2024-04-20T20:58:23.312 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-20T20:58:23.312 DEBUG:teuthology.run:Teuthology command: teuthology --name rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665408 --verbose --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/forward-scrub} -- /home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665408/orig.config.yaml 2024-04-20T20:58:23.359 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-20T20:58:23.444 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665408 branch: wip-rishabh-testing-20240416.193735 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/forward-scrub} email: null first_in_suite: false job_id: '7665408' kernel: flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240416.193735 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - inode wrongly marked free - bad backtrace on inode - inode table repaired for inode - Scrub error on inode - Scrub error on dir - Metadata damage detected sha1: a8c84f5997950fd7d008204839253291d52e6eda ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: a8c84f5997950fd7d008204839253291d52e6eda selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240416.193735 sha1: a8c84f5997950fd7d008204839253291d52e6eda owner: scheduled_rishabh@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 7954 sha1: a8c84f5997950fd7d008204839253291d52e6eda sleep_before_teardown: 0 subset: 364/512 suite: fs suite_branch: wip-rishabh-testing-20240416.193735 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: a8c84f5997950fd7d008204839253291d52e6eda syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi086.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHZnpT/76zTvzEFODAI0hhznpXMK/yubClrbEpPoEgNSA8R/v4zsK8z84AoBNTlVHHCt97Nlq4Z56KKW9fxSmvY= smithi144.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNFjKrxoh8SrDM4g/XgOPDuG61tFRgAkBwBCkI70X6ampv0lTPrAbP6bOnQPzoCAqUAMhMswMK2ehmSHQ/ViGVw= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_forward_scrub teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-20_20:50:37 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-20T20:58:23.444 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa; will attempt to use it 2024-04-20T20:58:23.445 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks 2024-04-20T20:58:23.445 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-20T20:58:23.447 INFO:teuthology.task.internal:Checking packages... 2024-04-20T20:58:23.471 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'a8c84f5997950fd7d008204839253291d52e6eda' 2024-04-20T20:58:23.471 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-20T20:58:23.471 INFO:teuthology.packaging:ref: None 2024-04-20T20:58:23.471 INFO:teuthology.packaging:tag: None 2024-04-20T20:58:23.471 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240416.193735 2024-04-20T20:58:23.471 INFO:teuthology.packaging:sha1: a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T20:58:23.472 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240416.193735 2024-04-20T20:58:23.706 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3133.ga8c84f59 2024-04-20T20:58:23.707 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-20T20:58:23.717 INFO:teuthology.task.internal:no buildpackages task found 2024-04-20T20:58:23.717 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-20T20:58:23.731 INFO:teuthology.task.internal:Saving configuration 2024-04-20T20:58:23.748 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-20T20:58:23.759 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-20T20:58:23.783 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi086.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665408', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-20 20:51:45.034137', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHZnpT/76zTvzEFODAI0hhznpXMK/yubClrbEpPoEgNSA8R/v4zsK8z84AoBNTlVHHCt97Nlq4Z56KKW9fxSmvY='} 2024-04-20T20:58:23.806 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi144.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665408', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-20 20:51:45.035608', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNFjKrxoh8SrDM4g/XgOPDuG61tFRgAkBwBCkI70X6ampv0lTPrAbP6bOnQPzoCAqUAMhMswMK2ehmSHQ/ViGVw='} 2024-04-20T20:58:23.806 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-20T20:58:23.817 INFO:teuthology.task.internal:roles: ubuntu@smithi086.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-20T20:58:23.817 INFO:teuthology.task.internal:roles: ubuntu@smithi144.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-20T20:58:23.818 INFO:teuthology.run_tasks:Running task console_log... 2024-04-20T20:58:23.897 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f27e77d8160>, signals=[15]) 2024-04-20T20:58:23.897 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-20T20:58:23.906 INFO:teuthology.task.internal:Opening connections... 2024-04-20T20:58:23.907 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi086.front.sepia.ceph.com 2024-04-20T20:58:23.909 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T20:58:23.987 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi144.front.sepia.ceph.com 2024-04-20T20:58:23.987 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T20:58:24.060 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-20T20:58:24.067 DEBUG:teuthology.orchestra.run.smithi086:> uname -m 2024-04-20T20:58:24.084 INFO:teuthology.orchestra.run.smithi086.stdout:x86_64 2024-04-20T20:58:24.085 DEBUG:teuthology.orchestra.run.smithi086:> cat /etc/os-release 2024-04-20T20:58:24.141 INFO:teuthology.orchestra.run.smithi086.stdout:NAME="CentOS Stream" 2024-04-20T20:58:24.141 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION="9" 2024-04-20T20:58:24.141 INFO:teuthology.orchestra.run.smithi086.stdout:ID="centos" 2024-04-20T20:58:24.141 INFO:teuthology.orchestra.run.smithi086.stdout:ID_LIKE="rhel fedora" 2024-04-20T20:58:24.141 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION_ID="9" 2024-04-20T20:58:24.141 INFO:teuthology.orchestra.run.smithi086.stdout:PLATFORM_ID="platform:el9" 2024-04-20T20:58:24.141 INFO:teuthology.orchestra.run.smithi086.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-20T20:58:24.141 INFO:teuthology.orchestra.run.smithi086.stdout:ANSI_COLOR="0;31" 2024-04-20T20:58:24.141 INFO:teuthology.orchestra.run.smithi086.stdout:LOGO="fedora-logo-icon" 2024-04-20T20:58:24.142 INFO:teuthology.orchestra.run.smithi086.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-20T20:58:24.142 INFO:teuthology.orchestra.run.smithi086.stdout:HOME_URL="https://centos.org/" 2024-04-20T20:58:24.142 INFO:teuthology.orchestra.run.smithi086.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-20T20:58:24.142 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-20T20:58:24.142 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-20T20:58:24.142 INFO:teuthology.lock.ops:Updating smithi086.front.sepia.ceph.com on lock server 2024-04-20T20:58:24.167 DEBUG:teuthology.orchestra.run.smithi144:> uname -m 2024-04-20T20:58:24.185 INFO:teuthology.orchestra.run.smithi144.stdout:x86_64 2024-04-20T20:58:24.185 DEBUG:teuthology.orchestra.run.smithi144:> cat /etc/os-release 2024-04-20T20:58:24.243 INFO:teuthology.orchestra.run.smithi144.stdout:NAME="CentOS Stream" 2024-04-20T20:58:24.243 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION="9" 2024-04-20T20:58:24.243 INFO:teuthology.orchestra.run.smithi144.stdout:ID="centos" 2024-04-20T20:58:24.243 INFO:teuthology.orchestra.run.smithi144.stdout:ID_LIKE="rhel fedora" 2024-04-20T20:58:24.243 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION_ID="9" 2024-04-20T20:58:24.243 INFO:teuthology.orchestra.run.smithi144.stdout:PLATFORM_ID="platform:el9" 2024-04-20T20:58:24.243 INFO:teuthology.orchestra.run.smithi144.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-20T20:58:24.243 INFO:teuthology.orchestra.run.smithi144.stdout:ANSI_COLOR="0;31" 2024-04-20T20:58:24.243 INFO:teuthology.orchestra.run.smithi144.stdout:LOGO="fedora-logo-icon" 2024-04-20T20:58:24.243 INFO:teuthology.orchestra.run.smithi144.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-20T20:58:24.243 INFO:teuthology.orchestra.run.smithi144.stdout:HOME_URL="https://centos.org/" 2024-04-20T20:58:24.243 INFO:teuthology.orchestra.run.smithi144.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-20T20:58:24.244 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-20T20:58:24.244 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-20T20:58:24.244 INFO:teuthology.lock.ops:Updating smithi144.front.sepia.ceph.com on lock server 2024-04-20T20:58:24.329 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-20T20:58:24.339 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-20T20:58:24.351 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-20T20:58:24.351 DEBUG:teuthology.orchestra.run.smithi086:> test '!' -e /home/ubuntu/cephtest 2024-04-20T20:58:24.355 DEBUG:teuthology.orchestra.run.smithi144:> test '!' -e /home/ubuntu/cephtest 2024-04-20T20:58:24.371 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-20T20:58:24.382 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-20T20:58:24.382 DEBUG:teuthology.orchestra.run.smithi086:> test -z $(ls -A /var/lib/ceph) 2024-04-20T20:58:24.412 DEBUG:teuthology.orchestra.run.smithi144:> test -z $(ls -A /var/lib/ceph) 2024-04-20T20:58:24.502 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-20T20:58:24.555 INFO:teuthology.run_tasks:Running task kernel... 2024-04-20T20:58:24.574 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-04-20T20:58:24.574 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}}, timeout 300 2024-04-20T20:58:24.575 DEBUG:teuthology.orchestra.run.smithi086:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-20T20:58:24.575 DEBUG:teuthology.orchestra.run.smithi144:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-20T20:58:24.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T20:58:24.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T20:58:24.593 DEBUG:teuthology.task.kernel:sha1 for mon.b is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T20:58:24.593 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-20T20:58:24.593 DEBUG:teuthology.orchestra.run.smithi144:> uname -r 2024-04-20T20:58:24.593 DEBUG:teuthology.task.kernel:sha1 for mon.a is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T20:58:24.593 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-20T20:58:24.594 DEBUG:teuthology.orchestra.run.smithi086:> uname -r 2024-04-20T20:58:24.650 INFO:teuthology.orchestra.run.smithi086.stdout:5.14.0-437.el9.x86_64 2024-04-20T20:58:24.650 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T20:58:24.650 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-20T20:58:24.651 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T20:58:24.654 INFO:teuthology.orchestra.run.smithi144.stdout:5.14.0-437.el9.x86_64 2024-04-20T20:58:24.655 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T20:58:24.655 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-20T20:58:24.655 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T20:58:24.793 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum upgrade -y linux-firmware 2024-04-20T20:58:24.797 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum upgrade -y linux-firmware 2024-04-20T20:58:27.303 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - BaseOS 9.5 MB/s | 8.0 MB 00:00 2024-04-20T20:58:28.798 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - BaseOS 3.5 MB/s | 8.0 MB 00:02 2024-04-20T20:58:31.290 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - AppStream 13 MB/s | 19 MB 00:01 2024-04-20T20:58:36.174 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - CRB 4.8 MB/s | 6.2 MB 00:01 2024-04-20T20:58:37.583 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - Extras packages 60 kB/s | 16 kB 00:00 2024-04-20T20:58:39.462 INFO:teuthology.orchestra.run.smithi086.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-04-20T20:58:43.251 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - AppStream 1.3 MB/s | 19 MB 00:14 2024-04-20T20:58:44.467 INFO:teuthology.orchestra.run.smithi086.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-04-20T20:58:46.475 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-04-20T20:58:46.476 INFO:teuthology.orchestra.run.smithi086.stdout:Nothing to do. 2024-04-20T20:58:46.476 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-04-20T20:58:46.583 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.a... 2024-04-20T20:58:46.584 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T20:58:46.729 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-20T20:58:46.729 DEBUG:teuthology.orchestra.run.smithi086:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-20T20:58:46.734 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi086.front.sepia.ceph.com to complete... 2024-04-20T20:58:47.627 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - CRB 7.7 MB/s | 6.2 MB 00:00 2024-04-20T20:58:49.353 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - Extras packages 43 kB/s | 16 kB 00:00 2024-04-20T20:58:53.657 INFO:teuthology.orchestra.run.smithi144.stdout:Extra Packages for Enterprise Linux 5.0 MB/s | 21 MB 00:04 2024-04-20T20:58:58.620 INFO:teuthology.orchestra.run.smithi144.stdout:lab-extras 37 kB/s | 1.7 kB 00:00 2024-04-20T20:59:00.677 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-04-20T20:59:00.678 INFO:teuthology.orchestra.run.smithi144.stdout:Nothing to do. 2024-04-20T20:59:00.678 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-04-20T20:59:00.804 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.b... 2024-04-20T20:59:00.804 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T20:59:01.023 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-20T20:59:01.023 DEBUG:teuthology.orchestra.run.smithi144:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-20T20:59:01.025 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi144.front.sepia.ceph.com to complete... 2024-04-20T21:06:40.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20 21:06:40 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-20T21:06:40.063 INFO:teuthology.orchestra.run.smithi086.stderr:FINISHED --2024-04-20 21:06:40-- 2024-04-20T21:06:40.063 INFO:teuthology.orchestra.run.smithi086.stderr:Total wall clock time: 7m 53s 2024-04-20T21:06:40.063 INFO:teuthology.orchestra.run.smithi086.stderr:Downloaded: 1 files, 341M in 7m 53s (738 KB/s) 2024-04-20T21:06:40.066 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.a... 2024-04-20T21:06:40.066 DEBUG:teuthology.orchestra.run.smithi086:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-20T21:06:42.149 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying... ######################################## 2024-04-20T21:06:42.371 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing... ######################################## 2024-04-20T21:06:42.488 INFO:teuthology.orchestra.run.smithi086.stdout:Updating / installing... 2024-04-20T21:06:50.422 INFO:teuthology.orchestra.run.smithi086.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-20T21:07:26.932 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi086.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-20T21:07:26.933 DEBUG:teuthology.orchestra.run.smithi086:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-20T21:07:27.039 INFO:teuthology.orchestra.run.smithi086.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-20T21:07:27.040 INFO:teuthology.orchestra.run.smithi086.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-20T21:07:27.040 INFO:teuthology.orchestra.run.smithi086.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-20T21:07:27.040 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-20T21:07:27.040 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-20T21:07:27.040 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-20T21:07:27.040 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-20T21:07:27.040 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-20T21:07:27.040 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-20T21:07:27.040 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-20T21:07:27.040 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-20T21:07:27.040 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-20T21:07:27.040 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-20T21:07:27.040 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-20T21:07:27.041 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-20T21:07:27.041 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-20T21:07:27.041 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-20T21:07:27.041 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-20T21:07:27.041 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-20T21:07:27.041 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-20T21:07:27.041 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-20T21:07:27.041 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-20T21:07:27.041 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-20T21:07:27.041 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-20T21:07:27.041 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-20T21:07:27.041 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-20T21:07:27.041 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-20T21:07:27.041 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-20T21:07:27.042 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-20T21:07:27.042 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-20T21:07:27.042 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-20T21:07:27.042 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-20T21:07:27.042 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-20T21:07:27.042 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-20T21:07:27.042 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-20T21:07:27.042 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-20T21:07:27.042 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-20T21:07:27.042 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-20T21:07:27.042 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-20T21:07:27.042 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-20T21:07:27.043 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-20T21:07:27.043 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-20T21:07:27.043 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-20T21:07:27.043 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-20T21:07:27.043 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-20T21:07:27.043 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-20T21:07:27.043 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-20T21:07:27.043 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-20T21:07:27.043 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-20T21:07:27.043 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-20T21:07:27.043 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-20T21:07:27.043 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-20T21:07:27.043 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-20T21:07:27.044 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-20T21:07:27.044 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-20T21:07:27.044 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-20T21:07:27.044 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-20T21:07:27.044 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-20T21:07:27.044 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-20T21:07:27.044 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-20T21:07:27.044 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-20T21:07:27.044 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-20T21:07:27.044 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-20T21:07:27.044 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-20T21:07:27.044 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-20T21:07:27.044 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-20T21:07:27.044 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-20T21:07:27.045 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-20T21:07:27.045 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-20T21:07:27.045 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-20T21:07:27.045 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-20T21:07:27.045 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-20T21:07:27.045 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-20T21:07:27.045 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-20T21:07:27.045 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-20T21:07:27.045 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-20T21:07:27.045 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-20T21:07:27.045 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-20T21:07:27.045 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-20T21:07:27.045 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-20T21:07:27.045 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-20T21:07:27.046 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-20T21:07:27.046 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-20T21:07:27.046 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-20T21:07:27.046 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-20T21:07:27.046 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-20T21:07:27.046 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-20T21:07:27.046 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-20T21:07:27.046 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-20T21:07:27.046 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-20T21:07:27.046 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-20T21:07:27.046 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-20T21:07:27.046 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-20T21:07:27.047 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-20T21:07:27.047 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-20T21:07:27.047 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-20T21:07:27.047 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-20T21:07:27.047 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-20T21:07:27.047 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-20T21:07:27.047 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-20T21:07:27.047 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-20T21:07:27.047 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-20T21:07:27.047 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-20T21:07:27.047 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-20T21:07:27.047 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-20T21:07:27.047 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-20T21:07:27.047 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-20T21:07:27.047 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-20T21:07:27.048 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-20T21:07:27.048 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-20T21:07:27.048 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-20T21:07:27.048 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-20T21:07:27.048 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-20T21:07:27.048 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-20T21:07:27.048 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-20T21:07:27.048 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-20T21:07:27.048 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-20T21:07:27.048 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-20T21:07:27.048 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-20T21:07:27.048 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-20T21:07:27.048 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-20T21:07:27.048 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-20T21:07:27.049 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-20T21:07:27.049 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-20T21:07:27.049 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-20T21:07:27.049 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-20T21:07:27.049 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-20T21:07:27.049 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-20T21:07:27.049 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-20T21:07:27.049 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-20T21:07:27.049 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-20T21:07:27.049 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-20T21:07:27.049 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-20T21:07:27.049 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-20T21:07:27.049 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-20T21:07:27.049 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-20T21:07:27.049 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-20T21:07:27.050 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-20T21:07:27.050 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-20T21:07:27.050 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-20T21:07:27.050 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-20T21:07:27.050 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-20T21:07:27.050 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-20T21:07:27.050 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-20T21:07:27.050 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-20T21:07:27.050 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-20T21:07:27.050 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-20T21:07:27.050 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-20T21:07:27.050 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-20T21:07:27.050 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-20T21:07:27.051 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-20T21:07:27.051 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-20T21:07:27.051 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-20T21:07:27.051 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-20T21:07:27.051 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-20T21:07:27.051 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-20T21:07:27.051 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-20T21:07:27.051 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-20T21:07:27.051 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-20T21:07:27.051 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-20T21:07:27.051 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-20T21:07:27.051 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-20T21:07:27.052 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-20T21:07:27.052 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-20T21:07:27.052 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-20T21:07:27.052 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-20T21:07:27.052 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-20T21:07:27.052 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-20T21:07:27.052 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-20T21:07:27.052 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-20T21:07:27.052 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-20T21:07:27.052 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-20T21:07:27.053 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-20T21:07:27.053 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-20T21:07:27.053 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-20T21:07:27.053 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-20T21:07:27.053 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-20T21:07:27.053 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-20T21:07:27.053 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-20T21:07:27.053 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-20T21:07:27.053 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-20T21:07:27.053 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-20T21:07:27.053 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-20T21:07:27.053 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-20T21:07:27.053 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-20T21:07:27.054 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-20T21:07:27.054 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-20T21:07:27.054 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-20T21:07:27.054 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-20T21:07:27.054 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-20T21:07:27.054 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-20T21:07:27.054 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-20T21:07:27.054 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-20T21:07:27.054 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-20T21:07:27.054 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-20T21:07:27.054 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-20T21:07:27.054 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-20T21:07:27.054 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-20T21:07:27.055 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-20T21:07:27.055 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-20T21:07:27.055 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-20T21:07:27.055 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-20T21:07:27.055 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-20T21:07:27.055 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-20T21:07:27.055 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-20T21:07:27.055 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-20T21:07:27.055 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-20T21:07:27.055 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-20T21:07:27.055 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-20T21:07:27.055 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-20T21:07:27.055 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-20T21:07:27.055 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-20T21:07:27.056 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-20T21:07:27.056 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-20T21:07:27.056 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-20T21:07:27.056 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-20T21:07:27.056 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-20T21:07:27.056 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-20T21:07:27.056 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-20T21:07:27.056 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-20T21:07:27.056 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-20T21:07:27.056 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-20T21:07:27.056 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-20T21:07:27.056 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-20T21:07:27.056 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-20T21:07:27.057 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-20T21:07:27.057 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-20T21:07:27.057 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-20T21:07:27.057 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-20T21:07:27.057 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-20T21:07:27.057 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-20T21:07:27.057 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-20T21:07:27.057 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-20T21:07:27.057 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-20T21:07:27.057 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-20T21:07:27.057 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-20T21:07:27.057 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-20T21:07:27.057 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-20T21:07:27.057 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-20T21:07:27.058 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-20T21:07:27.058 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-20T21:07:27.058 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-20T21:07:27.058 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-20T21:07:27.058 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-20T21:07:27.058 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-20T21:07:27.058 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-20T21:07:27.058 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-20T21:07:27.058 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-20T21:07:27.058 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-20T21:07:27.058 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-20T21:07:27.058 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-20T21:07:27.058 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-20T21:07:27.058 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-20T21:07:27.059 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-20T21:07:27.059 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-20T21:07:27.059 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-20T21:07:27.059 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-20T21:07:27.059 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-20T21:07:27.059 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-20T21:07:27.059 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-20T21:07:27.059 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-20T21:07:27.059 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-20T21:07:27.059 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-20T21:07:27.059 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-20T21:07:27.059 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-20T21:07:27.059 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-20T21:07:27.059 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-20T21:07:27.059 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-20T21:07:27.060 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-20T21:07:27.060 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-20T21:07:27.060 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-20T21:07:27.060 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-20T21:07:27.060 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-20T21:07:27.060 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-20T21:07:27.060 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-20T21:07:27.060 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-20T21:07:27.060 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-20T21:07:27.060 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-20T21:07:27.060 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-20T21:07:27.060 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-20T21:07:27.060 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-20T21:07:27.061 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-20T21:07:27.061 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-20T21:07:27.061 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-20T21:07:27.061 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-20T21:07:27.061 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-20T21:07:27.061 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-20T21:07:27.061 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-20T21:07:27.061 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-20T21:07:27.061 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-20T21:07:27.061 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-20T21:07:27.061 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-20T21:07:27.061 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-20T21:07:27.061 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-20T21:07:27.061 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-20T21:07:27.062 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-20T21:07:27.062 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-20T21:07:27.062 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-20T21:07:27.062 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-20T21:07:27.062 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-20T21:07:27.062 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-20T21:07:27.062 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-20T21:07:27.062 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-20T21:07:27.062 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-20T21:07:27.062 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-20T21:07:27.062 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-20T21:07:27.062 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-20T21:07:27.062 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-20T21:07:27.062 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-20T21:07:27.063 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-20T21:07:27.063 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-20T21:07:27.063 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-20T21:07:27.063 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-20T21:07:27.063 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-20T21:07:27.063 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-20T21:07:27.063 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-20T21:07:27.063 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-20T21:07:27.063 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-20T21:07:27.063 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-20T21:07:27.063 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-20T21:07:27.063 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-20T21:07:27.063 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-20T21:07:27.063 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-20T21:07:27.064 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-20T21:07:27.064 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-20T21:07:27.064 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-20T21:07:27.064 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-20T21:07:27.064 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-20T21:07:27.064 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-20T21:07:27.064 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-20T21:07:27.064 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-20T21:07:27.064 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-20T21:07:27.064 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-20T21:07:27.064 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-20T21:07:27.064 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-20T21:07:27.064 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-20T21:07:27.064 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-20T21:07:27.065 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-20T21:07:27.065 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-20T21:07:27.065 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-20T21:07:27.065 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-20T21:07:27.065 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-20T21:07:27.065 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-20T21:07:27.065 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-20T21:07:27.065 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-20T21:07:27.065 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-20T21:07:27.065 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-20T21:07:27.065 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-20T21:07:27.065 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-20T21:07:27.065 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-20T21:07:27.065 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-20T21:07:27.065 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-20T21:07:27.066 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-20T21:07:27.066 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-20T21:07:27.066 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-20T21:07:27.066 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-20T21:07:27.066 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-20T21:07:27.066 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-20T21:07:27.066 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-20T21:07:27.066 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-20T21:07:27.066 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-20T21:07:27.066 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-20T21:07:27.066 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-20T21:07:27.066 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-20T21:07:27.066 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-20T21:07:27.066 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-20T21:07:27.067 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-20T21:07:27.067 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-20T21:07:27.067 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-20T21:07:27.067 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-20T21:07:27.067 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-20T21:07:27.067 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-20T21:07:27.067 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-20T21:07:27.067 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-20T21:07:27.067 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-20T21:07:27.067 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-20T21:07:27.067 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-20T21:07:27.067 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-20T21:07:27.068 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-20T21:07:27.068 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-20T21:07:27.068 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-20T21:07:27.068 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-20T21:07:27.068 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-20T21:07:27.068 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-20T21:07:27.068 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-20T21:07:27.068 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-20T21:07:27.068 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-20T21:07:27.068 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-20T21:07:27.068 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-20T21:07:27.068 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-20T21:07:27.068 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-20T21:07:27.068 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-20T21:07:27.068 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-20T21:07:27.069 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-20T21:07:27.069 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-20T21:07:27.069 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-20T21:07:27.069 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-20T21:07:27.069 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-20T21:07:27.069 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-20T21:07:27.069 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-20T21:07:27.069 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-20T21:07:27.069 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-20T21:07:27.069 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-20T21:07:27.069 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-20T21:07:27.069 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-20T21:07:27.069 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-20T21:07:27.069 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-20T21:07:27.070 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-20T21:07:27.070 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-20T21:07:27.070 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-20T21:07:27.070 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-20T21:07:27.070 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-20T21:07:27.070 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-20T21:07:27.070 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-20T21:07:27.070 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-20T21:07:27.070 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-20T21:07:27.070 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-20T21:07:27.070 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-20T21:07:27.070 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-20T21:07:27.070 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-20T21:07:27.070 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-20T21:07:27.070 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-20T21:07:27.071 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-20T21:07:27.071 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-20T21:07:27.071 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-20T21:07:27.071 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-20T21:07:27.071 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-20T21:07:27.071 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-20T21:07:27.071 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-20T21:07:27.071 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-20T21:07:27.071 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-20T21:07:27.071 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-20T21:07:27.072 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-20T21:07:27.072 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-20T21:07:27.072 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-20T21:07:27.072 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-20T21:07:27.072 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-20T21:07:27.072 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-20T21:07:27.072 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-20T21:07:27.072 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-20T21:07:27.072 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-20T21:07:27.072 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-20T21:07:27.072 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-20T21:07:27.072 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-20T21:07:27.072 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-20T21:07:27.072 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-20T21:07:27.073 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-20T21:07:27.073 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-20T21:07:27.073 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-20T21:07:27.073 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-20T21:07:27.073 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-20T21:07:27.073 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-20T21:07:27.073 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-20T21:07:27.073 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-20T21:07:27.073 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-20T21:07:27.073 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-20T21:07:27.073 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-20T21:07:27.073 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-20T21:07:27.073 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-20T21:07:27.073 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-20T21:07:27.073 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-20T21:07:27.074 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-20T21:07:27.074 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-20T21:07:27.074 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-20T21:07:27.074 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-20T21:07:27.074 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-20T21:07:27.074 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-20T21:07:27.074 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-20T21:07:27.074 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-20T21:07:27.074 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-20T21:07:27.074 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-20T21:07:27.074 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-20T21:07:27.074 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-20T21:07:27.074 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-20T21:07:27.074 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-20T21:07:27.075 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-20T21:07:27.075 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-20T21:07:27.075 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-20T21:07:27.075 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-20T21:07:27.075 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-20T21:07:27.075 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-20T21:07:27.075 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-20T21:07:27.075 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-20T21:07:27.075 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-20T21:07:27.075 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-20T21:07:27.075 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-20T21:07:27.075 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-20T21:07:27.076 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-20T21:07:27.076 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-20T21:07:27.076 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-20T21:07:27.076 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-20T21:07:27.076 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-20T21:07:27.076 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-20T21:07:27.076 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-20T21:07:27.076 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-20T21:07:27.076 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-20T21:07:27.076 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-20T21:07:27.076 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-20T21:07:27.077 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-20T21:07:27.077 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-20T21:07:27.077 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-20T21:07:27.077 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-20T21:07:27.077 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-20T21:07:27.077 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-20T21:07:27.077 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-20T21:07:27.077 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-20T21:07:27.077 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-20T21:07:27.077 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-20T21:07:27.077 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-20T21:07:27.077 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-20T21:07:27.077 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-20T21:07:27.078 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-20T21:07:27.078 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-20T21:07:27.078 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-20T21:07:27.078 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-20T21:07:27.078 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-20T21:07:27.078 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-20T21:07:27.078 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-20T21:07:27.078 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-20T21:07:27.078 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-20T21:07:27.078 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-20T21:07:27.078 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-20T21:07:27.078 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-20T21:07:27.078 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-20T21:07:27.078 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-20T21:07:27.079 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-20T21:07:27.079 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-20T21:07:27.079 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-20T21:07:27.079 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-20T21:07:27.079 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-20T21:07:27.079 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-20T21:07:27.079 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-20T21:07:27.079 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-20T21:07:27.079 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-20T21:07:27.079 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-20T21:07:27.079 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-20T21:07:27.079 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-20T21:07:27.079 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-20T21:07:27.079 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-20T21:07:27.080 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-20T21:07:27.080 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-20T21:07:27.080 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-20T21:07:27.080 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-20T21:07:27.080 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-20T21:07:27.080 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-20T21:07:27.080 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-20T21:07:27.080 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-20T21:07:27.080 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-20T21:07:27.080 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-20T21:07:27.080 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-20T21:07:27.080 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-20T21:07:27.080 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-20T21:07:27.080 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-20T21:07:27.080 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-20T21:07:27.081 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-20T21:07:27.081 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-20T21:07:27.081 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-20T21:07:27.081 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-20T21:07:27.081 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-20T21:07:27.081 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-20T21:07:27.081 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-20T21:07:27.081 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-20T21:07:27.081 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-20T21:07:27.081 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-20T21:07:27.081 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-20T21:07:27.081 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-20T21:07:27.081 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-20T21:07:27.081 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-20T21:07:27.081 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-20T21:07:27.082 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-20T21:07:27.082 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-20T21:07:27.082 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-20T21:07:27.082 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-20T21:07:27.082 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-20T21:07:27.082 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-20T21:07:27.082 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-20T21:07:27.082 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-20T21:07:27.082 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-20T21:07:27.082 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-20T21:07:27.082 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-20T21:07:27.082 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-20T21:07:27.082 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-20T21:07:27.082 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-20T21:07:27.083 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-20T21:07:27.083 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-20T21:07:27.083 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-20T21:07:27.083 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-20T21:07:27.083 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-20T21:07:27.083 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-20T21:07:27.083 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-20T21:07:27.083 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-20T21:07:27.083 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-20T21:07:27.083 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-20T21:07:27.083 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-20T21:07:27.083 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-20T21:07:27.084 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-20T21:07:27.084 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-20T21:07:27.084 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-20T21:07:27.084 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-20T21:07:27.084 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-20T21:07:27.084 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-20T21:07:27.084 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-20T21:07:27.084 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-20T21:07:27.084 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-20T21:07:27.084 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-20T21:07:27.084 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-20T21:07:27.084 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-20T21:07:27.084 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-20T21:07:27.084 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-20T21:07:27.084 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-20T21:07:27.085 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-20T21:07:27.085 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-20T21:07:27.085 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-20T21:07:27.085 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-20T21:07:27.085 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-20T21:07:27.085 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-20T21:07:27.085 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-20T21:07:27.085 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-20T21:07:27.085 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-20T21:07:27.085 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-20T21:07:27.085 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-20T21:07:27.085 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-20T21:07:27.085 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-20T21:07:27.085 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-20T21:07:27.086 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-20T21:07:27.086 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-20T21:07:27.086 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-20T21:07:27.086 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-20T21:07:27.086 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-20T21:07:27.086 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-20T21:07:27.086 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-20T21:07:27.086 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-20T21:07:27.086 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-20T21:07:27.086 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-20T21:07:27.086 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-20T21:07:27.086 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-20T21:07:27.086 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-20T21:07:27.087 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-20T21:07:27.087 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-20T21:07:27.087 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-20T21:07:27.087 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-20T21:07:27.087 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-20T21:07:27.087 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-20T21:07:27.087 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-20T21:07:27.087 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-20T21:07:27.087 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-20T21:07:27.087 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-20T21:07:27.087 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-20T21:07:27.087 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-20T21:07:27.087 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-20T21:07:27.087 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-20T21:07:27.087 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-20T21:07:27.088 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-20T21:07:27.088 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-20T21:07:27.088 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-20T21:07:27.088 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-20T21:07:27.088 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-20T21:07:27.088 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-20T21:07:27.088 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-20T21:07:27.088 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-20T21:07:27.088 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-20T21:07:27.088 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-20T21:07:27.088 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-20T21:07:27.088 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-20T21:07:27.088 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-20T21:07:27.088 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-20T21:07:27.089 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-20T21:07:27.089 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-20T21:07:27.089 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-20T21:07:27.089 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-20T21:07:27.089 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-20T21:07:27.089 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-20T21:07:27.089 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-20T21:07:27.089 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-20T21:07:27.089 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-20T21:07:27.089 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-20T21:07:27.089 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-20T21:07:27.089 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-20T21:07:27.089 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-20T21:07:27.089 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-20T21:07:27.090 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-20T21:07:27.090 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-20T21:07:27.090 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-20T21:07:27.090 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-20T21:07:27.090 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-20T21:07:27.090 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-20T21:07:27.090 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-20T21:07:27.090 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-20T21:07:27.090 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-20T21:07:27.090 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-20T21:07:27.090 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-20T21:07:27.090 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-20T21:07:27.091 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-20T21:07:27.091 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-20T21:07:27.091 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-20T21:07:27.091 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-20T21:07:27.091 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-20T21:07:27.091 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-20T21:07:27.091 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-20T21:07:27.091 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-20T21:07:27.091 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-20T21:07:27.091 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-20T21:07:27.091 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-20T21:07:27.092 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-20T21:07:27.092 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-20T21:07:27.092 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-20T21:07:27.092 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-20T21:07:27.092 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-20T21:07:27.092 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-20T21:07:27.092 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-20T21:07:27.092 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-20T21:07:27.092 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-20T21:07:27.092 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-20T21:07:27.092 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-20T21:07:27.092 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-20T21:07:27.092 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-20T21:07:27.092 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-20T21:07:27.093 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-20T21:07:27.093 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-20T21:07:27.093 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-20T21:07:27.093 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-20T21:07:27.093 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-20T21:07:27.093 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-20T21:07:27.093 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-20T21:07:27.093 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-20T21:07:27.093 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-20T21:07:27.093 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-20T21:07:27.093 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-20T21:07:27.093 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-20T21:07:27.093 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-20T21:07:27.093 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-20T21:07:27.093 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-20T21:07:27.094 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-20T21:07:27.094 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-20T21:07:27.094 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-20T21:07:27.094 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-20T21:07:27.094 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-20T21:07:27.094 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-20T21:07:27.094 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-20T21:07:27.094 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-20T21:07:27.094 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-20T21:07:27.094 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-20T21:07:27.094 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-20T21:07:27.094 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-20T21:07:27.094 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-20T21:07:27.095 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-20T21:07:27.095 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-20T21:07:27.095 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-20T21:07:27.095 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-20T21:07:27.095 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-20T21:07:27.095 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-20T21:07:27.095 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-20T21:07:27.095 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-20T21:07:27.095 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-20T21:07:27.095 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-20T21:07:27.095 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-20T21:07:27.095 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-20T21:07:27.095 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-20T21:07:27.096 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-20T21:07:27.096 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-20T21:07:27.096 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-20T21:07:27.096 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-20T21:07:27.096 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-20T21:07:27.096 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-20T21:07:27.096 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-20T21:07:27.096 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-20T21:07:27.096 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-20T21:07:27.096 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-20T21:07:27.096 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-20T21:07:27.096 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-20T21:07:27.097 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-20T21:07:27.097 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-20T21:07:27.097 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-20T21:07:27.097 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-20T21:07:27.097 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-20T21:07:27.097 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-20T21:07:27.097 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-20T21:07:27.097 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-20T21:07:27.097 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-20T21:07:27.097 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-20T21:07:27.097 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-20T21:07:27.097 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-20T21:07:27.097 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-20T21:07:27.097 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-20T21:07:27.097 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-20T21:07:27.098 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-20T21:07:27.098 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-20T21:07:27.098 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-20T21:07:27.098 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-20T21:07:27.098 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-20T21:07:27.098 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-20T21:07:27.098 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-20T21:07:27.098 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-20T21:07:27.098 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-20T21:07:27.098 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-20T21:07:27.098 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-20T21:07:27.098 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-20T21:07:27.099 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-20T21:07:27.099 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-20T21:07:27.099 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-20T21:07:27.099 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-20T21:07:27.099 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-20T21:07:27.099 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-20T21:07:27.099 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-20T21:07:27.099 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-20T21:07:27.099 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-20T21:07:27.099 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-20T21:07:27.099 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-20T21:07:27.099 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-20T21:07:27.099 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-20T21:07:27.099 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-20T21:07:27.099 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-20T21:07:27.100 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-20T21:07:27.100 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-20T21:07:27.100 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-20T21:07:27.100 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-20T21:07:27.100 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-20T21:07:27.100 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-20T21:07:27.100 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-20T21:07:27.100 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-20T21:07:27.100 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-20T21:07:27.100 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-20T21:07:27.100 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-20T21:07:27.100 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-20T21:07:27.100 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-20T21:07:27.100 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-20T21:07:27.100 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-20T21:07:27.101 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-20T21:07:27.101 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-20T21:07:27.101 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-20T21:07:27.101 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-20T21:07:27.101 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-20T21:07:27.101 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-20T21:07:27.101 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-20T21:07:27.101 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-20T21:07:27.101 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-20T21:07:27.101 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-20T21:07:27.101 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-20T21:07:27.101 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-20T21:07:27.101 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-20T21:07:27.101 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-20T21:07:27.101 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-20T21:07:27.102 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-20T21:07:27.102 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-20T21:07:27.102 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-20T21:07:27.102 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-20T21:07:27.102 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-20T21:07:27.102 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-20T21:07:27.102 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-20T21:07:27.102 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-20T21:07:27.102 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-20T21:07:27.102 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-20T21:07:27.102 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-20T21:07:27.102 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-20T21:07:27.102 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-20T21:07:27.102 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-20T21:07:27.103 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-20T21:07:27.103 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-20T21:07:27.103 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-20T21:07:27.103 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-20T21:07:27.103 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-20T21:07:27.103 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-20T21:07:27.103 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-20T21:07:27.103 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-20T21:07:27.103 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-20T21:07:27.103 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-20T21:07:27.103 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-20T21:07:27.103 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-20T21:07:27.103 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-20T21:07:27.103 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-20T21:07:27.103 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-20T21:07:27.104 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-20T21:07:27.104 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-20T21:07:27.104 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-20T21:07:27.104 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-20T21:07:27.104 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-20T21:07:27.104 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-20T21:07:27.104 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-20T21:07:27.104 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-20T21:07:27.104 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-20T21:07:27.104 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-20T21:07:27.104 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-20T21:07:27.104 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-20T21:07:27.104 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-20T21:07:27.104 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-20T21:07:27.105 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-20T21:07:27.105 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-20T21:07:27.105 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-20T21:07:27.105 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-20T21:07:27.105 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-20T21:07:27.105 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-20T21:07:27.105 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-20T21:07:27.105 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-20T21:07:27.105 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-20T21:07:27.105 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-20T21:07:27.105 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-20T21:07:27.105 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-20T21:07:27.105 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-20T21:07:27.105 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-20T21:07:27.105 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-20T21:07:27.106 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-20T21:07:27.106 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-20T21:07:27.106 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-20T21:07:27.106 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-20T21:07:27.106 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-20T21:07:27.106 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-20T21:07:27.106 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-20T21:07:27.106 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-20T21:07:27.106 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-20T21:07:27.106 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-20T21:07:27.106 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-20T21:07:27.106 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-20T21:07:27.106 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-20T21:07:27.107 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-20T21:07:27.107 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-20T21:07:27.107 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-20T21:07:27.107 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-20T21:07:27.107 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-20T21:07:27.107 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-20T21:07:27.107 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-20T21:07:27.107 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-20T21:07:27.107 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-20T21:07:27.107 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-20T21:07:27.107 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-20T21:07:27.107 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-20T21:07:27.107 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-20T21:07:27.107 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-20T21:07:27.107 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-20T21:07:27.108 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-20T21:07:27.108 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-20T21:07:27.108 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-20T21:07:27.108 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-20T21:07:27.108 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-20T21:07:27.108 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-20T21:07:27.108 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-20T21:07:27.108 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-20T21:07:27.108 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-20T21:07:27.108 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-20T21:07:27.108 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-20T21:07:27.108 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-20T21:07:27.108 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-20T21:07:27.108 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-20T21:07:27.109 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-20T21:07:27.109 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-20T21:07:27.109 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-20T21:07:27.109 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-20T21:07:27.109 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-20T21:07:27.109 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-20T21:07:27.109 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-20T21:07:27.109 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-20T21:07:27.109 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-20T21:07:27.109 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-20T21:07:27.109 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-20T21:07:27.109 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-20T21:07:27.109 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-20T21:07:27.109 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-20T21:07:27.109 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-20T21:07:27.110 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-20T21:07:27.110 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-20T21:07:27.110 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-20T21:07:27.110 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-20T21:07:27.110 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-20T21:07:27.110 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-20T21:07:27.110 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-20T21:07:27.110 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-20T21:07:27.110 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-20T21:07:27.110 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-20T21:07:27.110 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-20T21:07:27.110 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-20T21:07:27.110 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-20T21:07:27.110 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-20T21:07:27.111 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-20T21:07:27.111 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-20T21:07:27.111 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-20T21:07:27.111 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-20T21:07:27.111 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-20T21:07:27.111 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-20T21:07:27.111 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-20T21:07:27.111 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-20T21:07:27.111 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-20T21:07:27.111 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-20T21:07:27.111 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-20T21:07:27.111 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-20T21:07:27.111 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-20T21:07:27.111 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-20T21:07:27.111 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-20T21:07:27.112 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-20T21:07:27.112 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-20T21:07:27.112 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-20T21:07:27.112 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-20T21:07:27.112 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-20T21:07:27.112 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-20T21:07:27.112 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-20T21:07:27.112 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-20T21:07:27.112 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-20T21:07:27.112 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-20T21:07:27.112 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-20T21:07:27.112 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-20T21:07:27.112 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-20T21:07:27.112 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-20T21:07:27.113 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-20T21:07:27.113 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-20T21:07:27.113 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-20T21:07:27.113 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-20T21:07:27.113 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-20T21:07:27.113 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-20T21:07:27.113 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-20T21:07:27.113 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-20T21:07:27.113 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-20T21:07:27.113 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-20T21:07:27.113 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-20T21:07:27.113 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-20T21:07:27.113 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-20T21:07:27.113 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-20T21:07:27.113 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-20T21:07:27.114 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-20T21:07:27.114 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-20T21:07:27.114 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-20T21:07:27.114 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-20T21:07:27.114 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-20T21:07:27.114 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-20T21:07:27.114 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-20T21:07:27.114 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-20T21:07:27.114 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-20T21:07:27.114 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-20T21:07:27.114 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-20T21:07:27.114 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-20T21:07:27.114 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-20T21:07:27.115 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-20T21:07:27.115 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-20T21:07:27.115 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-20T21:07:27.115 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-20T21:07:27.115 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-20T21:07:27.115 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-20T21:07:27.115 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-20T21:07:27.115 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-20T21:07:27.115 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-20T21:07:27.115 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-20T21:07:27.115 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-20T21:07:27.115 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-20T21:07:27.115 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-20T21:07:27.115 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-20T21:07:27.115 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-20T21:07:27.116 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-20T21:07:27.116 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-20T21:07:27.116 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-20T21:07:27.116 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-20T21:07:27.116 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-20T21:07:27.116 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-20T21:07:27.116 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-20T21:07:27.116 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-20T21:07:27.116 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-20T21:07:27.116 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-20T21:07:27.116 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-20T21:07:27.116 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-20T21:07:27.116 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-20T21:07:27.116 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-20T21:07:27.117 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-20T21:07:27.117 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-20T21:07:27.117 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-20T21:07:27.117 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-20T21:07:27.117 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-20T21:07:27.117 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-20T21:07:27.117 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-20T21:07:27.117 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-20T21:07:27.117 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-20T21:07:27.117 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-20T21:07:27.117 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-20T21:07:27.117 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-20T21:07:27.117 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-20T21:07:27.117 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-20T21:07:27.117 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-20T21:07:27.118 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-20T21:07:27.118 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-20T21:07:27.118 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-20T21:07:27.118 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-20T21:07:27.118 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-20T21:07:27.118 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-20T21:07:27.118 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-20T21:07:27.118 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-20T21:07:27.118 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-20T21:07:27.118 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-20T21:07:27.118 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-20T21:07:27.118 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-20T21:07:27.118 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-20T21:07:27.118 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-20T21:07:27.119 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-20T21:07:27.119 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-20T21:07:27.119 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-20T21:07:27.119 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-20T21:07:27.119 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-20T21:07:27.119 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-20T21:07:27.119 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-20T21:07:27.119 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-20T21:07:27.119 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-20T21:07:27.119 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-20T21:07:27.119 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-20T21:07:27.119 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-20T21:07:27.119 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-20T21:07:27.119 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-20T21:07:27.119 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-20T21:07:27.120 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-20T21:07:27.120 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-20T21:07:27.120 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-20T21:07:27.120 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-20T21:07:27.120 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-20T21:07:27.120 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-20T21:07:27.120 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-20T21:07:27.120 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-20T21:07:27.120 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-20T21:07:27.120 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-20T21:07:27.120 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-20T21:07:27.120 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-20T21:07:27.120 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-20T21:07:27.120 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-20T21:07:27.121 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-20T21:07:27.121 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-20T21:07:27.121 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-20T21:07:27.121 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-20T21:07:27.121 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-20T21:07:27.121 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-20T21:07:27.121 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-20T21:07:27.121 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-20T21:07:27.121 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-20T21:07:27.121 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-20T21:07:27.121 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-20T21:07:27.121 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-20T21:07:27.121 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-20T21:07:27.121 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-20T21:07:27.122 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-20T21:07:27.122 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-20T21:07:27.122 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-20T21:07:27.122 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-20T21:07:27.122 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-20T21:07:27.122 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-20T21:07:27.122 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-20T21:07:27.122 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-20T21:07:27.122 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-20T21:07:27.122 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-20T21:07:27.122 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-20T21:07:27.122 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-20T21:07:27.122 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-20T21:07:27.122 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-20T21:07:27.123 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-20T21:07:27.123 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-20T21:07:27.123 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-20T21:07:27.123 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-20T21:07:27.123 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-20T21:07:27.123 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-20T21:07:27.123 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-20T21:07:27.123 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-20T21:07:27.124 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-20T21:07:27.124 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-20T21:07:27.124 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-20T21:07:27.124 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-20T21:07:27.124 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-20T21:07:27.124 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-20T21:07:27.124 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-20T21:07:27.124 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-20T21:07:27.124 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-20T21:07:27.124 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-20T21:07:27.125 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-20T21:07:27.125 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-20T21:07:27.125 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-20T21:07:27.125 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-20T21:07:27.125 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-20T21:07:27.125 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-20T21:07:27.125 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-20T21:07:27.125 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-20T21:07:27.126 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-20T21:07:27.126 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-20T21:07:27.126 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-20T21:07:27.126 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-20T21:07:27.126 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-20T21:07:27.126 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-20T21:07:27.126 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-20T21:07:27.126 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-20T21:07:27.126 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-20T21:07:27.126 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-20T21:07:27.127 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-20T21:07:27.127 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-20T21:07:27.127 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-20T21:07:27.127 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-20T21:07:27.127 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-20T21:07:27.127 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-20T21:07:27.127 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-20T21:07:27.127 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-20T21:07:27.127 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-20T21:07:27.127 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-20T21:07:27.127 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-20T21:07:27.127 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-20T21:07:27.127 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-20T21:07:27.128 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-20T21:07:27.128 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-20T21:07:27.128 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-20T21:07:27.128 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-20T21:07:27.128 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-20T21:07:27.128 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-20T21:07:27.128 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-20T21:07:27.128 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-20T21:07:27.128 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-20T21:07:27.128 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-20T21:07:27.128 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-20T21:07:27.128 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-20T21:07:27.129 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-20T21:07:27.129 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-20T21:07:27.129 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-20T21:07:27.129 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-20T21:07:27.129 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-20T21:07:27.129 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-20T21:07:27.129 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-20T21:07:27.129 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-20T21:07:27.129 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-20T21:07:27.129 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-20T21:07:27.129 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-20T21:07:27.129 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-20T21:07:27.129 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-20T21:07:27.129 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-20T21:07:27.129 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-20T21:07:27.130 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-20T21:07:27.130 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-20T21:07:27.130 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-20T21:07:27.130 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-20T21:07:27.130 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-20T21:07:27.130 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-20T21:07:27.130 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-20T21:07:27.130 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-20T21:07:27.130 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-20T21:07:27.130 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-20T21:07:27.130 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-20T21:07:27.130 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-20T21:07:27.130 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-20T21:07:27.130 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-20T21:07:27.131 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-20T21:07:27.131 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-20T21:07:27.131 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-20T21:07:27.131 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-20T21:07:27.131 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-20T21:07:27.131 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-20T21:07:27.131 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-20T21:07:27.131 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-20T21:07:27.131 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-20T21:07:27.131 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-20T21:07:27.131 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-20T21:07:27.131 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-20T21:07:27.131 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-20T21:07:27.132 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-20T21:07:27.132 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-20T21:07:27.132 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-20T21:07:27.132 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-20T21:07:27.132 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-20T21:07:27.132 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-20T21:07:27.132 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-20T21:07:27.132 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-20T21:07:27.132 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-20T21:07:27.132 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-20T21:07:27.132 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-20T21:07:27.132 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-20T21:07:27.132 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-20T21:07:27.132 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-20T21:07:27.133 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-20T21:07:27.133 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-20T21:07:27.133 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-20T21:07:27.133 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-20T21:07:27.133 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-20T21:07:27.133 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-20T21:07:27.133 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-20T21:07:27.133 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-20T21:07:27.133 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-20T21:07:27.133 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-20T21:07:27.133 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-20T21:07:27.133 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-20T21:07:27.133 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-20T21:07:27.133 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-20T21:07:27.133 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-20T21:07:27.133 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-20T21:07:27.134 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-20T21:07:27.134 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-20T21:07:27.134 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-20T21:07:27.134 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-20T21:07:27.134 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-20T21:07:27.134 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-20T21:07:27.134 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-20T21:07:27.134 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-20T21:07:27.134 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-20T21:07:27.134 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-20T21:07:27.134 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-20T21:07:27.134 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-20T21:07:27.134 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-20T21:07:27.135 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-20T21:07:27.135 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-20T21:07:27.135 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-20T21:07:27.135 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-20T21:07:27.135 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-20T21:07:27.135 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-20T21:07:27.135 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-20T21:07:27.135 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-20T21:07:27.135 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-20T21:07:27.135 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-20T21:07:27.135 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-20T21:07:27.135 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-20T21:07:27.135 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-20T21:07:27.135 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-20T21:07:27.135 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-20T21:07:27.136 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-20T21:07:27.136 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-20T21:07:27.136 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-20T21:07:27.136 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-20T21:07:27.136 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-20T21:07:27.136 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-20T21:07:27.136 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-20T21:07:27.136 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-20T21:07:27.136 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-20T21:07:27.136 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-20T21:07:27.136 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-20T21:07:27.136 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-20T21:07:27.136 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-20T21:07:27.136 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-20T21:07:27.136 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-20T21:07:27.137 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-20T21:07:27.137 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-20T21:07:27.137 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-20T21:07:27.137 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-20T21:07:27.137 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-20T21:07:27.137 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-20T21:07:27.137 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-20T21:07:27.137 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-20T21:07:27.137 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-20T21:07:27.137 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-20T21:07:27.137 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-20T21:07:27.137 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-20T21:07:27.137 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-20T21:07:27.137 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-20T21:07:27.137 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-20T21:07:27.138 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-20T21:07:27.138 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-20T21:07:27.138 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-20T21:07:27.138 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-20T21:07:27.138 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-20T21:07:27.138 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-20T21:07:27.138 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-20T21:07:27.138 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-20T21:07:27.138 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-20T21:07:27.138 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-20T21:07:27.138 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-20T21:07:27.138 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-20T21:07:27.138 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-20T21:07:27.138 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-20T21:07:27.139 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-20T21:07:27.139 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-20T21:07:27.139 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-20T21:07:27.139 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-20T21:07:27.139 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-20T21:07:27.139 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-20T21:07:27.139 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-20T21:07:27.139 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-20T21:07:27.139 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-20T21:07:27.139 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-20T21:07:27.139 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-20T21:07:27.139 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-20T21:07:27.139 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-20T21:07:27.139 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-20T21:07:27.139 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-20T21:07:27.140 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-20T21:07:27.140 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-20T21:07:27.140 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-20T21:07:27.140 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-20T21:07:27.140 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-20T21:07:27.140 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-20T21:07:27.140 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-20T21:07:27.140 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-20T21:07:27.140 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-20T21:07:27.140 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-20T21:07:27.140 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-20T21:07:27.140 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-20T21:07:27.140 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-20T21:07:27.140 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-20T21:07:27.140 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-20T21:07:27.141 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-20T21:07:27.141 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-20T21:07:27.141 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-20T21:07:27.141 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-20T21:07:27.141 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-20T21:07:27.141 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-20T21:07:27.141 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-20T21:07:27.141 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-20T21:07:27.141 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-20T21:07:27.141 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-20T21:07:27.141 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-20T21:07:27.141 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-20T21:07:27.141 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-20T21:07:27.141 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-20T21:07:27.141 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-20T21:07:27.142 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-20T21:07:27.142 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-20T21:07:27.142 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-20T21:07:27.142 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-20T21:07:27.142 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-20T21:07:27.142 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-20T21:07:27.142 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-20T21:07:27.142 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-20T21:07:27.142 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-20T21:07:27.142 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-20T21:07:27.142 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-20T21:07:27.142 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-20T21:07:27.142 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-20T21:07:27.142 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-20T21:07:27.143 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-20T21:07:27.143 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-20T21:07:27.143 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-20T21:07:27.143 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-20T21:07:27.143 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-20T21:07:27.143 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-20T21:07:27.143 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-20T21:07:27.143 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-20T21:07:27.143 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-20T21:07:27.143 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-20T21:07:27.143 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-20T21:07:27.143 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-20T21:07:27.143 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-20T21:07:27.143 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-20T21:07:27.143 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-20T21:07:27.144 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-20T21:07:27.144 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-20T21:07:27.144 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-20T21:07:27.144 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-20T21:07:27.144 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-20T21:07:27.144 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-20T21:07:27.144 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-20T21:07:27.144 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-20T21:07:27.144 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-20T21:07:27.144 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-20T21:07:27.144 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-20T21:07:27.144 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-20T21:07:27.144 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-20T21:07:27.144 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-20T21:07:27.145 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-20T21:07:27.145 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-20T21:07:27.145 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-20T21:07:27.145 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-20T21:07:27.145 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-20T21:07:27.145 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-20T21:07:27.145 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-20T21:07:27.145 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-20T21:07:27.145 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-20T21:07:27.145 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-20T21:07:27.145 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-20T21:07:27.145 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-20T21:07:27.145 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-20T21:07:27.145 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-20T21:07:27.145 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-20T21:07:27.146 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-20T21:07:27.146 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-20T21:07:27.146 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-20T21:07:27.146 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-20T21:07:27.146 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-20T21:07:27.146 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-20T21:07:27.146 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-20T21:07:27.146 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-20T21:07:27.146 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-20T21:07:27.146 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-20T21:07:27.146 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-20T21:07:27.146 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-20T21:07:27.146 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-20T21:07:27.146 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-20T21:07:27.147 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-20T21:07:27.147 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-20T21:07:27.147 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-20T21:07:27.147 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-20T21:07:27.147 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-20T21:07:27.147 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-20T21:07:27.147 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-20T21:07:27.147 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-20T21:07:27.147 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-20T21:07:27.147 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-20T21:07:27.147 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-20T21:07:27.147 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-20T21:07:27.147 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-20T21:07:27.147 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-20T21:07:27.147 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-20T21:07:27.148 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-20T21:07:27.148 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-20T21:07:27.148 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-20T21:07:27.148 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-20T21:07:27.148 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-20T21:07:27.148 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-20T21:07:27.148 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-20T21:07:27.148 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-20T21:07:27.148 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-20T21:07:27.148 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-20T21:07:27.148 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-20T21:07:27.148 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-20T21:07:27.148 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-20T21:07:27.148 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-20T21:07:27.149 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-20T21:07:27.149 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-20T21:07:27.149 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-20T21:07:27.149 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-20T21:07:27.149 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-20T21:07:27.149 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-20T21:07:27.149 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-20T21:07:27.149 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-20T21:07:27.149 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-20T21:07:27.149 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-20T21:07:27.149 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-20T21:07:27.149 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-20T21:07:27.149 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-20T21:07:27.149 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-20T21:07:27.149 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-20T21:07:27.149 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-20T21:07:27.150 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-20T21:07:27.150 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-20T21:07:27.150 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-20T21:07:27.150 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-20T21:07:27.150 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-20T21:07:27.150 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-20T21:07:27.150 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-20T21:07:27.150 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-20T21:07:27.150 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-20T21:07:27.150 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-20T21:07:27.150 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-20T21:07:27.150 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-20T21:07:27.151 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-20T21:07:27.151 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-20T21:07:27.151 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-20T21:07:27.151 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-20T21:07:27.151 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-20T21:07:27.151 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-20T21:07:27.151 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-20T21:07:27.151 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-20T21:07:27.151 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-20T21:07:27.151 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-20T21:07:27.151 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-20T21:07:27.151 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-20T21:07:27.151 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-20T21:07:27.151 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-20T21:07:27.151 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-20T21:07:27.152 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-20T21:07:27.152 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-20T21:07:27.152 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-20T21:07:27.152 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-20T21:07:27.152 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-20T21:07:27.152 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-20T21:07:27.152 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-20T21:07:27.152 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-20T21:07:27.152 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-20T21:07:27.153 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-20T21:07:27.153 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-20T21:07:27.153 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-20T21:07:27.153 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-20T21:07:27.153 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-20T21:07:27.153 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-20T21:07:27.153 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-20T21:07:27.153 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-20T21:07:27.153 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-20T21:07:27.153 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-20T21:07:27.153 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-20T21:07:27.153 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-20T21:07:27.153 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-20T21:07:27.153 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-20T21:07:27.154 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-20T21:07:27.154 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-20T21:07:27.154 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-20T21:07:27.154 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-20T21:07:27.154 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-20T21:07:27.154 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-20T21:07:27.154 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-20T21:07:27.154 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-20T21:07:27.154 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-20T21:07:27.154 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-20T21:07:27.154 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-20T21:07:27.154 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-20T21:07:27.154 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-20T21:07:27.154 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-20T21:07:27.155 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-20T21:07:27.155 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-20T21:07:27.155 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-20T21:07:27.155 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-20T21:07:27.155 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-20T21:07:27.155 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-20T21:07:27.155 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-20T21:07:27.155 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-20T21:07:27.155 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-20T21:07:27.155 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-20T21:07:27.155 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-20T21:07:27.155 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-20T21:07:27.155 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-20T21:07:27.155 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-20T21:07:27.155 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-20T21:07:27.156 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-20T21:07:27.156 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-20T21:07:27.156 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-20T21:07:27.156 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-20T21:07:27.156 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-20T21:07:27.156 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-20T21:07:27.156 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-20T21:07:27.156 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-20T21:07:27.156 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-20T21:07:27.156 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-20T21:07:27.156 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-20T21:07:27.156 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-20T21:07:27.156 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-20T21:07:27.157 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-20T21:07:27.157 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-20T21:07:27.157 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-20T21:07:27.157 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-20T21:07:27.157 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-20T21:07:27.157 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-20T21:07:27.157 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-20T21:07:27.157 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-20T21:07:27.157 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-20T21:07:27.157 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-20T21:07:27.157 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-20T21:07:27.157 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-20T21:07:27.157 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-20T21:07:27.157 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-20T21:07:27.158 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-20T21:07:27.158 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-20T21:07:27.158 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-20T21:07:27.158 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-20T21:07:27.158 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-20T21:07:27.158 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-20T21:07:27.158 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-20T21:07:27.158 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-20T21:07:27.158 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-20T21:07:27.158 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-20T21:07:27.159 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-20T21:07:27.159 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-20T21:07:27.159 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-20T21:07:27.159 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-20T21:07:27.159 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-20T21:07:27.159 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-20T21:07:27.159 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-20T21:07:27.159 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-20T21:07:27.159 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-20T21:07:27.160 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-20T21:07:27.160 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-20T21:07:27.160 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-20T21:07:27.160 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-20T21:07:27.160 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-20T21:07:27.160 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-20T21:07:27.160 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-20T21:07:27.160 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-20T21:07:27.160 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-20T21:07:27.160 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-20T21:07:27.160 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-20T21:07:27.160 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-20T21:07:27.160 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-20T21:07:27.160 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-20T21:07:27.161 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-20T21:07:27.161 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-20T21:07:27.161 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-20T21:07:27.161 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-20T21:07:27.161 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-20T21:07:27.161 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-20T21:07:27.161 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-20T21:07:27.161 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-20T21:07:27.161 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-20T21:07:27.161 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-20T21:07:27.161 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-20T21:07:27.161 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-20T21:07:27.161 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-20T21:07:27.161 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-20T21:07:27.162 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-20T21:07:27.162 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-20T21:07:27.162 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-20T21:07:27.162 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-20T21:07:27.162 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-20T21:07:27.162 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-20T21:07:27.162 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-20T21:07:27.162 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-20T21:07:27.162 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-20T21:07:27.162 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-20T21:07:27.162 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-20T21:07:27.162 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-20T21:07:27.162 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-20T21:07:27.162 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-20T21:07:27.163 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-20T21:07:27.163 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-20T21:07:27.163 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-20T21:07:27.163 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-20T21:07:27.163 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-20T21:07:27.163 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-20T21:07:27.163 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-20T21:07:27.163 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-20T21:07:27.163 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-20T21:07:27.163 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-20T21:07:27.163 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-20T21:07:27.163 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-20T21:07:27.163 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-20T21:07:27.163 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-20T21:07:27.163 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-20T21:07:27.164 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-20T21:07:27.164 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-20T21:07:27.164 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-20T21:07:27.164 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-20T21:07:27.164 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-20T21:07:27.164 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-20T21:07:27.164 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-20T21:07:27.164 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-20T21:07:27.164 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-20T21:07:27.164 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-20T21:07:27.164 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-20T21:07:27.164 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-20T21:07:27.164 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-20T21:07:27.164 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-20T21:07:27.164 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-20T21:07:27.165 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-20T21:07:27.165 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-20T21:07:27.165 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-20T21:07:27.165 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-20T21:07:27.165 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-20T21:07:27.165 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-20T21:07:27.165 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-20T21:07:27.165 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-20T21:07:27.165 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-20T21:07:27.165 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-20T21:07:27.165 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-20T21:07:27.165 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-20T21:07:27.165 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-20T21:07:27.165 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-20T21:07:27.165 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-20T21:07:27.166 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-20T21:07:27.166 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-20T21:07:27.166 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-20T21:07:27.166 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-20T21:07:27.166 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-20T21:07:27.166 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-20T21:07:27.166 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-20T21:07:27.166 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-20T21:07:27.166 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-20T21:07:27.166 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-20T21:07:27.166 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-20T21:07:27.166 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-20T21:07:27.166 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-20T21:07:27.167 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-20T21:07:27.167 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-20T21:07:27.167 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-20T21:07:27.167 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-20T21:07:27.167 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-20T21:07:27.167 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-20T21:07:27.167 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-20T21:07:27.167 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-20T21:07:27.167 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-20T21:07:27.167 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-20T21:07:27.167 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-20T21:07:27.167 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-20T21:07:27.167 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-20T21:07:27.167 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-20T21:07:27.167 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-20T21:07:27.168 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-20T21:07:27.168 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-20T21:07:27.168 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-20T21:07:27.168 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-20T21:07:27.168 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-20T21:07:27.168 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-20T21:07:27.168 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-20T21:07:27.168 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-20T21:07:27.168 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-20T21:07:27.168 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-20T21:07:27.168 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-20T21:07:27.168 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-20T21:07:27.169 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-20T21:07:27.169 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-20T21:07:27.169 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-20T21:07:27.169 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-20T21:07:27.169 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-20T21:07:27.169 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-20T21:07:27.169 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-20T21:07:27.169 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-20T21:07:27.169 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-20T21:07:27.170 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-20T21:07:27.170 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-20T21:07:27.170 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-20T21:07:27.170 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-20T21:07:27.170 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-20T21:07:27.170 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-20T21:07:27.170 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-20T21:07:27.170 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-20T21:07:27.170 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-20T21:07:27.170 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-20T21:07:27.170 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-20T21:07:27.170 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-20T21:07:27.170 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-20T21:07:27.171 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-20T21:07:27.171 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-20T21:07:27.171 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-20T21:07:27.171 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-20T21:07:27.171 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-20T21:07:27.171 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-20T21:07:27.171 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-20T21:07:27.171 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-20T21:07:27.171 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-20T21:07:27.171 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-20T21:07:27.171 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-20T21:07:27.171 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-20T21:07:27.171 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-20T21:07:27.171 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-20T21:07:27.172 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-20T21:07:27.172 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-20T21:07:27.172 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-20T21:07:27.172 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-20T21:07:27.172 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-20T21:07:27.172 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-20T21:07:27.172 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-20T21:07:27.172 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-20T21:07:27.172 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-20T21:07:27.172 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-20T21:07:27.172 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-20T21:07:27.173 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-20T21:07:27.173 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-20T21:07:27.173 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-20T21:07:27.173 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-20T21:07:27.173 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-20T21:07:27.173 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-20T21:07:27.173 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-20T21:07:27.173 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-20T21:07:27.173 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-20T21:07:27.173 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-20T21:07:27.173 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-20T21:07:27.173 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-20T21:07:27.173 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-20T21:07:27.173 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-20T21:07:27.173 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-20T21:07:27.174 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-20T21:07:27.174 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-20T21:07:27.174 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-20T21:07:27.174 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-20T21:07:27.174 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-20T21:07:27.174 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-20T21:07:27.174 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-20T21:07:27.174 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-20T21:07:27.174 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-20T21:07:27.175 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-20T21:07:27.175 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-20T21:07:27.175 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-20T21:07:27.175 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-20T21:07:27.175 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-20T21:07:27.175 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-20T21:07:27.175 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-20T21:07:27.175 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-20T21:07:27.175 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-20T21:07:27.176 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-20T21:07:27.176 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-20T21:07:27.176 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-20T21:07:27.176 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-20T21:07:27.176 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-20T21:07:27.176 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-20T21:07:27.176 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-20T21:07:27.176 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-20T21:07:27.176 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-20T21:07:27.177 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-20T21:07:27.177 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-20T21:07:27.177 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-20T21:07:27.177 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-20T21:07:27.177 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-20T21:07:27.177 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-20T21:07:27.177 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-20T21:07:27.177 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-20T21:07:27.178 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-20T21:07:27.178 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-20T21:07:27.178 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-20T21:07:27.178 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-20T21:07:27.178 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-20T21:07:27.178 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-20T21:07:27.178 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-20T21:07:27.178 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-20T21:07:27.178 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-20T21:07:27.179 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-20T21:07:27.179 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-20T21:07:27.179 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-20T21:07:27.179 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-20T21:07:27.179 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-20T21:07:27.179 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-20T21:07:27.179 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-20T21:07:27.179 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-20T21:07:27.180 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-20T21:07:27.180 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-20T21:07:27.180 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-20T21:07:27.180 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-20T21:07:27.180 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-20T21:07:27.180 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-20T21:07:27.180 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-20T21:07:27.180 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-20T21:07:27.181 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-20T21:07:27.181 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-20T21:07:27.181 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-20T21:07:27.181 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-20T21:07:27.181 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-20T21:07:27.181 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-20T21:07:27.181 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-20T21:07:27.181 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-20T21:07:27.182 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-20T21:07:27.182 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-20T21:07:27.182 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-20T21:07:27.182 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-20T21:07:27.182 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-20T21:07:27.182 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-20T21:07:27.182 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-20T21:07:27.182 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-20T21:07:27.182 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-20T21:07:27.182 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-20T21:07:27.182 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-20T21:07:27.182 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-20T21:07:27.182 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-20T21:07:27.182 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-20T21:07:27.183 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-20T21:07:27.183 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-20T21:07:27.183 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-20T21:07:27.183 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-20T21:07:27.183 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-20T21:07:27.183 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-20T21:07:27.183 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-20T21:07:27.183 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-20T21:07:27.183 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-20T21:07:27.183 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-20T21:07:27.183 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-20T21:07:27.183 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-20T21:07:27.183 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-20T21:07:27.184 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-20T21:07:27.184 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-20T21:07:27.184 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-20T21:07:27.184 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-20T21:07:27.184 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-20T21:07:27.184 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-20T21:07:27.184 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-20T21:07:27.184 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-20T21:07:27.184 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-20T21:07:27.184 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-20T21:07:27.184 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-20T21:07:27.184 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-20T21:07:27.184 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-20T21:07:27.184 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-20T21:07:27.184 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-20T21:07:27.185 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-20T21:07:27.185 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-20T21:07:27.185 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-20T21:07:27.185 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-20T21:07:27.185 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-20T21:07:27.185 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-20T21:07:27.185 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-20T21:07:27.185 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-20T21:07:27.185 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-20T21:07:27.185 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-20T21:07:27.185 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-20T21:07:27.185 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-20T21:07:27.185 INFO:teuthology.orchestra.run.smithi086.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-20T21:07:27.189 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-20T21:07:27.190 DEBUG:teuthology.orchestra.run.smithi086:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-20T21:07:27.212 INFO:teuthology.orchestra.run.smithi086.stdout:postinstall scriptlet (using /bin/sh): 2024-04-20T21:07:27.213 INFO:teuthology.orchestra.run.smithi086.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-20T21:07:27.213 INFO:teuthology.orchestra.run.smithi086.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-20T21:07:27.213 INFO:teuthology.orchestra.run.smithi086.stdout:fi 2024-04-20T21:07:27.213 INFO:teuthology.orchestra.run.smithi086.stdout:for file in vmlinuz System.map config; do 2024-04-20T21:07:27.213 INFO:teuthology.orchestra.run.smithi086.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-20T21:07:27.213 INFO:teuthology.orchestra.run.smithi086.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-20T21:07:27.213 INFO:teuthology.orchestra.run.smithi086.stdout: fi 2024-04-20T21:07:27.213 INFO:teuthology.orchestra.run.smithi086.stdout:done 2024-04-20T21:07:27.213 INFO:teuthology.orchestra.run.smithi086.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-20T21:07:27.213 INFO:teuthology.orchestra.run.smithi086.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-20T21:07:27.213 INFO:teuthology.orchestra.run.smithi086.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-20T21:07:27.213 INFO:teuthology.orchestra.run.smithi086.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-20T21:07:27.213 INFO:teuthology.orchestra.run.smithi086.stdout:fi 2024-04-20T21:07:27.213 INFO:teuthology.orchestra.run.smithi086.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-20T21:07:27.214 INFO:teuthology.orchestra.run.smithi086.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-20T21:07:27.214 INFO:teuthology.orchestra.run.smithi086.stdout:fi 2024-04-20T21:07:27.214 INFO:teuthology.orchestra.run.smithi086.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-20T21:07:27.214 INFO:teuthology.orchestra.run.smithi086.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-20T21:07:27.214 INFO:teuthology.orchestra.run.smithi086.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-20T21:07:27.214 INFO:teuthology.orchestra.run.smithi086.stdout:fi 2024-04-20T21:07:27.215 DEBUG:teuthology.orchestra.run.smithi086:> sudo rpm -qi grub2-tools 2024-04-20T21:07:27.290 INFO:teuthology.orchestra.run.smithi086.stdout:Name : grub2-tools 2024-04-20T21:07:27.290 INFO:teuthology.orchestra.run.smithi086.stdout:Epoch : 1 2024-04-20T21:07:27.290 INFO:teuthology.orchestra.run.smithi086.stdout:Version : 2.06 2024-04-20T21:07:27.290 INFO:teuthology.orchestra.run.smithi086.stdout:Release : 68.el9 2024-04-20T21:07:27.290 INFO:teuthology.orchestra.run.smithi086.stdout:Architecture: x86_64 2024-04-20T21:07:27.290 INFO:teuthology.orchestra.run.smithi086.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-20T21:07:27.291 INFO:teuthology.orchestra.run.smithi086.stdout:Group : Unspecified 2024-04-20T21:07:27.291 INFO:teuthology.orchestra.run.smithi086.stdout:Size : 8264254 2024-04-20T21:07:27.291 INFO:teuthology.orchestra.run.smithi086.stdout:License : GPLv3+ 2024-04-20T21:07:27.291 INFO:teuthology.orchestra.run.smithi086.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-20T21:07:27.291 INFO:teuthology.orchestra.run.smithi086.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-20T21:07:27.291 INFO:teuthology.orchestra.run.smithi086.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-20T21:07:27.291 INFO:teuthology.orchestra.run.smithi086.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-20T21:07:27.291 INFO:teuthology.orchestra.run.smithi086.stdout:Packager : builder@centos.org 2024-04-20T21:07:27.291 INFO:teuthology.orchestra.run.smithi086.stdout:Vendor : CentOS 2024-04-20T21:07:27.291 INFO:teuthology.orchestra.run.smithi086.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-20T21:07:27.291 INFO:teuthology.orchestra.run.smithi086.stdout:Summary : Support tools for GRUB. 2024-04-20T21:07:27.292 INFO:teuthology.orchestra.run.smithi086.stdout:Description : 2024-04-20T21:07:27.292 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:07:27.292 INFO:teuthology.orchestra.run.smithi086.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-20T21:07:27.292 INFO:teuthology.orchestra.run.smithi086.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-20T21:07:27.292 INFO:teuthology.orchestra.run.smithi086.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-20T21:07:27.292 INFO:teuthology.orchestra.run.smithi086.stdout:hardware devices. 2024-04-20T21:07:27.292 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:07:27.292 INFO:teuthology.orchestra.run.smithi086.stdout:This subpackage provides tools for support of all platforms. 2024-04-20T21:07:27.293 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-20T21:07:27.293 INFO:teuthology.task.kernel:Updating grub on smithi086 to boot 6.9.0-rc3-g806ec36b2112 2024-04-20T21:07:27.293 DEBUG:teuthology.orchestra.run.smithi086:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-20T21:07:27.871 INFO:teuthology.orchestra.run.smithi086.stderr:Generating grub configuration file ... 2024-04-20T21:07:29.295 INFO:teuthology.orchestra.run.smithi086.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-20T21:07:29.340 INFO:teuthology.orchestra.run.smithi086.stderr:done 2024-04-20T21:07:29.342 DEBUG:teuthology.orchestra.run.smithi086:> mktemp 2024-04-20T21:07:29.363 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/tmp.nIZ4cJuJbX 2024-04-20T21:07:29.363 DEBUG:teuthology.orchestra.run.smithi086:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.nIZ4cJuJbX 2024-04-20T21:07:29.429 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 0666 /tmp/tmp.nIZ4cJuJbX 2024-04-20T21:07:29.565 DEBUG:teuthology.orchestra.remote:smithi086:/tmp/tmp.nIZ4cJuJbX is 6KB 2024-04-20T21:07:29.614 DEBUG:teuthology.orchestra.run.smithi086:> rm -fr /tmp/tmp.nIZ4cJuJbX 2024-04-20T21:07:29.628 DEBUG:teuthology.orchestra.run.smithi086:> sudo /bin/ls /boot/loader/entries || true 2024-04-20T21:07:29.707 INFO:teuthology.orchestra.run.smithi086.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-20T21:07:29.707 INFO:teuthology.orchestra.run.smithi086.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-20T21:07:29.707 INFO:teuthology.orchestra.run.smithi086.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-20T21:07:29.707 INFO:teuthology.orchestra.run.smithi086.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-20T21:07:29.708 DEBUG:teuthology.orchestra.run.smithi086:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-20T21:07:29.841 DEBUG:teuthology.orchestra.run.smithi086:> sudo shutdown -r now 2024-04-20T21:07:59.844 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-20T21:07:59.845 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-20T21:07:59.846 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T21:08:18.295 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20 21:08:18 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-20T21:08:18.295 INFO:teuthology.orchestra.run.smithi144.stderr:FINISHED --2024-04-20 21:08:18-- 2024-04-20T21:08:18.295 INFO:teuthology.orchestra.run.smithi144.stderr:Total wall clock time: 9m 17s 2024-04-20T21:08:18.295 INFO:teuthology.orchestra.run.smithi144.stderr:Downloaded: 1 files, 341M in 9m 17s (627 KB/s) 2024-04-20T21:08:18.299 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.b... 2024-04-20T21:08:18.299 DEBUG:teuthology.orchestra.run.smithi144:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-20T21:08:20.388 INFO:teuthology.orchestra.run.smithi144.stdout:Verifying... ######################################## 2024-04-20T21:08:20.622 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing... ######################################## 2024-04-20T21:08:20.737 INFO:teuthology.orchestra.run.smithi144.stdout:Updating / installing... 2024-04-20T21:08:28.724 INFO:teuthology.orchestra.run.smithi144.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-20T21:08:59.849 DEBUG:teuthology.orchestra.remote:timed out 2024-04-20T21:09:06.002 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi144.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-20T21:09:06.002 DEBUG:teuthology.orchestra.run.smithi144:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-20T21:09:06.096 INFO:teuthology.orchestra.run.smithi144.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-20T21:09:06.096 INFO:teuthology.orchestra.run.smithi144.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-20T21:09:06.096 INFO:teuthology.orchestra.run.smithi144.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-20T21:09:06.096 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-20T21:09:06.096 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-20T21:09:06.096 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-20T21:09:06.096 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-20T21:09:06.097 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-20T21:09:06.097 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-20T21:09:06.097 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-20T21:09:06.097 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-20T21:09:06.097 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-20T21:09:06.097 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-20T21:09:06.097 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-20T21:09:06.097 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-20T21:09:06.097 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-20T21:09:06.097 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-20T21:09:06.097 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-20T21:09:06.097 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-20T21:09:06.097 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-20T21:09:06.098 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-20T21:09:06.098 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-20T21:09:06.098 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-20T21:09:06.098 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-20T21:09:06.098 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-20T21:09:06.098 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-20T21:09:06.098 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-20T21:09:06.098 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-20T21:09:06.098 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-20T21:09:06.098 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-20T21:09:06.098 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-20T21:09:06.098 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-20T21:09:06.098 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-20T21:09:06.099 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-20T21:09:06.099 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-20T21:09:06.099 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-20T21:09:06.099 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-20T21:09:06.099 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-20T21:09:06.099 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-20T21:09:06.099 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-20T21:09:06.099 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-20T21:09:06.099 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-20T21:09:06.099 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-20T21:09:06.099 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-20T21:09:06.099 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-20T21:09:06.100 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-20T21:09:06.100 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-20T21:09:06.100 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-20T21:09:06.100 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-20T21:09:06.100 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-20T21:09:06.100 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-20T21:09:06.100 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-20T21:09:06.100 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-20T21:09:06.100 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-20T21:09:06.100 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-20T21:09:06.100 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-20T21:09:06.100 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-20T21:09:06.100 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-20T21:09:06.101 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-20T21:09:06.101 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-20T21:09:06.101 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-20T21:09:06.101 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-20T21:09:06.101 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-20T21:09:06.101 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-20T21:09:06.101 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-20T21:09:06.101 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-20T21:09:06.101 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-20T21:09:06.101 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-20T21:09:06.101 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-20T21:09:06.101 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-20T21:09:06.101 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-20T21:09:06.101 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-20T21:09:06.102 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-20T21:09:06.102 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-20T21:09:06.102 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-20T21:09:06.102 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-20T21:09:06.102 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-20T21:09:06.102 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-20T21:09:06.102 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-20T21:09:06.102 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-20T21:09:06.102 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-20T21:09:06.102 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-20T21:09:06.102 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-20T21:09:06.102 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-20T21:09:06.102 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-20T21:09:06.102 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-20T21:09:06.103 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-20T21:09:06.103 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-20T21:09:06.103 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-20T21:09:06.103 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-20T21:09:06.103 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-20T21:09:06.103 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-20T21:09:06.103 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-20T21:09:06.103 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-20T21:09:06.103 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-20T21:09:06.103 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-20T21:09:06.103 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-20T21:09:06.103 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-20T21:09:06.103 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-20T21:09:06.103 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-20T21:09:06.103 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-20T21:09:06.103 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-20T21:09:06.104 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-20T21:09:06.104 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-20T21:09:06.104 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-20T21:09:06.104 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-20T21:09:06.104 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-20T21:09:06.104 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-20T21:09:06.104 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-20T21:09:06.104 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-20T21:09:06.104 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-20T21:09:06.104 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-20T21:09:06.104 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-20T21:09:06.104 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-20T21:09:06.104 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-20T21:09:06.104 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-20T21:09:06.105 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-20T21:09:06.105 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-20T21:09:06.105 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-20T21:09:06.105 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-20T21:09:06.105 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-20T21:09:06.105 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-20T21:09:06.105 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-20T21:09:06.105 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-20T21:09:06.105 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-20T21:09:06.105 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-20T21:09:06.105 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-20T21:09:06.105 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-20T21:09:06.105 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-20T21:09:06.105 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-20T21:09:06.106 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-20T21:09:06.106 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-20T21:09:06.106 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-20T21:09:06.106 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-20T21:09:06.106 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-20T21:09:06.106 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-20T21:09:06.106 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-20T21:09:06.106 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-20T21:09:06.106 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-20T21:09:06.106 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-20T21:09:06.106 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-20T21:09:06.106 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-20T21:09:06.106 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-20T21:09:06.107 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-20T21:09:06.107 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-20T21:09:06.107 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-20T21:09:06.107 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-20T21:09:06.107 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-20T21:09:06.107 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-20T21:09:06.107 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-20T21:09:06.107 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-20T21:09:06.107 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-20T21:09:06.107 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-20T21:09:06.107 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-20T21:09:06.107 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-20T21:09:06.107 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-20T21:09:06.107 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-20T21:09:06.108 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-20T21:09:06.108 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-20T21:09:06.108 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-20T21:09:06.108 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-20T21:09:06.108 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-20T21:09:06.108 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-20T21:09:06.108 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-20T21:09:06.108 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-20T21:09:06.108 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-20T21:09:06.108 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-20T21:09:06.108 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-20T21:09:06.108 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-20T21:09:06.108 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-20T21:09:06.108 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-20T21:09:06.109 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-20T21:09:06.109 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-20T21:09:06.109 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-20T21:09:06.109 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-20T21:09:06.109 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-20T21:09:06.109 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-20T21:09:06.109 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-20T21:09:06.109 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-20T21:09:06.109 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-20T21:09:06.109 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-20T21:09:06.109 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-20T21:09:06.109 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-20T21:09:06.109 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-20T21:09:06.109 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-20T21:09:06.110 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-20T21:09:06.110 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-20T21:09:06.110 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-20T21:09:06.110 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-20T21:09:06.110 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-20T21:09:06.110 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-20T21:09:06.110 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-20T21:09:06.110 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-20T21:09:06.110 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-20T21:09:06.110 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-20T21:09:06.110 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-20T21:09:06.110 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-20T21:09:06.110 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-20T21:09:06.110 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-20T21:09:06.111 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-20T21:09:06.111 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-20T21:09:06.111 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-20T21:09:06.111 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-20T21:09:06.111 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-20T21:09:06.111 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-20T21:09:06.111 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-20T21:09:06.111 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-20T21:09:06.111 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-20T21:09:06.111 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-20T21:09:06.111 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-20T21:09:06.111 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-20T21:09:06.111 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-20T21:09:06.111 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-20T21:09:06.112 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-20T21:09:06.112 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-20T21:09:06.112 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-20T21:09:06.112 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-20T21:09:06.112 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-20T21:09:06.112 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-20T21:09:06.112 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-20T21:09:06.112 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-20T21:09:06.112 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-20T21:09:06.112 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-20T21:09:06.112 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-20T21:09:06.112 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-20T21:09:06.112 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-20T21:09:06.113 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-20T21:09:06.113 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-20T21:09:06.113 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-20T21:09:06.113 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-20T21:09:06.113 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-20T21:09:06.113 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-20T21:09:06.113 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-20T21:09:06.113 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-20T21:09:06.113 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-20T21:09:06.113 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-20T21:09:06.113 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-20T21:09:06.113 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-20T21:09:06.113 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-20T21:09:06.113 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-20T21:09:06.114 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-20T21:09:06.114 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-20T21:09:06.114 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-20T21:09:06.114 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-20T21:09:06.114 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-20T21:09:06.114 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-20T21:09:06.114 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-20T21:09:06.114 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-20T21:09:06.114 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-20T21:09:06.114 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-20T21:09:06.114 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-20T21:09:06.114 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-20T21:09:06.114 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-20T21:09:06.115 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-20T21:09:06.115 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-20T21:09:06.115 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-20T21:09:06.115 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-20T21:09:06.115 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-20T21:09:06.115 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-20T21:09:06.115 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-20T21:09:06.115 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-20T21:09:06.115 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-20T21:09:06.115 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-20T21:09:06.115 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-20T21:09:06.115 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-20T21:09:06.115 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-20T21:09:06.115 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-20T21:09:06.115 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-20T21:09:06.116 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-20T21:09:06.116 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-20T21:09:06.116 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-20T21:09:06.116 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-20T21:09:06.116 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-20T21:09:06.116 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-20T21:09:06.116 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-20T21:09:06.116 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-20T21:09:06.116 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-20T21:09:06.116 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-20T21:09:06.116 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-20T21:09:06.116 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-20T21:09:06.117 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-20T21:09:06.117 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-20T21:09:06.117 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-20T21:09:06.117 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-20T21:09:06.117 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-20T21:09:06.117 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-20T21:09:06.117 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-20T21:09:06.117 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-20T21:09:06.117 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-20T21:09:06.117 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-20T21:09:06.117 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-20T21:09:06.117 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-20T21:09:06.117 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-20T21:09:06.117 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-20T21:09:06.117 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-20T21:09:06.118 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-20T21:09:06.118 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-20T21:09:06.118 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-20T21:09:06.118 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-20T21:09:06.118 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-20T21:09:06.118 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-20T21:09:06.118 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-20T21:09:06.118 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-20T21:09:06.118 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-20T21:09:06.118 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-20T21:09:06.118 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-20T21:09:06.118 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-20T21:09:06.118 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-20T21:09:06.118 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-20T21:09:06.119 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-20T21:09:06.119 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-20T21:09:06.119 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-20T21:09:06.119 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-20T21:09:06.119 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-20T21:09:06.119 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-20T21:09:06.119 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-20T21:09:06.119 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-20T21:09:06.119 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-20T21:09:06.119 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-20T21:09:06.119 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-20T21:09:06.119 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-20T21:09:06.119 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-20T21:09:06.119 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-20T21:09:06.120 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-20T21:09:06.120 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-20T21:09:06.120 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-20T21:09:06.120 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-20T21:09:06.120 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-20T21:09:06.120 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-20T21:09:06.120 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-20T21:09:06.120 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-20T21:09:06.120 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-20T21:09:06.120 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-20T21:09:06.120 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-20T21:09:06.120 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-20T21:09:06.120 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-20T21:09:06.120 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-20T21:09:06.120 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-20T21:09:06.121 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-20T21:09:06.121 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-20T21:09:06.121 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-20T21:09:06.121 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-20T21:09:06.121 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-20T21:09:06.121 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-20T21:09:06.121 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-20T21:09:06.121 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-20T21:09:06.121 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-20T21:09:06.121 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-20T21:09:06.121 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-20T21:09:06.121 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-20T21:09:06.121 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-20T21:09:06.121 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-20T21:09:06.121 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-20T21:09:06.122 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-20T21:09:06.122 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-20T21:09:06.122 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-20T21:09:06.122 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-20T21:09:06.122 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-20T21:09:06.122 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-20T21:09:06.122 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-20T21:09:06.122 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-20T21:09:06.122 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-20T21:09:06.122 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-20T21:09:06.122 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-20T21:09:06.122 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-20T21:09:06.122 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-20T21:09:06.123 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-20T21:09:06.123 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-20T21:09:06.123 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-20T21:09:06.123 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-20T21:09:06.123 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-20T21:09:06.123 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-20T21:09:06.123 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-20T21:09:06.123 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-20T21:09:06.123 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-20T21:09:06.123 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-20T21:09:06.123 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-20T21:09:06.123 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-20T21:09:06.123 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-20T21:09:06.123 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-20T21:09:06.123 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-20T21:09:06.124 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-20T21:09:06.124 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-20T21:09:06.124 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-20T21:09:06.124 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-20T21:09:06.125 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-20T21:09:06.125 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-20T21:09:06.125 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-20T21:09:06.125 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-20T21:09:06.125 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-20T21:09:06.125 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-20T21:09:06.126 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-20T21:09:06.126 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-20T21:09:06.126 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-20T21:09:06.126 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-20T21:09:06.126 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-20T21:09:06.126 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-20T21:09:06.126 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-20T21:09:06.126 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-20T21:09:06.126 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-20T21:09:06.126 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-20T21:09:06.126 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-20T21:09:06.126 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-20T21:09:06.126 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-20T21:09:06.126 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-20T21:09:06.127 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-20T21:09:06.127 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-20T21:09:06.127 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-20T21:09:06.127 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-20T21:09:06.127 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-20T21:09:06.127 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-20T21:09:06.127 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-20T21:09:06.127 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-20T21:09:06.127 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-20T21:09:06.127 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-20T21:09:06.127 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-20T21:09:06.127 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-20T21:09:06.127 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-20T21:09:06.127 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-20T21:09:06.127 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-20T21:09:06.128 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-20T21:09:06.128 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-20T21:09:06.128 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-20T21:09:06.128 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-20T21:09:06.128 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-20T21:09:06.128 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-20T21:09:06.128 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-20T21:09:06.128 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-20T21:09:06.128 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-20T21:09:06.128 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-20T21:09:06.128 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-20T21:09:06.128 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-20T21:09:06.128 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-20T21:09:06.128 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-20T21:09:06.129 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-20T21:09:06.129 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-20T21:09:06.129 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-20T21:09:06.129 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-20T21:09:06.129 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-20T21:09:06.129 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-20T21:09:06.129 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-20T21:09:06.129 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-20T21:09:06.129 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-20T21:09:06.129 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-20T21:09:06.129 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-20T21:09:06.129 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-20T21:09:06.129 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-20T21:09:06.129 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-20T21:09:06.130 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-20T21:09:06.130 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-20T21:09:06.130 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-20T21:09:06.130 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-20T21:09:06.130 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-20T21:09:06.130 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-20T21:09:06.130 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-20T21:09:06.130 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-20T21:09:06.130 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-20T21:09:06.130 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-20T21:09:06.130 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-20T21:09:06.130 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-20T21:09:06.130 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-20T21:09:06.131 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-20T21:09:06.131 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-20T21:09:06.131 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-20T21:09:06.131 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-20T21:09:06.131 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-20T21:09:06.131 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-20T21:09:06.131 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-20T21:09:06.131 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-20T21:09:06.131 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-20T21:09:06.131 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-20T21:09:06.131 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-20T21:09:06.132 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-20T21:09:06.132 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-20T21:09:06.132 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-20T21:09:06.132 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-20T21:09:06.132 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-20T21:09:06.132 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-20T21:09:06.132 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-20T21:09:06.132 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-20T21:09:06.132 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-20T21:09:06.132 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-20T21:09:06.132 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-20T21:09:06.132 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-20T21:09:06.132 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-20T21:09:06.132 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-20T21:09:06.133 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-20T21:09:06.133 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-20T21:09:06.133 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-20T21:09:06.133 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-20T21:09:06.133 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-20T21:09:06.133 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-20T21:09:06.133 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-20T21:09:06.133 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-20T21:09:06.133 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-20T21:09:06.133 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-20T21:09:06.133 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-20T21:09:06.133 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-20T21:09:06.133 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-20T21:09:06.133 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-20T21:09:06.134 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-20T21:09:06.134 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-20T21:09:06.134 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-20T21:09:06.134 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-20T21:09:06.134 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-20T21:09:06.134 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-20T21:09:06.134 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-20T21:09:06.134 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-20T21:09:06.134 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-20T21:09:06.134 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-20T21:09:06.134 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-20T21:09:06.134 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-20T21:09:06.134 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-20T21:09:06.134 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-20T21:09:06.135 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-20T21:09:06.135 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-20T21:09:06.135 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-20T21:09:06.135 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-20T21:09:06.135 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-20T21:09:06.135 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-20T21:09:06.135 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-20T21:09:06.135 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-20T21:09:06.135 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-20T21:09:06.135 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-20T21:09:06.135 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-20T21:09:06.135 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-20T21:09:06.135 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-20T21:09:06.135 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-20T21:09:06.135 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-20T21:09:06.136 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-20T21:09:06.136 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-20T21:09:06.136 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-20T21:09:06.136 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-20T21:09:06.136 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-20T21:09:06.136 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-20T21:09:06.136 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-20T21:09:06.136 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-20T21:09:06.136 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-20T21:09:06.136 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-20T21:09:06.136 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-20T21:09:06.136 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-20T21:09:06.136 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-20T21:09:06.136 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-20T21:09:06.136 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-20T21:09:06.137 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-20T21:09:06.137 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-20T21:09:06.137 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-20T21:09:06.137 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-20T21:09:06.137 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-20T21:09:06.137 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-20T21:09:06.137 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-20T21:09:06.137 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-20T21:09:06.137 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-20T21:09:06.137 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-20T21:09:06.137 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-20T21:09:06.137 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-20T21:09:06.137 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-20T21:09:06.137 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-20T21:09:06.138 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-20T21:09:06.138 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-20T21:09:06.138 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-20T21:09:06.138 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-20T21:09:06.138 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-20T21:09:06.138 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-20T21:09:06.138 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-20T21:09:06.138 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-20T21:09:06.138 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-20T21:09:06.138 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-20T21:09:06.138 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-20T21:09:06.138 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-20T21:09:06.138 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-20T21:09:06.139 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-20T21:09:06.139 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-20T21:09:06.139 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-20T21:09:06.139 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-20T21:09:06.139 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-20T21:09:06.139 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-20T21:09:06.139 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-20T21:09:06.139 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-20T21:09:06.139 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-20T21:09:06.139 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-20T21:09:06.139 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-20T21:09:06.139 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-20T21:09:06.139 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-20T21:09:06.139 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-20T21:09:06.140 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-20T21:09:06.140 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-20T21:09:06.140 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-20T21:09:06.140 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-20T21:09:06.140 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-20T21:09:06.140 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-20T21:09:06.140 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-20T21:09:06.140 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-20T21:09:06.140 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-20T21:09:06.140 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-20T21:09:06.140 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-20T21:09:06.140 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-20T21:09:06.140 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-20T21:09:06.140 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-20T21:09:06.140 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-20T21:09:06.141 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-20T21:09:06.141 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-20T21:09:06.141 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-20T21:09:06.141 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-20T21:09:06.141 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-20T21:09:06.141 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-20T21:09:06.141 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-20T21:09:06.141 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-20T21:09:06.141 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-20T21:09:06.141 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-20T21:09:06.141 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-20T21:09:06.141 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-20T21:09:06.141 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-20T21:09:06.141 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-20T21:09:06.141 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-20T21:09:06.142 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-20T21:09:06.142 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-20T21:09:06.142 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-20T21:09:06.142 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-20T21:09:06.142 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-20T21:09:06.142 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-20T21:09:06.142 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-20T21:09:06.142 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-20T21:09:06.142 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-20T21:09:06.142 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-20T21:09:06.142 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-20T21:09:06.142 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-20T21:09:06.142 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-20T21:09:06.142 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-20T21:09:06.143 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-20T21:09:06.143 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-20T21:09:06.143 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-20T21:09:06.143 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-20T21:09:06.143 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-20T21:09:06.143 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-20T21:09:06.143 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-20T21:09:06.143 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-20T21:09:06.143 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-20T21:09:06.143 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-20T21:09:06.143 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-20T21:09:06.143 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-20T21:09:06.143 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-20T21:09:06.143 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-20T21:09:06.143 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-20T21:09:06.144 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-20T21:09:06.144 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-20T21:09:06.144 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-20T21:09:06.144 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-20T21:09:06.144 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-20T21:09:06.144 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-20T21:09:06.144 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-20T21:09:06.144 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-20T21:09:06.144 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-20T21:09:06.144 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-20T21:09:06.144 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-20T21:09:06.144 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-20T21:09:06.144 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-20T21:09:06.144 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-20T21:09:06.145 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-20T21:09:06.145 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-20T21:09:06.145 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-20T21:09:06.145 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-20T21:09:06.145 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-20T21:09:06.145 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-20T21:09:06.145 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-20T21:09:06.145 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-20T21:09:06.145 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-20T21:09:06.145 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-20T21:09:06.145 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-20T21:09:06.145 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-20T21:09:06.145 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-20T21:09:06.145 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-20T21:09:06.146 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-20T21:09:06.146 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-20T21:09:06.146 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-20T21:09:06.146 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-20T21:09:06.146 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-20T21:09:06.146 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-20T21:09:06.146 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-20T21:09:06.146 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-20T21:09:06.146 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-20T21:09:06.146 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-20T21:09:06.146 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-20T21:09:06.146 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-20T21:09:06.146 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-20T21:09:06.146 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-20T21:09:06.147 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-20T21:09:06.147 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-20T21:09:06.147 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-20T21:09:06.147 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-20T21:09:06.147 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-20T21:09:06.147 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-20T21:09:06.147 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-20T21:09:06.147 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-20T21:09:06.147 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-20T21:09:06.147 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-20T21:09:06.147 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-20T21:09:06.147 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-20T21:09:06.147 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-20T21:09:06.147 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-20T21:09:06.148 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-20T21:09:06.148 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-20T21:09:06.148 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-20T21:09:06.148 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-20T21:09:06.148 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-20T21:09:06.148 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-20T21:09:06.148 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-20T21:09:06.148 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-20T21:09:06.148 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-20T21:09:06.148 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-20T21:09:06.148 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-20T21:09:06.148 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-20T21:09:06.148 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-20T21:09:06.149 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-20T21:09:06.149 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-20T21:09:06.149 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-20T21:09:06.149 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-20T21:09:06.149 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-20T21:09:06.149 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-20T21:09:06.149 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-20T21:09:06.149 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-20T21:09:06.149 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-20T21:09:06.149 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-20T21:09:06.149 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-20T21:09:06.149 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-20T21:09:06.149 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-20T21:09:06.149 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-20T21:09:06.149 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-20T21:09:06.150 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-20T21:09:06.150 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-20T21:09:06.150 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-20T21:09:06.150 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-20T21:09:06.150 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-20T21:09:06.150 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-20T21:09:06.150 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-20T21:09:06.150 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-20T21:09:06.150 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-20T21:09:06.150 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-20T21:09:06.150 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-20T21:09:06.150 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-20T21:09:06.150 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-20T21:09:06.150 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-20T21:09:06.151 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-20T21:09:06.151 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-20T21:09:06.151 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-20T21:09:06.151 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-20T21:09:06.151 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-20T21:09:06.151 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-20T21:09:06.151 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-20T21:09:06.151 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-20T21:09:06.151 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-20T21:09:06.151 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-20T21:09:06.151 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-20T21:09:06.151 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-20T21:09:06.151 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-20T21:09:06.152 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-20T21:09:06.152 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-20T21:09:06.152 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-20T21:09:06.152 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-20T21:09:06.152 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-20T21:09:06.152 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-20T21:09:06.152 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-20T21:09:06.152 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-20T21:09:06.152 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-20T21:09:06.152 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-20T21:09:06.152 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-20T21:09:06.152 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-20T21:09:06.152 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-20T21:09:06.152 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-20T21:09:06.152 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-20T21:09:06.153 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-20T21:09:06.153 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-20T21:09:06.153 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-20T21:09:06.153 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-20T21:09:06.153 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-20T21:09:06.153 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-20T21:09:06.153 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-20T21:09:06.153 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-20T21:09:06.153 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-20T21:09:06.153 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-20T21:09:06.153 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-20T21:09:06.153 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-20T21:09:06.153 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-20T21:09:06.153 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-20T21:09:06.153 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-20T21:09:06.153 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-20T21:09:06.154 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-20T21:09:06.154 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-20T21:09:06.154 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-20T21:09:06.154 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-20T21:09:06.154 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-20T21:09:06.154 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-20T21:09:06.154 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-20T21:09:06.154 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-20T21:09:06.154 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-20T21:09:06.154 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-20T21:09:06.154 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-20T21:09:06.154 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-20T21:09:06.154 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-20T21:09:06.155 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-20T21:09:06.155 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-20T21:09:06.155 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-20T21:09:06.155 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-20T21:09:06.155 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-20T21:09:06.155 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-20T21:09:06.155 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-20T21:09:06.155 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-20T21:09:06.155 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-20T21:09:06.155 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-20T21:09:06.155 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-20T21:09:06.155 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-20T21:09:06.155 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-20T21:09:06.155 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-20T21:09:06.155 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-20T21:09:06.156 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-20T21:09:06.156 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-20T21:09:06.156 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-20T21:09:06.156 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-20T21:09:06.156 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-20T21:09:06.156 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-20T21:09:06.156 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-20T21:09:06.156 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-20T21:09:06.156 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-20T21:09:06.156 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-20T21:09:06.156 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-20T21:09:06.156 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-20T21:09:06.156 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-20T21:09:06.156 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-20T21:09:06.157 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-20T21:09:06.157 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-20T21:09:06.157 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-20T21:09:06.157 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-20T21:09:06.157 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-20T21:09:06.157 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-20T21:09:06.157 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-20T21:09:06.157 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-20T21:09:06.157 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-20T21:09:06.157 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-20T21:09:06.157 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-20T21:09:06.157 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-20T21:09:06.157 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-20T21:09:06.157 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-20T21:09:06.158 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-20T21:09:06.158 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-20T21:09:06.158 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-20T21:09:06.158 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-20T21:09:06.158 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-20T21:09:06.158 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-20T21:09:06.158 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-20T21:09:06.158 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-20T21:09:06.158 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-20T21:09:06.158 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-20T21:09:06.158 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-20T21:09:06.158 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-20T21:09:06.158 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-20T21:09:06.158 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-20T21:09:06.159 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-20T21:09:06.159 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-20T21:09:06.159 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-20T21:09:06.159 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-20T21:09:06.159 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-20T21:09:06.159 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-20T21:09:06.159 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-20T21:09:06.159 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-20T21:09:06.159 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-20T21:09:06.159 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-20T21:09:06.159 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-20T21:09:06.159 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-20T21:09:06.159 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-20T21:09:06.159 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-20T21:09:06.159 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-20T21:09:06.160 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-20T21:09:06.160 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-20T21:09:06.160 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-20T21:09:06.160 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-20T21:09:06.160 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-20T21:09:06.160 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-20T21:09:06.160 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-20T21:09:06.160 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-20T21:09:06.160 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-20T21:09:06.160 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-20T21:09:06.160 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-20T21:09:06.160 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-20T21:09:06.160 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-20T21:09:06.160 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-20T21:09:06.161 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-20T21:09:06.161 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-20T21:09:06.161 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-20T21:09:06.161 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-20T21:09:06.161 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-20T21:09:06.161 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-20T21:09:06.161 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-20T21:09:06.161 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-20T21:09:06.161 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-20T21:09:06.161 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-20T21:09:06.161 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-20T21:09:06.161 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-20T21:09:06.161 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-20T21:09:06.161 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-20T21:09:06.161 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-20T21:09:06.161 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-20T21:09:06.162 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-20T21:09:06.162 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-20T21:09:06.162 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-20T21:09:06.162 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-20T21:09:06.162 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-20T21:09:06.162 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-20T21:09:06.162 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-20T21:09:06.162 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-20T21:09:06.162 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-20T21:09:06.162 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-20T21:09:06.162 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-20T21:09:06.162 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-20T21:09:06.162 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-20T21:09:06.162 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-20T21:09:06.163 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-20T21:09:06.163 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-20T21:09:06.163 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-20T21:09:06.163 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-20T21:09:06.163 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-20T21:09:06.163 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-20T21:09:06.163 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-20T21:09:06.163 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-20T21:09:06.163 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-20T21:09:06.163 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-20T21:09:06.163 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-20T21:09:06.163 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-20T21:09:06.163 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-20T21:09:06.163 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-20T21:09:06.164 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-20T21:09:06.164 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-20T21:09:06.164 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-20T21:09:06.164 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-20T21:09:06.164 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-20T21:09:06.164 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-20T21:09:06.164 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-20T21:09:06.164 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-20T21:09:06.164 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-20T21:09:06.164 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-20T21:09:06.164 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-20T21:09:06.164 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-20T21:09:06.164 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-20T21:09:06.164 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-20T21:09:06.164 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-20T21:09:06.165 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-20T21:09:06.165 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-20T21:09:06.165 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-20T21:09:06.165 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-20T21:09:06.165 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-20T21:09:06.165 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-20T21:09:06.165 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-20T21:09:06.165 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-20T21:09:06.165 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-20T21:09:06.165 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-20T21:09:06.165 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-20T21:09:06.165 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-20T21:09:06.165 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-20T21:09:06.165 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-20T21:09:06.165 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-20T21:09:06.165 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-20T21:09:06.166 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-20T21:09:06.166 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-20T21:09:06.166 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-20T21:09:06.166 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-20T21:09:06.166 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-20T21:09:06.166 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-20T21:09:06.166 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-20T21:09:06.166 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-20T21:09:06.166 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-20T21:09:06.166 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-20T21:09:06.166 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-20T21:09:06.166 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-20T21:09:06.166 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-20T21:09:06.166 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-20T21:09:06.167 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-20T21:09:06.167 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-20T21:09:06.167 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-20T21:09:06.167 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-20T21:09:06.167 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-20T21:09:06.167 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-20T21:09:06.167 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-20T21:09:06.167 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-20T21:09:06.167 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-20T21:09:06.167 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-20T21:09:06.167 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-20T21:09:06.167 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-20T21:09:06.167 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-20T21:09:06.167 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-20T21:09:06.167 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-20T21:09:06.168 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-20T21:09:06.168 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-20T21:09:06.168 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-20T21:09:06.168 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-20T21:09:06.168 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-20T21:09:06.168 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-20T21:09:06.168 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-20T21:09:06.168 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-20T21:09:06.168 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-20T21:09:06.168 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-20T21:09:06.168 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-20T21:09:06.168 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-20T21:09:06.168 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-20T21:09:06.168 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-20T21:09:06.168 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-20T21:09:06.169 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-20T21:09:06.169 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-20T21:09:06.169 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-20T21:09:06.169 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-20T21:09:06.169 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-20T21:09:06.169 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-20T21:09:06.169 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-20T21:09:06.169 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-20T21:09:06.169 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-20T21:09:06.169 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-20T21:09:06.169 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-20T21:09:06.169 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-20T21:09:06.169 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-20T21:09:06.169 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-20T21:09:06.169 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-20T21:09:06.170 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-20T21:09:06.170 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-20T21:09:06.170 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-20T21:09:06.170 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-20T21:09:06.170 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-20T21:09:06.170 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-20T21:09:06.170 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-20T21:09:06.170 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-20T21:09:06.170 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-20T21:09:06.170 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-20T21:09:06.170 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-20T21:09:06.170 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-20T21:09:06.170 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-20T21:09:06.171 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-20T21:09:06.171 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-20T21:09:06.171 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-20T21:09:06.171 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-20T21:09:06.171 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-20T21:09:06.171 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-20T21:09:06.171 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-20T21:09:06.171 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-20T21:09:06.171 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-20T21:09:06.171 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-20T21:09:06.171 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-20T21:09:06.171 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-20T21:09:06.171 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-20T21:09:06.171 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-20T21:09:06.171 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-20T21:09:06.171 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-20T21:09:06.171 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-20T21:09:06.172 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-20T21:09:06.172 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-20T21:09:06.172 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-20T21:09:06.172 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-20T21:09:06.172 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-20T21:09:06.172 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-20T21:09:06.172 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-20T21:09:06.172 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-20T21:09:06.172 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-20T21:09:06.172 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-20T21:09:06.172 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-20T21:09:06.172 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-20T21:09:06.172 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-20T21:09:06.172 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-20T21:09:06.172 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-20T21:09:06.173 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-20T21:09:06.173 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-20T21:09:06.173 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-20T21:09:06.173 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-20T21:09:06.173 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-20T21:09:06.173 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-20T21:09:06.173 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-20T21:09:06.173 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-20T21:09:06.173 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-20T21:09:06.173 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-20T21:09:06.173 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-20T21:09:06.173 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-20T21:09:06.173 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-20T21:09:06.173 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-20T21:09:06.174 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-20T21:09:06.174 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-20T21:09:06.174 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-20T21:09:06.174 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-20T21:09:06.174 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-20T21:09:06.174 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-20T21:09:06.174 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-20T21:09:06.174 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-20T21:09:06.174 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-20T21:09:06.174 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-20T21:09:06.174 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-20T21:09:06.174 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-20T21:09:06.174 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-20T21:09:06.174 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-20T21:09:06.175 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-20T21:09:06.175 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-20T21:09:06.175 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-20T21:09:06.175 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-20T21:09:06.175 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-20T21:09:06.175 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-20T21:09:06.175 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-20T21:09:06.175 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-20T21:09:06.175 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-20T21:09:06.175 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-20T21:09:06.175 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-20T21:09:06.175 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-20T21:09:06.175 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-20T21:09:06.175 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-20T21:09:06.175 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-20T21:09:06.176 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-20T21:09:06.176 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-20T21:09:06.176 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-20T21:09:06.176 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-20T21:09:06.176 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-20T21:09:06.176 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-20T21:09:06.176 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-20T21:09:06.176 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-20T21:09:06.176 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-20T21:09:06.176 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-20T21:09:06.176 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-20T21:09:06.176 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-20T21:09:06.176 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-20T21:09:06.176 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-20T21:09:06.176 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-20T21:09:06.176 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-20T21:09:06.177 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-20T21:09:06.177 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-20T21:09:06.177 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-20T21:09:06.177 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-20T21:09:06.177 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-20T21:09:06.177 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-20T21:09:06.177 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-20T21:09:06.177 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-20T21:09:06.177 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-20T21:09:06.177 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-20T21:09:06.177 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-20T21:09:06.177 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-20T21:09:06.177 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-20T21:09:06.177 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-20T21:09:06.177 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-20T21:09:06.177 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-20T21:09:06.178 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-20T21:09:06.178 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-20T21:09:06.178 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-20T21:09:06.178 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-20T21:09:06.178 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-20T21:09:06.178 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-20T21:09:06.178 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-20T21:09:06.178 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-20T21:09:06.178 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-20T21:09:06.178 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-20T21:09:06.178 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-20T21:09:06.178 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-20T21:09:06.178 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-20T21:09:06.178 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-20T21:09:06.179 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-20T21:09:06.179 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-20T21:09:06.179 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-20T21:09:06.179 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-20T21:09:06.179 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-20T21:09:06.179 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-20T21:09:06.179 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-20T21:09:06.179 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-20T21:09:06.179 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-20T21:09:06.179 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-20T21:09:06.179 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-20T21:09:06.179 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-20T21:09:06.179 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-20T21:09:06.179 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-20T21:09:06.180 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-20T21:09:06.180 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-20T21:09:06.180 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-20T21:09:06.180 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-20T21:09:06.180 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-20T21:09:06.180 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-20T21:09:06.180 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-20T21:09:06.180 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-20T21:09:06.180 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-20T21:09:06.180 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-20T21:09:06.180 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-20T21:09:06.180 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-20T21:09:06.180 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-20T21:09:06.180 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-20T21:09:06.180 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-20T21:09:06.181 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-20T21:09:06.181 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-20T21:09:06.181 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-20T21:09:06.181 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-20T21:09:06.181 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-20T21:09:06.181 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-20T21:09:06.181 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-20T21:09:06.181 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-20T21:09:06.181 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-20T21:09:06.181 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-20T21:09:06.181 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-20T21:09:06.181 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-20T21:09:06.181 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-20T21:09:06.181 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-20T21:09:06.182 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-20T21:09:06.182 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-20T21:09:06.182 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-20T21:09:06.182 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-20T21:09:06.182 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-20T21:09:06.182 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-20T21:09:06.182 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-20T21:09:06.182 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-20T21:09:06.182 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-20T21:09:06.182 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-20T21:09:06.182 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-20T21:09:06.182 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-20T21:09:06.182 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-20T21:09:06.182 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-20T21:09:06.183 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-20T21:09:06.183 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-20T21:09:06.183 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-20T21:09:06.183 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-20T21:09:06.183 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-20T21:09:06.183 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-20T21:09:06.183 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-20T21:09:06.183 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-20T21:09:06.183 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-20T21:09:06.183 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-20T21:09:06.183 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-20T21:09:06.183 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-20T21:09:06.183 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-20T21:09:06.183 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-20T21:09:06.184 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-20T21:09:06.184 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-20T21:09:06.184 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-20T21:09:06.184 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-20T21:09:06.184 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-20T21:09:06.184 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-20T21:09:06.184 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-20T21:09:06.184 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-20T21:09:06.184 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-20T21:09:06.184 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-20T21:09:06.184 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-20T21:09:06.184 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-20T21:09:06.184 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-20T21:09:06.184 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-20T21:09:06.184 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-20T21:09:06.185 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-20T21:09:06.185 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-20T21:09:06.185 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-20T21:09:06.185 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-20T21:09:06.185 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-20T21:09:06.185 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-20T21:09:06.185 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-20T21:09:06.185 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-20T21:09:06.185 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-20T21:09:06.185 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-20T21:09:06.185 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-20T21:09:06.185 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-20T21:09:06.185 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-20T21:09:06.186 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-20T21:09:06.186 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-20T21:09:06.186 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-20T21:09:06.186 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-20T21:09:06.186 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-20T21:09:06.186 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-20T21:09:06.186 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-20T21:09:06.186 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-20T21:09:06.186 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-20T21:09:06.186 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-20T21:09:06.186 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-20T21:09:06.186 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-20T21:09:06.186 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-20T21:09:06.187 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-20T21:09:06.187 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-20T21:09:06.187 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-20T21:09:06.187 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-20T21:09:06.187 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-20T21:09:06.187 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-20T21:09:06.187 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-20T21:09:06.187 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-20T21:09:06.187 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-20T21:09:06.187 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-20T21:09:06.187 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-20T21:09:06.187 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-20T21:09:06.187 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-20T21:09:06.187 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-20T21:09:06.188 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-20T21:09:06.188 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-20T21:09:06.188 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-20T21:09:06.188 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-20T21:09:06.188 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-20T21:09:06.188 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-20T21:09:06.188 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-20T21:09:06.188 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-20T21:09:06.188 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-20T21:09:06.188 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-20T21:09:06.188 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-20T21:09:06.188 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-20T21:09:06.189 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-20T21:09:06.189 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-20T21:09:06.189 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-20T21:09:06.189 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-20T21:09:06.189 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-20T21:09:06.189 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-20T21:09:06.189 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-20T21:09:06.189 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-20T21:09:06.189 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-20T21:09:06.189 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-20T21:09:06.189 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-20T21:09:06.189 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-20T21:09:06.189 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-20T21:09:06.189 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-20T21:09:06.190 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-20T21:09:06.190 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-20T21:09:06.190 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-20T21:09:06.190 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-20T21:09:06.190 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-20T21:09:06.190 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-20T21:09:06.190 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-20T21:09:06.190 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-20T21:09:06.190 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-20T21:09:06.190 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-20T21:09:06.190 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-20T21:09:06.190 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-20T21:09:06.191 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-20T21:09:06.191 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-20T21:09:06.191 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-20T21:09:06.191 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-20T21:09:06.191 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-20T21:09:06.191 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-20T21:09:06.191 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-20T21:09:06.191 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-20T21:09:06.191 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-20T21:09:06.191 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-20T21:09:06.191 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-20T21:09:06.191 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-20T21:09:06.191 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-20T21:09:06.191 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-20T21:09:06.191 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-20T21:09:06.192 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-20T21:09:06.192 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-20T21:09:06.192 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-20T21:09:06.192 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-20T21:09:06.192 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-20T21:09:06.192 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-20T21:09:06.192 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-20T21:09:06.192 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-20T21:09:06.192 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-20T21:09:06.192 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-20T21:09:06.192 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-20T21:09:06.192 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-20T21:09:06.192 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-20T21:09:06.192 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-20T21:09:06.192 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-20T21:09:06.193 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-20T21:09:06.193 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-20T21:09:06.193 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-20T21:09:06.193 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-20T21:09:06.193 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-20T21:09:06.193 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-20T21:09:06.193 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-20T21:09:06.193 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-20T21:09:06.193 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-20T21:09:06.193 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-20T21:09:06.193 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-20T21:09:06.193 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-20T21:09:06.193 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-20T21:09:06.193 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-20T21:09:06.193 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-20T21:09:06.194 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-20T21:09:06.194 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-20T21:09:06.194 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-20T21:09:06.194 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-20T21:09:06.194 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-20T21:09:06.194 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-20T21:09:06.194 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-20T21:09:06.194 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-20T21:09:06.194 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-20T21:09:06.194 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-20T21:09:06.194 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-20T21:09:06.194 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-20T21:09:06.194 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-20T21:09:06.194 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-20T21:09:06.195 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-20T21:09:06.195 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-20T21:09:06.195 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-20T21:09:06.195 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-20T21:09:06.195 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-20T21:09:06.195 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-20T21:09:06.195 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-20T21:09:06.195 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-20T21:09:06.195 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-20T21:09:06.195 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-20T21:09:06.195 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-20T21:09:06.195 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-20T21:09:06.195 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-20T21:09:06.195 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-20T21:09:06.196 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-20T21:09:06.196 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-20T21:09:06.196 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-20T21:09:06.196 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-20T21:09:06.196 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-20T21:09:06.196 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-20T21:09:06.196 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-20T21:09:06.196 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-20T21:09:06.196 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-20T21:09:06.196 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-20T21:09:06.196 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-20T21:09:06.196 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-20T21:09:06.196 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-20T21:09:06.197 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-20T21:09:06.197 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-20T21:09:06.197 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-20T21:09:06.197 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-20T21:09:06.197 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-20T21:09:06.197 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-20T21:09:06.197 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-20T21:09:06.197 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-20T21:09:06.197 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-20T21:09:06.197 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-20T21:09:06.197 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-20T21:09:06.197 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-20T21:09:06.197 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-20T21:09:06.197 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-20T21:09:06.198 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-20T21:09:06.198 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-20T21:09:06.198 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-20T21:09:06.198 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-20T21:09:06.198 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-20T21:09:06.198 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-20T21:09:06.198 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-20T21:09:06.198 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-20T21:09:06.198 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-20T21:09:06.198 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-20T21:09:06.198 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-20T21:09:06.198 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-20T21:09:06.198 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-20T21:09:06.199 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-20T21:09:06.199 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-20T21:09:06.199 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-20T21:09:06.199 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-20T21:09:06.199 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-20T21:09:06.199 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-20T21:09:06.199 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-20T21:09:06.199 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-20T21:09:06.199 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-20T21:09:06.199 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-20T21:09:06.199 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-20T21:09:06.199 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-20T21:09:06.199 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-20T21:09:06.199 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-20T21:09:06.199 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-20T21:09:06.200 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-20T21:09:06.200 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-20T21:09:06.200 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-20T21:09:06.200 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-20T21:09:06.200 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-20T21:09:06.200 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-20T21:09:06.200 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-20T21:09:06.200 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-20T21:09:06.200 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-20T21:09:06.200 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-20T21:09:06.200 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-20T21:09:06.200 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-20T21:09:06.200 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-20T21:09:06.200 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-20T21:09:06.201 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-20T21:09:06.201 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-20T21:09:06.201 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-20T21:09:06.201 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-20T21:09:06.201 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-20T21:09:06.201 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-20T21:09:06.201 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-20T21:09:06.201 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-20T21:09:06.201 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-20T21:09:06.201 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-20T21:09:06.201 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-20T21:09:06.201 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-20T21:09:06.201 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-20T21:09:06.201 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-20T21:09:06.201 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-20T21:09:06.202 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-20T21:09:06.202 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-20T21:09:06.202 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-20T21:09:06.202 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-20T21:09:06.202 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-20T21:09:06.202 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-20T21:09:06.202 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-20T21:09:06.202 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-20T21:09:06.202 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-20T21:09:06.202 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-20T21:09:06.202 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-20T21:09:06.202 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-20T21:09:06.202 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-20T21:09:06.203 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-20T21:09:06.203 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-20T21:09:06.203 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-20T21:09:06.203 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-20T21:09:06.203 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-20T21:09:06.203 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-20T21:09:06.203 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-20T21:09:06.203 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-20T21:09:06.203 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-20T21:09:06.203 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-20T21:09:06.203 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-20T21:09:06.203 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-20T21:09:06.203 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-20T21:09:06.203 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-20T21:09:06.204 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-20T21:09:06.204 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-20T21:09:06.204 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-20T21:09:06.204 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-20T21:09:06.204 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-20T21:09:06.204 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-20T21:09:06.204 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-20T21:09:06.204 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-20T21:09:06.204 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-20T21:09:06.204 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-20T21:09:06.204 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-20T21:09:06.204 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-20T21:09:06.204 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-20T21:09:06.204 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-20T21:09:06.205 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-20T21:09:06.205 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-20T21:09:06.205 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-20T21:09:06.205 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-20T21:09:06.205 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-20T21:09:06.205 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-20T21:09:06.205 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-20T21:09:06.205 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-20T21:09:06.205 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-20T21:09:06.205 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-20T21:09:06.205 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-20T21:09:06.205 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-20T21:09:06.205 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-20T21:09:06.205 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-20T21:09:06.206 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-20T21:09:06.206 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-20T21:09:06.206 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-20T21:09:06.206 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-20T21:09:06.206 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-20T21:09:06.206 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-20T21:09:06.206 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-20T21:09:06.206 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-20T21:09:06.206 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-20T21:09:06.206 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-20T21:09:06.206 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-20T21:09:06.207 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-20T21:09:06.207 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-20T21:09:06.207 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-20T21:09:06.207 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-20T21:09:06.207 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-20T21:09:06.207 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-20T21:09:06.207 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-20T21:09:06.207 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-20T21:09:06.207 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-20T21:09:06.207 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-20T21:09:06.207 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-20T21:09:06.207 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-20T21:09:06.207 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-20T21:09:06.207 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-20T21:09:06.208 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-20T21:09:06.208 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-20T21:09:06.208 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-20T21:09:06.208 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-20T21:09:06.208 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-20T21:09:06.208 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-20T21:09:06.208 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-20T21:09:06.208 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-20T21:09:06.208 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-20T21:09:06.208 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-20T21:09:06.208 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-20T21:09:06.208 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-20T21:09:06.209 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-20T21:09:06.209 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-20T21:09:06.209 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-20T21:09:06.209 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-20T21:09:06.209 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-20T21:09:06.209 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-20T21:09:06.209 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-20T21:09:06.209 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-20T21:09:06.209 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-20T21:09:06.209 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-20T21:09:06.209 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-20T21:09:06.209 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-20T21:09:06.209 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-20T21:09:06.209 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-20T21:09:06.210 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-20T21:09:06.210 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-20T21:09:06.210 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-20T21:09:06.210 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-20T21:09:06.210 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-20T21:09:06.210 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-20T21:09:06.210 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-20T21:09:06.210 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-20T21:09:06.210 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-20T21:09:06.210 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-20T21:09:06.210 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-20T21:09:06.210 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-20T21:09:06.210 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-20T21:09:06.210 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-20T21:09:06.211 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-20T21:09:06.211 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-20T21:09:06.211 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-20T21:09:06.211 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-20T21:09:06.211 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-20T21:09:06.211 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-20T21:09:06.211 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-20T21:09:06.211 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-20T21:09:06.211 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-20T21:09:06.211 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-20T21:09:06.211 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-20T21:09:06.211 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-20T21:09:06.211 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-20T21:09:06.211 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-20T21:09:06.212 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-20T21:09:06.212 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-20T21:09:06.212 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-20T21:09:06.212 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-20T21:09:06.212 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-20T21:09:06.212 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-20T21:09:06.212 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-20T21:09:06.212 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-20T21:09:06.212 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-20T21:09:06.212 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-20T21:09:06.212 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-20T21:09:06.212 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-20T21:09:06.212 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-20T21:09:06.212 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-20T21:09:06.213 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-20T21:09:06.213 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-20T21:09:06.213 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-20T21:09:06.213 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-20T21:09:06.213 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-20T21:09:06.213 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-20T21:09:06.213 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-20T21:09:06.213 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-20T21:09:06.213 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-20T21:09:06.213 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-20T21:09:06.213 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-20T21:09:06.213 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-20T21:09:06.213 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-20T21:09:06.213 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-20T21:09:06.213 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-20T21:09:06.214 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-20T21:09:06.214 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-20T21:09:06.214 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-20T21:09:06.214 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-20T21:09:06.214 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-20T21:09:06.214 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-20T21:09:06.214 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-20T21:09:06.214 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-20T21:09:06.214 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-20T21:09:06.214 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-20T21:09:06.214 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-20T21:09:06.214 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-20T21:09:06.214 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-20T21:09:06.214 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-20T21:09:06.215 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-20T21:09:06.215 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-20T21:09:06.215 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-20T21:09:06.215 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-20T21:09:06.215 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-20T21:09:06.215 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-20T21:09:06.215 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-20T21:09:06.215 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-20T21:09:06.215 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-20T21:09:06.215 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-20T21:09:06.215 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-20T21:09:06.215 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-20T21:09:06.215 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-20T21:09:06.215 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-20T21:09:06.215 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-20T21:09:06.216 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-20T21:09:06.216 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-20T21:09:06.216 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-20T21:09:06.216 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-20T21:09:06.216 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-20T21:09:06.216 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-20T21:09:06.216 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-20T21:09:06.216 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-20T21:09:06.216 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-20T21:09:06.216 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-20T21:09:06.216 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-20T21:09:06.216 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-20T21:09:06.216 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-20T21:09:06.216 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-20T21:09:06.217 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-20T21:09:06.217 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-20T21:09:06.217 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-20T21:09:06.217 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-20T21:09:06.217 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-20T21:09:06.217 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-20T21:09:06.217 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-20T21:09:06.217 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-20T21:09:06.217 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-20T21:09:06.217 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-20T21:09:06.217 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-20T21:09:06.217 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-20T21:09:06.217 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-20T21:09:06.217 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-20T21:09:06.217 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-20T21:09:06.217 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-20T21:09:06.218 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-20T21:09:06.218 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-20T21:09:06.218 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-20T21:09:06.218 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-20T21:09:06.218 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-20T21:09:06.218 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-20T21:09:06.218 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-20T21:09:06.218 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-20T21:09:06.218 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-20T21:09:06.218 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-20T21:09:06.218 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-20T21:09:06.218 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-20T21:09:06.218 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-20T21:09:06.218 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-20T21:09:06.219 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-20T21:09:06.219 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-20T21:09:06.219 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-20T21:09:06.219 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-20T21:09:06.219 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-20T21:09:06.219 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-20T21:09:06.219 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-20T21:09:06.219 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-20T21:09:06.219 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-20T21:09:06.219 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-20T21:09:06.219 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-20T21:09:06.219 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-20T21:09:06.219 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-20T21:09:06.219 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-20T21:09:06.220 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-20T21:09:06.220 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-20T21:09:06.220 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-20T21:09:06.220 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-20T21:09:06.220 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-20T21:09:06.220 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-20T21:09:06.220 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-20T21:09:06.220 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-20T21:09:06.220 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-20T21:09:06.220 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-20T21:09:06.220 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-20T21:09:06.220 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-20T21:09:06.220 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-20T21:09:06.220 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-20T21:09:06.220 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-20T21:09:06.221 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-20T21:09:06.221 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-20T21:09:06.221 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-20T21:09:06.221 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-20T21:09:06.221 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-20T21:09:06.221 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-20T21:09:06.221 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-20T21:09:06.221 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-20T21:09:06.221 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-20T21:09:06.221 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-20T21:09:06.221 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-20T21:09:06.221 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-20T21:09:06.221 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-20T21:09:06.221 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-20T21:09:06.222 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-20T21:09:06.222 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-20T21:09:06.222 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-20T21:09:06.222 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-20T21:09:06.222 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-20T21:09:06.222 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-20T21:09:06.222 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-20T21:09:06.222 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-20T21:09:06.222 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-20T21:09:06.222 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-20T21:09:06.222 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-20T21:09:06.222 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-20T21:09:06.222 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-20T21:09:06.222 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-20T21:09:06.223 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-20T21:09:06.223 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-20T21:09:06.223 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-20T21:09:06.223 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-20T21:09:06.223 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-20T21:09:06.223 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-20T21:09:06.223 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-20T21:09:06.223 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-20T21:09:06.223 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-20T21:09:06.223 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-20T21:09:06.223 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-20T21:09:06.223 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-20T21:09:06.223 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-20T21:09:06.223 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-20T21:09:06.223 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-20T21:09:06.223 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-20T21:09:06.224 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-20T21:09:06.224 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-20T21:09:06.224 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-20T21:09:06.224 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-20T21:09:06.224 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-20T21:09:06.224 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-20T21:09:06.224 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-20T21:09:06.224 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-20T21:09:06.224 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-20T21:09:06.224 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-20T21:09:06.224 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-20T21:09:06.224 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-20T21:09:06.224 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-20T21:09:06.224 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-20T21:09:06.224 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-20T21:09:06.224 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-20T21:09:06.225 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-20T21:09:06.225 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-20T21:09:06.225 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-20T21:09:06.225 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-20T21:09:06.225 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-20T21:09:06.225 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-20T21:09:06.225 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-20T21:09:06.225 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-20T21:09:06.225 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-20T21:09:06.225 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-20T21:09:06.225 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-20T21:09:06.225 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-20T21:09:06.225 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-20T21:09:06.225 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-20T21:09:06.226 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-20T21:09:06.226 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-20T21:09:06.226 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-20T21:09:06.226 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-20T21:09:06.226 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-20T21:09:06.226 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-20T21:09:06.226 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-20T21:09:06.226 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-20T21:09:06.226 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-20T21:09:06.226 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-20T21:09:06.226 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-20T21:09:06.226 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-20T21:09:06.226 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-20T21:09:06.226 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-20T21:09:06.226 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-20T21:09:06.227 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-20T21:09:06.227 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-20T21:09:06.227 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-20T21:09:06.227 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-20T21:09:06.227 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-20T21:09:06.227 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-20T21:09:06.227 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-20T21:09:06.227 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-20T21:09:06.227 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-20T21:09:06.227 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-20T21:09:06.227 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-20T21:09:06.227 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-20T21:09:06.227 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-20T21:09:06.227 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-20T21:09:06.227 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-20T21:09:06.228 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-20T21:09:06.228 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-20T21:09:06.228 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-20T21:09:06.228 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-20T21:09:06.228 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-20T21:09:06.228 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-20T21:09:06.228 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-20T21:09:06.228 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-20T21:09:06.228 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-20T21:09:06.228 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-20T21:09:06.228 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-20T21:09:06.228 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-20T21:09:06.228 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-20T21:09:06.228 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-20T21:09:06.228 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-20T21:09:06.229 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-20T21:09:06.229 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-20T21:09:06.229 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-20T21:09:06.229 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-20T21:09:06.229 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-20T21:09:06.229 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-20T21:09:06.229 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-20T21:09:06.229 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-20T21:09:06.229 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-20T21:09:06.229 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-20T21:09:06.229 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-20T21:09:06.229 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-20T21:09:06.229 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-20T21:09:06.229 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-20T21:09:06.230 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-20T21:09:06.230 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-20T21:09:06.230 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-20T21:09:06.230 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-20T21:09:06.230 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-20T21:09:06.230 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-20T21:09:06.230 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-20T21:09:06.230 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-20T21:09:06.230 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-20T21:09:06.230 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-20T21:09:06.230 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-20T21:09:06.230 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-20T21:09:06.230 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-20T21:09:06.230 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-20T21:09:06.230 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-20T21:09:06.231 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-20T21:09:06.231 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-20T21:09:06.231 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-20T21:09:06.231 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-20T21:09:06.231 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-20T21:09:06.231 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-20T21:09:06.231 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-20T21:09:06.231 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-20T21:09:06.231 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-20T21:09:06.231 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-20T21:09:06.231 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-20T21:09:06.231 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-20T21:09:06.231 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-20T21:09:06.231 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-20T21:09:06.231 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-20T21:09:06.232 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-20T21:09:06.232 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-20T21:09:06.232 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-20T21:09:06.232 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-20T21:09:06.232 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-20T21:09:06.232 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-20T21:09:06.232 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-20T21:09:06.232 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-20T21:09:06.232 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-20T21:09:06.232 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-20T21:09:06.232 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-20T21:09:06.232 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-20T21:09:06.232 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-20T21:09:06.232 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-20T21:09:06.232 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-20T21:09:06.233 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-20T21:09:06.233 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-20T21:09:06.233 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-20T21:09:06.233 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-20T21:09:06.233 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-20T21:09:06.233 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-20T21:09:06.233 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-20T21:09:06.233 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-20T21:09:06.233 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-20T21:09:06.233 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-20T21:09:06.233 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-20T21:09:06.233 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-20T21:09:06.233 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-20T21:09:06.233 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-20T21:09:06.234 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-20T21:09:06.234 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-20T21:09:06.234 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-20T21:09:06.234 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-20T21:09:06.234 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-20T21:09:06.234 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-20T21:09:06.234 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-20T21:09:06.234 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-20T21:09:06.234 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-20T21:09:06.234 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-20T21:09:06.234 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-20T21:09:06.234 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-20T21:09:06.234 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-20T21:09:06.234 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-20T21:09:06.234 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-20T21:09:06.235 INFO:teuthology.orchestra.run.smithi144.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-20T21:09:06.238 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-20T21:09:06.238 DEBUG:teuthology.orchestra.run.smithi144:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-20T21:09:06.260 INFO:teuthology.orchestra.run.smithi144.stdout:postinstall scriptlet (using /bin/sh): 2024-04-20T21:09:06.261 INFO:teuthology.orchestra.run.smithi144.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-20T21:09:06.261 INFO:teuthology.orchestra.run.smithi144.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-20T21:09:06.261 INFO:teuthology.orchestra.run.smithi144.stdout:fi 2024-04-20T21:09:06.261 INFO:teuthology.orchestra.run.smithi144.stdout:for file in vmlinuz System.map config; do 2024-04-20T21:09:06.261 INFO:teuthology.orchestra.run.smithi144.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-20T21:09:06.261 INFO:teuthology.orchestra.run.smithi144.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-20T21:09:06.261 INFO:teuthology.orchestra.run.smithi144.stdout: fi 2024-04-20T21:09:06.261 INFO:teuthology.orchestra.run.smithi144.stdout:done 2024-04-20T21:09:06.261 INFO:teuthology.orchestra.run.smithi144.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-20T21:09:06.261 INFO:teuthology.orchestra.run.smithi144.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-20T21:09:06.261 INFO:teuthology.orchestra.run.smithi144.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-20T21:09:06.261 INFO:teuthology.orchestra.run.smithi144.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-20T21:09:06.261 INFO:teuthology.orchestra.run.smithi144.stdout:fi 2024-04-20T21:09:06.261 INFO:teuthology.orchestra.run.smithi144.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-20T21:09:06.262 INFO:teuthology.orchestra.run.smithi144.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-20T21:09:06.262 INFO:teuthology.orchestra.run.smithi144.stdout:fi 2024-04-20T21:09:06.262 INFO:teuthology.orchestra.run.smithi144.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-20T21:09:06.262 INFO:teuthology.orchestra.run.smithi144.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-20T21:09:06.262 INFO:teuthology.orchestra.run.smithi144.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-20T21:09:06.262 INFO:teuthology.orchestra.run.smithi144.stdout:fi 2024-04-20T21:09:06.263 DEBUG:teuthology.orchestra.run.smithi144:> sudo rpm -qi grub2-tools 2024-04-20T21:09:06.335 INFO:teuthology.orchestra.run.smithi144.stdout:Name : grub2-tools 2024-04-20T21:09:06.335 INFO:teuthology.orchestra.run.smithi144.stdout:Epoch : 1 2024-04-20T21:09:06.335 INFO:teuthology.orchestra.run.smithi144.stdout:Version : 2.06 2024-04-20T21:09:06.336 INFO:teuthology.orchestra.run.smithi144.stdout:Release : 68.el9 2024-04-20T21:09:06.336 INFO:teuthology.orchestra.run.smithi144.stdout:Architecture: x86_64 2024-04-20T21:09:06.336 INFO:teuthology.orchestra.run.smithi144.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-20T21:09:06.336 INFO:teuthology.orchestra.run.smithi144.stdout:Group : Unspecified 2024-04-20T21:09:06.336 INFO:teuthology.orchestra.run.smithi144.stdout:Size : 8264254 2024-04-20T21:09:06.336 INFO:teuthology.orchestra.run.smithi144.stdout:License : GPLv3+ 2024-04-20T21:09:06.336 INFO:teuthology.orchestra.run.smithi144.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-20T21:09:06.336 INFO:teuthology.orchestra.run.smithi144.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-20T21:09:06.336 INFO:teuthology.orchestra.run.smithi144.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-20T21:09:06.336 INFO:teuthology.orchestra.run.smithi144.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-20T21:09:06.336 INFO:teuthology.orchestra.run.smithi144.stdout:Packager : builder@centos.org 2024-04-20T21:09:06.336 INFO:teuthology.orchestra.run.smithi144.stdout:Vendor : CentOS 2024-04-20T21:09:06.336 INFO:teuthology.orchestra.run.smithi144.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-20T21:09:06.336 INFO:teuthology.orchestra.run.smithi144.stdout:Summary : Support tools for GRUB. 2024-04-20T21:09:06.336 INFO:teuthology.orchestra.run.smithi144.stdout:Description : 2024-04-20T21:09:06.337 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:09:06.337 INFO:teuthology.orchestra.run.smithi144.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-20T21:09:06.337 INFO:teuthology.orchestra.run.smithi144.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-20T21:09:06.337 INFO:teuthology.orchestra.run.smithi144.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-20T21:09:06.337 INFO:teuthology.orchestra.run.smithi144.stdout:hardware devices. 2024-04-20T21:09:06.337 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:09:06.337 INFO:teuthology.orchestra.run.smithi144.stdout:This subpackage provides tools for support of all platforms. 2024-04-20T21:09:06.338 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-20T21:09:06.338 INFO:teuthology.task.kernel:Updating grub on smithi144 to boot 6.9.0-rc3-g806ec36b2112 2024-04-20T21:09:06.338 DEBUG:teuthology.orchestra.run.smithi144:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-20T21:09:06.893 INFO:teuthology.orchestra.run.smithi144.stderr:Generating grub configuration file ... 2024-04-20T21:09:08.230 INFO:teuthology.orchestra.run.smithi144.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-20T21:09:08.258 INFO:teuthology.orchestra.run.smithi144.stderr:done 2024-04-20T21:09:08.260 DEBUG:teuthology.orchestra.run.smithi144:> mktemp 2024-04-20T21:09:08.275 INFO:teuthology.orchestra.run.smithi144.stdout:/tmp/tmp.Vrc4Ebpo4F 2024-04-20T21:09:08.275 DEBUG:teuthology.orchestra.run.smithi144:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Vrc4Ebpo4F 2024-04-20T21:09:08.342 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 0666 /tmp/tmp.Vrc4Ebpo4F 2024-04-20T21:09:08.476 DEBUG:teuthology.orchestra.remote:smithi144:/tmp/tmp.Vrc4Ebpo4F is 6KB 2024-04-20T21:09:08.525 DEBUG:teuthology.orchestra.run.smithi144:> rm -fr /tmp/tmp.Vrc4Ebpo4F 2024-04-20T21:09:08.539 DEBUG:teuthology.orchestra.run.smithi144:> sudo /bin/ls /boot/loader/entries || true 2024-04-20T21:09:08.601 INFO:teuthology.orchestra.run.smithi144.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-20T21:09:08.601 INFO:teuthology.orchestra.run.smithi144.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-20T21:09:08.601 INFO:teuthology.orchestra.run.smithi144.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-20T21:09:08.601 INFO:teuthology.orchestra.run.smithi144.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-20T21:09:08.602 DEBUG:teuthology.orchestra.run.smithi144:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-20T21:09:08.794 DEBUG:teuthology.orchestra.run.smithi144:> sudo shutdown -r now 2024-04-20T21:09:08.850 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-20T21:09:08.851 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T21:09:16.233 DEBUG:teuthology.orchestra.run.smithi086:> true 2024-04-20T21:09:16.944 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-20T21:09:16.944 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.9.0-rc3-g806ec36b2112"... 2024-04-20T21:09:16.944 DEBUG:teuthology.orchestra.run.smithi086:> uname -r 2024-04-20T21:09:16.985 INFO:teuthology.orchestra.run.smithi086.stdout:6.9.0-rc3-g806ec36b2112 2024-04-20T21:09:16.986 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-20T21:09:16.986 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-20T21:09:16.986 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-20T21:09:17.987 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-20T21:09:17.987 DEBUG:teuthology.orchestra.run.smithi086:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-20T21:09:18.106 INFO:teuthology.orchestra.run.smithi086.stdout:ttyS1 2024-04-20T21:09:18.179 DEBUG:teuthology.parallel:result is None 2024-04-20T21:09:38.798 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-04-20T21:09:38.798 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-04-20T21:09:38.799 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T21:10:38.802 DEBUG:teuthology.orchestra.remote:timed out 2024-04-20T21:10:47.803 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-04-20T21:10:47.804 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T21:10:55.353 DEBUG:teuthology.orchestra.run.smithi144:> true 2024-04-20T21:10:55.973 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-04-20T21:10:55.973 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.9.0-rc3-g806ec36b2112"... 2024-04-20T21:10:55.973 DEBUG:teuthology.orchestra.run.smithi144:> uname -r 2024-04-20T21:10:56.014 INFO:teuthology.orchestra.run.smithi144.stdout:6.9.0-rc3-g806ec36b2112 2024-04-20T21:10:56.015 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-20T21:10:56.015 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-20T21:10:56.015 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-20T21:10:57.016 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-20T21:10:57.016 DEBUG:teuthology.orchestra.run.smithi144:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-20T21:10:57.135 INFO:teuthology.orchestra.run.smithi144.stdout:ttyS1 2024-04-20T21:10:57.194 DEBUG:teuthology.parallel:result is None 2024-04-20T21:10:57.194 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-20T21:10:57.201 INFO:teuthology.task.internal:Creating test directory... 2024-04-20T21:10:57.201 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-20T21:10:57.210 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-20T21:10:57.260 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-20T21:10:57.267 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-20T21:10:57.272 INFO:teuthology.task.internal:Creating archive directory... 2024-04-20T21:10:57.273 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-20T21:10:57.278 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-20T21:10:57.348 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-20T21:10:57.355 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-20T21:10:57.356 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-20T21:10:57.363 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-20T21:10:57.456 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T21:10:57.488 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T21:10:57.511 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T21:10:57.539 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T21:10:57.542 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-20T21:10:57.549 INFO:teuthology.task.internal:Configuring sudo... 2024-04-20T21:10:57.550 DEBUG:teuthology.orchestra.run.smithi086:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-20T21:10:57.557 DEBUG:teuthology.orchestra.run.smithi144:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-20T21:10:57.641 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-20T21:10:57.649 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-20T21:10:57.650 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-20T21:10:57.657 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-20T21:10:57.705 DEBUG:teuthology.orchestra.run.smithi086:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T21:10:57.783 DEBUG:teuthology.orchestra.run.smithi086:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T21:10:57.908 DEBUG:teuthology.orchestra.run.smithi086:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T21:10:57.993 DEBUG:teuthology.orchestra.run.smithi086:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T21:10:58.080 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:10:58.080 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-20T21:10:58.163 DEBUG:teuthology.orchestra.run.smithi144:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T21:10:58.237 DEBUG:teuthology.orchestra.run.smithi144:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T21:10:58.333 DEBUG:teuthology.orchestra.run.smithi144:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T21:10:58.401 DEBUG:teuthology.orchestra.run.smithi144:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T21:10:58.481 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:10:58.481 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-20T21:10:58.544 DEBUG:teuthology.orchestra.run.smithi086:> sudo service rsyslog restart 2024-04-20T21:10:58.551 DEBUG:teuthology.orchestra.run.smithi144:> sudo service rsyslog restart 2024-04-20T21:10:58.660 INFO:teuthology.orchestra.run.smithi086.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T21:10:58.700 INFO:teuthology.orchestra.run.smithi144.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T21:10:59.018 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-20T21:10:59.025 INFO:teuthology.task.internal:Starting timer... 2024-04-20T21:10:59.025 INFO:teuthology.run_tasks:Running task pcp... 2024-04-20T21:10:59.035 INFO:teuthology.run_tasks:Running task selinux... 2024-04-20T21:10:59.043 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-20T21:10:59.044 DEBUG:teuthology.orchestra.run.smithi086:> sudo service auditd rotate 2024-04-20T21:10:59.171 INFO:teuthology.orchestra.run.smithi086.stdout:Rotating logs: 2024-04-20T21:10:59.176 DEBUG:teuthology.orchestra.run.smithi144:> sudo service auditd rotate 2024-04-20T21:10:59.341 INFO:teuthology.orchestra.run.smithi144.stdout:Rotating logs: 2024-04-20T21:10:59.346 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-20T21:10:59.346 DEBUG:teuthology.orchestra.run.smithi086:> /usr/sbin/getenforce 2024-04-20T21:10:59.441 INFO:teuthology.orchestra.run.smithi086.stdout:Permissive 2024-04-20T21:10:59.441 DEBUG:teuthology.orchestra.run.smithi144:> /usr/sbin/getenforce 2024-04-20T21:10:59.495 INFO:teuthology.orchestra.run.smithi144.stdout:Permissive 2024-04-20T21:10:59.496 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi086.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi144.front.sepia.ceph.com': 'permissive'} 2024-04-20T21:10:59.496 DEBUG:teuthology.orchestra.run.smithi086:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-20T21:10:59.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:10:59.565 DEBUG:teuthology.orchestra.run.smithi144:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-20T21:10:59.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:10:59.647 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-20T21:10:59.647 DEBUG:teuthology.orchestra.run.smithi086:> sudo /usr/sbin/setenforce permissive 2024-04-20T21:10:59.717 DEBUG:teuthology.orchestra.run.smithi144:> sudo /usr/sbin/setenforce permissive 2024-04-20T21:10:59.786 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-20T21:10:59.798 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-20T21:10:59.817 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-20T21:10:59.835 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi086.front.sepia.ceph.com,smithi144.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-20T21:13:20.925 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi086.front.sepia.ceph.com'), Remote(name='ubuntu@smithi144.front.sepia.ceph.com')] 2024-04-20T21:13:20.926 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-20T21:13:20.927 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T21:13:21.048 DEBUG:teuthology.orchestra.run.smithi086:> true 2024-04-20T21:13:21.205 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-20T21:13:21.206 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-04-20T21:13:21.206 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T21:13:21.316 DEBUG:teuthology.orchestra.run.smithi144:> true 2024-04-20T21:13:21.471 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-04-20T21:13:21.471 INFO:teuthology.run_tasks:Running task clock... 2024-04-20T21:13:21.480 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-20T21:13:21.481 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-20T21:13:21.481 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T21:13:21.489 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-20T21:13:21.489 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T21:13:21.575 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-20T21:13:21.576 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-20T21:13:21.615 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-20T21:13:21.619 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-20T21:13:21.686 INFO:teuthology.orchestra.run.smithi086.stderr:sudo: ntpd: command not found 2024-04-20T21:13:21.690 INFO:teuthology.orchestra.run.smithi144.stderr:sudo: ntpd: command not found 2024-04-20T21:13:21.718 INFO:teuthology.orchestra.run.smithi086.stdout:506 Cannot talk to daemon 2024-04-20T21:13:21.722 INFO:teuthology.orchestra.run.smithi144.stdout:506 Cannot talk to daemon 2024-04-20T21:13:21.757 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-20T21:13:21.761 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-20T21:13:21.798 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-20T21:13:21.802 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-20T21:13:21.938 INFO:teuthology.orchestra.run.smithi086.stderr:bash: line 1: ntpq: command not found 2024-04-20T21:13:21.942 INFO:teuthology.orchestra.run.smithi144.stderr:bash: line 1: ntpq: command not found 2024-04-20T21:13:21.946 INFO:teuthology.orchestra.run.smithi086.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T21:13:21.946 INFO:teuthology.orchestra.run.smithi086.stdout:=============================================================================== 2024-04-20T21:13:21.946 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T21:13:21.946 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T21:13:21.946 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T21:13:21.946 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T21:13:21.949 INFO:teuthology.orchestra.run.smithi144.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T21:13:21.950 INFO:teuthology.orchestra.run.smithi144.stdout:=============================================================================== 2024-04-20T21:13:21.950 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T21:13:21.950 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T21:13:21.950 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T21:13:21.950 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T21:13:21.951 INFO:teuthology.run_tasks:Running task install... 2024-04-20T21:13:21.961 DEBUG:teuthology.task.install:project ceph 2024-04-20T21:13:21.961 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'}} 2024-04-20T21:13:21.961 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'} 2024-04-20T21:13:21.961 INFO:teuthology.task.install:Using flavor: default 2024-04-20T21:13:21.966 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-20T21:13:21.967 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-20T21:13:21.967 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'tag': None, 'wait_for_package': False} 2024-04-20T21:13:21.968 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T21:13:21.969 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'tag': None, 'wait_for_package': False} 2024-04-20T21:13:21.969 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T21:13:22.203 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/ 2024-04-20T21:13:22.203 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3133.ga8c84f59 2024-04-20T21:13:22.207 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/ 2024-04-20T21:13:22.207 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3133.ga8c84f59 2024-04-20T21:13:22.533 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-20T21:13:22.533 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:13:22.533 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-20T21:13:22.544 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-20T21:13:22.544 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:13:22.544 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-20T21:13:22.625 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-20T21:13:22.626 DEBUG:teuthology.orchestra.run.smithi144:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a8c84f5997950fd7d008204839253291d52e6eda/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-20T21:13:22.642 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-20T21:13:22.643 DEBUG:teuthology.orchestra.run.smithi086:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a8c84f5997950fd7d008204839253291d52e6eda/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-20T21:13:22.755 DEBUG:teuthology.orchestra.run.smithi144:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-20T21:13:22.785 DEBUG:teuthology.orchestra.run.smithi086:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-20T21:13:22.877 DEBUG:teuthology.orchestra.run.smithi144:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-20T21:13:22.905 DEBUG:teuthology.orchestra.run.smithi086:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-20T21:13:22.967 INFO:teuthology.orchestra.run.smithi144.stdout:check_obsoletes = 1 2024-04-20T21:13:22.973 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum clean all 2024-04-20T21:13:22.989 INFO:teuthology.orchestra.run.smithi086.stdout:check_obsoletes = 1 2024-04-20T21:13:22.994 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum clean all 2024-04-20T21:13:23.315 INFO:teuthology.orchestra.run.smithi144.stdout:48 files removed 2024-04-20T21:13:23.343 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-20T21:13:23.343 INFO:teuthology.orchestra.run.smithi086.stdout:48 files removed 2024-04-20T21:13:23.382 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-20T21:13:25.101 INFO:teuthology.orchestra.run.smithi086.stdout:ceph packages for x86_64 65 kB/s | 84 kB 00:01 2024-04-20T21:13:25.180 INFO:teuthology.orchestra.run.smithi144.stdout:ceph packages for x86_64 60 kB/s | 84 kB 00:01 2024-04-20T21:13:25.985 INFO:teuthology.orchestra.run.smithi086.stdout:ceph noarch packages 15 kB/s | 12 kB 00:00 2024-04-20T21:13:26.230 INFO:teuthology.orchestra.run.smithi144.stdout:ceph noarch packages 12 kB/s | 12 kB 00:00 2024-04-20T21:13:27.014 INFO:teuthology.orchestra.run.smithi086.stdout:ceph source packages 2.0 kB/s | 1.9 kB 00:00 2024-04-20T21:13:27.240 INFO:teuthology.orchestra.run.smithi144.stdout:ceph source packages 2.1 kB/s | 1.9 kB 00:00 2024-04-20T21:13:28.537 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - BaseOS 6.6 MB/s | 8.0 MB 00:01 2024-04-20T21:13:29.083 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - BaseOS 4.0 MB/s | 8.0 MB 00:01 2024-04-20T21:13:30.724 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-04-20T21:13:35.190 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-04-20T21:13:36.864 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - Extras packages 44 kB/s | 16 kB 00:00 2024-04-20T21:13:38.762 INFO:teuthology.orchestra.run.smithi144.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-04-20T21:13:44.694 INFO:teuthology.orchestra.run.smithi144.stdout:lab-extras 18 kB/s | 1.7 kB 00:00 2024-04-20T21:13:47.154 INFO:teuthology.orchestra.run.smithi144.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-20T21:13:47.156 INFO:teuthology.orchestra.run.smithi144.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-20T21:13:47.179 INFO:teuthology.orchestra.run.smithi144.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-20T21:13:47.179 INFO:teuthology.orchestra.run.smithi144.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-20T21:13:47.179 INFO:teuthology.orchestra.run.smithi144.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-20T21:13:47.180 INFO:teuthology.orchestra.run.smithi144.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-20T21:13:47.183 INFO:teuthology.orchestra.run.smithi144.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-20T21:13:47.186 INFO:teuthology.orchestra.run.smithi144.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-20T21:13:47.187 INFO:teuthology.orchestra.run.smithi144.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-20T21:13:47.259 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-04-20T21:13:47.282 INFO:teuthology.orchestra.run.smithi144.stdout:======================================================================================= 2024-04-20T21:13:47.282 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repository Size 2024-04-20T21:13:47.282 INFO:teuthology.orchestra.run.smithi144.stdout:======================================================================================= 2024-04-20T21:13:47.282 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-04-20T21:13:47.283 INFO:teuthology.orchestra.run.smithi144.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-20T21:13:47.283 INFO:teuthology.orchestra.run.smithi144.stdout: ceph x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 6.6 k 2024-04-20T21:13:47.283 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-base x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-20T21:13:47.283 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 893 k 2024-04-20T21:13:47.283 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 145 k 2024-04-20T21:13:47.283 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 1.5 M 2024-04-20T21:13:47.283 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 142 k 2024-04-20T21:13:47.283 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 3.5 M 2024-04-20T21:13:47.283 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 7.4 M 2024-04-20T21:13:47.283 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-rook noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 49 k 2024-04-20T21:13:47.283 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-radosgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 11 M 2024-04-20T21:13:47.283 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-test x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 49 M 2024-04-20T21:13:47.283 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-volume noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 273 k 2024-04-20T21:13:47.284 INFO:teuthology.orchestra.run.smithi144.stdout: cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 763 k 2024-04-20T21:13:47.284 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 225 k 2024-04-20T21:13:47.284 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-shell x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 27 k 2024-04-20T21:13:47.284 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-top noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 18 k 2024-04-20T21:13:47.284 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-20T21:13:47.284 INFO:teuthology.orchestra.run.smithi144.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-20T21:13:47.284 INFO:teuthology.orchestra.run.smithi144.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-20T21:13:47.284 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 32 k 2024-04-20T21:13:47.284 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 748 k 2024-04-20T21:13:47.284 INFO:teuthology.orchestra.run.smithi144.stdout: librados-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 126 k 2024-04-20T21:13:47.284 INFO:teuthology.orchestra.run.smithi144.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-20T21:13:47.284 INFO:teuthology.orchestra.run.smithi144.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-20T21:13:47.284 INFO:teuthology.orchestra.run.smithi144.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-20T21:13:47.285 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-20T21:13:47.285 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-20T21:13:47.285 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-20T21:13:47.285 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cephfs x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 162 k 2024-04-20T21:13:47.285 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rados x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 321 k 2024-04-20T21:13:47.285 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 299 k 2024-04-20T21:13:47.285 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 99 k 2024-04-20T21:13:47.285 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 85 k 2024-04-20T21:13:47.285 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-20T21:13:47.285 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-nbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 170 k 2024-04-20T21:13:47.285 INFO:teuthology.orchestra.run.smithi144.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-20T21:13:47.285 INFO:teuthology.orchestra.run.smithi144.stdout:Upgrading: 2024-04-20T21:13:47.286 INFO:teuthology.orchestra.run.smithi144.stdout: librados2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.4 M 2024-04-20T21:13:47.286 INFO:teuthology.orchestra.run.smithi144.stdout: librbd1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-20T21:13:47.286 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2024-04-20T21:13:47.286 INFO:teuthology.orchestra.run.smithi144.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-20T21:13:47.286 INFO:teuthology.orchestra.run.smithi144.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-20T21:13:47.286 INFO:teuthology.orchestra.run.smithi144.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-20T21:13:47.286 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 21 M 2024-04-20T21:13:47.286 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 34 k 2024-04-20T21:13:47.286 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mds x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 2.4 M 2024-04-20T21:13:47.286 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 249 k 2024-04-20T21:13:47.286 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mon x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 4.7 M 2024-04-20T21:13:47.287 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-osd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 17 M 2024-04-20T21:13:47.287 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 16 k 2024-04-20T21:13:47.287 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-selinux x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 25 k 2024-04-20T21:13:47.287 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-20T21:13:47.287 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-20T21:13:47.287 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-20T21:13:47.287 INFO:teuthology.orchestra.run.smithi144.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-20T21:13:47.287 INFO:teuthology.orchestra.run.smithi144.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-20T21:13:47.287 INFO:teuthology.orchestra.run.smithi144.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-20T21:13:47.287 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-20T21:13:47.287 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-20T21:13:47.287 INFO:teuthology.orchestra.run.smithi144.stdout: libcephsqlite x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 163 k 2024-04-20T21:13:47.288 INFO:teuthology.orchestra.run.smithi144.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-20T21:13:47.288 INFO:teuthology.orchestra.run.smithi144.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-20T21:13:47.288 INFO:teuthology.orchestra.run.smithi144.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-20T21:13:47.288 INFO:teuthology.orchestra.run.smithi144.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-20T21:13:47.288 INFO:teuthology.orchestra.run.smithi144.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-20T21:13:47.288 INFO:teuthology.orchestra.run.smithi144.stdout: libradosstriper1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 501 k 2024-04-20T21:13:47.289 INFO:teuthology.orchestra.run.smithi144.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-20T21:13:47.289 INFO:teuthology.orchestra.run.smithi144.stdout: librgw2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-20T21:13:47.289 INFO:teuthology.orchestra.run.smithi144.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-20T21:13:47.289 INFO:teuthology.orchestra.run.smithi144.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-20T21:13:47.289 INFO:teuthology.orchestra.run.smithi144.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-20T21:13:47.289 INFO:teuthology.orchestra.run.smithi144.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-20T21:13:47.289 INFO:teuthology.orchestra.run.smithi144.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-20T21:13:47.289 INFO:teuthology.orchestra.run.smithi144.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-20T21:13:47.289 INFO:teuthology.orchestra.run.smithi144.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-20T21:13:47.289 INFO:teuthology.orchestra.run.smithi144.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-20T21:13:47.290 INFO:teuthology.orchestra.run.smithi144.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-20T21:13:47.290 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-20T21:13:47.290 INFO:teuthology.orchestra.run.smithi144.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-20T21:13:47.290 INFO:teuthology.orchestra.run.smithi144.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-20T21:13:47.291 INFO:teuthology.orchestra.run.smithi144.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-20T21:13:47.291 INFO:teuthology.orchestra.run.smithi144.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-20T21:13:47.291 INFO:teuthology.orchestra.run.smithi144.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-20T21:13:47.291 INFO:teuthology.orchestra.run.smithi144.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-20T21:13:47.291 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-20T21:13:47.292 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-argparse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 45 k 2024-04-20T21:13:47.292 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 135 k 2024-04-20T21:13:47.292 INFO:teuthology.orchestra.run.smithi144.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-20T21:13:47.292 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-20T21:13:47.292 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-20T21:13:47.292 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-20T21:13:47.292 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-20T21:13:47.292 INFO:teuthology.orchestra.run.smithi144.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-20T21:13:47.292 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-20T21:13:47.292 INFO:teuthology.orchestra.run.smithi144.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-20T21:13:47.292 INFO:teuthology.orchestra.run.smithi144.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-20T21:13:47.292 INFO:teuthology.orchestra.run.smithi144.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-20T21:13:47.293 INFO:teuthology.orchestra.run.smithi144.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-20T21:13:47.293 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-20T21:13:47.293 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-20T21:13:47.293 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-20T21:13:47.293 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-20T21:13:47.293 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-20T21:13:47.293 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-20T21:13:47.293 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-20T21:13:47.293 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-20T21:13:47.293 INFO:teuthology.orchestra.run.smithi144.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-20T21:13:47.293 INFO:teuthology.orchestra.run.smithi144.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-20T21:13:47.293 INFO:teuthology.orchestra.run.smithi144.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-20T21:13:47.293 INFO:teuthology.orchestra.run.smithi144.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-20T21:13:47.294 INFO:teuthology.orchestra.run.smithi144.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-20T21:13:47.294 INFO:teuthology.orchestra.run.smithi144.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-20T21:13:47.294 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-20T21:13:47.294 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-20T21:13:47.294 INFO:teuthology.orchestra.run.smithi144.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-20T21:13:47.294 INFO:teuthology.orchestra.run.smithi144.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-20T21:13:47.294 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-20T21:13:47.294 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-20T21:13:47.294 INFO:teuthology.orchestra.run.smithi144.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-20T21:13:47.294 INFO:teuthology.orchestra.run.smithi144.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-20T21:13:47.294 INFO:teuthology.orchestra.run.smithi144.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-20T21:13:47.294 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-20T21:13:47.294 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-20T21:13:47.294 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-20T21:13:47.295 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-20T21:13:47.295 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-20T21:13:47.295 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-20T21:13:47.295 INFO:teuthology.orchestra.run.smithi144.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-20T21:13:47.295 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-20T21:13:47.295 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-20T21:13:47.295 INFO:teuthology.orchestra.run.smithi144.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-20T21:13:47.295 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-20T21:13:47.295 INFO:teuthology.orchestra.run.smithi144.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-20T21:13:47.295 INFO:teuthology.orchestra.run.smithi144.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-20T21:13:47.295 INFO:teuthology.orchestra.run.smithi144.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-20T21:13:47.295 INFO:teuthology.orchestra.run.smithi144.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-20T21:13:47.295 INFO:teuthology.orchestra.run.smithi144.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-20T21:13:47.296 INFO:teuthology.orchestra.run.smithi144.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-20T21:13:47.296 INFO:teuthology.orchestra.run.smithi144.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-20T21:13:47.296 INFO:teuthology.orchestra.run.smithi144.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-20T21:13:47.296 INFO:teuthology.orchestra.run.smithi144.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-20T21:13:47.296 INFO:teuthology.orchestra.run.smithi144.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-20T21:13:47.296 INFO:teuthology.orchestra.run.smithi144.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-20T21:13:47.296 INFO:teuthology.orchestra.run.smithi144.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-20T21:13:47.296 INFO:teuthology.orchestra.run.smithi144.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-20T21:13:47.296 INFO:teuthology.orchestra.run.smithi144.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-20T21:13:47.296 INFO:teuthology.orchestra.run.smithi144.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-20T21:13:47.296 INFO:teuthology.orchestra.run.smithi144.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-20T21:13:47.296 INFO:teuthology.orchestra.run.smithi144.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-20T21:13:47.296 INFO:teuthology.orchestra.run.smithi144.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-20T21:13:47.296 INFO:teuthology.orchestra.run.smithi144.stdout:Installing weak dependencies: 2024-04-20T21:13:47.297 INFO:teuthology.orchestra.run.smithi144.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-20T21:13:47.297 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:13:47.297 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-04-20T21:13:47.297 INFO:teuthology.orchestra.run.smithi144.stdout:======================================================================================= 2024-04-20T21:13:47.297 INFO:teuthology.orchestra.run.smithi144.stdout:Install 148 Packages 2024-04-20T21:13:47.297 INFO:teuthology.orchestra.run.smithi144.stdout:Upgrade 2 Packages 2024-04-20T21:13:47.297 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:13:47.298 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 218 M 2024-04-20T21:13:47.298 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-04-20T21:13:48.427 INFO:teuthology.orchestra.run.smithi144.stdout:(1/150): ceph-19.0.0-3133.ga8c84f59.el9.x86_64. 15 kB/s | 6.6 kB 00:00 2024-04-20T21:13:49.787 INFO:teuthology.orchestra.run.smithi144.stdout:(2/150): ceph-fuse-19.0.0-3133.ga8c84f59.el9.x8 657 kB/s | 893 kB 00:01 2024-04-20T21:13:50.054 INFO:teuthology.orchestra.run.smithi144.stdout:(3/150): ceph-immutable-object-cache-19.0.0-313 544 kB/s | 145 kB 00:00 2024-04-20T21:13:52.133 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - AppStream 892 kB/s | 19 MB 00:21 2024-04-20T21:13:53.121 INFO:teuthology.orchestra.run.smithi144.stdout:(4/150): ceph-base-19.0.0-3133.ga8c84f59.el9.x8 1.0 MB/s | 5.4 MB 00:05 2024-04-20T21:13:53.997 INFO:teuthology.orchestra.run.smithi144.stdout:(5/150): ceph-mds-19.0.0-3133.ga8c84f59.el9.x86 613 kB/s | 2.4 MB 00:03 2024-04-20T21:13:54.613 INFO:teuthology.orchestra.run.smithi144.stdout:(6/150): ceph-mgr-19.0.0-3133.ga8c84f59.el9.x86 1.0 MB/s | 1.5 MB 00:01 2024-04-20T21:13:56.914 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - CRB 6.9 MB/s | 6.2 MB 00:00 2024-04-20T21:13:58.528 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - Extras packages 59 kB/s | 16 kB 00:00 2024-04-20T21:14:00.104 INFO:teuthology.orchestra.run.smithi144.stdout:(7/150): ceph-mon-19.0.0-3133.ga8c84f59.el9.x86 788 kB/s | 4.7 MB 00:06 2024-04-20T21:14:00.763 INFO:teuthology.orchestra.run.smithi086.stdout:Extra Packages for Enterprise Linux 10 MB/s | 21 MB 00:02 2024-04-20T21:14:06.647 INFO:teuthology.orchestra.run.smithi086.stdout:lab-extras 21 kB/s | 1.7 kB 00:00 2024-04-20T21:14:09.089 INFO:teuthology.orchestra.run.smithi086.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-20T21:14:09.092 INFO:teuthology.orchestra.run.smithi086.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-20T21:14:09.111 INFO:teuthology.orchestra.run.smithi086.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-20T21:14:09.112 INFO:teuthology.orchestra.run.smithi086.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-20T21:14:09.114 INFO:teuthology.orchestra.run.smithi086.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-20T21:14:09.115 INFO:teuthology.orchestra.run.smithi086.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-20T21:14:09.118 INFO:teuthology.orchestra.run.smithi086.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-20T21:14:09.121 INFO:teuthology.orchestra.run.smithi086.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-20T21:14:09.122 INFO:teuthology.orchestra.run.smithi086.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-20T21:14:09.191 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-04-20T21:14:09.209 INFO:teuthology.orchestra.run.smithi086.stdout:======================================================================================= 2024-04-20T21:14:09.209 INFO:teuthology.orchestra.run.smithi086.stdout: Package Arch Version Repository Size 2024-04-20T21:14:09.209 INFO:teuthology.orchestra.run.smithi086.stdout:======================================================================================= 2024-04-20T21:14:09.209 INFO:teuthology.orchestra.run.smithi086.stdout:Installing: 2024-04-20T21:14:09.209 INFO:teuthology.orchestra.run.smithi086.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-20T21:14:09.209 INFO:teuthology.orchestra.run.smithi086.stdout: ceph x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 6.6 k 2024-04-20T21:14:09.209 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-base x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-20T21:14:09.209 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 893 k 2024-04-20T21:14:09.209 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 145 k 2024-04-20T21:14:09.209 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 1.5 M 2024-04-20T21:14:09.210 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 142 k 2024-04-20T21:14:09.210 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 3.5 M 2024-04-20T21:14:09.210 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 7.4 M 2024-04-20T21:14:09.210 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-rook noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 49 k 2024-04-20T21:14:09.210 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-radosgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 11 M 2024-04-20T21:14:09.210 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-test x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 49 M 2024-04-20T21:14:09.210 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-volume noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 273 k 2024-04-20T21:14:09.210 INFO:teuthology.orchestra.run.smithi086.stdout: cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 763 k 2024-04-20T21:14:09.211 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 225 k 2024-04-20T21:14:09.211 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-shell x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 27 k 2024-04-20T21:14:09.211 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-top noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 18 k 2024-04-20T21:14:09.211 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-20T21:14:09.211 INFO:teuthology.orchestra.run.smithi086.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-20T21:14:09.211 INFO:teuthology.orchestra.run.smithi086.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-20T21:14:09.211 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 32 k 2024-04-20T21:14:09.211 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 748 k 2024-04-20T21:14:09.211 INFO:teuthology.orchestra.run.smithi086.stdout: librados-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 126 k 2024-04-20T21:14:09.211 INFO:teuthology.orchestra.run.smithi086.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-20T21:14:09.211 INFO:teuthology.orchestra.run.smithi086.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-20T21:14:09.211 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-20T21:14:09.211 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-20T21:14:09.212 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-20T21:14:09.212 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-20T21:14:09.212 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cephfs x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 162 k 2024-04-20T21:14:09.212 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rados x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 321 k 2024-04-20T21:14:09.212 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 299 k 2024-04-20T21:14:09.212 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 99 k 2024-04-20T21:14:09.212 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 85 k 2024-04-20T21:14:09.212 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-20T21:14:09.212 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-nbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 170 k 2024-04-20T21:14:09.212 INFO:teuthology.orchestra.run.smithi086.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-20T21:14:09.212 INFO:teuthology.orchestra.run.smithi086.stdout:Upgrading: 2024-04-20T21:14:09.212 INFO:teuthology.orchestra.run.smithi086.stdout: librados2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.4 M 2024-04-20T21:14:09.213 INFO:teuthology.orchestra.run.smithi086.stdout: librbd1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-20T21:14:09.213 INFO:teuthology.orchestra.run.smithi086.stdout:Installing dependencies: 2024-04-20T21:14:09.213 INFO:teuthology.orchestra.run.smithi086.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-20T21:14:09.213 INFO:teuthology.orchestra.run.smithi086.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-20T21:14:09.213 INFO:teuthology.orchestra.run.smithi086.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-20T21:14:09.213 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 21 M 2024-04-20T21:14:09.213 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 34 k 2024-04-20T21:14:09.215 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mds x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 2.4 M 2024-04-20T21:14:09.215 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 249 k 2024-04-20T21:14:09.215 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mon x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 4.7 M 2024-04-20T21:14:09.215 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-osd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 17 M 2024-04-20T21:14:09.215 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 16 k 2024-04-20T21:14:09.215 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-selinux x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 25 k 2024-04-20T21:14:09.215 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-20T21:14:09.215 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-20T21:14:09.215 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-20T21:14:09.215 INFO:teuthology.orchestra.run.smithi086.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-20T21:14:09.215 INFO:teuthology.orchestra.run.smithi086.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-20T21:14:09.216 INFO:teuthology.orchestra.run.smithi086.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-20T21:14:09.216 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-20T21:14:09.216 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-20T21:14:09.216 INFO:teuthology.orchestra.run.smithi086.stdout: libcephsqlite x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 163 k 2024-04-20T21:14:09.216 INFO:teuthology.orchestra.run.smithi086.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-20T21:14:09.216 INFO:teuthology.orchestra.run.smithi086.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-20T21:14:09.216 INFO:teuthology.orchestra.run.smithi086.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-20T21:14:09.216 INFO:teuthology.orchestra.run.smithi086.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-20T21:14:09.216 INFO:teuthology.orchestra.run.smithi086.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-20T21:14:09.216 INFO:teuthology.orchestra.run.smithi086.stdout: libradosstriper1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 501 k 2024-04-20T21:14:09.217 INFO:teuthology.orchestra.run.smithi086.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-20T21:14:09.217 INFO:teuthology.orchestra.run.smithi086.stdout: librgw2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-20T21:14:09.217 INFO:teuthology.orchestra.run.smithi086.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-20T21:14:09.217 INFO:teuthology.orchestra.run.smithi086.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-20T21:14:09.217 INFO:teuthology.orchestra.run.smithi086.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-20T21:14:09.217 INFO:teuthology.orchestra.run.smithi086.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-20T21:14:09.217 INFO:teuthology.orchestra.run.smithi086.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-20T21:14:09.217 INFO:teuthology.orchestra.run.smithi086.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-20T21:14:09.217 INFO:teuthology.orchestra.run.smithi086.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-20T21:14:09.217 INFO:teuthology.orchestra.run.smithi086.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-20T21:14:09.218 INFO:teuthology.orchestra.run.smithi086.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-20T21:14:09.218 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-20T21:14:09.218 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-20T21:14:09.218 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-20T21:14:09.218 INFO:teuthology.orchestra.run.smithi086.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-20T21:14:09.218 INFO:teuthology.orchestra.run.smithi086.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-20T21:14:09.219 INFO:teuthology.orchestra.run.smithi086.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-20T21:14:09.219 INFO:teuthology.orchestra.run.smithi086.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-20T21:14:09.219 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-20T21:14:09.219 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-argparse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 45 k 2024-04-20T21:14:09.219 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 135 k 2024-04-20T21:14:09.219 INFO:teuthology.orchestra.run.smithi086.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-20T21:14:09.219 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-20T21:14:09.219 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-20T21:14:09.219 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-20T21:14:09.219 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-20T21:14:09.219 INFO:teuthology.orchestra.run.smithi086.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-20T21:14:09.219 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-20T21:14:09.219 INFO:teuthology.orchestra.run.smithi086.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-20T21:14:09.219 INFO:teuthology.orchestra.run.smithi086.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-20T21:14:09.220 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-20T21:14:09.220 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-20T21:14:09.220 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-20T21:14:09.220 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-20T21:14:09.220 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-20T21:14:09.220 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-20T21:14:09.220 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-20T21:14:09.220 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-20T21:14:09.220 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-20T21:14:09.220 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-20T21:14:09.220 INFO:teuthology.orchestra.run.smithi086.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-20T21:14:09.220 INFO:teuthology.orchestra.run.smithi086.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-20T21:14:09.220 INFO:teuthology.orchestra.run.smithi086.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-20T21:14:09.220 INFO:teuthology.orchestra.run.smithi086.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-20T21:14:09.221 INFO:teuthology.orchestra.run.smithi086.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-20T21:14:09.221 INFO:teuthology.orchestra.run.smithi086.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-20T21:14:09.221 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-20T21:14:09.221 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-20T21:14:09.221 INFO:teuthology.orchestra.run.smithi086.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-20T21:14:09.221 INFO:teuthology.orchestra.run.smithi086.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-20T21:14:09.221 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-20T21:14:09.222 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-20T21:14:09.222 INFO:teuthology.orchestra.run.smithi086.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-20T21:14:09.222 INFO:teuthology.orchestra.run.smithi086.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-20T21:14:09.222 INFO:teuthology.orchestra.run.smithi086.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-20T21:14:09.222 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-20T21:14:09.222 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-20T21:14:09.222 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-20T21:14:09.222 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-20T21:14:09.222 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-20T21:14:09.222 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-20T21:14:09.222 INFO:teuthology.orchestra.run.smithi086.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-20T21:14:09.222 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-20T21:14:09.222 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-20T21:14:09.222 INFO:teuthology.orchestra.run.smithi086.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-20T21:14:09.223 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-20T21:14:09.223 INFO:teuthology.orchestra.run.smithi086.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-20T21:14:09.223 INFO:teuthology.orchestra.run.smithi086.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-20T21:14:09.223 INFO:teuthology.orchestra.run.smithi086.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-20T21:14:09.223 INFO:teuthology.orchestra.run.smithi086.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-20T21:14:09.223 INFO:teuthology.orchestra.run.smithi086.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-20T21:14:09.223 INFO:teuthology.orchestra.run.smithi086.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-20T21:14:09.223 INFO:teuthology.orchestra.run.smithi086.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-20T21:14:09.223 INFO:teuthology.orchestra.run.smithi086.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-20T21:14:09.223 INFO:teuthology.orchestra.run.smithi086.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-20T21:14:09.223 INFO:teuthology.orchestra.run.smithi086.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-20T21:14:09.223 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-20T21:14:09.223 INFO:teuthology.orchestra.run.smithi086.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-20T21:14:09.224 INFO:teuthology.orchestra.run.smithi086.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-20T21:14:09.224 INFO:teuthology.orchestra.run.smithi086.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-20T21:14:09.224 INFO:teuthology.orchestra.run.smithi086.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-20T21:14:09.225 INFO:teuthology.orchestra.run.smithi086.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-20T21:14:09.225 INFO:teuthology.orchestra.run.smithi086.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-20T21:14:09.225 INFO:teuthology.orchestra.run.smithi086.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-20T21:14:09.225 INFO:teuthology.orchestra.run.smithi086.stdout:Installing weak dependencies: 2024-04-20T21:14:09.225 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-20T21:14:09.225 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:14:09.226 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-04-20T21:14:09.226 INFO:teuthology.orchestra.run.smithi086.stdout:======================================================================================= 2024-04-20T21:14:09.226 INFO:teuthology.orchestra.run.smithi086.stdout:Install 148 Packages 2024-04-20T21:14:09.226 INFO:teuthology.orchestra.run.smithi086.stdout:Upgrade 2 Packages 2024-04-20T21:14:09.226 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:14:09.226 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 218 M 2024-04-20T21:14:09.227 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-04-20T21:14:10.448 INFO:teuthology.orchestra.run.smithi086.stdout:(1/150): ceph-19.0.0-3133.ga8c84f59.el9.x86_64. 16 kB/s | 6.6 kB 00:00 2024-04-20T21:14:10.495 INFO:teuthology.orchestra.run.smithi144.stdout:(8/150): ceph-radosgw-19.0.0-3133.ga8c84f59.el9 1.0 MB/s | 11 MB 00:10 2024-04-20T21:14:10.628 INFO:teuthology.orchestra.run.smithi144.stdout:(9/150): ceph-selinux-19.0.0-3133.ga8c84f59.el9 190 kB/s | 25 kB 00:00 2024-04-20T21:14:10.920 INFO:teuthology.orchestra.run.smithi144.stdout:(10/150): ceph-common-19.0.0-3133.ga8c84f59.el9 943 kB/s | 21 MB 00:22 2024-04-20T21:14:11.262 INFO:teuthology.orchestra.run.smithi144.stdout:(11/150): cephfs-mirror-19.0.0-3133.ga8c84f59.e 660 kB/s | 225 kB 00:00 2024-04-20T21:14:11.370 INFO:teuthology.orchestra.run.smithi144.stdout:(12/150): cephfs-shell-19.0.0-3133.ga8c84f59.el 251 kB/s | 27 kB 00:00 2024-04-20T21:14:11.456 INFO:teuthology.orchestra.run.smithi086.stdout:(2/150): ceph-fuse-19.0.0-3133.ga8c84f59.el9.x8 887 kB/s | 893 kB 00:01 2024-04-20T21:14:11.471 INFO:teuthology.orchestra.run.smithi144.stdout:(13/150): libcephfs-devel-19.0.0-3133.ga8c84f59 320 kB/s | 32 kB 00:00 2024-04-20T21:14:11.798 INFO:teuthology.orchestra.run.smithi086.stdout:(3/150): ceph-immutable-object-cache-19.0.0-313 425 kB/s | 145 kB 00:00 2024-04-20T21:14:11.913 INFO:teuthology.orchestra.run.smithi144.stdout:(14/150): ceph-osd-19.0.0-3133.ga8c84f59.el9.x8 996 kB/s | 17 MB 00:17 2024-04-20T21:14:12.130 INFO:teuthology.orchestra.run.smithi144.stdout:(15/150): libcephfs2-19.0.0-3133.ga8c84f59.el9. 1.1 MB/s | 748 kB 00:00 2024-04-20T21:14:12.155 INFO:teuthology.orchestra.run.smithi144.stdout:(16/150): libcephsqlite-19.0.0-3133.ga8c84f59.e 678 kB/s | 163 kB 00:00 2024-04-20T21:14:12.297 INFO:teuthology.orchestra.run.smithi144.stdout:(17/150): librados-devel-19.0.0-3133.ga8c84f59. 762 kB/s | 126 kB 00:00 2024-04-20T21:14:12.755 INFO:teuthology.orchestra.run.smithi144.stdout:(18/150): libradosstriper1-19.0.0-3133.ga8c84f5 834 kB/s | 501 kB 00:00 2024-04-20T21:14:12.847 INFO:teuthology.orchestra.run.smithi144.stdout:(19/150): python3-ceph-argparse-19.0.0-3133.ga8 498 kB/s | 45 kB 00:00 2024-04-20T21:14:13.048 INFO:teuthology.orchestra.run.smithi144.stdout:(20/150): python3-ceph-common-19.0.0-3133.ga8c8 675 kB/s | 135 kB 00:00 2024-04-20T21:14:13.298 INFO:teuthology.orchestra.run.smithi144.stdout:(21/150): python3-cephfs-19.0.0-3133.ga8c84f59. 651 kB/s | 162 kB 00:00 2024-04-20T21:14:13.590 INFO:teuthology.orchestra.run.smithi086.stdout:(4/150): ceph-base-19.0.0-3133.ga8c84f59.el9.x8 1.5 MB/s | 5.4 MB 00:03 2024-04-20T21:14:13.623 INFO:teuthology.orchestra.run.smithi144.stdout:(22/150): python3-rados-19.0.0-3133.ga8c84f59.e 990 kB/s | 321 kB 00:00 2024-04-20T21:14:13.984 INFO:teuthology.orchestra.run.smithi144.stdout:(23/150): python3-rbd-19.0.0-3133.ga8c84f59.el9 830 kB/s | 299 kB 00:00 2024-04-20T21:14:14.023 INFO:teuthology.orchestra.run.smithi086.stdout:(5/150): ceph-mds-19.0.0-3133.ga8c84f59.el9.x86 1.1 MB/s | 2.4 MB 00:02 2024-04-20T21:14:14.199 INFO:teuthology.orchestra.run.smithi144.stdout:(24/150): python3-rgw-19.0.0-3133.ga8c84f59.el9 465 kB/s | 99 kB 00:00 2024-04-20T21:14:14.291 INFO:teuthology.orchestra.run.smithi144.stdout:(25/150): rbd-fuse-19.0.0-3133.ga8c84f59.el9.x8 940 kB/s | 85 kB 00:00 2024-04-20T21:14:14.474 INFO:teuthology.orchestra.run.smithi086.stdout:(6/150): ceph-mgr-19.0.0-3133.ga8c84f59.el9.x86 1.7 MB/s | 1.5 MB 00:00 2024-04-20T21:14:16.657 INFO:teuthology.orchestra.run.smithi144.stdout:(26/150): librgw2-19.0.0-3133.ga8c84f59.el9.x86 1.2 MB/s | 5.4 MB 00:04 2024-04-20T21:14:16.741 INFO:teuthology.orchestra.run.smithi144.stdout:(27/150): rbd-mirror-19.0.0-3133.ga8c84f59.el9. 1.3 MB/s | 3.1 MB 00:02 2024-04-20T21:14:16.833 INFO:teuthology.orchestra.run.smithi144.stdout:(28/150): ceph-grafana-dashboards-19.0.0-3133.g 377 kB/s | 34 kB 00:00 2024-04-20T21:14:16.900 INFO:teuthology.orchestra.run.smithi144.stdout:(29/150): rbd-nbd-19.0.0-3133.ga8c84f59.el9.x86 706 kB/s | 170 kB 00:00 2024-04-20T21:14:16.991 INFO:teuthology.orchestra.run.smithi144.stdout:(30/150): ceph-mgr-cephadm-19.0.0-3133.ga8c84f5 900 kB/s | 142 kB 00:00 2024-04-20T21:14:18.400 INFO:teuthology.orchestra.run.smithi086.stdout:(7/150): ceph-mon-19.0.0-3133.ga8c84f59.el9.x86 1.1 MB/s | 4.7 MB 00:04 2024-04-20T21:14:19.933 INFO:teuthology.orchestra.run.smithi144.stdout:(31/150): ceph-mgr-dashboard-19.0.0-3133.ga8c84 1.2 MB/s | 3.5 MB 00:03 2024-04-20T21:14:20.192 INFO:teuthology.orchestra.run.smithi144.stdout:(32/150): ceph-mgr-modules-core-19.0.0-3133.ga8 967 kB/s | 249 kB 00:00 2024-04-20T21:14:20.292 INFO:teuthology.orchestra.run.smithi144.stdout:(33/150): ceph-mgr-rook-19.0.0-3133.ga8c84f59.e 497 kB/s | 49 kB 00:00 2024-04-20T21:14:20.384 INFO:teuthology.orchestra.run.smithi144.stdout:(34/150): ceph-prometheus-alerts-19.0.0-3133.ga 181 kB/s | 16 kB 00:00 2024-04-20T21:14:20.634 INFO:teuthology.orchestra.run.smithi144.stdout:(35/150): ceph-volume-19.0.0-3133.ga8c84f59.el9 1.1 MB/s | 273 kB 00:00 2024-04-20T21:14:21.160 INFO:teuthology.orchestra.run.smithi144.stdout:(36/150): cephadm-19.0.0-3133.ga8c84f59.el9.noa 1.4 MB/s | 763 kB 00:00 2024-04-20T21:14:21.252 INFO:teuthology.orchestra.run.smithi144.stdout:(37/150): cephfs-top-19.0.0-3133.ga8c84f59.el9. 201 kB/s | 18 kB 00:00 2024-04-20T21:14:21.386 INFO:teuthology.orchestra.run.smithi144.stdout:(38/150): fuse-2.9.9-15.el9.x86_64.rpm 611 kB/s | 80 kB 00:00 2024-04-20T21:14:21.586 INFO:teuthology.orchestra.run.smithi144.stdout:(39/150): libgfortran-11.4.1-3.el9.x86_64.rpm 3.9 MB/s | 803 kB 00:00 2024-04-20T21:14:21.636 INFO:teuthology.orchestra.run.smithi144.stdout:(40/150): libquadmath-11.4.1-3.el9.x86_64.rpm 3.9 MB/s | 195 kB 00:00 2024-04-20T21:14:21.695 INFO:teuthology.orchestra.run.smithi144.stdout:(41/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.3 MB/s | 253 kB 00:00 2024-04-20T21:14:21.879 INFO:teuthology.orchestra.run.smithi144.stdout:(42/150): python3-cryptography-36.0.1-4.el9.x86 6.7 MB/s | 1.2 MB 00:00 2024-04-20T21:14:21.937 INFO:teuthology.orchestra.run.smithi144.stdout:(43/150): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-20T21:14:21.987 INFO:teuthology.orchestra.run.smithi144.stdout:(44/150): python3-pycparser-2.20-6.el9.noarch.r 2.7 MB/s | 135 kB 00:00 2024-04-20T21:14:22.038 INFO:teuthology.orchestra.run.smithi144.stdout:(45/150): python3-requests-2.25.1-8.el9.noarch. 2.5 MB/s | 125 kB 00:00 2024-04-20T21:14:22.088 INFO:teuthology.orchestra.run.smithi144.stdout:(46/150): python3-urllib3-1.26.5-5.el9.noarch.r 4.3 MB/s | 215 kB 00:00 2024-04-20T21:14:22.413 INFO:teuthology.orchestra.run.smithi144.stdout:(47/150): automake-1.16.2-8.el9.noarch.rpm 2.1 MB/s | 689 kB 00:00 2024-04-20T21:14:22.706 INFO:teuthology.orchestra.run.smithi144.stdout:(48/150): ceph-mgr-diskprediction-local-19.0.0- 1.3 MB/s | 7.4 MB 00:05 2024-04-20T21:14:22.747 INFO:teuthology.orchestra.run.smithi144.stdout:(49/150): bison-3.7.4-5.el9.x86_64.rpm 2.8 MB/s | 943 kB 00:00 2024-04-20T21:14:22.828 INFO:teuthology.orchestra.run.smithi144.stdout:(50/150): elfutils-libelf-devel-0.190-2.el9.x86 289 kB/s | 21 kB 00:00 2024-04-20T21:14:22.898 INFO:teuthology.orchestra.run.smithi144.stdout:(51/150): boost-program-options-1.75.0-8.el9.x8 560 kB/s | 107 kB 00:00 2024-04-20T21:14:22.932 INFO:teuthology.orchestra.run.smithi144.stdout:(52/150): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-04-20T21:14:22.974 INFO:teuthology.orchestra.run.smithi144.stdout:(53/150): flexiblas-3.0.4-8.el9.x86_64.rpm 437 kB/s | 33 kB 00:00 2024-04-20T21:14:23.049 INFO:teuthology.orchestra.run.smithi144.stdout:(54/150): flexiblas-openblas-openmp-3.0.4-8.el9 246 kB/s | 18 kB 00:00 2024-04-20T21:14:23.142 INFO:teuthology.orchestra.run.smithi144.stdout:(55/150): libaio-devel-0.3.111-13.el9.x86_64.rp 121 kB/s | 11 kB 00:00 2024-04-20T21:14:23.241 INFO:teuthology.orchestra.run.smithi144.stdout:(56/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 160 kB 00:00 2024-04-20T21:14:23.317 INFO:teuthology.orchestra.run.smithi144.stdout:(57/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 609 kB/s | 45 kB 00:00 2024-04-20T21:14:23.408 INFO:teuthology.orchestra.run.smithi144.stdout:(58/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.3 MB/s | 3.0 MB 00:00 2024-04-20T21:14:23.525 INFO:teuthology.orchestra.run.smithi144.stdout:(59/150): librdkafka-1.6.1-102.el9.x86_64.rpm 3.1 MB/s | 662 kB 00:00 2024-04-20T21:14:23.559 INFO:teuthology.orchestra.run.smithi144.stdout:(60/150): libtool-2.4.6-45.el9.x86_64.rpm 3.8 MB/s | 582 kB 00:00 2024-04-20T21:14:23.609 INFO:teuthology.orchestra.run.smithi144.stdout:(61/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 307 kB/s | 25 kB 00:00 2024-04-20T21:14:23.677 INFO:teuthology.orchestra.run.smithi144.stdout:(62/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 407 kB/s | 47 kB 00:00 2024-04-20T21:14:23.719 INFO:teuthology.orchestra.run.smithi144.stdout:(63/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-04-20T21:14:23.752 INFO:teuthology.orchestra.run.smithi144.stdout:(64/150): lua-5.4.4-4.el9.x86_64.rpm 2.5 MB/s | 188 kB 00:00 2024-04-20T21:14:23.803 INFO:teuthology.orchestra.run.smithi144.stdout:(65/150): openblas-0.3.21-2.el9.x86_64.rpm 425 kB/s | 35 kB 00:00 2024-04-20T21:14:24.028 INFO:teuthology.orchestra.run.smithi144.stdout:(66/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 18 MB/s | 4.1 MB 00:00 2024-04-20T21:14:24.145 INFO:teuthology.orchestra.run.smithi144.stdout:(67/150): postgresql-13.14-1.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-04-20T21:14:24.270 INFO:teuthology.orchestra.run.smithi144.stdout:(68/150): openblas-openmp-0.3.21-2.el9.x86_64.r 9.2 MB/s | 4.7 MB 00:00 2024-04-20T21:14:24.329 INFO:teuthology.orchestra.run.smithi144.stdout:(69/150): postgresql-contrib-13.14-1.el9.x86_64 4.9 MB/s | 911 kB 00:00 2024-04-20T21:14:24.362 INFO:teuthology.orchestra.run.smithi144.stdout:(70/150): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-04-20T21:14:24.479 INFO:teuthology.orchestra.run.smithi144.stdout:(71/150): protobuf-3.14.0-13.el9.x86_64.rpm 8.7 MB/s | 1.0 MB 00:00 2024-04-20T21:14:24.633 INFO:teuthology.orchestra.run.smithi144.stdout:(72/150): postgresql-server-13.14-1.el9.x86_64. 19 MB/s | 5.8 MB 00:00 2024-04-20T21:14:24.663 INFO:teuthology.orchestra.run.smithi144.stdout:(73/150): python3-attrs-20.3.0-7.el9.noarch.rpm 479 kB/s | 87 kB 00:00 2024-04-20T21:14:24.730 INFO:teuthology.orchestra.run.smithi144.stdout:(74/150): python3-devel-3.9.18-3.el9.x86_64.rpm 3.6 MB/s | 244 kB 00:00 2024-04-20T21:14:24.797 INFO:teuthology.orchestra.run.smithi144.stdout:(75/150): python3-jinja2-2.11.3-5.el9.noarch.rp 3.7 MB/s | 248 kB 00:00 2024-04-20T21:14:24.922 INFO:teuthology.orchestra.run.smithi144.stdout:(76/150): python3-babel-2.9.1-2.el9.noarch.rpm 21 MB/s | 6.0 MB 00:00 2024-04-20T21:14:24.964 INFO:teuthology.orchestra.run.smithi144.stdout:(77/150): python3-jsonpatch-1.21-16.el9.noarch. 154 kB/s | 26 kB 00:00 2024-04-20T21:14:25.006 INFO:teuthology.orchestra.run.smithi144.stdout:(78/150): python3-jsonpointer-2.0-4.el9.noarch. 231 kB/s | 19 kB 00:00 2024-04-20T21:14:25.040 INFO:teuthology.orchestra.run.smithi144.stdout:(79/150): python3-mako-1.1.4-6.el9.noarch.rpm 2.3 MB/s | 172 kB 00:00 2024-04-20T21:14:25.082 INFO:teuthology.orchestra.run.smithi144.stdout:(80/150): python3-markupsafe-1.1.1-12.el9.x86_6 467 kB/s | 35 kB 00:00 2024-04-20T21:14:25.165 INFO:teuthology.orchestra.run.smithi144.stdout:(81/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.0 MB/s | 250 kB 00:00 2024-04-20T21:14:25.232 INFO:teuthology.orchestra.run.smithi144.stdout:(82/150): python3-oauthlib-3.1.1-5.el9.noarch.r 3.3 MB/s | 222 kB 00:00 2024-04-20T21:14:25.291 INFO:teuthology.orchestra.run.smithi144.stdout:(83/150): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-20T21:14:25.366 INFO:teuthology.orchestra.run.smithi144.stdout:(84/150): python3-prettytable-0.7.2-27.el9.noar 570 kB/s | 42 kB 00:00 2024-04-20T21:14:25.483 INFO:teuthology.orchestra.run.smithi144.stdout:(85/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-04-20T21:14:25.517 INFO:teuthology.orchestra.run.smithi144.stdout:(86/150): python3-protobuf-3.14.0-13.el9.noarch 1.8 MB/s | 269 kB 00:00 2024-04-20T21:14:25.542 INFO:teuthology.orchestra.run.smithi144.stdout:(87/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-04-20T21:14:25.548 INFO:teuthology.orchestra.run.smithi086.stdout:(8/150): ceph-common-19.0.0-3133.ga8c84f59.el9. 1.4 MB/s | 21 MB 00:15 2024-04-20T21:14:25.605 INFO:teuthology.orchestra.run.smithi144.stdout:(88/150): python3-pyasn1-modules-0.4.8-6.el9.no 3.1 MB/s | 279 kB 00:00 2024-04-20T21:14:25.651 INFO:teuthology.orchestra.run.smithi144.stdout:(89/150): python3-pytz-2021.1-5.el9.noarch.rpm 472 kB/s | 51 kB 00:00 2024-04-20T21:14:25.665 INFO:teuthology.orchestra.run.smithi086.stdout:(9/150): ceph-selinux-19.0.0-3133.ga8c84f59.el9 218 kB/s | 25 kB 00:00 2024-04-20T21:14:25.694 INFO:teuthology.orchestra.run.smithi144.stdout:(90/150): python3-requests-oauthlib-1.3.0-12.el 614 kB/s | 54 kB 00:00 2024-04-20T21:14:25.785 INFO:teuthology.orchestra.run.smithi144.stdout:(91/150): python3-toml-0.10.2-6.el9.noarch.rpm 459 kB/s | 42 kB 00:00 2024-04-20T21:14:25.861 INFO:teuthology.orchestra.run.smithi144.stdout:(92/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 595 kB/s | 44 kB 00:00 2024-04-20T21:14:25.928 INFO:teuthology.orchestra.run.smithi144.stdout:(93/150): qatlib-24.02.0-1.el9.x86_64.rpm 3.3 MB/s | 221 kB 00:00 2024-04-20T21:14:26.003 INFO:teuthology.orchestra.run.smithi144.stdout:(94/150): qatlib-service-24.02.0-1.el9.x86_64.r 484 kB/s | 36 kB 00:00 2024-04-20T21:14:26.095 INFO:teuthology.orchestra.run.smithi144.stdout:(95/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 521 kB/s | 47 kB 00:00 2024-04-20T21:14:26.162 INFO:teuthology.orchestra.run.smithi144.stdout:(96/150): socat-1.7.4.1-5.el9.x86_64.rpm 4.5 MB/s | 305 kB 00:00 2024-04-20T21:14:26.237 INFO:teuthology.orchestra.run.smithi144.stdout:(97/150): uuid-1.6.2-55.el9.x86_64.rpm 779 kB/s | 58 kB 00:00 2024-04-20T21:14:26.479 INFO:teuthology.orchestra.run.smithi144.stdout:(98/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 20 MB/s | 16 MB 00:00 2024-04-20T21:14:26.496 INFO:teuthology.orchestra.run.smithi144.stdout:(99/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 364 kB/s | 94 kB 00:00 2024-04-20T21:14:26.555 INFO:teuthology.orchestra.run.smithi144.stdout:(100/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 856 kB/s | 64 kB 00:00 2024-04-20T21:14:26.622 INFO:teuthology.orchestra.run.smithi144.stdout:(101/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 359 kB/s | 45 kB 00:00 2024-04-20T21:14:26.880 INFO:teuthology.orchestra.run.smithi144.stdout:(102/150): lua-devel-5.4.4-4.el9.x86_64.rpm 69 kB/s | 22 kB 00:00 2024-04-20T21:14:27.069 INFO:teuthology.orchestra.run.smithi144.stdout:(103/150): protobuf-compiler-3.14.0-13.el9.x86_ 1.9 MB/s | 863 kB 00:00 2024-04-20T21:14:27.431 INFO:teuthology.orchestra.run.smithi144.stdout:(104/150): abseil-cpp-20211102.0-3.el9.x86_64.r 998 kB/s | 548 kB 00:00 2024-04-20T21:14:27.531 INFO:teuthology.orchestra.run.smithi144.stdout:(105/150): grpc-data-1.46.7-10.el9.noarch.rpm 196 kB/s | 19 kB 00:00 2024-04-20T21:14:27.581 INFO:teuthology.orchestra.run.smithi144.stdout:(106/150): gperftools-libs-2.9.1-2.el9.x86_64.r 606 kB/s | 309 kB 00:00 2024-04-20T21:14:27.682 INFO:teuthology.orchestra.run.smithi144.stdout:(107/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 261 kB/s | 26 kB 00:00 2024-04-20T21:14:27.790 INFO:teuthology.orchestra.run.smithi144.stdout:(108/150): liboath-2.6.7-2.el9.x86_64.rpm 454 kB/s | 49 kB 00:00 2024-04-20T21:14:27.891 INFO:teuthology.orchestra.run.smithi144.stdout:(109/150): libarrow-9.0.0-10.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-04-20T21:14:27.916 INFO:teuthology.orchestra.run.smithi144.stdout:(110/150): libunwind-1.6.2-1.el9.x86_64.rpm 541 kB/s | 67 kB 00:00 2024-04-20T21:14:27.966 INFO:teuthology.orchestra.run.smithi144.stdout:(111/150): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-04-20T21:14:28.067 INFO:teuthology.orchestra.run.smithi144.stdout:(112/150): python3-asyncssh-2.13.2-5.el9.noarch 5.4 MB/s | 548 kB 00:00 2024-04-20T21:14:28.117 INFO:teuthology.orchestra.run.smithi144.stdout:(113/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.1 MB/s | 838 kB 00:00 2024-04-20T21:14:28.149 INFO:teuthology.orchestra.run.smithi086.stdout:(10/150): ceph-osd-19.0.0-3133.ga8c84f59.el9.x8 1.2 MB/s | 17 MB 00:13 2024-04-20T21:14:28.167 INFO:teuthology.orchestra.run.smithi144.stdout:(114/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 437 kB/s | 43 kB 00:00 2024-04-20T21:14:28.218 INFO:teuthology.orchestra.run.smithi144.stdout:(115/150): python3-cachetools-4.2.4-1.el9.noarc 323 kB/s | 32 kB 00:00 2024-04-20T21:14:28.268 INFO:teuthology.orchestra.run.smithi144.stdout:(116/150): python3-certifi-2021.10.8-2.el9.noar 150 kB/s | 15 kB 00:00 2024-04-20T21:14:28.302 INFO:teuthology.orchestra.run.smithi144.stdout:(117/150): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-04-20T21:14:28.308 INFO:teuthology.orchestra.run.smithi086.stdout:(11/150): ceph-radosgw-19.0.0-3133.ga8c84f59.el 1.1 MB/s | 11 MB 00:09 2024-04-20T21:14:28.360 INFO:teuthology.orchestra.run.smithi144.stdout:(118/150): python3-cherrypy-18.6.1-2.el9.noarch 3.9 MB/s | 358 kB 00:00 2024-04-20T21:14:28.394 INFO:teuthology.orchestra.run.smithi144.stdout:(119/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-04-20T21:14:28.424 INFO:teuthology.orchestra.run.smithi086.stdout:(12/150): cephfs-shell-19.0.0-3133.ga8c84f59.el 234 kB/s | 27 kB 00:00 2024-04-20T21:14:28.453 INFO:teuthology.orchestra.run.smithi144.stdout:(120/150): python3-colorama-0.4.4-12.el9.noarch 388 kB/s | 35 kB 00:00 2024-04-20T21:14:28.486 INFO:teuthology.orchestra.run.smithi144.stdout:(121/150): python3-google-auth-2.28.2-1.el9.noa 2.2 MB/s | 203 kB 00:00 2024-04-20T21:14:28.551 INFO:teuthology.orchestra.run.smithi086.stdout:(13/150): libcephfs-devel-19.0.0-3133.ga8c84f59 255 kB/s | 32 kB 00:00 2024-04-20T21:14:28.561 INFO:teuthology.orchestra.run.smithi144.stdout:(122/150): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-04-20T21:14:28.637 INFO:teuthology.orchestra.run.smithi144.stdout:(123/150): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-04-20T21:14:28.679 INFO:teuthology.orchestra.run.smithi144.stdout:(124/150): python3-jaraco-8.2.1-3.el9.noarch.rp 92 kB/s | 11 kB 00:00 2024-04-20T21:14:28.729 INFO:teuthology.orchestra.run.smithi144.stdout:(125/150): python3-jaraco-classes-3.2.1-5.el9.n 194 kB/s | 18 kB 00:00 2024-04-20T21:14:28.750 INFO:teuthology.orchestra.run.smithi086.stdout:(14/150): cephfs-mirror-19.0.0-3133.ga8c84f59.e 375 kB/s | 225 kB 00:00 2024-04-20T21:14:28.771 INFO:teuthology.orchestra.run.smithi144.stdout:(126/150): python3-jaraco-collections-3.0.0-8.e 254 kB/s | 23 kB 00:00 2024-04-20T21:14:28.821 INFO:teuthology.orchestra.run.smithi144.stdout:(127/150): python3-jaraco-functools-3.5.0-2.el9 214 kB/s | 19 kB 00:00 2024-04-20T21:14:28.872 INFO:teuthology.orchestra.run.smithi144.stdout:(128/150): python3-jaraco-text-3.2.0-6.el9.noar 197 kB/s | 20 kB 00:00 2024-04-20T21:14:28.930 INFO:teuthology.orchestra.run.smithi144.stdout:(129/150): python3-kubernetes-26.1.0-2.el9.noar 9.5 MB/s | 1.0 MB 00:00 2024-04-20T21:14:28.972 INFO:teuthology.orchestra.run.smithi144.stdout:(130/150): python3-logutils-0.3.5-21.el9.noarch 469 kB/s | 46 kB 00:00 2024-04-20T21:14:29.000 INFO:teuthology.orchestra.run.smithi086.stdout:(15/150): libcephsqlite-19.0.0-3133.ga8c84f59.e 655 kB/s | 163 kB 00:00 2024-04-20T21:14:29.022 INFO:teuthology.orchestra.run.smithi144.stdout:(131/150): python3-more-itertools-8.12.0-2.el9. 866 kB/s | 79 kB 00:00 2024-04-20T21:14:29.064 INFO:teuthology.orchestra.run.smithi144.stdout:(132/150): python3-natsort-7.1.1-5.el9.noarch.r 633 kB/s | 58 kB 00:00 2024-04-20T21:14:29.107 INFO:teuthology.orchestra.run.smithi144.stdout:(133/150): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-04-20T21:14:29.165 INFO:teuthology.orchestra.run.smithi144.stdout:(134/150): python3-portend-3.1.0-2.el9.noarch.r 165 kB/s | 16 kB 00:00 2024-04-20T21:14:29.182 INFO:teuthology.orchestra.run.smithi144.stdout:(135/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-20T21:14:29.183 INFO:teuthology.orchestra.run.smithi086.stdout:(16/150): libcephfs2-19.0.0-3133.ga8c84f59.el9. 1.2 MB/s | 748 kB 00:00 2024-04-20T21:14:29.266 INFO:teuthology.orchestra.run.smithi144.stdout:(136/150): python3-pyperclip-1.8.2-2.el9.noarch 284 kB/s | 28 kB 00:00 2024-04-20T21:14:29.275 INFO:teuthology.orchestra.run.smithi086.stdout:(17/150): librados-devel-19.0.0-3133.ga8c84f59. 461 kB/s | 126 kB 00:00 2024-04-20T21:14:29.316 INFO:teuthology.orchestra.run.smithi144.stdout:(137/150): python3-repoze-lru-0.7-16.el9.noarch 231 kB/s | 31 kB 00:00 2024-04-20T21:14:29.350 INFO:teuthology.orchestra.run.smithi144.stdout:(138/150): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-04-20T21:14:29.416 INFO:teuthology.orchestra.run.smithi144.stdout:(139/150): python3-rsa-4.9-2.el9.noarch.rpm 594 kB/s | 59 kB 00:00 2024-04-20T21:14:29.467 INFO:teuthology.orchestra.run.smithi144.stdout:(140/150): python3-tempora-5.0.0-2.el9.noarch.r 308 kB/s | 36 kB 00:00 2024-04-20T21:14:29.534 INFO:teuthology.orchestra.run.smithi144.stdout:(141/150): python3-typing-extensions-4.4.0-2.el 439 kB/s | 51 kB 00:00 2024-04-20T21:14:29.569 INFO:teuthology.orchestra.run.smithi144.stdout:(142/150): python3-webob-1.8.7-6.el9.noarch.rpm 2.3 MB/s | 230 kB 00:00 2024-04-20T21:14:29.610 INFO:teuthology.orchestra.run.smithi144.stdout:(143/150): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-20T21:14:29.617 INFO:teuthology.orchestra.run.smithi086.stdout:(18/150): libradosstriper1-19.0.0-3133.ga8c84f5 1.1 MB/s | 501 kB 00:00 2024-04-20T21:14:29.652 INFO:teuthology.orchestra.run.smithi144.stdout:(144/150): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-04-20T21:14:29.703 INFO:teuthology.orchestra.run.smithi144.stdout:(145/150): python3-zc-lockfile-2.0-10.el9.noarc 218 kB/s | 20 kB 00:00 2024-04-20T21:14:29.736 INFO:teuthology.orchestra.run.smithi144.stdout:(146/150): re2-20211101-3.el9.x86_64.rpm 2.3 MB/s | 192 kB 00:00 2024-04-20T21:14:29.743 INFO:teuthology.orchestra.run.smithi086.stdout:(19/150): python3-ceph-argparse-19.0.0-3133.ga8 363 kB/s | 45 kB 00:00 2024-04-20T21:14:29.844 INFO:teuthology.orchestra.run.smithi144.stdout:(147/150): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-20T21:14:29.928 INFO:teuthology.orchestra.run.smithi086.stdout:(20/150): python3-ceph-common-19.0.0-3133.ga8c8 772 kB/s | 135 kB 00:00 2024-04-20T21:14:30.295 INFO:teuthology.orchestra.run.smithi086.stdout:(21/150): python3-cephfs-19.0.0-3133.ga8c84f59. 433 kB/s | 162 kB 00:00 2024-04-20T21:14:30.910 INFO:teuthology.orchestra.run.smithi086.stdout:(22/150): python3-rados-19.0.0-3133.ga8c84f59.e 521 kB/s | 321 kB 00:00 2024-04-20T21:14:31.361 INFO:teuthology.orchestra.run.smithi086.stdout:(23/150): python3-rbd-19.0.0-3133.ga8c84f59.el9 665 kB/s | 299 kB 00:00 2024-04-20T21:14:31.536 INFO:teuthology.orchestra.run.smithi086.stdout:(24/150): python3-rgw-19.0.0-3133.ga8c84f59.el9 572 kB/s | 99 kB 00:00 2024-04-20T21:14:31.711 INFO:teuthology.orchestra.run.smithi086.stdout:(25/150): rbd-fuse-19.0.0-3133.ga8c84f59.el9.x8 488 kB/s | 85 kB 00:00 2024-04-20T21:14:32.002 INFO:teuthology.orchestra.run.smithi144.stdout:(148/150): librbd1-19.0.0-3133.ga8c84f59.el9.x8 1.5 MB/s | 3.1 MB 00:02 2024-04-20T21:14:32.261 INFO:teuthology.orchestra.run.smithi144.stdout:(149/150): librados2-19.0.0-3133.ga8c84f59.el9. 1.3 MB/s | 3.4 MB 00:02 2024-04-20T21:14:34.720 INFO:teuthology.orchestra.run.smithi086.stdout:(26/150): rbd-mirror-19.0.0-3133.ga8c84f59.el9. 1.0 MB/s | 3.1 MB 00:03 2024-04-20T21:14:34.895 INFO:teuthology.orchestra.run.smithi086.stdout:(27/150): rbd-nbd-19.0.0-3133.ga8c84f59.el9.x86 980 kB/s | 170 kB 00:00 2024-04-20T21:14:34.986 INFO:teuthology.orchestra.run.smithi086.stdout:(28/150): ceph-grafana-dashboards-19.0.0-3133.g 377 kB/s | 34 kB 00:00 2024-04-20T21:14:35.104 INFO:teuthology.orchestra.run.smithi086.stdout:(29/150): librgw2-19.0.0-3133.ga8c84f59.el9.x86 944 kB/s | 5.4 MB 00:05 2024-04-20T21:14:35.179 INFO:teuthology.orchestra.run.smithi086.stdout:(30/150): ceph-mgr-cephadm-19.0.0-3133.ga8c84f5 743 kB/s | 142 kB 00:00 2024-04-20T21:14:38.179 INFO:teuthology.orchestra.run.smithi086.stdout:(31/150): ceph-mgr-dashboard-19.0.0-3133.ga8c84 1.2 MB/s | 3.5 MB 00:03 2024-04-20T21:14:38.474 INFO:teuthology.orchestra.run.smithi086.stdout:(32/150): ceph-mgr-modules-core-19.0.0-3133.ga8 848 kB/s | 249 kB 00:00 2024-04-20T21:14:38.588 INFO:teuthology.orchestra.run.smithi086.stdout:(33/150): ceph-mgr-rook-19.0.0-3133.ga8c84f59.e 437 kB/s | 49 kB 00:00 2024-04-20T21:14:38.713 INFO:teuthology.orchestra.run.smithi086.stdout:(34/150): ceph-prometheus-alerts-19.0.0-3133.ga 132 kB/s | 16 kB 00:00 2024-04-20T21:14:38.980 INFO:teuthology.orchestra.run.smithi086.stdout:(35/150): ceph-volume-19.0.0-3133.ga8c84f59.el9 1.0 MB/s | 273 kB 00:00 2024-04-20T21:14:39.822 INFO:teuthology.orchestra.run.smithi086.stdout:(36/150): cephadm-19.0.0-3133.ga8c84f59.el9.noa 906 kB/s | 763 kB 00:00 2024-04-20T21:14:39.939 INFO:teuthology.orchestra.run.smithi086.stdout:(37/150): cephfs-top-19.0.0-3133.ga8c84f59.el9. 158 kB/s | 18 kB 00:00 2024-04-20T21:14:40.055 INFO:teuthology.orchestra.run.smithi086.stdout:(38/150): fuse-2.9.9-15.el9.x86_64.rpm 695 kB/s | 80 kB 00:00 2024-04-20T21:14:40.239 INFO:teuthology.orchestra.run.smithi086.stdout:(39/150): ceph-mgr-diskprediction-local-19.0.0- 1.5 MB/s | 7.4 MB 00:05 2024-04-20T21:14:40.331 INFO:teuthology.orchestra.run.smithi086.stdout:(40/150): libgfortran-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 803 kB 00:00 2024-04-20T21:14:40.390 INFO:teuthology.orchestra.run.smithi086.stdout:(41/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.3 MB/s | 253 kB 00:00 2024-04-20T21:14:40.423 INFO:teuthology.orchestra.run.smithi086.stdout:(42/150): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 195 kB 00:00 2024-04-20T21:14:40.490 INFO:teuthology.orchestra.run.smithi086.stdout:(43/150): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.2 MB 00:00 2024-04-20T21:14:40.515 INFO:teuthology.orchestra.run.smithi086.stdout:(44/150): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 106 kB 00:00 2024-04-20T21:14:40.570 INFO:teuthology.orchestra.run.smithi086.stdout:(45/150): python3-pycparser-2.20-6.el9.noarch.r 1.7 MB/s | 135 kB 00:00 2024-04-20T21:14:40.599 INFO:teuthology.orchestra.run.smithi086.stdout:(46/150): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-04-20T21:14:40.633 INFO:teuthology.orchestra.run.smithi086.stdout:(47/150): python3-urllib3-1.26.5-5.el9.noarch.r 3.3 MB/s | 215 kB 00:00 2024-04-20T21:14:41.143 INFO:teuthology.orchestra.run.smithi144.stdout:(150/150): ceph-test-19.0.0-3133.ga8c84f59.el9. 1.6 MB/s | 49 MB 00:30 2024-04-20T21:14:41.159 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-04-20T21:14:41.160 INFO:teuthology.orchestra.run.smithi144.stdout:Total 4.0 MB/s | 218 MB 00:53 2024-04-20T21:14:42.092 INFO:teuthology.orchestra.run.smithi086.stdout:(48/150): automake-1.16.2-8.el9.noarch.rpm 462 kB/s | 689 kB 00:01 2024-04-20T21:14:42.517 INFO:teuthology.orchestra.run.smithi086.stdout:(49/150): bison-3.7.4-5.el9.x86_64.rpm 501 kB/s | 943 kB 00:01 2024-04-20T21:14:42.892 INFO:teuthology.orchestra.run.smithi086.stdout:(50/150): boost-program-options-1.75.0-8.el9.x8 134 kB/s | 107 kB 00:00 2024-04-20T21:14:43.151 INFO:teuthology.orchestra.run.smithi086.stdout:(51/150): elfutils-libelf-devel-0.190-2.el9.x86 34 kB/s | 21 kB 00:00 2024-04-20T21:14:43.893 INFO:teuthology.orchestra.run.smithi086.stdout:(52/150): flexiblas-3.0.4-8.el9.x86_64.rpm 49 kB/s | 33 kB 00:00 2024-04-20T21:14:43.956 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-04-20T21:14:44.052 INFO:teuthology.orchestra.run.smithi086.stdout:(53/150): flex-2.6.4-9.el9.x86_64.rpm 271 kB/s | 313 kB 00:01 2024-04-20T21:14:44.168 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-04-20T21:14:44.168 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-04-20T21:14:44.744 INFO:teuthology.orchestra.run.smithi086.stdout:(54/150): flexiblas-openblas-openmp-3.0.4-8.el9 26 kB/s | 18 kB 00:00 2024-04-20T21:14:45.310 INFO:teuthology.orchestra.run.smithi086.stdout:(55/150): libaio-devel-0.3.111-13.el9.x86_64.rp 20 kB/s | 11 kB 00:00 2024-04-20T21:14:46.202 INFO:teuthology.orchestra.run.smithi086.stdout:(56/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 180 kB/s | 160 kB 00:00 2024-04-20T21:14:46.719 INFO:teuthology.orchestra.run.smithi086.stdout:(57/150): ceph-test-19.0.0-3133.ga8c84f59.el9.x 2.3 MB/s | 49 MB 00:21 2024-04-20T21:14:46.778 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-04-20T21:14:46.778 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-04-20T21:14:47.328 INFO:teuthology.orchestra.run.smithi086.stdout:(58/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 40 kB/s | 45 kB 00:01 2024-04-20T21:14:48.321 INFO:teuthology.orchestra.run.smithi086.stdout:(59/150): librdkafka-1.6.1-102.el9.x86_64.rpm 414 kB/s | 662 kB 00:01 2024-04-20T21:14:48.494 INFO:teuthology.orchestra.run.smithi086.stdout:(60/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 665 kB/s | 3.0 MB 00:04 2024-04-20T21:14:48.795 INFO:teuthology.orchestra.run.smithi086.stdout:(61/150): libtool-2.4.6-45.el9.x86_64.rpm 397 kB/s | 582 kB 00:01 2024-04-20T21:14:48.995 INFO:teuthology.orchestra.run.smithi086.stdout:(62/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 38 kB/s | 25 kB 00:00 2024-04-20T21:14:49.345 INFO:teuthology.orchestra.run.smithi086.stdout:(63/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 56 kB/s | 47 kB 00:00 2024-04-20T21:14:49.921 INFO:teuthology.orchestra.run.smithi086.stdout:(64/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 277 kB/s | 292 kB 00:01 2024-04-20T21:14:49.954 INFO:teuthology.orchestra.run.smithi086.stdout:(65/150): lua-5.4.4-4.el9.x86_64.rpm 197 kB/s | 188 kB 00:00 2024-04-20T21:14:50.004 INFO:teuthology.orchestra.run.smithi086.stdout:(66/150): openblas-0.3.21-2.el9.x86_64.rpm 54 kB/s | 35 kB 00:00 2024-04-20T21:14:52.788 INFO:teuthology.orchestra.run.smithi086.stdout:(67/150): postgresql-13.14-1.el9.x86_64.rpm 584 kB/s | 1.6 MB 00:02 2024-04-20T21:14:53.093 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-04-20T21:14:53.227 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-20T21:14:53.290 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-20T21:14:53.416 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-20T21:14:53.822 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-20T21:14:53.955 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 5/152 2024-04-20T21:14:54.131 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 5/152 2024-04-20T21:14:54.165 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T21:14:54.296 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T21:14:54.367 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/152 2024-04-20T21:14:54.405 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-20T21:14:54.441 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-20T21:14:54.579 INFO:teuthology.orchestra.run.smithi086.stdout:(68/150): postgresql-contrib-13.14-1.el9.x86_64 509 kB/s | 911 kB 00:01 2024-04-20T21:14:54.817 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-20T21:14:54.877 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-20T21:14:54.902 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T21:14:55.059 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T21:14:55.087 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 13/152 2024-04-20T21:14:55.177 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 13/152 2024-04-20T21:14:55.204 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-20T21:14:55.287 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-20T21:14:55.421 INFO:teuthology.orchestra.run.smithi086.stdout:(69/150): postgresql-private-libs-13.14-1.el9.x 159 kB/s | 133 kB 00:00 2024-04-20T21:14:55.913 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-20T21:14:55.962 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-20T21:14:56.139 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-20T21:14:56.171 INFO:teuthology.orchestra.run.smithi086.stdout:(70/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 681 kB/s | 4.1 MB 00:06 2024-04-20T21:14:56.261 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-20T21:14:56.305 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-20T21:14:56.353 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-20T21:14:56.455 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-20T21:14:56.519 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-20T21:14:56.579 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-20T21:14:56.630 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-20T21:14:56.639 INFO:teuthology.orchestra.run.smithi086.stdout:(71/150): openblas-openmp-0.3.21-2.el9.x86_64.r 722 kB/s | 4.7 MB 00:06 2024-04-20T21:14:56.696 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-20T21:14:56.819 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-20T21:14:56.889 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-20T21:14:56.925 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-20T21:14:56.982 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-20T21:14:57.087 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-20T21:14:57.128 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 30/152 2024-04-20T21:14:57.198 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 31/152 2024-04-20T21:14:57.236 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 32/152 2024-04-20T21:14:57.352 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-20T21:14:57.415 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-20T21:14:57.490 INFO:teuthology.orchestra.run.smithi086.stdout:(72/150): python3-attrs-20.3.0-7.el9.noarch.rpm 103 kB/s | 87 kB 00:00 2024-04-20T21:14:57.523 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-20T21:14:57.702 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-20T21:14:57.769 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-20T21:14:58.256 INFO:teuthology.orchestra.run.smithi086.stdout:(73/150): protobuf-3.14.0-13.el9.x86_64.rpm 496 kB/s | 1.0 MB 00:02 2024-04-20T21:14:58.470 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-20T21:14:58.565 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-20T21:14:58.625 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-20T21:14:58.672 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-20T21:14:58.729 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 42/152 2024-04-20T21:14:58.772 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-20T21:14:58.858 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-20T21:14:58.961 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-20T21:14:59.000 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-20T21:14:59.040 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-20T21:14:59.106 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-20T21:14:59.156 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-20T21:14:59.218 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-20T21:14:59.276 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-20T21:14:59.298 INFO:teuthology.orchestra.run.smithi086.stdout:(74/150): python3-devel-3.9.18-3.el9.x86_64.rpm 235 kB/s | 244 kB 00:01 2024-04-20T21:14:59.471 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-20T21:14:59.529 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-20T21:14:59.566 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-20T21:14:59.638 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-20T21:14:59.770 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-20T21:14:59.808 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-20T21:14:59.854 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-20T21:14:59.890 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-20T21:14:59.923 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-20T21:15:00.108 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-20T21:15:00.289 INFO:teuthology.orchestra.run.smithi086.stdout:(75/150): python3-jinja2-2.11.3-5.el9.noarch.rp 250 kB/s | 248 kB 00:00 2024-04-20T21:15:00.292 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-20T21:15:00.348 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-20T21:15:00.385 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-20T21:15:00.444 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-20T21:15:00.529 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-20T21:15:00.600 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-20T21:15:00.653 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-20T21:15:00.702 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-20T21:15:00.756 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-20T21:15:00.805 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-20T21:15:00.882 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-20T21:15:00.898 INFO:teuthology.orchestra.run.smithi086.stdout:(76/150): python3-jsonpatch-1.21-16.el9.noarch. 42 kB/s | 26 kB 00:00 2024-04-20T21:15:00.928 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-20T21:15:01.072 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-20T21:15:01.143 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-20T21:15:01.186 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-20T21:15:01.257 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-20T21:15:01.407 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-20T21:15:01.475 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-20T21:15:02.123 INFO:teuthology.orchestra.run.smithi086.stdout:(77/150): python3-jsonpointer-2.0-4.el9.noarch. 16 kB/s | 19 kB 00:01 2024-04-20T21:15:02.489 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-20T21:15:02.548 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-20T21:15:02.582 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-20T21:15:02.709 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-20T21:15:02.777 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-20T21:15:02.885 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-20T21:15:02.934 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-20T21:15:03.023 INFO:teuthology.orchestra.run.smithi086.stdout:(78/150): python3-mako-1.1.4-6.el9.noarch.rpm 192 kB/s | 172 kB 00:00 2024-04-20T21:15:03.192 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-20T21:15:03.230 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-20T21:15:03.326 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-20T21:15:03.690 INFO:teuthology.orchestra.run.smithi086.stdout:(79/150): python3-markupsafe-1.1.1-12.el9.x86_6 52 kB/s | 35 kB 00:00 2024-04-20T21:15:03.899 INFO:teuthology.orchestra.run.smithi086.stdout:(80/150): postgresql-server-13.14-1.el9.x86_64. 698 kB/s | 5.8 MB 00:08 2024-04-20T21:15:04.163 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-20T21:15:04.340 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-20T21:15:04.999 INFO:teuthology.orchestra.run.smithi086.stdout:(81/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 228 kB/s | 250 kB 00:01 2024-04-20T21:15:05.907 INFO:teuthology.orchestra.run.smithi086.stdout:(82/150): python3-oauthlib-3.1.1-5.el9.noarch.r 245 kB/s | 222 kB 00:00 2024-04-20T21:15:06.015 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-20T21:15:06.120 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-20T21:15:06.174 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-20T21:15:06.224 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-20T21:15:06.399 INFO:teuthology.orchestra.run.smithi086.stdout:(83/150): python3-babel-2.9.1-2.el9.noarch.rpm 685 kB/s | 6.0 MB 00:08 2024-04-20T21:15:06.609 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-20T21:15:06.658 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 97/152 2024-04-20T21:15:06.724 INFO:teuthology.orchestra.run.smithi086.stdout:(84/150): python3-packaging-20.9-5.el9.noarch.r 95 kB/s | 77 kB 00:00 2024-04-20T21:15:06.779 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 97/152 2024-04-20T21:15:06.832 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 98/152 2024-04-20T21:15:06.890 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-20T21:15:07.091 INFO:teuthology.orchestra.run.smithi086.stdout:(85/150): python3-prettytable-0.7.2-27.el9.noar 61 kB/s | 42 kB 00:00 2024-04-20T21:15:07.434 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-20T21:15:07.469 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 101/152 2024-04-20T21:15:07.561 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 101/152 2024-04-20T21:15:07.588 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 102/152 2024-04-20T21:15:07.742 INFO:teuthology.orchestra.run.smithi086.stdout:(86/150): python3-protobuf-3.14.0-13.el9.noarch 265 kB/s | 269 kB 00:01 2024-04-20T21:15:08.092 INFO:teuthology.orchestra.run.smithi086.stdout:(87/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 159 kB/s | 159 kB 00:00 2024-04-20T21:15:08.813 INFO:teuthology.orchestra.run.smithi086.stdout:(88/150): python3-pyasn1-modules-0.4.8-6.el9.no 261 kB/s | 279 kB 00:01 2024-04-20T21:15:08.851 INFO:teuthology.orchestra.run.smithi086.stdout:(89/150): python3-pytz-2021.1-5.el9.noarch.rpm 67 kB/s | 51 kB 00:00 2024-04-20T21:15:09.534 INFO:teuthology.orchestra.run.smithi086.stdout:(90/150): python3-requests-oauthlib-1.3.0-12.el 75 kB/s | 54 kB 00:00 2024-04-20T21:15:10.226 INFO:teuthology.orchestra.run.smithi086.stdout:(91/150): python3-toml-0.10.2-6.el9.noarch.rpm 60 kB/s | 42 kB 00:00 2024-04-20T21:15:10.466 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T21:15:10.510 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T21:15:10.909 INFO:teuthology.orchestra.run.smithi086.stdout:(92/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 65 kB/s | 44 kB 00:00 2024-04-20T21:15:11.140 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T21:15:11.151 INFO:teuthology.orchestra.run.smithi086.stdout:(93/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 700 kB/s | 5.1 MB 00:07 2024-04-20T21:15:11.188 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 104/152 2024-04-20T21:15:11.369 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 104/152 2024-04-20T21:15:11.369 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-20T21:15:11.370 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-20T21:15:11.370 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:15:11.414 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 105/152 2024-04-20T21:15:11.818 INFO:teuthology.orchestra.run.smithi086.stdout:(94/150): qatlib-service-24.02.0-1.el9.x86_64.r 54 kB/s | 36 kB 00:00 2024-04-20T21:15:11.994 INFO:teuthology.orchestra.run.smithi086.stdout:(95/150): qatlib-24.02.0-1.el9.x86_64.rpm 204 kB/s | 221 kB 00:01 2024-04-20T21:15:12.634 INFO:teuthology.orchestra.run.smithi086.stdout:(96/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 58 kB/s | 47 kB 00:00 2024-04-20T21:15:13.152 INFO:teuthology.orchestra.run.smithi086.stdout:(97/150): socat-1.7.4.1-5.el9.x86_64.rpm 263 kB/s | 305 kB 00:01 2024-04-20T21:15:13.369 INFO:teuthology.orchestra.run.smithi086.stdout:(98/150): uuid-1.6.2-55.el9.x86_64.rpm 79 kB/s | 58 kB 00:00 2024-04-20T21:15:14.036 INFO:teuthology.orchestra.run.smithi086.stdout:(99/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 106 kB/s | 94 kB 00:00 2024-04-20T21:15:14.103 INFO:teuthology.orchestra.run.smithi086.stdout:(100/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 87 kB/s | 64 kB 00:00 2024-04-20T21:15:14.287 INFO:teuthology.orchestra.run.smithi086.stdout:(101/150): lua-devel-5.4.4-4.el9.x86_64.rpm 122 kB/s | 22 kB 00:00 2024-04-20T21:15:14.454 INFO:teuthology.orchestra.run.smithi086.stdout:(102/150): protobuf-compiler-3.14.0-13.el9.x86_ 5.1 MB/s | 863 kB 00:00 2024-04-20T21:15:14.570 INFO:teuthology.orchestra.run.smithi086.stdout:(103/150): abseil-cpp-20211102.0-3.el9.x86_64.r 4.7 MB/s | 548 kB 00:00 2024-04-20T21:15:14.612 INFO:teuthology.orchestra.run.smithi086.stdout:(104/150): gperftools-libs-2.9.1-2.el9.x86_64.r 7.4 MB/s | 309 kB 00:00 2024-04-20T21:15:14.646 INFO:teuthology.orchestra.run.smithi086.stdout:(105/150): grpc-data-1.46.7-10.el9.noarch.rpm 593 kB/s | 19 kB 00:00 2024-04-20T21:15:14.796 INFO:teuthology.orchestra.run.smithi086.stdout:(106/150): libarrow-9.0.0-10.el9.x86_64.rpm 30 MB/s | 4.4 MB 00:00 2024-04-20T21:15:14.821 INFO:teuthology.orchestra.run.smithi086.stdout:(107/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 57 kB/s | 45 kB 00:00 2024-04-20T21:15:14.847 INFO:teuthology.orchestra.run.smithi086.stdout:(108/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 548 kB/s | 26 kB 00:00 2024-04-20T21:15:14.881 INFO:teuthology.orchestra.run.smithi086.stdout:(109/150): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-04-20T21:15:14.906 INFO:teuthology.orchestra.run.smithi086.stdout:(110/150): liboath-2.6.7-2.el9.x86_64.rpm 584 kB/s | 49 kB 00:00 2024-04-20T21:15:14.931 INFO:teuthology.orchestra.run.smithi086.stdout:(111/150): luarocks-3.9.2-1.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-04-20T21:15:14.981 INFO:teuthology.orchestra.run.smithi086.stdout:(112/150): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2024-04-20T21:15:15.051 INFO:teuthology.orchestra.run.smithi086.stdout:(113/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 5.7 MB/s | 838 kB 00:00 2024-04-20T21:15:15.074 INFO:teuthology.orchestra.run.smithi086.stdout:(114/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 476 kB/s | 43 kB 00:00 2024-04-20T21:15:15.091 INFO:teuthology.orchestra.run.smithi086.stdout:(115/150): python3-cachetools-4.2.4-1.el9.noarc 820 kB/s | 32 kB 00:00 2024-04-20T21:15:15.107 INFO:teuthology.orchestra.run.smithi086.stdout:(116/150): python3-certifi-2021.10.8-2.el9.noar 458 kB/s | 15 kB 00:00 2024-04-20T21:15:15.141 INFO:teuthology.orchestra.run.smithi086.stdout:(117/150): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-04-20T21:15:15.175 INFO:teuthology.orchestra.run.smithi086.stdout:(118/150): python3-cherrypy-18.6.1-2.el9.noarch 5.3 MB/s | 358 kB 00:00 2024-04-20T21:15:15.208 INFO:teuthology.orchestra.run.smithi086.stdout:(119/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.5 MB/s | 303 kB 00:00 2024-04-20T21:15:15.233 INFO:teuthology.orchestra.run.smithi086.stdout:(120/150): python3-colorama-0.4.4-12.el9.noarch 611 kB/s | 35 kB 00:00 2024-04-20T21:15:15.268 INFO:teuthology.orchestra.run.smithi086.stdout:(121/150): python3-google-auth-2.28.2-1.el9.noa 3.4 MB/s | 203 kB 00:00 2024-04-20T21:15:15.309 INFO:teuthology.orchestra.run.smithi086.stdout:(122/150): python3-grpcio-tools-1.46.7-10.el9.x 3.6 MB/s | 144 kB 00:00 2024-04-20T21:15:15.411 INFO:teuthology.orchestra.run.smithi086.stdout:(123/150): python3-grpcio-1.46.7-10.el9.x86_64. 12 MB/s | 2.0 MB 00:00 2024-04-20T21:15:15.443 INFO:teuthology.orchestra.run.smithi086.stdout:(124/150): python3-jaraco-8.2.1-3.el9.noarch.rp 80 kB/s | 11 kB 00:00 2024-04-20T21:15:15.468 INFO:teuthology.orchestra.run.smithi086.stdout:(125/150): python3-jaraco-classes-3.2.1-5.el9.n 316 kB/s | 18 kB 00:00 2024-04-20T21:15:15.493 INFO:teuthology.orchestra.run.smithi086.stdout:(126/150): python3-jaraco-collections-3.0.0-8.e 470 kB/s | 23 kB 00:00 2024-04-20T21:15:15.519 INFO:teuthology.orchestra.run.smithi086.stdout:(127/150): python3-jaraco-functools-3.5.0-2.el9 393 kB/s | 19 kB 00:00 2024-04-20T21:15:15.544 INFO:teuthology.orchestra.run.smithi086.stdout:(128/150): python3-jaraco-text-3.2.0-6.el9.noar 397 kB/s | 20 kB 00:00 2024-04-20T21:15:15.594 INFO:teuthology.orchestra.run.smithi086.stdout:(129/150): python3-kubernetes-26.1.0-2.el9.noar 14 MB/s | 1.0 MB 00:00 2024-04-20T21:15:15.620 INFO:teuthology.orchestra.run.smithi086.stdout:(130/150): python3-logutils-0.3.5-21.el9.noarch 621 kB/s | 46 kB 00:00 2024-04-20T21:15:15.645 INFO:teuthology.orchestra.run.smithi086.stdout:(131/150): python3-more-itertools-8.12.0-2.el9. 1.6 MB/s | 79 kB 00:00 2024-04-20T21:15:15.670 INFO:teuthology.orchestra.run.smithi086.stdout:(132/150): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-04-20T21:15:15.704 INFO:teuthology.orchestra.run.smithi086.stdout:(133/150): python3-pecan-1.4.2-3.el9.noarch.rpm 4.6 MB/s | 272 kB 00:00 2024-04-20T21:15:15.729 INFO:teuthology.orchestra.run.smithi086.stdout:(134/150): python3-portend-3.1.0-2.el9.noarch.r 284 kB/s | 16 kB 00:00 2024-04-20T21:15:15.771 INFO:teuthology.orchestra.run.smithi086.stdout:(135/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-04-20T21:15:15.796 INFO:teuthology.orchestra.run.smithi086.stdout:(136/150): python3-pyperclip-1.8.2-2.el9.noarch 428 kB/s | 28 kB 00:00 2024-04-20T21:15:15.822 INFO:teuthology.orchestra.run.smithi086.stdout:(137/150): python3-repoze-lru-0.7-16.el9.noarch 621 kB/s | 31 kB 00:00 2024-04-20T21:15:15.855 INFO:teuthology.orchestra.run.smithi086.stdout:(138/150): python3-routes-2.5.1-5.el9.noarch.rp 3.2 MB/s | 188 kB 00:00 2024-04-20T21:15:15.889 INFO:teuthology.orchestra.run.smithi086.stdout:(139/150): python3-rsa-4.9-2.el9.noarch.rpm 891 kB/s | 59 kB 00:00 2024-04-20T21:15:15.914 INFO:teuthology.orchestra.run.smithi086.stdout:(140/150): python3-tempora-5.0.0-2.el9.noarch.r 618 kB/s | 36 kB 00:00 2024-04-20T21:15:15.956 INFO:teuthology.orchestra.run.smithi086.stdout:(141/150): python3-typing-extensions-4.4.0-2.el 770 kB/s | 51 kB 00:00 2024-04-20T21:15:15.990 INFO:teuthology.orchestra.run.smithi086.stdout:(142/150): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-20T21:15:16.023 INFO:teuthology.orchestra.run.smithi086.stdout:(143/150): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-04-20T21:15:16.065 INFO:teuthology.orchestra.run.smithi086.stdout:(144/150): python3-werkzeug-2.0.3-3.el9.1.noarc 5.6 MB/s | 427 kB 00:00 2024-04-20T21:15:16.124 INFO:teuthology.orchestra.run.smithi086.stdout:(145/150): python3-zc-lockfile-2.0-10.el9.noarc 201 kB/s | 20 kB 00:00 2024-04-20T21:15:16.158 INFO:teuthology.orchestra.run.smithi086.stdout:(146/150): re2-20211101-3.el9.x86_64.rpm 2.1 MB/s | 192 kB 00:00 2024-04-20T21:15:16.233 INFO:teuthology.orchestra.run.smithi086.stdout:(147/150): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-04-20T21:15:22.899 INFO:teuthology.orchestra.run.smithi086.stdout:(148/150): librbd1-19.0.0-3133.ga8c84f59.el9.x8 483 kB/s | 3.1 MB 00:06 2024-04-20T21:15:24.225 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 105/152 2024-04-20T21:15:24.226 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /sys 2024-04-20T21:15:24.226 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /proc 2024-04-20T21:15:24.226 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /mnt 2024-04-20T21:15:24.226 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /var/tmp 2024-04-20T21:15:24.226 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /home 2024-04-20T21:15:24.226 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /root 2024-04-20T21:15:24.226 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /tmp 2024-04-20T21:15:24.226 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:15:24.469 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 106/152 2024-04-20T21:15:24.517 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 106/152 2024-04-20T21:15:24.518 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:15:24.518 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-20T21:15:24.518 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:15:24.666 INFO:teuthology.orchestra.run.smithi086.stdout:(149/150): librados2-19.0.0-3133.ga8c84f59.el9. 409 kB/s | 3.4 MB 00:08 2024-04-20T21:15:26.172 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 107/152 2024-04-20T21:15:26.250 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 107/152 2024-04-20T21:15:26.250 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:15:26.250 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-20T21:15:26.251 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T21:15:26.251 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T21:15:26.251 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:15:26.497 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 108/152 2024-04-20T21:15:26.565 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 108/152 2024-04-20T21:15:26.565 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:15:26.565 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-20T21:15:26.566 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T21:15:26.566 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T21:15:26.566 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:15:27.047 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 109/152 2024-04-20T21:15:27.115 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 109/152 2024-04-20T21:15:27.116 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:15:27.116 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-20T21:15:27.116 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T21:15:27.116 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T21:15:27.116 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:15:27.256 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-20T21:15:27.346 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-20T21:15:27.438 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-20T21:15:27.663 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-20T21:15:27.718 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-20T21:15:27.836 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-20T21:15:27.947 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-20T21:15:28.109 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-20T21:15:28.161 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-20T21:15:28.201 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-20T21:15:28.272 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-20T21:15:28.916 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 121/152 2024-04-20T21:15:28.962 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 121/152 2024-04-20T21:15:29.016 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 122/152 2024-04-20T21:15:29.125 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 123/152 2024-04-20T21:15:29.156 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 124/152 2024-04-20T21:15:30.458 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 124/152 2024-04-20T21:15:30.500 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 125/152 2024-04-20T21:15:30.707 INFO:teuthology.orchestra.run.smithi086.stdout:(150/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 761 kB/s | 16 MB 00:21 2024-04-20T21:15:30.719 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-04-20T21:15:30.719 INFO:teuthology.orchestra.run.smithi086.stdout:Total 2.7 MB/s | 218 MB 01:21 2024-04-20T21:15:31.593 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 125/152 2024-04-20T21:15:31.637 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 126/152 2024-04-20T21:15:31.783 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 126/152 2024-04-20T21:15:31.974 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 127/152 2024-04-20T21:15:32.002 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/152 2024-04-20T21:15:32.074 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/152 2024-04-20T21:15:32.074 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:15:32.074 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-20T21:15:32.074 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T21:15:32.074 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T21:15:32.074 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:15:32.153 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 129/152 2024-04-20T21:15:32.191 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 129/152 2024-04-20T21:15:32.328 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 130/152 2024-04-20T21:15:32.399 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 131/152 2024-04-20T21:15:33.456 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-20T21:15:33.465 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-04-20T21:15:33.489 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 133/152 2024-04-20T21:15:33.556 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 133/152 2024-04-20T21:15:33.556 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:15:33.556 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-20T21:15:33.556 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T21:15:33.556 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T21:15:33.556 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:15:33.614 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 134/152 2024-04-20T21:15:33.679 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-04-20T21:15:33.679 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-04-20T21:15:33.683 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 134/152 2024-04-20T21:15:33.683 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:15:33.683 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-20T21:15:33.683 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:15:33.765 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/152 2024-04-20T21:15:33.846 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/152 2024-04-20T21:15:33.846 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:15:33.847 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-20T21:15:33.847 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-20T21:15:33.847 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-20T21:15:33.847 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:15:34.215 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/152 2024-04-20T21:15:34.307 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/152 2024-04-20T21:15:34.307 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:15:34.308 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-20T21:15:34.308 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T21:15:34.308 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T21:15:34.308 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:15:36.400 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-04-20T21:15:36.401 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-04-20T21:15:39.380 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 137/152 2024-04-20T21:15:39.434 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 138/152 2024-04-20T21:15:39.599 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 139/152 2024-04-20T21:15:39.895 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-20T21:15:39.943 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-20T21:15:40.254 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 142/152 2024-04-20T21:15:40.318 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-20T21:15:40.371 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 144/152 2024-04-20T21:15:41.550 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T21:15:41.611 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T21:15:41.673 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T21:15:42.759 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-04-20T21:15:42.876 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-20T21:15:42.912 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-20T21:15:42.956 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 146/152 2024-04-20T21:15:43.051 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-20T21:15:43.084 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-20T21:15:43.174 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-20T21:15:43.338 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-20T21:15:43.390 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-20T21:15:43.390 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-20T21:15:43.500 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-20T21:15:43.523 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-20T21:15:43.523 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T21:15:43.610 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 5/152 2024-04-20T21:15:43.777 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 5/152 2024-04-20T21:15:43.811 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T21:15:43.963 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T21:15:44.029 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/152 2024-04-20T21:15:44.029 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T21:15:44.029 INFO:teuthology.orchestra.run.smithi144.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-20T21:15:44.029 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:15:44.059 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-20T21:15:44.096 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-20T21:15:44.498 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-20T21:15:44.593 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-20T21:15:44.624 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T21:15:44.795 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T21:15:44.826 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 13/152 2024-04-20T21:15:44.947 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 13/152 2024-04-20T21:15:44.977 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-20T21:15:45.047 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-20T21:15:45.532 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-20T21:15:45.605 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-20T21:15:45.812 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-20T21:15:45.941 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-20T21:15:45.986 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-20T21:15:46.035 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-20T21:15:46.162 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-20T21:15:46.216 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-20T21:15:46.277 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-20T21:15:46.333 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-20T21:15:46.366 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-20T21:15:46.485 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-20T21:15:46.555 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-20T21:15:46.598 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-20T21:15:46.666 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-20T21:15:46.778 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-20T21:15:46.823 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 30/152 2024-04-20T21:15:46.867 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 31/152 2024-04-20T21:15:46.896 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 32/152 2024-04-20T21:15:47.009 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-20T21:15:47.077 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-20T21:15:47.186 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-20T21:15:47.369 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-20T21:15:47.439 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-20T21:15:48.149 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-20T21:15:48.253 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-20T21:15:48.295 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-20T21:15:48.340 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-20T21:15:48.389 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 42/152 2024-04-20T21:15:48.427 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-20T21:15:48.515 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-20T21:15:48.611 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-20T21:15:48.653 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-20T21:15:48.693 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-20T21:15:48.776 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-20T21:15:48.815 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-20T21:15:48.879 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-20T21:15:48.937 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-20T21:15:49.129 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-20T21:15:49.182 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-20T21:15:49.219 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-20T21:15:49.274 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-20T21:15:49.406 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-20T21:15:49.451 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-20T21:15:49.512 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-20T21:15:49.559 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-20T21:15:49.592 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-20T21:15:49.828 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-20T21:15:49.927 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 1/152 2024-04-20T21:15:49.927 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2/152 2024-04-20T21:15:49.927 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 3/152 2024-04-20T21:15:49.927 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 4/152 2024-04-20T21:15:49.927 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 5/152 2024-04-20T21:15:49.927 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T21:15:49.927 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/152 2024-04-20T21:15:49.927 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 8/152 2024-04-20T21:15:49.928 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 9/152 2024-04-20T21:15:49.928 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 10/152 2024-04-20T21:15:49.928 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 11/152 2024-04-20T21:15:49.928 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T21:15:49.928 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 13/152 2024-04-20T21:15:49.928 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 14/152 2024-04-20T21:15:49.928 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 15/152 2024-04-20T21:15:49.928 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 16/152 2024-04-20T21:15:49.928 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 17/152 2024-04-20T21:15:49.928 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 18/152 2024-04-20T21:15:49.928 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 19/152 2024-04-20T21:15:49.928 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 20/152 2024-04-20T21:15:49.928 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 21/152 2024-04-20T21:15:49.929 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 22/152 2024-04-20T21:15:49.929 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 23/152 2024-04-20T21:15:49.929 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 24/152 2024-04-20T21:15:49.929 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 25/152 2024-04-20T21:15:49.929 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 26/152 2024-04-20T21:15:49.929 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 27/152 2024-04-20T21:15:49.929 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 28/152 2024-04-20T21:15:49.929 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 29/152 2024-04-20T21:15:49.929 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 30/152 2024-04-20T21:15:49.929 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 31/152 2024-04-20T21:15:49.929 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 32/152 2024-04-20T21:15:49.929 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 33/152 2024-04-20T21:15:49.929 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 34/152 2024-04-20T21:15:49.929 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 35/152 2024-04-20T21:15:49.930 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 36/152 2024-04-20T21:15:49.930 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 37/152 2024-04-20T21:15:49.930 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 38/152 2024-04-20T21:15:49.930 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 39/152 2024-04-20T21:15:49.930 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-20T21:15:49.930 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-20T21:15:49.930 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-20T21:15:49.930 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-20T21:15:49.930 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-20T21:15:49.930 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-20T21:15:49.930 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-20T21:15:49.930 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-20T21:15:49.930 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-20T21:15:49.931 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-20T21:15:49.931 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-20T21:15:49.931 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-20T21:15:49.931 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-20T21:15:49.931 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-20T21:15:49.931 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-20T21:15:49.931 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-20T21:15:49.931 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-20T21:15:49.931 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-20T21:15:49.931 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-20T21:15:49.931 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-20T21:15:49.931 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-20T21:15:49.932 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-20T21:15:49.932 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-20T21:15:49.932 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-20T21:15:49.932 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-20T21:15:49.932 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-20T21:15:49.932 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-20T21:15:49.932 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-20T21:15:49.932 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-20T21:15:49.932 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-20T21:15:49.932 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-20T21:15:49.932 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-20T21:15:49.932 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-20T21:15:49.932 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-20T21:15:49.932 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-20T21:15:49.932 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-20T21:15:49.933 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-20T21:15:49.933 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-20T21:15:49.933 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-20T21:15:49.933 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-20T21:15:49.933 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-20T21:15:49.933 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-20T21:15:49.933 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-20T21:15:49.933 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-20T21:15:49.933 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-20T21:15:49.933 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-20T21:15:49.933 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-20T21:15:49.933 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-20T21:15:49.933 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-20T21:15:49.933 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-20T21:15:49.934 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-20T21:15:49.934 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-20T21:15:49.934 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-20T21:15:49.934 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-20T21:15:49.934 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-20T21:15:49.934 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-20T21:15:49.934 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-20T21:15:49.934 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-20T21:15:49.934 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-20T21:15:49.934 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-20T21:15:49.934 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-20T21:15:49.934 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-20T21:15:49.934 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-20T21:15:49.935 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-20T21:15:49.935 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-20T21:15:49.935 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-20T21:15:49.935 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-20T21:15:49.935 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-20T21:15:49.935 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-20T21:15:49.935 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-20T21:15:49.935 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-20T21:15:49.935 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-20T21:15:49.935 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-20T21:15:49.935 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-20T21:15:49.935 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-20T21:15:49.935 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-20T21:15:49.935 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-20T21:15:49.936 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-20T21:15:49.936 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-20T21:15:49.936 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-20T21:15:49.936 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-20T21:15:49.936 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-20T21:15:49.936 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-20T21:15:49.936 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-20T21:15:49.936 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-20T21:15:49.936 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-20T21:15:49.936 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-20T21:15:49.936 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-20T21:15:49.936 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-20T21:15:49.936 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-20T21:15:49.936 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-20T21:15:49.937 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-20T21:15:49.937 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-20T21:15:49.937 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-20T21:15:49.937 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-20T21:15:49.937 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-20T21:15:49.937 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-20T21:15:49.937 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-20T21:15:49.937 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-20T21:15:49.937 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-20T21:15:49.937 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-20T21:15:49.937 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-20T21:15:49.937 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-20T21:15:49.937 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-20T21:15:49.937 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-20T21:15:49.937 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-20T21:15:49.938 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-20T21:15:49.938 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-20T21:15:49.938 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-20T21:15:49.938 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 149/152 2024-04-20T21:15:49.938 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-20T21:15:49.938 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 151/152 2024-04-20T21:15:50.016 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-20T21:15:50.060 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-20T21:15:50.135 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-20T21:15:50.192 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-20T21:15:50.273 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-20T21:15:50.345 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-20T21:15:50.416 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-20T21:15:50.488 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-20T21:15:50.543 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-20T21:15:50.593 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-20T21:15:50.667 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-20T21:15:50.713 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-20T21:15:50.860 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-20T21:15:50.930 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-20T21:15:50.968 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-20T21:15:51.032 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-20T21:15:51.183 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-20T21:15:51.253 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-20T21:15:52.250 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-20T21:15:52.301 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-20T21:15:52.334 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-20T21:15:52.353 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T21:15:52.353 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:15:52.353 INFO:teuthology.orchestra.run.smithi144.stdout:Upgraded: 2024-04-20T21:15:52.354 INFO:teuthology.orchestra.run.smithi144.stdout: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:52.354 INFO:teuthology.orchestra.run.smithi144.stdout: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:52.354 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-04-20T21:15:52.354 INFO:teuthology.orchestra.run.smithi144.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-20T21:15:52.354 INFO:teuthology.orchestra.run.smithi144.stdout: automake-1.16.2-8.el9.noarch 2024-04-20T21:15:52.354 INFO:teuthology.orchestra.run.smithi144.stdout: bison-3.7.4-5.el9.x86_64 2024-04-20T21:15:52.354 INFO:teuthology.orchestra.run.smithi144.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-20T21:15:52.354 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:52.354 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:52.354 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:52.354 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:52.354 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:15:52.355 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:52.355 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:52.355 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:52.355 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:15:52.355 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:15:52.355 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:15:52.355 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:15:52.355 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:15:52.355 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:52.355 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:52.355 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:15:52.355 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:52.355 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:52.356 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:52.356 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:15:52.356 INFO:teuthology.orchestra.run.smithi144.stdout: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:15:52.356 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:52.356 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:52.356 INFO:teuthology.orchestra.run.smithi144.stdout: cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:15:52.356 INFO:teuthology.orchestra.run.smithi144.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-20T21:15:52.356 INFO:teuthology.orchestra.run.smithi144.stdout: flex-2.6.4-9.el9.x86_64 2024-04-20T21:15:52.356 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-20T21:15:52.356 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-20T21:15:52.356 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-20T21:15:52.356 INFO:teuthology.orchestra.run.smithi144.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-20T21:15:52.356 INFO:teuthology.orchestra.run.smithi144.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-20T21:15:52.357 INFO:teuthology.orchestra.run.smithi144.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-20T21:15:52.357 INFO:teuthology.orchestra.run.smithi144.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-20T21:15:52.357 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-20T21:15:52.357 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-20T21:15:52.357 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:52.357 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:52.357 INFO:teuthology.orchestra.run.smithi144.stdout: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:52.357 INFO:teuthology.orchestra.run.smithi144.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-20T21:15:52.357 INFO:teuthology.orchestra.run.smithi144.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-20T21:15:52.357 INFO:teuthology.orchestra.run.smithi144.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-20T21:15:52.357 INFO:teuthology.orchestra.run.smithi144.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-20T21:15:52.357 INFO:teuthology.orchestra.run.smithi144.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-20T21:15:52.357 INFO:teuthology.orchestra.run.smithi144.stdout: librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:52.358 INFO:teuthology.orchestra.run.smithi144.stdout: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:52.358 INFO:teuthology.orchestra.run.smithi144.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-20T21:15:52.358 INFO:teuthology.orchestra.run.smithi144.stdout: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:52.358 INFO:teuthology.orchestra.run.smithi144.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-20T21:15:52.358 INFO:teuthology.orchestra.run.smithi144.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-20T21:15:52.358 INFO:teuthology.orchestra.run.smithi144.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-20T21:15:52.358 INFO:teuthology.orchestra.run.smithi144.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-20T21:15:52.362 INFO:teuthology.orchestra.run.smithi144.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-20T21:15:52.362 INFO:teuthology.orchestra.run.smithi144.stdout: lua-5.4.4-4.el9.x86_64 2024-04-20T21:15:52.362 INFO:teuthology.orchestra.run.smithi144.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-20T21:15:52.362 INFO:teuthology.orchestra.run.smithi144.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-20T21:15:52.362 INFO:teuthology.orchestra.run.smithi144.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-20T21:15:52.363 INFO:teuthology.orchestra.run.smithi144.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-20T21:15:52.363 INFO:teuthology.orchestra.run.smithi144.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-20T21:15:52.363 INFO:teuthology.orchestra.run.smithi144.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-20T21:15:52.363 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-13.14-1.el9.x86_64 2024-04-20T21:15:52.363 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-20T21:15:52.363 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-20T21:15:52.364 INFO:teuthology.orchestra.run.smithi144.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-20T21:15:52.364 INFO:teuthology.orchestra.run.smithi144.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-20T21:15:52.364 INFO:teuthology.orchestra.run.smithi144.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-20T21:15:52.364 INFO:teuthology.orchestra.run.smithi144.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-20T21:15:52.364 INFO:teuthology.orchestra.run.smithi144.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-20T21:15:52.365 INFO:teuthology.orchestra.run.smithi144.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-20T21:15:52.365 INFO:teuthology.orchestra.run.smithi144.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-20T21:15:52.365 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-20T21:15:52.365 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:52.365 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:52.365 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:52.365 INFO:teuthology.orchestra.run.smithi144.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-20T21:15:52.366 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-20T21:15:52.366 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-20T21:15:52.366 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-20T21:15:52.366 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-20T21:15:52.366 INFO:teuthology.orchestra.run.smithi144.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-20T21:15:52.366 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-20T21:15:52.366 INFO:teuthology.orchestra.run.smithi144.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-20T21:15:52.366 INFO:teuthology.orchestra.run.smithi144.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-20T21:15:52.366 INFO:teuthology.orchestra.run.smithi144.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-20T21:15:52.366 INFO:teuthology.orchestra.run.smithi144.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-20T21:15:52.366 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-20T21:15:52.366 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-20T21:15:52.367 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-20T21:15:52.367 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-20T21:15:52.367 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-20T21:15:52.367 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-20T21:15:52.367 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-20T21:15:52.367 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-20T21:15:52.367 INFO:teuthology.orchestra.run.smithi144.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-20T21:15:52.367 INFO:teuthology.orchestra.run.smithi144.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-20T21:15:52.368 INFO:teuthology.orchestra.run.smithi144.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-20T21:15:52.368 INFO:teuthology.orchestra.run.smithi144.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-20T21:15:52.368 INFO:teuthology.orchestra.run.smithi144.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-20T21:15:52.368 INFO:teuthology.orchestra.run.smithi144.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-20T21:15:52.368 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-20T21:15:52.368 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-20T21:15:52.368 INFO:teuthology.orchestra.run.smithi144.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-20T21:15:52.368 INFO:teuthology.orchestra.run.smithi144.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-20T21:15:52.368 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-20T21:15:52.369 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ply-3.11-14.el9.noarch 2024-04-20T21:15:52.369 INFO:teuthology.orchestra.run.smithi144.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-20T21:15:52.369 INFO:teuthology.orchestra.run.smithi144.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-20T21:15:52.369 INFO:teuthology.orchestra.run.smithi144.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-20T21:15:52.369 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-20T21:15:52.369 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-20T21:15:52.369 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-20T21:15:52.369 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-20T21:15:52.369 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-20T21:15:52.369 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-20T21:15:52.369 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:52.369 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:52.369 INFO:teuthology.orchestra.run.smithi144.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-20T21:15:52.369 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-20T21:15:52.370 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-20T21:15:52.370 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:52.370 INFO:teuthology.orchestra.run.smithi144.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-20T21:15:52.370 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-20T21:15:52.370 INFO:teuthology.orchestra.run.smithi144.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-20T21:15:52.370 INFO:teuthology.orchestra.run.smithi144.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-20T21:15:52.370 INFO:teuthology.orchestra.run.smithi144.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-20T21:15:52.370 INFO:teuthology.orchestra.run.smithi144.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-20T21:15:52.370 INFO:teuthology.orchestra.run.smithi144.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-20T21:15:52.370 INFO:teuthology.orchestra.run.smithi144.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-20T21:15:52.370 INFO:teuthology.orchestra.run.smithi144.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-20T21:15:52.370 INFO:teuthology.orchestra.run.smithi144.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-20T21:15:52.370 INFO:teuthology.orchestra.run.smithi144.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-20T21:15:52.371 INFO:teuthology.orchestra.run.smithi144.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-20T21:15:52.371 INFO:teuthology.orchestra.run.smithi144.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-20T21:15:52.371 INFO:teuthology.orchestra.run.smithi144.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-20T21:15:52.371 INFO:teuthology.orchestra.run.smithi144.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-20T21:15:52.371 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:52.371 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:52.371 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:15:52.371 INFO:teuthology.orchestra.run.smithi144.stdout: re2-1:20211101-3.el9.x86_64 2024-04-20T21:15:52.371 INFO:teuthology.orchestra.run.smithi144.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-20T21:15:52.371 INFO:teuthology.orchestra.run.smithi144.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-20T21:15:52.371 INFO:teuthology.orchestra.run.smithi144.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-20T21:15:52.371 INFO:teuthology.orchestra.run.smithi144.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-20T21:15:52.371 INFO:teuthology.orchestra.run.smithi144.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-20T21:15:52.372 INFO:teuthology.orchestra.run.smithi144.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-20T21:15:52.372 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:15:52.372 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-04-20T21:15:52.455 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-20T21:15:52.555 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-20T21:15:52.662 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-20T21:15:52.703 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-20T21:15:52.933 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-20T21:15:52.966 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-20T21:15:52.986 DEBUG:teuthology.parallel:result is None 2024-04-20T21:15:53.080 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-20T21:15:53.927 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-20T21:15:54.075 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-20T21:15:55.731 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-20T21:15:55.831 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-20T21:15:55.877 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-20T21:15:55.926 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-20T21:15:56.398 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-20T21:15:56.443 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 97/152 2024-04-20T21:15:56.564 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 97/152 2024-04-20T21:15:56.600 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 98/152 2024-04-20T21:15:56.658 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-20T21:15:57.229 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-20T21:15:57.271 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 101/152 2024-04-20T21:15:57.390 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 101/152 2024-04-20T21:15:57.423 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 102/152 2024-04-20T21:16:00.146 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T21:16:00.194 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T21:16:00.824 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T21:16:00.873 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 104/152 2024-04-20T21:16:01.025 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 104/152 2024-04-20T21:16:01.025 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-20T21:16:01.025 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-20T21:16:01.025 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:16:01.074 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 105/152 2024-04-20T21:16:13.577 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 105/152 2024-04-20T21:16:13.578 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /sys 2024-04-20T21:16:13.578 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /proc 2024-04-20T21:16:13.578 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /mnt 2024-04-20T21:16:13.578 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /var/tmp 2024-04-20T21:16:13.578 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /home 2024-04-20T21:16:13.578 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /root 2024-04-20T21:16:13.578 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /tmp 2024-04-20T21:16:13.578 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:16:13.812 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 106/152 2024-04-20T21:16:13.856 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 106/152 2024-04-20T21:16:13.856 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:16:13.856 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-20T21:16:13.856 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:16:15.497 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 107/152 2024-04-20T21:16:15.602 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 107/152 2024-04-20T21:16:15.603 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:16:15.603 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-20T21:16:15.603 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T21:16:15.603 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T21:16:15.603 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:16:15.856 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 108/152 2024-04-20T21:16:15.950 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 108/152 2024-04-20T21:16:15.951 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:16:15.951 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-20T21:16:15.951 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T21:16:15.951 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T21:16:15.951 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:16:16.440 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 109/152 2024-04-20T21:16:16.536 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 109/152 2024-04-20T21:16:16.536 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:16:16.536 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-20T21:16:16.536 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T21:16:16.537 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T21:16:16.537 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:16:16.684 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-20T21:16:16.759 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-20T21:16:16.850 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-20T21:16:17.111 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-20T21:16:17.171 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-20T21:16:17.262 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-20T21:16:17.382 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-20T21:16:17.563 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-20T21:16:17.630 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-20T21:16:17.669 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-20T21:16:17.701 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-20T21:16:18.466 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 121/152 2024-04-20T21:16:18.513 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 121/152 2024-04-20T21:16:18.570 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 122/152 2024-04-20T21:16:18.678 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 123/152 2024-04-20T21:16:18.705 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 124/152 2024-04-20T21:16:20.021 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 124/152 2024-04-20T21:16:20.058 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 125/152 2024-04-20T21:16:21.151 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 125/152 2024-04-20T21:16:21.191 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 126/152 2024-04-20T21:16:21.335 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 126/152 2024-04-20T21:16:21.519 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 127/152 2024-04-20T21:16:21.585 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/152 2024-04-20T21:16:21.656 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/152 2024-04-20T21:16:21.656 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:16:21.657 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-20T21:16:21.657 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T21:16:21.657 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T21:16:21.657 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:16:21.718 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 129/152 2024-04-20T21:16:21.758 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 129/152 2024-04-20T21:16:21.902 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 130/152 2024-04-20T21:16:21.973 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 131/152 2024-04-20T21:16:22.992 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-20T21:16:23.030 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 133/152 2024-04-20T21:16:23.118 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 133/152 2024-04-20T21:16:23.118 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:16:23.118 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-20T21:16:23.118 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T21:16:23.118 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T21:16:23.118 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:16:23.172 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 134/152 2024-04-20T21:16:23.267 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 134/152 2024-04-20T21:16:23.267 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:16:23.267 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-20T21:16:23.267 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:16:23.348 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/152 2024-04-20T21:16:23.452 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/152 2024-04-20T21:16:23.452 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:16:23.452 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-20T21:16:23.452 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-20T21:16:23.453 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-20T21:16:23.453 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:16:23.856 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/152 2024-04-20T21:16:23.948 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/152 2024-04-20T21:16:23.948 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T21:16:23.948 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-20T21:16:23.948 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T21:16:23.948 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T21:16:23.948 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:16:29.015 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 137/152 2024-04-20T21:16:29.073 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 138/152 2024-04-20T21:16:29.207 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 139/152 2024-04-20T21:16:29.263 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-20T21:16:29.309 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-20T21:16:29.644 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 142/152 2024-04-20T21:16:29.701 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-20T21:16:29.729 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 144/152 2024-04-20T21:16:30.687 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T21:16:30.735 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T21:16:30.790 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T21:16:32.058 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 146/152 2024-04-20T21:16:32.174 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-20T21:16:32.268 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-20T21:16:32.455 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-20T21:16:32.498 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-20T21:16:32.498 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-20T21:16:32.655 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-20T21:16:32.655 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T21:16:33.196 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T21:16:33.196 INFO:teuthology.orchestra.run.smithi086.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-20T21:16:33.197 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:16:39.018 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 1/152 2024-04-20T21:16:39.018 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2/152 2024-04-20T21:16:39.018 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 3/152 2024-04-20T21:16:39.019 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 4/152 2024-04-20T21:16:39.019 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 5/152 2024-04-20T21:16:39.019 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T21:16:39.019 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/152 2024-04-20T21:16:39.019 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 8/152 2024-04-20T21:16:39.019 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 9/152 2024-04-20T21:16:39.019 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 10/152 2024-04-20T21:16:39.019 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 11/152 2024-04-20T21:16:39.019 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T21:16:39.019 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 13/152 2024-04-20T21:16:39.019 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 14/152 2024-04-20T21:16:39.019 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 15/152 2024-04-20T21:16:39.020 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 16/152 2024-04-20T21:16:39.020 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 17/152 2024-04-20T21:16:39.020 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 18/152 2024-04-20T21:16:39.020 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 19/152 2024-04-20T21:16:39.020 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 20/152 2024-04-20T21:16:39.020 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 21/152 2024-04-20T21:16:39.020 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 22/152 2024-04-20T21:16:39.020 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 23/152 2024-04-20T21:16:39.020 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 24/152 2024-04-20T21:16:39.020 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 25/152 2024-04-20T21:16:39.020 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 26/152 2024-04-20T21:16:39.020 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 27/152 2024-04-20T21:16:39.033 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 28/152 2024-04-20T21:16:39.033 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 29/152 2024-04-20T21:16:39.033 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 30/152 2024-04-20T21:16:39.033 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 31/152 2024-04-20T21:16:39.033 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 32/152 2024-04-20T21:16:39.034 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 33/152 2024-04-20T21:16:39.034 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 34/152 2024-04-20T21:16:39.034 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 35/152 2024-04-20T21:16:39.034 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 36/152 2024-04-20T21:16:39.034 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 37/152 2024-04-20T21:16:39.034 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 38/152 2024-04-20T21:16:39.034 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 39/152 2024-04-20T21:16:39.034 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-20T21:16:39.034 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-20T21:16:39.034 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-20T21:16:39.034 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-20T21:16:39.034 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-20T21:16:39.035 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-20T21:16:39.035 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-20T21:16:39.035 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-20T21:16:39.035 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-20T21:16:39.035 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-20T21:16:39.035 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-20T21:16:39.035 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-20T21:16:39.035 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-20T21:16:39.035 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-20T21:16:39.035 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-20T21:16:39.035 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-20T21:16:39.035 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-20T21:16:39.035 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-20T21:16:39.036 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-20T21:16:39.036 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-20T21:16:39.036 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-20T21:16:39.036 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-20T21:16:39.036 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-20T21:16:39.036 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-20T21:16:39.036 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-20T21:16:39.036 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-20T21:16:39.036 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-20T21:16:39.036 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-20T21:16:39.036 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-20T21:16:39.036 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-20T21:16:39.037 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-20T21:16:39.037 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-20T21:16:39.037 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-20T21:16:39.037 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-20T21:16:39.037 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-20T21:16:39.037 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-20T21:16:39.037 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-20T21:16:39.037 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-20T21:16:39.037 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-20T21:16:39.037 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-20T21:16:39.037 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-20T21:16:39.037 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-20T21:16:39.037 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-20T21:16:39.037 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-20T21:16:39.038 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-20T21:16:39.038 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-20T21:16:39.038 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-20T21:16:39.038 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-20T21:16:39.038 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-20T21:16:39.038 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-20T21:16:39.038 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-20T21:16:39.038 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-20T21:16:39.038 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-20T21:16:39.038 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-20T21:16:39.038 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-20T21:16:39.038 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-20T21:16:39.038 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-20T21:16:39.039 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-20T21:16:39.039 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-20T21:16:39.039 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-20T21:16:39.039 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-20T21:16:39.039 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-20T21:16:39.039 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-20T21:16:39.039 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-20T21:16:39.039 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-20T21:16:39.039 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-20T21:16:39.039 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-20T21:16:39.039 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-20T21:16:39.039 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-20T21:16:39.039 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-20T21:16:39.039 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-20T21:16:39.040 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-20T21:16:39.040 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-20T21:16:39.040 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-20T21:16:39.040 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-20T21:16:39.041 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-20T21:16:39.041 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-20T21:16:39.041 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-20T21:16:39.041 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-20T21:16:39.041 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-20T21:16:39.041 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-20T21:16:39.041 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-20T21:16:39.041 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-20T21:16:39.041 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-20T21:16:39.041 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-20T21:16:39.042 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-20T21:16:39.043 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-20T21:16:39.043 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-20T21:16:39.043 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-20T21:16:39.043 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-20T21:16:39.043 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-20T21:16:39.043 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-20T21:16:39.043 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-20T21:16:39.043 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-20T21:16:39.043 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-20T21:16:39.043 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-20T21:16:39.043 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-20T21:16:39.044 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-20T21:16:39.044 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-20T21:16:39.044 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-20T21:16:39.044 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-20T21:16:39.044 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-20T21:16:39.044 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-20T21:16:39.044 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-20T21:16:39.044 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-20T21:16:39.044 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-20T21:16:39.044 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-20T21:16:39.044 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-20T21:16:39.044 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-20T21:16:39.044 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 149/152 2024-04-20T21:16:39.044 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-20T21:16:39.044 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 151/152 2024-04-20T21:16:41.522 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T21:16:41.522 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:16:41.522 INFO:teuthology.orchestra.run.smithi086.stdout:Upgraded: 2024-04-20T21:16:41.522 INFO:teuthology.orchestra.run.smithi086.stdout: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:16:41.522 INFO:teuthology.orchestra.run.smithi086.stdout: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:16:41.522 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-04-20T21:16:41.522 INFO:teuthology.orchestra.run.smithi086.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-20T21:16:41.523 INFO:teuthology.orchestra.run.smithi086.stdout: automake-1.16.2-8.el9.noarch 2024-04-20T21:16:41.523 INFO:teuthology.orchestra.run.smithi086.stdout: bison-3.7.4-5.el9.x86_64 2024-04-20T21:16:41.523 INFO:teuthology.orchestra.run.smithi086.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-20T21:16:41.523 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:16:41.523 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:16:41.523 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:16:41.523 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:16:41.523 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:16:41.523 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:16:41.523 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:16:41.523 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:16:41.523 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:16:41.523 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:16:41.523 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:16:41.523 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:16:41.524 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:16:41.524 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:16:41.524 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:16:41.524 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:16:41.524 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:16:41.524 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:16:41.524 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:16:41.524 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:16:41.524 INFO:teuthology.orchestra.run.smithi086.stdout: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:16:41.524 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:16:41.524 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:16:41.524 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T21:16:41.524 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-20T21:16:41.524 INFO:teuthology.orchestra.run.smithi086.stdout: flex-2.6.4-9.el9.x86_64 2024-04-20T21:16:41.525 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-20T21:16:41.525 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-20T21:16:41.525 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-20T21:16:41.525 INFO:teuthology.orchestra.run.smithi086.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-20T21:16:41.525 INFO:teuthology.orchestra.run.smithi086.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-20T21:16:41.525 INFO:teuthology.orchestra.run.smithi086.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-20T21:16:41.525 INFO:teuthology.orchestra.run.smithi086.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-20T21:16:41.525 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-20T21:16:41.525 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-20T21:16:41.525 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:16:41.525 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:16:41.525 INFO:teuthology.orchestra.run.smithi086.stdout: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:16:41.525 INFO:teuthology.orchestra.run.smithi086.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-20T21:16:41.525 INFO:teuthology.orchestra.run.smithi086.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-20T21:16:41.526 INFO:teuthology.orchestra.run.smithi086.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-20T21:16:41.526 INFO:teuthology.orchestra.run.smithi086.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-20T21:16:41.526 INFO:teuthology.orchestra.run.smithi086.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-20T21:16:41.526 INFO:teuthology.orchestra.run.smithi086.stdout: librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:16:41.526 INFO:teuthology.orchestra.run.smithi086.stdout: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:16:41.526 INFO:teuthology.orchestra.run.smithi086.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-20T21:16:41.526 INFO:teuthology.orchestra.run.smithi086.stdout: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:16:41.526 INFO:teuthology.orchestra.run.smithi086.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-20T21:16:41.526 INFO:teuthology.orchestra.run.smithi086.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-20T21:16:41.526 INFO:teuthology.orchestra.run.smithi086.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-20T21:16:41.526 INFO:teuthology.orchestra.run.smithi086.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-20T21:16:41.526 INFO:teuthology.orchestra.run.smithi086.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-20T21:16:41.526 INFO:teuthology.orchestra.run.smithi086.stdout: lua-5.4.4-4.el9.x86_64 2024-04-20T21:16:41.527 INFO:teuthology.orchestra.run.smithi086.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-20T21:16:41.527 INFO:teuthology.orchestra.run.smithi086.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-20T21:16:41.527 INFO:teuthology.orchestra.run.smithi086.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-20T21:16:41.527 INFO:teuthology.orchestra.run.smithi086.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-20T21:16:41.527 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-20T21:16:41.527 INFO:teuthology.orchestra.run.smithi086.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-20T21:16:41.527 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-13.14-1.el9.x86_64 2024-04-20T21:16:41.527 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-20T21:16:41.527 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-20T21:16:41.527 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-20T21:16:41.527 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-20T21:16:41.527 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-20T21:16:41.527 INFO:teuthology.orchestra.run.smithi086.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-20T21:16:41.527 INFO:teuthology.orchestra.run.smithi086.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-20T21:16:41.527 INFO:teuthology.orchestra.run.smithi086.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-20T21:16:41.528 INFO:teuthology.orchestra.run.smithi086.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-20T21:16:41.528 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-20T21:16:41.528 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:16:41.528 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:16:41.528 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:16:41.528 INFO:teuthology.orchestra.run.smithi086.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-20T21:16:41.528 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-20T21:16:41.528 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-20T21:16:41.528 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-20T21:16:41.528 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-20T21:16:41.528 INFO:teuthology.orchestra.run.smithi086.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-20T21:16:41.528 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-20T21:16:41.528 INFO:teuthology.orchestra.run.smithi086.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-20T21:16:41.528 INFO:teuthology.orchestra.run.smithi086.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-20T21:16:41.528 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-20T21:16:41.529 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-20T21:16:41.529 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-20T21:16:41.529 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-20T21:16:41.529 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-20T21:16:41.529 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-20T21:16:41.529 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-20T21:16:41.529 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-20T21:16:41.529 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-20T21:16:41.529 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-20T21:16:41.529 INFO:teuthology.orchestra.run.smithi086.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-20T21:16:41.529 INFO:teuthology.orchestra.run.smithi086.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-20T21:16:41.529 INFO:teuthology.orchestra.run.smithi086.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-20T21:16:41.529 INFO:teuthology.orchestra.run.smithi086.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-20T21:16:41.529 INFO:teuthology.orchestra.run.smithi086.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-20T21:16:41.530 INFO:teuthology.orchestra.run.smithi086.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-20T21:16:41.530 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-20T21:16:41.530 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-20T21:16:41.530 INFO:teuthology.orchestra.run.smithi086.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-20T21:16:41.530 INFO:teuthology.orchestra.run.smithi086.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-20T21:16:41.530 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-20T21:16:41.530 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ply-3.11-14.el9.noarch 2024-04-20T21:16:41.530 INFO:teuthology.orchestra.run.smithi086.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-20T21:16:41.530 INFO:teuthology.orchestra.run.smithi086.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-20T21:16:41.530 INFO:teuthology.orchestra.run.smithi086.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-20T21:16:41.530 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-20T21:16:41.530 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-20T21:16:41.530 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-20T21:16:41.530 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-20T21:16:41.531 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-20T21:16:41.531 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-20T21:16:41.531 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:16:41.531 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:16:41.531 INFO:teuthology.orchestra.run.smithi086.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-20T21:16:41.531 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-20T21:16:41.531 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-20T21:16:41.531 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:16:41.531 INFO:teuthology.orchestra.run.smithi086.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-20T21:16:41.531 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-20T21:16:41.531 INFO:teuthology.orchestra.run.smithi086.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-20T21:16:41.531 INFO:teuthology.orchestra.run.smithi086.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-20T21:16:41.531 INFO:teuthology.orchestra.run.smithi086.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-20T21:16:41.531 INFO:teuthology.orchestra.run.smithi086.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-20T21:16:41.531 INFO:teuthology.orchestra.run.smithi086.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-20T21:16:41.531 INFO:teuthology.orchestra.run.smithi086.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-20T21:16:41.532 INFO:teuthology.orchestra.run.smithi086.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-20T21:16:41.532 INFO:teuthology.orchestra.run.smithi086.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-20T21:16:41.532 INFO:teuthology.orchestra.run.smithi086.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-20T21:16:41.532 INFO:teuthology.orchestra.run.smithi086.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-20T21:16:41.532 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-20T21:16:41.532 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-20T21:16:41.532 INFO:teuthology.orchestra.run.smithi086.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-20T21:16:41.532 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:16:41.532 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:16:41.532 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T21:16:41.532 INFO:teuthology.orchestra.run.smithi086.stdout: re2-1:20211101-3.el9.x86_64 2024-04-20T21:16:41.532 INFO:teuthology.orchestra.run.smithi086.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-20T21:16:41.532 INFO:teuthology.orchestra.run.smithi086.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-20T21:16:41.532 INFO:teuthology.orchestra.run.smithi086.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-20T21:16:41.532 INFO:teuthology.orchestra.run.smithi086.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-20T21:16:41.533 INFO:teuthology.orchestra.run.smithi086.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-20T21:16:41.533 INFO:teuthology.orchestra.run.smithi086.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-20T21:16:41.533 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:16:41.533 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-04-20T21:16:42.126 DEBUG:teuthology.parallel:result is None 2024-04-20T21:16:42.127 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T21:16:42.341 DEBUG:teuthology.orchestra.run.smithi086:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-20T21:16:42.395 INFO:teuthology.orchestra.run.smithi086.stdout:19.0.0-3133.ga8c84f59.el9 2024-04-20T21:16:42.396 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3133.ga8c84f59.el9 2024-04-20T21:16:42.396 INFO:teuthology.task.install:The correct ceph version 19.0.0-3133.ga8c84f59 is installed. 2024-04-20T21:16:42.399 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T21:16:42.620 DEBUG:teuthology.orchestra.run.smithi144:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-20T21:16:42.693 INFO:teuthology.orchestra.run.smithi144.stdout:19.0.0-3133.ga8c84f59.el9 2024-04-20T21:16:42.693 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3133.ga8c84f59.el9 2024-04-20T21:16:42.693 INFO:teuthology.task.install:The correct ceph version 19.0.0-3133.ga8c84f59 is installed. 2024-04-20T21:16:42.695 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-20T21:16:42.695 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:16:42.695 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-20T21:16:42.768 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:16:42.768 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-20T21:16:42.860 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-20T21:16:42.860 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:16:42.860 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/daemon-helper 2024-04-20T21:16:42.928 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-20T21:16:43.032 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:16:43.032 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/daemon-helper 2024-04-20T21:16:43.121 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-20T21:16:43.247 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-20T21:16:43.247 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:16:43.247 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-20T21:16:43.312 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-20T21:16:43.416 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:16:43.416 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-20T21:16:43.496 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-20T21:16:43.621 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-20T21:16:43.621 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:16:43.621 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/stdin-killer 2024-04-20T21:16:43.690 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-20T21:16:43.792 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:16:43.792 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/stdin-killer 2024-04-20T21:16:43.879 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-20T21:16:43.961 INFO:teuthology.run_tasks:Running task ceph... 2024-04-20T21:16:44.044 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-20T21:16:44.044 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 777 /var/log/ceph 2024-04-20T21:16:44.049 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 777 /var/log/ceph 2024-04-20T21:16:44.135 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-20T21:16:44.135 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-20T21:16:44.139 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-20T21:16:44.222 INFO:tasks.ceph:Creating extra log directories... 2024-04-20T21:16:44.222 DEBUG:teuthology.orchestra.run.smithi086:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-20T21:16:44.227 DEBUG:teuthology.orchestra.run.smithi144:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-20T21:16:44.312 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-20T21:16:44.313 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-20T21:16:44.313 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-20T21:16:44.313 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:16:44.313 DEBUG:teuthology.orchestra.run.smithi086:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-20T21:16:44.349 DEBUG:teuthology.orchestra.run.smithi086:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-20T21:16:44.560 DEBUG:teuthology.orchestra.run.smithi086:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-20T21:16:44.622 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:16:44.622 DEBUG:teuthology.orchestra.run.smithi144:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-20T21:16:44.677 DEBUG:teuthology.orchestra.run.smithi144:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-20T21:16:44.887 DEBUG:teuthology.orchestra.run.smithi144:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-20T21:16:44.979 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-20T21:16:44.979 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-20T21:16:44.979 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665408', 'branch': 'wip-rishabh-testing-20240416.193735', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/forward-scrub}', 'email': None, 'first_in_suite': False, 'job_id': '7665408', 'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-testing-20240416.193735'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-testing-20240416.193735', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 75, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 7954, 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'sleep_before_teardown': 0, 'subset': '364/512', 'suite': 'fs', 'suite_branch': 'wip-rishabh-testing-20240416.193735', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi086.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHZnpT/76zTvzEFODAI0hhznpXMK/yubClrbEpPoEgNSA8R/v4zsK8z84AoBNTlVHHCt97Nlq4Z56KKW9fxSmvY=', 'smithi144.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNFjKrxoh8SrDM4g/XgOPDuG61tFRgAkBwBCkI70X6ampv0lTPrAbP6bOnQPzoCAqUAMhMswMK2ehmSHQ/ViGVw='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_forward_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-20_20:50:37', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-20T21:16:44.980 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-20T21:16:44.985 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-20T21:16:45.038 DEBUG:teuthology.orchestra.run.smithi086:> sudo install -d -m0777 -- /var/run/ceph 2024-04-20T21:16:45.068 DEBUG:teuthology.orchestra.run.smithi144:> sudo install -d -m0777 -- /var/run/ceph 2024-04-20T21:16:45.152 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:16:45.152 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/scratch_devs of=/dev/stdout 2024-04-20T21:16:45.191 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-20T21:16:45.191 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_1 2024-04-20T21:16:45.269 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-20T21:16:45.270 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T21:16:45.270 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-04-20T21:16:45.270 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T21:16:45.270 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T21:16:45.270 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-04-20 21:16:33.672569302 +0000 2024-04-20T21:16:45.270 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-04-20 21:12:51.676760648 +0000 2024-04-20T21:16:45.270 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-04-20 21:12:51.676760648 +0000 2024-04-20T21:16:45.270 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-04-20 21:12:51.676760648 +0000 2024-04-20T21:16:45.270 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-20T21:16:45.373 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-20T21:16:45.373 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-20T21:16:45.373 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000390518 s, 1.3 MB/s 2024-04-20T21:16:45.376 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-20T21:16:45.419 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_2 2024-04-20T21:16:45.498 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-20T21:16:45.499 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T21:16:45.499 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 6h/6d Inode: 652 Links: 1 2024-04-20T21:16:45.499 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T21:16:45.499 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T21:16:45.499 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-04-20 21:16:33.673569281 +0000 2024-04-20T21:16:45.499 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-04-20 21:12:52.284747900 +0000 2024-04-20T21:16:45.499 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-04-20 21:12:52.284747900 +0000 2024-04-20T21:16:45.499 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-04-20 21:12:52.284747900 +0000 2024-04-20T21:16:45.500 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-20T21:16:45.602 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-20T21:16:45.602 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-20T21:16:45.602 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000268367 s, 1.9 MB/s 2024-04-20T21:16:45.607 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-20T21:16:45.657 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_3 2024-04-20T21:16:45.696 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-20T21:16:45.696 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T21:16:45.696 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 6h/6d Inode: 670 Links: 1 2024-04-20T21:16:45.696 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T21:16:45.696 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T21:16:45.696 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-04-20 21:16:33.674569260 +0000 2024-04-20T21:16:45.696 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-04-20 21:12:52.861735802 +0000 2024-04-20T21:16:45.697 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-04-20 21:12:52.861735802 +0000 2024-04-20T21:16:45.697 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-04-20 21:12:52.861735802 +0000 2024-04-20T21:16:45.697 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-20T21:16:45.802 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-20T21:16:45.802 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-20T21:16:45.802 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000390241 s, 1.3 MB/s 2024-04-20T21:16:45.805 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-20T21:16:45.849 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_4 2024-04-20T21:16:45.928 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-20T21:16:45.929 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T21:16:45.929 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-04-20T21:16:45.929 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T21:16:45.929 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T21:16:45.929 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-04-20 21:16:33.675569239 +0000 2024-04-20T21:16:45.929 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-04-20 21:12:53.447723516 +0000 2024-04-20T21:16:45.929 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-04-20 21:12:53.447723516 +0000 2024-04-20T21:16:45.929 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-04-20 21:12:53.447723516 +0000 2024-04-20T21:16:45.929 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-20T21:16:46.029 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-20T21:16:46.030 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-20T21:16:46.030 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000294759 s, 1.7 MB/s 2024-04-20T21:16:46.033 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-20T21:16:46.075 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-20T21:16:46.075 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:16:46.075 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/scratch_devs of=/dev/stdout 2024-04-20T21:16:46.133 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-20T21:16:46.133 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_1 2024-04-20T21:16:46.187 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-20T21:16:46.187 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T21:16:46.187 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 6h/6d Inode: 633 Links: 1 2024-04-20T21:16:46.187 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T21:16:46.187 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T21:16:46.187 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-04-20 21:15:44.510208353 +0000 2024-04-20T21:16:46.187 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-04-20 21:12:51.684568075 +0000 2024-04-20T21:16:46.187 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-04-20 21:12:51.684568075 +0000 2024-04-20T21:16:46.187 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-04-20 21:12:51.684568075 +0000 2024-04-20T21:16:46.188 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-20T21:16:46.270 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-04-20T21:16:46.270 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-04-20T21:16:46.270 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000372162 s, 1.4 MB/s 2024-04-20T21:16:46.271 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-20T21:16:46.328 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_2 2024-04-20T21:16:46.378 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-20T21:16:46.379 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T21:16:46.379 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 6h/6d Inode: 650 Links: 1 2024-04-20T21:16:46.379 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T21:16:46.379 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T21:16:46.379 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-04-20 21:15:44.511208336 +0000 2024-04-20T21:16:46.379 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-04-20 21:12:52.294557666 +0000 2024-04-20T21:16:46.379 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-04-20 21:12:52.294557666 +0000 2024-04-20T21:16:46.379 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-04-20 21:12:52.294557666 +0000 2024-04-20T21:16:46.379 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-20T21:16:46.456 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-04-20T21:16:46.456 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-04-20T21:16:46.456 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000459568 s, 1.1 MB/s 2024-04-20T21:16:46.459 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-20T21:16:46.515 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_3 2024-04-20T21:16:46.567 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-20T21:16:46.567 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T21:16:46.567 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-04-20T21:16:46.567 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T21:16:46.567 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T21:16:46.567 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-04-20 21:15:44.511208336 +0000 2024-04-20T21:16:46.567 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-04-20 21:12:52.892547461 +0000 2024-04-20T21:16:46.567 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-04-20 21:12:52.892547461 +0000 2024-04-20T21:16:46.567 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-04-20 21:12:52.892547461 +0000 2024-04-20T21:16:46.567 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-20T21:16:46.642 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-04-20T21:16:46.642 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-04-20T21:16:46.642 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000339664 s, 1.5 MB/s 2024-04-20T21:16:46.646 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-20T21:16:46.700 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_4 2024-04-20T21:16:46.748 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-20T21:16:46.749 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T21:16:46.749 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 6h/6d Inode: 688 Links: 1 2024-04-20T21:16:46.749 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T21:16:46.749 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T21:16:46.749 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-04-20 21:15:44.513208303 +0000 2024-04-20T21:16:46.749 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-04-20 21:12:53.495537171 +0000 2024-04-20T21:16:46.749 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-04-20 21:12:53.495537171 +0000 2024-04-20T21:16:46.749 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-04-20 21:12:53.495537171 +0000 2024-04-20T21:16:46.749 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-20T21:16:46.825 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-04-20T21:16:46.825 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-04-20T21:16:46.825 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000305181 s, 1.7 MB/s 2024-04-20T21:16:46.828 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-20T21:16:46.886 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-20T21:16:46.887 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi086.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi144.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-20T21:16:46.887 INFO:tasks.ceph:Generating config... 2024-04-20T21:16:46.888 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-20T21:16:46.888 INFO:tasks.ceph:[client] debug client = 20 2024-04-20T21:16:46.888 INFO:tasks.ceph:[client] debug ms = 1 2024-04-20T21:16:46.888 INFO:tasks.ceph:[client] pid file = 2024-04-20T21:16:46.888 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-20T21:16:46.888 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-20T21:16:46.889 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-20T21:16:46.889 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-20T21:16:46.889 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-20T21:16:46.889 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-20T21:16:46.889 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-20T21:16:46.889 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-20T21:16:46.889 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-20T21:16:46.889 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-20T21:16:46.889 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-20T21:16:46.889 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-20T21:16:46.889 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-20T21:16:46.889 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-20T21:16:46.889 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-20T21:16:46.889 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-20T21:16:46.889 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-20T21:16:46.890 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-20T21:16:46.890 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-20T21:16:46.890 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-20T21:16:46.890 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-20T21:16:46.890 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-20T21:16:46.890 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-20T21:16:46.890 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-20T21:16:46.890 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-20T21:16:46.890 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-20T21:16:46.890 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-20T21:16:46.890 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-20T21:16:46.890 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-20T21:16:46.890 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-20T21:16:46.890 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-20T21:16:46.891 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-20T21:16:46.891 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-20T21:16:46.891 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-20T21:16:46.891 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-20T21:16:46.891 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-20T21:16:46.891 INFO:tasks.ceph:Setting up mon.a... 2024-04-20T21:16:46.891 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-20T21:16:46.993 INFO:teuthology.orchestra.run.smithi086.stdout:creating /etc/ceph/ceph.keyring 2024-04-20T21:16:46.999 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-20T21:16:47.091 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-20T21:16:47.149 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.86'), ('mon.b', '172.21.15.144'), ('mon.c', '[v2:172.21.15.144:3301,v1:172.21.15.144:6790]')] 2024-04-20T21:16:47.150 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.86,172.21.15.144,[v2:172.21.15.144:3301,v1:172.21.15.144:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-20T21:16:47.151 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:16:47.151 DEBUG:teuthology.orchestra.run.smithi086:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-20T21:16:47.186 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.86 --add b 172.21.15.144 --addv c '[v2:172.21.15.144:3301,v1:172.21.15.144:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-20T21:16:47.312 INFO:teuthology.orchestra.run.smithi086.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-20T21:16:47.312 INFO:teuthology.orchestra.run.smithi086.stdout:monmaptool: generated fsid 9c2dec1d-d7ae-4a83-a812-79922808e278 2024-04-20T21:16:47.312 INFO:teuthology.orchestra.run.smithi086.stdout:setting min_mon_release = quincy 2024-04-20T21:16:47.312 INFO:teuthology.orchestra.run.smithi086.stdout:epoch 0 2024-04-20T21:16:47.312 INFO:teuthology.orchestra.run.smithi086.stdout:fsid 9c2dec1d-d7ae-4a83-a812-79922808e278 2024-04-20T21:16:47.312 INFO:teuthology.orchestra.run.smithi086.stdout:last_changed 2024-04-20T21:16:47.313647+0000 2024-04-20T21:16:47.312 INFO:teuthology.orchestra.run.smithi086.stdout:created 2024-04-20T21:16:47.313647+0000 2024-04-20T21:16:47.312 INFO:teuthology.orchestra.run.smithi086.stdout:min_mon_release 17 (quincy) 2024-04-20T21:16:47.312 INFO:teuthology.orchestra.run.smithi086.stdout:election_strategy: 1 2024-04-20T21:16:47.313 INFO:teuthology.orchestra.run.smithi086.stdout:0: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.a 2024-04-20T21:16:47.313 INFO:teuthology.orchestra.run.smithi086.stdout:1: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.b 2024-04-20T21:16:47.313 INFO:teuthology.orchestra.run.smithi086.stdout:2: [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] mon.c 2024-04-20T21:16:47.313 INFO:teuthology.orchestra.run.smithi086.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-20T21:16:47.315 DEBUG:teuthology.orchestra.run.smithi086:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-20T21:16:47.353 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 9c2dec1d-d7ae-4a83-a812-79922808e278... 2024-04-20T21:16:47.354 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-20T21:16:47.397 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-20T21:16:47.510 INFO:teuthology.orchestra.run.smithi086.stdout:[global] 2024-04-20T21:16:47.511 INFO:teuthology.orchestra.run.smithi086.stdout: chdir = "" 2024-04-20T21:16:47.511 INFO:teuthology.orchestra.run.smithi086.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-20T21:16:47.511 INFO:teuthology.orchestra.run.smithi086.stdout: auth supported = cephx 2024-04-20T21:16:47.511 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:16:47.511 INFO:teuthology.orchestra.run.smithi086.stdout: filestore xattr use omap = true 2024-04-20T21:16:47.511 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:16:47.511 INFO:teuthology.orchestra.run.smithi086.stdout: mon clock drift allowed = 1.000 2024-04-20T21:16:47.511 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:16:47.511 INFO:teuthology.orchestra.run.smithi086.stdout: osd crush chooseleaf type = 0 2024-04-20T21:16:47.511 INFO:teuthology.orchestra.run.smithi086.stdout: auth debug = true 2024-04-20T21:16:47.511 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:16:47.511 INFO:teuthology.orchestra.run.smithi086.stdout: ms die on old message = true 2024-04-20T21:16:47.511 INFO:teuthology.orchestra.run.smithi086.stdout: ms die on bug = true 2024-04-20T21:16:47.511 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:16:47.511 INFO:teuthology.orchestra.run.smithi086.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-20T21:16:47.511 INFO:teuthology.orchestra.run.smithi086.stdout: mon pg warn max object skew = 0 2024-04-20T21:16:47.512 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:16:47.512 INFO:teuthology.orchestra.run.smithi086.stdout: # disable pg_autoscaler by default for new pools 2024-04-20T21:16:47.512 INFO:teuthology.orchestra.run.smithi086.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-20T21:16:47.512 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:16:47.512 INFO:teuthology.orchestra.run.smithi086.stdout: osd pool default size = 2 2024-04-20T21:16:47.512 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:16:47.512 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd allow primary affinity = true 2024-04-20T21:16:47.512 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd allow pg remap = true 2024-04-20T21:16:47.512 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on legacy crush tunables = false 2024-04-20T21:16:47.512 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on crush straw calc version zero = false 2024-04-20T21:16:47.512 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on no sortbitwise = false 2024-04-20T21:16:47.512 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on osd down out interval zero = false 2024-04-20T21:16:47.512 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on too few osds = false 2024-04-20T21:16:47.512 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-20T21:16:47.512 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-20T21:16:47.512 INFO:teuthology.orchestra.run.smithi086.stdout: mon_allow_pool_size_one = true 2024-04-20T21:16:47.513 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:16:47.513 INFO:teuthology.orchestra.run.smithi086.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-20T21:16:47.513 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:16:47.513 INFO:teuthology.orchestra.run.smithi086.stdout: osd default data pool replay window = 5 2024-04-20T21:16:47.513 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:16:47.513 INFO:teuthology.orchestra.run.smithi086.stdout: mon allow pool delete = true 2024-04-20T21:16:47.513 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:16:47.513 INFO:teuthology.orchestra.run.smithi086.stdout: mon cluster log file level = debug 2024-04-20T21:16:47.513 INFO:teuthology.orchestra.run.smithi086.stdout: debug asserts on shutdown = true 2024-04-20T21:16:47.513 INFO:teuthology.orchestra.run.smithi086.stdout: mon health detail to clog = false 2024-04-20T21:16:47.513 INFO:teuthology.orchestra.run.smithi086.stdout: mon host = "172.21.15.86,172.21.15.144,[v2:172.21.15.144:3301,v1:172.21.15.144:6790]" 2024-04-20T21:16:47.513 INFO:teuthology.orchestra.run.smithi086.stdout: fsid = 9c2dec1d-d7ae-4a83-a812-79922808e278 2024-04-20T21:16:47.513 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:16:47.513 INFO:teuthology.orchestra.run.smithi086.stdout:[osd] 2024-04-20T21:16:47.513 INFO:teuthology.orchestra.run.smithi086.stdout: osd journal size = 100 2024-04-20T21:16:47.513 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:16:47.513 INFO:teuthology.orchestra.run.smithi086.stdout: osd scrub load threshold = 5.0 2024-04-20T21:16:47.514 INFO:teuthology.orchestra.run.smithi086.stdout: osd scrub max interval = 600 2024-04-20T21:16:47.514 INFO:teuthology.orchestra.run.smithi086.stdout: osd mclock profile = high_recovery_ops 2024-04-20T21:16:47.514 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:16:47.514 INFO:teuthology.orchestra.run.smithi086.stdout: osd recover clone overlap = true 2024-04-20T21:16:47.514 INFO:teuthology.orchestra.run.smithi086.stdout: osd recovery max chunk = 1048576 2024-04-20T21:16:47.514 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:16:47.514 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug shutdown = true 2024-04-20T21:16:47.514 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug op order = true 2024-04-20T21:16:47.514 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug verify stray on activate = true 2024-04-20T21:16:47.514 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug trim objects = true 2024-04-20T21:16:47.514 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:16:47.514 INFO:teuthology.orchestra.run.smithi086.stdout: osd open classes on start = true 2024-04-20T21:16:47.514 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug pg log writeout = true 2024-04-20T21:16:47.514 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:16:47.515 INFO:teuthology.orchestra.run.smithi086.stdout: osd deep scrub update digest min age = 30 2024-04-20T21:16:47.515 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:16:47.515 INFO:teuthology.orchestra.run.smithi086.stdout: osd map max advance = 10 2024-04-20T21:16:47.515 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:16:47.515 INFO:teuthology.orchestra.run.smithi086.stdout: journal zero on create = true 2024-04-20T21:16:47.515 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:16:47.515 INFO:teuthology.orchestra.run.smithi086.stdout: filestore ondisk finisher threads = 3 2024-04-20T21:16:47.515 INFO:teuthology.orchestra.run.smithi086.stdout: filestore apply finisher threads = 3 2024-04-20T21:16:47.515 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:16:47.515 INFO:teuthology.orchestra.run.smithi086.stdout: bdev debug aio = true 2024-04-20T21:16:47.515 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug misdirected ops = true 2024-04-20T21:16:47.515 INFO:teuthology.orchestra.run.smithi086.stdout: bdev async discard = True 2024-04-20T21:16:47.515 INFO:teuthology.orchestra.run.smithi086.stdout: bdev enable discard = True 2024-04-20T21:16:47.515 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore allocator = bitmap 2024-04-20T21:16:47.515 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore block size = 96636764160 2024-04-20T21:16:47.516 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore fsck on mount = True 2024-04-20T21:16:47.516 INFO:teuthology.orchestra.run.smithi086.stdout: debug bluefs = 1/20 2024-04-20T21:16:47.516 INFO:teuthology.orchestra.run.smithi086.stdout: debug bluestore = 1/20 2024-04-20T21:16:47.516 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-20T21:16:47.516 INFO:teuthology.orchestra.run.smithi086.stdout: debug osd = 20 2024-04-20T21:16:47.516 INFO:teuthology.orchestra.run.smithi086.stdout: debug rocksdb = 4/10 2024-04-20T21:16:47.516 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd backfillfull_ratio = 0.85 2024-04-20T21:16:47.516 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd full ratio = 0.9 2024-04-20T21:16:47.516 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd nearfull ratio = 0.8 2024-04-20T21:16:47.516 INFO:teuthology.orchestra.run.smithi086.stdout: osd failsafe full ratio = 0.95 2024-04-20T21:16:47.516 INFO:teuthology.orchestra.run.smithi086.stdout: osd objectstore = bluestore 2024-04-20T21:16:47.516 INFO:teuthology.orchestra.run.smithi086.stdout: osd op complaint time = 180 2024-04-20T21:16:47.516 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:16:47.516 INFO:teuthology.orchestra.run.smithi086.stdout:[mgr] 2024-04-20T21:16:47.516 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-20T21:16:47.516 INFO:teuthology.orchestra.run.smithi086.stdout: debug mgr = 20 2024-04-20T21:16:47.516 INFO:teuthology.orchestra.run.smithi086.stdout: debug mon = 20 2024-04-20T21:16:47.517 INFO:teuthology.orchestra.run.smithi086.stdout: debug auth = 20 2024-04-20T21:16:47.517 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min pgs per osd = 4 2024-04-20T21:16:47.517 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min bytes per osd = 10 2024-04-20T21:16:47.517 INFO:teuthology.orchestra.run.smithi086.stdout: mgr/telemetry/nag = false 2024-04-20T21:16:47.517 INFO:teuthology.orchestra.run.smithi086.stdout: debug client = 20 2024-04-20T21:16:47.517 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:16:47.517 INFO:teuthology.orchestra.run.smithi086.stdout:[mon] 2024-04-20T21:16:47.517 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-20T21:16:47.517 INFO:teuthology.orchestra.run.smithi086.stdout: debug mon = 20 2024-04-20T21:16:47.517 INFO:teuthology.orchestra.run.smithi086.stdout: debug paxos = 20 2024-04-20T21:16:47.517 INFO:teuthology.orchestra.run.smithi086.stdout: debug auth = 20 2024-04-20T21:16:47.517 INFO:teuthology.orchestra.run.smithi086.stdout: mon data avail warn = 5 2024-04-20T21:16:47.517 INFO:teuthology.orchestra.run.smithi086.stdout: mon mgr mkfs grace = 240 2024-04-20T21:16:47.517 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min pgs per osd = 4 2024-04-20T21:16:47.517 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd reporter subtree level = osd 2024-04-20T21:16:47.517 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd prime pg temp = true 2024-04-20T21:16:47.518 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min bytes per osd = 10 2024-04-20T21:16:47.518 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:16:47.518 INFO:teuthology.orchestra.run.smithi086.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-20T21:16:47.518 INFO:teuthology.orchestra.run.smithi086.stdout: auth mon ticket ttl = 660 # 11m 2024-04-20T21:16:47.518 INFO:teuthology.orchestra.run.smithi086.stdout: auth service ticket ttl = 240 # 4m 2024-04-20T21:16:47.518 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:16:47.518 INFO:teuthology.orchestra.run.smithi086.stdout: # don't complain about insecure global_id in the test suite 2024-04-20T21:16:47.518 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-20T21:16:47.518 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-20T21:16:47.518 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:16:47.518 INFO:teuthology.orchestra.run.smithi086.stdout: # 1m isn't quite enough 2024-04-20T21:16:47.518 INFO:teuthology.orchestra.run.smithi086.stdout: mon_down_mkfs_grace = 2m 2024-04-20T21:16:47.518 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:16:47.519 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_filestore_osds = false 2024-04-20T21:16:47.519 INFO:teuthology.orchestra.run.smithi086.stdout: mon down mkfs grace = 300 2024-04-20T21:16:47.519 INFO:teuthology.orchestra.run.smithi086.stdout: mon op complaint time = 120 2024-04-20T21:16:47.519 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:16:47.519 INFO:teuthology.orchestra.run.smithi086.stdout:[client] 2024-04-20T21:16:47.519 INFO:teuthology.orchestra.run.smithi086.stdout: rgw cache enabled = true 2024-04-20T21:16:47.519 INFO:teuthology.orchestra.run.smithi086.stdout: rgw enable ops log = true 2024-04-20T21:16:47.519 INFO:teuthology.orchestra.run.smithi086.stdout: rgw enable usage log = true 2024-04-20T21:16:47.519 INFO:teuthology.orchestra.run.smithi086.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-20T21:16:47.519 INFO:teuthology.orchestra.run.smithi086.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-20T21:16:47.519 INFO:teuthology.orchestra.run.smithi086.stdout: client mount timeout = 600 2024-04-20T21:16:47.519 INFO:teuthology.orchestra.run.smithi086.stdout: debug client = 20 2024-04-20T21:16:47.519 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-20T21:16:47.519 INFO:teuthology.orchestra.run.smithi086.stdout: pid file = "" 2024-04-20T21:16:47.519 INFO:teuthology.orchestra.run.smithi086.stdout: rados mon op timeout = 900 2024-04-20T21:16:47.520 INFO:teuthology.orchestra.run.smithi086.stdout: rados osd op timeout = 900 2024-04-20T21:16:47.520 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.a] 2024-04-20T21:16:47.520 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.b] 2024-04-20T21:16:47.520 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.c] 2024-04-20T21:16:47.520 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.a] 2024-04-20T21:16:47.520 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.c] 2024-04-20T21:16:47.520 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.b] 2024-04-20T21:16:47.520 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.d] 2024-04-20T21:16:47.520 INFO:teuthology.orchestra.run.smithi086.stdout:[mds] 2024-04-20T21:16:47.520 INFO:teuthology.orchestra.run.smithi086.stdout: debug mds = 20 2024-04-20T21:16:47.520 INFO:teuthology.orchestra.run.smithi086.stdout: debug mds balancer = 20 2024-04-20T21:16:47.520 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-20T21:16:47.520 INFO:teuthology.orchestra.run.smithi086.stdout: mds debug frag = True 2024-04-20T21:16:47.520 INFO:teuthology.orchestra.run.smithi086.stdout: mds debug scatterstat = True 2024-04-20T21:16:47.520 INFO:teuthology.orchestra.run.smithi086.stdout: mds op complaint time = 180 2024-04-20T21:16:47.520 INFO:teuthology.orchestra.run.smithi086.stdout: mds verify scatter = True 2024-04-20T21:16:47.520 INFO:teuthology.orchestra.run.smithi086.stdout: osd op complaint time = 180 2024-04-20T21:16:47.521 INFO:teuthology.orchestra.run.smithi086.stdout: rados mon op timeout = 900 2024-04-20T21:16:47.521 INFO:teuthology.orchestra.run.smithi086.stdout: rados osd op timeout = 900 2024-04-20T21:16:47.541 INFO:teuthology.orchestra.run.smithi144.stdout:[global] 2024-04-20T21:16:47.541 INFO:teuthology.orchestra.run.smithi144.stdout: chdir = "" 2024-04-20T21:16:47.541 INFO:teuthology.orchestra.run.smithi144.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-20T21:16:47.542 INFO:teuthology.orchestra.run.smithi144.stdout: auth supported = cephx 2024-04-20T21:16:47.542 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:16:47.542 INFO:teuthology.orchestra.run.smithi144.stdout: filestore xattr use omap = true 2024-04-20T21:16:47.542 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:16:47.542 INFO:teuthology.orchestra.run.smithi144.stdout: mon clock drift allowed = 1.000 2024-04-20T21:16:47.542 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:16:47.542 INFO:teuthology.orchestra.run.smithi144.stdout: osd crush chooseleaf type = 0 2024-04-20T21:16:47.542 INFO:teuthology.orchestra.run.smithi144.stdout: auth debug = true 2024-04-20T21:16:47.542 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:16:47.542 INFO:teuthology.orchestra.run.smithi144.stdout: ms die on old message = true 2024-04-20T21:16:47.542 INFO:teuthology.orchestra.run.smithi144.stdout: ms die on bug = true 2024-04-20T21:16:47.542 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:16:47.542 INFO:teuthology.orchestra.run.smithi144.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-20T21:16:47.542 INFO:teuthology.orchestra.run.smithi144.stdout: mon pg warn max object skew = 0 2024-04-20T21:16:47.542 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:16:47.543 INFO:teuthology.orchestra.run.smithi144.stdout: # disable pg_autoscaler by default for new pools 2024-04-20T21:16:47.543 INFO:teuthology.orchestra.run.smithi144.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-20T21:16:47.543 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:16:47.543 INFO:teuthology.orchestra.run.smithi144.stdout: osd pool default size = 2 2024-04-20T21:16:47.543 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:16:47.543 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd allow primary affinity = true 2024-04-20T21:16:47.543 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd allow pg remap = true 2024-04-20T21:16:47.543 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on legacy crush tunables = false 2024-04-20T21:16:47.543 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on crush straw calc version zero = false 2024-04-20T21:16:47.543 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on no sortbitwise = false 2024-04-20T21:16:47.543 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on osd down out interval zero = false 2024-04-20T21:16:47.543 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on too few osds = false 2024-04-20T21:16:47.543 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-20T21:16:47.543 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-20T21:16:47.543 INFO:teuthology.orchestra.run.smithi144.stdout: mon_allow_pool_size_one = true 2024-04-20T21:16:47.543 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:16:47.544 INFO:teuthology.orchestra.run.smithi144.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-20T21:16:47.544 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:16:47.544 INFO:teuthology.orchestra.run.smithi144.stdout: osd default data pool replay window = 5 2024-04-20T21:16:47.544 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:16:47.544 INFO:teuthology.orchestra.run.smithi144.stdout: mon allow pool delete = true 2024-04-20T21:16:47.544 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:16:47.544 INFO:teuthology.orchestra.run.smithi144.stdout: mon cluster log file level = debug 2024-04-20T21:16:47.544 INFO:teuthology.orchestra.run.smithi144.stdout: debug asserts on shutdown = true 2024-04-20T21:16:47.544 INFO:teuthology.orchestra.run.smithi144.stdout: mon health detail to clog = false 2024-04-20T21:16:47.544 INFO:teuthology.orchestra.run.smithi144.stdout: mon host = "172.21.15.86,172.21.15.144,[v2:172.21.15.144:3301,v1:172.21.15.144:6790]" 2024-04-20T21:16:47.544 INFO:teuthology.orchestra.run.smithi144.stdout: fsid = 9c2dec1d-d7ae-4a83-a812-79922808e278 2024-04-20T21:16:47.544 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:16:47.544 INFO:teuthology.orchestra.run.smithi144.stdout:[osd] 2024-04-20T21:16:47.544 INFO:teuthology.orchestra.run.smithi144.stdout: osd journal size = 100 2024-04-20T21:16:47.544 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:16:47.544 INFO:teuthology.orchestra.run.smithi144.stdout: osd scrub load threshold = 5.0 2024-04-20T21:16:47.544 INFO:teuthology.orchestra.run.smithi144.stdout: osd scrub max interval = 600 2024-04-20T21:16:47.545 INFO:teuthology.orchestra.run.smithi144.stdout: osd mclock profile = high_recovery_ops 2024-04-20T21:16:47.545 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:16:47.545 INFO:teuthology.orchestra.run.smithi144.stdout: osd recover clone overlap = true 2024-04-20T21:16:47.545 INFO:teuthology.orchestra.run.smithi144.stdout: osd recovery max chunk = 1048576 2024-04-20T21:16:47.545 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:16:47.545 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug shutdown = true 2024-04-20T21:16:47.545 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug op order = true 2024-04-20T21:16:47.545 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug verify stray on activate = true 2024-04-20T21:16:47.545 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug trim objects = true 2024-04-20T21:16:47.545 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:16:47.545 INFO:teuthology.orchestra.run.smithi144.stdout: osd open classes on start = true 2024-04-20T21:16:47.545 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug pg log writeout = true 2024-04-20T21:16:47.545 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:16:47.545 INFO:teuthology.orchestra.run.smithi144.stdout: osd deep scrub update digest min age = 30 2024-04-20T21:16:47.545 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:16:47.546 INFO:teuthology.orchestra.run.smithi144.stdout: osd map max advance = 10 2024-04-20T21:16:47.546 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:16:47.546 INFO:teuthology.orchestra.run.smithi144.stdout: journal zero on create = true 2024-04-20T21:16:47.546 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:16:47.546 INFO:teuthology.orchestra.run.smithi144.stdout: filestore ondisk finisher threads = 3 2024-04-20T21:16:47.546 INFO:teuthology.orchestra.run.smithi144.stdout: filestore apply finisher threads = 3 2024-04-20T21:16:47.546 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:16:47.546 INFO:teuthology.orchestra.run.smithi144.stdout: bdev debug aio = true 2024-04-20T21:16:47.546 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug misdirected ops = true 2024-04-20T21:16:47.546 INFO:teuthology.orchestra.run.smithi144.stdout: bdev async discard = True 2024-04-20T21:16:47.546 INFO:teuthology.orchestra.run.smithi144.stdout: bdev enable discard = True 2024-04-20T21:16:47.546 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore allocator = bitmap 2024-04-20T21:16:47.546 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore block size = 96636764160 2024-04-20T21:16:47.547 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore fsck on mount = True 2024-04-20T21:16:47.547 INFO:teuthology.orchestra.run.smithi144.stdout: debug bluefs = 1/20 2024-04-20T21:16:47.547 INFO:teuthology.orchestra.run.smithi144.stdout: debug bluestore = 1/20 2024-04-20T21:16:47.547 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-04-20T21:16:47.547 INFO:teuthology.orchestra.run.smithi144.stdout: debug osd = 20 2024-04-20T21:16:47.547 INFO:teuthology.orchestra.run.smithi144.stdout: debug rocksdb = 4/10 2024-04-20T21:16:47.547 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd backfillfull_ratio = 0.85 2024-04-20T21:16:47.547 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd full ratio = 0.9 2024-04-20T21:16:47.547 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd nearfull ratio = 0.8 2024-04-20T21:16:47.547 INFO:teuthology.orchestra.run.smithi144.stdout: osd failsafe full ratio = 0.95 2024-04-20T21:16:47.547 INFO:teuthology.orchestra.run.smithi144.stdout: osd objectstore = bluestore 2024-04-20T21:16:47.547 INFO:teuthology.orchestra.run.smithi144.stdout: osd op complaint time = 180 2024-04-20T21:16:47.547 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:16:47.547 INFO:teuthology.orchestra.run.smithi144.stdout:[mgr] 2024-04-20T21:16:47.547 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-04-20T21:16:47.547 INFO:teuthology.orchestra.run.smithi144.stdout: debug mgr = 20 2024-04-20T21:16:47.548 INFO:teuthology.orchestra.run.smithi144.stdout: debug mon = 20 2024-04-20T21:16:47.548 INFO:teuthology.orchestra.run.smithi144.stdout: debug auth = 20 2024-04-20T21:16:47.548 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min pgs per osd = 4 2024-04-20T21:16:47.548 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min bytes per osd = 10 2024-04-20T21:16:47.548 INFO:teuthology.orchestra.run.smithi144.stdout: mgr/telemetry/nag = false 2024-04-20T21:16:47.548 INFO:teuthology.orchestra.run.smithi144.stdout: debug client = 20 2024-04-20T21:16:47.548 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:16:47.548 INFO:teuthology.orchestra.run.smithi144.stdout:[mon] 2024-04-20T21:16:47.548 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-04-20T21:16:47.548 INFO:teuthology.orchestra.run.smithi144.stdout: debug mon = 20 2024-04-20T21:16:47.548 INFO:teuthology.orchestra.run.smithi144.stdout: debug paxos = 20 2024-04-20T21:16:47.548 INFO:teuthology.orchestra.run.smithi144.stdout: debug auth = 20 2024-04-20T21:16:47.548 INFO:teuthology.orchestra.run.smithi144.stdout: mon data avail warn = 5 2024-04-20T21:16:47.548 INFO:teuthology.orchestra.run.smithi144.stdout: mon mgr mkfs grace = 240 2024-04-20T21:16:47.549 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min pgs per osd = 4 2024-04-20T21:16:47.549 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd reporter subtree level = osd 2024-04-20T21:16:47.549 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd prime pg temp = true 2024-04-20T21:16:47.549 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min bytes per osd = 10 2024-04-20T21:16:47.549 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:16:47.549 INFO:teuthology.orchestra.run.smithi144.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-20T21:16:47.549 INFO:teuthology.orchestra.run.smithi144.stdout: auth mon ticket ttl = 660 # 11m 2024-04-20T21:16:47.549 INFO:teuthology.orchestra.run.smithi144.stdout: auth service ticket ttl = 240 # 4m 2024-04-20T21:16:47.549 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:16:47.549 INFO:teuthology.orchestra.run.smithi144.stdout: # don't complain about insecure global_id in the test suite 2024-04-20T21:16:47.549 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-20T21:16:47.549 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-20T21:16:47.549 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:16:47.549 INFO:teuthology.orchestra.run.smithi144.stdout: # 1m isn't quite enough 2024-04-20T21:16:47.549 INFO:teuthology.orchestra.run.smithi144.stdout: mon_down_mkfs_grace = 2m 2024-04-20T21:16:47.549 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:16:47.549 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_filestore_osds = false 2024-04-20T21:16:47.550 INFO:teuthology.orchestra.run.smithi144.stdout: mon down mkfs grace = 300 2024-04-20T21:16:47.550 INFO:teuthology.orchestra.run.smithi144.stdout: mon op complaint time = 120 2024-04-20T21:16:47.550 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:16:47.550 INFO:teuthology.orchestra.run.smithi144.stdout:[client] 2024-04-20T21:16:47.550 INFO:teuthology.orchestra.run.smithi144.stdout: rgw cache enabled = true 2024-04-20T21:16:47.550 INFO:teuthology.orchestra.run.smithi144.stdout: rgw enable ops log = true 2024-04-20T21:16:47.550 INFO:teuthology.orchestra.run.smithi144.stdout: rgw enable usage log = true 2024-04-20T21:16:47.550 INFO:teuthology.orchestra.run.smithi144.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-20T21:16:47.550 INFO:teuthology.orchestra.run.smithi144.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-20T21:16:47.550 INFO:teuthology.orchestra.run.smithi144.stdout: client mount timeout = 600 2024-04-20T21:16:47.550 INFO:teuthology.orchestra.run.smithi144.stdout: debug client = 20 2024-04-20T21:16:47.550 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-04-20T21:16:47.550 INFO:teuthology.orchestra.run.smithi144.stdout: pid file = "" 2024-04-20T21:16:47.550 INFO:teuthology.orchestra.run.smithi144.stdout: rados mon op timeout = 900 2024-04-20T21:16:47.550 INFO:teuthology.orchestra.run.smithi144.stdout: rados osd op timeout = 900 2024-04-20T21:16:47.551 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.a] 2024-04-20T21:16:47.551 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.b] 2024-04-20T21:16:47.551 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.c] 2024-04-20T21:16:47.551 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.a] 2024-04-20T21:16:47.551 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.c] 2024-04-20T21:16:47.551 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.b] 2024-04-20T21:16:47.551 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.d] 2024-04-20T21:16:47.551 INFO:teuthology.orchestra.run.smithi144.stdout:[mds] 2024-04-20T21:16:47.551 INFO:teuthology.orchestra.run.smithi144.stdout: debug mds = 20 2024-04-20T21:16:47.551 INFO:teuthology.orchestra.run.smithi144.stdout: debug mds balancer = 20 2024-04-20T21:16:47.551 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-04-20T21:16:47.551 INFO:teuthology.orchestra.run.smithi144.stdout: mds debug frag = True 2024-04-20T21:16:47.551 INFO:teuthology.orchestra.run.smithi144.stdout: mds debug scatterstat = True 2024-04-20T21:16:47.551 INFO:teuthology.orchestra.run.smithi144.stdout: mds op complaint time = 180 2024-04-20T21:16:47.551 INFO:teuthology.orchestra.run.smithi144.stdout: mds verify scatter = True 2024-04-20T21:16:47.551 INFO:teuthology.orchestra.run.smithi144.stdout: osd op complaint time = 180 2024-04-20T21:16:47.552 INFO:teuthology.orchestra.run.smithi144.stdout: rados mon op timeout = 900 2024-04-20T21:16:47.552 INFO:teuthology.orchestra.run.smithi144.stdout: rados osd op timeout = 900 2024-04-20T21:16:47.574 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-20T21:16:47.574 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-20T21:16:47.687 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-20T21:16:47.687 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:16:47.687 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-20T21:16:47.725 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:16:47.725 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-20T21:16:47.807 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi086.front.sepia.ceph.com 2024-04-20T21:16:47.807 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:16:47.807 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-20T21:16:47.807 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-20T21:16:47.965 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:16:47.965 DEBUG:teuthology.orchestra.run.smithi086:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-20T21:16:48.057 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi144.front.sepia.ceph.com 2024-04-20T21:16:48.057 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:16:48.057 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-20T21:16:48.057 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-20T21:16:48.166 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:16:48.166 DEBUG:teuthology.orchestra.run.smithi144:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-20T21:16:48.259 INFO:tasks.ceph:Setting up mon nodes... 2024-04-20T21:16:48.260 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-20T21:16:48.260 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-20T21:16:48.415 INFO:teuthology.orchestra.run.smithi086.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-20T21:16:48.424 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-20T21:16:48.588 INFO:teuthology.orchestra.run.smithi144.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-20T21:16:48.598 INFO:tasks.ceph:Setting up mds nodes... 2024-04-20T21:16:48.598 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-20T21:16:48.765 INFO:teuthology.orchestra.run.smithi086.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-20T21:16:48.775 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-20T21:16:48.856 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-20T21:16:48.996 INFO:teuthology.orchestra.run.smithi086.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-20T21:16:49.005 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-20T21:16:49.079 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-20T21:16:49.232 INFO:teuthology.orchestra.run.smithi144.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-20T21:16:49.240 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-20T21:16:49.322 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-20T21:16:49.482 INFO:teuthology.orchestra.run.smithi144.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-20T21:16:49.491 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-20T21:16:49.578 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-20T21:16:49.579 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-20T21:16:49.693 INFO:teuthology.orchestra.run.smithi086.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-20T21:16:49.734 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-20T21:16:49.842 INFO:teuthology.orchestra.run.smithi086.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-20T21:16:49.877 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-20T21:16:49.992 INFO:teuthology.orchestra.run.smithi144.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-20T21:16:50.026 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-20T21:16:50.133 INFO:teuthology.orchestra.run.smithi144.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-20T21:16:50.181 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-20T21:16:50.181 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi086.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi144.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-20T21:16:50.181 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-20T21:16:50.260 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-20T21:16:50.261 INFO:tasks.ceph:role: osd.0 2024-04-20T21:16:50.261 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi086.front.sepia.ceph.com 2024-04-20T21:16:50.261 DEBUG:teuthology.orchestra.run.smithi086:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-20T21:16:50.333 INFO:teuthology.orchestra.run.smithi086.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T21:16:50.334 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T21:16:50.334 INFO:teuthology.orchestra.run.smithi086.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T21:16:50.334 INFO:teuthology.orchestra.run.smithi086.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-20T21:16:50.334 INFO:teuthology.orchestra.run.smithi086.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T21:16:50.334 INFO:teuthology.orchestra.run.smithi086.stdout: = sunit=0 swidth=0 blks 2024-04-20T21:16:50.334 INFO:teuthology.orchestra.run.smithi086.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T21:16:50.334 INFO:teuthology.orchestra.run.smithi086.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-20T21:16:50.334 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T21:16:50.334 INFO:teuthology.orchestra.run.smithi086.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T21:16:50.743 INFO:teuthology.orchestra.run.smithi086.stdout:Discarding blocks...Done. 2024-04-20T21:16:50.751 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi086.front.sepia.ceph.com -o noatime 2024-04-20T21:16:50.751 DEBUG:teuthology.orchestra.run.smithi086:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-20T21:16:50.915 DEBUG:teuthology.orchestra.run.smithi086:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-20T21:16:51.010 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-20T21:16:51.089 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-20T21:16:51.089 INFO:tasks.ceph:role: osd.1 2024-04-20T21:16:51.089 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi086.front.sepia.ceph.com 2024-04-20T21:16:51.089 DEBUG:teuthology.orchestra.run.smithi086:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-20T21:16:51.176 INFO:teuthology.orchestra.run.smithi086.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T21:16:51.176 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T21:16:51.176 INFO:teuthology.orchestra.run.smithi086.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T21:16:51.177 INFO:teuthology.orchestra.run.smithi086.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-20T21:16:51.177 INFO:teuthology.orchestra.run.smithi086.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T21:16:51.177 INFO:teuthology.orchestra.run.smithi086.stdout: = sunit=0 swidth=0 blks 2024-04-20T21:16:51.177 INFO:teuthology.orchestra.run.smithi086.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T21:16:51.177 INFO:teuthology.orchestra.run.smithi086.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-20T21:16:51.177 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T21:16:51.177 INFO:teuthology.orchestra.run.smithi086.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T21:16:51.535 INFO:teuthology.orchestra.run.smithi086.stdout:Discarding blocks...Done. 2024-04-20T21:16:51.540 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi086.front.sepia.ceph.com -o noatime 2024-04-20T21:16:51.540 DEBUG:teuthology.orchestra.run.smithi086:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-20T21:16:51.648 DEBUG:teuthology.orchestra.run.smithi086:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-20T21:16:51.728 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-20T21:16:51.811 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-20T21:16:51.811 INFO:tasks.ceph:role: osd.2 2024-04-20T21:16:51.811 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi086.front.sepia.ceph.com 2024-04-20T21:16:51.811 DEBUG:teuthology.orchestra.run.smithi086:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-20T21:16:51.899 INFO:teuthology.orchestra.run.smithi086.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T21:16:51.899 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T21:16:51.899 INFO:teuthology.orchestra.run.smithi086.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T21:16:51.899 INFO:teuthology.orchestra.run.smithi086.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-20T21:16:51.899 INFO:teuthology.orchestra.run.smithi086.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T21:16:51.899 INFO:teuthology.orchestra.run.smithi086.stdout: = sunit=0 swidth=0 blks 2024-04-20T21:16:51.899 INFO:teuthology.orchestra.run.smithi086.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T21:16:51.899 INFO:teuthology.orchestra.run.smithi086.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-20T21:16:51.899 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T21:16:51.899 INFO:teuthology.orchestra.run.smithi086.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T21:16:52.294 INFO:teuthology.orchestra.run.smithi086.stdout:Discarding blocks...Done. 2024-04-20T21:16:52.300 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi086.front.sepia.ceph.com -o noatime 2024-04-20T21:16:52.300 DEBUG:teuthology.orchestra.run.smithi086:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-20T21:16:52.459 DEBUG:teuthology.orchestra.run.smithi086:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-20T21:16:52.551 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-20T21:16:52.630 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-20T21:16:52.630 INFO:tasks.ceph:role: osd.3 2024-04-20T21:16:52.631 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi086.front.sepia.ceph.com 2024-04-20T21:16:52.631 DEBUG:teuthology.orchestra.run.smithi086:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-20T21:16:52.711 INFO:teuthology.orchestra.run.smithi086.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T21:16:52.712 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T21:16:52.712 INFO:teuthology.orchestra.run.smithi086.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T21:16:52.712 INFO:teuthology.orchestra.run.smithi086.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-20T21:16:52.712 INFO:teuthology.orchestra.run.smithi086.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T21:16:52.712 INFO:teuthology.orchestra.run.smithi086.stdout: = sunit=0 swidth=0 blks 2024-04-20T21:16:52.712 INFO:teuthology.orchestra.run.smithi086.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T21:16:52.712 INFO:teuthology.orchestra.run.smithi086.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-20T21:16:52.712 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T21:16:52.712 INFO:teuthology.orchestra.run.smithi086.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T21:16:53.129 INFO:teuthology.orchestra.run.smithi086.stdout:Discarding blocks...Done. 2024-04-20T21:16:53.136 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi086.front.sepia.ceph.com -o noatime 2024-04-20T21:16:53.136 DEBUG:teuthology.orchestra.run.smithi086:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-20T21:16:53.284 DEBUG:teuthology.orchestra.run.smithi086:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-20T21:16:53.369 DEBUG:teuthology.orchestra.run.smithi086:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T21:16:53.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:16:53.508+0000 7f837961f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-20T21:16:53.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:16:53.509+0000 7f837961f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-20T21:16:53.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:16:53.509+0000 7f837961f640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-20T21:16:53.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:16:53.509+0000 7f837961f640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-20T21:16:53.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:16:53.509+0000 7f837961f640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-20T21:16:53.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:16:53.512+0000 7f837961f640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-20T21:16:56.487 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-20T21:16:56.573 DEBUG:teuthology.orchestra.run.smithi086:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T21:16:56.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:16:56.702+0000 7fe303ea4640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-20T21:16:56.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:16:56.703+0000 7fe303ea4640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-20T21:16:56.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:16:56.703+0000 7fe303ea4640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-20T21:16:56.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:16:56.703+0000 7fe303ea4640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-20T21:16:56.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:16:56.703+0000 7fe303ea4640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-20T21:16:56.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:16:56.706+0000 7fe303ea4640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-20T21:16:59.682 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-20T21:16:59.771 DEBUG:teuthology.orchestra.run.smithi086:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T21:16:59.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:16:59.901+0000 7fe13f225640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-20T21:16:59.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:16:59.901+0000 7fe13f225640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-20T21:16:59.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:16:59.901+0000 7fe13f225640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-20T21:16:59.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:16:59.902+0000 7fe13f225640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-20T21:16:59.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:16:59.902+0000 7fe13f225640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-20T21:16:59.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:16:59.904+0000 7fe13f225640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-20T21:17:03.118 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-20T21:17:03.201 DEBUG:teuthology.orchestra.run.smithi086:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T21:17:03.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:03.330+0000 7fec6ce0c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-20T21:17:03.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:03.330+0000 7fec6ce0c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-20T21:17:03.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:03.330+0000 7fec6ce0c640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-20T21:17:03.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:03.331+0000 7fec6ce0c640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-20T21:17:03.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:03.331+0000 7fec6ce0c640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-20T21:17:03.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:03.333+0000 7fec6ce0c640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-20T21:17:06.310 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-20T21:17:06.397 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-20T21:17:06.486 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-20T21:17:06.486 INFO:tasks.ceph:role: osd.4 2024-04-20T21:17:06.487 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi144.front.sepia.ceph.com 2024-04-20T21:17:06.487 DEBUG:teuthology.orchestra.run.smithi144:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-20T21:17:06.576 INFO:teuthology.orchestra.run.smithi144.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T21:17:06.576 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T21:17:06.576 INFO:teuthology.orchestra.run.smithi144.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T21:17:06.576 INFO:teuthology.orchestra.run.smithi144.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-20T21:17:06.576 INFO:teuthology.orchestra.run.smithi144.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T21:17:06.576 INFO:teuthology.orchestra.run.smithi144.stdout: = sunit=0 swidth=0 blks 2024-04-20T21:17:06.576 INFO:teuthology.orchestra.run.smithi144.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T21:17:06.576 INFO:teuthology.orchestra.run.smithi144.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-20T21:17:06.576 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T21:17:06.576 INFO:teuthology.orchestra.run.smithi144.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T21:17:06.995 INFO:teuthology.orchestra.run.smithi144.stdout:Discarding blocks...Done. 2024-04-20T21:17:07.003 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi144.front.sepia.ceph.com -o noatime 2024-04-20T21:17:07.003 DEBUG:teuthology.orchestra.run.smithi144:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-20T21:17:07.154 DEBUG:teuthology.orchestra.run.smithi144:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-20T21:17:07.247 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-20T21:17:07.331 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-20T21:17:07.332 INFO:tasks.ceph:role: osd.5 2024-04-20T21:17:07.332 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi144.front.sepia.ceph.com 2024-04-20T21:17:07.332 DEBUG:teuthology.orchestra.run.smithi144:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-20T21:17:07.413 INFO:teuthology.orchestra.run.smithi144.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T21:17:07.414 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T21:17:07.414 INFO:teuthology.orchestra.run.smithi144.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T21:17:07.414 INFO:teuthology.orchestra.run.smithi144.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-20T21:17:07.414 INFO:teuthology.orchestra.run.smithi144.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T21:17:07.414 INFO:teuthology.orchestra.run.smithi144.stdout: = sunit=0 swidth=0 blks 2024-04-20T21:17:07.414 INFO:teuthology.orchestra.run.smithi144.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T21:17:07.414 INFO:teuthology.orchestra.run.smithi144.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-20T21:17:07.414 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T21:17:07.414 INFO:teuthology.orchestra.run.smithi144.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T21:17:07.770 INFO:teuthology.orchestra.run.smithi144.stdout:Discarding blocks...Done. 2024-04-20T21:17:07.775 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi144.front.sepia.ceph.com -o noatime 2024-04-20T21:17:07.775 DEBUG:teuthology.orchestra.run.smithi144:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-20T21:17:07.890 DEBUG:teuthology.orchestra.run.smithi144:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-20T21:17:07.978 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-20T21:17:08.056 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-20T21:17:08.057 INFO:tasks.ceph:role: osd.6 2024-04-20T21:17:08.057 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi144.front.sepia.ceph.com 2024-04-20T21:17:08.057 DEBUG:teuthology.orchestra.run.smithi144:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-20T21:17:08.141 INFO:teuthology.orchestra.run.smithi144.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T21:17:08.141 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T21:17:08.141 INFO:teuthology.orchestra.run.smithi144.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T21:17:08.141 INFO:teuthology.orchestra.run.smithi144.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-20T21:17:08.141 INFO:teuthology.orchestra.run.smithi144.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T21:17:08.141 INFO:teuthology.orchestra.run.smithi144.stdout: = sunit=0 swidth=0 blks 2024-04-20T21:17:08.141 INFO:teuthology.orchestra.run.smithi144.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T21:17:08.141 INFO:teuthology.orchestra.run.smithi144.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-20T21:17:08.141 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T21:17:08.141 INFO:teuthology.orchestra.run.smithi144.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T21:17:08.538 INFO:teuthology.orchestra.run.smithi144.stdout:Discarding blocks...Done. 2024-04-20T21:17:08.544 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi144.front.sepia.ceph.com -o noatime 2024-04-20T21:17:08.544 DEBUG:teuthology.orchestra.run.smithi144:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-20T21:17:08.690 DEBUG:teuthology.orchestra.run.smithi144:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-20T21:17:08.772 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-20T21:17:08.847 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-20T21:17:08.847 INFO:tasks.ceph:role: osd.7 2024-04-20T21:17:08.847 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi144.front.sepia.ceph.com 2024-04-20T21:17:08.847 DEBUG:teuthology.orchestra.run.smithi144:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-20T21:17:08.929 INFO:teuthology.orchestra.run.smithi144.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T21:17:08.930 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T21:17:08.930 INFO:teuthology.orchestra.run.smithi144.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T21:17:08.930 INFO:teuthology.orchestra.run.smithi144.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-20T21:17:08.930 INFO:teuthology.orchestra.run.smithi144.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T21:17:08.930 INFO:teuthology.orchestra.run.smithi144.stdout: = sunit=0 swidth=0 blks 2024-04-20T21:17:08.930 INFO:teuthology.orchestra.run.smithi144.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T21:17:08.930 INFO:teuthology.orchestra.run.smithi144.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-20T21:17:08.930 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T21:17:08.930 INFO:teuthology.orchestra.run.smithi144.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T21:17:09.336 INFO:teuthology.orchestra.run.smithi144.stdout:Discarding blocks...Done. 2024-04-20T21:17:09.343 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi144.front.sepia.ceph.com -o noatime 2024-04-20T21:17:09.343 DEBUG:teuthology.orchestra.run.smithi144:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-20T21:17:09.485 DEBUG:teuthology.orchestra.run.smithi144:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-20T21:17:09.567 DEBUG:teuthology.orchestra.run.smithi144:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T21:17:09.694 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:09.694+0000 7fc7a80a4640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-20T21:17:09.694 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:09.694+0000 7fc7a80a4640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-20T21:17:09.695 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:09.694+0000 7fc7a80a4640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-20T21:17:09.695 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:09.694+0000 7fc7a80a4640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-20T21:17:09.695 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:09.695+0000 7fc7a80a4640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-20T21:17:09.698 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:09.698+0000 7fc7a80a4640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-20T21:17:12.687 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-20T21:17:12.773 DEBUG:teuthology.orchestra.run.smithi144:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T21:17:12.901 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:12.902+0000 7f41cb226640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-20T21:17:12.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:12.902+0000 7f41cb226640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-20T21:17:12.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:12.902+0000 7f41cb226640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-20T21:17:12.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:12.902+0000 7f41cb226640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-20T21:17:12.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:12.902+0000 7f41cb226640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-20T21:17:12.904 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:12.905+0000 7f41cb226640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-20T21:17:14.981 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:17:14.989 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:17:15.866 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-20T21:17:15.951 DEBUG:teuthology.orchestra.run.smithi144:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T21:17:16.073 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:16.074+0000 7f09f8b99640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-20T21:17:16.074 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:16.074+0000 7f09f8b99640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-20T21:17:16.074 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:16.074+0000 7f09f8b99640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-20T21:17:16.074 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:16.074+0000 7f09f8b99640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-20T21:17:16.074 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:16.074+0000 7f09f8b99640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-20T21:17:16.076 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:16.077+0000 7f09f8b99640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-20T21:17:19.037 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-20T21:17:19.118 DEBUG:teuthology.orchestra.run.smithi144:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T21:17:19.247 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:19.247+0000 7fcc8e621640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-20T21:17:19.247 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:19.247+0000 7fcc8e621640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-20T21:17:19.248 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:19.248+0000 7fcc8e621640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-20T21:17:19.248 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:19.248+0000 7fcc8e621640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-20T21:17:19.248 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:19.248+0000 7fcc8e621640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-20T21:17:19.250 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:19.250+0000 7fcc8e621640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-20T21:17:22.218 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-20T21:17:22.304 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-20T21:17:22.304 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:17:22.304 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-20T21:17:22.392 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:17:22.392 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-20T21:17:22.476 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:17:22.476 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-20T21:17:22.559 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:17:22.559 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-20T21:17:22.687 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:17:22.687 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-20T21:17:22.765 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:17:22.765 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-20T21:17:22.846 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:17:22.846 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-20T21:17:22.928 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:17:22.929 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-20T21:17:23.015 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:17:23.015 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-20T21:17:23.095 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:17:23.095 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-20T21:17:23.174 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:17:23.174 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-20T21:17:23.256 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:17:23.257 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-20T21:17:23.336 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:17:23.336 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-20T21:17:23.410 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:17:23.410 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-20T21:17:23.491 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:17:23.491 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-20T21:17:23.548 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:17:23.548 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-20T21:17:23.600 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:17:23.600 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-20T21:17:23.658 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:17:23.658 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-20T21:17:23.708 INFO:tasks.ceph:Adding keys to all mons... 2024-04-20T21:17:23.709 DEBUG:teuthology.orchestra.run.smithi086:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-20T21:17:23.716 DEBUG:teuthology.orchestra.run.smithi144:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-20T21:17:23.792 INFO:teuthology.orchestra.run.smithi086.stdout:[mgr.y] 2024-04-20T21:17:23.792 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQDAMCRmSn3VGBAA0eu3R53f3gFgscWFMRkDzg== 2024-04-20T21:17:23.792 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.a] 2024-04-20T21:17:23.792 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQDAMCRmOpS/LRAAPZUUQNf6TyeJ2tF7YPnIKw== 2024-04-20T21:17:23.792 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.c] 2024-04-20T21:17:23.792 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQDAMCRmPHOFOxAA4tUsw/n6oSKrNmzPD+NKyw== 2024-04-20T21:17:23.792 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.0] 2024-04-20T21:17:23.792 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQDFMCRm4w9UHhAA56y4dJyDASI+IoZv2XrEoQ== 2024-04-20T21:17:23.793 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.1] 2024-04-20T21:17:23.793 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQDIMCRmFo7xKRAAou5B04+J1gZtTzW/Mo+nRQ== 2024-04-20T21:17:23.793 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.2] 2024-04-20T21:17:23.793 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQDLMCRm7/jFNRAA4KcRsapPFLaKf0NaH5MApg== 2024-04-20T21:17:23.793 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.3] 2024-04-20T21:17:23.793 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQDPMCRmUTPMExAA0/dubb4ehsWIwZPS/ao68w== 2024-04-20T21:17:23.793 INFO:teuthology.orchestra.run.smithi086.stdout:[mgr.x] 2024-04-20T21:17:23.793 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQDAMCRmvTgjIxAACeSV8gNkL5LV3xzPS6VPeg== 2024-04-20T21:17:23.793 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.b] 2024-04-20T21:17:23.793 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQDBMCRmCJLpDRAA/JiLRSAlQRNMpDC+qDSr0g== 2024-04-20T21:17:23.793 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.d] 2024-04-20T21:17:23.793 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQDBMCRmZnPXHBAA0jXwBmGehaO7qens4boRIQ== 2024-04-20T21:17:23.793 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.4] 2024-04-20T21:17:23.793 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQDVMCRmB4F2KRAA8Xj0DfH5Ou8TUrO8X/BQyw== 2024-04-20T21:17:23.794 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.5] 2024-04-20T21:17:23.794 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQDYMCRmSOvYNRAAODYXmRHMGPrCAxcHwlS7Ww== 2024-04-20T21:17:23.794 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.6] 2024-04-20T21:17:23.794 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQDcMCRmENJ9BBAAi/KBUhbYGNXEyiSjSHavWQ== 2024-04-20T21:17:23.794 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.7] 2024-04-20T21:17:23.794 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQDfMCRmfTHTDhAALL9GZBSGjWYrhgFw/YI6IA== 2024-04-20T21:17:23.794 INFO:teuthology.orchestra.run.smithi086.stdout:[client.0] 2024-04-20T21:17:23.794 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQDBMCRmHWxyKRAANzhRi4eajp68h/jWkhFMnw== 2024-04-20T21:17:23.794 INFO:teuthology.orchestra.run.smithi086.stdout:[client.2] 2024-04-20T21:17:23.794 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQDBMCRmQAdJMhAAiUmKhPEeuV+UPqQ3gynG9w== 2024-04-20T21:17:23.794 INFO:teuthology.orchestra.run.smithi086.stdout:[client.1] 2024-04-20T21:17:23.794 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQDBMCRmgBJAOxAADJLzO97tjEpKn6kUs8g2QA== 2024-04-20T21:17:23.794 INFO:teuthology.orchestra.run.smithi086.stdout:[client.3] 2024-04-20T21:17:23.795 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQDCMCRm8yMSCBAAFgSVW0XYu7fa40YxHITljQ== 2024-04-20T21:17:23.795 INFO:teuthology.orchestra.run.smithi144.stdout:[mgr.y] 2024-04-20T21:17:23.795 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDAMCRmSn3VGBAA0eu3R53f3gFgscWFMRkDzg== 2024-04-20T21:17:23.795 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.a] 2024-04-20T21:17:23.796 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDAMCRmOpS/LRAAPZUUQNf6TyeJ2tF7YPnIKw== 2024-04-20T21:17:23.796 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.c] 2024-04-20T21:17:23.796 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDAMCRmPHOFOxAA4tUsw/n6oSKrNmzPD+NKyw== 2024-04-20T21:17:23.796 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.0] 2024-04-20T21:17:23.796 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDFMCRm4w9UHhAA56y4dJyDASI+IoZv2XrEoQ== 2024-04-20T21:17:23.796 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.1] 2024-04-20T21:17:23.796 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDIMCRmFo7xKRAAou5B04+J1gZtTzW/Mo+nRQ== 2024-04-20T21:17:23.796 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.2] 2024-04-20T21:17:23.796 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDLMCRm7/jFNRAA4KcRsapPFLaKf0NaH5MApg== 2024-04-20T21:17:23.796 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.3] 2024-04-20T21:17:23.797 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDPMCRmUTPMExAA0/dubb4ehsWIwZPS/ao68w== 2024-04-20T21:17:23.797 INFO:teuthology.orchestra.run.smithi144.stdout:[mgr.x] 2024-04-20T21:17:23.797 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDAMCRmvTgjIxAACeSV8gNkL5LV3xzPS6VPeg== 2024-04-20T21:17:23.797 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.b] 2024-04-20T21:17:23.797 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDBMCRmCJLpDRAA/JiLRSAlQRNMpDC+qDSr0g== 2024-04-20T21:17:23.797 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.d] 2024-04-20T21:17:23.797 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDBMCRmZnPXHBAA0jXwBmGehaO7qens4boRIQ== 2024-04-20T21:17:23.797 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.4] 2024-04-20T21:17:23.797 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDVMCRmB4F2KRAA8Xj0DfH5Ou8TUrO8X/BQyw== 2024-04-20T21:17:23.797 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.5] 2024-04-20T21:17:23.798 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDYMCRmSOvYNRAAODYXmRHMGPrCAxcHwlS7Ww== 2024-04-20T21:17:23.798 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.6] 2024-04-20T21:17:23.798 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDcMCRmENJ9BBAAi/KBUhbYGNXEyiSjSHavWQ== 2024-04-20T21:17:23.798 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.7] 2024-04-20T21:17:23.798 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDfMCRmfTHTDhAALL9GZBSGjWYrhgFw/YI6IA== 2024-04-20T21:17:23.798 INFO:teuthology.orchestra.run.smithi144.stdout:[client.0] 2024-04-20T21:17:23.798 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDBMCRmHWxyKRAANzhRi4eajp68h/jWkhFMnw== 2024-04-20T21:17:23.798 INFO:teuthology.orchestra.run.smithi144.stdout:[client.2] 2024-04-20T21:17:23.798 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDBMCRmQAdJMhAAiUmKhPEeuV+UPqQ3gynG9w== 2024-04-20T21:17:23.798 INFO:teuthology.orchestra.run.smithi144.stdout:[client.1] 2024-04-20T21:17:23.799 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDBMCRmgBJAOxAADJLzO97tjEpKn6kUs8g2QA== 2024-04-20T21:17:23.799 INFO:teuthology.orchestra.run.smithi144.stdout:[client.3] 2024-04-20T21:17:23.799 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQDCMCRm8yMSCBAAFgSVW0XYu7fa40YxHITljQ== 2024-04-20T21:17:23.800 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-20T21:17:23.848 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-20T21:17:23.986 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T21:17:23.993 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T21:17:24.130 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T21:17:24.135 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T21:17:24.269 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T21:17:24.276 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T21:17:24.411 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T21:17:24.417 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T21:17:24.549 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T21:17:24.555 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T21:17:24.681 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T21:17:24.686 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T21:17:24.820 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-20T21:17:24.827 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-20T21:17:24.958 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T21:17:24.963 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T21:17:25.099 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T21:17:25.105 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T21:17:25.239 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T21:17:25.244 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T21:17:25.379 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T21:17:25.385 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T21:17:25.516 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T21:17:25.522 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T21:17:25.655 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T21:17:25.662 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T21:17:25.794 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-20T21:17:25.799 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-20T21:17:25.935 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-20T21:17:25.939 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-20T21:17:26.066 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-20T21:17:26.072 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-20T21:17:26.205 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-20T21:17:26.211 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-20T21:17:26.345 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-20T21:17:26.345 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-20T21:17:26.425 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-20T21:17:26.574 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-20T21:17:26.641 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-20T21:17:26.729 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-20T21:17:26.877 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-20T21:17:26.954 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-20T21:17:27.033 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-20T21:17:27.165 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-20T21:17:27.242 DEBUG:teuthology.orchestra.run.smithi086:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-20T21:17:27.250 DEBUG:teuthology.orchestra.run.smithi144:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-20T21:17:27.299 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-20T21:17:27.299 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-20T21:17:27.299 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-20T21:17:27.304 INFO:tasks.ceph.mon.a:Started 2024-04-20T21:17:27.305 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-20T21:17:27.305 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-20T21:17:27.309 INFO:tasks.ceph.mon.b:Started 2024-04-20T21:17:27.309 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-20T21:17:27.309 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-20T21:17:27.314 INFO:tasks.ceph.mon.c:Started 2024-04-20T21:17:27.314 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-20T21:17:27.314 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-20T21:17:27.314 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-20T21:17:27.318 INFO:tasks.ceph.mgr.y:Started 2024-04-20T21:17:27.319 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-20T21:17:27.319 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-20T21:17:27.323 INFO:tasks.ceph.mgr.x:Started 2024-04-20T21:17:27.323 INFO:tasks.ceph:Setting crush tunables to default 2024-04-20T21:17:27.324 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph osd crush tunables default 2024-04-20T21:17:27.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.493+0000 7f390ac00640 1 Processor -- start 2024-04-20T21:17:27.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.493+0000 7f390ac00640 1 -- start start 2024-04-20T21:17:27.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.495+0000 7f390ac00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f39040b3ba0 0x7f390406eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:27.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.495+0000 7f390ac00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f390406f3f0 0x7f3904071ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:27.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.495+0000 7f390ac00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f390406f800 con 0x7f39040cfc70 2024-04-20T21:17:27.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.495+0000 7f390ac00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f390415c8e0 con 0x7f39040b3ba0 2024-04-20T21:17:27.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.495+0000 7f390ac00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f39040cddb0 con 0x7f390406f3f0 2024-04-20T21:17:27.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.495+0000 7f390a200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f390406f3f0 0x7f3904071ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:27.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.495+0000 7f3908e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f39040b3ba0 0x7f390406eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:27.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.496+0000 7f3908e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f39040b3ba0 0x7f390406eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.86:54908/0 (socket says 172.21.15.86:54908) 2024-04-20T21:17:27.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.496+0000 7f3908e00640 1 -- 172.21.15.86:0/246674038 learned_addr learned my addr 172.21.15.86:0/246674038 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:27.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.496+0000 7f390a200640 1 -- 172.21.15.86:0/246674038 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f390406f3f0 msgr2=0x7f3904071ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T21:17:27.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.496+0000 7f390a200640 1 -- 172.21.15.86:0/246674038 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f390406f3f0 msgr2=0x7f3904071ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T21:17:27.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.496+0000 7f390a200640 1 --2- 172.21.15.86:0/246674038 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f390406f3f0 0x7f3904071ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T21:17:27.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.496+0000 7f3908e00640 1 -- 172.21.15.86:0/246674038 >> v2:172.21.15.144:3300/0 conn(0x7f39040b3ba0 msgr2=0x7f390406eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T21:17:27.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.496+0000 7f3908e00640 1 -- 172.21.15.86:0/246674038 >> v2:172.21.15.144:3300/0 conn(0x7f39040b3ba0 msgr2=0x7f390406eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T21:17:27.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.496+0000 7f3908e00640 1 --2- 172.21.15.86:0/246674038 >> v2:172.21.15.144:3300/0 conn(0x7f39040b3ba0 0x7f390406eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T21:17:27.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.496+0000 7f390a200640 1 --2- 172.21.15.86:0/246674038 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f390406f3f0 0x7f3904071ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:17:27.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.496+0000 7f3908e00640 1 --2- 172.21.15.86:0/246674038 >> v2:172.21.15.144:3300/0 conn(0x7f39040b3ba0 0x7f390406eeb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T21:17:27.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.516+0000 7f3903e00640 1 -- 172.21.15.86:0/246674038 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 212369615 0 0) 0x7f38f8001110 con 0x7f39040cfc70 2024-04-20T21:17:27.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.516+0000 7f3903e00640 1 -- 172.21.15.86:0/246674038 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f38ec005730 con 0x7f39040cfc70 2024-04-20T21:17:27.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.516+0000 7f3903e00640 1 -- 172.21.15.86:0/246674038 <== mon.1 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 63016518 0 0) 0x7f38ec005730 con 0x7f39040cfc70 2024-04-20T21:17:27.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.517+0000 7f3903e00640 1 -- 172.21.15.86:0/246674038 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f390406f3f0 msgr2=0x7f3904071ef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:17:27.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.517+0000 7f3903e00640 1 --2- 172.21.15.86:0/246674038 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f390406f3f0 0x7f3904071ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:27.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.517+0000 7f3903e00640 1 -- 172.21.15.86:0/246674038 >> v2:172.21.15.144:3300/0 conn(0x7f39040b3ba0 msgr2=0x7f390406eeb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:17:27.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.517+0000 7f3903e00640 1 --2- 172.21.15.86:0/246674038 >> v2:172.21.15.144:3300/0 conn(0x7f39040b3ba0 0x7f390406eeb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:27.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.517+0000 7f3903e00640 1 -- 172.21.15.86:0/246674038 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f390415bcd0 con 0x7f39040cfc70 2024-04-20T21:17:27.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.517+0000 7f3903e00640 1 -- 172.21.15.86:0/246674038 <== mon.1 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f38f8001860 con 0x7f39040cfc70 2024-04-20T21:17:27.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.517+0000 7f3903e00640 1 -- 172.21.15.86:0/246674038 >> v1:172.21.15.144:6789/0 conn(0x7f39040cfc70 legacy=0x7f39040cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:27.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.517+0000 7f3903e00640 1 --2- 172.21.15.86:0/246674038 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f390406f3f0 0x7f38ec005c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:27.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.517+0000 7f3903e00640 1 --2- 172.21.15.86:0/246674038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38ec006260 0x7f38ec006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:27.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.517+0000 7f3903e00640 1 --2- 172.21.15.86:0/246674038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38ec006f50 0x7f38ec0073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:27.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.517+0000 7f3903e00640 1 -- 172.21.15.86:0/246674038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f390415c8e0 con 0x7f38ec006260 2024-04-20T21:17:27.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.517+0000 7f3903e00640 1 -- 172.21.15.86:0/246674038 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f39040cddb0 con 0x7f38ec006f50 2024-04-20T21:17:27.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.517+0000 7f3903e00640 1 -- 172.21.15.86:0/246674038 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f38ec00dd60 con 0x7f390406f3f0 2024-04-20T21:17:27.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.517+0000 7f3909800640 1 --2- 172.21.15.86:0/246674038 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f390406f3f0 0x7f38ec005c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:27.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.517+0000 7f390a200640 1 --2- 172.21.15.86:0/246674038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38ec006f50 0x7f38ec0073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:27.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.517+0000 7f3908e00640 1 --2- 172.21.15.86:0/246674038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38ec006260 0x7f38ec006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:27.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.518+0000 7f390a200640 1 -- 172.21.15.86:0/246674038 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f390406f3f0 msgr2=0x7f38ec005c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:27.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.518+0000 7f390a200640 1 --2- 172.21.15.86:0/246674038 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f390406f3f0 0x7f38ec005c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:27.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.518+0000 7f390a200640 1 -- 172.21.15.86:0/246674038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38ec006260 msgr2=0x7f38ec006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:27.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.518+0000 7f390a200640 1 --2- 172.21.15.86:0/246674038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38ec006260 0x7f38ec006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:27.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.518+0000 7f390a200640 1 -- 172.21.15.86:0/246674038 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38ec005730 con 0x7f38ec006f50 2024-04-20T21:17:27.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.518+0000 7f3909800640 1 --2- 172.21.15.86:0/246674038 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f390406f3f0 0x7f38ec005c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:27.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.519+0000 7f390a200640 1 --2- 172.21.15.86:0/246674038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38ec006f50 0x7f38ec0073d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f38fc009610 tx=0x7f38ec007a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51704cba01bab5e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:27.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.519+0000 7f3903e00640 1 -- 172.21.15.86:0/246674038 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38fc039070 con 0x7f38ec006f50 2024-04-20T21:17:27.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.519+0000 7f3903e00640 1 -- 172.21.15.86:0/246674038 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38fc02cc70 con 0x7f38ec006f50 2024-04-20T21:17:27.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.519+0000 7f3903e00640 1 -- 172.21.15.86:0/246674038 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38fc009bd0 con 0x7f38ec006f50 2024-04-20T21:17:27.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.520+0000 7f390ac00640 1 -- 172.21.15.86:0/246674038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38ec006f50 msgr2=0x7f38ec0073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:27.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.520+0000 7f390ac00640 1 --2- 172.21.15.86:0/246674038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38ec006f50 0x7f38ec0073d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f38fc009610 tx=0x7f38ec007a40 comp rx=0 tx=0).stop 2024-04-20T21:17:27.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.520+0000 7f390a200640 1 -- 172.21.15.86:0/246674038 reap_dead start 2024-04-20T21:17:27.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.520+0000 7f390ac00640 1 -- 172.21.15.86:0/246674038 shutdown_connections 2024-04-20T21:17:27.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.520+0000 7f390ac00640 1 -- 172.21.15.86:0/246674038 >> 172.21.15.86:0/246674038 conn(0x7f3904067cf0 msgr2=0x7f390406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:27.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.520+0000 7f390ac00640 1 -- 172.21.15.86:0/246674038 shutdown_connections 2024-04-20T21:17:27.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.520+0000 7f390ac00640 1 -- 172.21.15.86:0/246674038 wait complete. 2024-04-20T21:17:27.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.522+0000 7f390ac00640 1 Processor -- start 2024-04-20T21:17:27.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.522+0000 7f390ac00640 1 -- start start 2024-04-20T21:17:27.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.523+0000 7f390ac00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38ec006f50 0x7f39040af410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:27.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.523+0000 7f390ac00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39040b0eb0 0x7f39040ab8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:27.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.523+0000 7f390ac00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f39040b1310 0x7f39040abdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:27.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.523+0000 7f390ac00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f390415c570 con 0x7f39040b0eb0 2024-04-20T21:17:27.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.523+0000 7f390ac00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f390415c290 con 0x7f38ec006f50 2024-04-20T21:17:27.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.523+0000 7f390ac00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f39040b3710 con 0x7f39040b1310 2024-04-20T21:17:27.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.524+0000 7f3909800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38ec006f50 0x7f39040af410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:27.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.524+0000 7f3908e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39040b0eb0 0x7f39040ab8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:27.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.524+0000 7f390a200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f39040b1310 0x7f39040abdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:27.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.524+0000 7f3909800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38ec006f50 0x7f39040af410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.86:54924/0 (socket says 172.21.15.86:54924) 2024-04-20T21:17:27.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.524+0000 7f3909800640 1 -- 172.21.15.86:0/819620127 learned_addr learned my addr 172.21.15.86:0/819620127 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:27.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.524+0000 7f3908e00640 1 -- 172.21.15.86:0/819620127 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f39040b1310 msgr2=0x7f39040abdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:27.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.524+0000 7f3908e00640 1 --2- 172.21.15.86:0/819620127 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f39040b1310 0x7f39040abdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:27.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.524+0000 7f3908e00640 1 -- 172.21.15.86:0/819620127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38ec006f50 msgr2=0x7f39040af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:27.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.524+0000 7f3908e00640 1 --2- 172.21.15.86:0/819620127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38ec006f50 0x7f39040af410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:27.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.524+0000 7f3908e00640 1 -- 172.21.15.86:0/819620127 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39040ac9c0 con 0x7f39040b0eb0 2024-04-20T21:17:27.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.524+0000 7f3909800640 1 --2- 172.21.15.86:0/819620127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38ec006f50 0x7f39040af410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:27.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.524+0000 7f3908e00640 1 --2- 172.21.15.86:0/819620127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39040b0eb0 0x7f39040ab8b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f38f0010890 tx=0x7f38f0010c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:27.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.525+0000 7f3901600640 1 -- 172.21.15.86:0/819620127 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f001a030 con 0x7f39040b0eb0 2024-04-20T21:17:27.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.525+0000 7f3901600640 1 -- 172.21.15.86:0/819620127 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38f0010e80 con 0x7f39040b0eb0 2024-04-20T21:17:27.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.525+0000 7f3901600640 1 -- 172.21.15.86:0/819620127 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f000f040 con 0x7f39040b0eb0 2024-04-20T21:17:27.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.525+0000 7f390ac00640 1 -- 172.21.15.86:0/819620127 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39040acca0 con 0x7f39040b0eb0 2024-04-20T21:17:27.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.525+0000 7f390ac00640 1 -- 172.21.15.86:0/819620127 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39040ad260 con 0x7f39040b0eb0 2024-04-20T21:17:27.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.525+0000 7f3901600640 1 -- 172.21.15.86:0/819620127 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f38f001d070 con 0x7f39040b0eb0 2024-04-20T21:17:27.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.525+0000 7f3901600640 1 -- 172.21.15.86:0/819620127 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f38f0016760 con 0x7f39040b0eb0 2024-04-20T21:17:27.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.526+0000 7f390ac00640 1 -- 172.21.15.86:0/819620127 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f390404fd90 con 0x7f39040b0eb0 2024-04-20T21:17:27.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.529+0000 7f3901600640 1 -- 172.21.15.86:0/819620127 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f38f0015d90 con 0x7f39040b0eb0 2024-04-20T21:17:27.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.601+0000 7f390ac00640 1 -- 172.21.15.86:0/819620127 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f39040af950 con 0x7f39040b0eb0 2024-04-20T21:17:27.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.604+0000 7f3901600640 1 -- 172.21.15.86:0/819620127 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f38f000ec50 con 0x7f39040b0eb0 2024-04-20T21:17:27.603 INFO:teuthology.orchestra.run.smithi086.stderr:adjusted tunables profile to default 2024-04-20T21:17:27.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.605+0000 7f390ac00640 1 -- 172.21.15.86:0/819620127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39040b0eb0 msgr2=0x7f39040ab8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:27.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.605+0000 7f390ac00640 1 --2- 172.21.15.86:0/819620127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39040b0eb0 0x7f39040ab8b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f38f0010890 tx=0x7f38f0010c90 comp rx=0 tx=0).stop 2024-04-20T21:17:27.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.605+0000 7f390ac00640 1 -- 172.21.15.86:0/819620127 shutdown_connections 2024-04-20T21:17:27.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.605+0000 7f390ac00640 1 --2- 172.21.15.86:0/819620127 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f39040b1310 0x7f39040abdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:27.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.605+0000 7f390ac00640 1 --2- 172.21.15.86:0/819620127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39040b0eb0 0x7f39040ab8b0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:27.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.605+0000 7f390ac00640 1 --2- 172.21.15.86:0/819620127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38ec006f50 0x7f39040af410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:27.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.605+0000 7f390ac00640 1 -- 172.21.15.86:0/819620127 >> 172.21.15.86:0/819620127 conn(0x7f3904067cf0 msgr2=0x7f390415ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:27.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.606+0000 7f390ac00640 1 -- 172.21.15.86:0/819620127 shutdown_connections 2024-04-20T21:17:27.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:27.606+0000 7f390ac00640 1 -- 172.21.15.86:0/819620127 wait complete. 2024-04-20T21:17:27.623 INFO:tasks.ceph:check_enable_crimson: False 2024-04-20T21:17:27.623 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-20T21:17:27.623 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:17:27.623 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-20T21:17:27.689 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:17:27.689 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-20T21:17:27.798 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:17:27.798 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-20T21:17:27.799 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-04-20T21:17:27.799+0000 7fe5f8bc0040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-20T21:17:27.803 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-20T21:17:27.803+0000 7f0a0524b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-20T21:17:27.864 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:17:27.864 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-20T21:17:27.968 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:17:27.968 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-20T21:17:27.978 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-04-20T21:17:27.978+0000 7fe5f8bc0040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-20T21:17:27.994 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-20T21:17:27.994+0000 7f0a0524b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-20T21:17:28.042 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:17:28.042 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-20T21:17:28.046 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-04-20T21:17:28.046+0000 7fe5f8bc0040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-20T21:17:28.060 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-20T21:17:28.061+0000 7f0a0524b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-20T21:17:28.116 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:17:28.116 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-20T21:17:28.229 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:17:28.229 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-20T21:17:28.343 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new 87a2269d-c3c3-4e14-9d35-03ea51d17677 0 2024-04-20T21:17:28.381 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-20T21:17:28.382+0000 7f0a0524b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-20T21:17:28.384 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-04-20T21:17:28.384+0000 7fe5f8bc0040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-20T21:17:28.439 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-20T21:17:28.440+0000 7f0a0524b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-20T21:17:28.441 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-04-20T21:17:28.441+0000 7fe5f8bc0040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-20T21:17:28.567 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.567+0000 7f6eaf200640 1 Processor -- start 2024-04-20T21:17:28.567 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.567+0000 7f6eaf200640 1 -- start start 2024-04-20T21:17:28.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.568+0000 7f6eaf200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6eb015e8e0 0x7f6eb00b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:28.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.568+0000 7f6eaf200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6eb00d1aa0 0x7f6eb00d4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:28.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.568+0000 7f6eaf200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6eb00d4a80 0x7f6eb0162530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:28.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.568+0000 7f6eaf200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f6eb00b4ab0 con 0x7f6eb015e8e0 2024-04-20T21:17:28.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.568+0000 7f6eaf200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f6eb00b4d90 con 0x7f6eb00d4a80 2024-04-20T21:17:28.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.568+0000 7f6eaf200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6eb00b5070 con 0x7f6eb00d1aa0 2024-04-20T21:17:28.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.569+0000 7f6ead400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6eb00d1aa0 0x7f6eb00d4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:28.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.569+0000 7f6ead400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6eb00d1aa0 0x7f6eb00d4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.144:44612/0 (socket says 172.21.15.144:44612) 2024-04-20T21:17:28.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.569+0000 7f6ead400640 1 -- 172.21.15.144:0/3170795027 learned_addr learned my addr 172.21.15.144:0/3170795027 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-04-20T21:17:28.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.569+0000 7f6eae800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6eb00d4a80 0x7f6eb0162530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:28.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.569+0000 7f6eade00640 1 --2- 172.21.15.144:0/3170795027 >> v2:172.21.15.86:3300/0 conn(0x7f6eb015e8e0 0x7f6eb00b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:28.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.569+0000 7f6ead400640 1 -- 172.21.15.144:0/3170795027 >> v2:172.21.15.144:3300/0 conn(0x7f6eb00d4a80 msgr2=0x7f6eb0162530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:28.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.569+0000 7f6ead400640 1 --2- 172.21.15.144:0/3170795027 >> v2:172.21.15.144:3300/0 conn(0x7f6eb00d4a80 0x7f6eb0162530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:28.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.569+0000 7f6ead400640 1 -- 172.21.15.144:0/3170795027 >> v2:172.21.15.86:3300/0 conn(0x7f6eb015e8e0 msgr2=0x7f6eb00b44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:28.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.569+0000 7f6ead400640 1 --2- 172.21.15.144:0/3170795027 >> v2:172.21.15.86:3300/0 conn(0x7f6eb015e8e0 0x7f6eb00b44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:28.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.569+0000 7f6ead400640 1 -- 172.21.15.144:0/3170795027 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eb00b5350 con 0x7f6eb00d1aa0 2024-04-20T21:17:28.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.569+0000 7f6eade00640 1 --2- 172.21.15.144:0/3170795027 >> v2:172.21.15.86:3300/0 conn(0x7f6eb015e8e0 0x7f6eb00b44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:17:28.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.570+0000 7f6ead400640 1 --2- 172.21.15.144:0/3170795027 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6eb00d1aa0 0x7f6eb00d4540 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6eb008db10 tx=0x7f6ea002be40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17d640a18701ad49 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:28.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.570+0000 7f6eaca00640 1 -- 172.21.15.144:0/3170795027 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea002c9e0 con 0x7f6eb00d1aa0 2024-04-20T21:17:28.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.571+0000 7f6eaca00640 1 -- 172.21.15.144:0/3170795027 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ea002cb80 con 0x7f6eb00d1aa0 2024-04-20T21:17:28.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.571+0000 7f6eaca00640 1 -- 172.21.15.144:0/3170795027 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea00343f0 con 0x7f6eb00d1aa0 2024-04-20T21:17:28.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.571+0000 7f6eaf200640 1 -- 172.21.15.144:0/3170795027 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6eb00d1aa0 msgr2=0x7f6eb00d4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:28.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.571+0000 7f6eaf200640 1 --2- 172.21.15.144:0/3170795027 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6eb00d1aa0 0x7f6eb00d4540 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6eb008db10 tx=0x7f6ea002be40 comp rx=0 tx=0).stop 2024-04-20T21:17:28.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.571+0000 7f6eaf200640 1 -- 172.21.15.144:0/3170795027 shutdown_connections 2024-04-20T21:17:28.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.571+0000 7f6eaf200640 1 --2- 172.21.15.144:0/3170795027 >> v2:172.21.15.144:3300/0 conn(0x7f6eb00d4a80 0x7f6eb0162530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:28.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.571+0000 7f6eaf200640 1 --2- 172.21.15.144:0/3170795027 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6eb00d1aa0 0x7f6eb00d4540 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:28.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.571+0000 7f6eaf200640 1 --2- 172.21.15.144:0/3170795027 >> v2:172.21.15.86:3300/0 conn(0x7f6eb015e8e0 0x7f6eb00b44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:28.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.571+0000 7f6eaf200640 1 -- 172.21.15.144:0/3170795027 >> 172.21.15.144:0/3170795027 conn(0x7f6eb009d9e0 msgr2=0x7f6eb008bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:28.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.571+0000 7f6eaf200640 1 -- 172.21.15.144:0/3170795027 shutdown_connections 2024-04-20T21:17:28.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.571+0000 7f6eaf200640 1 -- 172.21.15.144:0/3170795027 wait complete. 2024-04-20T21:17:28.573 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.573+0000 7f6eaf200640 1 Processor -- start 2024-04-20T21:17:28.573 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.573+0000 7f6eaf200640 1 -- start start 2024-04-20T21:17:28.575 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.574+0000 7f6eaf200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6eb00d1aa0 0x7f6eb01f3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:28.575 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.574+0000 7f6eaf200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6eb00d4a80 0x7f6eb00b6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:28.575 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.574+0000 7f6eaf200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6eb015e8e0 0x7f6eb00b6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:28.575 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.574+0000 7f6eaf200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb00d1f30 con 0x7f6eb00d1aa0 2024-04-20T21:17:28.575 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.574+0000 7f6eaf200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb00d4ed0 con 0x7f6eb015e8e0 2024-04-20T21:17:28.575 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.574+0000 7f6eaf200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6eb015ecc0 con 0x7f6eb00d4a80 2024-04-20T21:17:28.575 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.574+0000 7f6ead400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6eb00d4a80 0x7f6eb00b6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:28.575 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.574+0000 7f6eae800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6eb015e8e0 0x7f6eb00b6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:28.575 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.574+0000 7f6ead400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6eb00d4a80 0x7f6eb00b6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.144:44616/0 (socket says 172.21.15.144:44616) 2024-04-20T21:17:28.575 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.575+0000 7f6ead400640 1 -- 172.21.15.144:0/3605808735 learned_addr learned my addr 172.21.15.144:0/3605808735 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-04-20T21:17:28.576 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.575+0000 7f6eade00640 1 --2- 172.21.15.144:0/3605808735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6eb00d1aa0 0x7f6eb01f3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:28.576 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.575+0000 7f6eae800640 1 -- 172.21.15.144:0/3605808735 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6eb00d4a80 msgr2=0x7f6eb00b6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:28.576 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.575+0000 7f6eae800640 1 --2- 172.21.15.144:0/3605808735 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6eb00d4a80 0x7f6eb00b6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:28.576 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.575+0000 7f6eae800640 1 -- 172.21.15.144:0/3605808735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6eb00d1aa0 msgr2=0x7f6eb01f3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:28.576 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.575+0000 7f6eae800640 1 --2- 172.21.15.144:0/3605808735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6eb00d1aa0 0x7f6eb01f3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:28.576 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.575+0000 7f6eae800640 1 -- 172.21.15.144:0/3605808735 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eb00b8080 con 0x7f6eb015e8e0 2024-04-20T21:17:28.576 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.575+0000 7f6ead400640 1 --2- 172.21.15.144:0/3605808735 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6eb00d4a80 0x7f6eb00b6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:28.576 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.575+0000 7f6eae800640 1 --2- 172.21.15.144:0/3605808735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6eb015e8e0 0x7f6eb00b6960 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6ea800e7f0 tx=0x7f6ea800ebf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:28.577 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.576+0000 7f6ea6000640 1 -- 172.21.15.144:0/3605808735 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea8017070 con 0x7f6eb015e8e0 2024-04-20T21:17:28.577 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.576+0000 7f6ea6000640 1 -- 172.21.15.144:0/3605808735 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ea800ee40 con 0x7f6eb015e8e0 2024-04-20T21:17:28.577 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.576+0000 7f6ea6000640 1 -- 172.21.15.144:0/3605808735 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea800cb60 con 0x7f6eb015e8e0 2024-04-20T21:17:28.577 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.576+0000 7f6eaf200640 1 -- 172.21.15.144:0/3605808735 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6eb00b8320 con 0x7f6eb015e8e0 2024-04-20T21:17:28.577 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.576+0000 7f6eaf200640 1 -- 172.21.15.144:0/3605808735 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6eb00b88e0 con 0x7f6eb015e8e0 2024-04-20T21:17:28.577 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.576+0000 7f6ea6000640 1 -- 172.21.15.144:0/3605808735 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6ea801a070 con 0x7f6eb015e8e0 2024-04-20T21:17:28.577 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.576+0000 7f6ea6000640 1 -- 172.21.15.144:0/3605808735 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f6ea8003950 con 0x7f6eb015e8e0 2024-04-20T21:17:28.578 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.577+0000 7f6eaf200640 1 -- 172.21.15.144:0/3605808735 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6eb004fd90 con 0x7f6eb015e8e0 2024-04-20T21:17:28.579 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.580+0000 7f6ea6000640 1 -- 172.21.15.144:0/3605808735 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f6ea8005990 con 0x7f6eb015e8e0 2024-04-20T21:17:28.658 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.658+0000 7f6eaf200640 1 -- 172.21.15.144:0/3605808735 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd new", "uuid": "87a2269d-c3c3-4e14-9d35-03ea51d17677", "id": 0} v 0) -- 0x7f6eb00b42b0 con 0x7f6eb015e8e0 2024-04-20T21:17:28.665 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.665+0000 7f6ea6000640 1 -- 172.21.15.144:0/3605808735 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "87a2269d-c3c3-4e14-9d35-03ea51d17677", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f6ea8003b50 con 0x7f6eb015e8e0 2024-04-20T21:17:28.665 INFO:teuthology.orchestra.run.smithi144.stdout:0 2024-04-20T21:17:28.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.667+0000 7f6eaf200640 1 -- 172.21.15.144:0/3605808735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6eb015e8e0 msgr2=0x7f6eb00b6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:28.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.667+0000 7f6eaf200640 1 --2- 172.21.15.144:0/3605808735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6eb015e8e0 0x7f6eb00b6960 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6ea800e7f0 tx=0x7f6ea800ebf0 comp rx=0 tx=0).stop 2024-04-20T21:17:28.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.667+0000 7f6eaf200640 1 -- 172.21.15.144:0/3605808735 shutdown_connections 2024-04-20T21:17:28.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.667+0000 7f6eaf200640 1 --2- 172.21.15.144:0/3605808735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6eb015e8e0 0x7f6eb00b6960 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:28.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.667+0000 7f6eaf200640 1 --2- 172.21.15.144:0/3605808735 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6eb00d4a80 0x7f6eb00b6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:28.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.667+0000 7f6eaf200640 1 --2- 172.21.15.144:0/3605808735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6eb00d1aa0 0x7f6eb01f3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:28.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.667+0000 7f6eaf200640 1 -- 172.21.15.144:0/3605808735 >> 172.21.15.144:0/3605808735 conn(0x7f6eb009d9e0 msgr2=0x7f6eb00b5910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:28.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.667+0000 7f6eaf200640 1 -- 172.21.15.144:0/3605808735 shutdown_connections 2024-04-20T21:17:28.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.667+0000 7f6eaf200640 1 -- 172.21.15.144:0/3605808735 wait complete. 2024-04-20T21:17:28.685 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new 52e5994e-f24d-4f0c-bbea-e064c89e8af9 1 2024-04-20T21:17:28.850 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.851+0000 7fde31000640 1 Processor -- start 2024-04-20T21:17:28.851 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.851+0000 7fde31000640 1 -- start start 2024-04-20T21:17:28.852 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.852+0000 7fde31000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fde2c0ad850 0x7fde2c0b4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:28.853 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.852+0000 7fde31000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde2c0b4660 0x7fde2c06e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:28.853 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.852+0000 7fde31000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde2c0adee0 con 0x7fde2c161c90 2024-04-20T21:17:28.853 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.852+0000 7fde31000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fde2c0ae1c0 con 0x7fde2c0ad850 2024-04-20T21:17:28.853 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.852+0000 7fde31000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fde2c0ae4a0 con 0x7fde2c0b4660 2024-04-20T21:17:28.853 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.853+0000 7fde2be00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde2c0b4660 0x7fde2c06e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:28.853 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.853+0000 7fde2b400640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fde2c161c90 0x7fde2c0aef80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:52820/0 (socket says 172.21.15.144:52820) 2024-04-20T21:17:28.853 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.853+0000 7fde2b400640 1 -- 172.21.15.144:0/3933883956 learned_addr learned my addr 172.21.15.144:0/3933883956 (peer_addr_for_me v1:172.21.15.144:0/0) 2024-04-20T21:17:28.853 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.853+0000 7fde2aa00640 1 --2- 172.21.15.144:0/3933883956 >> v2:172.21.15.86:3300/0 conn(0x7fde2c0ad850 0x7fde2c0b4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:28.854 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.853+0000 7fde2be00640 1 -- 172.21.15.144:0/3933883956 >> v2:172.21.15.86:3300/0 conn(0x7fde2c0ad850 msgr2=0x7fde2c0b4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:28.854 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.853+0000 7fde2be00640 1 --2- 172.21.15.144:0/3933883956 >> v2:172.21.15.86:3300/0 conn(0x7fde2c0ad850 0x7fde2c0b4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:28.854 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.853+0000 7fde2be00640 1 -- 172.21.15.144:0/3933883956 >> v1:172.21.15.144:6789/0 conn(0x7fde2c161c90 legacy=0x7fde2c0aef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:28.854 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.853+0000 7fde2be00640 1 -- 172.21.15.144:0/3933883956 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde2c0ae780 con 0x7fde2c0b4660 2024-04-20T21:17:28.854 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.853+0000 7fde2a000640 1 -- 172.21.15.144:0/3933883956 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1961991721 0 0) 0x7fde2c0adee0 con 0x7fde2c161c90 2024-04-20T21:17:28.854 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.854+0000 7fde2be00640 1 --2- 172.21.15.144:0/3933883956 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde2c0b4660 0x7fde2c06e540 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fde24009d90 tx=0x7fde2402bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4744685f1b301829 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:28.855 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.855+0000 7fde2a000640 1 -- 172.21.15.144:0/3933883956 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde24039070 con 0x7fde2c0b4660 2024-04-20T21:17:28.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.855+0000 7fde2a000640 1 -- 172.21.15.144:0/3933883956 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde2402cc50 con 0x7fde2c0b4660 2024-04-20T21:17:28.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.855+0000 7fde2a000640 1 -- 172.21.15.144:0/3933883956 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde24034440 con 0x7fde2c0b4660 2024-04-20T21:17:28.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.855+0000 7fde31000640 1 -- 172.21.15.144:0/3933883956 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde2c0b4660 msgr2=0x7fde2c06e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:28.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.855+0000 7fde31000640 1 --2- 172.21.15.144:0/3933883956 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde2c0b4660 0x7fde2c06e540 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fde24009d90 tx=0x7fde2402bea0 comp rx=0 tx=0).stop 2024-04-20T21:17:28.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.855+0000 7fde31000640 1 -- 172.21.15.144:0/3933883956 shutdown_connections 2024-04-20T21:17:28.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.856+0000 7fde31000640 1 --2- 172.21.15.144:0/3933883956 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde2c0b4660 0x7fde2c06e540 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:28.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.856+0000 7fde31000640 1 --2- 172.21.15.144:0/3933883956 >> v2:172.21.15.86:3300/0 conn(0x7fde2c0ad850 0x7fde2c0b4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:28.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.856+0000 7fde31000640 1 -- 172.21.15.144:0/3933883956 >> 172.21.15.144:0/3933883956 conn(0x7fde2c0733d0 msgr2=0x7fde2c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:28.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.856+0000 7fde31000640 1 -- 172.21.15.144:0/3933883956 shutdown_connections 2024-04-20T21:17:28.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.856+0000 7fde31000640 1 -- 172.21.15.144:0/3933883956 wait complete. 2024-04-20T21:17:28.857 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.857+0000 7fde31000640 1 Processor -- start 2024-04-20T21:17:28.857 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.857+0000 7fde31000640 1 -- start start 2024-04-20T21:17:28.860 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.858+0000 7fde31000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde2c0ad850 0x7fde2c158780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:28.860 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.858+0000 7fde31000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde2c0b4660 0x7fde2c160030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:28.860 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.858+0000 7fde31000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde2c161c90 0x7fde2c14b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:28.860 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.858+0000 7fde31000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fde2c0af690 con 0x7fde2c0ad850 2024-04-20T21:17:28.860 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.858+0000 7fde31000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fde2c070d60 con 0x7fde2c0b4660 2024-04-20T21:17:28.860 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.858+0000 7fde31000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fde2c162070 con 0x7fde2c161c90 2024-04-20T21:17:28.860 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.859+0000 7fde2aa00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde2c0b4660 0x7fde2c160030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:28.860 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.859+0000 7fde2aa00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde2c0b4660 0x7fde2c160030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34974/0 (socket says 172.21.15.144:34974) 2024-04-20T21:17:28.861 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.859+0000 7fde2aa00640 1 -- 172.21.15.144:0/3832047026 learned_addr learned my addr 172.21.15.144:0/3832047026 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-04-20T21:17:28.861 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.859+0000 7fde2be00640 1 --2- 172.21.15.144:0/3832047026 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde2c161c90 0x7fde2c14b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:28.861 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.859+0000 7fde2b400640 1 --2- 172.21.15.144:0/3832047026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde2c0ad850 0x7fde2c158780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:28.861 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.859+0000 7fde2aa00640 1 -- 172.21.15.144:0/3832047026 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde2c161c90 msgr2=0x7fde2c14b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:28.861 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.859+0000 7fde2aa00640 1 --2- 172.21.15.144:0/3832047026 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde2c161c90 0x7fde2c14b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:28.861 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.859+0000 7fde2aa00640 1 -- 172.21.15.144:0/3832047026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde2c0ad850 msgr2=0x7fde2c158780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:28.861 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.859+0000 7fde2aa00640 1 --2- 172.21.15.144:0/3832047026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde2c0ad850 0x7fde2c158780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:28.861 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.859+0000 7fde2aa00640 1 -- 172.21.15.144:0/3832047026 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde2c159080 con 0x7fde2c0b4660 2024-04-20T21:17:28.861 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.859+0000 7fde2be00640 1 --2- 172.21.15.144:0/3832047026 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde2c161c90 0x7fde2c14b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:28.861 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.860+0000 7fde2aa00640 1 --2- 172.21.15.144:0/3832047026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde2c0b4660 0x7fde2c160030 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fde1800d800 tx=0x7fde1800dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:28.861 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.860+0000 7fde23400640 1 -- 172.21.15.144:0/3832047026 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1800b6c0 con 0x7fde2c0b4660 2024-04-20T21:17:28.861 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.860+0000 7fde23400640 1 -- 172.21.15.144:0/3832047026 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde18019070 con 0x7fde2c0b4660 2024-04-20T21:17:28.862 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.860+0000 7fde23400640 1 -- 172.21.15.144:0/3832047026 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1800ba80 con 0x7fde2c0b4660 2024-04-20T21:17:28.862 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.860+0000 7fde31000640 1 -- 172.21.15.144:0/3832047026 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde2c159640 con 0x7fde2c0b4660 2024-04-20T21:17:28.862 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.860+0000 7fde31000640 1 -- 172.21.15.144:0/3832047026 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde2c159c00 con 0x7fde2c0b4660 2024-04-20T21:17:28.862 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.860+0000 7fde23400640 1 -- 172.21.15.144:0/3832047026 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fde1800bc20 con 0x7fde2c0b4660 2024-04-20T21:17:28.862 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.860+0000 7fde23400640 1 -- 172.21.15.144:0/3832047026 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fde18014910 con 0x7fde2c0b4660 2024-04-20T21:17:28.862 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.861+0000 7fde31000640 1 -- 172.21.15.144:0/3832047026 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde2c15f080 con 0x7fde2c0b4660 2024-04-20T21:17:28.863 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.864+0000 7fde23400640 1 -- 172.21.15.144:0/3832047026 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7fde18014b10 con 0x7fde2c0b4660 2024-04-20T21:17:28.942 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.942+0000 7fde31000640 1 -- 172.21.15.144:0/3832047026 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd new", "uuid": "52e5994e-f24d-4f0c-bbea-e064c89e8af9", "id": 1} v 0) -- 0x7fde2c04fd90 con 0x7fde2c0b4660 2024-04-20T21:17:28.948 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.949+0000 7fde23400640 1 -- 172.21.15.144:0/3832047026 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "52e5994e-f24d-4f0c-bbea-e064c89e8af9", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fde18014dc0 con 0x7fde2c0b4660 2024-04-20T21:17:28.948 INFO:teuthology.orchestra.run.smithi144.stdout:1 2024-04-20T21:17:28.950 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.950+0000 7fde31000640 1 -- 172.21.15.144:0/3832047026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde2c0b4660 msgr2=0x7fde2c160030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:28.950 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.950+0000 7fde31000640 1 --2- 172.21.15.144:0/3832047026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde2c0b4660 0x7fde2c160030 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fde1800d800 tx=0x7fde1800dcc0 comp rx=0 tx=0).stop 2024-04-20T21:17:28.950 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.950+0000 7fde31000640 1 -- 172.21.15.144:0/3832047026 shutdown_connections 2024-04-20T21:17:28.950 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.950+0000 7fde31000640 1 --2- 172.21.15.144:0/3832047026 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde2c161c90 0x7fde2c14b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:28.950 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.950+0000 7fde31000640 1 --2- 172.21.15.144:0/3832047026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde2c0b4660 0x7fde2c160030 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:28.950 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.950+0000 7fde31000640 1 --2- 172.21.15.144:0/3832047026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde2c0ad850 0x7fde2c158780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:28.950 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.950+0000 7fde31000640 1 -- 172.21.15.144:0/3832047026 >> 172.21.15.144:0/3832047026 conn(0x7fde2c0733d0 msgr2=0x7fde2c0b4ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:28.950 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.950+0000 7fde31000640 1 -- 172.21.15.144:0/3832047026 shutdown_connections 2024-04-20T21:17:28.950 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:28.950+0000 7fde31000640 1 -- 172.21.15.144:0/3832047026 wait complete. 2024-04-20T21:17:28.966 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new 1fe89fb0-bf19-49ae-80ef-13880cf7144c 2 2024-04-20T21:17:29.136 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.136+0000 7f8d7a400640 1 Processor -- start 2024-04-20T21:17:29.137 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.137+0000 7f8d7a400640 1 -- start start 2024-04-20T21:17:29.138 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.138+0000 7f8d7a400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8d740b44d0 0x7f8d740d4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:29.138 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.138+0000 7f8d7a400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d740b4ab0 con 0x7f8d7415e8e0 2024-04-20T21:17:29.138 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.138+0000 7f8d7a400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d740b4d90 con 0x7f8d740d4a80 2024-04-20T21:17:29.138 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.138+0000 7f8d7a400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f8d740b5070 con 0x7f8d740b44d0 2024-04-20T21:17:29.138 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.138+0000 7f8d73e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8d740b44d0 0x7f8d740d4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:29.138 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.138+0000 7f8d73e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8d740b44d0 0x7f8d740d4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34990/0 (socket says 172.21.15.144:34990) 2024-04-20T21:17:29.139 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.138+0000 7f8d73e00640 1 -- 172.21.15.144:0/2740660717 learned_addr learned my addr 172.21.15.144:0/2740660717 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-04-20T21:17:29.139 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.139+0000 7f8d73e00640 1 -- 172.21.15.144:0/2740660717 >> v1:172.21.15.144:6789/0 conn(0x7f8d740d4a80 legacy=0x7f8d740d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:29.139 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.139+0000 7f8d73e00640 1 -- 172.21.15.144:0/2740660717 >> v1:172.21.15.86:6789/0 conn(0x7f8d7415e8e0 legacy=0x7f8d740d1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:29.139 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.139+0000 7f8d73e00640 1 -- 172.21.15.144:0/2740660717 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d740b5350 con 0x7f8d740b44d0 2024-04-20T21:17:29.140 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.140+0000 7f8d73e00640 1 --2- 172.21.15.144:0/2740660717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d740b44d0 0x7f8d740d4540 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8d60009870 tx=0x7f8d6002b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1ff4117921a9b10 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:29.140 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.140+0000 7f8d73400640 1 -- 172.21.15.144:0/2740660717 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d60035020 con 0x7f8d740b44d0 2024-04-20T21:17:29.140 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.140+0000 7f8d73400640 1 -- 172.21.15.144:0/2740660717 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d60031870 con 0x7f8d740b44d0 2024-04-20T21:17:29.140 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.140+0000 7f8d73400640 1 -- 172.21.15.144:0/2740660717 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d60031c40 con 0x7f8d740b44d0 2024-04-20T21:17:29.141 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.141+0000 7f8d7a400640 1 -- 172.21.15.144:0/2740660717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d740b44d0 msgr2=0x7f8d740d4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:29.141 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.141+0000 7f8d7a400640 1 --2- 172.21.15.144:0/2740660717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d740b44d0 0x7f8d740d4540 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8d60009870 tx=0x7f8d6002b910 comp rx=0 tx=0).stop 2024-04-20T21:17:29.141 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.141+0000 7f8d7a400640 1 -- 172.21.15.144:0/2740660717 shutdown_connections 2024-04-20T21:17:29.141 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.141+0000 7f8d7a400640 1 --2- 172.21.15.144:0/2740660717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d740b44d0 0x7f8d740d4540 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:29.141 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.141+0000 7f8d7a400640 1 -- 172.21.15.144:0/2740660717 >> 172.21.15.144:0/2740660717 conn(0x7f8d7409d9e0 msgr2=0x7f8d7408bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:29.141 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.141+0000 7f8d7a400640 1 -- 172.21.15.144:0/2740660717 shutdown_connections 2024-04-20T21:17:29.141 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.141+0000 7f8d7a400640 1 -- 172.21.15.144:0/2740660717 wait complete. 2024-04-20T21:17:29.142 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.143+0000 7f8d7a400640 1 Processor -- start 2024-04-20T21:17:29.142 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.143+0000 7f8d7a400640 1 -- start start 2024-04-20T21:17:29.145 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.144+0000 7f8d7a400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d740b44d0 0x7f8d74155dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:29.145 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.144+0000 7f8d7a400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d740d4a80 0x7f8d740cc720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:29.145 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.144+0000 7f8d7a400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d7415e8e0 0x7f8d740c95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:29.145 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.144+0000 7f8d7a400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7415ecc0 con 0x7f8d740b44d0 2024-04-20T21:17:29.145 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.144+0000 7f8d7a400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8d740d4e90 con 0x7f8d740d4a80 2024-04-20T21:17:29.145 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.144+0000 7f8d7a400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8d7415ee40 con 0x7f8d7415e8e0 2024-04-20T21:17:29.145 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.144+0000 7f8d79a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d7415e8e0 0x7f8d740c95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:29.145 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.144+0000 7f8d73e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d740d4a80 0x7f8d740cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:29.145 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.144+0000 7f8d79a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d7415e8e0 0x7f8d740c95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.144:44630/0 (socket says 172.21.15.144:44630) 2024-04-20T21:17:29.145 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.144+0000 7f8d79a00640 1 -- 172.21.15.144:0/1255945368 learned_addr learned my addr 172.21.15.144:0/1255945368 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-04-20T21:17:29.146 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.144+0000 7f8d79000640 1 --2- 172.21.15.144:0/1255945368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d740b44d0 0x7f8d74155dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:29.146 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.144+0000 7f8d79a00640 1 -- 172.21.15.144:0/1255945368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d740d4a80 msgr2=0x7f8d740cc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:29.146 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.144+0000 7f8d79a00640 1 --2- 172.21.15.144:0/1255945368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d740d4a80 0x7f8d740cc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:29.146 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.145+0000 7f8d79a00640 1 -- 172.21.15.144:0/1255945368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d740b44d0 msgr2=0x7f8d74155dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:29.146 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.145+0000 7f8d79a00640 1 --2- 172.21.15.144:0/1255945368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d740b44d0 0x7f8d74155dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:29.146 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.145+0000 7f8d79a00640 1 -- 172.21.15.144:0/1255945368 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d740c9ea0 con 0x7f8d7415e8e0 2024-04-20T21:17:29.146 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.145+0000 7f8d79000640 1 --2- 172.21.15.144:0/1255945368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d740b44d0 0x7f8d74155dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:17:29.146 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.145+0000 7f8d79a00640 1 --2- 172.21.15.144:0/1255945368 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d7415e8e0 0x7f8d740c95a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c00c7a0 tx=0x7f8d6c00cc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:29.146 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.145+0000 7f8d70c00640 1 -- 172.21.15.144:0/1255945368 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c003b70 con 0x7f8d7415e8e0 2024-04-20T21:17:29.146 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.145+0000 7f8d70c00640 1 -- 172.21.15.144:0/1255945368 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d6c003d30 con 0x7f8d7415e8e0 2024-04-20T21:17:29.146 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.145+0000 7f8d70c00640 1 -- 172.21.15.144:0/1255945368 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c00aa90 con 0x7f8d7415e8e0 2024-04-20T21:17:29.146 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.145+0000 7f8d7a400640 1 -- 172.21.15.144:0/1255945368 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d740ca140 con 0x7f8d7415e8e0 2024-04-20T21:17:29.146 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.145+0000 7f8d7a400640 1 -- 172.21.15.144:0/1255945368 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d740ca700 con 0x7f8d7415e8e0 2024-04-20T21:17:29.147 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.146+0000 7f8d70c00640 1 -- 172.21.15.144:0/1255945368 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8d6c00ac30 con 0x7f8d7415e8e0 2024-04-20T21:17:29.147 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.146+0000 7f8d70c00640 1 -- 172.21.15.144:0/1255945368 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f8d6c0054f0 con 0x7f8d7415e8e0 2024-04-20T21:17:29.147 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.146+0000 7f8d7a400640 1 -- 172.21.15.144:0/1255945368 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d74073510 con 0x7f8d7415e8e0 2024-04-20T21:17:29.149 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.149+0000 7f8d70c00640 1 -- 172.21.15.144:0/1255945368 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f8d74073510 con 0x7f8d7415e8e0 2024-04-20T21:17:29.228 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.228+0000 7f8d7a400640 1 -- 172.21.15.144:0/1255945368 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd new", "uuid": "1fe89fb0-bf19-49ae-80ef-13880cf7144c", "id": 2} v 0) -- 0x7f8d7404fd90 con 0x7f8d7415e8e0 2024-04-20T21:17:29.235 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.235+0000 7f8d70c00640 1 -- 172.21.15.144:0/1255945368 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1fe89fb0-bf19-49ae-80ef-13880cf7144c", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f8d6c0050d0 con 0x7f8d7415e8e0 2024-04-20T21:17:29.235 INFO:teuthology.orchestra.run.smithi144.stdout:2 2024-04-20T21:17:29.237 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.237+0000 7f8d7a400640 1 -- 172.21.15.144:0/1255945368 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d7415e8e0 msgr2=0x7f8d740c95a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:29.237 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.237+0000 7f8d7a400640 1 --2- 172.21.15.144:0/1255945368 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d7415e8e0 0x7f8d740c95a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c00c7a0 tx=0x7f8d6c00cc60 comp rx=0 tx=0).stop 2024-04-20T21:17:29.237 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.237+0000 7f8d7a400640 1 -- 172.21.15.144:0/1255945368 shutdown_connections 2024-04-20T21:17:29.237 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.237+0000 7f8d7a400640 1 --2- 172.21.15.144:0/1255945368 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d7415e8e0 0x7f8d740c95a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:29.237 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.237+0000 7f8d7a400640 1 --2- 172.21.15.144:0/1255945368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d740d4a80 0x7f8d740cc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:29.237 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.237+0000 7f8d7a400640 1 --2- 172.21.15.144:0/1255945368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d740b44d0 0x7f8d74155dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:29.237 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.237+0000 7f8d7a400640 1 -- 172.21.15.144:0/1255945368 >> 172.21.15.144:0/1255945368 conn(0x7f8d7409d9e0 msgr2=0x7f8d7406ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:29.237 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.237+0000 7f8d7a400640 1 -- 172.21.15.144:0/1255945368 shutdown_connections 2024-04-20T21:17:29.238 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.237+0000 7f8d7a400640 1 -- 172.21.15.144:0/1255945368 wait complete. 2024-04-20T21:17:29.253 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new 728f9832-283e-4e71-a815-33afa71d1344 3 2024-04-20T21:17:29.354 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-20T21:17:29.354+0000 7f0a0524b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-20T21:17:29.390 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-04-20T21:17:29.390+0000 7fe5f8bc0040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-20T21:17:29.433 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.433+0000 7f3f1d400640 1 Processor -- start 2024-04-20T21:17:29.433 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.433+0000 7f3f1d400640 1 -- start start 2024-04-20T21:17:29.435 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.434+0000 7f3f1d400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f1815e8e0 0x7f3f180b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:29.435 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.435+0000 7f3f1d400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3f180d4c50 0x7f3f180d24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:29.435 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.435+0000 7f3f1d400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f180b4ab0 con 0x7f3f180d1aa0 2024-04-20T21:17:29.435 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.435+0000 7f3f1d400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f3f180b4d90 con 0x7f3f180d4c50 2024-04-20T21:17:29.435 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.435+0000 7f3f1d400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3f180b5070 con 0x7f3f1815e8e0 2024-04-20T21:17:29.435 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.435+0000 7f3f0f400640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f3f180d1aa0 0x7f3f180d4540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:52840/0 (socket says 172.21.15.144:52840) 2024-04-20T21:17:29.435 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.435+0000 7f3f0f400640 1 -- 172.21.15.144:0/2883677093 learned_addr learned my addr 172.21.15.144:0/2883677093 (peer_addr_for_me v1:172.21.15.144:0/0) 2024-04-20T21:17:29.435 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.435+0000 7f3f17e00640 1 --2- 172.21.15.144:0/2883677093 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f1815e8e0 0x7f3f180b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:29.435 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.435+0000 7f3f1ca00640 1 --2- 172.21.15.144:0/2883677093 >> v2:172.21.15.144:3300/0 conn(0x7f3f180d4c50 0x7f3f180d24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:29.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.435+0000 7f3f17400640 1 -- 172.21.15.144:0/2883677093 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 147103643 0 0) 0x7f3f180b4ab0 con 0x7f3f180d1aa0 2024-04-20T21:17:29.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.435+0000 7f3f17400640 1 -- 172.21.15.144:0/2883677093 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3efc003520 con 0x7f3f180d1aa0 2024-04-20T21:17:29.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.435+0000 7f3f17e00640 1 -- 172.21.15.144:0/2883677093 >> v2:172.21.15.144:3300/0 conn(0x7f3f180d4c50 msgr2=0x7f3f180d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:29.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.436+0000 7f3f17e00640 1 --2- 172.21.15.144:0/2883677093 >> v2:172.21.15.144:3300/0 conn(0x7f3f180d4c50 0x7f3f180d24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:29.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.436+0000 7f3f17e00640 1 -- 172.21.15.144:0/2883677093 >> v1:172.21.15.144:6789/0 conn(0x7f3f180d1aa0 legacy=0x7f3f180d4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:29.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.436+0000 7f3f17e00640 1 -- 172.21.15.144:0/2883677093 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f180b5350 con 0x7f3f1815e8e0 2024-04-20T21:17:29.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.436+0000 7f3f1ca00640 1 --2- 172.21.15.144:0/2883677093 >> v2:172.21.15.144:3300/0 conn(0x7f3f180d4c50 0x7f3f180d24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:29.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.437+0000 7f3f17e00640 1 --2- 172.21.15.144:0/2883677093 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f1815e8e0 0x7f3f180b44d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3f180d3a30 tx=0x7f3f0802be00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=337d87f79eddac28 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:29.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.437+0000 7f3f17400640 1 -- 172.21.15.144:0/2883677093 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f08039070 con 0x7f3f1815e8e0 2024-04-20T21:17:29.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.437+0000 7f3f17400640 1 -- 172.21.15.144:0/2883677093 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f0802cbb0 con 0x7f3f1815e8e0 2024-04-20T21:17:29.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.437+0000 7f3f17400640 1 -- 172.21.15.144:0/2883677093 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f08034440 con 0x7f3f1815e8e0 2024-04-20T21:17:29.438 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.437+0000 7f3f1d400640 1 -- 172.21.15.144:0/2883677093 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f1815e8e0 msgr2=0x7f3f180b44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:29.438 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.437+0000 7f3f1d400640 1 --2- 172.21.15.144:0/2883677093 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f1815e8e0 0x7f3f180b44d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3f180d3a30 tx=0x7f3f0802be00 comp rx=0 tx=0).stop 2024-04-20T21:17:29.438 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.438+0000 7f3f1d400640 1 -- 172.21.15.144:0/2883677093 shutdown_connections 2024-04-20T21:17:29.438 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.438+0000 7f3f1d400640 1 --2- 172.21.15.144:0/2883677093 >> v2:172.21.15.144:3300/0 conn(0x7f3f180d4c50 0x7f3f180d24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:29.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.438+0000 7f3f1d400640 1 --2- 172.21.15.144:0/2883677093 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f1815e8e0 0x7f3f180b44d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:29.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.438+0000 7f3f1d400640 1 -- 172.21.15.144:0/2883677093 >> 172.21.15.144:0/2883677093 conn(0x7f3f1809d9e0 msgr2=0x7f3f1808bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:29.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.438+0000 7f3f1d400640 1 -- 172.21.15.144:0/2883677093 shutdown_connections 2024-04-20T21:17:29.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.438+0000 7f3f1d400640 1 -- 172.21.15.144:0/2883677093 wait complete. 2024-04-20T21:17:29.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.439+0000 7f3f1d400640 1 Processor -- start 2024-04-20T21:17:29.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.439+0000 7f3f1d400640 1 -- start start 2024-04-20T21:17:29.441 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.440+0000 7f3f1d400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f180d1aa0 0x7f3f180c82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:29.441 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.440+0000 7f3f1d400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f180d4c50 0x7f3f180c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:29.441 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.440+0000 7f3f1d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f1815e8e0 0x7f3f180c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:29.441 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.440+0000 7f3f1d400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3f180d1f00 con 0x7f3f1815e8e0 2024-04-20T21:17:29.441 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.440+0000 7f3f1d400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1815ecc0 con 0x7f3f180d4c50 2024-04-20T21:17:29.441 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.440+0000 7f3f1d400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3f180b3830 con 0x7f3f180d1aa0 2024-04-20T21:17:29.441 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.441+0000 7f3f17e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f180d1aa0 0x7f3f180c82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:29.441 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.441+0000 7f3f17e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f180d1aa0 0x7f3f180c82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.144:44636/0 (socket says 172.21.15.144:44636) 2024-04-20T21:17:29.441 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.441+0000 7f3f17e00640 1 -- 172.21.15.144:0/1494403440 learned_addr learned my addr 172.21.15.144:0/1494403440 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-04-20T21:17:29.441 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.441+0000 7f3f0f400640 1 --2- 172.21.15.144:0/1494403440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f180d4c50 0x7f3f180c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:29.442 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.441+0000 7f3f17e00640 1 -- 172.21.15.144:0/1494403440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f180d4c50 msgr2=0x7f3f180c87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:29.442 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.441+0000 7f3f17e00640 1 --2- 172.21.15.144:0/1494403440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f180d4c50 0x7f3f180c87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:29.442 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.441+0000 7f3f17e00640 1 -- 172.21.15.144:0/1494403440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f1815e8e0 msgr2=0x7f3f180c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:29.442 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.441+0000 7f3f17e00640 1 --2- 172.21.15.144:0/1494403440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f1815e8e0 0x7f3f180c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:29.442 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.441+0000 7f3f17e00640 1 -- 172.21.15.144:0/1494403440 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f180b3270 con 0x7f3f180d1aa0 2024-04-20T21:17:29.442 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.441+0000 7f3f17e00640 1 --2- 172.21.15.144:0/1494403440 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f180d1aa0 0x7f3f180c82b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3f08037040 tx=0x7f3f08004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:29.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.442+0000 7f3f14c00640 1 -- 172.21.15.144:0/1494403440 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f08039050 con 0x7f3f180d1aa0 2024-04-20T21:17:29.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.442+0000 7f3f14c00640 1 -- 172.21.15.144:0/1494403440 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f08045070 con 0x7f3f180d1aa0 2024-04-20T21:17:29.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.442+0000 7f3f14c00640 1 -- 172.21.15.144:0/1494403440 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f08009450 con 0x7f3f180d1aa0 2024-04-20T21:17:29.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.442+0000 7f3f1d400640 1 -- 172.21.15.144:0/1494403440 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f180b3550 con 0x7f3f180d1aa0 2024-04-20T21:17:29.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.442+0000 7f3f1d400640 1 -- 172.21.15.144:0/1494403440 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f18155fc0 con 0x7f3f180d1aa0 2024-04-20T21:17:29.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.442+0000 7f3f14c00640 1 -- 172.21.15.144:0/1494403440 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3f0803e850 con 0x7f3f180d1aa0 2024-04-20T21:17:29.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.442+0000 7f3f14c00640 1 -- 172.21.15.144:0/1494403440 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f3f0804a8f0 con 0x7f3f180d1aa0 2024-04-20T21:17:29.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.443+0000 7f3f1d400640 1 -- 172.21.15.144:0/1494403440 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f1804fd90 con 0x7f3f180d1aa0 2024-04-20T21:17:29.447 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.447+0000 7f3f14c00640 1 -- 172.21.15.144:0/1494403440 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f3f0803eb40 con 0x7f3f180d1aa0 2024-04-20T21:17:29.456 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-20T21:17:29.457+0000 7f0a0524b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-20T21:17:29.501 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-04-20T21:17:29.501+0000 7fe5f8bc0040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-20T21:17:29.506 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-20T21:17:29.507+0000 7f0a0524b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-20T21:17:29.527 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.527+0000 7f3f1d400640 1 -- 172.21.15.144:0/1494403440 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd new", "uuid": "728f9832-283e-4e71-a815-33afa71d1344", "id": 3} v 0) -- 0x7f3f180b42b0 con 0x7f3f180d1aa0 2024-04-20T21:17:29.534 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.534+0000 7f3f14c00640 1 -- 172.21.15.144:0/1494403440 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "728f9832-283e-4e71-a815-33afa71d1344", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f3f0804a460 con 0x7f3f180d1aa0 2024-04-20T21:17:29.534 INFO:teuthology.orchestra.run.smithi144.stdout:3 2024-04-20T21:17:29.536 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.536+0000 7f3f1d400640 1 -- 172.21.15.144:0/1494403440 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f180d1aa0 msgr2=0x7f3f180c82b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:29.536 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.536+0000 7f3f1d400640 1 --2- 172.21.15.144:0/1494403440 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f180d1aa0 0x7f3f180c82b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3f08037040 tx=0x7f3f08004a40 comp rx=0 tx=0).stop 2024-04-20T21:17:29.536 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.536+0000 7f3f1d400640 1 -- 172.21.15.144:0/1494403440 shutdown_connections 2024-04-20T21:17:29.536 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.536+0000 7f3f1d400640 1 --2- 172.21.15.144:0/1494403440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f1815e8e0 0x7f3f180c9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:29.536 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.536+0000 7f3f1d400640 1 --2- 172.21.15.144:0/1494403440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f180d4c50 0x7f3f180c87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:29.536 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.536+0000 7f3f1d400640 1 --2- 172.21.15.144:0/1494403440 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f180d1aa0 0x7f3f180c82b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:29.536 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.536+0000 7f3f1d400640 1 -- 172.21.15.144:0/1494403440 >> 172.21.15.144:0/1494403440 conn(0x7f3f1809d9e0 msgr2=0x7f3f180b6320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:29.537 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.536+0000 7f3f1d400640 1 -- 172.21.15.144:0/1494403440 shutdown_connections 2024-04-20T21:17:29.537 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.536+0000 7f3f1d400640 1 -- 172.21.15.144:0/1494403440 wait complete. 2024-04-20T21:17:29.552 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new 38b3b5f6-6c5a-4145-a43d-f88b190e50b0 4 2024-04-20T21:17:29.556 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-04-20T21:17:29.555+0000 7fe5f8bc0040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-20T21:17:29.696 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-20T21:17:29.697+0000 7f0a0524b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-20T21:17:29.727 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.727+0000 7fb2cf200640 1 Processor -- start 2024-04-20T21:17:29.727 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.727+0000 7fb2cf200640 1 -- start start 2024-04-20T21:17:29.729 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.728+0000 7fb2cf200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb2d015e8e0 0x7fb2d006d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:29.729 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.729+0000 7fb2cf200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb2d00d4d70 0x7fb2d00cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:29.729 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.729+0000 7fb2cf200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2d00d1f60 con 0x7fb2d015ecc0 2024-04-20T21:17:29.729 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.729+0000 7fb2cf200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb2d00cf7f0 con 0x7fb2d00d4d70 2024-04-20T21:17:29.729 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.729+0000 7fb2cf200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fb2d00d19a0 con 0x7fb2d015e8e0 2024-04-20T21:17:29.729 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.729+0000 7fb2cde00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb2d015e8e0 0x7fb2d006d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:29.730 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.729+0000 7fb2cde00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb2d015e8e0 0x7fb2d006d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35036/0 (socket says 172.21.15.144:35036) 2024-04-20T21:17:29.730 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.729+0000 7fb2cde00640 1 -- 172.21.15.144:0/211988117 learned_addr learned my addr 172.21.15.144:0/211988117 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-04-20T21:17:29.730 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.729+0000 7fb2ce800640 1 --2- 172.21.15.144:0/211988117 >> v2:172.21.15.86:3300/0 conn(0x7fb2d00d4d70 0x7fb2d00cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:29.730 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.729+0000 7fb2cde00640 1 -- 172.21.15.144:0/211988117 >> v2:172.21.15.86:3300/0 conn(0x7fb2d00d4d70 msgr2=0x7fb2d00cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:29.730 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.729+0000 7fb2cde00640 1 --2- 172.21.15.144:0/211988117 >> v2:172.21.15.86:3300/0 conn(0x7fb2d00d4d70 0x7fb2d00cfc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:29.730 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.729+0000 7fb2cde00640 1 -- 172.21.15.144:0/211988117 >> v1:172.21.15.86:6789/0 conn(0x7fb2d015ecc0 legacy=0x7fb2d00d4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:29.730 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.729+0000 7fb2cde00640 1 -- 172.21.15.144:0/211988117 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2d00d1c80 con 0x7fb2d015e8e0 2024-04-20T21:17:29.730 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.731+0000 7fb2cde00640 1 --2- 172.21.15.144:0/211988117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2d015e8e0 0x7fb2d006d200 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb2d008c790 tx=0x7fb2c002b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=772d13ea043a4221 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:29.731 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.731+0000 7fb2cd400640 1 -- 172.21.15.144:0/211988117 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0035020 con 0x7fb2d015e8e0 2024-04-20T21:17:29.731 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.731+0000 7fb2cd400640 1 -- 172.21.15.144:0/211988117 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2c0031640 con 0x7fb2d015e8e0 2024-04-20T21:17:29.731 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.731+0000 7fb2cd400640 1 -- 172.21.15.144:0/211988117 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c002d040 con 0x7fb2d015e8e0 2024-04-20T21:17:29.732 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.732+0000 7fb2cf200640 1 -- 172.21.15.144:0/211988117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2d015e8e0 msgr2=0x7fb2d006d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:29.732 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.732+0000 7fb2cf200640 1 --2- 172.21.15.144:0/211988117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2d015e8e0 0x7fb2d006d200 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb2d008c790 tx=0x7fb2c002b8f0 comp rx=0 tx=0).stop 2024-04-20T21:17:29.732 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.732+0000 7fb2cf200640 1 -- 172.21.15.144:0/211988117 shutdown_connections 2024-04-20T21:17:29.732 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.732+0000 7fb2cf200640 1 --2- 172.21.15.144:0/211988117 >> v2:172.21.15.86:3300/0 conn(0x7fb2d00d4d70 0x7fb2d00cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:29.733 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.732+0000 7fb2cf200640 1 --2- 172.21.15.144:0/211988117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2d015e8e0 0x7fb2d006d200 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:29.733 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.732+0000 7fb2cf200640 1 -- 172.21.15.144:0/211988117 >> 172.21.15.144:0/211988117 conn(0x7fb2d009d9d0 msgr2=0x7fb2d008bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:29.733 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.732+0000 7fb2cf200640 1 -- 172.21.15.144:0/211988117 shutdown_connections 2024-04-20T21:17:29.733 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.732+0000 7fb2cf200640 1 -- 172.21.15.144:0/211988117 wait complete. 2024-04-20T21:17:29.734 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.735+0000 7fb2cf200640 1 Processor -- start 2024-04-20T21:17:29.734 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.735+0000 7fb2cf200640 1 -- start start 2024-04-20T21:17:29.738 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.736+0000 7fb2cf200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2d00d4d70 0x7fb2d014e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:29.738 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.736+0000 7fb2cf200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2d015e8e0 0x7fb2d014b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:29.738 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.736+0000 7fb2cf200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2d00cd100 0x7fb2d00d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:29.738 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.736+0000 7fb2cf200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d006d8c0 con 0x7fb2d00d4d70 2024-04-20T21:17:29.738 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.736+0000 7fb2cf200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb2d006d740 con 0x7fb2d015e8e0 2024-04-20T21:17:29.738 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.736+0000 7fb2cf200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb2d00d01c0 con 0x7fb2d00cd100 2024-04-20T21:17:29.738 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.736+0000 7fb2c7e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2d015e8e0 0x7fb2d014b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:29.738 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.737+0000 7fb2c7e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2d015e8e0 0x7fb2d014b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35038/0 (socket says 172.21.15.144:35038) 2024-04-20T21:17:29.738 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.737+0000 7fb2c7e00640 1 -- 172.21.15.144:0/1485410318 learned_addr learned my addr 172.21.15.144:0/1485410318 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-04-20T21:17:29.739 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.737+0000 7fb2ce800640 1 --2- 172.21.15.144:0/1485410318 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2d00cd100 0x7fb2d00d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:29.739 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.737+0000 7fb2cde00640 1 --2- 172.21.15.144:0/1485410318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2d00d4d70 0x7fb2d014e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:29.739 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.737+0000 7fb2c7e00640 1 -- 172.21.15.144:0/1485410318 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2d00cd100 msgr2=0x7fb2d00d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:29.739 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.737+0000 7fb2c7e00640 1 --2- 172.21.15.144:0/1485410318 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2d00cd100 0x7fb2d00d05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:29.739 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.737+0000 7fb2c7e00640 1 -- 172.21.15.144:0/1485410318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2d00d4d70 msgr2=0x7fb2d014e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:29.739 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.737+0000 7fb2c7e00640 1 --2- 172.21.15.144:0/1485410318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2d00d4d70 0x7fb2d014e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:29.739 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.737+0000 7fb2c7e00640 1 -- 172.21.15.144:0/1485410318 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2d014ea30 con 0x7fb2d015e8e0 2024-04-20T21:17:29.739 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.737+0000 7fb2cde00640 1 --2- 172.21.15.144:0/1485410318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2d00d4d70 0x7fb2d014e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:17:29.739 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.737+0000 7fb2ce800640 1 --2- 172.21.15.144:0/1485410318 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2d00cd100 0x7fb2d00d05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:29.739 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.737+0000 7fb2c7e00640 1 --2- 172.21.15.144:0/1485410318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2d015e8e0 0x7fb2d014b140 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb2b800b3f0 tx=0x7fb2b800b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:29.739 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.738+0000 7fb2c6000640 1 -- 172.21.15.144:0/1485410318 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b8015020 con 0x7fb2d015e8e0 2024-04-20T21:17:29.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.738+0000 7fb2c6000640 1 -- 172.21.15.144:0/1485410318 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2b8011830 con 0x7fb2d015e8e0 2024-04-20T21:17:29.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.738+0000 7fb2cf200640 1 -- 172.21.15.144:0/1485410318 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2d00cd5e0 con 0x7fb2d015e8e0 2024-04-20T21:17:29.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.738+0000 7fb2c6000640 1 -- 172.21.15.144:0/1485410318 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b8011c20 con 0x7fb2d015e8e0 2024-04-20T21:17:29.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.738+0000 7fb2cf200640 1 -- 172.21.15.144:0/1485410318 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2d00cdba0 con 0x7fb2d015e8e0 2024-04-20T21:17:29.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.738+0000 7fb2c6000640 1 -- 172.21.15.144:0/1485410318 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb2b8018020 con 0x7fb2d015e8e0 2024-04-20T21:17:29.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.738+0000 7fb2c6000640 1 -- 172.21.15.144:0/1485410318 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7fb2b801a910 con 0x7fb2d015e8e0 2024-04-20T21:17:29.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.739+0000 7fb2cf200640 1 -- 172.21.15.144:0/1485410318 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2d00ce370 con 0x7fb2d015e8e0 2024-04-20T21:17:29.741 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.741+0000 7fb2c6000640 1 -- 172.21.15.144:0/1485410318 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7fb2b801f030 con 0x7fb2d015e8e0 2024-04-20T21:17:29.756 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-04-20T21:17:29.756+0000 7fe5f8bc0040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-20T21:17:29.760 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-20T21:17:29.761+0000 7f0a0524b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-20T21:17:29.813 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-04-20T21:17:29.814+0000 7fe5f8bc0040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-20T21:17:29.820 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-20T21:17:29.821+0000 7f0a0524b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-20T21:17:29.822 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.822+0000 7fb2cf200640 1 -- 172.21.15.144:0/1485410318 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd new", "uuid": "38b3b5f6-6c5a-4145-a43d-f88b190e50b0", "id": 4} v 0) -- 0x7fb2d004fd90 con 0x7fb2d015e8e0 2024-04-20T21:17:29.829 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.830+0000 7fb2c6000640 1 -- 172.21.15.144:0/1485410318 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "38b3b5f6-6c5a-4145-a43d-f88b190e50b0", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7fb2b801a400 con 0x7fb2d015e8e0 2024-04-20T21:17:29.829 INFO:teuthology.orchestra.run.smithi144.stdout:4 2024-04-20T21:17:29.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.831+0000 7fb2cf200640 1 -- 172.21.15.144:0/1485410318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2d015e8e0 msgr2=0x7fb2d014b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:29.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.831+0000 7fb2cf200640 1 --2- 172.21.15.144:0/1485410318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2d015e8e0 0x7fb2d014b140 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb2b800b3f0 tx=0x7fb2b800b8b0 comp rx=0 tx=0).stop 2024-04-20T21:17:29.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.831+0000 7fb2cf200640 1 -- 172.21.15.144:0/1485410318 shutdown_connections 2024-04-20T21:17:29.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.831+0000 7fb2cf200640 1 --2- 172.21.15.144:0/1485410318 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2d00cd100 0x7fb2d00d05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:29.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.831+0000 7fb2cf200640 1 --2- 172.21.15.144:0/1485410318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2d015e8e0 0x7fb2d014b140 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:29.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.831+0000 7fb2cf200640 1 --2- 172.21.15.144:0/1485410318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2d00d4d70 0x7fb2d014e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:29.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.831+0000 7fb2cf200640 1 -- 172.21.15.144:0/1485410318 >> 172.21.15.144:0/1485410318 conn(0x7fb2d009d9d0 msgr2=0x7fb2d00d4660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:29.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.831+0000 7fb2cf200640 1 -- 172.21.15.144:0/1485410318 shutdown_connections 2024-04-20T21:17:29.832 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:29.831+0000 7fb2cf200640 1 -- 172.21.15.144:0/1485410318 wait complete. 2024-04-20T21:17:29.846 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new 05a16fec-9d93-4955-a3b1-5d1742bdb8fc 5 2024-04-20T21:17:29.880 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-04-20T21:17:29.880+0000 7fe5f8bc0040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-20T21:17:29.947 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-20T21:17:29.947+0000 7f0a0524b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-20T21:17:30.011 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-04-20T21:17:30.011+0000 7fe5f8bc0040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-20T21:17:30.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.029+0000 7fcbde200640 1 Processor -- start 2024-04-20T21:17:30.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.029+0000 7fcbde200640 1 -- start start 2024-04-20T21:17:30.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.030+0000 7fcbde200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbd815e8e0 0x7fcbd806d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:30.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.030+0000 7fcbde200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcbd80d4d70 0x7fcbd80cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:30.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.030+0000 7fcbde200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbd80d1f60 con 0x7fcbd815ecc0 2024-04-20T21:17:30.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.030+0000 7fcbde200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fcbd80cf7f0 con 0x7fcbd80d4d70 2024-04-20T21:17:30.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.030+0000 7fcbde200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fcbd80d19a0 con 0x7fcbd815e8e0 2024-04-20T21:17:30.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.031+0000 7fcbdce00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbd815e8e0 0x7fcbd806d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:30.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.031+0000 7fcbd7e00640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fcbd815ecc0 0x7fcbd80d4660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:52842/0 (socket says 172.21.15.144:52842) 2024-04-20T21:17:30.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.031+0000 7fcbd7e00640 1 -- 172.21.15.144:0/4091139502 learned_addr learned my addr 172.21.15.144:0/4091139502 (peer_addr_for_me v1:172.21.15.144:0/0) 2024-04-20T21:17:30.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.031+0000 7fcbdd800640 1 --2- 172.21.15.144:0/4091139502 >> v2:172.21.15.86:3300/0 conn(0x7fcbd80d4d70 0x7fcbd80cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:30.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.031+0000 7fcbdce00640 1 -- 172.21.15.144:0/4091139502 >> v2:172.21.15.86:3300/0 conn(0x7fcbd80d4d70 msgr2=0x7fcbd80cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:30.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.031+0000 7fcbd7400640 1 -- 172.21.15.144:0/4091139502 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3302533245 0 0) 0x7fcbd80d1f60 con 0x7fcbd815ecc0 2024-04-20T21:17:30.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.031+0000 7fcbdce00640 1 --2- 172.21.15.144:0/4091139502 >> v2:172.21.15.86:3300/0 conn(0x7fcbd80d4d70 0x7fcbd80cfc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:30.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.031+0000 7fcbdce00640 1 -- 172.21.15.144:0/4091139502 >> v1:172.21.15.144:6789/0 conn(0x7fcbd815ecc0 legacy=0x7fcbd80d4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:30.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.031+0000 7fcbdce00640 1 -- 172.21.15.144:0/4091139502 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbd80d1c80 con 0x7fcbd815e8e0 2024-04-20T21:17:30.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.032+0000 7fcbdce00640 1 --2- 172.21.15.144:0/4091139502 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbd815e8e0 0x7fcbd806d200 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc0097a0 tx=0x7fcbcc02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3aaf9aba52f0e8e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:30.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.033+0000 7fcbd7400640 1 -- 172.21.15.144:0/4091139502 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc035020 con 0x7fcbd815e8e0 2024-04-20T21:17:30.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.033+0000 7fcbd7400640 1 -- 172.21.15.144:0/4091139502 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbcc031830 con 0x7fcbd815e8e0 2024-04-20T21:17:30.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.033+0000 7fcbd7400640 1 -- 172.21.15.144:0/4091139502 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc031c00 con 0x7fcbd815e8e0 2024-04-20T21:17:30.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.033+0000 7fcbde200640 1 -- 172.21.15.144:0/4091139502 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbd815e8e0 msgr2=0x7fcbd806d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:30.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.033+0000 7fcbde200640 1 --2- 172.21.15.144:0/4091139502 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbd815e8e0 0x7fcbd806d200 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc0097a0 tx=0x7fcbcc02b8d0 comp rx=0 tx=0).stop 2024-04-20T21:17:30.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.033+0000 7fcbde200640 1 -- 172.21.15.144:0/4091139502 shutdown_connections 2024-04-20T21:17:30.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.033+0000 7fcbde200640 1 --2- 172.21.15.144:0/4091139502 >> v2:172.21.15.86:3300/0 conn(0x7fcbd80d4d70 0x7fcbd80cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:30.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.033+0000 7fcbde200640 1 --2- 172.21.15.144:0/4091139502 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbd815e8e0 0x7fcbd806d200 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:30.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.033+0000 7fcbde200640 1 -- 172.21.15.144:0/4091139502 >> 172.21.15.144:0/4091139502 conn(0x7fcbd809d9d0 msgr2=0x7fcbd808bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:30.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.034+0000 7fcbde200640 1 -- 172.21.15.144:0/4091139502 shutdown_connections 2024-04-20T21:17:30.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.034+0000 7fcbde200640 1 -- 172.21.15.144:0/4091139502 wait complete. 2024-04-20T21:17:30.035 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.035+0000 7fcbde200640 1 Processor -- start 2024-04-20T21:17:30.035 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.035+0000 7fcbde200640 1 -- start start 2024-04-20T21:17:30.038 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.036+0000 7fcbde200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbd80d4d70 0x7fcbd80c90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:30.038 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.036+0000 7fcbde200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbd815e8e0 0x7fcbd815a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:30.038 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.036+0000 7fcbde200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbd815f0b0 0x7fcbd80b3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:30.038 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.036+0000 7fcbde200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd806d8c0 con 0x7fcbd815f0b0 2024-04-20T21:17:30.038 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.036+0000 7fcbde200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd806d740 con 0x7fcbd80d4d70 2024-04-20T21:17:30.038 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.036+0000 7fcbde200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fcbd80d01c0 con 0x7fcbd815e8e0 2024-04-20T21:17:30.038 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.036+0000 7fcbdce00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbd80d4d70 0x7fcbd80c90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:30.038 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.036+0000 7fcbdce00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbd80d4d70 0x7fcbd80c90f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35046/0 (socket says 172.21.15.144:35046) 2024-04-20T21:17:30.038 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.036+0000 7fcbdce00640 1 -- 172.21.15.144:0/615730427 learned_addr learned my addr 172.21.15.144:0/615730427 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-04-20T21:17:30.038 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.036+0000 7fcbd7e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbd815e8e0 0x7fcbd815a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:30.039 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.037+0000 7fcbdd800640 1 --2- 172.21.15.144:0/615730427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbd815f0b0 0x7fcbd80b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:30.039 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.037+0000 7fcbdce00640 1 -- 172.21.15.144:0/615730427 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbd815e8e0 msgr2=0x7fcbd815a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:30.039 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.037+0000 7fcbdce00640 1 --2- 172.21.15.144:0/615730427 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbd815e8e0 0x7fcbd815a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:30.039 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.037+0000 7fcbdce00640 1 -- 172.21.15.144:0/615730427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbd815f0b0 msgr2=0x7fcbd80b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:30.039 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.037+0000 7fcbdce00640 1 --2- 172.21.15.144:0/615730427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbd815f0b0 0x7fcbd80b3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:30.039 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.037+0000 7fcbdce00640 1 -- 172.21.15.144:0/615730427 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbd815f560 con 0x7fcbd80d4d70 2024-04-20T21:17:30.039 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.037+0000 7fcbd7e00640 1 --2- 172.21.15.144:0/615730427 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbd815e8e0 0x7fcbd815a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:30.039 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.037+0000 7fcbdce00640 1 --2- 172.21.15.144:0/615730427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbd80d4d70 0x7fcbd80c90f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc0098c0 tx=0x7fcbcc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:30.039 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.037+0000 7fcbd4c00640 1 -- 172.21.15.144:0/615730427 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc041070 con 0x7fcbd80d4d70 2024-04-20T21:17:30.039 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.038+0000 7fcbd4c00640 1 -- 172.21.15.144:0/615730427 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbcc03aa30 con 0x7fcbd80d4d70 2024-04-20T21:17:30.039 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.038+0000 7fcbde200640 1 -- 172.21.15.144:0/615730427 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbd815f820 con 0x7fcbd80d4d70 2024-04-20T21:17:30.039 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.038+0000 7fcbd4c00640 1 -- 172.21.15.144:0/615730427 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc035040 con 0x7fcbd80d4d70 2024-04-20T21:17:30.039 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.038+0000 7fcbde200640 1 -- 172.21.15.144:0/615730427 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbd815fde0 con 0x7fcbd80d4d70 2024-04-20T21:17:30.039 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.038+0000 7fcbd4c00640 1 -- 172.21.15.144:0/615730427 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcbcc04c060 con 0x7fcbd80d4d70 2024-04-20T21:17:30.040 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.038+0000 7fcbd4c00640 1 -- 172.21.15.144:0/615730427 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7fcbcc03bb40 con 0x7fcbd80d4d70 2024-04-20T21:17:30.040 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.039+0000 7fcbde200640 1 -- 172.21.15.144:0/615730427 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbd804fd90 con 0x7fcbd80d4d70 2024-04-20T21:17:30.041 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.041+0000 7fcbd4c00640 1 -- 172.21.15.144:0/615730427 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7fcbcc03be20 con 0x7fcbd80d4d70 2024-04-20T21:17:30.120 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.120+0000 7fcbde200640 1 -- 172.21.15.144:0/615730427 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd new", "uuid": "05a16fec-9d93-4955-a3b1-5d1742bdb8fc", "id": 5} v 0) -- 0x7fcbd806cd10 con 0x7fcbd80d4d70 2024-04-20T21:17:30.127 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.128+0000 7fcbd4c00640 1 -- 172.21.15.144:0/615730427 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "05a16fec-9d93-4955-a3b1-5d1742bdb8fc", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7fcbcc053020 con 0x7fcbd80d4d70 2024-04-20T21:17:30.128 INFO:teuthology.orchestra.run.smithi144.stdout:5 2024-04-20T21:17:30.130 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.129+0000 7fcbde200640 1 -- 172.21.15.144:0/615730427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbd80d4d70 msgr2=0x7fcbd80c90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:30.130 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.129+0000 7fcbde200640 1 --2- 172.21.15.144:0/615730427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbd80d4d70 0x7fcbd80c90f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc0098c0 tx=0x7fcbcc0046c0 comp rx=0 tx=0).stop 2024-04-20T21:17:30.130 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.130+0000 7fcbde200640 1 -- 172.21.15.144:0/615730427 shutdown_connections 2024-04-20T21:17:30.130 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.130+0000 7fcbde200640 1 --2- 172.21.15.144:0/615730427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbd815f0b0 0x7fcbd80b3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:30.130 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.130+0000 7fcbde200640 1 --2- 172.21.15.144:0/615730427 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbd815e8e0 0x7fcbd815a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:30.130 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.130+0000 7fcbde200640 1 --2- 172.21.15.144:0/615730427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbd80d4d70 0x7fcbd80c90f0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:30.130 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.130+0000 7fcbde200640 1 -- 172.21.15.144:0/615730427 >> 172.21.15.144:0/615730427 conn(0x7fcbd809d9d0 msgr2=0x7fcbd808bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:30.130 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.130+0000 7fcbde200640 1 -- 172.21.15.144:0/615730427 shutdown_connections 2024-04-20T21:17:30.130 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.130+0000 7fcbde200640 1 -- 172.21.15.144:0/615730427 wait complete. 2024-04-20T21:17:30.146 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new f6e35481-399e-4da1-be5c-bfe52662d6be 6 2024-04-20T21:17:30.368 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.368+0000 7f0886e00640 1 Processor -- start 2024-04-20T21:17:30.368 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.368+0000 7f0886e00640 1 -- start start 2024-04-20T21:17:30.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.369+0000 7f0886e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0880160310 0x7f08800cd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:30.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.369+0000 7f0886e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f08801606f0 0x7f088009b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:30.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.369+0000 7f0886e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08800cd5e0 con 0x7f088015ca00 2024-04-20T21:17:30.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.370+0000 7f0886e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f08800ce1f0 con 0x7f0880160310 2024-04-20T21:17:30.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.370+0000 7f0886e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f08800cd8c0 con 0x7f08801606f0 2024-04-20T21:17:30.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.370+0000 7f0885000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f08801606f0 0x7f088009b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:30.371 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.370+0000 7f0885a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0880160310 0x7f08800cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:30.371 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.370+0000 7f0885000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f08801606f0 0x7f088009b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.144:44684/0 (socket says 172.21.15.144:44684) 2024-04-20T21:17:30.371 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.370+0000 7f0885000640 1 -- 172.21.15.144:0/2275886731 learned_addr learned my addr 172.21.15.144:0/2275886731 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-04-20T21:17:30.371 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.370+0000 7f0885a00640 1 -- 172.21.15.144:0/2275886731 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f08801606f0 msgr2=0x7f088009b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:30.371 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.370+0000 7f0885a00640 1 --2- 172.21.15.144:0/2275886731 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f08801606f0 0x7f088009b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:30.371 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.370+0000 7f0885a00640 1 -- 172.21.15.144:0/2275886731 >> v1:172.21.15.86:6789/0 conn(0x7f088015ca00 legacy=0x7f088015ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:30.371 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.370+0000 7f0885a00640 1 -- 172.21.15.144:0/2275886731 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08800cdba0 con 0x7f0880160310 2024-04-20T21:17:30.371 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.370+0000 7f0885000640 1 --2- 172.21.15.144:0/2275886731 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f08801606f0 0x7f088009b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:30.372 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.372+0000 7f0885a00640 1 --2- 172.21.15.144:0/2275886731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0880160310 0x7f08800cd000 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f08740097a0 tx=0x7f087402b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73dcbac3f16207bf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:30.372 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.372+0000 7f087fe00640 1 -- 172.21.15.144:0/2275886731 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0874039070 con 0x7f0880160310 2024-04-20T21:17:30.372 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.372+0000 7f087fe00640 1 -- 172.21.15.144:0/2275886731 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08740316f0 con 0x7f0880160310 2024-04-20T21:17:30.372 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.372+0000 7f087fe00640 1 -- 172.21.15.144:0/2275886731 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0874031ac0 con 0x7f0880160310 2024-04-20T21:17:30.373 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.373+0000 7f0886e00640 1 -- 172.21.15.144:0/2275886731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0880160310 msgr2=0x7f08800cd000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:30.373 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.373+0000 7f0886e00640 1 --2- 172.21.15.144:0/2275886731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0880160310 0x7f08800cd000 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f08740097a0 tx=0x7f087402b790 comp rx=0 tx=0).stop 2024-04-20T21:17:30.373 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.373+0000 7f0886e00640 1 -- 172.21.15.144:0/2275886731 shutdown_connections 2024-04-20T21:17:30.373 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.373+0000 7f0886e00640 1 --2- 172.21.15.144:0/2275886731 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f08801606f0 0x7f088009b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:30.373 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.373+0000 7f0886e00640 1 --2- 172.21.15.144:0/2275886731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0880160310 0x7f08800cd000 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:30.373 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.373+0000 7f0886e00640 1 -- 172.21.15.144:0/2275886731 >> 172.21.15.144:0/2275886731 conn(0x7f08800d2530 msgr2=0x7f08800d2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:30.373 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.373+0000 7f0886e00640 1 -- 172.21.15.144:0/2275886731 shutdown_connections 2024-04-20T21:17:30.373 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.373+0000 7f0886e00640 1 -- 172.21.15.144:0/2275886731 wait complete. 2024-04-20T21:17:30.375 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.375+0000 7f0886e00640 1 Processor -- start 2024-04-20T21:17:30.375 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.375+0000 7f0886e00640 1 -- start start 2024-04-20T21:17:30.378 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.376+0000 7f0886e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f088015ca00 0x7f08801519c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:30.378 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.376+0000 7f0886e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0880160310 0x7f08800b0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:30.378 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.376+0000 7f0886e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08801606f0 0x7f08800b08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:30.378 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.376+0000 7f0886e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f088009b950 con 0x7f08801606f0 2024-04-20T21:17:30.378 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.376+0000 7f0886e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0880160b50 con 0x7f088015ca00 2024-04-20T21:17:30.378 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.376+0000 7f0886e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f088009bad0 con 0x7f0880160310 2024-04-20T21:17:30.378 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.377+0000 7f0885a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f088015ca00 0x7f08801519c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:30.378 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.377+0000 7f0885000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0880160310 0x7f08800b0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:30.379 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.377+0000 7f0885a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f088015ca00 0x7f08801519c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35060/0 (socket says 172.21.15.144:35060) 2024-04-20T21:17:30.379 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.377+0000 7f0885a00640 1 -- 172.21.15.144:0/162660746 learned_addr learned my addr 172.21.15.144:0/162660746 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-04-20T21:17:30.379 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.377+0000 7f0886400640 1 --2- 172.21.15.144:0/162660746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08801606f0 0x7f08800b08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:30.379 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.377+0000 7f0885a00640 1 -- 172.21.15.144:0/162660746 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0880160310 msgr2=0x7f08800b0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:30.379 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.377+0000 7f0885a00640 1 --2- 172.21.15.144:0/162660746 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0880160310 0x7f08800b0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:30.379 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.377+0000 7f0885a00640 1 -- 172.21.15.144:0/162660746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08801606f0 msgr2=0x7f08800b08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:30.379 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.377+0000 7f0885a00640 1 --2- 172.21.15.144:0/162660746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08801606f0 0x7f08800b08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:30.379 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.377+0000 7f0885a00640 1 -- 172.21.15.144:0/162660746 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f088006ca90 con 0x7f088015ca00 2024-04-20T21:17:30.379 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.377+0000 7f0885a00640 1 --2- 172.21.15.144:0/162660746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f088015ca00 0x7f08801519c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0874037040 tx=0x7f08740046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:30.379 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.378+0000 7f087d600640 1 -- 172.21.15.144:0/162660746 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0874043070 con 0x7f088015ca00 2024-04-20T21:17:30.379 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.378+0000 7f087d600640 1 -- 172.21.15.144:0/162660746 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08740346a0 con 0x7f088015ca00 2024-04-20T21:17:30.379 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.378+0000 7f087d600640 1 -- 172.21.15.144:0/162660746 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0874039040 con 0x7f088015ca00 2024-04-20T21:17:30.379 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.378+0000 7f0886e00640 1 -- 172.21.15.144:0/162660746 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08800cb1a0 con 0x7f088015ca00 2024-04-20T21:17:30.380 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.378+0000 7f0886e00640 1 -- 172.21.15.144:0/162660746 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f088006cd50 con 0x7f088015ca00 2024-04-20T21:17:30.380 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.378+0000 7f087d600640 1 -- 172.21.15.144:0/162660746 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0874046070 con 0x7f088015ca00 2024-04-20T21:17:30.380 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.378+0000 7f087d600640 1 -- 172.21.15.144:0/162660746 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f08740315b0 con 0x7f088015ca00 2024-04-20T21:17:30.380 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.379+0000 7f0886e00640 1 -- 172.21.15.144:0/162660746 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08800b3b00 con 0x7f088015ca00 2024-04-20T21:17:30.381 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.381+0000 7f087d600640 1 -- 172.21.15.144:0/162660746 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f08800b3b00 con 0x7f088015ca00 2024-04-20T21:17:30.399 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-20T21:17:30.400+0000 7f0a0524b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-20T21:17:30.459 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.460+0000 7f0886e00640 1 -- 172.21.15.144:0/162660746 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f6e35481-399e-4da1-be5c-bfe52662d6be", "id": 6} v 0) -- 0x7f088004fd90 con 0x7f088015ca00 2024-04-20T21:17:30.467 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.467+0000 7f087d600640 1 -- 172.21.15.144:0/162660746 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f6e35481-399e-4da1-be5c-bfe52662d6be", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f0874034a90 con 0x7f088015ca00 2024-04-20T21:17:30.468 INFO:teuthology.orchestra.run.smithi144.stdout:6 2024-04-20T21:17:30.469 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.469+0000 7f0886e00640 1 -- 172.21.15.144:0/162660746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f088015ca00 msgr2=0x7f08801519c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:30.469 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.469+0000 7f0886e00640 1 --2- 172.21.15.144:0/162660746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f088015ca00 0x7f08801519c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0874037040 tx=0x7f08740046c0 comp rx=0 tx=0).stop 2024-04-20T21:17:30.469 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.469+0000 7f0886e00640 1 -- 172.21.15.144:0/162660746 shutdown_connections 2024-04-20T21:17:30.469 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.469+0000 7f0886e00640 1 --2- 172.21.15.144:0/162660746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08801606f0 0x7f08800b08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:30.469 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.469+0000 7f0886e00640 1 --2- 172.21.15.144:0/162660746 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0880160310 0x7f08800b0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:30.469 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.469+0000 7f0886e00640 1 --2- 172.21.15.144:0/162660746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f088015ca00 0x7f08801519c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:30.469 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.469+0000 7f0886e00640 1 -- 172.21.15.144:0/162660746 >> 172.21.15.144:0/162660746 conn(0x7f08800d2530 msgr2=0x7f08800b33f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:30.469 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.469+0000 7f0886e00640 1 -- 172.21.15.144:0/162660746 shutdown_connections 2024-04-20T21:17:30.469 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.469+0000 7f0886e00640 1 -- 172.21.15.144:0/162660746 wait complete. 2024-04-20T21:17:30.486 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster ceph osd new d965de5d-e7cf-40fc-99b3-42ea692295a7 7 2024-04-20T21:17:30.493 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-04-20T21:17:30.494+0000 7fe5f8bc0040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-20T21:17:30.658 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.658+0000 7fec2b200640 1 Processor -- start 2024-04-20T21:17:30.658 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.658+0000 7fec2b200640 1 -- start start 2024-04-20T21:17:30.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.660+0000 7fec2b200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec2c15e8e0 0x7fec2c06d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:30.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.660+0000 7fec2b200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fec2c15ecc0 0x7fec2c0d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:30.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.660+0000 7fec2b200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec2c0d1f60 con 0x7fec2c06d740 2024-04-20T21:17:30.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.660+0000 7fec2b200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fec2c0cf7f0 con 0x7fec2c15ecc0 2024-04-20T21:17:30.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.660+0000 7fec2b200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fec2c0d19a0 con 0x7fec2c15e8e0 2024-04-20T21:17:30.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.660+0000 7fec29e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec2c15e8e0 0x7fec2c06d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:30.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.660+0000 7fec23e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fec2c15ecc0 0x7fec2c0d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:30.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.660+0000 7fec2a800640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fec2c06d740 0x7fec2c0cfc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.144:52852/0 (socket says 172.21.15.144:52852) 2024-04-20T21:17:30.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.660+0000 7fec2a800640 1 -- 172.21.15.144:0/1383361424 learned_addr learned my addr 172.21.15.144:0/1383361424 (peer_addr_for_me v1:172.21.15.144:0/0) 2024-04-20T21:17:30.661 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.660+0000 7fec29400640 1 -- 172.21.15.144:0/1383361424 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 670244498 0 0) 0x7fec2c0d1f60 con 0x7fec2c06d740 2024-04-20T21:17:30.661 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.661+0000 7fec29400640 1 -- 172.21.15.144:0/1383361424 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec10003520 con 0x7fec2c06d740 2024-04-20T21:17:30.661 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.661+0000 7fec29e00640 1 -- 172.21.15.144:0/1383361424 >> v2:172.21.15.144:3300/0 conn(0x7fec2c15ecc0 msgr2=0x7fec2c0d4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:30.661 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.661+0000 7fec29e00640 1 --2- 172.21.15.144:0/1383361424 >> v2:172.21.15.144:3300/0 conn(0x7fec2c15ecc0 0x7fec2c0d4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:30.661 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.661+0000 7fec29e00640 1 -- 172.21.15.144:0/1383361424 >> v1:172.21.15.144:6789/0 conn(0x7fec2c06d740 legacy=0x7fec2c0cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:30.661 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.661+0000 7fec29e00640 1 -- 172.21.15.144:0/1383361424 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec2c0d1c80 con 0x7fec2c15e8e0 2024-04-20T21:17:30.662 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.661+0000 7fec23e00640 1 --2- 172.21.15.144:0/1383361424 >> v2:172.21.15.144:3300/0 conn(0x7fec2c15ecc0 0x7fec2c0d4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:30.662 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.662+0000 7fec29e00640 1 --2- 172.21.15.144:0/1383361424 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec2c15e8e0 0x7fec2c06d200 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fec1c0097a0 tx=0x7fec1c02b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2a489c8561431e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:30.663 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.662+0000 7fec29400640 1 -- 172.21.15.144:0/1383361424 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec1c035020 con 0x7fec2c15e8e0 2024-04-20T21:17:30.663 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.662+0000 7fec29400640 1 -- 172.21.15.144:0/1383361424 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec1c031810 con 0x7fec2c15e8e0 2024-04-20T21:17:30.663 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.662+0000 7fec29400640 1 -- 172.21.15.144:0/1383361424 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec1c031be0 con 0x7fec2c15e8e0 2024-04-20T21:17:30.663 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.663+0000 7fec2b200640 1 -- 172.21.15.144:0/1383361424 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec2c15e8e0 msgr2=0x7fec2c06d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:30.663 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.663+0000 7fec2b200640 1 --2- 172.21.15.144:0/1383361424 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec2c15e8e0 0x7fec2c06d200 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fec1c0097a0 tx=0x7fec1c02b8b0 comp rx=0 tx=0).stop 2024-04-20T21:17:30.663 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.663+0000 7fec2b200640 1 -- 172.21.15.144:0/1383361424 shutdown_connections 2024-04-20T21:17:30.663 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.663+0000 7fec2b200640 1 --2- 172.21.15.144:0/1383361424 >> v2:172.21.15.144:3300/0 conn(0x7fec2c15ecc0 0x7fec2c0d4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:30.663 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.663+0000 7fec2b200640 1 --2- 172.21.15.144:0/1383361424 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec2c15e8e0 0x7fec2c06d200 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:30.663 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.663+0000 7fec2b200640 1 -- 172.21.15.144:0/1383361424 >> 172.21.15.144:0/1383361424 conn(0x7fec2c09d9d0 msgr2=0x7fec2c08bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:30.663 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.663+0000 7fec2b200640 1 -- 172.21.15.144:0/1383361424 shutdown_connections 2024-04-20T21:17:30.663 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.663+0000 7fec2b200640 1 -- 172.21.15.144:0/1383361424 wait complete. 2024-04-20T21:17:30.664 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.664+0000 7fec2b200640 1 Processor -- start 2024-04-20T21:17:30.664 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.665+0000 7fec2b200640 1 -- start start 2024-04-20T21:17:30.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.665+0000 7fec2b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec2c06d740 0x7fec2c0ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:30.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.665+0000 7fec2b200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec2c15e8e0 0x7fec2c0ca970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:30.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.666+0000 7fec2b200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec2c0ccd80 0x7fec2c0c8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:30.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.666+0000 7fec2b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fec2c0d0510 con 0x7fec2c06d740 2024-04-20T21:17:30.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.666+0000 7fec2b200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fec2c0d0390 con 0x7fec2c0ccd80 2024-04-20T21:17:30.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.666+0000 7fec2b200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fec2c0d0690 con 0x7fec2c15e8e0 2024-04-20T21:17:30.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.666+0000 7fec23e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec2c15e8e0 0x7fec2c0ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:30.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.666+0000 7fec23e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec2c15e8e0 0x7fec2c0ca970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.144:44702/0 (socket says 172.21.15.144:44702) 2024-04-20T21:17:30.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.666+0000 7fec23e00640 1 -- 172.21.15.144:0/4112915364 learned_addr learned my addr 172.21.15.144:0/4112915364 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-04-20T21:17:30.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.666+0000 7fec2a800640 1 --2- 172.21.15.144:0/4112915364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec2c0ccd80 0x7fec2c0c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:30.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.666+0000 7fec29e00640 1 --2- 172.21.15.144:0/4112915364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec2c06d740 0x7fec2c0ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:30.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.666+0000 7fec23e00640 1 -- 172.21.15.144:0/4112915364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec2c0ccd80 msgr2=0x7fec2c0c8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:30.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.666+0000 7fec23e00640 1 --2- 172.21.15.144:0/4112915364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec2c0ccd80 0x7fec2c0c8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:30.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.666+0000 7fec23e00640 1 -- 172.21.15.144:0/4112915364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec2c06d740 msgr2=0x7fec2c0ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:30.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.666+0000 7fec23e00640 1 --2- 172.21.15.144:0/4112915364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec2c06d740 0x7fec2c0ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:30.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.666+0000 7fec23e00640 1 -- 172.21.15.144:0/4112915364 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec2c0cd310 con 0x7fec2c15e8e0 2024-04-20T21:17:30.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.666+0000 7fec29e00640 1 --2- 172.21.15.144:0/4112915364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec2c06d740 0x7fec2c0ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:17:30.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.667+0000 7fec23e00640 1 --2- 172.21.15.144:0/4112915364 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec2c15e8e0 0x7fec2c0ca970 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fec1400e960 tx=0x7fec1400ee20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:30.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.667+0000 7fec22000640 1 -- 172.21.15.144:0/4112915364 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec14017070 con 0x7fec2c15e8e0 2024-04-20T21:17:30.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.667+0000 7fec22000640 1 -- 172.21.15.144:0/4112915364 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec1400ca00 con 0x7fec2c15e8e0 2024-04-20T21:17:30.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.667+0000 7fec22000640 1 -- 172.21.15.144:0/4112915364 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec1400cdd0 con 0x7fec2c15e8e0 2024-04-20T21:17:30.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.667+0000 7fec2b200640 1 -- 172.21.15.144:0/4112915364 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec2c0cd5f0 con 0x7fec2c15e8e0 2024-04-20T21:17:30.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.667+0000 7fec2b200640 1 -- 172.21.15.144:0/4112915364 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fec2c0cdbb0 con 0x7fec2c15e8e0 2024-04-20T21:17:30.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.667+0000 7fec22000640 1 -- 172.21.15.144:0/4112915364 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fec1401a070 con 0x7fec2c15e8e0 2024-04-20T21:17:30.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.667+0000 7fec22000640 1 -- 172.21.15.144:0/4112915364 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7fec14009400 con 0x7fec2c15e8e0 2024-04-20T21:17:30.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.668+0000 7fec2b200640 1 -- 172.21.15.144:0/4112915364 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec2c04fd90 con 0x7fec2c15e8e0 2024-04-20T21:17:30.670 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.670+0000 7fec22000640 1 -- 172.21.15.144:0/4112915364 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7fec14013470 con 0x7fec2c15e8e0 2024-04-20T21:17:30.693 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-20T21:17:30.694+0000 7f0a0524b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-20T21:17:30.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.749+0000 7fec2b200640 1 -- 172.21.15.144:0/4112915364 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d965de5d-e7cf-40fc-99b3-42ea692295a7", "id": 7} v 0) -- 0x7fec2c06cd10 con 0x7fec2c15e8e0 2024-04-20T21:17:30.754 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-20T21:17:30.755+0000 7f0a0524b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-20T21:17:30.760 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.761+0000 7fec22000640 1 -- 172.21.15.144:0/4112915364 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d965de5d-e7cf-40fc-99b3-42ea692295a7", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7fec1402a020 con 0x7fec2c15e8e0 2024-04-20T21:17:30.761 INFO:teuthology.orchestra.run.smithi144.stdout:7 2024-04-20T21:17:30.762 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.762+0000 7fec2b200640 1 -- 172.21.15.144:0/4112915364 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec2c15e8e0 msgr2=0x7fec2c0ca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:30.762 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.762+0000 7fec2b200640 1 --2- 172.21.15.144:0/4112915364 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec2c15e8e0 0x7fec2c0ca970 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fec1400e960 tx=0x7fec1400ee20 comp rx=0 tx=0).stop 2024-04-20T21:17:30.762 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.762+0000 7fec2b200640 1 -- 172.21.15.144:0/4112915364 shutdown_connections 2024-04-20T21:17:30.762 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.762+0000 7fec2b200640 1 --2- 172.21.15.144:0/4112915364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec2c0ccd80 0x7fec2c0c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:30.762 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.762+0000 7fec2b200640 1 --2- 172.21.15.144:0/4112915364 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec2c15e8e0 0x7fec2c0ca970 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:30.762 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.762+0000 7fec2b200640 1 --2- 172.21.15.144:0/4112915364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec2c06d740 0x7fec2c0ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:30.762 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.762+0000 7fec2b200640 1 -- 172.21.15.144:0/4112915364 >> 172.21.15.144:0/4112915364 conn(0x7fec2c09d9d0 msgr2=0x7fec2c15f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:30.763 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.762+0000 7fec2b200640 1 -- 172.21.15.144:0/4112915364 shutdown_connections 2024-04-20T21:17:30.763 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:17:30.762+0000 7fec2b200640 1 -- 172.21.15.144:0/4112915364 wait complete. 2024-04-20T21:17:30.778 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-20T21:17:30.778 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-20T21:17:30.786 INFO:tasks.ceph.osd.0:Started 2024-04-20T21:17:30.786 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-20T21:17:30.786 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-20T21:17:30.796 INFO:tasks.ceph.osd.1:Started 2024-04-20T21:17:30.797 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-20T21:17:30.797 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-20T21:17:30.803 INFO:tasks.ceph.osd.2:Started 2024-04-20T21:17:30.803 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-20T21:17:30.804 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-20T21:17:30.806 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-04-20T21:17:30.806+0000 7fe5f8bc0040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-20T21:17:30.809 INFO:tasks.ceph.osd.3:Started 2024-04-20T21:17:30.809 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-20T21:17:30.810 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-20T21:17:30.815 INFO:tasks.ceph.osd.4:Started 2024-04-20T21:17:30.816 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-20T21:17:30.816 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-20T21:17:30.821 INFO:tasks.ceph.osd.5:Started 2024-04-20T21:17:30.821 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-20T21:17:30.821 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-20T21:17:30.826 INFO:tasks.ceph.osd.6:Started 2024-04-20T21:17:30.826 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-20T21:17:30.827 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-20T21:17:30.832 INFO:tasks.ceph.osd.7:Started 2024-04-20T21:17:30.832 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:17:30.891 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-04-20T21:17:30.891+0000 7fe5f8bc0040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-20T21:17:30.930 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-20T21:17:30.931+0000 7f0a0524b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-20T21:17:31.053 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-04-20T21:17:31.053+0000 7fe5f8bc0040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-20T21:17:31.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.072+0000 7fab1aa00640 1 Processor -- start 2024-04-20T21:17:31.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.072+0000 7fab1aa00640 1 -- start start 2024-04-20T21:17:31.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.074+0000 7fab1aa00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fab1415eb10 0x7fab1409a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:31.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.074+0000 7fab1aa00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab1409b330 con 0x7fab1409af20 2024-04-20T21:17:31.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.074+0000 7fab1aa00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab1406eaa0 con 0x7fab1415e730 2024-04-20T21:17:31.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.074+0000 7fab1aa00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fab140713c0 con 0x7fab1415eb10 2024-04-20T21:17:31.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.074+0000 7fab18a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fab1409af20 0x7fab141687d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57582/0 (socket says 172.21.15.86:57582) 2024-04-20T21:17:31.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.074+0000 7fab18a00640 1 -- 172.21.15.86:0/3361750909 learned_addr learned my addr 172.21.15.86:0/3361750909 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:17:31.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.074+0000 7fab13400640 1 --2- 172.21.15.86:0/3361750909 >> v2:172.21.15.144:3300/0 conn(0x7fab1415eb10 0x7fab1409a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:31.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.075+0000 7fab12a00640 1 -- 172.21.15.86:0/3361750909 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4234188696 0 0) 0x7fab0c001490 con 0x7fab1409af20 2024-04-20T21:17:31.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.075+0000 7fab12a00640 1 -- 172.21.15.86:0/3361750909 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faaf8003520 con 0x7fab1409af20 2024-04-20T21:17:31.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.075+0000 7fab13400640 1 -- 172.21.15.86:0/3361750909 >> v1:172.21.15.144:6789/0 conn(0x7fab1415e730 legacy=0x7fab140d1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:31.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.075+0000 7fab13400640 1 -- 172.21.15.86:0/3361750909 >> v1:172.21.15.86:6789/0 conn(0x7fab1409af20 legacy=0x7fab141687d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:31.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.075+0000 7fab13400640 1 -- 172.21.15.86:0/3361750909 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab14070320 con 0x7fab1415eb10 2024-04-20T21:17:31.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.076+0000 7fab13400640 1 --2- 172.21.15.86:0/3361750909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab1415eb10 0x7fab1409a9e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fab000097a0 tx=0x7fab0002b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2a7667a4dd819f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:31.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.076+0000 7fab12a00640 1 -- 172.21.15.86:0/3361750909 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0002f810 con 0x7fab1415eb10 2024-04-20T21:17:31.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.076+0000 7fab12a00640 1 -- 172.21.15.86:0/3361750909 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab0002f9b0 con 0x7fab1415eb10 2024-04-20T21:17:31.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.076+0000 7fab12a00640 1 -- 172.21.15.86:0/3361750909 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0002fd80 con 0x7fab1415eb10 2024-04-20T21:17:31.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.077+0000 7fab1aa00640 1 -- 172.21.15.86:0/3361750909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab1415eb10 msgr2=0x7fab1409a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:31.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.077+0000 7fab1aa00640 1 --2- 172.21.15.86:0/3361750909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab1415eb10 0x7fab1409a9e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fab000097a0 tx=0x7fab0002b9f0 comp rx=0 tx=0).stop 2024-04-20T21:17:31.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.077+0000 7fab1aa00640 1 -- 172.21.15.86:0/3361750909 shutdown_connections 2024-04-20T21:17:31.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.077+0000 7fab1aa00640 1 --2- 172.21.15.86:0/3361750909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab1415eb10 0x7fab1409a9e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:31.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.077+0000 7fab1aa00640 1 -- 172.21.15.86:0/3361750909 >> 172.21.15.86:0/3361750909 conn(0x7fab140b4080 msgr2=0x7fab140b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:31.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.077+0000 7fab1aa00640 1 -- 172.21.15.86:0/3361750909 shutdown_connections 2024-04-20T21:17:31.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.077+0000 7fab1aa00640 1 -- 172.21.15.86:0/3361750909 wait complete. 2024-04-20T21:17:31.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.080+0000 7fab1aa00640 1 Processor -- start 2024-04-20T21:17:31.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.080+0000 7fab1aa00640 1 -- start start 2024-04-20T21:17:31.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.081+0000 7fab1aa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab1409af20 0x7fab140b5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:31.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.081+0000 7fab1aa00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab1415e730 0x7fab140c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:31.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.081+0000 7fab1aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab1415eb10 0x7fab14157650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:31.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.081+0000 7fab1aa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fab14070bc0 con 0x7fab1415eb10 2024-04-20T21:17:31.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.081+0000 7fab1aa00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fab140708e0 con 0x7fab1415e730 2024-04-20T21:17:31.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.081+0000 7fab1aa00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fab14070ea0 con 0x7fab1409af20 2024-04-20T21:17:31.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.081+0000 7fab18a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab1415eb10 0x7fab14157650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:31.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.082+0000 7fab13400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab1415e730 0x7fab140c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:31.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.082+0000 7fab18a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab1415eb10 0x7fab14157650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48592/0 (socket says 172.21.15.86:48592) 2024-04-20T21:17:31.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.082+0000 7fab18a00640 1 -- 172.21.15.86:0/1004109371 learned_addr learned my addr 172.21.15.86:0/1004109371 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:31.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.082+0000 7fab13e00640 1 --2- 172.21.15.86:0/1004109371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab1409af20 0x7fab140b5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:31.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.082+0000 7fab18a00640 1 -- 172.21.15.86:0/1004109371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab1409af20 msgr2=0x7fab140b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:31.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.082+0000 7fab18a00640 1 --2- 172.21.15.86:0/1004109371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab1409af20 0x7fab140b5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:31.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.082+0000 7fab18a00640 1 -- 172.21.15.86:0/1004109371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab1415e730 msgr2=0x7fab140c8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:31.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.082+0000 7fab18a00640 1 --2- 172.21.15.86:0/1004109371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab1415e730 0x7fab140c8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:31.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.082+0000 7fab18a00640 1 -- 172.21.15.86:0/1004109371 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab14157b90 con 0x7fab1415eb10 2024-04-20T21:17:31.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.082+0000 7fab18a00640 1 --2- 172.21.15.86:0/1004109371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab1415eb10 0x7fab14157650 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fab0c00cb50 tx=0x7fab0c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:31.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.082+0000 7fab0be00640 1 -- 172.21.15.86:0/1004109371 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0c017070 con 0x7fab1415eb10 2024-04-20T21:17:31.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.082+0000 7fab0be00640 1 -- 172.21.15.86:0/1004109371 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab0c00ad40 con 0x7fab1415eb10 2024-04-20T21:17:31.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.082+0000 7fab0be00640 1 -- 172.21.15.86:0/1004109371 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0c012620 con 0x7fab1415eb10 2024-04-20T21:17:31.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.082+0000 7fab1aa00640 1 -- 172.21.15.86:0/1004109371 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab140b61a0 con 0x7fab1415eb10 2024-04-20T21:17:31.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.083+0000 7fab1aa00640 1 -- 172.21.15.86:0/1004109371 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab140c6540 con 0x7fab1415eb10 2024-04-20T21:17:31.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.083+0000 7fab0be00640 1 -- 172.21.15.86:0/1004109371 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fab0c01a070 con 0x7fab1415eb10 2024-04-20T21:17:31.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.083+0000 7fab0be00640 1 -- 172.21.15.86:0/1004109371 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fab0c01c440 con 0x7fab1415eb10 2024-04-20T21:17:31.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.084+0000 7fab1aa00640 1 -- 172.21.15.86:0/1004109371 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab140d1b30 con 0x7fab1415eb10 2024-04-20T21:17:31.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.086+0000 7fab0be00640 1 -- 172.21.15.86:0/1004109371 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7fab140d1b30 con 0x7fab1415eb10 2024-04-20T21:17:31.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.150+0000 7fab1aa00640 1 -- 172.21.15.86:0/1004109371 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fab140689e0 con 0x7fab1415eb10 2024-04-20T21:17:31.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.151+0000 7fab0be00640 1 -- 172.21.15.86:0/1004109371 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fab0c021080 con 0x7fab1415eb10 2024-04-20T21:17:31.151 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:17:31.151 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":10,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:17:30.752446+0000","last_up_change":"0.000000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:17:31.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.152+0000 7fab1aa00640 1 -- 172.21.15.86:0/1004109371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab1415eb10 msgr2=0x7fab14157650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:31.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.152+0000 7fab1aa00640 1 --2- 172.21.15.86:0/1004109371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab1415eb10 0x7fab14157650 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fab0c00cb50 tx=0x7fab0c00a430 comp rx=0 tx=0).stop 2024-04-20T21:17:31.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.152+0000 7fab1aa00640 1 -- 172.21.15.86:0/1004109371 shutdown_connections 2024-04-20T21:17:31.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.152+0000 7fab1aa00640 1 --2- 172.21.15.86:0/1004109371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab1415eb10 0x7fab14157650 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:31.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.153+0000 7fab1aa00640 1 --2- 172.21.15.86:0/1004109371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab1415e730 0x7fab140c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:31.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.153+0000 7fab1aa00640 1 --2- 172.21.15.86:0/1004109371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab1409af20 0x7fab140b5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:31.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.153+0000 7fab1aa00640 1 -- 172.21.15.86:0/1004109371 >> 172.21.15.86:0/1004109371 conn(0x7fab140b4080 msgr2=0x7fab1415d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:31.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.153+0000 7fab1aa00640 1 -- 172.21.15.86:0/1004109371 shutdown_connections 2024-04-20T21:17:31.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:31.153+0000 7fab1aa00640 1 -- 172.21.15.86:0/1004109371 wait complete. 2024-04-20T21:17:31.169 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-20T21:17:31.169 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-20T21:17:31.323 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-20T21:17:31.324+0000 7f0a0524b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-20T21:17:31.378 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-20T21:17:31.379+0000 7f0a0524b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-20T21:17:31.381 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-04-20T21:17:31.381+0000 7fe5f8bc0040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-20T21:17:31.436 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-04-20T21:17:31.435+0000 7fe5f8bc0040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-20T21:17:31.483 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-20T21:17:31.483+0000 7f95b5c1b640 -1 Falling back to public interface 2024-04-20T21:17:31.494 INFO:tasks.ceph.osd.3.smithi086.stderr:2024-04-20T21:17:31.493+0000 7fbeedc0f640 -1 Falling back to public interface 2024-04-20T21:17:31.495 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-04-20T21:17:31.496+0000 7fa38b4b5640 -1 Falling back to public interface 2024-04-20T21:17:31.503 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-04-20T21:17:31.503+0000 7f007f22d640 -1 Falling back to public interface 2024-04-20T21:17:31.512 INFO:tasks.ceph.osd.1.smithi086.stderr:2024-04-20T21:17:31.513+0000 7fda25c0f640 -1 Falling back to public interface 2024-04-20T21:17:31.512 INFO:tasks.ceph.osd.2.smithi086.stderr:2024-04-20T21:17:31.513+0000 7f8043dff640 -1 Falling back to public interface 2024-04-20T21:17:31.524 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-04-20T21:17:31.524+0000 7fe6c6011640 -1 Falling back to public interface 2024-04-20T21:17:31.531 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-04-20T21:17:31.531+0000 7f3b71014640 -1 Falling back to public interface 2024-04-20T21:17:31.563 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-20T21:17:31.564+0000 7f0a0524b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-20T21:17:31.627 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-04-20T21:17:31.627+0000 7fe5f8bc0040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-20T21:17:31.729 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-20T21:17:31.730+0000 7f0a0524b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-20T21:17:31.789 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-20T21:17:31.790+0000 7f0a0524b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-20T21:17:31.791 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-04-20T21:17:31.791+0000 7fe5f8bc0040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-20T21:17:31.851 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-04-20T21:17:31.852+0000 7fe5f8bc0040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-20T21:17:31.870 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-20T21:17:31.871+0000 7f0a0524b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-20T21:17:31.923 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-20T21:17:31.923+0000 7f0a0524b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-20T21:17:31.932 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-04-20T21:17:31.933+0000 7fe5f8bc0040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-20T21:17:31.975 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-20T21:17:31.984 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-04-20T21:17:31.984+0000 7fe5f8bc0040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-20T21:17:32.136 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.137+0000 7f69a9e00640 1 Processor -- start 2024-04-20T21:17:32.136 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.137+0000 7f69a9e00640 1 -- start start 2024-04-20T21:17:32.137 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.138+0000 7f69a9e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f69a40cd7b0 0x7f69a40b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:32.138 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.138+0000 7f69a9e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f69a40cdc10 0x7f69a40b4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:32.138 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.138+0000 7f69a9e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69a415f230 con 0x7f69a406ee70 2024-04-20T21:17:32.138 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.138+0000 7f69a9e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f69a41600e0 con 0x7f69a40cdc10 2024-04-20T21:17:32.138 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.138+0000 7f69a9e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f69a415f4d0 con 0x7f69a40cd7b0 2024-04-20T21:17:32.138 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.138+0000 7f69a3400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f69a40cdc10 0x7f69a40b4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:32.138 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.138+0000 7f69a3400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f69a40cdc10 0x7f69a40b4500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48604/0 (socket says 172.21.15.86:48604) 2024-04-20T21:17:32.138 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.138+0000 7f69a3400640 1 -- 172.21.15.86:0/1233221403 learned_addr learned my addr 172.21.15.86:0/1233221403 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:32.138 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.139+0000 7f699be00640 1 --2- 172.21.15.86:0/1233221403 >> v2:172.21.15.144:3300/0 conn(0x7f69a40cd7b0 0x7f69a40b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:32.139 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.139+0000 7f69a3400640 1 -- 172.21.15.86:0/1233221403 >> v2:172.21.15.144:3300/0 conn(0x7f69a40cd7b0 msgr2=0x7f69a40b3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:32.139 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.139+0000 7f69a3400640 1 --2- 172.21.15.86:0/1233221403 >> v2:172.21.15.144:3300/0 conn(0x7f69a40cd7b0 0x7f69a40b3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:32.139 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.139+0000 7f69a3400640 1 -- 172.21.15.86:0/1233221403 >> v1:172.21.15.144:6789/0 conn(0x7f69a406ee70 legacy=0x7f69a406f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:32.139 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.139+0000 7f69a3400640 1 -- 172.21.15.86:0/1233221403 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69a415f7b0 con 0x7f69a40cdc10 2024-04-20T21:17:32.139 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.140+0000 7f69a3400640 1 --2- 172.21.15.86:0/1233221403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69a40cdc10 0x7f69a40b4500 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f69a406ff20 tx=0x7f699c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daf909093b9fea59 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:32.140 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.140+0000 7f69a2000640 1 -- 172.21.15.86:0/1233221403 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699c02ca80 con 0x7f69a40cdc10 2024-04-20T21:17:32.140 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.140+0000 7f69a2000640 1 -- 172.21.15.86:0/1233221403 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f699c02cc20 con 0x7f69a40cdc10 2024-04-20T21:17:32.140 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.140+0000 7f69a2000640 1 -- 172.21.15.86:0/1233221403 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699c0343f0 con 0x7f69a40cdc10 2024-04-20T21:17:32.141 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.141+0000 7f69a9e00640 1 -- 172.21.15.86:0/1233221403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69a40cdc10 msgr2=0x7f69a40b4500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:32.141 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.141+0000 7f69a9e00640 1 --2- 172.21.15.86:0/1233221403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69a40cdc10 0x7f69a40b4500 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f69a406ff20 tx=0x7f699c02bee0 comp rx=0 tx=0).stop 2024-04-20T21:17:32.141 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.141+0000 7f69a9e00640 1 -- 172.21.15.86:0/1233221403 shutdown_connections 2024-04-20T21:17:32.141 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.141+0000 7f69a9e00640 1 --2- 172.21.15.86:0/1233221403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69a40cdc10 0x7f69a40b4500 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:32.141 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.141+0000 7f69a9e00640 1 --2- 172.21.15.86:0/1233221403 >> v2:172.21.15.144:3300/0 conn(0x7f69a40cd7b0 0x7f69a40b3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:32.141 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.141+0000 7f69a9e00640 1 -- 172.21.15.86:0/1233221403 >> 172.21.15.86:0/1233221403 conn(0x7f69a4074fd0 msgr2=0x7f69a406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:32.141 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.141+0000 7f69a9e00640 1 -- 172.21.15.86:0/1233221403 shutdown_connections 2024-04-20T21:17:32.141 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.141+0000 7f69a9e00640 1 -- 172.21.15.86:0/1233221403 wait complete. 2024-04-20T21:17:32.142 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.143+0000 7f69a9e00640 1 Processor -- start 2024-04-20T21:17:32.142 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.143+0000 7f69a9e00640 1 -- start start 2024-04-20T21:17:32.144 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.144+0000 7f69a9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69a406ee70 0x7f69a415e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:32.144 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.144+0000 7f69a9e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69a40cd7b0 0x7f69a4157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:32.144 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.144+0000 7f69a9e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69a40cdc10 0x7f69a4140950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:32.144 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.144+0000 7f69a9e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f69a40cd040 con 0x7f69a406ee70 2024-04-20T21:17:32.144 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.144+0000 7f69a9e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f69a415fd70 con 0x7f69a40cd7b0 2024-04-20T21:17:32.144 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.144+0000 7f69a9e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f69a4141fe0 con 0x7f69a40cdc10 2024-04-20T21:17:32.144 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.144+0000 7f69a2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69a406ee70 0x7f69a415e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:32.145 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.144+0000 7f69a2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69a406ee70 0x7f69a415e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48612/0 (socket says 172.21.15.86:48612) 2024-04-20T21:17:32.145 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.144+0000 7f69a2a00640 1 -- 172.21.15.86:0/4141067759 learned_addr learned my addr 172.21.15.86:0/4141067759 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:32.145 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.144+0000 7f699be00640 1 --2- 172.21.15.86:0/4141067759 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69a40cd7b0 0x7f69a4157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:32.145 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.145+0000 7f69a3400640 1 --2- 172.21.15.86:0/4141067759 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69a40cdc10 0x7f69a4140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:32.145 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.145+0000 7f69a2a00640 1 -- 172.21.15.86:0/4141067759 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69a40cdc10 msgr2=0x7f69a4140950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:32.145 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.145+0000 7f69a2a00640 1 --2- 172.21.15.86:0/4141067759 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69a40cdc10 0x7f69a4140950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:32.145 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.145+0000 7f69a2a00640 1 -- 172.21.15.86:0/4141067759 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69a40cd7b0 msgr2=0x7f69a4157bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:32.145 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.145+0000 7f69a2a00640 1 --2- 172.21.15.86:0/4141067759 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69a40cd7b0 0x7f69a4157bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:32.145 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.145+0000 7f69a2a00640 1 -- 172.21.15.86:0/4141067759 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69a41f7d80 con 0x7f69a406ee70 2024-04-20T21:17:32.145 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.145+0000 7f69a2a00640 1 --2- 172.21.15.86:0/4141067759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69a406ee70 0x7f69a415e920 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f699400c430 tx=0x7f699400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:32.146 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.145+0000 7f699b400640 1 -- 172.21.15.86:0/4141067759 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6994011790 con 0x7f69a406ee70 2024-04-20T21:17:32.146 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.146+0000 7f699b400640 1 -- 172.21.15.86:0/4141067759 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6994017070 con 0x7f69a406ee70 2024-04-20T21:17:32.146 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.146+0000 7f69a9e00640 1 -- 172.21.15.86:0/4141067759 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69a41f8060 con 0x7f69a406ee70 2024-04-20T21:17:32.147 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.146+0000 7f699b400640 1 -- 172.21.15.86:0/4141067759 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6994011bb0 con 0x7f69a406ee70 2024-04-20T21:17:32.147 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.146+0000 7f69a9e00640 1 -- 172.21.15.86:0/4141067759 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69a41f8620 con 0x7f69a406ee70 2024-04-20T21:17:32.147 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.146+0000 7f699b400640 1 -- 172.21.15.86:0/4141067759 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f699401c460 con 0x7f69a406ee70 2024-04-20T21:17:32.147 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.146+0000 7f699b400640 1 -- 172.21.15.86:0/4141067759 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f6994011d50 con 0x7f69a406ee70 2024-04-20T21:17:32.147 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.147+0000 7f69a9e00640 1 -- 172.21.15.86:0/4141067759 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69a404fcf0 con 0x7f69a406ee70 2024-04-20T21:17:32.148 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.149+0000 7f699b400640 1 -- 172.21.15.86:0/4141067759 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f6994012740 con 0x7f69a406ee70 2024-04-20T21:17:32.215 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.216+0000 7f69a9e00640 1 -- 172.21.15.86:0/4141067759 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f69a40b3780 con 0x7f69a406ee70 2024-04-20T21:17:32.216 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.217+0000 7f699b400640 1 -- 172.21.15.86:0/4141067759 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f6994012920 con 0x7f69a406ee70 2024-04-20T21:17:32.217 INFO:teuthology.misc.health.smithi086.stdout: 2024-04-20T21:17:32.217 INFO:teuthology.misc.health.smithi086.stdout:{"epoch":10,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:17:30.752446+0000","last_up_change":"0.000000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:17:32.218 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.218+0000 7f69a9e00640 1 -- 172.21.15.86:0/4141067759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69a406ee70 msgr2=0x7f69a415e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:32.218 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.218+0000 7f69a9e00640 1 --2- 172.21.15.86:0/4141067759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69a406ee70 0x7f69a415e920 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f699400c430 tx=0x7f699400c8f0 comp rx=0 tx=0).stop 2024-04-20T21:17:32.218 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.218+0000 7f69a9e00640 1 -- 172.21.15.86:0/4141067759 shutdown_connections 2024-04-20T21:17:32.218 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.218+0000 7f69a9e00640 1 --2- 172.21.15.86:0/4141067759 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69a40cdc10 0x7f69a4140950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:32.218 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.218+0000 7f69a9e00640 1 --2- 172.21.15.86:0/4141067759 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69a40cd7b0 0x7f69a4157bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:32.218 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.218+0000 7f69a9e00640 1 --2- 172.21.15.86:0/4141067759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69a406ee70 0x7f69a415e920 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:32.218 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.218+0000 7f69a9e00640 1 -- 172.21.15.86:0/4141067759 >> 172.21.15.86:0/4141067759 conn(0x7f69a4074fd0 msgr2=0x7f69a41582a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:32.218 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.218+0000 7f69a9e00640 1 -- 172.21.15.86:0/4141067759 shutdown_connections 2024-04-20T21:17:32.218 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:32.218+0000 7f69a9e00640 1 -- 172.21.15.86:0/4141067759 wait complete. 2024-04-20T21:17:32.230 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-20T21:17:32.478 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-20T21:17:32.478+0000 7f0a0524b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-20T21:17:32.525 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-04-20T21:17:32.525+0000 7fe5f8bc0040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-20T21:17:32.611 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-20T21:17:32.612+0000 7f0a0524b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-20T21:17:32.660 INFO:tasks.ceph.mgr.x.smithi144.stderr:2024-04-20T21:17:32.660+0000 7fe5f8bc0040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-20T21:17:33.651 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-20T21:17:33.652+0000 7f09a5c00640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-20T21:17:33.651 INFO:tasks.ceph.mgr.y.smithi086.stderr:2024-04-20T21:17:33.652+0000 7f09a5c00640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-20T21:17:34.888 INFO:tasks.ceph.osd.1.smithi086.stderr:2024-04-20T21:17:34.888+0000 7fda25c0f640 -1 osd.1 0 log_to_monitors true 2024-04-20T21:17:35.052 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-20T21:17:35.053+0000 7f95b5c1b640 -1 osd.0 0 log_to_monitors true 2024-04-20T21:17:35.126 INFO:tasks.ceph.osd.3.smithi086.stderr:2024-04-20T21:17:35.127+0000 7fbeedc0f640 -1 osd.3 0 log_to_monitors true 2024-04-20T21:17:35.142 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-04-20T21:17:35.142+0000 7f007f22d640 -1 osd.5 0 log_to_monitors true 2024-04-20T21:17:35.149 INFO:tasks.ceph.osd.2.smithi086.stderr:2024-04-20T21:17:35.150+0000 7f8043dff640 -1 osd.2 0 log_to_monitors true 2024-04-20T21:17:35.168 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-04-20T21:17:35.168+0000 7f3b71014640 -1 osd.6 0 log_to_monitors true 2024-04-20T21:17:35.364 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-04-20T21:17:35.364+0000 7fa38b4b5640 -1 osd.4 0 log_to_monitors true 2024-04-20T21:17:35.393 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-04-20T21:17:35.393+0000 7fe6c6011640 -1 osd.7 0 log_to_monitors true 2024-04-20T21:17:39.000 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-04-20T21:17:39.000+0000 7fa386400640 -1 osd.4 0 waiting for initial osdmap 2024-04-20T21:17:39.012 INFO:tasks.ceph.osd.4.smithi144.stderr:2024-04-20T21:17:39.012+0000 7fa37c400640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T21:17:39.020 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-20T21:17:39.021+0000 7f95b1000640 -1 osd.0 0 waiting for initial osdmap 2024-04-20T21:17:39.028 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-04-20T21:17:39.028+0000 7f0079600640 -1 osd.5 0 waiting for initial osdmap 2024-04-20T21:17:39.028 INFO:tasks.ceph.osd.0.smithi086.stderr:2024-04-20T21:17:39.029+0000 7f95a6600640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T21:17:39.036 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-20T21:17:39.043 INFO:tasks.ceph.osd.5.smithi144.stderr:2024-04-20T21:17:39.044+0000 7f006f600640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T21:17:39.050 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-04-20T21:17:39.050+0000 7f3b6c400640 -1 osd.6 0 waiting for initial osdmap 2024-04-20T21:17:39.050 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-04-20T21:17:39.050+0000 7fe6c0800640 -1 osd.7 0 waiting for initial osdmap 2024-04-20T21:17:39.058 INFO:tasks.ceph.osd.2.smithi086.stderr:2024-04-20T21:17:39.059+0000 7f803ec00640 -1 osd.2 0 waiting for initial osdmap 2024-04-20T21:17:39.058 INFO:tasks.ceph.osd.6.smithi144.stderr:2024-04-20T21:17:39.058+0000 7f3b61a00640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T21:17:39.062 INFO:tasks.ceph.osd.7.smithi144.stderr:2024-04-20T21:17:39.062+0000 7fe6b6800640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T21:17:39.063 INFO:tasks.ceph.osd.3.smithi086.stderr:2024-04-20T21:17:39.064+0000 7fbee8600640 -1 osd.3 0 waiting for initial osdmap 2024-04-20T21:17:39.068 INFO:tasks.ceph.osd.2.smithi086.stderr:2024-04-20T21:17:39.069+0000 7f8034c00640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T21:17:39.073 INFO:tasks.ceph.osd.3.smithi086.stderr:2024-04-20T21:17:39.074+0000 7fbede600640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T21:17:39.075 INFO:tasks.ceph.osd.1.smithi086.stderr:2024-04-20T21:17:39.076+0000 7fda20400640 -1 osd.1 0 waiting for initial osdmap 2024-04-20T21:17:39.086 INFO:tasks.ceph.osd.1.smithi086.stderr:2024-04-20T21:17:39.087+0000 7fda16400640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T21:17:39.193 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.194+0000 7f549b200640 1 Processor -- start 2024-04-20T21:17:39.194 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.194+0000 7f549b200640 1 -- start start 2024-04-20T21:17:39.195 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.195+0000 7f549b200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f549406f050 0x7f5494071d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:39.195 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.195+0000 7f549b200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f549406f4b0 0x7f5494072280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:39.196 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.196+0000 7f549b200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5494143190 con 0x7f5494142db0 2024-04-20T21:17:39.196 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.196+0000 7f549b200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f549406dd60 con 0x7f549406f4b0 2024-04-20T21:17:39.196 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.196+0000 7f549b200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f54940cf010 con 0x7f549406f050 2024-04-20T21:17:39.196 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.196+0000 7f5499200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f549406f4b0 0x7f5494072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:39.196 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.196+0000 7f5499200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f549406f4b0 0x7f5494072280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38578/0 (socket says 172.21.15.86:38578) 2024-04-20T21:17:39.196 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.196+0000 7f5499200640 1 -- 172.21.15.86:0/3642187270 learned_addr learned my addr 172.21.15.86:0/3642187270 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:39.196 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.196+0000 7f5499200640 1 -- 172.21.15.86:0/3642187270 >> v2:172.21.15.144:3300/0 conn(0x7f549406f050 msgr2=0x7f5494071d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:39.196 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.196+0000 7f5499200640 1 --2- 172.21.15.86:0/3642187270 >> v2:172.21.15.144:3300/0 conn(0x7f549406f050 0x7f5494071d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:39.196 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.196+0000 7f5499200640 1 -- 172.21.15.86:0/3642187270 >> v1:172.21.15.144:6789/0 conn(0x7f5494142db0 legacy=0x7f54940ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:39.196 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.196+0000 7f5499200640 1 -- 172.21.15.86:0/3642187270 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f549406dfb0 con 0x7f549406f4b0 2024-04-20T21:17:39.197 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.197+0000 7f5499200640 1 --2- 172.21.15.86:0/3642187270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f549406f4b0 0x7f5494072280 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f549406c860 tx=0x7f548c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56951d60d5248073 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:39.197 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.197+0000 7f5493400640 1 -- 172.21.15.86:0/3642187270 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548c02ca80 con 0x7f549406f4b0 2024-04-20T21:17:39.198 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.197+0000 7f5493400640 1 -- 172.21.15.86:0/3642187270 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f548c02cc20 con 0x7f549406f4b0 2024-04-20T21:17:39.198 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.197+0000 7f5493400640 1 -- 172.21.15.86:0/3642187270 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548c0343f0 con 0x7f549406f4b0 2024-04-20T21:17:39.198 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.198+0000 7f549b200640 1 -- 172.21.15.86:0/3642187270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f549406f4b0 msgr2=0x7f5494072280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:39.198 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.198+0000 7f549b200640 1 --2- 172.21.15.86:0/3642187270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f549406f4b0 0x7f5494072280 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f549406c860 tx=0x7f548c02bee0 comp rx=0 tx=0).stop 2024-04-20T21:17:39.198 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.198+0000 7f549b200640 1 -- 172.21.15.86:0/3642187270 shutdown_connections 2024-04-20T21:17:39.198 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.198+0000 7f549b200640 1 --2- 172.21.15.86:0/3642187270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f549406f4b0 0x7f5494072280 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:39.198 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.198+0000 7f549b200640 1 --2- 172.21.15.86:0/3642187270 >> v2:172.21.15.144:3300/0 conn(0x7f549406f050 0x7f5494071d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:39.198 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.198+0000 7f549b200640 1 -- 172.21.15.86:0/3642187270 >> 172.21.15.86:0/3642187270 conn(0x7f5494074fd0 msgr2=0x7f549406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:39.198 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.198+0000 7f549b200640 1 -- 172.21.15.86:0/3642187270 shutdown_connections 2024-04-20T21:17:39.198 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.198+0000 7f549b200640 1 -- 172.21.15.86:0/3642187270 wait complete. 2024-04-20T21:17:39.199 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.200+0000 7f549b200640 1 Processor -- start 2024-04-20T21:17:39.199 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.200+0000 7f549b200640 1 -- start start 2024-04-20T21:17:39.201 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.201+0000 7f549b200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f549406f050 0x7f54940b2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:39.202 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.201+0000 7f549b200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f549406f4b0 0x7f54940b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:39.202 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.201+0000 7f549b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5494142db0 0x7f54940b4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:39.202 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.201+0000 7f549b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f549406e850 con 0x7f5494142db0 2024-04-20T21:17:39.202 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.201+0000 7f549b200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f549406e570 con 0x7f549406f050 2024-04-20T21:17:39.202 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.201+0000 7f549b200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f549406eb30 con 0x7f549406f4b0 2024-04-20T21:17:39.202 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.201+0000 7f5499200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5494142db0 0x7f54940b4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:39.202 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.201+0000 7f5499200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5494142db0 0x7f54940b4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38594/0 (socket says 172.21.15.86:38594) 2024-04-20T21:17:39.202 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.201+0000 7f5499200640 1 -- 172.21.15.86:0/1284530015 learned_addr learned my addr 172.21.15.86:0/1284530015 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:39.202 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.202+0000 7f548b400640 1 --2- 172.21.15.86:0/1284530015 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f549406f4b0 0x7f54940b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:39.202 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.202+0000 7f5499200640 1 -- 172.21.15.86:0/1284530015 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f549406f4b0 msgr2=0x7f54940b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:39.202 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.202+0000 7f5499200640 1 --2- 172.21.15.86:0/1284530015 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f549406f4b0 0x7f54940b41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:39.203 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.202+0000 7f5493e00640 1 --2- 172.21.15.86:0/1284530015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f549406f050 0x7f54940b2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:39.203 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.202+0000 7f5499200640 1 -- 172.21.15.86:0/1284530015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f549406f050 msgr2=0x7f54940b2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:39.203 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.202+0000 7f5499200640 1 --2- 172.21.15.86:0/1284530015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f549406f050 0x7f54940b2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:39.203 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.202+0000 7f5499200640 1 -- 172.21.15.86:0/1284530015 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5494136570 con 0x7f5494142db0 2024-04-20T21:17:39.203 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.202+0000 7f548b400640 1 --2- 172.21.15.86:0/1284530015 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f549406f4b0 0x7f54940b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:17:39.203 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.202+0000 7f5499200640 1 --2- 172.21.15.86:0/1284530015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5494142db0 0x7f54940b4700 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f548c036040 tx=0x7f548c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:39.203 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.202+0000 7f5490c00640 1 -- 172.21.15.86:0/1284530015 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548c043020 con 0x7f5494142db0 2024-04-20T21:17:39.203 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.202+0000 7f5490c00640 1 -- 172.21.15.86:0/1284530015 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f548c03e450 con 0x7f5494142db0 2024-04-20T21:17:39.203 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.202+0000 7f5490c00640 1 -- 172.21.15.86:0/1284530015 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548c03e820 con 0x7f5494142db0 2024-04-20T21:17:39.203 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.202+0000 7f549b200640 1 -- 172.21.15.86:0/1284530015 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5494136850 con 0x7f5494142db0 2024-04-20T21:17:39.203 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.202+0000 7f549b200640 1 -- 172.21.15.86:0/1284530015 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54940aff20 con 0x7f5494142db0 2024-04-20T21:17:39.204 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.203+0000 7f549b200640 1 -- 172.21.15.86:0/1284530015 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f549404fcf0 con 0x7f5494142db0 2024-04-20T21:17:39.204 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.204+0000 7f5490c00640 1 -- 172.21.15.86:0/1284530015 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f548c046020 con 0x7f5494142db0 2024-04-20T21:17:39.204 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.205+0000 7f5490c00640 1 --2- 172.21.15.86:0/1284530015 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f547006c890 0x7f547006ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:39.204 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.205+0000 7f5490c00640 1 -- 172.21.15.86:0/1284530015 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7f548c0ad380 con 0x7f5494142db0 2024-04-20T21:17:39.205 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.205+0000 7f5493e00640 1 --2- 172.21.15.86:0/1284530015 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f547006c890 0x7f547006ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:39.205 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.206+0000 7f5493e00640 1 --2- 172.21.15.86:0/1284530015 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f547006c890 0x7f547006ed50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f549415c530 tx=0x7f5484008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:39.209 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.210+0000 7f5490c00640 1 -- 172.21.15.86:0/1284530015 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f548c04d070 con 0x7f5494142db0 2024-04-20T21:17:39.394 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.395+0000 7f549b200640 1 -- 172.21.15.86:0/1284530015 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f54940b2e30 con 0x7f5494142db0 2024-04-20T21:17:39.395 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.396+0000 7f5490c00640 1 -- 172.21.15.86:0/1284530015 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7f548c07c610 con 0x7f5494142db0 2024-04-20T21:17:39.395 INFO:teuthology.misc.health.smithi086.stdout: 2024-04-20T21:17:39.395 INFO:teuthology.misc.health.smithi086.stdout:{"epoch":12,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:17:36.652933+0000","last_up_change":"0.000000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:17:39.398 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.398+0000 7f549b200640 1 -- 172.21.15.86:0/1284530015 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f547006c890 msgr2=0x7f547006ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:39.398 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.398+0000 7f549b200640 1 --2- 172.21.15.86:0/1284530015 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f547006c890 0x7f547006ed50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f549415c530 tx=0x7f5484008040 comp rx=0 tx=0).stop 2024-04-20T21:17:39.398 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.398+0000 7f549b200640 1 -- 172.21.15.86:0/1284530015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5494142db0 msgr2=0x7f54940b4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:39.398 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.398+0000 7f549b200640 1 --2- 172.21.15.86:0/1284530015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5494142db0 0x7f54940b4700 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f548c036040 tx=0x7f548c004a40 comp rx=0 tx=0).stop 2024-04-20T21:17:39.398 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.399+0000 7f549b200640 1 -- 172.21.15.86:0/1284530015 shutdown_connections 2024-04-20T21:17:39.399 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.399+0000 7f549b200640 1 --2- 172.21.15.86:0/1284530015 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f547006c890 0x7f547006ed50 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:39.399 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.399+0000 7f549b200640 1 --2- 172.21.15.86:0/1284530015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5494142db0 0x7f54940b4700 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:39.399 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.399+0000 7f549b200640 1 --2- 172.21.15.86:0/1284530015 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f549406f4b0 0x7f54940b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:39.399 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.399+0000 7f549b200640 1 --2- 172.21.15.86:0/1284530015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f549406f050 0x7f54940b2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:39.399 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.399+0000 7f549b200640 1 -- 172.21.15.86:0/1284530015 >> 172.21.15.86:0/1284530015 conn(0x7f5494074fd0 msgr2=0x7f5494141c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:39.399 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.399+0000 7f549b200640 1 -- 172.21.15.86:0/1284530015 shutdown_connections 2024-04-20T21:17:39.399 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:39.399+0000 7f549b200640 1 -- 172.21.15.86:0/1284530015 wait complete. 2024-04-20T21:17:39.411 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-20T21:17:45.120 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:17:45.129 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:17:46.218 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-20T21:17:46.397 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.398+0000 7fb61f200640 1 Processor -- start 2024-04-20T21:17:46.398 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.398+0000 7fb61f200640 1 -- start start 2024-04-20T21:17:46.399 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.400+0000 7fb61f200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6200d0540 0x7fb6200d09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:46.399 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.400+0000 7fb61f200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb62015c9d0 con 0x7fb62015c5f0 2024-04-20T21:17:46.399 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.400+0000 7fb61f200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6200cd650 con 0x7fb6200cc920 2024-04-20T21:17:46.399 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.400+0000 7fb61f200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb6200d0ee0 con 0x7fb6200d0540 2024-04-20T21:17:46.399 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.400+0000 7fb61de00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fb62015c5f0 0x7fb62006ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39212/0 (socket says 172.21.15.86:39212) 2024-04-20T21:17:46.400 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.400+0000 7fb61de00640 1 -- 172.21.15.86:0/950104476 learned_addr learned my addr 172.21.15.86:0/950104476 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:17:46.402 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.401+0000 7fb61d400640 1 -- 172.21.15.86:0/950104476 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3797433680 0 0) 0x7fb62015c9d0 con 0x7fb62015c5f0 2024-04-20T21:17:46.402 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.401+0000 7fb617e00640 1 --2- 172.21.15.86:0/950104476 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6200d0540 0x7fb6200d09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:46.402 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.401+0000 7fb61d400640 1 -- 172.21.15.86:0/950104476 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb604003520 con 0x7fb62015c5f0 2024-04-20T21:17:46.402 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.401+0000 7fb61d400640 1 -- 172.21.15.86:0/950104476 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1780298527 0 0) 0x7fb604003520 con 0x7fb62015c5f0 2024-04-20T21:17:46.402 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.401+0000 7fb61d400640 1 -- 172.21.15.86:0/950104476 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6200d0540 msgr2=0x7fb6200d09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:46.402 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.401+0000 7fb61d400640 1 --2- 172.21.15.86:0/950104476 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6200d0540 0x7fb6200d09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:46.402 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.401+0000 7fb61d400640 1 -- 172.21.15.86:0/950104476 >> v1:172.21.15.144:6789/0 conn(0x7fb6200cc920 legacy=0x7fb6200ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:46.402 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.401+0000 7fb61d400640 1 -- 172.21.15.86:0/950104476 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6200cd930 con 0x7fb62015c5f0 2024-04-20T21:17:46.402 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.401+0000 7fb61d400640 1 -- 172.21.15.86:0/950104476 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fb610001670 con 0x7fb62015c5f0 2024-04-20T21:17:46.403 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.401+0000 7fb61d400640 1 -- 172.21.15.86:0/950104476 >> v1:172.21.15.86:6789/0 conn(0x7fb62015c5f0 legacy=0x7fb62006ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:46.403 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.401+0000 7fb61d400640 1 --2- 172.21.15.86:0/950104476 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6200d0540 0x7fb6040039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:46.403 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.401+0000 7fb61d400640 1 --2- 172.21.15.86:0/950104476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb604003f60 0x7fb604004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:46.403 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.401+0000 7fb61d400640 1 --2- 172.21.15.86:0/950104476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb604004d40 0x7fb6040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:46.403 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.401+0000 7fb61d400640 1 -- 172.21.15.86:0/950104476 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb60400ba70 con 0x7fb604003f60 2024-04-20T21:17:46.403 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.401+0000 7fb61d400640 1 -- 172.21.15.86:0/950104476 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb60400bc20 con 0x7fb604004d40 2024-04-20T21:17:46.403 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.402+0000 7fb61d400640 1 -- 172.21.15.86:0/950104476 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb60400be20 con 0x7fb6200d0540 2024-04-20T21:17:46.403 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.402+0000 7fb617e00640 1 --2- 172.21.15.86:0/950104476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb604003f60 0x7fb604004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:46.403 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.402+0000 7fb61de00640 1 --2- 172.21.15.86:0/950104476 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6200d0540 0x7fb6040039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:46.403 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.402+0000 7fb617e00640 1 -- 172.21.15.86:0/950104476 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6200d0540 msgr2=0x7fb6040039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:46.403 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.402+0000 7fb617e00640 1 --2- 172.21.15.86:0/950104476 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb6200d0540 0x7fb6040039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:46.403 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.402+0000 7fb617e00640 1 -- 172.21.15.86:0/950104476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb604004d40 msgr2=0x7fb6040051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:46.403 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.402+0000 7fb617e00640 1 --2- 172.21.15.86:0/950104476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb604004d40 0x7fb6040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:46.403 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.402+0000 7fb617e00640 1 -- 172.21.15.86:0/950104476 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb604003520 con 0x7fb604003f60 2024-04-20T21:17:46.405 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.403+0000 7fb617e00640 1 --2- 172.21.15.86:0/950104476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb604003f60 0x7fb604004800 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb608009ac0 tx=0x7fb604005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e4d4319abacc8c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:46.406 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.403+0000 7fb61d400640 1 -- 172.21.15.86:0/950104476 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb608039070 con 0x7fb604003f60 2024-04-20T21:17:46.406 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.403+0000 7fb61d400640 1 -- 172.21.15.86:0/950104476 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb60802c6b0 con 0x7fb604003f60 2024-04-20T21:17:46.406 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.403+0000 7fb61d400640 1 -- 172.21.15.86:0/950104476 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb608009690 con 0x7fb604003f60 2024-04-20T21:17:46.406 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.404+0000 7fb61f200640 1 -- 172.21.15.86:0/950104476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb604003f60 msgr2=0x7fb604004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:46.406 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.404+0000 7fb61f200640 1 --2- 172.21.15.86:0/950104476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb604003f60 0x7fb604004800 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb608009ac0 tx=0x7fb604005830 comp rx=0 tx=0).stop 2024-04-20T21:17:46.406 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.404+0000 7fb61e800640 1 -- 172.21.15.86:0/950104476 reap_dead start 2024-04-20T21:17:46.406 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.404+0000 7fb61f200640 1 -- 172.21.15.86:0/950104476 shutdown_connections 2024-04-20T21:17:46.406 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.404+0000 7fb61f200640 1 -- 172.21.15.86:0/950104476 >> 172.21.15.86:0/950104476 conn(0x7fb62006bda0 msgr2=0x7fb62006c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:46.406 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.404+0000 7fb61f200640 1 -- 172.21.15.86:0/950104476 shutdown_connections 2024-04-20T21:17:46.406 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.404+0000 7fb61f200640 1 -- 172.21.15.86:0/950104476 wait complete. 2024-04-20T21:17:46.406 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.406+0000 7fb61f200640 1 Processor -- start 2024-04-20T21:17:46.406 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.406+0000 7fb61f200640 1 -- start start 2024-04-20T21:17:46.409 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.407+0000 7fb61f200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb604003f60 0x7fb62009a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:46.409 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.407+0000 7fb61f200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb620099c90 0x7fb62009ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:46.409 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.407+0000 7fb61f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb62009a6d0 0x7fb62009b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:46.409 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.407+0000 7fb61f200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb62006e7e0 con 0x7fb62009a6d0 2024-04-20T21:17:46.409 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.407+0000 7fb61f200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb62006e500 con 0x7fb620099c90 2024-04-20T21:17:46.409 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.407+0000 7fb61f200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb6200d09a0 con 0x7fb604003f60 2024-04-20T21:17:46.409 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.407+0000 7fb61e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb62009a6d0 0x7fb62009b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:46.409 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.407+0000 7fb61e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb62009a6d0 0x7fb62009b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46500/0 (socket says 172.21.15.86:46500) 2024-04-20T21:17:46.409 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.407+0000 7fb61e800640 1 -- 172.21.15.86:0/2996308076 learned_addr learned my addr 172.21.15.86:0/2996308076 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:46.409 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.408+0000 7fb61de00640 1 --2- 172.21.15.86:0/2996308076 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb604003f60 0x7fb62009a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:46.409 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.408+0000 7fb617e00640 1 --2- 172.21.15.86:0/2996308076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb620099c90 0x7fb62009ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:46.409 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.408+0000 7fb61e800640 1 -- 172.21.15.86:0/2996308076 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb604003f60 msgr2=0x7fb62009a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:46.410 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.408+0000 7fb61e800640 1 --2- 172.21.15.86:0/2996308076 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb604003f60 0x7fb62009a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:46.410 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.408+0000 7fb61e800640 1 -- 172.21.15.86:0/2996308076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb620099c90 msgr2=0x7fb62009ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:46.410 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.408+0000 7fb61e800640 1 --2- 172.21.15.86:0/2996308076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb620099c90 0x7fb62009ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:46.410 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.408+0000 7fb61e800640 1 -- 172.21.15.86:0/2996308076 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6201f3180 con 0x7fb62009a6d0 2024-04-20T21:17:46.410 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.408+0000 7fb617e00640 1 --2- 172.21.15.86:0/2996308076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb620099c90 0x7fb62009ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:17:46.410 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.408+0000 7fb61e800640 1 --2- 172.21.15.86:0/2996308076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb62009a6d0 0x7fb62009b2a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb62006d8d0 tx=0x7fb61800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:46.410 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.408+0000 7fb616000640 1 -- 172.21.15.86:0/2996308076 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb618017070 con 0x7fb62009a6d0 2024-04-20T21:17:46.410 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.408+0000 7fb616000640 1 -- 172.21.15.86:0/2996308076 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb618005d60 con 0x7fb62009a6d0 2024-04-20T21:17:46.410 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.408+0000 7fb616000640 1 -- 172.21.15.86:0/2996308076 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61800a850 con 0x7fb62009a6d0 2024-04-20T21:17:46.410 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.408+0000 7fb61f200640 1 -- 172.21.15.86:0/2996308076 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6201f23c0 con 0x7fb62009a6d0 2024-04-20T21:17:46.410 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.408+0000 7fb61f200640 1 -- 172.21.15.86:0/2996308076 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6201f2980 con 0x7fb62009a6d0 2024-04-20T21:17:46.410 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.410+0000 7fb61f200640 1 -- 172.21.15.86:0/2996308076 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb620075d70 con 0x7fb62009a6d0 2024-04-20T21:17:46.411 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.410+0000 7fb616000640 1 -- 172.21.15.86:0/2996308076 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb61800a9f0 con 0x7fb62009a6d0 2024-04-20T21:17:46.411 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.411+0000 7fb616000640 1 --2- 172.21.15.86:0/2996308076 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb5fc06c8f0 0x7fb5fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:46.411 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.411+0000 7fb616000640 1 -- 172.21.15.86:0/2996308076 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fb61808aa50 con 0x7fb62009a6d0 2024-04-20T21:17:46.412 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.412+0000 7fb61de00640 1 --2- 172.21.15.86:0/2996308076 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb5fc06c8f0 0x7fb5fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:46.412 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.412+0000 7fb61de00640 1 --2- 172.21.15.86:0/2996308076 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb5fc06c8f0 0x7fb5fc06edb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb610001cd0 tx=0x7fb6100014f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:46.415 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.416+0000 7fb616000640 1 -- 172.21.15.86:0/2996308076 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb6180591c0 con 0x7fb62009a6d0 2024-04-20T21:17:46.626 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.623+0000 7fb61f200640 1 -- 172.21.15.86:0/2996308076 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb6200ce500 con 0x7fb62009a6d0 2024-04-20T21:17:46.626 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.625+0000 7fb616000640 1 -- 172.21.15.86:0/2996308076 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12884 (secure 0 0 0) 0x7fb618058ac0 con 0x7fb62009a6d0 2024-04-20T21:17:46.626 INFO:teuthology.misc.health.smithi086.stdout: 2024-04-20T21:17:46.627 INFO:teuthology.misc.health.smithi086.stdout:{"epoch":16,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:17:43.058910+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:17:46.628 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.628+0000 7fb61f200640 1 -- 172.21.15.86:0/2996308076 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb5fc06c8f0 msgr2=0x7fb5fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:46.628 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.628+0000 7fb61f200640 1 --2- 172.21.15.86:0/2996308076 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb5fc06c8f0 0x7fb5fc06edb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb610001cd0 tx=0x7fb6100014f0 comp rx=0 tx=0).stop 2024-04-20T21:17:46.628 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.628+0000 7fb61f200640 1 -- 172.21.15.86:0/2996308076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb62009a6d0 msgr2=0x7fb62009b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:46.628 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.628+0000 7fb61f200640 1 --2- 172.21.15.86:0/2996308076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb62009a6d0 0x7fb62009b2a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb62006d8d0 tx=0x7fb61800cb90 comp rx=0 tx=0).stop 2024-04-20T21:17:46.628 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.628+0000 7fb61f200640 1 -- 172.21.15.86:0/2996308076 shutdown_connections 2024-04-20T21:17:46.628 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.628+0000 7fb61f200640 1 --2- 172.21.15.86:0/2996308076 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb5fc06c8f0 0x7fb5fc06edb0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:46.628 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.628+0000 7fb61f200640 1 --2- 172.21.15.86:0/2996308076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb62009a6d0 0x7fb62009b2a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:46.628 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.628+0000 7fb61f200640 1 --2- 172.21.15.86:0/2996308076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb620099c90 0x7fb62009ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:46.628 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.628+0000 7fb61f200640 1 --2- 172.21.15.86:0/2996308076 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb604003f60 0x7fb62009a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:46.628 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.628+0000 7fb61f200640 1 -- 172.21.15.86:0/2996308076 >> 172.21.15.86:0/2996308076 conn(0x7fb62006bda0 msgr2=0x7fb6200ad010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:46.628 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.628+0000 7fb61f200640 1 -- 172.21.15.86:0/2996308076 shutdown_connections 2024-04-20T21:17:46.628 INFO:teuthology.misc.health.smithi086.stderr:2024-04-20T21:17:46.628+0000 7fb61f200640 1 -- 172.21.15.86:0/2996308076 wait complete. 2024-04-20T21:17:46.642 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-20T21:17:46.643 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-20T21:17:46.643 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-20T21:17:46.643 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-20T21:17:46.651 INFO:tasks.ceph.mds.a:Started 2024-04-20T21:17:46.651 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-20T21:17:46.651 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-20T21:17:46.657 INFO:tasks.ceph.mds.c:Started 2024-04-20T21:17:46.657 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-20T21:17:46.658 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-20T21:17:46.665 INFO:tasks.ceph.mds.b:Started 2024-04-20T21:17:46.665 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-20T21:17:46.665 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-20T21:17:46.672 INFO:tasks.ceph.mds.d:Started 2024-04-20T21:17:46.672 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-20T21:17:46.673 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:17:46.792 INFO:tasks.ceph.mds.c.smithi086.stdout:starting mds.c at 2024-04-20T21:17:46.794 INFO:tasks.ceph.mds.a.smithi086.stdout:starting mds.a at 2024-04-20T21:17:46.820 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2024-04-20T21:17:46.821 INFO:tasks.ceph.mds.d.smithi144.stdout:starting mds.d at 2024-04-20T21:17:46.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.837+0000 7fe076e00640 1 Processor -- start 2024-04-20T21:17:46.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.837+0000 7fe076e00640 1 -- start start 2024-04-20T21:17:46.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.838+0000 7fe076e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe07015ece0 0x7fe07015f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:46.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.838+0000 7fe076e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe0700d50c0 0x7fe0700d3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:46.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.838+0000 7fe076e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe07015f600 con 0x7fe0700cf9c0 2024-04-20T21:17:46.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.838+0000 7fe076e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe0700d3950 con 0x7fe07015ece0 2024-04-20T21:17:46.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.838+0000 7fe076e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fe0700cec00 con 0x7fe0700d50c0 2024-04-20T21:17:46.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.839+0000 7fe075a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe07015ece0 0x7fe07015f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:46.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.839+0000 7fe076400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe0700cf9c0 0x7fe070160000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39230/0 (socket says 172.21.15.86:39230) 2024-04-20T21:17:46.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.839+0000 7fe076400640 1 -- 172.21.15.86:0/1072930362 learned_addr learned my addr 172.21.15.86:0/1072930362 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:17:46.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.839+0000 7fe075000640 1 --2- 172.21.15.86:0/1072930362 >> v2:172.21.15.144:3300/0 conn(0x7fe0700d50c0 0x7fe0700d3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:46.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.839+0000 7fe075a00640 1 -- 172.21.15.86:0/1072930362 >> v2:172.21.15.144:3300/0 conn(0x7fe0700d50c0 msgr2=0x7fe0700d3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:46.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.839+0000 7fe075a00640 1 --2- 172.21.15.86:0/1072930362 >> v2:172.21.15.144:3300/0 conn(0x7fe0700d50c0 0x7fe0700d3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:46.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.839+0000 7fe075a00640 1 -- 172.21.15.86:0/1072930362 >> v1:172.21.15.86:6789/0 conn(0x7fe0700cf9c0 legacy=0x7fe070160000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:46.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.839+0000 7fe075a00640 1 -- 172.21.15.86:0/1072930362 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0700ceee0 con 0x7fe07015ece0 2024-04-20T21:17:46.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.840+0000 7fe075a00640 1 --2- 172.21.15.86:0/1072930362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe07015ece0 0x7fe07015f0c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe0700ceac0 tx=0x7fe06402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb5990e77d17ed server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:46.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.840+0000 7fe06fe00640 1 -- 172.21.15.86:0/1072930362 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0640315a0 con 0x7fe07015ece0 2024-04-20T21:17:46.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.840+0000 7fe06fe00640 1 -- 172.21.15.86:0/1072930362 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe064037070 con 0x7fe07015ece0 2024-04-20T21:17:46.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.840+0000 7fe06fe00640 1 -- 172.21.15.86:0/1072930362 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe064031960 con 0x7fe07015ece0 2024-04-20T21:17:46.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.841+0000 7fe076e00640 1 -- 172.21.15.86:0/1072930362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe07015ece0 msgr2=0x7fe07015f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:46.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.841+0000 7fe076e00640 1 --2- 172.21.15.86:0/1072930362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe07015ece0 0x7fe07015f0c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe0700ceac0 tx=0x7fe06402b840 comp rx=0 tx=0).stop 2024-04-20T21:17:46.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.841+0000 7fe076e00640 1 -- 172.21.15.86:0/1072930362 shutdown_connections 2024-04-20T21:17:46.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.841+0000 7fe076e00640 1 --2- 172.21.15.86:0/1072930362 >> v2:172.21.15.144:3300/0 conn(0x7fe0700d50c0 0x7fe0700d3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:46.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.841+0000 7fe076e00640 1 --2- 172.21.15.86:0/1072930362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe07015ece0 0x7fe07015f0c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:46.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.841+0000 7fe076e00640 1 -- 172.21.15.86:0/1072930362 >> 172.21.15.86:0/1072930362 conn(0x7fe07008bf70 msgr2=0x7fe0700d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:46.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.841+0000 7fe076e00640 1 -- 172.21.15.86:0/1072930362 shutdown_connections 2024-04-20T21:17:46.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.841+0000 7fe076e00640 1 -- 172.21.15.86:0/1072930362 wait complete. 2024-04-20T21:17:46.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.842+0000 7fe076e00640 1 Processor -- start 2024-04-20T21:17:46.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.843+0000 7fe076e00640 1 -- start start 2024-04-20T21:17:46.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.843+0000 7fe076e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0700cf9c0 0x7fe0700cda20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:46.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.843+0000 7fe076e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0700d50c0 0x7fe0701e8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:46.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.843+0000 7fe076e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe07015ece0 0x7fe07015d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:46.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.843+0000 7fe076e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe0700d3640 con 0x7fe0700cf9c0 2024-04-20T21:17:46.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.843+0000 7fe076e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe0700cf4a0 con 0x7fe07015ece0 2024-04-20T21:17:46.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.843+0000 7fe076e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe07015ea50 con 0x7fe0700d50c0 2024-04-20T21:17:46.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.844+0000 7fe075a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0700cf9c0 0x7fe0700cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:46.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.844+0000 7fe075a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0700cf9c0 0x7fe0700cda20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46550/0 (socket says 172.21.15.86:46550) 2024-04-20T21:17:46.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.844+0000 7fe075a00640 1 -- 172.21.15.86:0/3922617602 learned_addr learned my addr 172.21.15.86:0/3922617602 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:46.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.844+0000 7fe075000640 1 --2- 172.21.15.86:0/3922617602 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0700d50c0 0x7fe0701e8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:46.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.844+0000 7fe076400640 1 --2- 172.21.15.86:0/3922617602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe07015ece0 0x7fe07015d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:46.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.844+0000 7fe075a00640 1 -- 172.21.15.86:0/3922617602 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0700d50c0 msgr2=0x7fe0701e8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:46.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.844+0000 7fe075a00640 1 --2- 172.21.15.86:0/3922617602 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0700d50c0 0x7fe0701e8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:46.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.844+0000 7fe075a00640 1 -- 172.21.15.86:0/3922617602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe07015ece0 msgr2=0x7fe07015d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:46.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.844+0000 7fe075a00640 1 --2- 172.21.15.86:0/3922617602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe07015ece0 0x7fe07015d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:46.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.844+0000 7fe075a00640 1 -- 172.21.15.86:0/3922617602 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0701e9530 con 0x7fe0700cf9c0 2024-04-20T21:17:46.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.844+0000 7fe076400640 1 --2- 172.21.15.86:0/3922617602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe07015ece0 0x7fe07015d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:17:46.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.844+0000 7fe075a00640 1 --2- 172.21.15.86:0/3922617602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0700cf9c0 0x7fe0700cda20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe064009890 tx=0x7fe0640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:46.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.845+0000 7fe06d600640 1 -- 172.21.15.86:0/3922617602 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe064032630 con 0x7fe0700cf9c0 2024-04-20T21:17:46.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.845+0000 7fe06d600640 1 -- 172.21.15.86:0/3922617602 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0640327d0 con 0x7fe0700cf9c0 2024-04-20T21:17:46.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.845+0000 7fe06d600640 1 -- 172.21.15.86:0/3922617602 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe064032bc0 con 0x7fe0700cf9c0 2024-04-20T21:17:46.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.845+0000 7fe076e00640 1 -- 172.21.15.86:0/3922617602 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0701e9810 con 0x7fe0700cf9c0 2024-04-20T21:17:46.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.845+0000 7fe076e00640 1 -- 172.21.15.86:0/3922617602 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0701e9dd0 con 0x7fe0700cf9c0 2024-04-20T21:17:46.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.846+0000 7fe076e00640 1 -- 172.21.15.86:0/3922617602 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe07004fcf0 con 0x7fe0700cf9c0 2024-04-20T21:17:46.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.847+0000 7fe06d600640 1 -- 172.21.15.86:0/3922617602 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe06403b3f0 con 0x7fe0700cf9c0 2024-04-20T21:17:46.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.847+0000 7fe06d600640 1 --2- 172.21.15.86:0/3922617602 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe05006c940 0x7fe05006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:46.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.848+0000 7fe06d600640 1 -- 172.21.15.86:0/3922617602 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fe06402f030 con 0x7fe0700cf9c0 2024-04-20T21:17:46.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.848+0000 7fe075000640 1 --2- 172.21.15.86:0/3922617602 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe05006c940 0x7fe05006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:46.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.848+0000 7fe075000640 1 --2- 172.21.15.86:0/3922617602 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe05006c940 0x7fe05006ee00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe0700a23b0 tx=0x7fe05c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:46.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:46.852+0000 7fe06d600640 1 -- 172.21.15.86:0/3922617602 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe064040070 con 0x7fe0700cf9c0 2024-04-20T21:17:47.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.037+0000 7fe076e00640 1 -- 172.21.15.86:0/3922617602 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe0700ba3d0 con 0x7fe0700cf9c0 2024-04-20T21:17:47.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.038+0000 7fe06d600640 1 -- 172.21.15.86:0/3922617602 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12884 (secure 0 0 0) 0x7fe06407bd10 con 0x7fe0700cf9c0 2024-04-20T21:17:47.038 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:17:47.038 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":16,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:17:43.058910+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:17:47.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.041+0000 7fe076e00640 1 -- 172.21.15.86:0/3922617602 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe05006c940 msgr2=0x7fe05006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:47.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.041+0000 7fe076e00640 1 --2- 172.21.15.86:0/3922617602 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe05006c940 0x7fe05006ee00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe0700a23b0 tx=0x7fe05c009190 comp rx=0 tx=0).stop 2024-04-20T21:17:47.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.041+0000 7fe076e00640 1 -- 172.21.15.86:0/3922617602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0700cf9c0 msgr2=0x7fe0700cda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:47.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.041+0000 7fe076e00640 1 --2- 172.21.15.86:0/3922617602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0700cf9c0 0x7fe0700cda20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe064009890 tx=0x7fe0640046c0 comp rx=0 tx=0).stop 2024-04-20T21:17:47.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.041+0000 7fe076e00640 1 -- 172.21.15.86:0/3922617602 shutdown_connections 2024-04-20T21:17:47.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.041+0000 7fe076e00640 1 --2- 172.21.15.86:0/3922617602 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe05006c940 0x7fe05006ee00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:47.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.041+0000 7fe076e00640 1 --2- 172.21.15.86:0/3922617602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe07015ece0 0x7fe07015d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:47.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.041+0000 7fe076e00640 1 --2- 172.21.15.86:0/3922617602 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0700d50c0 0x7fe0701e8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:47.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.041+0000 7fe076e00640 1 --2- 172.21.15.86:0/3922617602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0700cf9c0 0x7fe0700cda20 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:47.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.041+0000 7fe076e00640 1 -- 172.21.15.86:0/3922617602 >> 172.21.15.86:0/3922617602 conn(0x7fe07008bf70 msgr2=0x7fe0700d25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:47.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.042+0000 7fe076e00640 1 -- 172.21.15.86:0/3922617602 shutdown_connections 2024-04-20T21:17:47.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.042+0000 7fe076e00640 1 -- 172.21.15.86:0/3922617602 wait complete. 2024-04-20T21:17:47.058 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:17:47.058 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:17:47.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.262+0000 7f1363200640 1 Processor -- start 2024-04-20T21:17:47.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.262+0000 7f1363200640 1 -- start start 2024-04-20T21:17:47.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.263+0000 7f1363200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f136415b4b0 0x7f13640cc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:47.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.263+0000 7f1363200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f136415ef50 0x7f136415f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:47.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.264+0000 7f1363200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13640729f0 con 0x7f136415b890 2024-04-20T21:17:47.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.264+0000 7f1363200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f136415fe10 con 0x7f136415ef50 2024-04-20T21:17:47.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.264+0000 7f1363200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f136415f8d0 con 0x7f136415b4b0 2024-04-20T21:17:47.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.264+0000 7f1362800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f136415b890 0x7f136415bd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39234/0 (socket says 172.21.15.86:39234) 2024-04-20T21:17:47.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.264+0000 7f1362800640 1 -- 172.21.15.86:0/155059612 learned_addr learned my addr 172.21.15.86:0/155059612 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:17:47.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.264+0000 7f1361e00640 1 --2- 172.21.15.86:0/155059612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f136415b4b0 0x7f13640cc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:47.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.264+0000 7f1361400640 1 --2- 172.21.15.86:0/155059612 >> v2:172.21.15.144:3300/0 conn(0x7f136415ef50 0x7f136415f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:47.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.264+0000 7f1360a00640 1 -- 172.21.15.86:0/155059612 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2509874728 0 0) 0x7f13640729f0 con 0x7f136415b890 2024-04-20T21:17:47.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.264+0000 7f1360a00640 1 -- 172.21.15.86:0/155059612 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1348003520 con 0x7f136415b890 2024-04-20T21:17:47.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.265+0000 7f1360a00640 1 -- 172.21.15.86:0/155059612 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 438949893 0 0) 0x7f1348003520 con 0x7f136415b890 2024-04-20T21:17:47.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.265+0000 7f1360a00640 1 -- 172.21.15.86:0/155059612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f136415b4b0 msgr2=0x7f13640cc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:47.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.265+0000 7f1360a00640 1 --2- 172.21.15.86:0/155059612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f136415b4b0 0x7f13640cc3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:47.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.265+0000 7f1360a00640 1 -- 172.21.15.86:0/155059612 >> v2:172.21.15.144:3300/0 conn(0x7f136415ef50 msgr2=0x7f136415f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:47.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.265+0000 7f1360a00640 1 --2- 172.21.15.86:0/155059612 >> v2:172.21.15.144:3300/0 conn(0x7f136415ef50 0x7f136415f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:47.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.265+0000 7f1360a00640 1 -- 172.21.15.86:0/155059612 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1364072cd0 con 0x7f136415b890 2024-04-20T21:17:47.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.265+0000 7f1360a00640 1 -- 172.21.15.86:0/155059612 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f135c0019f0 con 0x7f136415b890 2024-04-20T21:17:47.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.265+0000 7f1360a00640 1 -- 172.21.15.86:0/155059612 >> v1:172.21.15.86:6789/0 conn(0x7f136415b890 legacy=0x7f136415bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:47.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.265+0000 7f1360a00640 1 --2- 172.21.15.86:0/155059612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13480039b0 0x7f1348003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:47.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.265+0000 7f1360a00640 1 --2- 172.21.15.86:0/155059612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1348004800 0x7f1348004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:47.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.265+0000 7f1360a00640 1 --2- 172.21.15.86:0/155059612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f136415b4b0 0x7f13480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:47.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.265+0000 7f1360a00640 1 -- 172.21.15.86:0/155059612 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f136415fe10 con 0x7f1348004800 2024-04-20T21:17:47.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.265+0000 7f1360a00640 1 -- 172.21.15.86:0/155059612 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f136415f8d0 con 0x7f13480039b0 2024-04-20T21:17:47.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.265+0000 7f1360a00640 1 -- 172.21.15.86:0/155059612 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f134800b9f0 con 0x7f136415b4b0 2024-04-20T21:17:47.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.265+0000 7f1361400640 1 --2- 172.21.15.86:0/155059612 >> v2:172.21.15.144:3300/0 conn(0x7f136415ef50 0x7f136415f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:17:47.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.265+0000 7f1361400640 1 --2- 172.21.15.86:0/155059612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1348004800 0x7f1348004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:47.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.265+0000 7f1362800640 1 --2- 172.21.15.86:0/155059612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f136415b4b0 0x7f13480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:47.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.266+0000 7f1361e00640 1 --2- 172.21.15.86:0/155059612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13480039b0 0x7f1348003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:47.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.266+0000 7f1361400640 1 -- 172.21.15.86:0/155059612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f136415b4b0 msgr2=0x7f13480051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:47.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.266+0000 7f1361400640 1 --2- 172.21.15.86:0/155059612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f136415b4b0 0x7f13480051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:47.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.266+0000 7f1361400640 1 -- 172.21.15.86:0/155059612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13480039b0 msgr2=0x7f1348003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:47.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.266+0000 7f1361400640 1 --2- 172.21.15.86:0/155059612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13480039b0 0x7f1348003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:47.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.266+0000 7f1361400640 1 -- 172.21.15.86:0/155059612 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1348003520 con 0x7f1348004800 2024-04-20T21:17:47.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.266+0000 7f1361e00640 1 --2- 172.21.15.86:0/155059612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13480039b0 0x7f1348003df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:17:47.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.267+0000 7f1361400640 1 --2- 172.21.15.86:0/155059612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1348004800 0x7f1348004c80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f13500098a0 tx=0x7f1348005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37937bfc8914c8cb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:47.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.267+0000 7f1360a00640 1 -- 172.21.15.86:0/155059612 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1350034070 con 0x7f1348004800 2024-04-20T21:17:47.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.267+0000 7f1360a00640 1 -- 172.21.15.86:0/155059612 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1350030670 con 0x7f1348004800 2024-04-20T21:17:47.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.267+0000 7f1360a00640 1 -- 172.21.15.86:0/155059612 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135000f040 con 0x7f1348004800 2024-04-20T21:17:47.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.268+0000 7f1363200640 1 -- 172.21.15.86:0/155059612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1348004800 msgr2=0x7f1348004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:47.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.268+0000 7f1363200640 1 --2- 172.21.15.86:0/155059612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1348004800 0x7f1348004c80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f13500098a0 tx=0x7f1348005830 comp rx=0 tx=0).stop 2024-04-20T21:17:47.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.268+0000 7f1362800640 1 -- 172.21.15.86:0/155059612 reap_dead start 2024-04-20T21:17:47.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.268+0000 7f1363200640 1 -- 172.21.15.86:0/155059612 shutdown_connections 2024-04-20T21:17:47.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.268+0000 7f1363200640 1 -- 172.21.15.86:0/155059612 >> 172.21.15.86:0/155059612 conn(0x7f13640b1f00 msgr2=0x7f13640b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:47.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.268+0000 7f1363200640 1 -- 172.21.15.86:0/155059612 shutdown_connections 2024-04-20T21:17:47.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.268+0000 7f1363200640 1 -- 172.21.15.86:0/155059612 wait complete. 2024-04-20T21:17:47.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.269+0000 7f1363200640 1 Processor -- start 2024-04-20T21:17:47.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.270+0000 7f1363200640 1 -- start start 2024-04-20T21:17:47.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.271+0000 7f1363200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1348004800 0x7f13641462d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:47.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.271+0000 7f1363200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1364148780 0x7f13640b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:47.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.271+0000 7f1363200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1364147a30 0x7f1364147eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:47.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.271+0000 7f1363200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1364073570 con 0x7f1348004800 2024-04-20T21:17:47.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.271+0000 7f1363200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1364073290 con 0x7f1364147a30 2024-04-20T21:17:47.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.271+0000 7f1363200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1364072650 con 0x7f1364148780 2024-04-20T21:17:47.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.271+0000 7f1361e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1348004800 0x7f13641462d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:47.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.271+0000 7f1361e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1348004800 0x7f13641462d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46572/0 (socket says 172.21.15.86:46572) 2024-04-20T21:17:47.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.271+0000 7f1361e00640 1 -- 172.21.15.86:0/2806052961 learned_addr learned my addr 172.21.15.86:0/2806052961 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:47.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.271+0000 7f1361e00640 1 -- 172.21.15.86:0/2806052961 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1364148780 msgr2=0x7f13640b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:47.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.271+0000 7f1361e00640 1 --2- 172.21.15.86:0/2806052961 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1364148780 0x7f13640b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:47.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.271+0000 7f1361e00640 1 -- 172.21.15.86:0/2806052961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1364147a30 msgr2=0x7f1364147eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:47.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.271+0000 7f1361e00640 1 --2- 172.21.15.86:0/2806052961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1364147a30 0x7f1364147eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:47.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.271+0000 7f1361e00640 1 -- 172.21.15.86:0/2806052961 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13640b50e0 con 0x7f1348004800 2024-04-20T21:17:47.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.272+0000 7f1361e00640 1 --2- 172.21.15.86:0/2806052961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1348004800 0x7f13641462d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f13540108b0 tx=0x7f1354010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:47.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.272+0000 7f135a000640 1 -- 172.21.15.86:0/2806052961 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135400f040 con 0x7f1348004800 2024-04-20T21:17:47.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.272+0000 7f135a000640 1 -- 172.21.15.86:0/2806052961 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f135400ea80 con 0x7f1348004800 2024-04-20T21:17:47.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.272+0000 7f1363200640 1 -- 172.21.15.86:0/2806052961 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f136414a230 con 0x7f1348004800 2024-04-20T21:17:47.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.272+0000 7f135a000640 1 -- 172.21.15.86:0/2806052961 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135400ee30 con 0x7f1348004800 2024-04-20T21:17:47.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.272+0000 7f1363200640 1 -- 172.21.15.86:0/2806052961 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1364149ee0 con 0x7f1348004800 2024-04-20T21:17:47.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.273+0000 7f1363200640 1 -- 172.21.15.86:0/2806052961 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f136404fd90 con 0x7f1348004800 2024-04-20T21:17:47.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.274+0000 7f135a000640 1 -- 172.21.15.86:0/2806052961 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f135401c400 con 0x7f1348004800 2024-04-20T21:17:47.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.275+0000 7f135a000640 1 --2- 172.21.15.86:0/2806052961 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f134406c920 0x7f134406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:47.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.275+0000 7f135a000640 1 -- 172.21.15.86:0/2806052961 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f135408bcc0 con 0x7f1348004800 2024-04-20T21:17:47.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.276+0000 7f1361400640 1 --2- 172.21.15.86:0/2806052961 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f134406c920 0x7f134406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:47.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.276+0000 7f1361400640 1 --2- 172.21.15.86:0/2806052961 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f134406c920 0x7f134406ede0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1350036000 tx=0x7f1350030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:47.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.280+0000 7f135a000640 1 -- 172.21.15.86:0/2806052961 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1354092080 con 0x7f1348004800 2024-04-20T21:17:47.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.481+0000 7f1363200640 1 -- 172.21.15.86:0/2806052961 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f136415fbf0 con 0x7f1348004800 2024-04-20T21:17:47.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.482+0000 7f135a000640 1 -- 172.21.15.86:0/2806052961 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f1354018030 con 0x7f1348004800 2024-04-20T21:17:47.482 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:17:47.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.485+0000 7f1363200640 1 -- 172.21.15.86:0/2806052961 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f134406c920 msgr2=0x7f134406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:47.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.485+0000 7f1363200640 1 --2- 172.21.15.86:0/2806052961 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f134406c920 0x7f134406ede0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1350036000 tx=0x7f1350030000 comp rx=0 tx=0).stop 2024-04-20T21:17:47.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.485+0000 7f1363200640 1 -- 172.21.15.86:0/2806052961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1348004800 msgr2=0x7f13641462d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:47.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.485+0000 7f1363200640 1 --2- 172.21.15.86:0/2806052961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1348004800 0x7f13641462d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f13540108b0 tx=0x7f1354010d70 comp rx=0 tx=0).stop 2024-04-20T21:17:47.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.485+0000 7f1363200640 1 -- 172.21.15.86:0/2806052961 shutdown_connections 2024-04-20T21:17:47.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.485+0000 7f1363200640 1 --2- 172.21.15.86:0/2806052961 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f134406c920 0x7f134406ede0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:47.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.485+0000 7f1363200640 1 --2- 172.21.15.86:0/2806052961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1364147a30 0x7f1364147eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:47.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.485+0000 7f1363200640 1 --2- 172.21.15.86:0/2806052961 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1364148780 0x7f13640b3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:47.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.485+0000 7f1363200640 1 --2- 172.21.15.86:0/2806052961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1348004800 0x7f13641462d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:47.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.485+0000 7f1363200640 1 -- 172.21.15.86:0/2806052961 >> 172.21.15.86:0/2806052961 conn(0x7f13640b1f00 msgr2=0x7f13641688e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:47.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.485+0000 7f1363200640 1 -- 172.21.15.86:0/2806052961 shutdown_connections 2024-04-20T21:17:47.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.485+0000 7f1363200640 1 -- 172.21.15.86:0/2806052961 wait complete. 2024-04-20T21:17:47.503 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:17:47.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.702+0000 7fdf9e800640 1 Processor -- start 2024-04-20T21:17:47.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.702+0000 7fdf9e800640 1 -- start start 2024-04-20T21:17:47.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.703+0000 7fdf9e800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdf98146130 0x7fdf9806e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:47.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.703+0000 7fdf9e800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdf980ccdd0 0x7fdf980cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:47.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.703+0000 7fdf9e800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf98071d00 0x7fdf98072190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:47.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.703+0000 7fdf9e800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fdf9806dc30 con 0x7fdf98146130 2024-04-20T21:17:47.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.703+0000 7fdf9e800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fdf980cd8e0 con 0x7fdf980ccdd0 2024-04-20T21:17:47.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.703+0000 7fdf9e800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdf980cdbc0 con 0x7fdf98071d00 2024-04-20T21:17:47.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.704+0000 7fdf97400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdf98146130 0x7fdf9806e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:47.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.704+0000 7fdf97400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdf98146130 0x7fdf9806e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46580/0 (socket says 172.21.15.86:46580) 2024-04-20T21:17:47.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.704+0000 7fdf97400640 1 -- 172.21.15.86:0/181801995 learned_addr learned my addr 172.21.15.86:0/181801995 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:47.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.704+0000 7fdf97400640 1 -- 172.21.15.86:0/181801995 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf98071d00 msgr2=0x7fdf98072190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:47.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.704+0000 7fdf97400640 1 --2- 172.21.15.86:0/181801995 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf98071d00 0x7fdf98072190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:47.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.704+0000 7fdf8fe00640 1 --2- 172.21.15.86:0/181801995 >> v2:172.21.15.144:3300/0 conn(0x7fdf980ccdd0 0x7fdf980cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:47.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.704+0000 7fdf97400640 1 -- 172.21.15.86:0/181801995 >> v2:172.21.15.144:3300/0 conn(0x7fdf980ccdd0 msgr2=0x7fdf980cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:47.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.704+0000 7fdf97400640 1 --2- 172.21.15.86:0/181801995 >> v2:172.21.15.144:3300/0 conn(0x7fdf980ccdd0 0x7fdf980cd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:47.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.704+0000 7fdf97400640 1 -- 172.21.15.86:0/181801995 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf9806ded0 con 0x7fdf98146130 2024-04-20T21:17:47.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.705+0000 7fdf97400640 1 --2- 172.21.15.86:0/181801995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf98146130 0x7fdf9806e6d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdf98073120 tx=0x7fdf8802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee3352eb9e48410b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:47.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.706+0000 7fdf96a00640 1 -- 172.21.15.86:0/181801995 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf88034070 con 0x7fdf98146130 2024-04-20T21:17:47.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.706+0000 7fdf96a00640 1 -- 172.21.15.86:0/181801995 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf88030860 con 0x7fdf98146130 2024-04-20T21:17:47.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.706+0000 7fdf96a00640 1 -- 172.21.15.86:0/181801995 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf88030c30 con 0x7fdf98146130 2024-04-20T21:17:47.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.706+0000 7fdf9e800640 1 -- 172.21.15.86:0/181801995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf98146130 msgr2=0x7fdf9806e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:47.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.706+0000 7fdf9e800640 1 --2- 172.21.15.86:0/181801995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf98146130 0x7fdf9806e6d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdf98073120 tx=0x7fdf8802b9a0 comp rx=0 tx=0).stop 2024-04-20T21:17:47.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.707+0000 7fdf9e800640 1 -- 172.21.15.86:0/181801995 shutdown_connections 2024-04-20T21:17:47.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.707+0000 7fdf9e800640 1 --2- 172.21.15.86:0/181801995 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf98071d00 0x7fdf98072190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:47.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.707+0000 7fdf9e800640 1 --2- 172.21.15.86:0/181801995 >> v2:172.21.15.144:3300/0 conn(0x7fdf980ccdd0 0x7fdf980cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:47.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.707+0000 7fdf9e800640 1 --2- 172.21.15.86:0/181801995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf98146130 0x7fdf9806e6d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:47.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.707+0000 7fdf9e800640 1 -- 172.21.15.86:0/181801995 >> 172.21.15.86:0/181801995 conn(0x7fdf980b2360 msgr2=0x7fdf980b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:47.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.707+0000 7fdf9e800640 1 -- 172.21.15.86:0/181801995 shutdown_connections 2024-04-20T21:17:47.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.707+0000 7fdf9e800640 1 -- 172.21.15.86:0/181801995 wait complete. 2024-04-20T21:17:47.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.708+0000 7fdf9e800640 1 Processor -- start 2024-04-20T21:17:47.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.708+0000 7fdf9e800640 1 -- start start 2024-04-20T21:17:47.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.709+0000 7fdf9e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf98071d00 0x7fdf98136e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:47.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.709+0000 7fdf9e800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf980ccdd0 0x7fdf9813f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:47.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.709+0000 7fdf9e800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf98146130 0x7fdf9813fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:47.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.709+0000 7fdf9e800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9815cc20 con 0x7fdf98071d00 2024-04-20T21:17:47.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.709+0000 7fdf9e800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9806f150 con 0x7fdf980ccdd0 2024-04-20T21:17:47.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.709+0000 7fdf9e800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdf9815cda0 con 0x7fdf98146130 2024-04-20T21:17:47.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.710+0000 7fdf97400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf98071d00 0x7fdf98136e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:47.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.710+0000 7fdf97400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf98071d00 0x7fdf98136e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46586/0 (socket says 172.21.15.86:46586) 2024-04-20T21:17:47.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.710+0000 7fdf97400640 1 -- 172.21.15.86:0/3117333645 learned_addr learned my addr 172.21.15.86:0/3117333645 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:47.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.710+0000 7fdf97400640 1 -- 172.21.15.86:0/3117333645 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf98146130 msgr2=0x7fdf9813fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:47.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.710+0000 7fdf97400640 1 --2- 172.21.15.86:0/3117333645 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf98146130 0x7fdf9813fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:47.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.710+0000 7fdf97400640 1 -- 172.21.15.86:0/3117333645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf980ccdd0 msgr2=0x7fdf9813f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:47.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.710+0000 7fdf8fe00640 1 --2- 172.21.15.86:0/3117333645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf980ccdd0 0x7fdf9813f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:47.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.710+0000 7fdf97400640 1 --2- 172.21.15.86:0/3117333645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf980ccdd0 0x7fdf9813f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:47.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.710+0000 7fdf97400640 1 -- 172.21.15.86:0/3117333645 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf980b4730 con 0x7fdf98071d00 2024-04-20T21:17:47.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.710+0000 7fdf97400640 1 --2- 172.21.15.86:0/3117333645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf98071d00 0x7fdf98136e20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdf88000c00 tx=0x7fdf880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:47.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.711+0000 7fdf8f400640 1 -- 172.21.15.86:0/3117333645 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf88034040 con 0x7fdf98071d00 2024-04-20T21:17:47.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.711+0000 7fdf8f400640 1 -- 172.21.15.86:0/3117333645 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf880433f0 con 0x7fdf98071d00 2024-04-20T21:17:47.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.711+0000 7fdf8f400640 1 -- 172.21.15.86:0/3117333645 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf880437e0 con 0x7fdf98071d00 2024-04-20T21:17:47.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.711+0000 7fdf9e800640 1 -- 172.21.15.86:0/3117333645 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf980b4a10 con 0x7fdf98071d00 2024-04-20T21:17:47.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.711+0000 7fdf9e800640 1 -- 172.21.15.86:0/3117333645 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf980b4fd0 con 0x7fdf98071d00 2024-04-20T21:17:47.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.712+0000 7fdf9e800640 1 -- 172.21.15.86:0/3117333645 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf980689e0 con 0x7fdf98071d00 2024-04-20T21:17:47.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.713+0000 7fdf8f400640 1 -- 172.21.15.86:0/3117333645 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdf8804a060 con 0x7fdf98071d00 2024-04-20T21:17:47.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.714+0000 7fdf8f400640 1 --2- 172.21.15.86:0/3117333645 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdf7406c920 0x7fdf7406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:47.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.714+0000 7fdf8f400640 1 -- 172.21.15.86:0/3117333645 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fdf880adba0 con 0x7fdf98071d00 2024-04-20T21:17:47.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.714+0000 7fdf8fe00640 1 --2- 172.21.15.86:0/3117333645 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdf7406c920 0x7fdf7406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:47.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.715+0000 7fdf8fe00640 1 --2- 172.21.15.86:0/3117333645 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdf7406c920 0x7fdf7406ede0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdf80006fd0 tx=0x7fdf80008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:47.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.719+0000 7fdf8f400640 1 -- 172.21.15.86:0/3117333645 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fdf8803e070 con 0x7fdf98071d00 2024-04-20T21:17:47.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.959+0000 7fdf9e800640 1 -- 172.21.15.86:0/3117333645 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf9806eaf0 con 0x7fdf98071d00 2024-04-20T21:17:47.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.960+0000 7fdf8f400640 1 -- 172.21.15.86:0/3117333645 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3867 (secure 0 0 0) 0x7fdf8804be90 con 0x7fdf98071d00 2024-04-20T21:17:47.960 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:17:47.960 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/179773217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":179773217},{"type":"v1","addr":"172.21.15.144:6834","nonce":179773217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3179785507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3179785507},{"type":"v1","addr":"172.21.15.86:6835","nonce":3179785507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4270,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2780853729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2780853729},{"type":"v1","addr":"172.21.15.86:6837","nonce":2780853729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4272,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2436181584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2436181584},{"type":"v1","addr":"172.21.15.144:6835","nonce":2436181584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-20T21:17:47.960 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 3 2024-04-20T21:17:47.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.963+0000 7fdf9e800640 1 -- 172.21.15.86:0/3117333645 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdf7406c920 msgr2=0x7fdf7406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:47.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.963+0000 7fdf9e800640 1 --2- 172.21.15.86:0/3117333645 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdf7406c920 0x7fdf7406ede0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdf80006fd0 tx=0x7fdf80008040 comp rx=0 tx=0).stop 2024-04-20T21:17:47.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.963+0000 7fdf9e800640 1 -- 172.21.15.86:0/3117333645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf98071d00 msgr2=0x7fdf98136e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:47.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.963+0000 7fdf9e800640 1 --2- 172.21.15.86:0/3117333645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf98071d00 0x7fdf98136e20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdf88000c00 tx=0x7fdf880046c0 comp rx=0 tx=0).stop 2024-04-20T21:17:47.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.963+0000 7fdf9e800640 1 -- 172.21.15.86:0/3117333645 shutdown_connections 2024-04-20T21:17:47.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.963+0000 7fdf9e800640 1 --2- 172.21.15.86:0/3117333645 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdf7406c920 0x7fdf7406ede0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:47.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.963+0000 7fdf9e800640 1 --2- 172.21.15.86:0/3117333645 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf98146130 0x7fdf9813fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:47.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.963+0000 7fdf9e800640 1 --2- 172.21.15.86:0/3117333645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf980ccdd0 0x7fdf9813f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:47.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.963+0000 7fdf9e800640 1 --2- 172.21.15.86:0/3117333645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf98071d00 0x7fdf98136e20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:47.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.963+0000 7fdf9e800640 1 -- 172.21.15.86:0/3117333645 >> 172.21.15.86:0/3117333645 conn(0x7fdf980b2360 msgr2=0x7fdf9813eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:47.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.963+0000 7fdf9e800640 1 -- 172.21.15.86:0/3117333645 shutdown_connections 2024-04-20T21:17:47.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:47.964+0000 7fdf9e800640 1 -- 172.21.15.86:0/3117333645 wait complete. 2024-04-20T21:17:47.982 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:17:48.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.175+0000 7fceb2600640 1 Processor -- start 2024-04-20T21:17:48.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.175+0000 7fceb2600640 1 -- start start 2024-04-20T21:17:48.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.176+0000 7fceb2600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fceac06e6d0 0x7fceac071d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:48.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.176+0000 7fceb2600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fceac072240 0x7fceac15cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:48.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.176+0000 7fceb2600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fceac06eb30 con 0x7fceac146130 2024-04-20T21:17:48.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.176+0000 7fceb2600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fceac072690 con 0x7fceac06e6d0 2024-04-20T21:17:48.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.176+0000 7fceb2600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fceac06dc30 con 0x7fceac072240 2024-04-20T21:17:48.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.177+0000 7fceab400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fceac146130 0x7fceac0ccdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39244/0 (socket says 172.21.15.86:39244) 2024-04-20T21:17:48.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.177+0000 7fceab400640 1 -- 172.21.15.86:0/30041332 learned_addr learned my addr 172.21.15.86:0/30041332 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:17:48.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.177+0000 7fcea2a00640 1 --2- 172.21.15.86:0/30041332 >> v2:172.21.15.86:3300/0 conn(0x7fceac06e6d0 0x7fceac071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:48.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.177+0000 7fceabe00640 1 --2- 172.21.15.86:0/30041332 >> v2:172.21.15.144:3300/0 conn(0x7fceac072240 0x7fceac15cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:48.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.177+0000 7fceaaa00640 1 -- 172.21.15.86:0/30041332 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2505253182 0 0) 0x7fce9c001110 con 0x7fceac146130 2024-04-20T21:17:48.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.177+0000 7fceaaa00640 1 -- 172.21.15.86:0/30041332 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce90003520 con 0x7fceac146130 2024-04-20T21:17:48.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.177+0000 7fcea2a00640 1 -- 172.21.15.86:0/30041332 >> v2:172.21.15.144:3300/0 conn(0x7fceac072240 msgr2=0x7fceac15cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:48.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.177+0000 7fcea2a00640 1 --2- 172.21.15.86:0/30041332 >> v2:172.21.15.144:3300/0 conn(0x7fceac072240 0x7fceac15cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:48.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.178+0000 7fcea2a00640 1 -- 172.21.15.86:0/30041332 >> v1:172.21.15.86:6789/0 conn(0x7fceac146130 legacy=0x7fceac0ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:48.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.178+0000 7fceaaa00640 1 -- 172.21.15.86:0/30041332 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1492982827 0 0) 0x7fce90003520 con 0x7fceac146130 2024-04-20T21:17:48.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.178+0000 7fcea2a00640 1 -- 172.21.15.86:0/30041332 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceac0cd8e0 con 0x7fceac06e6d0 2024-04-20T21:17:48.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.179+0000 7fcea2a00640 1 --2- 172.21.15.86:0/30041332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fceac06e6d0 0x7fceac071d00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fceac071c90 tx=0x7fce9402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dece4555f532828 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:48.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.179+0000 7fceaaa00640 1 -- 172.21.15.86:0/30041332 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce94039070 con 0x7fceac06e6d0 2024-04-20T21:17:48.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.179+0000 7fceaaa00640 1 -- 172.21.15.86:0/30041332 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce9402c950 con 0x7fceac06e6d0 2024-04-20T21:17:48.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.179+0000 7fceaaa00640 1 -- 172.21.15.86:0/30041332 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9c001840 con 0x7fceac06e6d0 2024-04-20T21:17:48.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.180+0000 7fceb2600640 1 -- 172.21.15.86:0/30041332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fceac06e6d0 msgr2=0x7fceac071d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:48.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.180+0000 7fceb2600640 1 --2- 172.21.15.86:0/30041332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fceac06e6d0 0x7fceac071d00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fceac071c90 tx=0x7fce9402be20 comp rx=0 tx=0).stop 2024-04-20T21:17:48.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.180+0000 7fceb2600640 1 -- 172.21.15.86:0/30041332 shutdown_connections 2024-04-20T21:17:48.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.180+0000 7fceb2600640 1 --2- 172.21.15.86:0/30041332 >> v2:172.21.15.144:3300/0 conn(0x7fceac072240 0x7fceac15cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:48.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.180+0000 7fceb2600640 1 --2- 172.21.15.86:0/30041332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fceac06e6d0 0x7fceac071d00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:48.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.180+0000 7fceb2600640 1 -- 172.21.15.86:0/30041332 >> 172.21.15.86:0/30041332 conn(0x7fceac0b2360 msgr2=0x7fceac0b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:48.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.180+0000 7fceb2600640 1 -- 172.21.15.86:0/30041332 shutdown_connections 2024-04-20T21:17:48.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.180+0000 7fceb2600640 1 -- 172.21.15.86:0/30041332 wait complete. 2024-04-20T21:17:48.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.182+0000 7fceb2600640 1 Processor -- start 2024-04-20T21:17:48.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.182+0000 7fceb2600640 1 -- start start 2024-04-20T21:17:48.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.183+0000 7fceb2600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fceac06e6d0 0x7fceac15f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:48.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.183+0000 7fceb2600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fceac072240 0x7fceac154920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:48.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.183+0000 7fceb2600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fceac146130 0x7fceac154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:48.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.183+0000 7fceb2600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fceac06e1b0 con 0x7fceac072240 2024-04-20T21:17:48.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.183+0000 7fceb2600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fceac06ded0 con 0x7fceac146130 2024-04-20T21:17:48.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.183+0000 7fceb2600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fceac06f150 con 0x7fceac06e6d0 2024-04-20T21:17:48.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.183+0000 7fcea2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fceac072240 0x7fceac154920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:48.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.183+0000 7fcea2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fceac072240 0x7fceac154920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46590/0 (socket says 172.21.15.86:46590) 2024-04-20T21:17:48.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.183+0000 7fcea2a00640 1 -- 172.21.15.86:0/1986539543 learned_addr learned my addr 172.21.15.86:0/1986539543 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:48.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.183+0000 7fcea2a00640 1 -- 172.21.15.86:0/1986539543 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fceac06e6d0 msgr2=0x7fceac15f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:48.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.183+0000 7fceabe00640 1 --2- 172.21.15.86:0/1986539543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fceac146130 0x7fceac154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:48.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.183+0000 7fcea2a00640 1 --2- 172.21.15.86:0/1986539543 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fceac06e6d0 0x7fceac15f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:48.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.184+0000 7fcea2a00640 1 -- 172.21.15.86:0/1986539543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fceac146130 msgr2=0x7fceac154e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:48.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.184+0000 7fcea2a00640 1 --2- 172.21.15.86:0/1986539543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fceac146130 0x7fceac154e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:48.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.184+0000 7fcea2a00640 1 -- 172.21.15.86:0/1986539543 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceac13e7c0 con 0x7fceac072240 2024-04-20T21:17:48.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.184+0000 7fceabe00640 1 --2- 172.21.15.86:0/1986539543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fceac146130 0x7fceac154e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:17:48.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.184+0000 7fcea2a00640 1 --2- 172.21.15.86:0/1986539543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fceac072240 0x7fceac154920 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fce94009e80 tx=0x7fce94004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:48.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.184+0000 7fcea3e00640 1 -- 172.21.15.86:0/1986539543 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce94039050 con 0x7fceac072240 2024-04-20T21:17:48.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.184+0000 7fcea3e00640 1 -- 172.21.15.86:0/1986539543 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce94045070 con 0x7fceac072240 2024-04-20T21:17:48.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.184+0000 7fceb2600640 1 -- 172.21.15.86:0/1986539543 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceac13bbc0 con 0x7fceac072240 2024-04-20T21:17:48.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.184+0000 7fcea3e00640 1 -- 172.21.15.86:0/1986539543 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce94040690 con 0x7fceac072240 2024-04-20T21:17:48.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.184+0000 7fceb2600640 1 -- 172.21.15.86:0/1986539543 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fceac15cbc0 con 0x7fceac072240 2024-04-20T21:17:48.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.186+0000 7fceb2600640 1 -- 172.21.15.86:0/1986539543 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fceac0689e0 con 0x7fceac072240 2024-04-20T21:17:48.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.186+0000 7fcea3e00640 1 -- 172.21.15.86:0/1986539543 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fce94040830 con 0x7fceac072240 2024-04-20T21:17:48.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.187+0000 7fcea3e00640 1 --2- 172.21.15.86:0/1986539543 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fce8806c8f0 0x7fce8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:48.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.187+0000 7fcea3e00640 1 -- 172.21.15.86:0/1986539543 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fce940ae1f0 con 0x7fceac072240 2024-04-20T21:17:48.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.187+0000 7fceab400640 1 --2- 172.21.15.86:0/1986539543 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fce8806c8f0 0x7fce8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:48.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.188+0000 7fceab400640 1 --2- 172.21.15.86:0/1986539543 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fce8806c8f0 0x7fce8806edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fce9c0078f0 tx=0x7fce9c000ef0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:48.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.192+0000 7fcea3e00640 1 -- 172.21.15.86:0/1986539543 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fceac13e7c0 con 0x7fceac072240 2024-04-20T21:17:48.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.390+0000 7fceb2600640 1 -- 172.21.15.86:0/1986539543 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fceac0cf000 con 0x7fceac072240 2024-04-20T21:17:48.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.391+0000 7fcea3e00640 1 -- 172.21.15.86:0/1986539543 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12884 (secure 0 0 0) 0x7fce9407c9f0 con 0x7fceac072240 2024-04-20T21:17:48.392 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:17:48.392 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":16,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:17:43.058910+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:17:48.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.394+0000 7fceb2600640 1 -- 172.21.15.86:0/1986539543 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fce8806c8f0 msgr2=0x7fce8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:48.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.394+0000 7fceb2600640 1 --2- 172.21.15.86:0/1986539543 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fce8806c8f0 0x7fce8806edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fce9c0078f0 tx=0x7fce9c000ef0 comp rx=0 tx=0).stop 2024-04-20T21:17:48.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.394+0000 7fceb2600640 1 -- 172.21.15.86:0/1986539543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fceac072240 msgr2=0x7fceac154920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:48.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.394+0000 7fceb2600640 1 --2- 172.21.15.86:0/1986539543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fceac072240 0x7fceac154920 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fce94009e80 tx=0x7fce94004a40 comp rx=0 tx=0).stop 2024-04-20T21:17:48.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.394+0000 7fceb2600640 1 -- 172.21.15.86:0/1986539543 shutdown_connections 2024-04-20T21:17:48.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.394+0000 7fceb2600640 1 --2- 172.21.15.86:0/1986539543 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fce8806c8f0 0x7fce8806edb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:48.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.394+0000 7fceb2600640 1 --2- 172.21.15.86:0/1986539543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fceac146130 0x7fceac154e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:48.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.394+0000 7fceb2600640 1 --2- 172.21.15.86:0/1986539543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fceac072240 0x7fceac154920 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:48.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.394+0000 7fceb2600640 1 --2- 172.21.15.86:0/1986539543 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fceac06e6d0 0x7fceac15f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:48.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.394+0000 7fceb2600640 1 -- 172.21.15.86:0/1986539543 >> 172.21.15.86:0/1986539543 conn(0x7fceac0b2360 msgr2=0x7fceac0b3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:48.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.394+0000 7fceb2600640 1 -- 172.21.15.86:0/1986539543 shutdown_connections 2024-04-20T21:17:48.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.394+0000 7fceb2600640 1 -- 172.21.15.86:0/1986539543 wait complete. 2024-04-20T21:17:48.412 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:17:48.412 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:17:48.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.618+0000 7fb4dee00640 1 Processor -- start 2024-04-20T21:17:48.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.618+0000 7fb4dee00640 1 -- start start 2024-04-20T21:17:48.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.619+0000 7fb4dee00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb4d806bbf0 0x7fb4d806e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:48.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.619+0000 7fb4dee00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4d806bfd0 0x7fb4d8145b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:48.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.620+0000 7fb4dee00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4d815cd40 con 0x7fb4d8146040 2024-04-20T21:17:48.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.620+0000 7fb4dee00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb4d815d000 con 0x7fb4d806bbf0 2024-04-20T21:17:48.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.620+0000 7fb4dee00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb4d815d2e0 con 0x7fb4d806bfd0 2024-04-20T21:17:48.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.620+0000 7fb4d7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb4d806bbf0 0x7fb4d806e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:48.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.620+0000 7fb4d7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb4d806bbf0 0x7fb4d806e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46602/0 (socket says 172.21.15.86:46602) 2024-04-20T21:17:48.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.620+0000 7fb4d7e00640 1 -- 172.21.15.86:0/3225716562 learned_addr learned my addr 172.21.15.86:0/3225716562 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:48.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.620+0000 7fb4dce00640 1 --1- 172.21.15.86:0/3225716562 >> v1:172.21.15.86:6789/0 conn(0x7fb4d8146040 0x7fb4d80cd540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39254/0 (socket says 172.21.15.86:39254) 2024-04-20T21:17:48.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.620+0000 7fb4d7400640 1 --2- 172.21.15.86:0/3225716562 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4d806bfd0 0x7fb4d8145b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:48.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.620+0000 7fb4d7e00640 1 -- 172.21.15.86:0/3225716562 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4d806bfd0 msgr2=0x7fb4d8145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:48.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.620+0000 7fb4d7e00640 1 --2- 172.21.15.86:0/3225716562 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4d806bfd0 0x7fb4d8145b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:48.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.620+0000 7fb4d7e00640 1 -- 172.21.15.86:0/3225716562 >> v1:172.21.15.86:6789/0 conn(0x7fb4d8146040 legacy=0x7fb4d80cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:48.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.620+0000 7fb4d7e00640 1 -- 172.21.15.86:0/3225716562 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4d815d5c0 con 0x7fb4d806bbf0 2024-04-20T21:17:48.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.620+0000 7fb4d7400640 1 --2- 172.21.15.86:0/3225716562 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4d806bfd0 0x7fb4d8145b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:17:48.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.620+0000 7fb4d6a00640 1 -- 172.21.15.86:0/3225716562 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2902564433 0 0) 0x7fb4d815cd40 con 0x7fb4d8146040 2024-04-20T21:17:48.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.622+0000 7fb4d7e00640 1 --2- 172.21.15.86:0/3225716562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4d806bbf0 0x7fb4d806e000 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8009e50 tx=0x7fb4c802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1fa7ea20138342b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:48.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.622+0000 7fb4d6a00640 1 -- 172.21.15.86:0/3225716562 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c802cb00 con 0x7fb4d806bbf0 2024-04-20T21:17:48.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.622+0000 7fb4d6a00640 1 -- 172.21.15.86:0/3225716562 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4c802cca0 con 0x7fb4d806bbf0 2024-04-20T21:17:48.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.622+0000 7fb4d6a00640 1 -- 172.21.15.86:0/3225716562 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c8034610 con 0x7fb4d806bbf0 2024-04-20T21:17:48.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.622+0000 7fb4dee00640 1 -- 172.21.15.86:0/3225716562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4d806bbf0 msgr2=0x7fb4d806e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:48.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.622+0000 7fb4dee00640 1 --2- 172.21.15.86:0/3225716562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4d806bbf0 0x7fb4d806e000 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8009e50 tx=0x7fb4c802bf60 comp rx=0 tx=0).stop 2024-04-20T21:17:48.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.622+0000 7fb4dee00640 1 -- 172.21.15.86:0/3225716562 shutdown_connections 2024-04-20T21:17:48.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.622+0000 7fb4dee00640 1 --2- 172.21.15.86:0/3225716562 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4d806bfd0 0x7fb4d8145b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:48.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.622+0000 7fb4dee00640 1 --2- 172.21.15.86:0/3225716562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4d806bbf0 0x7fb4d806e000 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:48.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.622+0000 7fb4dee00640 1 -- 172.21.15.86:0/3225716562 >> 172.21.15.86:0/3225716562 conn(0x7fb4d80b2110 msgr2=0x7fb4d80b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:48.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.623+0000 7fb4dee00640 1 -- 172.21.15.86:0/3225716562 shutdown_connections 2024-04-20T21:17:48.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.623+0000 7fb4dee00640 1 -- 172.21.15.86:0/3225716562 wait complete. 2024-04-20T21:17:48.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.624+0000 7fb4dee00640 1 Processor -- start 2024-04-20T21:17:48.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.624+0000 7fb4dee00640 1 -- start start 2024-04-20T21:17:48.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.625+0000 7fb4dee00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4d806bbf0 0x7fb4d8144740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:48.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.625+0000 7fb4dee00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4d806bfd0 0x7fb4d81e67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:48.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.625+0000 7fb4dee00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4d8146040 0x7fb4d81eb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:48.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.625+0000 7fb4dee00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d80cddd0 con 0x7fb4d806bbf0 2024-04-20T21:17:48.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.625+0000 7fb4dee00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d80cdc50 con 0x7fb4d8146040 2024-04-20T21:17:48.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.625+0000 7fb4dee00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb4d80cdf50 con 0x7fb4d806bfd0 2024-04-20T21:17:48.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.625+0000 7fb4d7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4d806bbf0 0x7fb4d8144740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:48.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.626+0000 7fb4d7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4d806bbf0 0x7fb4d8144740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46618/0 (socket says 172.21.15.86:46618) 2024-04-20T21:17:48.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.626+0000 7fb4d7e00640 1 -- 172.21.15.86:0/1778559562 learned_addr learned my addr 172.21.15.86:0/1778559562 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:48.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.626+0000 7fb4d7e00640 1 -- 172.21.15.86:0/1778559562 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4d806bfd0 msgr2=0x7fb4d81e67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:48.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.626+0000 7fb4d7e00640 1 --2- 172.21.15.86:0/1778559562 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4d806bfd0 0x7fb4d81e67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:48.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.626+0000 7fb4d7e00640 1 -- 172.21.15.86:0/1778559562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4d8146040 msgr2=0x7fb4d81eb7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:48.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.626+0000 7fb4d7e00640 1 --2- 172.21.15.86:0/1778559562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4d8146040 0x7fb4d81eb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:48.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.626+0000 7fb4d7e00640 1 -- 172.21.15.86:0/1778559562 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4d81e6d20 con 0x7fb4d806bbf0 2024-04-20T21:17:48.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.626+0000 7fb4d7e00640 1 --2- 172.21.15.86:0/1778559562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4d806bbf0 0x7fb4d8144740 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8036040 tx=0x7fb4c8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:48.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.626+0000 7fb4cfe00640 1 -- 172.21.15.86:0/1778559562 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c8043020 con 0x7fb4d806bbf0 2024-04-20T21:17:48.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.627+0000 7fb4cfe00640 1 -- 172.21.15.86:0/1778559562 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4c802ce50 con 0x7fb4d806bbf0 2024-04-20T21:17:48.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.627+0000 7fb4cfe00640 1 -- 172.21.15.86:0/1778559562 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c8009510 con 0x7fb4d806bbf0 2024-04-20T21:17:48.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.627+0000 7fb4dee00640 1 -- 172.21.15.86:0/1778559562 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4d81e73a0 con 0x7fb4d806bbf0 2024-04-20T21:17:48.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.627+0000 7fb4dee00640 1 -- 172.21.15.86:0/1778559562 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4d81e7960 con 0x7fb4d806bbf0 2024-04-20T21:17:48.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.628+0000 7fb4dee00640 1 -- 172.21.15.86:0/1778559562 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4d80689e0 con 0x7fb4d806bbf0 2024-04-20T21:17:48.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.629+0000 7fb4cfe00640 1 -- 172.21.15.86:0/1778559562 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb4c8046020 con 0x7fb4d806bbf0 2024-04-20T21:17:48.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.630+0000 7fb4cfe00640 1 --2- 172.21.15.86:0/1778559562 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb4b806c8f0 0x7fb4b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:48.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.630+0000 7fb4cfe00640 1 -- 172.21.15.86:0/1778559562 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fb4c80adfa0 con 0x7fb4d806bbf0 2024-04-20T21:17:48.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.630+0000 7fb4d7400640 1 --2- 172.21.15.86:0/1778559562 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb4b806c8f0 0x7fb4b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:48.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.631+0000 7fb4d7400640 1 --2- 172.21.15.86:0/1778559562 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb4b806c8f0 0x7fb4b806edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb4c4006fd0 tx=0x7fb4c4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:48.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.634+0000 7fb4cfe00640 1 -- 172.21.15.86:0/1778559562 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb4c804d030 con 0x7fb4d806bbf0 2024-04-20T21:17:48.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.832+0000 7fb4dee00640 1 -- 172.21.15.86:0/1778559562 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb4d80b1ef0 con 0x7fb4d806bbf0 2024-04-20T21:17:48.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.833+0000 7fb4cfe00640 1 -- 172.21.15.86:0/1778559562 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7fb4c807c720 con 0x7fb4d806bbf0 2024-04-20T21:17:48.833 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:17:48.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.836+0000 7fb4dee00640 1 -- 172.21.15.86:0/1778559562 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb4b806c8f0 msgr2=0x7fb4b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:48.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.836+0000 7fb4dee00640 1 --2- 172.21.15.86:0/1778559562 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb4b806c8f0 0x7fb4b806edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb4c4006fd0 tx=0x7fb4c4008040 comp rx=0 tx=0).stop 2024-04-20T21:17:48.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.836+0000 7fb4dee00640 1 -- 172.21.15.86:0/1778559562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4d806bbf0 msgr2=0x7fb4d8144740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:48.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.836+0000 7fb4dee00640 1 --2- 172.21.15.86:0/1778559562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4d806bbf0 0x7fb4d8144740 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8036040 tx=0x7fb4c8004a40 comp rx=0 tx=0).stop 2024-04-20T21:17:48.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.836+0000 7fb4dee00640 1 -- 172.21.15.86:0/1778559562 shutdown_connections 2024-04-20T21:17:48.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.836+0000 7fb4dee00640 1 --2- 172.21.15.86:0/1778559562 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb4b806c8f0 0x7fb4b806edb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:48.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.836+0000 7fb4dee00640 1 --2- 172.21.15.86:0/1778559562 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4d8146040 0x7fb4d81eb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:48.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.836+0000 7fb4dee00640 1 --2- 172.21.15.86:0/1778559562 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4d806bfd0 0x7fb4d81e67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:48.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.836+0000 7fb4dee00640 1 --2- 172.21.15.86:0/1778559562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4d806bbf0 0x7fb4d8144740 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:48.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.836+0000 7fb4dee00640 1 -- 172.21.15.86:0/1778559562 >> 172.21.15.86:0/1778559562 conn(0x7fb4d80b2110 msgr2=0x7fb4d815e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:48.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.836+0000 7fb4dee00640 1 -- 172.21.15.86:0/1778559562 shutdown_connections 2024-04-20T21:17:48.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:48.836+0000 7fb4dee00640 1 -- 172.21.15.86:0/1778559562 wait complete. 2024-04-20T21:17:48.855 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-20T21:17:49.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.054+0000 7fd237400640 1 Processor -- start 2024-04-20T21:17:49.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.054+0000 7fd237400640 1 -- start start 2024-04-20T21:17:49.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.056+0000 7fd237400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd230146130 0x7fd23006e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:49.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.056+0000 7fd237400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd2300ccdd0 0x7fd2300cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:49.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.056+0000 7fd237400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd23006dc30 con 0x7fd230071d00 2024-04-20T21:17:49.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.056+0000 7fd237400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd2300cd8e0 con 0x7fd230146130 2024-04-20T21:17:49.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.056+0000 7fd237400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fd2300cdbc0 con 0x7fd2300ccdd0 2024-04-20T21:17:49.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.056+0000 7fd234a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd230146130 0x7fd23006e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:49.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.056+0000 7fd234a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd230146130 0x7fd23006e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46622/0 (socket says 172.21.15.86:46622) 2024-04-20T21:17:49.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.056+0000 7fd234a00640 1 -- 172.21.15.86:0/1456434413 learned_addr learned my addr 172.21.15.86:0/1456434413 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:49.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.056+0000 7fd235400640 1 --1- 172.21.15.86:0/1456434413 >> v1:172.21.15.86:6789/0 conn(0x7fd230071d00 0x7fd23015cc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39266/0 (socket says 172.21.15.86:39266) 2024-04-20T21:17:49.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.057+0000 7fd22fe00640 1 --2- 172.21.15.86:0/1456434413 >> v2:172.21.15.144:3300/0 conn(0x7fd2300ccdd0 0x7fd2300cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:49.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.057+0000 7fd234a00640 1 -- 172.21.15.86:0/1456434413 >> v2:172.21.15.144:3300/0 conn(0x7fd2300ccdd0 msgr2=0x7fd2300cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:49.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.057+0000 7fd234a00640 1 --2- 172.21.15.86:0/1456434413 >> v2:172.21.15.144:3300/0 conn(0x7fd2300ccdd0 0x7fd2300cd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:49.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.057+0000 7fd234a00640 1 -- 172.21.15.86:0/1456434413 >> v1:172.21.15.86:6789/0 conn(0x7fd230071d00 legacy=0x7fd23015cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:49.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.057+0000 7fd234a00640 1 -- 172.21.15.86:0/1456434413 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd23006ded0 con 0x7fd230146130 2024-04-20T21:17:49.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.057+0000 7fd22fe00640 1 --2- 172.21.15.86:0/1456434413 >> v2:172.21.15.144:3300/0 conn(0x7fd2300ccdd0 0x7fd2300cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:17:49.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.057+0000 7fd22f400640 1 -- 172.21.15.86:0/1456434413 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2266039771 0 0) 0x7fd23006dc30 con 0x7fd230071d00 2024-04-20T21:17:49.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.058+0000 7fd234a00640 1 --2- 172.21.15.86:0/1456434413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd230146130 0x7fd23006e6d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd23006de00 tx=0x7fd22402ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71a891ea9d66ea1d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:49.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.058+0000 7fd22f400640 1 -- 172.21.15.86:0/1456434413 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd22402f840 con 0x7fd230146130 2024-04-20T21:17:49.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.058+0000 7fd22f400640 1 -- 172.21.15.86:0/1456434413 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd22402f9e0 con 0x7fd230146130 2024-04-20T21:17:49.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.058+0000 7fd22f400640 1 -- 172.21.15.86:0/1456434413 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd22402fdb0 con 0x7fd230146130 2024-04-20T21:17:49.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.059+0000 7fd237400640 1 -- 172.21.15.86:0/1456434413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd230146130 msgr2=0x7fd23006e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:49.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.059+0000 7fd237400640 1 --2- 172.21.15.86:0/1456434413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd230146130 0x7fd23006e6d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd23006de00 tx=0x7fd22402ba20 comp rx=0 tx=0).stop 2024-04-20T21:17:49.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.059+0000 7fd237400640 1 -- 172.21.15.86:0/1456434413 shutdown_connections 2024-04-20T21:17:49.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.059+0000 7fd237400640 1 --2- 172.21.15.86:0/1456434413 >> v2:172.21.15.144:3300/0 conn(0x7fd2300ccdd0 0x7fd2300cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:49.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.059+0000 7fd237400640 1 --2- 172.21.15.86:0/1456434413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd230146130 0x7fd23006e6d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:49.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.059+0000 7fd237400640 1 -- 172.21.15.86:0/1456434413 >> 172.21.15.86:0/1456434413 conn(0x7fd2300b2360 msgr2=0x7fd2300b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:49.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.059+0000 7fd237400640 1 -- 172.21.15.86:0/1456434413 shutdown_connections 2024-04-20T21:17:49.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.059+0000 7fd237400640 1 -- 172.21.15.86:0/1456434413 wait complete. 2024-04-20T21:17:49.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.061+0000 7fd237400640 1 Processor -- start 2024-04-20T21:17:49.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.061+0000 7fd237400640 1 -- start start 2024-04-20T21:17:49.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.062+0000 7fd237400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd230071d00 0x7fd23015cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:49.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.062+0000 7fd237400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2300ccdd0 0x7fd23015c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:49.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.062+0000 7fd237400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd230146130 0x7fd23015f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:49.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.062+0000 7fd237400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd230072150 con 0x7fd230071d00 2024-04-20T21:17:49.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.062+0000 7fd237400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd23006f150 con 0x7fd2300ccdd0 2024-04-20T21:17:49.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.062+0000 7fd237400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd2300722d0 con 0x7fd230146130 2024-04-20T21:17:49.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.062+0000 7fd234a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd230071d00 0x7fd23015cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:49.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.062+0000 7fd234a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd230071d00 0x7fd23015cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46624/0 (socket says 172.21.15.86:46624) 2024-04-20T21:17:49.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.062+0000 7fd234a00640 1 -- 172.21.15.86:0/2293362002 learned_addr learned my addr 172.21.15.86:0/2293362002 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:49.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.062+0000 7fd235400640 1 --2- 172.21.15.86:0/2293362002 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd230146130 0x7fd23015f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:49.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.062+0000 7fd234a00640 1 -- 172.21.15.86:0/2293362002 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd230146130 msgr2=0x7fd23015f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:49.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.062+0000 7fd22fe00640 1 --2- 172.21.15.86:0/2293362002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2300ccdd0 0x7fd23015c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:49.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.062+0000 7fd234a00640 1 --2- 172.21.15.86:0/2293362002 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd230146130 0x7fd23015f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:49.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.062+0000 7fd234a00640 1 -- 172.21.15.86:0/2293362002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2300ccdd0 msgr2=0x7fd23015c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:49.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.063+0000 7fd234a00640 1 --2- 172.21.15.86:0/2293362002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2300ccdd0 0x7fd23015c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:49.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.063+0000 7fd234a00640 1 -- 172.21.15.86:0/2293362002 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd23015c620 con 0x7fd230071d00 2024-04-20T21:17:49.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.063+0000 7fd234a00640 1 --2- 172.21.15.86:0/2293362002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd230071d00 0x7fd23015cb20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd224000c00 tx=0x7fd224004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:49.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.063+0000 7fd22cc00640 1 -- 172.21.15.86:0/2293362002 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd224047070 con 0x7fd230071d00 2024-04-20T21:17:49.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.063+0000 7fd22cc00640 1 -- 172.21.15.86:0/2293362002 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd224037d40 con 0x7fd230071d00 2024-04-20T21:17:49.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.063+0000 7fd22cc00640 1 -- 172.21.15.86:0/2293362002 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd224004270 con 0x7fd230071d00 2024-04-20T21:17:49.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.063+0000 7fd237400640 1 -- 172.21.15.86:0/2293362002 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd23015d1e0 con 0x7fd230071d00 2024-04-20T21:17:49.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.063+0000 7fd237400640 1 -- 172.21.15.86:0/2293362002 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd23015d7a0 con 0x7fd230071d00 2024-04-20T21:17:49.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.065+0000 7fd237400640 1 -- 172.21.15.86:0/2293362002 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2300689e0 con 0x7fd230071d00 2024-04-20T21:17:49.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.065+0000 7fd22cc00640 1 -- 172.21.15.86:0/2293362002 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd224003610 con 0x7fd230071d00 2024-04-20T21:17:49.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.067+0000 7fd22cc00640 1 --2- 172.21.15.86:0/2293362002 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd21006c8f0 0x7fd21006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:49.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.067+0000 7fd22cc00640 1 -- 172.21.15.86:0/2293362002 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fd2240adaf0 con 0x7fd230071d00 2024-04-20T21:17:49.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.067+0000 7fd22fe00640 1 --2- 172.21.15.86:0/2293362002 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd21006c8f0 0x7fd21006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:49.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.068+0000 7fd22fe00640 1 --2- 172.21.15.86:0/2293362002 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd21006c8f0 0x7fd21006edb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd2300c6c00 tx=0x7fd21c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:49.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.071+0000 7fd22cc00640 1 -- 172.21.15.86:0/2293362002 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd23015c620 con 0x7fd230071d00 2024-04-20T21:17:49.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.273+0000 7fd237400640 1 -- 172.21.15.86:0/2293362002 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fd23015ff50 con 0x7fd230071d00 2024-04-20T21:17:49.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.274+0000 7fd22cc00640 1 -- 172.21.15.86:0/2293362002 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7fd224033030 con 0x7fd230071d00 2024-04-20T21:17:49.275 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:17:49.275 INFO:teuthology.orchestra.run.smithi086.stdout:[ 2024-04-20T21:17:49.275 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:17:49.275 INFO:teuthology.orchestra.run.smithi086.stdout: "poolnum": 1, 2024-04-20T21:17:49.275 INFO:teuthology.orchestra.run.smithi086.stdout: "poolname": ".mgr" 2024-04-20T21:17:49.275 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:17:49.275 INFO:teuthology.orchestra.run.smithi086.stdout:] 2024-04-20T21:17:49.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.277+0000 7fd237400640 1 -- 172.21.15.86:0/2293362002 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd21006c8f0 msgr2=0x7fd21006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:49.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.277+0000 7fd237400640 1 --2- 172.21.15.86:0/2293362002 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd21006c8f0 0x7fd21006edb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd2300c6c00 tx=0x7fd21c008040 comp rx=0 tx=0).stop 2024-04-20T21:17:49.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.277+0000 7fd237400640 1 -- 172.21.15.86:0/2293362002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd230071d00 msgr2=0x7fd23015cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:49.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.277+0000 7fd237400640 1 --2- 172.21.15.86:0/2293362002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd230071d00 0x7fd23015cb20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd224000c00 tx=0x7fd224004730 comp rx=0 tx=0).stop 2024-04-20T21:17:49.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.277+0000 7fd237400640 1 -- 172.21.15.86:0/2293362002 shutdown_connections 2024-04-20T21:17:49.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.277+0000 7fd237400640 1 --2- 172.21.15.86:0/2293362002 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd21006c8f0 0x7fd21006edb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:49.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.277+0000 7fd237400640 1 --2- 172.21.15.86:0/2293362002 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd230146130 0x7fd23015f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:49.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.277+0000 7fd237400640 1 --2- 172.21.15.86:0/2293362002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2300ccdd0 0x7fd23015c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:49.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.277+0000 7fd237400640 1 --2- 172.21.15.86:0/2293362002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd230071d00 0x7fd23015cb20 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:49.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.277+0000 7fd237400640 1 -- 172.21.15.86:0/2293362002 >> 172.21.15.86:0/2293362002 conn(0x7fd2300b2360 msgr2=0x7fd2300b4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:49.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.277+0000 7fd237400640 1 -- 172.21.15.86:0/2293362002 shutdown_connections 2024-04-20T21:17:49.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.277+0000 7fd237400640 1 -- 172.21.15.86:0/2293362002 wait complete. 2024-04-20T21:17:49.296 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-20T21:17:49.296 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-20T21:17:49.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.487+0000 7f867ae00640 1 Processor -- start 2024-04-20T21:17:49.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.488+0000 7f867ae00640 1 -- start start 2024-04-20T21:17:49.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.489+0000 7f867ae00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8674153590 0x7f86740d0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:49.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.489+0000 7f867ae00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8674071ad0 0x7f86740d0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:49.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.489+0000 7f867ae00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f867415cbe0 con 0x7f86740af160 2024-04-20T21:17:49.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.489+0000 7f867ae00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8674153970 con 0x7f8674153590 2024-04-20T21:17:49.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.489+0000 7f867ae00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f867415cea0 con 0x7f8674071ad0 2024-04-20T21:17:49.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.489+0000 7f8679a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8674153590 0x7f86740d0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:49.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.489+0000 7f8679a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8674153590 0x7f86740d0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46628/0 (socket says 172.21.15.86:46628) 2024-04-20T21:17:49.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.490+0000 7f8679a00640 1 -- 172.21.15.86:0/4259434189 learned_addr learned my addr 172.21.15.86:0/4259434189 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:49.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.490+0000 7f8679000640 1 --2- 172.21.15.86:0/4259434189 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8674071ad0 0x7f86740d0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:49.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.490+0000 7f8679a00640 1 -- 172.21.15.86:0/4259434189 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8674071ad0 msgr2=0x7f86740d0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:49.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.490+0000 7f8679a00640 1 --2- 172.21.15.86:0/4259434189 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8674071ad0 0x7f86740d0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:49.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.490+0000 7f8679a00640 1 -- 172.21.15.86:0/4259434189 >> v1:172.21.15.144:6789/0 conn(0x7f86740af160 legacy=0x7f867406dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:49.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.490+0000 7f8679a00640 1 -- 172.21.15.86:0/4259434189 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f867415d180 con 0x7f8674153590 2024-04-20T21:17:49.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.491+0000 7f8679a00640 1 --2- 172.21.15.86:0/4259434189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8674153590 0x7f86740d0610 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8668009870 tx=0x7f866802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5692dd752dc883de server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:49.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.491+0000 7f8673e00640 1 -- 172.21.15.86:0/4259434189 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8668035020 con 0x7f8674153590 2024-04-20T21:17:49.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.491+0000 7f8673e00640 1 -- 172.21.15.86:0/4259434189 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8668031640 con 0x7f8674153590 2024-04-20T21:17:49.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.491+0000 7f8673e00640 1 -- 172.21.15.86:0/4259434189 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866802d040 con 0x7f8674153590 2024-04-20T21:17:49.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.492+0000 7f867ae00640 1 -- 172.21.15.86:0/4259434189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8674153590 msgr2=0x7f86740d0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:49.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.492+0000 7f867ae00640 1 --2- 172.21.15.86:0/4259434189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8674153590 0x7f86740d0610 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8668009870 tx=0x7f866802b8f0 comp rx=0 tx=0).stop 2024-04-20T21:17:49.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.492+0000 7f867ae00640 1 -- 172.21.15.86:0/4259434189 shutdown_connections 2024-04-20T21:17:49.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.492+0000 7f867ae00640 1 --2- 172.21.15.86:0/4259434189 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8674071ad0 0x7f86740d0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:49.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.492+0000 7f867ae00640 1 --2- 172.21.15.86:0/4259434189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8674153590 0x7f86740d0610 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:49.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.492+0000 7f867ae00640 1 -- 172.21.15.86:0/4259434189 >> 172.21.15.86:0/4259434189 conn(0x7f86740b2180 msgr2=0x7f86740b2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:49.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.492+0000 7f867ae00640 1 -- 172.21.15.86:0/4259434189 shutdown_connections 2024-04-20T21:17:49.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.492+0000 7f867ae00640 1 -- 172.21.15.86:0/4259434189 wait complete. 2024-04-20T21:17:49.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.494+0000 7f867ae00640 1 Processor -- start 2024-04-20T21:17:49.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.494+0000 7f867ae00640 1 -- start start 2024-04-20T21:17:49.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.495+0000 7f867ae00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8674071ad0 0x7f867414d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:49.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.495+0000 7f867ae00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86740af160 0x7f867414ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:49.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.495+0000 7f867ae00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8674153590 0x7f86740b0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:49.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.495+0000 7f867ae00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f86740af5f0 con 0x7f86740af160 2024-04-20T21:17:49.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.495+0000 7f867ae00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f867415d740 con 0x7f8674153590 2024-04-20T21:17:49.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.495+0000 7f867ae00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f86740af770 con 0x7f8674071ad0 2024-04-20T21:17:49.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.496+0000 7f8679000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86740af160 0x7f867414ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:49.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.496+0000 7f8679000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86740af160 0x7f867414ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46632/0 (socket says 172.21.15.86:46632) 2024-04-20T21:17:49.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.496+0000 7f8679000640 1 -- 172.21.15.86:0/3084272954 learned_addr learned my addr 172.21.15.86:0/3084272954 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:49.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.496+0000 7f8679a00640 1 --2- 172.21.15.86:0/3084272954 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8674071ad0 0x7f867414d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:49.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.496+0000 7f867a400640 1 --2- 172.21.15.86:0/3084272954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8674153590 0x7f86740b0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:49.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.496+0000 7f8679000640 1 -- 172.21.15.86:0/3084272954 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8674071ad0 msgr2=0x7f867414d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:49.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.496+0000 7f8679000640 1 --2- 172.21.15.86:0/3084272954 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8674071ad0 0x7f867414d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:49.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.496+0000 7f8679000640 1 -- 172.21.15.86:0/3084272954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8674153590 msgr2=0x7f86740b0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:49.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.496+0000 7f8679000640 1 --2- 172.21.15.86:0/3084272954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8674153590 0x7f86740b0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:49.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.496+0000 7f8679000640 1 -- 172.21.15.86:0/3084272954 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86740afd00 con 0x7f86740af160 2024-04-20T21:17:49.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.496+0000 7f8679000640 1 --2- 172.21.15.86:0/3084272954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86740af160 0x7f867414ee20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f866000d800 tx=0x7f866000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:49.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.497+0000 7f8671600640 1 -- 172.21.15.86:0/3084272954 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866000b6c0 con 0x7f86740af160 2024-04-20T21:17:49.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.497+0000 7f8671600640 1 -- 172.21.15.86:0/3084272954 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8660019070 con 0x7f86740af160 2024-04-20T21:17:49.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.497+0000 7f8671600640 1 -- 172.21.15.86:0/3084272954 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866000ba80 con 0x7f86740af160 2024-04-20T21:17:49.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.497+0000 7f867ae00640 1 -- 172.21.15.86:0/3084272954 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86740affe0 con 0x7f86740af160 2024-04-20T21:17:49.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.497+0000 7f867ae00640 1 -- 172.21.15.86:0/3084272954 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86740b05a0 con 0x7f86740af160 2024-04-20T21:17:49.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.498+0000 7f867ae00640 1 -- 172.21.15.86:0/3084272954 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f867404fcf0 con 0x7f86740af160 2024-04-20T21:17:49.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.499+0000 7f8671600640 1 -- 172.21.15.86:0/3084272954 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f866000bc20 con 0x7f86740af160 2024-04-20T21:17:49.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.500+0000 7f8671600640 1 --2- 172.21.15.86:0/3084272954 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f865406c8f0 0x7f865406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:49.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.500+0000 7f8671600640 1 -- 172.21.15.86:0/3084272954 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f866008bc30 con 0x7f86740af160 2024-04-20T21:17:49.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.500+0000 7f8679a00640 1 --2- 172.21.15.86:0/3084272954 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f865406c8f0 0x7f865406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:49.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.501+0000 7f8679a00640 1 --2- 172.21.15.86:0/3084272954 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f865406c8f0 0x7f865406edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8668036000 tx=0x7f866803c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:49.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.505+0000 7f8671600640 1 -- 172.21.15.86:0/3084272954 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f866005a3b0 con 0x7f86740af160 2024-04-20T21:17:49.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:49.705+0000 7f867ae00640 1 -- 172.21.15.86:0/3084272954 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f86740cf370 con 0x7f86740af160 2024-04-20T21:17:50.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.227+0000 7f8671600640 1 -- 172.21.15.86:0/3084272954 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7f8660059cb0 con 0x7f86740af160 2024-04-20T21:17:50.228 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' created 2024-04-20T21:17:50.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.232+0000 7f867ae00640 1 -- 172.21.15.86:0/3084272954 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f865406c8f0 msgr2=0x7f865406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:50.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.232+0000 7f867ae00640 1 --2- 172.21.15.86:0/3084272954 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f865406c8f0 0x7f865406edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8668036000 tx=0x7f866803c040 comp rx=0 tx=0).stop 2024-04-20T21:17:50.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.232+0000 7f867ae00640 1 -- 172.21.15.86:0/3084272954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86740af160 msgr2=0x7f867414ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:50.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.232+0000 7f867ae00640 1 --2- 172.21.15.86:0/3084272954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86740af160 0x7f867414ee20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f866000d800 tx=0x7f866000dcc0 comp rx=0 tx=0).stop 2024-04-20T21:17:50.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.233+0000 7f867ae00640 1 -- 172.21.15.86:0/3084272954 shutdown_connections 2024-04-20T21:17:50.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.233+0000 7f867ae00640 1 --2- 172.21.15.86:0/3084272954 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f865406c8f0 0x7f865406edb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:50.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.233+0000 7f867ae00640 1 --2- 172.21.15.86:0/3084272954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8674153590 0x7f86740b0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:50.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.233+0000 7f867ae00640 1 --2- 172.21.15.86:0/3084272954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86740af160 0x7f867414ee20 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:50.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.233+0000 7f867ae00640 1 --2- 172.21.15.86:0/3084272954 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8674071ad0 0x7f867414d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:50.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.233+0000 7f867ae00640 1 -- 172.21.15.86:0/3084272954 >> 172.21.15.86:0/3084272954 conn(0x7f86740b2180 msgr2=0x7f867406e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:50.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.233+0000 7f867ae00640 1 -- 172.21.15.86:0/3084272954 shutdown_connections 2024-04-20T21:17:50.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.234+0000 7f867ae00640 1 -- 172.21.15.86:0/3084272954 wait complete. 2024-04-20T21:17:50.260 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-20T21:17:50.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.451+0000 7f8a39000640 1 Processor -- start 2024-04-20T21:17:50.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.451+0000 7f8a39000640 1 -- start start 2024-04-20T21:17:50.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.452+0000 7f8a39000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a3415eb30 0x7f8a340b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:50.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.452+0000 7f8a39000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8a340aee50 0x7f8a340b4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:50.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.452+0000 7f8a39000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8a3406b980 0x7f8a3406be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:50.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.452+0000 7f8a39000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8a340af7d0 con 0x7f8a340aee50 2024-04-20T21:17:50.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.453+0000 7f8a39000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f8a3406c370 con 0x7f8a3406b980 2024-04-20T21:17:50.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.453+0000 7f8a39000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8a340adb10 con 0x7f8a3415eb30 2024-04-20T21:17:50.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.453+0000 7f8a29400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8a340aee50 0x7f8a340b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:50.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.453+0000 7f8a29400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8a340aee50 0x7f8a340b4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46638/0 (socket says 172.21.15.86:46638) 2024-04-20T21:17:50.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.453+0000 7f8a29400640 1 -- 172.21.15.86:0/2950746695 learned_addr learned my addr 172.21.15.86:0/2950746695 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:50.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.453+0000 7f8a31e00640 1 --2- 172.21.15.86:0/2950746695 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a3415eb30 0x7f8a340b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:50.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.453+0000 7f8a32800640 1 --2- 172.21.15.86:0/2950746695 >> v2:172.21.15.144:3300/0 conn(0x7f8a3406b980 0x7f8a3406be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:50.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.453+0000 7f8a29400640 1 -- 172.21.15.86:0/2950746695 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a3415eb30 msgr2=0x7f8a340b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:50.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.453+0000 7f8a29400640 1 --2- 172.21.15.86:0/2950746695 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a3415eb30 0x7f8a340b3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:50.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.453+0000 7f8a29400640 1 -- 172.21.15.86:0/2950746695 >> v2:172.21.15.144:3300/0 conn(0x7f8a3406b980 msgr2=0x7f8a3406be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:50.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.453+0000 7f8a29400640 1 --2- 172.21.15.86:0/2950746695 >> v2:172.21.15.144:3300/0 conn(0x7f8a3406b980 0x7f8a3406be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:50.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.453+0000 7f8a29400640 1 -- 172.21.15.86:0/2950746695 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a340addb0 con 0x7f8a340aee50 2024-04-20T21:17:50.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.455+0000 7f8a29400640 1 --2- 172.21.15.86:0/2950746695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a340aee50 0x7f8a340b4360 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8a340b0010 tx=0x7f8a1c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2be3d0e7c3c189bc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:50.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.455+0000 7f8a31400640 1 -- 172.21.15.86:0/2950746695 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c035020 con 0x7f8a340aee50 2024-04-20T21:17:50.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.455+0000 7f8a31400640 1 -- 172.21.15.86:0/2950746695 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a1c031870 con 0x7f8a340aee50 2024-04-20T21:17:50.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.455+0000 7f8a31400640 1 -- 172.21.15.86:0/2950746695 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c031c40 con 0x7f8a340aee50 2024-04-20T21:17:50.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.456+0000 7f8a39000640 1 -- 172.21.15.86:0/2950746695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a340aee50 msgr2=0x7f8a340b4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:50.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.456+0000 7f8a39000640 1 --2- 172.21.15.86:0/2950746695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a340aee50 0x7f8a340b4360 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8a340b0010 tx=0x7f8a1c02b910 comp rx=0 tx=0).stop 2024-04-20T21:17:50.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.456+0000 7f8a39000640 1 -- 172.21.15.86:0/2950746695 shutdown_connections 2024-04-20T21:17:50.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.456+0000 7f8a39000640 1 --2- 172.21.15.86:0/2950746695 >> v2:172.21.15.144:3300/0 conn(0x7f8a3406b980 0x7f8a3406be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:50.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.456+0000 7f8a39000640 1 --2- 172.21.15.86:0/2950746695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a340aee50 0x7f8a340b4360 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:50.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.456+0000 7f8a39000640 1 --2- 172.21.15.86:0/2950746695 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a3415eb30 0x7f8a340b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:50.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.456+0000 7f8a39000640 1 -- 172.21.15.86:0/2950746695 >> 172.21.15.86:0/2950746695 conn(0x7f8a340b2110 msgr2=0x7f8a340b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:50.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.456+0000 7f8a39000640 1 -- 172.21.15.86:0/2950746695 shutdown_connections 2024-04-20T21:17:50.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.456+0000 7f8a39000640 1 -- 172.21.15.86:0/2950746695 wait complete. 2024-04-20T21:17:50.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.458+0000 7f8a39000640 1 Processor -- start 2024-04-20T21:17:50.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.458+0000 7f8a39000640 1 -- start start 2024-04-20T21:17:50.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.459+0000 7f8a39000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a3406b980 0x7f8a340cf460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:50.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.459+0000 7f8a39000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a340aee50 0x7f8a340cf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:50.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.459+0000 7f8a39000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a3415eb30 0x7f8a341485e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:50.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.459+0000 7f8a39000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8a340ae650 con 0x7f8a3415eb30 2024-04-20T21:17:50.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.459+0000 7f8a39000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8a340ae370 con 0x7f8a3406b980 2024-04-20T21:17:50.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.459+0000 7f8a39000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8a340ae930 con 0x7f8a340aee50 2024-04-20T21:17:50.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.460+0000 7f8a32800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a3415eb30 0x7f8a341485e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:50.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.460+0000 7f8a32800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a3415eb30 0x7f8a341485e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46648/0 (socket says 172.21.15.86:46648) 2024-04-20T21:17:50.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.460+0000 7f8a32800640 1 -- 172.21.15.86:0/1937389291 learned_addr learned my addr 172.21.15.86:0/1937389291 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:50.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.460+0000 7f8a31e00640 1 --2- 172.21.15.86:0/1937389291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a3406b980 0x7f8a340cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:50.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.460+0000 7f8a29400640 1 --2- 172.21.15.86:0/1937389291 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a340aee50 0x7f8a340cf9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:50.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.460+0000 7f8a32800640 1 -- 172.21.15.86:0/1937389291 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a340aee50 msgr2=0x7f8a340cf9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:50.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.460+0000 7f8a32800640 1 --2- 172.21.15.86:0/1937389291 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a340aee50 0x7f8a340cf9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:50.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.460+0000 7f8a32800640 1 -- 172.21.15.86:0/1937389291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a3406b980 msgr2=0x7f8a340cf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:50.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.460+0000 7f8a32800640 1 --2- 172.21.15.86:0/1937389291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a3406b980 0x7f8a340cf460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:50.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.460+0000 7f8a32800640 1 -- 172.21.15.86:0/1937389291 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a34157370 con 0x7f8a3415eb30 2024-04-20T21:17:50.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.460+0000 7f8a32800640 1 --2- 172.21.15.86:0/1937389291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a3415eb30 0x7f8a341485e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c007d20 tx=0x7f8a2c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:50.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.461+0000 7f8a2aa00640 1 -- 172.21.15.86:0/1937389291 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c014420 con 0x7f8a3415eb30 2024-04-20T21:17:50.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.461+0000 7f8a2aa00640 1 -- 172.21.15.86:0/1937389291 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a2c019070 con 0x7f8a3415eb30 2024-04-20T21:17:50.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.461+0000 7f8a39000640 1 -- 172.21.15.86:0/1937389291 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a34157650 con 0x7f8a3415eb30 2024-04-20T21:17:50.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.461+0000 7f8a2aa00640 1 -- 172.21.15.86:0/1937389291 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c014820 con 0x7f8a3415eb30 2024-04-20T21:17:50.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.461+0000 7f8a39000640 1 -- 172.21.15.86:0/1937389291 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a34149580 con 0x7f8a3415eb30 2024-04-20T21:17:50.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.462+0000 7f8a39000640 1 -- 172.21.15.86:0/1937389291 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a340689e0 con 0x7f8a3415eb30 2024-04-20T21:17:50.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.463+0000 7f8a2aa00640 1 -- 172.21.15.86:0/1937389291 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8a2c0149c0 con 0x7f8a3415eb30 2024-04-20T21:17:50.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.464+0000 7f8a2aa00640 1 --2- 172.21.15.86:0/1937389291 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8a1006c8f0 0x7f8a1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:50.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.464+0000 7f8a2aa00640 1 -- 172.21.15.86:0/1937389291 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7f8a2c08d0d0 con 0x7f8a3415eb30 2024-04-20T21:17:50.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.465+0000 7f8a31e00640 1 --2- 172.21.15.86:0/1937389291 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8a1006c8f0 0x7f8a1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:50.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.465+0000 7f8a31e00640 1 --2- 172.21.15.86:0/1937389291 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8a1006c8f0 0x7f8a1006edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8a24006fd0 tx=0x7f8a24008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:50.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.470+0000 7f8a2aa00640 1 -- 172.21.15.86:0/1937389291 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8a2c01c360 con 0x7f8a3415eb30 2024-04-20T21:17:50.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:50.669+0000 7f8a39000640 1 -- 172.21.15.86:0/1937389291 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f8a340b4270 con 0x7f8a3415eb30 2024-04-20T21:17:51.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.234+0000 7f8a2aa00640 1 -- 172.21.15.86:0/1937389291 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) ==== 170+0+0 (secure 0 0 0) 0x7f8a2c05b6c0 con 0x7f8a3415eb30 2024-04-20T21:17:51.235 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' created 2024-04-20T21:17:51.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.238+0000 7f8a39000640 1 -- 172.21.15.86:0/1937389291 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8a1006c8f0 msgr2=0x7f8a1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:51.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.238+0000 7f8a39000640 1 --2- 172.21.15.86:0/1937389291 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8a1006c8f0 0x7f8a1006edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8a24006fd0 tx=0x7f8a24008040 comp rx=0 tx=0).stop 2024-04-20T21:17:51.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.238+0000 7f8a39000640 1 -- 172.21.15.86:0/1937389291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a3415eb30 msgr2=0x7f8a341485e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:51.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.238+0000 7f8a39000640 1 --2- 172.21.15.86:0/1937389291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a3415eb30 0x7f8a341485e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c007d20 tx=0x7f8a2c00e4b0 comp rx=0 tx=0).stop 2024-04-20T21:17:51.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.238+0000 7f8a39000640 1 -- 172.21.15.86:0/1937389291 shutdown_connections 2024-04-20T21:17:51.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.238+0000 7f8a39000640 1 --2- 172.21.15.86:0/1937389291 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8a1006c8f0 0x7f8a1006edb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:51.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.238+0000 7f8a39000640 1 --2- 172.21.15.86:0/1937389291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a3415eb30 0x7f8a341485e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:51.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.238+0000 7f8a39000640 1 --2- 172.21.15.86:0/1937389291 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a340aee50 0x7f8a340cf9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:51.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.238+0000 7f8a39000640 1 --2- 172.21.15.86:0/1937389291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a3406b980 0x7f8a340cf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:51.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.238+0000 7f8a39000640 1 -- 172.21.15.86:0/1937389291 >> 172.21.15.86:0/1937389291 conn(0x7f8a340b2110 msgr2=0x7f8a340b1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:51.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.238+0000 7f8a39000640 1 -- 172.21.15.86:0/1937389291 shutdown_connections 2024-04-20T21:17:51.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.239+0000 7f8a39000640 1 -- 172.21.15.86:0/1937389291 wait complete. 2024-04-20T21:17:51.275 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-20T21:17:51.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.443+0000 7f9c43200640 1 Processor -- start 2024-04-20T21:17:51.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.443+0000 7f9c43200640 1 -- start start 2024-04-20T21:17:51.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.445+0000 7f9c43200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9c44161c60 0x7f9c4415f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:51.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.445+0000 7f9c43200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9c4415ff10 0x7f9c4406dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:51.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.445+0000 7f9c43200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9c44071d40 0x7f9c440721d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:51.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.445+0000 7f9c43200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9c440aecd0 con 0x7f9c4415ff10 2024-04-20T21:17:51.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.445+0000 7f9c43200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f9c4406e2e0 con 0x7f9c44161c60 2024-04-20T21:17:51.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.445+0000 7f9c43200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9c4406e580 con 0x7f9c44071d40 2024-04-20T21:17:51.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.445+0000 7f9c39400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9c4415ff10 0x7f9c4406dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:51.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.445+0000 7f9c39400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9c4415ff10 0x7f9c4406dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46654/0 (socket says 172.21.15.86:46654) 2024-04-20T21:17:51.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.445+0000 7f9c39400640 1 -- 172.21.15.86:0/3543333389 learned_addr learned my addr 172.21.15.86:0/3543333389 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:51.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.445+0000 7f9c42800640 1 --2- 172.21.15.86:0/3543333389 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9c44071d40 0x7f9c440721d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:51.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.445+0000 7f9c39400640 1 -- 172.21.15.86:0/3543333389 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9c44071d40 msgr2=0x7f9c440721d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:51.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.445+0000 7f9c39400640 1 --2- 172.21.15.86:0/3543333389 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9c44071d40 0x7f9c440721d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:51.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.445+0000 7f9c39400640 1 -- 172.21.15.86:0/3543333389 >> v2:172.21.15.144:3300/0 conn(0x7f9c44161c60 msgr2=0x7f9c4415f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:51.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.445+0000 7f9c39400640 1 --2- 172.21.15.86:0/3543333389 >> v2:172.21.15.144:3300/0 conn(0x7f9c44161c60 0x7f9c4415f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:51.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.445+0000 7f9c39400640 1 -- 172.21.15.86:0/3543333389 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c4406e860 con 0x7f9c4415ff10 2024-04-20T21:17:51.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.446+0000 7f9c42800640 1 --2- 172.21.15.86:0/3543333389 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9c44071d40 0x7f9c440721d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:17:51.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.447+0000 7f9c39400640 1 --2- 172.21.15.86:0/3543333389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c4415ff10 0x7f9c4406dda0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9c4406c4d0 tx=0x7f9c2c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecb2c8dd687b2d75 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:51.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.447+0000 7f9c41400640 1 -- 172.21.15.86:0/3543333389 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c035020 con 0x7f9c4415ff10 2024-04-20T21:17:51.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.447+0000 7f9c41400640 1 -- 172.21.15.86:0/3543333389 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c2c031870 con 0x7f9c4415ff10 2024-04-20T21:17:51.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.447+0000 7f9c41400640 1 -- 172.21.15.86:0/3543333389 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c031c40 con 0x7f9c4415ff10 2024-04-20T21:17:51.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.447+0000 7f9c43200640 1 -- 172.21.15.86:0/3543333389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c4415ff10 msgr2=0x7f9c4406dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:51.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.447+0000 7f9c43200640 1 --2- 172.21.15.86:0/3543333389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c4415ff10 0x7f9c4406dda0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9c4406c4d0 tx=0x7f9c2c02b910 comp rx=0 tx=0).stop 2024-04-20T21:17:51.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.448+0000 7f9c43200640 1 -- 172.21.15.86:0/3543333389 shutdown_connections 2024-04-20T21:17:51.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.448+0000 7f9c43200640 1 --2- 172.21.15.86:0/3543333389 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9c44071d40 0x7f9c440721d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:51.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.448+0000 7f9c43200640 1 --2- 172.21.15.86:0/3543333389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c4415ff10 0x7f9c4406dda0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:51.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.448+0000 7f9c43200640 1 --2- 172.21.15.86:0/3543333389 >> v2:172.21.15.144:3300/0 conn(0x7f9c44161c60 0x7f9c4415f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:51.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.448+0000 7f9c43200640 1 -- 172.21.15.86:0/3543333389 >> 172.21.15.86:0/3543333389 conn(0x7f9c44067cf0 msgr2=0x7f9c4406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:51.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.448+0000 7f9c43200640 1 -- 172.21.15.86:0/3543333389 shutdown_connections 2024-04-20T21:17:51.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.448+0000 7f9c43200640 1 -- 172.21.15.86:0/3543333389 wait complete. 2024-04-20T21:17:51.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.449+0000 7f9c43200640 1 Processor -- start 2024-04-20T21:17:51.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.449+0000 7f9c43200640 1 -- start start 2024-04-20T21:17:51.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.450+0000 7f9c43200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c44071d40 0x7f9c441f7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:51.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.450+0000 7f9c43200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c4415ff10 0x7f9c44158250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:51.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.450+0000 7f9c43200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9c44161c60 0x7f9c440c53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:51.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.450+0000 7f9c43200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9c44162040 con 0x7f9c4415ff10 2024-04-20T21:17:51.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.450+0000 7f9c43200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9c4406ee20 con 0x7f9c44071d40 2024-04-20T21:17:51.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.450+0000 7f9c43200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9c44160370 con 0x7f9c44161c60 2024-04-20T21:17:51.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.451+0000 7f9c39400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c4415ff10 0x7f9c44158250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:51.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.451+0000 7f9c39400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c4415ff10 0x7f9c44158250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46670/0 (socket says 172.21.15.86:46670) 2024-04-20T21:17:51.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.451+0000 7f9c39400640 1 -- 172.21.15.86:0/1659893272 learned_addr learned my addr 172.21.15.86:0/1659893272 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:51.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.451+0000 7f9c41e00640 1 --2- 172.21.15.86:0/1659893272 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c44071d40 0x7f9c441f7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:51.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.451+0000 7f9c39400640 1 -- 172.21.15.86:0/1659893272 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9c44161c60 msgr2=0x7f9c440c53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:51.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.451+0000 7f9c42800640 1 --2- 172.21.15.86:0/1659893272 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9c44161c60 0x7f9c440c53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:51.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.451+0000 7f9c39400640 1 --2- 172.21.15.86:0/1659893272 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9c44161c60 0x7f9c440c53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:51.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.451+0000 7f9c39400640 1 -- 172.21.15.86:0/1659893272 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c44071d40 msgr2=0x7f9c441f7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:51.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.451+0000 7f9c39400640 1 --2- 172.21.15.86:0/1659893272 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c44071d40 0x7f9c441f7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:51.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.451+0000 7f9c39400640 1 -- 172.21.15.86:0/1659893272 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c441f6030 con 0x7f9c4415ff10 2024-04-20T21:17:51.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.451+0000 7f9c41e00640 1 --2- 172.21.15.86:0/1659893272 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c44071d40 0x7f9c441f7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:17:51.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.451+0000 7f9c39400640 1 --2- 172.21.15.86:0/1659893272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c4415ff10 0x7f9c44158250 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c000c00 tx=0x7f9c2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:51.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.451+0000 7f9c3aa00640 1 -- 172.21.15.86:0/1659893272 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c040070 con 0x7f9c4415ff10 2024-04-20T21:17:51.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.452+0000 7f9c3aa00640 1 -- 172.21.15.86:0/1659893272 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c2c03a420 con 0x7f9c4415ff10 2024-04-20T21:17:51.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.452+0000 7f9c43200640 1 -- 172.21.15.86:0/1659893272 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c441f62d0 con 0x7f9c4415ff10 2024-04-20T21:17:51.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.452+0000 7f9c3aa00640 1 -- 172.21.15.86:0/1659893272 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c035040 con 0x7f9c4415ff10 2024-04-20T21:17:51.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.452+0000 7f9c43200640 1 -- 172.21.15.86:0/1659893272 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c441f6890 con 0x7f9c4415ff10 2024-04-20T21:17:51.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.453+0000 7f9c43200640 1 -- 172.21.15.86:0/1659893272 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c4404fd90 con 0x7f9c4415ff10 2024-04-20T21:17:51.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.454+0000 7f9c3aa00640 1 -- 172.21.15.86:0/1659893272 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9c2c04b060 con 0x7f9c4415ff10 2024-04-20T21:17:51.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.455+0000 7f9c3aa00640 1 --2- 172.21.15.86:0/1659893272 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9c2006c8f0 0x7f9c2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:51.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.455+0000 7f9c3aa00640 1 -- 172.21.15.86:0/1659893272 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7f9c2c0adaf0 con 0x7f9c4415ff10 2024-04-20T21:17:51.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.455+0000 7f9c41e00640 1 --2- 172.21.15.86:0/1659893272 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9c2006c8f0 0x7f9c2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:51.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.456+0000 7f9c41e00640 1 --2- 172.21.15.86:0/1659893272 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9c2006c8f0 0x7f9c2006edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9c440b5a00 tx=0x7f9c34008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:51.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.460+0000 7f9c3aa00640 1 -- 172.21.15.86:0/1659893272 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9c2c07bf70 con 0x7f9c4415ff10 2024-04-20T21:17:51.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:51.697+0000 7f9c43200640 1 -- 172.21.15.86:0/1659893272 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f9c44072bb0 con 0x7f9c4415ff10 2024-04-20T21:17:52.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.273+0000 7f9c3aa00640 1 -- 172.21.15.86:0/1659893272 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f9c2c07b870 con 0x7f9c4415ff10 2024-04-20T21:17:52.274 INFO:teuthology.orchestra.run.smithi086.stderr:new fs with metadata pool 2 and data pool 3 2024-04-20T21:17:52.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.287+0000 7f9c43200640 1 -- 172.21.15.86:0/1659893272 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9c2006c8f0 msgr2=0x7f9c2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:52.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.287+0000 7f9c43200640 1 --2- 172.21.15.86:0/1659893272 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9c2006c8f0 0x7f9c2006edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9c440b5a00 tx=0x7f9c34008040 comp rx=0 tx=0).stop 2024-04-20T21:17:52.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.287+0000 7f9c43200640 1 -- 172.21.15.86:0/1659893272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c4415ff10 msgr2=0x7f9c44158250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:52.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.287+0000 7f9c43200640 1 --2- 172.21.15.86:0/1659893272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c4415ff10 0x7f9c44158250 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c000c00 tx=0x7f9c2c0046c0 comp rx=0 tx=0).stop 2024-04-20T21:17:52.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.288+0000 7f9c43200640 1 -- 172.21.15.86:0/1659893272 shutdown_connections 2024-04-20T21:17:52.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.288+0000 7f9c43200640 1 --2- 172.21.15.86:0/1659893272 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9c2006c8f0 0x7f9c2006edb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:52.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.288+0000 7f9c43200640 1 --2- 172.21.15.86:0/1659893272 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9c44161c60 0x7f9c440c53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:52.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.288+0000 7f9c43200640 1 --2- 172.21.15.86:0/1659893272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c4415ff10 0x7f9c44158250 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:52.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.288+0000 7f9c43200640 1 --2- 172.21.15.86:0/1659893272 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c44071d40 0x7f9c441f7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:52.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.288+0000 7f9c43200640 1 -- 172.21.15.86:0/1659893272 >> 172.21.15.86:0/1659893272 conn(0x7f9c44067cf0 msgr2=0x7f9c440b3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:52.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.289+0000 7f9c43200640 1 -- 172.21.15.86:0/1659893272 shutdown_connections 2024-04-20T21:17:52.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.289+0000 7f9c43200640 1 -- 172.21.15.86:0/1659893272 wait complete. 2024-04-20T21:17:52.324 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:17:52.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.487+0000 7faa9f400640 1 Processor -- start 2024-04-20T21:17:52.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.488+0000 7faa9f400640 1 -- start start 2024-04-20T21:17:52.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.489+0000 7faa9f400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7faa98072410 0x7faa9815cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:52.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.489+0000 7faa9f400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa9806eb30 con 0x7faa98146130 2024-04-20T21:17:52.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.489+0000 7faa9f400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa9806dc30 con 0x7faa9806e6d0 2024-04-20T21:17:52.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.489+0000 7faa9f400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7faa980cd8e0 con 0x7faa98072410 2024-04-20T21:17:52.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.489+0000 7faa9ca00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7faa98146130 0x7faa980ccdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39274/0 (socket says 172.21.15.86:39274) 2024-04-20T21:17:52.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.489+0000 7faa9ca00640 1 -- 172.21.15.86:0/4149051875 learned_addr learned my addr 172.21.15.86:0/4149051875 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:17:52.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.489+0000 7faa9d400640 1 --2- 172.21.15.86:0/4149051875 >> v2:172.21.15.144:3300/0 conn(0x7faa98072410 0x7faa9815cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:52.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.489+0000 7faa97e00640 1 -- 172.21.15.86:0/4149051875 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3896430344 0 0) 0x7faa80001110 con 0x7faa98146130 2024-04-20T21:17:52.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.489+0000 7faa97e00640 1 -- 172.21.15.86:0/4149051875 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa78003520 con 0x7faa98146130 2024-04-20T21:17:52.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.490+0000 7faa97e00640 1 -- 172.21.15.86:0/4149051875 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1587969387 0 0) 0x7faa78003520 con 0x7faa98146130 2024-04-20T21:17:52.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.490+0000 7faa97e00640 1 -- 172.21.15.86:0/4149051875 >> v2:172.21.15.144:3300/0 conn(0x7faa98072410 msgr2=0x7faa9815cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:52.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.490+0000 7faa97e00640 1 --2- 172.21.15.86:0/4149051875 >> v2:172.21.15.144:3300/0 conn(0x7faa98072410 0x7faa9815cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:52.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.490+0000 7faa97e00640 1 -- 172.21.15.86:0/4149051875 >> v1:172.21.15.144:6789/0 conn(0x7faa9806e6d0 legacy=0x7faa98071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:52.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.490+0000 7faa97e00640 1 -- 172.21.15.86:0/4149051875 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa980cdbc0 con 0x7faa98146130 2024-04-20T21:17:52.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.490+0000 7faa97e00640 1 -- 172.21.15.86:0/4149051875 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7faa80001840 con 0x7faa98146130 2024-04-20T21:17:52.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.490+0000 7faa97e00640 1 -- 172.21.15.86:0/4149051875 >> v1:172.21.15.86:6789/0 conn(0x7faa98146130 legacy=0x7faa980ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:52.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.490+0000 7faa97e00640 1 --2- 172.21.15.86:0/4149051875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa780039b0 0x7faa78003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:52.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.490+0000 7faa97e00640 1 --2- 172.21.15.86:0/4149051875 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa78004800 0x7faa78004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:52.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.490+0000 7faa97e00640 1 --2- 172.21.15.86:0/4149051875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa780051c0 0x7faa7800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:52.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.490+0000 7faa97e00640 1 -- 172.21.15.86:0/4149051875 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa980cd8e0 con 0x7faa780039b0 2024-04-20T21:17:52.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.490+0000 7faa97e00640 1 -- 172.21.15.86:0/4149051875 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7faa7800c070 con 0x7faa780051c0 2024-04-20T21:17:52.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.490+0000 7faa97e00640 1 -- 172.21.15.86:0/4149051875 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7faa7800c270 con 0x7faa78004800 2024-04-20T21:17:52.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.490+0000 7faa9ca00640 1 --2- 172.21.15.86:0/4149051875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa780039b0 0x7faa78003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:52.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.490+0000 7faa8fe00640 1 --2- 172.21.15.86:0/4149051875 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa78004800 0x7faa78004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:52.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.490+0000 7faa9d400640 1 --2- 172.21.15.86:0/4149051875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa780051c0 0x7faa7800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:52.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.491+0000 7faa9ca00640 1 -- 172.21.15.86:0/4149051875 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa78004800 msgr2=0x7faa78004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:52.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.491+0000 7faa9ca00640 1 --2- 172.21.15.86:0/4149051875 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa78004800 0x7faa78004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:52.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.491+0000 7faa9ca00640 1 -- 172.21.15.86:0/4149051875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa780051c0 msgr2=0x7faa7800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:52.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.491+0000 7faa9ca00640 1 --2- 172.21.15.86:0/4149051875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa780051c0 0x7faa7800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:52.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.491+0000 7faa9ca00640 1 -- 172.21.15.86:0/4149051875 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa78003520 con 0x7faa780039b0 2024-04-20T21:17:52.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.491+0000 7faa8fe00640 1 --2- 172.21.15.86:0/4149051875 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa78004800 0x7faa78004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:17:52.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.491+0000 7faa9d400640 1 --2- 172.21.15.86:0/4149051875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa780051c0 0x7faa7800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:17:52.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.491+0000 7faa9d400640 1 -- 172.21.15.86:0/4149051875 reap_dead start 2024-04-20T21:17:52.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.492+0000 7faa9ca00640 1 --2- 172.21.15.86:0/4149051875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa780039b0 0x7faa78003dd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7faa8000c4c0 tx=0x7faa78005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d45b05ef765cb00b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:52.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.492+0000 7faa97e00640 1 -- 172.21.15.86:0/4149051875 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8000c0e0 con 0x7faa780039b0 2024-04-20T21:17:52.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.492+0000 7faa97e00640 1 -- 172.21.15.86:0/4149051875 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa80030070 con 0x7faa780039b0 2024-04-20T21:17:52.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.492+0000 7faa97e00640 1 -- 172.21.15.86:0/4149051875 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa800058d0 con 0x7faa780039b0 2024-04-20T21:17:52.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.492+0000 7faa9f400640 1 -- 172.21.15.86:0/4149051875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa780039b0 msgr2=0x7faa78003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:52.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.492+0000 7faa9f400640 1 --2- 172.21.15.86:0/4149051875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa780039b0 0x7faa78003dd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7faa8000c4c0 tx=0x7faa78005830 comp rx=0 tx=0).stop 2024-04-20T21:17:52.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.492+0000 7faa9f400640 1 -- 172.21.15.86:0/4149051875 shutdown_connections 2024-04-20T21:17:52.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.492+0000 7faa9f400640 1 --2- 172.21.15.86:0/4149051875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa780039b0 0x7faa78003dd0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:52.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.492+0000 7faa9f400640 1 -- 172.21.15.86:0/4149051875 >> 172.21.15.86:0/4149051875 conn(0x7faa980b2360 msgr2=0x7faa980b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:52.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.493+0000 7faa9f400640 1 -- 172.21.15.86:0/4149051875 shutdown_connections 2024-04-20T21:17:52.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.493+0000 7faa9f400640 1 -- 172.21.15.86:0/4149051875 wait complete. 2024-04-20T21:17:52.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.494+0000 7faa9f400640 1 Processor -- start 2024-04-20T21:17:52.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.494+0000 7faa9f400640 1 -- start start 2024-04-20T21:17:52.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.495+0000 7faa9f400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa780039b0 0x7faa980af0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:52.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.495+0000 7faa9f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa9815b4b0 0x7faa9815b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:52.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.495+0000 7faa9f400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa98139970 0x7faa98139e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:52.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.495+0000 7faa9f400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa9806f150 con 0x7faa9815b4b0 2024-04-20T21:17:52.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.495+0000 7faa9f400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7faa9806e1b0 con 0x7faa98139970 2024-04-20T21:17:52.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.495+0000 7faa9f400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7faa98071d00 con 0x7faa780039b0 2024-04-20T21:17:52.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.496+0000 7faa8fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa9815b4b0 0x7faa9815b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:52.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.496+0000 7faa8fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa9815b4b0 0x7faa9815b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46682/0 (socket says 172.21.15.86:46682) 2024-04-20T21:17:52.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.496+0000 7faa8fe00640 1 -- 172.21.15.86:0/3321009943 learned_addr learned my addr 172.21.15.86:0/3321009943 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:52.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.496+0000 7faa9ca00640 1 --2- 172.21.15.86:0/3321009943 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa780039b0 0x7faa980af0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:52.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.496+0000 7faa9d400640 1 --2- 172.21.15.86:0/3321009943 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa98139970 0x7faa98139e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:52.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.496+0000 7faa8fe00640 1 -- 172.21.15.86:0/3321009943 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa780039b0 msgr2=0x7faa980af0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:52.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.496+0000 7faa8fe00640 1 --2- 172.21.15.86:0/3321009943 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa780039b0 0x7faa980af0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:52.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.496+0000 7faa8fe00640 1 -- 172.21.15.86:0/3321009943 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa98139970 msgr2=0x7faa98139e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:52.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.496+0000 7faa8fe00640 1 --2- 172.21.15.86:0/3321009943 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa98139970 0x7faa98139e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:52.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.496+0000 7faa8fe00640 1 -- 172.21.15.86:0/3321009943 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa9813c480 con 0x7faa9815b4b0 2024-04-20T21:17:52.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.496+0000 7faa8fe00640 1 --2- 172.21.15.86:0/3321009943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa9815b4b0 0x7faa9815b910 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faa8800e480 tx=0x7faa8800e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:52.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.496+0000 7faa95600640 1 -- 172.21.15.86:0/3321009943 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa880052e0 con 0x7faa9815b4b0 2024-04-20T21:17:52.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.496+0000 7faa95600640 1 -- 172.21.15.86:0/3321009943 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa88018070 con 0x7faa9815b4b0 2024-04-20T21:17:52.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.496+0000 7faa95600640 1 -- 172.21.15.86:0/3321009943 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa88003810 con 0x7faa9815b4b0 2024-04-20T21:17:52.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.496+0000 7faa9f400640 1 -- 172.21.15.86:0/3321009943 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa9813c760 con 0x7faa9815b4b0 2024-04-20T21:17:52.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.497+0000 7faa9f400640 1 -- 172.21.15.86:0/3321009943 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa9813cd20 con 0x7faa9815b4b0 2024-04-20T21:17:52.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.498+0000 7faa9f400640 1 -- 172.21.15.86:0/3321009943 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa980689e0 con 0x7faa9815b4b0 2024-04-20T21:17:52.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.499+0000 7faa95600640 1 -- 172.21.15.86:0/3321009943 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faa880039b0 con 0x7faa9815b4b0 2024-04-20T21:17:52.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.499+0000 7faa95600640 1 --2- 172.21.15.86:0/3321009943 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7faa7406c8f0 0x7faa7406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:52.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.500+0000 7faa95600640 1 -- 172.21.15.86:0/3321009943 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7faa8808b880 con 0x7faa9815b4b0 2024-04-20T21:17:52.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.500+0000 7faa9ca00640 1 --2- 172.21.15.86:0/3321009943 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7faa7406c8f0 0x7faa7406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:52.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.500+0000 7faa9ca00640 1 --2- 172.21.15.86:0/3321009943 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7faa7406c8f0 0x7faa7406edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7faa80034000 tx=0x7faa80009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:52.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.505+0000 7faa95600640 1 -- 172.21.15.86:0/3321009943 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7faa88059cb0 con 0x7faa9815b4b0 2024-04-20T21:17:52.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.706+0000 7faa9f400640 1 -- 172.21.15.86:0/3321009943 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faa9815ff50 con 0x7faa9815b4b0 2024-04-20T21:17:52.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.708+0000 7faa95600640 1 -- 172.21.15.86:0/3321009943 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16618 (secure 0 0 0) 0x7faa880595b0 con 0x7faa9815b4b0 2024-04-20T21:17:52.709 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:17:52.709 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":19,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:17:51.700447+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:17:49.718799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:17:50.683477+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:17:52.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.711+0000 7faa9f400640 1 -- 172.21.15.86:0/3321009943 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7faa7406c8f0 msgr2=0x7faa7406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:52.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.711+0000 7faa9f400640 1 --2- 172.21.15.86:0/3321009943 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7faa7406c8f0 0x7faa7406edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7faa80034000 tx=0x7faa80009030 comp rx=0 tx=0).stop 2024-04-20T21:17:52.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.711+0000 7faa9f400640 1 -- 172.21.15.86:0/3321009943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa9815b4b0 msgr2=0x7faa9815b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:52.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.711+0000 7faa9f400640 1 --2- 172.21.15.86:0/3321009943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa9815b4b0 0x7faa9815b910 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faa8800e480 tx=0x7faa8800e940 comp rx=0 tx=0).stop 2024-04-20T21:17:52.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.711+0000 7faa9f400640 1 -- 172.21.15.86:0/3321009943 shutdown_connections 2024-04-20T21:17:52.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.711+0000 7faa9f400640 1 --2- 172.21.15.86:0/3321009943 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7faa7406c8f0 0x7faa7406edb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:52.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.711+0000 7faa9f400640 1 --2- 172.21.15.86:0/3321009943 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa98139970 0x7faa98139e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:52.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.711+0000 7faa9f400640 1 --2- 172.21.15.86:0/3321009943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa9815b4b0 0x7faa9815b910 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:52.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.711+0000 7faa9f400640 1 --2- 172.21.15.86:0/3321009943 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa780039b0 0x7faa980af0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:52.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.711+0000 7faa9f400640 1 -- 172.21.15.86:0/3321009943 >> 172.21.15.86:0/3321009943 conn(0x7faa980b2360 msgr2=0x7faa980b17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:52.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.711+0000 7faa9f400640 1 -- 172.21.15.86:0/3321009943 shutdown_connections 2024-04-20T21:17:52.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.711+0000 7faa9f400640 1 -- 172.21.15.86:0/3321009943 wait complete. 2024-04-20T21:17:52.729 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:17:52.820 INFO:tasks.ceph.mds.d.smithi144.stderr:2024-04-20T21:17:52.822+0000 7f61d2e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T21:17:52.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.922+0000 7f5cdda00640 1 Processor -- start 2024-04-20T21:17:52.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.922+0000 7f5cdda00640 1 -- start start 2024-04-20T21:17:52.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.923+0000 7f5cdda00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5cd806fbd0 0x7f5cd815c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:52.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.923+0000 7f5cdda00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cd80ae710 con 0x7f5cd80af7b0 2024-04-20T21:17:52.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.923+0000 7f5cdda00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cd80ae9f0 con 0x7f5cd815c010 2024-04-20T21:17:52.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.923+0000 7f5cdda00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f5cd80af320 con 0x7f5cd806fbd0 2024-04-20T21:17:52.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.924+0000 7f5cdd000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5cd806fbd0 0x7f5cd815c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:52.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.924+0000 7f5cd7400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f5cd80af7b0 0x7f5cd806f4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39284/0 (socket says 172.21.15.86:39284) 2024-04-20T21:17:52.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.924+0000 7f5cd7400640 1 -- 172.21.15.86:0/666821958 learned_addr learned my addr 172.21.15.86:0/666821958 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:17:52.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.924+0000 7f5cdd000640 1 -- 172.21.15.86:0/666821958 >> v1:172.21.15.144:6789/0 conn(0x7f5cd815c010 legacy=0x7f5cd80adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:52.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.924+0000 7f5cdd000640 1 -- 172.21.15.86:0/666821958 >> v1:172.21.15.86:6789/0 conn(0x7f5cd80af7b0 legacy=0x7f5cd806f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:52.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.924+0000 7f5cdd000640 1 -- 172.21.15.86:0/666821958 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cd80aecd0 con 0x7f5cd806fbd0 2024-04-20T21:17:52.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.925+0000 7f5cdd000640 1 --2- 172.21.15.86:0/666821958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5cd806fbd0 0x7f5cd815c3f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0009d90 tx=0x7f5cd002e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e3e82751ad4ebad server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:52.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.925+0000 7f5cd6a00640 1 -- 172.21.15.86:0/666821958 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd003b070 con 0x7f5cd806fbd0 2024-04-20T21:17:52.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.925+0000 7f5cd6a00640 1 -- 172.21.15.86:0/666821958 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cd0036440 con 0x7f5cd806fbd0 2024-04-20T21:17:52.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.925+0000 7f5cd6a00640 1 -- 172.21.15.86:0/666821958 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd0036810 con 0x7f5cd806fbd0 2024-04-20T21:17:52.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.926+0000 7f5cdda00640 1 -- 172.21.15.86:0/666821958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5cd806fbd0 msgr2=0x7f5cd815c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:52.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.926+0000 7f5cdda00640 1 --2- 172.21.15.86:0/666821958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5cd806fbd0 0x7f5cd815c3f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0009d90 tx=0x7f5cd002e0c0 comp rx=0 tx=0).stop 2024-04-20T21:17:52.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.926+0000 7f5cdda00640 1 -- 172.21.15.86:0/666821958 shutdown_connections 2024-04-20T21:17:52.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.926+0000 7f5cdda00640 1 --2- 172.21.15.86:0/666821958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5cd806fbd0 0x7f5cd815c3f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:52.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.926+0000 7f5cdda00640 1 -- 172.21.15.86:0/666821958 >> 172.21.15.86:0/666821958 conn(0x7f5cd80d0170 msgr2=0x7f5cd80d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:52.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.926+0000 7f5cdda00640 1 -- 172.21.15.86:0/666821958 shutdown_connections 2024-04-20T21:17:52.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.926+0000 7f5cdda00640 1 -- 172.21.15.86:0/666821958 wait complete. 2024-04-20T21:17:52.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.928+0000 7f5cdda00640 1 Processor -- start 2024-04-20T21:17:52.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.928+0000 7f5cdda00640 1 -- start start 2024-04-20T21:17:52.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.929+0000 7f5cdda00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5cd806fbd0 0x7f5cd809db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:52.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.929+0000 7f5cdda00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5cd80af7b0 0x7f5cd81e5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:52.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.929+0000 7f5cdda00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5cd815c010 0x7f5cd81452b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:52.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.929+0000 7f5cdda00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd815c930 con 0x7f5cd80af7b0 2024-04-20T21:17:52.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.929+0000 7f5cdda00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd80ae4b0 con 0x7f5cd815c010 2024-04-20T21:17:52.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.929+0000 7f5cdda00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5cd806e3b0 con 0x7f5cd806fbd0 2024-04-20T21:17:52.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.929+0000 7f5cd7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5cd80af7b0 0x7f5cd81e5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:52.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.929+0000 7f5cd7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5cd80af7b0 0x7f5cd81e5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46702/0 (socket says 172.21.15.86:46702) 2024-04-20T21:17:52.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.929+0000 7f5cd7400640 1 -- 172.21.15.86:0/3480781123 learned_addr learned my addr 172.21.15.86:0/3480781123 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:52.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.929+0000 7f5cdd000640 1 --2- 172.21.15.86:0/3480781123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5cd815c010 0x7f5cd81452b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:52.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.929+0000 7f5cd7e00640 1 --2- 172.21.15.86:0/3480781123 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5cd806fbd0 0x7f5cd809db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:52.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.929+0000 7f5cd7400640 1 -- 172.21.15.86:0/3480781123 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5cd806fbd0 msgr2=0x7f5cd809db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:52.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.930+0000 7f5cd7400640 1 --2- 172.21.15.86:0/3480781123 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5cd806fbd0 0x7f5cd809db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:52.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.930+0000 7f5cd7400640 1 -- 172.21.15.86:0/3480781123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5cd815c010 msgr2=0x7f5cd81452b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:52.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.930+0000 7f5cd7400640 1 --2- 172.21.15.86:0/3480781123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5cd815c010 0x7f5cd81452b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:52.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.930+0000 7f5cd7400640 1 -- 172.21.15.86:0/3480781123 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cd8148340 con 0x7f5cd80af7b0 2024-04-20T21:17:52.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.930+0000 7f5cdd000640 1 --2- 172.21.15.86:0/3480781123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5cd815c010 0x7f5cd81452b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:17:52.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.930+0000 7f5cd7400640 1 --2- 172.21.15.86:0/3480781123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5cd80af7b0 0x7f5cd81e5fe0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5cc400c400 tx=0x7f5cc400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:52.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.930+0000 7f5ccfe00640 1 -- 172.21.15.86:0/3480781123 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4017070 con 0x7f5cd80af7b0 2024-04-20T21:17:52.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.930+0000 7f5ccfe00640 1 -- 172.21.15.86:0/3480781123 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cc4005ce0 con 0x7f5cd80af7b0 2024-04-20T21:17:52.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.930+0000 7f5ccfe00640 1 -- 172.21.15.86:0/3480781123 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc40072f0 con 0x7f5cd80af7b0 2024-04-20T21:17:52.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.930+0000 7f5cdda00640 1 -- 172.21.15.86:0/3480781123 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cd8148620 con 0x7f5cd80af7b0 2024-04-20T21:17:52.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.930+0000 7f5cdda00640 1 -- 172.21.15.86:0/3480781123 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cd8148be0 con 0x7f5cd80af7b0 2024-04-20T21:17:52.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.931+0000 7f5cdda00640 1 -- 172.21.15.86:0/3480781123 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cd804fcf0 con 0x7f5cd80af7b0 2024-04-20T21:17:52.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.932+0000 7f5ccfe00640 1 -- 172.21.15.86:0/3480781123 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5cc401a070 con 0x7f5cd80af7b0 2024-04-20T21:17:52.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.934+0000 7f5ccfe00640 1 --2- 172.21.15.86:0/3480781123 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5cb806c8f0 0x7f5cb806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:52.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.934+0000 7f5ccfe00640 1 -- 172.21.15.86:0/3480781123 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f5cc408afe0 con 0x7f5cd80af7b0 2024-04-20T21:17:52.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.934+0000 7f5cd7e00640 1 --2- 172.21.15.86:0/3480781123 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5cb806c8f0 0x7f5cb806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:52.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.934+0000 7f5cd7e00640 1 --2- 172.21.15.86:0/3480781123 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5cb806c8f0 0x7f5cb806edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5cc800a750 tx=0x7f5cc8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:52.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:52.938+0000 7f5ccfe00640 1 -- 172.21.15.86:0/3480781123 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5cc4059390 con 0x7f5cd80af7b0 2024-04-20T21:17:53.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.147+0000 7f5cdda00640 1 -- 172.21.15.86:0/3480781123 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5cd8149080 con 0x7f5cd80af7b0 2024-04-20T21:17:53.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.150+0000 7f5ccfe00640 1 -- 172.21.15.86:0/3480781123 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16618 (secure 0 0 0) 0x7f5cc4058c90 con 0x7f5cd80af7b0 2024-04-20T21:17:53.151 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:17:53.151 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":19,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:17:51.700447+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:17:49.718799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:17:50.683477+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:17:53.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.152+0000 7f5cdda00640 1 -- 172.21.15.86:0/3480781123 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5cb806c8f0 msgr2=0x7f5cb806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:53.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.152+0000 7f5cdda00640 1 --2- 172.21.15.86:0/3480781123 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5cb806c8f0 0x7f5cb806edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5cc800a750 tx=0x7f5cc8008040 comp rx=0 tx=0).stop 2024-04-20T21:17:53.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.152+0000 7f5cdda00640 1 -- 172.21.15.86:0/3480781123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5cd80af7b0 msgr2=0x7f5cd81e5fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:53.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.152+0000 7f5cdda00640 1 --2- 172.21.15.86:0/3480781123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5cd80af7b0 0x7f5cd81e5fe0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5cc400c400 tx=0x7f5cc400c8c0 comp rx=0 tx=0).stop 2024-04-20T21:17:53.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.153+0000 7f5cdda00640 1 -- 172.21.15.86:0/3480781123 shutdown_connections 2024-04-20T21:17:53.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.153+0000 7f5cdda00640 1 --2- 172.21.15.86:0/3480781123 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5cb806c8f0 0x7f5cb806edb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:53.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.153+0000 7f5cdda00640 1 --2- 172.21.15.86:0/3480781123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5cd815c010 0x7f5cd81452b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:53.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.153+0000 7f5cdda00640 1 --2- 172.21.15.86:0/3480781123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5cd80af7b0 0x7f5cd81e5fe0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:53.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.153+0000 7f5cdda00640 1 --2- 172.21.15.86:0/3480781123 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5cd806fbd0 0x7f5cd809db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:53.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.153+0000 7f5cdda00640 1 -- 172.21.15.86:0/3480781123 >> 172.21.15.86:0/3480781123 conn(0x7f5cd80d0170 msgr2=0x7f5cd815e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:53.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.153+0000 7f5cdda00640 1 -- 172.21.15.86:0/3480781123 shutdown_connections 2024-04-20T21:17:53.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.153+0000 7f5cdda00640 1 -- 172.21.15.86:0/3480781123 wait complete. 2024-04-20T21:17:53.171 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-20T21:17:53.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.376+0000 7fd9abc00640 1 Processor -- start 2024-04-20T21:17:53.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.376+0000 7fd9abc00640 1 -- start start 2024-04-20T21:17:53.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.378+0000 7fd9abc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9a40cdc50 0x7fd9a406e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:53.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.378+0000 7fd9abc00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9a415cd40 con 0x7fd9a4146210 2024-04-20T21:17:53.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.378+0000 7fd9abc00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9a415d000 con 0x7fd9a406bbf0 2024-04-20T21:17:53.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.378+0000 7fd9abc00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd9a415d2e0 con 0x7fd9a40cdc50 2024-04-20T21:17:53.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.378+0000 7fd99be00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd9a4146210 0x7fd9a40cd540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39290/0 (socket says 172.21.15.86:39290) 2024-04-20T21:17:53.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.378+0000 7fd99be00640 1 -- 172.21.15.86:0/824215527 learned_addr learned my addr 172.21.15.86:0/824215527 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:17:53.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.379+0000 7fd9a9c00640 1 --2- 172.21.15.86:0/824215527 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9a40cdc50 0x7fd9a406e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:53.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.379+0000 7fd9a3e00640 1 -- 172.21.15.86:0/824215527 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 937568992 0 0) 0x7fd9a415cd40 con 0x7fd9a4146210 2024-04-20T21:17:53.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.379+0000 7fd9a3e00640 1 -- 172.21.15.86:0/824215527 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd988003520 con 0x7fd9a4146210 2024-04-20T21:17:53.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.379+0000 7fd9a3e00640 1 -- 172.21.15.86:0/824215527 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2478462419 0 0) 0x7fd988003520 con 0x7fd9a4146210 2024-04-20T21:17:53.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.379+0000 7fd9a3e00640 1 -- 172.21.15.86:0/824215527 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9a40cdc50 msgr2=0x7fd9a406e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:53.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.379+0000 7fd9a3e00640 1 --2- 172.21.15.86:0/824215527 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9a40cdc50 0x7fd9a406e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:53.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.379+0000 7fd9a3e00640 1 -- 172.21.15.86:0/824215527 >> v1:172.21.15.144:6789/0 conn(0x7fd9a406bbf0 legacy=0x7fd9a4145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:53.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.379+0000 7fd9a3e00640 1 -- 172.21.15.86:0/824215527 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a415d5c0 con 0x7fd9a4146210 2024-04-20T21:17:53.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.379+0000 7fd9a3e00640 1 -- 172.21.15.86:0/824215527 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fd98c001690 con 0x7fd9a4146210 2024-04-20T21:17:53.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.379+0000 7fd9a3e00640 1 -- 172.21.15.86:0/824215527 >> v1:172.21.15.86:6789/0 conn(0x7fd9a4146210 legacy=0x7fd9a40cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:53.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.379+0000 7fd9a3e00640 1 --2- 172.21.15.86:0/824215527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9880039e0 0x7fd988003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:53.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.379+0000 7fd9a3e00640 1 --2- 172.21.15.86:0/824215527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd988004800 0x7fd988004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:53.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.379+0000 7fd9a3e00640 1 --2- 172.21.15.86:0/824215527 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9a40cdc50 0x7fd9880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:53.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.379+0000 7fd9a3e00640 1 -- 172.21.15.86:0/824215527 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a415d2e0 con 0x7fd9880039e0 2024-04-20T21:17:53.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.379+0000 7fd9a3e00640 1 -- 172.21.15.86:0/824215527 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd98800baa0 con 0x7fd988004800 2024-04-20T21:17:53.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.379+0000 7fd9a3e00640 1 -- 172.21.15.86:0/824215527 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd98800bca0 con 0x7fd9a40cdc50 2024-04-20T21:17:53.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.380+0000 7fd9a9200640 1 --2- 172.21.15.86:0/824215527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9880039e0 0x7fd988003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:53.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.380+0000 7fd9a9c00640 1 --2- 172.21.15.86:0/824215527 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9a40cdc50 0x7fd9880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:53.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.380+0000 7fd99be00640 1 --2- 172.21.15.86:0/824215527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd988004800 0x7fd988004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:53.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.380+0000 7fd9a9200640 1 -- 172.21.15.86:0/824215527 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9a40cdc50 msgr2=0x7fd9880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:53.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.380+0000 7fd9a9200640 1 --2- 172.21.15.86:0/824215527 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9a40cdc50 0x7fd9880051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:53.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.380+0000 7fd9a9200640 1 -- 172.21.15.86:0/824215527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd988004800 msgr2=0x7fd988004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:53.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.380+0000 7fd9a9200640 1 --2- 172.21.15.86:0/824215527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd988004800 0x7fd988004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:53.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.380+0000 7fd9a9200640 1 -- 172.21.15.86:0/824215527 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd988003520 con 0x7fd9880039e0 2024-04-20T21:17:53.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.380+0000 7fd9a9c00640 1 --2- 172.21.15.86:0/824215527 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9a40cdc50 0x7fd9880051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:17:53.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.380+0000 7fd99be00640 1 --2- 172.21.15.86:0/824215527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd988004800 0x7fd988004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:17:53.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.381+0000 7fd9a9200640 1 --2- 172.21.15.86:0/824215527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9880039e0 0x7fd988003e00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd994001720 tx=0x7fd988005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=461d9108e5c49479 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:53.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.381+0000 7fd9a3e00640 1 -- 172.21.15.86:0/824215527 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd994001370 con 0x7fd9880039e0 2024-04-20T21:17:53.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.381+0000 7fd9a3e00640 1 -- 172.21.15.86:0/824215527 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9940052d0 con 0x7fd9880039e0 2024-04-20T21:17:53.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.381+0000 7fd9a3e00640 1 -- 172.21.15.86:0/824215527 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd994003830 con 0x7fd9880039e0 2024-04-20T21:17:53.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.382+0000 7fd9abc00640 1 -- 172.21.15.86:0/824215527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9880039e0 msgr2=0x7fd988003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:53.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.382+0000 7fd9abc00640 1 --2- 172.21.15.86:0/824215527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9880039e0 0x7fd988003e00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd994001720 tx=0x7fd988005830 comp rx=0 tx=0).stop 2024-04-20T21:17:53.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.382+0000 7fd9a9c00640 1 -- 172.21.15.86:0/824215527 reap_dead start 2024-04-20T21:17:53.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.382+0000 7fd9abc00640 1 -- 172.21.15.86:0/824215527 shutdown_connections 2024-04-20T21:17:53.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.382+0000 7fd9abc00640 1 -- 172.21.15.86:0/824215527 >> 172.21.15.86:0/824215527 conn(0x7fd9a40b2110 msgr2=0x7fd9a40b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:53.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.382+0000 7fd9abc00640 1 -- 172.21.15.86:0/824215527 shutdown_connections 2024-04-20T21:17:53.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.382+0000 7fd9abc00640 1 -- 172.21.15.86:0/824215527 wait complete. 2024-04-20T21:17:53.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.384+0000 7fd9abc00640 1 Processor -- start 2024-04-20T21:17:53.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.384+0000 7fd9abc00640 1 -- start start 2024-04-20T21:17:53.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.385+0000 7fd9abc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9880039e0 0x7fd9a415c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:53.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.385+0000 7fd9abc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9a41324f0 0x7fd9a40ae020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:53.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.385+0000 7fd9abc00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9a4132970 0x7fd9a40ae560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:53.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.385+0000 7fd9abc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a406c150 con 0x7fd9a41324f0 2024-04-20T21:17:53.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.385+0000 7fd9abc00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a406bfd0 con 0x7fd9a4132970 2024-04-20T21:17:53.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.385+0000 7fd9abc00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd9a406c2d0 con 0x7fd9880039e0 2024-04-20T21:17:53.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.385+0000 7fd99be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9a41324f0 0x7fd9a40ae020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:53.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.385+0000 7fd99be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9a41324f0 0x7fd9a40ae020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46732/0 (socket says 172.21.15.86:46732) 2024-04-20T21:17:53.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.385+0000 7fd99be00640 1 -- 172.21.15.86:0/1176277414 learned_addr learned my addr 172.21.15.86:0/1176277414 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:53.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.385+0000 7fd9a9c00640 1 --2- 172.21.15.86:0/1176277414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9a4132970 0x7fd9a40ae560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:53.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.385+0000 7fd9a9200640 1 --2- 172.21.15.86:0/1176277414 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9880039e0 0x7fd9a415c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:53.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.386+0000 7fd99be00640 1 -- 172.21.15.86:0/1176277414 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9880039e0 msgr2=0x7fd9a415c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:53.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.386+0000 7fd99be00640 1 --2- 172.21.15.86:0/1176277414 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9880039e0 0x7fd9a415c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:53.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.386+0000 7fd99be00640 1 -- 172.21.15.86:0/1176277414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9a4132970 msgr2=0x7fd9a40ae560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:53.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.386+0000 7fd99be00640 1 --2- 172.21.15.86:0/1176277414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9a4132970 0x7fd9a40ae560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:53.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.386+0000 7fd99be00640 1 -- 172.21.15.86:0/1176277414 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a4132f40 con 0x7fd9a41324f0 2024-04-20T21:17:53.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.386+0000 7fd9a9200640 1 --2- 172.21.15.86:0/1176277414 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9880039e0 0x7fd9a415c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:17:53.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.386+0000 7fd9a9c00640 1 --2- 172.21.15.86:0/1176277414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9a4132970 0x7fd9a40ae560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:17:53.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.386+0000 7fd99be00640 1 --2- 172.21.15.86:0/1176277414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9a41324f0 0x7fd9a40ae020 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd98c00d770 tx=0x7fd98c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:53.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.386+0000 7fd9a1600640 1 -- 172.21.15.86:0/1176277414 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c019070 con 0x7fd9a41324f0 2024-04-20T21:17:53.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.386+0000 7fd9a1600640 1 -- 172.21.15.86:0/1176277414 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd98c0074e0 con 0x7fd9a41324f0 2024-04-20T21:17:53.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.386+0000 7fd9abc00640 1 -- 172.21.15.86:0/1176277414 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9a4133220 con 0x7fd9a41324f0 2024-04-20T21:17:53.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.386+0000 7fd9a1600640 1 -- 172.21.15.86:0/1176277414 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c0049e0 con 0x7fd9a41324f0 2024-04-20T21:17:53.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.386+0000 7fd9abc00640 1 -- 172.21.15.86:0/1176277414 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9a41e5350 con 0x7fd9a41324f0 2024-04-20T21:17:53.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.387+0000 7fd9abc00640 1 -- 172.21.15.86:0/1176277414 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9a40689e0 con 0x7fd9a41324f0 2024-04-20T21:17:53.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.388+0000 7fd9a1600640 1 -- 172.21.15.86:0/1176277414 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd98c004b80 con 0x7fd9a41324f0 2024-04-20T21:17:53.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.389+0000 7fd9a1600640 1 --2- 172.21.15.86:0/1176277414 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd98006c920 0x7fd98006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:53.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.389+0000 7fd9a1600640 1 -- 172.21.15.86:0/1176277414 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd98c08b4e0 con 0x7fd9a41324f0 2024-04-20T21:17:53.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.390+0000 7fd9a9200640 1 --2- 172.21.15.86:0/1176277414 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd98006c920 0x7fd98006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:53.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.390+0000 7fd9a9200640 1 --2- 172.21.15.86:0/1176277414 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd98006c920 0x7fd98006ede0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd994033000 tx=0x7fd994009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:53.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.393+0000 7fd9a1600640 1 -- 172.21.15.86:0/1176277414 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd98c059910 con 0x7fd9a41324f0 2024-04-20T21:17:53.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:53.632+0000 7fd9abc00640 1 -- 172.21.15.86:0/1176277414 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fd9a41333f0 con 0x7fd9a41324f0 2024-04-20T21:17:54.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.485+0000 7fd9a1600640 1 -- 172.21.15.86:0/1176277414 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) ==== 118+0+0 (secure 0 0 0) 0x7fd98c059210 con 0x7fd9a41324f0 2024-04-20T21:17:54.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.489+0000 7fd9abc00640 1 -- 172.21.15.86:0/1176277414 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd98006c920 msgr2=0x7fd98006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:54.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.489+0000 7fd9abc00640 1 --2- 172.21.15.86:0/1176277414 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd98006c920 0x7fd98006ede0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd994033000 tx=0x7fd994009030 comp rx=0 tx=0).stop 2024-04-20T21:17:54.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.489+0000 7fd9abc00640 1 -- 172.21.15.86:0/1176277414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9a41324f0 msgr2=0x7fd9a40ae020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:54.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.489+0000 7fd9abc00640 1 --2- 172.21.15.86:0/1176277414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9a41324f0 0x7fd9a40ae020 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd98c00d770 tx=0x7fd98c00dc30 comp rx=0 tx=0).stop 2024-04-20T21:17:54.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.489+0000 7fd9abc00640 1 -- 172.21.15.86:0/1176277414 shutdown_connections 2024-04-20T21:17:54.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.489+0000 7fd9abc00640 1 --2- 172.21.15.86:0/1176277414 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd98006c920 0x7fd98006ede0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:54.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.489+0000 7fd9abc00640 1 --2- 172.21.15.86:0/1176277414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9a4132970 0x7fd9a40ae560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:54.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.489+0000 7fd9abc00640 1 --2- 172.21.15.86:0/1176277414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9a41324f0 0x7fd9a40ae020 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:54.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.489+0000 7fd9abc00640 1 --2- 172.21.15.86:0/1176277414 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd9880039e0 0x7fd9a415c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:54.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.489+0000 7fd9abc00640 1 -- 172.21.15.86:0/1176277414 >> 172.21.15.86:0/1176277414 conn(0x7fd9a40b2110 msgr2=0x7fd9a40affd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:54.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.489+0000 7fd9abc00640 1 -- 172.21.15.86:0/1176277414 shutdown_connections 2024-04-20T21:17:54.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.489+0000 7fd9abc00640 1 -- 172.21.15.86:0/1176277414 wait complete. 2024-04-20T21:17:54.512 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-20T21:17:54.512 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-20T21:17:54.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.716+0000 7fad89600640 1 Processor -- start 2024-04-20T21:17:54.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.717+0000 7fad89600640 1 -- start start 2024-04-20T21:17:54.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.718+0000 7fad89600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fad840cfc70 0x7fad840cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:54.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.718+0000 7fad89600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad840cdbe0 0x7fad8406eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:54.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.718+0000 7fad89600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad8415bcd0 con 0x7fad840b3ba0 2024-04-20T21:17:54.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.718+0000 7fad89600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fad8415c8e0 con 0x7fad840cfc70 2024-04-20T21:17:54.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.718+0000 7fad89600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fad8415bfb0 con 0x7fad840cdbe0 2024-04-20T21:17:54.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.718+0000 7fad83e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fad840cfc70 0x7fad840cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:54.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.718+0000 7fad83e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fad840cfc70 0x7fad840cd6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46738/0 (socket says 172.21.15.86:46738) 2024-04-20T21:17:54.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.718+0000 7fad83e00640 1 -- 172.21.15.86:0/3824255110 learned_addr learned my addr 172.21.15.86:0/3824255110 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:54.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.719+0000 7fad83400640 1 --2- 172.21.15.86:0/3824255110 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad840cdbe0 0x7fad8406eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:54.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.719+0000 7fad83e00640 1 -- 172.21.15.86:0/3824255110 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad840cdbe0 msgr2=0x7fad8406eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:54.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.719+0000 7fad83e00640 1 --2- 172.21.15.86:0/3824255110 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad840cdbe0 0x7fad8406eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:54.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.719+0000 7fad83e00640 1 -- 172.21.15.86:0/3824255110 >> v1:172.21.15.144:6789/0 conn(0x7fad840b3ba0 legacy=0x7fad84071ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:54.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.719+0000 7fad83e00640 1 -- 172.21.15.86:0/3824255110 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad8415c290 con 0x7fad840cfc70 2024-04-20T21:17:54.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.719+0000 7fad83400640 1 --2- 172.21.15.86:0/3824255110 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad840cdbe0 0x7fad8406eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:17:54.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.720+0000 7fad83e00640 1 --2- 172.21.15.86:0/3824255110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad840cfc70 0x7fad840cd6a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fad74009870 tx=0x7fad7402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abe843990bbd92c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:54.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.720+0000 7fad82a00640 1 -- 172.21.15.86:0/3824255110 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad74035020 con 0x7fad840cfc70 2024-04-20T21:17:54.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.720+0000 7fad82a00640 1 -- 172.21.15.86:0/3824255110 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad74031640 con 0x7fad840cfc70 2024-04-20T21:17:54.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.720+0000 7fad82a00640 1 -- 172.21.15.86:0/3824255110 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7402d040 con 0x7fad840cfc70 2024-04-20T21:17:54.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.721+0000 7fad89600640 1 -- 172.21.15.86:0/3824255110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad840cfc70 msgr2=0x7fad840cd6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:54.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.721+0000 7fad89600640 1 --2- 172.21.15.86:0/3824255110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad840cfc70 0x7fad840cd6a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fad74009870 tx=0x7fad7402b8f0 comp rx=0 tx=0).stop 2024-04-20T21:17:54.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.721+0000 7fad89600640 1 -- 172.21.15.86:0/3824255110 shutdown_connections 2024-04-20T21:17:54.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.721+0000 7fad89600640 1 --2- 172.21.15.86:0/3824255110 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad840cdbe0 0x7fad8406eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:54.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.721+0000 7fad89600640 1 --2- 172.21.15.86:0/3824255110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad840cfc70 0x7fad840cd6a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:54.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.721+0000 7fad89600640 1 -- 172.21.15.86:0/3824255110 >> 172.21.15.86:0/3824255110 conn(0x7fad84067cf0 msgr2=0x7fad8406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:54.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.721+0000 7fad89600640 1 -- 172.21.15.86:0/3824255110 shutdown_connections 2024-04-20T21:17:54.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.721+0000 7fad89600640 1 -- 172.21.15.86:0/3824255110 wait complete. 2024-04-20T21:17:54.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.723+0000 7fad89600640 1 Processor -- start 2024-04-20T21:17:54.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.723+0000 7fad89600640 1 -- start start 2024-04-20T21:17:54.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.724+0000 7fad89600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad840b3ba0 0x7fad840c5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:54.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.724+0000 7fad89600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad840cdbe0 0x7fad8414c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:54.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.724+0000 7fad89600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad840cfc70 0x7fad8415a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:54.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.724+0000 7fad89600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fad840d0050 con 0x7fad840cfc70 2024-04-20T21:17:54.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.724+0000 7fad89600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fad840b3710 con 0x7fad840cdbe0 2024-04-20T21:17:54.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.724+0000 7fad89600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fad840d01d0 con 0x7fad840b3ba0 2024-04-20T21:17:54.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.724+0000 7fad88c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad840cfc70 0x7fad8415a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:54.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.724+0000 7fad88c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad840cfc70 0x7fad8415a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46744/0 (socket says 172.21.15.86:46744) 2024-04-20T21:17:54.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.724+0000 7fad88c00640 1 -- 172.21.15.86:0/3275193780 learned_addr learned my addr 172.21.15.86:0/3275193780 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:54.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.724+0000 7fad83400640 1 --2- 172.21.15.86:0/3275193780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad840cdbe0 0x7fad8414c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:54.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.725+0000 7fad83e00640 1 --2- 172.21.15.86:0/3275193780 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad840b3ba0 0x7fad840c5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:54.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.725+0000 7fad88c00640 1 -- 172.21.15.86:0/3275193780 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad840b3ba0 msgr2=0x7fad840c5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:54.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.725+0000 7fad88c00640 1 --2- 172.21.15.86:0/3275193780 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad840b3ba0 0x7fad840c5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:54.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.725+0000 7fad88c00640 1 -- 172.21.15.86:0/3275193780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad840cdbe0 msgr2=0x7fad8414c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:54.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.725+0000 7fad88c00640 1 --2- 172.21.15.86:0/3275193780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad840cdbe0 0x7fad8414c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:54.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.725+0000 7fad88c00640 1 -- 172.21.15.86:0/3275193780 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad840c6610 con 0x7fad840cfc70 2024-04-20T21:17:54.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.725+0000 7fad83e00640 1 --2- 172.21.15.86:0/3275193780 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad840b3ba0 0x7fad840c5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:17:54.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.725+0000 7fad83400640 1 --2- 172.21.15.86:0/3275193780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad840cdbe0 0x7fad8414c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:17:54.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.725+0000 7fad88c00640 1 --2- 172.21.15.86:0/3275193780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad840cfc70 0x7fad8415a500 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fad7c00b970 tx=0x7fad7c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:54.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.725+0000 7fad7be00640 1 -- 172.21.15.86:0/3275193780 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7c019070 con 0x7fad840cfc70 2024-04-20T21:17:54.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.725+0000 7fad7be00640 1 -- 172.21.15.86:0/3275193780 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad7c00cb40 con 0x7fad840cfc70 2024-04-20T21:17:54.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.725+0000 7fad89600640 1 -- 172.21.15.86:0/3275193780 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad840c68f0 con 0x7fad840cfc70 2024-04-20T21:17:54.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.725+0000 7fad7be00640 1 -- 172.21.15.86:0/3275193780 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7c014440 con 0x7fad840cfc70 2024-04-20T21:17:54.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.725+0000 7fad89600640 1 -- 172.21.15.86:0/3275193780 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad840c6320 con 0x7fad840cfc70 2024-04-20T21:17:54.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.727+0000 7fad89600640 1 -- 172.21.15.86:0/3275193780 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad8404fd90 con 0x7fad840cfc70 2024-04-20T21:17:54.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.728+0000 7fad7be00640 1 -- 172.21.15.86:0/3275193780 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fad7c01c070 con 0x7fad840cfc70 2024-04-20T21:17:54.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.729+0000 7fad7be00640 1 --2- 172.21.15.86:0/3275193780 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fad6406c8f0 0x7fad6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:54.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.729+0000 7fad7be00640 1 -- 172.21.15.86:0/3275193780 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fad7c08b6a0 con 0x7fad840cfc70 2024-04-20T21:17:54.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.729+0000 7fad83e00640 1 --2- 172.21.15.86:0/3275193780 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fad6406c8f0 0x7fad6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:54.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.730+0000 7fad83e00640 1 --2- 172.21.15.86:0/3275193780 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fad6406c8f0 0x7fad6406edb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fad74036000 tx=0x7fad74008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:54.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.733+0000 7fad7be00640 1 -- 172.21.15.86:0/3275193780 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fad7c059b60 con 0x7fad840cfc70 2024-04-20T21:17:54.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:54.971+0000 7fad89600640 1 -- 172.21.15.86:0/3275193780 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fad8414f3a0 con 0x7fad840cfc70 2024-04-20T21:17:55.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.498+0000 7fad7be00640 1 -- 172.21.15.86:0/3275193780 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) ==== 122+0+0 (secure 0 0 0) 0x7fad7c059460 con 0x7fad840cfc70 2024-04-20T21:17:55.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.501+0000 7fad89600640 1 -- 172.21.15.86:0/3275193780 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fad6406c8f0 msgr2=0x7fad6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:55.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.501+0000 7fad89600640 1 --2- 172.21.15.86:0/3275193780 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fad6406c8f0 0x7fad6406edb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fad74036000 tx=0x7fad74008030 comp rx=0 tx=0).stop 2024-04-20T21:17:55.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.502+0000 7fad89600640 1 -- 172.21.15.86:0/3275193780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad840cfc70 msgr2=0x7fad8415a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:55.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.502+0000 7fad89600640 1 --2- 172.21.15.86:0/3275193780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad840cfc70 0x7fad8415a500 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fad7c00b970 tx=0x7fad7c00be30 comp rx=0 tx=0).stop 2024-04-20T21:17:55.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.502+0000 7fad89600640 1 -- 172.21.15.86:0/3275193780 shutdown_connections 2024-04-20T21:17:55.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.502+0000 7fad89600640 1 --2- 172.21.15.86:0/3275193780 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fad6406c8f0 0x7fad6406edb0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:55.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.502+0000 7fad89600640 1 --2- 172.21.15.86:0/3275193780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad840cfc70 0x7fad8415a500 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:55.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.502+0000 7fad89600640 1 --2- 172.21.15.86:0/3275193780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad840cdbe0 0x7fad8414c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:55.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.502+0000 7fad89600640 1 --2- 172.21.15.86:0/3275193780 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad840b3ba0 0x7fad840c5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:55.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.502+0000 7fad89600640 1 -- 172.21.15.86:0/3275193780 >> 172.21.15.86:0/3275193780 conn(0x7fad84067cf0 msgr2=0x7fad840b4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:55.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.502+0000 7fad89600640 1 -- 172.21.15.86:0/3275193780 shutdown_connections 2024-04-20T21:17:55.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.502+0000 7fad89600640 1 -- 172.21.15.86:0/3275193780 wait complete. 2024-04-20T21:17:55.524 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-20T21:17:55.524 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:17:55.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.725+0000 7f74c2c00640 1 Processor -- start 2024-04-20T21:17:55.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.725+0000 7f74c2c00640 1 -- start start 2024-04-20T21:17:55.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.727+0000 7f74c2c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74bc144e90 0x7f74bc0b4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:55.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.727+0000 7f74c2c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f74bc0cd170 0x7f74bc06ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:55.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.727+0000 7f74c2c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74bc0cdb50 con 0x7f74bc0cd5d0 2024-04-20T21:17:55.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.727+0000 7f74c2c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f74bc145270 con 0x7f74bc0cd170 2024-04-20T21:17:55.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.727+0000 7f74c2c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f74bc0b3e50 con 0x7f74bc144e90 2024-04-20T21:17:55.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.727+0000 7f74c0c00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f74bc0cd5d0 0x7f74bc15bb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39294/0 (socket says 172.21.15.86:39294) 2024-04-20T21:17:55.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.727+0000 7f74c0c00640 1 -- 172.21.15.86:0/3435478161 learned_addr learned my addr 172.21.15.86:0/3435478161 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:17:55.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.727+0000 7f74bbe00640 1 --2- 172.21.15.86:0/3435478161 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74bc144e90 0x7f74bc0b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:55.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.727+0000 7f74bb400640 1 --2- 172.21.15.86:0/3435478161 >> v2:172.21.15.144:3300/0 conn(0x7f74bc0cd170 0x7f74bc06ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:55.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.728+0000 7f74baa00640 1 -- 172.21.15.86:0/3435478161 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3721644397 0 0) 0x7f74bc0cdb50 con 0x7f74bc0cd5d0 2024-04-20T21:17:55.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.728+0000 7f74baa00640 1 -- 172.21.15.86:0/3435478161 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f74a0003520 con 0x7f74bc0cd5d0 2024-04-20T21:17:55.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.728+0000 7f74baa00640 1 -- 172.21.15.86:0/3435478161 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1635895418 0 0) 0x7f74a0003520 con 0x7f74bc0cd5d0 2024-04-20T21:17:55.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.728+0000 7f74baa00640 1 -- 172.21.15.86:0/3435478161 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74bc144e90 msgr2=0x7f74bc0b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:55.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.728+0000 7f74baa00640 1 --2- 172.21.15.86:0/3435478161 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74bc144e90 0x7f74bc0b4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:55.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.728+0000 7f74baa00640 1 -- 172.21.15.86:0/3435478161 >> v2:172.21.15.144:3300/0 conn(0x7f74bc0cd170 msgr2=0x7f74bc06ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:55.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.728+0000 7f74baa00640 1 --2- 172.21.15.86:0/3435478161 >> v2:172.21.15.144:3300/0 conn(0x7f74bc0cd170 0x7f74bc06ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:55.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.728+0000 7f74baa00640 1 -- 172.21.15.86:0/3435478161 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74bc0b4130 con 0x7f74bc0cd5d0 2024-04-20T21:17:55.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.728+0000 7f74baa00640 1 -- 172.21.15.86:0/3435478161 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f74b40019f0 con 0x7f74bc0cd5d0 2024-04-20T21:17:55.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.728+0000 7f74baa00640 1 -- 172.21.15.86:0/3435478161 >> v1:172.21.15.86:6789/0 conn(0x7f74bc0cd5d0 legacy=0x7f74bc15bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:55.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.728+0000 7f74baa00640 1 --2- 172.21.15.86:0/3435478161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74a00039b0 0x7f74a0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:55.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.728+0000 7f74baa00640 1 --2- 172.21.15.86:0/3435478161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74a0004800 0x7f74a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:55.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.728+0000 7f74baa00640 1 --2- 172.21.15.86:0/3435478161 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74bc144e90 0x7f74a00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:55.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.728+0000 7f74baa00640 1 -- 172.21.15.86:0/3435478161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f74bc145270 con 0x7f74a0004800 2024-04-20T21:17:55.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.728+0000 7f74baa00640 1 -- 172.21.15.86:0/3435478161 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f74bc0b3e50 con 0x7f74a00039b0 2024-04-20T21:17:55.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.728+0000 7f74baa00640 1 -- 172.21.15.86:0/3435478161 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f74a000ba10 con 0x7f74bc144e90 2024-04-20T21:17:55.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.728+0000 7f74bb400640 1 --2- 172.21.15.86:0/3435478161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74a0004800 0x7f74a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:55.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.729+0000 7f74bbe00640 1 --2- 172.21.15.86:0/3435478161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74a00039b0 0x7f74a0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:55.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.729+0000 7f74bb400640 1 -- 172.21.15.86:0/3435478161 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74bc144e90 msgr2=0x7f74a00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:55.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.729+0000 7f74bb400640 1 --2- 172.21.15.86:0/3435478161 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74bc144e90 0x7f74a00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:55.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.729+0000 7f74bb400640 1 -- 172.21.15.86:0/3435478161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74a00039b0 msgr2=0x7f74a0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:55.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.729+0000 7f74bb400640 1 --2- 172.21.15.86:0/3435478161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74a00039b0 0x7f74a0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:55.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.729+0000 7f74bb400640 1 -- 172.21.15.86:0/3435478161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74a0003520 con 0x7f74a0004800 2024-04-20T21:17:55.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.730+0000 7f74bb400640 1 --2- 172.21.15.86:0/3435478161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74a0004800 0x7f74a0004c80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f74a8009af0 tx=0x7f74a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e177e581872e423 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:55.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.730+0000 7f74baa00640 1 -- 172.21.15.86:0/3435478161 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a8039070 con 0x7f74a0004800 2024-04-20T21:17:55.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.730+0000 7f74baa00640 1 -- 172.21.15.86:0/3435478161 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74a802c710 con 0x7f74a0004800 2024-04-20T21:17:55.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.730+0000 7f74baa00640 1 -- 172.21.15.86:0/3435478161 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a8009690 con 0x7f74a0004800 2024-04-20T21:17:55.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.731+0000 7f74c2c00640 1 -- 172.21.15.86:0/3435478161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74a0004800 msgr2=0x7f74a0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:55.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.731+0000 7f74c2c00640 1 --2- 172.21.15.86:0/3435478161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74a0004800 0x7f74a0004c80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f74a8009af0 tx=0x7f74a0005830 comp rx=0 tx=0).stop 2024-04-20T21:17:55.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.731+0000 7f74c0c00640 1 -- 172.21.15.86:0/3435478161 reap_dead start 2024-04-20T21:17:55.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.731+0000 7f74c2c00640 1 -- 172.21.15.86:0/3435478161 shutdown_connections 2024-04-20T21:17:55.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.731+0000 7f74c2c00640 1 -- 172.21.15.86:0/3435478161 >> 172.21.15.86:0/3435478161 conn(0x7f74bc0b2360 msgr2=0x7f74bc0b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:55.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.731+0000 7f74c2c00640 1 -- 172.21.15.86:0/3435478161 shutdown_connections 2024-04-20T21:17:55.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.731+0000 7f74c2c00640 1 -- 172.21.15.86:0/3435478161 wait complete. 2024-04-20T21:17:55.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.732+0000 7f74c2c00640 1 Processor -- start 2024-04-20T21:17:55.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.733+0000 7f74c2c00640 1 -- start start 2024-04-20T21:17:55.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.734+0000 7f74c2c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74a0004800 0x7f74bc15af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:55.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.734+0000 7f74c2c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74bc156500 0x7f74bc1339d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:55.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.734+0000 7f74c2c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74bc156980 0x7f74bc133f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:55.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.734+0000 7f74c2c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f74bc0d0190 con 0x7f74bc156980 2024-04-20T21:17:55.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.734+0000 7f74c2c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f74bc0d0010 con 0x7f74a0004800 2024-04-20T21:17:55.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.734+0000 7f74c2c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f74bc06f280 con 0x7f74bc156500 2024-04-20T21:17:55.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.734+0000 7f74c0c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74bc156980 0x7f74bc133f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:55.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.734+0000 7f74c0c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74bc156980 0x7f74bc133f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46758/0 (socket says 172.21.15.86:46758) 2024-04-20T21:17:55.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.734+0000 7f74c0c00640 1 -- 172.21.15.86:0/1533268862 learned_addr learned my addr 172.21.15.86:0/1533268862 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:55.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.734+0000 7f74bbe00640 1 --2- 172.21.15.86:0/1533268862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74a0004800 0x7f74bc15af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:55.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.734+0000 7f74bb400640 1 --2- 172.21.15.86:0/1533268862 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74bc156500 0x7f74bc1339d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:55.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.734+0000 7f74c0c00640 1 -- 172.21.15.86:0/1533268862 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74bc156500 msgr2=0x7f74bc1339d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:55.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.734+0000 7f74c0c00640 1 --2- 172.21.15.86:0/1533268862 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74bc156500 0x7f74bc1339d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:55.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.734+0000 7f74c0c00640 1 -- 172.21.15.86:0/1533268862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74a0004800 msgr2=0x7f74bc15af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:55.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.734+0000 7f74c0c00640 1 --2- 172.21.15.86:0/1533268862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74a0004800 0x7f74bc15af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:55.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.734+0000 7f74c0c00640 1 -- 172.21.15.86:0/1533268862 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74bc135c20 con 0x7f74bc156980 2024-04-20T21:17:55.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.735+0000 7f74c0c00640 1 --2- 172.21.15.86:0/1533268862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74bc156980 0x7f74bc133f10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f74bc06f920 tx=0x7f74b400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:55.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.735+0000 7f74b3e00640 1 -- 172.21.15.86:0/1533268862 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b400cea0 con 0x7f74bc156980 2024-04-20T21:17:55.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.735+0000 7f74b3e00640 1 -- 172.21.15.86:0/1533268862 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74b400a610 con 0x7f74bc156980 2024-04-20T21:17:55.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.735+0000 7f74b3e00640 1 -- 172.21.15.86:0/1533268862 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b400a9e0 con 0x7f74bc156980 2024-04-20T21:17:55.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.735+0000 7f74c2c00640 1 -- 172.21.15.86:0/1533268862 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74bc135f00 con 0x7f74bc156980 2024-04-20T21:17:55.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.735+0000 7f74c2c00640 1 -- 172.21.15.86:0/1533268862 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74bc15a780 con 0x7f74bc156980 2024-04-20T21:17:55.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.736+0000 7f74c2c00640 1 -- 172.21.15.86:0/1533268862 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74bc0689e0 con 0x7f74bc156980 2024-04-20T21:17:55.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.737+0000 7f74b3e00640 1 -- 172.21.15.86:0/1533268862 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f74b4011070 con 0x7f74bc156980 2024-04-20T21:17:55.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.738+0000 7f74b3e00640 1 --2- 172.21.15.86:0/1533268862 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f749c06c920 0x7f749c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:55.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.738+0000 7f74b3e00640 1 -- 172.21.15.86:0/1533268862 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f74b408b1e0 con 0x7f74bc156980 2024-04-20T21:17:55.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.738+0000 7f74bbe00640 1 --2- 172.21.15.86:0/1533268862 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f749c06c920 0x7f749c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:55.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.739+0000 7f74bbe00640 1 --2- 172.21.15.86:0/1533268862 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f749c06c920 0x7f749c06ede0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f74bc13ef00 tx=0x7f74ac008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:55.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.742+0000 7f74b3e00640 1 -- 172.21.15.86:0/1533268862 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f74b4019060 con 0x7f74bc156980 2024-04-20T21:17:55.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.982+0000 7f74c2c00640 1 -- 172.21.15.86:0/1533268862 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f74bc06e510 con 0x7f74bc156980 2024-04-20T21:17:55.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.983+0000 7f74b3e00640 1 -- 172.21.15.86:0/1533268862 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5196 (secure 0 0 0) 0x7f74b4011e40 con 0x7f74bc156980 2024-04-20T21:17:55.983 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:17:55.983 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/179773217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":179773217},{"type":"v1","addr":"172.21.15.144:6834","nonce":179773217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3179785507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3179785507},{"type":"v1","addr":"172.21.15.86:6835","nonce":3179785507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4270,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2780853729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2780853729},{"type":"v1","addr":"172.21.15.86:6837","nonce":2780853729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:17:51.700362+0000","modified":"2024-04-20T21:17:54.973792+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6835/2436181584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2436181584},{"type":"v1","addr":"172.21.15.144:6835","nonce":2436181584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4272,"qdb_cluster":[4272]},"id":1}]} 2024-04-20T21:17:55.983 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 7 2024-04-20T21:17:55.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.985+0000 7f74c2c00640 1 -- 172.21.15.86:0/1533268862 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f749c06c920 msgr2=0x7f749c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:55.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.985+0000 7f74c2c00640 1 --2- 172.21.15.86:0/1533268862 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f749c06c920 0x7f749c06ede0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f74bc13ef00 tx=0x7f74ac008040 comp rx=0 tx=0).stop 2024-04-20T21:17:55.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.985+0000 7f74c2c00640 1 -- 172.21.15.86:0/1533268862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74bc156980 msgr2=0x7f74bc133f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:55.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.985+0000 7f74c2c00640 1 --2- 172.21.15.86:0/1533268862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74bc156980 0x7f74bc133f10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f74bc06f920 tx=0x7f74b400cc50 comp rx=0 tx=0).stop 2024-04-20T21:17:55.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.985+0000 7f74c2c00640 1 -- 172.21.15.86:0/1533268862 shutdown_connections 2024-04-20T21:17:55.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.985+0000 7f74c2c00640 1 --2- 172.21.15.86:0/1533268862 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f749c06c920 0x7f749c06ede0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:55.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.985+0000 7f74c2c00640 1 --2- 172.21.15.86:0/1533268862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74bc156980 0x7f74bc133f10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:55.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.985+0000 7f74c2c00640 1 --2- 172.21.15.86:0/1533268862 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74bc156500 0x7f74bc1339d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:55.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.985+0000 7f74c2c00640 1 --2- 172.21.15.86:0/1533268862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74a0004800 0x7f74bc15af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:55.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.985+0000 7f74c2c00640 1 -- 172.21.15.86:0/1533268862 >> 172.21.15.86:0/1533268862 conn(0x7f74bc0b2360 msgr2=0x7f74bc0b0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:55.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.986+0000 7f74c2c00640 1 -- 172.21.15.86:0/1533268862 shutdown_connections 2024-04-20T21:17:55.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:55.986+0000 7f74c2c00640 1 -- 172.21.15.86:0/1533268862 wait complete. 2024-04-20T21:17:56.003 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:17:56.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.186+0000 7fd2c6000640 1 Processor -- start 2024-04-20T21:17:56.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.186+0000 7fd2c6000640 1 -- start start 2024-04-20T21:17:56.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.188+0000 7fd2c6000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd2c015c010 0x7fd2c015c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:56.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.188+0000 7fd2c6000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2c00ae710 con 0x7fd2c00af7b0 2024-04-20T21:17:56.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.188+0000 7fd2c6000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2c00ae9f0 con 0x7fd2c006f4c0 2024-04-20T21:17:56.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.188+0000 7fd2c6000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fd2c00af320 con 0x7fd2c015c010 2024-04-20T21:17:56.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.188+0000 7fd2bfe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd2c00af7b0 0x7fd2c00adda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:43442/0 (socket says 172.21.15.86:43442) 2024-04-20T21:17:56.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.188+0000 7fd2bfe00640 1 -- 172.21.15.86:0/2562761187 learned_addr learned my addr 172.21.15.86:0/2562761187 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:17:56.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.188+0000 7fd2c4c00640 1 --2- 172.21.15.86:0/2562761187 >> v2:172.21.15.144:3300/0 conn(0x7fd2c015c010 0x7fd2c015c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:56.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.188+0000 7fd2bf400640 1 -- 172.21.15.86:0/2562761187 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3773174718 0 0) 0x7fd2c00ae710 con 0x7fd2c00af7b0 2024-04-20T21:17:56.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.188+0000 7fd2bf400640 1 -- 172.21.15.86:0/2562761187 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd2a8003520 con 0x7fd2c00af7b0 2024-04-20T21:17:56.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.189+0000 7fd2bf400640 1 -- 172.21.15.86:0/2562761187 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1965375257 0 0) 0x7fd2a8003520 con 0x7fd2c00af7b0 2024-04-20T21:17:56.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.189+0000 7fd2bf400640 1 -- 172.21.15.86:0/2562761187 >> v2:172.21.15.144:3300/0 conn(0x7fd2c015c010 msgr2=0x7fd2c015c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:56.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.189+0000 7fd2bf400640 1 --2- 172.21.15.86:0/2562761187 >> v2:172.21.15.144:3300/0 conn(0x7fd2c015c010 0x7fd2c015c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:56.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.189+0000 7fd2bf400640 1 -- 172.21.15.86:0/2562761187 >> v1:172.21.15.144:6789/0 conn(0x7fd2c006f4c0 legacy=0x7fd2c0071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:56.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.189+0000 7fd2bf400640 1 -- 172.21.15.86:0/2562761187 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2c00aecd0 con 0x7fd2c00af7b0 2024-04-20T21:17:56.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.189+0000 7fd2bf400640 1 -- 172.21.15.86:0/2562761187 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fd2ac001670 con 0x7fd2c00af7b0 2024-04-20T21:17:56.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.189+0000 7fd2bf400640 1 -- 172.21.15.86:0/2562761187 >> v1:172.21.15.86:6789/0 conn(0x7fd2c00af7b0 legacy=0x7fd2c00adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:56.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.189+0000 7fd2bf400640 1 --2- 172.21.15.86:0/2562761187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2a8003a00 0x7fd2a8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:56.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.189+0000 7fd2bf400640 1 --2- 172.21.15.86:0/2562761187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2a8004800 0x7fd2a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:56.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.189+0000 7fd2bf400640 1 --2- 172.21.15.86:0/2562761187 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2a8005210 0x7fd2a800fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:56.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.189+0000 7fd2bf400640 1 -- 172.21.15.86:0/2562761187 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c00af320 con 0x7fd2a8004800 2024-04-20T21:17:56.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.189+0000 7fd2bf400640 1 -- 172.21.15.86:0/2562761187 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a8010550 con 0x7fd2a8003a00 2024-04-20T21:17:56.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.189+0000 7fd2bf400640 1 -- 172.21.15.86:0/2562761187 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd2a8010750 con 0x7fd2a8005210 2024-04-20T21:17:56.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.189+0000 7fd2bfe00640 1 --2- 172.21.15.86:0/2562761187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2a8004800 0x7fd2a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:56.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.189+0000 7fd2c4c00640 1 --2- 172.21.15.86:0/2562761187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2a8003a00 0x7fd2a8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:56.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.189+0000 7fd2bfe00640 1 -- 172.21.15.86:0/2562761187 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2a8005210 msgr2=0x7fd2a800fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:56.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.189+0000 7fd2c5600640 1 --2- 172.21.15.86:0/2562761187 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2a8005210 0x7fd2a800fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:56.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.190+0000 7fd2bfe00640 1 --2- 172.21.15.86:0/2562761187 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2a8005210 0x7fd2a800fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:56.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.190+0000 7fd2bfe00640 1 -- 172.21.15.86:0/2562761187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2a8003a00 msgr2=0x7fd2a8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:56.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.190+0000 7fd2bfe00640 1 --2- 172.21.15.86:0/2562761187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2a8003a00 0x7fd2a8003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:56.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.190+0000 7fd2bfe00640 1 -- 172.21.15.86:0/2562761187 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a8003520 con 0x7fd2a8004800 2024-04-20T21:17:56.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.190+0000 7fd2c4c00640 1 --2- 172.21.15.86:0/2562761187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2a8003a00 0x7fd2a8003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:17:56.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.190+0000 7fd2c5600640 1 --2- 172.21.15.86:0/2562761187 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2a8005210 0x7fd2a800fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:17:56.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.190+0000 7fd2c5600640 1 -- 172.21.15.86:0/2562761187 reap_dead start 2024-04-20T21:17:56.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.191+0000 7fd2bfe00640 1 --2- 172.21.15.86:0/2562761187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2a8004800 0x7fd2a8004c80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac00c160 tx=0x7fd2a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27afc70aebaad630 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:56.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.191+0000 7fd2bf400640 1 -- 172.21.15.86:0/2562761187 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac007c60 con 0x7fd2a8004800 2024-04-20T21:17:56.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.191+0000 7fd2bf400640 1 -- 172.21.15.86:0/2562761187 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2ac030070 con 0x7fd2a8004800 2024-04-20T21:17:56.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.191+0000 7fd2bf400640 1 -- 172.21.15.86:0/2562761187 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac005d50 con 0x7fd2a8004800 2024-04-20T21:17:56.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.191+0000 7fd2c6000640 1 -- 172.21.15.86:0/2562761187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2a8004800 msgr2=0x7fd2a8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:56.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.191+0000 7fd2c6000640 1 --2- 172.21.15.86:0/2562761187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2a8004800 0x7fd2a8004c80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac00c160 tx=0x7fd2a8005830 comp rx=0 tx=0).stop 2024-04-20T21:17:56.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.192+0000 7fd2c6000640 1 -- 172.21.15.86:0/2562761187 shutdown_connections 2024-04-20T21:17:56.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.192+0000 7fd2c6000640 1 --2- 172.21.15.86:0/2562761187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2a8004800 0x7fd2a8004c80 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:56.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.192+0000 7fd2c6000640 1 -- 172.21.15.86:0/2562761187 >> 172.21.15.86:0/2562761187 conn(0x7fd2c00d0170 msgr2=0x7fd2c00d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:56.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.192+0000 7fd2c6000640 1 -- 172.21.15.86:0/2562761187 shutdown_connections 2024-04-20T21:17:56.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.192+0000 7fd2c6000640 1 -- 172.21.15.86:0/2562761187 wait complete. 2024-04-20T21:17:56.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.193+0000 7fd2c6000640 1 Processor -- start 2024-04-20T21:17:56.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.194+0000 7fd2c6000640 1 -- start start 2024-04-20T21:17:56.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.194+0000 7fd2c6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2a8004800 0x7fd2c009de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:56.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.194+0000 7fd2c6000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2c01384b0 0x7fd2c01f7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:56.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.195+0000 7fd2c6000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2c01f8220 0x7fd2c00b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:56.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.195+0000 7fd2c6000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c015c570 con 0x7fd2a8004800 2024-04-20T21:17:56.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.195+0000 7fd2c6000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c015c3f0 con 0x7fd2c01f8220 2024-04-20T21:17:56.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.195+0000 7fd2c6000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd2c015c6f0 con 0x7fd2c01384b0 2024-04-20T21:17:56.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.195+0000 7fd2c4c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2a8004800 0x7fd2c009de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:56.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.195+0000 7fd2c4c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2a8004800 0x7fd2c009de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57570/0 (socket says 172.21.15.86:57570) 2024-04-20T21:17:56.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.195+0000 7fd2c4c00640 1 -- 172.21.15.86:0/854068594 learned_addr learned my addr 172.21.15.86:0/854068594 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:56.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.195+0000 7fd2bfe00640 1 --2- 172.21.15.86:0/854068594 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2c01384b0 0x7fd2c01f7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:56.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.195+0000 7fd2c4c00640 1 -- 172.21.15.86:0/854068594 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2c01384b0 msgr2=0x7fd2c01f7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:56.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.195+0000 7fd2c4c00640 1 --2- 172.21.15.86:0/854068594 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2c01384b0 0x7fd2c01f7ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:56.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.195+0000 7fd2c5600640 1 --2- 172.21.15.86:0/854068594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2c01f8220 0x7fd2c00b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:56.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.195+0000 7fd2c4c00640 1 -- 172.21.15.86:0/854068594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2c01f8220 msgr2=0x7fd2c00b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:56.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.195+0000 7fd2c4c00640 1 --2- 172.21.15.86:0/854068594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2c01f8220 0x7fd2c00b3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:56.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.195+0000 7fd2c4c00640 1 -- 172.21.15.86:0/854068594 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2c0138930 con 0x7fd2a8004800 2024-04-20T21:17:56.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.196+0000 7fd2c4c00640 1 --2- 172.21.15.86:0/854068594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2a8004800 0x7fd2c009de90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd2b400e840 tx=0x7fd2b400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:56.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.196+0000 7fd2bcc00640 1 -- 172.21.15.86:0/854068594 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b4017070 con 0x7fd2a8004800 2024-04-20T21:17:56.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.196+0000 7fd2bcc00640 1 -- 172.21.15.86:0/854068594 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2b400c8d0 con 0x7fd2a8004800 2024-04-20T21:17:56.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.196+0000 7fd2bcc00640 1 -- 172.21.15.86:0/854068594 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b400f040 con 0x7fd2a8004800 2024-04-20T21:17:56.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.196+0000 7fd2c6000640 1 -- 172.21.15.86:0/854068594 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2c015b200 con 0x7fd2a8004800 2024-04-20T21:17:56.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.196+0000 7fd2c6000640 1 -- 172.21.15.86:0/854068594 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2c015a720 con 0x7fd2a8004800 2024-04-20T21:17:56.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.197+0000 7fd2c6000640 1 -- 172.21.15.86:0/854068594 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2c004fcf0 con 0x7fd2a8004800 2024-04-20T21:17:56.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.198+0000 7fd2bcc00640 1 -- 172.21.15.86:0/854068594 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd2b401a070 con 0x7fd2a8004800 2024-04-20T21:17:56.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.199+0000 7fd2bcc00640 1 --2- 172.21.15.86:0/854068594 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd2a006c8f0 0x7fd2a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:56.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.199+0000 7fd2bcc00640 1 -- 172.21.15.86:0/854068594 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd2b408b820 con 0x7fd2a8004800 2024-04-20T21:17:56.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.199+0000 7fd2bfe00640 1 --2- 172.21.15.86:0/854068594 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd2a006c8f0 0x7fd2a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:56.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.200+0000 7fd2bfe00640 1 --2- 172.21.15.86:0/854068594 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd2a006c8f0 0x7fd2a006edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac034000 tx=0x7fd2ac039040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:56.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.203+0000 7fd2bcc00640 1 -- 172.21.15.86:0/854068594 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd2b4059bb0 con 0x7fd2a8004800 2024-04-20T21:17:56.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.402+0000 7fd2c6000640 1 -- 172.21.15.86:0/854068594 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd2c0072ca0 con 0x7fd2a8004800 2024-04-20T21:17:56.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.404+0000 7fd2bcc00640 1 -- 172.21.15.86:0/854068594 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7fd2b40594b0 con 0x7fd2a8004800 2024-04-20T21:17:56.405 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:17:56.405 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":20,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:17:53.274462+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:17:49.718799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:17:50.683477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:17:56.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.406+0000 7fd2c6000640 1 -- 172.21.15.86:0/854068594 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd2a006c8f0 msgr2=0x7fd2a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:56.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.406+0000 7fd2c6000640 1 --2- 172.21.15.86:0/854068594 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd2a006c8f0 0x7fd2a006edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac034000 tx=0x7fd2ac039040 comp rx=0 tx=0).stop 2024-04-20T21:17:56.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.406+0000 7fd2c6000640 1 -- 172.21.15.86:0/854068594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2a8004800 msgr2=0x7fd2c009de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:56.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.406+0000 7fd2c6000640 1 --2- 172.21.15.86:0/854068594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2a8004800 0x7fd2c009de90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd2b400e840 tx=0x7fd2b400ed00 comp rx=0 tx=0).stop 2024-04-20T21:17:56.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.406+0000 7fd2c6000640 1 -- 172.21.15.86:0/854068594 shutdown_connections 2024-04-20T21:17:56.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.406+0000 7fd2c6000640 1 --2- 172.21.15.86:0/854068594 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd2a006c8f0 0x7fd2a006edb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:56.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.406+0000 7fd2c6000640 1 --2- 172.21.15.86:0/854068594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2c01f8220 0x7fd2c00b3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:56.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.406+0000 7fd2c6000640 1 --2- 172.21.15.86:0/854068594 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2c01384b0 0x7fd2c01f7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:56.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.406+0000 7fd2c6000640 1 --2- 172.21.15.86:0/854068594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2a8004800 0x7fd2c009de90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:56.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.406+0000 7fd2c6000640 1 -- 172.21.15.86:0/854068594 >> 172.21.15.86:0/854068594 conn(0x7fd2c00d0170 msgr2=0x7fd2c00731c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:56.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.407+0000 7fd2c6000640 1 -- 172.21.15.86:0/854068594 shutdown_connections 2024-04-20T21:17:56.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.407+0000 7fd2c6000640 1 -- 172.21.15.86:0/854068594 wait complete. 2024-04-20T21:17:56.426 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-20T21:17:56.427 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T21:17:56.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.616+0000 7f72c0000640 1 Processor -- start 2024-04-20T21:17:56.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.616+0000 7f72c0000640 1 -- start start 2024-04-20T21:17:56.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.617+0000 7f72c0000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f72b815ac10 0x7f72b80b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:56.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.617+0000 7f72c0000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72b806dcd0 con 0x7f72b815f210 2024-04-20T21:17:56.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.617+0000 7f72c0000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72b806dfb0 con 0x7f72b8072ca0 2024-04-20T21:17:56.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.617+0000 7f72c0000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f72b806e8e0 con 0x7f72b815ac10 2024-04-20T21:17:56.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.617+0000 7f72bd600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f72b815ac10 0x7f72b80b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:56.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.617+0000 7f72bcc00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f72b815f210 0x7f72b8072590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:43454/0 (socket says 172.21.15.86:43454) 2024-04-20T21:17:56.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.617+0000 7f72bcc00640 1 -- 172.21.15.86:0/774178548 learned_addr learned my addr 172.21.15.86:0/774178548 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:17:56.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.618+0000 7f72bd600640 1 -- 172.21.15.86:0/774178548 >> v1:172.21.15.144:6789/0 conn(0x7f72b8072ca0 legacy=0x7f72b80cea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:56.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.618+0000 7f72bd600640 1 -- 172.21.15.86:0/774178548 >> v1:172.21.15.86:6789/0 conn(0x7f72b815f210 legacy=0x7f72b8072590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:56.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.618+0000 7f72bd600640 1 -- 172.21.15.86:0/774178548 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72b806e290 con 0x7f72b815ac10 2024-04-20T21:17:56.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.618+0000 7f72b7e00640 1 -- 172.21.15.86:0/774178548 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2028962777 0 0) 0x7f72b806dcd0 con 0x7f72b815f210 2024-04-20T21:17:56.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.619+0000 7f72bd600640 1 --2- 172.21.15.86:0/774178548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72b815ac10 0x7f72b80b53b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f72ac0097a0 tx=0x7f72ac02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32424ffa761904c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:56.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.619+0000 7f72b7e00640 1 -- 172.21.15.86:0/774178548 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ac0315a0 con 0x7f72b815ac10 2024-04-20T21:17:56.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.619+0000 7f72b7e00640 1 -- 172.21.15.86:0/774178548 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72ac037070 con 0x7f72b815ac10 2024-04-20T21:17:56.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.619+0000 7f72b7e00640 1 -- 172.21.15.86:0/774178548 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ac031960 con 0x7f72b815ac10 2024-04-20T21:17:56.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.620+0000 7f72c0000640 1 -- 172.21.15.86:0/774178548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72b815ac10 msgr2=0x7f72b80b53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:56.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.620+0000 7f72c0000640 1 --2- 172.21.15.86:0/774178548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72b815ac10 0x7f72b80b53b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f72ac0097a0 tx=0x7f72ac02b840 comp rx=0 tx=0).stop 2024-04-20T21:17:56.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.620+0000 7f72c0000640 1 -- 172.21.15.86:0/774178548 shutdown_connections 2024-04-20T21:17:56.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.620+0000 7f72c0000640 1 --2- 172.21.15.86:0/774178548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72b815ac10 0x7f72b80b53b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:56.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.620+0000 7f72c0000640 1 -- 172.21.15.86:0/774178548 >> 172.21.15.86:0/774178548 conn(0x7f72b80b2360 msgr2=0x7f72b80b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:56.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.620+0000 7f72c0000640 1 -- 172.21.15.86:0/774178548 shutdown_connections 2024-04-20T21:17:56.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.620+0000 7f72c0000640 1 -- 172.21.15.86:0/774178548 wait complete. 2024-04-20T21:17:56.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.622+0000 7f72c0000640 1 Processor -- start 2024-04-20T21:17:56.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.622+0000 7f72c0000640 1 -- start start 2024-04-20T21:17:56.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.623+0000 7f72c0000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72b8072ca0 0x7f72b814b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:56.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.623+0000 7f72c0000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72b815ac10 0x7f72b814ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:56.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.623+0000 7f72c0000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72b815f210 0x7f72b81494b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:56.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.623+0000 7f72c0000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f72b815f610 con 0x7f72b815f210 2024-04-20T21:17:56.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.623+0000 7f72c0000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f72b815aff0 con 0x7f72b815ac10 2024-04-20T21:17:56.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.623+0000 7f72c0000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f72b815f790 con 0x7f72b8072ca0 2024-04-20T21:17:56.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.623+0000 7f72be000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72b815f210 0x7f72b81494b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:56.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.623+0000 7f72be000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72b815f210 0x7f72b81494b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57588/0 (socket says 172.21.15.86:57588) 2024-04-20T21:17:56.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.623+0000 7f72be000640 1 -- 172.21.15.86:0/1426768954 learned_addr learned my addr 172.21.15.86:0/1426768954 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:56.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.623+0000 7f72bcc00640 1 --2- 172.21.15.86:0/1426768954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72b815ac10 0x7f72b814ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:56.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.623+0000 7f72be000640 1 -- 172.21.15.86:0/1426768954 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72b8072ca0 msgr2=0x7f72b814b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:56.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.623+0000 7f72bd600640 1 --2- 172.21.15.86:0/1426768954 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72b8072ca0 0x7f72b814b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:56.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.623+0000 7f72be000640 1 --2- 172.21.15.86:0/1426768954 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72b8072ca0 0x7f72b814b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:56.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.623+0000 7f72be000640 1 -- 172.21.15.86:0/1426768954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72b815ac10 msgr2=0x7f72b814ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:56.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.623+0000 7f72be000640 1 --2- 172.21.15.86:0/1426768954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72b815ac10 0x7f72b814ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:56.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.623+0000 7f72be000640 1 -- 172.21.15.86:0/1426768954 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72b8152480 con 0x7f72b815f210 2024-04-20T21:17:56.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.623+0000 7f72bcc00640 1 --2- 172.21.15.86:0/1426768954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72b815ac10 0x7f72b814ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:17:56.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.624+0000 7f72be000640 1 --2- 172.21.15.86:0/1426768954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72b815f210 0x7f72b81494b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f72b000c730 tx=0x7f72b000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:56.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.624+0000 7f72b5600640 1 -- 172.21.15.86:0/1426768954 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b0016020 con 0x7f72b815f210 2024-04-20T21:17:56.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.624+0000 7f72b5600640 1 -- 172.21.15.86:0/1426768954 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72b000aa50 con 0x7f72b815f210 2024-04-20T21:17:56.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.624+0000 7f72b5600640 1 -- 172.21.15.86:0/1426768954 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b000ce40 con 0x7f72b815f210 2024-04-20T21:17:56.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.624+0000 7f72c0000640 1 -- 172.21.15.86:0/1426768954 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72b81513e0 con 0x7f72b815f210 2024-04-20T21:17:56.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.624+0000 7f72c0000640 1 -- 172.21.15.86:0/1426768954 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72b81519a0 con 0x7f72b815f210 2024-04-20T21:17:56.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.625+0000 7f72c0000640 1 -- 172.21.15.86:0/1426768954 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72b80689e0 con 0x7f72b815f210 2024-04-20T21:17:56.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.626+0000 7f72b5600640 1 -- 172.21.15.86:0/1426768954 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f72b0019020 con 0x7f72b815f210 2024-04-20T21:17:56.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.627+0000 7f72b5600640 1 --2- 172.21.15.86:0/1426768954 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f729806c920 0x7f729806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:56.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.627+0000 7f72b5600640 1 -- 172.21.15.86:0/1426768954 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f72b008bb70 con 0x7f72b815f210 2024-04-20T21:17:56.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.627+0000 7f72bd600640 1 --2- 172.21.15.86:0/1426768954 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f729806c920 0x7f729806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:56.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.628+0000 7f72bd600640 1 --2- 172.21.15.86:0/1426768954 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f729806c920 0x7f729806ede0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f72ac036000 tx=0x7f72ac008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:56.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.632+0000 7f72b5600640 1 -- 172.21.15.86:0/1426768954 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f72b0020030 con 0x7f72b815f210 2024-04-20T21:17:56.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.828+0000 7f72c0000640 1 -- 172.21.15.86:0/1426768954 --> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f72b80b47c0 con 0x7f729806c920 2024-04-20T21:17:56.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.840+0000 7f72b5600640 1 -- 172.21.15.86:0/1426768954 <== mgr.4107 v2:172.21.15.86:6832/302045240 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251948 (secure 0 0 0) 0x7f72b80b47c0 con 0x7f729806c920 2024-04-20T21:17:56.845 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:17:56.847 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-20T21:17:56.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.844+0000 7f72c0000640 1 -- 172.21.15.86:0/1426768954 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f729806c920 msgr2=0x7f729806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:56.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.844+0000 7f72c0000640 1 --2- 172.21.15.86:0/1426768954 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f729806c920 0x7f729806ede0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f72ac036000 tx=0x7f72ac008030 comp rx=0 tx=0).stop 2024-04-20T21:17:56.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.844+0000 7f72c0000640 1 -- 172.21.15.86:0/1426768954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72b815f210 msgr2=0x7f72b81494b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:56.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.844+0000 7f72c0000640 1 --2- 172.21.15.86:0/1426768954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72b815f210 0x7f72b81494b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f72b000c730 tx=0x7f72b000cbf0 comp rx=0 tx=0).stop 2024-04-20T21:17:56.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.844+0000 7f72c0000640 1 -- 172.21.15.86:0/1426768954 shutdown_connections 2024-04-20T21:17:56.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.844+0000 7f72c0000640 1 --2- 172.21.15.86:0/1426768954 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f729806c920 0x7f729806ede0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:56.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.844+0000 7f72c0000640 1 --2- 172.21.15.86:0/1426768954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f72b815f210 0x7f72b81494b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:56.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.844+0000 7f72c0000640 1 --2- 172.21.15.86:0/1426768954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72b815ac10 0x7f72b814ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:56.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.844+0000 7f72c0000640 1 --2- 172.21.15.86:0/1426768954 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72b8072ca0 0x7f72b814b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:56.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.844+0000 7f72c0000640 1 -- 172.21.15.86:0/1426768954 >> 172.21.15.86:0/1426768954 conn(0x7f72b80b2360 msgr2=0x7f72b80b1910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:56.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.844+0000 7f72c0000640 1 -- 172.21.15.86:0/1426768954 shutdown_connections 2024-04-20T21:17:56.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:56.845+0000 7f72c0000640 1 -- 172.21.15.86:0/1426768954 wait complete. 2024-04-20T21:17:56.861 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-20T21:17:56.631964+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":180,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219000,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755720,"statfs":{"total":773094113280,"available":772869857280,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001720"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.607719+0000","last_change":"2024-04-20T21:17:52.309045+0000","last_active":"2024-04-20T21:17:53.607719+0000","last_peered":"2024-04-20T21:17:53.607719+0000","last_clean":"2024-04-20T21:17:53.607719+0000","last_became_active":"2024-04-20T21:17:52.308808+0000","last_became_peered":"2024-04-20T21:17:52.308808+0000","last_unstale":"2024-04-20T21:17:53.607719+0000","last_undegraded":"2024-04-20T21:17:53.607719+0000","last_fullsized":"2024-04-20T21:17:53.607719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:15:54.762913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.281640+0000","last_change":"2024-04-20T21:17:52.303716+0000","last_active":"2024-04-20T21:17:53.281640+0000","last_peered":"2024-04-20T21:17:53.281640+0000","last_clean":"2024-04-20T21:17:53.281640+0000","last_became_active":"2024-04-20T21:17:52.303460+0000","last_became_peered":"2024-04-20T21:17:52.303460+0000","last_unstale":"2024-04-20T21:17:53.281640+0000","last_undegraded":"2024-04-20T21:17:53.281640+0000","last_fullsized":"2024-04-20T21:17:53.281640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:34:47.842666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.684324+0000","last_change":"2024-04-20T21:17:52.297807+0000","last_active":"2024-04-20T21:17:54.684324+0000","last_peered":"2024-04-20T21:17:54.684324+0000","last_clean":"2024-04-20T21:17:54.684324+0000","last_became_active":"2024-04-20T21:17:52.296178+0000","last_became_peered":"2024-04-20T21:17:52.296178+0000","last_unstale":"2024-04-20T21:17:54.684324+0000","last_undegraded":"2024-04-20T21:17:54.684324+0000","last_fullsized":"2024-04-20T21:17:54.684324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:42:57.243810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.357862+0000","last_change":"2024-04-20T21:17:52.315284+0000","last_active":"2024-04-20T21:17:54.357862+0000","last_peered":"2024-04-20T21:17:54.357862+0000","last_clean":"2024-04-20T21:17:54.357862+0000","last_became_active":"2024-04-20T21:17:52.314657+0000","last_became_peered":"2024-04-20T21:17:52.314657+0000","last_unstale":"2024-04-20T21:17:54.357862+0000","last_undegraded":"2024-04-20T21:17:54.357862+0000","last_fullsized":"2024-04-20T21:17:54.357862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:52:22.418925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.606807+0000","last_change":"2024-04-20T21:17:52.299456+0000","last_active":"2024-04-20T21:17:53.606807+0000","last_peered":"2024-04-20T21:17:53.606807+0000","last_clean":"2024-04-20T21:17:53.606807+0000","last_became_active":"2024-04-20T21:17:52.299016+0000","last_became_peered":"2024-04-20T21:17:52.299016+0000","last_unstale":"2024-04-20T21:17:53.606807+0000","last_undegraded":"2024-04-20T21:17:53.606807+0000","last_fullsized":"2024-04-20T21:17:53.606807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:04:26.111984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.357200+0000","last_change":"2024-04-20T21:17:52.305054+0000","last_active":"2024-04-20T21:17:54.357200+0000","last_peered":"2024-04-20T21:17:54.357200+0000","last_clean":"2024-04-20T21:17:54.357200+0000","last_became_active":"2024-04-20T21:17:52.303307+0000","last_became_peered":"2024-04-20T21:17:52.303307+0000","last_unstale":"2024-04-20T21:17:54.357200+0000","last_undegraded":"2024-04-20T21:17:54.357200+0000","last_fullsized":"2024-04-20T21:17:54.357200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:34:47.842666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.606631+0000","last_change":"2024-04-20T21:17:52.306121+0000","last_active":"2024-04-20T21:17:53.606631+0000","last_peered":"2024-04-20T21:17:53.606631+0000","last_clean":"2024-04-20T21:17:53.606631+0000","last_became_active":"2024-04-20T21:17:52.305749+0000","last_became_peered":"2024-04-20T21:17:52.305749+0000","last_unstale":"2024-04-20T21:17:53.606631+0000","last_undegraded":"2024-04-20T21:17:53.606631+0000","last_fullsized":"2024-04-20T21:17:53.606631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:10:11.436747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.682205+0000","last_change":"2024-04-20T21:17:52.292183+0000","last_active":"2024-04-20T21:17:54.682205+0000","last_peered":"2024-04-20T21:17:54.682205+0000","last_clean":"2024-04-20T21:17:54.682205+0000","last_became_active":"2024-04-20T21:17:52.291906+0000","last_became_peered":"2024-04-20T21:17:52.291906+0000","last_unstale":"2024-04-20T21:17:54.682205+0000","last_undegraded":"2024-04-20T21:17:54.682205+0000","last_fullsized":"2024-04-20T21:17:54.682205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:42:57.243810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.607755+0000","last_change":"2024-04-20T21:17:52.316938+0000","last_active":"2024-04-20T21:17:53.607755+0000","last_peered":"2024-04-20T21:17:53.607755+0000","last_clean":"2024-04-20T21:17:53.607755+0000","last_became_active":"2024-04-20T21:17:52.316386+0000","last_became_peered":"2024-04-20T21:17:52.316386+0000","last_unstale":"2024-04-20T21:17:53.607755+0000","last_undegraded":"2024-04-20T21:17:53.607755+0000","last_fullsized":"2024-04-20T21:17:53.607755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:23:58.112035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.289257+0000","last_change":"2024-04-20T21:17:52.299679+0000","last_active":"2024-04-20T21:17:53.289257+0000","last_peered":"2024-04-20T21:17:53.289257+0000","last_clean":"2024-04-20T21:17:53.289257+0000","last_became_active":"2024-04-20T21:17:52.299325+0000","last_became_peered":"2024-04-20T21:17:52.299325+0000","last_unstale":"2024-04-20T21:17:53.289257+0000","last_undegraded":"2024-04-20T21:17:53.289257+0000","last_fullsized":"2024-04-20T21:17:53.289257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:42:57.243810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.289696+0000","last_change":"2024-04-20T21:17:52.298416+0000","last_active":"2024-04-20T21:17:53.289696+0000","last_peered":"2024-04-20T21:17:53.289696+0000","last_clean":"2024-04-20T21:17:53.289696+0000","last_became_active":"2024-04-20T21:17:52.297990+0000","last_became_peered":"2024-04-20T21:17:52.297990+0000","last_unstale":"2024-04-20T21:17:53.289696+0000","last_undegraded":"2024-04-20T21:17:53.289696+0000","last_fullsized":"2024-04-20T21:17:53.289696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:11:50.878621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.287223+0000","last_change":"2024-04-20T21:17:52.279605+0000","last_active":"2024-04-20T21:17:53.287223+0000","last_peered":"2024-04-20T21:17:53.287223+0000","last_clean":"2024-04-20T21:17:53.287223+0000","last_became_active":"2024-04-20T21:17:52.277631+0000","last_became_peered":"2024-04-20T21:17:52.277631+0000","last_unstale":"2024-04-20T21:17:53.287223+0000","last_undegraded":"2024-04-20T21:17:53.287223+0000","last_fullsized":"2024-04-20T21:17:53.287223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:29:18.971384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.680792+0000","last_change":"2024-04-20T21:17:52.300679+0000","last_active":"2024-04-20T21:17:54.680792+0000","last_peered":"2024-04-20T21:17:54.680792+0000","last_clean":"2024-04-20T21:17:54.680792+0000","last_became_active":"2024-04-20T21:17:52.300387+0000","last_became_peered":"2024-04-20T21:17:52.300387+0000","last_unstale":"2024-04-20T21:17:54.680792+0000","last_undegraded":"2024-04-20T21:17:54.680792+0000","last_fullsized":"2024-04-20T21:17:54.680792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:09:02.488057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.682763+0000","last_change":"2024-04-20T21:17:52.301182+0000","last_active":"2024-04-20T21:17:54.682763+0000","last_peered":"2024-04-20T21:17:54.682763+0000","last_clean":"2024-04-20T21:17:54.682763+0000","last_became_active":"2024-04-20T21:17:52.300852+0000","last_became_peered":"2024-04-20T21:17:52.300852+0000","last_unstale":"2024-04-20T21:17:54.682763+0000","last_undegraded":"2024-04-20T21:17:54.682763+0000","last_fullsized":"2024-04-20T21:17:54.682763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:38:56.558892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.281082+0000","last_change":"2024-04-20T21:17:52.282408+0000","last_active":"2024-04-20T21:17:53.281082+0000","last_peered":"2024-04-20T21:17:53.281082+0000","last_clean":"2024-04-20T21:17:53.281082+0000","last_became_active":"2024-04-20T21:17:52.282044+0000","last_became_peered":"2024-04-20T21:17:52.282044+0000","last_unstale":"2024-04-20T21:17:53.281082+0000","last_undegraded":"2024-04-20T21:17:53.281082+0000","last_fullsized":"2024-04-20T21:17:53.281082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:27:15.675834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.357121+0000","last_change":"2024-04-20T21:17:52.315912+0000","last_active":"2024-04-20T21:17:54.357121+0000","last_peered":"2024-04-20T21:17:54.357121+0000","last_clean":"2024-04-20T21:17:54.357121+0000","last_became_active":"2024-04-20T21:17:52.315642+0000","last_became_peered":"2024-04-20T21:17:52.315642+0000","last_unstale":"2024-04-20T21:17:54.357121+0000","last_undegraded":"2024-04-20T21:17:54.357121+0000","last_fullsized":"2024-04-20T21:17:54.357121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:11:50.878621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.358578+0000","last_change":"2024-04-20T21:17:52.315150+0000","last_active":"2024-04-20T21:17:54.358578+0000","last_peered":"2024-04-20T21:17:54.358578+0000","last_clean":"2024-04-20T21:17:54.358578+0000","last_became_active":"2024-04-20T21:17:52.314856+0000","last_became_peered":"2024-04-20T21:17:52.314856+0000","last_unstale":"2024-04-20T21:17:54.358578+0000","last_undegraded":"2024-04-20T21:17:54.358578+0000","last_fullsized":"2024-04-20T21:17:54.358578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:15:54.762913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.679762+0000","last_change":"2024-04-20T21:17:52.286233+0000","last_active":"2024-04-20T21:17:54.679762+0000","last_peered":"2024-04-20T21:17:54.679762+0000","last_clean":"2024-04-20T21:17:54.679762+0000","last_became_active":"2024-04-20T21:17:52.285872+0000","last_became_peered":"2024-04-20T21:17:52.285872+0000","last_unstale":"2024-04-20T21:17:54.679762+0000","last_undegraded":"2024-04-20T21:17:54.679762+0000","last_fullsized":"2024-04-20T21:17:54.679762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:18:32.885585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.607283+0000","last_change":"2024-04-20T21:17:52.308569+0000","last_active":"2024-04-20T21:17:53.607283+0000","last_peered":"2024-04-20T21:17:53.607283+0000","last_clean":"2024-04-20T21:17:53.607283+0000","last_became_active":"2024-04-20T21:17:52.307606+0000","last_became_peered":"2024-04-20T21:17:52.307606+0000","last_unstale":"2024-04-20T21:17:53.607283+0000","last_undegraded":"2024-04-20T21:17:53.607283+0000","last_fullsized":"2024-04-20T21:17:53.607283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:34:46.630106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.288753+0000","last_change":"2024-04-20T21:17:52.281953+0000","last_active":"2024-04-20T21:17:53.288753+0000","last_peered":"2024-04-20T21:17:53.288753+0000","last_clean":"2024-04-20T21:17:53.288753+0000","last_became_active":"2024-04-20T21:17:52.281675+0000","last_became_peered":"2024-04-20T21:17:52.281675+0000","last_unstale":"2024-04-20T21:17:53.288753+0000","last_undegraded":"2024-04-20T21:17:53.288753+0000","last_fullsized":"2024-04-20T21:17:53.288753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:27:15.675834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.683439+0000","last_change":"2024-04-20T21:17:52.308448+0000","last_active":"2024-04-20T21:17:54.683439+0000","last_peered":"2024-04-20T21:17:54.683439+0000","last_clean":"2024-04-20T21:17:54.683439+0000","last_became_active":"2024-04-20T21:17:52.308158+0000","last_became_peered":"2024-04-20T21:17:52.308158+0000","last_unstale":"2024-04-20T21:17:54.683439+0000","last_undegraded":"2024-04-20T21:17:54.683439+0000","last_fullsized":"2024-04-20T21:17:54.683439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:09:02.488057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.357412+0000","last_change":"2024-04-20T21:17:52.304114+0000","last_active":"2024-04-20T21:17:54.357412+0000","last_peered":"2024-04-20T21:17:54.357412+0000","last_clean":"2024-04-20T21:17:54.357412+0000","last_became_active":"2024-04-20T21:17:52.303016+0000","last_became_peered":"2024-04-20T21:17:52.303016+0000","last_unstale":"2024-04-20T21:17:54.357412+0000","last_undegraded":"2024-04-20T21:17:54.357412+0000","last_fullsized":"2024-04-20T21:17:54.357412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:26:30.156714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.681213+0000","last_change":"2024-04-20T21:17:52.297418+0000","last_active":"2024-04-20T21:17:54.681213+0000","last_peered":"2024-04-20T21:17:54.681213+0000","last_clean":"2024-04-20T21:17:54.681213+0000","last_became_active":"2024-04-20T21:17:52.297018+0000","last_became_peered":"2024-04-20T21:17:52.297018+0000","last_unstale":"2024-04-20T21:17:54.681213+0000","last_undegraded":"2024-04-20T21:17:54.681213+0000","last_fullsized":"2024-04-20T21:17:54.681213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:12:56.626507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.357408+0000","last_change":"2024-04-20T21:17:52.306133+0000","last_active":"2024-04-20T21:17:54.357408+0000","last_peered":"2024-04-20T21:17:54.357408+0000","last_clean":"2024-04-20T21:17:54.357408+0000","last_became_active":"2024-04-20T21:17:52.304108+0000","last_became_peered":"2024-04-20T21:17:52.304108+0000","last_unstale":"2024-04-20T21:17:54.357408+0000","last_undegraded":"2024-04-20T21:17:54.357408+0000","last_fullsized":"2024-04-20T21:17:54.357408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:10:11.436747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.683656+0000","last_change":"2024-04-20T21:17:52.301781+0000","last_active":"2024-04-20T21:17:54.683656+0000","last_peered":"2024-04-20T21:17:54.683656+0000","last_clean":"2024-04-20T21:17:54.683656+0000","last_became_active":"2024-04-20T21:17:52.301194+0000","last_became_peered":"2024-04-20T21:17:52.301194+0000","last_unstale":"2024-04-20T21:17:54.683656+0000","last_undegraded":"2024-04-20T21:17:54.683656+0000","last_fullsized":"2024-04-20T21:17:54.683656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:10:11.436747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.289706+0000","last_change":"2024-04-20T21:17:52.303955+0000","last_active":"2024-04-20T21:17:53.289706+0000","last_peered":"2024-04-20T21:17:53.289706+0000","last_clean":"2024-04-20T21:17:53.289706+0000","last_became_active":"2024-04-20T21:17:52.303421+0000","last_became_peered":"2024-04-20T21:17:52.303421+0000","last_unstale":"2024-04-20T21:17:53.289706+0000","last_undegraded":"2024-04-20T21:17:53.289706+0000","last_fullsized":"2024-04-20T21:17:53.289706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:34:47.842666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.281621+0000","last_change":"2024-04-20T21:17:52.304472+0000","last_active":"2024-04-20T21:17:53.281621+0000","last_peered":"2024-04-20T21:17:53.281621+0000","last_clean":"2024-04-20T21:17:53.281621+0000","last_became_active":"2024-04-20T21:17:52.303141+0000","last_became_peered":"2024-04-20T21:17:52.303141+0000","last_unstale":"2024-04-20T21:17:53.281621+0000","last_undegraded":"2024-04-20T21:17:53.281621+0000","last_fullsized":"2024-04-20T21:17:53.281621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:11:50.878621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.681465+0000","last_change":"2024-04-20T21:17:52.302610+0000","last_active":"2024-04-20T21:17:54.681465+0000","last_peered":"2024-04-20T21:17:54.681465+0000","last_clean":"2024-04-20T21:17:54.681465+0000","last_became_active":"2024-04-20T21:17:52.302271+0000","last_became_peered":"2024-04-20T21:17:52.302271+0000","last_unstale":"2024-04-20T21:17:54.681465+0000","last_undegraded":"2024-04-20T21:17:54.681465+0000","last_fullsized":"2024-04-20T21:17:54.681465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:27:15.675834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.683628+0000","last_change":"2024-04-20T21:17:52.307809+0000","last_active":"2024-04-20T21:17:54.683628+0000","last_peered":"2024-04-20T21:17:54.683628+0000","last_clean":"2024-04-20T21:17:54.683628+0000","last_became_active":"2024-04-20T21:17:52.307459+0000","last_became_peered":"2024-04-20T21:17:52.307459+0000","last_unstale":"2024-04-20T21:17:54.683628+0000","last_undegraded":"2024-04-20T21:17:54.683628+0000","last_fullsized":"2024-04-20T21:17:54.683628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:52:26.845286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.357745+0000","last_change":"2024-04-20T21:17:52.305447+0000","last_active":"2024-04-20T21:17:54.357745+0000","last_peered":"2024-04-20T21:17:54.357745+0000","last_clean":"2024-04-20T21:17:54.357745+0000","last_became_active":"2024-04-20T21:17:52.303784+0000","last_became_peered":"2024-04-20T21:17:52.303784+0000","last_unstale":"2024-04-20T21:17:54.357745+0000","last_undegraded":"2024-04-20T21:17:54.357745+0000","last_fullsized":"2024-04-20T21:17:54.357745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:04:26.111984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.679196+0000","last_change":"2024-04-20T21:17:52.294245+0000","last_active":"2024-04-20T21:17:54.679196+0000","last_peered":"2024-04-20T21:17:54.679196+0000","last_clean":"2024-04-20T21:17:54.679196+0000","last_became_active":"2024-04-20T21:17:52.293880+0000","last_became_peered":"2024-04-20T21:17:52.293880+0000","last_unstale":"2024-04-20T21:17:54.679196+0000","last_undegraded":"2024-04-20T21:17:54.679196+0000","last_fullsized":"2024-04-20T21:17:54.679196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:51:58.727652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.287037+0000","last_change":"2024-04-20T21:17:52.303676+0000","last_active":"2024-04-20T21:17:53.287037+0000","last_peered":"2024-04-20T21:17:53.287037+0000","last_clean":"2024-04-20T21:17:53.287037+0000","last_became_active":"2024-04-20T21:17:52.303369+0000","last_became_peered":"2024-04-20T21:17:52.303369+0000","last_unstale":"2024-04-20T21:17:53.287037+0000","last_undegraded":"2024-04-20T21:17:53.287037+0000","last_fullsized":"2024-04-20T21:17:53.287037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:09:02.488057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.287416+0000","last_change":"2024-04-20T21:17:52.277151+0000","last_active":"2024-04-20T21:17:53.287416+0000","last_peered":"2024-04-20T21:17:53.287416+0000","last_clean":"2024-04-20T21:17:53.287416+0000","last_became_active":"2024-04-20T21:17:52.276740+0000","last_became_peered":"2024-04-20T21:17:52.276740+0000","last_unstale":"2024-04-20T21:17:53.287416+0000","last_undegraded":"2024-04-20T21:17:53.287416+0000","last_fullsized":"2024-04-20T21:17:53.287416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:34:27.917112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.680860+0000","last_change":"2024-04-20T21:17:51.275103+0000","last_active":"2024-04-20T21:17:54.680860+0000","last_peered":"2024-04-20T21:17:54.680860+0000","last_clean":"2024-04-20T21:17:54.680860+0000","last_became_active":"2024-04-20T21:17:51.274773+0000","last_became_peered":"2024-04-20T21:17:51.274773+0000","last_unstale":"2024-04-20T21:17:54.680860+0000","last_undegraded":"2024-04-20T21:17:54.680860+0000","last_fullsized":"2024-04-20T21:17:54.680860+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:53:16.864261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.683824+0000","last_change":"2024-04-20T21:17:52.312353+0000","last_active":"2024-04-20T21:17:54.683824+0000","last_peered":"2024-04-20T21:17:54.683824+0000","last_clean":"2024-04-20T21:17:54.683824+0000","last_became_active":"2024-04-20T21:17:52.311933+0000","last_became_peered":"2024-04-20T21:17:52.311933+0000","last_unstale":"2024-04-20T21:17:54.683824+0000","last_undegraded":"2024-04-20T21:17:54.683824+0000","last_fullsized":"2024-04-20T21:17:54.683824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:12:56.626507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.288439+0000","last_change":"2024-04-20T21:17:51.269484+0000","last_active":"2024-04-20T21:17:53.288439+0000","last_peered":"2024-04-20T21:17:53.288439+0000","last_clean":"2024-04-20T21:17:53.288439+0000","last_became_active":"2024-04-20T21:17:51.269196+0000","last_became_peered":"2024-04-20T21:17:51.269196+0000","last_unstale":"2024-04-20T21:17:53.288439+0000","last_undegraded":"2024-04-20T21:17:53.288439+0000","last_fullsized":"2024-04-20T21:17:53.288439+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:18:39.465785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.681496+0000","last_change":"2024-04-20T21:17:51.273330+0000","last_active":"2024-04-20T21:17:54.681496+0000","last_peered":"2024-04-20T21:17:54.681496+0000","last_clean":"2024-04-20T21:17:54.681496+0000","last_became_active":"2024-04-20T21:17:51.273036+0000","last_became_peered":"2024-04-20T21:17:51.273036+0000","last_unstale":"2024-04-20T21:17:54.681496+0000","last_undegraded":"2024-04-20T21:17:54.681496+0000","last_fullsized":"2024-04-20T21:17:54.681496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:07:01.697230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.682607+0000","last_change":"2024-04-20T21:17:52.300897+0000","last_active":"2024-04-20T21:17:54.682607+0000","last_peered":"2024-04-20T21:17:54.682607+0000","last_clean":"2024-04-20T21:17:54.682607+0000","last_became_active":"2024-04-20T21:17:52.300600+0000","last_became_peered":"2024-04-20T21:17:52.300600+0000","last_unstale":"2024-04-20T21:17:54.682607+0000","last_undegraded":"2024-04-20T21:17:54.682607+0000","last_fullsized":"2024-04-20T21:17:54.682607+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:11:50.878621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.357354+0000","last_change":"2024-04-20T21:17:51.276810+0000","last_active":"2024-04-20T21:17:54.357354+0000","last_peered":"2024-04-20T21:17:54.357354+0000","last_clean":"2024-04-20T21:17:54.357354+0000","last_became_active":"2024-04-20T21:17:51.276547+0000","last_became_peered":"2024-04-20T21:17:51.276547+0000","last_unstale":"2024-04-20T21:17:54.357354+0000","last_undegraded":"2024-04-20T21:17:54.357354+0000","last_fullsized":"2024-04-20T21:17:54.357354+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:04:47.881403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.281043+0000","last_change":"2024-04-20T21:17:52.299388+0000","last_active":"2024-04-20T21:17:53.281043+0000","last_peered":"2024-04-20T21:17:53.281043+0000","last_clean":"2024-04-20T21:17:53.281043+0000","last_became_active":"2024-04-20T21:17:52.298531+0000","last_became_peered":"2024-04-20T21:17:52.298531+0000","last_unstale":"2024-04-20T21:17:53.281043+0000","last_undegraded":"2024-04-20T21:17:53.281043+0000","last_fullsized":"2024-04-20T21:17:53.281043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:12:56.626507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.288661+0000","last_change":"2024-04-20T21:17:51.273343+0000","last_active":"2024-04-20T21:17:53.288661+0000","last_peered":"2024-04-20T21:17:53.288661+0000","last_clean":"2024-04-20T21:17:53.288661+0000","last_became_active":"2024-04-20T21:17:51.273018+0000","last_became_peered":"2024-04-20T21:17:51.273018+0000","last_unstale":"2024-04-20T21:17:53.288661+0000","last_undegraded":"2024-04-20T21:17:53.288661+0000","last_fullsized":"2024-04-20T21:17:53.288661+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:34:26.903632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.358664+0000","last_change":"2024-04-20T21:17:52.300438+0000","last_active":"2024-04-20T21:17:54.358664+0000","last_peered":"2024-04-20T21:17:54.358664+0000","last_clean":"2024-04-20T21:17:54.358664+0000","last_became_active":"2024-04-20T21:17:52.300082+0000","last_became_peered":"2024-04-20T21:17:52.300082+0000","last_unstale":"2024-04-20T21:17:54.358664+0000","last_undegraded":"2024-04-20T21:17:54.358664+0000","last_fullsized":"2024-04-20T21:17:54.358664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:23:58.112035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.358530+0000","last_change":"2024-04-20T21:17:51.272721+0000","last_active":"2024-04-20T21:17:54.358530+0000","last_peered":"2024-04-20T21:17:54.358530+0000","last_clean":"2024-04-20T21:17:54.358530+0000","last_became_active":"2024-04-20T21:17:51.272403+0000","last_became_peered":"2024-04-20T21:17:51.272403+0000","last_unstale":"2024-04-20T21:17:54.358530+0000","last_undegraded":"2024-04-20T21:17:54.358530+0000","last_fullsized":"2024-04-20T21:17:54.358530+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:09:01.474577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.684541+0000","last_change":"2024-04-20T21:17:52.316437+0000","last_active":"2024-04-20T21:17:54.684541+0000","last_peered":"2024-04-20T21:17:54.684541+0000","last_clean":"2024-04-20T21:17:54.684541+0000","last_became_active":"2024-04-20T21:17:52.316129+0000","last_became_peered":"2024-04-20T21:17:52.316129+0000","last_unstale":"2024-04-20T21:17:54.684541+0000","last_undegraded":"2024-04-20T21:17:54.684541+0000","last_fullsized":"2024-04-20T21:17:54.684541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:34:47.842666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.281424+0000","last_change":"2024-04-20T21:17:51.268776+0000","last_active":"2024-04-20T21:17:53.281424+0000","last_peered":"2024-04-20T21:17:53.281424+0000","last_clean":"2024-04-20T21:17:53.281424+0000","last_became_active":"2024-04-20T21:17:51.268468+0000","last_became_peered":"2024-04-20T21:17:51.268468+0000","last_unstale":"2024-04-20T21:17:53.281424+0000","last_undegraded":"2024-04-20T21:17:53.281424+0000","last_fullsized":"2024-04-20T21:17:53.281424+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:34:26.903632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.287471+0000","last_change":"2024-04-20T21:17:52.300758+0000","last_active":"2024-04-20T21:17:53.287471+0000","last_peered":"2024-04-20T21:17:53.287471+0000","last_clean":"2024-04-20T21:17:53.287471+0000","last_became_active":"2024-04-20T21:17:52.300476+0000","last_became_peered":"2024-04-20T21:17:52.300476+0000","last_unstale":"2024-04-20T21:17:53.287471+0000","last_undegraded":"2024-04-20T21:17:53.287471+0000","last_fullsized":"2024-04-20T21:17:53.287471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:51:58.727652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.288618+0000","last_change":"2024-04-20T21:17:51.270900+0000","last_active":"2024-04-20T21:17:53.288618+0000","last_peered":"2024-04-20T21:17:53.288618+0000","last_clean":"2024-04-20T21:17:53.288618+0000","last_became_active":"2024-04-20T21:17:51.270618+0000","last_became_peered":"2024-04-20T21:17:51.270618+0000","last_unstale":"2024-04-20T21:17:53.288618+0000","last_undegraded":"2024-04-20T21:17:53.288618+0000","last_fullsized":"2024-04-20T21:17:53.288618+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:04:47.881403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.606885+0000","last_change":"2024-04-20T21:17:52.313555+0000","last_active":"2024-04-20T21:17:53.606885+0000","last_peered":"2024-04-20T21:17:53.606885+0000","last_clean":"2024-04-20T21:17:53.606885+0000","last_became_active":"2024-04-20T21:17:52.312197+0000","last_became_peered":"2024-04-20T21:17:52.312197+0000","last_unstale":"2024-04-20T21:17:53.606885+0000","last_undegraded":"2024-04-20T21:17:53.606885+0000","last_fullsized":"2024-04-20T21:17:53.606885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:52:22.418925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.683925+0000","last_change":"2024-04-20T21:17:51.275126+0000","last_active":"2024-04-20T21:17:54.683925+0000","last_peered":"2024-04-20T21:17:54.683925+0000","last_clean":"2024-04-20T21:17:54.683925+0000","last_became_active":"2024-04-20T21:17:51.273060+0000","last_became_peered":"2024-04-20T21:17:51.273060+0000","last_unstale":"2024-04-20T21:17:54.683925+0000","last_undegraded":"2024-04-20T21:17:54.683925+0000","last_fullsized":"2024-04-20T21:17:54.683925+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:06:19.740358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.683147+0000","last_change":"2024-04-20T21:17:52.302899+0000","last_active":"2024-04-20T21:17:54.683147+0000","last_peered":"2024-04-20T21:17:54.683147+0000","last_clean":"2024-04-20T21:17:54.683147+0000","last_became_active":"2024-04-20T21:17:52.302606+0000","last_became_peered":"2024-04-20T21:17:52.302606+0000","last_unstale":"2024-04-20T21:17:54.683147+0000","last_undegraded":"2024-04-20T21:17:54.683147+0000","last_fullsized":"2024-04-20T21:17:54.683147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:13:52.887276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.357587+0000","last_change":"2024-04-20T21:17:51.278078+0000","last_active":"2024-04-20T21:17:54.357587+0000","last_peered":"2024-04-20T21:17:54.357587+0000","last_clean":"2024-04-20T21:17:54.357587+0000","last_became_active":"2024-04-20T21:17:51.277775+0000","last_became_peered":"2024-04-20T21:17:51.277775+0000","last_unstale":"2024-04-20T21:17:54.357587+0000","last_undegraded":"2024-04-20T21:17:54.357587+0000","last_fullsized":"2024-04-20T21:17:54.357587+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:23:42.473358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.607236+0000","last_change":"2024-04-20T21:17:52.295456+0000","last_active":"2024-04-20T21:17:53.607236+0000","last_peered":"2024-04-20T21:17:53.607236+0000","last_clean":"2024-04-20T21:17:53.607236+0000","last_became_active":"2024-04-20T21:17:52.294825+0000","last_became_peered":"2024-04-20T21:17:52.294825+0000","last_unstale":"2024-04-20T21:17:53.607236+0000","last_undegraded":"2024-04-20T21:17:53.607236+0000","last_fullsized":"2024-04-20T21:17:53.607236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:43:32.983640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.288028+0000","last_change":"2024-04-20T21:17:51.264742+0000","last_active":"2024-04-20T21:17:53.288028+0000","last_peered":"2024-04-20T21:17:53.288028+0000","last_clean":"2024-04-20T21:17:53.288028+0000","last_became_active":"2024-04-20T21:17:51.264009+0000","last_became_peered":"2024-04-20T21:17:51.264009+0000","last_unstale":"2024-04-20T21:17:53.288028+0000","last_undegraded":"2024-04-20T21:17:53.288028+0000","last_fullsized":"2024-04-20T21:17:53.288028+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:29:17.957905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.288331+0000","last_change":"2024-04-20T21:17:52.313950+0000","last_active":"2024-04-20T21:17:53.288331+0000","last_peered":"2024-04-20T21:17:53.288331+0000","last_clean":"2024-04-20T21:17:53.288331+0000","last_became_active":"2024-04-20T21:17:52.311961+0000","last_became_peered":"2024-04-20T21:17:52.311961+0000","last_unstale":"2024-04-20T21:17:53.288331+0000","last_undegraded":"2024-04-20T21:17:53.288331+0000","last_fullsized":"2024-04-20T21:17:53.288331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:12:56.626507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.678630+0000","last_change":"2024-04-20T21:17:51.254940+0000","last_active":"2024-04-20T21:17:54.678630+0000","last_peered":"2024-04-20T21:17:54.678630+0000","last_clean":"2024-04-20T21:17:54.678630+0000","last_became_active":"2024-04-20T21:17:51.254597+0000","last_became_peered":"2024-04-20T21:17:51.254597+0000","last_unstale":"2024-04-20T21:17:54.678630+0000","last_undegraded":"2024-04-20T21:17:54.678630+0000","last_fullsized":"2024-04-20T21:17:54.678630+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:43:48.697965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.606512+0000","last_change":"2024-04-20T21:17:52.303514+0000","last_active":"2024-04-20T21:17:53.606512+0000","last_peered":"2024-04-20T21:17:53.606512+0000","last_clean":"2024-04-20T21:17:53.606512+0000","last_became_active":"2024-04-20T21:17:52.303225+0000","last_became_peered":"2024-04-20T21:17:52.303225+0000","last_unstale":"2024-04-20T21:17:53.606512+0000","last_undegraded":"2024-04-20T21:17:53.606512+0000","last_fullsized":"2024-04-20T21:17:53.606512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:26:30.156714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.287716+0000","last_change":"2024-04-20T21:17:51.276458+0000","last_active":"2024-04-20T21:17:53.287716+0000","last_peered":"2024-04-20T21:17:53.287716+0000","last_clean":"2024-04-20T21:17:53.287716+0000","last_became_active":"2024-04-20T21:17:51.276033+0000","last_became_peered":"2024-04-20T21:17:51.276033+0000","last_unstale":"2024-04-20T21:17:53.287716+0000","last_undegraded":"2024-04-20T21:17:53.287716+0000","last_fullsized":"2024-04-20T21:17:53.287716+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:00:14.426522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.607498+0000","last_change":"2024-04-20T21:17:52.310874+0000","last_active":"2024-04-20T21:17:53.607498+0000","last_peered":"2024-04-20T21:17:53.607498+0000","last_clean":"2024-04-20T21:17:53.607498+0000","last_became_active":"2024-04-20T21:17:52.310612+0000","last_became_peered":"2024-04-20T21:17:52.310612+0000","last_unstale":"2024-04-20T21:17:53.607498+0000","last_undegraded":"2024-04-20T21:17:53.607498+0000","last_fullsized":"2024-04-20T21:17:53.607498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:59:58.666249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.679906+0000","last_change":"2024-04-20T21:17:51.266671+0000","last_active":"2024-04-20T21:17:54.679906+0000","last_peered":"2024-04-20T21:17:54.679906+0000","last_clean":"2024-04-20T21:17:54.679906+0000","last_became_active":"2024-04-20T21:17:51.264486+0000","last_became_peered":"2024-04-20T21:17:51.264486+0000","last_unstale":"2024-04-20T21:17:54.679906+0000","last_undegraded":"2024-04-20T21:17:54.679906+0000","last_fullsized":"2024-04-20T21:17:54.679906+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:53:16.864261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.607949+0000","last_change":"2024-04-20T21:17:52.309913+0000","last_active":"2024-04-20T21:17:53.607949+0000","last_peered":"2024-04-20T21:17:53.607949+0000","last_clean":"2024-04-20T21:17:53.607949+0000","last_became_active":"2024-04-20T21:17:52.309643+0000","last_became_peered":"2024-04-20T21:17:52.309643+0000","last_unstale":"2024-04-20T21:17:53.607949+0000","last_undegraded":"2024-04-20T21:17:53.607949+0000","last_fullsized":"2024-04-20T21:17:53.607949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:48:28.191286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.281734+0000","last_change":"2024-04-20T21:17:51.272265+0000","last_active":"2024-04-20T21:17:53.281734+0000","last_peered":"2024-04-20T21:17:53.281734+0000","last_clean":"2024-04-20T21:17:53.281734+0000","last_became_active":"2024-04-20T21:17:51.271937+0000","last_became_peered":"2024-04-20T21:17:51.271937+0000","last_unstale":"2024-04-20T21:17:53.281734+0000","last_undegraded":"2024-04-20T21:17:53.281734+0000","last_fullsized":"2024-04-20T21:17:53.281734+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:51:57.714172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.358372+0000","last_change":"2024-04-20T21:17:52.304344+0000","last_active":"2024-04-20T21:17:54.358372+0000","last_peered":"2024-04-20T21:17:54.358372+0000","last_clean":"2024-04-20T21:17:54.358372+0000","last_became_active":"2024-04-20T21:17:52.304006+0000","last_became_peered":"2024-04-20T21:17:52.304006+0000","last_unstale":"2024-04-20T21:17:54.358372+0000","last_undegraded":"2024-04-20T21:17:54.358372+0000","last_fullsized":"2024-04-20T21:17:54.358372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:59:58.666249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.679057+0000","last_change":"2024-04-20T21:17:51.272508+0000","last_active":"2024-04-20T21:17:54.679057+0000","last_peered":"2024-04-20T21:17:54.679057+0000","last_clean":"2024-04-20T21:17:54.679057+0000","last_became_active":"2024-04-20T21:17:51.272056+0000","last_became_peered":"2024-04-20T21:17:51.272056+0000","last_unstale":"2024-04-20T21:17:54.679057+0000","last_undegraded":"2024-04-20T21:17:54.679057+0000","last_fullsized":"2024-04-20T21:17:54.679057+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:06:19.740358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.606888+0000","last_change":"2024-04-20T21:17:52.308203+0000","last_active":"2024-04-20T21:17:53.606888+0000","last_peered":"2024-04-20T21:17:53.606888+0000","last_clean":"2024-04-20T21:17:53.606888+0000","last_became_active":"2024-04-20T21:17:52.307941+0000","last_became_peered":"2024-04-20T21:17:52.307941+0000","last_unstale":"2024-04-20T21:17:53.606888+0000","last_undegraded":"2024-04-20T21:17:53.606888+0000","last_fullsized":"2024-04-20T21:17:53.606888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:51:21.491774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.606871+0000","last_change":"2024-04-20T21:17:43.059637+0000","last_active":"2024-04-20T21:17:53.606871+0000","last_peered":"2024-04-20T21:17:53.606871+0000","last_clean":"2024-04-20T21:17:53.606871+0000","last_became_active":"2024-04-20T21:17:43.056207+0000","last_became_peered":"2024-04-20T21:17:43.056207+0000","last_unstale":"2024-04-20T21:17:53.606871+0000","last_undegraded":"2024-04-20T21:17:53.606871+0000","last_fullsized":"2024-04-20T21:17:53.606871+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:41.036290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:41.036290+0000","last_clean_scrub_stamp":"2024-04-20T21:17:41.036290+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:58:51.091337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.357115+0000","last_change":"2024-04-20T21:17:51.277973+0000","last_active":"2024-04-20T21:17:54.357115+0000","last_peered":"2024-04-20T21:17:54.357115+0000","last_clean":"2024-04-20T21:17:54.357115+0000","last_became_active":"2024-04-20T21:17:51.277643+0000","last_became_peered":"2024-04-20T21:17:51.277643+0000","last_unstale":"2024-04-20T21:17:54.357115+0000","last_undegraded":"2024-04-20T21:17:54.357115+0000","last_fullsized":"2024-04-20T21:17:54.357115+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:41:08.560538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.683244+0000","last_change":"2024-04-20T21:17:52.316313+0000","last_active":"2024-04-20T21:17:54.683244+0000","last_peered":"2024-04-20T21:17:54.683244+0000","last_clean":"2024-04-20T21:17:54.683244+0000","last_became_active":"2024-04-20T21:17:52.316054+0000","last_became_peered":"2024-04-20T21:17:52.316054+0000","last_unstale":"2024-04-20T21:17:54.683244+0000","last_undegraded":"2024-04-20T21:17:54.683244+0000","last_fullsized":"2024-04-20T21:17:54.683244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:26:30.156714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.606138+0000","last_change":"2024-04-20T21:17:51.282977+0000","last_active":"2024-04-20T21:17:53.606138+0000","last_peered":"2024-04-20T21:17:53.606138+0000","last_clean":"2024-04-20T21:17:53.606138+0000","last_became_active":"2024-04-20T21:17:51.282699+0000","last_became_peered":"2024-04-20T21:17:51.282699+0000","last_unstale":"2024-04-20T21:17:53.606138+0000","last_undegraded":"2024-04-20T21:17:53.606138+0000","last_fullsized":"2024-04-20T21:17:53.606138+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:04:47.881403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.682600+0000","last_change":"2024-04-20T21:17:52.301574+0000","last_active":"2024-04-20T21:17:54.682600+0000","last_peered":"2024-04-20T21:17:54.682600+0000","last_clean":"2024-04-20T21:17:54.682600+0000","last_became_active":"2024-04-20T21:17:52.299834+0000","last_became_peered":"2024-04-20T21:17:52.299834+0000","last_unstale":"2024-04-20T21:17:54.682600+0000","last_undegraded":"2024-04-20T21:17:54.682600+0000","last_fullsized":"2024-04-20T21:17:54.682600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:23:43.486837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.679134+0000","last_change":"2024-04-20T21:17:51.268326+0000","last_active":"2024-04-20T21:17:54.679134+0000","last_peered":"2024-04-20T21:17:54.679134+0000","last_clean":"2024-04-20T21:17:54.679134+0000","last_became_active":"2024-04-20T21:17:51.268016+0000","last_became_peered":"2024-04-20T21:17:51.268016+0000","last_unstale":"2024-04-20T21:17:54.679134+0000","last_undegraded":"2024-04-20T21:17:54.679134+0000","last_fullsized":"2024-04-20T21:17:54.679134+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:18:39.465785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.280844+0000","last_change":"2024-04-20T21:17:52.289513+0000","last_active":"2024-04-20T21:17:53.280844+0000","last_peered":"2024-04-20T21:17:53.280844+0000","last_clean":"2024-04-20T21:17:53.280844+0000","last_became_active":"2024-04-20T21:17:52.288677+0000","last_became_peered":"2024-04-20T21:17:52.288677+0000","last_unstale":"2024-04-20T21:17:53.280844+0000","last_undegraded":"2024-04-20T21:17:53.280844+0000","last_fullsized":"2024-04-20T21:17:53.280844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:52:26.845286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.358151+0000","last_change":"2024-04-20T21:17:51.275850+0000","last_active":"2024-04-20T21:17:54.358151+0000","last_peered":"2024-04-20T21:17:54.358151+0000","last_clean":"2024-04-20T21:17:54.358151+0000","last_became_active":"2024-04-20T21:17:51.275350+0000","last_became_peered":"2024-04-20T21:17:51.275350+0000","last_unstale":"2024-04-20T21:17:54.358151+0000","last_undegraded":"2024-04-20T21:17:54.358151+0000","last_fullsized":"2024-04-20T21:17:54.358151+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:38:55.545413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.287199+0000","last_change":"2024-04-20T21:17:52.305922+0000","last_active":"2024-04-20T21:17:53.287199+0000","last_peered":"2024-04-20T21:17:53.287199+0000","last_clean":"2024-04-20T21:17:53.287199+0000","last_became_active":"2024-04-20T21:17:52.304705+0000","last_became_peered":"2024-04-20T21:17:52.304705+0000","last_unstale":"2024-04-20T21:17:53.287199+0000","last_undegraded":"2024-04-20T21:17:53.287199+0000","last_fullsized":"2024-04-20T21:17:53.287199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:04:48.894882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.281179+0000","last_change":"2024-04-20T21:17:51.260375+0000","last_active":"2024-04-20T21:17:53.281179+0000","last_peered":"2024-04-20T21:17:53.281179+0000","last_clean":"2024-04-20T21:17:53.281179+0000","last_became_active":"2024-04-20T21:17:51.259993+0000","last_became_peered":"2024-04-20T21:17:51.259993+0000","last_unstale":"2024-04-20T21:17:53.281179+0000","last_undegraded":"2024-04-20T21:17:53.281179+0000","last_fullsized":"2024-04-20T21:17:53.281179+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:04:47.881403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.358295+0000","last_change":"2024-04-20T21:17:52.308697+0000","last_active":"2024-04-20T21:17:54.358295+0000","last_peered":"2024-04-20T21:17:54.358295+0000","last_clean":"2024-04-20T21:17:54.358295+0000","last_became_active":"2024-04-20T21:17:52.308037+0000","last_became_peered":"2024-04-20T21:17:52.308037+0000","last_unstale":"2024-04-20T21:17:54.358295+0000","last_undegraded":"2024-04-20T21:17:54.358295+0000","last_fullsized":"2024-04-20T21:17:54.358295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:34:46.630106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.607409+0000","last_change":"2024-04-20T21:17:51.283426+0000","last_active":"2024-04-20T21:17:53.607409+0000","last_peered":"2024-04-20T21:17:53.607409+0000","last_clean":"2024-04-20T21:17:53.607409+0000","last_became_active":"2024-04-20T21:17:51.283162+0000","last_became_peered":"2024-04-20T21:17:51.283162+0000","last_unstale":"2024-04-20T21:17:53.607409+0000","last_undegraded":"2024-04-20T21:17:53.607409+0000","last_fullsized":"2024-04-20T21:17:53.607409+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:18:31.872106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.281429+0000","last_change":"2024-04-20T21:17:52.285628+0000","last_active":"2024-04-20T21:17:53.281429+0000","last_peered":"2024-04-20T21:17:53.281429+0000","last_clean":"2024-04-20T21:17:53.281429+0000","last_became_active":"2024-04-20T21:17:52.284918+0000","last_became_peered":"2024-04-20T21:17:52.284918+0000","last_unstale":"2024-04-20T21:17:53.281429+0000","last_undegraded":"2024-04-20T21:17:53.281429+0000","last_fullsized":"2024-04-20T21:17:53.281429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:42:57.243810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.281782+0000","last_change":"2024-04-20T21:17:51.271506+0000","last_active":"2024-04-20T21:17:53.281782+0000","last_peered":"2024-04-20T21:17:53.281782+0000","last_clean":"2024-04-20T21:17:53.281782+0000","last_became_active":"2024-04-20T21:17:51.271243+0000","last_became_peered":"2024-04-20T21:17:51.271243+0000","last_unstale":"2024-04-20T21:17:53.281782+0000","last_undegraded":"2024-04-20T21:17:53.281782+0000","last_fullsized":"2024-04-20T21:17:53.281782+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:29:17.957905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.684078+0000","last_change":"2024-04-20T21:17:52.301465+0000","last_active":"2024-04-20T21:17:54.684078+0000","last_peered":"2024-04-20T21:17:54.684078+0000","last_clean":"2024-04-20T21:17:54.684078+0000","last_became_active":"2024-04-20T21:17:52.300849+0000","last_became_peered":"2024-04-20T21:17:52.300849+0000","last_unstale":"2024-04-20T21:17:54.684078+0000","last_undegraded":"2024-04-20T21:17:54.684078+0000","last_fullsized":"2024-04-20T21:17:54.684078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:04:26.111984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.288907+0000","last_change":"2024-04-20T21:17:51.282509+0000","last_active":"2024-04-20T21:17:53.288907+0000","last_peered":"2024-04-20T21:17:53.288907+0000","last_clean":"2024-04-20T21:17:53.288907+0000","last_became_active":"2024-04-20T21:17:51.281343+0000","last_became_peered":"2024-04-20T21:17:51.281343+0000","last_unstale":"2024-04-20T21:17:53.288907+0000","last_undegraded":"2024-04-20T21:17:53.288907+0000","last_fullsized":"2024-04-20T21:17:53.288907+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:43:48.697965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.684280+0000","last_change":"2024-04-20T21:17:52.311617+0000","last_active":"2024-04-20T21:17:54.684280+0000","last_peered":"2024-04-20T21:17:54.684280+0000","last_clean":"2024-04-20T21:17:54.684280+0000","last_became_active":"2024-04-20T21:17:52.311367+0000","last_became_peered":"2024-04-20T21:17:52.311367+0000","last_unstale":"2024-04-20T21:17:54.684280+0000","last_undegraded":"2024-04-20T21:17:54.684280+0000","last_fullsized":"2024-04-20T21:17:54.684280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:27:15.675834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.607489+0000","last_change":"2024-04-20T21:17:51.283039+0000","last_active":"2024-04-20T21:17:53.607489+0000","last_peered":"2024-04-20T21:17:53.607489+0000","last_clean":"2024-04-20T21:17:53.607489+0000","last_became_active":"2024-04-20T21:17:51.282780+0000","last_became_peered":"2024-04-20T21:17:51.282780+0000","last_unstale":"2024-04-20T21:17:53.607489+0000","last_undegraded":"2024-04-20T21:17:53.607489+0000","last_fullsized":"2024-04-20T21:17:53.607489+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:18:39.465785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.286949+0000","last_change":"2024-04-20T21:17:52.293046+0000","last_active":"2024-04-20T21:17:53.286949+0000","last_peered":"2024-04-20T21:17:53.286949+0000","last_clean":"2024-04-20T21:17:53.286949+0000","last_became_active":"2024-04-20T21:17:52.288365+0000","last_became_peered":"2024-04-20T21:17:52.288365+0000","last_unstale":"2024-04-20T21:17:53.286949+0000","last_undegraded":"2024-04-20T21:17:53.286949+0000","last_fullsized":"2024-04-20T21:17:53.286949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:18:40.479264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.281228+0000","last_change":"2024-04-20T21:17:51.280889+0000","last_active":"2024-04-20T21:17:53.281228+0000","last_peered":"2024-04-20T21:17:53.281228+0000","last_clean":"2024-04-20T21:17:53.281228+0000","last_became_active":"2024-04-20T21:17:51.280582+0000","last_became_peered":"2024-04-20T21:17:51.280582+0000","last_unstale":"2024-04-20T21:17:53.281228+0000","last_undegraded":"2024-04-20T21:17:53.281228+0000","last_fullsized":"2024-04-20T21:17:53.281228+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:18:39.465785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.682757+0000","last_change":"2024-04-20T21:17:52.292459+0000","last_active":"2024-04-20T21:17:54.682757+0000","last_peered":"2024-04-20T21:17:54.682757+0000","last_clean":"2024-04-20T21:17:54.682757+0000","last_became_active":"2024-04-20T21:17:52.292182+0000","last_became_peered":"2024-04-20T21:17:52.292182+0000","last_unstale":"2024-04-20T21:17:54.682757+0000","last_undegraded":"2024-04-20T21:17:54.682757+0000","last_fullsized":"2024-04-20T21:17:54.682757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:34:47.842666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.281162+0000","last_change":"2024-04-20T21:17:51.269365+0000","last_active":"2024-04-20T21:17:53.281162+0000","last_peered":"2024-04-20T21:17:53.281162+0000","last_clean":"2024-04-20T21:17:53.281162+0000","last_became_active":"2024-04-20T21:17:51.267902+0000","last_became_peered":"2024-04-20T21:17:51.267902+0000","last_unstale":"2024-04-20T21:17:53.281162+0000","last_undegraded":"2024-04-20T21:17:53.281162+0000","last_fullsized":"2024-04-20T21:17:53.281162+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:43:48.697965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.681157+0000","last_change":"2024-04-20T21:17:52.315096+0000","last_active":"2024-04-20T21:17:54.681157+0000","last_peered":"2024-04-20T21:17:54.681157+0000","last_clean":"2024-04-20T21:17:54.681157+0000","last_became_active":"2024-04-20T21:17:52.314742+0000","last_became_peered":"2024-04-20T21:17:52.314742+0000","last_unstale":"2024-04-20T21:17:54.681157+0000","last_undegraded":"2024-04-20T21:17:54.681157+0000","last_fullsized":"2024-04-20T21:17:54.681157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:52:26.845286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.607525+0000","last_change":"2024-04-20T21:17:51.288583+0000","last_active":"2024-04-20T21:17:53.607525+0000","last_peered":"2024-04-20T21:17:53.607525+0000","last_clean":"2024-04-20T21:17:53.607525+0000","last_became_active":"2024-04-20T21:17:51.288299+0000","last_became_peered":"2024-04-20T21:17:51.288299+0000","last_unstale":"2024-04-20T21:17:53.607525+0000","last_undegraded":"2024-04-20T21:17:53.607525+0000","last_fullsized":"2024-04-20T21:17:53.607525+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:29:17.957905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.288285+0000","last_change":"2024-04-20T21:17:52.283186+0000","last_active":"2024-04-20T21:17:53.288285+0000","last_peered":"2024-04-20T21:17:53.288285+0000","last_clean":"2024-04-20T21:17:53.288285+0000","last_became_active":"2024-04-20T21:17:52.280776+0000","last_became_peered":"2024-04-20T21:17:52.280776+0000","last_unstale":"2024-04-20T21:17:53.288285+0000","last_undegraded":"2024-04-20T21:17:53.288285+0000","last_fullsized":"2024-04-20T21:17:53.288285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:52:26.845286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.358061+0000","last_change":"2024-04-20T21:17:52.302224+0000","last_active":"2024-04-20T21:17:54.358061+0000","last_peered":"2024-04-20T21:17:54.358061+0000","last_clean":"2024-04-20T21:17:54.358061+0000","last_became_active":"2024-04-20T21:17:52.301869+0000","last_became_peered":"2024-04-20T21:17:52.301869+0000","last_unstale":"2024-04-20T21:17:54.358061+0000","last_undegraded":"2024-04-20T21:17:54.358061+0000","last_fullsized":"2024-04-20T21:17:54.358061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:43:32.983640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.606747+0000","last_change":"2024-04-20T21:17:51.288486+0000","last_active":"2024-04-20T21:17:53.606747+0000","last_peered":"2024-04-20T21:17:53.606747+0000","last_clean":"2024-04-20T21:17:53.606747+0000","last_became_active":"2024-04-20T21:17:51.288131+0000","last_became_peered":"2024-04-20T21:17:51.288131+0000","last_unstale":"2024-04-20T21:17:53.606747+0000","last_undegraded":"2024-04-20T21:17:53.606747+0000","last_fullsized":"2024-04-20T21:17:53.606747+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:51:57.714172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.357879+0000","last_change":"2024-04-20T21:17:52.302297+0000","last_active":"2024-04-20T21:17:54.357879+0000","last_peered":"2024-04-20T21:17:54.357879+0000","last_clean":"2024-04-20T21:17:54.357879+0000","last_became_active":"2024-04-20T21:17:52.301982+0000","last_became_peered":"2024-04-20T21:17:52.301982+0000","last_unstale":"2024-04-20T21:17:54.357879+0000","last_undegraded":"2024-04-20T21:17:54.357879+0000","last_fullsized":"2024-04-20T21:17:54.357879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:51:21.491774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.607170+0000","last_change":"2024-04-20T21:17:51.285151+0000","last_active":"2024-04-20T21:17:53.607170+0000","last_peered":"2024-04-20T21:17:53.607170+0000","last_clean":"2024-04-20T21:17:53.607170+0000","last_became_active":"2024-04-20T21:17:51.284879+0000","last_became_peered":"2024-04-20T21:17:51.284879+0000","last_unstale":"2024-04-20T21:17:53.607170+0000","last_undegraded":"2024-04-20T21:17:53.607170+0000","last_fullsized":"2024-04-20T21:17:53.607170+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:34:26.903632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.684538+0000","last_change":"2024-04-20T21:17:52.317666+0000","last_active":"2024-04-20T21:17:54.684538+0000","last_peered":"2024-04-20T21:17:54.684538+0000","last_clean":"2024-04-20T21:17:54.684538+0000","last_became_active":"2024-04-20T21:17:52.316819+0000","last_became_peered":"2024-04-20T21:17:52.316819+0000","last_unstale":"2024-04-20T21:17:54.684538+0000","last_undegraded":"2024-04-20T21:17:54.684538+0000","last_fullsized":"2024-04-20T21:17:54.684538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:11:50.878621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.682331+0000","last_change":"2024-04-20T21:17:51.276188+0000","last_active":"2024-04-20T21:17:54.682331+0000","last_peered":"2024-04-20T21:17:54.682331+0000","last_clean":"2024-04-20T21:17:54.682331+0000","last_became_active":"2024-04-20T21:17:51.275862+0000","last_became_peered":"2024-04-20T21:17:51.275862+0000","last_unstale":"2024-04-20T21:17:54.682331+0000","last_undegraded":"2024-04-20T21:17:54.682331+0000","last_fullsized":"2024-04-20T21:17:54.682331+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:18:39.465785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.684488+0000","last_change":"2024-04-20T21:17:52.310311+0000","last_active":"2024-04-20T21:17:54.684488+0000","last_peered":"2024-04-20T21:17:54.684488+0000","last_clean":"2024-04-20T21:17:54.684488+0000","last_became_active":"2024-04-20T21:17:52.307410+0000","last_became_peered":"2024-04-20T21:17:52.307410+0000","last_unstale":"2024-04-20T21:17:54.684488+0000","last_undegraded":"2024-04-20T21:17:54.684488+0000","last_fullsized":"2024-04-20T21:17:54.684488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:52:22.418925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.684750+0000","last_change":"2024-04-20T21:17:51.287639+0000","last_active":"2024-04-20T21:17:54.684750+0000","last_peered":"2024-04-20T21:17:54.684750+0000","last_clean":"2024-04-20T21:17:54.684750+0000","last_became_active":"2024-04-20T21:17:51.287421+0000","last_became_peered":"2024-04-20T21:17:51.287421+0000","last_unstale":"2024-04-20T21:17:54.684750+0000","last_undegraded":"2024-04-20T21:17:54.684750+0000","last_fullsized":"2024-04-20T21:17:54.684750+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:07:01.697230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T21:17:56.868 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T21:17:57.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.101+0000 7f9ac3200640 1 Processor -- start 2024-04-20T21:17:57.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.101+0000 7f9ac3200640 1 -- start start 2024-04-20T21:17:57.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.103+0000 7f9ac3200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ac415d6a0 0x7f9ac40ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:57.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.103+0000 7f9ac3200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ac4160890 con 0x7f9ac40ad620 2024-04-20T21:17:57.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.103+0000 7f9ac3200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ac4160b70 con 0x7f9ac4156ae0 2024-04-20T21:17:57.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.103+0000 7f9ac3200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9ac406db20 con 0x7f9ac415d6a0 2024-04-20T21:17:57.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.103+0000 7f9ac2800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f9ac40ad620 0x7f9ac406e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:43470/0 (socket says 172.21.15.86:43470) 2024-04-20T21:17:57.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.103+0000 7f9ac2800640 1 -- 172.21.15.86:0/2939418182 learned_addr learned my addr 172.21.15.86:0/2939418182 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:17:57.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.103+0000 7f9ac1400640 1 --2- 172.21.15.86:0/2939418182 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ac415d6a0 0x7f9ac40ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:57.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.103+0000 7f9ac0a00640 1 -- 172.21.15.86:0/2939418182 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3430721280 0 0) 0x7f9ac4160890 con 0x7f9ac40ad620 2024-04-20T21:17:57.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.103+0000 7f9ac0a00640 1 -- 172.21.15.86:0/2939418182 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9aa8003520 con 0x7f9ac40ad620 2024-04-20T21:17:57.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.104+0000 7f9ac0a00640 1 -- 172.21.15.86:0/2939418182 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 686161628 0 0) 0x7f9aa8003520 con 0x7f9ac40ad620 2024-04-20T21:17:57.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.104+0000 7f9ac0a00640 1 -- 172.21.15.86:0/2939418182 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ac415d6a0 msgr2=0x7f9ac40ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:57.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.104+0000 7f9ac0a00640 1 --2- 172.21.15.86:0/2939418182 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ac415d6a0 0x7f9ac40ad0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:57.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.104+0000 7f9ac0a00640 1 -- 172.21.15.86:0/2939418182 >> v1:172.21.15.144:6789/0 conn(0x7f9ac4156ae0 legacy=0x7f9ac4160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:57.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.104+0000 7f9ac0a00640 1 -- 172.21.15.86:0/2939418182 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ac4160e50 con 0x7f9ac40ad620 2024-04-20T21:17:57.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.104+0000 7f9ac0a00640 1 -- 172.21.15.86:0/2939418182 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f9abc0019f0 con 0x7f9ac40ad620 2024-04-20T21:17:57.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.104+0000 7f9ac0a00640 1 -- 172.21.15.86:0/2939418182 >> v1:172.21.15.86:6789/0 conn(0x7f9ac40ad620 legacy=0x7f9ac406e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:57.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.104+0000 7f9ac0a00640 1 --2- 172.21.15.86:0/2939418182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9aa80039e0 0x7f9aa8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:57.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.104+0000 7f9ac0a00640 1 --2- 172.21.15.86:0/2939418182 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ac415d6a0 0x7f9aa8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:57.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.104+0000 7f9ac0a00640 1 --2- 172.21.15.86:0/2939418182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9aa8004d40 0x7f9aa80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:57.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.104+0000 7f9ac0a00640 1 -- 172.21.15.86:0/2939418182 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac406db20 con 0x7f9aa80039e0 2024-04-20T21:17:57.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.104+0000 7f9ac0a00640 1 -- 172.21.15.86:0/2939418182 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa800ba70 con 0x7f9aa8004d40 2024-04-20T21:17:57.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.104+0000 7f9ac0a00640 1 -- 172.21.15.86:0/2939418182 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9aa800bc70 con 0x7f9ac415d6a0 2024-04-20T21:17:57.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.104+0000 7f9ac1e00640 1 --2- 172.21.15.86:0/2939418182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9aa80039e0 0x7f9aa8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:57.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.105+0000 7f9ac1400640 1 --2- 172.21.15.86:0/2939418182 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ac415d6a0 0x7f9aa8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:57.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.105+0000 7f9ac1e00640 1 -- 172.21.15.86:0/2939418182 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ac415d6a0 msgr2=0x7f9aa8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:57.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.105+0000 7f9ac2800640 1 --2- 172.21.15.86:0/2939418182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9aa8004d40 0x7f9aa80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:57.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.105+0000 7f9ac1e00640 1 --2- 172.21.15.86:0/2939418182 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ac415d6a0 0x7f9aa8004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:57.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.105+0000 7f9ac1e00640 1 -- 172.21.15.86:0/2939418182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9aa8004d40 msgr2=0x7f9aa80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:57.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.105+0000 7f9ac1e00640 1 --2- 172.21.15.86:0/2939418182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9aa8004d40 0x7f9aa80051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:57.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.105+0000 7f9ac1e00640 1 -- 172.21.15.86:0/2939418182 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aa8003520 con 0x7f9aa80039e0 2024-04-20T21:17:57.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.106+0000 7f9ac1e00640 1 --2- 172.21.15.86:0/2939418182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9aa80039e0 0x7f9aa8003e00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9ab4001720 tx=0x7f9aa8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b407533fc9d334a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:57.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.106+0000 7f9ac0a00640 1 -- 172.21.15.86:0/2939418182 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab4001370 con 0x7f9aa80039e0 2024-04-20T21:17:57.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.106+0000 7f9ac0a00640 1 -- 172.21.15.86:0/2939418182 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ab40052d0 con 0x7f9aa80039e0 2024-04-20T21:17:57.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.106+0000 7f9ac0a00640 1 -- 172.21.15.86:0/2939418182 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab4003830 con 0x7f9aa80039e0 2024-04-20T21:17:57.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.106+0000 7f9ac3200640 1 -- 172.21.15.86:0/2939418182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9aa80039e0 msgr2=0x7f9aa8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:57.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.106+0000 7f9ac3200640 1 --2- 172.21.15.86:0/2939418182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9aa80039e0 0x7f9aa8003e00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9ab4001720 tx=0x7f9aa8005830 comp rx=0 tx=0).stop 2024-04-20T21:17:57.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.106+0000 7f9ac2800640 1 -- 172.21.15.86:0/2939418182 reap_dead start 2024-04-20T21:17:57.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.106+0000 7f9ac3200640 1 -- 172.21.15.86:0/2939418182 shutdown_connections 2024-04-20T21:17:57.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.106+0000 7f9ac3200640 1 -- 172.21.15.86:0/2939418182 >> 172.21.15.86:0/2939418182 conn(0x7f9ac40d0170 msgr2=0x7f9ac40d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:57.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.106+0000 7f9ac3200640 1 -- 172.21.15.86:0/2939418182 shutdown_connections 2024-04-20T21:17:57.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.107+0000 7f9ac3200640 1 -- 172.21.15.86:0/2939418182 wait complete. 2024-04-20T21:17:57.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.108+0000 7f9ac3200640 1 Processor -- start 2024-04-20T21:17:57.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.108+0000 7f9ac3200640 1 -- start start 2024-04-20T21:17:57.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.109+0000 7f9ac3200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9aa80039e0 0x7f9ac41e5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:57.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.109+0000 7f9ac3200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ac4145c40 0x7f9ac41e6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:57.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.109+0000 7f9ac3200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ac40b10f0 0x7f9ac41fd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:57.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.109+0000 7f9ac3200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac40ad0e0 con 0x7f9ac4145c40 2024-04-20T21:17:57.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.109+0000 7f9ac3200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac4156ec0 con 0x7f9aa80039e0 2024-04-20T21:17:57.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.109+0000 7f9ac3200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9ac40ad260 con 0x7f9ac40b10f0 2024-04-20T21:17:57.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.109+0000 7f9ac1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ac4145c40 0x7f9ac41e6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:57.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.109+0000 7f9ac1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ac4145c40 0x7f9ac41e6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57592/0 (socket says 172.21.15.86:57592) 2024-04-20T21:17:57.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.109+0000 7f9ac1400640 1 -- 172.21.15.86:0/2211981147 learned_addr learned my addr 172.21.15.86:0/2211981147 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:57.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.110+0000 7f9ac1e00640 1 --2- 172.21.15.86:0/2211981147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9aa80039e0 0x7f9ac41e5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:57.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.110+0000 7f9ac1400640 1 -- 172.21.15.86:0/2211981147 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ac40b10f0 msgr2=0x7f9ac41fd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:57.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.110+0000 7f9ac2800640 1 --2- 172.21.15.86:0/2211981147 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ac40b10f0 0x7f9ac41fd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:57.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.110+0000 7f9ac1400640 1 --2- 172.21.15.86:0/2211981147 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ac40b10f0 0x7f9ac41fd1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:57.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.110+0000 7f9ac1400640 1 -- 172.21.15.86:0/2211981147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9aa80039e0 msgr2=0x7f9ac41e5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:57.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.110+0000 7f9ac1400640 1 --2- 172.21.15.86:0/2211981147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9aa80039e0 0x7f9ac41e5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:57.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.110+0000 7f9ac1400640 1 -- 172.21.15.86:0/2211981147 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ac4157040 con 0x7f9ac4145c40 2024-04-20T21:17:57.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.110+0000 7f9ac1400640 1 --2- 172.21.15.86:0/2211981147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ac4145c40 0x7f9ac41e6380 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9ab00108c0 tx=0x7f9ab0010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:57.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.110+0000 7f9aba000640 1 -- 172.21.15.86:0/2211981147 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab000f040 con 0x7f9ac4145c40 2024-04-20T21:17:57.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.110+0000 7f9aba000640 1 -- 172.21.15.86:0/2211981147 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ab000eb60 con 0x7f9ac4145c40 2024-04-20T21:17:57.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.110+0000 7f9aba000640 1 -- 172.21.15.86:0/2211981147 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab001c400 con 0x7f9ac4145c40 2024-04-20T21:17:57.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.110+0000 7f9ac3200640 1 -- 172.21.15.86:0/2211981147 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ac4073b80 con 0x7f9ac4145c40 2024-04-20T21:17:57.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.111+0000 7f9ac3200640 1 -- 172.21.15.86:0/2211981147 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ac415f860 con 0x7f9ac4145c40 2024-04-20T21:17:57.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.112+0000 7f9ac3200640 1 -- 172.21.15.86:0/2211981147 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ac404fcf0 con 0x7f9ac4145c40 2024-04-20T21:17:57.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.113+0000 7f9aba000640 1 -- 172.21.15.86:0/2211981147 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9ab001c5a0 con 0x7f9ac4145c40 2024-04-20T21:17:57.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.114+0000 7f9aba000640 1 --2- 172.21.15.86:0/2211981147 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9aa406c8f0 0x7f9aa406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:57.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.114+0000 7f9aba000640 1 -- 172.21.15.86:0/2211981147 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9ab008c0e0 con 0x7f9ac4145c40 2024-04-20T21:17:57.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.114+0000 7f9ac1e00640 1 --2- 172.21.15.86:0/2211981147 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9aa406c8f0 0x7f9aa406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:57.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.114+0000 7f9ac1e00640 1 --2- 172.21.15.86:0/2211981147 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9aa406c8f0 0x7f9aa406edb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9ab4033000 tx=0x7f9ab4009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:57.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.118+0000 7f9aba000640 1 -- 172.21.15.86:0/2211981147 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9ac4073b80 con 0x7f9ac4145c40 2024-04-20T21:17:57.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.299+0000 7f9ac3200640 1 -- 172.21.15.86:0/2211981147 --> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9ac4159f60 con 0x7f9aa406c8f0 2024-04-20T21:17:57.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.317+0000 7f9aba000640 1 -- 172.21.15.86:0/2211981147 <== mgr.4107 v2:172.21.15.86:6832/302045240 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251948 (secure 0 0 0) 0x7f9ac4159f60 con 0x7f9aa406c8f0 2024-04-20T21:17:57.320 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:17:57.321 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-20T21:17:57.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.322+0000 7f9ac3200640 1 -- 172.21.15.86:0/2211981147 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9aa406c8f0 msgr2=0x7f9aa406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:57.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.322+0000 7f9ac3200640 1 --2- 172.21.15.86:0/2211981147 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9aa406c8f0 0x7f9aa406edb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9ab4033000 tx=0x7f9ab4009030 comp rx=0 tx=0).stop 2024-04-20T21:17:57.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.322+0000 7f9ac3200640 1 -- 172.21.15.86:0/2211981147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ac4145c40 msgr2=0x7f9ac41e6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:57.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.322+0000 7f9ac3200640 1 --2- 172.21.15.86:0/2211981147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ac4145c40 0x7f9ac41e6380 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9ab00108c0 tx=0x7f9ab0010d80 comp rx=0 tx=0).stop 2024-04-20T21:17:57.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.322+0000 7f9ac3200640 1 -- 172.21.15.86:0/2211981147 shutdown_connections 2024-04-20T21:17:57.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.322+0000 7f9ac3200640 1 --2- 172.21.15.86:0/2211981147 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9aa406c8f0 0x7f9aa406edb0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:57.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.322+0000 7f9ac3200640 1 --2- 172.21.15.86:0/2211981147 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ac40b10f0 0x7f9ac41fd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:57.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.322+0000 7f9ac3200640 1 --2- 172.21.15.86:0/2211981147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ac4145c40 0x7f9ac41e6380 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:57.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.322+0000 7f9ac3200640 1 --2- 172.21.15.86:0/2211981147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9aa80039e0 0x7f9ac41e5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:57.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.322+0000 7f9ac3200640 1 -- 172.21.15.86:0/2211981147 >> 172.21.15.86:0/2211981147 conn(0x7f9ac40d0170 msgr2=0x7f9ac406e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:57.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.322+0000 7f9ac3200640 1 -- 172.21.15.86:0/2211981147 shutdown_connections 2024-04-20T21:17:57.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.323+0000 7f9ac3200640 1 -- 172.21.15.86:0/2211981147 wait complete. 2024-04-20T21:17:57.340 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-20T21:17:56.631964+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":180,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219000,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755720,"statfs":{"total":773094113280,"available":772869857280,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001720"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.607719+0000","last_change":"2024-04-20T21:17:52.309045+0000","last_active":"2024-04-20T21:17:53.607719+0000","last_peered":"2024-04-20T21:17:53.607719+0000","last_clean":"2024-04-20T21:17:53.607719+0000","last_became_active":"2024-04-20T21:17:52.308808+0000","last_became_peered":"2024-04-20T21:17:52.308808+0000","last_unstale":"2024-04-20T21:17:53.607719+0000","last_undegraded":"2024-04-20T21:17:53.607719+0000","last_fullsized":"2024-04-20T21:17:53.607719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:15:54.762913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.281640+0000","last_change":"2024-04-20T21:17:52.303716+0000","last_active":"2024-04-20T21:17:53.281640+0000","last_peered":"2024-04-20T21:17:53.281640+0000","last_clean":"2024-04-20T21:17:53.281640+0000","last_became_active":"2024-04-20T21:17:52.303460+0000","last_became_peered":"2024-04-20T21:17:52.303460+0000","last_unstale":"2024-04-20T21:17:53.281640+0000","last_undegraded":"2024-04-20T21:17:53.281640+0000","last_fullsized":"2024-04-20T21:17:53.281640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:34:47.842666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.684324+0000","last_change":"2024-04-20T21:17:52.297807+0000","last_active":"2024-04-20T21:17:54.684324+0000","last_peered":"2024-04-20T21:17:54.684324+0000","last_clean":"2024-04-20T21:17:54.684324+0000","last_became_active":"2024-04-20T21:17:52.296178+0000","last_became_peered":"2024-04-20T21:17:52.296178+0000","last_unstale":"2024-04-20T21:17:54.684324+0000","last_undegraded":"2024-04-20T21:17:54.684324+0000","last_fullsized":"2024-04-20T21:17:54.684324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:42:57.243810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.357862+0000","last_change":"2024-04-20T21:17:52.315284+0000","last_active":"2024-04-20T21:17:54.357862+0000","last_peered":"2024-04-20T21:17:54.357862+0000","last_clean":"2024-04-20T21:17:54.357862+0000","last_became_active":"2024-04-20T21:17:52.314657+0000","last_became_peered":"2024-04-20T21:17:52.314657+0000","last_unstale":"2024-04-20T21:17:54.357862+0000","last_undegraded":"2024-04-20T21:17:54.357862+0000","last_fullsized":"2024-04-20T21:17:54.357862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:52:22.418925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.606807+0000","last_change":"2024-04-20T21:17:52.299456+0000","last_active":"2024-04-20T21:17:53.606807+0000","last_peered":"2024-04-20T21:17:53.606807+0000","last_clean":"2024-04-20T21:17:53.606807+0000","last_became_active":"2024-04-20T21:17:52.299016+0000","last_became_peered":"2024-04-20T21:17:52.299016+0000","last_unstale":"2024-04-20T21:17:53.606807+0000","last_undegraded":"2024-04-20T21:17:53.606807+0000","last_fullsized":"2024-04-20T21:17:53.606807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:04:26.111984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.357200+0000","last_change":"2024-04-20T21:17:52.305054+0000","last_active":"2024-04-20T21:17:54.357200+0000","last_peered":"2024-04-20T21:17:54.357200+0000","last_clean":"2024-04-20T21:17:54.357200+0000","last_became_active":"2024-04-20T21:17:52.303307+0000","last_became_peered":"2024-04-20T21:17:52.303307+0000","last_unstale":"2024-04-20T21:17:54.357200+0000","last_undegraded":"2024-04-20T21:17:54.357200+0000","last_fullsized":"2024-04-20T21:17:54.357200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:34:47.842666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.606631+0000","last_change":"2024-04-20T21:17:52.306121+0000","last_active":"2024-04-20T21:17:53.606631+0000","last_peered":"2024-04-20T21:17:53.606631+0000","last_clean":"2024-04-20T21:17:53.606631+0000","last_became_active":"2024-04-20T21:17:52.305749+0000","last_became_peered":"2024-04-20T21:17:52.305749+0000","last_unstale":"2024-04-20T21:17:53.606631+0000","last_undegraded":"2024-04-20T21:17:53.606631+0000","last_fullsized":"2024-04-20T21:17:53.606631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:10:11.436747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.682205+0000","last_change":"2024-04-20T21:17:52.292183+0000","last_active":"2024-04-20T21:17:54.682205+0000","last_peered":"2024-04-20T21:17:54.682205+0000","last_clean":"2024-04-20T21:17:54.682205+0000","last_became_active":"2024-04-20T21:17:52.291906+0000","last_became_peered":"2024-04-20T21:17:52.291906+0000","last_unstale":"2024-04-20T21:17:54.682205+0000","last_undegraded":"2024-04-20T21:17:54.682205+0000","last_fullsized":"2024-04-20T21:17:54.682205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:42:57.243810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.607755+0000","last_change":"2024-04-20T21:17:52.316938+0000","last_active":"2024-04-20T21:17:53.607755+0000","last_peered":"2024-04-20T21:17:53.607755+0000","last_clean":"2024-04-20T21:17:53.607755+0000","last_became_active":"2024-04-20T21:17:52.316386+0000","last_became_peered":"2024-04-20T21:17:52.316386+0000","last_unstale":"2024-04-20T21:17:53.607755+0000","last_undegraded":"2024-04-20T21:17:53.607755+0000","last_fullsized":"2024-04-20T21:17:53.607755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:23:58.112035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.289257+0000","last_change":"2024-04-20T21:17:52.299679+0000","last_active":"2024-04-20T21:17:53.289257+0000","last_peered":"2024-04-20T21:17:53.289257+0000","last_clean":"2024-04-20T21:17:53.289257+0000","last_became_active":"2024-04-20T21:17:52.299325+0000","last_became_peered":"2024-04-20T21:17:52.299325+0000","last_unstale":"2024-04-20T21:17:53.289257+0000","last_undegraded":"2024-04-20T21:17:53.289257+0000","last_fullsized":"2024-04-20T21:17:53.289257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:42:57.243810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.289696+0000","last_change":"2024-04-20T21:17:52.298416+0000","last_active":"2024-04-20T21:17:53.289696+0000","last_peered":"2024-04-20T21:17:53.289696+0000","last_clean":"2024-04-20T21:17:53.289696+0000","last_became_active":"2024-04-20T21:17:52.297990+0000","last_became_peered":"2024-04-20T21:17:52.297990+0000","last_unstale":"2024-04-20T21:17:53.289696+0000","last_undegraded":"2024-04-20T21:17:53.289696+0000","last_fullsized":"2024-04-20T21:17:53.289696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:11:50.878621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.287223+0000","last_change":"2024-04-20T21:17:52.279605+0000","last_active":"2024-04-20T21:17:53.287223+0000","last_peered":"2024-04-20T21:17:53.287223+0000","last_clean":"2024-04-20T21:17:53.287223+0000","last_became_active":"2024-04-20T21:17:52.277631+0000","last_became_peered":"2024-04-20T21:17:52.277631+0000","last_unstale":"2024-04-20T21:17:53.287223+0000","last_undegraded":"2024-04-20T21:17:53.287223+0000","last_fullsized":"2024-04-20T21:17:53.287223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:29:18.971384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.680792+0000","last_change":"2024-04-20T21:17:52.300679+0000","last_active":"2024-04-20T21:17:54.680792+0000","last_peered":"2024-04-20T21:17:54.680792+0000","last_clean":"2024-04-20T21:17:54.680792+0000","last_became_active":"2024-04-20T21:17:52.300387+0000","last_became_peered":"2024-04-20T21:17:52.300387+0000","last_unstale":"2024-04-20T21:17:54.680792+0000","last_undegraded":"2024-04-20T21:17:54.680792+0000","last_fullsized":"2024-04-20T21:17:54.680792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:09:02.488057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.682763+0000","last_change":"2024-04-20T21:17:52.301182+0000","last_active":"2024-04-20T21:17:54.682763+0000","last_peered":"2024-04-20T21:17:54.682763+0000","last_clean":"2024-04-20T21:17:54.682763+0000","last_became_active":"2024-04-20T21:17:52.300852+0000","last_became_peered":"2024-04-20T21:17:52.300852+0000","last_unstale":"2024-04-20T21:17:54.682763+0000","last_undegraded":"2024-04-20T21:17:54.682763+0000","last_fullsized":"2024-04-20T21:17:54.682763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:38:56.558892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.281082+0000","last_change":"2024-04-20T21:17:52.282408+0000","last_active":"2024-04-20T21:17:53.281082+0000","last_peered":"2024-04-20T21:17:53.281082+0000","last_clean":"2024-04-20T21:17:53.281082+0000","last_became_active":"2024-04-20T21:17:52.282044+0000","last_became_peered":"2024-04-20T21:17:52.282044+0000","last_unstale":"2024-04-20T21:17:53.281082+0000","last_undegraded":"2024-04-20T21:17:53.281082+0000","last_fullsized":"2024-04-20T21:17:53.281082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:27:15.675834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.357121+0000","last_change":"2024-04-20T21:17:52.315912+0000","last_active":"2024-04-20T21:17:54.357121+0000","last_peered":"2024-04-20T21:17:54.357121+0000","last_clean":"2024-04-20T21:17:54.357121+0000","last_became_active":"2024-04-20T21:17:52.315642+0000","last_became_peered":"2024-04-20T21:17:52.315642+0000","last_unstale":"2024-04-20T21:17:54.357121+0000","last_undegraded":"2024-04-20T21:17:54.357121+0000","last_fullsized":"2024-04-20T21:17:54.357121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:11:50.878621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.358578+0000","last_change":"2024-04-20T21:17:52.315150+0000","last_active":"2024-04-20T21:17:54.358578+0000","last_peered":"2024-04-20T21:17:54.358578+0000","last_clean":"2024-04-20T21:17:54.358578+0000","last_became_active":"2024-04-20T21:17:52.314856+0000","last_became_peered":"2024-04-20T21:17:52.314856+0000","last_unstale":"2024-04-20T21:17:54.358578+0000","last_undegraded":"2024-04-20T21:17:54.358578+0000","last_fullsized":"2024-04-20T21:17:54.358578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:15:54.762913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.679762+0000","last_change":"2024-04-20T21:17:52.286233+0000","last_active":"2024-04-20T21:17:54.679762+0000","last_peered":"2024-04-20T21:17:54.679762+0000","last_clean":"2024-04-20T21:17:54.679762+0000","last_became_active":"2024-04-20T21:17:52.285872+0000","last_became_peered":"2024-04-20T21:17:52.285872+0000","last_unstale":"2024-04-20T21:17:54.679762+0000","last_undegraded":"2024-04-20T21:17:54.679762+0000","last_fullsized":"2024-04-20T21:17:54.679762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:18:32.885585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.607283+0000","last_change":"2024-04-20T21:17:52.308569+0000","last_active":"2024-04-20T21:17:53.607283+0000","last_peered":"2024-04-20T21:17:53.607283+0000","last_clean":"2024-04-20T21:17:53.607283+0000","last_became_active":"2024-04-20T21:17:52.307606+0000","last_became_peered":"2024-04-20T21:17:52.307606+0000","last_unstale":"2024-04-20T21:17:53.607283+0000","last_undegraded":"2024-04-20T21:17:53.607283+0000","last_fullsized":"2024-04-20T21:17:53.607283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:34:46.630106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.288753+0000","last_change":"2024-04-20T21:17:52.281953+0000","last_active":"2024-04-20T21:17:53.288753+0000","last_peered":"2024-04-20T21:17:53.288753+0000","last_clean":"2024-04-20T21:17:53.288753+0000","last_became_active":"2024-04-20T21:17:52.281675+0000","last_became_peered":"2024-04-20T21:17:52.281675+0000","last_unstale":"2024-04-20T21:17:53.288753+0000","last_undegraded":"2024-04-20T21:17:53.288753+0000","last_fullsized":"2024-04-20T21:17:53.288753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:27:15.675834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.683439+0000","last_change":"2024-04-20T21:17:52.308448+0000","last_active":"2024-04-20T21:17:54.683439+0000","last_peered":"2024-04-20T21:17:54.683439+0000","last_clean":"2024-04-20T21:17:54.683439+0000","last_became_active":"2024-04-20T21:17:52.308158+0000","last_became_peered":"2024-04-20T21:17:52.308158+0000","last_unstale":"2024-04-20T21:17:54.683439+0000","last_undegraded":"2024-04-20T21:17:54.683439+0000","last_fullsized":"2024-04-20T21:17:54.683439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:09:02.488057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.357412+0000","last_change":"2024-04-20T21:17:52.304114+0000","last_active":"2024-04-20T21:17:54.357412+0000","last_peered":"2024-04-20T21:17:54.357412+0000","last_clean":"2024-04-20T21:17:54.357412+0000","last_became_active":"2024-04-20T21:17:52.303016+0000","last_became_peered":"2024-04-20T21:17:52.303016+0000","last_unstale":"2024-04-20T21:17:54.357412+0000","last_undegraded":"2024-04-20T21:17:54.357412+0000","last_fullsized":"2024-04-20T21:17:54.357412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:26:30.156714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.681213+0000","last_change":"2024-04-20T21:17:52.297418+0000","last_active":"2024-04-20T21:17:54.681213+0000","last_peered":"2024-04-20T21:17:54.681213+0000","last_clean":"2024-04-20T21:17:54.681213+0000","last_became_active":"2024-04-20T21:17:52.297018+0000","last_became_peered":"2024-04-20T21:17:52.297018+0000","last_unstale":"2024-04-20T21:17:54.681213+0000","last_undegraded":"2024-04-20T21:17:54.681213+0000","last_fullsized":"2024-04-20T21:17:54.681213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:12:56.626507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.357408+0000","last_change":"2024-04-20T21:17:52.306133+0000","last_active":"2024-04-20T21:17:54.357408+0000","last_peered":"2024-04-20T21:17:54.357408+0000","last_clean":"2024-04-20T21:17:54.357408+0000","last_became_active":"2024-04-20T21:17:52.304108+0000","last_became_peered":"2024-04-20T21:17:52.304108+0000","last_unstale":"2024-04-20T21:17:54.357408+0000","last_undegraded":"2024-04-20T21:17:54.357408+0000","last_fullsized":"2024-04-20T21:17:54.357408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:10:11.436747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.683656+0000","last_change":"2024-04-20T21:17:52.301781+0000","last_active":"2024-04-20T21:17:54.683656+0000","last_peered":"2024-04-20T21:17:54.683656+0000","last_clean":"2024-04-20T21:17:54.683656+0000","last_became_active":"2024-04-20T21:17:52.301194+0000","last_became_peered":"2024-04-20T21:17:52.301194+0000","last_unstale":"2024-04-20T21:17:54.683656+0000","last_undegraded":"2024-04-20T21:17:54.683656+0000","last_fullsized":"2024-04-20T21:17:54.683656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:10:11.436747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.289706+0000","last_change":"2024-04-20T21:17:52.303955+0000","last_active":"2024-04-20T21:17:53.289706+0000","last_peered":"2024-04-20T21:17:53.289706+0000","last_clean":"2024-04-20T21:17:53.289706+0000","last_became_active":"2024-04-20T21:17:52.303421+0000","last_became_peered":"2024-04-20T21:17:52.303421+0000","last_unstale":"2024-04-20T21:17:53.289706+0000","last_undegraded":"2024-04-20T21:17:53.289706+0000","last_fullsized":"2024-04-20T21:17:53.289706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:34:47.842666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.281621+0000","last_change":"2024-04-20T21:17:52.304472+0000","last_active":"2024-04-20T21:17:53.281621+0000","last_peered":"2024-04-20T21:17:53.281621+0000","last_clean":"2024-04-20T21:17:53.281621+0000","last_became_active":"2024-04-20T21:17:52.303141+0000","last_became_peered":"2024-04-20T21:17:52.303141+0000","last_unstale":"2024-04-20T21:17:53.281621+0000","last_undegraded":"2024-04-20T21:17:53.281621+0000","last_fullsized":"2024-04-20T21:17:53.281621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:11:50.878621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.681465+0000","last_change":"2024-04-20T21:17:52.302610+0000","last_active":"2024-04-20T21:17:54.681465+0000","last_peered":"2024-04-20T21:17:54.681465+0000","last_clean":"2024-04-20T21:17:54.681465+0000","last_became_active":"2024-04-20T21:17:52.302271+0000","last_became_peered":"2024-04-20T21:17:52.302271+0000","last_unstale":"2024-04-20T21:17:54.681465+0000","last_undegraded":"2024-04-20T21:17:54.681465+0000","last_fullsized":"2024-04-20T21:17:54.681465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:27:15.675834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.683628+0000","last_change":"2024-04-20T21:17:52.307809+0000","last_active":"2024-04-20T21:17:54.683628+0000","last_peered":"2024-04-20T21:17:54.683628+0000","last_clean":"2024-04-20T21:17:54.683628+0000","last_became_active":"2024-04-20T21:17:52.307459+0000","last_became_peered":"2024-04-20T21:17:52.307459+0000","last_unstale":"2024-04-20T21:17:54.683628+0000","last_undegraded":"2024-04-20T21:17:54.683628+0000","last_fullsized":"2024-04-20T21:17:54.683628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:52:26.845286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.357745+0000","last_change":"2024-04-20T21:17:52.305447+0000","last_active":"2024-04-20T21:17:54.357745+0000","last_peered":"2024-04-20T21:17:54.357745+0000","last_clean":"2024-04-20T21:17:54.357745+0000","last_became_active":"2024-04-20T21:17:52.303784+0000","last_became_peered":"2024-04-20T21:17:52.303784+0000","last_unstale":"2024-04-20T21:17:54.357745+0000","last_undegraded":"2024-04-20T21:17:54.357745+0000","last_fullsized":"2024-04-20T21:17:54.357745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:04:26.111984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.679196+0000","last_change":"2024-04-20T21:17:52.294245+0000","last_active":"2024-04-20T21:17:54.679196+0000","last_peered":"2024-04-20T21:17:54.679196+0000","last_clean":"2024-04-20T21:17:54.679196+0000","last_became_active":"2024-04-20T21:17:52.293880+0000","last_became_peered":"2024-04-20T21:17:52.293880+0000","last_unstale":"2024-04-20T21:17:54.679196+0000","last_undegraded":"2024-04-20T21:17:54.679196+0000","last_fullsized":"2024-04-20T21:17:54.679196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:51:58.727652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.287037+0000","last_change":"2024-04-20T21:17:52.303676+0000","last_active":"2024-04-20T21:17:53.287037+0000","last_peered":"2024-04-20T21:17:53.287037+0000","last_clean":"2024-04-20T21:17:53.287037+0000","last_became_active":"2024-04-20T21:17:52.303369+0000","last_became_peered":"2024-04-20T21:17:52.303369+0000","last_unstale":"2024-04-20T21:17:53.287037+0000","last_undegraded":"2024-04-20T21:17:53.287037+0000","last_fullsized":"2024-04-20T21:17:53.287037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:09:02.488057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.287416+0000","last_change":"2024-04-20T21:17:52.277151+0000","last_active":"2024-04-20T21:17:53.287416+0000","last_peered":"2024-04-20T21:17:53.287416+0000","last_clean":"2024-04-20T21:17:53.287416+0000","last_became_active":"2024-04-20T21:17:52.276740+0000","last_became_peered":"2024-04-20T21:17:52.276740+0000","last_unstale":"2024-04-20T21:17:53.287416+0000","last_undegraded":"2024-04-20T21:17:53.287416+0000","last_fullsized":"2024-04-20T21:17:53.287416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:34:27.917112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.680860+0000","last_change":"2024-04-20T21:17:51.275103+0000","last_active":"2024-04-20T21:17:54.680860+0000","last_peered":"2024-04-20T21:17:54.680860+0000","last_clean":"2024-04-20T21:17:54.680860+0000","last_became_active":"2024-04-20T21:17:51.274773+0000","last_became_peered":"2024-04-20T21:17:51.274773+0000","last_unstale":"2024-04-20T21:17:54.680860+0000","last_undegraded":"2024-04-20T21:17:54.680860+0000","last_fullsized":"2024-04-20T21:17:54.680860+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:53:16.864261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.683824+0000","last_change":"2024-04-20T21:17:52.312353+0000","last_active":"2024-04-20T21:17:54.683824+0000","last_peered":"2024-04-20T21:17:54.683824+0000","last_clean":"2024-04-20T21:17:54.683824+0000","last_became_active":"2024-04-20T21:17:52.311933+0000","last_became_peered":"2024-04-20T21:17:52.311933+0000","last_unstale":"2024-04-20T21:17:54.683824+0000","last_undegraded":"2024-04-20T21:17:54.683824+0000","last_fullsized":"2024-04-20T21:17:54.683824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:12:56.626507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.288439+0000","last_change":"2024-04-20T21:17:51.269484+0000","last_active":"2024-04-20T21:17:53.288439+0000","last_peered":"2024-04-20T21:17:53.288439+0000","last_clean":"2024-04-20T21:17:53.288439+0000","last_became_active":"2024-04-20T21:17:51.269196+0000","last_became_peered":"2024-04-20T21:17:51.269196+0000","last_unstale":"2024-04-20T21:17:53.288439+0000","last_undegraded":"2024-04-20T21:17:53.288439+0000","last_fullsized":"2024-04-20T21:17:53.288439+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:18:39.465785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.681496+0000","last_change":"2024-04-20T21:17:51.273330+0000","last_active":"2024-04-20T21:17:54.681496+0000","last_peered":"2024-04-20T21:17:54.681496+0000","last_clean":"2024-04-20T21:17:54.681496+0000","last_became_active":"2024-04-20T21:17:51.273036+0000","last_became_peered":"2024-04-20T21:17:51.273036+0000","last_unstale":"2024-04-20T21:17:54.681496+0000","last_undegraded":"2024-04-20T21:17:54.681496+0000","last_fullsized":"2024-04-20T21:17:54.681496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:07:01.697230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.682607+0000","last_change":"2024-04-20T21:17:52.300897+0000","last_active":"2024-04-20T21:17:54.682607+0000","last_peered":"2024-04-20T21:17:54.682607+0000","last_clean":"2024-04-20T21:17:54.682607+0000","last_became_active":"2024-04-20T21:17:52.300600+0000","last_became_peered":"2024-04-20T21:17:52.300600+0000","last_unstale":"2024-04-20T21:17:54.682607+0000","last_undegraded":"2024-04-20T21:17:54.682607+0000","last_fullsized":"2024-04-20T21:17:54.682607+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:11:50.878621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.357354+0000","last_change":"2024-04-20T21:17:51.276810+0000","last_active":"2024-04-20T21:17:54.357354+0000","last_peered":"2024-04-20T21:17:54.357354+0000","last_clean":"2024-04-20T21:17:54.357354+0000","last_became_active":"2024-04-20T21:17:51.276547+0000","last_became_peered":"2024-04-20T21:17:51.276547+0000","last_unstale":"2024-04-20T21:17:54.357354+0000","last_undegraded":"2024-04-20T21:17:54.357354+0000","last_fullsized":"2024-04-20T21:17:54.357354+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:04:47.881403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.281043+0000","last_change":"2024-04-20T21:17:52.299388+0000","last_active":"2024-04-20T21:17:53.281043+0000","last_peered":"2024-04-20T21:17:53.281043+0000","last_clean":"2024-04-20T21:17:53.281043+0000","last_became_active":"2024-04-20T21:17:52.298531+0000","last_became_peered":"2024-04-20T21:17:52.298531+0000","last_unstale":"2024-04-20T21:17:53.281043+0000","last_undegraded":"2024-04-20T21:17:53.281043+0000","last_fullsized":"2024-04-20T21:17:53.281043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:12:56.626507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.288661+0000","last_change":"2024-04-20T21:17:51.273343+0000","last_active":"2024-04-20T21:17:53.288661+0000","last_peered":"2024-04-20T21:17:53.288661+0000","last_clean":"2024-04-20T21:17:53.288661+0000","last_became_active":"2024-04-20T21:17:51.273018+0000","last_became_peered":"2024-04-20T21:17:51.273018+0000","last_unstale":"2024-04-20T21:17:53.288661+0000","last_undegraded":"2024-04-20T21:17:53.288661+0000","last_fullsized":"2024-04-20T21:17:53.288661+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:34:26.903632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.358664+0000","last_change":"2024-04-20T21:17:52.300438+0000","last_active":"2024-04-20T21:17:54.358664+0000","last_peered":"2024-04-20T21:17:54.358664+0000","last_clean":"2024-04-20T21:17:54.358664+0000","last_became_active":"2024-04-20T21:17:52.300082+0000","last_became_peered":"2024-04-20T21:17:52.300082+0000","last_unstale":"2024-04-20T21:17:54.358664+0000","last_undegraded":"2024-04-20T21:17:54.358664+0000","last_fullsized":"2024-04-20T21:17:54.358664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:23:58.112035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.358530+0000","last_change":"2024-04-20T21:17:51.272721+0000","last_active":"2024-04-20T21:17:54.358530+0000","last_peered":"2024-04-20T21:17:54.358530+0000","last_clean":"2024-04-20T21:17:54.358530+0000","last_became_active":"2024-04-20T21:17:51.272403+0000","last_became_peered":"2024-04-20T21:17:51.272403+0000","last_unstale":"2024-04-20T21:17:54.358530+0000","last_undegraded":"2024-04-20T21:17:54.358530+0000","last_fullsized":"2024-04-20T21:17:54.358530+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:09:01.474577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.684541+0000","last_change":"2024-04-20T21:17:52.316437+0000","last_active":"2024-04-20T21:17:54.684541+0000","last_peered":"2024-04-20T21:17:54.684541+0000","last_clean":"2024-04-20T21:17:54.684541+0000","last_became_active":"2024-04-20T21:17:52.316129+0000","last_became_peered":"2024-04-20T21:17:52.316129+0000","last_unstale":"2024-04-20T21:17:54.684541+0000","last_undegraded":"2024-04-20T21:17:54.684541+0000","last_fullsized":"2024-04-20T21:17:54.684541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:34:47.842666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.281424+0000","last_change":"2024-04-20T21:17:51.268776+0000","last_active":"2024-04-20T21:17:53.281424+0000","last_peered":"2024-04-20T21:17:53.281424+0000","last_clean":"2024-04-20T21:17:53.281424+0000","last_became_active":"2024-04-20T21:17:51.268468+0000","last_became_peered":"2024-04-20T21:17:51.268468+0000","last_unstale":"2024-04-20T21:17:53.281424+0000","last_undegraded":"2024-04-20T21:17:53.281424+0000","last_fullsized":"2024-04-20T21:17:53.281424+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:34:26.903632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.287471+0000","last_change":"2024-04-20T21:17:52.300758+0000","last_active":"2024-04-20T21:17:53.287471+0000","last_peered":"2024-04-20T21:17:53.287471+0000","last_clean":"2024-04-20T21:17:53.287471+0000","last_became_active":"2024-04-20T21:17:52.300476+0000","last_became_peered":"2024-04-20T21:17:52.300476+0000","last_unstale":"2024-04-20T21:17:53.287471+0000","last_undegraded":"2024-04-20T21:17:53.287471+0000","last_fullsized":"2024-04-20T21:17:53.287471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:51:58.727652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.288618+0000","last_change":"2024-04-20T21:17:51.270900+0000","last_active":"2024-04-20T21:17:53.288618+0000","last_peered":"2024-04-20T21:17:53.288618+0000","last_clean":"2024-04-20T21:17:53.288618+0000","last_became_active":"2024-04-20T21:17:51.270618+0000","last_became_peered":"2024-04-20T21:17:51.270618+0000","last_unstale":"2024-04-20T21:17:53.288618+0000","last_undegraded":"2024-04-20T21:17:53.288618+0000","last_fullsized":"2024-04-20T21:17:53.288618+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:04:47.881403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.606885+0000","last_change":"2024-04-20T21:17:52.313555+0000","last_active":"2024-04-20T21:17:53.606885+0000","last_peered":"2024-04-20T21:17:53.606885+0000","last_clean":"2024-04-20T21:17:53.606885+0000","last_became_active":"2024-04-20T21:17:52.312197+0000","last_became_peered":"2024-04-20T21:17:52.312197+0000","last_unstale":"2024-04-20T21:17:53.606885+0000","last_undegraded":"2024-04-20T21:17:53.606885+0000","last_fullsized":"2024-04-20T21:17:53.606885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:52:22.418925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.683925+0000","last_change":"2024-04-20T21:17:51.275126+0000","last_active":"2024-04-20T21:17:54.683925+0000","last_peered":"2024-04-20T21:17:54.683925+0000","last_clean":"2024-04-20T21:17:54.683925+0000","last_became_active":"2024-04-20T21:17:51.273060+0000","last_became_peered":"2024-04-20T21:17:51.273060+0000","last_unstale":"2024-04-20T21:17:54.683925+0000","last_undegraded":"2024-04-20T21:17:54.683925+0000","last_fullsized":"2024-04-20T21:17:54.683925+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:06:19.740358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.683147+0000","last_change":"2024-04-20T21:17:52.302899+0000","last_active":"2024-04-20T21:17:54.683147+0000","last_peered":"2024-04-20T21:17:54.683147+0000","last_clean":"2024-04-20T21:17:54.683147+0000","last_became_active":"2024-04-20T21:17:52.302606+0000","last_became_peered":"2024-04-20T21:17:52.302606+0000","last_unstale":"2024-04-20T21:17:54.683147+0000","last_undegraded":"2024-04-20T21:17:54.683147+0000","last_fullsized":"2024-04-20T21:17:54.683147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:13:52.887276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.357587+0000","last_change":"2024-04-20T21:17:51.278078+0000","last_active":"2024-04-20T21:17:54.357587+0000","last_peered":"2024-04-20T21:17:54.357587+0000","last_clean":"2024-04-20T21:17:54.357587+0000","last_became_active":"2024-04-20T21:17:51.277775+0000","last_became_peered":"2024-04-20T21:17:51.277775+0000","last_unstale":"2024-04-20T21:17:54.357587+0000","last_undegraded":"2024-04-20T21:17:54.357587+0000","last_fullsized":"2024-04-20T21:17:54.357587+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:23:42.473358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.607236+0000","last_change":"2024-04-20T21:17:52.295456+0000","last_active":"2024-04-20T21:17:53.607236+0000","last_peered":"2024-04-20T21:17:53.607236+0000","last_clean":"2024-04-20T21:17:53.607236+0000","last_became_active":"2024-04-20T21:17:52.294825+0000","last_became_peered":"2024-04-20T21:17:52.294825+0000","last_unstale":"2024-04-20T21:17:53.607236+0000","last_undegraded":"2024-04-20T21:17:53.607236+0000","last_fullsized":"2024-04-20T21:17:53.607236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:43:32.983640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.288028+0000","last_change":"2024-04-20T21:17:51.264742+0000","last_active":"2024-04-20T21:17:53.288028+0000","last_peered":"2024-04-20T21:17:53.288028+0000","last_clean":"2024-04-20T21:17:53.288028+0000","last_became_active":"2024-04-20T21:17:51.264009+0000","last_became_peered":"2024-04-20T21:17:51.264009+0000","last_unstale":"2024-04-20T21:17:53.288028+0000","last_undegraded":"2024-04-20T21:17:53.288028+0000","last_fullsized":"2024-04-20T21:17:53.288028+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:29:17.957905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.288331+0000","last_change":"2024-04-20T21:17:52.313950+0000","last_active":"2024-04-20T21:17:53.288331+0000","last_peered":"2024-04-20T21:17:53.288331+0000","last_clean":"2024-04-20T21:17:53.288331+0000","last_became_active":"2024-04-20T21:17:52.311961+0000","last_became_peered":"2024-04-20T21:17:52.311961+0000","last_unstale":"2024-04-20T21:17:53.288331+0000","last_undegraded":"2024-04-20T21:17:53.288331+0000","last_fullsized":"2024-04-20T21:17:53.288331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:12:56.626507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.678630+0000","last_change":"2024-04-20T21:17:51.254940+0000","last_active":"2024-04-20T21:17:54.678630+0000","last_peered":"2024-04-20T21:17:54.678630+0000","last_clean":"2024-04-20T21:17:54.678630+0000","last_became_active":"2024-04-20T21:17:51.254597+0000","last_became_peered":"2024-04-20T21:17:51.254597+0000","last_unstale":"2024-04-20T21:17:54.678630+0000","last_undegraded":"2024-04-20T21:17:54.678630+0000","last_fullsized":"2024-04-20T21:17:54.678630+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:43:48.697965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.606512+0000","last_change":"2024-04-20T21:17:52.303514+0000","last_active":"2024-04-20T21:17:53.606512+0000","last_peered":"2024-04-20T21:17:53.606512+0000","last_clean":"2024-04-20T21:17:53.606512+0000","last_became_active":"2024-04-20T21:17:52.303225+0000","last_became_peered":"2024-04-20T21:17:52.303225+0000","last_unstale":"2024-04-20T21:17:53.606512+0000","last_undegraded":"2024-04-20T21:17:53.606512+0000","last_fullsized":"2024-04-20T21:17:53.606512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:26:30.156714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.287716+0000","last_change":"2024-04-20T21:17:51.276458+0000","last_active":"2024-04-20T21:17:53.287716+0000","last_peered":"2024-04-20T21:17:53.287716+0000","last_clean":"2024-04-20T21:17:53.287716+0000","last_became_active":"2024-04-20T21:17:51.276033+0000","last_became_peered":"2024-04-20T21:17:51.276033+0000","last_unstale":"2024-04-20T21:17:53.287716+0000","last_undegraded":"2024-04-20T21:17:53.287716+0000","last_fullsized":"2024-04-20T21:17:53.287716+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:00:14.426522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.607498+0000","last_change":"2024-04-20T21:17:52.310874+0000","last_active":"2024-04-20T21:17:53.607498+0000","last_peered":"2024-04-20T21:17:53.607498+0000","last_clean":"2024-04-20T21:17:53.607498+0000","last_became_active":"2024-04-20T21:17:52.310612+0000","last_became_peered":"2024-04-20T21:17:52.310612+0000","last_unstale":"2024-04-20T21:17:53.607498+0000","last_undegraded":"2024-04-20T21:17:53.607498+0000","last_fullsized":"2024-04-20T21:17:53.607498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:59:58.666249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.679906+0000","last_change":"2024-04-20T21:17:51.266671+0000","last_active":"2024-04-20T21:17:54.679906+0000","last_peered":"2024-04-20T21:17:54.679906+0000","last_clean":"2024-04-20T21:17:54.679906+0000","last_became_active":"2024-04-20T21:17:51.264486+0000","last_became_peered":"2024-04-20T21:17:51.264486+0000","last_unstale":"2024-04-20T21:17:54.679906+0000","last_undegraded":"2024-04-20T21:17:54.679906+0000","last_fullsized":"2024-04-20T21:17:54.679906+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:53:16.864261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.607949+0000","last_change":"2024-04-20T21:17:52.309913+0000","last_active":"2024-04-20T21:17:53.607949+0000","last_peered":"2024-04-20T21:17:53.607949+0000","last_clean":"2024-04-20T21:17:53.607949+0000","last_became_active":"2024-04-20T21:17:52.309643+0000","last_became_peered":"2024-04-20T21:17:52.309643+0000","last_unstale":"2024-04-20T21:17:53.607949+0000","last_undegraded":"2024-04-20T21:17:53.607949+0000","last_fullsized":"2024-04-20T21:17:53.607949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:48:28.191286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.281734+0000","last_change":"2024-04-20T21:17:51.272265+0000","last_active":"2024-04-20T21:17:53.281734+0000","last_peered":"2024-04-20T21:17:53.281734+0000","last_clean":"2024-04-20T21:17:53.281734+0000","last_became_active":"2024-04-20T21:17:51.271937+0000","last_became_peered":"2024-04-20T21:17:51.271937+0000","last_unstale":"2024-04-20T21:17:53.281734+0000","last_undegraded":"2024-04-20T21:17:53.281734+0000","last_fullsized":"2024-04-20T21:17:53.281734+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:51:57.714172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.358372+0000","last_change":"2024-04-20T21:17:52.304344+0000","last_active":"2024-04-20T21:17:54.358372+0000","last_peered":"2024-04-20T21:17:54.358372+0000","last_clean":"2024-04-20T21:17:54.358372+0000","last_became_active":"2024-04-20T21:17:52.304006+0000","last_became_peered":"2024-04-20T21:17:52.304006+0000","last_unstale":"2024-04-20T21:17:54.358372+0000","last_undegraded":"2024-04-20T21:17:54.358372+0000","last_fullsized":"2024-04-20T21:17:54.358372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:59:58.666249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.679057+0000","last_change":"2024-04-20T21:17:51.272508+0000","last_active":"2024-04-20T21:17:54.679057+0000","last_peered":"2024-04-20T21:17:54.679057+0000","last_clean":"2024-04-20T21:17:54.679057+0000","last_became_active":"2024-04-20T21:17:51.272056+0000","last_became_peered":"2024-04-20T21:17:51.272056+0000","last_unstale":"2024-04-20T21:17:54.679057+0000","last_undegraded":"2024-04-20T21:17:54.679057+0000","last_fullsized":"2024-04-20T21:17:54.679057+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:06:19.740358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.606888+0000","last_change":"2024-04-20T21:17:52.308203+0000","last_active":"2024-04-20T21:17:53.606888+0000","last_peered":"2024-04-20T21:17:53.606888+0000","last_clean":"2024-04-20T21:17:53.606888+0000","last_became_active":"2024-04-20T21:17:52.307941+0000","last_became_peered":"2024-04-20T21:17:52.307941+0000","last_unstale":"2024-04-20T21:17:53.606888+0000","last_undegraded":"2024-04-20T21:17:53.606888+0000","last_fullsized":"2024-04-20T21:17:53.606888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:51:21.491774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.606871+0000","last_change":"2024-04-20T21:17:43.059637+0000","last_active":"2024-04-20T21:17:53.606871+0000","last_peered":"2024-04-20T21:17:53.606871+0000","last_clean":"2024-04-20T21:17:53.606871+0000","last_became_active":"2024-04-20T21:17:43.056207+0000","last_became_peered":"2024-04-20T21:17:43.056207+0000","last_unstale":"2024-04-20T21:17:53.606871+0000","last_undegraded":"2024-04-20T21:17:53.606871+0000","last_fullsized":"2024-04-20T21:17:53.606871+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:41.036290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:41.036290+0000","last_clean_scrub_stamp":"2024-04-20T21:17:41.036290+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:58:51.091337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.357115+0000","last_change":"2024-04-20T21:17:51.277973+0000","last_active":"2024-04-20T21:17:54.357115+0000","last_peered":"2024-04-20T21:17:54.357115+0000","last_clean":"2024-04-20T21:17:54.357115+0000","last_became_active":"2024-04-20T21:17:51.277643+0000","last_became_peered":"2024-04-20T21:17:51.277643+0000","last_unstale":"2024-04-20T21:17:54.357115+0000","last_undegraded":"2024-04-20T21:17:54.357115+0000","last_fullsized":"2024-04-20T21:17:54.357115+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:41:08.560538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.683244+0000","last_change":"2024-04-20T21:17:52.316313+0000","last_active":"2024-04-20T21:17:54.683244+0000","last_peered":"2024-04-20T21:17:54.683244+0000","last_clean":"2024-04-20T21:17:54.683244+0000","last_became_active":"2024-04-20T21:17:52.316054+0000","last_became_peered":"2024-04-20T21:17:52.316054+0000","last_unstale":"2024-04-20T21:17:54.683244+0000","last_undegraded":"2024-04-20T21:17:54.683244+0000","last_fullsized":"2024-04-20T21:17:54.683244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:26:30.156714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.606138+0000","last_change":"2024-04-20T21:17:51.282977+0000","last_active":"2024-04-20T21:17:53.606138+0000","last_peered":"2024-04-20T21:17:53.606138+0000","last_clean":"2024-04-20T21:17:53.606138+0000","last_became_active":"2024-04-20T21:17:51.282699+0000","last_became_peered":"2024-04-20T21:17:51.282699+0000","last_unstale":"2024-04-20T21:17:53.606138+0000","last_undegraded":"2024-04-20T21:17:53.606138+0000","last_fullsized":"2024-04-20T21:17:53.606138+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:04:47.881403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.682600+0000","last_change":"2024-04-20T21:17:52.301574+0000","last_active":"2024-04-20T21:17:54.682600+0000","last_peered":"2024-04-20T21:17:54.682600+0000","last_clean":"2024-04-20T21:17:54.682600+0000","last_became_active":"2024-04-20T21:17:52.299834+0000","last_became_peered":"2024-04-20T21:17:52.299834+0000","last_unstale":"2024-04-20T21:17:54.682600+0000","last_undegraded":"2024-04-20T21:17:54.682600+0000","last_fullsized":"2024-04-20T21:17:54.682600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:23:43.486837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.679134+0000","last_change":"2024-04-20T21:17:51.268326+0000","last_active":"2024-04-20T21:17:54.679134+0000","last_peered":"2024-04-20T21:17:54.679134+0000","last_clean":"2024-04-20T21:17:54.679134+0000","last_became_active":"2024-04-20T21:17:51.268016+0000","last_became_peered":"2024-04-20T21:17:51.268016+0000","last_unstale":"2024-04-20T21:17:54.679134+0000","last_undegraded":"2024-04-20T21:17:54.679134+0000","last_fullsized":"2024-04-20T21:17:54.679134+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:18:39.465785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.280844+0000","last_change":"2024-04-20T21:17:52.289513+0000","last_active":"2024-04-20T21:17:53.280844+0000","last_peered":"2024-04-20T21:17:53.280844+0000","last_clean":"2024-04-20T21:17:53.280844+0000","last_became_active":"2024-04-20T21:17:52.288677+0000","last_became_peered":"2024-04-20T21:17:52.288677+0000","last_unstale":"2024-04-20T21:17:53.280844+0000","last_undegraded":"2024-04-20T21:17:53.280844+0000","last_fullsized":"2024-04-20T21:17:53.280844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:52:26.845286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.358151+0000","last_change":"2024-04-20T21:17:51.275850+0000","last_active":"2024-04-20T21:17:54.358151+0000","last_peered":"2024-04-20T21:17:54.358151+0000","last_clean":"2024-04-20T21:17:54.358151+0000","last_became_active":"2024-04-20T21:17:51.275350+0000","last_became_peered":"2024-04-20T21:17:51.275350+0000","last_unstale":"2024-04-20T21:17:54.358151+0000","last_undegraded":"2024-04-20T21:17:54.358151+0000","last_fullsized":"2024-04-20T21:17:54.358151+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:38:55.545413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.287199+0000","last_change":"2024-04-20T21:17:52.305922+0000","last_active":"2024-04-20T21:17:53.287199+0000","last_peered":"2024-04-20T21:17:53.287199+0000","last_clean":"2024-04-20T21:17:53.287199+0000","last_became_active":"2024-04-20T21:17:52.304705+0000","last_became_peered":"2024-04-20T21:17:52.304705+0000","last_unstale":"2024-04-20T21:17:53.287199+0000","last_undegraded":"2024-04-20T21:17:53.287199+0000","last_fullsized":"2024-04-20T21:17:53.287199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:04:48.894882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.281179+0000","last_change":"2024-04-20T21:17:51.260375+0000","last_active":"2024-04-20T21:17:53.281179+0000","last_peered":"2024-04-20T21:17:53.281179+0000","last_clean":"2024-04-20T21:17:53.281179+0000","last_became_active":"2024-04-20T21:17:51.259993+0000","last_became_peered":"2024-04-20T21:17:51.259993+0000","last_unstale":"2024-04-20T21:17:53.281179+0000","last_undegraded":"2024-04-20T21:17:53.281179+0000","last_fullsized":"2024-04-20T21:17:53.281179+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:04:47.881403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.358295+0000","last_change":"2024-04-20T21:17:52.308697+0000","last_active":"2024-04-20T21:17:54.358295+0000","last_peered":"2024-04-20T21:17:54.358295+0000","last_clean":"2024-04-20T21:17:54.358295+0000","last_became_active":"2024-04-20T21:17:52.308037+0000","last_became_peered":"2024-04-20T21:17:52.308037+0000","last_unstale":"2024-04-20T21:17:54.358295+0000","last_undegraded":"2024-04-20T21:17:54.358295+0000","last_fullsized":"2024-04-20T21:17:54.358295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:34:46.630106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.607409+0000","last_change":"2024-04-20T21:17:51.283426+0000","last_active":"2024-04-20T21:17:53.607409+0000","last_peered":"2024-04-20T21:17:53.607409+0000","last_clean":"2024-04-20T21:17:53.607409+0000","last_became_active":"2024-04-20T21:17:51.283162+0000","last_became_peered":"2024-04-20T21:17:51.283162+0000","last_unstale":"2024-04-20T21:17:53.607409+0000","last_undegraded":"2024-04-20T21:17:53.607409+0000","last_fullsized":"2024-04-20T21:17:53.607409+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:18:31.872106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.281429+0000","last_change":"2024-04-20T21:17:52.285628+0000","last_active":"2024-04-20T21:17:53.281429+0000","last_peered":"2024-04-20T21:17:53.281429+0000","last_clean":"2024-04-20T21:17:53.281429+0000","last_became_active":"2024-04-20T21:17:52.284918+0000","last_became_peered":"2024-04-20T21:17:52.284918+0000","last_unstale":"2024-04-20T21:17:53.281429+0000","last_undegraded":"2024-04-20T21:17:53.281429+0000","last_fullsized":"2024-04-20T21:17:53.281429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:42:57.243810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.281782+0000","last_change":"2024-04-20T21:17:51.271506+0000","last_active":"2024-04-20T21:17:53.281782+0000","last_peered":"2024-04-20T21:17:53.281782+0000","last_clean":"2024-04-20T21:17:53.281782+0000","last_became_active":"2024-04-20T21:17:51.271243+0000","last_became_peered":"2024-04-20T21:17:51.271243+0000","last_unstale":"2024-04-20T21:17:53.281782+0000","last_undegraded":"2024-04-20T21:17:53.281782+0000","last_fullsized":"2024-04-20T21:17:53.281782+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:29:17.957905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.684078+0000","last_change":"2024-04-20T21:17:52.301465+0000","last_active":"2024-04-20T21:17:54.684078+0000","last_peered":"2024-04-20T21:17:54.684078+0000","last_clean":"2024-04-20T21:17:54.684078+0000","last_became_active":"2024-04-20T21:17:52.300849+0000","last_became_peered":"2024-04-20T21:17:52.300849+0000","last_unstale":"2024-04-20T21:17:54.684078+0000","last_undegraded":"2024-04-20T21:17:54.684078+0000","last_fullsized":"2024-04-20T21:17:54.684078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:04:26.111984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.288907+0000","last_change":"2024-04-20T21:17:51.282509+0000","last_active":"2024-04-20T21:17:53.288907+0000","last_peered":"2024-04-20T21:17:53.288907+0000","last_clean":"2024-04-20T21:17:53.288907+0000","last_became_active":"2024-04-20T21:17:51.281343+0000","last_became_peered":"2024-04-20T21:17:51.281343+0000","last_unstale":"2024-04-20T21:17:53.288907+0000","last_undegraded":"2024-04-20T21:17:53.288907+0000","last_fullsized":"2024-04-20T21:17:53.288907+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:43:48.697965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.684280+0000","last_change":"2024-04-20T21:17:52.311617+0000","last_active":"2024-04-20T21:17:54.684280+0000","last_peered":"2024-04-20T21:17:54.684280+0000","last_clean":"2024-04-20T21:17:54.684280+0000","last_became_active":"2024-04-20T21:17:52.311367+0000","last_became_peered":"2024-04-20T21:17:52.311367+0000","last_unstale":"2024-04-20T21:17:54.684280+0000","last_undegraded":"2024-04-20T21:17:54.684280+0000","last_fullsized":"2024-04-20T21:17:54.684280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:27:15.675834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.607489+0000","last_change":"2024-04-20T21:17:51.283039+0000","last_active":"2024-04-20T21:17:53.607489+0000","last_peered":"2024-04-20T21:17:53.607489+0000","last_clean":"2024-04-20T21:17:53.607489+0000","last_became_active":"2024-04-20T21:17:51.282780+0000","last_became_peered":"2024-04-20T21:17:51.282780+0000","last_unstale":"2024-04-20T21:17:53.607489+0000","last_undegraded":"2024-04-20T21:17:53.607489+0000","last_fullsized":"2024-04-20T21:17:53.607489+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:18:39.465785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.286949+0000","last_change":"2024-04-20T21:17:52.293046+0000","last_active":"2024-04-20T21:17:53.286949+0000","last_peered":"2024-04-20T21:17:53.286949+0000","last_clean":"2024-04-20T21:17:53.286949+0000","last_became_active":"2024-04-20T21:17:52.288365+0000","last_became_peered":"2024-04-20T21:17:52.288365+0000","last_unstale":"2024-04-20T21:17:53.286949+0000","last_undegraded":"2024-04-20T21:17:53.286949+0000","last_fullsized":"2024-04-20T21:17:53.286949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:18:40.479264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.281228+0000","last_change":"2024-04-20T21:17:51.280889+0000","last_active":"2024-04-20T21:17:53.281228+0000","last_peered":"2024-04-20T21:17:53.281228+0000","last_clean":"2024-04-20T21:17:53.281228+0000","last_became_active":"2024-04-20T21:17:51.280582+0000","last_became_peered":"2024-04-20T21:17:51.280582+0000","last_unstale":"2024-04-20T21:17:53.281228+0000","last_undegraded":"2024-04-20T21:17:53.281228+0000","last_fullsized":"2024-04-20T21:17:53.281228+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:18:39.465785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.682757+0000","last_change":"2024-04-20T21:17:52.292459+0000","last_active":"2024-04-20T21:17:54.682757+0000","last_peered":"2024-04-20T21:17:54.682757+0000","last_clean":"2024-04-20T21:17:54.682757+0000","last_became_active":"2024-04-20T21:17:52.292182+0000","last_became_peered":"2024-04-20T21:17:52.292182+0000","last_unstale":"2024-04-20T21:17:54.682757+0000","last_undegraded":"2024-04-20T21:17:54.682757+0000","last_fullsized":"2024-04-20T21:17:54.682757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:34:47.842666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.281162+0000","last_change":"2024-04-20T21:17:51.269365+0000","last_active":"2024-04-20T21:17:53.281162+0000","last_peered":"2024-04-20T21:17:53.281162+0000","last_clean":"2024-04-20T21:17:53.281162+0000","last_became_active":"2024-04-20T21:17:51.267902+0000","last_became_peered":"2024-04-20T21:17:51.267902+0000","last_unstale":"2024-04-20T21:17:53.281162+0000","last_undegraded":"2024-04-20T21:17:53.281162+0000","last_fullsized":"2024-04-20T21:17:53.281162+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:43:48.697965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.681157+0000","last_change":"2024-04-20T21:17:52.315096+0000","last_active":"2024-04-20T21:17:54.681157+0000","last_peered":"2024-04-20T21:17:54.681157+0000","last_clean":"2024-04-20T21:17:54.681157+0000","last_became_active":"2024-04-20T21:17:52.314742+0000","last_became_peered":"2024-04-20T21:17:52.314742+0000","last_unstale":"2024-04-20T21:17:54.681157+0000","last_undegraded":"2024-04-20T21:17:54.681157+0000","last_fullsized":"2024-04-20T21:17:54.681157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:52:26.845286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.607525+0000","last_change":"2024-04-20T21:17:51.288583+0000","last_active":"2024-04-20T21:17:53.607525+0000","last_peered":"2024-04-20T21:17:53.607525+0000","last_clean":"2024-04-20T21:17:53.607525+0000","last_became_active":"2024-04-20T21:17:51.288299+0000","last_became_peered":"2024-04-20T21:17:51.288299+0000","last_unstale":"2024-04-20T21:17:53.607525+0000","last_undegraded":"2024-04-20T21:17:53.607525+0000","last_fullsized":"2024-04-20T21:17:53.607525+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:29:17.957905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.288285+0000","last_change":"2024-04-20T21:17:52.283186+0000","last_active":"2024-04-20T21:17:53.288285+0000","last_peered":"2024-04-20T21:17:53.288285+0000","last_clean":"2024-04-20T21:17:53.288285+0000","last_became_active":"2024-04-20T21:17:52.280776+0000","last_became_peered":"2024-04-20T21:17:52.280776+0000","last_unstale":"2024-04-20T21:17:53.288285+0000","last_undegraded":"2024-04-20T21:17:53.288285+0000","last_fullsized":"2024-04-20T21:17:53.288285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:52:26.845286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.358061+0000","last_change":"2024-04-20T21:17:52.302224+0000","last_active":"2024-04-20T21:17:54.358061+0000","last_peered":"2024-04-20T21:17:54.358061+0000","last_clean":"2024-04-20T21:17:54.358061+0000","last_became_active":"2024-04-20T21:17:52.301869+0000","last_became_peered":"2024-04-20T21:17:52.301869+0000","last_unstale":"2024-04-20T21:17:54.358061+0000","last_undegraded":"2024-04-20T21:17:54.358061+0000","last_fullsized":"2024-04-20T21:17:54.358061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:43:32.983640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.606747+0000","last_change":"2024-04-20T21:17:51.288486+0000","last_active":"2024-04-20T21:17:53.606747+0000","last_peered":"2024-04-20T21:17:53.606747+0000","last_clean":"2024-04-20T21:17:53.606747+0000","last_became_active":"2024-04-20T21:17:51.288131+0000","last_became_peered":"2024-04-20T21:17:51.288131+0000","last_unstale":"2024-04-20T21:17:53.606747+0000","last_undegraded":"2024-04-20T21:17:53.606747+0000","last_fullsized":"2024-04-20T21:17:53.606747+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:51:57.714172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.357879+0000","last_change":"2024-04-20T21:17:52.302297+0000","last_active":"2024-04-20T21:17:54.357879+0000","last_peered":"2024-04-20T21:17:54.357879+0000","last_clean":"2024-04-20T21:17:54.357879+0000","last_became_active":"2024-04-20T21:17:52.301982+0000","last_became_peered":"2024-04-20T21:17:52.301982+0000","last_unstale":"2024-04-20T21:17:54.357879+0000","last_undegraded":"2024-04-20T21:17:54.357879+0000","last_fullsized":"2024-04-20T21:17:54.357879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:51:21.491774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.607170+0000","last_change":"2024-04-20T21:17:51.285151+0000","last_active":"2024-04-20T21:17:53.607170+0000","last_peered":"2024-04-20T21:17:53.607170+0000","last_clean":"2024-04-20T21:17:53.607170+0000","last_became_active":"2024-04-20T21:17:51.284879+0000","last_became_peered":"2024-04-20T21:17:51.284879+0000","last_unstale":"2024-04-20T21:17:53.607170+0000","last_undegraded":"2024-04-20T21:17:53.607170+0000","last_fullsized":"2024-04-20T21:17:53.607170+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:34:26.903632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.684538+0000","last_change":"2024-04-20T21:17:52.317666+0000","last_active":"2024-04-20T21:17:54.684538+0000","last_peered":"2024-04-20T21:17:54.684538+0000","last_clean":"2024-04-20T21:17:54.684538+0000","last_became_active":"2024-04-20T21:17:52.316819+0000","last_became_peered":"2024-04-20T21:17:52.316819+0000","last_unstale":"2024-04-20T21:17:54.684538+0000","last_undegraded":"2024-04-20T21:17:54.684538+0000","last_fullsized":"2024-04-20T21:17:54.684538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:11:50.878621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.682331+0000","last_change":"2024-04-20T21:17:51.276188+0000","last_active":"2024-04-20T21:17:54.682331+0000","last_peered":"2024-04-20T21:17:54.682331+0000","last_clean":"2024-04-20T21:17:54.682331+0000","last_became_active":"2024-04-20T21:17:51.275862+0000","last_became_peered":"2024-04-20T21:17:51.275862+0000","last_unstale":"2024-04-20T21:17:54.682331+0000","last_undegraded":"2024-04-20T21:17:54.682331+0000","last_fullsized":"2024-04-20T21:17:54.682331+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:18:39.465785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.684488+0000","last_change":"2024-04-20T21:17:52.310311+0000","last_active":"2024-04-20T21:17:54.684488+0000","last_peered":"2024-04-20T21:17:54.684488+0000","last_clean":"2024-04-20T21:17:54.684488+0000","last_became_active":"2024-04-20T21:17:52.307410+0000","last_became_peered":"2024-04-20T21:17:52.307410+0000","last_unstale":"2024-04-20T21:17:54.684488+0000","last_undegraded":"2024-04-20T21:17:54.684488+0000","last_fullsized":"2024-04-20T21:17:54.684488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:52:22.418925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.684750+0000","last_change":"2024-04-20T21:17:51.287639+0000","last_active":"2024-04-20T21:17:54.684750+0000","last_peered":"2024-04-20T21:17:54.684750+0000","last_clean":"2024-04-20T21:17:54.684750+0000","last_became_active":"2024-04-20T21:17:51.287421+0000","last_became_peered":"2024-04-20T21:17:51.287421+0000","last_unstale":"2024-04-20T21:17:54.684750+0000","last_undegraded":"2024-04-20T21:17:54.684750+0000","last_fullsized":"2024-04-20T21:17:54.684750+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:07:01.697230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T21:17:57.344 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-20T21:17:57.344 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-20T21:17:57.344 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-20T21:17:57.344 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-20T21:17:57.345 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-20T21:17:57.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.578+0000 7f7c25a00640 1 Processor -- start 2024-04-20T21:17:57.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.578+0000 7f7c25a00640 1 -- start start 2024-04-20T21:17:57.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.579+0000 7f7c25a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7c20144e90 0x7f7c2015ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:57.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.579+0000 7f7c25a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7c2006f140 0x7f7c200aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:57.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.579+0000 7f7c25a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c200af0b0 0x7f7c20071d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:57.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.579+0000 7f7c25a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f7c20145270 con 0x7f7c2006f140 2024-04-20T21:17:57.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.579+0000 7f7c25a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f7c2006dc30 con 0x7f7c20144e90 2024-04-20T21:17:57.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.579+0000 7f7c25a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7c2015f720 con 0x7f7c200af0b0 2024-04-20T21:17:57.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.580+0000 7f7c1e000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7c2006f140 0x7f7c200aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:57.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.580+0000 7f7c1e000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7c2006f140 0x7f7c200aeb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57606/0 (socket says 172.21.15.86:57606) 2024-04-20T21:17:57.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.580+0000 7f7c1e000640 1 -- 172.21.15.86:0/811282515 learned_addr learned my addr 172.21.15.86:0/811282515 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:57.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.580+0000 7f7c1e000640 1 -- 172.21.15.86:0/811282515 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c200af0b0 msgr2=0x7f7c20071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:57.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.580+0000 7f7c1f400640 1 --2- 172.21.15.86:0/811282515 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c200af0b0 0x7f7c20071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:57.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.580+0000 7f7c1ea00640 1 --2- 172.21.15.86:0/811282515 >> v2:172.21.15.144:3300/0 conn(0x7f7c20144e90 0x7f7c2015ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:57.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.580+0000 7f7c1e000640 1 --2- 172.21.15.86:0/811282515 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c200af0b0 0x7f7c20071d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:57.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.580+0000 7f7c1e000640 1 -- 172.21.15.86:0/811282515 >> v2:172.21.15.144:3300/0 conn(0x7f7c20144e90 msgr2=0x7f7c2015ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:57.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.580+0000 7f7c1e000640 1 --2- 172.21.15.86:0/811282515 >> v2:172.21.15.144:3300/0 conn(0x7f7c20144e90 0x7f7c2015ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:57.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.580+0000 7f7c1e000640 1 -- 172.21.15.86:0/811282515 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c2015fa00 con 0x7f7c2006f140 2024-04-20T21:17:57.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.581+0000 7f7c1e000640 1 --2- 172.21.15.86:0/811282515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c2006f140 0x7f7c200aeb70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7c2006dbb0 tx=0x7f7c0c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=351539fab4ceed4e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:57.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.582+0000 7f7c1d600640 1 -- 172.21.15.86:0/811282515 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c035020 con 0x7f7c2006f140 2024-04-20T21:17:57.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.582+0000 7f7c1d600640 1 -- 172.21.15.86:0/811282515 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c0c031870 con 0x7f7c2006f140 2024-04-20T21:17:57.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.582+0000 7f7c1d600640 1 -- 172.21.15.86:0/811282515 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c031c40 con 0x7f7c2006f140 2024-04-20T21:17:57.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.582+0000 7f7c25a00640 1 -- 172.21.15.86:0/811282515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c2006f140 msgr2=0x7f7c200aeb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:57.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.582+0000 7f7c25a00640 1 --2- 172.21.15.86:0/811282515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c2006f140 0x7f7c200aeb70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7c2006dbb0 tx=0x7f7c0c02b910 comp rx=0 tx=0).stop 2024-04-20T21:17:57.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.582+0000 7f7c25a00640 1 -- 172.21.15.86:0/811282515 shutdown_connections 2024-04-20T21:17:57.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.582+0000 7f7c25a00640 1 --2- 172.21.15.86:0/811282515 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c200af0b0 0x7f7c20071d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:57.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.582+0000 7f7c25a00640 1 --2- 172.21.15.86:0/811282515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c2006f140 0x7f7c200aeb70 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:57.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.582+0000 7f7c25a00640 1 --2- 172.21.15.86:0/811282515 >> v2:172.21.15.144:3300/0 conn(0x7f7c20144e90 0x7f7c2015ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:57.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.582+0000 7f7c25a00640 1 -- 172.21.15.86:0/811282515 >> 172.21.15.86:0/811282515 conn(0x7f7c200b2360 msgr2=0x7f7c200b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:57.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.582+0000 7f7c25a00640 1 -- 172.21.15.86:0/811282515 shutdown_connections 2024-04-20T21:17:57.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.582+0000 7f7c25a00640 1 -- 172.21.15.86:0/811282515 wait complete. 2024-04-20T21:17:57.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.584+0000 7f7c25a00640 1 Processor -- start 2024-04-20T21:17:57.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.584+0000 7f7c25a00640 1 -- start start 2024-04-20T21:17:57.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.585+0000 7f7c25a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c2006f140 0x7f7c2015e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:57.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.585+0000 7f7c25a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c200af0b0 0x7f7c20142e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:57.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.585+0000 7f7c25a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c20144e90 0x7f7c200cce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:57.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.585+0000 7f7c25a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2015f350 con 0x7f7c20144e90 2024-04-20T21:17:57.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.585+0000 7f7c25a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2006ded0 con 0x7f7c200af0b0 2024-04-20T21:17:57.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.585+0000 7f7c25a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7c2015f4d0 con 0x7f7c2006f140 2024-04-20T21:17:57.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.585+0000 7f7c1f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c20144e90 0x7f7c200cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:57.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.585+0000 7f7c1f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c20144e90 0x7f7c200cce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57618/0 (socket says 172.21.15.86:57618) 2024-04-20T21:17:57.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.585+0000 7f7c1f400640 1 -- 172.21.15.86:0/2968180077 learned_addr learned my addr 172.21.15.86:0/2968180077 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:57.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.586+0000 7f7c1e000640 1 --2- 172.21.15.86:0/2968180077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c200af0b0 0x7f7c20142e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:57.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.586+0000 7f7c1ea00640 1 --2- 172.21.15.86:0/2968180077 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c2006f140 0x7f7c2015e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:57.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.586+0000 7f7c1f400640 1 -- 172.21.15.86:0/2968180077 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c2006f140 msgr2=0x7f7c2015e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:57.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.586+0000 7f7c1f400640 1 --2- 172.21.15.86:0/2968180077 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c2006f140 0x7f7c2015e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:57.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.586+0000 7f7c1f400640 1 -- 172.21.15.86:0/2968180077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c200af0b0 msgr2=0x7f7c20142e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:57.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.586+0000 7f7c1f400640 1 --2- 172.21.15.86:0/2968180077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c200af0b0 0x7f7c20142e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:57.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.586+0000 7f7c1f400640 1 -- 172.21.15.86:0/2968180077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c20143910 con 0x7f7c20144e90 2024-04-20T21:17:57.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.586+0000 7f7c1f400640 1 --2- 172.21.15.86:0/2968180077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c20144e90 0x7f7c200cce60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7c1800cc30 tx=0x7f7c1800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:57.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.586+0000 7f7c16a00640 1 -- 172.21.15.86:0/2968180077 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1800ce40 con 0x7f7c20144e90 2024-04-20T21:17:57.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.586+0000 7f7c16a00640 1 -- 172.21.15.86:0/2968180077 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c18003990 con 0x7f7c20144e90 2024-04-20T21:17:57.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.586+0000 7f7c16a00640 1 -- 172.21.15.86:0/2968180077 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c18003d40 con 0x7f7c20144e90 2024-04-20T21:17:57.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.586+0000 7f7c25a00640 1 -- 172.21.15.86:0/2968180077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c20143bf0 con 0x7f7c20144e90 2024-04-20T21:17:57.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.587+0000 7f7c25a00640 1 -- 172.21.15.86:0/2968180077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c20145ff0 con 0x7f7c20144e90 2024-04-20T21:17:57.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.588+0000 7f7c25a00640 1 -- 172.21.15.86:0/2968180077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c200689e0 con 0x7f7c20144e90 2024-04-20T21:17:57.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.589+0000 7f7c16a00640 1 -- 172.21.15.86:0/2968180077 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7c18003ee0 con 0x7f7c20144e90 2024-04-20T21:17:57.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.589+0000 7f7c16a00640 1 --2- 172.21.15.86:0/2968180077 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7c0006c920 0x7f7c0006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:57.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.590+0000 7f7c16a00640 1 -- 172.21.15.86:0/2968180077 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7c1808bf40 con 0x7f7c20144e90 2024-04-20T21:17:57.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.590+0000 7f7c1ea00640 1 --2- 172.21.15.86:0/2968180077 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7c0006c920 0x7f7c0006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:57.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.590+0000 7f7c1ea00640 1 --2- 172.21.15.86:0/2968180077 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7c0006c920 0x7f7c0006ede0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7c10006fd0 tx=0x7f7c10008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:57.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.594+0000 7f7c16a00640 1 -- 172.21.15.86:0/2968180077 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7c18014030 con 0x7f7c20144e90 2024-04-20T21:17:57.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.848+0000 7f7c25a00640 1 -- 172.21.15.86:0/2968180077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f7c200b0ab0 con 0x7f7c20144e90 2024-04-20T21:17:57.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.852+0000 7f7c16a00640 1 -- 172.21.15.86:0/2968180077 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+174134 (secure 0 0 0) 0x7f7c1805a400 con 0x7f7c20144e90 2024-04-20T21:17:57.855 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:17:57.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.856+0000 7f7c25a00640 1 -- 172.21.15.86:0/2968180077 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7c0006c920 msgr2=0x7f7c0006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:57.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.856+0000 7f7c25a00640 1 --2- 172.21.15.86:0/2968180077 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7c0006c920 0x7f7c0006ede0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7c10006fd0 tx=0x7f7c10008040 comp rx=0 tx=0).stop 2024-04-20T21:17:57.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.856+0000 7f7c25a00640 1 -- 172.21.15.86:0/2968180077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c20144e90 msgr2=0x7f7c200cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:57.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.856+0000 7f7c25a00640 1 --2- 172.21.15.86:0/2968180077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c20144e90 0x7f7c200cce60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7c1800cc30 tx=0x7f7c1800a5c0 comp rx=0 tx=0).stop 2024-04-20T21:17:57.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.856+0000 7f7c25a00640 1 -- 172.21.15.86:0/2968180077 shutdown_connections 2024-04-20T21:17:57.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.856+0000 7f7c25a00640 1 --2- 172.21.15.86:0/2968180077 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7c0006c920 0x7f7c0006ede0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:57.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.856+0000 7f7c25a00640 1 --2- 172.21.15.86:0/2968180077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c20144e90 0x7f7c200cce60 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:57.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.856+0000 7f7c25a00640 1 --2- 172.21.15.86:0/2968180077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c200af0b0 0x7f7c20142e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:57.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.856+0000 7f7c25a00640 1 --2- 172.21.15.86:0/2968180077 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c2006f140 0x7f7c2015e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:57.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.856+0000 7f7c25a00640 1 -- 172.21.15.86:0/2968180077 >> 172.21.15.86:0/2968180077 conn(0x7f7c200b2360 msgr2=0x7f7c2006e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:57.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.857+0000 7f7c25a00640 1 -- 172.21.15.86:0/2968180077 shutdown_connections 2024-04-20T21:17:57.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:57.857+0000 7f7c25a00640 1 -- 172.21.15.86:0/2968180077 wait complete. 2024-04-20T21:17:57.877 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":5,"flags":0,"active_gid":4107,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":302045240},{"type":"v1","addr":"172.21.15.86:6833","nonce":302045240}]},"active_addr":"172.21.15.86:6833/302045240","active_change":"2024-04-20T21:17:32.614974+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4111,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":1350131512}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":1644357398}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":2978563069}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":3450122717}]}]} 2024-04-20T21:17:57.881 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-20T21:17:57.881 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-20T21:17:57.881 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:17:58.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.079+0000 7fa095e00640 1 Processor -- start 2024-04-20T21:17:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.079+0000 7fa095e00640 1 -- start start 2024-04-20T21:17:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.080+0000 7fa095e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa09015c010 0x7fa09015c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.080+0000 7fa095e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa0900af7b0 0x7fa0900adda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.080+0000 7fa095e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0900ae710 con 0x7fa09006f4c0 2024-04-20T21:17:58.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.080+0000 7fa095e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa0900af320 con 0x7fa09015c010 2024-04-20T21:17:58.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.080+0000 7fa095e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fa09015c930 con 0x7fa0900af7b0 2024-04-20T21:17:58.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.081+0000 7fa094a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa09015c010 0x7fa09015c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:58.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.081+0000 7fa094a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa09015c010 0x7fa09015c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57620/0 (socket says 172.21.15.86:57620) 2024-04-20T21:17:58.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.081+0000 7fa094a00640 1 -- 172.21.15.86:0/2515580476 learned_addr learned my addr 172.21.15.86:0/2515580476 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:58.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.081+0000 7fa08fe00640 1 --2- 172.21.15.86:0/2515580476 >> v2:172.21.15.144:3300/0 conn(0x7fa0900af7b0 0x7fa0900adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:58.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.081+0000 7fa094a00640 1 -- 172.21.15.86:0/2515580476 >> v2:172.21.15.144:3300/0 conn(0x7fa0900af7b0 msgr2=0x7fa0900adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:58.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.081+0000 7fa094a00640 1 --2- 172.21.15.86:0/2515580476 >> v2:172.21.15.144:3300/0 conn(0x7fa0900af7b0 0x7fa0900adda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:58.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.081+0000 7fa094a00640 1 -- 172.21.15.86:0/2515580476 >> v1:172.21.15.144:6789/0 conn(0x7fa09006f4c0 legacy=0x7fa09015ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:58.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.081+0000 7fa094a00640 1 -- 172.21.15.86:0/2515580476 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0900ae9f0 con 0x7fa09015c010 2024-04-20T21:17:58.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.082+0000 7fa094a00640 1 --2- 172.21.15.86:0/2515580476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa09015c010 0x7fa09015c3f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa090145230 tx=0x7fa08402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b31f9731f1830f61 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:58.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.083+0000 7fa08f400640 1 -- 172.21.15.86:0/2515580476 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa084035020 con 0x7fa09015c010 2024-04-20T21:17:58.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.083+0000 7fa08f400640 1 -- 172.21.15.86:0/2515580476 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa084031870 con 0x7fa09015c010 2024-04-20T21:17:58.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.083+0000 7fa08f400640 1 -- 172.21.15.86:0/2515580476 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa084031c40 con 0x7fa09015c010 2024-04-20T21:17:58.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.083+0000 7fa095e00640 1 -- 172.21.15.86:0/2515580476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa09015c010 msgr2=0x7fa09015c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:58.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.083+0000 7fa095e00640 1 --2- 172.21.15.86:0/2515580476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa09015c010 0x7fa09015c3f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa090145230 tx=0x7fa08402b910 comp rx=0 tx=0).stop 2024-04-20T21:17:58.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.083+0000 7fa095e00640 1 -- 172.21.15.86:0/2515580476 shutdown_connections 2024-04-20T21:17:58.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.083+0000 7fa095e00640 1 --2- 172.21.15.86:0/2515580476 >> v2:172.21.15.144:3300/0 conn(0x7fa0900af7b0 0x7fa0900adda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:58.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.083+0000 7fa095e00640 1 --2- 172.21.15.86:0/2515580476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa09015c010 0x7fa09015c3f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:58.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.083+0000 7fa095e00640 1 -- 172.21.15.86:0/2515580476 >> 172.21.15.86:0/2515580476 conn(0x7fa0900d0170 msgr2=0x7fa0900d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:58.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.084+0000 7fa095e00640 1 -- 172.21.15.86:0/2515580476 shutdown_connections 2024-04-20T21:17:58.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.084+0000 7fa095e00640 1 -- 172.21.15.86:0/2515580476 wait complete. 2024-04-20T21:17:58.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.085+0000 7fa095e00640 1 Processor -- start 2024-04-20T21:17:58.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.085+0000 7fa095e00640 1 -- start start 2024-04-20T21:17:58.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.086+0000 7fa095e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa09006f4c0 0x7fa0901f5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:58.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.086+0000 7fa095e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0900af7b0 0x7fa0901f2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:58.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.086+0000 7fa095e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa09015c010 0x7fa09014e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:58.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.086+0000 7fa095e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa0900ae2e0 con 0x7fa09015c010 2024-04-20T21:17:58.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.086+0000 7fa095e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa0900aefb0 con 0x7fa09006f4c0 2024-04-20T21:17:58.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.086+0000 7fa095e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa0900ae460 con 0x7fa0900af7b0 2024-04-20T21:17:58.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.086+0000 7fa095400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa09015c010 0x7fa09014e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:58.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.087+0000 7fa095400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa09015c010 0x7fa09014e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57632/0 (socket says 172.21.15.86:57632) 2024-04-20T21:17:58.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.087+0000 7fa095400640 1 -- 172.21.15.86:0/3359318137 learned_addr learned my addr 172.21.15.86:0/3359318137 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:58.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.087+0000 7fa094a00640 1 --2- 172.21.15.86:0/3359318137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa09006f4c0 0x7fa0901f5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:58.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.087+0000 7fa08fe00640 1 --2- 172.21.15.86:0/3359318137 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0900af7b0 0x7fa0901f2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:58.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.087+0000 7fa095400640 1 -- 172.21.15.86:0/3359318137 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0900af7b0 msgr2=0x7fa0901f2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:58.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.087+0000 7fa095400640 1 --2- 172.21.15.86:0/3359318137 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0900af7b0 0x7fa0901f2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:58.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.087+0000 7fa095400640 1 -- 172.21.15.86:0/3359318137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa09006f4c0 msgr2=0x7fa0901f5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:58.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.087+0000 7fa095400640 1 --2- 172.21.15.86:0/3359318137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa09006f4c0 0x7fa0901f5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:58.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.087+0000 7fa095400640 1 -- 172.21.15.86:0/3359318137 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0901514f0 con 0x7fa09015c010 2024-04-20T21:17:58.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.087+0000 7fa095400640 1 --2- 172.21.15.86:0/3359318137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa09015c010 0x7fa09014e290 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa08800c7b0 tx=0x7fa08800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:58.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.087+0000 7fa08cc00640 1 -- 172.21.15.86:0/3359318137 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa088017070 con 0x7fa09015c010 2024-04-20T21:17:58.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.088+0000 7fa08cc00640 1 -- 172.21.15.86:0/3359318137 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa088003da0 con 0x7fa09015c010 2024-04-20T21:17:58.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.088+0000 7fa095e00640 1 -- 172.21.15.86:0/3359318137 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0901517d0 con 0x7fa09015c010 2024-04-20T21:17:58.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.088+0000 7fa08cc00640 1 -- 172.21.15.86:0/3359318137 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0880052b0 con 0x7fa09015c010 2024-04-20T21:17:58.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.088+0000 7fa095e00640 1 -- 172.21.15.86:0/3359318137 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa090151d90 con 0x7fa09015c010 2024-04-20T21:17:58.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.089+0000 7fa095e00640 1 -- 172.21.15.86:0/3359318137 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa09006e180 con 0x7fa09015c010 2024-04-20T21:17:58.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.090+0000 7fa08cc00640 1 -- 172.21.15.86:0/3359318137 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa08801a070 con 0x7fa09015c010 2024-04-20T21:17:58.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.090+0000 7fa08cc00640 1 --2- 172.21.15.86:0/3359318137 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa07006c8f0 0x7fa07006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:58.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.090+0000 7fa08cc00640 1 -- 172.21.15.86:0/3359318137 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa08808b170 con 0x7fa09015c010 2024-04-20T21:17:58.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.091+0000 7fa094a00640 1 --2- 172.21.15.86:0/3359318137 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa07006c8f0 0x7fa07006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:58.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.091+0000 7fa094a00640 1 --2- 172.21.15.86:0/3359318137 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa07006c8f0 0x7fa07006edb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa084036000 tx=0x7fa08403c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:58.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.095+0000 7fa08cc00640 1 -- 172.21.15.86:0/3359318137 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa09006e180 con 0x7fa09015c010 2024-04-20T21:17:58.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.279+0000 7fa095e00640 1 -- 172.21.15.86:0/3359318137 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa09004fcf0 con 0x7fa09015c010 2024-04-20T21:17:58.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.281+0000 7fa08cc00640 1 -- 172.21.15.86:0/3359318137 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7fa088059630 con 0x7fa09015c010 2024-04-20T21:17:58.282 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:17:58.282 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":20,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:17:53.274462+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:17:49.718799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:17:50.683477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:17:58.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.284+0000 7fa095e00640 1 -- 172.21.15.86:0/3359318137 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa07006c8f0 msgr2=0x7fa07006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:58.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.284+0000 7fa095e00640 1 --2- 172.21.15.86:0/3359318137 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa07006c8f0 0x7fa07006edb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa084036000 tx=0x7fa08403c040 comp rx=0 tx=0).stop 2024-04-20T21:17:58.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.284+0000 7fa095e00640 1 -- 172.21.15.86:0/3359318137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa09015c010 msgr2=0x7fa09014e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:58.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.284+0000 7fa095e00640 1 --2- 172.21.15.86:0/3359318137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa09015c010 0x7fa09014e290 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa08800c7b0 tx=0x7fa08800cc70 comp rx=0 tx=0).stop 2024-04-20T21:17:58.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.284+0000 7fa095e00640 1 -- 172.21.15.86:0/3359318137 shutdown_connections 2024-04-20T21:17:58.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.284+0000 7fa095e00640 1 --2- 172.21.15.86:0/3359318137 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa07006c8f0 0x7fa07006edb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:58.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.284+0000 7fa095e00640 1 --2- 172.21.15.86:0/3359318137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa09015c010 0x7fa09014e290 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:58.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.284+0000 7fa095e00640 1 --2- 172.21.15.86:0/3359318137 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa0900af7b0 0x7fa0901f2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:58.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.284+0000 7fa095e00640 1 --2- 172.21.15.86:0/3359318137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa09006f4c0 0x7fa0901f5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:58.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.284+0000 7fa095e00640 1 -- 172.21.15.86:0/3359318137 >> 172.21.15.86:0/3359318137 conn(0x7fa0900d0170 msgr2=0x7fa090071b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:58.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.284+0000 7fa095e00640 1 -- 172.21.15.86:0/3359318137 shutdown_connections 2024-04-20T21:17:58.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.284+0000 7fa095e00640 1 -- 172.21.15.86:0/3359318137 wait complete. 2024-04-20T21:17:58.302 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-20T21:17:58.303 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:17:58.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.500+0000 7fbe1ee00640 1 Processor -- start 2024-04-20T21:17:58.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.500+0000 7fbe1ee00640 1 -- start start 2024-04-20T21:17:58.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.502+0000 7fbe1ee00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbe18162370 0x7fbe1815df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:58.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.502+0000 7fbe1ee00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe18162850 con 0x7fbe18160170 2024-04-20T21:17:58.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.502+0000 7fbe1ee00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe18162b30 con 0x7fbe1809a9e0 2024-04-20T21:17:58.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.502+0000 7fbe1ee00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fbe18162e10 con 0x7fbe18162370 2024-04-20T21:17:58.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.502+0000 7fbe1d000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbe18162370 0x7fbe1815df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:58.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.502+0000 7fbe1e400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fbe18160170 0x7fbe1815e4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:43482/0 (socket says 172.21.15.86:43482) 2024-04-20T21:17:58.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.502+0000 7fbe1e400640 1 -- 172.21.15.86:0/4108685684 learned_addr learned my addr 172.21.15.86:0/4108685684 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:17:58.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.503+0000 7fbe1d000640 1 -- 172.21.15.86:0/4108685684 >> v1:172.21.15.144:6789/0 conn(0x7fbe1809a9e0 legacy=0x7fbe1809adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:58.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.503+0000 7fbe1d000640 1 -- 172.21.15.86:0/4108685684 >> v1:172.21.15.86:6789/0 conn(0x7fbe18160170 legacy=0x7fbe1815e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:58.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.503+0000 7fbe1d000640 1 -- 172.21.15.86:0/4108685684 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe181630f0 con 0x7fbe18162370 2024-04-20T21:17:58.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.503+0000 7fbe17e00640 1 -- 172.21.15.86:0/4108685684 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 729074953 0 0) 0x7fbe18162850 con 0x7fbe18160170 2024-04-20T21:17:58.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.504+0000 7fbe1d000640 1 --2- 172.21.15.86:0/4108685684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe18162370 0x7fbe1815df90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbe04009d90 tx=0x7fbe0402bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=facd106feeb3e3c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:58.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.504+0000 7fbe17e00640 1 -- 172.21.15.86:0/4108685684 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe04039070 con 0x7fbe18162370 2024-04-20T21:17:58.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.504+0000 7fbe17e00640 1 -- 172.21.15.86:0/4108685684 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe0402cd80 con 0x7fbe18162370 2024-04-20T21:17:58.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.504+0000 7fbe17e00640 1 -- 172.21.15.86:0/4108685684 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe04034660 con 0x7fbe18162370 2024-04-20T21:17:58.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.505+0000 7fbe1ee00640 1 -- 172.21.15.86:0/4108685684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe18162370 msgr2=0x7fbe1815df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:58.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.505+0000 7fbe1ee00640 1 --2- 172.21.15.86:0/4108685684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe18162370 0x7fbe1815df90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbe04009d90 tx=0x7fbe0402bfd0 comp rx=0 tx=0).stop 2024-04-20T21:17:58.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.505+0000 7fbe1ee00640 1 -- 172.21.15.86:0/4108685684 shutdown_connections 2024-04-20T21:17:58.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.505+0000 7fbe1ee00640 1 --2- 172.21.15.86:0/4108685684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe18162370 0x7fbe1815df90 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:58.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.505+0000 7fbe1ee00640 1 -- 172.21.15.86:0/4108685684 >> 172.21.15.86:0/4108685684 conn(0x7fbe180b3e10 msgr2=0x7fbe180b4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:58.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.505+0000 7fbe1ee00640 1 -- 172.21.15.86:0/4108685684 shutdown_connections 2024-04-20T21:17:58.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.505+0000 7fbe1ee00640 1 -- 172.21.15.86:0/4108685684 wait complete. 2024-04-20T21:17:58.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.507+0000 7fbe1ee00640 1 Processor -- start 2024-04-20T21:17:58.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.507+0000 7fbe1ee00640 1 -- start start 2024-04-20T21:17:58.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.508+0000 7fbe1ee00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe1809a9e0 0x7fbe181548e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:58.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.508+0000 7fbe1ee00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe18160170 0x7fbe180c5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:58.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.508+0000 7fbe1ee00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe18162370 0x7fbe180b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:58.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.508+0000 7fbe1ee00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbe18161fc0 con 0x7fbe1809a9e0 2024-04-20T21:17:58.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.508+0000 7fbe1ee00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1815fbf0 con 0x7fbe18162370 2024-04-20T21:17:58.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.508+0000 7fbe1ee00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbe18162140 con 0x7fbe18160170 2024-04-20T21:17:58.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.508+0000 7fbe1da00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe1809a9e0 0x7fbe181548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:58.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.508+0000 7fbe1da00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe1809a9e0 0x7fbe181548e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57650/0 (socket says 172.21.15.86:57650) 2024-04-20T21:17:58.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.508+0000 7fbe1da00640 1 -- 172.21.15.86:0/1138677917 learned_addr learned my addr 172.21.15.86:0/1138677917 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:58.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.508+0000 7fbe1da00640 1 -- 172.21.15.86:0/1138677917 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe18160170 msgr2=0x7fbe180c5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:58.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.509+0000 7fbe1da00640 1 --2- 172.21.15.86:0/1138677917 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe18160170 0x7fbe180c5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:58.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.509+0000 7fbe1da00640 1 -- 172.21.15.86:0/1138677917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe18162370 msgr2=0x7fbe180b4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:58.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.509+0000 7fbe1da00640 1 --2- 172.21.15.86:0/1138677917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe18162370 0x7fbe180b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:58.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.509+0000 7fbe1da00640 1 -- 172.21.15.86:0/1138677917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe180c6820 con 0x7fbe1809a9e0 2024-04-20T21:17:58.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.509+0000 7fbe1da00640 1 --2- 172.21.15.86:0/1138677917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe1809a9e0 0x7fbe181548e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbe0800c450 tx=0x7fbe0800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:58.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.509+0000 7fbe15600640 1 -- 172.21.15.86:0/1138677917 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe080052d0 con 0x7fbe1809a9e0 2024-04-20T21:17:58.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.509+0000 7fbe15600640 1 -- 172.21.15.86:0/1138677917 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe08005470 con 0x7fbe1809a9e0 2024-04-20T21:17:58.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.509+0000 7fbe15600640 1 -- 172.21.15.86:0/1138677917 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe08012870 con 0x7fbe1809a9e0 2024-04-20T21:17:58.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.509+0000 7fbe1ee00640 1 -- 172.21.15.86:0/1138677917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe180cd7d0 con 0x7fbe1809a9e0 2024-04-20T21:17:58.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.509+0000 7fbe1ee00640 1 -- 172.21.15.86:0/1138677917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe180cdd90 con 0x7fbe1809a9e0 2024-04-20T21:17:58.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.511+0000 7fbe1ee00640 1 -- 172.21.15.86:0/1138677917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe180cdf90 con 0x7fbe1809a9e0 2024-04-20T21:17:58.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.511+0000 7fbe15600640 1 -- 172.21.15.86:0/1138677917 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbe08012a10 con 0x7fbe1809a9e0 2024-04-20T21:17:58.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.512+0000 7fbe15600640 1 --2- 172.21.15.86:0/1138677917 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbdf806c920 0x7fbdf806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:58.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.512+0000 7fbe15600640 1 -- 172.21.15.86:0/1138677917 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbe0808b6b0 con 0x7fbe1809a9e0 2024-04-20T21:17:58.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.513+0000 7fbe1d000640 1 --2- 172.21.15.86:0/1138677917 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbdf806c920 0x7fbdf806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:58.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.513+0000 7fbe1d000640 1 --2- 172.21.15.86:0/1138677917 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbdf806c920 0x7fbdf806ede0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbe04036000 tx=0x7fbe0402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:58.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.517+0000 7fbe15600640 1 -- 172.21.15.86:0/1138677917 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbe180cdf90 con 0x7fbe1809a9e0 2024-04-20T21:17:58.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.721+0000 7fbe1ee00640 1 -- 172.21.15.86:0/1138677917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbe1804fcf0 con 0x7fbe1809a9e0 2024-04-20T21:17:58.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.723+0000 7fbe15600640 1 -- 172.21.15.86:0/1138677917 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7fbe08059b70 con 0x7fbe1809a9e0 2024-04-20T21:17:58.724 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:17:58.724 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":20,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:17:53.274462+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:17:49.718799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:17:50.683477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:17:58.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.726+0000 7fbe1ee00640 1 -- 172.21.15.86:0/1138677917 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbdf806c920 msgr2=0x7fbdf806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:58.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.726+0000 7fbe1ee00640 1 --2- 172.21.15.86:0/1138677917 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbdf806c920 0x7fbdf806ede0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbe04036000 tx=0x7fbe0402e000 comp rx=0 tx=0).stop 2024-04-20T21:17:58.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.726+0000 7fbe1ee00640 1 -- 172.21.15.86:0/1138677917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe1809a9e0 msgr2=0x7fbe181548e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:58.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.726+0000 7fbe1ee00640 1 --2- 172.21.15.86:0/1138677917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe1809a9e0 0x7fbe181548e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbe0800c450 tx=0x7fbe0800c910 comp rx=0 tx=0).stop 2024-04-20T21:17:58.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.726+0000 7fbe1ee00640 1 -- 172.21.15.86:0/1138677917 shutdown_connections 2024-04-20T21:17:58.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.726+0000 7fbe1ee00640 1 --2- 172.21.15.86:0/1138677917 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbdf806c920 0x7fbdf806ede0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:58.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.726+0000 7fbe1ee00640 1 --2- 172.21.15.86:0/1138677917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe18162370 0x7fbe180b4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:58.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.726+0000 7fbe1ee00640 1 --2- 172.21.15.86:0/1138677917 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe18160170 0x7fbe180c5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:58.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.726+0000 7fbe1ee00640 1 --2- 172.21.15.86:0/1138677917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe1809a9e0 0x7fbe181548e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:58.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.726+0000 7fbe1ee00640 1 -- 172.21.15.86:0/1138677917 >> 172.21.15.86:0/1138677917 conn(0x7fbe180b3e10 msgr2=0x7fbe1806d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:58.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.726+0000 7fbe1ee00640 1 -- 172.21.15.86:0/1138677917 shutdown_connections 2024-04-20T21:17:58.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:58.726+0000 7fbe1ee00640 1 -- 172.21.15.86:0/1138677917 wait complete. 2024-04-20T21:17:58.744 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-20T21:17:58.744 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-20T21:17:58.744 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-20T21:17:58.745 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-20T21:17:58.745 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-20T21:17:58.745 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-20T21:17:58.746 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-20T21:17:58.746 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-20T21:17:59.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.033+0000 7f41bac00640 1 Processor -- start 2024-04-20T21:17:59.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.033+0000 7f41bac00640 1 -- start start 2024-04-20T21:17:59.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.034+0000 7f001be00640 1 Processor -- start 2024-04-20T21:17:59.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.034+0000 7f001be00640 1 -- start start 2024-04-20T21:17:59.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.035+0000 7f41bac00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f41ac0e3160 0x7f41ac0e7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.035+0000 7f41bac00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f41ac03c0a0 0x7f41ac058050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.035+0000 7f41bac00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41ac058590 con 0x7f41ac053cb0 2024-04-20T21:17:59.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.035+0000 7f41bac00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f41ac007b70 con 0x7f41ac0e3160 2024-04-20T21:17:59.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.035+0000 7f41bac00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f41ac058810 con 0x7f41ac03c0a0 2024-04-20T21:17:59.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.036+0000 7fa35d000640 1 Processor -- start 2024-04-20T21:17:59.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.036+0000 7f001be00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f001c0b32a0 0x7f001c0b1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.036+0000 7f001be00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f001c0b19b0 con 0x7f001c0c7990 2024-04-20T21:17:59.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.036+0000 7f001be00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f001c0d25c0 con 0x7f001c0b3680 2024-04-20T21:17:59.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.036+0000 7f001be00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f001c0cfce0 con 0x7f001c0b32a0 2024-04-20T21:17:59.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.037+0000 7f001a000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f001c0c7990 0x7f001c0cf5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:43496/0 (socket says 172.21.15.86:43496) 2024-04-20T21:17:59.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.037+0000 7f001a000640 1 -- 172.21.15.86:0/2301762567 learned_addr learned my addr 172.21.15.86:0/2301762567 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:17:59.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.037+0000 7f001aa00640 1 --2- 172.21.15.86:0/2301762567 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f001c0b32a0 0x7f001c0b1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.037+0000 7f41b9800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f41ac0e3160 0x7f41ac0e7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.037+0000 7f41b9800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f41ac0e3160 0x7f41ac0e7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57660/0 (socket says 172.21.15.86:57660) 2024-04-20T21:17:59.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.037+0000 7f41b9800640 1 -- 172.21.15.86:0/66220431 learned_addr learned my addr 172.21.15.86:0/66220431 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:59.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.036+0000 7fa35d000640 1 -- start start 2024-04-20T21:17:59.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.037+0000 7f0019600640 1 -- 172.21.15.86:0/2301762567 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2551065162 0 0) 0x7f001c0b19b0 con 0x7f001c0c7990 2024-04-20T21:17:59.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.038+0000 7f0019600640 1 -- 172.21.15.86:0/2301762567 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0000003520 con 0x7f001c0c7990 2024-04-20T21:17:59.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.038+0000 7f0019600640 1 -- 172.21.15.86:0/2301762567 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2911509949 0 0) 0x7f001c0d25c0 con 0x7f001c0b3680 2024-04-20T21:17:59.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.038+0000 7f0019600640 1 -- 172.21.15.86:0/2301762567 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f001c0b19b0 con 0x7f001c0b3680 2024-04-20T21:17:59.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.038+0000 7fa35d000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa35815eaf0 0x7fa3580d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.038+0000 7fa35d000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3580b13c0 0x7fa3580cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.038+0000 7fa35d000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa3580d44e0 0x7fa3580d4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.038+0000 7fa35d000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa3580d06f0 con 0x7fa3580d44e0 2024-04-20T21:17:59.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.038+0000 7fa35d000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fa3580d4e80 con 0x7fa35815eaf0 2024-04-20T21:17:59.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.038+0000 7fa35d000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa3580b4650 con 0x7fa3580b13c0 2024-04-20T21:17:59.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.038+0000 7f41b9800640 1 -- 172.21.15.86:0/66220431 >> v2:172.21.15.144:3300/0 conn(0x7f41ac03c0a0 msgr2=0x7f41ac058050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:59.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.038+0000 7f41b8e00640 1 --2- 172.21.15.86:0/66220431 >> v2:172.21.15.144:3300/0 conn(0x7f41ac03c0a0 0x7f41ac058050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.038+0000 7f41b9800640 1 --2- 172.21.15.86:0/66220431 >> v2:172.21.15.144:3300/0 conn(0x7f41ac03c0a0 0x7f41ac058050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.038+0000 7f41b9800640 1 -- 172.21.15.86:0/66220431 >> v1:172.21.15.86:6789/0 conn(0x7f41ac053cb0 legacy=0x7f41ac054100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.038+0000 7f9d39800640 1 Processor -- start 2024-04-20T21:17:59.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.038+0000 7fa355e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa35815eaf0 0x7fa3580d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.038+0000 7f0019600640 1 -- 172.21.15.86:0/2301762567 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1652980598 0 0) 0x7f0000003520 con 0x7f001c0c7990 2024-04-20T21:17:59.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.038+0000 7f0019600640 1 -- 172.21.15.86:0/2301762567 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f001c0b32a0 msgr2=0x7f001c0b1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:59.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.038+0000 7f0019600640 1 --2- 172.21.15.86:0/2301762567 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f001c0b32a0 0x7f001c0b1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.038+0000 7f0019600640 1 -- 172.21.15.86:0/2301762567 >> v1:172.21.15.144:6789/0 conn(0x7f001c0b3680 legacy=0x7f001c166ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.038+0000 7f41b9800640 1 -- 172.21.15.86:0/66220431 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41ac058af0 con 0x7f41ac0e3160 2024-04-20T21:17:59.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.038+0000 7f0019600640 1 -- 172.21.15.86:0/2301762567 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f001c0d28a0 con 0x7f001c0c7990 2024-04-20T21:17:59.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.039+0000 7f9d39800640 1 -- start start 2024-04-20T21:17:59.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.039+0000 7f0019600640 1 -- 172.21.15.86:0/2301762567 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f000c001670 con 0x7f001c0c7990 2024-04-20T21:17:59.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.039+0000 7f0019600640 1 -- 172.21.15.86:0/2301762567 >> v1:172.21.15.86:6789/0 conn(0x7f001c0c7990 legacy=0x7f001c0cf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.039+0000 7f0019600640 1 --2- 172.21.15.86:0/2301762567 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0000003b70 0x7f0000003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.039+0000 7f0019600640 1 --2- 172.21.15.86:0/2301762567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0000004800 0x7f0000004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.039+0000 7f0019600640 1 --2- 172.21.15.86:0/2301762567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00000051c0 0x7f000000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.039+0000 7f0019600640 1 -- 172.21.15.86:0/2301762567 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f001c0cfce0 con 0x7f00000051c0 2024-04-20T21:17:59.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.039+0000 7f0019600640 1 -- 172.21.15.86:0/2301762567 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f000000c090 con 0x7f0000004800 2024-04-20T21:17:59.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.039+0000 7f0019600640 1 -- 172.21.15.86:0/2301762567 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f000000c290 con 0x7f0000003b70 2024-04-20T21:17:59.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.039+0000 7f001aa00640 1 --2- 172.21.15.86:0/2301762567 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f001c0b32a0 0x7f001c0b1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:59.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.039+0000 7fa355400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3580b13c0 0x7fa3580cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.039+0000 7fa355e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa35815eaf0 0x7fa3580d34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.86:57846/0 (socket says 172.21.15.86:57846) 2024-04-20T21:17:59.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.039+0000 7fa355e00640 1 -- 172.21.15.86:0/723105699 learned_addr learned my addr 172.21.15.86:0/723105699 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:59.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.039+0000 7fa356800640 1 --2- 172.21.15.86:0/723105699 >> v2:172.21.15.86:3300/0 conn(0x7fa3580d44e0 0x7fa3580d4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.039+0000 7f41b8e00640 1 --2- 172.21.15.86:0/66220431 >> v2:172.21.15.144:3300/0 conn(0x7f41ac03c0a0 0x7f41ac058050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:17:59.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.040+0000 7f001aa00640 1 --2- 172.21.15.86:0/2301762567 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0000003b70 0x7f0000003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.040+0000 7f001a000640 1 --2- 172.21.15.86:0/2301762567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0000004800 0x7f0000004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.040+0000 7f001b400640 1 --2- 172.21.15.86:0/2301762567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00000051c0 0x7f000000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.040+0000 7fa355e00640 1 -- 172.21.15.86:0/723105699 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3580b13c0 msgr2=0x7fa3580cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:59.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.040+0000 7fa355e00640 1 --2- 172.21.15.86:0/723105699 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3580b13c0 0x7fa3580cfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.040+0000 7fa355e00640 1 -- 172.21.15.86:0/723105699 >> v2:172.21.15.86:3300/0 conn(0x7fa3580d44e0 msgr2=0x7fa3580d4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:59.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.040+0000 7fa355e00640 1 --2- 172.21.15.86:0/723105699 >> v2:172.21.15.86:3300/0 conn(0x7fa3580d44e0 0x7fa3580d4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.041+0000 7fa355e00640 1 -- 172.21.15.86:0/723105699 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3580b4930 con 0x7fa35815eaf0 2024-04-20T21:17:59.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.041+0000 7fa355400640 1 --2- 172.21.15.86:0/723105699 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3580b13c0 0x7fa3580cfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:59.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.041+0000 7f1750e00640 1 Processor -- start 2024-04-20T21:17:59.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.041+0000 7f41b9800640 1 --2- 172.21.15.86:0/66220431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41ac0e3160 0x7f41ac0e7780 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f41ac053700 tx=0x7f41a402b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad7d936968bfa07 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.042+0000 7f9d39800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d3415dd70 0x7f9d3415e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.042+0000 7f9d39800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9d340b3480 0x7f9d340ce640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.042+0000 7f9d39800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9d34071d40 0x7f9d340721d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.041+0000 7f001aa00640 1 -- 172.21.15.86:0/2301762567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0000004800 msgr2=0x7f0000004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:59.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.041+0000 7f001aa00640 1 --2- 172.21.15.86:0/2301762567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0000004800 0x7f0000004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.041+0000 7f001aa00640 1 -- 172.21.15.86:0/2301762567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00000051c0 msgr2=0x7f000000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:59.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.041+0000 7f001aa00640 1 --2- 172.21.15.86:0/2301762567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00000051c0 0x7f000000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.041+0000 7f001aa00640 1 -- 172.21.15.86:0/2301762567 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f001c0b19b0 con 0x7f0000003b70 2024-04-20T21:17:59.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.041+0000 7f41b3e00640 1 -- 172.21.15.86:0/66220431 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a4034070 con 0x7f41ac0e3160 2024-04-20T21:17:59.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.041+0000 7f41b3e00640 1 -- 172.21.15.86:0/66220431 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41a4030870 con 0x7f41ac0e3160 2024-04-20T21:17:59.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.042+0000 7f9d33400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9d34071d40 0x7f9d340721d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.042+0000 7f9d33400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9d34071d40 0x7f9d340721d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57692/0 (socket says 172.21.15.86:57692) 2024-04-20T21:17:59.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.042+0000 7f9d33400640 1 -- 172.21.15.86:0/3579968778 learned_addr learned my addr 172.21.15.86:0/3579968778 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:59.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.042+0000 7f9d39800640 1 -- 172.21.15.86:0/3579968778 --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9d340b48d0 con 0x7f9d34071d40 2024-04-20T21:17:59.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.042+0000 7f9d32000640 1 --2- 172.21.15.86:0/3579968778 >> v2:172.21.15.144:3300/0 conn(0x7f9d340b3480 0x7f9d340ce640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.041+0000 7f41b3e00640 1 -- 172.21.15.86:0/66220431 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a4030c40 con 0x7f41ac0e3160 2024-04-20T21:17:59.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.042+0000 7f41bac00640 1 -- 172.21.15.86:0/66220431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41ac0e3160 msgr2=0x7f41ac0e7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.042+0000 7f41bac00640 1 --2- 172.21.15.86:0/66220431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41ac0e3160 0x7f41ac0e7780 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f41ac053700 tx=0x7f41a402b9b0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.043+0000 7f41bac00640 1 -- 172.21.15.86:0/66220431 shutdown_connections 2024-04-20T21:17:59.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.043+0000 7f41bac00640 1 --2- 172.21.15.86:0/66220431 >> v2:172.21.15.144:3300/0 conn(0x7f41ac03c0a0 0x7f41ac058050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.043+0000 7f41bac00640 1 --2- 172.21.15.86:0/66220431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41ac0e3160 0x7f41ac0e7780 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.043+0000 7f41bac00640 1 -- 172.21.15.86:0/66220431 >> 172.21.15.86:0/66220431 conn(0x7f41ac058ee0 msgr2=0x7f41ac0592c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:59.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.043+0000 7f41bac00640 1 -- 172.21.15.86:0/66220431 shutdown_connections 2024-04-20T21:17:59.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.043+0000 7f41bac00640 1 -- 172.21.15.86:0/66220431 wait complete. 2024-04-20T21:17:59.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.041+0000 7f1750e00640 1 -- start start 2024-04-20T21:17:59.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.043+0000 7f1750e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f174403e830 0x7f1744053be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.043+0000 7f1750e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1744054120 0x7f17440068c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.043+0000 7f1750e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1744058560 con 0x7f17440e30b0 2024-04-20T21:17:59.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.043+0000 7f001aa00640 1 --2- 172.21.15.86:0/2301762567 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0000003b70 0x7f0000003f90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f001c073b30 tx=0x7f0000005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6dbf2e9a0e1dff55 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.043+0000 7fa355e00640 1 --2- 172.21.15.86:0/723105699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa35815eaf0 0x7fa3580d34b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa3580b6230 tx=0x7fa34802b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5b49d4e52b8204d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.043+0000 7f9d39800640 1 -- 172.21.15.86:0/3579968778 --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f9d3415e150 con 0x7f9d340b3480 2024-04-20T21:17:59.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.043+0000 7f9d39800640 1 -- 172.21.15.86:0/3579968778 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9d3415e980 con 0x7f9d3415dd70 2024-04-20T21:17:59.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.043+0000 7f9d32a00640 1 --2- 172.21.15.86:0/3579968778 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d3415dd70 0x7f9d3415e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.043+0000 7f12d4a00640 1 Processor -- start 2024-04-20T21:17:59.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.043+0000 7f1750e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f1744054590 con 0x7f1744054120 2024-04-20T21:17:59.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.043+0000 7f1750e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f1744058840 con 0x7f174403e830 2024-04-20T21:17:59.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.043+0000 7f9d33400640 1 -- 172.21.15.86:0/3579968778 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d3415dd70 msgr2=0x7f9d3415e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:59.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.043+0000 7f9d33400640 1 --2- 172.21.15.86:0/3579968778 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d3415dd70 0x7f9d3415e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.043+0000 7f9d33400640 1 -- 172.21.15.86:0/3579968778 >> v2:172.21.15.144:3300/0 conn(0x7f9d340b3480 msgr2=0x7f9d340ce640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:59.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.043+0000 7f9d33400640 1 --2- 172.21.15.86:0/3579968778 >> v2:172.21.15.144:3300/0 conn(0x7f9d340b3480 0x7f9d340ce640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.043+0000 7f9d33400640 1 -- 172.21.15.86:0/3579968778 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d3415ec60 con 0x7f9d34071d40 2024-04-20T21:17:59.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.044+0000 7f0019600640 1 -- 172.21.15.86:0/2301762567 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0014009d60 con 0x7f0000003b70 2024-04-20T21:17:59.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.044+0000 7fa354a00640 1 -- 172.21.15.86:0/723105699 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa348034070 con 0x7fa35815eaf0 2024-04-20T21:17:59.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.044+0000 7f1749400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f174403e830 0x7f1744053be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.044+0000 7f1749400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f174403e830 0x7f1744053be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.86:57882/0 (socket says 172.21.15.86:57882) 2024-04-20T21:17:59.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.044+0000 7f1749400640 1 -- 172.21.15.86:0/291716467 learned_addr learned my addr 172.21.15.86:0/291716467 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:59.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.044+0000 7f0019600640 1 -- 172.21.15.86:0/2301762567 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f001402cd20 con 0x7f0000003b70 2024-04-20T21:17:59.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.044+0000 7fa354a00640 1 -- 172.21.15.86:0/723105699 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa348030860 con 0x7fa35815eaf0 2024-04-20T21:17:59.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.044+0000 7fa354a00640 1 -- 172.21.15.86:0/723105699 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa348030c30 con 0x7fa35815eaf0 2024-04-20T21:17:59.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.044+0000 7f174a800640 1 --2- 172.21.15.86:0/291716467 >> v2:172.21.15.86:3300/0 conn(0x7f1744054120 0x7f17440068c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.044+0000 7f12d4a00640 1 -- start start 2024-04-20T21:17:59.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.045+0000 7f1748a00640 1 -- 172.21.15.86:0/291716467 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 322182905 0 0) 0x7f1744058560 con 0x7f17440e30b0 2024-04-20T21:17:59.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.045+0000 7f1749400640 1 -- 172.21.15.86:0/291716467 >> v2:172.21.15.86:3300/0 conn(0x7f1744054120 msgr2=0x7f17440068c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:59.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.045+0000 7f1749400640 1 --2- 172.21.15.86:0/291716467 >> v2:172.21.15.86:3300/0 conn(0x7f1744054120 0x7f17440068c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.045+0000 7f1749400640 1 -- 172.21.15.86:0/291716467 >> v1:172.21.15.86:6789/0 conn(0x7f17440e30b0 legacy=0x7f1744057d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.045+0000 7f001be00640 1 -- 172.21.15.86:0/2301762567 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0000003b70 msgr2=0x7f0000003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.045+0000 7f001be00640 1 --2- 172.21.15.86:0/2301762567 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0000003b70 0x7f0000003f90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f001c073b30 tx=0x7f0000005830 comp rx=0 tx=0).stop 2024-04-20T21:17:59.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.045+0000 7fa35d000640 1 -- 172.21.15.86:0/723105699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa35815eaf0 msgr2=0x7fa3580d34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.045+0000 7f001b400640 1 -- 172.21.15.86:0/2301762567 reap_dead start 2024-04-20T21:17:59.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.045+0000 7f1749400640 1 -- 172.21.15.86:0/291716467 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1744058b20 con 0x7f174403e830 2024-04-20T21:17:59.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.045+0000 7f174a800640 1 --2- 172.21.15.86:0/291716467 >> v2:172.21.15.86:3300/0 conn(0x7f1744054120 0x7f17440068c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:17:59.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.045+0000 7f9d33400640 1 --2- 172.21.15.86:0/3579968778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d34071d40 0x7f9d340721d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9d340731e0 tx=0x7f9d2002ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=634fb9a9010ed3f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.045+0000 7fa35d000640 1 --2- 172.21.15.86:0/723105699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa35815eaf0 0x7fa3580d34b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa3580b6230 tx=0x7fa34802b9a0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.046+0000 7fa35d000640 1 -- 172.21.15.86:0/723105699 shutdown_connections 2024-04-20T21:17:59.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.046+0000 7fa35d000640 1 --2- 172.21.15.86:0/723105699 >> v2:172.21.15.86:3300/0 conn(0x7fa3580d44e0 0x7fa3580d4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.046+0000 7fa35d000640 1 --2- 172.21.15.86:0/723105699 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3580b13c0 0x7fa3580cfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.046+0000 7fa35d000640 1 --2- 172.21.15.86:0/723105699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa35815eaf0 0x7fa3580d34b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.046+0000 7fa35d000640 1 -- 172.21.15.86:0/723105699 >> 172.21.15.86:0/723105699 conn(0x7fa35809da60 msgr2=0x7fa35808bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:59.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.046+0000 7f092d400640 1 Processor -- start 2024-04-20T21:17:59.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.046+0000 7f001be00640 1 -- 172.21.15.86:0/2301762567 shutdown_connections 2024-04-20T21:17:59.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.046+0000 7f001be00640 1 -- 172.21.15.86:0/2301762567 >> 172.21.15.86:0/2301762567 conn(0x7f001c08bf70 msgr2=0x7f001c0d4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:59.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.046+0000 7fa35d000640 1 -- 172.21.15.86:0/723105699 shutdown_connections 2024-04-20T21:17:59.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.046+0000 7f9d31600640 1 -- 172.21.15.86:0/3579968778 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d20030840 con 0x7f9d34071d40 2024-04-20T21:17:59.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.046+0000 7f9d31600640 1 -- 172.21.15.86:0/3579968778 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d200309e0 con 0x7f9d34071d40 2024-04-20T21:17:59.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.046+0000 7f9d31600640 1 -- 172.21.15.86:0/3579968778 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d20030db0 con 0x7f9d34071d40 2024-04-20T21:17:59.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.046+0000 7f001be00640 1 -- 172.21.15.86:0/2301762567 shutdown_connections 2024-04-20T21:17:59.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.046+0000 7f001be00640 1 -- 172.21.15.86:0/2301762567 wait complete. 2024-04-20T21:17:59.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.046+0000 7f092d400640 1 -- start start 2024-04-20T21:17:59.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.047+0000 7f1749400640 1 --2- 172.21.15.86:0/291716467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f174403e830 0x7f1744053be0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1744008670 tx=0x7f173802ba40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=358729682342ea44 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.046+0000 7fa35d000640 1 -- 172.21.15.86:0/723105699 wait complete. 2024-04-20T21:17:59.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.047+0000 7f12d4a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f12d0166eb0 0x7f12d006db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.047+0000 7f12d4a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f12d0167290 0x7f12d006e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.047+0000 7f12d4a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12d0072f60 con 0x7f12d006bda0 2024-04-20T21:17:59.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.047+0000 7f12d4a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f12d00b3b60 con 0x7f12d0166eb0 2024-04-20T21:17:59.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.047+0000 7f12d4a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f12d00b3230 con 0x7f12d0167290 2024-04-20T21:17:59.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.047+0000 7f1748a00640 1 -- 172.21.15.86:0/291716467 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173802f860 con 0x7f174403e830 2024-04-20T21:17:59.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.047+0000 7f1748a00640 1 -- 172.21.15.86:0/291716467 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f173802fa00 con 0x7f174403e830 2024-04-20T21:17:59.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.048+0000 7f1748a00640 1 -- 172.21.15.86:0/291716467 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173802fdd0 con 0x7f174403e830 2024-04-20T21:17:59.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.048+0000 7f41bac00640 1 Processor -- start 2024-04-20T21:17:59.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.048+0000 7f41bac00640 1 -- start start 2024-04-20T21:17:59.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.048+0000 7f12cf200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f12d006bda0 0x7f12d0162fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:43512/0 (socket says 172.21.15.86:43512) 2024-04-20T21:17:59.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.048+0000 7f12cf200640 1 -- 172.21.15.86:0/3987880181 learned_addr learned my addr 172.21.15.86:0/3987880181 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:17:59.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.048+0000 7f092d400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09280d0ee0 0x7f09280d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.048+0000 7f092d400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09280d33b0 con 0x7f092815e690 2024-04-20T21:17:59.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.048+0000 7f092d400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09280d3690 con 0x7f09280d25a0 2024-04-20T21:17:59.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.048+0000 7f092d400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f09280d1880 con 0x7f09280d0ee0 2024-04-20T21:17:59.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.048+0000 7f092ca00640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f09280d25a0 0x7f09280d29c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.86:36670/0 (socket says 172.21.15.86:36670) 2024-04-20T21:17:59.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.048+0000 7f092ca00640 1 -- 172.21.15.86:0/875219492 learned_addr learned my addr 172.21.15.86:0/875219492 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:17:59.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.049+0000 7f0917400640 1 --2- 172.21.15.86:0/875219492 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09280d0ee0 0x7f09280d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.049+0000 7fa35d000640 1 Processor -- start 2024-04-20T21:17:59.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.049+0000 7f12cd400640 1 -- 172.21.15.86:0/3987880181 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 358433966 0 0) 0x7f12d0072f60 con 0x7f12d006bda0 2024-04-20T21:17:59.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.049+0000 7f12cd400640 1 -- 172.21.15.86:0/3987880181 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f12b4003520 con 0x7f12d006bda0 2024-04-20T21:17:59.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.049+0000 7f9d39800640 1 -- 172.21.15.86:0/3579968778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d34071d40 msgr2=0x7f9d340721d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.049+0000 7f9d39800640 1 --2- 172.21.15.86:0/3579968778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d34071d40 0x7f9d340721d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9d340731e0 tx=0x7f9d2002ca20 comp rx=0 tx=0).stop 2024-04-20T21:17:59.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.049+0000 7f12cde00640 1 --2- 172.21.15.86:0/3987880181 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f12d0167290 0x7f12d006e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.049+0000 7f12ce800640 1 --2- 172.21.15.86:0/3987880181 >> v2:172.21.15.144:3300/0 conn(0x7f12d0166eb0 0x7f12d006db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.049+0000 7f9d39800640 1 -- 172.21.15.86:0/3579968778 shutdown_connections 2024-04-20T21:17:59.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.049+0000 7f9d39800640 1 --2- 172.21.15.86:0/3579968778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d34071d40 0x7f9d340721d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.049+0000 7f9d39800640 1 --2- 172.21.15.86:0/3579968778 >> v2:172.21.15.144:3300/0 conn(0x7f9d340b3480 0x7f9d340ce640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.049+0000 7f9d39800640 1 --2- 172.21.15.86:0/3579968778 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d3415dd70 0x7f9d3415e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.049+0000 7f9d39800640 1 -- 172.21.15.86:0/3579968778 >> 172.21.15.86:0/3579968778 conn(0x7f9d3406b980 msgr2=0x7f9d3406bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:59.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.049+0000 7f12cd400640 1 -- 172.21.15.86:0/3987880181 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2506954076 0 0) 0x7f12b4003520 con 0x7f12d006bda0 2024-04-20T21:17:59.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.049+0000 7f12cd400640 1 -- 172.21.15.86:0/3987880181 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f12d0167290 msgr2=0x7f12d006e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:59.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.049+0000 7f12cd400640 1 --2- 172.21.15.86:0/3987880181 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f12d0167290 0x7f12d006e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.049+0000 7f001be00640 1 Processor -- start 2024-04-20T21:17:59.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.049+0000 7f1750e00640 1 -- 172.21.15.86:0/291716467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f174403e830 msgr2=0x7f1744053be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.049+0000 7f1750e00640 1 --2- 172.21.15.86:0/291716467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f174403e830 0x7f1744053be0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1744008670 tx=0x7f173802ba40 comp rx=0 tx=0).stop 2024-04-20T21:17:59.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.050+0000 7fa35d000640 1 -- start start 2024-04-20T21:17:59.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.050+0000 7f12cd400640 1 -- 172.21.15.86:0/3987880181 >> v2:172.21.15.144:3300/0 conn(0x7f12d0166eb0 msgr2=0x7f12d006db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:59.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.049+0000 7f0917400640 1 -- 172.21.15.86:0/875219492 >> v1:172.21.15.144:6789/0 conn(0x7f09280d25a0 legacy=0x7f09280d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.049+0000 7f0917400640 1 -- 172.21.15.86:0/875219492 >> v1:172.21.15.86:6789/0 conn(0x7f092815e690 legacy=0x7f092806ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:59.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.049+0000 7f0917400640 1 -- 172.21.15.86:0/875219492 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09280d3970 con 0x7f09280d0ee0 2024-04-20T21:17:59.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.050+0000 7f12cd400640 1 --2- 172.21.15.86:0/3987880181 >> v2:172.21.15.144:3300/0 conn(0x7f12d0166eb0 0x7f12d006db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.050+0000 7f12cd400640 1 -- 172.21.15.86:0/3987880181 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12d00b3510 con 0x7f12d006bda0 2024-04-20T21:17:59.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.050+0000 7f12cd400640 1 -- 172.21.15.86:0/3987880181 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f12c80019d0 con 0x7f12d006bda0 2024-04-20T21:17:59.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.050+0000 7f12cd400640 1 -- 172.21.15.86:0/3987880181 >> v1:172.21.15.86:6789/0 conn(0x7f12d006bda0 legacy=0x7f12d0162fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.050+0000 7f9d39800640 1 -- 172.21.15.86:0/3579968778 shutdown_connections 2024-04-20T21:17:59.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.050+0000 7f001be00640 1 -- start start 2024-04-20T21:17:59.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.049+0000 7f41bac00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41ac03c0a0 0x7f41ac04f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.049+0000 7f41bac00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f41ac053cb0 0x7f41ac0e3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.049+0000 7f41bac00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41ac0e3160 0x7f41ac0e4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.050+0000 7f41bac00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f41ac0e3540 con 0x7f41ac0e3160 2024-04-20T21:17:59.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.050+0000 7f41bac00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f41ac006640 con 0x7f41ac03c0a0 2024-04-20T21:17:59.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.050+0000 7f12cd400640 1 --2- 172.21.15.86:0/3987880181 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f12b40039d0 0x7f12b4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.050+0000 7f12cd400640 1 --2- 172.21.15.86:0/3987880181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12b4004800 0x7f12b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.050+0000 7f41bac00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f41ac006b60 con 0x7f41ac053cb0 2024-04-20T21:17:59.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.050+0000 7f9d39800640 1 -- 172.21.15.86:0/3579968778 wait complete. 2024-04-20T21:17:59.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.050+0000 7f41b8e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f41ac053cb0 0x7f41ac0e3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.050+0000 7f41b8e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f41ac053cb0 0x7f41ac0e3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:55416/0 (socket says 172.21.15.86:55416) 2024-04-20T21:17:59.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.050+0000 7f41b8e00640 1 -- 172.21.15.86:0/1434226228 learned_addr learned my addr 172.21.15.86:0/1434226228 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:59.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.050+0000 7f1750e00640 1 -- 172.21.15.86:0/291716467 shutdown_connections 2024-04-20T21:17:59.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.050+0000 7f12cd400640 1 --2- 172.21.15.86:0/3987880181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12b40051c0 0x7f12b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.050+0000 7f12cd400640 1 -- 172.21.15.86:0/3987880181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f12d00b3230 con 0x7f12b4004800 2024-04-20T21:17:59.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.050+0000 7f12cd400640 1 -- 172.21.15.86:0/3987880181 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f12b400bfe0 con 0x7f12b40051c0 2024-04-20T21:17:59.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.050+0000 7f12cd400640 1 -- 172.21.15.86:0/3987880181 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f12b400c1e0 con 0x7f12b40039d0 2024-04-20T21:17:59.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.050+0000 7f1750e00640 1 --2- 172.21.15.86:0/291716467 >> v2:172.21.15.86:3300/0 conn(0x7f1744054120 0x7f17440068c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.050+0000 7f1750e00640 1 --2- 172.21.15.86:0/291716467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f174403e830 0x7f1744053be0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.050+0000 7f1750e00640 1 -- 172.21.15.86:0/291716467 >> 172.21.15.86:0/291716467 conn(0x7f17440559d0 msgr2=0x7f1744055de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:59.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.050+0000 7f41b9800640 1 --2- 172.21.15.86:0/1434226228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41ac03c0a0 0x7f41ac04f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.050+0000 7f12ce800640 1 --2- 172.21.15.86:0/3987880181 >> v2:172.21.15.144:3300/0 conn(0x7f12d0166eb0 0x7f12d006db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:17:59.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.051+0000 7f41ba200640 1 --2- 172.21.15.86:0/1434226228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41ac0e3160 0x7f41ac0e4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.051+0000 7f12cf200640 1 --2- 172.21.15.86:0/3987880181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12b40051c0 0x7f12b400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.051+0000 7f12cde00640 1 --2- 172.21.15.86:0/3987880181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12b4004800 0x7f12b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.051+0000 7f0917400640 1 --2- 172.21.15.86:0/875219492 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09280d0ee0 0x7f09280d1340 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f09280d1a00 tx=0x7f091002b880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68b205495c2de4a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.051+0000 7f41b8e00640 1 -- 172.21.15.86:0/1434226228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41ac03c0a0 msgr2=0x7f41ac04f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.051+0000 7f41b8e00640 1 --2- 172.21.15.86:0/1434226228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41ac03c0a0 0x7f41ac04f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.052+0000 7f41b8e00640 1 -- 172.21.15.86:0/1434226228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41ac0e3160 msgr2=0x7f41ac0e4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.052+0000 7f41b8e00640 1 --2- 172.21.15.86:0/1434226228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41ac0e3160 0x7f41ac0e4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.052+0000 7f41b8e00640 1 -- 172.21.15.86:0/1434226228 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41ac17dee0 con 0x7f41ac053cb0 2024-04-20T21:17:59.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.052+0000 7f001be00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0000003b70 0x7f001c15ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.052+0000 7f001be00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f001c0cd1b0 0x7f001c0c8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.052+0000 7f001be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f001c0cd630 0x7f001c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.052+0000 7f12ce800640 1 --2- 172.21.15.86:0/3987880181 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f12b40039d0 0x7f12b4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.052+0000 7f12cde00640 1 -- 172.21.15.86:0/3987880181 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f12b40039d0 msgr2=0x7f12b4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:59.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.052+0000 7f12cde00640 1 --2- 172.21.15.86:0/3987880181 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f12b40039d0 0x7f12b4003e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.052+0000 7f12cde00640 1 -- 172.21.15.86:0/3987880181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12b40051c0 msgr2=0x7f12b400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:59.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.052+0000 7f12cde00640 1 --2- 172.21.15.86:0/3987880181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12b40051c0 0x7f12b400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.052+0000 7f12cde00640 1 -- 172.21.15.86:0/3987880181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12d00b3510 con 0x7f12b4004800 2024-04-20T21:17:59.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.052+0000 7f0927400640 1 -- 172.21.15.86:0/875219492 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091002e070 con 0x7f09280d0ee0 2024-04-20T21:17:59.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.052+0000 7f001be00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f001c0d3140 con 0x7f001c0cd630 2024-04-20T21:17:59.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.052+0000 7f001be00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f001c0d2e60 con 0x7f0000003b70 2024-04-20T21:17:59.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.052+0000 7f001be00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f001c1671c0 con 0x7f001c0cd1b0 2024-04-20T21:17:59.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.053+0000 7f9d39800640 1 Processor -- start 2024-04-20T21:17:59.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.053+0000 7f001b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f001c0cd630 0x7f001c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.053+0000 7f001a000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f001c0cd1b0 0x7f001c0c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.053+0000 7f001a000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f001c0cd1b0 0x7f001c0c8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:55446/0 (socket says 172.21.15.86:55446) 2024-04-20T21:17:59.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.053+0000 7f001a000640 1 -- 172.21.15.86:0/3169744386 learned_addr learned my addr 172.21.15.86:0/3169744386 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:59.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.051+0000 7f1750e00640 1 -- 172.21.15.86:0/291716467 shutdown_connections 2024-04-20T21:17:59.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.051+0000 7f1750e00640 1 -- 172.21.15.86:0/291716467 wait complete. 2024-04-20T21:17:59.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.052+0000 7f41b8e00640 1 --2- 172.21.15.86:0/1434226228 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f41ac053cb0 0x7f41ac0e3d40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f41a800c7f0 tx=0x7f41a800cbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.053+0000 7f9d39800640 1 -- start start 2024-04-20T21:17:59.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.053+0000 7f0927400640 1 -- 172.21.15.86:0/875219492 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0910037070 con 0x7f09280d0ee0 2024-04-20T21:17:59.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.053+0000 7f0927400640 1 -- 172.21.15.86:0/875219492 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0910031a00 con 0x7f09280d0ee0 2024-04-20T21:17:59.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.053+0000 7f41b1600640 1 -- 172.21.15.86:0/1434226228 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a8015070 con 0x7f41ac053cb0 2024-04-20T21:17:59.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.053+0000 7f001aa00640 1 --2- 172.21.15.86:0/3169744386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0000003b70 0x7f001c15ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.053+0000 7f41b1600640 1 -- 172.21.15.86:0/1434226228 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41a8007820 con 0x7f41ac053cb0 2024-04-20T21:17:59.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.053+0000 7f41b1600640 1 -- 172.21.15.86:0/1434226228 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a8015070 con 0x7f41ac053cb0 2024-04-20T21:17:59.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.053+0000 7f001b400640 1 -- 172.21.15.86:0/3169744386 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f001c0cd1b0 msgr2=0x7f001c0c8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.053+0000 7f001b400640 1 --2- 172.21.15.86:0/3169744386 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f001c0cd1b0 0x7f001c0c8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.053+0000 7f001b400640 1 -- 172.21.15.86:0/3169744386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0000003b70 msgr2=0x7f001c15ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.053+0000 7f001b400640 1 --2- 172.21.15.86:0/3169744386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0000003b70 0x7f001c15ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.053+0000 7f001b400640 1 -- 172.21.15.86:0/3169744386 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f001c0b7b70 con 0x7f001c0cd630 2024-04-20T21:17:59.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.054+0000 7f1750e00640 1 Processor -- start 2024-04-20T21:17:59.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.054+0000 7f1750e00640 1 -- start start 2024-04-20T21:17:59.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.054+0000 7f9d39800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d34071d40 0x7f9d3415bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.054+0000 7f9d39800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d340b3480 0x7f9d3414bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.054+0000 7f9d39800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d3415dd70 0x7f9d3414c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.054+0000 7f001b400640 1 --2- 172.21.15.86:0/3169744386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f001c0cd630 0x7f001c0b4d90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f001c0b7160 tx=0x7f000400f930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.054+0000 7f12cde00640 1 --2- 172.21.15.86:0/3987880181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12b4004800 0x7f12b4004c80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f12d00b4930 tx=0x7f12b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9efa83c025d805c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.054+0000 7f092d400640 1 -- 172.21.15.86:0/875219492 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09280d0ee0 msgr2=0x7f09280d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.054+0000 7f092d400640 1 --2- 172.21.15.86:0/875219492 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09280d0ee0 0x7f09280d1340 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f09280d1a00 tx=0x7f091002b880 comp rx=0 tx=0).stop 2024-04-20T21:17:59.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.055+0000 7f092d400640 1 -- 172.21.15.86:0/875219492 shutdown_connections 2024-04-20T21:17:59.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.055+0000 7f092d400640 1 --2- 172.21.15.86:0/875219492 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09280d0ee0 0x7f09280d1340 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.055+0000 7f092d400640 1 -- 172.21.15.86:0/875219492 >> 172.21.15.86:0/875219492 conn(0x7f092809d8a0 msgr2=0x7f092808bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:59.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.055+0000 7f092d400640 1 -- 172.21.15.86:0/875219492 shutdown_connections 2024-04-20T21:17:59.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.055+0000 7f9d39800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3415f220 con 0x7f9d3415dd70 2024-04-20T21:17:59.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.055+0000 7f9d39800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9d340afe50 con 0x7f9d340b3480 2024-04-20T21:17:59.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.055+0000 7f9d39800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9d3406c470 con 0x7f9d34071d40 2024-04-20T21:17:59.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.055+0000 7f9d33400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d3415dd70 0x7f9d3414c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.055+0000 7f9d32a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d34071d40 0x7f9d3415bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.055+0000 7f9d32a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d34071d40 0x7f9d3415bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:55462/0 (socket says 172.21.15.86:55462) 2024-04-20T21:17:59.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.055+0000 7f9d33400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d3415dd70 0x7f9d3414c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57760/0 (socket says 172.21.15.86:57760) 2024-04-20T21:17:59.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.055+0000 7f9d32a00640 1 -- 172.21.15.86:0/3467630821 learned_addr learned my addr 172.21.15.86:0/3467630821 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:59.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.055+0000 7f092d400640 1 -- 172.21.15.86:0/875219492 wait complete. 2024-04-20T21:17:59.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.055+0000 7f0012a00640 1 -- 172.21.15.86:0/3169744386 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0004019070 con 0x7f001c0cd630 2024-04-20T21:17:59.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.055+0000 7f12cd400640 1 -- 172.21.15.86:0/3987880181 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c003a070 con 0x7f12b4004800 2024-04-20T21:17:59.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.055+0000 7f12cd400640 1 -- 172.21.15.86:0/3987880181 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12c0035660 con 0x7f12b4004800 2024-04-20T21:17:59.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.055+0000 7f12cd400640 1 -- 172.21.15.86:0/3987880181 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c000b530 con 0x7f12b4004800 2024-04-20T21:17:59.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.055+0000 7f0012a00640 1 -- 172.21.15.86:0/3169744386 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00040050d0 con 0x7f001c0cd630 2024-04-20T21:17:59.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.055+0000 7fa35d000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3580b13c0 0x7fa358200da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.055+0000 7fa35d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3580d44e0 0x7fa35806dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.055+0000 7fa35d000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa35815eaf0 0x7fa35806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.055+0000 7fa35d000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa3580b1820 con 0x7fa3580d44e0 2024-04-20T21:17:59.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.055+0000 7fa35d000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa3580b4ef0 con 0x7fa35815eaf0 2024-04-20T21:17:59.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.055+0000 7fa35d000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa3580d0340 con 0x7fa3580b13c0 2024-04-20T21:17:59.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.056+0000 7f41bac00640 1 -- 172.21.15.86:0/1434226228 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41ac17e1c0 con 0x7f41ac053cb0 2024-04-20T21:17:59.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.056+0000 7f9d32000640 1 --2- 172.21.15.86:0/3467630821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d340b3480 0x7f9d3414bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.056+0000 7f3a71000640 1 Processor -- start 2024-04-20T21:17:59.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.056+0000 7f3a71000640 1 -- start start 2024-04-20T21:17:59.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.056+0000 7f0012a00640 1 -- 172.21.15.86:0/3169744386 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00040069e0 con 0x7f001c0cd630 2024-04-20T21:17:59.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.056+0000 7f12d4a00640 1 -- 172.21.15.86:0/3987880181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12b4004800 msgr2=0x7f12b4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.056+0000 7f12d4a00640 1 --2- 172.21.15.86:0/3987880181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12b4004800 0x7f12b4004c80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f12d00b4930 tx=0x7f12b4005830 comp rx=0 tx=0).stop 2024-04-20T21:17:59.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.056+0000 7f12cf200640 1 -- 172.21.15.86:0/3987880181 reap_dead start 2024-04-20T21:17:59.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.056+0000 7f12d4a00640 1 -- 172.21.15.86:0/3987880181 shutdown_connections 2024-04-20T21:17:59.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.056+0000 7f12d4a00640 1 -- 172.21.15.86:0/3987880181 >> 172.21.15.86:0/3987880181 conn(0x7f12d00b2220 msgr2=0x7f12d00b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:59.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.056+0000 7fa356800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa35815eaf0 0x7fa35806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.056+0000 7f41bac00640 1 -- 172.21.15.86:0/1434226228 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f41ac051410 con 0x7f41ac053cb0 2024-04-20T21:17:59.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.056+0000 7fa356800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa35815eaf0 0x7fa35806e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.86:57944/0 (socket says 172.21.15.86:57944) 2024-04-20T21:17:59.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.056+0000 7fa356800640 1 -- 172.21.15.86:0/1827703005 learned_addr learned my addr 172.21.15.86:0/1827703005 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:59.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.057+0000 7fa355e00640 1 --2- 172.21.15.86:0/1827703005 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3580b13c0 0x7fa358200da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.057+0000 7f9d32a00640 1 -- 172.21.15.86:0/3467630821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d340b3480 msgr2=0x7f9d3414bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.057+0000 7f9d32a00640 1 --2- 172.21.15.86:0/3467630821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d340b3480 0x7f9d3414bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.057+0000 7f9d32a00640 1 -- 172.21.15.86:0/3467630821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d3415dd70 msgr2=0x7f9d3414c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.057+0000 7f9d32a00640 1 --2- 172.21.15.86:0/3467630821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d3415dd70 0x7f9d3414c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.057+0000 7f9d32a00640 1 -- 172.21.15.86:0/3467630821 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d340ae4d0 con 0x7f9d34071d40 2024-04-20T21:17:59.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.057+0000 7f001be00640 1 -- 172.21.15.86:0/3169744386 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f001c0cafb0 con 0x7f001c0cd630 2024-04-20T21:17:59.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.057+0000 7fa355400640 1 --2- 172.21.15.86:0/1827703005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3580d44e0 0x7fa35806dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.056+0000 7f12d4a00640 1 -- 172.21.15.86:0/3987880181 shutdown_connections 2024-04-20T21:17:59.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.056+0000 7f12d4a00640 1 -- 172.21.15.86:0/3987880181 wait complete. 2024-04-20T21:17:59.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.057+0000 7f3a71000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3a6c15ef60 0x7f3a6c0b7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.057+0000 7f1750e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f174403e830 0x7f17440d8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.057+0000 7f1750e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1744054120 0x7f17440dc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.057+0000 7f1750e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17440e30b0 0x7f17440dcb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.057+0000 7f1750e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f17440e3490 con 0x7f17440e30b0 2024-04-20T21:17:59.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.057+0000 7f1750e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f174403bfb0 con 0x7f174403e830 2024-04-20T21:17:59.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.057+0000 7f1750e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1744006e00 con 0x7f1744054120 2024-04-20T21:17:59.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.057+0000 7f3a71000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3a6c0d4260 0x7f3a6c0d0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.057+0000 7f3a71000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a6c0b6390 con 0x7f3a6c15eb80 2024-04-20T21:17:59.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.057+0000 7f3a71000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f3a6c0b6670 con 0x7f3a6c0d4260 2024-04-20T21:17:59.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.057+0000 7f3a71000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3a6c0b6950 con 0x7f3a6c15ef60 2024-04-20T21:17:59.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.058+0000 7fa355e00640 1 -- 172.21.15.86:0/1827703005 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa35815eaf0 msgr2=0x7fa35806e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.058+0000 7fa355e00640 1 --2- 172.21.15.86:0/1827703005 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa35815eaf0 0x7fa35806e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.058+0000 7fa355e00640 1 -- 172.21.15.86:0/1827703005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3580d44e0 msgr2=0x7fa35806dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.058+0000 7fa355e00640 1 --2- 172.21.15.86:0/1827703005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3580d44e0 0x7fa35806dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.058+0000 7fa355e00640 1 -- 172.21.15.86:0/1827703005 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3580b7770 con 0x7fa3580b13c0 2024-04-20T21:17:59.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.058+0000 7f1749e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f174403e830 0x7f17440d8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.057+0000 7f9d32a00640 1 --2- 172.21.15.86:0/3467630821 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d34071d40 0x7f9d3415bfe0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c00ccd0 tx=0x7f9d2c00a5c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.058+0000 7f1749400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1744054120 0x7f17440dc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.058+0000 7f1749e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f174403e830 0x7f17440d8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.86:57958/0 (socket says 172.21.15.86:57958) 2024-04-20T21:17:59.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.058+0000 7f1749e00640 1 -- 172.21.15.86:0/556424118 learned_addr learned my addr 172.21.15.86:0/556424118 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:59.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.058+0000 7f174a800640 1 --2- 172.21.15.86:0/556424118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17440e30b0 0x7f17440dcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.058+0000 7f3a69400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3a6c15ef60 0x7f3a6c0b7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.058+0000 7f001be00640 1 -- 172.21.15.86:0/3169744386 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f001c0b7df0 con 0x7f001c0cd630 2024-04-20T21:17:59.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.058+0000 7f092d400640 1 Processor -- start 2024-04-20T21:17:59.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.058+0000 7fa355e00640 1 --2- 172.21.15.86:0/1827703005 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3580b13c0 0x7fa358200da0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa348033000 tx=0x7fa3480046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.059+0000 7f3a6a800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3a6c0d4260 0x7f3a6c0d0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.059+0000 7f3a69e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f3a6c15eb80 0x7f3a6c0cedc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:43522/0 (socket says 172.21.15.86:43522) 2024-04-20T21:17:59.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.059+0000 7f3a69e00640 1 -- 172.21.15.86:0/1858764385 learned_addr learned my addr 172.21.15.86:0/1858764385 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:17:59.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.059+0000 7f41bac00640 1 -- 172.21.15.86:0/1434226228 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f41ac005390 con 0x7f41ac053cb0 2024-04-20T21:17:59.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.059+0000 7f092d400640 1 -- start start 2024-04-20T21:17:59.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.059+0000 7f9d2aa00640 1 -- 172.21.15.86:0/3467630821 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c003990 con 0x7f9d34071d40 2024-04-20T21:17:59.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.059+0000 7f12d4a00640 1 Processor -- start 2024-04-20T21:17:59.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.059+0000 7f12d4a00640 1 -- start start 2024-04-20T21:17:59.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.059+0000 7f9d39800640 1 -- 172.21.15.86:0/3467630821 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d340ae7b0 con 0x7f9d34071d40 2024-04-20T21:17:59.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.059+0000 7f3a6a800640 1 -- 172.21.15.86:0/1858764385 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3a6c15ef60 msgr2=0x7f3a6c0b7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:59.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.060+0000 7fa34e000640 1 -- 172.21.15.86:0/1827703005 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa348034040 con 0x7fa3580b13c0 2024-04-20T21:17:59.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.060+0000 7fa34e000640 1 -- 172.21.15.86:0/1827703005 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa348030e40 con 0x7fa3580b13c0 2024-04-20T21:17:59.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.060+0000 7fa34e000640 1 -- 172.21.15.86:0/1827703005 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3480433f0 con 0x7fa3580b13c0 2024-04-20T21:17:59.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.060+0000 7f9d2aa00640 1 -- 172.21.15.86:0/3467630821 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d2c003b30 con 0x7f9d34071d40 2024-04-20T21:17:59.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.060+0000 7f9d2aa00640 1 -- 172.21.15.86:0/3467630821 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c003990 con 0x7f9d34071d40 2024-04-20T21:17:59.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.059+0000 7f3a6a800640 1 --2- 172.21.15.86:0/1858764385 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3a6c15ef60 0x7f3a6c0b7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.059+0000 7f3a6a800640 1 -- 172.21.15.86:0/1858764385 >> v1:172.21.15.86:6789/0 conn(0x7f3a6c15eb80 legacy=0x7f3a6c0cedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.059+0000 7f3a6a800640 1 -- 172.21.15.86:0/1858764385 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a6c0b6c30 con 0x7f3a6c0d4260 2024-04-20T21:17:59.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.060+0000 7fa35d000640 1 -- 172.21.15.86:0/1827703005 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3580ba630 con 0x7fa3580b13c0 2024-04-20T21:17:59.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.060+0000 7fa35d000640 1 -- 172.21.15.86:0/1827703005 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa35815ef70 con 0x7fa3580b13c0 2024-04-20T21:17:59.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.060+0000 7f001be00640 1 -- 172.21.15.86:0/3169744386 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f001c04fcf0 con 0x7f001c0cd630 2024-04-20T21:17:59.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.061+0000 7f092d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09280d0ee0 0x7f0928155180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.060+0000 7f12d4a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f12b4004800 0x7f12d0163ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.060+0000 7f12d4a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12d01628a0 0x7f12d009b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.061+0000 7f12d4a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12d0150580 0x7f12d0164cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.061+0000 7f12d4a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f12d00b3ff0 con 0x7f12d01628a0 2024-04-20T21:17:59.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.061+0000 7f12d4a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f12d006e6c0 con 0x7f12d0150580 2024-04-20T21:17:59.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.061+0000 7f12d4a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f12d006e060 con 0x7f12b4004800 2024-04-20T21:17:59.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.061+0000 7f12ce800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f12b4004800 0x7f12d0163ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.061+0000 7f092d400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09280d25a0 0x7f09281f8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.061+0000 7f3a6a800640 1 --2- 172.21.15.86:0/1858764385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a6c0d4260 0x7f3a6c0d0ec0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3a64009d90 tx=0x7f3a6402beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f62be58b1f2a49d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.061+0000 7f3a68a00640 1 -- 172.21.15.86:0/1858764385 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a64039070 con 0x7f3a6c0d4260 2024-04-20T21:17:59.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.061+0000 7f3a68a00640 1 -- 172.21.15.86:0/1858764385 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a6402cc60 con 0x7f3a6c0d4260 2024-04-20T21:17:59.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.061+0000 7f3a68a00640 1 -- 172.21.15.86:0/1858764385 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a64034660 con 0x7f3a6c0d4260 2024-04-20T21:17:59.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.062+0000 7f092d400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f092815e690 0x7f0928157300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.060+0000 7f1749400640 1 -- 172.21.15.86:0/556424118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f174403e830 msgr2=0x7f17440d8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.060+0000 7f1749400640 1 --2- 172.21.15.86:0/556424118 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f174403e830 0x7f17440d8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.060+0000 7f1749400640 1 -- 172.21.15.86:0/556424118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17440e30b0 msgr2=0x7f17440dcb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.060+0000 7f1749400640 1 --2- 172.21.15.86:0/556424118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17440e30b0 0x7f17440dcb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.060+0000 7f1749400640 1 -- 172.21.15.86:0/556424118 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1744052340 con 0x7f1744054120 2024-04-20T21:17:59.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.061+0000 7fa343400640 1 -- 172.21.15.86:0/1827703005 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa3580690f0 con 0x7fa3580b13c0 2024-04-20T21:17:59.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.062+0000 7f12ce800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f12b4004800 0x7f12d0163ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:55482/0 (socket says 172.21.15.86:55482) 2024-04-20T21:17:59.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.062+0000 7f12ce800640 1 -- 172.21.15.86:0/2184573140 learned_addr learned my addr 172.21.15.86:0/2184573140 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:59.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.062+0000 7f12cf200640 1 --2- 172.21.15.86:0/2184573140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12d0150580 0x7f12d0164cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.062+0000 7f092d400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f09280d30d0 con 0x7f09280d0ee0 2024-04-20T21:17:59.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.062+0000 7f092d400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f09280cef40 con 0x7f092815e690 2024-04-20T21:17:59.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.062+0000 7f092d400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f092815ea70 con 0x7f09280d25a0 2024-04-20T21:17:59.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.062+0000 7f0927e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09280d0ee0 0x7f0928155180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.062+0000 7f0917400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09280d25a0 0x7f09281f8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.062+0000 7f3a71000640 1 -- 172.21.15.86:0/1858764385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a6c0d4260 msgr2=0x7f3a6c0d0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.062+0000 7f3a71000640 1 --2- 172.21.15.86:0/1858764385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a6c0d4260 0x7f3a6c0d0ec0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3a64009d90 tx=0x7f3a6402beb0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.062+0000 7f0927e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09280d0ee0 0x7f0928155180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57798/0 (socket says 172.21.15.86:57798) 2024-04-20T21:17:59.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.062+0000 7f0927e00640 1 -- 172.21.15.86:0/1477425991 learned_addr learned my addr 172.21.15.86:0/1477425991 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:59.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.062+0000 7f1749400640 1 --2- 172.21.15.86:0/556424118 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1744054120 0x7f17440dc5c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1738033000 tx=0x7f17380046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.062+0000 7f12cde00640 1 --2- 172.21.15.86:0/2184573140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12d01628a0 0x7f12d009b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.062+0000 7f3a71000640 1 -- 172.21.15.86:0/1858764385 shutdown_connections 2024-04-20T21:17:59.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.062+0000 7f3a71000640 1 --2- 172.21.15.86:0/1858764385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a6c0d4260 0x7f3a6c0d0ec0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.062+0000 7f3a71000640 1 --2- 172.21.15.86:0/1858764385 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3a6c15ef60 0x7f3a6c0b7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.062+0000 7f092ca00640 1 --2- 172.21.15.86:0/1477425991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f092815e690 0x7f0928157300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.062+0000 7f3a71000640 1 -- 172.21.15.86:0/1858764385 >> 172.21.15.86:0/1858764385 conn(0x7f3a6c06cc20 msgr2=0x7f3a6c06d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:59.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.063+0000 7f3a71000640 1 -- 172.21.15.86:0/1858764385 shutdown_connections 2024-04-20T21:17:59.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.063+0000 7f3a71000640 1 -- 172.21.15.86:0/1858764385 wait complete. 2024-04-20T21:17:59.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.062+0000 7f9d39800640 1 -- 172.21.15.86:0/3467630821 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d340aed70 con 0x7f9d34071d40 2024-04-20T21:17:59.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.063+0000 7f0012a00640 1 -- 172.21.15.86:0/3169744386 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0004006b80 con 0x7f001c0cd630 2024-04-20T21:17:59.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.064+0000 7f1742000640 1 -- 172.21.15.86:0/556424118 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1738042480 con 0x7f1744054120 2024-04-20T21:17:59.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.063+0000 7f12ce800640 1 -- 172.21.15.86:0/2184573140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12d0150580 msgr2=0x7f12d0164cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.063+0000 7f12ce800640 1 --2- 172.21.15.86:0/2184573140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12d0150580 0x7f12d0164cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.063+0000 7f12ce800640 1 -- 172.21.15.86:0/2184573140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12d01628a0 msgr2=0x7f12d009b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.063+0000 7f12ce800640 1 --2- 172.21.15.86:0/2184573140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12d01628a0 0x7f12d009b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.063+0000 7f12ce800640 1 -- 172.21.15.86:0/2184573140 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12d009ba50 con 0x7f12b4004800 2024-04-20T21:17:59.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.063+0000 7f092ca00640 1 -- 172.21.15.86:0/1477425991 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09280d25a0 msgr2=0x7f09281f8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.063+0000 7f092ca00640 1 --2- 172.21.15.86:0/1477425991 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09280d25a0 0x7f09281f8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.063+0000 7f092ca00640 1 -- 172.21.15.86:0/1477425991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09280d0ee0 msgr2=0x7f0928155180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.063+0000 7f092ca00640 1 --2- 172.21.15.86:0/1477425991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09280d0ee0 0x7f0928155180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.063+0000 7f092ca00640 1 -- 172.21.15.86:0/1477425991 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09281556c0 con 0x7f092815e690 2024-04-20T21:17:59.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.062+0000 7f41b1600640 1 -- 172.21.15.86:0/1434226228 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f41a8018070 con 0x7f41ac053cb0 2024-04-20T21:17:59.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.064+0000 7f41b1600640 1 --2- 172.21.15.86:0/1434226228 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f419806cb20 0x7f419806efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.064+0000 7f41b1600640 1 -- 172.21.15.86:0/1434226228 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f41a80579c0 con 0x7f41ac053cb0 2024-04-20T21:17:59.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.064+0000 7f9d39800640 1 -- 172.21.15.86:0/3467630821 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9d340689e0 con 0x7f9d34071d40 2024-04-20T21:17:59.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.064+0000 7f1742000640 1 -- 172.21.15.86:0/556424118 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1738047070 con 0x7f1744054120 2024-04-20T21:17:59.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.064+0000 7f1742000640 1 -- 172.21.15.86:0/556424118 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17380428a0 con 0x7f1744054120 2024-04-20T21:17:59.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.064+0000 7f092ca00640 1 --2- 172.21.15.86:0/1477425991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f092815e690 0x7f0928157300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f092000ca70 tx=0x7f092000cf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.064+0000 7f1750e00640 1 -- 172.21.15.86:0/556424118 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1744052620 con 0x7f1744054120 2024-04-20T21:17:59.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.064+0000 7f41b9800640 1 --2- 172.21.15.86:0/1434226228 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f419806cb20 0x7f419806efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.065+0000 7f41b1600640 1 --2- 172.21.15.86:0/1434226228 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x7f4198075990 0x7f4198077e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.065+0000 7f41b1600640 1 -- 172.21.15.86:0/1434226228 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4198078500 con 0x7f4198075990 2024-04-20T21:17:59.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.065+0000 7f41b1600640 1 -- 172.21.15.86:0/1434226228 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f41a8056350 con 0x7f41ac053cb0 2024-04-20T21:17:59.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.065+0000 7f12ce800640 1 --2- 172.21.15.86:0/2184573140 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f12b4004800 0x7f12d0163ee0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f12bc00f5a0 tx=0x7f12bc00f9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.065+0000 7f3a71000640 1 Processor -- start 2024-04-20T21:17:59.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.065+0000 7f3a71000640 1 -- start start 2024-04-20T21:17:59.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.065+0000 7f1750e00640 1 -- 172.21.15.86:0/556424118 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f17440dad10 con 0x7f1744054120 2024-04-20T21:17:59.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.065+0000 7f41ba200640 1 --2- 172.21.15.86:0/1434226228 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x7f4198075990 0x7f4198077e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.064+0000 7f0012a00640 1 --2- 172.21.15.86:0/3169744386 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efff806c8f0 0x7efff806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.065+0000 7f0012a00640 1 -- 172.21.15.86:0/3169744386 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f000408ac10 con 0x7f001c0cd630 2024-04-20T21:17:59.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.065+0000 7f0012a00640 1 --2- 172.21.15.86:0/3169744386 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x7efff8074b10 0x7efff8076f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.065+0000 7f0012a00640 1 -- 172.21.15.86:0/3169744386 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0004017e70 con 0x7efff8074b10 2024-04-20T21:17:59.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.066+0000 7f12c6a00640 1 -- 172.21.15.86:0/2184573140 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12bc018070 con 0x7f12b4004800 2024-04-20T21:17:59.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.066+0000 7f12c6a00640 1 -- 172.21.15.86:0/2184573140 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12bc014890 con 0x7f12b4004800 2024-04-20T21:17:59.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.066+0000 7f12c6a00640 1 -- 172.21.15.86:0/2184573140 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12bc014c60 con 0x7f12b4004800 2024-04-20T21:17:59.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.066+0000 7f12d4a00640 1 -- 172.21.15.86:0/2184573140 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12d0162e40 con 0x7f12b4004800 2024-04-20T21:17:59.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.066+0000 7f12d4a00640 1 -- 172.21.15.86:0/2184573140 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f12d0163400 con 0x7f12b4004800 2024-04-20T21:17:59.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.067+0000 7fa34e000640 1 -- 172.21.15.86:0/1827703005 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa34804a060 con 0x7fa3580b13c0 2024-04-20T21:17:59.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.066+0000 7f3a71000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a6c0d4260 0x7f3a6c1e7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.066+0000 7f3a71000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3a6c15eb80 0x7f3a6c14ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.067+0000 7f3a71000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a6c15ef60 0x7f3a6c14b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.067+0000 7f3a71000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6c0d1790 con 0x7f3a6c0d4260 2024-04-20T21:17:59.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.067+0000 7f3a71000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6c0d1610 con 0x7f3a6c15ef60 2024-04-20T21:17:59.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.067+0000 7f3a71000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3a6c0d1910 con 0x7f3a6c15eb80 2024-04-20T21:17:59.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.067+0000 7f3a6a800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a6c15ef60 0x7f3a6c14b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.067+0000 7f001aa00640 1 --2- 172.21.15.86:0/3169744386 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efff806c8f0 0x7efff806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.067+0000 7f3a6a800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a6c15ef60 0x7f3a6c14b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.86:58008/0 (socket says 172.21.15.86:58008) 2024-04-20T21:17:59.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.067+0000 7f3a6a800640 1 -- 172.21.15.86:0/3604546919 learned_addr learned my addr 172.21.15.86:0/3604546919 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:59.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.067+0000 7f12d4a00640 1 -- 172.21.15.86:0/2184573140 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f12d004fcf0 con 0x7f12b4004800 2024-04-20T21:17:59.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.065+0000 7f0924c00640 1 -- 172.21.15.86:0/1477425991 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0920017070 con 0x7f092815e690 2024-04-20T21:17:59.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.065+0000 7f0924c00640 1 -- 172.21.15.86:0/1477425991 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f092000ab70 con 0x7f092815e690 2024-04-20T21:17:59.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.067+0000 7f41ba200640 1 --2- 172.21.15.86:0/1434226228 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x7f4198075990 0x7f4198077e30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.065+0000 7f0924c00640 1 -- 172.21.15.86:0/1477425991 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0920012400 con 0x7f092815e690 2024-04-20T21:17:59.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.067+0000 7f0012a00640 1 -- 172.21.15.86:0/3169744386 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f000408d270 con 0x7f001c0cd630 2024-04-20T21:17:59.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.066+0000 7f9d2aa00640 1 -- 172.21.15.86:0/3467630821 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9d2c003ce0 con 0x7f9d34071d40 2024-04-20T21:17:59.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.068+0000 7f3a69e00640 1 --2- 172.21.15.86:0/3604546919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a6c0d4260 0x7f3a6c1e7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.068+0000 7f41b9800640 1 --2- 172.21.15.86:0/1434226228 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f419806cb20 0x7f419806efe0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f41a4036000 tx=0x7f41a403b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.068+0000 7f3a6a800640 1 -- 172.21.15.86:0/3604546919 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3a6c15eb80 msgr2=0x7f3a6c14ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.068+0000 7f3a6a800640 1 --2- 172.21.15.86:0/3604546919 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3a6c15eb80 0x7f3a6c14ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.068+0000 7f001aa00640 1 --2- 172.21.15.86:0/3169744386 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efff806c8f0 0x7efff806edb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0014035000 tx=0x7f001402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.069+0000 7f9d2aa00640 1 --2- 172.21.15.86:0/3467630821 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9d1406cb20 0x7f9d1406efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.067+0000 7f1750e00640 1 -- 172.21.15.86:0/556424118 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f17440057d0 con 0x7f1744054120 2024-04-20T21:17:59.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.068+0000 7f1742000640 1 -- 172.21.15.86:0/556424118 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1738040050 con 0x7f1744054120 2024-04-20T21:17:59.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.069+0000 7f001a000640 1 --2- 172.21.15.86:0/3169744386 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x7efff8074b10 0x7efff8076f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.068+0000 7f3a6a800640 1 -- 172.21.15.86:0/3604546919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a6c0d4260 msgr2=0x7f3a6c1e7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.069+0000 7fa34e000640 1 --2- 172.21.15.86:0/1827703005 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa33806c9c0 0x7fa33806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.069+0000 7f9d2aa00640 1 -- 172.21.15.86:0/3467630821 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9d2c08b330 con 0x7f9d34071d40 2024-04-20T21:17:59.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.069+0000 7fa34e000640 1 -- 172.21.15.86:0/1827703005 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa3480ae030 con 0x7fa3580b13c0 2024-04-20T21:17:59.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.069+0000 7f9d2aa00640 1 --2- 172.21.15.86:0/3467630821 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x7f9d14074d40 0x7f9d14077160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.069+0000 7fa34e000640 1 --2- 172.21.15.86:0/1827703005 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x7fa338074be0 0x7fa338077000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.069+0000 7fa34e000640 1 -- 172.21.15.86:0/1827703005 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa34802ce90 con 0x7fa338074be0 2024-04-20T21:17:59.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.069+0000 7fa34e000640 1 -- 172.21.15.86:0/1827703005 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa3480ae420 con 0x7fa3580b13c0 2024-04-20T21:17:59.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.069+0000 7f9d2aa00640 1 -- 172.21.15.86:0/3467630821 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9d14077810 con 0x7f9d14074d40 2024-04-20T21:17:59.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.069+0000 7f9d2aa00640 1 -- 172.21.15.86:0/3467630821 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f9d2c092050 con 0x7f9d34071d40 2024-04-20T21:17:59.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.069+0000 7f001a000640 1 --2- 172.21.15.86:0/3169744386 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x7efff8074b10 0x7efff8076f30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.070+0000 7f9d32000640 1 --2- 172.21.15.86:0/3467630821 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9d1406cb20 0x7f9d1406efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.068+0000 7f092d400640 1 -- 172.21.15.86:0/1477425991 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0928155940 con 0x7f092815e690 2024-04-20T21:17:59.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.069+0000 7f092d400640 1 -- 172.21.15.86:0/1477425991 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0928151520 con 0x7f092815e690 2024-04-20T21:17:59.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.070+0000 7fa356800640 1 --2- 172.21.15.86:0/1827703005 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x7fa338074be0 0x7fa338077000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.070+0000 7f092d400640 1 -- 172.21.15.86:0/1477425991 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f092804fcf0 con 0x7f092815e690 2024-04-20T21:17:59.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.070+0000 7f3a6a800640 1 --2- 172.21.15.86:0/3604546919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a6c0d4260 0x7f3a6c1e7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.070+0000 7f3a6a800640 1 -- 172.21.15.86:0/3604546919 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a6c1e7d40 con 0x7f3a6c15ef60 2024-04-20T21:17:59.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.070+0000 7f3a69e00640 1 --2- 172.21.15.86:0/3604546919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3a6c0d4260 0x7f3a6c1e7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:59.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.071+0000 7fa356800640 1 --2- 172.21.15.86:0/1827703005 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x7fa338074be0 0x7fa338077000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.071+0000 7fa355400640 1 --2- 172.21.15.86:0/1827703005 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa33806c9c0 0x7fa33806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.071+0000 7f9d32000640 1 --2- 172.21.15.86:0/3467630821 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9d1406cb20 0x7f9d1406efe0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9d341f8140 tx=0x7f9d24008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.071+0000 7f0012a00640 1 -- 172.21.15.86:0/3169744386 <== osd.1 v2:172.21.15.86:6817/3149564961 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0004017e70 con 0x7efff8074b10 2024-04-20T21:17:59.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.070+0000 7f1742000640 1 --2- 172.21.15.86:0/556424118 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f172c06c8f0 0x7f172c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.071+0000 7f1742000640 1 -- 172.21.15.86:0/556424118 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f17380ae750 con 0x7f1744054120 2024-04-20T21:17:59.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.071+0000 7f1742000640 1 --2- 172.21.15.86:0/556424118 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x7f172c074a90 0x7f172c076eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.071+0000 7f1742000640 1 -- 172.21.15.86:0/556424118 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1738001d30 con 0x7f172c074a90 2024-04-20T21:17:59.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.071+0000 7f1742000640 1 -- 172.21.15.86:0/556424118 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f17380b5050 con 0x7f1744054120 2024-04-20T21:17:59.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.071+0000 7f41b1600640 1 -- 172.21.15.86:0/1434226228 <== osd.3 v2:172.21.15.86:6808/794135519 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4198078500 con 0x7f4198075990 2024-04-20T21:17:59.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.071+0000 7f1749e00640 1 --2- 172.21.15.86:0/556424118 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f172c06c8f0 0x7f172c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.072+0000 7f174a800640 1 --2- 172.21.15.86:0/556424118 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x7f172c074a90 0x7f172c076eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.072+0000 7f9d33400640 1 --2- 172.21.15.86:0/3467630821 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x7f9d14074d40 0x7f9d14077160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.072+0000 7f3a6a800640 1 --2- 172.21.15.86:0/3604546919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a6c15ef60 0x7f3a6c14b5c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3a64009d90 tx=0x7f3a64004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.072+0000 7f12c6a00640 1 -- 172.21.15.86:0/2184573140 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f12bc01b070 con 0x7f12b4004800 2024-04-20T21:17:59.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.072+0000 7fa355400640 1 --2- 172.21.15.86:0/1827703005 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa33806c9c0 0x7fa33806ee80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa344009560 tx=0x7fa344008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.072+0000 7f3a62000640 1 -- 172.21.15.86:0/3604546919 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a64039050 con 0x7f3a6c15ef60 2024-04-20T21:17:59.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.072+0000 7f3a62000640 1 -- 172.21.15.86:0/3604546919 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a64045070 con 0x7f3a6c15ef60 2024-04-20T21:17:59.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.072+0000 7f3a62000640 1 -- 172.21.15.86:0/3604546919 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6403e830 con 0x7f3a6c15ef60 2024-04-20T21:17:59.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.072+0000 7f3a71000640 1 -- 172.21.15.86:0/3604546919 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a6c14b0b0 con 0x7f3a6c15ef60 2024-04-20T21:17:59.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.073+0000 7f3a71000640 1 -- 172.21.15.86:0/3604546919 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a6c0cd720 con 0x7f3a6c15ef60 2024-04-20T21:17:59.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.073+0000 7f9d33400640 1 --2- 172.21.15.86:0/3467630821 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x7f9d14074d40 0x7f9d14077160 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.073+0000 7f12c6a00640 1 --2- 172.21.15.86:0/2184573140 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f12b006c9f0 0x7f12b006eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.074+0000 7f12c6a00640 1 -- 172.21.15.86:0/2184573140 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f12bc08bf30 con 0x7f12b4004800 2024-04-20T21:17:59.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.074+0000 7f12c6a00640 1 --2- 172.21.15.86:0/2184573140 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x7f12b0074c10 0x7f12b0077030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.074+0000 7f12c6a00640 1 -- 172.21.15.86:0/2184573140 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f12bc001d30 con 0x7f12b0074c10 2024-04-20T21:17:59.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.072+0000 7f1749e00640 1 --2- 172.21.15.86:0/556424118 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f172c06c8f0 0x7f172c06edb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f173c007fd0 tx=0x7f173c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.074+0000 7f12c6a00640 1 -- 172.21.15.86:0/2184573140 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f12bc08c320 con 0x7f12b4004800 2024-04-20T21:17:59.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.074+0000 7f3a71000640 1 -- 172.21.15.86:0/3604546919 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3a44001030 con 0x7f3a6c15ef60 2024-04-20T21:17:59.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.074+0000 7fa34e000640 1 -- 172.21.15.86:0/1827703005 <== osd.2 v2:172.21.15.86:6816/3437611468 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa34802ce90 con 0x7fa338074be0 2024-04-20T21:17:59.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.074+0000 7f12cde00640 1 --2- 172.21.15.86:0/2184573140 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f12b006c9f0 0x7f12b006eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.074+0000 7f174a800640 1 --2- 172.21.15.86:0/556424118 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x7f172c074a90 0x7f172c076eb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.075+0000 7f12cf200640 1 --2- 172.21.15.86:0/2184573140 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x7f12b0074c10 0x7f12b0077030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.075+0000 7f12cde00640 1 --2- 172.21.15.86:0/2184573140 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f12b006c9f0 0x7f12b006eeb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f12c0006a90 tx=0x7f12c0030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.076+0000 7f9d2aa00640 1 -- 172.21.15.86:0/3467630821 <== osd.4 v2:172.21.15.144:6800/4087309475 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9d14077810 con 0x7f9d14074d40 2024-04-20T21:17:59.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.076+0000 7f1742000640 1 -- 172.21.15.86:0/556424118 <== osd.0 v2:172.21.15.86:6800/2471076035 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1738001d30 con 0x7f172c074a90 2024-04-20T21:17:59.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.077+0000 7f12cf200640 1 --2- 172.21.15.86:0/2184573140 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x7f12b0074c10 0x7f12b0077030 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.075+0000 7f0924c00640 1 -- 172.21.15.86:0/1477425991 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f092001a070 con 0x7f092815e690 2024-04-20T21:17:59.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.076+0000 7f0924c00640 1 --2- 172.21.15.86:0/1477425991 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f090406c9c0 0x7f090406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.077+0000 7f0924c00640 1 -- 172.21.15.86:0/1477425991 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f092008be30 con 0x7f092815e690 2024-04-20T21:17:59.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.077+0000 7f0924c00640 1 --2- 172.21.15.86:0/1477425991 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x7f0904074be0 0x7f0904077000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.077+0000 7f0924c00640 1 -- 172.21.15.86:0/1477425991 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0920020ec0 con 0x7f0904074be0 2024-04-20T21:17:59.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.077+0000 7f0924c00640 1 -- 172.21.15.86:0/1477425991 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f092008c220 con 0x7f092815e690 2024-04-20T21:17:59.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.077+0000 7f0927e00640 1 --2- 172.21.15.86:0/1477425991 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f090406c9c0 0x7f090406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.077+0000 7f3a62000640 1 -- 172.21.15.86:0/3604546919 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3a6403e9d0 con 0x7f3a6c15ef60 2024-04-20T21:17:59.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.078+0000 7f0917400640 1 --2- 172.21.15.86:0/1477425991 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x7f0904074be0 0x7f0904077000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.078+0000 7f0927e00640 1 --2- 172.21.15.86:0/1477425991 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f090406c9c0 0x7f090406ee80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f09180099a0 tx=0x7f09180090e0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.079+0000 7f3a62000640 1 --2- 172.21.15.86:0/3604546919 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3a4c06c9f0 0x7f3a4c06eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.079+0000 7f3a62000640 1 -- 172.21.15.86:0/3604546919 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3a640af200 con 0x7f3a6c15ef60 2024-04-20T21:17:59.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.079+0000 7f0917400640 1 --2- 172.21.15.86:0/1477425991 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x7f0904074be0 0x7f0904077000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.079+0000 7f3a62000640 1 --2- 172.21.15.86:0/3604546919 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x7f3a4c074c70 0x7f3a4c077090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.079+0000 7f3a62000640 1 -- 172.21.15.86:0/3604546919 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3a4c077740 con 0x7f3a4c074c70 2024-04-20T21:17:59.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.079+0000 7f3a62000640 1 -- 172.21.15.86:0/3604546919 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f3a640af5f0 con 0x7f3a6c15ef60 2024-04-20T21:17:59.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.080+0000 7f3a69e00640 1 --2- 172.21.15.86:0/3604546919 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3a4c06c9f0 0x7f3a4c06eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.080+0000 7f12c6a00640 1 -- 172.21.15.86:0/2184573140 <== osd.5 v2:172.21.15.144:6808/1635974704 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f12bc001d30 con 0x7f12b0074c10 2024-04-20T21:17:59.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.080+0000 7f3a69400640 1 --2- 172.21.15.86:0/3604546919 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x7f3a4c074c70 0x7f3a4c077090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.081+0000 7f3a69e00640 1 --2- 172.21.15.86:0/3604546919 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3a4c06c9f0 0x7f3a4c06eeb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c007fd0 tx=0x7f3a5c007e80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.082+0000 7f3a69400640 1 --2- 172.21.15.86:0/3604546919 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x7f3a4c074c70 0x7f3a4c077090 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.082+0000 7f0924c00640 1 -- 172.21.15.86:0/1477425991 <== osd.6 v2:172.21.15.144:6824/1893803079 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0920020ec0 con 0x7f0904074be0 2024-04-20T21:17:59.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.084+0000 7f3a62000640 1 -- 172.21.15.86:0/3604546919 <== osd.7 v2:172.21.15.144:6816/2100967057 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3a4c077740 con 0x7f3a4c074c70 2024-04-20T21:17:59.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.091+0000 7f41bac00640 1 -- 172.21.15.86:0/1434226228 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f41ac056bf0 con 0x7f4198075990 2024-04-20T21:17:59.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.096+0000 7f41b1600640 1 -- 172.21.15.86:0/1434226228 <== osd.3 v2:172.21.15.86:6808/794135519 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f41ac056bf0 con 0x7f4198075990 2024-04-20T21:17:59.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.096+0000 7f41bac00640 1 -- 172.21.15.86:0/1434226228 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x7f4198075990 msgr2=0x7f4198077e30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.096+0000 7f41bac00640 1 --2- 172.21.15.86:0/1434226228 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x7f4198075990 0x7f4198077e30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.096+0000 7f001be00640 1 -- 172.21.15.86:0/3169744386 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f001c08c4c0 con 0x7efff8074b10 2024-04-20T21:17:59.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.097+0000 7f41bac00640 1 -- 172.21.15.86:0/1434226228 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f419806cb20 msgr2=0x7f419806efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.097+0000 7f41bac00640 1 --2- 172.21.15.86:0/1434226228 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f419806cb20 0x7f419806efe0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f41a4036000 tx=0x7f41a403b040 comp rx=0 tx=0).stop 2024-04-20T21:17:59.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.097+0000 7f41bac00640 1 -- 172.21.15.86:0/1434226228 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f41ac053cb0 msgr2=0x7f41ac0e3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.097+0000 7f41bac00640 1 --2- 172.21.15.86:0/1434226228 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f41ac053cb0 0x7f41ac0e3d40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f41a800c7f0 tx=0x7f41a800cbf0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.097+0000 7f41ba200640 1 -- 172.21.15.86:0/1434226228 reap_dead start 2024-04-20T21:17:59.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.097+0000 7f41bac00640 1 -- 172.21.15.86:0/1434226228 shutdown_connections 2024-04-20T21:17:59.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.097+0000 7f41bac00640 1 -- 172.21.15.86:0/1434226228 >> 172.21.15.86:0/1434226228 conn(0x7f41ac058ee0 msgr2=0x7f41ac0399c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:59.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.097+0000 7f41bac00640 1 -- 172.21.15.86:0/1434226228 shutdown_connections 2024-04-20T21:17:59.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.097+0000 7f41bac00640 1 -- 172.21.15.86:0/1434226228 wait complete. 2024-04-20T21:17:59.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.097+0000 7f0012a00640 1 -- 172.21.15.86:0/3169744386 <== osd.1 v2:172.21.15.86:6817/3149564961 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f001c08c4c0 con 0x7efff8074b10 2024-04-20T21:17:59.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.098+0000 7f001be00640 1 -- 172.21.15.86:0/3169744386 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x7efff8074b10 msgr2=0x7efff8076f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.098+0000 7f001be00640 1 --2- 172.21.15.86:0/3169744386 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x7efff8074b10 0x7efff8076f30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.098+0000 7f001be00640 1 -- 172.21.15.86:0/3169744386 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efff806c8f0 msgr2=0x7efff806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.098+0000 7f001be00640 1 --2- 172.21.15.86:0/3169744386 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efff806c8f0 0x7efff806edb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0014035000 tx=0x7f001402e000 comp rx=0 tx=0).stop 2024-04-20T21:17:59.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.098+0000 7f001be00640 1 -- 172.21.15.86:0/3169744386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f001c0cd630 msgr2=0x7f001c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.098+0000 7f001be00640 1 --2- 172.21.15.86:0/3169744386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f001c0cd630 0x7f001c0b4d90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f001c0b7160 tx=0x7f000400f930 comp rx=0 tx=0).stop 2024-04-20T21:17:59.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.100+0000 7f001b400640 1 -- 172.21.15.86:0/3169744386 reap_dead start 2024-04-20T21:17:59.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.100+0000 7f001be00640 1 -- 172.21.15.86:0/3169744386 shutdown_connections 2024-04-20T21:17:59.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.100+0000 7f001be00640 1 -- 172.21.15.86:0/3169744386 >> 172.21.15.86:0/3169744386 conn(0x7f001c08bf70 msgr2=0x7f001c0b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:59.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.100+0000 7f001be00640 1 -- 172.21.15.86:0/3169744386 shutdown_connections 2024-04-20T21:17:59.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.101+0000 7f001be00640 1 -- 172.21.15.86:0/3169744386 wait complete. 2024-04-20T21:17:59.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.106+0000 7f092d400640 1 -- 172.21.15.86:0/1477425991 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0928072e90 con 0x7f0904074be0 2024-04-20T21:17:59.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.109+0000 7f0924c00640 1 -- 172.21.15.86:0/1477425991 <== osd.6 v2:172.21.15.144:6824/1893803079 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0928072e90 con 0x7f0904074be0 2024-04-20T21:17:59.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.110+0000 7f1750e00640 1 -- 172.21.15.86:0/556424118 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1744007540 con 0x7f172c074a90 2024-04-20T21:17:59.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.111+0000 7f1742000640 1 -- 172.21.15.86:0/556424118 <== osd.0 v2:172.21.15.86:6800/2471076035 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1744007540 con 0x7f172c074a90 2024-04-20T21:17:59.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.112+0000 7f1750e00640 1 -- 172.21.15.86:0/556424118 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x7f172c074a90 msgr2=0x7f172c076eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.112+0000 7f1750e00640 1 --2- 172.21.15.86:0/556424118 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x7f172c074a90 0x7f172c076eb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.113+0000 7f092d400640 1 -- 172.21.15.86:0/1477425991 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x7f0904074be0 msgr2=0x7f0904077000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.113+0000 7f092d400640 1 --2- 172.21.15.86:0/1477425991 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x7f0904074be0 0x7f0904077000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.113+0000 7fa343400640 1 -- 172.21.15.86:0/1827703005 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa3580d2e60 con 0x7fa338074be0 2024-04-20T21:17:59.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.113+0000 7f092d400640 1 -- 172.21.15.86:0/1477425991 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f090406c9c0 msgr2=0x7f090406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.113+0000 7f092d400640 1 --2- 172.21.15.86:0/1477425991 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f090406c9c0 0x7f090406ee80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f09180099a0 tx=0x7f09180090e0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.113+0000 7f092d400640 1 -- 172.21.15.86:0/1477425991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f092815e690 msgr2=0x7f0928157300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.113+0000 7f092d400640 1 --2- 172.21.15.86:0/1477425991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f092815e690 0x7f0928157300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f092000ca70 tx=0x7f092000cf30 comp rx=0 tx=0).stop 2024-04-20T21:17:59.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.113+0000 7f092ca00640 1 -- 172.21.15.86:0/1477425991 reap_dead start 2024-04-20T21:17:59.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.113+0000 7f1750e00640 1 -- 172.21.15.86:0/556424118 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f172c06c8f0 msgr2=0x7f172c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.113+0000 7f1750e00640 1 --2- 172.21.15.86:0/556424118 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f172c06c8f0 0x7f172c06edb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f173c007fd0 tx=0x7f173c009040 comp rx=0 tx=0).stop 2024-04-20T21:17:59.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.113+0000 7f1750e00640 1 -- 172.21.15.86:0/556424118 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1744054120 msgr2=0x7f17440dc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.113+0000 7f1750e00640 1 --2- 172.21.15.86:0/556424118 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1744054120 0x7f17440dc5c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1738033000 tx=0x7f17380046c0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.114+0000 7fa34e000640 1 -- 172.21.15.86:0/1827703005 <== osd.2 v2:172.21.15.86:6816/3437611468 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa3580d2e60 con 0x7fa338074be0 2024-04-20T21:17:59.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.114+0000 7f174a800640 1 -- 172.21.15.86:0/556424118 reap_dead start 2024-04-20T21:17:59.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.114+0000 7f1750e00640 1 -- 172.21.15.86:0/556424118 shutdown_connections 2024-04-20T21:17:59.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.114+0000 7f1750e00640 1 -- 172.21.15.86:0/556424118 >> 172.21.15.86:0/556424118 conn(0x7f17440559d0 msgr2=0x7f17440578c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:59.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.113+0000 7f092d400640 1 -- 172.21.15.86:0/1477425991 shutdown_connections 2024-04-20T21:17:59.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.114+0000 7fa343400640 1 -- 172.21.15.86:0/1827703005 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x7fa338074be0 msgr2=0x7fa338077000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.114+0000 7f3a71000640 1 -- 172.21.15.86:0/3604546919 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3a44002dd0 con 0x7f3a4c074c70 2024-04-20T21:17:59.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.114+0000 7fa343400640 1 --2- 172.21.15.86:0/1827703005 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x7fa338074be0 0x7fa338077000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.114+0000 7fa343400640 1 -- 172.21.15.86:0/1827703005 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa33806c9c0 msgr2=0x7fa33806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.114+0000 7fa343400640 1 --2- 172.21.15.86:0/1827703005 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa33806c9c0 0x7fa33806ee80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa344009560 tx=0x7fa344008040 comp rx=0 tx=0).stop 2024-04-20T21:17:59.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.114+0000 7fa343400640 1 -- 172.21.15.86:0/1827703005 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3580b13c0 msgr2=0x7fa358200da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.114+0000 7fa343400640 1 --2- 172.21.15.86:0/1827703005 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3580b13c0 0x7fa358200da0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa348033000 tx=0x7fa3480046c0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.115+0000 7fa356800640 1 -- 172.21.15.86:0/1827703005 reap_dead start 2024-04-20T21:17:59.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.115+0000 7fa343400640 1 -- 172.21.15.86:0/1827703005 shutdown_connections 2024-04-20T21:17:59.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.115+0000 7fa343400640 1 -- 172.21.15.86:0/1827703005 >> 172.21.15.86:0/1827703005 conn(0x7fa35809da60 msgr2=0x7fa3580749f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:59.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.115+0000 7f1750e00640 1 -- 172.21.15.86:0/556424118 shutdown_connections 2024-04-20T21:17:59.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.115+0000 7f1750e00640 1 -- 172.21.15.86:0/556424118 wait complete. 2024-04-20T21:17:59.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.113+0000 7f092d400640 1 -- 172.21.15.86:0/1477425991 >> 172.21.15.86:0/1477425991 conn(0x7f092809d8a0 msgr2=0x7f09280c74e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:59.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.115+0000 7fa343400640 1 -- 172.21.15.86:0/1827703005 shutdown_connections 2024-04-20T21:17:59.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.115+0000 7f12d4a00640 1 -- 172.21.15.86:0/2184573140 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f12d00b5290 con 0x7f12b0074c10 2024-04-20T21:17:59.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.115+0000 7fa343400640 1 -- 172.21.15.86:0/1827703005 wait complete. 2024-04-20T21:17:59.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.115+0000 7f092d400640 1 -- 172.21.15.86:0/1477425991 shutdown_connections 2024-04-20T21:17:59.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.116+0000 7f092d400640 1 -- 172.21.15.86:0/1477425991 wait complete. 2024-04-20T21:17:59.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.116+0000 7f3a62000640 1 -- 172.21.15.86:0/3604546919 <== osd.7 v2:172.21.15.144:6816/2100967057 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3a44002dd0 con 0x7f3a4c074c70 2024-04-20T21:17:59.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.116+0000 7f9d39800640 1 -- 172.21.15.86:0/3467630821 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9d34072c80 con 0x7f9d14074d40 2024-04-20T21:17:59.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.116+0000 7f12c6a00640 1 -- 172.21.15.86:0/2184573140 <== osd.5 v2:172.21.15.144:6808/1635974704 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f12d00b5290 con 0x7f12b0074c10 2024-04-20T21:17:59.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.117+0000 7f12d4a00640 1 -- 172.21.15.86:0/2184573140 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x7f12b0074c10 msgr2=0x7f12b0077030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.117+0000 7f3a71000640 1 -- 172.21.15.86:0/3604546919 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x7f3a4c074c70 msgr2=0x7f3a4c077090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.117+0000 7f3a71000640 1 --2- 172.21.15.86:0/3604546919 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x7f3a4c074c70 0x7f3a4c077090 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.117+0000 7f9d2aa00640 1 -- 172.21.15.86:0/3467630821 <== osd.4 v2:172.21.15.144:6800/4087309475 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9d34072c80 con 0x7f9d14074d40 2024-04-20T21:17:59.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.117+0000 7f12d4a00640 1 --2- 172.21.15.86:0/2184573140 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x7f12b0074c10 0x7f12b0077030 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.117+0000 7f12d4a00640 1 -- 172.21.15.86:0/2184573140 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f12b006c9f0 msgr2=0x7f12b006eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.117+0000 7f12d4a00640 1 --2- 172.21.15.86:0/2184573140 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f12b006c9f0 0x7f12b006eeb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f12c0006a90 tx=0x7f12c0030000 comp rx=0 tx=0).stop 2024-04-20T21:17:59.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.117+0000 7f12d4a00640 1 -- 172.21.15.86:0/2184573140 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f12b4004800 msgr2=0x7f12d0163ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.117+0000 7f12d4a00640 1 --2- 172.21.15.86:0/2184573140 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f12b4004800 0x7f12d0163ee0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f12bc00f5a0 tx=0x7f12bc00f9a0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.118+0000 7f12cf200640 1 -- 172.21.15.86:0/2184573140 reap_dead start 2024-04-20T21:17:59.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.118+0000 7f12d4a00640 1 -- 172.21.15.86:0/2184573140 shutdown_connections 2024-04-20T21:17:59.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.118+0000 7f12d4a00640 1 -- 172.21.15.86:0/2184573140 >> 172.21.15.86:0/2184573140 conn(0x7f12d00b2220 msgr2=0x7f12d00b4610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:59.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.118+0000 7f9d39800640 1 -- 172.21.15.86:0/3467630821 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x7f9d14074d40 msgr2=0x7f9d14077160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.118+0000 7f9d39800640 1 --2- 172.21.15.86:0/3467630821 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x7f9d14074d40 0x7f9d14077160 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.119+0000 7f9d39800640 1 -- 172.21.15.86:0/3467630821 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9d1406cb20 msgr2=0x7f9d1406efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.119+0000 7f9d39800640 1 --2- 172.21.15.86:0/3467630821 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9d1406cb20 0x7f9d1406efe0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9d341f8140 tx=0x7f9d24008040 comp rx=0 tx=0).stop 2024-04-20T21:17:59.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.119+0000 7f9d39800640 1 -- 172.21.15.86:0/3467630821 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d34071d40 msgr2=0x7f9d3415bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.119+0000 7f9d39800640 1 --2- 172.21.15.86:0/3467630821 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d34071d40 0x7f9d3415bfe0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c00ccd0 tx=0x7f9d2c00a5c0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.119+0000 7f3a71000640 1 -- 172.21.15.86:0/3604546919 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3a4c06c9f0 msgr2=0x7f3a4c06eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.119+0000 7f3a71000640 1 --2- 172.21.15.86:0/3604546919 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3a4c06c9f0 0x7f3a4c06eeb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c007fd0 tx=0x7f3a5c007e80 comp rx=0 tx=0).stop 2024-04-20T21:17:59.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.119+0000 7f3a71000640 1 -- 172.21.15.86:0/3604546919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a6c15ef60 msgr2=0x7f3a6c14b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.119+0000 7f3a71000640 1 --2- 172.21.15.86:0/3604546919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a6c15ef60 0x7f3a6c14b5c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3a64009d90 tx=0x7f3a64004a40 comp rx=0 tx=0).stop 2024-04-20T21:17:59.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.119+0000 7f12d4a00640 1 -- 172.21.15.86:0/2184573140 shutdown_connections 2024-04-20T21:17:59.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.119+0000 7f3a6a800640 1 -- 172.21.15.86:0/3604546919 reap_dead start 2024-04-20T21:17:59.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.119+0000 7f12d4a00640 1 -- 172.21.15.86:0/2184573140 wait complete. 2024-04-20T21:17:59.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.119+0000 7f3a71000640 1 -- 172.21.15.86:0/3604546919 shutdown_connections 2024-04-20T21:17:59.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.119+0000 7f3a71000640 1 -- 172.21.15.86:0/3604546919 >> 172.21.15.86:0/3604546919 conn(0x7f3a6c06cc20 msgr2=0x7f3a6c1615d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:59.154 INFO:teuthology.orchestra.run.smithi086.stdout:55834574855 2024-04-20T21:17:59.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.120+0000 7f9d33400640 1 -- 172.21.15.86:0/3467630821 reap_dead start 2024-04-20T21:17:59.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.120+0000 7f9d39800640 1 -- 172.21.15.86:0/3467630821 shutdown_connections 2024-04-20T21:17:59.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.120+0000 7f9d39800640 1 -- 172.21.15.86:0/3467630821 >> 172.21.15.86:0/3467630821 conn(0x7f9d3406b980 msgr2=0x7f9d340b1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:59.155 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-20T21:17:59.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.120+0000 7f9d39800640 1 -- 172.21.15.86:0/3467630821 shutdown_connections 2024-04-20T21:17:59.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.120+0000 7f9d39800640 1 -- 172.21.15.86:0/3467630821 wait complete. 2024-04-20T21:17:59.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.120+0000 7f3a71000640 1 -- 172.21.15.86:0/3604546919 shutdown_connections 2024-04-20T21:17:59.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.121+0000 7f3a71000640 1 -- 172.21.15.86:0/3604546919 wait complete. 2024-04-20T21:17:59.157 INFO:teuthology.orchestra.run.smithi086.stdout:55834574854 2024-04-20T21:17:59.157 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-20T21:17:59.158 INFO:teuthology.orchestra.run.smithi086.stdout:55834574854 2024-04-20T21:17:59.158 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-20T21:17:59.159 INFO:teuthology.orchestra.run.smithi086.stdout:55834574855 2024-04-20T21:17:59.159 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-20T21:17:59.160 INFO:teuthology.orchestra.run.smithi086.stdout:55834574855 2024-04-20T21:17:59.160 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-20T21:17:59.161 INFO:teuthology.orchestra.run.smithi086.stdout:55834574856 2024-04-20T21:17:59.161 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-20T21:17:59.162 INFO:teuthology.orchestra.run.smithi086.stdout:55834574855 2024-04-20T21:17:59.162 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-20T21:17:59.163 INFO:teuthology.orchestra.run.smithi086.stdout:55834574853 2024-04-20T21:17:59.163 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-20T21:17:59.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.474+0000 7fbd26400640 1 Processor -- start 2024-04-20T21:17:59.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.474+0000 7fbd26400640 1 -- start start 2024-04-20T21:17:59.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.476+0000 7fbd26400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbd180e7710 0x7fbd1804ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.476+0000 7fbd26400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbd180589c0 0x7fbd18053530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.477+0000 7fbd26400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd18057d90 con 0x7fbd18056ab0 2024-04-20T21:17:59.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.477+0000 7fbd26400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fbd18006260 con 0x7fbd180589c0 2024-04-20T21:17:59.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.477+0000 7fbd26400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fbd18006540 con 0x7fbd180e7710 2024-04-20T21:17:59.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.477+0000 7fbd1fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbd180589c0 0x7fbd18053530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.477+0000 7fbd25000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbd180e7710 0x7fbd1804ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.478+0000 7fbd25000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbd180e7710 0x7fbd1804ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.86:58024/0 (socket says 172.21.15.86:58024) 2024-04-20T21:17:59.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.478+0000 7fbd25000640 1 -- 172.21.15.86:0/147898311 learned_addr learned my addr 172.21.15.86:0/147898311 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:59.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.479+0000 7fbd25000640 1 -- 172.21.15.86:0/147898311 >> v2:172.21.15.86:3300/0 conn(0x7fbd180589c0 msgr2=0x7fbd18053530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:59.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.479+0000 7fbd25000640 1 --2- 172.21.15.86:0/147898311 >> v2:172.21.15.86:3300/0 conn(0x7fbd180589c0 0x7fbd18053530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.479+0000 7fbd25000640 1 -- 172.21.15.86:0/147898311 >> v1:172.21.15.86:6789/0 conn(0x7fbd18056ab0 legacy=0x7fbd18007020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:59.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.480+0000 7fbd1f400640 1 -- 172.21.15.86:0/147898311 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3358413630 0 0) 0x7fbd18057d90 con 0x7fbd18056ab0 2024-04-20T21:17:59.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.480+0000 7fbd25000640 1 -- 172.21.15.86:0/147898311 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd18006820 con 0x7fbd180e7710 2024-04-20T21:17:59.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.481+0000 7fbd25000640 1 --2- 172.21.15.86:0/147898311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd180e7710 0x7fbd1804ca30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbd20051240 tx=0x7fbd200a73f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17d78c38310fcbf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.482+0000 7fbd1f400640 1 -- 172.21.15.86:0/147898311 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd200b3070 con 0x7fbd180e7710 2024-04-20T21:17:59.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.482+0000 7fbd1f400640 1 -- 172.21.15.86:0/147898311 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd200ac810 con 0x7fbd180e7710 2024-04-20T21:17:59.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.483+0000 7fbd26400640 1 -- 172.21.15.86:0/147898311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd180e7710 msgr2=0x7fbd1804ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.483+0000 7fbd26400640 1 --2- 172.21.15.86:0/147898311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd180e7710 0x7fbd1804ca30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbd20051240 tx=0x7fbd200a73f0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.484+0000 7fbd26400640 1 -- 172.21.15.86:0/147898311 shutdown_connections 2024-04-20T21:17:59.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.484+0000 7fbd26400640 1 --2- 172.21.15.86:0/147898311 >> v2:172.21.15.86:3300/0 conn(0x7fbd180589c0 0x7fbd18053530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.484+0000 7fbd26400640 1 --2- 172.21.15.86:0/147898311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd180e7710 0x7fbd1804ca30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.484+0000 7fbd26400640 1 -- 172.21.15.86:0/147898311 >> 172.21.15.86:0/147898311 conn(0x7fbd18058ee0 msgr2=0x7fbd180592c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:59.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.484+0000 7fbd26400640 1 -- 172.21.15.86:0/147898311 shutdown_connections 2024-04-20T21:17:59.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.485+0000 7fbd26400640 1 -- 172.21.15.86:0/147898311 wait complete. 2024-04-20T21:17:59.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.487+0000 7f3083200640 1 Processor -- start 2024-04-20T21:17:59.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.487+0000 7f3083200640 1 -- start start 2024-04-20T21:17:59.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.487+0000 7fbd26400640 1 Processor -- start 2024-04-20T21:17:59.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.488+0000 7fbd26400640 1 -- start start 2024-04-20T21:17:59.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.489+0000 7f47d5a00640 1 Processor -- start 2024-04-20T21:17:59.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.489+0000 7f47d5a00640 1 -- start start 2024-04-20T21:17:59.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.490+0000 7fbd26400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd18056ab0 0x7fbd180eb070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.490+0000 7fbd26400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd180589c0 0x7fbd180dacb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.490+0000 7fbd26400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd180e7710 0x7fbd1817e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.489+0000 7f3083200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f308406f480 0x7f308415ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.490+0000 7f3081e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f308406f480 0x7f308415ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.490+0000 7f3081400640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f308415e320 0x7f30840b42e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.86:36678/0 (socket says 172.21.15.86:36678) 2024-04-20T21:17:59.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.490+0000 7f3081e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f308406f480 0x7f308415ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.86:58038/0 (socket says 172.21.15.86:58038) 2024-04-20T21:17:59.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.490+0000 7f3081400640 1 -- 172.21.15.86:0/1189566667 learned_addr learned my addr 172.21.15.86:0/1189566667 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:17:59.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.490+0000 7fbd26400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbd180579a0 con 0x7fbd18056ab0 2024-04-20T21:17:59.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.490+0000 7fbd26400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1804cf70 con 0x7fbd180589c0 2024-04-20T21:17:59.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.490+0000 7fbd26400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbd18057b20 con 0x7fbd180e7710 2024-04-20T21:17:59.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.491+0000 7fbd25000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd18056ab0 0x7fbd180eb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.491+0000 7f3083200640 1 --2- 172.21.15.86:0/1189566667 >> v2:172.21.15.86:3300/0 conn(0x7f308406fc60 0x7f30840cf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.491+0000 7fbd25000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd18056ab0 0x7fbd180eb070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57826/0 (socket says 172.21.15.86:57826) 2024-04-20T21:17:59.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.491+0000 7fbd25000640 1 -- 172.21.15.86:0/3009965254 learned_addr learned my addr 172.21.15.86:0/3009965254 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:59.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.491+0000 7f3083200640 1 -- 172.21.15.86:0/1189566667 --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f308415eb50 con 0x7f308415e320 2024-04-20T21:17:59.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.491+0000 7fbd25a00640 1 --2- 172.21.15.86:0/3009965254 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd180e7710 0x7fbd1817e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.491+0000 7f3083200640 1 -- 172.21.15.86:0/1189566667 --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f30840ced50 con 0x7f308406fc60 2024-04-20T21:17:59.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.491+0000 7f3083200640 1 -- 172.21.15.86:0/1189566667 --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f308415ee30 con 0x7f308406f480 2024-04-20T21:17:59.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.491+0000 7f3082800640 1 --2- 172.21.15.86:0/1189566667 >> v2:172.21.15.86:3300/0 conn(0x7f308406fc60 0x7f30840cf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.491+0000 7fbd1fe00640 1 --2- 172.21.15.86:0/3009965254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd180589c0 0x7fbd180dacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.492+0000 7fbd25000640 1 -- 172.21.15.86:0/3009965254 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd180e7710 msgr2=0x7fbd1817e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.492+0000 7f47d5a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47d006c090 0x7f47d00ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.492+0000 7fbd25000640 1 --2- 172.21.15.86:0/3009965254 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd180e7710 0x7fbd1817e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.492+0000 7fbd25000640 1 -- 172.21.15.86:0/3009965254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd180589c0 msgr2=0x7fbd180dacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.492+0000 7f47d5a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47d00b44f0 con 0x7f47d0162c70 2024-04-20T21:17:59.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.492+0000 7f47d5a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47d00b4210 con 0x7f47d0072a30 2024-04-20T21:17:59.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.492+0000 7f47d5a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f47d00725a0 con 0x7f47d006c090 2024-04-20T21:17:59.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.492+0000 7fbd25000640 1 --2- 172.21.15.86:0/3009965254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd180589c0 0x7fbd180dacb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.492+0000 7f3080a00640 1 -- 172.21.15.86:0/1189566667 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2866718003 0 0) 0x7f308415eb50 con 0x7f308415e320 2024-04-20T21:17:59.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.492+0000 7f47cea00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f47d0162c70 0x7f47d00b3800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:43548/0 (socket says 172.21.15.86:43548) 2024-04-20T21:17:59.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.492+0000 7f47cea00640 1 -- 172.21.15.86:0/1216634385 learned_addr learned my addr 172.21.15.86:0/1216634385 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:17:59.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.493+0000 7f3081e00640 1 -- 172.21.15.86:0/1189566667 >> v2:172.21.15.86:3300/0 conn(0x7f308406fc60 msgr2=0x7f30840cf4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:59.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.492+0000 7fbd25000640 1 -- 172.21.15.86:0/3009965254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd180e9870 con 0x7fbd18056ab0 2024-04-20T21:17:59.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.494+0000 7fbd25000640 1 --2- 172.21.15.86:0/3009965254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd18056ab0 0x7fbd180eb070 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbd2006a870 tx=0x7fbd20069a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.494+0000 7f47cd600640 1 -- 172.21.15.86:0/1216634385 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3751275248 0 0) 0x7f47bc001110 con 0x7f47d0162c70 2024-04-20T21:17:59.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.494+0000 7f47cd600640 1 -- 172.21.15.86:0/1216634385 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f47b4003520 con 0x7f47d0162c70 2024-04-20T21:17:59.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.494+0000 7fbd1cc00640 1 -- 172.21.15.86:0/3009965254 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd200b3040 con 0x7fbd18056ab0 2024-04-20T21:17:59.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.493+0000 7f3081e00640 1 --2- 172.21.15.86:0/1189566667 >> v2:172.21.15.86:3300/0 conn(0x7f308406fc60 0x7f30840cf4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.493+0000 7f3081e00640 1 -- 172.21.15.86:0/1189566667 >> v1:172.21.15.144:6789/0 conn(0x7f308415e320 legacy=0x7f30840b42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.493+0000 7f3081e00640 1 -- 172.21.15.86:0/1189566667 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f308415f110 con 0x7f308406f480 2024-04-20T21:17:59.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.494+0000 7f3082800640 1 --2- 172.21.15.86:0/1189566667 >> v2:172.21.15.86:3300/0 conn(0x7f308406fc60 0x7f30840cf4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:59.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.495+0000 7f47cd600640 1 -- 172.21.15.86:0/1216634385 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2198583054 0 0) 0x7f47b4003520 con 0x7f47d0162c70 2024-04-20T21:17:59.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.495+0000 7f47cd600640 1 -- 172.21.15.86:0/1216634385 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47d006c090 msgr2=0x7f47d00ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:59.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.495+0000 7f47cd600640 1 --2- 172.21.15.86:0/1216634385 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47d006c090 0x7f47d00ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.495+0000 7f47cd600640 1 -- 172.21.15.86:0/1216634385 >> v1:172.21.15.144:6789/0 conn(0x7f47d0072a30 legacy=0x7f47d006b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.495+0000 7f47cd600640 1 -- 172.21.15.86:0/1216634385 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47d0071f50 con 0x7f47d0162c70 2024-04-20T21:17:59.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.495+0000 7f47cd600640 1 -- 172.21.15.86:0/1216634385 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f47bc001840 con 0x7f47d0162c70 2024-04-20T21:17:59.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.495+0000 7f47cd600640 1 -- 172.21.15.86:0/1216634385 >> v1:172.21.15.86:6789/0 conn(0x7f47d0162c70 legacy=0x7f47d00b3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.495+0000 7f47cd600640 1 --2- 172.21.15.86:0/1216634385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47b40039e0 0x7f47b4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.495+0000 7f47cd600640 1 --2- 172.21.15.86:0/1216634385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47b4004800 0x7f47b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.495+0000 7f47cd600640 1 --2- 172.21.15.86:0/1216634385 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47b40052e0 0x7f47b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.495+0000 7f47cd600640 1 -- 172.21.15.86:0/1216634385 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f47d00725a0 con 0x7f47b4004800 2024-04-20T21:17:59.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.495+0000 7f47cd600640 1 -- 172.21.15.86:0/1216634385 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f47b400c160 con 0x7f47b40039e0 2024-04-20T21:17:59.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.495+0000 7f47cd600640 1 -- 172.21.15.86:0/1216634385 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f47b400c360 con 0x7f47b40052e0 2024-04-20T21:17:59.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.494+0000 7fbd26400640 1 -- 172.21.15.86:0/3009965254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd180e9b50 con 0x7fbd18056ab0 2024-04-20T21:17:59.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.494+0000 7fbd26400640 1 -- 172.21.15.86:0/3009965254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd180da790 con 0x7fbd18056ab0 2024-04-20T21:17:59.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.494+0000 7fbd1cc00640 1 -- 172.21.15.86:0/3009965254 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd20067880 con 0x7fbd18056ab0 2024-04-20T21:17:59.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.494+0000 7fbd1cc00640 1 -- 172.21.15.86:0/3009965254 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd20067c70 con 0x7fbd18056ab0 2024-04-20T21:17:59.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.495+0000 7f3081e00640 1 --2- 172.21.15.86:0/1189566667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f308406f480 0x7f308415ad30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f30840ceed0 tx=0x7f307002cdb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4fd4e11b02f322b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.496+0000 7f47cea00640 1 --2- 172.21.15.86:0/1216634385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47b40039e0 0x7f47b4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.496+0000 7f47cf400640 1 --2- 172.21.15.86:0/1216634385 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47b40052e0 0x7f47b400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.496+0000 7f47ce000640 1 --2- 172.21.15.86:0/1216634385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47b4004800 0x7f47b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.496+0000 7f3080a00640 1 -- 172.21.15.86:0/1189566667 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307003a070 con 0x7f308406f480 2024-04-20T21:17:59.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.496+0000 7f3080a00640 1 -- 172.21.15.86:0/1189566667 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f307002db60 con 0x7f308406f480 2024-04-20T21:17:59.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.496+0000 7f3080a00640 1 -- 172.21.15.86:0/1189566667 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307000b040 con 0x7f308406f480 2024-04-20T21:17:59.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.497+0000 7f47cea00640 1 -- 172.21.15.86:0/1216634385 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47b40052e0 msgr2=0x7f47b400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:59.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.497+0000 7f47cea00640 1 --2- 172.21.15.86:0/1216634385 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47b40052e0 0x7f47b400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.497+0000 7f47cea00640 1 -- 172.21.15.86:0/1216634385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47b4004800 msgr2=0x7f47b4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:59.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.498+0000 7fb821c00640 1 Processor -- start 2024-04-20T21:17:59.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.498+0000 7fbd26400640 1 -- 172.21.15.86:0/3009965254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcfc005290 con 0x7fbd18056ab0 2024-04-20T21:17:59.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.498+0000 7f3083200640 1 -- 172.21.15.86:0/1189566667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f308406f480 msgr2=0x7f308415ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.498+0000 7f3083200640 1 --2- 172.21.15.86:0/1189566667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f308406f480 0x7f308415ad30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f30840ceed0 tx=0x7f307002cdb0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.498+0000 7fb821c00640 1 -- start start 2024-04-20T21:17:59.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.498+0000 7f3083200640 1 -- 172.21.15.86:0/1189566667 shutdown_connections 2024-04-20T21:17:59.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.498+0000 7f3083200640 1 --2- 172.21.15.86:0/1189566667 >> v2:172.21.15.86:3300/0 conn(0x7f308406fc60 0x7f30840cf4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.498+0000 7f3083200640 1 --2- 172.21.15.86:0/1189566667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f308406f480 0x7f308415ad30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.499+0000 7f3083200640 1 -- 172.21.15.86:0/1189566667 >> 172.21.15.86:0/1189566667 conn(0x7f308406bb60 msgr2=0x7f308406bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:59.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.499+0000 7f3083200640 1 -- 172.21.15.86:0/1189566667 shutdown_connections 2024-04-20T21:17:59.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.499+0000 7fb821c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb81c15ed80 0x7fb81c0d3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.499+0000 7fb821c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb81c0d0fd0 0x7fb81c0d1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.499+0000 7f3083200640 1 -- 172.21.15.86:0/1189566667 wait complete. 2024-04-20T21:17:59.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.499+0000 7fb821c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb81c0b7120 con 0x7fb81c0cf810 2024-04-20T21:17:59.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.499+0000 7fb821c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fb81c0d1950 con 0x7fb81c0d0fd0 2024-04-20T21:17:59.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.499+0000 7fb821c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb81c0d1b80 con 0x7fb81c15ed80 2024-04-20T21:17:59.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.499+0000 7f47cea00640 1 --2- 172.21.15.86:0/1216634385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47b4004800 0x7f47b4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.499+0000 7f47cea00640 1 -- 172.21.15.86:0/1216634385 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47d0071f50 con 0x7f47b40039e0 2024-04-20T21:17:59.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.499+0000 7f47ce000640 1 --2- 172.21.15.86:0/1216634385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47b4004800 0x7f47b4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:17:59.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.500+0000 7f4c95a00640 1 Processor -- start 2024-04-20T21:17:59.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.500+0000 7f771fa00640 1 Processor -- start 2024-04-20T21:17:59.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.500+0000 7fb81aa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb81c15ed80 0x7fb81c0d3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.500+0000 7fb81aa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb81c15ed80 0x7fb81c0d3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:55538/0 (socket says 172.21.15.86:55538) 2024-04-20T21:17:59.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.500+0000 7fb81aa00640 1 -- 172.21.15.86:0/63755037 learned_addr learned my addr 172.21.15.86:0/63755037 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:59.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.500+0000 7fb81a000640 1 --2- 172.21.15.86:0/63755037 >> v2:172.21.15.144:3300/0 conn(0x7fb81c0d0fd0 0x7fb81c0d1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.500+0000 7f771fa00640 1 -- start start 2024-04-20T21:17:59.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.500+0000 7f4c95a00640 1 -- start start 2024-04-20T21:17:59.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.501+0000 7fb81a000640 1 -- 172.21.15.86:0/63755037 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb81c15ed80 msgr2=0x7fb81c0d3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:59.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.501+0000 7fb81a000640 1 --2- 172.21.15.86:0/63755037 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb81c15ed80 0x7fb81c0d3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.501+0000 7fb81a000640 1 -- 172.21.15.86:0/63755037 >> v1:172.21.15.144:6789/0 conn(0x7fb81c0cf810 legacy=0x7fb81c0b7cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:17:59.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.501+0000 7fb81a000640 1 -- 172.21.15.86:0/63755037 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb81c0d2e20 con 0x7fb81c0d0fd0 2024-04-20T21:17:59.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.502+0000 7f8f4c000640 1 Processor -- start 2024-04-20T21:17:59.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.502+0000 7f771fa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7718072fd0 0x7f77180733b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.502+0000 7f771fa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f77180d6030 0x7f77181668d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.502+0000 7f771fa00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77180738f0 con 0x7f77180d8640 2024-04-20T21:17:59.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.502+0000 7f771fa00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f7718163f50 con 0x7f77180d6030 2024-04-20T21:17:59.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.502+0000 7f771fa00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7718166640 con 0x7f7718072fd0 2024-04-20T21:17:59.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.502+0000 7f8f4c000640 1 -- start start 2024-04-20T21:17:59.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.502+0000 7f47cea00640 1 --2- 172.21.15.86:0/1216634385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47b40039e0 0x7f47b4003e00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f47bc00c510 tx=0x7f47b4005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=622c73f3a030cbd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.502+0000 7f3083200640 1 Processor -- start 2024-04-20T21:17:59.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.502+0000 7fbd1cc00640 1 -- 172.21.15.86:0/3009965254 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbd200c0030 con 0x7fbd18056ab0 2024-04-20T21:17:59.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.503+0000 7f3083200640 1 -- start start 2024-04-20T21:17:59.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.503+0000 7f47cd600640 1 -- 172.21.15.86:0/1216634385 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc00c0e0 con 0x7f47b40039e0 2024-04-20T21:17:59.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.503+0000 7f771d000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7718072fd0 0x7f77180733b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.503+0000 7f771d000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7718072fd0 0x7f77180733b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:55554/0 (socket says 172.21.15.86:55554) 2024-04-20T21:17:59.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.503+0000 7f771d000640 1 -- 172.21.15.86:0/1772266580 learned_addr learned my addr 172.21.15.86:0/1772266580 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:59.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.503+0000 7fb81a000640 1 --2- 172.21.15.86:0/63755037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb81c0d0fd0 0x7fb81c0d1410 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb81c08d0a0 tx=0x7fb80802b780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47886829fabcea1d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.503+0000 7fbd1cc00640 1 --2- 172.21.15.86:0/3009965254 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbd0406c8f0 0x7fbd0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.503+0000 7f770fe00640 1 --2- 172.21.15.86:0/1772266580 >> v2:172.21.15.86:3300/0 conn(0x7f77180d6030 0x7f77181668d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.504+0000 7fbd1cc00640 1 -- 172.21.15.86:0/3009965254 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbd20129070 con 0x7fbd18056ab0 2024-04-20T21:17:59.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.504+0000 7fbd1fe00640 1 --2- 172.21.15.86:0/3009965254 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbd0406c8f0 0x7fbd0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.504+0000 7f771d000640 1 -- 172.21.15.86:0/1772266580 >> v2:172.21.15.86:3300/0 conn(0x7f77180d6030 msgr2=0x7f77181668d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:59.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.504+0000 7f771d000640 1 --2- 172.21.15.86:0/1772266580 >> v2:172.21.15.86:3300/0 conn(0x7f77180d6030 0x7f77181668d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.504+0000 7f771d000640 1 -- 172.21.15.86:0/1772266580 >> v1:172.21.15.86:6789/0 conn(0x7f77180d8640 legacy=0x7f7718166e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.504+0000 7f771d000640 1 -- 172.21.15.86:0/1772266580 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7718163620 con 0x7f7718072fd0 2024-04-20T21:17:59.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.504+0000 7f47cd600640 1 -- 172.21.15.86:0/1216634385 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47bc030070 con 0x7f47b40039e0 2024-04-20T21:17:59.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.504+0000 7f770fe00640 1 --2- 172.21.15.86:0/1772266580 >> v2:172.21.15.86:3300/0 conn(0x7f77180d6030 0x7f77181668d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:17:59.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.504+0000 7f47cd600640 1 -- 172.21.15.86:0/1216634385 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc0058d0 con 0x7f47b40039e0 2024-04-20T21:17:59.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.505+0000 7f3083200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f308406f480 0x7f30840d06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.504+0000 7fb819600640 1 -- 172.21.15.86:0/63755037 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb808037070 con 0x7fb81c0d0fd0 2024-04-20T21:17:59.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.504+0000 7fb819600640 1 -- 172.21.15.86:0/63755037 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb808030710 con 0x7fb81c0d0fd0 2024-04-20T21:17:59.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.504+0000 7fb819600640 1 -- 172.21.15.86:0/63755037 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb808030ae0 con 0x7fb81c0d0fd0 2024-04-20T21:17:59.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.505+0000 7f3083200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f308406fc60 0x7f30841f8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.504+0000 7f8f4c000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8f4415e850 0x7f8f440cfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.505+0000 7f8f4c000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f440d24d0 con 0x7f8f440b68f0 2024-04-20T21:17:59.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.505+0000 7f8f4c000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f440d27b0 con 0x7f8f4415ec30 2024-04-20T21:17:59.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.505+0000 7f8f4c000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f8f440d22a0 con 0x7f8f4415e850 2024-04-20T21:17:59.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.505+0000 7f3083200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f308415e320 0x7f30841596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.505+0000 7f47d5a00640 1 -- 172.21.15.86:0/1216634385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47b40039e0 msgr2=0x7f47b4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.505+0000 7f47d5a00640 1 --2- 172.21.15.86:0/1216634385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47b40039e0 0x7f47b4003e00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f47bc00c510 tx=0x7f47b4005830 comp rx=0 tx=0).stop 2024-04-20T21:17:59.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.505+0000 7fbd1fe00640 1 --2- 172.21.15.86:0/3009965254 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbd0406c8f0 0x7fbd0406edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbd180dbe00 tx=0x7fbd10008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.505+0000 7f8f48c00640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f8f4415ec30 0x7f8f440b61e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.86:36698/0 (socket says 172.21.15.86:36698) 2024-04-20T21:17:59.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.505+0000 7f8f48c00640 1 -- 172.21.15.86:0/1925656238 learned_addr learned my addr 172.21.15.86:0/1925656238 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:17:59.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.505+0000 7f47cf400640 1 -- 172.21.15.86:0/1216634385 reap_dead start 2024-04-20T21:17:59.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.506+0000 7f3081400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f308406fc60 0x7f30841f8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.506+0000 7f47d5a00640 1 -- 172.21.15.86:0/1216634385 shutdown_connections 2024-04-20T21:17:59.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.506+0000 7f47d5a00640 1 -- 172.21.15.86:0/1216634385 >> 172.21.15.86:0/1216634385 conn(0x7f47d00b2110 msgr2=0x7f47d00b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:59.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.506+0000 7f47d5a00640 1 -- 172.21.15.86:0/1216634385 shutdown_connections 2024-04-20T21:17:59.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.506+0000 7f3082800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f308415e320 0x7f30841596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.506+0000 7f8f49600640 1 --2- 172.21.15.86:0/1925656238 >> v2:172.21.15.144:3300/0 conn(0x7f8f4415e850 0x7f8f440cfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.506+0000 7f47d5a00640 1 -- 172.21.15.86:0/1216634385 wait complete. 2024-04-20T21:17:59.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.506+0000 7fb821c00640 1 -- 172.21.15.86:0/63755037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb81c0d0fd0 msgr2=0x7fb81c0d1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.506+0000 7fb821c00640 1 --2- 172.21.15.86:0/63755037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb81c0d0fd0 0x7fb81c0d1410 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb81c08d0a0 tx=0x7fb80802b780 comp rx=0 tx=0).stop 2024-04-20T21:17:59.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.506+0000 7f3082800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f308415e320 0x7f30841596f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57872/0 (socket says 172.21.15.86:57872) 2024-04-20T21:17:59.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.506+0000 7fb821c00640 1 -- 172.21.15.86:0/63755037 shutdown_connections 2024-04-20T21:17:59.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.506+0000 7fb821c00640 1 --2- 172.21.15.86:0/63755037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb81c0d0fd0 0x7fb81c0d1410 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.506+0000 7f4c95a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4c9015e850 0x7f4c900cfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.506+0000 7fb821c00640 1 --2- 172.21.15.86:0/63755037 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb81c15ed80 0x7fb81c0d3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.506+0000 7fb821c00640 1 -- 172.21.15.86:0/63755037 >> 172.21.15.86:0/63755037 conn(0x7fb81c06cc20 msgr2=0x7fb81c06d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:59.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.506+0000 7f3081400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f308406fc60 0x7f30841f8bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:55564/0 (socket says 172.21.15.86:55564) 2024-04-20T21:17:59.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.506+0000 7f3081400640 1 -- 172.21.15.86:0/2389144901 learned_addr learned my addr 172.21.15.86:0/2389144901 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:59.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.506+0000 7f3083200640 1 -- 172.21.15.86:0/2389144901 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f308415e900 con 0x7f308415e320 2024-04-20T21:17:59.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.506+0000 7f3083200640 1 -- 172.21.15.86:0/2389144901 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f308415e780 con 0x7f308406f480 2024-04-20T21:17:59.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.506+0000 7f3083200640 1 -- 172.21.15.86:0/2389144901 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f308406f860 con 0x7f308406fc60 2024-04-20T21:17:59.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.506+0000 7f3081e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f308406f480 0x7f30840d06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.506+0000 7f4c95a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c9015ec30 0x7f4c900d0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.507+0000 7fb821c00640 1 -- 172.21.15.86:0/63755037 shutdown_connections 2024-04-20T21:17:59.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.507+0000 7f8f49600640 1 -- 172.21.15.86:0/1925656238 >> v1:172.21.15.144:6789/0 conn(0x7f8f4415ec30 legacy=0x7f8f440b61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.507+0000 7f8f49600640 1 -- 172.21.15.86:0/1925656238 >> v1:172.21.15.86:6789/0 conn(0x7f8f440b68f0 legacy=0x7f8f44168870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.507+0000 7f8f43e00640 1 -- 172.21.15.86:0/1925656238 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 136980989 0 0) 0x7f8f440d27b0 con 0x7f8f4415ec30 2024-04-20T21:17:59.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.507+0000 7f8f49600640 1 -- 172.21.15.86:0/1925656238 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f440d2a90 con 0x7f8f4415e850 2024-04-20T21:17:59.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.507+0000 7f4c8ea00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4c9015e850 0x7f4c900cfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.507+0000 7f4c8e000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c9015ec30 0x7f4c900d0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.507+0000 7f4c8ea00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4c9015e850 0x7f4c900cfcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.86:58096/0 (socket says 172.21.15.86:58096) 2024-04-20T21:17:59.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.507+0000 7f4c8e000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c9015ec30 0x7f4c900d0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:55574/0 (socket says 172.21.15.86:55574) 2024-04-20T21:17:59.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.507+0000 7fb821c00640 1 -- 172.21.15.86:0/63755037 wait complete. 2024-04-20T21:17:59.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.507+0000 7f771d000640 1 --2- 172.21.15.86:0/1772266580 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7718072fd0 0x7f77180733b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7708009870 tx=0x7f770802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40c911761e0fb729 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.508+0000 7f4c8ea00640 1 -- 172.21.15.86:0/2098950044 learned_addr learned my addr 172.21.15.86:0/2098950044 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:59.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.508+0000 7f4c95a00640 1 -- 172.21.15.86:0/2098950044 --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c900d24d0 con 0x7f4c900b61e0 2024-04-20T21:17:59.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.508+0000 7f3082800640 1 -- 172.21.15.86:0/2389144901 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f308406fc60 msgr2=0x7f30841f8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.508+0000 7f3082800640 1 --2- 172.21.15.86:0/2389144901 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f308406fc60 0x7f30841f8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.508+0000 7f3082800640 1 -- 172.21.15.86:0/2389144901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f308406f480 msgr2=0x7f30840d06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.508+0000 7f4c95a00640 1 -- 172.21.15.86:0/2098950044 --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f4c900d22a0 con 0x7f4c9015e850 2024-04-20T21:17:59.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.508+0000 7f3082800640 1 --2- 172.21.15.86:0/2389144901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f308406f480 0x7f30840d06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.508+0000 7f3082800640 1 -- 172.21.15.86:0/2389144901 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3084159c30 con 0x7f308415e320 2024-04-20T21:17:59.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.508+0000 7f4c95a00640 1 -- 172.21.15.86:0/2098950044 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f4c900d27b0 con 0x7f4c9015ec30 2024-04-20T21:17:59.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.508+0000 7f47d5a00640 1 Processor -- start 2024-04-20T21:17:59.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.508+0000 7f3082800640 1 --2- 172.21.15.86:0/2389144901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f308415e320 0x7f30841596f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f307c00efc0 tx=0x7f307c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.508+0000 7f47d5a00640 1 -- start start 2024-04-20T21:17:59.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.508+0000 7f70fb000640 1 Processor -- start 2024-04-20T21:17:59.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.509+0000 7f7717e00640 1 -- 172.21.15.86:0/1772266580 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7708034070 con 0x7f7718072fd0 2024-04-20T21:17:59.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.509+0000 7f7717e00640 1 -- 172.21.15.86:0/1772266580 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7708030840 con 0x7f7718072fd0 2024-04-20T21:17:59.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.509+0000 7f7717e00640 1 -- 172.21.15.86:0/1772266580 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7708030c10 con 0x7f7718072fd0 2024-04-20T21:17:59.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.509+0000 7f4c8e000640 1 -- 172.21.15.86:0/2098950044 >> v2:172.21.15.144:3300/0 conn(0x7f4c9015e850 msgr2=0x7f4c900cfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:59.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.509+0000 7f4c8e000640 1 --2- 172.21.15.86:0/2098950044 >> v2:172.21.15.144:3300/0 conn(0x7f4c9015e850 0x7f4c900cfcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.509+0000 7f4c8e000640 1 -- 172.21.15.86:0/2098950044 >> v1:172.21.15.144:6789/0 conn(0x7f4c900b61e0 legacy=0x7f4c900b65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:59.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.509+0000 7f4c8e000640 1 -- 172.21.15.86:0/2098950044 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c900d2a90 con 0x7f4c9015ec30 2024-04-20T21:17:59.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.509+0000 7f70fb000640 1 -- start start 2024-04-20T21:17:59.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.509+0000 7f771fa00640 1 -- 172.21.15.86:0/1772266580 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7718072fd0 msgr2=0x7f77180733b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.510+0000 7f771fa00640 1 --2- 172.21.15.86:0/1772266580 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7718072fd0 0x7f77180733b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7708009870 tx=0x7f770802b980 comp rx=0 tx=0).stop 2024-04-20T21:17:59.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.510+0000 7f771fa00640 1 -- 172.21.15.86:0/1772266580 shutdown_connections 2024-04-20T21:17:59.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.510+0000 7f771fa00640 1 --2- 172.21.15.86:0/1772266580 >> v2:172.21.15.86:3300/0 conn(0x7f77180d6030 0x7f77181668d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.510+0000 7f771fa00640 1 --2- 172.21.15.86:0/1772266580 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7718072fd0 0x7f77180733b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.510+0000 7f771fa00640 1 -- 172.21.15.86:0/1772266580 >> 172.21.15.86:0/1772266580 conn(0x7f77180d5540 msgr2=0x7f77180d5920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:59.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.510+0000 7fbd1cc00640 1 -- 172.21.15.86:0/3009965254 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbd200c7020 con 0x7fbd18056ab0 2024-04-20T21:17:59.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.510+0000 7f47d5a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47b40039e0 0x7f47d0158bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.510+0000 7f47d5a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47d01ed260 0x7f47d015d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.510+0000 7f47d5a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47d00cf350 0x7f47d01594a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.510+0000 7f47d5a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f47d00b3f10 con 0x7f47d01ed260 2024-04-20T21:17:59.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.510+0000 7fb821c00640 1 Processor -- start 2024-04-20T21:17:59.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.510+0000 7f47d5a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f47d0072e90 con 0x7f47d00cf350 2024-04-20T21:17:59.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.510+0000 7f47d5a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f47d006f080 con 0x7f47b40039e0 2024-04-20T21:17:59.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.510+0000 7f771fa00640 1 -- 172.21.15.86:0/1772266580 shutdown_connections 2024-04-20T21:17:59.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.510+0000 7f771fa00640 1 -- 172.21.15.86:0/1772266580 wait complete. 2024-04-20T21:17:59.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.510+0000 7fb821c00640 1 -- start start 2024-04-20T21:17:59.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.511+0000 7f47ce000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47d01ed260 0x7f47d015d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.511+0000 7f47ce000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47d01ed260 0x7f47d015d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57884/0 (socket says 172.21.15.86:57884) 2024-04-20T21:17:59.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.511+0000 7f47ce000640 1 -- 172.21.15.86:0/4161499405 learned_addr learned my addr 172.21.15.86:0/4161499405 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:59.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.511+0000 7f70fb000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f70f415e850 0x7f70f40cfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.511+0000 7f70fb000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f70f415ec30 0x7f70f40d0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.511+0000 7f70fb000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70f40b61e0 0x7f70f40b6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.511+0000 7f47cf400640 1 --2- 172.21.15.86:0/4161499405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47d00cf350 0x7f47d01594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.511+0000 7f70fb000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f70f40d22a0 con 0x7f70f415ec30 2024-04-20T21:17:59.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.511+0000 7f70fb000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f70f40b6b50 con 0x7f70f415e850 2024-04-20T21:17:59.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.511+0000 7f70fb000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f70f40d24d0 con 0x7f70f40b61e0 2024-04-20T21:17:59.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.511+0000 7f307a000640 1 -- 172.21.15.86:0/2389144901 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307c018020 con 0x7f308415e320 2024-04-20T21:17:59.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.511+0000 7f307a000640 1 -- 172.21.15.86:0/2389144901 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f307c014480 con 0x7f308415e320 2024-04-20T21:17:59.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.511+0000 7f307a000640 1 -- 172.21.15.86:0/2389144901 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307c014870 con 0x7f308415e320 2024-04-20T21:17:59.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.511+0000 7f47ce000640 1 -- 172.21.15.86:0/4161499405 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47b40039e0 msgr2=0x7f47d0158bd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-20T21:17:59.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.511+0000 7f47ce000640 1 --2- 172.21.15.86:0/4161499405 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47b40039e0 0x7f47d0158bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.511+0000 7f47ce000640 1 -- 172.21.15.86:0/4161499405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47d00cf350 msgr2=0x7f47d01594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.511+0000 7f47ce000640 1 --2- 172.21.15.86:0/4161499405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47d00cf350 0x7f47d01594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.511+0000 7f47ce000640 1 -- 172.21.15.86:0/4161499405 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47d015cac0 con 0x7f47d01ed260 2024-04-20T21:17:59.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.512+0000 7f47ce000640 1 --2- 172.21.15.86:0/4161499405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47d01ed260 0x7f47d015d3d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f47c000e4d0 tx=0x7f47c000e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.513+0000 7f47c6a00640 1 -- 172.21.15.86:0/4161499405 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c00053a0 con 0x7f47d01ed260 2024-04-20T21:17:59.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.513+0000 7f47c6a00640 1 -- 172.21.15.86:0/4161499405 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47c0007030 con 0x7f47d01ed260 2024-04-20T21:17:59.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.513+0000 7f47c6a00640 1 -- 172.21.15.86:0/4161499405 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c0007420 con 0x7f47d01ed260 2024-04-20T21:17:59.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.512+0000 7f4c8e000640 1 --2- 172.21.15.86:0/2098950044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c9015ec30 0x7f4c900d0230 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4c80009d90 tx=0x7f4c8002be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdeaa66578fde06e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.513+0000 7f47d5a00640 1 -- 172.21.15.86:0/4161499405 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47d015cd60 con 0x7f47d01ed260 2024-04-20T21:17:59.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.513+0000 7f47d5a00640 1 -- 172.21.15.86:0/4161499405 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47d00ceb50 con 0x7f47d01ed260 2024-04-20T21:17:59.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.512+0000 7f3083200640 1 -- 172.21.15.86:0/2389144901 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3084158950 con 0x7f308415e320 2024-04-20T21:17:59.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.513+0000 7f3083200640 1 -- 172.21.15.86:0/2389144901 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3084158c10 con 0x7f308415e320 2024-04-20T21:17:59.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.513+0000 7fb821c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb81c0cf810 0x7fb81c06fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.513+0000 7fb821c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb81c0d0fd0 0x7fb81c15a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.513+0000 7fb821c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb81c15ed80 0x7fb81c15f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.514+0000 7fb81aa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb81c0cf810 0x7fb81c06fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.514+0000 7fb81aa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb81c0cf810 0x7fb81c06fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:55602/0 (socket says 172.21.15.86:55602) 2024-04-20T21:17:59.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.514+0000 7fb81aa00640 1 -- 172.21.15.86:0/3689136490 learned_addr learned my addr 172.21.15.86:0/3689136490 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:59.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.514+0000 7fb821c00640 1 -- 172.21.15.86:0/3689136490 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb81c0d36c0 con 0x7fb81c0d0fd0 2024-04-20T21:17:59.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.514+0000 7fb821c00640 1 -- 172.21.15.86:0/3689136490 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb81c0d33e0 con 0x7fb81c15ed80 2024-04-20T21:17:59.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.513+0000 7f771fa00640 1 Processor -- start 2024-04-20T21:17:59.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.513+0000 7f771fa00640 1 -- start start 2024-04-20T21:17:59.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.514+0000 7f771fa00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7718072fd0 0x7f7718162300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.514+0000 7f771fa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77180d8640 0x7f77180a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.514+0000 7f771fa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f77180a56d0 0x7f77180a2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.514+0000 7f771fa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7718163230 con 0x7f77180d8640 2024-04-20T21:17:59.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.514+0000 7f771fa00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7718163be0 con 0x7f7718072fd0 2024-04-20T21:17:59.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.514+0000 7f771fa00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f77181633b0 con 0x7f77180a56d0 2024-04-20T21:17:59.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.515+0000 7fb821c00640 1 -- 172.21.15.86:0/3689136490 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb81c15f160 con 0x7fb81c0cf810 2024-04-20T21:17:59.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.511+0000 7f70f3400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f70f415ec30 0x7f70f40d0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.511+0000 7f70f3400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f70f415ec30 0x7f70f40d0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57888/0 (socket says 172.21.15.86:57888) 2024-04-20T21:17:59.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.511+0000 7f70f3400640 1 -- 172.21.15.86:0/1585497897 learned_addr learned my addr 172.21.15.86:0/1585497897 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:59.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.512+0000 7f70f9000640 1 --2- 172.21.15.86:0/1585497897 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70f40b61e0 0x7f70f40b6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.512+0000 7f70f3e00640 1 --2- 172.21.15.86:0/1585497897 >> v2:172.21.15.144:3300/0 conn(0x7f70f415e850 0x7f70f40cfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.512+0000 7f70f3400640 1 -- 172.21.15.86:0/1585497897 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70f40b61e0 msgr2=0x7f70f40b6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:59.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.512+0000 7f70f3400640 1 --2- 172.21.15.86:0/1585497897 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70f40b61e0 0x7f70f40b6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.512+0000 7f70f3400640 1 -- 172.21.15.86:0/1585497897 >> v2:172.21.15.144:3300/0 conn(0x7f70f415e850 msgr2=0x7f70f40cfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:17:59.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.512+0000 7f70f3400640 1 --2- 172.21.15.86:0/1585497897 >> v2:172.21.15.144:3300/0 conn(0x7f70f415e850 0x7f70f40cfcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.512+0000 7f70f3400640 1 -- 172.21.15.86:0/1585497897 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70f40d27b0 con 0x7f70f415ec30 2024-04-20T21:17:59.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.514+0000 7f70f3400640 1 --2- 172.21.15.86:0/1585497897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70f415ec30 0x7f70f40d0230 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f70f408e860 tx=0x7f70e0034120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98616f4a681fb2e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.515+0000 7f770fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77180d8640 0x7f77180a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.515+0000 7fb81a000640 1 --2- 172.21.15.86:0/3689136490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb81c0d0fd0 0x7fb81c15a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.515+0000 7f47bbe00640 1 -- 172.21.15.86:0/4161499405 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47a4005290 con 0x7f47d01ed260 2024-04-20T21:17:59.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.515+0000 7f770fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77180d8640 0x7f77180a5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57904/0 (socket says 172.21.15.86:57904) 2024-04-20T21:17:59.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.515+0000 7f770fe00640 1 -- 172.21.15.86:0/1471410674 learned_addr learned my addr 172.21.15.86:0/1471410674 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:59.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.515+0000 7f771d000640 1 --2- 172.21.15.86:0/1471410674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7718072fd0 0x7f7718162300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.515+0000 7f771da00640 1 --2- 172.21.15.86:0/1471410674 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f77180a56d0 0x7f77180a2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.515+0000 7f70f2a00640 1 -- 172.21.15.86:0/1585497897 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e0041070 con 0x7f70f415ec30 2024-04-20T21:17:59.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.515+0000 7f8f49600640 1 --2- 172.21.15.86:0/1925656238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f4415e850 0x7f8f440cfcf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8f38009790 tx=0x7f8f3802b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b135e1f94c38c196 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.516+0000 7fb81b400640 1 --2- 172.21.15.86:0/3689136490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb81c15ed80 0x7fb81c15f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.516+0000 7f8f43e00640 1 -- 172.21.15.86:0/1925656238 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f38034070 con 0x7f8f4415e850 2024-04-20T21:17:59.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.516+0000 7f8f43e00640 1 -- 172.21.15.86:0/1925656238 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f38030800 con 0x7f8f4415e850 2024-04-20T21:17:59.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.516+0000 7f8f43e00640 1 -- 172.21.15.86:0/1925656238 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f38030bd0 con 0x7f8f4415e850 2024-04-20T21:17:59.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.516+0000 7f770fe00640 1 -- 172.21.15.86:0/1471410674 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f77180a56d0 msgr2=0x7f77180a2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.516+0000 7f770fe00640 1 --2- 172.21.15.86:0/1471410674 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f77180a56d0 0x7f77180a2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.516+0000 7f770fe00640 1 -- 172.21.15.86:0/1471410674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7718072fd0 msgr2=0x7f7718162300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.516+0000 7f770fe00640 1 --2- 172.21.15.86:0/1471410674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7718072fd0 0x7f7718162300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.516+0000 7f770fe00640 1 -- 172.21.15.86:0/1471410674 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77180a38f0 con 0x7f77180d8640 2024-04-20T21:17:59.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.516+0000 7f771da00640 1 --2- 172.21.15.86:0/1471410674 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f77180a56d0 0x7f77180a2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:59.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.516+0000 7f70f2a00640 1 -- 172.21.15.86:0/1585497897 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70e003c440 con 0x7f70f415ec30 2024-04-20T21:17:59.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.516+0000 7f70f2a00640 1 -- 172.21.15.86:0/1585497897 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e003c810 con 0x7f70f415ec30 2024-04-20T21:17:59.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.516+0000 7f70fb000640 1 -- 172.21.15.86:0/1585497897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70f415ec30 msgr2=0x7f70f40d0230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.516+0000 7f70fb000640 1 --2- 172.21.15.86:0/1585497897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70f415ec30 0x7f70f40d0230 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f70f408e860 tx=0x7f70e0034120 comp rx=0 tx=0).stop 2024-04-20T21:17:59.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.516+0000 7f770fe00640 1 --2- 172.21.15.86:0/1471410674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77180d8640 0x7f77180a5190 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f770000c7f0 tx=0x7f770000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.516+0000 7f4c8d600640 1 -- 172.21.15.86:0/2098950044 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c80039070 con 0x7f4c9015ec30 2024-04-20T21:17:59.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.516+0000 7f4c8d600640 1 -- 172.21.15.86:0/2098950044 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c8002cbd0 con 0x7f4c9015ec30 2024-04-20T21:17:59.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.516+0000 7f4c8d600640 1 -- 172.21.15.86:0/2098950044 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c80034440 con 0x7f4c9015ec30 2024-04-20T21:17:59.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.517+0000 7f8f4c000640 1 -- 172.21.15.86:0/1925656238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f4415e850 msgr2=0x7f8f440cfcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.517+0000 7f8f4c000640 1 --2- 172.21.15.86:0/1925656238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f4415e850 0x7f8f440cfcf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8f38009790 tx=0x7f8f3802b940 comp rx=0 tx=0).stop 2024-04-20T21:17:59.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.517+0000 7f8f4c000640 1 -- 172.21.15.86:0/1925656238 shutdown_connections 2024-04-20T21:17:59.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.517+0000 7f8f4c000640 1 --2- 172.21.15.86:0/1925656238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f4415e850 0x7f8f440cfcf0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.517+0000 7f8f4c000640 1 -- 172.21.15.86:0/1925656238 >> 172.21.15.86:0/1925656238 conn(0x7f8f4408bf70 msgr2=0x7f8f440d4760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:59.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.517+0000 7f8f4c000640 1 -- 172.21.15.86:0/1925656238 shutdown_connections 2024-04-20T21:17:59.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.517+0000 7f8f4c000640 1 -- 172.21.15.86:0/1925656238 wait complete. 2024-04-20T21:17:59.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.518+0000 7f70fb000640 1 -- 172.21.15.86:0/1585497897 shutdown_connections 2024-04-20T21:17:59.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.518+0000 7f70fb000640 1 --2- 172.21.15.86:0/1585497897 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70f40b61e0 0x7f70f40b6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.518+0000 7f70fb000640 1 --2- 172.21.15.86:0/1585497897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70f415ec30 0x7f70f40d0230 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.518+0000 7f307a000640 1 -- 172.21.15.86:0/2389144901 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f307c01b020 con 0x7f308415e320 2024-04-20T21:17:59.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.516+0000 7fb81aa00640 1 -- 172.21.15.86:0/3689136490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb81c15ed80 msgr2=0x7fb81c15f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.516+0000 7fb81aa00640 1 --2- 172.21.15.86:0/3689136490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb81c15ed80 0x7fb81c15f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.516+0000 7fb81aa00640 1 -- 172.21.15.86:0/3689136490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb81c0d0fd0 msgr2=0x7fb81c15a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.516+0000 7fb81aa00640 1 --2- 172.21.15.86:0/3689136490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb81c0d0fd0 0x7fb81c15a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.518+0000 7f70fb000640 1 --2- 172.21.15.86:0/1585497897 >> v2:172.21.15.144:3300/0 conn(0x7f70f415e850 0x7f70f40cfcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.516+0000 7fb81aa00640 1 -- 172.21.15.86:0/3689136490 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb81c0caf80 con 0x7fb81c0cf810 2024-04-20T21:17:59.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.518+0000 7f70fb000640 1 -- 172.21.15.86:0/1585497897 >> 172.21.15.86:0/1585497897 conn(0x7f70f408bf70 msgr2=0x7f70f40d4760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:59.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.517+0000 7f47c6a00640 1 -- 172.21.15.86:0/4161499405 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f47c00067d0 con 0x7f47d01ed260 2024-04-20T21:17:59.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.518+0000 7f4c95a00640 1 -- 172.21.15.86:0/2098950044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c9015ec30 msgr2=0x7f4c900d0230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.518+0000 7f4c95a00640 1 --2- 172.21.15.86:0/2098950044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c9015ec30 0x7f4c900d0230 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4c80009d90 tx=0x7f4c8002be20 comp rx=0 tx=0).stop 2024-04-20T21:17:59.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.518+0000 7f70fb000640 1 -- 172.21.15.86:0/1585497897 shutdown_connections 2024-04-20T21:17:59.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.518+0000 7f70fb000640 1 -- 172.21.15.86:0/1585497897 wait complete. 2024-04-20T21:17:59.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.518+0000 7f4c95a00640 1 -- 172.21.15.86:0/2098950044 shutdown_connections 2024-04-20T21:17:59.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.518+0000 7fb81aa00640 1 --2- 172.21.15.86:0/3689136490 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb81c0cf810 0x7fb81c06fce0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb80c00e840 tx=0x7fb80c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.518+0000 7f4c95a00640 1 --2- 172.21.15.86:0/2098950044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c9015ec30 0x7f4c900d0230 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.518+0000 7f4c95a00640 1 --2- 172.21.15.86:0/2098950044 >> v2:172.21.15.144:3300/0 conn(0x7f4c9015e850 0x7f4c900cfcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.518+0000 7f4c95a00640 1 -- 172.21.15.86:0/2098950044 >> 172.21.15.86:0/2098950044 conn(0x7f4c9008bf70 msgr2=0x7f4c900d4760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:59.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.519+0000 7f3083200640 1 -- 172.21.15.86:0/2389144901 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f308404fcf0 con 0x7f308415e320 2024-04-20T21:17:59.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.519+0000 7fb812a00640 1 -- 172.21.15.86:0/3689136490 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80c017070 con 0x7fb81c0cf810 2024-04-20T21:17:59.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.519+0000 7f4c95a00640 1 -- 172.21.15.86:0/2098950044 shutdown_connections 2024-04-20T21:17:59.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.519+0000 7fb812a00640 1 -- 172.21.15.86:0/3689136490 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb80c00c8e0 con 0x7fb81c0cf810 2024-04-20T21:17:59.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.519+0000 7fb812a00640 1 -- 172.21.15.86:0/3689136490 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80c00ccd0 con 0x7fb81c0cf810 2024-04-20T21:17:59.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.519+0000 7fb821c00640 1 -- 172.21.15.86:0/3689136490 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb81c070350 con 0x7fb81c0cf810 2024-04-20T21:17:59.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.519+0000 7fb821c00640 1 -- 172.21.15.86:0/3689136490 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb81c070910 con 0x7fb81c0cf810 2024-04-20T21:17:59.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.519+0000 7f47c6a00640 1 --2- 172.21.15.86:0/4161499405 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f47b006c940 0x7f47b006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.520+0000 7f47c6a00640 1 -- 172.21.15.86:0/4161499405 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f47c008c090 con 0x7f47d01ed260 2024-04-20T21:17:59.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.520+0000 7f47cea00640 1 --2- 172.21.15.86:0/4161499405 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f47b006c940 0x7f47b006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.520+0000 7f307a000640 1 --2- 172.21.15.86:0/2389144901 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f306406c940 0x7f306406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.520+0000 7fb821c00640 1 -- 172.21.15.86:0/3689136490 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb81c0690f0 con 0x7fb81c0cf810 2024-04-20T21:17:59.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.520+0000 7f4c95a00640 1 -- 172.21.15.86:0/2098950044 wait complete. 2024-04-20T21:17:59.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.521+0000 7f3081e00640 1 --2- 172.21.15.86:0/2389144901 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f306406c940 0x7f306406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.521+0000 7f70fb000640 1 Processor -- start 2024-04-20T21:17:59.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.521+0000 7f70fb000640 1 -- start start 2024-04-20T21:17:59.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.521+0000 7f8f4c000640 1 Processor -- start 2024-04-20T21:17:59.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.522+0000 7f47cea00640 1 --2- 172.21.15.86:0/4161499405 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f47b006c940 0x7f47b006ee00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f47bc034000 tx=0x7f47bc02f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.523+0000 7f4c95a00640 1 Processor -- start 2024-04-20T21:17:59.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.523+0000 7f4c95a00640 1 -- start start 2024-04-20T21:17:59.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.523+0000 7f3081e00640 1 --2- 172.21.15.86:0/2389144901 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f306406c940 0x7f306406ee00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3070037000 tx=0x7f307000c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.522+0000 7f8f4c000640 1 -- start start 2024-04-20T21:17:59.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.523+0000 7fb812a00640 1 -- 172.21.15.86:0/3689136490 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb80c01a070 con 0x7fb81c0cf810 2024-04-20T21:17:59.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.523+0000 7f8f4c000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f440b68f0 0x7f8f4415ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.523+0000 7f8f4c000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8f4415e850 0x7f8f440cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.523+0000 7f8f4c000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f4415ec30 0x7f8f440cc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.523+0000 7f8f4c000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8f440d3570 con 0x7f8f4415ec30 2024-04-20T21:17:59.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.523+0000 7f8f4c000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8f440d3050 con 0x7f8f440b68f0 2024-04-20T21:17:59.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.523+0000 7f8f4c000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8f440d36f0 con 0x7f8f4415e850 2024-04-20T21:17:59.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.524+0000 7f70fb000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70f40b61e0 0x7f70f4156710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.524+0000 7f70fb000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70f415e850 0x7f70f40b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.524+0000 7f8f48c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8f4415e850 0x7f8f440cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.524+0000 7f8f48c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8f4415e850 0x7f8f440cbc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:55610/0 (socket says 172.21.15.86:55610) 2024-04-20T21:17:59.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.524+0000 7f8f48c00640 1 -- 172.21.15.86:0/1898482109 learned_addr learned my addr 172.21.15.86:0/1898482109 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:59.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.524+0000 7f8f4a000640 1 --2- 172.21.15.86:0/1898482109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f4415ec30 0x7f8f440cc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.524+0000 7f8f49600640 1 --2- 172.21.15.86:0/1898482109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f440b68f0 0x7f8f4415ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.524+0000 7f4c95a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c900b61e0 0x7f4c900cccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.524+0000 7fb812a00640 1 --2- 172.21.15.86:0/3689136490 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb7fc06c920 0x7fb7fc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.524+0000 7f4c95a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c9015e850 0x7f4c900b9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.524+0000 7f4c95a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c9015ec30 0x7f4c900b9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.524+0000 7f4c95a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4c900d3570 con 0x7f4c900b61e0 2024-04-20T21:17:59.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.524+0000 7f4c95a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4c900d3050 con 0x7f4c9015e850 2024-04-20T21:17:59.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.524+0000 7f4c95a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f4c900d36f0 con 0x7f4c9015ec30 2024-04-20T21:17:59.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.524+0000 7f70fb000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70f415ec30 0x7f70f4155ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.525+0000 7f307a000640 1 -- 172.21.15.86:0/2389144901 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f307c057930 con 0x7f308415e320 2024-04-20T21:17:59.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.525+0000 7f70f3e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70f40b61e0 0x7f70f4156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.525+0000 7f70f3e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70f40b61e0 0x7f70f4156710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:55618/0 (socket says 172.21.15.86:55618) 2024-04-20T21:17:59.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.525+0000 7f70f3e00640 1 -- 172.21.15.86:0/1554258679 learned_addr learned my addr 172.21.15.86:0/1554258679 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:59.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.525+0000 7f70fb000640 1 -- 172.21.15.86:0/1554258679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f70f40d3050 con 0x7f70f415e850 2024-04-20T21:17:59.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.525+0000 7f4c8f400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c9015ec30 0x7f4c900b9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.525+0000 7f4c8f400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c9015ec30 0x7f4c900b9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:55628/0 (socket says 172.21.15.86:55628) 2024-04-20T21:17:59.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.525+0000 7f4c8f400640 1 -- 172.21.15.86:0/3676161191 learned_addr learned my addr 172.21.15.86:0/3676161191 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:17:59.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.525+0000 7f70fb000640 1 -- 172.21.15.86:0/1554258679 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f70f40d2d70 con 0x7f70f415ec30 2024-04-20T21:17:59.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.525+0000 7f70fb000640 1 -- 172.21.15.86:0/1554258679 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f70f40d3570 con 0x7f70f40b61e0 2024-04-20T21:17:59.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.525+0000 7f70f9000640 1 --2- 172.21.15.86:0/1554258679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70f415ec30 0x7f70f4155ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.525+0000 7f7715600640 1 -- 172.21.15.86:0/1471410674 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7700015070 con 0x7f77180d8640 2024-04-20T21:17:59.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.525+0000 7f7715600640 1 -- 172.21.15.86:0/1471410674 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7700007a30 con 0x7f77180d8640 2024-04-20T21:17:59.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.525+0000 7f7715600640 1 -- 172.21.15.86:0/1471410674 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7700007e00 con 0x7f77180d8640 2024-04-20T21:17:59.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.526+0000 7fb81a000640 1 --2- 172.21.15.86:0/3689136490 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb7fc06c920 0x7fb7fc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.526+0000 7f70f3400640 1 --2- 172.21.15.86:0/1554258679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70f415e850 0x7f70f40b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.526+0000 7fb812a00640 1 -- 172.21.15.86:0/3689136490 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb80c08c0a0 con 0x7fb81c0cf810 2024-04-20T21:17:59.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.527+0000 7f47c6a00640 1 -- 172.21.15.86:0/4161499405 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f47c005a5a0 con 0x7f47d01ed260 2024-04-20T21:17:59.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.525+0000 7f8f4a000640 1 -- 172.21.15.86:0/1898482109 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8f4415e850 msgr2=0x7f8f440cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.527+0000 7f70f3e00640 1 -- 172.21.15.86:0/1554258679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70f415ec30 msgr2=0x7f70f4155ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.527+0000 7f70f3e00640 1 --2- 172.21.15.86:0/1554258679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70f415ec30 0x7f70f4155ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.527+0000 7fb81a000640 1 --2- 172.21.15.86:0/3689136490 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb7fc06c920 0x7fb7fc06ede0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb808034000 tx=0x7fb808008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.527+0000 7f70f3e00640 1 -- 172.21.15.86:0/1554258679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70f415e850 msgr2=0x7f70f40b7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.527+0000 7f70f3e00640 1 --2- 172.21.15.86:0/1554258679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70f415e850 0x7f70f40b7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.527+0000 7f70f3e00640 1 -- 172.21.15.86:0/1554258679 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70f40ccdf0 con 0x7f70f40b61e0 2024-04-20T21:17:59.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.527+0000 7f8f4a000640 1 --2- 172.21.15.86:0/1898482109 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8f4415e850 0x7f8f440cbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.527+0000 7f8f4a000640 1 -- 172.21.15.86:0/1898482109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f440b68f0 msgr2=0x7f8f4415ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.527+0000 7f8f4a000640 1 --2- 172.21.15.86:0/1898482109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f440b68f0 0x7f8f4415ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.527+0000 7f8f4a000640 1 -- 172.21.15.86:0/1898482109 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f440ba040 con 0x7f8f4415ec30 2024-04-20T21:17:59.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.527+0000 7f8f49600640 1 --2- 172.21.15.86:0/1898482109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f440b68f0 0x7f8f4415ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:59.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.525+0000 7f4c8e000640 1 --2- 172.21.15.86:0/3676161191 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c9015e850 0x7f4c900b9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.525+0000 7f4c8ea00640 1 --2- 172.21.15.86:0/3676161191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c900b61e0 0x7f4c900cccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.526+0000 7f4c8ea00640 1 -- 172.21.15.86:0/3676161191 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c9015ec30 msgr2=0x7f4c900b9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.526+0000 7f4c8ea00640 1 --2- 172.21.15.86:0/3676161191 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c9015ec30 0x7f4c900b9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.526+0000 7f4c8ea00640 1 -- 172.21.15.86:0/3676161191 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c9015e850 msgr2=0x7f4c900b9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.526+0000 7f4c8ea00640 1 --2- 172.21.15.86:0/3676161191 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c9015e850 0x7f4c900b9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.527+0000 7f70f3400640 1 --2- 172.21.15.86:0/1554258679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70f415e850 0x7f70f40b7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:59.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.526+0000 7f4c8ea00640 1 -- 172.21.15.86:0/3676161191 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c900ba2f0 con 0x7f4c900b61e0 2024-04-20T21:17:59.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.527+0000 7f4c8ea00640 1 --2- 172.21.15.86:0/3676161191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c900b61e0 0x7f4c900cccd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c00e840 tx=0x7f4c7c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.527+0000 7f70f9000640 1 --2- 172.21.15.86:0/1554258679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70f415ec30 0x7f70f4155ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:17:59.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.527+0000 7f8f4a000640 1 --2- 172.21.15.86:0/1898482109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f4415ec30 0x7f8f440cc1b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c00c690 tx=0x7f8f3c00cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.527+0000 7f771fa00640 1 -- 172.21.15.86:0/1471410674 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77180a3bd0 con 0x7f77180d8640 2024-04-20T21:17:59.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.527+0000 7f771fa00640 1 -- 172.21.15.86:0/1471410674 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77180a4190 con 0x7f77180d8640 2024-04-20T21:17:59.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.528+0000 7f70f3e00640 1 --2- 172.21.15.86:0/1554258679 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70f40b61e0 0x7f70f4156710 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f70e400c820 tx=0x7f70e400cce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.528+0000 7f771fa00640 1 -- 172.21.15.86:0/1471410674 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77180a5ae0 con 0x7f77180d8640 2024-04-20T21:17:59.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.534+0000 7f307a000640 1 -- 172.21.15.86:0/2389144901 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f307c010040 con 0x7f308415e320 2024-04-20T21:17:59.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.534+0000 7f4c86a00640 1 -- 172.21.15.86:0/3676161191 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c005990 con 0x7f4c900b61e0 2024-04-20T21:17:59.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.534+0000 7f4c95a00640 1 -- 172.21.15.86:0/3676161191 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c900b8540 con 0x7f4c900b61e0 2024-04-20T21:17:59.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.534+0000 7f4c95a00640 1 -- 172.21.15.86:0/3676161191 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c900b9590 con 0x7f4c900b61e0 2024-04-20T21:17:59.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.536+0000 7f4c86a00640 1 -- 172.21.15.86:0/3676161191 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c7c005b30 con 0x7f4c900b61e0 2024-04-20T21:17:59.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.536+0000 7f4c95a00640 1 -- 172.21.15.86:0/3676161191 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c900689e0 con 0x7f4c900b61e0 2024-04-20T21:17:59.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.536+0000 7f4c86a00640 1 -- 172.21.15.86:0/3676161191 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c020060 con 0x7f4c900b61e0 2024-04-20T21:17:59.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.537+0000 7f8f41600640 1 -- 172.21.15.86:0/1898482109 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c003b70 con 0x7f8f4415ec30 2024-04-20T21:17:59.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.537+0000 7f8f41600640 1 -- 172.21.15.86:0/1898482109 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f3c003d10 con 0x7f8f4415ec30 2024-04-20T21:17:59.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.537+0000 7f8f41600640 1 -- 172.21.15.86:0/1898482109 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c005310 con 0x7f8f4415ec30 2024-04-20T21:17:59.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.537+0000 7f8f4c000640 1 -- 172.21.15.86:0/1898482109 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f440ba2e0 con 0x7f8f4415ec30 2024-04-20T21:17:59.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.537+0000 7f8f4c000640 1 -- 172.21.15.86:0/1898482109 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f440a23a0 con 0x7f8f4415ec30 2024-04-20T21:17:59.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.538+0000 7f7715600640 1 -- 172.21.15.86:0/1471410674 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7700018070 con 0x7f77180d8640 2024-04-20T21:17:59.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.539+0000 7f8f4c000640 1 -- 172.21.15.86:0/1898482109 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f440689e0 con 0x7f8f4415ec30 2024-04-20T21:17:59.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.540+0000 7f7715600640 1 --2- 172.21.15.86:0/1471410674 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f76f406c940 0x7f76f406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.540+0000 7f7715600640 1 -- 172.21.15.86:0/1471410674 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f770008bec0 con 0x7f77180d8640 2024-04-20T21:17:59.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.541+0000 7f771d000640 1 --2- 172.21.15.86:0/1471410674 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f76f406c940 0x7f76f406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.541+0000 7f70ebe00640 1 -- 172.21.15.86:0/1554258679 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e4015070 con 0x7f70f40b61e0 2024-04-20T21:17:59.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.541+0000 7fb812a00640 1 -- 172.21.15.86:0/3689136490 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb80c059ef0 con 0x7fb81c0cf810 2024-04-20T21:17:59.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.539+0000 7f4c86a00640 1 -- 172.21.15.86:0/3676161191 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4c7c00c940 con 0x7f4c900b61e0 2024-04-20T21:17:59.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.541+0000 7f4c86a00640 1 --2- 172.21.15.86:0/3676161191 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4c7006c920 0x7f4c7006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.541+0000 7f70ebe00640 1 -- 172.21.15.86:0/1554258679 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70e4007a30 con 0x7f70f40b61e0 2024-04-20T21:17:59.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.542+0000 7f771d000640 1 --2- 172.21.15.86:0/1471410674 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f76f406c940 0x7f76f406ee00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7708036000 tx=0x7f770803b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.542+0000 7f70fb000640 1 -- 172.21.15.86:0/1554258679 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70f415d550 con 0x7f70f40b61e0 2024-04-20T21:17:59.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.542+0000 7f70fb000640 1 -- 172.21.15.86:0/1554258679 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f70f40cd090 con 0x7f70f40b61e0 2024-04-20T21:17:59.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.542+0000 7f70ebe00640 1 -- 172.21.15.86:0/1554258679 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e4007de0 con 0x7f70f40b61e0 2024-04-20T21:17:59.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.542+0000 7f4c8e000640 1 --2- 172.21.15.86:0/3676161191 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4c7006c920 0x7f4c7006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.544+0000 7f4c8e000640 1 --2- 172.21.15.86:0/3676161191 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4c7006c920 0x7f4c7006ede0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4c80036000 tx=0x7f4c80030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.544+0000 7f70fb000640 1 -- 172.21.15.86:0/1554258679 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70f40689e0 con 0x7f70f40b61e0 2024-04-20T21:17:59.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.545+0000 7f4c86a00640 1 -- 172.21.15.86:0/3676161191 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4c7c057d70 con 0x7f4c900b61e0 2024-04-20T21:17:59.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.546+0000 7f7715600640 1 -- 172.21.15.86:0/1471410674 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f77180a5ae0 con 0x7f77180d8640 2024-04-20T21:17:59.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.546+0000 7f70ebe00640 1 -- 172.21.15.86:0/1554258679 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f70e4018070 con 0x7f70f40b61e0 2024-04-20T21:17:59.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.547+0000 7f70ebe00640 1 --2- 172.21.15.86:0/1554258679 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f70d406c940 0x7f70d406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.548+0000 7f70ebe00640 1 -- 172.21.15.86:0/1554258679 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f70e408b5f0 con 0x7f70f40b61e0 2024-04-20T21:17:59.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.548+0000 7f8f41600640 1 -- 172.21.15.86:0/1898482109 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8f3c0054b0 con 0x7f8f4415ec30 2024-04-20T21:17:59.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.550+0000 7f8f41600640 1 --2- 172.21.15.86:0/1898482109 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8f2406c940 0x7f8f2406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:17:59.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.550+0000 7f70f3400640 1 --2- 172.21.15.86:0/1554258679 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f70d406c940 0x7f70d406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.550+0000 7f8f49600640 1 --2- 172.21.15.86:0/1898482109 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8f2406c940 0x7f8f2406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:17:59.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.551+0000 7f8f49600640 1 --2- 172.21.15.86:0/1898482109 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8f2406c940 0x7f8f2406ee00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8f38036000 tx=0x7f8f38008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.552+0000 7f70f3400640 1 --2- 172.21.15.86:0/1554258679 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f70d406c940 0x7f70d406ee00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f70e003e000 tx=0x7f70e0038000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:17:59.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.555+0000 7f4c86a00640 1 -- 172.21.15.86:0/3676161191 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4c7c017020 con 0x7f4c900b61e0 2024-04-20T21:17:59.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.558+0000 7f8f41600640 1 -- 172.21.15.86:0/1898482109 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8f3c0057a0 con 0x7f8f4415ec30 2024-04-20T21:17:59.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.560+0000 7f70ebe00640 1 -- 172.21.15.86:0/1554258679 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f70e40594b0 con 0x7f70f40b61e0 2024-04-20T21:17:59.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.568+0000 7f8f41600640 1 -- 172.21.15.86:0/1898482109 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8f3c05a140 con 0x7f8f4415ec30 2024-04-20T21:17:59.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.924+0000 7fbd26400640 1 -- 172.21.15.86:0/3009965254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fbcfc005580 con 0x7fbd18056ab0 2024-04-20T21:17:59.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.925+0000 7fbd1cc00640 1 -- 172.21.15.86:0/3009965254 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbd200f7490 con 0x7fbd18056ab0 2024-04-20T21:17:59.925 INFO:teuthology.orchestra.run.smithi086.stdout:55834574854 2024-04-20T21:17:59.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.928+0000 7fbd26400640 1 -- 172.21.15.86:0/3009965254 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbd0406c8f0 msgr2=0x7fbd0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.928+0000 7fbd26400640 1 --2- 172.21.15.86:0/3009965254 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbd0406c8f0 0x7fbd0406edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbd180dbe00 tx=0x7fbd10008040 comp rx=0 tx=0).stop 2024-04-20T21:17:59.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.928+0000 7fbd26400640 1 -- 172.21.15.86:0/3009965254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd18056ab0 msgr2=0x7fbd180eb070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.928+0000 7fbd26400640 1 --2- 172.21.15.86:0/3009965254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd18056ab0 0x7fbd180eb070 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbd2006a870 tx=0x7fbd20069a20 comp rx=0 tx=0).stop 2024-04-20T21:17:59.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.929+0000 7fbd26400640 1 -- 172.21.15.86:0/3009965254 shutdown_connections 2024-04-20T21:17:59.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.929+0000 7fbd26400640 1 --2- 172.21.15.86:0/3009965254 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbd0406c8f0 0x7fbd0406edb0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.929+0000 7fbd26400640 1 --2- 172.21.15.86:0/3009965254 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd180e7710 0x7fbd1817e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.929+0000 7fbd26400640 1 --2- 172.21.15.86:0/3009965254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd180589c0 0x7fbd180dacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.929+0000 7fbd26400640 1 --2- 172.21.15.86:0/3009965254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd18056ab0 0x7fbd180eb070 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.929+0000 7fbd26400640 1 -- 172.21.15.86:0/3009965254 >> 172.21.15.86:0/3009965254 conn(0x7fbd18058ee0 msgr2=0x7fbd1804d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:59.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.930+0000 7fbd26400640 1 -- 172.21.15.86:0/3009965254 shutdown_connections 2024-04-20T21:17:59.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.930+0000 7fbd26400640 1 -- 172.21.15.86:0/3009965254 wait complete. 2024-04-20T21:17:59.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.955+0000 7f47bbe00640 1 -- 172.21.15.86:0/4161499405 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f47a4005850 con 0x7f47d01ed260 2024-04-20T21:17:59.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.955+0000 7f47c6a00640 1 -- 172.21.15.86:0/4161499405 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f47c0059ea0 con 0x7f47d01ed260 2024-04-20T21:17:59.958 INFO:teuthology.orchestra.run.smithi086.stdout:55834574853 2024-04-20T21:17:59.959 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.3 2024-04-20T21:17:59.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.958+0000 7f47bbe00640 1 -- 172.21.15.86:0/4161499405 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f47b006c940 msgr2=0x7f47b006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.958+0000 7f47bbe00640 1 --2- 172.21.15.86:0/4161499405 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f47b006c940 0x7f47b006ee00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f47bc034000 tx=0x7f47bc02f000 comp rx=0 tx=0).stop 2024-04-20T21:17:59.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.959+0000 7f47bbe00640 1 -- 172.21.15.86:0/4161499405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47d01ed260 msgr2=0x7f47d015d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.959+0000 7f47bbe00640 1 --2- 172.21.15.86:0/4161499405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47d01ed260 0x7f47d015d3d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f47c000e4d0 tx=0x7f47c000e990 comp rx=0 tx=0).stop 2024-04-20T21:17:59.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.959+0000 7f47bbe00640 1 -- 172.21.15.86:0/4161499405 shutdown_connections 2024-04-20T21:17:59.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.959+0000 7f47bbe00640 1 --2- 172.21.15.86:0/4161499405 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f47b006c940 0x7f47b006ee00 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.959+0000 7f47bbe00640 1 --2- 172.21.15.86:0/4161499405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47d00cf350 0x7f47d01594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.959+0000 7f47bbe00640 1 --2- 172.21.15.86:0/4161499405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47d01ed260 0x7f47d015d3d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.959+0000 7f47bbe00640 1 --2- 172.21.15.86:0/4161499405 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47b40039e0 0x7f47d0158bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.959+0000 7f47bbe00640 1 -- 172.21.15.86:0/4161499405 >> 172.21.15.86:0/4161499405 conn(0x7f47d00b2110 msgr2=0x7f47d0163170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:59.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.960+0000 7f47bbe00640 1 -- 172.21.15.86:0/4161499405 shutdown_connections 2024-04-20T21:17:59.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.961+0000 7f47bbe00640 1 -- 172.21.15.86:0/4161499405 wait complete. 2024-04-20T21:17:59.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.968+0000 7f3083200640 1 -- 172.21.15.86:0/2389144901 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f30840ce4e0 con 0x7f308415e320 2024-04-20T21:17:59.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.968+0000 7fb821c00640 1 -- 172.21.15.86:0/3689136490 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fb81c0d2820 con 0x7fb81c0cf810 2024-04-20T21:17:59.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.969+0000 7f307a000640 1 -- 172.21.15.86:0/2389144901 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f307c059890 con 0x7f308415e320 2024-04-20T21:17:59.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.969+0000 7fb812a00640 1 -- 172.21.15.86:0/3689136490 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb80c05e1f0 con 0x7fb81c0cf810 2024-04-20T21:17:59.970 INFO:teuthology.orchestra.run.smithi086.stdout:55834574853 2024-04-20T21:17:59.970 INFO:teuthology.orchestra.run.smithi086.stdout:55834574854 2024-04-20T21:17:59.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.972+0000 7f3083200640 1 -- 172.21.15.86:0/2389144901 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f306406c940 msgr2=0x7f306406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.972+0000 7f3083200640 1 --2- 172.21.15.86:0/2389144901 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f306406c940 0x7f306406ee00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3070037000 tx=0x7f307000c040 comp rx=0 tx=0).stop 2024-04-20T21:17:59.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.972+0000 7f3083200640 1 -- 172.21.15.86:0/2389144901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f308415e320 msgr2=0x7f30841596f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.972+0000 7f3083200640 1 --2- 172.21.15.86:0/2389144901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f308415e320 0x7f30841596f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f307c00efc0 tx=0x7f307c00c490 comp rx=0 tx=0).stop 2024-04-20T21:17:59.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.972+0000 7fb821c00640 1 -- 172.21.15.86:0/3689136490 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb7fc06c920 msgr2=0x7fb7fc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.973+0000 7f3083200640 1 -- 172.21.15.86:0/2389144901 shutdown_connections 2024-04-20T21:17:59.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.973+0000 7f3083200640 1 --2- 172.21.15.86:0/2389144901 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f306406c940 0x7f306406ee00 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.973+0000 7f3083200640 1 --2- 172.21.15.86:0/2389144901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f308415e320 0x7f30841596f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.973+0000 7f3083200640 1 --2- 172.21.15.86:0/2389144901 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f308406fc60 0x7f30841f8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.973+0000 7f3083200640 1 --2- 172.21.15.86:0/2389144901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f308406f480 0x7f30840d06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.973+0000 7f3083200640 1 -- 172.21.15.86:0/2389144901 >> 172.21.15.86:0/2389144901 conn(0x7f308406bb60 msgr2=0x7f30840b42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:59.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.972+0000 7fb821c00640 1 --2- 172.21.15.86:0/3689136490 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb7fc06c920 0x7fb7fc06ede0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb808034000 tx=0x7fb808008030 comp rx=0 tx=0).stop 2024-04-20T21:17:59.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.973+0000 7fb821c00640 1 -- 172.21.15.86:0/3689136490 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb81c0cf810 msgr2=0x7fb81c06fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.973+0000 7fb821c00640 1 --2- 172.21.15.86:0/3689136490 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb81c0cf810 0x7fb81c06fce0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb80c00e840 tx=0x7fb80c00ed00 comp rx=0 tx=0).stop 2024-04-20T21:17:59.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.973+0000 7f3083200640 1 -- 172.21.15.86:0/2389144901 shutdown_connections 2024-04-20T21:17:59.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.973+0000 7f3083200640 1 -- 172.21.15.86:0/2389144901 wait complete. 2024-04-20T21:17:59.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.973+0000 7fb821c00640 1 -- 172.21.15.86:0/3689136490 shutdown_connections 2024-04-20T21:17:59.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.973+0000 7fb821c00640 1 --2- 172.21.15.86:0/3689136490 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb7fc06c920 0x7fb7fc06ede0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.973+0000 7fb821c00640 1 --2- 172.21.15.86:0/3689136490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb81c15ed80 0x7fb81c15f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.973+0000 7fb821c00640 1 --2- 172.21.15.86:0/3689136490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb81c0d0fd0 0x7fb81c15a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.973+0000 7fb821c00640 1 --2- 172.21.15.86:0/3689136490 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb81c0cf810 0x7fb81c06fce0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.973+0000 7fb821c00640 1 -- 172.21.15.86:0/3689136490 >> 172.21.15.86:0/3689136490 conn(0x7fb81c06cc20 msgr2=0x7fb81c0b5b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:59.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.974+0000 7fb821c00640 1 -- 172.21.15.86:0/3689136490 shutdown_connections 2024-04-20T21:17:59.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.974+0000 7fb821c00640 1 -- 172.21.15.86:0/3689136490 wait complete. 2024-04-20T21:17:59.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.981+0000 7f771fa00640 1 -- 172.21.15.86:0/1471410674 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f77180689e0 con 0x7f77180d8640 2024-04-20T21:17:59.982 INFO:teuthology.orchestra.run.smithi086.stdout:55834574854 2024-04-20T21:17:59.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.981+0000 7f7715600640 1 -- 172.21.15.86:0/1471410674 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7700059d80 con 0x7f77180d8640 2024-04-20T21:17:59.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.984+0000 7f771fa00640 1 -- 172.21.15.86:0/1471410674 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f76f406c940 msgr2=0x7f76f406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.984+0000 7f771fa00640 1 --2- 172.21.15.86:0/1471410674 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f76f406c940 0x7f76f406ee00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7708036000 tx=0x7f770803b040 comp rx=0 tx=0).stop 2024-04-20T21:17:59.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.984+0000 7f771fa00640 1 -- 172.21.15.86:0/1471410674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77180d8640 msgr2=0x7f77180a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.984+0000 7f771fa00640 1 --2- 172.21.15.86:0/1471410674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77180d8640 0x7f77180a5190 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f770000c7f0 tx=0x7f770000cbf0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.985+0000 7f70fb000640 1 -- 172.21.15.86:0/1554258679 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f70f40d0c70 con 0x7f70f40b61e0 2024-04-20T21:17:59.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.985+0000 7f771fa00640 1 -- 172.21.15.86:0/1471410674 shutdown_connections 2024-04-20T21:17:59.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.985+0000 7f771fa00640 1 --2- 172.21.15.86:0/1471410674 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f76f406c940 0x7f76f406ee00 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.985+0000 7f771fa00640 1 --2- 172.21.15.86:0/1471410674 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f77180a56d0 0x7f77180a2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.985+0000 7f771fa00640 1 --2- 172.21.15.86:0/1471410674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77180d8640 0x7f77180a5190 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.985+0000 7f771fa00640 1 --2- 172.21.15.86:0/1471410674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7718072fd0 0x7f7718162300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.985+0000 7f771fa00640 1 -- 172.21.15.86:0/1471410674 >> 172.21.15.86:0/1471410674 conn(0x7f77180d5540 msgr2=0x7f771806def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:59.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.985+0000 7f771fa00640 1 -- 172.21.15.86:0/1471410674 shutdown_connections 2024-04-20T21:17:59.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.985+0000 7f771fa00640 1 -- 172.21.15.86:0/1471410674 wait complete. 2024-04-20T21:17:59.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.986+0000 7f70ebe00640 1 -- 172.21.15.86:0/1554258679 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f70e405d7b0 con 0x7f70f40b61e0 2024-04-20T21:17:59.987 INFO:teuthology.orchestra.run.smithi086.stdout:55834574852 2024-04-20T21:17:59.988 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.1 2024-04-20T21:17:59.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.988+0000 7f70fb000640 1 -- 172.21.15.86:0/1554258679 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f70d406c940 msgr2=0x7f70d406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.988+0000 7f70fb000640 1 --2- 172.21.15.86:0/1554258679 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f70d406c940 0x7f70d406ee00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f70e003e000 tx=0x7f70e0038000 comp rx=0 tx=0).stop 2024-04-20T21:17:59.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.988+0000 7f70fb000640 1 -- 172.21.15.86:0/1554258679 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70f40b61e0 msgr2=0x7f70f4156710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:17:59.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.988+0000 7f70fb000640 1 --2- 172.21.15.86:0/1554258679 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70f40b61e0 0x7f70f4156710 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f70e400c820 tx=0x7f70e400cce0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.989+0000 7f70fb000640 1 -- 172.21.15.86:0/1554258679 shutdown_connections 2024-04-20T21:17:59.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.989+0000 7f70fb000640 1 --2- 172.21.15.86:0/1554258679 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f70d406c940 0x7f70d406ee00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.989+0000 7f70fb000640 1 --2- 172.21.15.86:0/1554258679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70f415ec30 0x7f70f4155ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.989+0000 7f70fb000640 1 --2- 172.21.15.86:0/1554258679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70f415e850 0x7f70f40b7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.989+0000 7f70fb000640 1 --2- 172.21.15.86:0/1554258679 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70f40b61e0 0x7f70f4156710 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:17:59.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.989+0000 7f70fb000640 1 -- 172.21.15.86:0/1554258679 >> 172.21.15.86:0/1554258679 conn(0x7f70f408bf70 msgr2=0x7f70f40b1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:17:59.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.989+0000 7f70fb000640 1 -- 172.21.15.86:0/1554258679 shutdown_connections 2024-04-20T21:17:59.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:17:59.989+0000 7f70fb000640 1 -- 172.21.15.86:0/1554258679 wait complete. 2024-04-20T21:17:59.999 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.6 2024-04-20T21:18:00.000 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.0 2024-04-20T21:18:00.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:00.002+0000 7f4c95a00640 1 -- 172.21.15.86:0/3676161191 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f4c900b6de0 con 0x7f4c900b61e0 2024-04-20T21:18:00.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:00.002+0000 7f4c86a00640 1 -- 172.21.15.86:0/3676161191 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4c7c059cd0 con 0x7f4c900b61e0 2024-04-20T21:18:00.002 INFO:teuthology.orchestra.run.smithi086.stdout:55834574854 2024-04-20T21:18:00.006 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.7 2024-04-20T21:18:00.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:00.007+0000 7f4c95a00640 1 -- 172.21.15.86:0/3676161191 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4c7006c920 msgr2=0x7f4c7006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:00.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:00.007+0000 7f4c95a00640 1 --2- 172.21.15.86:0/3676161191 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4c7006c920 0x7f4c7006ede0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4c80036000 tx=0x7f4c80030000 comp rx=0 tx=0).stop 2024-04-20T21:18:00.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:00.007+0000 7f4c95a00640 1 -- 172.21.15.86:0/3676161191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c900b61e0 msgr2=0x7f4c900cccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:00.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:00.007+0000 7f4c95a00640 1 --2- 172.21.15.86:0/3676161191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c900b61e0 0x7f4c900cccd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c00e840 tx=0x7f4c7c00ed00 comp rx=0 tx=0).stop 2024-04-20T21:18:00.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:00.007+0000 7f4c95a00640 1 -- 172.21.15.86:0/3676161191 shutdown_connections 2024-04-20T21:18:00.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:00.007+0000 7f4c95a00640 1 --2- 172.21.15.86:0/3676161191 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4c7006c920 0x7f4c7006ede0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:00.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:00.007+0000 7f4c95a00640 1 --2- 172.21.15.86:0/3676161191 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c9015ec30 0x7f4c900b9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:00.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:00.007+0000 7f4c95a00640 1 --2- 172.21.15.86:0/3676161191 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c9015e850 0x7f4c900b9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:00.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:00.007+0000 7f4c95a00640 1 --2- 172.21.15.86:0/3676161191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c900b61e0 0x7f4c900cccd0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:00.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:00.007+0000 7f4c95a00640 1 -- 172.21.15.86:0/3676161191 >> 172.21.15.86:0/3676161191 conn(0x7f4c9008bf70 msgr2=0x7f4c900b49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:00.010 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.2 2024-04-20T21:18:00.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:00.007+0000 7f4c95a00640 1 -- 172.21.15.86:0/3676161191 shutdown_connections 2024-04-20T21:18:00.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:00.007+0000 7f4c95a00640 1 -- 172.21.15.86:0/3676161191 wait complete. 2024-04-20T21:18:00.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:00.012+0000 7f8f4c000640 1 -- 172.21.15.86:0/1898482109 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f8f440cba50 con 0x7f8f4415ec30 2024-04-20T21:18:00.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:00.012+0000 7f8f41600640 1 -- 172.21.15.86:0/1898482109 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8f3c05e440 con 0x7f8f4415ec30 2024-04-20T21:18:00.012 INFO:teuthology.orchestra.run.smithi086.stdout:55834574855 2024-04-20T21:18:00.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:00.016+0000 7f8f4c000640 1 -- 172.21.15.86:0/1898482109 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8f2406c940 msgr2=0x7f8f2406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:00.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:00.016+0000 7f8f4c000640 1 --2- 172.21.15.86:0/1898482109 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8f2406c940 0x7f8f2406ee00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8f38036000 tx=0x7f8f38008030 comp rx=0 tx=0).stop 2024-04-20T21:18:00.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:00.016+0000 7f8f4c000640 1 -- 172.21.15.86:0/1898482109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f4415ec30 msgr2=0x7f8f440cc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:00.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:00.016+0000 7f8f4c000640 1 --2- 172.21.15.86:0/1898482109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f4415ec30 0x7f8f440cc1b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c00c690 tx=0x7f8f3c00cb50 comp rx=0 tx=0).stop 2024-04-20T21:18:00.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:00.016+0000 7f8f4c000640 1 -- 172.21.15.86:0/1898482109 shutdown_connections 2024-04-20T21:18:00.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:00.016+0000 7f8f4c000640 1 --2- 172.21.15.86:0/1898482109 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8f2406c940 0x7f8f2406ee00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:00.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:00.016+0000 7f8f4c000640 1 --2- 172.21.15.86:0/1898482109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f4415ec30 0x7f8f440cc1b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:00.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:00.016+0000 7f8f4c000640 1 --2- 172.21.15.86:0/1898482109 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8f4415e850 0x7f8f440cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:00.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:00.016+0000 7f8f4c000640 1 --2- 172.21.15.86:0/1898482109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f440b68f0 0x7f8f4415ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:00.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:00.016+0000 7f8f4c000640 1 -- 172.21.15.86:0/1898482109 >> 172.21.15.86:0/1898482109 conn(0x7f8f4408bf70 msgr2=0x7f8f440ca2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:00.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:00.016+0000 7f8f4c000640 1 -- 172.21.15.86:0/1898482109 shutdown_connections 2024-04-20T21:18:00.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:00.016+0000 7f8f4c000640 1 -- 172.21.15.86:0/1898482109 wait complete. 2024-04-20T21:18:00.026 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.4 2024-04-20T21:18:00.033 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.5 2024-04-20T21:18:00.961 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-20T21:18:00.990 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-20T21:18:01.000 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-20T21:18:01.001 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-20T21:18:01.007 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-20T21:18:01.011 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-20T21:18:01.027 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-20T21:18:01.033 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-20T21:18:01.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.231+0000 7f36b0c00640 1 Processor -- start 2024-04-20T21:18:01.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.231+0000 7f36b0c00640 1 -- start start 2024-04-20T21:18:01.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.234+0000 7f36b0c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36a4054690 0x7f36a40df5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:01.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.234+0000 7f36a9e00640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f36a40df1c0 0x7f36a404fff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.86:36714/0 (socket says 172.21.15.86:36714) 2024-04-20T21:18:01.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.234+0000 7f36a9e00640 1 -- 172.21.15.86:0/4118695706 learned_addr learned my addr 172.21.15.86:0/4118695706 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:18:01.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.235+0000 7f36b0c00640 1 -- 172.21.15.86:0/4118695706 --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36a4053310 con 0x7f36a40dfae0 2024-04-20T21:18:01.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.235+0000 7f36b0c00640 1 -- 172.21.15.86:0/4118695706 --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36a40535f0 con 0x7f36a40df1c0 2024-04-20T21:18:01.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.235+0000 7f36b0c00640 1 -- 172.21.15.86:0/4118695706 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f36a40538d0 con 0x7f36a4054690 2024-04-20T21:18:01.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.236+0000 7f36a9400640 1 --2- 172.21.15.86:0/4118695706 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36a4054690 0x7f36a40df5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:01.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.237+0000 7f36a8a00640 1 -- 172.21.15.86:0/4118695706 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2146803020 0 0) 0x7f36a40535f0 con 0x7f36a40df1c0 2024-04-20T21:18:01.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.237+0000 7f36a8a00640 1 -- 172.21.15.86:0/4118695706 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3690003440 con 0x7f36a40df1c0 2024-04-20T21:18:01.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.237+0000 7f36a8a00640 1 -- 172.21.15.86:0/4118695706 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2306121395 0 0) 0x7f36a4053310 con 0x7f36a40dfae0 2024-04-20T21:18:01.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.237+0000 7f36a8a00640 1 -- 172.21.15.86:0/4118695706 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f36a40535f0 con 0x7f36a40dfae0 2024-04-20T21:18:01.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.238+0000 7f36a8a00640 1 -- 172.21.15.86:0/4118695706 <== mon.1 v1:172.21.15.144:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 657370207 0 0) 0x7f3690003440 con 0x7f36a40df1c0 2024-04-20T21:18:01.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.238+0000 7f36a8a00640 1 -- 172.21.15.86:0/4118695706 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36a4054690 msgr2=0x7f36a40df5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:01.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.238+0000 7f36a8a00640 1 --2- 172.21.15.86:0/4118695706 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36a4054690 0x7f36a40df5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.238+0000 7f36a8a00640 1 -- 172.21.15.86:0/4118695706 >> v1:172.21.15.86:6789/0 conn(0x7f36a40dfae0 legacy=0x7f36a4005ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:01.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.238+0000 7f36a8a00640 1 -- 172.21.15.86:0/4118695706 --> v1:172.21.15.144:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36a4053bb0 con 0x7f36a40df1c0 2024-04-20T21:18:01.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.238+0000 7f36a8a00640 1 -- 172.21.15.86:0/4118695706 <== mon.1 v1:172.21.15.144:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f3698004cd0 con 0x7f36a40df1c0 2024-04-20T21:18:01.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.238+0000 7f36a8a00640 1 -- 172.21.15.86:0/4118695706 >> v1:172.21.15.144:6789/0 conn(0x7f36a40df1c0 legacy=0x7f36a404fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:01.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.238+0000 7f36a8a00640 1 --2- 172.21.15.86:0/4118695706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3690003cb0 0x7f369000ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:01.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.238+0000 7f36a8a00640 1 --2- 172.21.15.86:0/4118695706 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36900040b0 0x7f369000d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:01.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.238+0000 7f36a8a00640 1 --2- 172.21.15.86:0/4118695706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f369000d6e0 0x7f369000fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:01.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.238+0000 7f36a8a00640 1 -- 172.21.15.86:0/4118695706 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f36a40538d0 con 0x7f369000d6e0 2024-04-20T21:18:01.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.238+0000 7f36a8a00640 1 -- 172.21.15.86:0/4118695706 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3690006870 con 0x7f3690003cb0 2024-04-20T21:18:01.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.238+0000 7f36a8a00640 1 -- 172.21.15.86:0/4118695706 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3690006a70 con 0x7f36900040b0 2024-04-20T21:18:01.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.239+0000 7f36a9e00640 1 --2- 172.21.15.86:0/4118695706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3690003cb0 0x7f369000ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:01.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.240+0000 7f36aa800640 1 --2- 172.21.15.86:0/4118695706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f369000d6e0 0x7f369000fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:01.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.240+0000 7f36a9e00640 1 -- 172.21.15.86:0/4118695706 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36900040b0 msgr2=0x7f369000d1a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T21:18:01.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.240+0000 7f36a9e00640 1 --2- 172.21.15.86:0/4118695706 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36900040b0 0x7f369000d1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.240+0000 7f36a9e00640 1 -- 172.21.15.86:0/4118695706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f369000d6e0 msgr2=0x7f369000fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:01.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.240+0000 7f36a9e00640 1 --2- 172.21.15.86:0/4118695706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f369000d6e0 0x7f369000fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.240+0000 7f36a9e00640 1 -- 172.21.15.86:0/4118695706 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3690003440 con 0x7f3690003cb0 2024-04-20T21:18:01.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.241+0000 7f36aa800640 1 --2- 172.21.15.86:0/4118695706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f369000d6e0 0x7f369000fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:18:01.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.242+0000 7f36a9e00640 1 --2- 172.21.15.86:0/4118695706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3690003cb0 0x7f369000ac50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3698001e70 tx=0x7f3690003730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=152cd2408e8c1f84 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:01.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.242+0000 7f36a8a00640 1 -- 172.21.15.86:0/4118695706 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3698038070 con 0x7f3690003cb0 2024-04-20T21:18:01.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.242+0000 7f36a8a00640 1 -- 172.21.15.86:0/4118695706 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3698009040 con 0x7f3690003cb0 2024-04-20T21:18:01.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.242+0000 7f36a8a00640 1 -- 172.21.15.86:0/4118695706 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3698001a90 con 0x7f3690003cb0 2024-04-20T21:18:01.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.243+0000 7f36b0c00640 1 -- 172.21.15.86:0/4118695706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3690003cb0 msgr2=0x7f369000ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:01.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.243+0000 7f36b0c00640 1 --2- 172.21.15.86:0/4118695706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3690003cb0 0x7f369000ac50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3698001e70 tx=0x7f3690003730 comp rx=0 tx=0).stop 2024-04-20T21:18:01.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.244+0000 7f36aa800640 1 -- 172.21.15.86:0/4118695706 reap_dead start 2024-04-20T21:18:01.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.244+0000 7f36b0c00640 1 -- 172.21.15.86:0/4118695706 shutdown_connections 2024-04-20T21:18:01.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.244+0000 7f36b0c00640 1 -- 172.21.15.86:0/4118695706 >> 172.21.15.86:0/4118695706 conn(0x7f36a4009fe0 msgr2=0x7f36a400a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:01.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.245+0000 7f36b0c00640 1 -- 172.21.15.86:0/4118695706 shutdown_connections 2024-04-20T21:18:01.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.245+0000 7f36b0c00640 1 -- 172.21.15.86:0/4118695706 wait complete. 2024-04-20T21:18:01.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.247+0000 7f36b0c00640 1 Processor -- start 2024-04-20T21:18:01.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.248+0000 7f36b0c00640 1 -- start start 2024-04-20T21:18:01.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.250+0000 7f36b0c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3690003cb0 0x7f36a4047570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:01.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.251+0000 7f36b0c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36a40483a0 0x7f36a40c39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:01.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.251+0000 7f36b0c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36a40d96e0 0x7f36a40c3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:01.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.251+0000 7f36b0c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f36a40e04e0 con 0x7f3690003cb0 2024-04-20T21:18:01.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.251+0000 7f36b0c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f36a4054170 con 0x7f36a40483a0 2024-04-20T21:18:01.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.252+0000 7f36aa800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36a40d96e0 0x7f36a40c3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:01.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.252+0000 7f36aa800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36a40d96e0 0x7f36a40c3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:55652/0 (socket says 172.21.15.86:55652) 2024-04-20T21:18:01.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.252+0000 7f36aa800640 1 -- 172.21.15.86:0/2003620636 learned_addr learned my addr 172.21.15.86:0/2003620636 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:01.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.252+0000 7f36a9400640 1 --2- 172.21.15.86:0/2003620636 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36a40483a0 0x7f36a40c39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:01.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.252+0000 7f36a9e00640 1 --2- 172.21.15.86:0/2003620636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3690003cb0 0x7f36a4047570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:01.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.252+0000 7f36b0c00640 1 -- 172.21.15.86:0/2003620636 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f36a40dff10 con 0x7f36a40d96e0 2024-04-20T21:18:01.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.254+0000 7f36aa800640 1 -- 172.21.15.86:0/2003620636 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36a40483a0 msgr2=0x7f36a40c39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:01.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.254+0000 7f36aa800640 1 --2- 172.21.15.86:0/2003620636 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36a40483a0 0x7f36a40c39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.254+0000 7f36aa800640 1 -- 172.21.15.86:0/2003620636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3690003cb0 msgr2=0x7f36a4047570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:01.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.254+0000 7f36aa800640 1 --2- 172.21.15.86:0/2003620636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3690003cb0 0x7f36a4047570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.254+0000 7f36aa800640 1 -- 172.21.15.86:0/2003620636 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36a40d3db0 con 0x7f36a40d96e0 2024-04-20T21:18:01.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.254+0000 7f36a9e00640 1 --2- 172.21.15.86:0/2003620636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3690003cb0 0x7f36a4047570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:18:01.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.255+0000 7f36aa800640 1 --2- 172.21.15.86:0/2003620636 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36a40d96e0 0x7f36a40c3f20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f36ac076a30 tx=0x7f36ac0694c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:01.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.255+0000 7f36a2000640 1 -- 172.21.15.86:0/2003620636 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ac072730 con 0x7f36a40d96e0 2024-04-20T21:18:01.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.256+0000 7f36a2000640 1 -- 172.21.15.86:0/2003620636 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36ac0728d0 con 0x7f36a40d96e0 2024-04-20T21:18:01.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.256+0000 7f36b0c00640 1 -- 172.21.15.86:0/2003620636 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36a4048910 con 0x7f36a40d96e0 2024-04-20T21:18:01.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.256+0000 7f36b0c00640 1 -- 172.21.15.86:0/2003620636 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f36a40d8ee0 con 0x7f36a40d96e0 2024-04-20T21:18:01.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.256+0000 7f36a2000640 1 -- 172.21.15.86:0/2003620636 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ac072ce0 con 0x7f36a40d96e0 2024-04-20T21:18:01.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.259+0000 7f3697400640 1 -- 172.21.15.86:0/2003620636 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3680005290 con 0x7f36a40d96e0 2024-04-20T21:18:01.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.261+0000 7f36a2000640 1 -- 172.21.15.86:0/2003620636 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f36ac0ad060 con 0x7f36a40d96e0 2024-04-20T21:18:01.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.262+0000 7f36a2000640 1 --2- 172.21.15.86:0/2003620636 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f368c06c9f0 0x7f368c06eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:01.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.263+0000 7f36a2000640 1 -- 172.21.15.86:0/2003620636 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f36ac117920 con 0x7f36a40d96e0 2024-04-20T21:18:01.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.263+0000 7f36a9e00640 1 --2- 172.21.15.86:0/2003620636 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f368c06c9f0 0x7f368c06eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:01.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.265+0000 7f36a9e00640 1 --2- 172.21.15.86:0/2003620636 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f368c06c9f0 0x7f368c06eeb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3698035000 tx=0x7f3698030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:01.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.266+0000 7f989ac00640 1 Processor -- start 2024-04-20T21:18:01.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.267+0000 7f989ac00640 1 -- start start 2024-04-20T21:18:01.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.269+0000 7f989ac00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f98940cfc80 0x7f98940d2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:01.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.269+0000 7f989ac00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98940d02f0 con 0x7f989415e8d0 2024-04-20T21:18:01.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.269+0000 7f989ac00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98940d2340 con 0x7f98940d3360 2024-04-20T21:18:01.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.269+0000 7f989ac00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f98940d2990 con 0x7f98940cfc80 2024-04-20T21:18:01.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.270+0000 7f9899800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f989415e8d0 0x7f98940d4fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:43572/0 (socket says 172.21.15.86:43572) 2024-04-20T21:18:01.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.270+0000 7f9899800640 1 -- 172.21.15.86:0/3258359552 learned_addr learned my addr 172.21.15.86:0/3258359552 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:18:01.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.270+0000 7f9898e00640 1 --2- 172.21.15.86:0/3258359552 >> v2:172.21.15.86:3300/0 conn(0x7f98940cfc80 0x7f98940d2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:01.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.270+0000 7f9898e00640 1 -- 172.21.15.86:0/3258359552 >> v1:172.21.15.144:6789/0 conn(0x7f98940d3360 legacy=0x7f98940d0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:01.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.271+0000 7f9898e00640 1 -- 172.21.15.86:0/3258359552 >> v1:172.21.15.86:6789/0 conn(0x7f989415e8d0 legacy=0x7f98940d4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:01.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.271+0000 7f9898e00640 1 -- 172.21.15.86:0/3258359552 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98940d2620 con 0x7f98940cfc80 2024-04-20T21:18:01.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.270+0000 7f9893e00640 1 -- 172.21.15.86:0/3258359552 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3955569720 0 0) 0x7f98940d02f0 con 0x7f989415e8d0 2024-04-20T21:18:01.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.272+0000 7f9898e00640 1 --2- 172.21.15.86:0/3258359552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98940cfc80 0x7f98940d2e20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9880009790 tx=0x7f988002b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48cdaea4cbd3fa53 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:01.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.273+0000 7f9893e00640 1 -- 172.21.15.86:0/3258359552 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9880034070 con 0x7f98940cfc80 2024-04-20T21:18:01.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.273+0000 7f9893e00640 1 -- 172.21.15.86:0/3258359552 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9880030800 con 0x7f98940cfc80 2024-04-20T21:18:01.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.273+0000 7f9893e00640 1 -- 172.21.15.86:0/3258359552 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9880030bd0 con 0x7f98940cfc80 2024-04-20T21:18:01.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.274+0000 7f71c2600640 1 Processor -- start 2024-04-20T21:18:01.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.274+0000 7f989ac00640 1 -- 172.21.15.86:0/3258359552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98940cfc80 msgr2=0x7f98940d2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:01.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.274+0000 7f989ac00640 1 --2- 172.21.15.86:0/3258359552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98940cfc80 0x7f98940d2e20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9880009790 tx=0x7f988002b940 comp rx=0 tx=0).stop 2024-04-20T21:18:01.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.274+0000 7f71c2600640 1 -- start start 2024-04-20T21:18:01.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.275+0000 7f989ac00640 1 -- 172.21.15.86:0/3258359552 shutdown_connections 2024-04-20T21:18:01.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.275+0000 7f989ac00640 1 --2- 172.21.15.86:0/3258359552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98940cfc80 0x7f98940d2e20 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.275+0000 7f989ac00640 1 -- 172.21.15.86:0/3258359552 >> 172.21.15.86:0/3258359552 conn(0x7f989408bf70 msgr2=0x7f98940d45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:01.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.276+0000 7f36a2000640 1 -- 172.21.15.86:0/2003620636 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f36ac0a1070 con 0x7f36a40d96e0 2024-04-20T21:18:01.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.275+0000 7f989ac00640 1 -- 172.21.15.86:0/3258359552 shutdown_connections 2024-04-20T21:18:01.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.275+0000 7f989ac00640 1 -- 172.21.15.86:0/3258359552 wait complete. 2024-04-20T21:18:01.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.276+0000 7f71c2600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f71bc0cfc80 0x7f71bc0d2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:01.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.276+0000 7f71c2600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f71bc0d3360 0x7f71bc0d0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:01.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.276+0000 7f71c2600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71bc0d02f0 con 0x7f71bc15e8d0 2024-04-20T21:18:01.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.276+0000 7f71c2600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f71bc0d2990 con 0x7f71bc0cfc80 2024-04-20T21:18:01.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.276+0000 7f71c2600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f71bc0d2340 con 0x7f71bc0d3360 2024-04-20T21:18:01.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.276+0000 7f71bbe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f71bc0cfc80 0x7f71bc0d2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:01.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.277+0000 7f71bbe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f71bc0cfc80 0x7f71bc0d2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57972/0 (socket says 172.21.15.86:57972) 2024-04-20T21:18:01.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.277+0000 7f71bbe00640 1 -- 172.21.15.86:0/934493563 learned_addr learned my addr 172.21.15.86:0/934493563 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:01.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.277+0000 7f71c1c00640 1 --2- 172.21.15.86:0/934493563 >> v2:172.21.15.144:3300/0 conn(0x7f71bc0d3360 0x7f71bc0d0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:01.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.278+0000 7f989ac00640 1 Processor -- start 2024-04-20T21:18:01.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.278+0000 7f989ac00640 1 -- start start 2024-04-20T21:18:01.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.279+0000 7f989ac00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98940cfc80 0x7f98940ca980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:01.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.280+0000 7f989ac00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f989415e8d0 0x7f98940ce020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:01.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.280+0000 7f989ac00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9894155240 0x7f98940b71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:01.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.280+0000 7f989ac00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f98940d1610 con 0x7f9894155240 2024-04-20T21:18:01.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.280+0000 7f989ac00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f98940d1490 con 0x7f989415e8d0 2024-04-20T21:18:01.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.280+0000 7f989ac00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f989406ca80 con 0x7f98940cfc80 2024-04-20T21:18:01.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.280+0000 7f9899800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98940cfc80 0x7f98940ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:01.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.281+0000 7f9899800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98940cfc80 0x7f98940ca980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:55656/0 (socket says 172.21.15.86:55656) 2024-04-20T21:18:01.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.281+0000 7f9899800640 1 -- 172.21.15.86:0/2197057840 learned_addr learned my addr 172.21.15.86:0/2197057840 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:01.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.281+0000 7f989a200640 1 --2- 172.21.15.86:0/2197057840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9894155240 0x7f98940b71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:01.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.278+0000 7f71bbe00640 1 -- 172.21.15.86:0/934493563 >> v2:172.21.15.144:3300/0 conn(0x7f71bc0d3360 msgr2=0x7f71bc0d0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:01.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.278+0000 7f71bbe00640 1 --2- 172.21.15.86:0/934493563 >> v2:172.21.15.144:3300/0 conn(0x7f71bc0d3360 0x7f71bc0d0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.278+0000 7f71bbe00640 1 -- 172.21.15.86:0/934493563 >> v1:172.21.15.86:6789/0 conn(0x7f71bc15e8d0 legacy=0x7f71bc0d4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:01.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.279+0000 7f71bbe00640 1 -- 172.21.15.86:0/934493563 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71bc0d2620 con 0x7f71bc0cfc80 2024-04-20T21:18:01.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.279+0000 7f71c1c00640 1 --2- 172.21.15.86:0/934493563 >> v2:172.21.15.144:3300/0 conn(0x7f71bc0d3360 0x7f71bc0d0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:18:01.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.281+0000 7f71bbe00640 1 --2- 172.21.15.86:0/934493563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71bc0cfc80 0x7f71bc0d2e20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f71bc08e220 tx=0x7f71a802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19b6b3916cb3b896 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:01.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.281+0000 7f758fe00640 1 Processor -- start 2024-04-20T21:18:01.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.282+0000 7f758fe00640 1 -- start start 2024-04-20T21:18:01.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.282+0000 7f989a200640 1 -- 172.21.15.86:0/2197057840 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98940cfc80 msgr2=0x7f98940ca980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:01.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.282+0000 7f989a200640 1 --2- 172.21.15.86:0/2197057840 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98940cfc80 0x7f98940ca980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.282+0000 7f989a200640 1 -- 172.21.15.86:0/2197057840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f989415e8d0 msgr2=0x7f98940ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:01.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.282+0000 7f9898e00640 1 --2- 172.21.15.86:0/2197057840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f989415e8d0 0x7f98940ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:01.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.281+0000 7f71bb400640 1 -- 172.21.15.86:0/934493563 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a802d040 con 0x7f71bc0cfc80 2024-04-20T21:18:01.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.282+0000 7f989a200640 1 --2- 172.21.15.86:0/2197057840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f989415e8d0 0x7f98940ce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.282+0000 7f989a200640 1 -- 172.21.15.86:0/2197057840 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98940caf80 con 0x7f9894155240 2024-04-20T21:18:01.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.282+0000 7f9898e00640 1 --2- 172.21.15.86:0/2197057840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f989415e8d0 0x7f98940ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:01.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.282+0000 7f9899800640 1 --2- 172.21.15.86:0/2197057840 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98940cfc80 0x7f98940ca980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:18:01.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.282+0000 7f71bb400640 1 -- 172.21.15.86:0/934493563 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71a8004230 con 0x7f71bc0cfc80 2024-04-20T21:18:01.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.282+0000 7f71bb400640 1 -- 172.21.15.86:0/934493563 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a8003610 con 0x7f71bc0cfc80 2024-04-20T21:18:01.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.283+0000 7f989a200640 1 --2- 172.21.15.86:0/2197057840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9894155240 0x7f98940b71d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f988c00c730 tx=0x7f988c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:01.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.283+0000 7f9891600640 1 -- 172.21.15.86:0/2197057840 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c016020 con 0x7f9894155240 2024-04-20T21:18:01.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.283+0000 7f9891600640 1 -- 172.21.15.86:0/2197057840 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f988c006d50 con 0x7f9894155240 2024-04-20T21:18:01.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.283+0000 7f9891600640 1 -- 172.21.15.86:0/2197057840 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c003bb0 con 0x7f9894155240 2024-04-20T21:18:01.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.283+0000 7f71c2600640 1 -- 172.21.15.86:0/934493563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71bc0cfc80 msgr2=0x7f71bc0d2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:01.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.283+0000 7f71c2600640 1 --2- 172.21.15.86:0/934493563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71bc0cfc80 0x7f71bc0d2e20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f71bc08e220 tx=0x7f71a802b8f0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.283+0000 7f758fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f758803e840 0x7f7588053bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:01.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.284+0000 7f71c2600640 1 -- 172.21.15.86:0/934493563 shutdown_connections 2024-04-20T21:18:01.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.284+0000 7f71c2600640 1 --2- 172.21.15.86:0/934493563 >> v2:172.21.15.144:3300/0 conn(0x7f71bc0d3360 0x7f71bc0d0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.284+0000 7f71c2600640 1 --2- 172.21.15.86:0/934493563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71bc0cfc80 0x7f71bc0d2e20 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.284+0000 7f71c2600640 1 -- 172.21.15.86:0/934493563 >> 172.21.15.86:0/934493563 conn(0x7f71bc08bf70 msgr2=0x7f71bc0d45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:01.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.284+0000 7f758fe00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7588054130 0x7f75880068d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:01.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.284+0000 7f71c2600640 1 -- 172.21.15.86:0/934493563 shutdown_connections 2024-04-20T21:18:01.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.284+0000 7f71c2600640 1 -- 172.21.15.86:0/934493563 wait complete. 2024-04-20T21:18:01.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.284+0000 7f989ac00640 1 -- 172.21.15.86:0/2197057840 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98940cd540 con 0x7f9894155240 2024-04-20T21:18:01.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.285+0000 7f989ac00640 1 -- 172.21.15.86:0/2197057840 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98940b7710 con 0x7f9894155240 2024-04-20T21:18:01.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.284+0000 7f758fe00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7588058570 con 0x7f75880e30c0 2024-04-20T21:18:01.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.284+0000 7f758fe00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f75880545a0 con 0x7f758803e840 2024-04-20T21:18:01.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.284+0000 7f758fe00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f7588058850 con 0x7f7588054130 2024-04-20T21:18:01.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.284+0000 7f758ea00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f75880e30c0 0x7f7588057d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:43582/0 (socket says 172.21.15.86:43582) 2024-04-20T21:18:01.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.284+0000 7f758ea00640 1 -- 172.21.15.86:0/1705725746 learned_addr learned my addr 172.21.15.86:0/1705725746 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:18:01.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.285+0000 7f758f400640 1 --2- 172.21.15.86:0/1705725746 >> v2:172.21.15.144:3300/0 conn(0x7f7588054130 0x7f75880068d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:01.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.285+0000 7f758e000640 1 --2- 172.21.15.86:0/1705725746 >> v2:172.21.15.86:3300/0 conn(0x7f758803e840 0x7f7588053bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:01.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.286+0000 7f989ac00640 1 -- 172.21.15.86:0/2197057840 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98940cd350 con 0x7f9894155240 2024-04-20T21:18:01.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.286+0000 7f758f400640 1 -- 172.21.15.86:0/1705725746 >> v2:172.21.15.86:3300/0 conn(0x7f758803e840 msgr2=0x7f7588053bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:01.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.287+0000 7f758f400640 1 --2- 172.21.15.86:0/1705725746 >> v2:172.21.15.86:3300/0 conn(0x7f758803e840 0x7f7588053bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.287+0000 7f758f400640 1 -- 172.21.15.86:0/1705725746 >> v1:172.21.15.86:6789/0 conn(0x7f75880e30c0 legacy=0x7f7588057d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:01.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.287+0000 7f758d600640 1 -- 172.21.15.86:0/1705725746 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 508158866 0 0) 0x7f7588058570 con 0x7f75880e30c0 2024-04-20T21:18:01.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.287+0000 7f758f400640 1 -- 172.21.15.86:0/1705725746 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7588058b30 con 0x7f7588054130 2024-04-20T21:18:01.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.288+0000 7f758e000640 1 --2- 172.21.15.86:0/1705725746 >> v2:172.21.15.86:3300/0 conn(0x7f758803e840 0x7f7588053bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:18:01.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.289+0000 7f9891600640 1 -- 172.21.15.86:0/2197057840 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f988c0050d0 con 0x7f9894155240 2024-04-20T21:18:01.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.289+0000 7f71c2600640 1 Processor -- start 2024-04-20T21:18:01.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.289+0000 7f71c2600640 1 -- start start 2024-04-20T21:18:01.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.290+0000 7f758f400640 1 --2- 172.21.15.86:0/1705725746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7588054130 0x7f75880068d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f75880e7cd0 tx=0x7f757c02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=207ac13ab45d6f06 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:01.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.290+0000 7fede2400640 1 Processor -- start 2024-04-20T21:18:01.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.290+0000 7f9891600640 1 --2- 172.21.15.86:0/2197057840 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f987406c940 0x7f987406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:01.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.290+0000 7fede2400640 1 -- start start 2024-04-20T21:18:01.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.290+0000 7f9891600640 1 -- 172.21.15.86:0/2197057840 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f988c08b5c0 con 0x7f9894155240 2024-04-20T21:18:01.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.291+0000 7f9899800640 1 --2- 172.21.15.86:0/2197057840 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f987406c940 0x7f987406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:01.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.293+0000 7f9899800640 1 --2- 172.21.15.86:0/2197057840 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f987406c940 0x7f987406ee00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9888000f60 tx=0x7f9888009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:01.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.291+0000 7f71c2600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71bc0cfc80 0x7f71bc150d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:01.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.292+0000 7f71c2600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71bc15e8d0 0x7f71bc0b9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:01.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.292+0000 7f71c2600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71bc0b9970 0x7f71bc0c0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:01.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.292+0000 7f71c1200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71bc0cfc80 0x7f71bc150d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:01.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.292+0000 7f71c2600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f71bc0d1440 con 0x7f71bc0cfc80 2024-04-20T21:18:01.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.292+0000 7f71c2600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f71bc0d12c0 con 0x7f71bc15e8d0 2024-04-20T21:18:01.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.292+0000 7f71c2600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f71bc0d15c0 con 0x7f71bc0b9970 2024-04-20T21:18:01.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.292+0000 7f71c1200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71bc0cfc80 0x7f71bc150d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57988/0 (socket says 172.21.15.86:57988) 2024-04-20T21:18:01.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.292+0000 7f71c1200640 1 -- 172.21.15.86:0/962375563 learned_addr learned my addr 172.21.15.86:0/962375563 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:01.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.292+0000 7f71bbe00640 1 --2- 172.21.15.86:0/962375563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71bc15e8d0 0x7f71bc0b9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:01.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.292+0000 7f758d600640 1 -- 172.21.15.86:0/1705725746 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c035020 con 0x7f7588054130 2024-04-20T21:18:01.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.292+0000 7f758d600640 1 -- 172.21.15.86:0/1705725746 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f757c031810 con 0x7f7588054130 2024-04-20T21:18:01.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.292+0000 7f758d600640 1 -- 172.21.15.86:0/1705725746 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c031be0 con 0x7f7588054130 2024-04-20T21:18:01.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.292+0000 7fede2400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7feddc15ad50 0x7feddc0b4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:01.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.292+0000 7fede2400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7feddc0723e0 0x7feddc0cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:01.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.292+0000 7fede2400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feddc0b3cc0 con 0x7feddc06ef20 2024-04-20T21:18:01.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.292+0000 7fede2400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7feddc0b3f60 con 0x7feddc0723e0 2024-04-20T21:18:01.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.292+0000 7fede2400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7feddc0b4240 con 0x7feddc15ad50 2024-04-20T21:18:01.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.293+0000 7feddbe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7feddc0723e0 0x7feddc0cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:01.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.293+0000 7feddbe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7feddc0723e0 0x7feddc0cd8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57998/0 (socket says 172.21.15.86:57998) 2024-04-20T21:18:01.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.293+0000 7f758fe00640 1 -- 172.21.15.86:0/1705725746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7588054130 msgr2=0x7f75880068d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:01.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.293+0000 7f758fe00640 1 --2- 172.21.15.86:0/1705725746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7588054130 0x7f75880068d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f75880e7cd0 tx=0x7f757c02b8b0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.293+0000 7feddbe00640 1 -- 172.21.15.86:0/3167510737 learned_addr learned my addr 172.21.15.86:0/3167510737 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:01.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.294+0000 7f758fe00640 1 -- 172.21.15.86:0/1705725746 shutdown_connections 2024-04-20T21:18:01.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.294+0000 7f758fe00640 1 --2- 172.21.15.86:0/1705725746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7588054130 0x7f75880068d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.294+0000 7f71c1c00640 1 --2- 172.21.15.86:0/962375563 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71bc0b9970 0x7f71bc0c0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:01.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.294+0000 7f71bbe00640 1 -- 172.21.15.86:0/962375563 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71bc0b9970 msgr2=0x7f71bc0c0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:01.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.294+0000 7f71bbe00640 1 --2- 172.21.15.86:0/962375563 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71bc0b9970 0x7f71bc0c0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.294+0000 7f71bbe00640 1 -- 172.21.15.86:0/962375563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71bc0cfc80 msgr2=0x7f71bc150d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:01.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.294+0000 7f71bbe00640 1 --2- 172.21.15.86:0/962375563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71bc0cfc80 0x7f71bc150d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.294+0000 7feddbe00640 1 -- 172.21.15.86:0/3167510737 >> v2:172.21.15.144:3300/0 conn(0x7feddc15ad50 msgr2=0x7feddc0b4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:01.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.294+0000 7feddbe00640 1 --2- 172.21.15.86:0/3167510737 >> v2:172.21.15.144:3300/0 conn(0x7feddc15ad50 0x7feddc0b4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.294+0000 7f71bbe00640 1 -- 172.21.15.86:0/962375563 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71bc0b4160 con 0x7f71bc15e8d0 2024-04-20T21:18:01.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.294+0000 7f71c1200640 1 --2- 172.21.15.86:0/962375563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71bc0cfc80 0x7f71bc150d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:18:01.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.294+0000 7f71c1c00640 1 --2- 172.21.15.86:0/962375563 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71bc0b9970 0x7f71bc0c0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:01.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.294+0000 7feddbe00640 1 -- 172.21.15.86:0/3167510737 >> v1:172.21.15.144:6789/0 conn(0x7feddc06ef20 legacy=0x7feddc071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:01.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.294+0000 7feddbe00640 1 -- 172.21.15.86:0/3167510737 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feddc15f860 con 0x7feddc0723e0 2024-04-20T21:18:01.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.294+0000 7f758fe00640 1 --2- 172.21.15.86:0/1705725746 >> v2:172.21.15.86:3300/0 conn(0x7f758803e840 0x7f7588053bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.294+0000 7f758fe00640 1 -- 172.21.15.86:0/1705725746 >> 172.21.15.86:0/1705725746 conn(0x7f75880559e0 msgr2=0x7f7588055df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:01.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.295+0000 7f758fe00640 1 -- 172.21.15.86:0/1705725746 shutdown_connections 2024-04-20T21:18:01.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.295+0000 7f71bbe00640 1 --2- 172.21.15.86:0/962375563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71bc15e8d0 0x7f71bc0b9430 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f71a8004000 tx=0x7f71a80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:01.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.295+0000 7f758fe00640 1 -- 172.21.15.86:0/1705725746 wait complete. 2024-04-20T21:18:01.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.295+0000 7feddbe00640 1 --2- 172.21.15.86:0/3167510737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feddc0723e0 0x7feddc0cd8d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7feddc06d660 tx=0x7fedcc02df40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b9ee22e09ababc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:01.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.296+0000 7f71b8c00640 1 -- 172.21.15.86:0/962375563 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a8040070 con 0x7f71bc15e8d0 2024-04-20T21:18:01.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.296+0000 7f71b8c00640 1 -- 172.21.15.86:0/962375563 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71a8031650 con 0x7f71bc15e8d0 2024-04-20T21:18:01.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.296+0000 7f71b8c00640 1 -- 172.21.15.86:0/962375563 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a8031a20 con 0x7f71bc15e8d0 2024-04-20T21:18:01.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.297+0000 7f71c2600640 1 -- 172.21.15.86:0/962375563 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71bc0b4400 con 0x7f71bc15e8d0 2024-04-20T21:18:01.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.297+0000 7f71c2600640 1 -- 172.21.15.86:0/962375563 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71bc0b8c30 con 0x7f71bc15e8d0 2024-04-20T21:18:01.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.297+0000 7f758fe00640 1 Processor -- start 2024-04-20T21:18:01.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.299+0000 7f71c2600640 1 -- 172.21.15.86:0/962375563 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71bc160840 con 0x7f71bc15e8d0 2024-04-20T21:18:01.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.299+0000 7f758fe00640 1 -- start start 2024-04-20T21:18:01.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.300+0000 7fedda000640 1 -- 172.21.15.86:0/3167510737 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedcc034070 con 0x7feddc0723e0 2024-04-20T21:18:01.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.300+0000 7fedda000640 1 -- 172.21.15.86:0/3167510737 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedcc02ecf0 con 0x7feddc0723e0 2024-04-20T21:18:01.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.300+0000 7fedda000640 1 -- 172.21.15.86:0/3167510737 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedcc038620 con 0x7feddc0723e0 2024-04-20T21:18:01.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.300+0000 7f9891600640 1 -- 172.21.15.86:0/2197057840 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f988c020020 con 0x7f9894155240 2024-04-20T21:18:01.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.300+0000 7f71b8c00640 1 -- 172.21.15.86:0/962375563 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f71a8031bc0 con 0x7f71bc15e8d0 2024-04-20T21:18:01.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.301+0000 7f758fe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f758803e840 0x7f7588052550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:01.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.301+0000 7f758fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7588054130 0x7f758817ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:01.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.301+0000 7f758fe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75880e30c0 0x7f7588177300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:01.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.302+0000 7fede2400640 1 -- 172.21.15.86:0/3167510737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feddc0723e0 msgr2=0x7feddc0cd8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:01.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.302+0000 7fede2400640 1 --2- 172.21.15.86:0/3167510737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feddc0723e0 0x7feddc0cd8d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7feddc06d660 tx=0x7fedcc02df40 comp rx=0 tx=0).stop 2024-04-20T21:18:01.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.302+0000 7f71b8c00640 1 --2- 172.21.15.86:0/962375563 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f719c06c8f0 0x7f719c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:01.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.302+0000 7f758fe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f75880e34a0 con 0x7f7588054130 2024-04-20T21:18:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.302+0000 7f758fe00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f758803bfc0 con 0x7f75880e30c0 2024-04-20T21:18:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.302+0000 7f758fe00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7588006e10 con 0x7f758803e840 2024-04-20T21:18:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.302+0000 7f758ea00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f758803e840 0x7f7588052550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.302+0000 7fede2400640 1 -- 172.21.15.86:0/3167510737 shutdown_connections 2024-04-20T21:18:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.302+0000 7fede2400640 1 --2- 172.21.15.86:0/3167510737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feddc0723e0 0x7feddc0cd8d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.302+0000 7fede2400640 1 --2- 172.21.15.86:0/3167510737 >> v2:172.21.15.144:3300/0 conn(0x7feddc15ad50 0x7feddc0b4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.302+0000 7fede2400640 1 -- 172.21.15.86:0/3167510737 >> 172.21.15.86:0/3167510737 conn(0x7feddc06b980 msgr2=0x7feddc06bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.302+0000 7f71b8c00640 1 -- 172.21.15.86:0/962375563 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f71a80ada50 con 0x7f71bc15e8d0 2024-04-20T21:18:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.303+0000 7fede2400640 1 -- 172.21.15.86:0/3167510737 shutdown_connections 2024-04-20T21:18:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.302+0000 7f758ea00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f758803e840 0x7f7588052550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:55682/0 (socket says 172.21.15.86:55682) 2024-04-20T21:18:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.302+0000 7f758ea00640 1 -- 172.21.15.86:0/4039265863 learned_addr learned my addr 172.21.15.86:0/4039265863 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.303+0000 7f758f400640 1 --2- 172.21.15.86:0/4039265863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75880e30c0 0x7f7588177300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.303+0000 7f758e000640 1 --2- 172.21.15.86:0/4039265863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7588054130 0x7f758817ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:01.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.303+0000 7f71c1200640 1 --2- 172.21.15.86:0/962375563 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f719c06c8f0 0x7f719c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:01.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.303+0000 7fede2400640 1 -- 172.21.15.86:0/3167510737 wait complete. 2024-04-20T21:18:01.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.304+0000 7f758f400640 1 -- 172.21.15.86:0/4039265863 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f758803e840 msgr2=0x7f7588052550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:01.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.304+0000 7f758f400640 1 --2- 172.21.15.86:0/4039265863 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f758803e840 0x7f7588052550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.304+0000 7f758f400640 1 -- 172.21.15.86:0/4039265863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7588054130 msgr2=0x7f758817ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:01.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.304+0000 7f758f400640 1 --2- 172.21.15.86:0/4039265863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7588054130 0x7f758817ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.304+0000 7f758f400640 1 -- 172.21.15.86:0/4039265863 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f758817f4b0 con 0x7f75880e30c0 2024-04-20T21:18:01.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.304+0000 7f758e000640 1 --2- 172.21.15.86:0/4039265863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7588054130 0x7f758817ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:18:01.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.305+0000 7f71c1200640 1 --2- 172.21.15.86:0/962375563 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f719c06c8f0 0x7f719c06edb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f71b0007c60 tx=0x7f71b0007bb0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:01.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.305+0000 7f758f400640 1 --2- 172.21.15.86:0/4039265863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75880e30c0 0x7f7588177300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7588056930 tx=0x7f757c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:01.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.306+0000 7fede2400640 1 Processor -- start 2024-04-20T21:18:01.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.306+0000 7fede2400640 1 -- start start 2024-04-20T21:18:01.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.308+0000 7fede2400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feddc06ef20 0x7feddc0c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:01.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.308+0000 7fede2400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feddc0723e0 0x7feddc1542c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:01.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.308+0000 7fede2400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feddc15ad50 0x7feddc0cf540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:01.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.308+0000 7fede2400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feddc15f520 con 0x7feddc06ef20 2024-04-20T21:18:01.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.308+0000 7fede2400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7feddc15f3a0 con 0x7feddc0723e0 2024-04-20T21:18:01.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.308+0000 7fede2400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7feddc06f370 con 0x7feddc15ad50 2024-04-20T21:18:01.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.308+0000 7feddb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feddc06ef20 0x7feddc0c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:01.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.308+0000 7feddb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feddc06ef20 0x7feddc0c5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58024/0 (socket says 172.21.15.86:58024) 2024-04-20T21:18:01.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.309+0000 7feddb400640 1 -- 172.21.15.86:0/4170992164 learned_addr learned my addr 172.21.15.86:0/4170992164 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:01.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.309+0000 7feddbe00640 1 --2- 172.21.15.86:0/4170992164 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feddc15ad50 0x7feddc0cf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:01.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.309+0000 7feddaa00640 1 --2- 172.21.15.86:0/4170992164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feddc0723e0 0x7feddc1542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:01.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.309+0000 7feddb400640 1 -- 172.21.15.86:0/4170992164 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feddc15ad50 msgr2=0x7feddc0cf540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:01.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.309+0000 7feddb400640 1 --2- 172.21.15.86:0/4170992164 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feddc15ad50 0x7feddc0cf540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.309+0000 7feddb400640 1 -- 172.21.15.86:0/4170992164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feddc0723e0 msgr2=0x7feddc1542c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:01.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.309+0000 7feddb400640 1 --2- 172.21.15.86:0/4170992164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feddc0723e0 0x7feddc1542c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.309+0000 7feddb400640 1 -- 172.21.15.86:0/4170992164 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feddc0c6290 con 0x7feddc06ef20 2024-04-20T21:18:01.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.310+0000 7feddb400640 1 --2- 172.21.15.86:0/4170992164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feddc06ef20 0x7feddc0c5cb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fedc800b410 tx=0x7fedc800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:01.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.310+0000 7fedd3400640 1 -- 172.21.15.86:0/4170992164 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc8015020 con 0x7feddc06ef20 2024-04-20T21:18:01.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.310+0000 7fedd3400640 1 -- 172.21.15.86:0/4170992164 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedc8011850 con 0x7feddc06ef20 2024-04-20T21:18:01.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.311+0000 7fede2400640 1 -- 172.21.15.86:0/4170992164 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feddc0c6570 con 0x7feddc06ef20 2024-04-20T21:18:01.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.311+0000 7fedd3400640 1 -- 172.21.15.86:0/4170992164 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc8011c40 con 0x7feddc06ef20 2024-04-20T21:18:01.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.311+0000 7fede2400640 1 -- 172.21.15.86:0/4170992164 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feddc0c6b30 con 0x7feddc06ef20 2024-04-20T21:18:01.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.313+0000 7fede2400640 1 -- 172.21.15.86:0/4170992164 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feddc1597b0 con 0x7feddc06ef20 2024-04-20T21:18:01.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.315+0000 7fedd3400640 1 -- 172.21.15.86:0/4170992164 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fedc8018020 con 0x7feddc06ef20 2024-04-20T21:18:01.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.317+0000 7fedd3400640 1 --2- 172.21.15.86:0/4170992164 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fedbc06c940 0x7fedbc06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:01.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.317+0000 7fedd3400640 1 -- 172.21.15.86:0/4170992164 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fedc808b980 con 0x7feddc06ef20 2024-04-20T21:18:01.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.318+0000 7f71b8c00640 1 -- 172.21.15.86:0/962375563 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f71bc160840 con 0x7f71bc15e8d0 2024-04-20T21:18:01.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.318+0000 7f7586a00640 1 -- 172.21.15.86:0/4039265863 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c041070 con 0x7f75880e30c0 2024-04-20T21:18:01.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.318+0000 7f7586a00640 1 -- 172.21.15.86:0/4039265863 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f757c03f070 con 0x7f75880e30c0 2024-04-20T21:18:01.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.318+0000 7f7586a00640 1 -- 172.21.15.86:0/4039265863 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c035040 con 0x7f75880e30c0 2024-04-20T21:18:01.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.319+0000 7f758fe00640 1 -- 172.21.15.86:0/4039265863 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f758817f780 con 0x7f75880e30c0 2024-04-20T21:18:01.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.319+0000 7f758fe00640 1 -- 172.21.15.86:0/4039265863 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f758817fd40 con 0x7f75880e30c0 2024-04-20T21:18:01.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.319+0000 7feddaa00640 1 --2- 172.21.15.86:0/4170992164 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fedbc06c940 0x7fedbc06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:01.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.322+0000 7f758fe00640 1 -- 172.21.15.86:0/4039265863 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75880057e0 con 0x7f75880e30c0 2024-04-20T21:18:01.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.324+0000 7f7586a00640 1 -- 172.21.15.86:0/4039265863 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f757c038020 con 0x7f75880e30c0 2024-04-20T21:18:01.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.326+0000 7f7586a00640 1 --2- 172.21.15.86:0/4039265863 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f757006c9c0 0x7f757006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:01.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.326+0000 7f7586a00640 1 -- 172.21.15.86:0/4039265863 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f757c0ada60 con 0x7f75880e30c0 2024-04-20T21:18:01.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.328+0000 7f758ea00640 1 --2- 172.21.15.86:0/4039265863 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f757006c9c0 0x7f757006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:01.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.330+0000 7f758ea00640 1 --2- 172.21.15.86:0/4039265863 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f757006c9c0 0x7f757006ee80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7580001030 tx=0x7f7580007240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:01.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.329+0000 7feddaa00640 1 --2- 172.21.15.86:0/4170992164 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fedbc06c940 0x7fedbc06ee00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7feddc15cd10 tx=0x7fedd4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:01.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.330+0000 7fedd3400640 1 -- 172.21.15.86:0/4170992164 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7feddc1597b0 con 0x7feddc06ef20 2024-04-20T21:18:01.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.338+0000 7f7586a00640 1 -- 172.21.15.86:0/4039265863 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f757c07b8b0 con 0x7f75880e30c0 2024-04-20T21:18:01.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.348+0000 7fe20b600640 1 Processor -- start 2024-04-20T21:18:01.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.349+0000 7fe20b600640 1 -- start start 2024-04-20T21:18:01.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.351+0000 7faeee600640 1 Processor -- start 2024-04-20T21:18:01.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.351+0000 7fe20b600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe204162aa0 0x7fe2040b3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:01.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.351+0000 7fe20b600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe204166780 0x7fe204166be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:01.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.351+0000 7fe20b600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe20406f020 con 0x7fe204071cd0 2024-04-20T21:18:01.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.351+0000 7fe20b600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe204167120 con 0x7fe204166780 2024-04-20T21:18:01.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.351+0000 7fe20b600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fe20406ea60 con 0x7fe204162aa0 2024-04-20T21:18:01.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.351+0000 7faeee600640 1 -- start start 2024-04-20T21:18:01.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.352+0000 7fe203e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe204166780 0x7fe204166be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:01.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.352+0000 7fe208c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe204162aa0 0x7fe2040b3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:01.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.352+0000 7fe203e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe204166780 0x7fe204166be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58034/0 (socket says 172.21.15.86:58034) 2024-04-20T21:18:01.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.352+0000 7fe203e00640 1 -- 172.21.15.86:0/1529280991 learned_addr learned my addr 172.21.15.86:0/1529280991 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:01.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.353+0000 7faeee600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7faee815e8d0 0x7faee80cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:01.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.353+0000 7faeee600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7faee80d0d80 0x7faee80d11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:01.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.353+0000 7faeee600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faee80d3530 con 0x7faee80d4fa0 2024-04-20T21:18:01.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.353+0000 7faeee600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7faee80d2990 con 0x7faee815e8d0 2024-04-20T21:18:01.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.353+0000 7faeee600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7faee80d1710 con 0x7faee80d0d80 2024-04-20T21:18:01.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.353+0000 7fe203e00640 1 -- 172.21.15.86:0/1529280991 >> v2:172.21.15.144:3300/0 conn(0x7fe204162aa0 msgr2=0x7fe2040b3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:01.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.353+0000 7fe203e00640 1 --2- 172.21.15.86:0/1529280991 >> v2:172.21.15.144:3300/0 conn(0x7fe204162aa0 0x7fe2040b3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.353+0000 7fe203e00640 1 -- 172.21.15.86:0/1529280991 >> v1:172.21.15.144:6789/0 conn(0x7fe204071cd0 legacy=0x7fe204072120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:01.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.353+0000 7fe203e00640 1 -- 172.21.15.86:0/1529280991 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe20406ed40 con 0x7fe204166780 2024-04-20T21:18:01.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.354+0000 7faeed200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7faee815e8d0 0x7faee80cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:01.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.355+0000 7faeed200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7faee815e8d0 0x7faee80cfc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58048/0 (socket says 172.21.15.86:58048) 2024-04-20T21:18:01.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.355+0000 7faeed200640 1 -- 172.21.15.86:0/1257310275 learned_addr learned my addr 172.21.15.86:0/1257310275 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:01.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.355+0000 7faeedc00640 1 --2- 172.21.15.86:0/1257310275 >> v2:172.21.15.144:3300/0 conn(0x7faee80d0d80 0x7faee80d11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:01.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.356+0000 7fe203e00640 1 --2- 172.21.15.86:0/1529280991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe204166780 0x7fe204166be0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe2040730e0 tx=0x7fe1f002ba60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d70fc91f3e7eaeb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:01.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.357+0000 7fe203400640 1 -- 172.21.15.86:0/1529280991 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f0037680 con 0x7fe204166780 2024-04-20T21:18:01.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.357+0000 7fe203400640 1 -- 172.21.15.86:0/1529280991 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1f0037820 con 0x7fe204166780 2024-04-20T21:18:01.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.356+0000 7faeed200640 1 -- 172.21.15.86:0/1257310275 >> v2:172.21.15.144:3300/0 conn(0x7faee80d0d80 msgr2=0x7faee80d11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:01.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.356+0000 7faeed200640 1 --2- 172.21.15.86:0/1257310275 >> v2:172.21.15.144:3300/0 conn(0x7faee80d0d80 0x7faee80d11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.356+0000 7faeed200640 1 -- 172.21.15.86:0/1257310275 >> v1:172.21.15.86:6789/0 conn(0x7faee80d4fa0 legacy=0x7faee80d2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:01.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.356+0000 7faeed200640 1 -- 172.21.15.86:0/1257310275 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faee80d02f0 con 0x7faee815e8d0 2024-04-20T21:18:01.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.357+0000 7fe203400640 1 -- 172.21.15.86:0/1529280991 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f0037bf0 con 0x7fe204166780 2024-04-20T21:18:01.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.357+0000 7faeedc00640 1 --2- 172.21.15.86:0/1257310275 >> v2:172.21.15.144:3300/0 conn(0x7faee80d0d80 0x7faee80d11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:18:01.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.358+0000 7faeed200640 1 --2- 172.21.15.86:0/1257310275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faee815e8d0 0x7faee80cfc80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7faee80c7f60 tx=0x7faedc02b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ff638a99c2c761d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:01.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.359+0000 7fe20b600640 1 -- 172.21.15.86:0/1529280991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe204166780 msgr2=0x7fe204166be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:01.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.359+0000 7fe20b600640 1 --2- 172.21.15.86:0/1529280991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe204166780 0x7fe204166be0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe2040730e0 tx=0x7fe1f002ba60 comp rx=0 tx=0).stop 2024-04-20T21:18:01.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.359+0000 7faee7400640 1 -- 172.21.15.86:0/1257310275 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faedc035020 con 0x7faee815e8d0 2024-04-20T21:18:01.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.359+0000 7faee7400640 1 -- 172.21.15.86:0/1257310275 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faedc031800 con 0x7faee815e8d0 2024-04-20T21:18:01.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.359+0000 7faee7400640 1 -- 172.21.15.86:0/1257310275 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faedc031c10 con 0x7faee815e8d0 2024-04-20T21:18:01.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.359+0000 7fe20b600640 1 -- 172.21.15.86:0/1529280991 shutdown_connections 2024-04-20T21:18:01.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.359+0000 7fe20b600640 1 --2- 172.21.15.86:0/1529280991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe204166780 0x7fe204166be0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.359+0000 7fe20b600640 1 --2- 172.21.15.86:0/1529280991 >> v2:172.21.15.144:3300/0 conn(0x7fe204162aa0 0x7fe2040b3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.359+0000 7fe20b600640 1 -- 172.21.15.86:0/1529280991 >> 172.21.15.86:0/1529280991 conn(0x7fe20406b980 msgr2=0x7fe20406bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:01.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.360+0000 7fe20b600640 1 -- 172.21.15.86:0/1529280991 shutdown_connections 2024-04-20T21:18:01.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.360+0000 7faeee600640 1 -- 172.21.15.86:0/1257310275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faee815e8d0 msgr2=0x7faee80cfc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:01.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.361+0000 7faeee600640 1 --2- 172.21.15.86:0/1257310275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faee815e8d0 0x7faee80cfc80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7faee80c7f60 tx=0x7faedc02b8a0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.360+0000 7fe20b600640 1 -- 172.21.15.86:0/1529280991 wait complete. 2024-04-20T21:18:01.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.362+0000 7fe20b600640 1 Processor -- start 2024-04-20T21:18:01.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.363+0000 7faeee600640 1 -- 172.21.15.86:0/1257310275 shutdown_connections 2024-04-20T21:18:01.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.363+0000 7fe20b600640 1 -- start start 2024-04-20T21:18:01.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.363+0000 7faeee600640 1 --2- 172.21.15.86:0/1257310275 >> v2:172.21.15.144:3300/0 conn(0x7faee80d0d80 0x7faee80d11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.363+0000 7faeee600640 1 --2- 172.21.15.86:0/1257310275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faee815e8d0 0x7faee80cfc80 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.363+0000 7faeee600640 1 -- 172.21.15.86:0/1257310275 >> 172.21.15.86:0/1257310275 conn(0x7faee808bf70 msgr2=0x7faee80d45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:01.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.365+0000 7faeee600640 1 -- 172.21.15.86:0/1257310275 shutdown_connections 2024-04-20T21:18:01.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.365+0000 7fe20b600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe204071cd0 0x7fe204166070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:01.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.366+0000 7faeee600640 1 -- 172.21.15.86:0/1257310275 wait complete. 2024-04-20T21:18:01.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.366+0000 7fe20b600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe204162aa0 0x7fe204155730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:01.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.366+0000 7fe20b600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe204166780 0x7fe204162e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:01.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.366+0000 7fe20b600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe2040b3870 con 0x7fe204071cd0 2024-04-20T21:18:01.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.366+0000 7fe20b600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe2040b4150 con 0x7fe204166780 2024-04-20T21:18:01.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.367+0000 7fe208c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe204071cd0 0x7fe204166070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:01.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.367+0000 7fe20b600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe2040b39f0 con 0x7fe204162aa0 2024-04-20T21:18:01.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.367+0000 7fe203e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe204162aa0 0x7fe204155730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:01.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.367+0000 7fe203e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe204162aa0 0x7fe204155730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:55712/0 (socket says 172.21.15.86:55712) 2024-04-20T21:18:01.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.367+0000 7fe203e00640 1 -- 172.21.15.86:0/2905159268 learned_addr learned my addr 172.21.15.86:0/2905159268 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:01.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.367+0000 7fe209600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe204166780 0x7fe204162e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:01.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.368+0000 7fe203e00640 1 -- 172.21.15.86:0/2905159268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe204166780 msgr2=0x7fe204162e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:01.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.368+0000 7fe203e00640 1 --2- 172.21.15.86:0/2905159268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe204166780 0x7fe204162e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.368+0000 7fe203e00640 1 -- 172.21.15.86:0/2905159268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe204071cd0 msgr2=0x7fe204166070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:01.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.368+0000 7fe203e00640 1 --2- 172.21.15.86:0/2905159268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe204071cd0 0x7fe204166070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.368+0000 7fe203e00640 1 -- 172.21.15.86:0/2905159268 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe204155450 con 0x7fe204162aa0 2024-04-20T21:18:01.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.369+0000 7fe203e00640 1 --2- 172.21.15.86:0/2905159268 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe204162aa0 0x7fe204155730 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe1f002c070 tx=0x7fe1f00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:01.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.369+0000 7faeee600640 1 Processor -- start 2024-04-20T21:18:01.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.370+0000 7fe200c00640 1 -- 172.21.15.86:0/2905159268 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f0047070 con 0x7fe204162aa0 2024-04-20T21:18:01.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.370+0000 7fe200c00640 1 -- 172.21.15.86:0/2905159268 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1f0037de0 con 0x7fe204162aa0 2024-04-20T21:18:01.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.370+0000 7fe200c00640 1 -- 172.21.15.86:0/2905159268 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f0003830 con 0x7fe204162aa0 2024-04-20T21:18:01.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.370+0000 7fe20b600640 1 -- 172.21.15.86:0/2905159268 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2041596c0 con 0x7fe204162aa0 2024-04-20T21:18:01.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.372+0000 7fe20b600640 1 -- 172.21.15.86:0/2905159268 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe204159c80 con 0x7fe204162aa0 2024-04-20T21:18:01.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.370+0000 7faeee600640 1 -- start start 2024-04-20T21:18:01.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.372+0000 7faeee600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faee80d0d80 0x7faee8158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:01.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.372+0000 7faeee600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faee80d4fa0 0x7faee80ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:01.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.372+0000 7faeee600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faee815e8d0 0x7faee806d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:01.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.372+0000 7faeee600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faee80d5430 con 0x7faee815e8d0 2024-04-20T21:18:01.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.372+0000 7faeee600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7faee80d2620 con 0x7faee80d0d80 2024-04-20T21:18:01.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.372+0000 7faeee600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7faee815ecb0 con 0x7faee80d4fa0 2024-04-20T21:18:01.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.374+0000 7faeedc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faee815e8d0 0x7faee806d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:01.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.374+0000 7fe20b600640 1 -- 172.21.15.86:0/2905159268 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe204159e80 con 0x7fe204162aa0 2024-04-20T21:18:01.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.374+0000 7faeedc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faee815e8d0 0x7faee806d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58064/0 (socket says 172.21.15.86:58064) 2024-04-20T21:18:01.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.374+0000 7faeedc00640 1 -- 172.21.15.86:0/2520469054 learned_addr learned my addr 172.21.15.86:0/2520469054 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:01.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.375+0000 7faee7e00640 1 --2- 172.21.15.86:0/2520469054 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faee80d4fa0 0x7faee80ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:01.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.375+0000 7faeedc00640 1 -- 172.21.15.86:0/2520469054 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faee80d4fa0 msgr2=0x7faee80ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:01.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.375+0000 7faeedc00640 1 --2- 172.21.15.86:0/2520469054 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faee80d4fa0 0x7faee80ca7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.375+0000 7faeedc00640 1 -- 172.21.15.86:0/2520469054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faee80d0d80 msgr2=0x7faee8158270 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T21:18:01.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.375+0000 7faeedc00640 1 --2- 172.21.15.86:0/2520469054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faee80d0d80 0x7faee8158270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.375+0000 7faeedc00640 1 -- 172.21.15.86:0/2520469054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faee80b95d0 con 0x7faee815e8d0 2024-04-20T21:18:01.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.376+0000 7faeedc00640 1 --2- 172.21.15.86:0/2520469054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faee815e8d0 0x7faee806d240 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7faee0009e60 tx=0x7faee000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:01.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.377+0000 7faee4c00640 1 -- 172.21.15.86:0/2520469054 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee0018030 con 0x7faee815e8d0 2024-04-20T21:18:01.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.377+0000 7faee4c00640 1 -- 172.21.15.86:0/2520469054 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faee000cdf0 con 0x7faee815e8d0 2024-04-20T21:18:01.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.377+0000 7faee4c00640 1 -- 172.21.15.86:0/2520469054 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee0003bb0 con 0x7faee815e8d0 2024-04-20T21:18:01.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.377+0000 7faeee600640 1 -- 172.21.15.86:0/2520469054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faee80b9870 con 0x7faee815e8d0 2024-04-20T21:18:01.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.378+0000 7faeee600640 1 -- 172.21.15.86:0/2520469054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faee80caeb0 con 0x7faee815e8d0 2024-04-20T21:18:01.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.381+0000 7faee4c00640 1 -- 172.21.15.86:0/2520469054 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faee0005d10 con 0x7faee815e8d0 2024-04-20T21:18:01.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.382+0000 7faeee600640 1 -- 172.21.15.86:0/2520469054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faee804fcf0 con 0x7faee815e8d0 2024-04-20T21:18:01.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.383+0000 7faee4c00640 1 --2- 172.21.15.86:0/2520469054 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7faec806c940 0x7faec806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:01.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.383+0000 7faee4c00640 1 -- 172.21.15.86:0/2520469054 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7faee008c500 con 0x7faee815e8d0 2024-04-20T21:18:01.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.385+0000 7fe200c00640 1 -- 172.21.15.86:0/2905159268 <== mon.2 v2:172.21.15.144:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe1f004a070 con 0x7fe204162aa0 2024-04-20T21:18:01.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.385+0000 7fe200c00640 1 --2- 172.21.15.86:0/2905159268 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe1e406c9c0 0x7fe1e406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:01.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.386+0000 7faeed200640 1 --2- 172.21.15.86:0/2520469054 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7faec806c940 0x7faec806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:01.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.386+0000 7fe200c00640 1 -- 172.21.15.86:0/2905159268 <== mon.2 v2:172.21.15.144:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe1f00ad7a0 con 0x7fe204162aa0 2024-04-20T21:18:01.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.390+0000 7fe208c00640 1 --2- 172.21.15.86:0/2905159268 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe1e406c9c0 0x7fe1e406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:01.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.391+0000 7faeed200640 1 --2- 172.21.15.86:0/2520469054 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7faec806c940 0x7faec806ee00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7faedc036000 tx=0x7faedc008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:01.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.394+0000 7faee4c00640 1 -- 172.21.15.86:0/2520469054 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7faee0022070 con 0x7faee815e8d0 2024-04-20T21:18:01.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.391+0000 7fe208c00640 1 --2- 172.21.15.86:0/2905159268 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe1e406c9c0 0x7fe1e406ee80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8006fd0 tx=0x7fe1f8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:01.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.393+0000 7fe200c00640 1 -- 172.21.15.86:0/2905159268 <== mon.2 v2:172.21.15.144:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe204159e80 con 0x7fe204162aa0 2024-04-20T21:18:01.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.409+0000 7f177e000640 1 Processor -- start 2024-04-20T21:18:01.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.409+0000 7f177e000640 1 -- start start 2024-04-20T21:18:01.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.411+0000 7f177e000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f17780d0ee0 0x7f17780d1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:01.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.411+0000 7f177e000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17780d3100 con 0x7f17780d25a0 2024-04-20T21:18:01.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.411+0000 7f177e000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17780d33b0 con 0x7f177815e690 2024-04-20T21:18:01.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.411+0000 7f177e000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f17780d18b0 con 0x7f17780d0ee0 2024-04-20T21:18:01.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.412+0000 7f177cc00640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f177815e690 0x7f177806ce20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.86:36756/0 (socket says 172.21.15.86:36756) 2024-04-20T21:18:01.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.412+0000 7f177cc00640 1 -- 172.21.15.86:0/474200631 learned_addr learned my addr 172.21.15.86:0/474200631 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:18:01.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.412+0000 7f1777e00640 1 --2- 172.21.15.86:0/474200631 >> v2:172.21.15.86:3300/0 conn(0x7f17780d0ee0 0x7f17780d1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:01.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.414+0000 7f1777400640 1 -- 172.21.15.86:0/474200631 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1106162554 0 0) 0x7f17780d3100 con 0x7f17780d25a0 2024-04-20T21:18:01.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.414+0000 7f1777400640 1 -- 172.21.15.86:0/474200631 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1760003520 con 0x7f17780d25a0 2024-04-20T21:18:01.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.414+0000 7f1777400640 1 -- 172.21.15.86:0/474200631 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3533366819 0 0) 0x7f17780d33b0 con 0x7f177815e690 2024-04-20T21:18:01.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.414+0000 7f1777400640 1 -- 172.21.15.86:0/474200631 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f17780d3100 con 0x7f177815e690 2024-04-20T21:18:01.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.414+0000 7f1777e00640 1 -- 172.21.15.86:0/474200631 >> v1:172.21.15.144:6789/0 conn(0x7f177815e690 legacy=0x7f177806ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:01.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.414+0000 7f1777e00640 1 -- 172.21.15.86:0/474200631 >> v1:172.21.15.86:6789/0 conn(0x7f17780d25a0 legacy=0x7f17780d29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:01.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.414+0000 7f1777e00640 1 -- 172.21.15.86:0/474200631 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17780d3690 con 0x7f17780d0ee0 2024-04-20T21:18:01.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.416+0000 7f1777e00640 1 --2- 172.21.15.86:0/474200631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17780d0ee0 0x7f17780d1370 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f176c009a00 tx=0x7f176c02bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1e2b2ae8cfa494f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:01.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.418+0000 7f1777400640 1 -- 172.21.15.86:0/474200631 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176c02d700 con 0x7f17780d0ee0 2024-04-20T21:18:01.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.418+0000 7f1777400640 1 -- 172.21.15.86:0/474200631 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f176c02d8a0 con 0x7f17780d0ee0 2024-04-20T21:18:01.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.418+0000 7f1777400640 1 -- 172.21.15.86:0/474200631 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176c02dcb0 con 0x7f17780d0ee0 2024-04-20T21:18:01.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.419+0000 7f177e000640 1 -- 172.21.15.86:0/474200631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17780d0ee0 msgr2=0x7f17780d1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:01.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.419+0000 7f177e000640 1 --2- 172.21.15.86:0/474200631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17780d0ee0 0x7f17780d1370 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f176c009a00 tx=0x7f176c02bb50 comp rx=0 tx=0).stop 2024-04-20T21:18:01.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.422+0000 7f177e000640 1 -- 172.21.15.86:0/474200631 shutdown_connections 2024-04-20T21:18:01.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.422+0000 7f177e000640 1 --2- 172.21.15.86:0/474200631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f17780d0ee0 0x7f17780d1370 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.422+0000 7f177e000640 1 -- 172.21.15.86:0/474200631 >> 172.21.15.86:0/474200631 conn(0x7f177809d8a0 msgr2=0x7f177808bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:01.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.422+0000 7f177e000640 1 -- 172.21.15.86:0/474200631 shutdown_connections 2024-04-20T21:18:01.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.422+0000 7f177e000640 1 -- 172.21.15.86:0/474200631 wait complete. 2024-04-20T21:18:01.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.425+0000 7f177e000640 1 Processor -- start 2024-04-20T21:18:01.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.425+0000 7f177e000640 1 -- start start 2024-04-20T21:18:01.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.427+0000 7f177e000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17780d0ee0 0x7f177815efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:01.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.427+0000 7f177e000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f17780d25a0 0x7f17780b1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:01.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.427+0000 7f177e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f177815e690 0x7f1778145140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:01.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.427+0000 7f177d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f177815e690 0x7f1778145140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:01.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.427+0000 7f177cc00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17780d0ee0 0x7f177815efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:01.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.428+0000 7f177cc00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17780d0ee0 0x7f177815efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.86:58272/0 (socket says 172.21.15.86:58272) 2024-04-20T21:18:01.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.428+0000 7f177d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f177815e690 0x7f1778145140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58094/0 (socket says 172.21.15.86:58094) 2024-04-20T21:18:01.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.428+0000 7f1777e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f17780d25a0 0x7f17780b1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:01.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.428+0000 7f177e000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f17780cef00 con 0x7f177815e690 2024-04-20T21:18:01.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.428+0000 7f177e000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f17780cec20 con 0x7f17780d0ee0 2024-04-20T21:18:01.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.428+0000 7f177e000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f177815ea70 con 0x7f17780d25a0 2024-04-20T21:18:01.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.428+0000 7f177cc00640 1 -- 172.21.15.86:0/2229541422 learned_addr learned my addr 172.21.15.86:0/2229541422 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:01.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.429+0000 7f177cc00640 1 -- 172.21.15.86:0/2229541422 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f17780d25a0 msgr2=0x7f17780b1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:01.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.429+0000 7f177cc00640 1 --2- 172.21.15.86:0/2229541422 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f17780d25a0 0x7f17780b1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.429+0000 7f177cc00640 1 -- 172.21.15.86:0/2229541422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f177815e690 msgr2=0x7f1778145140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:01.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.429+0000 7f177cc00640 1 --2- 172.21.15.86:0/2229541422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f177815e690 0x7f1778145140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.429+0000 7f177cc00640 1 -- 172.21.15.86:0/2229541422 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17780b1b50 con 0x7f17780d0ee0 2024-04-20T21:18:01.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.429+0000 7f177cc00640 1 --2- 172.21.15.86:0/2229541422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17780d0ee0 0x7f177815efe0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f176800c410 tx=0x7f176800c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:01.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.430+0000 7f1774c00640 1 -- 172.21.15.86:0/2229541422 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17680074e0 con 0x7f17780d0ee0 2024-04-20T21:18:01.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.430+0000 7f177e000640 1 -- 172.21.15.86:0/2229541422 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17780b1e30 con 0x7f17780d0ee0 2024-04-20T21:18:01.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.430+0000 7f177e000640 1 -- 172.21.15.86:0/2229541422 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17780b23f0 con 0x7f17780d0ee0 2024-04-20T21:18:01.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.431+0000 7f1774c00640 1 -- 172.21.15.86:0/2229541422 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17680047e0 con 0x7f17780d0ee0 2024-04-20T21:18:01.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.432+0000 7f1774c00640 1 -- 172.21.15.86:0/2229541422 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17680074e0 con 0x7f17780d0ee0 2024-04-20T21:18:01.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.433+0000 7f177e000640 1 -- 172.21.15.86:0/2229541422 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1754005290 con 0x7f17780d0ee0 2024-04-20T21:18:01.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.436+0000 7f1774c00640 1 -- 172.21.15.86:0/2229541422 <== mon.1 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1768011070 con 0x7f17780d0ee0 2024-04-20T21:18:01.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.438+0000 7f1774c00640 1 --2- 172.21.15.86:0/2229541422 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f175c06ccf0 0x7f175c06f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:01.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.438+0000 7f1774c00640 1 -- 172.21.15.86:0/2229541422 <== mon.1 v2:172.21.15.144:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f17680072e0 con 0x7f17780d0ee0 2024-04-20T21:18:01.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.439+0000 7f1777e00640 1 --2- 172.21.15.86:0/2229541422 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f175c06ccf0 0x7f175c06f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:01.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.440+0000 7f1777e00640 1 --2- 172.21.15.86:0/2229541422 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f175c06ccf0 0x7f175c06f1b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f176c036000 tx=0x7f176c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:01.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.451+0000 7f1774c00640 1 -- 172.21.15.86:0/2229541422 <== mon.1 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1768059b40 con 0x7f17780d0ee0 2024-04-20T21:18:01.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.692+0000 7f3697400640 1 -- 172.21.15.86:0/2003620636 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f3680005580 con 0x7f36a40d96e0 2024-04-20T21:18:01.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.694+0000 7f36a2000640 1 -- 172.21.15.86:0/2003620636 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f36ac0e56e0 con 0x7f36a40d96e0 2024-04-20T21:18:01.694 INFO:teuthology.orchestra.run.smithi086.stdout:55834574855 2024-04-20T21:18:01.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.697+0000 7f3697400640 1 -- 172.21.15.86:0/2003620636 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f368c06c9f0 msgr2=0x7f368c06eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:01.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.697+0000 7f3697400640 1 --2- 172.21.15.86:0/2003620636 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f368c06c9f0 0x7f368c06eeb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3698035000 tx=0x7f3698030000 comp rx=0 tx=0).stop 2024-04-20T21:18:01.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.698+0000 7f3697400640 1 -- 172.21.15.86:0/2003620636 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36a40d96e0 msgr2=0x7f36a40c3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:01.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.698+0000 7f3697400640 1 --2- 172.21.15.86:0/2003620636 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36a40d96e0 0x7f36a40c3f20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f36ac076a30 tx=0x7f36ac0694c0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.698+0000 7f3697400640 1 -- 172.21.15.86:0/2003620636 shutdown_connections 2024-04-20T21:18:01.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.698+0000 7f3697400640 1 --2- 172.21.15.86:0/2003620636 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f368c06c9f0 0x7f368c06eeb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.698+0000 7f3697400640 1 --2- 172.21.15.86:0/2003620636 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36a40d96e0 0x7f36a40c3f20 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.698+0000 7f3697400640 1 --2- 172.21.15.86:0/2003620636 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36a40483a0 0x7f36a40c39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.699+0000 7f3697400640 1 --2- 172.21.15.86:0/2003620636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3690003cb0 0x7f36a4047570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.699+0000 7f3697400640 1 -- 172.21.15.86:0/2003620636 >> 172.21.15.86:0/2003620636 conn(0x7f36a4009fe0 msgr2=0x7f36a4007280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:01.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.699+0000 7f3697400640 1 -- 172.21.15.86:0/2003620636 shutdown_connections 2024-04-20T21:18:01.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.699+0000 7f3697400640 1 -- 172.21.15.86:0/2003620636 wait complete. 2024-04-20T21:18:01.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.708+0000 7f71c2600640 1 -- 172.21.15.86:0/962375563 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f71bc04fcf0 con 0x7f71bc15e8d0 2024-04-20T21:18:01.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.709+0000 7f71b8c00640 1 -- 172.21.15.86:0/962375563 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f71a807bef0 con 0x7f71bc15e8d0 2024-04-20T21:18:01.712 INFO:teuthology.orchestra.run.smithi086.stdout:55834574854 2024-04-20T21:18:01.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.714+0000 7f71c2600640 1 -- 172.21.15.86:0/962375563 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f719c06c8f0 msgr2=0x7f719c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:01.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.714+0000 7f71c2600640 1 --2- 172.21.15.86:0/962375563 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f719c06c8f0 0x7f719c06edb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f71b0007c60 tx=0x7f71b0007bb0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.714+0000 7f71c2600640 1 -- 172.21.15.86:0/962375563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71bc15e8d0 msgr2=0x7f71bc0b9430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:01.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.714+0000 7f71c2600640 1 --2- 172.21.15.86:0/962375563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71bc15e8d0 0x7f71bc0b9430 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f71a8004000 tx=0x7f71a80046c0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.715+0000 7f71c2600640 1 -- 172.21.15.86:0/962375563 shutdown_connections 2024-04-20T21:18:01.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.715+0000 7f71c2600640 1 --2- 172.21.15.86:0/962375563 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f719c06c8f0 0x7f719c06edb0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.715+0000 7f71c2600640 1 --2- 172.21.15.86:0/962375563 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71bc0b9970 0x7f71bc0c0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.715+0000 7f71c2600640 1 --2- 172.21.15.86:0/962375563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71bc15e8d0 0x7f71bc0b9430 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.715+0000 7f71c2600640 1 --2- 172.21.15.86:0/962375563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71bc0cfc80 0x7f71bc150d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.715+0000 7f71c2600640 1 -- 172.21.15.86:0/962375563 >> 172.21.15.86:0/962375563 conn(0x7f71bc08bf70 msgr2=0x7f71bc0d04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:01.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.715+0000 7f71c2600640 1 -- 172.21.15.86:0/962375563 shutdown_connections 2024-04-20T21:18:01.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.715+0000 7f71c2600640 1 -- 172.21.15.86:0/962375563 wait complete. 2024-04-20T21:18:01.728 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.3 2024-04-20T21:18:01.728 DEBUG:teuthology.parallel:result is None 2024-04-20T21:18:01.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.740+0000 7fede2400640 1 -- 172.21.15.86:0/4170992164 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7feddc0689e0 con 0x7feddc06ef20 2024-04-20T21:18:01.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.741+0000 7fedd3400640 1 -- 172.21.15.86:0/4170992164 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fedc801f020 con 0x7feddc06ef20 2024-04-20T21:18:01.741 INFO:teuthology.orchestra.run.smithi086.stdout:55834574853 2024-04-20T21:18:01.742 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.6 2024-04-20T21:18:01.742 DEBUG:teuthology.parallel:result is None 2024-04-20T21:18:01.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.743+0000 7fede2400640 1 -- 172.21.15.86:0/4170992164 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fedbc06c940 msgr2=0x7fedbc06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:01.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.743+0000 7fede2400640 1 --2- 172.21.15.86:0/4170992164 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fedbc06c940 0x7fedbc06ee00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7feddc15cd10 tx=0x7fedd4009040 comp rx=0 tx=0).stop 2024-04-20T21:18:01.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.743+0000 7fede2400640 1 -- 172.21.15.86:0/4170992164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feddc06ef20 msgr2=0x7feddc0c5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:01.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.743+0000 7fede2400640 1 --2- 172.21.15.86:0/4170992164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feddc06ef20 0x7feddc0c5cb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fedc800b410 tx=0x7fedc800b8d0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.743+0000 7fede2400640 1 -- 172.21.15.86:0/4170992164 shutdown_connections 2024-04-20T21:18:01.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.743+0000 7fede2400640 1 --2- 172.21.15.86:0/4170992164 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fedbc06c940 0x7fedbc06ee00 secure :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7feddc15cd10 tx=0x7fedd4009040 comp rx=0 tx=0).stop 2024-04-20T21:18:01.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.743+0000 7fede2400640 1 --2- 172.21.15.86:0/4170992164 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feddc15ad50 0x7feddc0cf540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.743+0000 7fede2400640 1 --2- 172.21.15.86:0/4170992164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feddc0723e0 0x7feddc1542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.743+0000 7fede2400640 1 --2- 172.21.15.86:0/4170992164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feddc06ef20 0x7feddc0c5cb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.744+0000 7fede2400640 1 -- 172.21.15.86:0/4170992164 >> 172.21.15.86:0/4170992164 conn(0x7feddc06b980 msgr2=0x7feddc1590a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:01.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.744+0000 7fede2400640 1 -- 172.21.15.86:0/4170992164 shutdown_connections 2024-04-20T21:18:01.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.744+0000 7fede2400640 1 -- 172.21.15.86:0/4170992164 wait complete. 2024-04-20T21:18:01.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.754+0000 7f989ac00640 1 -- 172.21.15.86:0/2197057840 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f989404fcf0 con 0x7f9894155240 2024-04-20T21:18:01.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.755+0000 7f9891600640 1 -- 172.21.15.86:0/2197057840 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f988c059a80 con 0x7f9894155240 2024-04-20T21:18:01.757 INFO:teuthology.orchestra.run.smithi086.stdout:55834574854 2024-04-20T21:18:01.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.758+0000 7f989ac00640 1 -- 172.21.15.86:0/2197057840 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f987406c940 msgr2=0x7f987406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:01.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.758+0000 7f989ac00640 1 --2- 172.21.15.86:0/2197057840 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f987406c940 0x7f987406ee00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9888000f60 tx=0x7f9888009040 comp rx=0 tx=0).stop 2024-04-20T21:18:01.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.758+0000 7f989ac00640 1 -- 172.21.15.86:0/2197057840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9894155240 msgr2=0x7f98940b71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:01.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.758+0000 7f989ac00640 1 --2- 172.21.15.86:0/2197057840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9894155240 0x7f98940b71d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f988c00c730 tx=0x7f988c00cbf0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.758+0000 7f989ac00640 1 -- 172.21.15.86:0/2197057840 shutdown_connections 2024-04-20T21:18:01.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.758+0000 7f989ac00640 1 --2- 172.21.15.86:0/2197057840 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f987406c940 0x7f987406ee00 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.758+0000 7f989ac00640 1 --2- 172.21.15.86:0/2197057840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9894155240 0x7f98940b71d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.758+0000 7f989ac00640 1 --2- 172.21.15.86:0/2197057840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f989415e8d0 0x7f98940ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.758+0000 7f989ac00640 1 --2- 172.21.15.86:0/2197057840 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98940cfc80 0x7f98940ca980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.758+0000 7f989ac00640 1 -- 172.21.15.86:0/2197057840 >> 172.21.15.86:0/2197057840 conn(0x7f989408bf70 msgr2=0x7f98941618b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:01.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.758+0000 7f989ac00640 1 -- 172.21.15.86:0/2197057840 shutdown_connections 2024-04-20T21:18:01.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.759+0000 7f989ac00640 1 -- 172.21.15.86:0/2197057840 wait complete. 2024-04-20T21:18:01.770 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.7 2024-04-20T21:18:01.770 DEBUG:teuthology.parallel:result is None 2024-04-20T21:18:01.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.776+0000 7f758fe00640 1 -- 172.21.15.86:0/4039265863 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f758800df40 con 0x7f75880e30c0 2024-04-20T21:18:01.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.777+0000 7f7586a00640 1 -- 172.21.15.86:0/4039265863 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f757c0820d0 con 0x7f75880e30c0 2024-04-20T21:18:01.777 INFO:teuthology.orchestra.run.smithi086.stdout:55834574855 2024-04-20T21:18:01.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.779+0000 7f758fe00640 1 -- 172.21.15.86:0/4039265863 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f757006c9c0 msgr2=0x7f757006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:01.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.779+0000 7f758fe00640 1 --2- 172.21.15.86:0/4039265863 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f757006c9c0 0x7f757006ee80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7580001030 tx=0x7f7580007240 comp rx=0 tx=0).stop 2024-04-20T21:18:01.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.779+0000 7f758fe00640 1 -- 172.21.15.86:0/4039265863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75880e30c0 msgr2=0x7f7588177300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:01.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.779+0000 7f758fe00640 1 --2- 172.21.15.86:0/4039265863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75880e30c0 0x7f7588177300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7588056930 tx=0x7f757c0046c0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.779+0000 7f758fe00640 1 -- 172.21.15.86:0/4039265863 shutdown_connections 2024-04-20T21:18:01.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.779+0000 7f758fe00640 1 --2- 172.21.15.86:0/4039265863 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f757006c9c0 0x7f757006ee80 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.779+0000 7f758fe00640 1 --2- 172.21.15.86:0/4039265863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75880e30c0 0x7f7588177300 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.779+0000 7f758fe00640 1 --2- 172.21.15.86:0/4039265863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7588054130 0x7f758817ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.779+0000 7f758fe00640 1 --2- 172.21.15.86:0/4039265863 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f758803e840 0x7f7588052550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.779+0000 7f758fe00640 1 -- 172.21.15.86:0/4039265863 >> 172.21.15.86:0/4039265863 conn(0x7f75880559e0 msgr2=0x7f7588039cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:01.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.780+0000 7f758fe00640 1 -- 172.21.15.86:0/4039265863 shutdown_connections 2024-04-20T21:18:01.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.780+0000 7f758fe00640 1 -- 172.21.15.86:0/4039265863 wait complete. 2024-04-20T21:18:01.782 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2024-04-20T21:18:01.782 DEBUG:teuthology.parallel:result is None 2024-04-20T21:18:01.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.791+0000 7faeee600640 1 -- 172.21.15.86:0/2520469054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7faee80d1d30 con 0x7faee815e8d0 2024-04-20T21:18:01.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.791+0000 7faee4c00640 1 -- 172.21.15.86:0/2520469054 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7faee0010040 con 0x7faee815e8d0 2024-04-20T21:18:01.792 INFO:teuthology.orchestra.run.smithi086.stdout:55834574855 2024-04-20T21:18:01.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.794+0000 7faeee600640 1 -- 172.21.15.86:0/2520469054 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7faec806c940 msgr2=0x7faec806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:01.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.794+0000 7faeee600640 1 --2- 172.21.15.86:0/2520469054 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7faec806c940 0x7faec806ee00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7faedc036000 tx=0x7faedc008030 comp rx=0 tx=0).stop 2024-04-20T21:18:01.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.794+0000 7faeee600640 1 -- 172.21.15.86:0/2520469054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faee815e8d0 msgr2=0x7faee806d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:01.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.794+0000 7faeee600640 1 --2- 172.21.15.86:0/2520469054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faee815e8d0 0x7faee806d240 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7faee0009e60 tx=0x7faee000c490 comp rx=0 tx=0).stop 2024-04-20T21:18:01.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.794+0000 7faeee600640 1 -- 172.21.15.86:0/2520469054 shutdown_connections 2024-04-20T21:18:01.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.794+0000 7faeee600640 1 --2- 172.21.15.86:0/2520469054 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7faec806c940 0x7faec806ee00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.794+0000 7faeee600640 1 --2- 172.21.15.86:0/2520469054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faee815e8d0 0x7faee806d240 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.794+0000 7faeee600640 1 --2- 172.21.15.86:0/2520469054 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faee80d4fa0 0x7faee80ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.794+0000 7faeee600640 1 --2- 172.21.15.86:0/2520469054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faee80d0d80 0x7faee8158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.794+0000 7faeee600640 1 -- 172.21.15.86:0/2520469054 >> 172.21.15.86:0/2520469054 conn(0x7faee808bf70 msgr2=0x7faee80b1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:01.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.794+0000 7faeee600640 1 -- 172.21.15.86:0/2520469054 shutdown_connections 2024-04-20T21:18:01.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.794+0000 7faeee600640 1 -- 172.21.15.86:0/2520469054 wait complete. 2024-04-20T21:18:01.796 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.0 2024-04-20T21:18:01.796 DEBUG:teuthology.parallel:result is None 2024-04-20T21:18:01.810 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2024-04-20T21:18:01.810 DEBUG:teuthology.parallel:result is None 2024-04-20T21:18:01.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.825+0000 7fe20b600640 1 -- 172.21.15.86:0/2905159268 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fe2040689e0 con 0x7fe204162aa0 2024-04-20T21:18:01.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.826+0000 7fe200c00640 1 -- 172.21.15.86:0/2905159268 <== mon.2 v2:172.21.15.144:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe1f007bc40 con 0x7fe204162aa0 2024-04-20T21:18:01.826 INFO:teuthology.orchestra.run.smithi086.stdout:55834574855 2024-04-20T21:18:01.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.828+0000 7fe20b600640 1 -- 172.21.15.86:0/2905159268 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe1e406c9c0 msgr2=0x7fe1e406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:01.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.828+0000 7fe20b600640 1 --2- 172.21.15.86:0/2905159268 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe1e406c9c0 0x7fe1e406ee80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8006fd0 tx=0x7fe1f8008040 comp rx=0 tx=0).stop 2024-04-20T21:18:01.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.828+0000 7fe20b600640 1 -- 172.21.15.86:0/2905159268 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe204162aa0 msgr2=0x7fe204155730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:01.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.828+0000 7fe20b600640 1 --2- 172.21.15.86:0/2905159268 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe204162aa0 0x7fe204155730 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe1f002c070 tx=0x7fe1f00046c0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.829+0000 7fe20b600640 1 -- 172.21.15.86:0/2905159268 shutdown_connections 2024-04-20T21:18:01.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.829+0000 7fe20b600640 1 --2- 172.21.15.86:0/2905159268 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe1e406c9c0 0x7fe1e406ee80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.829+0000 7fe20b600640 1 --2- 172.21.15.86:0/2905159268 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe204166780 0x7fe204162e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.829+0000 7fe20b600640 1 --2- 172.21.15.86:0/2905159268 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe204162aa0 0x7fe204155730 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.829+0000 7fe20b600640 1 --2- 172.21.15.86:0/2905159268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe204071cd0 0x7fe204166070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.829+0000 7fe20b600640 1 -- 172.21.15.86:0/2905159268 >> 172.21.15.86:0/2905159268 conn(0x7fe20406b980 msgr2=0x7fe2040b28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:01.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.829+0000 7fe20b600640 1 -- 172.21.15.86:0/2905159268 shutdown_connections 2024-04-20T21:18:01.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.829+0000 7fe20b600640 1 -- 172.21.15.86:0/2905159268 wait complete. 2024-04-20T21:18:01.844 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.4 2024-04-20T21:18:01.844 DEBUG:teuthology.parallel:result is None 2024-04-20T21:18:01.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.854+0000 7f177e000640 1 -- 172.21.15.86:0/2229541422 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f1754005850 con 0x7f17780d0ee0 2024-04-20T21:18:01.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.855+0000 7f1774c00640 1 -- 172.21.15.86:0/2229541422 <== mon.1 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1768059440 con 0x7f17780d0ee0 2024-04-20T21:18:01.854 INFO:teuthology.orchestra.run.smithi086.stdout:55834574856 2024-04-20T21:18:01.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.857+0000 7f177e000640 1 -- 172.21.15.86:0/2229541422 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f175c06ccf0 msgr2=0x7f175c06f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:01.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.857+0000 7f177e000640 1 --2- 172.21.15.86:0/2229541422 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f175c06ccf0 0x7f175c06f1b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f176c036000 tx=0x7f176c008030 comp rx=0 tx=0).stop 2024-04-20T21:18:01.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.857+0000 7f177e000640 1 -- 172.21.15.86:0/2229541422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17780d0ee0 msgr2=0x7f177815efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:01.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.857+0000 7f177e000640 1 --2- 172.21.15.86:0/2229541422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17780d0ee0 0x7f177815efe0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f176800c410 tx=0x7f176800c8d0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.857+0000 7f177e000640 1 -- 172.21.15.86:0/2229541422 shutdown_connections 2024-04-20T21:18:01.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.857+0000 7f177e000640 1 --2- 172.21.15.86:0/2229541422 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f175c06ccf0 0x7f175c06f1b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.857+0000 7f177e000640 1 --2- 172.21.15.86:0/2229541422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f177815e690 0x7f1778145140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.857+0000 7f177e000640 1 --2- 172.21.15.86:0/2229541422 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f17780d25a0 0x7f17780b1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.857+0000 7f177e000640 1 --2- 172.21.15.86:0/2229541422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17780d0ee0 0x7f177815efe0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:01.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.857+0000 7f177e000640 1 -- 172.21.15.86:0/2229541422 >> 172.21.15.86:0/2229541422 conn(0x7f177809d8a0 msgr2=0x7f177815aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:01.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.857+0000 7f177e000640 1 -- 172.21.15.86:0/2229541422 shutdown_connections 2024-04-20T21:18:01.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:01.857+0000 7f177e000640 1 -- 172.21.15.86:0/2229541422 wait complete. 2024-04-20T21:18:01.873 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.5 2024-04-20T21:18:01.873 DEBUG:teuthology.parallel:result is None 2024-04-20T21:18:01.873 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-20T21:18:01.873 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T21:18:02.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.107+0000 7f7bfcc00640 1 Processor -- start 2024-04-20T21:18:02.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.107+0000 7f7bfcc00640 1 -- start start 2024-04-20T21:18:02.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.109+0000 7f7bfcc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7bf8161f30 0x7f7bf80cf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:02.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.109+0000 7f7bfcc00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7bf80d3a80 0x7f7bf80d3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:02.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.109+0000 7f7bfcc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7bf80d0e50 0x7f7bf80d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:02.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.109+0000 7f7bfcc00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f7bf806dc80 con 0x7f7bf8161f30 2024-04-20T21:18:02.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.109+0000 7f7bfcc00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f7bf806cdb0 con 0x7f7bf80d3a80 2024-04-20T21:18:02.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.109+0000 7f7bfcc00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7bf806d000 con 0x7f7bf80d0e50 2024-04-20T21:18:02.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.109+0000 7f7bf7400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7bf8161f30 0x7f7bf80cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:02.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.109+0000 7f7bf7400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7bf8161f30 0x7f7bf80cf000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58096/0 (socket says 172.21.15.86:58096) 2024-04-20T21:18:02.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.109+0000 7f7bf7400640 1 -- 172.21.15.86:0/2507446173 learned_addr learned my addr 172.21.15.86:0/2507446173 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:02.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.110+0000 7f7bf7400640 1 -- 172.21.15.86:0/2507446173 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7bf80d0e50 msgr2=0x7f7bf80d12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:02.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.110+0000 7f7bf7400640 1 --2- 172.21.15.86:0/2507446173 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7bf80d0e50 0x7f7bf80d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:02.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.110+0000 7f7bf7400640 1 -- 172.21.15.86:0/2507446173 >> v2:172.21.15.144:3300/0 conn(0x7f7bf80d3a80 msgr2=0x7f7bf80d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:02.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.110+0000 7f7bf7400640 1 --2- 172.21.15.86:0/2507446173 >> v2:172.21.15.144:3300/0 conn(0x7f7bf80d3a80 0x7f7bf80d3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:02.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.110+0000 7f7bf7400640 1 -- 172.21.15.86:0/2507446173 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bf806d2e0 con 0x7f7bf8161f30 2024-04-20T21:18:02.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.111+0000 7f7bf7400640 1 --2- 172.21.15.86:0/2507446173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bf8161f30 0x7f7bf80cf000 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7bf80ba640 tx=0x7f7be002b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf2f45ef74769707 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:02.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.111+0000 7f7bf6000640 1 -- 172.21.15.86:0/2507446173 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be0038070 con 0x7f7bf8161f30 2024-04-20T21:18:02.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.111+0000 7f7bf6000640 1 -- 172.21.15.86:0/2507446173 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7be002f050 con 0x7f7bf8161f30 2024-04-20T21:18:02.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.111+0000 7f7bf6000640 1 -- 172.21.15.86:0/2507446173 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be0031970 con 0x7f7bf8161f30 2024-04-20T21:18:02.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.112+0000 7f7bfcc00640 1 -- 172.21.15.86:0/2507446173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bf8161f30 msgr2=0x7f7bf80cf000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:02.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.112+0000 7f7bfcc00640 1 --2- 172.21.15.86:0/2507446173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bf8161f30 0x7f7bf80cf000 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7bf80ba640 tx=0x7f7be002b7f0 comp rx=0 tx=0).stop 2024-04-20T21:18:02.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.112+0000 7f7bfcc00640 1 -- 172.21.15.86:0/2507446173 shutdown_connections 2024-04-20T21:18:02.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.112+0000 7f7bfcc00640 1 --2- 172.21.15.86:0/2507446173 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7bf80d0e50 0x7f7bf80d12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:02.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.112+0000 7f7bfcc00640 1 --2- 172.21.15.86:0/2507446173 >> v2:172.21.15.144:3300/0 conn(0x7f7bf80d3a80 0x7f7bf80d3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:02.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.112+0000 7f7bfcc00640 1 --2- 172.21.15.86:0/2507446173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bf8161f30 0x7f7bf80cf000 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:02.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.112+0000 7f7bfcc00640 1 -- 172.21.15.86:0/2507446173 >> 172.21.15.86:0/2507446173 conn(0x7f7bf808bf70 msgr2=0x7f7bf80d4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:02.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.112+0000 7f7bfcc00640 1 -- 172.21.15.86:0/2507446173 shutdown_connections 2024-04-20T21:18:02.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.112+0000 7f7bfcc00640 1 -- 172.21.15.86:0/2507446173 wait complete. 2024-04-20T21:18:02.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.114+0000 7f7bfcc00640 1 Processor -- start 2024-04-20T21:18:02.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.114+0000 7f7bfcc00640 1 -- start start 2024-04-20T21:18:02.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.115+0000 7f7bfcc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7bf80d0e50 0x7f7bf8152d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:02.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.115+0000 7f7bfcc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bf80d3a80 0x7f7bf80c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:02.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.115+0000 7f7bfcc00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bf8161f30 0x7f7bf81f47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:02.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.115+0000 7f7bfcc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf8162310 con 0x7f7bf80d3a80 2024-04-20T21:18:02.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.115+0000 7f7bfcc00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf806d8a0 con 0x7f7bf8161f30 2024-04-20T21:18:02.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.115+0000 7f7bfcc00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7bf80cf540 con 0x7f7bf80d0e50 2024-04-20T21:18:02.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.116+0000 7f7bf6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bf80d3a80 0x7f7bf80c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:02.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.116+0000 7f7bf6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bf80d3a80 0x7f7bf80c92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58106/0 (socket says 172.21.15.86:58106) 2024-04-20T21:18:02.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.116+0000 7f7bf6a00640 1 -- 172.21.15.86:0/1478513566 learned_addr learned my addr 172.21.15.86:0/1478513566 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:02.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.116+0000 7f7bf7400640 1 --2- 172.21.15.86:0/1478513566 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7bf80d0e50 0x7f7bf8152d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:02.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.116+0000 7f7bf7e00640 1 --2- 172.21.15.86:0/1478513566 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bf8161f30 0x7f7bf81f47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:02.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.116+0000 7f7bf6a00640 1 -- 172.21.15.86:0/1478513566 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7bf80d0e50 msgr2=0x7f7bf8152d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:02.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.116+0000 7f7bf6a00640 1 --2- 172.21.15.86:0/1478513566 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7bf80d0e50 0x7f7bf8152d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:02.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.116+0000 7f7bf6a00640 1 -- 172.21.15.86:0/1478513566 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bf8161f30 msgr2=0x7f7bf81f47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:02.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.116+0000 7f7bf6a00640 1 --2- 172.21.15.86:0/1478513566 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bf8161f30 0x7f7bf81f47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:02.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.116+0000 7f7bf6a00640 1 -- 172.21.15.86:0/1478513566 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bf80cc8f0 con 0x7f7bf80d3a80 2024-04-20T21:18:02.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.116+0000 7f7bf7e00640 1 --2- 172.21.15.86:0/1478513566 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bf8161f30 0x7f7bf81f47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:02.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.116+0000 7f7bf6a00640 1 --2- 172.21.15.86:0/1478513566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bf80d3a80 0x7f7bf80c92c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7be800b410 tx=0x7f7be800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:02.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.117+0000 7f7bef400640 1 -- 172.21.15.86:0/1478513566 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be8015020 con 0x7f7bf80d3a80 2024-04-20T21:18:02.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.117+0000 7f7bef400640 1 -- 172.21.15.86:0/1478513566 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7be8011850 con 0x7f7bf80d3a80 2024-04-20T21:18:02.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.117+0000 7f7bef400640 1 -- 172.21.15.86:0/1478513566 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be8011c40 con 0x7f7bf80d3a80 2024-04-20T21:18:02.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.117+0000 7f7bfcc00640 1 -- 172.21.15.86:0/1478513566 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bf81f5000 con 0x7f7bf80d3a80 2024-04-20T21:18:02.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.117+0000 7f7bfcc00640 1 -- 172.21.15.86:0/1478513566 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bf81f55c0 con 0x7f7bf80d3a80 2024-04-20T21:18:02.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.118+0000 7f7bfcc00640 1 -- 172.21.15.86:0/1478513566 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bf804fcf0 con 0x7f7bf80d3a80 2024-04-20T21:18:02.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.119+0000 7f7bef400640 1 -- 172.21.15.86:0/1478513566 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7be8018020 con 0x7f7bf80d3a80 2024-04-20T21:18:02.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.119+0000 7f7bef400640 1 --2- 172.21.15.86:0/1478513566 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7bd406c8f0 0x7f7bd406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:02.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.120+0000 7f7bef400640 1 -- 172.21.15.86:0/1478513566 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7be808bb10 con 0x7f7bf80d3a80 2024-04-20T21:18:02.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.120+0000 7f7bf7400640 1 --2- 172.21.15.86:0/1478513566 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7bd406c8f0 0x7f7bd406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:02.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.121+0000 7f7bf7400640 1 --2- 172.21.15.86:0/1478513566 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7bd406c8f0 0x7f7bd406edb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7be0035000 tx=0x7f7be0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:02.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.124+0000 7f7bef400640 1 -- 172.21.15.86:0/1478513566 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7be801f020 con 0x7f7bf80d3a80 2024-04-20T21:18:02.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.307+0000 7f7bfcc00640 1 -- 172.21.15.86:0/1478513566 --> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7bf808c4c0 con 0x7f7bd406c8f0 2024-04-20T21:18:02.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.326+0000 7f7bef400640 1 -- 172.21.15.86:0/1478513566 <== mgr.4107 v2:172.21.15.86:6832/302045240 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7f7bf808c4c0 con 0x7f7bd406c8f0 2024-04-20T21:18:02.328 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:18:02.329 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-20T21:18:02.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.330+0000 7f7bfcc00640 1 -- 172.21.15.86:0/1478513566 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7bd406c8f0 msgr2=0x7f7bd406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:02.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.330+0000 7f7bfcc00640 1 --2- 172.21.15.86:0/1478513566 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7bd406c8f0 0x7f7bd406edb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7be0035000 tx=0x7f7be0008030 comp rx=0 tx=0).stop 2024-04-20T21:18:02.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.330+0000 7f7bfcc00640 1 -- 172.21.15.86:0/1478513566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bf80d3a80 msgr2=0x7f7bf80c92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:02.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.330+0000 7f7bfcc00640 1 --2- 172.21.15.86:0/1478513566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bf80d3a80 0x7f7bf80c92c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7be800b410 tx=0x7f7be800b8d0 comp rx=0 tx=0).stop 2024-04-20T21:18:02.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.330+0000 7f7bfcc00640 1 -- 172.21.15.86:0/1478513566 shutdown_connections 2024-04-20T21:18:02.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.330+0000 7f7bfcc00640 1 --2- 172.21.15.86:0/1478513566 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7bd406c8f0 0x7f7bd406edb0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:02.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.330+0000 7f7bfcc00640 1 --2- 172.21.15.86:0/1478513566 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7bf8161f30 0x7f7bf81f47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:02.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.330+0000 7f7bfcc00640 1 --2- 172.21.15.86:0/1478513566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bf80d3a80 0x7f7bf80c92c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:02.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.330+0000 7f7bfcc00640 1 --2- 172.21.15.86:0/1478513566 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7bf80d0e50 0x7f7bf8152d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:02.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.330+0000 7f7bfcc00640 1 -- 172.21.15.86:0/1478513566 >> 172.21.15.86:0/1478513566 conn(0x7f7bf808bf70 msgr2=0x7f7bf80b33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:02.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.330+0000 7f7bfcc00640 1 -- 172.21.15.86:0/1478513566 shutdown_connections 2024-04-20T21:18:02.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.330+0000 7f7bfcc00640 1 -- 172.21.15.86:0/1478513566 wait complete. 2024-04-20T21:18:02.346 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-04-20T21:18:00.633002+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219000,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755720,"statfs":{"total":773094113280,"available":772869857280,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.372710"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.607719+0000","last_change":"2024-04-20T21:17:52.309045+0000","last_active":"2024-04-20T21:17:53.607719+0000","last_peered":"2024-04-20T21:17:53.607719+0000","last_clean":"2024-04-20T21:17:53.607719+0000","last_became_active":"2024-04-20T21:17:52.308808+0000","last_became_peered":"2024-04-20T21:17:52.308808+0000","last_unstale":"2024-04-20T21:17:53.607719+0000","last_undegraded":"2024-04-20T21:17:53.607719+0000","last_fullsized":"2024-04-20T21:17:53.607719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:15:54.762913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.281640+0000","last_change":"2024-04-20T21:17:52.303716+0000","last_active":"2024-04-20T21:17:53.281640+0000","last_peered":"2024-04-20T21:17:53.281640+0000","last_clean":"2024-04-20T21:17:53.281640+0000","last_became_active":"2024-04-20T21:17:52.303460+0000","last_became_peered":"2024-04-20T21:17:52.303460+0000","last_unstale":"2024-04-20T21:17:53.281640+0000","last_undegraded":"2024-04-20T21:17:53.281640+0000","last_fullsized":"2024-04-20T21:17:53.281640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:34:47.842666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.684324+0000","last_change":"2024-04-20T21:17:52.297807+0000","last_active":"2024-04-20T21:17:54.684324+0000","last_peered":"2024-04-20T21:17:54.684324+0000","last_clean":"2024-04-20T21:17:54.684324+0000","last_became_active":"2024-04-20T21:17:52.296178+0000","last_became_peered":"2024-04-20T21:17:52.296178+0000","last_unstale":"2024-04-20T21:17:54.684324+0000","last_undegraded":"2024-04-20T21:17:54.684324+0000","last_fullsized":"2024-04-20T21:17:54.684324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:42:57.243810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.357862+0000","last_change":"2024-04-20T21:17:52.315284+0000","last_active":"2024-04-20T21:17:54.357862+0000","last_peered":"2024-04-20T21:17:54.357862+0000","last_clean":"2024-04-20T21:17:54.357862+0000","last_became_active":"2024-04-20T21:17:52.314657+0000","last_became_peered":"2024-04-20T21:17:52.314657+0000","last_unstale":"2024-04-20T21:17:54.357862+0000","last_undegraded":"2024-04-20T21:17:54.357862+0000","last_fullsized":"2024-04-20T21:17:54.357862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:52:22.418925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.606807+0000","last_change":"2024-04-20T21:17:52.299456+0000","last_active":"2024-04-20T21:17:53.606807+0000","last_peered":"2024-04-20T21:17:53.606807+0000","last_clean":"2024-04-20T21:17:53.606807+0000","last_became_active":"2024-04-20T21:17:52.299016+0000","last_became_peered":"2024-04-20T21:17:52.299016+0000","last_unstale":"2024-04-20T21:17:53.606807+0000","last_undegraded":"2024-04-20T21:17:53.606807+0000","last_fullsized":"2024-04-20T21:17:53.606807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:04:26.111984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.357200+0000","last_change":"2024-04-20T21:17:52.305054+0000","last_active":"2024-04-20T21:17:54.357200+0000","last_peered":"2024-04-20T21:17:54.357200+0000","last_clean":"2024-04-20T21:17:54.357200+0000","last_became_active":"2024-04-20T21:17:52.303307+0000","last_became_peered":"2024-04-20T21:17:52.303307+0000","last_unstale":"2024-04-20T21:17:54.357200+0000","last_undegraded":"2024-04-20T21:17:54.357200+0000","last_fullsized":"2024-04-20T21:17:54.357200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:34:47.842666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.606631+0000","last_change":"2024-04-20T21:17:52.306121+0000","last_active":"2024-04-20T21:17:53.606631+0000","last_peered":"2024-04-20T21:17:53.606631+0000","last_clean":"2024-04-20T21:17:53.606631+0000","last_became_active":"2024-04-20T21:17:52.305749+0000","last_became_peered":"2024-04-20T21:17:52.305749+0000","last_unstale":"2024-04-20T21:17:53.606631+0000","last_undegraded":"2024-04-20T21:17:53.606631+0000","last_fullsized":"2024-04-20T21:17:53.606631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:10:11.436747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.682205+0000","last_change":"2024-04-20T21:17:52.292183+0000","last_active":"2024-04-20T21:17:54.682205+0000","last_peered":"2024-04-20T21:17:54.682205+0000","last_clean":"2024-04-20T21:17:54.682205+0000","last_became_active":"2024-04-20T21:17:52.291906+0000","last_became_peered":"2024-04-20T21:17:52.291906+0000","last_unstale":"2024-04-20T21:17:54.682205+0000","last_undegraded":"2024-04-20T21:17:54.682205+0000","last_fullsized":"2024-04-20T21:17:54.682205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:42:57.243810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.607755+0000","last_change":"2024-04-20T21:17:52.316938+0000","last_active":"2024-04-20T21:17:53.607755+0000","last_peered":"2024-04-20T21:17:53.607755+0000","last_clean":"2024-04-20T21:17:53.607755+0000","last_became_active":"2024-04-20T21:17:52.316386+0000","last_became_peered":"2024-04-20T21:17:52.316386+0000","last_unstale":"2024-04-20T21:17:53.607755+0000","last_undegraded":"2024-04-20T21:17:53.607755+0000","last_fullsized":"2024-04-20T21:17:53.607755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:23:58.112035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.289257+0000","last_change":"2024-04-20T21:17:52.299679+0000","last_active":"2024-04-20T21:17:53.289257+0000","last_peered":"2024-04-20T21:17:53.289257+0000","last_clean":"2024-04-20T21:17:53.289257+0000","last_became_active":"2024-04-20T21:17:52.299325+0000","last_became_peered":"2024-04-20T21:17:52.299325+0000","last_unstale":"2024-04-20T21:17:53.289257+0000","last_undegraded":"2024-04-20T21:17:53.289257+0000","last_fullsized":"2024-04-20T21:17:53.289257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:42:57.243810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.289696+0000","last_change":"2024-04-20T21:17:52.298416+0000","last_active":"2024-04-20T21:17:53.289696+0000","last_peered":"2024-04-20T21:17:53.289696+0000","last_clean":"2024-04-20T21:17:53.289696+0000","last_became_active":"2024-04-20T21:17:52.297990+0000","last_became_peered":"2024-04-20T21:17:52.297990+0000","last_unstale":"2024-04-20T21:17:53.289696+0000","last_undegraded":"2024-04-20T21:17:53.289696+0000","last_fullsized":"2024-04-20T21:17:53.289696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:11:50.878621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.287223+0000","last_change":"2024-04-20T21:17:52.279605+0000","last_active":"2024-04-20T21:17:53.287223+0000","last_peered":"2024-04-20T21:17:53.287223+0000","last_clean":"2024-04-20T21:17:53.287223+0000","last_became_active":"2024-04-20T21:17:52.277631+0000","last_became_peered":"2024-04-20T21:17:52.277631+0000","last_unstale":"2024-04-20T21:17:53.287223+0000","last_undegraded":"2024-04-20T21:17:53.287223+0000","last_fullsized":"2024-04-20T21:17:53.287223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:29:18.971384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.680792+0000","last_change":"2024-04-20T21:17:52.300679+0000","last_active":"2024-04-20T21:17:54.680792+0000","last_peered":"2024-04-20T21:17:54.680792+0000","last_clean":"2024-04-20T21:17:54.680792+0000","last_became_active":"2024-04-20T21:17:52.300387+0000","last_became_peered":"2024-04-20T21:17:52.300387+0000","last_unstale":"2024-04-20T21:17:54.680792+0000","last_undegraded":"2024-04-20T21:17:54.680792+0000","last_fullsized":"2024-04-20T21:17:54.680792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:09:02.488057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.682763+0000","last_change":"2024-04-20T21:17:52.301182+0000","last_active":"2024-04-20T21:17:54.682763+0000","last_peered":"2024-04-20T21:17:54.682763+0000","last_clean":"2024-04-20T21:17:54.682763+0000","last_became_active":"2024-04-20T21:17:52.300852+0000","last_became_peered":"2024-04-20T21:17:52.300852+0000","last_unstale":"2024-04-20T21:17:54.682763+0000","last_undegraded":"2024-04-20T21:17:54.682763+0000","last_fullsized":"2024-04-20T21:17:54.682763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:38:56.558892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.281082+0000","last_change":"2024-04-20T21:17:52.282408+0000","last_active":"2024-04-20T21:17:53.281082+0000","last_peered":"2024-04-20T21:17:53.281082+0000","last_clean":"2024-04-20T21:17:53.281082+0000","last_became_active":"2024-04-20T21:17:52.282044+0000","last_became_peered":"2024-04-20T21:17:52.282044+0000","last_unstale":"2024-04-20T21:17:53.281082+0000","last_undegraded":"2024-04-20T21:17:53.281082+0000","last_fullsized":"2024-04-20T21:17:53.281082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:27:15.675834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.357121+0000","last_change":"2024-04-20T21:17:52.315912+0000","last_active":"2024-04-20T21:17:54.357121+0000","last_peered":"2024-04-20T21:17:54.357121+0000","last_clean":"2024-04-20T21:17:54.357121+0000","last_became_active":"2024-04-20T21:17:52.315642+0000","last_became_peered":"2024-04-20T21:17:52.315642+0000","last_unstale":"2024-04-20T21:17:54.357121+0000","last_undegraded":"2024-04-20T21:17:54.357121+0000","last_fullsized":"2024-04-20T21:17:54.357121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:11:50.878621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.358578+0000","last_change":"2024-04-20T21:17:52.315150+0000","last_active":"2024-04-20T21:17:54.358578+0000","last_peered":"2024-04-20T21:17:54.358578+0000","last_clean":"2024-04-20T21:17:54.358578+0000","last_became_active":"2024-04-20T21:17:52.314856+0000","last_became_peered":"2024-04-20T21:17:52.314856+0000","last_unstale":"2024-04-20T21:17:54.358578+0000","last_undegraded":"2024-04-20T21:17:54.358578+0000","last_fullsized":"2024-04-20T21:17:54.358578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:15:54.762913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.679762+0000","last_change":"2024-04-20T21:17:52.286233+0000","last_active":"2024-04-20T21:17:54.679762+0000","last_peered":"2024-04-20T21:17:54.679762+0000","last_clean":"2024-04-20T21:17:54.679762+0000","last_became_active":"2024-04-20T21:17:52.285872+0000","last_became_peered":"2024-04-20T21:17:52.285872+0000","last_unstale":"2024-04-20T21:17:54.679762+0000","last_undegraded":"2024-04-20T21:17:54.679762+0000","last_fullsized":"2024-04-20T21:17:54.679762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:18:32.885585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.607283+0000","last_change":"2024-04-20T21:17:52.308569+0000","last_active":"2024-04-20T21:17:53.607283+0000","last_peered":"2024-04-20T21:17:53.607283+0000","last_clean":"2024-04-20T21:17:53.607283+0000","last_became_active":"2024-04-20T21:17:52.307606+0000","last_became_peered":"2024-04-20T21:17:52.307606+0000","last_unstale":"2024-04-20T21:17:53.607283+0000","last_undegraded":"2024-04-20T21:17:53.607283+0000","last_fullsized":"2024-04-20T21:17:53.607283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:34:46.630106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.288753+0000","last_change":"2024-04-20T21:17:52.281953+0000","last_active":"2024-04-20T21:17:53.288753+0000","last_peered":"2024-04-20T21:17:53.288753+0000","last_clean":"2024-04-20T21:17:53.288753+0000","last_became_active":"2024-04-20T21:17:52.281675+0000","last_became_peered":"2024-04-20T21:17:52.281675+0000","last_unstale":"2024-04-20T21:17:53.288753+0000","last_undegraded":"2024-04-20T21:17:53.288753+0000","last_fullsized":"2024-04-20T21:17:53.288753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:27:15.675834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.683439+0000","last_change":"2024-04-20T21:17:52.308448+0000","last_active":"2024-04-20T21:17:54.683439+0000","last_peered":"2024-04-20T21:17:54.683439+0000","last_clean":"2024-04-20T21:17:54.683439+0000","last_became_active":"2024-04-20T21:17:52.308158+0000","last_became_peered":"2024-04-20T21:17:52.308158+0000","last_unstale":"2024-04-20T21:17:54.683439+0000","last_undegraded":"2024-04-20T21:17:54.683439+0000","last_fullsized":"2024-04-20T21:17:54.683439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:09:02.488057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.357412+0000","last_change":"2024-04-20T21:17:52.304114+0000","last_active":"2024-04-20T21:17:54.357412+0000","last_peered":"2024-04-20T21:17:54.357412+0000","last_clean":"2024-04-20T21:17:54.357412+0000","last_became_active":"2024-04-20T21:17:52.303016+0000","last_became_peered":"2024-04-20T21:17:52.303016+0000","last_unstale":"2024-04-20T21:17:54.357412+0000","last_undegraded":"2024-04-20T21:17:54.357412+0000","last_fullsized":"2024-04-20T21:17:54.357412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:26:30.156714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.681213+0000","last_change":"2024-04-20T21:17:52.297418+0000","last_active":"2024-04-20T21:17:54.681213+0000","last_peered":"2024-04-20T21:17:54.681213+0000","last_clean":"2024-04-20T21:17:54.681213+0000","last_became_active":"2024-04-20T21:17:52.297018+0000","last_became_peered":"2024-04-20T21:17:52.297018+0000","last_unstale":"2024-04-20T21:17:54.681213+0000","last_undegraded":"2024-04-20T21:17:54.681213+0000","last_fullsized":"2024-04-20T21:17:54.681213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:12:56.626507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.357408+0000","last_change":"2024-04-20T21:17:52.306133+0000","last_active":"2024-04-20T21:17:54.357408+0000","last_peered":"2024-04-20T21:17:54.357408+0000","last_clean":"2024-04-20T21:17:54.357408+0000","last_became_active":"2024-04-20T21:17:52.304108+0000","last_became_peered":"2024-04-20T21:17:52.304108+0000","last_unstale":"2024-04-20T21:17:54.357408+0000","last_undegraded":"2024-04-20T21:17:54.357408+0000","last_fullsized":"2024-04-20T21:17:54.357408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:10:11.436747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.683656+0000","last_change":"2024-04-20T21:17:52.301781+0000","last_active":"2024-04-20T21:17:54.683656+0000","last_peered":"2024-04-20T21:17:54.683656+0000","last_clean":"2024-04-20T21:17:54.683656+0000","last_became_active":"2024-04-20T21:17:52.301194+0000","last_became_peered":"2024-04-20T21:17:52.301194+0000","last_unstale":"2024-04-20T21:17:54.683656+0000","last_undegraded":"2024-04-20T21:17:54.683656+0000","last_fullsized":"2024-04-20T21:17:54.683656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:10:11.436747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.289706+0000","last_change":"2024-04-20T21:17:52.303955+0000","last_active":"2024-04-20T21:17:53.289706+0000","last_peered":"2024-04-20T21:17:53.289706+0000","last_clean":"2024-04-20T21:17:53.289706+0000","last_became_active":"2024-04-20T21:17:52.303421+0000","last_became_peered":"2024-04-20T21:17:52.303421+0000","last_unstale":"2024-04-20T21:17:53.289706+0000","last_undegraded":"2024-04-20T21:17:53.289706+0000","last_fullsized":"2024-04-20T21:17:53.289706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:34:47.842666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.281621+0000","last_change":"2024-04-20T21:17:52.304472+0000","last_active":"2024-04-20T21:17:53.281621+0000","last_peered":"2024-04-20T21:17:53.281621+0000","last_clean":"2024-04-20T21:17:53.281621+0000","last_became_active":"2024-04-20T21:17:52.303141+0000","last_became_peered":"2024-04-20T21:17:52.303141+0000","last_unstale":"2024-04-20T21:17:53.281621+0000","last_undegraded":"2024-04-20T21:17:53.281621+0000","last_fullsized":"2024-04-20T21:17:53.281621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:11:50.878621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.681465+0000","last_change":"2024-04-20T21:17:52.302610+0000","last_active":"2024-04-20T21:17:54.681465+0000","last_peered":"2024-04-20T21:17:54.681465+0000","last_clean":"2024-04-20T21:17:54.681465+0000","last_became_active":"2024-04-20T21:17:52.302271+0000","last_became_peered":"2024-04-20T21:17:52.302271+0000","last_unstale":"2024-04-20T21:17:54.681465+0000","last_undegraded":"2024-04-20T21:17:54.681465+0000","last_fullsized":"2024-04-20T21:17:54.681465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:27:15.675834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.683628+0000","last_change":"2024-04-20T21:17:52.307809+0000","last_active":"2024-04-20T21:17:54.683628+0000","last_peered":"2024-04-20T21:17:54.683628+0000","last_clean":"2024-04-20T21:17:54.683628+0000","last_became_active":"2024-04-20T21:17:52.307459+0000","last_became_peered":"2024-04-20T21:17:52.307459+0000","last_unstale":"2024-04-20T21:17:54.683628+0000","last_undegraded":"2024-04-20T21:17:54.683628+0000","last_fullsized":"2024-04-20T21:17:54.683628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:52:26.845286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.357745+0000","last_change":"2024-04-20T21:17:52.305447+0000","last_active":"2024-04-20T21:17:54.357745+0000","last_peered":"2024-04-20T21:17:54.357745+0000","last_clean":"2024-04-20T21:17:54.357745+0000","last_became_active":"2024-04-20T21:17:52.303784+0000","last_became_peered":"2024-04-20T21:17:52.303784+0000","last_unstale":"2024-04-20T21:17:54.357745+0000","last_undegraded":"2024-04-20T21:17:54.357745+0000","last_fullsized":"2024-04-20T21:17:54.357745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:04:26.111984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.679196+0000","last_change":"2024-04-20T21:17:52.294245+0000","last_active":"2024-04-20T21:17:54.679196+0000","last_peered":"2024-04-20T21:17:54.679196+0000","last_clean":"2024-04-20T21:17:54.679196+0000","last_became_active":"2024-04-20T21:17:52.293880+0000","last_became_peered":"2024-04-20T21:17:52.293880+0000","last_unstale":"2024-04-20T21:17:54.679196+0000","last_undegraded":"2024-04-20T21:17:54.679196+0000","last_fullsized":"2024-04-20T21:17:54.679196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:51:58.727652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.287037+0000","last_change":"2024-04-20T21:17:52.303676+0000","last_active":"2024-04-20T21:17:53.287037+0000","last_peered":"2024-04-20T21:17:53.287037+0000","last_clean":"2024-04-20T21:17:53.287037+0000","last_became_active":"2024-04-20T21:17:52.303369+0000","last_became_peered":"2024-04-20T21:17:52.303369+0000","last_unstale":"2024-04-20T21:17:53.287037+0000","last_undegraded":"2024-04-20T21:17:53.287037+0000","last_fullsized":"2024-04-20T21:17:53.287037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:09:02.488057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.287416+0000","last_change":"2024-04-20T21:17:52.277151+0000","last_active":"2024-04-20T21:17:53.287416+0000","last_peered":"2024-04-20T21:17:53.287416+0000","last_clean":"2024-04-20T21:17:53.287416+0000","last_became_active":"2024-04-20T21:17:52.276740+0000","last_became_peered":"2024-04-20T21:17:52.276740+0000","last_unstale":"2024-04-20T21:17:53.287416+0000","last_undegraded":"2024-04-20T21:17:53.287416+0000","last_fullsized":"2024-04-20T21:17:53.287416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:34:27.917112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.680860+0000","last_change":"2024-04-20T21:17:51.275103+0000","last_active":"2024-04-20T21:17:54.680860+0000","last_peered":"2024-04-20T21:17:54.680860+0000","last_clean":"2024-04-20T21:17:54.680860+0000","last_became_active":"2024-04-20T21:17:51.274773+0000","last_became_peered":"2024-04-20T21:17:51.274773+0000","last_unstale":"2024-04-20T21:17:54.680860+0000","last_undegraded":"2024-04-20T21:17:54.680860+0000","last_fullsized":"2024-04-20T21:17:54.680860+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:53:16.864261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.683824+0000","last_change":"2024-04-20T21:17:52.312353+0000","last_active":"2024-04-20T21:17:54.683824+0000","last_peered":"2024-04-20T21:17:54.683824+0000","last_clean":"2024-04-20T21:17:54.683824+0000","last_became_active":"2024-04-20T21:17:52.311933+0000","last_became_peered":"2024-04-20T21:17:52.311933+0000","last_unstale":"2024-04-20T21:17:54.683824+0000","last_undegraded":"2024-04-20T21:17:54.683824+0000","last_fullsized":"2024-04-20T21:17:54.683824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:12:56.626507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.288439+0000","last_change":"2024-04-20T21:17:51.269484+0000","last_active":"2024-04-20T21:17:53.288439+0000","last_peered":"2024-04-20T21:17:53.288439+0000","last_clean":"2024-04-20T21:17:53.288439+0000","last_became_active":"2024-04-20T21:17:51.269196+0000","last_became_peered":"2024-04-20T21:17:51.269196+0000","last_unstale":"2024-04-20T21:17:53.288439+0000","last_undegraded":"2024-04-20T21:17:53.288439+0000","last_fullsized":"2024-04-20T21:17:53.288439+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:18:39.465785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.681496+0000","last_change":"2024-04-20T21:17:51.273330+0000","last_active":"2024-04-20T21:17:54.681496+0000","last_peered":"2024-04-20T21:17:54.681496+0000","last_clean":"2024-04-20T21:17:54.681496+0000","last_became_active":"2024-04-20T21:17:51.273036+0000","last_became_peered":"2024-04-20T21:17:51.273036+0000","last_unstale":"2024-04-20T21:17:54.681496+0000","last_undegraded":"2024-04-20T21:17:54.681496+0000","last_fullsized":"2024-04-20T21:17:54.681496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:07:01.697230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.682607+0000","last_change":"2024-04-20T21:17:52.300897+0000","last_active":"2024-04-20T21:17:54.682607+0000","last_peered":"2024-04-20T21:17:54.682607+0000","last_clean":"2024-04-20T21:17:54.682607+0000","last_became_active":"2024-04-20T21:17:52.300600+0000","last_became_peered":"2024-04-20T21:17:52.300600+0000","last_unstale":"2024-04-20T21:17:54.682607+0000","last_undegraded":"2024-04-20T21:17:54.682607+0000","last_fullsized":"2024-04-20T21:17:54.682607+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:11:50.878621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.357354+0000","last_change":"2024-04-20T21:17:51.276810+0000","last_active":"2024-04-20T21:17:54.357354+0000","last_peered":"2024-04-20T21:17:54.357354+0000","last_clean":"2024-04-20T21:17:54.357354+0000","last_became_active":"2024-04-20T21:17:51.276547+0000","last_became_peered":"2024-04-20T21:17:51.276547+0000","last_unstale":"2024-04-20T21:17:54.357354+0000","last_undegraded":"2024-04-20T21:17:54.357354+0000","last_fullsized":"2024-04-20T21:17:54.357354+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:04:47.881403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.281043+0000","last_change":"2024-04-20T21:17:52.299388+0000","last_active":"2024-04-20T21:17:53.281043+0000","last_peered":"2024-04-20T21:17:53.281043+0000","last_clean":"2024-04-20T21:17:53.281043+0000","last_became_active":"2024-04-20T21:17:52.298531+0000","last_became_peered":"2024-04-20T21:17:52.298531+0000","last_unstale":"2024-04-20T21:17:53.281043+0000","last_undegraded":"2024-04-20T21:17:53.281043+0000","last_fullsized":"2024-04-20T21:17:53.281043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:12:56.626507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.288661+0000","last_change":"2024-04-20T21:17:51.273343+0000","last_active":"2024-04-20T21:17:53.288661+0000","last_peered":"2024-04-20T21:17:53.288661+0000","last_clean":"2024-04-20T21:17:53.288661+0000","last_became_active":"2024-04-20T21:17:51.273018+0000","last_became_peered":"2024-04-20T21:17:51.273018+0000","last_unstale":"2024-04-20T21:17:53.288661+0000","last_undegraded":"2024-04-20T21:17:53.288661+0000","last_fullsized":"2024-04-20T21:17:53.288661+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:34:26.903632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.358664+0000","last_change":"2024-04-20T21:17:52.300438+0000","last_active":"2024-04-20T21:17:54.358664+0000","last_peered":"2024-04-20T21:17:54.358664+0000","last_clean":"2024-04-20T21:17:54.358664+0000","last_became_active":"2024-04-20T21:17:52.300082+0000","last_became_peered":"2024-04-20T21:17:52.300082+0000","last_unstale":"2024-04-20T21:17:54.358664+0000","last_undegraded":"2024-04-20T21:17:54.358664+0000","last_fullsized":"2024-04-20T21:17:54.358664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:23:58.112035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.358530+0000","last_change":"2024-04-20T21:17:51.272721+0000","last_active":"2024-04-20T21:17:54.358530+0000","last_peered":"2024-04-20T21:17:54.358530+0000","last_clean":"2024-04-20T21:17:54.358530+0000","last_became_active":"2024-04-20T21:17:51.272403+0000","last_became_peered":"2024-04-20T21:17:51.272403+0000","last_unstale":"2024-04-20T21:17:54.358530+0000","last_undegraded":"2024-04-20T21:17:54.358530+0000","last_fullsized":"2024-04-20T21:17:54.358530+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:09:01.474577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.684541+0000","last_change":"2024-04-20T21:17:52.316437+0000","last_active":"2024-04-20T21:17:54.684541+0000","last_peered":"2024-04-20T21:17:54.684541+0000","last_clean":"2024-04-20T21:17:54.684541+0000","last_became_active":"2024-04-20T21:17:52.316129+0000","last_became_peered":"2024-04-20T21:17:52.316129+0000","last_unstale":"2024-04-20T21:17:54.684541+0000","last_undegraded":"2024-04-20T21:17:54.684541+0000","last_fullsized":"2024-04-20T21:17:54.684541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:34:47.842666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.281424+0000","last_change":"2024-04-20T21:17:51.268776+0000","last_active":"2024-04-20T21:17:53.281424+0000","last_peered":"2024-04-20T21:17:53.281424+0000","last_clean":"2024-04-20T21:17:53.281424+0000","last_became_active":"2024-04-20T21:17:51.268468+0000","last_became_peered":"2024-04-20T21:17:51.268468+0000","last_unstale":"2024-04-20T21:17:53.281424+0000","last_undegraded":"2024-04-20T21:17:53.281424+0000","last_fullsized":"2024-04-20T21:17:53.281424+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:34:26.903632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.287471+0000","last_change":"2024-04-20T21:17:52.300758+0000","last_active":"2024-04-20T21:17:53.287471+0000","last_peered":"2024-04-20T21:17:53.287471+0000","last_clean":"2024-04-20T21:17:53.287471+0000","last_became_active":"2024-04-20T21:17:52.300476+0000","last_became_peered":"2024-04-20T21:17:52.300476+0000","last_unstale":"2024-04-20T21:17:53.287471+0000","last_undegraded":"2024-04-20T21:17:53.287471+0000","last_fullsized":"2024-04-20T21:17:53.287471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:51:58.727652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.288618+0000","last_change":"2024-04-20T21:17:51.270900+0000","last_active":"2024-04-20T21:17:53.288618+0000","last_peered":"2024-04-20T21:17:53.288618+0000","last_clean":"2024-04-20T21:17:53.288618+0000","last_became_active":"2024-04-20T21:17:51.270618+0000","last_became_peered":"2024-04-20T21:17:51.270618+0000","last_unstale":"2024-04-20T21:17:53.288618+0000","last_undegraded":"2024-04-20T21:17:53.288618+0000","last_fullsized":"2024-04-20T21:17:53.288618+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:04:47.881403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.606885+0000","last_change":"2024-04-20T21:17:52.313555+0000","last_active":"2024-04-20T21:17:53.606885+0000","last_peered":"2024-04-20T21:17:53.606885+0000","last_clean":"2024-04-20T21:17:53.606885+0000","last_became_active":"2024-04-20T21:17:52.312197+0000","last_became_peered":"2024-04-20T21:17:52.312197+0000","last_unstale":"2024-04-20T21:17:53.606885+0000","last_undegraded":"2024-04-20T21:17:53.606885+0000","last_fullsized":"2024-04-20T21:17:53.606885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:52:22.418925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.683925+0000","last_change":"2024-04-20T21:17:51.275126+0000","last_active":"2024-04-20T21:17:54.683925+0000","last_peered":"2024-04-20T21:17:54.683925+0000","last_clean":"2024-04-20T21:17:54.683925+0000","last_became_active":"2024-04-20T21:17:51.273060+0000","last_became_peered":"2024-04-20T21:17:51.273060+0000","last_unstale":"2024-04-20T21:17:54.683925+0000","last_undegraded":"2024-04-20T21:17:54.683925+0000","last_fullsized":"2024-04-20T21:17:54.683925+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:06:19.740358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.683147+0000","last_change":"2024-04-20T21:17:52.302899+0000","last_active":"2024-04-20T21:17:54.683147+0000","last_peered":"2024-04-20T21:17:54.683147+0000","last_clean":"2024-04-20T21:17:54.683147+0000","last_became_active":"2024-04-20T21:17:52.302606+0000","last_became_peered":"2024-04-20T21:17:52.302606+0000","last_unstale":"2024-04-20T21:17:54.683147+0000","last_undegraded":"2024-04-20T21:17:54.683147+0000","last_fullsized":"2024-04-20T21:17:54.683147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:13:52.887276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.357587+0000","last_change":"2024-04-20T21:17:51.278078+0000","last_active":"2024-04-20T21:17:54.357587+0000","last_peered":"2024-04-20T21:17:54.357587+0000","last_clean":"2024-04-20T21:17:54.357587+0000","last_became_active":"2024-04-20T21:17:51.277775+0000","last_became_peered":"2024-04-20T21:17:51.277775+0000","last_unstale":"2024-04-20T21:17:54.357587+0000","last_undegraded":"2024-04-20T21:17:54.357587+0000","last_fullsized":"2024-04-20T21:17:54.357587+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:23:42.473358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.607236+0000","last_change":"2024-04-20T21:17:52.295456+0000","last_active":"2024-04-20T21:17:53.607236+0000","last_peered":"2024-04-20T21:17:53.607236+0000","last_clean":"2024-04-20T21:17:53.607236+0000","last_became_active":"2024-04-20T21:17:52.294825+0000","last_became_peered":"2024-04-20T21:17:52.294825+0000","last_unstale":"2024-04-20T21:17:53.607236+0000","last_undegraded":"2024-04-20T21:17:53.607236+0000","last_fullsized":"2024-04-20T21:17:53.607236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:43:32.983640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.288028+0000","last_change":"2024-04-20T21:17:51.264742+0000","last_active":"2024-04-20T21:17:53.288028+0000","last_peered":"2024-04-20T21:17:53.288028+0000","last_clean":"2024-04-20T21:17:53.288028+0000","last_became_active":"2024-04-20T21:17:51.264009+0000","last_became_peered":"2024-04-20T21:17:51.264009+0000","last_unstale":"2024-04-20T21:17:53.288028+0000","last_undegraded":"2024-04-20T21:17:53.288028+0000","last_fullsized":"2024-04-20T21:17:53.288028+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:29:17.957905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.288331+0000","last_change":"2024-04-20T21:17:52.313950+0000","last_active":"2024-04-20T21:17:53.288331+0000","last_peered":"2024-04-20T21:17:53.288331+0000","last_clean":"2024-04-20T21:17:53.288331+0000","last_became_active":"2024-04-20T21:17:52.311961+0000","last_became_peered":"2024-04-20T21:17:52.311961+0000","last_unstale":"2024-04-20T21:17:53.288331+0000","last_undegraded":"2024-04-20T21:17:53.288331+0000","last_fullsized":"2024-04-20T21:17:53.288331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:12:56.626507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.678630+0000","last_change":"2024-04-20T21:17:51.254940+0000","last_active":"2024-04-20T21:17:54.678630+0000","last_peered":"2024-04-20T21:17:54.678630+0000","last_clean":"2024-04-20T21:17:54.678630+0000","last_became_active":"2024-04-20T21:17:51.254597+0000","last_became_peered":"2024-04-20T21:17:51.254597+0000","last_unstale":"2024-04-20T21:17:54.678630+0000","last_undegraded":"2024-04-20T21:17:54.678630+0000","last_fullsized":"2024-04-20T21:17:54.678630+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:43:48.697965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.606512+0000","last_change":"2024-04-20T21:17:52.303514+0000","last_active":"2024-04-20T21:17:53.606512+0000","last_peered":"2024-04-20T21:17:53.606512+0000","last_clean":"2024-04-20T21:17:53.606512+0000","last_became_active":"2024-04-20T21:17:52.303225+0000","last_became_peered":"2024-04-20T21:17:52.303225+0000","last_unstale":"2024-04-20T21:17:53.606512+0000","last_undegraded":"2024-04-20T21:17:53.606512+0000","last_fullsized":"2024-04-20T21:17:53.606512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:26:30.156714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.287716+0000","last_change":"2024-04-20T21:17:51.276458+0000","last_active":"2024-04-20T21:17:53.287716+0000","last_peered":"2024-04-20T21:17:53.287716+0000","last_clean":"2024-04-20T21:17:53.287716+0000","last_became_active":"2024-04-20T21:17:51.276033+0000","last_became_peered":"2024-04-20T21:17:51.276033+0000","last_unstale":"2024-04-20T21:17:53.287716+0000","last_undegraded":"2024-04-20T21:17:53.287716+0000","last_fullsized":"2024-04-20T21:17:53.287716+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:00:14.426522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.607498+0000","last_change":"2024-04-20T21:17:52.310874+0000","last_active":"2024-04-20T21:17:53.607498+0000","last_peered":"2024-04-20T21:17:53.607498+0000","last_clean":"2024-04-20T21:17:53.607498+0000","last_became_active":"2024-04-20T21:17:52.310612+0000","last_became_peered":"2024-04-20T21:17:52.310612+0000","last_unstale":"2024-04-20T21:17:53.607498+0000","last_undegraded":"2024-04-20T21:17:53.607498+0000","last_fullsized":"2024-04-20T21:17:53.607498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:59:58.666249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.679906+0000","last_change":"2024-04-20T21:17:51.266671+0000","last_active":"2024-04-20T21:17:54.679906+0000","last_peered":"2024-04-20T21:17:54.679906+0000","last_clean":"2024-04-20T21:17:54.679906+0000","last_became_active":"2024-04-20T21:17:51.264486+0000","last_became_peered":"2024-04-20T21:17:51.264486+0000","last_unstale":"2024-04-20T21:17:54.679906+0000","last_undegraded":"2024-04-20T21:17:54.679906+0000","last_fullsized":"2024-04-20T21:17:54.679906+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:53:16.864261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.607949+0000","last_change":"2024-04-20T21:17:52.309913+0000","last_active":"2024-04-20T21:17:53.607949+0000","last_peered":"2024-04-20T21:17:53.607949+0000","last_clean":"2024-04-20T21:17:53.607949+0000","last_became_active":"2024-04-20T21:17:52.309643+0000","last_became_peered":"2024-04-20T21:17:52.309643+0000","last_unstale":"2024-04-20T21:17:53.607949+0000","last_undegraded":"2024-04-20T21:17:53.607949+0000","last_fullsized":"2024-04-20T21:17:53.607949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:48:28.191286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.281734+0000","last_change":"2024-04-20T21:17:51.272265+0000","last_active":"2024-04-20T21:17:53.281734+0000","last_peered":"2024-04-20T21:17:53.281734+0000","last_clean":"2024-04-20T21:17:53.281734+0000","last_became_active":"2024-04-20T21:17:51.271937+0000","last_became_peered":"2024-04-20T21:17:51.271937+0000","last_unstale":"2024-04-20T21:17:53.281734+0000","last_undegraded":"2024-04-20T21:17:53.281734+0000","last_fullsized":"2024-04-20T21:17:53.281734+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:51:57.714172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.358372+0000","last_change":"2024-04-20T21:17:52.304344+0000","last_active":"2024-04-20T21:17:54.358372+0000","last_peered":"2024-04-20T21:17:54.358372+0000","last_clean":"2024-04-20T21:17:54.358372+0000","last_became_active":"2024-04-20T21:17:52.304006+0000","last_became_peered":"2024-04-20T21:17:52.304006+0000","last_unstale":"2024-04-20T21:17:54.358372+0000","last_undegraded":"2024-04-20T21:17:54.358372+0000","last_fullsized":"2024-04-20T21:17:54.358372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:59:58.666249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.679057+0000","last_change":"2024-04-20T21:17:51.272508+0000","last_active":"2024-04-20T21:17:54.679057+0000","last_peered":"2024-04-20T21:17:54.679057+0000","last_clean":"2024-04-20T21:17:54.679057+0000","last_became_active":"2024-04-20T21:17:51.272056+0000","last_became_peered":"2024-04-20T21:17:51.272056+0000","last_unstale":"2024-04-20T21:17:54.679057+0000","last_undegraded":"2024-04-20T21:17:54.679057+0000","last_fullsized":"2024-04-20T21:17:54.679057+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:06:19.740358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.606888+0000","last_change":"2024-04-20T21:17:52.308203+0000","last_active":"2024-04-20T21:17:53.606888+0000","last_peered":"2024-04-20T21:17:53.606888+0000","last_clean":"2024-04-20T21:17:53.606888+0000","last_became_active":"2024-04-20T21:17:52.307941+0000","last_became_peered":"2024-04-20T21:17:52.307941+0000","last_unstale":"2024-04-20T21:17:53.606888+0000","last_undegraded":"2024-04-20T21:17:53.606888+0000","last_fullsized":"2024-04-20T21:17:53.606888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:51:21.491774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.606871+0000","last_change":"2024-04-20T21:17:43.059637+0000","last_active":"2024-04-20T21:17:53.606871+0000","last_peered":"2024-04-20T21:17:53.606871+0000","last_clean":"2024-04-20T21:17:53.606871+0000","last_became_active":"2024-04-20T21:17:43.056207+0000","last_became_peered":"2024-04-20T21:17:43.056207+0000","last_unstale":"2024-04-20T21:17:53.606871+0000","last_undegraded":"2024-04-20T21:17:53.606871+0000","last_fullsized":"2024-04-20T21:17:53.606871+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:41.036290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:41.036290+0000","last_clean_scrub_stamp":"2024-04-20T21:17:41.036290+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:58:51.091337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.357115+0000","last_change":"2024-04-20T21:17:51.277973+0000","last_active":"2024-04-20T21:17:54.357115+0000","last_peered":"2024-04-20T21:17:54.357115+0000","last_clean":"2024-04-20T21:17:54.357115+0000","last_became_active":"2024-04-20T21:17:51.277643+0000","last_became_peered":"2024-04-20T21:17:51.277643+0000","last_unstale":"2024-04-20T21:17:54.357115+0000","last_undegraded":"2024-04-20T21:17:54.357115+0000","last_fullsized":"2024-04-20T21:17:54.357115+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:41:08.560538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.683244+0000","last_change":"2024-04-20T21:17:52.316313+0000","last_active":"2024-04-20T21:17:54.683244+0000","last_peered":"2024-04-20T21:17:54.683244+0000","last_clean":"2024-04-20T21:17:54.683244+0000","last_became_active":"2024-04-20T21:17:52.316054+0000","last_became_peered":"2024-04-20T21:17:52.316054+0000","last_unstale":"2024-04-20T21:17:54.683244+0000","last_undegraded":"2024-04-20T21:17:54.683244+0000","last_fullsized":"2024-04-20T21:17:54.683244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:26:30.156714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.606138+0000","last_change":"2024-04-20T21:17:51.282977+0000","last_active":"2024-04-20T21:17:53.606138+0000","last_peered":"2024-04-20T21:17:53.606138+0000","last_clean":"2024-04-20T21:17:53.606138+0000","last_became_active":"2024-04-20T21:17:51.282699+0000","last_became_peered":"2024-04-20T21:17:51.282699+0000","last_unstale":"2024-04-20T21:17:53.606138+0000","last_undegraded":"2024-04-20T21:17:53.606138+0000","last_fullsized":"2024-04-20T21:17:53.606138+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:04:47.881403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.682600+0000","last_change":"2024-04-20T21:17:52.301574+0000","last_active":"2024-04-20T21:17:54.682600+0000","last_peered":"2024-04-20T21:17:54.682600+0000","last_clean":"2024-04-20T21:17:54.682600+0000","last_became_active":"2024-04-20T21:17:52.299834+0000","last_became_peered":"2024-04-20T21:17:52.299834+0000","last_unstale":"2024-04-20T21:17:54.682600+0000","last_undegraded":"2024-04-20T21:17:54.682600+0000","last_fullsized":"2024-04-20T21:17:54.682600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:23:43.486837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.679134+0000","last_change":"2024-04-20T21:17:51.268326+0000","last_active":"2024-04-20T21:17:54.679134+0000","last_peered":"2024-04-20T21:17:54.679134+0000","last_clean":"2024-04-20T21:17:54.679134+0000","last_became_active":"2024-04-20T21:17:51.268016+0000","last_became_peered":"2024-04-20T21:17:51.268016+0000","last_unstale":"2024-04-20T21:17:54.679134+0000","last_undegraded":"2024-04-20T21:17:54.679134+0000","last_fullsized":"2024-04-20T21:17:54.679134+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:18:39.465785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.280844+0000","last_change":"2024-04-20T21:17:52.289513+0000","last_active":"2024-04-20T21:17:53.280844+0000","last_peered":"2024-04-20T21:17:53.280844+0000","last_clean":"2024-04-20T21:17:53.280844+0000","last_became_active":"2024-04-20T21:17:52.288677+0000","last_became_peered":"2024-04-20T21:17:52.288677+0000","last_unstale":"2024-04-20T21:17:53.280844+0000","last_undegraded":"2024-04-20T21:17:53.280844+0000","last_fullsized":"2024-04-20T21:17:53.280844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:52:26.845286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.358151+0000","last_change":"2024-04-20T21:17:51.275850+0000","last_active":"2024-04-20T21:17:54.358151+0000","last_peered":"2024-04-20T21:17:54.358151+0000","last_clean":"2024-04-20T21:17:54.358151+0000","last_became_active":"2024-04-20T21:17:51.275350+0000","last_became_peered":"2024-04-20T21:17:51.275350+0000","last_unstale":"2024-04-20T21:17:54.358151+0000","last_undegraded":"2024-04-20T21:17:54.358151+0000","last_fullsized":"2024-04-20T21:17:54.358151+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:38:55.545413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.287199+0000","last_change":"2024-04-20T21:17:52.305922+0000","last_active":"2024-04-20T21:17:53.287199+0000","last_peered":"2024-04-20T21:17:53.287199+0000","last_clean":"2024-04-20T21:17:53.287199+0000","last_became_active":"2024-04-20T21:17:52.304705+0000","last_became_peered":"2024-04-20T21:17:52.304705+0000","last_unstale":"2024-04-20T21:17:53.287199+0000","last_undegraded":"2024-04-20T21:17:53.287199+0000","last_fullsized":"2024-04-20T21:17:53.287199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:04:48.894882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.281179+0000","last_change":"2024-04-20T21:17:51.260375+0000","last_active":"2024-04-20T21:17:53.281179+0000","last_peered":"2024-04-20T21:17:53.281179+0000","last_clean":"2024-04-20T21:17:53.281179+0000","last_became_active":"2024-04-20T21:17:51.259993+0000","last_became_peered":"2024-04-20T21:17:51.259993+0000","last_unstale":"2024-04-20T21:17:53.281179+0000","last_undegraded":"2024-04-20T21:17:53.281179+0000","last_fullsized":"2024-04-20T21:17:53.281179+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:04:47.881403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.358295+0000","last_change":"2024-04-20T21:17:52.308697+0000","last_active":"2024-04-20T21:17:54.358295+0000","last_peered":"2024-04-20T21:17:54.358295+0000","last_clean":"2024-04-20T21:17:54.358295+0000","last_became_active":"2024-04-20T21:17:52.308037+0000","last_became_peered":"2024-04-20T21:17:52.308037+0000","last_unstale":"2024-04-20T21:17:54.358295+0000","last_undegraded":"2024-04-20T21:17:54.358295+0000","last_fullsized":"2024-04-20T21:17:54.358295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:34:46.630106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.607409+0000","last_change":"2024-04-20T21:17:51.283426+0000","last_active":"2024-04-20T21:17:53.607409+0000","last_peered":"2024-04-20T21:17:53.607409+0000","last_clean":"2024-04-20T21:17:53.607409+0000","last_became_active":"2024-04-20T21:17:51.283162+0000","last_became_peered":"2024-04-20T21:17:51.283162+0000","last_unstale":"2024-04-20T21:17:53.607409+0000","last_undegraded":"2024-04-20T21:17:53.607409+0000","last_fullsized":"2024-04-20T21:17:53.607409+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:18:31.872106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.281429+0000","last_change":"2024-04-20T21:17:52.285628+0000","last_active":"2024-04-20T21:17:53.281429+0000","last_peered":"2024-04-20T21:17:53.281429+0000","last_clean":"2024-04-20T21:17:53.281429+0000","last_became_active":"2024-04-20T21:17:52.284918+0000","last_became_peered":"2024-04-20T21:17:52.284918+0000","last_unstale":"2024-04-20T21:17:53.281429+0000","last_undegraded":"2024-04-20T21:17:53.281429+0000","last_fullsized":"2024-04-20T21:17:53.281429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:42:57.243810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.281782+0000","last_change":"2024-04-20T21:17:51.271506+0000","last_active":"2024-04-20T21:17:53.281782+0000","last_peered":"2024-04-20T21:17:53.281782+0000","last_clean":"2024-04-20T21:17:53.281782+0000","last_became_active":"2024-04-20T21:17:51.271243+0000","last_became_peered":"2024-04-20T21:17:51.271243+0000","last_unstale":"2024-04-20T21:17:53.281782+0000","last_undegraded":"2024-04-20T21:17:53.281782+0000","last_fullsized":"2024-04-20T21:17:53.281782+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:29:17.957905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.684078+0000","last_change":"2024-04-20T21:17:52.301465+0000","last_active":"2024-04-20T21:17:54.684078+0000","last_peered":"2024-04-20T21:17:54.684078+0000","last_clean":"2024-04-20T21:17:54.684078+0000","last_became_active":"2024-04-20T21:17:52.300849+0000","last_became_peered":"2024-04-20T21:17:52.300849+0000","last_unstale":"2024-04-20T21:17:54.684078+0000","last_undegraded":"2024-04-20T21:17:54.684078+0000","last_fullsized":"2024-04-20T21:17:54.684078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:04:26.111984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.288907+0000","last_change":"2024-04-20T21:17:51.282509+0000","last_active":"2024-04-20T21:17:53.288907+0000","last_peered":"2024-04-20T21:17:53.288907+0000","last_clean":"2024-04-20T21:17:53.288907+0000","last_became_active":"2024-04-20T21:17:51.281343+0000","last_became_peered":"2024-04-20T21:17:51.281343+0000","last_unstale":"2024-04-20T21:17:53.288907+0000","last_undegraded":"2024-04-20T21:17:53.288907+0000","last_fullsized":"2024-04-20T21:17:53.288907+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:43:48.697965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.684280+0000","last_change":"2024-04-20T21:17:52.311617+0000","last_active":"2024-04-20T21:17:54.684280+0000","last_peered":"2024-04-20T21:17:54.684280+0000","last_clean":"2024-04-20T21:17:54.684280+0000","last_became_active":"2024-04-20T21:17:52.311367+0000","last_became_peered":"2024-04-20T21:17:52.311367+0000","last_unstale":"2024-04-20T21:17:54.684280+0000","last_undegraded":"2024-04-20T21:17:54.684280+0000","last_fullsized":"2024-04-20T21:17:54.684280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:27:15.675834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.607489+0000","last_change":"2024-04-20T21:17:51.283039+0000","last_active":"2024-04-20T21:17:53.607489+0000","last_peered":"2024-04-20T21:17:53.607489+0000","last_clean":"2024-04-20T21:17:53.607489+0000","last_became_active":"2024-04-20T21:17:51.282780+0000","last_became_peered":"2024-04-20T21:17:51.282780+0000","last_unstale":"2024-04-20T21:17:53.607489+0000","last_undegraded":"2024-04-20T21:17:53.607489+0000","last_fullsized":"2024-04-20T21:17:53.607489+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:18:39.465785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.286949+0000","last_change":"2024-04-20T21:17:52.293046+0000","last_active":"2024-04-20T21:17:53.286949+0000","last_peered":"2024-04-20T21:17:53.286949+0000","last_clean":"2024-04-20T21:17:53.286949+0000","last_became_active":"2024-04-20T21:17:52.288365+0000","last_became_peered":"2024-04-20T21:17:52.288365+0000","last_unstale":"2024-04-20T21:17:53.286949+0000","last_undegraded":"2024-04-20T21:17:53.286949+0000","last_fullsized":"2024-04-20T21:17:53.286949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:18:40.479264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.281228+0000","last_change":"2024-04-20T21:17:51.280889+0000","last_active":"2024-04-20T21:17:53.281228+0000","last_peered":"2024-04-20T21:17:53.281228+0000","last_clean":"2024-04-20T21:17:53.281228+0000","last_became_active":"2024-04-20T21:17:51.280582+0000","last_became_peered":"2024-04-20T21:17:51.280582+0000","last_unstale":"2024-04-20T21:17:53.281228+0000","last_undegraded":"2024-04-20T21:17:53.281228+0000","last_fullsized":"2024-04-20T21:17:53.281228+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:18:39.465785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.682757+0000","last_change":"2024-04-20T21:17:52.292459+0000","last_active":"2024-04-20T21:17:54.682757+0000","last_peered":"2024-04-20T21:17:54.682757+0000","last_clean":"2024-04-20T21:17:54.682757+0000","last_became_active":"2024-04-20T21:17:52.292182+0000","last_became_peered":"2024-04-20T21:17:52.292182+0000","last_unstale":"2024-04-20T21:17:54.682757+0000","last_undegraded":"2024-04-20T21:17:54.682757+0000","last_fullsized":"2024-04-20T21:17:54.682757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:34:47.842666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.281162+0000","last_change":"2024-04-20T21:17:51.269365+0000","last_active":"2024-04-20T21:17:53.281162+0000","last_peered":"2024-04-20T21:17:53.281162+0000","last_clean":"2024-04-20T21:17:53.281162+0000","last_became_active":"2024-04-20T21:17:51.267902+0000","last_became_peered":"2024-04-20T21:17:51.267902+0000","last_unstale":"2024-04-20T21:17:53.281162+0000","last_undegraded":"2024-04-20T21:17:53.281162+0000","last_fullsized":"2024-04-20T21:17:53.281162+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:43:48.697965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.681157+0000","last_change":"2024-04-20T21:17:52.315096+0000","last_active":"2024-04-20T21:17:54.681157+0000","last_peered":"2024-04-20T21:17:54.681157+0000","last_clean":"2024-04-20T21:17:54.681157+0000","last_became_active":"2024-04-20T21:17:52.314742+0000","last_became_peered":"2024-04-20T21:17:52.314742+0000","last_unstale":"2024-04-20T21:17:54.681157+0000","last_undegraded":"2024-04-20T21:17:54.681157+0000","last_fullsized":"2024-04-20T21:17:54.681157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:52:26.845286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.607525+0000","last_change":"2024-04-20T21:17:51.288583+0000","last_active":"2024-04-20T21:17:53.607525+0000","last_peered":"2024-04-20T21:17:53.607525+0000","last_clean":"2024-04-20T21:17:53.607525+0000","last_became_active":"2024-04-20T21:17:51.288299+0000","last_became_peered":"2024-04-20T21:17:51.288299+0000","last_unstale":"2024-04-20T21:17:53.607525+0000","last_undegraded":"2024-04-20T21:17:53.607525+0000","last_fullsized":"2024-04-20T21:17:53.607525+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:29:17.957905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.288285+0000","last_change":"2024-04-20T21:17:52.283186+0000","last_active":"2024-04-20T21:17:53.288285+0000","last_peered":"2024-04-20T21:17:53.288285+0000","last_clean":"2024-04-20T21:17:53.288285+0000","last_became_active":"2024-04-20T21:17:52.280776+0000","last_became_peered":"2024-04-20T21:17:52.280776+0000","last_unstale":"2024-04-20T21:17:53.288285+0000","last_undegraded":"2024-04-20T21:17:53.288285+0000","last_fullsized":"2024-04-20T21:17:53.288285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:52:26.845286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.358061+0000","last_change":"2024-04-20T21:17:52.302224+0000","last_active":"2024-04-20T21:17:54.358061+0000","last_peered":"2024-04-20T21:17:54.358061+0000","last_clean":"2024-04-20T21:17:54.358061+0000","last_became_active":"2024-04-20T21:17:52.301869+0000","last_became_peered":"2024-04-20T21:17:52.301869+0000","last_unstale":"2024-04-20T21:17:54.358061+0000","last_undegraded":"2024-04-20T21:17:54.358061+0000","last_fullsized":"2024-04-20T21:17:54.358061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:43:32.983640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.606747+0000","last_change":"2024-04-20T21:17:51.288486+0000","last_active":"2024-04-20T21:17:53.606747+0000","last_peered":"2024-04-20T21:17:53.606747+0000","last_clean":"2024-04-20T21:17:53.606747+0000","last_became_active":"2024-04-20T21:17:51.288131+0000","last_became_peered":"2024-04-20T21:17:51.288131+0000","last_unstale":"2024-04-20T21:17:53.606747+0000","last_undegraded":"2024-04-20T21:17:53.606747+0000","last_fullsized":"2024-04-20T21:17:53.606747+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:51:57.714172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.357879+0000","last_change":"2024-04-20T21:17:52.302297+0000","last_active":"2024-04-20T21:17:54.357879+0000","last_peered":"2024-04-20T21:17:54.357879+0000","last_clean":"2024-04-20T21:17:54.357879+0000","last_became_active":"2024-04-20T21:17:52.301982+0000","last_became_peered":"2024-04-20T21:17:52.301982+0000","last_unstale":"2024-04-20T21:17:54.357879+0000","last_undegraded":"2024-04-20T21:17:54.357879+0000","last_fullsized":"2024-04-20T21:17:54.357879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:51:21.491774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.607170+0000","last_change":"2024-04-20T21:17:51.285151+0000","last_active":"2024-04-20T21:17:53.607170+0000","last_peered":"2024-04-20T21:17:53.607170+0000","last_clean":"2024-04-20T21:17:53.607170+0000","last_became_active":"2024-04-20T21:17:51.284879+0000","last_became_peered":"2024-04-20T21:17:51.284879+0000","last_unstale":"2024-04-20T21:17:53.607170+0000","last_undegraded":"2024-04-20T21:17:53.607170+0000","last_fullsized":"2024-04-20T21:17:53.607170+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:34:26.903632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.684538+0000","last_change":"2024-04-20T21:17:52.317666+0000","last_active":"2024-04-20T21:17:54.684538+0000","last_peered":"2024-04-20T21:17:54.684538+0000","last_clean":"2024-04-20T21:17:54.684538+0000","last_became_active":"2024-04-20T21:17:52.316819+0000","last_became_peered":"2024-04-20T21:17:52.316819+0000","last_unstale":"2024-04-20T21:17:54.684538+0000","last_undegraded":"2024-04-20T21:17:54.684538+0000","last_fullsized":"2024-04-20T21:17:54.684538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:11:50.878621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.682331+0000","last_change":"2024-04-20T21:17:51.276188+0000","last_active":"2024-04-20T21:17:54.682331+0000","last_peered":"2024-04-20T21:17:54.682331+0000","last_clean":"2024-04-20T21:17:54.682331+0000","last_became_active":"2024-04-20T21:17:51.275862+0000","last_became_peered":"2024-04-20T21:17:51.275862+0000","last_unstale":"2024-04-20T21:17:54.682331+0000","last_undegraded":"2024-04-20T21:17:54.682331+0000","last_fullsized":"2024-04-20T21:17:54.682331+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:18:39.465785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.684488+0000","last_change":"2024-04-20T21:17:52.310311+0000","last_active":"2024-04-20T21:17:54.684488+0000","last_peered":"2024-04-20T21:17:54.684488+0000","last_clean":"2024-04-20T21:17:54.684488+0000","last_became_active":"2024-04-20T21:17:52.307410+0000","last_became_peered":"2024-04-20T21:17:52.307410+0000","last_unstale":"2024-04-20T21:17:54.684488+0000","last_undegraded":"2024-04-20T21:17:54.684488+0000","last_fullsized":"2024-04-20T21:17:54.684488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:52:22.418925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.684750+0000","last_change":"2024-04-20T21:17:51.287639+0000","last_active":"2024-04-20T21:17:54.684750+0000","last_peered":"2024-04-20T21:17:54.684750+0000","last_clean":"2024-04-20T21:17:54.684750+0000","last_became_active":"2024-04-20T21:17:51.287421+0000","last_became_peered":"2024-04-20T21:17:51.287421+0000","last_unstale":"2024-04-20T21:17:54.684750+0000","last_undegraded":"2024-04-20T21:17:54.684750+0000","last_fullsized":"2024-04-20T21:17:54.684750+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:07:01.697230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T21:18:02.350 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T21:18:02.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.581+0000 7fcf50e00640 1 Processor -- start 2024-04-20T21:18:02.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.581+0000 7fcf50e00640 1 -- start start 2024-04-20T21:18:02.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.582+0000 7fcf50e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcf4c156ae0 0x7fcf4c15d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:02.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.582+0000 7fcf50e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcf4c0ad7f0 0x7fcf4c06e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:02.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.582+0000 7fcf50e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf4c160890 con 0x7fcf4c160080 2024-04-20T21:18:02.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.582+0000 7fcf50e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fcf4c06db20 con 0x7fcf4c156ae0 2024-04-20T21:18:02.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.582+0000 7fcf50e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fcf4c160b70 con 0x7fcf4c0ad7f0 2024-04-20T21:18:02.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.582+0000 7fcf4b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcf4c156ae0 0x7fcf4c15d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:02.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.582+0000 7fcf4b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcf4c156ae0 0x7fcf4c15d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58112/0 (socket says 172.21.15.86:58112) 2024-04-20T21:18:02.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.582+0000 7fcf4b400640 1 -- 172.21.15.86:0/1524029018 learned_addr learned my addr 172.21.15.86:0/1524029018 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:02.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.583+0000 7fcf4b400640 1 -- 172.21.15.86:0/1524029018 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcf4c0ad7f0 msgr2=0x7fcf4c06e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:02.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.582+0000 7fcf4be00640 1 --2- 172.21.15.86:0/1524029018 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcf4c0ad7f0 0x7fcf4c06e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:02.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.583+0000 7fcf4b400640 1 --2- 172.21.15.86:0/1524029018 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcf4c0ad7f0 0x7fcf4c06e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:02.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.583+0000 7fcf4b400640 1 -- 172.21.15.86:0/1524029018 >> v1:172.21.15.144:6789/0 conn(0x7fcf4c160080 legacy=0x7fcf4c0ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:02.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.583+0000 7fcf4b400640 1 -- 172.21.15.86:0/1524029018 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf4c160e50 con 0x7fcf4c156ae0 2024-04-20T21:18:02.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.584+0000 7fcf4b400640 1 --2- 172.21.15.86:0/1524029018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf4c156ae0 0x7fcf4c15d6a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c009870 tx=0x7fcf3c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ee8ebeac00de4b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:02.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.584+0000 7fcf4a000640 1 -- 172.21.15.86:0/1524029018 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c035020 con 0x7fcf4c156ae0 2024-04-20T21:18:02.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.584+0000 7fcf4a000640 1 -- 172.21.15.86:0/1524029018 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf3c031640 con 0x7fcf4c156ae0 2024-04-20T21:18:02.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.584+0000 7fcf4a000640 1 -- 172.21.15.86:0/1524029018 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c02d040 con 0x7fcf4c156ae0 2024-04-20T21:18:02.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.584+0000 7fcf50e00640 1 -- 172.21.15.86:0/1524029018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf4c156ae0 msgr2=0x7fcf4c15d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:02.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.584+0000 7fcf50e00640 1 --2- 172.21.15.86:0/1524029018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf4c156ae0 0x7fcf4c15d6a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c009870 tx=0x7fcf3c02b8f0 comp rx=0 tx=0).stop 2024-04-20T21:18:02.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.585+0000 7fcf50e00640 1 -- 172.21.15.86:0/1524029018 shutdown_connections 2024-04-20T21:18:02.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.585+0000 7fcf50e00640 1 --2- 172.21.15.86:0/1524029018 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcf4c0ad7f0 0x7fcf4c06e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:02.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.585+0000 7fcf50e00640 1 --2- 172.21.15.86:0/1524029018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf4c156ae0 0x7fcf4c15d6a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:02.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.585+0000 7fcf50e00640 1 -- 172.21.15.86:0/1524029018 >> 172.21.15.86:0/1524029018 conn(0x7fcf4c0d0170 msgr2=0x7fcf4c0d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:02.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.585+0000 7fcf50e00640 1 -- 172.21.15.86:0/1524029018 shutdown_connections 2024-04-20T21:18:02.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.585+0000 7fcf50e00640 1 -- 172.21.15.86:0/1524029018 wait complete. 2024-04-20T21:18:02.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.586+0000 7fcf50e00640 1 Processor -- start 2024-04-20T21:18:02.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.586+0000 7fcf50e00640 1 -- start start 2024-04-20T21:18:02.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.587+0000 7fcf50e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcf4c0ad7f0 0x7fcf4c073380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:02.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.587+0000 7fcf50e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf4c156ae0 0x7fcf4c145100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:02.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.587+0000 7fcf50e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf4c160080 0x7fcf4c153870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:02.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.587+0000 7fcf50e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4c160500 con 0x7fcf4c160080 2024-04-20T21:18:02.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.587+0000 7fcf50e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4c156ec0 con 0x7fcf4c156ae0 2024-04-20T21:18:02.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.587+0000 7fcf50e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fcf4c06e7d0 con 0x7fcf4c0ad7f0 2024-04-20T21:18:02.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.588+0000 7fcf4be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf4c160080 0x7fcf4c153870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:02.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.588+0000 7fcf4be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf4c160080 0x7fcf4c153870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58124/0 (socket says 172.21.15.86:58124) 2024-04-20T21:18:02.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.588+0000 7fcf4be00640 1 -- 172.21.15.86:0/1817609251 learned_addr learned my addr 172.21.15.86:0/1817609251 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:02.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.588+0000 7fcf4aa00640 1 --2- 172.21.15.86:0/1817609251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf4c156ae0 0x7fcf4c145100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:02.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.588+0000 7fcf4b400640 1 --2- 172.21.15.86:0/1817609251 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcf4c0ad7f0 0x7fcf4c073380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:02.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.588+0000 7fcf4be00640 1 -- 172.21.15.86:0/1817609251 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcf4c0ad7f0 msgr2=0x7fcf4c073380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:02.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.588+0000 7fcf4be00640 1 --2- 172.21.15.86:0/1817609251 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcf4c0ad7f0 0x7fcf4c073380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:02.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.588+0000 7fcf4be00640 1 -- 172.21.15.86:0/1817609251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf4c156ae0 msgr2=0x7fcf4c145100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:02.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.588+0000 7fcf4be00640 1 --2- 172.21.15.86:0/1817609251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf4c156ae0 0x7fcf4c145100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:02.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.588+0000 7fcf4be00640 1 -- 172.21.15.86:0/1817609251 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf4c142ed0 con 0x7fcf4c160080 2024-04-20T21:18:02.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.588+0000 7fcf4aa00640 1 --2- 172.21.15.86:0/1817609251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf4c156ae0 0x7fcf4c145100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:02.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.589+0000 7fcf4be00640 1 --2- 172.21.15.86:0/1817609251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf4c160080 0x7fcf4c153870 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcf4400cd90 tx=0x7fcf4400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:02.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.589+0000 7fcf43400640 1 -- 172.21.15.86:0/1817609251 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf44003b90 con 0x7fcf4c160080 2024-04-20T21:18:02.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.589+0000 7fcf43400640 1 -- 172.21.15.86:0/1817609251 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf44003d50 con 0x7fcf4c160080 2024-04-20T21:18:02.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.589+0000 7fcf50e00640 1 -- 172.21.15.86:0/1817609251 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf4c0ce970 con 0x7fcf4c160080 2024-04-20T21:18:02.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.589+0000 7fcf43400640 1 -- 172.21.15.86:0/1817609251 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf44004260 con 0x7fcf4c160080 2024-04-20T21:18:02.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.589+0000 7fcf50e00640 1 -- 172.21.15.86:0/1817609251 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf4c1547d0 con 0x7fcf4c160080 2024-04-20T21:18:02.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.590+0000 7fcf50e00640 1 -- 172.21.15.86:0/1817609251 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf4c04fcf0 con 0x7fcf4c160080 2024-04-20T21:18:02.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.592+0000 7fcf43400640 1 -- 172.21.15.86:0/1817609251 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcf44004400 con 0x7fcf4c160080 2024-04-20T21:18:02.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.593+0000 7fcf43400640 1 --2- 172.21.15.86:0/1817609251 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcf2c06c8f0 0x7fcf2c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:02.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.593+0000 7fcf43400640 1 -- 172.21.15.86:0/1817609251 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcf4408c730 con 0x7fcf4c160080 2024-04-20T21:18:02.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.593+0000 7fcf4b400640 1 --2- 172.21.15.86:0/1817609251 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcf2c06c8f0 0x7fcf2c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:02.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.594+0000 7fcf4b400640 1 --2- 172.21.15.86:0/1817609251 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcf2c06c8f0 0x7fcf2c06edb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c036000 tx=0x7fcf3c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:02.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.596+0000 7fcf43400640 1 -- 172.21.15.86:0/1817609251 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fcf4405abd0 con 0x7fcf4c160080 2024-04-20T21:18:02.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.779+0000 7fcf50e00640 1 -- 172.21.15.86:0/1817609251 --> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcf4c0cfb30 con 0x7fcf2c06c8f0 2024-04-20T21:18:02.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.792+0000 7fcf43400640 1 -- 172.21.15.86:0/1817609251 <== mgr.4107 v2:172.21.15.86:6832/302045240 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251837 (secure 0 0 0) 0x7fcf3c031760 con 0x7fcf2c06c8f0 2024-04-20T21:18:02.793 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:18:02.796 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-20T21:18:02.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.798+0000 7fcf50e00640 1 -- 172.21.15.86:0/1817609251 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcf2c06c8f0 msgr2=0x7fcf2c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:02.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.798+0000 7fcf50e00640 1 --2- 172.21.15.86:0/1817609251 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcf2c06c8f0 0x7fcf2c06edb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c036000 tx=0x7fcf3c008030 comp rx=0 tx=0).stop 2024-04-20T21:18:02.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.798+0000 7fcf50e00640 1 -- 172.21.15.86:0/1817609251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf4c160080 msgr2=0x7fcf4c153870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:02.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.798+0000 7fcf50e00640 1 --2- 172.21.15.86:0/1817609251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf4c160080 0x7fcf4c153870 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcf4400cd90 tx=0x7fcf4400a660 comp rx=0 tx=0).stop 2024-04-20T21:18:02.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.799+0000 7fcf50e00640 1 -- 172.21.15.86:0/1817609251 shutdown_connections 2024-04-20T21:18:02.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.799+0000 7fcf50e00640 1 --2- 172.21.15.86:0/1817609251 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcf2c06c8f0 0x7fcf2c06edb0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:02.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.799+0000 7fcf50e00640 1 --2- 172.21.15.86:0/1817609251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf4c160080 0x7fcf4c153870 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:02.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.799+0000 7fcf50e00640 1 --2- 172.21.15.86:0/1817609251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf4c156ae0 0x7fcf4c145100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:02.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.799+0000 7fcf50e00640 1 --2- 172.21.15.86:0/1817609251 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcf4c0ad7f0 0x7fcf4c073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:02.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.799+0000 7fcf50e00640 1 -- 172.21.15.86:0/1817609251 >> 172.21.15.86:0/1817609251 conn(0x7fcf4c0d0170 msgr2=0x7fcf4c0aef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:02.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.799+0000 7fcf50e00640 1 -- 172.21.15.86:0/1817609251 shutdown_connections 2024-04-20T21:18:02.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:02.799+0000 7fcf50e00640 1 -- 172.21.15.86:0/1817609251 wait complete. 2024-04-20T21:18:02.814 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-20T21:18:02.633925+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219000,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755720,"statfs":{"total":773094113280,"available":772869857280,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003121"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.607719+0000","last_change":"2024-04-20T21:17:52.309045+0000","last_active":"2024-04-20T21:17:53.607719+0000","last_peered":"2024-04-20T21:17:53.607719+0000","last_clean":"2024-04-20T21:17:53.607719+0000","last_became_active":"2024-04-20T21:17:52.308808+0000","last_became_peered":"2024-04-20T21:17:52.308808+0000","last_unstale":"2024-04-20T21:17:53.607719+0000","last_undegraded":"2024-04-20T21:17:53.607719+0000","last_fullsized":"2024-04-20T21:17:53.607719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:15:54.762913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.281640+0000","last_change":"2024-04-20T21:17:52.303716+0000","last_active":"2024-04-20T21:17:53.281640+0000","last_peered":"2024-04-20T21:17:53.281640+0000","last_clean":"2024-04-20T21:17:53.281640+0000","last_became_active":"2024-04-20T21:17:52.303460+0000","last_became_peered":"2024-04-20T21:17:52.303460+0000","last_unstale":"2024-04-20T21:17:53.281640+0000","last_undegraded":"2024-04-20T21:17:53.281640+0000","last_fullsized":"2024-04-20T21:17:53.281640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:34:47.842666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.684324+0000","last_change":"2024-04-20T21:17:52.297807+0000","last_active":"2024-04-20T21:17:54.684324+0000","last_peered":"2024-04-20T21:17:54.684324+0000","last_clean":"2024-04-20T21:17:54.684324+0000","last_became_active":"2024-04-20T21:17:52.296178+0000","last_became_peered":"2024-04-20T21:17:52.296178+0000","last_unstale":"2024-04-20T21:17:54.684324+0000","last_undegraded":"2024-04-20T21:17:54.684324+0000","last_fullsized":"2024-04-20T21:17:54.684324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:42:57.243810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.357862+0000","last_change":"2024-04-20T21:17:52.315284+0000","last_active":"2024-04-20T21:17:54.357862+0000","last_peered":"2024-04-20T21:17:54.357862+0000","last_clean":"2024-04-20T21:17:54.357862+0000","last_became_active":"2024-04-20T21:17:52.314657+0000","last_became_peered":"2024-04-20T21:17:52.314657+0000","last_unstale":"2024-04-20T21:17:54.357862+0000","last_undegraded":"2024-04-20T21:17:54.357862+0000","last_fullsized":"2024-04-20T21:17:54.357862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:52:22.418925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.606807+0000","last_change":"2024-04-20T21:17:52.299456+0000","last_active":"2024-04-20T21:17:53.606807+0000","last_peered":"2024-04-20T21:17:53.606807+0000","last_clean":"2024-04-20T21:17:53.606807+0000","last_became_active":"2024-04-20T21:17:52.299016+0000","last_became_peered":"2024-04-20T21:17:52.299016+0000","last_unstale":"2024-04-20T21:17:53.606807+0000","last_undegraded":"2024-04-20T21:17:53.606807+0000","last_fullsized":"2024-04-20T21:17:53.606807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:04:26.111984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.357200+0000","last_change":"2024-04-20T21:17:52.305054+0000","last_active":"2024-04-20T21:17:54.357200+0000","last_peered":"2024-04-20T21:17:54.357200+0000","last_clean":"2024-04-20T21:17:54.357200+0000","last_became_active":"2024-04-20T21:17:52.303307+0000","last_became_peered":"2024-04-20T21:17:52.303307+0000","last_unstale":"2024-04-20T21:17:54.357200+0000","last_undegraded":"2024-04-20T21:17:54.357200+0000","last_fullsized":"2024-04-20T21:17:54.357200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:34:47.842666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.606631+0000","last_change":"2024-04-20T21:17:52.306121+0000","last_active":"2024-04-20T21:17:53.606631+0000","last_peered":"2024-04-20T21:17:53.606631+0000","last_clean":"2024-04-20T21:17:53.606631+0000","last_became_active":"2024-04-20T21:17:52.305749+0000","last_became_peered":"2024-04-20T21:17:52.305749+0000","last_unstale":"2024-04-20T21:17:53.606631+0000","last_undegraded":"2024-04-20T21:17:53.606631+0000","last_fullsized":"2024-04-20T21:17:53.606631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:10:11.436747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.682205+0000","last_change":"2024-04-20T21:17:52.292183+0000","last_active":"2024-04-20T21:17:54.682205+0000","last_peered":"2024-04-20T21:17:54.682205+0000","last_clean":"2024-04-20T21:17:54.682205+0000","last_became_active":"2024-04-20T21:17:52.291906+0000","last_became_peered":"2024-04-20T21:17:52.291906+0000","last_unstale":"2024-04-20T21:17:54.682205+0000","last_undegraded":"2024-04-20T21:17:54.682205+0000","last_fullsized":"2024-04-20T21:17:54.682205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:42:57.243810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.607755+0000","last_change":"2024-04-20T21:17:52.316938+0000","last_active":"2024-04-20T21:17:53.607755+0000","last_peered":"2024-04-20T21:17:53.607755+0000","last_clean":"2024-04-20T21:17:53.607755+0000","last_became_active":"2024-04-20T21:17:52.316386+0000","last_became_peered":"2024-04-20T21:17:52.316386+0000","last_unstale":"2024-04-20T21:17:53.607755+0000","last_undegraded":"2024-04-20T21:17:53.607755+0000","last_fullsized":"2024-04-20T21:17:53.607755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:23:58.112035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.289257+0000","last_change":"2024-04-20T21:17:52.299679+0000","last_active":"2024-04-20T21:17:53.289257+0000","last_peered":"2024-04-20T21:17:53.289257+0000","last_clean":"2024-04-20T21:17:53.289257+0000","last_became_active":"2024-04-20T21:17:52.299325+0000","last_became_peered":"2024-04-20T21:17:52.299325+0000","last_unstale":"2024-04-20T21:17:53.289257+0000","last_undegraded":"2024-04-20T21:17:53.289257+0000","last_fullsized":"2024-04-20T21:17:53.289257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:42:57.243810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.289696+0000","last_change":"2024-04-20T21:17:52.298416+0000","last_active":"2024-04-20T21:17:53.289696+0000","last_peered":"2024-04-20T21:17:53.289696+0000","last_clean":"2024-04-20T21:17:53.289696+0000","last_became_active":"2024-04-20T21:17:52.297990+0000","last_became_peered":"2024-04-20T21:17:52.297990+0000","last_unstale":"2024-04-20T21:17:53.289696+0000","last_undegraded":"2024-04-20T21:17:53.289696+0000","last_fullsized":"2024-04-20T21:17:53.289696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:11:50.878621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.287223+0000","last_change":"2024-04-20T21:17:52.279605+0000","last_active":"2024-04-20T21:17:53.287223+0000","last_peered":"2024-04-20T21:17:53.287223+0000","last_clean":"2024-04-20T21:17:53.287223+0000","last_became_active":"2024-04-20T21:17:52.277631+0000","last_became_peered":"2024-04-20T21:17:52.277631+0000","last_unstale":"2024-04-20T21:17:53.287223+0000","last_undegraded":"2024-04-20T21:17:53.287223+0000","last_fullsized":"2024-04-20T21:17:53.287223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:29:18.971384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.680792+0000","last_change":"2024-04-20T21:17:52.300679+0000","last_active":"2024-04-20T21:17:54.680792+0000","last_peered":"2024-04-20T21:17:54.680792+0000","last_clean":"2024-04-20T21:17:54.680792+0000","last_became_active":"2024-04-20T21:17:52.300387+0000","last_became_peered":"2024-04-20T21:17:52.300387+0000","last_unstale":"2024-04-20T21:17:54.680792+0000","last_undegraded":"2024-04-20T21:17:54.680792+0000","last_fullsized":"2024-04-20T21:17:54.680792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:09:02.488057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.682763+0000","last_change":"2024-04-20T21:17:52.301182+0000","last_active":"2024-04-20T21:17:54.682763+0000","last_peered":"2024-04-20T21:17:54.682763+0000","last_clean":"2024-04-20T21:17:54.682763+0000","last_became_active":"2024-04-20T21:17:52.300852+0000","last_became_peered":"2024-04-20T21:17:52.300852+0000","last_unstale":"2024-04-20T21:17:54.682763+0000","last_undegraded":"2024-04-20T21:17:54.682763+0000","last_fullsized":"2024-04-20T21:17:54.682763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:38:56.558892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.281082+0000","last_change":"2024-04-20T21:17:52.282408+0000","last_active":"2024-04-20T21:17:53.281082+0000","last_peered":"2024-04-20T21:17:53.281082+0000","last_clean":"2024-04-20T21:17:53.281082+0000","last_became_active":"2024-04-20T21:17:52.282044+0000","last_became_peered":"2024-04-20T21:17:52.282044+0000","last_unstale":"2024-04-20T21:17:53.281082+0000","last_undegraded":"2024-04-20T21:17:53.281082+0000","last_fullsized":"2024-04-20T21:17:53.281082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:27:15.675834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.357121+0000","last_change":"2024-04-20T21:17:52.315912+0000","last_active":"2024-04-20T21:17:54.357121+0000","last_peered":"2024-04-20T21:17:54.357121+0000","last_clean":"2024-04-20T21:17:54.357121+0000","last_became_active":"2024-04-20T21:17:52.315642+0000","last_became_peered":"2024-04-20T21:17:52.315642+0000","last_unstale":"2024-04-20T21:17:54.357121+0000","last_undegraded":"2024-04-20T21:17:54.357121+0000","last_fullsized":"2024-04-20T21:17:54.357121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:11:50.878621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.358578+0000","last_change":"2024-04-20T21:17:52.315150+0000","last_active":"2024-04-20T21:17:54.358578+0000","last_peered":"2024-04-20T21:17:54.358578+0000","last_clean":"2024-04-20T21:17:54.358578+0000","last_became_active":"2024-04-20T21:17:52.314856+0000","last_became_peered":"2024-04-20T21:17:52.314856+0000","last_unstale":"2024-04-20T21:17:54.358578+0000","last_undegraded":"2024-04-20T21:17:54.358578+0000","last_fullsized":"2024-04-20T21:17:54.358578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:15:54.762913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.679762+0000","last_change":"2024-04-20T21:17:52.286233+0000","last_active":"2024-04-20T21:17:54.679762+0000","last_peered":"2024-04-20T21:17:54.679762+0000","last_clean":"2024-04-20T21:17:54.679762+0000","last_became_active":"2024-04-20T21:17:52.285872+0000","last_became_peered":"2024-04-20T21:17:52.285872+0000","last_unstale":"2024-04-20T21:17:54.679762+0000","last_undegraded":"2024-04-20T21:17:54.679762+0000","last_fullsized":"2024-04-20T21:17:54.679762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:18:32.885585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.607283+0000","last_change":"2024-04-20T21:17:52.308569+0000","last_active":"2024-04-20T21:17:53.607283+0000","last_peered":"2024-04-20T21:17:53.607283+0000","last_clean":"2024-04-20T21:17:53.607283+0000","last_became_active":"2024-04-20T21:17:52.307606+0000","last_became_peered":"2024-04-20T21:17:52.307606+0000","last_unstale":"2024-04-20T21:17:53.607283+0000","last_undegraded":"2024-04-20T21:17:53.607283+0000","last_fullsized":"2024-04-20T21:17:53.607283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:34:46.630106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.288753+0000","last_change":"2024-04-20T21:17:52.281953+0000","last_active":"2024-04-20T21:17:53.288753+0000","last_peered":"2024-04-20T21:17:53.288753+0000","last_clean":"2024-04-20T21:17:53.288753+0000","last_became_active":"2024-04-20T21:17:52.281675+0000","last_became_peered":"2024-04-20T21:17:52.281675+0000","last_unstale":"2024-04-20T21:17:53.288753+0000","last_undegraded":"2024-04-20T21:17:53.288753+0000","last_fullsized":"2024-04-20T21:17:53.288753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:27:15.675834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.683439+0000","last_change":"2024-04-20T21:17:52.308448+0000","last_active":"2024-04-20T21:17:54.683439+0000","last_peered":"2024-04-20T21:17:54.683439+0000","last_clean":"2024-04-20T21:17:54.683439+0000","last_became_active":"2024-04-20T21:17:52.308158+0000","last_became_peered":"2024-04-20T21:17:52.308158+0000","last_unstale":"2024-04-20T21:17:54.683439+0000","last_undegraded":"2024-04-20T21:17:54.683439+0000","last_fullsized":"2024-04-20T21:17:54.683439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:09:02.488057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.357412+0000","last_change":"2024-04-20T21:17:52.304114+0000","last_active":"2024-04-20T21:17:54.357412+0000","last_peered":"2024-04-20T21:17:54.357412+0000","last_clean":"2024-04-20T21:17:54.357412+0000","last_became_active":"2024-04-20T21:17:52.303016+0000","last_became_peered":"2024-04-20T21:17:52.303016+0000","last_unstale":"2024-04-20T21:17:54.357412+0000","last_undegraded":"2024-04-20T21:17:54.357412+0000","last_fullsized":"2024-04-20T21:17:54.357412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:26:30.156714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.681213+0000","last_change":"2024-04-20T21:17:52.297418+0000","last_active":"2024-04-20T21:17:54.681213+0000","last_peered":"2024-04-20T21:17:54.681213+0000","last_clean":"2024-04-20T21:17:54.681213+0000","last_became_active":"2024-04-20T21:17:52.297018+0000","last_became_peered":"2024-04-20T21:17:52.297018+0000","last_unstale":"2024-04-20T21:17:54.681213+0000","last_undegraded":"2024-04-20T21:17:54.681213+0000","last_fullsized":"2024-04-20T21:17:54.681213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:12:56.626507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.357408+0000","last_change":"2024-04-20T21:17:52.306133+0000","last_active":"2024-04-20T21:17:54.357408+0000","last_peered":"2024-04-20T21:17:54.357408+0000","last_clean":"2024-04-20T21:17:54.357408+0000","last_became_active":"2024-04-20T21:17:52.304108+0000","last_became_peered":"2024-04-20T21:17:52.304108+0000","last_unstale":"2024-04-20T21:17:54.357408+0000","last_undegraded":"2024-04-20T21:17:54.357408+0000","last_fullsized":"2024-04-20T21:17:54.357408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:10:11.436747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.683656+0000","last_change":"2024-04-20T21:17:52.301781+0000","last_active":"2024-04-20T21:17:54.683656+0000","last_peered":"2024-04-20T21:17:54.683656+0000","last_clean":"2024-04-20T21:17:54.683656+0000","last_became_active":"2024-04-20T21:17:52.301194+0000","last_became_peered":"2024-04-20T21:17:52.301194+0000","last_unstale":"2024-04-20T21:17:54.683656+0000","last_undegraded":"2024-04-20T21:17:54.683656+0000","last_fullsized":"2024-04-20T21:17:54.683656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:10:11.436747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.289706+0000","last_change":"2024-04-20T21:17:52.303955+0000","last_active":"2024-04-20T21:17:53.289706+0000","last_peered":"2024-04-20T21:17:53.289706+0000","last_clean":"2024-04-20T21:17:53.289706+0000","last_became_active":"2024-04-20T21:17:52.303421+0000","last_became_peered":"2024-04-20T21:17:52.303421+0000","last_unstale":"2024-04-20T21:17:53.289706+0000","last_undegraded":"2024-04-20T21:17:53.289706+0000","last_fullsized":"2024-04-20T21:17:53.289706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:34:47.842666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.281621+0000","last_change":"2024-04-20T21:17:52.304472+0000","last_active":"2024-04-20T21:17:53.281621+0000","last_peered":"2024-04-20T21:17:53.281621+0000","last_clean":"2024-04-20T21:17:53.281621+0000","last_became_active":"2024-04-20T21:17:52.303141+0000","last_became_peered":"2024-04-20T21:17:52.303141+0000","last_unstale":"2024-04-20T21:17:53.281621+0000","last_undegraded":"2024-04-20T21:17:53.281621+0000","last_fullsized":"2024-04-20T21:17:53.281621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:11:50.878621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.681465+0000","last_change":"2024-04-20T21:17:52.302610+0000","last_active":"2024-04-20T21:17:54.681465+0000","last_peered":"2024-04-20T21:17:54.681465+0000","last_clean":"2024-04-20T21:17:54.681465+0000","last_became_active":"2024-04-20T21:17:52.302271+0000","last_became_peered":"2024-04-20T21:17:52.302271+0000","last_unstale":"2024-04-20T21:17:54.681465+0000","last_undegraded":"2024-04-20T21:17:54.681465+0000","last_fullsized":"2024-04-20T21:17:54.681465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:27:15.675834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.683628+0000","last_change":"2024-04-20T21:17:52.307809+0000","last_active":"2024-04-20T21:17:54.683628+0000","last_peered":"2024-04-20T21:17:54.683628+0000","last_clean":"2024-04-20T21:17:54.683628+0000","last_became_active":"2024-04-20T21:17:52.307459+0000","last_became_peered":"2024-04-20T21:17:52.307459+0000","last_unstale":"2024-04-20T21:17:54.683628+0000","last_undegraded":"2024-04-20T21:17:54.683628+0000","last_fullsized":"2024-04-20T21:17:54.683628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:52:26.845286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.357745+0000","last_change":"2024-04-20T21:17:52.305447+0000","last_active":"2024-04-20T21:17:54.357745+0000","last_peered":"2024-04-20T21:17:54.357745+0000","last_clean":"2024-04-20T21:17:54.357745+0000","last_became_active":"2024-04-20T21:17:52.303784+0000","last_became_peered":"2024-04-20T21:17:52.303784+0000","last_unstale":"2024-04-20T21:17:54.357745+0000","last_undegraded":"2024-04-20T21:17:54.357745+0000","last_fullsized":"2024-04-20T21:17:54.357745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:04:26.111984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.679196+0000","last_change":"2024-04-20T21:17:52.294245+0000","last_active":"2024-04-20T21:17:54.679196+0000","last_peered":"2024-04-20T21:17:54.679196+0000","last_clean":"2024-04-20T21:17:54.679196+0000","last_became_active":"2024-04-20T21:17:52.293880+0000","last_became_peered":"2024-04-20T21:17:52.293880+0000","last_unstale":"2024-04-20T21:17:54.679196+0000","last_undegraded":"2024-04-20T21:17:54.679196+0000","last_fullsized":"2024-04-20T21:17:54.679196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:51:58.727652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.287037+0000","last_change":"2024-04-20T21:17:52.303676+0000","last_active":"2024-04-20T21:17:53.287037+0000","last_peered":"2024-04-20T21:17:53.287037+0000","last_clean":"2024-04-20T21:17:53.287037+0000","last_became_active":"2024-04-20T21:17:52.303369+0000","last_became_peered":"2024-04-20T21:17:52.303369+0000","last_unstale":"2024-04-20T21:17:53.287037+0000","last_undegraded":"2024-04-20T21:17:53.287037+0000","last_fullsized":"2024-04-20T21:17:53.287037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:09:02.488057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.287416+0000","last_change":"2024-04-20T21:17:52.277151+0000","last_active":"2024-04-20T21:17:53.287416+0000","last_peered":"2024-04-20T21:17:53.287416+0000","last_clean":"2024-04-20T21:17:53.287416+0000","last_became_active":"2024-04-20T21:17:52.276740+0000","last_became_peered":"2024-04-20T21:17:52.276740+0000","last_unstale":"2024-04-20T21:17:53.287416+0000","last_undegraded":"2024-04-20T21:17:53.287416+0000","last_fullsized":"2024-04-20T21:17:53.287416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:34:27.917112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.680860+0000","last_change":"2024-04-20T21:17:51.275103+0000","last_active":"2024-04-20T21:17:54.680860+0000","last_peered":"2024-04-20T21:17:54.680860+0000","last_clean":"2024-04-20T21:17:54.680860+0000","last_became_active":"2024-04-20T21:17:51.274773+0000","last_became_peered":"2024-04-20T21:17:51.274773+0000","last_unstale":"2024-04-20T21:17:54.680860+0000","last_undegraded":"2024-04-20T21:17:54.680860+0000","last_fullsized":"2024-04-20T21:17:54.680860+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:53:16.864261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.683824+0000","last_change":"2024-04-20T21:17:52.312353+0000","last_active":"2024-04-20T21:17:54.683824+0000","last_peered":"2024-04-20T21:17:54.683824+0000","last_clean":"2024-04-20T21:17:54.683824+0000","last_became_active":"2024-04-20T21:17:52.311933+0000","last_became_peered":"2024-04-20T21:17:52.311933+0000","last_unstale":"2024-04-20T21:17:54.683824+0000","last_undegraded":"2024-04-20T21:17:54.683824+0000","last_fullsized":"2024-04-20T21:17:54.683824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:12:56.626507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.288439+0000","last_change":"2024-04-20T21:17:51.269484+0000","last_active":"2024-04-20T21:17:53.288439+0000","last_peered":"2024-04-20T21:17:53.288439+0000","last_clean":"2024-04-20T21:17:53.288439+0000","last_became_active":"2024-04-20T21:17:51.269196+0000","last_became_peered":"2024-04-20T21:17:51.269196+0000","last_unstale":"2024-04-20T21:17:53.288439+0000","last_undegraded":"2024-04-20T21:17:53.288439+0000","last_fullsized":"2024-04-20T21:17:53.288439+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:18:39.465785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.681496+0000","last_change":"2024-04-20T21:17:51.273330+0000","last_active":"2024-04-20T21:17:54.681496+0000","last_peered":"2024-04-20T21:17:54.681496+0000","last_clean":"2024-04-20T21:17:54.681496+0000","last_became_active":"2024-04-20T21:17:51.273036+0000","last_became_peered":"2024-04-20T21:17:51.273036+0000","last_unstale":"2024-04-20T21:17:54.681496+0000","last_undegraded":"2024-04-20T21:17:54.681496+0000","last_fullsized":"2024-04-20T21:17:54.681496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:07:01.697230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.682607+0000","last_change":"2024-04-20T21:17:52.300897+0000","last_active":"2024-04-20T21:17:54.682607+0000","last_peered":"2024-04-20T21:17:54.682607+0000","last_clean":"2024-04-20T21:17:54.682607+0000","last_became_active":"2024-04-20T21:17:52.300600+0000","last_became_peered":"2024-04-20T21:17:52.300600+0000","last_unstale":"2024-04-20T21:17:54.682607+0000","last_undegraded":"2024-04-20T21:17:54.682607+0000","last_fullsized":"2024-04-20T21:17:54.682607+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:11:50.878621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.357354+0000","last_change":"2024-04-20T21:17:51.276810+0000","last_active":"2024-04-20T21:17:54.357354+0000","last_peered":"2024-04-20T21:17:54.357354+0000","last_clean":"2024-04-20T21:17:54.357354+0000","last_became_active":"2024-04-20T21:17:51.276547+0000","last_became_peered":"2024-04-20T21:17:51.276547+0000","last_unstale":"2024-04-20T21:17:54.357354+0000","last_undegraded":"2024-04-20T21:17:54.357354+0000","last_fullsized":"2024-04-20T21:17:54.357354+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:04:47.881403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.281043+0000","last_change":"2024-04-20T21:17:52.299388+0000","last_active":"2024-04-20T21:17:53.281043+0000","last_peered":"2024-04-20T21:17:53.281043+0000","last_clean":"2024-04-20T21:17:53.281043+0000","last_became_active":"2024-04-20T21:17:52.298531+0000","last_became_peered":"2024-04-20T21:17:52.298531+0000","last_unstale":"2024-04-20T21:17:53.281043+0000","last_undegraded":"2024-04-20T21:17:53.281043+0000","last_fullsized":"2024-04-20T21:17:53.281043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:12:56.626507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.288661+0000","last_change":"2024-04-20T21:17:51.273343+0000","last_active":"2024-04-20T21:17:53.288661+0000","last_peered":"2024-04-20T21:17:53.288661+0000","last_clean":"2024-04-20T21:17:53.288661+0000","last_became_active":"2024-04-20T21:17:51.273018+0000","last_became_peered":"2024-04-20T21:17:51.273018+0000","last_unstale":"2024-04-20T21:17:53.288661+0000","last_undegraded":"2024-04-20T21:17:53.288661+0000","last_fullsized":"2024-04-20T21:17:53.288661+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:34:26.903632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.358664+0000","last_change":"2024-04-20T21:17:52.300438+0000","last_active":"2024-04-20T21:17:54.358664+0000","last_peered":"2024-04-20T21:17:54.358664+0000","last_clean":"2024-04-20T21:17:54.358664+0000","last_became_active":"2024-04-20T21:17:52.300082+0000","last_became_peered":"2024-04-20T21:17:52.300082+0000","last_unstale":"2024-04-20T21:17:54.358664+0000","last_undegraded":"2024-04-20T21:17:54.358664+0000","last_fullsized":"2024-04-20T21:17:54.358664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:23:58.112035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.358530+0000","last_change":"2024-04-20T21:17:51.272721+0000","last_active":"2024-04-20T21:17:54.358530+0000","last_peered":"2024-04-20T21:17:54.358530+0000","last_clean":"2024-04-20T21:17:54.358530+0000","last_became_active":"2024-04-20T21:17:51.272403+0000","last_became_peered":"2024-04-20T21:17:51.272403+0000","last_unstale":"2024-04-20T21:17:54.358530+0000","last_undegraded":"2024-04-20T21:17:54.358530+0000","last_fullsized":"2024-04-20T21:17:54.358530+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:09:01.474577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.684541+0000","last_change":"2024-04-20T21:17:52.316437+0000","last_active":"2024-04-20T21:17:54.684541+0000","last_peered":"2024-04-20T21:17:54.684541+0000","last_clean":"2024-04-20T21:17:54.684541+0000","last_became_active":"2024-04-20T21:17:52.316129+0000","last_became_peered":"2024-04-20T21:17:52.316129+0000","last_unstale":"2024-04-20T21:17:54.684541+0000","last_undegraded":"2024-04-20T21:17:54.684541+0000","last_fullsized":"2024-04-20T21:17:54.684541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:34:47.842666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.281424+0000","last_change":"2024-04-20T21:17:51.268776+0000","last_active":"2024-04-20T21:17:53.281424+0000","last_peered":"2024-04-20T21:17:53.281424+0000","last_clean":"2024-04-20T21:17:53.281424+0000","last_became_active":"2024-04-20T21:17:51.268468+0000","last_became_peered":"2024-04-20T21:17:51.268468+0000","last_unstale":"2024-04-20T21:17:53.281424+0000","last_undegraded":"2024-04-20T21:17:53.281424+0000","last_fullsized":"2024-04-20T21:17:53.281424+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:34:26.903632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.287471+0000","last_change":"2024-04-20T21:17:52.300758+0000","last_active":"2024-04-20T21:17:53.287471+0000","last_peered":"2024-04-20T21:17:53.287471+0000","last_clean":"2024-04-20T21:17:53.287471+0000","last_became_active":"2024-04-20T21:17:52.300476+0000","last_became_peered":"2024-04-20T21:17:52.300476+0000","last_unstale":"2024-04-20T21:17:53.287471+0000","last_undegraded":"2024-04-20T21:17:53.287471+0000","last_fullsized":"2024-04-20T21:17:53.287471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:51:58.727652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.288618+0000","last_change":"2024-04-20T21:17:51.270900+0000","last_active":"2024-04-20T21:17:53.288618+0000","last_peered":"2024-04-20T21:17:53.288618+0000","last_clean":"2024-04-20T21:17:53.288618+0000","last_became_active":"2024-04-20T21:17:51.270618+0000","last_became_peered":"2024-04-20T21:17:51.270618+0000","last_unstale":"2024-04-20T21:17:53.288618+0000","last_undegraded":"2024-04-20T21:17:53.288618+0000","last_fullsized":"2024-04-20T21:17:53.288618+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:04:47.881403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.606885+0000","last_change":"2024-04-20T21:17:52.313555+0000","last_active":"2024-04-20T21:17:53.606885+0000","last_peered":"2024-04-20T21:17:53.606885+0000","last_clean":"2024-04-20T21:17:53.606885+0000","last_became_active":"2024-04-20T21:17:52.312197+0000","last_became_peered":"2024-04-20T21:17:52.312197+0000","last_unstale":"2024-04-20T21:17:53.606885+0000","last_undegraded":"2024-04-20T21:17:53.606885+0000","last_fullsized":"2024-04-20T21:17:53.606885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:52:22.418925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.683925+0000","last_change":"2024-04-20T21:17:51.275126+0000","last_active":"2024-04-20T21:17:54.683925+0000","last_peered":"2024-04-20T21:17:54.683925+0000","last_clean":"2024-04-20T21:17:54.683925+0000","last_became_active":"2024-04-20T21:17:51.273060+0000","last_became_peered":"2024-04-20T21:17:51.273060+0000","last_unstale":"2024-04-20T21:17:54.683925+0000","last_undegraded":"2024-04-20T21:17:54.683925+0000","last_fullsized":"2024-04-20T21:17:54.683925+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:06:19.740358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.683147+0000","last_change":"2024-04-20T21:17:52.302899+0000","last_active":"2024-04-20T21:17:54.683147+0000","last_peered":"2024-04-20T21:17:54.683147+0000","last_clean":"2024-04-20T21:17:54.683147+0000","last_became_active":"2024-04-20T21:17:52.302606+0000","last_became_peered":"2024-04-20T21:17:52.302606+0000","last_unstale":"2024-04-20T21:17:54.683147+0000","last_undegraded":"2024-04-20T21:17:54.683147+0000","last_fullsized":"2024-04-20T21:17:54.683147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:13:52.887276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.357587+0000","last_change":"2024-04-20T21:17:51.278078+0000","last_active":"2024-04-20T21:17:54.357587+0000","last_peered":"2024-04-20T21:17:54.357587+0000","last_clean":"2024-04-20T21:17:54.357587+0000","last_became_active":"2024-04-20T21:17:51.277775+0000","last_became_peered":"2024-04-20T21:17:51.277775+0000","last_unstale":"2024-04-20T21:17:54.357587+0000","last_undegraded":"2024-04-20T21:17:54.357587+0000","last_fullsized":"2024-04-20T21:17:54.357587+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:23:42.473358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.607236+0000","last_change":"2024-04-20T21:17:52.295456+0000","last_active":"2024-04-20T21:17:53.607236+0000","last_peered":"2024-04-20T21:17:53.607236+0000","last_clean":"2024-04-20T21:17:53.607236+0000","last_became_active":"2024-04-20T21:17:52.294825+0000","last_became_peered":"2024-04-20T21:17:52.294825+0000","last_unstale":"2024-04-20T21:17:53.607236+0000","last_undegraded":"2024-04-20T21:17:53.607236+0000","last_fullsized":"2024-04-20T21:17:53.607236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:43:32.983640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.288028+0000","last_change":"2024-04-20T21:17:51.264742+0000","last_active":"2024-04-20T21:17:53.288028+0000","last_peered":"2024-04-20T21:17:53.288028+0000","last_clean":"2024-04-20T21:17:53.288028+0000","last_became_active":"2024-04-20T21:17:51.264009+0000","last_became_peered":"2024-04-20T21:17:51.264009+0000","last_unstale":"2024-04-20T21:17:53.288028+0000","last_undegraded":"2024-04-20T21:17:53.288028+0000","last_fullsized":"2024-04-20T21:17:53.288028+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:29:17.957905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.288331+0000","last_change":"2024-04-20T21:17:52.313950+0000","last_active":"2024-04-20T21:17:53.288331+0000","last_peered":"2024-04-20T21:17:53.288331+0000","last_clean":"2024-04-20T21:17:53.288331+0000","last_became_active":"2024-04-20T21:17:52.311961+0000","last_became_peered":"2024-04-20T21:17:52.311961+0000","last_unstale":"2024-04-20T21:17:53.288331+0000","last_undegraded":"2024-04-20T21:17:53.288331+0000","last_fullsized":"2024-04-20T21:17:53.288331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:12:56.626507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.678630+0000","last_change":"2024-04-20T21:17:51.254940+0000","last_active":"2024-04-20T21:17:54.678630+0000","last_peered":"2024-04-20T21:17:54.678630+0000","last_clean":"2024-04-20T21:17:54.678630+0000","last_became_active":"2024-04-20T21:17:51.254597+0000","last_became_peered":"2024-04-20T21:17:51.254597+0000","last_unstale":"2024-04-20T21:17:54.678630+0000","last_undegraded":"2024-04-20T21:17:54.678630+0000","last_fullsized":"2024-04-20T21:17:54.678630+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:43:48.697965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.606512+0000","last_change":"2024-04-20T21:17:52.303514+0000","last_active":"2024-04-20T21:17:53.606512+0000","last_peered":"2024-04-20T21:17:53.606512+0000","last_clean":"2024-04-20T21:17:53.606512+0000","last_became_active":"2024-04-20T21:17:52.303225+0000","last_became_peered":"2024-04-20T21:17:52.303225+0000","last_unstale":"2024-04-20T21:17:53.606512+0000","last_undegraded":"2024-04-20T21:17:53.606512+0000","last_fullsized":"2024-04-20T21:17:53.606512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:26:30.156714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.287716+0000","last_change":"2024-04-20T21:17:51.276458+0000","last_active":"2024-04-20T21:17:53.287716+0000","last_peered":"2024-04-20T21:17:53.287716+0000","last_clean":"2024-04-20T21:17:53.287716+0000","last_became_active":"2024-04-20T21:17:51.276033+0000","last_became_peered":"2024-04-20T21:17:51.276033+0000","last_unstale":"2024-04-20T21:17:53.287716+0000","last_undegraded":"2024-04-20T21:17:53.287716+0000","last_fullsized":"2024-04-20T21:17:53.287716+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:00:14.426522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.607498+0000","last_change":"2024-04-20T21:17:52.310874+0000","last_active":"2024-04-20T21:17:53.607498+0000","last_peered":"2024-04-20T21:17:53.607498+0000","last_clean":"2024-04-20T21:17:53.607498+0000","last_became_active":"2024-04-20T21:17:52.310612+0000","last_became_peered":"2024-04-20T21:17:52.310612+0000","last_unstale":"2024-04-20T21:17:53.607498+0000","last_undegraded":"2024-04-20T21:17:53.607498+0000","last_fullsized":"2024-04-20T21:17:53.607498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:59:58.666249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.679906+0000","last_change":"2024-04-20T21:17:51.266671+0000","last_active":"2024-04-20T21:17:54.679906+0000","last_peered":"2024-04-20T21:17:54.679906+0000","last_clean":"2024-04-20T21:17:54.679906+0000","last_became_active":"2024-04-20T21:17:51.264486+0000","last_became_peered":"2024-04-20T21:17:51.264486+0000","last_unstale":"2024-04-20T21:17:54.679906+0000","last_undegraded":"2024-04-20T21:17:54.679906+0000","last_fullsized":"2024-04-20T21:17:54.679906+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:53:16.864261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.607949+0000","last_change":"2024-04-20T21:17:52.309913+0000","last_active":"2024-04-20T21:17:53.607949+0000","last_peered":"2024-04-20T21:17:53.607949+0000","last_clean":"2024-04-20T21:17:53.607949+0000","last_became_active":"2024-04-20T21:17:52.309643+0000","last_became_peered":"2024-04-20T21:17:52.309643+0000","last_unstale":"2024-04-20T21:17:53.607949+0000","last_undegraded":"2024-04-20T21:17:53.607949+0000","last_fullsized":"2024-04-20T21:17:53.607949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:48:28.191286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.281734+0000","last_change":"2024-04-20T21:17:51.272265+0000","last_active":"2024-04-20T21:17:53.281734+0000","last_peered":"2024-04-20T21:17:53.281734+0000","last_clean":"2024-04-20T21:17:53.281734+0000","last_became_active":"2024-04-20T21:17:51.271937+0000","last_became_peered":"2024-04-20T21:17:51.271937+0000","last_unstale":"2024-04-20T21:17:53.281734+0000","last_undegraded":"2024-04-20T21:17:53.281734+0000","last_fullsized":"2024-04-20T21:17:53.281734+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:51:57.714172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.358372+0000","last_change":"2024-04-20T21:17:52.304344+0000","last_active":"2024-04-20T21:17:54.358372+0000","last_peered":"2024-04-20T21:17:54.358372+0000","last_clean":"2024-04-20T21:17:54.358372+0000","last_became_active":"2024-04-20T21:17:52.304006+0000","last_became_peered":"2024-04-20T21:17:52.304006+0000","last_unstale":"2024-04-20T21:17:54.358372+0000","last_undegraded":"2024-04-20T21:17:54.358372+0000","last_fullsized":"2024-04-20T21:17:54.358372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:59:58.666249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.679057+0000","last_change":"2024-04-20T21:17:51.272508+0000","last_active":"2024-04-20T21:17:54.679057+0000","last_peered":"2024-04-20T21:17:54.679057+0000","last_clean":"2024-04-20T21:17:54.679057+0000","last_became_active":"2024-04-20T21:17:51.272056+0000","last_became_peered":"2024-04-20T21:17:51.272056+0000","last_unstale":"2024-04-20T21:17:54.679057+0000","last_undegraded":"2024-04-20T21:17:54.679057+0000","last_fullsized":"2024-04-20T21:17:54.679057+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:06:19.740358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.606888+0000","last_change":"2024-04-20T21:17:52.308203+0000","last_active":"2024-04-20T21:17:53.606888+0000","last_peered":"2024-04-20T21:17:53.606888+0000","last_clean":"2024-04-20T21:17:53.606888+0000","last_became_active":"2024-04-20T21:17:52.307941+0000","last_became_peered":"2024-04-20T21:17:52.307941+0000","last_unstale":"2024-04-20T21:17:53.606888+0000","last_undegraded":"2024-04-20T21:17:53.606888+0000","last_fullsized":"2024-04-20T21:17:53.606888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:51:21.491774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.606871+0000","last_change":"2024-04-20T21:17:43.059637+0000","last_active":"2024-04-20T21:17:53.606871+0000","last_peered":"2024-04-20T21:17:53.606871+0000","last_clean":"2024-04-20T21:17:53.606871+0000","last_became_active":"2024-04-20T21:17:43.056207+0000","last_became_peered":"2024-04-20T21:17:43.056207+0000","last_unstale":"2024-04-20T21:17:53.606871+0000","last_undegraded":"2024-04-20T21:17:53.606871+0000","last_fullsized":"2024-04-20T21:17:53.606871+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:41.036290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:41.036290+0000","last_clean_scrub_stamp":"2024-04-20T21:17:41.036290+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:58:51.091337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.357115+0000","last_change":"2024-04-20T21:17:51.277973+0000","last_active":"2024-04-20T21:17:54.357115+0000","last_peered":"2024-04-20T21:17:54.357115+0000","last_clean":"2024-04-20T21:17:54.357115+0000","last_became_active":"2024-04-20T21:17:51.277643+0000","last_became_peered":"2024-04-20T21:17:51.277643+0000","last_unstale":"2024-04-20T21:17:54.357115+0000","last_undegraded":"2024-04-20T21:17:54.357115+0000","last_fullsized":"2024-04-20T21:17:54.357115+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:41:08.560538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.683244+0000","last_change":"2024-04-20T21:17:52.316313+0000","last_active":"2024-04-20T21:17:54.683244+0000","last_peered":"2024-04-20T21:17:54.683244+0000","last_clean":"2024-04-20T21:17:54.683244+0000","last_became_active":"2024-04-20T21:17:52.316054+0000","last_became_peered":"2024-04-20T21:17:52.316054+0000","last_unstale":"2024-04-20T21:17:54.683244+0000","last_undegraded":"2024-04-20T21:17:54.683244+0000","last_fullsized":"2024-04-20T21:17:54.683244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:26:30.156714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.606138+0000","last_change":"2024-04-20T21:17:51.282977+0000","last_active":"2024-04-20T21:17:53.606138+0000","last_peered":"2024-04-20T21:17:53.606138+0000","last_clean":"2024-04-20T21:17:53.606138+0000","last_became_active":"2024-04-20T21:17:51.282699+0000","last_became_peered":"2024-04-20T21:17:51.282699+0000","last_unstale":"2024-04-20T21:17:53.606138+0000","last_undegraded":"2024-04-20T21:17:53.606138+0000","last_fullsized":"2024-04-20T21:17:53.606138+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:04:47.881403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.682600+0000","last_change":"2024-04-20T21:17:52.301574+0000","last_active":"2024-04-20T21:17:54.682600+0000","last_peered":"2024-04-20T21:17:54.682600+0000","last_clean":"2024-04-20T21:17:54.682600+0000","last_became_active":"2024-04-20T21:17:52.299834+0000","last_became_peered":"2024-04-20T21:17:52.299834+0000","last_unstale":"2024-04-20T21:17:54.682600+0000","last_undegraded":"2024-04-20T21:17:54.682600+0000","last_fullsized":"2024-04-20T21:17:54.682600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:23:43.486837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.679134+0000","last_change":"2024-04-20T21:17:51.268326+0000","last_active":"2024-04-20T21:17:54.679134+0000","last_peered":"2024-04-20T21:17:54.679134+0000","last_clean":"2024-04-20T21:17:54.679134+0000","last_became_active":"2024-04-20T21:17:51.268016+0000","last_became_peered":"2024-04-20T21:17:51.268016+0000","last_unstale":"2024-04-20T21:17:54.679134+0000","last_undegraded":"2024-04-20T21:17:54.679134+0000","last_fullsized":"2024-04-20T21:17:54.679134+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:18:39.465785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.280844+0000","last_change":"2024-04-20T21:17:52.289513+0000","last_active":"2024-04-20T21:17:53.280844+0000","last_peered":"2024-04-20T21:17:53.280844+0000","last_clean":"2024-04-20T21:17:53.280844+0000","last_became_active":"2024-04-20T21:17:52.288677+0000","last_became_peered":"2024-04-20T21:17:52.288677+0000","last_unstale":"2024-04-20T21:17:53.280844+0000","last_undegraded":"2024-04-20T21:17:53.280844+0000","last_fullsized":"2024-04-20T21:17:53.280844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:52:26.845286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.358151+0000","last_change":"2024-04-20T21:17:51.275850+0000","last_active":"2024-04-20T21:17:54.358151+0000","last_peered":"2024-04-20T21:17:54.358151+0000","last_clean":"2024-04-20T21:17:54.358151+0000","last_became_active":"2024-04-20T21:17:51.275350+0000","last_became_peered":"2024-04-20T21:17:51.275350+0000","last_unstale":"2024-04-20T21:17:54.358151+0000","last_undegraded":"2024-04-20T21:17:54.358151+0000","last_fullsized":"2024-04-20T21:17:54.358151+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:38:55.545413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.287199+0000","last_change":"2024-04-20T21:17:52.305922+0000","last_active":"2024-04-20T21:17:53.287199+0000","last_peered":"2024-04-20T21:17:53.287199+0000","last_clean":"2024-04-20T21:17:53.287199+0000","last_became_active":"2024-04-20T21:17:52.304705+0000","last_became_peered":"2024-04-20T21:17:52.304705+0000","last_unstale":"2024-04-20T21:17:53.287199+0000","last_undegraded":"2024-04-20T21:17:53.287199+0000","last_fullsized":"2024-04-20T21:17:53.287199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:04:48.894882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.281179+0000","last_change":"2024-04-20T21:17:51.260375+0000","last_active":"2024-04-20T21:17:53.281179+0000","last_peered":"2024-04-20T21:17:53.281179+0000","last_clean":"2024-04-20T21:17:53.281179+0000","last_became_active":"2024-04-20T21:17:51.259993+0000","last_became_peered":"2024-04-20T21:17:51.259993+0000","last_unstale":"2024-04-20T21:17:53.281179+0000","last_undegraded":"2024-04-20T21:17:53.281179+0000","last_fullsized":"2024-04-20T21:17:53.281179+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:04:47.881403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.358295+0000","last_change":"2024-04-20T21:17:52.308697+0000","last_active":"2024-04-20T21:17:54.358295+0000","last_peered":"2024-04-20T21:17:54.358295+0000","last_clean":"2024-04-20T21:17:54.358295+0000","last_became_active":"2024-04-20T21:17:52.308037+0000","last_became_peered":"2024-04-20T21:17:52.308037+0000","last_unstale":"2024-04-20T21:17:54.358295+0000","last_undegraded":"2024-04-20T21:17:54.358295+0000","last_fullsized":"2024-04-20T21:17:54.358295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:34:46.630106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.607409+0000","last_change":"2024-04-20T21:17:51.283426+0000","last_active":"2024-04-20T21:17:53.607409+0000","last_peered":"2024-04-20T21:17:53.607409+0000","last_clean":"2024-04-20T21:17:53.607409+0000","last_became_active":"2024-04-20T21:17:51.283162+0000","last_became_peered":"2024-04-20T21:17:51.283162+0000","last_unstale":"2024-04-20T21:17:53.607409+0000","last_undegraded":"2024-04-20T21:17:53.607409+0000","last_fullsized":"2024-04-20T21:17:53.607409+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:18:31.872106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.281429+0000","last_change":"2024-04-20T21:17:52.285628+0000","last_active":"2024-04-20T21:17:53.281429+0000","last_peered":"2024-04-20T21:17:53.281429+0000","last_clean":"2024-04-20T21:17:53.281429+0000","last_became_active":"2024-04-20T21:17:52.284918+0000","last_became_peered":"2024-04-20T21:17:52.284918+0000","last_unstale":"2024-04-20T21:17:53.281429+0000","last_undegraded":"2024-04-20T21:17:53.281429+0000","last_fullsized":"2024-04-20T21:17:53.281429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:42:57.243810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.281782+0000","last_change":"2024-04-20T21:17:51.271506+0000","last_active":"2024-04-20T21:17:53.281782+0000","last_peered":"2024-04-20T21:17:53.281782+0000","last_clean":"2024-04-20T21:17:53.281782+0000","last_became_active":"2024-04-20T21:17:51.271243+0000","last_became_peered":"2024-04-20T21:17:51.271243+0000","last_unstale":"2024-04-20T21:17:53.281782+0000","last_undegraded":"2024-04-20T21:17:53.281782+0000","last_fullsized":"2024-04-20T21:17:53.281782+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:29:17.957905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.684078+0000","last_change":"2024-04-20T21:17:52.301465+0000","last_active":"2024-04-20T21:17:54.684078+0000","last_peered":"2024-04-20T21:17:54.684078+0000","last_clean":"2024-04-20T21:17:54.684078+0000","last_became_active":"2024-04-20T21:17:52.300849+0000","last_became_peered":"2024-04-20T21:17:52.300849+0000","last_unstale":"2024-04-20T21:17:54.684078+0000","last_undegraded":"2024-04-20T21:17:54.684078+0000","last_fullsized":"2024-04-20T21:17:54.684078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:04:26.111984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.288907+0000","last_change":"2024-04-20T21:17:51.282509+0000","last_active":"2024-04-20T21:17:53.288907+0000","last_peered":"2024-04-20T21:17:53.288907+0000","last_clean":"2024-04-20T21:17:53.288907+0000","last_became_active":"2024-04-20T21:17:51.281343+0000","last_became_peered":"2024-04-20T21:17:51.281343+0000","last_unstale":"2024-04-20T21:17:53.288907+0000","last_undegraded":"2024-04-20T21:17:53.288907+0000","last_fullsized":"2024-04-20T21:17:53.288907+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:43:48.697965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.684280+0000","last_change":"2024-04-20T21:17:52.311617+0000","last_active":"2024-04-20T21:17:54.684280+0000","last_peered":"2024-04-20T21:17:54.684280+0000","last_clean":"2024-04-20T21:17:54.684280+0000","last_became_active":"2024-04-20T21:17:52.311367+0000","last_became_peered":"2024-04-20T21:17:52.311367+0000","last_unstale":"2024-04-20T21:17:54.684280+0000","last_undegraded":"2024-04-20T21:17:54.684280+0000","last_fullsized":"2024-04-20T21:17:54.684280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:27:15.675834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.607489+0000","last_change":"2024-04-20T21:17:51.283039+0000","last_active":"2024-04-20T21:17:53.607489+0000","last_peered":"2024-04-20T21:17:53.607489+0000","last_clean":"2024-04-20T21:17:53.607489+0000","last_became_active":"2024-04-20T21:17:51.282780+0000","last_became_peered":"2024-04-20T21:17:51.282780+0000","last_unstale":"2024-04-20T21:17:53.607489+0000","last_undegraded":"2024-04-20T21:17:53.607489+0000","last_fullsized":"2024-04-20T21:17:53.607489+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:18:39.465785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.286949+0000","last_change":"2024-04-20T21:17:52.293046+0000","last_active":"2024-04-20T21:17:53.286949+0000","last_peered":"2024-04-20T21:17:53.286949+0000","last_clean":"2024-04-20T21:17:53.286949+0000","last_became_active":"2024-04-20T21:17:52.288365+0000","last_became_peered":"2024-04-20T21:17:52.288365+0000","last_unstale":"2024-04-20T21:17:53.286949+0000","last_undegraded":"2024-04-20T21:17:53.286949+0000","last_fullsized":"2024-04-20T21:17:53.286949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:18:40.479264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.281228+0000","last_change":"2024-04-20T21:17:51.280889+0000","last_active":"2024-04-20T21:17:53.281228+0000","last_peered":"2024-04-20T21:17:53.281228+0000","last_clean":"2024-04-20T21:17:53.281228+0000","last_became_active":"2024-04-20T21:17:51.280582+0000","last_became_peered":"2024-04-20T21:17:51.280582+0000","last_unstale":"2024-04-20T21:17:53.281228+0000","last_undegraded":"2024-04-20T21:17:53.281228+0000","last_fullsized":"2024-04-20T21:17:53.281228+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:18:39.465785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.682757+0000","last_change":"2024-04-20T21:17:52.292459+0000","last_active":"2024-04-20T21:17:54.682757+0000","last_peered":"2024-04-20T21:17:54.682757+0000","last_clean":"2024-04-20T21:17:54.682757+0000","last_became_active":"2024-04-20T21:17:52.292182+0000","last_became_peered":"2024-04-20T21:17:52.292182+0000","last_unstale":"2024-04-20T21:17:54.682757+0000","last_undegraded":"2024-04-20T21:17:54.682757+0000","last_fullsized":"2024-04-20T21:17:54.682757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:34:47.842666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.281162+0000","last_change":"2024-04-20T21:17:51.269365+0000","last_active":"2024-04-20T21:17:53.281162+0000","last_peered":"2024-04-20T21:17:53.281162+0000","last_clean":"2024-04-20T21:17:53.281162+0000","last_became_active":"2024-04-20T21:17:51.267902+0000","last_became_peered":"2024-04-20T21:17:51.267902+0000","last_unstale":"2024-04-20T21:17:53.281162+0000","last_undegraded":"2024-04-20T21:17:53.281162+0000","last_fullsized":"2024-04-20T21:17:53.281162+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:43:48.697965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.681157+0000","last_change":"2024-04-20T21:17:52.315096+0000","last_active":"2024-04-20T21:17:54.681157+0000","last_peered":"2024-04-20T21:17:54.681157+0000","last_clean":"2024-04-20T21:17:54.681157+0000","last_became_active":"2024-04-20T21:17:52.314742+0000","last_became_peered":"2024-04-20T21:17:52.314742+0000","last_unstale":"2024-04-20T21:17:54.681157+0000","last_undegraded":"2024-04-20T21:17:54.681157+0000","last_fullsized":"2024-04-20T21:17:54.681157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:52:26.845286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.607525+0000","last_change":"2024-04-20T21:17:51.288583+0000","last_active":"2024-04-20T21:17:53.607525+0000","last_peered":"2024-04-20T21:17:53.607525+0000","last_clean":"2024-04-20T21:17:53.607525+0000","last_became_active":"2024-04-20T21:17:51.288299+0000","last_became_peered":"2024-04-20T21:17:51.288299+0000","last_unstale":"2024-04-20T21:17:53.607525+0000","last_undegraded":"2024-04-20T21:17:53.607525+0000","last_fullsized":"2024-04-20T21:17:53.607525+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:29:17.957905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.288285+0000","last_change":"2024-04-20T21:17:52.283186+0000","last_active":"2024-04-20T21:17:53.288285+0000","last_peered":"2024-04-20T21:17:53.288285+0000","last_clean":"2024-04-20T21:17:53.288285+0000","last_became_active":"2024-04-20T21:17:52.280776+0000","last_became_peered":"2024-04-20T21:17:52.280776+0000","last_unstale":"2024-04-20T21:17:53.288285+0000","last_undegraded":"2024-04-20T21:17:53.288285+0000","last_fullsized":"2024-04-20T21:17:53.288285+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:52:26.845286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.358061+0000","last_change":"2024-04-20T21:17:52.302224+0000","last_active":"2024-04-20T21:17:54.358061+0000","last_peered":"2024-04-20T21:17:54.358061+0000","last_clean":"2024-04-20T21:17:54.358061+0000","last_became_active":"2024-04-20T21:17:52.301869+0000","last_became_peered":"2024-04-20T21:17:52.301869+0000","last_unstale":"2024-04-20T21:17:54.358061+0000","last_undegraded":"2024-04-20T21:17:54.358061+0000","last_fullsized":"2024-04-20T21:17:54.358061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:43:32.983640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.606747+0000","last_change":"2024-04-20T21:17:51.288486+0000","last_active":"2024-04-20T21:17:53.606747+0000","last_peered":"2024-04-20T21:17:53.606747+0000","last_clean":"2024-04-20T21:17:53.606747+0000","last_became_active":"2024-04-20T21:17:51.288131+0000","last_became_peered":"2024-04-20T21:17:51.288131+0000","last_unstale":"2024-04-20T21:17:53.606747+0000","last_undegraded":"2024-04-20T21:17:53.606747+0000","last_fullsized":"2024-04-20T21:17:53.606747+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:51:57.714172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.357879+0000","last_change":"2024-04-20T21:17:52.302297+0000","last_active":"2024-04-20T21:17:54.357879+0000","last_peered":"2024-04-20T21:17:54.357879+0000","last_clean":"2024-04-20T21:17:54.357879+0000","last_became_active":"2024-04-20T21:17:52.301982+0000","last_became_peered":"2024-04-20T21:17:52.301982+0000","last_unstale":"2024-04-20T21:17:54.357879+0000","last_undegraded":"2024-04-20T21:17:54.357879+0000","last_fullsized":"2024-04-20T21:17:54.357879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:51:21.491774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:53.607170+0000","last_change":"2024-04-20T21:17:51.285151+0000","last_active":"2024-04-20T21:17:53.607170+0000","last_peered":"2024-04-20T21:17:53.607170+0000","last_clean":"2024-04-20T21:17:53.607170+0000","last_became_active":"2024-04-20T21:17:51.284879+0000","last_became_peered":"2024-04-20T21:17:51.284879+0000","last_unstale":"2024-04-20T21:17:53.607170+0000","last_undegraded":"2024-04-20T21:17:53.607170+0000","last_fullsized":"2024-04-20T21:17:53.607170+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:34:26.903632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.684538+0000","last_change":"2024-04-20T21:17:52.317666+0000","last_active":"2024-04-20T21:17:54.684538+0000","last_peered":"2024-04-20T21:17:54.684538+0000","last_clean":"2024-04-20T21:17:54.684538+0000","last_became_active":"2024-04-20T21:17:52.316819+0000","last_became_peered":"2024-04-20T21:17:52.316819+0000","last_unstale":"2024-04-20T21:17:54.684538+0000","last_undegraded":"2024-04-20T21:17:54.684538+0000","last_fullsized":"2024-04-20T21:17:54.684538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:11:50.878621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.682331+0000","last_change":"2024-04-20T21:17:51.276188+0000","last_active":"2024-04-20T21:17:54.682331+0000","last_peered":"2024-04-20T21:17:54.682331+0000","last_clean":"2024-04-20T21:17:54.682331+0000","last_became_active":"2024-04-20T21:17:51.275862+0000","last_became_peered":"2024-04-20T21:17:51.275862+0000","last_unstale":"2024-04-20T21:17:54.682331+0000","last_undegraded":"2024-04-20T21:17:54.682331+0000","last_fullsized":"2024-04-20T21:17:54.682331+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:18:39.465785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.684488+0000","last_change":"2024-04-20T21:17:52.310311+0000","last_active":"2024-04-20T21:17:54.684488+0000","last_peered":"2024-04-20T21:17:54.684488+0000","last_clean":"2024-04-20T21:17:54.684488+0000","last_became_active":"2024-04-20T21:17:52.307410+0000","last_became_peered":"2024-04-20T21:17:52.307410+0000","last_unstale":"2024-04-20T21:17:54.684488+0000","last_undegraded":"2024-04-20T21:17:54.684488+0000","last_fullsized":"2024-04-20T21:17:54.684488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:51.228800+0000","last_clean_scrub_stamp":"2024-04-20T21:17:51.228800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:52:22.418925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-20T21:17:54.684750+0000","last_change":"2024-04-20T21:17:51.287639+0000","last_active":"2024-04-20T21:17:54.684750+0000","last_peered":"2024-04-20T21:17:54.684750+0000","last_clean":"2024-04-20T21:17:54.684750+0000","last_became_active":"2024-04-20T21:17:51.287421+0000","last_became_peered":"2024-04-20T21:17:51.287421+0000","last_unstale":"2024-04-20T21:17:54.684750+0000","last_undegraded":"2024-04-20T21:17:54.684750+0000","last_fullsized":"2024-04-20T21:17:54.684750+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:50.215321+0000","last_clean_scrub_stamp":"2024-04-20T21:17:50.215321+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:07:01.697230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T21:18:02.818 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-20T21:18:02.818 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-20T21:18:02.818 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-20T21:18:02.819 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-20T21:18:03.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.047+0000 7fb95de00640 1 Processor -- start 2024-04-20T21:18:03.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.047+0000 7fb95de00640 1 -- start start 2024-04-20T21:18:03.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.049+0000 7fb95de00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb95806fbd0 0x7fb95815c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:03.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.049+0000 7fb95de00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9580ae710 con 0x7fb95815c010 2024-04-20T21:18:03.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.049+0000 7fb95de00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9580ae9f0 con 0x7fb9580af7b0 2024-04-20T21:18:03.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.049+0000 7fb95de00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb9580af320 con 0x7fb95806fbd0 2024-04-20T21:18:03.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.049+0000 7fb95ca00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fb95815c010 0x7fb9580adda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:43610/0 (socket says 172.21.15.86:43610) 2024-04-20T21:18:03.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.049+0000 7fb95ca00640 1 -- 172.21.15.86:0/954501413 learned_addr learned my addr 172.21.15.86:0/954501413 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:18:03.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.049+0000 7fb95d400640 1 --2- 172.21.15.86:0/954501413 >> v2:172.21.15.86:3300/0 conn(0x7fb95806fbd0 0x7fb95815c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:03.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.049+0000 7fb957400640 1 -- 172.21.15.86:0/954501413 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 448300141 0 0) 0x7fb9580ae710 con 0x7fb95815c010 2024-04-20T21:18:03.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.049+0000 7fb957400640 1 -- 172.21.15.86:0/954501413 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb940003520 con 0x7fb95815c010 2024-04-20T21:18:03.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.050+0000 7fb95d400640 1 -- 172.21.15.86:0/954501413 >> v1:172.21.15.144:6789/0 conn(0x7fb9580af7b0 legacy=0x7fb95806f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:03.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.050+0000 7fb95d400640 1 -- 172.21.15.86:0/954501413 >> v1:172.21.15.86:6789/0 conn(0x7fb95815c010 legacy=0x7fb9580adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:03.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.050+0000 7fb957400640 1 -- 172.21.15.86:0/954501413 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4094619152 0 0) 0x7fb940003520 con 0x7fb95815c010 2024-04-20T21:18:03.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.050+0000 7fb95d400640 1 -- 172.21.15.86:0/954501413 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9580aecd0 con 0x7fb95806fbd0 2024-04-20T21:18:03.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.051+0000 7fb95d400640 1 --2- 172.21.15.86:0/954501413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb95806fbd0 0x7fb95815c3f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb950009d90 tx=0x7fb95002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fc16be62825a76 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:03.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.051+0000 7fb957400640 1 -- 172.21.15.86:0/954501413 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb950039070 con 0x7fb95806fbd0 2024-04-20T21:18:03.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.051+0000 7fb957400640 1 -- 172.21.15.86:0/954501413 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb95002c940 con 0x7fb95806fbd0 2024-04-20T21:18:03.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.051+0000 7fb957400640 1 -- 172.21.15.86:0/954501413 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94c001670 con 0x7fb95806fbd0 2024-04-20T21:18:03.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.052+0000 7fb95de00640 1 -- 172.21.15.86:0/954501413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb95806fbd0 msgr2=0x7fb95815c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:03.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.052+0000 7fb95de00640 1 --2- 172.21.15.86:0/954501413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb95806fbd0 0x7fb95815c3f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb950009d90 tx=0x7fb95002be10 comp rx=0 tx=0).stop 2024-04-20T21:18:03.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.052+0000 7fb95de00640 1 -- 172.21.15.86:0/954501413 shutdown_connections 2024-04-20T21:18:03.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.052+0000 7fb95de00640 1 --2- 172.21.15.86:0/954501413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb95806fbd0 0x7fb95815c3f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:03.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.052+0000 7fb95de00640 1 -- 172.21.15.86:0/954501413 >> 172.21.15.86:0/954501413 conn(0x7fb9580d0170 msgr2=0x7fb9580d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:03.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.052+0000 7fb95de00640 1 -- 172.21.15.86:0/954501413 shutdown_connections 2024-04-20T21:18:03.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.052+0000 7fb95de00640 1 -- 172.21.15.86:0/954501413 wait complete. 2024-04-20T21:18:03.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.053+0000 7fb95de00640 1 Processor -- start 2024-04-20T21:18:03.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.054+0000 7fb95de00640 1 -- start start 2024-04-20T21:18:03.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.054+0000 7fb95de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb95806fbd0 0x7fb9580b41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:03.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.054+0000 7fb95de00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb9580af7b0 0x7fb95809d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:03.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.054+0000 7fb95de00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb95815c010 0x7fb958157b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:03.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.054+0000 7fb95de00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb95815c930 con 0x7fb95806fbd0 2024-04-20T21:18:03.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.054+0000 7fb95de00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb9580ae4b0 con 0x7fb95815c010 2024-04-20T21:18:03.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.055+0000 7fb95de00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb95806e3b0 con 0x7fb9580af7b0 2024-04-20T21:18:03.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.055+0000 7fb95ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb95806fbd0 0x7fb9580b41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:03.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.055+0000 7fb95ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb95806fbd0 0x7fb9580b41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58142/0 (socket says 172.21.15.86:58142) 2024-04-20T21:18:03.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.055+0000 7fb95ca00640 1 -- 172.21.15.86:0/417189155 learned_addr learned my addr 172.21.15.86:0/417189155 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:03.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.055+0000 7fb95ca00640 1 -- 172.21.15.86:0/417189155 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb9580af7b0 msgr2=0x7fb95809d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:03.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.055+0000 7fb95ca00640 1 --2- 172.21.15.86:0/417189155 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb9580af7b0 0x7fb95809d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:03.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.055+0000 7fb95ca00640 1 -- 172.21.15.86:0/417189155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb95815c010 msgr2=0x7fb958157b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:03.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.055+0000 7fb95ca00640 1 --2- 172.21.15.86:0/417189155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb95815c010 0x7fb958157b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:03.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.055+0000 7fb95ca00640 1 -- 172.21.15.86:0/417189155 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb95809df20 con 0x7fb95806fbd0 2024-04-20T21:18:03.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.055+0000 7fb95ca00640 1 --2- 172.21.15.86:0/417189155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb95806fbd0 0x7fb9580b41b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb94c00c700 tx=0x7fb94c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:03.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.056+0000 7fb954c00640 1 -- 172.21.15.86:0/417189155 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94c005ad0 con 0x7fb95806fbd0 2024-04-20T21:18:03.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.056+0000 7fb954c00640 1 -- 172.21.15.86:0/417189155 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb94c005c70 con 0x7fb95806fbd0 2024-04-20T21:18:03.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.056+0000 7fb954c00640 1 -- 172.21.15.86:0/417189155 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94c005e60 con 0x7fb95806fbd0 2024-04-20T21:18:03.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.056+0000 7fb95de00640 1 -- 172.21.15.86:0/417189155 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9581f0dc0 con 0x7fb95806fbd0 2024-04-20T21:18:03.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.056+0000 7fb95de00640 1 -- 172.21.15.86:0/417189155 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9581f1380 con 0x7fb95806fbd0 2024-04-20T21:18:03.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.057+0000 7fb95de00640 1 -- 172.21.15.86:0/417189155 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb95804fcf0 con 0x7fb95806fbd0 2024-04-20T21:18:03.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.058+0000 7fb954c00640 1 -- 172.21.15.86:0/417189155 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb94c007290 con 0x7fb95806fbd0 2024-04-20T21:18:03.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.059+0000 7fb954c00640 1 --2- 172.21.15.86:0/417189155 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb93806c8f0 0x7fb93806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:03.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.059+0000 7fb954c00640 1 -- 172.21.15.86:0/417189155 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb94c08b960 con 0x7fb95806fbd0 2024-04-20T21:18:03.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.059+0000 7fb957e00640 1 --2- 172.21.15.86:0/417189155 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb93806c8f0 0x7fb93806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:03.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.060+0000 7fb957e00640 1 --2- 172.21.15.86:0/417189155 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb93806c8f0 0x7fb93806edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb944000f60 tx=0x7fb9440079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:03.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.063+0000 7fb954c00640 1 -- 172.21.15.86:0/417189155 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb94c059d80 con 0x7fb95806fbd0 2024-04-20T21:18:03.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.287+0000 7fb95de00640 1 -- 172.21.15.86:0/417189155 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fb95815b260 con 0x7fb95806fbd0 2024-04-20T21:18:03.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.287+0000 7fb954c00640 1 -- 172.21.15.86:0/417189155 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fb94c059680 con 0x7fb95806fbd0 2024-04-20T21:18:03.287 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:18:03.287 INFO:teuthology.orchestra.run.smithi086.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-20T21:18:03.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.290+0000 7fb95de00640 1 -- 172.21.15.86:0/417189155 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb93806c8f0 msgr2=0x7fb93806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:03.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.290+0000 7fb95de00640 1 --2- 172.21.15.86:0/417189155 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb93806c8f0 0x7fb93806edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb944000f60 tx=0x7fb9440079b0 comp rx=0 tx=0).stop 2024-04-20T21:18:03.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.290+0000 7fb95de00640 1 -- 172.21.15.86:0/417189155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb95806fbd0 msgr2=0x7fb9580b41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:03.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.290+0000 7fb95de00640 1 --2- 172.21.15.86:0/417189155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb95806fbd0 0x7fb9580b41b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb94c00c700 tx=0x7fb94c00cbc0 comp rx=0 tx=0).stop 2024-04-20T21:18:03.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.290+0000 7fb95de00640 1 -- 172.21.15.86:0/417189155 shutdown_connections 2024-04-20T21:18:03.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.290+0000 7fb95de00640 1 --2- 172.21.15.86:0/417189155 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb93806c8f0 0x7fb93806edb0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:03.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.290+0000 7fb95de00640 1 --2- 172.21.15.86:0/417189155 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb95815c010 0x7fb958157b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:03.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.290+0000 7fb95de00640 1 --2- 172.21.15.86:0/417189155 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb9580af7b0 0x7fb95809d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:03.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.290+0000 7fb95de00640 1 --2- 172.21.15.86:0/417189155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb95806fbd0 0x7fb9580b41b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:03.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.290+0000 7fb95de00640 1 -- 172.21.15.86:0/417189155 >> 172.21.15.86:0/417189155 conn(0x7fb9580d0170 msgr2=0x7fb9580b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:03.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.290+0000 7fb95de00640 1 -- 172.21.15.86:0/417189155 shutdown_connections 2024-04-20T21:18:03.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.290+0000 7fb95de00640 1 -- 172.21.15.86:0/417189155 wait complete. 2024-04-20T21:18:03.307 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-20T21:18:03.307 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:18:03.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.498+0000 7f0e1dc00640 1 Processor -- start 2024-04-20T21:18:03.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.498+0000 7f0e1dc00640 1 -- start start 2024-04-20T21:18:03.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.500+0000 7f0e1dc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e180d1ed0 0x7f0e18169900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:03.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.500+0000 7f0e1dc00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e1815eed0 con 0x7f0e180b4030 2024-04-20T21:18:03.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.500+0000 7f0e1dc00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e181627e0 con 0x7f0e1815eaf0 2024-04-20T21:18:03.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.500+0000 7f0e1dc00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0e18163110 con 0x7f0e180d1ed0 2024-04-20T21:18:03.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.500+0000 7f0e0fe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f0e180b4030 0x7f0e180d17c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:43626/0 (socket says 172.21.15.86:43626) 2024-04-20T21:18:03.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.500+0000 7f0e0fe00640 1 -- 172.21.15.86:0/962401393 learned_addr learned my addr 172.21.15.86:0/962401393 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:18:03.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.501+0000 7f0e17400640 1 --2- 172.21.15.86:0/962401393 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e180d1ed0 0x7f0e18169900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:03.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.501+0000 7f0e16000640 1 -- 172.21.15.86:0/962401393 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2297195316 0 0) 0x7f0e1815eed0 con 0x7f0e180b4030 2024-04-20T21:18:03.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.501+0000 7f0e16000640 1 -- 172.21.15.86:0/962401393 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0dfc003520 con 0x7f0e180b4030 2024-04-20T21:18:03.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.501+0000 7f0e16000640 1 -- 172.21.15.86:0/962401393 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1738329335 0 0) 0x7f0dfc003520 con 0x7f0e180b4030 2024-04-20T21:18:03.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.501+0000 7f0e16000640 1 -- 172.21.15.86:0/962401393 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e180d1ed0 msgr2=0x7f0e18169900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:03.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.501+0000 7f0e16000640 1 --2- 172.21.15.86:0/962401393 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e180d1ed0 0x7f0e18169900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:03.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.501+0000 7f0e16000640 1 -- 172.21.15.86:0/962401393 >> v1:172.21.15.144:6789/0 conn(0x7f0e1815eaf0 legacy=0x7f0e180ba1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:03.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.501+0000 7f0e16000640 1 -- 172.21.15.86:0/962401393 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e18162ac0 con 0x7f0e180b4030 2024-04-20T21:18:03.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.501+0000 7f0e16000640 1 -- 172.21.15.86:0/962401393 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f0e00001670 con 0x7f0e180b4030 2024-04-20T21:18:03.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.501+0000 7f0e16000640 1 -- 172.21.15.86:0/962401393 >> v1:172.21.15.86:6789/0 conn(0x7f0e180b4030 legacy=0x7f0e180d17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:03.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.501+0000 7f0e16000640 1 --2- 172.21.15.86:0/962401393 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e180d1ed0 0x7f0dfc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:03.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.501+0000 7f0e16000640 1 --2- 172.21.15.86:0/962401393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dfc003fb0 0x7f0dfc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:03.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.501+0000 7f0e16000640 1 --2- 172.21.15.86:0/962401393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0dfc004d40 0x7f0dfc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:03.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.501+0000 7f0e16000640 1 -- 172.21.15.86:0/962401393 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0e18163110 con 0x7f0dfc003fb0 2024-04-20T21:18:03.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.501+0000 7f0e16000640 1 -- 172.21.15.86:0/962401393 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0dfc00bb90 con 0x7f0dfc004d40 2024-04-20T21:18:03.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.501+0000 7f0e16000640 1 -- 172.21.15.86:0/962401393 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0dfc00bd90 con 0x7f0e180d1ed0 2024-04-20T21:18:03.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.502+0000 7f0e0fe00640 1 --2- 172.21.15.86:0/962401393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dfc003fb0 0x7f0dfc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:03.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.502+0000 7f0e16a00640 1 --2- 172.21.15.86:0/962401393 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e180d1ed0 0x7f0dfc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:03.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.502+0000 7f0e17400640 1 --2- 172.21.15.86:0/962401393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0dfc004d40 0x7f0dfc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:03.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.502+0000 7f0e0fe00640 1 -- 172.21.15.86:0/962401393 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e180d1ed0 msgr2=0x7f0dfc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:03.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.502+0000 7f0e0fe00640 1 --2- 172.21.15.86:0/962401393 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e180d1ed0 0x7f0dfc0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:03.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.502+0000 7f0e0fe00640 1 -- 172.21.15.86:0/962401393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0dfc004d40 msgr2=0x7f0dfc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:03.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.502+0000 7f0e0fe00640 1 --2- 172.21.15.86:0/962401393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0dfc004d40 0x7f0dfc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:03.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.502+0000 7f0e0fe00640 1 -- 172.21.15.86:0/962401393 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dfc003520 con 0x7f0dfc003fb0 2024-04-20T21:18:03.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.502+0000 7f0e17400640 1 --2- 172.21.15.86:0/962401393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0dfc004d40 0x7f0dfc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:03.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.503+0000 7f0e0fe00640 1 --2- 172.21.15.86:0/962401393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dfc003fb0 0x7f0dfc004800 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0e0000c0e0 tx=0x7f0dfc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38ac90b02fece1f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:03.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.503+0000 7f0e16000640 1 -- 172.21.15.86:0/962401393 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e00007c40 con 0x7f0dfc003fb0 2024-04-20T21:18:03.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.503+0000 7f0e16000640 1 -- 172.21.15.86:0/962401393 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e00030070 con 0x7f0dfc003fb0 2024-04-20T21:18:03.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.503+0000 7f0e16000640 1 -- 172.21.15.86:0/962401393 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e00005d70 con 0x7f0dfc003fb0 2024-04-20T21:18:03.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.504+0000 7f0e1dc00640 1 -- 172.21.15.86:0/962401393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dfc003fb0 msgr2=0x7f0dfc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:03.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.504+0000 7f0e1dc00640 1 --2- 172.21.15.86:0/962401393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dfc003fb0 0x7f0dfc004800 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0e0000c0e0 tx=0x7f0dfc005830 comp rx=0 tx=0).stop 2024-04-20T21:18:03.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.504+0000 7f0e17400640 1 -- 172.21.15.86:0/962401393 reap_dead start 2024-04-20T21:18:03.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.504+0000 7f0e1dc00640 1 -- 172.21.15.86:0/962401393 shutdown_connections 2024-04-20T21:18:03.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.504+0000 7f0e1dc00640 1 -- 172.21.15.86:0/962401393 >> 172.21.15.86:0/962401393 conn(0x7f0e180678a0 msgr2=0x7f0e18067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:03.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.504+0000 7f0e1dc00640 1 -- 172.21.15.86:0/962401393 shutdown_connections 2024-04-20T21:18:03.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.504+0000 7f0e1dc00640 1 -- 172.21.15.86:0/962401393 wait complete. 2024-04-20T21:18:03.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.506+0000 7f0e1dc00640 1 Processor -- start 2024-04-20T21:18:03.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.506+0000 7f0e1dc00640 1 -- start start 2024-04-20T21:18:03.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.507+0000 7f0e1dc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dfc003fb0 0x7f0e180cec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:03.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.507+0000 7f0e1dc00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e180cc090 0x7f0e18158fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:03.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.507+0000 7f0e1dc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e1815a5d0 0x7f0e180b6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:03.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.507+0000 7f0e1dc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0e180d3d40 con 0x7f0dfc003fb0 2024-04-20T21:18:03.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.507+0000 7f0e1dc00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0e180b7a00 con 0x7f0e180cc090 2024-04-20T21:18:03.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.507+0000 7f0e1dc00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0e181678f0 con 0x7f0e1815a5d0 2024-04-20T21:18:03.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.507+0000 7f0e16a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dfc003fb0 0x7f0e180cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:03.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.507+0000 7f0e16a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dfc003fb0 0x7f0e180cec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58158/0 (socket says 172.21.15.86:58158) 2024-04-20T21:18:03.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.507+0000 7f0e16a00640 1 -- 172.21.15.86:0/1848311660 learned_addr learned my addr 172.21.15.86:0/1848311660 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:03.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.507+0000 7f0e16a00640 1 -- 172.21.15.86:0/1848311660 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e1815a5d0 msgr2=0x7f0e180b6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:03.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.507+0000 7f0e0fe00640 1 --2- 172.21.15.86:0/1848311660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e180cc090 0x7f0e18158fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:03.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.507+0000 7f0e17400640 1 --2- 172.21.15.86:0/1848311660 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e1815a5d0 0x7f0e180b6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:03.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.507+0000 7f0e16a00640 1 --2- 172.21.15.86:0/1848311660 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e1815a5d0 0x7f0e180b6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:03.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.507+0000 7f0e16a00640 1 -- 172.21.15.86:0/1848311660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e180cc090 msgr2=0x7f0e18158fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:03.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.507+0000 7f0e16a00640 1 --2- 172.21.15.86:0/1848311660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e180cc090 0x7f0e18158fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:03.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.507+0000 7f0e16a00640 1 -- 172.21.15.86:0/1848311660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e180cc510 con 0x7f0dfc003fb0 2024-04-20T21:18:03.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.507+0000 7f0e0fe00640 1 --2- 172.21.15.86:0/1848311660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e180cc090 0x7f0e18158fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:03.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.508+0000 7f0e16a00640 1 --2- 172.21.15.86:0/1848311660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dfc003fb0 0x7f0e180cec60 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0e0800e5b0 tx=0x7f0e0800ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:03.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.508+0000 7f0e0ea00640 1 -- 172.21.15.86:0/1848311660 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e080052e0 con 0x7f0dfc003fb0 2024-04-20T21:18:03.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.508+0000 7f0e0ea00640 1 -- 172.21.15.86:0/1848311660 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e08018070 con 0x7f0dfc003fb0 2024-04-20T21:18:03.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.508+0000 7f0e0ea00640 1 -- 172.21.15.86:0/1848311660 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e08003810 con 0x7f0dfc003fb0 2024-04-20T21:18:03.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.508+0000 7f0e1dc00640 1 -- 172.21.15.86:0/1848311660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e1815ad90 con 0x7f0dfc003fb0 2024-04-20T21:18:03.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.508+0000 7f0e1dc00640 1 -- 172.21.15.86:0/1848311660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e180b4990 con 0x7f0dfc003fb0 2024-04-20T21:18:03.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.509+0000 7f0e1dc00640 1 -- 172.21.15.86:0/1848311660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e180689e0 con 0x7f0dfc003fb0 2024-04-20T21:18:03.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.510+0000 7f0e0ea00640 1 -- 172.21.15.86:0/1848311660 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0e080039b0 con 0x7f0dfc003fb0 2024-04-20T21:18:03.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.511+0000 7f0e0ea00640 1 --2- 172.21.15.86:0/1848311660 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0df406c8f0 0x7f0df406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:03.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.511+0000 7f0e0ea00640 1 -- 172.21.15.86:0/1848311660 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0e0808b780 con 0x7f0dfc003fb0 2024-04-20T21:18:03.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.511+0000 7f0e0fe00640 1 --2- 172.21.15.86:0/1848311660 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0df406c8f0 0x7f0df406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:03.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.512+0000 7f0e0fe00640 1 --2- 172.21.15.86:0/1848311660 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0df406c8f0 0x7f0df406edb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0e00034000 tx=0x7f0e00009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:03.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.515+0000 7f0e0ea00640 1 -- 172.21.15.86:0/1848311660 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0e0800ce60 con 0x7f0dfc003fb0 2024-04-20T21:18:03.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.706+0000 7f0e1dc00640 1 -- 172.21.15.86:0/1848311660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0e180d0490 con 0x7f0dfc003fb0 2024-04-20T21:18:03.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.709+0000 7f0e0ea00640 1 -- 172.21.15.86:0/1848311660 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7f0e08059c40 con 0x7f0dfc003fb0 2024-04-20T21:18:03.709 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:18:03.709 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":20,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:17:53.274462+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:17:49.718799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:17:50.683477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:18:03.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.712+0000 7f0e1dc00640 1 -- 172.21.15.86:0/1848311660 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0df406c8f0 msgr2=0x7f0df406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:03.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.712+0000 7f0e1dc00640 1 --2- 172.21.15.86:0/1848311660 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0df406c8f0 0x7f0df406edb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0e00034000 tx=0x7f0e00009030 comp rx=0 tx=0).stop 2024-04-20T21:18:03.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.712+0000 7f0e1dc00640 1 -- 172.21.15.86:0/1848311660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dfc003fb0 msgr2=0x7f0e180cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:03.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.712+0000 7f0e1dc00640 1 --2- 172.21.15.86:0/1848311660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dfc003fb0 0x7f0e180cec60 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0e0800e5b0 tx=0x7f0e0800ea70 comp rx=0 tx=0).stop 2024-04-20T21:18:03.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.712+0000 7f0e1dc00640 1 -- 172.21.15.86:0/1848311660 shutdown_connections 2024-04-20T21:18:03.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.712+0000 7f0e1dc00640 1 --2- 172.21.15.86:0/1848311660 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0df406c8f0 0x7f0df406edb0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:03.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.712+0000 7f0e1dc00640 1 --2- 172.21.15.86:0/1848311660 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e1815a5d0 0x7f0e180b6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:03.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.712+0000 7f0e1dc00640 1 --2- 172.21.15.86:0/1848311660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e180cc090 0x7f0e18158fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:03.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.712+0000 7f0e1dc00640 1 --2- 172.21.15.86:0/1848311660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0dfc003fb0 0x7f0e180cec60 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:03.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.712+0000 7f0e1dc00640 1 -- 172.21.15.86:0/1848311660 >> 172.21.15.86:0/1848311660 conn(0x7f0e180678a0 msgr2=0x7f0e18067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:03.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.712+0000 7f0e1dc00640 1 -- 172.21.15.86:0/1848311660 shutdown_connections 2024-04-20T21:18:03.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.712+0000 7f0e1dc00640 1 -- 172.21.15.86:0/1848311660 wait complete. 2024-04-20T21:18:03.730 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:17:49.718799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:17:50.683477+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:18:03.731 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:18:03.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.932+0000 7fd695e00640 1 Processor -- start 2024-04-20T21:18:03.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.932+0000 7fd695e00640 1 -- start start 2024-04-20T21:18:03.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.934+0000 7fd695e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd69006ff30 0x7fd690166f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:03.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.934+0000 7fd695e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6900cd080 con 0x7fd690162c60 2024-04-20T21:18:03.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.934+0000 7fd695e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd69006d090 con 0x7fd69006b980 2024-04-20T21:18:03.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.934+0000 7fd695e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fd69006d370 con 0x7fd69006ff30 2024-04-20T21:18:03.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.934+0000 7fd68ea00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd690162c60 0x7fd690166810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:43640/0 (socket says 172.21.15.86:43640) 2024-04-20T21:18:03.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.934+0000 7fd68ea00640 1 -- 172.21.15.86:0/701433456 learned_addr learned my addr 172.21.15.86:0/701433456 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:18:03.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.934+0000 7fd68e000640 1 --2- 172.21.15.86:0/701433456 >> v2:172.21.15.144:3300/0 conn(0x7fd69006ff30 0x7fd690166f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:03.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.935+0000 7fd68d600640 1 -- 172.21.15.86:0/701433456 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 962070353 0 0) 0x7fd6900cd080 con 0x7fd690162c60 2024-04-20T21:18:03.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.935+0000 7fd68d600640 1 -- 172.21.15.86:0/701433456 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd674003520 con 0x7fd690162c60 2024-04-20T21:18:03.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.935+0000 7fd68d600640 1 -- 172.21.15.86:0/701433456 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3650413271 0 0) 0x7fd674003520 con 0x7fd690162c60 2024-04-20T21:18:03.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.935+0000 7fd68d600640 1 -- 172.21.15.86:0/701433456 >> v2:172.21.15.144:3300/0 conn(0x7fd69006ff30 msgr2=0x7fd690166f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:03.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.935+0000 7fd68d600640 1 --2- 172.21.15.86:0/701433456 >> v2:172.21.15.144:3300/0 conn(0x7fd69006ff30 0x7fd690166f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:03.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.935+0000 7fd68d600640 1 -- 172.21.15.86:0/701433456 >> v1:172.21.15.144:6789/0 conn(0x7fd69006b980 legacy=0x7fd69006be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:03.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.935+0000 7fd68d600640 1 -- 172.21.15.86:0/701433456 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd69006e440 con 0x7fd690162c60 2024-04-20T21:18:03.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.935+0000 7fd68d600640 1 -- 172.21.15.86:0/701433456 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fd680001670 con 0x7fd690162c60 2024-04-20T21:18:03.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.935+0000 7fd68d600640 1 -- 172.21.15.86:0/701433456 >> v1:172.21.15.86:6789/0 conn(0x7fd690162c60 legacy=0x7fd690166810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:03.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.935+0000 7fd68d600640 1 --2- 172.21.15.86:0/701433456 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd674003a00 0x7fd674003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:03.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.935+0000 7fd68d600640 1 --2- 172.21.15.86:0/701433456 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd674004800 0x7fd674004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:03.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.935+0000 7fd68d600640 1 --2- 172.21.15.86:0/701433456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd674005210 0x7fd67400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:03.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.935+0000 7fd68d600640 1 -- 172.21.15.86:0/701433456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd69006d370 con 0x7fd674005210 2024-04-20T21:18:03.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.935+0000 7fd68d600640 1 -- 172.21.15.86:0/701433456 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd67400c160 con 0x7fd674004800 2024-04-20T21:18:03.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.935+0000 7fd68d600640 1 -- 172.21.15.86:0/701433456 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd67400c360 con 0x7fd674003a00 2024-04-20T21:18:03.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.935+0000 7fd68f400640 1 --2- 172.21.15.86:0/701433456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd674005210 0x7fd67400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:03.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.936+0000 7fd68e000640 1 --2- 172.21.15.86:0/701433456 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd674004800 0x7fd674004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:03.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.936+0000 7fd68ea00640 1 --2- 172.21.15.86:0/701433456 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd674003a00 0x7fd674003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:03.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.936+0000 7fd68f400640 1 -- 172.21.15.86:0/701433456 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd674003a00 msgr2=0x7fd674003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:03.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.936+0000 7fd68f400640 1 --2- 172.21.15.86:0/701433456 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd674003a00 0x7fd674003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:03.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.936+0000 7fd68f400640 1 -- 172.21.15.86:0/701433456 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd674004800 msgr2=0x7fd674004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:03.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.936+0000 7fd68f400640 1 --2- 172.21.15.86:0/701433456 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd674004800 0x7fd674004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:03.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.936+0000 7fd68f400640 1 -- 172.21.15.86:0/701433456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd674003520 con 0x7fd674005210 2024-04-20T21:18:03.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.937+0000 7fd68f400640 1 -- 172.21.15.86:0/701433456 reap_dead start 2024-04-20T21:18:03.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.937+0000 7fd68f400640 1 --2- 172.21.15.86:0/701433456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd674005210 0x7fd67400b9c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd688001e20 tx=0x7fd674005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c07829409fdb440f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:03.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.937+0000 7fd68d600640 1 -- 172.21.15.86:0/701433456 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd688001a70 con 0x7fd674005210 2024-04-20T21:18:03.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.937+0000 7fd68d600640 1 -- 172.21.15.86:0/701433456 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd688003b70 con 0x7fd674005210 2024-04-20T21:18:03.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.937+0000 7fd68d600640 1 -- 172.21.15.86:0/701433456 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6880050d0 con 0x7fd674005210 2024-04-20T21:18:03.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.938+0000 7fd695e00640 1 -- 172.21.15.86:0/701433456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd674005210 msgr2=0x7fd67400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:03.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.938+0000 7fd695e00640 1 --2- 172.21.15.86:0/701433456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd674005210 0x7fd67400b9c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd688001e20 tx=0x7fd674005830 comp rx=0 tx=0).stop 2024-04-20T21:18:03.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.938+0000 7fd695e00640 1 -- 172.21.15.86:0/701433456 shutdown_connections 2024-04-20T21:18:03.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.938+0000 7fd695e00640 1 --2- 172.21.15.86:0/701433456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd674005210 0x7fd67400b9c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:03.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.938+0000 7fd695e00640 1 -- 172.21.15.86:0/701433456 >> 172.21.15.86:0/701433456 conn(0x7fd6900b3380 msgr2=0x7fd6900d0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:03.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.938+0000 7fd695e00640 1 -- 172.21.15.86:0/701433456 shutdown_connections 2024-04-20T21:18:03.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.938+0000 7fd695e00640 1 -- 172.21.15.86:0/701433456 wait complete. 2024-04-20T21:18:03.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.940+0000 7fd695e00640 1 Processor -- start 2024-04-20T21:18:03.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.940+0000 7fd695e00640 1 -- start start 2024-04-20T21:18:03.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.941+0000 7fd695e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd674005210 0x7fd6900ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:03.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.941+0000 7fd695e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd690150110 0x7fd6900b4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:03.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.941+0000 7fd695e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6900b4db0 0x7fd690165790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:03.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.941+0000 7fd695e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd690166810 con 0x7fd690150110 2024-04-20T21:18:03.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.941+0000 7fd695e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd690163040 con 0x7fd674005210 2024-04-20T21:18:03.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.941+0000 7fd695e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd690166990 con 0x7fd6900b4db0 2024-04-20T21:18:03.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.941+0000 7fd68e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd690150110 0x7fd6900b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:03.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.941+0000 7fd68e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd690150110 0x7fd6900b4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58184/0 (socket says 172.21.15.86:58184) 2024-04-20T21:18:03.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.941+0000 7fd68e000640 1 -- 172.21.15.86:0/1495478580 learned_addr learned my addr 172.21.15.86:0/1495478580 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:03.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.941+0000 7fd68ea00640 1 --2- 172.21.15.86:0/1495478580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd674005210 0x7fd6900ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:03.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.942+0000 7fd68e000640 1 -- 172.21.15.86:0/1495478580 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6900b4db0 msgr2=0x7fd690165790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:03.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.942+0000 7fd68f400640 1 --2- 172.21.15.86:0/1495478580 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6900b4db0 0x7fd690165790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:03.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.942+0000 7fd68e000640 1 --2- 172.21.15.86:0/1495478580 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6900b4db0 0x7fd690165790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:03.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.942+0000 7fd68e000640 1 -- 172.21.15.86:0/1495478580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd674005210 msgr2=0x7fd6900ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:03.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.942+0000 7fd68e000640 1 --2- 172.21.15.86:0/1495478580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd674005210 0x7fd6900ce2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:03.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.942+0000 7fd68e000640 1 -- 172.21.15.86:0/1495478580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6900ce7e0 con 0x7fd690150110 2024-04-20T21:18:03.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.942+0000 7fd68e000640 1 --2- 172.21.15.86:0/1495478580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd690150110 0x7fd6900b4870 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd67c0115a0 tx=0x7fd67c011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:03.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.942+0000 7fd687400640 1 -- 172.21.15.86:0/1495478580 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c015930 con 0x7fd690150110 2024-04-20T21:18:03.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.942+0000 7fd687400640 1 -- 172.21.15.86:0/1495478580 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd67c015ad0 con 0x7fd690150110 2024-04-20T21:18:03.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.942+0000 7fd695e00640 1 -- 172.21.15.86:0/1495478580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6900cd200 con 0x7fd690150110 2024-04-20T21:18:03.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.942+0000 7fd687400640 1 -- 172.21.15.86:0/1495478580 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c01d400 con 0x7fd690150110 2024-04-20T21:18:03.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.943+0000 7fd695e00640 1 -- 172.21.15.86:0/1495478580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6900cd7c0 con 0x7fd690150110 2024-04-20T21:18:03.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.944+0000 7fd695e00640 1 -- 172.21.15.86:0/1495478580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6900690f0 con 0x7fd690150110 2024-04-20T21:18:03.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.945+0000 7fd687400640 1 -- 172.21.15.86:0/1495478580 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd67c005990 con 0x7fd690150110 2024-04-20T21:18:03.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.945+0000 7fd687400640 1 --2- 172.21.15.86:0/1495478580 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd67006c8f0 0x7fd67006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:03.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.945+0000 7fd687400640 1 -- 172.21.15.86:0/1495478580 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd67c08c890 con 0x7fd690150110 2024-04-20T21:18:03.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.946+0000 7fd68ea00640 1 --2- 172.21.15.86:0/1495478580 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd67006c8f0 0x7fd67006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:03.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.946+0000 7fd68ea00640 1 --2- 172.21.15.86:0/1495478580 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd67006c8f0 0x7fd67006edb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd680000c00 tx=0x7fd680009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:03.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:03.950+0000 7fd687400640 1 -- 172.21.15.86:0/1495478580 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd67c022060 con 0x7fd690150110 2024-04-20T21:18:04.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.147+0000 7fd695e00640 1 -- 172.21.15.86:0/1495478580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd6900b5190 con 0x7fd690150110 2024-04-20T21:18:04.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.148+0000 7fd687400640 1 -- 172.21.15.86:0/1495478580 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fd6900ce7e0 con 0x7fd690150110 2024-04-20T21:18:04.148 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:18:04.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.151+0000 7fd695e00640 1 -- 172.21.15.86:0/1495478580 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd67006c8f0 msgr2=0x7fd67006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:04.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.151+0000 7fd695e00640 1 --2- 172.21.15.86:0/1495478580 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd67006c8f0 0x7fd67006edb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd680000c00 tx=0x7fd680009040 comp rx=0 tx=0).stop 2024-04-20T21:18:04.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.151+0000 7fd695e00640 1 -- 172.21.15.86:0/1495478580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd690150110 msgr2=0x7fd6900b4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:04.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.151+0000 7fd695e00640 1 --2- 172.21.15.86:0/1495478580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd690150110 0x7fd6900b4870 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd67c0115a0 tx=0x7fd67c011a60 comp rx=0 tx=0).stop 2024-04-20T21:18:04.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.151+0000 7fd695e00640 1 -- 172.21.15.86:0/1495478580 shutdown_connections 2024-04-20T21:18:04.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.151+0000 7fd695e00640 1 --2- 172.21.15.86:0/1495478580 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd67006c8f0 0x7fd67006edb0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:04.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.151+0000 7fd695e00640 1 --2- 172.21.15.86:0/1495478580 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6900b4db0 0x7fd690165790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:04.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.151+0000 7fd695e00640 1 --2- 172.21.15.86:0/1495478580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd690150110 0x7fd6900b4870 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:04.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.151+0000 7fd695e00640 1 --2- 172.21.15.86:0/1495478580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd674005210 0x7fd6900ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:04.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.151+0000 7fd695e00640 1 -- 172.21.15.86:0/1495478580 >> 172.21.15.86:0/1495478580 conn(0x7fd6900b3380 msgr2=0x7fd690161f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:04.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.151+0000 7fd695e00640 1 -- 172.21.15.86:0/1495478580 shutdown_connections 2024-04-20T21:18:04.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.151+0000 7fd695e00640 1 -- 172.21.15.86:0/1495478580 wait complete. 2024-04-20T21:18:04.169 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:18:04.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.375+0000 7fd6c7e00640 1 Processor -- start 2024-04-20T21:18:04.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.375+0000 7fd6c7e00640 1 -- start start 2024-04-20T21:18:04.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.377+0000 7fd6c7e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd6c80cfc70 0x7fd6c80cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:04.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.377+0000 7fd6c7e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6c80cdbe0 0x7fd6c806eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:04.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.377+0000 7fd6c7e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6c815bcd0 con 0x7fd6c80b3ba0 2024-04-20T21:18:04.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.377+0000 7fd6c7e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fd6c815c8e0 con 0x7fd6c80cfc70 2024-04-20T21:18:04.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.377+0000 7fd6c7e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd6c815bfb0 con 0x7fd6c80cdbe0 2024-04-20T21:18:04.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.377+0000 7fd6c6000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6c80cdbe0 0x7fd6c806eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:04.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.378+0000 7fd6c6a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd6c80cfc70 0x7fd6c80cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:04.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.378+0000 7fd6c6000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6c80cdbe0 0x7fd6c806eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:55794/0 (socket says 172.21.15.86:55794) 2024-04-20T21:18:04.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.378+0000 7fd6c6000640 1 -- 172.21.15.86:0/2627638520 learned_addr learned my addr 172.21.15.86:0/2627638520 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:04.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.378+0000 7fd6c6000640 1 -- 172.21.15.86:0/2627638520 >> v2:172.21.15.144:3300/0 conn(0x7fd6c80cfc70 msgr2=0x7fd6c80cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:04.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.378+0000 7fd6c6000640 1 --2- 172.21.15.86:0/2627638520 >> v2:172.21.15.144:3300/0 conn(0x7fd6c80cfc70 0x7fd6c80cd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:04.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.378+0000 7fd6c6000640 1 -- 172.21.15.86:0/2627638520 >> v1:172.21.15.144:6789/0 conn(0x7fd6c80b3ba0 legacy=0x7fd6c8071ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:04.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.378+0000 7fd6c6000640 1 -- 172.21.15.86:0/2627638520 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6c815c290 con 0x7fd6c80cdbe0 2024-04-20T21:18:04.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.380+0000 7fd6c6000640 1 --2- 172.21.15.86:0/2627638520 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6c80cdbe0 0x7fd6c806eeb0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd6b400bb10 tx=0x7fd6b402e150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a212dbd67ff7d52 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:04.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.380+0000 7fd6c5600640 1 -- 172.21.15.86:0/2627638520 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b403b070 con 0x7fd6c80cdbe0 2024-04-20T21:18:04.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.380+0000 7fd6c5600640 1 -- 172.21.15.86:0/2627638520 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6b4036440 con 0x7fd6c80cdbe0 2024-04-20T21:18:04.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.380+0000 7fd6c5600640 1 -- 172.21.15.86:0/2627638520 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b4036810 con 0x7fd6c80cdbe0 2024-04-20T21:18:04.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.381+0000 7fd6c7e00640 1 -- 172.21.15.86:0/2627638520 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6c80cdbe0 msgr2=0x7fd6c806eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:04.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.381+0000 7fd6c7e00640 1 --2- 172.21.15.86:0/2627638520 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6c80cdbe0 0x7fd6c806eeb0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd6b400bb10 tx=0x7fd6b402e150 comp rx=0 tx=0).stop 2024-04-20T21:18:04.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.381+0000 7fd6c7e00640 1 -- 172.21.15.86:0/2627638520 shutdown_connections 2024-04-20T21:18:04.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.381+0000 7fd6c7e00640 1 --2- 172.21.15.86:0/2627638520 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6c80cdbe0 0x7fd6c806eeb0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:04.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.381+0000 7fd6c7e00640 1 --2- 172.21.15.86:0/2627638520 >> v2:172.21.15.144:3300/0 conn(0x7fd6c80cfc70 0x7fd6c80cd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:04.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.381+0000 7fd6c7e00640 1 -- 172.21.15.86:0/2627638520 >> 172.21.15.86:0/2627638520 conn(0x7fd6c8067cf0 msgr2=0x7fd6c806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:04.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.381+0000 7fd6c7e00640 1 -- 172.21.15.86:0/2627638520 shutdown_connections 2024-04-20T21:18:04.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.381+0000 7fd6c7e00640 1 -- 172.21.15.86:0/2627638520 wait complete. 2024-04-20T21:18:04.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.382+0000 7fd6c7e00640 1 Processor -- start 2024-04-20T21:18:04.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.383+0000 7fd6c7e00640 1 -- start start 2024-04-20T21:18:04.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.384+0000 7fd6c7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6c80b3ba0 0x7fd6c81360f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:04.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.384+0000 7fd6c7e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6c80cdbe0 0x7fd6c8148920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:04.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.384+0000 7fd6c7e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6c80cfc70 0x7fd6c8071ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:04.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.384+0000 7fd6c7e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c80d0050 con 0x7fd6c80b3ba0 2024-04-20T21:18:04.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.384+0000 7fd6c7e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c80b3710 con 0x7fd6c80cdbe0 2024-04-20T21:18:04.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.384+0000 7fd6c7e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd6c80d01d0 con 0x7fd6c80cfc70 2024-04-20T21:18:04.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.384+0000 7fd6c6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6c80b3ba0 0x7fd6c81360f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:04.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.384+0000 7fd6c6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6c80b3ba0 0x7fd6c81360f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58196/0 (socket says 172.21.15.86:58196) 2024-04-20T21:18:04.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.384+0000 7fd6c6a00640 1 -- 172.21.15.86:0/488430135 learned_addr learned my addr 172.21.15.86:0/488430135 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:04.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.384+0000 7fd6c6000640 1 --2- 172.21.15.86:0/488430135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6c80cdbe0 0x7fd6c8148920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:04.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.384+0000 7fd6c6a00640 1 -- 172.21.15.86:0/488430135 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6c80cfc70 msgr2=0x7fd6c8071ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:04.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.384+0000 7fd6c6a00640 1 --2- 172.21.15.86:0/488430135 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6c80cfc70 0x7fd6c8071ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:04.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.384+0000 7fd6c6a00640 1 -- 172.21.15.86:0/488430135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6c80cdbe0 msgr2=0x7fd6c8148920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:04.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.384+0000 7fd6c6a00640 1 --2- 172.21.15.86:0/488430135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6c80cdbe0 0x7fd6c8148920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:04.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.384+0000 7fd6c6a00640 1 -- 172.21.15.86:0/488430135 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6c81efd70 con 0x7fd6c80b3ba0 2024-04-20T21:18:04.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.385+0000 7fd6c6a00640 1 --2- 172.21.15.86:0/488430135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6c80b3ba0 0x7fd6c81360f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd6b800e9e0 tx=0x7fd6b800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:04.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.385+0000 7fd6bea00640 1 -- 172.21.15.86:0/488430135 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b8017070 con 0x7fd6c80b3ba0 2024-04-20T21:18:04.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.385+0000 7fd6bea00640 1 -- 172.21.15.86:0/488430135 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6b800c9d0 con 0x7fd6c80b3ba0 2024-04-20T21:18:04.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.385+0000 7fd6c7e00640 1 -- 172.21.15.86:0/488430135 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6c81f0000 con 0x7fd6c80b3ba0 2024-04-20T21:18:04.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.385+0000 7fd6bea00640 1 -- 172.21.15.86:0/488430135 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b800cdc0 con 0x7fd6c80b3ba0 2024-04-20T21:18:04.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.385+0000 7fd6c7e00640 1 -- 172.21.15.86:0/488430135 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6c81f05c0 con 0x7fd6c80b3ba0 2024-04-20T21:18:04.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.386+0000 7fd6c7e00640 1 -- 172.21.15.86:0/488430135 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6c804fd90 con 0x7fd6c80b3ba0 2024-04-20T21:18:04.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.387+0000 7fd6bea00640 1 -- 172.21.15.86:0/488430135 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd6b801a070 con 0x7fd6c80b3ba0 2024-04-20T21:18:04.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.388+0000 7fd6bea00640 1 --2- 172.21.15.86:0/488430135 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd6a806c8f0 0x7fd6a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:04.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.388+0000 7fd6bea00640 1 -- 172.21.15.86:0/488430135 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd6b808bac0 con 0x7fd6c80b3ba0 2024-04-20T21:18:04.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.388+0000 7fd6c6000640 1 --2- 172.21.15.86:0/488430135 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd6a806c8f0 0x7fd6a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:04.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.389+0000 7fd6c6000640 1 --2- 172.21.15.86:0/488430135 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd6a806c8f0 0x7fd6a806edb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd6b4038000 tx=0x7fd6b4032000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:04.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.392+0000 7fd6bea00640 1 -- 172.21.15.86:0/488430135 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd6b8059f80 con 0x7fd6c80b3ba0 2024-04-20T21:18:04.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.593+0000 7fd6c7e00640 1 -- 172.21.15.86:0/488430135 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd6c806dff0 con 0x7fd6c80b3ba0 2024-04-20T21:18:04.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.593+0000 7fd6bea00640 1 -- 172.21.15.86:0/488430135 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fd6b8059880 con 0x7fd6c80b3ba0 2024-04-20T21:18:04.593 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-20T21:18:04.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.596+0000 7fd6c7e00640 1 -- 172.21.15.86:0/488430135 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd6a806c8f0 msgr2=0x7fd6a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:04.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.596+0000 7fd6c7e00640 1 --2- 172.21.15.86:0/488430135 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd6a806c8f0 0x7fd6a806edb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd6b4038000 tx=0x7fd6b4032000 comp rx=0 tx=0).stop 2024-04-20T21:18:04.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.596+0000 7fd6c7e00640 1 -- 172.21.15.86:0/488430135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6c80b3ba0 msgr2=0x7fd6c81360f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:04.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.596+0000 7fd6c7e00640 1 --2- 172.21.15.86:0/488430135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6c80b3ba0 0x7fd6c81360f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd6b800e9e0 tx=0x7fd6b800eea0 comp rx=0 tx=0).stop 2024-04-20T21:18:04.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.596+0000 7fd6c7e00640 1 -- 172.21.15.86:0/488430135 shutdown_connections 2024-04-20T21:18:04.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.596+0000 7fd6c7e00640 1 --2- 172.21.15.86:0/488430135 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd6a806c8f0 0x7fd6a806edb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:04.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.596+0000 7fd6c7e00640 1 --2- 172.21.15.86:0/488430135 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6c80cfc70 0x7fd6c8071ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:04.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.596+0000 7fd6c7e00640 1 --2- 172.21.15.86:0/488430135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6c80cdbe0 0x7fd6c8148920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:04.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.596+0000 7fd6c7e00640 1 --2- 172.21.15.86:0/488430135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6c80b3ba0 0x7fd6c81360f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:04.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.596+0000 7fd6c7e00640 1 -- 172.21.15.86:0/488430135 >> 172.21.15.86:0/488430135 conn(0x7fd6c8067cf0 msgr2=0x7fd6c815cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:04.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.596+0000 7fd6c7e00640 1 -- 172.21.15.86:0/488430135 shutdown_connections 2024-04-20T21:18:04.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.596+0000 7fd6c7e00640 1 -- 172.21.15.86:0/488430135 wait complete. 2024-04-20T21:18:04.614 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:18:04.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.804+0000 7f361f400640 1 Processor -- start 2024-04-20T21:18:04.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.805+0000 7f361f400640 1 -- start start 2024-04-20T21:18:04.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.806+0000 7f361f400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f361815ee00 0x7f36181634c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:04.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.806+0000 7f361f400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36180cf830 0x7f361815fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:04.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.806+0000 7f361f400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36180cff90 con 0x7f36180d1260 2024-04-20T21:18:04.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.806+0000 7f361f400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f36180d0550 con 0x7f361815ee00 2024-04-20T21:18:04.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.806+0000 7f361f400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f36180d0270 con 0x7f36180cf830 2024-04-20T21:18:04.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.806+0000 7f361d400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f36180d1260 0x7f36180d3040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:43652/0 (socket says 172.21.15.86:43652) 2024-04-20T21:18:04.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.806+0000 7f361d400640 1 -- 172.21.15.86:0/3826819126 learned_addr learned my addr 172.21.15.86:0/3826819126 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:18:04.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.807+0000 7f3617e00640 1 --2- 172.21.15.86:0/3826819126 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36180cf830 0x7f361815fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:04.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.807+0000 7f361ca00640 1 --2- 172.21.15.86:0/3826819126 >> v2:172.21.15.144:3300/0 conn(0x7f361815ee00 0x7f36181634c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:04.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.807+0000 7f3617400640 1 -- 172.21.15.86:0/3826819126 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1357170106 0 0) 0x7f36180cff90 con 0x7f36180d1260 2024-04-20T21:18:04.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.807+0000 7f3617400640 1 -- 172.21.15.86:0/3826819126 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f35fc003520 con 0x7f36180d1260 2024-04-20T21:18:04.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.807+0000 7f3617400640 1 -- 172.21.15.86:0/3826819126 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3776177309 0 0) 0x7f35fc003520 con 0x7f36180d1260 2024-04-20T21:18:04.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.807+0000 7f3617400640 1 -- 172.21.15.86:0/3826819126 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36180cf830 msgr2=0x7f361815fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:04.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.807+0000 7f3617400640 1 --2- 172.21.15.86:0/3826819126 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36180cf830 0x7f361815fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:04.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.807+0000 7f3617400640 1 -- 172.21.15.86:0/3826819126 >> v2:172.21.15.144:3300/0 conn(0x7f361815ee00 msgr2=0x7f36181634c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:04.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.807+0000 7f3617400640 1 --2- 172.21.15.86:0/3826819126 >> v2:172.21.15.144:3300/0 conn(0x7f361815ee00 0x7f36181634c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:04.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.807+0000 7f3617400640 1 -- 172.21.15.86:0/3826819126 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f361815f640 con 0x7f36180d1260 2024-04-20T21:18:04.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.807+0000 7f3617400640 1 -- 172.21.15.86:0/3826819126 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f3610001670 con 0x7f36180d1260 2024-04-20T21:18:04.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.807+0000 7f3617400640 1 -- 172.21.15.86:0/3826819126 >> v1:172.21.15.86:6789/0 conn(0x7f36180d1260 legacy=0x7f36180d3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:04.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.807+0000 7f3617400640 1 --2- 172.21.15.86:0/3826819126 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36180cf830 0x7f35fc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:04.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.807+0000 7f3617400640 1 --2- 172.21.15.86:0/3826819126 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35fc003f80 0x7f35fc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:04.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.807+0000 7f3617400640 1 --2- 172.21.15.86:0/3826819126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35fc004d40 0x7f35fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:04.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.807+0000 7f3617400640 1 -- 172.21.15.86:0/3826819126 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f36180d0550 con 0x7f35fc004d40 2024-04-20T21:18:04.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.807+0000 7f3617400640 1 -- 172.21.15.86:0/3826819126 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f36180d0270 con 0x7f35fc003f80 2024-04-20T21:18:04.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.807+0000 7f3617400640 1 -- 172.21.15.86:0/3826819126 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f35fc00ba70 con 0x7f36180cf830 2024-04-20T21:18:04.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.808+0000 7f361d400640 1 --2- 172.21.15.86:0/3826819126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35fc004d40 0x7f35fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:04.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.808+0000 7f361ca00640 1 --2- 172.21.15.86:0/3826819126 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36180cf830 0x7f35fc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:04.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.808+0000 7f3617e00640 1 --2- 172.21.15.86:0/3826819126 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35fc003f80 0x7f35fc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:04.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.808+0000 7f361d400640 1 -- 172.21.15.86:0/3826819126 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36180cf830 msgr2=0x7f35fc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:04.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.808+0000 7f361d400640 1 --2- 172.21.15.86:0/3826819126 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36180cf830 0x7f35fc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:04.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.808+0000 7f361d400640 1 -- 172.21.15.86:0/3826819126 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35fc003f80 msgr2=0x7f35fc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:04.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.808+0000 7f361d400640 1 --2- 172.21.15.86:0/3826819126 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35fc003f80 0x7f35fc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:04.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.808+0000 7f361d400640 1 -- 172.21.15.86:0/3826819126 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35fc003520 con 0x7f35fc004d40 2024-04-20T21:18:04.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.809+0000 7f361d400640 1 --2- 172.21.15.86:0/3826819126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35fc004d40 0x7f35fc0051c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f361000c160 tx=0x7f35fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb774bd55c4d8d69 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:04.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.810+0000 7f3617400640 1 -- 172.21.15.86:0/3826819126 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3610007c80 con 0x7f35fc004d40 2024-04-20T21:18:04.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.810+0000 7f3617400640 1 -- 172.21.15.86:0/3826819126 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3610030070 con 0x7f35fc004d40 2024-04-20T21:18:04.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.810+0000 7f3617400640 1 -- 172.21.15.86:0/3826819126 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3610005d70 con 0x7f35fc004d40 2024-04-20T21:18:04.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.810+0000 7f361f400640 1 -- 172.21.15.86:0/3826819126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35fc004d40 msgr2=0x7f35fc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:04.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.810+0000 7f361f400640 1 --2- 172.21.15.86:0/3826819126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35fc004d40 0x7f35fc0051c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f361000c160 tx=0x7f35fc005830 comp rx=0 tx=0).stop 2024-04-20T21:18:04.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.810+0000 7f361d400640 1 -- 172.21.15.86:0/3826819126 reap_dead start 2024-04-20T21:18:04.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.810+0000 7f361f400640 1 -- 172.21.15.86:0/3826819126 shutdown_connections 2024-04-20T21:18:04.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.810+0000 7f361f400640 1 -- 172.21.15.86:0/3826819126 >> 172.21.15.86:0/3826819126 conn(0x7f361806cc20 msgr2=0x7f361806d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:04.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.810+0000 7f361f400640 1 -- 172.21.15.86:0/3826819126 shutdown_connections 2024-04-20T21:18:04.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.810+0000 7f361f400640 1 -- 172.21.15.86:0/3826819126 wait complete. 2024-04-20T21:18:04.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.812+0000 7f361f400640 1 Processor -- start 2024-04-20T21:18:04.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.812+0000 7f361f400640 1 -- start start 2024-04-20T21:18:04.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.813+0000 7f361f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35fc004d40 0x7f3618154090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:04.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.813+0000 7f361f400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36180cc470 0x7f36180cd500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:04.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.813+0000 7f361f400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36180cc8f0 0x7f36180cda40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:04.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.813+0000 7f361f400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f361815f360 con 0x7f35fc004d40 2024-04-20T21:18:04.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.813+0000 7f361f400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f361815f1e0 con 0x7f36180cc470 2024-04-20T21:18:04.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.813+0000 7f361f400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f36180d3750 con 0x7f36180cc8f0 2024-04-20T21:18:04.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.813+0000 7f361ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35fc004d40 0x7f3618154090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:04.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.813+0000 7f361ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35fc004d40 0x7f3618154090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58212/0 (socket says 172.21.15.86:58212) 2024-04-20T21:18:04.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.813+0000 7f361ca00640 1 -- 172.21.15.86:0/2945675222 learned_addr learned my addr 172.21.15.86:0/2945675222 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:04.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.814+0000 7f361d400640 1 --2- 172.21.15.86:0/2945675222 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36180cc8f0 0x7f36180cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:04.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.814+0000 7f3617e00640 1 --2- 172.21.15.86:0/2945675222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36180cc470 0x7f36180cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:04.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.814+0000 7f361ca00640 1 -- 172.21.15.86:0/2945675222 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36180cc8f0 msgr2=0x7f36180cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:04.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.814+0000 7f361ca00640 1 --2- 172.21.15.86:0/2945675222 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36180cc8f0 0x7f36180cda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:04.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.814+0000 7f361ca00640 1 -- 172.21.15.86:0/2945675222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36180cc470 msgr2=0x7f36180cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:04.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.814+0000 7f361ca00640 1 --2- 172.21.15.86:0/2945675222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36180cc470 0x7f36180cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:04.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.814+0000 7f361ca00640 1 -- 172.21.15.86:0/2945675222 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3618154b80 con 0x7f35fc004d40 2024-04-20T21:18:04.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.814+0000 7f361d400640 1 --2- 172.21.15.86:0/2945675222 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36180cc8f0 0x7f36180cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:04.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.814+0000 7f3617e00640 1 --2- 172.21.15.86:0/2945675222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36180cc470 0x7f36180cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:04.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.814+0000 7f361ca00640 1 --2- 172.21.15.86:0/2945675222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35fc004d40 0x7f3618154090 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f360000f630 tx=0x7f360000faf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:04.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.814+0000 7f3614c00640 1 -- 172.21.15.86:0/2945675222 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3600019020 con 0x7f35fc004d40 2024-04-20T21:18:04.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.814+0000 7f3614c00640 1 -- 172.21.15.86:0/2945675222 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3600014870 con 0x7f35fc004d40 2024-04-20T21:18:04.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.814+0000 7f3614c00640 1 -- 172.21.15.86:0/2945675222 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3600014c20 con 0x7f35fc004d40 2024-04-20T21:18:04.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.814+0000 7f361f400640 1 -- 172.21.15.86:0/2945675222 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36180ccec0 con 0x7f35fc004d40 2024-04-20T21:18:04.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.815+0000 7f361f400640 1 -- 172.21.15.86:0/2945675222 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36180cd1a0 con 0x7f35fc004d40 2024-04-20T21:18:04.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.816+0000 7f361f400640 1 -- 172.21.15.86:0/2945675222 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36180689e0 con 0x7f35fc004d40 2024-04-20T21:18:04.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.817+0000 7f3614c00640 1 -- 172.21.15.86:0/2945675222 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f360001c020 con 0x7f35fc004d40 2024-04-20T21:18:04.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.817+0000 7f3614c00640 1 --2- 172.21.15.86:0/2945675222 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f35f406c8f0 0x7f35f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:04.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.817+0000 7f3614c00640 1 -- 172.21.15.86:0/2945675222 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f360008c4f0 con 0x7f35fc004d40 2024-04-20T21:18:04.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.818+0000 7f3617e00640 1 --2- 172.21.15.86:0/2945675222 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f35f406c8f0 0x7f35f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:04.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.818+0000 7f3617e00640 1 --2- 172.21.15.86:0/2945675222 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f35f406c8f0 0x7f35f406edb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3618161060 tx=0x7f360800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:04.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:04.822+0000 7f3614c00640 1 -- 172.21.15.86:0/2945675222 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3600023020 con 0x7f35fc004d40 2024-04-20T21:18:05.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.020+0000 7f361f400640 1 -- 172.21.15.86:0/2945675222 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f36180d29c0 con 0x7f35fc004d40 2024-04-20T21:18:05.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.021+0000 7f3614c00640 1 -- 172.21.15.86:0/2945675222 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f360005a880 con 0x7f35fc004d40 2024-04-20T21:18:05.021 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-20T21:18:05.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.024+0000 7f361f400640 1 -- 172.21.15.86:0/2945675222 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f35f406c8f0 msgr2=0x7f35f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:05.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.024+0000 7f361f400640 1 --2- 172.21.15.86:0/2945675222 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f35f406c8f0 0x7f35f406edb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3618161060 tx=0x7f360800b040 comp rx=0 tx=0).stop 2024-04-20T21:18:05.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.024+0000 7f361f400640 1 -- 172.21.15.86:0/2945675222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35fc004d40 msgr2=0x7f3618154090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:05.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.024+0000 7f361f400640 1 --2- 172.21.15.86:0/2945675222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35fc004d40 0x7f3618154090 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f360000f630 tx=0x7f360000faf0 comp rx=0 tx=0).stop 2024-04-20T21:18:05.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.024+0000 7f361f400640 1 -- 172.21.15.86:0/2945675222 shutdown_connections 2024-04-20T21:18:05.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.024+0000 7f361f400640 1 --2- 172.21.15.86:0/2945675222 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f35f406c8f0 0x7f35f406edb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:05.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.024+0000 7f361f400640 1 --2- 172.21.15.86:0/2945675222 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f36180cc8f0 0x7f36180cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:05.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.024+0000 7f361f400640 1 --2- 172.21.15.86:0/2945675222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36180cc470 0x7f36180cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:05.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.024+0000 7f361f400640 1 --2- 172.21.15.86:0/2945675222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35fc004d40 0x7f3618154090 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:05.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.024+0000 7f361f400640 1 -- 172.21.15.86:0/2945675222 >> 172.21.15.86:0/2945675222 conn(0x7f361806cc20 msgr2=0x7f361815e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:05.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.024+0000 7f361f400640 1 -- 172.21.15.86:0/2945675222 shutdown_connections 2024-04-20T21:18:05.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.024+0000 7f361f400640 1 -- 172.21.15.86:0/2945675222 wait complete. 2024-04-20T21:18:05.041 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:18:05.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.243+0000 7fd883800640 1 Processor -- start 2024-04-20T21:18:05.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.243+0000 7fd883800640 1 -- start start 2024-04-20T21:18:05.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.244+0000 7fd883800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd87c15ca60 0x7fd87c160400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:05.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.244+0000 7fd883800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd87c0d0160 con 0x7fd87c15c680 2024-04-20T21:18:05.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.245+0000 7fd883800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd87c0d0440 con 0x7fd87c160940 2024-04-20T21:18:05.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.245+0000 7fd883800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd87c0b3f00 con 0x7fd87c15ca60 2024-04-20T21:18:05.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.245+0000 7fd880e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd87c15c680 0x7fd87c0cdc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:43664/0 (socket says 172.21.15.86:43664) 2024-04-20T21:18:05.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.245+0000 7fd880e00640 1 -- 172.21.15.86:0/636236969 learned_addr learned my addr 172.21.15.86:0/636236969 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:18:05.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.245+0000 7fd873e00640 1 --2- 172.21.15.86:0/636236969 >> v2:172.21.15.86:3300/0 conn(0x7fd87c15ca60 0x7fd87c160400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:05.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.245+0000 7fd87be00640 1 -- 172.21.15.86:0/636236969 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3603311362 0 0) 0x7fd87c0d0160 con 0x7fd87c15c680 2024-04-20T21:18:05.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.245+0000 7fd87be00640 1 -- 172.21.15.86:0/636236969 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd860003520 con 0x7fd87c15c680 2024-04-20T21:18:05.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.245+0000 7fd873e00640 1 -- 172.21.15.86:0/636236969 >> v1:172.21.15.144:6789/0 conn(0x7fd87c160940 legacy=0x7fd87c166730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:05.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.245+0000 7fd873e00640 1 -- 172.21.15.86:0/636236969 >> v1:172.21.15.86:6789/0 conn(0x7fd87c15c680 legacy=0x7fd87c0cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:05.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.245+0000 7fd873e00640 1 -- 172.21.15.86:0/636236969 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd87c0d0720 con 0x7fd87c15ca60 2024-04-20T21:18:05.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.246+0000 7fd873e00640 1 --2- 172.21.15.86:0/636236969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd87c15ca60 0x7fd87c160400 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd8640097a0 tx=0x7fd86402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7adb5ca1c4047b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:05.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.247+0000 7fd87be00640 1 -- 172.21.15.86:0/636236969 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd864035020 con 0x7fd87c15ca60 2024-04-20T21:18:05.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.247+0000 7fd87be00640 1 -- 172.21.15.86:0/636236969 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd864031830 con 0x7fd87c15ca60 2024-04-20T21:18:05.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.247+0000 7fd87be00640 1 -- 172.21.15.86:0/636236969 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd864031be0 con 0x7fd87c15ca60 2024-04-20T21:18:05.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.247+0000 7fd883800640 1 -- 172.21.15.86:0/636236969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd87c15ca60 msgr2=0x7fd87c160400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:05.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.247+0000 7fd883800640 1 --2- 172.21.15.86:0/636236969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd87c15ca60 0x7fd87c160400 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd8640097a0 tx=0x7fd86402b8d0 comp rx=0 tx=0).stop 2024-04-20T21:18:05.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.247+0000 7fd883800640 1 -- 172.21.15.86:0/636236969 shutdown_connections 2024-04-20T21:18:05.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.247+0000 7fd883800640 1 --2- 172.21.15.86:0/636236969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd87c15ca60 0x7fd87c160400 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:05.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.247+0000 7fd883800640 1 -- 172.21.15.86:0/636236969 >> 172.21.15.86:0/636236969 conn(0x7fd87c0b4080 msgr2=0x7fd87c0b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:05.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.247+0000 7fd883800640 1 -- 172.21.15.86:0/636236969 shutdown_connections 2024-04-20T21:18:05.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.248+0000 7fd883800640 1 -- 172.21.15.86:0/636236969 wait complete. 2024-04-20T21:18:05.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.249+0000 7fd883800640 1 Processor -- start 2024-04-20T21:18:05.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.249+0000 7fd883800640 1 -- start start 2024-04-20T21:18:05.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.250+0000 7fd883800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd87c15c680 0x7fd87c0cbb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:05.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.250+0000 7fd883800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd87c15ca60 0x7fd87c0b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:05.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.250+0000 7fd883800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd87c160940 0x7fd87c0b56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:05.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.250+0000 7fd883800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c0d1050 con 0x7fd87c160940 2024-04-20T21:18:05.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.250+0000 7fd883800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c160d90 con 0x7fd87c15c680 2024-04-20T21:18:05.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.250+0000 7fd883800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd87c0d0ce0 con 0x7fd87c15ca60 2024-04-20T21:18:05.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.251+0000 7fd881800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd87c160940 0x7fd87c0b56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:05.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.251+0000 7fd881800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd87c160940 0x7fd87c0b56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58222/0 (socket says 172.21.15.86:58222) 2024-04-20T21:18:05.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.251+0000 7fd881800640 1 -- 172.21.15.86:0/3477971189 learned_addr learned my addr 172.21.15.86:0/3477971189 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:05.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.251+0000 7fd873e00640 1 --2- 172.21.15.86:0/3477971189 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd87c15ca60 0x7fd87c0b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:05.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.251+0000 7fd880e00640 1 --2- 172.21.15.86:0/3477971189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd87c15c680 0x7fd87c0cbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:05.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.251+0000 7fd881800640 1 -- 172.21.15.86:0/3477971189 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd87c15ca60 msgr2=0x7fd87c0b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:05.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.251+0000 7fd881800640 1 --2- 172.21.15.86:0/3477971189 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd87c15ca60 0x7fd87c0b51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:05.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.251+0000 7fd881800640 1 -- 172.21.15.86:0/3477971189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd87c15c680 msgr2=0x7fd87c0cbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:05.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.251+0000 7fd881800640 1 --2- 172.21.15.86:0/3477971189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd87c15c680 0x7fd87c0cbb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:05.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.251+0000 7fd881800640 1 -- 172.21.15.86:0/3477971189 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd87c15d5b0 con 0x7fd87c160940 2024-04-20T21:18:05.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.251+0000 7fd880e00640 1 --2- 172.21.15.86:0/3477971189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd87c15c680 0x7fd87c0cbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:05.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.251+0000 7fd881800640 1 --2- 172.21.15.86:0/3477971189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd87c160940 0x7fd87c0b56f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd87400c410 tx=0x7fd87400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:05.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.251+0000 7fd879600640 1 -- 172.21.15.86:0/3477971189 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd874011790 con 0x7fd87c160940 2024-04-20T21:18:05.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.251+0000 7fd879600640 1 -- 172.21.15.86:0/3477971189 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd874017070 con 0x7fd87c160940 2024-04-20T21:18:05.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.252+0000 7fd879600640 1 -- 172.21.15.86:0/3477971189 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd874011bb0 con 0x7fd87c160940 2024-04-20T21:18:05.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.252+0000 7fd883800640 1 -- 172.21.15.86:0/3477971189 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd87c15d890 con 0x7fd87c160940 2024-04-20T21:18:05.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.252+0000 7fd883800640 1 -- 172.21.15.86:0/3477971189 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd87c15de50 con 0x7fd87c160940 2024-04-20T21:18:05.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.253+0000 7fd883800640 1 -- 172.21.15.86:0/3477971189 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd87c06c5f0 con 0x7fd87c160940 2024-04-20T21:18:05.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.254+0000 7fd879600640 1 -- 172.21.15.86:0/3477971189 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd87401c460 con 0x7fd87c160940 2024-04-20T21:18:05.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.255+0000 7fd879600640 1 --2- 172.21.15.86:0/3477971189 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd85806c8f0 0x7fd85806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:05.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.255+0000 7fd879600640 1 -- 172.21.15.86:0/3477971189 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd87408b790 con 0x7fd87c160940 2024-04-20T21:18:05.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.255+0000 7fd880e00640 1 --2- 172.21.15.86:0/3477971189 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd85806c8f0 0x7fd85806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:05.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.256+0000 7fd880e00640 1 --2- 172.21.15.86:0/3477971189 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd85806c8f0 0x7fd85806edb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd86c005c00 tx=0x7fd86c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:05.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.259+0000 7fd879600640 1 -- 172.21.15.86:0/3477971189 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd874021020 con 0x7fd87c160940 2024-04-20T21:18:05.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.501+0000 7fd883800640 1 -- 172.21.15.86:0/3477971189 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd87c0690f0 con 0x7fd87c160940 2024-04-20T21:18:05.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.503+0000 7fd879600640 1 -- 172.21.15.86:0/3477971189 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5196 (secure 0 0 0) 0x7fd874059bc0 con 0x7fd87c160940 2024-04-20T21:18:05.502 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:18:05.502 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/179773217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":179773217},{"type":"v1","addr":"172.21.15.144:6834","nonce":179773217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3179785507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3179785507},{"type":"v1","addr":"172.21.15.86:6835","nonce":3179785507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4270,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2780853729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2780853729},{"type":"v1","addr":"172.21.15.86:6837","nonce":2780853729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:17:51.700362+0000","modified":"2024-04-20T21:17:54.973792+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6835/2436181584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2436181584},{"type":"v1","addr":"172.21.15.144:6835","nonce":2436181584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4272,"qdb_cluster":[4272]},"id":1}]} 2024-04-20T21:18:05.503 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 7 2024-04-20T21:18:05.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.505+0000 7fd883800640 1 -- 172.21.15.86:0/3477971189 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd85806c8f0 msgr2=0x7fd85806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:05.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.505+0000 7fd883800640 1 --2- 172.21.15.86:0/3477971189 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd85806c8f0 0x7fd85806edb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd86c005c00 tx=0x7fd86c009040 comp rx=0 tx=0).stop 2024-04-20T21:18:05.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.505+0000 7fd883800640 1 -- 172.21.15.86:0/3477971189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd87c160940 msgr2=0x7fd87c0b56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:05.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.505+0000 7fd883800640 1 --2- 172.21.15.86:0/3477971189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd87c160940 0x7fd87c0b56f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd87400c410 tx=0x7fd87400c8d0 comp rx=0 tx=0).stop 2024-04-20T21:18:05.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.506+0000 7fd883800640 1 -- 172.21.15.86:0/3477971189 shutdown_connections 2024-04-20T21:18:05.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.506+0000 7fd883800640 1 --2- 172.21.15.86:0/3477971189 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd85806c8f0 0x7fd85806edb0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:05.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.506+0000 7fd883800640 1 --2- 172.21.15.86:0/3477971189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd87c160940 0x7fd87c0b56f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:05.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.506+0000 7fd883800640 1 --2- 172.21.15.86:0/3477971189 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd87c15ca60 0x7fd87c0b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:05.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.506+0000 7fd883800640 1 --2- 172.21.15.86:0/3477971189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd87c15c680 0x7fd87c0cbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:05.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.506+0000 7fd883800640 1 -- 172.21.15.86:0/3477971189 >> 172.21.15.86:0/3477971189 conn(0x7fd87c0b4080 msgr2=0x7fd87c0b08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:05.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.506+0000 7fd883800640 1 -- 172.21.15.86:0/3477971189 shutdown_connections 2024-04-20T21:18:05.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.506+0000 7fd883800640 1 -- 172.21.15.86:0/3477971189 wait complete. 2024-04-20T21:18:05.524 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:18:05.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.724+0000 7ff298c00640 1 Processor -- start 2024-04-20T21:18:05.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.724+0000 7ff298c00640 1 -- start start 2024-04-20T21:18:05.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.726+0000 7ff298c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff29415eaf0 0x7ff2940ba1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:05.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.726+0000 7ff298c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff2940b4030 0x7ff2940d17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:05.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.726+0000 7ff298c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff29415eed0 con 0x7ff2940d1d00 2024-04-20T21:18:05.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.726+0000 7ff298c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff294163110 con 0x7ff29415eaf0 2024-04-20T21:18:05.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.726+0000 7ff298c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7ff2940b7a00 con 0x7ff2940b4030 2024-04-20T21:18:05.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.726+0000 7ff291e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff29415eaf0 0x7ff2940ba1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:05.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.726+0000 7ff291e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff29415eaf0 0x7ff2940ba1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58232/0 (socket says 172.21.15.86:58232) 2024-04-20T21:18:05.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.726+0000 7ff291e00640 1 -- 172.21.15.86:0/3907909763 learned_addr learned my addr 172.21.15.86:0/3907909763 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:05.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.726+0000 7ff291400640 1 --2- 172.21.15.86:0/3907909763 >> v2:172.21.15.144:3300/0 conn(0x7ff2940b4030 0x7ff2940d17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:05.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.727+0000 7ff291e00640 1 -- 172.21.15.86:0/3907909763 >> v2:172.21.15.144:3300/0 conn(0x7ff2940b4030 msgr2=0x7ff2940d17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:05.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.727+0000 7ff291e00640 1 --2- 172.21.15.86:0/3907909763 >> v2:172.21.15.144:3300/0 conn(0x7ff2940b4030 0x7ff2940d17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:05.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.727+0000 7ff291e00640 1 -- 172.21.15.86:0/3907909763 >> v1:172.21.15.144:6789/0 conn(0x7ff2940d1d00 legacy=0x7ff2940b6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:05.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.727+0000 7ff291e00640 1 -- 172.21.15.86:0/3907909763 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2940d2110 con 0x7ff29415eaf0 2024-04-20T21:18:05.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.728+0000 7ff291e00640 1 --2- 172.21.15.86:0/3907909763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff29415eaf0 0x7ff2940ba1f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff29406d8e0 tx=0x7ff28402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfb15e86eca7ddf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:05.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.728+0000 7ff290a00640 1 -- 172.21.15.86:0/3907909763 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28402caf0 con 0x7ff29415eaf0 2024-04-20T21:18:05.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.728+0000 7ff290a00640 1 -- 172.21.15.86:0/3907909763 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff28402cc90 con 0x7ff29415eaf0 2024-04-20T21:18:05.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.728+0000 7ff290a00640 1 -- 172.21.15.86:0/3907909763 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff284034610 con 0x7ff29415eaf0 2024-04-20T21:18:05.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.729+0000 7ff298c00640 1 -- 172.21.15.86:0/3907909763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff29415eaf0 msgr2=0x7ff2940ba1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:05.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.729+0000 7ff298c00640 1 --2- 172.21.15.86:0/3907909763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff29415eaf0 0x7ff2940ba1f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff29406d8e0 tx=0x7ff28402bf50 comp rx=0 tx=0).stop 2024-04-20T21:18:05.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.729+0000 7ff298c00640 1 -- 172.21.15.86:0/3907909763 shutdown_connections 2024-04-20T21:18:05.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.729+0000 7ff298c00640 1 --2- 172.21.15.86:0/3907909763 >> v2:172.21.15.144:3300/0 conn(0x7ff2940b4030 0x7ff2940d17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:05.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.729+0000 7ff298c00640 1 --2- 172.21.15.86:0/3907909763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff29415eaf0 0x7ff2940ba1f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:05.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.729+0000 7ff298c00640 1 -- 172.21.15.86:0/3907909763 >> 172.21.15.86:0/3907909763 conn(0x7ff2940678a0 msgr2=0x7ff294067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:05.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.729+0000 7ff298c00640 1 -- 172.21.15.86:0/3907909763 shutdown_connections 2024-04-20T21:18:05.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.729+0000 7ff298c00640 1 -- 172.21.15.86:0/3907909763 wait complete. 2024-04-20T21:18:05.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.731+0000 7ff298c00640 1 Processor -- start 2024-04-20T21:18:05.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.731+0000 7ff298c00640 1 -- start start 2024-04-20T21:18:05.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.732+0000 7ff298c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2940b4030 0x7ff29414a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:05.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.732+0000 7ff298c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2940d1d00 0x7ff2941f7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:05.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.732+0000 7ff298c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff29415eaf0 0x7ff2941f8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:05.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.732+0000 7ff298c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff294162da0 con 0x7ff29415eaf0 2024-04-20T21:18:05.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.732+0000 7ff298c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff294162ac0 con 0x7ff2940d1d00 2024-04-20T21:18:05.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.732+0000 7ff298c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff2940d3d40 con 0x7ff2940b4030 2024-04-20T21:18:05.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.732+0000 7ff292800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff29415eaf0 0x7ff2941f8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:05.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.732+0000 7ff292800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff29415eaf0 0x7ff2941f8440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58234/0 (socket says 172.21.15.86:58234) 2024-04-20T21:18:05.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.732+0000 7ff292800640 1 -- 172.21.15.86:0/52995002 learned_addr learned my addr 172.21.15.86:0/52995002 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:05.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.733+0000 7ff291e00640 1 --2- 172.21.15.86:0/52995002 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2940b4030 0x7ff29414a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:05.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.733+0000 7ff291400640 1 --2- 172.21.15.86:0/52995002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2940d1d00 0x7ff2941f7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:05.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.733+0000 7ff292800640 1 -- 172.21.15.86:0/52995002 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2940b4030 msgr2=0x7ff29414a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:05.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.733+0000 7ff292800640 1 --2- 172.21.15.86:0/52995002 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2940b4030 0x7ff29414a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:05.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.733+0000 7ff292800640 1 -- 172.21.15.86:0/52995002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2940d1d00 msgr2=0x7ff2941f7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:05.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.733+0000 7ff292800640 1 --2- 172.21.15.86:0/52995002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2940d1d00 0x7ff2941f7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:05.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.733+0000 7ff292800640 1 -- 172.21.15.86:0/52995002 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2940b5220 con 0x7ff29415eaf0 2024-04-20T21:18:05.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.733+0000 7ff292800640 1 --2- 172.21.15.86:0/52995002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff29415eaf0 0x7ff2941f8440 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff28c00c440 tx=0x7ff28c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:05.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.733+0000 7ff28a000640 1 -- 172.21.15.86:0/52995002 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28c011790 con 0x7ff29415eaf0 2024-04-20T21:18:05.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.733+0000 7ff28a000640 1 -- 172.21.15.86:0/52995002 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff28c017070 con 0x7ff29415eaf0 2024-04-20T21:18:05.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.734+0000 7ff298c00640 1 -- 172.21.15.86:0/52995002 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2940b5500 con 0x7ff29415eaf0 2024-04-20T21:18:05.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.734+0000 7ff28a000640 1 -- 172.21.15.86:0/52995002 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28c011b70 con 0x7ff29415eaf0 2024-04-20T21:18:05.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.734+0000 7ff298c00640 1 -- 172.21.15.86:0/52995002 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2940b5ac0 con 0x7ff29415eaf0 2024-04-20T21:18:05.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.735+0000 7ff298c00640 1 -- 172.21.15.86:0/52995002 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2940689e0 con 0x7ff29415eaf0 2024-04-20T21:18:05.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.736+0000 7ff28a000640 1 -- 172.21.15.86:0/52995002 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff28c011d10 con 0x7ff29415eaf0 2024-04-20T21:18:05.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.737+0000 7ff28a000640 1 --2- 172.21.15.86:0/52995002 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff27406c8f0 0x7ff27406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:05.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.737+0000 7ff28a000640 1 -- 172.21.15.86:0/52995002 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff28c08b930 con 0x7ff29415eaf0 2024-04-20T21:18:05.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.737+0000 7ff291e00640 1 --2- 172.21.15.86:0/52995002 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff27406c8f0 0x7ff27406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:05.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.738+0000 7ff291e00640 1 --2- 172.21.15.86:0/52995002 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff27406c8f0 0x7ff27406edb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff284035000 tx=0x7ff284030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:05.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.741+0000 7ff28a000640 1 -- 172.21.15.86:0/52995002 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff28c059d60 con 0x7ff29415eaf0 2024-04-20T21:18:05.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.943+0000 7ff298c00640 1 -- 172.21.15.86:0/52995002 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff2940c81a0 con 0x7ff29415eaf0 2024-04-20T21:18:05.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.946+0000 7ff28a000640 1 -- 172.21.15.86:0/52995002 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7ff28c059660 con 0x7ff29415eaf0 2024-04-20T21:18:05.945 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:18:05.946 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":20,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:17:53.274462+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:17:49.718799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:17:50.683477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:18:05.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.948+0000 7ff298c00640 1 -- 172.21.15.86:0/52995002 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff27406c8f0 msgr2=0x7ff27406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:05.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.948+0000 7ff298c00640 1 --2- 172.21.15.86:0/52995002 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff27406c8f0 0x7ff27406edb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff284035000 tx=0x7ff284030000 comp rx=0 tx=0).stop 2024-04-20T21:18:05.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.948+0000 7ff298c00640 1 -- 172.21.15.86:0/52995002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff29415eaf0 msgr2=0x7ff2941f8440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:05.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.948+0000 7ff298c00640 1 --2- 172.21.15.86:0/52995002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff29415eaf0 0x7ff2941f8440 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff28c00c440 tx=0x7ff28c00c900 comp rx=0 tx=0).stop 2024-04-20T21:18:05.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.949+0000 7ff298c00640 1 -- 172.21.15.86:0/52995002 shutdown_connections 2024-04-20T21:18:05.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.949+0000 7ff298c00640 1 --2- 172.21.15.86:0/52995002 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff27406c8f0 0x7ff27406edb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:05.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.949+0000 7ff298c00640 1 --2- 172.21.15.86:0/52995002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff29415eaf0 0x7ff2941f8440 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:05.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.949+0000 7ff298c00640 1 --2- 172.21.15.86:0/52995002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2940d1d00 0x7ff2941f7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:05.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.949+0000 7ff298c00640 1 --2- 172.21.15.86:0/52995002 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2940b4030 0x7ff29414a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:05.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.949+0000 7ff298c00640 1 -- 172.21.15.86:0/52995002 >> 172.21.15.86:0/52995002 conn(0x7ff2940678a0 msgr2=0x7ff2941600b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:05.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.949+0000 7ff298c00640 1 -- 172.21.15.86:0/52995002 shutdown_connections 2024-04-20T21:18:05.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:05.949+0000 7ff298c00640 1 -- 172.21.15.86:0/52995002 wait complete. 2024-04-20T21:18:05.966 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:17:49.718799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:17:50.683477+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:18:05.967 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:18:06.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.161+0000 7fdff5600640 1 Processor -- start 2024-04-20T21:18:06.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.161+0000 7fdff5600640 1 -- start start 2024-04-20T21:18:06.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.163+0000 7fdff5600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdff00cdc50 0x7fdff006e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:06.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.163+0000 7fdff5600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdff015cd40 con 0x7fdff0146210 2024-04-20T21:18:06.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.163+0000 7fdff5600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdff015d000 con 0x7fdff006bbf0 2024-04-20T21:18:06.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.163+0000 7fdff5600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdff015d2e0 con 0x7fdff00cdc50 2024-04-20T21:18:06.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.163+0000 7fdfe6000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fdff0146210 0x7fdff00cd540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:48612/0 (socket says 172.21.15.86:48612) 2024-04-20T21:18:06.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.163+0000 7fdfe6000640 1 -- 172.21.15.86:0/107341716 learned_addr learned my addr 172.21.15.86:0/107341716 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:18:06.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.164+0000 7fdfef400640 1 --2- 172.21.15.86:0/107341716 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdff00cdc50 0x7fdff006e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:06.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.164+0000 7fdfee000640 1 -- 172.21.15.86:0/107341716 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1876082206 0 0) 0x7fdff015cd40 con 0x7fdff0146210 2024-04-20T21:18:06.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.164+0000 7fdfee000640 1 -- 172.21.15.86:0/107341716 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdfd4003520 con 0x7fdff0146210 2024-04-20T21:18:06.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.164+0000 7fdfee000640 1 -- 172.21.15.86:0/107341716 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1275765543 0 0) 0x7fdfd4003520 con 0x7fdff0146210 2024-04-20T21:18:06.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.164+0000 7fdfee000640 1 -- 172.21.15.86:0/107341716 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdff00cdc50 msgr2=0x7fdff006e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:06.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.164+0000 7fdfee000640 1 --2- 172.21.15.86:0/107341716 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdff00cdc50 0x7fdff006e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:06.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.164+0000 7fdfee000640 1 -- 172.21.15.86:0/107341716 >> v1:172.21.15.144:6789/0 conn(0x7fdff006bbf0 legacy=0x7fdff0145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:06.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.164+0000 7fdfee000640 1 -- 172.21.15.86:0/107341716 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdff015d5c0 con 0x7fdff0146210 2024-04-20T21:18:06.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.164+0000 7fdfee000640 1 -- 172.21.15.86:0/107341716 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fdfd8001690 con 0x7fdff0146210 2024-04-20T21:18:06.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.164+0000 7fdfee000640 1 -- 172.21.15.86:0/107341716 >> v1:172.21.15.86:6789/0 conn(0x7fdff0146210 legacy=0x7fdff00cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:06.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.164+0000 7fdfee000640 1 --2- 172.21.15.86:0/107341716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfd40039e0 0x7fdfd4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:06.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.164+0000 7fdfee000640 1 --2- 172.21.15.86:0/107341716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfd4004800 0x7fdfd4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:06.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.164+0000 7fdfee000640 1 --2- 172.21.15.86:0/107341716 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdff00cdc50 0x7fdfd40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:06.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.164+0000 7fdfee000640 1 -- 172.21.15.86:0/107341716 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdff015d2e0 con 0x7fdfd40039e0 2024-04-20T21:18:06.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.164+0000 7fdfee000640 1 -- 172.21.15.86:0/107341716 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd400bad0 con 0x7fdfd4004800 2024-04-20T21:18:06.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.165+0000 7fdfee000640 1 -- 172.21.15.86:0/107341716 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdfd400bcd0 con 0x7fdff00cdc50 2024-04-20T21:18:06.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.165+0000 7fdfeea00640 1 --2- 172.21.15.86:0/107341716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfd40039e0 0x7fdfd4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:06.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.165+0000 7fdfe6000640 1 --2- 172.21.15.86:0/107341716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfd4004800 0x7fdfd4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:06.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.165+0000 7fdfef400640 1 --2- 172.21.15.86:0/107341716 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdff00cdc50 0x7fdfd40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:06.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.165+0000 7fdfeea00640 1 -- 172.21.15.86:0/107341716 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdff00cdc50 msgr2=0x7fdfd40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:06.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.165+0000 7fdfeea00640 1 --2- 172.21.15.86:0/107341716 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdff00cdc50 0x7fdfd40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:06.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.165+0000 7fdfeea00640 1 -- 172.21.15.86:0/107341716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfd4004800 msgr2=0x7fdfd4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:06.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.165+0000 7fdfeea00640 1 --2- 172.21.15.86:0/107341716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfd4004800 0x7fdfd4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:06.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.165+0000 7fdfeea00640 1 -- 172.21.15.86:0/107341716 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfd4003520 con 0x7fdfd40039e0 2024-04-20T21:18:06.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.165+0000 7fdfef400640 1 --2- 172.21.15.86:0/107341716 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdff00cdc50 0x7fdfd40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:06.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.165+0000 7fdfe6000640 1 --2- 172.21.15.86:0/107341716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfd4004800 0x7fdfd4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:06.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.166+0000 7fdfeea00640 1 --2- 172.21.15.86:0/107341716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfd40039e0 0x7fdfd4003e00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0001e20 tx=0x7fdfd4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54a04b1dfcc76e13 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:06.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.166+0000 7fdfee000640 1 -- 172.21.15.86:0/107341716 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe0001a70 con 0x7fdfd40039e0 2024-04-20T21:18:06.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.166+0000 7fdfee000640 1 -- 172.21.15.86:0/107341716 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfe0003b70 con 0x7fdfd40039e0 2024-04-20T21:18:06.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.166+0000 7fdfee000640 1 -- 172.21.15.86:0/107341716 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe00050d0 con 0x7fdfd40039e0 2024-04-20T21:18:06.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.167+0000 7fdff5600640 1 -- 172.21.15.86:0/107341716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfd40039e0 msgr2=0x7fdfd4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:06.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.167+0000 7fdff5600640 1 --2- 172.21.15.86:0/107341716 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfd40039e0 0x7fdfd4003e00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0001e20 tx=0x7fdfd4005830 comp rx=0 tx=0).stop 2024-04-20T21:18:06.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.167+0000 7fdfef400640 1 -- 172.21.15.86:0/107341716 reap_dead start 2024-04-20T21:18:06.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.167+0000 7fdff5600640 1 -- 172.21.15.86:0/107341716 shutdown_connections 2024-04-20T21:18:06.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.167+0000 7fdff5600640 1 -- 172.21.15.86:0/107341716 >> 172.21.15.86:0/107341716 conn(0x7fdff00b2110 msgr2=0x7fdff00b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:06.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.167+0000 7fdff5600640 1 -- 172.21.15.86:0/107341716 shutdown_connections 2024-04-20T21:18:06.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.167+0000 7fdff5600640 1 -- 172.21.15.86:0/107341716 wait complete. 2024-04-20T21:18:06.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.169+0000 7fdff5600640 1 Processor -- start 2024-04-20T21:18:06.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.169+0000 7fdff5600640 1 -- start start 2024-04-20T21:18:06.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.170+0000 7fdff5600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfd40039e0 0x7fdff013fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:06.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.170+0000 7fdff5600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdff00a9db0 0x7fdff00b0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:06.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.170+0000 7fdff5600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdff00aa230 0x7fdff00b0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:06.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.170+0000 7fdff5600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdff006c150 con 0x7fdff00a9db0 2024-04-20T21:18:06.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.170+0000 7fdff5600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdff006bfd0 con 0x7fdff00aa230 2024-04-20T21:18:06.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.170+0000 7fdff5600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdff006c2d0 con 0x7fdfd40039e0 2024-04-20T21:18:06.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.170+0000 7fdfe6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdff00a9db0 0x7fdff00b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:06.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.170+0000 7fdfe6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdff00a9db0 0x7fdff00b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46336/0 (socket says 172.21.15.86:46336) 2024-04-20T21:18:06.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.170+0000 7fdfe6000640 1 -- 172.21.15.86:0/793529195 learned_addr learned my addr 172.21.15.86:0/793529195 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:06.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.171+0000 7fdfeea00640 1 --2- 172.21.15.86:0/793529195 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfd40039e0 0x7fdff013fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:06.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.171+0000 7fdfef400640 1 --2- 172.21.15.86:0/793529195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdff00aa230 0x7fdff00b0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:06.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.171+0000 7fdfe6000640 1 -- 172.21.15.86:0/793529195 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfd40039e0 msgr2=0x7fdff013fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:06.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.171+0000 7fdfe6000640 1 --2- 172.21.15.86:0/793529195 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfd40039e0 0x7fdff013fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:06.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.171+0000 7fdfe6000640 1 -- 172.21.15.86:0/793529195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdff00aa230 msgr2=0x7fdff00b0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:06.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.171+0000 7fdfe6000640 1 --2- 172.21.15.86:0/793529195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdff00aa230 0x7fdff00b0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:06.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.171+0000 7fdfe6000640 1 -- 172.21.15.86:0/793529195 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdff01401c0 con 0x7fdff00a9db0 2024-04-20T21:18:06.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.171+0000 7fdfe6000640 1 --2- 172.21.15.86:0/793529195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdff00a9db0 0x7fdff00b0870 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fdfd8001eb0 tx=0x7fdfd800dc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:06.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.171+0000 7fdfe7400640 1 -- 172.21.15.86:0/793529195 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd8019070 con 0x7fdff00a9db0 2024-04-20T21:18:06.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.171+0000 7fdfe7400640 1 -- 172.21.15.86:0/793529195 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfd80074e0 con 0x7fdff00a9db0 2024-04-20T21:18:06.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.171+0000 7fdfe7400640 1 -- 172.21.15.86:0/793529195 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd80049e0 con 0x7fdff00a9db0 2024-04-20T21:18:06.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.171+0000 7fdff5600640 1 -- 172.21.15.86:0/793529195 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdff013ebe0 con 0x7fdff00a9db0 2024-04-20T21:18:06.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.172+0000 7fdff5600640 1 -- 172.21.15.86:0/793529195 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdff013f1a0 con 0x7fdff00a9db0 2024-04-20T21:18:06.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.173+0000 7fdff5600640 1 -- 172.21.15.86:0/793529195 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdff00689e0 con 0x7fdff00a9db0 2024-04-20T21:18:06.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.174+0000 7fdfe7400640 1 -- 172.21.15.86:0/793529195 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdfd8004b80 con 0x7fdff00a9db0 2024-04-20T21:18:06.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.174+0000 7fdfe7400640 1 --2- 172.21.15.86:0/793529195 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdfcc06c8f0 0x7fdfcc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:06.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.175+0000 7fdfe7400640 1 -- 172.21.15.86:0/793529195 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdfd808b580 con 0x7fdff00a9db0 2024-04-20T21:18:06.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.175+0000 7fdfeea00640 1 --2- 172.21.15.86:0/793529195 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdfcc06c8f0 0x7fdfcc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:06.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.175+0000 7fdfeea00640 1 --2- 172.21.15.86:0/793529195 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdfcc06c8f0 0x7fdfcc06edb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0034000 tx=0x7fdfe0009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:06.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.179+0000 7fdfe7400640 1 -- 172.21.15.86:0/793529195 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fdfd8059aa0 con 0x7fdff00a9db0 2024-04-20T21:18:06.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.361+0000 7fdff5600640 1 -- 172.21.15.86:0/793529195 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdff00b1ef0 con 0x7fdff00a9db0 2024-04-20T21:18:06.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.362+0000 7fdfe7400640 1 -- 172.21.15.86:0/793529195 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fdfd80593a0 con 0x7fdff00a9db0 2024-04-20T21:18:06.361 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:18:06.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.364+0000 7fdff5600640 1 -- 172.21.15.86:0/793529195 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdfcc06c8f0 msgr2=0x7fdfcc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:06.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.364+0000 7fdff5600640 1 --2- 172.21.15.86:0/793529195 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdfcc06c8f0 0x7fdfcc06edb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0034000 tx=0x7fdfe0009030 comp rx=0 tx=0).stop 2024-04-20T21:18:06.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.364+0000 7fdff5600640 1 -- 172.21.15.86:0/793529195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdff00a9db0 msgr2=0x7fdff00b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:06.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.364+0000 7fdff5600640 1 --2- 172.21.15.86:0/793529195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdff00a9db0 0x7fdff00b0870 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fdfd8001eb0 tx=0x7fdfd800dc70 comp rx=0 tx=0).stop 2024-04-20T21:18:06.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.364+0000 7fdff5600640 1 -- 172.21.15.86:0/793529195 shutdown_connections 2024-04-20T21:18:06.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.364+0000 7fdff5600640 1 --2- 172.21.15.86:0/793529195 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdfcc06c8f0 0x7fdfcc06edb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:06.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.364+0000 7fdff5600640 1 --2- 172.21.15.86:0/793529195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdff00aa230 0x7fdff00b0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:06.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.364+0000 7fdff5600640 1 --2- 172.21.15.86:0/793529195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdff00a9db0 0x7fdff00b0870 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:06.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.364+0000 7fdff5600640 1 --2- 172.21.15.86:0/793529195 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfd40039e0 0x7fdff013fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:06.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.364+0000 7fdff5600640 1 -- 172.21.15.86:0/793529195 >> 172.21.15.86:0/793529195 conn(0x7fdff00b2110 msgr2=0x7fdff006e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:06.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.365+0000 7fdff5600640 1 -- 172.21.15.86:0/793529195 shutdown_connections 2024-04-20T21:18:06.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.365+0000 7fdff5600640 1 -- 172.21.15.86:0/793529195 wait complete. 2024-04-20T21:18:06.382 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:18:06.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.581+0000 7fe902a00640 1 Processor -- start 2024-04-20T21:18:06.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.581+0000 7fe902a00640 1 -- start start 2024-04-20T21:18:06.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.583+0000 7fe902a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe8fc15e7d0 0x7fe8fc0ba020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:06.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.583+0000 7fe902a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe8fc15ebb0 0x7fe8fc0d10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:06.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.583+0000 7fe902a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe8fc0d1630 0x7fe8fc162550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:06.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.583+0000 7fe902a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe8fc0cf5b0 con 0x7fe8fc15ebb0 2024-04-20T21:18:06.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.583+0000 7fe902a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fe8fc0cf890 con 0x7fe8fc0d1630 2024-04-20T21:18:06.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.583+0000 7fe902a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe8fc0cfb70 con 0x7fe8fc15e7d0 2024-04-20T21:18:06.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.583+0000 7fe900c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe8fc15ebb0 0x7fe8fc0d10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:06.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.583+0000 7fe900c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe8fc15ebb0 0x7fe8fc0d10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46350/0 (socket says 172.21.15.86:46350) 2024-04-20T21:18:06.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.583+0000 7fe900c00640 1 -- 172.21.15.86:0/2910907200 learned_addr learned my addr 172.21.15.86:0/2910907200 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:06.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.583+0000 7fe901600640 1 --2- 172.21.15.86:0/2910907200 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe8fc15e7d0 0x7fe8fc0ba020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:06.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.583+0000 7fe902000640 1 --2- 172.21.15.86:0/2910907200 >> v2:172.21.15.144:3300/0 conn(0x7fe8fc0d1630 0x7fe8fc162550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:06.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.584+0000 7fe900c00640 1 -- 172.21.15.86:0/2910907200 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe8fc15e7d0 msgr2=0x7fe8fc0ba020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:06.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.584+0000 7fe900c00640 1 --2- 172.21.15.86:0/2910907200 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe8fc15e7d0 0x7fe8fc0ba020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:06.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.584+0000 7fe900c00640 1 -- 172.21.15.86:0/2910907200 >> v2:172.21.15.144:3300/0 conn(0x7fe8fc0d1630 msgr2=0x7fe8fc162550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:06.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.584+0000 7fe900c00640 1 --2- 172.21.15.86:0/2910907200 >> v2:172.21.15.144:3300/0 conn(0x7fe8fc0d1630 0x7fe8fc162550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:06.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.584+0000 7fe900c00640 1 -- 172.21.15.86:0/2910907200 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8fc0b8500 con 0x7fe8fc15ebb0 2024-04-20T21:18:06.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.585+0000 7fe900c00640 1 --2- 172.21.15.86:0/2910907200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8fc15ebb0 0x7fe8fc0d10f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8009860 tx=0x7fe8e802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c9be60c0bca090f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:06.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.585+0000 7fe8fbe00640 1 -- 172.21.15.86:0/2910907200 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e80317c0 con 0x7fe8fc15ebb0 2024-04-20T21:18:06.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.585+0000 7fe8fbe00640 1 -- 172.21.15.86:0/2910907200 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8e8037070 con 0x7fe8fc15ebb0 2024-04-20T21:18:06.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.585+0000 7fe8fbe00640 1 -- 172.21.15.86:0/2910907200 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8031b80 con 0x7fe8fc15ebb0 2024-04-20T21:18:06.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.586+0000 7fe902a00640 1 -- 172.21.15.86:0/2910907200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8fc15ebb0 msgr2=0x7fe8fc0d10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:06.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.586+0000 7fe902a00640 1 --2- 172.21.15.86:0/2910907200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8fc15ebb0 0x7fe8fc0d10f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8009860 tx=0x7fe8e802b860 comp rx=0 tx=0).stop 2024-04-20T21:18:06.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.586+0000 7fe902a00640 1 -- 172.21.15.86:0/2910907200 shutdown_connections 2024-04-20T21:18:06.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.586+0000 7fe902a00640 1 --2- 172.21.15.86:0/2910907200 >> v2:172.21.15.144:3300/0 conn(0x7fe8fc0d1630 0x7fe8fc162550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:06.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.586+0000 7fe902a00640 1 --2- 172.21.15.86:0/2910907200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8fc15ebb0 0x7fe8fc0d10f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:06.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.586+0000 7fe902a00640 1 --2- 172.21.15.86:0/2910907200 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe8fc15e7d0 0x7fe8fc0ba020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:06.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.586+0000 7fe902a00640 1 -- 172.21.15.86:0/2910907200 >> 172.21.15.86:0/2910907200 conn(0x7fe8fc06cc70 msgr2=0x7fe8fc06d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:06.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.586+0000 7fe902a00640 1 -- 172.21.15.86:0/2910907200 shutdown_connections 2024-04-20T21:18:06.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.586+0000 7fe902a00640 1 -- 172.21.15.86:0/2910907200 wait complete. 2024-04-20T21:18:06.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.588+0000 7fe902a00640 1 Processor -- start 2024-04-20T21:18:06.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.588+0000 7fe902a00640 1 -- start start 2024-04-20T21:18:06.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.589+0000 7fe902a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe8fc0d1630 0x7fe8fc0adbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:06.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.589+0000 7fe902a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8fc15e7d0 0x7fe8fc153740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:06.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.589+0000 7fe902a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8fc15ebb0 0x7fe8fc153c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:06.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.589+0000 7fe902a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc0b3c70 con 0x7fe8fc15e7d0 2024-04-20T21:18:06.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.589+0000 7fe902a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc0ba560 con 0x7fe8fc15ebb0 2024-04-20T21:18:06.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.590+0000 7fe902a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe8fc0b81b0 con 0x7fe8fc0d1630 2024-04-20T21:18:06.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.590+0000 7fe900c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8fc15e7d0 0x7fe8fc153740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:06.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.590+0000 7fe900c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8fc15e7d0 0x7fe8fc153740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46352/0 (socket says 172.21.15.86:46352) 2024-04-20T21:18:06.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.590+0000 7fe900c00640 1 -- 172.21.15.86:0/3154709441 learned_addr learned my addr 172.21.15.86:0/3154709441 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:06.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.590+0000 7fe902000640 1 --2- 172.21.15.86:0/3154709441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8fc15ebb0 0x7fe8fc153c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:06.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.590+0000 7fe901600640 1 --2- 172.21.15.86:0/3154709441 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe8fc0d1630 0x7fe8fc0adbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:06.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.590+0000 7fe900c00640 1 -- 172.21.15.86:0/3154709441 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe8fc0d1630 msgr2=0x7fe8fc0adbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:06.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.590+0000 7fe900c00640 1 --2- 172.21.15.86:0/3154709441 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe8fc0d1630 0x7fe8fc0adbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:06.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.590+0000 7fe900c00640 1 -- 172.21.15.86:0/3154709441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8fc15ebb0 msgr2=0x7fe8fc153c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:06.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.590+0000 7fe900c00640 1 --2- 172.21.15.86:0/3154709441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8fc15ebb0 0x7fe8fc153c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:06.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.590+0000 7fe900c00640 1 -- 172.21.15.86:0/3154709441 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8fc0cd810 con 0x7fe8fc15e7d0 2024-04-20T21:18:06.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.590+0000 7fe902000640 1 --2- 172.21.15.86:0/3154709441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8fc15ebb0 0x7fe8fc153c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:06.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.591+0000 7fe900c00640 1 --2- 172.21.15.86:0/3154709441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8fc15e7d0 0x7fe8fc153740 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe8e802b8a0 tx=0x7fe8e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:06.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.591+0000 7fe8f9600640 1 -- 172.21.15.86:0/3154709441 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8032860 con 0x7fe8fc15e7d0 2024-04-20T21:18:06.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.591+0000 7fe8f9600640 1 -- 172.21.15.86:0/3154709441 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8e8032a20 con 0x7fe8fc15e7d0 2024-04-20T21:18:06.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.591+0000 7fe8f9600640 1 -- 172.21.15.86:0/3154709441 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8032e10 con 0x7fe8fc15e7d0 2024-04-20T21:18:06.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.591+0000 7fe902a00640 1 -- 172.21.15.86:0/3154709441 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8fc0cdaf0 con 0x7fe8fc15e7d0 2024-04-20T21:18:06.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.591+0000 7fe902a00640 1 -- 172.21.15.86:0/3154709441 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8fc0ce0b0 con 0x7fe8fc15e7d0 2024-04-20T21:18:06.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.592+0000 7fe902a00640 1 -- 172.21.15.86:0/3154709441 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8fc04fd90 con 0x7fe8fc15e7d0 2024-04-20T21:18:06.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.593+0000 7fe8f9600640 1 -- 172.21.15.86:0/3154709441 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe8e804c060 con 0x7fe8fc15e7d0 2024-04-20T21:18:06.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.594+0000 7fe8f9600640 1 --2- 172.21.15.86:0/3154709441 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe8dc06c8f0 0x7fe8dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:06.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.594+0000 7fe8f9600640 1 -- 172.21.15.86:0/3154709441 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe8e80ad790 con 0x7fe8fc15e7d0 2024-04-20T21:18:06.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.594+0000 7fe901600640 1 --2- 172.21.15.86:0/3154709441 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe8dc06c8f0 0x7fe8dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:06.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.595+0000 7fe901600640 1 --2- 172.21.15.86:0/3154709441 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe8dc06c8f0 0x7fe8dc06edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0006fd0 tx=0x7fe8f0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:06.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.598+0000 7fe8f9600640 1 -- 172.21.15.86:0/3154709441 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe8e8040070 con 0x7fe8fc15e7d0 2024-04-20T21:18:06.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.796+0000 7fe902a00640 1 -- 172.21.15.86:0/3154709441 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe8fc0d22b0 con 0x7fe8fc15e7d0 2024-04-20T21:18:06.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.797+0000 7fe8f9600640 1 -- 172.21.15.86:0/3154709441 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fe8e8031e90 con 0x7fe8fc15e7d0 2024-04-20T21:18:06.796 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-20T21:18:06.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.799+0000 7fe902a00640 1 -- 172.21.15.86:0/3154709441 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe8dc06c8f0 msgr2=0x7fe8dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:06.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.799+0000 7fe902a00640 1 --2- 172.21.15.86:0/3154709441 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe8dc06c8f0 0x7fe8dc06edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0006fd0 tx=0x7fe8f0008040 comp rx=0 tx=0).stop 2024-04-20T21:18:06.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.799+0000 7fe902a00640 1 -- 172.21.15.86:0/3154709441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8fc15e7d0 msgr2=0x7fe8fc153740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:06.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.799+0000 7fe902a00640 1 --2- 172.21.15.86:0/3154709441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8fc15e7d0 0x7fe8fc153740 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe8e802b8a0 tx=0x7fe8e80046c0 comp rx=0 tx=0).stop 2024-04-20T21:18:06.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.799+0000 7fe902a00640 1 -- 172.21.15.86:0/3154709441 shutdown_connections 2024-04-20T21:18:06.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.799+0000 7fe902a00640 1 --2- 172.21.15.86:0/3154709441 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe8dc06c8f0 0x7fe8dc06edb0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:06.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.799+0000 7fe902a00640 1 --2- 172.21.15.86:0/3154709441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8fc15ebb0 0x7fe8fc153c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:06.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.799+0000 7fe902a00640 1 --2- 172.21.15.86:0/3154709441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8fc15e7d0 0x7fe8fc153740 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:06.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.799+0000 7fe902a00640 1 --2- 172.21.15.86:0/3154709441 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe8fc0d1630 0x7fe8fc0adbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:06.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.799+0000 7fe902a00640 1 -- 172.21.15.86:0/3154709441 >> 172.21.15.86:0/3154709441 conn(0x7fe8fc06cc70 msgr2=0x7fe8fc0d27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:06.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.800+0000 7fe902a00640 1 -- 172.21.15.86:0/3154709441 shutdown_connections 2024-04-20T21:18:06.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:06.800+0000 7fe902a00640 1 -- 172.21.15.86:0/3154709441 wait complete. 2024-04-20T21:18:06.815 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:18:07.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.011+0000 7f95b0000640 1 Processor -- start 2024-04-20T21:18:07.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.011+0000 7f95b0000640 1 -- start start 2024-04-20T21:18:07.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.013+0000 7f95b0000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f95a8161fa0 0x7f95a80cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:07.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.013+0000 7f95b0000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95a80b0570 0x7f95a80704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:07.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.013+0000 7f95b0000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95a80701c0 con 0x7f95a8070a10 2024-04-20T21:18:07.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.013+0000 7f95b0000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f95a8070e80 con 0x7f95a8161fa0 2024-04-20T21:18:07.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.013+0000 7f95b0000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f95a8162380 con 0x7f95a80b0570 2024-04-20T21:18:07.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.014+0000 7f95ad600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f95a8161fa0 0x7f95a80cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:07.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.014+0000 7f95acc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95a80b0570 0x7f95a80704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:07.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.014+0000 7f95ad600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f95a8161fa0 0x7f95a80cdb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.86:43962/0 (socket says 172.21.15.86:43962) 2024-04-20T21:18:07.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.014+0000 7f95ad600640 1 -- 172.21.15.86:0/3376115828 learned_addr learned my addr 172.21.15.86:0/3376115828 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:07.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.014+0000 7f95acc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95a80b0570 0x7f95a80704d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:45928/0 (socket says 172.21.15.86:45928) 2024-04-20T21:18:07.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.014+0000 7f95ae000640 1 --1- 172.21.15.86:0/3376115828 >> v1:172.21.15.144:6789/0 conn(0x7f95a8070a10 0x7f95a806b980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.86:56940/0 (socket says 172.21.15.86:56940) 2024-04-20T21:18:07.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.014+0000 7f95ad600640 1 -- 172.21.15.86:0/3376115828 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95a80b0570 msgr2=0x7f95a80704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:07.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.014+0000 7f95ad600640 1 --2- 172.21.15.86:0/3376115828 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95a80b0570 0x7f95a80704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:07.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.014+0000 7f95ad600640 1 -- 172.21.15.86:0/3376115828 >> v1:172.21.15.144:6789/0 conn(0x7f95a8070a10 legacy=0x7f95a806b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:07.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.014+0000 7f95ad600640 1 -- 172.21.15.86:0/3376115828 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95a80ce110 con 0x7f95a8161fa0 2024-04-20T21:18:07.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.016+0000 7f95ad600640 1 --2- 172.21.15.86:0/3376115828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95a8161fa0 0x7f95a80cdb30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f959c009870 tx=0x7f959c02b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef7281df4f92b0db server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:07.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.016+0000 7f95a7e00640 1 -- 172.21.15.86:0/3376115828 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c035020 con 0x7f95a8161fa0 2024-04-20T21:18:07.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.016+0000 7f95a7e00640 1 -- 172.21.15.86:0/3376115828 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f959c031640 con 0x7f95a8161fa0 2024-04-20T21:18:07.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.016+0000 7f95a7e00640 1 -- 172.21.15.86:0/3376115828 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c02d040 con 0x7f95a8161fa0 2024-04-20T21:18:07.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.017+0000 7f95b0000640 1 -- 172.21.15.86:0/3376115828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95a8161fa0 msgr2=0x7f95a80cdb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:07.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.017+0000 7f95b0000640 1 --2- 172.21.15.86:0/3376115828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95a8161fa0 0x7f95a80cdb30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f959c009870 tx=0x7f959c02b8f0 comp rx=0 tx=0).stop 2024-04-20T21:18:07.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.017+0000 7f95b0000640 1 -- 172.21.15.86:0/3376115828 shutdown_connections 2024-04-20T21:18:07.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.017+0000 7f95b0000640 1 --2- 172.21.15.86:0/3376115828 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95a80b0570 0x7f95a80704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:07.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.017+0000 7f95b0000640 1 --2- 172.21.15.86:0/3376115828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95a8161fa0 0x7f95a80cdb30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:07.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.017+0000 7f95b0000640 1 -- 172.21.15.86:0/3376115828 >> 172.21.15.86:0/3376115828 conn(0x7f95a809a9e0 msgr2=0x7f95a80b2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:07.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.017+0000 7f95b0000640 1 -- 172.21.15.86:0/3376115828 shutdown_connections 2024-04-20T21:18:07.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.017+0000 7f95b0000640 1 -- 172.21.15.86:0/3376115828 wait complete. 2024-04-20T21:18:07.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.019+0000 7f95b0000640 1 Processor -- start 2024-04-20T21:18:07.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.020+0000 7f95b0000640 1 -- start start 2024-04-20T21:18:07.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.021+0000 7f95b0000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95a8070a10 0x7f95a8152a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:07.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.021+0000 7f95b0000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95a80b0570 0x7f95a80b3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:07.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.021+0000 7f95b0000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95a8161fa0 0x7f95a80cb200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:07.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.021+0000 7f95b0000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f95a80ce6d0 con 0x7f95a80b0570 2024-04-20T21:18:07.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.021+0000 7f95b0000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f95a80ced20 con 0x7f95a8070a10 2024-04-20T21:18:07.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.021+0000 7f95b0000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f95a80ce9b0 con 0x7f95a8161fa0 2024-04-20T21:18:07.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.021+0000 7f95acc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95a80b0570 0x7f95a80b3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:07.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.021+0000 7f95acc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95a80b0570 0x7f95a80b3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46364/0 (socket says 172.21.15.86:46364) 2024-04-20T21:18:07.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.021+0000 7f95acc00640 1 -- 172.21.15.86:0/3162128318 learned_addr learned my addr 172.21.15.86:0/3162128318 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:07.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.021+0000 7f95ad600640 1 --2- 172.21.15.86:0/3162128318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95a8070a10 0x7f95a8152a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:07.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.021+0000 7f95ae000640 1 --2- 172.21.15.86:0/3162128318 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95a8161fa0 0x7f95a80cb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:07.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.022+0000 7f95acc00640 1 -- 172.21.15.86:0/3162128318 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95a8161fa0 msgr2=0x7f95a80cb200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:07.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.022+0000 7f95acc00640 1 --2- 172.21.15.86:0/3162128318 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95a8161fa0 0x7f95a80cb200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:07.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.022+0000 7f95acc00640 1 -- 172.21.15.86:0/3162128318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95a8070a10 msgr2=0x7f95a8152a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:07.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.022+0000 7f95acc00640 1 --2- 172.21.15.86:0/3162128318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95a8070a10 0x7f95a8152a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:07.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.022+0000 7f95acc00640 1 -- 172.21.15.86:0/3162128318 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95a815e250 con 0x7f95a80b0570 2024-04-20T21:18:07.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.022+0000 7f95ae000640 1 --2- 172.21.15.86:0/3162128318 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95a8161fa0 0x7f95a80cb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:07.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.022+0000 7f95acc00640 1 --2- 172.21.15.86:0/3162128318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95a80b0570 0x7f95a80b3c80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f959400e9b0 tx=0x7f959400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:07.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.022+0000 7f95a5600640 1 -- 172.21.15.86:0/3162128318 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9594017070 con 0x7f95a80b0570 2024-04-20T21:18:07.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.022+0000 7f95a5600640 1 -- 172.21.15.86:0/3162128318 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f959400c9a0 con 0x7f95a80b0570 2024-04-20T21:18:07.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.022+0000 7f95b0000640 1 -- 172.21.15.86:0/3162128318 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95a815e530 con 0x7f95a80b0570 2024-04-20T21:18:07.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.022+0000 7f95a5600640 1 -- 172.21.15.86:0/3162128318 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959400cd90 con 0x7f95a80b0570 2024-04-20T21:18:07.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.022+0000 7f95b0000640 1 -- 172.21.15.86:0/3162128318 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95a815d4d0 con 0x7f95a80b0570 2024-04-20T21:18:07.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.023+0000 7f95b0000640 1 -- 172.21.15.86:0/3162128318 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95a80689e0 con 0x7f95a80b0570 2024-04-20T21:18:07.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.025+0000 7f95a5600640 1 -- 172.21.15.86:0/3162128318 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f959401a070 con 0x7f95a80b0570 2024-04-20T21:18:07.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.026+0000 7f95a5600640 1 --2- 172.21.15.86:0/3162128318 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f958806c920 0x7f958806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:07.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.026+0000 7f95a5600640 1 -- 172.21.15.86:0/3162128318 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f959408bef0 con 0x7f95a80b0570 2024-04-20T21:18:07.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.026+0000 7f95ad600640 1 --2- 172.21.15.86:0/3162128318 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f958806c920 0x7f958806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:07.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.027+0000 7f95ad600640 1 --2- 172.21.15.86:0/3162128318 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f958806c920 0x7f958806ede0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f959c036000 tx=0x7f959c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:07.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.030+0000 7f95a5600640 1 -- 172.21.15.86:0/3162128318 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f959405a310 con 0x7f95a80b0570 2024-04-20T21:18:07.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.219+0000 7f95b0000640 1 -- 172.21.15.86:0/3162128318 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f95a8161380 con 0x7f95a80b0570 2024-04-20T21:18:07.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.220+0000 7f95a5600640 1 -- 172.21.15.86:0/3162128318 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f9594059c10 con 0x7f95a80b0570 2024-04-20T21:18:07.219 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-20T21:18:07.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.223+0000 7f95b0000640 1 -- 172.21.15.86:0/3162128318 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f958806c920 msgr2=0x7f958806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:07.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.223+0000 7f95b0000640 1 --2- 172.21.15.86:0/3162128318 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f958806c920 0x7f958806ede0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f959c036000 tx=0x7f959c008030 comp rx=0 tx=0).stop 2024-04-20T21:18:07.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.223+0000 7f95b0000640 1 -- 172.21.15.86:0/3162128318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95a80b0570 msgr2=0x7f95a80b3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:07.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.223+0000 7f95b0000640 1 --2- 172.21.15.86:0/3162128318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95a80b0570 0x7f95a80b3c80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f959400e9b0 tx=0x7f959400ee70 comp rx=0 tx=0).stop 2024-04-20T21:18:07.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.223+0000 7f95b0000640 1 -- 172.21.15.86:0/3162128318 shutdown_connections 2024-04-20T21:18:07.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.223+0000 7f95b0000640 1 --2- 172.21.15.86:0/3162128318 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f958806c920 0x7f958806ede0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:07.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.223+0000 7f95b0000640 1 --2- 172.21.15.86:0/3162128318 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95a8161fa0 0x7f95a80cb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:07.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.223+0000 7f95b0000640 1 --2- 172.21.15.86:0/3162128318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95a80b0570 0x7f95a80b3c80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:07.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.223+0000 7f95b0000640 1 --2- 172.21.15.86:0/3162128318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95a8070a10 0x7f95a8152a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:07.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.223+0000 7f95b0000640 1 -- 172.21.15.86:0/3162128318 >> 172.21.15.86:0/3162128318 conn(0x7f95a809a9e0 msgr2=0x7f95a80cba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:07.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.223+0000 7f95b0000640 1 -- 172.21.15.86:0/3162128318 shutdown_connections 2024-04-20T21:18:07.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.223+0000 7f95b0000640 1 -- 172.21.15.86:0/3162128318 wait complete. 2024-04-20T21:18:07.239 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:18:07.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.435+0000 7fa756800640 1 Processor -- start 2024-04-20T21:18:07.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.435+0000 7fa756800640 1 -- start start 2024-04-20T21:18:07.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.436+0000 7fa756800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa750144e90 0x7fa7500b4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:07.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.436+0000 7fa756800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7500cd170 0x7fa75006ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:07.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.436+0000 7fa756800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7500cdb50 con 0x7fa7500cd5d0 2024-04-20T21:18:07.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.436+0000 7fa756800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa750145270 con 0x7fa750144e90 2024-04-20T21:18:07.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.436+0000 7fa756800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa7500b3e50 con 0x7fa7500cd170 2024-04-20T21:18:07.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.436+0000 7fa74f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa750144e90 0x7fa7500b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:07.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.437+0000 7fa74fe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fa7500cd5d0 0x7fa75015bb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:48622/0 (socket says 172.21.15.86:48622) 2024-04-20T21:18:07.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.437+0000 7fa74f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa750144e90 0x7fa7500b4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46368/0 (socket says 172.21.15.86:46368) 2024-04-20T21:18:07.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.437+0000 7fa74fe00640 1 -- 172.21.15.86:0/2113933899 learned_addr learned my addr 172.21.15.86:0/2113933899 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:18:07.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.437+0000 7fa747e00640 1 --2- 172.21.15.86:0/2113933899 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7500cd170 0x7fa75006ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:07.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.437+0000 7fa74f400640 1 -- 172.21.15.86:0/2113933899 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7500cd170 msgr2=0x7fa75006ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:07.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.437+0000 7fa74f400640 1 --2- 172.21.15.86:0/2113933899 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7500cd170 0x7fa75006ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:07.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.437+0000 7fa74f400640 1 -- 172.21.15.86:0/2113933899 >> v1:172.21.15.86:6789/0 conn(0x7fa7500cd5d0 legacy=0x7fa75015bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:07.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.437+0000 7fa74f400640 1 -- 172.21.15.86:0/2113933899 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7500b4130 con 0x7fa750144e90 2024-04-20T21:18:07.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.437+0000 7fa74ea00640 1 -- 172.21.15.86:0/2113933899 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2869331757 0 0) 0x7fa7500cdb50 con 0x7fa7500cd5d0 2024-04-20T21:18:07.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.438+0000 7fa74f400640 1 --2- 172.21.15.86:0/2113933899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa750144e90 0x7fa7500b4930 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa74000ba40 tx=0x7fa74002dff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e5571157d931eab server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:07.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.438+0000 7fa74ea00640 1 -- 172.21.15.86:0/2113933899 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74003b070 con 0x7fa750144e90 2024-04-20T21:18:07.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.438+0000 7fa74ea00640 1 -- 172.21.15.86:0/2113933899 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa74002eda0 con 0x7fa750144e90 2024-04-20T21:18:07.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.438+0000 7fa74ea00640 1 -- 172.21.15.86:0/2113933899 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa740036660 con 0x7fa750144e90 2024-04-20T21:18:07.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.439+0000 7fa756800640 1 -- 172.21.15.86:0/2113933899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa750144e90 msgr2=0x7fa7500b4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:07.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.439+0000 7fa756800640 1 --2- 172.21.15.86:0/2113933899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa750144e90 0x7fa7500b4930 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa74000ba40 tx=0x7fa74002dff0 comp rx=0 tx=0).stop 2024-04-20T21:18:07.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.439+0000 7fa756800640 1 -- 172.21.15.86:0/2113933899 shutdown_connections 2024-04-20T21:18:07.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.439+0000 7fa756800640 1 --2- 172.21.15.86:0/2113933899 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7500cd170 0x7fa75006ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:07.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.439+0000 7fa756800640 1 --2- 172.21.15.86:0/2113933899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa750144e90 0x7fa7500b4930 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:07.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.439+0000 7fa756800640 1 -- 172.21.15.86:0/2113933899 >> 172.21.15.86:0/2113933899 conn(0x7fa7500b2360 msgr2=0x7fa7500b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:07.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.439+0000 7fa756800640 1 -- 172.21.15.86:0/2113933899 shutdown_connections 2024-04-20T21:18:07.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.439+0000 7fa756800640 1 -- 172.21.15.86:0/2113933899 wait complete. 2024-04-20T21:18:07.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.441+0000 7fa756800640 1 Processor -- start 2024-04-20T21:18:07.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.441+0000 7fa756800640 1 -- start start 2024-04-20T21:18:07.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.442+0000 7fa756800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7500cd170 0x7fa7500c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:07.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.442+0000 7fa756800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7500cd5d0 0x7fa7501e1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:07.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.442+0000 7fa756800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa750144e90 0x7fa7501f41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:07.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.442+0000 7fa756800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa7500d0010 con 0x7fa7500cd170 2024-04-20T21:18:07.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.442+0000 7fa756800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa7500b4e70 con 0x7fa750144e90 2024-04-20T21:18:07.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.442+0000 7fa756800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa7500d0190 con 0x7fa7500cd5d0 2024-04-20T21:18:07.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.442+0000 7fa74f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7500cd170 0x7fa7500c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:07.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.442+0000 7fa74f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7500cd170 0x7fa7500c7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46374/0 (socket says 172.21.15.86:46374) 2024-04-20T21:18:07.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.442+0000 7fa74f400640 1 -- 172.21.15.86:0/2881251302 learned_addr learned my addr 172.21.15.86:0/2881251302 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:07.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.442+0000 7fa74f400640 1 -- 172.21.15.86:0/2881251302 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7500cd5d0 msgr2=0x7fa7501e1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:07.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.442+0000 7fa74f400640 1 --2- 172.21.15.86:0/2881251302 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7500cd5d0 0x7fa7501e1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:07.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.442+0000 7fa74f400640 1 -- 172.21.15.86:0/2881251302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa750144e90 msgr2=0x7fa7501f41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:07.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.442+0000 7fa74f400640 1 --2- 172.21.15.86:0/2881251302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa750144e90 0x7fa7501f41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:07.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.442+0000 7fa74f400640 1 -- 172.21.15.86:0/2881251302 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7501e1950 con 0x7fa7500cd170 2024-04-20T21:18:07.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.443+0000 7fa74f400640 1 --2- 172.21.15.86:0/2881251302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7500cd170 0x7fa7500c7e10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa740039040 tx=0x7fa7400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:07.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.443+0000 7fa747400640 1 -- 172.21.15.86:0/2881251302 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74003b040 con 0x7fa7500cd170 2024-04-20T21:18:07.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.443+0000 7fa747400640 1 -- 172.21.15.86:0/2881251302 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa740044070 con 0x7fa7500cd170 2024-04-20T21:18:07.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.443+0000 7fa747400640 1 -- 172.21.15.86:0/2881251302 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74003f680 con 0x7fa7500cd170 2024-04-20T21:18:07.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.443+0000 7fa756800640 1 -- 172.21.15.86:0/2881251302 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7500c8760 con 0x7fa7500cd170 2024-04-20T21:18:07.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.443+0000 7fa756800640 1 -- 172.21.15.86:0/2881251302 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7500c8d20 con 0x7fa7500cd170 2024-04-20T21:18:07.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.444+0000 7fa756800640 1 -- 172.21.15.86:0/2881251302 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7500c84e0 con 0x7fa7500cd170 2024-04-20T21:18:07.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.445+0000 7fa747400640 1 -- 172.21.15.86:0/2881251302 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa74003f820 con 0x7fa7500cd170 2024-04-20T21:18:07.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.446+0000 7fa747400640 1 --2- 172.21.15.86:0/2881251302 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa72c06c8f0 0x7fa72c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:07.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.446+0000 7fa747400640 1 -- 172.21.15.86:0/2881251302 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa7400adb30 con 0x7fa7500cd170 2024-04-20T21:18:07.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.446+0000 7fa747e00640 1 --2- 172.21.15.86:0/2881251302 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa72c06c8f0 0x7fa72c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:07.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.447+0000 7fa747e00640 1 --2- 172.21.15.86:0/2881251302 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa72c06c8f0 0x7fa72c06edb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa7500c91f0 tx=0x7fa738008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:07.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.450+0000 7fa747400640 1 -- 172.21.15.86:0/2881251302 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa7500c84e0 con 0x7fa7500cd170 2024-04-20T21:18:07.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.675+0000 7fa756800640 1 -- 172.21.15.86:0/2881251302 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa7500689e0 con 0x7fa7500cd170 2024-04-20T21:18:07.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.677+0000 7fa747400640 1 -- 172.21.15.86:0/2881251302 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5196 (secure 0 0 0) 0x7fa74007bff0 con 0x7fa7500cd170 2024-04-20T21:18:07.676 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:18:07.676 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/179773217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":179773217},{"type":"v1","addr":"172.21.15.144:6834","nonce":179773217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3179785507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3179785507},{"type":"v1","addr":"172.21.15.86:6835","nonce":3179785507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4270,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2780853729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2780853729},{"type":"v1","addr":"172.21.15.86:6837","nonce":2780853729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:17:51.700362+0000","modified":"2024-04-20T21:17:54.973792+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6835/2436181584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2436181584},{"type":"v1","addr":"172.21.15.144:6835","nonce":2436181584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4272,"qdb_cluster":[4272]},"id":1}]} 2024-04-20T21:18:07.677 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 7 2024-04-20T21:18:07.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.679+0000 7fa756800640 1 -- 172.21.15.86:0/2881251302 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa72c06c8f0 msgr2=0x7fa72c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:07.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.679+0000 7fa756800640 1 --2- 172.21.15.86:0/2881251302 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa72c06c8f0 0x7fa72c06edb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa7500c91f0 tx=0x7fa738008040 comp rx=0 tx=0).stop 2024-04-20T21:18:07.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.679+0000 7fa756800640 1 -- 172.21.15.86:0/2881251302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7500cd170 msgr2=0x7fa7500c7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:07.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.679+0000 7fa756800640 1 --2- 172.21.15.86:0/2881251302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7500cd170 0x7fa7500c7e10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa740039040 tx=0x7fa7400046c0 comp rx=0 tx=0).stop 2024-04-20T21:18:07.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.680+0000 7fa756800640 1 -- 172.21.15.86:0/2881251302 shutdown_connections 2024-04-20T21:18:07.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.680+0000 7fa756800640 1 --2- 172.21.15.86:0/2881251302 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa72c06c8f0 0x7fa72c06edb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:07.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.680+0000 7fa756800640 1 --2- 172.21.15.86:0/2881251302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa750144e90 0x7fa7501f41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:07.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.680+0000 7fa756800640 1 --2- 172.21.15.86:0/2881251302 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7500cd5d0 0x7fa7501e1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:07.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.680+0000 7fa756800640 1 --2- 172.21.15.86:0/2881251302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7500cd170 0x7fa7500c7e10 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:07.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.680+0000 7fa756800640 1 -- 172.21.15.86:0/2881251302 >> 172.21.15.86:0/2881251302 conn(0x7fa7500b2360 msgr2=0x7fa750071d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:07.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.680+0000 7fa756800640 1 -- 172.21.15.86:0/2881251302 shutdown_connections 2024-04-20T21:18:07.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.680+0000 7fa756800640 1 -- 172.21.15.86:0/2881251302 wait complete. 2024-04-20T21:18:07.695 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:18:07.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.887+0000 7f7311e00640 1 Processor -- start 2024-04-20T21:18:07.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.887+0000 7f7311e00640 1 -- start start 2024-04-20T21:18:07.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.888+0000 7f7311e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f730c15d2c0 0x7f730c06f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:07.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.889+0000 7f7311e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f730c06f5f0 0x7f730c071d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:07.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.889+0000 7f7311e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f730c06fb70 con 0x7f730c072240 2024-04-20T21:18:07.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.889+0000 7f7311e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f730c0726b0 con 0x7f730c15d2c0 2024-04-20T21:18:07.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.889+0000 7f7311e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f730c164210 con 0x7f730c06f5f0 2024-04-20T21:18:07.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.889+0000 7f730aa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f730c15d2c0 0x7f730c06f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:07.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.889+0000 7f730aa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f730c15d2c0 0x7f730c06f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46384/0 (socket says 172.21.15.86:46384) 2024-04-20T21:18:07.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.889+0000 7f730aa00640 1 -- 172.21.15.86:0/231533461 learned_addr learned my addr 172.21.15.86:0/231533461 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:07.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.889+0000 7f730a000640 1 --2- 172.21.15.86:0/231533461 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f730c06f5f0 0x7f730c071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:07.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.890+0000 7f730aa00640 1 -- 172.21.15.86:0/231533461 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f730c06f5f0 msgr2=0x7f730c071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:07.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.890+0000 7f730aa00640 1 --2- 172.21.15.86:0/231533461 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f730c06f5f0 0x7f730c071d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:07.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.890+0000 7f730aa00640 1 -- 172.21.15.86:0/231533461 >> v1:172.21.15.144:6789/0 conn(0x7f730c072240 legacy=0x7f730c06dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:07.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.890+0000 7f730aa00640 1 -- 172.21.15.86:0/231533461 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f730c15d6a0 con 0x7f730c15d2c0 2024-04-20T21:18:07.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.891+0000 7f730aa00640 1 --2- 172.21.15.86:0/231533461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f730c15d2c0 0x7f730c06f0b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7304009870 tx=0x7f730402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40560b4aa0a7aba0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:07.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.891+0000 7f7309600640 1 -- 172.21.15.86:0/231533461 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7304034070 con 0x7f730c15d2c0 2024-04-20T21:18:07.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.891+0000 7f7309600640 1 -- 172.21.15.86:0/231533461 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7304030840 con 0x7f730c15d2c0 2024-04-20T21:18:07.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.891+0000 7f7309600640 1 -- 172.21.15.86:0/231533461 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7304030c10 con 0x7f730c15d2c0 2024-04-20T21:18:07.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.892+0000 7f7311e00640 1 -- 172.21.15.86:0/231533461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f730c15d2c0 msgr2=0x7f730c06f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:07.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.892+0000 7f7311e00640 1 --2- 172.21.15.86:0/231533461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f730c15d2c0 0x7f730c06f0b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7304009870 tx=0x7f730402b980 comp rx=0 tx=0).stop 2024-04-20T21:18:07.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.892+0000 7f7311e00640 1 -- 172.21.15.86:0/231533461 shutdown_connections 2024-04-20T21:18:07.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.892+0000 7f7311e00640 1 --2- 172.21.15.86:0/231533461 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f730c06f5f0 0x7f730c071d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:07.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.892+0000 7f7311e00640 1 --2- 172.21.15.86:0/231533461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f730c15d2c0 0x7f730c06f0b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:07.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.892+0000 7f7311e00640 1 -- 172.21.15.86:0/231533461 >> 172.21.15.86:0/231533461 conn(0x7f730c0b2360 msgr2=0x7f730c0b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:07.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.892+0000 7f7311e00640 1 -- 172.21.15.86:0/231533461 shutdown_connections 2024-04-20T21:18:07.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.892+0000 7f7311e00640 1 -- 172.21.15.86:0/231533461 wait complete. 2024-04-20T21:18:07.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.894+0000 7f7311e00640 1 Processor -- start 2024-04-20T21:18:07.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.894+0000 7f7311e00640 1 -- start start 2024-04-20T21:18:07.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.895+0000 7f7311e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f730c06f5f0 0x7f730c145520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:07.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.895+0000 7f7311e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f730c072240 0x7f730c150880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:07.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.895+0000 7f7311e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f730c15d2c0 0x7f730c15fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:07.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.895+0000 7f7311e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f730c166ec0 con 0x7f730c06f5f0 2024-04-20T21:18:07.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.895+0000 7f7311e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f730c166be0 con 0x7f730c15d2c0 2024-04-20T21:18:07.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.895+0000 7f7311e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f730c1671a0 con 0x7f730c072240 2024-04-20T21:18:07.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.895+0000 7f730aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f730c06f5f0 0x7f730c145520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:07.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.895+0000 7f730aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f730c06f5f0 0x7f730c145520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46394/0 (socket says 172.21.15.86:46394) 2024-04-20T21:18:07.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.895+0000 7f730aa00640 1 -- 172.21.15.86:0/1807715105 learned_addr learned my addr 172.21.15.86:0/1807715105 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:07.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.896+0000 7f730a000640 1 --2- 172.21.15.86:0/1807715105 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f730c072240 0x7f730c150880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:07.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.896+0000 7f730b400640 1 --2- 172.21.15.86:0/1807715105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f730c15d2c0 0x7f730c15fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:07.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.896+0000 7f730aa00640 1 -- 172.21.15.86:0/1807715105 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f730c072240 msgr2=0x7f730c150880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:07.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.896+0000 7f730aa00640 1 --2- 172.21.15.86:0/1807715105 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f730c072240 0x7f730c150880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:07.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.896+0000 7f730aa00640 1 -- 172.21.15.86:0/1807715105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f730c15d2c0 msgr2=0x7f730c15fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:07.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.896+0000 7f730aa00640 1 --2- 172.21.15.86:0/1807715105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f730c15d2c0 0x7f730c15fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:07.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.896+0000 7f730aa00640 1 -- 172.21.15.86:0/1807715105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f730c151260 con 0x7f730c06f5f0 2024-04-20T21:18:07.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.896+0000 7f730b400640 1 --2- 172.21.15.86:0/1807715105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f730c15d2c0 0x7f730c15fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:07.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.896+0000 7f730aa00640 1 --2- 172.21.15.86:0/1807715105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f730c06f5f0 0x7f730c145520 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7304000c00 tx=0x7f73040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:07.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.896+0000 7f7303400640 1 -- 172.21.15.86:0/1807715105 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7304034040 con 0x7f730c06f5f0 2024-04-20T21:18:07.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.896+0000 7f7303400640 1 -- 172.21.15.86:0/1807715105 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7304039d90 con 0x7f730c06f5f0 2024-04-20T21:18:07.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.896+0000 7f7311e00640 1 -- 172.21.15.86:0/1807715105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f730c151540 con 0x7f730c06f5f0 2024-04-20T21:18:07.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.897+0000 7f7303400640 1 -- 172.21.15.86:0/1807715105 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7304043610 con 0x7f730c06f5f0 2024-04-20T21:18:07.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.897+0000 7f7311e00640 1 -- 172.21.15.86:0/1807715105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f730c15dd40 con 0x7f730c06f5f0 2024-04-20T21:18:07.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.898+0000 7f7311e00640 1 -- 172.21.15.86:0/1807715105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f730c06fe10 con 0x7f730c06f5f0 2024-04-20T21:18:07.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.899+0000 7f7303400640 1 -- 172.21.15.86:0/1807715105 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f730404a060 con 0x7f730c06f5f0 2024-04-20T21:18:07.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.899+0000 7f7303400640 1 --2- 172.21.15.86:0/1807715105 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f72e806c920 0x7f72e806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:07.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.899+0000 7f7303400640 1 -- 172.21.15.86:0/1807715105 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f73040adb30 con 0x7f730c06f5f0 2024-04-20T21:18:07.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.900+0000 7f730a000640 1 --2- 172.21.15.86:0/1807715105 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f72e806c920 0x7f72e806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:07.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.900+0000 7f730a000640 1 --2- 172.21.15.86:0/1807715105 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f72e806c920 0x7f72e806ede0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f730c0b4060 tx=0x7f72f4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:07.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:07.904+0000 7f7303400640 1 -- 172.21.15.86:0/1807715105 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f730c06fe10 con 0x7f730c06f5f0 2024-04-20T21:18:08.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.106+0000 7f7311e00640 1 -- 172.21.15.86:0/1807715105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f730c0689e0 con 0x7f730c06f5f0 2024-04-20T21:18:08.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.109+0000 7f7303400640 1 -- 172.21.15.86:0/1807715105 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7f730403e070 con 0x7f730c06f5f0 2024-04-20T21:18:08.108 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:18:08.108 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":20,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:17:53.274462+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:17:49.718799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:17:50.683477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:18:08.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.111+0000 7f7311e00640 1 -- 172.21.15.86:0/1807715105 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f72e806c920 msgr2=0x7f72e806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:08.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.111+0000 7f7311e00640 1 --2- 172.21.15.86:0/1807715105 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f72e806c920 0x7f72e806ede0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f730c0b4060 tx=0x7f72f4008040 comp rx=0 tx=0).stop 2024-04-20T21:18:08.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.111+0000 7f7311e00640 1 -- 172.21.15.86:0/1807715105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f730c06f5f0 msgr2=0x7f730c145520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:08.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.111+0000 7f7311e00640 1 --2- 172.21.15.86:0/1807715105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f730c06f5f0 0x7f730c145520 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7304000c00 tx=0x7f73040046c0 comp rx=0 tx=0).stop 2024-04-20T21:18:08.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.112+0000 7f7311e00640 1 -- 172.21.15.86:0/1807715105 shutdown_connections 2024-04-20T21:18:08.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.112+0000 7f7311e00640 1 --2- 172.21.15.86:0/1807715105 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f72e806c920 0x7f72e806ede0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:08.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.112+0000 7f7311e00640 1 --2- 172.21.15.86:0/1807715105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f730c15d2c0 0x7f730c15fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:08.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.112+0000 7f7311e00640 1 --2- 172.21.15.86:0/1807715105 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f730c072240 0x7f730c150880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:08.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.112+0000 7f7311e00640 1 --2- 172.21.15.86:0/1807715105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f730c06f5f0 0x7f730c145520 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:08.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.112+0000 7f7311e00640 1 -- 172.21.15.86:0/1807715105 >> 172.21.15.86:0/1807715105 conn(0x7f730c0b2360 msgr2=0x7f730c0b3920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:08.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.112+0000 7f7311e00640 1 -- 172.21.15.86:0/1807715105 shutdown_connections 2024-04-20T21:18:08.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.112+0000 7f7311e00640 1 -- 172.21.15.86:0/1807715105 wait complete. 2024-04-20T21:18:08.129 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:18:08.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.328+0000 7f011aa00640 1 Processor -- start 2024-04-20T21:18:08.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.328+0000 7f011aa00640 1 -- start start 2024-04-20T21:18:08.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.330+0000 7f011aa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f01140d50c0 0x7f01140d3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:08.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.330+0000 7f011aa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01140cf9c0 0x7f0114160000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:08.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.330+0000 7f011aa00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01140d3950 con 0x7f011415ece0 2024-04-20T21:18:08.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.330+0000 7f011aa00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f01140cec00 con 0x7f01140d50c0 2024-04-20T21:18:08.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.330+0000 7f011aa00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f01140ceee0 con 0x7f01140cf9c0 2024-04-20T21:18:08.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.330+0000 7f0119600640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f011415ece0 0x7f011415f0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:48632/0 (socket says 172.21.15.86:48632) 2024-04-20T21:18:08.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.330+0000 7f0119600640 1 -- 172.21.15.86:0/2474950764 learned_addr learned my addr 172.21.15.86:0/2474950764 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:18:08.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.330+0000 7f0118c00640 1 --2- 172.21.15.86:0/2474950764 >> v2:172.21.15.86:3300/0 conn(0x7f01140d50c0 0x7f01140d3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:08.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.331+0000 7f011a000640 1 --2- 172.21.15.86:0/2474950764 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01140cf9c0 0x7f0114160000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:08.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.331+0000 7f0118c00640 1 -- 172.21.15.86:0/2474950764 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01140cf9c0 msgr2=0x7f0114160000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:08.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.331+0000 7f0113e00640 1 -- 172.21.15.86:0/2474950764 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1082303766 0 0) 0x7f01140d3950 con 0x7f011415ece0 2024-04-20T21:18:08.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.331+0000 7f0118c00640 1 --2- 172.21.15.86:0/2474950764 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01140cf9c0 0x7f0114160000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:08.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.331+0000 7f0118c00640 1 -- 172.21.15.86:0/2474950764 >> v1:172.21.15.86:6789/0 conn(0x7f011415ece0 legacy=0x7f011415f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:08.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.331+0000 7f0118c00640 1 -- 172.21.15.86:0/2474950764 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01140cf1c0 con 0x7f01140d50c0 2024-04-20T21:18:08.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.331+0000 7f011a000640 1 --2- 172.21.15.86:0/2474950764 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01140cf9c0 0x7f0114160000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:08.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.332+0000 7f0118c00640 1 --2- 172.21.15.86:0/2474950764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01140d50c0 0x7f01140d3100 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f01000097a0 tx=0x7f010002b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65a14b2bea18ef36 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:08.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.332+0000 7f0113e00640 1 -- 172.21.15.86:0/2474950764 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0100031780 con 0x7f01140d50c0 2024-04-20T21:18:08.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.332+0000 7f0113e00640 1 -- 172.21.15.86:0/2474950764 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0100037070 con 0x7f01140d50c0 2024-04-20T21:18:08.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.332+0000 7f0113e00640 1 -- 172.21.15.86:0/2474950764 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0100031b40 con 0x7f01140d50c0 2024-04-20T21:18:08.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.333+0000 7f011aa00640 1 -- 172.21.15.86:0/2474950764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01140d50c0 msgr2=0x7f01140d3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:08.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.333+0000 7f011aa00640 1 --2- 172.21.15.86:0/2474950764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01140d50c0 0x7f01140d3100 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f01000097a0 tx=0x7f010002b820 comp rx=0 tx=0).stop 2024-04-20T21:18:08.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.333+0000 7f011aa00640 1 -- 172.21.15.86:0/2474950764 shutdown_connections 2024-04-20T21:18:08.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.333+0000 7f011aa00640 1 --2- 172.21.15.86:0/2474950764 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01140cf9c0 0x7f0114160000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:08.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.333+0000 7f011aa00640 1 --2- 172.21.15.86:0/2474950764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01140d50c0 0x7f01140d3100 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:08.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.333+0000 7f011aa00640 1 -- 172.21.15.86:0/2474950764 >> 172.21.15.86:0/2474950764 conn(0x7f011408bf70 msgr2=0x7f01140d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:08.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.333+0000 7f011aa00640 1 -- 172.21.15.86:0/2474950764 shutdown_connections 2024-04-20T21:18:08.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.333+0000 7f011aa00640 1 -- 172.21.15.86:0/2474950764 wait complete. 2024-04-20T21:18:08.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.334+0000 7f011aa00640 1 Processor -- start 2024-04-20T21:18:08.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.335+0000 7f011aa00640 1 -- start start 2024-04-20T21:18:08.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.335+0000 7f011aa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01140cf9c0 0x7f01140c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:08.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.335+0000 7f011aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01140d50c0 0x7f0114149820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:08.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.335+0000 7f011aa00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f011415ece0 0x7f01140b8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:08.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.335+0000 7f011aa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f011415ea50 con 0x7f01140d50c0 2024-04-20T21:18:08.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.335+0000 7f011aa00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f01140d3640 con 0x7f011415ece0 2024-04-20T21:18:08.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.335+0000 7f011aa00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f01140cfe70 con 0x7f01140cf9c0 2024-04-20T21:18:08.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.336+0000 7f0118c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01140d50c0 0x7f0114149820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:08.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.336+0000 7f0118c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01140d50c0 0x7f0114149820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46408/0 (socket says 172.21.15.86:46408) 2024-04-20T21:18:08.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.336+0000 7f0118c00640 1 -- 172.21.15.86:0/4113358284 learned_addr learned my addr 172.21.15.86:0/4113358284 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:08.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.336+0000 7f0119600640 1 --2- 172.21.15.86:0/4113358284 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01140cf9c0 0x7f01140c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:08.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.336+0000 7f011a000640 1 --2- 172.21.15.86:0/4113358284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f011415ece0 0x7f01140b8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:08.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.336+0000 7f0118c00640 1 -- 172.21.15.86:0/4113358284 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01140cf9c0 msgr2=0x7f01140c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:08.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.336+0000 7f0118c00640 1 --2- 172.21.15.86:0/4113358284 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01140cf9c0 0x7f01140c8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:08.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.336+0000 7f0118c00640 1 -- 172.21.15.86:0/4113358284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f011415ece0 msgr2=0x7f01140b8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:08.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.336+0000 7f0118c00640 1 --2- 172.21.15.86:0/4113358284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f011415ece0 0x7f01140b8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:08.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.336+0000 7f0118c00640 1 -- 172.21.15.86:0/4113358284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01140cb690 con 0x7f01140d50c0 2024-04-20T21:18:08.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.337+0000 7f0118c00640 1 --2- 172.21.15.86:0/4113358284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01140d50c0 0x7f0114149820 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0100034040 tx=0x7f01000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:08.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.337+0000 7f0111600640 1 -- 172.21.15.86:0/4113358284 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0100032820 con 0x7f01140d50c0 2024-04-20T21:18:08.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.337+0000 7f0111600640 1 -- 172.21.15.86:0/4113358284 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01000329c0 con 0x7f01140d50c0 2024-04-20T21:18:08.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.337+0000 7f011aa00640 1 -- 172.21.15.86:0/4113358284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01140caf70 con 0x7f01140d50c0 2024-04-20T21:18:08.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.337+0000 7f0111600640 1 -- 172.21.15.86:0/4113358284 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0100032db0 con 0x7f01140d50c0 2024-04-20T21:18:08.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.337+0000 7f011aa00640 1 -- 172.21.15.86:0/4113358284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01140cbc10 con 0x7f01140d50c0 2024-04-20T21:18:08.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.338+0000 7f011aa00640 1 -- 172.21.15.86:0/4113358284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f011404fcf0 con 0x7f01140d50c0 2024-04-20T21:18:08.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.339+0000 7f0111600640 1 -- 172.21.15.86:0/4113358284 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f010004e440 con 0x7f01140d50c0 2024-04-20T21:18:08.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.340+0000 7f0111600640 1 --2- 172.21.15.86:0/4113358284 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f00f406c8f0 0x7f00f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:08.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.340+0000 7f0111600640 1 -- 172.21.15.86:0/4113358284 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f01000adaf0 con 0x7f01140d50c0 2024-04-20T21:18:08.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.340+0000 7f0119600640 1 --2- 172.21.15.86:0/4113358284 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f00f406c8f0 0x7f00f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:08.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.341+0000 7f0119600640 1 --2- 172.21.15.86:0/4113358284 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f00f406c8f0 0x7f00f406edb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0108007ed0 tx=0x7f0108007e60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:08.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.344+0000 7f0111600640 1 -- 172.21.15.86:0/4113358284 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0100040070 con 0x7f01140d50c0 2024-04-20T21:18:08.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.568+0000 7f011aa00640 1 -- 172.21.15.86:0/4113358284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01141626a0 con 0x7f01140d50c0 2024-04-20T21:18:08.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.569+0000 7f0111600640 1 -- 172.21.15.86:0/4113358284 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5196 (secure 0 0 0) 0x7f010002f050 con 0x7f01140d50c0 2024-04-20T21:18:08.569 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:18:08.569 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/179773217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":179773217},{"type":"v1","addr":"172.21.15.144:6834","nonce":179773217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3179785507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3179785507},{"type":"v1","addr":"172.21.15.86:6835","nonce":3179785507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4270,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2780853729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2780853729},{"type":"v1","addr":"172.21.15.86:6837","nonce":2780853729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:17:51.700362+0000","modified":"2024-04-20T21:17:54.973792+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6835/2436181584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2436181584},{"type":"v1","addr":"172.21.15.144:6835","nonce":2436181584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4272,"qdb_cluster":[4272]},"id":1}]} 2024-04-20T21:18:08.569 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 7 2024-04-20T21:18:08.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.572+0000 7f011aa00640 1 -- 172.21.15.86:0/4113358284 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f00f406c8f0 msgr2=0x7f00f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:08.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.572+0000 7f011aa00640 1 --2- 172.21.15.86:0/4113358284 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f00f406c8f0 0x7f00f406edb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0108007ed0 tx=0x7f0108007e60 comp rx=0 tx=0).stop 2024-04-20T21:18:08.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.572+0000 7f011aa00640 1 -- 172.21.15.86:0/4113358284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01140d50c0 msgr2=0x7f0114149820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:08.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.572+0000 7f011aa00640 1 --2- 172.21.15.86:0/4113358284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01140d50c0 0x7f0114149820 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0100034040 tx=0x7f01000046c0 comp rx=0 tx=0).stop 2024-04-20T21:18:08.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.572+0000 7f011aa00640 1 -- 172.21.15.86:0/4113358284 shutdown_connections 2024-04-20T21:18:08.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.572+0000 7f011aa00640 1 --2- 172.21.15.86:0/4113358284 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f00f406c8f0 0x7f00f406edb0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:08.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.572+0000 7f011aa00640 1 --2- 172.21.15.86:0/4113358284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f011415ece0 0x7f01140b8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:08.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.572+0000 7f011aa00640 1 --2- 172.21.15.86:0/4113358284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01140d50c0 0x7f0114149820 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:08.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.572+0000 7f011aa00640 1 --2- 172.21.15.86:0/4113358284 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01140cf9c0 0x7f01140c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:08.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.572+0000 7f011aa00640 1 -- 172.21.15.86:0/4113358284 >> 172.21.15.86:0/4113358284 conn(0x7f011408bf70 msgr2=0x7f01140d25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:08.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.572+0000 7f011aa00640 1 -- 172.21.15.86:0/4113358284 shutdown_connections 2024-04-20T21:18:08.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:08.572+0000 7f011aa00640 1 -- 172.21.15.86:0/4113358284 wait complete. 2024-04-20T21:18:08.990 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T21:17:51.700362+0000', 'modified': '2024-04-20T21:17:54.973792+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4272}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4272': {'gid': 4272, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.144:6835/2436181584', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6833', 'nonce': 2436181584}, {'type': 'v1', 'addr': '172.21.15.144:6835', 'nonce': 2436181584}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4272, 'qdb_cluster': [4272]} 2024-04-20T21:18:08.990 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-20T21:18:08.990 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-20T21:18:09.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.200+0000 7fa2f0400640 1 Processor -- start 2024-04-20T21:18:09.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.200+0000 7fa2f0400640 1 -- start start 2024-04-20T21:18:09.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.202+0000 7fa2f0400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2e815b5f0 0x7fa2e80b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:09.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.202+0000 7fa2f0400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa2e806f070 0x7fa2e80b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:09.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.202+0000 7fa2f0400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa2e80cff80 0x7fa2e80d0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:09.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.202+0000 7fa2f0400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa2e80b4650 con 0x7fa2e806f070 2024-04-20T21:18:09.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.202+0000 7fa2f0400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fa2e815b9d0 con 0x7fa2e80cff80 2024-04-20T21:18:09.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.202+0000 7fa2f0400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa2e80af850 con 0x7fa2e815b5f0 2024-04-20T21:18:09.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.202+0000 7fa2ed000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa2e806f070 0x7fa2e80b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:09.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.202+0000 7fa2ed000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa2e806f070 0x7fa2e80b50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46424/0 (socket says 172.21.15.86:46424) 2024-04-20T21:18:09.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.202+0000 7fa2ed000640 1 -- 172.21.15.86:0/308361561 learned_addr learned my addr 172.21.15.86:0/308361561 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:09.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.203+0000 7fa2eda00640 1 --2- 172.21.15.86:0/308361561 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2e815b5f0 0x7fa2e80b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:09.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.203+0000 7fa2ee400640 1 --2- 172.21.15.86:0/308361561 >> v2:172.21.15.144:3300/0 conn(0x7fa2e80cff80 0x7fa2e80d0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:09.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.203+0000 7fa2ed000640 1 -- 172.21.15.86:0/308361561 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2e815b5f0 msgr2=0x7fa2e80b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:09.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.203+0000 7fa2ed000640 1 --2- 172.21.15.86:0/308361561 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2e815b5f0 0x7fa2e80b3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:09.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.203+0000 7fa2ed000640 1 -- 172.21.15.86:0/308361561 >> v2:172.21.15.144:3300/0 conn(0x7fa2e80cff80 msgr2=0x7fa2e80d0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:09.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.203+0000 7fa2ed000640 1 --2- 172.21.15.86:0/308361561 >> v2:172.21.15.144:3300/0 conn(0x7fa2e80cff80 0x7fa2e80d0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:09.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.203+0000 7fa2ed000640 1 -- 172.21.15.86:0/308361561 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2e80afb30 con 0x7fa2e806f070 2024-04-20T21:18:09.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.203+0000 7fa2ee400640 1 --2- 172.21.15.86:0/308361561 >> v2:172.21.15.144:3300/0 conn(0x7fa2e80cff80 0x7fa2e80d0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:09.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.203+0000 7fa2eda00640 1 --2- 172.21.15.86:0/308361561 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2e815b5f0 0x7fa2e80b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:09.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.204+0000 7fa2ed000640 1 --2- 172.21.15.86:0/308361561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2e806f070 0x7fa2e80b50a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa2e80701c0 tx=0x7fa2d402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44b949a2ae3ebd90 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:09.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.204+0000 7fa2e7e00640 1 -- 172.21.15.86:0/308361561 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d4035020 con 0x7fa2e806f070 2024-04-20T21:18:09.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.204+0000 7fa2e7e00640 1 -- 172.21.15.86:0/308361561 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2d4031870 con 0x7fa2e806f070 2024-04-20T21:18:09.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.204+0000 7fa2e7e00640 1 -- 172.21.15.86:0/308361561 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d4031c40 con 0x7fa2e806f070 2024-04-20T21:18:09.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.205+0000 7fa2f0400640 1 -- 172.21.15.86:0/308361561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2e806f070 msgr2=0x7fa2e80b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:09.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.205+0000 7fa2f0400640 1 --2- 172.21.15.86:0/308361561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2e806f070 0x7fa2e80b50a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa2e80701c0 tx=0x7fa2d402b910 comp rx=0 tx=0).stop 2024-04-20T21:18:09.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.205+0000 7fa2f0400640 1 -- 172.21.15.86:0/308361561 shutdown_connections 2024-04-20T21:18:09.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.205+0000 7fa2f0400640 1 --2- 172.21.15.86:0/308361561 >> v2:172.21.15.144:3300/0 conn(0x7fa2e80cff80 0x7fa2e80d0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:09.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.205+0000 7fa2f0400640 1 --2- 172.21.15.86:0/308361561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2e806f070 0x7fa2e80b50a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:09.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.205+0000 7fa2f0400640 1 --2- 172.21.15.86:0/308361561 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2e815b5f0 0x7fa2e80b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:09.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.205+0000 7fa2f0400640 1 -- 172.21.15.86:0/308361561 >> 172.21.15.86:0/308361561 conn(0x7fa2e806b980 msgr2=0x7fa2e806bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:09.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.205+0000 7fa2f0400640 1 -- 172.21.15.86:0/308361561 shutdown_connections 2024-04-20T21:18:09.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.205+0000 7fa2f0400640 1 -- 172.21.15.86:0/308361561 wait complete. 2024-04-20T21:18:09.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.207+0000 7fa2f0400640 1 Processor -- start 2024-04-20T21:18:09.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.207+0000 7fa2f0400640 1 -- start start 2024-04-20T21:18:09.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.208+0000 7fa2f0400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2e806f070 0x7fa2e815d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:09.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.208+0000 7fa2f0400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2e80cff80 0x7fa2e815dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:09.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.208+0000 7fa2f0400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2e815b5f0 0x7fa2e815e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:09.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.208+0000 7fa2f0400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e80b55e0 con 0x7fa2e815b5f0 2024-04-20T21:18:09.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.208+0000 7fa2f0400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e80b4b80 con 0x7fa2e80cff80 2024-04-20T21:18:09.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.208+0000 7fa2f0400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa2e80ce130 con 0x7fa2e806f070 2024-04-20T21:18:09.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.208+0000 7fa2ee400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2e815b5f0 0x7fa2e815e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:09.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.208+0000 7fa2ee400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2e815b5f0 0x7fa2e815e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46438/0 (socket says 172.21.15.86:46438) 2024-04-20T21:18:09.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.208+0000 7fa2ee400640 1 -- 172.21.15.86:0/205976270 learned_addr learned my addr 172.21.15.86:0/205976270 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:09.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.208+0000 7fa2ee400640 1 -- 172.21.15.86:0/205976270 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2e806f070 msgr2=0x7fa2e815d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:09.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.208+0000 7fa2ee400640 1 --2- 172.21.15.86:0/205976270 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2e806f070 0x7fa2e815d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:09.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.208+0000 7fa2ed000640 1 --2- 172.21.15.86:0/205976270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2e80cff80 0x7fa2e815dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:09.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.208+0000 7fa2ee400640 1 -- 172.21.15.86:0/205976270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2e80cff80 msgr2=0x7fa2e815dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:09.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.208+0000 7fa2ee400640 1 --2- 172.21.15.86:0/205976270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2e80cff80 0x7fa2e815dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:09.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.208+0000 7fa2ee400640 1 -- 172.21.15.86:0/205976270 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2e815d870 con 0x7fa2e815b5f0 2024-04-20T21:18:09.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.209+0000 7fa2ee400640 1 --2- 172.21.15.86:0/205976270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2e815b5f0 0x7fa2e815e200 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa2e000cef0 tx=0x7fa2e000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:09.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.209+0000 7fa2e5600640 1 -- 172.21.15.86:0/205976270 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e001a070 con 0x7fa2e815b5f0 2024-04-20T21:18:09.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.209+0000 7fa2e5600640 1 -- 172.21.15.86:0/205976270 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2e000d860 con 0x7fa2e815b5f0 2024-04-20T21:18:09.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.209+0000 7fa2e5600640 1 -- 172.21.15.86:0/205976270 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e000dc50 con 0x7fa2e815b5f0 2024-04-20T21:18:09.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.209+0000 7fa2f0400640 1 -- 172.21.15.86:0/205976270 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2e81f5e80 con 0x7fa2e815b5f0 2024-04-20T21:18:09.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.209+0000 7fa2f0400640 1 -- 172.21.15.86:0/205976270 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2e81f6440 con 0x7fa2e815b5f0 2024-04-20T21:18:09.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.211+0000 7fa2e5600640 1 -- 172.21.15.86:0/205976270 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa2e001d070 con 0x7fa2e815b5f0 2024-04-20T21:18:09.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.212+0000 7fa2e5600640 1 --2- 172.21.15.86:0/205976270 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa2c806c8f0 0x7fa2c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:09.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.212+0000 7fa2e5600640 1 -- 172.21.15.86:0/205976270 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa2e008c420 con 0x7fa2e815b5f0 2024-04-20T21:18:09.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.212+0000 7fa2eda00640 1 --2- 172.21.15.86:0/205976270 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa2c806c8f0 0x7fa2c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:09.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.213+0000 7fa2f31ce740 1 Processor -- start 2024-04-20T21:18:09.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.213+0000 7fa2eda00640 1 --2- 172.21.15.86:0/205976270 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa2c806c8f0 0x7fa2c806edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc006b10 tx=0x7fa2dc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:09.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.213+0000 7fa2f31ce740 1 -- start start 2024-04-20T21:18:09.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.214+0000 7fa2f31ce740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561a0ad51fc0 0x561a0ad70a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:09.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.214+0000 7fa2f31ce740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a0ad701b0 0x561a0ad2a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:09.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.214+0000 7fa2f31ce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a0ad54770 0x561a0ad54bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:09.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.214+0000 7fa2f31ce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561a0ac909d0 con 0x561a0ad54770 2024-04-20T21:18:09.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.214+0000 7fa2f31ce740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561a0ac37270 con 0x561a0ad701b0 2024-04-20T21:18:09.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.214+0000 7fa2f31ce740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x561a0ac8c990 con 0x561a0ad51fc0 2024-04-20T21:18:09.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.214+0000 7fa2ee400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a0ad54770 0x561a0ad54bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:09.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.214+0000 7fa2ee400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a0ad54770 0x561a0ad54bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46450/0 (socket says 172.21.15.86:46450) 2024-04-20T21:18:09.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.214+0000 7fa2ee400640 1 -- 172.21.15.86:0/1145717108 learned_addr learned my addr 172.21.15.86:0/1145717108 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:09.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.214+0000 7fa2ee400640 1 -- 172.21.15.86:0/1145717108 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561a0ad51fc0 msgr2=0x561a0ad70a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:09.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.214+0000 7fa2ee400640 1 --2- 172.21.15.86:0/1145717108 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561a0ad51fc0 0x561a0ad70a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:09.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.214+0000 7fa2ee400640 1 -- 172.21.15.86:0/1145717108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a0ad701b0 msgr2=0x561a0ad2a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:09.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.214+0000 7fa2ee400640 1 --2- 172.21.15.86:0/1145717108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a0ad701b0 0x561a0ad2a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:09.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.214+0000 7fa2ee400640 1 -- 172.21.15.86:0/1145717108 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a0a9a32d0 con 0x561a0ad54770 2024-04-20T21:18:09.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.215+0000 7fa2ee400640 1 --2- 172.21.15.86:0/1145717108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a0ad54770 0x561a0ad54bf0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa2e00628c0 tx=0x7fa2e005a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f971679523052c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:09.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.215+0000 7fa2da000640 1 -- 172.21.15.86:0/1145717108 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e0058510 con 0x561a0ad54770 2024-04-20T21:18:09.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.215+0000 7fa2da000640 1 -- 172.21.15.86:0/1145717108 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2e0056880 con 0x561a0ad54770 2024-04-20T21:18:09.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.215+0000 7fa2da000640 1 -- 172.21.15.86:0/1145717108 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e005e370 con 0x561a0ad54770 2024-04-20T21:18:09.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.216+0000 7fa2f31ce740 1 -- 172.21.15.86:0/1145717108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a0ad54770 msgr2=0x561a0ad54bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:09.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.216+0000 7fa2f31ce740 1 --2- 172.21.15.86:0/1145717108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a0ad54770 0x561a0ad54bf0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa2e00628c0 tx=0x7fa2e005a110 comp rx=0 tx=0).stop 2024-04-20T21:18:09.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.216+0000 7fa2f31ce740 1 -- 172.21.15.86:0/1145717108 shutdown_connections 2024-04-20T21:18:09.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.216+0000 7fa2f31ce740 1 --2- 172.21.15.86:0/1145717108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a0ad54770 0x561a0ad54bf0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:09.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.216+0000 7fa2f31ce740 1 --2- 172.21.15.86:0/1145717108 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a0ad701b0 0x561a0ad2a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:09.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.216+0000 7fa2f31ce740 1 --2- 172.21.15.86:0/1145717108 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561a0ad51fc0 0x561a0ad70a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:09.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.216+0000 7fa2f31ce740 1 -- 172.21.15.86:0/1145717108 >> 172.21.15.86:0/1145717108 conn(0x561a0ad26bf0 msgr2=0x561a0ad6f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:09.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.216+0000 7fa2f31ce740 1 -- 172.21.15.86:0/1145717108 shutdown_connections 2024-04-20T21:18:09.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.216+0000 7fa2f31ce740 1 -- 172.21.15.86:0/1145717108 wait complete. 2024-04-20T21:18:09.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.217+0000 7fa2f31ce740 1 -- start start 2024-04-20T21:18:09.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.217+0000 7fa2f31ce740 1 Processor -- start 2024-04-20T21:18:09.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.218+0000 7fa2f31ce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a0ad54770 0x561a0ad6ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:09.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.218+0000 7fa2f31ce740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561a0ad701b0 0x561a0ad728d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:09.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.218+0000 7fa2f31ce740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a0ad72e10 0x561a0ad777e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:09.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.218+0000 7fa2f31ce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561a0ad523a0 con 0x561a0ad54770 2024-04-20T21:18:09.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.218+0000 7fa2f31ce740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561a0ac4a5e0 con 0x561a0ad72e10 2024-04-20T21:18:09.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.218+0000 7fa2f31ce740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x561a0ad24930 con 0x561a0ad701b0 2024-04-20T21:18:09.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.218+0000 7fa2f31ce740 10 client.0 _ll_register_callbacks cb 0x561a0a9a2ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:18:09.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.218+0000 7fa2eda00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a0ad54770 0x561a0ad6ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:09.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.218+0000 7fa2eda00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a0ad54770 0x561a0ad6ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46452/0 (socket says 172.21.15.86:46452) 2024-04-20T21:18:09.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.218+0000 7fa2eda00640 1 -- 172.21.15.86:0/1528648643 learned_addr learned my addr 172.21.15.86:0/1528648643 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:09.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.218+0000 7fa2eda00640 1 -- 172.21.15.86:0/1528648643 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561a0ad701b0 msgr2=0x561a0ad728d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:09.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.218+0000 7fa2eda00640 1 --2- 172.21.15.86:0/1528648643 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561a0ad701b0 0x561a0ad728d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:09.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.218+0000 7fa2eda00640 1 -- 172.21.15.86:0/1528648643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a0ad72e10 msgr2=0x561a0ad777e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:09.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.218+0000 7fa2eda00640 1 --2- 172.21.15.86:0/1528648643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a0ad72e10 0x561a0ad777e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:09.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.218+0000 7fa2eda00640 1 -- 172.21.15.86:0/1528648643 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a0aa6d6b0 con 0x561a0ad54770 2024-04-20T21:18:09.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.219+0000 7fa2eda00640 1 --2- 172.21.15.86:0/1528648643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a0ad54770 0x561a0ad6ef10 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc00ce60 tx=0x7fa2dc0137f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a309aed515668cc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:09.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.219+0000 7fa2cea00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:18:09.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.219+0000 7fa2cea00640 1 -- 172.21.15.86:0/1528648643 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc023060 con 0x561a0ad54770 2024-04-20T21:18:09.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.219+0000 7fa2cea00640 1 -- 172.21.15.86:0/1528648643 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa2dc0117e0 con 0x561a0ad54770 2024-04-20T21:18:09.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.219+0000 7fa2f31ce740 1 -- 172.21.15.86:0/1528648643 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2e0056880 con 0x561a0ad54770 2024-04-20T21:18:09.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.219+0000 7fa2cea00640 1 -- 172.21.15.86:0/1528648643 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7fa2dc013e20 con 0x561a0ad54770 2024-04-20T21:18:09.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.219+0000 7fa2f31ce740 10 client.4665 fetch_fsmap learned FSMap version 7 2024-04-20T21:18:09.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.219+0000 7fa2f31ce740 1 -- 172.21.15.86:0/1528648643 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x561a0ad75540 con 0x561a0ad54770 2024-04-20T21:18:09.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.219+0000 7fa2cea00640 1 -- 172.21.15.86:0/1528648643 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2e0056880 con 0x561a0ad54770 2024-04-20T21:18:09.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.219+0000 7fa2cea00640 1 -- 172.21.15.86:0/1528648643 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 7) ==== 3495+0+0 (secure 0 0 0) 0x7fa2dc02d080 con 0x561a0ad54770 2024-04-20T21:18:09.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.220+0000 7fa2f31ce740 10 client.4665 fetch_fsmap finished waiting for FSMap version 7 2024-04-20T21:18:09.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.220+0000 7fa2f31ce740 10 client.4665 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-20T21:18:09.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.220+0000 7fa2f31ce740 20 client.4665 populate_metadata read hostname 'smithi086' 2024-04-20T21:18:09.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.220+0000 7fa2f31ce740 1 --2- 172.21.15.86:0/1528648643 >> [v2:172.21.15.144:6833/2436181584,v1:172.21.15.144:6835/2436181584] conn(0x561a0ad88770 0x561a0ad765b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:09.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.220+0000 7fa2f31ce740 4 client.4665 mds_command: new command op to 4272 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:18:09.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.220+0000 7fa2f31ce740 1 -- 172.21.15.86:0/1528648643 --> [v2:172.21.15.144:6833/2436181584,v1:172.21.15.144:6835/2436181584] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a0ac9e3f0 con 0x561a0ad88770 2024-04-20T21:18:09.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.220+0000 7fa2ed000640 1 --2- 172.21.15.86:0/1528648643 >> [v2:172.21.15.144:6833/2436181584,v1:172.21.15.144:6835/2436181584] conn(0x561a0ad88770 0x561a0ad765b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:09.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.221+0000 7fa2ed000640 1 --2- 172.21.15.86:0/1528648643 >> [v2:172.21.15.144:6833/2436181584,v1:172.21.15.144:6835/2436181584] conn(0x561a0ad88770 0x561a0ad765b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=470f075ebb1ecc4d server_cookie=a6537ef2ce1b8165 in_seq=0 out_seq=0 2024-04-20T21:18:09.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.221+0000 7fa2cea00640 10 client.4665 ms_handle_connect on v2:172.21.15.144:6833/2436181584 2024-04-20T21:18:09.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.224+0000 7fa2cea00640 1 -- 172.21.15.86:0/1528648643 <== mds.0 v2:172.21.15.144:6833/2436181584 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561a0ac9e3f0 con 0x561a0ad88770 2024-04-20T21:18:09.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.224+0000 7fa2cea00640 10 client.4665 handle_command_reply: tid=0 2024-04-20T21:18:09.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.224+0000 7fa2f31ce740 1 client.4665 shutdown 2024-04-20T21:18:09.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.224+0000 7fa2f31ce740 1 -- 172.21.15.86:0/1528648643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a0ad54770 msgr2=0x561a0ad6ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:09.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.224+0000 7fa2f31ce740 1 --2- 172.21.15.86:0/1528648643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a0ad54770 0x561a0ad6ef10 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc00ce60 tx=0x7fa2dc0137f0 comp rx=0 tx=0).stop 2024-04-20T21:18:09.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.224+0000 7fa2f31ce740 1 -- 172.21.15.86:0/1528648643 shutdown_connections 2024-04-20T21:18:09.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.224+0000 7fa2f31ce740 1 --2- 172.21.15.86:0/1528648643 >> [v2:172.21.15.144:6833/2436181584,v1:172.21.15.144:6835/2436181584] conn(0x561a0ad88770 0x561a0ad765b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:09.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.224+0000 7fa2f31ce740 1 --2- 172.21.15.86:0/1528648643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a0ad72e10 0x561a0ad777e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:09.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.224+0000 7fa2f31ce740 1 --2- 172.21.15.86:0/1528648643 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561a0ad701b0 0x561a0ad728d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:09.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.224+0000 7fa2f31ce740 1 --2- 172.21.15.86:0/1528648643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a0ad54770 0x561a0ad6ef10 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:09.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.224+0000 7fa2cea00640 0 client.4665 ms_handle_reset on v2:172.21.15.144:6833/2436181584 2024-04-20T21:18:09.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.224+0000 7fa2f31ce740 1 -- 172.21.15.86:0/1528648643 >> 172.21.15.86:0/1528648643 conn(0x561a0ad51fc0 msgr2=0x561a0ad53a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:09.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.224+0000 7fa2f31ce740 1 -- 172.21.15.86:0/1528648643 shutdown_connections 2024-04-20T21:18:09.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.224+0000 7fa2f31ce740 1 -- 172.21.15.86:0/1528648643 wait complete. 2024-04-20T21:18:09.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.225+0000 7fa2f31ce740 20 client.4665 trim_cache size 0 max 16384 2024-04-20T21:18:09.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.244+0000 7fa2f31ce740 1 Processor -- start 2024-04-20T21:18:09.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.244+0000 7fa2f31ce740 1 -- start start 2024-04-20T21:18:09.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.245+0000 7fa2f31ce740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561a0ad54770 0x561a0ad74970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:09.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.245+0000 7fa2f31ce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a0ad701b0 0x561a0ad53fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:09.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.245+0000 7fa2f31ce740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a0ad72e10 0x561a0ad54b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:09.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.245+0000 7fa2f31ce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e815a750 con 0x561a0ad701b0 2024-04-20T21:18:09.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.245+0000 7fa2f31ce740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e8159760 con 0x561a0ad72e10 2024-04-20T21:18:09.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.245+0000 7fa2f31ce740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa2e8158ac0 con 0x561a0ad54770 2024-04-20T21:18:09.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.245+0000 7fa2ed000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a0ad701b0 0x561a0ad53fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:09.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.245+0000 7fa2ed000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a0ad701b0 0x561a0ad53fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46466/0 (socket says 172.21.15.86:46466) 2024-04-20T21:18:09.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.245+0000 7fa2ed000640 1 -- 172.21.15.86:0/662215007 learned_addr learned my addr 172.21.15.86:0/662215007 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:09.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.245+0000 7fa2ed000640 1 -- 172.21.15.86:0/662215007 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561a0ad54770 msgr2=0x561a0ad74970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:09.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.245+0000 7fa2ed000640 1 --2- 172.21.15.86:0/662215007 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561a0ad54770 0x561a0ad74970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:09.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.245+0000 7fa2ed000640 1 -- 172.21.15.86:0/662215007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a0ad72e10 msgr2=0x561a0ad54b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:09.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.245+0000 7fa2ed000640 1 --2- 172.21.15.86:0/662215007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a0ad72e10 0x561a0ad54b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:09.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.245+0000 7fa2ed000640 1 -- 172.21.15.86:0/662215007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a0ad26300 con 0x561a0ad701b0 2024-04-20T21:18:09.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.246+0000 7fa2ed000640 1 --2- 172.21.15.86:0/662215007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a0ad701b0 0x561a0ad53fa0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa2d402b950 tx=0x7fa2d402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19d80829849ff4a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:09.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.246+0000 7fa2d9600640 1 -- 172.21.15.86:0/662215007 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d404a070 con 0x561a0ad701b0 2024-04-20T21:18:09.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.246+0000 7fa2d9600640 1 -- 172.21.15.86:0/662215007 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2d4031bb0 con 0x561a0ad701b0 2024-04-20T21:18:09.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.246+0000 7fa2d9600640 1 -- 172.21.15.86:0/662215007 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d4044030 con 0x561a0ad701b0 2024-04-20T21:18:09.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.246+0000 7fa2f31ce740 1 -- 172.21.15.86:0/662215007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a0ad701b0 msgr2=0x561a0ad53fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:09.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.246+0000 7fa2f31ce740 1 --2- 172.21.15.86:0/662215007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a0ad701b0 0x561a0ad53fa0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa2d402b950 tx=0x7fa2d402e000 comp rx=0 tx=0).stop 2024-04-20T21:18:09.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.246+0000 7fa2f31ce740 1 -- 172.21.15.86:0/662215007 shutdown_connections 2024-04-20T21:18:09.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.246+0000 7fa2f31ce740 1 --2- 172.21.15.86:0/662215007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a0ad72e10 0x561a0ad54b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:09.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.246+0000 7fa2f31ce740 1 --2- 172.21.15.86:0/662215007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a0ad701b0 0x561a0ad53fa0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:09.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.246+0000 7fa2f31ce740 1 --2- 172.21.15.86:0/662215007 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561a0ad54770 0x561a0ad74970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:09.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.246+0000 7fa2f31ce740 1 -- 172.21.15.86:0/662215007 >> 172.21.15.86:0/662215007 conn(0x561a0ad51fc0 msgr2=0x561a0ad24e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:09.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.246+0000 7fa2f31ce740 1 -- 172.21.15.86:0/662215007 shutdown_connections 2024-04-20T21:18:09.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.247+0000 7fa2f31ce740 1 -- 172.21.15.86:0/662215007 wait complete. 2024-04-20T21:18:09.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.247+0000 7fa2f31ce740 1 -- start start 2024-04-20T21:18:09.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.247+0000 7fa2f31ce740 1 Processor -- start 2024-04-20T21:18:09.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.248+0000 7fa2f31ce740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561a0ad54770 0x561a0ad550f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:09.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.248+0000 7fa2f31ce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a0ad701b0 0x561a0ad71380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:09.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.248+0000 7fa2f31ce740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a0ad72e10 0x561a0adf2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:09.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.248+0000 7fa2f31ce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561a0ad885f0 con 0x561a0ad701b0 2024-04-20T21:18:09.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.248+0000 7fa2f31ce740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e8159110 con 0x561a0ad72e10 2024-04-20T21:18:09.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.248+0000 7fa2f31ce740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x561a0ad88c10 con 0x561a0ad54770 2024-04-20T21:18:09.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.248+0000 7fa2f31ce740 10 client.0 _ll_register_callbacks cb 0x7fa2dc00ad80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:18:09.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.249+0000 7fa2ed000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a0ad701b0 0x561a0ad71380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:09.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.249+0000 7fa2ed000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a0ad701b0 0x561a0ad71380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46472/0 (socket says 172.21.15.86:46472) 2024-04-20T21:18:09.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.249+0000 7fa2ed000640 1 -- 172.21.15.86:0/2465626216 learned_addr learned my addr 172.21.15.86:0/2465626216 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:09.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.249+0000 7fa2eda00640 1 --2- 172.21.15.86:0/2465626216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561a0ad54770 0x561a0ad550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:09.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.249+0000 7fa2ee400640 1 --2- 172.21.15.86:0/2465626216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a0ad72e10 0x561a0adf2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:09.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.249+0000 7fa2ed000640 1 -- 172.21.15.86:0/2465626216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561a0ad54770 msgr2=0x561a0ad550f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:09.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.249+0000 7fa2ed000640 1 --2- 172.21.15.86:0/2465626216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561a0ad54770 0x561a0ad550f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:09.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.249+0000 7fa2ed000640 1 -- 172.21.15.86:0/2465626216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a0ad72e10 msgr2=0x561a0adf2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:09.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.249+0000 7fa2ed000640 1 --2- 172.21.15.86:0/2465626216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a0ad72e10 0x561a0adf2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:09.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.249+0000 7fa2ed000640 1 -- 172.21.15.86:0/2465626216 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a0ad86940 con 0x561a0ad701b0 2024-04-20T21:18:09.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.249+0000 7fa2ee400640 1 --2- 172.21.15.86:0/2465626216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a0ad72e10 0x561a0adf2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:09.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.249+0000 7fa2ed000640 1 --2- 172.21.15.86:0/2465626216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a0ad701b0 0x561a0ad71380 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4048040 tx=0x7fa2d4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69fdcd0f034da84d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:09.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.249+0000 7fa2cea00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:18:09.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.250+0000 7fa2cea00640 1 -- 172.21.15.86:0/2465626216 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d404a040 con 0x561a0ad701b0 2024-04-20T21:18:09.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.250+0000 7fa2cea00640 1 -- 172.21.15.86:0/2465626216 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa2d4004040 con 0x561a0ad701b0 2024-04-20T21:18:09.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.250+0000 7fa2f31ce740 1 -- 172.21.15.86:0/2465626216 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2d4031bb0 con 0x561a0ad701b0 2024-04-20T21:18:09.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.250+0000 7fa2cea00640 1 -- 172.21.15.86:0/2465626216 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7fa2d4057050 con 0x561a0ad701b0 2024-04-20T21:18:09.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.250+0000 7fa2f31ce740 10 client.4671 fetch_fsmap learned FSMap version 7 2024-04-20T21:18:09.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.250+0000 7fa2f31ce740 1 -- 172.21.15.86:0/2465626216 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x561a0ad98060 con 0x561a0ad701b0 2024-04-20T21:18:09.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.250+0000 7fa2cea00640 1 -- 172.21.15.86:0/2465626216 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2d4031bb0 con 0x561a0ad701b0 2024-04-20T21:18:09.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.250+0000 7fa2cea00640 1 -- 172.21.15.86:0/2465626216 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 7) ==== 3495+0+0 (secure 0 0 0) 0x7fa2d405a020 con 0x561a0ad701b0 2024-04-20T21:18:09.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.250+0000 7fa2f31ce740 10 client.4671 fetch_fsmap finished waiting for FSMap version 7 2024-04-20T21:18:09.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.250+0000 7fa2f31ce740 10 client.4671 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-20T21:18:09.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.250+0000 7fa2f31ce740 20 client.4671 populate_metadata read hostname 'smithi086' 2024-04-20T21:18:09.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.250+0000 7fa2f31ce740 1 --2- 172.21.15.86:0/2465626216 >> [v2:172.21.15.144:6833/2436181584,v1:172.21.15.144:6835/2436181584] conn(0x561a0ad83ca0 0x561a0ad719e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:09.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.250+0000 7fa2f31ce740 4 client.4671 mds_command: new command op to 4272 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T21:18:09.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.250+0000 7fa2f31ce740 1 -- 172.21.15.86:0/2465626216 --> [v2:172.21.15.144:6833/2436181584,v1:172.21.15.144:6835/2436181584] -- command(tid 0: {"prefix": "status"}) -- 0x561a0ac98260 con 0x561a0ad83ca0 2024-04-20T21:18:09.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.251+0000 7fa2eda00640 1 --2- 172.21.15.86:0/2465626216 >> [v2:172.21.15.144:6833/2436181584,v1:172.21.15.144:6835/2436181584] conn(0x561a0ad83ca0 0x561a0ad719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:09.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.252+0000 7fa2eda00640 1 --2- 172.21.15.86:0/2465626216 >> [v2:172.21.15.144:6833/2436181584,v1:172.21.15.144:6835/2436181584] conn(0x561a0ad83ca0 0x561a0ad719e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff26e944aaa194ce server_cookie=85f70821a293d324 in_seq=0 out_seq=0 2024-04-20T21:18:09.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.252+0000 7fa2cea00640 10 client.4671 ms_handle_connect on v2:172.21.15.144:6833/2436181584 2024-04-20T21:18:09.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.252+0000 7fa2cea00640 1 -- 172.21.15.86:0/2465626216 <== mds.0 v2:172.21.15.144:6833/2436181584 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x561a0ac98260 con 0x561a0ad83ca0 2024-04-20T21:18:09.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.252+0000 7fa2cea00640 10 client.4671 handle_command_reply: tid=0 2024-04-20T21:18:09.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.252+0000 7fa2f31ce740 1 client.4671 shutdown 2024-04-20T21:18:09.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.252+0000 7fa2f31ce740 1 -- 172.21.15.86:0/2465626216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a0ad701b0 msgr2=0x561a0ad71380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:09.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.252+0000 7fa2f31ce740 1 --2- 172.21.15.86:0/2465626216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a0ad701b0 0x561a0ad71380 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4048040 tx=0x7fa2d4004730 comp rx=0 tx=0).stop 2024-04-20T21:18:09.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.253+0000 7fa2f31ce740 1 -- 172.21.15.86:0/2465626216 shutdown_connections 2024-04-20T21:18:09.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.253+0000 7fa2f31ce740 1 --2- 172.21.15.86:0/2465626216 >> [v2:172.21.15.144:6833/2436181584,v1:172.21.15.144:6835/2436181584] conn(0x561a0ad83ca0 0x561a0ad719e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:09.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.253+0000 7fa2f31ce740 1 --2- 172.21.15.86:0/2465626216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561a0ad72e10 0x561a0adf2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:09.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.253+0000 7fa2f31ce740 1 --2- 172.21.15.86:0/2465626216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a0ad701b0 0x561a0ad71380 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:09.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.253+0000 7fa2f31ce740 1 --2- 172.21.15.86:0/2465626216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561a0ad54770 0x561a0ad550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:09.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.253+0000 7fa2cea00640 0 client.4671 ms_handle_reset on v2:172.21.15.144:6833/2436181584 2024-04-20T21:18:09.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.253+0000 7fa2f31ce740 1 -- 172.21.15.86:0/2465626216 >> 172.21.15.86:0/2465626216 conn(0x561a0ad51fc0 msgr2=0x561a0ad75800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:09.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.253+0000 7fa2f31ce740 1 -- 172.21.15.86:0/2465626216 shutdown_connections 2024-04-20T21:18:09.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.253+0000 7fa2f31ce740 1 -- 172.21.15.86:0/2465626216 wait complete. 2024-04-20T21:18:09.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.253+0000 7fa2f31ce740 20 client.4671 trim_cache size 0 max 16384 2024-04-20T21:18:09.255 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:18:09.255 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "9c2dec1d-d7ae-4a83-a812-79922808e278", 2024-04-20T21:18:09.255 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-04-20T21:18:09.255 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 4272, 2024-04-20T21:18:09.255 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-20T21:18:09.256 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-20T21:18:09.256 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-20T21:18:09.256 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 16.973199812000001, 2024-04-20T21:18:09.256 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 7, 2024-04-20T21:18:09.256 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 20, 2024-04-20T21:18:09.256 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 20, 2024-04-20T21:18:09.256 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 22.425569461999999 2024-04-20T21:18:09.256 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:18:09.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.253+0000 7fa2f0400640 1 -- 172.21.15.86:0/205976270 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa2c806c8f0 msgr2=0x7fa2c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:09.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.253+0000 7fa2f0400640 1 --2- 172.21.15.86:0/205976270 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa2c806c8f0 0x7fa2c806edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc006b10 tx=0x7fa2dc008040 comp rx=0 tx=0).stop 2024-04-20T21:18:09.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.253+0000 7fa2f0400640 1 -- 172.21.15.86:0/205976270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2e815b5f0 msgr2=0x7fa2e815e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:09.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.253+0000 7fa2f0400640 1 --2- 172.21.15.86:0/205976270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2e815b5f0 0x7fa2e815e200 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa2e000cef0 tx=0x7fa2e000a710 comp rx=0 tx=0).stop 2024-04-20T21:18:09.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.254+0000 7fa2f0400640 1 -- 172.21.15.86:0/205976270 shutdown_connections 2024-04-20T21:18:09.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.254+0000 7fa2f0400640 1 --2- 172.21.15.86:0/205976270 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa2c806c8f0 0x7fa2c806edb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:09.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.254+0000 7fa2f0400640 1 --2- 172.21.15.86:0/205976270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2e815b5f0 0x7fa2e815e200 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:09.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.254+0000 7fa2f0400640 1 --2- 172.21.15.86:0/205976270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2e80cff80 0x7fa2e815dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:09.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.254+0000 7fa2f0400640 1 --2- 172.21.15.86:0/205976270 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa2e806f070 0x7fa2e815d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:09.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.254+0000 7fa2f0400640 1 -- 172.21.15.86:0/205976270 >> 172.21.15.86:0/205976270 conn(0x7fa2e806b980 msgr2=0x7fa2e81582f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:09.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.254+0000 7fa2f0400640 1 -- 172.21.15.86:0/205976270 shutdown_connections 2024-04-20T21:18:09.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.254+0000 7fa2f0400640 1 -- 172.21.15.86:0/205976270 wait complete. 2024-04-20T21:18:09.270 INFO:teuthology.run_tasks:Running task sequential... 2024-04-20T21:18:09.279 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-20T21:18:09.279 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-20T21:18:09.280 INFO:teuthology.task.print:Enabling mgr modules 2024-04-20T21:18:09.280 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-20T21:18:09.306 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-20T21:18:09.306 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-20T21:18:09.306 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-20T21:18:09.306 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-20T21:18:09.306 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T21:18:09.306 INFO:tasks.ceph_fuse:client.2 config is {} 2024-04-20T21:18:09.306 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-20T21:18:09.306 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-20T21:18:09.306 INFO:tasks.ceph_fuse:client.1 config is {} 2024-04-20T21:18:09.307 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-20T21:18:09.307 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-20T21:18:09.307 INFO:tasks.ceph_fuse:client.3 config is {} 2024-04-20T21:18:09.307 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-20T21:18:09.307 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-20T21:18:09.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:09.307 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list 2024-04-20T21:18:09.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:09.367 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link delete ceph-brx 2024-04-20T21:18:09.453 INFO:teuthology.orchestra.run.smithi144.stderr:Cannot find device "ceph-brx" 2024-04-20T21:18:09.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:18:09.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:09.457 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-20T21:18:09.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:09.515 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link delete ceph-brx 2024-04-20T21:18:09.604 INFO:teuthology.orchestra.run.smithi086.stderr:Cannot find device "ceph-brx" 2024-04-20T21:18:09.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:18:09.608 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-20T21:18:09.608 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T21:18:09.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.811+0000 7f56c3200640 1 Processor -- start 2024-04-20T21:18:09.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.811+0000 7f56c3200640 1 -- start start 2024-04-20T21:18:09.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.813+0000 7f56c3200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56c415c010 0x7f56c415c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:09.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.813+0000 7f56c3200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f56c40af7b0 0x7f56c40adda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:09.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.813+0000 7f56c3200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56c40ae710 con 0x7f56c406f4c0 2024-04-20T21:18:09.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.813+0000 7f56c3200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f56c40af320 con 0x7f56c40af7b0 2024-04-20T21:18:09.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.813+0000 7f56c3200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f56c415c930 con 0x7f56c415c010 2024-04-20T21:18:09.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.813+0000 7f56c1400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f56c40af7b0 0x7f56c40adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:09.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.813+0000 7f56c1400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f56c40af7b0 0x7f56c40adda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46474/0 (socket says 172.21.15.86:46474) 2024-04-20T21:18:09.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.813+0000 7f56c1400640 1 -- 172.21.15.86:0/3404133608 learned_addr learned my addr 172.21.15.86:0/3404133608 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:09.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.814+0000 7f56c1e00640 1 --2- 172.21.15.86:0/3404133608 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56c415c010 0x7f56c415c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:09.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.814+0000 7f56c1400640 1 -- 172.21.15.86:0/3404133608 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56c415c010 msgr2=0x7f56c415c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:09.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.814+0000 7f56c1400640 1 --2- 172.21.15.86:0/3404133608 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56c415c010 0x7f56c415c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:09.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.814+0000 7f56c1400640 1 -- 172.21.15.86:0/3404133608 >> v1:172.21.15.86:6789/0 conn(0x7f56c406f4c0 legacy=0x7f56c415ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:09.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.814+0000 7f56c1400640 1 -- 172.21.15.86:0/3404133608 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56c40ae9f0 con 0x7f56c40af7b0 2024-04-20T21:18:09.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.815+0000 7f56c1400640 1 --2- 172.21.15.86:0/3404133608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56c40af7b0 0x7f56c40adda0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f56b0009870 tx=0x7f56b002db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43beb9662e0215a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:09.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.815+0000 7f56c0a00640 1 -- 172.21.15.86:0/3404133608 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b003b070 con 0x7f56c40af7b0 2024-04-20T21:18:09.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.815+0000 7f56c0a00640 1 -- 172.21.15.86:0/3404133608 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56b002e940 con 0x7f56c40af7b0 2024-04-20T21:18:09.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.815+0000 7f56c0a00640 1 -- 172.21.15.86:0/3404133608 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b002ed10 con 0x7f56c40af7b0 2024-04-20T21:18:09.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.816+0000 7f56c3200640 1 -- 172.21.15.86:0/3404133608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56c40af7b0 msgr2=0x7f56c40adda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:09.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.816+0000 7f56c3200640 1 --2- 172.21.15.86:0/3404133608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56c40af7b0 0x7f56c40adda0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f56b0009870 tx=0x7f56b002db90 comp rx=0 tx=0).stop 2024-04-20T21:18:09.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.816+0000 7f56c3200640 1 -- 172.21.15.86:0/3404133608 shutdown_connections 2024-04-20T21:18:09.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.816+0000 7f56c3200640 1 --2- 172.21.15.86:0/3404133608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56c40af7b0 0x7f56c40adda0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:09.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.816+0000 7f56c3200640 1 --2- 172.21.15.86:0/3404133608 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56c415c010 0x7f56c415c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:09.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.816+0000 7f56c3200640 1 -- 172.21.15.86:0/3404133608 >> 172.21.15.86:0/3404133608 conn(0x7f56c40d0170 msgr2=0x7f56c40d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:09.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.816+0000 7f56c3200640 1 -- 172.21.15.86:0/3404133608 shutdown_connections 2024-04-20T21:18:09.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.816+0000 7f56c3200640 1 -- 172.21.15.86:0/3404133608 wait complete. 2024-04-20T21:18:09.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.819+0000 7f56c3200640 1 Processor -- start 2024-04-20T21:18:09.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.819+0000 7f56c3200640 1 -- start start 2024-04-20T21:18:09.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.820+0000 7f56c3200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56c406f4c0 0x7f56c41454c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:09.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.820+0000 7f56c3200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56c40af7b0 0x7f56c40cee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:09.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.820+0000 7f56c3200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56c415c010 0x7f56c409b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:09.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.820+0000 7f56c3200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f56c40ae2e0 con 0x7f56c406f4c0 2024-04-20T21:18:09.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.820+0000 7f56c3200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f56c40aefb0 con 0x7f56c40af7b0 2024-04-20T21:18:09.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.820+0000 7f56c3200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f56c40ae460 con 0x7f56c415c010 2024-04-20T21:18:09.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.820+0000 7f56c1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56c406f4c0 0x7f56c41454c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:09.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.820+0000 7f56c1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56c406f4c0 0x7f56c41454c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46492/0 (socket says 172.21.15.86:46492) 2024-04-20T21:18:09.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.820+0000 7f56c1e00640 1 -- 172.21.15.86:0/2483715805 learned_addr learned my addr 172.21.15.86:0/2483715805 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:09.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.821+0000 7f56c1400640 1 --2- 172.21.15.86:0/2483715805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56c40af7b0 0x7f56c40cee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:09.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.821+0000 7f56c2800640 1 --2- 172.21.15.86:0/2483715805 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56c415c010 0x7f56c409b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:09.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.821+0000 7f56c1e00640 1 -- 172.21.15.86:0/2483715805 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56c415c010 msgr2=0x7f56c409b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:09.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.821+0000 7f56c1e00640 1 --2- 172.21.15.86:0/2483715805 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56c415c010 0x7f56c409b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:09.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.821+0000 7f56c1e00640 1 -- 172.21.15.86:0/2483715805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56c40af7b0 msgr2=0x7f56c40cee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:09.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.821+0000 7f56c1e00640 1 --2- 172.21.15.86:0/2483715805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56c40af7b0 0x7f56c40cee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:09.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.821+0000 7f56c1e00640 1 -- 172.21.15.86:0/2483715805 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56c409e210 con 0x7f56c406f4c0 2024-04-20T21:18:09.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.821+0000 7f56c1e00640 1 --2- 172.21.15.86:0/2483715805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56c406f4c0 0x7f56c41454c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f56b4007e40 tx=0x7f56b400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:09.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.821+0000 7f56ba000640 1 -- 172.21.15.86:0/2483715805 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b4014620 con 0x7f56c406f4c0 2024-04-20T21:18:09.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.821+0000 7f56ba000640 1 -- 172.21.15.86:0/2483715805 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56b4019070 con 0x7f56c406f4c0 2024-04-20T21:18:09.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.821+0000 7f56ba000640 1 -- 172.21.15.86:0/2483715805 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b4014a00 con 0x7f56c406f4c0 2024-04-20T21:18:09.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.821+0000 7f56c3200640 1 -- 172.21.15.86:0/2483715805 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56c409df30 con 0x7f56c406f4c0 2024-04-20T21:18:09.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.822+0000 7f56c3200640 1 -- 172.21.15.86:0/2483715805 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56c4159470 con 0x7f56c406f4c0 2024-04-20T21:18:09.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.823+0000 7f56c3200640 1 -- 172.21.15.86:0/2483715805 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56c404fcf0 con 0x7f56c406f4c0 2024-04-20T21:18:09.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.824+0000 7f56ba000640 1 -- 172.21.15.86:0/2483715805 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f56b4014ba0 con 0x7f56c406f4c0 2024-04-20T21:18:09.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.825+0000 7f56ba000640 1 --2- 172.21.15.86:0/2483715805 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f56a406c8f0 0x7f56a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:09.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.825+0000 7f56ba000640 1 -- 172.21.15.86:0/2483715805 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f56b408c3b0 con 0x7f56c406f4c0 2024-04-20T21:18:09.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.825+0000 7f56c1400640 1 --2- 172.21.15.86:0/2483715805 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f56a406c8f0 0x7f56a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:09.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.826+0000 7f56c1400640 1 --2- 172.21.15.86:0/2483715805 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f56a406c8f0 0x7f56a406edb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f56b0038000 tx=0x7f56b0032000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:09.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:09.829+0000 7f56ba000640 1 -- 172.21.15.86:0/2483715805 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f56b405a870 con 0x7f56c406f4c0 2024-04-20T21:18:10.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:10.072+0000 7f56c3200640 1 -- 172.21.15.86:0/2483715805 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f56c415e0a0 con 0x7f56c406f4c0 2024-04-20T21:18:10.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:10.073+0000 7f56ba000640 1 -- 172.21.15.86:0/2483715805 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f56b405a170 con 0x7f56c406f4c0 2024-04-20T21:18:10.072 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T21:18:10.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:10.075+0000 7f56c3200640 1 -- 172.21.15.86:0/2483715805 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f56a406c8f0 msgr2=0x7f56a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:10.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:10.075+0000 7f56c3200640 1 --2- 172.21.15.86:0/2483715805 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f56a406c8f0 0x7f56a406edb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f56b0038000 tx=0x7f56b0032000 comp rx=0 tx=0).stop 2024-04-20T21:18:10.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:10.075+0000 7f56c3200640 1 -- 172.21.15.86:0/2483715805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56c406f4c0 msgr2=0x7f56c41454c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:10.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:10.075+0000 7f56c3200640 1 --2- 172.21.15.86:0/2483715805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56c406f4c0 0x7f56c41454c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f56b4007e40 tx=0x7f56b400e4b0 comp rx=0 tx=0).stop 2024-04-20T21:18:10.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:10.076+0000 7f56c3200640 1 -- 172.21.15.86:0/2483715805 shutdown_connections 2024-04-20T21:18:10.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:10.076+0000 7f56c3200640 1 --2- 172.21.15.86:0/2483715805 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f56a406c8f0 0x7f56a406edb0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:10.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:10.076+0000 7f56c3200640 1 --2- 172.21.15.86:0/2483715805 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56c415c010 0x7f56c409b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:10.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:10.076+0000 7f56c3200640 1 --2- 172.21.15.86:0/2483715805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56c40af7b0 0x7f56c40cee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:10.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:10.076+0000 7f56c3200640 1 --2- 172.21.15.86:0/2483715805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56c406f4c0 0x7f56c41454c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:10.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:10.076+0000 7f56c3200640 1 -- 172.21.15.86:0/2483715805 >> 172.21.15.86:0/2483715805 conn(0x7f56c40d0170 msgr2=0x7f56c40b05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:10.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:10.076+0000 7f56c3200640 1 -- 172.21.15.86:0/2483715805 shutdown_connections 2024-04-20T21:18:10.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:10.076+0000 7f56c3200640 1 -- 172.21.15.86:0/2483715805 wait complete. 2024-04-20T21:18:10.092 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T21:18:10.092 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T21:18:10.092 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-04-20T21:18:10.092 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-20T21:18:10.092 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T21:18:10.092 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T21:18:10.092 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T21:18:10.092 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T21:18:10.092 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:18:10.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:10.093 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-04-20T21:18:10.142 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T21:18:10.142 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T21:18:10.142 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T21:18:10.142 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:18:10.142 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-20T21:18:10.142 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:18:10.142 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:18:10.143 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:18:10.143 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-04-20T21:18:10.143 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:18:10.143 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:18:10.143 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-04-20T21:18:10.143 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:18:10.143 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:18:10.143 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T21:18:10.143 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:18:10.143 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T21:18:10.143 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42666sec preferred_lft 42666sec 2024-04-20T21:18:10.143 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-04-20T21:18:10.143 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:18:10.144 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-20T21:18:10.145 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:18:10.145 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-20T21:18:10.145 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link add name ceph-brx type bridge 2024-04-20T21:18:10.145 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip addr flush dev ceph-brx 2024-04-20T21:18:10.145 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set ceph-brx up 2024-04-20T21:18:10.145 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-20T21:18:10.145 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-20T21:18:10.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:18:10.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:18:10.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:10.668 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T21:18:10.729 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-20T21:18:10.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:10.732 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-04-20T21:18:10.794 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-04-20T21:18:10.794 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T21:18:10.794 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T21:18:10.795 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T21:18:10.795 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T21:18:10.795 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:18:10.795 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-20T21:18:10.795 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T21:18:10.795 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T21:18:10.795 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T21:18:10.795 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-20T21:18:10.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:18:11.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:18:11.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:11.560 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-20T21:18:11.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:11.600 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list-id 2024-04-20T21:18:11.683 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:18:11.683 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-20T21:18:11.683 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:18:11.683 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-20T21:18:11.683 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-20T21:18:11.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:18:11.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:18:11.889 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-20T21:18:11.889 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:18:11.889 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-20T21:18:11.889 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-20T21:18:11.889 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-20T21:18:11.890 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-20T21:18:11.890 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-20T21:18:11.890 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-20T21:18:11.890 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-20T21:18:11.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:18:12.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:18:12.158 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:18:12.158 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-20T21:18:12.158 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-04-20T21:18:12.158 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set dev brx.0 master ceph-brx 2024-04-20T21:18:12.158 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-20T21:18:12.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:18:12.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:18:12.338 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-20T21:18:12.338 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T21:18:12.339 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-20T21:18:12.378 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:18:12.378 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T21:18:12.378 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-20T21:18:12.459 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-04-20T21:18:12.568 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T21:18:12.611 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-04-20T21:18:12.611 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-04-20T21:18:12.611 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-04-20T21:18:12.611 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-04-20T21:18:12.611 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-04-20T21:18:12.611 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-04-20T21:18:12.611 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-04-20T21:18:12.611 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-04-20T21:18:12.612 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-04-20T21:18:12.612 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-04-20T21:18:12.612 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-04-20T21:18:12.612 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/selinux 2024-04-20T21:18:12.612 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T21:18:12.612 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-04-20T21:18:12.612 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-04-20T21:18:12.612 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-04-20T21:18:12.612 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-04-20T21:18:12.612 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-04-20T21:18:12.612 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-04-20T21:18:12.612 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T21:18:12.612 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T21:18:12.612 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T21:18:12.613 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T21:18:12.613 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-04-20T21:18:12.613 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-04-20T21:18:12.613 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T21:18:12.613 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T21:18:12.613 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T21:18:12.613 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T21:18:12.613 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-04-20T21:18:12.613 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:18:12.613 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:18:12.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:12.614 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-04-20T21:18:12.706 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T21:18:12.706 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-20T21:18:12.711 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-04-20T21:18:12.776 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T21:18:12.815 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-04-20T21:18:12.815 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-04-20T21:18:12.816 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-04-20T21:18:12.816 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-04-20T21:18:12.816 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-04-20T21:18:12.816 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-04-20T21:18:12.816 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-04-20T21:18:12.816 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-04-20T21:18:12.816 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-04-20T21:18:12.816 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-04-20T21:18:12.816 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-04-20T21:18:12.816 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/selinux 2024-04-20T21:18:12.816 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T21:18:12.816 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-04-20T21:18:12.816 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-04-20T21:18:12.817 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-04-20T21:18:12.817 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-04-20T21:18:12.817 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-04-20T21:18:12.817 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-04-20T21:18:12.817 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T21:18:12.817 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T21:18:12.817 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T21:18:12.817 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T21:18:12.817 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-04-20T21:18:12.817 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-04-20T21:18:12.817 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T21:18:12.817 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T21:18:12.817 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T21:18:12.817 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T21:18:12.818 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-04-20T21:18:12.818 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:18:12.818 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:18:12.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:12.819 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-04-20T21:18:12.884 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:2024-04-20T21:18:12.885+0000 7f902d635480 -1 init, newargv = 0x55f329ce63e0 newargc=15 2024-04-20T21:18:12.884 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[31256]: starting ceph client 2024-04-20T21:18:12.896 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[31256]: starting fuse 2024-04-20T21:18:12.901 INFO:teuthology.orchestra.run.smithi086.stdout:41 2024-04-20T21:18:12.901 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-04-20T21:18:12.901 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer -- python3 -c ' 2024-04-20T21:18:12.901 DEBUG:teuthology.orchestra.run.smithi086:> import glob 2024-04-20T21:18:12.901 DEBUG:teuthology.orchestra.run.smithi086:> import re 2024-04-20T21:18:12.901 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:18:12.901 DEBUG:teuthology.orchestra.run.smithi086:> import subprocess 2024-04-20T21:18:12.901 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:18:12.901 DEBUG:teuthology.orchestra.run.smithi086:> def _find_admin_socket(client_name): 2024-04-20T21:18:12.902 DEBUG:teuthology.orchestra.run.smithi086:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-20T21:18:12.902 DEBUG:teuthology.orchestra.run.smithi086:> files = glob.glob(asok_path) 2024-04-20T21:18:12.902 DEBUG:teuthology.orchestra.run.smithi086:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-20T21:18:12.902 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:18:12.902 DEBUG:teuthology.orchestra.run.smithi086:> # Given a non-glob path, it better be there 2024-04-20T21:18:12.902 DEBUG:teuthology.orchestra.run.smithi086:> if "*" not in asok_path: 2024-04-20T21:18:12.902 DEBUG:teuthology.orchestra.run.smithi086:> assert(len(files) == 1) 2024-04-20T21:18:12.902 DEBUG:teuthology.orchestra.run.smithi086:> return files[0] 2024-04-20T21:18:12.902 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:18:12.902 DEBUG:teuthology.orchestra.run.smithi086:> for f in files: 2024-04-20T21:18:12.902 DEBUG:teuthology.orchestra.run.smithi086:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T21:18:12.902 DEBUG:teuthology.orchestra.run.smithi086:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T21:18:12.902 DEBUG:teuthology.orchestra.run.smithi086:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T21:18:12.902 DEBUG:teuthology.orchestra.run.smithi086:> contents = proc_f.read() 2024-04-20T21:18:12.902 DEBUG:teuthology.orchestra.run.smithi086:> if mountpoint in contents: 2024-04-20T21:18:12.902 DEBUG:teuthology.orchestra.run.smithi086:> return f 2024-04-20T21:18:12.903 DEBUG:teuthology.orchestra.run.smithi086:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T21:18:12.903 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:18:12.903 DEBUG:teuthology.orchestra.run.smithi086:> print(_find_admin_socket("client.0")) 2024-04-20T21:18:12.903 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:18:13.079 INFO:teuthology.orchestra.run.smithi086.stdout:/var/run/ceph/ceph-client.0.31256.asok 2024-04-20T21:18:13.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:18:13.095 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.31256.asok 2024-04-20T21:18:13.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:13.095 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.31256.asok status 2024-04-20T21:18:13.318 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:18:13.318 INFO:teuthology.orchestra.run.smithi086.stdout: "metadata": { 2024-04-20T21:18:13.318 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T21:18:13.318 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T21:18:13.318 INFO:teuthology.orchestra.run.smithi086.stdout: "entity_id": "0", 2024-04-20T21:18:13.318 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": "smithi086", 2024-04-20T21:18:13.318 INFO:teuthology.orchestra.run.smithi086.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-20T21:18:13.318 INFO:teuthology.orchestra.run.smithi086.stdout: "pid": "31256", 2024-04-20T21:18:13.318 INFO:teuthology.orchestra.run.smithi086.stdout: "root": "/" 2024-04-20T21:18:13.318 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:18:13.319 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_count": 0, 2024-04-20T21:18:13.319 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_pinned_count": 0, 2024-04-20T21:18:13.319 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 4686, 2024-04-20T21:18:13.319 INFO:teuthology.orchestra.run.smithi086.stdout: "inst": { 2024-04-20T21:18:13.319 INFO:teuthology.orchestra.run.smithi086.stdout: "name": { 2024-04-20T21:18:13.319 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "client", 2024-04-20T21:18:13.319 INFO:teuthology.orchestra.run.smithi086.stdout: "num": 4686 2024-04-20T21:18:13.319 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:18:13.319 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2024-04-20T21:18:13.319 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-04-20T21:18:13.319 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2024-04-20T21:18:13.319 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 3777284032 2024-04-20T21:18:13.319 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:18:13.319 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:18:13.319 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2024-04-20T21:18:13.319 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-04-20T21:18:13.320 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2024-04-20T21:18:13.320 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 3777284032 2024-04-20T21:18:13.320 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:18:13.320 INFO:teuthology.orchestra.run.smithi086.stdout: "inst_str": "client.4686 192.168.0.1:0/3777284032", 2024-04-20T21:18:13.320 INFO:teuthology.orchestra.run.smithi086.stdout: "addr_str": "192.168.0.1:0/3777284032", 2024-04-20T21:18:13.320 INFO:teuthology.orchestra.run.smithi086.stdout: "inode_count": 1, 2024-04-20T21:18:13.320 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_epoch": 7, 2024-04-20T21:18:13.320 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch": 20, 2024-04-20T21:18:13.320 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch_barrier": 0, 2024-04-20T21:18:13.320 INFO:teuthology.orchestra.run.smithi086.stdout: "blocklisted": false, 2024-04-20T21:18:13.320 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs" 2024-04-20T21:18:13.320 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:18:13.332 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T21:18:13.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.530+0000 7f1ffca00640 1 Processor -- start 2024-04-20T21:18:13.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.530+0000 7f1ffca00640 1 -- start start 2024-04-20T21:18:13.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.531+0000 7f1ffca00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1ff815ece0 0x7f1ff815f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:13.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.531+0000 7f1ffca00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1ff80d50c0 0x7f1ff80d3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:13.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.531+0000 7f1ffca00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ff80cf9c0 0x7f1ff8160000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:13.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.531+0000 7f1ffca00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f1ff815f600 con 0x7f1ff815ece0 2024-04-20T21:18:13.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.531+0000 7f1ffca00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f1ff80d3950 con 0x7f1ff80d50c0 2024-04-20T21:18:13.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.531+0000 7f1ffca00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1ff80cec00 con 0x7f1ff80cf9c0 2024-04-20T21:18:13.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.532+0000 7f1ff7400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1ff815ece0 0x7f1ff815f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:13.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.532+0000 7f1ff7400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1ff815ece0 0x7f1ff815f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46504/0 (socket says 172.21.15.86:46504) 2024-04-20T21:18:13.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.532+0000 7f1ff7400640 1 -- 172.21.15.86:0/60480245 learned_addr learned my addr 172.21.15.86:0/60480245 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:13.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.532+0000 7f1ff6a00640 1 --2- 172.21.15.86:0/60480245 >> v2:172.21.15.144:3300/0 conn(0x7f1ff80d50c0 0x7f1ff80d3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:13.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.532+0000 7f1ff7e00640 1 --2- 172.21.15.86:0/60480245 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ff80cf9c0 0x7f1ff8160000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:13.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.532+0000 7f1ff7400640 1 -- 172.21.15.86:0/60480245 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ff80cf9c0 msgr2=0x7f1ff8160000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:13.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.532+0000 7f1ff7400640 1 --2- 172.21.15.86:0/60480245 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ff80cf9c0 0x7f1ff8160000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:13.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.532+0000 7f1ff7400640 1 -- 172.21.15.86:0/60480245 >> v2:172.21.15.144:3300/0 conn(0x7f1ff80d50c0 msgr2=0x7f1ff80d3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:13.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.532+0000 7f1ff7400640 1 --2- 172.21.15.86:0/60480245 >> v2:172.21.15.144:3300/0 conn(0x7f1ff80d50c0 0x7f1ff80d3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:13.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.532+0000 7f1ff7400640 1 -- 172.21.15.86:0/60480245 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ff80ceee0 con 0x7f1ff815ece0 2024-04-20T21:18:13.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.533+0000 7f1ff7400640 1 --2- 172.21.15.86:0/60480245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ff815ece0 0x7f1ff815f0c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1ff80ceac0 tx=0x7f1fe802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9efa74bf9e266e1e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:13.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.534+0000 7f1ff6000640 1 -- 172.21.15.86:0/60480245 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe802e070 con 0x7f1ff815ece0 2024-04-20T21:18:13.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.534+0000 7f1ff6000640 1 -- 172.21.15.86:0/60480245 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fe8037070 con 0x7f1ff815ece0 2024-04-20T21:18:13.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.534+0000 7f1ff6000640 1 -- 172.21.15.86:0/60480245 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe8031a00 con 0x7f1ff815ece0 2024-04-20T21:18:13.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.534+0000 7f1ffca00640 1 -- 172.21.15.86:0/60480245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ff815ece0 msgr2=0x7f1ff815f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:13.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.534+0000 7f1ffca00640 1 --2- 172.21.15.86:0/60480245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ff815ece0 0x7f1ff815f0c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1ff80ceac0 tx=0x7f1fe802b880 comp rx=0 tx=0).stop 2024-04-20T21:18:13.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.534+0000 7f1ffca00640 1 -- 172.21.15.86:0/60480245 shutdown_connections 2024-04-20T21:18:13.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.534+0000 7f1ffca00640 1 --2- 172.21.15.86:0/60480245 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ff80cf9c0 0x7f1ff8160000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:13.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.534+0000 7f1ffca00640 1 --2- 172.21.15.86:0/60480245 >> v2:172.21.15.144:3300/0 conn(0x7f1ff80d50c0 0x7f1ff80d3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:13.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.534+0000 7f1ffca00640 1 --2- 172.21.15.86:0/60480245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ff815ece0 0x7f1ff815f0c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:13.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.534+0000 7f1ffca00640 1 -- 172.21.15.86:0/60480245 >> 172.21.15.86:0/60480245 conn(0x7f1ff808bf70 msgr2=0x7f1ff80d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:13.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.534+0000 7f1ffca00640 1 -- 172.21.15.86:0/60480245 shutdown_connections 2024-04-20T21:18:13.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.534+0000 7f1ffca00640 1 -- 172.21.15.86:0/60480245 wait complete. 2024-04-20T21:18:13.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.536+0000 7f1ffca00640 1 Processor -- start 2024-04-20T21:18:13.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.536+0000 7f1ffca00640 1 -- start start 2024-04-20T21:18:13.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.537+0000 7f1ffca00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ff80cf9c0 0x7f1ff80b1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:13.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.538+0000 7f1ffca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ff80d50c0 0x7f1ff80a1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:13.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.538+0000 7f1ffca00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ff815ece0 0x7f1ff8162420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:13.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.538+0000 7f1ffca00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff80d3640 con 0x7f1ff80d50c0 2024-04-20T21:18:13.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.538+0000 7f1ffca00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff80cf4a0 con 0x7f1ff815ece0 2024-04-20T21:18:13.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.538+0000 7f1ffca00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1ff815ea50 con 0x7f1ff80cf9c0 2024-04-20T21:18:13.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.538+0000 7f1ff6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ff80d50c0 0x7f1ff80a1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:13.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.538+0000 7f1ff6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ff80d50c0 0x7f1ff80a1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46512/0 (socket says 172.21.15.86:46512) 2024-04-20T21:18:13.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.538+0000 7f1ff6a00640 1 -- 172.21.15.86:0/2596762250 learned_addr learned my addr 172.21.15.86:0/2596762250 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:13.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.538+0000 7f1ff7400640 1 --2- 172.21.15.86:0/2596762250 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ff80cf9c0 0x7f1ff80b1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:13.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.538+0000 7f1ff7e00640 1 --2- 172.21.15.86:0/2596762250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ff815ece0 0x7f1ff8162420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:13.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.538+0000 7f1ff6a00640 1 -- 172.21.15.86:0/2596762250 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ff80cf9c0 msgr2=0x7f1ff80b1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:13.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.538+0000 7f1ff6a00640 1 --2- 172.21.15.86:0/2596762250 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ff80cf9c0 0x7f1ff80b1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:13.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.538+0000 7f1ff6a00640 1 -- 172.21.15.86:0/2596762250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ff815ece0 msgr2=0x7f1ff8162420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:13.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.538+0000 7f1ff6a00640 1 --2- 172.21.15.86:0/2596762250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ff815ece0 0x7f1ff8162420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:13.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.538+0000 7f1ff6a00640 1 -- 172.21.15.86:0/2596762250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ff809fe30 con 0x7f1ff80d50c0 2024-04-20T21:18:13.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.538+0000 7f1ff7400640 1 --2- 172.21.15.86:0/2596762250 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ff80cf9c0 0x7f1ff80b1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:13.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.539+0000 7f1ff6a00640 1 --2- 172.21.15.86:0/2596762250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ff80d50c0 0x7f1ff80a1c50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1fe400e980 tx=0x7f1fe400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:13.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.539+0000 7f1fef400640 1 -- 172.21.15.86:0/2596762250 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe4017070 con 0x7f1ff80d50c0 2024-04-20T21:18:13.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.539+0000 7f1fef400640 1 -- 172.21.15.86:0/2596762250 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fe400ca00 con 0x7f1ff80d50c0 2024-04-20T21:18:13.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.539+0000 7f1fef400640 1 -- 172.21.15.86:0/2596762250 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe400cdd0 con 0x7f1ff80d50c0 2024-04-20T21:18:13.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.539+0000 7f1ffca00640 1 -- 172.21.15.86:0/2596762250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ff80a0110 con 0x7f1ff80d50c0 2024-04-20T21:18:13.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.539+0000 7f1ffca00640 1 -- 172.21.15.86:0/2596762250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ff80a06d0 con 0x7f1ff80d50c0 2024-04-20T21:18:13.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.540+0000 7f1ffca00640 1 -- 172.21.15.86:0/2596762250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ff80b9f70 con 0x7f1ff80d50c0 2024-04-20T21:18:13.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.541+0000 7f1fef400640 1 -- 172.21.15.86:0/2596762250 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1fe401a070 con 0x7f1ff80d50c0 2024-04-20T21:18:13.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.542+0000 7f1fef400640 1 --2- 172.21.15.86:0/2596762250 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1fd806c8f0 0x7f1fd806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:13.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.542+0000 7f1fef400640 1 -- 172.21.15.86:0/2596762250 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1fe408bef0 con 0x7f1ff80d50c0 2024-04-20T21:18:13.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.543+0000 7f1ff7400640 1 --2- 172.21.15.86:0/2596762250 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1fd806c8f0 0x7f1fd806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:13.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.543+0000 7f1ff7400640 1 --2- 172.21.15.86:0/2596762250 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1fd806c8f0 0x7f1fd806edb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8036000 tx=0x7f1fe8008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:13.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.547+0000 7f1fef400640 1 -- 172.21.15.86:0/2596762250 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1ff80b9f70 con 0x7f1ff80d50c0 2024-04-20T21:18:13.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.771+0000 7f1ffca00640 1 -- 172.21.15.86:0/2596762250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1ff808e320 con 0x7f1ff80d50c0 2024-04-20T21:18:13.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.772+0000 7f1fef400640 1 -- 172.21.15.86:0/2596762250 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f1fe405a310 con 0x7f1ff80d50c0 2024-04-20T21:18:13.771 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T21:18:13.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.774+0000 7f1ffca00640 1 -- 172.21.15.86:0/2596762250 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1fd806c8f0 msgr2=0x7f1fd806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:13.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.774+0000 7f1ffca00640 1 --2- 172.21.15.86:0/2596762250 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1fd806c8f0 0x7f1fd806edb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8036000 tx=0x7f1fe8008030 comp rx=0 tx=0).stop 2024-04-20T21:18:13.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.774+0000 7f1ffca00640 1 -- 172.21.15.86:0/2596762250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ff80d50c0 msgr2=0x7f1ff80a1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:13.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.774+0000 7f1ffca00640 1 --2- 172.21.15.86:0/2596762250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ff80d50c0 0x7f1ff80a1c50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1fe400e980 tx=0x7f1fe400ee40 comp rx=0 tx=0).stop 2024-04-20T21:18:13.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.774+0000 7f1ffca00640 1 -- 172.21.15.86:0/2596762250 shutdown_connections 2024-04-20T21:18:13.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.774+0000 7f1ffca00640 1 --2- 172.21.15.86:0/2596762250 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1fd806c8f0 0x7f1fd806edb0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:13.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.774+0000 7f1ffca00640 1 --2- 172.21.15.86:0/2596762250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ff815ece0 0x7f1ff8162420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:13.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.774+0000 7f1ffca00640 1 --2- 172.21.15.86:0/2596762250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ff80d50c0 0x7f1ff80a1c50 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:13.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.774+0000 7f1ffca00640 1 --2- 172.21.15.86:0/2596762250 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ff80cf9c0 0x7f1ff80b1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:13.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.774+0000 7f1ffca00640 1 -- 172.21.15.86:0/2596762250 >> 172.21.15.86:0/2596762250 conn(0x7f1ff808bf70 msgr2=0x7f1ff8160990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:13.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.775+0000 7f1ffca00640 1 -- 172.21.15.86:0/2596762250 shutdown_connections 2024-04-20T21:18:13.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:13.775+0000 7f1ffca00640 1 -- 172.21.15.86:0/2596762250 wait complete. 2024-04-20T21:18:13.791 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T21:18:13.792 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T21:18:13.792 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-04-20T21:18:13.792 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-20T21:18:13.792 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-20T21:18:13.792 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T21:18:13.792 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T21:18:13.792 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T21:18:13.792 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-20T21:18:13.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:13.792 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-04-20T21:18:13.840 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T21:18:13.841 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T21:18:13.841 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T21:18:13.841 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:18:13.841 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-20T21:18:13.841 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:18:13.841 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:18:13.841 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:18:13.841 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-04-20T21:18:13.841 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:18:13.841 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:18:13.841 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-04-20T21:18:13.841 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:18:13.841 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:18:13.841 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T21:18:13.841 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:18:13.842 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T21:18:13.842 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42662sec preferred_lft 42662sec 2024-04-20T21:18:13.842 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-04-20T21:18:13.842 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:18:13.842 INFO:teuthology.orchestra.run.smithi086.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T21:18:13.842 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 86:25:97:2c:9c:bb brd ff:ff:ff:ff:ff:ff 2024-04-20T21:18:13.842 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-20T21:18:13.842 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:18:13.842 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::a4fb:30ff:fe90:2d6a/64 scope link 2024-04-20T21:18:13.842 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:18:13.842 INFO:teuthology.orchestra.run.smithi086.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T21:18:13.842 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 86:25:97:2c:9c:bb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:18:13.842 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::8425:97ff:fe2c:9cbb/64 scope link 2024-04-20T21:18:13.842 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:18:13.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:13.843 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T21:18:13.922 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-20T21:18:13.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:13.927 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-04-20T21:18:13.976 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-04-20T21:18:13.976 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T21:18:13.976 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T21:18:13.976 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T21:18:13.976 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T21:18:13.977 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:18:13.977 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-20T21:18:13.977 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T21:18:13.977 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T21:18:13.977 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T21:18:13.977 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-20T21:18:14.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:18:14.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:18:14.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:14.181 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-20T21:18:14.228 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-20T21:18:14.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:14.229 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list-id 2024-04-20T21:18:14.321 INFO:teuthology.orchestra.run.smithi086.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-20T21:18:14.322 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:18:14.322 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-20T21:18:14.323 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:18:14.323 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-20T21:18:14.323 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-20T21:18:14.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:18:14.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:18:14.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:14.505 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-20T21:18:14.579 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T21:18:14.579 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T21:18:14.580 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T21:18:14.580 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:18:14.580 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-20T21:18:14.580 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:18:14.580 INFO:teuthology.orchestra.run.smithi086.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T21:18:14.580 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 3e:74:4a:57:ec:c0 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-20T21:18:14.580 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-20T21:18:14.580 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:18:14.580 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::3c74:4aff:fe57:ecc0/64 scope link 2024-04-20T21:18:14.580 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:18:14.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:14.584 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-20T21:18:14.666 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T21:18:14.666 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T21:18:14.666 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T21:18:14.666 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:18:14.666 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-20T21:18:14.666 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:18:14.666 INFO:teuthology.orchestra.run.smithi086.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T21:18:14.666 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 3e:74:4a:57:ec:c0 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-20T21:18:14.666 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-20T21:18:14.667 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:18:14.667 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::3c74:4aff:fe57:ecc0/64 scope link 2024-04-20T21:18:14.667 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:18:14.670 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-20T21:18:14.670 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:18:14.670 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-20T21:18:14.670 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-20T21:18:14.670 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-20T21:18:14.671 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-20T21:18:14.671 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-20T21:18:14.671 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-20T21:18:14.671 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-20T21:18:14.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:18:14.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:18:14.942 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:18:14.942 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-20T21:18:14.942 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.1 up 2024-04-20T21:18:14.943 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set dev brx.1 master ceph-brx 2024-04-20T21:18:14.943 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-20T21:18:15.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:18:15.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:18:15.148 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-04-20T21:18:15.148 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T21:18:15.148 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-20T21:18:15.192 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-20T21:18:15.193 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T21:18:15.193 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-20T21:18:15.270 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:18:15.287 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:18:15.288 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-04-20T21:18:15.401 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T21:18:15.494 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-04-20T21:18:15.494 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-04-20T21:18:15.495 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-04-20T21:18:15.495 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-04-20T21:18:15.495 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-04-20T21:18:15.495 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-04-20T21:18:15.495 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-04-20T21:18:15.495 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-04-20T21:18:15.495 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-04-20T21:18:15.495 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-04-20T21:18:15.495 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-04-20T21:18:15.495 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/selinux 2024-04-20T21:18:15.495 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T21:18:15.495 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-04-20T21:18:15.495 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-04-20T21:18:15.496 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-04-20T21:18:15.496 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-04-20T21:18:15.496 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-04-20T21:18:15.496 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-04-20T21:18:15.496 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T21:18:15.496 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T21:18:15.496 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T21:18:15.496 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T21:18:15.496 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-04-20T21:18:15.496 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-04-20T21:18:15.496 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T21:18:15.496 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T21:18:15.496 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T21:18:15.496 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T21:18:15.497 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-04-20T21:18:15.497 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:18:15.497 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:18:15.497 INFO:teuthology.orchestra.run.smithi086.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-20T21:18:15.497 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:18:15.497 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:18:15.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:15.498 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-04-20T21:18:15.549 INFO:teuthology.orchestra.run.smithi086.stdout:41 2024-04-20T21:18:15.549 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [41] 2024-04-20T21:18:15.550 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-04-20T21:18:15.557 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-04-20T21:18:15.626 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T21:18:15.668 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-04-20T21:18:15.669 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-04-20T21:18:15.669 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-04-20T21:18:15.669 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-04-20T21:18:15.669 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-04-20T21:18:15.669 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-04-20T21:18:15.669 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-04-20T21:18:15.669 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-04-20T21:18:15.669 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-04-20T21:18:15.669 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-04-20T21:18:15.669 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-04-20T21:18:15.669 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/selinux 2024-04-20T21:18:15.669 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T21:18:15.670 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-04-20T21:18:15.670 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-04-20T21:18:15.670 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-04-20T21:18:15.670 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-04-20T21:18:15.670 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-04-20T21:18:15.670 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-04-20T21:18:15.670 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T21:18:15.670 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T21:18:15.670 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T21:18:15.670 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T21:18:15.670 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-04-20T21:18:15.670 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-04-20T21:18:15.670 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T21:18:15.670 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T21:18:15.670 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T21:18:15.671 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T21:18:15.671 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-04-20T21:18:15.671 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:18:15.671 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:18:15.671 INFO:teuthology.orchestra.run.smithi086.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-20T21:18:15.671 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:18:15.671 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:18:15.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:15.672 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-04-20T21:18:15.722 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi086.stderr:2024-04-20T21:18:15.723+0000 7f0dbe43c480 -1 init, newargv = 0x55bb186451e0 newargc=15 2024-04-20T21:18:15.723 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi086.stderr:ceph-fuse[31772]: starting ceph client 2024-04-20T21:18:15.734 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi086.stderr:ceph-fuse[31772]: starting fuse 2024-04-20T21:18:15.753 INFO:teuthology.orchestra.run.smithi086.stdout:41 2024-04-20T21:18:15.753 INFO:teuthology.orchestra.run.smithi086.stdout:43 2024-04-20T21:18:15.753 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41, 43] 2024-04-20T21:18:15.753 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer -- python3 -c ' 2024-04-20T21:18:15.753 DEBUG:teuthology.orchestra.run.smithi086:> import glob 2024-04-20T21:18:15.753 DEBUG:teuthology.orchestra.run.smithi086:> import re 2024-04-20T21:18:15.753 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:18:15.753 DEBUG:teuthology.orchestra.run.smithi086:> import subprocess 2024-04-20T21:18:15.754 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:18:15.754 DEBUG:teuthology.orchestra.run.smithi086:> def _find_admin_socket(client_name): 2024-04-20T21:18:15.754 DEBUG:teuthology.orchestra.run.smithi086:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-04-20T21:18:15.754 DEBUG:teuthology.orchestra.run.smithi086:> files = glob.glob(asok_path) 2024-04-20T21:18:15.754 DEBUG:teuthology.orchestra.run.smithi086:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-04-20T21:18:15.754 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:18:15.754 DEBUG:teuthology.orchestra.run.smithi086:> # Given a non-glob path, it better be there 2024-04-20T21:18:15.754 DEBUG:teuthology.orchestra.run.smithi086:> if "*" not in asok_path: 2024-04-20T21:18:15.754 DEBUG:teuthology.orchestra.run.smithi086:> assert(len(files) == 1) 2024-04-20T21:18:15.754 DEBUG:teuthology.orchestra.run.smithi086:> return files[0] 2024-04-20T21:18:15.754 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:18:15.754 DEBUG:teuthology.orchestra.run.smithi086:> for f in files: 2024-04-20T21:18:15.754 DEBUG:teuthology.orchestra.run.smithi086:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T21:18:15.754 DEBUG:teuthology.orchestra.run.smithi086:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T21:18:15.754 DEBUG:teuthology.orchestra.run.smithi086:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T21:18:15.754 DEBUG:teuthology.orchestra.run.smithi086:> contents = proc_f.read() 2024-04-20T21:18:15.755 DEBUG:teuthology.orchestra.run.smithi086:> if mountpoint in contents: 2024-04-20T21:18:15.755 DEBUG:teuthology.orchestra.run.smithi086:> return f 2024-04-20T21:18:15.755 DEBUG:teuthology.orchestra.run.smithi086:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T21:18:15.755 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:18:15.755 DEBUG:teuthology.orchestra.run.smithi086:> print(_find_admin_socket("client.2")) 2024-04-20T21:18:15.755 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:18:15.943 INFO:teuthology.orchestra.run.smithi086.stdout:/var/run/ceph/ceph-client.2.31772.asok 2024-04-20T21:18:15.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:18:15.960 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.31772.asok 2024-04-20T21:18:15.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:15.960 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.31772.asok status 2024-04-20T21:18:16.182 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:18:16.182 INFO:teuthology.orchestra.run.smithi086.stdout: "metadata": { 2024-04-20T21:18:16.182 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T21:18:16.182 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T21:18:16.182 INFO:teuthology.orchestra.run.smithi086.stdout: "entity_id": "2", 2024-04-20T21:18:16.182 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": "smithi086", 2024-04-20T21:18:16.183 INFO:teuthology.orchestra.run.smithi086.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-04-20T21:18:16.183 INFO:teuthology.orchestra.run.smithi086.stdout: "pid": "31772", 2024-04-20T21:18:16.183 INFO:teuthology.orchestra.run.smithi086.stdout: "root": "/" 2024-04-20T21:18:16.183 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:18:16.183 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_count": 0, 2024-04-20T21:18:16.183 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_pinned_count": 0, 2024-04-20T21:18:16.183 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 4695, 2024-04-20T21:18:16.183 INFO:teuthology.orchestra.run.smithi086.stdout: "inst": { 2024-04-20T21:18:16.183 INFO:teuthology.orchestra.run.smithi086.stdout: "name": { 2024-04-20T21:18:16.183 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "client", 2024-04-20T21:18:16.183 INFO:teuthology.orchestra.run.smithi086.stdout: "num": 4695 2024-04-20T21:18:16.183 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:18:16.183 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2024-04-20T21:18:16.183 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-04-20T21:18:16.183 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.2:0", 2024-04-20T21:18:16.184 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 4237161102 2024-04-20T21:18:16.184 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:18:16.184 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:18:16.184 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2024-04-20T21:18:16.184 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-04-20T21:18:16.184 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.2:0", 2024-04-20T21:18:16.184 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 4237161102 2024-04-20T21:18:16.184 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:18:16.184 INFO:teuthology.orchestra.run.smithi086.stdout: "inst_str": "client.4695 192.168.0.2:0/4237161102", 2024-04-20T21:18:16.184 INFO:teuthology.orchestra.run.smithi086.stdout: "addr_str": "192.168.0.2:0/4237161102", 2024-04-20T21:18:16.184 INFO:teuthology.orchestra.run.smithi086.stdout: "inode_count": 1, 2024-04-20T21:18:16.184 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_epoch": 7, 2024-04-20T21:18:16.184 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch": 20, 2024-04-20T21:18:16.184 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch_barrier": 0, 2024-04-20T21:18:16.184 INFO:teuthology.orchestra.run.smithi086.stdout: "blocklisted": false, 2024-04-20T21:18:16.185 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs" 2024-04-20T21:18:16.185 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:18:16.196 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T21:18:16.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.402+0000 7f4455800640 1 Processor -- start 2024-04-20T21:18:16.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.402+0000 7f4455800640 1 -- start start 2024-04-20T21:18:16.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.404+0000 7f4455800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4450146130 0x7f445006e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:16.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.404+0000 7f4455800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4450072410 0x7f44500cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:16.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.404+0000 7f4455800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f445006dc30 con 0x7f44500ccdd0 2024-04-20T21:18:16.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.404+0000 7f4455800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f44500cd8e0 con 0x7f4450146130 2024-04-20T21:18:16.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.404+0000 7f4455800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f44500cdbc0 con 0x7f4450072410 2024-04-20T21:18:16.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.404+0000 7f444f400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4450072410 0x7f44500cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:16.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.405+0000 7f444ea00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4450146130 0x7f445006e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:16.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.405+0000 7f444f400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4450072410 0x7f44500cd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:57004/0 (socket says 172.21.15.86:57004) 2024-04-20T21:18:16.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.405+0000 7f444f400640 1 -- 172.21.15.86:0/2088549890 learned_addr learned my addr 172.21.15.86:0/2088549890 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:16.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.405+0000 7f444f400640 1 -- 172.21.15.86:0/2088549890 >> v2:172.21.15.144:3300/0 conn(0x7f4450146130 msgr2=0x7f445006e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:16.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.405+0000 7f444f400640 1 --2- 172.21.15.86:0/2088549890 >> v2:172.21.15.144:3300/0 conn(0x7f4450146130 0x7f445006e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:16.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.405+0000 7f444f400640 1 -- 172.21.15.86:0/2088549890 >> v1:172.21.15.144:6789/0 conn(0x7f44500ccdd0 legacy=0x7f4450071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:16.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.405+0000 7f444f400640 1 -- 172.21.15.86:0/2088549890 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f445006ded0 con 0x7f4450072410 2024-04-20T21:18:16.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.407+0000 7f444f400640 1 --2- 172.21.15.86:0/2088549890 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4450072410 0x7f44500cd230 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4448009e50 tx=0x7f444802bff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad63759403ed77d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:16.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.407+0000 7f444e000640 1 -- 172.21.15.86:0/2088549890 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f444802cb90 con 0x7f4450072410 2024-04-20T21:18:16.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.407+0000 7f444e000640 1 -- 172.21.15.86:0/2088549890 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f444802cd30 con 0x7f4450072410 2024-04-20T21:18:16.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.407+0000 7f444e000640 1 -- 172.21.15.86:0/2088549890 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4448034610 con 0x7f4450072410 2024-04-20T21:18:16.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.408+0000 7f4455800640 1 -- 172.21.15.86:0/2088549890 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4450072410 msgr2=0x7f44500cd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:16.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.408+0000 7f4455800640 1 --2- 172.21.15.86:0/2088549890 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4450072410 0x7f44500cd230 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4448009e50 tx=0x7f444802bff0 comp rx=0 tx=0).stop 2024-04-20T21:18:16.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.408+0000 7f4455800640 1 -- 172.21.15.86:0/2088549890 shutdown_connections 2024-04-20T21:18:16.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.408+0000 7f4455800640 1 --2- 172.21.15.86:0/2088549890 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4450072410 0x7f44500cd230 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:16.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.408+0000 7f4455800640 1 --2- 172.21.15.86:0/2088549890 >> v2:172.21.15.144:3300/0 conn(0x7f4450146130 0x7f445006e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:16.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.408+0000 7f4455800640 1 -- 172.21.15.86:0/2088549890 >> 172.21.15.86:0/2088549890 conn(0x7f44500b2360 msgr2=0x7f44500b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:16.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.408+0000 7f4455800640 1 -- 172.21.15.86:0/2088549890 shutdown_connections 2024-04-20T21:18:16.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.408+0000 7f4455800640 1 -- 172.21.15.86:0/2088549890 wait complete. 2024-04-20T21:18:16.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.410+0000 7f4455800640 1 Processor -- start 2024-04-20T21:18:16.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.410+0000 7f4455800640 1 -- start start 2024-04-20T21:18:16.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.412+0000 7f4455800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4450072410 0x7f4450148a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:16.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.412+0000 7f4455800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44500ccdd0 0x7f44501f6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:16.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.412+0000 7f4455800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4450146130 0x7f445014aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:16.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.412+0000 7f4455800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f445015cc20 con 0x7f4450072410 2024-04-20T21:18:16.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.412+0000 7f4455800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f445006f150 con 0x7f4450146130 2024-04-20T21:18:16.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.412+0000 7f4455800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f445015cda0 con 0x7f44500ccdd0 2024-04-20T21:18:16.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.412+0000 7f444ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4450072410 0x7f4450148a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:16.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.412+0000 7f444ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4450072410 0x7f4450148a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44876/0 (socket says 172.21.15.86:44876) 2024-04-20T21:18:16.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.412+0000 7f444ea00640 1 -- 172.21.15.86:0/520104781 learned_addr learned my addr 172.21.15.86:0/520104781 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:16.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.412+0000 7f4447e00640 1 --2- 172.21.15.86:0/520104781 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44500ccdd0 0x7f44501f6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:16.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.412+0000 7f444f400640 1 --2- 172.21.15.86:0/520104781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4450146130 0x7f445014aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:16.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.412+0000 7f444ea00640 1 -- 172.21.15.86:0/520104781 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44500ccdd0 msgr2=0x7f44501f6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:16.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.412+0000 7f444ea00640 1 --2- 172.21.15.86:0/520104781 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44500ccdd0 0x7f44501f6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:16.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.412+0000 7f444ea00640 1 -- 172.21.15.86:0/520104781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4450146130 msgr2=0x7f445014aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:16.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.412+0000 7f444ea00640 1 --2- 172.21.15.86:0/520104781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4450146130 0x7f445014aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:16.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.412+0000 7f444ea00640 1 -- 172.21.15.86:0/520104781 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f445014b1f0 con 0x7f4450072410 2024-04-20T21:18:16.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.412+0000 7f4447e00640 1 --2- 172.21.15.86:0/520104781 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44500ccdd0 0x7f44501f6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:16.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.413+0000 7f444ea00640 1 --2- 172.21.15.86:0/520104781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4450072410 0x7f4450148a10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f444000e870 tx=0x7f444000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:16.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.413+0000 7f4446a00640 1 -- 172.21.15.86:0/520104781 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4440017070 con 0x7f4450072410 2024-04-20T21:18:16.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.413+0000 7f4446a00640 1 -- 172.21.15.86:0/520104781 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f444000c940 con 0x7f4450072410 2024-04-20T21:18:16.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.413+0000 7f4455800640 1 -- 172.21.15.86:0/520104781 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f445014b4d0 con 0x7f4450072410 2024-04-20T21:18:16.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.413+0000 7f4446a00640 1 -- 172.21.15.86:0/520104781 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f444000cd30 con 0x7f4450072410 2024-04-20T21:18:16.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.413+0000 7f4455800640 1 -- 172.21.15.86:0/520104781 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f445013f8d0 con 0x7f4450072410 2024-04-20T21:18:16.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.415+0000 7f4455800640 1 -- 172.21.15.86:0/520104781 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f445014b9f0 con 0x7f4450072410 2024-04-20T21:18:16.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.415+0000 7f4446a00640 1 -- 172.21.15.86:0/520104781 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f444001a070 con 0x7f4450072410 2024-04-20T21:18:16.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.417+0000 7f4446a00640 1 --2- 172.21.15.86:0/520104781 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f442c06c8f0 0x7f442c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:16.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.417+0000 7f4446a00640 1 -- 172.21.15.86:0/520104781 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f444008bac0 con 0x7f4450072410 2024-04-20T21:18:16.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.417+0000 7f4447e00640 1 --2- 172.21.15.86:0/520104781 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f442c06c8f0 0x7f442c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:16.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.418+0000 7f4447e00640 1 --2- 172.21.15.86:0/520104781 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f442c06c8f0 0x7f442c06edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f445015bac0 tx=0x7f4438009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:16.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.423+0000 7f4446a00640 1 -- 172.21.15.86:0/520104781 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f445014b9f0 con 0x7f4450072410 2024-04-20T21:18:16.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.665+0000 7f4455800640 1 -- 172.21.15.86:0/520104781 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f44500689e0 con 0x7f4450072410 2024-04-20T21:18:16.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.666+0000 7f4446a00640 1 -- 172.21.15.86:0/520104781 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f4440059e50 con 0x7f4450072410 2024-04-20T21:18:16.665 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T21:18:16.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.668+0000 7f4455800640 1 -- 172.21.15.86:0/520104781 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f442c06c8f0 msgr2=0x7f442c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:16.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.668+0000 7f4455800640 1 --2- 172.21.15.86:0/520104781 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f442c06c8f0 0x7f442c06edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f445015bac0 tx=0x7f4438009040 comp rx=0 tx=0).stop 2024-04-20T21:18:16.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.668+0000 7f4455800640 1 -- 172.21.15.86:0/520104781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4450072410 msgr2=0x7f4450148a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:16.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.668+0000 7f4455800640 1 --2- 172.21.15.86:0/520104781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4450072410 0x7f4450148a10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f444000e870 tx=0x7f444000ed30 comp rx=0 tx=0).stop 2024-04-20T21:18:16.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.669+0000 7f4455800640 1 -- 172.21.15.86:0/520104781 shutdown_connections 2024-04-20T21:18:16.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.669+0000 7f4455800640 1 --2- 172.21.15.86:0/520104781 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f442c06c8f0 0x7f442c06edb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:16.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.669+0000 7f4455800640 1 --2- 172.21.15.86:0/520104781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4450146130 0x7f445014aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:16.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.669+0000 7f4455800640 1 --2- 172.21.15.86:0/520104781 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44500ccdd0 0x7f44501f6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:16.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.669+0000 7f4455800640 1 --2- 172.21.15.86:0/520104781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4450072410 0x7f4450148a10 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:16.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.669+0000 7f4455800640 1 -- 172.21.15.86:0/520104781 >> 172.21.15.86:0/520104781 conn(0x7f44500b2360 msgr2=0x7f4450071d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:16.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.669+0000 7f4455800640 1 -- 172.21.15.86:0/520104781 shutdown_connections 2024-04-20T21:18:16.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:16.669+0000 7f4455800640 1 -- 172.21.15.86:0/520104781 wait complete. 2024-04-20T21:18:16.689 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T21:18:16.689 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T21:18:16.689 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi144.front.sepia.ceph.com 2024-04-20T21:18:16.689 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-20T21:18:16.689 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-20T21:18:16.689 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T21:18:16.689 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T21:18:16.689 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T21:18:16.689 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-20T21:18:16.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:16.690 DEBUG:teuthology.orchestra.run.smithi144:> ip addr 2024-04-20T21:18:16.744 INFO:teuthology.orchestra.run.smithi144.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T21:18:16.744 INFO:teuthology.orchestra.run.smithi144.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T21:18:16.744 INFO:teuthology.orchestra.run.smithi144.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T21:18:16.744 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:18:16.744 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 ::1/128 scope host 2024-04-20T21:18:16.744 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:18:16.744 INFO:teuthology.orchestra.run.smithi144.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:18:16.744 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:22 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:18:16.744 INFO:teuthology.orchestra.run.smithi144.stdout: altname enp7s0f0 2024-04-20T21:18:16.745 INFO:teuthology.orchestra.run.smithi144.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:18:16.745 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:23 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:18:16.745 INFO:teuthology.orchestra.run.smithi144.stdout: altname enp7s0f1 2024-04-20T21:18:16.745 INFO:teuthology.orchestra.run.smithi144.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:18:16.745 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5e brd ff:ff:ff:ff:ff:ff 2024-04-20T21:18:16.745 INFO:teuthology.orchestra.run.smithi144.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T21:18:16.745 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5f brd ff:ff:ff:ff:ff:ff 2024-04-20T21:18:16.745 INFO:teuthology.orchestra.run.smithi144.stdout: inet 172.21.15.144/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T21:18:16.745 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft 42756sec preferred_lft 42756sec 2024-04-20T21:18:16.745 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::ec4:7aff:fe88:7f5f/64 scope link 2024-04-20T21:18:16.745 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:18:16.746 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-20T21:18:16.746 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:18:16.747 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2024-04-20T21:18:16.747 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link add name ceph-brx type bridge 2024-04-20T21:18:16.747 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip addr flush dev ceph-brx 2024-04-20T21:18:16.747 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link set ceph-brx up 2024-04-20T21:18:16.747 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-20T21:18:16.747 DEBUG:teuthology.orchestra.run.smithi144:> ') 2024-04-20T21:18:16.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:18:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:18:17.195 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:18:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:18:17.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:17.204 DEBUG:teuthology.orchestra.run.smithi144:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T21:18:17.265 INFO:teuthology.orchestra.run.smithi144.stdout:1 2024-04-20T21:18:17.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:17.269 DEBUG:teuthology.orchestra.run.smithi144:> route 2024-04-20T21:18:17.320 INFO:teuthology.orchestra.run.smithi144.stdout:Kernel IP routing table 2024-04-20T21:18:17.320 INFO:teuthology.orchestra.run.smithi144.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T21:18:17.320 INFO:teuthology.orchestra.run.smithi144.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T21:18:17.320 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T21:18:17.320 INFO:teuthology.orchestra.run.smithi144.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T21:18:17.320 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:18:17.320 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2024-04-20T21:18:17.320 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T21:18:17.321 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T21:18:17.321 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T21:18:17.321 DEBUG:teuthology.orchestra.run.smithi144:> ') 2024-04-20T21:18:17.408 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:18:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:18:17.983 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:18:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:18:17.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:17.992 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list 2024-04-20T21:18:18.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:18.037 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list-id 2024-04-20T21:18:18.125 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:18:18.125 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2024-04-20T21:18:18.125 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T21:18:18.126 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-20T21:18:18.126 DEBUG:teuthology.orchestra.run.smithi144:> ') 2024-04-20T21:18:18.212 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:18:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:18:18.270 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:18:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:18:18.279 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-20T21:18:18.279 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:18:18.279 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2024-04-20T21:18:18.279 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-20T21:18:18.279 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-20T21:18:18.279 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-20T21:18:18.279 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-20T21:18:18.279 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-20T21:18:18.279 DEBUG:teuthology.orchestra.run.smithi144:> ') 2024-04-20T21:18:18.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:18:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:18:18.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:18:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:18:18.582 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:18:18.582 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2024-04-20T21:18:18.582 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link set brx.0 up 2024-04-20T21:18:18.582 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link set dev brx.0 master ceph-brx 2024-04-20T21:18:18.582 DEBUG:teuthology.orchestra.run.smithi144:> ') 2024-04-20T21:18:18.678 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:18:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:18:18.775 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:18:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:18:18.784 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-20T21:18:18.784 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T21:18:18.784 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-20T21:18:18.827 INFO:teuthology.orchestra.run.smithi144.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-20T21:18:18.827 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T21:18:18.827 DEBUG:teuthology.orchestra.run.smithi144:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-20T21:18:18.922 DEBUG:teuthology.orchestra.run.smithi144:> sudo modprobe fuse 2024-04-20T21:18:18.999 DEBUG:teuthology.orchestra.run.smithi144:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T21:18:19.048 INFO:teuthology.orchestra.run.smithi144.stdout:/proc 2024-04-20T21:18:19.048 INFO:teuthology.orchestra.run.smithi144.stdout:/sys 2024-04-20T21:18:19.049 INFO:teuthology.orchestra.run.smithi144.stdout:/dev 2024-04-20T21:18:19.049 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/security 2024-04-20T21:18:19.049 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/shm 2024-04-20T21:18:19.049 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/pts 2024-04-20T21:18:19.049 INFO:teuthology.orchestra.run.smithi144.stdout:/run 2024-04-20T21:18:19.049 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/cgroup 2024-04-20T21:18:19.049 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/pstore 2024-04-20T21:18:19.049 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/bpf 2024-04-20T21:18:19.049 INFO:teuthology.orchestra.run.smithi144.stdout:/ 2024-04-20T21:18:19.049 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/selinux 2024-04-20T21:18:19.049 INFO:teuthology.orchestra.run.smithi144.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T21:18:19.049 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/hugepages 2024-04-20T21:18:19.050 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/mqueue 2024-04-20T21:18:19.050 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/debug 2024-04-20T21:18:19.050 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/tracing 2024-04-20T21:18:19.050 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/fuse/connections 2024-04-20T21:18:19.050 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/config 2024-04-20T21:18:19.050 INFO:teuthology.orchestra.run.smithi144.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T21:18:19.050 INFO:teuthology.orchestra.run.smithi144.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T21:18:19.050 INFO:teuthology.orchestra.run.smithi144.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T21:18:19.050 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T21:18:19.050 INFO:teuthology.orchestra.run.smithi144.stdout:/run/user/1000 2024-04-20T21:18:19.050 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph 2024-04-20T21:18:19.050 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph/osd/ceph-4 2024-04-20T21:18:19.050 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph/osd/ceph-5 2024-04-20T21:18:19.050 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph/osd/ceph-6 2024-04-20T21:18:19.050 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph/osd/ceph-7 2024-04-20T21:18:19.051 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns 2024-04-20T21:18:19.051 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T21:18:19.051 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T21:18:19.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:19.052 DEBUG:teuthology.orchestra.run.smithi144:> ls /sys/fs/fuse/connections 2024-04-20T21:18:19.099 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T21:18:19.099 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-20T21:18:19.104 DEBUG:teuthology.orchestra.run.smithi144:> sudo modprobe fuse 2024-04-20T21:18:19.170 DEBUG:teuthology.orchestra.run.smithi144:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T21:18:19.213 INFO:teuthology.orchestra.run.smithi144.stdout:/proc 2024-04-20T21:18:19.213 INFO:teuthology.orchestra.run.smithi144.stdout:/sys 2024-04-20T21:18:19.213 INFO:teuthology.orchestra.run.smithi144.stdout:/dev 2024-04-20T21:18:19.213 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/security 2024-04-20T21:18:19.213 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/shm 2024-04-20T21:18:19.213 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/pts 2024-04-20T21:18:19.214 INFO:teuthology.orchestra.run.smithi144.stdout:/run 2024-04-20T21:18:19.214 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/cgroup 2024-04-20T21:18:19.214 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/pstore 2024-04-20T21:18:19.214 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/bpf 2024-04-20T21:18:19.214 INFO:teuthology.orchestra.run.smithi144.stdout:/ 2024-04-20T21:18:19.214 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/selinux 2024-04-20T21:18:19.214 INFO:teuthology.orchestra.run.smithi144.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T21:18:19.214 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/hugepages 2024-04-20T21:18:19.214 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/mqueue 2024-04-20T21:18:19.214 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/debug 2024-04-20T21:18:19.214 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/tracing 2024-04-20T21:18:19.215 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/fuse/connections 2024-04-20T21:18:19.215 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/config 2024-04-20T21:18:19.215 INFO:teuthology.orchestra.run.smithi144.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T21:18:19.215 INFO:teuthology.orchestra.run.smithi144.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T21:18:19.215 INFO:teuthology.orchestra.run.smithi144.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T21:18:19.215 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T21:18:19.215 INFO:teuthology.orchestra.run.smithi144.stdout:/run/user/1000 2024-04-20T21:18:19.215 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph 2024-04-20T21:18:19.215 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph/osd/ceph-4 2024-04-20T21:18:19.215 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph/osd/ceph-5 2024-04-20T21:18:19.215 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph/osd/ceph-6 2024-04-20T21:18:19.215 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph/osd/ceph-7 2024-04-20T21:18:19.215 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns 2024-04-20T21:18:19.215 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T21:18:19.216 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T21:18:19.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:19.217 DEBUG:teuthology.orchestra.run.smithi144:> ls /sys/fs/fuse/connections 2024-04-20T21:18:19.294 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi144.stderr:2024-04-20T21:18:19.295+0000 7f172de44480 -1 init, newargv = 0x5599b8002280 newargc=15 2024-04-20T21:18:19.295 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi144.stderr:ceph-fuse[28410]: starting ceph client 2024-04-20T21:18:19.306 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi144.stderr:ceph-fuse[28410]: starting fuse 2024-04-20T21:18:20.402 DEBUG:teuthology.orchestra.run.smithi144:> sudo modprobe fuse 2024-04-20T21:18:20.492 DEBUG:teuthology.orchestra.run.smithi144:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T21:18:20.548 INFO:teuthology.orchestra.run.smithi144.stdout:/proc 2024-04-20T21:18:20.548 INFO:teuthology.orchestra.run.smithi144.stdout:/sys 2024-04-20T21:18:20.548 INFO:teuthology.orchestra.run.smithi144.stdout:/dev 2024-04-20T21:18:20.548 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/security 2024-04-20T21:18:20.548 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/shm 2024-04-20T21:18:20.549 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/pts 2024-04-20T21:18:20.549 INFO:teuthology.orchestra.run.smithi144.stdout:/run 2024-04-20T21:18:20.549 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/cgroup 2024-04-20T21:18:20.549 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/pstore 2024-04-20T21:18:20.549 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/bpf 2024-04-20T21:18:20.549 INFO:teuthology.orchestra.run.smithi144.stdout:/ 2024-04-20T21:18:20.549 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/selinux 2024-04-20T21:18:20.550 INFO:teuthology.orchestra.run.smithi144.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T21:18:20.550 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/hugepages 2024-04-20T21:18:20.550 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/mqueue 2024-04-20T21:18:20.550 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/debug 2024-04-20T21:18:20.550 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/tracing 2024-04-20T21:18:20.550 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/fuse/connections 2024-04-20T21:18:20.550 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/config 2024-04-20T21:18:20.551 INFO:teuthology.orchestra.run.smithi144.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T21:18:20.551 INFO:teuthology.orchestra.run.smithi144.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T21:18:20.551 INFO:teuthology.orchestra.run.smithi144.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T21:18:20.551 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T21:18:20.551 INFO:teuthology.orchestra.run.smithi144.stdout:/run/user/1000 2024-04-20T21:18:20.551 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph 2024-04-20T21:18:20.552 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph/osd/ceph-4 2024-04-20T21:18:20.552 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph/osd/ceph-5 2024-04-20T21:18:20.552 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph/osd/ceph-6 2024-04-20T21:18:20.552 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph/osd/ceph-7 2024-04-20T21:18:20.552 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns 2024-04-20T21:18:20.552 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T21:18:20.552 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T21:18:20.553 INFO:teuthology.orchestra.run.smithi144.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-20T21:18:20.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:20.554 DEBUG:teuthology.orchestra.run.smithi144:> ls /sys/fs/fuse/connections 2024-04-20T21:18:20.603 INFO:teuthology.orchestra.run.smithi144.stdout:40 2024-04-20T21:18:20.604 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2024-04-20T21:18:20.604 DEBUG:teuthology.orchestra.run.smithi144:> sudo stdin-killer -- python3 -c ' 2024-04-20T21:18:20.604 DEBUG:teuthology.orchestra.run.smithi144:> import glob 2024-04-20T21:18:20.604 DEBUG:teuthology.orchestra.run.smithi144:> import re 2024-04-20T21:18:20.604 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-04-20T21:18:20.605 DEBUG:teuthology.orchestra.run.smithi144:> import subprocess 2024-04-20T21:18:20.605 DEBUG:teuthology.orchestra.run.smithi144:> 2024-04-20T21:18:20.605 DEBUG:teuthology.orchestra.run.smithi144:> def _find_admin_socket(client_name): 2024-04-20T21:18:20.605 DEBUG:teuthology.orchestra.run.smithi144:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-20T21:18:20.605 DEBUG:teuthology.orchestra.run.smithi144:> files = glob.glob(asok_path) 2024-04-20T21:18:20.605 DEBUG:teuthology.orchestra.run.smithi144:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-20T21:18:20.606 DEBUG:teuthology.orchestra.run.smithi144:> 2024-04-20T21:18:20.606 DEBUG:teuthology.orchestra.run.smithi144:> # Given a non-glob path, it better be there 2024-04-20T21:18:20.606 DEBUG:teuthology.orchestra.run.smithi144:> if "*" not in asok_path: 2024-04-20T21:18:20.606 DEBUG:teuthology.orchestra.run.smithi144:> assert(len(files) == 1) 2024-04-20T21:18:20.606 DEBUG:teuthology.orchestra.run.smithi144:> return files[0] 2024-04-20T21:18:20.606 DEBUG:teuthology.orchestra.run.smithi144:> 2024-04-20T21:18:20.607 DEBUG:teuthology.orchestra.run.smithi144:> for f in files: 2024-04-20T21:18:20.607 DEBUG:teuthology.orchestra.run.smithi144:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T21:18:20.607 DEBUG:teuthology.orchestra.run.smithi144:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T21:18:20.607 DEBUG:teuthology.orchestra.run.smithi144:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T21:18:20.607 DEBUG:teuthology.orchestra.run.smithi144:> contents = proc_f.read() 2024-04-20T21:18:20.607 DEBUG:teuthology.orchestra.run.smithi144:> if mountpoint in contents: 2024-04-20T21:18:20.607 DEBUG:teuthology.orchestra.run.smithi144:> return f 2024-04-20T21:18:20.608 DEBUG:teuthology.orchestra.run.smithi144:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T21:18:20.608 DEBUG:teuthology.orchestra.run.smithi144:> 2024-04-20T21:18:20.608 DEBUG:teuthology.orchestra.run.smithi144:> print(_find_admin_socket("client.1")) 2024-04-20T21:18:20.608 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-04-20T21:18:20.770 INFO:teuthology.orchestra.run.smithi144.stdout:/var/run/ceph/ceph-client.1.28410.asok 2024-04-20T21:18:20.774 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:18:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:18:20.787 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.28410.asok 2024-04-20T21:18:20.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:20.787 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.28410.asok status 2024-04-20T21:18:21.012 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-04-20T21:18:21.012 INFO:teuthology.orchestra.run.smithi144.stdout: "metadata": { 2024-04-20T21:18:21.012 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T21:18:21.012 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T21:18:21.012 INFO:teuthology.orchestra.run.smithi144.stdout: "entity_id": "1", 2024-04-20T21:18:21.012 INFO:teuthology.orchestra.run.smithi144.stdout: "hostname": "smithi144", 2024-04-20T21:18:21.012 INFO:teuthology.orchestra.run.smithi144.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-20T21:18:21.012 INFO:teuthology.orchestra.run.smithi144.stdout: "pid": "28410", 2024-04-20T21:18:21.012 INFO:teuthology.orchestra.run.smithi144.stdout: "root": "/" 2024-04-20T21:18:21.012 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-04-20T21:18:21.013 INFO:teuthology.orchestra.run.smithi144.stdout: "dentry_count": 0, 2024-04-20T21:18:21.013 INFO:teuthology.orchestra.run.smithi144.stdout: "dentry_pinned_count": 0, 2024-04-20T21:18:21.013 INFO:teuthology.orchestra.run.smithi144.stdout: "id": 4424, 2024-04-20T21:18:21.013 INFO:teuthology.orchestra.run.smithi144.stdout: "inst": { 2024-04-20T21:18:21.013 INFO:teuthology.orchestra.run.smithi144.stdout: "name": { 2024-04-20T21:18:21.013 INFO:teuthology.orchestra.run.smithi144.stdout: "type": "client", 2024-04-20T21:18:21.013 INFO:teuthology.orchestra.run.smithi144.stdout: "num": 4424 2024-04-20T21:18:21.013 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-04-20T21:18:21.013 INFO:teuthology.orchestra.run.smithi144.stdout: "addr": { 2024-04-20T21:18:21.013 INFO:teuthology.orchestra.run.smithi144.stdout: "type": "v1", 2024-04-20T21:18:21.013 INFO:teuthology.orchestra.run.smithi144.stdout: "addr": "192.168.0.1:0", 2024-04-20T21:18:21.013 INFO:teuthology.orchestra.run.smithi144.stdout: "nonce": 3330228370 2024-04-20T21:18:21.013 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-04-20T21:18:21.013 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-04-20T21:18:21.014 INFO:teuthology.orchestra.run.smithi144.stdout: "addr": { 2024-04-20T21:18:21.014 INFO:teuthology.orchestra.run.smithi144.stdout: "type": "v1", 2024-04-20T21:18:21.014 INFO:teuthology.orchestra.run.smithi144.stdout: "addr": "192.168.0.1:0", 2024-04-20T21:18:21.014 INFO:teuthology.orchestra.run.smithi144.stdout: "nonce": 3330228370 2024-04-20T21:18:21.014 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-04-20T21:18:21.014 INFO:teuthology.orchestra.run.smithi144.stdout: "inst_str": "client.4424 192.168.0.1:0/3330228370", 2024-04-20T21:18:21.014 INFO:teuthology.orchestra.run.smithi144.stdout: "addr_str": "192.168.0.1:0/3330228370", 2024-04-20T21:18:21.014 INFO:teuthology.orchestra.run.smithi144.stdout: "inode_count": 1, 2024-04-20T21:18:21.014 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_epoch": 7, 2024-04-20T21:18:21.014 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_epoch": 20, 2024-04-20T21:18:21.014 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_epoch_barrier": 0, 2024-04-20T21:18:21.014 INFO:teuthology.orchestra.run.smithi144.stdout: "blocklisted": false, 2024-04-20T21:18:21.014 INFO:teuthology.orchestra.run.smithi144.stdout: "fs_name": "cephfs" 2024-04-20T21:18:21.014 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-04-20T21:18:21.026 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T21:18:21.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.234+0000 7f85ace00640 1 Processor -- start 2024-04-20T21:18:21.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.234+0000 7f85ace00640 1 -- start start 2024-04-20T21:18:21.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.235+0000 7f85ace00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f85a815a820 0x7f85a80b3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:21.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.235+0000 7f85ace00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85a8073670 0x7f85a815e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:21.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.235+0000 7f85ace00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85a806f5c0 con 0x7f85a8073b40 2024-04-20T21:18:21.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.235+0000 7f85ace00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f85a806ee30 con 0x7f85a815a820 2024-04-20T21:18:21.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.235+0000 7f85ace00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f85a815ac00 con 0x7f85a8073670 2024-04-20T21:18:21.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.236+0000 7f85a7400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f85a815a820 0x7f85a80b3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:21.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.236+0000 7f85a7400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f85a815a820 0x7f85a80b3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44882/0 (socket says 172.21.15.86:44882) 2024-04-20T21:18:21.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.236+0000 7f85a7400640 1 -- 172.21.15.86:0/2151413891 learned_addr learned my addr 172.21.15.86:0/2151413891 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:21.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.236+0000 7f859ea00640 1 --2- 172.21.15.86:0/2151413891 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85a8073670 0x7f85a815e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:21.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.236+0000 7f85a7400640 1 -- 172.21.15.86:0/2151413891 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85a8073670 msgr2=0x7f85a815e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:21.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.236+0000 7f85a7400640 1 --2- 172.21.15.86:0/2151413891 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85a8073670 0x7f85a815e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:21.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.236+0000 7f85a7400640 1 -- 172.21.15.86:0/2151413891 >> v1:172.21.15.144:6789/0 conn(0x7f85a8073b40 legacy=0x7f85a815e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:21.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.236+0000 7f85a7400640 1 -- 172.21.15.86:0/2151413891 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85a806f970 con 0x7f85a815a820 2024-04-20T21:18:21.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.237+0000 7f85a7400640 1 --2- 172.21.15.86:0/2151413891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85a815a820 0x7f85a80b3100 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8598009870 tx=0x7f859802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28eb653248d483b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:21.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.238+0000 7f85a6a00640 1 -- 172.21.15.86:0/2151413891 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8598035020 con 0x7f85a815a820 2024-04-20T21:18:21.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.238+0000 7f85a6a00640 1 -- 172.21.15.86:0/2151413891 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8598031640 con 0x7f85a815a820 2024-04-20T21:18:21.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.238+0000 7f85a6a00640 1 -- 172.21.15.86:0/2151413891 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859802d040 con 0x7f85a815a820 2024-04-20T21:18:21.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.238+0000 7f85ace00640 1 -- 172.21.15.86:0/2151413891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85a815a820 msgr2=0x7f85a80b3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:21.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.238+0000 7f85ace00640 1 --2- 172.21.15.86:0/2151413891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85a815a820 0x7f85a80b3100 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8598009870 tx=0x7f859802b8f0 comp rx=0 tx=0).stop 2024-04-20T21:18:21.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.238+0000 7f85ace00640 1 -- 172.21.15.86:0/2151413891 shutdown_connections 2024-04-20T21:18:21.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.239+0000 7f85ace00640 1 --2- 172.21.15.86:0/2151413891 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85a8073670 0x7f85a815e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:21.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.239+0000 7f85ace00640 1 --2- 172.21.15.86:0/2151413891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85a815a820 0x7f85a80b3100 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:21.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.239+0000 7f85ace00640 1 -- 172.21.15.86:0/2151413891 >> 172.21.15.86:0/2151413891 conn(0x7f85a80b5840 msgr2=0x7f85a8072f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:21.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.239+0000 7f85ace00640 1 -- 172.21.15.86:0/2151413891 shutdown_connections 2024-04-20T21:18:21.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.239+0000 7f85ace00640 1 -- 172.21.15.86:0/2151413891 wait complete. 2024-04-20T21:18:21.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.240+0000 7f85ace00640 1 Processor -- start 2024-04-20T21:18:21.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.241+0000 7f85ace00640 1 -- start start 2024-04-20T21:18:21.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.241+0000 7f85ace00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85a8073670 0x7f85a80b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:21.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.241+0000 7f85ace00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85a8073b40 0x7f85a814faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:21.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.241+0000 7f85ace00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85a815a820 0x7f85a8146020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:21.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.242+0000 7f85ace00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f85a815ad80 con 0x7f85a8073670 2024-04-20T21:18:21.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.242+0000 7f85ace00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f85a8072300 con 0x7f85a8073b40 2024-04-20T21:18:21.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.242+0000 7f85ace00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f85a80ad2c0 con 0x7f85a815a820 2024-04-20T21:18:21.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.242+0000 7f85a7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85a8073670 0x7f85a80b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:21.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.242+0000 7f85a7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85a8073670 0x7f85a80b32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44894/0 (socket says 172.21.15.86:44894) 2024-04-20T21:18:21.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.242+0000 7f85a7400640 1 -- 172.21.15.86:0/36741585 learned_addr learned my addr 172.21.15.86:0/36741585 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:21.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.242+0000 7f859ea00640 1 --2- 172.21.15.86:0/36741585 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85a8073b40 0x7f85a814faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:21.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.242+0000 7f85a7400640 1 -- 172.21.15.86:0/36741585 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85a815a820 msgr2=0x7f85a8146020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:21.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.242+0000 7f85a7400640 1 --2- 172.21.15.86:0/36741585 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85a815a820 0x7f85a8146020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:21.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.242+0000 7f85a7400640 1 -- 172.21.15.86:0/36741585 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85a8073b40 msgr2=0x7f85a814faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:21.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.242+0000 7f85a7400640 1 --2- 172.21.15.86:0/36741585 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85a8073b40 0x7f85a814faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:21.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.242+0000 7f85a7400640 1 -- 172.21.15.86:0/36741585 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85a80b3810 con 0x7f85a8073670 2024-04-20T21:18:21.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.242+0000 7f859ea00640 1 --2- 172.21.15.86:0/36741585 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85a8073b40 0x7f85a814faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:21.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.243+0000 7f85a7400640 1 --2- 172.21.15.86:0/36741585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85a8073670 0x7f85a80b32d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f859802b930 tx=0x7f85980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:21.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.243+0000 7f859fe00640 1 -- 172.21.15.86:0/36741585 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8598040070 con 0x7f85a8073670 2024-04-20T21:18:21.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.243+0000 7f859fe00640 1 -- 172.21.15.86:0/36741585 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85980399f0 con 0x7f85a8073670 2024-04-20T21:18:21.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.243+0000 7f85ace00640 1 -- 172.21.15.86:0/36741585 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85a81415d0 con 0x7f85a8073670 2024-04-20T21:18:21.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.243+0000 7f859fe00640 1 -- 172.21.15.86:0/36741585 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8598035040 con 0x7f85a8073670 2024-04-20T21:18:21.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.243+0000 7f85ace00640 1 -- 172.21.15.86:0/36741585 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85a8141b90 con 0x7f85a8073670 2024-04-20T21:18:21.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.244+0000 7f85ace00640 1 -- 172.21.15.86:0/36741585 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85a804fcf0 con 0x7f85a8073670 2024-04-20T21:18:21.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.245+0000 7f859fe00640 1 -- 172.21.15.86:0/36741585 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f859804b060 con 0x7f85a8073670 2024-04-20T21:18:21.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.246+0000 7f859fe00640 1 --2- 172.21.15.86:0/36741585 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f858406c8f0 0x7f858406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:21.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.246+0000 7f859fe00640 1 -- 172.21.15.86:0/36741585 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f85980ad440 con 0x7f85a8073670 2024-04-20T21:18:21.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.246+0000 7f859ea00640 1 --2- 172.21.15.86:0/36741585 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f858406c8f0 0x7f858406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:21.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.247+0000 7f859ea00640 1 --2- 172.21.15.86:0/36741585 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f858406c8f0 0x7f858406edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f85a8073640 tx=0x7f859000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:21.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.251+0000 7f859fe00640 1 -- 172.21.15.86:0/36741585 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f859807b860 con 0x7f85a8073670 2024-04-20T21:18:21.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.490+0000 7f85ace00640 1 -- 172.21.15.86:0/36741585 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f85a80b5190 con 0x7f85a8073670 2024-04-20T21:18:21.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.491+0000 7f859fe00640 1 -- 172.21.15.86:0/36741585 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f859807b160 con 0x7f85a8073670 2024-04-20T21:18:21.490 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T21:18:21.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.494+0000 7f85ace00640 1 -- 172.21.15.86:0/36741585 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f858406c8f0 msgr2=0x7f858406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:21.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.494+0000 7f85ace00640 1 --2- 172.21.15.86:0/36741585 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f858406c8f0 0x7f858406edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f85a8073640 tx=0x7f859000b040 comp rx=0 tx=0).stop 2024-04-20T21:18:21.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.494+0000 7f85ace00640 1 -- 172.21.15.86:0/36741585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85a8073670 msgr2=0x7f85a80b32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:21.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.494+0000 7f85ace00640 1 --2- 172.21.15.86:0/36741585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85a8073670 0x7f85a80b32d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f859802b930 tx=0x7f85980046c0 comp rx=0 tx=0).stop 2024-04-20T21:18:21.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.494+0000 7f85ace00640 1 -- 172.21.15.86:0/36741585 shutdown_connections 2024-04-20T21:18:21.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.494+0000 7f85ace00640 1 --2- 172.21.15.86:0/36741585 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f858406c8f0 0x7f858406edb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:21.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.494+0000 7f85ace00640 1 --2- 172.21.15.86:0/36741585 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85a815a820 0x7f85a8146020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:21.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.494+0000 7f85ace00640 1 --2- 172.21.15.86:0/36741585 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85a8073b40 0x7f85a814faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:21.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.494+0000 7f85ace00640 1 --2- 172.21.15.86:0/36741585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85a8073670 0x7f85a80b32d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:21.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.494+0000 7f85ace00640 1 -- 172.21.15.86:0/36741585 >> 172.21.15.86:0/36741585 conn(0x7f85a80b5840 msgr2=0x7f85a8150ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:21.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.494+0000 7f85ace00640 1 -- 172.21.15.86:0/36741585 shutdown_connections 2024-04-20T21:18:21.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:21.494+0000 7f85ace00640 1 -- 172.21.15.86:0/36741585 wait complete. 2024-04-20T21:18:21.513 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T21:18:21.513 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T21:18:21.513 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi144.front.sepia.ceph.com 2024-04-20T21:18:21.513 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-20T21:18:21.513 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-20T21:18:21.513 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T21:18:21.513 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T21:18:21.513 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T21:18:21.513 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-20T21:18:21.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:21.514 DEBUG:teuthology.orchestra.run.smithi144:> ip addr 2024-04-20T21:18:21.572 INFO:teuthology.orchestra.run.smithi144.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T21:18:21.572 INFO:teuthology.orchestra.run.smithi144.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T21:18:21.572 INFO:teuthology.orchestra.run.smithi144.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T21:18:21.572 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:18:21.572 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 ::1/128 scope host 2024-04-20T21:18:21.572 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:18:21.572 INFO:teuthology.orchestra.run.smithi144.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:18:21.572 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:22 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:18:21.572 INFO:teuthology.orchestra.run.smithi144.stdout: altname enp7s0f0 2024-04-20T21:18:21.572 INFO:teuthology.orchestra.run.smithi144.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:18:21.573 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:23 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:18:21.573 INFO:teuthology.orchestra.run.smithi144.stdout: altname enp7s0f1 2024-04-20T21:18:21.573 INFO:teuthology.orchestra.run.smithi144.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:18:21.573 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5e brd ff:ff:ff:ff:ff:ff 2024-04-20T21:18:21.573 INFO:teuthology.orchestra.run.smithi144.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T21:18:21.573 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5f brd ff:ff:ff:ff:ff:ff 2024-04-20T21:18:21.573 INFO:teuthology.orchestra.run.smithi144.stdout: inet 172.21.15.144/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T21:18:21.573 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft 42751sec preferred_lft 42751sec 2024-04-20T21:18:21.573 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::ec4:7aff:fe88:7f5f/64 scope link 2024-04-20T21:18:21.573 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:18:21.573 INFO:teuthology.orchestra.run.smithi144.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T21:18:21.573 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 86:2d:29:6b:82:c6 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:18:21.573 INFO:teuthology.orchestra.run.smithi144.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-20T21:18:21.573 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:18:21.573 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::44d3:26ff:fe69:b044/64 scope link 2024-04-20T21:18:21.574 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:18:21.574 INFO:teuthology.orchestra.run.smithi144.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T21:18:21.574 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 86:2d:29:6b:82:c6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T21:18:21.574 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::842d:29ff:fe6b:82c6/64 scope link 2024-04-20T21:18:21.574 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:18:21.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:21.575 DEBUG:teuthology.orchestra.run.smithi144:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T21:18:21.655 INFO:teuthology.orchestra.run.smithi144.stdout:1 2024-04-20T21:18:21.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:21.660 DEBUG:teuthology.orchestra.run.smithi144:> route 2024-04-20T21:18:21.712 INFO:teuthology.orchestra.run.smithi144.stdout:Kernel IP routing table 2024-04-20T21:18:21.712 INFO:teuthology.orchestra.run.smithi144.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T21:18:21.712 INFO:teuthology.orchestra.run.smithi144.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T21:18:21.712 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T21:18:21.712 INFO:teuthology.orchestra.run.smithi144.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T21:18:21.713 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:18:21.713 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2024-04-20T21:18:21.713 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T21:18:21.713 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T21:18:21.713 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T21:18:21.713 DEBUG:teuthology.orchestra.run.smithi144:> ') 2024-04-20T21:18:21.819 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:18:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:18:21.912 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:18:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:18:21.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:21.922 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list 2024-04-20T21:18:21.968 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-20T21:18:21.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:21.969 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list-id 2024-04-20T21:18:22.018 INFO:teuthology.orchestra.run.smithi144.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-20T21:18:22.019 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:18:22.019 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2024-04-20T21:18:22.019 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T21:18:22.019 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-20T21:18:22.019 DEBUG:teuthology.orchestra.run.smithi144:> ') 2024-04-20T21:18:22.125 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:18:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:18:22.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:18:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:18:22.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:22.203 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-20T21:18:22.274 INFO:teuthology.orchestra.run.smithi144.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T21:18:22.274 INFO:teuthology.orchestra.run.smithi144.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T21:18:22.275 INFO:teuthology.orchestra.run.smithi144.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T21:18:22.275 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:18:22.275 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 ::1/128 scope host 2024-04-20T21:18:22.275 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:18:22.275 INFO:teuthology.orchestra.run.smithi144.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T21:18:22.275 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether de:79:a9:d2:4d:a6 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-20T21:18:22.275 INFO:teuthology.orchestra.run.smithi144.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-20T21:18:22.275 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:18:22.275 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::dc79:a9ff:fed2:4da6/64 scope link 2024-04-20T21:18:22.275 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:18:22.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:22.278 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-20T21:18:22.358 INFO:teuthology.orchestra.run.smithi144.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T21:18:22.358 INFO:teuthology.orchestra.run.smithi144.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T21:18:22.358 INFO:teuthology.orchestra.run.smithi144.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T21:18:22.358 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:18:22.358 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 ::1/128 scope host 2024-04-20T21:18:22.358 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:18:22.358 INFO:teuthology.orchestra.run.smithi144.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T21:18:22.358 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether de:79:a9:d2:4d:a6 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-20T21:18:22.358 INFO:teuthology.orchestra.run.smithi144.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-20T21:18:22.358 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:18:22.358 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::dc79:a9ff:fed2:4da6/64 scope link 2024-04-20T21:18:22.358 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:18:22.362 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-20T21:18:22.363 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:18:22.363 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2024-04-20T21:18:22.363 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-20T21:18:22.363 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-20T21:18:22.363 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-20T21:18:22.363 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-20T21:18:22.363 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-20T21:18:22.363 DEBUG:teuthology.orchestra.run.smithi144:> ') 2024-04-20T21:18:22.468 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:18:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:18:22.623 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:18:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:18:22.633 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:18:22.633 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2024-04-20T21:18:22.633 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link set brx.1 up 2024-04-20T21:18:22.633 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link set dev brx.1 master ceph-brx 2024-04-20T21:18:22.633 DEBUG:teuthology.orchestra.run.smithi144:> ') 2024-04-20T21:18:22.724 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:18:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:18:22.821 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:18:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:18:22.830 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-04-20T21:18:22.830 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T21:18:22.830 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-20T21:18:22.875 INFO:teuthology.orchestra.run.smithi144.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-20T21:18:22.875 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T21:18:22.875 DEBUG:teuthology.orchestra.run.smithi144:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-20T21:18:22.968 DEBUG:teuthology.orchestra.run.smithi144:> sudo modprobe fuse 2024-04-20T21:18:23.046 DEBUG:teuthology.orchestra.run.smithi144:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T21:18:23.094 INFO:teuthology.orchestra.run.smithi144.stdout:/proc 2024-04-20T21:18:23.094 INFO:teuthology.orchestra.run.smithi144.stdout:/sys 2024-04-20T21:18:23.094 INFO:teuthology.orchestra.run.smithi144.stdout:/dev 2024-04-20T21:18:23.095 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/security 2024-04-20T21:18:23.095 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/shm 2024-04-20T21:18:23.095 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/pts 2024-04-20T21:18:23.095 INFO:teuthology.orchestra.run.smithi144.stdout:/run 2024-04-20T21:18:23.095 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/cgroup 2024-04-20T21:18:23.095 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/pstore 2024-04-20T21:18:23.095 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/bpf 2024-04-20T21:18:23.095 INFO:teuthology.orchestra.run.smithi144.stdout:/ 2024-04-20T21:18:23.095 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/selinux 2024-04-20T21:18:23.095 INFO:teuthology.orchestra.run.smithi144.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T21:18:23.095 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/hugepages 2024-04-20T21:18:23.095 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/mqueue 2024-04-20T21:18:23.095 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/debug 2024-04-20T21:18:23.095 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/tracing 2024-04-20T21:18:23.095 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/fuse/connections 2024-04-20T21:18:23.096 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/config 2024-04-20T21:18:23.096 INFO:teuthology.orchestra.run.smithi144.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T21:18:23.096 INFO:teuthology.orchestra.run.smithi144.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T21:18:23.096 INFO:teuthology.orchestra.run.smithi144.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T21:18:23.096 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T21:18:23.096 INFO:teuthology.orchestra.run.smithi144.stdout:/run/user/1000 2024-04-20T21:18:23.096 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph 2024-04-20T21:18:23.096 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph/osd/ceph-4 2024-04-20T21:18:23.096 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph/osd/ceph-5 2024-04-20T21:18:23.096 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph/osd/ceph-6 2024-04-20T21:18:23.096 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph/osd/ceph-7 2024-04-20T21:18:23.096 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns 2024-04-20T21:18:23.096 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T21:18:23.096 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T21:18:23.096 INFO:teuthology.orchestra.run.smithi144.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-20T21:18:23.097 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T21:18:23.097 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T21:18:23.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:23.098 DEBUG:teuthology.orchestra.run.smithi144:> ls /sys/fs/fuse/connections 2024-04-20T21:18:23.144 INFO:teuthology.orchestra.run.smithi144.stdout:40 2024-04-20T21:18:23.144 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [40] 2024-04-20T21:18:23.144 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-04-20T21:18:23.192 DEBUG:teuthology.orchestra.run.smithi144:> sudo modprobe fuse 2024-04-20T21:18:23.263 DEBUG:teuthology.orchestra.run.smithi144:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T21:18:23.305 INFO:teuthology.orchestra.run.smithi144.stdout:/proc 2024-04-20T21:18:23.305 INFO:teuthology.orchestra.run.smithi144.stdout:/sys 2024-04-20T21:18:23.306 INFO:teuthology.orchestra.run.smithi144.stdout:/dev 2024-04-20T21:18:23.306 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/security 2024-04-20T21:18:23.306 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/shm 2024-04-20T21:18:23.306 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/pts 2024-04-20T21:18:23.306 INFO:teuthology.orchestra.run.smithi144.stdout:/run 2024-04-20T21:18:23.306 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/cgroup 2024-04-20T21:18:23.306 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/pstore 2024-04-20T21:18:23.306 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/bpf 2024-04-20T21:18:23.306 INFO:teuthology.orchestra.run.smithi144.stdout:/ 2024-04-20T21:18:23.306 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/selinux 2024-04-20T21:18:23.306 INFO:teuthology.orchestra.run.smithi144.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T21:18:23.306 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/hugepages 2024-04-20T21:18:23.306 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/mqueue 2024-04-20T21:18:23.307 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/debug 2024-04-20T21:18:23.307 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/tracing 2024-04-20T21:18:23.307 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/fuse/connections 2024-04-20T21:18:23.307 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/config 2024-04-20T21:18:23.307 INFO:teuthology.orchestra.run.smithi144.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T21:18:23.307 INFO:teuthology.orchestra.run.smithi144.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T21:18:23.307 INFO:teuthology.orchestra.run.smithi144.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T21:18:23.307 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T21:18:23.307 INFO:teuthology.orchestra.run.smithi144.stdout:/run/user/1000 2024-04-20T21:18:23.307 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph 2024-04-20T21:18:23.307 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph/osd/ceph-4 2024-04-20T21:18:23.307 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph/osd/ceph-5 2024-04-20T21:18:23.307 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph/osd/ceph-6 2024-04-20T21:18:23.307 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph/osd/ceph-7 2024-04-20T21:18:23.307 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns 2024-04-20T21:18:23.307 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T21:18:23.307 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T21:18:23.308 INFO:teuthology.orchestra.run.smithi144.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-20T21:18:23.308 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T21:18:23.308 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T21:18:23.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:23.309 DEBUG:teuthology.orchestra.run.smithi144:> ls /sys/fs/fuse/connections 2024-04-20T21:18:23.365 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi144.stderr:2024-04-20T21:18:23.365+0000 7f8a8680d480 -1 init, newargv = 0x5632cbe6c440 newargc=15 2024-04-20T21:18:23.365 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi144.stderr:ceph-fuse[28934]: starting ceph client 2024-04-20T21:18:23.375 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi144.stderr:ceph-fuse[28934]: starting fuse 2024-04-20T21:18:23.388 INFO:teuthology.orchestra.run.smithi144.stdout:40 2024-04-20T21:18:23.388 INFO:teuthology.orchestra.run.smithi144.stdout:43 2024-04-20T21:18:23.389 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40, 43] 2024-04-20T21:18:23.389 DEBUG:teuthology.orchestra.run.smithi144:> sudo stdin-killer -- python3 -c ' 2024-04-20T21:18:23.389 DEBUG:teuthology.orchestra.run.smithi144:> import glob 2024-04-20T21:18:23.389 DEBUG:teuthology.orchestra.run.smithi144:> import re 2024-04-20T21:18:23.389 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-04-20T21:18:23.389 DEBUG:teuthology.orchestra.run.smithi144:> import subprocess 2024-04-20T21:18:23.389 DEBUG:teuthology.orchestra.run.smithi144:> 2024-04-20T21:18:23.389 DEBUG:teuthology.orchestra.run.smithi144:> def _find_admin_socket(client_name): 2024-04-20T21:18:23.389 DEBUG:teuthology.orchestra.run.smithi144:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-04-20T21:18:23.389 DEBUG:teuthology.orchestra.run.smithi144:> files = glob.glob(asok_path) 2024-04-20T21:18:23.390 DEBUG:teuthology.orchestra.run.smithi144:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-04-20T21:18:23.390 DEBUG:teuthology.orchestra.run.smithi144:> 2024-04-20T21:18:23.390 DEBUG:teuthology.orchestra.run.smithi144:> # Given a non-glob path, it better be there 2024-04-20T21:18:23.390 DEBUG:teuthology.orchestra.run.smithi144:> if "*" not in asok_path: 2024-04-20T21:18:23.390 DEBUG:teuthology.orchestra.run.smithi144:> assert(len(files) == 1) 2024-04-20T21:18:23.390 DEBUG:teuthology.orchestra.run.smithi144:> return files[0] 2024-04-20T21:18:23.390 DEBUG:teuthology.orchestra.run.smithi144:> 2024-04-20T21:18:23.390 DEBUG:teuthology.orchestra.run.smithi144:> for f in files: 2024-04-20T21:18:23.390 DEBUG:teuthology.orchestra.run.smithi144:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T21:18:23.390 DEBUG:teuthology.orchestra.run.smithi144:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T21:18:23.390 DEBUG:teuthology.orchestra.run.smithi144:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T21:18:23.391 DEBUG:teuthology.orchestra.run.smithi144:> contents = proc_f.read() 2024-04-20T21:18:23.391 DEBUG:teuthology.orchestra.run.smithi144:> if mountpoint in contents: 2024-04-20T21:18:23.391 DEBUG:teuthology.orchestra.run.smithi144:> return f 2024-04-20T21:18:23.391 DEBUG:teuthology.orchestra.run.smithi144:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T21:18:23.391 DEBUG:teuthology.orchestra.run.smithi144:> 2024-04-20T21:18:23.391 DEBUG:teuthology.orchestra.run.smithi144:> print(_find_admin_socket("client.3")) 2024-04-20T21:18:23.391 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-04-20T21:18:23.576 INFO:teuthology.orchestra.run.smithi144.stdout:/var/run/ceph/ceph-client.3.28934.asok 2024-04-20T21:18:23.581 INFO:teuthology.orchestra.run.smithi144.stderr:2024-04-20T21:18:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:18:23.593 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.28934.asok 2024-04-20T21:18:23.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:23.593 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.28934.asok status 2024-04-20T21:18:23.817 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-04-20T21:18:23.817 INFO:teuthology.orchestra.run.smithi144.stdout: "metadata": { 2024-04-20T21:18:23.817 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T21:18:23.817 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T21:18:23.817 INFO:teuthology.orchestra.run.smithi144.stdout: "entity_id": "3", 2024-04-20T21:18:23.817 INFO:teuthology.orchestra.run.smithi144.stdout: "hostname": "smithi144", 2024-04-20T21:18:23.817 INFO:teuthology.orchestra.run.smithi144.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-04-20T21:18:23.817 INFO:teuthology.orchestra.run.smithi144.stdout: "pid": "28934", 2024-04-20T21:18:23.817 INFO:teuthology.orchestra.run.smithi144.stdout: "root": "/" 2024-04-20T21:18:23.817 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-04-20T21:18:23.818 INFO:teuthology.orchestra.run.smithi144.stdout: "dentry_count": 0, 2024-04-20T21:18:23.818 INFO:teuthology.orchestra.run.smithi144.stdout: "dentry_pinned_count": 0, 2024-04-20T21:18:23.818 INFO:teuthology.orchestra.run.smithi144.stdout: "id": 4549, 2024-04-20T21:18:23.818 INFO:teuthology.orchestra.run.smithi144.stdout: "inst": { 2024-04-20T21:18:23.818 INFO:teuthology.orchestra.run.smithi144.stdout: "name": { 2024-04-20T21:18:23.818 INFO:teuthology.orchestra.run.smithi144.stdout: "type": "client", 2024-04-20T21:18:23.818 INFO:teuthology.orchestra.run.smithi144.stdout: "num": 4549 2024-04-20T21:18:23.818 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-04-20T21:18:23.818 INFO:teuthology.orchestra.run.smithi144.stdout: "addr": { 2024-04-20T21:18:23.818 INFO:teuthology.orchestra.run.smithi144.stdout: "type": "v1", 2024-04-20T21:18:23.818 INFO:teuthology.orchestra.run.smithi144.stdout: "addr": "192.168.0.2:0", 2024-04-20T21:18:23.818 INFO:teuthology.orchestra.run.smithi144.stdout: "nonce": 3792151529 2024-04-20T21:18:23.818 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-04-20T21:18:23.819 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-04-20T21:18:23.819 INFO:teuthology.orchestra.run.smithi144.stdout: "addr": { 2024-04-20T21:18:23.819 INFO:teuthology.orchestra.run.smithi144.stdout: "type": "v1", 2024-04-20T21:18:23.819 INFO:teuthology.orchestra.run.smithi144.stdout: "addr": "192.168.0.2:0", 2024-04-20T21:18:23.819 INFO:teuthology.orchestra.run.smithi144.stdout: "nonce": 3792151529 2024-04-20T21:18:23.819 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-04-20T21:18:23.819 INFO:teuthology.orchestra.run.smithi144.stdout: "inst_str": "client.4549 192.168.0.2:0/3792151529", 2024-04-20T21:18:23.819 INFO:teuthology.orchestra.run.smithi144.stdout: "addr_str": "192.168.0.2:0/3792151529", 2024-04-20T21:18:23.819 INFO:teuthology.orchestra.run.smithi144.stdout: "inode_count": 1, 2024-04-20T21:18:23.819 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_epoch": 7, 2024-04-20T21:18:23.819 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_epoch": 20, 2024-04-20T21:18:23.819 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_epoch_barrier": 0, 2024-04-20T21:18:23.819 INFO:teuthology.orchestra.run.smithi144.stdout: "blocklisted": false, 2024-04-20T21:18:23.819 INFO:teuthology.orchestra.run.smithi144.stdout: "fs_name": "cephfs" 2024-04-20T21:18:23.819 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-04-20T21:18:23.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:23.832 DEBUG:teuthology.orchestra.run.smithi086:> stat --file-system '--printf=%T 2024-04-20T21:18:23.832 DEBUG:teuthology.orchestra.run.smithi086:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-20T21:18:23.892 INFO:teuthology.orchestra.run.smithi086.stdout:fuseblk 2024-04-20T21:18:23.892 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-20T21:18:23.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:23.892 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-20T21:18:23.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:23.979 DEBUG:teuthology.orchestra.run.smithi086:> stat --file-system '--printf=%T 2024-04-20T21:18:23.979 DEBUG:teuthology.orchestra.run.smithi086:> ' -- /home/ubuntu/cephtest/mnt.2 2024-04-20T21:18:24.034 INFO:teuthology.orchestra.run.smithi086.stdout:fuseblk 2024-04-20T21:18:24.035 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-04-20T21:18:24.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:24.035 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-20T21:18:24.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:24.127 DEBUG:teuthology.orchestra.run.smithi144:> stat --file-system '--printf=%T 2024-04-20T21:18:24.127 DEBUG:teuthology.orchestra.run.smithi144:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-20T21:18:24.185 INFO:teuthology.orchestra.run.smithi144.stdout:fuseblk 2024-04-20T21:18:24.185 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-20T21:18:24.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:24.186 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-20T21:18:24.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:24.269 DEBUG:teuthology.orchestra.run.smithi144:> stat --file-system '--printf=%T 2024-04-20T21:18:24.269 DEBUG:teuthology.orchestra.run.smithi144:> ' -- /home/ubuntu/cephtest/mnt.3 2024-04-20T21:18:24.324 INFO:teuthology.orchestra.run.smithi144.stdout:fuseblk 2024-04-20T21:18:24.324 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-04-20T21:18:24.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:24.324 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-20T21:18:24.395 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-20T21:18:24.410 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:18:24.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.617+0000 7fe455600640 1 Processor -- start 2024-04-20T21:18:24.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.617+0000 7fe455600640 1 -- start start 2024-04-20T21:18:24.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.618+0000 7fe455600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe45015ac10 0x7fe4500b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:24.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.618+0000 7fe455600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe450072ca0 0x7fe4500cea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:24.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.618+0000 7fe455600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe45006dcd0 con 0x7fe45015f210 2024-04-20T21:18:24.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.618+0000 7fe455600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe45006e8e0 con 0x7fe45015ac10 2024-04-20T21:18:24.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.618+0000 7fe455600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fe45006dfb0 con 0x7fe450072ca0 2024-04-20T21:18:24.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.618+0000 7fe44ea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe45015ac10 0x7fe4500b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:24.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.619+0000 7fe44ea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe45015ac10 0x7fe4500b53b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44910/0 (socket says 172.21.15.86:44910) 2024-04-20T21:18:24.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.619+0000 7fe44ea00640 1 -- 172.21.15.86:0/3448711817 learned_addr learned my addr 172.21.15.86:0/3448711817 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:24.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.619+0000 7fe44f400640 1 --2- 172.21.15.86:0/3448711817 >> v2:172.21.15.144:3300/0 conn(0x7fe450072ca0 0x7fe4500cea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:24.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.619+0000 7fe44ea00640 1 -- 172.21.15.86:0/3448711817 >> v2:172.21.15.144:3300/0 conn(0x7fe450072ca0 msgr2=0x7fe4500cea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:24.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.619+0000 7fe44ea00640 1 --2- 172.21.15.86:0/3448711817 >> v2:172.21.15.144:3300/0 conn(0x7fe450072ca0 0x7fe4500cea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:24.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.619+0000 7fe44ea00640 1 -- 172.21.15.86:0/3448711817 >> v1:172.21.15.144:6789/0 conn(0x7fe45015f210 legacy=0x7fe450072590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:24.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.619+0000 7fe44ea00640 1 -- 172.21.15.86:0/3448711817 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe45006e290 con 0x7fe45015ac10 2024-04-20T21:18:24.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.620+0000 7fe44ea00640 1 --2- 172.21.15.86:0/3448711817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe45015ac10 0x7fe4500b53b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe450073590 tx=0x7fe44002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b672a29c665bdf13 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:24.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.620+0000 7fe44d600640 1 -- 172.21.15.86:0/3448711817 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44002c9f0 con 0x7fe45015ac10 2024-04-20T21:18:24.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.620+0000 7fe44d600640 1 -- 172.21.15.86:0/3448711817 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe44002cb90 con 0x7fe45015ac10 2024-04-20T21:18:24.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.620+0000 7fe44d600640 1 -- 172.21.15.86:0/3448711817 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4400343f0 con 0x7fe45015ac10 2024-04-20T21:18:24.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.621+0000 7fe455600640 1 -- 172.21.15.86:0/3448711817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe45015ac10 msgr2=0x7fe4500b53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:24.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.621+0000 7fe455600640 1 --2- 172.21.15.86:0/3448711817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe45015ac10 0x7fe4500b53b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe450073590 tx=0x7fe44002be50 comp rx=0 tx=0).stop 2024-04-20T21:18:24.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.621+0000 7fe455600640 1 -- 172.21.15.86:0/3448711817 shutdown_connections 2024-04-20T21:18:24.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.621+0000 7fe455600640 1 --2- 172.21.15.86:0/3448711817 >> v2:172.21.15.144:3300/0 conn(0x7fe450072ca0 0x7fe4500cea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:24.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.621+0000 7fe455600640 1 --2- 172.21.15.86:0/3448711817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe45015ac10 0x7fe4500b53b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:24.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.621+0000 7fe455600640 1 -- 172.21.15.86:0/3448711817 >> 172.21.15.86:0/3448711817 conn(0x7fe4500b2360 msgr2=0x7fe4500b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:24.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.621+0000 7fe455600640 1 -- 172.21.15.86:0/3448711817 shutdown_connections 2024-04-20T21:18:24.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.621+0000 7fe455600640 1 -- 172.21.15.86:0/3448711817 wait complete. 2024-04-20T21:18:24.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.623+0000 7fe455600640 1 Processor -- start 2024-04-20T21:18:24.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.623+0000 7fe455600640 1 -- start start 2024-04-20T21:18:24.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.624+0000 7fe455600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe450072ca0 0x7fe45015b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:24.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.624+0000 7fe455600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe45015ac10 0x7fe450139d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:24.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.624+0000 7fe455600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe45015f210 0x7fe45013a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:24.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.624+0000 7fe455600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe45015f650 con 0x7fe45015ac10 2024-04-20T21:18:24.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.624+0000 7fe455600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe45015aff0 con 0x7fe450072ca0 2024-04-20T21:18:24.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.624+0000 7fe455600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe45015f7d0 con 0x7fe45015f210 2024-04-20T21:18:24.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.624+0000 7fe44e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe45015ac10 0x7fe450139d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:24.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.624+0000 7fe44e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe45015ac10 0x7fe450139d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44920/0 (socket says 172.21.15.86:44920) 2024-04-20T21:18:24.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.624+0000 7fe44e000640 1 -- 172.21.15.86:0/3381942310 learned_addr learned my addr 172.21.15.86:0/3381942310 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:24.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.625+0000 7fe44ea00640 1 --2- 172.21.15.86:0/3381942310 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe450072ca0 0x7fe45015b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:24.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.625+0000 7fe44e000640 1 -- 172.21.15.86:0/3381942310 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe45015f210 msgr2=0x7fe45013a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:24.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.625+0000 7fe44e000640 1 --2- 172.21.15.86:0/3381942310 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe45015f210 0x7fe45013a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:24.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.625+0000 7fe44e000640 1 -- 172.21.15.86:0/3381942310 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe450072ca0 msgr2=0x7fe45015b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:24.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.625+0000 7fe44e000640 1 --2- 172.21.15.86:0/3381942310 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe450072ca0 0x7fe45015b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:24.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.625+0000 7fe44e000640 1 -- 172.21.15.86:0/3381942310 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe45014d0c0 con 0x7fe45015ac10 2024-04-20T21:18:24.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.625+0000 7fe44e000640 1 --2- 172.21.15.86:0/3381942310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe45015ac10 0x7fe450139d50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe43800b3e0 tx=0x7fe43800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:24.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.625+0000 7fe446a00640 1 -- 172.21.15.86:0/3381942310 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe438015020 con 0x7fe45015ac10 2024-04-20T21:18:24.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.625+0000 7fe446a00640 1 -- 172.21.15.86:0/3381942310 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe438011850 con 0x7fe45015ac10 2024-04-20T21:18:24.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.625+0000 7fe446a00640 1 -- 172.21.15.86:0/3381942310 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe438011c40 con 0x7fe45015ac10 2024-04-20T21:18:24.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.625+0000 7fe455600640 1 -- 172.21.15.86:0/3381942310 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe45013e430 con 0x7fe45015ac10 2024-04-20T21:18:24.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.625+0000 7fe455600640 1 -- 172.21.15.86:0/3381942310 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe45014d3a0 con 0x7fe45015ac10 2024-04-20T21:18:24.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.626+0000 7fe455600640 1 -- 172.21.15.86:0/3381942310 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4500689e0 con 0x7fe45015ac10 2024-04-20T21:18:24.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.628+0000 7fe446a00640 1 -- 172.21.15.86:0/3381942310 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe438018020 con 0x7fe45015ac10 2024-04-20T21:18:24.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.628+0000 7fe446a00640 1 --2- 172.21.15.86:0/3381942310 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe42c06c920 0x7fe42c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:24.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.629+0000 7fe446a00640 1 -- 172.21.15.86:0/3381942310 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe43808bae0 con 0x7fe45015ac10 2024-04-20T21:18:24.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.629+0000 7fe44ea00640 1 --2- 172.21.15.86:0/3381942310 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe42c06c920 0x7fe42c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:24.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.629+0000 7fe44ea00640 1 --2- 172.21.15.86:0/3381942310 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe42c06c920 0x7fe42c06ede0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe440035000 tx=0x7fe440030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:24.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.633+0000 7fe446a00640 1 -- 172.21.15.86:0/3381942310 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe43801f020 con 0x7fe45015ac10 2024-04-20T21:18:24.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.840+0000 7fe455600640 1 -- 172.21.15.86:0/3381942310 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe45006f450 con 0x7fe45015ac10 2024-04-20T21:18:24.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.842+0000 7fe446a00640 1 -- 172.21.15.86:0/3381942310 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7fe4380172c0 con 0x7fe45015ac10 2024-04-20T21:18:24.841 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:18:24.842 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":20,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:17:53.274462+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:17:49.718799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:17:50.683477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:18:24.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.845+0000 7fe455600640 1 -- 172.21.15.86:0/3381942310 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe42c06c920 msgr2=0x7fe42c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:24.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.845+0000 7fe455600640 1 --2- 172.21.15.86:0/3381942310 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe42c06c920 0x7fe42c06ede0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe440035000 tx=0x7fe440030000 comp rx=0 tx=0).stop 2024-04-20T21:18:24.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.845+0000 7fe455600640 1 -- 172.21.15.86:0/3381942310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe45015ac10 msgr2=0x7fe450139d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:24.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.845+0000 7fe455600640 1 --2- 172.21.15.86:0/3381942310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe45015ac10 0x7fe450139d50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe43800b3e0 tx=0x7fe43800b8a0 comp rx=0 tx=0).stop 2024-04-20T21:18:24.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.845+0000 7fe455600640 1 -- 172.21.15.86:0/3381942310 shutdown_connections 2024-04-20T21:18:24.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.845+0000 7fe455600640 1 --2- 172.21.15.86:0/3381942310 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe42c06c920 0x7fe42c06ede0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:24.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.845+0000 7fe455600640 1 --2- 172.21.15.86:0/3381942310 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe45015f210 0x7fe45013a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:24.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.845+0000 7fe455600640 1 --2- 172.21.15.86:0/3381942310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe45015ac10 0x7fe450139d50 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:24.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.845+0000 7fe455600640 1 --2- 172.21.15.86:0/3381942310 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe450072ca0 0x7fe45015b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:24.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.845+0000 7fe455600640 1 -- 172.21.15.86:0/3381942310 >> 172.21.15.86:0/3381942310 conn(0x7fe4500b2360 msgr2=0x7fe450150b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:24.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.845+0000 7fe455600640 1 -- 172.21.15.86:0/3381942310 shutdown_connections 2024-04-20T21:18:24.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:24.845+0000 7fe455600640 1 -- 172.21.15.86:0/3381942310 wait complete. 2024-04-20T21:18:24.864 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:17:49.718799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:17:50.683477+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:18:24.864 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:18:25.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.067+0000 7fa7e2a00640 1 Processor -- start 2024-04-20T21:18:25.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.067+0000 7fa7e2a00640 1 -- start start 2024-04-20T21:18:25.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.069+0000 7fa7e2a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa7dc06bbf0 0x7fa7dc06e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:25.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.069+0000 7fa7e2a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa7dc146210 0x7fa7dc0cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:25.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.069+0000 7fa7e2a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7dc15cd40 con 0x7fa7dc06bfd0 2024-04-20T21:18:25.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.069+0000 7fa7e2a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa7dc15d000 con 0x7fa7dc06bbf0 2024-04-20T21:18:25.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.069+0000 7fa7e2a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fa7dc15d2e0 con 0x7fa7dc146210 2024-04-20T21:18:25.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.069+0000 7fa7dbe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa7dc06bbf0 0x7fa7dc06e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:25.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.069+0000 7fa7db400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fa7dc06bfd0 0x7fa7dc145b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:55818/0 (socket says 172.21.15.86:55818) 2024-04-20T21:18:25.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.069+0000 7fa7db400640 1 -- 172.21.15.86:0/2153838597 learned_addr learned my addr 172.21.15.86:0/2153838597 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:18:25.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.069+0000 7fa7e0a00640 1 --2- 172.21.15.86:0/2153838597 >> v2:172.21.15.144:3300/0 conn(0x7fa7dc146210 0x7fa7dc0cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:25.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.070+0000 7fa7daa00640 1 -- 172.21.15.86:0/2153838597 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3195977099 0 0) 0x7fa7dc15cd40 con 0x7fa7dc06bfd0 2024-04-20T21:18:25.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.070+0000 7fa7daa00640 1 -- 172.21.15.86:0/2153838597 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa7c0003520 con 0x7fa7dc06bfd0 2024-04-20T21:18:25.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.070+0000 7fa7dbe00640 1 -- 172.21.15.86:0/2153838597 >> v2:172.21.15.144:3300/0 conn(0x7fa7dc146210 msgr2=0x7fa7dc0cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:25.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.070+0000 7fa7dbe00640 1 --2- 172.21.15.86:0/2153838597 >> v2:172.21.15.144:3300/0 conn(0x7fa7dc146210 0x7fa7dc0cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:25.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.070+0000 7fa7dbe00640 1 -- 172.21.15.86:0/2153838597 >> v1:172.21.15.86:6789/0 conn(0x7fa7dc06bfd0 legacy=0x7fa7dc145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:25.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.070+0000 7fa7dbe00640 1 -- 172.21.15.86:0/2153838597 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7dc15d5c0 con 0x7fa7dc06bbf0 2024-04-20T21:18:25.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.071+0000 7fa7dbe00640 1 --2- 172.21.15.86:0/2153838597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7dc06bbf0 0x7fa7dc06e000 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc06f6c0 tx=0x7fa7c802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f2041ebe1113093 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:25.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.072+0000 7fa7daa00640 1 -- 172.21.15.86:0/2153838597 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c8035020 con 0x7fa7dc06bbf0 2024-04-20T21:18:25.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.072+0000 7fa7daa00640 1 -- 172.21.15.86:0/2153838597 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7c8031830 con 0x7fa7dc06bbf0 2024-04-20T21:18:25.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.072+0000 7fa7daa00640 1 -- 172.21.15.86:0/2153838597 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c8031be0 con 0x7fa7dc06bbf0 2024-04-20T21:18:25.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.072+0000 7fa7e2a00640 1 -- 172.21.15.86:0/2153838597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7dc06bbf0 msgr2=0x7fa7dc06e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:25.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.072+0000 7fa7e2a00640 1 --2- 172.21.15.86:0/2153838597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7dc06bbf0 0x7fa7dc06e000 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc06f6c0 tx=0x7fa7c802b8d0 comp rx=0 tx=0).stop 2024-04-20T21:18:25.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.072+0000 7fa7e2a00640 1 -- 172.21.15.86:0/2153838597 shutdown_connections 2024-04-20T21:18:25.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.072+0000 7fa7e2a00640 1 --2- 172.21.15.86:0/2153838597 >> v2:172.21.15.144:3300/0 conn(0x7fa7dc146210 0x7fa7dc0cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:25.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.072+0000 7fa7e2a00640 1 --2- 172.21.15.86:0/2153838597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7dc06bbf0 0x7fa7dc06e000 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:25.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.072+0000 7fa7e2a00640 1 -- 172.21.15.86:0/2153838597 >> 172.21.15.86:0/2153838597 conn(0x7fa7dc0b2110 msgr2=0x7fa7dc0b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:25.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.073+0000 7fa7e2a00640 1 -- 172.21.15.86:0/2153838597 shutdown_connections 2024-04-20T21:18:25.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.073+0000 7fa7e2a00640 1 -- 172.21.15.86:0/2153838597 wait complete. 2024-04-20T21:18:25.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.074+0000 7fa7e2a00640 1 Processor -- start 2024-04-20T21:18:25.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.074+0000 7fa7e2a00640 1 -- start start 2024-04-20T21:18:25.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.075+0000 7fa7e2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7dc06bbf0 0x7fa7dc143220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:25.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.075+0000 7fa7e2a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7dc06bfd0 0x7fa7dc1f5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:25.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.075+0000 7fa7e2a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7dc146210 0x7fa7dc1e8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:25.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.075+0000 7fa7e2a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa7dc0cdc00 con 0x7fa7dc06bbf0 2024-04-20T21:18:25.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.075+0000 7fa7e2a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa7dc0cda80 con 0x7fa7dc146210 2024-04-20T21:18:25.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.075+0000 7fa7e2a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa7dc0cdd80 con 0x7fa7dc06bfd0 2024-04-20T21:18:25.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.076+0000 7fa7dbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7dc06bbf0 0x7fa7dc143220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:25.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.076+0000 7fa7dbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7dc06bbf0 0x7fa7dc143220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44940/0 (socket says 172.21.15.86:44940) 2024-04-20T21:18:25.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.076+0000 7fa7dbe00640 1 -- 172.21.15.86:0/3275315488 learned_addr learned my addr 172.21.15.86:0/3275315488 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:25.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.076+0000 7fa7db400640 1 --2- 172.21.15.86:0/3275315488 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7dc06bfd0 0x7fa7dc1f5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:25.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.076+0000 7fa7dbe00640 1 -- 172.21.15.86:0/3275315488 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7dc06bfd0 msgr2=0x7fa7dc1f5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:25.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.076+0000 7fa7dbe00640 1 --2- 172.21.15.86:0/3275315488 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7dc06bfd0 0x7fa7dc1f5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:25.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.076+0000 7fa7e0a00640 1 --2- 172.21.15.86:0/3275315488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7dc146210 0x7fa7dc1e8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:25.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.076+0000 7fa7dbe00640 1 -- 172.21.15.86:0/3275315488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7dc146210 msgr2=0x7fa7dc1e8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:25.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.076+0000 7fa7dbe00640 1 --2- 172.21.15.86:0/3275315488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7dc146210 0x7fa7dc1e8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:25.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.076+0000 7fa7dbe00640 1 -- 172.21.15.86:0/3275315488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7dc1f5960 con 0x7fa7dc06bbf0 2024-04-20T21:18:25.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.077+0000 7fa7dbe00640 1 --2- 172.21.15.86:0/3275315488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7dc06bbf0 0x7fa7dc143220 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa7c802b910 tx=0x7fa7c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:25.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.077+0000 7fa7d3e00640 1 -- 172.21.15.86:0/3275315488 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c8035050 con 0x7fa7dc06bbf0 2024-04-20T21:18:25.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.077+0000 7fa7d3e00640 1 -- 172.21.15.86:0/3275315488 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7c8003610 con 0x7fa7dc06bbf0 2024-04-20T21:18:25.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.077+0000 7fa7d3e00640 1 -- 172.21.15.86:0/3275315488 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c80039e0 con 0x7fa7dc06bbf0 2024-04-20T21:18:25.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.077+0000 7fa7e2a00640 1 -- 172.21.15.86:0/3275315488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7dc13df40 con 0x7fa7dc06bbf0 2024-04-20T21:18:25.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.077+0000 7fa7e2a00640 1 -- 172.21.15.86:0/3275315488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7dc13e500 con 0x7fa7dc06bbf0 2024-04-20T21:18:25.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.078+0000 7fa7e2a00640 1 -- 172.21.15.86:0/3275315488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7dc0689e0 con 0x7fa7dc06bbf0 2024-04-20T21:18:25.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.079+0000 7fa7d3e00640 1 -- 172.21.15.86:0/3275315488 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa7c8038020 con 0x7fa7dc06bbf0 2024-04-20T21:18:25.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.081+0000 7fa7d3e00640 1 --2- 172.21.15.86:0/3275315488 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa7bc06c8f0 0x7fa7bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:25.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.081+0000 7fa7d3e00640 1 -- 172.21.15.86:0/3275315488 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa7c80ada00 con 0x7fa7dc06bbf0 2024-04-20T21:18:25.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.081+0000 7fa7db400640 1 --2- 172.21.15.86:0/3275315488 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa7bc06c8f0 0x7fa7bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:25.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.082+0000 7fa7db400640 1 --2- 172.21.15.86:0/3275315488 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa7bc06c8f0 0x7fa7bc06edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc15c720 tx=0x7fa7cc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:25.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.085+0000 7fa7d3e00640 1 -- 172.21.15.86:0/3275315488 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa7c807bd90 con 0x7fa7dc06bbf0 2024-04-20T21:18:25.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.288+0000 7fa7e2a00640 1 -- 172.21.15.86:0/3275315488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa7dc06c780 con 0x7fa7dc06bbf0 2024-04-20T21:18:25.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.289+0000 7fa7d3e00640 1 -- 172.21.15.86:0/3275315488 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fa7c807b690 con 0x7fa7dc06bbf0 2024-04-20T21:18:25.288 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:18:25.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.291+0000 7fa7e2a00640 1 -- 172.21.15.86:0/3275315488 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa7bc06c8f0 msgr2=0x7fa7bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:25.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.291+0000 7fa7e2a00640 1 --2- 172.21.15.86:0/3275315488 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa7bc06c8f0 0x7fa7bc06edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc15c720 tx=0x7fa7cc009040 comp rx=0 tx=0).stop 2024-04-20T21:18:25.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.291+0000 7fa7e2a00640 1 -- 172.21.15.86:0/3275315488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7dc06bbf0 msgr2=0x7fa7dc143220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:25.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.291+0000 7fa7e2a00640 1 --2- 172.21.15.86:0/3275315488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7dc06bbf0 0x7fa7dc143220 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa7c802b910 tx=0x7fa7c80046c0 comp rx=0 tx=0).stop 2024-04-20T21:18:25.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.291+0000 7fa7e2a00640 1 -- 172.21.15.86:0/3275315488 shutdown_connections 2024-04-20T21:18:25.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.291+0000 7fa7e2a00640 1 --2- 172.21.15.86:0/3275315488 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa7bc06c8f0 0x7fa7bc06edb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:25.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.291+0000 7fa7e2a00640 1 --2- 172.21.15.86:0/3275315488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7dc146210 0x7fa7dc1e8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:25.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.291+0000 7fa7e2a00640 1 --2- 172.21.15.86:0/3275315488 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7dc06bfd0 0x7fa7dc1f5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:25.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.291+0000 7fa7e2a00640 1 --2- 172.21.15.86:0/3275315488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7dc06bbf0 0x7fa7dc143220 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:25.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.292+0000 7fa7e2a00640 1 -- 172.21.15.86:0/3275315488 >> 172.21.15.86:0/3275315488 conn(0x7fa7dc0b2110 msgr2=0x7fa7dc15dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:25.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.292+0000 7fa7e2a00640 1 -- 172.21.15.86:0/3275315488 shutdown_connections 2024-04-20T21:18:25.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.292+0000 7fa7e2a00640 1 -- 172.21.15.86:0/3275315488 wait complete. 2024-04-20T21:18:25.311 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:18:25.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.512+0000 7fd271200640 1 Processor -- start 2024-04-20T21:18:25.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.512+0000 7fd271200640 1 -- start start 2024-04-20T21:18:25.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.513+0000 7fd271200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd26c06bbf0 0x7fd26c06e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:25.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.513+0000 7fd271200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd26c06bfd0 0x7fd26c145b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:25.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.513+0000 7fd271200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd26c146040 0x7fd26c0cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:25.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.513+0000 7fd271200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd26c15cd40 con 0x7fd26c06bbf0 2024-04-20T21:18:25.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.513+0000 7fd271200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fd26c15d000 con 0x7fd26c146040 2024-04-20T21:18:25.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.513+0000 7fd271200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd26c15d2e0 con 0x7fd26c06bfd0 2024-04-20T21:18:25.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.514+0000 7fd269e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd26c06bbf0 0x7fd26c06e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:25.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.514+0000 7fd269e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd26c06bbf0 0x7fd26c06e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44950/0 (socket says 172.21.15.86:44950) 2024-04-20T21:18:25.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.514+0000 7fd269e00640 1 -- 172.21.15.86:0/1814226417 learned_addr learned my addr 172.21.15.86:0/1814226417 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:25.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.514+0000 7fd261400640 1 --2- 172.21.15.86:0/1814226417 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd26c06bfd0 0x7fd26c145b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:25.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.514+0000 7fd26a800640 1 --2- 172.21.15.86:0/1814226417 >> v2:172.21.15.144:3300/0 conn(0x7fd26c146040 0x7fd26c0cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:25.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.514+0000 7fd269e00640 1 -- 172.21.15.86:0/1814226417 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd26c06bfd0 msgr2=0x7fd26c145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:25.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.514+0000 7fd269e00640 1 --2- 172.21.15.86:0/1814226417 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd26c06bfd0 0x7fd26c145b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:25.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.514+0000 7fd269e00640 1 -- 172.21.15.86:0/1814226417 >> v2:172.21.15.144:3300/0 conn(0x7fd26c146040 msgr2=0x7fd26c0cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:25.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.514+0000 7fd269e00640 1 --2- 172.21.15.86:0/1814226417 >> v2:172.21.15.144:3300/0 conn(0x7fd26c146040 0x7fd26c0cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:25.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.514+0000 7fd269e00640 1 -- 172.21.15.86:0/1814226417 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd26c15d5c0 con 0x7fd26c06bbf0 2024-04-20T21:18:25.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.515+0000 7fd269e00640 1 --2- 172.21.15.86:0/1814226417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd26c06bbf0 0x7fd26c06e000 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd26c0cf110 tx=0x7fd25c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e112e4bbba908ef server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:25.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.516+0000 7fd269400640 1 -- 172.21.15.86:0/1814226417 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd25c035020 con 0x7fd26c06bbf0 2024-04-20T21:18:25.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.516+0000 7fd269400640 1 -- 172.21.15.86:0/1814226417 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd25c031870 con 0x7fd26c06bbf0 2024-04-20T21:18:25.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.516+0000 7fd269400640 1 -- 172.21.15.86:0/1814226417 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd25c031c40 con 0x7fd26c06bbf0 2024-04-20T21:18:25.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.516+0000 7fd271200640 1 -- 172.21.15.86:0/1814226417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd26c06bbf0 msgr2=0x7fd26c06e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:25.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.516+0000 7fd271200640 1 --2- 172.21.15.86:0/1814226417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd26c06bbf0 0x7fd26c06e000 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd26c0cf110 tx=0x7fd25c02b910 comp rx=0 tx=0).stop 2024-04-20T21:18:25.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.517+0000 7fd271200640 1 -- 172.21.15.86:0/1814226417 shutdown_connections 2024-04-20T21:18:25.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.517+0000 7fd271200640 1 --2- 172.21.15.86:0/1814226417 >> v2:172.21.15.144:3300/0 conn(0x7fd26c146040 0x7fd26c0cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:25.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.517+0000 7fd271200640 1 --2- 172.21.15.86:0/1814226417 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd26c06bfd0 0x7fd26c145b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:25.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.517+0000 7fd271200640 1 --2- 172.21.15.86:0/1814226417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd26c06bbf0 0x7fd26c06e000 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:25.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.517+0000 7fd271200640 1 -- 172.21.15.86:0/1814226417 >> 172.21.15.86:0/1814226417 conn(0x7fd26c0b2110 msgr2=0x7fd26c0b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:25.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.517+0000 7fd271200640 1 -- 172.21.15.86:0/1814226417 shutdown_connections 2024-04-20T21:18:25.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.517+0000 7fd271200640 1 -- 172.21.15.86:0/1814226417 wait complete. 2024-04-20T21:18:25.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.518+0000 7fd271200640 1 Processor -- start 2024-04-20T21:18:25.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.518+0000 7fd271200640 1 -- start start 2024-04-20T21:18:25.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.519+0000 7fd271200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd26c06bbf0 0x7fd26c0c9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:25.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.519+0000 7fd271200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd26c06bfd0 0x7fd26c1e78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:25.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.519+0000 7fd271200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd26c146040 0x7fd26c1e7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:25.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.519+0000 7fd271200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd26c0cdc00 con 0x7fd26c06bfd0 2024-04-20T21:18:25.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.519+0000 7fd271200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd26c0cda80 con 0x7fd26c06bbf0 2024-04-20T21:18:25.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.519+0000 7fd271200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd26c0cdd80 con 0x7fd26c146040 2024-04-20T21:18:25.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.520+0000 7fd261400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd26c06bfd0 0x7fd26c1e78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:25.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.520+0000 7fd261400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd26c06bfd0 0x7fd26c1e78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44966/0 (socket says 172.21.15.86:44966) 2024-04-20T21:18:25.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.520+0000 7fd261400640 1 -- 172.21.15.86:0/2330470177 learned_addr learned my addr 172.21.15.86:0/2330470177 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:25.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.520+0000 7fd269e00640 1 --2- 172.21.15.86:0/2330470177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd26c06bbf0 0x7fd26c0c9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:25.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.520+0000 7fd261400640 1 -- 172.21.15.86:0/2330470177 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd26c146040 msgr2=0x7fd26c1e7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:25.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.520+0000 7fd261400640 1 --2- 172.21.15.86:0/2330470177 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd26c146040 0x7fd26c1e7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:25.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.520+0000 7fd261400640 1 -- 172.21.15.86:0/2330470177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd26c06bbf0 msgr2=0x7fd26c0c9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:25.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.520+0000 7fd261400640 1 --2- 172.21.15.86:0/2330470177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd26c06bbf0 0x7fd26c0c9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:25.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.520+0000 7fd261400640 1 -- 172.21.15.86:0/2330470177 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd26c1e8350 con 0x7fd26c06bfd0 2024-04-20T21:18:25.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.520+0000 7fd261400640 1 --2- 172.21.15.86:0/2330470177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd26c06bfd0 0x7fd26c1e78d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd25400d800 tx=0x7fd25400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:25.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.521+0000 7fd262a00640 1 -- 172.21.15.86:0/2330470177 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd254018030 con 0x7fd26c06bfd0 2024-04-20T21:18:25.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.521+0000 7fd262a00640 1 -- 172.21.15.86:0/2330470177 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2540096a0 con 0x7fd26c06bfd0 2024-04-20T21:18:25.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.521+0000 7fd271200640 1 -- 172.21.15.86:0/2330470177 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd26c1e85c0 con 0x7fd26c06bfd0 2024-04-20T21:18:25.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.521+0000 7fd262a00640 1 -- 172.21.15.86:0/2330470177 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd254009870 con 0x7fd26c06bfd0 2024-04-20T21:18:25.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.521+0000 7fd271200640 1 -- 172.21.15.86:0/2330470177 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd26c12eea0 con 0x7fd26c06bfd0 2024-04-20T21:18:25.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.522+0000 7fd271200640 1 -- 172.21.15.86:0/2330470177 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd26c0689e0 con 0x7fd26c06bfd0 2024-04-20T21:18:25.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.523+0000 7fd262a00640 1 -- 172.21.15.86:0/2330470177 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd25401b070 con 0x7fd26c06bfd0 2024-04-20T21:18:25.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.524+0000 7fd262a00640 1 --2- 172.21.15.86:0/2330470177 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd24806c8f0 0x7fd24806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:25.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.524+0000 7fd262a00640 1 -- 172.21.15.86:0/2330470177 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd25408bdb0 con 0x7fd26c06bfd0 2024-04-20T21:18:25.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.524+0000 7fd269e00640 1 --2- 172.21.15.86:0/2330470177 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd24806c8f0 0x7fd24806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:25.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.525+0000 7fd269e00640 1 --2- 172.21.15.86:0/2330470177 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd24806c8f0 0x7fd24806edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd25c036000 tx=0x7fd25c03c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:25.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.528+0000 7fd262a00640 1 -- 172.21.15.86:0/2330470177 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd25401be40 con 0x7fd26c06bfd0 2024-04-20T21:18:25.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.729+0000 7fd271200640 1 -- 172.21.15.86:0/2330470177 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd26c06c780 con 0x7fd26c06bfd0 2024-04-20T21:18:25.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.730+0000 7fd262a00640 1 -- 172.21.15.86:0/2330470177 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fd25405a1e0 con 0x7fd26c06bfd0 2024-04-20T21:18:25.730 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-20T21:18:25.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.733+0000 7fd271200640 1 -- 172.21.15.86:0/2330470177 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd24806c8f0 msgr2=0x7fd24806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:25.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.733+0000 7fd271200640 1 --2- 172.21.15.86:0/2330470177 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd24806c8f0 0x7fd24806edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd25c036000 tx=0x7fd25c03c040 comp rx=0 tx=0).stop 2024-04-20T21:18:25.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.733+0000 7fd271200640 1 -- 172.21.15.86:0/2330470177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd26c06bfd0 msgr2=0x7fd26c1e78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:25.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.733+0000 7fd271200640 1 --2- 172.21.15.86:0/2330470177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd26c06bfd0 0x7fd26c1e78d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd25400d800 tx=0x7fd25400dcc0 comp rx=0 tx=0).stop 2024-04-20T21:18:25.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.733+0000 7fd271200640 1 -- 172.21.15.86:0/2330470177 shutdown_connections 2024-04-20T21:18:25.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.733+0000 7fd271200640 1 --2- 172.21.15.86:0/2330470177 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd24806c8f0 0x7fd24806edb0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:25.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.733+0000 7fd271200640 1 --2- 172.21.15.86:0/2330470177 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd26c146040 0x7fd26c1e7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:25.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.733+0000 7fd271200640 1 --2- 172.21.15.86:0/2330470177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd26c06bfd0 0x7fd26c1e78d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:25.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.733+0000 7fd271200640 1 --2- 172.21.15.86:0/2330470177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd26c06bbf0 0x7fd26c0c9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:25.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.733+0000 7fd271200640 1 -- 172.21.15.86:0/2330470177 >> 172.21.15.86:0/2330470177 conn(0x7fd26c0b2110 msgr2=0x7fd26c15f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:25.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.734+0000 7fd271200640 1 -- 172.21.15.86:0/2330470177 shutdown_connections 2024-04-20T21:18:25.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.734+0000 7fd271200640 1 -- 172.21.15.86:0/2330470177 wait complete. 2024-04-20T21:18:25.752 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:18:25.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.957+0000 7f9621e00640 1 Processor -- start 2024-04-20T21:18:25.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.958+0000 7f9621e00640 1 -- start start 2024-04-20T21:18:25.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.959+0000 7f9621e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f961c15e8f0 0x7f961c162640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:25.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.959+0000 7f9621e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f961c0d29a0 con 0x7f961c0cfd90 2024-04-20T21:18:25.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.959+0000 7f9621e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f961c1631b0 con 0x7f961c15ecd0 2024-04-20T21:18:25.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.959+0000 7f9621e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f961c0d2fd0 con 0x7f961c15e8f0 2024-04-20T21:18:25.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.959+0000 7f961b400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f961c0cfd90 0x7f961c0d3460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60376/0 (socket says 172.21.15.86:60376) 2024-04-20T21:18:25.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.959+0000 7f961b400640 1 -- 172.21.15.86:0/2823735408 learned_addr learned my addr 172.21.15.86:0/2823735408 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:18:25.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.960+0000 7f961aa00640 1 --2- 172.21.15.86:0/2823735408 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f961c15e8f0 0x7f961c162640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:25.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.960+0000 7f9619600640 1 -- 172.21.15.86:0/2823735408 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3742741257 0 0) 0x7f961c0d29a0 con 0x7f961c0cfd90 2024-04-20T21:18:25.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.960+0000 7f9619600640 1 -- 172.21.15.86:0/2823735408 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9600003520 con 0x7f961c0cfd90 2024-04-20T21:18:25.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.960+0000 7f9619600640 1 -- 172.21.15.86:0/2823735408 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2929683088 0 0) 0x7f9600003520 con 0x7f961c0cfd90 2024-04-20T21:18:25.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.960+0000 7f9619600640 1 -- 172.21.15.86:0/2823735408 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f961c15e8f0 msgr2=0x7f961c162640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:25.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.960+0000 7f9619600640 1 --2- 172.21.15.86:0/2823735408 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f961c15e8f0 0x7f961c162640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:25.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.960+0000 7f9619600640 1 -- 172.21.15.86:0/2823735408 >> v1:172.21.15.144:6789/0 conn(0x7f961c15ecd0 legacy=0x7f961c0cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:25.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.960+0000 7f9619600640 1 -- 172.21.15.86:0/2823735408 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f961c163490 con 0x7f961c0cfd90 2024-04-20T21:18:25.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.960+0000 7f9619600640 1 -- 172.21.15.86:0/2823735408 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f9614001670 con 0x7f961c0cfd90 2024-04-20T21:18:25.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.960+0000 7f9619600640 1 -- 172.21.15.86:0/2823735408 >> v1:172.21.15.86:6789/0 conn(0x7f961c0cfd90 legacy=0x7f961c0d3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:25.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.960+0000 7f9619600640 1 --2- 172.21.15.86:0/2823735408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96000039e0 0x7f9600003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:25.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.960+0000 7f9619600640 1 --2- 172.21.15.86:0/2823735408 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f961c15e8f0 0x7f9600004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:25.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.961+0000 7f9619600640 1 --2- 172.21.15.86:0/2823735408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9600004d40 0x7f96000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:25.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.961+0000 7f9619600640 1 -- 172.21.15.86:0/2823735408 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f961c0d2fd0 con 0x7f9600004d40 2024-04-20T21:18:25.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.961+0000 7f9619600640 1 -- 172.21.15.86:0/2823735408 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f960000ba70 con 0x7f96000039e0 2024-04-20T21:18:25.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.961+0000 7f9619600640 1 -- 172.21.15.86:0/2823735408 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f960000bc70 con 0x7f961c15e8f0 2024-04-20T21:18:25.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.961+0000 7f961b400640 1 --2- 172.21.15.86:0/2823735408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9600004d40 0x7f96000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:25.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.961+0000 7f961aa00640 1 --2- 172.21.15.86:0/2823735408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96000039e0 0x7f9600003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:25.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.961+0000 7f961a000640 1 --2- 172.21.15.86:0/2823735408 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f961c15e8f0 0x7f9600004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:25.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.961+0000 7f961b400640 1 -- 172.21.15.86:0/2823735408 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f961c15e8f0 msgr2=0x7f9600004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:25.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.961+0000 7f961b400640 1 --2- 172.21.15.86:0/2823735408 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f961c15e8f0 0x7f9600004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:25.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.961+0000 7f961b400640 1 -- 172.21.15.86:0/2823735408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96000039e0 msgr2=0x7f9600003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:25.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.961+0000 7f961b400640 1 --2- 172.21.15.86:0/2823735408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96000039e0 0x7f9600003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:25.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.961+0000 7f961b400640 1 -- 172.21.15.86:0/2823735408 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9600003520 con 0x7f9600004d40 2024-04-20T21:18:25.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.962+0000 7f961b400640 1 --2- 172.21.15.86:0/2823735408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9600004d40 0x7f96000051c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f961400c210 tx=0x7f9600005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f89efb8d38baa8a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:25.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.963+0000 7f9619600640 1 -- 172.21.15.86:0/2823735408 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9614007d10 con 0x7f9600004d40 2024-04-20T21:18:25.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.963+0000 7f9619600640 1 -- 172.21.15.86:0/2823735408 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9614030070 con 0x7f9600004d40 2024-04-20T21:18:25.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.963+0000 7f9619600640 1 -- 172.21.15.86:0/2823735408 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9614005d30 con 0x7f9600004d40 2024-04-20T21:18:25.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.963+0000 7f9621e00640 1 -- 172.21.15.86:0/2823735408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9600004d40 msgr2=0x7f96000051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:25.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.963+0000 7f9621e00640 1 --2- 172.21.15.86:0/2823735408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9600004d40 0x7f96000051c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f961400c210 tx=0x7f9600005830 comp rx=0 tx=0).stop 2024-04-20T21:18:25.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.963+0000 7f961b400640 1 -- 172.21.15.86:0/2823735408 reap_dead start 2024-04-20T21:18:25.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.963+0000 7f9621e00640 1 -- 172.21.15.86:0/2823735408 shutdown_connections 2024-04-20T21:18:25.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.963+0000 7f9621e00640 1 -- 172.21.15.86:0/2823735408 >> 172.21.15.86:0/2823735408 conn(0x7f961c06cc70 msgr2=0x7f961c06d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:25.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.963+0000 7f9621e00640 1 -- 172.21.15.86:0/2823735408 shutdown_connections 2024-04-20T21:18:25.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.963+0000 7f9621e00640 1 -- 172.21.15.86:0/2823735408 wait complete. 2024-04-20T21:18:25.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.965+0000 7f9621e00640 1 Processor -- start 2024-04-20T21:18:25.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.965+0000 7f9621e00640 1 -- start start 2024-04-20T21:18:25.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.966+0000 7f9621e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9600004d40 0x7f961c0d3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:25.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.966+0000 7f9621e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f961c1f7b80 0x7f961c1f8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:25.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.966+0000 7f9621e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f961c152900 0x7f961c152db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:25.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.966+0000 7f9621e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f961c0ba7f0 con 0x7f9600004d40 2024-04-20T21:18:25.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.966+0000 7f9621e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f961c0b7a50 con 0x7f961c152900 2024-04-20T21:18:25.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.966+0000 7f9621e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f961c162640 con 0x7f961c1f7b80 2024-04-20T21:18:25.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.967+0000 7f961aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9600004d40 0x7f961c0d3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:25.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.967+0000 7f961aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9600004d40 0x7f961c0d3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58846/0 (socket says 172.21.15.86:58846) 2024-04-20T21:18:25.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.967+0000 7f961aa00640 1 -- 172.21.15.86:0/904700026 learned_addr learned my addr 172.21.15.86:0/904700026 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:25.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.967+0000 7f961a000640 1 --2- 172.21.15.86:0/904700026 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f961c1f7b80 0x7f961c1f8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:25.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.967+0000 7f961b400640 1 --2- 172.21.15.86:0/904700026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f961c152900 0x7f961c152db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:25.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.967+0000 7f961aa00640 1 -- 172.21.15.86:0/904700026 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f961c1f7b80 msgr2=0x7f961c1f8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:25.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.967+0000 7f961aa00640 1 --2- 172.21.15.86:0/904700026 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f961c1f7b80 0x7f961c1f8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:25.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.967+0000 7f961aa00640 1 -- 172.21.15.86:0/904700026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f961c152900 msgr2=0x7f961c152db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:25.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.967+0000 7f961aa00640 1 --2- 172.21.15.86:0/904700026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f961c152900 0x7f961c152db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:25.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.967+0000 7f961aa00640 1 -- 172.21.15.86:0/904700026 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f961c1532f0 con 0x7f9600004d40 2024-04-20T21:18:25.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.968+0000 7f961aa00640 1 --2- 172.21.15.86:0/904700026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9600004d40 0x7f961c0d3e80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f960c011600 tx=0x7f960c011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:25.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.968+0000 7f9613400640 1 -- 172.21.15.86:0/904700026 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960c00f920 con 0x7f9600004d40 2024-04-20T21:18:25.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.968+0000 7f9613400640 1 -- 172.21.15.86:0/904700026 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f960c00fac0 con 0x7f9600004d40 2024-04-20T21:18:25.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.968+0000 7f9613400640 1 -- 172.21.15.86:0/904700026 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960c00fe70 con 0x7f9600004d40 2024-04-20T21:18:25.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.968+0000 7f9621e00640 1 -- 172.21.15.86:0/904700026 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f961c1f8640 con 0x7f9600004d40 2024-04-20T21:18:25.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.968+0000 7f9621e00640 1 -- 172.21.15.86:0/904700026 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f961c1537f0 con 0x7f9600004d40 2024-04-20T21:18:25.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.969+0000 7f9621e00640 1 -- 172.21.15.86:0/904700026 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f961c1535e0 con 0x7f9600004d40 2024-04-20T21:18:25.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.970+0000 7f9613400640 1 -- 172.21.15.86:0/904700026 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f960c015460 con 0x7f9600004d40 2024-04-20T21:18:25.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.972+0000 7f9613400640 1 --2- 172.21.15.86:0/904700026 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f95fc06c8f0 0x7f95fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:25.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.972+0000 7f9613400640 1 -- 172.21.15.86:0/904700026 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f960c08c490 con 0x7f9600004d40 2024-04-20T21:18:25.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.972+0000 7f961a000640 1 --2- 172.21.15.86:0/904700026 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f95fc06c8f0 0x7f95fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:25.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.972+0000 7f961a000640 1 --2- 172.21.15.86:0/904700026 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f95fc06c8f0 0x7f95fc06edb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f961c0b8e10 tx=0x7f9608009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:25.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:25.976+0000 7f9613400640 1 -- 172.21.15.86:0/904700026 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f960c019030 con 0x7f9600004d40 2024-04-20T21:18:26.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.177+0000 7f9621e00640 1 -- 172.21.15.86:0/904700026 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f961c0d2550 con 0x7f9600004d40 2024-04-20T21:18:26.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.178+0000 7f9613400640 1 -- 172.21.15.86:0/904700026 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f960c05a950 con 0x7f9600004d40 2024-04-20T21:18:26.177 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-20T21:18:26.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.180+0000 7f9621e00640 1 -- 172.21.15.86:0/904700026 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f95fc06c8f0 msgr2=0x7f95fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:26.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.180+0000 7f9621e00640 1 --2- 172.21.15.86:0/904700026 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f95fc06c8f0 0x7f95fc06edb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f961c0b8e10 tx=0x7f9608009040 comp rx=0 tx=0).stop 2024-04-20T21:18:26.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.181+0000 7f9621e00640 1 -- 172.21.15.86:0/904700026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9600004d40 msgr2=0x7f961c0d3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:26.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.181+0000 7f9621e00640 1 --2- 172.21.15.86:0/904700026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9600004d40 0x7f961c0d3e80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f960c011600 tx=0x7f960c011ac0 comp rx=0 tx=0).stop 2024-04-20T21:18:26.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.181+0000 7f9621e00640 1 -- 172.21.15.86:0/904700026 shutdown_connections 2024-04-20T21:18:26.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.181+0000 7f9621e00640 1 --2- 172.21.15.86:0/904700026 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f95fc06c8f0 0x7f95fc06edb0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:26.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.181+0000 7f9621e00640 1 --2- 172.21.15.86:0/904700026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f961c152900 0x7f961c152db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:26.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.181+0000 7f9621e00640 1 --2- 172.21.15.86:0/904700026 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f961c1f7b80 0x7f961c1f8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:26.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.181+0000 7f9621e00640 1 --2- 172.21.15.86:0/904700026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9600004d40 0x7f961c0d3e80 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:26.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.181+0000 7f9621e00640 1 -- 172.21.15.86:0/904700026 >> 172.21.15.86:0/904700026 conn(0x7f961c06cc70 msgr2=0x7f961c0b9780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:26.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.181+0000 7f9621e00640 1 -- 172.21.15.86:0/904700026 shutdown_connections 2024-04-20T21:18:26.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.181+0000 7f9621e00640 1 -- 172.21.15.86:0/904700026 wait complete. 2024-04-20T21:18:26.199 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:18:26.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.401+0000 7f93a5600640 1 Processor -- start 2024-04-20T21:18:26.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.401+0000 7f93a5600640 1 -- start start 2024-04-20T21:18:26.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.403+0000 7f93a5600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93a0156ae0 0x7f93a015d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:26.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.403+0000 7f93a5600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f93a00ad7f0 0x7f93a006e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:26.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.403+0000 7f93a5600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93a0160890 con 0x7f93a0160080 2024-04-20T21:18:26.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.403+0000 7f93a5600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f93a006db20 con 0x7f93a00ad7f0 2024-04-20T21:18:26.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.403+0000 7f93a5600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f93a0160b70 con 0x7f93a0156ae0 2024-04-20T21:18:26.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.403+0000 7f939f400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f93a0160080 0x7f93a00ad0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60388/0 (socket says 172.21.15.86:60388) 2024-04-20T21:18:26.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.403+0000 7f939f400640 1 -- 172.21.15.86:0/3678956570 learned_addr learned my addr 172.21.15.86:0/3678956570 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:18:26.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.404+0000 7f93a4c00640 1 --2- 172.21.15.86:0/3678956570 >> v2:172.21.15.144:3300/0 conn(0x7f93a00ad7f0 0x7f93a006e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:26.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.404+0000 7f939fe00640 1 --2- 172.21.15.86:0/3678956570 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93a0156ae0 0x7f93a015d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:26.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.404+0000 7f939ea00640 1 -- 172.21.15.86:0/3678956570 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 903000197 0 0) 0x7f93a0160890 con 0x7f93a0160080 2024-04-20T21:18:26.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.404+0000 7f939ea00640 1 -- 172.21.15.86:0/3678956570 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9380003520 con 0x7f93a0160080 2024-04-20T21:18:26.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.404+0000 7f939ea00640 1 -- 172.21.15.86:0/3678956570 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 413345296 0 0) 0x7f9380003520 con 0x7f93a0160080 2024-04-20T21:18:26.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.404+0000 7f939ea00640 1 -- 172.21.15.86:0/3678956570 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93a0156ae0 msgr2=0x7f93a015d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:26.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.404+0000 7f939ea00640 1 --2- 172.21.15.86:0/3678956570 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93a0156ae0 0x7f93a015d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:26.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.404+0000 7f939ea00640 1 -- 172.21.15.86:0/3678956570 >> v2:172.21.15.144:3300/0 conn(0x7f93a00ad7f0 msgr2=0x7f93a006e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:26.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.404+0000 7f939ea00640 1 --2- 172.21.15.86:0/3678956570 >> v2:172.21.15.144:3300/0 conn(0x7f93a00ad7f0 0x7f93a006e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:26.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.404+0000 7f939ea00640 1 -- 172.21.15.86:0/3678956570 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93a0160e50 con 0x7f93a0160080 2024-04-20T21:18:26.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.404+0000 7f939ea00640 1 -- 172.21.15.86:0/3678956570 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f93880019f0 con 0x7f93a0160080 2024-04-20T21:18:26.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.404+0000 7f939ea00640 1 -- 172.21.15.86:0/3678956570 >> v1:172.21.15.86:6789/0 conn(0x7f93a0160080 legacy=0x7f93a00ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:26.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.404+0000 7f939ea00640 1 --2- 172.21.15.86:0/3678956570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93800039b0 0x7f9380003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:26.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.404+0000 7f939ea00640 1 --2- 172.21.15.86:0/3678956570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9380004800 0x7f9380004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:26.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.404+0000 7f939ea00640 1 --2- 172.21.15.86:0/3678956570 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93a0156ae0 0x7f93800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:26.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.404+0000 7f939ea00640 1 -- 172.21.15.86:0/3678956570 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f93a006db20 con 0x7f93800039b0 2024-04-20T21:18:26.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.404+0000 7f939ea00640 1 -- 172.21.15.86:0/3678956570 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f93a0160b70 con 0x7f9380004800 2024-04-20T21:18:26.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.404+0000 7f939ea00640 1 -- 172.21.15.86:0/3678956570 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f938000b9c0 con 0x7f93a0156ae0 2024-04-20T21:18:26.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.405+0000 7f939fe00640 1 --2- 172.21.15.86:0/3678956570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93800039b0 0x7f9380003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:26.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.405+0000 7f939f400640 1 --2- 172.21.15.86:0/3678956570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9380004800 0x7f9380004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:26.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.405+0000 7f93a4c00640 1 --2- 172.21.15.86:0/3678956570 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93a0156ae0 0x7f93800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:26.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.405+0000 7f939fe00640 1 -- 172.21.15.86:0/3678956570 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93a0156ae0 msgr2=0x7f93800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:26.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.405+0000 7f939fe00640 1 --2- 172.21.15.86:0/3678956570 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93a0156ae0 0x7f93800051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:26.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.405+0000 7f939fe00640 1 -- 172.21.15.86:0/3678956570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9380004800 msgr2=0x7f9380004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:26.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.405+0000 7f939fe00640 1 --2- 172.21.15.86:0/3678956570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9380004800 0x7f9380004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:26.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.405+0000 7f939fe00640 1 -- 172.21.15.86:0/3678956570 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9380003520 con 0x7f93800039b0 2024-04-20T21:18:26.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.405+0000 7f939f400640 1 --2- 172.21.15.86:0/3678956570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9380004800 0x7f9380004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:26.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.405+0000 7f93a4c00640 1 --2- 172.21.15.86:0/3678956570 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93a0156ae0 0x7f93800051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:26.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.406+0000 7f939fe00640 1 --2- 172.21.15.86:0/3678956570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93800039b0 0x7f9380003dd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9398009ac0 tx=0x7f9380005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b98a1ea94fa6ce4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:26.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.406+0000 7f939ea00640 1 -- 172.21.15.86:0/3678956570 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9398039070 con 0x7f93800039b0 2024-04-20T21:18:26.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.406+0000 7f939ea00640 1 -- 172.21.15.86:0/3678956570 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f939802c6b0 con 0x7f93800039b0 2024-04-20T21:18:26.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.406+0000 7f939ea00640 1 -- 172.21.15.86:0/3678956570 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9398009690 con 0x7f93800039b0 2024-04-20T21:18:26.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.407+0000 7f93a5600640 1 -- 172.21.15.86:0/3678956570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93800039b0 msgr2=0x7f9380003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:26.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.407+0000 7f93a5600640 1 --2- 172.21.15.86:0/3678956570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93800039b0 0x7f9380003dd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9398009ac0 tx=0x7f9380005830 comp rx=0 tx=0).stop 2024-04-20T21:18:26.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.407+0000 7f93a4c00640 1 -- 172.21.15.86:0/3678956570 reap_dead start 2024-04-20T21:18:26.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.407+0000 7f93a5600640 1 -- 172.21.15.86:0/3678956570 shutdown_connections 2024-04-20T21:18:26.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.407+0000 7f93a5600640 1 -- 172.21.15.86:0/3678956570 >> 172.21.15.86:0/3678956570 conn(0x7f93a00d0170 msgr2=0x7f93a00d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:26.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.407+0000 7f93a5600640 1 -- 172.21.15.86:0/3678956570 shutdown_connections 2024-04-20T21:18:26.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.407+0000 7f93a5600640 1 -- 172.21.15.86:0/3678956570 wait complete. 2024-04-20T21:18:26.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.409+0000 7f93a5600640 1 Processor -- start 2024-04-20T21:18:26.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.409+0000 7f93a5600640 1 -- start start 2024-04-20T21:18:26.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.410+0000 7f93a5600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93800039b0 0x7f93a01445c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:26.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.410+0000 7f93a5600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93a0142da0 0x7f93a0144b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:26.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.410+0000 7f93a5600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93a01f4350 0x7f93a01f47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:26.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.410+0000 7f93a5600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f93a0160500 con 0x7f93a0142da0 2024-04-20T21:18:26.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.410+0000 7f93a5600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f93a0156ec0 con 0x7f93800039b0 2024-04-20T21:18:26.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.410+0000 7f93a5600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f93a015d6a0 con 0x7f93a01f4350 2024-04-20T21:18:26.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.411+0000 7f939f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93a0142da0 0x7f93a0144b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:26.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.411+0000 7f939f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93a0142da0 0x7f93a0144b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58854/0 (socket says 172.21.15.86:58854) 2024-04-20T21:18:26.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.411+0000 7f939f400640 1 -- 172.21.15.86:0/321195582 learned_addr learned my addr 172.21.15.86:0/321195582 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:26.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.411+0000 7f939fe00640 1 --2- 172.21.15.86:0/321195582 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93800039b0 0x7f93a01445c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:26.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.411+0000 7f93a4c00640 1 --2- 172.21.15.86:0/321195582 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93a01f4350 0x7f93a01f47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:26.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.411+0000 7f939f400640 1 -- 172.21.15.86:0/321195582 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93a01f4350 msgr2=0x7f93a01f47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:26.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.411+0000 7f939f400640 1 --2- 172.21.15.86:0/321195582 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93a01f4350 0x7f93a01f47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:26.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.411+0000 7f939f400640 1 -- 172.21.15.86:0/321195582 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93800039b0 msgr2=0x7f93a01445c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:26.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.411+0000 7f939f400640 1 --2- 172.21.15.86:0/321195582 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93800039b0 0x7f93a01445c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:26.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.411+0000 7f939f400640 1 -- 172.21.15.86:0/321195582 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93a015c630 con 0x7f93a0142da0 2024-04-20T21:18:26.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.412+0000 7f939f400640 1 --2- 172.21.15.86:0/321195582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93a0142da0 0x7f93a0144b00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9388001aa0 tx=0x7f938800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:26.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.412+0000 7f9397e00640 1 -- 172.21.15.86:0/321195582 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9388019070 con 0x7f93a0142da0 2024-04-20T21:18:26.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.412+0000 7f9397e00640 1 -- 172.21.15.86:0/321195582 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9388004d60 con 0x7f93a0142da0 2024-04-20T21:18:26.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.412+0000 7f9397e00640 1 -- 172.21.15.86:0/321195582 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9388007310 con 0x7f93a0142da0 2024-04-20T21:18:26.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.412+0000 7f93a5600640 1 -- 172.21.15.86:0/321195582 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93a0143280 con 0x7f93a0142da0 2024-04-20T21:18:26.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.412+0000 7f93a5600640 1 -- 172.21.15.86:0/321195582 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93a0143840 con 0x7f93a0142da0 2024-04-20T21:18:26.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.413+0000 7f93a5600640 1 -- 172.21.15.86:0/321195582 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93a004fcf0 con 0x7f93a0142da0 2024-04-20T21:18:26.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.414+0000 7f9397e00640 1 -- 172.21.15.86:0/321195582 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f93880074b0 con 0x7f93a0142da0 2024-04-20T21:18:26.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.415+0000 7f9397e00640 1 --2- 172.21.15.86:0/321195582 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f937c06c8f0 0x7f937c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:26.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.415+0000 7f9397e00640 1 -- 172.21.15.86:0/321195582 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f938808bca0 con 0x7f93a0142da0 2024-04-20T21:18:26.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.415+0000 7f939fe00640 1 --2- 172.21.15.86:0/321195582 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f937c06c8f0 0x7f937c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:26.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.416+0000 7f939fe00640 1 --2- 172.21.15.86:0/321195582 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f937c06c8f0 0x7f937c06edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9398036000 tx=0x7f9398030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:26.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.419+0000 7f9397e00640 1 -- 172.21.15.86:0/321195582 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f93880599e0 con 0x7f93a0142da0 2024-04-20T21:18:26.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.624+0000 7f93a5600640 1 -- 172.21.15.86:0/321195582 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f93a006fae0 con 0x7f93a0142da0 2024-04-20T21:18:26.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.627+0000 7f9397e00640 1 -- 172.21.15.86:0/321195582 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7f938805dce0 con 0x7f93a0142da0 2024-04-20T21:18:26.627 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:18:26.627 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":20,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:17:53.274462+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:17:49.718799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:17:50.683477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:18:26.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.630+0000 7f93a5600640 1 -- 172.21.15.86:0/321195582 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f937c06c8f0 msgr2=0x7f937c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:26.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.630+0000 7f93a5600640 1 --2- 172.21.15.86:0/321195582 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f937c06c8f0 0x7f937c06edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9398036000 tx=0x7f9398030000 comp rx=0 tx=0).stop 2024-04-20T21:18:26.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.630+0000 7f93a5600640 1 -- 172.21.15.86:0/321195582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93a0142da0 msgr2=0x7f93a0144b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:26.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.630+0000 7f93a5600640 1 --2- 172.21.15.86:0/321195582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93a0142da0 0x7f93a0144b00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9388001aa0 tx=0x7f938800b4a0 comp rx=0 tx=0).stop 2024-04-20T21:18:26.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.630+0000 7f93a5600640 1 -- 172.21.15.86:0/321195582 shutdown_connections 2024-04-20T21:18:26.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.630+0000 7f93a5600640 1 --2- 172.21.15.86:0/321195582 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f937c06c8f0 0x7f937c06edb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:26.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.630+0000 7f93a5600640 1 --2- 172.21.15.86:0/321195582 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93a01f4350 0x7f93a01f47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:26.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.630+0000 7f93a5600640 1 --2- 172.21.15.86:0/321195582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93a0142da0 0x7f93a0144b00 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:26.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.630+0000 7f93a5600640 1 --2- 172.21.15.86:0/321195582 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93800039b0 0x7f93a01445c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:26.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.630+0000 7f93a5600640 1 -- 172.21.15.86:0/321195582 >> 172.21.15.86:0/321195582 conn(0x7f93a00d0170 msgr2=0x7f93a0154740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:26.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.630+0000 7f93a5600640 1 -- 172.21.15.86:0/321195582 shutdown_connections 2024-04-20T21:18:26.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.630+0000 7f93a5600640 1 -- 172.21.15.86:0/321195582 wait complete. 2024-04-20T21:18:26.649 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:17:49.718799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:17:50.683477+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:18:26.650 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:18:26.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.847+0000 7f1969800640 1 Processor -- start 2024-04-20T21:18:26.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.848+0000 7f1969800640 1 -- start start 2024-04-20T21:18:26.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.849+0000 7f1969800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f196415ee00 0x7f19641634c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:26.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.849+0000 7f1969800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f19640cf830 0x7f196415fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:26.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.849+0000 7f1969800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19640cff90 con 0x7f19640d1260 2024-04-20T21:18:26.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.849+0000 7f1969800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f19640d0550 con 0x7f19640cf830 2024-04-20T21:18:26.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.849+0000 7f1969800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f19640d0270 con 0x7f196415ee00 2024-04-20T21:18:26.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.849+0000 7f1962000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f19640cf830 0x7f196415fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:26.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.849+0000 7f1962000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f19640cf830 0x7f196415fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58864/0 (socket says 172.21.15.86:58864) 2024-04-20T21:18:26.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.849+0000 7f1962000640 1 -- 172.21.15.86:0/2587354114 learned_addr learned my addr 172.21.15.86:0/2587354114 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:26.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.850+0000 7f1962000640 1 -- 172.21.15.86:0/2587354114 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f196415ee00 msgr2=0x7f19641634c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:26.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.850+0000 7f1962000640 1 --2- 172.21.15.86:0/2587354114 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f196415ee00 0x7f19641634c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:26.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.850+0000 7f1962000640 1 -- 172.21.15.86:0/2587354114 >> v1:172.21.15.144:6789/0 conn(0x7f19640d1260 legacy=0x7f19640d3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:26.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.850+0000 7f1962000640 1 -- 172.21.15.86:0/2587354114 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f196415f640 con 0x7f19640cf830 2024-04-20T21:18:26.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.851+0000 7f1962000640 1 --2- 172.21.15.86:0/2587354114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19640cf830 0x7f196415fe40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f194c009e50 tx=0x7f194c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7e7a88234354da6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:26.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.851+0000 7f1961600640 1 -- 172.21.15.86:0/2587354114 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194c02ca70 con 0x7f19640cf830 2024-04-20T21:18:26.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.851+0000 7f1961600640 1 -- 172.21.15.86:0/2587354114 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f194c02cc10 con 0x7f19640cf830 2024-04-20T21:18:26.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.851+0000 7f1961600640 1 -- 172.21.15.86:0/2587354114 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194c0343f0 con 0x7f19640cf830 2024-04-20T21:18:26.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.852+0000 7f1969800640 1 -- 172.21.15.86:0/2587354114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19640cf830 msgr2=0x7f196415fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:26.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.852+0000 7f1969800640 1 --2- 172.21.15.86:0/2587354114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19640cf830 0x7f196415fe40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f194c009e50 tx=0x7f194c02bed0 comp rx=0 tx=0).stop 2024-04-20T21:18:26.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.852+0000 7f1969800640 1 -- 172.21.15.86:0/2587354114 shutdown_connections 2024-04-20T21:18:26.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.852+0000 7f1969800640 1 --2- 172.21.15.86:0/2587354114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19640cf830 0x7f196415fe40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:26.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.852+0000 7f1969800640 1 --2- 172.21.15.86:0/2587354114 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f196415ee00 0x7f19641634c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:26.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.852+0000 7f1969800640 1 -- 172.21.15.86:0/2587354114 >> 172.21.15.86:0/2587354114 conn(0x7f196406cc20 msgr2=0x7f196406d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:26.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.852+0000 7f1969800640 1 -- 172.21.15.86:0/2587354114 shutdown_connections 2024-04-20T21:18:26.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.852+0000 7f1969800640 1 -- 172.21.15.86:0/2587354114 wait complete. 2024-04-20T21:18:26.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.854+0000 7f1969800640 1 Processor -- start 2024-04-20T21:18:26.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.854+0000 7f1969800640 1 -- start start 2024-04-20T21:18:26.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.855+0000 7f1969800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f19640cf830 0x7f1964153b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:26.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.855+0000 7f1969800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19640d1260 0x7f19640cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:26.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.855+0000 7f1969800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f196415ee00 0x7f19641f8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:26.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.855+0000 7f1969800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f196415f1e0 con 0x7f196415ee00 2024-04-20T21:18:26.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.855+0000 7f1969800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1964160380 con 0x7f19640d1260 2024-04-20T21:18:26.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.855+0000 7f1969800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f196415f360 con 0x7f19640cf830 2024-04-20T21:18:26.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.855+0000 7f1963400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f196415ee00 0x7f19641f8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:26.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.855+0000 7f1963400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f196415ee00 0x7f19641f8910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58876/0 (socket says 172.21.15.86:58876) 2024-04-20T21:18:26.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.855+0000 7f1963400640 1 -- 172.21.15.86:0/3288403466 learned_addr learned my addr 172.21.15.86:0/3288403466 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:26.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.855+0000 7f1962000640 1 --2- 172.21.15.86:0/3288403466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19640d1260 0x7f19640cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:26.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.855+0000 7f1962a00640 1 --2- 172.21.15.86:0/3288403466 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f19640cf830 0x7f1964153b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:26.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.855+0000 7f1963400640 1 -- 172.21.15.86:0/3288403466 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f19640cf830 msgr2=0x7f1964153b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:26.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.855+0000 7f1963400640 1 --2- 172.21.15.86:0/3288403466 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f19640cf830 0x7f1964153b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:26.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.855+0000 7f1963400640 1 -- 172.21.15.86:0/3288403466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19640d1260 msgr2=0x7f19640cbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:26.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.855+0000 7f1963400640 1 --2- 172.21.15.86:0/3288403466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19640d1260 0x7f19640cbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:26.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.855+0000 7f1963400640 1 -- 172.21.15.86:0/3288403466 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19641550d0 con 0x7f196415ee00 2024-04-20T21:18:26.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.855+0000 7f1962000640 1 --2- 172.21.15.86:0/3288403466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19640d1260 0x7f19640cbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:26.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.856+0000 7f1963400640 1 --2- 172.21.15.86:0/3288403466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f196415ee00 0x7f19641f8910 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f195c00c410 tx=0x7f195c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:26.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.856+0000 7f195aa00640 1 -- 172.21.15.86:0/3288403466 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195c011790 con 0x7f196415ee00 2024-04-20T21:18:26.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.856+0000 7f195aa00640 1 -- 172.21.15.86:0/3288403466 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f195c017070 con 0x7f196415ee00 2024-04-20T21:18:26.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.856+0000 7f1969800640 1 -- 172.21.15.86:0/3288403466 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19641553b0 con 0x7f196415ee00 2024-04-20T21:18:26.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.856+0000 7f195aa00640 1 -- 172.21.15.86:0/3288403466 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195c011bb0 con 0x7f196415ee00 2024-04-20T21:18:26.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.856+0000 7f1969800640 1 -- 172.21.15.86:0/3288403466 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1964155970 con 0x7f196415ee00 2024-04-20T21:18:26.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.857+0000 7f1969800640 1 -- 172.21.15.86:0/3288403466 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19640b83c0 con 0x7f196415ee00 2024-04-20T21:18:26.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.858+0000 7f195aa00640 1 -- 172.21.15.86:0/3288403466 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f195c01c460 con 0x7f196415ee00 2024-04-20T21:18:26.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.859+0000 7f195aa00640 1 --2- 172.21.15.86:0/3288403466 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f194006c8f0 0x7f194006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:26.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.859+0000 7f195aa00640 1 -- 172.21.15.86:0/3288403466 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f195c08ba10 con 0x7f196415ee00 2024-04-20T21:18:26.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.859+0000 7f1962a00640 1 --2- 172.21.15.86:0/3288403466 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f194006c8f0 0x7f194006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:26.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.860+0000 7f1962a00640 1 --2- 172.21.15.86:0/3288403466 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f194006c8f0 0x7f194006edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f195400a770 tx=0x7f1954008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:26.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:26.864+0000 7f195aa00640 1 -- 172.21.15.86:0/3288403466 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f19640b83c0 con 0x7f196415ee00 2024-04-20T21:18:27.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.063+0000 7f1969800640 1 -- 172.21.15.86:0/3288403466 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1964155e40 con 0x7f196415ee00 2024-04-20T21:18:27.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.064+0000 7f195aa00640 1 -- 172.21.15.86:0/3288403466 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f195c059da0 con 0x7f196415ee00 2024-04-20T21:18:27.063 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:18:27.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.066+0000 7f1969800640 1 -- 172.21.15.86:0/3288403466 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f194006c8f0 msgr2=0x7f194006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:27.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.066+0000 7f1969800640 1 --2- 172.21.15.86:0/3288403466 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f194006c8f0 0x7f194006edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f195400a770 tx=0x7f1954008040 comp rx=0 tx=0).stop 2024-04-20T21:18:27.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.066+0000 7f1969800640 1 -- 172.21.15.86:0/3288403466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f196415ee00 msgr2=0x7f19641f8910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:27.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.066+0000 7f1969800640 1 --2- 172.21.15.86:0/3288403466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f196415ee00 0x7f19641f8910 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f195c00c410 tx=0x7f195c00c8d0 comp rx=0 tx=0).stop 2024-04-20T21:18:27.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.067+0000 7f1969800640 1 -- 172.21.15.86:0/3288403466 shutdown_connections 2024-04-20T21:18:27.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.067+0000 7f1969800640 1 --2- 172.21.15.86:0/3288403466 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f194006c8f0 0x7f194006edb0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:27.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.067+0000 7f1969800640 1 --2- 172.21.15.86:0/3288403466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f196415ee00 0x7f19641f8910 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:27.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.067+0000 7f1969800640 1 --2- 172.21.15.86:0/3288403466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19640d1260 0x7f19640cbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:27.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.067+0000 7f1969800640 1 --2- 172.21.15.86:0/3288403466 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f19640cf830 0x7f1964153b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:27.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.067+0000 7f1969800640 1 -- 172.21.15.86:0/3288403466 >> 172.21.15.86:0/3288403466 conn(0x7f196406cc20 msgr2=0x7f1964154610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:27.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.067+0000 7f1969800640 1 -- 172.21.15.86:0/3288403466 shutdown_connections 2024-04-20T21:18:27.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.067+0000 7f1969800640 1 -- 172.21.15.86:0/3288403466 wait complete. 2024-04-20T21:18:27.085 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:18:27.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.272+0000 7fc06b200640 1 Processor -- start 2024-04-20T21:18:27.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.272+0000 7fc06b200640 1 -- start start 2024-04-20T21:18:27.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.274+0000 7fc06b200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc06c0cfc70 0x7fc06c0cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:27.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.274+0000 7fc06b200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc06c15bcd0 con 0x7fc06c0cdbe0 2024-04-20T21:18:27.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.274+0000 7fc06b200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc06c15bfb0 con 0x7fc06c06f5c0 2024-04-20T21:18:27.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.274+0000 7fc06b200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc06c15c8e0 con 0x7fc06c0cfc70 2024-04-20T21:18:27.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.274+0000 7fc069e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc06c0cfc70 0x7fc06c0cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:27.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.274+0000 7fc061400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fc06c0cdbe0 0x7fc06c06eeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60398/0 (socket says 172.21.15.86:60398) 2024-04-20T21:18:27.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.274+0000 7fc061400640 1 -- 172.21.15.86:0/1132618869 learned_addr learned my addr 172.21.15.86:0/1132618869 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:18:27.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.274+0000 7fc069e00640 1 -- 172.21.15.86:0/1132618869 >> v1:172.21.15.144:6789/0 conn(0x7fc06c06f5c0 legacy=0x7fc06c071ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:27.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.274+0000 7fc069e00640 1 -- 172.21.15.86:0/1132618869 >> v1:172.21.15.86:6789/0 conn(0x7fc06c0cdbe0 legacy=0x7fc06c06eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:27.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.274+0000 7fc069e00640 1 -- 172.21.15.86:0/1132618869 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc06c15c290 con 0x7fc06c0cfc70 2024-04-20T21:18:27.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.275+0000 7fc069e00640 1 --2- 172.21.15.86:0/1132618869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc06c0cfc70 0x7fc06c0cd6a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc06c15ca60 tx=0x7fc05c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b78e3ddbd8c83f11 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:27.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.276+0000 7fc069400640 1 -- 172.21.15.86:0/1132618869 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc05c035020 con 0x7fc06c0cfc70 2024-04-20T21:18:27.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.276+0000 7fc069400640 1 -- 172.21.15.86:0/1132618869 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc05c031810 con 0x7fc06c0cfc70 2024-04-20T21:18:27.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.276+0000 7fc069400640 1 -- 172.21.15.86:0/1132618869 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc05c031be0 con 0x7fc06c0cfc70 2024-04-20T21:18:27.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.276+0000 7fc06b200640 1 -- 172.21.15.86:0/1132618869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc06c0cfc70 msgr2=0x7fc06c0cd6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:27.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.276+0000 7fc06b200640 1 --2- 172.21.15.86:0/1132618869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc06c0cfc70 0x7fc06c0cd6a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc06c15ca60 tx=0x7fc05c02b8b0 comp rx=0 tx=0).stop 2024-04-20T21:18:27.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.276+0000 7fc06b200640 1 -- 172.21.15.86:0/1132618869 shutdown_connections 2024-04-20T21:18:27.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.276+0000 7fc06b200640 1 --2- 172.21.15.86:0/1132618869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc06c0cfc70 0x7fc06c0cd6a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:27.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.276+0000 7fc06b200640 1 -- 172.21.15.86:0/1132618869 >> 172.21.15.86:0/1132618869 conn(0x7fc06c067cf0 msgr2=0x7fc06c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:27.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.277+0000 7fc06b200640 1 -- 172.21.15.86:0/1132618869 shutdown_connections 2024-04-20T21:18:27.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.277+0000 7fc06b200640 1 -- 172.21.15.86:0/1132618869 wait complete. 2024-04-20T21:18:27.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.278+0000 7fc06b200640 1 Processor -- start 2024-04-20T21:18:27.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.278+0000 7fc06b200640 1 -- start start 2024-04-20T21:18:27.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.279+0000 7fc06b200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc06c06f5c0 0x7fc06c153d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:27.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.279+0000 7fc06b200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc06c0cdbe0 0x7fc06c1542c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:27.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.279+0000 7fc06b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc06c0cfc70 0x7fc06c14a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:27.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.279+0000 7fc06b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c0d0050 con 0x7fc06c0cfc70 2024-04-20T21:18:27.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.279+0000 7fc06b200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c0b3710 con 0x7fc06c0cdbe0 2024-04-20T21:18:27.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.279+0000 7fc06b200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc06c0d01d0 con 0x7fc06c06f5c0 2024-04-20T21:18:27.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.279+0000 7fc06a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc06c0cfc70 0x7fc06c14a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:27.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.279+0000 7fc06a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc06c0cfc70 0x7fc06c14a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58892/0 (socket says 172.21.15.86:58892) 2024-04-20T21:18:27.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.279+0000 7fc06a800640 1 -- 172.21.15.86:0/3558149607 learned_addr learned my addr 172.21.15.86:0/3558149607 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:27.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.280+0000 7fc061400640 1 --2- 172.21.15.86:0/3558149607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc06c0cdbe0 0x7fc06c1542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:27.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.280+0000 7fc06a800640 1 -- 172.21.15.86:0/3558149607 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc06c06f5c0 msgr2=0x7fc06c153d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:27.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.280+0000 7fc069e00640 1 --2- 172.21.15.86:0/3558149607 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc06c06f5c0 0x7fc06c153d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:27.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.280+0000 7fc06a800640 1 --2- 172.21.15.86:0/3558149607 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc06c06f5c0 0x7fc06c153d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:27.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.280+0000 7fc06a800640 1 -- 172.21.15.86:0/3558149607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc06c0cdbe0 msgr2=0x7fc06c1542c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:27.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.280+0000 7fc06a800640 1 --2- 172.21.15.86:0/3558149607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc06c0cdbe0 0x7fc06c1542c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:27.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.280+0000 7fc06a800640 1 -- 172.21.15.86:0/3558149607 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc06c1442e0 con 0x7fc06c0cfc70 2024-04-20T21:18:27.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.280+0000 7fc069e00640 1 --2- 172.21.15.86:0/3558149607 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc06c06f5c0 0x7fc06c153d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:27.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.280+0000 7fc06a800640 1 --2- 172.21.15.86:0/3558149607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc06c0cfc70 0x7fc06c14a0c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc06400c780 tx=0x7fc06400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:27.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.280+0000 7fc062a00640 1 -- 172.21.15.86:0/3558149607 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc064017070 con 0x7fc06c0cfc70 2024-04-20T21:18:27.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.280+0000 7fc062a00640 1 -- 172.21.15.86:0/3558149607 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc064003da0 con 0x7fc06c0cfc70 2024-04-20T21:18:27.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.280+0000 7fc062a00640 1 -- 172.21.15.86:0/3558149607 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0640052b0 con 0x7fc06c0cfc70 2024-04-20T21:18:27.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.280+0000 7fc06b200640 1 -- 172.21.15.86:0/3558149607 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc06c14c4f0 con 0x7fc06c0cfc70 2024-04-20T21:18:27.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.281+0000 7fc06b200640 1 -- 172.21.15.86:0/3558149607 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc06c14cab0 con 0x7fc06c0cfc70 2024-04-20T21:18:27.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.281+0000 7fc06b200640 1 -- 172.21.15.86:0/3558149607 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc06c143620 con 0x7fc06c0cfc70 2024-04-20T21:18:27.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.282+0000 7fc062a00640 1 -- 172.21.15.86:0/3558149607 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc06401a070 con 0x7fc06c0cfc70 2024-04-20T21:18:27.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.283+0000 7fc062a00640 1 --2- 172.21.15.86:0/3558149607 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc04806c8f0 0x7fc04806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:27.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.283+0000 7fc062a00640 1 -- 172.21.15.86:0/3558149607 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc06408b1a0 con 0x7fc06c0cfc70 2024-04-20T21:18:27.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.284+0000 7fc069e00640 1 --2- 172.21.15.86:0/3558149607 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc04806c8f0 0x7fc04806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:27.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.284+0000 7fc069e00640 1 --2- 172.21.15.86:0/3558149607 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc04806c8f0 0x7fc04806edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc05c036000 tx=0x7fc05c03c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:27.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.288+0000 7fc062a00640 1 -- 172.21.15.86:0/3558149607 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc06c1442e0 con 0x7fc06c0cfc70 2024-04-20T21:18:27.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.469+0000 7fc06b200640 1 -- 172.21.15.86:0/3558149607 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc06c1439a0 con 0x7fc06c0cfc70 2024-04-20T21:18:27.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.470+0000 7fc062a00640 1 -- 172.21.15.86:0/3558149607 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fc0640595c0 con 0x7fc06c0cfc70 2024-04-20T21:18:27.469 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-20T21:18:27.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.472+0000 7fc06b200640 1 -- 172.21.15.86:0/3558149607 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc04806c8f0 msgr2=0x7fc04806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:27.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.472+0000 7fc06b200640 1 --2- 172.21.15.86:0/3558149607 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc04806c8f0 0x7fc04806edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc05c036000 tx=0x7fc05c03c040 comp rx=0 tx=0).stop 2024-04-20T21:18:27.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.472+0000 7fc06b200640 1 -- 172.21.15.86:0/3558149607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc06c0cfc70 msgr2=0x7fc06c14a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:27.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.472+0000 7fc06b200640 1 --2- 172.21.15.86:0/3558149607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc06c0cfc70 0x7fc06c14a0c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc06400c780 tx=0x7fc06400cc40 comp rx=0 tx=0).stop 2024-04-20T21:18:27.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.472+0000 7fc06b200640 1 -- 172.21.15.86:0/3558149607 shutdown_connections 2024-04-20T21:18:27.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.472+0000 7fc06b200640 1 --2- 172.21.15.86:0/3558149607 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc04806c8f0 0x7fc04806edb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:27.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.472+0000 7fc06b200640 1 --2- 172.21.15.86:0/3558149607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc06c0cfc70 0x7fc06c14a0c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:27.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.472+0000 7fc06b200640 1 --2- 172.21.15.86:0/3558149607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc06c0cdbe0 0x7fc06c1542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:27.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.472+0000 7fc06b200640 1 --2- 172.21.15.86:0/3558149607 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc06c06f5c0 0x7fc06c153d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:27.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.472+0000 7fc06b200640 1 -- 172.21.15.86:0/3558149607 >> 172.21.15.86:0/3558149607 conn(0x7fc06c067cf0 msgr2=0x7fc06c15f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:27.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.472+0000 7fc06b200640 1 -- 172.21.15.86:0/3558149607 shutdown_connections 2024-04-20T21:18:27.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.472+0000 7fc06b200640 1 -- 172.21.15.86:0/3558149607 wait complete. 2024-04-20T21:18:27.490 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:18:27.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.681+0000 7f3177200640 1 Processor -- start 2024-04-20T21:18:27.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.681+0000 7f3177200640 1 -- start start 2024-04-20T21:18:27.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.683+0000 7f3177200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f317815f3c0 0x7f31780ae770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:27.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.683+0000 7f3177200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31780729f0 con 0x7f317815b4b0 2024-04-20T21:18:27.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.683+0000 7f3177200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3178072cd0 con 0x7f317815ef50 2024-04-20T21:18:27.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.683+0000 7f3177200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f317815fe10 con 0x7f317815f3c0 2024-04-20T21:18:27.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.683+0000 7f3175e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f317815b4b0 0x7f31780cc3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60404/0 (socket says 172.21.15.86:60404) 2024-04-20T21:18:27.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.683+0000 7f3175e00640 1 -- 172.21.15.86:0/1161659535 learned_addr learned my addr 172.21.15.86:0/1161659535 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:18:27.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.683+0000 7f3176800640 1 --2- 172.21.15.86:0/1161659535 >> v2:172.21.15.144:3300/0 conn(0x7f317815f3c0 0x7f31780ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:27.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.683+0000 7f3174a00640 1 -- 172.21.15.86:0/1161659535 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3783098107 0 0) 0x7f31780729f0 con 0x7f317815b4b0 2024-04-20T21:18:27.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.683+0000 7f3174a00640 1 -- 172.21.15.86:0/1161659535 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f315c003520 con 0x7f317815b4b0 2024-04-20T21:18:27.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.684+0000 7f3174a00640 1 -- 172.21.15.86:0/1161659535 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1957654047 0 0) 0x7f315c003520 con 0x7f317815b4b0 2024-04-20T21:18:27.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.684+0000 7f3174a00640 1 -- 172.21.15.86:0/1161659535 >> v2:172.21.15.144:3300/0 conn(0x7f317815f3c0 msgr2=0x7f31780ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:27.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.684+0000 7f3174a00640 1 --2- 172.21.15.86:0/1161659535 >> v2:172.21.15.144:3300/0 conn(0x7f317815f3c0 0x7f31780ae770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:27.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.684+0000 7f3174a00640 1 -- 172.21.15.86:0/1161659535 >> v1:172.21.15.144:6789/0 conn(0x7f317815ef50 legacy=0x7f31780ae060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:27.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.684+0000 7f3174a00640 1 -- 172.21.15.86:0/1161659535 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3178072fb0 con 0x7f317815b4b0 2024-04-20T21:18:27.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.684+0000 7f3174a00640 1 -- 172.21.15.86:0/1161659535 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f3168001670 con 0x7f317815b4b0 2024-04-20T21:18:27.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.684+0000 7f3174a00640 1 -- 172.21.15.86:0/1161659535 >> v1:172.21.15.86:6789/0 conn(0x7f317815b4b0 legacy=0x7f31780cc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:27.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.684+0000 7f3174a00640 1 --2- 172.21.15.86:0/1161659535 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f315c003a00 0x7f315c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:27.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.684+0000 7f3174a00640 1 --2- 172.21.15.86:0/1161659535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f315c004800 0x7f315c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:27.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.684+0000 7f3174a00640 1 --2- 172.21.15.86:0/1161659535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f315c005210 0x7f315c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:27.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.684+0000 7f3174a00640 1 -- 172.21.15.86:0/1161659535 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f317815fe10 con 0x7f315c005210 2024-04-20T21:18:27.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.684+0000 7f3174a00640 1 -- 172.21.15.86:0/1161659535 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f315c00c140 con 0x7f315c004800 2024-04-20T21:18:27.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.684+0000 7f3174a00640 1 -- 172.21.15.86:0/1161659535 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f315c00c340 con 0x7f315c003a00 2024-04-20T21:18:27.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.684+0000 7f3176800640 1 --2- 172.21.15.86:0/1161659535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f315c005210 0x7f315c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:27.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.684+0000 7f3175400640 1 --2- 172.21.15.86:0/1161659535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f315c004800 0x7f315c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:27.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.684+0000 7f3175e00640 1 --2- 172.21.15.86:0/1161659535 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f315c003a00 0x7f315c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:27.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.685+0000 7f3176800640 1 -- 172.21.15.86:0/1161659535 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f315c003a00 msgr2=0x7f315c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:27.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.685+0000 7f3176800640 1 --2- 172.21.15.86:0/1161659535 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f315c003a00 0x7f315c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:27.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.685+0000 7f3176800640 1 -- 172.21.15.86:0/1161659535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f315c004800 msgr2=0x7f315c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:27.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.685+0000 7f3176800640 1 --2- 172.21.15.86:0/1161659535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f315c004800 0x7f315c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:27.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.685+0000 7f3176800640 1 -- 172.21.15.86:0/1161659535 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f315c003520 con 0x7f315c005210 2024-04-20T21:18:27.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.685+0000 7f3175400640 1 --2- 172.21.15.86:0/1161659535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f315c004800 0x7f315c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:27.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.685+0000 7f3176800640 1 -- 172.21.15.86:0/1161659535 reap_dead start 2024-04-20T21:18:27.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.686+0000 7f3176800640 1 --2- 172.21.15.86:0/1161659535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f315c005210 0x7f315c00b9c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3170007330 tx=0x7f315c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e0d002216d4746c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:27.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.686+0000 7f3174a00640 1 -- 172.21.15.86:0/1161659535 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3170009be0 con 0x7f315c005210 2024-04-20T21:18:27.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.686+0000 7f3174a00640 1 -- 172.21.15.86:0/1161659535 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f317002ccd0 con 0x7f315c005210 2024-04-20T21:18:27.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.686+0000 7f3174a00640 1 -- 172.21.15.86:0/1161659535 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3170003bd0 con 0x7f315c005210 2024-04-20T21:18:27.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.686+0000 7f3177200640 1 -- 172.21.15.86:0/1161659535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f315c005210 msgr2=0x7f315c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:27.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.686+0000 7f3177200640 1 --2- 172.21.15.86:0/1161659535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f315c005210 0x7f315c00b9c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3170007330 tx=0x7f315c005830 comp rx=0 tx=0).stop 2024-04-20T21:18:27.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.686+0000 7f3177200640 1 -- 172.21.15.86:0/1161659535 shutdown_connections 2024-04-20T21:18:27.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.686+0000 7f3177200640 1 --2- 172.21.15.86:0/1161659535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f315c005210 0x7f315c00b9c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:27.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.686+0000 7f3177200640 1 -- 172.21.15.86:0/1161659535 >> 172.21.15.86:0/1161659535 conn(0x7f31780b1f00 msgr2=0x7f31780b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:27.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.687+0000 7f3177200640 1 -- 172.21.15.86:0/1161659535 shutdown_connections 2024-04-20T21:18:27.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.687+0000 7f3177200640 1 -- 172.21.15.86:0/1161659535 wait complete. 2024-04-20T21:18:27.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.688+0000 7f3177200640 1 Processor -- start 2024-04-20T21:18:27.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.688+0000 7f3177200640 1 -- start start 2024-04-20T21:18:27.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.689+0000 7f3177200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f315c005210 0x7f31780999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:27.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.689+0000 7f3177200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f317814c7a0 0x7f31781f0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:27.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.689+0000 7f3177200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31781f08a0 0x7f3178159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:27.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.689+0000 7f3177200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f317815f7e0 con 0x7f315c005210 2024-04-20T21:18:27.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.689+0000 7f3177200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3178073570 con 0x7f31781f08a0 2024-04-20T21:18:27.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.689+0000 7f3177200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3178072650 con 0x7f317814c7a0 2024-04-20T21:18:27.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.689+0000 7f3175e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f315c005210 0x7f31780999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:27.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.689+0000 7f3175e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f315c005210 0x7f31780999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58922/0 (socket says 172.21.15.86:58922) 2024-04-20T21:18:27.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.689+0000 7f3175e00640 1 -- 172.21.15.86:0/3583192985 learned_addr learned my addr 172.21.15.86:0/3583192985 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:27.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.690+0000 7f3176800640 1 --2- 172.21.15.86:0/3583192985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31781f08a0 0x7f3178159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:27.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.690+0000 7f3175400640 1 --2- 172.21.15.86:0/3583192985 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f317814c7a0 0x7f31781f0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:27.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.690+0000 7f3175e00640 1 -- 172.21.15.86:0/3583192985 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f317814c7a0 msgr2=0x7f31781f0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:27.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.690+0000 7f3175e00640 1 --2- 172.21.15.86:0/3583192985 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f317814c7a0 0x7f31781f0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:27.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.690+0000 7f3175e00640 1 -- 172.21.15.86:0/3583192985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31781f08a0 msgr2=0x7f3178159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:27.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.690+0000 7f3175e00640 1 --2- 172.21.15.86:0/3583192985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31781f08a0 0x7f3178159f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:27.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.690+0000 7f3175e00640 1 -- 172.21.15.86:0/3583192985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3178158e80 con 0x7f315c005210 2024-04-20T21:18:27.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.690+0000 7f3175400640 1 --2- 172.21.15.86:0/3583192985 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f317814c7a0 0x7f31781f0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:27.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.690+0000 7f3175e00640 1 --2- 172.21.15.86:0/3583192985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f315c005210 0x7f31780999c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f316800e7b0 tx=0x7f316800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:27.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.690+0000 7f316e000640 1 -- 172.21.15.86:0/3583192985 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316800ee60 con 0x7f315c005210 2024-04-20T21:18:27.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.690+0000 7f316e000640 1 -- 172.21.15.86:0/3583192985 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3168018070 con 0x7f315c005210 2024-04-20T21:18:27.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.690+0000 7f316e000640 1 -- 172.21.15.86:0/3583192985 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316800ca90 con 0x7f315c005210 2024-04-20T21:18:27.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.690+0000 7f3177200640 1 -- 172.21.15.86:0/3583192985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3178159160 con 0x7f315c005210 2024-04-20T21:18:27.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.691+0000 7f3177200640 1 -- 172.21.15.86:0/3583192985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3178159720 con 0x7f315c005210 2024-04-20T21:18:27.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.692+0000 7f3177200640 1 -- 172.21.15.86:0/3583192985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f317804fd90 con 0x7f315c005210 2024-04-20T21:18:27.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.693+0000 7f316e000640 1 -- 172.21.15.86:0/3583192985 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f316800cc30 con 0x7f315c005210 2024-04-20T21:18:27.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.694+0000 7f316e000640 1 --2- 172.21.15.86:0/3583192985 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f315806c920 0x7f315806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:27.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.694+0000 7f316e000640 1 -- 172.21.15.86:0/3583192985 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f316808b660 con 0x7f315c005210 2024-04-20T21:18:27.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.694+0000 7f3175400640 1 --2- 172.21.15.86:0/3583192985 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f315806c920 0x7f315806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:27.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.695+0000 7f3175400640 1 --2- 172.21.15.86:0/3583192985 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f315806c920 0x7f315806ede0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3164007fd0 tx=0x7f3164009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:27.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.698+0000 7f316e000640 1 -- 172.21.15.86:0/3583192985 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3168059a00 con 0x7f315c005210 2024-04-20T21:18:27.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.903+0000 7f3177200640 1 -- 172.21.15.86:0/3583192985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f317815fbf0 con 0x7f315c005210 2024-04-20T21:18:27.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.904+0000 7f316e000640 1 -- 172.21.15.86:0/3583192985 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f3168059300 con 0x7f315c005210 2024-04-20T21:18:27.903 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-20T21:18:27.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.906+0000 7f3177200640 1 -- 172.21.15.86:0/3583192985 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f315806c920 msgr2=0x7f315806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:27.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.906+0000 7f3177200640 1 --2- 172.21.15.86:0/3583192985 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f315806c920 0x7f315806ede0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3164007fd0 tx=0x7f3164009040 comp rx=0 tx=0).stop 2024-04-20T21:18:27.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.906+0000 7f3177200640 1 -- 172.21.15.86:0/3583192985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f315c005210 msgr2=0x7f31780999c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:27.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.906+0000 7f3177200640 1 --2- 172.21.15.86:0/3583192985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f315c005210 0x7f31780999c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f316800e7b0 tx=0x7f316800ec70 comp rx=0 tx=0).stop 2024-04-20T21:18:27.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.907+0000 7f3177200640 1 -- 172.21.15.86:0/3583192985 shutdown_connections 2024-04-20T21:18:27.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.907+0000 7f3177200640 1 --2- 172.21.15.86:0/3583192985 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f315806c920 0x7f315806ede0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:27.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.907+0000 7f3177200640 1 --2- 172.21.15.86:0/3583192985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31781f08a0 0x7f3178159f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:27.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.907+0000 7f3177200640 1 --2- 172.21.15.86:0/3583192985 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f317814c7a0 0x7f31781f0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:27.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.907+0000 7f3177200640 1 --2- 172.21.15.86:0/3583192985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f315c005210 0x7f31780999c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:27.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.907+0000 7f3177200640 1 -- 172.21.15.86:0/3583192985 >> 172.21.15.86:0/3583192985 conn(0x7f31780b1f00 msgr2=0x7f317806ebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:27.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.907+0000 7f3177200640 1 -- 172.21.15.86:0/3583192985 shutdown_connections 2024-04-20T21:18:27.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:27.907+0000 7f3177200640 1 -- 172.21.15.86:0/3583192985 wait complete. 2024-04-20T21:18:27.924 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:18:28.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.122+0000 7f6afc200640 1 Processor -- start 2024-04-20T21:18:28.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.122+0000 7f6afc200640 1 -- start start 2024-04-20T21:18:28.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.124+0000 7f6afc200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6af4146130 0x7f6af406e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:28.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.124+0000 7f6afc200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6af406dc30 con 0x7f6af4072410 2024-04-20T21:18:28.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.124+0000 7f6afc200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6af40cd8e0 con 0x7f6af40ccdd0 2024-04-20T21:18:28.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.124+0000 7f6afc200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6af40cdbc0 con 0x7f6af4146130 2024-04-20T21:18:28.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.124+0000 7f6afa200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f6af4072410 0x7f6af415cc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60406/0 (socket says 172.21.15.86:60406) 2024-04-20T21:18:28.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.124+0000 7f6afa200640 1 -- 172.21.15.86:0/3675496367 learned_addr learned my addr 172.21.15.86:0/3675496367 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:18:28.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.125+0000 7f6af9800640 1 --2- 172.21.15.86:0/3675496367 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6af4146130 0x7f6af406e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:28.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.125+0000 7f6af3e00640 1 -- 172.21.15.86:0/3675496367 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3945640657 0 0) 0x7f6af406dc30 con 0x7f6af4072410 2024-04-20T21:18:28.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.125+0000 7f6af3e00640 1 -- 172.21.15.86:0/3675496367 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6adc003520 con 0x7f6af4072410 2024-04-20T21:18:28.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.125+0000 7f6af3e00640 1 -- 172.21.15.86:0/3675496367 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 676549354 0 0) 0x7f6adc003520 con 0x7f6af4072410 2024-04-20T21:18:28.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.125+0000 7f6af3e00640 1 -- 172.21.15.86:0/3675496367 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6af4146130 msgr2=0x7f6af406e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:28.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.125+0000 7f6af3e00640 1 --2- 172.21.15.86:0/3675496367 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6af4146130 0x7f6af406e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:28.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.125+0000 7f6af3e00640 1 -- 172.21.15.86:0/3675496367 >> v1:172.21.15.144:6789/0 conn(0x7f6af40ccdd0 legacy=0x7f6af4071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:28.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.125+0000 7f6af3e00640 1 -- 172.21.15.86:0/3675496367 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6af406ded0 con 0x7f6af4072410 2024-04-20T21:18:28.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.125+0000 7f6af3e00640 1 -- 172.21.15.86:0/3675496367 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f6aec0019f0 con 0x7f6af4072410 2024-04-20T21:18:28.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.125+0000 7f6af3e00640 1 -- 172.21.15.86:0/3675496367 >> v1:172.21.15.86:6789/0 conn(0x7f6af4072410 legacy=0x7f6af415cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:28.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.125+0000 7f6af3e00640 1 --2- 172.21.15.86:0/3675496367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6adc0039e0 0x7f6adc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:28.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.125+0000 7f6af3e00640 1 --2- 172.21.15.86:0/3675496367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6adc004800 0x7f6adc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:28.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.126+0000 7f6af3e00640 1 --2- 172.21.15.86:0/3675496367 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6af4146130 0x7f6adc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:28.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.126+0000 7f6af3e00640 1 -- 172.21.15.86:0/3675496367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6af40cdbc0 con 0x7f6adc004800 2024-04-20T21:18:28.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.126+0000 7f6af3e00640 1 -- 172.21.15.86:0/3675496367 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6adc00bb00 con 0x7f6adc0039e0 2024-04-20T21:18:28.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.126+0000 7f6af3e00640 1 -- 172.21.15.86:0/3675496367 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6adc00bd00 con 0x7f6af4146130 2024-04-20T21:18:28.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.126+0000 7f6af8e00640 1 --2- 172.21.15.86:0/3675496367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6adc004800 0x7f6adc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:28.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.126+0000 7f6af9800640 1 --2- 172.21.15.86:0/3675496367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6adc0039e0 0x7f6adc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:28.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.126+0000 7f6af8e00640 1 -- 172.21.15.86:0/3675496367 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6af4146130 msgr2=0x7f6adc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:28.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.126+0000 7f6af8e00640 1 --2- 172.21.15.86:0/3675496367 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6af4146130 0x7f6adc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:28.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.126+0000 7f6af8e00640 1 -- 172.21.15.86:0/3675496367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6adc0039e0 msgr2=0x7f6adc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:28.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.126+0000 7f6af8e00640 1 --2- 172.21.15.86:0/3675496367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6adc0039e0 0x7f6adc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:28.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.126+0000 7f6af8e00640 1 -- 172.21.15.86:0/3675496367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6adc003520 con 0x7f6adc004800 2024-04-20T21:18:28.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.127+0000 7f6af8e00640 1 --2- 172.21.15.86:0/3675496367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6adc004800 0x7f6adc004c80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0001720 tx=0x7f6adc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=417c063d650cd1c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:28.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.127+0000 7f6af3e00640 1 -- 172.21.15.86:0/3675496367 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0001370 con 0x7f6adc004800 2024-04-20T21:18:28.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.128+0000 7f6af3e00640 1 -- 172.21.15.86:0/3675496367 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ae00052d0 con 0x7f6adc004800 2024-04-20T21:18:28.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.128+0000 7f6af3e00640 1 -- 172.21.15.86:0/3675496367 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0003830 con 0x7f6adc004800 2024-04-20T21:18:28.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.128+0000 7f6afc200640 1 -- 172.21.15.86:0/3675496367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6adc004800 msgr2=0x7f6adc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:28.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.128+0000 7f6afc200640 1 --2- 172.21.15.86:0/3675496367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6adc004800 0x7f6adc004c80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0001720 tx=0x7f6adc005830 comp rx=0 tx=0).stop 2024-04-20T21:18:28.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.128+0000 7f6afa200640 1 -- 172.21.15.86:0/3675496367 reap_dead start 2024-04-20T21:18:28.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.128+0000 7f6afc200640 1 -- 172.21.15.86:0/3675496367 shutdown_connections 2024-04-20T21:18:28.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.128+0000 7f6afc200640 1 -- 172.21.15.86:0/3675496367 >> 172.21.15.86:0/3675496367 conn(0x7f6af40b2360 msgr2=0x7f6af40b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:28.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.128+0000 7f6afc200640 1 -- 172.21.15.86:0/3675496367 shutdown_connections 2024-04-20T21:18:28.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.128+0000 7f6afc200640 1 -- 172.21.15.86:0/3675496367 wait complete. 2024-04-20T21:18:28.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.130+0000 7f6afc200640 1 Processor -- start 2024-04-20T21:18:28.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.130+0000 7f6afc200640 1 -- start start 2024-04-20T21:18:28.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.131+0000 7f6afc200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6adc004800 0x7f6af41400d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:28.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.131+0000 7f6afc200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6af4144590 0x7f6af4144a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:28.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.131+0000 7f6afc200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6af41422f0 0x7f6af4142770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:28.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.131+0000 7f6afc200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6af40cd230 con 0x7f6af41422f0 2024-04-20T21:18:28.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.131+0000 7f6afc200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6af406f150 con 0x7f6adc004800 2024-04-20T21:18:28.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.131+0000 7f6afc200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6af40cd3b0 con 0x7f6af4144590 2024-04-20T21:18:28.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.131+0000 7f6afa200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6af41422f0 0x7f6af4142770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:28.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.131+0000 7f6afa200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6af41422f0 0x7f6af4142770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58944/0 (socket says 172.21.15.86:58944) 2024-04-20T21:18:28.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.131+0000 7f6afa200640 1 -- 172.21.15.86:0/123815044 learned_addr learned my addr 172.21.15.86:0/123815044 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:28.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.132+0000 7f6af9800640 1 --2- 172.21.15.86:0/123815044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6adc004800 0x7f6af41400d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:28.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.132+0000 7f6afa200640 1 -- 172.21.15.86:0/123815044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6af4144590 msgr2=0x7f6af4144a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:28.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.132+0000 7f6af8e00640 1 --2- 172.21.15.86:0/123815044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6af4144590 0x7f6af4144a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:28.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.132+0000 7f6afa200640 1 --2- 172.21.15.86:0/123815044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6af4144590 0x7f6af4144a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:28.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.132+0000 7f6afa200640 1 -- 172.21.15.86:0/123815044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6adc004800 msgr2=0x7f6af41400d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:28.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.132+0000 7f6afa200640 1 --2- 172.21.15.86:0/123815044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6adc004800 0x7f6af41400d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:28.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.132+0000 7f6afa200640 1 -- 172.21.15.86:0/123815044 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6af4143020 con 0x7f6af41422f0 2024-04-20T21:18:28.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.132+0000 7f6af9800640 1 --2- 172.21.15.86:0/123815044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6adc004800 0x7f6af41400d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:28.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.132+0000 7f6afa200640 1 --2- 172.21.15.86:0/123815044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6af41422f0 0x7f6af4142770 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6af4072d50 tx=0x7f6aec00ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:28.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.132+0000 7f6af1600640 1 -- 172.21.15.86:0/123815044 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aec00a5b0 con 0x7f6af41422f0 2024-04-20T21:18:28.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.132+0000 7f6af1600640 1 -- 172.21.15.86:0/123815044 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6aec00a750 con 0x7f6af41422f0 2024-04-20T21:18:28.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.132+0000 7f6afc200640 1 -- 172.21.15.86:0/123815044 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6af4143300 con 0x7f6af41422f0 2024-04-20T21:18:28.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.132+0000 7f6af1600640 1 -- 172.21.15.86:0/123815044 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aec00ab20 con 0x7f6af41422f0 2024-04-20T21:18:28.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.132+0000 7f6afc200640 1 -- 172.21.15.86:0/123815044 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6af40a9720 con 0x7f6af41422f0 2024-04-20T21:18:28.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.134+0000 7f6afc200640 1 -- 172.21.15.86:0/123815044 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6af40689e0 con 0x7f6af41422f0 2024-04-20T21:18:28.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.135+0000 7f6af1600640 1 -- 172.21.15.86:0/123815044 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6aec011070 con 0x7f6af41422f0 2024-04-20T21:18:28.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.136+0000 7f6af1600640 1 --2- 172.21.15.86:0/123815044 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6ad406c920 0x7f6ad406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:28.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.136+0000 7f6af1600640 1 -- 172.21.15.86:0/123815044 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6aec08aeb0 con 0x7f6af41422f0 2024-04-20T21:18:28.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.136+0000 7f6af9800640 1 --2- 172.21.15.86:0/123815044 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6ad406c920 0x7f6ad406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:28.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.137+0000 7f6af9800640 1 --2- 172.21.15.86:0/123815044 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6ad406c920 0x7f6ad406ede0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8009a00 tx=0x7f6ae8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:28.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.140+0000 7f6af1600640 1 -- 172.21.15.86:0/123815044 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6aec019060 con 0x7f6af41422f0 2024-04-20T21:18:28.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.337+0000 7f6afc200640 1 -- 172.21.15.86:0/123815044 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6af40ccbb0 con 0x7f6af41422f0 2024-04-20T21:18:28.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.339+0000 7f6af1600640 1 -- 172.21.15.86:0/123815044 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7f6aec0102c0 con 0x7f6af41422f0 2024-04-20T21:18:28.339 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:18:28.339 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":20,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:17:53.274462+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:17:49.718799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:17:50.683477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:18:28.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.341+0000 7f6afc200640 1 -- 172.21.15.86:0/123815044 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6ad406c920 msgr2=0x7f6ad406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:28.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.341+0000 7f6afc200640 1 --2- 172.21.15.86:0/123815044 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6ad406c920 0x7f6ad406ede0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8009a00 tx=0x7f6ae8008040 comp rx=0 tx=0).stop 2024-04-20T21:18:28.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.341+0000 7f6afc200640 1 -- 172.21.15.86:0/123815044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6af41422f0 msgr2=0x7f6af4142770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:28.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.341+0000 7f6afc200640 1 --2- 172.21.15.86:0/123815044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6af41422f0 0x7f6af4142770 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6af4072d50 tx=0x7f6aec00ccd0 comp rx=0 tx=0).stop 2024-04-20T21:18:28.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.341+0000 7f6afc200640 1 -- 172.21.15.86:0/123815044 shutdown_connections 2024-04-20T21:18:28.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.341+0000 7f6afc200640 1 --2- 172.21.15.86:0/123815044 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6ad406c920 0x7f6ad406ede0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:28.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.342+0000 7f6afc200640 1 --2- 172.21.15.86:0/123815044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6af41422f0 0x7f6af4142770 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:28.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.342+0000 7f6afc200640 1 --2- 172.21.15.86:0/123815044 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6af4144590 0x7f6af4144a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:28.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.342+0000 7f6afc200640 1 --2- 172.21.15.86:0/123815044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6adc004800 0x7f6af41400d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:28.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.342+0000 7f6afc200640 1 -- 172.21.15.86:0/123815044 >> 172.21.15.86:0/123815044 conn(0x7f6af40b2360 msgr2=0x7f6af40b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:28.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.342+0000 7f6afc200640 1 -- 172.21.15.86:0/123815044 shutdown_connections 2024-04-20T21:18:28.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.342+0000 7f6afc200640 1 -- 172.21.15.86:0/123815044 wait complete. 2024-04-20T21:18:28.359 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:17:49.718799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:17:50.683477+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:18:28.360 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:18:28.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.552+0000 7f7ae4600640 1 Processor -- start 2024-04-20T21:18:28.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.552+0000 7f7ae4600640 1 -- start start 2024-04-20T21:18:28.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.554+0000 7f7ae4600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7adc146210 0x7f7adc06e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:28.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.554+0000 7f7ae4600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7adc06bfd0 0x7f7adc0cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:28.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.554+0000 7f7ae4600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7adc15cd40 con 0x7f7adc06bbf0 2024-04-20T21:18:28.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.554+0000 7f7ae4600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f7adc15d000 con 0x7f7adc06bfd0 2024-04-20T21:18:28.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.554+0000 7f7ae4600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7adc15d2e0 con 0x7f7adc146210 2024-04-20T21:18:28.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.554+0000 7f7ae2600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7adc06bfd0 0x7f7adc0cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:28.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.555+0000 7f7ae2600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7adc06bfd0 0x7f7adc0cd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58950/0 (socket says 172.21.15.86:58950) 2024-04-20T21:18:28.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.555+0000 7f7ae2600640 1 -- 172.21.15.86:0/831658201 learned_addr learned my addr 172.21.15.86:0/831658201 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:28.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.555+0000 7f7ae1200640 1 --2- 172.21.15.86:0/831658201 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7adc146210 0x7f7adc06e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:28.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.555+0000 7f7ae2600640 1 -- 172.21.15.86:0/831658201 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7adc146210 msgr2=0x7f7adc06e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:28.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.555+0000 7f7ae2600640 1 --2- 172.21.15.86:0/831658201 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7adc146210 0x7f7adc06e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:28.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.555+0000 7f7ae2600640 1 -- 172.21.15.86:0/831658201 >> v1:172.21.15.144:6789/0 conn(0x7f7adc06bbf0 legacy=0x7f7adc145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:28.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.555+0000 7f7ae2600640 1 -- 172.21.15.86:0/831658201 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7adc15d5c0 con 0x7f7adc06bfd0 2024-04-20T21:18:28.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.556+0000 7f7ae2600640 1 --2- 172.21.15.86:0/831658201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7adc06bfd0 0x7f7adc0cd540 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4009e50 tx=0x7f7ad402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a25549961151bb10 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:28.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.556+0000 7f7adbe00640 1 -- 172.21.15.86:0/831658201 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad402ca70 con 0x7f7adc06bfd0 2024-04-20T21:18:28.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.556+0000 7f7adbe00640 1 -- 172.21.15.86:0/831658201 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ad402cc10 con 0x7f7adc06bfd0 2024-04-20T21:18:28.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.556+0000 7f7adbe00640 1 -- 172.21.15.86:0/831658201 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad40343f0 con 0x7f7adc06bfd0 2024-04-20T21:18:28.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.557+0000 7f7ae4600640 1 -- 172.21.15.86:0/831658201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7adc06bfd0 msgr2=0x7f7adc0cd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:28.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.557+0000 7f7ae4600640 1 --2- 172.21.15.86:0/831658201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7adc06bfd0 0x7f7adc0cd540 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4009e50 tx=0x7f7ad402bed0 comp rx=0 tx=0).stop 2024-04-20T21:18:28.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.557+0000 7f7ae4600640 1 -- 172.21.15.86:0/831658201 shutdown_connections 2024-04-20T21:18:28.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.557+0000 7f7ae4600640 1 --2- 172.21.15.86:0/831658201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7adc06bfd0 0x7f7adc0cd540 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:28.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.557+0000 7f7ae4600640 1 --2- 172.21.15.86:0/831658201 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7adc146210 0x7f7adc06e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:28.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.557+0000 7f7ae4600640 1 -- 172.21.15.86:0/831658201 >> 172.21.15.86:0/831658201 conn(0x7f7adc0b2110 msgr2=0x7f7adc0b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:28.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.557+0000 7f7ae4600640 1 -- 172.21.15.86:0/831658201 shutdown_connections 2024-04-20T21:18:28.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.557+0000 7f7ae4600640 1 -- 172.21.15.86:0/831658201 wait complete. 2024-04-20T21:18:28.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.559+0000 7f7ae4600640 1 Processor -- start 2024-04-20T21:18:28.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.559+0000 7f7ae4600640 1 -- start start 2024-04-20T21:18:28.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.560+0000 7f7ae4600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7adc06bbf0 0x7f7adc0cc750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:28.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.560+0000 7f7ae4600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7adc06bfd0 0x7f7adc0c5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:28.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.560+0000 7f7ae4600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7adc146210 0x7f7adc0c60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:28.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.560+0000 7f7ae4600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7adc0cdc00 con 0x7f7adc146210 2024-04-20T21:18:28.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.560+0000 7f7ae4600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7adc0cda80 con 0x7f7adc06bfd0 2024-04-20T21:18:28.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.560+0000 7f7ae4600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7adc0cdd80 con 0x7f7adc06bbf0 2024-04-20T21:18:28.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.560+0000 7f7ae2600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7adc146210 0x7f7adc0c60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:28.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.560+0000 7f7ae2600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7adc146210 0x7f7adc0c60e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58966/0 (socket says 172.21.15.86:58966) 2024-04-20T21:18:28.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.560+0000 7f7ae2600640 1 -- 172.21.15.86:0/310784048 learned_addr learned my addr 172.21.15.86:0/310784048 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:28.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.561+0000 7f7ae1200640 1 --2- 172.21.15.86:0/310784048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7adc06bfd0 0x7f7adc0c5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:28.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.561+0000 7f7ae1c00640 1 --2- 172.21.15.86:0/310784048 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7adc06bbf0 0x7f7adc0cc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:28.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.561+0000 7f7ae2600640 1 -- 172.21.15.86:0/310784048 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7adc06bbf0 msgr2=0x7f7adc0cc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:28.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.561+0000 7f7ae2600640 1 --2- 172.21.15.86:0/310784048 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7adc06bbf0 0x7f7adc0cc750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:28.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.561+0000 7f7ae2600640 1 -- 172.21.15.86:0/310784048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7adc06bfd0 msgr2=0x7f7adc0c5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:28.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.561+0000 7f7ae2600640 1 --2- 172.21.15.86:0/310784048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7adc06bfd0 0x7f7adc0c5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:28.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.561+0000 7f7ae2600640 1 -- 172.21.15.86:0/310784048 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7adc0ccc90 con 0x7f7adc146210 2024-04-20T21:18:28.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.561+0000 7f7ae1200640 1 --2- 172.21.15.86:0/310784048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7adc06bfd0 0x7f7adc0c5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:28.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.561+0000 7f7ae1c00640 1 --2- 172.21.15.86:0/310784048 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7adc06bbf0 0x7f7adc0cc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:28.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.561+0000 7f7ae2600640 1 --2- 172.21.15.86:0/310784048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7adc146210 0x7f7adc0c60e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4009f70 tx=0x7f7ad4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:28.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.561+0000 7f7ad9600640 1 -- 172.21.15.86:0/310784048 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4043020 con 0x7f7adc146210 2024-04-20T21:18:28.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.561+0000 7f7ad9600640 1 -- 172.21.15.86:0/310784048 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ad403e450 con 0x7f7adc146210 2024-04-20T21:18:28.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.561+0000 7f7ae4600640 1 -- 172.21.15.86:0/310784048 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7adc13a2b0 con 0x7f7adc146210 2024-04-20T21:18:28.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.561+0000 7f7ad9600640 1 -- 172.21.15.86:0/310784048 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4009510 con 0x7f7adc146210 2024-04-20T21:18:28.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.562+0000 7f7ae4600640 1 -- 172.21.15.86:0/310784048 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7adc0ae920 con 0x7f7adc146210 2024-04-20T21:18:28.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.563+0000 7f7ae4600640 1 -- 172.21.15.86:0/310784048 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7adc15fb90 con 0x7f7adc146210 2024-04-20T21:18:28.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.564+0000 7f7ad9600640 1 -- 172.21.15.86:0/310784048 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7ad4046020 con 0x7f7adc146210 2024-04-20T21:18:28.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.565+0000 7f7ad9600640 1 --2- 172.21.15.86:0/310784048 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7ac006c8f0 0x7f7ac006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:28.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.565+0000 7f7ad9600640 1 -- 172.21.15.86:0/310784048 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7ad40ae2f0 con 0x7f7adc146210 2024-04-20T21:18:28.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.565+0000 7f7ae1c00640 1 --2- 172.21.15.86:0/310784048 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7ac006c8f0 0x7f7ac006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:28.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.566+0000 7f7ae1c00640 1 --2- 172.21.15.86:0/310784048 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7ac006c8f0 0x7f7ac006edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0007fd0 tx=0x7f7ad0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:28.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.569+0000 7f7ad9600640 1 -- 172.21.15.86:0/310784048 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7adc15fb90 con 0x7f7adc146210 2024-04-20T21:18:28.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.769+0000 7f7ae4600640 1 -- 172.21.15.86:0/310784048 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7adc0689e0 con 0x7f7adc146210 2024-04-20T21:18:28.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.770+0000 7f7ad9600640 1 -- 172.21.15.86:0/310784048 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f7ad404d030 con 0x7f7adc146210 2024-04-20T21:18:28.769 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:18:28.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.772+0000 7f7ae4600640 1 -- 172.21.15.86:0/310784048 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7ac006c8f0 msgr2=0x7f7ac006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:28.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.772+0000 7f7ae4600640 1 --2- 172.21.15.86:0/310784048 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7ac006c8f0 0x7f7ac006edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0007fd0 tx=0x7f7ad0009040 comp rx=0 tx=0).stop 2024-04-20T21:18:28.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.772+0000 7f7ae4600640 1 -- 172.21.15.86:0/310784048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7adc146210 msgr2=0x7f7adc0c60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:28.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.772+0000 7f7ae4600640 1 --2- 172.21.15.86:0/310784048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7adc146210 0x7f7adc0c60e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4009f70 tx=0x7f7ad4004a40 comp rx=0 tx=0).stop 2024-04-20T21:18:28.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.772+0000 7f7ae4600640 1 -- 172.21.15.86:0/310784048 shutdown_connections 2024-04-20T21:18:28.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.772+0000 7f7ae4600640 1 --2- 172.21.15.86:0/310784048 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7ac006c8f0 0x7f7ac006edb0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:28.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.772+0000 7f7ae4600640 1 --2- 172.21.15.86:0/310784048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7adc146210 0x7f7adc0c60e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:28.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.772+0000 7f7ae4600640 1 --2- 172.21.15.86:0/310784048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7adc06bfd0 0x7f7adc0c5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:28.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.772+0000 7f7ae4600640 1 --2- 172.21.15.86:0/310784048 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7adc06bbf0 0x7f7adc0cc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:28.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.772+0000 7f7ae4600640 1 -- 172.21.15.86:0/310784048 >> 172.21.15.86:0/310784048 conn(0x7f7adc0b2110 msgr2=0x7f7adc071f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:28.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.772+0000 7f7ae4600640 1 -- 172.21.15.86:0/310784048 shutdown_connections 2024-04-20T21:18:28.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.773+0000 7f7ae4600640 1 -- 172.21.15.86:0/310784048 wait complete. 2024-04-20T21:18:28.788 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:18:28.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.979+0000 7f3611000640 1 Processor -- start 2024-04-20T21:18:28.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.979+0000 7f3611000640 1 -- start start 2024-04-20T21:18:28.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.981+0000 7f3611000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f360c155d30 0x7f360c0cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:28.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.981+0000 7f3611000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f360c156110 con 0x7f360c0b52b0 2024-04-20T21:18:28.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.981+0000 7f3611000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f360c0ce460 con 0x7f360c06b980 2024-04-20T21:18:28.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.981+0000 7f3611000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f360c0b56e0 con 0x7f360c155d30 2024-04-20T21:18:28.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.981+0000 7f360a800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f360c0b52b0 0x7f360c06ddf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60408/0 (socket says 172.21.15.86:60408) 2024-04-20T21:18:28.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.981+0000 7f360a800640 1 -- 172.21.15.86:0/1101833342 learned_addr learned my addr 172.21.15.86:0/1101833342 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:18:28.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.981+0000 7f3609e00640 1 --2- 172.21.15.86:0/1101833342 >> v2:172.21.15.144:3300/0 conn(0x7f360c155d30 0x7f360c0cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:28.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.982+0000 7f3609400640 1 -- 172.21.15.86:0/1101833342 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2412126079 0 0) 0x7f360c156110 con 0x7f360c0b52b0 2024-04-20T21:18:28.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.982+0000 7f3609400640 1 -- 172.21.15.86:0/1101833342 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f35f0003520 con 0x7f360c0b52b0 2024-04-20T21:18:28.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.982+0000 7f3609400640 1 -- 172.21.15.86:0/1101833342 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1834989937 0 0) 0x7f35f0003520 con 0x7f360c0b52b0 2024-04-20T21:18:28.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.982+0000 7f3609400640 1 -- 172.21.15.86:0/1101833342 >> v2:172.21.15.144:3300/0 conn(0x7f360c155d30 msgr2=0x7f360c0cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:28.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.982+0000 7f3609400640 1 --2- 172.21.15.86:0/1101833342 >> v2:172.21.15.144:3300/0 conn(0x7f360c155d30 0x7f360c0cf500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:28.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.982+0000 7f3609400640 1 -- 172.21.15.86:0/1101833342 >> v1:172.21.15.144:6789/0 conn(0x7f360c06b980 legacy=0x7f360c0b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:28.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.982+0000 7f3609400640 1 -- 172.21.15.86:0/1101833342 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f360c0ce740 con 0x7f360c0b52b0 2024-04-20T21:18:28.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.982+0000 7f3609400640 1 -- 172.21.15.86:0/1101833342 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f36040019f0 con 0x7f360c0b52b0 2024-04-20T21:18:28.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.982+0000 7f3609400640 1 -- 172.21.15.86:0/1101833342 >> v1:172.21.15.86:6789/0 conn(0x7f360c0b52b0 legacy=0x7f360c06ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:28.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.982+0000 7f3609400640 1 --2- 172.21.15.86:0/1101833342 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35f00039b0 0x7f35f0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:28.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.982+0000 7f3609400640 1 --2- 172.21.15.86:0/1101833342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35f0004800 0x7f35f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:28.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.982+0000 7f3609400640 1 --2- 172.21.15.86:0/1101833342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35f00052e0 0x7f35f000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:28.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.982+0000 7f3609400640 1 -- 172.21.15.86:0/1101833342 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f360c0b56e0 con 0x7f35f0004800 2024-04-20T21:18:28.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.982+0000 7f3609400640 1 -- 172.21.15.86:0/1101833342 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f35f000c1b0 con 0x7f35f00052e0 2024-04-20T21:18:28.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.982+0000 7f3609400640 1 -- 172.21.15.86:0/1101833342 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f35f000c3b0 con 0x7f35f00039b0 2024-04-20T21:18:28.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.983+0000 7f3603e00640 1 --2- 172.21.15.86:0/1101833342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35f0004800 0x7f35f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:28.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.983+0000 7f3603e00640 1 -- 172.21.15.86:0/1101833342 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35f00039b0 msgr2=0x7f35f0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:28.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.983+0000 7f3603e00640 1 --2- 172.21.15.86:0/1101833342 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35f00039b0 0x7f35f0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:28.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.983+0000 7f3603e00640 1 -- 172.21.15.86:0/1101833342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35f00052e0 msgr2=0x7f35f000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:28.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.983+0000 7f3603e00640 1 --2- 172.21.15.86:0/1101833342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35f00052e0 0x7f35f000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:28.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.983+0000 7f360a800640 1 -- 172.21.15.86:0/1101833342 reap_dead start 2024-04-20T21:18:28.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.983+0000 7f3603e00640 1 -- 172.21.15.86:0/1101833342 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35f0003520 con 0x7f35f0004800 2024-04-20T21:18:28.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.984+0000 7f3603e00640 1 --2- 172.21.15.86:0/1101833342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35f0004800 0x7f35f0004c80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f35f4001720 tx=0x7f35f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be15fc71328bdd7b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:28.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.984+0000 7f3609400640 1 -- 172.21.15.86:0/1101833342 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f4001370 con 0x7f35f0004800 2024-04-20T21:18:28.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.984+0000 7f3609400640 1 -- 172.21.15.86:0/1101833342 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35f40052d0 con 0x7f35f0004800 2024-04-20T21:18:28.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.984+0000 7f3609400640 1 -- 172.21.15.86:0/1101833342 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f4003830 con 0x7f35f0004800 2024-04-20T21:18:28.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.985+0000 7f3611000640 1 -- 172.21.15.86:0/1101833342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35f0004800 msgr2=0x7f35f0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:28.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.985+0000 7f3611000640 1 --2- 172.21.15.86:0/1101833342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35f0004800 0x7f35f0004c80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f35f4001720 tx=0x7f35f0005830 comp rx=0 tx=0).stop 2024-04-20T21:18:28.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.985+0000 7f3611000640 1 -- 172.21.15.86:0/1101833342 shutdown_connections 2024-04-20T21:18:28.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.985+0000 7f3611000640 1 --2- 172.21.15.86:0/1101833342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f35f0004800 0x7f35f0004c80 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:28.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.985+0000 7f3611000640 1 -- 172.21.15.86:0/1101833342 >> 172.21.15.86:0/1101833342 conn(0x7f360c0b2110 msgr2=0x7f360c0b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:28.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.985+0000 7f3611000640 1 -- 172.21.15.86:0/1101833342 shutdown_connections 2024-04-20T21:18:28.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.985+0000 7f3611000640 1 -- 172.21.15.86:0/1101833342 wait complete. 2024-04-20T21:18:28.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.987+0000 7f3611000640 1 Processor -- start 2024-04-20T21:18:28.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.987+0000 7f3611000640 1 -- start start 2024-04-20T21:18:28.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.988+0000 7f3611000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35f0004800 0x7f360c140fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:28.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.988+0000 7f3611000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f360c06e360 0x7f360c06e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:28.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.988+0000 7f3611000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f360c15d4f0 0x7f360c15d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:28.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.988+0000 7f3611000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f360c0cefe0 con 0x7f360c15d4f0 2024-04-20T21:18:28.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.988+0000 7f3611000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f360c0ced00 con 0x7f360c06e360 2024-04-20T21:18:28.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.988+0000 7f3611000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f360c0cf500 con 0x7f35f0004800 2024-04-20T21:18:28.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.988+0000 7f360a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f360c15d4f0 0x7f360c15d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:28.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.988+0000 7f360a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f360c15d4f0 0x7f360c15d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58986/0 (socket says 172.21.15.86:58986) 2024-04-20T21:18:28.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.989+0000 7f360a800640 1 -- 172.21.15.86:0/4028528126 learned_addr learned my addr 172.21.15.86:0/4028528126 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:28.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.989+0000 7f3609e00640 1 --2- 172.21.15.86:0/4028528126 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35f0004800 0x7f360c140fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:28.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.989+0000 7f3603e00640 1 --2- 172.21.15.86:0/4028528126 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f360c06e360 0x7f360c06e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:28.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.989+0000 7f360a800640 1 -- 172.21.15.86:0/4028528126 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35f0004800 msgr2=0x7f360c140fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:28.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.989+0000 7f360a800640 1 --2- 172.21.15.86:0/4028528126 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35f0004800 0x7f360c140fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:28.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.989+0000 7f360a800640 1 -- 172.21.15.86:0/4028528126 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f360c06e360 msgr2=0x7f360c06e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:28.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.989+0000 7f360a800640 1 --2- 172.21.15.86:0/4028528126 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f360c06e360 0x7f360c06e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:28.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.989+0000 7f360a800640 1 -- 172.21.15.86:0/4028528126 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f360c15deb0 con 0x7f360c15d4f0 2024-04-20T21:18:28.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.989+0000 7f360a800640 1 --2- 172.21.15.86:0/4028528126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f360c15d4f0 0x7f360c15d970 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3604001560 tx=0x7f360400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:28.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.989+0000 7f3602000640 1 -- 172.21.15.86:0/4028528126 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360400ce70 con 0x7f360c15d4f0 2024-04-20T21:18:28.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.989+0000 7f3602000640 1 -- 172.21.15.86:0/4028528126 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f360400a610 con 0x7f360c15d4f0 2024-04-20T21:18:28.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.989+0000 7f3602000640 1 -- 172.21.15.86:0/4028528126 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360400aa00 con 0x7f360c15d4f0 2024-04-20T21:18:28.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.989+0000 7f3611000640 1 -- 172.21.15.86:0/4028528126 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f360c15e170 con 0x7f360c15d4f0 2024-04-20T21:18:28.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.990+0000 7f3611000640 1 -- 172.21.15.86:0/4028528126 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f360c15e730 con 0x7f360c15d4f0 2024-04-20T21:18:28.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.991+0000 7f3611000640 1 -- 172.21.15.86:0/4028528126 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f360c14a690 con 0x7f360c15d4f0 2024-04-20T21:18:28.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.992+0000 7f3602000640 1 -- 172.21.15.86:0/4028528126 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f360400aba0 con 0x7f360c15d4f0 2024-04-20T21:18:28.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.993+0000 7f3602000640 1 --2- 172.21.15.86:0/4028528126 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f35e806c8f0 0x7f35e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:28.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.993+0000 7f3602000640 1 -- 172.21.15.86:0/4028528126 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f360408b030 con 0x7f360c15d4f0 2024-04-20T21:18:28.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.993+0000 7f3609e00640 1 --2- 172.21.15.86:0/4028528126 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f35e806c8f0 0x7f35e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:28.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.994+0000 7f3609e00640 1 --2- 172.21.15.86:0/4028528126 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f35e806c8f0 0x7f35e806edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f360c06a0f0 tx=0x7f35fc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:28.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:28.997+0000 7f3602000640 1 -- 172.21.15.86:0/4028528126 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f360401a030 con 0x7f360c15d4f0 2024-04-20T21:18:29.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.199+0000 7f3611000640 1 -- 172.21.15.86:0/4028528126 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f360c0689e0 con 0x7f360c15d4f0 2024-04-20T21:18:29.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.200+0000 7f3602000640 1 -- 172.21.15.86:0/4028528126 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f360401b2b0 con 0x7f360c15d4f0 2024-04-20T21:18:29.199 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-20T21:18:29.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.203+0000 7f3611000640 1 -- 172.21.15.86:0/4028528126 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f35e806c8f0 msgr2=0x7f35e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:29.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.203+0000 7f3611000640 1 --2- 172.21.15.86:0/4028528126 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f35e806c8f0 0x7f35e806edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f360c06a0f0 tx=0x7f35fc008040 comp rx=0 tx=0).stop 2024-04-20T21:18:29.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.203+0000 7f3611000640 1 -- 172.21.15.86:0/4028528126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f360c15d4f0 msgr2=0x7f360c15d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:29.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.203+0000 7f3611000640 1 --2- 172.21.15.86:0/4028528126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f360c15d4f0 0x7f360c15d970 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3604001560 tx=0x7f360400cc20 comp rx=0 tx=0).stop 2024-04-20T21:18:29.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.203+0000 7f3611000640 1 -- 172.21.15.86:0/4028528126 shutdown_connections 2024-04-20T21:18:29.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.203+0000 7f3611000640 1 --2- 172.21.15.86:0/4028528126 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f35e806c8f0 0x7f35e806edb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:29.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.203+0000 7f3611000640 1 --2- 172.21.15.86:0/4028528126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f360c15d4f0 0x7f360c15d970 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:29.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.203+0000 7f3611000640 1 --2- 172.21.15.86:0/4028528126 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f360c06e360 0x7f360c06e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:29.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.203+0000 7f3611000640 1 --2- 172.21.15.86:0/4028528126 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35f0004800 0x7f360c140fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:29.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.203+0000 7f3611000640 1 -- 172.21.15.86:0/4028528126 >> 172.21.15.86:0/4028528126 conn(0x7f360c0b2110 msgr2=0x7f360c071db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:29.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.203+0000 7f3611000640 1 -- 172.21.15.86:0/4028528126 shutdown_connections 2024-04-20T21:18:29.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.203+0000 7f3611000640 1 -- 172.21.15.86:0/4028528126 wait complete. 2024-04-20T21:18:29.220 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:18:29.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.421+0000 7f7942800640 1 Processor -- start 2024-04-20T21:18:29.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.421+0000 7f7942800640 1 -- start start 2024-04-20T21:18:29.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.422+0000 7f7942800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f793c15e850 0x7f793c0b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:29.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.422+0000 7f7942800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f793c0b7d30 0x7f793c0d1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:29.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.422+0000 7f7942800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f793c0d3870 con 0x7f793c0d1c70 2024-04-20T21:18:29.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.422+0000 7f7942800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f793c0d20d0 con 0x7f793c15e850 2024-04-20T21:18:29.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.422+0000 7f7942800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f793c0d3d40 con 0x7f793c0b7d30 2024-04-20T21:18:29.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.423+0000 7f793b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f793c15e850 0x7f793c0b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:29.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.423+0000 7f793b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f793c15e850 0x7f793c0b77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59000/0 (socket says 172.21.15.86:59000) 2024-04-20T21:18:29.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.423+0000 7f793b400640 1 -- 172.21.15.86:0/1017224644 learned_addr learned my addr 172.21.15.86:0/1017224644 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:29.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.423+0000 7f793b400640 1 -- 172.21.15.86:0/1017224644 >> v2:172.21.15.144:3300/0 conn(0x7f793c0b7d30 msgr2=0x7f793c0d1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:29.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.423+0000 7f793b400640 1 --2- 172.21.15.86:0/1017224644 >> v2:172.21.15.144:3300/0 conn(0x7f793c0b7d30 0x7f793c0d1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:29.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.423+0000 7f793b400640 1 -- 172.21.15.86:0/1017224644 >> v1:172.21.15.144:6789/0 conn(0x7f793c0d1c70 legacy=0x7f793c0d4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:29.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.423+0000 7f793b400640 1 -- 172.21.15.86:0/1017224644 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f793c0d4020 con 0x7f793c15e850 2024-04-20T21:18:29.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.424+0000 7f793b400640 1 --2- 172.21.15.86:0/1017224644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f793c15e850 0x7f793c0b77f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f793c0b6450 tx=0x7f792c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ee1b1b48b2d6f10 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:29.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.424+0000 7f793a000640 1 -- 172.21.15.86:0/1017224644 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f792c035020 con 0x7f793c15e850 2024-04-20T21:18:29.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.425+0000 7f793a000640 1 -- 172.21.15.86:0/1017224644 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f792c031870 con 0x7f793c15e850 2024-04-20T21:18:29.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.425+0000 7f793a000640 1 -- 172.21.15.86:0/1017224644 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f792c031c40 con 0x7f793c15e850 2024-04-20T21:18:29.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.425+0000 7f7942800640 1 -- 172.21.15.86:0/1017224644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f793c15e850 msgr2=0x7f793c0b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:29.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.425+0000 7f7942800640 1 --2- 172.21.15.86:0/1017224644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f793c15e850 0x7f793c0b77f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f793c0b6450 tx=0x7f792c02b910 comp rx=0 tx=0).stop 2024-04-20T21:18:29.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.425+0000 7f7942800640 1 -- 172.21.15.86:0/1017224644 shutdown_connections 2024-04-20T21:18:29.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.425+0000 7f7942800640 1 --2- 172.21.15.86:0/1017224644 >> v2:172.21.15.144:3300/0 conn(0x7f793c0b7d30 0x7f793c0d1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:29.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.425+0000 7f7942800640 1 --2- 172.21.15.86:0/1017224644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f793c15e850 0x7f793c0b77f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:29.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.425+0000 7f7942800640 1 -- 172.21.15.86:0/1017224644 >> 172.21.15.86:0/1017224644 conn(0x7f793c067ae0 msgr2=0x7f793c067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:29.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.425+0000 7f7942800640 1 -- 172.21.15.86:0/1017224644 shutdown_connections 2024-04-20T21:18:29.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.425+0000 7f7942800640 1 -- 172.21.15.86:0/1017224644 wait complete. 2024-04-20T21:18:29.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.427+0000 7f7942800640 1 Processor -- start 2024-04-20T21:18:29.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.427+0000 7f7942800640 1 -- start start 2024-04-20T21:18:29.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.428+0000 7f7942800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f793c0b7d30 0x7f793c15fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:29.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.428+0000 7f7942800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f793c0d1c70 0x7f793c14fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:29.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.428+0000 7f7942800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f793c15e850 0x7f793c0ab150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:29.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.428+0000 7f7942800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f793c15ec30 con 0x7f793c15e850 2024-04-20T21:18:29.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.428+0000 7f7942800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f793c0d45e0 con 0x7f793c0b7d30 2024-04-20T21:18:29.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.428+0000 7f7942800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f793c0d5210 con 0x7f793c0d1c70 2024-04-20T21:18:29.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.429+0000 7f793be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f793c15e850 0x7f793c0ab150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:29.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.429+0000 7f793be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f793c15e850 0x7f793c0ab150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59012/0 (socket says 172.21.15.86:59012) 2024-04-20T21:18:29.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.429+0000 7f793be00640 1 -- 172.21.15.86:0/3355850000 learned_addr learned my addr 172.21.15.86:0/3355850000 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:29.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.429+0000 7f793b400640 1 --2- 172.21.15.86:0/3355850000 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f793c0b7d30 0x7f793c15fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:29.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.429+0000 7f793aa00640 1 --2- 172.21.15.86:0/3355850000 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f793c0d1c70 0x7f793c14fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:29.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.429+0000 7f793be00640 1 -- 172.21.15.86:0/3355850000 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f793c0d1c70 msgr2=0x7f793c14fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:29.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.429+0000 7f793be00640 1 --2- 172.21.15.86:0/3355850000 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f793c0d1c70 0x7f793c14fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:29.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.429+0000 7f793be00640 1 -- 172.21.15.86:0/3355850000 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f793c0b7d30 msgr2=0x7f793c15fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:29.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.429+0000 7f793be00640 1 --2- 172.21.15.86:0/3355850000 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f793c0b7d30 0x7f793c15fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:29.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.429+0000 7f793be00640 1 -- 172.21.15.86:0/3355850000 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f793c0ab690 con 0x7f793c15e850 2024-04-20T21:18:29.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.429+0000 7f793be00640 1 --2- 172.21.15.86:0/3355850000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f793c15e850 0x7f793c0ab150 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f793400c780 tx=0x7f793400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:29.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.430+0000 7f7933400640 1 -- 172.21.15.86:0/3355850000 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7934017070 con 0x7f793c15e850 2024-04-20T21:18:29.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.430+0000 7f7933400640 1 -- 172.21.15.86:0/3355850000 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7934003da0 con 0x7f793c15e850 2024-04-20T21:18:29.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.430+0000 7f7933400640 1 -- 172.21.15.86:0/3355850000 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79340052b0 con 0x7f793c15e850 2024-04-20T21:18:29.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.430+0000 7f7942800640 1 -- 172.21.15.86:0/3355850000 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f793c161ef0 con 0x7f793c15e850 2024-04-20T21:18:29.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.430+0000 7f7942800640 1 -- 172.21.15.86:0/3355850000 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f793c161410 con 0x7f793c15e850 2024-04-20T21:18:29.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.431+0000 7f7942800640 1 -- 172.21.15.86:0/3355850000 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f793c0690f0 con 0x7f793c15e850 2024-04-20T21:18:29.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.432+0000 7f7933400640 1 -- 172.21.15.86:0/3355850000 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7934005450 con 0x7f793c15e850 2024-04-20T21:18:29.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.433+0000 7f7933400640 1 --2- 172.21.15.86:0/3355850000 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f791806c8f0 0x7f791806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:29.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.433+0000 7f7933400640 1 -- 172.21.15.86:0/3355850000 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f793408b1c0 con 0x7f793c15e850 2024-04-20T21:18:29.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.433+0000 7f793b400640 1 --2- 172.21.15.86:0/3355850000 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f791806c8f0 0x7f791806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:29.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.434+0000 7f793b400640 1 --2- 172.21.15.86:0/3355850000 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f791806c8f0 0x7f791806edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f792c036000 tx=0x7f792c03c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:29.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.438+0000 7f7933400640 1 -- 172.21.15.86:0/3355850000 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f793c0ab690 con 0x7f793c15e850 2024-04-20T21:18:29.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.635+0000 7f7942800640 1 -- 172.21.15.86:0/3355850000 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f793c1602b0 con 0x7f793c15e850 2024-04-20T21:18:29.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.636+0000 7f7933400640 1 -- 172.21.15.86:0/3355850000 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f7934059550 con 0x7f793c15e850 2024-04-20T21:18:29.635 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-20T21:18:29.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.638+0000 7f7942800640 1 -- 172.21.15.86:0/3355850000 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f791806c8f0 msgr2=0x7f791806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:29.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.638+0000 7f7942800640 1 --2- 172.21.15.86:0/3355850000 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f791806c8f0 0x7f791806edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f792c036000 tx=0x7f792c03c040 comp rx=0 tx=0).stop 2024-04-20T21:18:29.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.638+0000 7f7942800640 1 -- 172.21.15.86:0/3355850000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f793c15e850 msgr2=0x7f793c0ab150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:29.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.638+0000 7f7942800640 1 --2- 172.21.15.86:0/3355850000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f793c15e850 0x7f793c0ab150 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f793400c780 tx=0x7f793400cc40 comp rx=0 tx=0).stop 2024-04-20T21:18:29.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.639+0000 7f7942800640 1 -- 172.21.15.86:0/3355850000 shutdown_connections 2024-04-20T21:18:29.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.639+0000 7f7942800640 1 --2- 172.21.15.86:0/3355850000 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f791806c8f0 0x7f791806edb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:29.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.639+0000 7f7942800640 1 --2- 172.21.15.86:0/3355850000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f793c15e850 0x7f793c0ab150 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:29.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.639+0000 7f7942800640 1 --2- 172.21.15.86:0/3355850000 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f793c0d1c70 0x7f793c14fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:29.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.639+0000 7f7942800640 1 --2- 172.21.15.86:0/3355850000 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f793c0b7d30 0x7f793c15fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:29.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.639+0000 7f7942800640 1 -- 172.21.15.86:0/3355850000 >> 172.21.15.86:0/3355850000 conn(0x7f793c067ae0 msgr2=0x7f793c0cb930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:29.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.639+0000 7f7942800640 1 -- 172.21.15.86:0/3355850000 shutdown_connections 2024-04-20T21:18:29.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.639+0000 7f7942800640 1 -- 172.21.15.86:0/3355850000 wait complete. 2024-04-20T21:18:29.656 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:18:29.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.860+0000 7fa493800640 1 Processor -- start 2024-04-20T21:18:29.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.861+0000 7fa493800640 1 -- start start 2024-04-20T21:18:29.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.862+0000 7fa493800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa48c144e90 0x7fa48c0b4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:29.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.862+0000 7fa493800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa48c06f450 0x7fa48c15bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:29.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.862+0000 7fa493800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa48c0cdb50 con 0x7fa48c0cd170 2024-04-20T21:18:29.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.862+0000 7fa493800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fa48c145270 con 0x7fa48c06f450 2024-04-20T21:18:29.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.862+0000 7fa493800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa48c0b3e50 con 0x7fa48c144e90 2024-04-20T21:18:29.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.863+0000 7fa491800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa48c06f450 0x7fa48c15bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:29.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.863+0000 7fa490e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa48c144e90 0x7fa48c0b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:29.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.863+0000 7fa491800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa48c06f450 0x7fa48c15bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.86:43824/0 (socket says 172.21.15.86:43824) 2024-04-20T21:18:29.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.863+0000 7fa491800640 1 -- 172.21.15.86:0/3279535313 learned_addr learned my addr 172.21.15.86:0/3279535313 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:29.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.863+0000 7fa48be00640 1 --1- 172.21.15.86:0/3279535313 >> v1:172.21.15.144:6789/0 conn(0x7fa48c0cd170 0x7fa48c06ed40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.86:48162/0 (socket says 172.21.15.86:48162) 2024-04-20T21:18:29.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.864+0000 7fa491800640 1 -- 172.21.15.86:0/3279535313 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa48c144e90 msgr2=0x7fa48c0b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:29.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.864+0000 7fa491800640 1 --2- 172.21.15.86:0/3279535313 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa48c144e90 0x7fa48c0b4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:29.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.864+0000 7fa491800640 1 -- 172.21.15.86:0/3279535313 >> v1:172.21.15.144:6789/0 conn(0x7fa48c0cd170 legacy=0x7fa48c06ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:29.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.864+0000 7fa491800640 1 -- 172.21.15.86:0/3279535313 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa48c0b4130 con 0x7fa48c06f450 2024-04-20T21:18:29.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.865+0000 7fa491800640 1 --2- 172.21.15.86:0/3279535313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa48c06f450 0x7fa48c15bb50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa484009e50 tx=0x7fa48402bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71ece987a9fd94c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:29.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.866+0000 7fa48b400640 1 -- 172.21.15.86:0/3279535313 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48402cb00 con 0x7fa48c06f450 2024-04-20T21:18:29.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.866+0000 7fa48b400640 1 -- 172.21.15.86:0/3279535313 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa48402cca0 con 0x7fa48c06f450 2024-04-20T21:18:29.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.866+0000 7fa48b400640 1 -- 172.21.15.86:0/3279535313 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa484034610 con 0x7fa48c06f450 2024-04-20T21:18:29.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.866+0000 7fa493800640 1 -- 172.21.15.86:0/3279535313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa48c06f450 msgr2=0x7fa48c15bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:29.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.866+0000 7fa493800640 1 --2- 172.21.15.86:0/3279535313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa48c06f450 0x7fa48c15bb50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa484009e50 tx=0x7fa48402bf60 comp rx=0 tx=0).stop 2024-04-20T21:18:29.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.866+0000 7fa493800640 1 -- 172.21.15.86:0/3279535313 shutdown_connections 2024-04-20T21:18:29.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.866+0000 7fa493800640 1 --2- 172.21.15.86:0/3279535313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa48c06f450 0x7fa48c15bb50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:29.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.866+0000 7fa493800640 1 --2- 172.21.15.86:0/3279535313 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa48c144e90 0x7fa48c0b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:29.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.866+0000 7fa493800640 1 -- 172.21.15.86:0/3279535313 >> 172.21.15.86:0/3279535313 conn(0x7fa48c0b2360 msgr2=0x7fa48c0b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:29.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.866+0000 7fa493800640 1 -- 172.21.15.86:0/3279535313 shutdown_connections 2024-04-20T21:18:29.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.866+0000 7fa493800640 1 -- 172.21.15.86:0/3279535313 wait complete. 2024-04-20T21:18:29.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.868+0000 7fa493800640 1 Processor -- start 2024-04-20T21:18:29.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.868+0000 7fa493800640 1 -- start start 2024-04-20T21:18:29.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.869+0000 7fa493800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa48c06f450 0x7fa48c1e26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:29.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.869+0000 7fa493800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa48c0cd170 0x7fa48c1e5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:29.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.869+0000 7fa493800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa48c144e90 0x7fa48c15ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:29.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.869+0000 7fa493800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c0d0010 con 0x7fa48c144e90 2024-04-20T21:18:29.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.869+0000 7fa493800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c0b4e70 con 0x7fa48c06f450 2024-04-20T21:18:29.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.869+0000 7fa493800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa48c0d0190 con 0x7fa48c0cd170 2024-04-20T21:18:29.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.870+0000 7fa491800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa48c144e90 0x7fa48c15ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:29.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.870+0000 7fa491800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa48c144e90 0x7fa48c15ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59026/0 (socket says 172.21.15.86:59026) 2024-04-20T21:18:29.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.870+0000 7fa491800640 1 -- 172.21.15.86:0/2090130887 learned_addr learned my addr 172.21.15.86:0/2090130887 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:29.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.870+0000 7fa490e00640 1 --2- 172.21.15.86:0/2090130887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa48c06f450 0x7fa48c1e26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:29.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.870+0000 7fa491800640 1 -- 172.21.15.86:0/2090130887 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa48c0cd170 msgr2=0x7fa48c1e5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:29.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.870+0000 7fa48be00640 1 --2- 172.21.15.86:0/2090130887 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa48c0cd170 0x7fa48c1e5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:29.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.870+0000 7fa491800640 1 --2- 172.21.15.86:0/2090130887 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa48c0cd170 0x7fa48c1e5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:29.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.870+0000 7fa491800640 1 -- 172.21.15.86:0/2090130887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa48c06f450 msgr2=0x7fa48c1e26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:29.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.870+0000 7fa491800640 1 --2- 172.21.15.86:0/2090130887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa48c06f450 0x7fa48c1e26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:29.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.870+0000 7fa491800640 1 -- 172.21.15.86:0/2090130887 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa48c130e10 con 0x7fa48c144e90 2024-04-20T21:18:29.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.871+0000 7fa491800640 1 --2- 172.21.15.86:0/2090130887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa48c144e90 0x7fa48c15ad50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa484036040 tx=0x7fa484004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:29.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.871+0000 7fa488c00640 1 -- 172.21.15.86:0/2090130887 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa484043020 con 0x7fa48c144e90 2024-04-20T21:18:29.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.871+0000 7fa488c00640 1 -- 172.21.15.86:0/2090130887 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa48402ce50 con 0x7fa48c144e90 2024-04-20T21:18:29.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.871+0000 7fa493800640 1 -- 172.21.15.86:0/2090130887 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa48c1310f0 con 0x7fa48c144e90 2024-04-20T21:18:29.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.871+0000 7fa488c00640 1 -- 172.21.15.86:0/2090130887 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa484009510 con 0x7fa48c144e90 2024-04-20T21:18:29.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.871+0000 7fa493800640 1 -- 172.21.15.86:0/2090130887 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa48c1316b0 con 0x7fa48c144e90 2024-04-20T21:18:29.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.872+0000 7fa493800640 1 -- 172.21.15.86:0/2090130887 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa48c0689e0 con 0x7fa48c144e90 2024-04-20T21:18:29.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.873+0000 7fa488c00640 1 -- 172.21.15.86:0/2090130887 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa484046020 con 0x7fa48c144e90 2024-04-20T21:18:29.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.875+0000 7fa488c00640 1 --2- 172.21.15.86:0/2090130887 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa46c06c8f0 0x7fa46c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:29.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.875+0000 7fa488c00640 1 -- 172.21.15.86:0/2090130887 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa4840ae090 con 0x7fa48c144e90 2024-04-20T21:18:29.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.875+0000 7fa490e00640 1 --2- 172.21.15.86:0/2090130887 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa46c06c8f0 0x7fa46c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:29.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.876+0000 7fa490e00640 1 --2- 172.21.15.86:0/2090130887 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa46c06c8f0 0x7fa46c06edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa480009a30 tx=0x7fa480008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:29.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:29.879+0000 7fa488c00640 1 -- 172.21.15.86:0/2090130887 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa48404d030 con 0x7fa48c144e90 2024-04-20T21:18:30.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.081+0000 7fa493800640 1 -- 172.21.15.86:0/2090130887 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa48c06cce0 con 0x7fa48c144e90 2024-04-20T21:18:30.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.083+0000 7fa488c00640 1 -- 172.21.15.86:0/2090130887 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7fa48407c550 con 0x7fa48c144e90 2024-04-20T21:18:30.083 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:18:30.083 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":20,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:17:53.274462+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:17:49.718799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:17:50.683477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:18:30.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.085+0000 7fa493800640 1 -- 172.21.15.86:0/2090130887 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa46c06c8f0 msgr2=0x7fa46c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:30.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.085+0000 7fa493800640 1 --2- 172.21.15.86:0/2090130887 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa46c06c8f0 0x7fa46c06edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa480009a30 tx=0x7fa480008040 comp rx=0 tx=0).stop 2024-04-20T21:18:30.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.085+0000 7fa493800640 1 -- 172.21.15.86:0/2090130887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa48c144e90 msgr2=0x7fa48c15ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:30.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.085+0000 7fa493800640 1 --2- 172.21.15.86:0/2090130887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa48c144e90 0x7fa48c15ad50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa484036040 tx=0x7fa484004a40 comp rx=0 tx=0).stop 2024-04-20T21:18:30.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.085+0000 7fa493800640 1 -- 172.21.15.86:0/2090130887 shutdown_connections 2024-04-20T21:18:30.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.085+0000 7fa493800640 1 --2- 172.21.15.86:0/2090130887 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa46c06c8f0 0x7fa46c06edb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:30.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.085+0000 7fa493800640 1 --2- 172.21.15.86:0/2090130887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa48c144e90 0x7fa48c15ad50 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:30.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.085+0000 7fa493800640 1 --2- 172.21.15.86:0/2090130887 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa48c0cd170 0x7fa48c1e5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:30.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.085+0000 7fa493800640 1 --2- 172.21.15.86:0/2090130887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa48c06f450 0x7fa48c1e26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:30.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.086+0000 7fa493800640 1 -- 172.21.15.86:0/2090130887 >> 172.21.15.86:0/2090130887 conn(0x7fa48c0b2360 msgr2=0x7fa48c0b00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:30.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.086+0000 7fa493800640 1 -- 172.21.15.86:0/2090130887 shutdown_connections 2024-04-20T21:18:30.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.086+0000 7fa493800640 1 -- 172.21.15.86:0/2090130887 wait complete. 2024-04-20T21:18:30.104 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:17:49.718799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:17:50.683477+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:18:30.104 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:18:30.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.311+0000 7f9215a00640 1 Processor -- start 2024-04-20T21:18:30.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.312+0000 7f9215a00640 1 -- start start 2024-04-20T21:18:30.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.313+0000 7f9215a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f921015eb30 0x7f92100b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:30.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.313+0000 7f9215a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f921006c090 0x7f92100b4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:30.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.313+0000 7f9215a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92100adb10 con 0x7f92100aee50 2024-04-20T21:18:30.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.313+0000 7f9215a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f92100af7d0 con 0x7f921006c090 2024-04-20T21:18:30.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.313+0000 7f9215a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f92100addb0 con 0x7f921015eb30 2024-04-20T21:18:30.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.314+0000 7f920f400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f921006c090 0x7f92100b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:30.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.314+0000 7f920f400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f921006c090 0x7f92100b4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.86:43842/0 (socket says 172.21.15.86:43842) 2024-04-20T21:18:30.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.314+0000 7f920f400640 1 -- 172.21.15.86:0/2824198400 learned_addr learned my addr 172.21.15.86:0/2824198400 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:30.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.314+0000 7f920ea00640 1 --2- 172.21.15.86:0/2824198400 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f921015eb30 0x7f92100b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:30.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.315+0000 7f920f400640 1 -- 172.21.15.86:0/2824198400 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f921015eb30 msgr2=0x7f92100b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:30.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.315+0000 7f920f400640 1 --2- 172.21.15.86:0/2824198400 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f921015eb30 0x7f92100b3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:30.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.315+0000 7f920f400640 1 -- 172.21.15.86:0/2824198400 >> v1:172.21.15.144:6789/0 conn(0x7f92100aee50 legacy=0x7f921006b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:30.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.315+0000 7f920f400640 1 -- 172.21.15.86:0/2824198400 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92100ae090 con 0x7f921006c090 2024-04-20T21:18:30.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.316+0000 7f920f400640 1 --2- 172.21.15.86:0/2824198400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f921006c090 0x7f92100b4360 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9208009e50 tx=0x7f920802bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83fc9bd1eac391fc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:30.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.317+0000 7f920d600640 1 -- 172.21.15.86:0/2824198400 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920802cb00 con 0x7f921006c090 2024-04-20T21:18:30.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.317+0000 7f920d600640 1 -- 172.21.15.86:0/2824198400 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f920802cca0 con 0x7f921006c090 2024-04-20T21:18:30.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.317+0000 7f920d600640 1 -- 172.21.15.86:0/2824198400 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9208034610 con 0x7f921006c090 2024-04-20T21:18:30.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.318+0000 7f9215a00640 1 -- 172.21.15.86:0/2824198400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f921006c090 msgr2=0x7f92100b4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:30.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.318+0000 7f9215a00640 1 --2- 172.21.15.86:0/2824198400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f921006c090 0x7f92100b4360 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9208009e50 tx=0x7f920802bf60 comp rx=0 tx=0).stop 2024-04-20T21:18:30.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.318+0000 7f9215a00640 1 -- 172.21.15.86:0/2824198400 shutdown_connections 2024-04-20T21:18:30.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.318+0000 7f9215a00640 1 --2- 172.21.15.86:0/2824198400 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f921006c090 0x7f92100b4360 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:30.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.318+0000 7f9215a00640 1 --2- 172.21.15.86:0/2824198400 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f921015eb30 0x7f92100b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:30.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.318+0000 7f9215a00640 1 -- 172.21.15.86:0/2824198400 >> 172.21.15.86:0/2824198400 conn(0x7f92100b2110 msgr2=0x7f92100b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:30.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.318+0000 7f9215a00640 1 -- 172.21.15.86:0/2824198400 shutdown_connections 2024-04-20T21:18:30.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.318+0000 7f9215a00640 1 -- 172.21.15.86:0/2824198400 wait complete. 2024-04-20T21:18:30.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.320+0000 7f9215a00640 1 Processor -- start 2024-04-20T21:18:30.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.320+0000 7f9215a00640 1 -- start start 2024-04-20T21:18:30.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.321+0000 7f9215a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f921006c090 0x7f9210155b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:30.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.321+0000 7f9215a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92100aee50 0x7f9210145300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:30.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.321+0000 7f9215a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f921015eb30 0x7f9210146b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:30.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.321+0000 7f9215a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f92100ae930 con 0x7f92100aee50 2024-04-20T21:18:30.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.321+0000 7f9215a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f92100ae650 con 0x7f921015eb30 2024-04-20T21:18:30.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.321+0000 7f9215a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f921006e160 con 0x7f921006c090 2024-04-20T21:18:30.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.322+0000 7f920e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92100aee50 0x7f9210145300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:30.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.322+0000 7f920e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92100aee50 0x7f9210145300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59032/0 (socket says 172.21.15.86:59032) 2024-04-20T21:18:30.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.322+0000 7f920e000640 1 -- 172.21.15.86:0/850882022 learned_addr learned my addr 172.21.15.86:0/850882022 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:30.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.322+0000 7f920ea00640 1 --2- 172.21.15.86:0/850882022 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f921006c090 0x7f9210155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:30.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.322+0000 7f920f400640 1 --2- 172.21.15.86:0/850882022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f921015eb30 0x7f9210146b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:30.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.322+0000 7f920e000640 1 -- 172.21.15.86:0/850882022 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f921006c090 msgr2=0x7f9210155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:30.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.322+0000 7f920e000640 1 --2- 172.21.15.86:0/850882022 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f921006c090 0x7f9210155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:30.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.322+0000 7f920e000640 1 -- 172.21.15.86:0/850882022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f921015eb30 msgr2=0x7f9210146b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:30.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.322+0000 7f920e000640 1 --2- 172.21.15.86:0/850882022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f921015eb30 0x7f9210146b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:30.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.322+0000 7f920e000640 1 -- 172.21.15.86:0/850882022 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9210145840 con 0x7f92100aee50 2024-04-20T21:18:30.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.323+0000 7f920e000640 1 --2- 172.21.15.86:0/850882022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92100aee50 0x7f9210145300 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f91fc00c400 tx=0x7f91fc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:30.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.323+0000 7f9206a00640 1 -- 172.21.15.86:0/850882022 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91fc0052d0 con 0x7f92100aee50 2024-04-20T21:18:30.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.323+0000 7f9206a00640 1 -- 172.21.15.86:0/850882022 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91fc005490 con 0x7f92100aee50 2024-04-20T21:18:30.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.323+0000 7f9206a00640 1 -- 172.21.15.86:0/850882022 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91fc007260 con 0x7f92100aee50 2024-04-20T21:18:30.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.323+0000 7f9215a00640 1 -- 172.21.15.86:0/850882022 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9210145ac0 con 0x7f92100aee50 2024-04-20T21:18:30.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.323+0000 7f9215a00640 1 -- 172.21.15.86:0/850882022 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9210146080 con 0x7f92100aee50 2024-04-20T21:18:30.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.325+0000 7f9215a00640 1 -- 172.21.15.86:0/850882022 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92100689e0 con 0x7f92100aee50 2024-04-20T21:18:30.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.326+0000 7f9206a00640 1 -- 172.21.15.86:0/850882022 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f91fc007400 con 0x7f92100aee50 2024-04-20T21:18:30.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.327+0000 7f9206a00640 1 --2- 172.21.15.86:0/850882022 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f91f006c920 0x7f91f006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:30.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.327+0000 7f9206a00640 1 -- 172.21.15.86:0/850882022 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f91fc08b060 con 0x7f92100aee50 2024-04-20T21:18:30.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.327+0000 7f920ea00640 1 --2- 172.21.15.86:0/850882022 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f91f006c920 0x7f91f006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:30.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.328+0000 7f920ea00640 1 --2- 172.21.15.86:0/850882022 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f91f006c920 0x7f91f006ede0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9210156300 tx=0x7f9200008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:30.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.331+0000 7f9206a00640 1 -- 172.21.15.86:0/850882022 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9210145840 con 0x7f92100aee50 2024-04-20T21:18:30.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.531+0000 7f9215a00640 1 -- 172.21.15.86:0/850882022 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f92100d0ed0 con 0x7f92100aee50 2024-04-20T21:18:30.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.532+0000 7f9206a00640 1 -- 172.21.15.86:0/850882022 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f91fc059490 con 0x7f92100aee50 2024-04-20T21:18:30.531 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:18:30.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.534+0000 7f9215a00640 1 -- 172.21.15.86:0/850882022 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f91f006c920 msgr2=0x7f91f006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:30.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.534+0000 7f9215a00640 1 --2- 172.21.15.86:0/850882022 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f91f006c920 0x7f91f006ede0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9210156300 tx=0x7f9200008040 comp rx=0 tx=0).stop 2024-04-20T21:18:30.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.534+0000 7f9215a00640 1 -- 172.21.15.86:0/850882022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92100aee50 msgr2=0x7f9210145300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:30.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.534+0000 7f9215a00640 1 --2- 172.21.15.86:0/850882022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92100aee50 0x7f9210145300 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f91fc00c400 tx=0x7f91fc00c8c0 comp rx=0 tx=0).stop 2024-04-20T21:18:30.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.534+0000 7f9215a00640 1 -- 172.21.15.86:0/850882022 shutdown_connections 2024-04-20T21:18:30.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.534+0000 7f9215a00640 1 --2- 172.21.15.86:0/850882022 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f91f006c920 0x7f91f006ede0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:30.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.534+0000 7f9215a00640 1 --2- 172.21.15.86:0/850882022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f921015eb30 0x7f9210146b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:30.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.534+0000 7f9215a00640 1 --2- 172.21.15.86:0/850882022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92100aee50 0x7f9210145300 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:30.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.535+0000 7f9215a00640 1 --2- 172.21.15.86:0/850882022 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f921006c090 0x7f9210155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:30.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.535+0000 7f9215a00640 1 -- 172.21.15.86:0/850882022 >> 172.21.15.86:0/850882022 conn(0x7f92100b2110 msgr2=0x7f92100d0420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:30.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.535+0000 7f9215a00640 1 -- 172.21.15.86:0/850882022 shutdown_connections 2024-04-20T21:18:30.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.535+0000 7f9215a00640 1 -- 172.21.15.86:0/850882022 wait complete. 2024-04-20T21:18:30.552 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:18:30.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.756+0000 7fca1d600640 1 Processor -- start 2024-04-20T21:18:30.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.756+0000 7fca1d600640 1 -- start start 2024-04-20T21:18:30.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.758+0000 7fca1d600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fca18071d40 0x7fca180ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:30.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.758+0000 7fca1d600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca18162dd0 0x7fca180b3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:30.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.758+0000 7fca1d600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca180b43a0 con 0x7fca18072120 2024-04-20T21:18:30.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.758+0000 7fca1d600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fca180ce6f0 con 0x7fca18071d40 2024-04-20T21:18:30.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.758+0000 7fca1d600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fca180cf550 con 0x7fca18162dd0 2024-04-20T21:18:30.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.758+0000 7fca1cc00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fca18072120 0x7fca180b4b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60424/0 (socket says 172.21.15.86:60424) 2024-04-20T21:18:30.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.758+0000 7fca1cc00640 1 -- 172.21.15.86:0/3224677964 learned_addr learned my addr 172.21.15.86:0/3224677964 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:18:30.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.758+0000 7fca17400640 1 --2- 172.21.15.86:0/3224677964 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca18162dd0 0x7fca180b3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:30.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.758+0000 7fca17e00640 1 --2- 172.21.15.86:0/3224677964 >> v2:172.21.15.144:3300/0 conn(0x7fca18071d40 0x7fca180ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:30.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.759+0000 7fca16a00640 1 -- 172.21.15.86:0/3224677964 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3518772789 0 0) 0x7fca180b43a0 con 0x7fca18072120 2024-04-20T21:18:30.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.759+0000 7fca16a00640 1 -- 172.21.15.86:0/3224677964 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc9fc003520 con 0x7fca18072120 2024-04-20T21:18:30.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.759+0000 7fca16a00640 1 -- 172.21.15.86:0/3224677964 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1648487045 0 0) 0x7fc9fc003520 con 0x7fca18072120 2024-04-20T21:18:30.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.759+0000 7fca16a00640 1 -- 172.21.15.86:0/3224677964 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca18162dd0 msgr2=0x7fca180b3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:30.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.759+0000 7fca16a00640 1 --2- 172.21.15.86:0/3224677964 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca18162dd0 0x7fca180b3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:30.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.759+0000 7fca16a00640 1 -- 172.21.15.86:0/3224677964 >> v2:172.21.15.144:3300/0 conn(0x7fca18071d40 msgr2=0x7fca180ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:30.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.759+0000 7fca16a00640 1 --2- 172.21.15.86:0/3224677964 >> v2:172.21.15.144:3300/0 conn(0x7fca18071d40 0x7fca180ce1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:30.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.759+0000 7fca16a00640 1 -- 172.21.15.86:0/3224677964 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca180ce940 con 0x7fca18072120 2024-04-20T21:18:30.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.759+0000 7fca16a00640 1 -- 172.21.15.86:0/3224677964 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fca100019f0 con 0x7fca18072120 2024-04-20T21:18:30.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.759+0000 7fca16a00640 1 -- 172.21.15.86:0/3224677964 >> v1:172.21.15.86:6789/0 conn(0x7fca18072120 legacy=0x7fca180b4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:30.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.759+0000 7fca16a00640 1 --2- 172.21.15.86:0/3224677964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9fc003a20 0x7fc9fc003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:30.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.759+0000 7fca16a00640 1 --2- 172.21.15.86:0/3224677964 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca18162dd0 0x7fc9fc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:30.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.759+0000 7fca16a00640 1 --2- 172.21.15.86:0/3224677964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9fc004d40 0x7fc9fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:30.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.759+0000 7fca16a00640 1 -- 172.21.15.86:0/3224677964 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fca180ce6f0 con 0x7fc9fc004d40 2024-04-20T21:18:30.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.759+0000 7fca16a00640 1 -- 172.21.15.86:0/3224677964 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fca180cf550 con 0x7fc9fc003a20 2024-04-20T21:18:30.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.759+0000 7fca16a00640 1 -- 172.21.15.86:0/3224677964 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc9fc00b9c0 con 0x7fca18162dd0 2024-04-20T21:18:30.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.759+0000 7fca1cc00640 1 --2- 172.21.15.86:0/3224677964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9fc004d40 0x7fc9fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:30.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.760+0000 7fca17400640 1 --2- 172.21.15.86:0/3224677964 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca18162dd0 0x7fc9fc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:30.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.760+0000 7fca17e00640 1 --2- 172.21.15.86:0/3224677964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9fc003a20 0x7fc9fc003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:30.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.760+0000 7fca1cc00640 1 -- 172.21.15.86:0/3224677964 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca18162dd0 msgr2=0x7fc9fc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:30.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.760+0000 7fca1cc00640 1 --2- 172.21.15.86:0/3224677964 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca18162dd0 0x7fc9fc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:30.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.760+0000 7fca1cc00640 1 -- 172.21.15.86:0/3224677964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9fc003a20 msgr2=0x7fc9fc003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:30.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.760+0000 7fca1cc00640 1 --2- 172.21.15.86:0/3224677964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9fc003a20 0x7fc9fc003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:30.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.760+0000 7fca1cc00640 1 -- 172.21.15.86:0/3224677964 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9fc003520 con 0x7fc9fc004d40 2024-04-20T21:18:30.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.761+0000 7fca1cc00640 1 --2- 172.21.15.86:0/3224677964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9fc004d40 0x7fc9fc0051c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fca1000a8a0 tx=0x7fc9fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72c6ae9e58cb534b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:30.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.761+0000 7fca16a00640 1 -- 172.21.15.86:0/3224677964 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1000a4c0 con 0x7fc9fc004d40 2024-04-20T21:18:30.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.761+0000 7fca16a00640 1 -- 172.21.15.86:0/3224677964 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca10034650 con 0x7fc9fc004d40 2024-04-20T21:18:30.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.761+0000 7fca16a00640 1 -- 172.21.15.86:0/3224677964 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca10034a40 con 0x7fc9fc004d40 2024-04-20T21:18:30.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.762+0000 7fca1d600640 1 -- 172.21.15.86:0/3224677964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9fc004d40 msgr2=0x7fc9fc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:30.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.762+0000 7fca1d600640 1 --2- 172.21.15.86:0/3224677964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9fc004d40 0x7fc9fc0051c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fca1000a8a0 tx=0x7fc9fc005830 comp rx=0 tx=0).stop 2024-04-20T21:18:30.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.762+0000 7fca1cc00640 1 -- 172.21.15.86:0/3224677964 reap_dead start 2024-04-20T21:18:30.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.762+0000 7fca1d600640 1 -- 172.21.15.86:0/3224677964 shutdown_connections 2024-04-20T21:18:30.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.762+0000 7fca1d600640 1 -- 172.21.15.86:0/3224677964 >> 172.21.15.86:0/3224677964 conn(0x7fca18067cf0 msgr2=0x7fca1806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:30.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.762+0000 7fca1d600640 1 -- 172.21.15.86:0/3224677964 shutdown_connections 2024-04-20T21:18:30.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.762+0000 7fca1d600640 1 -- 172.21.15.86:0/3224677964 wait complete. 2024-04-20T21:18:30.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.764+0000 7fca1d600640 1 Processor -- start 2024-04-20T21:18:30.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.764+0000 7fca1d600640 1 -- start start 2024-04-20T21:18:30.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.765+0000 7fca1d600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9fc004d40 0x7fca1814b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:30.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.765+0000 7fca1d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca181f8790 0x7fca18156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:30.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.765+0000 7fca1d600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca181570d0 0x7fca1815aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:30.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.765+0000 7fca1d600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fca180cf1e0 con 0x7fca181f8790 2024-04-20T21:18:30.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.765+0000 7fca1d600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fca180cef00 con 0x7fc9fc004d40 2024-04-20T21:18:30.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.765+0000 7fca1d600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fca180725d0 con 0x7fca181570d0 2024-04-20T21:18:30.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.765+0000 7fca17400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca181f8790 0x7fca18156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:30.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.765+0000 7fca17400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca181f8790 0x7fca18156b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59056/0 (socket says 172.21.15.86:59056) 2024-04-20T21:18:30.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.765+0000 7fca17400640 1 -- 172.21.15.86:0/2213994714 learned_addr learned my addr 172.21.15.86:0/2213994714 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:30.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.765+0000 7fca17e00640 1 --2- 172.21.15.86:0/2213994714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9fc004d40 0x7fca1814b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:30.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.766+0000 7fca1cc00640 1 --2- 172.21.15.86:0/2213994714 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca181570d0 0x7fca1815aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:30.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.766+0000 7fca17400640 1 -- 172.21.15.86:0/2213994714 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca181570d0 msgr2=0x7fca1815aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:30.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.766+0000 7fca17400640 1 --2- 172.21.15.86:0/2213994714 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca181570d0 0x7fca1815aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:30.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.766+0000 7fca17400640 1 -- 172.21.15.86:0/2213994714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9fc004d40 msgr2=0x7fca1814b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:30.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.766+0000 7fca17400640 1 --2- 172.21.15.86:0/2213994714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9fc004d40 0x7fca1814b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:30.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.766+0000 7fca17400640 1 -- 172.21.15.86:0/2213994714 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca181546a0 con 0x7fca181f8790 2024-04-20T21:18:30.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.766+0000 7fca1cc00640 1 --2- 172.21.15.86:0/2213994714 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca181570d0 0x7fca1815aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:30.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.766+0000 7fca17400640 1 --2- 172.21.15.86:0/2213994714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca181f8790 0x7fca18156b90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fca040115d0 tx=0x7fca04011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:30.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.767+0000 7fca0fe00640 1 -- 172.21.15.86:0/2213994714 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca04015870 con 0x7fca181f8790 2024-04-20T21:18:30.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.767+0000 7fca0fe00640 1 -- 172.21.15.86:0/2213994714 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca04015a10 con 0x7fca181f8790 2024-04-20T21:18:30.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.767+0000 7fca0fe00640 1 -- 172.21.15.86:0/2213994714 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca04015e00 con 0x7fca181f8790 2024-04-20T21:18:30.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.767+0000 7fca1d600640 1 -- 172.21.15.86:0/2213994714 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca18154980 con 0x7fca181f8790 2024-04-20T21:18:30.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.767+0000 7fca1d600640 1 -- 172.21.15.86:0/2213994714 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca18154f40 con 0x7fca181f8790 2024-04-20T21:18:30.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.768+0000 7fca1d600640 1 -- 172.21.15.86:0/2213994714 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca1804fd90 con 0x7fca181f8790 2024-04-20T21:18:30.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.769+0000 7fca0fe00640 1 -- 172.21.15.86:0/2213994714 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fca0401d400 con 0x7fca181f8790 2024-04-20T21:18:30.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.770+0000 7fca0fe00640 1 --2- 172.21.15.86:0/2213994714 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc9f806c920 0x7fc9f806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:30.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.770+0000 7fca0fe00640 1 -- 172.21.15.86:0/2213994714 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fca0408c490 con 0x7fca181f8790 2024-04-20T21:18:30.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.770+0000 7fca17e00640 1 --2- 172.21.15.86:0/2213994714 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc9f806c920 0x7fc9f806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:30.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.771+0000 7fca17e00640 1 --2- 172.21.15.86:0/2213994714 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc9f806c920 0x7fc9f806ede0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fca08009a10 tx=0x7fca08008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:30.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.774+0000 7fca0fe00640 1 -- 172.21.15.86:0/2213994714 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fca04019030 con 0x7fca181f8790 2024-04-20T21:18:30.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.974+0000 7fca1d600640 1 -- 172.21.15.86:0/2213994714 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fca181574b0 con 0x7fca181f8790 2024-04-20T21:18:30.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.975+0000 7fca0fe00640 1 -- 172.21.15.86:0/2213994714 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fca0405a820 con 0x7fca181f8790 2024-04-20T21:18:30.974 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-20T21:18:30.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.977+0000 7fca1d600640 1 -- 172.21.15.86:0/2213994714 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc9f806c920 msgr2=0x7fc9f806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:30.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.977+0000 7fca1d600640 1 --2- 172.21.15.86:0/2213994714 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc9f806c920 0x7fc9f806ede0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fca08009a10 tx=0x7fca08008040 comp rx=0 tx=0).stop 2024-04-20T21:18:30.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.977+0000 7fca1d600640 1 -- 172.21.15.86:0/2213994714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca181f8790 msgr2=0x7fca18156b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:30.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.977+0000 7fca1d600640 1 --2- 172.21.15.86:0/2213994714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca181f8790 0x7fca18156b90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fca040115d0 tx=0x7fca04011a90 comp rx=0 tx=0).stop 2024-04-20T21:18:30.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.978+0000 7fca1d600640 1 -- 172.21.15.86:0/2213994714 shutdown_connections 2024-04-20T21:18:30.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.978+0000 7fca1d600640 1 --2- 172.21.15.86:0/2213994714 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc9f806c920 0x7fc9f806ede0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:30.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.978+0000 7fca1d600640 1 --2- 172.21.15.86:0/2213994714 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca181570d0 0x7fca1815aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:30.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.978+0000 7fca1d600640 1 --2- 172.21.15.86:0/2213994714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca181f8790 0x7fca18156b90 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:30.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.978+0000 7fca1d600640 1 --2- 172.21.15.86:0/2213994714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9fc004d40 0x7fca1814b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:30.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.978+0000 7fca1d600640 1 -- 172.21.15.86:0/2213994714 >> 172.21.15.86:0/2213994714 conn(0x7fca18067cf0 msgr2=0x7fca1806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:30.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.978+0000 7fca1d600640 1 -- 172.21.15.86:0/2213994714 shutdown_connections 2024-04-20T21:18:30.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:30.978+0000 7fca1d600640 1 -- 172.21.15.86:0/2213994714 wait complete. 2024-04-20T21:18:30.996 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:18:31.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.200+0000 7fcc3b600640 1 Processor -- start 2024-04-20T21:18:31.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.201+0000 7fcc3b600640 1 -- start start 2024-04-20T21:18:31.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.202+0000 7fcc3b600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcc340d1970 0x7fcc340cf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:31.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.202+0000 7fcc3b600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc34163970 con 0x7fcc3415ee00 2024-04-20T21:18:31.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.202+0000 7fcc3b600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc340cff90 con 0x7fcc341634c0 2024-04-20T21:18:31.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.202+0000 7fcc3b600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fcc340cfd70 con 0x7fcc340d1970 2024-04-20T21:18:31.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.202+0000 7fcc39600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcc340d1970 0x7fcc340cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:31.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.202+0000 7fcc38c00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fcc3415ee00 0x7fcc3415fe40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60432/0 (socket says 172.21.15.86:60432) 2024-04-20T21:18:31.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.202+0000 7fcc38c00640 1 -- 172.21.15.86:0/4169888065 learned_addr learned my addr 172.21.15.86:0/4169888065 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:18:31.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.203+0000 7fcc33400640 1 -- 172.21.15.86:0/4169888065 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4228573215 0 0) 0x7fcc24001490 con 0x7fcc3415ee00 2024-04-20T21:18:31.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.203+0000 7fcc39600640 1 -- 172.21.15.86:0/4169888065 >> v1:172.21.15.144:6789/0 conn(0x7fcc341634c0 legacy=0x7fcc340d1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:31.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.203+0000 7fcc39600640 1 -- 172.21.15.86:0/4169888065 >> v1:172.21.15.86:6789/0 conn(0x7fcc3415ee00 legacy=0x7fcc3415fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:31.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.203+0000 7fcc39600640 1 -- 172.21.15.86:0/4169888065 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc340d0270 con 0x7fcc340d1970 2024-04-20T21:18:31.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.204+0000 7fcc39600640 1 --2- 172.21.15.86:0/4169888065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc340d1970 0x7fcc340cf830 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c0097a0 tx=0x7fcc2c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3cbd657479d8169 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:31.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.204+0000 7fcc33400640 1 -- 172.21.15.86:0/4169888065 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c035020 con 0x7fcc340d1970 2024-04-20T21:18:31.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.204+0000 7fcc33400640 1 -- 172.21.15.86:0/4169888065 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc2c031830 con 0x7fcc340d1970 2024-04-20T21:18:31.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.204+0000 7fcc33400640 1 -- 172.21.15.86:0/4169888065 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c031c00 con 0x7fcc340d1970 2024-04-20T21:18:31.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.205+0000 7fcc3b600640 1 -- 172.21.15.86:0/4169888065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc340d1970 msgr2=0x7fcc340cf830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:31.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.205+0000 7fcc3b600640 1 --2- 172.21.15.86:0/4169888065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc340d1970 0x7fcc340cf830 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c0097a0 tx=0x7fcc2c02b8d0 comp rx=0 tx=0).stop 2024-04-20T21:18:31.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.205+0000 7fcc3b600640 1 -- 172.21.15.86:0/4169888065 shutdown_connections 2024-04-20T21:18:31.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.205+0000 7fcc3b600640 1 --2- 172.21.15.86:0/4169888065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc340d1970 0x7fcc340cf830 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:31.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.205+0000 7fcc3b600640 1 -- 172.21.15.86:0/4169888065 >> 172.21.15.86:0/4169888065 conn(0x7fcc3406cc20 msgr2=0x7fcc3406d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:31.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.205+0000 7fcc3b600640 1 -- 172.21.15.86:0/4169888065 shutdown_connections 2024-04-20T21:18:31.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.205+0000 7fcc3b600640 1 -- 172.21.15.86:0/4169888065 wait complete. 2024-04-20T21:18:31.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.207+0000 7fcc3b600640 1 Processor -- start 2024-04-20T21:18:31.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.207+0000 7fcc3b600640 1 -- start start 2024-04-20T21:18:31.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.208+0000 7fcc3b600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc340d1970 0x7fcc340ad7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:31.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.208+0000 7fcc3b600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc3415ee00 0x7fcc340add20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:31.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.208+0000 7fcc3b600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc341634c0 0x7fcc3414bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:31.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.208+0000 7fcc3b600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3415f1e0 con 0x7fcc341634c0 2024-04-20T21:18:31.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.208+0000 7fcc3b600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcc340d0550 con 0x7fcc3415ee00 2024-04-20T21:18:31.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.208+0000 7fcc3b600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fcc3415f360 con 0x7fcc340d1970 2024-04-20T21:18:31.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.208+0000 7fcc39600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc341634c0 0x7fcc3414bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:31.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.208+0000 7fcc39600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc341634c0 0x7fcc3414bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59072/0 (socket says 172.21.15.86:59072) 2024-04-20T21:18:31.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.208+0000 7fcc39600640 1 -- 172.21.15.86:0/1679039085 learned_addr learned my addr 172.21.15.86:0/1679039085 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:31.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.209+0000 7fcc39600640 1 -- 172.21.15.86:0/1679039085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc340d1970 msgr2=0x7fcc340ad7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:31.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.209+0000 7fcc39600640 1 --2- 172.21.15.86:0/1679039085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc340d1970 0x7fcc340ad7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:31.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.209+0000 7fcc33e00640 1 --2- 172.21.15.86:0/1679039085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc3415ee00 0x7fcc340add20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:31.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.209+0000 7fcc39600640 1 -- 172.21.15.86:0/1679039085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc3415ee00 msgr2=0x7fcc340add20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:31.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.209+0000 7fcc39600640 1 --2- 172.21.15.86:0/1679039085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc3415ee00 0x7fcc340add20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:31.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.209+0000 7fcc39600640 1 -- 172.21.15.86:0/1679039085 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc340ce320 con 0x7fcc341634c0 2024-04-20T21:18:31.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.209+0000 7fcc39600640 1 --2- 172.21.15.86:0/1679039085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc341634c0 0x7fcc3414bc20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c0098c0 tx=0x7fcc2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:31.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.209+0000 7fcc30c00640 1 -- 172.21.15.86:0/1679039085 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c03b940 con 0x7fcc341634c0 2024-04-20T21:18:31.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.209+0000 7fcc30c00640 1 -- 172.21.15.86:0/1679039085 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc2c03bae0 con 0x7fcc341634c0 2024-04-20T21:18:31.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.209+0000 7fcc3b600640 1 -- 172.21.15.86:0/1679039085 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc34149d90 con 0x7fcc341634c0 2024-04-20T21:18:31.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.209+0000 7fcc30c00640 1 -- 172.21.15.86:0/1679039085 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c0453f0 con 0x7fcc341634c0 2024-04-20T21:18:31.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.209+0000 7fcc3b600640 1 -- 172.21.15.86:0/1679039085 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc3414c160 con 0x7fcc341634c0 2024-04-20T21:18:31.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.211+0000 7fcc3b600640 1 -- 172.21.15.86:0/1679039085 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc340689e0 con 0x7fcc341634c0 2024-04-20T21:18:31.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.211+0000 7fcc30c00640 1 -- 172.21.15.86:0/1679039085 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcc2c038020 con 0x7fcc341634c0 2024-04-20T21:18:31.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.212+0000 7fcc30c00640 1 --2- 172.21.15.86:0/1679039085 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcc1406c920 0x7fcc1406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:31.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.212+0000 7fcc30c00640 1 -- 172.21.15.86:0/1679039085 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcc2c0ad960 con 0x7fcc341634c0 2024-04-20T21:18:31.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.213+0000 7fcc38c00640 1 --2- 172.21.15.86:0/1679039085 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcc1406c920 0x7fcc1406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:31.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.213+0000 7fcc38c00640 1 --2- 172.21.15.86:0/1679039085 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcc1406c920 0x7fcc1406ede0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcc24007fd0 tx=0x7fcc24001690 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:31.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.217+0000 7fcc30c00640 1 -- 172.21.15.86:0/1679039085 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fcc2c040070 con 0x7fcc341634c0 2024-04-20T21:18:31.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.418+0000 7fcc3b600640 1 -- 172.21.15.86:0/1679039085 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcc340d29c0 con 0x7fcc341634c0 2024-04-20T21:18:31.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.418+0000 7fcc30c00640 1 -- 172.21.15.86:0/1679039085 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fcc2c07bdc0 con 0x7fcc341634c0 2024-04-20T21:18:31.418 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-20T21:18:31.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.421+0000 7fcc3b600640 1 -- 172.21.15.86:0/1679039085 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcc1406c920 msgr2=0x7fcc1406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:31.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.421+0000 7fcc3b600640 1 --2- 172.21.15.86:0/1679039085 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcc1406c920 0x7fcc1406ede0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcc24007fd0 tx=0x7fcc24001690 comp rx=0 tx=0).stop 2024-04-20T21:18:31.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.421+0000 7fcc3b600640 1 -- 172.21.15.86:0/1679039085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc341634c0 msgr2=0x7fcc3414bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:31.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.421+0000 7fcc3b600640 1 --2- 172.21.15.86:0/1679039085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc341634c0 0x7fcc3414bc20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c0098c0 tx=0x7fcc2c0046c0 comp rx=0 tx=0).stop 2024-04-20T21:18:31.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.421+0000 7fcc3b600640 1 -- 172.21.15.86:0/1679039085 shutdown_connections 2024-04-20T21:18:31.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.421+0000 7fcc3b600640 1 --2- 172.21.15.86:0/1679039085 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcc1406c920 0x7fcc1406ede0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:31.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.421+0000 7fcc3b600640 1 --2- 172.21.15.86:0/1679039085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc341634c0 0x7fcc3414bc20 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:31.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.421+0000 7fcc3b600640 1 --2- 172.21.15.86:0/1679039085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc3415ee00 0x7fcc340add20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:31.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.421+0000 7fcc3b600640 1 --2- 172.21.15.86:0/1679039085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc340d1970 0x7fcc340ad7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:31.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.421+0000 7fcc3b600640 1 -- 172.21.15.86:0/1679039085 >> 172.21.15.86:0/1679039085 conn(0x7fcc3406cc20 msgr2=0x7fcc3415d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:31.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.421+0000 7fcc3b600640 1 -- 172.21.15.86:0/1679039085 shutdown_connections 2024-04-20T21:18:31.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.421+0000 7fcc3b600640 1 -- 172.21.15.86:0/1679039085 wait complete. 2024-04-20T21:18:31.466 INFO:tasks.cephfs_test_runner:Starting test: test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-20T21:18:31.466 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:18:31.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.665+0000 7ff357200640 1 Processor -- start 2024-04-20T21:18:31.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.665+0000 7ff357200640 1 -- start start 2024-04-20T21:18:31.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.666+0000 7ff357200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff3580b3810 0x7ff358141f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:31.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.666+0000 7ff357200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff358162c40 0x7ff358072990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:31.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.667+0000 7ff357200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff358071c10 con 0x7ff3580af8d0 2024-04-20T21:18:31.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.667+0000 7ff357200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff358142490 con 0x7ff3580b3810 2024-04-20T21:18:31.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.667+0000 7ff357200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7ff35815eaf0 con 0x7ff358162c40 2024-04-20T21:18:31.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.667+0000 7ff355400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff3580b3810 0x7ff358141f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:31.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.667+0000 7ff355e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7ff3580af8d0 0x7ff3580b3100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60436/0 (socket says 172.21.15.86:60436) 2024-04-20T21:18:31.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.667+0000 7ff355e00640 1 -- 172.21.15.86:0/1421667065 learned_addr learned my addr 172.21.15.86:0/1421667065 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:18:31.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.667+0000 7ff356800640 1 --2- 172.21.15.86:0/1421667065 >> v2:172.21.15.144:3300/0 conn(0x7ff358162c40 0x7ff358072990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:31.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.667+0000 7ff355400640 1 -- 172.21.15.86:0/1421667065 >> v2:172.21.15.144:3300/0 conn(0x7ff358162c40 msgr2=0x7ff358072990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:31.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.667+0000 7ff355400640 1 --2- 172.21.15.86:0/1421667065 >> v2:172.21.15.144:3300/0 conn(0x7ff358162c40 0x7ff358072990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:31.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.667+0000 7ff355400640 1 -- 172.21.15.86:0/1421667065 >> v1:172.21.15.86:6789/0 conn(0x7ff3580af8d0 legacy=0x7ff3580b3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:31.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.667+0000 7ff355400640 1 -- 172.21.15.86:0/1421667065 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff358071eb0 con 0x7ff3580b3810 2024-04-20T21:18:31.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.667+0000 7ff354a00640 1 -- 172.21.15.86:0/1421667065 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3967852676 0 0) 0x7ff358071c10 con 0x7ff3580af8d0 2024-04-20T21:18:31.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.667+0000 7ff356800640 1 --2- 172.21.15.86:0/1421667065 >> v2:172.21.15.144:3300/0 conn(0x7ff358162c40 0x7ff358072990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:31.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.669+0000 7ff355400640 1 --2- 172.21.15.86:0/1421667065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3580b3810 0x7ff358141f50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff358141e20 tx=0x7ff34402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=605ffe34fad603c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:31.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.669+0000 7ff354a00640 1 -- 172.21.15.86:0/1421667065 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3440315a0 con 0x7ff3580b3810 2024-04-20T21:18:31.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.669+0000 7ff354a00640 1 -- 172.21.15.86:0/1421667065 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff344037070 con 0x7ff3580b3810 2024-04-20T21:18:31.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.669+0000 7ff354a00640 1 -- 172.21.15.86:0/1421667065 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff344031960 con 0x7ff3580b3810 2024-04-20T21:18:31.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.669+0000 7ff357200640 1 -- 172.21.15.86:0/1421667065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3580b3810 msgr2=0x7ff358141f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:31.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.669+0000 7ff357200640 1 --2- 172.21.15.86:0/1421667065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3580b3810 0x7ff358141f50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff358141e20 tx=0x7ff34402b840 comp rx=0 tx=0).stop 2024-04-20T21:18:31.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.669+0000 7ff357200640 1 -- 172.21.15.86:0/1421667065 shutdown_connections 2024-04-20T21:18:31.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.669+0000 7ff357200640 1 --2- 172.21.15.86:0/1421667065 >> v2:172.21.15.144:3300/0 conn(0x7ff358162c40 0x7ff358072990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:31.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.669+0000 7ff357200640 1 --2- 172.21.15.86:0/1421667065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3580b3810 0x7ff358141f50 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:31.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.669+0000 7ff357200640 1 -- 172.21.15.86:0/1421667065 >> 172.21.15.86:0/1421667065 conn(0x7ff3580d0170 msgr2=0x7ff3580d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:31.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.670+0000 7ff357200640 1 -- 172.21.15.86:0/1421667065 shutdown_connections 2024-04-20T21:18:31.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.670+0000 7ff357200640 1 -- 172.21.15.86:0/1421667065 wait complete. 2024-04-20T21:18:31.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.671+0000 7ff357200640 1 Processor -- start 2024-04-20T21:18:31.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.671+0000 7ff357200640 1 -- start start 2024-04-20T21:18:31.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.672+0000 7ff357200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3580af8d0 0x7ff358160a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:31.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.672+0000 7ff357200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff3580b3810 0x7ff358099a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:31.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.672+0000 7ff357200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff358162c40 0x7ff35815f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:31.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.672+0000 7ff357200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff358163100 con 0x7ff3580af8d0 2024-04-20T21:18:31.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.672+0000 7ff357200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff358072470 con 0x7ff358162c40 2024-04-20T21:18:31.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.672+0000 7ff357200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff358148050 con 0x7ff3580b3810 2024-04-20T21:18:31.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.673+0000 7ff355e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3580af8d0 0x7ff358160a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:31.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.673+0000 7ff355e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3580af8d0 0x7ff358160a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59088/0 (socket says 172.21.15.86:59088) 2024-04-20T21:18:31.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.673+0000 7ff355e00640 1 -- 172.21.15.86:0/2512041148 learned_addr learned my addr 172.21.15.86:0/2512041148 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:31.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.673+0000 7ff355400640 1 --2- 172.21.15.86:0/2512041148 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff3580b3810 0x7ff358099a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:31.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.673+0000 7ff355e00640 1 -- 172.21.15.86:0/2512041148 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff3580b3810 msgr2=0x7ff358099a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:31.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.673+0000 7ff356800640 1 --2- 172.21.15.86:0/2512041148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff358162c40 0x7ff35815f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:31.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.673+0000 7ff355e00640 1 --2- 172.21.15.86:0/2512041148 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff3580b3810 0x7ff358099a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:31.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.673+0000 7ff355e00640 1 -- 172.21.15.86:0/2512041148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff358162c40 msgr2=0x7ff35815f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:31.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.673+0000 7ff355e00640 1 --2- 172.21.15.86:0/2512041148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff358162c40 0x7ff35815f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:31.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.673+0000 7ff355e00640 1 -- 172.21.15.86:0/2512041148 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff35815fdb0 con 0x7ff3580af8d0 2024-04-20T21:18:31.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.673+0000 7ff355400640 1 --2- 172.21.15.86:0/2512041148 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff3580b3810 0x7ff358099a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:31.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.673+0000 7ff355e00640 1 --2- 172.21.15.86:0/2512041148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3580af8d0 0x7ff358160a70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff34800c4d0 tx=0x7ff34800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:31.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.674+0000 7ff34e000640 1 -- 172.21.15.86:0/2512041148 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff348017070 con 0x7ff3580af8d0 2024-04-20T21:18:31.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.674+0000 7ff34e000640 1 -- 172.21.15.86:0/2512041148 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff348005d00 con 0x7ff3580af8d0 2024-04-20T21:18:31.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.674+0000 7ff34e000640 1 -- 172.21.15.86:0/2512041148 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3480072b0 con 0x7ff3580af8d0 2024-04-20T21:18:31.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.674+0000 7ff357200640 1 -- 172.21.15.86:0/2512041148 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff358161080 con 0x7ff3580af8d0 2024-04-20T21:18:31.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.674+0000 7ff357200640 1 -- 172.21.15.86:0/2512041148 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3580ad350 con 0x7ff3580af8d0 2024-04-20T21:18:31.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.675+0000 7ff357200640 1 -- 172.21.15.86:0/2512041148 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3580694b0 con 0x7ff3580af8d0 2024-04-20T21:18:31.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.676+0000 7ff34e000640 1 -- 172.21.15.86:0/2512041148 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff34801a070 con 0x7ff3580af8d0 2024-04-20T21:18:31.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.677+0000 7ff34e000640 1 --2- 172.21.15.86:0/2512041148 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff33806c8f0 0x7ff33806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:31.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.678+0000 7ff34e000640 1 -- 172.21.15.86:0/2512041148 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff34808afe0 con 0x7ff3580af8d0 2024-04-20T21:18:31.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.678+0000 7ff355400640 1 --2- 172.21.15.86:0/2512041148 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff33806c8f0 0x7ff33806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:31.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.678+0000 7ff355400640 1 --2- 172.21.15.86:0/2512041148 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff33806c8f0 0x7ff33806edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff344036000 tx=0x7ff344008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:31.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.682+0000 7ff34e000640 1 -- 172.21.15.86:0/2512041148 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff3580694b0 con 0x7ff3580af8d0 2024-04-20T21:18:31.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.876+0000 7ff357200640 1 -- 172.21.15.86:0/2512041148 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff3580730c0 con 0x7ff3580af8d0 2024-04-20T21:18:31.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.879+0000 7ff34e000640 1 -- 172.21.15.86:0/2512041148 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16605 (secure 0 0 0) 0x7ff3580730c0 con 0x7ff3580af8d0 2024-04-20T21:18:31.878 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:18:31.878 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":20,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:17:53.274462+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:17:49.718799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:17:50.683477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:18:31.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.881+0000 7ff357200640 1 -- 172.21.15.86:0/2512041148 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff33806c8f0 msgr2=0x7ff33806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:31.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.881+0000 7ff357200640 1 --2- 172.21.15.86:0/2512041148 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff33806c8f0 0x7ff33806edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff344036000 tx=0x7ff344008030 comp rx=0 tx=0).stop 2024-04-20T21:18:31.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.881+0000 7ff357200640 1 -- 172.21.15.86:0/2512041148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3580af8d0 msgr2=0x7ff358160a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:31.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.881+0000 7ff357200640 1 --2- 172.21.15.86:0/2512041148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3580af8d0 0x7ff358160a70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff34800c4d0 tx=0x7ff34800c990 comp rx=0 tx=0).stop 2024-04-20T21:18:31.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.881+0000 7ff357200640 1 -- 172.21.15.86:0/2512041148 shutdown_connections 2024-04-20T21:18:31.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.881+0000 7ff357200640 1 --2- 172.21.15.86:0/2512041148 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff33806c8f0 0x7ff33806edb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:31.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.881+0000 7ff357200640 1 --2- 172.21.15.86:0/2512041148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff358162c40 0x7ff35815f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:31.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.881+0000 7ff357200640 1 --2- 172.21.15.86:0/2512041148 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff3580b3810 0x7ff358099a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:31.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.881+0000 7ff357200640 1 --2- 172.21.15.86:0/2512041148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3580af8d0 0x7ff358160a70 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:31.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.881+0000 7ff357200640 1 -- 172.21.15.86:0/2512041148 >> 172.21.15.86:0/2512041148 conn(0x7ff3580d0170 msgr2=0x7ff3580733a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:31.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.881+0000 7ff357200640 1 -- 172.21.15.86:0/2512041148 shutdown_connections 2024-04-20T21:18:31.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:31.881+0000 7ff357200640 1 -- 172.21.15.86:0/2512041148 wait complete. 2024-04-20T21:18:31.898 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:17:49.718799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:17:50.683477+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:18:31.898 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:18:32.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.094+0000 7fb35a800640 1 Processor -- start 2024-04-20T21:18:32.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.094+0000 7fb35a800640 1 -- start start 2024-04-20T21:18:32.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.095+0000 7fb35a800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb3540af1f0 0x7fb35406ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:32.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.095+0000 7fb35a800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb35406d9c0 0x7fb35415d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:32.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.095+0000 7fb35a800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb35415da60 con 0x7fb3540af5d0 2024-04-20T21:18:32.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.096+0000 7fb35a800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb35406f460 con 0x7fb3540af1f0 2024-04-20T21:18:32.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.096+0000 7fb35a800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fb35406de80 con 0x7fb35406d9c0 2024-04-20T21:18:32.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.096+0000 7fb359400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb3540af1f0 0x7fb35406ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:32.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.096+0000 7fb359400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb3540af1f0 0x7fb35406ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59100/0 (socket says 172.21.15.86:59100) 2024-04-20T21:18:32.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.096+0000 7fb359400640 1 -- 172.21.15.86:0/130702488 learned_addr learned my addr 172.21.15.86:0/130702488 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:32.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.096+0000 7fb358a00640 1 --2- 172.21.15.86:0/130702488 >> v2:172.21.15.144:3300/0 conn(0x7fb35406d9c0 0x7fb35415d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:32.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.096+0000 7fb359400640 1 -- 172.21.15.86:0/130702488 >> v2:172.21.15.144:3300/0 conn(0x7fb35406d9c0 msgr2=0x7fb35415d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:32.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.096+0000 7fb359400640 1 --2- 172.21.15.86:0/130702488 >> v2:172.21.15.144:3300/0 conn(0x7fb35406d9c0 0x7fb35415d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:32.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.096+0000 7fb359400640 1 -- 172.21.15.86:0/130702488 >> v1:172.21.15.86:6789/0 conn(0x7fb3540af5d0 legacy=0x7fb35415de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:32.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.096+0000 7fb359400640 1 -- 172.21.15.86:0/130702488 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb35406e160 con 0x7fb3540af1f0 2024-04-20T21:18:32.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.096+0000 7fb353e00640 1 -- 172.21.15.86:0/130702488 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3983227833 0 0) 0x7fb34c001490 con 0x7fb3540af5d0 2024-04-20T21:18:32.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.097+0000 7fb358a00640 1 --2- 172.21.15.86:0/130702488 >> v2:172.21.15.144:3300/0 conn(0x7fb35406d9c0 0x7fb35415d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:32.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.098+0000 7fb359400640 1 --2- 172.21.15.86:0/130702488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3540af1f0 0x7fb35406ef20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb3540d1380 tx=0x7fb34802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b737190827d29ac server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:32.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.098+0000 7fb353e00640 1 -- 172.21.15.86:0/130702488 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb348034070 con 0x7fb3540af1f0 2024-04-20T21:18:32.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.098+0000 7fb353e00640 1 -- 172.21.15.86:0/130702488 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb348030860 con 0x7fb3540af1f0 2024-04-20T21:18:32.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.098+0000 7fb353e00640 1 -- 172.21.15.86:0/130702488 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb348030c30 con 0x7fb3540af1f0 2024-04-20T21:18:32.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.098+0000 7fb35a800640 1 -- 172.21.15.86:0/130702488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3540af1f0 msgr2=0x7fb35406ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:32.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.098+0000 7fb35a800640 1 --2- 172.21.15.86:0/130702488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3540af1f0 0x7fb35406ef20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb3540d1380 tx=0x7fb34802b9a0 comp rx=0 tx=0).stop 2024-04-20T21:18:32.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.098+0000 7fb35a800640 1 -- 172.21.15.86:0/130702488 shutdown_connections 2024-04-20T21:18:32.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.099+0000 7fb35a800640 1 --2- 172.21.15.86:0/130702488 >> v2:172.21.15.144:3300/0 conn(0x7fb35406d9c0 0x7fb35415d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:32.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.099+0000 7fb35a800640 1 --2- 172.21.15.86:0/130702488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3540af1f0 0x7fb35406ef20 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:32.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.099+0000 7fb35a800640 1 -- 172.21.15.86:0/130702488 >> 172.21.15.86:0/130702488 conn(0x7fb3540b1f00 msgr2=0x7fb3540b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:32.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.099+0000 7fb35a800640 1 -- 172.21.15.86:0/130702488 shutdown_connections 2024-04-20T21:18:32.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.099+0000 7fb35a800640 1 -- 172.21.15.86:0/130702488 wait complete. 2024-04-20T21:18:32.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.100+0000 7fb35a800640 1 Processor -- start 2024-04-20T21:18:32.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.100+0000 7fb35a800640 1 -- start start 2024-04-20T21:18:32.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.101+0000 7fb35a800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb35406d9c0 0x7fb354158160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:32.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.101+0000 7fb35a800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3540af1f0 0x7fb354157770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:32.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.101+0000 7fb35a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3540af5d0 0x7fb3541f3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:32.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.101+0000 7fb35a800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb35406ea00 con 0x7fb3540af5d0 2024-04-20T21:18:32.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.101+0000 7fb35a800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb35406e720 con 0x7fb3540af1f0 2024-04-20T21:18:32.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.101+0000 7fb35a800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb3540afa40 con 0x7fb35406d9c0 2024-04-20T21:18:32.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.102+0000 7fb359e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3540af5d0 0x7fb3541f3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:32.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.102+0000 7fb359e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3540af5d0 0x7fb3541f3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59114/0 (socket says 172.21.15.86:59114) 2024-04-20T21:18:32.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.102+0000 7fb359e00640 1 -- 172.21.15.86:0/1576388883 learned_addr learned my addr 172.21.15.86:0/1576388883 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:32.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.102+0000 7fb359400640 1 --2- 172.21.15.86:0/1576388883 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb35406d9c0 0x7fb354158160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:32.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.102+0000 7fb359e00640 1 -- 172.21.15.86:0/1576388883 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb35406d9c0 msgr2=0x7fb354158160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:32.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.102+0000 7fb358a00640 1 --2- 172.21.15.86:0/1576388883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3540af1f0 0x7fb354157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:32.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.102+0000 7fb359e00640 1 --2- 172.21.15.86:0/1576388883 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb35406d9c0 0x7fb354158160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:32.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.102+0000 7fb359e00640 1 -- 172.21.15.86:0/1576388883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3540af1f0 msgr2=0x7fb354157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:32.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.102+0000 7fb359e00640 1 --2- 172.21.15.86:0/1576388883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3540af1f0 0x7fb354157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:32.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.102+0000 7fb359e00640 1 -- 172.21.15.86:0/1576388883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb354159810 con 0x7fb3540af5d0 2024-04-20T21:18:32.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.102+0000 7fb359400640 1 --2- 172.21.15.86:0/1576388883 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb35406d9c0 0x7fb354158160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:32.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.102+0000 7fb358a00640 1 --2- 172.21.15.86:0/1576388883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3540af1f0 0x7fb354157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:32.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.102+0000 7fb359e00640 1 --2- 172.21.15.86:0/1576388883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3540af5d0 0x7fb3541f3fb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb34c00c7c0 tx=0x7fb34c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:32.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.103+0000 7fb351600640 1 -- 172.21.15.86:0/1576388883 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34c003da0 con 0x7fb3540af5d0 2024-04-20T21:18:32.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.103+0000 7fb351600640 1 -- 172.21.15.86:0/1576388883 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb34c005070 con 0x7fb3540af5d0 2024-04-20T21:18:32.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.103+0000 7fb351600640 1 -- 172.21.15.86:0/1576388883 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34c005460 con 0x7fb3540af5d0 2024-04-20T21:18:32.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.103+0000 7fb35a800640 1 -- 172.21.15.86:0/1576388883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb354158a20 con 0x7fb3540af5d0 2024-04-20T21:18:32.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.103+0000 7fb35a800640 1 -- 172.21.15.86:0/1576388883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb354158fe0 con 0x7fb3540af5d0 2024-04-20T21:18:32.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.104+0000 7fb35a800640 1 -- 172.21.15.86:0/1576388883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3540adbc0 con 0x7fb3540af5d0 2024-04-20T21:18:32.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.105+0000 7fb351600640 1 -- 172.21.15.86:0/1576388883 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb34c005600 con 0x7fb3540af5d0 2024-04-20T21:18:32.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.106+0000 7fb351600640 1 --2- 172.21.15.86:0/1576388883 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb33406c8f0 0x7fb33406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:32.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.106+0000 7fb351600640 1 -- 172.21.15.86:0/1576388883 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb34c08bab0 con 0x7fb3540af5d0 2024-04-20T21:18:32.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.106+0000 7fb359400640 1 --2- 172.21.15.86:0/1576388883 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb33406c8f0 0x7fb33406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:32.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.107+0000 7fb359400640 1 --2- 172.21.15.86:0/1576388883 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb33406c8f0 0x7fb33406edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb348036000 tx=0x7fb34803b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:32.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.110+0000 7fb351600640 1 -- 172.21.15.86:0/1576388883 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb3540adbc0 con 0x7fb3540af5d0 2024-04-20T21:18:32.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.312+0000 7fb35a800640 1 -- 172.21.15.86:0/1576388883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb354156430 con 0x7fb3540af5d0 2024-04-20T21:18:32.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.313+0000 7fb351600640 1 -- 172.21.15.86:0/1576388883 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fb35415da60 con 0x7fb3540af5d0 2024-04-20T21:18:32.312 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:18:32.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.316+0000 7fb35a800640 1 -- 172.21.15.86:0/1576388883 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb33406c8f0 msgr2=0x7fb33406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:32.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.316+0000 7fb35a800640 1 --2- 172.21.15.86:0/1576388883 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb33406c8f0 0x7fb33406edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb348036000 tx=0x7fb34803b040 comp rx=0 tx=0).stop 2024-04-20T21:18:32.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.316+0000 7fb35a800640 1 -- 172.21.15.86:0/1576388883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3540af5d0 msgr2=0x7fb3541f3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:32.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.316+0000 7fb35a800640 1 --2- 172.21.15.86:0/1576388883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3540af5d0 0x7fb3541f3fb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb34c00c7c0 tx=0x7fb34c00cc80 comp rx=0 tx=0).stop 2024-04-20T21:18:32.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.317+0000 7fb35a800640 1 -- 172.21.15.86:0/1576388883 shutdown_connections 2024-04-20T21:18:32.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.317+0000 7fb35a800640 1 --2- 172.21.15.86:0/1576388883 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb33406c8f0 0x7fb33406edb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:32.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.317+0000 7fb35a800640 1 --2- 172.21.15.86:0/1576388883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3540af5d0 0x7fb3541f3fb0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:32.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.317+0000 7fb35a800640 1 --2- 172.21.15.86:0/1576388883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb3540af1f0 0x7fb354157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:32.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.317+0000 7fb35a800640 1 --2- 172.21.15.86:0/1576388883 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb35406d9c0 0x7fb354158160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:32.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.317+0000 7fb35a800640 1 -- 172.21.15.86:0/1576388883 >> 172.21.15.86:0/1576388883 conn(0x7fb3540b1f00 msgr2=0x7fb35415a630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:32.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.317+0000 7fb35a800640 1 -- 172.21.15.86:0/1576388883 shutdown_connections 2024-04-20T21:18:32.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.317+0000 7fb35a800640 1 -- 172.21.15.86:0/1576388883 wait complete. 2024-04-20T21:18:32.336 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:18:32.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.533+0000 7f0b08200640 1 Processor -- start 2024-04-20T21:18:32.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.533+0000 7f0b08200640 1 -- start start 2024-04-20T21:18:32.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.534+0000 7f0b08200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0b0006bbf0 0x7f0b0006e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:32.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.535+0000 7f0b08200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0b00146210 0x7f0b000cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:32.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.535+0000 7f0b08200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b0015cd40 con 0x7f0b0006bfd0 2024-04-20T21:18:32.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.535+0000 7f0b08200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f0b0015d000 con 0x7f0b0006bbf0 2024-04-20T21:18:32.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.535+0000 7f0b08200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f0b0015d2e0 con 0x7f0b00146210 2024-04-20T21:18:32.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.535+0000 7f0b05800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0b0006bbf0 0x7f0b0006e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:32.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.535+0000 7f0b05800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0b0006bbf0 0x7f0b0006e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59120/0 (socket says 172.21.15.86:59120) 2024-04-20T21:18:32.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.535+0000 7f0b05800640 1 -- 172.21.15.86:0/3926580868 learned_addr learned my addr 172.21.15.86:0/3926580868 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:32.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.535+0000 7f0b06200640 1 --2- 172.21.15.86:0/3926580868 >> v2:172.21.15.144:3300/0 conn(0x7f0b00146210 0x7f0b000cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:32.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.536+0000 7f0b05800640 1 -- 172.21.15.86:0/3926580868 >> v2:172.21.15.144:3300/0 conn(0x7f0b00146210 msgr2=0x7f0b000cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:32.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.536+0000 7f0b05800640 1 --2- 172.21.15.86:0/3926580868 >> v2:172.21.15.144:3300/0 conn(0x7f0b00146210 0x7f0b000cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:32.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.536+0000 7f0b05800640 1 -- 172.21.15.86:0/3926580868 >> v1:172.21.15.144:6789/0 conn(0x7f0b0006bfd0 legacy=0x7f0b00145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:32.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.536+0000 7f0b05800640 1 -- 172.21.15.86:0/3926580868 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b0015d5c0 con 0x7f0b0006bbf0 2024-04-20T21:18:32.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.536+0000 7f0b06200640 1 --2- 172.21.15.86:0/3926580868 >> v2:172.21.15.144:3300/0 conn(0x7f0b00146210 0x7f0b000cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:18:32.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.537+0000 7f0b05800640 1 --2- 172.21.15.86:0/3926580868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b0006bbf0 0x7f0b0006e000 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0b0006f6c0 tx=0x7f0af402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5771e4fe8f47a74 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:32.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.538+0000 7f0affe00640 1 -- 172.21.15.86:0/3926580868 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af4035020 con 0x7f0b0006bbf0 2024-04-20T21:18:32.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.538+0000 7f0affe00640 1 -- 172.21.15.86:0/3926580868 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0af4031870 con 0x7f0b0006bbf0 2024-04-20T21:18:32.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.538+0000 7f0affe00640 1 -- 172.21.15.86:0/3926580868 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af4031c40 con 0x7f0b0006bbf0 2024-04-20T21:18:32.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.538+0000 7f0b08200640 1 -- 172.21.15.86:0/3926580868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b0006bbf0 msgr2=0x7f0b0006e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:32.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.538+0000 7f0b08200640 1 --2- 172.21.15.86:0/3926580868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b0006bbf0 0x7f0b0006e000 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0b0006f6c0 tx=0x7f0af402b910 comp rx=0 tx=0).stop 2024-04-20T21:18:32.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.538+0000 7f0b08200640 1 -- 172.21.15.86:0/3926580868 shutdown_connections 2024-04-20T21:18:32.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.538+0000 7f0b08200640 1 --2- 172.21.15.86:0/3926580868 >> v2:172.21.15.144:3300/0 conn(0x7f0b00146210 0x7f0b000cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:32.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.538+0000 7f0b08200640 1 --2- 172.21.15.86:0/3926580868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b0006bbf0 0x7f0b0006e000 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:32.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.538+0000 7f0b08200640 1 -- 172.21.15.86:0/3926580868 >> 172.21.15.86:0/3926580868 conn(0x7f0b000b2110 msgr2=0x7f0b000b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:32.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.538+0000 7f0b08200640 1 -- 172.21.15.86:0/3926580868 shutdown_connections 2024-04-20T21:18:32.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.539+0000 7f0b08200640 1 -- 172.21.15.86:0/3926580868 wait complete. 2024-04-20T21:18:32.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.540+0000 7f0b08200640 1 Processor -- start 2024-04-20T21:18:32.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.540+0000 7f0b08200640 1 -- start start 2024-04-20T21:18:32.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.541+0000 7f0b08200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b0006bbf0 0x7f0b0013e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:32.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.541+0000 7f0b08200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b0006bfd0 0x7f0b000cad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:32.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.541+0000 7f0b08200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b00146210 0x7f0b001e4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:32.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.541+0000 7f0b08200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0b000cdc00 con 0x7f0b0006bbf0 2024-04-20T21:18:32.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.541+0000 7f0b08200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0b000cda80 con 0x7f0b00146210 2024-04-20T21:18:32.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.541+0000 7f0b08200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0b000cdd80 con 0x7f0b0006bfd0 2024-04-20T21:18:32.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.542+0000 7f0b05800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b0006bbf0 0x7f0b0013e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:32.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.542+0000 7f0b05800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b0006bbf0 0x7f0b0013e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59134/0 (socket says 172.21.15.86:59134) 2024-04-20T21:18:32.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.542+0000 7f0b05800640 1 -- 172.21.15.86:0/3962430865 learned_addr learned my addr 172.21.15.86:0/3962430865 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:32.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.542+0000 7f0b05800640 1 -- 172.21.15.86:0/3962430865 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b0006bfd0 msgr2=0x7f0b000cad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:32.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.542+0000 7f0b04e00640 1 --2- 172.21.15.86:0/3962430865 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b0006bfd0 0x7f0b000cad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:32.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.542+0000 7f0b06200640 1 --2- 172.21.15.86:0/3962430865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b00146210 0x7f0b001e4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:32.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.542+0000 7f0b05800640 1 --2- 172.21.15.86:0/3962430865 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b0006bfd0 0x7f0b000cad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:32.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.542+0000 7f0b05800640 1 -- 172.21.15.86:0/3962430865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b00146210 msgr2=0x7f0b001e4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:32.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.542+0000 7f0b05800640 1 --2- 172.21.15.86:0/3962430865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b00146210 0x7f0b001e4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:32.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.542+0000 7f0b05800640 1 -- 172.21.15.86:0/3962430865 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b000cb840 con 0x7f0b0006bbf0 2024-04-20T21:18:32.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.542+0000 7f0b04e00640 1 --2- 172.21.15.86:0/3962430865 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b0006bfd0 0x7f0b000cad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:32.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.542+0000 7f0b06200640 1 --2- 172.21.15.86:0/3962430865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b00146210 0x7f0b001e4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:32.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.542+0000 7f0b05800640 1 --2- 172.21.15.86:0/3962430865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b0006bbf0 0x7f0b0013e9e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0af4000c00 tx=0x7f0af40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:32.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.543+0000 7f0afd600640 1 -- 172.21.15.86:0/3962430865 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af4040070 con 0x7f0b0006bbf0 2024-04-20T21:18:32.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.543+0000 7f0afd600640 1 -- 172.21.15.86:0/3962430865 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0af4030d00 con 0x7f0b0006bbf0 2024-04-20T21:18:32.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.543+0000 7f0afd600640 1 -- 172.21.15.86:0/3962430865 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af4035040 con 0x7f0b0006bbf0 2024-04-20T21:18:32.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.543+0000 7f0b08200640 1 -- 172.21.15.86:0/3962430865 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b000cbb20 con 0x7f0b0006bbf0 2024-04-20T21:18:32.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.543+0000 7f0b08200640 1 -- 172.21.15.86:0/3962430865 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b000cc0e0 con 0x7f0b0006bbf0 2024-04-20T21:18:32.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.544+0000 7f0b08200640 1 -- 172.21.15.86:0/3962430865 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b000689e0 con 0x7f0b0006bbf0 2024-04-20T21:18:32.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.545+0000 7f0afd600640 1 -- 172.21.15.86:0/3962430865 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0af404b060 con 0x7f0b0006bbf0 2024-04-20T21:18:32.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.546+0000 7f0afd600640 1 --2- 172.21.15.86:0/3962430865 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0ae006c8f0 0x7f0ae006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:32.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.546+0000 7f0afd600640 1 -- 172.21.15.86:0/3962430865 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0af40ad900 con 0x7f0b0006bbf0 2024-04-20T21:18:32.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.546+0000 7f0b04e00640 1 --2- 172.21.15.86:0/3962430865 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0ae006c8f0 0x7f0ae006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:32.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.547+0000 7f0b04e00640 1 --2- 172.21.15.86:0/3962430865 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0ae006c8f0 0x7f0ae006edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0b0015c820 tx=0x7f0aec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:32.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.551+0000 7f0afd600640 1 -- 172.21.15.86:0/3962430865 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0af407bd20 con 0x7f0b0006bbf0 2024-04-20T21:18:32.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.754+0000 7f0b08200640 1 -- 172.21.15.86:0/3962430865 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0b0006c780 con 0x7f0b0006bbf0 2024-04-20T21:18:32.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.755+0000 7f0afd600640 1 -- 172.21.15.86:0/3962430865 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f0af40ad0a0 con 0x7f0b0006bbf0 2024-04-20T21:18:32.754 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-20T21:18:32.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.757+0000 7f0b08200640 1 -- 172.21.15.86:0/3962430865 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0ae006c8f0 msgr2=0x7f0ae006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:32.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.757+0000 7f0b08200640 1 --2- 172.21.15.86:0/3962430865 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0ae006c8f0 0x7f0ae006edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0b0015c820 tx=0x7f0aec008040 comp rx=0 tx=0).stop 2024-04-20T21:18:32.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.757+0000 7f0b08200640 1 -- 172.21.15.86:0/3962430865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b0006bbf0 msgr2=0x7f0b0013e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:32.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.757+0000 7f0b08200640 1 --2- 172.21.15.86:0/3962430865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b0006bbf0 0x7f0b0013e9e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0af4000c00 tx=0x7f0af40046c0 comp rx=0 tx=0).stop 2024-04-20T21:18:32.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.757+0000 7f0b08200640 1 -- 172.21.15.86:0/3962430865 shutdown_connections 2024-04-20T21:18:32.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.757+0000 7f0b08200640 1 --2- 172.21.15.86:0/3962430865 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0ae006c8f0 0x7f0ae006edb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:32.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.757+0000 7f0b08200640 1 --2- 172.21.15.86:0/3962430865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b00146210 0x7f0b001e4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:32.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.757+0000 7f0b08200640 1 --2- 172.21.15.86:0/3962430865 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b0006bfd0 0x7f0b000cad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:32.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.757+0000 7f0b08200640 1 --2- 172.21.15.86:0/3962430865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b0006bbf0 0x7f0b0013e9e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:32.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.757+0000 7f0b08200640 1 -- 172.21.15.86:0/3962430865 >> 172.21.15.86:0/3962430865 conn(0x7f0b000b2110 msgr2=0x7f0b0006e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:32.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.758+0000 7f0b08200640 1 -- 172.21.15.86:0/3962430865 shutdown_connections 2024-04-20T21:18:32.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.758+0000 7f0b08200640 1 -- 172.21.15.86:0/3962430865 wait complete. 2024-04-20T21:18:32.776 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:18:32.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.976+0000 7f2d9be00640 1 Processor -- start 2024-04-20T21:18:32.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.976+0000 7f2d9be00640 1 -- start start 2024-04-20T21:18:32.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.978+0000 7f2d9be00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2d9c0af1f0 0x7f2d9c06ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:32.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.978+0000 7f2d9be00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d9c06de80 con 0x7f2d9c0af5d0 2024-04-20T21:18:32.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.978+0000 7f2d9be00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d9c06e160 con 0x7f2d9c06d9c0 2024-04-20T21:18:32.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.978+0000 7f2d9be00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f2d9c06f460 con 0x7f2d9c0af1f0 2024-04-20T21:18:32.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.978+0000 7f2d9b400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f2d9c0af5d0 0x7f2d9c15de70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60454/0 (socket says 172.21.15.86:60454) 2024-04-20T21:18:32.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.978+0000 7f2d9b400640 1 -- 172.21.15.86:0/1537474212 learned_addr learned my addr 172.21.15.86:0/1537474212 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:18:32.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.978+0000 7f2d9aa00640 1 --2- 172.21.15.86:0/1537474212 >> v2:172.21.15.144:3300/0 conn(0x7f2d9c0af1f0 0x7f2d9c06ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:32.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.979+0000 7f2d99600640 1 -- 172.21.15.86:0/1537474212 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1392471068 0 0) 0x7f2d9c06de80 con 0x7f2d9c0af5d0 2024-04-20T21:18:32.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.979+0000 7f2d99600640 1 -- 172.21.15.86:0/1537474212 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d80003520 con 0x7f2d9c0af5d0 2024-04-20T21:18:32.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.979+0000 7f2d99600640 1 -- 172.21.15.86:0/1537474212 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2241958276 0 0) 0x7f2d80003520 con 0x7f2d9c0af5d0 2024-04-20T21:18:32.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.979+0000 7f2d99600640 1 -- 172.21.15.86:0/1537474212 >> v2:172.21.15.144:3300/0 conn(0x7f2d9c0af1f0 msgr2=0x7f2d9c06ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:32.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.979+0000 7f2d99600640 1 --2- 172.21.15.86:0/1537474212 >> v2:172.21.15.144:3300/0 conn(0x7f2d9c0af1f0 0x7f2d9c06ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:32.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.979+0000 7f2d99600640 1 -- 172.21.15.86:0/1537474212 >> v1:172.21.15.144:6789/0 conn(0x7f2d9c06d9c0 legacy=0x7f2d9c15d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:32.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.979+0000 7f2d99600640 1 -- 172.21.15.86:0/1537474212 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d9c06e440 con 0x7f2d9c0af5d0 2024-04-20T21:18:32.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.979+0000 7f2d99600640 1 -- 172.21.15.86:0/1537474212 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f2d940019f0 con 0x7f2d9c0af5d0 2024-04-20T21:18:32.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.979+0000 7f2d99600640 1 -- 172.21.15.86:0/1537474212 >> v1:172.21.15.86:6789/0 conn(0x7f2d9c0af5d0 legacy=0x7f2d9c15de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:32.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.979+0000 7f2d99600640 1 --2- 172.21.15.86:0/1537474212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d80003a00 0x7f2d80003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:32.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.979+0000 7f2d99600640 1 --2- 172.21.15.86:0/1537474212 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d80004800 0x7f2d80004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:32.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.979+0000 7f2d99600640 1 --2- 172.21.15.86:0/1537474212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d800051c0 0x7f2d8000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:32.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.979+0000 7f2d99600640 1 -- 172.21.15.86:0/1537474212 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9c06f460 con 0x7f2d800051c0 2024-04-20T21:18:32.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.979+0000 7f2d99600640 1 -- 172.21.15.86:0/1537474212 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8000c0c0 con 0x7f2d80003a00 2024-04-20T21:18:32.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.979+0000 7f2d99600640 1 -- 172.21.15.86:0/1537474212 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2d8000c2c0 con 0x7f2d80004800 2024-04-20T21:18:32.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.980+0000 7f2d9b400640 1 --2- 172.21.15.86:0/1537474212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d800051c0 0x7f2d8000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:32.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.980+0000 7f2d9aa00640 1 --2- 172.21.15.86:0/1537474212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d80003a00 0x7f2d80003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:32.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.980+0000 7f2d9a000640 1 --2- 172.21.15.86:0/1537474212 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d80004800 0x7f2d80004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:32.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.980+0000 7f2d9b400640 1 -- 172.21.15.86:0/1537474212 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d80004800 msgr2=0x7f2d80004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:32.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.980+0000 7f2d9b400640 1 --2- 172.21.15.86:0/1537474212 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d80004800 0x7f2d80004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:32.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.980+0000 7f2d9b400640 1 -- 172.21.15.86:0/1537474212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d80003a00 msgr2=0x7f2d80003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:32.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.980+0000 7f2d9b400640 1 --2- 172.21.15.86:0/1537474212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d80003a00 0x7f2d80003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:32.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.980+0000 7f2d9b400640 1 -- 172.21.15.86:0/1537474212 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d80003520 con 0x7f2d800051c0 2024-04-20T21:18:32.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.981+0000 7f2d9b400640 1 -- 172.21.15.86:0/1537474212 reap_dead start 2024-04-20T21:18:32.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.981+0000 7f2d9b400640 1 --2- 172.21.15.86:0/1537474212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d800051c0 0x7f2d8000b9c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2d9400a7b0 tx=0x7f2d80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92ca702985ec0a9c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:32.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.981+0000 7f2d99600640 1 -- 172.21.15.86:0/1537474212 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9400a3d0 con 0x7f2d800051c0 2024-04-20T21:18:32.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.981+0000 7f2d99600640 1 -- 172.21.15.86:0/1537474212 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d94034650 con 0x7f2d800051c0 2024-04-20T21:18:32.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.982+0000 7f2d99600640 1 -- 172.21.15.86:0/1537474212 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d94034a40 con 0x7f2d800051c0 2024-04-20T21:18:32.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.982+0000 7f2d9be00640 1 -- 172.21.15.86:0/1537474212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d800051c0 msgr2=0x7f2d8000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:32.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.982+0000 7f2d9be00640 1 --2- 172.21.15.86:0/1537474212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d800051c0 0x7f2d8000b9c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2d9400a7b0 tx=0x7f2d80005830 comp rx=0 tx=0).stop 2024-04-20T21:18:32.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.982+0000 7f2d9be00640 1 -- 172.21.15.86:0/1537474212 shutdown_connections 2024-04-20T21:18:32.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.982+0000 7f2d9be00640 1 --2- 172.21.15.86:0/1537474212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d800051c0 0x7f2d8000b9c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:32.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.982+0000 7f2d9be00640 1 -- 172.21.15.86:0/1537474212 >> 172.21.15.86:0/1537474212 conn(0x7f2d9c0b1f00 msgr2=0x7f2d9c0b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:32.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.982+0000 7f2d9be00640 1 -- 172.21.15.86:0/1537474212 shutdown_connections 2024-04-20T21:18:32.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.982+0000 7f2d9be00640 1 -- 172.21.15.86:0/1537474212 wait complete. 2024-04-20T21:18:32.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.984+0000 7f2d9be00640 1 Processor -- start 2024-04-20T21:18:32.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.984+0000 7f2d9be00640 1 -- start start 2024-04-20T21:18:32.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.985+0000 7f2d9be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d800051c0 0x7f2d9c154bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:32.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.985+0000 7f2d9be00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d9c1557f0 0x7f2d9c0b0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:32.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.985+0000 7f2d9be00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d9c1552a0 0x7f2d9c0b14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:32.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.985+0000 7f2d9be00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9c0afa40 con 0x7f2d800051c0 2024-04-20T21:18:32.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.985+0000 7f2d9be00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9c06ea00 con 0x7f2d9c1552a0 2024-04-20T21:18:32.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.985+0000 7f2d9be00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2d9c15a3c0 con 0x7f2d9c1557f0 2024-04-20T21:18:32.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.985+0000 7f2d9aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d800051c0 0x7f2d9c154bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:32.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.985+0000 7f2d9aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d800051c0 0x7f2d9c154bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59140/0 (socket says 172.21.15.86:59140) 2024-04-20T21:18:32.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.985+0000 7f2d9aa00640 1 -- 172.21.15.86:0/2175736551 learned_addr learned my addr 172.21.15.86:0/2175736551 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:32.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.986+0000 7f2d9a000640 1 --2- 172.21.15.86:0/2175736551 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d9c1557f0 0x7f2d9c0b0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:32.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.986+0000 7f2d9b400640 1 --2- 172.21.15.86:0/2175736551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d9c1552a0 0x7f2d9c0b14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:32.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.986+0000 7f2d9aa00640 1 -- 172.21.15.86:0/2175736551 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d9c1557f0 msgr2=0x7f2d9c0b0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:32.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.986+0000 7f2d9aa00640 1 --2- 172.21.15.86:0/2175736551 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d9c1557f0 0x7f2d9c0b0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:32.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.986+0000 7f2d9aa00640 1 -- 172.21.15.86:0/2175736551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d9c1552a0 msgr2=0x7f2d9c0b14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:32.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.986+0000 7f2d9aa00640 1 --2- 172.21.15.86:0/2175736551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d9c1552a0 0x7f2d9c0b14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:32.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.986+0000 7f2d9aa00640 1 -- 172.21.15.86:0/2175736551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d9c145f90 con 0x7f2d800051c0 2024-04-20T21:18:32.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.986+0000 7f2d9b400640 1 --2- 172.21.15.86:0/2175736551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d9c1552a0 0x7f2d9c0b14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:32.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.986+0000 7f2d9aa00640 1 --2- 172.21.15.86:0/2175736551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d800051c0 0x7f2d9c154bf0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c011540 tx=0x7f2d8c011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:32.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.987+0000 7f2d92a00640 1 -- 172.21.15.86:0/2175736551 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c015930 con 0x7f2d800051c0 2024-04-20T21:18:32.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.987+0000 7f2d92a00640 1 -- 172.21.15.86:0/2175736551 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d8c015ad0 con 0x7f2d800051c0 2024-04-20T21:18:32.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.987+0000 7f2d92a00640 1 -- 172.21.15.86:0/2175736551 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c00f460 con 0x7f2d800051c0 2024-04-20T21:18:32.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.987+0000 7f2d9be00640 1 -- 172.21.15.86:0/2175736551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d9c143aa0 con 0x7f2d800051c0 2024-04-20T21:18:32.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.987+0000 7f2d9be00640 1 -- 172.21.15.86:0/2175736551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d9c145790 con 0x7f2d800051c0 2024-04-20T21:18:32.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.988+0000 7f2d9be00640 1 -- 172.21.15.86:0/2175736551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d9c04fd90 con 0x7f2d800051c0 2024-04-20T21:18:32.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.989+0000 7f2d92a00640 1 -- 172.21.15.86:0/2175736551 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2d8c00f600 con 0x7f2d800051c0 2024-04-20T21:18:32.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.990+0000 7f2d92a00640 1 --2- 172.21.15.86:0/2175736551 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2d7c06c920 0x7f2d7c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:32.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.990+0000 7f2d92a00640 1 -- 172.21.15.86:0/2175736551 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5825+0+0 (secure 0 0 0) 0x7f2d8c08c4b0 con 0x7f2d800051c0 2024-04-20T21:18:32.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.990+0000 7f2d9a000640 1 --2- 172.21.15.86:0/2175736551 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2d7c06c920 0x7f2d7c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:32.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.991+0000 7f2d9a000640 1 --2- 172.21.15.86:0/2175736551 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2d7c06c920 0x7f2d7c06ede0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c0b4770 tx=0x7f2d88009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:32.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:32.994+0000 7f2d92a00640 1 -- 172.21.15.86:0/2175736551 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2d9c145f90 con 0x7f2d800051c0 2024-04-20T21:18:33.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.194+0000 7f2d9be00640 1 -- 172.21.15.86:0/2175736551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2d9c0b3300 con 0x7f2d800051c0 2024-04-20T21:18:33.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.195+0000 7f2d92a00640 1 -- 172.21.15.86:0/2175736551 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f2d8c019030 con 0x7f2d800051c0 2024-04-20T21:18:33.194 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-20T21:18:33.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.197+0000 7f2d9be00640 1 -- 172.21.15.86:0/2175736551 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2d7c06c920 msgr2=0x7f2d7c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:33.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.197+0000 7f2d9be00640 1 --2- 172.21.15.86:0/2175736551 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2d7c06c920 0x7f2d7c06ede0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c0b4770 tx=0x7f2d88009040 comp rx=0 tx=0).stop 2024-04-20T21:18:33.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.197+0000 7f2d9be00640 1 -- 172.21.15.86:0/2175736551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d800051c0 msgr2=0x7f2d9c154bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:33.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.197+0000 7f2d9be00640 1 --2- 172.21.15.86:0/2175736551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d800051c0 0x7f2d9c154bf0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c011540 tx=0x7f2d8c011a00 comp rx=0 tx=0).stop 2024-04-20T21:18:33.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.197+0000 7f2d9be00640 1 -- 172.21.15.86:0/2175736551 shutdown_connections 2024-04-20T21:18:33.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.197+0000 7f2d9be00640 1 --2- 172.21.15.86:0/2175736551 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2d7c06c920 0x7f2d7c06ede0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:33.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.197+0000 7f2d9be00640 1 --2- 172.21.15.86:0/2175736551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d9c1552a0 0x7f2d9c0b14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:33.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.197+0000 7f2d9be00640 1 --2- 172.21.15.86:0/2175736551 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d9c1557f0 0x7f2d9c0b0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:33.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.197+0000 7f2d9be00640 1 --2- 172.21.15.86:0/2175736551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d800051c0 0x7f2d9c154bf0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:33.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.197+0000 7f2d9be00640 1 -- 172.21.15.86:0/2175736551 >> 172.21.15.86:0/2175736551 conn(0x7f2d9c0b1f00 msgr2=0x7f2d9c15b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:33.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.197+0000 7f2d9be00640 1 -- 172.21.15.86:0/2175736551 shutdown_connections 2024-04-20T21:18:33.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.197+0000 7f2d9be00640 1 -- 172.21.15.86:0/2175736551 wait complete. 2024-04-20T21:18:33.215 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-04-20T21:18:33.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.415+0000 7f64dbe00640 1 Processor -- start 2024-04-20T21:18:33.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.415+0000 7f64dbe00640 1 -- start start 2024-04-20T21:18:33.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.416+0000 7f64dbe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f64dc15c010 0x7f64dc15c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:33.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.416+0000 7f64dbe00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f64dc0af7b0 0x7f64dc0adda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:33.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.417+0000 7f64dbe00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64dc0ae710 con 0x7f64dc06f4c0 2024-04-20T21:18:33.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.417+0000 7f64dbe00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f64dc0af320 con 0x7f64dc15c010 2024-04-20T21:18:33.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.417+0000 7f64dbe00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f64dc15c930 con 0x7f64dc0af7b0 2024-04-20T21:18:33.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.417+0000 7f64daa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f64dc15c010 0x7f64dc15c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:33.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.417+0000 7f64db400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f64dc06f4c0 0x7f64dc15ef70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60458/0 (socket says 172.21.15.86:60458) 2024-04-20T21:18:33.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.417+0000 7f64db400640 1 -- 172.21.15.86:0/152555529 learned_addr learned my addr 172.21.15.86:0/152555529 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:18:33.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.417+0000 7f64da000640 1 --2- 172.21.15.86:0/152555529 >> v2:172.21.15.144:3300/0 conn(0x7f64dc0af7b0 0x7f64dc0adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:33.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.417+0000 7f64daa00640 1 -- 172.21.15.86:0/152555529 >> v2:172.21.15.144:3300/0 conn(0x7f64dc0af7b0 msgr2=0x7f64dc0adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:33.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.417+0000 7f64daa00640 1 --2- 172.21.15.86:0/152555529 >> v2:172.21.15.144:3300/0 conn(0x7f64dc0af7b0 0x7f64dc0adda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:33.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.417+0000 7f64daa00640 1 -- 172.21.15.86:0/152555529 >> v1:172.21.15.86:6789/0 conn(0x7f64dc06f4c0 legacy=0x7f64dc15ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:33.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.417+0000 7f64daa00640 1 -- 172.21.15.86:0/152555529 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64dc0ae9f0 con 0x7f64dc15c010 2024-04-20T21:18:33.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.419+0000 7f64daa00640 1 --2- 172.21.15.86:0/152555529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64dc15c010 0x7f64dc15c3f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f64dc145230 tx=0x7f64cc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=633f0721f0dfe23a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:33.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.419+0000 7f64d9600640 1 -- 172.21.15.86:0/152555529 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64cc035020 con 0x7f64dc15c010 2024-04-20T21:18:33.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.419+0000 7f64d9600640 1 -- 172.21.15.86:0/152555529 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64cc031830 con 0x7f64dc15c010 2024-04-20T21:18:33.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.419+0000 7f64d9600640 1 -- 172.21.15.86:0/152555529 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64cc031c00 con 0x7f64dc15c010 2024-04-20T21:18:33.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.419+0000 7f64dbe00640 1 -- 172.21.15.86:0/152555529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64dc15c010 msgr2=0x7f64dc15c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:33.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.419+0000 7f64dbe00640 1 --2- 172.21.15.86:0/152555529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64dc15c010 0x7f64dc15c3f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f64dc145230 tx=0x7f64cc02b8d0 comp rx=0 tx=0).stop 2024-04-20T21:18:33.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.420+0000 7f64dbe00640 1 -- 172.21.15.86:0/152555529 shutdown_connections 2024-04-20T21:18:33.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.420+0000 7f64dbe00640 1 --2- 172.21.15.86:0/152555529 >> v2:172.21.15.144:3300/0 conn(0x7f64dc0af7b0 0x7f64dc0adda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:33.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.420+0000 7f64dbe00640 1 --2- 172.21.15.86:0/152555529 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64dc15c010 0x7f64dc15c3f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:33.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.420+0000 7f64dbe00640 1 -- 172.21.15.86:0/152555529 >> 172.21.15.86:0/152555529 conn(0x7f64dc0d0170 msgr2=0x7f64dc0d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:33.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.420+0000 7f64dbe00640 1 -- 172.21.15.86:0/152555529 shutdown_connections 2024-04-20T21:18:33.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.420+0000 7f64dbe00640 1 -- 172.21.15.86:0/152555529 wait complete. 2024-04-20T21:18:33.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.421+0000 7f64dbe00640 1 Processor -- start 2024-04-20T21:18:33.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.422+0000 7f64dbe00640 1 -- start start 2024-04-20T21:18:33.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.422+0000 7f64dbe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64dc06f4c0 0x7f64dc09ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:33.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.422+0000 7f64dbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64dc0af7b0 0x7f64dc09b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:33.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.423+0000 7f64dbe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64dc15c010 0x7f64dc155170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:33.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.423+0000 7f64dbe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f64dc0ae2e0 con 0x7f64dc0af7b0 2024-04-20T21:18:33.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.423+0000 7f64dbe00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f64dc0aefb0 con 0x7f64dc15c010 2024-04-20T21:18:33.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.423+0000 7f64dbe00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f64dc0ae460 con 0x7f64dc06f4c0 2024-04-20T21:18:33.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.423+0000 7f64da000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64dc0af7b0 0x7f64dc09b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:33.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.423+0000 7f64da000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64dc0af7b0 0x7f64dc09b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59158/0 (socket says 172.21.15.86:59158) 2024-04-20T21:18:33.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.423+0000 7f64da000640 1 -- 172.21.15.86:0/3590050619 learned_addr learned my addr 172.21.15.86:0/3590050619 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:33.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.423+0000 7f64da000640 1 -- 172.21.15.86:0/3590050619 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64dc06f4c0 msgr2=0x7f64dc09ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:33.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.423+0000 7f64da000640 1 --2- 172.21.15.86:0/3590050619 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64dc06f4c0 0x7f64dc09ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:33.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.423+0000 7f64da000640 1 -- 172.21.15.86:0/3590050619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64dc15c010 msgr2=0x7f64dc155170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:33.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.423+0000 7f64da000640 1 --2- 172.21.15.86:0/3590050619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64dc15c010 0x7f64dc155170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:33.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.423+0000 7f64da000640 1 -- 172.21.15.86:0/3590050619 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64dc09a300 con 0x7f64dc0af7b0 2024-04-20T21:18:33.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.424+0000 7f64da000640 1 --2- 172.21.15.86:0/3590050619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64dc0af7b0 0x7f64dc09b3a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f64c800c7f0 tx=0x7f64c800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:33.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.424+0000 7f64d2a00640 1 -- 172.21.15.86:0/3590050619 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c8015070 con 0x7f64dc0af7b0 2024-04-20T21:18:33.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.424+0000 7f64d2a00640 1 -- 172.21.15.86:0/3590050619 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64c8004240 con 0x7f64dc0af7b0 2024-04-20T21:18:33.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.424+0000 7f64dbe00640 1 -- 172.21.15.86:0/3590050619 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64dc09a5e0 con 0x7f64dc0af7b0 2024-04-20T21:18:33.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.424+0000 7f64d2a00640 1 -- 172.21.15.86:0/3590050619 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c8007620 con 0x7f64dc0af7b0 2024-04-20T21:18:33.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.424+0000 7f64dbe00640 1 -- 172.21.15.86:0/3590050619 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64dc09aba0 con 0x7f64dc0af7b0 2024-04-20T21:18:33.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.425+0000 7f64dbe00640 1 -- 172.21.15.86:0/3590050619 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64dc04fcf0 con 0x7f64dc0af7b0 2024-04-20T21:18:33.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.426+0000 7f64d2a00640 1 -- 172.21.15.86:0/3590050619 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f64c80079d0 con 0x7f64dc0af7b0 2024-04-20T21:18:33.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.427+0000 7f64d2a00640 1 --2- 172.21.15.86:0/3590050619 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f64bc06c920 0x7f64bc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:33.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.427+0000 7f64d2a00640 1 -- 172.21.15.86:0/3590050619 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5825+0+0 (secure 0 0 0) 0x7f64c808bd30 con 0x7f64dc0af7b0 2024-04-20T21:18:33.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.427+0000 7f64daa00640 1 --2- 172.21.15.86:0/3590050619 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f64bc06c920 0x7f64bc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:33.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.428+0000 7f64daa00640 1 --2- 172.21.15.86:0/3590050619 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f64bc06c920 0x7f64bc06ede0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f64cc036000 tx=0x7f64cc030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:33.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.431+0000 7f64d2a00640 1 -- 172.21.15.86:0/3590050619 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f64c805a040 con 0x7f64dc0af7b0 2024-04-20T21:18:33.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.704+0000 7f64dbe00640 1 -- 172.21.15.86:0/3590050619 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) -- 0x7f64dc06cb50 con 0x7f64dc0af7b0 2024-04-20T21:18:33.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.803+0000 7f64d2a00640 1 -- 172.21.15.86:0/3590050619 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v63) ==== 145+0+0 (secure 0 0 0) 0x7f64c8059940 con 0x7f64dc0af7b0 2024-04-20T21:18:33.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.807+0000 7f64dbe00640 1 -- 172.21.15.86:0/3590050619 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f64bc06c920 msgr2=0x7f64bc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:33.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.807+0000 7f64dbe00640 1 --2- 172.21.15.86:0/3590050619 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f64bc06c920 0x7f64bc06ede0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f64cc036000 tx=0x7f64cc030000 comp rx=0 tx=0).stop 2024-04-20T21:18:33.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.807+0000 7f64dbe00640 1 -- 172.21.15.86:0/3590050619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64dc0af7b0 msgr2=0x7f64dc09b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:33.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.807+0000 7f64dbe00640 1 --2- 172.21.15.86:0/3590050619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64dc0af7b0 0x7f64dc09b3a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f64c800c7f0 tx=0x7f64c800ccb0 comp rx=0 tx=0).stop 2024-04-20T21:18:33.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.808+0000 7f64dbe00640 1 -- 172.21.15.86:0/3590050619 shutdown_connections 2024-04-20T21:18:33.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.808+0000 7f64dbe00640 1 --2- 172.21.15.86:0/3590050619 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f64bc06c920 0x7f64bc06ede0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:33.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.808+0000 7f64dbe00640 1 --2- 172.21.15.86:0/3590050619 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64dc15c010 0x7f64dc155170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:33.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.808+0000 7f64dbe00640 1 --2- 172.21.15.86:0/3590050619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64dc0af7b0 0x7f64dc09b3a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:33.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.808+0000 7f64dbe00640 1 --2- 172.21.15.86:0/3590050619 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64dc06f4c0 0x7f64dc09ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:33.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.808+0000 7f64dbe00640 1 -- 172.21.15.86:0/3590050619 >> 172.21.15.86:0/3590050619 conn(0x7f64dc0d0170 msgr2=0x7f64dc071b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:33.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.808+0000 7f64dbe00640 1 -- 172.21.15.86:0/3590050619 shutdown_connections 2024-04-20T21:18:33.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:33.808+0000 7f64dbe00640 1 -- 172.21.15.86:0/3590050619 wait complete. 2024-04-20T21:18:33.830 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-20T21:18:34.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.035+0000 7fe966600640 1 Processor -- start 2024-04-20T21:18:34.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.035+0000 7fe966600640 1 -- start start 2024-04-20T21:18:34.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.037+0000 7fe966600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe960070300 0x7fe96015fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:34.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.037+0000 7fe966600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9600d2020 con 0x7fe96015f940 2024-04-20T21:18:34.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.037+0000 7fe966600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9600d22e0 con 0x7fe960160260 2024-04-20T21:18:34.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.037+0000 7fe966600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe960070850 con 0x7fe960070300 2024-04-20T21:18:34.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.037+0000 7fe957e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe960070300 0x7fe96015fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:34.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.037+0000 7fe957e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe960070300 0x7fe96015fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59166/0 (socket says 172.21.15.86:59166) 2024-04-20T21:18:34.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.037+0000 7fe957e00640 1 -- 172.21.15.86:0/122275009 learned_addr learned my addr 172.21.15.86:0/122275009 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:34.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.037+0000 7fe95f400640 1 --1- 172.21.15.86:0/122275009 >> v1:172.21.15.86:6789/0 conn(0x7fe96015f940 0x7fe9600cfd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:60462/0 (socket says 172.21.15.86:60462) 2024-04-20T21:18:34.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.038+0000 7fe957e00640 1 -- 172.21.15.86:0/122275009 >> v1:172.21.15.144:6789/0 conn(0x7fe960160260 legacy=0x7fe9600aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:34.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.038+0000 7fe957e00640 1 -- 172.21.15.86:0/122275009 >> v1:172.21.15.86:6789/0 conn(0x7fe96015f940 legacy=0x7fe9600cfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:34.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.038+0000 7fe957e00640 1 -- 172.21.15.86:0/122275009 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe960070aa0 con 0x7fe960070300 2024-04-20T21:18:34.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.039+0000 7fe957e00640 1 --2- 172.21.15.86:0/122275009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe960070300 0x7fe96015fd20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe9600709d0 tx=0x7fe94802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=401e665097260d23 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:34.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.039+0000 7fe95ea00640 1 -- 172.21.15.86:0/122275009 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe948034070 con 0x7fe960070300 2024-04-20T21:18:34.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.039+0000 7fe95ea00640 1 -- 172.21.15.86:0/122275009 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe948030840 con 0x7fe960070300 2024-04-20T21:18:34.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.039+0000 7fe95ea00640 1 -- 172.21.15.86:0/122275009 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe948030c10 con 0x7fe960070300 2024-04-20T21:18:34.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.040+0000 7fe966600640 1 -- 172.21.15.86:0/122275009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe960070300 msgr2=0x7fe96015fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:34.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.040+0000 7fe966600640 1 --2- 172.21.15.86:0/122275009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe960070300 0x7fe96015fd20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe9600709d0 tx=0x7fe94802b980 comp rx=0 tx=0).stop 2024-04-20T21:18:34.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.040+0000 7fe966600640 1 -- 172.21.15.86:0/122275009 shutdown_connections 2024-04-20T21:18:34.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.040+0000 7fe966600640 1 --2- 172.21.15.86:0/122275009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe960070300 0x7fe96015fd20 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:34.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.040+0000 7fe966600640 1 -- 172.21.15.86:0/122275009 >> 172.21.15.86:0/122275009 conn(0x7fe9600d1500 msgr2=0x7fe9600d1910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:34.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.041+0000 7fe966600640 1 -- 172.21.15.86:0/122275009 shutdown_connections 2024-04-20T21:18:34.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.041+0000 7fe966600640 1 -- 172.21.15.86:0/122275009 wait complete. 2024-04-20T21:18:34.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.042+0000 7fe966600640 1 Processor -- start 2024-04-20T21:18:34.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.042+0000 7fe966600640 1 -- start start 2024-04-20T21:18:34.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.044+0000 7fe966600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe960070300 0x7fe960157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:34.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.044+0000 7fe966600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe96015f940 0x7fe9601584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:34.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.044+0000 7fe966600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe960160260 0x7fe9600c9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:34.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.044+0000 7fe966600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe9600af570 con 0x7fe960160260 2024-04-20T21:18:34.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.044+0000 7fe966600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe9600af3f0 con 0x7fe96015f940 2024-04-20T21:18:34.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.044+0000 7fe966600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe9600af6f0 con 0x7fe960070300 2024-04-20T21:18:34.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.044+0000 7fe95fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe960160260 0x7fe9600c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:34.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.044+0000 7fe95fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe960160260 0x7fe9600c9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59168/0 (socket says 172.21.15.86:59168) 2024-04-20T21:18:34.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.044+0000 7fe95fe00640 1 -- 172.21.15.86:0/465383103 learned_addr learned my addr 172.21.15.86:0/465383103 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:34.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.044+0000 7fe957e00640 1 --2- 172.21.15.86:0/465383103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe96015f940 0x7fe9601584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:34.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.044+0000 7fe95f400640 1 --2- 172.21.15.86:0/465383103 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe960070300 0x7fe960157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:34.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.044+0000 7fe95fe00640 1 -- 172.21.15.86:0/465383103 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe960070300 msgr2=0x7fe960157f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:34.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.044+0000 7fe95fe00640 1 --2- 172.21.15.86:0/465383103 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe960070300 0x7fe960157f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:34.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.045+0000 7fe95fe00640 1 -- 172.21.15.86:0/465383103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe96015f940 msgr2=0x7fe9601584b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:34.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.045+0000 7fe95fe00640 1 --2- 172.21.15.86:0/465383103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe96015f940 0x7fe9601584b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:34.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.045+0000 7fe95fe00640 1 -- 172.21.15.86:0/465383103 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe96015bbf0 con 0x7fe960160260 2024-04-20T21:18:34.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.045+0000 7fe957e00640 1 --2- 172.21.15.86:0/465383103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe96015f940 0x7fe9601584b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:34.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.045+0000 7fe95f400640 1 --2- 172.21.15.86:0/465383103 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe960070300 0x7fe960157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:34.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.045+0000 7fe95fe00640 1 --2- 172.21.15.86:0/465383103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe960160260 0x7fe9600c9620 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe95800b970 tx=0x7fe95800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:34.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.045+0000 7fe957400640 1 -- 172.21.15.86:0/465383103 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe958019070 con 0x7fe960160260 2024-04-20T21:18:34.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.045+0000 7fe957400640 1 -- 172.21.15.86:0/465383103 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe95800cae0 con 0x7fe960160260 2024-04-20T21:18:34.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.045+0000 7fe957400640 1 -- 172.21.15.86:0/465383103 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe958014440 con 0x7fe960160260 2024-04-20T21:18:34.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.045+0000 7fe966600640 1 -- 172.21.15.86:0/465383103 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe960156ad0 con 0x7fe960160260 2024-04-20T21:18:34.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.045+0000 7fe966600640 1 -- 172.21.15.86:0/465383103 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe960156510 con 0x7fe960160260 2024-04-20T21:18:34.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.046+0000 7fe966600640 1 -- 172.21.15.86:0/465383103 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9600b1700 con 0x7fe960160260 2024-04-20T21:18:34.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.048+0000 7fe957400640 1 -- 172.21.15.86:0/465383103 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe95801c070 con 0x7fe960160260 2024-04-20T21:18:34.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.049+0000 7fe957400640 1 --2- 172.21.15.86:0/465383103 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe93c06c920 0x7fe93c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:34.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.049+0000 7fe957400640 1 -- 172.21.15.86:0/465383103 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fe95808b710 con 0x7fe960160260 2024-04-20T21:18:34.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.049+0000 7fe95f400640 1 --2- 172.21.15.86:0/465383103 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe93c06c920 0x7fe93c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:34.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.050+0000 7fe95f400640 1 --2- 172.21.15.86:0/465383103 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe93c06c920 0x7fe93c06ede0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe950000f60 tx=0x7fe950009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:34.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.053+0000 7fe957400640 1 -- 172.21.15.86:0/465383103 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe9600b1700 con 0x7fe960160260 2024-04-20T21:18:34.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.251+0000 7fe966600640 1 -- 172.21.15.86:0/465383103 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fe9600690f0 con 0x7fe960160260 2024-04-20T21:18:34.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.255+0000 7fe957400640 1 -- 172.21.15.86:0/465383103 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7fe958059ab0 con 0x7fe960160260 2024-04-20T21:18:34.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.259+0000 7fe966600640 1 -- 172.21.15.86:0/465383103 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe93c06c920 msgr2=0x7fe93c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:34.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.259+0000 7fe966600640 1 --2- 172.21.15.86:0/465383103 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe93c06c920 0x7fe93c06ede0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe950000f60 tx=0x7fe950009040 comp rx=0 tx=0).stop 2024-04-20T21:18:34.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.259+0000 7fe966600640 1 -- 172.21.15.86:0/465383103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe960160260 msgr2=0x7fe9600c9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:34.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.259+0000 7fe966600640 1 --2- 172.21.15.86:0/465383103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe960160260 0x7fe9600c9620 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe95800b970 tx=0x7fe95800be30 comp rx=0 tx=0).stop 2024-04-20T21:18:34.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.259+0000 7fe966600640 1 -- 172.21.15.86:0/465383103 shutdown_connections 2024-04-20T21:18:34.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.259+0000 7fe966600640 1 --2- 172.21.15.86:0/465383103 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe93c06c920 0x7fe93c06ede0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:34.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.259+0000 7fe966600640 1 --2- 172.21.15.86:0/465383103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe960160260 0x7fe9600c9620 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:34.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.259+0000 7fe966600640 1 --2- 172.21.15.86:0/465383103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe96015f940 0x7fe9601584b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:34.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.259+0000 7fe966600640 1 --2- 172.21.15.86:0/465383103 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe960070300 0x7fe960157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:34.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.259+0000 7fe966600640 1 -- 172.21.15.86:0/465383103 >> 172.21.15.86:0/465383103 conn(0x7fe9600d1500 msgr2=0x7fe9600b1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:34.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.259+0000 7fe966600640 1 -- 172.21.15.86:0/465383103 shutdown_connections 2024-04-20T21:18:34.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.259+0000 7fe966600640 1 -- 172.21.15.86:0/465383103 wait complete. 2024-04-20T21:18:34.278 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-20T21:18:34.278 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T21:18:34.387 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T21:18:34.503 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-20T21:18:34.503 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:18:34.503 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:18:34.560 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:18:34.560 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:18:34.612 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:18:34.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.812+0000 7f38ed600640 1 Processor -- start 2024-04-20T21:18:34.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.812+0000 7f38ed600640 1 -- start start 2024-04-20T21:18:34.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.814+0000 7f38ed600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f38e80b5990 0x7f38e80cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:34.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.814+0000 7f38ed600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f38e80cfcb0 0x7f38e815baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:34.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.814+0000 7f38ed600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38e80d04a0 con 0x7f38e8155af0 2024-04-20T21:18:34.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.814+0000 7f38ed600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f38e815f110 con 0x7f38e80cfcb0 2024-04-20T21:18:34.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.814+0000 7f38ed600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f38e815f3f0 con 0x7f38e80b5990 2024-04-20T21:18:34.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.815+0000 7f38ecc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f38e80cfcb0 0x7f38e815baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:34.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.815+0000 7f38ecc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f38e80cfcb0 0x7f38e815baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59178/0 (socket says 172.21.15.86:59178) 2024-04-20T21:18:34.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.815+0000 7f38ecc00640 1 -- 172.21.15.86:0/4151359659 learned_addr learned my addr 172.21.15.86:0/4151359659 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:34.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.815+0000 7f38e7400640 1 --2- 172.21.15.86:0/4151359659 >> v2:172.21.15.144:3300/0 conn(0x7f38e80b5990 0x7f38e80cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:34.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.815+0000 7f38ecc00640 1 -- 172.21.15.86:0/4151359659 >> v2:172.21.15.144:3300/0 conn(0x7f38e80b5990 msgr2=0x7f38e80cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:34.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.815+0000 7f38ecc00640 1 --2- 172.21.15.86:0/4151359659 >> v2:172.21.15.144:3300/0 conn(0x7f38e80b5990 0x7f38e80cf770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:34.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.815+0000 7f38ecc00640 1 -- 172.21.15.86:0/4151359659 >> v1:172.21.15.144:6789/0 conn(0x7f38e8155af0 legacy=0x7f38e80b5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:34.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.815+0000 7f38ecc00640 1 -- 172.21.15.86:0/4151359659 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38e815f6d0 con 0x7f38e80cfcb0 2024-04-20T21:18:34.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.817+0000 7f38ecc00640 1 --2- 172.21.15.86:0/4151359659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38e80cfcb0 0x7f38e815baa0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f38e8071f20 tx=0x7f38e002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=738bc5046c39ff91 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:34.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.817+0000 7f38e6a00640 1 -- 172.21.15.86:0/4151359659 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e002ca80 con 0x7f38e80cfcb0 2024-04-20T21:18:34.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.817+0000 7f38e6a00640 1 -- 172.21.15.86:0/4151359659 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38e002cc20 con 0x7f38e80cfcb0 2024-04-20T21:18:34.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.817+0000 7f38e6a00640 1 -- 172.21.15.86:0/4151359659 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e00343f0 con 0x7f38e80cfcb0 2024-04-20T21:18:34.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.818+0000 7f38ed600640 1 -- 172.21.15.86:0/4151359659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38e80cfcb0 msgr2=0x7f38e815baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:34.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.818+0000 7f38ed600640 1 --2- 172.21.15.86:0/4151359659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38e80cfcb0 0x7f38e815baa0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f38e8071f20 tx=0x7f38e002bee0 comp rx=0 tx=0).stop 2024-04-20T21:18:34.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.818+0000 7f38ed600640 1 -- 172.21.15.86:0/4151359659 shutdown_connections 2024-04-20T21:18:34.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.818+0000 7f38ed600640 1 --2- 172.21.15.86:0/4151359659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38e80cfcb0 0x7f38e815baa0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:34.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.818+0000 7f38ed600640 1 --2- 172.21.15.86:0/4151359659 >> v2:172.21.15.144:3300/0 conn(0x7f38e80b5990 0x7f38e80cf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:34.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.818+0000 7f38ed600640 1 -- 172.21.15.86:0/4151359659 >> 172.21.15.86:0/4151359659 conn(0x7f38e806b980 msgr2=0x7f38e806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:34.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.818+0000 7f38ed600640 1 -- 172.21.15.86:0/4151359659 shutdown_connections 2024-04-20T21:18:34.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.818+0000 7f38ed600640 1 -- 172.21.15.86:0/4151359659 wait complete. 2024-04-20T21:18:34.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.820+0000 7f38ed600640 1 Processor -- start 2024-04-20T21:18:34.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.820+0000 7f38ed600640 1 -- start start 2024-04-20T21:18:34.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.821+0000 7f38ed600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38e80b5990 0x7f38e806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:34.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.821+0000 7f38ed600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38e80cfcb0 0x7f38e806f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:34.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.821+0000 7f38ed600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38e8155af0 0x7f38e814a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:34.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.821+0000 7f38ed600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f38e815fed0 con 0x7f38e8155af0 2024-04-20T21:18:34.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.821+0000 7f38ed600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f38e8155ed0 con 0x7f38e80cfcb0 2024-04-20T21:18:34.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.821+0000 7f38ed600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f38e80d00e0 con 0x7f38e80b5990 2024-04-20T21:18:34.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.821+0000 7f38ecc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38e8155af0 0x7f38e814a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:34.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.821+0000 7f38ecc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38e8155af0 0x7f38e814a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59184/0 (socket says 172.21.15.86:59184) 2024-04-20T21:18:34.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.821+0000 7f38ecc00640 1 -- 172.21.15.86:0/1425364318 learned_addr learned my addr 172.21.15.86:0/1425364318 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:34.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.821+0000 7f38ecc00640 1 -- 172.21.15.86:0/1425364318 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38e80b5990 msgr2=0x7f38e806f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:34.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.821+0000 7f38ecc00640 1 --2- 172.21.15.86:0/1425364318 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38e80b5990 0x7f38e806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:34.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.821+0000 7f38ecc00640 1 -- 172.21.15.86:0/1425364318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38e80cfcb0 msgr2=0x7f38e806f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:34.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.821+0000 7f38ecc00640 1 --2- 172.21.15.86:0/1425364318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38e80cfcb0 0x7f38e806f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:34.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.821+0000 7f38ecc00640 1 -- 172.21.15.86:0/1425364318 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38e81682b0 con 0x7f38e8155af0 2024-04-20T21:18:34.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.822+0000 7f38ecc00640 1 --2- 172.21.15.86:0/1425364318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38e8155af0 0x7f38e814a990 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f38e0036040 tx=0x7f38e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:34.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.822+0000 7f38dfe00640 1 -- 172.21.15.86:0/1425364318 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e0043020 con 0x7f38e8155af0 2024-04-20T21:18:34.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.822+0000 7f38dfe00640 1 -- 172.21.15.86:0/1425364318 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38e003e450 con 0x7f38e8155af0 2024-04-20T21:18:34.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.822+0000 7f38dfe00640 1 -- 172.21.15.86:0/1425364318 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e0009510 con 0x7f38e8155af0 2024-04-20T21:18:34.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.822+0000 7f38ed600640 1 -- 172.21.15.86:0/1425364318 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38e8168590 con 0x7f38e8155af0 2024-04-20T21:18:34.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.822+0000 7f38ed600640 1 -- 172.21.15.86:0/1425364318 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38e8168b50 con 0x7f38e8155af0 2024-04-20T21:18:34.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.823+0000 7f38ed600640 1 -- 172.21.15.86:0/1425364318 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38e804fcf0 con 0x7f38e8155af0 2024-04-20T21:18:34.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.824+0000 7f38dfe00640 1 -- 172.21.15.86:0/1425364318 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f38e0005d10 con 0x7f38e8155af0 2024-04-20T21:18:34.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.825+0000 7f38dfe00640 1 --2- 172.21.15.86:0/1425364318 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f38c806c920 0x7f38c806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:34.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.825+0000 7f38dfe00640 1 -- 172.21.15.86:0/1425364318 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f38e00ae080 con 0x7f38e8155af0 2024-04-20T21:18:34.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.825+0000 7f38e7e00640 1 --2- 172.21.15.86:0/1425364318 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f38c806c920 0x7f38c806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:34.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.826+0000 7f38e7e00640 1 --2- 172.21.15.86:0/1425364318 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f38c806c920 0x7f38c806ede0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f38e806fd60 tx=0x7f38d8006ba0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:34.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:34.830+0000 7f38dfe00640 1 -- 172.21.15.86:0/1425364318 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f38e004d020 con 0x7f38e8155af0 2024-04-20T21:18:35.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.030+0000 7f38ed600640 1 -- 172.21.15.86:0/1425364318 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f38e80b2f80 con 0x7f38e8155af0 2024-04-20T21:18:35.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.032+0000 7f38dfe00640 1 -- 172.21.15.86:0/1425364318 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16854 (secure 0 0 0) 0x7f38e007c420 con 0x7f38e8155af0 2024-04-20T21:18:35.032 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:18:35.032 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":22,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:18:33.788155+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:17:49.718799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:17:50.683477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:18:35.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.035+0000 7f38ed600640 1 -- 172.21.15.86:0/1425364318 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f38c806c920 msgr2=0x7f38c806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:35.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.035+0000 7f38ed600640 1 --2- 172.21.15.86:0/1425364318 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f38c806c920 0x7f38c806ede0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f38e806fd60 tx=0x7f38d8006ba0 comp rx=0 tx=0).stop 2024-04-20T21:18:35.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.035+0000 7f38ed600640 1 -- 172.21.15.86:0/1425364318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38e8155af0 msgr2=0x7f38e814a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:35.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.035+0000 7f38ed600640 1 --2- 172.21.15.86:0/1425364318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38e8155af0 0x7f38e814a990 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f38e0036040 tx=0x7f38e0004a40 comp rx=0 tx=0).stop 2024-04-20T21:18:35.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.035+0000 7f38ed600640 1 -- 172.21.15.86:0/1425364318 shutdown_connections 2024-04-20T21:18:35.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.035+0000 7f38ed600640 1 --2- 172.21.15.86:0/1425364318 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f38c806c920 0x7f38c806ede0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:35.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.035+0000 7f38ed600640 1 --2- 172.21.15.86:0/1425364318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38e8155af0 0x7f38e814a990 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:35.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.035+0000 7f38ed600640 1 --2- 172.21.15.86:0/1425364318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38e80cfcb0 0x7f38e806f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:35.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.035+0000 7f38ed600640 1 --2- 172.21.15.86:0/1425364318 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38e80b5990 0x7f38e806f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:35.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.035+0000 7f38ed600640 1 -- 172.21.15.86:0/1425364318 >> 172.21.15.86:0/1425364318 conn(0x7f38e806b980 msgr2=0x7f38e814edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:35.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.035+0000 7f38ed600640 1 -- 172.21.15.86:0/1425364318 shutdown_connections 2024-04-20T21:18:35.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.035+0000 7f38ed600640 1 -- 172.21.15.86:0/1425364318 wait complete. 2024-04-20T21:18:35.053 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:17:49.718799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:17:50.683477+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:18:35.053 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:18:35.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.249+0000 7f2786c00640 1 Processor -- start 2024-04-20T21:18:35.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.250+0000 7f2786c00640 1 -- start start 2024-04-20T21:18:35.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.251+0000 7f2786c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f278006bbf0 0x7f278006e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:35.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.251+0000 7f2786c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f278006bfd0 0x7f2780145b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:35.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.251+0000 7f2786c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f278015cd40 con 0x7f2780146040 2024-04-20T21:18:35.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.251+0000 7f2786c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f278015d000 con 0x7f278006bfd0 2024-04-20T21:18:35.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.251+0000 7f2786c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f278015d2e0 con 0x7f278006bbf0 2024-04-20T21:18:35.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.252+0000 7f2777e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f278006bfd0 0x7f2780145b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:35.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.252+0000 7f2777e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f278006bfd0 0x7f2780145b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59196/0 (socket says 172.21.15.86:59196) 2024-04-20T21:18:35.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.252+0000 7f2777e00640 1 -- 172.21.15.86:0/1537714288 learned_addr learned my addr 172.21.15.86:0/1537714288 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:35.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.252+0000 7f277fe00640 1 --2- 172.21.15.86:0/1537714288 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f278006bbf0 0x7f278006e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:35.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.252+0000 7f2777e00640 1 -- 172.21.15.86:0/1537714288 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f278006bbf0 msgr2=0x7f278006e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:35.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.252+0000 7f2777e00640 1 --2- 172.21.15.86:0/1537714288 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f278006bbf0 0x7f278006e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:35.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.252+0000 7f2777e00640 1 -- 172.21.15.86:0/1537714288 >> v1:172.21.15.144:6789/0 conn(0x7f2780146040 legacy=0x7f27800cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:35.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.252+0000 7f2777e00640 1 -- 172.21.15.86:0/1537714288 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f278015d5c0 con 0x7f278006bfd0 2024-04-20T21:18:35.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.254+0000 7f2777e00640 1 --2- 172.21.15.86:0/1537714288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f278006bfd0 0x7f2780145b00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2768009870 tx=0x7f276802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8326bd64bc1f7e9f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:35.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.254+0000 7f277f400640 1 -- 172.21.15.86:0/1537714288 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2768034070 con 0x7f278006bfd0 2024-04-20T21:18:35.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.254+0000 7f277f400640 1 -- 172.21.15.86:0/1537714288 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2768030840 con 0x7f278006bfd0 2024-04-20T21:18:35.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.254+0000 7f277f400640 1 -- 172.21.15.86:0/1537714288 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2768030c10 con 0x7f278006bfd0 2024-04-20T21:18:35.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.254+0000 7f2786c00640 1 -- 172.21.15.86:0/1537714288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f278006bfd0 msgr2=0x7f2780145b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:35.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.254+0000 7f2786c00640 1 --2- 172.21.15.86:0/1537714288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f278006bfd0 0x7f2780145b00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2768009870 tx=0x7f276802b980 comp rx=0 tx=0).stop 2024-04-20T21:18:35.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.255+0000 7f2786c00640 1 -- 172.21.15.86:0/1537714288 shutdown_connections 2024-04-20T21:18:35.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.255+0000 7f2786c00640 1 --2- 172.21.15.86:0/1537714288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f278006bfd0 0x7f2780145b00 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:35.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.255+0000 7f2786c00640 1 --2- 172.21.15.86:0/1537714288 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f278006bbf0 0x7f278006e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:35.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.255+0000 7f2786c00640 1 -- 172.21.15.86:0/1537714288 >> 172.21.15.86:0/1537714288 conn(0x7f27800b2110 msgr2=0x7f27800b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:35.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.255+0000 7f2786c00640 1 -- 172.21.15.86:0/1537714288 shutdown_connections 2024-04-20T21:18:35.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.255+0000 7f2786c00640 1 -- 172.21.15.86:0/1537714288 wait complete. 2024-04-20T21:18:35.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.256+0000 7f2786c00640 1 Processor -- start 2024-04-20T21:18:35.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.257+0000 7f2786c00640 1 -- start start 2024-04-20T21:18:35.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.257+0000 7f2786c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f278006bbf0 0x7f27801441f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:35.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.258+0000 7f2786c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f278006bfd0 0x7f27801f7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:35.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.258+0000 7f2786c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2780146040 0x7f27800b0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:35.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.258+0000 7f2786c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f27800cddd0 con 0x7f278006bfd0 2024-04-20T21:18:35.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.258+0000 7f2786c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f27800cdc50 con 0x7f2780146040 2024-04-20T21:18:35.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.258+0000 7f2786c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f27800cdf50 con 0x7f278006bbf0 2024-04-20T21:18:35.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.258+0000 7f2777e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f278006bfd0 0x7f27801f7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:35.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.258+0000 7f2777e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f278006bfd0 0x7f27801f7c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59212/0 (socket says 172.21.15.86:59212) 2024-04-20T21:18:35.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.258+0000 7f2777e00640 1 -- 172.21.15.86:0/3906935009 learned_addr learned my addr 172.21.15.86:0/3906935009 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:35.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.258+0000 7f2777e00640 1 -- 172.21.15.86:0/3906935009 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f278006bbf0 msgr2=0x7f27801441f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:35.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.258+0000 7f2777e00640 1 --2- 172.21.15.86:0/3906935009 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f278006bbf0 0x7f27801441f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:35.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.258+0000 7f2777e00640 1 -- 172.21.15.86:0/3906935009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2780146040 msgr2=0x7f27800b0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:35.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.258+0000 7f2777e00640 1 --2- 172.21.15.86:0/3906935009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2780146040 0x7f27800b0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:35.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.258+0000 7f2777e00640 1 -- 172.21.15.86:0/3906935009 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27801f8b70 con 0x7f278006bfd0 2024-04-20T21:18:35.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.259+0000 7f2777e00640 1 --2- 172.21.15.86:0/3906935009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f278006bfd0 0x7f27801f7c80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2768000c00 tx=0x7f2768004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:35.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.259+0000 7f277cc00640 1 -- 172.21.15.86:0/3906935009 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2768048070 con 0x7f278006bfd0 2024-04-20T21:18:35.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.259+0000 7f277cc00640 1 -- 172.21.15.86:0/3906935009 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2768030640 con 0x7f278006bfd0 2024-04-20T21:18:35.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.259+0000 7f277cc00640 1 -- 172.21.15.86:0/3906935009 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2768034040 con 0x7f278006bfd0 2024-04-20T21:18:35.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.259+0000 7f2786c00640 1 -- 172.21.15.86:0/3906935009 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27801f85b0 con 0x7f278006bfd0 2024-04-20T21:18:35.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.259+0000 7f2786c00640 1 -- 172.21.15.86:0/3906935009 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27801e6d90 con 0x7f278006bfd0 2024-04-20T21:18:35.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.260+0000 7f2786c00640 1 -- 172.21.15.86:0/3906935009 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27800689e0 con 0x7f278006bfd0 2024-04-20T21:18:35.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.261+0000 7f277cc00640 1 -- 172.21.15.86:0/3906935009 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2768037070 con 0x7f278006bfd0 2024-04-20T21:18:35.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.263+0000 7f277cc00640 1 --2- 172.21.15.86:0/3906935009 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f275c06c920 0x7f275c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:35.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.263+0000 7f277cc00640 1 -- 172.21.15.86:0/3906935009 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f27680ad9b0 con 0x7f278006bfd0 2024-04-20T21:18:35.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.263+0000 7f277fe00640 1 --2- 172.21.15.86:0/3906935009 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f275c06c920 0x7f275c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:35.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.263+0000 7f277fe00640 1 --2- 172.21.15.86:0/3906935009 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f275c06c920 0x7f275c06ede0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f27700098f0 tx=0x7f2770008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:35.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.267+0000 7f277cc00640 1 -- 172.21.15.86:0/3906935009 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f276807be70 con 0x7f278006bfd0 2024-04-20T21:18:35.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.464+0000 7f2786c00640 1 -- 172.21.15.86:0/3906935009 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f278006c780 con 0x7f278006bfd0 2024-04-20T21:18:35.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.465+0000 7f277cc00640 1 -- 172.21.15.86:0/3906935009 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f276807b770 con 0x7f278006bfd0 2024-04-20T21:18:35.466 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:18:35.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.468+0000 7f2786c00640 1 -- 172.21.15.86:0/3906935009 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f275c06c920 msgr2=0x7f275c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:35.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.468+0000 7f2786c00640 1 --2- 172.21.15.86:0/3906935009 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f275c06c920 0x7f275c06ede0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f27700098f0 tx=0x7f2770008040 comp rx=0 tx=0).stop 2024-04-20T21:18:35.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.468+0000 7f2786c00640 1 -- 172.21.15.86:0/3906935009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f278006bfd0 msgr2=0x7f27801f7c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:35.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.468+0000 7f2786c00640 1 --2- 172.21.15.86:0/3906935009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f278006bfd0 0x7f27801f7c80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2768000c00 tx=0x7f2768004730 comp rx=0 tx=0).stop 2024-04-20T21:18:35.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.468+0000 7f2786c00640 1 -- 172.21.15.86:0/3906935009 shutdown_connections 2024-04-20T21:18:35.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.468+0000 7f2786c00640 1 --2- 172.21.15.86:0/3906935009 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f275c06c920 0x7f275c06ede0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:35.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.468+0000 7f2786c00640 1 --2- 172.21.15.86:0/3906935009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2780146040 0x7f27800b0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:35.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.468+0000 7f2786c00640 1 --2- 172.21.15.86:0/3906935009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f278006bfd0 0x7f27801f7c80 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:35.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.468+0000 7f2786c00640 1 --2- 172.21.15.86:0/3906935009 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f278006bbf0 0x7f27801441f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:35.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.468+0000 7f2786c00640 1 -- 172.21.15.86:0/3906935009 >> 172.21.15.86:0/3906935009 conn(0x7f27800b2110 msgr2=0x7f278006e450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:35.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.468+0000 7f2786c00640 1 -- 172.21.15.86:0/3906935009 shutdown_connections 2024-04-20T21:18:35.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.469+0000 7f2786c00640 1 -- 172.21.15.86:0/3906935009 wait complete. 2024-04-20T21:18:35.487 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:18:35.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.691+0000 7f0995a00640 1 Processor -- start 2024-04-20T21:18:35.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.691+0000 7f0995a00640 1 -- start start 2024-04-20T21:18:35.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.692+0000 7f0995a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f09901634c0 0x7f09900cf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:35.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.692+0000 7f0995a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f09900d1260 0x7f09900d16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:35.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.692+0000 7f0995a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0990163970 con 0x7f099015ee00 2024-04-20T21:18:35.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.693+0000 7f0995a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f09900cfd70 con 0x7f09901634c0 2024-04-20T21:18:35.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.693+0000 7f0995a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f09900d0550 con 0x7f09900d1260 2024-04-20T21:18:35.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.693+0000 7f098e000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f09901634c0 0x7f09900cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:35.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.693+0000 7f098e000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f09901634c0 0x7f09900cf830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59226/0 (socket says 172.21.15.86:59226) 2024-04-20T21:18:35.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.693+0000 7f098e000640 1 -- 172.21.15.86:0/804814846 learned_addr learned my addr 172.21.15.86:0/804814846 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:35.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.693+0000 7f098f400640 1 --2- 172.21.15.86:0/804814846 >> v2:172.21.15.144:3300/0 conn(0x7f09900d1260 0x7f09900d16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:35.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.694+0000 7f098f400640 1 -- 172.21.15.86:0/804814846 >> v2:172.21.15.86:3300/0 conn(0x7f09901634c0 msgr2=0x7f09900cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:35.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.694+0000 7f098f400640 1 --2- 172.21.15.86:0/804814846 >> v2:172.21.15.86:3300/0 conn(0x7f09901634c0 0x7f09900cf830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:35.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.694+0000 7f098f400640 1 -- 172.21.15.86:0/804814846 >> v1:172.21.15.86:6789/0 conn(0x7f099015ee00 legacy=0x7f099015fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:35.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.694+0000 7f098f400640 1 -- 172.21.15.86:0/804814846 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09900cff90 con 0x7f09900d1260 2024-04-20T21:18:35.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.695+0000 7f098f400640 1 --2- 172.21.15.86:0/804814846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09900d1260 0x7f09900d16a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f09900d2960 tx=0x7f098802bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5db325c1c256081 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:35.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.696+0000 7f098d600640 1 -- 172.21.15.86:0/804814846 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0988039070 con 0x7f09900d1260 2024-04-20T21:18:35.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.696+0000 7f098d600640 1 -- 172.21.15.86:0/804814846 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f098802cc80 con 0x7f09900d1260 2024-04-20T21:18:35.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.696+0000 7f098d600640 1 -- 172.21.15.86:0/804814846 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0988034660 con 0x7f09900d1260 2024-04-20T21:18:35.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.696+0000 7f0995a00640 1 -- 172.21.15.86:0/804814846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09900d1260 msgr2=0x7f09900d16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:35.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.696+0000 7f0995a00640 1 --2- 172.21.15.86:0/804814846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09900d1260 0x7f09900d16a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f09900d2960 tx=0x7f098802bed0 comp rx=0 tx=0).stop 2024-04-20T21:18:35.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.697+0000 7f0995a00640 1 -- 172.21.15.86:0/804814846 shutdown_connections 2024-04-20T21:18:35.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.697+0000 7f0995a00640 1 --2- 172.21.15.86:0/804814846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09900d1260 0x7f09900d16a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:35.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.697+0000 7f0995a00640 1 --2- 172.21.15.86:0/804814846 >> v2:172.21.15.86:3300/0 conn(0x7f09901634c0 0x7f09900cf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:35.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.697+0000 7f0995a00640 1 -- 172.21.15.86:0/804814846 >> 172.21.15.86:0/804814846 conn(0x7f099006cc20 msgr2=0x7f099006d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:35.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.697+0000 7f0995a00640 1 -- 172.21.15.86:0/804814846 shutdown_connections 2024-04-20T21:18:35.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.697+0000 7f0995a00640 1 -- 172.21.15.86:0/804814846 wait complete. 2024-04-20T21:18:35.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.698+0000 7f0995a00640 1 Processor -- start 2024-04-20T21:18:35.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.698+0000 7f0995a00640 1 -- start start 2024-04-20T21:18:35.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.699+0000 7f0995a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09900d1260 0x7f09900cb920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:35.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.699+0000 7f0995a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f099015ee00 0x7f09901580c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:35.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.699+0000 7f0995a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09901634c0 0x7f0990152f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:35.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.699+0000 7f0995a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f099015f640 con 0x7f09900d1260 2024-04-20T21:18:35.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.699+0000 7f0995a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f09900d1be0 con 0x7f099015ee00 2024-04-20T21:18:35.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.699+0000 7f0995a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f099015f1e0 con 0x7f09901634c0 2024-04-20T21:18:35.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.700+0000 7f098ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09900d1260 0x7f09900cb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:35.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.700+0000 7f098ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09900d1260 0x7f09900cb920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59228/0 (socket says 172.21.15.86:59228) 2024-04-20T21:18:35.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.700+0000 7f098ea00640 1 -- 172.21.15.86:0/3093883000 learned_addr learned my addr 172.21.15.86:0/3093883000 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:35.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.700+0000 7f098e000640 1 --2- 172.21.15.86:0/3093883000 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f099015ee00 0x7f09901580c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:35.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.700+0000 7f098ea00640 1 -- 172.21.15.86:0/3093883000 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09901634c0 msgr2=0x7f0990152f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:35.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.700+0000 7f098ea00640 1 --2- 172.21.15.86:0/3093883000 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09901634c0 0x7f0990152f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:35.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.700+0000 7f098ea00640 1 -- 172.21.15.86:0/3093883000 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f099015ee00 msgr2=0x7f09901580c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:35.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.700+0000 7f098ea00640 1 --2- 172.21.15.86:0/3093883000 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f099015ee00 0x7f09901580c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:35.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.700+0000 7f098ea00640 1 -- 172.21.15.86:0/3093883000 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09901587c0 con 0x7f09900d1260 2024-04-20T21:18:35.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.700+0000 7f098ea00640 1 --2- 172.21.15.86:0/3093883000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09900d1260 0x7f09900cb920 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f098000c430 tx=0x7f098000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:35.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.700+0000 7f0986a00640 1 -- 172.21.15.86:0/3093883000 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09800052d0 con 0x7f09900d1260 2024-04-20T21:18:35.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.701+0000 7f0986a00640 1 -- 172.21.15.86:0/3093883000 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0980005490 con 0x7f09900d1260 2024-04-20T21:18:35.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.701+0000 7f0995a00640 1 -- 172.21.15.86:0/3093883000 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0990158aa0 con 0x7f09900d1260 2024-04-20T21:18:35.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.701+0000 7f0986a00640 1 -- 172.21.15.86:0/3093883000 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0980007260 con 0x7f09900d1260 2024-04-20T21:18:35.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.701+0000 7f0995a00640 1 -- 172.21.15.86:0/3093883000 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0990154e10 con 0x7f09900d1260 2024-04-20T21:18:35.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.702+0000 7f0995a00640 1 -- 172.21.15.86:0/3093883000 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09900689e0 con 0x7f09900d1260 2024-04-20T21:18:35.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.703+0000 7f0986a00640 1 -- 172.21.15.86:0/3093883000 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0980007400 con 0x7f09900d1260 2024-04-20T21:18:35.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.704+0000 7f0986a00640 1 --2- 172.21.15.86:0/3093883000 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f097006c8f0 0x7f097006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:35.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.704+0000 7f0986a00640 1 -- 172.21.15.86:0/3093883000 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f098008b130 con 0x7f09900d1260 2024-04-20T21:18:35.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.704+0000 7f098e000640 1 --2- 172.21.15.86:0/3093883000 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f097006c8f0 0x7f097006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:35.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.705+0000 7f098e000640 1 --2- 172.21.15.86:0/3093883000 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f097006c8f0 0x7f097006edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f09900b8df0 tx=0x7f097c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:35.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.709+0000 7f0986a00640 1 -- 172.21.15.86:0/3093883000 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0990163970 con 0x7f09900d1260 2024-04-20T21:18:35.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.908+0000 7f0995a00640 1 -- 172.21.15.86:0/3093883000 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f09900d29c0 con 0x7f09900d1260 2024-04-20T21:18:35.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.909+0000 7f0986a00640 1 -- 172.21.15.86:0/3093883000 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f09800594e0 con 0x7f09900d1260 2024-04-20T21:18:35.909 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-20T21:18:35.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.912+0000 7f0995a00640 1 -- 172.21.15.86:0/3093883000 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f097006c8f0 msgr2=0x7f097006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:35.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.912+0000 7f0995a00640 1 --2- 172.21.15.86:0/3093883000 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f097006c8f0 0x7f097006edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f09900b8df0 tx=0x7f097c008040 comp rx=0 tx=0).stop 2024-04-20T21:18:35.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.912+0000 7f0995a00640 1 -- 172.21.15.86:0/3093883000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09900d1260 msgr2=0x7f09900cb920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:35.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.912+0000 7f0995a00640 1 --2- 172.21.15.86:0/3093883000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09900d1260 0x7f09900cb920 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f098000c430 tx=0x7f098000c8f0 comp rx=0 tx=0).stop 2024-04-20T21:18:35.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.912+0000 7f0995a00640 1 -- 172.21.15.86:0/3093883000 shutdown_connections 2024-04-20T21:18:35.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.912+0000 7f0995a00640 1 --2- 172.21.15.86:0/3093883000 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f097006c8f0 0x7f097006edb0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:35.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.912+0000 7f0995a00640 1 --2- 172.21.15.86:0/3093883000 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09901634c0 0x7f0990152f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:35.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.912+0000 7f0995a00640 1 --2- 172.21.15.86:0/3093883000 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f099015ee00 0x7f09901580c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:35.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.912+0000 7f0995a00640 1 --2- 172.21.15.86:0/3093883000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09900d1260 0x7f09900cb920 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:35.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.912+0000 7f0995a00640 1 -- 172.21.15.86:0/3093883000 >> 172.21.15.86:0/3093883000 conn(0x7f099006cc20 msgr2=0x7f09900b93b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:35.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.912+0000 7f0995a00640 1 -- 172.21.15.86:0/3093883000 shutdown_connections 2024-04-20T21:18:35.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:35.912+0000 7f0995a00640 1 -- 172.21.15.86:0/3093883000 wait complete. 2024-04-20T21:18:35.930 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:18:36.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.135+0000 7fa1e1400640 1 Processor -- start 2024-04-20T21:18:36.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.136+0000 7fa1e1400640 1 -- start start 2024-04-20T21:18:36.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.137+0000 7fa1e1400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa1dc0af1f0 0x7fa1dc06ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:36.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.137+0000 7fa1e1400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa1dc0af5d0 0x7fa1dc15de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:36.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.137+0000 7fa1e1400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1dc06de80 con 0x7fa1dc06d9c0 2024-04-20T21:18:36.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.137+0000 7fa1e1400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa1dc06f460 con 0x7fa1dc0af5d0 2024-04-20T21:18:36.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.137+0000 7fa1e1400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fa1dc06e160 con 0x7fa1dc0af1f0 2024-04-20T21:18:36.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.138+0000 7fa1e0a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa1dc0af5d0 0x7fa1dc15de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:36.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.138+0000 7fa1e0a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa1dc0af5d0 0x7fa1dc15de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35838/0 (socket says 172.21.15.86:35838) 2024-04-20T21:18:36.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.138+0000 7fa1e0a00640 1 -- 172.21.15.86:0/501168956 learned_addr learned my addr 172.21.15.86:0/501168956 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:36.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.138+0000 7fa1dbe00640 1 --2- 172.21.15.86:0/501168956 >> v2:172.21.15.144:3300/0 conn(0x7fa1dc0af1f0 0x7fa1dc06ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:36.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.138+0000 7fa1e0a00640 1 -- 172.21.15.86:0/501168956 >> v2:172.21.15.144:3300/0 conn(0x7fa1dc0af1f0 msgr2=0x7fa1dc06ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:36.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.138+0000 7fa1e0a00640 1 --2- 172.21.15.86:0/501168956 >> v2:172.21.15.144:3300/0 conn(0x7fa1dc0af1f0 0x7fa1dc06ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:36.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.138+0000 7fa1e0a00640 1 -- 172.21.15.86:0/501168956 >> v1:172.21.15.144:6789/0 conn(0x7fa1dc06d9c0 legacy=0x7fa1dc15d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:36.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.138+0000 7fa1e0a00640 1 -- 172.21.15.86:0/501168956 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1dc06e440 con 0x7fa1dc0af5d0 2024-04-20T21:18:36.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.139+0000 7fa1e0a00640 1 --2- 172.21.15.86:0/501168956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1dc0af5d0 0x7fa1dc15de70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc06d3d0 tx=0x7fa1d402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89b5e95a65011c76 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:36.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.139+0000 7fa1daa00640 1 -- 172.21.15.86:0/501168956 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d4035020 con 0x7fa1dc0af5d0 2024-04-20T21:18:36.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.140+0000 7fa1daa00640 1 -- 172.21.15.86:0/501168956 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1d4031870 con 0x7fa1dc0af5d0 2024-04-20T21:18:36.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.140+0000 7fa1daa00640 1 -- 172.21.15.86:0/501168956 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d4031c40 con 0x7fa1dc0af5d0 2024-04-20T21:18:36.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.140+0000 7fa1e1400640 1 -- 172.21.15.86:0/501168956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1dc0af5d0 msgr2=0x7fa1dc15de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:36.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.140+0000 7fa1e1400640 1 --2- 172.21.15.86:0/501168956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1dc0af5d0 0x7fa1dc15de70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc06d3d0 tx=0x7fa1d402b910 comp rx=0 tx=0).stop 2024-04-20T21:18:36.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.140+0000 7fa1e1400640 1 -- 172.21.15.86:0/501168956 shutdown_connections 2024-04-20T21:18:36.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.140+0000 7fa1e1400640 1 --2- 172.21.15.86:0/501168956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1dc0af5d0 0x7fa1dc15de70 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:36.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.140+0000 7fa1e1400640 1 --2- 172.21.15.86:0/501168956 >> v2:172.21.15.144:3300/0 conn(0x7fa1dc0af1f0 0x7fa1dc06ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:36.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.140+0000 7fa1e1400640 1 -- 172.21.15.86:0/501168956 >> 172.21.15.86:0/501168956 conn(0x7fa1dc0b1f00 msgr2=0x7fa1dc0b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:36.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.140+0000 7fa1e1400640 1 -- 172.21.15.86:0/501168956 shutdown_connections 2024-04-20T21:18:36.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.140+0000 7fa1e1400640 1 -- 172.21.15.86:0/501168956 wait complete. 2024-04-20T21:18:36.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.142+0000 7fa1e1400640 1 Processor -- start 2024-04-20T21:18:36.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.142+0000 7fa1e1400640 1 -- start start 2024-04-20T21:18:36.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.143+0000 7fa1e1400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa1dc06d9c0 0x7fa1dc142fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:36.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.143+0000 7fa1e1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1dc0af1f0 0x7fa1dc1f4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:36.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.143+0000 7fa1e1400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1dc0af5d0 0x7fa1dc15c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:36.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.143+0000 7fa1e1400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa1dc0afa40 con 0x7fa1dc0af1f0 2024-04-20T21:18:36.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.143+0000 7fa1e1400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa1dc06ea00 con 0x7fa1dc0af5d0 2024-04-20T21:18:36.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.143+0000 7fa1e1400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa1dc15a3c0 con 0x7fa1dc06d9c0 2024-04-20T21:18:36.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.143+0000 7fa1db400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1dc0af1f0 0x7fa1dc1f4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:36.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.143+0000 7fa1db400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1dc0af1f0 0x7fa1dc1f4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35848/0 (socket says 172.21.15.86:35848) 2024-04-20T21:18:36.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.143+0000 7fa1db400640 1 -- 172.21.15.86:0/2589247815 learned_addr learned my addr 172.21.15.86:0/2589247815 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:36.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.144+0000 7fa1dbe00640 1 --2- 172.21.15.86:0/2589247815 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa1dc06d9c0 0x7fa1dc142fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:36.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.144+0000 7fa1e0a00640 1 --2- 172.21.15.86:0/2589247815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1dc0af5d0 0x7fa1dc15c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:36.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.144+0000 7fa1db400640 1 -- 172.21.15.86:0/2589247815 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa1dc06d9c0 msgr2=0x7fa1dc142fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:36.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.144+0000 7fa1db400640 1 --2- 172.21.15.86:0/2589247815 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa1dc06d9c0 0x7fa1dc142fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:36.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.144+0000 7fa1db400640 1 -- 172.21.15.86:0/2589247815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1dc0af5d0 msgr2=0x7fa1dc15c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:36.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.144+0000 7fa1db400640 1 --2- 172.21.15.86:0/2589247815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1dc0af5d0 0x7fa1dc15c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:36.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.144+0000 7fa1db400640 1 -- 172.21.15.86:0/2589247815 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1dc15f0f0 con 0x7fa1dc0af1f0 2024-04-20T21:18:36.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.144+0000 7fa1e0a00640 1 --2- 172.21.15.86:0/2589247815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1dc0af5d0 0x7fa1dc15c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:36.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.144+0000 7fa1db400640 1 --2- 172.21.15.86:0/2589247815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1dc0af1f0 0x7fa1dc1f4220 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa1c400b420 tx=0x7fa1c400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:36.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.144+0000 7fa1d3e00640 1 -- 172.21.15.86:0/2589247815 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c4011830 con 0x7fa1dc0af1f0 2024-04-20T21:18:36.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.145+0000 7fa1d3e00640 1 -- 172.21.15.86:0/2589247815 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1c4016070 con 0x7fa1dc0af1f0 2024-04-20T21:18:36.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.145+0000 7fa1d3e00640 1 -- 172.21.15.86:0/2589247815 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c4011c10 con 0x7fa1dc0af1f0 2024-04-20T21:18:36.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.145+0000 7fa1e1400640 1 -- 172.21.15.86:0/2589247815 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1dc15f3d0 con 0x7fa1dc0af1f0 2024-04-20T21:18:36.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.145+0000 7fa1e1400640 1 -- 172.21.15.86:0/2589247815 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1dc1435b0 con 0x7fa1dc0af1f0 2024-04-20T21:18:36.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.146+0000 7fa1e1400640 1 -- 172.21.15.86:0/2589247815 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1dc15fa90 con 0x7fa1dc0af1f0 2024-04-20T21:18:36.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.147+0000 7fa1d3e00640 1 -- 172.21.15.86:0/2589247815 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa1c401b460 con 0x7fa1dc0af1f0 2024-04-20T21:18:36.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.148+0000 7fa1d3e00640 1 --2- 172.21.15.86:0/2589247815 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa1b806c920 0x7fa1b806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:36.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.148+0000 7fa1d3e00640 1 -- 172.21.15.86:0/2589247815 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fa1c408b930 con 0x7fa1dc0af1f0 2024-04-20T21:18:36.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.149+0000 7fa1dbe00640 1 --2- 172.21.15.86:0/2589247815 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa1b806c920 0x7fa1b806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:36.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.149+0000 7fa1dbe00640 1 --2- 172.21.15.86:0/2589247815 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa1b806c920 0x7fa1b806ede0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc0099c0 tx=0x7fa1cc003990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:36.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.153+0000 7fa1d3e00640 1 -- 172.21.15.86:0/2589247815 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa1c4059c40 con 0x7fa1dc0af1f0 2024-04-20T21:18:36.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.351+0000 7fa1e1400640 1 -- 172.21.15.86:0/2589247815 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa1dc0b3300 con 0x7fa1dc0af1f0 2024-04-20T21:18:36.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.352+0000 7fa1d3e00640 1 -- 172.21.15.86:0/2589247815 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7fa1c4059540 con 0x7fa1dc0af1f0 2024-04-20T21:18:36.352 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-20T21:18:36.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.354+0000 7fa1e1400640 1 -- 172.21.15.86:0/2589247815 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa1b806c920 msgr2=0x7fa1b806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:36.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.354+0000 7fa1e1400640 1 --2- 172.21.15.86:0/2589247815 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa1b806c920 0x7fa1b806ede0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc0099c0 tx=0x7fa1cc003990 comp rx=0 tx=0).stop 2024-04-20T21:18:36.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.354+0000 7fa1e1400640 1 -- 172.21.15.86:0/2589247815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1dc0af1f0 msgr2=0x7fa1dc1f4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:36.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.354+0000 7fa1e1400640 1 --2- 172.21.15.86:0/2589247815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1dc0af1f0 0x7fa1dc1f4220 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa1c400b420 tx=0x7fa1c400b8e0 comp rx=0 tx=0).stop 2024-04-20T21:18:36.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.354+0000 7fa1e1400640 1 -- 172.21.15.86:0/2589247815 shutdown_connections 2024-04-20T21:18:36.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.354+0000 7fa1e1400640 1 --2- 172.21.15.86:0/2589247815 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa1b806c920 0x7fa1b806ede0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:36.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.354+0000 7fa1e1400640 1 --2- 172.21.15.86:0/2589247815 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1dc0af5d0 0x7fa1dc15c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:36.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.354+0000 7fa1e1400640 1 --2- 172.21.15.86:0/2589247815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1dc0af1f0 0x7fa1dc1f4220 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:36.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.354+0000 7fa1e1400640 1 --2- 172.21.15.86:0/2589247815 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa1dc06d9c0 0x7fa1dc142fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:36.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.354+0000 7fa1e1400640 1 -- 172.21.15.86:0/2589247815 >> 172.21.15.86:0/2589247815 conn(0x7fa1dc0b1f00 msgr2=0x7fa1dc160770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:36.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.354+0000 7fa1e1400640 1 -- 172.21.15.86:0/2589247815 shutdown_connections 2024-04-20T21:18:36.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.354+0000 7fa1e1400640 1 -- 172.21.15.86:0/2589247815 wait complete. 2024-04-20T21:18:36.373 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T21:18:36.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.573+0000 7fde81a00640 1 Processor -- start 2024-04-20T21:18:36.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.574+0000 7fde81a00640 1 -- start start 2024-04-20T21:18:36.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.575+0000 7fde81a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde7c0d4230 0x7fde7c0d2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:36.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.575+0000 7fde81a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fde7c0b7a80 0x7fde7c0b7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:36.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.575+0000 7fde81a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde7c0d4690 con 0x7fde7c0d1360 2024-04-20T21:18:36.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.575+0000 7fde81a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fde7c0d2a60 con 0x7fde7c0b7a80 2024-04-20T21:18:36.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.575+0000 7fde81a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fde7c0d2c90 con 0x7fde7c0d4230 2024-04-20T21:18:36.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.575+0000 7fde7b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fde7c0b7a80 0x7fde7c0b7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:36.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.575+0000 7fde7b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fde7c0b7a80 0x7fde7c0b7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35852/0 (socket says 172.21.15.86:35852) 2024-04-20T21:18:36.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.575+0000 7fde7b400640 1 -- 172.21.15.86:0/2222984264 learned_addr learned my addr 172.21.15.86:0/2222984264 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:36.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.576+0000 7fde7a000640 1 --2- 172.21.15.86:0/2222984264 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde7c0d4230 0x7fde7c0d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:36.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.576+0000 7fde7b400640 1 -- 172.21.15.86:0/2222984264 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde7c0d4230 msgr2=0x7fde7c0d2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:36.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.576+0000 7fde7b400640 1 --2- 172.21.15.86:0/2222984264 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde7c0d4230 0x7fde7c0d2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:36.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.576+0000 7fde7b400640 1 -- 172.21.15.86:0/2222984264 >> v1:172.21.15.144:6789/0 conn(0x7fde7c0d1360 legacy=0x7fde7c0d1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:36.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.576+0000 7fde7b400640 1 -- 172.21.15.86:0/2222984264 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde7c0d2f70 con 0x7fde7c0b7a80 2024-04-20T21:18:36.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.577+0000 7fde7b400640 1 --2- 172.21.15.86:0/2222984264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde7c0b7a80 0x7fde7c0b7f10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fde74009870 tx=0x7fde7402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a6e3b35ec2b128e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:36.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.578+0000 7fde79600640 1 -- 172.21.15.86:0/2222984264 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde74034070 con 0x7fde7c0b7a80 2024-04-20T21:18:36.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.578+0000 7fde79600640 1 -- 172.21.15.86:0/2222984264 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde74030840 con 0x7fde7c0b7a80 2024-04-20T21:18:36.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.578+0000 7fde79600640 1 -- 172.21.15.86:0/2222984264 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde74030c10 con 0x7fde7c0b7a80 2024-04-20T21:18:36.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.578+0000 7fde81a00640 1 -- 172.21.15.86:0/2222984264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde7c0b7a80 msgr2=0x7fde7c0b7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:36.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.578+0000 7fde81a00640 1 --2- 172.21.15.86:0/2222984264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde7c0b7a80 0x7fde7c0b7f10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fde74009870 tx=0x7fde7402b980 comp rx=0 tx=0).stop 2024-04-20T21:18:36.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.578+0000 7fde81a00640 1 -- 172.21.15.86:0/2222984264 shutdown_connections 2024-04-20T21:18:36.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.578+0000 7fde81a00640 1 --2- 172.21.15.86:0/2222984264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde7c0b7a80 0x7fde7c0b7f10 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:36.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.578+0000 7fde81a00640 1 --2- 172.21.15.86:0/2222984264 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde7c0d4230 0x7fde7c0d2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:36.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.578+0000 7fde81a00640 1 -- 172.21.15.86:0/2222984264 >> 172.21.15.86:0/2222984264 conn(0x7fde7c0b64a0 msgr2=0x7fde7c0b68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:36.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.579+0000 7fde81a00640 1 -- 172.21.15.86:0/2222984264 shutdown_connections 2024-04-20T21:18:36.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.579+0000 7fde81a00640 1 -- 172.21.15.86:0/2222984264 wait complete. 2024-04-20T21:18:36.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.580+0000 7fde81a00640 1 Processor -- start 2024-04-20T21:18:36.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.580+0000 7fde81a00640 1 -- start start 2024-04-20T21:18:36.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.581+0000 7fde81a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde7c0b7a80 0x7fde7c0cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:36.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.581+0000 7fde81a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde7c0d1360 0x7fde7c1581f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:36.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.581+0000 7fde81a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde7c0d4230 0x7fde7c158730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:36.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.581+0000 7fde81a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c0d3810 con 0x7fde7c0d4230 2024-04-20T21:18:36.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.581+0000 7fde81a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c0d3530 con 0x7fde7c0b7a80 2024-04-20T21:18:36.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.581+0000 7fde81a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fde7c1620c0 con 0x7fde7c0d1360 2024-04-20T21:18:36.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.582+0000 7fde7b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde7c0d4230 0x7fde7c158730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:36.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.582+0000 7fde7b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde7c0d4230 0x7fde7c158730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35866/0 (socket says 172.21.15.86:35866) 2024-04-20T21:18:36.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.582+0000 7fde7b400640 1 -- 172.21.15.86:0/978681954 learned_addr learned my addr 172.21.15.86:0/978681954 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:36.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.582+0000 7fde7aa00640 1 --2- 172.21.15.86:0/978681954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde7c0b7a80 0x7fde7c0cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:36.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.582+0000 7fde7a000640 1 --2- 172.21.15.86:0/978681954 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde7c0d1360 0x7fde7c1581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:36.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.582+0000 7fde7b400640 1 -- 172.21.15.86:0/978681954 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde7c0d1360 msgr2=0x7fde7c1581f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:36.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.582+0000 7fde7b400640 1 --2- 172.21.15.86:0/978681954 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde7c0d1360 0x7fde7c1581f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:36.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.582+0000 7fde7b400640 1 -- 172.21.15.86:0/978681954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde7c0b7a80 msgr2=0x7fde7c0cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:36.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.582+0000 7fde7b400640 1 --2- 172.21.15.86:0/978681954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde7c0b7a80 0x7fde7c0cd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:36.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.582+0000 7fde7b400640 1 -- 172.21.15.86:0/978681954 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde7c15a8d0 con 0x7fde7c0d4230 2024-04-20T21:18:36.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.582+0000 7fde7aa00640 1 --2- 172.21.15.86:0/978681954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde7c0b7a80 0x7fde7c0cd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:18:36.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.583+0000 7fde7b400640 1 --2- 172.21.15.86:0/978681954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde7c0d4230 0x7fde7c158730 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fde74009870 tx=0x7fde74004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:36.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.583+0000 7fde72a00640 1 -- 172.21.15.86:0/978681954 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde74048070 con 0x7fde7c0d4230 2024-04-20T21:18:36.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.583+0000 7fde72a00640 1 -- 172.21.15.86:0/978681954 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde74030640 con 0x7fde7c0d4230 2024-04-20T21:18:36.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.583+0000 7fde72a00640 1 -- 172.21.15.86:0/978681954 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde74034040 con 0x7fde7c0d4230 2024-04-20T21:18:36.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.583+0000 7fde81a00640 1 -- 172.21.15.86:0/978681954 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde7c15abb0 con 0x7fde7c0d4230 2024-04-20T21:18:36.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.583+0000 7fde81a00640 1 -- 172.21.15.86:0/978681954 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde7c15b170 con 0x7fde7c0d4230 2024-04-20T21:18:36.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.584+0000 7fde81a00640 1 -- 172.21.15.86:0/978681954 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde7c0689e0 con 0x7fde7c0d4230 2024-04-20T21:18:36.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.585+0000 7fde72a00640 1 -- 172.21.15.86:0/978681954 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fde7404b070 con 0x7fde7c0d4230 2024-04-20T21:18:36.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.586+0000 7fde72a00640 1 --2- 172.21.15.86:0/978681954 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fde5806c8f0 0x7fde5806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:36.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.586+0000 7fde72a00640 1 -- 172.21.15.86:0/978681954 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fde740adb30 con 0x7fde7c0d4230 2024-04-20T21:18:36.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.586+0000 7fde7aa00640 1 --2- 172.21.15.86:0/978681954 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fde5806c8f0 0x7fde5806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:36.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.587+0000 7fde7aa00640 1 --2- 172.21.15.86:0/978681954 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fde5806c8f0 0x7fde5806edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fde64001700 tx=0x7fde64009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:36.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.590+0000 7fde72a00640 1 -- 172.21.15.86:0/978681954 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fde7407bed0 con 0x7fde7c0d4230 2024-04-20T21:18:36.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.807+0000 7fde81a00640 1 -- 172.21.15.86:0/978681954 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fde7c0b6210 con 0x7fde7c0d4230 2024-04-20T21:18:36.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.808+0000 7fde72a00640 1 -- 172.21.15.86:0/978681954 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7fde7407b7d0 con 0x7fde7c0d4230 2024-04-20T21:18:36.808 INFO:teuthology.orchestra.run.smithi086.stderr:listed 0 entries 2024-04-20T21:18:36.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.811+0000 7fde81a00640 1 -- 172.21.15.86:0/978681954 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fde5806c8f0 msgr2=0x7fde5806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:36.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.811+0000 7fde81a00640 1 --2- 172.21.15.86:0/978681954 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fde5806c8f0 0x7fde5806edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fde64001700 tx=0x7fde64009040 comp rx=0 tx=0).stop 2024-04-20T21:18:36.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.811+0000 7fde81a00640 1 -- 172.21.15.86:0/978681954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde7c0d4230 msgr2=0x7fde7c158730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:36.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.811+0000 7fde81a00640 1 --2- 172.21.15.86:0/978681954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde7c0d4230 0x7fde7c158730 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fde74009870 tx=0x7fde74004730 comp rx=0 tx=0).stop 2024-04-20T21:18:36.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.811+0000 7fde81a00640 1 -- 172.21.15.86:0/978681954 shutdown_connections 2024-04-20T21:18:36.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.811+0000 7fde81a00640 1 --2- 172.21.15.86:0/978681954 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fde5806c8f0 0x7fde5806edb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:36.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.811+0000 7fde81a00640 1 --2- 172.21.15.86:0/978681954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde7c0d4230 0x7fde7c158730 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:36.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.811+0000 7fde81a00640 1 --2- 172.21.15.86:0/978681954 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde7c0d1360 0x7fde7c1581f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:36.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.811+0000 7fde81a00640 1 --2- 172.21.15.86:0/978681954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde7c0b7a80 0x7fde7c0cd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:36.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.811+0000 7fde81a00640 1 -- 172.21.15.86:0/978681954 >> 172.21.15.86:0/978681954 conn(0x7fde7c0b64a0 msgr2=0x7fde7c0b45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:36.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.812+0000 7fde81a00640 1 -- 172.21.15.86:0/978681954 shutdown_connections 2024-04-20T21:18:36.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:36.812+0000 7fde81a00640 1 -- 172.21.15.86:0/978681954 wait complete. 2024-04-20T21:18:36.832 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:18:36.832 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:18:36.878 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:18:37.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.120+0000 7f8807e00640 1 Processor -- start 2024-04-20T21:18:37.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.120+0000 7f8807e00640 1 -- start start 2024-04-20T21:18:37.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.121+0000 7f8807e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8808156ae0 0x7f880815d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:37.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.122+0000 7f8807e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8808160080 0x7f88080ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:37.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.122+0000 7f8807e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8808160890 con 0x7f88080ad620 2024-04-20T21:18:37.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.122+0000 7f8807e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f880806db20 con 0x7f8808160080 2024-04-20T21:18:37.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.122+0000 7f8807e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8808160b70 con 0x7f8808156ae0 2024-04-20T21:18:37.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.122+0000 7f8806000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8808160080 0x7f88080ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:37.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.122+0000 7f8806000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8808160080 0x7f88080ad0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35880/0 (socket says 172.21.15.86:35880) 2024-04-20T21:18:37.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.122+0000 7f8806000640 1 -- 172.21.15.86:0/1864574597 learned_addr learned my addr 172.21.15.86:0/1864574597 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:37.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.122+0000 7f8806a00640 1 --2- 172.21.15.86:0/1864574597 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8808156ae0 0x7f880815d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:37.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.122+0000 7f8806000640 1 -- 172.21.15.86:0/1864574597 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8808156ae0 msgr2=0x7f880815d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:37.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.122+0000 7f8806000640 1 --2- 172.21.15.86:0/1864574597 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8808156ae0 0x7f880815d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:37.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.122+0000 7f8806000640 1 -- 172.21.15.86:0/1864574597 >> v1:172.21.15.86:6789/0 conn(0x7f88080ad620 legacy=0x7f8808141cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:37.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.123+0000 7f8806000640 1 -- 172.21.15.86:0/1864574597 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8808160e50 con 0x7f8808160080 2024-04-20T21:18:37.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.124+0000 7f8806000640 1 --2- 172.21.15.86:0/1864574597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8808160080 0x7f88080ad0e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f87f4009870 tx=0x7f87f402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d15136d912acee server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:37.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.124+0000 7f8805600640 1 -- 172.21.15.86:0/1864574597 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f4034070 con 0x7f8808160080 2024-04-20T21:18:37.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.124+0000 7f8805600640 1 -- 172.21.15.86:0/1864574597 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87f4030840 con 0x7f8808160080 2024-04-20T21:18:37.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.124+0000 7f8805600640 1 -- 172.21.15.86:0/1864574597 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f4030c10 con 0x7f8808160080 2024-04-20T21:18:37.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.124+0000 7f8807e00640 1 -- 172.21.15.86:0/1864574597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8808160080 msgr2=0x7f88080ad0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:37.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.125+0000 7f8807e00640 1 --2- 172.21.15.86:0/1864574597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8808160080 0x7f88080ad0e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f87f4009870 tx=0x7f87f402b980 comp rx=0 tx=0).stop 2024-04-20T21:18:37.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.125+0000 7f8807e00640 1 -- 172.21.15.86:0/1864574597 shutdown_connections 2024-04-20T21:18:37.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.125+0000 7f8807e00640 1 --2- 172.21.15.86:0/1864574597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8808160080 0x7f88080ad0e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:37.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.125+0000 7f8807e00640 1 --2- 172.21.15.86:0/1864574597 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8808156ae0 0x7f880815d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:37.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.125+0000 7f8807e00640 1 -- 172.21.15.86:0/1864574597 >> 172.21.15.86:0/1864574597 conn(0x7f88080d0170 msgr2=0x7f88080d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:37.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.125+0000 7f8807e00640 1 -- 172.21.15.86:0/1864574597 shutdown_connections 2024-04-20T21:18:37.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.125+0000 7f8807e00640 1 -- 172.21.15.86:0/1864574597 wait complete. 2024-04-20T21:18:37.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.127+0000 7f8807e00640 1 Processor -- start 2024-04-20T21:18:37.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.127+0000 7f8807e00640 1 -- start start 2024-04-20T21:18:37.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.128+0000 7f8807e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88080ad620 0x7f88081f3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:37.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.128+0000 7f8807e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8808156ae0 0x7f88080b4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:37.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.128+0000 7f8807e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8808160080 0x7f88080b4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:37.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.128+0000 7f8807e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8808160500 con 0x7f88080ad620 2024-04-20T21:18:37.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.128+0000 7f8807e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8808156ec0 con 0x7f8808156ae0 2024-04-20T21:18:37.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.128+0000 7f8807e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f880806f2a0 con 0x7f8808160080 2024-04-20T21:18:37.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.128+0000 7f8806a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88080ad620 0x7f88081f3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:37.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.128+0000 7f8806a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88080ad620 0x7f88081f3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35882/0 (socket says 172.21.15.86:35882) 2024-04-20T21:18:37.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.128+0000 7f8806a00640 1 -- 172.21.15.86:0/283809589 learned_addr learned my addr 172.21.15.86:0/283809589 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:37.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.129+0000 7f8806a00640 1 -- 172.21.15.86:0/283809589 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8808160080 msgr2=0x7f88080b4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:37.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.129+0000 7f8806a00640 1 --2- 172.21.15.86:0/283809589 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8808160080 0x7f88080b4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:37.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.129+0000 7f8806a00640 1 -- 172.21.15.86:0/283809589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8808156ae0 msgr2=0x7f88080b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:37.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.129+0000 7f8806a00640 1 --2- 172.21.15.86:0/283809589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8808156ae0 0x7f88080b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:37.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.129+0000 7f8806a00640 1 -- 172.21.15.86:0/283809589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8808157cf0 con 0x7f88080ad620 2024-04-20T21:18:37.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.129+0000 7f8806a00640 1 --2- 172.21.15.86:0/283809589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88080ad620 0x7f88081f3fa0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f87f800d800 tx=0x7f87f800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:37.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.129+0000 7f87fea00640 1 -- 172.21.15.86:0/283809589 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f800b6c0 con 0x7f88080ad620 2024-04-20T21:18:37.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.129+0000 7f87fea00640 1 -- 172.21.15.86:0/283809589 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87f8019070 con 0x7f88080ad620 2024-04-20T21:18:37.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.129+0000 7f87fea00640 1 -- 172.21.15.86:0/283809589 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f800ba60 con 0x7f88080ad620 2024-04-20T21:18:37.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.129+0000 7f8807e00640 1 -- 172.21.15.86:0/283809589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8808157fd0 con 0x7f88080ad620 2024-04-20T21:18:37.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.129+0000 7f8807e00640 1 -- 172.21.15.86:0/283809589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8808157a00 con 0x7f88080ad620 2024-04-20T21:18:37.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.131+0000 7f8807e00640 1 -- 172.21.15.86:0/283809589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f880804fcf0 con 0x7f88080ad620 2024-04-20T21:18:37.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.132+0000 7f87fea00640 1 -- 172.21.15.86:0/283809589 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f87f800bc00 con 0x7f88080ad620 2024-04-20T21:18:37.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.133+0000 7f87fea00640 1 --2- 172.21.15.86:0/283809589 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f87e806c8f0 0x7f87e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:37.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.133+0000 7f87fea00640 1 -- 172.21.15.86:0/283809589 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f87f808c0b0 con 0x7f88080ad620 2024-04-20T21:18:37.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.133+0000 7f8806000640 1 --2- 172.21.15.86:0/283809589 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f87e806c8f0 0x7f87e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:37.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.134+0000 7f8806000640 1 --2- 172.21.15.86:0/283809589 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f87e806c8f0 0x7f87e806edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f87f4036000 tx=0x7f87f403b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:37.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.137+0000 7f87fea00640 1 -- 172.21.15.86:0/283809589 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f87f805a450 con 0x7f88080ad620 2024-04-20T21:18:37.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.339+0000 7f8807e00640 1 -- 172.21.15.86:0/283809589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f880806fb40 con 0x7f88080ad620 2024-04-20T21:18:37.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.342+0000 7f87fea00640 1 -- 172.21.15.86:0/283809589 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16854 (secure 0 0 0) 0x7f87f8059d50 con 0x7f88080ad620 2024-04-20T21:18:37.342 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:18:37.343 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":22,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:18:33.788155+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:17:49.718799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:17:50.683477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:18:37.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.345+0000 7f8807e00640 1 -- 172.21.15.86:0/283809589 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f87e806c8f0 msgr2=0x7f87e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:37.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.345+0000 7f8807e00640 1 --2- 172.21.15.86:0/283809589 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f87e806c8f0 0x7f87e806edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f87f4036000 tx=0x7f87f403b040 comp rx=0 tx=0).stop 2024-04-20T21:18:37.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.345+0000 7f8807e00640 1 -- 172.21.15.86:0/283809589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88080ad620 msgr2=0x7f88081f3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:37.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.345+0000 7f8807e00640 1 --2- 172.21.15.86:0/283809589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88080ad620 0x7f88081f3fa0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f87f800d800 tx=0x7f87f800dcc0 comp rx=0 tx=0).stop 2024-04-20T21:18:37.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.345+0000 7f8807e00640 1 -- 172.21.15.86:0/283809589 shutdown_connections 2024-04-20T21:18:37.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.345+0000 7f8807e00640 1 --2- 172.21.15.86:0/283809589 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f87e806c8f0 0x7f87e806edb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:37.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.345+0000 7f8807e00640 1 --2- 172.21.15.86:0/283809589 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8808160080 0x7f88080b4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:37.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.345+0000 7f8807e00640 1 --2- 172.21.15.86:0/283809589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8808156ae0 0x7f88080b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:37.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.345+0000 7f8807e00640 1 --2- 172.21.15.86:0/283809589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88080ad620 0x7f88081f3fa0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:37.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.345+0000 7f8807e00640 1 -- 172.21.15.86:0/283809589 >> 172.21.15.86:0/283809589 conn(0x7f88080d0170 msgr2=0x7f88080b07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:37.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.345+0000 7f8807e00640 1 -- 172.21.15.86:0/283809589 shutdown_connections 2024-04-20T21:18:37.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.345+0000 7f8807e00640 1 -- 172.21.15.86:0/283809589 wait complete. 2024-04-20T21:18:37.364 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:17:49.718799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:17:50.683477+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:18:37.364 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:18:37.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.568+0000 7f8022800640 1 Processor -- start 2024-04-20T21:18:37.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.569+0000 7f8022800640 1 -- start start 2024-04-20T21:18:37.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.570+0000 7f8022800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f801c15ee00 0x7f801c1634c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:37.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.570+0000 7f8022800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f801c0d1260 0x7f801c0d16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:37.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.570+0000 7f8022800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f801c0cfce0 con 0x7f801c0cf830 2024-04-20T21:18:37.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.570+0000 7f8022800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f801c0d0550 con 0x7f801c15ee00 2024-04-20T21:18:37.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.570+0000 7f8022800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f801c0cff90 con 0x7f801c0d1260 2024-04-20T21:18:37.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.570+0000 7f801b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f801c15ee00 0x7f801c1634c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:37.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.571+0000 7f801b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f801c15ee00 0x7f801c1634c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35890/0 (socket says 172.21.15.86:35890) 2024-04-20T21:18:37.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.571+0000 7f801b400640 1 -- 172.21.15.86:0/789184406 learned_addr learned my addr 172.21.15.86:0/789184406 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:37.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.571+0000 7f801be00640 1 --2- 172.21.15.86:0/789184406 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f801c0d1260 0x7f801c0d16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:37.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.571+0000 7f801b400640 1 -- 172.21.15.86:0/789184406 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f801c0d1260 msgr2=0x7f801c0d16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:37.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.571+0000 7f801b400640 1 --2- 172.21.15.86:0/789184406 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f801c0d1260 0x7f801c0d16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:37.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.571+0000 7f801b400640 1 -- 172.21.15.86:0/789184406 >> v1:172.21.15.144:6789/0 conn(0x7f801c0cf830 legacy=0x7f801c15fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:37.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.571+0000 7f801b400640 1 -- 172.21.15.86:0/789184406 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f801c0d0270 con 0x7f801c15ee00 2024-04-20T21:18:37.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.572+0000 7f801b400640 1 --2- 172.21.15.86:0/789184406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f801c15ee00 0x7f801c1634c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f801c0d1eb0 tx=0x7f800c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc3ad56624384376 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:37.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.573+0000 7f801a000640 1 -- 172.21.15.86:0/789184406 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800c035020 con 0x7f801c15ee00 2024-04-20T21:18:37.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.573+0000 7f801a000640 1 -- 172.21.15.86:0/789184406 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f800c031830 con 0x7f801c15ee00 2024-04-20T21:18:37.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.573+0000 7f801a000640 1 -- 172.21.15.86:0/789184406 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800c031c00 con 0x7f801c15ee00 2024-04-20T21:18:37.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.573+0000 7f8022800640 1 -- 172.21.15.86:0/789184406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f801c15ee00 msgr2=0x7f801c1634c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:37.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.573+0000 7f8022800640 1 --2- 172.21.15.86:0/789184406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f801c15ee00 0x7f801c1634c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f801c0d1eb0 tx=0x7f800c02b8d0 comp rx=0 tx=0).stop 2024-04-20T21:18:37.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.573+0000 7f8022800640 1 -- 172.21.15.86:0/789184406 shutdown_connections 2024-04-20T21:18:37.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.573+0000 7f8022800640 1 --2- 172.21.15.86:0/789184406 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f801c0d1260 0x7f801c0d16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:37.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.573+0000 7f8022800640 1 --2- 172.21.15.86:0/789184406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f801c15ee00 0x7f801c1634c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:37.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.573+0000 7f8022800640 1 -- 172.21.15.86:0/789184406 >> 172.21.15.86:0/789184406 conn(0x7f801c06cc20 msgr2=0x7f801c06d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:37.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.573+0000 7f8022800640 1 -- 172.21.15.86:0/789184406 shutdown_connections 2024-04-20T21:18:37.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.573+0000 7f8022800640 1 -- 172.21.15.86:0/789184406 wait complete. 2024-04-20T21:18:37.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.575+0000 7f8022800640 1 Processor -- start 2024-04-20T21:18:37.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.575+0000 7f8022800640 1 -- start start 2024-04-20T21:18:37.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.576+0000 7f8022800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f801c0cf830 0x7f801c157e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:37.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.576+0000 7f8022800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f801c0d1260 0x7f801c1ea5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:37.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.576+0000 7f8022800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f801c15ee00 0x7f801c1eaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:37.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.576+0000 7f8022800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f801c15f360 con 0x7f801c0cf830 2024-04-20T21:18:37.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.576+0000 7f8022800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f801c15f1e0 con 0x7f801c0d1260 2024-04-20T21:18:37.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.576+0000 7f8022800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f801c0d3040 con 0x7f801c15ee00 2024-04-20T21:18:37.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.576+0000 7f801b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f801c0cf830 0x7f801c157e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:37.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.576+0000 7f801b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f801c0cf830 0x7f801c157e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35904/0 (socket says 172.21.15.86:35904) 2024-04-20T21:18:37.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.576+0000 7f801b400640 1 -- 172.21.15.86:0/1216502285 learned_addr learned my addr 172.21.15.86:0/1216502285 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:37.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.577+0000 7f801b400640 1 -- 172.21.15.86:0/1216502285 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f801c15ee00 msgr2=0x7f801c1eaaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:37.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.577+0000 7f801b400640 1 --2- 172.21.15.86:0/1216502285 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f801c15ee00 0x7f801c1eaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:37.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.577+0000 7f801b400640 1 -- 172.21.15.86:0/1216502285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f801c0d1260 msgr2=0x7f801c1ea5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:37.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.577+0000 7f801aa00640 1 --2- 172.21.15.86:0/1216502285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f801c0d1260 0x7f801c1ea5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:37.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.577+0000 7f801b400640 1 --2- 172.21.15.86:0/1216502285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f801c0d1260 0x7f801c1ea5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:37.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.577+0000 7f801b400640 1 -- 172.21.15.86:0/1216502285 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f801c1583c0 con 0x7f801c0cf830 2024-04-20T21:18:37.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.577+0000 7f801b400640 1 --2- 172.21.15.86:0/1216502285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f801c0cf830 0x7f801c157e80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f800c000c00 tx=0x7f800c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:37.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.577+0000 7f8013400640 1 -- 172.21.15.86:0/1216502285 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800c040070 con 0x7f801c0cf830 2024-04-20T21:18:37.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.578+0000 7f8013400640 1 -- 172.21.15.86:0/1216502285 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f800c0399c0 con 0x7f801c0cf830 2024-04-20T21:18:37.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.578+0000 7f8013400640 1 -- 172.21.15.86:0/1216502285 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800c035040 con 0x7f801c0cf830 2024-04-20T21:18:37.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.578+0000 7f8022800640 1 -- 172.21.15.86:0/1216502285 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f801c1f5070 con 0x7f801c0cf830 2024-04-20T21:18:37.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.578+0000 7f8022800640 1 -- 172.21.15.86:0/1216502285 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f801c1f5630 con 0x7f801c0cf830 2024-04-20T21:18:37.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.579+0000 7f8022800640 1 -- 172.21.15.86:0/1216502285 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f801c0689e0 con 0x7f801c0cf830 2024-04-20T21:18:37.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.580+0000 7f8013400640 1 -- 172.21.15.86:0/1216502285 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f800c04b060 con 0x7f801c0cf830 2024-04-20T21:18:37.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.581+0000 7f8013400640 1 --2- 172.21.15.86:0/1216502285 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7ff806c8f0 0x7f7ff806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:37.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.581+0000 7f8013400640 1 -- 172.21.15.86:0/1216502285 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f800c0ad5b0 con 0x7f801c0cf830 2024-04-20T21:18:37.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.581+0000 7f801aa00640 1 --2- 172.21.15.86:0/1216502285 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7ff806c8f0 0x7f7ff806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:37.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.581+0000 7f801aa00640 1 --2- 172.21.15.86:0/1216502285 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7ff806c8f0 0x7f7ff806edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f801c160740 tx=0x7f8004009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:37.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.586+0000 7f8013400640 1 -- 172.21.15.86:0/1216502285 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f800c07b9f0 con 0x7f801c0cf830 2024-04-20T21:18:37.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.781+0000 7f8022800640 1 -- 172.21.15.86:0/1216502285 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f801c0d29c0 con 0x7f801c0cf830 2024-04-20T21:18:37.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.782+0000 7f8013400640 1 -- 172.21.15.86:0/1216502285 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f800c07b2f0 con 0x7f801c0cf830 2024-04-20T21:18:37.782 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:18:37.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.784+0000 7f8022800640 1 -- 172.21.15.86:0/1216502285 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7ff806c8f0 msgr2=0x7f7ff806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:37.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.784+0000 7f8022800640 1 --2- 172.21.15.86:0/1216502285 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7ff806c8f0 0x7f7ff806edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f801c160740 tx=0x7f8004009040 comp rx=0 tx=0).stop 2024-04-20T21:18:37.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.784+0000 7f8022800640 1 -- 172.21.15.86:0/1216502285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f801c0cf830 msgr2=0x7f801c157e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:37.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.784+0000 7f8022800640 1 --2- 172.21.15.86:0/1216502285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f801c0cf830 0x7f801c157e80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f800c000c00 tx=0x7f800c0046c0 comp rx=0 tx=0).stop 2024-04-20T21:18:37.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.784+0000 7f8022800640 1 -- 172.21.15.86:0/1216502285 shutdown_connections 2024-04-20T21:18:37.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.784+0000 7f8022800640 1 --2- 172.21.15.86:0/1216502285 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7ff806c8f0 0x7f7ff806edb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:37.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.784+0000 7f8022800640 1 --2- 172.21.15.86:0/1216502285 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f801c15ee00 0x7f801c1eaaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:37.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.784+0000 7f8022800640 1 --2- 172.21.15.86:0/1216502285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f801c0d1260 0x7f801c1ea5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:37.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.784+0000 7f8022800640 1 --2- 172.21.15.86:0/1216502285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f801c0cf830 0x7f801c157e80 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:37.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.784+0000 7f8022800640 1 -- 172.21.15.86:0/1216502285 >> 172.21.15.86:0/1216502285 conn(0x7f801c06cc20 msgr2=0x7f801c0b9a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:37.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.785+0000 7f8022800640 1 -- 172.21.15.86:0/1216502285 shutdown_connections 2024-04-20T21:18:37.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.785+0000 7f8022800640 1 -- 172.21.15.86:0/1216502285 wait complete. 2024-04-20T21:18:37.803 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:18:37.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.989+0000 7f6c2ba00640 1 Processor -- start 2024-04-20T21:18:37.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.989+0000 7f6c2ba00640 1 -- start start 2024-04-20T21:18:37.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.990+0000 7f6c2ba00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c240b03e0 0x7f6c2415f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:37.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.990+0000 7f6c2ba00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6c240b4360 0x7f6c24162a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:37.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.990+0000 7f6c2ba00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c2406e760 con 0x7f6c240b3ee0 2024-04-20T21:18:37.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.990+0000 7f6c2ba00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f6c2406ea40 con 0x7f6c240b4360 2024-04-20T21:18:37.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.990+0000 7f6c2ba00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6c2406ed20 con 0x7f6c240b03e0 2024-04-20T21:18:37.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.991+0000 7f6c29a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6c240b4360 0x7f6c24162a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:37.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.991+0000 7f6c29a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6c240b4360 0x7f6c24162a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35908/0 (socket says 172.21.15.86:35908) 2024-04-20T21:18:37.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.991+0000 7f6c29a00640 1 -- 172.21.15.86:0/2987030449 learned_addr learned my addr 172.21.15.86:0/2987030449 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:37.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.991+0000 7f6c29000640 1 --2- 172.21.15.86:0/2987030449 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c240b03e0 0x7f6c2415f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:37.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.991+0000 7f6c29a00640 1 -- 172.21.15.86:0/2987030449 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c240b03e0 msgr2=0x7f6c2415f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:37.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.991+0000 7f6c29a00640 1 --2- 172.21.15.86:0/2987030449 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c240b03e0 0x7f6c2415f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:37.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.991+0000 7f6c29a00640 1 -- 172.21.15.86:0/2987030449 >> v1:172.21.15.144:6789/0 conn(0x7f6c240b3ee0 legacy=0x7f6c24162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:37.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.992+0000 7f6c29a00640 1 -- 172.21.15.86:0/2987030449 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c2406f000 con 0x7f6c240b4360 2024-04-20T21:18:37.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.992+0000 7f6c29000640 1 --2- 172.21.15.86:0/2987030449 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c240b03e0 0x7f6c2415f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:37.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.993+0000 7f6c29a00640 1 --2- 172.21.15.86:0/2987030449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c240b4360 0x7f6c24162a70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c009870 tx=0x7f6c1c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10c4cb740cff7e62 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:37.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.993+0000 7f6c23400640 1 -- 172.21.15.86:0/2987030449 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c035020 con 0x7f6c240b4360 2024-04-20T21:18:37.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.993+0000 7f6c23400640 1 -- 172.21.15.86:0/2987030449 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c1c02d040 con 0x7f6c240b4360 2024-04-20T21:18:37.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.993+0000 7f6c23400640 1 -- 172.21.15.86:0/2987030449 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c031a80 con 0x7f6c240b4360 2024-04-20T21:18:37.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.994+0000 7f6c2ba00640 1 -- 172.21.15.86:0/2987030449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c240b4360 msgr2=0x7f6c24162a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:37.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.994+0000 7f6c2ba00640 1 --2- 172.21.15.86:0/2987030449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c240b4360 0x7f6c24162a70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c009870 tx=0x7f6c1c02b900 comp rx=0 tx=0).stop 2024-04-20T21:18:37.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.994+0000 7f6c2ba00640 1 -- 172.21.15.86:0/2987030449 shutdown_connections 2024-04-20T21:18:37.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.994+0000 7f6c2ba00640 1 --2- 172.21.15.86:0/2987030449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c240b4360 0x7f6c24162a70 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:37.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.994+0000 7f6c2ba00640 1 --2- 172.21.15.86:0/2987030449 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c240b03e0 0x7f6c2415f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:37.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.994+0000 7f6c2ba00640 1 -- 172.21.15.86:0/2987030449 >> 172.21.15.86:0/2987030449 conn(0x7f6c240b3380 msgr2=0x7f6c240d0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:37.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.994+0000 7f6c2ba00640 1 -- 172.21.15.86:0/2987030449 shutdown_connections 2024-04-20T21:18:37.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.994+0000 7f6c2ba00640 1 -- 172.21.15.86:0/2987030449 wait complete. 2024-04-20T21:18:37.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.995+0000 7f6c2ba00640 1 Processor -- start 2024-04-20T21:18:37.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.996+0000 7f6c2ba00640 1 -- start start 2024-04-20T21:18:37.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.997+0000 7f6c2ba00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c240b03e0 0x7f6c240d0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:37.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.997+0000 7f6c2ba00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c240b3ee0 0x7f6c24160180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:37.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.997+0000 7f6c2ba00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c240b4360 0x7f6c241e94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:37.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.997+0000 7f6c2ba00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6c240b4790 con 0x7f6c240b4360 2024-04-20T21:18:37.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.997+0000 7f6c2ba00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6c24070330 con 0x7f6c240b03e0 2024-04-20T21:18:37.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.997+0000 7f6c2ba00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6c2406e310 con 0x7f6c240b3ee0 2024-04-20T21:18:37.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.997+0000 7f6c29a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c240b4360 0x7f6c241e94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:37.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.997+0000 7f6c29a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c240b4360 0x7f6c241e94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35920/0 (socket says 172.21.15.86:35920) 2024-04-20T21:18:37.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.997+0000 7f6c29a00640 1 -- 172.21.15.86:0/1194438577 learned_addr learned my addr 172.21.15.86:0/1194438577 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:37.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.997+0000 7f6c29000640 1 --2- 172.21.15.86:0/1194438577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c240b03e0 0x7f6c240d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:37.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.997+0000 7f6c29a00640 1 -- 172.21.15.86:0/1194438577 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c240b3ee0 msgr2=0x7f6c24160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:37.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.997+0000 7f6c23e00640 1 --2- 172.21.15.86:0/1194438577 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c240b3ee0 0x7f6c24160180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:37.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.997+0000 7f6c29a00640 1 --2- 172.21.15.86:0/1194438577 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c240b3ee0 0x7f6c24160180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:37.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.997+0000 7f6c29a00640 1 -- 172.21.15.86:0/1194438577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c240b03e0 msgr2=0x7f6c240d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:37.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.998+0000 7f6c29a00640 1 --2- 172.21.15.86:0/1194438577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c240b03e0 0x7f6c240d0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:37.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.998+0000 7f6c29a00640 1 -- 172.21.15.86:0/1194438577 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c240cf260 con 0x7f6c240b4360 2024-04-20T21:18:37.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.998+0000 7f6c29000640 1 --2- 172.21.15.86:0/1194438577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c240b03e0 0x7f6c240d0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:37.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.998+0000 7f6c23e00640 1 --2- 172.21.15.86:0/1194438577 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c240b3ee0 0x7f6c24160180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:38.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.998+0000 7f6c29a00640 1 --2- 172.21.15.86:0/1194438577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c240b4360 0x7f6c241e94f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c009870 tx=0x7f6c1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:38.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.998+0000 7f6c20c00640 1 -- 172.21.15.86:0/1194438577 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c040070 con 0x7f6c240b4360 2024-04-20T21:18:38.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.998+0000 7f6c20c00640 1 -- 172.21.15.86:0/1194438577 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c1c039c10 con 0x7f6c240b4360 2024-04-20T21:18:38.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.998+0000 7f6c20c00640 1 -- 172.21.15.86:0/1194438577 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c035040 con 0x7f6c240b4360 2024-04-20T21:18:38.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.998+0000 7f6c2ba00640 1 -- 172.21.15.86:0/1194438577 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c240cf540 con 0x7f6c240b4360 2024-04-20T21:18:38.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.998+0000 7f6c2ba00640 1 -- 172.21.15.86:0/1194438577 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c240cfb00 con 0x7f6c240b4360 2024-04-20T21:18:38.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:37.999+0000 7f6c2ba00640 1 -- 172.21.15.86:0/1194438577 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c240690f0 con 0x7f6c240b4360 2024-04-20T21:18:38.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.000+0000 7f6c20c00640 1 -- 172.21.15.86:0/1194438577 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6c1c04b060 con 0x7f6c240b4360 2024-04-20T21:18:38.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.002+0000 7f6c20c00640 1 --2- 172.21.15.86:0/1194438577 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6c0006c8f0 0x7f6c0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:38.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.002+0000 7f6c20c00640 1 -- 172.21.15.86:0/1194438577 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f6c1c0ad910 con 0x7f6c240b4360 2024-04-20T21:18:38.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.002+0000 7f6c29000640 1 --2- 172.21.15.86:0/1194438577 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6c0006c8f0 0x7f6c0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:38.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.003+0000 7f6c29000640 1 --2- 172.21.15.86:0/1194438577 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6c0006c8f0 0x7f6c0006edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6c241f7800 tx=0x7f6c0c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:38.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.006+0000 7f6c20c00640 1 -- 172.21.15.86:0/1194438577 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6c1c07bcb0 con 0x7f6c240b4360 2024-04-20T21:18:38.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.193+0000 7f6c2ba00640 1 -- 172.21.15.86:0/1194438577 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6c24158af0 con 0x7f6c240b4360 2024-04-20T21:18:38.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.194+0000 7f6c20c00640 1 -- 172.21.15.86:0/1194438577 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f6c1c07b5b0 con 0x7f6c240b4360 2024-04-20T21:18:38.194 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-20T21:18:38.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.196+0000 7f6c2ba00640 1 -- 172.21.15.86:0/1194438577 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6c0006c8f0 msgr2=0x7f6c0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:38.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.196+0000 7f6c2ba00640 1 --2- 172.21.15.86:0/1194438577 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6c0006c8f0 0x7f6c0006edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6c241f7800 tx=0x7f6c0c008040 comp rx=0 tx=0).stop 2024-04-20T21:18:38.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.196+0000 7f6c2ba00640 1 -- 172.21.15.86:0/1194438577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c240b4360 msgr2=0x7f6c241e94f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:38.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.196+0000 7f6c2ba00640 1 --2- 172.21.15.86:0/1194438577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c240b4360 0x7f6c241e94f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c009870 tx=0x7f6c1c0046c0 comp rx=0 tx=0).stop 2024-04-20T21:18:38.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.197+0000 7f6c2ba00640 1 -- 172.21.15.86:0/1194438577 shutdown_connections 2024-04-20T21:18:38.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.197+0000 7f6c2ba00640 1 --2- 172.21.15.86:0/1194438577 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6c0006c8f0 0x7f6c0006edb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:38.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.197+0000 7f6c2ba00640 1 --2- 172.21.15.86:0/1194438577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c240b4360 0x7f6c241e94f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:38.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.197+0000 7f6c2ba00640 1 --2- 172.21.15.86:0/1194438577 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c240b3ee0 0x7f6c24160180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:38.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.197+0000 7f6c2ba00640 1 --2- 172.21.15.86:0/1194438577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c240b03e0 0x7f6c240d0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:38.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.197+0000 7f6c2ba00640 1 -- 172.21.15.86:0/1194438577 >> 172.21.15.86:0/1194438577 conn(0x7f6c240b3380 msgr2=0x7f6c24160de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:38.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.197+0000 7f6c2ba00640 1 -- 172.21.15.86:0/1194438577 shutdown_connections 2024-04-20T21:18:38.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.197+0000 7f6c2ba00640 1 -- 172.21.15.86:0/1194438577 wait complete. 2024-04-20T21:18:38.215 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:18:38.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.418+0000 7f4e88c00640 1 Processor -- start 2024-04-20T21:18:38.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.419+0000 7f4e88c00640 1 -- start start 2024-04-20T21:18:38.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.420+0000 7f4e88c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4e840af1f0 0x7f4e8406ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:38.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.420+0000 7f4e88c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e8406de80 con 0x7f4e8406d9c0 2024-04-20T21:18:38.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.420+0000 7f4e88c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e8406e160 con 0x7f4e840af5d0 2024-04-20T21:18:38.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.420+0000 7f4e88c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f4e8406f460 con 0x7f4e840af1f0 2024-04-20T21:18:38.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.420+0000 7f4e83400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4e840af1f0 0x7f4e8406ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:38.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.421+0000 7f4e83400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4e840af1f0 0x7f4e8406ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35926/0 (socket says 172.21.15.86:35926) 2024-04-20T21:18:38.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.421+0000 7f4e83400640 1 -- 172.21.15.86:0/3593683852 learned_addr learned my addr 172.21.15.86:0/3593683852 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:38.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.421+0000 7f4e83400640 1 -- 172.21.15.86:0/3593683852 >> v1:172.21.15.144:6789/0 conn(0x7f4e840af5d0 legacy=0x7f4e8415de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:38.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.421+0000 7f4e83400640 1 -- 172.21.15.86:0/3593683852 >> v1:172.21.15.86:6789/0 conn(0x7f4e8406d9c0 legacy=0x7f4e8415d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:38.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.421+0000 7f4e83400640 1 -- 172.21.15.86:0/3593683852 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e8406e440 con 0x7f4e840af1f0 2024-04-20T21:18:38.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.422+0000 7f4e83400640 1 --2- 172.21.15.86:0/3593683852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e840af1f0 0x7f4e8406ef20 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4e74009870 tx=0x7f4e7402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=816e23cc462a0d46 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:38.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.422+0000 7f4e82a00640 1 -- 172.21.15.86:0/3593683852 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e74034070 con 0x7f4e840af1f0 2024-04-20T21:18:38.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.422+0000 7f4e82a00640 1 -- 172.21.15.86:0/3593683852 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e74030860 con 0x7f4e840af1f0 2024-04-20T21:18:38.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.422+0000 7f4e82a00640 1 -- 172.21.15.86:0/3593683852 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e74030c30 con 0x7f4e840af1f0 2024-04-20T21:18:38.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.423+0000 7f4e88c00640 1 -- 172.21.15.86:0/3593683852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e840af1f0 msgr2=0x7f4e8406ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:38.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.423+0000 7f4e88c00640 1 --2- 172.21.15.86:0/3593683852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e840af1f0 0x7f4e8406ef20 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4e74009870 tx=0x7f4e7402b9a0 comp rx=0 tx=0).stop 2024-04-20T21:18:38.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.423+0000 7f4e88c00640 1 -- 172.21.15.86:0/3593683852 shutdown_connections 2024-04-20T21:18:38.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.423+0000 7f4e88c00640 1 --2- 172.21.15.86:0/3593683852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e840af1f0 0x7f4e8406ef20 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:38.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.423+0000 7f4e88c00640 1 -- 172.21.15.86:0/3593683852 >> 172.21.15.86:0/3593683852 conn(0x7f4e840b1f00 msgr2=0x7f4e840b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:38.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.423+0000 7f4e88c00640 1 -- 172.21.15.86:0/3593683852 shutdown_connections 2024-04-20T21:18:38.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.423+0000 7f4e88c00640 1 -- 172.21.15.86:0/3593683852 wait complete. 2024-04-20T21:18:38.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.425+0000 7f4e88c00640 1 Processor -- start 2024-04-20T21:18:38.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.425+0000 7f4e88c00640 1 -- start start 2024-04-20T21:18:38.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.426+0000 7f4e88c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e8406d9c0 0x7f4e841f4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:38.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.426+0000 7f4e88c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e840af1f0 0x7f4e84099040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:38.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.426+0000 7f4e88c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e840af5d0 0x7f4e84156e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:38.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.426+0000 7f4e88c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4e840afa40 con 0x7f4e840af5d0 2024-04-20T21:18:38.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.426+0000 7f4e88c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8406ea00 con 0x7f4e840af1f0 2024-04-20T21:18:38.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.426+0000 7f4e88c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f4e8415a3c0 con 0x7f4e8406d9c0 2024-04-20T21:18:38.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.426+0000 7f4e83e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e840af5d0 0x7f4e84156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:38.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.426+0000 7f4e83e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e840af5d0 0x7f4e84156e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35930/0 (socket says 172.21.15.86:35930) 2024-04-20T21:18:38.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.426+0000 7f4e83e00640 1 -- 172.21.15.86:0/1124139559 learned_addr learned my addr 172.21.15.86:0/1124139559 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:38.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.427+0000 7f4e7aa00640 1 --2- 172.21.15.86:0/1124139559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e840af1f0 0x7f4e84099040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:38.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.427+0000 7f4e83e00640 1 -- 172.21.15.86:0/1124139559 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e8406d9c0 msgr2=0x7f4e841f4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:38.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.427+0000 7f4e83e00640 1 --2- 172.21.15.86:0/1124139559 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e8406d9c0 0x7f4e841f4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:38.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.427+0000 7f4e83e00640 1 -- 172.21.15.86:0/1124139559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e840af1f0 msgr2=0x7f4e84099040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:38.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.427+0000 7f4e83e00640 1 --2- 172.21.15.86:0/1124139559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e840af1f0 0x7f4e84099040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:38.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.427+0000 7f4e83e00640 1 -- 172.21.15.86:0/1124139559 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e84071f70 con 0x7f4e840af5d0 2024-04-20T21:18:38.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.427+0000 7f4e7aa00640 1 --2- 172.21.15.86:0/1124139559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e840af1f0 0x7f4e84099040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:38.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.427+0000 7f4e83e00640 1 --2- 172.21.15.86:0/1124139559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e840af5d0 0x7f4e84156e90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c00c780 tx=0x7f4e7c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:38.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.427+0000 7f4e7be00640 1 -- 172.21.15.86:0/1124139559 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c017070 con 0x7f4e840af5d0 2024-04-20T21:18:38.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.427+0000 7f4e7be00640 1 -- 172.21.15.86:0/1124139559 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e7c003da0 con 0x7f4e840af5d0 2024-04-20T21:18:38.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.427+0000 7f4e88c00640 1 -- 172.21.15.86:0/1124139559 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e84072250 con 0x7f4e840af5d0 2024-04-20T21:18:38.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.427+0000 7f4e7be00640 1 -- 172.21.15.86:0/1124139559 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c0052b0 con 0x7f4e840af5d0 2024-04-20T21:18:38.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.427+0000 7f4e88c00640 1 -- 172.21.15.86:0/1124139559 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e84072810 con 0x7f4e840af5d0 2024-04-20T21:18:38.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.429+0000 7f4e88c00640 1 -- 172.21.15.86:0/1124139559 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e8404fd90 con 0x7f4e840af5d0 2024-04-20T21:18:38.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.430+0000 7f4e7be00640 1 -- 172.21.15.86:0/1124139559 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4e7c01a070 con 0x7f4e840af5d0 2024-04-20T21:18:38.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.430+0000 7f4e7be00640 1 --2- 172.21.15.86:0/1124139559 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4e6006c8f0 0x7f4e6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:38.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.431+0000 7f4e7be00640 1 -- 172.21.15.86:0/1124139559 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f4e7c08b2e0 con 0x7f4e840af5d0 2024-04-20T21:18:38.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.431+0000 7f4e83400640 1 --2- 172.21.15.86:0/1124139559 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4e6006c8f0 0x7f4e6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:38.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.431+0000 7f4e83400640 1 --2- 172.21.15.86:0/1124139559 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4e6006c8f0 0x7f4e6006edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4e74036000 tx=0x7f4e7403b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:38.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.435+0000 7f4e7be00640 1 -- 172.21.15.86:0/1124139559 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4e7c0595f0 con 0x7f4e840af5d0 2024-04-20T21:18:38.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.632+0000 7f4e88c00640 1 -- 172.21.15.86:0/1124139559 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4e840b3300 con 0x7f4e840af5d0 2024-04-20T21:18:38.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.633+0000 7f4e7be00640 1 -- 172.21.15.86:0/1124139559 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f4e7c058ef0 con 0x7f4e840af5d0 2024-04-20T21:18:38.633 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-20T21:18:38.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.636+0000 7f4e88c00640 1 -- 172.21.15.86:0/1124139559 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4e6006c8f0 msgr2=0x7f4e6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:38.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.636+0000 7f4e88c00640 1 --2- 172.21.15.86:0/1124139559 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4e6006c8f0 0x7f4e6006edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4e74036000 tx=0x7f4e7403b040 comp rx=0 tx=0).stop 2024-04-20T21:18:38.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.636+0000 7f4e88c00640 1 -- 172.21.15.86:0/1124139559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e840af5d0 msgr2=0x7f4e84156e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:38.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.636+0000 7f4e88c00640 1 --2- 172.21.15.86:0/1124139559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e840af5d0 0x7f4e84156e90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c00c780 tx=0x7f4e7c00cc40 comp rx=0 tx=0).stop 2024-04-20T21:18:38.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.636+0000 7f4e88c00640 1 -- 172.21.15.86:0/1124139559 shutdown_connections 2024-04-20T21:18:38.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.636+0000 7f4e88c00640 1 --2- 172.21.15.86:0/1124139559 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4e6006c8f0 0x7f4e6006edb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:38.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.636+0000 7f4e88c00640 1 --2- 172.21.15.86:0/1124139559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e840af5d0 0x7f4e84156e90 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:38.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.636+0000 7f4e88c00640 1 --2- 172.21.15.86:0/1124139559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e840af1f0 0x7f4e84099040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:38.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.636+0000 7f4e88c00640 1 --2- 172.21.15.86:0/1124139559 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e8406d9c0 0x7f4e841f4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:38.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.636+0000 7f4e88c00640 1 -- 172.21.15.86:0/1124139559 >> 172.21.15.86:0/1124139559 conn(0x7f4e840b1f00 msgr2=0x7f4e8415bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:38.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.636+0000 7f4e88c00640 1 -- 172.21.15.86:0/1124139559 shutdown_connections 2024-04-20T21:18:38.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.636+0000 7f4e88c00640 1 -- 172.21.15.86:0/1124139559 wait complete. 2024-04-20T21:18:38.655 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T21:18:38.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.845+0000 7f52bda00640 1 Processor -- start 2024-04-20T21:18:38.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.845+0000 7f52bda00640 1 -- start start 2024-04-20T21:18:38.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.846+0000 7f52bda00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52b815e850 0x7f52b80cfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:38.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.847+0000 7f52bda00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f52b815ec30 0x7f52b80d0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:38.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.847+0000 7f52bda00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52b80d24d0 con 0x7f52b80b61e0 2024-04-20T21:18:38.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.847+0000 7f52bda00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f52b80d22a0 con 0x7f52b815ec30 2024-04-20T21:18:38.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.847+0000 7f52bda00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f52b80d27b0 con 0x7f52b815e850 2024-04-20T21:18:38.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.847+0000 7f52b7400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f52b80b61e0 0x7f52b80b65f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41758/0 (socket says 172.21.15.86:41758) 2024-04-20T21:18:38.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.847+0000 7f52b7400640 1 -- 172.21.15.86:0/2629431623 learned_addr learned my addr 172.21.15.86:0/2629431623 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:18:38.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.847+0000 7f52b6000640 1 --2- 172.21.15.86:0/2629431623 >> v2:172.21.15.144:3300/0 conn(0x7f52b815ec30 0x7f52b80d0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:38.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.847+0000 7f52b6a00640 1 --2- 172.21.15.86:0/2629431623 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52b815e850 0x7f52b80cfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:38.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.848+0000 7f52b5600640 1 -- 172.21.15.86:0/2629431623 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1193489514 0 0) 0x7f52b80d24d0 con 0x7f52b80b61e0 2024-04-20T21:18:38.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.848+0000 7f52b5600640 1 -- 172.21.15.86:0/2629431623 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f529c003520 con 0x7f52b80b61e0 2024-04-20T21:18:38.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.848+0000 7f52b5600640 1 -- 172.21.15.86:0/2629431623 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1658553144 0 0) 0x7f529c003520 con 0x7f52b80b61e0 2024-04-20T21:18:38.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.848+0000 7f52b5600640 1 -- 172.21.15.86:0/2629431623 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52b815e850 msgr2=0x7f52b80cfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:38.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.848+0000 7f52b5600640 1 --2- 172.21.15.86:0/2629431623 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52b815e850 0x7f52b80cfcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:38.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.848+0000 7f52b5600640 1 -- 172.21.15.86:0/2629431623 >> v2:172.21.15.144:3300/0 conn(0x7f52b815ec30 msgr2=0x7f52b80d0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:38.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.848+0000 7f52b5600640 1 --2- 172.21.15.86:0/2629431623 >> v2:172.21.15.144:3300/0 conn(0x7f52b815ec30 0x7f52b80d0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:38.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.848+0000 7f52b5600640 1 -- 172.21.15.86:0/2629431623 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b80d2a90 con 0x7f52b80b61e0 2024-04-20T21:18:38.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.848+0000 7f52b5600640 1 -- 172.21.15.86:0/2629431623 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f52b00019f0 con 0x7f52b80b61e0 2024-04-20T21:18:38.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.848+0000 7f52b5600640 1 -- 172.21.15.86:0/2629431623 >> v1:172.21.15.86:6789/0 conn(0x7f52b80b61e0 legacy=0x7f52b80b65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:38.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.848+0000 7f52b5600640 1 --2- 172.21.15.86:0/2629431623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f529c0039b0 0x7f529c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:38.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.848+0000 7f52b5600640 1 --2- 172.21.15.86:0/2629431623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f529c004800 0x7f529c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:38.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.848+0000 7f52b5600640 1 --2- 172.21.15.86:0/2629431623 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52b815e850 0x7f529c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:38.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.848+0000 7f52b5600640 1 -- 172.21.15.86:0/2629431623 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f52b80d27b0 con 0x7f529c004800 2024-04-20T21:18:38.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.848+0000 7f52b5600640 1 -- 172.21.15.86:0/2629431623 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f529c00b9c0 con 0x7f529c0039b0 2024-04-20T21:18:38.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.848+0000 7f52b5600640 1 -- 172.21.15.86:0/2629431623 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f529c00bb40 con 0x7f52b815e850 2024-04-20T21:18:38.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.848+0000 7f52b6000640 1 --2- 172.21.15.86:0/2629431623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f529c004800 0x7f529c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:38.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.849+0000 7f52b6a00640 1 --2- 172.21.15.86:0/2629431623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f529c0039b0 0x7f529c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:38.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.849+0000 7f52b7400640 1 --2- 172.21.15.86:0/2629431623 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52b815e850 0x7f529c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:38.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.849+0000 7f52b6000640 1 -- 172.21.15.86:0/2629431623 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52b815e850 msgr2=0x7f529c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:38.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.849+0000 7f52b6000640 1 --2- 172.21.15.86:0/2629431623 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52b815e850 0x7f529c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:38.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.849+0000 7f52b6000640 1 -- 172.21.15.86:0/2629431623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f529c0039b0 msgr2=0x7f529c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:38.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.849+0000 7f52b6000640 1 --2- 172.21.15.86:0/2629431623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f529c0039b0 0x7f529c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:38.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.849+0000 7f52b6000640 1 -- 172.21.15.86:0/2629431623 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f529c003520 con 0x7f529c004800 2024-04-20T21:18:38.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.849+0000 7f52b7400640 1 --2- 172.21.15.86:0/2629431623 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52b815e850 0x7f529c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:38.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.850+0000 7f52b6000640 1 --2- 172.21.15.86:0/2629431623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f529c004800 0x7f529c004c80 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f52a4009ac0 tx=0x7f529c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d155eadb7a282e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:38.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.850+0000 7f52b5600640 1 -- 172.21.15.86:0/2629431623 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a4039070 con 0x7f529c004800 2024-04-20T21:18:38.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.850+0000 7f52b5600640 1 -- 172.21.15.86:0/2629431623 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52a402c710 con 0x7f529c004800 2024-04-20T21:18:38.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.850+0000 7f52b5600640 1 -- 172.21.15.86:0/2629431623 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a4009690 con 0x7f529c004800 2024-04-20T21:18:38.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.851+0000 7f52bda00640 1 -- 172.21.15.86:0/2629431623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f529c004800 msgr2=0x7f529c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:38.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.851+0000 7f52bda00640 1 --2- 172.21.15.86:0/2629431623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f529c004800 0x7f529c004c80 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f52a4009ac0 tx=0x7f529c005830 comp rx=0 tx=0).stop 2024-04-20T21:18:38.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.851+0000 7f52b7400640 1 -- 172.21.15.86:0/2629431623 reap_dead start 2024-04-20T21:18:38.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.851+0000 7f52bda00640 1 -- 172.21.15.86:0/2629431623 shutdown_connections 2024-04-20T21:18:38.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.851+0000 7f52bda00640 1 -- 172.21.15.86:0/2629431623 >> 172.21.15.86:0/2629431623 conn(0x7f52b808bf70 msgr2=0x7f52b80d4760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:38.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.851+0000 7f52bda00640 1 -- 172.21.15.86:0/2629431623 shutdown_connections 2024-04-20T21:18:38.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.851+0000 7f52bda00640 1 -- 172.21.15.86:0/2629431623 wait complete. 2024-04-20T21:18:38.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.853+0000 7f52bda00640 1 Processor -- start 2024-04-20T21:18:38.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.853+0000 7f52bda00640 1 -- start start 2024-04-20T21:18:38.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.854+0000 7f52bda00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f529c004800 0x7f52b80c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:38.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.854+0000 7f52bda00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52b815d4f0 0x7f52b80ca230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:38.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.854+0000 7f52bda00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52b80ca770 0x7f52b815f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:38.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.854+0000 7f52bda00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f52b80d3570 con 0x7f529c004800 2024-04-20T21:18:38.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.854+0000 7f52bda00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f52b80d3050 con 0x7f52b80ca770 2024-04-20T21:18:38.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.854+0000 7f52bda00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f52b80d36f0 con 0x7f52b815d4f0 2024-04-20T21:18:38.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.854+0000 7f52b6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f529c004800 0x7f52b80c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:38.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.854+0000 7f52b6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f529c004800 0x7f52b80c7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35944/0 (socket says 172.21.15.86:35944) 2024-04-20T21:18:38.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.854+0000 7f52b6a00640 1 -- 172.21.15.86:0/757297888 learned_addr learned my addr 172.21.15.86:0/757297888 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:38.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.855+0000 7f52b7400640 1 --2- 172.21.15.86:0/757297888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52b80ca770 0x7f52b815f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:38.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.855+0000 7f52b6000640 1 --2- 172.21.15.86:0/757297888 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52b815d4f0 0x7f52b80ca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:38.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.855+0000 7f52b6a00640 1 -- 172.21.15.86:0/757297888 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52b815d4f0 msgr2=0x7f52b80ca230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:38.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.855+0000 7f52b6a00640 1 --2- 172.21.15.86:0/757297888 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52b815d4f0 0x7f52b80ca230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:38.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.855+0000 7f52b6a00640 1 -- 172.21.15.86:0/757297888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52b80ca770 msgr2=0x7f52b815f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:38.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.855+0000 7f52b6a00640 1 --2- 172.21.15.86:0/757297888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52b80ca770 0x7f52b815f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:38.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.855+0000 7f52b6a00640 1 -- 172.21.15.86:0/757297888 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b80b9300 con 0x7f529c004800 2024-04-20T21:18:38.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.855+0000 7f52b6a00640 1 --2- 172.21.15.86:0/757297888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f529c004800 0x7f52b80c7b80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f52a80115e0 tx=0x7f52a8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:38.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.856+0000 7f52aea00640 1 -- 172.21.15.86:0/757297888 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a8019070 con 0x7f529c004800 2024-04-20T21:18:38.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.856+0000 7f52aea00640 1 -- 172.21.15.86:0/757297888 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52a8005da0 con 0x7f529c004800 2024-04-20T21:18:38.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.856+0000 7f52aea00640 1 -- 172.21.15.86:0/757297888 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a8006250 con 0x7f529c004800 2024-04-20T21:18:38.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.856+0000 7f52bda00640 1 -- 172.21.15.86:0/757297888 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52b80b95e0 con 0x7f529c004800 2024-04-20T21:18:38.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.856+0000 7f52bda00640 1 -- 172.21.15.86:0/757297888 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52b80b9ba0 con 0x7f529c004800 2024-04-20T21:18:38.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.857+0000 7f52bda00640 1 -- 172.21.15.86:0/757297888 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52b80689e0 con 0x7f529c004800 2024-04-20T21:18:38.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.858+0000 7f52aea00640 1 -- 172.21.15.86:0/757297888 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f52a801c070 con 0x7f529c004800 2024-04-20T21:18:38.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.859+0000 7f52aea00640 1 --2- 172.21.15.86:0/757297888 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f529806c920 0x7f529806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:38.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.859+0000 7f52aea00640 1 -- 172.21.15.86:0/757297888 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f52a808bfe0 con 0x7f529c004800 2024-04-20T21:18:38.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.859+0000 7f52b6000640 1 --2- 172.21.15.86:0/757297888 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f529806c920 0x7f529806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:38.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.860+0000 7f52b6000640 1 --2- 172.21.15.86:0/757297888 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f529806c920 0x7f529806ede0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f52a4036000 tx=0x7f52a4030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:38.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:38.863+0000 7f52aea00640 1 -- 172.21.15.86:0/757297888 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f52a805a380 con 0x7f529c004800 2024-04-20T21:18:39.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.076+0000 7f52bda00640 1 -- 172.21.15.86:0/757297888 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f52b80cab50 con 0x7f529c004800 2024-04-20T21:18:39.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.077+0000 7f52aea00640 1 -- 172.21.15.86:0/757297888 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f52a8059c80 con 0x7f529c004800 2024-04-20T21:18:39.078 INFO:teuthology.orchestra.run.smithi086.stderr:listed 0 entries 2024-04-20T21:18:39.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.080+0000 7f52bda00640 1 -- 172.21.15.86:0/757297888 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f529806c920 msgr2=0x7f529806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:39.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.080+0000 7f52bda00640 1 --2- 172.21.15.86:0/757297888 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f529806c920 0x7f529806ede0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f52a4036000 tx=0x7f52a4030000 comp rx=0 tx=0).stop 2024-04-20T21:18:39.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.080+0000 7f52bda00640 1 -- 172.21.15.86:0/757297888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f529c004800 msgr2=0x7f52b80c7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:39.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.080+0000 7f52bda00640 1 --2- 172.21.15.86:0/757297888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f529c004800 0x7f52b80c7b80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f52a80115e0 tx=0x7f52a8011aa0 comp rx=0 tx=0).stop 2024-04-20T21:18:39.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.080+0000 7f52bda00640 1 -- 172.21.15.86:0/757297888 shutdown_connections 2024-04-20T21:18:39.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.080+0000 7f52bda00640 1 --2- 172.21.15.86:0/757297888 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f529806c920 0x7f529806ede0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:39.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.080+0000 7f52bda00640 1 --2- 172.21.15.86:0/757297888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52b80ca770 0x7f52b815f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:39.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.080+0000 7f52bda00640 1 --2- 172.21.15.86:0/757297888 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f52b815d4f0 0x7f52b80ca230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:39.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.080+0000 7f52bda00640 1 --2- 172.21.15.86:0/757297888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f529c004800 0x7f52b80c7b80 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:39.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.080+0000 7f52bda00640 1 -- 172.21.15.86:0/757297888 >> 172.21.15.86:0/757297888 conn(0x7f52b808bf70 msgr2=0x7f52b806cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:39.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.080+0000 7f52bda00640 1 -- 172.21.15.86:0/757297888 shutdown_connections 2024-04-20T21:18:39.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.080+0000 7f52bda00640 1 -- 172.21.15.86:0/757297888 wait complete. 2024-04-20T21:18:39.100 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi086.front.sepia.ceph.com... 2024-04-20T21:18:39.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:39.100 DEBUG:teuthology.orchestra.run.smithi086:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-20T21:18:39.183 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:18:39.183 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-20T21:18:39.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:39.184 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-20T21:18:39.244 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:18:39.244 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:18:39.301 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:18:39.302 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:18:39.353 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:18:39.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.561+0000 7f49d2c00640 1 Processor -- start 2024-04-20T21:18:39.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.561+0000 7f49d2c00640 1 -- start start 2024-04-20T21:18:39.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.562+0000 7f49d2c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f49cc06fbd0 0x7f49cc15c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:39.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.562+0000 7f49d2c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49cc0ae710 con 0x7f49cc15c010 2024-04-20T21:18:39.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.562+0000 7f49d2c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49cc0ae9f0 con 0x7f49cc0af7b0 2024-04-20T21:18:39.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.562+0000 7f49d2c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f49cc0af320 con 0x7f49cc06fbd0 2024-04-20T21:18:39.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.563+0000 7f49d2200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f49cc06fbd0 0x7f49cc15c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:39.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.563+0000 7f49d1800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f49cc15c010 0x7f49cc0adda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41760/0 (socket says 172.21.15.86:41760) 2024-04-20T21:18:39.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.563+0000 7f49d2200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f49cc06fbd0 0x7f49cc15c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35948/0 (socket says 172.21.15.86:35948) 2024-04-20T21:18:39.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.563+0000 7f49d1800640 1 -- 172.21.15.86:0/441387805 learned_addr learned my addr 172.21.15.86:0/441387805 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:18:39.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.563+0000 7f49d2200640 1 -- 172.21.15.86:0/441387805 >> v1:172.21.15.144:6789/0 conn(0x7f49cc0af7b0 legacy=0x7f49cc06f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:39.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.563+0000 7f49d2200640 1 -- 172.21.15.86:0/441387805 >> v1:172.21.15.86:6789/0 conn(0x7f49cc15c010 legacy=0x7f49cc0adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:39.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.563+0000 7f49d2200640 1 -- 172.21.15.86:0/441387805 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49cc0aecd0 con 0x7f49cc06fbd0 2024-04-20T21:18:39.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.564+0000 7f49d2200640 1 --2- 172.21.15.86:0/441387805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49cc06fbd0 0x7f49cc15c3f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f49c4009d90 tx=0x7f49c402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99726652e57075ef server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:39.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.565+0000 7f49cbe00640 1 -- 172.21.15.86:0/441387805 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c4039070 con 0x7f49cc06fbd0 2024-04-20T21:18:39.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.565+0000 7f49cbe00640 1 -- 172.21.15.86:0/441387805 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49c402cc60 con 0x7f49cc06fbd0 2024-04-20T21:18:39.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.565+0000 7f49cbe00640 1 -- 172.21.15.86:0/441387805 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c4034660 con 0x7f49cc06fbd0 2024-04-20T21:18:39.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.565+0000 7f49d2c00640 1 -- 172.21.15.86:0/441387805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49cc06fbd0 msgr2=0x7f49cc15c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:39.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.565+0000 7f49d2c00640 1 --2- 172.21.15.86:0/441387805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49cc06fbd0 0x7f49cc15c3f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f49c4009d90 tx=0x7f49c402beb0 comp rx=0 tx=0).stop 2024-04-20T21:18:39.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.565+0000 7f49d2c00640 1 -- 172.21.15.86:0/441387805 shutdown_connections 2024-04-20T21:18:39.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.565+0000 7f49d2c00640 1 --2- 172.21.15.86:0/441387805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49cc06fbd0 0x7f49cc15c3f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:39.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.565+0000 7f49d2c00640 1 -- 172.21.15.86:0/441387805 >> 172.21.15.86:0/441387805 conn(0x7f49cc0d0170 msgr2=0x7f49cc0d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:39.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.565+0000 7f49d2c00640 1 -- 172.21.15.86:0/441387805 shutdown_connections 2024-04-20T21:18:39.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.565+0000 7f49d2c00640 1 -- 172.21.15.86:0/441387805 wait complete. 2024-04-20T21:18:39.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.567+0000 7f49d2c00640 1 Processor -- start 2024-04-20T21:18:39.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.567+0000 7f49d2c00640 1 -- start start 2024-04-20T21:18:39.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.568+0000 7f49d2c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49cc06fbd0 0x7f49cc09cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:39.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.568+0000 7f49d2c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49cc0af7b0 0x7f49cc15aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:39.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.568+0000 7f49d2c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f49cc15c010 0x7f49cc1589c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:39.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.568+0000 7f49d2c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f49cc15c930 con 0x7f49cc06fbd0 2024-04-20T21:18:39.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.568+0000 7f49d2c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f49cc0ae4b0 con 0x7f49cc0af7b0 2024-04-20T21:18:39.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.568+0000 7f49d2c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f49cc06e3b0 con 0x7f49cc15c010 2024-04-20T21:18:39.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.568+0000 7f49d1800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49cc06fbd0 0x7f49cc09cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:39.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.568+0000 7f49d1800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49cc06fbd0 0x7f49cc09cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35952/0 (socket says 172.21.15.86:35952) 2024-04-20T21:18:39.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.568+0000 7f49d1800640 1 -- 172.21.15.86:0/2788131622 learned_addr learned my addr 172.21.15.86:0/2788131622 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:39.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.569+0000 7f49d0e00640 1 --2- 172.21.15.86:0/2788131622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49cc0af7b0 0x7f49cc15aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:39.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.569+0000 7f49d1800640 1 -- 172.21.15.86:0/2788131622 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f49cc15c010 msgr2=0x7f49cc1589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:39.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.569+0000 7f49d1800640 1 --2- 172.21.15.86:0/2788131622 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f49cc15c010 0x7f49cc1589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:39.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.569+0000 7f49d1800640 1 -- 172.21.15.86:0/2788131622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49cc0af7b0 msgr2=0x7f49cc15aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:39.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.569+0000 7f49d1800640 1 --2- 172.21.15.86:0/2788131622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49cc0af7b0 0x7f49cc15aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:39.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.569+0000 7f49d1800640 1 -- 172.21.15.86:0/2788131622 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49cc155430 con 0x7f49cc06fbd0 2024-04-20T21:18:39.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.569+0000 7f49d1800640 1 --2- 172.21.15.86:0/2788131622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49cc06fbd0 0x7f49cc09cc80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f49c000c3c0 tx=0x7f49c000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:39.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.569+0000 7f49c9600640 1 -- 172.21.15.86:0/2788131622 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c0017070 con 0x7f49cc06fbd0 2024-04-20T21:18:39.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.569+0000 7f49c9600640 1 -- 172.21.15.86:0/2788131622 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49c0005ce0 con 0x7f49cc06fbd0 2024-04-20T21:18:39.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.569+0000 7f49c9600640 1 -- 172.21.15.86:0/2788131622 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c00072f0 con 0x7f49cc06fbd0 2024-04-20T21:18:39.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.569+0000 7f49d2c00640 1 -- 172.21.15.86:0/2788131622 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49cc09d5f0 con 0x7f49cc06fbd0 2024-04-20T21:18:39.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.570+0000 7f49d2c00640 1 -- 172.21.15.86:0/2788131622 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49cc09dbb0 con 0x7f49cc06fbd0 2024-04-20T21:18:39.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.571+0000 7f49d2c00640 1 -- 172.21.15.86:0/2788131622 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49cc04fcf0 con 0x7f49cc06fbd0 2024-04-20T21:18:39.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.572+0000 7f49c9600640 1 -- 172.21.15.86:0/2788131622 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f49c001a070 con 0x7f49cc06fbd0 2024-04-20T21:18:39.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.573+0000 7f49c9600640 1 --2- 172.21.15.86:0/2788131622 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f49ac06c920 0x7f49ac06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:39.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.573+0000 7f49c9600640 1 -- 172.21.15.86:0/2788131622 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f49c008af20 con 0x7f49cc06fbd0 2024-04-20T21:18:39.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.573+0000 7f49d0e00640 1 --2- 172.21.15.86:0/2788131622 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f49ac06c920 0x7f49ac06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:39.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.573+0000 7f49d0e00640 1 --2- 172.21.15.86:0/2788131622 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f49ac06c920 0x7f49ac06ede0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f49cc0cfbf0 tx=0x7f49b8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:39.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.577+0000 7f49c9600640 1 -- 172.21.15.86:0/2788131622 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f49c0059360 con 0x7f49cc06fbd0 2024-04-20T21:18:39.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.779+0000 7f49d2c00640 1 -- 172.21.15.86:0/2788131622 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f49cc072d20 con 0x7f49cc06fbd0 2024-04-20T21:18:39.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.783+0000 7f49c9600640 1 -- 172.21.15.86:0/2788131622 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16854 (secure 0 0 0) 0x7f49c0058c60 con 0x7f49cc06fbd0 2024-04-20T21:18:39.784 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:18:39.784 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":22,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:18:33.788155+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:17:49.718799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:17:50.683477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:18:39.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.785+0000 7f49d2c00640 1 -- 172.21.15.86:0/2788131622 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f49ac06c920 msgr2=0x7f49ac06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:39.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.785+0000 7f49d2c00640 1 --2- 172.21.15.86:0/2788131622 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f49ac06c920 0x7f49ac06ede0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f49cc0cfbf0 tx=0x7f49b8009040 comp rx=0 tx=0).stop 2024-04-20T21:18:39.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.785+0000 7f49d2c00640 1 -- 172.21.15.86:0/2788131622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49cc06fbd0 msgr2=0x7f49cc09cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:39.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.785+0000 7f49d2c00640 1 --2- 172.21.15.86:0/2788131622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49cc06fbd0 0x7f49cc09cc80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f49c000c3c0 tx=0x7f49c000c880 comp rx=0 tx=0).stop 2024-04-20T21:18:39.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.785+0000 7f49d2c00640 1 -- 172.21.15.86:0/2788131622 shutdown_connections 2024-04-20T21:18:39.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.785+0000 7f49d2c00640 1 --2- 172.21.15.86:0/2788131622 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f49ac06c920 0x7f49ac06ede0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:39.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.785+0000 7f49d2c00640 1 --2- 172.21.15.86:0/2788131622 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f49cc15c010 0x7f49cc1589c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:39.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.785+0000 7f49d2c00640 1 --2- 172.21.15.86:0/2788131622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49cc0af7b0 0x7f49cc15aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:39.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.785+0000 7f49d2c00640 1 --2- 172.21.15.86:0/2788131622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49cc06fbd0 0x7f49cc09cc80 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:39.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.785+0000 7f49d2c00640 1 -- 172.21.15.86:0/2788131622 >> 172.21.15.86:0/2788131622 conn(0x7f49cc0d0170 msgr2=0x7f49cc0b07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:39.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.786+0000 7f49d2c00640 1 -- 172.21.15.86:0/2788131622 shutdown_connections 2024-04-20T21:18:39.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:39.786+0000 7f49d2c00640 1 -- 172.21.15.86:0/2788131622 wait complete. 2024-04-20T21:18:39.805 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:17:49.718799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:17:50.683477+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:18:39.805 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:18:40.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.006+0000 7f2fc8a00640 1 Processor -- start 2024-04-20T21:18:40.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.006+0000 7f2fc8a00640 1 -- start start 2024-04-20T21:18:40.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.008+0000 7f2fc8a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fc415eb30 0x7f2fc40b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:40.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.008+0000 7f2fc8a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2fc40aee50 0x7f2fc40b4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:40.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.008+0000 7f2fc8a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fc40adb10 con 0x7f2fc406b980 2024-04-20T21:18:40.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.008+0000 7f2fc8a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f2fc40af7d0 con 0x7f2fc40aee50 2024-04-20T21:18:40.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.008+0000 7f2fc8a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2fc40addb0 con 0x7f2fc415eb30 2024-04-20T21:18:40.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.008+0000 7f2fb9400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2fc40aee50 0x7f2fc40b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:40.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.008+0000 7f2fb9400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2fc40aee50 0x7f2fc40b4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35964/0 (socket says 172.21.15.86:35964) 2024-04-20T21:18:40.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.008+0000 7f2fb9400640 1 -- 172.21.15.86:0/3473920885 learned_addr learned my addr 172.21.15.86:0/3473920885 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:40.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.009+0000 7f2fb9400640 1 -- 172.21.15.86:0/3473920885 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fc415eb30 msgr2=0x7f2fc40b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:40.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.009+0000 7f2fb9400640 1 --2- 172.21.15.86:0/3473920885 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fc415eb30 0x7f2fc40b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:40.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.009+0000 7f2fb9400640 1 -- 172.21.15.86:0/3473920885 >> v1:172.21.15.144:6789/0 conn(0x7f2fc406b980 legacy=0x7f2fc4071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:40.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.009+0000 7f2fb9400640 1 -- 172.21.15.86:0/3473920885 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fc40ae090 con 0x7f2fc40aee50 2024-04-20T21:18:40.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.010+0000 7f2fb9400640 1 --2- 172.21.15.86:0/3473920885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fc40aee50 0x7f2fc40b4360 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2fac009870 tx=0x7f2fac02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1d0b1ba31485159 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:40.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.010+0000 7f2fc1400640 1 -- 172.21.15.86:0/3473920885 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fac035020 con 0x7f2fc40aee50 2024-04-20T21:18:40.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.010+0000 7f2fc1400640 1 -- 172.21.15.86:0/3473920885 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fac031640 con 0x7f2fc40aee50 2024-04-20T21:18:40.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.010+0000 7f2fc1400640 1 -- 172.21.15.86:0/3473920885 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fac02d040 con 0x7f2fc40aee50 2024-04-20T21:18:40.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.011+0000 7f2fc8a00640 1 -- 172.21.15.86:0/3473920885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fc40aee50 msgr2=0x7f2fc40b4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:40.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.011+0000 7f2fc8a00640 1 --2- 172.21.15.86:0/3473920885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fc40aee50 0x7f2fc40b4360 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2fac009870 tx=0x7f2fac02b8f0 comp rx=0 tx=0).stop 2024-04-20T21:18:40.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.011+0000 7f2fc8a00640 1 -- 172.21.15.86:0/3473920885 shutdown_connections 2024-04-20T21:18:40.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.011+0000 7f2fc8a00640 1 --2- 172.21.15.86:0/3473920885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fc40aee50 0x7f2fc40b4360 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:40.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.011+0000 7f2fc8a00640 1 --2- 172.21.15.86:0/3473920885 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fc415eb30 0x7f2fc40b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:40.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.011+0000 7f2fc8a00640 1 -- 172.21.15.86:0/3473920885 >> 172.21.15.86:0/3473920885 conn(0x7f2fc40b2110 msgr2=0x7f2fc40b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:40.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.011+0000 7f2fc8a00640 1 -- 172.21.15.86:0/3473920885 shutdown_connections 2024-04-20T21:18:40.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.011+0000 7f2fc8a00640 1 -- 172.21.15.86:0/3473920885 wait complete. 2024-04-20T21:18:40.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.013+0000 7f2fc8a00640 1 Processor -- start 2024-04-20T21:18:40.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.013+0000 7f2fc8a00640 1 -- start start 2024-04-20T21:18:40.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.014+0000 7f2fc8a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fc406b980 0x7f2fc4159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:40.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.014+0000 7f2fc8a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fc40aee50 0x7f2fc415a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:40.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.014+0000 7f2fc8a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fc415eb30 0x7f2fc41f48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:40.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.014+0000 7f2fc8a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc40ae930 con 0x7f2fc406b980 2024-04-20T21:18:40.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.014+0000 7f2fc8a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc40ae650 con 0x7f2fc40aee50 2024-04-20T21:18:40.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.014+0000 7f2fc8a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2fc406be30 con 0x7f2fc415eb30 2024-04-20T21:18:40.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.014+0000 7f2fc1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fc406b980 0x7f2fc4159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:40.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.014+0000 7f2fc1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fc406b980 0x7f2fc4159bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35966/0 (socket says 172.21.15.86:35966) 2024-04-20T21:18:40.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.014+0000 7f2fc1e00640 1 -- 172.21.15.86:0/2317713245 learned_addr learned my addr 172.21.15.86:0/2317713245 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:40.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.015+0000 7f2fc1e00640 1 -- 172.21.15.86:0/2317713245 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fc415eb30 msgr2=0x7f2fc41f48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:40.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.015+0000 7f2fc1e00640 1 --2- 172.21.15.86:0/2317713245 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fc415eb30 0x7f2fc41f48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:40.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.015+0000 7f2fc1e00640 1 -- 172.21.15.86:0/2317713245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fc40aee50 msgr2=0x7f2fc415a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:40.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.015+0000 7f2fc1e00640 1 --2- 172.21.15.86:0/2317713245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fc40aee50 0x7f2fc415a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:40.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.015+0000 7f2fc1e00640 1 -- 172.21.15.86:0/2317713245 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fc415bda0 con 0x7f2fc406b980 2024-04-20T21:18:40.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.015+0000 7f2fc1e00640 1 --2- 172.21.15.86:0/2317713245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fc406b980 0x7f2fc4159bb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2fb400ba50 tx=0x7f2fb400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:40.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.015+0000 7f2fbaa00640 1 -- 172.21.15.86:0/2317713245 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb4019070 con 0x7f2fc406b980 2024-04-20T21:18:40.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.015+0000 7f2fbaa00640 1 -- 172.21.15.86:0/2317713245 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fb400cc00 con 0x7f2fc406b980 2024-04-20T21:18:40.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.015+0000 7f2fc8a00640 1 -- 172.21.15.86:0/2317713245 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fc415a720 con 0x7f2fc406b980 2024-04-20T21:18:40.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.015+0000 7f2fbaa00640 1 -- 172.21.15.86:0/2317713245 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb4014440 con 0x7f2fc406b980 2024-04-20T21:18:40.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.015+0000 7f2fc8a00640 1 -- 172.21.15.86:0/2317713245 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fc415ace0 con 0x7f2fc406b980 2024-04-20T21:18:40.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.016+0000 7f2fc8a00640 1 -- 172.21.15.86:0/2317713245 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fc40689e0 con 0x7f2fc406b980 2024-04-20T21:18:40.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.017+0000 7f2fbaa00640 1 -- 172.21.15.86:0/2317713245 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2fb401c070 con 0x7f2fc406b980 2024-04-20T21:18:40.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.018+0000 7f2fbaa00640 1 --2- 172.21.15.86:0/2317713245 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2fa006c8f0 0x7f2fa006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:40.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.018+0000 7f2fbaa00640 1 -- 172.21.15.86:0/2317713245 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f2fb408b970 con 0x7f2fc406b980 2024-04-20T21:18:40.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.019+0000 7f2fb9400640 1 --2- 172.21.15.86:0/2317713245 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2fa006c8f0 0x7f2fa006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:40.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.019+0000 7f2fb9400640 1 --2- 172.21.15.86:0/2317713245 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2fa006c8f0 0x7f2fa006edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2fac036000 tx=0x7f2fac030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:40.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.023+0000 7f2fbaa00640 1 -- 172.21.15.86:0/2317713245 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2fb4059d10 con 0x7f2fc406b980 2024-04-20T21:18:40.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.222+0000 7f2fc8a00640 1 -- 172.21.15.86:0/2317713245 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2fc40b0aa0 con 0x7f2fc406b980 2024-04-20T21:18:40.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.223+0000 7f2fbaa00640 1 -- 172.21.15.86:0/2317713245 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f2fb4059610 con 0x7f2fc406b980 2024-04-20T21:18:40.224 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:18:40.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.226+0000 7f2fc8a00640 1 -- 172.21.15.86:0/2317713245 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2fa006c8f0 msgr2=0x7f2fa006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:40.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.226+0000 7f2fc8a00640 1 --2- 172.21.15.86:0/2317713245 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2fa006c8f0 0x7f2fa006edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2fac036000 tx=0x7f2fac030000 comp rx=0 tx=0).stop 2024-04-20T21:18:40.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.226+0000 7f2fc8a00640 1 -- 172.21.15.86:0/2317713245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fc406b980 msgr2=0x7f2fc4159bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:40.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.226+0000 7f2fc8a00640 1 --2- 172.21.15.86:0/2317713245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fc406b980 0x7f2fc4159bb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2fb400ba50 tx=0x7f2fb400bf10 comp rx=0 tx=0).stop 2024-04-20T21:18:40.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.226+0000 7f2fc8a00640 1 -- 172.21.15.86:0/2317713245 shutdown_connections 2024-04-20T21:18:40.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.226+0000 7f2fc8a00640 1 --2- 172.21.15.86:0/2317713245 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2fa006c8f0 0x7f2fa006edb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:40.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.226+0000 7f2fc8a00640 1 --2- 172.21.15.86:0/2317713245 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fc415eb30 0x7f2fc41f48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:40.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.226+0000 7f2fc8a00640 1 --2- 172.21.15.86:0/2317713245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fc40aee50 0x7f2fc415a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:40.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.226+0000 7f2fc8a00640 1 --2- 172.21.15.86:0/2317713245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fc406b980 0x7f2fc4159bb0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:40.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.226+0000 7f2fc8a00640 1 -- 172.21.15.86:0/2317713245 >> 172.21.15.86:0/2317713245 conn(0x7f2fc40b2110 msgr2=0x7f2fc40d0420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:40.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.226+0000 7f2fc8a00640 1 -- 172.21.15.86:0/2317713245 shutdown_connections 2024-04-20T21:18:40.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.227+0000 7f2fc8a00640 1 -- 172.21.15.86:0/2317713245 wait complete. 2024-04-20T21:18:40.246 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:18:40.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.453+0000 7f788d000640 1 Processor -- start 2024-04-20T21:18:40.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.453+0000 7f788d000640 1 -- start start 2024-04-20T21:18:40.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.455+0000 7f788d000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7888145250 0x7f788806efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:40.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.455+0000 7f788d000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f788806f500 con 0x7f7888145f00 2024-04-20T21:18:40.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.455+0000 7f788d000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7888160410 con 0x7f7888072450 2024-04-20T21:18:40.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.455+0000 7f788d000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f78881606f0 con 0x7f7888145250 2024-04-20T21:18:40.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.455+0000 7f7886a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f7888145f00 0x7f7888071d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41764/0 (socket says 172.21.15.86:41764) 2024-04-20T21:18:40.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.455+0000 7f7886a00640 1 -- 172.21.15.86:0/678304021 learned_addr learned my addr 172.21.15.86:0/678304021 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:18:40.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.455+0000 7f7887400640 1 --2- 172.21.15.86:0/678304021 >> v2:172.21.15.86:3300/0 conn(0x7f7888145250 0x7f788806efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:40.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.456+0000 7f7887400640 1 -- 172.21.15.86:0/678304021 >> v1:172.21.15.144:6789/0 conn(0x7f7888072450 legacy=0x7f788815cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:40.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.456+0000 7f7887400640 1 -- 172.21.15.86:0/678304021 >> v1:172.21.15.86:6789/0 conn(0x7f7888145f00 legacy=0x7f7888071d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:40.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.456+0000 7f7887400640 1 -- 172.21.15.86:0/678304021 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78881609d0 con 0x7f7888145250 2024-04-20T21:18:40.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.457+0000 7f7887400640 1 --2- 172.21.15.86:0/678304021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7888145250 0x7f788806efc0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f78780097a0 tx=0x7f787802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd22f8bcd16810f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:40.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.457+0000 7f7886000640 1 -- 172.21.15.86:0/678304021 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78780315a0 con 0x7f7888145250 2024-04-20T21:18:40.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.457+0000 7f7886000640 1 -- 172.21.15.86:0/678304021 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7878037070 con 0x7f7888145250 2024-04-20T21:18:40.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.457+0000 7f7886000640 1 -- 172.21.15.86:0/678304021 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7878031960 con 0x7f7888145250 2024-04-20T21:18:40.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.458+0000 7f788d000640 1 -- 172.21.15.86:0/678304021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7888145250 msgr2=0x7f788806efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:40.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.458+0000 7f788d000640 1 --2- 172.21.15.86:0/678304021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7888145250 0x7f788806efc0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f78780097a0 tx=0x7f787802b840 comp rx=0 tx=0).stop 2024-04-20T21:18:40.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.458+0000 7f788d000640 1 -- 172.21.15.86:0/678304021 shutdown_connections 2024-04-20T21:18:40.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.458+0000 7f788d000640 1 --2- 172.21.15.86:0/678304021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7888145250 0x7f788806efc0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:40.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.458+0000 7f788d000640 1 -- 172.21.15.86:0/678304021 >> 172.21.15.86:0/678304021 conn(0x7f7888067cf0 msgr2=0x7f788806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:40.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.458+0000 7f788d000640 1 -- 172.21.15.86:0/678304021 shutdown_connections 2024-04-20T21:18:40.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.458+0000 7f788d000640 1 -- 172.21.15.86:0/678304021 wait complete. 2024-04-20T21:18:40.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.460+0000 7f788d000640 1 Processor -- start 2024-04-20T21:18:40.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.461+0000 7f788d000640 1 -- start start 2024-04-20T21:18:40.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.462+0000 7f788d000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7888072450 0x7f788813d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:40.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.462+0000 7f788d000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7888145250 0x7f788813c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:40.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.462+0000 7f788d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7888145f00 0x7f7888141df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:40.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.462+0000 7f788d000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f78881457b0 con 0x7f7888145f00 2024-04-20T21:18:40.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.462+0000 7f788d000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7888145630 con 0x7f7888145250 2024-04-20T21:18:40.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.462+0000 7f788d000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f788815d500 con 0x7f7888072450 2024-04-20T21:18:40.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.462+0000 7f7887e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7888145f00 0x7f7888141df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:40.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.462+0000 7f7887e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7888145f00 0x7f7888141df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35986/0 (socket says 172.21.15.86:35986) 2024-04-20T21:18:40.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.462+0000 7f7887e00640 1 -- 172.21.15.86:0/2570822671 learned_addr learned my addr 172.21.15.86:0/2570822671 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:40.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.463+0000 7f7887400640 1 --2- 172.21.15.86:0/2570822671 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7888072450 0x7f788813d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:40.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.463+0000 7f7886a00640 1 --2- 172.21.15.86:0/2570822671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7888145250 0x7f788813c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:40.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.463+0000 7f7887e00640 1 -- 172.21.15.86:0/2570822671 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7888072450 msgr2=0x7f788813d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:40.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.463+0000 7f7887e00640 1 --2- 172.21.15.86:0/2570822671 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7888072450 0x7f788813d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:40.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.463+0000 7f7887e00640 1 -- 172.21.15.86:0/2570822671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7888145250 msgr2=0x7f788813c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:40.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.463+0000 7f7887e00640 1 --2- 172.21.15.86:0/2570822671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7888145250 0x7f788813c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:40.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.463+0000 7f7887e00640 1 -- 172.21.15.86:0/2570822671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78881545b0 con 0x7f7888145f00 2024-04-20T21:18:40.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.463+0000 7f7886a00640 1 --2- 172.21.15.86:0/2570822671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7888145250 0x7f788813c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:40.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.463+0000 7f7887e00640 1 --2- 172.21.15.86:0/2570822671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7888145f00 0x7f7888141df0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f788000c700 tx=0x7f788000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:40.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.463+0000 7f787f400640 1 -- 172.21.15.86:0/2570822671 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7880016020 con 0x7f7888145f00 2024-04-20T21:18:40.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.463+0000 7f787f400640 1 -- 172.21.15.86:0/2570822671 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f788000aa50 con 0x7f7888145f00 2024-04-20T21:18:40.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.463+0000 7f787f400640 1 -- 172.21.15.86:0/2570822671 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788000ce10 con 0x7f7888145f00 2024-04-20T21:18:40.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.463+0000 7f788d000640 1 -- 172.21.15.86:0/2570822671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78881522a0 con 0x7f7888145f00 2024-04-20T21:18:40.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.464+0000 7f788d000640 1 -- 172.21.15.86:0/2570822671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7888152860 con 0x7f7888145f00 2024-04-20T21:18:40.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.465+0000 7f788d000640 1 -- 172.21.15.86:0/2570822671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f788804fd90 con 0x7f7888145f00 2024-04-20T21:18:40.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.466+0000 7f787f400640 1 -- 172.21.15.86:0/2570822671 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7880019020 con 0x7f7888145f00 2024-04-20T21:18:40.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.467+0000 7f787f400640 1 --2- 172.21.15.86:0/2570822671 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f786806c8f0 0x7f786806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:40.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.467+0000 7f787f400640 1 -- 172.21.15.86:0/2570822671 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f788008be00 con 0x7f7888145f00 2024-04-20T21:18:40.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.467+0000 7f7887400640 1 --2- 172.21.15.86:0/2570822671 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f786806c8f0 0x7f786806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:40.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.468+0000 7f7887400640 1 --2- 172.21.15.86:0/2570822671 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f786806c8f0 0x7f786806edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7878036000 tx=0x7f787803d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:40.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.472+0000 7f787f400640 1 -- 172.21.15.86:0/2570822671 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7880020020 con 0x7f7888145f00 2024-04-20T21:18:40.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.673+0000 7f788d000640 1 -- 172.21.15.86:0/2570822671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7888072ea0 con 0x7f7888145f00 2024-04-20T21:18:40.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.674+0000 7f787f400640 1 -- 172.21.15.86:0/2570822671 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f788005a240 con 0x7f7888145f00 2024-04-20T21:18:40.674 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-20T21:18:40.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.676+0000 7f788d000640 1 -- 172.21.15.86:0/2570822671 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f786806c8f0 msgr2=0x7f786806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:40.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.676+0000 7f788d000640 1 --2- 172.21.15.86:0/2570822671 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f786806c8f0 0x7f786806edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7878036000 tx=0x7f787803d040 comp rx=0 tx=0).stop 2024-04-20T21:18:40.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.676+0000 7f788d000640 1 -- 172.21.15.86:0/2570822671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7888145f00 msgr2=0x7f7888141df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:40.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.676+0000 7f788d000640 1 --2- 172.21.15.86:0/2570822671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7888145f00 0x7f7888141df0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f788000c700 tx=0x7f788000cbc0 comp rx=0 tx=0).stop 2024-04-20T21:18:40.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.676+0000 7f788d000640 1 -- 172.21.15.86:0/2570822671 shutdown_connections 2024-04-20T21:18:40.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.676+0000 7f788d000640 1 --2- 172.21.15.86:0/2570822671 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f786806c8f0 0x7f786806edb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:40.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.676+0000 7f788d000640 1 --2- 172.21.15.86:0/2570822671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7888145f00 0x7f7888141df0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:40.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.676+0000 7f788d000640 1 --2- 172.21.15.86:0/2570822671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7888145250 0x7f788813c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:40.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.676+0000 7f788d000640 1 --2- 172.21.15.86:0/2570822671 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7888072450 0x7f788813d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:40.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.676+0000 7f788d000640 1 -- 172.21.15.86:0/2570822671 >> 172.21.15.86:0/2570822671 conn(0x7f7888067cf0 msgr2=0x7f7888143690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:40.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.677+0000 7f788d000640 1 -- 172.21.15.86:0/2570822671 shutdown_connections 2024-04-20T21:18:40.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.677+0000 7f788d000640 1 -- 172.21.15.86:0/2570822671 wait complete. 2024-04-20T21:18:40.695 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:18:40.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.887+0000 7fdfb7200640 1 Processor -- start 2024-04-20T21:18:40.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.887+0000 7fdfb7200640 1 -- start start 2024-04-20T21:18:40.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.888+0000 7fdfb7200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfb80af1f0 0x7fdfb806ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:40.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.888+0000 7fdfb7200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdfb80af5d0 0x7fdfb815de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:40.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.888+0000 7fdfb7200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfb806de80 con 0x7fdfb806d9c0 2024-04-20T21:18:40.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.888+0000 7fdfb7200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fdfb806f460 con 0x7fdfb80af5d0 2024-04-20T21:18:40.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.888+0000 7fdfb7200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdfb806e160 con 0x7fdfb80af1f0 2024-04-20T21:18:40.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.889+0000 7fdfb6800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdfb80af5d0 0x7fdfb815de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:40.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.889+0000 7fdfb6800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdfb80af5d0 0x7fdfb815de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35996/0 (socket says 172.21.15.86:35996) 2024-04-20T21:18:40.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.889+0000 7fdfb6800640 1 -- 172.21.15.86:0/2096766257 learned_addr learned my addr 172.21.15.86:0/2096766257 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:40.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.889+0000 7fdfb5e00640 1 --2- 172.21.15.86:0/2096766257 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfb80af1f0 0x7fdfb806ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:40.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.889+0000 7fdfb6800640 1 -- 172.21.15.86:0/2096766257 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfb80af1f0 msgr2=0x7fdfb806ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:40.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.889+0000 7fdfb6800640 1 --2- 172.21.15.86:0/2096766257 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfb80af1f0 0x7fdfb806ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:40.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.889+0000 7fdfb6800640 1 -- 172.21.15.86:0/2096766257 >> v1:172.21.15.86:6789/0 conn(0x7fdfb806d9c0 legacy=0x7fdfb815d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:40.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.889+0000 7fdfb6800640 1 -- 172.21.15.86:0/2096766257 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfb806e440 con 0x7fdfb80af5d0 2024-04-20T21:18:40.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.890+0000 7fdfb6800640 1 --2- 172.21.15.86:0/2096766257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfb80af5d0 0x7fdfb815de70 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0009e50 tx=0x7fdfb002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b765fef134f40f9f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:40.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.891+0000 7fdfb5400640 1 -- 172.21.15.86:0/2096766257 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb002cb00 con 0x7fdfb80af5d0 2024-04-20T21:18:40.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.891+0000 7fdfb5400640 1 -- 172.21.15.86:0/2096766257 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfb002cca0 con 0x7fdfb80af5d0 2024-04-20T21:18:40.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.891+0000 7fdfb5400640 1 -- 172.21.15.86:0/2096766257 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0034610 con 0x7fdfb80af5d0 2024-04-20T21:18:40.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.892+0000 7fdfb7200640 1 -- 172.21.15.86:0/2096766257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfb80af5d0 msgr2=0x7fdfb815de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:40.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.892+0000 7fdfb7200640 1 --2- 172.21.15.86:0/2096766257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfb80af5d0 0x7fdfb815de70 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0009e50 tx=0x7fdfb002bf60 comp rx=0 tx=0).stop 2024-04-20T21:18:40.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.892+0000 7fdfb7200640 1 -- 172.21.15.86:0/2096766257 shutdown_connections 2024-04-20T21:18:40.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.892+0000 7fdfb7200640 1 --2- 172.21.15.86:0/2096766257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfb80af5d0 0x7fdfb815de70 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:40.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.892+0000 7fdfb7200640 1 --2- 172.21.15.86:0/2096766257 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfb80af1f0 0x7fdfb806ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:40.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.892+0000 7fdfb7200640 1 -- 172.21.15.86:0/2096766257 >> 172.21.15.86:0/2096766257 conn(0x7fdfb80b1f00 msgr2=0x7fdfb80b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:40.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.892+0000 7fdfb7200640 1 -- 172.21.15.86:0/2096766257 shutdown_connections 2024-04-20T21:18:40.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.892+0000 7fdfb7200640 1 -- 172.21.15.86:0/2096766257 wait complete. 2024-04-20T21:18:40.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.894+0000 7fdfb7200640 1 Processor -- start 2024-04-20T21:18:40.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.894+0000 7fdfb7200640 1 -- start start 2024-04-20T21:18:40.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.895+0000 7fdfb7200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfb806d9c0 0x7fdfb81f48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:40.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.895+0000 7fdfb7200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfb80af1f0 0x7fdfb8073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:40.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.895+0000 7fdfb7200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfb80af5d0 0x7fdfb8158830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:40.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.895+0000 7fdfb7200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb80afa80 con 0x7fdfb80af1f0 2024-04-20T21:18:40.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.895+0000 7fdfb7200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb806ea00 con 0x7fdfb80af5d0 2024-04-20T21:18:40.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.895+0000 7fdfb7200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdfb815a3c0 con 0x7fdfb806d9c0 2024-04-20T21:18:40.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.895+0000 7fdfafe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfb80af1f0 0x7fdfb8073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:40.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.895+0000 7fdfafe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfb80af1f0 0x7fdfb8073120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36008/0 (socket says 172.21.15.86:36008) 2024-04-20T21:18:40.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.895+0000 7fdfafe00640 1 -- 172.21.15.86:0/1961236796 learned_addr learned my addr 172.21.15.86:0/1961236796 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:40.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.895+0000 7fdfb5e00640 1 --2- 172.21.15.86:0/1961236796 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfb806d9c0 0x7fdfb81f48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:40.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.895+0000 7fdfb6800640 1 --2- 172.21.15.86:0/1961236796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfb80af5d0 0x7fdfb8158830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:40.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.896+0000 7fdfafe00640 1 -- 172.21.15.86:0/1961236796 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfb806d9c0 msgr2=0x7fdfb81f48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:40.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.896+0000 7fdfafe00640 1 --2- 172.21.15.86:0/1961236796 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfb806d9c0 0x7fdfb81f48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:40.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.896+0000 7fdfafe00640 1 -- 172.21.15.86:0/1961236796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfb80af5d0 msgr2=0x7fdfb8158830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:40.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.896+0000 7fdfafe00640 1 --2- 172.21.15.86:0/1961236796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfb80af5d0 0x7fdfb8158830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:40.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.896+0000 7fdfafe00640 1 -- 172.21.15.86:0/1961236796 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfb8073b00 con 0x7fdfb80af1f0 2024-04-20T21:18:40.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.896+0000 7fdfb6800640 1 --2- 172.21.15.86:0/1961236796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfb80af5d0 0x7fdfb8158830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:40.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.896+0000 7fdfafe00640 1 --2- 172.21.15.86:0/1961236796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfb80af1f0 0x7fdfb8073120 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fdfa000c3c0 tx=0x7fdfa000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:40.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.896+0000 7fdfae000640 1 -- 172.21.15.86:0/1961236796 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa00052d0 con 0x7fdfb80af1f0 2024-04-20T21:18:40.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.896+0000 7fdfae000640 1 -- 172.21.15.86:0/1961236796 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfa0005490 con 0x7fdfb80af1f0 2024-04-20T21:18:40.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.896+0000 7fdfae000640 1 -- 172.21.15.86:0/1961236796 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa0007260 con 0x7fdfb80af1f0 2024-04-20T21:18:40.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.896+0000 7fdfb7200640 1 -- 172.21.15.86:0/1961236796 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfb81f85e0 con 0x7fdfb80af1f0 2024-04-20T21:18:40.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.896+0000 7fdfb7200640 1 -- 172.21.15.86:0/1961236796 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfb81f90c0 con 0x7fdfb80af1f0 2024-04-20T21:18:40.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.897+0000 7fdfb7200640 1 -- 172.21.15.86:0/1961236796 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfb804fd90 con 0x7fdfb80af1f0 2024-04-20T21:18:40.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.898+0000 7fdfae000640 1 -- 172.21.15.86:0/1961236796 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdfa0007400 con 0x7fdfb80af1f0 2024-04-20T21:18:40.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.899+0000 7fdfae000640 1 --2- 172.21.15.86:0/1961236796 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdf9406c8f0 0x7fdf9406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:40.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.899+0000 7fdfae000640 1 -- 172.21.15.86:0/1961236796 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fdfa008b080 con 0x7fdfb80af1f0 2024-04-20T21:18:40.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.900+0000 7fdfb5e00640 1 --2- 172.21.15.86:0/1961236796 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdf9406c8f0 0x7fdf9406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:40.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.900+0000 7fdfb5e00640 1 --2- 172.21.15.86:0/1961236796 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdf9406c8f0 0x7fdf9406edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8006fd0 tx=0x7fdfa8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:40.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:40.904+0000 7fdfae000640 1 -- 172.21.15.86:0/1961236796 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fdfa0059420 con 0x7fdfb80af1f0 2024-04-20T21:18:41.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.103+0000 7fdfb7200640 1 -- 172.21.15.86:0/1961236796 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdfb8159be0 con 0x7fdfb80af1f0 2024-04-20T21:18:41.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.104+0000 7fdfae000640 1 -- 172.21.15.86:0/1961236796 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7fdfa0058d20 con 0x7fdfb80af1f0 2024-04-20T21:18:41.104 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-20T21:18:41.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.106+0000 7fdfb7200640 1 -- 172.21.15.86:0/1961236796 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdf9406c8f0 msgr2=0x7fdf9406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:41.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.106+0000 7fdfb7200640 1 --2- 172.21.15.86:0/1961236796 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdf9406c8f0 0x7fdf9406edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8006fd0 tx=0x7fdfa8008040 comp rx=0 tx=0).stop 2024-04-20T21:18:41.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.106+0000 7fdfb7200640 1 -- 172.21.15.86:0/1961236796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfb80af1f0 msgr2=0x7fdfb8073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:41.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.106+0000 7fdfb7200640 1 --2- 172.21.15.86:0/1961236796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfb80af1f0 0x7fdfb8073120 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fdfa000c3c0 tx=0x7fdfa000c880 comp rx=0 tx=0).stop 2024-04-20T21:18:41.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.107+0000 7fdfb7200640 1 -- 172.21.15.86:0/1961236796 shutdown_connections 2024-04-20T21:18:41.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.107+0000 7fdfb7200640 1 --2- 172.21.15.86:0/1961236796 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdf9406c8f0 0x7fdf9406edb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:41.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.107+0000 7fdfb7200640 1 --2- 172.21.15.86:0/1961236796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfb80af5d0 0x7fdfb8158830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:41.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.107+0000 7fdfb7200640 1 --2- 172.21.15.86:0/1961236796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfb80af1f0 0x7fdfb8073120 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:41.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.107+0000 7fdfb7200640 1 --2- 172.21.15.86:0/1961236796 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfb806d9c0 0x7fdfb81f48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:41.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.107+0000 7fdfb7200640 1 -- 172.21.15.86:0/1961236796 >> 172.21.15.86:0/1961236796 conn(0x7fdfb80b1f00 msgr2=0x7fdfb80b0890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:41.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.107+0000 7fdfb7200640 1 -- 172.21.15.86:0/1961236796 shutdown_connections 2024-04-20T21:18:41.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.107+0000 7fdfb7200640 1 -- 172.21.15.86:0/1961236796 wait complete. 2024-04-20T21:18:41.127 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T21:18:41.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.333+0000 7fe6d5200640 1 Processor -- start 2024-04-20T21:18:41.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.333+0000 7fe6d5200640 1 -- start start 2024-04-20T21:18:41.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.335+0000 7fe6d5200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe6d015b5f0 0x7fe6d00b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:41.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.335+0000 7fe6d5200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6d015b9d0 con 0x7fe6d006f070 2024-04-20T21:18:41.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.335+0000 7fe6d5200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6d00af850 con 0x7fe6d00cff80 2024-04-20T21:18:41.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.335+0000 7fe6d5200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe6d00b4650 con 0x7fe6d015b5f0 2024-04-20T21:18:41.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.335+0000 7fe6cde00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe6d015b5f0 0x7fe6d00b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:41.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.335+0000 7fe6cde00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe6d015b5f0 0x7fe6d00b3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36012/0 (socket says 172.21.15.86:36012) 2024-04-20T21:18:41.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.335+0000 7fe6cde00640 1 -- 172.21.15.86:0/1964711660 learned_addr learned my addr 172.21.15.86:0/1964711660 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:41.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.336+0000 7fe6cde00640 1 -- 172.21.15.86:0/1964711660 >> v1:172.21.15.144:6789/0 conn(0x7fe6d00cff80 legacy=0x7fe6d00ce130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:41.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.336+0000 7fe6cde00640 1 -- 172.21.15.86:0/1964711660 >> v1:172.21.15.86:6789/0 conn(0x7fe6d006f070 legacy=0x7fe6d00b50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:41.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.336+0000 7fe6cde00640 1 -- 172.21.15.86:0/1964711660 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6d00afb30 con 0x7fe6d015b5f0 2024-04-20T21:18:41.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.337+0000 7fe6cde00640 1 --2- 172.21.15.86:0/1964711660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6d015b5f0 0x7fe6d00b3d40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0009870 tx=0x7fe6c002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ac794691eac9c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:41.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.337+0000 7fe6cca00640 1 -- 172.21.15.86:0/1964711660 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c0034070 con 0x7fe6d015b5f0 2024-04-20T21:18:41.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.337+0000 7fe6cca00640 1 -- 172.21.15.86:0/1964711660 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6c0030860 con 0x7fe6d015b5f0 2024-04-20T21:18:41.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.337+0000 7fe6cca00640 1 -- 172.21.15.86:0/1964711660 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c0030c30 con 0x7fe6d015b5f0 2024-04-20T21:18:41.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.338+0000 7fe6d5200640 1 -- 172.21.15.86:0/1964711660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6d015b5f0 msgr2=0x7fe6d00b3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:41.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.338+0000 7fe6d5200640 1 --2- 172.21.15.86:0/1964711660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6d015b5f0 0x7fe6d00b3d40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0009870 tx=0x7fe6c002b9a0 comp rx=0 tx=0).stop 2024-04-20T21:18:41.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.338+0000 7fe6d5200640 1 -- 172.21.15.86:0/1964711660 shutdown_connections 2024-04-20T21:18:41.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.338+0000 7fe6d5200640 1 --2- 172.21.15.86:0/1964711660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6d015b5f0 0x7fe6d00b3d40 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:41.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.338+0000 7fe6d5200640 1 -- 172.21.15.86:0/1964711660 >> 172.21.15.86:0/1964711660 conn(0x7fe6d006b980 msgr2=0x7fe6d006bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:41.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.338+0000 7fe6d5200640 1 -- 172.21.15.86:0/1964711660 shutdown_connections 2024-04-20T21:18:41.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.338+0000 7fe6d5200640 1 -- 172.21.15.86:0/1964711660 wait complete. 2024-04-20T21:18:41.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.340+0000 7fe6d5200640 1 Processor -- start 2024-04-20T21:18:41.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.340+0000 7fe6d5200640 1 -- start start 2024-04-20T21:18:41.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.341+0000 7fe6d5200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6d006f070 0x7fe6d0153540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:41.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.341+0000 7fe6d5200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6d00cff80 0x7fe6d013f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:41.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.341+0000 7fe6d5200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6d015b5f0 0x7fe6d0142b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:41.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.341+0000 7fe6d5200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d00d03d0 con 0x7fe6d015b5f0 2024-04-20T21:18:41.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.341+0000 7fe6d5200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d00b4b80 con 0x7fe6d00cff80 2024-04-20T21:18:41.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.341+0000 7fe6d5200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe6d00d0550 con 0x7fe6d006f070 2024-04-20T21:18:41.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.342+0000 7fe6ce800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6d015b5f0 0x7fe6d0142b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:41.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.342+0000 7fe6ce800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6d015b5f0 0x7fe6d0142b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36022/0 (socket says 172.21.15.86:36022) 2024-04-20T21:18:41.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.342+0000 7fe6ce800640 1 -- 172.21.15.86:0/1742415374 learned_addr learned my addr 172.21.15.86:0/1742415374 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:41.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.342+0000 7fe6cde00640 1 --2- 172.21.15.86:0/1742415374 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6d006f070 0x7fe6d0153540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:41.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.342+0000 7fe6cd400640 1 --2- 172.21.15.86:0/1742415374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6d00cff80 0x7fe6d013f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:41.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.342+0000 7fe6ce800640 1 -- 172.21.15.86:0/1742415374 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6d006f070 msgr2=0x7fe6d0153540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:41.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.342+0000 7fe6ce800640 1 --2- 172.21.15.86:0/1742415374 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6d006f070 0x7fe6d0153540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:41.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.342+0000 7fe6ce800640 1 -- 172.21.15.86:0/1742415374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6d00cff80 msgr2=0x7fe6d013f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:41.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.342+0000 7fe6ce800640 1 --2- 172.21.15.86:0/1742415374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6d00cff80 0x7fe6d013f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:41.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.342+0000 7fe6ce800640 1 -- 172.21.15.86:0/1742415374 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6d013fe50 con 0x7fe6d015b5f0 2024-04-20T21:18:41.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.342+0000 7fe6ce800640 1 --2- 172.21.15.86:0/1742415374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6d015b5f0 0x7fe6d0142b00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe6c800c7b0 tx=0x7fe6c800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:41.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.343+0000 7fe6c6000640 1 -- 172.21.15.86:0/1742415374 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8004b60 con 0x7fe6d015b5f0 2024-04-20T21:18:41.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.343+0000 7fe6c6000640 1 -- 172.21.15.86:0/1742415374 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6c8004d00 con 0x7fe6d015b5f0 2024-04-20T21:18:41.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.343+0000 7fe6c6000640 1 -- 172.21.15.86:0/1742415374 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c800aa90 con 0x7fe6d015b5f0 2024-04-20T21:18:41.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.343+0000 7fe6d5200640 1 -- 172.21.15.86:0/1742415374 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6d0153b60 con 0x7fe6d015b5f0 2024-04-20T21:18:41.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.343+0000 7fe6d5200640 1 -- 172.21.15.86:0/1742415374 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6d0154120 con 0x7fe6d015b5f0 2024-04-20T21:18:41.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.344+0000 7fe6d5200640 1 -- 172.21.15.86:0/1742415374 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6d00689e0 con 0x7fe6d015b5f0 2024-04-20T21:18:41.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.345+0000 7fe6c6000640 1 -- 172.21.15.86:0/1742415374 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe6c800ac30 con 0x7fe6d015b5f0 2024-04-20T21:18:41.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.346+0000 7fe6c6000640 1 --2- 172.21.15.86:0/1742415374 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe6ac06c8f0 0x7fe6ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:41.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.346+0000 7fe6c6000640 1 -- 172.21.15.86:0/1742415374 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fe6c808bef0 con 0x7fe6d015b5f0 2024-04-20T21:18:41.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.346+0000 7fe6cde00640 1 --2- 172.21.15.86:0/1742415374 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe6ac06c8f0 0x7fe6ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:41.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.347+0000 7fe6cde00640 1 --2- 172.21.15.86:0/1742415374 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe6ac06c8f0 0x7fe6ac06edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0036000 tx=0x7fe6c003b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:41.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.350+0000 7fe6c6000640 1 -- 172.21.15.86:0/1742415374 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe6c8020070 con 0x7fe6d015b5f0 2024-04-20T21:18:41.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.564+0000 7fe6d5200640 1 -- 172.21.15.86:0/1742415374 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe6d00b20b0 con 0x7fe6d015b5f0 2024-04-20T21:18:41.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.565+0000 7fe6c6000640 1 -- 172.21.15.86:0/1742415374 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7fe6d013fe50 con 0x7fe6d015b5f0 2024-04-20T21:18:41.565 INFO:teuthology.orchestra.run.smithi086.stderr:listed 0 entries 2024-04-20T21:18:41.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.567+0000 7fe6d5200640 1 -- 172.21.15.86:0/1742415374 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe6ac06c8f0 msgr2=0x7fe6ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:41.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.567+0000 7fe6d5200640 1 --2- 172.21.15.86:0/1742415374 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe6ac06c8f0 0x7fe6ac06edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0036000 tx=0x7fe6c003b040 comp rx=0 tx=0).stop 2024-04-20T21:18:41.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.567+0000 7fe6d5200640 1 -- 172.21.15.86:0/1742415374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6d015b5f0 msgr2=0x7fe6d0142b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:41.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.567+0000 7fe6d5200640 1 --2- 172.21.15.86:0/1742415374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6d015b5f0 0x7fe6d0142b00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe6c800c7b0 tx=0x7fe6c800cc70 comp rx=0 tx=0).stop 2024-04-20T21:18:41.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.567+0000 7fe6d5200640 1 -- 172.21.15.86:0/1742415374 shutdown_connections 2024-04-20T21:18:41.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.567+0000 7fe6d5200640 1 --2- 172.21.15.86:0/1742415374 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe6ac06c8f0 0x7fe6ac06edb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:41.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.567+0000 7fe6d5200640 1 --2- 172.21.15.86:0/1742415374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6d015b5f0 0x7fe6d0142b00 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:41.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.567+0000 7fe6d5200640 1 --2- 172.21.15.86:0/1742415374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6d00cff80 0x7fe6d013f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:41.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.567+0000 7fe6d5200640 1 --2- 172.21.15.86:0/1742415374 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6d006f070 0x7fe6d0153540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:41.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.567+0000 7fe6d5200640 1 -- 172.21.15.86:0/1742415374 >> 172.21.15.86:0/1742415374 conn(0x7fe6d006b980 msgr2=0x7fe6d015a540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:41.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.568+0000 7fe6d5200640 1 -- 172.21.15.86:0/1742415374 shutdown_connections 2024-04-20T21:18:41.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.568+0000 7fe6d5200640 1 -- 172.21.15.86:0/1742415374 wait complete. 2024-04-20T21:18:41.586 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:18:41.586 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:18:41.644 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:18:41.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.847+0000 7fd71b200640 1 Processor -- start 2024-04-20T21:18:41.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.847+0000 7fd71b200640 1 -- start start 2024-04-20T21:18:41.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.849+0000 7fd71b200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd71c0b4270 0x7fd71c06efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:41.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.849+0000 7fd71b200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd71c073bc0 con 0x7fd71c0ad490 2024-04-20T21:18:41.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.849+0000 7fd71b200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd71c06e4e0 con 0x7fd71c15aa40 2024-04-20T21:18:41.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.849+0000 7fd71b200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fd71c073ea0 con 0x7fd71c0b4270 2024-04-20T21:18:41.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.849+0000 7fd711400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd71c0ad490 0x7fd71c0b3b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41794/0 (socket says 172.21.15.86:41794) 2024-04-20T21:18:41.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.849+0000 7fd711400640 1 -- 172.21.15.86:0/2158606802 learned_addr learned my addr 172.21.15.86:0/2158606802 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:18:41.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.849+0000 7fd71a800640 1 --2- 172.21.15.86:0/2158606802 >> v2:172.21.15.144:3300/0 conn(0x7fd71c0b4270 0x7fd71c06efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:41.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.849+0000 7fd719400640 1 -- 172.21.15.86:0/2158606802 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2705990986 0 0) 0x7fd71c073bc0 con 0x7fd71c0ad490 2024-04-20T21:18:41.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.849+0000 7fd719400640 1 -- 172.21.15.86:0/2158606802 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd700003520 con 0x7fd71c0ad490 2024-04-20T21:18:41.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.850+0000 7fd719400640 1 -- 172.21.15.86:0/2158606802 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 327165517 0 0) 0x7fd700003520 con 0x7fd71c0ad490 2024-04-20T21:18:41.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.850+0000 7fd719400640 1 -- 172.21.15.86:0/2158606802 >> v2:172.21.15.144:3300/0 conn(0x7fd71c0b4270 msgr2=0x7fd71c06efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:41.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.850+0000 7fd719400640 1 --2- 172.21.15.86:0/2158606802 >> v2:172.21.15.144:3300/0 conn(0x7fd71c0b4270 0x7fd71c06efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:41.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.850+0000 7fd719400640 1 -- 172.21.15.86:0/2158606802 >> v1:172.21.15.144:6789/0 conn(0x7fd71c15aa40 legacy=0x7fd71c073100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:41.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.850+0000 7fd719400640 1 -- 172.21.15.86:0/2158606802 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd71c06e7c0 con 0x7fd71c0ad490 2024-04-20T21:18:41.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.850+0000 7fd719400640 1 -- 172.21.15.86:0/2158606802 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fd704001670 con 0x7fd71c0ad490 2024-04-20T21:18:41.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.850+0000 7fd719400640 1 -- 172.21.15.86:0/2158606802 >> v1:172.21.15.86:6789/0 conn(0x7fd71c0ad490 legacy=0x7fd71c0b3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:41.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.850+0000 7fd719400640 1 --2- 172.21.15.86:0/2158606802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7000039b0 0x7fd700003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:41.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.850+0000 7fd719400640 1 --2- 172.21.15.86:0/2158606802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd700004800 0x7fd700004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:41.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.850+0000 7fd719400640 1 --2- 172.21.15.86:0/2158606802 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7000052e0 0x7fd70000dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:41.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.850+0000 7fd719400640 1 -- 172.21.15.86:0/2158606802 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd71c073ea0 con 0x7fd7000039b0 2024-04-20T21:18:41.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.850+0000 7fd719400640 1 -- 172.21.15.86:0/2158606802 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd70000e3a0 con 0x7fd700004800 2024-04-20T21:18:41.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.850+0000 7fd719400640 1 -- 172.21.15.86:0/2158606802 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd70000e5a0 con 0x7fd7000052e0 2024-04-20T21:18:41.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.850+0000 7fd719e00640 1 --2- 172.21.15.86:0/2158606802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7000039b0 0x7fd700003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:41.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.851+0000 7fd711400640 1 --2- 172.21.15.86:0/2158606802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd700004800 0x7fd700004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:41.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.851+0000 7fd719e00640 1 -- 172.21.15.86:0/2158606802 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7000052e0 msgr2=0x7fd70000dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:41.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.851+0000 7fd719e00640 1 --2- 172.21.15.86:0/2158606802 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7000052e0 0x7fd70000dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:41.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.851+0000 7fd719e00640 1 -- 172.21.15.86:0/2158606802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd700004800 msgr2=0x7fd700004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:41.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.851+0000 7fd719e00640 1 --2- 172.21.15.86:0/2158606802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd700004800 0x7fd700004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:41.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.851+0000 7fd719e00640 1 -- 172.21.15.86:0/2158606802 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd700003520 con 0x7fd7000039b0 2024-04-20T21:18:41.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.851+0000 7fd711400640 1 --2- 172.21.15.86:0/2158606802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd700004800 0x7fd700004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:41.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.851+0000 7fd71a800640 1 -- 172.21.15.86:0/2158606802 reap_dead start 2024-04-20T21:18:41.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.852+0000 7fd719e00640 1 --2- 172.21.15.86:0/2158606802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7000039b0 0x7fd700003db0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd70c001d20 tx=0x7fd700005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3646c0abe54a7fa server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:41.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.852+0000 7fd719400640 1 -- 172.21.15.86:0/2158606802 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd70c001920 con 0x7fd7000039b0 2024-04-20T21:18:41.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.852+0000 7fd719400640 1 -- 172.21.15.86:0/2158606802 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd70c035070 con 0x7fd7000039b0 2024-04-20T21:18:41.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.852+0000 7fd719400640 1 -- 172.21.15.86:0/2158606802 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd70c003df0 con 0x7fd7000039b0 2024-04-20T21:18:41.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.853+0000 7fd71b200640 1 -- 172.21.15.86:0/2158606802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7000039b0 msgr2=0x7fd700003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:41.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.853+0000 7fd71b200640 1 --2- 172.21.15.86:0/2158606802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7000039b0 0x7fd700003db0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd70c001d20 tx=0x7fd700005830 comp rx=0 tx=0).stop 2024-04-20T21:18:41.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.853+0000 7fd71b200640 1 -- 172.21.15.86:0/2158606802 shutdown_connections 2024-04-20T21:18:41.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.853+0000 7fd71b200640 1 --2- 172.21.15.86:0/2158606802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7000039b0 0x7fd700003db0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:41.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.853+0000 7fd71b200640 1 -- 172.21.15.86:0/2158606802 >> 172.21.15.86:0/2158606802 conn(0x7fd71c0d0170 msgr2=0x7fd71c0d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:41.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.853+0000 7fd71b200640 1 -- 172.21.15.86:0/2158606802 shutdown_connections 2024-04-20T21:18:41.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.853+0000 7fd71b200640 1 -- 172.21.15.86:0/2158606802 wait complete. 2024-04-20T21:18:41.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.855+0000 7fd71b200640 1 Processor -- start 2024-04-20T21:18:41.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.855+0000 7fd71b200640 1 -- start start 2024-04-20T21:18:41.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.856+0000 7fd71b200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7000039b0 0x7fd71c14b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:41.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.856+0000 7fd71b200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd71c15cb00 0x7fd71c099980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:41.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.856+0000 7fd71b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd71c072e60 0x7fd71c0732e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:41.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.856+0000 7fd71b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd71c0b33a0 con 0x7fd71c072e60 2024-04-20T21:18:41.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.856+0000 7fd71b200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd71c0ad920 con 0x7fd7000039b0 2024-04-20T21:18:41.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.856+0000 7fd71b200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd71c15ae20 con 0x7fd71c15cb00 2024-04-20T21:18:41.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.856+0000 7fd71a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd71c072e60 0x7fd71c0732e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:41.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.856+0000 7fd71a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd71c072e60 0x7fd71c0732e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36044/0 (socket says 172.21.15.86:36044) 2024-04-20T21:18:41.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.856+0000 7fd71a800640 1 -- 172.21.15.86:0/3974529086 learned_addr learned my addr 172.21.15.86:0/3974529086 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:41.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.856+0000 7fd719e00640 1 --2- 172.21.15.86:0/3974529086 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7000039b0 0x7fd71c14b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:41.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.856+0000 7fd71a800640 1 -- 172.21.15.86:0/3974529086 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd71c15cb00 msgr2=0x7fd71c099980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:41.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.856+0000 7fd71a800640 1 --2- 172.21.15.86:0/3974529086 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd71c15cb00 0x7fd71c099980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:41.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.856+0000 7fd71a800640 1 -- 172.21.15.86:0/3974529086 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7000039b0 msgr2=0x7fd71c14b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:41.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.856+0000 7fd71a800640 1 --2- 172.21.15.86:0/3974529086 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7000039b0 0x7fd71c14b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:41.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.856+0000 7fd71a800640 1 -- 172.21.15.86:0/3974529086 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd71c15d190 con 0x7fd71c072e60 2024-04-20T21:18:41.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.857+0000 7fd71a800640 1 --2- 172.21.15.86:0/3974529086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd71c072e60 0x7fd71c0732e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd71c06ff80 tx=0x7fd71400d9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:41.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.857+0000 7fd712a00640 1 -- 172.21.15.86:0/3974529086 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd714015070 con 0x7fd71c072e60 2024-04-20T21:18:41.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.857+0000 7fd712a00640 1 -- 172.21.15.86:0/3974529086 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd714004220 con 0x7fd71c072e60 2024-04-20T21:18:41.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.857+0000 7fd712a00640 1 -- 172.21.15.86:0/3974529086 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71400b040 con 0x7fd71c072e60 2024-04-20T21:18:41.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.857+0000 7fd71b200640 1 -- 172.21.15.86:0/3974529086 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd71c1524f0 con 0x7fd71c072e60 2024-04-20T21:18:41.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.857+0000 7fd71b200640 1 -- 172.21.15.86:0/3974529086 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd71c152a70 con 0x7fd71c072e60 2024-04-20T21:18:41.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.858+0000 7fd71b200640 1 -- 172.21.15.86:0/3974529086 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd71c04fcf0 con 0x7fd71c072e60 2024-04-20T21:18:41.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.859+0000 7fd712a00640 1 -- 172.21.15.86:0/3974529086 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd714011670 con 0x7fd71c072e60 2024-04-20T21:18:41.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.861+0000 7fd712a00640 1 --2- 172.21.15.86:0/3974529086 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd6f806c920 0x7fd6f806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:41.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.861+0000 7fd712a00640 1 -- 172.21.15.86:0/3974529086 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fd71408b770 con 0x7fd71c072e60 2024-04-20T21:18:41.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.861+0000 7fd719e00640 1 --2- 172.21.15.86:0/3974529086 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd6f806c920 0x7fd6f806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:41.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.862+0000 7fd719e00640 1 --2- 172.21.15.86:0/3974529086 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd6f806c920 0x7fd6f806ede0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd70c034000 tx=0x7fd70c009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:41.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:41.865+0000 7fd712a00640 1 -- 172.21.15.86:0/3974529086 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd714059b10 con 0x7fd71c072e60 2024-04-20T21:18:42.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.071+0000 7fd71b200640 1 -- 172.21.15.86:0/3974529086 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd71c072180 con 0x7fd71c072e60 2024-04-20T21:18:42.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.073+0000 7fd712a00640 1 -- 172.21.15.86:0/3974529086 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16854 (secure 0 0 0) 0x7fd714059410 con 0x7fd71c072e60 2024-04-20T21:18:42.074 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:18:42.074 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":22,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:18:33.788155+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:17:49.718799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:17:50.683477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:18:42.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.076+0000 7fd71b200640 1 -- 172.21.15.86:0/3974529086 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd6f806c920 msgr2=0x7fd6f806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:42.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.076+0000 7fd71b200640 1 --2- 172.21.15.86:0/3974529086 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd6f806c920 0x7fd6f806ede0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd70c034000 tx=0x7fd70c009030 comp rx=0 tx=0).stop 2024-04-20T21:18:42.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.076+0000 7fd71b200640 1 -- 172.21.15.86:0/3974529086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd71c072e60 msgr2=0x7fd71c0732e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:42.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.076+0000 7fd71b200640 1 --2- 172.21.15.86:0/3974529086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd71c072e60 0x7fd71c0732e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd71c06ff80 tx=0x7fd71400d9a0 comp rx=0 tx=0).stop 2024-04-20T21:18:42.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.076+0000 7fd71b200640 1 -- 172.21.15.86:0/3974529086 shutdown_connections 2024-04-20T21:18:42.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.076+0000 7fd71b200640 1 --2- 172.21.15.86:0/3974529086 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd6f806c920 0x7fd6f806ede0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:42.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.076+0000 7fd71b200640 1 --2- 172.21.15.86:0/3974529086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd71c072e60 0x7fd71c0732e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:42.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.076+0000 7fd71b200640 1 --2- 172.21.15.86:0/3974529086 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd71c15cb00 0x7fd71c099980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:42.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.076+0000 7fd71b200640 1 --2- 172.21.15.86:0/3974529086 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7000039b0 0x7fd71c14b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:42.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.076+0000 7fd71b200640 1 -- 172.21.15.86:0/3974529086 >> 172.21.15.86:0/3974529086 conn(0x7fd71c0d0170 msgr2=0x7fd71c14d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:42.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.076+0000 7fd71b200640 1 -- 172.21.15.86:0/3974529086 shutdown_connections 2024-04-20T21:18:42.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.076+0000 7fd71b200640 1 -- 172.21.15.86:0/3974529086 wait complete. 2024-04-20T21:18:42.096 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:17:49.718799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:17:50.683477+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:18:42.096 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:18:42.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.297+0000 7febe5400640 1 Processor -- start 2024-04-20T21:18:42.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.298+0000 7febe5400640 1 -- start start 2024-04-20T21:18:42.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.299+0000 7febe5400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7febe00cfb80 0x7febe00d0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:42.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.299+0000 7febe5400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febe00d2ff0 con 0x7febe015e870 2024-04-20T21:18:42.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.299+0000 7febe5400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febe00d32d0 con 0x7febe00d4310 2024-04-20T21:18:42.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.299+0000 7febe5400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7febe006d590 con 0x7febe00cfb80 2024-04-20T21:18:42.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.299+0000 7febdfe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7febe015e870 0x7febe006ce80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41798/0 (socket says 172.21.15.86:41798) 2024-04-20T21:18:42.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.299+0000 7febdfe00640 1 -- 172.21.15.86:0/1153693801 learned_addr learned my addr 172.21.15.86:0/1153693801 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:18:42.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.300+0000 7febe4a00640 1 --2- 172.21.15.86:0/1153693801 >> v2:172.21.15.144:3300/0 conn(0x7febe00cfb80 0x7febe00d0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:42.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.300+0000 7febdea00640 1 -- 172.21.15.86:0/1153693801 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2945125409 0 0) 0x7febe00d2ff0 con 0x7febe015e870 2024-04-20T21:18:42.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.300+0000 7febdea00640 1 -- 172.21.15.86:0/1153693801 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7febc4003520 con 0x7febe015e870 2024-04-20T21:18:42.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.300+0000 7febdea00640 1 -- 172.21.15.86:0/1153693801 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1676644783 0 0) 0x7febc4003520 con 0x7febe015e870 2024-04-20T21:18:42.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.300+0000 7febdea00640 1 -- 172.21.15.86:0/1153693801 >> v2:172.21.15.144:3300/0 conn(0x7febe00cfb80 msgr2=0x7febe00d0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:42.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.300+0000 7febdea00640 1 --2- 172.21.15.86:0/1153693801 >> v2:172.21.15.144:3300/0 conn(0x7febe00cfb80 0x7febe00d0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:42.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.300+0000 7febdea00640 1 -- 172.21.15.86:0/1153693801 >> v1:172.21.15.144:6789/0 conn(0x7febe00d4310 legacy=0x7febe00d4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:42.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.300+0000 7febdea00640 1 -- 172.21.15.86:0/1153693801 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febe00cf380 con 0x7febe015e870 2024-04-20T21:18:42.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.300+0000 7febdea00640 1 -- 172.21.15.86:0/1153693801 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7febd0001670 con 0x7febe015e870 2024-04-20T21:18:42.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.300+0000 7febdea00640 1 -- 172.21.15.86:0/1153693801 >> v1:172.21.15.86:6789/0 conn(0x7febe015e870 legacy=0x7febe006ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:42.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.300+0000 7febdea00640 1 --2- 172.21.15.86:0/1153693801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7febc4003a00 0x7febc4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:42.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.300+0000 7febdea00640 1 --2- 172.21.15.86:0/1153693801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc4004800 0x7febc4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:42.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.300+0000 7febdea00640 1 --2- 172.21.15.86:0/1153693801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febc40051c0 0x7febc400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:42.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.300+0000 7febdea00640 1 -- 172.21.15.86:0/1153693801 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7febe006d590 con 0x7febc4004800 2024-04-20T21:18:42.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.300+0000 7febdea00640 1 -- 172.21.15.86:0/1153693801 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7febc400c0c0 con 0x7febc40051c0 2024-04-20T21:18:42.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.300+0000 7febdea00640 1 -- 172.21.15.86:0/1153693801 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7febc400c2c0 con 0x7febc4003a00 2024-04-20T21:18:42.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.301+0000 7febdf400640 1 --2- 172.21.15.86:0/1153693801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc4004800 0x7febc4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:42.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.301+0000 7febdfe00640 1 --2- 172.21.15.86:0/1153693801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7febc4003a00 0x7febc4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:42.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.301+0000 7febe4a00640 1 --2- 172.21.15.86:0/1153693801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febc40051c0 0x7febc400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:42.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.301+0000 7febdf400640 1 -- 172.21.15.86:0/1153693801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7febc4003a00 msgr2=0x7febc4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:42.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.301+0000 7febdf400640 1 --2- 172.21.15.86:0/1153693801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7febc4003a00 0x7febc4003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:42.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.301+0000 7febdf400640 1 -- 172.21.15.86:0/1153693801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febc40051c0 msgr2=0x7febc400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:42.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.301+0000 7febdf400640 1 --2- 172.21.15.86:0/1153693801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febc40051c0 0x7febc400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:42.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.301+0000 7febdf400640 1 -- 172.21.15.86:0/1153693801 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febc4003520 con 0x7febc4004800 2024-04-20T21:18:42.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.301+0000 7febe4a00640 1 -- 172.21.15.86:0/1153693801 reap_dead start 2024-04-20T21:18:42.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.302+0000 7febdf400640 1 --2- 172.21.15.86:0/1153693801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc4004800 0x7febc4004c80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7febc8001750 tx=0x7febc4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc2e054df208475b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:42.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.302+0000 7febdea00640 1 -- 172.21.15.86:0/1153693801 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc8001370 con 0x7febc4004800 2024-04-20T21:18:42.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.302+0000 7febdea00640 1 -- 172.21.15.86:0/1153693801 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febc80052d0 con 0x7febc4004800 2024-04-20T21:18:42.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.302+0000 7febdea00640 1 -- 172.21.15.86:0/1153693801 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc8003830 con 0x7febc4004800 2024-04-20T21:18:42.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.303+0000 7febe5400640 1 -- 172.21.15.86:0/1153693801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc4004800 msgr2=0x7febc4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:42.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.303+0000 7febe5400640 1 --2- 172.21.15.86:0/1153693801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc4004800 0x7febc4004c80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7febc8001750 tx=0x7febc4005830 comp rx=0 tx=0).stop 2024-04-20T21:18:42.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.303+0000 7febe5400640 1 -- 172.21.15.86:0/1153693801 shutdown_connections 2024-04-20T21:18:42.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.303+0000 7febe5400640 1 --2- 172.21.15.86:0/1153693801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc4004800 0x7febc4004c80 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:42.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.303+0000 7febe5400640 1 -- 172.21.15.86:0/1153693801 >> 172.21.15.86:0/1153693801 conn(0x7febe009d9c0 msgr2=0x7febe008bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:42.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.303+0000 7febe5400640 1 -- 172.21.15.86:0/1153693801 shutdown_connections 2024-04-20T21:18:42.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.303+0000 7febe5400640 1 -- 172.21.15.86:0/1153693801 wait complete. 2024-04-20T21:18:42.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.305+0000 7febe5400640 1 Processor -- start 2024-04-20T21:18:42.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.305+0000 7febe5400640 1 -- start start 2024-04-20T21:18:42.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.306+0000 7febe5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febe00c99f0 0x7febe00ce1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:42.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.306+0000 7febe5400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febe0159ba0 0x7febe00cc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:42.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.306+0000 7febe5400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7febe00c9dd0 0x7febe01f4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:42.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.306+0000 7febe5400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7febe00d35b0 con 0x7febe00c99f0 2024-04-20T21:18:42.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.306+0000 7febe5400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7febe015ec50 con 0x7febe0159ba0 2024-04-20T21:18:42.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.306+0000 7febe5400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7febe015edd0 con 0x7febe00c9dd0 2024-04-20T21:18:42.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.306+0000 7febdfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febe00c99f0 0x7febe00ce1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:42.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.306+0000 7febdfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febe00c99f0 0x7febe00ce1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36058/0 (socket says 172.21.15.86:36058) 2024-04-20T21:18:42.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.306+0000 7febdfe00640 1 -- 172.21.15.86:0/2108209372 learned_addr learned my addr 172.21.15.86:0/2108209372 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:42.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.307+0000 7febe4a00640 1 --2- 172.21.15.86:0/2108209372 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7febe00c9dd0 0x7febe01f4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:42.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.307+0000 7febdf400640 1 --2- 172.21.15.86:0/2108209372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febe0159ba0 0x7febe00cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:42.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.307+0000 7febdfe00640 1 -- 172.21.15.86:0/2108209372 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7febe00c9dd0 msgr2=0x7febe01f4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:42.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.307+0000 7febdfe00640 1 --2- 172.21.15.86:0/2108209372 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7febe00c9dd0 0x7febe01f4bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:42.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.307+0000 7febdfe00640 1 -- 172.21.15.86:0/2108209372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febe0159ba0 msgr2=0x7febe00cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:42.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.307+0000 7febdfe00640 1 --2- 172.21.15.86:0/2108209372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febe0159ba0 0x7febe00cc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:42.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.307+0000 7febdfe00640 1 -- 172.21.15.86:0/2108209372 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febe00cce90 con 0x7febe00c99f0 2024-04-20T21:18:42.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.307+0000 7febdfe00640 1 --2- 172.21.15.86:0/2108209372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febe00c99f0 0x7febe00ce1d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7febd000d7a0 tx=0x7febd000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:42.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.307+0000 7febd7e00640 1 -- 172.21.15.86:0/2108209372 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd0019070 con 0x7febe00c99f0 2024-04-20T21:18:42.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.307+0000 7febd7e00640 1 -- 172.21.15.86:0/2108209372 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febd000de50 con 0x7febe00c99f0 2024-04-20T21:18:42.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.307+0000 7febd7e00640 1 -- 172.21.15.86:0/2108209372 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd00047e0 con 0x7febe00c99f0 2024-04-20T21:18:42.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.308+0000 7febe5400640 1 -- 172.21.15.86:0/2108209372 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febe00cd130 con 0x7febe00c99f0 2024-04-20T21:18:42.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.308+0000 7febe5400640 1 -- 172.21.15.86:0/2108209372 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febe00cd6f0 con 0x7febe00c99f0 2024-04-20T21:18:42.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.309+0000 7febe5400640 1 -- 172.21.15.86:0/2108209372 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febe004fd90 con 0x7febe00c99f0 2024-04-20T21:18:42.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.310+0000 7febd7e00640 1 -- 172.21.15.86:0/2108209372 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7febd0004980 con 0x7febe00c99f0 2024-04-20T21:18:42.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.311+0000 7febd7e00640 1 --2- 172.21.15.86:0/2108209372 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7febbc06c8f0 0x7febbc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:42.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.311+0000 7febd7e00640 1 -- 172.21.15.86:0/2108209372 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7febd008b520 con 0x7febe00c99f0 2024-04-20T21:18:42.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.311+0000 7febdf400640 1 --2- 172.21.15.86:0/2108209372 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7febbc06c8f0 0x7febbc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:42.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.312+0000 7febdf400640 1 --2- 172.21.15.86:0/2108209372 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7febbc06c8f0 0x7febbc06edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7febc8033000 tx=0x7febc8009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:42.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.315+0000 7febd7e00640 1 -- 172.21.15.86:0/2108209372 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7febd00598c0 con 0x7febe00c99f0 2024-04-20T21:18:42.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.515+0000 7febe5400640 1 -- 172.21.15.86:0/2108209372 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7febe006cc60 con 0x7febe00c99f0 2024-04-20T21:18:42.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.516+0000 7febd7e00640 1 -- 172.21.15.86:0/2108209372 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7febd00591c0 con 0x7febe00c99f0 2024-04-20T21:18:42.516 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:18:42.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.518+0000 7febe5400640 1 -- 172.21.15.86:0/2108209372 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7febbc06c8f0 msgr2=0x7febbc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:42.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.518+0000 7febe5400640 1 --2- 172.21.15.86:0/2108209372 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7febbc06c8f0 0x7febbc06edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7febc8033000 tx=0x7febc8009030 comp rx=0 tx=0).stop 2024-04-20T21:18:42.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.518+0000 7febe5400640 1 -- 172.21.15.86:0/2108209372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febe00c99f0 msgr2=0x7febe00ce1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:42.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.518+0000 7febe5400640 1 --2- 172.21.15.86:0/2108209372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febe00c99f0 0x7febe00ce1d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7febd000d7a0 tx=0x7febd000dc60 comp rx=0 tx=0).stop 2024-04-20T21:18:42.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.518+0000 7febe5400640 1 -- 172.21.15.86:0/2108209372 shutdown_connections 2024-04-20T21:18:42.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.518+0000 7febe5400640 1 --2- 172.21.15.86:0/2108209372 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7febbc06c8f0 0x7febbc06edb0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:42.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.518+0000 7febe5400640 1 --2- 172.21.15.86:0/2108209372 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7febe00c9dd0 0x7febe01f4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:42.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.518+0000 7febe5400640 1 --2- 172.21.15.86:0/2108209372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febe0159ba0 0x7febe00cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:42.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.519+0000 7febe5400640 1 --2- 172.21.15.86:0/2108209372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febe00c99f0 0x7febe00ce1d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:42.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.519+0000 7febe5400640 1 -- 172.21.15.86:0/2108209372 >> 172.21.15.86:0/2108209372 conn(0x7febe009d9c0 msgr2=0x7febe006ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:42.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.519+0000 7febe5400640 1 -- 172.21.15.86:0/2108209372 shutdown_connections 2024-04-20T21:18:42.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.519+0000 7febe5400640 1 -- 172.21.15.86:0/2108209372 wait complete. 2024-04-20T21:18:42.537 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:18:42.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.738+0000 7f9d27e00640 1 Processor -- start 2024-04-20T21:18:42.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.738+0000 7f9d27e00640 1 -- start start 2024-04-20T21:18:42.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.739+0000 7f9d27e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9d28161c60 0x7f9d2815f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:42.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.739+0000 7f9d27e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9d2815ff10 0x7f9d2806dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:42.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.740+0000 7f9d27e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d280aecd0 con 0x7f9d28071d40 2024-04-20T21:18:42.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.740+0000 7f9d27e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9d2806e2e0 con 0x7f9d2815ff10 2024-04-20T21:18:42.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.740+0000 7f9d27e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f9d2806e580 con 0x7f9d28161c60 2024-04-20T21:18:42.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.740+0000 7f9d1e000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9d2815ff10 0x7f9d2806dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:42.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.740+0000 7f9d1e000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9d2815ff10 0x7f9d2806dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36066/0 (socket says 172.21.15.86:36066) 2024-04-20T21:18:42.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.740+0000 7f9d1e000640 1 -- 172.21.15.86:0/1570062050 learned_addr learned my addr 172.21.15.86:0/1570062050 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:42.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.740+0000 7f9d1e000640 1 -- 172.21.15.86:0/1570062050 >> v2:172.21.15.144:3300/0 conn(0x7f9d28161c60 msgr2=0x7f9d2815f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:42.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.740+0000 7f9d26a00640 1 --2- 172.21.15.86:0/1570062050 >> v2:172.21.15.144:3300/0 conn(0x7f9d28161c60 0x7f9d2815f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:42.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.740+0000 7f9d1e000640 1 --2- 172.21.15.86:0/1570062050 >> v2:172.21.15.144:3300/0 conn(0x7f9d28161c60 0x7f9d2815f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:42.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.740+0000 7f9d1e000640 1 -- 172.21.15.86:0/1570062050 >> v1:172.21.15.86:6789/0 conn(0x7f9d28071d40 legacy=0x7f9d280af1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:42.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.740+0000 7f9d1e000640 1 -- 172.21.15.86:0/1570062050 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d2806e860 con 0x7f9d2815ff10 2024-04-20T21:18:42.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.740+0000 7f9d26a00640 1 --2- 172.21.15.86:0/1570062050 >> v2:172.21.15.144:3300/0 conn(0x7f9d28161c60 0x7f9d2815f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:42.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.742+0000 7f9d1e000640 1 --2- 172.21.15.86:0/1570062050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d2815ff10 0x7f9d2806dda0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9d2806c4d0 tx=0x7f9d1002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=103e696021f6e810 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:42.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.742+0000 7f9d26000640 1 -- 172.21.15.86:0/1570062050 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d10034070 con 0x7f9d2815ff10 2024-04-20T21:18:42.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.742+0000 7f9d26000640 1 -- 172.21.15.86:0/1570062050 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d10030860 con 0x7f9d2815ff10 2024-04-20T21:18:42.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.742+0000 7f9d26000640 1 -- 172.21.15.86:0/1570062050 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d10030c30 con 0x7f9d2815ff10 2024-04-20T21:18:42.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.742+0000 7f9d27e00640 1 -- 172.21.15.86:0/1570062050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d2815ff10 msgr2=0x7f9d2806dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:42.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.742+0000 7f9d27e00640 1 --2- 172.21.15.86:0/1570062050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d2815ff10 0x7f9d2806dda0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9d2806c4d0 tx=0x7f9d1002b9a0 comp rx=0 tx=0).stop 2024-04-20T21:18:42.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.743+0000 7f9d27e00640 1 -- 172.21.15.86:0/1570062050 shutdown_connections 2024-04-20T21:18:42.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.743+0000 7f9d27e00640 1 --2- 172.21.15.86:0/1570062050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d2815ff10 0x7f9d2806dda0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:42.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.743+0000 7f9d27e00640 1 --2- 172.21.15.86:0/1570062050 >> v2:172.21.15.144:3300/0 conn(0x7f9d28161c60 0x7f9d2815f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:42.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.743+0000 7f9d27e00640 1 -- 172.21.15.86:0/1570062050 >> 172.21.15.86:0/1570062050 conn(0x7f9d28067cf0 msgr2=0x7f9d2806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:42.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.743+0000 7f9d27e00640 1 -- 172.21.15.86:0/1570062050 shutdown_connections 2024-04-20T21:18:42.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.743+0000 7f9d27e00640 1 -- 172.21.15.86:0/1570062050 wait complete. 2024-04-20T21:18:42.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.744+0000 7f9d27e00640 1 Processor -- start 2024-04-20T21:18:42.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.745+0000 7f9d27e00640 1 -- start start 2024-04-20T21:18:42.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.745+0000 7f9d27e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d28071d40 0x7f9d281f49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:42.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.745+0000 7f9d27e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d2815ff10 0x7f9d281f7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:42.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.745+0000 7f9d27e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d28161c60 0x7f9d280cf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:42.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.745+0000 7f9d27e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9d28162040 con 0x7f9d28071d40 2024-04-20T21:18:42.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.745+0000 7f9d27e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9d2806ee20 con 0x7f9d28161c60 2024-04-20T21:18:42.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.745+0000 7f9d27e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9d28160370 con 0x7f9d2815ff10 2024-04-20T21:18:42.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.746+0000 7f9d26a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d28071d40 0x7f9d281f49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:42.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.746+0000 7f9d26a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d28071d40 0x7f9d281f49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36078/0 (socket says 172.21.15.86:36078) 2024-04-20T21:18:42.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.746+0000 7f9d26a00640 1 -- 172.21.15.86:0/1337479535 learned_addr learned my addr 172.21.15.86:0/1337479535 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:42.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.746+0000 7f9d26a00640 1 -- 172.21.15.86:0/1337479535 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d2815ff10 msgr2=0x7f9d281f7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:42.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.746+0000 7f9d26a00640 1 --2- 172.21.15.86:0/1337479535 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d2815ff10 0x7f9d281f7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:42.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.746+0000 7f9d26a00640 1 -- 172.21.15.86:0/1337479535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d28161c60 msgr2=0x7f9d280cf4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:42.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.746+0000 7f9d26a00640 1 --2- 172.21.15.86:0/1337479535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d28161c60 0x7f9d280cf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:42.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.746+0000 7f9d26a00640 1 -- 172.21.15.86:0/1337479535 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d2815e6e0 con 0x7f9d28071d40 2024-04-20T21:18:42.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.746+0000 7f9d26a00640 1 --2- 172.21.15.86:0/1337479535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d28071d40 0x7f9d281f49a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9d1800c820 tx=0x7f9d1800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:42.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.747+0000 7f9d1f400640 1 -- 172.21.15.86:0/1337479535 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d18015070 con 0x7f9d28071d40 2024-04-20T21:18:42.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.747+0000 7f9d1f400640 1 -- 172.21.15.86:0/1337479535 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d18007a30 con 0x7f9d28071d40 2024-04-20T21:18:42.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.747+0000 7f9d1f400640 1 -- 172.21.15.86:0/1337479535 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d18007de0 con 0x7f9d28071d40 2024-04-20T21:18:42.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.747+0000 7f9d27e00640 1 -- 172.21.15.86:0/1337479535 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d281f4ee0 con 0x7f9d28071d40 2024-04-20T21:18:42.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.747+0000 7f9d27e00640 1 -- 172.21.15.86:0/1337479535 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d281f8290 con 0x7f9d28071d40 2024-04-20T21:18:42.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.748+0000 7f9d27e00640 1 -- 172.21.15.86:0/1337479535 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d2804fd90 con 0x7f9d28071d40 2024-04-20T21:18:42.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.749+0000 7f9d1f400640 1 -- 172.21.15.86:0/1337479535 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9d18018070 con 0x7f9d28071d40 2024-04-20T21:18:42.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.750+0000 7f9d1f400640 1 --2- 172.21.15.86:0/1337479535 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9d0406c8f0 0x7f9d0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:42.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.750+0000 7f9d1f400640 1 -- 172.21.15.86:0/1337479535 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f9d1808bad0 con 0x7f9d28071d40 2024-04-20T21:18:42.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.750+0000 7f9d1e000640 1 --2- 172.21.15.86:0/1337479535 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9d0406c8f0 0x7f9d0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:42.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.751+0000 7f9d1e000640 1 --2- 172.21.15.86:0/1337479535 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9d0406c8f0 0x7f9d0406edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9d10036000 tx=0x7f9d1003b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:42.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.755+0000 7f9d1f400640 1 -- 172.21.15.86:0/1337479535 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9d281f4ee0 con 0x7f9d28071d40 2024-04-20T21:18:42.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.955+0000 7f9d27e00640 1 -- 172.21.15.86:0/1337479535 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9d28068e70 con 0x7f9d28071d40 2024-04-20T21:18:42.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.956+0000 7f9d1f400640 1 -- 172.21.15.86:0/1337479535 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f9d18059e70 con 0x7f9d28071d40 2024-04-20T21:18:42.956 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-20T21:18:42.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.959+0000 7f9d27e00640 1 -- 172.21.15.86:0/1337479535 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9d0406c8f0 msgr2=0x7f9d0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:42.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.959+0000 7f9d27e00640 1 --2- 172.21.15.86:0/1337479535 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9d0406c8f0 0x7f9d0406edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9d10036000 tx=0x7f9d1003b040 comp rx=0 tx=0).stop 2024-04-20T21:18:42.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.959+0000 7f9d27e00640 1 -- 172.21.15.86:0/1337479535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d28071d40 msgr2=0x7f9d281f49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:42.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.959+0000 7f9d27e00640 1 --2- 172.21.15.86:0/1337479535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d28071d40 0x7f9d281f49a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9d1800c820 tx=0x7f9d1800cce0 comp rx=0 tx=0).stop 2024-04-20T21:18:42.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.959+0000 7f9d27e00640 1 -- 172.21.15.86:0/1337479535 shutdown_connections 2024-04-20T21:18:42.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.959+0000 7f9d27e00640 1 --2- 172.21.15.86:0/1337479535 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9d0406c8f0 0x7f9d0406edb0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:42.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.959+0000 7f9d27e00640 1 --2- 172.21.15.86:0/1337479535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d28161c60 0x7f9d280cf4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:42.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.959+0000 7f9d27e00640 1 --2- 172.21.15.86:0/1337479535 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9d2815ff10 0x7f9d281f7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:42.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.959+0000 7f9d27e00640 1 --2- 172.21.15.86:0/1337479535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9d28071d40 0x7f9d281f49a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:42.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.959+0000 7f9d27e00640 1 -- 172.21.15.86:0/1337479535 >> 172.21.15.86:0/1337479535 conn(0x7f9d28067cf0 msgr2=0x7f9d28158c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:42.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.959+0000 7f9d27e00640 1 -- 172.21.15.86:0/1337479535 shutdown_connections 2024-04-20T21:18:42.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:42.959+0000 7f9d27e00640 1 -- 172.21.15.86:0/1337479535 wait complete. 2024-04-20T21:18:42.977 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:18:43.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.183+0000 7ff114c00640 1 Processor -- start 2024-04-20T21:18:43.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.183+0000 7ff114c00640 1 -- start start 2024-04-20T21:18:43.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.184+0000 7ff114c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1100af1f0 0x7ff11006ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:43.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.185+0000 7ff114c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff11006de80 con 0x7ff1100af5d0 2024-04-20T21:18:43.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.185+0000 7ff114c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff11006e160 con 0x7ff11006d9c0 2024-04-20T21:18:43.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.185+0000 7ff114c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff11006f460 con 0x7ff1100af1f0 2024-04-20T21:18:43.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.185+0000 7ff10fe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7ff1100af5d0 0x7ff11015de70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41820/0 (socket says 172.21.15.86:41820) 2024-04-20T21:18:43.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.185+0000 7ff10fe00640 1 -- 172.21.15.86:0/2835185672 learned_addr learned my addr 172.21.15.86:0/2835185672 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:18:43.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.185+0000 7ff10f400640 1 --2- 172.21.15.86:0/2835185672 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1100af1f0 0x7ff11006ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:43.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.185+0000 7ff10e000640 1 -- 172.21.15.86:0/2835185672 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3967831814 0 0) 0x7ff11006de80 con 0x7ff1100af5d0 2024-04-20T21:18:43.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.186+0000 7ff10e000640 1 -- 172.21.15.86:0/2835185672 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0f4003520 con 0x7ff1100af5d0 2024-04-20T21:18:43.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.186+0000 7ff10e000640 1 -- 172.21.15.86:0/2835185672 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3569922163 0 0) 0x7ff0f4003520 con 0x7ff1100af5d0 2024-04-20T21:18:43.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.186+0000 7ff10e000640 1 -- 172.21.15.86:0/2835185672 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1100af1f0 msgr2=0x7ff11006ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:43.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.186+0000 7ff10e000640 1 --2- 172.21.15.86:0/2835185672 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1100af1f0 0x7ff11006ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:43.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.186+0000 7ff10e000640 1 -- 172.21.15.86:0/2835185672 >> v1:172.21.15.144:6789/0 conn(0x7ff11006d9c0 legacy=0x7ff11015d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:43.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.186+0000 7ff10e000640 1 -- 172.21.15.86:0/2835185672 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff11006e440 con 0x7ff1100af5d0 2024-04-20T21:18:43.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.186+0000 7ff10e000640 1 -- 172.21.15.86:0/2835185672 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7ff1080019f0 con 0x7ff1100af5d0 2024-04-20T21:18:43.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.186+0000 7ff10e000640 1 -- 172.21.15.86:0/2835185672 >> v1:172.21.15.86:6789/0 conn(0x7ff1100af5d0 legacy=0x7ff11015de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:43.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.186+0000 7ff10e000640 1 --2- 172.21.15.86:0/2835185672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0f40039e0 0x7ff0f4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:43.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.186+0000 7ff10e000640 1 --2- 172.21.15.86:0/2835185672 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1100af1f0 0x7ff0f4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:43.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.186+0000 7ff10e000640 1 --2- 172.21.15.86:0/2835185672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0f4004d40 0x7ff0f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:43.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.186+0000 7ff10e000640 1 -- 172.21.15.86:0/2835185672 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff11006f460 con 0x7ff0f40039e0 2024-04-20T21:18:43.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.186+0000 7ff10e000640 1 -- 172.21.15.86:0/2835185672 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f400ba70 con 0x7ff0f4004d40 2024-04-20T21:18:43.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.186+0000 7ff10e000640 1 -- 172.21.15.86:0/2835185672 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff0f400bc70 con 0x7ff1100af1f0 2024-04-20T21:18:43.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.186+0000 7ff10f400640 1 --2- 172.21.15.86:0/2835185672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0f40039e0 0x7ff0f4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:43.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.187+0000 7ff10ea00640 1 --2- 172.21.15.86:0/2835185672 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1100af1f0 0x7ff0f4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:43.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.187+0000 7ff10fe00640 1 --2- 172.21.15.86:0/2835185672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0f4004d40 0x7ff0f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:43.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.187+0000 7ff10f400640 1 -- 172.21.15.86:0/2835185672 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1100af1f0 msgr2=0x7ff0f4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:43.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.187+0000 7ff10f400640 1 --2- 172.21.15.86:0/2835185672 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1100af1f0 0x7ff0f4004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:43.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.187+0000 7ff10f400640 1 -- 172.21.15.86:0/2835185672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0f4004d40 msgr2=0x7ff0f40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:43.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.187+0000 7ff10f400640 1 --2- 172.21.15.86:0/2835185672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0f4004d40 0x7ff0f40051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:43.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.187+0000 7ff10f400640 1 -- 172.21.15.86:0/2835185672 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0f4003520 con 0x7ff0f40039e0 2024-04-20T21:18:43.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.188+0000 7ff10f400640 1 --2- 172.21.15.86:0/2835185672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0f40039e0 0x7ff0f4003e00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff100009a90 tx=0x7ff0f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52c80668c91af0dd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:43.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.188+0000 7ff10e000640 1 -- 172.21.15.86:0/2835185672 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff100039070 con 0x7ff0f40039e0 2024-04-20T21:18:43.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.188+0000 7ff10e000640 1 -- 172.21.15.86:0/2835185672 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff10002c680 con 0x7ff0f40039e0 2024-04-20T21:18:43.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.188+0000 7ff10e000640 1 -- 172.21.15.86:0/2835185672 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff100009690 con 0x7ff0f40039e0 2024-04-20T21:18:43.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.189+0000 7ff114c00640 1 -- 172.21.15.86:0/2835185672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0f40039e0 msgr2=0x7ff0f4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:43.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.189+0000 7ff114c00640 1 --2- 172.21.15.86:0/2835185672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0f40039e0 0x7ff0f4003e00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff100009a90 tx=0x7ff0f4005830 comp rx=0 tx=0).stop 2024-04-20T21:18:43.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.189+0000 7ff10fe00640 1 -- 172.21.15.86:0/2835185672 reap_dead start 2024-04-20T21:18:43.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.189+0000 7ff114c00640 1 -- 172.21.15.86:0/2835185672 shutdown_connections 2024-04-20T21:18:43.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.189+0000 7ff114c00640 1 -- 172.21.15.86:0/2835185672 >> 172.21.15.86:0/2835185672 conn(0x7ff1100b1f00 msgr2=0x7ff1100b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:43.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.189+0000 7ff114c00640 1 -- 172.21.15.86:0/2835185672 shutdown_connections 2024-04-20T21:18:43.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.189+0000 7ff114c00640 1 -- 172.21.15.86:0/2835185672 wait complete. 2024-04-20T21:18:43.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.191+0000 7ff114c00640 1 Processor -- start 2024-04-20T21:18:43.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.191+0000 7ff114c00640 1 -- start start 2024-04-20T21:18:43.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.192+0000 7ff114c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0f40039e0 0x7ff110143f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:43.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.192+0000 7ff114c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff110145630 0x7ff11015ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:43.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.192+0000 7ff114c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1100994e0 0x7ff1101e7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:43.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.192+0000 7ff114c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff1100afa80 con 0x7ff1100994e0 2024-04-20T21:18:43.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.192+0000 7ff114c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff11006ea00 con 0x7ff0f40039e0 2024-04-20T21:18:43.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.192+0000 7ff114c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff11015a3c0 con 0x7ff110145630 2024-04-20T21:18:43.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.192+0000 7ff10fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1100994e0 0x7ff1101e7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:43.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.192+0000 7ff10fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1100994e0 0x7ff1101e7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36102/0 (socket says 172.21.15.86:36102) 2024-04-20T21:18:43.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.192+0000 7ff10fe00640 1 -- 172.21.15.86:0/2212973250 learned_addr learned my addr 172.21.15.86:0/2212973250 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:43.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.193+0000 7ff10f400640 1 --2- 172.21.15.86:0/2212973250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0f40039e0 0x7ff110143f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:43.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.193+0000 7ff10ea00640 1 --2- 172.21.15.86:0/2212973250 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff110145630 0x7ff11015ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:43.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.193+0000 7ff10fe00640 1 -- 172.21.15.86:0/2212973250 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff110145630 msgr2=0x7ff11015ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:43.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.193+0000 7ff10fe00640 1 --2- 172.21.15.86:0/2212973250 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff110145630 0x7ff11015ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:43.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.193+0000 7ff10fe00640 1 -- 172.21.15.86:0/2212973250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0f40039e0 msgr2=0x7ff110143f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:43.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.193+0000 7ff10fe00640 1 --2- 172.21.15.86:0/2212973250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0f40039e0 0x7ff110143f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:43.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.193+0000 7ff10fe00640 1 -- 172.21.15.86:0/2212973250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff110144590 con 0x7ff1100994e0 2024-04-20T21:18:43.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.193+0000 7ff10fe00640 1 --2- 172.21.15.86:0/2212973250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1100994e0 0x7ff1101e7f60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff108001350 tx=0x7ff10800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:43.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.193+0000 7ff107400640 1 -- 172.21.15.86:0/2212973250 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff108004d60 con 0x7ff1100994e0 2024-04-20T21:18:43.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.193+0000 7ff107400640 1 -- 172.21.15.86:0/2212973250 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff10800bb20 con 0x7ff1100994e0 2024-04-20T21:18:43.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.194+0000 7ff114c00640 1 -- 172.21.15.86:0/2212973250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff110144870 con 0x7ff1100994e0 2024-04-20T21:18:43.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.194+0000 7ff107400640 1 -- 172.21.15.86:0/2212973250 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff108007110 con 0x7ff1100994e0 2024-04-20T21:18:43.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.194+0000 7ff114c00640 1 -- 172.21.15.86:0/2212973250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff110144e30 con 0x7ff1100994e0 2024-04-20T21:18:43.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.195+0000 7ff114c00640 1 -- 172.21.15.86:0/2212973250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff11004fd90 con 0x7ff1100994e0 2024-04-20T21:18:43.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.196+0000 7ff107400640 1 -- 172.21.15.86:0/2212973250 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff10801b060 con 0x7ff1100994e0 2024-04-20T21:18:43.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.197+0000 7ff107400640 1 --2- 172.21.15.86:0/2212973250 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff0f006c8f0 0x7ff0f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:43.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.197+0000 7ff107400640 1 -- 172.21.15.86:0/2212973250 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7ff10808bad0 con 0x7ff1100994e0 2024-04-20T21:18:43.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.197+0000 7ff10f400640 1 --2- 172.21.15.86:0/2212973250 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff0f006c8f0 0x7ff0f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:43.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.198+0000 7ff10f400640 1 --2- 172.21.15.86:0/2212973250 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff0f006c8f0 0x7ff0f006edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff100036000 tx=0x7ff100030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:43.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.202+0000 7ff107400640 1 -- 172.21.15.86:0/2212973250 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff108059f10 con 0x7ff1100994e0 2024-04-20T21:18:43.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.396+0000 7ff114c00640 1 -- 172.21.15.86:0/2212973250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff1100b3300 con 0x7ff1100994e0 2024-04-20T21:18:43.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.397+0000 7ff107400640 1 -- 172.21.15.86:0/2212973250 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7ff108059810 con 0x7ff1100994e0 2024-04-20T21:18:43.397 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-20T21:18:43.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.400+0000 7ff114c00640 1 -- 172.21.15.86:0/2212973250 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff0f006c8f0 msgr2=0x7ff0f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:43.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.400+0000 7ff114c00640 1 --2- 172.21.15.86:0/2212973250 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff0f006c8f0 0x7ff0f006edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff100036000 tx=0x7ff100030000 comp rx=0 tx=0).stop 2024-04-20T21:18:43.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.400+0000 7ff114c00640 1 -- 172.21.15.86:0/2212973250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1100994e0 msgr2=0x7ff1101e7f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:43.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.400+0000 7ff114c00640 1 --2- 172.21.15.86:0/2212973250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1100994e0 0x7ff1101e7f60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff108001350 tx=0x7ff10800b4a0 comp rx=0 tx=0).stop 2024-04-20T21:18:43.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.400+0000 7ff114c00640 1 -- 172.21.15.86:0/2212973250 shutdown_connections 2024-04-20T21:18:43.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.400+0000 7ff114c00640 1 --2- 172.21.15.86:0/2212973250 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff0f006c8f0 0x7ff0f006edb0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:43.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.400+0000 7ff114c00640 1 --2- 172.21.15.86:0/2212973250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1100994e0 0x7ff1101e7f60 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:43.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.400+0000 7ff114c00640 1 --2- 172.21.15.86:0/2212973250 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff110145630 0x7ff11015ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:43.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.400+0000 7ff114c00640 1 --2- 172.21.15.86:0/2212973250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0f40039e0 0x7ff110143f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:43.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.400+0000 7ff114c00640 1 -- 172.21.15.86:0/2212973250 >> 172.21.15.86:0/2212973250 conn(0x7ff1100b1f00 msgr2=0x7ff1100b4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:43.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.400+0000 7ff114c00640 1 -- 172.21.15.86:0/2212973250 shutdown_connections 2024-04-20T21:18:43.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.400+0000 7ff114c00640 1 -- 172.21.15.86:0/2212973250 wait complete. 2024-04-20T21:18:43.417 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T21:18:43.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.618+0000 7fa78e400640 1 Processor -- start 2024-04-20T21:18:43.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.618+0000 7fa78e400640 1 -- start start 2024-04-20T21:18:43.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.619+0000 7fa78e400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa78815e690 0x7fa78806ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:43.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.620+0000 7fa78e400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7880d3100 con 0x7fa7880d0ee0 2024-04-20T21:18:43.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.620+0000 7fa78e400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7880d33b0 con 0x7fa7880d2cb0 2024-04-20T21:18:43.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.620+0000 7fa78e400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa7880d3690 con 0x7fa78815e690 2024-04-20T21:18:43.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.620+0000 7fa78d000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa78815e690 0x7fa78806ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:43.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.620+0000 7fa787e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fa7880d0ee0 0x7fa7880d25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41826/0 (socket says 172.21.15.86:41826) 2024-04-20T21:18:43.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.620+0000 7fa787e00640 1 -- 172.21.15.86:0/3201583711 learned_addr learned my addr 172.21.15.86:0/3201583711 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:18:43.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.620+0000 7fa78d000640 1 -- 172.21.15.86:0/3201583711 >> v1:172.21.15.144:6789/0 conn(0x7fa7880d2cb0 legacy=0x7fa7880cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:43.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.620+0000 7fa78d000640 1 -- 172.21.15.86:0/3201583711 >> v1:172.21.15.86:6789/0 conn(0x7fa7880d0ee0 legacy=0x7fa7880d25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:43.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.620+0000 7fa78d000640 1 -- 172.21.15.86:0/3201583711 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7880d3970 con 0x7fa78815e690 2024-04-20T21:18:43.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.622+0000 7fa78d000640 1 --2- 172.21.15.86:0/3201583711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa78815e690 0x7fa78806ce20 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa7780097a0 tx=0x7fa77802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b42051889b35cf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:43.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.622+0000 7fa787400640 1 -- 172.21.15.86:0/3201583711 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa778035020 con 0x7fa78815e690 2024-04-20T21:18:43.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.622+0000 7fa787400640 1 -- 172.21.15.86:0/3201583711 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa778031830 con 0x7fa78815e690 2024-04-20T21:18:43.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.622+0000 7fa787400640 1 -- 172.21.15.86:0/3201583711 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa778031c00 con 0x7fa78815e690 2024-04-20T21:18:43.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.622+0000 7fa78e400640 1 -- 172.21.15.86:0/3201583711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa78815e690 msgr2=0x7fa78806ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:43.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.622+0000 7fa78e400640 1 --2- 172.21.15.86:0/3201583711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa78815e690 0x7fa78806ce20 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa7780097a0 tx=0x7fa77802b8d0 comp rx=0 tx=0).stop 2024-04-20T21:18:43.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.623+0000 7fa78e400640 1 -- 172.21.15.86:0/3201583711 shutdown_connections 2024-04-20T21:18:43.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.623+0000 7fa78e400640 1 --2- 172.21.15.86:0/3201583711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa78815e690 0x7fa78806ce20 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:43.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.623+0000 7fa78e400640 1 -- 172.21.15.86:0/3201583711 >> 172.21.15.86:0/3201583711 conn(0x7fa78809d8a0 msgr2=0x7fa78808bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:43.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.623+0000 7fa78e400640 1 -- 172.21.15.86:0/3201583711 shutdown_connections 2024-04-20T21:18:43.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.623+0000 7fa78e400640 1 -- 172.21.15.86:0/3201583711 wait complete. 2024-04-20T21:18:43.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.624+0000 7fa78e400640 1 Processor -- start 2024-04-20T21:18:43.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.624+0000 7fa78e400640 1 -- start start 2024-04-20T21:18:43.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.625+0000 7fa78e400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7880d0ee0 0x7fa7880c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:43.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.625+0000 7fa78e400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7880d2cb0 0x7fa788152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:43.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.625+0000 7fa78e400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa78815e690 0x7fa7880c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:43.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.625+0000 7fa78e400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa78806d360 con 0x7fa7880d2cb0 2024-04-20T21:18:43.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.625+0000 7fa78e400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa7880cef40 con 0x7fa78815e690 2024-04-20T21:18:43.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.625+0000 7fa78e400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa78815ea70 con 0x7fa7880d0ee0 2024-04-20T21:18:43.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.626+0000 7fa787e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7880d2cb0 0x7fa788152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:43.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.626+0000 7fa787e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7880d2cb0 0x7fa788152ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36116/0 (socket says 172.21.15.86:36116) 2024-04-20T21:18:43.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.626+0000 7fa787e00640 1 -- 172.21.15.86:0/1980413331 learned_addr learned my addr 172.21.15.86:0/1980413331 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:43.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.626+0000 7fa787e00640 1 -- 172.21.15.86:0/1980413331 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7880d0ee0 msgr2=0x7fa7880c99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:43.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.626+0000 7fa787e00640 1 --2- 172.21.15.86:0/1980413331 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7880d0ee0 0x7fa7880c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:43.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.626+0000 7fa787e00640 1 -- 172.21.15.86:0/1980413331 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa78815e690 msgr2=0x7fa7880c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:43.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.626+0000 7fa78da00640 1 --2- 172.21.15.86:0/1980413331 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa78815e690 0x7fa7880c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:43.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.626+0000 7fa787e00640 1 --2- 172.21.15.86:0/1980413331 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa78815e690 0x7fa7880c8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:43.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.626+0000 7fa787e00640 1 -- 172.21.15.86:0/1980413331 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7880ca0a0 con 0x7fa7880d2cb0 2024-04-20T21:18:43.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.626+0000 7fa787e00640 1 --2- 172.21.15.86:0/1980413331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7880d2cb0 0x7fa788152ab0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa77000c7b0 tx=0x7fa77000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:43.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.626+0000 7fa784c00640 1 -- 172.21.15.86:0/1980413331 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa770017070 con 0x7fa7880d2cb0 2024-04-20T21:18:43.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.626+0000 7fa784c00640 1 -- 172.21.15.86:0/1980413331 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa77000a640 con 0x7fa7880d2cb0 2024-04-20T21:18:43.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.627+0000 7fa784c00640 1 -- 172.21.15.86:0/1980413331 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77000aa10 con 0x7fa7880d2cb0 2024-04-20T21:18:43.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.627+0000 7fa78e400640 1 -- 172.21.15.86:0/1980413331 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7880ca380 con 0x7fa7880d2cb0 2024-04-20T21:18:43.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.627+0000 7fa78e400640 1 -- 172.21.15.86:0/1980413331 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7880ca940 con 0x7fa7880d2cb0 2024-04-20T21:18:43.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.628+0000 7fa78e400640 1 -- 172.21.15.86:0/1980413331 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa78804fcf0 con 0x7fa7880d2cb0 2024-04-20T21:18:43.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.629+0000 7fa784c00640 1 -- 172.21.15.86:0/1980413331 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa77001a070 con 0x7fa7880d2cb0 2024-04-20T21:18:43.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.629+0000 7fa784c00640 1 --2- 172.21.15.86:0/1980413331 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa76406c8f0 0x7fa76406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:43.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.630+0000 7fa784c00640 1 -- 172.21.15.86:0/1980413331 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fa77008b2b0 con 0x7fa7880d2cb0 2024-04-20T21:18:43.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.630+0000 7fa78d000640 1 --2- 172.21.15.86:0/1980413331 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa76406c8f0 0x7fa76406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:43.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.630+0000 7fa78d000640 1 --2- 172.21.15.86:0/1980413331 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa76406c8f0 0x7fa76406edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa778036000 tx=0x7fa778030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:43.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.634+0000 7fa784c00640 1 -- 172.21.15.86:0/1980413331 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa7700595c0 con 0x7fa7880d2cb0 2024-04-20T21:18:43.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.845+0000 7fa78e400640 1 -- 172.21.15.86:0/1980413331 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa78806cc00 con 0x7fa7880d2cb0 2024-04-20T21:18:43.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.846+0000 7fa784c00640 1 -- 172.21.15.86:0/1980413331 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7fa770058ec0 con 0x7fa7880d2cb0 2024-04-20T21:18:43.847 INFO:teuthology.orchestra.run.smithi086.stderr:listed 0 entries 2024-04-20T21:18:43.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.848+0000 7fa78e400640 1 -- 172.21.15.86:0/1980413331 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa76406c8f0 msgr2=0x7fa76406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:43.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.848+0000 7fa78e400640 1 --2- 172.21.15.86:0/1980413331 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa76406c8f0 0x7fa76406edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa778036000 tx=0x7fa778030000 comp rx=0 tx=0).stop 2024-04-20T21:18:43.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.848+0000 7fa78e400640 1 -- 172.21.15.86:0/1980413331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7880d2cb0 msgr2=0x7fa788152ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:43.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.848+0000 7fa78e400640 1 --2- 172.21.15.86:0/1980413331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7880d2cb0 0x7fa788152ab0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa77000c7b0 tx=0x7fa77000cc70 comp rx=0 tx=0).stop 2024-04-20T21:18:43.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.849+0000 7fa78e400640 1 -- 172.21.15.86:0/1980413331 shutdown_connections 2024-04-20T21:18:43.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.849+0000 7fa78e400640 1 --2- 172.21.15.86:0/1980413331 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa76406c8f0 0x7fa76406edb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:43.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.849+0000 7fa78e400640 1 --2- 172.21.15.86:0/1980413331 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa78815e690 0x7fa7880c8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:43.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.849+0000 7fa78e400640 1 --2- 172.21.15.86:0/1980413331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7880d2cb0 0x7fa788152ab0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:43.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.849+0000 7fa78e400640 1 --2- 172.21.15.86:0/1980413331 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7880d0ee0 0x7fa7880c99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:43.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.849+0000 7fa78e400640 1 -- 172.21.15.86:0/1980413331 >> 172.21.15.86:0/1980413331 conn(0x7fa78809d8a0 msgr2=0x7fa7881537b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:43.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.849+0000 7fa78e400640 1 -- 172.21.15.86:0/1980413331 shutdown_connections 2024-04-20T21:18:43.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:43.849+0000 7fa78e400640 1 -- 172.21.15.86:0/1980413331 wait complete. 2024-04-20T21:18:43.867 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi144.front.sepia.ceph.com... 2024-04-20T21:18:43.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:43.867 DEBUG:teuthology.orchestra.run.smithi144:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-20T21:18:43.960 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:18:43.960 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2024-04-20T21:18:43.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:43.961 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-20T21:18:44.020 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:18:44.020 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:18:44.076 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:18:44.076 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:18:44.128 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:18:44.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.323+0000 7f8c76a00640 1 Processor -- start 2024-04-20T21:18:44.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.323+0000 7f8c76a00640 1 -- start start 2024-04-20T21:18:44.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.324+0000 7f8c76a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c7015d6a0 0x7f8c700ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:44.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.324+0000 7f8c76a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8c700ad620 0x7f8c7006e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:44.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.324+0000 7f8c76a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c70160890 con 0x7f8c70156ae0 2024-04-20T21:18:44.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.324+0000 7f8c76a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f8c7006db20 con 0x7f8c700ad620 2024-04-20T21:18:44.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.324+0000 7f8c76a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8c70160b70 con 0x7f8c7015d6a0 2024-04-20T21:18:44.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.325+0000 7f8c76000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8c700ad620 0x7f8c7006e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:44.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.325+0000 7f8c74c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c7015d6a0 0x7f8c700ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:44.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.325+0000 7f8c76000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8c700ad620 0x7f8c7006e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.86:51390/0 (socket says 172.21.15.86:51390) 2024-04-20T21:18:44.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.325+0000 7f8c76000640 1 -- 172.21.15.86:0/533610077 learned_addr learned my addr 172.21.15.86:0/533610077 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:44.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.326+0000 7f8c76000640 1 -- 172.21.15.86:0/533610077 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c7015d6a0 msgr2=0x7f8c700ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:44.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.326+0000 7f8c76000640 1 --2- 172.21.15.86:0/533610077 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c7015d6a0 0x7f8c700ad0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:44.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.326+0000 7f8c76000640 1 -- 172.21.15.86:0/533610077 >> v1:172.21.15.144:6789/0 conn(0x7f8c70156ae0 legacy=0x7f8c70160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:44.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.326+0000 7f8c76000640 1 -- 172.21.15.86:0/533610077 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c70160e50 con 0x7f8c700ad620 2024-04-20T21:18:44.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.327+0000 7f8c76000640 1 --2- 172.21.15.86:0/533610077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c700ad620 0x7f8c7006e290 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8c68009e50 tx=0x7f8c6802bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6472470d839c7f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:44.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.328+0000 7f8c6fe00640 1 -- 172.21.15.86:0/533610077 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6802ca70 con 0x7f8c700ad620 2024-04-20T21:18:44.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.328+0000 7f8c6fe00640 1 -- 172.21.15.86:0/533610077 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c6802cc10 con 0x7f8c700ad620 2024-04-20T21:18:44.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.328+0000 7f8c6fe00640 1 -- 172.21.15.86:0/533610077 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c680343f0 con 0x7f8c700ad620 2024-04-20T21:18:44.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.328+0000 7f8c76a00640 1 -- 172.21.15.86:0/533610077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c700ad620 msgr2=0x7f8c7006e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:44.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.328+0000 7f8c76a00640 1 --2- 172.21.15.86:0/533610077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c700ad620 0x7f8c7006e290 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8c68009e50 tx=0x7f8c6802bed0 comp rx=0 tx=0).stop 2024-04-20T21:18:44.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.329+0000 7f8c76a00640 1 -- 172.21.15.86:0/533610077 shutdown_connections 2024-04-20T21:18:44.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.329+0000 7f8c76a00640 1 --2- 172.21.15.86:0/533610077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c700ad620 0x7f8c7006e290 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:44.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.329+0000 7f8c76a00640 1 --2- 172.21.15.86:0/533610077 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c7015d6a0 0x7f8c700ad0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:44.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.329+0000 7f8c76a00640 1 -- 172.21.15.86:0/533610077 >> 172.21.15.86:0/533610077 conn(0x7f8c700d0170 msgr2=0x7f8c700d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:44.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.329+0000 7f8c76a00640 1 -- 172.21.15.86:0/533610077 shutdown_connections 2024-04-20T21:18:44.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.329+0000 7f8c76a00640 1 -- 172.21.15.86:0/533610077 wait complete. 2024-04-20T21:18:44.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.331+0000 7f8c76a00640 1 Processor -- start 2024-04-20T21:18:44.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.331+0000 7f8c76a00640 1 -- start start 2024-04-20T21:18:44.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.332+0000 7f8c76a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c700ad620 0x7f8c700982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:44.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.332+0000 7f8c76a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c70156ae0 0x7f8c70097290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:44.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.332+0000 7f8c76a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c7015d6a0 0x7f8c70145660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:44.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.332+0000 7f8c76a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8c7006e7d0 con 0x7f8c70156ae0 2024-04-20T21:18:44.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.332+0000 7f8c76a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8c70156ec0 con 0x7f8c700ad620 2024-04-20T21:18:44.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.332+0000 7f8c76a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8c7006e950 con 0x7f8c7015d6a0 2024-04-20T21:18:44.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.332+0000 7f8c74c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c70156ae0 0x7f8c70097290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:44.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.332+0000 7f8c74c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c70156ae0 0x7f8c70097290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36124/0 (socket says 172.21.15.86:36124) 2024-04-20T21:18:44.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.332+0000 7f8c74c00640 1 -- 172.21.15.86:0/3548954702 learned_addr learned my addr 172.21.15.86:0/3548954702 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:44.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.333+0000 7f8c75600640 1 --2- 172.21.15.86:0/3548954702 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c700ad620 0x7f8c700982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:44.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.333+0000 7f8c76000640 1 --2- 172.21.15.86:0/3548954702 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c7015d6a0 0x7f8c70145660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:44.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.333+0000 7f8c74c00640 1 -- 172.21.15.86:0/3548954702 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c7015d6a0 msgr2=0x7f8c70145660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:44.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.333+0000 7f8c74c00640 1 --2- 172.21.15.86:0/3548954702 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c7015d6a0 0x7f8c70145660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:44.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.333+0000 7f8c74c00640 1 -- 172.21.15.86:0/3548954702 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c700ad620 msgr2=0x7f8c700982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:44.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.333+0000 7f8c74c00640 1 --2- 172.21.15.86:0/3548954702 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c700ad620 0x7f8c700982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:44.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.333+0000 7f8c74c00640 1 -- 172.21.15.86:0/3548954702 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c70145ba0 con 0x7f8c70156ae0 2024-04-20T21:18:44.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.333+0000 7f8c74c00640 1 --2- 172.21.15.86:0/3548954702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c70156ae0 0x7f8c70097290 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c00e8a0 tx=0x7f8c5c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:44.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.333+0000 7f8c6d600640 1 -- 172.21.15.86:0/3548954702 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c017070 con 0x7f8c70156ae0 2024-04-20T21:18:44.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.333+0000 7f8c6d600640 1 -- 172.21.15.86:0/3548954702 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c5c00c940 con 0x7f8c70156ae0 2024-04-20T21:18:44.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.333+0000 7f8c6d600640 1 -- 172.21.15.86:0/3548954702 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c00cd30 con 0x7f8c70156ae0 2024-04-20T21:18:44.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.333+0000 7f8c76a00640 1 -- 172.21.15.86:0/3548954702 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c70098930 con 0x7f8c70156ae0 2024-04-20T21:18:44.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.334+0000 7f8c76a00640 1 -- 172.21.15.86:0/3548954702 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c7015df30 con 0x7f8c70156ae0 2024-04-20T21:18:44.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.335+0000 7f8c76a00640 1 -- 172.21.15.86:0/3548954702 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c7004fcf0 con 0x7f8c70156ae0 2024-04-20T21:18:44.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.336+0000 7f8c6d600640 1 -- 172.21.15.86:0/3548954702 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8c5c01a070 con 0x7f8c70156ae0 2024-04-20T21:18:44.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.337+0000 7f8c6d600640 1 --2- 172.21.15.86:0/3548954702 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8c5006c920 0x7f8c5006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:44.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.337+0000 7f8c6d600640 1 -- 172.21.15.86:0/3548954702 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f8c5c08be40 con 0x7f8c70156ae0 2024-04-20T21:18:44.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.337+0000 7f8c75600640 1 --2- 172.21.15.86:0/3548954702 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8c5006c920 0x7f8c5006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:44.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.338+0000 7f8c75600640 1 --2- 172.21.15.86:0/3548954702 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8c5006c920 0x7f8c5006ede0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8c7015b690 tx=0x7f8c64009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:44.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.341+0000 7f8c6d600640 1 -- 172.21.15.86:0/3548954702 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8c5c05a1e0 con 0x7f8c70156ae0 2024-04-20T21:18:44.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.527+0000 7f8c76a00640 1 -- 172.21.15.86:0/3548954702 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8c700b1090 con 0x7f8c70156ae0 2024-04-20T21:18:44.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.529+0000 7f8c6d600640 1 -- 172.21.15.86:0/3548954702 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16854 (secure 0 0 0) 0x7f8c5c059ae0 con 0x7f8c70156ae0 2024-04-20T21:18:44.530 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:18:44.530 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":22,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:18:33.788155+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:17:49.718799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:17:50.683477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:18:44.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.532+0000 7f8c76a00640 1 -- 172.21.15.86:0/3548954702 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8c5006c920 msgr2=0x7f8c5006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:44.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.532+0000 7f8c76a00640 1 --2- 172.21.15.86:0/3548954702 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8c5006c920 0x7f8c5006ede0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8c7015b690 tx=0x7f8c64009040 comp rx=0 tx=0).stop 2024-04-20T21:18:44.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.532+0000 7f8c76a00640 1 -- 172.21.15.86:0/3548954702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c70156ae0 msgr2=0x7f8c70097290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:44.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.532+0000 7f8c76a00640 1 --2- 172.21.15.86:0/3548954702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c70156ae0 0x7f8c70097290 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c00e8a0 tx=0x7f8c5c00eca0 comp rx=0 tx=0).stop 2024-04-20T21:18:44.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.532+0000 7f8c76a00640 1 -- 172.21.15.86:0/3548954702 shutdown_connections 2024-04-20T21:18:44.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.532+0000 7f8c76a00640 1 --2- 172.21.15.86:0/3548954702 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8c5006c920 0x7f8c5006ede0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:44.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.532+0000 7f8c76a00640 1 --2- 172.21.15.86:0/3548954702 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c7015d6a0 0x7f8c70145660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:44.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.532+0000 7f8c76a00640 1 --2- 172.21.15.86:0/3548954702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c70156ae0 0x7f8c70097290 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:44.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.532+0000 7f8c76a00640 1 --2- 172.21.15.86:0/3548954702 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c700ad620 0x7f8c700982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:44.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.532+0000 7f8c76a00640 1 -- 172.21.15.86:0/3548954702 >> 172.21.15.86:0/3548954702 conn(0x7f8c700d0170 msgr2=0x7f8c700b3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:44.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.533+0000 7f8c76a00640 1 -- 172.21.15.86:0/3548954702 shutdown_connections 2024-04-20T21:18:44.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.533+0000 7f8c76a00640 1 -- 172.21.15.86:0/3548954702 wait complete. 2024-04-20T21:18:44.550 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:17:49.718799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:17:50.683477+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:18:44.551 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:18:44.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.751+0000 7f4eb7400640 1 Processor -- start 2024-04-20T21:18:44.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.751+0000 7f4eb7400640 1 -- start start 2024-04-20T21:18:44.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.753+0000 7f4eb7400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4eb006bbf0 0x7f4eb006e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:44.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.753+0000 7f4eb7400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4eb0146210 0x7f4eb00cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:44.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.753+0000 7f4eb7400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4eb015cd40 con 0x7f4eb006bfd0 2024-04-20T21:18:44.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.753+0000 7f4eb7400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f4eb015d000 con 0x7f4eb0146210 2024-04-20T21:18:44.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.753+0000 7f4eb7400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f4eb015d2e0 con 0x7f4eb006bbf0 2024-04-20T21:18:44.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.753+0000 7f4ea7e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f4eb006bfd0 0x7f4eb0145b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41836/0 (socket says 172.21.15.86:41836) 2024-04-20T21:18:44.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.753+0000 7f4ea7e00640 1 -- 172.21.15.86:0/885610124 learned_addr learned my addr 172.21.15.86:0/885610124 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:18:44.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.753+0000 7f4eb5400640 1 --2- 172.21.15.86:0/885610124 >> v2:172.21.15.144:3300/0 conn(0x7f4eb0146210 0x7f4eb00cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:44.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.754+0000 7f4eb4a00640 1 --2- 172.21.15.86:0/885610124 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4eb006bbf0 0x7f4eb006e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:44.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.754+0000 7f4eafe00640 1 -- 172.21.15.86:0/885610124 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 960020779 0 0) 0x7f4eb015cd40 con 0x7f4eb006bfd0 2024-04-20T21:18:44.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.754+0000 7f4eafe00640 1 -- 172.21.15.86:0/885610124 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e90003520 con 0x7f4eb006bfd0 2024-04-20T21:18:44.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.754+0000 7f4eafe00640 1 -- 172.21.15.86:0/885610124 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 144859196 0 0) 0x7f4e90003520 con 0x7f4eb006bfd0 2024-04-20T21:18:44.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.754+0000 7f4eafe00640 1 -- 172.21.15.86:0/885610124 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4eb006bbf0 msgr2=0x7f4eb006e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:44.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.754+0000 7f4eafe00640 1 --2- 172.21.15.86:0/885610124 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4eb006bbf0 0x7f4eb006e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:44.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.754+0000 7f4eafe00640 1 -- 172.21.15.86:0/885610124 >> v2:172.21.15.144:3300/0 conn(0x7f4eb0146210 msgr2=0x7f4eb00cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:44.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.754+0000 7f4eafe00640 1 --2- 172.21.15.86:0/885610124 >> v2:172.21.15.144:3300/0 conn(0x7f4eb0146210 0x7f4eb00cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:44.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.754+0000 7f4eafe00640 1 -- 172.21.15.86:0/885610124 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb015d5c0 con 0x7f4eb006bfd0 2024-04-20T21:18:44.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.754+0000 7f4eafe00640 1 -- 172.21.15.86:0/885610124 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f4ea0001670 con 0x7f4eb006bfd0 2024-04-20T21:18:44.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.754+0000 7f4eafe00640 1 -- 172.21.15.86:0/885610124 >> v1:172.21.15.86:6789/0 conn(0x7f4eb006bfd0 legacy=0x7f4eb0145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:44.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.754+0000 7f4eafe00640 1 --2- 172.21.15.86:0/885610124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e900039b0 0x7f4e90003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:44.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.754+0000 7f4eafe00640 1 --2- 172.21.15.86:0/885610124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e90004800 0x7f4e90004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:44.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.754+0000 7f4eafe00640 1 --2- 172.21.15.86:0/885610124 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4eb006bbf0 0x7f4e900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:44.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.754+0000 7f4eafe00640 1 -- 172.21.15.86:0/885610124 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb015d000 con 0x7f4e90004800 2024-04-20T21:18:44.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.754+0000 7f4eafe00640 1 -- 172.21.15.86:0/885610124 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb015d2e0 con 0x7f4e900039b0 2024-04-20T21:18:44.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.754+0000 7f4eafe00640 1 -- 172.21.15.86:0/885610124 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f4e9000b9c0 con 0x7f4eb006bbf0 2024-04-20T21:18:44.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.755+0000 7f4ea7e00640 1 --2- 172.21.15.86:0/885610124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e90004800 0x7f4e90004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:44.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.755+0000 7f4eb4a00640 1 --2- 172.21.15.86:0/885610124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e900039b0 0x7f4e90003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:44.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.755+0000 7f4ea7e00640 1 -- 172.21.15.86:0/885610124 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4eb006bbf0 msgr2=0x7f4e900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:44.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.755+0000 7f4ea7e00640 1 --2- 172.21.15.86:0/885610124 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4eb006bbf0 0x7f4e900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:44.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.755+0000 7f4ea7e00640 1 -- 172.21.15.86:0/885610124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e900039b0 msgr2=0x7f4e90003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:44.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.755+0000 7f4ea7e00640 1 --2- 172.21.15.86:0/885610124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e900039b0 0x7f4e90003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:44.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.755+0000 7f4ea7e00640 1 -- 172.21.15.86:0/885610124 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e90003520 con 0x7f4e90004800 2024-04-20T21:18:44.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.756+0000 7f4ea7e00640 1 --2- 172.21.15.86:0/885610124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e90004800 0x7f4e90004c80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4ea000c240 tx=0x7f4e90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce7a517d47122b10 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:44.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.756+0000 7f4eafe00640 1 -- 172.21.15.86:0/885610124 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea0007d10 con 0x7f4e90004800 2024-04-20T21:18:44.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.756+0000 7f4eafe00640 1 -- 172.21.15.86:0/885610124 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ea0030070 con 0x7f4e90004800 2024-04-20T21:18:44.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.756+0000 7f4eafe00640 1 -- 172.21.15.86:0/885610124 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea002cd60 con 0x7f4e90004800 2024-04-20T21:18:44.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.757+0000 7f4eb7400640 1 -- 172.21.15.86:0/885610124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e90004800 msgr2=0x7f4e90004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:44.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.757+0000 7f4eb7400640 1 --2- 172.21.15.86:0/885610124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e90004800 0x7f4e90004c80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4ea000c240 tx=0x7f4e90005830 comp rx=0 tx=0).stop 2024-04-20T21:18:44.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.757+0000 7f4eb5400640 1 -- 172.21.15.86:0/885610124 reap_dead start 2024-04-20T21:18:44.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.757+0000 7f4eb7400640 1 -- 172.21.15.86:0/885610124 shutdown_connections 2024-04-20T21:18:44.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.757+0000 7f4eb7400640 1 -- 172.21.15.86:0/885610124 >> 172.21.15.86:0/885610124 conn(0x7f4eb00b2110 msgr2=0x7f4eb00b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:44.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.757+0000 7f4eb7400640 1 -- 172.21.15.86:0/885610124 shutdown_connections 2024-04-20T21:18:44.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.757+0000 7f4eb7400640 1 -- 172.21.15.86:0/885610124 wait complete. 2024-04-20T21:18:44.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.759+0000 7f4eb7400640 1 Processor -- start 2024-04-20T21:18:44.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.759+0000 7f4eb7400640 1 -- start start 2024-04-20T21:18:44.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.760+0000 7f4eb7400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e90004800 0x7f4eb01e7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:44.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.760+0000 7f4eb7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4eb01e82d0 0x7f4eb0132fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:44.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.760+0000 7f4eb7400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4eb01334e0 0x7f4eb0133960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:44.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.760+0000 7f4eb7400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb006e180 con 0x7f4eb01e82d0 2024-04-20T21:18:44.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.760+0000 7f4eb7400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb006e000 con 0x7f4e90004800 2024-04-20T21:18:44.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.760+0000 7f4eb7400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f4eb006e300 con 0x7f4eb01334e0 2024-04-20T21:18:44.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.760+0000 7f4ea7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4eb01e82d0 0x7f4eb0132fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:44.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.760+0000 7f4ea7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4eb01e82d0 0x7f4eb0132fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36136/0 (socket says 172.21.15.86:36136) 2024-04-20T21:18:44.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.760+0000 7f4ea7e00640 1 -- 172.21.15.86:0/2709597241 learned_addr learned my addr 172.21.15.86:0/2709597241 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:44.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.761+0000 7f4eb4a00640 1 --2- 172.21.15.86:0/2709597241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e90004800 0x7f4eb01e7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:44.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.761+0000 7f4ea7e00640 1 -- 172.21.15.86:0/2709597241 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4eb01334e0 msgr2=0x7f4eb0133960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:44.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.761+0000 7f4ea7e00640 1 --2- 172.21.15.86:0/2709597241 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4eb01334e0 0x7f4eb0133960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:44.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.761+0000 7f4ea7e00640 1 -- 172.21.15.86:0/2709597241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e90004800 msgr2=0x7f4eb01e7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:44.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.761+0000 7f4ea7e00640 1 --2- 172.21.15.86:0/2709597241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e90004800 0x7f4eb01e7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:44.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.761+0000 7f4ea7e00640 1 -- 172.21.15.86:0/2709597241 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb00cc650 con 0x7f4eb01e82d0 2024-04-20T21:18:44.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.761+0000 7f4eb4a00640 1 --2- 172.21.15.86:0/2709597241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e90004800 0x7f4eb01e7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:44.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.761+0000 7f4ea7e00640 1 --2- 172.21.15.86:0/2709597241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4eb01e82d0 0x7f4eb0132fa0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0001970 tx=0x7f4eb015d790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:44.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.761+0000 7f4ead600640 1 -- 172.21.15.86:0/2709597241 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea0007d10 con 0x7f4eb01e82d0 2024-04-20T21:18:44.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.761+0000 7f4ead600640 1 -- 172.21.15.86:0/2709597241 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ea0004bf0 con 0x7f4eb01e82d0 2024-04-20T21:18:44.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.761+0000 7f4ead600640 1 -- 172.21.15.86:0/2709597241 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea0007070 con 0x7f4eb01e82d0 2024-04-20T21:18:44.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.761+0000 7f4eb7400640 1 -- 172.21.15.86:0/2709597241 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eb0130f40 con 0x7f4eb01e82d0 2024-04-20T21:18:44.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.761+0000 7f4eb7400640 1 -- 172.21.15.86:0/2709597241 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eb0131500 con 0x7f4eb01e82d0 2024-04-20T21:18:44.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.763+0000 7f4eb7400640 1 -- 172.21.15.86:0/2709597241 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4eb00689e0 con 0x7f4eb01e82d0 2024-04-20T21:18:44.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.764+0000 7f4ead600640 1 -- 172.21.15.86:0/2709597241 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4ea0007210 con 0x7f4eb01e82d0 2024-04-20T21:18:44.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.764+0000 7f4ead600640 1 --2- 172.21.15.86:0/2709597241 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4e8c06c920 0x7f4e8c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:44.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.765+0000 7f4ead600640 1 -- 172.21.15.86:0/2709597241 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f4ea00ad330 con 0x7f4eb01e82d0 2024-04-20T21:18:44.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.765+0000 7f4eb4a00640 1 --2- 172.21.15.86:0/2709597241 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4e8c06c920 0x7f4e8c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:44.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.766+0000 7f4eb4a00640 1 --2- 172.21.15.86:0/2709597241 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4e8c06c920 0x7f4e8c06ede0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4e980098b0 tx=0x7f4e98008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:44.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.769+0000 7f4ead600640 1 -- 172.21.15.86:0/2709597241 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4ea007b640 con 0x7f4eb01e82d0 2024-04-20T21:18:44.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.968+0000 7f4eb7400640 1 -- 172.21.15.86:0/2709597241 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4eb015eaa0 con 0x7f4eb01e82d0 2024-04-20T21:18:44.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.969+0000 7f4ead600640 1 -- 172.21.15.86:0/2709597241 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f4ea007af40 con 0x7f4eb01e82d0 2024-04-20T21:18:44.970 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:18:44.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.972+0000 7f4eb7400640 1 -- 172.21.15.86:0/2709597241 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4e8c06c920 msgr2=0x7f4e8c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:44.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.972+0000 7f4eb7400640 1 --2- 172.21.15.86:0/2709597241 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4e8c06c920 0x7f4e8c06ede0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4e980098b0 tx=0x7f4e98008040 comp rx=0 tx=0).stop 2024-04-20T21:18:44.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.972+0000 7f4eb7400640 1 -- 172.21.15.86:0/2709597241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4eb01e82d0 msgr2=0x7f4eb0132fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:44.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.972+0000 7f4eb7400640 1 --2- 172.21.15.86:0/2709597241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4eb01e82d0 0x7f4eb0132fa0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0001970 tx=0x7f4eb015d790 comp rx=0 tx=0).stop 2024-04-20T21:18:44.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.972+0000 7f4eb7400640 1 -- 172.21.15.86:0/2709597241 shutdown_connections 2024-04-20T21:18:44.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.972+0000 7f4eb7400640 1 --2- 172.21.15.86:0/2709597241 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4e8c06c920 0x7f4e8c06ede0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:44.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.972+0000 7f4eb7400640 1 --2- 172.21.15.86:0/2709597241 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4eb01334e0 0x7f4eb0133960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:44.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.972+0000 7f4eb7400640 1 --2- 172.21.15.86:0/2709597241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4eb01e82d0 0x7f4eb0132fa0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:44.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.972+0000 7f4eb7400640 1 --2- 172.21.15.86:0/2709597241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e90004800 0x7f4eb01e7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:44.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.972+0000 7f4eb7400640 1 -- 172.21.15.86:0/2709597241 >> 172.21.15.86:0/2709597241 conn(0x7f4eb00b2110 msgr2=0x7f4eb006f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:44.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.972+0000 7f4eb7400640 1 -- 172.21.15.86:0/2709597241 shutdown_connections 2024-04-20T21:18:44.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:44.972+0000 7f4eb7400640 1 -- 172.21.15.86:0/2709597241 wait complete. 2024-04-20T21:18:44.991 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:18:45.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.187+0000 7f7f70c00640 1 Processor -- start 2024-04-20T21:18:45.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.188+0000 7f7f70c00640 1 -- start start 2024-04-20T21:18:45.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.189+0000 7f7f70c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7f6c0d45a0 0x7f7f6c0d3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:45.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.189+0000 7f7f70c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7f6c15ea70 0x7f7f6c0d4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:45.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.189+0000 7f7f70c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7f6c06d570 0x7f7f6c0d1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:45.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.189+0000 7f7f70c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f7f6c0cf120 con 0x7f7f6c0d45a0 2024-04-20T21:18:45.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.189+0000 7f7f70c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f7f6c06de70 con 0x7f7f6c15ea70 2024-04-20T21:18:45.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.189+0000 7f7f70c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7f6c06e150 con 0x7f7f6c06d570 2024-04-20T21:18:45.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.190+0000 7f7f6b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7f6c0d45a0 0x7f7f6c0d3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:45.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.190+0000 7f7f6b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7f6c0d45a0 0x7f7f6c0d3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36152/0 (socket says 172.21.15.86:36152) 2024-04-20T21:18:45.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.190+0000 7f7f6b400640 1 -- 172.21.15.86:0/594376389 learned_addr learned my addr 172.21.15.86:0/594376389 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:45.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.190+0000 7f7f6b400640 1 -- 172.21.15.86:0/594376389 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7f6c06d570 msgr2=0x7f7f6c0d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:45.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.190+0000 7f7f6b400640 1 --2- 172.21.15.86:0/594376389 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7f6c06d570 0x7f7f6c0d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:45.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.190+0000 7f7f6b400640 1 -- 172.21.15.86:0/594376389 >> v2:172.21.15.144:3300/0 conn(0x7f7f6c15ea70 msgr2=0x7f7f6c0d4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:45.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.190+0000 7f7f6b400640 1 --2- 172.21.15.86:0/594376389 >> v2:172.21.15.144:3300/0 conn(0x7f7f6c15ea70 0x7f7f6c0d4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:45.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.190+0000 7f7f6b400640 1 -- 172.21.15.86:0/594376389 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f6c06e430 con 0x7f7f6c0d45a0 2024-04-20T21:18:45.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.191+0000 7f7f6b400640 1 --2- 172.21.15.86:0/594376389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f6c0d45a0 0x7f7f6c0d3260 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c0b4120 tx=0x7f7f5c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a122b5935716be5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:45.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.192+0000 7f7f6a000640 1 -- 172.21.15.86:0/594376389 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c02e070 con 0x7f7f6c0d45a0 2024-04-20T21:18:45.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.192+0000 7f7f6a000640 1 -- 172.21.15.86:0/594376389 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f5c037070 con 0x7f7f6c0d45a0 2024-04-20T21:18:45.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.192+0000 7f7f6a000640 1 -- 172.21.15.86:0/594376389 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c031a00 con 0x7f7f6c0d45a0 2024-04-20T21:18:45.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.192+0000 7f7f70c00640 1 -- 172.21.15.86:0/594376389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f6c0d45a0 msgr2=0x7f7f6c0d3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:45.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.192+0000 7f7f70c00640 1 --2- 172.21.15.86:0/594376389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f6c0d45a0 0x7f7f6c0d3260 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c0b4120 tx=0x7f7f5c02b880 comp rx=0 tx=0).stop 2024-04-20T21:18:45.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.193+0000 7f7f70c00640 1 -- 172.21.15.86:0/594376389 shutdown_connections 2024-04-20T21:18:45.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.193+0000 7f7f70c00640 1 --2- 172.21.15.86:0/594376389 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7f6c06d570 0x7f7f6c0d1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:45.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.193+0000 7f7f70c00640 1 --2- 172.21.15.86:0/594376389 >> v2:172.21.15.144:3300/0 conn(0x7f7f6c15ea70 0x7f7f6c0d4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:45.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.193+0000 7f7f70c00640 1 --2- 172.21.15.86:0/594376389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f6c0d45a0 0x7f7f6c0d3260 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:45.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.193+0000 7f7f70c00640 1 -- 172.21.15.86:0/594376389 >> 172.21.15.86:0/594376389 conn(0x7f7f6c09d9d0 msgr2=0x7f7f6c08bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:45.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.193+0000 7f7f70c00640 1 -- 172.21.15.86:0/594376389 shutdown_connections 2024-04-20T21:18:45.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.193+0000 7f7f70c00640 1 -- 172.21.15.86:0/594376389 wait complete. 2024-04-20T21:18:45.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.194+0000 7f7f70c00640 1 Processor -- start 2024-04-20T21:18:45.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.195+0000 7f7f70c00640 1 -- start start 2024-04-20T21:18:45.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.196+0000 7f7f70c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7f6c06d570 0x7f7f6c160420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:45.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.196+0000 7f7f70c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f6c0d45a0 0x7f7f6c0c9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:45.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.196+0000 7f7f70c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f6c15ea70 0x7f7f6c0ca730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:45.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.196+0000 7f7f70c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c06d9e0 con 0x7f7f6c15ea70 2024-04-20T21:18:45.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.196+0000 7f7f70c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c0d4ec0 con 0x7f7f6c0d45a0 2024-04-20T21:18:45.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.196+0000 7f7f70c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7f6c06db60 con 0x7f7f6c06d570 2024-04-20T21:18:45.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.196+0000 7f7f6be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f6c15ea70 0x7f7f6c0ca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:45.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.196+0000 7f7f6be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f6c15ea70 0x7f7f6c0ca730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36162/0 (socket says 172.21.15.86:36162) 2024-04-20T21:18:45.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.196+0000 7f7f6be00640 1 -- 172.21.15.86:0/2442631210 learned_addr learned my addr 172.21.15.86:0/2442631210 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:45.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.196+0000 7f7f6be00640 1 -- 172.21.15.86:0/2442631210 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7f6c06d570 msgr2=0x7f7f6c160420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:45.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.196+0000 7f7f6be00640 1 --2- 172.21.15.86:0/2442631210 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7f6c06d570 0x7f7f6c160420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:45.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.196+0000 7f7f6be00640 1 -- 172.21.15.86:0/2442631210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f6c0d45a0 msgr2=0x7f7f6c0c9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:45.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.196+0000 7f7f6aa00640 1 --2- 172.21.15.86:0/2442631210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f6c0d45a0 0x7f7f6c0c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:45.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.196+0000 7f7f6be00640 1 --2- 172.21.15.86:0/2442631210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f6c0d45a0 0x7f7f6c0c9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:45.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.197+0000 7f7f6be00640 1 -- 172.21.15.86:0/2442631210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f6c0c8350 con 0x7f7f6c15ea70 2024-04-20T21:18:45.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.197+0000 7f7f6aa00640 1 --2- 172.21.15.86:0/2442631210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f6c0d45a0 0x7f7f6c0c9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:45.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.197+0000 7f7f6be00640 1 --2- 172.21.15.86:0/2442631210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f6c15ea70 0x7f7f6c0ca730 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7f6400b970 tx=0x7f7f6400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:45.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.197+0000 7f7f63400640 1 -- 172.21.15.86:0/2442631210 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f64019070 con 0x7f7f6c15ea70 2024-04-20T21:18:45.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.197+0000 7f7f63400640 1 -- 172.21.15.86:0/2442631210 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f6400cb60 con 0x7f7f6c15ea70 2024-04-20T21:18:45.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.197+0000 7f7f70c00640 1 -- 172.21.15.86:0/2442631210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f6c0c8630 con 0x7f7f6c15ea70 2024-04-20T21:18:45.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.197+0000 7f7f63400640 1 -- 172.21.15.86:0/2442631210 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f64014440 con 0x7f7f6c15ea70 2024-04-20T21:18:45.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.197+0000 7f7f70c00640 1 -- 172.21.15.86:0/2442631210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f6c161b00 con 0x7f7f6c15ea70 2024-04-20T21:18:45.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.199+0000 7f7f70c00640 1 -- 172.21.15.86:0/2442631210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f6c04fd90 con 0x7f7f6c15ea70 2024-04-20T21:18:45.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.199+0000 7f7f63400640 1 -- 172.21.15.86:0/2442631210 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7f6401c070 con 0x7f7f6c15ea70 2024-04-20T21:18:45.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.200+0000 7f7f63400640 1 --2- 172.21.15.86:0/2442631210 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7f4806c920 0x7f7f4806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:45.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.200+0000 7f7f63400640 1 -- 172.21.15.86:0/2442631210 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f7f6408b830 con 0x7f7f6c15ea70 2024-04-20T21:18:45.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.201+0000 7f7f6b400640 1 --2- 172.21.15.86:0/2442631210 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7f4806c920 0x7f7f4806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:45.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.201+0000 7f7f6b400640 1 --2- 172.21.15.86:0/2442631210 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7f4806c920 0x7f7f4806ede0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c036000 tx=0x7f7f5c030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:45.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.205+0000 7f7f63400640 1 -- 172.21.15.86:0/2442631210 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7f64059c70 con 0x7f7f6c15ea70 2024-04-20T21:18:45.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.395+0000 7f7f70c00640 1 -- 172.21.15.86:0/2442631210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7f6c0ca510 con 0x7f7f6c15ea70 2024-04-20T21:18:45.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.396+0000 7f7f63400640 1 -- 172.21.15.86:0/2442631210 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f7f64059570 con 0x7f7f6c15ea70 2024-04-20T21:18:45.397 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-20T21:18:45.399 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:18:45.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.399+0000 7f7f70c00640 1 -- 172.21.15.86:0/2442631210 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7f4806c920 msgr2=0x7f7f4806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:45.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.399+0000 7f7f70c00640 1 --2- 172.21.15.86:0/2442631210 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7f4806c920 0x7f7f4806ede0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c036000 tx=0x7f7f5c030000 comp rx=0 tx=0).stop 2024-04-20T21:18:45.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.399+0000 7f7f70c00640 1 -- 172.21.15.86:0/2442631210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f6c15ea70 msgr2=0x7f7f6c0ca730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:45.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.399+0000 7f7f70c00640 1 --2- 172.21.15.86:0/2442631210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f6c15ea70 0x7f7f6c0ca730 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7f6400b970 tx=0x7f7f6400be30 comp rx=0 tx=0).stop 2024-04-20T21:18:45.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.399+0000 7f7f70c00640 1 -- 172.21.15.86:0/2442631210 shutdown_connections 2024-04-20T21:18:45.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.399+0000 7f7f70c00640 1 --2- 172.21.15.86:0/2442631210 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7f4806c920 0x7f7f4806ede0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:45.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.399+0000 7f7f70c00640 1 --2- 172.21.15.86:0/2442631210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f6c15ea70 0x7f7f6c0ca730 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:45.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.399+0000 7f7f70c00640 1 --2- 172.21.15.86:0/2442631210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f6c0d45a0 0x7f7f6c0c9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:45.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.399+0000 7f7f70c00640 1 --2- 172.21.15.86:0/2442631210 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7f6c06d570 0x7f7f6c160420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:45.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.399+0000 7f7f70c00640 1 -- 172.21.15.86:0/2442631210 >> 172.21.15.86:0/2442631210 conn(0x7f7f6c09d9d0 msgr2=0x7f7f6c0b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:45.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.399+0000 7f7f70c00640 1 -- 172.21.15.86:0/2442631210 shutdown_connections 2024-04-20T21:18:45.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.399+0000 7f7f70c00640 1 -- 172.21.15.86:0/2442631210 wait complete. 2024-04-20T21:18:45.405 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:18:45.418 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:18:45.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.647+0000 7f1f85e00640 1 Processor -- start 2024-04-20T21:18:45.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.648+0000 7f1f85e00640 1 -- start start 2024-04-20T21:18:45.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.649+0000 7f1f85e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f800d4310 0x7f1f800d4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:45.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.649+0000 7f1f85e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1f800cfb80 0x7f1f800d0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:45.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.649+0000 7f1f85e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f800d4cb0 con 0x7f1f8015e870 2024-04-20T21:18:45.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.649+0000 7f1f85e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f1f8006d590 con 0x7f1f800cfb80 2024-04-20T21:18:45.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.649+0000 7f1f85e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1f800cf150 con 0x7f1f800d4310 2024-04-20T21:18:45.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.650+0000 7f1f85400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1f800cfb80 0x7f1f800d0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:45.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.650+0000 7f1f7fe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f800d4310 0x7f1f800d4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:45.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.650+0000 7f1f85400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1f800cfb80 0x7f1f800d0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.86:51446/0 (socket says 172.21.15.86:51446) 2024-04-20T21:18:45.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.650+0000 7f1f85400640 1 -- 172.21.15.86:0/276610143 learned_addr learned my addr 172.21.15.86:0/276610143 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:45.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.651+0000 7f1f7fe00640 1 -- 172.21.15.86:0/276610143 >> v2:172.21.15.144:3300/0 conn(0x7f1f800cfb80 msgr2=0x7f1f800d0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:45.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.651+0000 7f1f7fe00640 1 --2- 172.21.15.86:0/276610143 >> v2:172.21.15.144:3300/0 conn(0x7f1f800cfb80 0x7f1f800d0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:45.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.651+0000 7f1f7fe00640 1 -- 172.21.15.86:0/276610143 >> v1:172.21.15.144:6789/0 conn(0x7f1f8015e870 legacy=0x7f1f8006ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:45.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.651+0000 7f1f7fe00640 1 -- 172.21.15.86:0/276610143 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f800d2ff0 con 0x7f1f800d4310 2024-04-20T21:18:45.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.651+0000 7f1f85400640 1 --2- 172.21.15.86:0/276610143 >> v2:172.21.15.144:3300/0 conn(0x7f1f800cfb80 0x7f1f800d0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:18:45.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.652+0000 7f1f7fe00640 1 --2- 172.21.15.86:0/276610143 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f800d4310 0x7f1f800d4770 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1f70009d90 tx=0x7f1f7002bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2fc67c08610d7a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:45.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.652+0000 7f1f7f400640 1 -- 172.21.15.86:0/276610143 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f70039070 con 0x7f1f800d4310 2024-04-20T21:18:45.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.653+0000 7f1f7f400640 1 -- 172.21.15.86:0/276610143 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f7002cc50 con 0x7f1f800d4310 2024-04-20T21:18:45.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.653+0000 7f1f7f400640 1 -- 172.21.15.86:0/276610143 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f70034440 con 0x7f1f800d4310 2024-04-20T21:18:45.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.653+0000 7f1f85e00640 1 -- 172.21.15.86:0/276610143 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f800d4310 msgr2=0x7f1f800d4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:45.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.653+0000 7f1f85e00640 1 --2- 172.21.15.86:0/276610143 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f800d4310 0x7f1f800d4770 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1f70009d90 tx=0x7f1f7002bea0 comp rx=0 tx=0).stop 2024-04-20T21:18:45.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.653+0000 7f1f85e00640 1 -- 172.21.15.86:0/276610143 shutdown_connections 2024-04-20T21:18:45.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.653+0000 7f1f85e00640 1 --2- 172.21.15.86:0/276610143 >> v2:172.21.15.144:3300/0 conn(0x7f1f800cfb80 0x7f1f800d0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:45.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.653+0000 7f1f85e00640 1 --2- 172.21.15.86:0/276610143 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f800d4310 0x7f1f800d4770 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:45.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.653+0000 7f1f85e00640 1 -- 172.21.15.86:0/276610143 >> 172.21.15.86:0/276610143 conn(0x7f1f8009d9c0 msgr2=0x7f1f8008bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:45.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.653+0000 7f1f85e00640 1 -- 172.21.15.86:0/276610143 shutdown_connections 2024-04-20T21:18:45.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.653+0000 7f1f85e00640 1 -- 172.21.15.86:0/276610143 wait complete. 2024-04-20T21:18:45.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.655+0000 7f1f85e00640 1 Processor -- start 2024-04-20T21:18:45.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.655+0000 7f1f85e00640 1 -- start start 2024-04-20T21:18:45.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.656+0000 7f1f85e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f800cfb80 0x7f1f80155a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:45.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.656+0000 7f1f85e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f800d4310 0x7f1f800b6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:45.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.656+0000 7f1f85e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f8015e870 0x7f1f8015f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:45.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.656+0000 7f1f85e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1f800cf380 con 0x7f1f8015e870 2024-04-20T21:18:45.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.656+0000 7f1f85e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1f800d35b0 con 0x7f1f800d4310 2024-04-20T21:18:45.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.656+0000 7f1f85e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1f800cf660 con 0x7f1f800cfb80 2024-04-20T21:18:45.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.656+0000 7f1f85400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f8015e870 0x7f1f8015f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:45.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.656+0000 7f1f85400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f8015e870 0x7f1f8015f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36178/0 (socket says 172.21.15.86:36178) 2024-04-20T21:18:45.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.656+0000 7f1f85400640 1 -- 172.21.15.86:0/3153201559 learned_addr learned my addr 172.21.15.86:0/3153201559 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:45.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.657+0000 7f1f84a00640 1 --2- 172.21.15.86:0/3153201559 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f800cfb80 0x7f1f80155a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:45.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.657+0000 7f1f7fe00640 1 --2- 172.21.15.86:0/3153201559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f800d4310 0x7f1f800b6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:45.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.657+0000 7f1f85400640 1 -- 172.21.15.86:0/3153201559 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f800cfb80 msgr2=0x7f1f80155a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:45.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.657+0000 7f1f85400640 1 --2- 172.21.15.86:0/3153201559 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f800cfb80 0x7f1f80155a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:45.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.657+0000 7f1f85400640 1 -- 172.21.15.86:0/3153201559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f800d4310 msgr2=0x7f1f800b6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:45.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.657+0000 7f1f85400640 1 --2- 172.21.15.86:0/3153201559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f800d4310 0x7f1f800b6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:45.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.657+0000 7f1f85400640 1 -- 172.21.15.86:0/3153201559 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f800b53e0 con 0x7f1f8015e870 2024-04-20T21:18:45.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.657+0000 7f1f85400640 1 --2- 172.21.15.86:0/3153201559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f8015e870 0x7f1f8015f660 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1f7800e930 tx=0x7f1f7800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:45.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.657+0000 7f1f7cc00640 1 -- 172.21.15.86:0/3153201559 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f78017070 con 0x7f1f8015e870 2024-04-20T21:18:45.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.658+0000 7f1f7cc00640 1 -- 172.21.15.86:0/3153201559 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f7800ca00 con 0x7f1f8015e870 2024-04-20T21:18:45.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.658+0000 7f1f7cc00640 1 -- 172.21.15.86:0/3153201559 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7800cdd0 con 0x7f1f8015e870 2024-04-20T21:18:45.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.658+0000 7f1f85e00640 1 -- 172.21.15.86:0/3153201559 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f800b56c0 con 0x7f1f8015e870 2024-04-20T21:18:45.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.658+0000 7f1f85e00640 1 -- 172.21.15.86:0/3153201559 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f800b5c80 con 0x7f1f8015e870 2024-04-20T21:18:45.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.659+0000 7f1f85e00640 1 -- 172.21.15.86:0/3153201559 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f8004fd90 con 0x7f1f8015e870 2024-04-20T21:18:45.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.660+0000 7f1f7cc00640 1 -- 172.21.15.86:0/3153201559 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1f78013470 con 0x7f1f8015e870 2024-04-20T21:18:45.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.661+0000 7f1f7cc00640 1 --2- 172.21.15.86:0/3153201559 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1f5c06c8f0 0x7f1f5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:45.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.661+0000 7f1f7cc00640 1 -- 172.21.15.86:0/3153201559 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f1f7808bb00 con 0x7f1f8015e870 2024-04-20T21:18:45.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.661+0000 7f1f84a00640 1 --2- 172.21.15.86:0/3153201559 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1f5c06c8f0 0x7f1f5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:45.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.661+0000 7f1f84a00640 1 --2- 172.21.15.86:0/3153201559 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1f5c06c8f0 0x7f1f5c06edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1f68007fd0 tx=0x7f1f68009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:45.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.665+0000 7f1f7cc00640 1 -- 172.21.15.86:0/3153201559 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1f78059ee0 con 0x7f1f8015e870 2024-04-20T21:18:45.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.865+0000 7f1f85e00640 1 -- 172.21.15.86:0/3153201559 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1f8006cc60 con 0x7f1f8015e870 2024-04-20T21:18:45.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.866+0000 7f1f7cc00640 1 -- 172.21.15.86:0/3153201559 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f1f780597e0 con 0x7f1f8015e870 2024-04-20T21:18:45.866 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-20T21:18:45.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.868+0000 7f1f85e00640 1 -- 172.21.15.86:0/3153201559 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1f5c06c8f0 msgr2=0x7f1f5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:45.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.868+0000 7f1f85e00640 1 --2- 172.21.15.86:0/3153201559 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1f5c06c8f0 0x7f1f5c06edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1f68007fd0 tx=0x7f1f68009040 comp rx=0 tx=0).stop 2024-04-20T21:18:45.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.868+0000 7f1f85e00640 1 -- 172.21.15.86:0/3153201559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f8015e870 msgr2=0x7f1f8015f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:45.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.868+0000 7f1f85e00640 1 --2- 172.21.15.86:0/3153201559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f8015e870 0x7f1f8015f660 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1f7800e930 tx=0x7f1f7800edf0 comp rx=0 tx=0).stop 2024-04-20T21:18:45.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.868+0000 7f1f85e00640 1 -- 172.21.15.86:0/3153201559 shutdown_connections 2024-04-20T21:18:45.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.868+0000 7f1f85e00640 1 --2- 172.21.15.86:0/3153201559 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1f5c06c8f0 0x7f1f5c06edb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:45.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.868+0000 7f1f85e00640 1 --2- 172.21.15.86:0/3153201559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f8015e870 0x7f1f8015f660 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:45.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.868+0000 7f1f85e00640 1 --2- 172.21.15.86:0/3153201559 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f800d4310 0x7f1f800b6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:45.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.868+0000 7f1f85e00640 1 --2- 172.21.15.86:0/3153201559 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f800cfb80 0x7f1f80155a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:45.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.868+0000 7f1f85e00640 1 -- 172.21.15.86:0/3153201559 >> 172.21.15.86:0/3153201559 conn(0x7f1f8009d9c0 msgr2=0x7f1f800d0820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:45.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.869+0000 7f1f85e00640 1 -- 172.21.15.86:0/3153201559 shutdown_connections 2024-04-20T21:18:45.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:45.869+0000 7f1f85e00640 1 -- 172.21.15.86:0/3153201559 wait complete. 2024-04-20T21:18:45.886 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T21:18:46.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.085+0000 7fe3dd600640 1 Processor -- start 2024-04-20T21:18:46.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.086+0000 7fe3dd600640 1 -- start start 2024-04-20T21:18:46.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.087+0000 7fe3dd600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe3d806ef20 0x7fe3d80b4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:46.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.087+0000 7fe3dd600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe3d80723e0 0x7fe3d80cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:46.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.087+0000 7fe3dd600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3d80b3cc0 con 0x7fe3d815ad50 2024-04-20T21:18:46.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.087+0000 7fe3dd600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fe3d80b3f60 con 0x7fe3d80723e0 2024-04-20T21:18:46.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.087+0000 7fe3dd600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe3d80b4240 con 0x7fe3d806ef20 2024-04-20T21:18:46.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.089+0000 7fe3ce000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe3d806ef20 0x7fe3d80b4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:46.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.089+0000 7fe3d7400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe3d80723e0 0x7fe3d80cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:46.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.089+0000 7fe3ce000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe3d806ef20 0x7fe3d80b4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:34606/0 (socket says 172.21.15.86:34606) 2024-04-20T21:18:46.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.089+0000 7fe3ce000640 1 -- 172.21.15.86:0/325515282 learned_addr learned my addr 172.21.15.86:0/325515282 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:46.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.089+0000 7fe3d7400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe3d80723e0 0x7fe3d80cd8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.86:42662/0 (socket says 172.21.15.86:42662) 2024-04-20T21:18:46.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.090+0000 7fe3ce000640 1 -- 172.21.15.86:0/325515282 >> v2:172.21.15.144:3300/0 conn(0x7fe3d80723e0 msgr2=0x7fe3d80cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:46.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.090+0000 7fe3ce000640 1 --2- 172.21.15.86:0/325515282 >> v2:172.21.15.144:3300/0 conn(0x7fe3d80723e0 0x7fe3d80cd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:46.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.090+0000 7fe3ce000640 1 -- 172.21.15.86:0/325515282 >> v1:172.21.15.144:6789/0 conn(0x7fe3d815ad50 legacy=0x7fe3d8071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:46.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.090+0000 7fe3ce000640 1 -- 172.21.15.86:0/325515282 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3d815f860 con 0x7fe3d806ef20 2024-04-20T21:18:46.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.090+0000 7fe3d7400640 1 --2- 172.21.15.86:0/325515282 >> v2:172.21.15.144:3300/0 conn(0x7fe3d80723e0 0x7fe3d80cd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:18:46.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.091+0000 7fe3ce000640 1 --2- 172.21.15.86:0/325515282 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe3d806ef20 0x7fe3d80b4a40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe3d815fa90 tx=0x7fe3c002dba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9404b11baffee978 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:46.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.091+0000 7fe3d6000640 1 -- 172.21.15.86:0/325515282 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c003b070 con 0x7fe3d806ef20 2024-04-20T21:18:46.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.091+0000 7fe3d6000640 1 -- 172.21.15.86:0/325515282 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3c002e950 con 0x7fe3d806ef20 2024-04-20T21:18:46.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.091+0000 7fe3d6000640 1 -- 172.21.15.86:0/325515282 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c002ed20 con 0x7fe3d806ef20 2024-04-20T21:18:46.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.092+0000 7fe3dd600640 1 -- 172.21.15.86:0/325515282 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe3d806ef20 msgr2=0x7fe3d80b4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:46.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.092+0000 7fe3dd600640 1 --2- 172.21.15.86:0/325515282 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe3d806ef20 0x7fe3d80b4a40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe3d815fa90 tx=0x7fe3c002dba0 comp rx=0 tx=0).stop 2024-04-20T21:18:46.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.092+0000 7fe3dd600640 1 -- 172.21.15.86:0/325515282 shutdown_connections 2024-04-20T21:18:46.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.092+0000 7fe3dd600640 1 --2- 172.21.15.86:0/325515282 >> v2:172.21.15.144:3300/0 conn(0x7fe3d80723e0 0x7fe3d80cd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:46.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.092+0000 7fe3dd600640 1 --2- 172.21.15.86:0/325515282 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe3d806ef20 0x7fe3d80b4a40 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:46.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.092+0000 7fe3dd600640 1 -- 172.21.15.86:0/325515282 >> 172.21.15.86:0/325515282 conn(0x7fe3d806b980 msgr2=0x7fe3d806bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:46.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.092+0000 7fe3dd600640 1 -- 172.21.15.86:0/325515282 shutdown_connections 2024-04-20T21:18:46.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.092+0000 7fe3dd600640 1 -- 172.21.15.86:0/325515282 wait complete. 2024-04-20T21:18:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.094+0000 7fe3dd600640 1 Processor -- start 2024-04-20T21:18:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.094+0000 7fe3dd600640 1 -- start start 2024-04-20T21:18:46.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.095+0000 7fe3dd600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3d806ef20 0x7fe3d81f4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:46.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.095+0000 7fe3dd600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe3d80723e0 0x7fe3d80cff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:46.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.095+0000 7fe3dd600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3d815ad50 0x7fe3d80d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:46.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.095+0000 7fe3dd600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d815f520 con 0x7fe3d815ad50 2024-04-20T21:18:46.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.095+0000 7fe3dd600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d815f3a0 con 0x7fe3d806ef20 2024-04-20T21:18:46.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.095+0000 7fe3dd600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe3d806f370 con 0x7fe3d80723e0 2024-04-20T21:18:46.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.095+0000 7fe3d7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3d815ad50 0x7fe3d80d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:46.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.095+0000 7fe3d7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3d815ad50 0x7fe3d80d04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37158/0 (socket says 172.21.15.86:37158) 2024-04-20T21:18:46.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.095+0000 7fe3d7400640 1 -- 172.21.15.86:0/2150458280 learned_addr learned my addr 172.21.15.86:0/2150458280 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:46.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.095+0000 7fe3d6a00640 1 --2- 172.21.15.86:0/2150458280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3d806ef20 0x7fe3d81f4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:46.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.096+0000 7fe3ce000640 1 --2- 172.21.15.86:0/2150458280 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe3d80723e0 0x7fe3d80cff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:46.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.096+0000 7fe3d7400640 1 -- 172.21.15.86:0/2150458280 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe3d80723e0 msgr2=0x7fe3d80cff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:46.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.096+0000 7fe3d7400640 1 --2- 172.21.15.86:0/2150458280 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe3d80723e0 0x7fe3d80cff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:46.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.096+0000 7fe3d7400640 1 -- 172.21.15.86:0/2150458280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3d806ef20 msgr2=0x7fe3d81f4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:46.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.096+0000 7fe3d7400640 1 --2- 172.21.15.86:0/2150458280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3d806ef20 0x7fe3d81f4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:46.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.096+0000 7fe3d7400640 1 -- 172.21.15.86:0/2150458280 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3d80d0d90 con 0x7fe3d815ad50 2024-04-20T21:18:46.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.096+0000 7fe3d7400640 1 --2- 172.21.15.86:0/2150458280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3d815ad50 0x7fe3d80d04a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe3d000e900 tx=0x7fe3d000edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:46.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.096+0000 7fe3cf400640 1 -- 172.21.15.86:0/2150458280 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d0017070 con 0x7fe3d815ad50 2024-04-20T21:18:46.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.096+0000 7fe3cf400640 1 -- 172.21.15.86:0/2150458280 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3d000ca00 con 0x7fe3d815ad50 2024-04-20T21:18:46.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.096+0000 7fe3cf400640 1 -- 172.21.15.86:0/2150458280 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d000cdb0 con 0x7fe3d815ad50 2024-04-20T21:18:46.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.096+0000 7fe3dd600640 1 -- 172.21.15.86:0/2150458280 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3d80d0ab0 con 0x7fe3d815ad50 2024-04-20T21:18:46.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.097+0000 7fe3dd600640 1 -- 172.21.15.86:0/2150458280 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3d81516a0 con 0x7fe3d815ad50 2024-04-20T21:18:46.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.098+0000 7fe3dd600640 1 -- 172.21.15.86:0/2150458280 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3d80689e0 con 0x7fe3d815ad50 2024-04-20T21:18:46.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.099+0000 7fe3cf400640 1 -- 172.21.15.86:0/2150458280 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe3d001a070 con 0x7fe3d815ad50 2024-04-20T21:18:46.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.099+0000 7fe3cf400640 1 --2- 172.21.15.86:0/2150458280 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe3b406c8f0 0x7fe3b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:46.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.100+0000 7fe3cf400640 1 -- 172.21.15.86:0/2150458280 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fe3d008ba90 con 0x7fe3d815ad50 2024-04-20T21:18:46.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.100+0000 7fe3d6a00640 1 --2- 172.21.15.86:0/2150458280 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe3b406c8f0 0x7fe3b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:46.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.100+0000 7fe3d6a00640 1 --2- 172.21.15.86:0/2150458280 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe3b406c8f0 0x7fe3b406edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8007fd0 tx=0x7fe3c8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:46.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.104+0000 7fe3cf400640 1 -- 172.21.15.86:0/2150458280 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe3d0059e40 con 0x7fe3d815ad50 2024-04-20T21:18:46.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.300+0000 7fe3dd600640 1 -- 172.21.15.86:0/2150458280 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe3d80ae950 con 0x7fe3d815ad50 2024-04-20T21:18:46.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.301+0000 7fe3cf400640 1 -- 172.21.15.86:0/2150458280 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7fe3d0059740 con 0x7fe3d815ad50 2024-04-20T21:18:46.301 INFO:teuthology.orchestra.run.smithi086.stderr:listed 0 entries 2024-04-20T21:18:46.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.303+0000 7fe3dd600640 1 -- 172.21.15.86:0/2150458280 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe3b406c8f0 msgr2=0x7fe3b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:46.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.303+0000 7fe3dd600640 1 --2- 172.21.15.86:0/2150458280 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe3b406c8f0 0x7fe3b406edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8007fd0 tx=0x7fe3c8009040 comp rx=0 tx=0).stop 2024-04-20T21:18:46.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.303+0000 7fe3dd600640 1 -- 172.21.15.86:0/2150458280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3d815ad50 msgr2=0x7fe3d80d04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:46.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.303+0000 7fe3dd600640 1 --2- 172.21.15.86:0/2150458280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3d815ad50 0x7fe3d80d04a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe3d000e900 tx=0x7fe3d000edc0 comp rx=0 tx=0).stop 2024-04-20T21:18:46.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.303+0000 7fe3dd600640 1 -- 172.21.15.86:0/2150458280 shutdown_connections 2024-04-20T21:18:46.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.303+0000 7fe3dd600640 1 --2- 172.21.15.86:0/2150458280 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe3b406c8f0 0x7fe3b406edb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:46.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.303+0000 7fe3dd600640 1 --2- 172.21.15.86:0/2150458280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3d815ad50 0x7fe3d80d04a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:46.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.303+0000 7fe3dd600640 1 --2- 172.21.15.86:0/2150458280 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe3d80723e0 0x7fe3d80cff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:46.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.303+0000 7fe3dd600640 1 --2- 172.21.15.86:0/2150458280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3d806ef20 0x7fe3d81f4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:46.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.303+0000 7fe3dd600640 1 -- 172.21.15.86:0/2150458280 >> 172.21.15.86:0/2150458280 conn(0x7fe3d806b980 msgr2=0x7fe3d806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:46.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.304+0000 7fe3dd600640 1 -- 172.21.15.86:0/2150458280 shutdown_connections 2024-04-20T21:18:46.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.304+0000 7fe3dd600640 1 -- 172.21.15.86:0/2150458280 wait complete. 2024-04-20T21:18:46.322 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:18:46.322 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:18:46.369 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:18:46.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.609+0000 7f0c33200640 1 Processor -- start 2024-04-20T21:18:46.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.609+0000 7f0c33200640 1 -- start start 2024-04-20T21:18:46.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.611+0000 7f0c33200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c34156ae0 0x7f0c3415d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:46.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.611+0000 7f0c33200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0c34160080 0x7f0c340ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:46.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.611+0000 7f0c33200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c34160890 con 0x7f0c340ad620 2024-04-20T21:18:46.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.611+0000 7f0c33200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f0c3406db20 con 0x7f0c34160080 2024-04-20T21:18:46.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.611+0000 7f0c33200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0c34160b70 con 0x7f0c34156ae0 2024-04-20T21:18:46.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.611+0000 7f0c31e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c34156ae0 0x7f0c3415d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:46.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.611+0000 7f0c31400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0c34160080 0x7f0c340ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:46.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.611+0000 7f0c31e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c34156ae0 0x7f0c3415d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:34630/0 (socket says 172.21.15.86:34630) 2024-04-20T21:18:46.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.612+0000 7f0c31e00640 1 -- 172.21.15.86:0/935139365 learned_addr learned my addr 172.21.15.86:0/935139365 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:46.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.612+0000 7f0c31e00640 1 -- 172.21.15.86:0/935139365 >> v2:172.21.15.144:3300/0 conn(0x7f0c34160080 msgr2=0x7f0c340ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:46.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.612+0000 7f0c31e00640 1 --2- 172.21.15.86:0/935139365 >> v2:172.21.15.144:3300/0 conn(0x7f0c34160080 0x7f0c340ad0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:46.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.612+0000 7f0c31e00640 1 -- 172.21.15.86:0/935139365 >> v1:172.21.15.144:6789/0 conn(0x7f0c340ad620 legacy=0x7f0c34141cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:46.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.612+0000 7f0c31e00640 1 -- 172.21.15.86:0/935139365 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c34160e50 con 0x7f0c34156ae0 2024-04-20T21:18:46.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.613+0000 7f0c31e00640 1 --2- 172.21.15.86:0/935139365 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c34156ae0 0x7f0c3415d6a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0c24009870 tx=0x7f0c2402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4c1177deed8fa9c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:46.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.614+0000 7f0c30a00640 1 -- 172.21.15.86:0/935139365 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c24034070 con 0x7f0c34156ae0 2024-04-20T21:18:46.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.614+0000 7f0c30a00640 1 -- 172.21.15.86:0/935139365 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c24030840 con 0x7f0c34156ae0 2024-04-20T21:18:46.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.614+0000 7f0c30a00640 1 -- 172.21.15.86:0/935139365 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c24030c10 con 0x7f0c34156ae0 2024-04-20T21:18:46.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.615+0000 7f0c33200640 1 -- 172.21.15.86:0/935139365 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c34156ae0 msgr2=0x7f0c3415d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:46.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.615+0000 7f0c33200640 1 --2- 172.21.15.86:0/935139365 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c34156ae0 0x7f0c3415d6a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0c24009870 tx=0x7f0c2402b980 comp rx=0 tx=0).stop 2024-04-20T21:18:46.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.615+0000 7f0c33200640 1 -- 172.21.15.86:0/935139365 shutdown_connections 2024-04-20T21:18:46.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.615+0000 7f0c33200640 1 --2- 172.21.15.86:0/935139365 >> v2:172.21.15.144:3300/0 conn(0x7f0c34160080 0x7f0c340ad0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:46.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.615+0000 7f0c33200640 1 --2- 172.21.15.86:0/935139365 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c34156ae0 0x7f0c3415d6a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:46.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.615+0000 7f0c33200640 1 -- 172.21.15.86:0/935139365 >> 172.21.15.86:0/935139365 conn(0x7f0c340d0170 msgr2=0x7f0c340d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:46.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.615+0000 7f0c33200640 1 -- 172.21.15.86:0/935139365 shutdown_connections 2024-04-20T21:18:46.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.615+0000 7f0c33200640 1 -- 172.21.15.86:0/935139365 wait complete. 2024-04-20T21:18:46.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.616+0000 7f0c33200640 1 Processor -- start 2024-04-20T21:18:46.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.617+0000 7f0c33200640 1 -- start start 2024-04-20T21:18:46.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.617+0000 7f0c33200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c340ad620 0x7f0c341f4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:46.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.618+0000 7f0c33200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c34156ae0 0x7f0c34154190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:46.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.618+0000 7f0c33200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c34160080 0x7f0c3415b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:46.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.618+0000 7f0c33200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0c34160500 con 0x7f0c34160080 2024-04-20T21:18:46.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.618+0000 7f0c33200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0c34156ec0 con 0x7f0c340ad620 2024-04-20T21:18:46.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.618+0000 7f0c33200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0c3406f2a0 con 0x7f0c34156ae0 2024-04-20T21:18:46.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.618+0000 7f0c32800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c34160080 0x7f0c3415b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:46.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.618+0000 7f0c32800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c34160080 0x7f0c3415b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37170/0 (socket says 172.21.15.86:37170) 2024-04-20T21:18:46.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.618+0000 7f0c32800640 1 -- 172.21.15.86:0/1453574001 learned_addr learned my addr 172.21.15.86:0/1453574001 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:46.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.618+0000 7f0c31400640 1 --2- 172.21.15.86:0/1453574001 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c34156ae0 0x7f0c34154190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:46.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.618+0000 7f0c31e00640 1 --2- 172.21.15.86:0/1453574001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c340ad620 0x7f0c341f4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:46.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.618+0000 7f0c32800640 1 -- 172.21.15.86:0/1453574001 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c34156ae0 msgr2=0x7f0c34154190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:46.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.618+0000 7f0c32800640 1 --2- 172.21.15.86:0/1453574001 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c34156ae0 0x7f0c34154190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:46.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.618+0000 7f0c32800640 1 -- 172.21.15.86:0/1453574001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c340ad620 msgr2=0x7f0c341f4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:46.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.618+0000 7f0c32800640 1 --2- 172.21.15.86:0/1453574001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c340ad620 0x7f0c341f4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:46.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.618+0000 7f0c32800640 1 -- 172.21.15.86:0/1453574001 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c341f46b0 con 0x7f0c34160080 2024-04-20T21:18:46.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.619+0000 7f0c32800640 1 --2- 172.21.15.86:0/1453574001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c34160080 0x7f0c3415b6e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c00c6e0 tx=0x7f0c2c00cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:46.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.619+0000 7f0c2a000640 1 -- 172.21.15.86:0/1453574001 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c003b70 con 0x7f0c34160080 2024-04-20T21:18:46.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.619+0000 7f0c2a000640 1 -- 172.21.15.86:0/1453574001 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c2c003d30 con 0x7f0c34160080 2024-04-20T21:18:46.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.619+0000 7f0c2a000640 1 -- 172.21.15.86:0/1453574001 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c005310 con 0x7f0c34160080 2024-04-20T21:18:46.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.619+0000 7f0c33200640 1 -- 172.21.15.86:0/1453574001 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c34153560 con 0x7f0c34160080 2024-04-20T21:18:46.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.619+0000 7f0c33200640 1 -- 172.21.15.86:0/1453574001 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c34153b20 con 0x7f0c34160080 2024-04-20T21:18:46.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.620+0000 7f0c33200640 1 -- 172.21.15.86:0/1453574001 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c340694b0 con 0x7f0c34160080 2024-04-20T21:18:46.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.621+0000 7f0c2a000640 1 -- 172.21.15.86:0/1453574001 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0c2c0054b0 con 0x7f0c34160080 2024-04-20T21:18:46.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.622+0000 7f0c2a000640 1 --2- 172.21.15.86:0/1453574001 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0c1406c8f0 0x7f0c1406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:46.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.623+0000 7f0c2a000640 1 -- 172.21.15.86:0/1453574001 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f0c2c08bb90 con 0x7f0c34160080 2024-04-20T21:18:46.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.623+0000 7f0c31e00640 1 --2- 172.21.15.86:0/1453574001 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0c1406c8f0 0x7f0c1406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:46.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.623+0000 7f0c31e00640 1 --2- 172.21.15.86:0/1453574001 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0c1406c8f0 0x7f0c1406edb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0c24036000 tx=0x7f0c2403b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:46.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.627+0000 7f0c2a000640 1 -- 172.21.15.86:0/1453574001 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0c340694b0 con 0x7f0c34160080 2024-04-20T21:18:46.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.815+0000 7f0c33200640 1 -- 172.21.15.86:0/1453574001 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0c3404fcf0 con 0x7f0c34160080 2024-04-20T21:18:46.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.818+0000 7f0c2a000640 1 -- 172.21.15.86:0/1453574001 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16854 (secure 0 0 0) 0x7f0c2c059e10 con 0x7f0c34160080 2024-04-20T21:18:46.819 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:18:46.819 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":22,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:18:33.788155+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:17:49.718799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:17:50.683477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:18:46.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.821+0000 7f0c33200640 1 -- 172.21.15.86:0/1453574001 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0c1406c8f0 msgr2=0x7f0c1406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:46.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.821+0000 7f0c33200640 1 --2- 172.21.15.86:0/1453574001 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0c1406c8f0 0x7f0c1406edb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0c24036000 tx=0x7f0c2403b040 comp rx=0 tx=0).stop 2024-04-20T21:18:46.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.821+0000 7f0c33200640 1 -- 172.21.15.86:0/1453574001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c34160080 msgr2=0x7f0c3415b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:46.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.821+0000 7f0c33200640 1 --2- 172.21.15.86:0/1453574001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c34160080 0x7f0c3415b6e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c00c6e0 tx=0x7f0c2c00cba0 comp rx=0 tx=0).stop 2024-04-20T21:18:46.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.821+0000 7f0c33200640 1 -- 172.21.15.86:0/1453574001 shutdown_connections 2024-04-20T21:18:46.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.821+0000 7f0c33200640 1 --2- 172.21.15.86:0/1453574001 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0c1406c8f0 0x7f0c1406edb0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:46.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.821+0000 7f0c33200640 1 --2- 172.21.15.86:0/1453574001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c34160080 0x7f0c3415b6e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:46.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.821+0000 7f0c33200640 1 --2- 172.21.15.86:0/1453574001 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c34156ae0 0x7f0c34154190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:46.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.821+0000 7f0c33200640 1 --2- 172.21.15.86:0/1453574001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c340ad620 0x7f0c341f4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:46.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.821+0000 7f0c33200640 1 -- 172.21.15.86:0/1453574001 >> 172.21.15.86:0/1453574001 conn(0x7f0c340d0170 msgr2=0x7f0c34072080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:46.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.821+0000 7f0c33200640 1 -- 172.21.15.86:0/1453574001 shutdown_connections 2024-04-20T21:18:46.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:46.821+0000 7f0c33200640 1 -- 172.21.15.86:0/1453574001 wait complete. 2024-04-20T21:18:46.840 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:17:49.718799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:17:50.683477+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:18:46.840 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:18:47.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.036+0000 7eff65c00640 1 Processor -- start 2024-04-20T21:18:47.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.037+0000 7eff65c00640 1 -- start start 2024-04-20T21:18:47.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.038+0000 7eff65c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7eff601622d0 0x7eff600d0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:47.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.038+0000 7eff65c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7eff600d34c0 0x7eff600b8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:47.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.038+0000 7eff65c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff600d3b50 con 0x7eff600d14b0 2024-04-20T21:18:47.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.038+0000 7eff65c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7eff601626b0 con 0x7eff601622d0 2024-04-20T21:18:47.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.038+0000 7eff65c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7eff600b82a0 con 0x7eff600d34c0 2024-04-20T21:18:47.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.039+0000 7eff5f400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7eff600d34c0 0x7eff600b8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:47.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.039+0000 7eff5ea00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7eff601622d0 0x7eff600d0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:47.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.039+0000 7eff5ea00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7eff601622d0 0x7eff600d0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.86:42688/0 (socket says 172.21.15.86:42688) 2024-04-20T21:18:47.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.039+0000 7eff5f400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7eff600d34c0 0x7eff600b8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:34656/0 (socket says 172.21.15.86:34656) 2024-04-20T21:18:47.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.039+0000 7eff5ea00640 1 -- 172.21.15.86:0/152029276 learned_addr learned my addr 172.21.15.86:0/152029276 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:47.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.040+0000 7eff5d600640 1 -- 172.21.15.86:0/152029276 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3469258808 0 0) 0x7eff600d3b50 con 0x7eff600d14b0 2024-04-20T21:18:47.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.040+0000 7eff5f400640 1 -- 172.21.15.86:0/152029276 >> v2:172.21.15.144:3300/0 conn(0x7eff601622d0 msgr2=0x7eff600d0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:47.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.040+0000 7eff5f400640 1 --2- 172.21.15.86:0/152029276 >> v2:172.21.15.144:3300/0 conn(0x7eff601622d0 0x7eff600d0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:47.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.040+0000 7eff5f400640 1 -- 172.21.15.86:0/152029276 >> v1:172.21.15.144:6789/0 conn(0x7eff600d14b0 legacy=0x7eff600d2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:47.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.040+0000 7eff5f400640 1 -- 172.21.15.86:0/152029276 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff600d3e30 con 0x7eff600d34c0 2024-04-20T21:18:47.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.040+0000 7eff5ea00640 1 --2- 172.21.15.86:0/152029276 >> v2:172.21.15.144:3300/0 conn(0x7eff601622d0 0x7eff600d0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:18:47.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.041+0000 7eff5f400640 1 --2- 172.21.15.86:0/152029276 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7eff600d34c0 0x7eff600b8730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7eff580097a0 tx=0x7eff5802b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abe2a5362a11211c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:47.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.041+0000 7eff5d600640 1 -- 172.21.15.86:0/152029276 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff58034070 con 0x7eff600d34c0 2024-04-20T21:18:47.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.041+0000 7eff5d600640 1 -- 172.21.15.86:0/152029276 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff58030800 con 0x7eff600d34c0 2024-04-20T21:18:47.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.041+0000 7eff5d600640 1 -- 172.21.15.86:0/152029276 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff58030bd0 con 0x7eff600d34c0 2024-04-20T21:18:47.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.042+0000 7eff65c00640 1 -- 172.21.15.86:0/152029276 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7eff600d34c0 msgr2=0x7eff600b8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:47.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.042+0000 7eff65c00640 1 --2- 172.21.15.86:0/152029276 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7eff600d34c0 0x7eff600b8730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7eff580097a0 tx=0x7eff5802b940 comp rx=0 tx=0).stop 2024-04-20T21:18:47.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.042+0000 7eff65c00640 1 -- 172.21.15.86:0/152029276 shutdown_connections 2024-04-20T21:18:47.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.042+0000 7eff65c00640 1 --2- 172.21.15.86:0/152029276 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7eff600d34c0 0x7eff600b8730 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:47.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.042+0000 7eff65c00640 1 --2- 172.21.15.86:0/152029276 >> v2:172.21.15.144:3300/0 conn(0x7eff601622d0 0x7eff600d0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:47.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.042+0000 7eff65c00640 1 -- 172.21.15.86:0/152029276 >> 172.21.15.86:0/152029276 conn(0x7eff6006cbe0 msgr2=0x7eff6006cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:47.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.042+0000 7eff65c00640 1 -- 172.21.15.86:0/152029276 shutdown_connections 2024-04-20T21:18:47.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.042+0000 7eff65c00640 1 -- 172.21.15.86:0/152029276 wait complete. 2024-04-20T21:18:47.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.044+0000 7eff65c00640 1 Processor -- start 2024-04-20T21:18:47.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.044+0000 7eff65c00640 1 -- start start 2024-04-20T21:18:47.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.045+0000 7eff65c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff600d14b0 0x7eff60156ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:47.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.045+0000 7eff65c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff600d34c0 0x7eff600c82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:47.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.045+0000 7eff65c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7eff601622d0 0x7eff600c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:47.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.045+0000 7eff65c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7eff600b7c20 con 0x7eff600d14b0 2024-04-20T21:18:47.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.045+0000 7eff65c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7eff600d18f0 con 0x7eff600d34c0 2024-04-20T21:18:47.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.045+0000 7eff65c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7eff600b8c70 con 0x7eff601622d0 2024-04-20T21:18:47.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.045+0000 7eff5ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff600d14b0 0x7eff60156ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:47.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.045+0000 7eff5ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff600d14b0 0x7eff60156ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37180/0 (socket says 172.21.15.86:37180) 2024-04-20T21:18:47.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.045+0000 7eff5ea00640 1 -- 172.21.15.86:0/4126499087 learned_addr learned my addr 172.21.15.86:0/4126499087 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:47.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.046+0000 7eff5f400640 1 --2- 172.21.15.86:0/4126499087 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7eff601622d0 0x7eff600c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:47.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.046+0000 7eff5e000640 1 --2- 172.21.15.86:0/4126499087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff600d34c0 0x7eff600c82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:47.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.046+0000 7eff5ea00640 1 -- 172.21.15.86:0/4126499087 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7eff601622d0 msgr2=0x7eff600c87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:47.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.046+0000 7eff5ea00640 1 --2- 172.21.15.86:0/4126499087 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7eff601622d0 0x7eff600c87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:47.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.046+0000 7eff5ea00640 1 -- 172.21.15.86:0/4126499087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff600d34c0 msgr2=0x7eff600c82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:47.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.046+0000 7eff5ea00640 1 --2- 172.21.15.86:0/4126499087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff600d34c0 0x7eff600c82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:47.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.046+0000 7eff5ea00640 1 -- 172.21.15.86:0/4126499087 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff601f8950 con 0x7eff600d14b0 2024-04-20T21:18:47.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.046+0000 7eff5ea00640 1 --2- 172.21.15.86:0/4126499087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff600d14b0 0x7eff60156ce0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7eff5000e9e0 tx=0x7eff5000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:47.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.046+0000 7eff56a00640 1 -- 172.21.15.86:0/4126499087 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff50017070 con 0x7eff600d14b0 2024-04-20T21:18:47.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.047+0000 7eff56a00640 1 -- 172.21.15.86:0/4126499087 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff5000c9d0 con 0x7eff600d14b0 2024-04-20T21:18:47.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.047+0000 7eff56a00640 1 -- 172.21.15.86:0/4126499087 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5000cd80 con 0x7eff600d14b0 2024-04-20T21:18:47.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.047+0000 7eff65c00640 1 -- 172.21.15.86:0/4126499087 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff601f4860 con 0x7eff600d14b0 2024-04-20T21:18:47.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.047+0000 7eff65c00640 1 -- 172.21.15.86:0/4126499087 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff60157220 con 0x7eff600d14b0 2024-04-20T21:18:47.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.048+0000 7eff65c00640 1 -- 172.21.15.86:0/4126499087 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff600689e0 con 0x7eff600d14b0 2024-04-20T21:18:47.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.049+0000 7eff56a00640 1 -- 172.21.15.86:0/4126499087 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7eff5001a070 con 0x7eff600d14b0 2024-04-20T21:18:47.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.050+0000 7eff56a00640 1 --2- 172.21.15.86:0/4126499087 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7eff3c06c8f0 0x7eff3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:47.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.050+0000 7eff56a00640 1 -- 172.21.15.86:0/4126499087 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7eff5008be20 con 0x7eff600d14b0 2024-04-20T21:18:47.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.050+0000 7eff5e000640 1 --2- 172.21.15.86:0/4126499087 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7eff3c06c8f0 0x7eff3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:47.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.050+0000 7eff5e000640 1 --2- 172.21.15.86:0/4126499087 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7eff3c06c8f0 0x7eff3c06edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7eff48007fd0 tx=0x7eff48009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:47.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.054+0000 7eff56a00640 1 -- 172.21.15.86:0/4126499087 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7eff5005a1c0 con 0x7eff600d14b0 2024-04-20T21:18:47.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.256+0000 7eff65c00640 1 -- 172.21.15.86:0/4126499087 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7eff601f4620 con 0x7eff600d14b0 2024-04-20T21:18:47.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.256+0000 7eff56a00640 1 -- 172.21.15.86:0/4126499087 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7eff50059ac0 con 0x7eff600d14b0 2024-04-20T21:18:47.257 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:18:47.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.259+0000 7eff65c00640 1 -- 172.21.15.86:0/4126499087 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7eff3c06c8f0 msgr2=0x7eff3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:47.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.259+0000 7eff65c00640 1 --2- 172.21.15.86:0/4126499087 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7eff3c06c8f0 0x7eff3c06edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7eff48007fd0 tx=0x7eff48009040 comp rx=0 tx=0).stop 2024-04-20T21:18:47.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.259+0000 7eff65c00640 1 -- 172.21.15.86:0/4126499087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff600d14b0 msgr2=0x7eff60156ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:47.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.259+0000 7eff65c00640 1 --2- 172.21.15.86:0/4126499087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff600d14b0 0x7eff60156ce0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7eff5000e9e0 tx=0x7eff5000eea0 comp rx=0 tx=0).stop 2024-04-20T21:18:47.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.259+0000 7eff65c00640 1 -- 172.21.15.86:0/4126499087 shutdown_connections 2024-04-20T21:18:47.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.259+0000 7eff65c00640 1 --2- 172.21.15.86:0/4126499087 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7eff3c06c8f0 0x7eff3c06edb0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:47.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.259+0000 7eff65c00640 1 --2- 172.21.15.86:0/4126499087 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7eff601622d0 0x7eff600c87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:47.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.259+0000 7eff65c00640 1 --2- 172.21.15.86:0/4126499087 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff600d34c0 0x7eff600c82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:47.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.259+0000 7eff65c00640 1 --2- 172.21.15.86:0/4126499087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff600d14b0 0x7eff60156ce0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:47.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.259+0000 7eff65c00640 1 -- 172.21.15.86:0/4126499087 >> 172.21.15.86:0/4126499087 conn(0x7eff6006cbe0 msgr2=0x7eff600b8f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:47.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.259+0000 7eff65c00640 1 -- 172.21.15.86:0/4126499087 shutdown_connections 2024-04-20T21:18:47.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.259+0000 7eff65c00640 1 -- 172.21.15.86:0/4126499087 wait complete. 2024-04-20T21:18:47.279 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:18:47.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.474+0000 7fd734a00640 1 Processor -- start 2024-04-20T21:18:47.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.474+0000 7fd734a00640 1 -- start start 2024-04-20T21:18:47.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.475+0000 7fd734a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd730145250 0x7fd73006efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:47.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.475+0000 7fd734a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd73006f500 con 0x7fd730072450 2024-04-20T21:18:47.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.475+0000 7fd734a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd730160410 con 0x7fd730145f00 2024-04-20T21:18:47.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.475+0000 7fd734a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd7301606f0 con 0x7fd730145250 2024-04-20T21:18:47.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.476+0000 7fd72e800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd730145250 0x7fd73006efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:47.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.476+0000 7fd72f200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd730072450 0x7fd73015cdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42598/0 (socket says 172.21.15.86:42598) 2024-04-20T21:18:47.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.476+0000 7fd72f200640 1 -- 172.21.15.86:0/1164133537 learned_addr learned my addr 172.21.15.86:0/1164133537 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:18:47.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.476+0000 7fd72e800640 1 -- 172.21.15.86:0/1164133537 >> v1:172.21.15.144:6789/0 conn(0x7fd730145f00 legacy=0x7fd730071d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:47.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.476+0000 7fd72e800640 1 -- 172.21.15.86:0/1164133537 >> v1:172.21.15.86:6789/0 conn(0x7fd730072450 legacy=0x7fd73015cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:47.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.476+0000 7fd72e800640 1 -- 172.21.15.86:0/1164133537 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7301609d0 con 0x7fd730145250 2024-04-20T21:18:47.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.477+0000 7fd72e800640 1 --2- 172.21.15.86:0/1164133537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd730145250 0x7fd73006efc0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd7200097a0 tx=0x7fd72002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50ac66118a11cee8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:47.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.478+0000 7fd72d400640 1 -- 172.21.15.86:0/1164133537 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7200315a0 con 0x7fd730145250 2024-04-20T21:18:47.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.478+0000 7fd72d400640 1 -- 172.21.15.86:0/1164133537 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd720037070 con 0x7fd730145250 2024-04-20T21:18:47.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.478+0000 7fd72d400640 1 -- 172.21.15.86:0/1164133537 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd720031960 con 0x7fd730145250 2024-04-20T21:18:47.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.478+0000 7fd734a00640 1 -- 172.21.15.86:0/1164133537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd730145250 msgr2=0x7fd73006efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:47.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.478+0000 7fd734a00640 1 --2- 172.21.15.86:0/1164133537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd730145250 0x7fd73006efc0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd7200097a0 tx=0x7fd72002b840 comp rx=0 tx=0).stop 2024-04-20T21:18:47.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.478+0000 7fd734a00640 1 -- 172.21.15.86:0/1164133537 shutdown_connections 2024-04-20T21:18:47.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.478+0000 7fd734a00640 1 --2- 172.21.15.86:0/1164133537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd730145250 0x7fd73006efc0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:47.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.478+0000 7fd734a00640 1 -- 172.21.15.86:0/1164133537 >> 172.21.15.86:0/1164133537 conn(0x7fd730067cf0 msgr2=0x7fd73006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:47.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.479+0000 7fd734a00640 1 -- 172.21.15.86:0/1164133537 shutdown_connections 2024-04-20T21:18:47.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.479+0000 7fd734a00640 1 -- 172.21.15.86:0/1164133537 wait complete. 2024-04-20T21:18:47.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.480+0000 7fd734a00640 1 Processor -- start 2024-04-20T21:18:47.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.480+0000 7fd734a00640 1 -- start start 2024-04-20T21:18:47.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.481+0000 7fd734a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd730072450 0x7fd730153f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:47.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.481+0000 7fd734a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd730145250 0x7fd7300ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:47.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.481+0000 7fd734a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd730145f00 0x7fd7300b4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:47.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.481+0000 7fd734a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd7301457b0 con 0x7fd730145f00 2024-04-20T21:18:47.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.481+0000 7fd734a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd730145630 con 0x7fd730145250 2024-04-20T21:18:47.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.481+0000 7fd734a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd73015d500 con 0x7fd730072450 2024-04-20T21:18:47.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.482+0000 7fd72f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd730145f00 0x7fd7300b4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:47.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.482+0000 7fd72f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd730145f00 0x7fd7300b4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37192/0 (socket says 172.21.15.86:37192) 2024-04-20T21:18:47.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.482+0000 7fd72f200640 1 -- 172.21.15.86:0/622220205 learned_addr learned my addr 172.21.15.86:0/622220205 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:47.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.482+0000 7fd72e800640 1 --2- 172.21.15.86:0/622220205 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd730072450 0x7fd730153f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:47.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.482+0000 7fd72de00640 1 --2- 172.21.15.86:0/622220205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd730145250 0x7fd7300ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:47.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.482+0000 7fd72f200640 1 -- 172.21.15.86:0/622220205 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd730072450 msgr2=0x7fd730153f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:47.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.482+0000 7fd72f200640 1 --2- 172.21.15.86:0/622220205 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd730072450 0x7fd730153f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:47.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.482+0000 7fd72f200640 1 -- 172.21.15.86:0/622220205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd730145250 msgr2=0x7fd7300ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:47.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.482+0000 7fd72f200640 1 --2- 172.21.15.86:0/622220205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd730145250 0x7fd7300ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:47.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.482+0000 7fd72f200640 1 -- 172.21.15.86:0/622220205 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd73013d8f0 con 0x7fd730145f00 2024-04-20T21:18:47.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.482+0000 7fd72f200640 1 --2- 172.21.15.86:0/622220205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd730145f00 0x7fd7300b4700 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd72800cb20 tx=0x7fd72800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:47.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.483+0000 7fd726a00640 1 -- 172.21.15.86:0/622220205 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd728017070 con 0x7fd730145f00 2024-04-20T21:18:47.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.483+0000 7fd726a00640 1 -- 172.21.15.86:0/622220205 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd72800aca0 con 0x7fd730145f00 2024-04-20T21:18:47.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.483+0000 7fd734a00640 1 -- 172.21.15.86:0/622220205 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd73013dbd0 con 0x7fd730145f00 2024-04-20T21:18:47.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.483+0000 7fd726a00640 1 -- 172.21.15.86:0/622220205 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd728012600 con 0x7fd730145f00 2024-04-20T21:18:47.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.483+0000 7fd734a00640 1 -- 172.21.15.86:0/622220205 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd73013e190 con 0x7fd730145f00 2024-04-20T21:18:47.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.484+0000 7fd734a00640 1 -- 172.21.15.86:0/622220205 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd73004fd90 con 0x7fd730145f00 2024-04-20T21:18:47.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.485+0000 7fd726a00640 1 -- 172.21.15.86:0/622220205 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd72801a070 con 0x7fd730145f00 2024-04-20T21:18:47.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.486+0000 7fd726a00640 1 --2- 172.21.15.86:0/622220205 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd71006c920 0x7fd71006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:47.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.486+0000 7fd726a00640 1 -- 172.21.15.86:0/622220205 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fd72808b860 con 0x7fd730145f00 2024-04-20T21:18:47.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.486+0000 7fd72e800640 1 --2- 172.21.15.86:0/622220205 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd71006c920 0x7fd71006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:47.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.487+0000 7fd72e800640 1 --2- 172.21.15.86:0/622220205 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd71006c920 0x7fd71006ede0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd720036000 tx=0x7fd72003d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:47.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.490+0000 7fd726a00640 1 -- 172.21.15.86:0/622220205 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd73006f500 con 0x7fd730145f00 2024-04-20T21:18:47.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.690+0000 7fd734a00640 1 -- 172.21.15.86:0/622220205 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd730072ea0 con 0x7fd730145f00 2024-04-20T21:18:47.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.691+0000 7fd726a00640 1 -- 172.21.15.86:0/622220205 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7fd7280595c0 con 0x7fd730145f00 2024-04-20T21:18:47.691 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-20T21:18:47.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.694+0000 7fd734a00640 1 -- 172.21.15.86:0/622220205 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd71006c920 msgr2=0x7fd71006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:47.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.694+0000 7fd734a00640 1 --2- 172.21.15.86:0/622220205 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd71006c920 0x7fd71006ede0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd720036000 tx=0x7fd72003d040 comp rx=0 tx=0).stop 2024-04-20T21:18:47.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.694+0000 7fd734a00640 1 -- 172.21.15.86:0/622220205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd730145f00 msgr2=0x7fd7300b4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:47.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.694+0000 7fd734a00640 1 --2- 172.21.15.86:0/622220205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd730145f00 0x7fd7300b4700 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd72800cb20 tx=0x7fd72800a060 comp rx=0 tx=0).stop 2024-04-20T21:18:47.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.694+0000 7fd734a00640 1 -- 172.21.15.86:0/622220205 shutdown_connections 2024-04-20T21:18:47.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.694+0000 7fd734a00640 1 --2- 172.21.15.86:0/622220205 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd71006c920 0x7fd71006ede0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:47.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.694+0000 7fd734a00640 1 --2- 172.21.15.86:0/622220205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd730145f00 0x7fd7300b4700 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:47.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.694+0000 7fd734a00640 1 --2- 172.21.15.86:0/622220205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd730145250 0x7fd7300ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:47.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.694+0000 7fd734a00640 1 --2- 172.21.15.86:0/622220205 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd730072450 0x7fd730153f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:47.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.694+0000 7fd734a00640 1 -- 172.21.15.86:0/622220205 >> 172.21.15.86:0/622220205 conn(0x7fd730067cf0 msgr2=0x7fd7301444f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:47.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.694+0000 7fd734a00640 1 -- 172.21.15.86:0/622220205 shutdown_connections 2024-04-20T21:18:47.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.694+0000 7fd734a00640 1 -- 172.21.15.86:0/622220205 wait complete. 2024-04-20T21:18:47.713 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:18:47.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.911+0000 7ffb43e00640 1 Processor -- start 2024-04-20T21:18:47.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.911+0000 7ffb43e00640 1 -- start start 2024-04-20T21:18:47.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.913+0000 7ffb43e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ffb440b3200 0x7ffb4406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:47.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.913+0000 7ffb43e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb4406f040 0x7ffb44071c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:47.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.913+0000 7ffb43e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb4406f4d0 con 0x7ffb44163080 2024-04-20T21:18:47.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.913+0000 7ffb43e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7ffb4406ebb0 con 0x7ffb440b3200 2024-04-20T21:18:47.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.913+0000 7ffb43e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ffb4406dfa0 con 0x7ffb4406f040 2024-04-20T21:18:47.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.913+0000 7ffb42000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7ffb44163080 0x7ffb440b35e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42606/0 (socket says 172.21.15.86:42606) 2024-04-20T21:18:47.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.913+0000 7ffb42000640 1 -- 172.21.15.86:0/2116799920 learned_addr learned my addr 172.21.15.86:0/2116799920 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:18:47.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.914+0000 7ffb43400640 1 --2- 172.21.15.86:0/2116799920 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb4406f040 0x7ffb44071c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:47.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.914+0000 7ffb41600640 1 -- 172.21.15.86:0/2116799920 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2557398142 0 0) 0x7ffb4406f4d0 con 0x7ffb44163080 2024-04-20T21:18:47.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.914+0000 7ffb41600640 1 -- 172.21.15.86:0/2116799920 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb28003520 con 0x7ffb44163080 2024-04-20T21:18:47.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.914+0000 7ffb42a00640 1 --2- 172.21.15.86:0/2116799920 >> v2:172.21.15.144:3300/0 conn(0x7ffb440b3200 0x7ffb4406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:47.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.914+0000 7ffb41600640 1 -- 172.21.15.86:0/2116799920 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 308335927 0 0) 0x7ffb28003520 con 0x7ffb44163080 2024-04-20T21:18:47.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.914+0000 7ffb41600640 1 -- 172.21.15.86:0/2116799920 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb4406f040 msgr2=0x7ffb44071c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:47.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.914+0000 7ffb41600640 1 --2- 172.21.15.86:0/2116799920 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb4406f040 0x7ffb44071c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:47.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.914+0000 7ffb41600640 1 -- 172.21.15.86:0/2116799920 >> v2:172.21.15.144:3300/0 conn(0x7ffb440b3200 msgr2=0x7ffb4406d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:47.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.914+0000 7ffb41600640 1 --2- 172.21.15.86:0/2116799920 >> v2:172.21.15.144:3300/0 conn(0x7ffb440b3200 0x7ffb4406d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:47.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.914+0000 7ffb41600640 1 -- 172.21.15.86:0/2116799920 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb4406e280 con 0x7ffb44163080 2024-04-20T21:18:47.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.914+0000 7ffb41600640 1 -- 172.21.15.86:0/2116799920 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7ffb2c001670 con 0x7ffb44163080 2024-04-20T21:18:47.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.914+0000 7ffb41600640 1 -- 172.21.15.86:0/2116799920 >> v1:172.21.15.86:6789/0 conn(0x7ffb44163080 legacy=0x7ffb440b35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:47.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.914+0000 7ffb41600640 1 --2- 172.21.15.86:0/2116799920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb280039b0 0x7ffb28003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:47.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.914+0000 7ffb41600640 1 --2- 172.21.15.86:0/2116799920 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb4406f040 0x7ffb28004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:47.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.914+0000 7ffb41600640 1 --2- 172.21.15.86:0/2116799920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb28004d40 0x7ffb280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:47.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.914+0000 7ffb41600640 1 -- 172.21.15.86:0/2116799920 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffb4406ebb0 con 0x7ffb28004d40 2024-04-20T21:18:47.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.914+0000 7ffb41600640 1 -- 172.21.15.86:0/2116799920 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ffb4406dfa0 con 0x7ffb280039b0 2024-04-20T21:18:47.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.914+0000 7ffb41600640 1 -- 172.21.15.86:0/2116799920 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ffb2800bb20 con 0x7ffb4406f040 2024-04-20T21:18:47.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.915+0000 7ffb43400640 1 --2- 172.21.15.86:0/2116799920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb28004d40 0x7ffb280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:47.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.915+0000 7ffb42000640 1 --2- 172.21.15.86:0/2116799920 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb4406f040 0x7ffb28004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:47.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.915+0000 7ffb42a00640 1 --2- 172.21.15.86:0/2116799920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb280039b0 0x7ffb28003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:47.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.915+0000 7ffb43400640 1 -- 172.21.15.86:0/2116799920 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb4406f040 msgr2=0x7ffb28004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:47.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.915+0000 7ffb43400640 1 --2- 172.21.15.86:0/2116799920 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb4406f040 0x7ffb28004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:47.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.915+0000 7ffb43400640 1 -- 172.21.15.86:0/2116799920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb280039b0 msgr2=0x7ffb28003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:47.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.915+0000 7ffb43400640 1 --2- 172.21.15.86:0/2116799920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb280039b0 0x7ffb28003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:47.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.915+0000 7ffb43400640 1 -- 172.21.15.86:0/2116799920 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb28003520 con 0x7ffb28004d40 2024-04-20T21:18:47.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.916+0000 7ffb43400640 1 --2- 172.21.15.86:0/2116799920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb28004d40 0x7ffb280051c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c007360 tx=0x7ffb28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e66d167c6e56c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:47.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.917+0000 7ffb41600640 1 -- 172.21.15.86:0/2116799920 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c039070 con 0x7ffb28004d40 2024-04-20T21:18:47.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.917+0000 7ffb41600640 1 -- 172.21.15.86:0/2116799920 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb3c02ccd0 con 0x7ffb28004d40 2024-04-20T21:18:47.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.917+0000 7ffb41600640 1 -- 172.21.15.86:0/2116799920 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c009be0 con 0x7ffb28004d40 2024-04-20T21:18:47.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.917+0000 7ffb43e00640 1 -- 172.21.15.86:0/2116799920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb28004d40 msgr2=0x7ffb280051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:47.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.917+0000 7ffb43e00640 1 --2- 172.21.15.86:0/2116799920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb28004d40 0x7ffb280051c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c007360 tx=0x7ffb28005830 comp rx=0 tx=0).stop 2024-04-20T21:18:47.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.917+0000 7ffb43400640 1 -- 172.21.15.86:0/2116799920 reap_dead start 2024-04-20T21:18:47.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.917+0000 7ffb43e00640 1 -- 172.21.15.86:0/2116799920 shutdown_connections 2024-04-20T21:18:47.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.917+0000 7ffb43e00640 1 -- 172.21.15.86:0/2116799920 >> 172.21.15.86:0/2116799920 conn(0x7ffb440b1f00 msgr2=0x7ffb440b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:47.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.917+0000 7ffb43e00640 1 -- 172.21.15.86:0/2116799920 shutdown_connections 2024-04-20T21:18:47.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.917+0000 7ffb43e00640 1 -- 172.21.15.86:0/2116799920 wait complete. 2024-04-20T21:18:47.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.919+0000 7ffb43e00640 1 Processor -- start 2024-04-20T21:18:47.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.919+0000 7ffb43e00640 1 -- start start 2024-04-20T21:18:47.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.920+0000 7ffb43e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb28004d40 0x7ffb4415a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:47.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.920+0000 7ffb43e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb4415f330 0x7ffb440ae7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:47.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.920+0000 7ffb43e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb440aed10 0x7ffb4415d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:47.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.920+0000 7ffb43e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffb4406fa40 con 0x7ffb28004d40 2024-04-20T21:18:47.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.920+0000 7ffb43e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ffb4406e840 con 0x7ffb440aed10 2024-04-20T21:18:47.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.920+0000 7ffb43e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ffb4415d3d0 con 0x7ffb4415f330 2024-04-20T21:18:47.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.920+0000 7ffb42a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb28004d40 0x7ffb4415a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:47.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.921+0000 7ffb42a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb28004d40 0x7ffb4415a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37214/0 (socket says 172.21.15.86:37214) 2024-04-20T21:18:47.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.921+0000 7ffb42a00640 1 -- 172.21.15.86:0/2759051142 learned_addr learned my addr 172.21.15.86:0/2759051142 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:47.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.921+0000 7ffb42000640 1 --2- 172.21.15.86:0/2759051142 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb4415f330 0x7ffb440ae7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:47.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.921+0000 7ffb42a00640 1 -- 172.21.15.86:0/2759051142 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb4415f330 msgr2=0x7ffb440ae7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:47.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.921+0000 7ffb42a00640 1 --2- 172.21.15.86:0/2759051142 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb4415f330 0x7ffb440ae7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:47.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.921+0000 7ffb42a00640 1 -- 172.21.15.86:0/2759051142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb440aed10 msgr2=0x7ffb4415d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:47.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.921+0000 7ffb42a00640 1 --2- 172.21.15.86:0/2759051142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb440aed10 0x7ffb4415d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:47.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.921+0000 7ffb42a00640 1 -- 172.21.15.86:0/2759051142 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb4415e290 con 0x7ffb28004d40 2024-04-20T21:18:47.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.921+0000 7ffb42a00640 1 --2- 172.21.15.86:0/2759051142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb28004d40 0x7ffb4415a7d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ffb3400f600 tx=0x7ffb3400fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:47.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.921+0000 7ffb3aa00640 1 -- 172.21.15.86:0/2759051142 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb34019020 con 0x7ffb28004d40 2024-04-20T21:18:47.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.922+0000 7ffb3aa00640 1 -- 172.21.15.86:0/2759051142 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb34014870 con 0x7ffb28004d40 2024-04-20T21:18:47.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.922+0000 7ffb43e00640 1 -- 172.21.15.86:0/2759051142 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb4415e570 con 0x7ffb28004d40 2024-04-20T21:18:47.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.922+0000 7ffb3aa00640 1 -- 172.21.15.86:0/2759051142 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb34014c20 con 0x7ffb28004d40 2024-04-20T21:18:47.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.922+0000 7ffb43e00640 1 -- 172.21.15.86:0/2759051142 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb4415eb30 con 0x7ffb28004d40 2024-04-20T21:18:47.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.923+0000 7ffb43e00640 1 -- 172.21.15.86:0/2759051142 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb4404fd90 con 0x7ffb28004d40 2024-04-20T21:18:47.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.924+0000 7ffb3aa00640 1 -- 172.21.15.86:0/2759051142 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffb3401c020 con 0x7ffb28004d40 2024-04-20T21:18:47.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.925+0000 7ffb3aa00640 1 --2- 172.21.15.86:0/2759051142 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ffb2006c8f0 0x7ffb2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:47.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.925+0000 7ffb3aa00640 1 -- 172.21.15.86:0/2759051142 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7ffb3408c410 con 0x7ffb28004d40 2024-04-20T21:18:47.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.925+0000 7ffb42000640 1 --2- 172.21.15.86:0/2759051142 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ffb2006c8f0 0x7ffb2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:47.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.926+0000 7ffb42000640 1 --2- 172.21.15.86:0/2759051142 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ffb2006c8f0 0x7ffb2006edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ffb44099da0 tx=0x7ffb2c000f50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:47.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:47.929+0000 7ffb3aa00640 1 -- 172.21.15.86:0/2759051142 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ffb34023020 con 0x7ffb28004d40 2024-04-20T21:18:48.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.121+0000 7ffb43e00640 1 -- 172.21.15.86:0/2759051142 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ffb441636e0 con 0x7ffb28004d40 2024-04-20T21:18:48.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.121+0000 7ffb3aa00640 1 -- 172.21.15.86:0/2759051142 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7ffb34026350 con 0x7ffb28004d40 2024-04-20T21:18:48.122 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-20T21:18:48.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.124+0000 7ffb43e00640 1 -- 172.21.15.86:0/2759051142 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ffb2006c8f0 msgr2=0x7ffb2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:48.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.124+0000 7ffb43e00640 1 --2- 172.21.15.86:0/2759051142 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ffb2006c8f0 0x7ffb2006edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ffb44099da0 tx=0x7ffb2c000f50 comp rx=0 tx=0).stop 2024-04-20T21:18:48.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.124+0000 7ffb43e00640 1 -- 172.21.15.86:0/2759051142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb28004d40 msgr2=0x7ffb4415a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:48.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.124+0000 7ffb43e00640 1 --2- 172.21.15.86:0/2759051142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb28004d40 0x7ffb4415a7d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ffb3400f600 tx=0x7ffb3400fac0 comp rx=0 tx=0).stop 2024-04-20T21:18:48.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.124+0000 7ffb43e00640 1 -- 172.21.15.86:0/2759051142 shutdown_connections 2024-04-20T21:18:48.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.124+0000 7ffb43e00640 1 --2- 172.21.15.86:0/2759051142 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ffb2006c8f0 0x7ffb2006edb0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:48.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.124+0000 7ffb43e00640 1 --2- 172.21.15.86:0/2759051142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb440aed10 0x7ffb4415d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:48.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.124+0000 7ffb43e00640 1 --2- 172.21.15.86:0/2759051142 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb4415f330 0x7ffb440ae7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:48.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.124+0000 7ffb43e00640 1 --2- 172.21.15.86:0/2759051142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb28004d40 0x7ffb4415a7d0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:48.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.124+0000 7ffb43e00640 1 -- 172.21.15.86:0/2759051142 >> 172.21.15.86:0/2759051142 conn(0x7ffb440b1f00 msgr2=0x7ffb4415c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:48.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.125+0000 7ffb43e00640 1 -- 172.21.15.86:0/2759051142 shutdown_connections 2024-04-20T21:18:48.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.125+0000 7ffb43e00640 1 -- 172.21.15.86:0/2759051142 wait complete. 2024-04-20T21:18:48.142 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T21:18:48.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.342+0000 7fd7b1a00640 1 Processor -- start 2024-04-20T21:18:48.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.342+0000 7fd7b1a00640 1 -- start start 2024-04-20T21:18:48.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.342+0000 7fd7b1a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7ac145fe0 0x7fd7ac0d02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:48.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.342+0000 7fd7b1a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd7ac0d0810 0x7fd7ac0cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:48.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.343+0000 7fd7b1a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7ac15dcc0 con 0x7fd7ac0cd610 2024-04-20T21:18:48.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.343+0000 7fd7b1a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fd7ac0d00d0 con 0x7fd7ac0d0810 2024-04-20T21:18:48.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.343+0000 7fd7b1a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd7ac0d0c70 con 0x7fd7ac145fe0 2024-04-20T21:18:48.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.343+0000 7fd7ab400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd7ac0cd610 0x7fd7ac160350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42610/0 (socket says 172.21.15.86:42610) 2024-04-20T21:18:48.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.343+0000 7fd7ab400640 1 -- 172.21.15.86:0/797968021 learned_addr learned my addr 172.21.15.86:0/797968021 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:18:48.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.343+0000 7fd7aaa00640 1 --2- 172.21.15.86:0/797968021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7ac145fe0 0x7fd7ac0d02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:48.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.344+0000 7fd7aa000640 1 --2- 172.21.15.86:0/797968021 >> v2:172.21.15.144:3300/0 conn(0x7fd7ac0d0810 0x7fd7ac0cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:48.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.344+0000 7fd7a9600640 1 -- 172.21.15.86:0/797968021 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2761277033 0 0) 0x7fd7ac15dcc0 con 0x7fd7ac0cd610 2024-04-20T21:18:48.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.344+0000 7fd7a9600640 1 -- 172.21.15.86:0/797968021 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd790003520 con 0x7fd7ac0cd610 2024-04-20T21:18:48.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.344+0000 7fd7a9600640 1 -- 172.21.15.86:0/797968021 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1532984665 0 0) 0x7fd790003520 con 0x7fd7ac0cd610 2024-04-20T21:18:48.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.344+0000 7fd7a9600640 1 -- 172.21.15.86:0/797968021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7ac145fe0 msgr2=0x7fd7ac0d02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:48.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.344+0000 7fd7a9600640 1 --2- 172.21.15.86:0/797968021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7ac145fe0 0x7fd7ac0d02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:48.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.344+0000 7fd7a9600640 1 -- 172.21.15.86:0/797968021 >> v2:172.21.15.144:3300/0 conn(0x7fd7ac0d0810 msgr2=0x7fd7ac0cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:48.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.344+0000 7fd7a9600640 1 --2- 172.21.15.86:0/797968021 >> v2:172.21.15.144:3300/0 conn(0x7fd7ac0d0810 0x7fd7ac0cd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:48.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.344+0000 7fd7a9600640 1 -- 172.21.15.86:0/797968021 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7ac15cf00 con 0x7fd7ac0cd610 2024-04-20T21:18:48.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.344+0000 7fd7a9600640 1 -- 172.21.15.86:0/797968021 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fd7a4001670 con 0x7fd7ac0cd610 2024-04-20T21:18:48.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.344+0000 7fd7a9600640 1 -- 172.21.15.86:0/797968021 >> v1:172.21.15.86:6789/0 conn(0x7fd7ac0cd610 legacy=0x7fd7ac160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:48.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.344+0000 7fd7a9600640 1 --2- 172.21.15.86:0/797968021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7ac145fe0 0x7fd7900039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:48.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.344+0000 7fd7a9600640 1 --2- 172.21.15.86:0/797968021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd790003f30 0x7fd790004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:48.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.344+0000 7fd7a9600640 1 --2- 172.21.15.86:0/797968021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd790004d40 0x7fd7900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:48.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.344+0000 7fd7a9600640 1 -- 172.21.15.86:0/797968021 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd7ac0d00d0 con 0x7fd790004d40 2024-04-20T21:18:48.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.344+0000 7fd7a9600640 1 -- 172.21.15.86:0/797968021 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd79000b9c0 con 0x7fd790003f30 2024-04-20T21:18:48.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.344+0000 7fd7a9600640 1 -- 172.21.15.86:0/797968021 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd79000bb40 con 0x7fd7ac145fe0 2024-04-20T21:18:48.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.345+0000 7fd7ab400640 1 --2- 172.21.15.86:0/797968021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd790004d40 0x7fd7900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:48.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.345+0000 7fd7aaa00640 1 --2- 172.21.15.86:0/797968021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7ac145fe0 0x7fd7900039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:48.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.345+0000 7fd7aa000640 1 --2- 172.21.15.86:0/797968021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd790003f30 0x7fd790004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:48.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.345+0000 7fd7ab400640 1 -- 172.21.15.86:0/797968021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7ac145fe0 msgr2=0x7fd7900039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:48.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.345+0000 7fd7ab400640 1 --2- 172.21.15.86:0/797968021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7ac145fe0 0x7fd7900039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:48.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.345+0000 7fd7ab400640 1 -- 172.21.15.86:0/797968021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd790003f30 msgr2=0x7fd790004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:48.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.345+0000 7fd7ab400640 1 --2- 172.21.15.86:0/797968021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd790003f30 0x7fd790004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:48.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.345+0000 7fd7ab400640 1 -- 172.21.15.86:0/797968021 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd790003520 con 0x7fd790004d40 2024-04-20T21:18:48.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.346+0000 7fd7ab400640 1 --2- 172.21.15.86:0/797968021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd790004d40 0x7fd7900051c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd7a400c290 tx=0x7fd790005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6b71658015e460c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:48.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.346+0000 7fd7a9600640 1 -- 172.21.15.86:0/797968021 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a4007d50 con 0x7fd790004d40 2024-04-20T21:18:48.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.347+0000 7fd7a9600640 1 -- 172.21.15.86:0/797968021 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7a4030070 con 0x7fd790004d40 2024-04-20T21:18:48.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.347+0000 7fd7a9600640 1 -- 172.21.15.86:0/797968021 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a402cdb0 con 0x7fd790004d40 2024-04-20T21:18:48.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.347+0000 7fd7b1a00640 1 -- 172.21.15.86:0/797968021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd790004d40 msgr2=0x7fd7900051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:48.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.347+0000 7fd7b1a00640 1 --2- 172.21.15.86:0/797968021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd790004d40 0x7fd7900051c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd7a400c290 tx=0x7fd790005830 comp rx=0 tx=0).stop 2024-04-20T21:18:48.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.347+0000 7fd7ab400640 1 -- 172.21.15.86:0/797968021 reap_dead start 2024-04-20T21:18:48.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.347+0000 7fd7b1a00640 1 -- 172.21.15.86:0/797968021 shutdown_connections 2024-04-20T21:18:48.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.347+0000 7fd7b1a00640 1 -- 172.21.15.86:0/797968021 >> 172.21.15.86:0/797968021 conn(0x7fd7ac06b980 msgr2=0x7fd7ac06bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:48.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.347+0000 7fd7b1a00640 1 -- 172.21.15.86:0/797968021 shutdown_connections 2024-04-20T21:18:48.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.347+0000 7fd7b1a00640 1 -- 172.21.15.86:0/797968021 wait complete. 2024-04-20T21:18:48.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.349+0000 7fd7b1a00640 1 Processor -- start 2024-04-20T21:18:48.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.349+0000 7fd7b1a00640 1 -- start start 2024-04-20T21:18:48.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.350+0000 7fd7b1a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd790004d40 0x7fd7ac142350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:48.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.350+0000 7fd7b1a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7ac0c5cf0 0x7fd7ac15beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:48.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.350+0000 7fd7b1a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7ac0c6170 0x7fd7ac15c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:48.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.350+0000 7fd7b1a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd7ac0cdaa0 con 0x7fd7ac0c5cf0 2024-04-20T21:18:48.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.350+0000 7fd7b1a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd7ac15d4c0 con 0x7fd790004d40 2024-04-20T21:18:48.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.350+0000 7fd7b1a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd7ac15d7a0 con 0x7fd7ac0c6170 2024-04-20T21:18:48.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.350+0000 7fd7aa000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7ac0c5cf0 0x7fd7ac15beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:48.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.350+0000 7fd7aa000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7ac0c5cf0 0x7fd7ac15beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37224/0 (socket says 172.21.15.86:37224) 2024-04-20T21:18:48.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.350+0000 7fd7aa000640 1 -- 172.21.15.86:0/2309417040 learned_addr learned my addr 172.21.15.86:0/2309417040 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:48.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.351+0000 7fd7aaa00640 1 --2- 172.21.15.86:0/2309417040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd790004d40 0x7fd7ac142350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:48.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.351+0000 7fd7ab400640 1 --2- 172.21.15.86:0/2309417040 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7ac0c6170 0x7fd7ac15c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:48.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.351+0000 7fd7aa000640 1 -- 172.21.15.86:0/2309417040 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7ac0c6170 msgr2=0x7fd7ac15c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:48.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.351+0000 7fd7aa000640 1 --2- 172.21.15.86:0/2309417040 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7ac0c6170 0x7fd7ac15c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:48.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.351+0000 7fd7aa000640 1 -- 172.21.15.86:0/2309417040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd790004d40 msgr2=0x7fd7ac142350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:48.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.351+0000 7fd7aa000640 1 --2- 172.21.15.86:0/2309417040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd790004d40 0x7fd7ac142350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:48.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.351+0000 7fd7aa000640 1 -- 172.21.15.86:0/2309417040 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7ac143000 con 0x7fd7ac0c5cf0 2024-04-20T21:18:48.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.351+0000 7fd7aa000640 1 --2- 172.21.15.86:0/2309417040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7ac0c5cf0 0x7fd7ac15beb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd794011700 tx=0x7fd794011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:48.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.351+0000 7fd7a2a00640 1 -- 172.21.15.86:0/2309417040 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79400f9c0 con 0x7fd7ac0c5cf0 2024-04-20T21:18:48.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.351+0000 7fd7a2a00640 1 -- 172.21.15.86:0/2309417040 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd79400fb60 con 0x7fd7ac0c5cf0 2024-04-20T21:18:48.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.351+0000 7fd7a2a00640 1 -- 172.21.15.86:0/2309417040 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd794015460 con 0x7fd7ac0c5cf0 2024-04-20T21:18:48.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.351+0000 7fd7b1a00640 1 -- 172.21.15.86:0/2309417040 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7ac0b00d0 con 0x7fd7ac0c5cf0 2024-04-20T21:18:48.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.351+0000 7fd7b1a00640 1 -- 172.21.15.86:0/2309417040 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7ac0b0690 con 0x7fd7ac0c5cf0 2024-04-20T21:18:48.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.352+0000 7fd7b1a00640 1 -- 172.21.15.86:0/2309417040 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7ac06f040 con 0x7fd7ac0c5cf0 2024-04-20T21:18:48.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.353+0000 7fd7a2a00640 1 -- 172.21.15.86:0/2309417040 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd794015600 con 0x7fd7ac0c5cf0 2024-04-20T21:18:48.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.354+0000 7fd7a2a00640 1 --2- 172.21.15.86:0/2309417040 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd78806c8f0 0x7fd78806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:48.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.354+0000 7fd7a2a00640 1 -- 172.21.15.86:0/2309417040 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fd79408c5b0 con 0x7fd7ac0c5cf0 2024-04-20T21:18:48.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.355+0000 7fd7aaa00640 1 --2- 172.21.15.86:0/2309417040 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd78806c8f0 0x7fd78806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:48.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.355+0000 7fd7aaa00640 1 --2- 172.21.15.86:0/2309417040 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd78806c8f0 0x7fd78806edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac1f7cb0 tx=0x7fd79c00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:48.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.359+0000 7fd7a2a00640 1 -- 172.21.15.86:0/2309417040 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd7ac06f040 con 0x7fd7ac0c5cf0 2024-04-20T21:18:48.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.570+0000 7fd7b1a00640 1 -- 172.21.15.86:0/2309417040 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd7ac0689e0 con 0x7fd7ac0c5cf0 2024-04-20T21:18:48.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.570+0000 7fd7a2a00640 1 -- 172.21.15.86:0/2309417040 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7fd794019030 con 0x7fd7ac0c5cf0 2024-04-20T21:18:48.571 INFO:teuthology.orchestra.run.smithi086.stderr:listed 0 entries 2024-04-20T21:18:48.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.573+0000 7fd7b1a00640 1 -- 172.21.15.86:0/2309417040 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd78806c8f0 msgr2=0x7fd78806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:48.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.573+0000 7fd7b1a00640 1 --2- 172.21.15.86:0/2309417040 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd78806c8f0 0x7fd78806edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac1f7cb0 tx=0x7fd79c00b040 comp rx=0 tx=0).stop 2024-04-20T21:18:48.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.573+0000 7fd7b1a00640 1 -- 172.21.15.86:0/2309417040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7ac0c5cf0 msgr2=0x7fd7ac15beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:48.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.573+0000 7fd7b1a00640 1 --2- 172.21.15.86:0/2309417040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7ac0c5cf0 0x7fd7ac15beb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd794011700 tx=0x7fd794011bc0 comp rx=0 tx=0).stop 2024-04-20T21:18:48.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.573+0000 7fd7b1a00640 1 -- 172.21.15.86:0/2309417040 shutdown_connections 2024-04-20T21:18:48.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.573+0000 7fd7b1a00640 1 --2- 172.21.15.86:0/2309417040 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd78806c8f0 0x7fd78806edb0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:48.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.573+0000 7fd7b1a00640 1 --2- 172.21.15.86:0/2309417040 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7ac0c6170 0x7fd7ac15c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:48.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.573+0000 7fd7b1a00640 1 --2- 172.21.15.86:0/2309417040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7ac0c5cf0 0x7fd7ac15beb0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:48.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.573+0000 7fd7b1a00640 1 --2- 172.21.15.86:0/2309417040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd790004d40 0x7fd7ac142350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:48.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.573+0000 7fd7b1a00640 1 -- 172.21.15.86:0/2309417040 >> 172.21.15.86:0/2309417040 conn(0x7fd7ac06b980 msgr2=0x7fd7ac15f130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:48.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.573+0000 7fd7b1a00640 1 -- 172.21.15.86:0/2309417040 shutdown_connections 2024-04-20T21:18:48.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:48.573+0000 7fd7b1a00640 1 -- 172.21.15.86:0/2309417040 wait complete. 2024-04-20T21:18:48.593 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi086.front.sepia.ceph.com... 2024-04-20T21:18:48.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:48.593 DEBUG:teuthology.orchestra.run.smithi086:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-04-20T21:18:48.674 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:18:48.674 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-20T21:18:48.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:48.674 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-20T21:18:48.725 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:18:48.725 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:18:48.782 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:18:48.783 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:18:48.836 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:18:49.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.044+0000 7f942ee00640 1 Processor -- start 2024-04-20T21:18:49.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.044+0000 7f942ee00640 1 -- start start 2024-04-20T21:18:49.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.046+0000 7f942ee00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f942815ece0 0x7f942815f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:49.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.046+0000 7f942ee00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f942815f600 con 0x7f94280cf9c0 2024-04-20T21:18:49.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.046+0000 7f942ee00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94280d3950 con 0x7f94280d50c0 2024-04-20T21:18:49.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.046+0000 7f942ee00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f94280cec00 con 0x7f942815ece0 2024-04-20T21:18:49.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.046+0000 7f942da00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f942815ece0 0x7f942815f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:49.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.046+0000 7f942e400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f94280cf9c0 0x7f9428160000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42614/0 (socket says 172.21.15.86:42614) 2024-04-20T21:18:49.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.046+0000 7f942e400640 1 -- 172.21.15.86:0/2397024298 learned_addr learned my addr 172.21.15.86:0/2397024298 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:18:49.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.046+0000 7f942da00640 1 -- 172.21.15.86:0/2397024298 >> v1:172.21.15.144:6789/0 conn(0x7f94280d50c0 legacy=0x7f94280d3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:49.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.046+0000 7f942da00640 1 -- 172.21.15.86:0/2397024298 >> v1:172.21.15.86:6789/0 conn(0x7f94280cf9c0 legacy=0x7f9428160000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:49.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.046+0000 7f942da00640 1 -- 172.21.15.86:0/2397024298 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94280ceee0 con 0x7f942815ece0 2024-04-20T21:18:49.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.046+0000 7f9427e00640 1 -- 172.21.15.86:0/2397024298 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3418141870 0 0) 0x7f942815f600 con 0x7f94280cf9c0 2024-04-20T21:18:49.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.047+0000 7f942da00640 1 --2- 172.21.15.86:0/2397024298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f942815ece0 0x7f942815f0c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f941c0097a0 tx=0x7f941c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf2775769286fcbb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:49.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.048+0000 7f9427e00640 1 -- 172.21.15.86:0/2397024298 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941c0315a0 con 0x7f942815ece0 2024-04-20T21:18:49.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.048+0000 7f9427e00640 1 -- 172.21.15.86:0/2397024298 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f941c037070 con 0x7f942815ece0 2024-04-20T21:18:49.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.048+0000 7f9427e00640 1 -- 172.21.15.86:0/2397024298 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941c031960 con 0x7f942815ece0 2024-04-20T21:18:49.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.048+0000 7f942ee00640 1 -- 172.21.15.86:0/2397024298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f942815ece0 msgr2=0x7f942815f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:49.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.048+0000 7f942ee00640 1 --2- 172.21.15.86:0/2397024298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f942815ece0 0x7f942815f0c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f941c0097a0 tx=0x7f941c02b840 comp rx=0 tx=0).stop 2024-04-20T21:18:49.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.048+0000 7f942ee00640 1 -- 172.21.15.86:0/2397024298 shutdown_connections 2024-04-20T21:18:49.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.048+0000 7f942ee00640 1 --2- 172.21.15.86:0/2397024298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f942815ece0 0x7f942815f0c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:49.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.048+0000 7f942ee00640 1 -- 172.21.15.86:0/2397024298 >> 172.21.15.86:0/2397024298 conn(0x7f942808bf70 msgr2=0x7f94280d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:49.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.049+0000 7f942ee00640 1 -- 172.21.15.86:0/2397024298 shutdown_connections 2024-04-20T21:18:49.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.049+0000 7f942ee00640 1 -- 172.21.15.86:0/2397024298 wait complete. 2024-04-20T21:18:49.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.050+0000 7f942ee00640 1 Processor -- start 2024-04-20T21:18:49.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.050+0000 7f942ee00640 1 -- start start 2024-04-20T21:18:49.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.051+0000 7f942ee00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94280cf9c0 0x7f942814fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:49.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.051+0000 7f942ee00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94280d50c0 0x7f942809fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:49.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.051+0000 7f942ee00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f942815ece0 0x7f942814c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:49.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.051+0000 7f942ee00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f942815ea50 con 0x7f942815ece0 2024-04-20T21:18:49.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.051+0000 7f942ee00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f94280cf4a0 con 0x7f94280d50c0 2024-04-20T21:18:49.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.051+0000 7f942ee00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f94280cfe70 con 0x7f94280cf9c0 2024-04-20T21:18:49.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.052+0000 7f942e400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f942815ece0 0x7f942814c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:49.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.052+0000 7f942e400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f942815ece0 0x7f942814c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37238/0 (socket says 172.21.15.86:37238) 2024-04-20T21:18:49.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.052+0000 7f942e400640 1 -- 172.21.15.86:0/1048836812 learned_addr learned my addr 172.21.15.86:0/1048836812 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:49.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.052+0000 7f942da00640 1 --2- 172.21.15.86:0/1048836812 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94280cf9c0 0x7f942814fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:49.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.052+0000 7f942d000640 1 --2- 172.21.15.86:0/1048836812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94280d50c0 0x7f942809fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:49.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.052+0000 7f942e400640 1 -- 172.21.15.86:0/1048836812 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94280cf9c0 msgr2=0x7f942814fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:49.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.052+0000 7f942e400640 1 --2- 172.21.15.86:0/1048836812 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94280cf9c0 0x7f942814fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:49.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.052+0000 7f942e400640 1 -- 172.21.15.86:0/1048836812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94280d50c0 msgr2=0x7f942809fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:49.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.052+0000 7f942e400640 1 --2- 172.21.15.86:0/1048836812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94280d50c0 0x7f942809fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:49.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.052+0000 7f942e400640 1 -- 172.21.15.86:0/1048836812 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f942814cff0 con 0x7f942815ece0 2024-04-20T21:18:49.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.052+0000 7f942d000640 1 --2- 172.21.15.86:0/1048836812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94280d50c0 0x7f942809fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:49.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.052+0000 7f942e400640 1 --2- 172.21.15.86:0/1048836812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f942815ece0 0x7f942814c600 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f942000c700 tx=0x7f942000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:49.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.053+0000 7f9425600640 1 -- 172.21.15.86:0/1048836812 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9420018070 con 0x7f942815ece0 2024-04-20T21:18:49.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.053+0000 7f9425600640 1 -- 172.21.15.86:0/1048836812 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f942000ce10 con 0x7f942815ece0 2024-04-20T21:18:49.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.053+0000 7f9425600640 1 -- 172.21.15.86:0/1048836812 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f942000a880 con 0x7f942815ece0 2024-04-20T21:18:49.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.053+0000 7f942ee00640 1 -- 172.21.15.86:0/1048836812 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f942814d290 con 0x7f942815ece0 2024-04-20T21:18:49.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.053+0000 7f942ee00640 1 -- 172.21.15.86:0/1048836812 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f942814d850 con 0x7f942815ece0 2024-04-20T21:18:49.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.054+0000 7f942ee00640 1 -- 172.21.15.86:0/1048836812 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f942804fcf0 con 0x7f942815ece0 2024-04-20T21:18:49.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.055+0000 7f9425600640 1 -- 172.21.15.86:0/1048836812 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f942001b070 con 0x7f942815ece0 2024-04-20T21:18:49.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.056+0000 7f9425600640 1 --2- 172.21.15.86:0/1048836812 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f940806c8f0 0x7f940806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:49.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.056+0000 7f9425600640 1 -- 172.21.15.86:0/1048836812 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f942008b160 con 0x7f942815ece0 2024-04-20T21:18:49.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.056+0000 7f942da00640 1 --2- 172.21.15.86:0/1048836812 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f940806c8f0 0x7f940806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:49.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.057+0000 7f942da00640 1 --2- 172.21.15.86:0/1048836812 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f940806c8f0 0x7f940806edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f941c036000 tx=0x7f941c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:49.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.060+0000 7f9425600640 1 -- 172.21.15.86:0/1048836812 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9420059500 con 0x7f942815ece0 2024-04-20T21:18:49.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.262+0000 7f942ee00640 1 -- 172.21.15.86:0/1048836812 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f942806e2a0 con 0x7f942815ece0 2024-04-20T21:18:49.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.265+0000 7f9425600640 1 -- 172.21.15.86:0/1048836812 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16854 (secure 0 0 0) 0x7f9420058e00 con 0x7f942815ece0 2024-04-20T21:18:49.264 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:18:49.264 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":22,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:18:33.788155+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:17:49.718799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:17:50.683477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:18:49.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.267+0000 7f942ee00640 1 -- 172.21.15.86:0/1048836812 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f940806c8f0 msgr2=0x7f940806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:49.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.267+0000 7f942ee00640 1 --2- 172.21.15.86:0/1048836812 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f940806c8f0 0x7f940806edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f941c036000 tx=0x7f941c008030 comp rx=0 tx=0).stop 2024-04-20T21:18:49.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.267+0000 7f942ee00640 1 -- 172.21.15.86:0/1048836812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f942815ece0 msgr2=0x7f942814c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:49.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.267+0000 7f942ee00640 1 --2- 172.21.15.86:0/1048836812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f942815ece0 0x7f942814c600 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f942000c700 tx=0x7f942000cbc0 comp rx=0 tx=0).stop 2024-04-20T21:18:49.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.268+0000 7f942ee00640 1 -- 172.21.15.86:0/1048836812 shutdown_connections 2024-04-20T21:18:49.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.268+0000 7f942ee00640 1 --2- 172.21.15.86:0/1048836812 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f940806c8f0 0x7f940806edb0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:49.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.268+0000 7f942ee00640 1 --2- 172.21.15.86:0/1048836812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f942815ece0 0x7f942814c600 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:49.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.268+0000 7f942ee00640 1 --2- 172.21.15.86:0/1048836812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94280d50c0 0x7f942809fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:49.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.268+0000 7f942ee00640 1 --2- 172.21.15.86:0/1048836812 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94280cf9c0 0x7f942814fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:49.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.268+0000 7f942ee00640 1 -- 172.21.15.86:0/1048836812 >> 172.21.15.86:0/1048836812 conn(0x7f942808bf70 msgr2=0x7f942815dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:49.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.268+0000 7f942ee00640 1 -- 172.21.15.86:0/1048836812 shutdown_connections 2024-04-20T21:18:49.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.268+0000 7f942ee00640 1 -- 172.21.15.86:0/1048836812 wait complete. 2024-04-20T21:18:49.288 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:17:49.718799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:17:50.683477+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:18:49.288 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:18:49.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.493+0000 7f189f000640 1 Processor -- start 2024-04-20T21:18:49.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.493+0000 7f189f000640 1 -- start start 2024-04-20T21:18:49.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.494+0000 7f189f000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1898155d30 0x7f18980cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:49.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.495+0000 7f189f000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1898156110 con 0x7f18980b52b0 2024-04-20T21:18:49.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.495+0000 7f189f000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18980ce460 con 0x7f189806b980 2024-04-20T21:18:49.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.495+0000 7f189f000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f18980b56e0 con 0x7f1898155d30 2024-04-20T21:18:49.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.495+0000 7f189d000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f18980b52b0 0x7f189806ddf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42630/0 (socket says 172.21.15.86:42630) 2024-04-20T21:18:49.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.495+0000 7f189d000640 1 -- 172.21.15.86:0/3241144333 learned_addr learned my addr 172.21.15.86:0/3241144333 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:18:49.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.495+0000 7f1897e00640 1 --2- 172.21.15.86:0/3241144333 >> v2:172.21.15.144:3300/0 conn(0x7f1898155d30 0x7f18980cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:49.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.495+0000 7f1897400640 1 -- 172.21.15.86:0/3241144333 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3440088690 0 0) 0x7f1898156110 con 0x7f18980b52b0 2024-04-20T21:18:49.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.496+0000 7f1897400640 1 -- 172.21.15.86:0/3241144333 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f187c003520 con 0x7f18980b52b0 2024-04-20T21:18:49.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.496+0000 7f1897400640 1 -- 172.21.15.86:0/3241144333 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3411209950 0 0) 0x7f187c003520 con 0x7f18980b52b0 2024-04-20T21:18:49.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.496+0000 7f1897400640 1 -- 172.21.15.86:0/3241144333 >> v2:172.21.15.144:3300/0 conn(0x7f1898155d30 msgr2=0x7f18980cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:49.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.496+0000 7f1897400640 1 --2- 172.21.15.86:0/3241144333 >> v2:172.21.15.144:3300/0 conn(0x7f1898155d30 0x7f18980cf500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:49.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.496+0000 7f1897400640 1 -- 172.21.15.86:0/3241144333 >> v1:172.21.15.144:6789/0 conn(0x7f189806b980 legacy=0x7f18980b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:49.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.496+0000 7f1897400640 1 -- 172.21.15.86:0/3241144333 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18980ce740 con 0x7f18980b52b0 2024-04-20T21:18:49.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.496+0000 7f1897400640 1 -- 172.21.15.86:0/3241144333 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f18900019f0 con 0x7f18980b52b0 2024-04-20T21:18:49.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.496+0000 7f1897e00640 1 --2- 172.21.15.86:0/3241144333 >> v2:172.21.15.144:3300/0 conn(0x7f1898155d30 0x7f18980cf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:18:49.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.496+0000 7f1897400640 1 -- 172.21.15.86:0/3241144333 >> v1:172.21.15.86:6789/0 conn(0x7f18980b52b0 legacy=0x7f189806ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:49.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.496+0000 7f1897400640 1 --2- 172.21.15.86:0/3241144333 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f187c0039b0 0x7f187c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:49.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.496+0000 7f1897400640 1 --2- 172.21.15.86:0/3241144333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f187c004800 0x7f187c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:49.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.496+0000 7f1897400640 1 --2- 172.21.15.86:0/3241144333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f187c0052e0 0x7f187c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:49.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.496+0000 7f1897400640 1 -- 172.21.15.86:0/3241144333 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f18980b56e0 con 0x7f187c0052e0 2024-04-20T21:18:49.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.496+0000 7f1897400640 1 -- 172.21.15.86:0/3241144333 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f187c00c1b0 con 0x7f187c004800 2024-04-20T21:18:49.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.496+0000 7f1897400640 1 -- 172.21.15.86:0/3241144333 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f187c00c3b0 con 0x7f187c0039b0 2024-04-20T21:18:49.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.496+0000 7f189d000640 1 --2- 172.21.15.86:0/3241144333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f187c0052e0 0x7f187c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:49.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.497+0000 7f188f400640 1 --2- 172.21.15.86:0/3241144333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f187c004800 0x7f187c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:49.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.497+0000 7f1897e00640 1 --2- 172.21.15.86:0/3241144333 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f187c0039b0 0x7f187c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:49.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.497+0000 7f189d000640 1 -- 172.21.15.86:0/3241144333 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f187c0039b0 msgr2=0x7f187c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:49.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.497+0000 7f189d000640 1 --2- 172.21.15.86:0/3241144333 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f187c0039b0 0x7f187c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:49.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.497+0000 7f189d000640 1 -- 172.21.15.86:0/3241144333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f187c004800 msgr2=0x7f187c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:49.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.497+0000 7f189d000640 1 --2- 172.21.15.86:0/3241144333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f187c004800 0x7f187c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:49.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.497+0000 7f189d000640 1 -- 172.21.15.86:0/3241144333 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f187c003520 con 0x7f187c0052e0 2024-04-20T21:18:49.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.498+0000 7f189d000640 1 -- 172.21.15.86:0/3241144333 reap_dead start 2024-04-20T21:18:49.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.498+0000 7f189d000640 1 --2- 172.21.15.86:0/3241144333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f187c0052e0 0x7f187c00b9c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f189000a810 tx=0x7f187c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10d03533e7694fee server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:49.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.499+0000 7f1897400640 1 -- 172.21.15.86:0/3241144333 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189000a460 con 0x7f187c0052e0 2024-04-20T21:18:49.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.499+0000 7f1897400640 1 -- 172.21.15.86:0/3241144333 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1890034650 con 0x7f187c0052e0 2024-04-20T21:18:49.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.499+0000 7f1897400640 1 -- 172.21.15.86:0/3241144333 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1890034a40 con 0x7f187c0052e0 2024-04-20T21:18:49.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.499+0000 7f189f000640 1 -- 172.21.15.86:0/3241144333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f187c0052e0 msgr2=0x7f187c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:49.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.499+0000 7f189f000640 1 --2- 172.21.15.86:0/3241144333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f187c0052e0 0x7f187c00b9c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f189000a810 tx=0x7f187c005830 comp rx=0 tx=0).stop 2024-04-20T21:18:49.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.499+0000 7f189f000640 1 -- 172.21.15.86:0/3241144333 shutdown_connections 2024-04-20T21:18:49.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.499+0000 7f189f000640 1 --2- 172.21.15.86:0/3241144333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f187c0052e0 0x7f187c00b9c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:49.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.499+0000 7f189f000640 1 -- 172.21.15.86:0/3241144333 >> 172.21.15.86:0/3241144333 conn(0x7f18980b2110 msgr2=0x7f18980b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:49.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.499+0000 7f189f000640 1 -- 172.21.15.86:0/3241144333 shutdown_connections 2024-04-20T21:18:49.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.499+0000 7f189f000640 1 -- 172.21.15.86:0/3241144333 wait complete. 2024-04-20T21:18:49.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.502+0000 7f189f000640 1 Processor -- start 2024-04-20T21:18:49.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.502+0000 7f189f000640 1 -- start start 2024-04-20T21:18:49.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.503+0000 7f189f000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f187c0052e0 0x7f18981f4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:49.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.503+0000 7f189f000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f189814af50 0x7f189814a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:49.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.503+0000 7f189f000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f189814b7c0 0x7f189814bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:49.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.503+0000 7f189f000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f18980cefe0 con 0x7f187c0052e0 2024-04-20T21:18:49.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.503+0000 7f189f000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f18980ced00 con 0x7f189814af50 2024-04-20T21:18:49.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.503+0000 7f189f000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f18980cf500 con 0x7f189814b7c0 2024-04-20T21:18:49.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.503+0000 7f1897e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f187c0052e0 0x7f18981f4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:49.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.503+0000 7f1897e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f187c0052e0 0x7f18981f4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37262/0 (socket says 172.21.15.86:37262) 2024-04-20T21:18:49.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.503+0000 7f1897e00640 1 -- 172.21.15.86:0/4139029735 learned_addr learned my addr 172.21.15.86:0/4139029735 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:49.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.504+0000 7f188f400640 1 --2- 172.21.15.86:0/4139029735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f189814af50 0x7f189814a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:49.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.504+0000 7f189d000640 1 --2- 172.21.15.86:0/4139029735 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f189814b7c0 0x7f189814bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:49.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.504+0000 7f1897e00640 1 -- 172.21.15.86:0/4139029735 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f189814b7c0 msgr2=0x7f189814bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:49.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.504+0000 7f1897e00640 1 --2- 172.21.15.86:0/4139029735 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f189814b7c0 0x7f189814bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:49.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.504+0000 7f1897e00640 1 -- 172.21.15.86:0/4139029735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f189814af50 msgr2=0x7f189814a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:49.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.504+0000 7f1897e00640 1 --2- 172.21.15.86:0/4139029735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f189814af50 0x7f189814a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:49.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.504+0000 7f1897e00640 1 -- 172.21.15.86:0/4139029735 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18981f4aa0 con 0x7f187c0052e0 2024-04-20T21:18:49.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.504+0000 7f1897e00640 1 --2- 172.21.15.86:0/4139029735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f187c0052e0 0x7f18981f4560 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1888011720 tx=0x7f1888011be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:49.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.504+0000 7f1894c00640 1 -- 172.21.15.86:0/4139029735 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188800f960 con 0x7f187c0052e0 2024-04-20T21:18:49.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.504+0000 7f1894c00640 1 -- 172.21.15.86:0/4139029735 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f188800fb00 con 0x7f187c0052e0 2024-04-20T21:18:49.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.504+0000 7f1894c00640 1 -- 172.21.15.86:0/4139029735 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1888015460 con 0x7f187c0052e0 2024-04-20T21:18:49.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.504+0000 7f189f000640 1 -- 172.21.15.86:0/4139029735 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f189814d9e0 con 0x7f187c0052e0 2024-04-20T21:18:49.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.505+0000 7f189f000640 1 -- 172.21.15.86:0/4139029735 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f189814df80 con 0x7f187c0052e0 2024-04-20T21:18:49.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.506+0000 7f189f000640 1 -- 172.21.15.86:0/4139029735 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18980689e0 con 0x7f187c0052e0 2024-04-20T21:18:49.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.507+0000 7f1894c00640 1 -- 172.21.15.86:0/4139029735 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1888015600 con 0x7f187c0052e0 2024-04-20T21:18:49.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.508+0000 7f1894c00640 1 --2- 172.21.15.86:0/4139029735 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f187406c890 0x7f187406ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:49.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.508+0000 7f1894c00640 1 -- 172.21.15.86:0/4139029735 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f188808c5c0 con 0x7f187c0052e0 2024-04-20T21:18:49.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.508+0000 7f188f400640 1 --2- 172.21.15.86:0/4139029735 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f187406c890 0x7f187406ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:49.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.509+0000 7f188f400640 1 --2- 172.21.15.86:0/4139029735 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f187406c890 0x7f187406ed50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1880007fd0 tx=0x7f1880009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:49.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.512+0000 7f1894c00640 1 -- 172.21.15.86:0/4139029735 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f18981f4aa0 con 0x7f187c0052e0 2024-04-20T21:18:49.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.712+0000 7f189f000640 1 -- 172.21.15.86:0/4139029735 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1898150d50 con 0x7f187c0052e0 2024-04-20T21:18:49.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.713+0000 7f1894c00640 1 -- 172.21.15.86:0/4139029735 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f1888019030 con 0x7f187c0052e0 2024-04-20T21:18:49.712 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:18:49.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.715+0000 7f189f000640 1 -- 172.21.15.86:0/4139029735 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f187406c890 msgr2=0x7f187406ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:49.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.715+0000 7f189f000640 1 --2- 172.21.15.86:0/4139029735 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f187406c890 0x7f187406ed50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1880007fd0 tx=0x7f1880009040 comp rx=0 tx=0).stop 2024-04-20T21:18:49.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.715+0000 7f189f000640 1 -- 172.21.15.86:0/4139029735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f187c0052e0 msgr2=0x7f18981f4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:49.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.715+0000 7f189f000640 1 --2- 172.21.15.86:0/4139029735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f187c0052e0 0x7f18981f4560 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1888011720 tx=0x7f1888011be0 comp rx=0 tx=0).stop 2024-04-20T21:18:49.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.716+0000 7f189f000640 1 -- 172.21.15.86:0/4139029735 shutdown_connections 2024-04-20T21:18:49.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.716+0000 7f189f000640 1 --2- 172.21.15.86:0/4139029735 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f187406c890 0x7f187406ed50 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:49.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.716+0000 7f189f000640 1 --2- 172.21.15.86:0/4139029735 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f189814b7c0 0x7f189814bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:49.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.716+0000 7f189f000640 1 --2- 172.21.15.86:0/4139029735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f189814af50 0x7f189814a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:49.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.716+0000 7f189f000640 1 --2- 172.21.15.86:0/4139029735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f187c0052e0 0x7f18981f4560 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:49.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.716+0000 7f189f000640 1 -- 172.21.15.86:0/4139029735 >> 172.21.15.86:0/4139029735 conn(0x7f18980b2110 msgr2=0x7f189806ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:49.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.716+0000 7f189f000640 1 -- 172.21.15.86:0/4139029735 shutdown_connections 2024-04-20T21:18:49.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.716+0000 7f189f000640 1 -- 172.21.15.86:0/4139029735 wait complete. 2024-04-20T21:18:49.734 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:18:49.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.939+0000 7f96b3e00640 1 Processor -- start 2024-04-20T21:18:49.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.940+0000 7f96b3e00640 1 -- start start 2024-04-20T21:18:49.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.941+0000 7f96b3e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96b40d45a0 0x7f96b40d3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:49.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.941+0000 7f96b3e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f96b415ea70 0x7f96b40d4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:49.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.941+0000 7f96b3e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f96b406d570 0x7f96b40d1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:49.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.941+0000 7f96b3e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f96b40cf120 con 0x7f96b406d570 2024-04-20T21:18:49.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.941+0000 7f96b3e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f96b406de70 con 0x7f96b415ea70 2024-04-20T21:18:49.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.941+0000 7f96b3e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f96b406e150 con 0x7f96b40d45a0 2024-04-20T21:18:49.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.941+0000 7f96b3400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f96b406d570 0x7f96b40d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:49.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.941+0000 7f96b3400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f96b406d570 0x7f96b40d1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37270/0 (socket says 172.21.15.86:37270) 2024-04-20T21:18:49.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.941+0000 7f96b3400640 1 -- 172.21.15.86:0/315938575 learned_addr learned my addr 172.21.15.86:0/315938575 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:49.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.942+0000 7f96b2000640 1 --2- 172.21.15.86:0/315938575 >> v2:172.21.15.144:3300/0 conn(0x7f96b415ea70 0x7f96b40d4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:49.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.942+0000 7f96b2a00640 1 --2- 172.21.15.86:0/315938575 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96b40d45a0 0x7f96b40d3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:49.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.942+0000 7f96b3400640 1 -- 172.21.15.86:0/315938575 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96b40d45a0 msgr2=0x7f96b40d3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:49.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.942+0000 7f96b3400640 1 --2- 172.21.15.86:0/315938575 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96b40d45a0 0x7f96b40d3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:49.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.942+0000 7f96b3400640 1 -- 172.21.15.86:0/315938575 >> v2:172.21.15.144:3300/0 conn(0x7f96b415ea70 msgr2=0x7f96b40d4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:49.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.942+0000 7f96b3400640 1 --2- 172.21.15.86:0/315938575 >> v2:172.21.15.144:3300/0 conn(0x7f96b415ea70 0x7f96b40d4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:49.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.942+0000 7f96b3400640 1 -- 172.21.15.86:0/315938575 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b406e430 con 0x7f96b406d570 2024-04-20T21:18:49.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.943+0000 7f96b3400640 1 --2- 172.21.15.86:0/315938575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b406d570 0x7f96b40d1100 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f96b40d2200 tx=0x7f96ac02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=948ef9301ad9fbcd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:49.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.943+0000 7f96b1600640 1 -- 172.21.15.86:0/315938575 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ac02e070 con 0x7f96b406d570 2024-04-20T21:18:49.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.944+0000 7f96b1600640 1 -- 172.21.15.86:0/315938575 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96ac037070 con 0x7f96b406d570 2024-04-20T21:18:49.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.944+0000 7f96b1600640 1 -- 172.21.15.86:0/315938575 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ac031a00 con 0x7f96b406d570 2024-04-20T21:18:49.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.944+0000 7f96b3e00640 1 -- 172.21.15.86:0/315938575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b406d570 msgr2=0x7f96b40d1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:49.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.944+0000 7f96b3e00640 1 --2- 172.21.15.86:0/315938575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b406d570 0x7f96b40d1100 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f96b40d2200 tx=0x7f96ac02b880 comp rx=0 tx=0).stop 2024-04-20T21:18:49.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.944+0000 7f96b3e00640 1 -- 172.21.15.86:0/315938575 shutdown_connections 2024-04-20T21:18:49.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.944+0000 7f96b3e00640 1 --2- 172.21.15.86:0/315938575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b406d570 0x7f96b40d1100 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:49.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.944+0000 7f96b3e00640 1 --2- 172.21.15.86:0/315938575 >> v2:172.21.15.144:3300/0 conn(0x7f96b415ea70 0x7f96b40d4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:49.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.944+0000 7f96b3e00640 1 --2- 172.21.15.86:0/315938575 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96b40d45a0 0x7f96b40d3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:49.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.944+0000 7f96b3e00640 1 -- 172.21.15.86:0/315938575 >> 172.21.15.86:0/315938575 conn(0x7f96b409d9d0 msgr2=0x7f96b408bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:49.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.944+0000 7f96b3e00640 1 -- 172.21.15.86:0/315938575 shutdown_connections 2024-04-20T21:18:49.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.944+0000 7f96b3e00640 1 -- 172.21.15.86:0/315938575 wait complete. 2024-04-20T21:18:49.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.946+0000 7f96b3e00640 1 Processor -- start 2024-04-20T21:18:49.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.946+0000 7f96b3e00640 1 -- start start 2024-04-20T21:18:49.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.947+0000 7f96b3e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96b406d570 0x7f96b40ca5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:49.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.947+0000 7f96b3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b40d45a0 0x7f96b40ce0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:49.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.947+0000 7f96b3e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96b415ea70 0x7f96b40ce630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:49.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.947+0000 7f96b3e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f96b406da00 con 0x7f96b40d45a0 2024-04-20T21:18:49.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.947+0000 7f96b3e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f96b40d4ec0 con 0x7f96b415ea70 2024-04-20T21:18:49.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.947+0000 7f96b3e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f96b406db80 con 0x7f96b406d570 2024-04-20T21:18:49.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.948+0000 7f96b2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b40d45a0 0x7f96b40ce0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:49.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.948+0000 7f96b2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b40d45a0 0x7f96b40ce0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37280/0 (socket says 172.21.15.86:37280) 2024-04-20T21:18:49.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.948+0000 7f96b2000640 1 -- 172.21.15.86:0/2005417910 learned_addr learned my addr 172.21.15.86:0/2005417910 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:49.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.948+0000 7f96b2a00640 1 --2- 172.21.15.86:0/2005417910 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96b406d570 0x7f96b40ca5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:49.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.948+0000 7f96b3400640 1 --2- 172.21.15.86:0/2005417910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96b415ea70 0x7f96b40ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:49.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.948+0000 7f96b2000640 1 -- 172.21.15.86:0/2005417910 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96b406d570 msgr2=0x7f96b40ca5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:49.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.948+0000 7f96b2000640 1 --2- 172.21.15.86:0/2005417910 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96b406d570 0x7f96b40ca5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:49.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.948+0000 7f96b2000640 1 -- 172.21.15.86:0/2005417910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96b415ea70 msgr2=0x7f96b40ce630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:49.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.948+0000 7f96b2000640 1 --2- 172.21.15.86:0/2005417910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96b415ea70 0x7f96b40ce630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:49.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.948+0000 7f96b2000640 1 -- 172.21.15.86:0/2005417910 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b40b9750 con 0x7f96b40d45a0 2024-04-20T21:18:49.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.948+0000 7f96b2000640 1 --2- 172.21.15.86:0/2005417910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b40d45a0 0x7f96b40ce0f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f969c007e70 tx=0x7f969c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:49.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.949+0000 7f96aaa00640 1 -- 172.21.15.86:0/2005417910 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969c014620 con 0x7f96b40d45a0 2024-04-20T21:18:49.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.949+0000 7f96aaa00640 1 -- 172.21.15.86:0/2005417910 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f969c019070 con 0x7f96b40d45a0 2024-04-20T21:18:49.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.949+0000 7f96b3e00640 1 -- 172.21.15.86:0/2005417910 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96b40ba4a0 con 0x7f96b40d45a0 2024-04-20T21:18:49.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.949+0000 7f96aaa00640 1 -- 172.21.15.86:0/2005417910 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969c014a00 con 0x7f96b40d45a0 2024-04-20T21:18:49.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.949+0000 7f96b3e00640 1 -- 172.21.15.86:0/2005417910 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96b40b9ca0 con 0x7f96b40d45a0 2024-04-20T21:18:49.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.950+0000 7f96b3e00640 1 -- 172.21.15.86:0/2005417910 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96b404fd90 con 0x7f96b40d45a0 2024-04-20T21:18:49.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.951+0000 7f96aaa00640 1 -- 172.21.15.86:0/2005417910 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f969c014ba0 con 0x7f96b40d45a0 2024-04-20T21:18:49.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.952+0000 7f96aaa00640 1 --2- 172.21.15.86:0/2005417910 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f969006c8f0 0x7f969006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:49.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.952+0000 7f96aaa00640 1 -- 172.21.15.86:0/2005417910 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f969c08c410 con 0x7f96b40d45a0 2024-04-20T21:18:49.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.952+0000 7f96b2a00640 1 --2- 172.21.15.86:0/2005417910 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f969006c8f0 0x7f969006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:49.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.953+0000 7f96b2a00640 1 --2- 172.21.15.86:0/2005417910 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f969006c8f0 0x7f969006edb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f96a4006fd0 tx=0x7f96a4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:49.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:49.956+0000 7f96aaa00640 1 -- 172.21.15.86:0/2005417910 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f969c05a7a0 con 0x7f96b40d45a0 2024-04-20T21:18:50.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.153+0000 7f96b3e00640 1 -- 172.21.15.86:0/2005417910 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f96b40d0ee0 con 0x7f96b40d45a0 2024-04-20T21:18:50.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.154+0000 7f96aaa00640 1 -- 172.21.15.86:0/2005417910 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f969c05a0a0 con 0x7f96b40d45a0 2024-04-20T21:18:50.154 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-20T21:18:50.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.157+0000 7f96b3e00640 1 -- 172.21.15.86:0/2005417910 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f969006c8f0 msgr2=0x7f969006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:50.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.157+0000 7f96b3e00640 1 --2- 172.21.15.86:0/2005417910 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f969006c8f0 0x7f969006edb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f96a4006fd0 tx=0x7f96a4008040 comp rx=0 tx=0).stop 2024-04-20T21:18:50.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.157+0000 7f96b3e00640 1 -- 172.21.15.86:0/2005417910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b40d45a0 msgr2=0x7f96b40ce0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:50.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.157+0000 7f96b3e00640 1 --2- 172.21.15.86:0/2005417910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b40d45a0 0x7f96b40ce0f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f969c007e70 tx=0x7f969c00e4b0 comp rx=0 tx=0).stop 2024-04-20T21:18:50.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.157+0000 7f96b3e00640 1 -- 172.21.15.86:0/2005417910 shutdown_connections 2024-04-20T21:18:50.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.157+0000 7f96b3e00640 1 --2- 172.21.15.86:0/2005417910 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f969006c8f0 0x7f969006edb0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:50.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.157+0000 7f96b3e00640 1 --2- 172.21.15.86:0/2005417910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96b415ea70 0x7f96b40ce630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:50.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.157+0000 7f96b3e00640 1 --2- 172.21.15.86:0/2005417910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b40d45a0 0x7f96b40ce0f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:50.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.157+0000 7f96b3e00640 1 --2- 172.21.15.86:0/2005417910 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96b406d570 0x7f96b40ca5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:50.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.157+0000 7f96b3e00640 1 -- 172.21.15.86:0/2005417910 >> 172.21.15.86:0/2005417910 conn(0x7f96b409d9d0 msgr2=0x7f96b40b35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:50.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.157+0000 7f96b3e00640 1 -- 172.21.15.86:0/2005417910 shutdown_connections 2024-04-20T21:18:50.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.157+0000 7f96b3e00640 1 -- 172.21.15.86:0/2005417910 wait complete. 2024-04-20T21:18:50.175 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:18:50.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.367+0000 7f64c4e00640 1 Processor -- start 2024-04-20T21:18:50.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.368+0000 7f64c4e00640 1 -- start start 2024-04-20T21:18:50.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.369+0000 7f64c4e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64c00cf550 0x7f64c00b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:50.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.369+0000 7f64c4e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f64c00b5150 0x7f64c006ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:50.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.369+0000 7f64c4e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64c00ce4f0 con 0x7f64c006b980 2024-04-20T21:18:50.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.369+0000 7f64c4e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f64c00ce2c0 con 0x7f64c00b5150 2024-04-20T21:18:50.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.369+0000 7f64c4e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f64c00ce790 con 0x7f64c00cf550 2024-04-20T21:18:50.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.369+0000 7f64bde00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f64c006b980 0x7f64c006bd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42634/0 (socket says 172.21.15.86:42634) 2024-04-20T21:18:50.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.369+0000 7f64bde00640 1 -- 172.21.15.86:0/2389488978 learned_addr learned my addr 172.21.15.86:0/2389488978 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:18:50.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.370+0000 7f64bd400640 1 --2- 172.21.15.86:0/2389488978 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64c00cf550 0x7f64c00b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:50.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.370+0000 7f64be800640 1 --2- 172.21.15.86:0/2389488978 >> v2:172.21.15.144:3300/0 conn(0x7f64c00b5150 0x7f64c006ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:50.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.370+0000 7f64bca00640 1 -- 172.21.15.86:0/2389488978 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4000857888 0 0) 0x7f64c00ce4f0 con 0x7f64c006b980 2024-04-20T21:18:50.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.370+0000 7f64bca00640 1 -- 172.21.15.86:0/2389488978 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f64a4003520 con 0x7f64c006b980 2024-04-20T21:18:50.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.370+0000 7f64bca00640 1 -- 172.21.15.86:0/2389488978 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3866541688 0 0) 0x7f64a4003520 con 0x7f64c006b980 2024-04-20T21:18:50.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.370+0000 7f64bca00640 1 -- 172.21.15.86:0/2389488978 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64c00cf550 msgr2=0x7f64c00b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:50.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.370+0000 7f64bca00640 1 --2- 172.21.15.86:0/2389488978 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64c00cf550 0x7f64c00b4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:50.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.370+0000 7f64bca00640 1 -- 172.21.15.86:0/2389488978 >> v2:172.21.15.144:3300/0 conn(0x7f64c00b5150 msgr2=0x7f64c006ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:50.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.370+0000 7f64bca00640 1 --2- 172.21.15.86:0/2389488978 >> v2:172.21.15.144:3300/0 conn(0x7f64c00b5150 0x7f64c006ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:50.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.370+0000 7f64bca00640 1 -- 172.21.15.86:0/2389488978 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64c00cea70 con 0x7f64c006b980 2024-04-20T21:18:50.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.370+0000 7f64bca00640 1 -- 172.21.15.86:0/2389488978 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f64b0001670 con 0x7f64c006b980 2024-04-20T21:18:50.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.370+0000 7f64bca00640 1 -- 172.21.15.86:0/2389488978 >> v1:172.21.15.86:6789/0 conn(0x7f64c006b980 legacy=0x7f64c006bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:50.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.370+0000 7f64bca00640 1 --2- 172.21.15.86:0/2389488978 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64c00cf550 0x7f64a40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:50.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.370+0000 7f64bca00640 1 --2- 172.21.15.86:0/2389488978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64a4003f30 0x7f64a4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:50.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.370+0000 7f64bca00640 1 --2- 172.21.15.86:0/2389488978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64a4004d40 0x7f64a40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:50.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.370+0000 7f64bca00640 1 -- 172.21.15.86:0/2389488978 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f64c00ce790 con 0x7f64a4004d40 2024-04-20T21:18:50.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.370+0000 7f64bca00640 1 -- 172.21.15.86:0/2389488978 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f64a400b9c0 con 0x7f64a4003f30 2024-04-20T21:18:50.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.370+0000 7f64bca00640 1 -- 172.21.15.86:0/2389488978 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f64a400bb90 con 0x7f64c00cf550 2024-04-20T21:18:50.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.371+0000 7f64be800640 1 --2- 172.21.15.86:0/2389488978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64a4004d40 0x7f64a40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:50.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.371+0000 7f64bd400640 1 --2- 172.21.15.86:0/2389488978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64a4003f30 0x7f64a4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:50.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.371+0000 7f64bde00640 1 --2- 172.21.15.86:0/2389488978 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64c00cf550 0x7f64a40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:50.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.371+0000 7f64be800640 1 -- 172.21.15.86:0/2389488978 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64c00cf550 msgr2=0x7f64a40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:50.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.371+0000 7f64be800640 1 --2- 172.21.15.86:0/2389488978 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64c00cf550 0x7f64a40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:50.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.371+0000 7f64be800640 1 -- 172.21.15.86:0/2389488978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64a4003f30 msgr2=0x7f64a4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:50.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.371+0000 7f64be800640 1 --2- 172.21.15.86:0/2389488978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64a4003f30 0x7f64a4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:50.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.371+0000 7f64be800640 1 -- 172.21.15.86:0/2389488978 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64a4003520 con 0x7f64a4004d40 2024-04-20T21:18:50.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.372+0000 7f64be800640 1 --2- 172.21.15.86:0/2389488978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64a4004d40 0x7f64a40051c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f64b8009fc0 tx=0x7f64a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30826356be86f135 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:50.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.373+0000 7f64bca00640 1 -- 172.21.15.86:0/2389488978 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b8039070 con 0x7f64a4004d40 2024-04-20T21:18:50.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.373+0000 7f64bca00640 1 -- 172.21.15.86:0/2389488978 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64b802cc60 con 0x7f64a4004d40 2024-04-20T21:18:50.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.373+0000 7f64bca00640 1 -- 172.21.15.86:0/2389488978 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b8009c10 con 0x7f64a4004d40 2024-04-20T21:18:50.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.374+0000 7f64c4e00640 1 -- 172.21.15.86:0/2389488978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64a4004d40 msgr2=0x7f64a40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:50.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.374+0000 7f64c4e00640 1 --2- 172.21.15.86:0/2389488978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64a4004d40 0x7f64a40051c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f64b8009fc0 tx=0x7f64a4005830 comp rx=0 tx=0).stop 2024-04-20T21:18:50.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.374+0000 7f64be800640 1 -- 172.21.15.86:0/2389488978 reap_dead start 2024-04-20T21:18:50.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.374+0000 7f64c4e00640 1 -- 172.21.15.86:0/2389488978 shutdown_connections 2024-04-20T21:18:50.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.374+0000 7f64c4e00640 1 -- 172.21.15.86:0/2389488978 >> 172.21.15.86:0/2389488978 conn(0x7f64c00b2110 msgr2=0x7f64c00b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:50.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.374+0000 7f64c4e00640 1 -- 172.21.15.86:0/2389488978 shutdown_connections 2024-04-20T21:18:50.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.374+0000 7f64c4e00640 1 -- 172.21.15.86:0/2389488978 wait complete. 2024-04-20T21:18:50.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.376+0000 7f64c4e00640 1 Processor -- start 2024-04-20T21:18:50.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.376+0000 7f64c4e00640 1 -- start start 2024-04-20T21:18:50.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.377+0000 7f64c4e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64a4004d40 0x7f64c015c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:50.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.377+0000 7f64c4e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64c015e5f0 0x7f64c0072020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:50.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.377+0000 7f64c4e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64c00b0e80 0x7f64c00b1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:50.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.377+0000 7f64c4e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f64c00b55c0 con 0x7f64c00b0e80 2024-04-20T21:18:50.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.377+0000 7f64c4e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f64c00cf030 con 0x7f64a4004d40 2024-04-20T21:18:50.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.377+0000 7f64c4e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f64c0152a00 con 0x7f64c015e5f0 2024-04-20T21:18:50.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.377+0000 7f64be800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64c00b0e80 0x7f64c00b1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:50.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.377+0000 7f64be800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64c00b0e80 0x7f64c00b1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37292/0 (socket says 172.21.15.86:37292) 2024-04-20T21:18:50.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.377+0000 7f64be800640 1 -- 172.21.15.86:0/3799093639 learned_addr learned my addr 172.21.15.86:0/3799093639 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:50.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.377+0000 7f64bd400640 1 --2- 172.21.15.86:0/3799093639 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64c015e5f0 0x7f64c0072020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:50.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.377+0000 7f64bde00640 1 --2- 172.21.15.86:0/3799093639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64a4004d40 0x7f64c015c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:50.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.377+0000 7f64be800640 1 -- 172.21.15.86:0/3799093639 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64c015e5f0 msgr2=0x7f64c0072020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:50.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.377+0000 7f64be800640 1 --2- 172.21.15.86:0/3799093639 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64c015e5f0 0x7f64c0072020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:50.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.378+0000 7f64be800640 1 -- 172.21.15.86:0/3799093639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64a4004d40 msgr2=0x7f64c015c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:50.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.378+0000 7f64be800640 1 --2- 172.21.15.86:0/3799093639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64a4004d40 0x7f64c015c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:50.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.378+0000 7f64be800640 1 -- 172.21.15.86:0/3799093639 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64c015d200 con 0x7f64c00b0e80 2024-04-20T21:18:50.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.378+0000 7f64be800640 1 --2- 172.21.15.86:0/3799093639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64c00b0e80 0x7f64c00b1330 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f64b8002230 tx=0x7f64b8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:50.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.378+0000 7f64b6000640 1 -- 172.21.15.86:0/3799093639 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b8044020 con 0x7f64c00b0e80 2024-04-20T21:18:50.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.378+0000 7f64b6000640 1 -- 172.21.15.86:0/3799093639 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64b8035d70 con 0x7f64c00b0e80 2024-04-20T21:18:50.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.378+0000 7f64b6000640 1 -- 172.21.15.86:0/3799093639 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b8039040 con 0x7f64c00b0e80 2024-04-20T21:18:50.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.378+0000 7f64c4e00640 1 -- 172.21.15.86:0/3799093639 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64c015d4e0 con 0x7f64c00b0e80 2024-04-20T21:18:50.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.378+0000 7f64c4e00640 1 -- 172.21.15.86:0/3799093639 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64c015daa0 con 0x7f64c00b0e80 2024-04-20T21:18:50.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.380+0000 7f64c4e00640 1 -- 172.21.15.86:0/3799093639 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64c00689e0 con 0x7f64c00b0e80 2024-04-20T21:18:50.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.380+0000 7f64b6000640 1 -- 172.21.15.86:0/3799093639 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f64b8005d10 con 0x7f64c00b0e80 2024-04-20T21:18:50.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.382+0000 7f64b6000640 1 --2- 172.21.15.86:0/3799093639 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f649c06c8f0 0x7f649c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:50.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.382+0000 7f64b6000640 1 -- 172.21.15.86:0/3799093639 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f64b80ade10 con 0x7f64c00b0e80 2024-04-20T21:18:50.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.382+0000 7f64bde00640 1 --2- 172.21.15.86:0/3799093639 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f649c06c8f0 0x7f649c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:50.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.383+0000 7f64bde00640 1 --2- 172.21.15.86:0/3799093639 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f649c06c8f0 0x7f649c06edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f64b00058d0 tx=0x7f64b0001dd0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:50.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.386+0000 7f64b6000640 1 -- 172.21.15.86:0/3799093639 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f64b804e020 con 0x7f64c00b0e80 2024-04-20T21:18:50.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.583+0000 7f64c4e00640 1 -- 172.21.15.86:0/3799093639 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f64c006f430 con 0x7f64c00b0e80 2024-04-20T21:18:50.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.584+0000 7f64b6000640 1 -- 172.21.15.86:0/3799093639 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f64b807c230 con 0x7f64c00b0e80 2024-04-20T21:18:50.584 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-20T21:18:50.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.586+0000 7f64c4e00640 1 -- 172.21.15.86:0/3799093639 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f649c06c8f0 msgr2=0x7f649c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:50.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.586+0000 7f64c4e00640 1 --2- 172.21.15.86:0/3799093639 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f649c06c8f0 0x7f649c06edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f64b00058d0 tx=0x7f64b0001dd0 comp rx=0 tx=0).stop 2024-04-20T21:18:50.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.586+0000 7f64c4e00640 1 -- 172.21.15.86:0/3799093639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64c00b0e80 msgr2=0x7f64c00b1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:50.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.586+0000 7f64c4e00640 1 --2- 172.21.15.86:0/3799093639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64c00b0e80 0x7f64c00b1330 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f64b8002230 tx=0x7f64b8009750 comp rx=0 tx=0).stop 2024-04-20T21:18:50.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.587+0000 7f64c4e00640 1 -- 172.21.15.86:0/3799093639 shutdown_connections 2024-04-20T21:18:50.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.587+0000 7f64c4e00640 1 --2- 172.21.15.86:0/3799093639 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f649c06c8f0 0x7f649c06edb0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:50.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.587+0000 7f64c4e00640 1 --2- 172.21.15.86:0/3799093639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64c00b0e80 0x7f64c00b1330 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:50.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.587+0000 7f64c4e00640 1 --2- 172.21.15.86:0/3799093639 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64c015e5f0 0x7f64c0072020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:50.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.587+0000 7f64c4e00640 1 --2- 172.21.15.86:0/3799093639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64a4004d40 0x7f64c015c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:50.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.587+0000 7f64c4e00640 1 -- 172.21.15.86:0/3799093639 >> 172.21.15.86:0/3799093639 conn(0x7f64c00b2110 msgr2=0x7f64c0145ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:50.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.587+0000 7f64c4e00640 1 -- 172.21.15.86:0/3799093639 shutdown_connections 2024-04-20T21:18:50.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.587+0000 7f64c4e00640 1 -- 172.21.15.86:0/3799093639 wait complete. 2024-04-20T21:18:50.604 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T21:18:50.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.798+0000 7f9e3e800640 1 Processor -- start 2024-04-20T21:18:50.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.798+0000 7f9e3e800640 1 -- start start 2024-04-20T21:18:50.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.799+0000 7f9e3e800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9e38073540 0x7f9e380d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:50.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.799+0000 7f9e3e800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e380af870 0x7f9e380d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:50.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.799+0000 7f9e3e800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e380ce1c0 con 0x7f9e380cfa00 2024-04-20T21:18:50.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.799+0000 7f9e3e800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9e380ce4a0 con 0x7f9e38073540 2024-04-20T21:18:50.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.799+0000 7f9e3e800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9e380ce780 con 0x7f9e380af870 2024-04-20T21:18:50.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.799+0000 7f9e3d400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9e38073540 0x7f9e380d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:50.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.799+0000 7f9e3d400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9e38073540 0x7f9e380d00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37294/0 (socket says 172.21.15.86:37294) 2024-04-20T21:18:50.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.799+0000 7f9e3d400640 1 -- 172.21.15.86:0/392778207 learned_addr learned my addr 172.21.15.86:0/392778207 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:50.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.800+0000 7f9e3ca00640 1 --2- 172.21.15.86:0/392778207 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e380af870 0x7f9e380d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:50.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.800+0000 7f9e3d400640 1 -- 172.21.15.86:0/392778207 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e380af870 msgr2=0x7f9e380d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:50.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.800+0000 7f9e3d400640 1 --2- 172.21.15.86:0/392778207 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e380af870 0x7f9e380d05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:50.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.800+0000 7f9e3d400640 1 -- 172.21.15.86:0/392778207 >> v1:172.21.15.144:6789/0 conn(0x7f9e380cfa00 legacy=0x7f9e3815f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:50.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.800+0000 7f9e3d400640 1 -- 172.21.15.86:0/392778207 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e380cea60 con 0x7f9e38073540 2024-04-20T21:18:50.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.800+0000 7f9e3ca00640 1 --2- 172.21.15.86:0/392778207 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e380af870 0x7f9e380d05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:50.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.801+0000 7f9e3d400640 1 --2- 172.21.15.86:0/392778207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e38073540 0x7f9e380d00b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c009870 tx=0x7f9e2c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=305f8221db33b8bf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:50.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.801+0000 7f9e37e00640 1 -- 172.21.15.86:0/392778207 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c035020 con 0x7f9e38073540 2024-04-20T21:18:50.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.801+0000 7f9e37e00640 1 -- 172.21.15.86:0/392778207 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e2c031640 con 0x7f9e38073540 2024-04-20T21:18:50.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.801+0000 7f9e37e00640 1 -- 172.21.15.86:0/392778207 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c02d040 con 0x7f9e38073540 2024-04-20T21:18:50.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.802+0000 7f9e3e800640 1 -- 172.21.15.86:0/392778207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e38073540 msgr2=0x7f9e380d00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:50.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.802+0000 7f9e3e800640 1 --2- 172.21.15.86:0/392778207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e38073540 0x7f9e380d00b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c009870 tx=0x7f9e2c02b8f0 comp rx=0 tx=0).stop 2024-04-20T21:18:50.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.802+0000 7f9e3e800640 1 -- 172.21.15.86:0/392778207 shutdown_connections 2024-04-20T21:18:50.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.802+0000 7f9e3e800640 1 --2- 172.21.15.86:0/392778207 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e380af870 0x7f9e380d05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:50.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.802+0000 7f9e3e800640 1 --2- 172.21.15.86:0/392778207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e38073540 0x7f9e380d00b0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:50.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.802+0000 7f9e3e800640 1 -- 172.21.15.86:0/392778207 >> 172.21.15.86:0/392778207 conn(0x7f9e3806bb60 msgr2=0x7f9e3806bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:50.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.802+0000 7f9e3e800640 1 -- 172.21.15.86:0/392778207 shutdown_connections 2024-04-20T21:18:50.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.802+0000 7f9e3e800640 1 -- 172.21.15.86:0/392778207 wait complete. 2024-04-20T21:18:50.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.803+0000 7f9e3e800640 1 Processor -- start 2024-04-20T21:18:50.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.804+0000 7f9e3e800640 1 -- start start 2024-04-20T21:18:50.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.804+0000 7f9e3e800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e38073540 0x7f9e381f22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:50.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.804+0000 7f9e3e800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e380cfa00 0x7f9e3815c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:50.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.804+0000 7f9e3e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e3815caa0 0x7f9e380c7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:50.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.804+0000 7f9e3e800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9e38072b80 con 0x7f9e3815caa0 2024-04-20T21:18:50.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.804+0000 7f9e3e800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9e38073920 con 0x7f9e380cfa00 2024-04-20T21:18:50.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.804+0000 7f9e3e800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9e38099830 con 0x7f9e38073540 2024-04-20T21:18:50.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.805+0000 7f9e3de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e3815caa0 0x7f9e380c7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:50.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.805+0000 7f9e3de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e3815caa0 0x7f9e380c7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37302/0 (socket says 172.21.15.86:37302) 2024-04-20T21:18:50.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.805+0000 7f9e3de00640 1 -- 172.21.15.86:0/2448586228 learned_addr learned my addr 172.21.15.86:0/2448586228 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:50.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.805+0000 7f9e3ca00640 1 --2- 172.21.15.86:0/2448586228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e380cfa00 0x7f9e3815c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:50.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.805+0000 7f9e3d400640 1 --2- 172.21.15.86:0/2448586228 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e38073540 0x7f9e381f22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:50.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.805+0000 7f9e3de00640 1 -- 172.21.15.86:0/2448586228 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e38073540 msgr2=0x7f9e381f22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:50.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.805+0000 7f9e3de00640 1 --2- 172.21.15.86:0/2448586228 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e38073540 0x7f9e381f22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:50.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.805+0000 7f9e3de00640 1 -- 172.21.15.86:0/2448586228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e380cfa00 msgr2=0x7f9e3815c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:50.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.805+0000 7f9e3de00640 1 --2- 172.21.15.86:0/2448586228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e380cfa00 0x7f9e3815c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:50.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.805+0000 7f9e3de00640 1 -- 172.21.15.86:0/2448586228 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e380c7290 con 0x7f9e3815caa0 2024-04-20T21:18:50.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.806+0000 7f9e3de00640 1 --2- 172.21.15.86:0/2448586228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e3815caa0 0x7f9e380c7a70 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9e3000b410 tx=0x7f9e3000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:50.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.806+0000 7f9e35600640 1 -- 172.21.15.86:0/2448586228 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e30015030 con 0x7f9e3815caa0 2024-04-20T21:18:50.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.806+0000 7f9e35600640 1 -- 172.21.15.86:0/2448586228 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e30004020 con 0x7f9e3815caa0 2024-04-20T21:18:50.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.806+0000 7f9e3e800640 1 -- 172.21.15.86:0/2448586228 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e3815d010 con 0x7f9e3815caa0 2024-04-20T21:18:50.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.806+0000 7f9e35600640 1 -- 172.21.15.86:0/2448586228 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e30004410 con 0x7f9e3815caa0 2024-04-20T21:18:50.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.806+0000 7f9e3e800640 1 -- 172.21.15.86:0/2448586228 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e380c7550 con 0x7f9e3815caa0 2024-04-20T21:18:50.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.807+0000 7f9e3e800640 1 -- 172.21.15.86:0/2448586228 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e380c7750 con 0x7f9e3815caa0 2024-04-20T21:18:50.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.808+0000 7f9e35600640 1 -- 172.21.15.86:0/2448586228 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9e30018070 con 0x7f9e3815caa0 2024-04-20T21:18:50.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.809+0000 7f9e35600640 1 --2- 172.21.15.86:0/2448586228 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9e1c06c920 0x7f9e1c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:50.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.809+0000 7f9e35600640 1 -- 172.21.15.86:0/2448586228 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f9e3008b740 con 0x7f9e3815caa0 2024-04-20T21:18:50.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.809+0000 7f9e3d400640 1 --2- 172.21.15.86:0/2448586228 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9e1c06c920 0x7f9e1c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:50.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.810+0000 7f9e3d400640 1 --2- 172.21.15.86:0/2448586228 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9e1c06c920 0x7f9e1c06ede0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c036000 tx=0x7f9e2c03c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:50.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:50.813+0000 7f9e35600640 1 -- 172.21.15.86:0/2448586228 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9e380c7750 con 0x7f9e3815caa0 2024-04-20T21:18:51.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.013+0000 7f9e3e800640 1 -- 172.21.15.86:0/2448586228 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9e3804fcf0 con 0x7f9e3815caa0 2024-04-20T21:18:51.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.014+0000 7f9e35600640 1 -- 172.21.15.86:0/2448586228 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f9e300599c0 con 0x7f9e3815caa0 2024-04-20T21:18:51.013 INFO:teuthology.orchestra.run.smithi086.stderr:listed 0 entries 2024-04-20T21:18:51.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.016+0000 7f9e3e800640 1 -- 172.21.15.86:0/2448586228 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9e1c06c920 msgr2=0x7f9e1c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:51.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.016+0000 7f9e3e800640 1 --2- 172.21.15.86:0/2448586228 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9e1c06c920 0x7f9e1c06ede0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c036000 tx=0x7f9e2c03c040 comp rx=0 tx=0).stop 2024-04-20T21:18:51.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.016+0000 7f9e3e800640 1 -- 172.21.15.86:0/2448586228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e3815caa0 msgr2=0x7f9e380c7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:51.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.016+0000 7f9e3e800640 1 --2- 172.21.15.86:0/2448586228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e3815caa0 0x7f9e380c7a70 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9e3000b410 tx=0x7f9e3000b8d0 comp rx=0 tx=0).stop 2024-04-20T21:18:51.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.016+0000 7f9e3e800640 1 -- 172.21.15.86:0/2448586228 shutdown_connections 2024-04-20T21:18:51.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.016+0000 7f9e3e800640 1 --2- 172.21.15.86:0/2448586228 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9e1c06c920 0x7f9e1c06ede0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:51.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.016+0000 7f9e3e800640 1 --2- 172.21.15.86:0/2448586228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e3815caa0 0x7f9e380c7a70 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:51.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.016+0000 7f9e3e800640 1 --2- 172.21.15.86:0/2448586228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e380cfa00 0x7f9e3815c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:51.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.016+0000 7f9e3e800640 1 --2- 172.21.15.86:0/2448586228 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e38073540 0x7f9e381f22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:51.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.016+0000 7f9e3e800640 1 -- 172.21.15.86:0/2448586228 >> 172.21.15.86:0/2448586228 conn(0x7f9e3806bb60 msgr2=0x7f9e380cb150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:51.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.017+0000 7f9e3e800640 1 -- 172.21.15.86:0/2448586228 shutdown_connections 2024-04-20T21:18:51.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.017+0000 7f9e3e800640 1 -- 172.21.15.86:0/2448586228 wait complete. 2024-04-20T21:18:51.034 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:18:51.034 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:18:51.092 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:18:51.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.292+0000 7ff432800640 1 Processor -- start 2024-04-20T21:18:51.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.292+0000 7ff432800640 1 -- start start 2024-04-20T21:18:51.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.294+0000 7ff432800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff42c156ae0 0x7ff42c15d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:51.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.294+0000 7ff432800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff42c0ad7f0 0x7ff42c06e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:51.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.294+0000 7ff432800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff42c160890 con 0x7ff42c160080 2024-04-20T21:18:51.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.294+0000 7ff432800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff42c06db20 con 0x7ff42c0ad7f0 2024-04-20T21:18:51.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.294+0000 7ff432800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7ff42c160b70 con 0x7ff42c156ae0 2024-04-20T21:18:51.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.294+0000 7ff431e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff42c0ad7f0 0x7ff42c06e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:51.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.294+0000 7ff431e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff42c0ad7f0 0x7ff42c06e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37316/0 (socket says 172.21.15.86:37316) 2024-04-20T21:18:51.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.294+0000 7ff431e00640 1 -- 172.21.15.86:0/3267993104 learned_addr learned my addr 172.21.15.86:0/3267993104 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:51.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.294+0000 7ff430a00640 1 --1- 172.21.15.86:0/3267993104 >> v1:172.21.15.86:6789/0 conn(0x7ff42c160080 0x7ff42c0ad0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42640/0 (socket says 172.21.15.86:42640) 2024-04-20T21:18:51.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.295+0000 7ff431400640 1 --2- 172.21.15.86:0/3267993104 >> v2:172.21.15.144:3300/0 conn(0x7ff42c156ae0 0x7ff42c15d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:51.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.295+0000 7ff431e00640 1 -- 172.21.15.86:0/3267993104 >> v2:172.21.15.144:3300/0 conn(0x7ff42c156ae0 msgr2=0x7ff42c15d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:51.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.295+0000 7ff431e00640 1 --2- 172.21.15.86:0/3267993104 >> v2:172.21.15.144:3300/0 conn(0x7ff42c156ae0 0x7ff42c15d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:51.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.295+0000 7ff431e00640 1 -- 172.21.15.86:0/3267993104 >> v1:172.21.15.86:6789/0 conn(0x7ff42c160080 legacy=0x7ff42c0ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:51.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.295+0000 7ff431e00640 1 -- 172.21.15.86:0/3267993104 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff42c160e50 con 0x7ff42c0ad7f0 2024-04-20T21:18:51.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.295+0000 7ff431400640 1 --2- 172.21.15.86:0/3267993104 >> v2:172.21.15.144:3300/0 conn(0x7ff42c156ae0 0x7ff42c15d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:51.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.295+0000 7ff42be00640 1 -- 172.21.15.86:0/3267993104 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1996461957 0 0) 0x7ff42c160890 con 0x7ff42c160080 2024-04-20T21:18:51.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.296+0000 7ff431e00640 1 --2- 172.21.15.86:0/3267993104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff42c0ad7f0 0x7ff42c06e290 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff42c06d460 tx=0x7ff42402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40314e43c94e7274 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:51.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.296+0000 7ff42be00640 1 -- 172.21.15.86:0/3267993104 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42402cb20 con 0x7ff42c0ad7f0 2024-04-20T21:18:51.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.296+0000 7ff42be00640 1 -- 172.21.15.86:0/3267993104 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff42402ccc0 con 0x7ff42c0ad7f0 2024-04-20T21:18:51.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.296+0000 7ff42be00640 1 -- 172.21.15.86:0/3267993104 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff424034610 con 0x7ff42c0ad7f0 2024-04-20T21:18:51.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.297+0000 7ff432800640 1 -- 172.21.15.86:0/3267993104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff42c0ad7f0 msgr2=0x7ff42c06e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:51.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.297+0000 7ff432800640 1 --2- 172.21.15.86:0/3267993104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff42c0ad7f0 0x7ff42c06e290 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff42c06d460 tx=0x7ff42402bf80 comp rx=0 tx=0).stop 2024-04-20T21:18:51.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.297+0000 7ff432800640 1 -- 172.21.15.86:0/3267993104 shutdown_connections 2024-04-20T21:18:51.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.297+0000 7ff432800640 1 --2- 172.21.15.86:0/3267993104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff42c0ad7f0 0x7ff42c06e290 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:51.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.297+0000 7ff432800640 1 --2- 172.21.15.86:0/3267993104 >> v2:172.21.15.144:3300/0 conn(0x7ff42c156ae0 0x7ff42c15d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:51.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.297+0000 7ff432800640 1 -- 172.21.15.86:0/3267993104 >> 172.21.15.86:0/3267993104 conn(0x7ff42c0d0170 msgr2=0x7ff42c0d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:51.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.297+0000 7ff432800640 1 -- 172.21.15.86:0/3267993104 shutdown_connections 2024-04-20T21:18:51.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.297+0000 7ff432800640 1 -- 172.21.15.86:0/3267993104 wait complete. 2024-04-20T21:18:51.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.298+0000 7ff432800640 1 Processor -- start 2024-04-20T21:18:51.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.299+0000 7ff432800640 1 -- start start 2024-04-20T21:18:51.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.299+0000 7ff432800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff42c0ad7f0 0x7ff42c099a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:51.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.299+0000 7ff432800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff42c156ae0 0x7ff42c1599b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:51.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.299+0000 7ff432800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff42c160080 0x7ff42c144f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:51.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.299+0000 7ff432800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff42c160500 con 0x7ff42c156ae0 2024-04-20T21:18:51.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.299+0000 7ff432800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff42c156ec0 con 0x7ff42c0ad7f0 2024-04-20T21:18:51.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.299+0000 7ff432800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff42c06e7d0 con 0x7ff42c160080 2024-04-20T21:18:51.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.300+0000 7ff430a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff42c156ae0 0x7ff42c1599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:51.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.300+0000 7ff430a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff42c156ae0 0x7ff42c1599b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37318/0 (socket says 172.21.15.86:37318) 2024-04-20T21:18:51.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.300+0000 7ff430a00640 1 -- 172.21.15.86:0/2602326843 learned_addr learned my addr 172.21.15.86:0/2602326843 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:51.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.300+0000 7ff430a00640 1 -- 172.21.15.86:0/2602326843 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff42c160080 msgr2=0x7ff42c144f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:51.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.300+0000 7ff430a00640 1 --2- 172.21.15.86:0/2602326843 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff42c160080 0x7ff42c144f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:51.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.300+0000 7ff430a00640 1 -- 172.21.15.86:0/2602326843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff42c0ad7f0 msgr2=0x7ff42c099a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:51.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.300+0000 7ff431400640 1 --2- 172.21.15.86:0/2602326843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff42c0ad7f0 0x7ff42c099a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:51.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.300+0000 7ff430a00640 1 --2- 172.21.15.86:0/2602326843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff42c0ad7f0 0x7ff42c099a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:51.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.300+0000 7ff430a00640 1 -- 172.21.15.86:0/2602326843 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff42c159ef0 con 0x7ff42c156ae0 2024-04-20T21:18:51.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.300+0000 7ff430a00640 1 --2- 172.21.15.86:0/2602326843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff42c156ae0 0x7ff42c1599b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff41800c430 tx=0x7ff41800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:51.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.301+0000 7ff429600640 1 -- 172.21.15.86:0/2602326843 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4180074e0 con 0x7ff42c156ae0 2024-04-20T21:18:51.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.301+0000 7ff429600640 1 -- 172.21.15.86:0/2602326843 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4180047e0 con 0x7ff42c156ae0 2024-04-20T21:18:51.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.301+0000 7ff429600640 1 -- 172.21.15.86:0/2602326843 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff418004bd0 con 0x7ff42c156ae0 2024-04-20T21:18:51.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.301+0000 7ff432800640 1 -- 172.21.15.86:0/2602326843 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff42c158910 con 0x7ff42c156ae0 2024-04-20T21:18:51.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.301+0000 7ff432800640 1 -- 172.21.15.86:0/2602326843 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff42c158ed0 con 0x7ff42c156ae0 2024-04-20T21:18:51.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.302+0000 7ff432800640 1 -- 172.21.15.86:0/2602326843 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff42c0694b0 con 0x7ff42c156ae0 2024-04-20T21:18:51.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.303+0000 7ff429600640 1 -- 172.21.15.86:0/2602326843 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff418011070 con 0x7ff42c156ae0 2024-04-20T21:18:51.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.304+0000 7ff429600640 1 --2- 172.21.15.86:0/2602326843 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff40c06c8f0 0x7ff40c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:51.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.304+0000 7ff429600640 1 -- 172.21.15.86:0/2602326843 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7ff41808b800 con 0x7ff42c156ae0 2024-04-20T21:18:51.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.304+0000 7ff431400640 1 --2- 172.21.15.86:0/2602326843 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff40c06c8f0 0x7ff40c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:51.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.305+0000 7ff431400640 1 --2- 172.21.15.86:0/2602326843 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff40c06c8f0 0x7ff40c06edb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff420008850 tx=0x7ff420008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:51.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.308+0000 7ff429600640 1 -- 172.21.15.86:0/2602326843 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff42c0694b0 con 0x7ff42c156ae0 2024-04-20T21:18:51.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.507+0000 7ff432800640 1 -- 172.21.15.86:0/2602326843 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff42c04fcf0 con 0x7ff42c156ae0 2024-04-20T21:18:51.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.509+0000 7ff429600640 1 -- 172.21.15.86:0/2602326843 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16854 (secure 0 0 0) 0x7ff41800e280 con 0x7ff42c156ae0 2024-04-20T21:18:51.509 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:18:51.509 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":22,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:18:33.788155+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:17:49.718799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:17:50.683477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:18:51.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.511+0000 7ff432800640 1 -- 172.21.15.86:0/2602326843 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff40c06c8f0 msgr2=0x7ff40c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:51.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.511+0000 7ff432800640 1 --2- 172.21.15.86:0/2602326843 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff40c06c8f0 0x7ff40c06edb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff420008850 tx=0x7ff420008040 comp rx=0 tx=0).stop 2024-04-20T21:18:51.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.511+0000 7ff432800640 1 -- 172.21.15.86:0/2602326843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff42c156ae0 msgr2=0x7ff42c1599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:51.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.512+0000 7ff432800640 1 --2- 172.21.15.86:0/2602326843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff42c156ae0 0x7ff42c1599b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff41800c430 tx=0x7ff41800c8f0 comp rx=0 tx=0).stop 2024-04-20T21:18:51.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.512+0000 7ff432800640 1 -- 172.21.15.86:0/2602326843 shutdown_connections 2024-04-20T21:18:51.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.512+0000 7ff432800640 1 --2- 172.21.15.86:0/2602326843 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff40c06c8f0 0x7ff40c06edb0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:51.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.512+0000 7ff432800640 1 --2- 172.21.15.86:0/2602326843 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff42c160080 0x7ff42c144f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:51.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.512+0000 7ff432800640 1 --2- 172.21.15.86:0/2602326843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff42c156ae0 0x7ff42c1599b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:51.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.512+0000 7ff432800640 1 --2- 172.21.15.86:0/2602326843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff42c0ad7f0 0x7ff42c099a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:51.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.512+0000 7ff432800640 1 -- 172.21.15.86:0/2602326843 >> 172.21.15.86:0/2602326843 conn(0x7ff42c0d0170 msgr2=0x7ff42c155570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:51.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.512+0000 7ff432800640 1 -- 172.21.15.86:0/2602326843 shutdown_connections 2024-04-20T21:18:51.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.512+0000 7ff432800640 1 -- 172.21.15.86:0/2602326843 wait complete. 2024-04-20T21:18:51.529 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:17:49.718799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:17:50.683477+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:18:51.530 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:18:51.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.731+0000 7f56ab600640 1 Processor -- start 2024-04-20T21:18:51.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.732+0000 7f56ab600640 1 -- start start 2024-04-20T21:18:51.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.733+0000 7f56ab600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f56a40b9740 0x7f56a40d34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:51.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.733+0000 7f56ab600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56a40d3b50 con 0x7f56a40d1680 2024-04-20T21:18:51.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.733+0000 7f56ab600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56a40d3e30 con 0x7f56a41622d0 2024-04-20T21:18:51.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.733+0000 7f56ab600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f56a41626b0 con 0x7f56a40b9740 2024-04-20T21:18:51.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.733+0000 7f56a3e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f56a40d1680 0x7f56a40d2db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42648/0 (socket says 172.21.15.86:42648) 2024-04-20T21:18:51.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.733+0000 7f56a3e00640 1 -- 172.21.15.86:0/3367553310 learned_addr learned my addr 172.21.15.86:0/3367553310 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:18:51.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.734+0000 7f56a9600640 1 --2- 172.21.15.86:0/3367553310 >> v2:172.21.15.144:3300/0 conn(0x7f56a40b9740 0x7f56a40d34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:51.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.734+0000 7f56a3400640 1 -- 172.21.15.86:0/3367553310 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3010158022 0 0) 0x7f56a40d3b50 con 0x7f56a40d1680 2024-04-20T21:18:51.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.734+0000 7f56a3400640 1 -- 172.21.15.86:0/3367553310 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5688003520 con 0x7f56a40d1680 2024-04-20T21:18:51.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.734+0000 7f56a3400640 1 -- 172.21.15.86:0/3367553310 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3042635967 0 0) 0x7f5688003520 con 0x7f56a40d1680 2024-04-20T21:18:51.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.734+0000 7f56a3400640 1 -- 172.21.15.86:0/3367553310 >> v2:172.21.15.144:3300/0 conn(0x7f56a40b9740 msgr2=0x7f56a40d34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:51.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.734+0000 7f56a3400640 1 --2- 172.21.15.86:0/3367553310 >> v2:172.21.15.144:3300/0 conn(0x7f56a40b9740 0x7f56a40d34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:51.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.734+0000 7f56a3400640 1 -- 172.21.15.86:0/3367553310 >> v1:172.21.15.144:6789/0 conn(0x7f56a41622d0 legacy=0x7f56a40d0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:51.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.734+0000 7f56a3400640 1 -- 172.21.15.86:0/3367553310 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56a40b7f30 con 0x7f56a40d1680 2024-04-20T21:18:51.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.734+0000 7f56a3400640 1 -- 172.21.15.86:0/3367553310 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f568c001670 con 0x7f56a40d1680 2024-04-20T21:18:51.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.734+0000 7f56a3400640 1 -- 172.21.15.86:0/3367553310 >> v1:172.21.15.86:6789/0 conn(0x7f56a40d1680 legacy=0x7f56a40d2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:51.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.735+0000 7f56a3400640 1 --2- 172.21.15.86:0/3367553310 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5688003a00 0x7f5688003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:51.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.735+0000 7f56a3400640 1 --2- 172.21.15.86:0/3367553310 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5688004800 0x7f5688004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:51.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.735+0000 7f56a3400640 1 --2- 172.21.15.86:0/3367553310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56880051c0 0x7f568800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:51.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.735+0000 7f56a3400640 1 -- 172.21.15.86:0/3367553310 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f56a41626b0 con 0x7f56880051c0 2024-04-20T21:18:51.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.735+0000 7f56a3400640 1 -- 172.21.15.86:0/3367553310 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f568800c0a0 con 0x7f5688004800 2024-04-20T21:18:51.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.735+0000 7f56a3400640 1 -- 172.21.15.86:0/3367553310 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f568800c2a0 con 0x7f5688003a00 2024-04-20T21:18:51.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.735+0000 7f56a9600640 1 --2- 172.21.15.86:0/3367553310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56880051c0 0x7f568800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:51.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.735+0000 7f56a3e00640 1 --2- 172.21.15.86:0/3367553310 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5688004800 0x7f5688004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:51.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.735+0000 7f56a8c00640 1 --2- 172.21.15.86:0/3367553310 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5688003a00 0x7f5688003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:51.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.735+0000 7f56a9600640 1 -- 172.21.15.86:0/3367553310 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5688003a00 msgr2=0x7f5688003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:51.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.735+0000 7f56a9600640 1 --2- 172.21.15.86:0/3367553310 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5688003a00 0x7f5688003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:51.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.735+0000 7f56a9600640 1 -- 172.21.15.86:0/3367553310 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5688004800 msgr2=0x7f5688004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:51.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.735+0000 7f56a9600640 1 --2- 172.21.15.86:0/3367553310 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5688004800 0x7f5688004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:51.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.735+0000 7f56a9600640 1 -- 172.21.15.86:0/3367553310 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5688003520 con 0x7f56880051c0 2024-04-20T21:18:51.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.735+0000 7f56a3e00640 1 --2- 172.21.15.86:0/3367553310 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5688004800 0x7f5688004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:51.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.735+0000 7f56a8c00640 1 --2- 172.21.15.86:0/3367553310 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5688003a00 0x7f5688003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:51.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.736+0000 7f56a9600640 1 -- 172.21.15.86:0/3367553310 reap_dead start 2024-04-20T21:18:51.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.737+0000 7f56a9600640 1 --2- 172.21.15.86:0/3367553310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56880051c0 0x7f568800b9c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f569c009f90 tx=0x7f5688005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd6f11e458468d2d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:51.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.737+0000 7f56a3400640 1 -- 172.21.15.86:0/3367553310 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569c039070 con 0x7f56880051c0 2024-04-20T21:18:51.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.737+0000 7f56a3400640 1 -- 172.21.15.86:0/3367553310 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f569c02cc20 con 0x7f56880051c0 2024-04-20T21:18:51.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.737+0000 7f56a3400640 1 -- 172.21.15.86:0/3367553310 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569c009c10 con 0x7f56880051c0 2024-04-20T21:18:51.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.737+0000 7f56ab600640 1 -- 172.21.15.86:0/3367553310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56880051c0 msgr2=0x7f568800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:51.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.737+0000 7f56ab600640 1 --2- 172.21.15.86:0/3367553310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56880051c0 0x7f568800b9c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f569c009f90 tx=0x7f5688005830 comp rx=0 tx=0).stop 2024-04-20T21:18:51.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.738+0000 7f56ab600640 1 -- 172.21.15.86:0/3367553310 shutdown_connections 2024-04-20T21:18:51.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.738+0000 7f56ab600640 1 --2- 172.21.15.86:0/3367553310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56880051c0 0x7f568800b9c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:51.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.738+0000 7f56ab600640 1 -- 172.21.15.86:0/3367553310 >> 172.21.15.86:0/3367553310 conn(0x7f56a406cbe0 msgr2=0x7f56a406cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:51.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.738+0000 7f56ab600640 1 -- 172.21.15.86:0/3367553310 shutdown_connections 2024-04-20T21:18:51.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.738+0000 7f56ab600640 1 -- 172.21.15.86:0/3367553310 wait complete. 2024-04-20T21:18:51.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.739+0000 7f56ab600640 1 Processor -- start 2024-04-20T21:18:51.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.740+0000 7f56ab600640 1 -- start start 2024-04-20T21:18:51.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.740+0000 7f56ab600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56880051c0 0x7f56a4158fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:51.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.740+0000 7f56ab600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56a415afc0 0x7f56a40cd920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:51.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.740+0000 7f56ab600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56a40ca680 0x7f56a40cab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:51.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.740+0000 7f56ab600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f56a40d0f70 con 0x7f56a415afc0 2024-04-20T21:18:51.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.740+0000 7f56ab600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f56a40b82a0 con 0x7f56a40ca680 2024-04-20T21:18:51.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.740+0000 7f56ab600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f56a40d10f0 con 0x7f56880051c0 2024-04-20T21:18:51.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.741+0000 7f56a3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56a415afc0 0x7f56a40cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:51.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.741+0000 7f56a3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56a415afc0 0x7f56a40cd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37338/0 (socket says 172.21.15.86:37338) 2024-04-20T21:18:51.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.741+0000 7f56a3e00640 1 -- 172.21.15.86:0/1169701366 learned_addr learned my addr 172.21.15.86:0/1169701366 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:51.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.741+0000 7f56a8c00640 1 --2- 172.21.15.86:0/1169701366 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56880051c0 0x7f56a4158fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:51.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.741+0000 7f56a9600640 1 --2- 172.21.15.86:0/1169701366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56a40ca680 0x7f56a40cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:51.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.741+0000 7f56a3e00640 1 -- 172.21.15.86:0/1169701366 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56880051c0 msgr2=0x7f56a4158fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:51.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.741+0000 7f56a3e00640 1 --2- 172.21.15.86:0/1169701366 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56880051c0 0x7f56a4158fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:51.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.741+0000 7f56a3e00640 1 -- 172.21.15.86:0/1169701366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56a40ca680 msgr2=0x7f56a40cab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:51.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.741+0000 7f56a3e00640 1 --2- 172.21.15.86:0/1169701366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56a40ca680 0x7f56a40cab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:51.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.741+0000 7f56a3e00640 1 -- 172.21.15.86:0/1169701366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56a415ab20 con 0x7f56a415afc0 2024-04-20T21:18:51.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.741+0000 7f56a8c00640 1 --2- 172.21.15.86:0/1169701366 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56880051c0 0x7f56a4158fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:51.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.742+0000 7f56a3e00640 1 --2- 172.21.15.86:0/1169701366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56a415afc0 0x7f56a40cd920 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f568c00d6c0 tx=0x7f568c00db80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:51.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.742+0000 7f56a0c00640 1 -- 172.21.15.86:0/1169701366 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568c019070 con 0x7f56a415afc0 2024-04-20T21:18:51.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.742+0000 7f56a0c00640 1 -- 172.21.15.86:0/1169701366 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f568c0072d0 con 0x7f56a415afc0 2024-04-20T21:18:51.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.742+0000 7f56a0c00640 1 -- 172.21.15.86:0/1169701366 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568c0047e0 con 0x7f56a415afc0 2024-04-20T21:18:51.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.742+0000 7f56ab600640 1 -- 172.21.15.86:0/1169701366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56a41f49d0 con 0x7f56a415afc0 2024-04-20T21:18:51.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.742+0000 7f56ab600640 1 -- 172.21.15.86:0/1169701366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56a41f4f90 con 0x7f56a415afc0 2024-04-20T21:18:51.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.743+0000 7f56ab600640 1 -- 172.21.15.86:0/1169701366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56a40689e0 con 0x7f56a415afc0 2024-04-20T21:18:51.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.744+0000 7f56a0c00640 1 -- 172.21.15.86:0/1169701366 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f568c004980 con 0x7f56a415afc0 2024-04-20T21:18:51.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.745+0000 7f56a0c00640 1 --2- 172.21.15.86:0/1169701366 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f568006c8f0 0x7f568006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:51.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.745+0000 7f56a0c00640 1 -- 172.21.15.86:0/1169701366 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f568c08b550 con 0x7f56a415afc0 2024-04-20T21:18:51.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.745+0000 7f56a8c00640 1 --2- 172.21.15.86:0/1169701366 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f568006c8f0 0x7f568006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:51.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.746+0000 7f56a8c00640 1 --2- 172.21.15.86:0/1169701366 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f568006c8f0 0x7f568006edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f56940016f0 tx=0x7f5694009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:51.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.750+0000 7f56a0c00640 1 -- 172.21.15.86:0/1169701366 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f568c004c70 con 0x7f56a415afc0 2024-04-20T21:18:51.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.951+0000 7f56ab600640 1 -- 172.21.15.86:0/1169701366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f56a40cf860 con 0x7f56a415afc0 2024-04-20T21:18:51.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.952+0000 7f56a0c00640 1 -- 172.21.15.86:0/1169701366 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f568c089e20 con 0x7f56a415afc0 2024-04-20T21:18:51.951 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:18:51.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.954+0000 7f56ab600640 1 -- 172.21.15.86:0/1169701366 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f568006c8f0 msgr2=0x7f568006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:51.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.954+0000 7f56ab600640 1 --2- 172.21.15.86:0/1169701366 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f568006c8f0 0x7f568006edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f56940016f0 tx=0x7f5694009040 comp rx=0 tx=0).stop 2024-04-20T21:18:51.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.954+0000 7f56ab600640 1 -- 172.21.15.86:0/1169701366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56a415afc0 msgr2=0x7f56a40cd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:51.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.954+0000 7f56ab600640 1 --2- 172.21.15.86:0/1169701366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56a415afc0 0x7f56a40cd920 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f568c00d6c0 tx=0x7f568c00db80 comp rx=0 tx=0).stop 2024-04-20T21:18:51.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.954+0000 7f56ab600640 1 -- 172.21.15.86:0/1169701366 shutdown_connections 2024-04-20T21:18:51.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.954+0000 7f56ab600640 1 --2- 172.21.15.86:0/1169701366 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f568006c8f0 0x7f568006edb0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:51.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.954+0000 7f56ab600640 1 --2- 172.21.15.86:0/1169701366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56a40ca680 0x7f56a40cab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:51.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.954+0000 7f56ab600640 1 --2- 172.21.15.86:0/1169701366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56a415afc0 0x7f56a40cd920 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:51.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.954+0000 7f56ab600640 1 --2- 172.21.15.86:0/1169701366 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f56880051c0 0x7f56a4158fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:51.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.954+0000 7f56ab600640 1 -- 172.21.15.86:0/1169701366 >> 172.21.15.86:0/1169701366 conn(0x7f56a406cbe0 msgr2=0x7f56a415f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:51.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.954+0000 7f56ab600640 1 -- 172.21.15.86:0/1169701366 shutdown_connections 2024-04-20T21:18:51.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:51.955+0000 7f56ab600640 1 -- 172.21.15.86:0/1169701366 wait complete. 2024-04-20T21:18:51.971 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:18:52.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.168+0000 7f8ea5600640 1 Processor -- start 2024-04-20T21:18:52.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.168+0000 7f8ea5600640 1 -- start start 2024-04-20T21:18:52.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.169+0000 7f8ea5600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8ea0145f00 0x7f8ea0071d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:52.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.169+0000 7f8ea5600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8ea0072280 0x7f8ea015cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:52.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.169+0000 7f8ea5600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ea0160410 con 0x7f8ea0145250 2024-04-20T21:18:52.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.169+0000 7f8ea5600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f8ea01606f0 con 0x7f8ea0072280 2024-04-20T21:18:52.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.169+0000 7f8ea5600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8ea01609d0 con 0x7f8ea0145f00 2024-04-20T21:18:52.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.170+0000 7f8e9fe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f8ea0145250 0x7f8ea006efc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42652/0 (socket says 172.21.15.86:42652) 2024-04-20T21:18:52.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.170+0000 7f8e9fe00640 1 -- 172.21.15.86:0/18969250 learned_addr learned my addr 172.21.15.86:0/18969250 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:18:52.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.170+0000 7f8e97e00640 1 --2- 172.21.15.86:0/18969250 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8ea0145f00 0x7f8ea0071d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:52.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.170+0000 7f8ea4c00640 1 --2- 172.21.15.86:0/18969250 >> v2:172.21.15.144:3300/0 conn(0x7f8ea0072280 0x7f8ea015cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:52.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.170+0000 7f8e9f400640 1 -- 172.21.15.86:0/18969250 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1649214723 0 0) 0x7f8ea0160410 con 0x7f8ea0145250 2024-04-20T21:18:52.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.170+0000 7f8e9f400640 1 -- 172.21.15.86:0/18969250 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e84003520 con 0x7f8ea0145250 2024-04-20T21:18:52.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.171+0000 7f8e9f400640 1 -- 172.21.15.86:0/18969250 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1466965166 0 0) 0x7f8e84003520 con 0x7f8ea0145250 2024-04-20T21:18:52.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.171+0000 7f8e9f400640 1 -- 172.21.15.86:0/18969250 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8ea0145f00 msgr2=0x7f8ea0071d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:52.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.171+0000 7f8e9f400640 1 --2- 172.21.15.86:0/18969250 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8ea0145f00 0x7f8ea0071d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:52.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.171+0000 7f8e9f400640 1 -- 172.21.15.86:0/18969250 >> v2:172.21.15.144:3300/0 conn(0x7f8ea0072280 msgr2=0x7f8ea015cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:52.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.171+0000 7f8e9f400640 1 --2- 172.21.15.86:0/18969250 >> v2:172.21.15.144:3300/0 conn(0x7f8ea0072280 0x7f8ea015cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:52.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.171+0000 7f8e9f400640 1 -- 172.21.15.86:0/18969250 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ea0160cb0 con 0x7f8ea0145250 2024-04-20T21:18:52.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.171+0000 7f8e9f400640 1 -- 172.21.15.86:0/18969250 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f8e90001670 con 0x7f8ea0145250 2024-04-20T21:18:52.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.171+0000 7f8e9f400640 1 -- 172.21.15.86:0/18969250 >> v1:172.21.15.86:6789/0 conn(0x7f8ea0145250 legacy=0x7f8ea006efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:52.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.171+0000 7f8e9f400640 1 --2- 172.21.15.86:0/18969250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e84003a20 0x7f8e84003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:52.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.171+0000 7f8e9f400640 1 --2- 172.21.15.86:0/18969250 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8ea0145f00 0x7f8e84004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:52.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.171+0000 7f8e9f400640 1 --2- 172.21.15.86:0/18969250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e84004d40 0x7f8e840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:52.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.171+0000 7f8e9f400640 1 -- 172.21.15.86:0/18969250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea01606f0 con 0x7f8e84003a20 2024-04-20T21:18:52.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.171+0000 7f8e9f400640 1 -- 172.21.15.86:0/18969250 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea01609d0 con 0x7f8e84004d40 2024-04-20T21:18:52.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.171+0000 7f8e9f400640 1 -- 172.21.15.86:0/18969250 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8e8400baf0 con 0x7f8ea0145f00 2024-04-20T21:18:52.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.171+0000 7f8ea4c00640 1 --2- 172.21.15.86:0/18969250 >> v2:172.21.15.144:3300/0 conn(0x7f8ea0072280 0x7f8ea015cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:18:52.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.171+0000 7f8e9fe00640 1 --2- 172.21.15.86:0/18969250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e84003a20 0x7f8e84003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:52.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.171+0000 7f8e97e00640 1 --2- 172.21.15.86:0/18969250 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8ea0145f00 0x7f8e84004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:52.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.171+0000 7f8ea4c00640 1 --2- 172.21.15.86:0/18969250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e84004d40 0x7f8e840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:52.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.172+0000 7f8e9fe00640 1 -- 172.21.15.86:0/18969250 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8ea0145f00 msgr2=0x7f8e84004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:52.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.172+0000 7f8e9fe00640 1 --2- 172.21.15.86:0/18969250 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8ea0145f00 0x7f8e84004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:52.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.172+0000 7f8e9fe00640 1 -- 172.21.15.86:0/18969250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e84004d40 msgr2=0x7f8e840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:52.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.172+0000 7f8e9fe00640 1 --2- 172.21.15.86:0/18969250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e84004d40 0x7f8e840051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:52.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.172+0000 7f8e9fe00640 1 -- 172.21.15.86:0/18969250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e84003520 con 0x7f8e84003a20 2024-04-20T21:18:52.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.173+0000 7f8e9fe00640 1 --2- 172.21.15.86:0/18969250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e84003a20 0x7f8e84003e40 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8e9000c240 tx=0x7f8e84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61a9b72c93cc64c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:52.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.173+0000 7f8e9f400640 1 -- 172.21.15.86:0/18969250 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e90007d30 con 0x7f8e84003a20 2024-04-20T21:18:52.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.173+0000 7f8e9f400640 1 -- 172.21.15.86:0/18969250 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e90030070 con 0x7f8e84003a20 2024-04-20T21:18:52.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.173+0000 7f8e9f400640 1 -- 172.21.15.86:0/18969250 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e9002cd60 con 0x7f8e84003a20 2024-04-20T21:18:52.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.174+0000 7f8ea5600640 1 -- 172.21.15.86:0/18969250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e84003a20 msgr2=0x7f8e84003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:52.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.174+0000 7f8ea5600640 1 --2- 172.21.15.86:0/18969250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e84003a20 0x7f8e84003e40 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8e9000c240 tx=0x7f8e84005830 comp rx=0 tx=0).stop 2024-04-20T21:18:52.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.174+0000 7f8ea4c00640 1 -- 172.21.15.86:0/18969250 reap_dead start 2024-04-20T21:18:52.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.174+0000 7f8ea5600640 1 -- 172.21.15.86:0/18969250 shutdown_connections 2024-04-20T21:18:52.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.174+0000 7f8ea5600640 1 -- 172.21.15.86:0/18969250 >> 172.21.15.86:0/18969250 conn(0x7f8ea0067cf0 msgr2=0x7f8ea006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:52.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.174+0000 7f8ea5600640 1 -- 172.21.15.86:0/18969250 shutdown_connections 2024-04-20T21:18:52.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.174+0000 7f8ea5600640 1 -- 172.21.15.86:0/18969250 wait complete. 2024-04-20T21:18:52.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.176+0000 7f8ea5600640 1 Processor -- start 2024-04-20T21:18:52.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.177+0000 7f8ea5600640 1 -- start start 2024-04-20T21:18:52.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.178+0000 7f8ea5600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8e84003a20 0x7f8ea00c6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:52.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.178+0000 7f8ea5600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ea00b52b0 0x7f8ea015b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:52.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.178+0000 7f8ea5600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ea00c85f0 0x7f8ea013f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:52.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.178+0000 7f8ea5600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea0071ec0 con 0x7f8ea00c85f0 2024-04-20T21:18:52.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.178+0000 7f8ea5600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea0071d40 con 0x7f8ea00b52b0 2024-04-20T21:18:52.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.178+0000 7f8ea5600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8ea0072040 con 0x7f8e84003a20 2024-04-20T21:18:52.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.178+0000 7f8ea4c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ea00c85f0 0x7f8ea013f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:52.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.178+0000 7f8ea4c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ea00c85f0 0x7f8ea013f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37350/0 (socket says 172.21.15.86:37350) 2024-04-20T21:18:52.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.178+0000 7f8ea4c00640 1 -- 172.21.15.86:0/3082023331 learned_addr learned my addr 172.21.15.86:0/3082023331 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:52.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.178+0000 7f8e9fe00640 1 --2- 172.21.15.86:0/3082023331 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8e84003a20 0x7f8ea00c6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:52.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.178+0000 7f8e97e00640 1 --2- 172.21.15.86:0/3082023331 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ea00b52b0 0x7f8ea015b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:52.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.179+0000 7f8ea4c00640 1 -- 172.21.15.86:0/3082023331 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8e84003a20 msgr2=0x7f8ea00c6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:52.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.179+0000 7f8ea4c00640 1 --2- 172.21.15.86:0/3082023331 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8e84003a20 0x7f8ea00c6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:52.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.179+0000 7f8ea4c00640 1 -- 172.21.15.86:0/3082023331 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ea00b52b0 msgr2=0x7f8ea015b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:52.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.179+0000 7f8ea4c00640 1 --2- 172.21.15.86:0/3082023331 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ea00b52b0 0x7f8ea015b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:52.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.179+0000 7f8ea4c00640 1 -- 172.21.15.86:0/3082023331 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ea00c7550 con 0x7f8ea00c85f0 2024-04-20T21:18:52.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.179+0000 7f8e9fe00640 1 --2- 172.21.15.86:0/3082023331 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8e84003a20 0x7f8ea00c6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:52.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.179+0000 7f8e97e00640 1 --2- 172.21.15.86:0/3082023331 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ea00b52b0 0x7f8ea015b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:52.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.179+0000 7f8ea4c00640 1 --2- 172.21.15.86:0/3082023331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ea00c85f0 0x7f8ea013f830 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0145e90 tx=0x7f8e98007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:52.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.179+0000 7f8e9cc00640 1 -- 172.21.15.86:0/3082023331 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e98003ea0 con 0x7f8ea00c85f0 2024-04-20T21:18:52.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.179+0000 7f8e9cc00640 1 -- 172.21.15.86:0/3082023331 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e98019070 con 0x7f8ea00c85f0 2024-04-20T21:18:52.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.179+0000 7f8ea5600640 1 -- 172.21.15.86:0/3082023331 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ea00c7830 con 0x7f8ea00c85f0 2024-04-20T21:18:52.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.179+0000 7f8e9cc00640 1 -- 172.21.15.86:0/3082023331 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e98014630 con 0x7f8ea00c85f0 2024-04-20T21:18:52.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.179+0000 7f8ea5600640 1 -- 172.21.15.86:0/3082023331 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ea00c7df0 con 0x7f8ea00c85f0 2024-04-20T21:18:52.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.180+0000 7f8ea5600640 1 -- 172.21.15.86:0/3082023331 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ea004fd90 con 0x7f8ea00c85f0 2024-04-20T21:18:52.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.181+0000 7f8e9cc00640 1 -- 172.21.15.86:0/3082023331 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8e980147d0 con 0x7f8ea00c85f0 2024-04-20T21:18:52.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.182+0000 7f8e9cc00640 1 --2- 172.21.15.86:0/3082023331 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8e7c06c920 0x7f8e7c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:52.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.182+0000 7f8e9cc00640 1 -- 172.21.15.86:0/3082023331 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f8e9808c110 con 0x7f8ea00c85f0 2024-04-20T21:18:52.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.183+0000 7f8e9fe00640 1 --2- 172.21.15.86:0/3082023331 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8e7c06c920 0x7f8e7c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:52.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.183+0000 7f8e9fe00640 1 --2- 172.21.15.86:0/3082023331 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8e7c06c920 0x7f8e7c06ede0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8e90034000 tx=0x7f8e90007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:52.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.186+0000 7f8e9cc00640 1 -- 172.21.15.86:0/3082023331 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8e9805a4b0 con 0x7f8ea00c85f0 2024-04-20T21:18:52.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.386+0000 7f8ea5600640 1 -- 172.21.15.86:0/3082023331 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8ea006bc30 con 0x7f8ea00c85f0 2024-04-20T21:18:52.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.387+0000 7f8e9cc00640 1 -- 172.21.15.86:0/3082023331 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f8e98059db0 con 0x7f8ea00c85f0 2024-04-20T21:18:52.386 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-20T21:18:52.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.389+0000 7f8ea5600640 1 -- 172.21.15.86:0/3082023331 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8e7c06c920 msgr2=0x7f8e7c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:52.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.389+0000 7f8ea5600640 1 --2- 172.21.15.86:0/3082023331 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8e7c06c920 0x7f8e7c06ede0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8e90034000 tx=0x7f8e90007000 comp rx=0 tx=0).stop 2024-04-20T21:18:52.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.389+0000 7f8ea5600640 1 -- 172.21.15.86:0/3082023331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ea00c85f0 msgr2=0x7f8ea013f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:52.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.389+0000 7f8ea5600640 1 --2- 172.21.15.86:0/3082023331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ea00c85f0 0x7f8ea013f830 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0145e90 tx=0x7f8e98007700 comp rx=0 tx=0).stop 2024-04-20T21:18:52.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.389+0000 7f8ea5600640 1 -- 172.21.15.86:0/3082023331 shutdown_connections 2024-04-20T21:18:52.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.389+0000 7f8ea5600640 1 --2- 172.21.15.86:0/3082023331 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8e7c06c920 0x7f8e7c06ede0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:52.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.389+0000 7f8ea5600640 1 --2- 172.21.15.86:0/3082023331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ea00c85f0 0x7f8ea013f830 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:52.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.389+0000 7f8ea5600640 1 --2- 172.21.15.86:0/3082023331 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ea00b52b0 0x7f8ea015b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:52.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.389+0000 7f8ea5600640 1 --2- 172.21.15.86:0/3082023331 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8e84003a20 0x7f8ea00c6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:52.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.389+0000 7f8ea5600640 1 -- 172.21.15.86:0/3082023331 >> 172.21.15.86:0/3082023331 conn(0x7f8ea0067cf0 msgr2=0x7f8ea0142cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:52.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.390+0000 7f8ea5600640 1 -- 172.21.15.86:0/3082023331 shutdown_connections 2024-04-20T21:18:52.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.390+0000 7f8ea5600640 1 -- 172.21.15.86:0/3082023331 wait complete. 2024-04-20T21:18:52.408 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:18:52.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.607+0000 7fa4e3e00640 1 Processor -- start 2024-04-20T21:18:52.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.608+0000 7fa4e3e00640 1 -- start start 2024-04-20T21:18:52.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.609+0000 7fa4e3e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa4e40d4310 0x7fa4e40d4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:52.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.609+0000 7fa4e3e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa4e40cfb80 0x7fa4e40d0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:52.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.609+0000 7fa4e3e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4e40d4cb0 con 0x7fa4e415e870 2024-04-20T21:18:52.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.609+0000 7fa4e3e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa4e406d590 con 0x7fa4e40d4310 2024-04-20T21:18:52.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.609+0000 7fa4e3e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fa4e40cf150 con 0x7fa4e40cfb80 2024-04-20T21:18:52.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.610+0000 7fa4dbe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa4e40d4310 0x7fa4e40d4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:52.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.610+0000 7fa4dbe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa4e40d4310 0x7fa4e40d4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37364/0 (socket says 172.21.15.86:37364) 2024-04-20T21:18:52.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.610+0000 7fa4dbe00640 1 -- 172.21.15.86:0/959798363 learned_addr learned my addr 172.21.15.86:0/959798363 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:52.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.610+0000 7fa4dbe00640 1 -- 172.21.15.86:0/959798363 >> v2:172.21.15.144:3300/0 conn(0x7fa4e40cfb80 msgr2=0x7fa4e40d0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:52.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.610+0000 7fa4dbe00640 1 --2- 172.21.15.86:0/959798363 >> v2:172.21.15.144:3300/0 conn(0x7fa4e40cfb80 0x7fa4e40d0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:52.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.610+0000 7fa4dbe00640 1 -- 172.21.15.86:0/959798363 >> v1:172.21.15.144:6789/0 conn(0x7fa4e415e870 legacy=0x7fa4e406ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:52.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.610+0000 7fa4dbe00640 1 -- 172.21.15.86:0/959798363 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4e40d2ff0 con 0x7fa4e40d4310 2024-04-20T21:18:52.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.611+0000 7fa4dbe00640 1 --2- 172.21.15.86:0/959798363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4e40d4310 0x7fa4e40d4770 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa4e406ea40 tx=0x7fa4cc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc05e4c05dc56f33 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:52.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.611+0000 7fa4e2000640 1 -- 172.21.15.86:0/959798363 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc035020 con 0x7fa4e40d4310 2024-04-20T21:18:52.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.612+0000 7fa4e2000640 1 -- 172.21.15.86:0/959798363 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4cc031870 con 0x7fa4e40d4310 2024-04-20T21:18:52.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.612+0000 7fa4e2000640 1 -- 172.21.15.86:0/959798363 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc031c40 con 0x7fa4e40d4310 2024-04-20T21:18:52.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.612+0000 7fa4e3e00640 1 -- 172.21.15.86:0/959798363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4e40d4310 msgr2=0x7fa4e40d4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:52.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.612+0000 7fa4e3e00640 1 --2- 172.21.15.86:0/959798363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4e40d4310 0x7fa4e40d4770 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa4e406ea40 tx=0x7fa4cc02b910 comp rx=0 tx=0).stop 2024-04-20T21:18:52.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.612+0000 7fa4e3e00640 1 -- 172.21.15.86:0/959798363 shutdown_connections 2024-04-20T21:18:52.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.612+0000 7fa4e3e00640 1 --2- 172.21.15.86:0/959798363 >> v2:172.21.15.144:3300/0 conn(0x7fa4e40cfb80 0x7fa4e40d0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:52.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.612+0000 7fa4e3e00640 1 --2- 172.21.15.86:0/959798363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4e40d4310 0x7fa4e40d4770 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:52.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.613+0000 7fa4e3e00640 1 -- 172.21.15.86:0/959798363 >> 172.21.15.86:0/959798363 conn(0x7fa4e409d9c0 msgr2=0x7fa4e408bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:52.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.613+0000 7fa4e3e00640 1 -- 172.21.15.86:0/959798363 shutdown_connections 2024-04-20T21:18:52.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.613+0000 7fa4e3e00640 1 -- 172.21.15.86:0/959798363 wait complete. 2024-04-20T21:18:52.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.614+0000 7fa4e3e00640 1 Processor -- start 2024-04-20T21:18:52.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.614+0000 7fa4e3e00640 1 -- start start 2024-04-20T21:18:52.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.615+0000 7fa4e3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4e40cfb80 0x7fa4e40b6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:52.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.615+0000 7fa4e3e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4e40d4310 0x7fa4e415b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:52.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.615+0000 7fa4e3e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa4e415e870 0x7fa4e415c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:52.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.615+0000 7fa4e3e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e40cf380 con 0x7fa4e40cfb80 2024-04-20T21:18:52.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.615+0000 7fa4e3e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e40d35b0 con 0x7fa4e40d4310 2024-04-20T21:18:52.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.615+0000 7fa4e3e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa4e40cf660 con 0x7fa4e415e870 2024-04-20T21:18:52.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.616+0000 7fa4e2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4e40cfb80 0x7fa4e40b6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:52.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.616+0000 7fa4e2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4e40cfb80 0x7fa4e40b6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37368/0 (socket says 172.21.15.86:37368) 2024-04-20T21:18:52.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.616+0000 7fa4e2a00640 1 -- 172.21.15.86:0/1205329186 learned_addr learned my addr 172.21.15.86:0/1205329186 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:52.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.616+0000 7fa4dbe00640 1 --2- 172.21.15.86:0/1205329186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4e40d4310 0x7fa4e415b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:52.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.616+0000 7fa4e2a00640 1 -- 172.21.15.86:0/1205329186 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa4e415e870 msgr2=0x7fa4e415c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:52.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.616+0000 7fa4e2a00640 1 --2- 172.21.15.86:0/1205329186 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa4e415e870 0x7fa4e415c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:52.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.616+0000 7fa4e2a00640 1 -- 172.21.15.86:0/1205329186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4e40d4310 msgr2=0x7fa4e415b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:52.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.616+0000 7fa4e2a00640 1 --2- 172.21.15.86:0/1205329186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4e40d4310 0x7fa4e415b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:52.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.616+0000 7fa4e2a00640 1 -- 172.21.15.86:0/1205329186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4e40b4d20 con 0x7fa4e40cfb80 2024-04-20T21:18:52.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.616+0000 7fa4dbe00640 1 --2- 172.21.15.86:0/1205329186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4e40d4310 0x7fa4e415b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:52.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.616+0000 7fa4e2a00640 1 --2- 172.21.15.86:0/1205329186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4e40cfb80 0x7fa4e40b6ac0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa4d400b3f0 tx=0x7fa4d400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:52.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.617+0000 7fa4daa00640 1 -- 172.21.15.86:0/1205329186 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4015020 con 0x7fa4e40cfb80 2024-04-20T21:18:52.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.617+0000 7fa4daa00640 1 -- 172.21.15.86:0/1205329186 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4d4011830 con 0x7fa4e40cfb80 2024-04-20T21:18:52.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.617+0000 7fa4daa00640 1 -- 172.21.15.86:0/1205329186 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4011c20 con 0x7fa4e40cfb80 2024-04-20T21:18:52.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.617+0000 7fa4e3e00640 1 -- 172.21.15.86:0/1205329186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4e40b5000 con 0x7fa4e40cfb80 2024-04-20T21:18:52.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.617+0000 7fa4e3e00640 1 -- 172.21.15.86:0/1205329186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4e40b55c0 con 0x7fa4e40cfb80 2024-04-20T21:18:52.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.618+0000 7fa4e3e00640 1 -- 172.21.15.86:0/1205329186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4e404fd90 con 0x7fa4e40cfb80 2024-04-20T21:18:52.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.619+0000 7fa4daa00640 1 -- 172.21.15.86:0/1205329186 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa4d4018020 con 0x7fa4e40cfb80 2024-04-20T21:18:52.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.620+0000 7fa4daa00640 1 --2- 172.21.15.86:0/1205329186 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa4c006c8f0 0x7fa4c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:52.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.620+0000 7fa4daa00640 1 -- 172.21.15.86:0/1205329186 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fa4d408b7e0 con 0x7fa4e40cfb80 2024-04-20T21:18:52.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.620+0000 7fa4dbe00640 1 --2- 172.21.15.86:0/1205329186 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa4c006c8f0 0x7fa4c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:52.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.621+0000 7fa4dbe00640 1 --2- 172.21.15.86:0/1205329186 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa4c006c8f0 0x7fa4c006edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc036000 tx=0x7fa4cc03c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:52.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.625+0000 7fa4daa00640 1 -- 172.21.15.86:0/1205329186 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa4d401f030 con 0x7fa4e40cfb80 2024-04-20T21:18:52.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.827+0000 7fa4e3e00640 1 -- 172.21.15.86:0/1205329186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa4e40b1880 con 0x7fa4e40cfb80 2024-04-20T21:18:52.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.828+0000 7fa4daa00640 1 -- 172.21.15.86:0/1205329186 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7fa4d40172c0 con 0x7fa4e40cfb80 2024-04-20T21:18:52.827 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-20T21:18:52.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.831+0000 7fa4e3e00640 1 -- 172.21.15.86:0/1205329186 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa4c006c8f0 msgr2=0x7fa4c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:52.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.831+0000 7fa4e3e00640 1 --2- 172.21.15.86:0/1205329186 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa4c006c8f0 0x7fa4c006edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc036000 tx=0x7fa4cc03c040 comp rx=0 tx=0).stop 2024-04-20T21:18:52.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.831+0000 7fa4e3e00640 1 -- 172.21.15.86:0/1205329186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4e40cfb80 msgr2=0x7fa4e40b6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:52.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.831+0000 7fa4e3e00640 1 --2- 172.21.15.86:0/1205329186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4e40cfb80 0x7fa4e40b6ac0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa4d400b3f0 tx=0x7fa4d400b8b0 comp rx=0 tx=0).stop 2024-04-20T21:18:52.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.831+0000 7fa4e3e00640 1 -- 172.21.15.86:0/1205329186 shutdown_connections 2024-04-20T21:18:52.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.831+0000 7fa4e3e00640 1 --2- 172.21.15.86:0/1205329186 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa4c006c8f0 0x7fa4c006edb0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:52.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.831+0000 7fa4e3e00640 1 --2- 172.21.15.86:0/1205329186 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa4e415e870 0x7fa4e415c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:52.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.831+0000 7fa4e3e00640 1 --2- 172.21.15.86:0/1205329186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4e40d4310 0x7fa4e415b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:52.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.831+0000 7fa4e3e00640 1 --2- 172.21.15.86:0/1205329186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4e40cfb80 0x7fa4e40b6ac0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:52.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.831+0000 7fa4e3e00640 1 -- 172.21.15.86:0/1205329186 >> 172.21.15.86:0/1205329186 conn(0x7fa4e409d9c0 msgr2=0x7fa4e408bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:52.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.831+0000 7fa4e3e00640 1 -- 172.21.15.86:0/1205329186 shutdown_connections 2024-04-20T21:18:52.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:52.831+0000 7fa4e3e00640 1 -- 172.21.15.86:0/1205329186 wait complete. 2024-04-20T21:18:52.849 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T21:18:53.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.054+0000 7febc3200640 1 Processor -- start 2024-04-20T21:18:53.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.055+0000 7febc3200640 1 -- start start 2024-04-20T21:18:53.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.056+0000 7febc3200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7febc415c590 0x7febc40ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:53.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.056+0000 7febc3200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7febc40af7e0 0x7febc40cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:53.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.056+0000 7febc3200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febc4160160 con 0x7febc406daf0 2024-04-20T21:18:53.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.056+0000 7febc3200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7febc4160440 con 0x7febc415c590 2024-04-20T21:18:53.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.056+0000 7febc3200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7febc4160720 con 0x7febc40af7e0 2024-04-20T21:18:53.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.056+0000 7febc1e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7febc415c590 0x7febc40ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:53.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.056+0000 7febc1e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7febc415c590 0x7febc40ccb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37370/0 (socket says 172.21.15.86:37370) 2024-04-20T21:18:53.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.056+0000 7febc1e00640 1 -- 172.21.15.86:0/3810151567 learned_addr learned my addr 172.21.15.86:0/3810151567 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:53.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.057+0000 7febc1e00640 1 -- 172.21.15.86:0/3810151567 >> v2:172.21.15.144:3300/0 conn(0x7febc40af7e0 msgr2=0x7febc40cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:53.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.057+0000 7febb9400640 1 --2- 172.21.15.86:0/3810151567 >> v2:172.21.15.144:3300/0 conn(0x7febc40af7e0 0x7febc40cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:53.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.057+0000 7febc1e00640 1 --2- 172.21.15.86:0/3810151567 >> v2:172.21.15.144:3300/0 conn(0x7febc40af7e0 0x7febc40cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:53.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.057+0000 7febc1e00640 1 -- 172.21.15.86:0/3810151567 >> v1:172.21.15.144:6789/0 conn(0x7febc406daf0 legacy=0x7febc40d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:53.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.057+0000 7febc1e00640 1 -- 172.21.15.86:0/3810151567 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febc4160a00 con 0x7febc415c590 2024-04-20T21:18:53.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.058+0000 7febc1e00640 1 --2- 172.21.15.86:0/3810151567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc415c590 0x7febc40ccb10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7febc40cca40 tx=0x7febb402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=396fc034c9a96c89 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:53.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.058+0000 7febc1400640 1 -- 172.21.15.86:0/3810151567 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb4035020 con 0x7febc415c590 2024-04-20T21:18:53.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.058+0000 7febc1400640 1 -- 172.21.15.86:0/3810151567 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febb4031870 con 0x7febc415c590 2024-04-20T21:18:53.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.058+0000 7febc1400640 1 -- 172.21.15.86:0/3810151567 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb4031c40 con 0x7febc415c590 2024-04-20T21:18:53.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.059+0000 7febc3200640 1 -- 172.21.15.86:0/3810151567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc415c590 msgr2=0x7febc40ccb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:53.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.059+0000 7febc3200640 1 --2- 172.21.15.86:0/3810151567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc415c590 0x7febc40ccb10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7febc40cca40 tx=0x7febb402b910 comp rx=0 tx=0).stop 2024-04-20T21:18:53.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.059+0000 7febc3200640 1 -- 172.21.15.86:0/3810151567 shutdown_connections 2024-04-20T21:18:53.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.059+0000 7febc3200640 1 --2- 172.21.15.86:0/3810151567 >> v2:172.21.15.144:3300/0 conn(0x7febc40af7e0 0x7febc40cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:53.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.059+0000 7febc3200640 1 --2- 172.21.15.86:0/3810151567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc415c590 0x7febc40ccb10 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:53.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.059+0000 7febc3200640 1 -- 172.21.15.86:0/3810151567 >> 172.21.15.86:0/3810151567 conn(0x7febc406bb60 msgr2=0x7febc406bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:53.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.059+0000 7febc3200640 1 -- 172.21.15.86:0/3810151567 shutdown_connections 2024-04-20T21:18:53.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.059+0000 7febc3200640 1 -- 172.21.15.86:0/3810151567 wait complete. 2024-04-20T21:18:53.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.061+0000 7febc3200640 1 Processor -- start 2024-04-20T21:18:53.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.061+0000 7febc3200640 1 -- start start 2024-04-20T21:18:53.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.062+0000 7febc3200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7febc406daf0 0x7febc409dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:53.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.062+0000 7febc3200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febc40af7e0 0x7febc415e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:53.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.062+0000 7febc3200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc415c590 0x7febc41575d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:53.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.062+0000 7febc3200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7febc415c970 con 0x7febc415c590 2024-04-20T21:18:53.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.062+0000 7febc3200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7febc406f3d0 con 0x7febc40af7e0 2024-04-20T21:18:53.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.062+0000 7febc3200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7febc4160fc0 con 0x7febc406daf0 2024-04-20T21:18:53.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.062+0000 7febc2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc415c590 0x7febc41575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:53.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.062+0000 7febc2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc415c590 0x7febc41575d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37380/0 (socket says 172.21.15.86:37380) 2024-04-20T21:18:53.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.062+0000 7febc2800640 1 -- 172.21.15.86:0/799990159 learned_addr learned my addr 172.21.15.86:0/799990159 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:53.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.063+0000 7febb9400640 1 --2- 172.21.15.86:0/799990159 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febc40af7e0 0x7febc415e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:53.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.063+0000 7febc1e00640 1 --2- 172.21.15.86:0/799990159 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7febc406daf0 0x7febc409dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:53.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.063+0000 7febc2800640 1 -- 172.21.15.86:0/799990159 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7febc406daf0 msgr2=0x7febc409dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:53.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.063+0000 7febc2800640 1 --2- 172.21.15.86:0/799990159 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7febc406daf0 0x7febc409dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:53.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.063+0000 7febc2800640 1 -- 172.21.15.86:0/799990159 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febc40af7e0 msgr2=0x7febc415e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:53.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.063+0000 7febc2800640 1 --2- 172.21.15.86:0/799990159 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febc40af7e0 0x7febc415e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:53.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.063+0000 7febc2800640 1 -- 172.21.15.86:0/799990159 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febc415d500 con 0x7febc415c590 2024-04-20T21:18:53.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.063+0000 7febc2800640 1 --2- 172.21.15.86:0/799990159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc415c590 0x7febc41575d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7febbc00c780 tx=0x7febbc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:53.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.063+0000 7febbaa00640 1 -- 172.21.15.86:0/799990159 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febbc017070 con 0x7febc415c590 2024-04-20T21:18:53.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.063+0000 7febbaa00640 1 -- 172.21.15.86:0/799990159 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febbc003da0 con 0x7febc415c590 2024-04-20T21:18:53.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.063+0000 7febc3200640 1 -- 172.21.15.86:0/799990159 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febc415d7e0 con 0x7febc415c590 2024-04-20T21:18:53.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.063+0000 7febbaa00640 1 -- 172.21.15.86:0/799990159 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febbc0052b0 con 0x7febc415c590 2024-04-20T21:18:53.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.064+0000 7febc3200640 1 -- 172.21.15.86:0/799990159 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febc415dda0 con 0x7febc415c590 2024-04-20T21:18:53.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.065+0000 7febc3200640 1 -- 172.21.15.86:0/799990159 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febc40afcd0 con 0x7febc415c590 2024-04-20T21:18:53.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.066+0000 7febbaa00640 1 -- 172.21.15.86:0/799990159 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7febbc01a070 con 0x7febc415c590 2024-04-20T21:18:53.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.067+0000 7febbaa00640 1 --2- 172.21.15.86:0/799990159 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7feba006c8f0 0x7feba006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:53.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.067+0000 7febbaa00640 1 -- 172.21.15.86:0/799990159 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7febbc08b220 con 0x7febc415c590 2024-04-20T21:18:53.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.067+0000 7febc1e00640 1 --2- 172.21.15.86:0/799990159 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7feba006c8f0 0x7feba006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:53.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.068+0000 7febc1e00640 1 --2- 172.21.15.86:0/799990159 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7feba006c8f0 0x7feba006edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7febb4036000 tx=0x7febb403c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:53.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.071+0000 7febbaa00640 1 -- 172.21.15.86:0/799990159 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7febc40afcd0 con 0x7febc415c590 2024-04-20T21:18:53.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.286+0000 7febc3200640 1 -- 172.21.15.86:0/799990159 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7febc40ce580 con 0x7febc415c590 2024-04-20T21:18:53.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.287+0000 7febbaa00640 1 -- 172.21.15.86:0/799990159 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7febbc0595c0 con 0x7febc415c590 2024-04-20T21:18:53.286 INFO:teuthology.orchestra.run.smithi086.stderr:listed 0 entries 2024-04-20T21:18:53.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.290+0000 7febc3200640 1 -- 172.21.15.86:0/799990159 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7feba006c8f0 msgr2=0x7feba006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:53.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.290+0000 7febc3200640 1 --2- 172.21.15.86:0/799990159 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7feba006c8f0 0x7feba006edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7febb4036000 tx=0x7febb403c040 comp rx=0 tx=0).stop 2024-04-20T21:18:53.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.290+0000 7febc3200640 1 -- 172.21.15.86:0/799990159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc415c590 msgr2=0x7febc41575d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:53.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.290+0000 7febc3200640 1 --2- 172.21.15.86:0/799990159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc415c590 0x7febc41575d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7febbc00c780 tx=0x7febbc00cc40 comp rx=0 tx=0).stop 2024-04-20T21:18:53.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.290+0000 7febc3200640 1 -- 172.21.15.86:0/799990159 shutdown_connections 2024-04-20T21:18:53.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.290+0000 7febc3200640 1 --2- 172.21.15.86:0/799990159 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7feba006c8f0 0x7feba006edb0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:53.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.290+0000 7febc3200640 1 --2- 172.21.15.86:0/799990159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febc415c590 0x7febc41575d0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:53.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.290+0000 7febc3200640 1 --2- 172.21.15.86:0/799990159 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febc40af7e0 0x7febc415e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:53.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.290+0000 7febc3200640 1 --2- 172.21.15.86:0/799990159 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7febc406daf0 0x7febc409dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:53.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.290+0000 7febc3200640 1 -- 172.21.15.86:0/799990159 >> 172.21.15.86:0/799990159 conn(0x7febc406bb60 msgr2=0x7febc40ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:53.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.290+0000 7febc3200640 1 -- 172.21.15.86:0/799990159 shutdown_connections 2024-04-20T21:18:53.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.290+0000 7febc3200640 1 -- 172.21.15.86:0/799990159 wait complete. 2024-04-20T21:18:53.306 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi144.front.sepia.ceph.com... 2024-04-20T21:18:53.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:53.307 DEBUG:teuthology.orchestra.run.smithi144:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-04-20T21:18:53.396 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:18:53.396 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2024-04-20T21:18:53.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:18:53.396 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-20T21:18:53.448 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-20T21:18:53.448 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:18:53.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.653+0000 7faf1de00640 1 Processor -- start 2024-04-20T21:18:53.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.653+0000 7faf1de00640 1 -- start start 2024-04-20T21:18:53.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.654+0000 7faf1de00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faf18144e90 0x7faf1815ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:53.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.655+0000 7faf1de00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7faf1806f140 0x7faf180aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:53.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.655+0000 7faf1de00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf1806dc30 con 0x7faf180af0b0 2024-04-20T21:18:53.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.655+0000 7faf1de00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7faf18145270 con 0x7faf1806f140 2024-04-20T21:18:53.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.655+0000 7faf1de00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7faf1815f720 con 0x7faf18144e90 2024-04-20T21:18:53.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.655+0000 7faf0e000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7faf1806f140 0x7faf180aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:53.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.655+0000 7faf16a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faf18144e90 0x7faf1815ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:53.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.656+0000 7faf0e000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7faf1806f140 0x7faf180aeb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.86:42952/0 (socket says 172.21.15.86:42952) 2024-04-20T21:18:53.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.656+0000 7faf0e000640 1 -- 172.21.15.86:0/414969763 learned_addr learned my addr 172.21.15.86:0/414969763 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:53.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.656+0000 7faf0e000640 1 -- 172.21.15.86:0/414969763 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faf18144e90 msgr2=0x7faf1815ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:53.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.656+0000 7faf0e000640 1 --2- 172.21.15.86:0/414969763 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faf18144e90 0x7faf1815ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:53.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.656+0000 7faf0e000640 1 -- 172.21.15.86:0/414969763 >> v1:172.21.15.144:6789/0 conn(0x7faf180af0b0 legacy=0x7faf18071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:53.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.656+0000 7faf0e000640 1 -- 172.21.15.86:0/414969763 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf1815fa00 con 0x7faf1806f140 2024-04-20T21:18:53.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.656+0000 7faf16a00640 1 --2- 172.21.15.86:0/414969763 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faf18144e90 0x7faf1815ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:18:53.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.658+0000 7faf0e000640 1 --2- 172.21.15.86:0/414969763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf1806f140 0x7faf180aeb70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7faf181453f0 tx=0x7faf0002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdc7c25db4376b18 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:53.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.658+0000 7faf16000640 1 -- 172.21.15.86:0/414969763 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf00034070 con 0x7faf1806f140 2024-04-20T21:18:53.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.659+0000 7faf16000640 1 -- 172.21.15.86:0/414969763 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf00030840 con 0x7faf1806f140 2024-04-20T21:18:53.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.659+0000 7faf16000640 1 -- 172.21.15.86:0/414969763 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf00030c10 con 0x7faf1806f140 2024-04-20T21:18:53.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.659+0000 7faf1de00640 1 -- 172.21.15.86:0/414969763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf1806f140 msgr2=0x7faf180aeb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:53.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.659+0000 7faf1de00640 1 --2- 172.21.15.86:0/414969763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf1806f140 0x7faf180aeb70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7faf181453f0 tx=0x7faf0002b980 comp rx=0 tx=0).stop 2024-04-20T21:18:53.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.659+0000 7faf1de00640 1 -- 172.21.15.86:0/414969763 shutdown_connections 2024-04-20T21:18:53.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.659+0000 7faf1de00640 1 --2- 172.21.15.86:0/414969763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf1806f140 0x7faf180aeb70 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:53.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.659+0000 7faf1de00640 1 --2- 172.21.15.86:0/414969763 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faf18144e90 0x7faf1815ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:53.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.659+0000 7faf1de00640 1 -- 172.21.15.86:0/414969763 >> 172.21.15.86:0/414969763 conn(0x7faf180b2360 msgr2=0x7faf180b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:53.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.659+0000 7faf1de00640 1 -- 172.21.15.86:0/414969763 shutdown_connections 2024-04-20T21:18:53.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.659+0000 7faf1de00640 1 -- 172.21.15.86:0/414969763 wait complete. 2024-04-20T21:18:53.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.661+0000 7faf1de00640 1 Processor -- start 2024-04-20T21:18:53.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.661+0000 7faf1de00640 1 -- start start 2024-04-20T21:18:53.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.662+0000 7faf1de00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf1806f140 0x7faf1813ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:53.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.662+0000 7faf1de00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faf180af0b0 0x7faf180ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:53.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.662+0000 7faf1de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf18144e90 0x7faf180cf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:53.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.662+0000 7faf1de00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faf1815f350 con 0x7faf18144e90 2024-04-20T21:18:53.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.662+0000 7faf1de00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7faf1806ded0 con 0x7faf1806f140 2024-04-20T21:18:53.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.662+0000 7faf1de00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7faf1815f4d0 con 0x7faf180af0b0 2024-04-20T21:18:53.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.662+0000 7faf17400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf18144e90 0x7faf180cf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:53.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.662+0000 7faf17400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf18144e90 0x7faf180cf3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37386/0 (socket says 172.21.15.86:37386) 2024-04-20T21:18:53.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.663+0000 7faf17400640 1 -- 172.21.15.86:0/1384677890 learned_addr learned my addr 172.21.15.86:0/1384677890 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:53.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.663+0000 7faf16a00640 1 --2- 172.21.15.86:0/1384677890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf1806f140 0x7faf1813ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:53.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.663+0000 7faf0e000640 1 --2- 172.21.15.86:0/1384677890 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faf180af0b0 0x7faf180ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:53.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.663+0000 7faf17400640 1 -- 172.21.15.86:0/1384677890 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faf180af0b0 msgr2=0x7faf180ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:53.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.663+0000 7faf17400640 1 --2- 172.21.15.86:0/1384677890 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faf180af0b0 0x7faf180ceea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:53.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.663+0000 7faf17400640 1 -- 172.21.15.86:0/1384677890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf1806f140 msgr2=0x7faf1813ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:53.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.663+0000 7faf17400640 1 --2- 172.21.15.86:0/1384677890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf1806f140 0x7faf1813ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:53.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.663+0000 7faf17400640 1 -- 172.21.15.86:0/1384677890 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf18145d80 con 0x7faf18144e90 2024-04-20T21:18:53.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.663+0000 7faf17400640 1 --2- 172.21.15.86:0/1384677890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf18144e90 0x7faf180cf3e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7faf1000c7a0 tx=0x7faf1000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:53.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.663+0000 7faf0fe00640 1 -- 172.21.15.86:0/1384677890 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf10003b70 con 0x7faf18144e90 2024-04-20T21:18:53.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.664+0000 7faf0fe00640 1 -- 172.21.15.86:0/1384677890 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf10003d30 con 0x7faf18144e90 2024-04-20T21:18:53.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.664+0000 7faf0fe00640 1 -- 172.21.15.86:0/1384677890 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf1000aa90 con 0x7faf18144e90 2024-04-20T21:18:53.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.664+0000 7faf1de00640 1 -- 172.21.15.86:0/1384677890 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf181482a0 con 0x7faf18144e90 2024-04-20T21:18:53.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.664+0000 7faf1de00640 1 -- 172.21.15.86:0/1384677890 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf18146330 con 0x7faf18144e90 2024-04-20T21:18:53.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.665+0000 7faf1de00640 1 -- 172.21.15.86:0/1384677890 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf180689e0 con 0x7faf18144e90 2024-04-20T21:18:53.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.666+0000 7faf0fe00640 1 -- 172.21.15.86:0/1384677890 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faf1000ac30 con 0x7faf18144e90 2024-04-20T21:18:53.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.667+0000 7faf0fe00640 1 --2- 172.21.15.86:0/1384677890 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7faef406c8f0 0x7faef406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:53.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.667+0000 7faf0fe00640 1 -- 172.21.15.86:0/1384677890 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7faf1008bb40 con 0x7faf18144e90 2024-04-20T21:18:53.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.667+0000 7faf16a00640 1 --2- 172.21.15.86:0/1384677890 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7faef406c8f0 0x7faef406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:53.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.668+0000 7faf16a00640 1 --2- 172.21.15.86:0/1384677890 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7faef406c8f0 0x7faef406edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7faf08009560 tx=0x7faf08008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:53.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.671+0000 7faf0fe00640 1 -- 172.21.15.86:0/1384677890 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7faf10059ee0 con 0x7faf18144e90 2024-04-20T21:18:53.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.894+0000 7faf1de00640 1 -- 172.21.15.86:0/1384677890 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faf1806f690 con 0x7faf18144e90 2024-04-20T21:18:53.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.895+0000 7faf0fe00640 1 -- 172.21.15.86:0/1384677890 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5196 (secure 0 0 0) 0x7faf100597e0 con 0x7faf18144e90 2024-04-20T21:18:53.894 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:18:53.894 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/179773217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":179773217},{"type":"v1","addr":"172.21.15.144:6834","nonce":179773217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3179785507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3179785507},{"type":"v1","addr":"172.21.15.86:6835","nonce":3179785507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4270,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2780853729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2780853729},{"type":"v1","addr":"172.21.15.86:6837","nonce":2780853729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:17:51.700362+0000","modified":"2024-04-20T21:17:54.973792+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6835/2436181584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2436181584},{"type":"v1","addr":"172.21.15.144:6835","nonce":2436181584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4272,"qdb_cluster":[4272]},"id":1}]} 2024-04-20T21:18:53.895 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 7 2024-04-20T21:18:53.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.898+0000 7faf1de00640 1 -- 172.21.15.86:0/1384677890 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7faef406c8f0 msgr2=0x7faef406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:53.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.898+0000 7faf1de00640 1 --2- 172.21.15.86:0/1384677890 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7faef406c8f0 0x7faef406edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7faf08009560 tx=0x7faf08008040 comp rx=0 tx=0).stop 2024-04-20T21:18:53.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.898+0000 7faf1de00640 1 -- 172.21.15.86:0/1384677890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf18144e90 msgr2=0x7faf180cf3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:53.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.898+0000 7faf1de00640 1 --2- 172.21.15.86:0/1384677890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf18144e90 0x7faf180cf3e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7faf1000c7a0 tx=0x7faf1000cc60 comp rx=0 tx=0).stop 2024-04-20T21:18:53.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.898+0000 7faf1de00640 1 -- 172.21.15.86:0/1384677890 shutdown_connections 2024-04-20T21:18:53.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.898+0000 7faf1de00640 1 --2- 172.21.15.86:0/1384677890 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7faef406c8f0 0x7faef406edb0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:53.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.898+0000 7faf1de00640 1 --2- 172.21.15.86:0/1384677890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf18144e90 0x7faf180cf3e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:53.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.898+0000 7faf1de00640 1 --2- 172.21.15.86:0/1384677890 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faf180af0b0 0x7faf180ceea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:53.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.898+0000 7faf1de00640 1 --2- 172.21.15.86:0/1384677890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf1806f140 0x7faf1813ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:53.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.898+0000 7faf1de00640 1 -- 172.21.15.86:0/1384677890 >> 172.21.15.86:0/1384677890 conn(0x7faf180b2360 msgr2=0x7faf1806e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:53.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.898+0000 7faf1de00640 1 -- 172.21.15.86:0/1384677890 shutdown_connections 2024-04-20T21:18:53.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:53.898+0000 7faf1de00640 1 -- 172.21.15.86:0/1384677890 wait complete. 2024-04-20T21:18:53.914 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:18:54.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.108+0000 7f41c2200640 1 Processor -- start 2024-04-20T21:18:54.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.108+0000 7f41c2200640 1 -- start start 2024-04-20T21:18:54.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.109+0000 7f41c2200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f41bc0af8d0 0x7f41bc141f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:54.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.109+0000 7f41c2200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f41bc162c40 0x7f41bc0b3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:54.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.110+0000 7f41c2200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f41bc148050 0x7f41bc0b3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:54.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.110+0000 7f41c2200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f41bc142490 con 0x7f41bc0af8d0 2024-04-20T21:18:54.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.110+0000 7f41c2200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f41bc071c10 con 0x7f41bc162c40 2024-04-20T21:18:54.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.110+0000 7f41c2200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f41bc15eaf0 con 0x7f41bc148050 2024-04-20T21:18:54.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.110+0000 7f41c0e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f41bc0af8d0 0x7f41bc141f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:54.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.110+0000 7f41c0e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f41bc0af8d0 0x7f41bc141f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37388/0 (socket says 172.21.15.86:37388) 2024-04-20T21:18:54.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.110+0000 7f41c0e00640 1 -- 172.21.15.86:0/342843963 learned_addr learned my addr 172.21.15.86:0/342843963 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:54.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.110+0000 7f41c0e00640 1 -- 172.21.15.86:0/342843963 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f41bc148050 msgr2=0x7f41bc0b3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:54.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.110+0000 7f41c0e00640 1 --2- 172.21.15.86:0/342843963 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f41bc148050 0x7f41bc0b3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:54.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.110+0000 7f41c0e00640 1 -- 172.21.15.86:0/342843963 >> v2:172.21.15.144:3300/0 conn(0x7f41bc162c40 msgr2=0x7f41bc0b3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:54.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.110+0000 7f41c0e00640 1 --2- 172.21.15.86:0/342843963 >> v2:172.21.15.144:3300/0 conn(0x7f41bc162c40 0x7f41bc0b3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:54.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.110+0000 7f41c0e00640 1 -- 172.21.15.86:0/342843963 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41bc071eb0 con 0x7f41bc0af8d0 2024-04-20T21:18:54.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.111+0000 7f41c0e00640 1 --2- 172.21.15.86:0/342843963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41bc0af8d0 0x7f41bc141f50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f41bc09e430 tx=0x7f41b002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb387df1e6fc7f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:54.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.112+0000 7f41bb400640 1 -- 172.21.15.86:0/342843963 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b002e070 con 0x7f41bc0af8d0 2024-04-20T21:18:54.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.112+0000 7f41bb400640 1 -- 172.21.15.86:0/342843963 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41b0037070 con 0x7f41bc0af8d0 2024-04-20T21:18:54.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.112+0000 7f41bb400640 1 -- 172.21.15.86:0/342843963 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b0031a00 con 0x7f41bc0af8d0 2024-04-20T21:18:54.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.112+0000 7f41c2200640 1 -- 172.21.15.86:0/342843963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41bc0af8d0 msgr2=0x7f41bc141f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:54.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.112+0000 7f41c2200640 1 --2- 172.21.15.86:0/342843963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41bc0af8d0 0x7f41bc141f50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f41bc09e430 tx=0x7f41b002b880 comp rx=0 tx=0).stop 2024-04-20T21:18:54.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.113+0000 7f41c2200640 1 -- 172.21.15.86:0/342843963 shutdown_connections 2024-04-20T21:18:54.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.113+0000 7f41c2200640 1 --2- 172.21.15.86:0/342843963 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f41bc148050 0x7f41bc0b3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:54.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.113+0000 7f41c2200640 1 --2- 172.21.15.86:0/342843963 >> v2:172.21.15.144:3300/0 conn(0x7f41bc162c40 0x7f41bc0b3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:54.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.113+0000 7f41c2200640 1 --2- 172.21.15.86:0/342843963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41bc0af8d0 0x7f41bc141f50 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:54.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.113+0000 7f41c2200640 1 -- 172.21.15.86:0/342843963 >> 172.21.15.86:0/342843963 conn(0x7f41bc0d0170 msgr2=0x7f41bc0d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:54.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.113+0000 7f41c2200640 1 -- 172.21.15.86:0/342843963 shutdown_connections 2024-04-20T21:18:54.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.113+0000 7f41c2200640 1 -- 172.21.15.86:0/342843963 wait complete. 2024-04-20T21:18:54.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.114+0000 7f41c2200640 1 Processor -- start 2024-04-20T21:18:54.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.115+0000 7f41c2200640 1 -- start start 2024-04-20T21:18:54.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.115+0000 7f41c2200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41bc0af8d0 0x7f41bc161640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:54.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.115+0000 7f41c2200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41bc148050 0x7f41bc15be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:54.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.115+0000 7f41c2200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f41bc162c40 0x7f41bc161f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:54.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.115+0000 7f41c2200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f41bc163100 con 0x7f41bc0af8d0 2024-04-20T21:18:54.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.115+0000 7f41c2200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f41bc072470 con 0x7f41bc148050 2024-04-20T21:18:54.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.115+0000 7f41c2200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f41bc147a00 con 0x7f41bc162c40 2024-04-20T21:18:54.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.116+0000 7f41c0e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41bc0af8d0 0x7f41bc161640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:54.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.116+0000 7f41c0e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41bc0af8d0 0x7f41bc161640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37392/0 (socket says 172.21.15.86:37392) 2024-04-20T21:18:54.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.116+0000 7f41c0e00640 1 -- 172.21.15.86:0/1510277130 learned_addr learned my addr 172.21.15.86:0/1510277130 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:54.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.116+0000 7f41c1800640 1 --2- 172.21.15.86:0/1510277130 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f41bc162c40 0x7f41bc161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:54.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.116+0000 7f41c0e00640 1 -- 172.21.15.86:0/1510277130 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f41bc162c40 msgr2=0x7f41bc161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:54.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.116+0000 7f41c0e00640 1 --2- 172.21.15.86:0/1510277130 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f41bc162c40 0x7f41bc161f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:54.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.116+0000 7f41bbe00640 1 --2- 172.21.15.86:0/1510277130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41bc148050 0x7f41bc15be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:54.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.116+0000 7f41c0e00640 1 -- 172.21.15.86:0/1510277130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41bc148050 msgr2=0x7f41bc15be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:54.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.116+0000 7f41c0e00640 1 --2- 172.21.15.86:0/1510277130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41bc148050 0x7f41bc15be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:54.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.116+0000 7f41c0e00640 1 -- 172.21.15.86:0/1510277130 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41bc15c3a0 con 0x7f41bc0af8d0 2024-04-20T21:18:54.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.116+0000 7f41bbe00640 1 --2- 172.21.15.86:0/1510277130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41bc148050 0x7f41bc15be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:54.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.117+0000 7f41c0e00640 1 --2- 172.21.15.86:0/1510277130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41bc0af8d0 0x7f41bc161640 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f41b0000c00 tx=0x7f41b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:54.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.117+0000 7f41b8c00640 1 -- 172.21.15.86:0/1510277130 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b0032880 con 0x7f41bc0af8d0 2024-04-20T21:18:54.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.117+0000 7f41b8c00640 1 -- 172.21.15.86:0/1510277130 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41b0032a20 con 0x7f41bc0af8d0 2024-04-20T21:18:54.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.117+0000 7f41b8c00640 1 -- 172.21.15.86:0/1510277130 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b0032e10 con 0x7f41bc0af8d0 2024-04-20T21:18:54.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.117+0000 7f41c2200640 1 -- 172.21.15.86:0/1510277130 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41bc1f4fd0 con 0x7f41bc0af8d0 2024-04-20T21:18:54.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.117+0000 7f41c2200640 1 -- 172.21.15.86:0/1510277130 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41bc160e40 con 0x7f41bc0af8d0 2024-04-20T21:18:54.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.118+0000 7f41c2200640 1 -- 172.21.15.86:0/1510277130 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41bc04fcf0 con 0x7f41bc0af8d0 2024-04-20T21:18:54.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.119+0000 7f41b8c00640 1 -- 172.21.15.86:0/1510277130 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f41b004c060 con 0x7f41bc0af8d0 2024-04-20T21:18:54.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.120+0000 7f41b8c00640 1 --2- 172.21.15.86:0/1510277130 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f419c06c920 0x7f419c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:54.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.120+0000 7f41b8c00640 1 -- 172.21.15.86:0/1510277130 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f41b00ad480 con 0x7f41bc0af8d0 2024-04-20T21:18:54.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.120+0000 7f41bbe00640 1 --2- 172.21.15.86:0/1510277130 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f419c06c920 0x7f419c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:54.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.121+0000 7f41bbe00640 1 --2- 172.21.15.86:0/1510277130 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f419c06c920 0x7f419c06ede0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f41bc06f050 tx=0x7f41a8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:54.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.124+0000 7f41b8c00640 1 -- 172.21.15.86:0/1510277130 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f41bc15c3a0 con 0x7f41bc0af8d0 2024-04-20T21:18:54.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.324+0000 7f41c2200640 1 -- 172.21.15.86:0/1510277130 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f41bc0b35b0 con 0x7f41bc0af8d0 2024-04-20T21:18:54.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.327+0000 7f41b8c00640 1 -- 172.21.15.86:0/1510277130 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16854 (secure 0 0 0) 0x7f41b0040070 con 0x7f41bc0af8d0 2024-04-20T21:18:54.326 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:18:54.327 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":22,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:18:33.788155+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:17:49.718799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:17:50.683477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:18:54.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.329+0000 7f41c2200640 1 -- 172.21.15.86:0/1510277130 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f419c06c920 msgr2=0x7f419c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:54.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.329+0000 7f41c2200640 1 --2- 172.21.15.86:0/1510277130 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f419c06c920 0x7f419c06ede0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f41bc06f050 tx=0x7f41a8008040 comp rx=0 tx=0).stop 2024-04-20T21:18:54.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.329+0000 7f41c2200640 1 -- 172.21.15.86:0/1510277130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41bc0af8d0 msgr2=0x7f41bc161640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:54.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.329+0000 7f41c2200640 1 --2- 172.21.15.86:0/1510277130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41bc0af8d0 0x7f41bc161640 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f41b0000c00 tx=0x7f41b00046c0 comp rx=0 tx=0).stop 2024-04-20T21:18:54.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.329+0000 7f41c2200640 1 -- 172.21.15.86:0/1510277130 shutdown_connections 2024-04-20T21:18:54.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.329+0000 7f41c2200640 1 --2- 172.21.15.86:0/1510277130 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f419c06c920 0x7f419c06ede0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:54.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.329+0000 7f41c2200640 1 --2- 172.21.15.86:0/1510277130 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f41bc162c40 0x7f41bc161f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:54.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.329+0000 7f41c2200640 1 --2- 172.21.15.86:0/1510277130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41bc148050 0x7f41bc15be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:54.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.329+0000 7f41c2200640 1 --2- 172.21.15.86:0/1510277130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41bc0af8d0 0x7f41bc161640 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:54.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.329+0000 7f41c2200640 1 -- 172.21.15.86:0/1510277130 >> 172.21.15.86:0/1510277130 conn(0x7f41bc0d0170 msgr2=0x7f41bc142e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:54.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.329+0000 7f41c2200640 1 -- 172.21.15.86:0/1510277130 shutdown_connections 2024-04-20T21:18:54.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.330+0000 7f41c2200640 1 -- 172.21.15.86:0/1510277130 wait complete. 2024-04-20T21:18:54.347 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:17:49.718799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:17:50.683477+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:18:54.347 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:18:54.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.546+0000 7f1d43a00640 1 Processor -- start 2024-04-20T21:18:54.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.546+0000 7f1d43a00640 1 -- start start 2024-04-20T21:18:54.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.548+0000 7f1d43a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1d3c15eb30 0x7f1d3c0b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:54.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.548+0000 7f1d43a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1d3c06c090 0x7f1d3c0b4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:54.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.548+0000 7f1d43a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d3c0adb10 con 0x7f1d3c0aee50 2024-04-20T21:18:54.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.548+0000 7f1d43a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f1d3c0af7d0 con 0x7f1d3c06c090 2024-04-20T21:18:54.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.548+0000 7f1d43a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1d3c0addb0 con 0x7f1d3c15eb30 2024-04-20T21:18:54.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.548+0000 7f1d41000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1d3c15eb30 0x7f1d3c0b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:54.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.548+0000 7f1d41000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1d3c15eb30 0x7f1d3c0b3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:34890/0 (socket says 172.21.15.86:34890) 2024-04-20T21:18:54.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.548+0000 7f1d41000640 1 -- 172.21.15.86:0/3329529238 learned_addr learned my addr 172.21.15.86:0/3329529238 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:54.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.548+0000 7f1d41a00640 1 --2- 172.21.15.86:0/3329529238 >> v2:172.21.15.144:3300/0 conn(0x7f1d3c06c090 0x7f1d3c0b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:54.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.549+0000 7f1d41a00640 1 -- 172.21.15.86:0/3329529238 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1d3c15eb30 msgr2=0x7f1d3c0b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:54.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.549+0000 7f1d41a00640 1 --2- 172.21.15.86:0/3329529238 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1d3c15eb30 0x7f1d3c0b3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:54.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.549+0000 7f1d41a00640 1 -- 172.21.15.86:0/3329529238 >> v1:172.21.15.144:6789/0 conn(0x7f1d3c0aee50 legacy=0x7f1d3c06b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:54.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.549+0000 7f1d41a00640 1 -- 172.21.15.86:0/3329529238 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d3c0ae090 con 0x7f1d3c06c090 2024-04-20T21:18:54.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.549+0000 7f1d41000640 1 --2- 172.21.15.86:0/3329529238 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1d3c15eb30 0x7f1d3c0b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:18:54.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.551+0000 7f1d41a00640 1 --2- 172.21.15.86:0/3329529238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d3c06c090 0x7f1d3c0b4360 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1d34009d90 tx=0x7f1d3402bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77e93a65824e9515 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:54.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.551+0000 7f1d3b400640 1 -- 172.21.15.86:0/3329529238 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d34039070 con 0x7f1d3c06c090 2024-04-20T21:18:54.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.551+0000 7f1d3b400640 1 -- 172.21.15.86:0/3329529238 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d3402cc50 con 0x7f1d3c06c090 2024-04-20T21:18:54.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.551+0000 7f1d3b400640 1 -- 172.21.15.86:0/3329529238 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d34034440 con 0x7f1d3c06c090 2024-04-20T21:18:54.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.552+0000 7f1d43a00640 1 -- 172.21.15.86:0/3329529238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d3c06c090 msgr2=0x7f1d3c0b4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:54.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.552+0000 7f1d43a00640 1 --2- 172.21.15.86:0/3329529238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d3c06c090 0x7f1d3c0b4360 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1d34009d90 tx=0x7f1d3402bea0 comp rx=0 tx=0).stop 2024-04-20T21:18:54.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.552+0000 7f1d43a00640 1 -- 172.21.15.86:0/3329529238 shutdown_connections 2024-04-20T21:18:54.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.552+0000 7f1d43a00640 1 --2- 172.21.15.86:0/3329529238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d3c06c090 0x7f1d3c0b4360 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:54.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.552+0000 7f1d43a00640 1 --2- 172.21.15.86:0/3329529238 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1d3c15eb30 0x7f1d3c0b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:54.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.552+0000 7f1d43a00640 1 -- 172.21.15.86:0/3329529238 >> 172.21.15.86:0/3329529238 conn(0x7f1d3c0b2110 msgr2=0x7f1d3c0b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:54.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.552+0000 7f1d43a00640 1 -- 172.21.15.86:0/3329529238 shutdown_connections 2024-04-20T21:18:54.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.552+0000 7f1d43a00640 1 -- 172.21.15.86:0/3329529238 wait complete. 2024-04-20T21:18:54.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.554+0000 7f1d43a00640 1 Processor -- start 2024-04-20T21:18:54.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.554+0000 7f1d43a00640 1 -- start start 2024-04-20T21:18:54.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.555+0000 7f1d43a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d3c06c090 0x7f1d3c14dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:54.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.555+0000 7f1d43a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1d3c0aee50 0x7f1d3c15b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:54.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.555+0000 7f1d43a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d3c15eb30 0x7f1d3c15b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:54.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.555+0000 7f1d43a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1d3c0ae930 con 0x7f1d3c15eb30 2024-04-20T21:18:54.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.555+0000 7f1d43a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1d3c0ae650 con 0x7f1d3c06c090 2024-04-20T21:18:54.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.555+0000 7f1d43a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1d3c06e160 con 0x7f1d3c0aee50 2024-04-20T21:18:54.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.555+0000 7f1d41a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d3c15eb30 0x7f1d3c15b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:54.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.555+0000 7f1d41a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d3c15eb30 0x7f1d3c15b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37404/0 (socket says 172.21.15.86:37404) 2024-04-20T21:18:54.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.555+0000 7f1d41a00640 1 -- 172.21.15.86:0/2845662640 learned_addr learned my addr 172.21.15.86:0/2845662640 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:54.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.555+0000 7f1d41000640 1 --2- 172.21.15.86:0/2845662640 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d3c06c090 0x7f1d3c14dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:54.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.555+0000 7f1d3be00640 1 --2- 172.21.15.86:0/2845662640 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1d3c0aee50 0x7f1d3c15b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:54.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.555+0000 7f1d41a00640 1 -- 172.21.15.86:0/2845662640 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1d3c0aee50 msgr2=0x7f1d3c15b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:54.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.555+0000 7f1d41a00640 1 --2- 172.21.15.86:0/2845662640 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1d3c0aee50 0x7f1d3c15b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:54.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.555+0000 7f1d41a00640 1 -- 172.21.15.86:0/2845662640 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d3c06c090 msgr2=0x7f1d3c14dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:54.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.555+0000 7f1d41a00640 1 --2- 172.21.15.86:0/2845662640 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d3c06c090 0x7f1d3c14dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:54.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.555+0000 7f1d41a00640 1 -- 172.21.15.86:0/2845662640 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d3c1f4c30 con 0x7f1d3c15eb30 2024-04-20T21:18:54.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.556+0000 7f1d41a00640 1 --2- 172.21.15.86:0/2845662640 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d3c15eb30 0x7f1d3c15b6e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1d34009eb0 tx=0x7f1d34004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:54.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.556+0000 7f1d38c00640 1 -- 172.21.15.86:0/2845662640 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d34039050 con 0x7f1d3c15eb30 2024-04-20T21:18:54.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.556+0000 7f1d38c00640 1 -- 172.21.15.86:0/2845662640 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d34045070 con 0x7f1d3c15eb30 2024-04-20T21:18:54.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.556+0000 7f1d38c00640 1 -- 172.21.15.86:0/2845662640 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d34009450 con 0x7f1d3c15eb30 2024-04-20T21:18:54.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.556+0000 7f1d43a00640 1 -- 172.21.15.86:0/2845662640 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d3c1f4f10 con 0x7f1d3c15eb30 2024-04-20T21:18:54.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.556+0000 7f1d43a00640 1 -- 172.21.15.86:0/2845662640 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d3c1f54d0 con 0x7f1d3c15eb30 2024-04-20T21:18:54.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.557+0000 7f1d43a00640 1 -- 172.21.15.86:0/2845662640 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d3c0689e0 con 0x7f1d3c15eb30 2024-04-20T21:18:54.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.558+0000 7f1d38c00640 1 -- 172.21.15.86:0/2845662640 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1d3403e610 con 0x7f1d3c15eb30 2024-04-20T21:18:54.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.559+0000 7f1d38c00640 1 --2- 172.21.15.86:0/2845662640 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1d1c06c8f0 0x7f1d1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:54.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.559+0000 7f1d38c00640 1 -- 172.21.15.86:0/2845662640 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f1d340ae630 con 0x7f1d3c15eb30 2024-04-20T21:18:54.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.560+0000 7f1d41000640 1 --2- 172.21.15.86:0/2845662640 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1d1c06c8f0 0x7f1d1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:54.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.560+0000 7f1d41000640 1 --2- 172.21.15.86:0/2845662640 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1d1c06c8f0 0x7f1d1c06edb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c156b10 tx=0x7f1d30008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:54.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.563+0000 7f1d38c00640 1 -- 172.21.15.86:0/2845662640 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1d3407c9e0 con 0x7f1d3c15eb30 2024-04-20T21:18:54.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.762+0000 7f1d43a00640 1 -- 172.21.15.86:0/2845662640 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1d3c15c820 con 0x7f1d3c15eb30 2024-04-20T21:18:54.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.763+0000 7f1d38c00640 1 -- 172.21.15.86:0/2845662640 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f1d3407c2e0 con 0x7f1d3c15eb30 2024-04-20T21:18:54.762 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:18:54.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.765+0000 7f1d43a00640 1 -- 172.21.15.86:0/2845662640 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1d1c06c8f0 msgr2=0x7f1d1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:54.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.765+0000 7f1d43a00640 1 --2- 172.21.15.86:0/2845662640 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1d1c06c8f0 0x7f1d1c06edb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c156b10 tx=0x7f1d30008040 comp rx=0 tx=0).stop 2024-04-20T21:18:54.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.765+0000 7f1d43a00640 1 -- 172.21.15.86:0/2845662640 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d3c15eb30 msgr2=0x7f1d3c15b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:54.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.765+0000 7f1d43a00640 1 --2- 172.21.15.86:0/2845662640 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d3c15eb30 0x7f1d3c15b6e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1d34009eb0 tx=0x7f1d34004a40 comp rx=0 tx=0).stop 2024-04-20T21:18:54.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.765+0000 7f1d43a00640 1 -- 172.21.15.86:0/2845662640 shutdown_connections 2024-04-20T21:18:54.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.765+0000 7f1d43a00640 1 --2- 172.21.15.86:0/2845662640 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1d1c06c8f0 0x7f1d1c06edb0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:54.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.765+0000 7f1d43a00640 1 --2- 172.21.15.86:0/2845662640 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d3c15eb30 0x7f1d3c15b6e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:54.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.765+0000 7f1d43a00640 1 --2- 172.21.15.86:0/2845662640 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1d3c0aee50 0x7f1d3c15b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:54.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.765+0000 7f1d43a00640 1 --2- 172.21.15.86:0/2845662640 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d3c06c090 0x7f1d3c14dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:54.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.765+0000 7f1d43a00640 1 -- 172.21.15.86:0/2845662640 >> 172.21.15.86:0/2845662640 conn(0x7f1d3c0b2110 msgr2=0x7f1d3c15dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:54.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.766+0000 7f1d43a00640 1 -- 172.21.15.86:0/2845662640 shutdown_connections 2024-04-20T21:18:54.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.766+0000 7f1d43a00640 1 -- 172.21.15.86:0/2845662640 wait complete. 2024-04-20T21:18:54.782 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:18:54.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.988+0000 7f32cbe00640 1 Processor -- start 2024-04-20T21:18:54.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.988+0000 7f32cbe00640 1 -- start start 2024-04-20T21:18:54.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.990+0000 7f32cbe00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f32cc0d08f0 0x7f32cc0d3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:54.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.990+0000 7f32cbe00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32cc0cf120 con 0x7f32cc0d45a0 2024-04-20T21:18:54.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.990+0000 7f32cbe00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32cc06de70 con 0x7f32cc15ea70 2024-04-20T21:18:54.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.990+0000 7f32cbe00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f32cc06e150 con 0x7f32cc0d08f0 2024-04-20T21:18:54.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.990+0000 7f32caa00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f32cc0d45a0 0x7f32cc0d4980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42658/0 (socket says 172.21.15.86:42658) 2024-04-20T21:18:54.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.990+0000 7f32caa00640 1 -- 172.21.15.86:0/1911225776 learned_addr learned my addr 172.21.15.86:0/1911225776 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:18:54.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.991+0000 7f32c9600640 1 -- 172.21.15.86:0/1911225776 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2941208191 0 0) 0x7f32cc0cf120 con 0x7f32cc0d45a0 2024-04-20T21:18:54.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.991+0000 7f32c9600640 1 -- 172.21.15.86:0/1911225776 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32ac003520 con 0x7f32cc0d45a0 2024-04-20T21:18:54.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.991+0000 7f32cb400640 1 --2- 172.21.15.86:0/1911225776 >> v2:172.21.15.144:3300/0 conn(0x7f32cc0d08f0 0x7f32cc0d3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:54.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.991+0000 7f32c9600640 1 -- 172.21.15.86:0/1911225776 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2389417822 0 0) 0x7f32ac003520 con 0x7f32cc0d45a0 2024-04-20T21:18:54.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.991+0000 7f32c9600640 1 -- 172.21.15.86:0/1911225776 >> v2:172.21.15.144:3300/0 conn(0x7f32cc0d08f0 msgr2=0x7f32cc0d3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:54.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.991+0000 7f32c9600640 1 --2- 172.21.15.86:0/1911225776 >> v2:172.21.15.144:3300/0 conn(0x7f32cc0d08f0 0x7f32cc0d3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:54.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.991+0000 7f32c9600640 1 -- 172.21.15.86:0/1911225776 >> v1:172.21.15.144:6789/0 conn(0x7f32cc15ea70 legacy=0x7f32cc06d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:54.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.991+0000 7f32c9600640 1 -- 172.21.15.86:0/1911225776 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32cc06e430 con 0x7f32cc0d45a0 2024-04-20T21:18:54.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.991+0000 7f32c9600640 1 -- 172.21.15.86:0/1911225776 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f32bc001670 con 0x7f32cc0d45a0 2024-04-20T21:18:54.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.991+0000 7f32c9600640 1 -- 172.21.15.86:0/1911225776 >> v1:172.21.15.86:6789/0 conn(0x7f32cc0d45a0 legacy=0x7f32cc0d4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:54.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.991+0000 7f32c9600640 1 --2- 172.21.15.86:0/1911225776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32ac003a70 0x7f32ac003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:54.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.991+0000 7f32c9600640 1 --2- 172.21.15.86:0/1911225776 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32ac004800 0x7f32ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:54.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.991+0000 7f32c9600640 1 --2- 172.21.15.86:0/1911225776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32ac0053c0 0x7f32ac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:54.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.991+0000 7f32c9600640 1 -- 172.21.15.86:0/1911225776 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc06e150 con 0x7f32ac0053c0 2024-04-20T21:18:54.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.991+0000 7f32c9600640 1 -- 172.21.15.86:0/1911225776 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac00c280 con 0x7f32ac003a70 2024-04-20T21:18:54.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.991+0000 7f32c9600640 1 -- 172.21.15.86:0/1911225776 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f32ac00c480 con 0x7f32ac004800 2024-04-20T21:18:54.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.992+0000 7f32cb400640 1 --2- 172.21.15.86:0/1911225776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32ac0053c0 0x7f32ac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:54.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.992+0000 7f32caa00640 1 --2- 172.21.15.86:0/1911225776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32ac003a70 0x7f32ac003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:54.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.992+0000 7f32ca000640 1 --2- 172.21.15.86:0/1911225776 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32ac004800 0x7f32ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:54.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.992+0000 7f32cb400640 1 -- 172.21.15.86:0/1911225776 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32ac004800 msgr2=0x7f32ac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:54.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.992+0000 7f32cb400640 1 --2- 172.21.15.86:0/1911225776 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32ac004800 0x7f32ac004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:54.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.992+0000 7f32cb400640 1 -- 172.21.15.86:0/1911225776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32ac003a70 msgr2=0x7f32ac003eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:54.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.992+0000 7f32cb400640 1 --2- 172.21.15.86:0/1911225776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32ac003a70 0x7f32ac003eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:54.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.992+0000 7f32cb400640 1 -- 172.21.15.86:0/1911225776 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32ac003520 con 0x7f32ac0053c0 2024-04-20T21:18:54.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.993+0000 7f32cb400640 1 -- 172.21.15.86:0/1911225776 reap_dead start 2024-04-20T21:18:54.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.993+0000 7f32cb400640 1 --2- 172.21.15.86:0/1911225776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32ac0053c0 0x7f32ac00b9c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f32c400ba30 tx=0x7f32ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e29c492f7aa430d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:54.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.993+0000 7f32c9600640 1 -- 172.21.15.86:0/1911225776 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c4036030 con 0x7f32ac0053c0 2024-04-20T21:18:54.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.993+0000 7f32c9600640 1 -- 172.21.15.86:0/1911225776 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32c4004020 con 0x7f32ac0053c0 2024-04-20T21:18:54.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.994+0000 7f32c9600640 1 -- 172.21.15.86:0/1911225776 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c400b600 con 0x7f32ac0053c0 2024-04-20T21:18:54.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.994+0000 7f32cbe00640 1 -- 172.21.15.86:0/1911225776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32ac0053c0 msgr2=0x7f32ac00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:54.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.994+0000 7f32cbe00640 1 --2- 172.21.15.86:0/1911225776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32ac0053c0 0x7f32ac00b9c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f32c400ba30 tx=0x7f32ac005830 comp rx=0 tx=0).stop 2024-04-20T21:18:54.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.994+0000 7f32cbe00640 1 -- 172.21.15.86:0/1911225776 shutdown_connections 2024-04-20T21:18:54.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.994+0000 7f32cbe00640 1 --2- 172.21.15.86:0/1911225776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32ac0053c0 0x7f32ac00b9c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:54.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.994+0000 7f32cbe00640 1 -- 172.21.15.86:0/1911225776 >> 172.21.15.86:0/1911225776 conn(0x7f32cc09d9d0 msgr2=0x7f32cc08bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:54.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.994+0000 7f32cbe00640 1 -- 172.21.15.86:0/1911225776 shutdown_connections 2024-04-20T21:18:54.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.994+0000 7f32cbe00640 1 -- 172.21.15.86:0/1911225776 wait complete. 2024-04-20T21:18:54.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.996+0000 7f32cbe00640 1 Processor -- start 2024-04-20T21:18:54.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.996+0000 7f32cbe00640 1 -- start start 2024-04-20T21:18:54.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.997+0000 7f32cbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32ac0053c0 0x7f32cc14bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:54.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.997+0000 7f32cbe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32cc0b7200 0x7f32cc0b4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:54.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.997+0000 7f32cbe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32cc0b7680 0x7f32cc0b4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:54.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.997+0000 7f32cbe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc0d33e0 con 0x7f32ac0053c0 2024-04-20T21:18:54.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.997+0000 7f32cbe00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc0d3260 con 0x7f32cc0b7680 2024-04-20T21:18:54.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.997+0000 7f32cbe00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f32cc0d3560 con 0x7f32cc0b7200 2024-04-20T21:18:54.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.997+0000 7f32caa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32ac0053c0 0x7f32cc14bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:54.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.997+0000 7f32caa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32ac0053c0 0x7f32cc14bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37412/0 (socket says 172.21.15.86:37412) 2024-04-20T21:18:54.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.997+0000 7f32caa00640 1 -- 172.21.15.86:0/2040697586 learned_addr learned my addr 172.21.15.86:0/2040697586 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:54.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.998+0000 7f32ca000640 1 --2- 172.21.15.86:0/2040697586 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32cc0b7200 0x7f32cc0b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:54.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.998+0000 7f32caa00640 1 -- 172.21.15.86:0/2040697586 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32cc0b7200 msgr2=0x7f32cc0b4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:54.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.998+0000 7f32caa00640 1 --2- 172.21.15.86:0/2040697586 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32cc0b7200 0x7f32cc0b4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:54.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.998+0000 7f32cb400640 1 --2- 172.21.15.86:0/2040697586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32cc0b7680 0x7f32cc0b4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:54.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.998+0000 7f32caa00640 1 -- 172.21.15.86:0/2040697586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32cc0b7680 msgr2=0x7f32cc0b4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:54.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.998+0000 7f32caa00640 1 --2- 172.21.15.86:0/2040697586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32cc0b7680 0x7f32cc0b4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:55.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.998+0000 7f32caa00640 1 -- 172.21.15.86:0/2040697586 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32cc0b9ad0 con 0x7f32ac0053c0 2024-04-20T21:18:55.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.998+0000 7f32caa00640 1 --2- 172.21.15.86:0/2040697586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32ac0053c0 0x7f32cc14bb00 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f32bc00e780 tx=0x7f32bc00ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:55.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.998+0000 7f32c2a00640 1 -- 172.21.15.86:0/2040697586 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc00c660 con 0x7f32ac0053c0 2024-04-20T21:18:55.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.998+0000 7f32c2a00640 1 -- 172.21.15.86:0/2040697586 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32bc018070 con 0x7f32ac0053c0 2024-04-20T21:18:55.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.998+0000 7f32c2a00640 1 -- 172.21.15.86:0/2040697586 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc00ca20 con 0x7f32ac0053c0 2024-04-20T21:18:55.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.998+0000 7f32cbe00640 1 -- 172.21.15.86:0/2040697586 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32cc0b9db0 con 0x7f32ac0053c0 2024-04-20T21:18:55.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:54.999+0000 7f32cbe00640 1 -- 172.21.15.86:0/2040697586 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32cc0cbaa0 con 0x7f32ac0053c0 2024-04-20T21:18:55.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.000+0000 7f32cbe00640 1 -- 172.21.15.86:0/2040697586 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32cc04fd90 con 0x7f32ac0053c0 2024-04-20T21:18:55.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.001+0000 7f32c2a00640 1 -- 172.21.15.86:0/2040697586 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f32bc00cbc0 con 0x7f32ac0053c0 2024-04-20T21:18:55.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.002+0000 7f32c2a00640 1 --2- 172.21.15.86:0/2040697586 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f32a806c8f0 0x7f32a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:55.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.002+0000 7f32c2a00640 1 -- 172.21.15.86:0/2040697586 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f32bc08b970 con 0x7f32ac0053c0 2024-04-20T21:18:55.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.002+0000 7f32ca000640 1 --2- 172.21.15.86:0/2040697586 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f32a806c8f0 0x7f32a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:55.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.003+0000 7f32ca000640 1 --2- 172.21.15.86:0/2040697586 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f32a806c8f0 0x7f32a806edb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f32cc0d1390 tx=0x7f32b4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:55.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.007+0000 7f32c2a00640 1 -- 172.21.15.86:0/2040697586 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f32bc059d20 con 0x7f32ac0053c0 2024-04-20T21:18:55.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.209+0000 7f32cbe00640 1 -- 172.21.15.86:0/2040697586 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f32cc0d4fe0 con 0x7f32ac0053c0 2024-04-20T21:18:55.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.210+0000 7f32c2a00640 1 -- 172.21.15.86:0/2040697586 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f32bc059620 con 0x7f32ac0053c0 2024-04-20T21:18:55.209 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-20T21:18:55.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.212+0000 7f32cbe00640 1 -- 172.21.15.86:0/2040697586 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f32a806c8f0 msgr2=0x7f32a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:55.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.212+0000 7f32cbe00640 1 --2- 172.21.15.86:0/2040697586 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f32a806c8f0 0x7f32a806edb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f32cc0d1390 tx=0x7f32b4009040 comp rx=0 tx=0).stop 2024-04-20T21:18:55.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.212+0000 7f32cbe00640 1 -- 172.21.15.86:0/2040697586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32ac0053c0 msgr2=0x7f32cc14bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:55.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.212+0000 7f32cbe00640 1 --2- 172.21.15.86:0/2040697586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32ac0053c0 0x7f32cc14bb00 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f32bc00e780 tx=0x7f32bc00ec40 comp rx=0 tx=0).stop 2024-04-20T21:18:55.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.213+0000 7f32cbe00640 1 -- 172.21.15.86:0/2040697586 shutdown_connections 2024-04-20T21:18:55.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.213+0000 7f32cbe00640 1 --2- 172.21.15.86:0/2040697586 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f32a806c8f0 0x7f32a806edb0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:55.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.213+0000 7f32cbe00640 1 --2- 172.21.15.86:0/2040697586 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32cc0b7680 0x7f32cc0b4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:55.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.213+0000 7f32cbe00640 1 --2- 172.21.15.86:0/2040697586 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32cc0b7200 0x7f32cc0b4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:55.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.213+0000 7f32cbe00640 1 --2- 172.21.15.86:0/2040697586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32ac0053c0 0x7f32cc14bb00 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:55.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.213+0000 7f32cbe00640 1 -- 172.21.15.86:0/2040697586 >> 172.21.15.86:0/2040697586 conn(0x7f32cc09d9d0 msgr2=0x7f32cc0b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:55.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.213+0000 7f32cbe00640 1 -- 172.21.15.86:0/2040697586 shutdown_connections 2024-04-20T21:18:55.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.213+0000 7f32cbe00640 1 -- 172.21.15.86:0/2040697586 wait complete. 2024-04-20T21:18:55.232 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:18:55.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.440+0000 7f237ea00640 1 Processor -- start 2024-04-20T21:18:55.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.440+0000 7f237ea00640 1 -- start start 2024-04-20T21:18:55.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.442+0000 7f237ea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f237806d9c0 0x7f237806ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:55.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.442+0000 7f237ea00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f237806de80 con 0x7f23780af1f0 2024-04-20T21:18:55.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.442+0000 7f237ea00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f237806e160 con 0x7f23780af5d0 2024-04-20T21:18:55.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.442+0000 7f237ea00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f237806f460 con 0x7f237806d9c0 2024-04-20T21:18:55.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.442+0000 7f237d600640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f23780af1f0 0x7f237815d520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42666/0 (socket says 172.21.15.86:42666) 2024-04-20T21:18:55.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.442+0000 7f237d600640 1 -- 172.21.15.86:0/162488774 learned_addr learned my addr 172.21.15.86:0/162488774 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:18:55.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.442+0000 7f237cc00640 1 --2- 172.21.15.86:0/162488774 >> v2:172.21.15.86:3300/0 conn(0x7f237806d9c0 0x7f237806ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:55.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.443+0000 7f2377e00640 1 -- 172.21.15.86:0/162488774 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 512910871 0 0) 0x7f237806de80 con 0x7f23780af1f0 2024-04-20T21:18:55.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.443+0000 7f2377e00640 1 -- 172.21.15.86:0/162488774 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2360003520 con 0x7f23780af1f0 2024-04-20T21:18:55.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.443+0000 7f237cc00640 1 -- 172.21.15.86:0/162488774 >> v1:172.21.15.144:6789/0 conn(0x7f23780af5d0 legacy=0x7f237815de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:55.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.443+0000 7f237cc00640 1 -- 172.21.15.86:0/162488774 >> v1:172.21.15.86:6789/0 conn(0x7f23780af1f0 legacy=0x7f237815d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:55.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.443+0000 7f237cc00640 1 -- 172.21.15.86:0/162488774 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f237806e440 con 0x7f237806d9c0 2024-04-20T21:18:55.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.444+0000 7f237cc00640 1 --2- 172.21.15.86:0/162488774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f237806d9c0 0x7f237806ef20 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f23640097a0 tx=0x7f236402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aec9ec665a59899c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:55.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.444+0000 7f2377e00640 1 -- 172.21.15.86:0/162488774 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2364035020 con 0x7f237806d9c0 2024-04-20T21:18:55.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.444+0000 7f2377e00640 1 -- 172.21.15.86:0/162488774 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2364031830 con 0x7f237806d9c0 2024-04-20T21:18:55.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.444+0000 7f2377e00640 1 -- 172.21.15.86:0/162488774 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2364031c00 con 0x7f237806d9c0 2024-04-20T21:18:55.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.445+0000 7f237ea00640 1 -- 172.21.15.86:0/162488774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f237806d9c0 msgr2=0x7f237806ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:55.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.445+0000 7f237ea00640 1 --2- 172.21.15.86:0/162488774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f237806d9c0 0x7f237806ef20 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f23640097a0 tx=0x7f236402b8d0 comp rx=0 tx=0).stop 2024-04-20T21:18:55.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.445+0000 7f237ea00640 1 -- 172.21.15.86:0/162488774 shutdown_connections 2024-04-20T21:18:55.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.445+0000 7f237ea00640 1 --2- 172.21.15.86:0/162488774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f237806d9c0 0x7f237806ef20 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:55.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.445+0000 7f237ea00640 1 -- 172.21.15.86:0/162488774 >> 172.21.15.86:0/162488774 conn(0x7f23780b1f00 msgr2=0x7f23780b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:55.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.445+0000 7f237ea00640 1 -- 172.21.15.86:0/162488774 shutdown_connections 2024-04-20T21:18:55.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.445+0000 7f237ea00640 1 -- 172.21.15.86:0/162488774 wait complete. 2024-04-20T21:18:55.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.447+0000 7f237ea00640 1 Processor -- start 2024-04-20T21:18:55.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.448+0000 7f237ea00640 1 -- start start 2024-04-20T21:18:55.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.449+0000 7f237ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f237806d9c0 0x7f237815cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:55.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.449+0000 7f237ea00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23780af1f0 0x7f23781f46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:55.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.449+0000 7f237ea00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23780af5d0 0x7f237815f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:55.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.449+0000 7f237ea00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f23780afa40 con 0x7f237806d9c0 2024-04-20T21:18:55.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.449+0000 7f237ea00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f237806ea00 con 0x7f23780af1f0 2024-04-20T21:18:55.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.449+0000 7f237ea00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f237815a3c0 con 0x7f23780af5d0 2024-04-20T21:18:55.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.449+0000 7f237d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f237806d9c0 0x7f237815cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:55.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.449+0000 7f237d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f237806d9c0 0x7f237815cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37436/0 (socket says 172.21.15.86:37436) 2024-04-20T21:18:55.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.449+0000 7f237d600640 1 -- 172.21.15.86:0/4027160445 learned_addr learned my addr 172.21.15.86:0/4027160445 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:55.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.450+0000 7f237cc00640 1 --2- 172.21.15.86:0/4027160445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23780af1f0 0x7f23781f46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:55.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.450+0000 7f237e000640 1 --2- 172.21.15.86:0/4027160445 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23780af5d0 0x7f237815f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:55.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.450+0000 7f237d600640 1 -- 172.21.15.86:0/4027160445 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23780af5d0 msgr2=0x7f237815f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:55.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.450+0000 7f237d600640 1 --2- 172.21.15.86:0/4027160445 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23780af5d0 0x7f237815f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:55.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.450+0000 7f237d600640 1 -- 172.21.15.86:0/4027160445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23780af1f0 msgr2=0x7f23781f46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:55.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.450+0000 7f237d600640 1 --2- 172.21.15.86:0/4027160445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23780af1f0 0x7f23781f46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:55.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.450+0000 7f237d600640 1 -- 172.21.15.86:0/4027160445 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23781442f0 con 0x7f237806d9c0 2024-04-20T21:18:55.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.450+0000 7f237e000640 1 --2- 172.21.15.86:0/4027160445 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23780af5d0 0x7f237815f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:55.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.450+0000 7f237d600640 1 --2- 172.21.15.86:0/4027160445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f237806d9c0 0x7f237815cae0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f236c00c430 tx=0x7f236c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:55.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.450+0000 7f2375600640 1 -- 172.21.15.86:0/4027160445 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236c017070 con 0x7f237806d9c0 2024-04-20T21:18:55.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.450+0000 7f2375600640 1 -- 172.21.15.86:0/4027160445 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f236c005ce0 con 0x7f237806d9c0 2024-04-20T21:18:55.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.450+0000 7f2375600640 1 -- 172.21.15.86:0/4027160445 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236c0072f0 con 0x7f237806d9c0 2024-04-20T21:18:55.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.450+0000 7f237ea00640 1 -- 172.21.15.86:0/4027160445 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2378144590 con 0x7f237806d9c0 2024-04-20T21:18:55.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.451+0000 7f237ea00640 1 -- 172.21.15.86:0/4027160445 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2378144b50 con 0x7f237806d9c0 2024-04-20T21:18:55.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.452+0000 7f237ea00640 1 -- 172.21.15.86:0/4027160445 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f237804fd90 con 0x7f237806d9c0 2024-04-20T21:18:55.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.453+0000 7f2375600640 1 -- 172.21.15.86:0/4027160445 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f236c01a070 con 0x7f237806d9c0 2024-04-20T21:18:55.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.454+0000 7f2375600640 1 --2- 172.21.15.86:0/4027160445 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f235806c8f0 0x7f235806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:55.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.454+0000 7f2375600640 1 -- 172.21.15.86:0/4027160445 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f236c08b020 con 0x7f237806d9c0 2024-04-20T21:18:55.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.454+0000 7f237cc00640 1 --2- 172.21.15.86:0/4027160445 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f235806c8f0 0x7f235806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:55.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.455+0000 7f237cc00640 1 --2- 172.21.15.86:0/4027160445 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f235806c8f0 0x7f235806edb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2364036000 tx=0x7f236403c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:55.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.459+0000 7f2375600640 1 -- 172.21.15.86:0/4027160445 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f236c059460 con 0x7f237806d9c0 2024-04-20T21:18:55.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.658+0000 7f237ea00640 1 -- 172.21.15.86:0/4027160445 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f23780b3300 con 0x7f237806d9c0 2024-04-20T21:18:55.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.659+0000 7f2375600640 1 -- 172.21.15.86:0/4027160445 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f236c058d60 con 0x7f237806d9c0 2024-04-20T21:18:55.659 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-20T21:18:55.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.662+0000 7f237ea00640 1 -- 172.21.15.86:0/4027160445 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f235806c8f0 msgr2=0x7f235806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:55.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.662+0000 7f237ea00640 1 --2- 172.21.15.86:0/4027160445 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f235806c8f0 0x7f235806edb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2364036000 tx=0x7f236403c040 comp rx=0 tx=0).stop 2024-04-20T21:18:55.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.662+0000 7f237ea00640 1 -- 172.21.15.86:0/4027160445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f237806d9c0 msgr2=0x7f237815cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:55.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.662+0000 7f237ea00640 1 --2- 172.21.15.86:0/4027160445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f237806d9c0 0x7f237815cae0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f236c00c430 tx=0x7f236c00c8f0 comp rx=0 tx=0).stop 2024-04-20T21:18:55.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.662+0000 7f237ea00640 1 -- 172.21.15.86:0/4027160445 shutdown_connections 2024-04-20T21:18:55.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.662+0000 7f237ea00640 1 --2- 172.21.15.86:0/4027160445 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f235806c8f0 0x7f235806edb0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:55.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.662+0000 7f237ea00640 1 --2- 172.21.15.86:0/4027160445 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23780af5d0 0x7f237815f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:55.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.662+0000 7f237ea00640 1 --2- 172.21.15.86:0/4027160445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23780af1f0 0x7f23781f46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:55.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.662+0000 7f237ea00640 1 --2- 172.21.15.86:0/4027160445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f237806d9c0 0x7f237815cae0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:55.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.662+0000 7f237ea00640 1 -- 172.21.15.86:0/4027160445 >> 172.21.15.86:0/4027160445 conn(0x7f23780b1f00 msgr2=0x7f237815c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:55.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.662+0000 7f237ea00640 1 -- 172.21.15.86:0/4027160445 shutdown_connections 2024-04-20T21:18:55.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.662+0000 7f237ea00640 1 -- 172.21.15.86:0/4027160445 wait complete. 2024-04-20T21:18:55.679 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:18:55.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.880+0000 7f1174a00640 1 Processor -- start 2024-04-20T21:18:55.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.880+0000 7f1174a00640 1 -- start start 2024-04-20T21:18:55.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.882+0000 7f1174a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f117006fbd0 0x7f117015c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:55.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.882+0000 7f1174a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11700ae710 con 0x7f11700af7b0 2024-04-20T21:18:55.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.882+0000 7f1174a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11700ae9f0 con 0x7f117015c010 2024-04-20T21:18:55.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.882+0000 7f1174a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f11700af320 con 0x7f117006fbd0 2024-04-20T21:18:55.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.882+0000 7f116fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f117006fbd0 0x7f117015c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:55.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.882+0000 7f116fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f117006fbd0 0x7f117015c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55582/0 (socket says 172.21.15.86:55582) 2024-04-20T21:18:55.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.882+0000 7f1167e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f11700af7b0 0x7f117006f4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44358/0 (socket says 172.21.15.86:44358) 2024-04-20T21:18:55.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.882+0000 7f116fe00640 1 -- 172.21.15.86:0/1916494310 learned_addr learned my addr 172.21.15.86:0/1916494310 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:55.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.883+0000 7f116fe00640 1 -- 172.21.15.86:0/1916494310 >> v1:172.21.15.144:6789/0 conn(0x7f117015c010 legacy=0x7f11700adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:55.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.883+0000 7f116fe00640 1 -- 172.21.15.86:0/1916494310 >> v1:172.21.15.86:6789/0 conn(0x7f11700af7b0 legacy=0x7f117006f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:55.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.883+0000 7f116fe00640 1 -- 172.21.15.86:0/1916494310 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11700aecd0 con 0x7f117006fbd0 2024-04-20T21:18:55.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.884+0000 7f116fe00640 1 --2- 172.21.15.86:0/1916494310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f117006fbd0 0x7f117015c3f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1168009e50 tx=0x7f116802bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=884e056153508865 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:55.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.884+0000 7f116ea00640 1 -- 172.21.15.86:0/1916494310 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116802cb10 con 0x7f117006fbd0 2024-04-20T21:18:55.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.884+0000 7f116ea00640 1 -- 172.21.15.86:0/1916494310 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f116802ccb0 con 0x7f117006fbd0 2024-04-20T21:18:55.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.884+0000 7f116ea00640 1 -- 172.21.15.86:0/1916494310 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116802ce60 con 0x7f117006fbd0 2024-04-20T21:18:55.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.885+0000 7f1174a00640 1 -- 172.21.15.86:0/1916494310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f117006fbd0 msgr2=0x7f117015c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:55.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.885+0000 7f1174a00640 1 --2- 172.21.15.86:0/1916494310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f117006fbd0 0x7f117015c3f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1168009e50 tx=0x7f116802bf70 comp rx=0 tx=0).stop 2024-04-20T21:18:55.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.885+0000 7f1174a00640 1 -- 172.21.15.86:0/1916494310 shutdown_connections 2024-04-20T21:18:55.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.885+0000 7f1174a00640 1 --2- 172.21.15.86:0/1916494310 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f117006fbd0 0x7f117015c3f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:55.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.885+0000 7f1174a00640 1 -- 172.21.15.86:0/1916494310 >> 172.21.15.86:0/1916494310 conn(0x7f11700d0170 msgr2=0x7f11700d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:55.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.885+0000 7f1174a00640 1 -- 172.21.15.86:0/1916494310 shutdown_connections 2024-04-20T21:18:55.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.885+0000 7f1174a00640 1 -- 172.21.15.86:0/1916494310 wait complete. 2024-04-20T21:18:55.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.887+0000 7f1174a00640 1 Processor -- start 2024-04-20T21:18:55.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.887+0000 7f1174a00640 1 -- start start 2024-04-20T21:18:55.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.888+0000 7f1174a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f117006fbd0 0x7f1170071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:55.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.888+0000 7f1174a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11700af7b0 0x7f11701647b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:55.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.888+0000 7f1174a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f117015c010 0x7f11701486c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:55.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.888+0000 7f1174a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f117015c930 con 0x7f11700af7b0 2024-04-20T21:18:55.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.888+0000 7f1174a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f11700ae4b0 con 0x7f117015c010 2024-04-20T21:18:55.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.888+0000 7f1174a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f11701660d0 con 0x7f117006fbd0 2024-04-20T21:18:55.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.889+0000 7f1167e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11700af7b0 0x7f11701647b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:55.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.889+0000 7f1167e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11700af7b0 0x7f11701647b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55594/0 (socket says 172.21.15.86:55594) 2024-04-20T21:18:55.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.889+0000 7f1167e00640 1 -- 172.21.15.86:0/1749063754 learned_addr learned my addr 172.21.15.86:0/1749063754 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:55.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.889+0000 7f1167e00640 1 -- 172.21.15.86:0/1749063754 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f117006fbd0 msgr2=0x7f1170071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:55.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.889+0000 7f116f400640 1 --2- 172.21.15.86:0/1749063754 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f117006fbd0 0x7f1170071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:55.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.889+0000 7f116fe00640 1 --2- 172.21.15.86:0/1749063754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f117015c010 0x7f11701486c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:55.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.889+0000 7f1167e00640 1 --2- 172.21.15.86:0/1749063754 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f117006fbd0 0x7f1170071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:55.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.889+0000 7f1167e00640 1 -- 172.21.15.86:0/1749063754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f117015c010 msgr2=0x7f11701486c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:55.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.889+0000 7f1167e00640 1 --2- 172.21.15.86:0/1749063754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f117015c010 0x7f11701486c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:55.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.889+0000 7f1167e00640 1 -- 172.21.15.86:0/1749063754 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1170163710 con 0x7f11700af7b0 2024-04-20T21:18:55.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.890+0000 7f1167e00640 1 --2- 172.21.15.86:0/1749063754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11700af7b0 0x7f11701647b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f115800c430 tx=0x7f115800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:55.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.890+0000 7f1167400640 1 -- 172.21.15.86:0/1749063754 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11580074e0 con 0x7f11700af7b0 2024-04-20T21:18:55.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.890+0000 7f1167400640 1 -- 172.21.15.86:0/1749063754 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1158007680 con 0x7f11700af7b0 2024-04-20T21:18:55.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.890+0000 7f1174a00640 1 -- 172.21.15.86:0/1749063754 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11701639f0 con 0x7f11700af7b0 2024-04-20T21:18:55.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.890+0000 7f1167400640 1 -- 172.21.15.86:0/1749063754 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1158004a00 con 0x7f11700af7b0 2024-04-20T21:18:55.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.890+0000 7f1174a00640 1 -- 172.21.15.86:0/1749063754 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1170164290 con 0x7f11700af7b0 2024-04-20T21:18:55.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.891+0000 7f1174a00640 1 -- 172.21.15.86:0/1749063754 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1170163bc0 con 0x7f11700af7b0 2024-04-20T21:18:55.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.892+0000 7f1167400640 1 -- 172.21.15.86:0/1749063754 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f115801a020 con 0x7f11700af7b0 2024-04-20T21:18:55.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.893+0000 7f1167400640 1 --2- 172.21.15.86:0/1749063754 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f114c06c8f0 0x7f114c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:55.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.893+0000 7f1167400640 1 -- 172.21.15.86:0/1749063754 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f115808b4a0 con 0x7f11700af7b0 2024-04-20T21:18:55.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.893+0000 7f116f400640 1 --2- 172.21.15.86:0/1749063754 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f114c06c8f0 0x7f114c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:55.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.894+0000 7f116f400640 1 --2- 172.21.15.86:0/1749063754 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f114c06c8f0 0x7f114c06edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f116000a750 tx=0x7f1160008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:55.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:55.898+0000 7f1167400640 1 -- 172.21.15.86:0/1749063754 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f11580597b0 con 0x7f11700af7b0 2024-04-20T21:18:56.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.131+0000 7f1174a00640 1 -- 172.21.15.86:0/1749063754 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1170164490 con 0x7f11700af7b0 2024-04-20T21:18:56.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.133+0000 7f1167400640 1 -- 172.21.15.86:0/1749063754 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5196 (secure 0 0 0) 0x7f1170164490 con 0x7f11700af7b0 2024-04-20T21:18:56.132 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:18:56.132 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/179773217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":179773217},{"type":"v1","addr":"172.21.15.144:6834","nonce":179773217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3179785507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3179785507},{"type":"v1","addr":"172.21.15.86:6835","nonce":3179785507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4270,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2780853729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2780853729},{"type":"v1","addr":"172.21.15.86:6837","nonce":2780853729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:17:51.700362+0000","modified":"2024-04-20T21:17:54.973792+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6835/2436181584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2436181584},{"type":"v1","addr":"172.21.15.144:6835","nonce":2436181584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4272,"qdb_cluster":[4272]},"id":1}]} 2024-04-20T21:18:56.132 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 7 2024-04-20T21:18:56.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.136+0000 7f1174a00640 1 -- 172.21.15.86:0/1749063754 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f114c06c8f0 msgr2=0x7f114c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:56.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.136+0000 7f1174a00640 1 --2- 172.21.15.86:0/1749063754 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f114c06c8f0 0x7f114c06edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f116000a750 tx=0x7f1160008040 comp rx=0 tx=0).stop 2024-04-20T21:18:56.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.136+0000 7f1174a00640 1 -- 172.21.15.86:0/1749063754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11700af7b0 msgr2=0x7f11701647b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:56.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.136+0000 7f1174a00640 1 --2- 172.21.15.86:0/1749063754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11700af7b0 0x7f11701647b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f115800c430 tx=0x7f115800c8f0 comp rx=0 tx=0).stop 2024-04-20T21:18:56.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.136+0000 7f1174a00640 1 -- 172.21.15.86:0/1749063754 shutdown_connections 2024-04-20T21:18:56.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.136+0000 7f1174a00640 1 --2- 172.21.15.86:0/1749063754 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f114c06c8f0 0x7f114c06edb0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:56.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.136+0000 7f1174a00640 1 --2- 172.21.15.86:0/1749063754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f117015c010 0x7f11701486c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:56.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.136+0000 7f1174a00640 1 --2- 172.21.15.86:0/1749063754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11700af7b0 0x7f11701647b0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:56.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.136+0000 7f1174a00640 1 --2- 172.21.15.86:0/1749063754 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f117006fbd0 0x7f1170071de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:56.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.136+0000 7f1174a00640 1 -- 172.21.15.86:0/1749063754 >> 172.21.15.86:0/1749063754 conn(0x7f11700d0170 msgr2=0x7f117015de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:56.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.137+0000 7f1174a00640 1 -- 172.21.15.86:0/1749063754 shutdown_connections 2024-04-20T21:18:56.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.137+0000 7f1174a00640 1 -- 172.21.15.86:0/1749063754 wait complete. 2024-04-20T21:18:56.154 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:18:56.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.348+0000 7f6bd2800640 1 Processor -- start 2024-04-20T21:18:56.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.348+0000 7f6bd2800640 1 -- start start 2024-04-20T21:18:56.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.349+0000 7f6bd2800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6bcc144e90 0x7f6bcc0b4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:56.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.349+0000 7f6bd2800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6bcc0cd170 0x7f6bcc06ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:56.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.349+0000 7f6bd2800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6bcc0cd5d0 0x7f6bcc06f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:56.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.349+0000 7f6bd2800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f6bcc145270 con 0x7f6bcc0cd5d0 2024-04-20T21:18:56.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.349+0000 7f6bd2800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f6bcc0cdb50 con 0x7f6bcc144e90 2024-04-20T21:18:56.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.349+0000 7f6bd2800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6bcc0b3e50 con 0x7f6bcc0cd170 2024-04-20T21:18:56.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.350+0000 7f6bcbe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6bcc0cd5d0 0x7f6bcc06f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:56.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.350+0000 7f6bcbe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6bcc0cd5d0 0x7f6bcc06f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55598/0 (socket says 172.21.15.86:55598) 2024-04-20T21:18:56.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.350+0000 7f6bcbe00640 1 -- 172.21.15.86:0/2673710371 learned_addr learned my addr 172.21.15.86:0/2673710371 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:56.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.350+0000 7f6bcb400640 1 --2- 172.21.15.86:0/2673710371 >> v2:172.21.15.144:3300/0 conn(0x7f6bcc144e90 0x7f6bcc0b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:56.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.350+0000 7f6bcaa00640 1 --2- 172.21.15.86:0/2673710371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6bcc0cd170 0x7f6bcc06ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:56.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.350+0000 7f6bcbe00640 1 -- 172.21.15.86:0/2673710371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6bcc0cd170 msgr2=0x7f6bcc06ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:56.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.350+0000 7f6bcbe00640 1 --2- 172.21.15.86:0/2673710371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6bcc0cd170 0x7f6bcc06ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:56.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.350+0000 7f6bcbe00640 1 -- 172.21.15.86:0/2673710371 >> v2:172.21.15.144:3300/0 conn(0x7f6bcc144e90 msgr2=0x7f6bcc0b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:56.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.350+0000 7f6bcbe00640 1 --2- 172.21.15.86:0/2673710371 >> v2:172.21.15.144:3300/0 conn(0x7f6bcc144e90 0x7f6bcc0b4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:56.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.350+0000 7f6bcbe00640 1 -- 172.21.15.86:0/2673710371 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bcc0b4130 con 0x7f6bcc0cd5d0 2024-04-20T21:18:56.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.350+0000 7f6bcb400640 1 --2- 172.21.15.86:0/2673710371 >> v2:172.21.15.144:3300/0 conn(0x7f6bcc144e90 0x7f6bcc0b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:56.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.350+0000 7f6bcaa00640 1 --2- 172.21.15.86:0/2673710371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6bcc0cd170 0x7f6bcc06ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:56.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.351+0000 7f6bcbe00640 1 --2- 172.21.15.86:0/2673710371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bcc0cd5d0 0x7f6bcc06f280 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc06f920 tx=0x7f6bc402e060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ed67289e69a7da5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:56.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.352+0000 7f6bca000640 1 -- 172.21.15.86:0/2673710371 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc402ea00 con 0x7f6bcc0cd5d0 2024-04-20T21:18:56.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.352+0000 7f6bca000640 1 -- 172.21.15.86:0/2673710371 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bc402eba0 con 0x7f6bcc0cd5d0 2024-04-20T21:18:56.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.352+0000 7f6bca000640 1 -- 172.21.15.86:0/2673710371 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc4009600 con 0x7f6bcc0cd5d0 2024-04-20T21:18:56.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.352+0000 7f6bd2800640 1 -- 172.21.15.86:0/2673710371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bcc0cd5d0 msgr2=0x7f6bcc06f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:56.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.352+0000 7f6bd2800640 1 --2- 172.21.15.86:0/2673710371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bcc0cd5d0 0x7f6bcc06f280 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc06f920 tx=0x7f6bc402e060 comp rx=0 tx=0).stop 2024-04-20T21:18:56.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.352+0000 7f6bd2800640 1 -- 172.21.15.86:0/2673710371 shutdown_connections 2024-04-20T21:18:56.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.352+0000 7f6bd2800640 1 --2- 172.21.15.86:0/2673710371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bcc0cd5d0 0x7f6bcc06f280 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:56.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.352+0000 7f6bd2800640 1 --2- 172.21.15.86:0/2673710371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6bcc0cd170 0x7f6bcc06ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:56.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.352+0000 7f6bd2800640 1 --2- 172.21.15.86:0/2673710371 >> v2:172.21.15.144:3300/0 conn(0x7f6bcc144e90 0x7f6bcc0b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:56.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.352+0000 7f6bd2800640 1 -- 172.21.15.86:0/2673710371 >> 172.21.15.86:0/2673710371 conn(0x7f6bcc0b2360 msgr2=0x7f6bcc0b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:56.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.352+0000 7f6bd2800640 1 -- 172.21.15.86:0/2673710371 shutdown_connections 2024-04-20T21:18:56.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.352+0000 7f6bd2800640 1 -- 172.21.15.86:0/2673710371 wait complete. 2024-04-20T21:18:56.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.354+0000 7f6bd2800640 1 Processor -- start 2024-04-20T21:18:56.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.354+0000 7f6bd2800640 1 -- start start 2024-04-20T21:18:56.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.355+0000 7f6bd2800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bcc0cd170 0x7f6bcc0cc420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:56.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.355+0000 7f6bd2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bcc0cd5d0 0x7f6bcc15aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:56.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.355+0000 7f6bd2800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6bcc144e90 0x7f6bcc15afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:56.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.355+0000 7f6bd2800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6bcc0d0010 con 0x7f6bcc0cd5d0 2024-04-20T21:18:56.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.355+0000 7f6bd2800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6bcc0b4e70 con 0x7f6bcc0cd170 2024-04-20T21:18:56.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.355+0000 7f6bd2800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6bcc0d0190 con 0x7f6bcc144e90 2024-04-20T21:18:56.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.355+0000 7f6bcaa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bcc0cd5d0 0x7f6bcc15aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:56.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.355+0000 7f6bcaa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bcc0cd5d0 0x7f6bcc15aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55604/0 (socket says 172.21.15.86:55604) 2024-04-20T21:18:56.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.355+0000 7f6bcaa00640 1 -- 172.21.15.86:0/624518848 learned_addr learned my addr 172.21.15.86:0/624518848 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:56.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.356+0000 7f6bcaa00640 1 -- 172.21.15.86:0/624518848 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6bcc144e90 msgr2=0x7f6bcc15afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:56.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.356+0000 7f6bcaa00640 1 --2- 172.21.15.86:0/624518848 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6bcc144e90 0x7f6bcc15afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:56.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.356+0000 7f6bcb400640 1 --2- 172.21.15.86:0/624518848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bcc0cd170 0x7f6bcc0cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:56.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.356+0000 7f6bcaa00640 1 -- 172.21.15.86:0/624518848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bcc0cd170 msgr2=0x7f6bcc0cc420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:56.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.356+0000 7f6bcaa00640 1 --2- 172.21.15.86:0/624518848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bcc0cd170 0x7f6bcc0cc420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:56.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.356+0000 7f6bcaa00640 1 -- 172.21.15.86:0/624518848 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bcc0c99f0 con 0x7f6bcc0cd5d0 2024-04-20T21:18:56.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.356+0000 7f6bcaa00640 1 --2- 172.21.15.86:0/624518848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bcc0cd5d0 0x7f6bcc15aa70 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6bb400c7f0 tx=0x7f6bb400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:56.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.356+0000 7f6bc3400640 1 -- 172.21.15.86:0/624518848 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb4015070 con 0x7f6bcc0cd5d0 2024-04-20T21:18:56.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.356+0000 7f6bc3400640 1 -- 172.21.15.86:0/624518848 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bb4007820 con 0x7f6bcc0cd5d0 2024-04-20T21:18:56.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.356+0000 7f6bc3400640 1 -- 172.21.15.86:0/624518848 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb4007c10 con 0x7f6bcc0cd5d0 2024-04-20T21:18:56.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.356+0000 7f6bd2800640 1 -- 172.21.15.86:0/624518848 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bcc0c9cd0 con 0x7f6bcc0cd5d0 2024-04-20T21:18:56.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.356+0000 7f6bd2800640 1 -- 172.21.15.86:0/624518848 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bcc0ca290 con 0x7f6bcc0cd5d0 2024-04-20T21:18:56.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.357+0000 7f6bd2800640 1 -- 172.21.15.86:0/624518848 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bcc0689e0 con 0x7f6bcc0cd5d0 2024-04-20T21:18:56.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.358+0000 7f6bc3400640 1 -- 172.21.15.86:0/624518848 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6bb4018070 con 0x7f6bcc0cd5d0 2024-04-20T21:18:56.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.359+0000 7f6bc3400640 1 --2- 172.21.15.86:0/624518848 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6ba806c8f0 0x7f6ba806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:56.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.359+0000 7f6bc3400640 1 -- 172.21.15.86:0/624518848 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f6bb408bd80 con 0x7f6bcc0cd5d0 2024-04-20T21:18:56.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.359+0000 7f6bcb400640 1 --2- 172.21.15.86:0/624518848 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6ba806c8f0 0x7f6ba806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:56.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.360+0000 7f6bcb400640 1 --2- 172.21.15.86:0/624518848 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6ba806c8f0 0x7f6ba806edb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc008850 tx=0x7f6bbc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:56.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.363+0000 7f6bc3400640 1 -- 172.21.15.86:0/624518848 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6bb405a0a0 con 0x7f6bcc0cd5d0 2024-04-20T21:18:56.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.547+0000 7f6bd2800640 1 -- 172.21.15.86:0/624518848 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6bcc06e7c0 con 0x7f6bcc0cd5d0 2024-04-20T21:18:56.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.550+0000 7f6bc3400640 1 -- 172.21.15.86:0/624518848 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16854 (secure 0 0 0) 0x7f6bb40599a0 con 0x7f6bcc0cd5d0 2024-04-20T21:18:56.550 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:18:56.550 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":22,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:18:33.788155+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:17:49.718799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:17:50.683477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:18:56.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.553+0000 7f6bd2800640 1 -- 172.21.15.86:0/624518848 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6ba806c8f0 msgr2=0x7f6ba806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:56.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.553+0000 7f6bd2800640 1 --2- 172.21.15.86:0/624518848 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6ba806c8f0 0x7f6ba806edb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc008850 tx=0x7f6bbc008040 comp rx=0 tx=0).stop 2024-04-20T21:18:56.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.553+0000 7f6bd2800640 1 -- 172.21.15.86:0/624518848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bcc0cd5d0 msgr2=0x7f6bcc15aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:56.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.553+0000 7f6bd2800640 1 --2- 172.21.15.86:0/624518848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bcc0cd5d0 0x7f6bcc15aa70 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6bb400c7f0 tx=0x7f6bb400ccb0 comp rx=0 tx=0).stop 2024-04-20T21:18:56.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.553+0000 7f6bd2800640 1 -- 172.21.15.86:0/624518848 shutdown_connections 2024-04-20T21:18:56.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.553+0000 7f6bd2800640 1 --2- 172.21.15.86:0/624518848 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6ba806c8f0 0x7f6ba806edb0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:56.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.553+0000 7f6bd2800640 1 --2- 172.21.15.86:0/624518848 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6bcc144e90 0x7f6bcc15afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:56.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.553+0000 7f6bd2800640 1 --2- 172.21.15.86:0/624518848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bcc0cd5d0 0x7f6bcc15aa70 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:56.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.553+0000 7f6bd2800640 1 --2- 172.21.15.86:0/624518848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bcc0cd170 0x7f6bcc0cc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:56.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.553+0000 7f6bd2800640 1 -- 172.21.15.86:0/624518848 >> 172.21.15.86:0/624518848 conn(0x7f6bcc0b2360 msgr2=0x7f6bcc06efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:56.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.553+0000 7f6bd2800640 1 -- 172.21.15.86:0/624518848 shutdown_connections 2024-04-20T21:18:56.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.553+0000 7f6bd2800640 1 -- 172.21.15.86:0/624518848 wait complete. 2024-04-20T21:18:56.572 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-20T21:18:56.572 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:18:56.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.769+0000 7f1c78800640 1 Processor -- start 2024-04-20T21:18:56.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.770+0000 7f1c78800640 1 -- start start 2024-04-20T21:18:56.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.771+0000 7f1c78800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1c70146130 0x7f1c7006e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:56.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.771+0000 7f1c78800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c700ccdd0 0x7f1c700cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:56.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.771+0000 7f1c78800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c7006dc30 con 0x7f1c70071d00 2024-04-20T21:18:56.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.771+0000 7f1c78800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f1c700cd8e0 con 0x7f1c70146130 2024-04-20T21:18:56.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.771+0000 7f1c78800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1c700cdbc0 con 0x7f1c700ccdd0 2024-04-20T21:18:56.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.772+0000 7f1c75e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1c70146130 0x7f1c7006e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:56.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.772+0000 7f1c75e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1c70146130 0x7f1c7006e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55618/0 (socket says 172.21.15.86:55618) 2024-04-20T21:18:56.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.772+0000 7f1c75e00640 1 -- 172.21.15.86:0/2858221535 learned_addr learned my addr 172.21.15.86:0/2858221535 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:56.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.772+0000 7f1c75400640 1 --2- 172.21.15.86:0/2858221535 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c700ccdd0 0x7f1c700cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:56.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.772+0000 7f1c75e00640 1 -- 172.21.15.86:0/2858221535 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c700ccdd0 msgr2=0x7f1c700cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:56.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.772+0000 7f1c75e00640 1 --2- 172.21.15.86:0/2858221535 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c700ccdd0 0x7f1c700cd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:56.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.772+0000 7f1c75e00640 1 -- 172.21.15.86:0/2858221535 >> v1:172.21.15.144:6789/0 conn(0x7f1c70071d00 legacy=0x7f1c7015cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:56.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.772+0000 7f1c75e00640 1 -- 172.21.15.86:0/2858221535 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c7006ded0 con 0x7f1c70146130 2024-04-20T21:18:56.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.772+0000 7f1c75400640 1 --2- 172.21.15.86:0/2858221535 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c700ccdd0 0x7f1c700cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:56.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.774+0000 7f1c75e00640 1 --2- 172.21.15.86:0/2858221535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c70146130 0x7f1c7006e6d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1c64009870 tx=0x7f1c6402ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1c6c4da5a341e54 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:56.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.774+0000 7f1c74a00640 1 -- 172.21.15.86:0/2858221535 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6402f840 con 0x7f1c70146130 2024-04-20T21:18:56.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.774+0000 7f1c74a00640 1 -- 172.21.15.86:0/2858221535 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c6402f9e0 con 0x7f1c70146130 2024-04-20T21:18:56.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.774+0000 7f1c74a00640 1 -- 172.21.15.86:0/2858221535 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6402fdb0 con 0x7f1c70146130 2024-04-20T21:18:56.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.775+0000 7f1c78800640 1 -- 172.21.15.86:0/2858221535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c70146130 msgr2=0x7f1c7006e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:56.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.775+0000 7f1c78800640 1 --2- 172.21.15.86:0/2858221535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c70146130 0x7f1c7006e6d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1c64009870 tx=0x7f1c6402ba20 comp rx=0 tx=0).stop 2024-04-20T21:18:56.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.775+0000 7f1c78800640 1 -- 172.21.15.86:0/2858221535 shutdown_connections 2024-04-20T21:18:56.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.775+0000 7f1c78800640 1 --2- 172.21.15.86:0/2858221535 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c700ccdd0 0x7f1c700cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:56.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.775+0000 7f1c78800640 1 --2- 172.21.15.86:0/2858221535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c70146130 0x7f1c7006e6d0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:56.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.775+0000 7f1c78800640 1 -- 172.21.15.86:0/2858221535 >> 172.21.15.86:0/2858221535 conn(0x7f1c700b2360 msgr2=0x7f1c700b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:56.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.775+0000 7f1c78800640 1 -- 172.21.15.86:0/2858221535 shutdown_connections 2024-04-20T21:18:56.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.775+0000 7f1c78800640 1 -- 172.21.15.86:0/2858221535 wait complete. 2024-04-20T21:18:56.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.777+0000 7f1c78800640 1 Processor -- start 2024-04-20T21:18:56.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.777+0000 7f1c78800640 1 -- start start 2024-04-20T21:18:56.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.778+0000 7f1c78800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c70071d00 0x7f1c7015f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:56.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.778+0000 7f1c78800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c700ccdd0 0x7f1c701572c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:56.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.778+0000 7f1c78800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c70146130 0x7f1c70157800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:56.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.778+0000 7f1c78800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1c70072110 con 0x7f1c70071d00 2024-04-20T21:18:56.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.778+0000 7f1c78800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7006f150 con 0x7f1c70146130 2024-04-20T21:18:56.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.778+0000 7f1c78800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1c70072290 con 0x7f1c700ccdd0 2024-04-20T21:18:56.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.778+0000 7f1c75e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c70071d00 0x7f1c7015f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:56.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.778+0000 7f1c75e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c70071d00 0x7f1c7015f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55632/0 (socket says 172.21.15.86:55632) 2024-04-20T21:18:56.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.778+0000 7f1c75e00640 1 -- 172.21.15.86:0/1126092347 learned_addr learned my addr 172.21.15.86:0/1126092347 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:56.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.778+0000 7f1c75e00640 1 -- 172.21.15.86:0/1126092347 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c700ccdd0 msgr2=0x7f1c701572c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:56.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.778+0000 7f1c75e00640 1 --2- 172.21.15.86:0/1126092347 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c700ccdd0 0x7f1c701572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:56.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.778+0000 7f1c75e00640 1 -- 172.21.15.86:0/1126092347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c70146130 msgr2=0x7f1c70157800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:56.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.778+0000 7f1c75e00640 1 --2- 172.21.15.86:0/1126092347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c70146130 0x7f1c70157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:56.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.778+0000 7f1c75e00640 1 -- 172.21.15.86:0/1126092347 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c7013c440 con 0x7f1c70071d00 2024-04-20T21:18:56.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.779+0000 7f1c75e00640 1 --2- 172.21.15.86:0/1126092347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c70071d00 0x7f1c7015f160 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1c64009870 tx=0x7f1c64004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:56.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.779+0000 7f1c6a000640 1 -- 172.21.15.86:0/1126092347 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6402f800 con 0x7f1c70071d00 2024-04-20T21:18:56.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.779+0000 7f1c6a000640 1 -- 172.21.15.86:0/1126092347 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c64046070 con 0x7f1c70071d00 2024-04-20T21:18:56.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.779+0000 7f1c6a000640 1 -- 172.21.15.86:0/1126092347 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c64041640 con 0x7f1c70071d00 2024-04-20T21:18:56.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.779+0000 7f1c78800640 1 -- 172.21.15.86:0/1126092347 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c7015bf70 con 0x7f1c70071d00 2024-04-20T21:18:56.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.779+0000 7f1c78800640 1 -- 172.21.15.86:0/1126092347 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c7015c530 con 0x7f1c70071d00 2024-04-20T21:18:56.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.780+0000 7f1c78800640 1 -- 172.21.15.86:0/1126092347 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c700689e0 con 0x7f1c70071d00 2024-04-20T21:18:56.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.781+0000 7f1c6a000640 1 -- 172.21.15.86:0/1126092347 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1c64040070 con 0x7f1c70071d00 2024-04-20T21:18:56.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.782+0000 7f1c6a000640 1 --2- 172.21.15.86:0/1126092347 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1c5406c8f0 0x7f1c5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:56.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.782+0000 7f1c6a000640 1 -- 172.21.15.86:0/1126092347 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f1c640ae330 con 0x7f1c70071d00 2024-04-20T21:18:56.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.782+0000 7f1c75400640 1 --2- 172.21.15.86:0/1126092347 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1c5406c8f0 0x7f1c5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:56.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.783+0000 7f1c75400640 1 --2- 172.21.15.86:0/1126092347 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1c5406c8f0 0x7f1c5406edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1c60006fd0 tx=0x7f1c60008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:56.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:56.786+0000 7f1c6a000640 1 -- 172.21.15.86:0/1126092347 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1c7013c440 con 0x7f1c70071d00 2024-04-20T21:18:57.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.023+0000 7f1c78800640 1 -- 172.21.15.86:0/1126092347 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c7015ff50 con 0x7f1c70071d00 2024-04-20T21:18:57.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.024+0000 7f1c6a000640 1 -- 172.21.15.86:0/1126092347 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5196 (secure 0 0 0) 0x7f1c64033030 con 0x7f1c70071d00 2024-04-20T21:18:57.024 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:18:57.024 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/179773217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":179773217},{"type":"v1","addr":"172.21.15.144:6834","nonce":179773217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3179785507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3179785507},{"type":"v1","addr":"172.21.15.86:6835","nonce":3179785507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4270,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2780853729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2780853729},{"type":"v1","addr":"172.21.15.86:6837","nonce":2780853729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:17:51.700362+0000","modified":"2024-04-20T21:17:54.973792+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6835/2436181584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2436181584},{"type":"v1","addr":"172.21.15.144:6835","nonce":2436181584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4272,"qdb_cluster":[4272]},"id":1}]} 2024-04-20T21:18:57.024 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 7 2024-04-20T21:18:57.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.027+0000 7f1c78800640 1 -- 172.21.15.86:0/1126092347 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1c5406c8f0 msgr2=0x7f1c5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:57.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.027+0000 7f1c78800640 1 --2- 172.21.15.86:0/1126092347 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1c5406c8f0 0x7f1c5406edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1c60006fd0 tx=0x7f1c60008040 comp rx=0 tx=0).stop 2024-04-20T21:18:57.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.027+0000 7f1c78800640 1 -- 172.21.15.86:0/1126092347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c70071d00 msgr2=0x7f1c7015f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:57.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.027+0000 7f1c78800640 1 --2- 172.21.15.86:0/1126092347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c70071d00 0x7f1c7015f160 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1c64009870 tx=0x7f1c64004730 comp rx=0 tx=0).stop 2024-04-20T21:18:57.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.027+0000 7f1c78800640 1 -- 172.21.15.86:0/1126092347 shutdown_connections 2024-04-20T21:18:57.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.027+0000 7f1c78800640 1 --2- 172.21.15.86:0/1126092347 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1c5406c8f0 0x7f1c5406edb0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:57.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.027+0000 7f1c78800640 1 --2- 172.21.15.86:0/1126092347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c70146130 0x7f1c70157800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:57.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.027+0000 7f1c78800640 1 --2- 172.21.15.86:0/1126092347 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c700ccdd0 0x7f1c701572c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:57.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.027+0000 7f1c78800640 1 --2- 172.21.15.86:0/1126092347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c70071d00 0x7f1c7015f160 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:57.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.027+0000 7f1c78800640 1 -- 172.21.15.86:0/1126092347 >> 172.21.15.86:0/1126092347 conn(0x7f1c700b2360 msgr2=0x7f1c7015d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:57.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.027+0000 7f1c78800640 1 -- 172.21.15.86:0/1126092347 shutdown_connections 2024-04-20T21:18:57.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.027+0000 7f1c78800640 1 -- 172.21.15.86:0/1126092347 wait complete. 2024-04-20T21:18:57.045 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:18:57.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.244+0000 7fe736e00640 1 Processor -- start 2024-04-20T21:18:57.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.244+0000 7fe736e00640 1 -- start start 2024-04-20T21:18:57.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.245+0000 7fe736e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe73015c010 0x7fe73015c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:57.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.246+0000 7fe736e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe7300af7b0 0x7fe7300adda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:57.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.246+0000 7fe736e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe73006f4c0 0x7fe73006f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:57.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.246+0000 7fe736e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe7300af320 con 0x7fe73006f4c0 2024-04-20T21:18:57.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.246+0000 7fe736e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fe73015c930 con 0x7fe73015c010 2024-04-20T21:18:57.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.246+0000 7fe736e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe7300ae710 con 0x7fe7300af7b0 2024-04-20T21:18:57.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.246+0000 7fe736400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe73006f4c0 0x7fe73006f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:57.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.246+0000 7fe736400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe73006f4c0 0x7fe73006f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55642/0 (socket says 172.21.15.86:55642) 2024-04-20T21:18:57.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.246+0000 7fe736400640 1 -- 172.21.15.86:0/676481515 learned_addr learned my addr 172.21.15.86:0/676481515 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:57.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.246+0000 7fe735000640 1 --2- 172.21.15.86:0/676481515 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe7300af7b0 0x7fe7300adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:57.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.246+0000 7fe735a00640 1 --2- 172.21.15.86:0/676481515 >> v2:172.21.15.144:3300/0 conn(0x7fe73015c010 0x7fe73015c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:57.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.246+0000 7fe736400640 1 -- 172.21.15.86:0/676481515 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe7300af7b0 msgr2=0x7fe7300adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:57.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.247+0000 7fe736400640 1 --2- 172.21.15.86:0/676481515 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe7300af7b0 0x7fe7300adda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:57.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.247+0000 7fe736400640 1 -- 172.21.15.86:0/676481515 >> v2:172.21.15.144:3300/0 conn(0x7fe73015c010 msgr2=0x7fe73015c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:57.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.247+0000 7fe736400640 1 --2- 172.21.15.86:0/676481515 >> v2:172.21.15.144:3300/0 conn(0x7fe73015c010 0x7fe73015c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:57.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.247+0000 7fe736400640 1 -- 172.21.15.86:0/676481515 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7300ae9f0 con 0x7fe73006f4c0 2024-04-20T21:18:57.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.247+0000 7fe735a00640 1 --2- 172.21.15.86:0/676481515 >> v2:172.21.15.144:3300/0 conn(0x7fe73015c010 0x7fe73015c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:57.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.248+0000 7fe736400640 1 --2- 172.21.15.86:0/676481515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe73006f4c0 0x7fe73006f970 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe7300d0df0 tx=0x7fe72802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20cfed39627f626e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:57.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.248+0000 7fe72fe00640 1 -- 172.21.15.86:0/676481515 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72802ca70 con 0x7fe73006f4c0 2024-04-20T21:18:57.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.248+0000 7fe72fe00640 1 -- 172.21.15.86:0/676481515 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe72802cc10 con 0x7fe73006f4c0 2024-04-20T21:18:57.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.248+0000 7fe72fe00640 1 -- 172.21.15.86:0/676481515 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7280343f0 con 0x7fe73006f4c0 2024-04-20T21:18:57.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.249+0000 7fe736e00640 1 -- 172.21.15.86:0/676481515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe73006f4c0 msgr2=0x7fe73006f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:57.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.249+0000 7fe736e00640 1 --2- 172.21.15.86:0/676481515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe73006f4c0 0x7fe73006f970 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe7300d0df0 tx=0x7fe72802bed0 comp rx=0 tx=0).stop 2024-04-20T21:18:57.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.249+0000 7fe736e00640 1 -- 172.21.15.86:0/676481515 shutdown_connections 2024-04-20T21:18:57.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.249+0000 7fe736e00640 1 --2- 172.21.15.86:0/676481515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe73006f4c0 0x7fe73006f970 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:57.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.249+0000 7fe736e00640 1 --2- 172.21.15.86:0/676481515 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe7300af7b0 0x7fe7300adda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:57.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.249+0000 7fe736e00640 1 --2- 172.21.15.86:0/676481515 >> v2:172.21.15.144:3300/0 conn(0x7fe73015c010 0x7fe73015c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:57.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.249+0000 7fe736e00640 1 -- 172.21.15.86:0/676481515 >> 172.21.15.86:0/676481515 conn(0x7fe7300d0170 msgr2=0x7fe7300d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:57.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.249+0000 7fe736e00640 1 -- 172.21.15.86:0/676481515 shutdown_connections 2024-04-20T21:18:57.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.249+0000 7fe736e00640 1 -- 172.21.15.86:0/676481515 wait complete. 2024-04-20T21:18:57.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.251+0000 7fe736e00640 1 Processor -- start 2024-04-20T21:18:57.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.251+0000 7fe736e00640 1 -- start start 2024-04-20T21:18:57.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.252+0000 7fe736e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe73006f4c0 0x7fe730099880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:57.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.252+0000 7fe736e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe7300af7b0 0x7fe7300b0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:57.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.252+0000 7fe736e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe73015c010 0x7fe730154da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:57.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.252+0000 7fe736e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe7300ae2e0 con 0x7fe73006f4c0 2024-04-20T21:18:57.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.252+0000 7fe736e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe7300aefb0 con 0x7fe73015c010 2024-04-20T21:18:57.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.252+0000 7fe736e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe7300ae460 con 0x7fe7300af7b0 2024-04-20T21:18:57.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.252+0000 7fe735a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe73006f4c0 0x7fe730099880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:57.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.252+0000 7fe735a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe73006f4c0 0x7fe730099880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55656/0 (socket says 172.21.15.86:55656) 2024-04-20T21:18:57.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.252+0000 7fe735a00640 1 -- 172.21.15.86:0/1265616406 learned_addr learned my addr 172.21.15.86:0/1265616406 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:57.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.252+0000 7fe735000640 1 --2- 172.21.15.86:0/1265616406 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe7300af7b0 0x7fe7300b0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:57.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.252+0000 7fe736400640 1 --2- 172.21.15.86:0/1265616406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe73015c010 0x7fe730154da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:57.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.252+0000 7fe735a00640 1 -- 172.21.15.86:0/1265616406 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe7300af7b0 msgr2=0x7fe7300b0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:57.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.252+0000 7fe735a00640 1 --2- 172.21.15.86:0/1265616406 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe7300af7b0 0x7fe7300b0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:57.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.252+0000 7fe735a00640 1 -- 172.21.15.86:0/1265616406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe73015c010 msgr2=0x7fe730154da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:57.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.252+0000 7fe735a00640 1 --2- 172.21.15.86:0/1265616406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe73015c010 0x7fe730154da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:57.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.252+0000 7fe735a00640 1 -- 172.21.15.86:0/1265616406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe730150800 con 0x7fe73006f4c0 2024-04-20T21:18:57.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.252+0000 7fe735000640 1 --2- 172.21.15.86:0/1265616406 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe7300af7b0 0x7fe7300b0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:57.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.253+0000 7fe735a00640 1 --2- 172.21.15.86:0/1265616406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe73006f4c0 0x7fe730099880 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe72400c7f0 tx=0x7fe72400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:57.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.253+0000 7fe72d600640 1 -- 172.21.15.86:0/1265616406 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe724015070 con 0x7fe73006f4c0 2024-04-20T21:18:57.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.253+0000 7fe72d600640 1 -- 172.21.15.86:0/1265616406 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe724007820 con 0x7fe73006f4c0 2024-04-20T21:18:57.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.253+0000 7fe72d600640 1 -- 172.21.15.86:0/1265616406 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe724007bd0 con 0x7fe73006f4c0 2024-04-20T21:18:57.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.253+0000 7fe736e00640 1 -- 172.21.15.86:0/1265616406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe730150ae0 con 0x7fe73006f4c0 2024-04-20T21:18:57.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.253+0000 7fe736e00640 1 -- 172.21.15.86:0/1265616406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7301510a0 con 0x7fe73006f4c0 2024-04-20T21:18:57.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.254+0000 7fe736e00640 1 -- 172.21.15.86:0/1265616406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe73004fcf0 con 0x7fe73006f4c0 2024-04-20T21:18:57.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.255+0000 7fe72d600640 1 -- 172.21.15.86:0/1265616406 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe724018070 con 0x7fe73006f4c0 2024-04-20T21:18:57.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.256+0000 7fe72d600640 1 --2- 172.21.15.86:0/1265616406 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe71006c8f0 0x7fe71006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:57.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.256+0000 7fe72d600640 1 -- 172.21.15.86:0/1265616406 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fe72408bda0 con 0x7fe73006f4c0 2024-04-20T21:18:57.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.257+0000 7fe735000640 1 --2- 172.21.15.86:0/1265616406 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe71006c8f0 0x7fe71006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:57.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.257+0000 7fe735000640 1 --2- 172.21.15.86:0/1265616406 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe71006c8f0 0x7fe71006edb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe7300b18e0 tx=0x7fe71c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:57.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.261+0000 7fe72d600640 1 -- 172.21.15.86:0/1265616406 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe72405a0f0 con 0x7fe73006f4c0 2024-04-20T21:18:57.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.499+0000 7fe736e00640 1 -- 172.21.15.86:0/1265616406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7300b1630 con 0x7fe73006f4c0 2024-04-20T21:18:57.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.500+0000 7fe72d600640 1 -- 172.21.15.86:0/1265616406 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5196 (secure 0 0 0) 0x7fe7240599f0 con 0x7fe73006f4c0 2024-04-20T21:18:57.500 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:18:57.500 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/179773217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":179773217},{"type":"v1","addr":"172.21.15.144:6834","nonce":179773217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/3179785507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":3179785507},{"type":"v1","addr":"172.21.15.86:6835","nonce":3179785507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4270,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2780853729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":2780853729},{"type":"v1","addr":"172.21.15.86:6837","nonce":2780853729}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:17:51.700362+0000","modified":"2024-04-20T21:17:54.973792+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6835/2436181584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2436181584},{"type":"v1","addr":"172.21.15.144:6835","nonce":2436181584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4272,"qdb_cluster":[4272]},"id":1}]} 2024-04-20T21:18:57.500 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 7 2024-04-20T21:18:57.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.503+0000 7fe736e00640 1 -- 172.21.15.86:0/1265616406 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe71006c8f0 msgr2=0x7fe71006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:57.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.503+0000 7fe736e00640 1 --2- 172.21.15.86:0/1265616406 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe71006c8f0 0x7fe71006edb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe7300b18e0 tx=0x7fe71c008040 comp rx=0 tx=0).stop 2024-04-20T21:18:57.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.503+0000 7fe736e00640 1 -- 172.21.15.86:0/1265616406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe73006f4c0 msgr2=0x7fe730099880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:57.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.503+0000 7fe736e00640 1 --2- 172.21.15.86:0/1265616406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe73006f4c0 0x7fe730099880 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe72400c7f0 tx=0x7fe72400ccb0 comp rx=0 tx=0).stop 2024-04-20T21:18:57.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.503+0000 7fe736e00640 1 -- 172.21.15.86:0/1265616406 shutdown_connections 2024-04-20T21:18:57.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.503+0000 7fe736e00640 1 --2- 172.21.15.86:0/1265616406 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe71006c8f0 0x7fe71006edb0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:57.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.503+0000 7fe736e00640 1 --2- 172.21.15.86:0/1265616406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe73015c010 0x7fe730154da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:57.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.503+0000 7fe736e00640 1 --2- 172.21.15.86:0/1265616406 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe7300af7b0 0x7fe7300b0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:57.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.503+0000 7fe736e00640 1 --2- 172.21.15.86:0/1265616406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe73006f4c0 0x7fe730099880 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:57.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.503+0000 7fe736e00640 1 -- 172.21.15.86:0/1265616406 >> 172.21.15.86:0/1265616406 conn(0x7fe7300d0170 msgr2=0x7fe73006e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:57.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.503+0000 7fe736e00640 1 -- 172.21.15.86:0/1265616406 shutdown_connections 2024-04-20T21:18:57.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.503+0000 7fe736e00640 1 -- 172.21.15.86:0/1265616406 wait complete. 2024-04-20T21:18:57.520 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:18:57.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.721+0000 7fd4c5200640 1 Processor -- start 2024-04-20T21:18:57.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.722+0000 7fd4c5200640 1 -- start start 2024-04-20T21:18:57.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.723+0000 7fd4c5200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd4c0162180 0x7fd4c00cfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:57.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.723+0000 7fd4c5200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd4c00d52c0 0x7fd4c00d2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:57.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.723+0000 7fd4c5200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4c006ea70 con 0x7fd4c00b34b0 2024-04-20T21:18:57.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.723+0000 7fd4c5200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd4c006dc50 con 0x7fd4c00d52c0 2024-04-20T21:18:57.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.723+0000 7fd4c5200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fd4c00d1c80 con 0x7fd4c0162180 2024-04-20T21:18:57.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.723+0000 7fd4b7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd4c00d52c0 0x7fd4c00d2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:57.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.723+0000 7fd4bfe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd4c00b34b0 0x7fd4c00d0c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44372/0 (socket says 172.21.15.86:44372) 2024-04-20T21:18:57.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.723+0000 7fd4b7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd4c00d52c0 0x7fd4c00d2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55668/0 (socket says 172.21.15.86:55668) 2024-04-20T21:18:57.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.724+0000 7fd4b7e00640 1 -- 172.21.15.86:0/2270691015 learned_addr learned my addr 172.21.15.86:0/2270691015 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:57.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.724+0000 7fd4bf400640 1 --2- 172.21.15.86:0/2270691015 >> v2:172.21.15.144:3300/0 conn(0x7fd4c0162180 0x7fd4c00cfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:57.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.724+0000 7fd4b7e00640 1 -- 172.21.15.86:0/2270691015 >> v2:172.21.15.144:3300/0 conn(0x7fd4c0162180 msgr2=0x7fd4c00cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:57.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.724+0000 7fd4b7e00640 1 --2- 172.21.15.86:0/2270691015 >> v2:172.21.15.144:3300/0 conn(0x7fd4c0162180 0x7fd4c00cfa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:57.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.724+0000 7fd4b7e00640 1 -- 172.21.15.86:0/2270691015 >> v1:172.21.15.86:6789/0 conn(0x7fd4c00b34b0 legacy=0x7fd4c00d0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:57.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.724+0000 7fd4b7e00640 1 -- 172.21.15.86:0/2270691015 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4c006e4b0 con 0x7fd4c00d52c0 2024-04-20T21:18:57.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.724+0000 7fd4bea00640 1 -- 172.21.15.86:0/2270691015 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 967350445 0 0) 0x7fd4c006ea70 con 0x7fd4c00b34b0 2024-04-20T21:18:57.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.725+0000 7fd4b7e00640 1 --2- 172.21.15.86:0/2270691015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4c00d52c0 0x7fd4c00d2bc0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0073d80 tx=0x7fd4a802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32d9f5ba3bc02edf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:57.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.725+0000 7fd4bea00640 1 -- 172.21.15.86:0/2270691015 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a8035020 con 0x7fd4c00d52c0 2024-04-20T21:18:57.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.725+0000 7fd4bea00640 1 -- 172.21.15.86:0/2270691015 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4a8031870 con 0x7fd4c00d52c0 2024-04-20T21:18:57.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.725+0000 7fd4bea00640 1 -- 172.21.15.86:0/2270691015 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a8031c40 con 0x7fd4c00d52c0 2024-04-20T21:18:57.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.726+0000 7fd4c5200640 1 -- 172.21.15.86:0/2270691015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4c00d52c0 msgr2=0x7fd4c00d2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:57.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.726+0000 7fd4c5200640 1 --2- 172.21.15.86:0/2270691015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4c00d52c0 0x7fd4c00d2bc0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0073d80 tx=0x7fd4a802b910 comp rx=0 tx=0).stop 2024-04-20T21:18:57.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.726+0000 7fd4c5200640 1 -- 172.21.15.86:0/2270691015 shutdown_connections 2024-04-20T21:18:57.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.726+0000 7fd4c5200640 1 --2- 172.21.15.86:0/2270691015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4c00d52c0 0x7fd4c00d2bc0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:57.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.726+0000 7fd4c5200640 1 --2- 172.21.15.86:0/2270691015 >> v2:172.21.15.144:3300/0 conn(0x7fd4c0162180 0x7fd4c00cfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:57.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.726+0000 7fd4c5200640 1 -- 172.21.15.86:0/2270691015 >> 172.21.15.86:0/2270691015 conn(0x7fd4c008bf70 msgr2=0x7fd4c00d45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:57.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.726+0000 7fd4c5200640 1 -- 172.21.15.86:0/2270691015 shutdown_connections 2024-04-20T21:18:57.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.726+0000 7fd4c5200640 1 -- 172.21.15.86:0/2270691015 wait complete. 2024-04-20T21:18:57.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.728+0000 7fd4c5200640 1 Processor -- start 2024-04-20T21:18:57.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.728+0000 7fd4c5200640 1 -- start start 2024-04-20T21:18:57.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.729+0000 7fd4c5200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4c00b34b0 0x7fd4c00b7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:57.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.729+0000 7fd4c5200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4c00d52c0 0x7fd4c00cb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:57.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.729+0000 7fd4c5200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4c0162180 0x7fd4c00c8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:57.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.729+0000 7fd4c5200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c006e260 con 0x7fd4c00b34b0 2024-04-20T21:18:57.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.729+0000 7fd4c5200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c006e0e0 con 0x7fd4c00d52c0 2024-04-20T21:18:57.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.729+0000 7fd4c5200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd4c00cff40 con 0x7fd4c0162180 2024-04-20T21:18:57.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.729+0000 7fd4bf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4c00b34b0 0x7fd4c00b7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:57.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.729+0000 7fd4bf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4c00b34b0 0x7fd4c00b7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55680/0 (socket says 172.21.15.86:55680) 2024-04-20T21:18:57.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.729+0000 7fd4bf400640 1 -- 172.21.15.86:0/1466266365 learned_addr learned my addr 172.21.15.86:0/1466266365 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:57.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.730+0000 7fd4b7e00640 1 --2- 172.21.15.86:0/1466266365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4c00d52c0 0x7fd4c00cb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:57.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.730+0000 7fd4bfe00640 1 --2- 172.21.15.86:0/1466266365 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4c0162180 0x7fd4c00c8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:57.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.730+0000 7fd4bf400640 1 -- 172.21.15.86:0/1466266365 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4c0162180 msgr2=0x7fd4c00c8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:57.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.730+0000 7fd4bf400640 1 --2- 172.21.15.86:0/1466266365 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4c0162180 0x7fd4c00c8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:57.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.730+0000 7fd4bf400640 1 -- 172.21.15.86:0/1466266365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4c00d52c0 msgr2=0x7fd4c00cb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:57.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.730+0000 7fd4bf400640 1 --2- 172.21.15.86:0/1466266365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4c00d52c0 0x7fd4c00cb6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:57.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.730+0000 7fd4bf400640 1 -- 172.21.15.86:0/1466266365 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4c00b83a0 con 0x7fd4c00b34b0 2024-04-20T21:18:57.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.730+0000 7fd4bf400640 1 --2- 172.21.15.86:0/1466266365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4c00b34b0 0x7fd4c00b7dc0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd4b000c7f0 tx=0x7fd4b000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:57.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.730+0000 7fd4b7400640 1 -- 172.21.15.86:0/1466266365 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b0015070 con 0x7fd4c00b34b0 2024-04-20T21:18:57.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.731+0000 7fd4b7400640 1 -- 172.21.15.86:0/1466266365 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4b0004240 con 0x7fd4c00b34b0 2024-04-20T21:18:57.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.731+0000 7fd4b7400640 1 -- 172.21.15.86:0/1466266365 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b0007620 con 0x7fd4c00b34b0 2024-04-20T21:18:57.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.731+0000 7fd4c5200640 1 -- 172.21.15.86:0/1466266365 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4c00b8680 con 0x7fd4c00b34b0 2024-04-20T21:18:57.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.731+0000 7fd4c5200640 1 -- 172.21.15.86:0/1466266365 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4c00b8c40 con 0x7fd4c00b34b0 2024-04-20T21:18:57.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.732+0000 7fd4c5200640 1 -- 172.21.15.86:0/1466266365 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4c004fcf0 con 0x7fd4c00b34b0 2024-04-20T21:18:57.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.733+0000 7fd4b7400640 1 -- 172.21.15.86:0/1466266365 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd4b00079d0 con 0x7fd4c00b34b0 2024-04-20T21:18:57.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.734+0000 7fd4b7400640 1 --2- 172.21.15.86:0/1466266365 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd49c06c8f0 0x7fd49c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:57.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.734+0000 7fd4b7400640 1 -- 172.21.15.86:0/1466266365 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fd4b008c0f0 con 0x7fd4c00b34b0 2024-04-20T21:18:57.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.734+0000 7fd4b7e00640 1 --2- 172.21.15.86:0/1466266365 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd49c06c8f0 0x7fd49c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:57.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.735+0000 7fd4b7e00640 1 --2- 172.21.15.86:0/1466266365 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd49c06c8f0 0x7fd49c06edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8036000 tx=0x7fd4a8008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:57.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.739+0000 7fd4b7400640 1 -- 172.21.15.86:0/1466266365 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd4b005a410 con 0x7fd4c00b34b0 2024-04-20T21:18:57.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.940+0000 7fd4c5200640 1 -- 172.21.15.86:0/1466266365 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd4c00d4fc0 con 0x7fd4c00b34b0 2024-04-20T21:18:57.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.943+0000 7fd4b7400640 1 -- 172.21.15.86:0/1466266365 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16854 (secure 0 0 0) 0x7fd4b0059d10 con 0x7fd4c00b34b0 2024-04-20T21:18:57.943 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:18:57.943 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":22,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:18:33.788155+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:17:49.718799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T21:17:50.683477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":5,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:18:57.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.946+0000 7fd4c5200640 1 -- 172.21.15.86:0/1466266365 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd49c06c8f0 msgr2=0x7fd49c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:57.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.946+0000 7fd4c5200640 1 --2- 172.21.15.86:0/1466266365 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd49c06c8f0 0x7fd49c06edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8036000 tx=0x7fd4a8008030 comp rx=0 tx=0).stop 2024-04-20T21:18:57.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.946+0000 7fd4c5200640 1 -- 172.21.15.86:0/1466266365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4c00b34b0 msgr2=0x7fd4c00b7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:57.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.946+0000 7fd4c5200640 1 --2- 172.21.15.86:0/1466266365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4c00b34b0 0x7fd4c00b7dc0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd4b000c7f0 tx=0x7fd4b000ccb0 comp rx=0 tx=0).stop 2024-04-20T21:18:57.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.946+0000 7fd4c5200640 1 -- 172.21.15.86:0/1466266365 shutdown_connections 2024-04-20T21:18:57.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.946+0000 7fd4c5200640 1 --2- 172.21.15.86:0/1466266365 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd49c06c8f0 0x7fd49c06edb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:57.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.946+0000 7fd4c5200640 1 --2- 172.21.15.86:0/1466266365 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4c0162180 0x7fd4c00c8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:57.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.946+0000 7fd4c5200640 1 --2- 172.21.15.86:0/1466266365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4c00d52c0 0x7fd4c00cb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:57.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.946+0000 7fd4c5200640 1 --2- 172.21.15.86:0/1466266365 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4c00b34b0 0x7fd4c00b7dc0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:57.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.946+0000 7fd4c5200640 1 -- 172.21.15.86:0/1466266365 >> 172.21.15.86:0/1466266365 conn(0x7fd4c008bf70 msgr2=0x7fd4c015f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:57.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.946+0000 7fd4c5200640 1 -- 172.21.15.86:0/1466266365 shutdown_connections 2024-04-20T21:18:57.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:57.946+0000 7fd4c5200640 1 -- 172.21.15.86:0/1466266365 wait complete. 2024-04-20T21:18:57.965 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-20T21:18:58.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.163+0000 7fe26ea00640 1 Processor -- start 2024-04-20T21:18:58.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.163+0000 7fe26ea00640 1 -- start start 2024-04-20T21:18:58.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.165+0000 7fe26ea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe2680cd7e0 0x7fe2681676e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:58.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.165+0000 7fe26ea00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe26815dcc0 con 0x7fe2680d09e0 2024-04-20T21:18:58.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.165+0000 7fe26ea00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe26815cf00 con 0x7fe268145fe0 2024-04-20T21:18:58.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.165+0000 7fe26ea00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe2680d00d0 con 0x7fe2680cd7e0 2024-04-20T21:18:58.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.165+0000 7fe26ca00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe2680cd7e0 0x7fe2681676e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:58.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.165+0000 7fe26ca00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe2680cd7e0 0x7fe2681676e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55682/0 (socket says 172.21.15.86:55682) 2024-04-20T21:18:58.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.165+0000 7fe267400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe2680d09e0 0x7fe2680cd0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44388/0 (socket says 172.21.15.86:44388) 2024-04-20T21:18:58.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.165+0000 7fe26ca00640 1 -- 172.21.15.86:0/4266351482 learned_addr learned my addr 172.21.15.86:0/4266351482 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:58.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.165+0000 7fe26ca00640 1 -- 172.21.15.86:0/4266351482 >> v1:172.21.15.144:6789/0 conn(0x7fe268145fe0 legacy=0x7fe2680d02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:58.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.165+0000 7fe26ca00640 1 -- 172.21.15.86:0/4266351482 >> v1:172.21.15.86:6789/0 conn(0x7fe2680d09e0 legacy=0x7fe2680cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:58.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.165+0000 7fe26ca00640 1 -- 172.21.15.86:0/4266351482 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe26815d1e0 con 0x7fe2680cd7e0 2024-04-20T21:18:58.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.165+0000 7fe266a00640 1 -- 172.21.15.86:0/4266351482 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1679018059 0 0) 0x7fe26815dcc0 con 0x7fe2680d09e0 2024-04-20T21:18:58.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.167+0000 7fe26ca00640 1 --2- 172.21.15.86:0/4266351482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2680cd7e0 0x7fe2681676e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe260009e50 tx=0x7fe26002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d392721f18be4378 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:58.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.167+0000 7fe266a00640 1 -- 172.21.15.86:0/4266351482 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26002cb20 con 0x7fe2680cd7e0 2024-04-20T21:18:58.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.167+0000 7fe266a00640 1 -- 172.21.15.86:0/4266351482 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe26002ccc0 con 0x7fe2680cd7e0 2024-04-20T21:18:58.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.167+0000 7fe266a00640 1 -- 172.21.15.86:0/4266351482 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe260034610 con 0x7fe2680cd7e0 2024-04-20T21:18:58.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.167+0000 7fe26ea00640 1 -- 172.21.15.86:0/4266351482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2680cd7e0 msgr2=0x7fe2681676e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:58.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.167+0000 7fe26ea00640 1 --2- 172.21.15.86:0/4266351482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2680cd7e0 0x7fe2681676e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe260009e50 tx=0x7fe26002bf80 comp rx=0 tx=0).stop 2024-04-20T21:18:58.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.168+0000 7fe26ea00640 1 -- 172.21.15.86:0/4266351482 shutdown_connections 2024-04-20T21:18:58.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.168+0000 7fe26ea00640 1 --2- 172.21.15.86:0/4266351482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2680cd7e0 0x7fe2681676e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:58.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.168+0000 7fe26ea00640 1 -- 172.21.15.86:0/4266351482 >> 172.21.15.86:0/4266351482 conn(0x7fe26806b980 msgr2=0x7fe26806bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:58.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.168+0000 7fe26ea00640 1 -- 172.21.15.86:0/4266351482 shutdown_connections 2024-04-20T21:18:58.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.168+0000 7fe26ea00640 1 -- 172.21.15.86:0/4266351482 wait complete. 2024-04-20T21:18:58.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.169+0000 7fe26ea00640 1 Processor -- start 2024-04-20T21:18:58.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.170+0000 7fe26ea00640 1 -- start start 2024-04-20T21:18:58.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.170+0000 7fe26ea00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2680cd7e0 0x7fe26813cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:58.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.170+0000 7fe26ea00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2680d09e0 0x7fe26813d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:58.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.170+0000 7fe26ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe268145fe0 0x7fe2680b0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:58.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.171+0000 7fe26ea00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe26815d7a0 con 0x7fe268145fe0 2024-04-20T21:18:58.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.171+0000 7fe26ea00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe2681463c0 con 0x7fe2680cd7e0 2024-04-20T21:18:58.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.171+0000 7fe26ea00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe2680cdce0 con 0x7fe2680d09e0 2024-04-20T21:18:58.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.171+0000 7fe26ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe268145fe0 0x7fe2680b0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:58.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.171+0000 7fe26ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe268145fe0 0x7fe2680b0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55692/0 (socket says 172.21.15.86:55692) 2024-04-20T21:18:58.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.171+0000 7fe26ca00640 1 -- 172.21.15.86:0/185979904 learned_addr learned my addr 172.21.15.86:0/185979904 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:58.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.171+0000 7fe26ca00640 1 -- 172.21.15.86:0/185979904 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2680d09e0 msgr2=0x7fe26813d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:58.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.171+0000 7fe26ca00640 1 --2- 172.21.15.86:0/185979904 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2680d09e0 0x7fe26813d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:58.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.171+0000 7fe26ca00640 1 -- 172.21.15.86:0/185979904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2680cd7e0 msgr2=0x7fe26813cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:58.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.171+0000 7fe267e00640 1 --2- 172.21.15.86:0/185979904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2680cd7e0 0x7fe26813cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:58.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.171+0000 7fe26ca00640 1 --2- 172.21.15.86:0/185979904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2680cd7e0 0x7fe26813cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:58.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.171+0000 7fe26ca00640 1 -- 172.21.15.86:0/185979904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe268140bd0 con 0x7fe268145fe0 2024-04-20T21:18:58.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.172+0000 7fe26ca00640 1 --2- 172.21.15.86:0/185979904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe268145fe0 0x7fe2680b0ff0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe260009f70 tx=0x7fe260004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:58.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.172+0000 7fe25fe00640 1 -- 172.21.15.86:0/185979904 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe260043030 con 0x7fe268145fe0 2024-04-20T21:18:58.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.172+0000 7fe25fe00640 1 -- 172.21.15.86:0/185979904 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe26003e450 con 0x7fe268145fe0 2024-04-20T21:18:58.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.172+0000 7fe26ea00640 1 -- 172.21.15.86:0/185979904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2681e6570 con 0x7fe268145fe0 2024-04-20T21:18:58.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.172+0000 7fe25fe00640 1 -- 172.21.15.86:0/185979904 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe260009510 con 0x7fe268145fe0 2024-04-20T21:18:58.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.172+0000 7fe26ea00640 1 -- 172.21.15.86:0/185979904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2681e6b30 con 0x7fe268145fe0 2024-04-20T21:18:58.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.173+0000 7fe26ea00640 1 -- 172.21.15.86:0/185979904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2680689e0 con 0x7fe268145fe0 2024-04-20T21:18:58.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.174+0000 7fe25fe00640 1 -- 172.21.15.86:0/185979904 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe260046070 con 0x7fe268145fe0 2024-04-20T21:18:58.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.175+0000 7fe25fe00640 1 --2- 172.21.15.86:0/185979904 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe24806c8f0 0x7fe24806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:58.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.175+0000 7fe25fe00640 1 -- 172.21.15.86:0/185979904 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fe2600ae1a0 con 0x7fe268145fe0 2024-04-20T21:18:58.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.175+0000 7fe267e00640 1 --2- 172.21.15.86:0/185979904 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe24806c8f0 0x7fe24806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:58.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.176+0000 7fe267e00640 1 --2- 172.21.15.86:0/185979904 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe24806c8f0 0x7fe24806edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe25800a750 tx=0x7fe258008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:58.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.179+0000 7fe25fe00640 1 -- 172.21.15.86:0/185979904 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe26004d070 con 0x7fe268145fe0 2024-04-20T21:18:58.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.401+0000 7fe26ea00640 1 -- 172.21.15.86:0/185979904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe26815c830 con 0x7fe268145fe0 2024-04-20T21:18:58.401 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-04-20T21:18:58.402+0000 7f5bb9c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T21:18:58.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.405+0000 7fe25fe00640 1 -- 172.21.15.86:0/185979904 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) ==== 161+0+0 (secure 0 0 0) 0x7fe26002cb30 con 0x7fe268145fe0 2024-04-20T21:18:58.405 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-20T21:18:58.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.408+0000 7fe26ea00640 1 -- 172.21.15.86:0/185979904 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe24806c8f0 msgr2=0x7fe24806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:58.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.408+0000 7fe26ea00640 1 --2- 172.21.15.86:0/185979904 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe24806c8f0 0x7fe24806edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe25800a750 tx=0x7fe258008040 comp rx=0 tx=0).stop 2024-04-20T21:18:58.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.408+0000 7fe26ea00640 1 -- 172.21.15.86:0/185979904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe268145fe0 msgr2=0x7fe2680b0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:58.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.408+0000 7fe26ea00640 1 --2- 172.21.15.86:0/185979904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe268145fe0 0x7fe2680b0ff0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe260009f70 tx=0x7fe260004a40 comp rx=0 tx=0).stop 2024-04-20T21:18:58.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.408+0000 7fe26ea00640 1 -- 172.21.15.86:0/185979904 shutdown_connections 2024-04-20T21:18:58.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.408+0000 7fe26ea00640 1 --2- 172.21.15.86:0/185979904 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe24806c8f0 0x7fe24806edb0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:58.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.408+0000 7fe26ea00640 1 --2- 172.21.15.86:0/185979904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe268145fe0 0x7fe2680b0ff0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:58.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.408+0000 7fe26ea00640 1 --2- 172.21.15.86:0/185979904 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2680d09e0 0x7fe26813d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:58.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.408+0000 7fe26ea00640 1 --2- 172.21.15.86:0/185979904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2680cd7e0 0x7fe26813cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:58.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.408+0000 7fe26ea00640 1 -- 172.21.15.86:0/185979904 >> 172.21.15.86:0/185979904 conn(0x7fe26806b980 msgr2=0x7fe2680d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:58.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.409+0000 7fe26ea00640 1 -- 172.21.15.86:0/185979904 shutdown_connections 2024-04-20T21:18:58.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.409+0000 7fe26ea00640 1 -- 172.21.15.86:0/185979904 wait complete. 2024-04-20T21:18:58.427 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-20T21:18:58.436 INFO:tasks.ceph.mds.d.smithi144.stderr: -4196> 2024-04-20T21:17:52.822+0000 7f61d2e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T21:18:58.507 INFO:tasks.ceph.mds.d.smithi144.stdout:starting mds.d at 2024-04-20T21:18:58.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.620+0000 7f269be00640 1 Processor -- start 2024-04-20T21:18:58.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.621+0000 7f269be00640 1 -- start start 2024-04-20T21:18:58.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.622+0000 7f269be00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f269c0ce7f0 0x7f269c0c5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:58.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.622+0000 7f269be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f269c0ccd60 0x7f269c165270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:58.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.622+0000 7f269be00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f269c0cc010 con 0x7f269c15b8f0 2024-04-20T21:18:58.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.622+0000 7f269be00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f269c0c5b70 con 0x7f269c0ccd60 2024-04-20T21:18:58.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.622+0000 7f269be00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f269c0cd210 con 0x7f269c0ce7f0 2024-04-20T21:18:58.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.623+0000 7f269b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f269c0ccd60 0x7f269c165270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:58.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.623+0000 7f269b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f269c0ccd60 0x7f269c165270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55696/0 (socket says 172.21.15.86:55696) 2024-04-20T21:18:58.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.623+0000 7f269b400640 1 -- 172.21.15.86:0/3494196974 learned_addr learned my addr 172.21.15.86:0/3494196974 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:58.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.623+0000 7f269a000640 1 --2- 172.21.15.86:0/3494196974 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f269c0ce7f0 0x7f269c0c5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:58.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.623+0000 7f269b400640 1 -- 172.21.15.86:0/3494196974 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f269c0ce7f0 msgr2=0x7f269c0c5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:58.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.623+0000 7f269b400640 1 --2- 172.21.15.86:0/3494196974 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f269c0ce7f0 0x7f269c0c5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:58.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.623+0000 7f269b400640 1 -- 172.21.15.86:0/3494196974 >> v1:172.21.15.144:6789/0 conn(0x7f269c15b8f0 legacy=0x7f269c0c4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:58.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.623+0000 7f269b400640 1 -- 172.21.15.86:0/3494196974 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f269c0cc280 con 0x7f269c0ccd60 2024-04-20T21:18:58.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.624+0000 7f269b400640 1 --2- 172.21.15.86:0/3494196974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f269c0ccd60 0x7f269c165270 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f269400bb10 tx=0x7f269402e150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc7e2e858e4e28c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:58.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.624+0000 7f2699600640 1 -- 172.21.15.86:0/3494196974 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269403b070 con 0x7f269c0ccd60 2024-04-20T21:18:58.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.625+0000 7f2699600640 1 -- 172.21.15.86:0/3494196974 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2694036440 con 0x7f269c0ccd60 2024-04-20T21:18:58.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.625+0000 7f2699600640 1 -- 172.21.15.86:0/3494196974 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2694036810 con 0x7f269c0ccd60 2024-04-20T21:18:58.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.625+0000 7f269be00640 1 -- 172.21.15.86:0/3494196974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f269c0ccd60 msgr2=0x7f269c165270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:58.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.625+0000 7f269be00640 1 --2- 172.21.15.86:0/3494196974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f269c0ccd60 0x7f269c165270 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f269400bb10 tx=0x7f269402e150 comp rx=0 tx=0).stop 2024-04-20T21:18:58.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.625+0000 7f269be00640 1 -- 172.21.15.86:0/3494196974 shutdown_connections 2024-04-20T21:18:58.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.625+0000 7f269be00640 1 --2- 172.21.15.86:0/3494196974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f269c0ccd60 0x7f269c165270 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:58.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.625+0000 7f269be00640 1 --2- 172.21.15.86:0/3494196974 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f269c0ce7f0 0x7f269c0c5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:58.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.625+0000 7f269be00640 1 -- 172.21.15.86:0/3494196974 >> 172.21.15.86:0/3494196974 conn(0x7f269c0b2f60 msgr2=0x7f269c0b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:58.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.625+0000 7f269be00640 1 -- 172.21.15.86:0/3494196974 shutdown_connections 2024-04-20T21:18:58.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.626+0000 7f269be00640 1 -- 172.21.15.86:0/3494196974 wait complete. 2024-04-20T21:18:58.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.627+0000 7f269be00640 1 Processor -- start 2024-04-20T21:18:58.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.627+0000 7f269be00640 1 -- start start 2024-04-20T21:18:58.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.628+0000 7f269be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f269c0ccd60 0x7f269c1f3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:58.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.628+0000 7f269be00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f269c0ce7f0 0x7f269c0b4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:58.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.628+0000 7f269be00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f269c15b8f0 0x7f269c0aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:58.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.628+0000 7f269be00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f269c0cecd0 con 0x7f269c0ccd60 2024-04-20T21:18:58.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.628+0000 7f269be00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f269c0cc840 con 0x7f269c15b8f0 2024-04-20T21:18:58.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.628+0000 7f269be00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f269c0d0100 con 0x7f269c0ce7f0 2024-04-20T21:18:58.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.628+0000 7f269aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f269c0ccd60 0x7f269c1f3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:58.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.628+0000 7f269aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f269c0ccd60 0x7f269c1f3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55704/0 (socket says 172.21.15.86:55704) 2024-04-20T21:18:58.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.628+0000 7f269aa00640 1 -- 172.21.15.86:0/3609645848 learned_addr learned my addr 172.21.15.86:0/3609645848 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:58.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.629+0000 7f269b400640 1 --2- 172.21.15.86:0/3609645848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f269c15b8f0 0x7f269c0aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:58.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.629+0000 7f269aa00640 1 -- 172.21.15.86:0/3609645848 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f269c0ce7f0 msgr2=0x7f269c0b4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:58.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.629+0000 7f269aa00640 1 --2- 172.21.15.86:0/3609645848 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f269c0ce7f0 0x7f269c0b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:58.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.629+0000 7f269aa00640 1 -- 172.21.15.86:0/3609645848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f269c15b8f0 msgr2=0x7f269c0aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:58.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.629+0000 7f269aa00640 1 --2- 172.21.15.86:0/3609645848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f269c15b8f0 0x7f269c0aeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:58.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.629+0000 7f269aa00640 1 -- 172.21.15.86:0/3609645848 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f269c1f3ea0 con 0x7f269c0ccd60 2024-04-20T21:18:58.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.629+0000 7f269b400640 1 --2- 172.21.15.86:0/3609645848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f269c15b8f0 0x7f269c0aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:18:58.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.629+0000 7f269aa00640 1 --2- 172.21.15.86:0/3609645848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f269c0ccd60 0x7f269c1f3530 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f268c00c730 tx=0x7f268c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:58.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.629+0000 7f2692a00640 1 -- 172.21.15.86:0/3609645848 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268c016020 con 0x7f269c0ccd60 2024-04-20T21:18:58.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.629+0000 7f2692a00640 1 -- 172.21.15.86:0/3609645848 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f268c006d50 con 0x7f269c0ccd60 2024-04-20T21:18:58.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.629+0000 7f269be00640 1 -- 172.21.15.86:0/3609645848 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f269c1f4180 con 0x7f269c0ccd60 2024-04-20T21:18:58.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.629+0000 7f2692a00640 1 -- 172.21.15.86:0/3609645848 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268c003bb0 con 0x7f269c0ccd60 2024-04-20T21:18:58.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.630+0000 7f269be00640 1 -- 172.21.15.86:0/3609645848 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f269c0ae650 con 0x7f269c0ccd60 2024-04-20T21:18:58.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.631+0000 7f269be00640 1 -- 172.21.15.86:0/3609645848 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f269c04fd90 con 0x7f269c0ccd60 2024-04-20T21:18:58.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.632+0000 7f2692a00640 1 -- 172.21.15.86:0/3609645848 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f268c00a850 con 0x7f269c0ccd60 2024-04-20T21:18:58.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.633+0000 7f2692a00640 1 --2- 172.21.15.86:0/3609645848 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f267806c8f0 0x7f267806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:58.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.633+0000 7f2692a00640 1 -- 172.21.15.86:0/3609645848 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7f268c08b870 con 0x7f269c0ccd60 2024-04-20T21:18:58.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.633+0000 7f269a000640 1 --2- 172.21.15.86:0/3609645848 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f267806c8f0 0x7f267806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:58.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.634+0000 7f269a000640 1 --2- 172.21.15.86:0/3609645848 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f267806c8f0 0x7f267806edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f269c1f4630 tx=0x7f2684008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:58.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.638+0000 7f2692a00640 1 -- 172.21.15.86:0/3609645848 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f268c020020 con 0x7f269c0ccd60 2024-04-20T21:18:58.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:58.878+0000 7f269be00640 1 -- 172.21.15.86:0/3609645848 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f269c0b7080 con 0x7f269c0ccd60 2024-04-20T21:18:59.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.416+0000 7f2692a00640 1 -- 172.21.15.86:0/3609645848 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v9) ==== 104+0+0 (secure 0 0 0) 0x7f268c019310 con 0x7f269c0ccd60 2024-04-20T21:18:59.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.419+0000 7f269be00640 1 -- 172.21.15.86:0/3609645848 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f267806c8f0 msgr2=0x7f267806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:59.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.419+0000 7f269be00640 1 --2- 172.21.15.86:0/3609645848 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f267806c8f0 0x7f267806edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f269c1f4630 tx=0x7f2684008040 comp rx=0 tx=0).stop 2024-04-20T21:18:59.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.419+0000 7f269be00640 1 -- 172.21.15.86:0/3609645848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f269c0ccd60 msgr2=0x7f269c1f3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:59.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.419+0000 7f269be00640 1 --2- 172.21.15.86:0/3609645848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f269c0ccd60 0x7f269c1f3530 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f268c00c730 tx=0x7f268c00cbf0 comp rx=0 tx=0).stop 2024-04-20T21:18:59.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.420+0000 7f269be00640 1 -- 172.21.15.86:0/3609645848 shutdown_connections 2024-04-20T21:18:59.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.420+0000 7f269be00640 1 --2- 172.21.15.86:0/3609645848 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f267806c8f0 0x7f267806edb0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:59.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.420+0000 7f269be00640 1 --2- 172.21.15.86:0/3609645848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f269c15b8f0 0x7f269c0aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:59.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.420+0000 7f269be00640 1 --2- 172.21.15.86:0/3609645848 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f269c0ce7f0 0x7f269c0b4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:59.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.420+0000 7f269be00640 1 --2- 172.21.15.86:0/3609645848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f269c0ccd60 0x7f269c1f3530 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:59.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.420+0000 7f269be00640 1 -- 172.21.15.86:0/3609645848 >> 172.21.15.86:0/3609645848 conn(0x7f269c0b2f60 msgr2=0x7f269c074c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:59.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.420+0000 7f269be00640 1 -- 172.21.15.86:0/3609645848 shutdown_connections 2024-04-20T21:18:59.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.420+0000 7f269be00640 1 -- 172.21.15.86:0/3609645848 wait complete. 2024-04-20T21:18:59.440 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-20T21:18:59.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.641+0000 7fc2dfa00640 1 Processor -- start 2024-04-20T21:18:59.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.641+0000 7fc2dfa00640 1 -- start start 2024-04-20T21:18:59.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.643+0000 7fc2dfa00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc2d80afb60 0x7fc2d806ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:59.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.643+0000 7fc2dfa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2d80cd8d0 0x7fc2d80b39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:59.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.643+0000 7fc2dfa00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2d815ca50 con 0x7fc2d806f280 2024-04-20T21:18:59.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.643+0000 7fc2dfa00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fc2d80cdef0 con 0x7fc2d80afb60 2024-04-20T21:18:59.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.643+0000 7fc2dfa00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc2d815bc90 con 0x7fc2d80cd8d0 2024-04-20T21:18:59.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.643+0000 7fc2d7e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fc2d806f280 0x7fc2d80cd1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44392/0 (socket says 172.21.15.86:44392) 2024-04-20T21:18:59.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.643+0000 7fc2d7e00640 1 -- 172.21.15.86:0/2948757053 learned_addr learned my addr 172.21.15.86:0/2948757053 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:18:59.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.643+0000 7fc2d7400640 1 -- 172.21.15.86:0/2948757053 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1123331946 0 0) 0x7fc2d815ca50 con 0x7fc2d806f280 2024-04-20T21:18:59.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.643+0000 7fc2dda00640 1 --2- 172.21.15.86:0/2948757053 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2d80cd8d0 0x7fc2d80b39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:59.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.643+0000 7fc2d7400640 1 -- 172.21.15.86:0/2948757053 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc2bc003520 con 0x7fc2d806f280 2024-04-20T21:18:59.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.643+0000 7fc2dd000640 1 --2- 172.21.15.86:0/2948757053 >> v2:172.21.15.144:3300/0 conn(0x7fc2d80afb60 0x7fc2d806ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:59.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.644+0000 7fc2d7400640 1 -- 172.21.15.86:0/2948757053 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1611812173 0 0) 0x7fc2bc003520 con 0x7fc2d806f280 2024-04-20T21:18:59.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.644+0000 7fc2d7400640 1 -- 172.21.15.86:0/2948757053 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2d80cd8d0 msgr2=0x7fc2d80b39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:59.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.644+0000 7fc2d7400640 1 --2- 172.21.15.86:0/2948757053 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2d80cd8d0 0x7fc2d80b39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:59.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.644+0000 7fc2d7400640 1 -- 172.21.15.86:0/2948757053 >> v2:172.21.15.144:3300/0 conn(0x7fc2d80afb60 msgr2=0x7fc2d806ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:59.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.644+0000 7fc2d7400640 1 --2- 172.21.15.86:0/2948757053 >> v2:172.21.15.144:3300/0 conn(0x7fc2d80afb60 0x7fc2d806ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:59.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.644+0000 7fc2d7400640 1 -- 172.21.15.86:0/2948757053 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2d815bf70 con 0x7fc2d806f280 2024-04-20T21:18:59.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.644+0000 7fc2d7400640 1 -- 172.21.15.86:0/2948757053 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fc2c40017f0 con 0x7fc2d806f280 2024-04-20T21:18:59.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.644+0000 7fc2d7400640 1 -- 172.21.15.86:0/2948757053 >> v1:172.21.15.86:6789/0 conn(0x7fc2d806f280 legacy=0x7fc2d80cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:59.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.644+0000 7fc2d7400640 1 --2- 172.21.15.86:0/2948757053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2bc0039b0 0x7fc2bc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:59.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.644+0000 7fc2d7400640 1 --2- 172.21.15.86:0/2948757053 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2bc004800 0x7fc2bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:59.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.644+0000 7fc2d7400640 1 --2- 172.21.15.86:0/2948757053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2bc0051c0 0x7fc2bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:59.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.644+0000 7fc2d7400640 1 -- 172.21.15.86:0/2948757053 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d80cdef0 con 0x7fc2bc0051c0 2024-04-20T21:18:59.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.644+0000 7fc2d7400640 1 -- 172.21.15.86:0/2948757053 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d815bc90 con 0x7fc2bc0039b0 2024-04-20T21:18:59.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.644+0000 7fc2d7400640 1 -- 172.21.15.86:0/2948757053 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc2bc00c000 con 0x7fc2bc004800 2024-04-20T21:18:59.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.644+0000 7fc2dda00640 1 --2- 172.21.15.86:0/2948757053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2bc0051c0 0x7fc2bc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:59.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.645+0000 7fc2dd000640 1 --2- 172.21.15.86:0/2948757053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2bc0039b0 0x7fc2bc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:59.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.645+0000 7fc2d7e00640 1 --2- 172.21.15.86:0/2948757053 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2bc004800 0x7fc2bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:59.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.645+0000 7fc2dda00640 1 -- 172.21.15.86:0/2948757053 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2bc004800 msgr2=0x7fc2bc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:59.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.645+0000 7fc2dda00640 1 --2- 172.21.15.86:0/2948757053 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2bc004800 0x7fc2bc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:59.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.645+0000 7fc2dda00640 1 -- 172.21.15.86:0/2948757053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2bc0039b0 msgr2=0x7fc2bc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:18:59.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.645+0000 7fc2dda00640 1 --2- 172.21.15.86:0/2948757053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2bc0039b0 0x7fc2bc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:59.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.645+0000 7fc2dda00640 1 -- 172.21.15.86:0/2948757053 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2bc003520 con 0x7fc2bc0051c0 2024-04-20T21:18:59.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.646+0000 7fc2dda00640 1 --2- 172.21.15.86:0/2948757053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2bc0051c0 0x7fc2bc00b9c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc2d000b900 tx=0x7fc2bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbad66f9bbd52417 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:59.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.646+0000 7fc2d7400640 1 -- 172.21.15.86:0/2948757053 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d000b550 con 0x7fc2bc0051c0 2024-04-20T21:18:59.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.646+0000 7fc2d7400640 1 -- 172.21.15.86:0/2948757053 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2d0034610 con 0x7fc2bc0051c0 2024-04-20T21:18:59.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.646+0000 7fc2d7400640 1 -- 172.21.15.86:0/2948757053 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d00349e0 con 0x7fc2bc0051c0 2024-04-20T21:18:59.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.647+0000 7fc2dfa00640 1 -- 172.21.15.86:0/2948757053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2bc0051c0 msgr2=0x7fc2bc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:59.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.647+0000 7fc2dfa00640 1 --2- 172.21.15.86:0/2948757053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2bc0051c0 0x7fc2bc00b9c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc2d000b900 tx=0x7fc2bc005830 comp rx=0 tx=0).stop 2024-04-20T21:18:59.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.647+0000 7fc2dda00640 1 -- 172.21.15.86:0/2948757053 reap_dead start 2024-04-20T21:18:59.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.647+0000 7fc2dfa00640 1 -- 172.21.15.86:0/2948757053 shutdown_connections 2024-04-20T21:18:59.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.647+0000 7fc2dfa00640 1 -- 172.21.15.86:0/2948757053 >> 172.21.15.86:0/2948757053 conn(0x7fc2d80b2180 msgr2=0x7fc2d80b2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:18:59.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.647+0000 7fc2dfa00640 1 -- 172.21.15.86:0/2948757053 shutdown_connections 2024-04-20T21:18:59.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.647+0000 7fc2dfa00640 1 -- 172.21.15.86:0/2948757053 wait complete. 2024-04-20T21:18:59.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.649+0000 7fc2dfa00640 1 Processor -- start 2024-04-20T21:18:59.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.649+0000 7fc2dfa00640 1 -- start start 2024-04-20T21:18:59.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.650+0000 7fc2dfa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2d8159d70 0x7fc2d814f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:59.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.650+0000 7fc2dfa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2d8149560 0x7fc2d81499c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:59.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.650+0000 7fc2dfa00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2d814d630 0x7fc2d814dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:59.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.650+0000 7fc2dfa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d806dc30 con 0x7fc2d8149560 2024-04-20T21:18:59.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.650+0000 7fc2dfa00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d815c530 con 0x7fc2d814d630 2024-04-20T21:18:59.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.650+0000 7fc2dfa00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc2d806ddb0 con 0x7fc2d8159d70 2024-04-20T21:18:59.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.650+0000 7fc2d7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2d8149560 0x7fc2d81499c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:59.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.650+0000 7fc2d7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2d8149560 0x7fc2d81499c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55714/0 (socket says 172.21.15.86:55714) 2024-04-20T21:18:59.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.650+0000 7fc2d7e00640 1 -- 172.21.15.86:0/37058098 learned_addr learned my addr 172.21.15.86:0/37058098 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:18:59.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.650+0000 7fc2d7e00640 1 -- 172.21.15.86:0/37058098 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2d8159d70 msgr2=0x7fc2d814f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:59.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.650+0000 7fc2d7e00640 1 --2- 172.21.15.86:0/37058098 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2d8159d70 0x7fc2d814f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:59.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.650+0000 7fc2dda00640 1 --2- 172.21.15.86:0/37058098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2d814d630 0x7fc2d814dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:59.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.650+0000 7fc2d7e00640 1 -- 172.21.15.86:0/37058098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2d814d630 msgr2=0x7fc2d814dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:18:59.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.651+0000 7fc2d7e00640 1 --2- 172.21.15.86:0/37058098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2d814d630 0x7fc2d814dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:18:59.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.651+0000 7fc2d7e00640 1 -- 172.21.15.86:0/37058098 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2d814e320 con 0x7fc2d8149560 2024-04-20T21:18:59.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.651+0000 7fc2d7e00640 1 --2- 172.21.15.86:0/37058098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2d8149560 0x7fc2d81499c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc2c400e7e0 tx=0x7fc2c400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:59.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.651+0000 7fc2d4c00640 1 -- 172.21.15.86:0/37058098 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c400ee90 con 0x7fc2d8149560 2024-04-20T21:18:59.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.651+0000 7fc2d4c00640 1 -- 172.21.15.86:0/37058098 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2c4018070 con 0x7fc2d8149560 2024-04-20T21:18:59.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.651+0000 7fc2d4c00640 1 -- 172.21.15.86:0/37058098 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c4007520 con 0x7fc2d8149560 2024-04-20T21:18:59.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.651+0000 7fc2dfa00640 1 -- 172.21.15.86:0/37058098 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2d814e600 con 0x7fc2d8149560 2024-04-20T21:18:59.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.651+0000 7fc2dfa00640 1 -- 172.21.15.86:0/37058098 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2d814ebc0 con 0x7fc2d8149560 2024-04-20T21:18:59.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.652+0000 7fc2dfa00640 1 -- 172.21.15.86:0/37058098 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2d80689e0 con 0x7fc2d8149560 2024-04-20T21:18:59.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.653+0000 7fc2d4c00640 1 -- 172.21.15.86:0/37058098 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc2c40057a0 con 0x7fc2d8149560 2024-04-20T21:18:59.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.654+0000 7fc2d4c00640 1 --2- 172.21.15.86:0/37058098 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc2b806ca90 0x7fc2b806ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:18:59.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.654+0000 7fc2d4c00640 1 -- 172.21.15.86:0/37058098 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7fc2c408bc60 con 0x7fc2d8149560 2024-04-20T21:18:59.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.655+0000 7fc2dd000640 1 --2- 172.21.15.86:0/37058098 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc2b806ca90 0x7fc2b806ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:18:59.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.655+0000 7fc2dd000640 1 --2- 172.21.15.86:0/37058098 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc2b806ca90 0x7fc2b806ef50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc2c80042d0 tx=0x7fc2c80070f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:18:59.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.659+0000 7fc2d4c00640 1 -- 172.21.15.86:0/37058098 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc2c4059e90 con 0x7fc2d8149560 2024-04-20T21:18:59.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:18:59.860+0000 7fc2dfa00640 1 -- 172.21.15.86:0/37058098 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc2d814fc20 con 0x7fc2d8149560 2024-04-20T21:19:00.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.419+0000 7fc2d4c00640 1 -- 172.21.15.86:0/37058098 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) ==== 181+0+0 (secure 0 0 0) 0x7fc2c4059790 con 0x7fc2d8149560 2024-04-20T21:19:00.419 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' removed 2024-04-20T21:19:00.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.424+0000 7fc2dfa00640 1 -- 172.21.15.86:0/37058098 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc2b806ca90 msgr2=0x7fc2b806ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:00.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.424+0000 7fc2dfa00640 1 --2- 172.21.15.86:0/37058098 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc2b806ca90 0x7fc2b806ef50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc2c80042d0 tx=0x7fc2c80070f0 comp rx=0 tx=0).stop 2024-04-20T21:19:00.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.424+0000 7fc2dfa00640 1 -- 172.21.15.86:0/37058098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2d8149560 msgr2=0x7fc2d81499c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:00.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.424+0000 7fc2dfa00640 1 --2- 172.21.15.86:0/37058098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2d8149560 0x7fc2d81499c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc2c400e7e0 tx=0x7fc2c400eca0 comp rx=0 tx=0).stop 2024-04-20T21:19:00.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.425+0000 7fc2dfa00640 1 -- 172.21.15.86:0/37058098 shutdown_connections 2024-04-20T21:19:00.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.425+0000 7fc2dfa00640 1 --2- 172.21.15.86:0/37058098 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc2b806ca90 0x7fc2b806ef50 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:00.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.425+0000 7fc2dfa00640 1 --2- 172.21.15.86:0/37058098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2d814d630 0x7fc2d814dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:00.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.425+0000 7fc2dfa00640 1 --2- 172.21.15.86:0/37058098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2d8149560 0x7fc2d81499c0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:00.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.425+0000 7fc2dfa00640 1 --2- 172.21.15.86:0/37058098 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2d8159d70 0x7fc2d814f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:00.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.425+0000 7fc2dfa00640 1 -- 172.21.15.86:0/37058098 >> 172.21.15.86:0/37058098 conn(0x7fc2d80b2180 msgr2=0x7fc2d8150420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:00.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.426+0000 7fc2dfa00640 1 -- 172.21.15.86:0/37058098 shutdown_connections 2024-04-20T21:19:00.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.426+0000 7fc2dfa00640 1 -- 172.21.15.86:0/37058098 wait complete. 2024-04-20T21:19:00.447 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-20T21:19:00.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.650+0000 7fae84400640 1 Processor -- start 2024-04-20T21:19:00.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.650+0000 7fae84400640 1 -- start start 2024-04-20T21:19:00.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.652+0000 7fae84400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fae7c0cd8d0 0x7fae7c1660b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:00.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.652+0000 7fae84400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae7c15ca50 con 0x7fae7c06f450 2024-04-20T21:19:00.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.652+0000 7fae84400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae7c0cdef0 con 0x7fae7c0afb60 2024-04-20T21:19:00.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.652+0000 7fae84400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fae7c15bc90 con 0x7fae7c0cd8d0 2024-04-20T21:19:00.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.653+0000 7fae81000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fae7c06f450 0x7fae7c0cd1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44398/0 (socket says 172.21.15.86:44398) 2024-04-20T21:19:00.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.653+0000 7fae81000640 1 -- 172.21.15.86:0/3177688161 learned_addr learned my addr 172.21.15.86:0/3177688161 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:19:00.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.653+0000 7fae7be00640 1 -- 172.21.15.86:0/3177688161 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2781646868 0 0) 0x7fae7c15ca50 con 0x7fae7c06f450 2024-04-20T21:19:00.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.653+0000 7fae82400640 1 --2- 172.21.15.86:0/3177688161 >> v2:172.21.15.144:3300/0 conn(0x7fae7c0cd8d0 0x7fae7c1660b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:00.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.653+0000 7fae7be00640 1 -- 172.21.15.86:0/3177688161 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fae64003520 con 0x7fae7c06f450 2024-04-20T21:19:00.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.653+0000 7fae7be00640 1 -- 172.21.15.86:0/3177688161 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3481563725 0 0) 0x7fae64003520 con 0x7fae7c06f450 2024-04-20T21:19:00.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.653+0000 7fae7be00640 1 -- 172.21.15.86:0/3177688161 >> v2:172.21.15.144:3300/0 conn(0x7fae7c0cd8d0 msgr2=0x7fae7c1660b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:00.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.653+0000 7fae7be00640 1 --2- 172.21.15.86:0/3177688161 >> v2:172.21.15.144:3300/0 conn(0x7fae7c0cd8d0 0x7fae7c1660b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:00.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.653+0000 7fae7be00640 1 -- 172.21.15.86:0/3177688161 >> v1:172.21.15.144:6789/0 conn(0x7fae7c0afb60 legacy=0x7fae7c06ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:00.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.653+0000 7fae7be00640 1 -- 172.21.15.86:0/3177688161 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae7c15bf70 con 0x7fae7c06f450 2024-04-20T21:19:00.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.653+0000 7fae7be00640 1 -- 172.21.15.86:0/3177688161 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fae680017f0 con 0x7fae7c06f450 2024-04-20T21:19:00.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.653+0000 7fae7be00640 1 -- 172.21.15.86:0/3177688161 >> v1:172.21.15.86:6789/0 conn(0x7fae7c06f450 legacy=0x7fae7c0cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:00.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.653+0000 7fae7be00640 1 --2- 172.21.15.86:0/3177688161 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae64003a00 0x7fae64003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:00.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.654+0000 7fae7be00640 1 --2- 172.21.15.86:0/3177688161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae64004800 0x7fae64004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:00.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.654+0000 7fae7be00640 1 --2- 172.21.15.86:0/3177688161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae64005210 0x7fae6400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:00.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.654+0000 7fae7be00640 1 -- 172.21.15.86:0/3177688161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fae7c15bc90 con 0x7fae64004800 2024-04-20T21:19:00.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.654+0000 7fae7be00640 1 -- 172.21.15.86:0/3177688161 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fae6400c140 con 0x7fae64005210 2024-04-20T21:19:00.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.654+0000 7fae7be00640 1 -- 172.21.15.86:0/3177688161 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fae6400c340 con 0x7fae64003a00 2024-04-20T21:19:00.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.654+0000 7fae81000640 1 --2- 172.21.15.86:0/3177688161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae64004800 0x7fae64004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:00.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.654+0000 7fae82400640 1 --2- 172.21.15.86:0/3177688161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae64005210 0x7fae6400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:00.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.654+0000 7fae81a00640 1 --2- 172.21.15.86:0/3177688161 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae64003a00 0x7fae64003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:00.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.654+0000 7fae81000640 1 -- 172.21.15.86:0/3177688161 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae64003a00 msgr2=0x7fae64003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:00.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.654+0000 7fae81000640 1 --2- 172.21.15.86:0/3177688161 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae64003a00 0x7fae64003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:00.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.654+0000 7fae81000640 1 -- 172.21.15.86:0/3177688161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae64005210 msgr2=0x7fae6400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:00.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.654+0000 7fae81000640 1 --2- 172.21.15.86:0/3177688161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae64005210 0x7fae6400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:00.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.654+0000 7fae81000640 1 -- 172.21.15.86:0/3177688161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae64003520 con 0x7fae64004800 2024-04-20T21:19:00.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.654+0000 7fae82400640 1 --2- 172.21.15.86:0/3177688161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae64005210 0x7fae6400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:00.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.654+0000 7fae82400640 1 -- 172.21.15.86:0/3177688161 reap_dead start 2024-04-20T21:19:00.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.656+0000 7fae81000640 1 --2- 172.21.15.86:0/3177688161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae64004800 0x7fae64004c80 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fae68001eb0 tx=0x7fae64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b759f4ca084d114 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:00.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.656+0000 7fae7be00640 1 -- 172.21.15.86:0/3177688161 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae68007c60 con 0x7fae64004800 2024-04-20T21:19:00.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.656+0000 7fae7be00640 1 -- 172.21.15.86:0/3177688161 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae68030070 con 0x7fae64004800 2024-04-20T21:19:00.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.656+0000 7fae7be00640 1 -- 172.21.15.86:0/3177688161 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae68007510 con 0x7fae64004800 2024-04-20T21:19:00.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.657+0000 7fae84400640 1 -- 172.21.15.86:0/3177688161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae64004800 msgr2=0x7fae64004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:00.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.657+0000 7fae84400640 1 --2- 172.21.15.86:0/3177688161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae64004800 0x7fae64004c80 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fae68001eb0 tx=0x7fae64005830 comp rx=0 tx=0).stop 2024-04-20T21:19:00.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.657+0000 7fae84400640 1 -- 172.21.15.86:0/3177688161 shutdown_connections 2024-04-20T21:19:00.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.657+0000 7fae84400640 1 --2- 172.21.15.86:0/3177688161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae64004800 0x7fae64004c80 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:00.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.657+0000 7fae84400640 1 -- 172.21.15.86:0/3177688161 >> 172.21.15.86:0/3177688161 conn(0x7fae7c0b2180 msgr2=0x7fae7c0b2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:00.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.657+0000 7fae84400640 1 -- 172.21.15.86:0/3177688161 shutdown_connections 2024-04-20T21:19:00.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.657+0000 7fae84400640 1 -- 172.21.15.86:0/3177688161 wait complete. 2024-04-20T21:19:00.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.658+0000 7fae84400640 1 Processor -- start 2024-04-20T21:19:00.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.659+0000 7fae84400640 1 -- start start 2024-04-20T21:19:00.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.659+0000 7fae84400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae64004800 0x7fae7c15dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:00.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.659+0000 7fae84400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae7c157500 0x7fae7c157960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:00.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.659+0000 7fae84400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae7c14a730 0x7fae7c14abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:00.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.659+0000 7fae84400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fae7c0cd1c0 con 0x7fae7c14a730 2024-04-20T21:19:00.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.659+0000 7fae84400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fae7c15c530 con 0x7fae7c157500 2024-04-20T21:19:00.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.659+0000 7fae84400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fae7c0cd340 con 0x7fae64004800 2024-04-20T21:19:00.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.660+0000 7fae82400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae7c14a730 0x7fae7c14abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:00.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.660+0000 7fae82400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae7c14a730 0x7fae7c14abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55718/0 (socket says 172.21.15.86:55718) 2024-04-20T21:19:00.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.660+0000 7fae82400640 1 -- 172.21.15.86:0/3041028135 learned_addr learned my addr 172.21.15.86:0/3041028135 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:00.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.660+0000 7fae81a00640 1 --2- 172.21.15.86:0/3041028135 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae64004800 0x7fae7c15dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:00.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.660+0000 7fae81000640 1 --2- 172.21.15.86:0/3041028135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae7c157500 0x7fae7c157960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:00.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.660+0000 7fae82400640 1 -- 172.21.15.86:0/3041028135 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae64004800 msgr2=0x7fae7c15dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:00.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.660+0000 7fae82400640 1 --2- 172.21.15.86:0/3041028135 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae64004800 0x7fae7c15dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:00.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.660+0000 7fae82400640 1 -- 172.21.15.86:0/3041028135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae7c157500 msgr2=0x7fae7c157960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:00.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.660+0000 7fae82400640 1 --2- 172.21.15.86:0/3041028135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae7c157500 0x7fae7c157960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:00.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.660+0000 7fae82400640 1 -- 172.21.15.86:0/3041028135 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae7c157fa0 con 0x7fae7c14a730 2024-04-20T21:19:00.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.660+0000 7fae81a00640 1 --2- 172.21.15.86:0/3041028135 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae64004800 0x7fae7c15dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:00.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.660+0000 7fae81000640 1 --2- 172.21.15.86:0/3041028135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae7c157500 0x7fae7c157960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:00.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.661+0000 7fae82400640 1 --2- 172.21.15.86:0/3041028135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae7c14a730 0x7fae7c14abb0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fae7c0b5040 tx=0x7fae74009d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:00.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.661+0000 7fae79600640 1 -- 172.21.15.86:0/3041028135 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae74019070 con 0x7fae7c14a730 2024-04-20T21:19:00.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.661+0000 7fae79600640 1 -- 172.21.15.86:0/3041028135 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae74007cf0 con 0x7fae7c14a730 2024-04-20T21:19:00.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.661+0000 7fae79600640 1 -- 172.21.15.86:0/3041028135 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae74014640 con 0x7fae7c14a730 2024-04-20T21:19:00.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.661+0000 7fae84400640 1 -- 172.21.15.86:0/3041028135 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae7c158280 con 0x7fae7c14a730 2024-04-20T21:19:00.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.661+0000 7fae84400640 1 -- 172.21.15.86:0/3041028135 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae7c158840 con 0x7fae7c14a730 2024-04-20T21:19:00.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.662+0000 7fae84400640 1 -- 172.21.15.86:0/3041028135 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae7c0689e0 con 0x7fae7c14a730 2024-04-20T21:19:00.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.663+0000 7fae79600640 1 -- 172.21.15.86:0/3041028135 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fae7401c070 con 0x7fae7c14a730 2024-04-20T21:19:00.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.664+0000 7fae79600640 1 --2- 172.21.15.86:0/3041028135 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fae5c06c8f0 0x7fae5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:00.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.664+0000 7fae79600640 1 -- 172.21.15.86:0/3041028135 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5473+0+0 (secure 0 0 0) 0x7fae7408b7d0 con 0x7fae7c14a730 2024-04-20T21:19:00.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.664+0000 7fae81a00640 1 --2- 172.21.15.86:0/3041028135 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fae5c06c8f0 0x7fae5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:00.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.665+0000 7fae81a00640 1 --2- 172.21.15.86:0/3041028135 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fae5c06c8f0 0x7fae5c06edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fae7c06a0f0 tx=0x7fae70009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:00.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.668+0000 7fae79600640 1 -- 172.21.15.86:0/3041028135 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fae74059bb0 con 0x7fae7c14a730 2024-04-20T21:19:00.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:00.867+0000 7fae84400640 1 -- 172.21.15.86:0/3041028135 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fae7c0b2700 con 0x7fae7c14a730 2024-04-20T21:19:01.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.429+0000 7fae79600640 1 -- 172.21.15.86:0/3041028135 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) ==== 169+0+0 (secure 0 0 0) 0x7fae740594b0 con 0x7fae7c14a730 2024-04-20T21:19:01.429 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' removed 2024-04-20T21:19:01.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.433+0000 7fae84400640 1 -- 172.21.15.86:0/3041028135 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fae5c06c8f0 msgr2=0x7fae5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:01.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.433+0000 7fae84400640 1 --2- 172.21.15.86:0/3041028135 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fae5c06c8f0 0x7fae5c06edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fae7c06a0f0 tx=0x7fae70009040 comp rx=0 tx=0).stop 2024-04-20T21:19:01.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.433+0000 7fae84400640 1 -- 172.21.15.86:0/3041028135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae7c14a730 msgr2=0x7fae7c14abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:01.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.433+0000 7fae84400640 1 --2- 172.21.15.86:0/3041028135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae7c14a730 0x7fae7c14abb0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fae7c0b5040 tx=0x7fae74009d30 comp rx=0 tx=0).stop 2024-04-20T21:19:01.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.433+0000 7fae84400640 1 -- 172.21.15.86:0/3041028135 shutdown_connections 2024-04-20T21:19:01.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.433+0000 7fae84400640 1 --2- 172.21.15.86:0/3041028135 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fae5c06c8f0 0x7fae5c06edb0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:01.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.433+0000 7fae84400640 1 --2- 172.21.15.86:0/3041028135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fae7c14a730 0x7fae7c14abb0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:01.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.433+0000 7fae84400640 1 --2- 172.21.15.86:0/3041028135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae7c157500 0x7fae7c157960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:01.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.433+0000 7fae84400640 1 --2- 172.21.15.86:0/3041028135 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fae64004800 0x7fae7c15dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:01.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.433+0000 7fae84400640 1 -- 172.21.15.86:0/3041028135 >> 172.21.15.86:0/3041028135 conn(0x7fae7c0b2180 msgr2=0x7fae7c0b00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:01.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.434+0000 7fae84400640 1 -- 172.21.15.86:0/3041028135 shutdown_connections 2024-04-20T21:19:01.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.434+0000 7fae84400640 1 -- 172.21.15.86:0/3041028135 wait complete. 2024-04-20T21:19:01.464 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-20T21:19:01.464 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-20T21:19:01.464 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-20T21:19:01.464 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:19:01.464 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-20T21:19:01.465 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-20T21:19:01.465 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-20T21:19:01.465 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:19:01.465 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-20T21:19:01.465 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-20T21:19:01.465 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-20T21:19:01.466 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:19:01.466 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-20T21:19:01.466 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-20T21:19:01.466 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-20T21:19:01.466 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:19:01.485 INFO:tasks.ceph.mds.c:Stopped 2024-04-20T21:19:01.485 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-20T21:19:01.486 INFO:tasks.ceph.mds.a:Stopped 2024-04-20T21:19:01.486 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-20T21:19:01.487 INFO:tasks.ceph.mds.d:Stopped 2024-04-20T21:19:01.487 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-20T21:19:01.491 INFO:tasks.ceph.mds.b:Stopped 2024-04-20T21:19:01.491 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-20T21:19:01.533 INFO:tasks.ceph.mds.c:Started 2024-04-20T21:19:01.534 DEBUG:teuthology.parallel:result is None 2024-04-20T21:19:01.538 INFO:tasks.ceph.mds.d:Started 2024-04-20T21:19:01.538 DEBUG:teuthology.parallel:result is None 2024-04-20T21:19:01.539 INFO:tasks.ceph.mds.a:Started 2024-04-20T21:19:01.539 DEBUG:teuthology.parallel:result is None 2024-04-20T21:19:01.544 INFO:tasks.ceph.mds.b:Started 2024-04-20T21:19:01.544 DEBUG:teuthology.parallel:result is None 2024-04-20T21:19:01.545 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-20T21:19:01.685 INFO:tasks.ceph.mds.c.smithi086.stdout:starting mds.c at 2024-04-20T21:19:01.685 INFO:tasks.ceph.mds.a.smithi086.stdout:starting mds.a at 2024-04-20T21:19:01.694 INFO:tasks.ceph.mds.d.smithi144.stdout:starting mds.d at 2024-04-20T21:19:01.694 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2024-04-20T21:19:01.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.726+0000 7efccb200640 1 Processor -- start 2024-04-20T21:19:01.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.727+0000 7efccb200640 1 -- start start 2024-04-20T21:19:01.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.728+0000 7efccb200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7efccc0d2cb0 0x7efccc0d1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:01.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.728+0000 7efccb200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efccc0d3100 con 0x7efccc0d0ee0 2024-04-20T21:19:01.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.728+0000 7efccb200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efccc0d33b0 con 0x7efccc15e690 2024-04-20T21:19:01.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.728+0000 7efccb200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7efccc0d18b0 con 0x7efccc0d2cb0 2024-04-20T21:19:01.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.728+0000 7efcc1400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7efccc0d0ee0 0x7efccc0d25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44404/0 (socket says 172.21.15.86:44404) 2024-04-20T21:19:01.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.728+0000 7efcc1400640 1 -- 172.21.15.86:0/2026805777 learned_addr learned my addr 172.21.15.86:0/2026805777 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:19:01.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.728+0000 7efcca800640 1 --2- 172.21.15.86:0/2026805777 >> v2:172.21.15.144:3300/0 conn(0x7efccc0d2cb0 0x7efccc0d1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:01.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.728+0000 7efcc9400640 1 -- 172.21.15.86:0/2026805777 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4107878512 0 0) 0x7efccc0d3100 con 0x7efccc0d0ee0 2024-04-20T21:19:01.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.728+0000 7efcc9400640 1 -- 172.21.15.86:0/2026805777 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efcb0003520 con 0x7efccc0d0ee0 2024-04-20T21:19:01.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.728+0000 7efcc9400640 1 -- 172.21.15.86:0/2026805777 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3674560165 0 0) 0x7efccc0d33b0 con 0x7efccc15e690 2024-04-20T21:19:01.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.729+0000 7efcc9400640 1 -- 172.21.15.86:0/2026805777 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efccc0d3100 con 0x7efccc15e690 2024-04-20T21:19:01.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.729+0000 7efcca800640 1 -- 172.21.15.86:0/2026805777 >> v1:172.21.15.144:6789/0 conn(0x7efccc15e690 legacy=0x7efccc06ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:01.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.729+0000 7efcca800640 1 -- 172.21.15.86:0/2026805777 >> v1:172.21.15.86:6789/0 conn(0x7efccc0d0ee0 legacy=0x7efccc0d25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:01.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.729+0000 7efcca800640 1 -- 172.21.15.86:0/2026805777 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efccc0d3690 con 0x7efccc0d2cb0 2024-04-20T21:19:01.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.730+0000 7efcca800640 1 --2- 172.21.15.86:0/2026805777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efccc0d2cb0 0x7efccc0d1370 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efcc4009d90 tx=0x7efcc402bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2ebcf05231ccf49 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:01.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.730+0000 7efcc9400640 1 -- 172.21.15.86:0/2026805777 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc4039070 con 0x7efccc0d2cb0 2024-04-20T21:19:01.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.730+0000 7efcc9400640 1 -- 172.21.15.86:0/2026805777 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcc402cd00 con 0x7efccc0d2cb0 2024-04-20T21:19:01.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.730+0000 7efcc9400640 1 -- 172.21.15.86:0/2026805777 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc4034660 con 0x7efccc0d2cb0 2024-04-20T21:19:01.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.731+0000 7efccb200640 1 -- 172.21.15.86:0/2026805777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efccc0d2cb0 msgr2=0x7efccc0d1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:01.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.731+0000 7efccb200640 1 --2- 172.21.15.86:0/2026805777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efccc0d2cb0 0x7efccc0d1370 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efcc4009d90 tx=0x7efcc402bf50 comp rx=0 tx=0).stop 2024-04-20T21:19:01.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.731+0000 7efccb200640 1 -- 172.21.15.86:0/2026805777 shutdown_connections 2024-04-20T21:19:01.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.731+0000 7efccb200640 1 --2- 172.21.15.86:0/2026805777 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efccc0d2cb0 0x7efccc0d1370 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:01.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.731+0000 7efccb200640 1 -- 172.21.15.86:0/2026805777 >> 172.21.15.86:0/2026805777 conn(0x7efccc09d8a0 msgr2=0x7efccc08bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:01.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.731+0000 7efccb200640 1 -- 172.21.15.86:0/2026805777 shutdown_connections 2024-04-20T21:19:01.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.731+0000 7efccb200640 1 -- 172.21.15.86:0/2026805777 wait complete. 2024-04-20T21:19:01.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.733+0000 7efccb200640 1 Processor -- start 2024-04-20T21:19:01.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.733+0000 7efccb200640 1 -- start start 2024-04-20T21:19:01.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.734+0000 7efccb200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efccc0d0ee0 0x7efccc1f4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:01.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.734+0000 7efccb200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efccc0d2cb0 0x7efccc0c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:01.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.734+0000 7efccb200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efccc15e690 0x7efccc0c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:01.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.734+0000 7efccb200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efccc0cef00 con 0x7efccc0d2cb0 2024-04-20T21:19:01.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.734+0000 7efccb200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efccc0cec20 con 0x7efccc15e690 2024-04-20T21:19:01.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.734+0000 7efccb200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7efccc15ea70 con 0x7efccc0d0ee0 2024-04-20T21:19:01.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.734+0000 7efcc1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efccc0d2cb0 0x7efccc0c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:01.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.734+0000 7efcc1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efccc0d2cb0 0x7efccc0c8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55758/0 (socket says 172.21.15.86:55758) 2024-04-20T21:19:01.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.734+0000 7efcc1400640 1 -- 172.21.15.86:0/1911311350 learned_addr learned my addr 172.21.15.86:0/1911311350 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:01.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.734+0000 7efcca800640 1 --2- 172.21.15.86:0/1911311350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efccc15e690 0x7efccc0c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:01.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.734+0000 7efcc9e00640 1 --2- 172.21.15.86:0/1911311350 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efccc0d0ee0 0x7efccc1f4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:01.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.734+0000 7efcc1400640 1 -- 172.21.15.86:0/1911311350 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efccc0d0ee0 msgr2=0x7efccc1f4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:01.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.734+0000 7efcc1400640 1 --2- 172.21.15.86:0/1911311350 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efccc0d0ee0 0x7efccc1f4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:01.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.734+0000 7efcc1400640 1 -- 172.21.15.86:0/1911311350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efccc15e690 msgr2=0x7efccc0c8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:01.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.734+0000 7efcc1400640 1 --2- 172.21.15.86:0/1911311350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efccc15e690 0x7efccc0c8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:01.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.734+0000 7efcc1400640 1 -- 172.21.15.86:0/1911311350 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efccc156f00 con 0x7efccc0d2cb0 2024-04-20T21:19:01.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.735+0000 7efcc1400640 1 --2- 172.21.15.86:0/1911311350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efccc0d2cb0 0x7efccc0c8020 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7efcb400c4d0 tx=0x7efcb400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:01.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.735+0000 7efcc2a00640 1 -- 172.21.15.86:0/1911311350 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb4017070 con 0x7efccc0d2cb0 2024-04-20T21:19:01.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.735+0000 7efcc2a00640 1 -- 172.21.15.86:0/1911311350 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcb4007480 con 0x7efccc0d2cb0 2024-04-20T21:19:01.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.735+0000 7efcc2a00640 1 -- 172.21.15.86:0/1911311350 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb40059a0 con 0x7efccc0d2cb0 2024-04-20T21:19:01.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.735+0000 7efccb200640 1 -- 172.21.15.86:0/1911311350 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efccc1571e0 con 0x7efccc0d2cb0 2024-04-20T21:19:01.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.735+0000 7efccb200640 1 -- 172.21.15.86:0/1911311350 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efccc1577a0 con 0x7efccc0d2cb0 2024-04-20T21:19:01.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.736+0000 7efccb200640 1 -- 172.21.15.86:0/1911311350 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efccc04fcf0 con 0x7efccc0d2cb0 2024-04-20T21:19:01.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.737+0000 7efcc2a00640 1 -- 172.21.15.86:0/1911311350 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efcb401a070 con 0x7efccc0d2cb0 2024-04-20T21:19:01.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.738+0000 7efcc2a00640 1 --2- 172.21.15.86:0/1911311350 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efca806c8f0 0x7efca806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:01.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.739+0000 7efcc2a00640 1 -- 172.21.15.86:0/1911311350 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4943+0+0 (secure 0 0 0) 0x7efcb408ab80 con 0x7efccc0d2cb0 2024-04-20T21:19:01.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.739+0000 7efcc9e00640 1 --2- 172.21.15.86:0/1911311350 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efca806c8f0 0x7efca806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:01.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.739+0000 7efcc9e00640 1 --2- 172.21.15.86:0/1911311350 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efca806c8f0 0x7efca806edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efcbc007fd0 tx=0x7efcbc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:01.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.743+0000 7efcc2a00640 1 -- 172.21.15.86:0/1911311350 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7efcb4058bd0 con 0x7efccc0d2cb0 2024-04-20T21:19:01.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:01.955+0000 7efccb200640 1 -- 172.21.15.86:0/1911311350 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7efccc06cc00 con 0x7efccc0d2cb0 2024-04-20T21:19:02.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.458+0000 7efcc2a00640 1 -- 172.21.15.86:0/1911311350 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v27) ==== 97+0+0 (secure 0 0 0) 0x7efcb405ced0 con 0x7efccc0d2cb0 2024-04-20T21:19:02.457 INFO:teuthology.orchestra.run.smithi086.stderr: removed all blocklist entries 2024-04-20T21:19:02.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.461+0000 7efccb200640 1 -- 172.21.15.86:0/1911311350 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efca806c8f0 msgr2=0x7efca806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:02.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.461+0000 7efccb200640 1 --2- 172.21.15.86:0/1911311350 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efca806c8f0 0x7efca806edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efcbc007fd0 tx=0x7efcbc009040 comp rx=0 tx=0).stop 2024-04-20T21:19:02.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.461+0000 7efccb200640 1 -- 172.21.15.86:0/1911311350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efccc0d2cb0 msgr2=0x7efccc0c8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:02.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.461+0000 7efccb200640 1 --2- 172.21.15.86:0/1911311350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efccc0d2cb0 0x7efccc0c8020 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7efcb400c4d0 tx=0x7efcb400c990 comp rx=0 tx=0).stop 2024-04-20T21:19:02.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.461+0000 7efccb200640 1 -- 172.21.15.86:0/1911311350 shutdown_connections 2024-04-20T21:19:02.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.461+0000 7efccb200640 1 --2- 172.21.15.86:0/1911311350 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efca806c8f0 0x7efca806edb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:02.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.461+0000 7efccb200640 1 --2- 172.21.15.86:0/1911311350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efccc15e690 0x7efccc0c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:02.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.461+0000 7efccb200640 1 --2- 172.21.15.86:0/1911311350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efccc0d2cb0 0x7efccc0c8020 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:02.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.461+0000 7efccb200640 1 --2- 172.21.15.86:0/1911311350 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efccc0d0ee0 0x7efccc1f4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:02.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.461+0000 7efccb200640 1 -- 172.21.15.86:0/1911311350 >> 172.21.15.86:0/1911311350 conn(0x7efccc09d8a0 msgr2=0x7efccc153690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:02.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.462+0000 7efccb200640 1 -- 172.21.15.86:0/1911311350 shutdown_connections 2024-04-20T21:19:02.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.462+0000 7efccb200640 1 -- 172.21.15.86:0/1911311350 wait complete. 2024-04-20T21:19:02.486 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-20T21:19:02.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.665+0000 7f0295c00640 1 Processor -- start 2024-04-20T21:19:02.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.666+0000 7f0295c00640 1 -- start start 2024-04-20T21:19:02.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.667+0000 7f0295c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f02900d0450 0x7f029009a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:02.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.667+0000 7f0295c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f029015d1b0 con 0x7f02900cd640 2024-04-20T21:19:02.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.667+0000 7f0295c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f029009b050 con 0x7f0290070480 2024-04-20T21:19:02.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.667+0000 7f0295c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f029009b330 con 0x7f02900d0450 2024-04-20T21:19:02.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.668+0000 7f028f400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f02900cd640 0x7f02900cdad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44414/0 (socket says 172.21.15.86:44414) 2024-04-20T21:19:02.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.668+0000 7f028f400640 1 -- 172.21.15.86:0/3626913898 learned_addr learned my addr 172.21.15.86:0/3626913898 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:19:02.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.668+0000 7f0286000640 1 --2- 172.21.15.86:0/3626913898 >> v2:172.21.15.144:3300/0 conn(0x7f02900d0450 0x7f029009a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:02.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.668+0000 7f028e000640 1 -- 172.21.15.86:0/3626913898 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4282184880 0 0) 0x7f029015d1b0 con 0x7f02900cd640 2024-04-20T21:19:02.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.668+0000 7f028e000640 1 -- 172.21.15.86:0/3626913898 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0274003520 con 0x7f02900cd640 2024-04-20T21:19:02.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.669+0000 7f028e000640 1 -- 172.21.15.86:0/3626913898 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1210979082 0 0) 0x7f0274003520 con 0x7f02900cd640 2024-04-20T21:19:02.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.669+0000 7f028e000640 1 -- 172.21.15.86:0/3626913898 >> v2:172.21.15.144:3300/0 conn(0x7f02900d0450 msgr2=0x7f029009a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:02.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.669+0000 7f028e000640 1 --2- 172.21.15.86:0/3626913898 >> v2:172.21.15.144:3300/0 conn(0x7f02900d0450 0x7f029009a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:02.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.669+0000 7f028e000640 1 -- 172.21.15.86:0/3626913898 >> v1:172.21.15.144:6789/0 conn(0x7f0290070480 legacy=0x7f0290070860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:02.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.669+0000 7f028e000640 1 -- 172.21.15.86:0/3626913898 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f029009b610 con 0x7f02900cd640 2024-04-20T21:19:02.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.669+0000 7f028e000640 1 -- 172.21.15.86:0/3626913898 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f0288001670 con 0x7f02900cd640 2024-04-20T21:19:02.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.669+0000 7f028e000640 1 -- 172.21.15.86:0/3626913898 >> v1:172.21.15.86:6789/0 conn(0x7f02900cd640 legacy=0x7f02900cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:02.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.669+0000 7f028e000640 1 --2- 172.21.15.86:0/3626913898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02740039b0 0x7f0274003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:02.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.669+0000 7f028e000640 1 --2- 172.21.15.86:0/3626913898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0274004800 0x7f0274004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:02.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.669+0000 7f028e000640 1 --2- 172.21.15.86:0/3626913898 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02740051c0 0x7f027400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:02.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.669+0000 7f028e000640 1 -- 172.21.15.86:0/3626913898 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f029009b330 con 0x7f02740039b0 2024-04-20T21:19:02.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.669+0000 7f028e000640 1 -- 172.21.15.86:0/3626913898 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f027400c060 con 0x7f0274004800 2024-04-20T21:19:02.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.669+0000 7f028e000640 1 -- 172.21.15.86:0/3626913898 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f027400c260 con 0x7f02740051c0 2024-04-20T21:19:02.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.669+0000 7f028ea00640 1 --2- 172.21.15.86:0/3626913898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02740039b0 0x7f0274003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:02.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.669+0000 7f0286000640 1 --2- 172.21.15.86:0/3626913898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0274004800 0x7f0274004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:02.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.670+0000 7f028f400640 1 --2- 172.21.15.86:0/3626913898 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02740051c0 0x7f027400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:02.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.670+0000 7f028ea00640 1 -- 172.21.15.86:0/3626913898 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02740051c0 msgr2=0x7f027400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:02.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.670+0000 7f028ea00640 1 --2- 172.21.15.86:0/3626913898 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02740051c0 0x7f027400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:02.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.670+0000 7f028ea00640 1 -- 172.21.15.86:0/3626913898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0274004800 msgr2=0x7f0274004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:02.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.670+0000 7f028ea00640 1 --2- 172.21.15.86:0/3626913898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0274004800 0x7f0274004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:02.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.670+0000 7f028ea00640 1 -- 172.21.15.86:0/3626913898 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0274003520 con 0x7f02740039b0 2024-04-20T21:19:02.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.670+0000 7f028f400640 1 -- 172.21.15.86:0/3626913898 reap_dead start 2024-04-20T21:19:02.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.671+0000 7f028ea00640 1 --2- 172.21.15.86:0/3626913898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02740039b0 0x7f0274003dd0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0280001e50 tx=0x7f0274005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db8c4ecbc2e49ca9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:02.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.671+0000 7f028e000640 1 -- 172.21.15.86:0/3626913898 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0280001a70 con 0x7f02740039b0 2024-04-20T21:19:02.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.671+0000 7f028e000640 1 -- 172.21.15.86:0/3626913898 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0280003b70 con 0x7f02740039b0 2024-04-20T21:19:02.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.671+0000 7f028e000640 1 -- 172.21.15.86:0/3626913898 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02800050d0 con 0x7f02740039b0 2024-04-20T21:19:02.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.672+0000 7f0295c00640 1 -- 172.21.15.86:0/3626913898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02740039b0 msgr2=0x7f0274003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:02.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.672+0000 7f0295c00640 1 --2- 172.21.15.86:0/3626913898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02740039b0 0x7f0274003dd0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0280001e50 tx=0x7f0274005830 comp rx=0 tx=0).stop 2024-04-20T21:19:02.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.672+0000 7f0295c00640 1 -- 172.21.15.86:0/3626913898 shutdown_connections 2024-04-20T21:19:02.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.672+0000 7f0295c00640 1 --2- 172.21.15.86:0/3626913898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02740039b0 0x7f0274003dd0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:02.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.672+0000 7f0295c00640 1 -- 172.21.15.86:0/3626913898 >> 172.21.15.86:0/3626913898 conn(0x7f02900d2550 msgr2=0x7f02900d2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:02.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.672+0000 7f0295c00640 1 -- 172.21.15.86:0/3626913898 shutdown_connections 2024-04-20T21:19:02.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.672+0000 7f0295c00640 1 -- 172.21.15.86:0/3626913898 wait complete. 2024-04-20T21:19:02.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.674+0000 7f0295c00640 1 Processor -- start 2024-04-20T21:19:02.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.674+0000 7f0295c00640 1 -- start start 2024-04-20T21:19:02.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.675+0000 7f0295c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02740039b0 0x7f029015ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:02.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.675+0000 7f0295c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02900cbdd0 0x7f02900c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:02.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.675+0000 7f0295c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02900cc250 0x7f02900c8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:02.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.675+0000 7f0295c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0290071290 con 0x7f02900cc250 2024-04-20T21:19:02.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.675+0000 7f0295c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f02900d22c0 con 0x7f02900cbdd0 2024-04-20T21:19:02.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.675+0000 7f0295c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f029009a9e0 con 0x7f02740039b0 2024-04-20T21:19:02.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.675+0000 7f028f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02900cc250 0x7f02900c8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:02.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.675+0000 7f028f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02900cc250 0x7f02900c8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55778/0 (socket says 172.21.15.86:55778) 2024-04-20T21:19:02.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.675+0000 7f028f400640 1 -- 172.21.15.86:0/3016505888 learned_addr learned my addr 172.21.15.86:0/3016505888 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:02.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.675+0000 7f028f400640 1 -- 172.21.15.86:0/3016505888 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02740039b0 msgr2=0x7f029015ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:02.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.675+0000 7f028f400640 1 --2- 172.21.15.86:0/3016505888 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02740039b0 0x7f029015ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:02.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.675+0000 7f028f400640 1 -- 172.21.15.86:0/3016505888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02900cbdd0 msgr2=0x7f02900c81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:02.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.675+0000 7f028f400640 1 --2- 172.21.15.86:0/3016505888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02900cbdd0 0x7f02900c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:02.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.675+0000 7f028f400640 1 -- 172.21.15.86:0/3016505888 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02900c97a0 con 0x7f02900cc250 2024-04-20T21:19:02.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.676+0000 7f028f400640 1 --2- 172.21.15.86:0/3016505888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02900cc250 0x7f02900c8720 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f02900719b0 tx=0x7f028800d950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:02.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.676+0000 7f0287400640 1 -- 172.21.15.86:0/3016505888 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0288007440 con 0x7f02900cc250 2024-04-20T21:19:02.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.676+0000 7f0287400640 1 -- 172.21.15.86:0/3016505888 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0288007600 con 0x7f02900cc250 2024-04-20T21:19:02.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.676+0000 7f0295c00640 1 -- 172.21.15.86:0/3016505888 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02900b7090 con 0x7f02900cc250 2024-04-20T21:19:02.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.676+0000 7f0287400640 1 -- 172.21.15.86:0/3016505888 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0288004a20 con 0x7f02900cc250 2024-04-20T21:19:02.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.676+0000 7f0295c00640 1 -- 172.21.15.86:0/3016505888 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02900c6080 con 0x7f02900cc250 2024-04-20T21:19:02.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.677+0000 7f0295c00640 1 -- 172.21.15.86:0/3016505888 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02900690f0 con 0x7f02900cc250 2024-04-20T21:19:02.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.678+0000 7f0287400640 1 -- 172.21.15.86:0/3016505888 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0288004bc0 con 0x7f02900cc250 2024-04-20T21:19:02.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.679+0000 7f0287400640 1 --2- 172.21.15.86:0/3016505888 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f026c06c8f0 0x7f026c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:02.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.679+0000 7f0287400640 1 -- 172.21.15.86:0/3016505888 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f028808b410 con 0x7f02900cc250 2024-04-20T21:19:02.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.679+0000 7f028ea00640 1 --2- 172.21.15.86:0/3016505888 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f026c06c8f0 0x7f026c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:02.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.680+0000 7f028ea00640 1 --2- 172.21.15.86:0/3016505888 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f026c06c8f0 0x7f026c06edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0280034000 tx=0x7f0280030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:02.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.684+0000 7f0287400640 1 -- 172.21.15.86:0/3016505888 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f028801a060 con 0x7f02900cc250 2024-04-20T21:19:02.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.957+0000 7f0295c00640 1 -- 172.21.15.86:0/3016505888 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f02900ced40 con 0x7f02900cc250 2024-04-20T21:19:02.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.959+0000 7f0287400640 1 -- 172.21.15.86:0/3016505888 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) ==== 80+0+6645 (secure 0 0 0) 0x7f0288059af0 con 0x7f02900cc250 2024-04-20T21:19:02.958 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:19:02.958 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:19:02.958 INFO:teuthology.orchestra.run.smithi086.stdout: "auth_dump": [ 2024-04-20T21:19:02.958 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:19:02.958 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.a", 2024-04-20T21:19:02.959 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDAMCRmOpS/LRAAPZUUQNf6TyeJ2tF7YPnIKw==", 2024-04-20T21:19:02.959 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:19:02.959 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:19:02.959 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-20T21:19:02.959 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-20T21:19:02.959 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:19:02.959 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:19:02.959 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:19:02.959 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:19:02.959 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.b", 2024-04-20T21:19:02.959 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDBMCRmCJLpDRAA/JiLRSAlQRNMpDC+qDSr0g==", 2024-04-20T21:19:02.959 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:19:02.959 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:19:02.959 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-20T21:19:02.960 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-20T21:19:02.960 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:19:02.960 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:19:02.960 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:19:02.960 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:19:02.960 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.c", 2024-04-20T21:19:02.960 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDAMCRmPHOFOxAA4tUsw/n6oSKrNmzPD+NKyw==", 2024-04-20T21:19:02.960 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:19:02.960 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:19:02.960 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-20T21:19:02.960 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-20T21:19:02.960 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:19:02.961 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:19:02.961 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:19:02.961 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:19:02.961 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.d", 2024-04-20T21:19:02.961 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDBMCRmZnPXHBAA0jXwBmGehaO7qens4boRIQ==", 2024-04-20T21:19:02.961 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:19:02.961 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:19:02.961 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-20T21:19:02.961 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-20T21:19:02.961 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:19:02.961 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:19:02.961 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:19:02.961 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:19:02.961 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.0", 2024-04-20T21:19:02.961 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDFMCRm4w9UHhAA56y4dJyDASI+IoZv2XrEoQ==", 2024-04-20T21:19:02.962 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:19:02.962 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:19:02.962 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:19:02.962 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:19:02.962 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:19:02.962 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:19:02.962 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:19:02.962 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.1", 2024-04-20T21:19:02.962 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDIMCRmFo7xKRAAou5B04+J1gZtTzW/Mo+nRQ==", 2024-04-20T21:19:02.962 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:19:02.962 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:19:02.962 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:19:02.962 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:19:02.962 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:19:02.963 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:19:02.963 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:19:02.963 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.2", 2024-04-20T21:19:02.963 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDLMCRm7/jFNRAA4KcRsapPFLaKf0NaH5MApg==", 2024-04-20T21:19:02.963 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:19:02.963 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:19:02.963 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:19:02.963 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:19:02.963 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:19:02.963 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:19:02.963 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:19:02.963 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.3", 2024-04-20T21:19:02.963 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDPMCRmUTPMExAA0/dubb4ehsWIwZPS/ao68w==", 2024-04-20T21:19:02.963 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:19:02.963 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:19:02.963 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:19:02.964 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:19:02.964 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:19:02.964 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:19:02.964 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:19:02.964 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.4", 2024-04-20T21:19:02.964 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDVMCRmB4F2KRAA8Xj0DfH5Ou8TUrO8X/BQyw==", 2024-04-20T21:19:02.964 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:19:02.964 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:19:02.964 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:19:02.964 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:19:02.964 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:19:02.964 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:19:02.964 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:19:02.964 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.5", 2024-04-20T21:19:02.964 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDYMCRmSOvYNRAAODYXmRHMGPrCAxcHwlS7Ww==", 2024-04-20T21:19:02.964 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:19:02.965 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:19:02.965 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:19:02.965 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:19:02.965 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:19:02.965 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:19:02.965 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:19:02.965 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.6", 2024-04-20T21:19:02.965 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDcMCRmENJ9BBAAi/KBUhbYGNXEyiSjSHavWQ==", 2024-04-20T21:19:02.965 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:19:02.965 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:19:02.965 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:19:02.965 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:19:02.965 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:19:02.965 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:19:02.965 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:19:02.965 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.7", 2024-04-20T21:19:02.966 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDfMCRmfTHTDhAALL9GZBSGjWYrhgFw/YI6IA==", 2024-04-20T21:19:02.966 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:19:02.966 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:19:02.966 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:19:02.966 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:19:02.966 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:19:02.966 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:19:02.966 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:19:02.966 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.0", 2024-04-20T21:19:02.966 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDBMCRmHWxyKRAANzhRi4eajp68h/jWkhFMnw==", 2024-04-20T21:19:02.966 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:19:02.966 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:19:02.967 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow r", 2024-04-20T21:19:02.967 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow rw", 2024-04-20T21:19:02.967 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rwx" 2024-04-20T21:19:02.967 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:19:02.967 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:19:02.967 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:19:02.967 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.1", 2024-04-20T21:19:02.967 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDBMCRmgBJAOxAADJLzO97tjEpKn6kUs8g2QA==", 2024-04-20T21:19:02.967 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:19:02.967 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:19:02.967 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow r", 2024-04-20T21:19:02.967 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow rw", 2024-04-20T21:19:02.967 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rwx" 2024-04-20T21:19:02.967 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:19:02.967 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:19:02.967 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:19:02.968 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.2", 2024-04-20T21:19:02.968 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDBMCRmQAdJMhAAiUmKhPEeuV+UPqQ3gynG9w==", 2024-04-20T21:19:02.968 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:19:02.968 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:19:02.968 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow r", 2024-04-20T21:19:02.968 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow rw", 2024-04-20T21:19:02.968 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rwx" 2024-04-20T21:19:02.968 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:19:02.968 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:19:02.968 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:19:02.968 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.3", 2024-04-20T21:19:02.968 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDCMCRm8yMSCBAAFgSVW0XYu7fa40YxHITljQ==", 2024-04-20T21:19:02.968 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:19:02.968 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:19:02.968 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow r", 2024-04-20T21:19:02.968 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow rw", 2024-04-20T21:19:02.969 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rwx" 2024-04-20T21:19:02.969 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:19:02.969 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:19:02.969 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:19:02.969 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.admin", 2024-04-20T21:19:02.969 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQC/MCRmyru0KBAApSqv1hZRcLvrRwPMnqylqQ==", 2024-04-20T21:19:02.969 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:19:02.969 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-04-20T21:19:02.969 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-20T21:19:02.969 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-20T21:19:02.969 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:19:02.969 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:19:02.969 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:19:02.970 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:19:02.970 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.bootstrap-mds", 2024-04-20T21:19:02.970 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDnMCRmMkvFHRAAyaxySF++p7SmMkkaOx4trA==", 2024-04-20T21:19:02.970 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:19:02.970 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile bootstrap-mds" 2024-04-20T21:19:02.970 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:19:02.970 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:19:02.970 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:19:02.970 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.bootstrap-mgr", 2024-04-20T21:19:02.970 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDnMCRmFlfFHRAAEo13qWS2oc5oyRm0dQb3sg==", 2024-04-20T21:19:02.970 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:19:02.970 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-20T21:19:02.970 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:19:02.970 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:19:02.971 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:19:02.971 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.bootstrap-osd", 2024-04-20T21:19:02.971 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDnMCRmxl/FHRAA5Qn0mCl3dk7wWbWQESURcw==", 2024-04-20T21:19:02.971 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:19:02.971 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile bootstrap-osd" 2024-04-20T21:19:02.971 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:19:02.971 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:19:02.971 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:19:02.971 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.bootstrap-rbd", 2024-04-20T21:19:02.971 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDnMCRmOWjFHRAATHKhx5gzpj3yFqThM+Ch9w==", 2024-04-20T21:19:02.971 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:19:02.971 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-20T21:19:02.971 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:19:02.971 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:19:02.972 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:19:02.972 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-20T21:19:02.972 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDnMCRm23DFHRAAy7SqLSu7qcgaPX/BmZujLw==", 2024-04-20T21:19:02.972 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:19:02.972 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-20T21:19:02.972 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:19:02.972 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:19:02.972 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:19:02.972 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.bootstrap-rgw", 2024-04-20T21:19:02.972 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDnMCRm13nFHRAAw6T39xpRVQWSiou+Z2srWQ==", 2024-04-20T21:19:02.972 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:19:02.972 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-20T21:19:02.972 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:19:02.972 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:19:02.972 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:19:02.973 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mgr.x", 2024-04-20T21:19:02.973 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDAMCRmvTgjIxAACeSV8gNkL5LV3xzPS6VPeg==", 2024-04-20T21:19:02.973 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:19:02.973 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-04-20T21:19:02.973 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile mgr", 2024-04-20T21:19:02.973 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:19:02.973 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:19:02.973 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:19:02.973 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:19:02.973 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mgr.y", 2024-04-20T21:19:02.973 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDAMCRmSn3VGBAA0eu3R53f3gFgscWFMRkDzg==", 2024-04-20T21:19:02.973 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:19:02.973 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-04-20T21:19:02.974 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile mgr", 2024-04-20T21:19:02.974 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:19:02.974 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:19:02.974 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:19:02.974 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-20T21:19:02.974 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:19:02.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.961+0000 7f0295c00640 1 -- 172.21.15.86:0/3016505888 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f026c06c8f0 msgr2=0x7f026c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:02.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.961+0000 7f0295c00640 1 --2- 172.21.15.86:0/3016505888 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f026c06c8f0 0x7f026c06edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0280034000 tx=0x7f0280030000 comp rx=0 tx=0).stop 2024-04-20T21:19:02.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.961+0000 7f0295c00640 1 -- 172.21.15.86:0/3016505888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02900cc250 msgr2=0x7f02900c8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:02.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.961+0000 7f0295c00640 1 --2- 172.21.15.86:0/3016505888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02900cc250 0x7f02900c8720 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f02900719b0 tx=0x7f028800d950 comp rx=0 tx=0).stop 2024-04-20T21:19:02.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.962+0000 7f0295c00640 1 -- 172.21.15.86:0/3016505888 shutdown_connections 2024-04-20T21:19:02.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.962+0000 7f0295c00640 1 --2- 172.21.15.86:0/3016505888 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f026c06c8f0 0x7f026c06edb0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:02.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.962+0000 7f0295c00640 1 --2- 172.21.15.86:0/3016505888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02900cc250 0x7f02900c8720 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:02.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.962+0000 7f0295c00640 1 --2- 172.21.15.86:0/3016505888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02900cbdd0 0x7f02900c81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:02.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.962+0000 7f0295c00640 1 --2- 172.21.15.86:0/3016505888 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02740039b0 0x7f029015ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:02.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.962+0000 7f0295c00640 1 -- 172.21.15.86:0/3016505888 >> 172.21.15.86:0/3016505888 conn(0x7f02900d2550 msgr2=0x7f029015c880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:02.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.962+0000 7f0295c00640 1 -- 172.21.15.86:0/3016505888 shutdown_connections 2024-04-20T21:19:02.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:02.962+0000 7f0295c00640 1 -- 172.21.15.86:0/3016505888 wait complete. 2024-04-20T21:19:02.979 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-20T21:19:03.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.187+0000 7fac8f600640 1 Processor -- start 2024-04-20T21:19:03.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.187+0000 7fac8f600640 1 -- start start 2024-04-20T21:19:03.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.189+0000 7fac8f600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac88151db0 0x7fac880b52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:03.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.189+0000 7fac8f600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fac8806f560 0x7fac880725d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:03.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.189+0000 7fac8f600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac880b3620 con 0x7fac880cea30 2024-04-20T21:19:03.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.189+0000 7fac8f600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fac880b4230 con 0x7fac8806f560 2024-04-20T21:19:03.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.189+0000 7fac8f600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fac880b3900 con 0x7fac88151db0 2024-04-20T21:19:03.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.189+0000 7fac7fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fac8806f560 0x7fac880725d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:03.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.190+0000 7fac7fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fac8806f560 0x7fac880725d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55780/0 (socket says 172.21.15.86:55780) 2024-04-20T21:19:03.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.190+0000 7fac7fe00640 1 -- 172.21.15.86:0/442740402 learned_addr learned my addr 172.21.15.86:0/442740402 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:03.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.190+0000 7fac8cc00640 1 --2- 172.21.15.86:0/442740402 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac88151db0 0x7fac880b52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:03.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.190+0000 7fac7fe00640 1 -- 172.21.15.86:0/442740402 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac88151db0 msgr2=0x7fac880b52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:03.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.190+0000 7fac7fe00640 1 --2- 172.21.15.86:0/442740402 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac88151db0 0x7fac880b52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:03.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.190+0000 7fac7fe00640 1 -- 172.21.15.86:0/442740402 >> v1:172.21.15.144:6789/0 conn(0x7fac880cea30 legacy=0x7fac8815e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:03.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.190+0000 7fac7fe00640 1 -- 172.21.15.86:0/442740402 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac880b3be0 con 0x7fac8806f560 2024-04-20T21:19:03.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.191+0000 7fac7fe00640 1 --2- 172.21.15.86:0/442740402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac8806f560 0x7fac880725d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fac7800bb10 tx=0x7fac7802e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34345e39e35ba39d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:03.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.191+0000 7fac87e00640 1 -- 172.21.15.86:0/442740402 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7803b070 con 0x7fac8806f560 2024-04-20T21:19:03.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.192+0000 7fac87e00640 1 -- 172.21.15.86:0/442740402 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac78036440 con 0x7fac8806f560 2024-04-20T21:19:03.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.192+0000 7fac87e00640 1 -- 172.21.15.86:0/442740402 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac78036810 con 0x7fac8806f560 2024-04-20T21:19:03.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.192+0000 7fac8f600640 1 -- 172.21.15.86:0/442740402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac8806f560 msgr2=0x7fac880725d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:03.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.192+0000 7fac8f600640 1 --2- 172.21.15.86:0/442740402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac8806f560 0x7fac880725d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fac7800bb10 tx=0x7fac7802e0c0 comp rx=0 tx=0).stop 2024-04-20T21:19:03.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.192+0000 7fac8f600640 1 -- 172.21.15.86:0/442740402 shutdown_connections 2024-04-20T21:19:03.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.192+0000 7fac8f600640 1 --2- 172.21.15.86:0/442740402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac8806f560 0x7fac880725d0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:03.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.192+0000 7fac8f600640 1 --2- 172.21.15.86:0/442740402 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac88151db0 0x7fac880b52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:03.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.192+0000 7fac8f600640 1 -- 172.21.15.86:0/442740402 >> 172.21.15.86:0/442740402 conn(0x7fac8806b980 msgr2=0x7fac8806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:03.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.192+0000 7fac8f600640 1 -- 172.21.15.86:0/442740402 shutdown_connections 2024-04-20T21:19:03.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.192+0000 7fac8f600640 1 -- 172.21.15.86:0/442740402 wait complete. 2024-04-20T21:19:03.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.194+0000 7fac8f600640 1 Processor -- start 2024-04-20T21:19:03.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.194+0000 7fac8f600640 1 -- start start 2024-04-20T21:19:03.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.195+0000 7fac8f600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac8806f560 0x7fac88152190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:03.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.195+0000 7fac8f600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac880cea30 0x7fac881526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:03.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.195+0000 7fac8f600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac88151db0 0x7fac881540f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:03.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.195+0000 7fac8f600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fac880b57e0 con 0x7fac88151db0 2024-04-20T21:19:03.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.195+0000 7fac8f600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fac88072b10 con 0x7fac8806f560 2024-04-20T21:19:03.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.195+0000 7fac8f600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fac88159250 con 0x7fac880cea30 2024-04-20T21:19:03.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.195+0000 7fac8d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac88151db0 0x7fac881540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:03.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.196+0000 7fac8d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac88151db0 0x7fac881540f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55786/0 (socket says 172.21.15.86:55786) 2024-04-20T21:19:03.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.196+0000 7fac8d600640 1 -- 172.21.15.86:0/1172344210 learned_addr learned my addr 172.21.15.86:0/1172344210 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:03.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.196+0000 7fac8cc00640 1 --2- 172.21.15.86:0/1172344210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac8806f560 0x7fac88152190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:03.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.196+0000 7fac7fe00640 1 --2- 172.21.15.86:0/1172344210 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac880cea30 0x7fac881526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:03.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.196+0000 7fac8d600640 1 -- 172.21.15.86:0/1172344210 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac880cea30 msgr2=0x7fac881526d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:03.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.196+0000 7fac8d600640 1 --2- 172.21.15.86:0/1172344210 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac880cea30 0x7fac881526d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:03.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.196+0000 7fac8d600640 1 -- 172.21.15.86:0/1172344210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac8806f560 msgr2=0x7fac88152190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:03.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.196+0000 7fac8d600640 1 --2- 172.21.15.86:0/1172344210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac8806f560 0x7fac88152190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:03.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.196+0000 7fac8d600640 1 -- 172.21.15.86:0/1172344210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac88152d00 con 0x7fac88151db0 2024-04-20T21:19:03.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.196+0000 7fac7fe00640 1 --2- 172.21.15.86:0/1172344210 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac880cea30 0x7fac881526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:03.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.196+0000 7fac8d600640 1 --2- 172.21.15.86:0/1172344210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac88151db0 0x7fac881540f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fac8000c7b0 tx=0x7fac8000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:03.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.197+0000 7fac85600640 1 -- 172.21.15.86:0/1172344210 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac80004b60 con 0x7fac88151db0 2024-04-20T21:19:03.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.197+0000 7fac85600640 1 -- 172.21.15.86:0/1172344210 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac80004d00 con 0x7fac88151db0 2024-04-20T21:19:03.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.197+0000 7fac85600640 1 -- 172.21.15.86:0/1172344210 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac8000aa90 con 0x7fac88151db0 2024-04-20T21:19:03.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.197+0000 7fac8f600640 1 -- 172.21.15.86:0/1172344210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac88152fe0 con 0x7fac88151db0 2024-04-20T21:19:03.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.197+0000 7fac8f600640 1 -- 172.21.15.86:0/1172344210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac881535a0 con 0x7fac88151db0 2024-04-20T21:19:03.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.198+0000 7fac8f600640 1 -- 172.21.15.86:0/1172344210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac88155640 con 0x7fac88151db0 2024-04-20T21:19:03.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.199+0000 7fac85600640 1 -- 172.21.15.86:0/1172344210 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fac8000ac30 con 0x7fac88151db0 2024-04-20T21:19:03.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.200+0000 7fac85600640 1 --2- 172.21.15.86:0/1172344210 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fac6406c8f0 0x7fac6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:03.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.200+0000 7fac85600640 1 -- 172.21.15.86:0/1172344210 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fac8008bb90 con 0x7fac88151db0 2024-04-20T21:19:03.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.200+0000 7fac8cc00640 1 --2- 172.21.15.86:0/1172344210 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fac6406c8f0 0x7fac6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:03.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.201+0000 7fac8cc00640 1 --2- 172.21.15.86:0/1172344210 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fac6406c8f0 0x7fac6406edb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fac70006fd0 tx=0x7fac70008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:03.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.205+0000 7fac85600640 1 -- 172.21.15.86:0/1172344210 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fac80020070 con 0x7fac88151db0 2024-04-20T21:19:03.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.470+0000 7fac8f600640 1 -- 172.21.15.86:0/1172344210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7fac88072ee0 con 0x7fac88151db0 2024-04-20T21:19:03.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.475+0000 7fac85600640 1 -- 172.21.15.86:0/1172344210 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7fac8005a270 con 0x7fac88151db0 2024-04-20T21:19:03.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.477+0000 7fac8f600640 1 -- 172.21.15.86:0/1172344210 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fac6406c8f0 msgr2=0x7fac6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:03.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.477+0000 7fac8f600640 1 --2- 172.21.15.86:0/1172344210 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fac6406c8f0 0x7fac6406edb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fac70006fd0 tx=0x7fac70008040 comp rx=0 tx=0).stop 2024-04-20T21:19:03.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.477+0000 7fac8f600640 1 -- 172.21.15.86:0/1172344210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac88151db0 msgr2=0x7fac881540f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:03.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.477+0000 7fac8f600640 1 --2- 172.21.15.86:0/1172344210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac88151db0 0x7fac881540f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fac8000c7b0 tx=0x7fac8000cc70 comp rx=0 tx=0).stop 2024-04-20T21:19:03.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.478+0000 7fac8f600640 1 -- 172.21.15.86:0/1172344210 shutdown_connections 2024-04-20T21:19:03.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.478+0000 7fac8f600640 1 --2- 172.21.15.86:0/1172344210 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fac6406c8f0 0x7fac6406edb0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:03.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.478+0000 7fac8f600640 1 --2- 172.21.15.86:0/1172344210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac88151db0 0x7fac881540f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:03.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.478+0000 7fac8f600640 1 --2- 172.21.15.86:0/1172344210 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac880cea30 0x7fac881526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:03.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.478+0000 7fac8f600640 1 --2- 172.21.15.86:0/1172344210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac8806f560 0x7fac88152190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:03.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.478+0000 7fac8f600640 1 -- 172.21.15.86:0/1172344210 >> 172.21.15.86:0/1172344210 conn(0x7fac8806b980 msgr2=0x7fac88071d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:03.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.478+0000 7fac8f600640 1 -- 172.21.15.86:0/1172344210 shutdown_connections 2024-04-20T21:19:03.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.478+0000 7fac8f600640 1 -- 172.21.15.86:0/1172344210 wait complete. 2024-04-20T21:19:03.495 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-20T21:19:03.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.685+0000 7f61cfe00640 1 Processor -- start 2024-04-20T21:19:03.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.685+0000 7f61cfe00640 1 -- start start 2024-04-20T21:19:03.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.687+0000 7f61cfe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61d015e870 0x7f61d00b3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:03.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.687+0000 7f61cfe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f61d00cf550 0x7f61d00cf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:03.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.687+0000 7f61cfe00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61d00b4b50 con 0x7f61d00b45b0 2024-04-20T21:19:03.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.687+0000 7f61cfe00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f61d00b5c90 con 0x7f61d00cf550 2024-04-20T21:19:03.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.687+0000 7f61cfe00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f61d00cfef0 con 0x7f61d015e870 2024-04-20T21:19:03.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.687+0000 7f61cf400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f61d00cf550 0x7f61d00cf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:03.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.687+0000 7f61ce000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f61d00b45b0 0x7f61d00d26a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44418/0 (socket says 172.21.15.86:44418) 2024-04-20T21:19:03.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.687+0000 7f61ce000640 1 -- 172.21.15.86:0/1199131444 learned_addr learned my addr 172.21.15.86:0/1199131444 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:19:03.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.688+0000 7f61cea00640 1 --2- 172.21.15.86:0/1199131444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61d015e870 0x7f61d00b3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:03.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.688+0000 7f61cf400640 1 -- 172.21.15.86:0/1199131444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61d015e870 msgr2=0x7f61d00b3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:03.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.688+0000 7f61cf400640 1 --2- 172.21.15.86:0/1199131444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61d015e870 0x7f61d00b3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:03.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.688+0000 7f61cf400640 1 -- 172.21.15.86:0/1199131444 >> v1:172.21.15.86:6789/0 conn(0x7f61d00b45b0 legacy=0x7f61d00d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:03.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.688+0000 7f61cf400640 1 -- 172.21.15.86:0/1199131444 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61d00b4e30 con 0x7f61d00cf550 2024-04-20T21:19:03.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.688+0000 7f61cea00640 1 --2- 172.21.15.86:0/1199131444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61d015e870 0x7f61d00b3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:03.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.688+0000 7f61cd600640 1 -- 172.21.15.86:0/1199131444 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2018581098 0 0) 0x7f61d00b4b50 con 0x7f61d00b45b0 2024-04-20T21:19:03.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.689+0000 7f61cf400640 1 --2- 172.21.15.86:0/1199131444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61d00cf550 0x7f61d00cf9b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f61c80097a0 tx=0x7f61c802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5832c6ad7a9036e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:03.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.689+0000 7f61cd600640 1 -- 172.21.15.86:0/1199131444 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c8035020 con 0x7f61d00cf550 2024-04-20T21:19:03.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.689+0000 7f61cd600640 1 -- 172.21.15.86:0/1199131444 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61c8031810 con 0x7f61d00cf550 2024-04-20T21:19:03.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.689+0000 7f61cd600640 1 -- 172.21.15.86:0/1199131444 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c8031be0 con 0x7f61d00cf550 2024-04-20T21:19:03.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.690+0000 7f61cfe00640 1 -- 172.21.15.86:0/1199131444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61d00cf550 msgr2=0x7f61d00cf9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:03.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.690+0000 7f61cfe00640 1 --2- 172.21.15.86:0/1199131444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61d00cf550 0x7f61d00cf9b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f61c80097a0 tx=0x7f61c802b8b0 comp rx=0 tx=0).stop 2024-04-20T21:19:03.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.690+0000 7f61cfe00640 1 -- 172.21.15.86:0/1199131444 shutdown_connections 2024-04-20T21:19:03.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.690+0000 7f61cfe00640 1 --2- 172.21.15.86:0/1199131444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61d00cf550 0x7f61d00cf9b0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:03.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.690+0000 7f61cfe00640 1 --2- 172.21.15.86:0/1199131444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61d015e870 0x7f61d00b3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:03.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.690+0000 7f61cfe00640 1 -- 172.21.15.86:0/1199131444 >> 172.21.15.86:0/1199131444 conn(0x7f61d009d880 msgr2=0x7f61d008bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:03.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.690+0000 7f61cfe00640 1 -- 172.21.15.86:0/1199131444 shutdown_connections 2024-04-20T21:19:03.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.690+0000 7f61cfe00640 1 -- 172.21.15.86:0/1199131444 wait complete. 2024-04-20T21:19:03.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.692+0000 7f61cfe00640 1 Processor -- start 2024-04-20T21:19:03.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.692+0000 7f61cfe00640 1 -- start start 2024-04-20T21:19:03.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.693+0000 7f61cfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61d00b45b0 0x7f61d0154920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:03.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.693+0000 7f61cfe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61d00cf550 0x7f61d00ca430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:03.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.693+0000 7f61cfe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61d015e870 0x7f61d00ca970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:03.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.693+0000 7f61cfe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f61d015ec50 con 0x7f61d00b45b0 2024-04-20T21:19:03.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.693+0000 7f61cfe00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f61d00b53f0 con 0x7f61d00cf550 2024-04-20T21:19:03.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.693+0000 7f61cfe00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f61d00b6120 con 0x7f61d015e870 2024-04-20T21:19:03.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.693+0000 7f61cea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61d00b45b0 0x7f61d0154920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:03.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.693+0000 7f61cea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61d00b45b0 0x7f61d0154920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55800/0 (socket says 172.21.15.86:55800) 2024-04-20T21:19:03.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.693+0000 7f61cea00640 1 -- 172.21.15.86:0/3310909166 learned_addr learned my addr 172.21.15.86:0/3310909166 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:03.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.693+0000 7f61cea00640 1 -- 172.21.15.86:0/3310909166 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61d015e870 msgr2=0x7f61d00ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:03.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.693+0000 7f61cea00640 1 --2- 172.21.15.86:0/3310909166 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61d015e870 0x7f61d00ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:03.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.693+0000 7f61cea00640 1 -- 172.21.15.86:0/3310909166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61d00cf550 msgr2=0x7f61d00ca430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:03.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.693+0000 7f61cea00640 1 --2- 172.21.15.86:0/3310909166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61d00cf550 0x7f61d00ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:03.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.693+0000 7f61cea00640 1 -- 172.21.15.86:0/3310909166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61d00b9590 con 0x7f61d00b45b0 2024-04-20T21:19:03.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.694+0000 7f61cea00640 1 --2- 172.21.15.86:0/3310909166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61d00b45b0 0x7f61d0154920 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f61c000c7f0 tx=0x7f61c000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:03.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.694+0000 7f61c6a00640 1 -- 172.21.15.86:0/3310909166 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c0015070 con 0x7f61d00b45b0 2024-04-20T21:19:03.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.694+0000 7f61c6a00640 1 -- 172.21.15.86:0/3310909166 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61c0007820 con 0x7f61d00b45b0 2024-04-20T21:19:03.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.694+0000 7f61cfe00640 1 -- 172.21.15.86:0/3310909166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61d00b9870 con 0x7f61d00b45b0 2024-04-20T21:19:03.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.694+0000 7f61c6a00640 1 -- 172.21.15.86:0/3310909166 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c0007bf0 con 0x7f61d00b45b0 2024-04-20T21:19:03.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.694+0000 7f61cfe00640 1 -- 172.21.15.86:0/3310909166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61d00b9e30 con 0x7f61d00b45b0 2024-04-20T21:19:03.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.695+0000 7f61cfe00640 1 -- 172.21.15.86:0/3310909166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61d004fcf0 con 0x7f61d00b45b0 2024-04-20T21:19:03.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.696+0000 7f61c6a00640 1 -- 172.21.15.86:0/3310909166 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f61c0018070 con 0x7f61d00b45b0 2024-04-20T21:19:03.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.697+0000 7f61c6a00640 1 --2- 172.21.15.86:0/3310909166 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f61ac06c8f0 0x7f61ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:03.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.697+0000 7f61c6a00640 1 -- 172.21.15.86:0/3310909166 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f61c008b9e0 con 0x7f61d00b45b0 2024-04-20T21:19:03.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.697+0000 7f61ce000640 1 --2- 172.21.15.86:0/3310909166 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f61ac06c8f0 0x7f61ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:03.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.698+0000 7f61ce000640 1 --2- 172.21.15.86:0/3310909166 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f61ac06c8f0 0x7f61ac06edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f61b8001560 tx=0x7f61b8007ca0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:03.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.701+0000 7f61c6a00640 1 -- 172.21.15.86:0/3310909166 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f61c005a0c0 con 0x7f61d00b45b0 2024-04-20T21:19:03.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.971+0000 7f61cfe00640 1 -- 172.21.15.86:0/3310909166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f61d00cf330 con 0x7f61d00b45b0 2024-04-20T21:19:03.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.976+0000 7f61c6a00640 1 -- 172.21.15.86:0/3310909166 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7f61c00599c0 con 0x7f61d00b45b0 2024-04-20T21:19:03.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.978+0000 7f61cfe00640 1 -- 172.21.15.86:0/3310909166 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f61ac06c8f0 msgr2=0x7f61ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:03.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.978+0000 7f61cfe00640 1 --2- 172.21.15.86:0/3310909166 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f61ac06c8f0 0x7f61ac06edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f61b8001560 tx=0x7f61b8007ca0 comp rx=0 tx=0).stop 2024-04-20T21:19:03.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.978+0000 7f61cfe00640 1 -- 172.21.15.86:0/3310909166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61d00b45b0 msgr2=0x7f61d0154920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:03.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.978+0000 7f61cfe00640 1 --2- 172.21.15.86:0/3310909166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61d00b45b0 0x7f61d0154920 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f61c000c7f0 tx=0x7f61c000ccb0 comp rx=0 tx=0).stop 2024-04-20T21:19:03.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.978+0000 7f61cfe00640 1 -- 172.21.15.86:0/3310909166 shutdown_connections 2024-04-20T21:19:03.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.978+0000 7f61cfe00640 1 --2- 172.21.15.86:0/3310909166 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f61ac06c8f0 0x7f61ac06edb0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:03.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.978+0000 7f61cfe00640 1 --2- 172.21.15.86:0/3310909166 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61d015e870 0x7f61d00ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:03.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.978+0000 7f61cfe00640 1 --2- 172.21.15.86:0/3310909166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61d00cf550 0x7f61d00ca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:03.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.978+0000 7f61cfe00640 1 --2- 172.21.15.86:0/3310909166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61d00b45b0 0x7f61d0154920 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:03.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.978+0000 7f61cfe00640 1 -- 172.21.15.86:0/3310909166 >> 172.21.15.86:0/3310909166 conn(0x7f61d009d880 msgr2=0x7f61d00b64f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:03.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.979+0000 7f61cfe00640 1 -- 172.21.15.86:0/3310909166 shutdown_connections 2024-04-20T21:19:03.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:03.979+0000 7f61cfe00640 1 -- 172.21.15.86:0/3310909166 wait complete. 2024-04-20T21:19:03.996 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-20T21:19:04.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.192+0000 7f9cd8c00640 1 Processor -- start 2024-04-20T21:19:04.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.192+0000 7f9cd8c00640 1 -- start start 2024-04-20T21:19:04.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.194+0000 7f9cd8c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9cd4071d40 0x7f9cd40cead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:04.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.194+0000 7f9cd8c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9cd406efc0 0x7f9cd4072120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:04.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.194+0000 7f9cd8c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9cd406dda0 0x7f9cd406e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:04.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.194+0000 7f9cd8c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9cd41457e0 con 0x7f9cd406efc0 2024-04-20T21:19:04.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.194+0000 7f9cd8c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f9cd4160e10 con 0x7f9cd4071d40 2024-04-20T21:19:04.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.194+0000 7f9cd8c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9cd4072660 con 0x7f9cd406dda0 2024-04-20T21:19:04.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.194+0000 7f9cd1400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9cd406efc0 0x7f9cd4072120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:04.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.194+0000 7f9cd1400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9cd406efc0 0x7f9cd4072120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55802/0 (socket says 172.21.15.86:55802) 2024-04-20T21:19:04.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.194+0000 7f9cd1400640 1 -- 172.21.15.86:0/898704301 learned_addr learned my addr 172.21.15.86:0/898704301 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:04.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.195+0000 7f9cd1400640 1 -- 172.21.15.86:0/898704301 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9cd406dda0 msgr2=0x7f9cd406e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:04.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.195+0000 7f9cd2800640 1 --2- 172.21.15.86:0/898704301 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9cd406dda0 0x7f9cd406e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:04.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.195+0000 7f9cd1e00640 1 --2- 172.21.15.86:0/898704301 >> v2:172.21.15.144:3300/0 conn(0x7f9cd4071d40 0x7f9cd40cead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:04.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.195+0000 7f9cd1400640 1 --2- 172.21.15.86:0/898704301 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9cd406dda0 0x7f9cd406e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:04.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.195+0000 7f9cd1400640 1 -- 172.21.15.86:0/898704301 >> v2:172.21.15.144:3300/0 conn(0x7f9cd4071d40 msgr2=0x7f9cd40cead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:04.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.195+0000 7f9cd1400640 1 --2- 172.21.15.86:0/898704301 >> v2:172.21.15.144:3300/0 conn(0x7f9cd4071d40 0x7f9cd40cead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:04.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.195+0000 7f9cd1400640 1 -- 172.21.15.86:0/898704301 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cd4160200 con 0x7f9cd406efc0 2024-04-20T21:19:04.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.195+0000 7f9cd1e00640 1 --2- 172.21.15.86:0/898704301 >> v2:172.21.15.144:3300/0 conn(0x7f9cd4071d40 0x7f9cd40cead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:04.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.195+0000 7f9cd2800640 1 --2- 172.21.15.86:0/898704301 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9cd406dda0 0x7f9cd406e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:04.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.196+0000 7f9cd1400640 1 --2- 172.21.15.86:0/898704301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cd406efc0 0x7f9cd4072120 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9cd40b5a00 tx=0x7f9cc002b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f41f03226f955c04 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:04.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.197+0000 7f9cd0a00640 1 -- 172.21.15.86:0/898704301 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc0034070 con 0x7f9cd406efc0 2024-04-20T21:19:04.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.197+0000 7f9cd0a00640 1 -- 172.21.15.86:0/898704301 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cc0030850 con 0x7f9cd406efc0 2024-04-20T21:19:04.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.197+0000 7f9cd0a00640 1 -- 172.21.15.86:0/898704301 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc0030c20 con 0x7f9cd406efc0 2024-04-20T21:19:04.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.197+0000 7f9cd8c00640 1 -- 172.21.15.86:0/898704301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cd406efc0 msgr2=0x7f9cd4072120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:04.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.197+0000 7f9cd8c00640 1 --2- 172.21.15.86:0/898704301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cd406efc0 0x7f9cd4072120 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9cd40b5a00 tx=0x7f9cc002b990 comp rx=0 tx=0).stop 2024-04-20T21:19:04.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.198+0000 7f9cd8c00640 1 -- 172.21.15.86:0/898704301 shutdown_connections 2024-04-20T21:19:04.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.198+0000 7f9cd8c00640 1 --2- 172.21.15.86:0/898704301 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9cd406dda0 0x7f9cd406e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:04.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.198+0000 7f9cd8c00640 1 --2- 172.21.15.86:0/898704301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cd406efc0 0x7f9cd4072120 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:04.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.198+0000 7f9cd8c00640 1 --2- 172.21.15.86:0/898704301 >> v2:172.21.15.144:3300/0 conn(0x7f9cd4071d40 0x7f9cd40cead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:04.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.198+0000 7f9cd8c00640 1 -- 172.21.15.86:0/898704301 >> 172.21.15.86:0/898704301 conn(0x7f9cd406b980 msgr2=0x7f9cd406bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:04.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.198+0000 7f9cd8c00640 1 -- 172.21.15.86:0/898704301 shutdown_connections 2024-04-20T21:19:04.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.198+0000 7f9cd8c00640 1 -- 172.21.15.86:0/898704301 wait complete. 2024-04-20T21:19:04.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.200+0000 7f9cd8c00640 1 Processor -- start 2024-04-20T21:19:04.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.200+0000 7f9cd8c00640 1 -- start start 2024-04-20T21:19:04.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.201+0000 7f9cd8c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cd406dda0 0x7f9cd413c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:04.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.201+0000 7f9cd8c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cd406efc0 0x7f9cd40cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:04.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.201+0000 7f9cd8c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9cd4071d40 0x7f9cd40cc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:04.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.201+0000 7f9cd8c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd4160aa0 con 0x7f9cd406efc0 2024-04-20T21:19:04.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.201+0000 7f9cd8c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd41607c0 con 0x7f9cd406dda0 2024-04-20T21:19:04.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.201+0000 7f9cd8c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9cd406f430 con 0x7f9cd4071d40 2024-04-20T21:19:04.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.201+0000 7f9cd1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cd406efc0 0x7f9cd40cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:04.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.202+0000 7f9cd1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cd406efc0 0x7f9cd40cc4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55818/0 (socket says 172.21.15.86:55818) 2024-04-20T21:19:04.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.202+0000 7f9cd1400640 1 -- 172.21.15.86:0/723041891 learned_addr learned my addr 172.21.15.86:0/723041891 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:04.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.202+0000 7f9cd1e00640 1 --2- 172.21.15.86:0/723041891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cd406dda0 0x7f9cd413c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:04.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.202+0000 7f9cd1400640 1 -- 172.21.15.86:0/723041891 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9cd4071d40 msgr2=0x7f9cd40cc9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:04.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.202+0000 7f9cd2800640 1 --2- 172.21.15.86:0/723041891 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9cd4071d40 0x7f9cd40cc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:04.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.202+0000 7f9cd1400640 1 --2- 172.21.15.86:0/723041891 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9cd4071d40 0x7f9cd40cc9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:04.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.202+0000 7f9cd1400640 1 -- 172.21.15.86:0/723041891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cd406dda0 msgr2=0x7f9cd413c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:04.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.202+0000 7f9cd1400640 1 --2- 172.21.15.86:0/723041891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cd406dda0 0x7f9cd413c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:04.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.202+0000 7f9cd1400640 1 -- 172.21.15.86:0/723041891 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cd414c1a0 con 0x7f9cd406efc0 2024-04-20T21:19:04.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.202+0000 7f9cd1e00640 1 --2- 172.21.15.86:0/723041891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cd406dda0 0x7f9cd413c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:04.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.202+0000 7f9cd1400640 1 --2- 172.21.15.86:0/723041891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cd406efc0 0x7f9cd40cc4b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9cc002b9d0 tx=0x7f9cc00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:04.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.203+0000 7f9cca000640 1 -- 172.21.15.86:0/723041891 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc0034040 con 0x7f9cd406efc0 2024-04-20T21:19:04.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.203+0000 7f9cca000640 1 -- 172.21.15.86:0/723041891 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cc00433f0 con 0x7f9cd406efc0 2024-04-20T21:19:04.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.203+0000 7f9cca000640 1 -- 172.21.15.86:0/723041891 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc00437e0 con 0x7f9cd406efc0 2024-04-20T21:19:04.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.203+0000 7f9cd8c00640 1 -- 172.21.15.86:0/723041891 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cd414c480 con 0x7f9cd406efc0 2024-04-20T21:19:04.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.203+0000 7f9cd8c00640 1 -- 172.21.15.86:0/723041891 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cd414ca40 con 0x7f9cd406efc0 2024-04-20T21:19:04.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.204+0000 7f9cd8c00640 1 -- 172.21.15.86:0/723041891 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cd40cf700 con 0x7f9cd406efc0 2024-04-20T21:19:04.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.205+0000 7f9cca000640 1 -- 172.21.15.86:0/723041891 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9cc004a060 con 0x7f9cd406efc0 2024-04-20T21:19:04.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.206+0000 7f9cca000640 1 --2- 172.21.15.86:0/723041891 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9cb406c8f0 0x7f9cb406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:04.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.206+0000 7f9cca000640 1 -- 172.21.15.86:0/723041891 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f9cc00ada20 con 0x7f9cd406efc0 2024-04-20T21:19:04.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.206+0000 7f9cd1e00640 1 --2- 172.21.15.86:0/723041891 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9cb406c8f0 0x7f9cb406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:04.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.207+0000 7f9cd1e00640 1 --2- 172.21.15.86:0/723041891 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9cb406c8f0 0x7f9cb406edb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9cc40088b0 tx=0x7f9cc4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:04.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.212+0000 7f9cca000640 1 -- 172.21.15.86:0/723041891 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9cd40cf700 con 0x7f9cd406efc0 2024-04-20T21:19:04.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.468+0000 7f9cd8c00640 1 -- 172.21.15.86:0/723041891 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f9cd40689e0 con 0x7f9cd406efc0 2024-04-20T21:19:04.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.472+0000 7f9cca000640 1 -- 172.21.15.86:0/723041891 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7f9cc003e070 con 0x7f9cd406efc0 2024-04-20T21:19:04.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.474+0000 7f9cd8c00640 1 -- 172.21.15.86:0/723041891 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9cb406c8f0 msgr2=0x7f9cb406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:04.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.474+0000 7f9cd8c00640 1 --2- 172.21.15.86:0/723041891 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9cb406c8f0 0x7f9cb406edb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9cc40088b0 tx=0x7f9cc4008040 comp rx=0 tx=0).stop 2024-04-20T21:19:04.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.474+0000 7f9cd8c00640 1 -- 172.21.15.86:0/723041891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cd406efc0 msgr2=0x7f9cd40cc4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:04.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.474+0000 7f9cd8c00640 1 --2- 172.21.15.86:0/723041891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cd406efc0 0x7f9cd40cc4b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9cc002b9d0 tx=0x7f9cc00046c0 comp rx=0 tx=0).stop 2024-04-20T21:19:04.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.475+0000 7f9cd8c00640 1 -- 172.21.15.86:0/723041891 shutdown_connections 2024-04-20T21:19:04.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.475+0000 7f9cd8c00640 1 --2- 172.21.15.86:0/723041891 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9cb406c8f0 0x7f9cb406edb0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:04.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.475+0000 7f9cd8c00640 1 --2- 172.21.15.86:0/723041891 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9cd4071d40 0x7f9cd40cc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:04.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.475+0000 7f9cd8c00640 1 --2- 172.21.15.86:0/723041891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cd406efc0 0x7f9cd40cc4b0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:04.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.475+0000 7f9cd8c00640 1 --2- 172.21.15.86:0/723041891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cd406dda0 0x7f9cd413c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:04.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.475+0000 7f9cd8c00640 1 -- 172.21.15.86:0/723041891 >> 172.21.15.86:0/723041891 conn(0x7f9cd406b980 msgr2=0x7f9cd4142230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:04.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.475+0000 7f9cd8c00640 1 -- 172.21.15.86:0/723041891 shutdown_connections 2024-04-20T21:19:04.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.475+0000 7f9cd8c00640 1 -- 172.21.15.86:0/723041891 wait complete. 2024-04-20T21:19:04.492 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-20T21:19:04.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.687+0000 7f9e38e00640 1 Processor -- start 2024-04-20T21:19:04.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.687+0000 7f9e38e00640 1 -- start start 2024-04-20T21:19:04.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.688+0000 7f9e38e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e3406eeb0 0x7f9e340b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:04.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.688+0000 7f9e38e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9e3406f290 0x7f9e340cd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:04.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.688+0000 7f9e38e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9e340cdb70 0x7f9e3415bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:04.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.688+0000 7f9e38e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9e3415f160 con 0x7f9e340cdb70 2024-04-20T21:19:04.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.688+0000 7f9e38e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f9e3415f440 con 0x7f9e3406f290 2024-04-20T21:19:04.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.688+0000 7f9e38e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9e3415f720 con 0x7f9e3406eeb0 2024-04-20T21:19:04.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.688+0000 7f9e32800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9e340cdb70 0x7f9e3415bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:04.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.689+0000 7f9e32800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9e340cdb70 0x7f9e3415bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55830/0 (socket says 172.21.15.86:55830) 2024-04-20T21:19:04.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.689+0000 7f9e32800640 1 -- 172.21.15.86:0/3403023858 learned_addr learned my addr 172.21.15.86:0/3403023858 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:04.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.689+0000 7f9e31e00640 1 --2- 172.21.15.86:0/3403023858 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e3406eeb0 0x7f9e340b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:04.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.689+0000 7f9e32800640 1 -- 172.21.15.86:0/3403023858 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e3406eeb0 msgr2=0x7f9e340b3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:04.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.689+0000 7f9e29400640 1 --2- 172.21.15.86:0/3403023858 >> v2:172.21.15.144:3300/0 conn(0x7f9e3406f290 0x7f9e340cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:04.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.689+0000 7f9e32800640 1 --2- 172.21.15.86:0/3403023858 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e3406eeb0 0x7f9e340b3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:04.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.689+0000 7f9e32800640 1 -- 172.21.15.86:0/3403023858 >> v2:172.21.15.144:3300/0 conn(0x7f9e3406f290 msgr2=0x7f9e340cd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:04.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.689+0000 7f9e32800640 1 --2- 172.21.15.86:0/3403023858 >> v2:172.21.15.144:3300/0 conn(0x7f9e3406f290 0x7f9e340cd630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:04.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.689+0000 7f9e32800640 1 -- 172.21.15.86:0/3403023858 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e3415fa00 con 0x7f9e340cdb70 2024-04-20T21:19:04.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.689+0000 7f9e31e00640 1 --2- 172.21.15.86:0/3403023858 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e3406eeb0 0x7f9e340b3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:04.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.690+0000 7f9e32800640 1 --2- 172.21.15.86:0/3403023858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e340cdb70 0x7f9e3415bbb0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9e3406ffb0 tx=0x7f9e2c02bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77481b01e0ab972d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:04.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.690+0000 7f9e31400640 1 -- 172.21.15.86:0/3403023858 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c02c950 con 0x7f9e340cdb70 2024-04-20T21:19:04.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.690+0000 7f9e31400640 1 -- 172.21.15.86:0/3403023858 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e2c02caf0 con 0x7f9e340cdb70 2024-04-20T21:19:04.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.690+0000 7f9e31400640 1 -- 172.21.15.86:0/3403023858 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c0343f0 con 0x7f9e340cdb70 2024-04-20T21:19:04.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.691+0000 7f9e38e00640 1 -- 172.21.15.86:0/3403023858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e340cdb70 msgr2=0x7f9e3415bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:04.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.691+0000 7f9e38e00640 1 --2- 172.21.15.86:0/3403023858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e340cdb70 0x7f9e3415bbb0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9e3406ffb0 tx=0x7f9e2c02bdb0 comp rx=0 tx=0).stop 2024-04-20T21:19:04.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.691+0000 7f9e38e00640 1 -- 172.21.15.86:0/3403023858 shutdown_connections 2024-04-20T21:19:04.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.691+0000 7f9e38e00640 1 --2- 172.21.15.86:0/3403023858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e340cdb70 0x7f9e3415bbb0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:04.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.691+0000 7f9e38e00640 1 --2- 172.21.15.86:0/3403023858 >> v2:172.21.15.144:3300/0 conn(0x7f9e3406f290 0x7f9e340cd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:04.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.691+0000 7f9e38e00640 1 --2- 172.21.15.86:0/3403023858 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e3406eeb0 0x7f9e340b3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:04.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.691+0000 7f9e38e00640 1 -- 172.21.15.86:0/3403023858 >> 172.21.15.86:0/3403023858 conn(0x7f9e3406b980 msgr2=0x7f9e3406bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:04.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.691+0000 7f9e38e00640 1 -- 172.21.15.86:0/3403023858 shutdown_connections 2024-04-20T21:19:04.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.691+0000 7f9e38e00640 1 -- 172.21.15.86:0/3403023858 wait complete. 2024-04-20T21:19:04.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.693+0000 7f9e38e00640 1 Processor -- start 2024-04-20T21:19:04.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.693+0000 7f9e38e00640 1 -- start start 2024-04-20T21:19:04.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.694+0000 7f9e38e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e3406eeb0 0x7f9e340ad130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:04.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.694+0000 7f9e38e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e3406f290 0x7f9e340ae1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:04.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.694+0000 7f9e38e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e340cdb70 0x7f9e340ae710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:04.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.694+0000 7f9e38e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9e3406f6d0 con 0x7f9e3406f290 2024-04-20T21:19:04.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.694+0000 7f9e38e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9e340cff90 con 0x7f9e3406eeb0 2024-04-20T21:19:04.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.694+0000 7f9e38e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9e3414f460 con 0x7f9e340cdb70 2024-04-20T21:19:04.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.694+0000 7f9e29400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e3406f290 0x7f9e340ae1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:04.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.694+0000 7f9e29400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e3406f290 0x7f9e340ae1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55840/0 (socket says 172.21.15.86:55840) 2024-04-20T21:19:04.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.694+0000 7f9e29400640 1 -- 172.21.15.86:0/274540913 learned_addr learned my addr 172.21.15.86:0/274540913 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:04.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.695+0000 7f9e31e00640 1 --2- 172.21.15.86:0/274540913 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e3406eeb0 0x7f9e340ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:04.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.695+0000 7f9e32800640 1 --2- 172.21.15.86:0/274540913 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e340cdb70 0x7f9e340ae710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:04.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.695+0000 7f9e29400640 1 -- 172.21.15.86:0/274540913 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e340cdb70 msgr2=0x7f9e340ae710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:04.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.695+0000 7f9e29400640 1 --2- 172.21.15.86:0/274540913 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e340cdb70 0x7f9e340ae710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:04.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.695+0000 7f9e29400640 1 -- 172.21.15.86:0/274540913 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e3406eeb0 msgr2=0x7f9e340ad130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:04.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.695+0000 7f9e29400640 1 --2- 172.21.15.86:0/274540913 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e3406eeb0 0x7f9e340ad130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:04.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.695+0000 7f9e29400640 1 -- 172.21.15.86:0/274540913 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e340a9a00 con 0x7f9e3406f290 2024-04-20T21:19:04.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.695+0000 7f9e32800640 1 --2- 172.21.15.86:0/274540913 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e340cdb70 0x7f9e340ae710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:04.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.695+0000 7f9e31e00640 1 --2- 172.21.15.86:0/274540913 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e3406eeb0 0x7f9e340ad130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:04.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.695+0000 7f9e29400640 1 --2- 172.21.15.86:0/274540913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e3406f290 0x7f9e340ae1d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c00c820 tx=0x7f9e1c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:04.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.695+0000 7f9e2aa00640 1 -- 172.21.15.86:0/274540913 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c015070 con 0x7f9e3406f290 2024-04-20T21:19:04.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.695+0000 7f9e2aa00640 1 -- 172.21.15.86:0/274540913 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e1c004240 con 0x7f9e3406f290 2024-04-20T21:19:04.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.695+0000 7f9e38e00640 1 -- 172.21.15.86:0/274540913 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e340a9ce0 con 0x7f9e3406f290 2024-04-20T21:19:04.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.695+0000 7f9e2aa00640 1 -- 172.21.15.86:0/274540913 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c00ce10 con 0x7f9e3406f290 2024-04-20T21:19:04.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.696+0000 7f9e38e00640 1 -- 172.21.15.86:0/274540913 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e340aa2a0 con 0x7f9e3406f290 2024-04-20T21:19:04.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.697+0000 7f9e38e00640 1 -- 172.21.15.86:0/274540913 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e340689e0 con 0x7f9e3406f290 2024-04-20T21:19:04.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.698+0000 7f9e2aa00640 1 -- 172.21.15.86:0/274540913 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9e1c007830 con 0x7f9e3406f290 2024-04-20T21:19:04.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.699+0000 7f9e2aa00640 1 --2- 172.21.15.86:0/274540913 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9e1006c8f0 0x7f9e1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:04.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.699+0000 7f9e2aa00640 1 -- 172.21.15.86:0/274540913 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f9e1c08b5e0 con 0x7f9e3406f290 2024-04-20T21:19:04.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.699+0000 7f9e31e00640 1 --2- 172.21.15.86:0/274540913 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9e1006c8f0 0x7f9e1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:04.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.699+0000 7f9e31e00640 1 --2- 172.21.15.86:0/274540913 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9e1006c8f0 0x7f9e1006edb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9e24006fd0 tx=0x7f9e24008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:04.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.703+0000 7f9e2aa00640 1 -- 172.21.15.86:0/274540913 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9e1c059df0 con 0x7f9e3406f290 2024-04-20T21:19:04.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.970+0000 7f9e38e00640 1 -- 172.21.15.86:0/274540913 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f9e340ad670 con 0x7f9e3406f290 2024-04-20T21:19:04.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.975+0000 7f9e2aa00640 1 -- 172.21.15.86:0/274540913 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f9e1c0596f0 con 0x7f9e3406f290 2024-04-20T21:19:04.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.978+0000 7f9e38e00640 1 -- 172.21.15.86:0/274540913 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9e1006c8f0 msgr2=0x7f9e1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:04.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.978+0000 7f9e38e00640 1 --2- 172.21.15.86:0/274540913 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9e1006c8f0 0x7f9e1006edb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9e24006fd0 tx=0x7f9e24008040 comp rx=0 tx=0).stop 2024-04-20T21:19:04.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.978+0000 7f9e38e00640 1 -- 172.21.15.86:0/274540913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e3406f290 msgr2=0x7f9e340ae1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:04.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.978+0000 7f9e38e00640 1 --2- 172.21.15.86:0/274540913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e3406f290 0x7f9e340ae1d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c00c820 tx=0x7f9e1c00cce0 comp rx=0 tx=0).stop 2024-04-20T21:19:04.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.978+0000 7f9e38e00640 1 -- 172.21.15.86:0/274540913 shutdown_connections 2024-04-20T21:19:04.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.978+0000 7f9e38e00640 1 --2- 172.21.15.86:0/274540913 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9e1006c8f0 0x7f9e1006edb0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:04.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.978+0000 7f9e38e00640 1 --2- 172.21.15.86:0/274540913 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e340cdb70 0x7f9e340ae710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:04.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.978+0000 7f9e38e00640 1 --2- 172.21.15.86:0/274540913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e3406f290 0x7f9e340ae1d0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:04.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.978+0000 7f9e38e00640 1 --2- 172.21.15.86:0/274540913 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e3406eeb0 0x7f9e340ad130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:04.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.978+0000 7f9e38e00640 1 -- 172.21.15.86:0/274540913 >> 172.21.15.86:0/274540913 conn(0x7f9e3406b980 msgr2=0x7f9e3415aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:04.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.978+0000 7f9e38e00640 1 -- 172.21.15.86:0/274540913 shutdown_connections 2024-04-20T21:19:04.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:04.978+0000 7f9e38e00640 1 -- 172.21.15.86:0/274540913 wait complete. 2024-04-20T21:19:04.996 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-20T21:19:05.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.194+0000 7f6c0b800640 1 Processor -- start 2024-04-20T21:19:05.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.194+0000 7f6c0b800640 1 -- start start 2024-04-20T21:19:05.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.196+0000 7f6c0b800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6c0415e740 0x7f6c040d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:05.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.196+0000 7f6c0b800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c040d0fe0 0x7f6c041623d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:05.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.196+0000 7f6c0b800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c04163250 con 0x7f6c04162910 2024-04-20T21:19:05.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.196+0000 7f6c0b800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f6c040b34b0 con 0x7f6c0415e740 2024-04-20T21:19:05.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.196+0000 7f6c0b800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6c040b3790 con 0x7f6c040d0fe0 2024-04-20T21:19:05.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.196+0000 7f6c09800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f6c04162910 0x7f6c04071cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44432/0 (socket says 172.21.15.86:44432) 2024-04-20T21:19:05.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.196+0000 7f6c09800640 1 -- 172.21.15.86:0/1159921563 learned_addr learned my addr 172.21.15.86:0/1159921563 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:19:05.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.197+0000 7f6c03e00640 1 --2- 172.21.15.86:0/1159921563 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c040d0fe0 0x7f6c041623d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:05.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.197+0000 7f6c08e00640 1 --2- 172.21.15.86:0/1159921563 >> v2:172.21.15.144:3300/0 conn(0x7f6c0415e740 0x7f6c040d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:05.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.197+0000 7f6c03400640 1 -- 172.21.15.86:0/1159921563 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4210387732 0 0) 0x7f6c04163250 con 0x7f6c04162910 2024-04-20T21:19:05.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.197+0000 7f6c03400640 1 -- 172.21.15.86:0/1159921563 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6bec003520 con 0x7f6c04162910 2024-04-20T21:19:05.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.197+0000 7f6c03400640 1 -- 172.21.15.86:0/1159921563 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3443555323 0 0) 0x7f6bec003520 con 0x7f6c04162910 2024-04-20T21:19:05.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.197+0000 7f6c03400640 1 -- 172.21.15.86:0/1159921563 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c040d0fe0 msgr2=0x7f6c041623d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:05.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.197+0000 7f6c03400640 1 --2- 172.21.15.86:0/1159921563 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c040d0fe0 0x7f6c041623d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:05.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.197+0000 7f6c03400640 1 -- 172.21.15.86:0/1159921563 >> v2:172.21.15.144:3300/0 conn(0x7f6c0415e740 msgr2=0x7f6c040d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:05.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.197+0000 7f6c03400640 1 --2- 172.21.15.86:0/1159921563 >> v2:172.21.15.144:3300/0 conn(0x7f6c0415e740 0x7f6c040d0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:05.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.197+0000 7f6c03400640 1 -- 172.21.15.86:0/1159921563 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c040b3a70 con 0x7f6c04162910 2024-04-20T21:19:05.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.197+0000 7f6c03400640 1 -- 172.21.15.86:0/1159921563 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f6bfc0019f0 con 0x7f6c04162910 2024-04-20T21:19:05.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.197+0000 7f6c03400640 1 -- 172.21.15.86:0/1159921563 >> v1:172.21.15.86:6789/0 conn(0x7f6c04162910 legacy=0x7f6c04071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:05.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.197+0000 7f6c03400640 1 --2- 172.21.15.86:0/1159921563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bec0039b0 0x7f6bec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:05.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.198+0000 7f6c03400640 1 --2- 172.21.15.86:0/1159921563 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c040d0fe0 0x7f6bec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:05.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.198+0000 7f6c03400640 1 --2- 172.21.15.86:0/1159921563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bec004d40 0x7f6bec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:05.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.198+0000 7f6c03400640 1 -- 172.21.15.86:0/1159921563 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6c040b34b0 con 0x7f6bec0039b0 2024-04-20T21:19:05.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.198+0000 7f6c03400640 1 -- 172.21.15.86:0/1159921563 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6c040b3790 con 0x7f6bec004d40 2024-04-20T21:19:05.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.198+0000 7f6c03400640 1 -- 172.21.15.86:0/1159921563 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6bec00ba40 con 0x7f6c040d0fe0 2024-04-20T21:19:05.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.198+0000 7f6c08e00640 1 --2- 172.21.15.86:0/1159921563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bec0039b0 0x7f6bec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:05.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.198+0000 7f6c09800640 1 --2- 172.21.15.86:0/1159921563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bec004d40 0x7f6bec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:05.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.198+0000 7f6c03e00640 1 --2- 172.21.15.86:0/1159921563 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c040d0fe0 0x7f6bec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:05.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.198+0000 7f6c08e00640 1 -- 172.21.15.86:0/1159921563 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c040d0fe0 msgr2=0x7f6bec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:05.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.198+0000 7f6c08e00640 1 --2- 172.21.15.86:0/1159921563 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c040d0fe0 0x7f6bec004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:05.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.198+0000 7f6c08e00640 1 -- 172.21.15.86:0/1159921563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bec004d40 msgr2=0x7f6bec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:05.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.198+0000 7f6c08e00640 1 --2- 172.21.15.86:0/1159921563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bec004d40 0x7f6bec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:05.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.198+0000 7f6c08e00640 1 -- 172.21.15.86:0/1159921563 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bec003520 con 0x7f6bec0039b0 2024-04-20T21:19:05.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.198+0000 7f6c09800640 1 --2- 172.21.15.86:0/1159921563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bec004d40 0x7f6bec0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:05.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.198+0000 7f6c03e00640 1 --2- 172.21.15.86:0/1159921563 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c040d0fe0 0x7f6bec004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:05.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.200+0000 7f6c08e00640 1 --2- 172.21.15.86:0/1159921563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bec0039b0 0x7f6bec003dd0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6bf800ba30 tx=0x7f6bec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a291dc07a1c4249d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:05.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.200+0000 7f6c03400640 1 -- 172.21.15.86:0/1159921563 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf8036020 con 0x7f6bec0039b0 2024-04-20T21:19:05.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.200+0000 7f6c03400640 1 -- 172.21.15.86:0/1159921563 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bf8032680 con 0x7f6bec0039b0 2024-04-20T21:19:05.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.200+0000 7f6c03400640 1 -- 172.21.15.86:0/1159921563 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bf800b600 con 0x7f6bec0039b0 2024-04-20T21:19:05.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.200+0000 7f6c0b800640 1 -- 172.21.15.86:0/1159921563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bec0039b0 msgr2=0x7f6bec003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:05.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.200+0000 7f6c0b800640 1 --2- 172.21.15.86:0/1159921563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bec0039b0 0x7f6bec003dd0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6bf800ba30 tx=0x7f6bec005830 comp rx=0 tx=0).stop 2024-04-20T21:19:05.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.200+0000 7f6c09800640 1 -- 172.21.15.86:0/1159921563 reap_dead start 2024-04-20T21:19:05.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.201+0000 7f6c0b800640 1 -- 172.21.15.86:0/1159921563 shutdown_connections 2024-04-20T21:19:05.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.201+0000 7f6c0b800640 1 -- 172.21.15.86:0/1159921563 >> 172.21.15.86:0/1159921563 conn(0x7f6c0406b980 msgr2=0x7f6c0406bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:05.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.201+0000 7f6c0b800640 1 -- 172.21.15.86:0/1159921563 shutdown_connections 2024-04-20T21:19:05.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.201+0000 7f6c0b800640 1 -- 172.21.15.86:0/1159921563 wait complete. 2024-04-20T21:19:05.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.202+0000 7f6c0b800640 1 Processor -- start 2024-04-20T21:19:05.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.203+0000 7f6c0b800640 1 -- start start 2024-04-20T21:19:05.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.203+0000 7f6c0b800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6bec0039b0 0x7f6c041419c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:05.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.203+0000 7f6c0b800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c04152050 0x7f6c041524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:05.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.204+0000 7f6c0b800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c04152a10 0x7f6c040b0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:05.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.204+0000 7f6c0b800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6c040b4550 con 0x7f6c04152a10 2024-04-20T21:19:05.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.204+0000 7f6c0b800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6c040b4030 con 0x7f6c04152050 2024-04-20T21:19:05.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.204+0000 7f6c0b800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6c040aeb40 con 0x7f6bec0039b0 2024-04-20T21:19:05.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.204+0000 7f6c09800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c04152a10 0x7f6c040b0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:05.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.204+0000 7f6c09800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c04152a10 0x7f6c040b0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55862/0 (socket says 172.21.15.86:55862) 2024-04-20T21:19:05.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.204+0000 7f6c09800640 1 -- 172.21.15.86:0/3812918024 learned_addr learned my addr 172.21.15.86:0/3812918024 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:05.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.204+0000 7f6c08e00640 1 --2- 172.21.15.86:0/3812918024 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6bec0039b0 0x7f6c041419c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:05.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.204+0000 7f6c09800640 1 -- 172.21.15.86:0/3812918024 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6bec0039b0 msgr2=0x7f6c041419c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:05.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.204+0000 7f6c09800640 1 --2- 172.21.15.86:0/3812918024 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6bec0039b0 0x7f6c041419c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:05.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.204+0000 7f6c09800640 1 -- 172.21.15.86:0/3812918024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c04152050 msgr2=0x7f6c041524d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:05.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.204+0000 7f6c09800640 1 --2- 172.21.15.86:0/3812918024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c04152050 0x7f6c041524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:05.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.204+0000 7f6c09800640 1 -- 172.21.15.86:0/3812918024 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c04153720 con 0x7f6c04152a10 2024-04-20T21:19:05.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.204+0000 7f6c09800640 1 --2- 172.21.15.86:0/3812918024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c04152a10 0x7f6c040b0830 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6c04071c60 tx=0x7f6bfc00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:05.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.205+0000 7f6c00c00640 1 -- 172.21.15.86:0/3812918024 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc004d60 con 0x7f6c04152a10 2024-04-20T21:19:05.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.205+0000 7f6c00c00640 1 -- 172.21.15.86:0/3812918024 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bfc00bb20 con 0x7f6c04152a10 2024-04-20T21:19:05.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.205+0000 7f6c00c00640 1 -- 172.21.15.86:0/3812918024 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc007110 con 0x7f6c04152a10 2024-04-20T21:19:05.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.205+0000 7f6c0b800640 1 -- 172.21.15.86:0/3812918024 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c04151020 con 0x7f6c04152a10 2024-04-20T21:19:05.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.205+0000 7f6c0b800640 1 -- 172.21.15.86:0/3812918024 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c041515e0 con 0x7f6c04152a10 2024-04-20T21:19:05.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.206+0000 7f6c0b800640 1 -- 172.21.15.86:0/3812918024 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c040689e0 con 0x7f6c04152a10 2024-04-20T21:19:05.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.207+0000 7f6c00c00640 1 -- 172.21.15.86:0/3812918024 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6bfc01b020 con 0x7f6c04152a10 2024-04-20T21:19:05.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.209+0000 7f6c00c00640 1 --2- 172.21.15.86:0/3812918024 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6be406c920 0x7f6be406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:05.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.209+0000 7f6c00c00640 1 -- 172.21.15.86:0/3812918024 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f6bfc08b900 con 0x7f6c04152a10 2024-04-20T21:19:05.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.209+0000 7f6c08e00640 1 --2- 172.21.15.86:0/3812918024 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6be406c920 0x7f6be406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:05.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.210+0000 7f6c08e00640 1 --2- 172.21.15.86:0/3812918024 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6be406c920 0x7f6be406ede0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6bf800ba00 tx=0x7f6bf8030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:05.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.213+0000 7f6c00c00640 1 -- 172.21.15.86:0/3812918024 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6bfc05a070 con 0x7f6c04152a10 2024-04-20T21:19:05.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.481+0000 7f6c0b800640 1 -- 172.21.15.86:0/3812918024 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f6c04163250 con 0x7f6c04152a10 2024-04-20T21:19:05.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.486+0000 7f6c00c00640 1 -- 172.21.15.86:0/3812918024 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) ==== 97+0+0 (secure 0 0 0) 0x7f6bfc059970 con 0x7f6c04152a10 2024-04-20T21:19:05.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.489+0000 7f6c0b800640 1 -- 172.21.15.86:0/3812918024 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6be406c920 msgr2=0x7f6be406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:05.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.489+0000 7f6c0b800640 1 --2- 172.21.15.86:0/3812918024 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6be406c920 0x7f6be406ede0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6bf800ba00 tx=0x7f6bf8030000 comp rx=0 tx=0).stop 2024-04-20T21:19:05.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.489+0000 7f6c0b800640 1 -- 172.21.15.86:0/3812918024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c04152a10 msgr2=0x7f6c040b0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:05.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.489+0000 7f6c0b800640 1 --2- 172.21.15.86:0/3812918024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c04152a10 0x7f6c040b0830 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6c04071c60 tx=0x7f6bfc00b4a0 comp rx=0 tx=0).stop 2024-04-20T21:19:05.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.489+0000 7f6c0b800640 1 -- 172.21.15.86:0/3812918024 shutdown_connections 2024-04-20T21:19:05.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.489+0000 7f6c0b800640 1 --2- 172.21.15.86:0/3812918024 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6be406c920 0x7f6be406ede0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:05.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.489+0000 7f6c0b800640 1 --2- 172.21.15.86:0/3812918024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c04152a10 0x7f6c040b0830 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:05.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.489+0000 7f6c0b800640 1 --2- 172.21.15.86:0/3812918024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c04152050 0x7f6c041524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:05.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.489+0000 7f6c0b800640 1 --2- 172.21.15.86:0/3812918024 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6bec0039b0 0x7f6c041419c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:05.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.489+0000 7f6c0b800640 1 -- 172.21.15.86:0/3812918024 >> 172.21.15.86:0/3812918024 conn(0x7f6c0406b980 msgr2=0x7f6c04072540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:05.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.489+0000 7f6c0b800640 1 -- 172.21.15.86:0/3812918024 shutdown_connections 2024-04-20T21:19:05.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.489+0000 7f6c0b800640 1 -- 172.21.15.86:0/3812918024 wait complete. 2024-04-20T21:19:05.505 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-20T21:19:05.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.693+0000 7f0fa6c00640 1 Processor -- start 2024-04-20T21:19:05.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.694+0000 7f0fa6c00640 1 -- start start 2024-04-20T21:19:05.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.695+0000 7f0fa6c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fa006c4b0 0x7f0fa00aee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:05.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.695+0000 7f0fa6c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0fa00af3b0 0x7f0fa015c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:05.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.695+0000 7f0fa6c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fa015a3f0 con 0x7f0fa015a010 2024-04-20T21:19:05.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.695+0000 7f0fa6c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f0fa006fe50 con 0x7f0fa00af3b0 2024-04-20T21:19:05.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.695+0000 7f0fa6c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0fa00afb30 con 0x7f0fa006c4b0 2024-04-20T21:19:05.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.695+0000 7f0fa5800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f0fa015a010 0x7f0fa006bda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44434/0 (socket says 172.21.15.86:44434) 2024-04-20T21:19:05.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.695+0000 7f0fa5800640 1 -- 172.21.15.86:0/3172768968 learned_addr learned my addr 172.21.15.86:0/3172768968 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:19:05.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.696+0000 7f0fa4e00640 1 --2- 172.21.15.86:0/3172768968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fa006c4b0 0x7f0fa00aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:05.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.696+0000 7f0f9fe00640 1 -- 172.21.15.86:0/3172768968 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 844503281 0 0) 0x7f0fa015a3f0 con 0x7f0fa015a010 2024-04-20T21:19:05.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.696+0000 7f0fa6200640 1 --2- 172.21.15.86:0/3172768968 >> v2:172.21.15.144:3300/0 conn(0x7f0fa00af3b0 0x7f0fa015c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:05.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.696+0000 7f0f9fe00640 1 -- 172.21.15.86:0/3172768968 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f88003520 con 0x7f0fa015a010 2024-04-20T21:19:05.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.696+0000 7f0f9fe00640 1 -- 172.21.15.86:0/3172768968 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2242325370 0 0) 0x7f0f88003520 con 0x7f0fa015a010 2024-04-20T21:19:05.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.696+0000 7f0f9fe00640 1 -- 172.21.15.86:0/3172768968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fa006c4b0 msgr2=0x7f0fa00aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:05.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.696+0000 7f0f9fe00640 1 --2- 172.21.15.86:0/3172768968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fa006c4b0 0x7f0fa00aee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:05.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.696+0000 7f0f9fe00640 1 -- 172.21.15.86:0/3172768968 >> v2:172.21.15.144:3300/0 conn(0x7f0fa00af3b0 msgr2=0x7f0fa015c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:05.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.696+0000 7f0f9fe00640 1 --2- 172.21.15.86:0/3172768968 >> v2:172.21.15.144:3300/0 conn(0x7f0fa00af3b0 0x7f0fa015c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:05.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.696+0000 7f0f9fe00640 1 -- 172.21.15.86:0/3172768968 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fa00b31a0 con 0x7f0fa015a010 2024-04-20T21:19:05.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.696+0000 7f0f9fe00640 1 -- 172.21.15.86:0/3172768968 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f0f94001670 con 0x7f0fa015a010 2024-04-20T21:19:05.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.696+0000 7f0f9fe00640 1 -- 172.21.15.86:0/3172768968 >> v1:172.21.15.86:6789/0 conn(0x7f0fa015a010 legacy=0x7f0fa006bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:05.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.696+0000 7f0f9fe00640 1 --2- 172.21.15.86:0/3172768968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fa006c4b0 0x7f0f880039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:05.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.696+0000 7f0f9fe00640 1 --2- 172.21.15.86:0/3172768968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f88003f80 0x7f0f88004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:05.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.696+0000 7f0f9fe00640 1 --2- 172.21.15.86:0/3172768968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f88004d40 0x7f0f880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:05.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.696+0000 7f0f9fe00640 1 -- 172.21.15.86:0/3172768968 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa006fe50 con 0x7f0f88003f80 2024-04-20T21:19:05.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.696+0000 7f0f9fe00640 1 -- 172.21.15.86:0/3172768968 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa00afb30 con 0x7f0f88004d40 2024-04-20T21:19:05.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.696+0000 7f0f9fe00640 1 -- 172.21.15.86:0/3172768968 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0f8800ba70 con 0x7f0fa006c4b0 2024-04-20T21:19:05.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.696+0000 7f0fa6200640 1 --2- 172.21.15.86:0/3172768968 >> v2:172.21.15.144:3300/0 conn(0x7f0fa00af3b0 0x7f0fa015c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:19:05.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.696+0000 7f0fa4e00640 1 --2- 172.21.15.86:0/3172768968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f88003f80 0x7f0f88004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:05.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.697+0000 7f0fa5800640 1 --2- 172.21.15.86:0/3172768968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fa006c4b0 0x7f0f880039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:05.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.697+0000 7f0fa4e00640 1 -- 172.21.15.86:0/3172768968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fa006c4b0 msgr2=0x7f0f880039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:05.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.697+0000 7f0fa4e00640 1 --2- 172.21.15.86:0/3172768968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fa006c4b0 0x7f0f880039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:05.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.697+0000 7f0fa4e00640 1 -- 172.21.15.86:0/3172768968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f88004d40 msgr2=0x7f0f880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:05.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.697+0000 7f0fa4e00640 1 --2- 172.21.15.86:0/3172768968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f88004d40 0x7f0f880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:05.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.697+0000 7f0fa4e00640 1 -- 172.21.15.86:0/3172768968 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f88003520 con 0x7f0f88003f80 2024-04-20T21:19:05.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.697+0000 7f0fa5800640 1 --2- 172.21.15.86:0/3172768968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fa006c4b0 0x7f0f880039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:05.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.698+0000 7f0fa4e00640 1 --2- 172.21.15.86:0/3172768968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f88003f80 0x7f0f88004800 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c009ac0 tx=0x7f0f88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92a3a0f49eac0622 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:05.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.698+0000 7f0f9fe00640 1 -- 172.21.15.86:0/3172768968 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c039070 con 0x7f0f88003f80 2024-04-20T21:19:05.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.698+0000 7f0f9fe00640 1 -- 172.21.15.86:0/3172768968 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f8c02c6b0 con 0x7f0f88003f80 2024-04-20T21:19:05.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.698+0000 7f0f9fe00640 1 -- 172.21.15.86:0/3172768968 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c009690 con 0x7f0f88003f80 2024-04-20T21:19:05.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.699+0000 7f0fa6c00640 1 -- 172.21.15.86:0/3172768968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f88003f80 msgr2=0x7f0f88004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:05.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.699+0000 7f0fa6c00640 1 --2- 172.21.15.86:0/3172768968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f88003f80 0x7f0f88004800 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c009ac0 tx=0x7f0f88005830 comp rx=0 tx=0).stop 2024-04-20T21:19:05.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.699+0000 7f0fa6200640 1 -- 172.21.15.86:0/3172768968 reap_dead start 2024-04-20T21:19:05.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.699+0000 7f0fa6c00640 1 -- 172.21.15.86:0/3172768968 shutdown_connections 2024-04-20T21:19:05.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.699+0000 7f0fa6c00640 1 -- 172.21.15.86:0/3172768968 >> 172.21.15.86:0/3172768968 conn(0x7f0fa00b2220 msgr2=0x7f0fa00b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:05.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.699+0000 7f0fa6c00640 1 -- 172.21.15.86:0/3172768968 shutdown_connections 2024-04-20T21:19:05.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.699+0000 7f0fa6c00640 1 -- 172.21.15.86:0/3172768968 wait complete. 2024-04-20T21:19:05.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.701+0000 7f0fa6c00640 1 Processor -- start 2024-04-20T21:19:05.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.701+0000 7f0fa6c00640 1 -- start start 2024-04-20T21:19:05.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.702+0000 7f0fa6c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f88003f80 0x7f0fa014c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:05.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.702+0000 7f0fa6c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fa00c9480 0x7f0fa00c9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:05.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.702+0000 7f0fa6c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fa015ec70 0x7f0fa015f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:05.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.702+0000 7f0fa6c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa00b3a40 con 0x7f0fa015ec70 2024-04-20T21:19:05.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.702+0000 7f0fa6c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa00b3760 con 0x7f0fa00c9480 2024-04-20T21:19:05.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.702+0000 7f0fa6c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0fa00af800 con 0x7f0f88003f80 2024-04-20T21:19:05.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.702+0000 7f0fa6200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fa015ec70 0x7f0fa015f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:05.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.702+0000 7f0fa6200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fa015ec70 0x7f0fa015f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55892/0 (socket says 172.21.15.86:55892) 2024-04-20T21:19:05.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.702+0000 7f0fa6200640 1 -- 172.21.15.86:0/642201697 learned_addr learned my addr 172.21.15.86:0/642201697 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:05.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.702+0000 7f0fa6200640 1 -- 172.21.15.86:0/642201697 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f88003f80 msgr2=0x7f0fa014c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:05.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.702+0000 7f0fa6200640 1 --2- 172.21.15.86:0/642201697 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f88003f80 0x7f0fa014c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:05.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.702+0000 7f0fa6200640 1 -- 172.21.15.86:0/642201697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fa00c9480 msgr2=0x7f0fa00c9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:05.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.702+0000 7f0fa6200640 1 --2- 172.21.15.86:0/642201697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fa00c9480 0x7f0fa00c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:05.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.702+0000 7f0fa6200640 1 -- 172.21.15.86:0/642201697 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fa01f53b0 con 0x7f0fa015ec70 2024-04-20T21:19:05.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.703+0000 7f0fa6200640 1 --2- 172.21.15.86:0/642201697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fa015ec70 0x7f0fa015f120 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f0fa00b4d70 tx=0x7f0f9800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:05.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.703+0000 7f0f9d600640 1 -- 172.21.15.86:0/642201697 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9801a070 con 0x7f0fa015ec70 2024-04-20T21:19:05.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.703+0000 7f0f9d600640 1 -- 172.21.15.86:0/642201697 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f9800dc40 con 0x7f0fa015ec70 2024-04-20T21:19:05.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.703+0000 7f0f9d600640 1 -- 172.21.15.86:0/642201697 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f98003990 con 0x7f0fa015ec70 2024-04-20T21:19:05.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.703+0000 7f0fa6c00640 1 -- 172.21.15.86:0/642201697 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fa015deb0 con 0x7f0fa015ec70 2024-04-20T21:19:05.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.703+0000 7f0fa6c00640 1 -- 172.21.15.86:0/642201697 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fa015e470 con 0x7f0fa015ec70 2024-04-20T21:19:05.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.704+0000 7f0fa6c00640 1 -- 172.21.15.86:0/642201697 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fa00ad0e0 con 0x7f0fa015ec70 2024-04-20T21:19:05.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.705+0000 7f0f9d600640 1 -- 172.21.15.86:0/642201697 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0f9801d070 con 0x7f0fa015ec70 2024-04-20T21:19:05.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.706+0000 7f0f9d600640 1 --2- 172.21.15.86:0/642201697 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0f8006c920 0x7f0f8006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:05.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.706+0000 7f0f9d600640 1 -- 172.21.15.86:0/642201697 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f0f9808b610 con 0x7f0fa015ec70 2024-04-20T21:19:05.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.706+0000 7f0fa5800640 1 --2- 172.21.15.86:0/642201697 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0f8006c920 0x7f0f8006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:05.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.707+0000 7f0fa5800640 1 --2- 172.21.15.86:0/642201697 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0f8006c920 0x7f0f8006ede0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0159200 tx=0x7f0f94009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:05.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.710+0000 7f0f9d600640 1 -- 172.21.15.86:0/642201697 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0fa00ad0e0 con 0x7f0fa015ec70 2024-04-20T21:19:05.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.963+0000 7f0fa6c00640 1 -- 172.21.15.86:0/642201697 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f0fa004fcf0 con 0x7f0fa015ec70 2024-04-20T21:19:05.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.968+0000 7f0f9d600640 1 -- 172.21.15.86:0/642201697 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7f0f98010070 con 0x7f0fa015ec70 2024-04-20T21:19:05.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.970+0000 7f0fa6c00640 1 -- 172.21.15.86:0/642201697 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0f8006c920 msgr2=0x7f0f8006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:05.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.970+0000 7f0fa6c00640 1 --2- 172.21.15.86:0/642201697 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0f8006c920 0x7f0f8006ede0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0159200 tx=0x7f0f94009040 comp rx=0 tx=0).stop 2024-04-20T21:19:05.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.970+0000 7f0fa6c00640 1 -- 172.21.15.86:0/642201697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fa015ec70 msgr2=0x7f0fa015f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:05.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.970+0000 7f0fa6c00640 1 --2- 172.21.15.86:0/642201697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fa015ec70 0x7f0fa015f120 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f0fa00b4d70 tx=0x7f0f9800d4a0 comp rx=0 tx=0).stop 2024-04-20T21:19:05.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.971+0000 7f0fa6c00640 1 -- 172.21.15.86:0/642201697 shutdown_connections 2024-04-20T21:19:05.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.971+0000 7f0fa6c00640 1 --2- 172.21.15.86:0/642201697 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0f8006c920 0x7f0f8006ede0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:05.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.971+0000 7f0fa6c00640 1 --2- 172.21.15.86:0/642201697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fa015ec70 0x7f0fa015f120 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:05.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.971+0000 7f0fa6c00640 1 --2- 172.21.15.86:0/642201697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fa00c9480 0x7f0fa00c9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:05.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.971+0000 7f0fa6c00640 1 --2- 172.21.15.86:0/642201697 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f88003f80 0x7f0fa014c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:05.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.971+0000 7f0fa6c00640 1 -- 172.21.15.86:0/642201697 >> 172.21.15.86:0/642201697 conn(0x7f0fa00b2220 msgr2=0x7f0fa015a860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:05.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.971+0000 7f0fa6c00640 1 -- 172.21.15.86:0/642201697 shutdown_connections 2024-04-20T21:19:05.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:05.971+0000 7f0fa6c00640 1 -- 172.21.15.86:0/642201697 wait complete. 2024-04-20T21:19:05.987 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:19:06.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.172+0000 7f37b4a00640 1 Processor -- start 2024-04-20T21:19:06.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.172+0000 7f37b4a00640 1 -- start start 2024-04-20T21:19:06.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.174+0000 7f37b4a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37b015eaf0 0x7f37b00ba1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:06.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.174+0000 7f37b4a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f37b00d1ed0 0x7f37b00d2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:06.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.174+0000 7f37b4a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37b015eed0 con 0x7f37b00b4030 2024-04-20T21:19:06.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.174+0000 7f37b4a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f37b0163110 con 0x7f37b00d1ed0 2024-04-20T21:19:06.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.174+0000 7f37b4a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f37b00b7a00 con 0x7f37b015eaf0 2024-04-20T21:19:06.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.175+0000 7f37ade00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37b015eaf0 0x7f37b00ba1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:06.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.175+0000 7f37ade00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37b015eaf0 0x7f37b00ba1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:50578/0 (socket says 172.21.15.86:50578) 2024-04-20T21:19:06.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.175+0000 7f37ade00640 1 -- 172.21.15.86:0/1491497775 learned_addr learned my addr 172.21.15.86:0/1491497775 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:06.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.175+0000 7f37ad400640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f37b00b4030 0x7f37b00d17c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.86:54984/0 (socket says 172.21.15.86:54984) 2024-04-20T21:19:06.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.175+0000 7f37ae800640 1 --2- 172.21.15.86:0/1491497775 >> v2:172.21.15.144:3300/0 conn(0x7f37b00d1ed0 0x7f37b00d2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:06.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.175+0000 7f37ade00640 1 -- 172.21.15.86:0/1491497775 >> v2:172.21.15.144:3300/0 conn(0x7f37b00d1ed0 msgr2=0x7f37b00d2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:06.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.175+0000 7f37ade00640 1 --2- 172.21.15.86:0/1491497775 >> v2:172.21.15.144:3300/0 conn(0x7f37b00d1ed0 0x7f37b00d2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:06.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.175+0000 7f37ade00640 1 -- 172.21.15.86:0/1491497775 >> v1:172.21.15.144:6789/0 conn(0x7f37b00b4030 legacy=0x7f37b00d17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:06.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.175+0000 7f37ade00640 1 -- 172.21.15.86:0/1491497775 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37b01627e0 con 0x7f37b015eaf0 2024-04-20T21:19:06.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.177+0000 7f37ade00640 1 --2- 172.21.15.86:0/1491497775 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37b015eaf0 0x7f37b00ba1f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f37a0009870 tx=0x7f37a002ba20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76a90ede4f7be9c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:06.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.177+0000 7f37aca00640 1 -- 172.21.15.86:0/1491497775 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a002f840 con 0x7f37b015eaf0 2024-04-20T21:19:06.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.177+0000 7f37aca00640 1 -- 172.21.15.86:0/1491497775 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37a002f9e0 con 0x7f37b015eaf0 2024-04-20T21:19:06.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.177+0000 7f37aca00640 1 -- 172.21.15.86:0/1491497775 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a002fdb0 con 0x7f37b015eaf0 2024-04-20T21:19:06.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.178+0000 7f37b4a00640 1 -- 172.21.15.86:0/1491497775 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37b015eaf0 msgr2=0x7f37b00ba1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:06.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.178+0000 7f37b4a00640 1 --2- 172.21.15.86:0/1491497775 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37b015eaf0 0x7f37b00ba1f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f37a0009870 tx=0x7f37a002ba20 comp rx=0 tx=0).stop 2024-04-20T21:19:06.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.178+0000 7f37b4a00640 1 -- 172.21.15.86:0/1491497775 shutdown_connections 2024-04-20T21:19:06.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.178+0000 7f37b4a00640 1 --2- 172.21.15.86:0/1491497775 >> v2:172.21.15.144:3300/0 conn(0x7f37b00d1ed0 0x7f37b00d2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:06.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.178+0000 7f37b4a00640 1 --2- 172.21.15.86:0/1491497775 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37b015eaf0 0x7f37b00ba1f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:06.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.178+0000 7f37b4a00640 1 -- 172.21.15.86:0/1491497775 >> 172.21.15.86:0/1491497775 conn(0x7f37b00678a0 msgr2=0x7f37b0067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:06.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.178+0000 7f37b4a00640 1 -- 172.21.15.86:0/1491497775 shutdown_connections 2024-04-20T21:19:06.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.178+0000 7f37b4a00640 1 -- 172.21.15.86:0/1491497775 wait complete. 2024-04-20T21:19:06.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.180+0000 7f37b4a00640 1 Processor -- start 2024-04-20T21:19:06.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.180+0000 7f37b4a00640 1 -- start start 2024-04-20T21:19:06.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.181+0000 7f37b4a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37b00b4030 0x7f37b0159200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:06.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.181+0000 7f37b4a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37b00d1ed0 0x7f37b00cd060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:06.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.181+0000 7f37b4a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37b015eaf0 0x7f37b00c8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:06.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.181+0000 7f37b4a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f37b00ba730 con 0x7f37b00b4030 2024-04-20T21:19:06.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.181+0000 7f37b4a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f37b0162da0 con 0x7f37b00d1ed0 2024-04-20T21:19:06.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.181+0000 7f37b4a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f37b00ba8b0 con 0x7f37b015eaf0 2024-04-20T21:19:06.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.181+0000 7f37ade00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37b00b4030 0x7f37b0159200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:06.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.181+0000 7f37ade00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37b00b4030 0x7f37b0159200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42046/0 (socket says 172.21.15.86:42046) 2024-04-20T21:19:06.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.181+0000 7f37ade00640 1 -- 172.21.15.86:0/2215349025 learned_addr learned my addr 172.21.15.86:0/2215349025 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:06.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.181+0000 7f37ad400640 1 --2- 172.21.15.86:0/2215349025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37b00d1ed0 0x7f37b00cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:06.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.181+0000 7f37ade00640 1 -- 172.21.15.86:0/2215349025 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37b015eaf0 msgr2=0x7f37b00c8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:06.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.182+0000 7f37ade00640 1 --2- 172.21.15.86:0/2215349025 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37b015eaf0 0x7f37b00c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:06.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.182+0000 7f37ade00640 1 -- 172.21.15.86:0/2215349025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37b00d1ed0 msgr2=0x7f37b00cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:06.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.182+0000 7f37ade00640 1 --2- 172.21.15.86:0/2215349025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37b00d1ed0 0x7f37b00cd060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:06.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.182+0000 7f37ade00640 1 -- 172.21.15.86:0/2215349025 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37b00ce8b0 con 0x7f37b00b4030 2024-04-20T21:19:06.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.182+0000 7f37ad400640 1 --2- 172.21.15.86:0/2215349025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37b00d1ed0 0x7f37b00cd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:06.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.182+0000 7f37ade00640 1 --2- 172.21.15.86:0/2215349025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37b00b4030 0x7f37b0159200 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f37a0000c00 tx=0x7f37a0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:06.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.182+0000 7f37a6000640 1 -- 172.21.15.86:0/2215349025 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a0037bc0 con 0x7f37b00b4030 2024-04-20T21:19:06.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.182+0000 7f37a6000640 1 -- 172.21.15.86:0/2215349025 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37a0046070 con 0x7f37b00b4030 2024-04-20T21:19:06.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.182+0000 7f37b4a00640 1 -- 172.21.15.86:0/2215349025 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37b00ce5d0 con 0x7f37b00b4030 2024-04-20T21:19:06.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.182+0000 7f37a6000640 1 -- 172.21.15.86:0/2215349025 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a0041400 con 0x7f37b00b4030 2024-04-20T21:19:06.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.182+0000 7f37b4a00640 1 -- 172.21.15.86:0/2215349025 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37b00cc860 con 0x7f37b00b4030 2024-04-20T21:19:06.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.183+0000 7f37b4a00640 1 -- 172.21.15.86:0/2215349025 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37b00689e0 con 0x7f37b00b4030 2024-04-20T21:19:06.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.184+0000 7f37a6000640 1 -- 172.21.15.86:0/2215349025 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f37a0040070 con 0x7f37b00b4030 2024-04-20T21:19:06.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.185+0000 7f37a6000640 1 --2- 172.21.15.86:0/2215349025 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f378c06c920 0x7f378c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:06.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.185+0000 7f37a6000640 1 -- 172.21.15.86:0/2215349025 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f37a00adde0 con 0x7f37b00b4030 2024-04-20T21:19:06.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.185+0000 7f37ad400640 1 --2- 172.21.15.86:0/2215349025 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f378c06c920 0x7f378c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:06.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.186+0000 7f37ad400640 1 --2- 172.21.15.86:0/2215349025 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f378c06c920 0x7f378c06ede0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3798007fd0 tx=0x7f3798009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:06.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.190+0000 7f37a6000640 1 -- 172.21.15.86:0/2215349025 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f37a0033030 con 0x7f37b00b4030 2024-04-20T21:19:06.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.394+0000 7f37b4a00640 1 -- 172.21.15.86:0/2215349025 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f37b006dda0 con 0x7f37b00b4030 2024-04-20T21:19:06.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.396+0000 7f37a6000640 1 -- 172.21.15.86:0/2215349025 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+12891 (secure 0 0 0) 0x7f37a007c550 con 0x7f37b00b4030 2024-04-20T21:19:06.395 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:19:06.395 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:19:02.476469+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:19:06.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.398+0000 7f37b4a00640 1 -- 172.21.15.86:0/2215349025 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f378c06c920 msgr2=0x7f378c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:06.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.398+0000 7f37b4a00640 1 --2- 172.21.15.86:0/2215349025 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f378c06c920 0x7f378c06ede0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3798007fd0 tx=0x7f3798009040 comp rx=0 tx=0).stop 2024-04-20T21:19:06.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.398+0000 7f37b4a00640 1 -- 172.21.15.86:0/2215349025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37b00b4030 msgr2=0x7f37b0159200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:06.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.398+0000 7f37b4a00640 1 --2- 172.21.15.86:0/2215349025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37b00b4030 0x7f37b0159200 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f37a0000c00 tx=0x7f37a0004730 comp rx=0 tx=0).stop 2024-04-20T21:19:06.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.399+0000 7f37b4a00640 1 -- 172.21.15.86:0/2215349025 shutdown_connections 2024-04-20T21:19:06.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.399+0000 7f37b4a00640 1 --2- 172.21.15.86:0/2215349025 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f378c06c920 0x7f378c06ede0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:06.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.399+0000 7f37b4a00640 1 --2- 172.21.15.86:0/2215349025 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37b015eaf0 0x7f37b00c8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:06.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.399+0000 7f37b4a00640 1 --2- 172.21.15.86:0/2215349025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37b00d1ed0 0x7f37b00cd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:06.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.399+0000 7f37b4a00640 1 --2- 172.21.15.86:0/2215349025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37b00b4030 0x7f37b0159200 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:06.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.399+0000 7f37b4a00640 1 -- 172.21.15.86:0/2215349025 >> 172.21.15.86:0/2215349025 conn(0x7f37b00678a0 msgr2=0x7f37b015f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:06.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.399+0000 7f37b4a00640 1 -- 172.21.15.86:0/2215349025 shutdown_connections 2024-04-20T21:19:06.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.399+0000 7f37b4a00640 1 -- 172.21.15.86:0/2215349025 wait complete. 2024-04-20T21:19:06.418 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:19:06.418 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:19:06.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.624+0000 7fcc99600640 1 Processor -- start 2024-04-20T21:19:06.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.624+0000 7fcc99600640 1 -- start start 2024-04-20T21:19:06.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.626+0000 7fcc99600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc9415eb30 0x7fcc940b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:06.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.626+0000 7fcc99600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcc940aee50 0x7fcc940b4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:06.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.626+0000 7fcc99600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fcc9406b980 0x7fcc9406be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:06.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.626+0000 7fcc99600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fcc940af7d0 con 0x7fcc940aee50 2024-04-20T21:19:06.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.626+0000 7fcc99600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fcc9406c370 con 0x7fcc9406b980 2024-04-20T21:19:06.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.626+0000 7fcc99600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fcc940adb10 con 0x7fcc9415eb30 2024-04-20T21:19:06.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.626+0000 7fcc8a000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcc940aee50 0x7fcc940b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:06.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.626+0000 7fcc8a000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcc940aee50 0x7fcc940b4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42048/0 (socket says 172.21.15.86:42048) 2024-04-20T21:19:06.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.626+0000 7fcc8a000640 1 -- 172.21.15.86:0/2314441078 learned_addr learned my addr 172.21.15.86:0/2314441078 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:06.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.627+0000 7fcc8a000640 1 -- 172.21.15.86:0/2314441078 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc9415eb30 msgr2=0x7fcc940b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:06.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.627+0000 7fcc8a000640 1 --2- 172.21.15.86:0/2314441078 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc9415eb30 0x7fcc940b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:06.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.627+0000 7fcc8a000640 1 -- 172.21.15.86:0/2314441078 >> v2:172.21.15.144:3300/0 conn(0x7fcc9406b980 msgr2=0x7fcc9406be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:06.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.627+0000 7fcc8a000640 1 --2- 172.21.15.86:0/2314441078 >> v2:172.21.15.144:3300/0 conn(0x7fcc9406b980 0x7fcc9406be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:06.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.627+0000 7fcc8a000640 1 -- 172.21.15.86:0/2314441078 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc940addb0 con 0x7fcc940aee50 2024-04-20T21:19:06.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.628+0000 7fcc8a000640 1 --2- 172.21.15.86:0/2314441078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc940aee50 0x7fcc940b4360 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fcc940b0010 tx=0x7fcc7c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46ab32d1087ef280 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:06.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.628+0000 7fcc92000640 1 -- 172.21.15.86:0/2314441078 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c035020 con 0x7fcc940aee50 2024-04-20T21:19:06.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.628+0000 7fcc92000640 1 -- 172.21.15.86:0/2314441078 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc7c02d040 con 0x7fcc940aee50 2024-04-20T21:19:06.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.628+0000 7fcc92000640 1 -- 172.21.15.86:0/2314441078 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c031a80 con 0x7fcc940aee50 2024-04-20T21:19:06.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.629+0000 7fcc99600640 1 -- 172.21.15.86:0/2314441078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc940aee50 msgr2=0x7fcc940b4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:06.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.629+0000 7fcc99600640 1 --2- 172.21.15.86:0/2314441078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc940aee50 0x7fcc940b4360 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fcc940b0010 tx=0x7fcc7c02b900 comp rx=0 tx=0).stop 2024-04-20T21:19:06.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.629+0000 7fcc99600640 1 -- 172.21.15.86:0/2314441078 shutdown_connections 2024-04-20T21:19:06.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.629+0000 7fcc99600640 1 --2- 172.21.15.86:0/2314441078 >> v2:172.21.15.144:3300/0 conn(0x7fcc9406b980 0x7fcc9406be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:06.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.629+0000 7fcc99600640 1 --2- 172.21.15.86:0/2314441078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc940aee50 0x7fcc940b4360 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:06.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.629+0000 7fcc99600640 1 --2- 172.21.15.86:0/2314441078 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc9415eb30 0x7fcc940b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:06.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.629+0000 7fcc99600640 1 -- 172.21.15.86:0/2314441078 >> 172.21.15.86:0/2314441078 conn(0x7fcc940b2110 msgr2=0x7fcc940b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:06.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.629+0000 7fcc99600640 1 -- 172.21.15.86:0/2314441078 shutdown_connections 2024-04-20T21:19:06.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.629+0000 7fcc99600640 1 -- 172.21.15.86:0/2314441078 wait complete. 2024-04-20T21:19:06.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.631+0000 7fcc99600640 1 Processor -- start 2024-04-20T21:19:06.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.631+0000 7fcc99600640 1 -- start start 2024-04-20T21:19:06.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.632+0000 7fcc99600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc9406b980 0x7fcc94153a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:06.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.632+0000 7fcc99600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc940aee50 0x7fcc940cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:06.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.632+0000 7fcc99600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc9415eb30 0x7fcc9414bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:06.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.632+0000 7fcc99600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcc940ae650 con 0x7fcc9415eb30 2024-04-20T21:19:06.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.632+0000 7fcc99600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcc940ae370 con 0x7fcc940aee50 2024-04-20T21:19:06.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.632+0000 7fcc99600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fcc940ae930 con 0x7fcc9406b980 2024-04-20T21:19:06.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.632+0000 7fcc93400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc9415eb30 0x7fcc9414bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:06.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.632+0000 7fcc93400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc9415eb30 0x7fcc9414bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42050/0 (socket says 172.21.15.86:42050) 2024-04-20T21:19:06.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.632+0000 7fcc93400640 1 -- 172.21.15.86:0/283363668 learned_addr learned my addr 172.21.15.86:0/283363668 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:06.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.632+0000 7fcc93400640 1 -- 172.21.15.86:0/283363668 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc9406b980 msgr2=0x7fcc94153a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:06.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.632+0000 7fcc93400640 1 --2- 172.21.15.86:0/283363668 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc9406b980 0x7fcc94153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:06.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.632+0000 7fcc93400640 1 -- 172.21.15.86:0/283363668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc940aee50 msgr2=0x7fcc940cdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:06.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.632+0000 7fcc93400640 1 --2- 172.21.15.86:0/283363668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc940aee50 0x7fcc940cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:06.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.632+0000 7fcc93400640 1 -- 172.21.15.86:0/283363668 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc940ce400 con 0x7fcc9415eb30 2024-04-20T21:19:06.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.633+0000 7fcc93400640 1 --2- 172.21.15.86:0/283363668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc9415eb30 0x7fcc9414bdb0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c00b970 tx=0x7fcc8c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:06.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.633+0000 7fcc8b400640 1 -- 172.21.15.86:0/283363668 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c019070 con 0x7fcc9415eb30 2024-04-20T21:19:06.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.633+0000 7fcc8b400640 1 -- 172.21.15.86:0/283363668 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc8c00cb60 con 0x7fcc9415eb30 2024-04-20T21:19:06.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.633+0000 7fcc8b400640 1 -- 172.21.15.86:0/283363668 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c014440 con 0x7fcc9415eb30 2024-04-20T21:19:06.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.633+0000 7fcc99600640 1 -- 172.21.15.86:0/283363668 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc94154d00 con 0x7fcc9415eb30 2024-04-20T21:19:06.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.633+0000 7fcc99600640 1 -- 172.21.15.86:0/283363668 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc94154460 con 0x7fcc9415eb30 2024-04-20T21:19:06.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.635+0000 7fcc99600640 1 -- 172.21.15.86:0/283363668 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc940689e0 con 0x7fcc9415eb30 2024-04-20T21:19:06.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.636+0000 7fcc8b400640 1 -- 172.21.15.86:0/283363668 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcc8c01c070 con 0x7fcc9415eb30 2024-04-20T21:19:06.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.637+0000 7fcc8b400640 1 --2- 172.21.15.86:0/283363668 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcc7006c8f0 0x7fcc7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:06.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.637+0000 7fcc8b400640 1 -- 172.21.15.86:0/283363668 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fcc8c08b4b0 con 0x7fcc9415eb30 2024-04-20T21:19:06.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.637+0000 7fcc92a00640 1 --2- 172.21.15.86:0/283363668 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcc7006c8f0 0x7fcc7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:06.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.638+0000 7fcc92a00640 1 --2- 172.21.15.86:0/283363668 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcc7006c8f0 0x7fcc7006edb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcc84009710 tx=0x7fcc84006c90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:06.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.641+0000 7fcc8b400640 1 -- 172.21.15.86:0/283363668 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fcc8c059cc0 con 0x7fcc9415eb30 2024-04-20T21:19:06.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.841+0000 7fcc99600640 1 -- 172.21.15.86:0/283363668 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcc94159c40 con 0x7fcc9415eb30 2024-04-20T21:19:06.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.842+0000 7fcc8b400640 1 -- 172.21.15.86:0/283363668 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7fcc8c0595c0 con 0x7fcc9415eb30 2024-04-20T21:19:06.841 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:19:06.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.845+0000 7fcc99600640 1 -- 172.21.15.86:0/283363668 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcc7006c8f0 msgr2=0x7fcc7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:06.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.845+0000 7fcc99600640 1 --2- 172.21.15.86:0/283363668 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcc7006c8f0 0x7fcc7006edb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcc84009710 tx=0x7fcc84006c90 comp rx=0 tx=0).stop 2024-04-20T21:19:06.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.845+0000 7fcc99600640 1 -- 172.21.15.86:0/283363668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc9415eb30 msgr2=0x7fcc9414bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:06.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.845+0000 7fcc99600640 1 --2- 172.21.15.86:0/283363668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc9415eb30 0x7fcc9414bdb0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c00b970 tx=0x7fcc8c00be30 comp rx=0 tx=0).stop 2024-04-20T21:19:06.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.845+0000 7fcc99600640 1 -- 172.21.15.86:0/283363668 shutdown_connections 2024-04-20T21:19:06.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.845+0000 7fcc99600640 1 --2- 172.21.15.86:0/283363668 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcc7006c8f0 0x7fcc7006edb0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:06.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.845+0000 7fcc99600640 1 --2- 172.21.15.86:0/283363668 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc9415eb30 0x7fcc9414bdb0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:06.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.845+0000 7fcc99600640 1 --2- 172.21.15.86:0/283363668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc940aee50 0x7fcc940cdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:06.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.845+0000 7fcc99600640 1 --2- 172.21.15.86:0/283363668 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc9406b980 0x7fcc94153a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:06.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.845+0000 7fcc99600640 1 -- 172.21.15.86:0/283363668 >> 172.21.15.86:0/283363668 conn(0x7fcc940b2110 msgr2=0x7fcc9406e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:06.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.845+0000 7fcc99600640 1 -- 172.21.15.86:0/283363668 shutdown_connections 2024-04-20T21:19:06.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:06.845+0000 7fcc99600640 1 -- 172.21.15.86:0/283363668 wait complete. 2024-04-20T21:19:06.862 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-20T21:19:07.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.054+0000 7fadcfe00640 1 Processor -- start 2024-04-20T21:19:07.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.054+0000 7fadcfe00640 1 -- start start 2024-04-20T21:19:07.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.055+0000 7fadcfe00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fadd015d6a0 0x7fadd00ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:07.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.055+0000 7fadcfe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fadd00ad620 0x7fadd006e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:07.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.056+0000 7fadcfe00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadd0160890 con 0x7fadd0156ae0 2024-04-20T21:19:07.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.056+0000 7fadcfe00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fadd00ada90 con 0x7fadd015d6a0 2024-04-20T21:19:07.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.056+0000 7fadcfe00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fadd006db20 con 0x7fadd00ad620 2024-04-20T21:19:07.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.056+0000 7fadcf400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fadd00ad620 0x7fadd006e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:07.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.056+0000 7fadcf400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fadd00ad620 0x7fadd006e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:50622/0 (socket says 172.21.15.86:50622) 2024-04-20T21:19:07.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.056+0000 7fadcf400640 1 -- 172.21.15.86:0/3454705339 learned_addr learned my addr 172.21.15.86:0/3454705339 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:07.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.056+0000 7fadc7e00640 1 --2- 172.21.15.86:0/3454705339 >> v2:172.21.15.144:3300/0 conn(0x7fadd015d6a0 0x7fadd00ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:07.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.057+0000 7fadcf400640 1 -- 172.21.15.86:0/3454705339 >> v2:172.21.15.144:3300/0 conn(0x7fadd015d6a0 msgr2=0x7fadd00ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:07.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.057+0000 7fadcf400640 1 --2- 172.21.15.86:0/3454705339 >> v2:172.21.15.144:3300/0 conn(0x7fadd015d6a0 0x7fadd00ad0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:07.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.057+0000 7fadcf400640 1 -- 172.21.15.86:0/3454705339 >> v1:172.21.15.144:6789/0 conn(0x7fadd0156ae0 legacy=0x7fadd0160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:07.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.057+0000 7fadcf400640 1 -- 172.21.15.86:0/3454705339 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadd0160b70 con 0x7fadd00ad620 2024-04-20T21:19:07.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.058+0000 7fadcf400640 1 --2- 172.21.15.86:0/3454705339 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fadd00ad620 0x7fadd006e290 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fadc8009e50 tx=0x7fadc802bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=883f895fe5ae7bf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:07.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.059+0000 7fadce000640 1 -- 172.21.15.86:0/3454705339 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc802cb00 con 0x7fadd00ad620 2024-04-20T21:19:07.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.059+0000 7fadce000640 1 -- 172.21.15.86:0/3454705339 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadc802cca0 con 0x7fadd00ad620 2024-04-20T21:19:07.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.059+0000 7fadce000640 1 -- 172.21.15.86:0/3454705339 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc8034610 con 0x7fadd00ad620 2024-04-20T21:19:07.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.060+0000 7fadcfe00640 1 -- 172.21.15.86:0/3454705339 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fadd00ad620 msgr2=0x7fadd006e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:07.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.060+0000 7fadcfe00640 1 --2- 172.21.15.86:0/3454705339 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fadd00ad620 0x7fadd006e290 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fadc8009e50 tx=0x7fadc802bf60 comp rx=0 tx=0).stop 2024-04-20T21:19:07.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.060+0000 7fadcfe00640 1 -- 172.21.15.86:0/3454705339 shutdown_connections 2024-04-20T21:19:07.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.060+0000 7fadcfe00640 1 --2- 172.21.15.86:0/3454705339 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fadd00ad620 0x7fadd006e290 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:07.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.060+0000 7fadcfe00640 1 --2- 172.21.15.86:0/3454705339 >> v2:172.21.15.144:3300/0 conn(0x7fadd015d6a0 0x7fadd00ad0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:07.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.060+0000 7fadcfe00640 1 -- 172.21.15.86:0/3454705339 >> 172.21.15.86:0/3454705339 conn(0x7fadd00d0170 msgr2=0x7fadd00d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:07.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.060+0000 7fadcfe00640 1 -- 172.21.15.86:0/3454705339 shutdown_connections 2024-04-20T21:19:07.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.060+0000 7fadcfe00640 1 -- 172.21.15.86:0/3454705339 wait complete. 2024-04-20T21:19:07.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.062+0000 7fadcfe00640 1 Processor -- start 2024-04-20T21:19:07.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.062+0000 7fadcfe00640 1 -- start start 2024-04-20T21:19:07.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.063+0000 7fadcfe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fadd00ad620 0x7fadd0158b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:07.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.063+0000 7fadcfe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fadd0156ae0 0x7fadd0099830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:07.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.063+0000 7fadcfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fadd015d6a0 0x7fadd01e7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:07.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.063+0000 7fadcfe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fadd0156ec0 con 0x7fadd015d6a0 2024-04-20T21:19:07.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.063+0000 7fadcfe00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fadd006dd70 con 0x7fadd00ad620 2024-04-20T21:19:07.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.063+0000 7fadcfe00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fadd006e7d0 con 0x7fadd0156ae0 2024-04-20T21:19:07.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.063+0000 7fadcf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fadd015d6a0 0x7fadd01e7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:07.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.063+0000 7fadcf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fadd015d6a0 0x7fadd01e7a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42064/0 (socket says 172.21.15.86:42064) 2024-04-20T21:19:07.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.063+0000 7fadcf400640 1 -- 172.21.15.86:0/923238990 learned_addr learned my addr 172.21.15.86:0/923238990 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:07.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.063+0000 7fadc7e00640 1 --2- 172.21.15.86:0/923238990 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fadd0156ae0 0x7fadd0099830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:07.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.063+0000 7fadcea00640 1 --2- 172.21.15.86:0/923238990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fadd00ad620 0x7fadd0158b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:07.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.063+0000 7fadcf400640 1 -- 172.21.15.86:0/923238990 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fadd0156ae0 msgr2=0x7fadd0099830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:07.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.063+0000 7fadcf400640 1 --2- 172.21.15.86:0/923238990 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fadd0156ae0 0x7fadd0099830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:07.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.064+0000 7fadcf400640 1 -- 172.21.15.86:0/923238990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fadd00ad620 msgr2=0x7fadd0158b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:07.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.064+0000 7fadcf400640 1 --2- 172.21.15.86:0/923238990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fadd00ad620 0x7fadd0158b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:07.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.064+0000 7fadcf400640 1 -- 172.21.15.86:0/923238990 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadd0099d70 con 0x7fadd015d6a0 2024-04-20T21:19:07.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.064+0000 7fadcf400640 1 --2- 172.21.15.86:0/923238990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fadd015d6a0 0x7fadd01e7a00 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fadc8036040 tx=0x7fadc8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:07.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.064+0000 7fadc6a00640 1 -- 172.21.15.86:0/923238990 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc8043020 con 0x7fadd015d6a0 2024-04-20T21:19:07.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.064+0000 7fadc6a00640 1 -- 172.21.15.86:0/923238990 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadc802ce50 con 0x7fadd015d6a0 2024-04-20T21:19:07.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.064+0000 7fadc6a00640 1 -- 172.21.15.86:0/923238990 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc8009510 con 0x7fadd015d6a0 2024-04-20T21:19:07.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.064+0000 7fadcfe00640 1 -- 172.21.15.86:0/923238990 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadd01e8480 con 0x7fadd015d6a0 2024-04-20T21:19:07.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.064+0000 7fadcfe00640 1 -- 172.21.15.86:0/923238990 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadd0158040 con 0x7fadd015d6a0 2024-04-20T21:19:07.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.065+0000 7fadcfe00640 1 -- 172.21.15.86:0/923238990 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadd004fcf0 con 0x7fadd015d6a0 2024-04-20T21:19:07.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.066+0000 7fadc6a00640 1 -- 172.21.15.86:0/923238990 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fadc8046020 con 0x7fadd015d6a0 2024-04-20T21:19:07.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.067+0000 7fadc6a00640 1 --2- 172.21.15.86:0/923238990 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fadac06c8f0 0x7fadac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:07.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.067+0000 7fadc6a00640 1 -- 172.21.15.86:0/923238990 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fadc80adde0 con 0x7fadd015d6a0 2024-04-20T21:19:07.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.068+0000 7fadcea00640 1 --2- 172.21.15.86:0/923238990 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fadac06c8f0 0x7fadac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:07.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.068+0000 7fadcea00640 1 --2- 172.21.15.86:0/923238990 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fadac06c8f0 0x7fadac06edb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fadc0001620 tx=0x7fadc0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:07.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.072+0000 7fadc6a00640 1 -- 172.21.15.86:0/923238990 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fadc804d030 con 0x7fadd015d6a0 2024-04-20T21:19:07.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.278+0000 7fadcfe00640 1 -- 172.21.15.86:0/923238990 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fadd006fb40 con 0x7fadd015d6a0 2024-04-20T21:19:07.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.279+0000 7fadc6a00640 1 -- 172.21.15.86:0/923238990 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) ==== 84+0+65 (secure 0 0 0) 0x7fadc803c350 con 0x7fadd015d6a0 2024-04-20T21:19:07.278 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:19:07.278 INFO:teuthology.orchestra.run.smithi086.stdout:[ 2024-04-20T21:19:07.278 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:19:07.278 INFO:teuthology.orchestra.run.smithi086.stdout: "poolnum": 1, 2024-04-20T21:19:07.278 INFO:teuthology.orchestra.run.smithi086.stdout: "poolname": ".mgr" 2024-04-20T21:19:07.278 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:19:07.278 INFO:teuthology.orchestra.run.smithi086.stdout:] 2024-04-20T21:19:07.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.281+0000 7fadcfe00640 1 -- 172.21.15.86:0/923238990 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fadac06c8f0 msgr2=0x7fadac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:07.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.281+0000 7fadcfe00640 1 --2- 172.21.15.86:0/923238990 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fadac06c8f0 0x7fadac06edb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fadc0001620 tx=0x7fadc0009040 comp rx=0 tx=0).stop 2024-04-20T21:19:07.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.281+0000 7fadcfe00640 1 -- 172.21.15.86:0/923238990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fadd015d6a0 msgr2=0x7fadd01e7a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:07.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.281+0000 7fadcfe00640 1 --2- 172.21.15.86:0/923238990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fadd015d6a0 0x7fadd01e7a00 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fadc8036040 tx=0x7fadc8004a40 comp rx=0 tx=0).stop 2024-04-20T21:19:07.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.281+0000 7fadcfe00640 1 -- 172.21.15.86:0/923238990 shutdown_connections 2024-04-20T21:19:07.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.281+0000 7fadcfe00640 1 --2- 172.21.15.86:0/923238990 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fadac06c8f0 0x7fadac06edb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:07.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.281+0000 7fadcfe00640 1 --2- 172.21.15.86:0/923238990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fadd015d6a0 0x7fadd01e7a00 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:07.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.281+0000 7fadcfe00640 1 --2- 172.21.15.86:0/923238990 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fadd0156ae0 0x7fadd0099830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:07.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.281+0000 7fadcfe00640 1 --2- 172.21.15.86:0/923238990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fadd00ad620 0x7fadd0158b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:07.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.281+0000 7fadcfe00640 1 -- 172.21.15.86:0/923238990 >> 172.21.15.86:0/923238990 conn(0x7fadd00d0170 msgr2=0x7fadd00b3260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:07.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.281+0000 7fadcfe00640 1 -- 172.21.15.86:0/923238990 shutdown_connections 2024-04-20T21:19:07.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.281+0000 7fadcfe00640 1 -- 172.21.15.86:0/923238990 wait complete. 2024-04-20T21:19:07.301 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-20T21:19:07.301 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-20T21:19:07.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.501+0000 7fa328e00640 1 Processor -- start 2024-04-20T21:19:07.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.501+0000 7fa328e00640 1 -- start start 2024-04-20T21:19:07.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.503+0000 7fa328e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa32415cc70 0x7fa32415d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:07.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.503+0000 7fa328e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa32406e010 con 0x7fa324071d00 2024-04-20T21:19:07.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.503+0000 7fa328e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa32406e2f0 con 0x7fa32406f0b0 2024-04-20T21:19:07.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.503+0000 7fa328e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa32406e5d0 con 0x7fa32415cc70 2024-04-20T21:19:07.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.503+0000 7fa321e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fa324071d00 0x7fa3240720e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41304/0 (socket says 172.21.15.86:41304) 2024-04-20T21:19:07.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.503+0000 7fa321e00640 1 -- 172.21.15.86:0/823399269 learned_addr learned my addr 172.21.15.86:0/823399269 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:19:07.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.504+0000 7fa322800640 1 --2- 172.21.15.86:0/823399269 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa32415cc70 0x7fa32415d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:07.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.504+0000 7fa321400640 1 -- 172.21.15.86:0/823399269 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3605160521 0 0) 0x7fa32406e010 con 0x7fa324071d00 2024-04-20T21:19:07.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.504+0000 7fa321400640 1 -- 172.21.15.86:0/823399269 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa308003520 con 0x7fa324071d00 2024-04-20T21:19:07.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.504+0000 7fa321400640 1 -- 172.21.15.86:0/823399269 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2817035767 0 0) 0x7fa308003520 con 0x7fa324071d00 2024-04-20T21:19:07.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.504+0000 7fa321400640 1 -- 172.21.15.86:0/823399269 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa32415cc70 msgr2=0x7fa32415d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:07.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.504+0000 7fa321400640 1 --2- 172.21.15.86:0/823399269 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa32415cc70 0x7fa32415d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:07.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.504+0000 7fa321400640 1 -- 172.21.15.86:0/823399269 >> v1:172.21.15.144:6789/0 conn(0x7fa32406f0b0 legacy=0x7fa324145a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:07.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.504+0000 7fa321400640 1 -- 172.21.15.86:0/823399269 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa32406e8b0 con 0x7fa324071d00 2024-04-20T21:19:07.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.504+0000 7fa321400640 1 -- 172.21.15.86:0/823399269 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fa314001670 con 0x7fa324071d00 2024-04-20T21:19:07.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.504+0000 7fa321400640 1 -- 172.21.15.86:0/823399269 >> v1:172.21.15.86:6789/0 conn(0x7fa324071d00 legacy=0x7fa3240720e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:07.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.504+0000 7fa321400640 1 --2- 172.21.15.86:0/823399269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3080039e0 0x7fa308003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:07.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.504+0000 7fa321400640 1 --2- 172.21.15.86:0/823399269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa308004800 0x7fa308004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:07.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.504+0000 7fa321400640 1 --2- 172.21.15.86:0/823399269 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa32415cc70 0x7fa3080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:07.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.504+0000 7fa321400640 1 -- 172.21.15.86:0/823399269 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa32406e5d0 con 0x7fa3080039e0 2024-04-20T21:19:07.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.504+0000 7fa321400640 1 -- 172.21.15.86:0/823399269 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa30800baa0 con 0x7fa308004800 2024-04-20T21:19:07.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.505+0000 7fa321400640 1 -- 172.21.15.86:0/823399269 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa30800bca0 con 0x7fa32415cc70 2024-04-20T21:19:07.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.505+0000 7fa321e00640 1 --2- 172.21.15.86:0/823399269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3080039e0 0x7fa308003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:07.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.505+0000 7fa31be00640 1 --2- 172.21.15.86:0/823399269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa308004800 0x7fa308004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:07.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.505+0000 7fa321e00640 1 -- 172.21.15.86:0/823399269 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa32415cc70 msgr2=0x7fa3080051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:07.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.505+0000 7fa321e00640 1 --2- 172.21.15.86:0/823399269 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa32415cc70 0x7fa3080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:07.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.505+0000 7fa321e00640 1 -- 172.21.15.86:0/823399269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa308004800 msgr2=0x7fa308004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:07.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.505+0000 7fa321e00640 1 --2- 172.21.15.86:0/823399269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa308004800 0x7fa308004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:07.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.505+0000 7fa321e00640 1 -- 172.21.15.86:0/823399269 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa308003520 con 0x7fa3080039e0 2024-04-20T21:19:07.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.506+0000 7fa321e00640 1 --2- 172.21.15.86:0/823399269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3080039e0 0x7fa308003e00 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa314001eb0 tx=0x7fa308005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2a2bde3ca3a08de server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:07.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.506+0000 7fa321400640 1 -- 172.21.15.86:0/823399269 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa314007d30 con 0x7fa3080039e0 2024-04-20T21:19:07.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.507+0000 7fa321400640 1 -- 172.21.15.86:0/823399269 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa314030070 con 0x7fa3080039e0 2024-04-20T21:19:07.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.507+0000 7fa321400640 1 -- 172.21.15.86:0/823399269 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa314005d30 con 0x7fa3080039e0 2024-04-20T21:19:07.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.507+0000 7fa328e00640 1 -- 172.21.15.86:0/823399269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3080039e0 msgr2=0x7fa308003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:07.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.507+0000 7fa328e00640 1 --2- 172.21.15.86:0/823399269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3080039e0 0x7fa308003e00 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa314001eb0 tx=0x7fa308005830 comp rx=0 tx=0).stop 2024-04-20T21:19:07.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.507+0000 7fa322800640 1 -- 172.21.15.86:0/823399269 reap_dead start 2024-04-20T21:19:07.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.507+0000 7fa328e00640 1 -- 172.21.15.86:0/823399269 shutdown_connections 2024-04-20T21:19:07.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.507+0000 7fa328e00640 1 -- 172.21.15.86:0/823399269 >> 172.21.15.86:0/823399269 conn(0x7fa3240b2180 msgr2=0x7fa3240b2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:07.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.507+0000 7fa328e00640 1 -- 172.21.15.86:0/823399269 shutdown_connections 2024-04-20T21:19:07.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.507+0000 7fa328e00640 1 -- 172.21.15.86:0/823399269 wait complete. 2024-04-20T21:19:07.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.509+0000 7fa328e00640 1 Processor -- start 2024-04-20T21:19:07.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.509+0000 7fa328e00640 1 -- start start 2024-04-20T21:19:07.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.510+0000 7fa328e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3080039e0 0x7fa324139d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:07.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.510+0000 7fa328e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3241f0090 0x7fa3240b4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:07.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.510+0000 7fa328e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3240b49c0 0x7fa32413cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:07.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.510+0000 7fa328e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa3241461a0 con 0x7fa3080039e0 2024-04-20T21:19:07.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.510+0000 7fa328e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa32406f510 con 0x7fa3241f0090 2024-04-20T21:19:07.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.510+0000 7fa328e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa32406dc30 con 0x7fa3240b49c0 2024-04-20T21:19:07.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.510+0000 7fa321e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3080039e0 0x7fa324139d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:07.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.510+0000 7fa321e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3080039e0 0x7fa324139d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42086/0 (socket says 172.21.15.86:42086) 2024-04-20T21:19:07.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.510+0000 7fa321e00640 1 -- 172.21.15.86:0/2415984919 learned_addr learned my addr 172.21.15.86:0/2415984919 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:07.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.511+0000 7fa321e00640 1 -- 172.21.15.86:0/2415984919 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3240b49c0 msgr2=0x7fa32413cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:07.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.511+0000 7fa322800640 1 --2- 172.21.15.86:0/2415984919 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3240b49c0 0x7fa32413cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:07.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.511+0000 7fa321e00640 1 --2- 172.21.15.86:0/2415984919 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3240b49c0 0x7fa32413cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:07.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.511+0000 7fa321e00640 1 -- 172.21.15.86:0/2415984919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3241f0090 msgr2=0x7fa3240b4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:07.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.511+0000 7fa321e00640 1 --2- 172.21.15.86:0/2415984919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3241f0090 0x7fa3240b4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:07.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.511+0000 7fa321e00640 1 -- 172.21.15.86:0/2415984919 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa32413e0a0 con 0x7fa3080039e0 2024-04-20T21:19:07.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.511+0000 7fa321e00640 1 --2- 172.21.15.86:0/2415984919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3080039e0 0x7fa324139d50 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa31400c300 tx=0x7fa314001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:07.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.511+0000 7fa31a000640 1 -- 172.21.15.86:0/2415984919 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa314045070 con 0x7fa3080039e0 2024-04-20T21:19:07.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.511+0000 7fa31a000640 1 -- 172.21.15.86:0/2415984919 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa314000ed0 con 0x7fa3080039e0 2024-04-20T21:19:07.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.511+0000 7fa328e00640 1 -- 172.21.15.86:0/2415984919 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa32413e380 con 0x7fa3080039e0 2024-04-20T21:19:07.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.511+0000 7fa31a000640 1 -- 172.21.15.86:0/2415984919 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa314004df0 con 0x7fa3080039e0 2024-04-20T21:19:07.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.512+0000 7fa328e00640 1 -- 172.21.15.86:0/2415984919 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa32413e940 con 0x7fa3080039e0 2024-04-20T21:19:07.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.513+0000 7fa328e00640 1 -- 172.21.15.86:0/2415984919 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3240689e0 con 0x7fa3080039e0 2024-04-20T21:19:07.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.514+0000 7fa31a000640 1 -- 172.21.15.86:0/2415984919 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa314007070 con 0x7fa3080039e0 2024-04-20T21:19:07.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.515+0000 7fa31a000640 1 --2- 172.21.15.86:0/2415984919 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa30006c920 0x7fa30006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:07.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.515+0000 7fa31a000640 1 -- 172.21.15.86:0/2415984919 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fa3140acae0 con 0x7fa3080039e0 2024-04-20T21:19:07.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.515+0000 7fa31be00640 1 --2- 172.21.15.86:0/2415984919 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa30006c920 0x7fa30006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:07.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.516+0000 7fa31be00640 1 --2- 172.21.15.86:0/2415984919 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa30006c920 0x7fa30006ede0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa30c007fd0 tx=0x7fa30c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:07.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.520+0000 7fa31a000640 1 -- 172.21.15.86:0/2415984919 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa31407b350 con 0x7fa3080039e0 2024-04-20T21:19:07.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:07.718+0000 7fa328e00640 1 -- 172.21.15.86:0/2415984919 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fa324145850 con 0x7fa3080039e0 2024-04-20T21:19:08.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.504+0000 7fa31a000640 1 -- 172.21.15.86:0/2415984919 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v30) ==== 138+0+0 (secure 0 0 0) 0x7fa31407ac50 con 0x7fa3080039e0 2024-04-20T21:19:08.504 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' created 2024-04-20T21:19:08.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.508+0000 7fa328e00640 1 -- 172.21.15.86:0/2415984919 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa30006c920 msgr2=0x7fa30006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:08.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.508+0000 7fa328e00640 1 --2- 172.21.15.86:0/2415984919 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa30006c920 0x7fa30006ede0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa30c007fd0 tx=0x7fa30c009040 comp rx=0 tx=0).stop 2024-04-20T21:19:08.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.508+0000 7fa328e00640 1 -- 172.21.15.86:0/2415984919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3080039e0 msgr2=0x7fa324139d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:08.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.508+0000 7fa328e00640 1 --2- 172.21.15.86:0/2415984919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3080039e0 0x7fa324139d50 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa31400c300 tx=0x7fa314001a40 comp rx=0 tx=0).stop 2024-04-20T21:19:08.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.508+0000 7fa328e00640 1 -- 172.21.15.86:0/2415984919 shutdown_connections 2024-04-20T21:19:08.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.508+0000 7fa328e00640 1 --2- 172.21.15.86:0/2415984919 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa30006c920 0x7fa30006ede0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:08.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.508+0000 7fa328e00640 1 --2- 172.21.15.86:0/2415984919 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa3240b49c0 0x7fa32413cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:08.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.508+0000 7fa328e00640 1 --2- 172.21.15.86:0/2415984919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3241f0090 0x7fa3240b4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:08.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.508+0000 7fa328e00640 1 --2- 172.21.15.86:0/2415984919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3080039e0 0x7fa324139d50 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:08.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.508+0000 7fa328e00640 1 -- 172.21.15.86:0/2415984919 >> 172.21.15.86:0/2415984919 conn(0x7fa3240b2180 msgr2=0x7fa3240720e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:08.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.509+0000 7fa328e00640 1 -- 172.21.15.86:0/2415984919 shutdown_connections 2024-04-20T21:19:08.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.509+0000 7fa328e00640 1 -- 172.21.15.86:0/2415984919 wait complete. 2024-04-20T21:19:08.534 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-20T21:19:08.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.729+0000 7f6defe00640 1 Processor -- start 2024-04-20T21:19:08.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.729+0000 7f6defe00640 1 -- start start 2024-04-20T21:19:08.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.731+0000 7f6defe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6df00d45a0 0x7f6df00d3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:08.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.731+0000 7f6defe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6df015ea70 0x7f6df00d4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:08.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.731+0000 7f6defe00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6df006d570 0x7f6df00d1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:08.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.731+0000 7f6defe00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f6df00cf120 con 0x7f6df00d45a0 2024-04-20T21:19:08.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.731+0000 7f6defe00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f6df006de70 con 0x7f6df006d570 2024-04-20T21:19:08.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.731+0000 7f6defe00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6df006e150 con 0x7f6df015ea70 2024-04-20T21:19:08.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.731+0000 7f6deea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6df00d45a0 0x7f6df00d3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:08.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.731+0000 7f6deea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6df00d45a0 0x7f6df00d3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42094/0 (socket says 172.21.15.86:42094) 2024-04-20T21:19:08.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.731+0000 7f6deea00640 1 -- 172.21.15.86:0/2760479327 learned_addr learned my addr 172.21.15.86:0/2760479327 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:08.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.732+0000 7f6deea00640 1 -- 172.21.15.86:0/2760479327 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6df015ea70 msgr2=0x7f6df00d4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:08.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.732+0000 7f6deea00640 1 --2- 172.21.15.86:0/2760479327 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6df015ea70 0x7f6df00d4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:08.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.732+0000 7f6deea00640 1 -- 172.21.15.86:0/2760479327 >> v2:172.21.15.144:3300/0 conn(0x7f6df006d570 msgr2=0x7f6df00d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:08.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.732+0000 7f6deea00640 1 --2- 172.21.15.86:0/2760479327 >> v2:172.21.15.144:3300/0 conn(0x7f6df006d570 0x7f6df00d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:08.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.732+0000 7f6deea00640 1 -- 172.21.15.86:0/2760479327 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6df006e430 con 0x7f6df00d45a0 2024-04-20T21:19:08.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.733+0000 7f6deea00640 1 --2- 172.21.15.86:0/2760479327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6df00d45a0 0x7f6df00d3260 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6df00ceef0 tx=0x7f6de002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=905aba95a7b6cf54 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:08.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.733+0000 7f6ded600640 1 -- 172.21.15.86:0/2760479327 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de002c9f0 con 0x7f6df00d45a0 2024-04-20T21:19:08.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.733+0000 7f6ded600640 1 -- 172.21.15.86:0/2760479327 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6de002cb90 con 0x7f6df00d45a0 2024-04-20T21:19:08.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.733+0000 7f6ded600640 1 -- 172.21.15.86:0/2760479327 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de00343f0 con 0x7f6df00d45a0 2024-04-20T21:19:08.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.734+0000 7f6defe00640 1 -- 172.21.15.86:0/2760479327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6df00d45a0 msgr2=0x7f6df00d3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:08.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.734+0000 7f6defe00640 1 --2- 172.21.15.86:0/2760479327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6df00d45a0 0x7f6df00d3260 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6df00ceef0 tx=0x7f6de002be50 comp rx=0 tx=0).stop 2024-04-20T21:19:08.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.734+0000 7f6defe00640 1 -- 172.21.15.86:0/2760479327 shutdown_connections 2024-04-20T21:19:08.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.734+0000 7f6defe00640 1 --2- 172.21.15.86:0/2760479327 >> v2:172.21.15.144:3300/0 conn(0x7f6df006d570 0x7f6df00d1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:08.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.734+0000 7f6defe00640 1 --2- 172.21.15.86:0/2760479327 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6df015ea70 0x7f6df00d4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:08.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.734+0000 7f6defe00640 1 --2- 172.21.15.86:0/2760479327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6df00d45a0 0x7f6df00d3260 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:08.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.734+0000 7f6defe00640 1 -- 172.21.15.86:0/2760479327 >> 172.21.15.86:0/2760479327 conn(0x7f6df009d9d0 msgr2=0x7f6df008bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:08.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.735+0000 7f6defe00640 1 -- 172.21.15.86:0/2760479327 shutdown_connections 2024-04-20T21:19:08.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.735+0000 7f6defe00640 1 -- 172.21.15.86:0/2760479327 wait complete. 2024-04-20T21:19:08.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.736+0000 7f6defe00640 1 Processor -- start 2024-04-20T21:19:08.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.736+0000 7f6defe00640 1 -- start start 2024-04-20T21:19:08.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.737+0000 7f6defe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6df006d570 0x7f6df015a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:08.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.737+0000 7f6defe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6df00d45a0 0x7f6df015c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:08.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.737+0000 7f6defe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6df015ea70 0x7f6df01f4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:08.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.737+0000 7f6defe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6df006da00 con 0x7f6df00d45a0 2024-04-20T21:19:08.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.737+0000 7f6defe00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6df00d4ec0 con 0x7f6df006d570 2024-04-20T21:19:08.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.737+0000 7f6defe00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6df006db80 con 0x7f6df015ea70 2024-04-20T21:19:08.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.738+0000 7f6dee000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6df00d45a0 0x7f6df015c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:08.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.738+0000 7f6dee000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6df00d45a0 0x7f6df015c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42108/0 (socket says 172.21.15.86:42108) 2024-04-20T21:19:08.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.738+0000 7f6dee000640 1 -- 172.21.15.86:0/3676683084 learned_addr learned my addr 172.21.15.86:0/3676683084 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:08.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.738+0000 7f6def400640 1 --2- 172.21.15.86:0/3676683084 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6df015ea70 0x7f6df01f4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:08.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.738+0000 7f6dee000640 1 -- 172.21.15.86:0/3676683084 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6df015ea70 msgr2=0x7f6df01f4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:08.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.738+0000 7f6dee000640 1 --2- 172.21.15.86:0/3676683084 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6df015ea70 0x7f6df01f4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:08.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.738+0000 7f6dee000640 1 -- 172.21.15.86:0/3676683084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6df006d570 msgr2=0x7f6df015a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:08.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.738+0000 7f6deea00640 1 --2- 172.21.15.86:0/3676683084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6df006d570 0x7f6df015a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:08.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.738+0000 7f6dee000640 1 --2- 172.21.15.86:0/3676683084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6df006d570 0x7f6df015a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:08.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.738+0000 7f6dee000640 1 -- 172.21.15.86:0/3676683084 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6df00c8720 con 0x7f6df00d45a0 2024-04-20T21:19:08.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.739+0000 7f6dee000640 1 --2- 172.21.15.86:0/3676683084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6df00d45a0 0x7f6df015c550 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6dd800b410 tx=0x7f6dd800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:08.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.739+0000 7f6de6a00640 1 -- 172.21.15.86:0/3676683084 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd8015020 con 0x7f6df00d45a0 2024-04-20T21:19:08.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.739+0000 7f6de6a00640 1 -- 172.21.15.86:0/3676683084 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dd8011850 con 0x7f6df00d45a0 2024-04-20T21:19:08.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.739+0000 7f6de6a00640 1 -- 172.21.15.86:0/3676683084 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd8011c40 con 0x7f6df00d45a0 2024-04-20T21:19:08.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.739+0000 7f6defe00640 1 -- 172.21.15.86:0/3676683084 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6df00c8a00 con 0x7f6df00d45a0 2024-04-20T21:19:08.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.739+0000 7f6defe00640 1 -- 172.21.15.86:0/3676683084 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6df00c8fc0 con 0x7f6df00d45a0 2024-04-20T21:19:08.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.740+0000 7f6defe00640 1 -- 172.21.15.86:0/3676683084 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6df004fd90 con 0x7f6df00d45a0 2024-04-20T21:19:08.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.741+0000 7f6de6a00640 1 -- 172.21.15.86:0/3676683084 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6dd8018020 con 0x7f6df00d45a0 2024-04-20T21:19:08.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.742+0000 7f6de6a00640 1 --2- 172.21.15.86:0/3676683084 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6dcc06c8f0 0x7f6dcc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:08.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.742+0000 7f6de6a00640 1 -- 172.21.15.86:0/3676683084 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5227+0+0 (secure 0 0 0) 0x7f6dd808b7c0 con 0x7f6df00d45a0 2024-04-20T21:19:08.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.743+0000 7f6deea00640 1 --2- 172.21.15.86:0/3676683084 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6dcc06c8f0 0x7f6dcc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:08.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.743+0000 7f6deea00640 1 --2- 172.21.15.86:0/3676683084 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6dcc06c8f0 0x7f6dcc06edb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6de0035000 tx=0x7f6de0030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:08.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.748+0000 7f6de6a00640 1 -- 172.21.15.86:0/3676683084 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6dd801f020 con 0x7f6df00d45a0 2024-04-20T21:19:08.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:08.947+0000 7f6defe00640 1 -- 172.21.15.86:0/3676683084 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f6df015e100 con 0x7f6df00d45a0 2024-04-20T21:19:09.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.511+0000 7f6de6a00640 1 -- 172.21.15.86:0/3676683084 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v31) ==== 170+0+0 (secure 0 0 0) 0x7f6dd80172c0 con 0x7f6df00d45a0 2024-04-20T21:19:09.511 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' created 2024-04-20T21:19:09.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.515+0000 7f6defe00640 1 -- 172.21.15.86:0/3676683084 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6dcc06c8f0 msgr2=0x7f6dcc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:09.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.515+0000 7f6defe00640 1 --2- 172.21.15.86:0/3676683084 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6dcc06c8f0 0x7f6dcc06edb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6de0035000 tx=0x7f6de0030000 comp rx=0 tx=0).stop 2024-04-20T21:19:09.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.515+0000 7f6defe00640 1 -- 172.21.15.86:0/3676683084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6df00d45a0 msgr2=0x7f6df015c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:09.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.515+0000 7f6defe00640 1 --2- 172.21.15.86:0/3676683084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6df00d45a0 0x7f6df015c550 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6dd800b410 tx=0x7f6dd800b8d0 comp rx=0 tx=0).stop 2024-04-20T21:19:09.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.515+0000 7f6defe00640 1 -- 172.21.15.86:0/3676683084 shutdown_connections 2024-04-20T21:19:09.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.515+0000 7f6defe00640 1 --2- 172.21.15.86:0/3676683084 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6dcc06c8f0 0x7f6dcc06edb0 secure :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6de0035000 tx=0x7f6de0030000 comp rx=0 tx=0).stop 2024-04-20T21:19:09.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.515+0000 7f6defe00640 1 --2- 172.21.15.86:0/3676683084 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6df015ea70 0x7f6df01f4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:09.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.515+0000 7f6defe00640 1 --2- 172.21.15.86:0/3676683084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6df00d45a0 0x7f6df015c550 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:09.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.515+0000 7f6defe00640 1 --2- 172.21.15.86:0/3676683084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6df006d570 0x7f6df015a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:09.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.515+0000 7f6defe00640 1 -- 172.21.15.86:0/3676683084 >> 172.21.15.86:0/3676683084 conn(0x7f6df009d9d0 msgr2=0x7f6df008bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:09.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.516+0000 7f6defe00640 1 -- 172.21.15.86:0/3676683084 shutdown_connections 2024-04-20T21:19:09.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.516+0000 7f6defe00640 1 -- 172.21.15.86:0/3676683084 wait complete. 2024-04-20T21:19:09.550 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-20T21:19:09.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.725+0000 7fac18a00640 1 Processor -- start 2024-04-20T21:19:09.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.725+0000 7fac18a00640 1 -- start start 2024-04-20T21:19:09.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.727+0000 7fac18a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac14146210 0x7fac1406e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:09.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.727+0000 7fac18a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac1415cd40 con 0x7fac1406bbf0 2024-04-20T21:19:09.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.727+0000 7fac18a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac1415d000 con 0x7fac1406bfd0 2024-04-20T21:19:09.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.727+0000 7fac18a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fac1415d2e0 con 0x7fac14146210 2024-04-20T21:19:09.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.727+0000 7fac11e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fac1406bbf0 0x7fac14145b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41310/0 (socket says 172.21.15.86:41310) 2024-04-20T21:19:09.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.727+0000 7fac11e00640 1 -- 172.21.15.86:0/1865049479 learned_addr learned my addr 172.21.15.86:0/1865049479 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:19:09.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.727+0000 7fac09400640 1 --2- 172.21.15.86:0/1865049479 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac14146210 0x7fac1406e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:09.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.728+0000 7fac11400640 1 -- 172.21.15.86:0/1865049479 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1601474674 0 0) 0x7fac1415cd40 con 0x7fac1406bbf0 2024-04-20T21:19:09.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.728+0000 7fac11400640 1 -- 172.21.15.86:0/1865049479 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fabf8003520 con 0x7fac1406bbf0 2024-04-20T21:19:09.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.728+0000 7fac11400640 1 -- 172.21.15.86:0/1865049479 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3089201641 0 0) 0x7fabf8003520 con 0x7fac1406bbf0 2024-04-20T21:19:09.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.728+0000 7fac11400640 1 -- 172.21.15.86:0/1865049479 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac14146210 msgr2=0x7fac1406e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:09.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.728+0000 7fac11400640 1 --2- 172.21.15.86:0/1865049479 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac14146210 0x7fac1406e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:09.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.728+0000 7fac11400640 1 -- 172.21.15.86:0/1865049479 >> v1:172.21.15.144:6789/0 conn(0x7fac1406bfd0 legacy=0x7fac140cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:09.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.728+0000 7fac11400640 1 -- 172.21.15.86:0/1865049479 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac1415d5c0 con 0x7fac1406bbf0 2024-04-20T21:19:09.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.728+0000 7fac11400640 1 -- 172.21.15.86:0/1865049479 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fac04001670 con 0x7fac1406bbf0 2024-04-20T21:19:09.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.728+0000 7fac11400640 1 -- 172.21.15.86:0/1865049479 >> v1:172.21.15.86:6789/0 conn(0x7fac1406bbf0 legacy=0x7fac14145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:09.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.728+0000 7fac11400640 1 --2- 172.21.15.86:0/1865049479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabf80039e0 0x7fabf8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:09.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.728+0000 7fac11400640 1 --2- 172.21.15.86:0/1865049479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabf8004800 0x7fabf8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:09.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.728+0000 7fac11400640 1 --2- 172.21.15.86:0/1865049479 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac14146210 0x7fabf80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:09.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.728+0000 7fac11400640 1 -- 172.21.15.86:0/1865049479 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fac1415d2e0 con 0x7fabf80039e0 2024-04-20T21:19:09.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.728+0000 7fac11400640 1 -- 172.21.15.86:0/1865049479 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fabf800ba70 con 0x7fabf8004800 2024-04-20T21:19:09.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.728+0000 7fac11400640 1 -- 172.21.15.86:0/1865049479 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fabf800bc70 con 0x7fac14146210 2024-04-20T21:19:09.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.729+0000 7fac11e00640 1 --2- 172.21.15.86:0/1865049479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabf80039e0 0x7fabf8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:09.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.729+0000 7fac12800640 1 --2- 172.21.15.86:0/1865049479 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac14146210 0x7fabf80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:09.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.729+0000 7fac09400640 1 --2- 172.21.15.86:0/1865049479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabf8004800 0x7fabf8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:09.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.729+0000 7fac11e00640 1 -- 172.21.15.86:0/1865049479 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac14146210 msgr2=0x7fabf80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:09.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.729+0000 7fac11e00640 1 --2- 172.21.15.86:0/1865049479 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac14146210 0x7fabf80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:09.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.729+0000 7fac11e00640 1 -- 172.21.15.86:0/1865049479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabf8004800 msgr2=0x7fabf8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:09.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.729+0000 7fac11e00640 1 --2- 172.21.15.86:0/1865049479 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fabf8004800 0x7fabf8004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:09.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.729+0000 7fac11e00640 1 -- 172.21.15.86:0/1865049479 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabf8003520 con 0x7fabf80039e0 2024-04-20T21:19:09.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.729+0000 7fac12800640 1 --2- 172.21.15.86:0/1865049479 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac14146210 0x7fabf80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:09.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.730+0000 7fac11e00640 1 --2- 172.21.15.86:0/1865049479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabf80039e0 0x7fabf8003e00 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fac0400c210 tx=0x7fabf8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c276ff9f1b3fcdb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:09.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.730+0000 7fac11400640 1 -- 172.21.15.86:0/1865049479 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac04007cf0 con 0x7fabf80039e0 2024-04-20T21:19:09.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.730+0000 7fac11400640 1 -- 172.21.15.86:0/1865049479 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac04030070 con 0x7fabf80039e0 2024-04-20T21:19:09.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.730+0000 7fac11400640 1 -- 172.21.15.86:0/1865049479 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac04005d30 con 0x7fabf80039e0 2024-04-20T21:19:09.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.731+0000 7fac18a00640 1 -- 172.21.15.86:0/1865049479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabf80039e0 msgr2=0x7fabf8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:09.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.731+0000 7fac18a00640 1 --2- 172.21.15.86:0/1865049479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabf80039e0 0x7fabf8003e00 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fac0400c210 tx=0x7fabf8005830 comp rx=0 tx=0).stop 2024-04-20T21:19:09.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.731+0000 7fac12800640 1 -- 172.21.15.86:0/1865049479 reap_dead start 2024-04-20T21:19:09.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.731+0000 7fac18a00640 1 -- 172.21.15.86:0/1865049479 shutdown_connections 2024-04-20T21:19:09.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.731+0000 7fac18a00640 1 -- 172.21.15.86:0/1865049479 >> 172.21.15.86:0/1865049479 conn(0x7fac140b2110 msgr2=0x7fac140b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:09.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.731+0000 7fac18a00640 1 -- 172.21.15.86:0/1865049479 shutdown_connections 2024-04-20T21:19:09.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.731+0000 7fac18a00640 1 -- 172.21.15.86:0/1865049479 wait complete. 2024-04-20T21:19:09.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.733+0000 7fac18a00640 1 Processor -- start 2024-04-20T21:19:09.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.733+0000 7fac18a00640 1 -- start start 2024-04-20T21:19:09.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.734+0000 7fac18a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabf80039e0 0x7fac140aee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:09.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.734+0000 7fac18a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac1415c0a0 0x7fac1413ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:09.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.734+0000 7fac18a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac1415f310 0x7fac1415f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:09.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.734+0000 7fac18a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fac140cddd0 con 0x7fabf80039e0 2024-04-20T21:19:09.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.734+0000 7fac18a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fac140cdc50 con 0x7fac1415c0a0 2024-04-20T21:19:09.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.734+0000 7fac18a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fac140cdf50 con 0x7fac1415f310 2024-04-20T21:19:09.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.734+0000 7fac11e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabf80039e0 0x7fac140aee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:09.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.734+0000 7fac11e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabf80039e0 0x7fac140aee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42130/0 (socket says 172.21.15.86:42130) 2024-04-20T21:19:09.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.734+0000 7fac11e00640 1 -- 172.21.15.86:0/2004505563 learned_addr learned my addr 172.21.15.86:0/2004505563 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:09.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.735+0000 7fac12800640 1 --2- 172.21.15.86:0/2004505563 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac1415f310 0x7fac1415f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:09.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.735+0000 7fac09400640 1 --2- 172.21.15.86:0/2004505563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac1415c0a0 0x7fac1413ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:09.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.735+0000 7fac11e00640 1 -- 172.21.15.86:0/2004505563 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac1415f310 msgr2=0x7fac1415f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:09.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.735+0000 7fac11e00640 1 --2- 172.21.15.86:0/2004505563 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac1415f310 0x7fac1415f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:09.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.735+0000 7fac11e00640 1 -- 172.21.15.86:0/2004505563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac1415c0a0 msgr2=0x7fac1413ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:09.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.735+0000 7fac11e00640 1 --2- 172.21.15.86:0/2004505563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac1415c0a0 0x7fac1413ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:09.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.735+0000 7fac11e00640 1 -- 172.21.15.86:0/2004505563 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac140af3c0 con 0x7fabf80039e0 2024-04-20T21:19:09.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.735+0000 7fac11e00640 1 --2- 172.21.15.86:0/2004505563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabf80039e0 0x7fac140aee80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fac0400c330 tx=0x7fac1415d1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:09.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.736+0000 7fac0aa00640 1 -- 172.21.15.86:0/2004505563 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac04001c30 con 0x7fabf80039e0 2024-04-20T21:19:09.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.736+0000 7fac0aa00640 1 -- 172.21.15.86:0/2004505563 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac04004bf0 con 0x7fabf80039e0 2024-04-20T21:19:09.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.736+0000 7fac0aa00640 1 -- 172.21.15.86:0/2004505563 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac04007070 con 0x7fabf80039e0 2024-04-20T21:19:09.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.736+0000 7fac18a00640 1 -- 172.21.15.86:0/2004505563 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac140b4d90 con 0x7fabf80039e0 2024-04-20T21:19:09.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.736+0000 7fac18a00640 1 -- 172.21.15.86:0/2004505563 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac140b5310 con 0x7fabf80039e0 2024-04-20T21:19:09.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.737+0000 7fac18a00640 1 -- 172.21.15.86:0/2004505563 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac140b5510 con 0x7fabf80039e0 2024-04-20T21:19:09.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.739+0000 7fac0aa00640 1 -- 172.21.15.86:0/2004505563 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fac04007210 con 0x7fabf80039e0 2024-04-20T21:19:09.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.740+0000 7fac0aa00640 1 --2- 172.21.15.86:0/2004505563 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fabf006c8f0 0x7fabf006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:09.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.740+0000 7fac0aa00640 1 -- 172.21.15.86:0/2004505563 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5609+0+0 (secure 0 0 0) 0x7fac040ad490 con 0x7fabf80039e0 2024-04-20T21:19:09.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.740+0000 7fac09400640 1 --2- 172.21.15.86:0/2004505563 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fabf006c8f0 0x7fabf006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:09.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.741+0000 7fac09400640 1 --2- 172.21.15.86:0/2004505563 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fabf006c8f0 0x7fabf006edb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fac141f7ad0 tx=0x7fabfc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:09.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.744+0000 7fac0aa00640 1 -- 172.21.15.86:0/2004505563 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fac140b5510 con 0x7fabf80039e0 2024-04-20T21:19:09.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:09.977+0000 7fac18a00640 1 -- 172.21.15.86:0/2004505563 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fac140689e0 con 0x7fabf80039e0 2024-04-20T21:19:10.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.538+0000 7fac0aa00640 1 -- 172.21.15.86:0/2004505563 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v12) ==== 172+0+0 (secure 0 0 0) 0x7fac140af3c0 con 0x7fabf80039e0 2024-04-20T21:19:10.538 INFO:teuthology.orchestra.run.smithi086.stderr:new fs with metadata pool 4 and data pool 5 2024-04-20T21:19:10.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.543+0000 7fac18a00640 1 -- 172.21.15.86:0/2004505563 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fabf006c8f0 msgr2=0x7fabf006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:10.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.543+0000 7fac18a00640 1 --2- 172.21.15.86:0/2004505563 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fabf006c8f0 0x7fabf006edb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fac141f7ad0 tx=0x7fabfc008040 comp rx=0 tx=0).stop 2024-04-20T21:19:10.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.543+0000 7fac18a00640 1 -- 172.21.15.86:0/2004505563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabf80039e0 msgr2=0x7fac140aee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:10.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.543+0000 7fac18a00640 1 --2- 172.21.15.86:0/2004505563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabf80039e0 0x7fac140aee80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fac0400c330 tx=0x7fac1415d1d0 comp rx=0 tx=0).stop 2024-04-20T21:19:10.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.546+0000 7fac18a00640 1 -- 172.21.15.86:0/2004505563 shutdown_connections 2024-04-20T21:19:10.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.546+0000 7fac18a00640 1 --2- 172.21.15.86:0/2004505563 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fabf006c8f0 0x7fabf006edb0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:10.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.546+0000 7fac18a00640 1 --2- 172.21.15.86:0/2004505563 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fac1415f310 0x7fac1415f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:10.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.546+0000 7fac18a00640 1 --2- 172.21.15.86:0/2004505563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac1415c0a0 0x7fac1413ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:10.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.546+0000 7fac18a00640 1 --2- 172.21.15.86:0/2004505563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabf80039e0 0x7fac140aee80 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:10.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.546+0000 7fac18a00640 1 -- 172.21.15.86:0/2004505563 >> 172.21.15.86:0/2004505563 conn(0x7fac140b2110 msgr2=0x7fac140b0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:10.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.547+0000 7fac18a00640 1 -- 172.21.15.86:0/2004505563 shutdown_connections 2024-04-20T21:19:10.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.549+0000 7fac18a00640 1 -- 172.21.15.86:0/2004505563 wait complete. 2024-04-20T21:19:10.595 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:19:10.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.758+0000 7f857d800640 1 Processor -- start 2024-04-20T21:19:10.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.758+0000 7f857d800640 1 -- start start 2024-04-20T21:19:10.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.760+0000 7f857d800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8578146130 0x7f857806e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:10.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.760+0000 7f857d800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f857806dc30 con 0x7f8578072410 2024-04-20T21:19:10.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.760+0000 7f857d800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85780cd8e0 con 0x7f85780ccdd0 2024-04-20T21:19:10.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.760+0000 7f857d800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f85780cdbc0 con 0x7f8578146130 2024-04-20T21:19:10.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.760+0000 7f8576a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8578146130 0x7f857806e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:10.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.760+0000 7f8577400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f8578072410 0x7f857815cc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41320/0 (socket says 172.21.15.86:41320) 2024-04-20T21:19:10.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.760+0000 7f8577400640 1 -- 172.21.15.86:0/4070182091 learned_addr learned my addr 172.21.15.86:0/4070182091 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:19:10.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.760+0000 7f8576a00640 1 -- 172.21.15.86:0/4070182091 >> v1:172.21.15.144:6789/0 conn(0x7f85780ccdd0 legacy=0x7f8578071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:10.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.760+0000 7f8576a00640 1 -- 172.21.15.86:0/4070182091 >> v1:172.21.15.86:6789/0 conn(0x7f8578072410 legacy=0x7f857815cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:10.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.760+0000 7f8576a00640 1 -- 172.21.15.86:0/4070182091 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f857806ded0 con 0x7f8578146130 2024-04-20T21:19:10.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.762+0000 7f8576a00640 1 --2- 172.21.15.86:0/4070182091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8578146130 0x7f857806e6d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8568009d90 tx=0x7f856802bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14d632fba201c639 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:10.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.762+0000 7f8575600640 1 -- 172.21.15.86:0/4070182091 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8568039070 con 0x7f8578146130 2024-04-20T21:19:10.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.762+0000 7f8575600640 1 -- 172.21.15.86:0/4070182091 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f856802ccf0 con 0x7f8578146130 2024-04-20T21:19:10.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.762+0000 7f8575600640 1 -- 172.21.15.86:0/4070182091 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8568034660 con 0x7f8578146130 2024-04-20T21:19:10.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.762+0000 7f857d800640 1 -- 172.21.15.86:0/4070182091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8578146130 msgr2=0x7f857806e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:10.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.762+0000 7f857d800640 1 --2- 172.21.15.86:0/4070182091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8578146130 0x7f857806e6d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8568009d90 tx=0x7f856802bf40 comp rx=0 tx=0).stop 2024-04-20T21:19:10.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.763+0000 7f857d800640 1 -- 172.21.15.86:0/4070182091 shutdown_connections 2024-04-20T21:19:10.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.763+0000 7f857d800640 1 --2- 172.21.15.86:0/4070182091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8578146130 0x7f857806e6d0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:10.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.763+0000 7f857d800640 1 -- 172.21.15.86:0/4070182091 >> 172.21.15.86:0/4070182091 conn(0x7f85780b2360 msgr2=0x7f85780b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:10.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.763+0000 7f857d800640 1 -- 172.21.15.86:0/4070182091 shutdown_connections 2024-04-20T21:19:10.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.763+0000 7f857d800640 1 -- 172.21.15.86:0/4070182091 wait complete. 2024-04-20T21:19:10.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.764+0000 7f857d800640 1 Processor -- start 2024-04-20T21:19:10.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.764+0000 7f857d800640 1 -- start start 2024-04-20T21:19:10.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.765+0000 7f857d800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8578072410 0x7f85781303a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:10.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.765+0000 7f857d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85780ccdd0 0x7f857815c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:10.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.765+0000 7f857d800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8578146130 0x7f857812e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:10.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.765+0000 7f857d800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f85780cd230 con 0x7f85780ccdd0 2024-04-20T21:19:10.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.765+0000 7f857d800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f857806f150 con 0x7f8578072410 2024-04-20T21:19:10.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.765+0000 7f857d800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f85780cd3b0 con 0x7f8578146130 2024-04-20T21:19:10.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.766+0000 7f8576000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85780ccdd0 0x7f857815c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:10.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.766+0000 7f8576000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85780ccdd0 0x7f857815c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42154/0 (socket says 172.21.15.86:42154) 2024-04-20T21:19:10.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.766+0000 7f8576000640 1 -- 172.21.15.86:0/4189419571 learned_addr learned my addr 172.21.15.86:0/4189419571 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:10.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.766+0000 7f8576a00640 1 --2- 172.21.15.86:0/4189419571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8578072410 0x7f85781303a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:10.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.766+0000 7f8577400640 1 --2- 172.21.15.86:0/4189419571 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8578146130 0x7f857812e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:10.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.766+0000 7f8576000640 1 -- 172.21.15.86:0/4189419571 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8578146130 msgr2=0x7f857812e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:10.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.766+0000 7f8576000640 1 --2- 172.21.15.86:0/4189419571 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8578146130 0x7f857812e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:10.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.766+0000 7f8576000640 1 -- 172.21.15.86:0/4189419571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8578072410 msgr2=0x7f85781303a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:10.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.766+0000 7f8576000640 1 --2- 172.21.15.86:0/4189419571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8578072410 0x7f85781303a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:10.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.766+0000 7f8576000640 1 -- 172.21.15.86:0/4189419571 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f857815c570 con 0x7f85780ccdd0 2024-04-20T21:19:10.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.766+0000 7f8576a00640 1 --2- 172.21.15.86:0/4189419571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8578072410 0x7f85781303a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:10.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.766+0000 7f8576000640 1 --2- 172.21.15.86:0/4189419571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85780ccdd0 0x7f857815c030 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f856000c460 tx=0x7f856000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:10.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.766+0000 7f856ea00640 1 -- 172.21.15.86:0/4189419571 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8560016020 con 0x7f85780ccdd0 2024-04-20T21:19:10.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.767+0000 7f856ea00640 1 -- 172.21.15.86:0/4189419571 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8560012870 con 0x7f85780ccdd0 2024-04-20T21:19:10.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.767+0000 7f857d800640 1 -- 172.21.15.86:0/4189419571 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f857812f300 con 0x7f85780ccdd0 2024-04-20T21:19:10.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.767+0000 7f856ea00640 1 -- 172.21.15.86:0/4189419571 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8560012c60 con 0x7f85780ccdd0 2024-04-20T21:19:10.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.767+0000 7f857d800640 1 -- 172.21.15.86:0/4189419571 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f857812f8c0 con 0x7f85780ccdd0 2024-04-20T21:19:10.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.768+0000 7f857d800640 1 -- 172.21.15.86:0/4189419571 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85780689e0 con 0x7f85780ccdd0 2024-04-20T21:19:10.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.769+0000 7f856ea00640 1 -- 172.21.15.86:0/4189419571 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8560019020 con 0x7f85780ccdd0 2024-04-20T21:19:10.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.770+0000 7f856ea00640 1 --2- 172.21.15.86:0/4189419571 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f855406c920 0x7f855406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:10.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.770+0000 7f856ea00640 1 -- 172.21.15.86:0/4189419571 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f856008b970 con 0x7f85780ccdd0 2024-04-20T21:19:10.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.770+0000 7f8576a00640 1 --2- 172.21.15.86:0/4189419571 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f855406c920 0x7f855406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:10.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.770+0000 7f8576a00640 1 --2- 172.21.15.86:0/4189419571 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f855406c920 0x7f855406ede0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8568036000 tx=0x7f8568030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:10.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.774+0000 7f856ea00640 1 -- 172.21.15.86:0/4189419571 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8560020030 con 0x7f85780ccdd0 2024-04-20T21:19:10.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.972+0000 7f857d800640 1 -- 172.21.15.86:0/4189419571 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f85780ce6a0 con 0x7f85780ccdd0 2024-04-20T21:19:10.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.974+0000 7f856ea00640 1 -- 172.21.15.86:0/4189419571 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16682 (secure 0 0 0) 0x7f857815c570 con 0x7f85780ccdd0 2024-04-20T21:19:10.974 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:19:10.975 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":32,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:19:09.979388+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:19:07.732422+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-20T21:19:08.960268+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:19:10.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.977+0000 7f857d800640 1 -- 172.21.15.86:0/4189419571 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f855406c920 msgr2=0x7f855406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:10.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.977+0000 7f857d800640 1 --2- 172.21.15.86:0/4189419571 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f855406c920 0x7f855406ede0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8568036000 tx=0x7f8568030000 comp rx=0 tx=0).stop 2024-04-20T21:19:10.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.977+0000 7f857d800640 1 -- 172.21.15.86:0/4189419571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85780ccdd0 msgr2=0x7f857815c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:10.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.977+0000 7f857d800640 1 --2- 172.21.15.86:0/4189419571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85780ccdd0 0x7f857815c030 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f856000c460 tx=0x7f856000c920 comp rx=0 tx=0).stop 2024-04-20T21:19:10.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.977+0000 7f857d800640 1 -- 172.21.15.86:0/4189419571 shutdown_connections 2024-04-20T21:19:10.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.977+0000 7f857d800640 1 --2- 172.21.15.86:0/4189419571 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f855406c920 0x7f855406ede0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:10.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.977+0000 7f857d800640 1 --2- 172.21.15.86:0/4189419571 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8578146130 0x7f857812e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:10.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.977+0000 7f857d800640 1 --2- 172.21.15.86:0/4189419571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85780ccdd0 0x7f857815c030 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:10.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.977+0000 7f857d800640 1 --2- 172.21.15.86:0/4189419571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8578072410 0x7f85781303a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:10.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.977+0000 7f857d800640 1 -- 172.21.15.86:0/4189419571 >> 172.21.15.86:0/4189419571 conn(0x7f85780b2360 msgr2=0x7f857815cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:10.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.977+0000 7f857d800640 1 -- 172.21.15.86:0/4189419571 shutdown_connections 2024-04-20T21:19:10.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:10.977+0000 7f857d800640 1 -- 172.21.15.86:0/4189419571 wait complete. 2024-04-20T21:19:10.995 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:19:11.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.195+0000 7f1493e00640 1 Processor -- start 2024-04-20T21:19:11.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.196+0000 7f1493e00640 1 -- start start 2024-04-20T21:19:11.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.197+0000 7f1493e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f14940af7b0 0x7f149415c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:11.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.197+0000 7f1493e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14940ae710 con 0x7f149415c010 2024-04-20T21:19:11.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.197+0000 7f1493e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14940ae9f0 con 0x7f149406f4c0 2024-04-20T21:19:11.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.197+0000 7f1493e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f14940af320 con 0x7f14940af7b0 2024-04-20T21:19:11.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.197+0000 7f1492a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f149415c010 0x7f14940adda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41322/0 (socket says 172.21.15.86:41322) 2024-04-20T21:19:11.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.197+0000 7f1492a00640 1 -- 172.21.15.86:0/1924355556 learned_addr learned my addr 172.21.15.86:0/1924355556 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:19:11.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.198+0000 7f1492000640 1 --2- 172.21.15.86:0/1924355556 >> v2:172.21.15.86:3300/0 conn(0x7f14940af7b0 0x7f149415c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:11.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.198+0000 7f1491600640 1 -- 172.21.15.86:0/1924355556 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 137521823 0 0) 0x7f14940ae710 con 0x7f149415c010 2024-04-20T21:19:11.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.198+0000 7f1491600640 1 -- 172.21.15.86:0/1924355556 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1478003520 con 0x7f149415c010 2024-04-20T21:19:11.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.198+0000 7f1492000640 1 -- 172.21.15.86:0/1924355556 >> v1:172.21.15.144:6789/0 conn(0x7f149406f4c0 legacy=0x7f1494071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:11.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.198+0000 7f1492000640 1 -- 172.21.15.86:0/1924355556 >> v1:172.21.15.86:6789/0 conn(0x7f149415c010 legacy=0x7f14940adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:11.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.198+0000 7f1492000640 1 -- 172.21.15.86:0/1924355556 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14940aecd0 con 0x7f14940af7b0 2024-04-20T21:19:11.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.199+0000 7f1492000640 1 --2- 172.21.15.86:0/1924355556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14940af7b0 0x7f149415c3f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f147c0097a0 tx=0x7f147c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2de69150c102e526 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:11.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.199+0000 7f1491600640 1 -- 172.21.15.86:0/1924355556 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c034070 con 0x7f14940af7b0 2024-04-20T21:19:11.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.200+0000 7f1491600640 1 -- 172.21.15.86:0/1924355556 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f147c030610 con 0x7f14940af7b0 2024-04-20T21:19:11.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.200+0000 7f1491600640 1 -- 172.21.15.86:0/1924355556 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c0309e0 con 0x7f14940af7b0 2024-04-20T21:19:11.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.200+0000 7f1493e00640 1 -- 172.21.15.86:0/1924355556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14940af7b0 msgr2=0x7f149415c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:11.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.200+0000 7f1493e00640 1 --2- 172.21.15.86:0/1924355556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14940af7b0 0x7f149415c3f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f147c0097a0 tx=0x7f147c02b960 comp rx=0 tx=0).stop 2024-04-20T21:19:11.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.200+0000 7f1493e00640 1 -- 172.21.15.86:0/1924355556 shutdown_connections 2024-04-20T21:19:11.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.200+0000 7f1493e00640 1 --2- 172.21.15.86:0/1924355556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14940af7b0 0x7f149415c3f0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:11.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.200+0000 7f1493e00640 1 -- 172.21.15.86:0/1924355556 >> 172.21.15.86:0/1924355556 conn(0x7f14940d0170 msgr2=0x7f14940d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:11.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.200+0000 7f1493e00640 1 -- 172.21.15.86:0/1924355556 shutdown_connections 2024-04-20T21:19:11.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.200+0000 7f1493e00640 1 -- 172.21.15.86:0/1924355556 wait complete. 2024-04-20T21:19:11.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.202+0000 7f1493e00640 1 Processor -- start 2024-04-20T21:19:11.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.202+0000 7f1493e00640 1 -- start start 2024-04-20T21:19:11.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.203+0000 7f1493e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f149406f4c0 0x7f1494072620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:11.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.203+0000 7f1493e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14940af7b0 0x7f14941446e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:11.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.203+0000 7f1493e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f149415c010 0x7f14941539e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:11.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.203+0000 7f1493e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f149415c930 con 0x7f149415c010 2024-04-20T21:19:11.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.203+0000 7f1493e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f14940ae4b0 con 0x7f14940af7b0 2024-04-20T21:19:11.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.203+0000 7f1493e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f149406f8f0 con 0x7f149406f4c0 2024-04-20T21:19:11.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.203+0000 7f1493400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f149415c010 0x7f14941539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:11.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.203+0000 7f1493400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f149415c010 0x7f14941539e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42170/0 (socket says 172.21.15.86:42170) 2024-04-20T21:19:11.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.203+0000 7f1493400640 1 -- 172.21.15.86:0/2905822589 learned_addr learned my addr 172.21.15.86:0/2905822589 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:11.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.204+0000 7f1492a00640 1 --2- 172.21.15.86:0/2905822589 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f149406f4c0 0x7f1494072620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:11.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.204+0000 7f1492000640 1 --2- 172.21.15.86:0/2905822589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14940af7b0 0x7f14941446e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:11.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.204+0000 7f1493400640 1 -- 172.21.15.86:0/2905822589 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f149406f4c0 msgr2=0x7f1494072620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:11.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.204+0000 7f1493400640 1 --2- 172.21.15.86:0/2905822589 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f149406f4c0 0x7f1494072620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:11.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.204+0000 7f1493400640 1 -- 172.21.15.86:0/2905822589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14940af7b0 msgr2=0x7f14941446e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:11.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.204+0000 7f1493400640 1 --2- 172.21.15.86:0/2905822589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14940af7b0 0x7f14941446e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:11.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.204+0000 7f1493400640 1 -- 172.21.15.86:0/2905822589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14941526a0 con 0x7f149415c010 2024-04-20T21:19:11.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.204+0000 7f1493400640 1 --2- 172.21.15.86:0/2905822589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f149415c010 0x7f14941539e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f148c00c3c0 tx=0x7f148c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:11.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.204+0000 7f148aa00640 1 -- 172.21.15.86:0/2905822589 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f148c0052d0 con 0x7f149415c010 2024-04-20T21:19:11.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.204+0000 7f148aa00640 1 -- 172.21.15.86:0/2905822589 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f148c005490 con 0x7f149415c010 2024-04-20T21:19:11.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.204+0000 7f148aa00640 1 -- 172.21.15.86:0/2905822589 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f148c007260 con 0x7f149415c010 2024-04-20T21:19:11.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.204+0000 7f1493e00640 1 -- 172.21.15.86:0/2905822589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1494152940 con 0x7f149415c010 2024-04-20T21:19:11.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.205+0000 7f1493e00640 1 -- 172.21.15.86:0/2905822589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1494152f00 con 0x7f149415c010 2024-04-20T21:19:11.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.206+0000 7f1493e00640 1 -- 172.21.15.86:0/2905822589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f149404fcf0 con 0x7f149415c010 2024-04-20T21:19:11.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.207+0000 7f148aa00640 1 -- 172.21.15.86:0/2905822589 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f148c007400 con 0x7f149415c010 2024-04-20T21:19:11.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.208+0000 7f148aa00640 1 --2- 172.21.15.86:0/2905822589 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f147006c8f0 0x7f147006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:11.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.208+0000 7f148aa00640 1 -- 172.21.15.86:0/2905822589 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f148c08aff0 con 0x7f149415c010 2024-04-20T21:19:11.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.208+0000 7f1492a00640 1 --2- 172.21.15.86:0/2905822589 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f147006c8f0 0x7f147006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:11.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.209+0000 7f1492a00640 1 --2- 172.21.15.86:0/2905822589 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f147006c8f0 0x7f147006edb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1484001560 tx=0x7f1484009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:11.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.212+0000 7f148aa00640 1 -- 172.21.15.86:0/2905822589 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f148c0593e0 con 0x7f149415c010 2024-04-20T21:19:11.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.415+0000 7f1493e00640 1 -- 172.21.15.86:0/2905822589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f149415b690 con 0x7f149415c010 2024-04-20T21:19:11.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.418+0000 7f148aa00640 1 -- 172.21.15.86:0/2905822589 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16682 (secure 0 0 0) 0x7f148c058ce0 con 0x7f149415c010 2024-04-20T21:19:11.417 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:19:11.417 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":32,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:19:09.979388+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:19:07.732422+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-20T21:19:08.960268+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:19:11.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.421+0000 7f1493e00640 1 -- 172.21.15.86:0/2905822589 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f147006c8f0 msgr2=0x7f147006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:11.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.421+0000 7f1493e00640 1 --2- 172.21.15.86:0/2905822589 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f147006c8f0 0x7f147006edb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1484001560 tx=0x7f1484009040 comp rx=0 tx=0).stop 2024-04-20T21:19:11.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.421+0000 7f1493e00640 1 -- 172.21.15.86:0/2905822589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f149415c010 msgr2=0x7f14941539e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:11.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.421+0000 7f1493e00640 1 --2- 172.21.15.86:0/2905822589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f149415c010 0x7f14941539e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f148c00c3c0 tx=0x7f148c00c880 comp rx=0 tx=0).stop 2024-04-20T21:19:11.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.421+0000 7f1493e00640 1 -- 172.21.15.86:0/2905822589 shutdown_connections 2024-04-20T21:19:11.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.421+0000 7f1493e00640 1 --2- 172.21.15.86:0/2905822589 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f147006c8f0 0x7f147006edb0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:11.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.421+0000 7f1493e00640 1 --2- 172.21.15.86:0/2905822589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f149415c010 0x7f14941539e0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:11.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.421+0000 7f1493e00640 1 --2- 172.21.15.86:0/2905822589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14940af7b0 0x7f14941446e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:11.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.421+0000 7f1493e00640 1 --2- 172.21.15.86:0/2905822589 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f149406f4c0 0x7f1494072620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:11.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.421+0000 7f1493e00640 1 -- 172.21.15.86:0/2905822589 >> 172.21.15.86:0/2905822589 conn(0x7f14940d0170 msgr2=0x7f14941599d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:11.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.421+0000 7f1493e00640 1 -- 172.21.15.86:0/2905822589 shutdown_connections 2024-04-20T21:19:11.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.421+0000 7f1493e00640 1 -- 172.21.15.86:0/2905822589 wait complete. 2024-04-20T21:19:11.439 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-20T21:19:11.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.648+0000 7f7d2e400640 1 Processor -- start 2024-04-20T21:19:11.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.648+0000 7f7d2e400640 1 -- start start 2024-04-20T21:19:11.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.649+0000 7f7d2e400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7d2806c090 0x7f7d280cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:11.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.649+0000 7f7d2e400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d28156110 con 0x7f7d280b4ba0 2024-04-20T21:19:11.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.649+0000 7f7d2e400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d280ce460 con 0x7f7d28155d30 2024-04-20T21:19:11.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.649+0000 7f7d2e400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f7d280ce740 con 0x7f7d2806c090 2024-04-20T21:19:11.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.650+0000 7f7d27e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f7d280b4ba0 0x7f7d280b5030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41332/0 (socket says 172.21.15.86:41332) 2024-04-20T21:19:11.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.650+0000 7f7d27e00640 1 -- 172.21.15.86:0/1329219144 learned_addr learned my addr 172.21.15.86:0/1329219144 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:19:11.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.650+0000 7f7d26a00640 1 -- 172.21.15.86:0/1329219144 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1003176698 0 0) 0x7f7d28156110 con 0x7f7d280b4ba0 2024-04-20T21:19:11.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.650+0000 7f7d26a00640 1 -- 172.21.15.86:0/1329219144 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d0c003520 con 0x7f7d280b4ba0 2024-04-20T21:19:11.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.651+0000 7f7d1fe00640 1 --2- 172.21.15.86:0/1329219144 >> v2:172.21.15.144:3300/0 conn(0x7f7d2806c090 0x7f7d280cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:11.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.651+0000 7f7d26a00640 1 -- 172.21.15.86:0/1329219144 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1553798006 0 0) 0x7f7d0c003520 con 0x7f7d280b4ba0 2024-04-20T21:19:11.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.651+0000 7f7d26a00640 1 -- 172.21.15.86:0/1329219144 >> v2:172.21.15.144:3300/0 conn(0x7f7d2806c090 msgr2=0x7f7d280cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:11.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.651+0000 7f7d26a00640 1 --2- 172.21.15.86:0/1329219144 >> v2:172.21.15.144:3300/0 conn(0x7f7d2806c090 0x7f7d280cf500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:11.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.651+0000 7f7d26a00640 1 -- 172.21.15.86:0/1329219144 >> v1:172.21.15.144:6789/0 conn(0x7f7d28155d30 legacy=0x7f7d2806b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:11.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.651+0000 7f7d26a00640 1 -- 172.21.15.86:0/1329219144 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d280cea20 con 0x7f7d280b4ba0 2024-04-20T21:19:11.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.651+0000 7f7d26a00640 1 -- 172.21.15.86:0/1329219144 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f7d200019f0 con 0x7f7d280b4ba0 2024-04-20T21:19:11.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.651+0000 7f7d26a00640 1 -- 172.21.15.86:0/1329219144 >> v1:172.21.15.86:6789/0 conn(0x7f7d280b4ba0 legacy=0x7f7d280b5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:11.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.651+0000 7f7d26a00640 1 --2- 172.21.15.86:0/1329219144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d0c0039b0 0x7f7d0c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:11.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.651+0000 7f7d26a00640 1 --2- 172.21.15.86:0/1329219144 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7d0c004800 0x7f7d0c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:11.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.651+0000 7f7d26a00640 1 --2- 172.21.15.86:0/1329219144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d0c0053c0 0x7f7d0c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:11.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.651+0000 7f7d26a00640 1 -- 172.21.15.86:0/1329219144 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7d280ce740 con 0x7f7d0c0053c0 2024-04-20T21:19:11.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.651+0000 7f7d26a00640 1 -- 172.21.15.86:0/1329219144 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0c00c250 con 0x7f7d0c0039b0 2024-04-20T21:19:11.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.651+0000 7f7d26a00640 1 -- 172.21.15.86:0/1329219144 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7d0c00c450 con 0x7f7d0c004800 2024-04-20T21:19:11.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.651+0000 7f7d27e00640 1 --2- 172.21.15.86:0/1329219144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d0c0053c0 0x7f7d0c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:11.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.652+0000 7f7d27400640 1 --2- 172.21.15.86:0/1329219144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d0c0039b0 0x7f7d0c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:11.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.652+0000 7f7d27e00640 1 -- 172.21.15.86:0/1329219144 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7d0c004800 msgr2=0x7f7d0c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:11.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.652+0000 7f7d27e00640 1 --2- 172.21.15.86:0/1329219144 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7d0c004800 0x7f7d0c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:11.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.652+0000 7f7d27e00640 1 -- 172.21.15.86:0/1329219144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d0c0039b0 msgr2=0x7f7d0c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:11.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.652+0000 7f7d27e00640 1 --2- 172.21.15.86:0/1329219144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d0c0039b0 0x7f7d0c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:11.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.652+0000 7f7d27e00640 1 -- 172.21.15.86:0/1329219144 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d0c003520 con 0x7f7d0c0053c0 2024-04-20T21:19:11.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.652+0000 7f7d27400640 1 --2- 172.21.15.86:0/1329219144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d0c0039b0 0x7f7d0c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:11.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.653+0000 7f7d27e00640 1 -- 172.21.15.86:0/1329219144 reap_dead start 2024-04-20T21:19:11.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.653+0000 7f7d27e00640 1 --2- 172.21.15.86:0/1329219144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d0c0053c0 0x7f7d0c00b9c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f7d2000a840 tx=0x7f7d0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3dc7dffb239e638 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:11.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.653+0000 7f7d26a00640 1 -- 172.21.15.86:0/1329219144 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2000a460 con 0x7f7d0c0053c0 2024-04-20T21:19:11.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.653+0000 7f7d26a00640 1 -- 172.21.15.86:0/1329219144 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d20034650 con 0x7f7d0c0053c0 2024-04-20T21:19:11.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.653+0000 7f7d26a00640 1 -- 172.21.15.86:0/1329219144 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d20034a40 con 0x7f7d0c0053c0 2024-04-20T21:19:11.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.654+0000 7f7d2e400640 1 -- 172.21.15.86:0/1329219144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d0c0053c0 msgr2=0x7f7d0c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:11.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.654+0000 7f7d2e400640 1 --2- 172.21.15.86:0/1329219144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d0c0053c0 0x7f7d0c00b9c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f7d2000a840 tx=0x7f7d0c005830 comp rx=0 tx=0).stop 2024-04-20T21:19:11.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.654+0000 7f7d2e400640 1 -- 172.21.15.86:0/1329219144 shutdown_connections 2024-04-20T21:19:11.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.654+0000 7f7d2e400640 1 --2- 172.21.15.86:0/1329219144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d0c0053c0 0x7f7d0c00b9c0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:11.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.654+0000 7f7d2e400640 1 -- 172.21.15.86:0/1329219144 >> 172.21.15.86:0/1329219144 conn(0x7f7d280b2110 msgr2=0x7f7d280b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:11.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.654+0000 7f7d2e400640 1 -- 172.21.15.86:0/1329219144 shutdown_connections 2024-04-20T21:19:11.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.654+0000 7f7d2e400640 1 -- 172.21.15.86:0/1329219144 wait complete. 2024-04-20T21:19:11.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.656+0000 7f7d2e400640 1 Processor -- start 2024-04-20T21:19:11.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.656+0000 7f7d2e400640 1 -- start start 2024-04-20T21:19:11.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.657+0000 7f7d2e400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7d0c0053c0 0x7f7d2814a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:11.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.657+0000 7f7d2e400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d2814a590 0x7f7d2814b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:11.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.657+0000 7f7d2e400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d2814ddf0 0x7f7d2814b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:11.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.657+0000 7f7d2e400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7d280cf500 con 0x7f7d2814ddf0 2024-04-20T21:19:11.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.657+0000 7f7d2e400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7d280cefe0 con 0x7f7d2814a590 2024-04-20T21:19:11.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.657+0000 7f7d2e400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7d280cf680 con 0x7f7d0c0053c0 2024-04-20T21:19:11.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.657+0000 7f7d27e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d2814ddf0 0x7f7d2814b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:11.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.657+0000 7f7d27e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d2814ddf0 0x7f7d2814b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42200/0 (socket says 172.21.15.86:42200) 2024-04-20T21:19:11.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.657+0000 7f7d27e00640 1 -- 172.21.15.86:0/506476375 learned_addr learned my addr 172.21.15.86:0/506476375 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:11.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.657+0000 7f7d27400640 1 --2- 172.21.15.86:0/506476375 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7d0c0053c0 0x7f7d2814a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:11.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.657+0000 7f7d1fe00640 1 --2- 172.21.15.86:0/506476375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d2814a590 0x7f7d2814b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:11.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.657+0000 7f7d27e00640 1 -- 172.21.15.86:0/506476375 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7d0c0053c0 msgr2=0x7f7d2814a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:11.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.657+0000 7f7d27e00640 1 --2- 172.21.15.86:0/506476375 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7d0c0053c0 0x7f7d2814a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:11.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.658+0000 7f7d27e00640 1 -- 172.21.15.86:0/506476375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d2814a590 msgr2=0x7f7d2814b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:11.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.658+0000 7f7d27e00640 1 --2- 172.21.15.86:0/506476375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d2814a590 0x7f7d2814b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:11.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.658+0000 7f7d27e00640 1 -- 172.21.15.86:0/506476375 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d2814ad20 con 0x7f7d2814ddf0 2024-04-20T21:19:11.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.658+0000 7f7d27e00640 1 --2- 172.21.15.86:0/506476375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d2814ddf0 0x7f7d2814b6a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7d2002cc50 tx=0x7f7d280cebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:11.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.658+0000 7f7d1f400640 1 -- 172.21.15.86:0/506476375 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d20039040 con 0x7f7d2814ddf0 2024-04-20T21:19:11.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.658+0000 7f7d1f400640 1 -- 172.21.15.86:0/506476375 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d20043070 con 0x7f7d2814ddf0 2024-04-20T21:19:11.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.658+0000 7f7d2e400640 1 -- 172.21.15.86:0/506476375 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d2814aa40 con 0x7f7d2814ddf0 2024-04-20T21:19:11.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.658+0000 7f7d1f400640 1 -- 172.21.15.86:0/506476375 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d20004d60 con 0x7f7d2814ddf0 2024-04-20T21:19:11.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.658+0000 7f7d2e400640 1 -- 172.21.15.86:0/506476375 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d28149370 con 0x7f7d2814ddf0 2024-04-20T21:19:11.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.660+0000 7f7d2e400640 1 -- 172.21.15.86:0/506476375 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d280689e0 con 0x7f7d2814ddf0 2024-04-20T21:19:11.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.661+0000 7f7d1f400640 1 -- 172.21.15.86:0/506476375 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7d20034650 con 0x7f7d2814ddf0 2024-04-20T21:19:11.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.662+0000 7f7d1f400640 1 --2- 172.21.15.86:0/506476375 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7d0406c8f0 0x7f7d0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:11.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.662+0000 7f7d1f400640 1 -- 172.21.15.86:0/506476375 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f7d200ad8b0 con 0x7f7d2814ddf0 2024-04-20T21:19:11.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.662+0000 7f7d27400640 1 --2- 172.21.15.86:0/506476375 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7d0406c8f0 0x7f7d0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:11.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.663+0000 7f7d27400640 1 --2- 172.21.15.86:0/506476375 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7d0406c8f0 0x7f7d0406edb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7d18007fd0 tx=0x7f7d18009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:11.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.666+0000 7f7d1f400640 1 -- 172.21.15.86:0/506476375 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7d2007bc50 con 0x7f7d2814ddf0 2024-04-20T21:19:11.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:11.907+0000 7f7d2e400640 1 -- 172.21.15.86:0/506476375 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f7d28152f70 con 0x7f7d2814ddf0 2024-04-20T21:19:12.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.583+0000 7f7d1f400640 1 -- 172.21.15.86:0/506476375 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7f7d2007b550 con 0x7f7d2814ddf0 2024-04-20T21:19:12.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.586+0000 7f7d2e400640 1 -- 172.21.15.86:0/506476375 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7d0406c8f0 msgr2=0x7f7d0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:12.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.586+0000 7f7d2e400640 1 --2- 172.21.15.86:0/506476375 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7d0406c8f0 0x7f7d0406edb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7d18007fd0 tx=0x7f7d18009040 comp rx=0 tx=0).stop 2024-04-20T21:19:12.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.586+0000 7f7d2e400640 1 -- 172.21.15.86:0/506476375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d2814ddf0 msgr2=0x7f7d2814b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:12.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.586+0000 7f7d2e400640 1 --2- 172.21.15.86:0/506476375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d2814ddf0 0x7f7d2814b6a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7d2002cc50 tx=0x7f7d280cebf0 comp rx=0 tx=0).stop 2024-04-20T21:19:12.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.586+0000 7f7d2e400640 1 -- 172.21.15.86:0/506476375 shutdown_connections 2024-04-20T21:19:12.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.586+0000 7f7d2e400640 1 --2- 172.21.15.86:0/506476375 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7d0406c8f0 0x7f7d0406edb0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:12.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.586+0000 7f7d2e400640 1 --2- 172.21.15.86:0/506476375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d2814ddf0 0x7f7d2814b6a0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:12.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.586+0000 7f7d2e400640 1 --2- 172.21.15.86:0/506476375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d2814a590 0x7f7d2814b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:12.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.586+0000 7f7d2e400640 1 --2- 172.21.15.86:0/506476375 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7d0c0053c0 0x7f7d2814a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:12.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.586+0000 7f7d2e400640 1 -- 172.21.15.86:0/506476375 >> 172.21.15.86:0/506476375 conn(0x7f7d280b2110 msgr2=0x7f7d280b24f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:12.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.587+0000 7f7d2e400640 1 -- 172.21.15.86:0/506476375 shutdown_connections 2024-04-20T21:19:12.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.587+0000 7f7d2e400640 1 -- 172.21.15.86:0/506476375 wait complete. 2024-04-20T21:19:12.607 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-20T21:19:12.607 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-20T21:19:12.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.802+0000 7f29c7a00640 1 Processor -- start 2024-04-20T21:19:12.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.803+0000 7f29c7a00640 1 -- start start 2024-04-20T21:19:12.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.804+0000 7f29c7a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f29c009a9e0 0x7f29c009ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:12.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.804+0000 7f29c7a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29c00d13b0 con 0x7f29c015eb70 2024-04-20T21:19:12.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.804+0000 7f29c7a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29c00d1690 con 0x7f29c006cd70 2024-04-20T21:19:12.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.804+0000 7f29c7a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f29c009b390 con 0x7f29c009a9e0 2024-04-20T21:19:12.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.805+0000 7f29c5000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f29c015eb70 0x7f29c00d08f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41348/0 (socket says 172.21.15.86:41348) 2024-04-20T21:19:12.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.805+0000 7f29c5000640 1 -- 172.21.15.86:0/3913076607 learned_addr learned my addr 172.21.15.86:0/3913076607 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:19:12.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.805+0000 7f29bfe00640 1 -- 172.21.15.86:0/3913076607 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3249863935 0 0) 0x7f29c00d13b0 con 0x7f29c015eb70 2024-04-20T21:19:12.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.805+0000 7f29b7e00640 1 --2- 172.21.15.86:0/3913076607 >> v2:172.21.15.144:3300/0 conn(0x7f29c009a9e0 0x7f29c009ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:12.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.805+0000 7f29bfe00640 1 -- 172.21.15.86:0/3913076607 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f29a0003520 con 0x7f29c015eb70 2024-04-20T21:19:12.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.805+0000 7f29bfe00640 1 -- 172.21.15.86:0/3913076607 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3643040513 0 0) 0x7f29a0003520 con 0x7f29c015eb70 2024-04-20T21:19:12.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.805+0000 7f29bfe00640 1 -- 172.21.15.86:0/3913076607 >> v2:172.21.15.144:3300/0 conn(0x7f29c009a9e0 msgr2=0x7f29c009ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:12.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.805+0000 7f29bfe00640 1 --2- 172.21.15.86:0/3913076607 >> v2:172.21.15.144:3300/0 conn(0x7f29c009a9e0 0x7f29c009ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:12.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.805+0000 7f29bfe00640 1 -- 172.21.15.86:0/3913076607 >> v1:172.21.15.144:6789/0 conn(0x7f29c006cd70 legacy=0x7f29c006d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:12.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.805+0000 7f29bfe00640 1 -- 172.21.15.86:0/3913076607 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29c00d1970 con 0x7f29c015eb70 2024-04-20T21:19:12.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.806+0000 7f29bfe00640 1 -- 172.21.15.86:0/3913076607 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f29b0001670 con 0x7f29c015eb70 2024-04-20T21:19:12.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.806+0000 7f29bfe00640 1 -- 172.21.15.86:0/3913076607 >> v1:172.21.15.86:6789/0 conn(0x7f29c015eb70 legacy=0x7f29c00d08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:12.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.806+0000 7f29bfe00640 1 --2- 172.21.15.86:0/3913076607 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29a00039e0 0x7f29a0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:12.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.806+0000 7f29bfe00640 1 --2- 172.21.15.86:0/3913076607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29a0004800 0x7f29a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:12.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.806+0000 7f29bfe00640 1 --2- 172.21.15.86:0/3913076607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a00051c0 0x7f29a000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:12.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.806+0000 7f29bfe00640 1 -- 172.21.15.86:0/3913076607 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f29c009b390 con 0x7f29a00051c0 2024-04-20T21:19:12.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.806+0000 7f29bfe00640 1 -- 172.21.15.86:0/3913076607 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f29a000c090 con 0x7f29a0004800 2024-04-20T21:19:12.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.806+0000 7f29bfe00640 1 -- 172.21.15.86:0/3913076607 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f29a000c290 con 0x7f29a00039e0 2024-04-20T21:19:12.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.806+0000 7f29c5a00640 1 --2- 172.21.15.86:0/3913076607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a00051c0 0x7f29a000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:12.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.806+0000 7f29c5000640 1 --2- 172.21.15.86:0/3913076607 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29a00039e0 0x7f29a0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:12.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.806+0000 7f29c5a00640 1 -- 172.21.15.86:0/3913076607 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29a00039e0 msgr2=0x7f29a0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:12.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.806+0000 7f29c5a00640 1 --2- 172.21.15.86:0/3913076607 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29a00039e0 0x7f29a0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:12.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.806+0000 7f29c5a00640 1 -- 172.21.15.86:0/3913076607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29a0004800 msgr2=0x7f29a0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:12.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.806+0000 7f29c5a00640 1 --2- 172.21.15.86:0/3913076607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29a0004800 0x7f29a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:12.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.806+0000 7f29c5a00640 1 -- 172.21.15.86:0/3913076607 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29a0003520 con 0x7f29a00051c0 2024-04-20T21:19:12.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.806+0000 7f29c5000640 1 --2- 172.21.15.86:0/3913076607 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29a00039e0 0x7f29a0003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:12.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.807+0000 7f29c5a00640 1 -- 172.21.15.86:0/3913076607 reap_dead start 2024-04-20T21:19:12.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.808+0000 7f29c5a00640 1 --2- 172.21.15.86:0/3913076607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a00051c0 0x7f29a000b9c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f29b8001e50 tx=0x7f29a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8ff4197b85cdaf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:12.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.808+0000 7f29bfe00640 1 -- 172.21.15.86:0/3913076607 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b8001a70 con 0x7f29a00051c0 2024-04-20T21:19:12.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.808+0000 7f29bfe00640 1 -- 172.21.15.86:0/3913076607 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29b8003bb0 con 0x7f29a00051c0 2024-04-20T21:19:12.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.808+0000 7f29bfe00640 1 -- 172.21.15.86:0/3913076607 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b80050d0 con 0x7f29a00051c0 2024-04-20T21:19:12.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.808+0000 7f29c7a00640 1 -- 172.21.15.86:0/3913076607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a00051c0 msgr2=0x7f29a000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:12.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.808+0000 7f29c7a00640 1 --2- 172.21.15.86:0/3913076607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a00051c0 0x7f29a000b9c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f29b8001e50 tx=0x7f29a0005830 comp rx=0 tx=0).stop 2024-04-20T21:19:12.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.808+0000 7f29c7a00640 1 -- 172.21.15.86:0/3913076607 shutdown_connections 2024-04-20T21:19:12.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.808+0000 7f29c7a00640 1 --2- 172.21.15.86:0/3913076607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a00051c0 0x7f29a000b9c0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:12.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.808+0000 7f29c7a00640 1 -- 172.21.15.86:0/3913076607 >> 172.21.15.86:0/3913076607 conn(0x7f29c00b4070 msgr2=0x7f29c00b4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:12.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.809+0000 7f29c7a00640 1 -- 172.21.15.86:0/3913076607 shutdown_connections 2024-04-20T21:19:12.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.809+0000 7f29c7a00640 1 -- 172.21.15.86:0/3913076607 wait complete. 2024-04-20T21:19:12.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.810+0000 7f29c7a00640 1 Processor -- start 2024-04-20T21:19:12.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.810+0000 7f29c7a00640 1 -- start start 2024-04-20T21:19:12.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.811+0000 7f29c7a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a00051c0 0x7f29c00b5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:12.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.811+0000 7f29c7a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29c00b75f0 0x7f29c0161ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:12.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.811+0000 7f29c7a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29c0153590 0x7f29c0153a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:12.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.811+0000 7f29c7a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f29c015f160 con 0x7f29a00051c0 2024-04-20T21:19:12.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.811+0000 7f29c7a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f29c00d1f30 con 0x7f29c00b75f0 2024-04-20T21:19:12.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.811+0000 7f29c7a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f29c015f2e0 con 0x7f29c0153590 2024-04-20T21:19:12.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.812+0000 7f29c5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a00051c0 0x7f29c00b5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:12.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.812+0000 7f29c5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a00051c0 0x7f29c00b5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42214/0 (socket says 172.21.15.86:42214) 2024-04-20T21:19:12.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.812+0000 7f29c5000640 1 -- 172.21.15.86:0/2398406723 learned_addr learned my addr 172.21.15.86:0/2398406723 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:12.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.812+0000 7f29c5000640 1 -- 172.21.15.86:0/2398406723 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29c0153590 msgr2=0x7f29c0153a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:12.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.812+0000 7f29c5000640 1 --2- 172.21.15.86:0/2398406723 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29c0153590 0x7f29c0153a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:12.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.812+0000 7f29b7e00640 1 --2- 172.21.15.86:0/2398406723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29c00b75f0 0x7f29c0161ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:12.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.812+0000 7f29c5000640 1 -- 172.21.15.86:0/2398406723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29c00b75f0 msgr2=0x7f29c0161ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:12.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.812+0000 7f29c5000640 1 --2- 172.21.15.86:0/2398406723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29c00b75f0 0x7f29c0161ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:12.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.812+0000 7f29c5000640 1 -- 172.21.15.86:0/2398406723 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29c00b6550 con 0x7f29a00051c0 2024-04-20T21:19:12.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.812+0000 7f29c5000640 1 --2- 172.21.15.86:0/2398406723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a00051c0 0x7f29c00b5ec0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f29b0001200 tx=0x7f29b000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:12.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.813+0000 7f29bd600640 1 -- 172.21.15.86:0/2398406723 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b0019070 con 0x7f29a00051c0 2024-04-20T21:19:12.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.813+0000 7f29bd600640 1 -- 172.21.15.86:0/2398406723 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29b00074e0 con 0x7f29a00051c0 2024-04-20T21:19:12.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.813+0000 7f29bd600640 1 -- 172.21.15.86:0/2398406723 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b00049e0 con 0x7f29a00051c0 2024-04-20T21:19:12.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.813+0000 7f29c7a00640 1 -- 172.21.15.86:0/2398406723 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29c00b6830 con 0x7f29a00051c0 2024-04-20T21:19:12.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.813+0000 7f29c7a00640 1 -- 172.21.15.86:0/2398406723 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29c00b70d0 con 0x7f29a00051c0 2024-04-20T21:19:12.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.814+0000 7f29c7a00640 1 -- 172.21.15.86:0/2398406723 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29c00b6c20 con 0x7f29a00051c0 2024-04-20T21:19:12.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.815+0000 7f29bd600640 1 -- 172.21.15.86:0/2398406723 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f29b001c070 con 0x7f29a00051c0 2024-04-20T21:19:12.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.816+0000 7f29bd600640 1 --2- 172.21.15.86:0/2398406723 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f299c06c8f0 0x7f299c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:12.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.817+0000 7f29bd600640 1 -- 172.21.15.86:0/2398406723 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f29b008b660 con 0x7f29a00051c0 2024-04-20T21:19:12.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.817+0000 7f29b7e00640 1 --2- 172.21.15.86:0/2398406723 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f299c06c8f0 0x7f299c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:12.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.817+0000 7f29b7e00640 1 --2- 172.21.15.86:0/2398406723 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f299c06c8f0 0x7f299c06edb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f29c0159d70 tx=0x7f29a8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:12.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:12.821+0000 7f29bd600640 1 -- 172.21.15.86:0/2398406723 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f29b0001d00 con 0x7f29a00051c0 2024-04-20T21:19:13.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.062+0000 7f29c7a00640 1 -- 172.21.15.86:0/2398406723 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f29c00689e0 con 0x7f29a00051c0 2024-04-20T21:19:13.473 INFO:tasks.ceph.mds.a.smithi086.stderr:2024-04-20T21:19:13.474+0000 7f35c0200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T21:19:13.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.593+0000 7f29bd600640 1 -- 172.21.15.86:0/2398406723 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7f29b00599f0 con 0x7f29a00051c0 2024-04-20T21:19:13.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.597+0000 7f29c7a00640 1 -- 172.21.15.86:0/2398406723 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f299c06c8f0 msgr2=0x7f299c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:13.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.597+0000 7f29c7a00640 1 --2- 172.21.15.86:0/2398406723 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f299c06c8f0 0x7f299c06edb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f29c0159d70 tx=0x7f29a8008040 comp rx=0 tx=0).stop 2024-04-20T21:19:13.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.597+0000 7f29c7a00640 1 -- 172.21.15.86:0/2398406723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a00051c0 msgr2=0x7f29c00b5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:13.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.597+0000 7f29c7a00640 1 --2- 172.21.15.86:0/2398406723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a00051c0 0x7f29c00b5ec0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f29b0001200 tx=0x7f29b000dcc0 comp rx=0 tx=0).stop 2024-04-20T21:19:13.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.597+0000 7f29c7a00640 1 -- 172.21.15.86:0/2398406723 shutdown_connections 2024-04-20T21:19:13.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.597+0000 7f29c7a00640 1 --2- 172.21.15.86:0/2398406723 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f299c06c8f0 0x7f299c06edb0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:13.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.597+0000 7f29c7a00640 1 --2- 172.21.15.86:0/2398406723 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f29c0153590 0x7f29c0153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:13.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.597+0000 7f29c7a00640 1 --2- 172.21.15.86:0/2398406723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29c00b75f0 0x7f29c0161ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:13.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.597+0000 7f29c7a00640 1 --2- 172.21.15.86:0/2398406723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a00051c0 0x7f29c00b5ec0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:13.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.597+0000 7f29c7a00640 1 -- 172.21.15.86:0/2398406723 >> 172.21.15.86:0/2398406723 conn(0x7f29c00b4070 msgr2=0x7f29c015f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:13.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.597+0000 7f29c7a00640 1 -- 172.21.15.86:0/2398406723 shutdown_connections 2024-04-20T21:19:13.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.598+0000 7f29c7a00640 1 -- 172.21.15.86:0/2398406723 wait complete. 2024-04-20T21:19:13.617 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-20T21:19:13.617 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:19:13.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.819+0000 7f1e2e600640 1 Processor -- start 2024-04-20T21:19:13.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.819+0000 7f1e2e600640 1 -- start start 2024-04-20T21:19:13.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.821+0000 7f1e2e600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e28146130 0x7f1e2806e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:13.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.821+0000 7f1e2e600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1e280ccdd0 0x7f1e280cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:13.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.821+0000 7f1e2e600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e2806dc30 con 0x7f1e28071d00 2024-04-20T21:19:13.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.821+0000 7f1e2e600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f1e280cd8e0 con 0x7f1e280ccdd0 2024-04-20T21:19:13.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.821+0000 7f1e2e600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1e280cdbc0 con 0x7f1e28146130 2024-04-20T21:19:13.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.821+0000 7f1e26a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1e280ccdd0 0x7f1e280cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:13.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.821+0000 7f1e26a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1e280ccdd0 0x7f1e280cd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42230/0 (socket says 172.21.15.86:42230) 2024-04-20T21:19:13.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.821+0000 7f1e26a00640 1 -- 172.21.15.86:0/2908348170 learned_addr learned my addr 172.21.15.86:0/2908348170 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:13.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.821+0000 7f1e27400640 1 --2- 172.21.15.86:0/2908348170 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e28146130 0x7f1e2806e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:13.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.821+0000 7f1e26a00640 1 -- 172.21.15.86:0/2908348170 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e28146130 msgr2=0x7f1e2806e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:13.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.821+0000 7f1e26a00640 1 --2- 172.21.15.86:0/2908348170 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e28146130 0x7f1e2806e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:13.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.822+0000 7f1e26a00640 1 -- 172.21.15.86:0/2908348170 >> v1:172.21.15.144:6789/0 conn(0x7f1e28071d00 legacy=0x7f1e2815cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:13.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.822+0000 7f1e26a00640 1 -- 172.21.15.86:0/2908348170 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e2806ded0 con 0x7f1e280ccdd0 2024-04-20T21:19:13.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.822+0000 7f1e27400640 1 --2- 172.21.15.86:0/2908348170 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e28146130 0x7f1e2806e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:13.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.823+0000 7f1e26a00640 1 --2- 172.21.15.86:0/2908348170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e280ccdd0 0x7f1e280cd230 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1e14009870 tx=0x7f1e1402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47716977afc1f2ec server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:13.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.823+0000 7f1e26000640 1 -- 172.21.15.86:0/2908348170 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e14034070 con 0x7f1e280ccdd0 2024-04-20T21:19:13.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.823+0000 7f1e26000640 1 -- 172.21.15.86:0/2908348170 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e14030840 con 0x7f1e280ccdd0 2024-04-20T21:19:13.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.823+0000 7f1e26000640 1 -- 172.21.15.86:0/2908348170 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e14030c10 con 0x7f1e280ccdd0 2024-04-20T21:19:13.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.824+0000 7f1e2e600640 1 -- 172.21.15.86:0/2908348170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e280ccdd0 msgr2=0x7f1e280cd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:13.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.824+0000 7f1e2e600640 1 --2- 172.21.15.86:0/2908348170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e280ccdd0 0x7f1e280cd230 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1e14009870 tx=0x7f1e1402b980 comp rx=0 tx=0).stop 2024-04-20T21:19:13.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.824+0000 7f1e2e600640 1 -- 172.21.15.86:0/2908348170 shutdown_connections 2024-04-20T21:19:13.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.824+0000 7f1e2e600640 1 --2- 172.21.15.86:0/2908348170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e280ccdd0 0x7f1e280cd230 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:13.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.824+0000 7f1e2e600640 1 --2- 172.21.15.86:0/2908348170 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e28146130 0x7f1e2806e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:13.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.824+0000 7f1e2e600640 1 -- 172.21.15.86:0/2908348170 >> 172.21.15.86:0/2908348170 conn(0x7f1e280b2360 msgr2=0x7f1e280b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:13.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.824+0000 7f1e2e600640 1 -- 172.21.15.86:0/2908348170 shutdown_connections 2024-04-20T21:19:13.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.824+0000 7f1e2e600640 1 -- 172.21.15.86:0/2908348170 wait complete. 2024-04-20T21:19:13.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.826+0000 7f1e2e600640 1 Processor -- start 2024-04-20T21:19:13.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.826+0000 7f1e2e600640 1 -- start start 2024-04-20T21:19:13.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.827+0000 7f1e2e600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e28071d00 0x7f1e281f5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:13.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.827+0000 7f1e2e600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e280ccdd0 0x7f1e281e5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:13.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.827+0000 7f1e2e600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e28146130 0x7f1e28139a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:13.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.827+0000 7f1e2e600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1e28072190 con 0x7f1e280ccdd0 2024-04-20T21:19:13.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.827+0000 7f1e2e600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2806f150 con 0x7f1e28071d00 2024-04-20T21:19:13.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.827+0000 7f1e2e600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1e28072310 con 0x7f1e28146130 2024-04-20T21:19:13.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.827+0000 7f1e26a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e280ccdd0 0x7f1e281e5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:13.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.827+0000 7f1e26a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e280ccdd0 0x7f1e281e5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42242/0 (socket says 172.21.15.86:42242) 2024-04-20T21:19:13.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.827+0000 7f1e26a00640 1 -- 172.21.15.86:0/7178378 learned_addr learned my addr 172.21.15.86:0/7178378 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:13.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.828+0000 7f1e27e00640 1 --2- 172.21.15.86:0/7178378 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e28146130 0x7f1e28139a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:13.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.828+0000 7f1e27400640 1 --2- 172.21.15.86:0/7178378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e28071d00 0x7f1e281f5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:13.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.828+0000 7f1e26a00640 1 -- 172.21.15.86:0/7178378 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e28146130 msgr2=0x7f1e28139a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:13.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.828+0000 7f1e26a00640 1 --2- 172.21.15.86:0/7178378 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e28146130 0x7f1e28139a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:13.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.828+0000 7f1e26a00640 1 -- 172.21.15.86:0/7178378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e28071d00 msgr2=0x7f1e281f5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:13.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.828+0000 7f1e26a00640 1 --2- 172.21.15.86:0/7178378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e28071d00 0x7f1e281f5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:13.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.828+0000 7f1e26a00640 1 -- 172.21.15.86:0/7178378 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e2813b790 con 0x7f1e280ccdd0 2024-04-20T21:19:13.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.828+0000 7f1e26a00640 1 --2- 172.21.15.86:0/7178378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e280ccdd0 0x7f1e281e5690 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1e14009870 tx=0x7f1e14004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:13.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.828+0000 7f1e1f400640 1 -- 172.21.15.86:0/7178378 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e14048070 con 0x7f1e280ccdd0 2024-04-20T21:19:13.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.829+0000 7f1e1f400640 1 -- 172.21.15.86:0/7178378 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e14030640 con 0x7f1e280ccdd0 2024-04-20T21:19:13.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.829+0000 7f1e1f400640 1 -- 172.21.15.86:0/7178378 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e14034040 con 0x7f1e280ccdd0 2024-04-20T21:19:13.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.829+0000 7f1e2e600640 1 -- 172.21.15.86:0/7178378 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e2813ba00 con 0x7f1e280ccdd0 2024-04-20T21:19:13.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.829+0000 7f1e2e600640 1 -- 172.21.15.86:0/7178378 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e2813c2a0 con 0x7f1e280ccdd0 2024-04-20T21:19:13.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.830+0000 7f1e2e600640 1 -- 172.21.15.86:0/7178378 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e2813bbd0 con 0x7f1e280ccdd0 2024-04-20T21:19:13.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.831+0000 7f1e1f400640 1 -- 172.21.15.86:0/7178378 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1e14037070 con 0x7f1e280ccdd0 2024-04-20T21:19:13.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.832+0000 7f1e1f400640 1 --2- 172.21.15.86:0/7178378 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1e0806c8f0 0x7f1e0806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:13.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.832+0000 7f1e1f400640 1 -- 172.21.15.86:0/7178378 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f1e140ad9a0 con 0x7f1e280ccdd0 2024-04-20T21:19:13.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.832+0000 7f1e27400640 1 --2- 172.21.15.86:0/7178378 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1e0806c8f0 0x7f1e0806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:13.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.833+0000 7f1e27400640 1 --2- 172.21.15.86:0/7178378 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1e0806c8f0 0x7f1e0806edb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1e1800b840 tx=0x7f1e18008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:13.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:13.836+0000 7f1e1f400640 1 -- 172.21.15.86:0/7178378 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1e1407c040 con 0x7f1e280ccdd0 2024-04-20T21:19:14.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.075+0000 7f1e2e600640 1 -- 172.21.15.86:0/7178378 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1e280689e0 con 0x7f1e280ccdd0 2024-04-20T21:19:14.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.076+0000 7f1e1f400640 1 -- 172.21.15.86:0/7178378 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5205 (secure 0 0 0) 0x7f1e1407b940 con 0x7f1e280ccdd0 2024-04-20T21:19:14.076 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:19:14.076 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4604,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/1957936003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":1957936003},{"type":"v1","addr":"172.21.15.144:6834","nonce":1957936003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4807,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/1510649389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":1510649389},{"type":"v1","addr":"172.21.15.144:6835","nonce":1510649389}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2180009156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2180009156},{"type":"v1","addr":"172.21.15.86:6836","nonce":2180009156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:19:09.979345+0000","modified":"2024-04-20T21:19:13.064114+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5226":{"gid":5226,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6837/3715801064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3715801064},{"type":"v1","addr":"172.21.15.86:6837","nonce":3715801064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5226,"qdb_cluster":[5226]},"id":2}]} 2024-04-20T21:19:14.076 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 16 2024-04-20T21:19:14.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.079+0000 7f1e2e600640 1 -- 172.21.15.86:0/7178378 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1e0806c8f0 msgr2=0x7f1e0806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:14.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.079+0000 7f1e2e600640 1 --2- 172.21.15.86:0/7178378 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1e0806c8f0 0x7f1e0806edb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1e1800b840 tx=0x7f1e18008040 comp rx=0 tx=0).stop 2024-04-20T21:19:14.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.079+0000 7f1e2e600640 1 -- 172.21.15.86:0/7178378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e280ccdd0 msgr2=0x7f1e281e5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:14.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.079+0000 7f1e2e600640 1 --2- 172.21.15.86:0/7178378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e280ccdd0 0x7f1e281e5690 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1e14009870 tx=0x7f1e14004730 comp rx=0 tx=0).stop 2024-04-20T21:19:14.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.079+0000 7f1e2e600640 1 -- 172.21.15.86:0/7178378 shutdown_connections 2024-04-20T21:19:14.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.079+0000 7f1e2e600640 1 --2- 172.21.15.86:0/7178378 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1e0806c8f0 0x7f1e0806edb0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:14.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.079+0000 7f1e2e600640 1 --2- 172.21.15.86:0/7178378 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e28146130 0x7f1e28139a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:14.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.079+0000 7f1e2e600640 1 --2- 172.21.15.86:0/7178378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e280ccdd0 0x7f1e281e5690 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:14.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.079+0000 7f1e2e600640 1 --2- 172.21.15.86:0/7178378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e28071d00 0x7f1e281f5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:14.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.079+0000 7f1e2e600640 1 -- 172.21.15.86:0/7178378 >> 172.21.15.86:0/7178378 conn(0x7f1e280b2360 msgr2=0x7f1e28142110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:14.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.079+0000 7f1e2e600640 1 -- 172.21.15.86:0/7178378 shutdown_connections 2024-04-20T21:19:14.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.079+0000 7f1e2e600640 1 -- 172.21.15.86:0/7178378 wait complete. 2024-04-20T21:19:14.097 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:19:14.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.293+0000 7faa93e00640 1 Processor -- start 2024-04-20T21:19:14.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.293+0000 7faa93e00640 1 -- start start 2024-04-20T21:19:14.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.295+0000 7faa93e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7faa940af8d0 0x7faa94141f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:14.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.295+0000 7faa93e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa94071c10 con 0x7faa940b3810 2024-04-20T21:19:14.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.295+0000 7faa93e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa9415eaf0 con 0x7faa94162c40 2024-04-20T21:19:14.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.295+0000 7faa93e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7faa94142490 con 0x7faa940af8d0 2024-04-20T21:19:14.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.295+0000 7faa93400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7faa940b3810 0x7faa94072990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41362/0 (socket says 172.21.15.86:41362) 2024-04-20T21:19:14.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.295+0000 7faa93400640 1 -- 172.21.15.86:0/1541844560 learned_addr learned my addr 172.21.15.86:0/1541844560 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:19:14.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.295+0000 7faa92a00640 1 --2- 172.21.15.86:0/1541844560 >> v2:172.21.15.86:3300/0 conn(0x7faa940af8d0 0x7faa94141f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:14.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.296+0000 7faa92000640 1 -- 172.21.15.86:0/1541844560 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3241456483 0 0) 0x7faa94071c10 con 0x7faa940b3810 2024-04-20T21:19:14.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.296+0000 7faa92000640 1 -- 172.21.15.86:0/1541844560 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa78003520 con 0x7faa940b3810 2024-04-20T21:19:14.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.296+0000 7faa92a00640 1 -- 172.21.15.86:0/1541844560 >> v1:172.21.15.144:6789/0 conn(0x7faa94162c40 legacy=0x7faa940b3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:14.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.296+0000 7faa92a00640 1 -- 172.21.15.86:0/1541844560 >> v1:172.21.15.86:6789/0 conn(0x7faa940b3810 legacy=0x7faa94072990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:14.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.296+0000 7faa92a00640 1 -- 172.21.15.86:0/1541844560 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa94071eb0 con 0x7faa940af8d0 2024-04-20T21:19:14.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.297+0000 7faa92a00640 1 --2- 172.21.15.86:0/1541844560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa940af8d0 0x7faa94141f50 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7faa84009d90 tx=0x7faa8402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a16f0a947c9b081 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:14.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.297+0000 7faa92000640 1 -- 172.21.15.86:0/1541844560 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa84039070 con 0x7faa940af8d0 2024-04-20T21:19:14.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.297+0000 7faa92000640 1 -- 172.21.15.86:0/1541844560 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa8402cc60 con 0x7faa940af8d0 2024-04-20T21:19:14.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.297+0000 7faa92000640 1 -- 172.21.15.86:0/1541844560 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa84034660 con 0x7faa940af8d0 2024-04-20T21:19:14.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.298+0000 7faa93e00640 1 -- 172.21.15.86:0/1541844560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa940af8d0 msgr2=0x7faa94141f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:14.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.298+0000 7faa93e00640 1 --2- 172.21.15.86:0/1541844560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa940af8d0 0x7faa94141f50 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7faa84009d90 tx=0x7faa8402beb0 comp rx=0 tx=0).stop 2024-04-20T21:19:14.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.298+0000 7faa93e00640 1 -- 172.21.15.86:0/1541844560 shutdown_connections 2024-04-20T21:19:14.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.298+0000 7faa93e00640 1 --2- 172.21.15.86:0/1541844560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa940af8d0 0x7faa94141f50 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:14.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.298+0000 7faa93e00640 1 -- 172.21.15.86:0/1541844560 >> 172.21.15.86:0/1541844560 conn(0x7faa940d0170 msgr2=0x7faa940d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:14.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.298+0000 7faa93e00640 1 -- 172.21.15.86:0/1541844560 shutdown_connections 2024-04-20T21:19:14.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.298+0000 7faa93e00640 1 -- 172.21.15.86:0/1541844560 wait complete. 2024-04-20T21:19:14.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.300+0000 7faa93e00640 1 Processor -- start 2024-04-20T21:19:14.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.300+0000 7faa93e00640 1 -- start start 2024-04-20T21:19:14.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.301+0000 7faa93e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa940af8d0 0x7faa940b0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:14.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.301+0000 7faa93e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa940b3810 0x7faa941f4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:14.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.301+0000 7faa93e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa94162c40 0x7faa94146bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:14.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.301+0000 7faa93e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa94163100 con 0x7faa94162c40 2024-04-20T21:19:14.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.301+0000 7faa93e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7faa94072470 con 0x7faa940b3810 2024-04-20T21:19:14.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.301+0000 7faa93e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7faa94148050 con 0x7faa940af8d0 2024-04-20T21:19:14.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.301+0000 7faa93400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa94162c40 0x7faa94146bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:14.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.301+0000 7faa93400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa94162c40 0x7faa94146bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42266/0 (socket says 172.21.15.86:42266) 2024-04-20T21:19:14.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.301+0000 7faa93400640 1 -- 172.21.15.86:0/2623797026 learned_addr learned my addr 172.21.15.86:0/2623797026 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:14.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.301+0000 7faa93400640 1 -- 172.21.15.86:0/2623797026 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa940af8d0 msgr2=0x7faa940b0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:14.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.301+0000 7faa93400640 1 --2- 172.21.15.86:0/2623797026 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa940af8d0 0x7faa940b0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:14.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.301+0000 7faa93400640 1 -- 172.21.15.86:0/2623797026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa940b3810 msgr2=0x7faa941f4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:14.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.301+0000 7faa93400640 1 --2- 172.21.15.86:0/2623797026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa940b3810 0x7faa941f4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:14.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.301+0000 7faa93400640 1 -- 172.21.15.86:0/2623797026 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa9415fe70 con 0x7faa94162c40 2024-04-20T21:19:14.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.302+0000 7faa93400640 1 --2- 172.21.15.86:0/2623797026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa94162c40 0x7faa94146bf0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7faa8c00c420 tx=0x7faa8c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:14.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.302+0000 7faa8b400640 1 -- 172.21.15.86:0/2623797026 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8c017070 con 0x7faa94162c40 2024-04-20T21:19:14.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.302+0000 7faa8b400640 1 -- 172.21.15.86:0/2623797026 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa8c005ce0 con 0x7faa94162c40 2024-04-20T21:19:14.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.302+0000 7faa93e00640 1 -- 172.21.15.86:0/2623797026 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa94160150 con 0x7faa94162c40 2024-04-20T21:19:14.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.302+0000 7faa8b400640 1 -- 172.21.15.86:0/2623797026 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8c0072d0 con 0x7faa94162c40 2024-04-20T21:19:14.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.302+0000 7faa93e00640 1 -- 172.21.15.86:0/2623797026 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa94160710 con 0x7faa94162c40 2024-04-20T21:19:14.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.303+0000 7faa93e00640 1 -- 172.21.15.86:0/2623797026 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa9404fcf0 con 0x7faa94162c40 2024-04-20T21:19:14.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.304+0000 7faa8b400640 1 -- 172.21.15.86:0/2623797026 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faa8c01a070 con 0x7faa94162c40 2024-04-20T21:19:14.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.305+0000 7faa8b400640 1 --2- 172.21.15.86:0/2623797026 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7faa7006c920 0x7faa7006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:14.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.305+0000 7faa8b400640 1 -- 172.21.15.86:0/2623797026 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7faa8c08aed0 con 0x7faa94162c40 2024-04-20T21:19:14.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.305+0000 7faa92a00640 1 --2- 172.21.15.86:0/2623797026 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7faa7006c920 0x7faa7006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:14.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.306+0000 7faa92a00640 1 --2- 172.21.15.86:0/2623797026 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7faa7006c920 0x7faa7006ede0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7faa84036000 tx=0x7faa8402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:14.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.309+0000 7faa8b400640 1 -- 172.21.15.86:0/2623797026 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7faa8c058c10 con 0x7faa94162c40 2024-04-20T21:19:14.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.510+0000 7faa93e00640 1 -- 172.21.15.86:0/2623797026 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faa94147130 con 0x7faa94162c40 2024-04-20T21:19:14.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.513+0000 7faa8b400640 1 -- 172.21.15.86:0/2623797026 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16669 (secure 0 0 0) 0x7faa8c05cf10 con 0x7faa94162c40 2024-04-20T21:19:14.512 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:19:14.512 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":33,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:19:11.559613+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:19:07.732422+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-20T21:19:08.960268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:19:14.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.515+0000 7faa93e00640 1 -- 172.21.15.86:0/2623797026 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7faa7006c920 msgr2=0x7faa7006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:14.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.515+0000 7faa93e00640 1 --2- 172.21.15.86:0/2623797026 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7faa7006c920 0x7faa7006ede0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7faa84036000 tx=0x7faa8402e000 comp rx=0 tx=0).stop 2024-04-20T21:19:14.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.515+0000 7faa93e00640 1 -- 172.21.15.86:0/2623797026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa94162c40 msgr2=0x7faa94146bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:14.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.515+0000 7faa93e00640 1 --2- 172.21.15.86:0/2623797026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa94162c40 0x7faa94146bf0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7faa8c00c420 tx=0x7faa8c00c8e0 comp rx=0 tx=0).stop 2024-04-20T21:19:14.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.515+0000 7faa93e00640 1 -- 172.21.15.86:0/2623797026 shutdown_connections 2024-04-20T21:19:14.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.515+0000 7faa93e00640 1 --2- 172.21.15.86:0/2623797026 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7faa7006c920 0x7faa7006ede0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:14.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.516+0000 7faa93e00640 1 --2- 172.21.15.86:0/2623797026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa94162c40 0x7faa94146bf0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:14.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.516+0000 7faa93e00640 1 --2- 172.21.15.86:0/2623797026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa940b3810 0x7faa941f4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:14.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.516+0000 7faa93e00640 1 --2- 172.21.15.86:0/2623797026 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7faa940af8d0 0x7faa940b0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:14.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.516+0000 7faa93e00640 1 -- 172.21.15.86:0/2623797026 >> 172.21.15.86:0/2623797026 conn(0x7faa940d0170 msgr2=0x7faa94142620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:14.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.516+0000 7faa93e00640 1 -- 172.21.15.86:0/2623797026 shutdown_connections 2024-04-20T21:19:14.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.516+0000 7faa93e00640 1 -- 172.21.15.86:0/2623797026 wait complete. 2024-04-20T21:19:14.534 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-20T21:19:14.535 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T21:19:14.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.739+0000 7f0a45200640 1 Processor -- start 2024-04-20T21:19:14.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.739+0000 7f0a45200640 1 -- start start 2024-04-20T21:19:14.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.741+0000 7f0a45200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a4015e950 0x7f0a400ce8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:14.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.741+0000 7f0a45200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0a4006c090 0x7f0a400b28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:14.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.741+0000 7f0a45200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a400cd840 con 0x7f0a400b24d0 2024-04-20T21:19:14.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.741+0000 7f0a45200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f0a400cd610 con 0x7f0a4006c090 2024-04-20T21:19:14.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.741+0000 7f0a45200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0a400cf350 con 0x7f0a4015e950 2024-04-20T21:19:14.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.741+0000 7f0a3de00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a4015e950 0x7f0a400ce8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:14.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.742+0000 7f0a3de00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a4015e950 0x7f0a400ce8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:50818/0 (socket says 172.21.15.86:50818) 2024-04-20T21:19:14.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.742+0000 7f0a3de00640 1 -- 172.21.15.86:0/3035968067 learned_addr learned my addr 172.21.15.86:0/3035968067 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:14.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.742+0000 7f0a3e800640 1 --2- 172.21.15.86:0/3035968067 >> v2:172.21.15.144:3300/0 conn(0x7f0a4006c090 0x7f0a400b28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:14.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.742+0000 7f0a3de00640 1 -- 172.21.15.86:0/3035968067 >> v2:172.21.15.144:3300/0 conn(0x7f0a4006c090 msgr2=0x7f0a400b28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:14.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.742+0000 7f0a3de00640 1 --2- 172.21.15.86:0/3035968067 >> v2:172.21.15.144:3300/0 conn(0x7f0a4006c090 0x7f0a400b28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:14.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.742+0000 7f0a3de00640 1 -- 172.21.15.86:0/3035968067 >> v1:172.21.15.144:6789/0 conn(0x7f0a400b24d0 legacy=0x7f0a4006b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:14.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.742+0000 7f0a3de00640 1 -- 172.21.15.86:0/3035968067 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a400cdb20 con 0x7f0a4015e950 2024-04-20T21:19:14.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.744+0000 7f0a3de00640 1 --2- 172.21.15.86:0/3035968067 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a4015e950 0x7f0a400ce8e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0a30009870 tx=0x7f0a3002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9850e9f3ef1de450 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:14.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.744+0000 7f0a3d400640 1 -- 172.21.15.86:0/3035968067 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a30034070 con 0x7f0a4015e950 2024-04-20T21:19:14.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.744+0000 7f0a3d400640 1 -- 172.21.15.86:0/3035968067 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a30030840 con 0x7f0a4015e950 2024-04-20T21:19:14.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.744+0000 7f0a3d400640 1 -- 172.21.15.86:0/3035968067 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a30030c10 con 0x7f0a4015e950 2024-04-20T21:19:14.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.745+0000 7f0a45200640 1 -- 172.21.15.86:0/3035968067 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a4015e950 msgr2=0x7f0a400ce8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:14.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.745+0000 7f0a45200640 1 --2- 172.21.15.86:0/3035968067 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a4015e950 0x7f0a400ce8e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0a30009870 tx=0x7f0a3002b980 comp rx=0 tx=0).stop 2024-04-20T21:19:14.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.745+0000 7f0a45200640 1 -- 172.21.15.86:0/3035968067 shutdown_connections 2024-04-20T21:19:14.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.745+0000 7f0a45200640 1 --2- 172.21.15.86:0/3035968067 >> v2:172.21.15.144:3300/0 conn(0x7f0a4006c090 0x7f0a400b28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:14.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.745+0000 7f0a45200640 1 --2- 172.21.15.86:0/3035968067 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a4015e950 0x7f0a400ce8e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:14.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.745+0000 7f0a45200640 1 -- 172.21.15.86:0/3035968067 >> 172.21.15.86:0/3035968067 conn(0x7f0a4009a9e0 msgr2=0x7f0a400b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:14.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.745+0000 7f0a45200640 1 -- 172.21.15.86:0/3035968067 shutdown_connections 2024-04-20T21:19:14.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.745+0000 7f0a45200640 1 -- 172.21.15.86:0/3035968067 wait complete. 2024-04-20T21:19:14.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.747+0000 7f0a45200640 1 Processor -- start 2024-04-20T21:19:14.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.747+0000 7f0a45200640 1 -- start start 2024-04-20T21:19:14.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.748+0000 7f0a45200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a4006c090 0x7f0a40158130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:14.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.748+0000 7f0a45200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a400b24d0 0x7f0a400b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:14.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.748+0000 7f0a45200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a4015e950 0x7f0a4015c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:14.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.748+0000 7f0a45200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0a400ce3c0 con 0x7f0a4015e950 2024-04-20T21:19:14.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.748+0000 7f0a45200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0a400ce0e0 con 0x7f0a400b24d0 2024-04-20T21:19:14.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.748+0000 7f0a45200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0a4015ed30 con 0x7f0a4006c090 2024-04-20T21:19:14.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.748+0000 7f0a3e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a4015e950 0x7f0a4015c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:14.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.748+0000 7f0a3e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a4015e950 0x7f0a4015c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42272/0 (socket says 172.21.15.86:42272) 2024-04-20T21:19:14.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.748+0000 7f0a3e800640 1 -- 172.21.15.86:0/4140684733 learned_addr learned my addr 172.21.15.86:0/4140684733 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:14.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.749+0000 7f0a3de00640 1 --2- 172.21.15.86:0/4140684733 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a4006c090 0x7f0a40158130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:14.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.749+0000 7f0a35400640 1 --2- 172.21.15.86:0/4140684733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a400b24d0 0x7f0a400b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:14.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.749+0000 7f0a3e800640 1 -- 172.21.15.86:0/4140684733 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a4006c090 msgr2=0x7f0a40158130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:14.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.749+0000 7f0a3e800640 1 --2- 172.21.15.86:0/4140684733 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a4006c090 0x7f0a40158130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:14.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.749+0000 7f0a3e800640 1 -- 172.21.15.86:0/4140684733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a400b24d0 msgr2=0x7f0a400b4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:14.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.749+0000 7f0a3e800640 1 --2- 172.21.15.86:0/4140684733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a400b24d0 0x7f0a400b4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:14.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.749+0000 7f0a3e800640 1 -- 172.21.15.86:0/4140684733 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a40158670 con 0x7f0a4015e950 2024-04-20T21:19:14.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.749+0000 7f0a35400640 1 --2- 172.21.15.86:0/4140684733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a400b24d0 0x7f0a400b4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:14.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.749+0000 7f0a3e800640 1 --2- 172.21.15.86:0/4140684733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a4015e950 0x7f0a4015c640 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0a3800efc0 tx=0x7f0a3800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:14.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.749+0000 7f0a36a00640 1 -- 172.21.15.86:0/4140684733 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a38018020 con 0x7f0a4015e950 2024-04-20T21:19:14.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.749+0000 7f0a36a00640 1 -- 172.21.15.86:0/4140684733 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a38014480 con 0x7f0a4015e950 2024-04-20T21:19:14.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.749+0000 7f0a36a00640 1 -- 172.21.15.86:0/4140684733 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a38014870 con 0x7f0a4015e950 2024-04-20T21:19:14.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.749+0000 7f0a45200640 1 -- 172.21.15.86:0/4140684733 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a400c5050 con 0x7f0a4015e950 2024-04-20T21:19:14.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.750+0000 7f0a45200640 1 -- 172.21.15.86:0/4140684733 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a400c5610 con 0x7f0a4015e950 2024-04-20T21:19:14.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.751+0000 7f0a45200640 1 -- 172.21.15.86:0/4140684733 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a400689e0 con 0x7f0a4015e950 2024-04-20T21:19:14.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.752+0000 7f0a36a00640 1 -- 172.21.15.86:0/4140684733 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0a38005d10 con 0x7f0a4015e950 2024-04-20T21:19:14.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.752+0000 7f0a36a00640 1 --2- 172.21.15.86:0/4140684733 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0a1c06c920 0x7f0a1c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:14.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.753+0000 7f0a36a00640 1 -- 172.21.15.86:0/4140684733 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f0a3808c650 con 0x7f0a4015e950 2024-04-20T21:19:14.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.753+0000 7f0a3de00640 1 --2- 172.21.15.86:0/4140684733 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0a1c06c920 0x7f0a1c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:14.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.753+0000 7f0a3de00640 1 --2- 172.21.15.86:0/4140684733 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0a1c06c920 0x7f0a1c06ede0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0a30036000 tx=0x7f0a30008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:14.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.757+0000 7f0a36a00640 1 -- 172.21.15.86:0/4140684733 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0a38022070 con 0x7f0a4015e950 2024-04-20T21:19:14.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.950+0000 7f0a45200640 1 -- 172.21.15.86:0/4140684733 --> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0a400b4890 con 0x7f0a1c06c920 2024-04-20T21:19:14.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.959+0000 7f0a36a00640 1 -- 172.21.15.86:0/4140684733 <== mgr.4107 v2:172.21.15.86:6832/302045240 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251879 (secure 0 0 0) 0x7f0a400b4890 con 0x7f0a1c06c920 2024-04-20T21:19:14.962 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:19:14.963 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-20T21:19:14.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.965+0000 7f0a45200640 1 -- 172.21.15.86:0/4140684733 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0a1c06c920 msgr2=0x7f0a1c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:14.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.965+0000 7f0a45200640 1 --2- 172.21.15.86:0/4140684733 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0a1c06c920 0x7f0a1c06ede0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0a30036000 tx=0x7f0a30008030 comp rx=0 tx=0).stop 2024-04-20T21:19:14.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.965+0000 7f0a45200640 1 -- 172.21.15.86:0/4140684733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a4015e950 msgr2=0x7f0a4015c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:14.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.965+0000 7f0a45200640 1 --2- 172.21.15.86:0/4140684733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a4015e950 0x7f0a4015c640 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0a3800efc0 tx=0x7f0a3800c490 comp rx=0 tx=0).stop 2024-04-20T21:19:14.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.965+0000 7f0a45200640 1 -- 172.21.15.86:0/4140684733 shutdown_connections 2024-04-20T21:19:14.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.965+0000 7f0a45200640 1 --2- 172.21.15.86:0/4140684733 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0a1c06c920 0x7f0a1c06ede0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:14.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.965+0000 7f0a45200640 1 --2- 172.21.15.86:0/4140684733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a4015e950 0x7f0a4015c640 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:14.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.965+0000 7f0a45200640 1 --2- 172.21.15.86:0/4140684733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a400b24d0 0x7f0a400b4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:14.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.965+0000 7f0a45200640 1 --2- 172.21.15.86:0/4140684733 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a4006c090 0x7f0a40158130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:14.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.965+0000 7f0a45200640 1 -- 172.21.15.86:0/4140684733 >> 172.21.15.86:0/4140684733 conn(0x7f0a4009a9e0 msgr2=0x7f0a40070370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:14.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.965+0000 7f0a45200640 1 -- 172.21.15.86:0/4140684733 shutdown_connections 2024-04-20T21:19:14.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:14.965+0000 7f0a45200640 1 -- 172.21.15.86:0/4140684733 wait complete. 2024-04-20T21:19:14.982 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":73,"stamp":"2024-04-20T21:19:14.655067+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":69,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220108,"kb_used_data":3920,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754612,"statfs":{"total":773094113280,"available":772868722688,"internally_reserved":0,"allocated":4014080,"data_stored":2264184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001301"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.582138+0000","last_change":"2024-04-20T21:19:10.576352+0000","last_active":"2024-04-20T21:19:10.582138+0000","last_peered":"2024-04-20T21:19:10.582138+0000","last_clean":"2024-04-20T21:19:10.582138+0000","last_became_active":"2024-04-20T21:19:10.572686+0000","last_became_peered":"2024-04-20T21:19:10.572686+0000","last_unstale":"2024-04-20T21:19:10.582138+0000","last_undegraded":"2024-04-20T21:19:10.582138+0000","last_fullsized":"2024-04-20T21:19:10.582138+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:21:07.056413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.580263+0000","last_change":"2024-04-20T21:19:10.569472+0000","last_active":"2024-04-20T21:19:10.580263+0000","last_peered":"2024-04-20T21:19:10.580263+0000","last_clean":"2024-04-20T21:19:10.580263+0000","last_became_active":"2024-04-20T21:19:10.569195+0000","last_became_peered":"2024-04-20T21:19:10.569195+0000","last_unstale":"2024-04-20T21:19:10.580263+0000","last_undegraded":"2024-04-20T21:19:10.580263+0000","last_fullsized":"2024-04-20T21:19:10.580263+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:38:47.932291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.592435+0000","last_change":"2024-04-20T21:19:10.591084+0000","last_active":"2024-04-20T21:19:10.592435+0000","last_peered":"2024-04-20T21:19:10.592435+0000","last_clean":"2024-04-20T21:19:10.592435+0000","last_became_active":"2024-04-20T21:19:10.589939+0000","last_became_peered":"2024-04-20T21:19:10.589939+0000","last_unstale":"2024-04-20T21:19:10.592435+0000","last_undegraded":"2024-04-20T21:19:10.592435+0000","last_fullsized":"2024-04-20T21:19:10.592435+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:26:50.833532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.586058+0000","last_change":"2024-04-20T21:19:10.580766+0000","last_active":"2024-04-20T21:19:10.586058+0000","last_peered":"2024-04-20T21:19:10.586058+0000","last_clean":"2024-04-20T21:19:10.586058+0000","last_became_active":"2024-04-20T21:19:10.580370+0000","last_became_peered":"2024-04-20T21:19:10.580370+0000","last_unstale":"2024-04-20T21:19:10.586058+0000","last_undegraded":"2024-04-20T21:19:10.586058+0000","last_fullsized":"2024-04-20T21:19:10.586058+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:51:23.232062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.576872+0000","last_change":"2024-04-20T21:19:10.571809+0000","last_active":"2024-04-20T21:19:10.576872+0000","last_peered":"2024-04-20T21:19:10.576872+0000","last_clean":"2024-04-20T21:19:10.576872+0000","last_became_active":"2024-04-20T21:19:10.571446+0000","last_became_peered":"2024-04-20T21:19:10.571446+0000","last_unstale":"2024-04-20T21:19:10.576872+0000","last_undegraded":"2024-04-20T21:19:10.576872+0000","last_fullsized":"2024-04-20T21:19:10.576872+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:30:30.751151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.585539+0000","last_change":"2024-04-20T21:19:10.578313+0000","last_active":"2024-04-20T21:19:10.585539+0000","last_peered":"2024-04-20T21:19:10.585539+0000","last_clean":"2024-04-20T21:19:10.585539+0000","last_became_active":"2024-04-20T21:19:10.578025+0000","last_became_peered":"2024-04-20T21:19:10.578025+0000","last_unstale":"2024-04-20T21:19:10.585539+0000","last_undegraded":"2024-04-20T21:19:10.585539+0000","last_fullsized":"2024-04-20T21:19:10.585539+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:57:23.354844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.588335+0000","last_change":"2024-04-20T21:19:10.579851+0000","last_active":"2024-04-20T21:19:10.588335+0000","last_peered":"2024-04-20T21:19:10.588335+0000","last_clean":"2024-04-20T21:19:10.588335+0000","last_became_active":"2024-04-20T21:19:10.578964+0000","last_became_peered":"2024-04-20T21:19:10.578964+0000","last_unstale":"2024-04-20T21:19:10.588335+0000","last_undegraded":"2024-04-20T21:19:10.588335+0000","last_fullsized":"2024-04-20T21:19:10.588335+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:07:21.719322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.588144+0000","last_change":"2024-04-20T21:19:10.578080+0000","last_active":"2024-04-20T21:19:10.588144+0000","last_peered":"2024-04-20T21:19:10.588144+0000","last_clean":"2024-04-20T21:19:10.588144+0000","last_became_active":"2024-04-20T21:19:10.576963+0000","last_became_peered":"2024-04-20T21:19:10.576963+0000","last_unstale":"2024-04-20T21:19:10.588144+0000","last_undegraded":"2024-04-20T21:19:10.588144+0000","last_fullsized":"2024-04-20T21:19:10.588144+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:39:05.486077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.592015+0000","last_change":"2024-04-20T21:19:10.587702+0000","last_active":"2024-04-20T21:19:10.592015+0000","last_peered":"2024-04-20T21:19:10.592015+0000","last_clean":"2024-04-20T21:19:10.592015+0000","last_became_active":"2024-04-20T21:19:10.586205+0000","last_became_peered":"2024-04-20T21:19:10.586205+0000","last_unstale":"2024-04-20T21:19:10.592015+0000","last_undegraded":"2024-04-20T21:19:10.592015+0000","last_fullsized":"2024-04-20T21:19:10.592015+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:54:37.319387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.571489+0000","last_change":"2024-04-20T21:19:10.563877+0000","last_active":"2024-04-20T21:19:10.571489+0000","last_peered":"2024-04-20T21:19:10.571489+0000","last_clean":"2024-04-20T21:19:10.571489+0000","last_became_active":"2024-04-20T21:19:10.563438+0000","last_became_peered":"2024-04-20T21:19:10.563438+0000","last_unstale":"2024-04-20T21:19:10.571489+0000","last_undegraded":"2024-04-20T21:19:10.571489+0000","last_fullsized":"2024-04-20T21:19:10.571489+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:34:27.830777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.591026+0000","last_change":"2024-04-20T21:19:10.587379+0000","last_active":"2024-04-20T21:19:10.591026+0000","last_peered":"2024-04-20T21:19:10.591026+0000","last_clean":"2024-04-20T21:19:10.591026+0000","last_became_active":"2024-04-20T21:19:10.587057+0000","last_became_peered":"2024-04-20T21:19:10.587057+0000","last_unstale":"2024-04-20T21:19:10.591026+0000","last_undegraded":"2024-04-20T21:19:10.591026+0000","last_fullsized":"2024-04-20T21:19:10.591026+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:38:47.932291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.572558+0000","last_change":"2024-04-20T21:19:10.563725+0000","last_active":"2024-04-20T21:19:10.572558+0000","last_peered":"2024-04-20T21:19:10.572558+0000","last_clean":"2024-04-20T21:19:10.572558+0000","last_became_active":"2024-04-20T21:19:10.563427+0000","last_became_peered":"2024-04-20T21:19:10.563427+0000","last_unstale":"2024-04-20T21:19:10.572558+0000","last_undegraded":"2024-04-20T21:19:10.572558+0000","last_fullsized":"2024-04-20T21:19:10.572558+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:39:05.486077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.568373+0000","last_change":"2024-04-20T21:19:10.563737+0000","last_active":"2024-04-20T21:19:10.568373+0000","last_peered":"2024-04-20T21:19:10.568373+0000","last_clean":"2024-04-20T21:19:10.568373+0000","last_became_active":"2024-04-20T21:19:10.563419+0000","last_became_peered":"2024-04-20T21:19:10.563419+0000","last_unstale":"2024-04-20T21:19:10.568373+0000","last_undegraded":"2024-04-20T21:19:10.568373+0000","last_fullsized":"2024-04-20T21:19:10.568373+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:13:44.852599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.568332+0000","last_change":"2024-04-20T21:19:10.560575+0000","last_active":"2024-04-20T21:19:10.568332+0000","last_peered":"2024-04-20T21:19:10.568332+0000","last_clean":"2024-04-20T21:19:10.568332+0000","last_became_active":"2024-04-20T21:19:10.560313+0000","last_became_peered":"2024-04-20T21:19:10.560313+0000","last_unstale":"2024-04-20T21:19:10.568332+0000","last_undegraded":"2024-04-20T21:19:10.568332+0000","last_fullsized":"2024-04-20T21:19:10.568332+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:10:39.203202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.591289+0000","last_change":"2024-04-20T21:19:10.581154+0000","last_active":"2024-04-20T21:19:10.591289+0000","last_peered":"2024-04-20T21:19:10.591289+0000","last_clean":"2024-04-20T21:19:10.591289+0000","last_became_active":"2024-04-20T21:19:10.580796+0000","last_became_peered":"2024-04-20T21:19:10.580796+0000","last_unstale":"2024-04-20T21:19:10.591289+0000","last_undegraded":"2024-04-20T21:19:10.591289+0000","last_fullsized":"2024-04-20T21:19:10.591289+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:38:47.932291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.588344+0000","last_change":"2024-04-20T21:19:10.582619+0000","last_active":"2024-04-20T21:19:10.588344+0000","last_peered":"2024-04-20T21:19:10.588344+0000","last_clean":"2024-04-20T21:19:10.588344+0000","last_became_active":"2024-04-20T21:19:10.582291+0000","last_became_peered":"2024-04-20T21:19:10.582291+0000","last_unstale":"2024-04-20T21:19:10.588344+0000","last_undegraded":"2024-04-20T21:19:10.588344+0000","last_fullsized":"2024-04-20T21:19:10.588344+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:57:30.115297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.590663+0000","last_change":"2024-04-20T21:19:10.585810+0000","last_active":"2024-04-20T21:19:10.590663+0000","last_peered":"2024-04-20T21:19:10.590663+0000","last_clean":"2024-04-20T21:19:10.590663+0000","last_became_active":"2024-04-20T21:19:10.585504+0000","last_became_peered":"2024-04-20T21:19:10.585504+0000","last_unstale":"2024-04-20T21:19:10.590663+0000","last_undegraded":"2024-04-20T21:19:10.590663+0000","last_fullsized":"2024-04-20T21:19:10.590663+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:29:35.481155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.580284+0000","last_change":"2024-04-20T21:19:10.570881+0000","last_active":"2024-04-20T21:19:10.580284+0000","last_peered":"2024-04-20T21:19:10.580284+0000","last_clean":"2024-04-20T21:19:10.580284+0000","last_became_active":"2024-04-20T21:19:10.568851+0000","last_became_peered":"2024-04-20T21:19:10.568851+0000","last_unstale":"2024-04-20T21:19:10.580284+0000","last_undegraded":"2024-04-20T21:19:10.580284+0000","last_fullsized":"2024-04-20T21:19:10.580284+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:25:42.400241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.575132+0000","last_change":"2024-04-20T21:19:10.561188+0000","last_active":"2024-04-20T21:19:10.575132+0000","last_peered":"2024-04-20T21:19:10.575132+0000","last_clean":"2024-04-20T21:19:10.575132+0000","last_became_active":"2024-04-20T21:19:10.560920+0000","last_became_peered":"2024-04-20T21:19:10.560920+0000","last_unstale":"2024-04-20T21:19:10.575132+0000","last_undegraded":"2024-04-20T21:19:10.575132+0000","last_fullsized":"2024-04-20T21:19:10.575132+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:29:35.481155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.576896+0000","last_change":"2024-04-20T21:19:10.573630+0000","last_active":"2024-04-20T21:19:10.576896+0000","last_peered":"2024-04-20T21:19:10.576896+0000","last_clean":"2024-04-20T21:19:10.576896+0000","last_became_active":"2024-04-20T21:19:10.573363+0000","last_became_peered":"2024-04-20T21:19:10.573363+0000","last_unstale":"2024-04-20T21:19:10.576896+0000","last_undegraded":"2024-04-20T21:19:10.576896+0000","last_fullsized":"2024-04-20T21:19:10.576896+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:43:40.142483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.592099+0000","last_change":"2024-04-20T21:19:10.585234+0000","last_active":"2024-04-20T21:19:10.592099+0000","last_peered":"2024-04-20T21:19:10.592099+0000","last_clean":"2024-04-20T21:19:10.592099+0000","last_became_active":"2024-04-20T21:19:10.583464+0000","last_became_peered":"2024-04-20T21:19:10.583464+0000","last_unstale":"2024-04-20T21:19:10.592099+0000","last_undegraded":"2024-04-20T21:19:10.592099+0000","last_fullsized":"2024-04-20T21:19:10.592099+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:34:27.830777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.593744+0000","last_change":"2024-04-20T21:19:10.590871+0000","last_active":"2024-04-20T21:19:10.593744+0000","last_peered":"2024-04-20T21:19:10.593744+0000","last_clean":"2024-04-20T21:19:10.593744+0000","last_became_active":"2024-04-20T21:19:10.590605+0000","last_became_peered":"2024-04-20T21:19:10.590605+0000","last_unstale":"2024-04-20T21:19:10.593744+0000","last_undegraded":"2024-04-20T21:19:10.593744+0000","last_fullsized":"2024-04-20T21:19:10.593744+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:51:23.232062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.580430+0000","last_change":"2024-04-20T21:19:10.578864+0000","last_active":"2024-04-20T21:19:10.580430+0000","last_peered":"2024-04-20T21:19:10.580430+0000","last_clean":"2024-04-20T21:19:10.580430+0000","last_became_active":"2024-04-20T21:19:10.578560+0000","last_became_peered":"2024-04-20T21:19:10.578560+0000","last_unstale":"2024-04-20T21:19:10.580430+0000","last_undegraded":"2024-04-20T21:19:10.580430+0000","last_fullsized":"2024-04-20T21:19:10.580430+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:51:23.232062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.576391+0000","last_change":"2024-04-20T21:19:10.568598+0000","last_active":"2024-04-20T21:19:10.576391+0000","last_peered":"2024-04-20T21:19:10.576391+0000","last_clean":"2024-04-20T21:19:10.576391+0000","last_became_active":"2024-04-20T21:19:10.568178+0000","last_became_peered":"2024-04-20T21:19:10.568178+0000","last_unstale":"2024-04-20T21:19:10.576391+0000","last_undegraded":"2024-04-20T21:19:10.576391+0000","last_fullsized":"2024-04-20T21:19:10.576391+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:37:09.020189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.589843+0000","last_change":"2024-04-20T21:19:10.585546+0000","last_active":"2024-04-20T21:19:10.589843+0000","last_peered":"2024-04-20T21:19:10.589843+0000","last_clean":"2024-04-20T21:19:10.589843+0000","last_became_active":"2024-04-20T21:19:10.583941+0000","last_became_peered":"2024-04-20T21:19:10.583941+0000","last_unstale":"2024-04-20T21:19:10.589843+0000","last_undegraded":"2024-04-20T21:19:10.589843+0000","last_fullsized":"2024-04-20T21:19:10.589843+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:57:30.115297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.580302+0000","last_change":"2024-04-20T21:19:10.570110+0000","last_active":"2024-04-20T21:19:10.580302+0000","last_peered":"2024-04-20T21:19:10.580302+0000","last_clean":"2024-04-20T21:19:10.580302+0000","last_became_active":"2024-04-20T21:19:10.569836+0000","last_became_peered":"2024-04-20T21:19:10.569836+0000","last_unstale":"2024-04-20T21:19:10.580302+0000","last_undegraded":"2024-04-20T21:19:10.580302+0000","last_fullsized":"2024-04-20T21:19:10.580302+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:13:08.589597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.589861+0000","last_change":"2024-04-20T21:19:10.586097+0000","last_active":"2024-04-20T21:19:10.589861+0000","last_peered":"2024-04-20T21:19:10.589861+0000","last_clean":"2024-04-20T21:19:10.589861+0000","last_became_active":"2024-04-20T21:19:10.583964+0000","last_became_peered":"2024-04-20T21:19:10.583964+0000","last_unstale":"2024-04-20T21:19:10.589861+0000","last_undegraded":"2024-04-20T21:19:10.589861+0000","last_fullsized":"2024-04-20T21:19:10.589861+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:07:21.719322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.576347+0000","last_change":"2024-04-20T21:19:10.569345+0000","last_active":"2024-04-20T21:19:10.576347+0000","last_peered":"2024-04-20T21:19:10.576347+0000","last_clean":"2024-04-20T21:19:10.576347+0000","last_became_active":"2024-04-20T21:19:10.568842+0000","last_became_peered":"2024-04-20T21:19:10.568842+0000","last_unstale":"2024-04-20T21:19:10.576347+0000","last_undegraded":"2024-04-20T21:19:10.576347+0000","last_fullsized":"2024-04-20T21:19:10.576347+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:57:30.115297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.593599+0000","last_change":"2024-04-20T21:19:10.589230+0000","last_active":"2024-04-20T21:19:10.593599+0000","last_peered":"2024-04-20T21:19:10.593599+0000","last_clean":"2024-04-20T21:19:10.593599+0000","last_became_active":"2024-04-20T21:19:10.588970+0000","last_became_peered":"2024-04-20T21:19:10.588970+0000","last_unstale":"2024-04-20T21:19:10.593599+0000","last_undegraded":"2024-04-20T21:19:10.593599+0000","last_fullsized":"2024-04-20T21:19:10.593599+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:39:05.486077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.587976+0000","last_change":"2024-04-20T21:19:10.579024+0000","last_active":"2024-04-20T21:19:10.587976+0000","last_peered":"2024-04-20T21:19:10.587976+0000","last_clean":"2024-04-20T21:19:10.587976+0000","last_became_active":"2024-04-20T21:19:10.578750+0000","last_became_peered":"2024-04-20T21:19:10.578750+0000","last_unstale":"2024-04-20T21:19:10.587976+0000","last_undegraded":"2024-04-20T21:19:10.587976+0000","last_fullsized":"2024-04-20T21:19:10.587976+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:36:44.756836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.575351+0000","last_change":"2024-04-20T21:19:10.565245+0000","last_active":"2024-04-20T21:19:10.575351+0000","last_peered":"2024-04-20T21:19:10.575351+0000","last_clean":"2024-04-20T21:19:10.575351+0000","last_became_active":"2024-04-20T21:19:10.564910+0000","last_became_peered":"2024-04-20T21:19:10.564910+0000","last_unstale":"2024-04-20T21:19:10.575351+0000","last_undegraded":"2024-04-20T21:19:10.575351+0000","last_fullsized":"2024-04-20T21:19:10.575351+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:26:48.848073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.593674+0000","last_change":"2024-04-20T21:19:10.589881+0000","last_active":"2024-04-20T21:19:10.593674+0000","last_peered":"2024-04-20T21:19:10.593674+0000","last_clean":"2024-04-20T21:19:10.593674+0000","last_became_active":"2024-04-20T21:19:10.587418+0000","last_became_peered":"2024-04-20T21:19:10.587418+0000","last_unstale":"2024-04-20T21:19:10.593674+0000","last_undegraded":"2024-04-20T21:19:10.593674+0000","last_fullsized":"2024-04-20T21:19:10.593674+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:13:08.589597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.579718+0000","last_change":"2024-04-20T21:19:10.572752+0000","last_active":"2024-04-20T21:19:10.579718+0000","last_peered":"2024-04-20T21:19:10.579718+0000","last_clean":"2024-04-20T21:19:10.579718+0000","last_became_active":"2024-04-20T21:19:10.572410+0000","last_became_peered":"2024-04-20T21:19:10.572410+0000","last_unstale":"2024-04-20T21:19:10.579718+0000","last_undegraded":"2024-04-20T21:19:10.579718+0000","last_fullsized":"2024-04-20T21:19:10.579718+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:54:37.319387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.592512+0000","last_change":"2024-04-20T21:19:09.552557+0000","last_active":"2024-04-20T21:19:10.592512+0000","last_peered":"2024-04-20T21:19:10.592512+0000","last_clean":"2024-04-20T21:19:10.592512+0000","last_became_active":"2024-04-20T21:19:09.552209+0000","last_became_peered":"2024-04-20T21:19:09.552209+0000","last_unstale":"2024-04-20T21:19:10.592512+0000","last_undegraded":"2024-04-20T21:19:10.592512+0000","last_fullsized":"2024-04-20T21:19:10.592512+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:09:51.220179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.582159+0000","last_change":"2024-04-20T21:19:10.577010+0000","last_active":"2024-04-20T21:19:10.582159+0000","last_peered":"2024-04-20T21:19:10.582159+0000","last_clean":"2024-04-20T21:19:10.582159+0000","last_became_active":"2024-04-20T21:19:10.576148+0000","last_became_peered":"2024-04-20T21:19:10.576148+0000","last_unstale":"2024-04-20T21:19:10.582159+0000","last_undegraded":"2024-04-20T21:19:10.582159+0000","last_fullsized":"2024-04-20T21:19:10.582159+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:13:23.731257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.579585+0000","last_change":"2024-04-20T21:19:09.546615+0000","last_active":"2024-04-20T21:19:10.579585+0000","last_peered":"2024-04-20T21:19:10.579585+0000","last_clean":"2024-04-20T21:19:10.579585+0000","last_became_active":"2024-04-20T21:19:09.546318+0000","last_became_peered":"2024-04-20T21:19:09.546318+0000","last_unstale":"2024-04-20T21:19:10.579585+0000","last_undegraded":"2024-04-20T21:19:10.579585+0000","last_fullsized":"2024-04-20T21:19:10.579585+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:59:55.626583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.588154+0000","last_change":"2024-04-20T21:19:10.584575+0000","last_active":"2024-04-20T21:19:10.588154+0000","last_peered":"2024-04-20T21:19:10.588154+0000","last_clean":"2024-04-20T21:19:10.588154+0000","last_became_active":"2024-04-20T21:19:10.581940+0000","last_became_peered":"2024-04-20T21:19:10.581940+0000","last_unstale":"2024-04-20T21:19:10.588154+0000","last_undegraded":"2024-04-20T21:19:10.588154+0000","last_fullsized":"2024-04-20T21:19:10.588154+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:13:23.731257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.588632+0000","last_change":"2024-04-20T21:19:09.543512+0000","last_active":"2024-04-20T21:19:10.588632+0000","last_peered":"2024-04-20T21:19:10.588632+0000","last_clean":"2024-04-20T21:19:10.588632+0000","last_became_active":"2024-04-20T21:19:09.543220+0000","last_became_peered":"2024-04-20T21:19:09.543220+0000","last_unstale":"2024-04-20T21:19:10.588632+0000","last_undegraded":"2024-04-20T21:19:10.588632+0000","last_fullsized":"2024-04-20T21:19:10.588632+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:36:43.742787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.589836+0000","last_change":"2024-04-20T21:19:10.585454+0000","last_active":"2024-04-20T21:19:10.589836+0000","last_peered":"2024-04-20T21:19:10.589836+0000","last_clean":"2024-04-20T21:19:10.589836+0000","last_became_active":"2024-04-20T21:19:10.583609+0000","last_became_peered":"2024-04-20T21:19:10.583609+0000","last_unstale":"2024-04-20T21:19:10.589836+0000","last_undegraded":"2024-04-20T21:19:10.589836+0000","last_fullsized":"2024-04-20T21:19:10.589836+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:37:09.020189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.566139+0000","last_change":"2024-04-20T21:19:09.543512+0000","last_active":"2024-04-20T21:19:10.566139+0000","last_peered":"2024-04-20T21:19:10.566139+0000","last_clean":"2024-04-20T21:19:10.566139+0000","last_became_active":"2024-04-20T21:19:09.541949+0000","last_became_peered":"2024-04-20T21:19:09.541949+0000","last_unstale":"2024-04-20T21:19:10.566139+0000","last_undegraded":"2024-04-20T21:19:10.566139+0000","last_fullsized":"2024-04-20T21:19:10.566139+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:02:41.599087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.588311+0000","last_change":"2024-04-20T21:19:10.581639+0000","last_active":"2024-04-20T21:19:10.588311+0000","last_peered":"2024-04-20T21:19:10.588311+0000","last_clean":"2024-04-20T21:19:10.588311+0000","last_became_active":"2024-04-20T21:19:10.581328+0000","last_became_peered":"2024-04-20T21:19:10.581328+0000","last_unstale":"2024-04-20T21:19:10.588311+0000","last_undegraded":"2024-04-20T21:19:10.588311+0000","last_fullsized":"2024-04-20T21:19:10.588311+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:37:09.020189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.592544+0000","last_change":"2024-04-20T21:19:09.538794+0000","last_active":"2024-04-20T21:19:10.592544+0000","last_peered":"2024-04-20T21:19:10.592544+0000","last_clean":"2024-04-20T21:19:10.592544+0000","last_became_active":"2024-04-20T21:19:09.538483+0000","last_became_peered":"2024-04-20T21:19:09.538483+0000","last_unstale":"2024-04-20T21:19:10.592544+0000","last_undegraded":"2024-04-20T21:19:10.592544+0000","last_fullsized":"2024-04-20T21:19:10.592544+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:39:40.786910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.585676+0000","last_change":"2024-04-20T21:19:10.576637+0000","last_active":"2024-04-20T21:19:10.585676+0000","last_peered":"2024-04-20T21:19:10.585676+0000","last_clean":"2024-04-20T21:19:10.585676+0000","last_became_active":"2024-04-20T21:19:10.574494+0000","last_became_peered":"2024-04-20T21:19:10.574494+0000","last_unstale":"2024-04-20T21:19:10.585676+0000","last_undegraded":"2024-04-20T21:19:10.585676+0000","last_fullsized":"2024-04-20T21:19:10.585676+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:29:35.481155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.592545+0000","last_change":"2024-04-20T21:19:09.537483+0000","last_active":"2024-04-20T21:19:10.592545+0000","last_peered":"2024-04-20T21:19:10.592545+0000","last_clean":"2024-04-20T21:19:10.592545+0000","last_became_active":"2024-04-20T21:19:09.537075+0000","last_became_peered":"2024-04-20T21:19:09.537075+0000","last_unstale":"2024-04-20T21:19:10.592545+0000","last_undegraded":"2024-04-20T21:19:10.592545+0000","last_fullsized":"2024-04-20T21:19:10.592545+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:09:51.220179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.586797+0000","last_change":"2024-04-20T21:19:10.579780+0000","last_active":"2024-04-20T21:19:10.586797+0000","last_peered":"2024-04-20T21:19:10.586797+0000","last_clean":"2024-04-20T21:19:10.586797+0000","last_became_active":"2024-04-20T21:19:10.579485+0000","last_became_peered":"2024-04-20T21:19:10.579485+0000","last_unstale":"2024-04-20T21:19:10.586797+0000","last_undegraded":"2024-04-20T21:19:10.586797+0000","last_fullsized":"2024-04-20T21:19:10.586797+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:26:48.848073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.544523+0000","last_change":"2024-04-20T21:19:09.540890+0000","last_active":"2024-04-20T21:19:10.544523+0000","last_peered":"2024-04-20T21:19:10.544523+0000","last_clean":"2024-04-20T21:19:10.544523+0000","last_became_active":"2024-04-20T21:19:09.540445+0000","last_became_peered":"2024-04-20T21:19:09.540445+0000","last_unstale":"2024-04-20T21:19:10.544523+0000","last_undegraded":"2024-04-20T21:19:10.544523+0000","last_fullsized":"2024-04-20T21:19:10.544523+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:13:22.717208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.586442+0000","last_change":"2024-04-20T21:19:10.578871+0000","last_active":"2024-04-20T21:19:10.586442+0000","last_peered":"2024-04-20T21:19:10.586442+0000","last_clean":"2024-04-20T21:19:10.586442+0000","last_became_active":"2024-04-20T21:19:10.578563+0000","last_became_peered":"2024-04-20T21:19:10.578563+0000","last_unstale":"2024-04-20T21:19:10.586442+0000","last_undegraded":"2024-04-20T21:19:10.586442+0000","last_fullsized":"2024-04-20T21:19:10.586442+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:54:37.319387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.569308+0000","last_change":"2024-04-20T21:19:09.551243+0000","last_active":"2024-04-20T21:19:10.569308+0000","last_peered":"2024-04-20T21:19:10.569308+0000","last_clean":"2024-04-20T21:19:10.569308+0000","last_became_active":"2024-04-20T21:19:09.550916+0000","last_became_peered":"2024-04-20T21:19:09.550916+0000","last_unstale":"2024-04-20T21:19:10.569308+0000","last_undegraded":"2024-04-20T21:19:10.569308+0000","last_fullsized":"2024-04-20T21:19:10.569308+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:00:26.773145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.582602+0000","last_change":"2024-04-20T21:19:10.573585+0000","last_active":"2024-04-20T21:19:10.582602+0000","last_peered":"2024-04-20T21:19:10.582602+0000","last_clean":"2024-04-20T21:19:10.582602+0000","last_became_active":"2024-04-20T21:19:10.573188+0000","last_became_peered":"2024-04-20T21:19:10.573188+0000","last_unstale":"2024-04-20T21:19:10.582602+0000","last_undegraded":"2024-04-20T21:19:10.582602+0000","last_fullsized":"2024-04-20T21:19:10.582602+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:10:39.203202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.545169+0000","last_change":"2024-04-20T21:19:09.548020+0000","last_active":"2024-04-20T21:19:10.545169+0000","last_peered":"2024-04-20T21:19:10.545169+0000","last_clean":"2024-04-20T21:19:10.545169+0000","last_became_active":"2024-04-20T21:19:09.547756+0000","last_became_peered":"2024-04-20T21:19:09.547756+0000","last_unstale":"2024-04-20T21:19:10.545169+0000","last_undegraded":"2024-04-20T21:19:10.545169+0000","last_fullsized":"2024-04-20T21:19:10.545169+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:39:04.472028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.568512+0000","last_change":"2024-04-20T21:19:10.558497+0000","last_active":"2024-04-20T21:19:10.568512+0000","last_peered":"2024-04-20T21:19:10.568512+0000","last_clean":"2024-04-20T21:19:10.568512+0000","last_became_active":"2024-04-20T21:19:10.558228+0000","last_became_peered":"2024-04-20T21:19:10.558228+0000","last_unstale":"2024-04-20T21:19:10.568512+0000","last_undegraded":"2024-04-20T21:19:10.568512+0000","last_fullsized":"2024-04-20T21:19:10.568512+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:57:30.115297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.556848+0000","last_change":"2024-04-20T21:19:09.535468+0000","last_active":"2024-04-20T21:19:10.556848+0000","last_peered":"2024-04-20T21:19:10.556848+0000","last_clean":"2024-04-20T21:19:10.556848+0000","last_became_active":"2024-04-20T21:19:09.535123+0000","last_became_peered":"2024-04-20T21:19:09.535123+0000","last_unstale":"2024-04-20T21:19:10.556848+0000","last_undegraded":"2024-04-20T21:19:10.556848+0000","last_fullsized":"2024-04-20T21:19:10.556848+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:13:22.717208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.580885+0000","last_change":"2024-04-20T21:19:10.570617+0000","last_active":"2024-04-20T21:19:10.580885+0000","last_peered":"2024-04-20T21:19:10.580885+0000","last_clean":"2024-04-20T21:19:10.580885+0000","last_became_active":"2024-04-20T21:19:10.570333+0000","last_became_peered":"2024-04-20T21:19:10.570333+0000","last_unstale":"2024-04-20T21:19:10.580885+0000","last_undegraded":"2024-04-20T21:19:10.580885+0000","last_fullsized":"2024-04-20T21:19:10.580885+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:26:50.833532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.590288+0000","last_change":"2024-04-20T21:19:09.552829+0000","last_active":"2024-04-20T21:19:10.590288+0000","last_peered":"2024-04-20T21:19:10.590288+0000","last_clean":"2024-04-20T21:19:10.590288+0000","last_became_active":"2024-04-20T21:19:09.552531+0000","last_became_peered":"2024-04-20T21:19:09.552531+0000","last_unstale":"2024-04-20T21:19:10.590288+0000","last_undegraded":"2024-04-20T21:19:10.590288+0000","last_fullsized":"2024-04-20T21:19:10.590288+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:36:43.742787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.592211+0000","last_change":"2024-04-20T21:19:10.588160+0000","last_active":"2024-04-20T21:19:10.592211+0000","last_peered":"2024-04-20T21:19:10.592211+0000","last_clean":"2024-04-20T21:19:10.592211+0000","last_became_active":"2024-04-20T21:19:10.586777+0000","last_became_peered":"2024-04-20T21:19:10.586777+0000","last_unstale":"2024-04-20T21:19:10.592211+0000","last_undegraded":"2024-04-20T21:19:10.592211+0000","last_fullsized":"2024-04-20T21:19:10.592211+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:31:07.052296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.597529+0000","last_change":"2024-04-20T21:19:09.544496+0000","last_active":"2024-04-20T21:19:10.597529+0000","last_peered":"2024-04-20T21:19:10.597529+0000","last_clean":"2024-04-20T21:19:10.597529+0000","last_became_active":"2024-04-20T21:19:09.544216+0000","last_became_peered":"2024-04-20T21:19:09.544216+0000","last_unstale":"2024-04-20T21:19:10.597529+0000","last_undegraded":"2024-04-20T21:19:10.597529+0000","last_fullsized":"2024-04-20T21:19:10.597529+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:37:08.006140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.587865+0000","last_change":"2024-04-20T21:19:10.583052+0000","last_active":"2024-04-20T21:19:10.587865+0000","last_peered":"2024-04-20T21:19:10.587865+0000","last_clean":"2024-04-20T21:19:10.587865+0000","last_became_active":"2024-04-20T21:19:10.582768+0000","last_became_peered":"2024-04-20T21:19:10.582768+0000","last_unstale":"2024-04-20T21:19:10.587865+0000","last_undegraded":"2024-04-20T21:19:10.587865+0000","last_fullsized":"2024-04-20T21:19:10.587865+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:36:44.756836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.592529+0000","last_change":"2024-04-20T21:19:09.545708+0000","last_active":"2024-04-20T21:19:10.592529+0000","last_peered":"2024-04-20T21:19:10.592529+0000","last_clean":"2024-04-20T21:19:10.592529+0000","last_became_active":"2024-04-20T21:19:09.545373+0000","last_became_peered":"2024-04-20T21:19:09.545373+0000","last_unstale":"2024-04-20T21:19:10.592529+0000","last_undegraded":"2024-04-20T21:19:10.592529+0000","last_fullsized":"2024-04-20T21:19:10.592529+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:51:16.698689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.582749+0000","last_change":"2024-04-20T21:19:10.570085+0000","last_active":"2024-04-20T21:19:10.582749+0000","last_peered":"2024-04-20T21:19:10.582749+0000","last_clean":"2024-04-20T21:19:10.582749+0000","last_became_active":"2024-04-20T21:19:10.569846+0000","last_became_peered":"2024-04-20T21:19:10.569846+0000","last_unstale":"2024-04-20T21:19:10.582749+0000","last_undegraded":"2024-04-20T21:19:10.582749+0000","last_fullsized":"2024-04-20T21:19:10.582749+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:31:07.052296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.552562+0000","last_change":"2024-04-20T21:19:09.549906+0000","last_active":"2024-04-20T21:19:10.552562+0000","last_peered":"2024-04-20T21:19:10.552562+0000","last_clean":"2024-04-20T21:19:10.552562+0000","last_became_active":"2024-04-20T21:19:09.549637+0000","last_became_peered":"2024-04-20T21:19:09.549637+0000","last_unstale":"2024-04-20T21:19:10.552562+0000","last_undegraded":"2024-04-20T21:19:10.552562+0000","last_fullsized":"2024-04-20T21:19:10.552562+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:21:06.042364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.592010+0000","last_change":"2024-04-20T21:19:10.588038+0000","last_active":"2024-04-20T21:19:10.592010+0000","last_peered":"2024-04-20T21:19:10.592010+0000","last_clean":"2024-04-20T21:19:10.592010+0000","last_became_active":"2024-04-20T21:19:10.587755+0000","last_became_peered":"2024-04-20T21:19:10.587755+0000","last_unstale":"2024-04-20T21:19:10.592010+0000","last_undegraded":"2024-04-20T21:19:10.592010+0000","last_fullsized":"2024-04-20T21:19:10.592010+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:31:07.052296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.544539+0000","last_change":"2024-04-20T21:19:09.547370+0000","last_active":"2024-04-20T21:19:10.544539+0000","last_peered":"2024-04-20T21:19:10.544539+0000","last_clean":"2024-04-20T21:19:10.544539+0000","last_became_active":"2024-04-20T21:19:09.547129+0000","last_became_peered":"2024-04-20T21:19:09.547129+0000","last_unstale":"2024-04-20T21:19:10.544539+0000","last_undegraded":"2024-04-20T21:19:10.544539+0000","last_fullsized":"2024-04-20T21:19:10.544539+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:34:26.816727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.588946+0000","last_change":"2024-04-20T21:19:10.580561+0000","last_active":"2024-04-20T21:19:10.588946+0000","last_peered":"2024-04-20T21:19:10.588946+0000","last_clean":"2024-04-20T21:19:10.588946+0000","last_became_active":"2024-04-20T21:19:10.580251+0000","last_became_peered":"2024-04-20T21:19:10.580251+0000","last_unstale":"2024-04-20T21:19:10.588946+0000","last_undegraded":"2024-04-20T21:19:10.588946+0000","last_fullsized":"2024-04-20T21:19:10.588946+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:34:27.830777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.588089+0000","last_change":"2024-04-20T21:19:09.543558+0000","last_active":"2024-04-20T21:19:10.588089+0000","last_peered":"2024-04-20T21:19:10.588089+0000","last_clean":"2024-04-20T21:19:10.588089+0000","last_became_active":"2024-04-20T21:19:09.543279+0000","last_became_peered":"2024-04-20T21:19:09.543279+0000","last_unstale":"2024-04-20T21:19:10.588089+0000","last_undegraded":"2024-04-20T21:19:10.588089+0000","last_fullsized":"2024-04-20T21:19:10.588089+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:51:22.218013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.586876+0000","last_change":"2024-04-20T21:19:10.582195+0000","last_active":"2024-04-20T21:19:10.586876+0000","last_peered":"2024-04-20T21:19:10.586876+0000","last_clean":"2024-04-20T21:19:10.586876+0000","last_became_active":"2024-04-20T21:19:10.581906+0000","last_became_peered":"2024-04-20T21:19:10.581906+0000","last_unstale":"2024-04-20T21:19:10.586876+0000","last_undegraded":"2024-04-20T21:19:10.586876+0000","last_fullsized":"2024-04-20T21:19:10.586876+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:59:56.640632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.585592+0000","last_change":"2024-04-20T21:19:09.540882+0000","last_active":"2024-04-20T21:19:10.585592+0000","last_peered":"2024-04-20T21:19:10.585592+0000","last_clean":"2024-04-20T21:19:10.585592+0000","last_became_active":"2024-04-20T21:19:09.540531+0000","last_became_peered":"2024-04-20T21:19:09.540531+0000","last_unstale":"2024-04-20T21:19:10.585592+0000","last_undegraded":"2024-04-20T21:19:10.585592+0000","last_fullsized":"2024-04-20T21:19:10.585592+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:26:47.834024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.582174+0000","last_change":"2024-04-20T21:19:10.572217+0000","last_active":"2024-04-20T21:19:10.582174+0000","last_peered":"2024-04-20T21:19:10.582174+0000","last_clean":"2024-04-20T21:19:10.582174+0000","last_became_active":"2024-04-20T21:19:10.571913+0000","last_became_peered":"2024-04-20T21:19:10.571913+0000","last_unstale":"2024-04-20T21:19:10.582174+0000","last_undegraded":"2024-04-20T21:19:10.582174+0000","last_fullsized":"2024-04-20T21:19:10.582174+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:07:21.719322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.590656+0000","last_change":"2024-04-20T21:19:09.551656+0000","last_active":"2024-04-20T21:19:10.590656+0000","last_peered":"2024-04-20T21:19:10.590656+0000","last_clean":"2024-04-20T21:19:10.590656+0000","last_became_active":"2024-04-20T21:19:09.551360+0000","last_became_peered":"2024-04-20T21:19:09.551360+0000","last_unstale":"2024-04-20T21:19:10.590656+0000","last_undegraded":"2024-04-20T21:19:10.590656+0000","last_fullsized":"2024-04-20T21:19:10.590656+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:09:15.513908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.586014+0000","last_change":"2024-04-20T21:19:10.580564+0000","last_active":"2024-04-20T21:19:10.586014+0000","last_peered":"2024-04-20T21:19:10.586014+0000","last_clean":"2024-04-20T21:19:10.586014+0000","last_became_active":"2024-04-20T21:19:10.580073+0000","last_became_peered":"2024-04-20T21:19:10.580073+0000","last_unstale":"2024-04-20T21:19:10.586014+0000","last_undegraded":"2024-04-20T21:19:10.586014+0000","last_fullsized":"2024-04-20T21:19:10.586014+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:39:05.486077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.590865+0000","last_change":"2024-04-20T21:19:09.540957+0000","last_active":"2024-04-20T21:19:10.590865+0000","last_peered":"2024-04-20T21:19:10.590865+0000","last_clean":"2024-04-20T21:19:10.590865+0000","last_became_active":"2024-04-20T21:19:09.540659+0000","last_became_peered":"2024-04-20T21:19:09.540659+0000","last_unstale":"2024-04-20T21:19:10.590865+0000","last_undegraded":"2024-04-20T21:19:10.590865+0000","last_fullsized":"2024-04-20T21:19:10.590865+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:28:45.834437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.582659+0000","last_change":"2024-04-20T21:19:10.580229+0000","last_active":"2024-04-20T21:19:10.582659+0000","last_peered":"2024-04-20T21:19:10.582659+0000","last_clean":"2024-04-20T21:19:10.582659+0000","last_became_active":"2024-04-20T21:19:10.579921+0000","last_became_peered":"2024-04-20T21:19:10.579921+0000","last_unstale":"2024-04-20T21:19:10.582659+0000","last_undegraded":"2024-04-20T21:19:10.582659+0000","last_fullsized":"2024-04-20T21:19:10.582659+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:51:36.081243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.566700+0000","last_change":"2024-04-20T21:19:09.539463+0000","last_active":"2024-04-20T21:19:10.566700+0000","last_peered":"2024-04-20T21:19:10.566700+0000","last_clean":"2024-04-20T21:19:10.566700+0000","last_became_active":"2024-04-20T21:19:09.539003+0000","last_became_peered":"2024-04-20T21:19:09.539003+0000","last_unstale":"2024-04-20T21:19:10.566700+0000","last_undegraded":"2024-04-20T21:19:10.566700+0000","last_fullsized":"2024-04-20T21:19:10.566700+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:51:35.067194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.582677+0000","last_change":"2024-04-20T21:19:10.580604+0000","last_active":"2024-04-20T21:19:10.582677+0000","last_peered":"2024-04-20T21:19:10.582677+0000","last_clean":"2024-04-20T21:19:10.582677+0000","last_became_active":"2024-04-20T21:19:10.580391+0000","last_became_peered":"2024-04-20T21:19:10.580391+0000","last_unstale":"2024-04-20T21:19:10.582677+0000","last_undegraded":"2024-04-20T21:19:10.582677+0000","last_fullsized":"2024-04-20T21:19:10.582677+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:39:41.800959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.577630+0000","last_change":"2024-04-20T21:19:09.530587+0000","last_active":"2024-04-20T21:19:10.577630+0000","last_peered":"2024-04-20T21:19:10.577630+0000","last_clean":"2024-04-20T21:19:10.577630+0000","last_became_active":"2024-04-20T21:19:09.530276+0000","last_became_peered":"2024-04-20T21:19:09.530276+0000","last_unstale":"2024-04-20T21:19:10.577630+0000","last_undegraded":"2024-04-20T21:19:10.577630+0000","last_fullsized":"2024-04-20T21:19:10.577630+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:28:45.834437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.569090+0000","last_change":"2024-04-20T21:19:10.567504+0000","last_active":"2024-04-20T21:19:10.569090+0000","last_peered":"2024-04-20T21:19:10.569090+0000","last_clean":"2024-04-20T21:19:10.569090+0000","last_became_active":"2024-04-20T21:19:10.567256+0000","last_became_peered":"2024-04-20T21:19:10.567256+0000","last_unstale":"2024-04-20T21:19:10.569090+0000","last_undegraded":"2024-04-20T21:19:10.569090+0000","last_fullsized":"2024-04-20T21:19:10.569090+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:03:24.269920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.550901+0000","last_change":"2024-04-20T21:19:09.549958+0000","last_active":"2024-04-20T21:19:10.550901+0000","last_peered":"2024-04-20T21:19:10.550901+0000","last_clean":"2024-04-20T21:19:10.550901+0000","last_became_active":"2024-04-20T21:19:09.549493+0000","last_became_peered":"2024-04-20T21:19:09.549493+0000","last_unstale":"2024-04-20T21:19:10.550901+0000","last_undegraded":"2024-04-20T21:19:10.550901+0000","last_fullsized":"2024-04-20T21:19:10.550901+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:51:16.698689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.565552+0000","last_change":"2024-04-20T21:19:10.562046+0000","last_active":"2024-04-20T21:19:10.565552+0000","last_peered":"2024-04-20T21:19:10.565552+0000","last_clean":"2024-04-20T21:19:10.565552+0000","last_became_active":"2024-04-20T21:19:10.561740+0000","last_became_peered":"2024-04-20T21:19:10.561740+0000","last_unstale":"2024-04-20T21:19:10.565552+0000","last_undegraded":"2024-04-20T21:19:10.565552+0000","last_fullsized":"2024-04-20T21:19:10.565552+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:07:12.405934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.570140+0000","last_change":"2024-04-20T21:19:09.540244+0000","last_active":"2024-04-20T21:19:10.570140+0000","last_peered":"2024-04-20T21:19:10.570140+0000","last_clean":"2024-04-20T21:19:10.570140+0000","last_became_active":"2024-04-20T21:19:09.539969+0000","last_became_peered":"2024-04-20T21:19:09.539969+0000","last_unstale":"2024-04-20T21:19:10.570140+0000","last_undegraded":"2024-04-20T21:19:10.570140+0000","last_fullsized":"2024-04-20T21:19:10.570140+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:26:10.683679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.590625+0000","last_change":"2024-04-20T21:19:10.580138+0000","last_active":"2024-04-20T21:19:10.590625+0000","last_peered":"2024-04-20T21:19:10.590625+0000","last_clean":"2024-04-20T21:19:10.590625+0000","last_became_active":"2024-04-20T21:19:10.578940+0000","last_became_peered":"2024-04-20T21:19:10.578940+0000","last_unstale":"2024-04-20T21:19:10.590625+0000","last_undegraded":"2024-04-20T21:19:10.590625+0000","last_fullsized":"2024-04-20T21:19:10.590625+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:10:39.203202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.565477+0000","last_change":"2024-04-20T21:19:09.550130+0000","last_active":"2024-04-20T21:19:10.565477+0000","last_peered":"2024-04-20T21:19:10.565477+0000","last_clean":"2024-04-20T21:19:10.565477+0000","last_became_active":"2024-04-20T21:19:09.549820+0000","last_became_peered":"2024-04-20T21:19:09.549820+0000","last_unstale":"2024-04-20T21:19:10.565477+0000","last_undegraded":"2024-04-20T21:19:10.565477+0000","last_fullsized":"2024-04-20T21:19:10.565477+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:51:22.218013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.579688+0000","last_change":"2024-04-20T21:19:10.576293+0000","last_active":"2024-04-20T21:19:10.579688+0000","last_peered":"2024-04-20T21:19:10.579688+0000","last_clean":"2024-04-20T21:19:10.579688+0000","last_became_active":"2024-04-20T21:19:10.575982+0000","last_became_peered":"2024-04-20T21:19:10.575982+0000","last_unstale":"2024-04-20T21:19:10.579688+0000","last_undegraded":"2024-04-20T21:19:10.579688+0000","last_fullsized":"2024-04-20T21:19:10.579688+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:09:16.527958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.583028+0000","last_change":"2024-04-20T21:19:09.552625+0000","last_active":"2024-04-20T21:19:10.583028+0000","last_peered":"2024-04-20T21:19:10.583028+0000","last_clean":"2024-04-20T21:19:10.583028+0000","last_became_active":"2024-04-20T21:19:09.552313+0000","last_became_peered":"2024-04-20T21:19:09.552313+0000","last_unstale":"2024-04-20T21:19:10.583028+0000","last_undegraded":"2024-04-20T21:19:10.583028+0000","last_fullsized":"2024-04-20T21:19:10.583028+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:59:55.626583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.592293+0000","last_change":"2024-04-20T21:19:10.588770+0000","last_active":"2024-04-20T21:19:10.592293+0000","last_peered":"2024-04-20T21:19:10.592293+0000","last_clean":"2024-04-20T21:19:10.592293+0000","last_became_active":"2024-04-20T21:19:10.588447+0000","last_became_peered":"2024-04-20T21:19:10.588447+0000","last_unstale":"2024-04-20T21:19:10.592293+0000","last_undegraded":"2024-04-20T21:19:10.592293+0000","last_fullsized":"2024-04-20T21:19:10.592293+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:13:08.589597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.556666+0000","last_change":"2024-04-20T21:19:09.536991+0000","last_active":"2024-04-20T21:19:10.556666+0000","last_peered":"2024-04-20T21:19:10.556666+0000","last_clean":"2024-04-20T21:19:10.556666+0000","last_became_active":"2024-04-20T21:19:09.536698+0000","last_became_peered":"2024-04-20T21:19:09.536698+0000","last_unstale":"2024-04-20T21:19:10.556666+0000","last_undegraded":"2024-04-20T21:19:10.556666+0000","last_fullsized":"2024-04-20T21:19:10.556666+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:21:06.042364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.566533+0000","last_change":"2024-04-20T21:19:10.561355+0000","last_active":"2024-04-20T21:19:10.566533+0000","last_peered":"2024-04-20T21:19:10.566533+0000","last_clean":"2024-04-20T21:19:10.566533+0000","last_became_active":"2024-04-20T21:19:10.561078+0000","last_became_peered":"2024-04-20T21:19:10.561078+0000","last_unstale":"2024-04-20T21:19:10.566533+0000","last_undegraded":"2024-04-20T21:19:10.566533+0000","last_fullsized":"2024-04-20T21:19:10.566533+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:34:27.830777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.551096+0000","last_change":"2024-04-20T21:19:09.536528+0000","last_active":"2024-04-20T21:19:10.551096+0000","last_peered":"2024-04-20T21:19:10.551096+0000","last_clean":"2024-04-20T21:19:10.551096+0000","last_became_active":"2024-04-20T21:19:09.533219+0000","last_became_peered":"2024-04-20T21:19:09.533219+0000","last_unstale":"2024-04-20T21:19:10.551096+0000","last_undegraded":"2024-04-20T21:19:10.551096+0000","last_fullsized":"2024-04-20T21:19:10.551096+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:28:45.834437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.578089+0000","last_change":"2024-04-20T21:19:10.567447+0000","last_active":"2024-04-20T21:19:10.578089+0000","last_peered":"2024-04-20T21:19:10.578089+0000","last_clean":"2024-04-20T21:19:10.578089+0000","last_became_active":"2024-04-20T21:19:10.567147+0000","last_became_peered":"2024-04-20T21:19:10.567147+0000","last_unstale":"2024-04-20T21:19:10.578089+0000","last_undegraded":"2024-04-20T21:19:10.578089+0000","last_fullsized":"2024-04-20T21:19:10.578089+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:59:56.640632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.556430+0000","last_change":"2024-04-20T21:19:09.553345+0000","last_active":"2024-04-20T21:19:10.556430+0000","last_peered":"2024-04-20T21:19:10.556430+0000","last_clean":"2024-04-20T21:19:10.556430+0000","last_became_active":"2024-04-20T21:19:09.552947+0000","last_became_peered":"2024-04-20T21:19:09.552947+0000","last_unstale":"2024-04-20T21:19:10.556430+0000","last_undegraded":"2024-04-20T21:19:10.556430+0000","last_fullsized":"2024-04-20T21:19:10.556430+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:26:47.834024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.581095+0000","last_change":"2024-04-20T21:19:10.573357+0000","last_active":"2024-04-20T21:19:10.581095+0000","last_peered":"2024-04-20T21:19:10.581095+0000","last_clean":"2024-04-20T21:19:10.581095+0000","last_became_active":"2024-04-20T21:19:10.572636+0000","last_became_peered":"2024-04-20T21:19:10.572636+0000","last_unstale":"2024-04-20T21:19:10.581095+0000","last_undegraded":"2024-04-20T21:19:10.581095+0000","last_fullsized":"2024-04-20T21:19:10.581095+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:39:05.486077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.553521+0000","last_change":"2024-04-20T21:19:09.553778+0000","last_active":"2024-04-20T21:19:10.553521+0000","last_peered":"2024-04-20T21:19:10.553521+0000","last_clean":"2024-04-20T21:19:10.553521+0000","last_became_active":"2024-04-20T21:19:09.553494+0000","last_became_peered":"2024-04-20T21:19:09.553494+0000","last_unstale":"2024-04-20T21:19:10.553521+0000","last_undegraded":"2024-04-20T21:19:10.553521+0000","last_fullsized":"2024-04-20T21:19:10.553521+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:13:22.717208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.568302+0000","last_change":"2024-04-20T21:19:10.560259+0000","last_active":"2024-04-20T21:19:10.568302+0000","last_peered":"2024-04-20T21:19:10.568302+0000","last_clean":"2024-04-20T21:19:10.568302+0000","last_became_active":"2024-04-20T21:19:10.559961+0000","last_became_peered":"2024-04-20T21:19:10.559961+0000","last_unstale":"2024-04-20T21:19:10.568302+0000","last_undegraded":"2024-04-20T21:19:10.568302+0000","last_fullsized":"2024-04-20T21:19:10.568302+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:07:21.719322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.570309+0000","last_change":"2024-04-20T21:19:09.552929+0000","last_active":"2024-04-20T21:19:10.570309+0000","last_peered":"2024-04-20T21:19:10.570309+0000","last_clean":"2024-04-20T21:19:10.570309+0000","last_became_active":"2024-04-20T21:19:09.552191+0000","last_became_peered":"2024-04-20T21:19:09.552191+0000","last_unstale":"2024-04-20T21:19:10.570309+0000","last_undegraded":"2024-04-20T21:19:10.570309+0000","last_fullsized":"2024-04-20T21:19:10.570309+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:34:46.184597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.587411+0000","last_change":"2024-04-20T21:19:10.584427+0000","last_active":"2024-04-20T21:19:10.587411+0000","last_peered":"2024-04-20T21:19:10.587411+0000","last_clean":"2024-04-20T21:19:10.587411+0000","last_became_active":"2024-04-20T21:19:10.583454+0000","last_became_peered":"2024-04-20T21:19:10.583454+0000","last_unstale":"2024-04-20T21:19:10.587411+0000","last_undegraded":"2024-04-20T21:19:10.587411+0000","last_fullsized":"2024-04-20T21:19:10.587411+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:21:07.056413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.544626+0000","last_change":"2024-04-20T21:19:09.549646+0000","last_active":"2024-04-20T21:19:10.544626+0000","last_peered":"2024-04-20T21:19:10.544626+0000","last_clean":"2024-04-20T21:19:10.544626+0000","last_became_active":"2024-04-20T21:19:09.549359+0000","last_became_peered":"2024-04-20T21:19:09.549359+0000","last_unstale":"2024-04-20T21:19:10.544626+0000","last_undegraded":"2024-04-20T21:19:10.544626+0000","last_fullsized":"2024-04-20T21:19:10.544626+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:54:36.305338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.588251+0000","last_change":"2024-04-20T21:19:10.578207+0000","last_active":"2024-04-20T21:19:10.588251+0000","last_peered":"2024-04-20T21:19:10.588251+0000","last_clean":"2024-04-20T21:19:10.588251+0000","last_became_active":"2024-04-20T21:19:10.576939+0000","last_became_peered":"2024-04-20T21:19:10.576939+0000","last_unstale":"2024-04-20T21:19:10.588251+0000","last_undegraded":"2024-04-20T21:19:10.588251+0000","last_fullsized":"2024-04-20T21:19:10.588251+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:51:23.232062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.551816+0000","last_change":"2024-04-20T21:19:09.550683+0000","last_active":"2024-04-20T21:19:10.551816+0000","last_peered":"2024-04-20T21:19:10.551816+0000","last_clean":"2024-04-20T21:19:10.551816+0000","last_became_active":"2024-04-20T21:19:09.549140+0000","last_became_peered":"2024-04-20T21:19:09.549140+0000","last_unstale":"2024-04-20T21:19:10.551816+0000","last_undegraded":"2024-04-20T21:19:10.551816+0000","last_fullsized":"2024-04-20T21:19:10.551816+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:28:45.834437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":189,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.566054+0000","last_change":"2024-04-20T21:17:43.059637+0000","last_active":"2024-04-20T21:19:10.566054+0000","last_peered":"2024-04-20T21:19:10.566054+0000","last_clean":"2024-04-20T21:19:10.566054+0000","last_became_active":"2024-04-20T21:17:43.056207+0000","last_became_peered":"2024-04-20T21:17:43.056207+0000","last_unstale":"2024-04-20T21:19:10.566054+0000","last_undegraded":"2024-04-20T21:19:10.566054+0000","last_fullsized":"2024-04-20T21:19:10.566054+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:41.036290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:41.036290+0000","last_clean_scrub_stamp":"2024-04-20T21:17:41.036290+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:58:51.091337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":723901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":141942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":954368,"data_stored":730676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T21:19:14.987 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T21:19:15.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.224+0000 7f9df6800640 1 Processor -- start 2024-04-20T21:19:15.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.224+0000 7f9df6800640 1 -- start start 2024-04-20T21:19:15.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.225+0000 7f9df6800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9df00cd170 0x7f9df006ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:15.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.225+0000 7f9df6800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9df00cd600 0x7f9df006f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:15.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.225+0000 7f9df6800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9df00cdb50 con 0x7f9df0144e90 2024-04-20T21:19:15.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.225+0000 7f9df6800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9df0145270 con 0x7f9df00cd170 2024-04-20T21:19:15.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.225+0000 7f9df6800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f9df00b3e50 con 0x7f9df00cd600 2024-04-20T21:19:15.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.226+0000 7f9def400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f9df0144e90 0x7f9df00b4930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41364/0 (socket says 172.21.15.86:41364) 2024-04-20T21:19:15.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.226+0000 7f9def400640 1 -- 172.21.15.86:0/4221049495 learned_addr learned my addr 172.21.15.86:0/4221049495 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:19:15.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.226+0000 7f9deea00640 1 --2- 172.21.15.86:0/4221049495 >> v2:172.21.15.86:3300/0 conn(0x7f9df00cd170 0x7f9df006ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:15.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.226+0000 7f9defe00640 1 --2- 172.21.15.86:0/4221049495 >> v2:172.21.15.144:3300/0 conn(0x7f9df00cd600 0x7f9df006f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:15.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.226+0000 7f9dee000640 1 -- 172.21.15.86:0/4221049495 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 808166917 0 0) 0x7f9df00cdb50 con 0x7f9df0144e90 2024-04-20T21:19:15.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.226+0000 7f9deea00640 1 -- 172.21.15.86:0/4221049495 >> v2:172.21.15.144:3300/0 conn(0x7f9df00cd600 msgr2=0x7f9df006f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:15.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.226+0000 7f9deea00640 1 --2- 172.21.15.86:0/4221049495 >> v2:172.21.15.144:3300/0 conn(0x7f9df00cd600 0x7f9df006f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:15.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.226+0000 7f9deea00640 1 -- 172.21.15.86:0/4221049495 >> v1:172.21.15.86:6789/0 conn(0x7f9df0144e90 legacy=0x7f9df00b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:15.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.226+0000 7f9deea00640 1 -- 172.21.15.86:0/4221049495 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9df00b4130 con 0x7f9df00cd170 2024-04-20T21:19:15.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.227+0000 7f9deea00640 1 --2- 172.21.15.86:0/4221049495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9df00cd170 0x7f9df006ed40 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9df00b5320 tx=0x7f9ddc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7081c1daebe95ac server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:15.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.228+0000 7f9dee000640 1 -- 172.21.15.86:0/4221049495 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc0315a0 con 0x7f9df00cd170 2024-04-20T21:19:15.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.228+0000 7f9dee000640 1 -- 172.21.15.86:0/4221049495 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ddc037070 con 0x7f9df00cd170 2024-04-20T21:19:15.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.228+0000 7f9dee000640 1 -- 172.21.15.86:0/4221049495 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc031960 con 0x7f9df00cd170 2024-04-20T21:19:15.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.228+0000 7f9df6800640 1 -- 172.21.15.86:0/4221049495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9df00cd170 msgr2=0x7f9df006ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:15.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.228+0000 7f9df6800640 1 --2- 172.21.15.86:0/4221049495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9df00cd170 0x7f9df006ed40 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9df00b5320 tx=0x7f9ddc02b840 comp rx=0 tx=0).stop 2024-04-20T21:19:15.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.228+0000 7f9df6800640 1 -- 172.21.15.86:0/4221049495 shutdown_connections 2024-04-20T21:19:15.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.228+0000 7f9df6800640 1 --2- 172.21.15.86:0/4221049495 >> v2:172.21.15.144:3300/0 conn(0x7f9df00cd600 0x7f9df006f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:15.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.228+0000 7f9df6800640 1 --2- 172.21.15.86:0/4221049495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9df00cd170 0x7f9df006ed40 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:15.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.228+0000 7f9df6800640 1 -- 172.21.15.86:0/4221049495 >> 172.21.15.86:0/4221049495 conn(0x7f9df00b2360 msgr2=0x7f9df00b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:15.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.228+0000 7f9df6800640 1 -- 172.21.15.86:0/4221049495 shutdown_connections 2024-04-20T21:19:15.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.228+0000 7f9df6800640 1 -- 172.21.15.86:0/4221049495 wait complete. 2024-04-20T21:19:15.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.230+0000 7f9df6800640 1 Processor -- start 2024-04-20T21:19:15.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.230+0000 7f9df6800640 1 -- start start 2024-04-20T21:19:15.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.231+0000 7f9df6800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9df00cd170 0x7f9df01684c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:15.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.231+0000 7f9df6800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9df00cd600 0x7f9df0157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:15.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.231+0000 7f9df6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9df0144e90 0x7f9df0156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:15.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.231+0000 7f9df6800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9df00d0190 con 0x7f9df0144e90 2024-04-20T21:19:15.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.231+0000 7f9df6800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9df00d0010 con 0x7f9df00cd170 2024-04-20T21:19:15.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.231+0000 7f9df6800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9df015bb50 con 0x7f9df00cd600 2024-04-20T21:19:15.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.231+0000 7f9defe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9df0144e90 0x7f9df0156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:15.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.231+0000 7f9defe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9df0144e90 0x7f9df0156a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42300/0 (socket says 172.21.15.86:42300) 2024-04-20T21:19:15.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.231+0000 7f9defe00640 1 -- 172.21.15.86:0/3395111820 learned_addr learned my addr 172.21.15.86:0/3395111820 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:15.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.232+0000 7f9def400640 1 --2- 172.21.15.86:0/3395111820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9df00cd170 0x7f9df01684c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:15.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.232+0000 7f9deea00640 1 --2- 172.21.15.86:0/3395111820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9df00cd600 0x7f9df0157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:15.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.232+0000 7f9defe00640 1 -- 172.21.15.86:0/3395111820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9df00cd600 msgr2=0x7f9df0157f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:15.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.232+0000 7f9defe00640 1 --2- 172.21.15.86:0/3395111820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9df00cd600 0x7f9df0157f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:15.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.232+0000 7f9defe00640 1 -- 172.21.15.86:0/3395111820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9df00cd170 msgr2=0x7f9df01684c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:15.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.232+0000 7f9defe00640 1 --2- 172.21.15.86:0/3395111820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9df00cd170 0x7f9df01684c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:15.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.232+0000 7f9defe00640 1 -- 172.21.15.86:0/3395111820 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9df0168a00 con 0x7f9df0144e90 2024-04-20T21:19:15.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.232+0000 7f9deea00640 1 --2- 172.21.15.86:0/3395111820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9df00cd600 0x7f9df0157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:15.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.232+0000 7f9def400640 1 --2- 172.21.15.86:0/3395111820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9df00cd170 0x7f9df01684c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:15.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.232+0000 7f9defe00640 1 --2- 172.21.15.86:0/3395111820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9df0144e90 0x7f9df0156a40 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f9de800cd70 tx=0x7f9de800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:15.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.232+0000 7f9de7400640 1 -- 172.21.15.86:0/3395111820 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de8003b90 con 0x7f9df0144e90 2024-04-20T21:19:15.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.232+0000 7f9de7400640 1 -- 172.21.15.86:0/3395111820 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9de8003d30 con 0x7f9df0144e90 2024-04-20T21:19:15.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.232+0000 7f9de7400640 1 -- 172.21.15.86:0/3395111820 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de8004240 con 0x7f9df0144e90 2024-04-20T21:19:15.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.233+0000 7f9df6800640 1 -- 172.21.15.86:0/3395111820 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9df0168ca0 con 0x7f9df0144e90 2024-04-20T21:19:15.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.233+0000 7f9df6800640 1 -- 172.21.15.86:0/3395111820 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9df0157770 con 0x7f9df0144e90 2024-04-20T21:19:15.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.234+0000 7f9df6800640 1 -- 172.21.15.86:0/3395111820 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9df01691c0 con 0x7f9df0144e90 2024-04-20T21:19:15.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.235+0000 7f9de7400640 1 -- 172.21.15.86:0/3395111820 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9de80043e0 con 0x7f9df0144e90 2024-04-20T21:19:15.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.236+0000 7f9de7400640 1 --2- 172.21.15.86:0/3395111820 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9dd006c8f0 0x7f9dd006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:15.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.236+0000 7f9de7400640 1 -- 172.21.15.86:0/3395111820 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f9de808c640 con 0x7f9df0144e90 2024-04-20T21:19:15.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.236+0000 7f9def400640 1 --2- 172.21.15.86:0/3395111820 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9dd006c8f0 0x7f9dd006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:15.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.237+0000 7f9def400640 1 --2- 172.21.15.86:0/3395111820 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9dd006c8f0 0x7f9dd006edb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9de0000f60 tx=0x7f9de0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:15.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.240+0000 7f9de7400640 1 -- 172.21.15.86:0/3395111820 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9de805aa60 con 0x7f9df0144e90 2024-04-20T21:19:15.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.434+0000 7f9df6800640 1 -- 172.21.15.86:0/3395111820 --> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9df00689e0 con 0x7f9dd006c8f0 2024-04-20T21:19:15.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.443+0000 7f9de7400640 1 -- 172.21.15.86:0/3395111820 <== mgr.4107 v2:172.21.15.86:6832/302045240 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251879 (secure 0 0 0) 0x7f9df00689e0 con 0x7f9dd006c8f0 2024-04-20T21:19:15.445 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:19:15.446 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-20T21:19:15.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.447+0000 7f9df6800640 1 -- 172.21.15.86:0/3395111820 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9dd006c8f0 msgr2=0x7f9dd006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:15.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.447+0000 7f9df6800640 1 --2- 172.21.15.86:0/3395111820 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9dd006c8f0 0x7f9dd006edb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9de0000f60 tx=0x7f9de0009040 comp rx=0 tx=0).stop 2024-04-20T21:19:15.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.447+0000 7f9df6800640 1 -- 172.21.15.86:0/3395111820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9df0144e90 msgr2=0x7f9df0156a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:15.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.447+0000 7f9df6800640 1 --2- 172.21.15.86:0/3395111820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9df0144e90 0x7f9df0156a40 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f9de800cd70 tx=0x7f9de800a660 comp rx=0 tx=0).stop 2024-04-20T21:19:15.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.447+0000 7f9df6800640 1 -- 172.21.15.86:0/3395111820 shutdown_connections 2024-04-20T21:19:15.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.447+0000 7f9df6800640 1 --2- 172.21.15.86:0/3395111820 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9dd006c8f0 0x7f9dd006edb0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:15.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.447+0000 7f9df6800640 1 --2- 172.21.15.86:0/3395111820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9df0144e90 0x7f9df0156a40 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:15.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.447+0000 7f9df6800640 1 --2- 172.21.15.86:0/3395111820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9df00cd600 0x7f9df0157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:15.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.447+0000 7f9df6800640 1 --2- 172.21.15.86:0/3395111820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9df00cd170 0x7f9df01684c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:15.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.447+0000 7f9df6800640 1 -- 172.21.15.86:0/3395111820 >> 172.21.15.86:0/3395111820 conn(0x7f9df00b2360 msgr2=0x7f9df00b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:15.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.447+0000 7f9df6800640 1 -- 172.21.15.86:0/3395111820 shutdown_connections 2024-04-20T21:19:15.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.447+0000 7f9df6800640 1 -- 172.21.15.86:0/3395111820 wait complete. 2024-04-20T21:19:15.464 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":73,"stamp":"2024-04-20T21:19:14.655067+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":69,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220108,"kb_used_data":3920,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754612,"statfs":{"total":773094113280,"available":772868722688,"internally_reserved":0,"allocated":4014080,"data_stored":2264184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001301"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.582138+0000","last_change":"2024-04-20T21:19:10.576352+0000","last_active":"2024-04-20T21:19:10.582138+0000","last_peered":"2024-04-20T21:19:10.582138+0000","last_clean":"2024-04-20T21:19:10.582138+0000","last_became_active":"2024-04-20T21:19:10.572686+0000","last_became_peered":"2024-04-20T21:19:10.572686+0000","last_unstale":"2024-04-20T21:19:10.582138+0000","last_undegraded":"2024-04-20T21:19:10.582138+0000","last_fullsized":"2024-04-20T21:19:10.582138+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:21:07.056413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.580263+0000","last_change":"2024-04-20T21:19:10.569472+0000","last_active":"2024-04-20T21:19:10.580263+0000","last_peered":"2024-04-20T21:19:10.580263+0000","last_clean":"2024-04-20T21:19:10.580263+0000","last_became_active":"2024-04-20T21:19:10.569195+0000","last_became_peered":"2024-04-20T21:19:10.569195+0000","last_unstale":"2024-04-20T21:19:10.580263+0000","last_undegraded":"2024-04-20T21:19:10.580263+0000","last_fullsized":"2024-04-20T21:19:10.580263+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:38:47.932291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.592435+0000","last_change":"2024-04-20T21:19:10.591084+0000","last_active":"2024-04-20T21:19:10.592435+0000","last_peered":"2024-04-20T21:19:10.592435+0000","last_clean":"2024-04-20T21:19:10.592435+0000","last_became_active":"2024-04-20T21:19:10.589939+0000","last_became_peered":"2024-04-20T21:19:10.589939+0000","last_unstale":"2024-04-20T21:19:10.592435+0000","last_undegraded":"2024-04-20T21:19:10.592435+0000","last_fullsized":"2024-04-20T21:19:10.592435+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:26:50.833532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.586058+0000","last_change":"2024-04-20T21:19:10.580766+0000","last_active":"2024-04-20T21:19:10.586058+0000","last_peered":"2024-04-20T21:19:10.586058+0000","last_clean":"2024-04-20T21:19:10.586058+0000","last_became_active":"2024-04-20T21:19:10.580370+0000","last_became_peered":"2024-04-20T21:19:10.580370+0000","last_unstale":"2024-04-20T21:19:10.586058+0000","last_undegraded":"2024-04-20T21:19:10.586058+0000","last_fullsized":"2024-04-20T21:19:10.586058+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:51:23.232062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.576872+0000","last_change":"2024-04-20T21:19:10.571809+0000","last_active":"2024-04-20T21:19:10.576872+0000","last_peered":"2024-04-20T21:19:10.576872+0000","last_clean":"2024-04-20T21:19:10.576872+0000","last_became_active":"2024-04-20T21:19:10.571446+0000","last_became_peered":"2024-04-20T21:19:10.571446+0000","last_unstale":"2024-04-20T21:19:10.576872+0000","last_undegraded":"2024-04-20T21:19:10.576872+0000","last_fullsized":"2024-04-20T21:19:10.576872+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:30:30.751151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.585539+0000","last_change":"2024-04-20T21:19:10.578313+0000","last_active":"2024-04-20T21:19:10.585539+0000","last_peered":"2024-04-20T21:19:10.585539+0000","last_clean":"2024-04-20T21:19:10.585539+0000","last_became_active":"2024-04-20T21:19:10.578025+0000","last_became_peered":"2024-04-20T21:19:10.578025+0000","last_unstale":"2024-04-20T21:19:10.585539+0000","last_undegraded":"2024-04-20T21:19:10.585539+0000","last_fullsized":"2024-04-20T21:19:10.585539+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:57:23.354844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.588335+0000","last_change":"2024-04-20T21:19:10.579851+0000","last_active":"2024-04-20T21:19:10.588335+0000","last_peered":"2024-04-20T21:19:10.588335+0000","last_clean":"2024-04-20T21:19:10.588335+0000","last_became_active":"2024-04-20T21:19:10.578964+0000","last_became_peered":"2024-04-20T21:19:10.578964+0000","last_unstale":"2024-04-20T21:19:10.588335+0000","last_undegraded":"2024-04-20T21:19:10.588335+0000","last_fullsized":"2024-04-20T21:19:10.588335+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:07:21.719322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.588144+0000","last_change":"2024-04-20T21:19:10.578080+0000","last_active":"2024-04-20T21:19:10.588144+0000","last_peered":"2024-04-20T21:19:10.588144+0000","last_clean":"2024-04-20T21:19:10.588144+0000","last_became_active":"2024-04-20T21:19:10.576963+0000","last_became_peered":"2024-04-20T21:19:10.576963+0000","last_unstale":"2024-04-20T21:19:10.588144+0000","last_undegraded":"2024-04-20T21:19:10.588144+0000","last_fullsized":"2024-04-20T21:19:10.588144+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:39:05.486077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.592015+0000","last_change":"2024-04-20T21:19:10.587702+0000","last_active":"2024-04-20T21:19:10.592015+0000","last_peered":"2024-04-20T21:19:10.592015+0000","last_clean":"2024-04-20T21:19:10.592015+0000","last_became_active":"2024-04-20T21:19:10.586205+0000","last_became_peered":"2024-04-20T21:19:10.586205+0000","last_unstale":"2024-04-20T21:19:10.592015+0000","last_undegraded":"2024-04-20T21:19:10.592015+0000","last_fullsized":"2024-04-20T21:19:10.592015+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:54:37.319387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.571489+0000","last_change":"2024-04-20T21:19:10.563877+0000","last_active":"2024-04-20T21:19:10.571489+0000","last_peered":"2024-04-20T21:19:10.571489+0000","last_clean":"2024-04-20T21:19:10.571489+0000","last_became_active":"2024-04-20T21:19:10.563438+0000","last_became_peered":"2024-04-20T21:19:10.563438+0000","last_unstale":"2024-04-20T21:19:10.571489+0000","last_undegraded":"2024-04-20T21:19:10.571489+0000","last_fullsized":"2024-04-20T21:19:10.571489+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:34:27.830777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.591026+0000","last_change":"2024-04-20T21:19:10.587379+0000","last_active":"2024-04-20T21:19:10.591026+0000","last_peered":"2024-04-20T21:19:10.591026+0000","last_clean":"2024-04-20T21:19:10.591026+0000","last_became_active":"2024-04-20T21:19:10.587057+0000","last_became_peered":"2024-04-20T21:19:10.587057+0000","last_unstale":"2024-04-20T21:19:10.591026+0000","last_undegraded":"2024-04-20T21:19:10.591026+0000","last_fullsized":"2024-04-20T21:19:10.591026+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:38:47.932291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.572558+0000","last_change":"2024-04-20T21:19:10.563725+0000","last_active":"2024-04-20T21:19:10.572558+0000","last_peered":"2024-04-20T21:19:10.572558+0000","last_clean":"2024-04-20T21:19:10.572558+0000","last_became_active":"2024-04-20T21:19:10.563427+0000","last_became_peered":"2024-04-20T21:19:10.563427+0000","last_unstale":"2024-04-20T21:19:10.572558+0000","last_undegraded":"2024-04-20T21:19:10.572558+0000","last_fullsized":"2024-04-20T21:19:10.572558+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:39:05.486077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.568373+0000","last_change":"2024-04-20T21:19:10.563737+0000","last_active":"2024-04-20T21:19:10.568373+0000","last_peered":"2024-04-20T21:19:10.568373+0000","last_clean":"2024-04-20T21:19:10.568373+0000","last_became_active":"2024-04-20T21:19:10.563419+0000","last_became_peered":"2024-04-20T21:19:10.563419+0000","last_unstale":"2024-04-20T21:19:10.568373+0000","last_undegraded":"2024-04-20T21:19:10.568373+0000","last_fullsized":"2024-04-20T21:19:10.568373+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:13:44.852599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.568332+0000","last_change":"2024-04-20T21:19:10.560575+0000","last_active":"2024-04-20T21:19:10.568332+0000","last_peered":"2024-04-20T21:19:10.568332+0000","last_clean":"2024-04-20T21:19:10.568332+0000","last_became_active":"2024-04-20T21:19:10.560313+0000","last_became_peered":"2024-04-20T21:19:10.560313+0000","last_unstale":"2024-04-20T21:19:10.568332+0000","last_undegraded":"2024-04-20T21:19:10.568332+0000","last_fullsized":"2024-04-20T21:19:10.568332+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:10:39.203202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.591289+0000","last_change":"2024-04-20T21:19:10.581154+0000","last_active":"2024-04-20T21:19:10.591289+0000","last_peered":"2024-04-20T21:19:10.591289+0000","last_clean":"2024-04-20T21:19:10.591289+0000","last_became_active":"2024-04-20T21:19:10.580796+0000","last_became_peered":"2024-04-20T21:19:10.580796+0000","last_unstale":"2024-04-20T21:19:10.591289+0000","last_undegraded":"2024-04-20T21:19:10.591289+0000","last_fullsized":"2024-04-20T21:19:10.591289+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:38:47.932291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.588344+0000","last_change":"2024-04-20T21:19:10.582619+0000","last_active":"2024-04-20T21:19:10.588344+0000","last_peered":"2024-04-20T21:19:10.588344+0000","last_clean":"2024-04-20T21:19:10.588344+0000","last_became_active":"2024-04-20T21:19:10.582291+0000","last_became_peered":"2024-04-20T21:19:10.582291+0000","last_unstale":"2024-04-20T21:19:10.588344+0000","last_undegraded":"2024-04-20T21:19:10.588344+0000","last_fullsized":"2024-04-20T21:19:10.588344+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:57:30.115297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.590663+0000","last_change":"2024-04-20T21:19:10.585810+0000","last_active":"2024-04-20T21:19:10.590663+0000","last_peered":"2024-04-20T21:19:10.590663+0000","last_clean":"2024-04-20T21:19:10.590663+0000","last_became_active":"2024-04-20T21:19:10.585504+0000","last_became_peered":"2024-04-20T21:19:10.585504+0000","last_unstale":"2024-04-20T21:19:10.590663+0000","last_undegraded":"2024-04-20T21:19:10.590663+0000","last_fullsized":"2024-04-20T21:19:10.590663+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:29:35.481155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.580284+0000","last_change":"2024-04-20T21:19:10.570881+0000","last_active":"2024-04-20T21:19:10.580284+0000","last_peered":"2024-04-20T21:19:10.580284+0000","last_clean":"2024-04-20T21:19:10.580284+0000","last_became_active":"2024-04-20T21:19:10.568851+0000","last_became_peered":"2024-04-20T21:19:10.568851+0000","last_unstale":"2024-04-20T21:19:10.580284+0000","last_undegraded":"2024-04-20T21:19:10.580284+0000","last_fullsized":"2024-04-20T21:19:10.580284+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:25:42.400241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.575132+0000","last_change":"2024-04-20T21:19:10.561188+0000","last_active":"2024-04-20T21:19:10.575132+0000","last_peered":"2024-04-20T21:19:10.575132+0000","last_clean":"2024-04-20T21:19:10.575132+0000","last_became_active":"2024-04-20T21:19:10.560920+0000","last_became_peered":"2024-04-20T21:19:10.560920+0000","last_unstale":"2024-04-20T21:19:10.575132+0000","last_undegraded":"2024-04-20T21:19:10.575132+0000","last_fullsized":"2024-04-20T21:19:10.575132+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:29:35.481155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.576896+0000","last_change":"2024-04-20T21:19:10.573630+0000","last_active":"2024-04-20T21:19:10.576896+0000","last_peered":"2024-04-20T21:19:10.576896+0000","last_clean":"2024-04-20T21:19:10.576896+0000","last_became_active":"2024-04-20T21:19:10.573363+0000","last_became_peered":"2024-04-20T21:19:10.573363+0000","last_unstale":"2024-04-20T21:19:10.576896+0000","last_undegraded":"2024-04-20T21:19:10.576896+0000","last_fullsized":"2024-04-20T21:19:10.576896+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:43:40.142483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.592099+0000","last_change":"2024-04-20T21:19:10.585234+0000","last_active":"2024-04-20T21:19:10.592099+0000","last_peered":"2024-04-20T21:19:10.592099+0000","last_clean":"2024-04-20T21:19:10.592099+0000","last_became_active":"2024-04-20T21:19:10.583464+0000","last_became_peered":"2024-04-20T21:19:10.583464+0000","last_unstale":"2024-04-20T21:19:10.592099+0000","last_undegraded":"2024-04-20T21:19:10.592099+0000","last_fullsized":"2024-04-20T21:19:10.592099+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:34:27.830777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.593744+0000","last_change":"2024-04-20T21:19:10.590871+0000","last_active":"2024-04-20T21:19:10.593744+0000","last_peered":"2024-04-20T21:19:10.593744+0000","last_clean":"2024-04-20T21:19:10.593744+0000","last_became_active":"2024-04-20T21:19:10.590605+0000","last_became_peered":"2024-04-20T21:19:10.590605+0000","last_unstale":"2024-04-20T21:19:10.593744+0000","last_undegraded":"2024-04-20T21:19:10.593744+0000","last_fullsized":"2024-04-20T21:19:10.593744+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:51:23.232062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.580430+0000","last_change":"2024-04-20T21:19:10.578864+0000","last_active":"2024-04-20T21:19:10.580430+0000","last_peered":"2024-04-20T21:19:10.580430+0000","last_clean":"2024-04-20T21:19:10.580430+0000","last_became_active":"2024-04-20T21:19:10.578560+0000","last_became_peered":"2024-04-20T21:19:10.578560+0000","last_unstale":"2024-04-20T21:19:10.580430+0000","last_undegraded":"2024-04-20T21:19:10.580430+0000","last_fullsized":"2024-04-20T21:19:10.580430+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:51:23.232062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.576391+0000","last_change":"2024-04-20T21:19:10.568598+0000","last_active":"2024-04-20T21:19:10.576391+0000","last_peered":"2024-04-20T21:19:10.576391+0000","last_clean":"2024-04-20T21:19:10.576391+0000","last_became_active":"2024-04-20T21:19:10.568178+0000","last_became_peered":"2024-04-20T21:19:10.568178+0000","last_unstale":"2024-04-20T21:19:10.576391+0000","last_undegraded":"2024-04-20T21:19:10.576391+0000","last_fullsized":"2024-04-20T21:19:10.576391+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:37:09.020189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.589843+0000","last_change":"2024-04-20T21:19:10.585546+0000","last_active":"2024-04-20T21:19:10.589843+0000","last_peered":"2024-04-20T21:19:10.589843+0000","last_clean":"2024-04-20T21:19:10.589843+0000","last_became_active":"2024-04-20T21:19:10.583941+0000","last_became_peered":"2024-04-20T21:19:10.583941+0000","last_unstale":"2024-04-20T21:19:10.589843+0000","last_undegraded":"2024-04-20T21:19:10.589843+0000","last_fullsized":"2024-04-20T21:19:10.589843+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:57:30.115297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.580302+0000","last_change":"2024-04-20T21:19:10.570110+0000","last_active":"2024-04-20T21:19:10.580302+0000","last_peered":"2024-04-20T21:19:10.580302+0000","last_clean":"2024-04-20T21:19:10.580302+0000","last_became_active":"2024-04-20T21:19:10.569836+0000","last_became_peered":"2024-04-20T21:19:10.569836+0000","last_unstale":"2024-04-20T21:19:10.580302+0000","last_undegraded":"2024-04-20T21:19:10.580302+0000","last_fullsized":"2024-04-20T21:19:10.580302+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:13:08.589597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.589861+0000","last_change":"2024-04-20T21:19:10.586097+0000","last_active":"2024-04-20T21:19:10.589861+0000","last_peered":"2024-04-20T21:19:10.589861+0000","last_clean":"2024-04-20T21:19:10.589861+0000","last_became_active":"2024-04-20T21:19:10.583964+0000","last_became_peered":"2024-04-20T21:19:10.583964+0000","last_unstale":"2024-04-20T21:19:10.589861+0000","last_undegraded":"2024-04-20T21:19:10.589861+0000","last_fullsized":"2024-04-20T21:19:10.589861+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:07:21.719322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.576347+0000","last_change":"2024-04-20T21:19:10.569345+0000","last_active":"2024-04-20T21:19:10.576347+0000","last_peered":"2024-04-20T21:19:10.576347+0000","last_clean":"2024-04-20T21:19:10.576347+0000","last_became_active":"2024-04-20T21:19:10.568842+0000","last_became_peered":"2024-04-20T21:19:10.568842+0000","last_unstale":"2024-04-20T21:19:10.576347+0000","last_undegraded":"2024-04-20T21:19:10.576347+0000","last_fullsized":"2024-04-20T21:19:10.576347+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:57:30.115297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.593599+0000","last_change":"2024-04-20T21:19:10.589230+0000","last_active":"2024-04-20T21:19:10.593599+0000","last_peered":"2024-04-20T21:19:10.593599+0000","last_clean":"2024-04-20T21:19:10.593599+0000","last_became_active":"2024-04-20T21:19:10.588970+0000","last_became_peered":"2024-04-20T21:19:10.588970+0000","last_unstale":"2024-04-20T21:19:10.593599+0000","last_undegraded":"2024-04-20T21:19:10.593599+0000","last_fullsized":"2024-04-20T21:19:10.593599+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:39:05.486077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.587976+0000","last_change":"2024-04-20T21:19:10.579024+0000","last_active":"2024-04-20T21:19:10.587976+0000","last_peered":"2024-04-20T21:19:10.587976+0000","last_clean":"2024-04-20T21:19:10.587976+0000","last_became_active":"2024-04-20T21:19:10.578750+0000","last_became_peered":"2024-04-20T21:19:10.578750+0000","last_unstale":"2024-04-20T21:19:10.587976+0000","last_undegraded":"2024-04-20T21:19:10.587976+0000","last_fullsized":"2024-04-20T21:19:10.587976+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:36:44.756836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.575351+0000","last_change":"2024-04-20T21:19:10.565245+0000","last_active":"2024-04-20T21:19:10.575351+0000","last_peered":"2024-04-20T21:19:10.575351+0000","last_clean":"2024-04-20T21:19:10.575351+0000","last_became_active":"2024-04-20T21:19:10.564910+0000","last_became_peered":"2024-04-20T21:19:10.564910+0000","last_unstale":"2024-04-20T21:19:10.575351+0000","last_undegraded":"2024-04-20T21:19:10.575351+0000","last_fullsized":"2024-04-20T21:19:10.575351+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:26:48.848073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.593674+0000","last_change":"2024-04-20T21:19:10.589881+0000","last_active":"2024-04-20T21:19:10.593674+0000","last_peered":"2024-04-20T21:19:10.593674+0000","last_clean":"2024-04-20T21:19:10.593674+0000","last_became_active":"2024-04-20T21:19:10.587418+0000","last_became_peered":"2024-04-20T21:19:10.587418+0000","last_unstale":"2024-04-20T21:19:10.593674+0000","last_undegraded":"2024-04-20T21:19:10.593674+0000","last_fullsized":"2024-04-20T21:19:10.593674+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:13:08.589597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.579718+0000","last_change":"2024-04-20T21:19:10.572752+0000","last_active":"2024-04-20T21:19:10.579718+0000","last_peered":"2024-04-20T21:19:10.579718+0000","last_clean":"2024-04-20T21:19:10.579718+0000","last_became_active":"2024-04-20T21:19:10.572410+0000","last_became_peered":"2024-04-20T21:19:10.572410+0000","last_unstale":"2024-04-20T21:19:10.579718+0000","last_undegraded":"2024-04-20T21:19:10.579718+0000","last_fullsized":"2024-04-20T21:19:10.579718+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:54:37.319387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.592512+0000","last_change":"2024-04-20T21:19:09.552557+0000","last_active":"2024-04-20T21:19:10.592512+0000","last_peered":"2024-04-20T21:19:10.592512+0000","last_clean":"2024-04-20T21:19:10.592512+0000","last_became_active":"2024-04-20T21:19:09.552209+0000","last_became_peered":"2024-04-20T21:19:09.552209+0000","last_unstale":"2024-04-20T21:19:10.592512+0000","last_undegraded":"2024-04-20T21:19:10.592512+0000","last_fullsized":"2024-04-20T21:19:10.592512+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:09:51.220179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.582159+0000","last_change":"2024-04-20T21:19:10.577010+0000","last_active":"2024-04-20T21:19:10.582159+0000","last_peered":"2024-04-20T21:19:10.582159+0000","last_clean":"2024-04-20T21:19:10.582159+0000","last_became_active":"2024-04-20T21:19:10.576148+0000","last_became_peered":"2024-04-20T21:19:10.576148+0000","last_unstale":"2024-04-20T21:19:10.582159+0000","last_undegraded":"2024-04-20T21:19:10.582159+0000","last_fullsized":"2024-04-20T21:19:10.582159+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:13:23.731257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.579585+0000","last_change":"2024-04-20T21:19:09.546615+0000","last_active":"2024-04-20T21:19:10.579585+0000","last_peered":"2024-04-20T21:19:10.579585+0000","last_clean":"2024-04-20T21:19:10.579585+0000","last_became_active":"2024-04-20T21:19:09.546318+0000","last_became_peered":"2024-04-20T21:19:09.546318+0000","last_unstale":"2024-04-20T21:19:10.579585+0000","last_undegraded":"2024-04-20T21:19:10.579585+0000","last_fullsized":"2024-04-20T21:19:10.579585+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:59:55.626583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.588154+0000","last_change":"2024-04-20T21:19:10.584575+0000","last_active":"2024-04-20T21:19:10.588154+0000","last_peered":"2024-04-20T21:19:10.588154+0000","last_clean":"2024-04-20T21:19:10.588154+0000","last_became_active":"2024-04-20T21:19:10.581940+0000","last_became_peered":"2024-04-20T21:19:10.581940+0000","last_unstale":"2024-04-20T21:19:10.588154+0000","last_undegraded":"2024-04-20T21:19:10.588154+0000","last_fullsized":"2024-04-20T21:19:10.588154+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:13:23.731257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.588632+0000","last_change":"2024-04-20T21:19:09.543512+0000","last_active":"2024-04-20T21:19:10.588632+0000","last_peered":"2024-04-20T21:19:10.588632+0000","last_clean":"2024-04-20T21:19:10.588632+0000","last_became_active":"2024-04-20T21:19:09.543220+0000","last_became_peered":"2024-04-20T21:19:09.543220+0000","last_unstale":"2024-04-20T21:19:10.588632+0000","last_undegraded":"2024-04-20T21:19:10.588632+0000","last_fullsized":"2024-04-20T21:19:10.588632+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:36:43.742787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.589836+0000","last_change":"2024-04-20T21:19:10.585454+0000","last_active":"2024-04-20T21:19:10.589836+0000","last_peered":"2024-04-20T21:19:10.589836+0000","last_clean":"2024-04-20T21:19:10.589836+0000","last_became_active":"2024-04-20T21:19:10.583609+0000","last_became_peered":"2024-04-20T21:19:10.583609+0000","last_unstale":"2024-04-20T21:19:10.589836+0000","last_undegraded":"2024-04-20T21:19:10.589836+0000","last_fullsized":"2024-04-20T21:19:10.589836+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:37:09.020189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.566139+0000","last_change":"2024-04-20T21:19:09.543512+0000","last_active":"2024-04-20T21:19:10.566139+0000","last_peered":"2024-04-20T21:19:10.566139+0000","last_clean":"2024-04-20T21:19:10.566139+0000","last_became_active":"2024-04-20T21:19:09.541949+0000","last_became_peered":"2024-04-20T21:19:09.541949+0000","last_unstale":"2024-04-20T21:19:10.566139+0000","last_undegraded":"2024-04-20T21:19:10.566139+0000","last_fullsized":"2024-04-20T21:19:10.566139+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:02:41.599087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.588311+0000","last_change":"2024-04-20T21:19:10.581639+0000","last_active":"2024-04-20T21:19:10.588311+0000","last_peered":"2024-04-20T21:19:10.588311+0000","last_clean":"2024-04-20T21:19:10.588311+0000","last_became_active":"2024-04-20T21:19:10.581328+0000","last_became_peered":"2024-04-20T21:19:10.581328+0000","last_unstale":"2024-04-20T21:19:10.588311+0000","last_undegraded":"2024-04-20T21:19:10.588311+0000","last_fullsized":"2024-04-20T21:19:10.588311+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:37:09.020189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.592544+0000","last_change":"2024-04-20T21:19:09.538794+0000","last_active":"2024-04-20T21:19:10.592544+0000","last_peered":"2024-04-20T21:19:10.592544+0000","last_clean":"2024-04-20T21:19:10.592544+0000","last_became_active":"2024-04-20T21:19:09.538483+0000","last_became_peered":"2024-04-20T21:19:09.538483+0000","last_unstale":"2024-04-20T21:19:10.592544+0000","last_undegraded":"2024-04-20T21:19:10.592544+0000","last_fullsized":"2024-04-20T21:19:10.592544+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:39:40.786910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.585676+0000","last_change":"2024-04-20T21:19:10.576637+0000","last_active":"2024-04-20T21:19:10.585676+0000","last_peered":"2024-04-20T21:19:10.585676+0000","last_clean":"2024-04-20T21:19:10.585676+0000","last_became_active":"2024-04-20T21:19:10.574494+0000","last_became_peered":"2024-04-20T21:19:10.574494+0000","last_unstale":"2024-04-20T21:19:10.585676+0000","last_undegraded":"2024-04-20T21:19:10.585676+0000","last_fullsized":"2024-04-20T21:19:10.585676+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:29:35.481155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.592545+0000","last_change":"2024-04-20T21:19:09.537483+0000","last_active":"2024-04-20T21:19:10.592545+0000","last_peered":"2024-04-20T21:19:10.592545+0000","last_clean":"2024-04-20T21:19:10.592545+0000","last_became_active":"2024-04-20T21:19:09.537075+0000","last_became_peered":"2024-04-20T21:19:09.537075+0000","last_unstale":"2024-04-20T21:19:10.592545+0000","last_undegraded":"2024-04-20T21:19:10.592545+0000","last_fullsized":"2024-04-20T21:19:10.592545+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:09:51.220179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.586797+0000","last_change":"2024-04-20T21:19:10.579780+0000","last_active":"2024-04-20T21:19:10.586797+0000","last_peered":"2024-04-20T21:19:10.586797+0000","last_clean":"2024-04-20T21:19:10.586797+0000","last_became_active":"2024-04-20T21:19:10.579485+0000","last_became_peered":"2024-04-20T21:19:10.579485+0000","last_unstale":"2024-04-20T21:19:10.586797+0000","last_undegraded":"2024-04-20T21:19:10.586797+0000","last_fullsized":"2024-04-20T21:19:10.586797+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:26:48.848073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.544523+0000","last_change":"2024-04-20T21:19:09.540890+0000","last_active":"2024-04-20T21:19:10.544523+0000","last_peered":"2024-04-20T21:19:10.544523+0000","last_clean":"2024-04-20T21:19:10.544523+0000","last_became_active":"2024-04-20T21:19:09.540445+0000","last_became_peered":"2024-04-20T21:19:09.540445+0000","last_unstale":"2024-04-20T21:19:10.544523+0000","last_undegraded":"2024-04-20T21:19:10.544523+0000","last_fullsized":"2024-04-20T21:19:10.544523+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:13:22.717208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.586442+0000","last_change":"2024-04-20T21:19:10.578871+0000","last_active":"2024-04-20T21:19:10.586442+0000","last_peered":"2024-04-20T21:19:10.586442+0000","last_clean":"2024-04-20T21:19:10.586442+0000","last_became_active":"2024-04-20T21:19:10.578563+0000","last_became_peered":"2024-04-20T21:19:10.578563+0000","last_unstale":"2024-04-20T21:19:10.586442+0000","last_undegraded":"2024-04-20T21:19:10.586442+0000","last_fullsized":"2024-04-20T21:19:10.586442+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:54:37.319387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.569308+0000","last_change":"2024-04-20T21:19:09.551243+0000","last_active":"2024-04-20T21:19:10.569308+0000","last_peered":"2024-04-20T21:19:10.569308+0000","last_clean":"2024-04-20T21:19:10.569308+0000","last_became_active":"2024-04-20T21:19:09.550916+0000","last_became_peered":"2024-04-20T21:19:09.550916+0000","last_unstale":"2024-04-20T21:19:10.569308+0000","last_undegraded":"2024-04-20T21:19:10.569308+0000","last_fullsized":"2024-04-20T21:19:10.569308+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:00:26.773145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.582602+0000","last_change":"2024-04-20T21:19:10.573585+0000","last_active":"2024-04-20T21:19:10.582602+0000","last_peered":"2024-04-20T21:19:10.582602+0000","last_clean":"2024-04-20T21:19:10.582602+0000","last_became_active":"2024-04-20T21:19:10.573188+0000","last_became_peered":"2024-04-20T21:19:10.573188+0000","last_unstale":"2024-04-20T21:19:10.582602+0000","last_undegraded":"2024-04-20T21:19:10.582602+0000","last_fullsized":"2024-04-20T21:19:10.582602+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:10:39.203202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.545169+0000","last_change":"2024-04-20T21:19:09.548020+0000","last_active":"2024-04-20T21:19:10.545169+0000","last_peered":"2024-04-20T21:19:10.545169+0000","last_clean":"2024-04-20T21:19:10.545169+0000","last_became_active":"2024-04-20T21:19:09.547756+0000","last_became_peered":"2024-04-20T21:19:09.547756+0000","last_unstale":"2024-04-20T21:19:10.545169+0000","last_undegraded":"2024-04-20T21:19:10.545169+0000","last_fullsized":"2024-04-20T21:19:10.545169+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:39:04.472028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.568512+0000","last_change":"2024-04-20T21:19:10.558497+0000","last_active":"2024-04-20T21:19:10.568512+0000","last_peered":"2024-04-20T21:19:10.568512+0000","last_clean":"2024-04-20T21:19:10.568512+0000","last_became_active":"2024-04-20T21:19:10.558228+0000","last_became_peered":"2024-04-20T21:19:10.558228+0000","last_unstale":"2024-04-20T21:19:10.568512+0000","last_undegraded":"2024-04-20T21:19:10.568512+0000","last_fullsized":"2024-04-20T21:19:10.568512+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:57:30.115297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.556848+0000","last_change":"2024-04-20T21:19:09.535468+0000","last_active":"2024-04-20T21:19:10.556848+0000","last_peered":"2024-04-20T21:19:10.556848+0000","last_clean":"2024-04-20T21:19:10.556848+0000","last_became_active":"2024-04-20T21:19:09.535123+0000","last_became_peered":"2024-04-20T21:19:09.535123+0000","last_unstale":"2024-04-20T21:19:10.556848+0000","last_undegraded":"2024-04-20T21:19:10.556848+0000","last_fullsized":"2024-04-20T21:19:10.556848+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:13:22.717208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.580885+0000","last_change":"2024-04-20T21:19:10.570617+0000","last_active":"2024-04-20T21:19:10.580885+0000","last_peered":"2024-04-20T21:19:10.580885+0000","last_clean":"2024-04-20T21:19:10.580885+0000","last_became_active":"2024-04-20T21:19:10.570333+0000","last_became_peered":"2024-04-20T21:19:10.570333+0000","last_unstale":"2024-04-20T21:19:10.580885+0000","last_undegraded":"2024-04-20T21:19:10.580885+0000","last_fullsized":"2024-04-20T21:19:10.580885+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:26:50.833532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.590288+0000","last_change":"2024-04-20T21:19:09.552829+0000","last_active":"2024-04-20T21:19:10.590288+0000","last_peered":"2024-04-20T21:19:10.590288+0000","last_clean":"2024-04-20T21:19:10.590288+0000","last_became_active":"2024-04-20T21:19:09.552531+0000","last_became_peered":"2024-04-20T21:19:09.552531+0000","last_unstale":"2024-04-20T21:19:10.590288+0000","last_undegraded":"2024-04-20T21:19:10.590288+0000","last_fullsized":"2024-04-20T21:19:10.590288+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:36:43.742787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.592211+0000","last_change":"2024-04-20T21:19:10.588160+0000","last_active":"2024-04-20T21:19:10.592211+0000","last_peered":"2024-04-20T21:19:10.592211+0000","last_clean":"2024-04-20T21:19:10.592211+0000","last_became_active":"2024-04-20T21:19:10.586777+0000","last_became_peered":"2024-04-20T21:19:10.586777+0000","last_unstale":"2024-04-20T21:19:10.592211+0000","last_undegraded":"2024-04-20T21:19:10.592211+0000","last_fullsized":"2024-04-20T21:19:10.592211+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:31:07.052296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.597529+0000","last_change":"2024-04-20T21:19:09.544496+0000","last_active":"2024-04-20T21:19:10.597529+0000","last_peered":"2024-04-20T21:19:10.597529+0000","last_clean":"2024-04-20T21:19:10.597529+0000","last_became_active":"2024-04-20T21:19:09.544216+0000","last_became_peered":"2024-04-20T21:19:09.544216+0000","last_unstale":"2024-04-20T21:19:10.597529+0000","last_undegraded":"2024-04-20T21:19:10.597529+0000","last_fullsized":"2024-04-20T21:19:10.597529+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:37:08.006140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.587865+0000","last_change":"2024-04-20T21:19:10.583052+0000","last_active":"2024-04-20T21:19:10.587865+0000","last_peered":"2024-04-20T21:19:10.587865+0000","last_clean":"2024-04-20T21:19:10.587865+0000","last_became_active":"2024-04-20T21:19:10.582768+0000","last_became_peered":"2024-04-20T21:19:10.582768+0000","last_unstale":"2024-04-20T21:19:10.587865+0000","last_undegraded":"2024-04-20T21:19:10.587865+0000","last_fullsized":"2024-04-20T21:19:10.587865+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:36:44.756836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.592529+0000","last_change":"2024-04-20T21:19:09.545708+0000","last_active":"2024-04-20T21:19:10.592529+0000","last_peered":"2024-04-20T21:19:10.592529+0000","last_clean":"2024-04-20T21:19:10.592529+0000","last_became_active":"2024-04-20T21:19:09.545373+0000","last_became_peered":"2024-04-20T21:19:09.545373+0000","last_unstale":"2024-04-20T21:19:10.592529+0000","last_undegraded":"2024-04-20T21:19:10.592529+0000","last_fullsized":"2024-04-20T21:19:10.592529+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:51:16.698689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.582749+0000","last_change":"2024-04-20T21:19:10.570085+0000","last_active":"2024-04-20T21:19:10.582749+0000","last_peered":"2024-04-20T21:19:10.582749+0000","last_clean":"2024-04-20T21:19:10.582749+0000","last_became_active":"2024-04-20T21:19:10.569846+0000","last_became_peered":"2024-04-20T21:19:10.569846+0000","last_unstale":"2024-04-20T21:19:10.582749+0000","last_undegraded":"2024-04-20T21:19:10.582749+0000","last_fullsized":"2024-04-20T21:19:10.582749+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:31:07.052296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.552562+0000","last_change":"2024-04-20T21:19:09.549906+0000","last_active":"2024-04-20T21:19:10.552562+0000","last_peered":"2024-04-20T21:19:10.552562+0000","last_clean":"2024-04-20T21:19:10.552562+0000","last_became_active":"2024-04-20T21:19:09.549637+0000","last_became_peered":"2024-04-20T21:19:09.549637+0000","last_unstale":"2024-04-20T21:19:10.552562+0000","last_undegraded":"2024-04-20T21:19:10.552562+0000","last_fullsized":"2024-04-20T21:19:10.552562+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:21:06.042364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.592010+0000","last_change":"2024-04-20T21:19:10.588038+0000","last_active":"2024-04-20T21:19:10.592010+0000","last_peered":"2024-04-20T21:19:10.592010+0000","last_clean":"2024-04-20T21:19:10.592010+0000","last_became_active":"2024-04-20T21:19:10.587755+0000","last_became_peered":"2024-04-20T21:19:10.587755+0000","last_unstale":"2024-04-20T21:19:10.592010+0000","last_undegraded":"2024-04-20T21:19:10.592010+0000","last_fullsized":"2024-04-20T21:19:10.592010+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:31:07.052296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.544539+0000","last_change":"2024-04-20T21:19:09.547370+0000","last_active":"2024-04-20T21:19:10.544539+0000","last_peered":"2024-04-20T21:19:10.544539+0000","last_clean":"2024-04-20T21:19:10.544539+0000","last_became_active":"2024-04-20T21:19:09.547129+0000","last_became_peered":"2024-04-20T21:19:09.547129+0000","last_unstale":"2024-04-20T21:19:10.544539+0000","last_undegraded":"2024-04-20T21:19:10.544539+0000","last_fullsized":"2024-04-20T21:19:10.544539+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:34:26.816727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.588946+0000","last_change":"2024-04-20T21:19:10.580561+0000","last_active":"2024-04-20T21:19:10.588946+0000","last_peered":"2024-04-20T21:19:10.588946+0000","last_clean":"2024-04-20T21:19:10.588946+0000","last_became_active":"2024-04-20T21:19:10.580251+0000","last_became_peered":"2024-04-20T21:19:10.580251+0000","last_unstale":"2024-04-20T21:19:10.588946+0000","last_undegraded":"2024-04-20T21:19:10.588946+0000","last_fullsized":"2024-04-20T21:19:10.588946+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:34:27.830777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.588089+0000","last_change":"2024-04-20T21:19:09.543558+0000","last_active":"2024-04-20T21:19:10.588089+0000","last_peered":"2024-04-20T21:19:10.588089+0000","last_clean":"2024-04-20T21:19:10.588089+0000","last_became_active":"2024-04-20T21:19:09.543279+0000","last_became_peered":"2024-04-20T21:19:09.543279+0000","last_unstale":"2024-04-20T21:19:10.588089+0000","last_undegraded":"2024-04-20T21:19:10.588089+0000","last_fullsized":"2024-04-20T21:19:10.588089+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:51:22.218013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.586876+0000","last_change":"2024-04-20T21:19:10.582195+0000","last_active":"2024-04-20T21:19:10.586876+0000","last_peered":"2024-04-20T21:19:10.586876+0000","last_clean":"2024-04-20T21:19:10.586876+0000","last_became_active":"2024-04-20T21:19:10.581906+0000","last_became_peered":"2024-04-20T21:19:10.581906+0000","last_unstale":"2024-04-20T21:19:10.586876+0000","last_undegraded":"2024-04-20T21:19:10.586876+0000","last_fullsized":"2024-04-20T21:19:10.586876+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:59:56.640632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.585592+0000","last_change":"2024-04-20T21:19:09.540882+0000","last_active":"2024-04-20T21:19:10.585592+0000","last_peered":"2024-04-20T21:19:10.585592+0000","last_clean":"2024-04-20T21:19:10.585592+0000","last_became_active":"2024-04-20T21:19:09.540531+0000","last_became_peered":"2024-04-20T21:19:09.540531+0000","last_unstale":"2024-04-20T21:19:10.585592+0000","last_undegraded":"2024-04-20T21:19:10.585592+0000","last_fullsized":"2024-04-20T21:19:10.585592+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:26:47.834024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.582174+0000","last_change":"2024-04-20T21:19:10.572217+0000","last_active":"2024-04-20T21:19:10.582174+0000","last_peered":"2024-04-20T21:19:10.582174+0000","last_clean":"2024-04-20T21:19:10.582174+0000","last_became_active":"2024-04-20T21:19:10.571913+0000","last_became_peered":"2024-04-20T21:19:10.571913+0000","last_unstale":"2024-04-20T21:19:10.582174+0000","last_undegraded":"2024-04-20T21:19:10.582174+0000","last_fullsized":"2024-04-20T21:19:10.582174+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:07:21.719322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.590656+0000","last_change":"2024-04-20T21:19:09.551656+0000","last_active":"2024-04-20T21:19:10.590656+0000","last_peered":"2024-04-20T21:19:10.590656+0000","last_clean":"2024-04-20T21:19:10.590656+0000","last_became_active":"2024-04-20T21:19:09.551360+0000","last_became_peered":"2024-04-20T21:19:09.551360+0000","last_unstale":"2024-04-20T21:19:10.590656+0000","last_undegraded":"2024-04-20T21:19:10.590656+0000","last_fullsized":"2024-04-20T21:19:10.590656+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:09:15.513908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.586014+0000","last_change":"2024-04-20T21:19:10.580564+0000","last_active":"2024-04-20T21:19:10.586014+0000","last_peered":"2024-04-20T21:19:10.586014+0000","last_clean":"2024-04-20T21:19:10.586014+0000","last_became_active":"2024-04-20T21:19:10.580073+0000","last_became_peered":"2024-04-20T21:19:10.580073+0000","last_unstale":"2024-04-20T21:19:10.586014+0000","last_undegraded":"2024-04-20T21:19:10.586014+0000","last_fullsized":"2024-04-20T21:19:10.586014+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:39:05.486077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.590865+0000","last_change":"2024-04-20T21:19:09.540957+0000","last_active":"2024-04-20T21:19:10.590865+0000","last_peered":"2024-04-20T21:19:10.590865+0000","last_clean":"2024-04-20T21:19:10.590865+0000","last_became_active":"2024-04-20T21:19:09.540659+0000","last_became_peered":"2024-04-20T21:19:09.540659+0000","last_unstale":"2024-04-20T21:19:10.590865+0000","last_undegraded":"2024-04-20T21:19:10.590865+0000","last_fullsized":"2024-04-20T21:19:10.590865+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:28:45.834437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.582659+0000","last_change":"2024-04-20T21:19:10.580229+0000","last_active":"2024-04-20T21:19:10.582659+0000","last_peered":"2024-04-20T21:19:10.582659+0000","last_clean":"2024-04-20T21:19:10.582659+0000","last_became_active":"2024-04-20T21:19:10.579921+0000","last_became_peered":"2024-04-20T21:19:10.579921+0000","last_unstale":"2024-04-20T21:19:10.582659+0000","last_undegraded":"2024-04-20T21:19:10.582659+0000","last_fullsized":"2024-04-20T21:19:10.582659+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:51:36.081243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.566700+0000","last_change":"2024-04-20T21:19:09.539463+0000","last_active":"2024-04-20T21:19:10.566700+0000","last_peered":"2024-04-20T21:19:10.566700+0000","last_clean":"2024-04-20T21:19:10.566700+0000","last_became_active":"2024-04-20T21:19:09.539003+0000","last_became_peered":"2024-04-20T21:19:09.539003+0000","last_unstale":"2024-04-20T21:19:10.566700+0000","last_undegraded":"2024-04-20T21:19:10.566700+0000","last_fullsized":"2024-04-20T21:19:10.566700+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:51:35.067194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.582677+0000","last_change":"2024-04-20T21:19:10.580604+0000","last_active":"2024-04-20T21:19:10.582677+0000","last_peered":"2024-04-20T21:19:10.582677+0000","last_clean":"2024-04-20T21:19:10.582677+0000","last_became_active":"2024-04-20T21:19:10.580391+0000","last_became_peered":"2024-04-20T21:19:10.580391+0000","last_unstale":"2024-04-20T21:19:10.582677+0000","last_undegraded":"2024-04-20T21:19:10.582677+0000","last_fullsized":"2024-04-20T21:19:10.582677+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:39:41.800959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.577630+0000","last_change":"2024-04-20T21:19:09.530587+0000","last_active":"2024-04-20T21:19:10.577630+0000","last_peered":"2024-04-20T21:19:10.577630+0000","last_clean":"2024-04-20T21:19:10.577630+0000","last_became_active":"2024-04-20T21:19:09.530276+0000","last_became_peered":"2024-04-20T21:19:09.530276+0000","last_unstale":"2024-04-20T21:19:10.577630+0000","last_undegraded":"2024-04-20T21:19:10.577630+0000","last_fullsized":"2024-04-20T21:19:10.577630+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:28:45.834437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.569090+0000","last_change":"2024-04-20T21:19:10.567504+0000","last_active":"2024-04-20T21:19:10.569090+0000","last_peered":"2024-04-20T21:19:10.569090+0000","last_clean":"2024-04-20T21:19:10.569090+0000","last_became_active":"2024-04-20T21:19:10.567256+0000","last_became_peered":"2024-04-20T21:19:10.567256+0000","last_unstale":"2024-04-20T21:19:10.569090+0000","last_undegraded":"2024-04-20T21:19:10.569090+0000","last_fullsized":"2024-04-20T21:19:10.569090+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:03:24.269920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.550901+0000","last_change":"2024-04-20T21:19:09.549958+0000","last_active":"2024-04-20T21:19:10.550901+0000","last_peered":"2024-04-20T21:19:10.550901+0000","last_clean":"2024-04-20T21:19:10.550901+0000","last_became_active":"2024-04-20T21:19:09.549493+0000","last_became_peered":"2024-04-20T21:19:09.549493+0000","last_unstale":"2024-04-20T21:19:10.550901+0000","last_undegraded":"2024-04-20T21:19:10.550901+0000","last_fullsized":"2024-04-20T21:19:10.550901+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:51:16.698689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.565552+0000","last_change":"2024-04-20T21:19:10.562046+0000","last_active":"2024-04-20T21:19:10.565552+0000","last_peered":"2024-04-20T21:19:10.565552+0000","last_clean":"2024-04-20T21:19:10.565552+0000","last_became_active":"2024-04-20T21:19:10.561740+0000","last_became_peered":"2024-04-20T21:19:10.561740+0000","last_unstale":"2024-04-20T21:19:10.565552+0000","last_undegraded":"2024-04-20T21:19:10.565552+0000","last_fullsized":"2024-04-20T21:19:10.565552+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:07:12.405934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.570140+0000","last_change":"2024-04-20T21:19:09.540244+0000","last_active":"2024-04-20T21:19:10.570140+0000","last_peered":"2024-04-20T21:19:10.570140+0000","last_clean":"2024-04-20T21:19:10.570140+0000","last_became_active":"2024-04-20T21:19:09.539969+0000","last_became_peered":"2024-04-20T21:19:09.539969+0000","last_unstale":"2024-04-20T21:19:10.570140+0000","last_undegraded":"2024-04-20T21:19:10.570140+0000","last_fullsized":"2024-04-20T21:19:10.570140+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:26:10.683679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.590625+0000","last_change":"2024-04-20T21:19:10.580138+0000","last_active":"2024-04-20T21:19:10.590625+0000","last_peered":"2024-04-20T21:19:10.590625+0000","last_clean":"2024-04-20T21:19:10.590625+0000","last_became_active":"2024-04-20T21:19:10.578940+0000","last_became_peered":"2024-04-20T21:19:10.578940+0000","last_unstale":"2024-04-20T21:19:10.590625+0000","last_undegraded":"2024-04-20T21:19:10.590625+0000","last_fullsized":"2024-04-20T21:19:10.590625+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:10:39.203202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.565477+0000","last_change":"2024-04-20T21:19:09.550130+0000","last_active":"2024-04-20T21:19:10.565477+0000","last_peered":"2024-04-20T21:19:10.565477+0000","last_clean":"2024-04-20T21:19:10.565477+0000","last_became_active":"2024-04-20T21:19:09.549820+0000","last_became_peered":"2024-04-20T21:19:09.549820+0000","last_unstale":"2024-04-20T21:19:10.565477+0000","last_undegraded":"2024-04-20T21:19:10.565477+0000","last_fullsized":"2024-04-20T21:19:10.565477+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:51:22.218013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.579688+0000","last_change":"2024-04-20T21:19:10.576293+0000","last_active":"2024-04-20T21:19:10.579688+0000","last_peered":"2024-04-20T21:19:10.579688+0000","last_clean":"2024-04-20T21:19:10.579688+0000","last_became_active":"2024-04-20T21:19:10.575982+0000","last_became_peered":"2024-04-20T21:19:10.575982+0000","last_unstale":"2024-04-20T21:19:10.579688+0000","last_undegraded":"2024-04-20T21:19:10.579688+0000","last_fullsized":"2024-04-20T21:19:10.579688+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:09:16.527958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.583028+0000","last_change":"2024-04-20T21:19:09.552625+0000","last_active":"2024-04-20T21:19:10.583028+0000","last_peered":"2024-04-20T21:19:10.583028+0000","last_clean":"2024-04-20T21:19:10.583028+0000","last_became_active":"2024-04-20T21:19:09.552313+0000","last_became_peered":"2024-04-20T21:19:09.552313+0000","last_unstale":"2024-04-20T21:19:10.583028+0000","last_undegraded":"2024-04-20T21:19:10.583028+0000","last_fullsized":"2024-04-20T21:19:10.583028+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:59:55.626583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.592293+0000","last_change":"2024-04-20T21:19:10.588770+0000","last_active":"2024-04-20T21:19:10.592293+0000","last_peered":"2024-04-20T21:19:10.592293+0000","last_clean":"2024-04-20T21:19:10.592293+0000","last_became_active":"2024-04-20T21:19:10.588447+0000","last_became_peered":"2024-04-20T21:19:10.588447+0000","last_unstale":"2024-04-20T21:19:10.592293+0000","last_undegraded":"2024-04-20T21:19:10.592293+0000","last_fullsized":"2024-04-20T21:19:10.592293+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:13:08.589597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.556666+0000","last_change":"2024-04-20T21:19:09.536991+0000","last_active":"2024-04-20T21:19:10.556666+0000","last_peered":"2024-04-20T21:19:10.556666+0000","last_clean":"2024-04-20T21:19:10.556666+0000","last_became_active":"2024-04-20T21:19:09.536698+0000","last_became_peered":"2024-04-20T21:19:09.536698+0000","last_unstale":"2024-04-20T21:19:10.556666+0000","last_undegraded":"2024-04-20T21:19:10.556666+0000","last_fullsized":"2024-04-20T21:19:10.556666+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:21:06.042364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.566533+0000","last_change":"2024-04-20T21:19:10.561355+0000","last_active":"2024-04-20T21:19:10.566533+0000","last_peered":"2024-04-20T21:19:10.566533+0000","last_clean":"2024-04-20T21:19:10.566533+0000","last_became_active":"2024-04-20T21:19:10.561078+0000","last_became_peered":"2024-04-20T21:19:10.561078+0000","last_unstale":"2024-04-20T21:19:10.566533+0000","last_undegraded":"2024-04-20T21:19:10.566533+0000","last_fullsized":"2024-04-20T21:19:10.566533+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:34:27.830777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.551096+0000","last_change":"2024-04-20T21:19:09.536528+0000","last_active":"2024-04-20T21:19:10.551096+0000","last_peered":"2024-04-20T21:19:10.551096+0000","last_clean":"2024-04-20T21:19:10.551096+0000","last_became_active":"2024-04-20T21:19:09.533219+0000","last_became_peered":"2024-04-20T21:19:09.533219+0000","last_unstale":"2024-04-20T21:19:10.551096+0000","last_undegraded":"2024-04-20T21:19:10.551096+0000","last_fullsized":"2024-04-20T21:19:10.551096+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:28:45.834437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.578089+0000","last_change":"2024-04-20T21:19:10.567447+0000","last_active":"2024-04-20T21:19:10.578089+0000","last_peered":"2024-04-20T21:19:10.578089+0000","last_clean":"2024-04-20T21:19:10.578089+0000","last_became_active":"2024-04-20T21:19:10.567147+0000","last_became_peered":"2024-04-20T21:19:10.567147+0000","last_unstale":"2024-04-20T21:19:10.578089+0000","last_undegraded":"2024-04-20T21:19:10.578089+0000","last_fullsized":"2024-04-20T21:19:10.578089+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:59:56.640632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.556430+0000","last_change":"2024-04-20T21:19:09.553345+0000","last_active":"2024-04-20T21:19:10.556430+0000","last_peered":"2024-04-20T21:19:10.556430+0000","last_clean":"2024-04-20T21:19:10.556430+0000","last_became_active":"2024-04-20T21:19:09.552947+0000","last_became_peered":"2024-04-20T21:19:09.552947+0000","last_unstale":"2024-04-20T21:19:10.556430+0000","last_undegraded":"2024-04-20T21:19:10.556430+0000","last_fullsized":"2024-04-20T21:19:10.556430+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:26:47.834024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.581095+0000","last_change":"2024-04-20T21:19:10.573357+0000","last_active":"2024-04-20T21:19:10.581095+0000","last_peered":"2024-04-20T21:19:10.581095+0000","last_clean":"2024-04-20T21:19:10.581095+0000","last_became_active":"2024-04-20T21:19:10.572636+0000","last_became_peered":"2024-04-20T21:19:10.572636+0000","last_unstale":"2024-04-20T21:19:10.581095+0000","last_undegraded":"2024-04-20T21:19:10.581095+0000","last_fullsized":"2024-04-20T21:19:10.581095+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:39:05.486077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.553521+0000","last_change":"2024-04-20T21:19:09.553778+0000","last_active":"2024-04-20T21:19:10.553521+0000","last_peered":"2024-04-20T21:19:10.553521+0000","last_clean":"2024-04-20T21:19:10.553521+0000","last_became_active":"2024-04-20T21:19:09.553494+0000","last_became_peered":"2024-04-20T21:19:09.553494+0000","last_unstale":"2024-04-20T21:19:10.553521+0000","last_undegraded":"2024-04-20T21:19:10.553521+0000","last_fullsized":"2024-04-20T21:19:10.553521+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:13:22.717208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.568302+0000","last_change":"2024-04-20T21:19:10.560259+0000","last_active":"2024-04-20T21:19:10.568302+0000","last_peered":"2024-04-20T21:19:10.568302+0000","last_clean":"2024-04-20T21:19:10.568302+0000","last_became_active":"2024-04-20T21:19:10.559961+0000","last_became_peered":"2024-04-20T21:19:10.559961+0000","last_unstale":"2024-04-20T21:19:10.568302+0000","last_undegraded":"2024-04-20T21:19:10.568302+0000","last_fullsized":"2024-04-20T21:19:10.568302+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:07:21.719322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.570309+0000","last_change":"2024-04-20T21:19:09.552929+0000","last_active":"2024-04-20T21:19:10.570309+0000","last_peered":"2024-04-20T21:19:10.570309+0000","last_clean":"2024-04-20T21:19:10.570309+0000","last_became_active":"2024-04-20T21:19:09.552191+0000","last_became_peered":"2024-04-20T21:19:09.552191+0000","last_unstale":"2024-04-20T21:19:10.570309+0000","last_undegraded":"2024-04-20T21:19:10.570309+0000","last_fullsized":"2024-04-20T21:19:10.570309+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:34:46.184597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.587411+0000","last_change":"2024-04-20T21:19:10.584427+0000","last_active":"2024-04-20T21:19:10.587411+0000","last_peered":"2024-04-20T21:19:10.587411+0000","last_clean":"2024-04-20T21:19:10.587411+0000","last_became_active":"2024-04-20T21:19:10.583454+0000","last_became_peered":"2024-04-20T21:19:10.583454+0000","last_unstale":"2024-04-20T21:19:10.587411+0000","last_undegraded":"2024-04-20T21:19:10.587411+0000","last_fullsized":"2024-04-20T21:19:10.587411+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:21:07.056413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.544626+0000","last_change":"2024-04-20T21:19:09.549646+0000","last_active":"2024-04-20T21:19:10.544626+0000","last_peered":"2024-04-20T21:19:10.544626+0000","last_clean":"2024-04-20T21:19:10.544626+0000","last_became_active":"2024-04-20T21:19:09.549359+0000","last_became_peered":"2024-04-20T21:19:09.549359+0000","last_unstale":"2024-04-20T21:19:10.544626+0000","last_undegraded":"2024-04-20T21:19:10.544626+0000","last_fullsized":"2024-04-20T21:19:10.544626+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:54:36.305338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.588251+0000","last_change":"2024-04-20T21:19:10.578207+0000","last_active":"2024-04-20T21:19:10.588251+0000","last_peered":"2024-04-20T21:19:10.588251+0000","last_clean":"2024-04-20T21:19:10.588251+0000","last_became_active":"2024-04-20T21:19:10.576939+0000","last_became_peered":"2024-04-20T21:19:10.576939+0000","last_unstale":"2024-04-20T21:19:10.588251+0000","last_undegraded":"2024-04-20T21:19:10.588251+0000","last_fullsized":"2024-04-20T21:19:10.588251+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:09.504674+0000","last_clean_scrub_stamp":"2024-04-20T21:19:09.504674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:51:23.232062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.551816+0000","last_change":"2024-04-20T21:19:09.550683+0000","last_active":"2024-04-20T21:19:10.551816+0000","last_peered":"2024-04-20T21:19:10.551816+0000","last_clean":"2024-04-20T21:19:10.551816+0000","last_became_active":"2024-04-20T21:19:09.549140+0000","last_became_peered":"2024-04-20T21:19:09.549140+0000","last_unstale":"2024-04-20T21:19:10.551816+0000","last_undegraded":"2024-04-20T21:19:10.551816+0000","last_fullsized":"2024-04-20T21:19:10.551816+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:19:08.490624+0000","last_clean_scrub_stamp":"2024-04-20T21:19:08.490624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:28:45.834437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":189,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-20T21:19:10.566054+0000","last_change":"2024-04-20T21:17:43.059637+0000","last_active":"2024-04-20T21:19:10.566054+0000","last_peered":"2024-04-20T21:19:10.566054+0000","last_clean":"2024-04-20T21:19:10.566054+0000","last_became_active":"2024-04-20T21:17:43.056207+0000","last_became_peered":"2024-04-20T21:17:43.056207+0000","last_unstale":"2024-04-20T21:19:10.566054+0000","last_undegraded":"2024-04-20T21:19:10.566054+0000","last_fullsized":"2024-04-20T21:19:10.566054+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:41.036290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:41.036290+0000","last_clean_scrub_stamp":"2024-04-20T21:17:41.036290+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:58:51.091337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":723901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":141942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":133533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":954368,"data_stored":730676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T21:19:15.468 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-20T21:19:15.469 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-20T21:19:15.534 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:19:15.557 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:19:15.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.710+0000 7fa216000640 1 Processor -- start 2024-04-20T21:19:15.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.710+0000 7fa216000640 1 -- start start 2024-04-20T21:19:15.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.712+0000 7fa216000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa21015eb30 0x7fa2100b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:15.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.712+0000 7fa216000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa21006c090 0x7fa2100b4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:15.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.712+0000 7fa216000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2100adb10 con 0x7fa2100aee50 2024-04-20T21:19:15.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.712+0000 7fa216000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fa2100af7d0 con 0x7fa21006c090 2024-04-20T21:19:15.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.712+0000 7fa216000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa2100addb0 con 0x7fa21015eb30 2024-04-20T21:19:15.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.713+0000 7fa20f400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa21015eb30 0x7fa2100b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:15.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.713+0000 7fa20f400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa21015eb30 0x7fa2100b3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:50848/0 (socket says 172.21.15.86:50848) 2024-04-20T21:19:15.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.713+0000 7fa20f400640 1 -- 172.21.15.86:0/4052804904 learned_addr learned my addr 172.21.15.86:0/4052804904 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:15.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.713+0000 7fa20fe00640 1 --2- 172.21.15.86:0/4052804904 >> v2:172.21.15.144:3300/0 conn(0x7fa21006c090 0x7fa2100b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:15.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.714+0000 7fa20f400640 1 -- 172.21.15.86:0/4052804904 >> v2:172.21.15.144:3300/0 conn(0x7fa21006c090 msgr2=0x7fa2100b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:15.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.714+0000 7fa20f400640 1 --2- 172.21.15.86:0/4052804904 >> v2:172.21.15.144:3300/0 conn(0x7fa21006c090 0x7fa2100b4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:15.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.714+0000 7fa20f400640 1 -- 172.21.15.86:0/4052804904 >> v1:172.21.15.144:6789/0 conn(0x7fa2100aee50 legacy=0x7fa21006b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:15.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.714+0000 7fa20f400640 1 -- 172.21.15.86:0/4052804904 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2100ae090 con 0x7fa21015eb30 2024-04-20T21:19:15.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.715+0000 7fa20f400640 1 --2- 172.21.15.86:0/4052804904 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa21015eb30 0x7fa2100b3e20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa200009870 tx=0x7fa20002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae28ce5cb967bc12 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:15.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.716+0000 7fa20e000640 1 -- 172.21.15.86:0/4052804904 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa200034070 con 0x7fa21015eb30 2024-04-20T21:19:15.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.716+0000 7fa20e000640 1 -- 172.21.15.86:0/4052804904 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa200030840 con 0x7fa21015eb30 2024-04-20T21:19:15.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.716+0000 7fa20e000640 1 -- 172.21.15.86:0/4052804904 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa200030c10 con 0x7fa21015eb30 2024-04-20T21:19:15.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.716+0000 7fa216000640 1 -- 172.21.15.86:0/4052804904 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa21015eb30 msgr2=0x7fa2100b3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:15.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.716+0000 7fa216000640 1 --2- 172.21.15.86:0/4052804904 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa21015eb30 0x7fa2100b3e20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa200009870 tx=0x7fa20002b980 comp rx=0 tx=0).stop 2024-04-20T21:19:15.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.716+0000 7fa216000640 1 -- 172.21.15.86:0/4052804904 shutdown_connections 2024-04-20T21:19:15.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.716+0000 7fa216000640 1 --2- 172.21.15.86:0/4052804904 >> v2:172.21.15.144:3300/0 conn(0x7fa21006c090 0x7fa2100b4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:15.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.716+0000 7fa216000640 1 --2- 172.21.15.86:0/4052804904 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa21015eb30 0x7fa2100b3e20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:15.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.716+0000 7fa216000640 1 -- 172.21.15.86:0/4052804904 >> 172.21.15.86:0/4052804904 conn(0x7fa2100b2110 msgr2=0x7fa2100b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:15.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.716+0000 7fa216000640 1 -- 172.21.15.86:0/4052804904 shutdown_connections 2024-04-20T21:19:15.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.716+0000 7fa216000640 1 -- 172.21.15.86:0/4052804904 wait complete. 2024-04-20T21:19:15.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.718+0000 7fa216000640 1 Processor -- start 2024-04-20T21:19:15.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.718+0000 7fa216000640 1 -- start start 2024-04-20T21:19:15.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.719+0000 7fa216000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa21006c090 0x7fa21014f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:15.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.719+0000 7fa216000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2100aee50 0x7fa210158630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:15.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.719+0000 7fa216000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa21015eb30 0x7fa2100cec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:15.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.719+0000 7fa216000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa2100ae930 con 0x7fa2100aee50 2024-04-20T21:19:15.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.719+0000 7fa216000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa2100ae650 con 0x7fa21006c090 2024-04-20T21:19:15.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.719+0000 7fa216000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa21006e160 con 0x7fa21015eb30 2024-04-20T21:19:15.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.720+0000 7fa20ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2100aee50 0x7fa210158630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:15.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.720+0000 7fa20ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2100aee50 0x7fa210158630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42306/0 (socket says 172.21.15.86:42306) 2024-04-20T21:19:15.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.720+0000 7fa20ea00640 1 -- 172.21.15.86:0/2322213320 learned_addr learned my addr 172.21.15.86:0/2322213320 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:15.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.720+0000 7fa20ea00640 1 -- 172.21.15.86:0/2322213320 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa21015eb30 msgr2=0x7fa2100cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:15.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.720+0000 7fa20ea00640 1 --2- 172.21.15.86:0/2322213320 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa21015eb30 0x7fa2100cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:15.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.720+0000 7fa20ea00640 1 -- 172.21.15.86:0/2322213320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa21006c090 msgr2=0x7fa21014f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:15.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.720+0000 7fa20ea00640 1 --2- 172.21.15.86:0/2322213320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa21006c090 0x7fa21014f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:15.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.720+0000 7fa20ea00640 1 -- 172.21.15.86:0/2322213320 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2101497b0 con 0x7fa2100aee50 2024-04-20T21:19:15.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.721+0000 7fa20ea00640 1 --2- 172.21.15.86:0/2322213320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2100aee50 0x7fa210158630 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa1f800c380 tx=0x7fa1f800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:15.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.721+0000 7fa207400640 1 -- 172.21.15.86:0/2322213320 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f80052d0 con 0x7fa2100aee50 2024-04-20T21:19:15.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.721+0000 7fa207400640 1 -- 172.21.15.86:0/2322213320 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1f8005490 con 0x7fa2100aee50 2024-04-20T21:19:15.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.721+0000 7fa216000640 1 -- 172.21.15.86:0/2322213320 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa210149a90 con 0x7fa2100aee50 2024-04-20T21:19:15.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.721+0000 7fa207400640 1 -- 172.21.15.86:0/2322213320 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8007260 con 0x7fa2100aee50 2024-04-20T21:19:15.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.721+0000 7fa216000640 1 -- 172.21.15.86:0/2322213320 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa21014a050 con 0x7fa2100aee50 2024-04-20T21:19:15.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.722+0000 7fa216000640 1 -- 172.21.15.86:0/2322213320 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa21014a250 con 0x7fa2100aee50 2024-04-20T21:19:15.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.723+0000 7fa207400640 1 -- 172.21.15.86:0/2322213320 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa1f8007400 con 0x7fa2100aee50 2024-04-20T21:19:15.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.725+0000 7fa207400640 1 --2- 172.21.15.86:0/2322213320 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa1ec06c8f0 0x7fa1ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:15.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.725+0000 7fa207400640 1 -- 172.21.15.86:0/2322213320 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fa1f808b180 con 0x7fa2100aee50 2024-04-20T21:19:15.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.725+0000 7fa20f400640 1 --2- 172.21.15.86:0/2322213320 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa1ec06c8f0 0x7fa1ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:15.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.726+0000 7fa20f400640 1 --2- 172.21.15.86:0/2322213320 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa1ec06c8f0 0x7fa1ec06edb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa200036000 tx=0x7fa200030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:15.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:15.729+0000 7fa207400640 1 -- 172.21.15.86:0/2322213320 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa21014a250 con 0x7fa2100aee50 2024-04-20T21:19:16.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.004+0000 7fa216000640 1 -- 172.21.15.86:0/2322213320 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fa2100689e0 con 0x7fa2100aee50 2024-04-20T21:19:16.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.008+0000 7fa207400640 1 -- 172.21.15.86:0/2322213320 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7fa1f8059480 con 0x7fa2100aee50 2024-04-20T21:19:16.008 INFO:teuthology.orchestra.run.smithi086.stdout:[client.0] 2024-04-20T21:19:16.008 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQDBMCRmHWxyKRAANzhRi4eajp68h/jWkhFMnw== 2024-04-20T21:19:16.008 INFO:teuthology.orchestra.run.smithi086.stdout: caps mds = "allow" 2024-04-20T21:19:16.008 INFO:teuthology.orchestra.run.smithi086.stdout: caps mon = "allow r" 2024-04-20T21:19:16.008 INFO:teuthology.orchestra.run.smithi086.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-20T21:19:16.008 INFO:teuthology.orchestra.run.smithi086.stderr:updated caps for client.0 2024-04-20T21:19:16.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.010+0000 7fa216000640 1 -- 172.21.15.86:0/2322213320 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa1ec06c8f0 msgr2=0x7fa1ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:16.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.011+0000 7fa216000640 1 --2- 172.21.15.86:0/2322213320 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa1ec06c8f0 0x7fa1ec06edb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa200036000 tx=0x7fa200030000 comp rx=0 tx=0).stop 2024-04-20T21:19:16.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.011+0000 7fa216000640 1 -- 172.21.15.86:0/2322213320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2100aee50 msgr2=0x7fa210158630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:16.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.011+0000 7fa216000640 1 --2- 172.21.15.86:0/2322213320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2100aee50 0x7fa210158630 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa1f800c380 tx=0x7fa1f800c840 comp rx=0 tx=0).stop 2024-04-20T21:19:16.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.011+0000 7fa216000640 1 -- 172.21.15.86:0/2322213320 shutdown_connections 2024-04-20T21:19:16.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.011+0000 7fa216000640 1 --2- 172.21.15.86:0/2322213320 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa1ec06c8f0 0x7fa1ec06edb0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:16.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.011+0000 7fa216000640 1 --2- 172.21.15.86:0/2322213320 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa21015eb30 0x7fa2100cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:16.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.011+0000 7fa216000640 1 --2- 172.21.15.86:0/2322213320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa2100aee50 0x7fa210158630 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:16.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.011+0000 7fa216000640 1 --2- 172.21.15.86:0/2322213320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa21006c090 0x7fa21014f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:16.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.011+0000 7fa216000640 1 -- 172.21.15.86:0/2322213320 >> 172.21.15.86:0/2322213320 conn(0x7fa2100b2110 msgr2=0x7fa2100b1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:16.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.011+0000 7fa216000640 1 -- 172.21.15.86:0/2322213320 shutdown_connections 2024-04-20T21:19:16.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.011+0000 7fa216000640 1 -- 172.21.15.86:0/2322213320 wait complete. 2024-04-20T21:19:16.030 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:19:16.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.226+0000 7f81a3a00640 1 Processor -- start 2024-04-20T21:19:16.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.226+0000 7f81a3a00640 1 -- start start 2024-04-20T21:19:16.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.228+0000 7f81a3a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f819c144e90 0x7f819c0b4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:16.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.228+0000 7f81a3a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f819c0cdb50 con 0x7f819c06f450 2024-04-20T21:19:16.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.228+0000 7f81a3a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f819c0b3e50 con 0x7f819c0cd170 2024-04-20T21:19:16.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.228+0000 7f81a3a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f819c145270 con 0x7f819c144e90 2024-04-20T21:19:16.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.228+0000 7f81a1a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f819c06f450 0x7f819c15bb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:46242/0 (socket says 172.21.15.86:46242) 2024-04-20T21:19:16.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.228+0000 7f81a1a00640 1 -- 172.21.15.86:0/2968976486 learned_addr learned my addr 172.21.15.86:0/2968976486 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:19:16.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.229+0000 7f819be00640 1 -- 172.21.15.86:0/2968976486 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3352168619 0 0) 0x7f819c0cdb50 con 0x7f819c06f450 2024-04-20T21:19:16.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.229+0000 7f81a1000640 1 --2- 172.21.15.86:0/2968976486 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f819c144e90 0x7f819c0b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:16.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.229+0000 7f819be00640 1 -- 172.21.15.86:0/2968976486 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8180003520 con 0x7f819c06f450 2024-04-20T21:19:16.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.229+0000 7f819be00640 1 -- 172.21.15.86:0/2968976486 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1827101503 0 0) 0x7f8180003520 con 0x7f819c06f450 2024-04-20T21:19:16.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.229+0000 7f819be00640 1 -- 172.21.15.86:0/2968976486 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f819c144e90 msgr2=0x7f819c0b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:16.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.229+0000 7f819be00640 1 --2- 172.21.15.86:0/2968976486 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f819c144e90 0x7f819c0b4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:16.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.229+0000 7f819be00640 1 -- 172.21.15.86:0/2968976486 >> v1:172.21.15.144:6789/0 conn(0x7f819c0cd170 legacy=0x7f819c06ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:16.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.229+0000 7f819be00640 1 -- 172.21.15.86:0/2968976486 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f819c0b4130 con 0x7f819c06f450 2024-04-20T21:19:16.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.229+0000 7f819be00640 1 -- 172.21.15.86:0/2968976486 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f81940019f0 con 0x7f819c06f450 2024-04-20T21:19:16.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.229+0000 7f819be00640 1 -- 172.21.15.86:0/2968976486 >> v1:172.21.15.86:6789/0 conn(0x7f819c06f450 legacy=0x7f819c15bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:16.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.229+0000 7f819be00640 1 --2- 172.21.15.86:0/2968976486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81800039e0 0x7f8180003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:16.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.229+0000 7f819be00640 1 --2- 172.21.15.86:0/2968976486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8180004800 0x7f8180004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:16.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.229+0000 7f819be00640 1 --2- 172.21.15.86:0/2968976486 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f819c144e90 0x7f81800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:16.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.229+0000 7f819be00640 1 -- 172.21.15.86:0/2968976486 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f819c145270 con 0x7f8180004800 2024-04-20T21:19:16.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.229+0000 7f819be00640 1 -- 172.21.15.86:0/2968976486 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f818000ba70 con 0x7f81800039e0 2024-04-20T21:19:16.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.229+0000 7f819be00640 1 -- 172.21.15.86:0/2968976486 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f818000bc70 con 0x7f819c144e90 2024-04-20T21:19:16.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.229+0000 7f8193e00640 1 --2- 172.21.15.86:0/2968976486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8180004800 0x7f8180004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:16.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.230+0000 7f81a1000640 1 --2- 172.21.15.86:0/2968976486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81800039e0 0x7f8180003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:16.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.230+0000 7f8193e00640 1 -- 172.21.15.86:0/2968976486 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f819c144e90 msgr2=0x7f81800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:16.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.230+0000 7f8193e00640 1 --2- 172.21.15.86:0/2968976486 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f819c144e90 0x7f81800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:16.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.230+0000 7f8193e00640 1 -- 172.21.15.86:0/2968976486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81800039e0 msgr2=0x7f8180003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:16.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.230+0000 7f8193e00640 1 --2- 172.21.15.86:0/2968976486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81800039e0 0x7f8180003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:16.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.230+0000 7f8193e00640 1 -- 172.21.15.86:0/2968976486 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8180003520 con 0x7f8180004800 2024-04-20T21:19:16.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.231+0000 7f8193e00640 1 --2- 172.21.15.86:0/2968976486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8180004800 0x7f8180004c80 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f8184001720 tx=0x7f8180005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3fbe7d5ee764bab server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:16.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.231+0000 7f819be00640 1 -- 172.21.15.86:0/2968976486 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8184001370 con 0x7f8180004800 2024-04-20T21:19:16.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.231+0000 7f819be00640 1 -- 172.21.15.86:0/2968976486 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81840052d0 con 0x7f8180004800 2024-04-20T21:19:16.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.231+0000 7f819be00640 1 -- 172.21.15.86:0/2968976486 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8184003830 con 0x7f8180004800 2024-04-20T21:19:16.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.232+0000 7f81a3a00640 1 -- 172.21.15.86:0/2968976486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8180004800 msgr2=0x7f8180004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:16.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.232+0000 7f81a3a00640 1 --2- 172.21.15.86:0/2968976486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8180004800 0x7f8180004c80 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f8184001720 tx=0x7f8180005830 comp rx=0 tx=0).stop 2024-04-20T21:19:16.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.232+0000 7f81a1a00640 1 -- 172.21.15.86:0/2968976486 reap_dead start 2024-04-20T21:19:16.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.232+0000 7f81a3a00640 1 -- 172.21.15.86:0/2968976486 shutdown_connections 2024-04-20T21:19:16.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.232+0000 7f81a3a00640 1 -- 172.21.15.86:0/2968976486 >> 172.21.15.86:0/2968976486 conn(0x7f819c0b2360 msgr2=0x7f819c0b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:16.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.232+0000 7f81a3a00640 1 -- 172.21.15.86:0/2968976486 shutdown_connections 2024-04-20T21:19:16.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.232+0000 7f81a3a00640 1 -- 172.21.15.86:0/2968976486 wait complete. 2024-04-20T21:19:16.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.234+0000 7f81a3a00640 1 Processor -- start 2024-04-20T21:19:16.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.234+0000 7f81a3a00640 1 -- start start 2024-04-20T21:19:16.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.235+0000 7f81a3a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8180004800 0x7f819c1347a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:16.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.235+0000 7f81a3a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f819c1338b0 0x7f819c1324f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:16.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.235+0000 7f81a3a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f819c13b0d0 0x7f819c12fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:16.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.235+0000 7f81a3a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f819c0d0190 con 0x7f819c13b0d0 2024-04-20T21:19:16.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.235+0000 7f81a3a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f819c0d0010 con 0x7f8180004800 2024-04-20T21:19:16.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.235+0000 7f81a3a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f819c0cd5d0 con 0x7f819c1338b0 2024-04-20T21:19:16.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.235+0000 7f81a1a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f819c13b0d0 0x7f819c12fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:16.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.235+0000 7f81a1a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f819c13b0d0 0x7f819c12fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39698/0 (socket says 172.21.15.86:39698) 2024-04-20T21:19:16.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.235+0000 7f81a1a00640 1 -- 172.21.15.86:0/437705279 learned_addr learned my addr 172.21.15.86:0/437705279 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:16.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.235+0000 7f81a1a00640 1 -- 172.21.15.86:0/437705279 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f819c1338b0 msgr2=0x7f819c1324f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:16.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.235+0000 7f81a1a00640 1 --2- 172.21.15.86:0/437705279 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f819c1338b0 0x7f819c1324f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:16.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.235+0000 7f81a1a00640 1 -- 172.21.15.86:0/437705279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8180004800 msgr2=0x7f819c1347a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:16.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.235+0000 7f81a1a00640 1 --2- 172.21.15.86:0/437705279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8180004800 0x7f819c1347a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:16.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.235+0000 7f81a1a00640 1 -- 172.21.15.86:0/437705279 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f819c131450 con 0x7f819c13b0d0 2024-04-20T21:19:16.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.236+0000 7f81a1a00640 1 --2- 172.21.15.86:0/437705279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f819c13b0d0 0x7f819c12fc60 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f819c06f920 tx=0x7f819400ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:16.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.236+0000 7f8199600640 1 -- 172.21.15.86:0/437705279 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819400a5b0 con 0x7f819c13b0d0 2024-04-20T21:19:16.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.236+0000 7f8199600640 1 -- 172.21.15.86:0/437705279 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f819400a750 con 0x7f819c13b0d0 2024-04-20T21:19:16.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.236+0000 7f8199600640 1 -- 172.21.15.86:0/437705279 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819400ab40 con 0x7f819c13b0d0 2024-04-20T21:19:16.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.236+0000 7f81a3a00640 1 -- 172.21.15.86:0/437705279 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f819c131730 con 0x7f819c13b0d0 2024-04-20T21:19:16.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.236+0000 7f81a3a00640 1 -- 172.21.15.86:0/437705279 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f819c131cf0 con 0x7f819c13b0d0 2024-04-20T21:19:16.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.237+0000 7f81a3a00640 1 -- 172.21.15.86:0/437705279 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f819c06fe10 con 0x7f819c13b0d0 2024-04-20T21:19:16.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.238+0000 7f8199600640 1 -- 172.21.15.86:0/437705279 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8194011070 con 0x7f819c13b0d0 2024-04-20T21:19:16.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.239+0000 7f8199600640 1 --2- 172.21.15.86:0/437705279 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f817806c920 0x7f817806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:16.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.240+0000 7f8199600640 1 -- 172.21.15.86:0/437705279 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f819408b2b0 con 0x7f819c13b0d0 2024-04-20T21:19:16.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.240+0000 7f81a1000640 1 --2- 172.21.15.86:0/437705279 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f817806c920 0x7f817806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:16.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.240+0000 7f81a1000640 1 --2- 172.21.15.86:0/437705279 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f817806c920 0x7f817806ede0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f819c12f9a0 tx=0x7f818c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:16.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.244+0000 7f8199600640 1 -- 172.21.15.86:0/437705279 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f819c06fe10 con 0x7f819c13b0d0 2024-04-20T21:19:16.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.465+0000 7f81a3a00640 1 -- 172.21.15.86:0/437705279 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f819c0689e0 con 0x7f819c13b0d0 2024-04-20T21:19:16.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.467+0000 7f8199600640 1 -- 172.21.15.86:0/437705279 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5205 (secure 0 0 0) 0x7f8194019060 con 0x7f819c13b0d0 2024-04-20T21:19:16.466 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:19:16.466 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4604,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/1957936003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":1957936003},{"type":"v1","addr":"172.21.15.144:6834","nonce":1957936003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4807,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/1510649389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":1510649389},{"type":"v1","addr":"172.21.15.144:6835","nonce":1510649389}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2180009156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2180009156},{"type":"v1","addr":"172.21.15.86:6836","nonce":2180009156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:19:09.979345+0000","modified":"2024-04-20T21:19:13.064114+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5226":{"gid":5226,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6837/3715801064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3715801064},{"type":"v1","addr":"172.21.15.86:6837","nonce":3715801064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5226,"qdb_cluster":[5226]},"id":2}]} 2024-04-20T21:19:16.466 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 16 2024-04-20T21:19:16.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.469+0000 7f81a3a00640 1 -- 172.21.15.86:0/437705279 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f817806c920 msgr2=0x7f817806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:16.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.469+0000 7f81a3a00640 1 --2- 172.21.15.86:0/437705279 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f817806c920 0x7f817806ede0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f819c12f9a0 tx=0x7f818c008040 comp rx=0 tx=0).stop 2024-04-20T21:19:16.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.469+0000 7f81a3a00640 1 -- 172.21.15.86:0/437705279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f819c13b0d0 msgr2=0x7f819c12fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:16.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.469+0000 7f81a3a00640 1 --2- 172.21.15.86:0/437705279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f819c13b0d0 0x7f819c12fc60 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f819c06f920 tx=0x7f819400ccd0 comp rx=0 tx=0).stop 2024-04-20T21:19:16.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.470+0000 7f81a3a00640 1 -- 172.21.15.86:0/437705279 shutdown_connections 2024-04-20T21:19:16.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.470+0000 7f81a3a00640 1 --2- 172.21.15.86:0/437705279 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f817806c920 0x7f817806ede0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:16.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.470+0000 7f81a3a00640 1 --2- 172.21.15.86:0/437705279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f819c13b0d0 0x7f819c12fc60 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:16.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.470+0000 7f81a3a00640 1 --2- 172.21.15.86:0/437705279 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f819c1338b0 0x7f819c1324f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:16.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.470+0000 7f81a3a00640 1 --2- 172.21.15.86:0/437705279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8180004800 0x7f819c1347a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:16.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.470+0000 7f81a3a00640 1 -- 172.21.15.86:0/437705279 >> 172.21.15.86:0/437705279 conn(0x7f819c0b2360 msgr2=0x7f819c0b14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:16.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.470+0000 7f81a3a00640 1 -- 172.21.15.86:0/437705279 shutdown_connections 2024-04-20T21:19:16.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:16.470+0000 7f81a3a00640 1 -- 172.21.15.86:0/437705279 wait complete. 2024-04-20T21:19:16.892 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T21:19:09.979345+0000', 'modified': '2024-04-20T21:19:13.064114+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5226}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5226': {'gid': 5226, 'name': 'a', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.86:6837/3715801064', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6835', 'nonce': 3715801064}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3715801064}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5226, 'qdb_cluster': [5226]} 2024-04-20T21:19:16.892 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-20T21:19:16.892 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-20T21:19:17.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.103+0000 7f955fc00640 1 Processor -- start 2024-04-20T21:19:17.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.103+0000 7f955fc00640 1 -- start start 2024-04-20T21:19:17.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.105+0000 7f955fc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f955815e740 0x7f95580d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:17.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.105+0000 7f955fc00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9558163250 con 0x7f9558162ae0 2024-04-20T21:19:17.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.105+0000 7f955fc00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95580b34b0 con 0x7f95580d0fe0 2024-04-20T21:19:17.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.105+0000 7f955fc00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f95580b3790 con 0x7f955815e740 2024-04-20T21:19:17.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.105+0000 7f955dc00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f9558162ae0 0x7f95580adb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:46250/0 (socket says 172.21.15.86:46250) 2024-04-20T21:19:17.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.105+0000 7f955dc00640 1 -- 172.21.15.86:0/423645147 learned_addr learned my addr 172.21.15.86:0/423645147 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:19:17.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.105+0000 7f955d200640 1 --2- 172.21.15.86:0/423645147 >> v2:172.21.15.86:3300/0 conn(0x7f955815e740 0x7f95580d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:17.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.106+0000 7f955d200640 1 -- 172.21.15.86:0/423645147 >> v1:172.21.15.144:6789/0 conn(0x7f95580d0fe0 legacy=0x7f95581623d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:17.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.106+0000 7f955d200640 1 -- 172.21.15.86:0/423645147 >> v1:172.21.15.86:6789/0 conn(0x7f9558162ae0 legacy=0x7f95580adb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:17.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.106+0000 7f955d200640 1 -- 172.21.15.86:0/423645147 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95580b3a70 con 0x7f955815e740 2024-04-20T21:19:17.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.106+0000 7f9557400640 1 -- 172.21.15.86:0/423645147 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1995604490 0 0) 0x7f9558163250 con 0x7f9558162ae0 2024-04-20T21:19:17.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.107+0000 7f955d200640 1 --2- 172.21.15.86:0/423645147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f955815e740 0x7f95580d0aa0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f954c0097a0 tx=0x7f954c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e9b1a72776006b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:17.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.107+0000 7f9557400640 1 -- 172.21.15.86:0/423645147 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954c035020 con 0x7f955815e740 2024-04-20T21:19:17.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.107+0000 7f9557400640 1 -- 172.21.15.86:0/423645147 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f954c031830 con 0x7f955815e740 2024-04-20T21:19:17.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.107+0000 7f9557400640 1 -- 172.21.15.86:0/423645147 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954c031c00 con 0x7f955815e740 2024-04-20T21:19:17.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.108+0000 7f955fc00640 1 -- 172.21.15.86:0/423645147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f955815e740 msgr2=0x7f95580d0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:17.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.108+0000 7f955fc00640 1 --2- 172.21.15.86:0/423645147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f955815e740 0x7f95580d0aa0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f954c0097a0 tx=0x7f954c02b8d0 comp rx=0 tx=0).stop 2024-04-20T21:19:17.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.108+0000 7f955fc00640 1 -- 172.21.15.86:0/423645147 shutdown_connections 2024-04-20T21:19:17.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.108+0000 7f955fc00640 1 --2- 172.21.15.86:0/423645147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f955815e740 0x7f95580d0aa0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:17.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.108+0000 7f955fc00640 1 -- 172.21.15.86:0/423645147 >> 172.21.15.86:0/423645147 conn(0x7f955806b980 msgr2=0x7f955806bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:17.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.108+0000 7f955fc00640 1 -- 172.21.15.86:0/423645147 shutdown_connections 2024-04-20T21:19:17.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.108+0000 7f955fc00640 1 -- 172.21.15.86:0/423645147 wait complete. 2024-04-20T21:19:17.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.110+0000 7f955fc00640 1 Processor -- start 2024-04-20T21:19:17.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.110+0000 7f955fc00640 1 -- start start 2024-04-20T21:19:17.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.111+0000 7f955fc00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95580d0fe0 0x7f955814dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:17.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.111+0000 7f955fc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f955815e740 0x7f95580c6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:17.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.111+0000 7f955fc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9558162ae0 0x7f955815a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:17.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.111+0000 7f955fc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f95580b4550 con 0x7f955815e740 2024-04-20T21:19:17.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.111+0000 7f955fc00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f95580b4030 con 0x7f95580d0fe0 2024-04-20T21:19:17.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.111+0000 7f955fc00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f955815eb20 con 0x7f9558162ae0 2024-04-20T21:19:17.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.111+0000 7f9557e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f955815e740 0x7f95580c6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:17.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.111+0000 7f9557e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f955815e740 0x7f95580c6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39726/0 (socket says 172.21.15.86:39726) 2024-04-20T21:19:17.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.111+0000 7f9557e00640 1 -- 172.21.15.86:0/610071272 learned_addr learned my addr 172.21.15.86:0/610071272 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:17.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.111+0000 7f9557e00640 1 -- 172.21.15.86:0/610071272 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9558162ae0 msgr2=0x7f955815a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:17.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.111+0000 7f9557e00640 1 --2- 172.21.15.86:0/610071272 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9558162ae0 0x7f955815a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:17.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.111+0000 7f9557e00640 1 -- 172.21.15.86:0/610071272 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95580d0fe0 msgr2=0x7f955814dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:17.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.111+0000 7f9557e00640 1 --2- 172.21.15.86:0/610071272 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95580d0fe0 0x7f955814dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:17.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.111+0000 7f9557e00640 1 -- 172.21.15.86:0/610071272 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95581f8870 con 0x7f955815e740 2024-04-20T21:19:17.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.112+0000 7f9557e00640 1 --2- 172.21.15.86:0/610071272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f955815e740 0x7f95580c6100 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f954400c400 tx=0x7f954400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:17.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.112+0000 7f9554c00640 1 -- 172.21.15.86:0/610071272 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9544011790 con 0x7f955815e740 2024-04-20T21:19:17.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.112+0000 7f9554c00640 1 -- 172.21.15.86:0/610071272 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9544017070 con 0x7f955815e740 2024-04-20T21:19:17.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.112+0000 7f9554c00640 1 -- 172.21.15.86:0/610071272 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9544011bb0 con 0x7f955815e740 2024-04-20T21:19:17.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.112+0000 7f955fc00640 1 -- 172.21.15.86:0/610071272 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95580c5340 con 0x7f955815e740 2024-04-20T21:19:17.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.112+0000 7f955fc00640 1 -- 172.21.15.86:0/610071272 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95580c5900 con 0x7f955815e740 2024-04-20T21:19:17.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.114+0000 7f9554c00640 1 -- 172.21.15.86:0/610071272 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f954401c460 con 0x7f955815e740 2024-04-20T21:19:17.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.115+0000 7f9554c00640 1 --2- 172.21.15.86:0/610071272 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f953806c8f0 0x7f953806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:17.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.115+0000 7f9554c00640 1 -- 172.21.15.86:0/610071272 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f954408ba00 con 0x7f955815e740 2024-04-20T21:19:17.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.116+0000 7f955d200640 1 --2- 172.21.15.86:0/610071272 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f953806c8f0 0x7f953806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:17.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.116+0000 7f95625fd740 1 Processor -- start 2024-04-20T21:19:17.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.116+0000 7f95625fd740 1 -- start start 2024-04-20T21:19:17.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.116+0000 7f955d200640 1 --2- 172.21.15.86:0/610071272 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f953806c8f0 0x7f953806edb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f954c036000 tx=0x7f954c030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:17.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.117+0000 7f95625fd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56257e827460 0x56257e7dea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:17.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.117+0000 7f95625fd740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56257e7dd4e0 0x56257e7defd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:17.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.117+0000 7f95625fd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56257e7d9930 0x56257e809770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:17.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.117+0000 7f95625fd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56257e7459d0 con 0x56257e827460 2024-04-20T21:19:17.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.117+0000 7f95625fd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56257e6ec270 con 0x56257e7d9930 2024-04-20T21:19:17.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.117+0000 7f95625fd740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x56257e741990 con 0x56257e7dd4e0 2024-04-20T21:19:17.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.117+0000 7f955d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56257e827460 0x56257e7dea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:17.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.117+0000 7f955d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56257e827460 0x56257e7dea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39738/0 (socket says 172.21.15.86:39738) 2024-04-20T21:19:17.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.117+0000 7f955d200640 1 -- 172.21.15.86:0/1581011427 learned_addr learned my addr 172.21.15.86:0/1581011427 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:17.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.118+0000 7f9557e00640 1 --2- 172.21.15.86:0/1581011427 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56257e7dd4e0 0x56257e7defd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:17.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.118+0000 7f955d200640 1 -- 172.21.15.86:0/1581011427 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56257e7dd4e0 msgr2=0x56257e7defd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:17.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.118+0000 7f955d200640 1 --2- 172.21.15.86:0/1581011427 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56257e7dd4e0 0x56257e7defd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:17.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.118+0000 7f955dc00640 1 --2- 172.21.15.86:0/1581011427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56257e7d9930 0x56257e809770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:17.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.118+0000 7f955d200640 1 -- 172.21.15.86:0/1581011427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56257e7d9930 msgr2=0x56257e809770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:17.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.118+0000 7f955d200640 1 --2- 172.21.15.86:0/1581011427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56257e7d9930 0x56257e809770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:17.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.118+0000 7f955d200640 1 -- 172.21.15.86:0/1581011427 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56257e4582d0 con 0x56257e827460 2024-04-20T21:19:17.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.118+0000 7f955dc00640 1 --2- 172.21.15.86:0/1581011427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56257e7d9930 0x56257e809770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:17.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.118+0000 7f955d200640 1 --2- 172.21.15.86:0/1581011427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56257e827460 0x56257e7dea90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f954c02b910 tx=0x7f954c031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=442b02dab6e5ef2e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:17.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.119+0000 7f9549600640 1 -- 172.21.15.86:0/1581011427 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954c03cb90 con 0x56257e827460 2024-04-20T21:19:17.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.119+0000 7f9549600640 1 -- 172.21.15.86:0/1581011427 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f954c03cd30 con 0x56257e827460 2024-04-20T21:19:17.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.119+0000 7f9549600640 1 -- 172.21.15.86:0/1581011427 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954c03a670 con 0x56257e827460 2024-04-20T21:19:17.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.119+0000 7f95625fd740 1 -- 172.21.15.86:0/1581011427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56257e827460 msgr2=0x56257e7dea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:17.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.119+0000 7f95625fd740 1 --2- 172.21.15.86:0/1581011427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56257e827460 0x56257e7dea90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f954c02b910 tx=0x7f954c031000 comp rx=0 tx=0).stop 2024-04-20T21:19:17.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.119+0000 7f95625fd740 1 -- 172.21.15.86:0/1581011427 shutdown_connections 2024-04-20T21:19:17.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.119+0000 7f95625fd740 1 --2- 172.21.15.86:0/1581011427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56257e7d9930 0x56257e809770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:17.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.119+0000 7f95625fd740 1 --2- 172.21.15.86:0/1581011427 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56257e7dd4e0 0x56257e7defd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:17.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.119+0000 7f95625fd740 1 --2- 172.21.15.86:0/1581011427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56257e827460 0x56257e7dea90 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:17.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.119+0000 7f95625fd740 1 -- 172.21.15.86:0/1581011427 >> 172.21.15.86:0/1581011427 conn(0x56257e7dbbf0 msgr2=0x56257e824770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:17.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.119+0000 7f95625fd740 1 -- 172.21.15.86:0/1581011427 shutdown_connections 2024-04-20T21:19:17.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.120+0000 7f95625fd740 1 -- 172.21.15.86:0/1581011427 wait complete. 2024-04-20T21:19:17.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.120+0000 7f95625fd740 1 -- start start 2024-04-20T21:19:17.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.120+0000 7f95625fd740 1 Processor -- start 2024-04-20T21:19:17.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.121+0000 7f95625fd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56257e7dd4e0 0x56257e800470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:17.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.121+0000 7f95625fd740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56257e827460 0x56257e82bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:17.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.121+0000 7f95625fd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56257e800f60 0x56257e835340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:17.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.121+0000 7f95625fd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56257e7dd9f0 con 0x56257e800f60 2024-04-20T21:19:17.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.121+0000 7f95625fd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56257e6ff5e0 con 0x56257e7dd4e0 2024-04-20T21:19:17.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.121+0000 7f95625fd740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x56257e7ddcd0 con 0x56257e827460 2024-04-20T21:19:17.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.121+0000 7f95625fd740 10 client.0 _ll_register_callbacks cb 0x56257e457ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:19:17.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.122+0000 7f955dc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56257e800f60 0x56257e835340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:17.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.122+0000 7f955dc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56257e800f60 0x56257e835340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39750/0 (socket says 172.21.15.86:39750) 2024-04-20T21:19:17.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.122+0000 7f955dc00640 1 -- 172.21.15.86:0/244576377 learned_addr learned my addr 172.21.15.86:0/244576377 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:17.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.122+0000 7f9557e00640 1 --2- 172.21.15.86:0/244576377 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56257e827460 0x56257e82bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:17.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.122+0000 7f955d200640 1 --2- 172.21.15.86:0/244576377 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56257e7dd4e0 0x56257e800470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:17.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.122+0000 7f955dc00640 1 -- 172.21.15.86:0/244576377 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56257e827460 msgr2=0x56257e82bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:17.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.122+0000 7f955dc00640 1 --2- 172.21.15.86:0/244576377 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56257e827460 0x56257e82bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:17.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.122+0000 7f955dc00640 1 -- 172.21.15.86:0/244576377 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56257e7dd4e0 msgr2=0x56257e800470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:17.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.122+0000 7f955dc00640 1 --2- 172.21.15.86:0/244576377 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56257e7dd4e0 0x56257e800470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:17.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.122+0000 7f955dc00640 1 -- 172.21.15.86:0/244576377 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56257e5226b0 con 0x56257e800f60 2024-04-20T21:19:17.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.122+0000 7f955dc00640 1 --2- 172.21.15.86:0/244576377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56257e800f60 0x56257e835340 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f955000d4b0 tx=0x7f955000d970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac8da6d6dd54fd56 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:17.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.123+0000 7f953e000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:19:17.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.123+0000 7f953e000640 1 -- 172.21.15.86:0/244576377 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95500070d0 con 0x56257e800f60 2024-04-20T21:19:17.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.123+0000 7f953e000640 1 -- 172.21.15.86:0/244576377 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f9550013a80 con 0x56257e800f60 2024-04-20T21:19:17.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.123+0000 7f95625fd740 1 -- 172.21.15.86:0/244576377 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f954c03cd30 con 0x56257e800f60 2024-04-20T21:19:17.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.123+0000 7f953e000640 1 -- 172.21.15.86:0/244576377 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f9550013c80 con 0x56257e800f60 2024-04-20T21:19:17.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.123+0000 7f95625fd740 10 client.5394 fetch_fsmap learned FSMap version 16 2024-04-20T21:19:17.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.123+0000 7f95625fd740 1 -- 172.21.15.86:0/244576377 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x56257e82b6c0 con 0x56257e800f60 2024-04-20T21:19:17.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.123+0000 7f953e000640 1 -- 172.21.15.86:0/244576377 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f954c03cd30 con 0x56257e800f60 2024-04-20T21:19:17.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.123+0000 7f953e000640 1 -- 172.21.15.86:0/244576377 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 16) ==== 3495+0+0 (secure 0 0 0) 0x7f9550013450 con 0x56257e800f60 2024-04-20T21:19:17.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.124+0000 7f95625fd740 10 client.5394 fetch_fsmap finished waiting for FSMap version 16 2024-04-20T21:19:17.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.124+0000 7f95625fd740 10 client.5394 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-20T21:19:17.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.124+0000 7f95625fd740 20 client.5394 populate_metadata read hostname 'smithi086' 2024-04-20T21:19:17.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.124+0000 7f95625fd740 1 --2- 172.21.15.86:0/244576377 >> [v2:172.21.15.86:6835/3715801064,v1:172.21.15.86:6837/3715801064] conn(0x56257e829650 0x56257e843c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:17.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.124+0000 7f95625fd740 4 client.5394 mds_command: new command op to 5226 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:19:17.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.124+0000 7f95625fd740 1 -- 172.21.15.86:0/244576377 --> [v2:172.21.15.86:6835/3715801064,v1:172.21.15.86:6837/3715801064] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56257e7533f0 con 0x56257e829650 2024-04-20T21:19:17.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.124+0000 7f955d200640 1 --2- 172.21.15.86:0/244576377 >> [v2:172.21.15.86:6835/3715801064,v1:172.21.15.86:6837/3715801064] conn(0x56257e829650 0x56257e843c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:17.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.124+0000 7f955d200640 1 --2- 172.21.15.86:0/244576377 >> [v2:172.21.15.86:6835/3715801064,v1:172.21.15.86:6837/3715801064] conn(0x56257e829650 0x56257e843c50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d832694a4a9d32f server_cookie=7bed25e8acf4f7b7 in_seq=0 out_seq=0 2024-04-20T21:19:17.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.124+0000 7f953e000640 10 client.5394 ms_handle_connect on v2:172.21.15.86:6835/3715801064 2024-04-20T21:19:17.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.125+0000 7f953e000640 1 -- 172.21.15.86:0/244576377 <== mds.0 v2:172.21.15.86:6835/3715801064 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56257e7533f0 con 0x56257e829650 2024-04-20T21:19:17.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.125+0000 7f953e000640 10 client.5394 handle_command_reply: tid=0 2024-04-20T21:19:17.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.125+0000 7f95625fd740 1 client.5394 shutdown 2024-04-20T21:19:17.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.126+0000 7f95625fd740 1 -- 172.21.15.86:0/244576377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56257e800f60 msgr2=0x56257e835340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:17.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.126+0000 7f95625fd740 1 --2- 172.21.15.86:0/244576377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56257e800f60 0x56257e835340 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f955000d4b0 tx=0x7f955000d970 comp rx=0 tx=0).stop 2024-04-20T21:19:17.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.126+0000 7f95625fd740 1 -- 172.21.15.86:0/244576377 shutdown_connections 2024-04-20T21:19:17.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.126+0000 7f95625fd740 1 --2- 172.21.15.86:0/244576377 >> [v2:172.21.15.86:6835/3715801064,v1:172.21.15.86:6837/3715801064] conn(0x56257e829650 0x56257e843c50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:17.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.126+0000 7f95625fd740 1 --2- 172.21.15.86:0/244576377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56257e800f60 0x56257e835340 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:17.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.126+0000 7f95625fd740 1 --2- 172.21.15.86:0/244576377 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56257e827460 0x56257e82bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:17.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.126+0000 7f95625fd740 1 --2- 172.21.15.86:0/244576377 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56257e7dd4e0 0x56257e800470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:17.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.126+0000 7f953e000640 0 client.5394 ms_handle_reset on v2:172.21.15.86:6835/3715801064 2024-04-20T21:19:17.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.126+0000 7f95625fd740 1 -- 172.21.15.86:0/244576377 >> 172.21.15.86:0/244576377 conn(0x56257e7d9930 msgr2=0x56257e824570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:17.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.126+0000 7f95625fd740 1 -- 172.21.15.86:0/244576377 shutdown_connections 2024-04-20T21:19:17.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.126+0000 7f95625fd740 1 -- 172.21.15.86:0/244576377 wait complete. 2024-04-20T21:19:17.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.126+0000 7f95625fd740 20 client.5394 trim_cache size 0 max 16384 2024-04-20T21:19:17.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.146+0000 7f95625fd740 1 Processor -- start 2024-04-20T21:19:17.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.146+0000 7f95625fd740 1 -- start start 2024-04-20T21:19:17.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.147+0000 7f95625fd740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56257e7dd4e0 0x56257e7df790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:17.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.147+0000 7f95625fd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56257e800f60 0x56257e807650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:17.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.147+0000 7f95625fd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56257e827460 0x56257e807b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:17.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.147+0000 7f95625fd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f95581ffd70 con 0x56257e800f60 2024-04-20T21:19:17.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.147+0000 7f95625fd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f955815c880 con 0x56257e827460 2024-04-20T21:19:17.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.147+0000 7f95625fd740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f955815ced0 con 0x56257e7dd4e0 2024-04-20T21:19:17.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.147+0000 7f9557e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56257e800f60 0x56257e807650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:17.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.147+0000 7f9557e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56257e800f60 0x56257e807650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39762/0 (socket says 172.21.15.86:39762) 2024-04-20T21:19:17.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.148+0000 7f9557e00640 1 -- 172.21.15.86:0/1215270466 learned_addr learned my addr 172.21.15.86:0/1215270466 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:17.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.148+0000 7f955d200640 1 --2- 172.21.15.86:0/1215270466 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56257e7dd4e0 0x56257e7df790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:17.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.148+0000 7f955dc00640 1 --2- 172.21.15.86:0/1215270466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56257e827460 0x56257e807b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:17.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.148+0000 7f9557e00640 1 -- 172.21.15.86:0/1215270466 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56257e7dd4e0 msgr2=0x56257e7df790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:17.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.148+0000 7f9557e00640 1 --2- 172.21.15.86:0/1215270466 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56257e7dd4e0 0x56257e7df790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:17.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.148+0000 7f9557e00640 1 -- 172.21.15.86:0/1215270466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56257e827460 msgr2=0x56257e807b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:17.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.148+0000 7f9557e00640 1 --2- 172.21.15.86:0/1215270466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56257e827460 0x56257e807b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:17.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.148+0000 7f9557e00640 1 -- 172.21.15.86:0/1215270466 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56257e6c47b0 con 0x56257e800f60 2024-04-20T21:19:17.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.148+0000 7f955d200640 1 --2- 172.21.15.86:0/1215270466 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56257e7dd4e0 0x56257e7df790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:19:17.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.148+0000 7f9557e00640 1 --2- 172.21.15.86:0/1215270466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56257e800f60 0x56257e807650 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f9544055b80 tx=0x7f9544060040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8536f1231f48f14e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:17.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.149+0000 7f9548c00640 1 -- 172.21.15.86:0/1215270466 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9544088070 con 0x56257e800f60 2024-04-20T21:19:17.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.149+0000 7f9548c00640 1 -- 172.21.15.86:0/1215270466 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f954405dad0 con 0x56257e800f60 2024-04-20T21:19:17.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.149+0000 7f9548c00640 1 -- 172.21.15.86:0/1215270466 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9544062300 con 0x56257e800f60 2024-04-20T21:19:17.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.149+0000 7f95625fd740 1 -- 172.21.15.86:0/1215270466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56257e800f60 msgr2=0x56257e807650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:17.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.149+0000 7f95625fd740 1 --2- 172.21.15.86:0/1215270466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56257e800f60 0x56257e807650 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f9544055b80 tx=0x7f9544060040 comp rx=0 tx=0).stop 2024-04-20T21:19:17.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.149+0000 7f95625fd740 1 -- 172.21.15.86:0/1215270466 shutdown_connections 2024-04-20T21:19:17.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.149+0000 7f95625fd740 1 --2- 172.21.15.86:0/1215270466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56257e827460 0x56257e807b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:17.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.149+0000 7f95625fd740 1 --2- 172.21.15.86:0/1215270466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56257e800f60 0x56257e807650 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:17.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.149+0000 7f95625fd740 1 --2- 172.21.15.86:0/1215270466 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56257e7dd4e0 0x56257e7df790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:17.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.149+0000 7f95625fd740 1 -- 172.21.15.86:0/1215270466 >> 172.21.15.86:0/1215270466 conn(0x56257e7d9930 msgr2=0x56257e8350d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:17.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.149+0000 7f95625fd740 1 -- 172.21.15.86:0/1215270466 shutdown_connections 2024-04-20T21:19:17.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.149+0000 7f95625fd740 1 -- 172.21.15.86:0/1215270466 wait complete. 2024-04-20T21:19:17.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.150+0000 7f95625fd740 1 -- start start 2024-04-20T21:19:17.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.150+0000 7f95625fd740 1 Processor -- start 2024-04-20T21:19:17.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.151+0000 7f95625fd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56257e7dd4e0 0x56257e89f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:17.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.151+0000 7f95625fd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56257e800f60 0x56257e848f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:17.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.151+0000 7f95625fd740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56257e827460 0x56257e849480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:17.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.151+0000 7f95625fd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56257e835e00 con 0x56257e7dd4e0 2024-04-20T21:19:17.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.151+0000 7f95625fd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56257e82c1d0 con 0x56257e800f60 2024-04-20T21:19:17.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.151+0000 7f95625fd740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x56257e805130 con 0x56257e827460 2024-04-20T21:19:17.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.151+0000 7f95625fd740 10 client.0 _ll_register_callbacks cb 0x7f9550001530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:19:17.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.152+0000 7f955d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56257e7dd4e0 0x56257e89f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:17.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.152+0000 7f955d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56257e7dd4e0 0x56257e89f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39776/0 (socket says 172.21.15.86:39776) 2024-04-20T21:19:17.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.152+0000 7f955d200640 1 -- 172.21.15.86:0/3050883207 learned_addr learned my addr 172.21.15.86:0/3050883207 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:17.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.152+0000 7f9557e00640 1 --2- 172.21.15.86:0/3050883207 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56257e800f60 0x56257e848f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:17.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.152+0000 7f955dc00640 1 --2- 172.21.15.86:0/3050883207 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56257e827460 0x56257e849480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:17.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.152+0000 7f955d200640 1 -- 172.21.15.86:0/3050883207 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56257e827460 msgr2=0x56257e849480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:17.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.152+0000 7f955d200640 1 --2- 172.21.15.86:0/3050883207 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56257e827460 0x56257e849480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:17.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.152+0000 7f955d200640 1 -- 172.21.15.86:0/3050883207 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56257e800f60 msgr2=0x56257e848f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:17.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.152+0000 7f955d200640 1 --2- 172.21.15.86:0/3050883207 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56257e800f60 0x56257e848f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:17.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.152+0000 7f955d200640 1 -- 172.21.15.86:0/3050883207 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56257e7d90d0 con 0x56257e7dd4e0 2024-04-20T21:19:17.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.152+0000 7f9557e00640 1 --2- 172.21.15.86:0/3050883207 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56257e800f60 0x56257e848f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:17.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.152+0000 7f955d200640 1 --2- 172.21.15.86:0/3050883207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56257e7dd4e0 0x56257e89f9d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f954c0098c0 tx=0x7f954c02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10d41eac69037647 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:17.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.152+0000 7f953e000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:19:17.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.153+0000 7f953e000640 1 -- 172.21.15.86:0/3050883207 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954c049830 con 0x56257e7dd4e0 2024-04-20T21:19:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.153+0000 7f953e000640 1 -- 172.21.15.86:0/3050883207 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f954c0499d0 con 0x56257e7dd4e0 2024-04-20T21:19:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.153+0000 7f95625fd740 1 -- 172.21.15.86:0/3050883207 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f954405dad0 con 0x56257e7dd4e0 2024-04-20T21:19:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.153+0000 7f953e000640 1 -- 172.21.15.86:0/3050883207 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f954c066050 con 0x56257e7dd4e0 2024-04-20T21:19:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.153+0000 7f95625fd740 10 client.5400 fetch_fsmap learned FSMap version 16 2024-04-20T21:19:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.153+0000 7f95625fd740 1 -- 172.21.15.86:0/3050883207 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x56257e7d95d0 con 0x56257e7dd4e0 2024-04-20T21:19:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.153+0000 7f953e000640 1 -- 172.21.15.86:0/3050883207 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f954405dad0 con 0x56257e7dd4e0 2024-04-20T21:19:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.153+0000 7f953e000640 1 -- 172.21.15.86:0/3050883207 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 16) ==== 3495+0+0 (secure 0 0 0) 0x7f954c069020 con 0x56257e7dd4e0 2024-04-20T21:19:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.153+0000 7f95625fd740 10 client.5400 fetch_fsmap finished waiting for FSMap version 16 2024-04-20T21:19:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.153+0000 7f95625fd740 10 client.5400 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-20T21:19:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.153+0000 7f95625fd740 20 client.5400 populate_metadata read hostname 'smithi086' 2024-04-20T21:19:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.153+0000 7f95625fd740 1 --2- 172.21.15.86:0/3050883207 >> [v2:172.21.15.86:6835/3715801064,v1:172.21.15.86:6837/3715801064] conn(0x56257e807e70 0x56257e8259e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.153+0000 7f95625fd740 4 client.5400 mds_command: new command op to 5226 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T21:19:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.153+0000 7f95625fd740 1 -- 172.21.15.86:0/3050883207 --> [v2:172.21.15.86:6835/3715801064,v1:172.21.15.86:6837/3715801064] -- command(tid 0: {"prefix": "status"}) -- 0x56257e74d260 con 0x56257e807e70 2024-04-20T21:19:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.154+0000 7f9557e00640 1 --2- 172.21.15.86:0/3050883207 >> [v2:172.21.15.86:6835/3715801064,v1:172.21.15.86:6837/3715801064] conn(0x56257e807e70 0x56257e8259e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:17.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.154+0000 7f9557e00640 1 --2- 172.21.15.86:0/3050883207 >> [v2:172.21.15.86:6835/3715801064,v1:172.21.15.86:6837/3715801064] conn(0x56257e807e70 0x56257e8259e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccc624ea162dcc2d server_cookie=aa7b8f708220c415 in_seq=0 out_seq=0 2024-04-20T21:19:17.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.154+0000 7f953e000640 10 client.5400 ms_handle_connect on v2:172.21.15.86:6835/3715801064 2024-04-20T21:19:17.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.154+0000 7f953e000640 1 -- 172.21.15.86:0/3050883207 <== mds.0 v2:172.21.15.86:6835/3715801064 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x56257e74d260 con 0x56257e807e70 2024-04-20T21:19:17.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.154+0000 7f953e000640 10 client.5400 handle_command_reply: tid=0 2024-04-20T21:19:17.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.154+0000 7f95625fd740 1 client.5400 shutdown 2024-04-20T21:19:17.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.154+0000 7f95625fd740 1 -- 172.21.15.86:0/3050883207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56257e7dd4e0 msgr2=0x56257e89f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:17.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.154+0000 7f95625fd740 1 --2- 172.21.15.86:0/3050883207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56257e7dd4e0 0x56257e89f9d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f954c0098c0 tx=0x7f954c02e000 comp rx=0 tx=0).stop 2024-04-20T21:19:17.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.154+0000 7f95625fd740 1 -- 172.21.15.86:0/3050883207 shutdown_connections 2024-04-20T21:19:17.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.154+0000 7f95625fd740 1 --2- 172.21.15.86:0/3050883207 >> [v2:172.21.15.86:6835/3715801064,v1:172.21.15.86:6837/3715801064] conn(0x56257e807e70 0x56257e8259e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:17.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.154+0000 7f95625fd740 1 --2- 172.21.15.86:0/3050883207 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56257e827460 0x56257e849480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:17.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.154+0000 7f95625fd740 1 --2- 172.21.15.86:0/3050883207 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56257e800f60 0x56257e848f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:17.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.154+0000 7f95625fd740 1 --2- 172.21.15.86:0/3050883207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56257e7dd4e0 0x56257e89f9d0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:17.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.154+0000 7f953e000640 0 client.5400 ms_handle_reset on v2:172.21.15.86:6835/3715801064 2024-04-20T21:19:17.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.155+0000 7f95625fd740 1 -- 172.21.15.86:0/3050883207 >> 172.21.15.86:0/3050883207 conn(0x56257e7d9930 msgr2=0x56257e8350d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:17.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.155+0000 7f95625fd740 1 -- 172.21.15.86:0/3050883207 shutdown_connections 2024-04-20T21:19:17.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.155+0000 7f95625fd740 1 -- 172.21.15.86:0/3050883207 wait complete. 2024-04-20T21:19:17.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.155+0000 7f95625fd740 20 client.5400 trim_cache size 0 max 16384 2024-04-20T21:19:17.162 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:19:17.162 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "9c2dec1d-d7ae-4a83-a812-79922808e278", 2024-04-20T21:19:17.162 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-04-20T21:19:17.162 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 5226, 2024-04-20T21:19:17.162 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-20T21:19:17.163 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-20T21:19:17.163 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-20T21:19:17.163 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 6.6142931010000003, 2024-04-20T21:19:17.163 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 16, 2024-04-20T21:19:17.163 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 33, 2024-04-20T21:19:17.163 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 33, 2024-04-20T21:19:17.163 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 15.462449149999999 2024-04-20T21:19:17.163 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:19:17.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.155+0000 7f955fc00640 1 -- 172.21.15.86:0/610071272 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f953806c8f0 msgr2=0x7f953806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:17.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.155+0000 7f955fc00640 1 --2- 172.21.15.86:0/610071272 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f953806c8f0 0x7f953806edb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f954c036000 tx=0x7f954c030000 comp rx=0 tx=0).stop 2024-04-20T21:19:17.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.155+0000 7f955fc00640 1 -- 172.21.15.86:0/610071272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f955815e740 msgr2=0x7f95580c6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:17.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.155+0000 7f955fc00640 1 --2- 172.21.15.86:0/610071272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f955815e740 0x7f95580c6100 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f954400c400 tx=0x7f954400c8c0 comp rx=0 tx=0).stop 2024-04-20T21:19:17.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.156+0000 7f955fc00640 1 -- 172.21.15.86:0/610071272 shutdown_connections 2024-04-20T21:19:17.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.156+0000 7f955fc00640 1 --2- 172.21.15.86:0/610071272 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f953806c8f0 0x7f953806edb0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:17.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.156+0000 7f955fc00640 1 --2- 172.21.15.86:0/610071272 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9558162ae0 0x7f955815a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:17.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.156+0000 7f955fc00640 1 --2- 172.21.15.86:0/610071272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f955815e740 0x7f95580c6100 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:17.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.156+0000 7f955fc00640 1 --2- 172.21.15.86:0/610071272 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95580d0fe0 0x7f955814dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:17.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.156+0000 7f955fc00640 1 -- 172.21.15.86:0/610071272 >> 172.21.15.86:0/610071272 conn(0x7f955806b980 msgr2=0x7f9558158bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:17.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.156+0000 7f955fc00640 1 -- 172.21.15.86:0/610071272 shutdown_connections 2024-04-20T21:19:17.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.156+0000 7f955fc00640 1 -- 172.21.15.86:0/610071272 wait complete. 2024-04-20T21:19:17.172 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T21:19:17.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.366+0000 7f916aa00640 1 Processor -- start 2024-04-20T21:19:17.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.366+0000 7f916aa00640 1 -- start start 2024-04-20T21:19:17.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.367+0000 7f916aa00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9164144e90 0x7f916415ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:17.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.367+0000 7f916aa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f916406f140 0x7f91640aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:17.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.367+0000 7f916aa00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f916406dc30 con 0x7f91640af0b0 2024-04-20T21:19:17.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.367+0000 7f916aa00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f9164145270 con 0x7f9164144e90 2024-04-20T21:19:17.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.367+0000 7f916aa00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f916415f720 con 0x7f916406f140 2024-04-20T21:19:17.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.368+0000 7f9163400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f916406f140 0x7f91640aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:17.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.368+0000 7f9163400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f916406f140 0x7f91640aeb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:37394/0 (socket says 172.21.15.86:37394) 2024-04-20T21:19:17.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.368+0000 7f9163400640 1 -- 172.21.15.86:0/2923294873 learned_addr learned my addr 172.21.15.86:0/2923294873 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:17.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.368+0000 7f9163e00640 1 --2- 172.21.15.86:0/2923294873 >> v2:172.21.15.144:3300/0 conn(0x7f9164144e90 0x7f916415ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:17.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.369+0000 7f9163400640 1 -- 172.21.15.86:0/2923294873 >> v2:172.21.15.144:3300/0 conn(0x7f9164144e90 msgr2=0x7f916415ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:17.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.369+0000 7f9163400640 1 --2- 172.21.15.86:0/2923294873 >> v2:172.21.15.144:3300/0 conn(0x7f9164144e90 0x7f916415ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:17.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.369+0000 7f9163400640 1 -- 172.21.15.86:0/2923294873 >> v1:172.21.15.144:6789/0 conn(0x7f91640af0b0 legacy=0x7f9164071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:17.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.369+0000 7f9163400640 1 -- 172.21.15.86:0/2923294873 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f916415fa00 con 0x7f916406f140 2024-04-20T21:19:17.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.369+0000 7f9163e00640 1 --2- 172.21.15.86:0/2923294873 >> v2:172.21.15.144:3300/0 conn(0x7f9164144e90 0x7f916415ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:19:17.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.371+0000 7f9163400640 1 --2- 172.21.15.86:0/2923294873 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f916406f140 0x7f91640aeb70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f91641453f0 tx=0x7f915002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fea272163b5a5555 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:17.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.371+0000 7f9162a00640 1 -- 172.21.15.86:0/2923294873 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9150034070 con 0x7f916406f140 2024-04-20T21:19:17.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.371+0000 7f9162a00640 1 -- 172.21.15.86:0/2923294873 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9150030840 con 0x7f916406f140 2024-04-20T21:19:17.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.371+0000 7f9162a00640 1 -- 172.21.15.86:0/2923294873 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9150030c10 con 0x7f916406f140 2024-04-20T21:19:17.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.372+0000 7f916aa00640 1 -- 172.21.15.86:0/2923294873 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f916406f140 msgr2=0x7f91640aeb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:17.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.372+0000 7f916aa00640 1 --2- 172.21.15.86:0/2923294873 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f916406f140 0x7f91640aeb70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f91641453f0 tx=0x7f915002b980 comp rx=0 tx=0).stop 2024-04-20T21:19:17.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.372+0000 7f916aa00640 1 -- 172.21.15.86:0/2923294873 shutdown_connections 2024-04-20T21:19:17.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.372+0000 7f916aa00640 1 --2- 172.21.15.86:0/2923294873 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f916406f140 0x7f91640aeb70 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:17.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.372+0000 7f916aa00640 1 --2- 172.21.15.86:0/2923294873 >> v2:172.21.15.144:3300/0 conn(0x7f9164144e90 0x7f916415ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:17.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.372+0000 7f916aa00640 1 -- 172.21.15.86:0/2923294873 >> 172.21.15.86:0/2923294873 conn(0x7f91640b2360 msgr2=0x7f91640b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:17.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.372+0000 7f916aa00640 1 -- 172.21.15.86:0/2923294873 shutdown_connections 2024-04-20T21:19:17.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.372+0000 7f916aa00640 1 -- 172.21.15.86:0/2923294873 wait complete. 2024-04-20T21:19:17.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.374+0000 7f916aa00640 1 Processor -- start 2024-04-20T21:19:17.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.374+0000 7f916aa00640 1 -- start start 2024-04-20T21:19:17.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.375+0000 7f916aa00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f916406f140 0x7f91640cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:17.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.375+0000 7f916aa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f91640af0b0 0x7f916415dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:17.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.375+0000 7f916aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9164144e90 0x7f9164147ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:17.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.375+0000 7f916aa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f916415f350 con 0x7f9164144e90 2024-04-20T21:19:17.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.375+0000 7f916aa00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f916406ded0 con 0x7f916406f140 2024-04-20T21:19:17.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.375+0000 7f916aa00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f916415f4d0 con 0x7f91640af0b0 2024-04-20T21:19:17.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.376+0000 7f9163400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f91640af0b0 0x7f916415dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:17.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.376+0000 7f9168a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9164144e90 0x7f9164147ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:17.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.376+0000 7f9163e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f916406f140 0x7f91640cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:17.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.376+0000 7f9163400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f91640af0b0 0x7f916415dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:37396/0 (socket says 172.21.15.86:37396) 2024-04-20T21:19:17.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.376+0000 7f9163400640 1 -- 172.21.15.86:0/3974180080 learned_addr learned my addr 172.21.15.86:0/3974180080 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:17.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.376+0000 7f9168a00640 1 -- 172.21.15.86:0/3974180080 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f91640af0b0 msgr2=0x7f916415dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:17.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.376+0000 7f9168a00640 1 --2- 172.21.15.86:0/3974180080 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f91640af0b0 0x7f916415dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:17.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.376+0000 7f9168a00640 1 -- 172.21.15.86:0/3974180080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f916406f140 msgr2=0x7f91640cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:17.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.376+0000 7f9168a00640 1 --2- 172.21.15.86:0/3974180080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f916406f140 0x7f91640cfaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:17.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.376+0000 7f9168a00640 1 -- 172.21.15.86:0/3974180080 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f916415e210 con 0x7f9164144e90 2024-04-20T21:19:17.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.377+0000 7f9168a00640 1 --2- 172.21.15.86:0/3974180080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9164144e90 0x7f9164147ed0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f915c00c390 tx=0x7f915c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:17.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.377+0000 7f915be00640 1 -- 172.21.15.86:0/3974180080 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915c017070 con 0x7f9164144e90 2024-04-20T21:19:17.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.377+0000 7f915be00640 1 -- 172.21.15.86:0/3974180080 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f915c0052d0 con 0x7f9164144e90 2024-04-20T21:19:17.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.377+0000 7f915be00640 1 -- 172.21.15.86:0/3974180080 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915c003830 con 0x7f9164144e90 2024-04-20T21:19:17.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.377+0000 7f916aa00640 1 -- 172.21.15.86:0/3974180080 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9164148410 con 0x7f9164144e90 2024-04-20T21:19:17.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.377+0000 7f916aa00640 1 -- 172.21.15.86:0/3974180080 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91640cece0 con 0x7f9164144e90 2024-04-20T21:19:17.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.378+0000 7f916aa00640 1 -- 172.21.15.86:0/3974180080 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91640689e0 con 0x7f9164144e90 2024-04-20T21:19:17.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.379+0000 7f915be00640 1 -- 172.21.15.86:0/3974180080 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f915c01a070 con 0x7f9164144e90 2024-04-20T21:19:17.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.380+0000 7f915be00640 1 --2- 172.21.15.86:0/3974180080 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f914406c8f0 0x7f914406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:17.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.380+0000 7f915be00640 1 -- 172.21.15.86:0/3974180080 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f915c08ad10 con 0x7f9164144e90 2024-04-20T21:19:17.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.380+0000 7f9163e00640 1 --2- 172.21.15.86:0/3974180080 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f914406c8f0 0x7f914406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:17.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.381+0000 7f9163e00640 1 --2- 172.21.15.86:0/3974180080 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f914406c8f0 0x7f914406edb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9154009bd0 tx=0x7f915400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:17.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.385+0000 7f915be00640 1 -- 172.21.15.86:0/3974180080 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f915c090020 con 0x7f9164144e90 2024-04-20T21:19:17.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.634+0000 7f916aa00640 1 -- 172.21.15.86:0/3974180080 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f916413fb20 con 0x7f9164144e90 2024-04-20T21:19:17.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.634+0000 7f915be00640 1 -- 172.21.15.86:0/3974180080 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+73 (secure 0 0 0) 0x7f915c058ba0 con 0x7f9164144e90 2024-04-20T21:19:17.634 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T21:19:17.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.637+0000 7f916aa00640 1 -- 172.21.15.86:0/3974180080 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f914406c8f0 msgr2=0x7f914406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:17.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.637+0000 7f916aa00640 1 --2- 172.21.15.86:0/3974180080 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f914406c8f0 0x7f914406edb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9154009bd0 tx=0x7f915400b040 comp rx=0 tx=0).stop 2024-04-20T21:19:17.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.637+0000 7f916aa00640 1 -- 172.21.15.86:0/3974180080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9164144e90 msgr2=0x7f9164147ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:17.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.637+0000 7f916aa00640 1 --2- 172.21.15.86:0/3974180080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9164144e90 0x7f9164147ed0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f915c00c390 tx=0x7f915c00c850 comp rx=0 tx=0).stop 2024-04-20T21:19:17.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.637+0000 7f916aa00640 1 -- 172.21.15.86:0/3974180080 shutdown_connections 2024-04-20T21:19:17.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.637+0000 7f916aa00640 1 --2- 172.21.15.86:0/3974180080 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f914406c8f0 0x7f914406edb0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:17.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.637+0000 7f916aa00640 1 --2- 172.21.15.86:0/3974180080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9164144e90 0x7f9164147ed0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:17.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.637+0000 7f916aa00640 1 --2- 172.21.15.86:0/3974180080 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f91640af0b0 0x7f916415dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:17.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.637+0000 7f916aa00640 1 --2- 172.21.15.86:0/3974180080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f916406f140 0x7f91640cfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:17.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.637+0000 7f916aa00640 1 -- 172.21.15.86:0/3974180080 >> 172.21.15.86:0/3974180080 conn(0x7f91640b2360 msgr2=0x7f916406f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:17.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.637+0000 7f916aa00640 1 -- 172.21.15.86:0/3974180080 shutdown_connections 2024-04-20T21:19:17.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17.638+0000 7f916aa00640 1 -- 172.21.15.86:0/3974180080 wait complete. 2024-04-20T21:19:17.655 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T21:19:17.656 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T21:19:17.656 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-04-20T21:19:17.656 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-20T21:19:17.656 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T21:19:17.656 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T21:19:17.656 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T21:19:17.656 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T21:19:17.656 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:19:17.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:19:17.656 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-04-20T21:19:17.704 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T21:19:17.704 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T21:19:17.704 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T21:19:17.704 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:19:17.704 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-20T21:19:17.704 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:19:17.704 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:19:17.704 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:19:17.704 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-04-20T21:19:17.704 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:19:17.704 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:19:17.705 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-04-20T21:19:17.705 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:19:17.705 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:19:17.705 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T21:19:17.705 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:19:17.705 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T21:19:17.705 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42598sec preferred_lft 42598sec 2024-04-20T21:19:17.705 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-04-20T21:19:17.705 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:19:17.705 INFO:teuthology.orchestra.run.smithi086.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T21:19:17.705 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 86:25:97:2c:9c:bb brd ff:ff:ff:ff:ff:ff 2024-04-20T21:19:17.705 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-20T21:19:17.705 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:19:17.705 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::a4fb:30ff:fe90:2d6a/64 scope link 2024-04-20T21:19:17.705 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:19:17.706 INFO:teuthology.orchestra.run.smithi086.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T21:19:17.706 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 86:25:97:2c:9c:bb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:19:17.706 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::8425:97ff:fe2c:9cbb/64 scope link 2024-04-20T21:19:17.706 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:19:17.706 INFO:teuthology.orchestra.run.smithi086.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T21:19:17.706 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c2:9c:8b:55:ca:84 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:19:17.706 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c09c:8bff:fe55:ca84/64 scope link 2024-04-20T21:19:17.706 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:19:17.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:19:17.707 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T21:19:17.819 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-20T21:19:17.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:19:17.823 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-04-20T21:19:17.880 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-04-20T21:19:17.880 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T21:19:17.880 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T21:19:17.880 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T21:19:17.880 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T21:19:17.881 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:19:17.881 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-20T21:19:17.881 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T21:19:17.881 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T21:19:17.881 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T21:19:17.881 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-20T21:19:17.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:19:18.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:19:18.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:19:18.087 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-20T21:19:18.133 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-20T21:19:18.133 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-20T21:19:18.133 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:19:18.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:19:18.134 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-04-20T21:19:18.264 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-20T21:19:18.265 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T21:19:18.265 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-20T21:19:18.315 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:19:18.315 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T21:19:18.316 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-20T21:19:18.366 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-04-20T21:19:18.447 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T21:19:18.502 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-04-20T21:19:18.502 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-04-20T21:19:18.502 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-04-20T21:19:18.502 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-04-20T21:19:18.503 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-04-20T21:19:18.503 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-04-20T21:19:18.503 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-04-20T21:19:18.503 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-04-20T21:19:18.503 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-04-20T21:19:18.503 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-04-20T21:19:18.503 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-04-20T21:19:18.503 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/selinux 2024-04-20T21:19:18.503 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T21:19:18.503 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-04-20T21:19:18.503 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-04-20T21:19:18.504 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-04-20T21:19:18.504 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-04-20T21:19:18.504 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-04-20T21:19:18.504 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-04-20T21:19:18.504 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T21:19:18.504 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T21:19:18.504 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T21:19:18.504 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T21:19:18.504 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-04-20T21:19:18.504 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-04-20T21:19:18.504 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T21:19:18.504 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T21:19:18.504 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T21:19:18.504 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T21:19:18.505 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-04-20T21:19:18.505 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:19:18.505 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:19:18.505 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:19:18.505 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:19:18.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:19:18.506 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-04-20T21:19:18.556 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T21:19:18.556 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-20T21:19:18.562 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-04-20T21:19:18.630 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T21:19:18.669 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-04-20T21:19:18.670 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-04-20T21:19:18.670 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-04-20T21:19:18.670 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-04-20T21:19:18.670 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-04-20T21:19:18.670 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-04-20T21:19:18.670 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-04-20T21:19:18.670 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-04-20T21:19:18.670 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-04-20T21:19:18.670 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-04-20T21:19:18.670 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-04-20T21:19:18.670 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/selinux 2024-04-20T21:19:18.671 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T21:19:18.671 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-04-20T21:19:18.671 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-04-20T21:19:18.671 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-04-20T21:19:18.671 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-04-20T21:19:18.671 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-04-20T21:19:18.671 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-04-20T21:19:18.671 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T21:19:18.671 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T21:19:18.671 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T21:19:18.671 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T21:19:18.671 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-04-20T21:19:18.671 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-04-20T21:19:18.671 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T21:19:18.672 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T21:19:18.672 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T21:19:18.672 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T21:19:18.672 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-04-20T21:19:18.672 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:19:18.672 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:19:18.672 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:19:18.672 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:19:18.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:19:18.673 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-04-20T21:19:18.729 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:2024-04-20T21:19:18.731+0000 7fa4dc51b480 -1 init, newargv = 0x562fc0530300 newargc=15 2024-04-20T21:19:18.730 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[36892]: starting ceph client 2024-04-20T21:19:18.737 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[36892]: starting fuse 2024-04-20T21:19:18.757 INFO:teuthology.orchestra.run.smithi086.stdout:41 2024-04-20T21:19:18.758 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-04-20T21:19:18.758 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer -- python3 -c ' 2024-04-20T21:19:18.758 DEBUG:teuthology.orchestra.run.smithi086:> import glob 2024-04-20T21:19:18.758 DEBUG:teuthology.orchestra.run.smithi086:> import re 2024-04-20T21:19:18.758 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:19:18.758 DEBUG:teuthology.orchestra.run.smithi086:> import subprocess 2024-04-20T21:19:18.759 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:19:18.759 DEBUG:teuthology.orchestra.run.smithi086:> def _find_admin_socket(client_name): 2024-04-20T21:19:18.759 DEBUG:teuthology.orchestra.run.smithi086:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-20T21:19:18.759 DEBUG:teuthology.orchestra.run.smithi086:> files = glob.glob(asok_path) 2024-04-20T21:19:18.759 DEBUG:teuthology.orchestra.run.smithi086:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-20T21:19:18.759 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:19:18.759 DEBUG:teuthology.orchestra.run.smithi086:> # Given a non-glob path, it better be there 2024-04-20T21:19:18.759 DEBUG:teuthology.orchestra.run.smithi086:> if "*" not in asok_path: 2024-04-20T21:19:18.760 DEBUG:teuthology.orchestra.run.smithi086:> assert(len(files) == 1) 2024-04-20T21:19:18.760 DEBUG:teuthology.orchestra.run.smithi086:> return files[0] 2024-04-20T21:19:18.760 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:19:18.760 DEBUG:teuthology.orchestra.run.smithi086:> for f in files: 2024-04-20T21:19:18.760 DEBUG:teuthology.orchestra.run.smithi086:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T21:19:18.760 DEBUG:teuthology.orchestra.run.smithi086:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T21:19:18.760 DEBUG:teuthology.orchestra.run.smithi086:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T21:19:18.760 DEBUG:teuthology.orchestra.run.smithi086:> contents = proc_f.read() 2024-04-20T21:19:18.760 DEBUG:teuthology.orchestra.run.smithi086:> if mountpoint in contents: 2024-04-20T21:19:18.760 DEBUG:teuthology.orchestra.run.smithi086:> return f 2024-04-20T21:19:18.761 DEBUG:teuthology.orchestra.run.smithi086:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T21:19:18.761 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:19:18.761 DEBUG:teuthology.orchestra.run.smithi086:> print(_find_admin_socket("client.0")) 2024-04-20T21:19:18.761 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:19:18.947 INFO:teuthology.orchestra.run.smithi086.stdout:/var/run/ceph/ceph-client.0.36892.asok 2024-04-20T21:19:18.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:19:18.964 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.36892.asok 2024-04-20T21:19:18.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:19:18.964 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.36892.asok status 2024-04-20T21:19:19.192 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:19:19.192 INFO:teuthology.orchestra.run.smithi086.stdout: "metadata": { 2024-04-20T21:19:19.192 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T21:19:19.192 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T21:19:19.192 INFO:teuthology.orchestra.run.smithi086.stdout: "entity_id": "0", 2024-04-20T21:19:19.192 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": "smithi086", 2024-04-20T21:19:19.192 INFO:teuthology.orchestra.run.smithi086.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-20T21:19:19.192 INFO:teuthology.orchestra.run.smithi086.stdout: "pid": "36892", 2024-04-20T21:19:19.192 INFO:teuthology.orchestra.run.smithi086.stdout: "root": "/" 2024-04-20T21:19:19.192 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:19:19.192 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_count": 0, 2024-04-20T21:19:19.193 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_pinned_count": 0, 2024-04-20T21:19:19.193 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 5409, 2024-04-20T21:19:19.193 INFO:teuthology.orchestra.run.smithi086.stdout: "inst": { 2024-04-20T21:19:19.193 INFO:teuthology.orchestra.run.smithi086.stdout: "name": { 2024-04-20T21:19:19.193 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "client", 2024-04-20T21:19:19.193 INFO:teuthology.orchestra.run.smithi086.stdout: "num": 5409 2024-04-20T21:19:19.193 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:19:19.193 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2024-04-20T21:19:19.193 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-04-20T21:19:19.193 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2024-04-20T21:19:19.193 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 1727745977 2024-04-20T21:19:19.193 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:19:19.193 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:19:19.193 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2024-04-20T21:19:19.194 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-04-20T21:19:19.194 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2024-04-20T21:19:19.194 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 1727745977 2024-04-20T21:19:19.194 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:19:19.194 INFO:teuthology.orchestra.run.smithi086.stdout: "inst_str": "client.5409 192.168.0.1:0/1727745977", 2024-04-20T21:19:19.194 INFO:teuthology.orchestra.run.smithi086.stdout: "addr_str": "192.168.0.1:0/1727745977", 2024-04-20T21:19:19.194 INFO:teuthology.orchestra.run.smithi086.stdout: "inode_count": 1, 2024-04-20T21:19:19.194 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_epoch": 16, 2024-04-20T21:19:19.194 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch": 33, 2024-04-20T21:19:19.194 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch_barrier": 0, 2024-04-20T21:19:19.194 INFO:teuthology.orchestra.run.smithi086.stdout: "blocklisted": false, 2024-04-20T21:19:19.194 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs" 2024-04-20T21:19:19.194 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:19:19.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:19:19.206 DEBUG:teuthology.orchestra.run.smithi086:> stat --file-system '--printf=%T 2024-04-20T21:19:19.206 DEBUG:teuthology.orchestra.run.smithi086:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-20T21:19:19.254 INFO:teuthology.orchestra.run.smithi086.stdout:fuseblk 2024-04-20T21:19:19.255 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-20T21:19:19.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:19:19.255 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-20T21:19:19.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:19:19.382 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir) 2024-04-20T21:19:19.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:19:19.442 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir/childdir) 2024-04-20T21:19:19.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:19:19.494 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch rfile) 2024-04-20T21:19:19.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:19:19.549 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/pfile) 2024-04-20T21:19:19.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:19:19.611 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/childdir/cfile) 2024-04-20T21:19:19.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:19:19.667 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-20T21:19:19.737 INFO:teuthology.orchestra.run.smithi086.stdout:./ 2024-04-20T21:19:19.737 INFO:teuthology.orchestra.run.smithi086.stdout:./rfile 2024-04-20T21:19:19.737 INFO:teuthology.orchestra.run.smithi086.stdout:./parentdir 2024-04-20T21:19:19.737 INFO:teuthology.orchestra.run.smithi086.stdout:./parentdir/pfile 2024-04-20T21:19:19.737 INFO:teuthology.orchestra.run.smithi086.stdout:./parentdir/childdir 2024-04-20T21:19:19.737 INFO:teuthology.orchestra.run.smithi086.stdout:./parentdir/childdir/cfile 2024-04-20T21:19:19.739 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-04-20T21:19:19.739 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:19:19.739 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-04-20T21:19:19.739 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:19:19.739 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-20T21:19:19.739 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:19:19.868 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-20T21:19:19.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:19:19.881 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-04-20T21:19:19.881 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:19:19.881 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-04-20T21:19:19.881 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:19:19.881 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./rfile").st_ino) 2024-04-20T21:19:19.882 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:19:20.063 INFO:teuthology.orchestra.run.smithi086.stdout:1099511627778 2024-04-20T21:19:20.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:19:20.077 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-04-20T21:19:20.077 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:19:20.077 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-04-20T21:19:20.077 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:19:20.078 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir").st_ino) 2024-04-20T21:19:20.078 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:19:20.261 INFO:teuthology.orchestra.run.smithi086.stdout:1099511627776 2024-04-20T21:19:20.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:19:20.274 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-04-20T21:19:20.274 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:19:20.275 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-04-20T21:19:20.275 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:19:20.275 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/pfile").st_ino) 2024-04-20T21:19:20.275 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:19:20.464 INFO:teuthology.orchestra.run.smithi086.stdout:1099511628279 2024-04-20T21:19:20.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:19:20.477 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-04-20T21:19:20.477 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:19:20.477 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-04-20T21:19:20.477 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:19:20.477 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir").st_ino) 2024-04-20T21:19:20.477 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:19:20.660 INFO:teuthology.orchestra.run.smithi086.stdout:1099511627777 2024-04-20T21:19:20.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:19:20.674 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-04-20T21:19:20.674 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:19:20.674 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-04-20T21:19:20.674 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:19:20.674 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir/cfile").st_ino) 2024-04-20T21:19:20.674 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:19:20.857 INFO:teuthology.orchestra.run.smithi086.stdout:1099511628280 2024-04-20T21:19:20.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:19:20.870 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:19:20.870 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:19:20.969 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:19:21.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.171+0000 7f5197200640 1 Processor -- start 2024-04-20T21:19:21.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.171+0000 7f5197200640 1 -- start start 2024-04-20T21:19:21.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.173+0000 7f5197200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5198162180 0x7f51980cfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:21.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.173+0000 7f5197200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f51980b34b0 0x7f51980d0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:21.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.173+0000 7f5197200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f519806ea70 con 0x7f51980d52c0 2024-04-20T21:19:21.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.173+0000 7f5197200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f519806dc50 con 0x7f5198162180 2024-04-20T21:19:21.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.173+0000 7f5197200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f51980d1c80 con 0x7f51980b34b0 2024-04-20T21:19:21.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.173+0000 7f5195400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f51980d52c0 0x7f51980d2bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:46258/0 (socket says 172.21.15.86:46258) 2024-04-20T21:19:21.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.173+0000 7f5195400640 1 -- 172.21.15.86:0/959855440 learned_addr learned my addr 172.21.15.86:0/959855440 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:19:21.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.173+0000 7f5196800640 1 --2- 172.21.15.86:0/959855440 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f51980b34b0 0x7f51980d0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:21.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.173+0000 7f5194a00640 1 -- 172.21.15.86:0/959855440 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1352573022 0 0) 0x7f519806ea70 con 0x7f51980d52c0 2024-04-20T21:19:21.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.173+0000 7f5195e00640 1 --2- 172.21.15.86:0/959855440 >> v2:172.21.15.144:3300/0 conn(0x7f5198162180 0x7f51980cfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:21.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.174+0000 7f5194a00640 1 -- 172.21.15.86:0/959855440 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f517c003520 con 0x7f51980d52c0 2024-04-20T21:19:21.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.174+0000 7f5194a00640 1 -- 172.21.15.86:0/959855440 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2254623771 0 0) 0x7f517c003520 con 0x7f51980d52c0 2024-04-20T21:19:21.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.174+0000 7f5194a00640 1 -- 172.21.15.86:0/959855440 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f51980b34b0 msgr2=0x7f51980d0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:21.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.174+0000 7f5194a00640 1 --2- 172.21.15.86:0/959855440 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f51980b34b0 0x7f51980d0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:21.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.174+0000 7f5194a00640 1 -- 172.21.15.86:0/959855440 >> v2:172.21.15.144:3300/0 conn(0x7f5198162180 msgr2=0x7f51980cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:21.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.174+0000 7f5194a00640 1 --2- 172.21.15.86:0/959855440 >> v2:172.21.15.144:3300/0 conn(0x7f5198162180 0x7f51980cfa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:21.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.174+0000 7f5194a00640 1 -- 172.21.15.86:0/959855440 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f519806e4b0 con 0x7f51980d52c0 2024-04-20T21:19:21.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.174+0000 7f5194a00640 1 -- 172.21.15.86:0/959855440 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f5184001670 con 0x7f51980d52c0 2024-04-20T21:19:21.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.174+0000 7f5194a00640 1 -- 172.21.15.86:0/959855440 >> v1:172.21.15.86:6789/0 conn(0x7f51980d52c0 legacy=0x7f51980d2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:21.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.174+0000 7f5194a00640 1 --2- 172.21.15.86:0/959855440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f517c0039b0 0x7f517c003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:21.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.174+0000 7f5194a00640 1 --2- 172.21.15.86:0/959855440 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f517c004800 0x7f517c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:21.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.174+0000 7f5194a00640 1 --2- 172.21.15.86:0/959855440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f517c0051c0 0x7f517c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:21.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.174+0000 7f5194a00640 1 -- 172.21.15.86:0/959855440 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f519806dc50 con 0x7f517c0051c0 2024-04-20T21:19:21.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.174+0000 7f5194a00640 1 -- 172.21.15.86:0/959855440 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f51980d1c80 con 0x7f517c0039b0 2024-04-20T21:19:21.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.174+0000 7f5194a00640 1 -- 172.21.15.86:0/959855440 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f517c00c060 con 0x7f517c004800 2024-04-20T21:19:21.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.174+0000 7f5196800640 1 --2- 172.21.15.86:0/959855440 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f51980b34b0 0x7f51980d0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:19:21.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.174+0000 7f5196800640 1 --2- 172.21.15.86:0/959855440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f517c0051c0 0x7f517c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:21.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.175+0000 7f5195400640 1 --2- 172.21.15.86:0/959855440 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f517c004800 0x7f517c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:21.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.175+0000 7f5196800640 1 -- 172.21.15.86:0/959855440 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f517c004800 msgr2=0x7f517c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:21.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.175+0000 7f5196800640 1 --2- 172.21.15.86:0/959855440 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f517c004800 0x7f517c004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:21.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.175+0000 7f5196800640 1 -- 172.21.15.86:0/959855440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f517c0039b0 msgr2=0x7f517c003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:21.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.175+0000 7f5196800640 1 --2- 172.21.15.86:0/959855440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f517c0039b0 0x7f517c003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:21.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.175+0000 7f5196800640 1 -- 172.21.15.86:0/959855440 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f517c003520 con 0x7f517c0051c0 2024-04-20T21:19:21.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.176+0000 7f5196800640 1 --2- 172.21.15.86:0/959855440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f517c0051c0 0x7f517c00b9c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f51980b3460 tx=0x7f517c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f52849957b509352 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:21.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.176+0000 7f5194a00640 1 -- 172.21.15.86:0/959855440 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5190032070 con 0x7f517c0051c0 2024-04-20T21:19:21.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.176+0000 7f5194a00640 1 -- 172.21.15.86:0/959855440 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5190030070 con 0x7f517c0051c0 2024-04-20T21:19:21.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.176+0000 7f5194a00640 1 -- 172.21.15.86:0/959855440 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5190009b50 con 0x7f517c0051c0 2024-04-20T21:19:21.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.177+0000 7f5197200640 1 -- 172.21.15.86:0/959855440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f517c0051c0 msgr2=0x7f517c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:21.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.177+0000 7f5197200640 1 --2- 172.21.15.86:0/959855440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f517c0051c0 0x7f517c00b9c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f51980b3460 tx=0x7f517c005830 comp rx=0 tx=0).stop 2024-04-20T21:19:21.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.177+0000 7f5196800640 1 -- 172.21.15.86:0/959855440 reap_dead start 2024-04-20T21:19:21.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.177+0000 7f5197200640 1 -- 172.21.15.86:0/959855440 shutdown_connections 2024-04-20T21:19:21.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.177+0000 7f5197200640 1 -- 172.21.15.86:0/959855440 >> 172.21.15.86:0/959855440 conn(0x7f519808bf70 msgr2=0x7f51980d45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:21.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.177+0000 7f5197200640 1 -- 172.21.15.86:0/959855440 shutdown_connections 2024-04-20T21:19:21.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.177+0000 7f5197200640 1 -- 172.21.15.86:0/959855440 wait complete. 2024-04-20T21:19:21.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.179+0000 7f5197200640 1 Processor -- start 2024-04-20T21:19:21.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.179+0000 7f5197200640 1 -- start start 2024-04-20T21:19:21.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.180+0000 7f5197200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f517c0051c0 0x7f5198152c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:21.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.180+0000 7f5197200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f51980cdf10 0x7f51980c8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:21.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.180+0000 7f5197200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51980ce370 0x7f51980b96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:21.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.180+0000 7f5197200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f519806e260 con 0x7f517c0051c0 2024-04-20T21:19:21.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.180+0000 7f5197200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f519806e0e0 con 0x7f51980ce370 2024-04-20T21:19:21.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.180+0000 7f5197200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f51980cff40 con 0x7f51980cdf10 2024-04-20T21:19:21.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.180+0000 7f5195e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f517c0051c0 0x7f5198152c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:21.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.180+0000 7f5195e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f517c0051c0 0x7f5198152c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39796/0 (socket says 172.21.15.86:39796) 2024-04-20T21:19:21.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.180+0000 7f5195e00640 1 -- 172.21.15.86:0/957352859 learned_addr learned my addr 172.21.15.86:0/957352859 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:21.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.180+0000 7f5195e00640 1 -- 172.21.15.86:0/957352859 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f51980cdf10 msgr2=0x7f51980c8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:21.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.180+0000 7f5195e00640 1 --2- 172.21.15.86:0/957352859 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f51980cdf10 0x7f51980c8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:21.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.180+0000 7f5195e00640 1 -- 172.21.15.86:0/957352859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51980ce370 msgr2=0x7f51980b96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:21.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.180+0000 7f5195e00640 1 --2- 172.21.15.86:0/957352859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51980ce370 0x7f51980b96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:21.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.180+0000 7f5195e00640 1 -- 172.21.15.86:0/957352859 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51980b8640 con 0x7f517c0051c0 2024-04-20T21:19:21.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.181+0000 7f5195e00640 1 --2- 172.21.15.86:0/957352859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f517c0051c0 0x7f5198152c50 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f518800e7c0 tx=0x7f518800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:21.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.181+0000 7f518e000640 1 -- 172.21.15.86:0/957352859 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5188017070 con 0x7f517c0051c0 2024-04-20T21:19:21.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.181+0000 7f518e000640 1 -- 172.21.15.86:0/957352859 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f518800c8c0 con 0x7f517c0051c0 2024-04-20T21:19:21.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.181+0000 7f518e000640 1 -- 172.21.15.86:0/957352859 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518800ccb0 con 0x7f517c0051c0 2024-04-20T21:19:21.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.181+0000 7f5197200640 1 -- 172.21.15.86:0/957352859 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51980b8920 con 0x7f517c0051c0 2024-04-20T21:19:21.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.181+0000 7f5197200640 1 -- 172.21.15.86:0/957352859 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51980b8ee0 con 0x7f517c0051c0 2024-04-20T21:19:21.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.182+0000 7f5197200640 1 -- 172.21.15.86:0/957352859 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f519804fcf0 con 0x7f517c0051c0 2024-04-20T21:19:21.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.183+0000 7f518e000640 1 -- 172.21.15.86:0/957352859 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f518801a070 con 0x7f517c0051c0 2024-04-20T21:19:21.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.184+0000 7f518e000640 1 --2- 172.21.15.86:0/957352859 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f517806c8f0 0x7f517806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:21.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.184+0000 7f518e000640 1 -- 172.21.15.86:0/957352859 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f518808bde0 con 0x7f517c0051c0 2024-04-20T21:19:21.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.184+0000 7f5195400640 1 --2- 172.21.15.86:0/957352859 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f517806c8f0 0x7f517806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:21.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.185+0000 7f5195400640 1 --2- 172.21.15.86:0/957352859 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f517806c8f0 0x7f517806edb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5184000c00 tx=0x7f5184009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:21.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.189+0000 7f518e000640 1 -- 172.21.15.86:0/957352859 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f518805a170 con 0x7f517c0051c0 2024-04-20T21:19:21.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.388+0000 7f5197200640 1 -- 172.21.15.86:0/957352859 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f51980d4fc0 con 0x7f517c0051c0 2024-04-20T21:19:21.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.391+0000 7f518e000640 1 -- 172.21.15.86:0/957352859 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16669 (secure 0 0 0) 0x7f5188059a70 con 0x7f517c0051c0 2024-04-20T21:19:21.391 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:19:21.391 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":33,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:19:11.559613+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:19:07.732422+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-20T21:19:08.960268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:19:21.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.393+0000 7f5197200640 1 -- 172.21.15.86:0/957352859 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f517806c8f0 msgr2=0x7f517806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:21.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.393+0000 7f5197200640 1 --2- 172.21.15.86:0/957352859 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f517806c8f0 0x7f517806edb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5184000c00 tx=0x7f5184009040 comp rx=0 tx=0).stop 2024-04-20T21:19:21.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.393+0000 7f5197200640 1 -- 172.21.15.86:0/957352859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f517c0051c0 msgr2=0x7f5198152c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:21.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.393+0000 7f5197200640 1 --2- 172.21.15.86:0/957352859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f517c0051c0 0x7f5198152c50 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f518800e7c0 tx=0x7f518800ec80 comp rx=0 tx=0).stop 2024-04-20T21:19:21.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.394+0000 7f5197200640 1 -- 172.21.15.86:0/957352859 shutdown_connections 2024-04-20T21:19:21.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.394+0000 7f5197200640 1 --2- 172.21.15.86:0/957352859 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f517806c8f0 0x7f517806edb0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:21.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.394+0000 7f5197200640 1 --2- 172.21.15.86:0/957352859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51980ce370 0x7f51980b96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:21.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.394+0000 7f5197200640 1 --2- 172.21.15.86:0/957352859 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f51980cdf10 0x7f51980c8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:21.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.394+0000 7f5197200640 1 --2- 172.21.15.86:0/957352859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f517c0051c0 0x7f5198152c50 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:21.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.394+0000 7f5197200640 1 -- 172.21.15.86:0/957352859 >> 172.21.15.86:0/957352859 conn(0x7f519808bf70 msgr2=0x7f51980d0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:21.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.394+0000 7f5197200640 1 -- 172.21.15.86:0/957352859 shutdown_connections 2024-04-20T21:19:21.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.394+0000 7f5197200640 1 -- 172.21.15.86:0/957352859 wait complete. 2024-04-20T21:19:21.413 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:19:07.732422+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:19:08.960268+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:19:21.413 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:19:21.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.620+0000 7f3505400640 1 Processor -- start 2024-04-20T21:19:21.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.620+0000 7f3505400640 1 -- start start 2024-04-20T21:19:21.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.622+0000 7f3505400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f350015ef50 0x7f350015f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:21.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.622+0000 7f3505400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f35000ae060 0x7f35000ae4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:21.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.622+0000 7f3505400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35000729f0 con 0x7f350015b4b0 2024-04-20T21:19:21.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.622+0000 7f3505400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f350015fe10 con 0x7f35000ae060 2024-04-20T21:19:21.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.622+0000 7f3505400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f35000ccb00 con 0x7f350015ef50 2024-04-20T21:19:21.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.622+0000 7f34ffe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f350015b4b0 0x7f35000cc3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:46262/0 (socket says 172.21.15.86:46262) 2024-04-20T21:19:21.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.622+0000 7f34ffe00640 1 -- 172.21.15.86:0/3128268172 learned_addr learned my addr 172.21.15.86:0/3128268172 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:19:21.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.623+0000 7f3504a00640 1 --2- 172.21.15.86:0/3128268172 >> v2:172.21.15.144:3300/0 conn(0x7f35000ae060 0x7f35000ae4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:21.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.623+0000 7f34f7400640 1 --2- 172.21.15.86:0/3128268172 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f350015ef50 0x7f350015f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:21.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.623+0000 7f34ff400640 1 -- 172.21.15.86:0/3128268172 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2273389848 0 0) 0x7f35000729f0 con 0x7f350015b4b0 2024-04-20T21:19:21.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.623+0000 7f34ff400640 1 -- 172.21.15.86:0/3128268172 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f34e4003520 con 0x7f350015b4b0 2024-04-20T21:19:21.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.623+0000 7f34ff400640 1 -- 172.21.15.86:0/3128268172 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4147372596 0 0) 0x7f34e4003520 con 0x7f350015b4b0 2024-04-20T21:19:21.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.623+0000 7f34ff400640 1 -- 172.21.15.86:0/3128268172 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f350015ef50 msgr2=0x7f350015f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:21.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.623+0000 7f34ff400640 1 --2- 172.21.15.86:0/3128268172 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f350015ef50 0x7f350015f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:21.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.623+0000 7f34ff400640 1 -- 172.21.15.86:0/3128268172 >> v2:172.21.15.144:3300/0 conn(0x7f35000ae060 msgr2=0x7f35000ae4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:21.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.623+0000 7f34ff400640 1 --2- 172.21.15.86:0/3128268172 >> v2:172.21.15.144:3300/0 conn(0x7f35000ae060 0x7f35000ae4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:21.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.623+0000 7f34ff400640 1 -- 172.21.15.86:0/3128268172 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3500072cd0 con 0x7f350015b4b0 2024-04-20T21:19:21.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.623+0000 7f34ff400640 1 -- 172.21.15.86:0/3128268172 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f34f0001670 con 0x7f350015b4b0 2024-04-20T21:19:21.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.623+0000 7f34ff400640 1 -- 172.21.15.86:0/3128268172 >> v1:172.21.15.86:6789/0 conn(0x7f350015b4b0 legacy=0x7f35000cc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:21.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.623+0000 7f34ff400640 1 --2- 172.21.15.86:0/3128268172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34e40039b0 0x7f34e4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:21.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.623+0000 7f34ff400640 1 --2- 172.21.15.86:0/3128268172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34e4004800 0x7f34e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:21.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.623+0000 7f34ff400640 1 --2- 172.21.15.86:0/3128268172 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f350015ef50 0x7f34e40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:21.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.623+0000 7f34ff400640 1 -- 172.21.15.86:0/3128268172 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f350015fe10 con 0x7f34e40039b0 2024-04-20T21:19:21.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.623+0000 7f34ff400640 1 -- 172.21.15.86:0/3128268172 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f35000ccb00 con 0x7f34e4004800 2024-04-20T21:19:21.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.623+0000 7f34ff400640 1 -- 172.21.15.86:0/3128268172 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f34e400b9f0 con 0x7f350015ef50 2024-04-20T21:19:21.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.624+0000 7f34ffe00640 1 --2- 172.21.15.86:0/3128268172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34e40039b0 0x7f34e4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:21.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.624+0000 7f3504a00640 1 --2- 172.21.15.86:0/3128268172 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f350015ef50 0x7f34e40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:21.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.624+0000 7f34ffe00640 1 -- 172.21.15.86:0/3128268172 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f350015ef50 msgr2=0x7f34e40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:21.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.624+0000 7f34ffe00640 1 --2- 172.21.15.86:0/3128268172 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f350015ef50 0x7f34e40051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:21.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.624+0000 7f34ffe00640 1 -- 172.21.15.86:0/3128268172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34e4004800 msgr2=0x7f34e4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:21.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.624+0000 7f34ffe00640 1 --2- 172.21.15.86:0/3128268172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34e4004800 0x7f34e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:21.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.624+0000 7f34ffe00640 1 -- 172.21.15.86:0/3128268172 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34e4003520 con 0x7f34e40039b0 2024-04-20T21:19:21.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.625+0000 7f34ffe00640 1 --2- 172.21.15.86:0/3128268172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34e40039b0 0x7f34e4003dd0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f34f000c290 tx=0x7f34e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b8a944ffff585ae server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:21.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.625+0000 7f34ff400640 1 -- 172.21.15.86:0/3128268172 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f0007d50 con 0x7f34e40039b0 2024-04-20T21:19:21.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.625+0000 7f34ff400640 1 -- 172.21.15.86:0/3128268172 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34f0030070 con 0x7f34e40039b0 2024-04-20T21:19:21.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.625+0000 7f34ff400640 1 -- 172.21.15.86:0/3128268172 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f002cd80 con 0x7f34e40039b0 2024-04-20T21:19:21.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.626+0000 7f3505400640 1 -- 172.21.15.86:0/3128268172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34e40039b0 msgr2=0x7f34e4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:21.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.626+0000 7f3505400640 1 --2- 172.21.15.86:0/3128268172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34e40039b0 0x7f34e4003dd0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f34f000c290 tx=0x7f34e4005830 comp rx=0 tx=0).stop 2024-04-20T21:19:21.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.626+0000 7f3504a00640 1 -- 172.21.15.86:0/3128268172 reap_dead start 2024-04-20T21:19:21.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.626+0000 7f3505400640 1 -- 172.21.15.86:0/3128268172 shutdown_connections 2024-04-20T21:19:21.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.626+0000 7f3505400640 1 -- 172.21.15.86:0/3128268172 >> 172.21.15.86:0/3128268172 conn(0x7f35000b1f00 msgr2=0x7f35000b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:21.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.626+0000 7f3505400640 1 -- 172.21.15.86:0/3128268172 shutdown_connections 2024-04-20T21:19:21.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.626+0000 7f3505400640 1 -- 172.21.15.86:0/3128268172 wait complete. 2024-04-20T21:19:21.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.628+0000 7f3505400640 1 Processor -- start 2024-04-20T21:19:21.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.628+0000 7f3505400640 1 -- start start 2024-04-20T21:19:21.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.629+0000 7f3505400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34e40039b0 0x7f350015b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:21.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.629+0000 7f3505400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35001e7980 0x7f35000af150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:21.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.629+0000 7f3505400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f350014adf0 0x7f350014b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:21.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.629+0000 7f3505400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3500073570 con 0x7f34e40039b0 2024-04-20T21:19:21.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.629+0000 7f3505400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3500073290 con 0x7f35001e7980 2024-04-20T21:19:21.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.629+0000 7f3505400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3500072650 con 0x7f350014adf0 2024-04-20T21:19:21.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.629+0000 7f34ffe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34e40039b0 0x7f350015b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:21.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.629+0000 7f34ffe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34e40039b0 0x7f350015b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39808/0 (socket says 172.21.15.86:39808) 2024-04-20T21:19:21.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.629+0000 7f34ffe00640 1 -- 172.21.15.86:0/1609993533 learned_addr learned my addr 172.21.15.86:0/1609993533 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:21.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.630+0000 7f34f7400640 1 --2- 172.21.15.86:0/1609993533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35001e7980 0x7f35000af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:21.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.630+0000 7f3504a00640 1 --2- 172.21.15.86:0/1609993533 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f350014adf0 0x7f350014b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:21.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.630+0000 7f34ffe00640 1 -- 172.21.15.86:0/1609993533 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f350014adf0 msgr2=0x7f350014b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:21.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.630+0000 7f34ffe00640 1 --2- 172.21.15.86:0/1609993533 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f350014adf0 0x7f350014b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:21.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.630+0000 7f34ffe00640 1 -- 172.21.15.86:0/1609993533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35001e7980 msgr2=0x7f35000af150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:21.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.630+0000 7f34ffe00640 1 --2- 172.21.15.86:0/1609993533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35001e7980 0x7f35000af150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:21.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.630+0000 7f34ffe00640 1 -- 172.21.15.86:0/1609993533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f350015bf90 con 0x7f34e40039b0 2024-04-20T21:19:21.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.630+0000 7f34ffe00640 1 --2- 172.21.15.86:0/1609993533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34e40039b0 0x7f350015b890 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f34f0001970 tx=0x7f3500072bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:21.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.630+0000 7f34fcc00640 1 -- 172.21.15.86:0/1609993533 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f0001c80 con 0x7f34e40039b0 2024-04-20T21:19:21.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.630+0000 7f34fcc00640 1 -- 172.21.15.86:0/1609993533 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34f0004bf0 con 0x7f34e40039b0 2024-04-20T21:19:21.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.630+0000 7f34fcc00640 1 -- 172.21.15.86:0/1609993533 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f0007070 con 0x7f34e40039b0 2024-04-20T21:19:21.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.630+0000 7f3505400640 1 -- 172.21.15.86:0/1609993533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f350015c270 con 0x7f34e40039b0 2024-04-20T21:19:21.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.631+0000 7f3505400640 1 -- 172.21.15.86:0/1609993533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f350015c830 con 0x7f34e40039b0 2024-04-20T21:19:21.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.632+0000 7f3505400640 1 -- 172.21.15.86:0/1609993533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f350004fd90 con 0x7f34e40039b0 2024-04-20T21:19:21.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.633+0000 7f34fcc00640 1 -- 172.21.15.86:0/1609993533 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f34f0007210 con 0x7f34e40039b0 2024-04-20T21:19:21.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.633+0000 7f34fcc00640 1 --2- 172.21.15.86:0/1609993533 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f34dc06c8f0 0x7f34dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:21.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.634+0000 7f34fcc00640 1 -- 172.21.15.86:0/1609993533 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f34f00acbf0 con 0x7f34e40039b0 2024-04-20T21:19:21.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.634+0000 7f34f7400640 1 --2- 172.21.15.86:0/1609993533 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f34dc06c8f0 0x7f34dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:21.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.634+0000 7f34f7400640 1 --2- 172.21.15.86:0/1609993533 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f34dc06c8f0 0x7f34dc06edb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f34e8006a90 tx=0x7f34e8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:21.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.638+0000 7f34fcc00640 1 -- 172.21.15.86:0/1609993533 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f34f007b0b0 con 0x7f34e40039b0 2024-04-20T21:19:21.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.841+0000 7f3505400640 1 -- 172.21.15.86:0/1609993533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f350015fbf0 con 0x7f34e40039b0 2024-04-20T21:19:21.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.841+0000 7f34fcc00640 1 -- 172.21.15.86:0/1609993533 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) ==== 93+0+10 (secure 0 0 0) 0x7f34f007a9b0 con 0x7f34e40039b0 2024-04-20T21:19:21.841 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:19:21.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.844+0000 7f3505400640 1 -- 172.21.15.86:0/1609993533 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f34dc06c8f0 msgr2=0x7f34dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:21.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.844+0000 7f3505400640 1 --2- 172.21.15.86:0/1609993533 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f34dc06c8f0 0x7f34dc06edb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f34e8006a90 tx=0x7f34e8008040 comp rx=0 tx=0).stop 2024-04-20T21:19:21.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.844+0000 7f3505400640 1 -- 172.21.15.86:0/1609993533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34e40039b0 msgr2=0x7f350015b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:21.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.844+0000 7f3505400640 1 --2- 172.21.15.86:0/1609993533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34e40039b0 0x7f350015b890 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f34f0001970 tx=0x7f3500072bc0 comp rx=0 tx=0).stop 2024-04-20T21:19:21.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.844+0000 7f3505400640 1 -- 172.21.15.86:0/1609993533 shutdown_connections 2024-04-20T21:19:21.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.844+0000 7f3505400640 1 --2- 172.21.15.86:0/1609993533 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f34dc06c8f0 0x7f34dc06edb0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:21.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.844+0000 7f3505400640 1 --2- 172.21.15.86:0/1609993533 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f350014adf0 0x7f350014b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:21.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.844+0000 7f3505400640 1 --2- 172.21.15.86:0/1609993533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35001e7980 0x7f35000af150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:21.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.844+0000 7f3505400640 1 --2- 172.21.15.86:0/1609993533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34e40039b0 0x7f350015b890 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:21.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.844+0000 7f3505400640 1 -- 172.21.15.86:0/1609993533 >> 172.21.15.86:0/1609993533 conn(0x7f35000b1f00 msgr2=0x7f35000b1750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:21.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.844+0000 7f3505400640 1 -- 172.21.15.86:0/1609993533 shutdown_connections 2024-04-20T21:19:21.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:21.844+0000 7f3505400640 1 -- 172.21.15.86:0/1609993533 wait complete. 2024-04-20T21:19:21.863 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:19:22.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.065+0000 7fda0fe00640 1 Processor -- start 2024-04-20T21:19:22.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.066+0000 7fda0fe00640 1 -- start start 2024-04-20T21:19:22.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.067+0000 7fda0fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fda1015e8e0 0x7fda100b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:22.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.067+0000 7fda0fe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda100d4c50 0x7fda100d24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:22.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.067+0000 7fda0fe00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda100b4ab0 con 0x7fda100d1aa0 2024-04-20T21:19:22.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.067+0000 7fda0fe00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fda100b4d90 con 0x7fda1015e8e0 2024-04-20T21:19:22.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.067+0000 7fda0fe00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fda100b5070 con 0x7fda100d4c50 2024-04-20T21:19:22.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.067+0000 7fda0ea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fda1015e8e0 0x7fda100b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:22.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.068+0000 7fda0e000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fda100d1aa0 0x7fda100d4540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:46264/0 (socket says 172.21.15.86:46264) 2024-04-20T21:19:22.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.068+0000 7fda0e000640 1 -- 172.21.15.86:0/51884007 learned_addr learned my addr 172.21.15.86:0/51884007 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:19:22.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.068+0000 7fda0ea00640 1 -- 172.21.15.86:0/51884007 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda100d4c50 msgr2=0x7fda100d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:22.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.068+0000 7fda0f400640 1 --2- 172.21.15.86:0/51884007 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda100d4c50 0x7fda100d24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:22.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.068+0000 7fda0ea00640 1 --2- 172.21.15.86:0/51884007 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda100d4c50 0x7fda100d24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:22.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.068+0000 7fda0ea00640 1 -- 172.21.15.86:0/51884007 >> v1:172.21.15.86:6789/0 conn(0x7fda100d1aa0 legacy=0x7fda100d4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:22.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.068+0000 7fda0d600640 1 -- 172.21.15.86:0/51884007 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1405851388 0 0) 0x7fda100b4ab0 con 0x7fda100d1aa0 2024-04-20T21:19:22.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.068+0000 7fda0ea00640 1 -- 172.21.15.86:0/51884007 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda100b5350 con 0x7fda1015e8e0 2024-04-20T21:19:22.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.068+0000 7fda0f400640 1 --2- 172.21.15.86:0/51884007 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda100d4c50 0x7fda100d24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:22.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.070+0000 7fda0ea00640 1 --2- 172.21.15.86:0/51884007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda1015e8e0 0x7fda100b44d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fda000097a0 tx=0x7fda0002b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cebf54ddabbf4516 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:22.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.070+0000 7fda0d600640 1 -- 172.21.15.86:0/51884007 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda00031780 con 0x7fda1015e8e0 2024-04-20T21:19:22.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.070+0000 7fda0d600640 1 -- 172.21.15.86:0/51884007 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda00037070 con 0x7fda1015e8e0 2024-04-20T21:19:22.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.070+0000 7fda0d600640 1 -- 172.21.15.86:0/51884007 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda00031b40 con 0x7fda1015e8e0 2024-04-20T21:19:22.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.070+0000 7fda0fe00640 1 -- 172.21.15.86:0/51884007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda1015e8e0 msgr2=0x7fda100b44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:22.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.070+0000 7fda0fe00640 1 --2- 172.21.15.86:0/51884007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda1015e8e0 0x7fda100b44d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fda000097a0 tx=0x7fda0002b820 comp rx=0 tx=0).stop 2024-04-20T21:19:22.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.071+0000 7fda0fe00640 1 -- 172.21.15.86:0/51884007 shutdown_connections 2024-04-20T21:19:22.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.071+0000 7fda0fe00640 1 --2- 172.21.15.86:0/51884007 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda100d4c50 0x7fda100d24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:22.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.071+0000 7fda0fe00640 1 --2- 172.21.15.86:0/51884007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda1015e8e0 0x7fda100b44d0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:22.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.071+0000 7fda0fe00640 1 -- 172.21.15.86:0/51884007 >> 172.21.15.86:0/51884007 conn(0x7fda1009d9e0 msgr2=0x7fda1008bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:22.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.071+0000 7fda0fe00640 1 -- 172.21.15.86:0/51884007 shutdown_connections 2024-04-20T21:19:22.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.071+0000 7fda0fe00640 1 -- 172.21.15.86:0/51884007 wait complete. 2024-04-20T21:19:22.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.072+0000 7fda0fe00640 1 Processor -- start 2024-04-20T21:19:22.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.073+0000 7fda0fe00640 1 -- start start 2024-04-20T21:19:22.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.073+0000 7fda0fe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda100d1aa0 0x7fda100cad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:22.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.073+0000 7fda0fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda100d4c50 0x7fda100cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:22.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.073+0000 7fda0fe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda1015e8e0 0x7fda10158d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:22.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.074+0000 7fda0fe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fda1015ecc0 con 0x7fda100d4c50 2024-04-20T21:19:22.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.074+0000 7fda0fe00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fda100d1f30 con 0x7fda100d1aa0 2024-04-20T21:19:22.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.074+0000 7fda0fe00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fda1015ee40 con 0x7fda1015e8e0 2024-04-20T21:19:22.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.074+0000 7fda0e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda100d4c50 0x7fda100cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:22.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.074+0000 7fda0e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda100d4c50 0x7fda100cb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39820/0 (socket says 172.21.15.86:39820) 2024-04-20T21:19:22.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.074+0000 7fda0e000640 1 -- 172.21.15.86:0/1886908468 learned_addr learned my addr 172.21.15.86:0/1886908468 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:22.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.074+0000 7fda0ea00640 1 --2- 172.21.15.86:0/1886908468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda100d1aa0 0x7fda100cad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:22.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.074+0000 7fda0f400640 1 --2- 172.21.15.86:0/1886908468 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda1015e8e0 0x7fda10158d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:22.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.074+0000 7fda0e000640 1 -- 172.21.15.86:0/1886908468 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda1015e8e0 msgr2=0x7fda10158d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:22.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.074+0000 7fda0e000640 1 --2- 172.21.15.86:0/1886908468 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda1015e8e0 0x7fda10158d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:22.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.074+0000 7fda0e000640 1 -- 172.21.15.86:0/1886908468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda100d1aa0 msgr2=0x7fda100cad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:22.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.074+0000 7fda0e000640 1 --2- 172.21.15.86:0/1886908468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda100d1aa0 0x7fda100cad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:22.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.074+0000 7fda0e000640 1 -- 172.21.15.86:0/1886908468 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda100cc230 con 0x7fda100d4c50 2024-04-20T21:19:22.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.074+0000 7fda0ea00640 1 --2- 172.21.15.86:0/1886908468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda100d1aa0 0x7fda100cad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:22.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.075+0000 7fda0e000640 1 --2- 172.21.15.86:0/1886908468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda100d4c50 0x7fda100cb9e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd9f800c890 tx=0x7fd9f800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:22.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.075+0000 7fda06a00640 1 -- 172.21.15.86:0/1886908468 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8018070 con 0x7fda100d4c50 2024-04-20T21:19:22.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.075+0000 7fda06a00640 1 -- 172.21.15.86:0/1886908468 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9f800a640 con 0x7fda100d4c50 2024-04-20T21:19:22.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.075+0000 7fda06a00640 1 -- 172.21.15.86:0/1886908468 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f800aa30 con 0x7fda100d4c50 2024-04-20T21:19:22.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.075+0000 7fda0fe00640 1 -- 172.21.15.86:0/1886908468 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda100cc510 con 0x7fda100d4c50 2024-04-20T21:19:22.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.075+0000 7fda0fe00640 1 -- 172.21.15.86:0/1886908468 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda100ccdb0 con 0x7fda100d4c50 2024-04-20T21:19:22.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.076+0000 7fda0fe00640 1 -- 172.21.15.86:0/1886908468 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda100cc6e0 con 0x7fda100d4c50 2024-04-20T21:19:22.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.077+0000 7fda06a00640 1 -- 172.21.15.86:0/1886908468 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd9f800abd0 con 0x7fda100d4c50 2024-04-20T21:19:22.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.078+0000 7fda06a00640 1 --2- 172.21.15.86:0/1886908468 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd9ec06c8f0 0x7fd9ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:22.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.078+0000 7fda06a00640 1 -- 172.21.15.86:0/1886908468 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fd9f808b1d0 con 0x7fda100d4c50 2024-04-20T21:19:22.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.078+0000 7fda0ea00640 1 --2- 172.21.15.86:0/1886908468 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd9ec06c8f0 0x7fd9ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:22.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.079+0000 7fda0ea00640 1 --2- 172.21.15.86:0/1886908468 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd9ec06c8f0 0x7fd9ec06edb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fda00036000 tx=0x7fda00008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:22.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.083+0000 7fda06a00640 1 -- 172.21.15.86:0/1886908468 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd9f80595f0 con 0x7fda100d4c50 2024-04-20T21:19:22.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.284+0000 7fda0fe00640 1 -- 172.21.15.86:0/1886908468 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fda1004fd90 con 0x7fda100d4c50 2024-04-20T21:19:22.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.285+0000 7fda06a00640 1 -- 172.21.15.86:0/1886908468 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) ==== 104+0+11 (secure 0 0 0) 0x7fd9f8058ef0 con 0x7fda100d4c50 2024-04-20T21:19:22.284 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-20T21:19:22.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.287+0000 7fda0fe00640 1 -- 172.21.15.86:0/1886908468 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd9ec06c8f0 msgr2=0x7fd9ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:22.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.287+0000 7fda0fe00640 1 --2- 172.21.15.86:0/1886908468 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd9ec06c8f0 0x7fd9ec06edb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fda00036000 tx=0x7fda00008030 comp rx=0 tx=0).stop 2024-04-20T21:19:22.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.287+0000 7fda0fe00640 1 -- 172.21.15.86:0/1886908468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda100d4c50 msgr2=0x7fda100cb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:22.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.287+0000 7fda0fe00640 1 --2- 172.21.15.86:0/1886908468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda100d4c50 0x7fda100cb9e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd9f800c890 tx=0x7fd9f800cd50 comp rx=0 tx=0).stop 2024-04-20T21:19:22.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.288+0000 7fda0fe00640 1 -- 172.21.15.86:0/1886908468 shutdown_connections 2024-04-20T21:19:22.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.288+0000 7fda0fe00640 1 --2- 172.21.15.86:0/1886908468 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd9ec06c8f0 0x7fd9ec06edb0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:22.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.288+0000 7fda0fe00640 1 --2- 172.21.15.86:0/1886908468 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda1015e8e0 0x7fda10158d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:22.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.288+0000 7fda0fe00640 1 --2- 172.21.15.86:0/1886908468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda100d4c50 0x7fda100cb9e0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:22.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.288+0000 7fda0fe00640 1 --2- 172.21.15.86:0/1886908468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda100d1aa0 0x7fda100cad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:22.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.288+0000 7fda0fe00640 1 -- 172.21.15.86:0/1886908468 >> 172.21.15.86:0/1886908468 conn(0x7fda1009d9e0 msgr2=0x7fda100b6a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:22.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.288+0000 7fda0fe00640 1 -- 172.21.15.86:0/1886908468 shutdown_connections 2024-04-20T21:19:22.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.288+0000 7fda0fe00640 1 -- 172.21.15.86:0/1886908468 wait complete. 2024-04-20T21:19:22.306 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:19:22.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.502+0000 7fc611200640 1 Processor -- start 2024-04-20T21:19:22.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.502+0000 7fc611200640 1 -- start start 2024-04-20T21:19:22.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.504+0000 7fc611200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc60c15e870 0x7fc60c06ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:22.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.504+0000 7fc611200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc60c0cfb80 0x7fc60c0d0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:22.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.504+0000 7fc611200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc60c0d2ff0 con 0x7fc60c0d4310 2024-04-20T21:19:22.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.504+0000 7fc611200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc60c0cf150 con 0x7fc60c0cfb80 2024-04-20T21:19:22.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.504+0000 7fc611200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc60c0d35b0 con 0x7fc60c15e870 2024-04-20T21:19:22.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.504+0000 7fc60be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc60c0cfb80 0x7fc60c0d0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:22.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.505+0000 7fc60be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc60c0cfb80 0x7fc60c0d0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39836/0 (socket says 172.21.15.86:39836) 2024-04-20T21:19:22.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.505+0000 7fc60be00640 1 -- 172.21.15.86:0/1937445312 learned_addr learned my addr 172.21.15.86:0/1937445312 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:22.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.505+0000 7fc60b400640 1 --2- 172.21.15.86:0/1937445312 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc60c15e870 0x7fc60c06ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:22.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.505+0000 7fc60be00640 1 -- 172.21.15.86:0/1937445312 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc60c15e870 msgr2=0x7fc60c06ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:22.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.505+0000 7fc60be00640 1 --2- 172.21.15.86:0/1937445312 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc60c15e870 0x7fc60c06ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:22.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.505+0000 7fc60be00640 1 -- 172.21.15.86:0/1937445312 >> v1:172.21.15.144:6789/0 conn(0x7fc60c0d4310 legacy=0x7fc60c0d4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:22.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.505+0000 7fc60be00640 1 -- 172.21.15.86:0/1937445312 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc60c0d32d0 con 0x7fc60c0cfb80 2024-04-20T21:19:22.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.507+0000 7fc60be00640 1 --2- 172.21.15.86:0/1937445312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc60c0cfb80 0x7fc60c0d0010 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fc604009870 tx=0x7fc60402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23012fe3b9d9e977 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:22.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.507+0000 7fc60a000640 1 -- 172.21.15.86:0/1937445312 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc604034070 con 0x7fc60c0cfb80 2024-04-20T21:19:22.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.507+0000 7fc60a000640 1 -- 172.21.15.86:0/1937445312 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc604030840 con 0x7fc60c0cfb80 2024-04-20T21:19:22.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.507+0000 7fc60a000640 1 -- 172.21.15.86:0/1937445312 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc604030c10 con 0x7fc60c0cfb80 2024-04-20T21:19:22.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.508+0000 7fc611200640 1 -- 172.21.15.86:0/1937445312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc60c0cfb80 msgr2=0x7fc60c0d0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:22.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.508+0000 7fc611200640 1 --2- 172.21.15.86:0/1937445312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc60c0cfb80 0x7fc60c0d0010 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fc604009870 tx=0x7fc60402b980 comp rx=0 tx=0).stop 2024-04-20T21:19:22.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.508+0000 7fc611200640 1 -- 172.21.15.86:0/1937445312 shutdown_connections 2024-04-20T21:19:22.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.508+0000 7fc611200640 1 --2- 172.21.15.86:0/1937445312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc60c0cfb80 0x7fc60c0d0010 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:22.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.508+0000 7fc611200640 1 --2- 172.21.15.86:0/1937445312 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc60c15e870 0x7fc60c06ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:22.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.508+0000 7fc611200640 1 -- 172.21.15.86:0/1937445312 >> 172.21.15.86:0/1937445312 conn(0x7fc60c09d9c0 msgr2=0x7fc60c08bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:22.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.508+0000 7fc611200640 1 -- 172.21.15.86:0/1937445312 shutdown_connections 2024-04-20T21:19:22.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.508+0000 7fc611200640 1 -- 172.21.15.86:0/1937445312 wait complete. 2024-04-20T21:19:22.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.510+0000 7fc611200640 1 Processor -- start 2024-04-20T21:19:22.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.510+0000 7fc611200640 1 -- start start 2024-04-20T21:19:22.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.511+0000 7fc611200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc60c0cfb80 0x7fc60c14b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:22.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.511+0000 7fc611200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc60c0d4310 0x7fc60c1519e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:22.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.511+0000 7fc611200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc60c15e870 0x7fc60c0ca920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:22.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.511+0000 7fc611200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc60c0d0550 con 0x7fc60c0d4310 2024-04-20T21:19:22.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.511+0000 7fc611200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc60c0cf660 con 0x7fc60c15e870 2024-04-20T21:19:22.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.511+0000 7fc611200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc60c06d3c0 con 0x7fc60c0cfb80 2024-04-20T21:19:22.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.511+0000 7fc60aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc60c0d4310 0x7fc60c1519e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:22.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.511+0000 7fc60aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc60c0d4310 0x7fc60c1519e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39840/0 (socket says 172.21.15.86:39840) 2024-04-20T21:19:22.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.511+0000 7fc60aa00640 1 -- 172.21.15.86:0/1402228345 learned_addr learned my addr 172.21.15.86:0/1402228345 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:22.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.512+0000 7fc60b400640 1 --2- 172.21.15.86:0/1402228345 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc60c0cfb80 0x7fc60c14b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:22.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.512+0000 7fc60be00640 1 --2- 172.21.15.86:0/1402228345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc60c15e870 0x7fc60c0ca920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:22.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.512+0000 7fc60aa00640 1 -- 172.21.15.86:0/1402228345 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc60c0cfb80 msgr2=0x7fc60c14b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:22.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.512+0000 7fc60aa00640 1 --2- 172.21.15.86:0/1402228345 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc60c0cfb80 0x7fc60c14b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:22.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.512+0000 7fc60aa00640 1 -- 172.21.15.86:0/1402228345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc60c15e870 msgr2=0x7fc60c0ca920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:22.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.512+0000 7fc60aa00640 1 --2- 172.21.15.86:0/1402228345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc60c15e870 0x7fc60c0ca920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:22.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.512+0000 7fc60aa00640 1 -- 172.21.15.86:0/1402228345 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc60c0ce670 con 0x7fc60c0d4310 2024-04-20T21:19:22.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.512+0000 7fc60be00640 1 --2- 172.21.15.86:0/1402228345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc60c15e870 0x7fc60c0ca920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:22.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.512+0000 7fc60aa00640 1 --2- 172.21.15.86:0/1402228345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc60c0d4310 0x7fc60c1519e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc5f800c790 tx=0x7fc5f800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:22.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.512+0000 7fc603400640 1 -- 172.21.15.86:0/1402228345 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8016020 con 0x7fc60c0d4310 2024-04-20T21:19:22.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.512+0000 7fc603400640 1 -- 172.21.15.86:0/1402228345 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5f8006d50 con 0x7fc60c0d4310 2024-04-20T21:19:22.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.512+0000 7fc603400640 1 -- 172.21.15.86:0/1402228345 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8003bb0 con 0x7fc60c0d4310 2024-04-20T21:19:22.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.512+0000 7fc611200640 1 -- 172.21.15.86:0/1402228345 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc60c0ce950 con 0x7fc60c0d4310 2024-04-20T21:19:22.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.512+0000 7fc611200640 1 -- 172.21.15.86:0/1402228345 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc60c0cbad0 con 0x7fc60c0d4310 2024-04-20T21:19:22.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.513+0000 7fc611200640 1 -- 172.21.15.86:0/1402228345 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc60c04fd90 con 0x7fc60c0d4310 2024-04-20T21:19:22.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.514+0000 7fc603400640 1 -- 172.21.15.86:0/1402228345 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc5f800a850 con 0x7fc60c0d4310 2024-04-20T21:19:22.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.516+0000 7fc603400640 1 --2- 172.21.15.86:0/1402228345 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc5ec06c8f0 0x7fc5ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:22.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.516+0000 7fc603400640 1 -- 172.21.15.86:0/1402228345 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fc5f808b810 con 0x7fc60c0d4310 2024-04-20T21:19:22.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.516+0000 7fc60b400640 1 --2- 172.21.15.86:0/1402228345 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc5ec06c8f0 0x7fc5ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:22.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.517+0000 7fc60b400640 1 --2- 172.21.15.86:0/1402228345 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc5ec06c8f0 0x7fc5ec06edb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc006fd0 tx=0x7fc5fc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:22.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.521+0000 7fc603400640 1 -- 172.21.15.86:0/1402228345 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc5f8020020 con 0x7fc60c0d4310 2024-04-20T21:19:22.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.722+0000 7fc611200640 1 -- 172.21.15.86:0/1402228345 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc60c06cc60 con 0x7fc60c0d4310 2024-04-20T21:19:22.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.723+0000 7fc603400640 1 -- 172.21.15.86:0/1402228345 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) ==== 100+0+11 (secure 0 0 0) 0x7fc5f8019310 con 0x7fc60c0d4310 2024-04-20T21:19:22.723 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-20T21:19:22.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.725+0000 7fc611200640 1 -- 172.21.15.86:0/1402228345 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc5ec06c8f0 msgr2=0x7fc5ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:22.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.725+0000 7fc611200640 1 --2- 172.21.15.86:0/1402228345 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc5ec06c8f0 0x7fc5ec06edb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc006fd0 tx=0x7fc5fc008040 comp rx=0 tx=0).stop 2024-04-20T21:19:22.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.725+0000 7fc611200640 1 -- 172.21.15.86:0/1402228345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc60c0d4310 msgr2=0x7fc60c1519e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:22.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.725+0000 7fc611200640 1 --2- 172.21.15.86:0/1402228345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc60c0d4310 0x7fc60c1519e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc5f800c790 tx=0x7fc5f800cc50 comp rx=0 tx=0).stop 2024-04-20T21:19:22.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.726+0000 7fc611200640 1 -- 172.21.15.86:0/1402228345 shutdown_connections 2024-04-20T21:19:22.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.726+0000 7fc611200640 1 --2- 172.21.15.86:0/1402228345 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc5ec06c8f0 0x7fc5ec06edb0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:22.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.726+0000 7fc611200640 1 --2- 172.21.15.86:0/1402228345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc60c15e870 0x7fc60c0ca920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:22.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.726+0000 7fc611200640 1 --2- 172.21.15.86:0/1402228345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc60c0d4310 0x7fc60c1519e0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:22.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.726+0000 7fc611200640 1 --2- 172.21.15.86:0/1402228345 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc60c0cfb80 0x7fc60c14b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:22.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.726+0000 7fc611200640 1 -- 172.21.15.86:0/1402228345 >> 172.21.15.86:0/1402228345 conn(0x7fc60c09d9c0 msgr2=0x7fc60c0b1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:22.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.726+0000 7fc611200640 1 -- 172.21.15.86:0/1402228345 shutdown_connections 2024-04-20T21:19:22.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.726+0000 7fc611200640 1 -- 172.21.15.86:0/1402228345 wait complete. 2024-04-20T21:19:22.743 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T21:19:22.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.941+0000 7fc259200640 1 Processor -- start 2024-04-20T21:19:22.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.942+0000 7fc259200640 1 -- start start 2024-04-20T21:19:22.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.943+0000 7fc259200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc25415e9b0 0x7fc25415f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:22.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.943+0000 7fc259200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc254072280 0x7fc25406e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:22.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.943+0000 7fc259200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2540b49a0 con 0x7fc25406e1b0 2024-04-20T21:19:22.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.943+0000 7fc259200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc25406dd20 con 0x7fc254072280 2024-04-20T21:19:22.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.943+0000 7fc259200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fc2540b4c40 con 0x7fc25415e9b0 2024-04-20T21:19:22.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.943+0000 7fc253e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc254072280 0x7fc25406e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:22.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.943+0000 7fc253e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc254072280 0x7fc25406e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39846/0 (socket says 172.21.15.86:39846) 2024-04-20T21:19:22.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.943+0000 7fc253e00640 1 -- 172.21.15.86:0/1540073349 learned_addr learned my addr 172.21.15.86:0/1540073349 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:22.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.944+0000 7fc253400640 1 --2- 172.21.15.86:0/1540073349 >> v2:172.21.15.144:3300/0 conn(0x7fc25415e9b0 0x7fc25415f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:22.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.944+0000 7fc253e00640 1 -- 172.21.15.86:0/1540073349 >> v2:172.21.15.144:3300/0 conn(0x7fc25415e9b0 msgr2=0x7fc25415f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:22.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.944+0000 7fc253e00640 1 --2- 172.21.15.86:0/1540073349 >> v2:172.21.15.144:3300/0 conn(0x7fc25415e9b0 0x7fc25415f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:22.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.944+0000 7fc253e00640 1 -- 172.21.15.86:0/1540073349 >> v1:172.21.15.144:6789/0 conn(0x7fc25406e1b0 legacy=0x7fc254071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:22.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.944+0000 7fc253e00640 1 -- 172.21.15.86:0/1540073349 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2540b4f20 con 0x7fc254072280 2024-04-20T21:19:22.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.944+0000 7fc253400640 1 --2- 172.21.15.86:0/1540073349 >> v2:172.21.15.144:3300/0 conn(0x7fc25415e9b0 0x7fc25415f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:22.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.945+0000 7fc253e00640 1 --2- 172.21.15.86:0/1540073349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc254072280 0x7fc25406e700 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc24c009e50 tx=0x7fc24c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cfc5873e2364319 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:22.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.945+0000 7fc252a00640 1 -- 172.21.15.86:0/1540073349 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24c02ca80 con 0x7fc254072280 2024-04-20T21:19:22.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.945+0000 7fc252a00640 1 -- 172.21.15.86:0/1540073349 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc24c02cc20 con 0x7fc254072280 2024-04-20T21:19:22.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.945+0000 7fc252a00640 1 -- 172.21.15.86:0/1540073349 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24c0343f0 con 0x7fc254072280 2024-04-20T21:19:22.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.946+0000 7fc259200640 1 -- 172.21.15.86:0/1540073349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc254072280 msgr2=0x7fc25406e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:22.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.946+0000 7fc259200640 1 --2- 172.21.15.86:0/1540073349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc254072280 0x7fc25406e700 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc24c009e50 tx=0x7fc24c02bee0 comp rx=0 tx=0).stop 2024-04-20T21:19:22.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.946+0000 7fc259200640 1 -- 172.21.15.86:0/1540073349 shutdown_connections 2024-04-20T21:19:22.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.946+0000 7fc259200640 1 --2- 172.21.15.86:0/1540073349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc254072280 0x7fc25406e700 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:22.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.946+0000 7fc259200640 1 --2- 172.21.15.86:0/1540073349 >> v2:172.21.15.144:3300/0 conn(0x7fc25415e9b0 0x7fc25415f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:22.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.946+0000 7fc259200640 1 -- 172.21.15.86:0/1540073349 >> 172.21.15.86:0/1540073349 conn(0x7fc25406bb60 msgr2=0x7fc25406bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:22.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.946+0000 7fc259200640 1 -- 172.21.15.86:0/1540073349 shutdown_connections 2024-04-20T21:19:22.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.946+0000 7fc259200640 1 -- 172.21.15.86:0/1540073349 wait complete. 2024-04-20T21:19:22.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.948+0000 7fc259200640 1 Processor -- start 2024-04-20T21:19:22.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.948+0000 7fc259200640 1 -- start start 2024-04-20T21:19:22.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.949+0000 7fc259200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc25406e1b0 0x7fc2540cd170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:22.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.949+0000 7fc259200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc254072280 0x7fc25409da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:22.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.949+0000 7fc259200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc25415e9b0 0x7fc2540ca750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:22.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.949+0000 7fc259200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc25415ed90 con 0x7fc254072280 2024-04-20T21:19:22.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.949+0000 7fc259200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc25415f400 con 0x7fc25406e1b0 2024-04-20T21:19:22.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.949+0000 7fc259200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc25415fe60 con 0x7fc25415e9b0 2024-04-20T21:19:22.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.949+0000 7fc24be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc254072280 0x7fc25409da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:22.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.949+0000 7fc24be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc254072280 0x7fc25409da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39852/0 (socket says 172.21.15.86:39852) 2024-04-20T21:19:22.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.949+0000 7fc24be00640 1 -- 172.21.15.86:0/2689207167 learned_addr learned my addr 172.21.15.86:0/2689207167 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:22.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.949+0000 7fc253e00640 1 --2- 172.21.15.86:0/2689207167 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc25415e9b0 0x7fc2540ca750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:22.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.949+0000 7fc253400640 1 --2- 172.21.15.86:0/2689207167 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc25406e1b0 0x7fc2540cd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:22.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.949+0000 7fc24be00640 1 -- 172.21.15.86:0/2689207167 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc25415e9b0 msgr2=0x7fc2540ca750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:22.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.950+0000 7fc24be00640 1 --2- 172.21.15.86:0/2689207167 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc25415e9b0 0x7fc2540ca750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:22.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.950+0000 7fc24be00640 1 -- 172.21.15.86:0/2689207167 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc25406e1b0 msgr2=0x7fc2540cd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:22.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.950+0000 7fc24be00640 1 --2- 172.21.15.86:0/2689207167 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc25406e1b0 0x7fc2540cd170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:22.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.950+0000 7fc24be00640 1 -- 172.21.15.86:0/2689207167 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2540c83e0 con 0x7fc254072280 2024-04-20T21:19:22.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.950+0000 7fc24be00640 1 --2- 172.21.15.86:0/2689207167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc254072280 0x7fc25409da90 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc23c00c460 tx=0x7fc23c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:22.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.950+0000 7fc24b400640 1 -- 172.21.15.86:0/2689207167 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c016020 con 0x7fc254072280 2024-04-20T21:19:22.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.950+0000 7fc24b400640 1 -- 172.21.15.86:0/2689207167 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc23c012870 con 0x7fc254072280 2024-04-20T21:19:22.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.950+0000 7fc24b400640 1 -- 172.21.15.86:0/2689207167 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c012c60 con 0x7fc254072280 2024-04-20T21:19:22.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.950+0000 7fc259200640 1 -- 172.21.15.86:0/2689207167 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc25409dfd0 con 0x7fc254072280 2024-04-20T21:19:22.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.950+0000 7fc259200640 1 -- 172.21.15.86:0/2689207167 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2540cb280 con 0x7fc254072280 2024-04-20T21:19:22.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.951+0000 7fc259200640 1 -- 172.21.15.86:0/2689207167 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc25404fcf0 con 0x7fc254072280 2024-04-20T21:19:22.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.952+0000 7fc24b400640 1 -- 172.21.15.86:0/2689207167 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc23c019020 con 0x7fc254072280 2024-04-20T21:19:22.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.954+0000 7fc24b400640 1 --2- 172.21.15.86:0/2689207167 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc23006c8f0 0x7fc23006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:22.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.954+0000 7fc24b400640 1 -- 172.21.15.86:0/2689207167 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fc23c08b670 con 0x7fc254072280 2024-04-20T21:19:22.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.954+0000 7fc253400640 1 --2- 172.21.15.86:0/2689207167 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc23006c8f0 0x7fc23006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:22.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.955+0000 7fc253400640 1 --2- 172.21.15.86:0/2689207167 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc23006c8f0 0x7fc23006edb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc25406c470 tx=0x7fc244008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:22.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:22.958+0000 7fc24b400640 1 -- 172.21.15.86:0/2689207167 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc23c020070 con 0x7fc254072280 2024-04-20T21:19:23.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.169+0000 7fc259200640 1 -- 172.21.15.86:0/2689207167 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc254072060 con 0x7fc254072280 2024-04-20T21:19:23.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.170+0000 7fc24b400640 1 -- 172.21.15.86:0/2689207167 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v33) ==== 80+0+0 (secure 0 0 0) 0x7fc23c059a00 con 0x7fc254072280 2024-04-20T21:19:23.170 INFO:teuthology.orchestra.run.smithi086.stderr:listed 0 entries 2024-04-20T21:19:23.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.172+0000 7fc259200640 1 -- 172.21.15.86:0/2689207167 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc23006c8f0 msgr2=0x7fc23006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:23.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.172+0000 7fc259200640 1 --2- 172.21.15.86:0/2689207167 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc23006c8f0 0x7fc23006edb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc25406c470 tx=0x7fc244008040 comp rx=0 tx=0).stop 2024-04-20T21:19:23.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.172+0000 7fc259200640 1 -- 172.21.15.86:0/2689207167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc254072280 msgr2=0x7fc25409da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:23.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.172+0000 7fc259200640 1 --2- 172.21.15.86:0/2689207167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc254072280 0x7fc25409da90 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc23c00c460 tx=0x7fc23c00c920 comp rx=0 tx=0).stop 2024-04-20T21:19:23.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.173+0000 7fc259200640 1 -- 172.21.15.86:0/2689207167 shutdown_connections 2024-04-20T21:19:23.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.173+0000 7fc259200640 1 --2- 172.21.15.86:0/2689207167 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc23006c8f0 0x7fc23006edb0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:23.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.173+0000 7fc259200640 1 --2- 172.21.15.86:0/2689207167 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc25415e9b0 0x7fc2540ca750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:23.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.173+0000 7fc259200640 1 --2- 172.21.15.86:0/2689207167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc254072280 0x7fc25409da90 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:23.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.173+0000 7fc259200640 1 --2- 172.21.15.86:0/2689207167 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc25406e1b0 0x7fc2540cd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:23.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.173+0000 7fc259200640 1 -- 172.21.15.86:0/2689207167 >> 172.21.15.86:0/2689207167 conn(0x7fc25406bb60 msgr2=0x7fc2540b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:23.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.173+0000 7fc259200640 1 -- 172.21.15.86:0/2689207167 shutdown_connections 2024-04-20T21:19:23.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.173+0000 7fc259200640 1 -- 172.21.15.86:0/2689207167 wait complete. 2024-04-20T21:19:23.190 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:19:23.190 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:19:23.237 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:19:23.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.479+0000 7fc579200640 1 Processor -- start 2024-04-20T21:19:23.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.479+0000 7fc579200640 1 -- start start 2024-04-20T21:19:23.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.481+0000 7fc579200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc574146130 0x7fc57406e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:23.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.481+0000 7fc579200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc574072410 0x7fc5740cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:23.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.481+0000 7fc579200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc57406dc30 con 0x7fc5740ccdd0 2024-04-20T21:19:23.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.481+0000 7fc579200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc5740cd8e0 con 0x7fc574146130 2024-04-20T21:19:23.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.481+0000 7fc579200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc5740cdbc0 con 0x7fc574072410 2024-04-20T21:19:23.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.481+0000 7fc571e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc574146130 0x7fc57406e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:23.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.481+0000 7fc569400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fc5740ccdd0 0x7fc574071d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:46268/0 (socket says 172.21.15.86:46268) 2024-04-20T21:19:23.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.481+0000 7fc571e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc574146130 0x7fc57406e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39868/0 (socket says 172.21.15.86:39868) 2024-04-20T21:19:23.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.481+0000 7fc569400640 1 -- 172.21.15.86:0/3547763545 learned_addr learned my addr 172.21.15.86:0/3547763545 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:19:23.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.482+0000 7fc572800640 1 --2- 172.21.15.86:0/3547763545 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc574072410 0x7fc5740cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:23.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.482+0000 7fc571e00640 1 -- 172.21.15.86:0/3547763545 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc574072410 msgr2=0x7fc5740cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:23.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.482+0000 7fc571e00640 1 --2- 172.21.15.86:0/3547763545 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc574072410 0x7fc5740cd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:23.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.482+0000 7fc571e00640 1 -- 172.21.15.86:0/3547763545 >> v1:172.21.15.86:6789/0 conn(0x7fc5740ccdd0 legacy=0x7fc574071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:23.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.482+0000 7fc571400640 1 -- 172.21.15.86:0/3547763545 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2046763673 0 0) 0x7fc57406dc30 con 0x7fc5740ccdd0 2024-04-20T21:19:23.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.482+0000 7fc571e00640 1 -- 172.21.15.86:0/3547763545 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc57406ded0 con 0x7fc574146130 2024-04-20T21:19:23.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.483+0000 7fc571e00640 1 --2- 172.21.15.86:0/3547763545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc574146130 0x7fc57406e6d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc564009d90 tx=0x7fc56402bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6695a9963d4294b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:23.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.483+0000 7fc571400640 1 -- 172.21.15.86:0/3547763545 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc564039070 con 0x7fc574146130 2024-04-20T21:19:23.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.483+0000 7fc571400640 1 -- 172.21.15.86:0/3547763545 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc56402cce0 con 0x7fc574146130 2024-04-20T21:19:23.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.483+0000 7fc571400640 1 -- 172.21.15.86:0/3547763545 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc564034660 con 0x7fc574146130 2024-04-20T21:19:23.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.484+0000 7fc579200640 1 -- 172.21.15.86:0/3547763545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc574146130 msgr2=0x7fc57406e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:23.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.484+0000 7fc579200640 1 --2- 172.21.15.86:0/3547763545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc574146130 0x7fc57406e6d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc564009d90 tx=0x7fc56402bf30 comp rx=0 tx=0).stop 2024-04-20T21:19:23.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.484+0000 7fc579200640 1 -- 172.21.15.86:0/3547763545 shutdown_connections 2024-04-20T21:19:23.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.484+0000 7fc579200640 1 --2- 172.21.15.86:0/3547763545 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc574072410 0x7fc5740cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:23.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.484+0000 7fc579200640 1 --2- 172.21.15.86:0/3547763545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc574146130 0x7fc57406e6d0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:23.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.484+0000 7fc579200640 1 -- 172.21.15.86:0/3547763545 >> 172.21.15.86:0/3547763545 conn(0x7fc5740b2360 msgr2=0x7fc5740b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:23.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.484+0000 7fc579200640 1 -- 172.21.15.86:0/3547763545 shutdown_connections 2024-04-20T21:19:23.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.484+0000 7fc579200640 1 -- 172.21.15.86:0/3547763545 wait complete. 2024-04-20T21:19:23.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.486+0000 7fc579200640 1 Processor -- start 2024-04-20T21:19:23.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.486+0000 7fc579200640 1 -- start start 2024-04-20T21:19:23.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.487+0000 7fc579200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc574072410 0x7fc57413ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:23.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.487+0000 7fc579200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5740ccdd0 0x7fc57415ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:23.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.487+0000 7fc579200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc574146130 0x7fc57415f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:23.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.487+0000 7fc579200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc57415cc20 con 0x7fc574146130 2024-04-20T21:19:23.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.487+0000 7fc579200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc57406f150 con 0x7fc5740ccdd0 2024-04-20T21:19:23.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.487+0000 7fc579200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc57415cda0 con 0x7fc574072410 2024-04-20T21:19:23.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.487+0000 7fc572800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc574146130 0x7fc57415f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:23.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.487+0000 7fc572800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc574146130 0x7fc57415f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39878/0 (socket says 172.21.15.86:39878) 2024-04-20T21:19:23.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.487+0000 7fc572800640 1 -- 172.21.15.86:0/2691088931 learned_addr learned my addr 172.21.15.86:0/2691088931 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:23.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.488+0000 7fc571e00640 1 --2- 172.21.15.86:0/2691088931 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc574072410 0x7fc57413ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:23.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.488+0000 7fc569400640 1 --2- 172.21.15.86:0/2691088931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5740ccdd0 0x7fc57415ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:23.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.488+0000 7fc572800640 1 -- 172.21.15.86:0/2691088931 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc574072410 msgr2=0x7fc57413ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:23.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.488+0000 7fc572800640 1 --2- 172.21.15.86:0/2691088931 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc574072410 0x7fc57413ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:23.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.488+0000 7fc572800640 1 -- 172.21.15.86:0/2691088931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5740ccdd0 msgr2=0x7fc57415ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:23.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.488+0000 7fc572800640 1 --2- 172.21.15.86:0/2691088931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5740ccdd0 0x7fc57415ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:23.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.488+0000 7fc572800640 1 -- 172.21.15.86:0/2691088931 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5740b3790 con 0x7fc574146130 2024-04-20T21:19:23.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.488+0000 7fc572800640 1 --2- 172.21.15.86:0/2691088931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc574146130 0x7fc57415f380 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc56c00d5a0 tx=0x7fc56c00da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:23.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.489+0000 7fc56aa00640 1 -- 172.21.15.86:0/2691088931 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56c0125f0 con 0x7fc574146130 2024-04-20T21:19:23.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.489+0000 7fc56aa00640 1 -- 172.21.15.86:0/2691088931 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc56c019070 con 0x7fc574146130 2024-04-20T21:19:23.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.489+0000 7fc56aa00640 1 -- 172.21.15.86:0/2691088931 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56c0129d0 con 0x7fc574146130 2024-04-20T21:19:23.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.489+0000 7fc579200640 1 -- 172.21.15.86:0/2691088931 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5740b3a70 con 0x7fc574146130 2024-04-20T21:19:23.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.489+0000 7fc579200640 1 -- 172.21.15.86:0/2691088931 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5740b4030 con 0x7fc574146130 2024-04-20T21:19:23.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.490+0000 7fc579200640 1 -- 172.21.15.86:0/2691088931 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5740689e0 con 0x7fc574146130 2024-04-20T21:19:23.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.491+0000 7fc56aa00640 1 -- 172.21.15.86:0/2691088931 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc56c012b70 con 0x7fc574146130 2024-04-20T21:19:23.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.492+0000 7fc56aa00640 1 --2- 172.21.15.86:0/2691088931 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc55006c8f0 0x7fc55006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:23.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.492+0000 7fc56aa00640 1 -- 172.21.15.86:0/2691088931 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fc56c08bdd0 con 0x7fc574146130 2024-04-20T21:19:23.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.492+0000 7fc571e00640 1 --2- 172.21.15.86:0/2691088931 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc55006c8f0 0x7fc55006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:23.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.493+0000 7fc571e00640 1 --2- 172.21.15.86:0/2691088931 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc55006c8f0 0x7fc55006edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc564036000 tx=0x7fc564030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:23.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.497+0000 7fc56aa00640 1 -- 172.21.15.86:0/2691088931 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc56c05a290 con 0x7fc574146130 2024-04-20T21:19:23.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.698+0000 7fc579200640 1 -- 172.21.15.86:0/2691088931 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc57406e6d0 con 0x7fc574146130 2024-04-20T21:19:23.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.700+0000 7fc56aa00640 1 -- 172.21.15.86:0/2691088931 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16669 (secure 0 0 0) 0x7fc56c08d070 con 0x7fc574146130 2024-04-20T21:19:23.701 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:19:23.701 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":33,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:19:11.559613+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:19:07.732422+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-20T21:19:08.960268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:19:23.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.704+0000 7fc579200640 1 -- 172.21.15.86:0/2691088931 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc55006c8f0 msgr2=0x7fc55006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:23.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.704+0000 7fc579200640 1 --2- 172.21.15.86:0/2691088931 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc55006c8f0 0x7fc55006edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc564036000 tx=0x7fc564030000 comp rx=0 tx=0).stop 2024-04-20T21:19:23.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.704+0000 7fc579200640 1 -- 172.21.15.86:0/2691088931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc574146130 msgr2=0x7fc57415f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:23.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.704+0000 7fc579200640 1 --2- 172.21.15.86:0/2691088931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc574146130 0x7fc57415f380 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc56c00d5a0 tx=0x7fc56c00da60 comp rx=0 tx=0).stop 2024-04-20T21:19:23.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.704+0000 7fc579200640 1 -- 172.21.15.86:0/2691088931 shutdown_connections 2024-04-20T21:19:23.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.704+0000 7fc579200640 1 --2- 172.21.15.86:0/2691088931 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc55006c8f0 0x7fc55006edb0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:23.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.704+0000 7fc579200640 1 --2- 172.21.15.86:0/2691088931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc574146130 0x7fc57415f380 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:23.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.704+0000 7fc579200640 1 --2- 172.21.15.86:0/2691088931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5740ccdd0 0x7fc57415ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:23.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.704+0000 7fc579200640 1 --2- 172.21.15.86:0/2691088931 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc574072410 0x7fc57413ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:23.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.704+0000 7fc579200640 1 -- 172.21.15.86:0/2691088931 >> 172.21.15.86:0/2691088931 conn(0x7fc5740b2360 msgr2=0x7fc5740b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:23.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.704+0000 7fc579200640 1 -- 172.21.15.86:0/2691088931 shutdown_connections 2024-04-20T21:19:23.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.705+0000 7fc579200640 1 -- 172.21.15.86:0/2691088931 wait complete. 2024-04-20T21:19:23.724 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:19:07.732422+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:19:08.960268+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:19:23.724 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:19:23.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.926+0000 7feb69400640 1 Processor -- start 2024-04-20T21:19:23.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.926+0000 7feb69400640 1 -- start start 2024-04-20T21:19:23.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.927+0000 7feb69400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7feb64155120 0x7feb640b3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:23.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.928+0000 7feb69400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb6406e960 0x7feb6415c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:23.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.928+0000 7feb69400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb6415b920 con 0x7feb6415b0d0 2024-04-20T21:19:23.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.928+0000 7feb69400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7feb6415c530 con 0x7feb64155120 2024-04-20T21:19:23.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.928+0000 7feb69400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7feb6415bc00 con 0x7feb6406e960 2024-04-20T21:19:23.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.928+0000 7feb68a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb6406e960 0x7feb6415c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:23.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.928+0000 7feb68a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb6406e960 0x7feb6415c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:37496/0 (socket says 172.21.15.86:37496) 2024-04-20T21:19:23.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.928+0000 7feb68a00640 1 -- 172.21.15.86:0/74263150 learned_addr learned my addr 172.21.15.86:0/74263150 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:23.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.929+0000 7feb63e00640 1 --2- 172.21.15.86:0/74263150 >> v2:172.21.15.144:3300/0 conn(0x7feb64155120 0x7feb640b3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:23.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.929+0000 7feb68a00640 1 -- 172.21.15.86:0/74263150 >> v2:172.21.15.144:3300/0 conn(0x7feb64155120 msgr2=0x7feb640b3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:23.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.929+0000 7feb68a00640 1 --2- 172.21.15.86:0/74263150 >> v2:172.21.15.144:3300/0 conn(0x7feb64155120 0x7feb640b3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:23.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.929+0000 7feb68a00640 1 -- 172.21.15.86:0/74263150 >> v1:172.21.15.144:6789/0 conn(0x7feb6415b0d0 legacy=0x7feb6406e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:23.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.929+0000 7feb68a00640 1 -- 172.21.15.86:0/74263150 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb6415bee0 con 0x7feb6406e960 2024-04-20T21:19:23.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.931+0000 7feb68a00640 1 --2- 172.21.15.86:0/74263150 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb6406e960 0x7feb6415c9c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7feb5c009e50 tx=0x7feb5c02bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=440721cb505d7c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:23.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.931+0000 7feb62a00640 1 -- 172.21.15.86:0/74263150 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5c02cb00 con 0x7feb6406e960 2024-04-20T21:19:23.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.931+0000 7feb62a00640 1 -- 172.21.15.86:0/74263150 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb5c02cca0 con 0x7feb6406e960 2024-04-20T21:19:23.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.931+0000 7feb62a00640 1 -- 172.21.15.86:0/74263150 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5c034610 con 0x7feb6406e960 2024-04-20T21:19:23.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.932+0000 7feb69400640 1 -- 172.21.15.86:0/74263150 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb6406e960 msgr2=0x7feb6415c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:23.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.932+0000 7feb69400640 1 --2- 172.21.15.86:0/74263150 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb6406e960 0x7feb6415c9c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7feb5c009e50 tx=0x7feb5c02bf60 comp rx=0 tx=0).stop 2024-04-20T21:19:23.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.932+0000 7feb69400640 1 -- 172.21.15.86:0/74263150 shutdown_connections 2024-04-20T21:19:23.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.932+0000 7feb69400640 1 --2- 172.21.15.86:0/74263150 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb6406e960 0x7feb6415c9c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:23.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.932+0000 7feb69400640 1 --2- 172.21.15.86:0/74263150 >> v2:172.21.15.144:3300/0 conn(0x7feb64155120 0x7feb640b3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:23.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.932+0000 7feb69400640 1 -- 172.21.15.86:0/74263150 >> 172.21.15.86:0/74263150 conn(0x7feb640b1f00 msgr2=0x7feb640b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:23.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.932+0000 7feb69400640 1 -- 172.21.15.86:0/74263150 shutdown_connections 2024-04-20T21:19:23.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.932+0000 7feb69400640 1 -- 172.21.15.86:0/74263150 wait complete. 2024-04-20T21:19:23.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.934+0000 7feb69400640 1 Processor -- start 2024-04-20T21:19:23.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.934+0000 7feb69400640 1 -- start start 2024-04-20T21:19:23.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.935+0000 7feb69400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb6406e960 0x7feb64154970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:23.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.935+0000 7feb69400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb64155120 0x7feb6409a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:23.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.935+0000 7feb69400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb6415b0d0 0x7feb640ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:23.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.935+0000 7feb69400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feb6415b6b0 con 0x7feb6406e960 2024-04-20T21:19:23.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.935+0000 7feb69400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7feb6415b530 con 0x7feb6415b0d0 2024-04-20T21:19:23.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.935+0000 7feb69400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7feb6406d9c0 con 0x7feb64155120 2024-04-20T21:19:23.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.935+0000 7feb63e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb6406e960 0x7feb64154970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:23.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.935+0000 7feb63e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb6406e960 0x7feb64154970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39886/0 (socket says 172.21.15.86:39886) 2024-04-20T21:19:23.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.935+0000 7feb63e00640 1 -- 172.21.15.86:0/3494734920 learned_addr learned my addr 172.21.15.86:0/3494734920 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:23.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.935+0000 7feb63400640 1 --2- 172.21.15.86:0/3494734920 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb64155120 0x7feb6409a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:23.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.936+0000 7feb68a00640 1 --2- 172.21.15.86:0/3494734920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb6415b0d0 0x7feb640ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:23.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.936+0000 7feb63e00640 1 -- 172.21.15.86:0/3494734920 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb64155120 msgr2=0x7feb6409a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:23.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.936+0000 7feb63e00640 1 --2- 172.21.15.86:0/3494734920 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb64155120 0x7feb6409a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:23.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.936+0000 7feb63e00640 1 -- 172.21.15.86:0/3494734920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb6415b0d0 msgr2=0x7feb640ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:23.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.936+0000 7feb63e00640 1 --2- 172.21.15.86:0/3494734920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb6415b0d0 0x7feb640ae880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:23.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.936+0000 7feb63e00640 1 -- 172.21.15.86:0/3494734920 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb64147f50 con 0x7feb6406e960 2024-04-20T21:19:23.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.936+0000 7feb63e00640 1 --2- 172.21.15.86:0/3494734920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb6406e960 0x7feb64154970 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7feb4c00e9b0 tx=0x7feb4c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:23.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.936+0000 7feb5be00640 1 -- 172.21.15.86:0/3494734920 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4c017070 con 0x7feb6406e960 2024-04-20T21:19:23.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.936+0000 7feb5be00640 1 -- 172.21.15.86:0/3494734920 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb4c00c9a0 con 0x7feb6406e960 2024-04-20T21:19:23.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.936+0000 7feb69400640 1 -- 172.21.15.86:0/3494734920 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb6409acf0 con 0x7feb6406e960 2024-04-20T21:19:23.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.936+0000 7feb5be00640 1 -- 172.21.15.86:0/3494734920 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4c00cd90 con 0x7feb6406e960 2024-04-20T21:19:23.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.936+0000 7feb69400640 1 -- 172.21.15.86:0/3494734920 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb6409b2b0 con 0x7feb6406e960 2024-04-20T21:19:23.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.938+0000 7feb69400640 1 -- 172.21.15.86:0/3494734920 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb6404fd90 con 0x7feb6406e960 2024-04-20T21:19:23.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.939+0000 7feb5be00640 1 -- 172.21.15.86:0/3494734920 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feb4c01a070 con 0x7feb6406e960 2024-04-20T21:19:23.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.940+0000 7feb5be00640 1 --2- 172.21.15.86:0/3494734920 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7feb4006c8f0 0x7feb4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:23.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.940+0000 7feb5be00640 1 -- 172.21.15.86:0/3494734920 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7feb4c08be90 con 0x7feb6406e960 2024-04-20T21:19:23.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.940+0000 7feb63400640 1 --2- 172.21.15.86:0/3494734920 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7feb4006c8f0 0x7feb4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:23.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.941+0000 7feb63400640 1 --2- 172.21.15.86:0/3494734920 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7feb4006c8f0 0x7feb4006edb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7feb640b1eb0 tx=0x7feb54009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:23.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:23.945+0000 7feb5be00640 1 -- 172.21.15.86:0/3494734920 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7feb64147f50 con 0x7feb6406e960 2024-04-20T21:19:24.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.146+0000 7feb69400640 1 -- 172.21.15.86:0/3494734920 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7feb640afc00 con 0x7feb6406e960 2024-04-20T21:19:24.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.147+0000 7feb5be00640 1 -- 172.21.15.86:0/3494734920 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) ==== 93+0+10 (secure 0 0 0) 0x7feb4c05a350 con 0x7feb6406e960 2024-04-20T21:19:24.148 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:19:24.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.150+0000 7feb69400640 1 -- 172.21.15.86:0/3494734920 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7feb4006c8f0 msgr2=0x7feb4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:24.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.150+0000 7feb69400640 1 --2- 172.21.15.86:0/3494734920 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7feb4006c8f0 0x7feb4006edb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7feb640b1eb0 tx=0x7feb54009040 comp rx=0 tx=0).stop 2024-04-20T21:19:24.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.150+0000 7feb69400640 1 -- 172.21.15.86:0/3494734920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb6406e960 msgr2=0x7feb64154970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:24.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.150+0000 7feb69400640 1 --2- 172.21.15.86:0/3494734920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb6406e960 0x7feb64154970 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7feb4c00e9b0 tx=0x7feb4c00ee70 comp rx=0 tx=0).stop 2024-04-20T21:19:24.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.151+0000 7feb69400640 1 -- 172.21.15.86:0/3494734920 shutdown_connections 2024-04-20T21:19:24.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.151+0000 7feb69400640 1 --2- 172.21.15.86:0/3494734920 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7feb4006c8f0 0x7feb4006edb0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:24.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.151+0000 7feb69400640 1 --2- 172.21.15.86:0/3494734920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb6415b0d0 0x7feb640ae880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:24.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.151+0000 7feb69400640 1 --2- 172.21.15.86:0/3494734920 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb64155120 0x7feb6409a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:24.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.151+0000 7feb69400640 1 --2- 172.21.15.86:0/3494734920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb6406e960 0x7feb64154970 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:24.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.151+0000 7feb69400640 1 -- 172.21.15.86:0/3494734920 >> 172.21.15.86:0/3494734920 conn(0x7feb640b1f00 msgr2=0x7feb6414a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:24.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.151+0000 7feb69400640 1 -- 172.21.15.86:0/3494734920 shutdown_connections 2024-04-20T21:19:24.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.151+0000 7feb69400640 1 -- 172.21.15.86:0/3494734920 wait complete. 2024-04-20T21:19:24.170 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:19:24.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.375+0000 7f18ef200640 1 Processor -- start 2024-04-20T21:19:24.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.375+0000 7f18ef200640 1 -- start start 2024-04-20T21:19:24.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.377+0000 7f18ef200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18e8072450 0x7f18e8167670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:24.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.377+0000 7f18ef200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18e8160410 con 0x7f18e8145f00 2024-04-20T21:19:24.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.377+0000 7f18ef200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18e81606f0 con 0x7f18e8145250 2024-04-20T21:19:24.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.377+0000 7f18ef200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f18e81609d0 con 0x7f18e8072450 2024-04-20T21:19:24.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.377+0000 7f18ed400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f18e8145f00 0x7f18e8071d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:46274/0 (socket says 172.21.15.86:46274) 2024-04-20T21:19:24.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.377+0000 7f18ed400640 1 -- 172.21.15.86:0/823072138 learned_addr learned my addr 172.21.15.86:0/823072138 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:19:24.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.378+0000 7f18ee800640 1 --2- 172.21.15.86:0/823072138 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18e8072450 0x7f18e8167670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:24.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.378+0000 7f18eca00640 1 -- 172.21.15.86:0/823072138 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 499906149 0 0) 0x7f18e8160410 con 0x7f18e8145f00 2024-04-20T21:19:24.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.378+0000 7f18eca00640 1 -- 172.21.15.86:0/823072138 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f18d0003520 con 0x7f18e8145f00 2024-04-20T21:19:24.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.378+0000 7f18eca00640 1 -- 172.21.15.86:0/823072138 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4128158914 0 0) 0x7f18d0003520 con 0x7f18e8145f00 2024-04-20T21:19:24.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.378+0000 7f18eca00640 1 -- 172.21.15.86:0/823072138 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18e8072450 msgr2=0x7f18e8167670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:24.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.378+0000 7f18eca00640 1 --2- 172.21.15.86:0/823072138 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18e8072450 0x7f18e8167670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:24.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.378+0000 7f18eca00640 1 -- 172.21.15.86:0/823072138 >> v1:172.21.15.144:6789/0 conn(0x7f18e8145250 legacy=0x7f18e806efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:24.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.378+0000 7f18eca00640 1 -- 172.21.15.86:0/823072138 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18e8160cb0 con 0x7f18e8145f00 2024-04-20T21:19:24.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.378+0000 7f18eca00640 1 -- 172.21.15.86:0/823072138 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f18d8001670 con 0x7f18e8145f00 2024-04-20T21:19:24.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.378+0000 7f18eca00640 1 -- 172.21.15.86:0/823072138 >> v1:172.21.15.86:6789/0 conn(0x7f18e8145f00 legacy=0x7f18e8071d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:24.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.378+0000 7f18eca00640 1 --2- 172.21.15.86:0/823072138 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18d0003a50 0x7f18d0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:24.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.378+0000 7f18eca00640 1 --2- 172.21.15.86:0/823072138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18d0004800 0x7f18d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:24.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.378+0000 7f18eca00640 1 --2- 172.21.15.86:0/823072138 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18e8072450 0x7f18d0005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:24.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.378+0000 7f18eca00640 1 -- 172.21.15.86:0/823072138 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f18e81609d0 con 0x7f18d0004800 2024-04-20T21:19:24.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.378+0000 7f18eca00640 1 -- 172.21.15.86:0/823072138 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f18d000bbb0 con 0x7f18d0003a50 2024-04-20T21:19:24.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.378+0000 7f18eca00640 1 -- 172.21.15.86:0/823072138 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f18d000bdb0 con 0x7f18e8072450 2024-04-20T21:19:24.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.379+0000 7f18ed400640 1 --2- 172.21.15.86:0/823072138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18d0004800 0x7f18d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:24.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.379+0000 7f18ede00640 1 --2- 172.21.15.86:0/823072138 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18d0003a50 0x7f18d0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:24.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.379+0000 7f18ee800640 1 --2- 172.21.15.86:0/823072138 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18e8072450 0x7f18d0005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:24.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.379+0000 7f18ed400640 1 -- 172.21.15.86:0/823072138 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18e8072450 msgr2=0x7f18d0005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:24.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.379+0000 7f18ed400640 1 --2- 172.21.15.86:0/823072138 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18e8072450 0x7f18d0005270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:24.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.379+0000 7f18ed400640 1 -- 172.21.15.86:0/823072138 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18d0003a50 msgr2=0x7f18d0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:24.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.379+0000 7f18ed400640 1 --2- 172.21.15.86:0/823072138 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18d0003a50 0x7f18d0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:24.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.379+0000 7f18ed400640 1 -- 172.21.15.86:0/823072138 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18d0003520 con 0x7f18d0004800 2024-04-20T21:19:24.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.379+0000 7f18ede00640 1 --2- 172.21.15.86:0/823072138 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18d0003a50 0x7f18d0003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:24.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.379+0000 7f18ee800640 1 --2- 172.21.15.86:0/823072138 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18e8072450 0x7f18d0005270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:24.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.380+0000 7f18ed400640 1 --2- 172.21.15.86:0/823072138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18d0004800 0x7f18d0004c80 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f18d800c210 tx=0x7f18d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a21b957af979da07 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:24.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.380+0000 7f18eca00640 1 -- 172.21.15.86:0/823072138 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d8007cf0 con 0x7f18d0004800 2024-04-20T21:19:24.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.380+0000 7f18eca00640 1 -- 172.21.15.86:0/823072138 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18d8030070 con 0x7f18d0004800 2024-04-20T21:19:24.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.380+0000 7f18eca00640 1 -- 172.21.15.86:0/823072138 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d8005d30 con 0x7f18d0004800 2024-04-20T21:19:24.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.381+0000 7f18ef200640 1 -- 172.21.15.86:0/823072138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18d0004800 msgr2=0x7f18d0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:24.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.381+0000 7f18ef200640 1 --2- 172.21.15.86:0/823072138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18d0004800 0x7f18d0004c80 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f18d800c210 tx=0x7f18d0005830 comp rx=0 tx=0).stop 2024-04-20T21:19:24.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.381+0000 7f18ee800640 1 -- 172.21.15.86:0/823072138 reap_dead start 2024-04-20T21:19:24.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.381+0000 7f18ef200640 1 -- 172.21.15.86:0/823072138 shutdown_connections 2024-04-20T21:19:24.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.381+0000 7f18ef200640 1 -- 172.21.15.86:0/823072138 >> 172.21.15.86:0/823072138 conn(0x7f18e8067cf0 msgr2=0x7f18e806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:24.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.381+0000 7f18ef200640 1 -- 172.21.15.86:0/823072138 shutdown_connections 2024-04-20T21:19:24.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.381+0000 7f18ef200640 1 -- 172.21.15.86:0/823072138 wait complete. 2024-04-20T21:19:24.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.383+0000 7f18ef200640 1 Processor -- start 2024-04-20T21:19:24.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.383+0000 7f18ef200640 1 -- start start 2024-04-20T21:19:24.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.384+0000 7f18ef200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18d0004800 0x7f18e813ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:24.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.384+0000 7f18ef200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18e8139f20 0x7f18e8153db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:24.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.384+0000 7f18ef200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18e80cc580 0x7f18e80cca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:24.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.384+0000 7f18ef200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f18e815df80 con 0x7f18e80cc580 2024-04-20T21:19:24.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.384+0000 7f18ef200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f18e815de00 con 0x7f18e8139f20 2024-04-20T21:19:24.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.384+0000 7f18ef200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f18e815e100 con 0x7f18d0004800 2024-04-20T21:19:24.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.384+0000 7f18ee800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18e80cc580 0x7f18e80cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:24.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.384+0000 7f18ee800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18e80cc580 0x7f18e80cca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39912/0 (socket says 172.21.15.86:39912) 2024-04-20T21:19:24.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.384+0000 7f18ee800640 1 -- 172.21.15.86:0/2674085818 learned_addr learned my addr 172.21.15.86:0/2674085818 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:24.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.385+0000 7f18ede00640 1 --2- 172.21.15.86:0/2674085818 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18d0004800 0x7f18e813ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:24.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.385+0000 7f18ed400640 1 --2- 172.21.15.86:0/2674085818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18e8139f20 0x7f18e8153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:24.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.385+0000 7f18ee800640 1 -- 172.21.15.86:0/2674085818 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18d0004800 msgr2=0x7f18e813ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:24.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.385+0000 7f18ee800640 1 --2- 172.21.15.86:0/2674085818 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18d0004800 0x7f18e813ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:24.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.385+0000 7f18ee800640 1 -- 172.21.15.86:0/2674085818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18e8139f20 msgr2=0x7f18e8153db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:24.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.385+0000 7f18ee800640 1 --2- 172.21.15.86:0/2674085818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18e8139f20 0x7f18e8153db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:24.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.385+0000 7f18ee800640 1 -- 172.21.15.86:0/2674085818 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18e815c310 con 0x7f18e80cc580 2024-04-20T21:19:24.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.385+0000 7f18ed400640 1 --2- 172.21.15.86:0/2674085818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18e8139f20 0x7f18e8153db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:24.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.385+0000 7f18ee800640 1 --2- 172.21.15.86:0/2674085818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18e80cc580 0x7f18e80cca00 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f18e8145eb0 tx=0x7f18e400f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:24.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.385+0000 7f18e2000640 1 -- 172.21.15.86:0/2674085818 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e401c070 con 0x7f18e80cc580 2024-04-20T21:19:24.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.385+0000 7f18e2000640 1 -- 172.21.15.86:0/2674085818 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18e4017400 con 0x7f18e80cc580 2024-04-20T21:19:24.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.385+0000 7f18e2000640 1 -- 172.21.15.86:0/2674085818 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e40177f0 con 0x7f18e80cc580 2024-04-20T21:19:24.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.385+0000 7f18ef200640 1 -- 172.21.15.86:0/2674085818 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18e81542f0 con 0x7f18e80cc580 2024-04-20T21:19:24.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.386+0000 7f18ef200640 1 -- 172.21.15.86:0/2674085818 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18e80cbaa0 con 0x7f18e80cc580 2024-04-20T21:19:24.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.387+0000 7f18ef200640 1 -- 172.21.15.86:0/2674085818 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18e804fd90 con 0x7f18e80cc580 2024-04-20T21:19:24.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.388+0000 7f18e2000640 1 -- 172.21.15.86:0/2674085818 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f18e4005d10 con 0x7f18e80cc580 2024-04-20T21:19:24.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.389+0000 7f18e2000640 1 --2- 172.21.15.86:0/2674085818 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f18cc06c8f0 0x7f18cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:24.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.389+0000 7f18e2000640 1 -- 172.21.15.86:0/2674085818 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f18e408bcf0 con 0x7f18e80cc580 2024-04-20T21:19:24.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.389+0000 7f18ede00640 1 --2- 172.21.15.86:0/2674085818 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f18cc06c8f0 0x7f18cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:24.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.390+0000 7f18ede00640 1 --2- 172.21.15.86:0/2674085818 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f18cc06c8f0 0x7f18cc06edb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f18dc000f60 tx=0x7f18dc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:24.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.394+0000 7f18e2000640 1 -- 172.21.15.86:0/2674085818 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f18e4090070 con 0x7f18e80cc580 2024-04-20T21:19:24.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.593+0000 7f18ef200640 1 -- 172.21.15.86:0/2674085818 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f18e813a300 con 0x7f18e80cc580 2024-04-20T21:19:24.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.594+0000 7f18e2000640 1 -- 172.21.15.86:0/2674085818 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) ==== 104+0+11 (secure 0 0 0) 0x7f18e4012070 con 0x7f18e80cc580 2024-04-20T21:19:24.594 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-20T21:19:24.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.597+0000 7f18ef200640 1 -- 172.21.15.86:0/2674085818 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f18cc06c8f0 msgr2=0x7f18cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:24.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.597+0000 7f18ef200640 1 --2- 172.21.15.86:0/2674085818 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f18cc06c8f0 0x7f18cc06edb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f18dc000f60 tx=0x7f18dc009040 comp rx=0 tx=0).stop 2024-04-20T21:19:24.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.597+0000 7f18ef200640 1 -- 172.21.15.86:0/2674085818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18e80cc580 msgr2=0x7f18e80cca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:24.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.597+0000 7f18ef200640 1 --2- 172.21.15.86:0/2674085818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18e80cc580 0x7f18e80cca00 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f18e8145eb0 tx=0x7f18e400f4a0 comp rx=0 tx=0).stop 2024-04-20T21:19:24.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.597+0000 7f18ef200640 1 -- 172.21.15.86:0/2674085818 shutdown_connections 2024-04-20T21:19:24.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.597+0000 7f18ef200640 1 --2- 172.21.15.86:0/2674085818 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f18cc06c8f0 0x7f18cc06edb0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:24.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.597+0000 7f18ef200640 1 --2- 172.21.15.86:0/2674085818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f18e80cc580 0x7f18e80cca00 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:24.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.597+0000 7f18ef200640 1 --2- 172.21.15.86:0/2674085818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18e8139f20 0x7f18e8153db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:24.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.597+0000 7f18ef200640 1 --2- 172.21.15.86:0/2674085818 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f18d0004800 0x7f18e813ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:24.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.597+0000 7f18ef200640 1 -- 172.21.15.86:0/2674085818 >> 172.21.15.86:0/2674085818 conn(0x7f18e8067cf0 msgr2=0x7f18e8071d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:24.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.597+0000 7f18ef200640 1 -- 172.21.15.86:0/2674085818 shutdown_connections 2024-04-20T21:19:24.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.597+0000 7f18ef200640 1 -- 172.21.15.86:0/2674085818 wait complete. 2024-04-20T21:19:24.616 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:19:24.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.821+0000 7f2f2a200640 1 Processor -- start 2024-04-20T21:19:24.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.821+0000 7f2f2a200640 1 -- start start 2024-04-20T21:19:24.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.823+0000 7f2f2a200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f2415b4b0 0x7f2f240cc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:24.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.823+0000 7f2f2a200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2f2415ef50 0x7f2f2415f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:24.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.823+0000 7f2f2a200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f240729f0 con 0x7f2f2415b890 2024-04-20T21:19:24.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.823+0000 7f2f2a200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f2f2415fe10 con 0x7f2f2415ef50 2024-04-20T21:19:24.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.823+0000 7f2f2a200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2f2415f8d0 con 0x7f2f2415b4b0 2024-04-20T21:19:24.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.823+0000 7f2f28e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f2415b4b0 0x7f2f240cc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:24.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.824+0000 7f2f29800640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f2f2415b890 0x7f2f2415bd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.86:37012/0 (socket says 172.21.15.86:37012) 2024-04-20T21:19:24.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.824+0000 7f2f1be00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2f2415ef50 0x7f2f2415f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:24.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.824+0000 7f2f29800640 1 -- 172.21.15.86:0/216440613 learned_addr learned my addr 172.21.15.86:0/216440613 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:19:24.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.824+0000 7f2f1be00640 1 -- 172.21.15.86:0/216440613 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f2415b4b0 msgr2=0x7f2f240cc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:24.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.824+0000 7f2f1be00640 1 --2- 172.21.15.86:0/216440613 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f2415b4b0 0x7f2f240cc3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:24.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.824+0000 7f2f1be00640 1 -- 172.21.15.86:0/216440613 >> v1:172.21.15.144:6789/0 conn(0x7f2f2415b890 legacy=0x7f2f2415bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:24.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.824+0000 7f2f1be00640 1 -- 172.21.15.86:0/216440613 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f24072cd0 con 0x7f2f2415ef50 2024-04-20T21:19:24.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.826+0000 7f2f1be00640 1 --2- 172.21.15.86:0/216440613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f2415ef50 0x7f2f2415f390 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2f2415fa50 tx=0x7f2f0c02dab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54a3df18d289a5a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:24.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.826+0000 7f2f23e00640 1 -- 172.21.15.86:0/216440613 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c02f870 con 0x7f2f2415ef50 2024-04-20T21:19:24.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.826+0000 7f2f23e00640 1 -- 172.21.15.86:0/216440613 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f0c02fa10 con 0x7f2f2415ef50 2024-04-20T21:19:24.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.826+0000 7f2f23e00640 1 -- 172.21.15.86:0/216440613 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c02fde0 con 0x7f2f2415ef50 2024-04-20T21:19:24.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.827+0000 7f2f2a200640 1 -- 172.21.15.86:0/216440613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f2415ef50 msgr2=0x7f2f2415f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:24.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.827+0000 7f2f2a200640 1 --2- 172.21.15.86:0/216440613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f2415ef50 0x7f2f2415f390 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2f2415fa50 tx=0x7f2f0c02dab0 comp rx=0 tx=0).stop 2024-04-20T21:19:24.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.827+0000 7f2f2a200640 1 -- 172.21.15.86:0/216440613 shutdown_connections 2024-04-20T21:19:24.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.827+0000 7f2f2a200640 1 --2- 172.21.15.86:0/216440613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f2415ef50 0x7f2f2415f390 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:24.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.827+0000 7f2f2a200640 1 --2- 172.21.15.86:0/216440613 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f2415b4b0 0x7f2f240cc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:24.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.827+0000 7f2f2a200640 1 -- 172.21.15.86:0/216440613 >> 172.21.15.86:0/216440613 conn(0x7f2f240b1f00 msgr2=0x7f2f240b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:24.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.828+0000 7f2f2a200640 1 -- 172.21.15.86:0/216440613 shutdown_connections 2024-04-20T21:19:24.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.828+0000 7f2f2a200640 1 -- 172.21.15.86:0/216440613 wait complete. 2024-04-20T21:19:24.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.829+0000 7f2f2a200640 1 Processor -- start 2024-04-20T21:19:24.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.829+0000 7f2f2a200640 1 -- start start 2024-04-20T21:19:24.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.830+0000 7f2f2a200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f2415b4b0 0x7f2f240af240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:24.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.830+0000 7f2f2a200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f2415b890 0x7f2f24152160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:24.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.830+0000 7f2f2a200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f2415ef50 0x7f2f2414d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:24.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.830+0000 7f2f2a200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2f24073570 con 0x7f2f2415b890 2024-04-20T21:19:24.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.830+0000 7f2f2a200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2f24073290 con 0x7f2f2415b4b0 2024-04-20T21:19:24.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.830+0000 7f2f2a200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2f24072650 con 0x7f2f2415ef50 2024-04-20T21:19:24.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.831+0000 7f2f1be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f2415b890 0x7f2f24152160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:24.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.831+0000 7f2f1be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f2415b890 0x7f2f24152160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39920/0 (socket says 172.21.15.86:39920) 2024-04-20T21:19:24.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.831+0000 7f2f1be00640 1 -- 172.21.15.86:0/1311564599 learned_addr learned my addr 172.21.15.86:0/1311564599 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:24.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.831+0000 7f2f28e00640 1 --2- 172.21.15.86:0/1311564599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f2415b4b0 0x7f2f240af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:24.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.831+0000 7f2f29800640 1 --2- 172.21.15.86:0/1311564599 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f2415ef50 0x7f2f2414d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:24.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.831+0000 7f2f1be00640 1 -- 172.21.15.86:0/1311564599 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f2415ef50 msgr2=0x7f2f2414d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:24.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.831+0000 7f2f1be00640 1 --2- 172.21.15.86:0/1311564599 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f2415ef50 0x7f2f2414d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:24.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.831+0000 7f2f1be00640 1 -- 172.21.15.86:0/1311564599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f2415b4b0 msgr2=0x7f2f240af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:24.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.831+0000 7f2f1be00640 1 --2- 172.21.15.86:0/1311564599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f2415b4b0 0x7f2f240af240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:24.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.831+0000 7f2f1be00640 1 -- 172.21.15.86:0/1311564599 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f24155650 con 0x7f2f2415b890 2024-04-20T21:19:24.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.831+0000 7f2f28e00640 1 --2- 172.21.15.86:0/1311564599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f2415b4b0 0x7f2f240af240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:24.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.832+0000 7f2f1be00640 1 --2- 172.21.15.86:0/1311564599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f2415b890 0x7f2f24152160 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c02e0c0 tx=0x7f2f0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:24.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.832+0000 7f2f21600640 1 -- 172.21.15.86:0/1311564599 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c047070 con 0x7f2f2415b890 2024-04-20T21:19:24.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.832+0000 7f2f21600640 1 -- 172.21.15.86:0/1311564599 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f0c030a80 con 0x7f2f2415b890 2024-04-20T21:19:24.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.832+0000 7f2f21600640 1 -- 172.21.15.86:0/1311564599 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c042410 con 0x7f2f2415b890 2024-04-20T21:19:24.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.832+0000 7f2f2a200640 1 -- 172.21.15.86:0/1311564599 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f24155930 con 0x7f2f2415b890 2024-04-20T21:19:24.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.832+0000 7f2f2a200640 1 -- 172.21.15.86:0/1311564599 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f24155ef0 con 0x7f2f2415b890 2024-04-20T21:19:24.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.833+0000 7f2f2a200640 1 -- 172.21.15.86:0/1311564599 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f2404fd90 con 0x7f2f2415b890 2024-04-20T21:19:24.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.834+0000 7f2f21600640 1 -- 172.21.15.86:0/1311564599 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2f0c039400 con 0x7f2f2415b890 2024-04-20T21:19:24.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.835+0000 7f2f21600640 1 --2- 172.21.15.86:0/1311564599 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2f0006c8f0 0x7f2f0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:24.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.835+0000 7f2f21600640 1 -- 172.21.15.86:0/1311564599 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f2f0c0ad610 con 0x7f2f2415b890 2024-04-20T21:19:24.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.835+0000 7f2f28e00640 1 --2- 172.21.15.86:0/1311564599 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2f0006c8f0 0x7f2f0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:24.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.836+0000 7f2f28e00640 1 --2- 172.21.15.86:0/1311564599 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2f0006c8f0 0x7f2f0006edb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2f14009ac0 tx=0x7f2f14008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:24.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:24.840+0000 7f2f21600640 1 -- 172.21.15.86:0/1311564599 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2f0c035030 con 0x7f2f2415b890 2024-04-20T21:19:25.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.040+0000 7f2f2a200640 1 -- 172.21.15.86:0/1311564599 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2f2415fbf0 con 0x7f2f2415b890 2024-04-20T21:19:25.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.041+0000 7f2f21600640 1 -- 172.21.15.86:0/1311564599 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) ==== 100+0+11 (secure 0 0 0) 0x7f2f0c07ba60 con 0x7f2f2415b890 2024-04-20T21:19:25.041 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-20T21:19:25.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.044+0000 7f2f2a200640 1 -- 172.21.15.86:0/1311564599 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2f0006c8f0 msgr2=0x7f2f0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:25.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.044+0000 7f2f2a200640 1 --2- 172.21.15.86:0/1311564599 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2f0006c8f0 0x7f2f0006edb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2f14009ac0 tx=0x7f2f14008040 comp rx=0 tx=0).stop 2024-04-20T21:19:25.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.044+0000 7f2f2a200640 1 -- 172.21.15.86:0/1311564599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f2415b890 msgr2=0x7f2f24152160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:25.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.044+0000 7f2f2a200640 1 --2- 172.21.15.86:0/1311564599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f2415b890 0x7f2f24152160 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c02e0c0 tx=0x7f2f0c0046c0 comp rx=0 tx=0).stop 2024-04-20T21:19:25.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.044+0000 7f2f2a200640 1 -- 172.21.15.86:0/1311564599 shutdown_connections 2024-04-20T21:19:25.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.044+0000 7f2f2a200640 1 --2- 172.21.15.86:0/1311564599 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2f0006c8f0 0x7f2f0006edb0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:25.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.044+0000 7f2f2a200640 1 --2- 172.21.15.86:0/1311564599 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2f2415ef50 0x7f2f2414d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:25.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.044+0000 7f2f2a200640 1 --2- 172.21.15.86:0/1311564599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2f2415b890 0x7f2f24152160 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:25.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.044+0000 7f2f2a200640 1 --2- 172.21.15.86:0/1311564599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f2415b4b0 0x7f2f240af240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:25.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.044+0000 7f2f2a200640 1 -- 172.21.15.86:0/1311564599 >> 172.21.15.86:0/1311564599 conn(0x7f2f240b1f00 msgr2=0x7f2f240d0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:25.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.045+0000 7f2f2a200640 1 -- 172.21.15.86:0/1311564599 shutdown_connections 2024-04-20T21:19:25.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.045+0000 7f2f2a200640 1 -- 172.21.15.86:0/1311564599 wait complete. 2024-04-20T21:19:25.062 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T21:19:25.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.253+0000 7f0f02800640 1 Processor -- start 2024-04-20T21:19:25.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.253+0000 7f0f02800640 1 -- start start 2024-04-20T21:19:25.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.255+0000 7f0f02800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0efc0cd7e0 0x7f0efc1676e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:25.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.255+0000 7f0f02800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0efc15dcc0 con 0x7f0efc0d09e0 2024-04-20T21:19:25.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.255+0000 7f0f02800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0efc15cf00 con 0x7f0efc145fe0 2024-04-20T21:19:25.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.255+0000 7f0f02800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f0efc0d00d0 con 0x7f0efc0cd7e0 2024-04-20T21:19:25.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.255+0000 7f0ef3e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f0efc0d09e0 0x7f0efc0cd0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:46284/0 (socket says 172.21.15.86:46284) 2024-04-20T21:19:25.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.255+0000 7f0ef3e00640 1 -- 172.21.15.86:0/349000599 learned_addr learned my addr 172.21.15.86:0/349000599 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:19:25.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.256+0000 7f0efaa00640 1 -- 172.21.15.86:0/349000599 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2531740717 0 0) 0x7f0efc15dcc0 con 0x7f0efc0d09e0 2024-04-20T21:19:25.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.256+0000 7f0efaa00640 1 -- 172.21.15.86:0/349000599 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ee0003520 con 0x7f0efc0d09e0 2024-04-20T21:19:25.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.256+0000 7f0efbe00640 1 --2- 172.21.15.86:0/349000599 >> v2:172.21.15.144:3300/0 conn(0x7f0efc0cd7e0 0x7f0efc1676e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:25.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.256+0000 7f0efaa00640 1 -- 172.21.15.86:0/349000599 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 398331772 0 0) 0x7f0ee0003520 con 0x7f0efc0d09e0 2024-04-20T21:19:25.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.256+0000 7f0efaa00640 1 -- 172.21.15.86:0/349000599 >> v2:172.21.15.144:3300/0 conn(0x7f0efc0cd7e0 msgr2=0x7f0efc1676e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:25.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.256+0000 7f0efaa00640 1 --2- 172.21.15.86:0/349000599 >> v2:172.21.15.144:3300/0 conn(0x7f0efc0cd7e0 0x7f0efc1676e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:25.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.256+0000 7f0efaa00640 1 -- 172.21.15.86:0/349000599 >> v1:172.21.15.144:6789/0 conn(0x7f0efc145fe0 legacy=0x7f0efc0d02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:25.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.256+0000 7f0efaa00640 1 -- 172.21.15.86:0/349000599 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0efc15d1e0 con 0x7f0efc0d09e0 2024-04-20T21:19:25.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.256+0000 7f0efaa00640 1 -- 172.21.15.86:0/349000599 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f0ee4001670 con 0x7f0efc0d09e0 2024-04-20T21:19:25.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.256+0000 7f0efaa00640 1 -- 172.21.15.86:0/349000599 >> v1:172.21.15.86:6789/0 conn(0x7f0efc0d09e0 legacy=0x7f0efc0cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:25.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.256+0000 7f0efaa00640 1 --2- 172.21.15.86:0/349000599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ee0003a00 0x7f0ee0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:25.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.256+0000 7f0efaa00640 1 --2- 172.21.15.86:0/349000599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ee0004800 0x7f0ee0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:25.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.256+0000 7f0efaa00640 1 --2- 172.21.15.86:0/349000599 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ee0005210 0x7f0ee000fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:25.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.256+0000 7f0efaa00640 1 -- 172.21.15.86:0/349000599 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0efc0d00d0 con 0x7f0ee0003a00 2024-04-20T21:19:25.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.256+0000 7f0efaa00640 1 -- 172.21.15.86:0/349000599 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee0010570 con 0x7f0ee0004800 2024-04-20T21:19:25.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.256+0000 7f0efaa00640 1 -- 172.21.15.86:0/349000599 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0ee0010770 con 0x7f0ee0005210 2024-04-20T21:19:25.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.256+0000 7f0efb400640 1 --2- 172.21.15.86:0/349000599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ee0003a00 0x7f0ee0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:25.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.257+0000 7f0efbe00640 1 --2- 172.21.15.86:0/349000599 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ee0005210 0x7f0ee000fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:25.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.257+0000 7f0ef3e00640 1 --2- 172.21.15.86:0/349000599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ee0004800 0x7f0ee0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:25.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.257+0000 7f0efb400640 1 -- 172.21.15.86:0/349000599 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ee0005210 msgr2=0x7f0ee000fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:25.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.257+0000 7f0efb400640 1 --2- 172.21.15.86:0/349000599 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ee0005210 0x7f0ee000fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:25.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.257+0000 7f0efb400640 1 -- 172.21.15.86:0/349000599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ee0004800 msgr2=0x7f0ee0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:25.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.257+0000 7f0efb400640 1 --2- 172.21.15.86:0/349000599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ee0004800 0x7f0ee0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:25.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.257+0000 7f0efbe00640 1 -- 172.21.15.86:0/349000599 reap_dead start 2024-04-20T21:19:25.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.257+0000 7f0efb400640 1 -- 172.21.15.86:0/349000599 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ee0003520 con 0x7f0ee0003a00 2024-04-20T21:19:25.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.258+0000 7f0efb400640 1 --2- 172.21.15.86:0/349000599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ee0003a00 0x7f0ee0003e20 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f0eec001750 tx=0x7f0ee0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d203d220a9bca02 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:25.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.259+0000 7f0efaa00640 1 -- 172.21.15.86:0/349000599 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eec001370 con 0x7f0ee0003a00 2024-04-20T21:19:25.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.259+0000 7f0efaa00640 1 -- 172.21.15.86:0/349000599 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0eec0052d0 con 0x7f0ee0003a00 2024-04-20T21:19:25.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.259+0000 7f0efaa00640 1 -- 172.21.15.86:0/349000599 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eec003830 con 0x7f0ee0003a00 2024-04-20T21:19:25.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.259+0000 7f0f02800640 1 -- 172.21.15.86:0/349000599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ee0003a00 msgr2=0x7f0ee0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:25.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.259+0000 7f0f02800640 1 --2- 172.21.15.86:0/349000599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ee0003a00 0x7f0ee0003e20 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f0eec001750 tx=0x7f0ee0005830 comp rx=0 tx=0).stop 2024-04-20T21:19:25.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.259+0000 7f0f02800640 1 -- 172.21.15.86:0/349000599 shutdown_connections 2024-04-20T21:19:25.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.259+0000 7f0f02800640 1 --2- 172.21.15.86:0/349000599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ee0003a00 0x7f0ee0003e20 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:25.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.259+0000 7f0f02800640 1 -- 172.21.15.86:0/349000599 >> 172.21.15.86:0/349000599 conn(0x7f0efc06b980 msgr2=0x7f0efc06bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:25.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.259+0000 7f0f02800640 1 -- 172.21.15.86:0/349000599 shutdown_connections 2024-04-20T21:19:25.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.260+0000 7f0f02800640 1 -- 172.21.15.86:0/349000599 wait complete. 2024-04-20T21:19:25.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.261+0000 7f0f02800640 1 Processor -- start 2024-04-20T21:19:25.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.261+0000 7f0f02800640 1 -- start start 2024-04-20T21:19:25.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.262+0000 7f0f02800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ee0003a00 0x7f0efc15b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:25.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.262+0000 7f0f02800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0efc15b370 0x7f0efc15e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:25.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.262+0000 7f0f02800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0efc15bdd0 0x7f0efc15e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:25.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.262+0000 7f0f02800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0efc1463c0 con 0x7f0efc15b370 2024-04-20T21:19:25.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.262+0000 7f0f02800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0efc15d7a0 con 0x7f0efc15bdd0 2024-04-20T21:19:25.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.262+0000 7f0f02800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0efc0cdce0 con 0x7f0ee0003a00 2024-04-20T21:19:25.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.262+0000 7f0ef3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0efc15b370 0x7f0efc15e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:25.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.263+0000 7f0ef3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0efc15b370 0x7f0efc15e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39944/0 (socket says 172.21.15.86:39944) 2024-04-20T21:19:25.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.263+0000 7f0ef3e00640 1 -- 172.21.15.86:0/3117405447 learned_addr learned my addr 172.21.15.86:0/3117405447 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:25.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.263+0000 7f0efb400640 1 --2- 172.21.15.86:0/3117405447 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ee0003a00 0x7f0efc15b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:25.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.263+0000 7f0ef3e00640 1 -- 172.21.15.86:0/3117405447 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ee0003a00 msgr2=0x7f0efc15b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:25.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.263+0000 7f0efbe00640 1 --2- 172.21.15.86:0/3117405447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0efc15bdd0 0x7f0efc15e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:25.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.263+0000 7f0ef3e00640 1 --2- 172.21.15.86:0/3117405447 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ee0003a00 0x7f0efc15b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:25.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.263+0000 7f0ef3e00640 1 -- 172.21.15.86:0/3117405447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0efc15bdd0 msgr2=0x7f0efc15e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:25.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.263+0000 7f0ef3e00640 1 --2- 172.21.15.86:0/3117405447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0efc15bdd0 0x7f0efc15e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:25.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.263+0000 7f0ef3e00640 1 -- 172.21.15.86:0/3117405447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0efc1405e0 con 0x7f0efc15b370 2024-04-20T21:19:25.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.263+0000 7f0efb400640 1 --2- 172.21.15.86:0/3117405447 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ee0003a00 0x7f0efc15b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:25.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.263+0000 7f0ef3e00640 1 --2- 172.21.15.86:0/3117405447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0efc15b370 0x7f0efc15e1a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0ee400e700 tx=0x7f0ee400ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:25.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.263+0000 7f0ef3400640 1 -- 172.21.15.86:0/3117405447 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee400c6c0 con 0x7f0efc15b370 2024-04-20T21:19:25.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.263+0000 7f0ef3400640 1 -- 172.21.15.86:0/3117405447 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ee4018070 con 0x7f0efc15b370 2024-04-20T21:19:25.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.264+0000 7f0f02800640 1 -- 172.21.15.86:0/3117405447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0efc140880 con 0x7f0efc15b370 2024-04-20T21:19:25.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.264+0000 7f0ef3400640 1 -- 172.21.15.86:0/3117405447 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee400cac0 con 0x7f0efc15b370 2024-04-20T21:19:25.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.264+0000 7f0f02800640 1 -- 172.21.15.86:0/3117405447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0efc140b20 con 0x7f0efc15b370 2024-04-20T21:19:25.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.265+0000 7f0f02800640 1 -- 172.21.15.86:0/3117405447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0efc0689e0 con 0x7f0efc15b370 2024-04-20T21:19:25.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.266+0000 7f0ef3400640 1 -- 172.21.15.86:0/3117405447 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0ee400cc60 con 0x7f0efc15b370 2024-04-20T21:19:25.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.266+0000 7f0ef3400640 1 --2- 172.21.15.86:0/3117405447 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0ed806c920 0x7f0ed806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:25.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.267+0000 7f0ef3400640 1 -- 172.21.15.86:0/3117405447 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f0ee408b950 con 0x7f0efc15b370 2024-04-20T21:19:25.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.267+0000 7f0efb400640 1 --2- 172.21.15.86:0/3117405447 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0ed806c920 0x7f0ed806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:25.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.268+0000 7f0efb400640 1 --2- 172.21.15.86:0/3117405447 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0ed806c920 0x7f0ed806ede0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0eec033000 tx=0x7f0eec009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:25.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.271+0000 7f0ef3400640 1 -- 172.21.15.86:0/3117405447 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0ee4059ce0 con 0x7f0efc15b370 2024-04-20T21:19:25.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.481+0000 7f0f02800640 1 -- 172.21.15.86:0/3117405447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0efc0d1020 con 0x7f0efc15b370 2024-04-20T21:19:25.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.482+0000 7f0ef3400640 1 -- 172.21.15.86:0/3117405447 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v33) ==== 80+0+0 (secure 0 0 0) 0x7f0ee40595e0 con 0x7f0efc15b370 2024-04-20T21:19:25.482 INFO:teuthology.orchestra.run.smithi086.stderr:listed 0 entries 2024-04-20T21:19:25.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.486+0000 7f0f02800640 1 -- 172.21.15.86:0/3117405447 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0ed806c920 msgr2=0x7f0ed806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:25.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.486+0000 7f0f02800640 1 --2- 172.21.15.86:0/3117405447 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0ed806c920 0x7f0ed806ede0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0eec033000 tx=0x7f0eec009030 comp rx=0 tx=0).stop 2024-04-20T21:19:25.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.486+0000 7f0f02800640 1 -- 172.21.15.86:0/3117405447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0efc15b370 msgr2=0x7f0efc15e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:25.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.486+0000 7f0f02800640 1 --2- 172.21.15.86:0/3117405447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0efc15b370 0x7f0efc15e1a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0ee400e700 tx=0x7f0ee400ebc0 comp rx=0 tx=0).stop 2024-04-20T21:19:25.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.486+0000 7f0f02800640 1 -- 172.21.15.86:0/3117405447 shutdown_connections 2024-04-20T21:19:25.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.486+0000 7f0f02800640 1 --2- 172.21.15.86:0/3117405447 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0ed806c920 0x7f0ed806ede0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:25.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.486+0000 7f0f02800640 1 --2- 172.21.15.86:0/3117405447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0efc15bdd0 0x7f0efc15e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:25.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.486+0000 7f0f02800640 1 --2- 172.21.15.86:0/3117405447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0efc15b370 0x7f0efc15e1a0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:25.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.486+0000 7f0f02800640 1 --2- 172.21.15.86:0/3117405447 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ee0003a00 0x7f0efc15b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:25.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.486+0000 7f0f02800640 1 -- 172.21.15.86:0/3117405447 >> 172.21.15.86:0/3117405447 conn(0x7f0efc06b980 msgr2=0x7f0efc0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:25.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.486+0000 7f0f02800640 1 -- 172.21.15.86:0/3117405447 shutdown_connections 2024-04-20T21:19:25.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:25.486+0000 7f0f02800640 1 -- 172.21.15.86:0/3117405447 wait complete. 2024-04-20T21:19:25.504 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi086.front.sepia.ceph.com... 2024-04-20T21:19:25.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:19:25.504 DEBUG:teuthology.orchestra.run.smithi086:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-20T21:19:25.583 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[36892]: fuse finished with error 0 and tester_r 0 2024-04-20T21:19:25.588 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:19:31.690 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-20T21:19:31.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:19:31.690 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-20T21:19:31.750 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:19:31.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.949+0000 7f151b200640 1 Processor -- start 2024-04-20T21:19:31.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.949+0000 7f151b200640 1 -- start start 2024-04-20T21:19:31.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.951+0000 7f151b200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f151406e6d0 0x7f1514071d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:31.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.951+0000 7f151b200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1514072240 0x7f151415cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:31.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.951+0000 7f151b200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f151406eb30 con 0x7f1514146130 2024-04-20T21:19:31.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.951+0000 7f151b200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f151406dc30 con 0x7f151406e6d0 2024-04-20T21:19:31.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.951+0000 7f151b200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f15140cd8e0 con 0x7f1514072240 2024-04-20T21:19:31.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.951+0000 7f1513400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f151406e6d0 0x7f1514071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:31.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.951+0000 7f1513400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f151406e6d0 0x7f1514071d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43574/0 (socket says 172.21.15.86:43574) 2024-04-20T21:19:31.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.951+0000 7f1513400640 1 -- 172.21.15.86:0/33720688 learned_addr learned my addr 172.21.15.86:0/33720688 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:31.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.951+0000 7f1519200640 1 --2- 172.21.15.86:0/33720688 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1514072240 0x7f151415cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:31.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.952+0000 7f1513400640 1 -- 172.21.15.86:0/33720688 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1514072240 msgr2=0x7f151415cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:31.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.952+0000 7f1513400640 1 --2- 172.21.15.86:0/33720688 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1514072240 0x7f151415cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:31.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.952+0000 7f1513400640 1 -- 172.21.15.86:0/33720688 >> v1:172.21.15.144:6789/0 conn(0x7f1514146130 legacy=0x7f15140ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:31.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.952+0000 7f1513400640 1 -- 172.21.15.86:0/33720688 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15140cdbc0 con 0x7f151406e6d0 2024-04-20T21:19:31.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.953+0000 7f1513400640 1 --2- 172.21.15.86:0/33720688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f151406e6d0 0x7f1514071d00 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f1500009870 tx=0x7f150002ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f23696d3f2f9dfde server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:31.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.953+0000 7f1512a00640 1 -- 172.21.15.86:0/33720688 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150002f840 con 0x7f151406e6d0 2024-04-20T21:19:31.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.953+0000 7f1512a00640 1 -- 172.21.15.86:0/33720688 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f150002f9e0 con 0x7f151406e6d0 2024-04-20T21:19:31.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.953+0000 7f1512a00640 1 -- 172.21.15.86:0/33720688 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150002fdb0 con 0x7f151406e6d0 2024-04-20T21:19:31.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.954+0000 7f151b200640 1 -- 172.21.15.86:0/33720688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f151406e6d0 msgr2=0x7f1514071d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:31.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.954+0000 7f151b200640 1 --2- 172.21.15.86:0/33720688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f151406e6d0 0x7f1514071d00 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f1500009870 tx=0x7f150002ba20 comp rx=0 tx=0).stop 2024-04-20T21:19:31.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.954+0000 7f151b200640 1 -- 172.21.15.86:0/33720688 shutdown_connections 2024-04-20T21:19:31.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.954+0000 7f151b200640 1 --2- 172.21.15.86:0/33720688 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1514072240 0x7f151415cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:31.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.954+0000 7f151b200640 1 --2- 172.21.15.86:0/33720688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f151406e6d0 0x7f1514071d00 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:31.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.954+0000 7f151b200640 1 -- 172.21.15.86:0/33720688 >> 172.21.15.86:0/33720688 conn(0x7f15140b2360 msgr2=0x7f15140b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:31.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.954+0000 7f151b200640 1 -- 172.21.15.86:0/33720688 shutdown_connections 2024-04-20T21:19:31.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.954+0000 7f151b200640 1 -- 172.21.15.86:0/33720688 wait complete. 2024-04-20T21:19:31.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.956+0000 7f151b200640 1 Processor -- start 2024-04-20T21:19:31.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.956+0000 7f151b200640 1 -- start start 2024-04-20T21:19:31.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.957+0000 7f151b200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f151406e6d0 0x7f151415c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:31.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.957+0000 7f151b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1514072240 0x7f15141e7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:31.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.957+0000 7f151b200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1514146130 0x7f15141e7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:31.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.957+0000 7f151b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f151406f150 con 0x7f1514072240 2024-04-20T21:19:31.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.957+0000 7f151b200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f151406e1b0 con 0x7f151406e6d0 2024-04-20T21:19:31.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.957+0000 7f151b200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f15140cd4e0 con 0x7f1514146130 2024-04-20T21:19:31.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.957+0000 7f1513400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1514072240 0x7f15141e7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:31.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.957+0000 7f1513400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1514072240 0x7f15141e7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43588/0 (socket says 172.21.15.86:43588) 2024-04-20T21:19:31.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.957+0000 7f1513400640 1 -- 172.21.15.86:0/3361397904 learned_addr learned my addr 172.21.15.86:0/3361397904 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:31.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.957+0000 7f1519200640 1 --2- 172.21.15.86:0/3361397904 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1514146130 0x7f15141e7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:31.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.957+0000 7f1513e00640 1 --2- 172.21.15.86:0/3361397904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f151406e6d0 0x7f151415c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:31.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.957+0000 7f1513400640 1 -- 172.21.15.86:0/3361397904 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1514146130 msgr2=0x7f15141e7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:31.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.957+0000 7f1513400640 1 --2- 172.21.15.86:0/3361397904 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1514146130 0x7f15141e7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:31.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.957+0000 7f1513400640 1 -- 172.21.15.86:0/3361397904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f151406e6d0 msgr2=0x7f151415c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:31.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.957+0000 7f1513400640 1 --2- 172.21.15.86:0/3361397904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f151406e6d0 0x7f151415c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:31.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.957+0000 7f1513400640 1 -- 172.21.15.86:0/3361397904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15141e51b0 con 0x7f1514072240 2024-04-20T21:19:31.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.958+0000 7f1513400640 1 --2- 172.21.15.86:0/3361397904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1514072240 0x7f15141e7950 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f1500000c00 tx=0x7f1500004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:31.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.958+0000 7f150be00640 1 -- 172.21.15.86:0/3361397904 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1500041400 con 0x7f1514072240 2024-04-20T21:19:31.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.958+0000 7f150be00640 1 -- 172.21.15.86:0/3361397904 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1500046070 con 0x7f1514072240 2024-04-20T21:19:31.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.958+0000 7f150be00640 1 -- 172.21.15.86:0/3361397904 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1500041800 con 0x7f1514072240 2024-04-20T21:19:31.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.958+0000 7f151b200640 1 -- 172.21.15.86:0/3361397904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15141e5490 con 0x7f1514072240 2024-04-20T21:19:31.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.958+0000 7f151b200640 1 -- 172.21.15.86:0/3361397904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15141e5a50 con 0x7f1514072240 2024-04-20T21:19:31.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.959+0000 7f151b200640 1 -- 172.21.15.86:0/3361397904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1514143480 con 0x7f1514072240 2024-04-20T21:19:31.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.960+0000 7f150be00640 1 -- 172.21.15.86:0/3361397904 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1500040070 con 0x7f1514072240 2024-04-20T21:19:31.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.961+0000 7f150be00640 1 --2- 172.21.15.86:0/3361397904 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f14f406c8f0 0x7f14f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:31.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.961+0000 7f150be00640 1 -- 172.21.15.86:0/3361397904 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f15000ae450 con 0x7f1514072240 2024-04-20T21:19:31.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.962+0000 7f1513e00640 1 --2- 172.21.15.86:0/3361397904 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f14f406c8f0 0x7f14f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:31.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.962+0000 7f1513e00640 1 --2- 172.21.15.86:0/3361397904 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f14f406c8f0 0x7f14f406edb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1504007fd0 tx=0x7f1504009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:31.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:31.966+0000 7f150be00640 1 -- 172.21.15.86:0/3361397904 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1514143480 con 0x7f1514072240 2024-04-20T21:19:32.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.201+0000 7f151b200640 1 -- 172.21.15.86:0/3361397904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15140689e0 con 0x7f1514072240 2024-04-20T21:19:32.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.202+0000 7f150be00640 1 -- 172.21.15.86:0/3361397904 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5205 (secure 0 0 0) 0x7f1500033030 con 0x7f1514072240 2024-04-20T21:19:32.203 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:19:32.203 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4604,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/1957936003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":1957936003},{"type":"v1","addr":"172.21.15.144:6834","nonce":1957936003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4807,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/1510649389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":1510649389},{"type":"v1","addr":"172.21.15.144:6835","nonce":1510649389}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2180009156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2180009156},{"type":"v1","addr":"172.21.15.86:6836","nonce":2180009156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:19:09.979345+0000","modified":"2024-04-20T21:19:13.064114+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5226":{"gid":5226,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6837/3715801064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3715801064},{"type":"v1","addr":"172.21.15.86:6837","nonce":3715801064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5226,"qdb_cluster":[5226]},"id":2}]} 2024-04-20T21:19:32.203 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 16 2024-04-20T21:19:32.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.205+0000 7f151b200640 1 -- 172.21.15.86:0/3361397904 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f14f406c8f0 msgr2=0x7f14f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:32.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.205+0000 7f151b200640 1 --2- 172.21.15.86:0/3361397904 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f14f406c8f0 0x7f14f406edb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1504007fd0 tx=0x7f1504009040 comp rx=0 tx=0).stop 2024-04-20T21:19:32.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.205+0000 7f151b200640 1 -- 172.21.15.86:0/3361397904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1514072240 msgr2=0x7f15141e7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:32.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.205+0000 7f151b200640 1 --2- 172.21.15.86:0/3361397904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1514072240 0x7f15141e7950 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f1500000c00 tx=0x7f1500004730 comp rx=0 tx=0).stop 2024-04-20T21:19:32.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.205+0000 7f151b200640 1 -- 172.21.15.86:0/3361397904 shutdown_connections 2024-04-20T21:19:32.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.205+0000 7f151b200640 1 --2- 172.21.15.86:0/3361397904 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f14f406c8f0 0x7f14f406edb0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:32.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.205+0000 7f151b200640 1 --2- 172.21.15.86:0/3361397904 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1514146130 0x7f15141e7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:32.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.205+0000 7f151b200640 1 --2- 172.21.15.86:0/3361397904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1514072240 0x7f15141e7950 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:32.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.205+0000 7f151b200640 1 --2- 172.21.15.86:0/3361397904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f151406e6d0 0x7f151415c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:32.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.205+0000 7f151b200640 1 -- 172.21.15.86:0/3361397904 >> 172.21.15.86:0/3361397904 conn(0x7f15140b2360 msgr2=0x7f15140b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:32.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.206+0000 7f151b200640 1 -- 172.21.15.86:0/3361397904 shutdown_connections 2024-04-20T21:19:32.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.206+0000 7f151b200640 1 -- 172.21.15.86:0/3361397904 wait complete. 2024-04-20T21:19:32.224 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-20T21:19:32.465 INFO:teuthology.orchestra.run.smithi086.stdout:{"message":"","return_code":0} 2024-04-20T21:19:32.465 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-20T21:19:32.465 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:19:32.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.656+0000 7f69c8200640 1 Processor -- start 2024-04-20T21:19:32.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.656+0000 7f69c8200640 1 -- start start 2024-04-20T21:19:32.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.657+0000 7f69c8200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f69c0144e90 0x7f69c00b4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:32.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.658+0000 7f69c8200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f69c00cd170 0x7f69c006ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:32.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.658+0000 7f69c8200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69c00cdb50 con 0x7f69c00cd600 2024-04-20T21:19:32.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.658+0000 7f69c8200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f69c0145270 con 0x7f69c0144e90 2024-04-20T21:19:32.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.658+0000 7f69c8200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f69c00b3e50 con 0x7f69c00cd170 2024-04-20T21:19:32.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.658+0000 7f69c5800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f69c0144e90 0x7f69c00b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:32.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.658+0000 7f69c6200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f69c00cd600 0x7f69c015bb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:49782/0 (socket says 172.21.15.86:49782) 2024-04-20T21:19:32.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.658+0000 7f69c6200640 1 -- 172.21.15.86:0/773650753 learned_addr learned my addr 172.21.15.86:0/773650753 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:19:32.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.658+0000 7f69c4e00640 1 --2- 172.21.15.86:0/773650753 >> v2:172.21.15.144:3300/0 conn(0x7f69c00cd170 0x7f69c006ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:32.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.658+0000 7f69c5800640 1 -- 172.21.15.86:0/773650753 >> v2:172.21.15.144:3300/0 conn(0x7f69c00cd170 msgr2=0x7f69c006ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:32.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.658+0000 7f69c5800640 1 --2- 172.21.15.86:0/773650753 >> v2:172.21.15.144:3300/0 conn(0x7f69c00cd170 0x7f69c006ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:32.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.658+0000 7f69c5800640 1 -- 172.21.15.86:0/773650753 >> v1:172.21.15.86:6789/0 conn(0x7f69c00cd600 legacy=0x7f69c015bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:32.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.658+0000 7f69c5800640 1 -- 172.21.15.86:0/773650753 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69c00b4130 con 0x7f69c0144e90 2024-04-20T21:19:32.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.659+0000 7f69c4e00640 1 --2- 172.21.15.86:0/773650753 >> v2:172.21.15.144:3300/0 conn(0x7f69c00cd170 0x7f69c006ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:32.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.658+0000 7f69bfe00640 1 -- 172.21.15.86:0/773650753 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 407700912 0 0) 0x7f69c00cdb50 con 0x7f69c00cd600 2024-04-20T21:19:32.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.660+0000 7f69c5800640 1 --2- 172.21.15.86:0/773650753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69c0144e90 0x7f69c00b4930 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f69c006fa20 tx=0x7f69b802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e615a6f616b5e341 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:32.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.660+0000 7f69bfe00640 1 -- 172.21.15.86:0/773650753 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b8039070 con 0x7f69c0144e90 2024-04-20T21:19:32.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.660+0000 7f69bfe00640 1 -- 172.21.15.86:0/773650753 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69b802cc80 con 0x7f69c0144e90 2024-04-20T21:19:32.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.660+0000 7f69bfe00640 1 -- 172.21.15.86:0/773650753 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b8034660 con 0x7f69c0144e90 2024-04-20T21:19:32.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.661+0000 7f69c8200640 1 -- 172.21.15.86:0/773650753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69c0144e90 msgr2=0x7f69c00b4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:32.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.661+0000 7f69c8200640 1 --2- 172.21.15.86:0/773650753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69c0144e90 0x7f69c00b4930 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f69c006fa20 tx=0x7f69b802bed0 comp rx=0 tx=0).stop 2024-04-20T21:19:32.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.661+0000 7f69c8200640 1 -- 172.21.15.86:0/773650753 shutdown_connections 2024-04-20T21:19:32.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.661+0000 7f69c8200640 1 --2- 172.21.15.86:0/773650753 >> v2:172.21.15.144:3300/0 conn(0x7f69c00cd170 0x7f69c006ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:32.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.661+0000 7f69c8200640 1 --2- 172.21.15.86:0/773650753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69c0144e90 0x7f69c00b4930 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:32.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.661+0000 7f69c8200640 1 -- 172.21.15.86:0/773650753 >> 172.21.15.86:0/773650753 conn(0x7f69c00b2360 msgr2=0x7f69c00b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:32.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.661+0000 7f69c8200640 1 -- 172.21.15.86:0/773650753 shutdown_connections 2024-04-20T21:19:32.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.661+0000 7f69c8200640 1 -- 172.21.15.86:0/773650753 wait complete. 2024-04-20T21:19:32.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.663+0000 7f69c8200640 1 Processor -- start 2024-04-20T21:19:32.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.663+0000 7f69c8200640 1 -- start start 2024-04-20T21:19:32.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.664+0000 7f69c8200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69c00cd170 0x7f69c013a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:32.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.664+0000 7f69c8200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69c00cd600 0x7f69c013c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:32.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.664+0000 7f69c8200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69c0144e90 0x7f69c015aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:32.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.664+0000 7f69c8200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f69c00d0010 con 0x7f69c00cd600 2024-04-20T21:19:32.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.664+0000 7f69c8200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f69c00b4e70 con 0x7f69c00cd170 2024-04-20T21:19:32.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.664+0000 7f69c8200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f69c00d0190 con 0x7f69c0144e90 2024-04-20T21:19:32.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.664+0000 7f69c4e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69c00cd600 0x7f69c013c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:32.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.664+0000 7f69c4e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69c00cd600 0x7f69c013c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43592/0 (socket says 172.21.15.86:43592) 2024-04-20T21:19:32.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.664+0000 7f69c4e00640 1 -- 172.21.15.86:0/4146666051 learned_addr learned my addr 172.21.15.86:0/4146666051 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:32.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.664+0000 7f69c5800640 1 --2- 172.21.15.86:0/4146666051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69c00cd170 0x7f69c013a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:32.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.664+0000 7f69c6200640 1 --2- 172.21.15.86:0/4146666051 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69c0144e90 0x7f69c015aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:32.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.664+0000 7f69c4e00640 1 -- 172.21.15.86:0/4146666051 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69c0144e90 msgr2=0x7f69c015aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:32.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.665+0000 7f69c4e00640 1 --2- 172.21.15.86:0/4146666051 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69c0144e90 0x7f69c015aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:32.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.665+0000 7f69c4e00640 1 -- 172.21.15.86:0/4146666051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69c00cd170 msgr2=0x7f69c013a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:32.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.665+0000 7f69c4e00640 1 --2- 172.21.15.86:0/4146666051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69c00cd170 0x7f69c013a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:32.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.665+0000 7f69c4e00640 1 -- 172.21.15.86:0/4146666051 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69c013abb0 con 0x7f69c00cd600 2024-04-20T21:19:32.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.665+0000 7f69c5800640 1 --2- 172.21.15.86:0/4146666051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69c00cd170 0x7f69c013a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:32.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.665+0000 7f69c6200640 1 --2- 172.21.15.86:0/4146666051 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69c0144e90 0x7f69c015aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:32.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.665+0000 7f69c4e00640 1 --2- 172.21.15.86:0/4146666051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69c00cd600 0x7f69c013c140 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f69ac00c820 tx=0x7f69ac00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:32.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.665+0000 7f69bd600640 1 -- 172.21.15.86:0/4146666051 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ac015070 con 0x7f69c00cd600 2024-04-20T21:19:32.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.665+0000 7f69bd600640 1 -- 172.21.15.86:0/4146666051 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69ac007a30 con 0x7f69c00cd600 2024-04-20T21:19:32.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.665+0000 7f69bd600640 1 -- 172.21.15.86:0/4146666051 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ac007e00 con 0x7f69c00cd600 2024-04-20T21:19:32.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.665+0000 7f69c8200640 1 -- 172.21.15.86:0/4146666051 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69c013ae90 con 0x7f69c00cd600 2024-04-20T21:19:32.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.666+0000 7f69c8200640 1 -- 172.21.15.86:0/4146666051 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69c013b450 con 0x7f69c00cd600 2024-04-20T21:19:32.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.667+0000 7f69c8200640 1 -- 172.21.15.86:0/4146666051 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69c00689e0 con 0x7f69c00cd600 2024-04-20T21:19:32.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.668+0000 7f69bd600640 1 -- 172.21.15.86:0/4146666051 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f69ac018070 con 0x7f69c00cd600 2024-04-20T21:19:32.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.669+0000 7f69bd600640 1 --2- 172.21.15.86:0/4146666051 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f69a006c8f0 0x7f69a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:32.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.669+0000 7f69bd600640 1 -- 172.21.15.86:0/4146666051 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f69ac08bb30 con 0x7f69c00cd600 2024-04-20T21:19:32.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.669+0000 7f69c5800640 1 --2- 172.21.15.86:0/4146666051 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f69a006c8f0 0x7f69a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:32.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.670+0000 7f69c5800640 1 --2- 172.21.15.86:0/4146666051 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f69a006c8f0 0x7f69a006edb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f69b8036000 tx=0x7f69b8030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:32.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.673+0000 7f69bd600640 1 -- 172.21.15.86:0/4146666051 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f69ac059f50 con 0x7f69c00cd600 2024-04-20T21:19:32.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.922+0000 7f69c8200640 1 -- 172.21.15.86:0/4146666051 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f69c006cce0 con 0x7f69c00cd600 2024-04-20T21:19:32.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.924+0000 7f69bd600640 1 -- 172.21.15.86:0/4146666051 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5205 (secure 0 0 0) 0x7f69ac059850 con 0x7f69c00cd600 2024-04-20T21:19:32.924 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:19:32.924 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4604,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/1957936003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":1957936003},{"type":"v1","addr":"172.21.15.144:6834","nonce":1957936003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4807,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/1510649389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":1510649389},{"type":"v1","addr":"172.21.15.144:6835","nonce":1510649389}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2180009156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2180009156},{"type":"v1","addr":"172.21.15.86:6836","nonce":2180009156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:19:09.979345+0000","modified":"2024-04-20T21:19:13.064114+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5226":{"gid":5226,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6837/3715801064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3715801064},{"type":"v1","addr":"172.21.15.86:6837","nonce":3715801064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5226,"qdb_cluster":[5226]},"id":2}]} 2024-04-20T21:19:32.924 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 16 2024-04-20T21:19:32.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.926+0000 7f69c8200640 1 -- 172.21.15.86:0/4146666051 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f69a006c8f0 msgr2=0x7f69a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:32.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.926+0000 7f69c8200640 1 --2- 172.21.15.86:0/4146666051 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f69a006c8f0 0x7f69a006edb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f69b8036000 tx=0x7f69b8030000 comp rx=0 tx=0).stop 2024-04-20T21:19:32.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.926+0000 7f69c8200640 1 -- 172.21.15.86:0/4146666051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69c00cd600 msgr2=0x7f69c013c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:32.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.926+0000 7f69c8200640 1 --2- 172.21.15.86:0/4146666051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69c00cd600 0x7f69c013c140 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f69ac00c820 tx=0x7f69ac00cc20 comp rx=0 tx=0).stop 2024-04-20T21:19:32.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.927+0000 7f69c8200640 1 -- 172.21.15.86:0/4146666051 shutdown_connections 2024-04-20T21:19:32.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.927+0000 7f69c8200640 1 --2- 172.21.15.86:0/4146666051 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f69a006c8f0 0x7f69a006edb0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:32.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.927+0000 7f69c8200640 1 --2- 172.21.15.86:0/4146666051 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69c0144e90 0x7f69c015aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:32.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.927+0000 7f69c8200640 1 --2- 172.21.15.86:0/4146666051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69c00cd600 0x7f69c013c140 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:32.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.927+0000 7f69c8200640 1 --2- 172.21.15.86:0/4146666051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69c00cd170 0x7f69c013a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:32.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.927+0000 7f69c8200640 1 -- 172.21.15.86:0/4146666051 >> 172.21.15.86:0/4146666051 conn(0x7f69c00b2360 msgr2=0x7f69c00cc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:32.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.927+0000 7f69c8200640 1 -- 172.21.15.86:0/4146666051 shutdown_connections 2024-04-20T21:19:32.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:32.927+0000 7f69c8200640 1 -- 172.21.15.86:0/4146666051 wait complete. 2024-04-20T21:19:32.946 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json tag path /parentdir mytag 2024-04-20T21:19:33.162 INFO:teuthology.orchestra.run.smithi086.stdout:{"return_code":0,"scrub_tag":"mytag","mode":"asynchronous"} 2024-04-20T21:19:33.162 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag" } 2024-04-20T21:19:43.163 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000000.00000000 scrub_tag 2024-04-20T21:19:43.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.281+0000 7f0287eb5f40 1 Processor -- start 2024-04-20T21:19:43.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.281+0000 7f0287eb5f40 1 -- start start 2024-04-20T21:19:43.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.282+0000 7f0287eb5f40 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561f01da1640 0x561f01da0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:43.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.282+0000 7f0287eb5f40 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f01e7d9d0 0x561f01e7e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:43.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.282+0000 7f0287eb5f40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f01e7ddb0 0x561f01f10330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:43.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.282+0000 7f0287eb5f40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561f01d49cc0 con 0x561f01e7ddb0 2024-04-20T21:19:43.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.282+0000 7f0287eb5f40 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561f01dd22b0 con 0x561f01e7d9d0 2024-04-20T21:19:43.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.282+0000 7f0287eb5f40 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x561f01dd1b30 con 0x561f01da1640 2024-04-20T21:19:43.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.282+0000 7f0285a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f01e7ddb0 0x561f01f10330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:43.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.282+0000 7f0285a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f01e7ddb0 0x561f01f10330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50510/0 (socket says 172.21.15.86:50510) 2024-04-20T21:19:43.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.282+0000 7f0285a00640 1 -- 172.21.15.86:0/2397836490 learned_addr learned my addr 172.21.15.86:0/2397836490 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:43.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.282+0000 7f0285000640 1 --2- 172.21.15.86:0/2397836490 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561f01da1640 0x561f01da0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:43.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.282+0000 7f0285a00640 1 -- 172.21.15.86:0/2397836490 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561f01da1640 msgr2=0x561f01da0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:43.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.282+0000 7f027fe00640 1 --2- 172.21.15.86:0/2397836490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f01e7d9d0 0x561f01e7e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:43.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.283+0000 7f0285a00640 1 --2- 172.21.15.86:0/2397836490 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561f01da1640 0x561f01da0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:43.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.283+0000 7f0285a00640 1 -- 172.21.15.86:0/2397836490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f01e7d9d0 msgr2=0x561f01e7e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:43.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.283+0000 7f0285a00640 1 --2- 172.21.15.86:0/2397836490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f01e7d9d0 0x561f01e7e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:43.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.283+0000 7f0285a00640 1 -- 172.21.15.86:0/2397836490 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f01dcf530 con 0x561f01e7ddb0 2024-04-20T21:19:43.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.283+0000 7f0285000640 1 --2- 172.21.15.86:0/2397836490 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561f01da1640 0x561f01da0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:43.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.283+0000 7f027fe00640 1 --2- 172.21.15.86:0/2397836490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f01e7d9d0 0x561f01e7e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:43.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.283+0000 7f0285a00640 1 --2- 172.21.15.86:0/2397836490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f01e7ddb0 0x561f01f10330 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f028000a190 tx=0x7f028004f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12e007819899c4f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:43.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.283+0000 7f027ea00640 1 -- 172.21.15.86:0/2397836490 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028005a040 con 0x561f01e7ddb0 2024-04-20T21:19:43.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.283+0000 7f027ea00640 1 -- 172.21.15.86:0/2397836490 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0280064070 con 0x561f01e7ddb0 2024-04-20T21:19:43.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.283+0000 7f027ea00640 1 -- 172.21.15.86:0/2397836490 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028004dc50 con 0x561f01e7ddb0 2024-04-20T21:19:43.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.284+0000 7f0287eb5f40 1 -- 172.21.15.86:0/2397836490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f01e7ddb0 msgr2=0x561f01f10330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:43.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.284+0000 7f0287eb5f40 1 --2- 172.21.15.86:0/2397836490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f01e7ddb0 0x561f01f10330 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f028000a190 tx=0x7f028004f000 comp rx=0 tx=0).stop 2024-04-20T21:19:43.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.284+0000 7f0287eb5f40 1 -- 172.21.15.86:0/2397836490 shutdown_connections 2024-04-20T21:19:43.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.284+0000 7f0287eb5f40 1 --2- 172.21.15.86:0/2397836490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f01e7ddb0 0x561f01f10330 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:43.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.284+0000 7f0287eb5f40 1 --2- 172.21.15.86:0/2397836490 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f01e7d9d0 0x561f01e7e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:43.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.284+0000 7f0287eb5f40 1 --2- 172.21.15.86:0/2397836490 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561f01da1640 0x561f01da0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:43.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.284+0000 7f0287eb5f40 1 -- 172.21.15.86:0/2397836490 >> 172.21.15.86:0/2397836490 conn(0x561f01da0220 msgr2=0x561f01da0600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:43.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.284+0000 7f0287eb5f40 1 -- 172.21.15.86:0/2397836490 shutdown_connections 2024-04-20T21:19:43.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.284+0000 7f0287eb5f40 1 -- 172.21.15.86:0/2397836490 wait complete. 2024-04-20T21:19:43.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.285+0000 7f0287eb5f40 1 Processor -- start 2024-04-20T21:19:43.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.285+0000 7f0287eb5f40 1 -- start start 2024-04-20T21:19:43.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.286+0000 7f0287eb5f40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f01da1640 0x561f01e656a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:43.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.286+0000 7f0287eb5f40 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561f01e7d9d0 0x561f01e78e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:43.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.286+0000 7f0287eb5f40 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f01e7ddb0 0x561f01e79360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:43.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.286+0000 7f0287eb5f40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561f01dd18d0 con 0x561f01da1640 2024-04-20T21:19:43.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.286+0000 7f0287eb5f40 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561f01da1440 con 0x561f01e7ddb0 2024-04-20T21:19:43.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.286+0000 7f0287eb5f40 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x561f01e6ef90 con 0x561f01e7d9d0 2024-04-20T21:19:43.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.286+0000 7f0285000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f01da1640 0x561f01e656a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:43.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.286+0000 7f0285000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f01da1640 0x561f01e656a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50516/0 (socket says 172.21.15.86:50516) 2024-04-20T21:19:43.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.286+0000 7f0285000640 1 -- 172.21.15.86:0/3237916610 learned_addr learned my addr 172.21.15.86:0/3237916610 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:43.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.287+0000 7f0285a00640 1 --2- 172.21.15.86:0/3237916610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f01e7ddb0 0x561f01e79360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:43.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.287+0000 7f027fe00640 1 --2- 172.21.15.86:0/3237916610 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561f01e7d9d0 0x561f01e78e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:43.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.287+0000 7f0285000640 1 -- 172.21.15.86:0/3237916610 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561f01e7d9d0 msgr2=0x561f01e78e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:43.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.287+0000 7f0285000640 1 --2- 172.21.15.86:0/3237916610 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561f01e7d9d0 0x561f01e78e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:43.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.287+0000 7f0285000640 1 -- 172.21.15.86:0/3237916610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f01e7ddb0 msgr2=0x561f01e79360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:43.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.287+0000 7f0285000640 1 --2- 172.21.15.86:0/3237916610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f01e7ddb0 0x561f01e79360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:43.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.287+0000 7f0285000640 1 -- 172.21.15.86:0/3237916610 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f01e6f230 con 0x561f01da1640 2024-04-20T21:19:43.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.287+0000 7f0285a00640 1 --2- 172.21.15.86:0/3237916610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561f01e7ddb0 0x561f01e79360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:43.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.287+0000 7f0285000640 1 --2- 172.21.15.86:0/3237916610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f01da1640 0x561f01e656a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0278007e00 tx=0x7f0278006750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:43.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.287+0000 7f027d600640 1 -- 172.21.15.86:0/3237916610 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0278010710 con 0x561f01da1640 2024-04-20T21:19:43.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.287+0000 7f027d600640 1 -- 172.21.15.86:0/3237916610 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02780108b0 con 0x561f01da1640 2024-04-20T21:19:43.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.287+0000 7f027d600640 1 -- 172.21.15.86:0/3237916610 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0278010ca0 con 0x561f01da1640 2024-04-20T21:19:43.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.287+0000 7f0287eb5f40 1 -- 172.21.15.86:0/3237916610 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561f01e6f510 con 0x561f01da1640 2024-04-20T21:19:43.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.287+0000 7f0287eb5f40 1 -- 172.21.15.86:0/3237916610 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f01e6fad0 con 0x561f01da1640 2024-04-20T21:19:43.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.289+0000 7f027d600640 1 -- 172.21.15.86:0/3237916610 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0278012400 con 0x561f01da1640 2024-04-20T21:19:43.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.290+0000 7f027d600640 1 --2- 172.21.15.86:0/3237916610 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f026406b2d0 0x7f026406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:43.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.290+0000 7f027d600640 1 -- 172.21.15.86:0/3237916610 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f027808a0b0 con 0x561f01da1640 2024-04-20T21:19:43.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.291+0000 7f027fe00640 1 --2- 172.21.15.86:0/3237916610 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f026406b2d0 0x7f026406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:43.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.291+0000 7f0287eb5f40 1 --2- 172.21.15.86:0/3237916610 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x561f01dd0e80 0x561f01f1fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:43.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.291+0000 7f0287eb5f40 1 -- 172.21.15.86:0/3237916610 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:1 4.10 4:0cf6ad0f:::10000000000.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x561f01f20120 con 0x561f01dd0e80 2024-04-20T21:19:43.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.291+0000 7f0285a00640 1 --2- 172.21.15.86:0/3237916610 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x561f01dd0e80 0x561f01f1fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:43.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.291+0000 7f027fe00640 1 --2- 172.21.15.86:0/3237916610 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f026406b2d0 0x7f026406d790 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x561f01e73660 tx=0x7f02700055b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:43.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.292+0000 7f0285a00640 1 --2- 172.21.15.86:0/3237916610 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x561f01dd0e80 0x561f01f1fbe0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:43.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.292+0000 7f0285a00640 1 -- 172.21.15.86:0/3237916610 <== osd.3 v2:172.21.15.86:6808/794135519 1 ==== osd_op_reply(1 10000000000.00000000 [getxattr (9) out=9b] v0'0 uv11 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7f028005d040 con 0x561f01dd0e80 2024-04-20T21:19:43.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.292+0000 7f0287eb5f40 1 -- 172.21.15.86:0/3237916610 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x561f01dd0e80 msgr2=0x561f01f1fbe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:43.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.292+0000 7f0287eb5f40 1 --2- 172.21.15.86:0/3237916610 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x561f01dd0e80 0x561f01f1fbe0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:43.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.292+0000 7f0287eb5f40 1 -- 172.21.15.86:0/3237916610 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f026406b2d0 msgr2=0x7f026406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:43.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.292+0000 7f0287eb5f40 1 --2- 172.21.15.86:0/3237916610 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f026406b2d0 0x7f026406d790 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x561f01e73660 tx=0x7f02700055b0 comp rx=0 tx=0).stop 2024-04-20T21:19:43.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.292+0000 7f0287eb5f40 1 -- 172.21.15.86:0/3237916610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f01da1640 msgr2=0x561f01e656a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:43.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.292+0000 7f0287eb5f40 1 --2- 172.21.15.86:0/3237916610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f01da1640 0x561f01e656a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0278007e00 tx=0x7f0278006750 comp rx=0 tx=0).stop 2024-04-20T21:19:43.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.292+0000 7f0285a00640 1 -- 172.21.15.86:0/3237916610 reap_dead start 2024-04-20T21:19:43.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.292+0000 7f0287eb5f40 1 -- 172.21.15.86:0/3237916610 shutdown_connections 2024-04-20T21:19:43.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.292+0000 7f0287eb5f40 1 -- 172.21.15.86:0/3237916610 >> 172.21.15.86:0/3237916610 conn(0x561f01da0220 msgr2=0x561f01da0600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:43.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.293+0000 7f0287eb5f40 1 -- 172.21.15.86:0/3237916610 shutdown_connections 2024-04-20T21:19:43.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.293+0000 7f0287eb5f40 1 -- 172.21.15.86:0/3237916610 wait complete. 2024-04-20T21:19:43.296 INFO:teuthology.orchestra.run.smithi086.stdout:mytag 2024-04-20T21:19:43.297 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000001.00000000 scrub_tag 2024-04-20T21:19:43.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.396+0000 7fa5af786f40 1 Processor -- start 2024-04-20T21:19:43.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.397+0000 7fa5af786f40 1 -- start start 2024-04-20T21:19:43.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.397+0000 7fa5af786f40 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5900022f0 0x55f9d104f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:43.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.397+0000 7fa5af786f40 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9d0f9db80 0x55f9d0f9b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:43.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.397+0000 7fa5af786f40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9d0f9c8a0 0x55f9d0f9cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:43.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.397+0000 7fa5af786f40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f9d0f41cc0 con 0x55f9d0f9c8a0 2024-04-20T21:19:43.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.397+0000 7fa5af786f40 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f9d0f45d00 con 0x55f9d0f9db80 2024-04-20T21:19:43.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.397+0000 7fa5af786f40 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55f9d10734c0 con 0x7fa5900022f0 2024-04-20T21:19:43.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.398+0000 7fa5ad400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9d0f9c8a0 0x55f9d0f9cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:43.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.398+0000 7fa5ad400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9d0f9c8a0 0x55f9d0f9cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50532/0 (socket says 172.21.15.86:50532) 2024-04-20T21:19:43.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.398+0000 7fa5ad400640 1 -- 172.21.15.86:0/2635998554 learned_addr learned my addr 172.21.15.86:0/2635998554 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:43.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.398+0000 7fa5aca00640 1 --2- 172.21.15.86:0/2635998554 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5900022f0 0x55f9d104f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:43.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.398+0000 7fa5ad400640 1 -- 172.21.15.86:0/2635998554 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5900022f0 msgr2=0x55f9d104f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:43.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.398+0000 7fa5ad400640 1 --2- 172.21.15.86:0/2635998554 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5900022f0 0x55f9d104f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:43.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.398+0000 7fa5ac000640 1 --2- 172.21.15.86:0/2635998554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9d0f9db80 0x55f9d0f9b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:43.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.398+0000 7fa5ad400640 1 -- 172.21.15.86:0/2635998554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9d0f9db80 msgr2=0x55f9d0f9b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:43.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.398+0000 7fa5ad400640 1 --2- 172.21.15.86:0/2635998554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9d0f9db80 0x55f9d0f9b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:43.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.398+0000 7fa5ad400640 1 -- 172.21.15.86:0/2635998554 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9d104fc20 con 0x55f9d0f9c8a0 2024-04-20T21:19:43.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.398+0000 7fa5aca00640 1 --2- 172.21.15.86:0/2635998554 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5900022f0 0x55f9d104f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:43.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.398+0000 7fa5ad400640 1 --2- 172.21.15.86:0/2635998554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9d0f9c8a0 0x55f9d0f9cce0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa59c003c00 tx=0x7fa59c00c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f086c2f7fcbd0c8b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:43.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.399+0000 7fa5aac00640 1 -- 172.21.15.86:0/2635998554 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c010470 con 0x55f9d0f9c8a0 2024-04-20T21:19:43.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.399+0000 7fa5aac00640 1 -- 172.21.15.86:0/2635998554 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa59c010610 con 0x55f9d0f9c8a0 2024-04-20T21:19:43.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.399+0000 7fa5aac00640 1 -- 172.21.15.86:0/2635998554 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c010a00 con 0x55f9d0f9c8a0 2024-04-20T21:19:43.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.399+0000 7fa5af786f40 1 -- 172.21.15.86:0/2635998554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9d0f9c8a0 msgr2=0x55f9d0f9cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:43.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.399+0000 7fa5af786f40 1 --2- 172.21.15.86:0/2635998554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9d0f9c8a0 0x55f9d0f9cce0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa59c003c00 tx=0x7fa59c00c620 comp rx=0 tx=0).stop 2024-04-20T21:19:43.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.399+0000 7fa5af786f40 1 -- 172.21.15.86:0/2635998554 shutdown_connections 2024-04-20T21:19:43.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.399+0000 7fa5af786f40 1 --2- 172.21.15.86:0/2635998554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9d0f9c8a0 0x55f9d0f9cce0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:43.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.399+0000 7fa5af786f40 1 --2- 172.21.15.86:0/2635998554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9d0f9db80 0x55f9d0f9b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:43.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.399+0000 7fa5af786f40 1 --2- 172.21.15.86:0/2635998554 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5900022f0 0x55f9d104f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:43.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.399+0000 7fa5af786f40 1 -- 172.21.15.86:0/2635998554 >> 172.21.15.86:0/2635998554 conn(0x55f9d0f98220 msgr2=0x55f9d0f9d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:43.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.399+0000 7fa5af786f40 1 -- 172.21.15.86:0/2635998554 shutdown_connections 2024-04-20T21:19:43.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.400+0000 7fa5af786f40 1 -- 172.21.15.86:0/2635998554 wait complete. 2024-04-20T21:19:43.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.400+0000 7fa5af786f40 1 Processor -- start 2024-04-20T21:19:43.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.400+0000 7fa5af786f40 1 -- start start 2024-04-20T21:19:43.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.401+0000 7fa5af786f40 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9d0f9c8a0 0x55f9d1068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:43.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.401+0000 7fa5af786f40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9d0f9db80 0x55f9d0f99250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:43.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.401+0000 7fa5af786f40 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5900022f0 0x55f9d0f99790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:43.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.401+0000 7fa5af786f40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f9d0f9b540 con 0x55f9d0f9db80 2024-04-20T21:19:43.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.401+0000 7fa5af786f40 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f9d10665e0 con 0x55f9d0f9c8a0 2024-04-20T21:19:43.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.401+0000 7fa5af786f40 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55f9d0f9d220 con 0x7fa5900022f0 2024-04-20T21:19:43.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.402+0000 7fa5ac000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9d0f9db80 0x55f9d0f99250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:43.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.402+0000 7fa5ac000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9d0f9db80 0x55f9d0f99250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50544/0 (socket says 172.21.15.86:50544) 2024-04-20T21:19:43.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.402+0000 7fa5ac000640 1 -- 172.21.15.86:0/2458350098 learned_addr learned my addr 172.21.15.86:0/2458350098 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:43.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.402+0000 7fa5ad400640 1 --2- 172.21.15.86:0/2458350098 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5900022f0 0x55f9d0f99790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:43.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.402+0000 7fa5aca00640 1 --2- 172.21.15.86:0/2458350098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9d0f9c8a0 0x55f9d1068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:43.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.402+0000 7fa5ac000640 1 -- 172.21.15.86:0/2458350098 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5900022f0 msgr2=0x55f9d0f99790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:43.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.402+0000 7fa5ac000640 1 --2- 172.21.15.86:0/2458350098 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5900022f0 0x55f9d0f99790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:43.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.402+0000 7fa5ac000640 1 -- 172.21.15.86:0/2458350098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9d0f9c8a0 msgr2=0x55f9d1068ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:43.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.402+0000 7fa5ac000640 1 --2- 172.21.15.86:0/2458350098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f9d0f9c8a0 0x55f9d1068ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:43.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.402+0000 7fa5ac000640 1 -- 172.21.15.86:0/2458350098 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9d0f99ed0 con 0x55f9d0f9db80 2024-04-20T21:19:43.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.402+0000 7fa5ac000640 1 --2- 172.21.15.86:0/2458350098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9d0f9db80 0x55f9d0f99250 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa59800f710 tx=0x7fa59800fbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:43.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.403+0000 7fa5a9800640 1 -- 172.21.15.86:0/2458350098 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59800acd0 con 0x55f9d0f9db80 2024-04-20T21:19:43.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.403+0000 7fa5a9800640 1 -- 172.21.15.86:0/2458350098 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa598018440 con 0x55f9d0f9db80 2024-04-20T21:19:43.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.403+0000 7fa5a9800640 1 -- 172.21.15.86:0/2458350098 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa598018810 con 0x55f9d0f9db80 2024-04-20T21:19:43.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.403+0000 7fa5af786f40 1 -- 172.21.15.86:0/2458350098 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f9d0f9a1b0 con 0x55f9d0f9db80 2024-04-20T21:19:43.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.403+0000 7fa5af786f40 1 -- 172.21.15.86:0/2458350098 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9d0f9a770 con 0x55f9d0f9db80 2024-04-20T21:19:43.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.405+0000 7fa5a9800640 1 -- 172.21.15.86:0/2458350098 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa5980213a0 con 0x55f9d0f9db80 2024-04-20T21:19:43.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.406+0000 7fa5a9800640 1 --2- 172.21.15.86:0/2458350098 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa58c06b2d0 0x7fa58c06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:43.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.406+0000 7fa5a9800640 1 -- 172.21.15.86:0/2458350098 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fa59808d640 con 0x55f9d0f9db80 2024-04-20T21:19:43.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.406+0000 7fa5aca00640 1 --2- 172.21.15.86:0/2458350098 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa58c06b2d0 0x7fa58c06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:43.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.406+0000 7fa5af786f40 1 --2- 172.21.15.86:0/2458350098 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x55f9d1069020 0x55f9d1107ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:43.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.406+0000 7fa5af786f40 1 -- 172.21.15.86:0/2458350098 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:1 4.7 4:e2e07930:::10000000001.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55f9d1113840 con 0x55f9d1069020 2024-04-20T21:19:43.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.406+0000 7fa5ad400640 1 --2- 172.21.15.86:0/2458350098 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x55f9d1069020 0x55f9d1107ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:43.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.407+0000 7fa5aca00640 1 --2- 172.21.15.86:0/2458350098 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa58c06b2d0 0x7fa58c06d790 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4056000 tx=0x7fa5a404d000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:43.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.407+0000 7fa5ad400640 1 --2- 172.21.15.86:0/2458350098 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x55f9d1069020 0x55f9d1107ba0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:43.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.407+0000 7fa5ad400640 1 -- 172.21.15.86:0/2458350098 <== osd.1 v2:172.21.15.86:6817/3149564961 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (9) out=9b] v0'0 uv5 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7fa59c010840 con 0x55f9d1069020 2024-04-20T21:19:43.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.407+0000 7fa5af786f40 1 -- 172.21.15.86:0/2458350098 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x55f9d1069020 msgr2=0x55f9d1107ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:43.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.407+0000 7fa5af786f40 1 --2- 172.21.15.86:0/2458350098 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x55f9d1069020 0x55f9d1107ba0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:43.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.408+0000 7fa5af786f40 1 -- 172.21.15.86:0/2458350098 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa58c06b2d0 msgr2=0x7fa58c06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:43.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.408+0000 7fa5af786f40 1 --2- 172.21.15.86:0/2458350098 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa58c06b2d0 0x7fa58c06d790 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4056000 tx=0x7fa5a404d000 comp rx=0 tx=0).stop 2024-04-20T21:19:43.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.408+0000 7fa5af786f40 1 -- 172.21.15.86:0/2458350098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9d0f9db80 msgr2=0x55f9d0f99250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:43.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.408+0000 7fa5af786f40 1 --2- 172.21.15.86:0/2458350098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f9d0f9db80 0x55f9d0f99250 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa59800f710 tx=0x7fa59800fbd0 comp rx=0 tx=0).stop 2024-04-20T21:19:43.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.408+0000 7fa5ad400640 1 -- 172.21.15.86:0/2458350098 reap_dead start 2024-04-20T21:19:43.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.408+0000 7fa5af786f40 1 -- 172.21.15.86:0/2458350098 shutdown_connections 2024-04-20T21:19:43.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.408+0000 7fa5af786f40 1 -- 172.21.15.86:0/2458350098 >> 172.21.15.86:0/2458350098 conn(0x55f9d0f98220 msgr2=0x55f9d0f9d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:43.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.408+0000 7fa5af786f40 1 -- 172.21.15.86:0/2458350098 shutdown_connections 2024-04-20T21:19:43.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.408+0000 7fa5af786f40 1 -- 172.21.15.86:0/2458350098 wait complete. 2024-04-20T21:19:43.411 INFO:teuthology.orchestra.run.smithi086.stdout:mytag 2024-04-20T21:19:43.413 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-04-20T21:19:43.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.506+0000 7f2d17e41f40 1 Processor -- start 2024-04-20T21:19:43.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.507+0000 7f2d17e41f40 1 -- start start 2024-04-20T21:19:43.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.507+0000 7f2d17e41f40 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564562784e30 0x56456276ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:43.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.507+0000 7f2d17e41f40 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5645627f7610 0x56456276b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:43.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.507+0000 7f2d17e41f40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645627faed0 0x5645628140a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:43.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.507+0000 7f2d17e41f40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564562786cf0 con 0x5645627faed0 2024-04-20T21:19:43.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.507+0000 7f2d17e41f40 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564562786a50 con 0x564562784e30 2024-04-20T21:19:43.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.507+0000 7f2d17e41f40 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x564562786f90 con 0x5645627f7610 2024-04-20T21:19:43.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.508+0000 7f2d15200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645627faed0 0x5645628140a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:43.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.508+0000 7f2d15200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645627faed0 0x5645628140a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50572/0 (socket says 172.21.15.86:50572) 2024-04-20T21:19:43.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.508+0000 7f2d15200640 1 -- 172.21.15.86:0/85105034 learned_addr learned my addr 172.21.15.86:0/85105034 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:43.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.508+0000 7f2d13e00640 1 --2- 172.21.15.86:0/85105034 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5645627f7610 0x56456276b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:43.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.508+0000 7f2d14800640 1 --2- 172.21.15.86:0/85105034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564562784e30 0x56456276ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:43.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.508+0000 7f2d15200640 1 -- 172.21.15.86:0/85105034 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5645627f7610 msgr2=0x56456276b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:43.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.508+0000 7f2d15200640 1 --2- 172.21.15.86:0/85105034 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5645627f7610 0x56456276b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:43.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.508+0000 7f2d15200640 1 -- 172.21.15.86:0/85105034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564562784e30 msgr2=0x56456276ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:43.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.508+0000 7f2d15200640 1 --2- 172.21.15.86:0/85105034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564562784e30 0x56456276ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:43.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.508+0000 7f2d15200640 1 -- 172.21.15.86:0/85105034 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564562665b30 con 0x5645627faed0 2024-04-20T21:19:43.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.509+0000 7f2d15200640 1 --2- 172.21.15.86:0/85105034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645627faed0 0x5645628140a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c01bdc0 tx=0x7f2d0c04c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3350b734231548f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:43.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.509+0000 7f2d12a00640 1 -- 172.21.15.86:0/85105034 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c057040 con 0x5645627faed0 2024-04-20T21:19:43.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.509+0000 7f2d12a00640 1 -- 172.21.15.86:0/85105034 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d0c063070 con 0x5645627faed0 2024-04-20T21:19:43.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.509+0000 7f2d12a00640 1 -- 172.21.15.86:0/85105034 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c05d8d0 con 0x5645627faed0 2024-04-20T21:19:43.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.509+0000 7f2d17e41f40 1 -- 172.21.15.86:0/85105034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645627faed0 msgr2=0x5645628140a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:43.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.509+0000 7f2d17e41f40 1 --2- 172.21.15.86:0/85105034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645627faed0 0x5645628140a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c01bdc0 tx=0x7f2d0c04c000 comp rx=0 tx=0).stop 2024-04-20T21:19:43.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.509+0000 7f2d17e41f40 1 -- 172.21.15.86:0/85105034 shutdown_connections 2024-04-20T21:19:43.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.509+0000 7f2d17e41f40 1 --2- 172.21.15.86:0/85105034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645627faed0 0x5645628140a0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:43.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.509+0000 7f2d17e41f40 1 --2- 172.21.15.86:0/85105034 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5645627f7610 0x56456276b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:43.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.509+0000 7f2d17e41f40 1 --2- 172.21.15.86:0/85105034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564562784e30 0x56456276ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:43.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.509+0000 7f2d17e41f40 1 -- 172.21.15.86:0/85105034 >> 172.21.15.86:0/85105034 conn(0x56456273f950 msgr2=0x564562738b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:43.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.510+0000 7f2d17e41f40 1 -- 172.21.15.86:0/85105034 shutdown_connections 2024-04-20T21:19:43.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.510+0000 7f2d17e41f40 1 -- 172.21.15.86:0/85105034 wait complete. 2024-04-20T21:19:43.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.510+0000 7f2d17e41f40 1 Processor -- start 2024-04-20T21:19:43.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.510+0000 7f2d17e41f40 1 -- start start 2024-04-20T21:19:43.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.511+0000 7f2d17e41f40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564562784e30 0x56456276bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:43.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.511+0000 7f2d17e41f40 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5645627f7610 0x56456280d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:43.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.511+0000 7f2d17e41f40 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645627faed0 0x5645627f20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:43.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.511+0000 7f2d17e41f40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5645627f7f30 con 0x564562784e30 2024-04-20T21:19:43.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.511+0000 7f2d17e41f40 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564562811870 con 0x5645627faed0 2024-04-20T21:19:43.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.511+0000 7f2d17e41f40 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5645627f81d0 con 0x5645627f7610 2024-04-20T21:19:43.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.512+0000 7f2d14800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564562784e30 0x56456276bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:43.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.512+0000 7f2d14800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564562784e30 0x56456276bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50588/0 (socket says 172.21.15.86:50588) 2024-04-20T21:19:43.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.512+0000 7f2d14800640 1 -- 172.21.15.86:0/3941795064 learned_addr learned my addr 172.21.15.86:0/3941795064 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:43.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.512+0000 7f2d13e00640 1 --2- 172.21.15.86:0/3941795064 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5645627f7610 0x56456280d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:43.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.512+0000 7f2d14800640 1 -- 172.21.15.86:0/3941795064 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5645627f7610 msgr2=0x56456280d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:43.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.512+0000 7f2d15200640 1 --2- 172.21.15.86:0/3941795064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645627faed0 0x5645627f20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:43.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.512+0000 7f2d14800640 1 --2- 172.21.15.86:0/3941795064 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5645627f7610 0x56456280d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:43.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.512+0000 7f2d14800640 1 -- 172.21.15.86:0/3941795064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645627faed0 msgr2=0x5645627f20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:43.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.512+0000 7f2d14800640 1 --2- 172.21.15.86:0/3941795064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5645627faed0 0x5645627f20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:43.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.512+0000 7f2d14800640 1 -- 172.21.15.86:0/3941795064 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645627696a0 con 0x564562784e30 2024-04-20T21:19:43.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.512+0000 7f2d14800640 1 --2- 172.21.15.86:0/3941795064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564562784e30 0x56456276bff0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f2d0400a470 tx=0x7f2d0400a930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:43.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.513+0000 7f2d11600640 1 -- 172.21.15.86:0/3941795064 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d04015020 con 0x564562784e30 2024-04-20T21:19:43.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.513+0000 7f2d11600640 1 -- 172.21.15.86:0/3941795064 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d04011890 con 0x564562784e30 2024-04-20T21:19:43.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.513+0000 7f2d11600640 1 -- 172.21.15.86:0/3941795064 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d04011c40 con 0x564562784e30 2024-04-20T21:19:43.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.513+0000 7f2d17e41f40 1 -- 172.21.15.86:0/3941795064 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5645627eb300 con 0x564562784e30 2024-04-20T21:19:43.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.513+0000 7f2d17e41f40 1 -- 172.21.15.86:0/3941795064 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645627eb8c0 con 0x564562784e30 2024-04-20T21:19:43.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.515+0000 7f2d11600640 1 -- 172.21.15.86:0/3941795064 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2d04018020 con 0x564562784e30 2024-04-20T21:19:43.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.516+0000 7f2d11600640 1 --2- 172.21.15.86:0/3941795064 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2cf406b2d0 0x7f2cf406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:43.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.516+0000 7f2d11600640 1 -- 172.21.15.86:0/3941795064 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f2d0408ae10 con 0x564562784e30 2024-04-20T21:19:43.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.516+0000 7f2d13e00640 1 --2- 172.21.15.86:0/3941795064 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2cf406b2d0 0x7f2cf406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:43.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.516+0000 7f2d17e41f40 1 --2- 172.21.15.86:0/3941795064 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x56456280dcc0 0x56456280eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:43.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.516+0000 7f2d17e41f40 1 -- 172.21.15.86:0/3941795064 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:1 5.36 5:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x5645627ebde0 con 0x56456280dcc0 2024-04-20T21:19:43.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.517+0000 7f2d15200640 1 --2- 172.21.15.86:0/3941795064 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x56456280dcc0 0x56456280eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:43.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.517+0000 7f2d13e00640 1 --2- 172.21.15.86:0/3941795064 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2cf406b2d0 0x7f2cf406d790 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x564562739440 tx=0x7f2d0000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:43.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.517+0000 7f2d15200640 1 --2- 172.21.15.86:0/3941795064 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x56456280dcc0 0x56456280eb70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:43.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.517+0000 7f2d15200640 1 -- 172.21.15.86:0/3941795064 <== osd.1 v2:172.21.15.86:6817/3149564961 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr (9) out=9b] v0'0 uv17 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7f2d0c05a040 con 0x56456280dcc0 2024-04-20T21:19:43.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.518+0000 7f2d17e41f40 1 -- 172.21.15.86:0/3941795064 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x56456280dcc0 msgr2=0x56456280eb70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:43.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.518+0000 7f2d17e41f40 1 --2- 172.21.15.86:0/3941795064 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x56456280dcc0 0x56456280eb70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:43.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.518+0000 7f2d17e41f40 1 -- 172.21.15.86:0/3941795064 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2cf406b2d0 msgr2=0x7f2cf406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:43.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.518+0000 7f2d17e41f40 1 --2- 172.21.15.86:0/3941795064 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2cf406b2d0 0x7f2cf406d790 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x564562739440 tx=0x7f2d0000b040 comp rx=0 tx=0).stop 2024-04-20T21:19:43.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.518+0000 7f2d17e41f40 1 -- 172.21.15.86:0/3941795064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564562784e30 msgr2=0x56456276bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:43.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.518+0000 7f2d17e41f40 1 --2- 172.21.15.86:0/3941795064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564562784e30 0x56456276bff0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f2d0400a470 tx=0x7f2d0400a930 comp rx=0 tx=0).stop 2024-04-20T21:19:43.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.518+0000 7f2d15200640 1 -- 172.21.15.86:0/3941795064 reap_dead start 2024-04-20T21:19:43.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.518+0000 7f2d17e41f40 1 -- 172.21.15.86:0/3941795064 shutdown_connections 2024-04-20T21:19:43.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.518+0000 7f2d17e41f40 1 -- 172.21.15.86:0/3941795064 >> 172.21.15.86:0/3941795064 conn(0x56456273f950 msgr2=0x56456276c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:43.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.518+0000 7f2d17e41f40 1 -- 172.21.15.86:0/3941795064 shutdown_connections 2024-04-20T21:19:43.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.518+0000 7f2d17e41f40 1 -- 172.21.15.86:0/3941795064 wait complete. 2024-04-20T21:19:43.521 INFO:teuthology.orchestra.run.smithi086.stdout:mytag 2024-04-20T21:19:43.522 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-04-20T21:19:43.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.618+0000 7f69fa60ff40 1 Processor -- start 2024-04-20T21:19:43.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.618+0000 7f69fa60ff40 1 -- start start 2024-04-20T21:19:43.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.619+0000 7f69fa60ff40 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d8f0c35e30 0x55d8f0cbfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:43.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.619+0000 7f69fa60ff40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8f0ca8610 0x55d8f0ca7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:43.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.619+0000 7f69fa60ff40 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8f0cabed0 0x55d8f0d57540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:43.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.619+0000 7f69fa60ff40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d8f0c37cf0 con 0x55d8f0ca8610 2024-04-20T21:19:43.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.619+0000 7f69fa60ff40 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d8f0c37a50 con 0x55d8f0cabed0 2024-04-20T21:19:43.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.619+0000 7f69fa60ff40 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55d8f0c37f90 con 0x55d8f0c35e30 2024-04-20T21:19:43.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.619+0000 7f69f6200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8f0ca8610 0x55d8f0ca7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:43.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.619+0000 7f69f6200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8f0ca8610 0x55d8f0ca7aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50598/0 (socket says 172.21.15.86:50598) 2024-04-20T21:19:43.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.619+0000 7f69f6200640 1 -- 172.21.15.86:0/4196555151 learned_addr learned my addr 172.21.15.86:0/4196555151 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:43.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.619+0000 7f69f6c00640 1 --2- 172.21.15.86:0/4196555151 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d8f0c35e30 0x55d8f0cbfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:43.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.619+0000 7f69f6200640 1 -- 172.21.15.86:0/4196555151 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d8f0c35e30 msgr2=0x55d8f0cbfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:43.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.619+0000 7f69f6200640 1 --2- 172.21.15.86:0/4196555151 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d8f0c35e30 0x55d8f0cbfe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:43.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.619+0000 7f69f6200640 1 -- 172.21.15.86:0/4196555151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8f0cabed0 msgr2=0x55d8f0d57540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:43.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.619+0000 7f69f6200640 1 --2- 172.21.15.86:0/4196555151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8f0cabed0 0x55d8f0d57540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:43.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.619+0000 7f69f6200640 1 -- 172.21.15.86:0/4196555151 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8f0b16b30 con 0x55d8f0ca8610 2024-04-20T21:19:43.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.620+0000 7f69f6200640 1 --2- 172.21.15.86:0/4196555151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8f0ca8610 0x55d8f0ca7aa0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f69e40443c0 tx=0x7f69e404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d6bec657b2b0824 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:43.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.620+0000 7f69f4e00640 1 -- 172.21.15.86:0/4196555151 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e4057040 con 0x55d8f0ca8610 2024-04-20T21:19:43.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.620+0000 7f69f4e00640 1 -- 172.21.15.86:0/4196555151 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69e4063070 con 0x55d8f0ca8610 2024-04-20T21:19:43.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.620+0000 7f69f4e00640 1 -- 172.21.15.86:0/4196555151 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e405d8d0 con 0x55d8f0ca8610 2024-04-20T21:19:43.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.620+0000 7f69fa60ff40 1 -- 172.21.15.86:0/4196555151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8f0ca8610 msgr2=0x55d8f0ca7aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:43.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.620+0000 7f69fa60ff40 1 --2- 172.21.15.86:0/4196555151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8f0ca8610 0x55d8f0ca7aa0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f69e40443c0 tx=0x7f69e404c000 comp rx=0 tx=0).stop 2024-04-20T21:19:43.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.621+0000 7f69fa60ff40 1 -- 172.21.15.86:0/4196555151 shutdown_connections 2024-04-20T21:19:43.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.621+0000 7f69fa60ff40 1 --2- 172.21.15.86:0/4196555151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8f0cabed0 0x55d8f0d57540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:43.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.621+0000 7f69fa60ff40 1 --2- 172.21.15.86:0/4196555151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8f0ca8610 0x55d8f0ca7aa0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:43.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.621+0000 7f69fa60ff40 1 --2- 172.21.15.86:0/4196555151 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d8f0c35e30 0x55d8f0cbfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:43.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.621+0000 7f69fa60ff40 1 -- 172.21.15.86:0/4196555151 >> 172.21.15.86:0/4196555151 conn(0x55d8f0bf0950 msgr2=0x55d8f0c351e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:43.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.621+0000 7f69fa60ff40 1 -- 172.21.15.86:0/4196555151 shutdown_connections 2024-04-20T21:19:43.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.621+0000 7f69fa60ff40 1 -- 172.21.15.86:0/4196555151 wait complete. 2024-04-20T21:19:43.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.621+0000 7f69fa60ff40 1 Processor -- start 2024-04-20T21:19:43.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.622+0000 7f69fa60ff40 1 -- start start 2024-04-20T21:19:43.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.622+0000 7f69fa60ff40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8f0c35e30 0x55d8f0cccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:43.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.622+0000 7f69fa60ff40 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d8f0ca8610 0x55d8f0c1d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:43.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.622+0000 7f69fa60ff40 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8f0cabed0 0x55d8f0ca55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:43.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.622+0000 7f69fa60ff40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d8f0cc5b10 con 0x55d8f0c35e30 2024-04-20T21:19:43.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.622+0000 7f69fa60ff40 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d8f0cc2f90 con 0x55d8f0cabed0 2024-04-20T21:19:43.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.622+0000 7f69fa60ff40 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55d8f0cab6c0 con 0x55d8f0ca8610 2024-04-20T21:19:43.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.623+0000 7f69f6c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8f0c35e30 0x55d8f0cccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:43.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.623+0000 7f69f6c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8f0c35e30 0x55d8f0cccdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50602/0 (socket says 172.21.15.86:50602) 2024-04-20T21:19:43.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.623+0000 7f69f6c00640 1 -- 172.21.15.86:0/3968070771 learned_addr learned my addr 172.21.15.86:0/3968070771 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:43.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.623+0000 7f69f6200640 1 --2- 172.21.15.86:0/3968070771 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d8f0ca8610 0x55d8f0c1d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:43.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.623+0000 7f69f6c00640 1 -- 172.21.15.86:0/3968070771 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d8f0ca8610 msgr2=0x55d8f0c1d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:43.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.623+0000 7f69f6c00640 1 --2- 172.21.15.86:0/3968070771 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d8f0ca8610 0x55d8f0c1d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:43.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.623+0000 7f69f6c00640 1 -- 172.21.15.86:0/3968070771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8f0cabed0 msgr2=0x55d8f0ca55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:43.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.623+0000 7f69f6c00640 1 --2- 172.21.15.86:0/3968070771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d8f0cabed0 0x55d8f0ca55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:43.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.623+0000 7f69f6c00640 1 -- 172.21.15.86:0/3968070771 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8f0ca7110 con 0x55d8f0c35e30 2024-04-20T21:19:43.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.624+0000 7f69f6c00640 1 --2- 172.21.15.86:0/3968070771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8f0c35e30 0x55d8f0cccdf0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f69e8008480 tx=0x7f69e8008940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:43.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.624+0000 7f69ef400640 1 -- 172.21.15.86:0/3968070771 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e800b070 con 0x55d8f0c35e30 2024-04-20T21:19:43.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.624+0000 7f69ef400640 1 -- 172.21.15.86:0/3968070771 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69e8017070 con 0x55d8f0c35e30 2024-04-20T21:19:43.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.624+0000 7f69ef400640 1 -- 172.21.15.86:0/3968070771 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e800d030 con 0x55d8f0c35e30 2024-04-20T21:19:43.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.624+0000 7f69fa60ff40 1 -- 172.21.15.86:0/3968070771 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d8f0ca73f0 con 0x55d8f0c35e30 2024-04-20T21:19:43.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.624+0000 7f69fa60ff40 1 -- 172.21.15.86:0/3968070771 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8f0cc7810 con 0x55d8f0c35e30 2024-04-20T21:19:43.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.626+0000 7f69ef400640 1 -- 172.21.15.86:0/3968070771 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f69e8012810 con 0x55d8f0c35e30 2024-04-20T21:19:43.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.627+0000 7f69ef400640 1 --2- 172.21.15.86:0/3968070771 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f69d806b2d0 0x7f69d806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:43.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.627+0000 7f69ef400640 1 -- 172.21.15.86:0/3968070771 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f69e808a390 con 0x55d8f0c35e30 2024-04-20T21:19:43.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.627+0000 7f69f6200640 1 --2- 172.21.15.86:0/3968070771 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f69d806b2d0 0x7f69d806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:43.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.627+0000 7f69fa60ff40 1 --2- 172.21.15.86:0/3968070771 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x55d8f0cc7d30 0x55d8f0cc8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:43.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.627+0000 7f69fa60ff40 1 -- 172.21.15.86:0/3968070771 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:1 5.19 5:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55d8f0cc86c0 con 0x55d8f0cc7d30 2024-04-20T21:19:43.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.628+0000 7f69f7600640 1 --2- 172.21.15.86:0/3968070771 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x55d8f0cc7d30 0x55d8f0cc8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:43.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.628+0000 7f69f6200640 1 --2- 172.21.15.86:0/3968070771 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f69d806b2d0 0x7f69d806d790 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55d8f0c37680 tx=0x7f69e404f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:43.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.628+0000 7f69f7600640 1 --2- 172.21.15.86:0/3968070771 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x55d8f0cc7d30 0x55d8f0cc8180 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:43.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.629+0000 7f69f7600640 1 -- 172.21.15.86:0/3968070771 <== osd.1 v2:172.21.15.86:6817/3149564961 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (9) out=9b] v0'0 uv12 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7f69f000a500 con 0x55d8f0cc7d30 2024-04-20T21:19:43.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.629+0000 7f69fa60ff40 1 -- 172.21.15.86:0/3968070771 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x55d8f0cc7d30 msgr2=0x55d8f0cc8180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:43.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.629+0000 7f69fa60ff40 1 --2- 172.21.15.86:0/3968070771 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x55d8f0cc7d30 0x55d8f0cc8180 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:43.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.629+0000 7f69fa60ff40 1 -- 172.21.15.86:0/3968070771 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f69d806b2d0 msgr2=0x7f69d806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:43.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.629+0000 7f69fa60ff40 1 --2- 172.21.15.86:0/3968070771 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f69d806b2d0 0x7f69d806d790 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55d8f0c37680 tx=0x7f69e404f000 comp rx=0 tx=0).stop 2024-04-20T21:19:43.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.629+0000 7f69fa60ff40 1 -- 172.21.15.86:0/3968070771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8f0c35e30 msgr2=0x55d8f0cccdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:43.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.629+0000 7f69fa60ff40 1 --2- 172.21.15.86:0/3968070771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8f0c35e30 0x55d8f0cccdf0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f69e8008480 tx=0x7f69e8008940 comp rx=0 tx=0).stop 2024-04-20T21:19:43.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.629+0000 7f69f7600640 1 -- 172.21.15.86:0/3968070771 reap_dead start 2024-04-20T21:19:43.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.629+0000 7f69fa60ff40 1 -- 172.21.15.86:0/3968070771 shutdown_connections 2024-04-20T21:19:43.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.629+0000 7f69fa60ff40 1 -- 172.21.15.86:0/3968070771 >> 172.21.15.86:0/3968070771 conn(0x55d8f0bf0950 msgr2=0x55d8f0cc3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:43.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.629+0000 7f69fa60ff40 1 -- 172.21.15.86:0/3968070771 shutdown_connections 2024-04-20T21:19:43.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.629+0000 7f69fa60ff40 1 -- 172.21.15.86:0/3968070771 wait complete. 2024-04-20T21:19:43.632 INFO:teuthology.orchestra.run.smithi086.stdout:mytag 2024-04-20T21:19:43.633 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-04-20T21:19:43.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.718+0000 7efc727e9f40 1 Processor -- start 2024-04-20T21:19:43.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.718+0000 7efc727e9f40 1 -- start start 2024-04-20T21:19:43.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.719+0000 7efc727e9f40 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc540030f0 0x55a8189ecf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:43.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.719+0000 7efc727e9f40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a818a07960 0x55a81895f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:43.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.719+0000 7efc727e9f40 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a81895fe20 0x55a818a07e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:43.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.719+0000 7efc727e9f40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a8188d5cc0 con 0x55a818a07960 2024-04-20T21:19:43.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.719+0000 7efc727e9f40 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a8188d9d00 con 0x55a81895fe20 2024-04-20T21:19:43.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.719+0000 7efc727e9f40 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55a81897a7a0 con 0x7efc540030f0 2024-04-20T21:19:43.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.719+0000 7efc6f000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a818a07960 0x55a81895f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:43.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.719+0000 7efc6f000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a818a07960 0x55a81895f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50634/0 (socket says 172.21.15.86:50634) 2024-04-20T21:19:43.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.719+0000 7efc6f000640 1 -- 172.21.15.86:0/1975773162 learned_addr learned my addr 172.21.15.86:0/1975773162 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:43.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.719+0000 7efc70400640 1 --2- 172.21.15.86:0/1975773162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a81895fe20 0x55a818a07e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:43.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.720+0000 7efc6fa00640 1 --2- 172.21.15.86:0/1975773162 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc540030f0 0x55a8189ecf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:43.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.720+0000 7efc6f000640 1 -- 172.21.15.86:0/1975773162 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc540030f0 msgr2=0x55a8189ecf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:43.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.720+0000 7efc6f000640 1 --2- 172.21.15.86:0/1975773162 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc540030f0 0x55a8189ecf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:43.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.720+0000 7efc6f000640 1 -- 172.21.15.86:0/1975773162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a81895fe20 msgr2=0x55a818a07e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:43.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.720+0000 7efc6f000640 1 --2- 172.21.15.86:0/1975773162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a81895fe20 0x55a818a07e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:43.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.720+0000 7efc6f000640 1 -- 172.21.15.86:0/1975773162 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a818a0d490 con 0x55a818a07960 2024-04-20T21:19:43.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.720+0000 7efc6f000640 1 --2- 172.21.15.86:0/1975773162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a818a07960 0x55a81895f8e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7efc58000c00 tx=0x7efc5804c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cd0ad91bd2451d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:43.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.720+0000 7efc6dc00640 1 -- 172.21.15.86:0/1975773162 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc58061070 con 0x55a818a07960 2024-04-20T21:19:43.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.720+0000 7efc6dc00640 1 -- 172.21.15.86:0/1975773162 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc58052660 con 0x55a818a07960 2024-04-20T21:19:43.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.720+0000 7efc6dc00640 1 -- 172.21.15.86:0/1975773162 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc58052a50 con 0x55a818a07960 2024-04-20T21:19:43.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.721+0000 7efc727e9f40 1 -- 172.21.15.86:0/1975773162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a818a07960 msgr2=0x55a81895f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:43.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.721+0000 7efc727e9f40 1 --2- 172.21.15.86:0/1975773162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a818a07960 0x55a81895f8e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7efc58000c00 tx=0x7efc5804c070 comp rx=0 tx=0).stop 2024-04-20T21:19:43.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.721+0000 7efc727e9f40 1 -- 172.21.15.86:0/1975773162 shutdown_connections 2024-04-20T21:19:43.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.721+0000 7efc727e9f40 1 --2- 172.21.15.86:0/1975773162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a81895fe20 0x55a818a07e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:43.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.721+0000 7efc727e9f40 1 --2- 172.21.15.86:0/1975773162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a818a07960 0x55a81895f8e0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:43.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.721+0000 7efc727e9f40 1 --2- 172.21.15.86:0/1975773162 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc540030f0 0x55a8189ecf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:43.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.721+0000 7efc727e9f40 1 -- 172.21.15.86:0/1975773162 >> 172.21.15.86:0/1975773162 conn(0x55a818933950 msgr2=0x55a81895eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:43.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.721+0000 7efc727e9f40 1 -- 172.21.15.86:0/1975773162 shutdown_connections 2024-04-20T21:19:43.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.721+0000 7efc727e9f40 1 -- 172.21.15.86:0/1975773162 wait complete. 2024-04-20T21:19:43.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.722+0000 7efc727e9f40 1 Processor -- start 2024-04-20T21:19:43.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.722+0000 7efc727e9f40 1 -- start start 2024-04-20T21:19:43.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.723+0000 7efc727e9f40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a81895fe20 0x55a8189e7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:43.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.723+0000 7efc727e9f40 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a818a07960 0x55a818a01b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:43.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.723+0000 7efc727e9f40 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc540030f0 0x55a818a05f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:43.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.723+0000 7efc727e9f40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a818960670 con 0x55a81895fe20 2024-04-20T21:19:43.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.723+0000 7efc727e9f40 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a818a0dc20 con 0x55a818a07960 2024-04-20T21:19:43.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.723+0000 7efc727e9f40 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55a818960910 con 0x7efc540030f0 2024-04-20T21:19:43.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.723+0000 7efc6fa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a81895fe20 0x55a8189e7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:43.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.723+0000 7efc6fa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a81895fe20 0x55a8189e7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50650/0 (socket says 172.21.15.86:50650) 2024-04-20T21:19:43.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.723+0000 7efc6fa00640 1 -- 172.21.15.86:0/217446379 learned_addr learned my addr 172.21.15.86:0/217446379 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:43.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.723+0000 7efc6f000640 1 --2- 172.21.15.86:0/217446379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a818a07960 0x55a818a01b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:43.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.723+0000 7efc70400640 1 --2- 172.21.15.86:0/217446379 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc540030f0 0x55a818a05f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:43.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.723+0000 7efc6fa00640 1 -- 172.21.15.86:0/217446379 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc540030f0 msgr2=0x55a818a05f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:43.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.723+0000 7efc6fa00640 1 --2- 172.21.15.86:0/217446379 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc540030f0 0x55a818a05f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:43.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.724+0000 7efc6fa00640 1 -- 172.21.15.86:0/217446379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a818a07960 msgr2=0x55a818a01b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:43.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.724+0000 7efc6fa00640 1 --2- 172.21.15.86:0/217446379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a818a07960 0x55a818a01b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:43.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.724+0000 7efc6fa00640 1 -- 172.21.15.86:0/217446379 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8189e6f30 con 0x55a81895fe20 2024-04-20T21:19:43.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.724+0000 7efc6fa00640 1 --2- 172.21.15.86:0/217446379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a81895fe20 0x55a8189e7dd0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7efc6800d420 tx=0x7efc6800d820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:43.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.724+0000 7efc67e00640 1 -- 172.21.15.86:0/217446379 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc68018070 con 0x55a81895fe20 2024-04-20T21:19:43.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.724+0000 7efc67e00640 1 -- 172.21.15.86:0/217446379 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc68003650 con 0x55a81895fe20 2024-04-20T21:19:43.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.724+0000 7efc67e00640 1 -- 172.21.15.86:0/217446379 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc680062e0 con 0x55a81895fe20 2024-04-20T21:19:43.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.724+0000 7efc727e9f40 1 -- 172.21.15.86:0/217446379 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a818a072a0 con 0x55a81895fe20 2024-04-20T21:19:43.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.724+0000 7efc727e9f40 1 -- 172.21.15.86:0/217446379 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a818a06f30 con 0x55a81895fe20 2024-04-20T21:19:43.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.727+0000 7efc67e00640 1 -- 172.21.15.86:0/217446379 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efc68003a20 con 0x55a81895fe20 2024-04-20T21:19:43.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.727+0000 7efc67e00640 1 --2- 172.21.15.86:0/217446379 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efc4c06b300 0x7efc4c06d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:43.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.728+0000 7efc67e00640 1 -- 172.21.15.86:0/217446379 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7efc6808a270 con 0x55a81895fe20 2024-04-20T21:19:43.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.728+0000 7efc6f000640 1 --2- 172.21.15.86:0/217446379 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efc4c06b300 0x7efc4c06d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:43.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.728+0000 7efc727e9f40 1 --2- 172.21.15.86:0/217446379 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x55a818a020a0 0x55a8189ea330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:43.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.728+0000 7efc727e9f40 1 -- 172.21.15.86:0/217446379 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:1 5.12 5:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) -- 0x55a8189e89d0 con 0x55a818a020a0 2024-04-20T21:19:43.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.728+0000 7efc70400640 1 --2- 172.21.15.86:0/217446379 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x55a818a020a0 0x55a8189ea330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:43.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.729+0000 7efc6f000640 1 --2- 172.21.15.86:0/217446379 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efc4c06b300 0x7efc4c06d7c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7efc5805e000 tx=0x7efc5804e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:43.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.729+0000 7efc70400640 1 --2- 172.21.15.86:0/217446379 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x55a818a020a0 0x55a8189ea330 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:43.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.729+0000 7efc70400640 1 -- 172.21.15.86:0/217446379 <== osd.1 v2:172.21.15.86:6817/3149564961 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) ==== 164+0+0 (crc 0 0 0) 0x7efc600073f0 con 0x55a818a020a0 2024-04-20T21:19:43.729 INFO:teuthology.orchestra.run.smithi086.stderr:error getting xattr cephfs_data/10000000002.00000000/scrub_tag: (61) No data available 2024-04-20T21:19:43.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.729+0000 7efc727e9f40 1 -- 172.21.15.86:0/217446379 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x55a818a020a0 msgr2=0x55a8189ea330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:43.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.729+0000 7efc727e9f40 1 --2- 172.21.15.86:0/217446379 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x55a818a020a0 0x55a8189ea330 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:43.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.729+0000 7efc727e9f40 1 -- 172.21.15.86:0/217446379 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efc4c06b300 msgr2=0x7efc4c06d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:43.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.729+0000 7efc727e9f40 1 --2- 172.21.15.86:0/217446379 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efc4c06b300 0x7efc4c06d7c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7efc5805e000 tx=0x7efc5804e000 comp rx=0 tx=0).stop 2024-04-20T21:19:43.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.729+0000 7efc727e9f40 1 -- 172.21.15.86:0/217446379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a81895fe20 msgr2=0x55a8189e7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:43.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.729+0000 7efc727e9f40 1 --2- 172.21.15.86:0/217446379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a81895fe20 0x55a8189e7dd0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7efc6800d420 tx=0x7efc6800d820 comp rx=0 tx=0).stop 2024-04-20T21:19:43.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.730+0000 7efc70400640 1 -- 172.21.15.86:0/217446379 reap_dead start 2024-04-20T21:19:43.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.730+0000 7efc727e9f40 1 -- 172.21.15.86:0/217446379 shutdown_connections 2024-04-20T21:19:43.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.730+0000 7efc727e9f40 1 -- 172.21.15.86:0/217446379 >> 172.21.15.86:0/217446379 conn(0x55a818933950 msgr2=0x55a81895eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:43.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.730+0000 7efc727e9f40 1 -- 172.21.15.86:0/217446379 shutdown_connections 2024-04-20T21:19:43.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:43.730+0000 7efc727e9f40 1 -- 172.21.15.86:0/217446379 wait complete. 2024-04-20T21:19:43.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:19:43.733 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-20T21:19:43.734 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T21:19:43.833 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T21:19:43.942 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-20T21:19:43.942 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T21:19:43.943 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:19:43.943 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:19:43.999 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052731,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248996k,nr_inodes=812249,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T21:19:43.999 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-20T21:19:44.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:19:44.000 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-20T21:19:44.054 INFO:teuthology.orchestra.run.smithi086.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-20T21:19:44.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:19:44.055 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T21:19:44.055 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:19:44.055 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:19:44.113 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T21:19:44.113 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2024-04-20T21:19:44.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:19:44.113 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-20T21:19:44.161 INFO:teuthology.orchestra.run.smithi144.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-20T21:19:44.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:19:44.163 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T21:19:44.163 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:19:44.163 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:19:44.220 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052731,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248996k,nr_inodes=812249,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T21:19:44.220 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-20T21:19:44.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:19:44.220 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-20T21:19:44.271 INFO:teuthology.orchestra.run.smithi086.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-20T21:19:44.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:19:44.272 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T21:19:44.272 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:19:44.272 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:19:44.329 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T21:19:44.329 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2024-04-20T21:19:44.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:19:44.329 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-20T21:19:44.380 INFO:teuthology.orchestra.run.smithi144.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-20T21:19:44.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:19:44.381 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:19:44.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.590+0000 7f6b93200640 1 Processor -- start 2024-04-20T21:19:44.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.591+0000 7f6b93200640 1 -- start start 2024-04-20T21:19:44.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.592+0000 7f6b93200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b8c1629e0 0x7f6b8c0b3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:44.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.592+0000 7f6b93200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b8c06e640 con 0x7f6b8c06f6e0 2024-04-20T21:19:44.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.592+0000 7f6b93200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b8c06e920 con 0x7f6b8c162dc0 2024-04-20T21:19:44.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.592+0000 7f6b93200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6b8c06ec00 con 0x7f6b8c1629e0 2024-04-20T21:19:44.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.593+0000 7f6b92800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f6b8c06f6e0 0x7f6b8c0ad7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:37808/0 (socket says 172.21.15.86:37808) 2024-04-20T21:19:44.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.593+0000 7f6b92800640 1 -- 172.21.15.86:0/2832188086 learned_addr learned my addr 172.21.15.86:0/2832188086 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:19:44.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.593+0000 7f6b91e00640 1 --2- 172.21.15.86:0/2832188086 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b8c1629e0 0x7f6b8c0b3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:44.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.593+0000 7f6b90a00640 1 -- 172.21.15.86:0/2832188086 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1527985000 0 0) 0x7f6b8c06e640 con 0x7f6b8c06f6e0 2024-04-20T21:19:44.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.593+0000 7f6b90a00640 1 -- 172.21.15.86:0/2832188086 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6b74003520 con 0x7f6b8c06f6e0 2024-04-20T21:19:44.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.593+0000 7f6b90a00640 1 -- 172.21.15.86:0/2832188086 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 286139786 0 0) 0x7f6b74003520 con 0x7f6b8c06f6e0 2024-04-20T21:19:44.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.593+0000 7f6b90a00640 1 -- 172.21.15.86:0/2832188086 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b8c1629e0 msgr2=0x7f6b8c0b3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:44.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.593+0000 7f6b90a00640 1 --2- 172.21.15.86:0/2832188086 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b8c1629e0 0x7f6b8c0b3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:44.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.593+0000 7f6b90a00640 1 -- 172.21.15.86:0/2832188086 >> v1:172.21.15.144:6789/0 conn(0x7f6b8c162dc0 legacy=0x7f6b8c0ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:44.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.593+0000 7f6b90a00640 1 -- 172.21.15.86:0/2832188086 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b8c06eee0 con 0x7f6b8c06f6e0 2024-04-20T21:19:44.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.594+0000 7f6b90a00640 1 -- 172.21.15.86:0/2832188086 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f6b88001a10 con 0x7f6b8c06f6e0 2024-04-20T21:19:44.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.594+0000 7f6b90a00640 1 -- 172.21.15.86:0/2832188086 >> v1:172.21.15.86:6789/0 conn(0x7f6b8c06f6e0 legacy=0x7f6b8c0ad7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:44.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.594+0000 7f6b90a00640 1 --2- 172.21.15.86:0/2832188086 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b740039e0 0x7f6b74003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:44.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.594+0000 7f6b90a00640 1 --2- 172.21.15.86:0/2832188086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b74004800 0x7f6b74004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:44.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.594+0000 7f6b90a00640 1 --2- 172.21.15.86:0/2832188086 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b8c1629e0 0x7f6b740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:44.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.594+0000 7f6b90a00640 1 -- 172.21.15.86:0/2832188086 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8c06ec00 con 0x7f6b74004800 2024-04-20T21:19:44.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.594+0000 7f6b90a00640 1 -- 172.21.15.86:0/2832188086 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7400ba70 con 0x7f6b740039e0 2024-04-20T21:19:44.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.594+0000 7f6b90a00640 1 -- 172.21.15.86:0/2832188086 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6b7400bc70 con 0x7f6b8c1629e0 2024-04-20T21:19:44.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.594+0000 7f6b91400640 1 --2- 172.21.15.86:0/2832188086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b74004800 0x7f6b74004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:44.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.594+0000 7f6b91400640 1 -- 172.21.15.86:0/2832188086 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b8c1629e0 msgr2=0x7f6b740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:44.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.594+0000 7f6b91400640 1 --2- 172.21.15.86:0/2832188086 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b8c1629e0 0x7f6b740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:44.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.594+0000 7f6b91400640 1 -- 172.21.15.86:0/2832188086 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b740039e0 msgr2=0x7f6b74003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:44.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.594+0000 7f6b91400640 1 --2- 172.21.15.86:0/2832188086 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b740039e0 0x7f6b74003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:44.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.594+0000 7f6b91400640 1 -- 172.21.15.86:0/2832188086 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b74003520 con 0x7f6b74004800 2024-04-20T21:19:44.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.596+0000 7f6b91400640 1 --2- 172.21.15.86:0/2832188086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b74004800 0x7f6b74004c80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c001720 tx=0x7f6b74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20612dc496836cd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:44.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.596+0000 7f6b90a00640 1 -- 172.21.15.86:0/2832188086 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c001370 con 0x7f6b74004800 2024-04-20T21:19:44.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.596+0000 7f6b90a00640 1 -- 172.21.15.86:0/2832188086 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b7c0052d0 con 0x7f6b74004800 2024-04-20T21:19:44.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.596+0000 7f6b90a00640 1 -- 172.21.15.86:0/2832188086 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c003830 con 0x7f6b74004800 2024-04-20T21:19:44.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.596+0000 7f6b93200640 1 -- 172.21.15.86:0/2832188086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b74004800 msgr2=0x7f6b74004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:44.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.597+0000 7f6b93200640 1 --2- 172.21.15.86:0/2832188086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b74004800 0x7f6b74004c80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c001720 tx=0x7f6b74005830 comp rx=0 tx=0).stop 2024-04-20T21:19:44.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.597+0000 7f6b92800640 1 -- 172.21.15.86:0/2832188086 reap_dead start 2024-04-20T21:19:44.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.597+0000 7f6b93200640 1 -- 172.21.15.86:0/2832188086 shutdown_connections 2024-04-20T21:19:44.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.597+0000 7f6b93200640 1 -- 172.21.15.86:0/2832188086 >> 172.21.15.86:0/2832188086 conn(0x7f6b8c0d0170 msgr2=0x7f6b8c0d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:44.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.597+0000 7f6b93200640 1 -- 172.21.15.86:0/2832188086 shutdown_connections 2024-04-20T21:19:44.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.597+0000 7f6b93200640 1 -- 172.21.15.86:0/2832188086 wait complete. 2024-04-20T21:19:44.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.598+0000 7f6b93200640 1 Processor -- start 2024-04-20T21:19:44.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.599+0000 7f6b93200640 1 -- start start 2024-04-20T21:19:44.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.600+0000 7f6b93200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b74004800 0x7f6b8c147750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:44.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.600+0000 7f6b93200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b8c1eb350 0x7f6b8c146f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:44.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.600+0000 7f6b93200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b8c148e10 0x7f6b8c1497b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:44.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.600+0000 7f6b93200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8c0b3280 con 0x7f6b8c1eb350 2024-04-20T21:19:44.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.600+0000 7f6b93200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8c0b3100 con 0x7f6b8c148e10 2024-04-20T21:19:44.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.600+0000 7f6b93200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6b8c0b3400 con 0x7f6b74004800 2024-04-20T21:19:44.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.600+0000 7f6b91400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b8c1eb350 0x7f6b8c146f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:44.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.600+0000 7f6b91400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b8c1eb350 0x7f6b8c146f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50670/0 (socket says 172.21.15.86:50670) 2024-04-20T21:19:44.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.600+0000 7f6b91400640 1 -- 172.21.15.86:0/1943224973 learned_addr learned my addr 172.21.15.86:0/1943224973 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:44.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.600+0000 7f6b92800640 1 --2- 172.21.15.86:0/1943224973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b8c148e10 0x7f6b8c1497b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:44.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.600+0000 7f6b91e00640 1 --2- 172.21.15.86:0/1943224973 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b74004800 0x7f6b8c147750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:44.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.600+0000 7f6b91400640 1 -- 172.21.15.86:0/1943224973 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b74004800 msgr2=0x7f6b8c147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:44.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.600+0000 7f6b91400640 1 --2- 172.21.15.86:0/1943224973 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b74004800 0x7f6b8c147750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:44.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.600+0000 7f6b91400640 1 -- 172.21.15.86:0/1943224973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b8c148e10 msgr2=0x7f6b8c1497b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:44.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.600+0000 7f6b91400640 1 --2- 172.21.15.86:0/1943224973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b8c148e10 0x7f6b8c1497b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:44.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.600+0000 7f6b91400640 1 -- 172.21.15.86:0/1943224973 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b8c147440 con 0x7f6b8c1eb350 2024-04-20T21:19:44.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.601+0000 7f6b91400640 1 --2- 172.21.15.86:0/1943224973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b8c1eb350 0x7f6b8c146f00 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c008fd0 tx=0x7f6b7c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:44.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.601+0000 7f6b86000640 1 -- 172.21.15.86:0/1943224973 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c041070 con 0x7f6b8c1eb350 2024-04-20T21:19:44.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.601+0000 7f6b86000640 1 -- 172.21.15.86:0/1943224973 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b7c000ed0 con 0x7f6b8c1eb350 2024-04-20T21:19:44.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.601+0000 7f6b93200640 1 -- 172.21.15.86:0/1943224973 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b8c147d70 con 0x7f6b8c1eb350 2024-04-20T21:19:44.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.601+0000 7f6b86000640 1 -- 172.21.15.86:0/1943224973 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c007a30 con 0x7f6b8c1eb350 2024-04-20T21:19:44.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.601+0000 7f6b93200640 1 -- 172.21.15.86:0/1943224973 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b8c148330 con 0x7f6b8c1eb350 2024-04-20T21:19:44.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.602+0000 7f6b93200640 1 -- 172.21.15.86:0/1943224973 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b8c04fcf0 con 0x7f6b8c1eb350 2024-04-20T21:19:44.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.603+0000 7f6b86000640 1 -- 172.21.15.86:0/1943224973 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6b7c044070 con 0x7f6b8c1eb350 2024-04-20T21:19:44.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.605+0000 7f6b86000640 1 --2- 172.21.15.86:0/1943224973 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6b7006c920 0x7f6b7006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:44.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.605+0000 7f6b86000640 1 -- 172.21.15.86:0/1943224973 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f6b7c0ad770 con 0x7f6b8c1eb350 2024-04-20T21:19:44.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.605+0000 7f6b91e00640 1 --2- 172.21.15.86:0/1943224973 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6b7006c920 0x7f6b7006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:44.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.606+0000 7f6b91e00640 1 --2- 172.21.15.86:0/1943224973 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6b7006c920 0x7f6b7006ede0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6b80009970 tx=0x7f6b80008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:44.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.610+0000 7f6b86000640 1 -- 172.21.15.86:0/1943224973 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6b7c07bb90 con 0x7f6b8c1eb350 2024-04-20T21:19:44.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.850+0000 7f6b93200640 1 -- 172.21.15.86:0/1943224973 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b8c1eb7b0 con 0x7f6b8c1eb350 2024-04-20T21:19:44.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.851+0000 7f6b86000640 1 -- 172.21.15.86:0/1943224973 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5205 (secure 0 0 0) 0x7f6b7c07b490 con 0x7f6b8c1eb350 2024-04-20T21:19:44.850 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:19:44.850 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4604,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/1957936003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":1957936003},{"type":"v1","addr":"172.21.15.144:6834","nonce":1957936003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4807,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/1510649389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":1510649389},{"type":"v1","addr":"172.21.15.144:6835","nonce":1510649389}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2180009156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2180009156},{"type":"v1","addr":"172.21.15.86:6836","nonce":2180009156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:19:09.979345+0000","modified":"2024-04-20T21:19:13.064114+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5226":{"gid":5226,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6837/3715801064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3715801064},{"type":"v1","addr":"172.21.15.86:6837","nonce":3715801064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5226,"qdb_cluster":[5226]},"id":2}]} 2024-04-20T21:19:44.851 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 16 2024-04-20T21:19:44.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.853+0000 7f6b93200640 1 -- 172.21.15.86:0/1943224973 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6b7006c920 msgr2=0x7f6b7006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:44.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.853+0000 7f6b93200640 1 --2- 172.21.15.86:0/1943224973 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6b7006c920 0x7f6b7006ede0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6b80009970 tx=0x7f6b80008040 comp rx=0 tx=0).stop 2024-04-20T21:19:44.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.854+0000 7f6b93200640 1 -- 172.21.15.86:0/1943224973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b8c1eb350 msgr2=0x7f6b8c146f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:44.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.854+0000 7f6b93200640 1 --2- 172.21.15.86:0/1943224973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b8c1eb350 0x7f6b8c146f00 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c008fd0 tx=0x7f6b7c001120 comp rx=0 tx=0).stop 2024-04-20T21:19:44.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.854+0000 7f6b93200640 1 -- 172.21.15.86:0/1943224973 shutdown_connections 2024-04-20T21:19:44.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.854+0000 7f6b93200640 1 --2- 172.21.15.86:0/1943224973 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6b7006c920 0x7f6b7006ede0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:44.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.854+0000 7f6b93200640 1 --2- 172.21.15.86:0/1943224973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b8c148e10 0x7f6b8c1497b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:44.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.854+0000 7f6b93200640 1 --2- 172.21.15.86:0/1943224973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b8c1eb350 0x7f6b8c146f00 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:44.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.854+0000 7f6b93200640 1 --2- 172.21.15.86:0/1943224973 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6b74004800 0x7f6b8c147750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:44.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.854+0000 7f6b93200640 1 -- 172.21.15.86:0/1943224973 >> 172.21.15.86:0/1943224973 conn(0x7f6b8c0d0170 msgr2=0x7f6b8c071b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:44.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.854+0000 7f6b93200640 1 -- 172.21.15.86:0/1943224973 shutdown_connections 2024-04-20T21:19:44.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:44.854+0000 7f6b93200640 1 -- 172.21.15.86:0/1943224973 wait complete. 2024-04-20T21:19:44.872 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:19:45.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.071+0000 7fe26ca00640 1 Processor -- start 2024-04-20T21:19:45.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.072+0000 7fe26ca00640 1 -- start start 2024-04-20T21:19:45.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.073+0000 7fe26ca00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe26815ece0 0x7fe26815f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:45.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.073+0000 7fe26ca00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe26815f600 con 0x7fe2680d50c0 2024-04-20T21:19:45.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.073+0000 7fe26ca00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2680d3950 con 0x7fe2680cf9c0 2024-04-20T21:19:45.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.073+0000 7fe26ca00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fe2680cec00 con 0x7fe26815ece0 2024-04-20T21:19:45.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.074+0000 7fe265e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe2680d50c0 0x7fe2680d3100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:37818/0 (socket says 172.21.15.86:37818) 2024-04-20T21:19:45.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.074+0000 7fe265e00640 1 -- 172.21.15.86:0/3686032701 learned_addr learned my addr 172.21.15.86:0/3686032701 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:19:45.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.074+0000 7fe266800640 1 --2- 172.21.15.86:0/3686032701 >> v2:172.21.15.144:3300/0 conn(0x7fe26815ece0 0x7fe26815f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:45.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.074+0000 7fe265400640 1 -- 172.21.15.86:0/3686032701 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3612402427 0 0) 0x7fe26815f600 con 0x7fe2680d50c0 2024-04-20T21:19:45.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.074+0000 7fe265400640 1 -- 172.21.15.86:0/3686032701 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe248003520 con 0x7fe2680d50c0 2024-04-20T21:19:45.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.074+0000 7fe265400640 1 -- 172.21.15.86:0/3686032701 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1764479330 0 0) 0x7fe248003520 con 0x7fe2680d50c0 2024-04-20T21:19:45.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.074+0000 7fe265400640 1 -- 172.21.15.86:0/3686032701 >> v2:172.21.15.144:3300/0 conn(0x7fe26815ece0 msgr2=0x7fe26815f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:45.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.074+0000 7fe265400640 1 --2- 172.21.15.86:0/3686032701 >> v2:172.21.15.144:3300/0 conn(0x7fe26815ece0 0x7fe26815f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:45.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.074+0000 7fe265400640 1 -- 172.21.15.86:0/3686032701 >> v1:172.21.15.144:6789/0 conn(0x7fe2680cf9c0 legacy=0x7fe268160000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:45.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.074+0000 7fe265400640 1 -- 172.21.15.86:0/3686032701 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2680ceee0 con 0x7fe2680d50c0 2024-04-20T21:19:45.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.074+0000 7fe265400640 1 -- 172.21.15.86:0/3686032701 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fe2500019f0 con 0x7fe2680d50c0 2024-04-20T21:19:45.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.074+0000 7fe265400640 1 -- 172.21.15.86:0/3686032701 >> v1:172.21.15.86:6789/0 conn(0x7fe2680d50c0 legacy=0x7fe2680d3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:45.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.075+0000 7fe265400640 1 --2- 172.21.15.86:0/3686032701 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2480039b0 0x7fe248003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:45.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.075+0000 7fe265400640 1 --2- 172.21.15.86:0/3686032701 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe248004800 0x7fe248004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:45.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.075+0000 7fe265400640 1 --2- 172.21.15.86:0/3686032701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2480051c0 0x7fe24800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:45.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.075+0000 7fe265400640 1 -- 172.21.15.86:0/3686032701 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe2680cec00 con 0x7fe2480051c0 2024-04-20T21:19:45.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.075+0000 7fe265400640 1 -- 172.21.15.86:0/3686032701 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe24800c0f0 con 0x7fe2480039b0 2024-04-20T21:19:45.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.075+0000 7fe265400640 1 -- 172.21.15.86:0/3686032701 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe24800c2f0 con 0x7fe248004800 2024-04-20T21:19:45.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.075+0000 7fe267200640 1 --2- 172.21.15.86:0/3686032701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2480051c0 0x7fe24800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:45.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.075+0000 7fe266800640 1 --2- 172.21.15.86:0/3686032701 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2480039b0 0x7fe248003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:45.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.075+0000 7fe267200640 1 -- 172.21.15.86:0/3686032701 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe248004800 msgr2=0x7fe248004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:45.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.075+0000 7fe265e00640 1 --2- 172.21.15.86:0/3686032701 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe248004800 0x7fe248004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:45.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.075+0000 7fe267200640 1 --2- 172.21.15.86:0/3686032701 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe248004800 0x7fe248004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:45.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.075+0000 7fe267200640 1 -- 172.21.15.86:0/3686032701 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2480039b0 msgr2=0x7fe248003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:45.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.075+0000 7fe267200640 1 --2- 172.21.15.86:0/3686032701 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2480039b0 0x7fe248003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:45.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.075+0000 7fe267200640 1 -- 172.21.15.86:0/3686032701 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe248003520 con 0x7fe2480051c0 2024-04-20T21:19:45.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.075+0000 7fe266800640 1 --2- 172.21.15.86:0/3686032701 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2480039b0 0x7fe248003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:45.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.076+0000 7fe267200640 1 -- 172.21.15.86:0/3686032701 reap_dead start 2024-04-20T21:19:45.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.076+0000 7fe267200640 1 --2- 172.21.15.86:0/3686032701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2480051c0 0x7fe24800b9c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fe260003db0 tx=0x7fe248005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa2ded44b1b35f0d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:45.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.077+0000 7fe265400640 1 -- 172.21.15.86:0/3686032701 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe260003980 con 0x7fe2480051c0 2024-04-20T21:19:45.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.077+0000 7fe265400640 1 -- 172.21.15.86:0/3686032701 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe260007240 con 0x7fe2480051c0 2024-04-20T21:19:45.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.077+0000 7fe265400640 1 -- 172.21.15.86:0/3686032701 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe260007630 con 0x7fe2480051c0 2024-04-20T21:19:45.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.077+0000 7fe26ca00640 1 -- 172.21.15.86:0/3686032701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2480051c0 msgr2=0x7fe24800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:45.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.077+0000 7fe26ca00640 1 --2- 172.21.15.86:0/3686032701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2480051c0 0x7fe24800b9c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fe260003db0 tx=0x7fe248005830 comp rx=0 tx=0).stop 2024-04-20T21:19:45.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.077+0000 7fe26ca00640 1 -- 172.21.15.86:0/3686032701 shutdown_connections 2024-04-20T21:19:45.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.077+0000 7fe26ca00640 1 --2- 172.21.15.86:0/3686032701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2480051c0 0x7fe24800b9c0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:45.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.077+0000 7fe26ca00640 1 -- 172.21.15.86:0/3686032701 >> 172.21.15.86:0/3686032701 conn(0x7fe26808bf70 msgr2=0x7fe2680d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:45.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.077+0000 7fe26ca00640 1 -- 172.21.15.86:0/3686032701 shutdown_connections 2024-04-20T21:19:45.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.077+0000 7fe26ca00640 1 -- 172.21.15.86:0/3686032701 wait complete. 2024-04-20T21:19:45.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.079+0000 7fe26ca00640 1 Processor -- start 2024-04-20T21:19:45.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.079+0000 7fe26ca00640 1 -- start start 2024-04-20T21:19:45.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.080+0000 7fe26ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2480051c0 0x7fe26809f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:45.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.080+0000 7fe26ca00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe26809f800 0x7fe2680ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:45.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.080+0000 7fe26ca00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2680cd070 0x7fe2680c8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:45.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.080+0000 7fe26ca00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe26815ea50 con 0x7fe2480051c0 2024-04-20T21:19:45.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.080+0000 7fe26ca00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe2680cf4a0 con 0x7fe26809f800 2024-04-20T21:19:45.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.080+0000 7fe26ca00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe2680cfe70 con 0x7fe2680cd070 2024-04-20T21:19:45.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.080+0000 7fe266800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2480051c0 0x7fe26809f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:45.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.080+0000 7fe266800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2480051c0 0x7fe26809f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50680/0 (socket says 172.21.15.86:50680) 2024-04-20T21:19:45.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.080+0000 7fe266800640 1 -- 172.21.15.86:0/1274037007 learned_addr learned my addr 172.21.15.86:0/1274037007 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:45.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.081+0000 7fe265e00640 1 --2- 172.21.15.86:0/1274037007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe26809f800 0x7fe2680ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:45.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.081+0000 7fe267200640 1 --2- 172.21.15.86:0/1274037007 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2680cd070 0x7fe2680c8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:45.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.081+0000 7fe266800640 1 -- 172.21.15.86:0/1274037007 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2680cd070 msgr2=0x7fe2680c8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:45.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.081+0000 7fe266800640 1 --2- 172.21.15.86:0/1274037007 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2680cd070 0x7fe2680c8a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:45.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.081+0000 7fe266800640 1 -- 172.21.15.86:0/1274037007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe26809f800 msgr2=0x7fe2680ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:45.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.081+0000 7fe266800640 1 --2- 172.21.15.86:0/1274037007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe26809f800 0x7fe2680ccb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:45.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.081+0000 7fe266800640 1 -- 172.21.15.86:0/1274037007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2680cbb00 con 0x7fe2480051c0 2024-04-20T21:19:45.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.081+0000 7fe266800640 1 --2- 172.21.15.86:0/1274037007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2480051c0 0x7fe26809f2c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fe2580108b0 tx=0x7fe258010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:45.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.081+0000 7fe25ea00640 1 -- 172.21.15.86:0/1274037007 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25800f040 con 0x7fe2480051c0 2024-04-20T21:19:45.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.081+0000 7fe25ea00640 1 -- 172.21.15.86:0/1274037007 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe25800eb60 con 0x7fe2480051c0 2024-04-20T21:19:45.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.081+0000 7fe25ea00640 1 -- 172.21.15.86:0/1274037007 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25801c400 con 0x7fe2480051c0 2024-04-20T21:19:45.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.082+0000 7fe26ca00640 1 -- 172.21.15.86:0/1274037007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2680cd500 con 0x7fe2480051c0 2024-04-20T21:19:45.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.082+0000 7fe26ca00640 1 -- 172.21.15.86:0/1274037007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2680cc050 con 0x7fe2480051c0 2024-04-20T21:19:45.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.083+0000 7fe26ca00640 1 -- 172.21.15.86:0/1274037007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe26804fcf0 con 0x7fe2480051c0 2024-04-20T21:19:45.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.084+0000 7fe25ea00640 1 -- 172.21.15.86:0/1274037007 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe25801c5a0 con 0x7fe2480051c0 2024-04-20T21:19:45.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.084+0000 7fe25ea00640 1 --2- 172.21.15.86:0/1274037007 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe24406c920 0x7fe24406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:45.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.085+0000 7fe25ea00640 1 -- 172.21.15.86:0/1274037007 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fe25808c2b0 con 0x7fe2480051c0 2024-04-20T21:19:45.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.085+0000 7fe265e00640 1 --2- 172.21.15.86:0/1274037007 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe24406c920 0x7fe24406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:45.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.086+0000 7fe265e00640 1 --2- 172.21.15.86:0/1274037007 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe24406c920 0x7fe24406ede0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe250001b60 tx=0x7fe250009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:45.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.089+0000 7fe25ea00640 1 -- 172.21.15.86:0/1274037007 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe258018030 con 0x7fe2480051c0 2024-04-20T21:19:45.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.289+0000 7fe26ca00640 1 -- 172.21.15.86:0/1274037007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe26808e320 con 0x7fe2480051c0 2024-04-20T21:19:45.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.292+0000 7fe25ea00640 1 -- 172.21.15.86:0/1274037007 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16669 (secure 0 0 0) 0x7fe25805a080 con 0x7fe2480051c0 2024-04-20T21:19:45.292 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:19:45.292 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":33,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:19:11.559613+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:19:07.732422+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-20T21:19:08.960268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:19:45.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.295+0000 7fe26ca00640 1 -- 172.21.15.86:0/1274037007 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe24406c920 msgr2=0x7fe24406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:45.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.295+0000 7fe26ca00640 1 --2- 172.21.15.86:0/1274037007 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe24406c920 0x7fe24406ede0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe250001b60 tx=0x7fe250009040 comp rx=0 tx=0).stop 2024-04-20T21:19:45.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.295+0000 7fe26ca00640 1 -- 172.21.15.86:0/1274037007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2480051c0 msgr2=0x7fe26809f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:45.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.295+0000 7fe26ca00640 1 --2- 172.21.15.86:0/1274037007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2480051c0 0x7fe26809f2c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fe2580108b0 tx=0x7fe258010d70 comp rx=0 tx=0).stop 2024-04-20T21:19:45.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.295+0000 7fe26ca00640 1 -- 172.21.15.86:0/1274037007 shutdown_connections 2024-04-20T21:19:45.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.295+0000 7fe26ca00640 1 --2- 172.21.15.86:0/1274037007 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe24406c920 0x7fe24406ede0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:45.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.295+0000 7fe26ca00640 1 --2- 172.21.15.86:0/1274037007 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2680cd070 0x7fe2680c8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:45.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.295+0000 7fe26ca00640 1 --2- 172.21.15.86:0/1274037007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe26809f800 0x7fe2680ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:45.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.295+0000 7fe26ca00640 1 --2- 172.21.15.86:0/1274037007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2480051c0 0x7fe26809f2c0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:45.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.295+0000 7fe26ca00640 1 -- 172.21.15.86:0/1274037007 >> 172.21.15.86:0/1274037007 conn(0x7fe26808bf70 msgr2=0x7fe2680d25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:45.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.296+0000 7fe26ca00640 1 -- 172.21.15.86:0/1274037007 shutdown_connections 2024-04-20T21:19:45.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.296+0000 7fe26ca00640 1 -- 172.21.15.86:0/1274037007 wait complete. 2024-04-20T21:19:45.313 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:19:07.732422+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:19:08.960268+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:19:45.313 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:19:45.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.517+0000 7fa994200640 1 Processor -- start 2024-04-20T21:19:45.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.517+0000 7fa994200640 1 -- start start 2024-04-20T21:19:45.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.518+0000 7fa994200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa98c09a9e0 0x7fa98c09ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:45.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.518+0000 7fa994200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa98c1604d0 0x7fa98c160940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:45.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.518+0000 7fa994200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa98c0d15f0 con 0x7fa98c162160 2024-04-20T21:19:45.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.518+0000 7fa994200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa98c09b360 con 0x7fa98c09a9e0 2024-04-20T21:19:45.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.518+0000 7fa994200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa98c0d0550 con 0x7fa98c1604d0 2024-04-20T21:19:45.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.518+0000 7fa990e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa98c09a9e0 0x7fa98c09ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:45.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.519+0000 7fa990e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa98c09a9e0 0x7fa98c09ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50684/0 (socket says 172.21.15.86:50684) 2024-04-20T21:19:45.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.519+0000 7fa990e00640 1 -- 172.21.15.86:0/2662616896 learned_addr learned my addr 172.21.15.86:0/2662616896 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:45.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.519+0000 7fa992200640 1 --2- 172.21.15.86:0/2662616896 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa98c1604d0 0x7fa98c160940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:45.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.519+0000 7fa990e00640 1 -- 172.21.15.86:0/2662616896 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa98c1604d0 msgr2=0x7fa98c160940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:45.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.519+0000 7fa990e00640 1 --2- 172.21.15.86:0/2662616896 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa98c1604d0 0x7fa98c160940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:45.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.519+0000 7fa990e00640 1 -- 172.21.15.86:0/2662616896 >> v1:172.21.15.144:6789/0 conn(0x7fa98c162160 legacy=0x7fa98c06dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:45.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.519+0000 7fa990e00640 1 -- 172.21.15.86:0/2662616896 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa98c0d0830 con 0x7fa98c09a9e0 2024-04-20T21:19:45.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.519+0000 7fa992200640 1 --2- 172.21.15.86:0/2662616896 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa98c1604d0 0x7fa98c160940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:45.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.520+0000 7fa990e00640 1 --2- 172.21.15.86:0/2662616896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa98c09a9e0 0x7fa98c09ae20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa978009870 tx=0x7fa97802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=773df5c52caede26 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:45.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.520+0000 7fa98be00640 1 -- 172.21.15.86:0/2662616896 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa978034070 con 0x7fa98c09a9e0 2024-04-20T21:19:45.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.521+0000 7fa98be00640 1 -- 172.21.15.86:0/2662616896 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa978030840 con 0x7fa98c09a9e0 2024-04-20T21:19:45.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.521+0000 7fa98be00640 1 -- 172.21.15.86:0/2662616896 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa978030c10 con 0x7fa98c09a9e0 2024-04-20T21:19:45.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.521+0000 7fa994200640 1 -- 172.21.15.86:0/2662616896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa98c09a9e0 msgr2=0x7fa98c09ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:45.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.521+0000 7fa994200640 1 --2- 172.21.15.86:0/2662616896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa98c09a9e0 0x7fa98c09ae20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa978009870 tx=0x7fa97802b980 comp rx=0 tx=0).stop 2024-04-20T21:19:45.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.521+0000 7fa994200640 1 -- 172.21.15.86:0/2662616896 shutdown_connections 2024-04-20T21:19:45.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.521+0000 7fa994200640 1 --2- 172.21.15.86:0/2662616896 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa98c1604d0 0x7fa98c160940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:45.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.521+0000 7fa994200640 1 --2- 172.21.15.86:0/2662616896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa98c09a9e0 0x7fa98c09ae20 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:45.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.521+0000 7fa994200640 1 -- 172.21.15.86:0/2662616896 >> 172.21.15.86:0/2662616896 conn(0x7fa98c0d2630 msgr2=0x7fa98c0d2a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:45.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.521+0000 7fa994200640 1 -- 172.21.15.86:0/2662616896 shutdown_connections 2024-04-20T21:19:45.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.521+0000 7fa994200640 1 -- 172.21.15.86:0/2662616896 wait complete. 2024-04-20T21:19:45.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.523+0000 7fa994200640 1 Processor -- start 2024-04-20T21:19:45.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.523+0000 7fa994200640 1 -- start start 2024-04-20T21:19:45.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.524+0000 7fa994200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa98c09a9e0 0x7fa98c0cb310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:45.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.524+0000 7fa994200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa98c1604d0 0x7fa98c0b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:45.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.524+0000 7fa994200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa98c162160 0x7fa98c0b7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:45.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.524+0000 7fa994200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa98c0d10d0 con 0x7fa98c1604d0 2024-04-20T21:19:45.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.524+0000 7fa994200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa98c0d0df0 con 0x7fa98c162160 2024-04-20T21:19:45.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.524+0000 7fa994200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa98c0d0200 con 0x7fa98c09a9e0 2024-04-20T21:19:45.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.524+0000 7fa990e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa98c1604d0 0x7fa98c0b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:45.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.524+0000 7fa990e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa98c1604d0 0x7fa98c0b73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50698/0 (socket says 172.21.15.86:50698) 2024-04-20T21:19:45.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.524+0000 7fa990e00640 1 -- 172.21.15.86:0/681188934 learned_addr learned my addr 172.21.15.86:0/681188934 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:45.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.525+0000 7fa991800640 1 --2- 172.21.15.86:0/681188934 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa98c09a9e0 0x7fa98c0cb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:45.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.525+0000 7fa992200640 1 --2- 172.21.15.86:0/681188934 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa98c162160 0x7fa98c0b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:45.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.525+0000 7fa990e00640 1 -- 172.21.15.86:0/681188934 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa98c09a9e0 msgr2=0x7fa98c0cb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:45.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.525+0000 7fa990e00640 1 --2- 172.21.15.86:0/681188934 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa98c09a9e0 0x7fa98c0cb310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:45.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.525+0000 7fa990e00640 1 -- 172.21.15.86:0/681188934 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa98c162160 msgr2=0x7fa98c0b7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:45.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.525+0000 7fa990e00640 1 --2- 172.21.15.86:0/681188934 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa98c162160 0x7fa98c0b7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:45.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.525+0000 7fa990e00640 1 -- 172.21.15.86:0/681188934 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa98c0cb850 con 0x7fa98c1604d0 2024-04-20T21:19:45.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.525+0000 7fa992200640 1 --2- 172.21.15.86:0/681188934 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa98c162160 0x7fa98c0b7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:45.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.525+0000 7fa990e00640 1 --2- 172.21.15.86:0/681188934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa98c1604d0 0x7fa98c0b73e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa978009870 tx=0x7fa978004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:45.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.526+0000 7fa989600640 1 -- 172.21.15.86:0/681188934 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa978048070 con 0x7fa98c1604d0 2024-04-20T21:19:45.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.526+0000 7fa989600640 1 -- 172.21.15.86:0/681188934 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa978030640 con 0x7fa98c1604d0 2024-04-20T21:19:45.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.526+0000 7fa989600640 1 -- 172.21.15.86:0/681188934 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa978034040 con 0x7fa98c1604d0 2024-04-20T21:19:45.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.526+0000 7fa994200640 1 -- 172.21.15.86:0/681188934 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa98c0cbaf0 con 0x7fa98c1604d0 2024-04-20T21:19:45.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.526+0000 7fa994200640 1 -- 172.21.15.86:0/681188934 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa98c0cc0b0 con 0x7fa98c1604d0 2024-04-20T21:19:45.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.527+0000 7fa994200640 1 -- 172.21.15.86:0/681188934 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa98c0690f0 con 0x7fa98c1604d0 2024-04-20T21:19:45.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.528+0000 7fa989600640 1 -- 172.21.15.86:0/681188934 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa97804b070 con 0x7fa98c1604d0 2024-04-20T21:19:45.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.529+0000 7fa989600640 1 --2- 172.21.15.86:0/681188934 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa96c06c8f0 0x7fa96c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:45.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.529+0000 7fa989600640 1 -- 172.21.15.86:0/681188934 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fa9780adc80 con 0x7fa98c1604d0 2024-04-20T21:19:45.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.529+0000 7fa991800640 1 --2- 172.21.15.86:0/681188934 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa96c06c8f0 0x7fa96c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:45.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.530+0000 7fa991800640 1 --2- 172.21.15.86:0/681188934 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa96c06c8f0 0x7fa96c06edb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa980009f20 tx=0x7fa9800094d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:45.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.533+0000 7fa989600640 1 -- 172.21.15.86:0/681188934 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa97807c0a0 con 0x7fa98c1604d0 2024-04-20T21:19:45.671 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:19:45.679 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:19:45.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.732+0000 7fa994200640 1 -- 172.21.15.86:0/681188934 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa98c0d3420 con 0x7fa98c1604d0 2024-04-20T21:19:45.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.733+0000 7fa989600640 1 -- 172.21.15.86:0/681188934 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) ==== 93+0+10 (secure 0 0 0) 0x7fa97807b9a0 con 0x7fa98c1604d0 2024-04-20T21:19:45.732 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:19:45.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.735+0000 7fa994200640 1 -- 172.21.15.86:0/681188934 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa96c06c8f0 msgr2=0x7fa96c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:45.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.735+0000 7fa994200640 1 --2- 172.21.15.86:0/681188934 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa96c06c8f0 0x7fa96c06edb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa980009f20 tx=0x7fa9800094d0 comp rx=0 tx=0).stop 2024-04-20T21:19:45.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.735+0000 7fa994200640 1 -- 172.21.15.86:0/681188934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa98c1604d0 msgr2=0x7fa98c0b73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:45.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.735+0000 7fa994200640 1 --2- 172.21.15.86:0/681188934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa98c1604d0 0x7fa98c0b73e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa978009870 tx=0x7fa978004730 comp rx=0 tx=0).stop 2024-04-20T21:19:45.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.735+0000 7fa994200640 1 -- 172.21.15.86:0/681188934 shutdown_connections 2024-04-20T21:19:45.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.735+0000 7fa994200640 1 --2- 172.21.15.86:0/681188934 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa96c06c8f0 0x7fa96c06edb0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:45.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.735+0000 7fa994200640 1 --2- 172.21.15.86:0/681188934 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa98c162160 0x7fa98c0b7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:45.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.735+0000 7fa994200640 1 --2- 172.21.15.86:0/681188934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa98c1604d0 0x7fa98c0b73e0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:45.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.735+0000 7fa994200640 1 --2- 172.21.15.86:0/681188934 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa98c09a9e0 0x7fa98c0cb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:45.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.735+0000 7fa994200640 1 -- 172.21.15.86:0/681188934 >> 172.21.15.86:0/681188934 conn(0x7fa98c0d2630 msgr2=0x7fa98c0ce280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:45.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.735+0000 7fa994200640 1 -- 172.21.15.86:0/681188934 shutdown_connections 2024-04-20T21:19:45.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.735+0000 7fa994200640 1 -- 172.21.15.86:0/681188934 wait complete. 2024-04-20T21:19:45.753 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:19:45.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.948+0000 7f1340000640 1 Processor -- start 2024-04-20T21:19:45.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.948+0000 7f1340000640 1 -- start start 2024-04-20T21:19:45.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.950+0000 7f1340000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f133806bbf0 0x7f133806e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:45.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.950+0000 7f1340000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f133806bfd0 0x7f1338145b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:45.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.950+0000 7f1340000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f133815cd40 con 0x7f1338146040 2024-04-20T21:19:45.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.950+0000 7f1340000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f133815d000 con 0x7f133806bfd0 2024-04-20T21:19:45.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.950+0000 7f1340000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f133815d2e0 con 0x7f133806bbf0 2024-04-20T21:19:45.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.950+0000 7f133e000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f1338146040 0x7f13380cd540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:47232/0 (socket says 172.21.15.86:47232) 2024-04-20T21:19:45.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.950+0000 7f133e000640 1 -- 172.21.15.86:0/982964713 learned_addr learned my addr 172.21.15.86:0/982964713 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:19:45.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.951+0000 7f133cc00640 1 --2- 172.21.15.86:0/982964713 >> v2:172.21.15.144:3300/0 conn(0x7f133806bfd0 0x7f1338145b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:45.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.951+0000 7f133d600640 1 --2- 172.21.15.86:0/982964713 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f133806bbf0 0x7f133806e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:45.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.951+0000 7f1337e00640 1 -- 172.21.15.86:0/982964713 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1563255206 0 0) 0x7f133815cd40 con 0x7f1338146040 2024-04-20T21:19:45.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.951+0000 7f1337e00640 1 -- 172.21.15.86:0/982964713 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1320003520 con 0x7f1338146040 2024-04-20T21:19:45.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.951+0000 7f1337e00640 1 -- 172.21.15.86:0/982964713 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3804149156 0 0) 0x7f1320003520 con 0x7f1338146040 2024-04-20T21:19:45.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.951+0000 7f1337e00640 1 -- 172.21.15.86:0/982964713 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f133806bbf0 msgr2=0x7f133806e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:45.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.951+0000 7f1337e00640 1 --2- 172.21.15.86:0/982964713 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f133806bbf0 0x7f133806e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:45.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.951+0000 7f1337e00640 1 -- 172.21.15.86:0/982964713 >> v2:172.21.15.144:3300/0 conn(0x7f133806bfd0 msgr2=0x7f1338145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:45.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.951+0000 7f1337e00640 1 --2- 172.21.15.86:0/982964713 >> v2:172.21.15.144:3300/0 conn(0x7f133806bfd0 0x7f1338145b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:45.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.951+0000 7f1337e00640 1 -- 172.21.15.86:0/982964713 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f133815d5c0 con 0x7f1338146040 2024-04-20T21:19:45.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.951+0000 7f1337e00640 1 -- 172.21.15.86:0/982964713 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f13300019f0 con 0x7f1338146040 2024-04-20T21:19:45.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.951+0000 7f1337e00640 1 -- 172.21.15.86:0/982964713 >> v1:172.21.15.86:6789/0 conn(0x7f1338146040 legacy=0x7f13380cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:45.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.951+0000 7f1337e00640 1 --2- 172.21.15.86:0/982964713 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f133806bbf0 0x7f13200039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:45.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.951+0000 7f1337e00640 1 --2- 172.21.15.86:0/982964713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1320003f30 0x7f1320004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:45.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.951+0000 7f1337e00640 1 --2- 172.21.15.86:0/982964713 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1320004d40 0x7f13200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:45.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.951+0000 7f1337e00640 1 -- 172.21.15.86:0/982964713 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f133815d000 con 0x7f1320003f30 2024-04-20T21:19:45.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.951+0000 7f1337e00640 1 -- 172.21.15.86:0/982964713 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f133815d2e0 con 0x7f1320004d40 2024-04-20T21:19:45.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.951+0000 7f1337e00640 1 -- 172.21.15.86:0/982964713 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f132000b9c0 con 0x7f133806bbf0 2024-04-20T21:19:45.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.952+0000 7f133cc00640 1 --2- 172.21.15.86:0/982964713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1320003f30 0x7f1320004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:45.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.952+0000 7f133cc00640 1 -- 172.21.15.86:0/982964713 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f133806bbf0 msgr2=0x7f13200039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:45.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.952+0000 7f133e000640 1 --2- 172.21.15.86:0/982964713 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1320004d40 0x7f13200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:45.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.952+0000 7f133d600640 1 --2- 172.21.15.86:0/982964713 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f133806bbf0 0x7f13200039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:45.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.952+0000 7f133cc00640 1 --2- 172.21.15.86:0/982964713 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f133806bbf0 0x7f13200039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:45.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.952+0000 7f133cc00640 1 -- 172.21.15.86:0/982964713 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1320004d40 msgr2=0x7f13200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:45.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.952+0000 7f133cc00640 1 --2- 172.21.15.86:0/982964713 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1320004d40 0x7f13200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:45.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.952+0000 7f133cc00640 1 -- 172.21.15.86:0/982964713 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1320003520 con 0x7f1320003f30 2024-04-20T21:19:45.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.952+0000 7f133e000640 1 --2- 172.21.15.86:0/982964713 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1320004d40 0x7f13200051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:45.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.953+0000 7f133cc00640 1 --2- 172.21.15.86:0/982964713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1320003f30 0x7f1320004800 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f1324009ac0 tx=0x7f1320005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=479e0b241363f477 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:45.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.953+0000 7f1337e00640 1 -- 172.21.15.86:0/982964713 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1324039070 con 0x7f1320003f30 2024-04-20T21:19:45.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.953+0000 7f1337e00640 1 -- 172.21.15.86:0/982964713 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f132402c6d0 con 0x7f1320003f30 2024-04-20T21:19:45.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.953+0000 7f1337e00640 1 -- 172.21.15.86:0/982964713 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1324009690 con 0x7f1320003f30 2024-04-20T21:19:45.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.954+0000 7f1340000640 1 -- 172.21.15.86:0/982964713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1320003f30 msgr2=0x7f1320004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:45.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.954+0000 7f1340000640 1 --2- 172.21.15.86:0/982964713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1320003f30 0x7f1320004800 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f1324009ac0 tx=0x7f1320005830 comp rx=0 tx=0).stop 2024-04-20T21:19:45.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.954+0000 7f133e000640 1 -- 172.21.15.86:0/982964713 reap_dead start 2024-04-20T21:19:45.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.954+0000 7f1340000640 1 -- 172.21.15.86:0/982964713 shutdown_connections 2024-04-20T21:19:45.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.954+0000 7f1340000640 1 -- 172.21.15.86:0/982964713 >> 172.21.15.86:0/982964713 conn(0x7f13380b2110 msgr2=0x7f13380b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:45.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.954+0000 7f1340000640 1 -- 172.21.15.86:0/982964713 shutdown_connections 2024-04-20T21:19:45.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.954+0000 7f1340000640 1 -- 172.21.15.86:0/982964713 wait complete. 2024-04-20T21:19:45.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.956+0000 7f1340000640 1 Processor -- start 2024-04-20T21:19:45.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.956+0000 7f1340000640 1 -- start start 2024-04-20T21:19:45.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.957+0000 7f1340000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1320003f30 0x7f13381f5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:45.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.957+0000 7f1340000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13380b3510 0x7f13381f4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:45.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.957+0000 7f1340000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f133813e2b0 0x7f13381f6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:45.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.957+0000 7f1340000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f13380cddd0 con 0x7f133813e2b0 2024-04-20T21:19:45.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.957+0000 7f1340000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f13380cdc50 con 0x7f1320003f30 2024-04-20T21:19:45.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.957+0000 7f1340000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f13380cdf50 con 0x7f13380b3510 2024-04-20T21:19:45.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.957+0000 7f133e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f133813e2b0 0x7f13381f6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:45.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.958+0000 7f133e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f133813e2b0 0x7f13381f6d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39486/0 (socket says 172.21.15.86:39486) 2024-04-20T21:19:45.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.958+0000 7f133e000640 1 -- 172.21.15.86:0/4020659475 learned_addr learned my addr 172.21.15.86:0/4020659475 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:45.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.958+0000 7f133cc00640 1 --2- 172.21.15.86:0/4020659475 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13380b3510 0x7f13381f4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:45.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.958+0000 7f133d600640 1 --2- 172.21.15.86:0/4020659475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1320003f30 0x7f13381f5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:45.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.958+0000 7f133e000640 1 -- 172.21.15.86:0/4020659475 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13380b3510 msgr2=0x7f13381f4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:45.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.958+0000 7f133e000640 1 --2- 172.21.15.86:0/4020659475 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13380b3510 0x7f13381f4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:45.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.958+0000 7f133e000640 1 -- 172.21.15.86:0/4020659475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1320003f30 msgr2=0x7f13381f5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:45.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.958+0000 7f133e000640 1 --2- 172.21.15.86:0/4020659475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1320003f30 0x7f13381f5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:45.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.958+0000 7f133e000640 1 -- 172.21.15.86:0/4020659475 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13380b3990 con 0x7f133813e2b0 2024-04-20T21:19:45.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.958+0000 7f133d600640 1 --2- 172.21.15.86:0/4020659475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1320003f30 0x7f13381f5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:45.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.958+0000 7f133e000640 1 --2- 172.21.15.86:0/4020659475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f133813e2b0 0x7f13381f6d50 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1338073250 tx=0x7f133000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:45.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.959+0000 7f1335600640 1 -- 172.21.15.86:0/4020659475 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f133000de90 con 0x7f133813e2b0 2024-04-20T21:19:45.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.959+0000 7f1335600640 1 -- 172.21.15.86:0/4020659475 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1330004b60 con 0x7f133813e2b0 2024-04-20T21:19:45.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.959+0000 7f1335600640 1 -- 172.21.15.86:0/4020659475 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1330007070 con 0x7f133813e2b0 2024-04-20T21:19:45.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.959+0000 7f1340000640 1 -- 172.21.15.86:0/4020659475 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13380b3c40 con 0x7f133813e2b0 2024-04-20T21:19:45.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.959+0000 7f1340000640 1 -- 172.21.15.86:0/4020659475 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13380b4200 con 0x7f133813e2b0 2024-04-20T21:19:45.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.960+0000 7f1340000640 1 -- 172.21.15.86:0/4020659475 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13380689e0 con 0x7f133813e2b0 2024-04-20T21:19:45.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.961+0000 7f1335600640 1 -- 172.21.15.86:0/4020659475 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f133001b020 con 0x7f133813e2b0 2024-04-20T21:19:45.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.962+0000 7f1335600640 1 --2- 172.21.15.86:0/4020659475 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f131806ca90 0x7f131806ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:45.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.962+0000 7f1335600640 1 -- 172.21.15.86:0/4020659475 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f133008bde0 con 0x7f133813e2b0 2024-04-20T21:19:45.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.962+0000 7f133d600640 1 --2- 172.21.15.86:0/4020659475 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f131806ca90 0x7f131806ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:45.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.963+0000 7f133d600640 1 --2- 172.21.15.86:0/4020659475 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f131806ca90 0x7f131806ef50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f132c009910 tx=0x7f132c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:45.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:45.967+0000 7f1335600640 1 -- 172.21.15.86:0/4020659475 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f133005a170 con 0x7f133813e2b0 2024-04-20T21:19:46.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.167+0000 7f1340000640 1 -- 172.21.15.86:0/4020659475 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f13381408a0 con 0x7f133813e2b0 2024-04-20T21:19:46.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.168+0000 7f1335600640 1 -- 172.21.15.86:0/4020659475 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) ==== 104+0+11 (secure 0 0 0) 0x7f1330059a70 con 0x7f133813e2b0 2024-04-20T21:19:46.167 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-20T21:19:46.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.171+0000 7f1340000640 1 -- 172.21.15.86:0/4020659475 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f131806ca90 msgr2=0x7f131806ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:46.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.171+0000 7f1340000640 1 --2- 172.21.15.86:0/4020659475 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f131806ca90 0x7f131806ef50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f132c009910 tx=0x7f132c008040 comp rx=0 tx=0).stop 2024-04-20T21:19:46.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.171+0000 7f1340000640 1 -- 172.21.15.86:0/4020659475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f133813e2b0 msgr2=0x7f13381f6d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:46.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.171+0000 7f1340000640 1 --2- 172.21.15.86:0/4020659475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f133813e2b0 0x7f13381f6d50 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1338073250 tx=0x7f133000b4a0 comp rx=0 tx=0).stop 2024-04-20T21:19:46.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.171+0000 7f1340000640 1 -- 172.21.15.86:0/4020659475 shutdown_connections 2024-04-20T21:19:46.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.171+0000 7f1340000640 1 --2- 172.21.15.86:0/4020659475 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f131806ca90 0x7f131806ef50 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:46.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.171+0000 7f1340000640 1 --2- 172.21.15.86:0/4020659475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f133813e2b0 0x7f13381f6d50 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:46.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.171+0000 7f1340000640 1 --2- 172.21.15.86:0/4020659475 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13380b3510 0x7f13381f4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:46.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.171+0000 7f1340000640 1 --2- 172.21.15.86:0/4020659475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1320003f30 0x7f13381f5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:46.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.171+0000 7f1340000640 1 -- 172.21.15.86:0/4020659475 >> 172.21.15.86:0/4020659475 conn(0x7f13380b2110 msgr2=0x7f133815f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:46.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.171+0000 7f1340000640 1 -- 172.21.15.86:0/4020659475 shutdown_connections 2024-04-20T21:19:46.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.171+0000 7f1340000640 1 -- 172.21.15.86:0/4020659475 wait complete. 2024-04-20T21:19:46.189 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:19:46.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.387+0000 7f0cce600640 1 Processor -- start 2024-04-20T21:19:46.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.388+0000 7f0cce600640 1 -- start start 2024-04-20T21:19:46.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.389+0000 7f0cce600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0cc8155d30 0x7f0cc80cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:46.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.389+0000 7f0cce600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0cc806b980 0x7f0cc806be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:46.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.389+0000 7f0cce600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cc8156110 con 0x7f0cc80b4ba0 2024-04-20T21:19:46.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.389+0000 7f0cce600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f0cc80b56e0 con 0x7f0cc8155d30 2024-04-20T21:19:46.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.389+0000 7f0cce600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f0cc80ce460 con 0x7f0cc806b980 2024-04-20T21:19:46.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.390+0000 7f0cc7400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0cc8155d30 0x7f0cc80cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:46.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.390+0000 7f0cc7e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f0cc80b4ba0 0x7f0cc80b4fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:47240/0 (socket says 172.21.15.86:47240) 2024-04-20T21:19:46.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.390+0000 7f0cc7e00640 1 -- 172.21.15.86:0/1239941802 learned_addr learned my addr 172.21.15.86:0/1239941802 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:19:46.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.390+0000 7f0cc6a00640 1 --2- 172.21.15.86:0/1239941802 >> v2:172.21.15.144:3300/0 conn(0x7f0cc806b980 0x7f0cc806be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:46.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.390+0000 7f0cc7400640 1 -- 172.21.15.86:0/1239941802 >> v2:172.21.15.144:3300/0 conn(0x7f0cc806b980 msgr2=0x7f0cc806be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:46.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.390+0000 7f0cc7400640 1 --2- 172.21.15.86:0/1239941802 >> v2:172.21.15.144:3300/0 conn(0x7f0cc806b980 0x7f0cc806be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:46.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.390+0000 7f0cc7400640 1 -- 172.21.15.86:0/1239941802 >> v1:172.21.15.86:6789/0 conn(0x7f0cc80b4ba0 legacy=0x7f0cc80b4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:46.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.390+0000 7f0cc7400640 1 -- 172.21.15.86:0/1239941802 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cc80ce740 con 0x7f0cc8155d30 2024-04-20T21:19:46.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.391+0000 7f0cc7400640 1 --2- 172.21.15.86:0/1239941802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cc8155d30 0x7f0cc80cf500 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8072f40 tx=0x7f0cb802b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11149deb7cc57bbf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:46.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.392+0000 7f0cc6000640 1 -- 172.21.15.86:0/1239941802 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb8038070 con 0x7f0cc8155d30 2024-04-20T21:19:46.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.392+0000 7f0cc6000640 1 -- 172.21.15.86:0/1239941802 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cb8031710 con 0x7f0cc8155d30 2024-04-20T21:19:46.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.392+0000 7f0cc6000640 1 -- 172.21.15.86:0/1239941802 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb8031ae0 con 0x7f0cc8155d30 2024-04-20T21:19:46.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.392+0000 7f0cce600640 1 -- 172.21.15.86:0/1239941802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cc8155d30 msgr2=0x7f0cc80cf500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:46.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.392+0000 7f0cce600640 1 --2- 172.21.15.86:0/1239941802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cc8155d30 0x7f0cc80cf500 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8072f40 tx=0x7f0cb802b7b0 comp rx=0 tx=0).stop 2024-04-20T21:19:46.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.393+0000 7f0cce600640 1 -- 172.21.15.86:0/1239941802 shutdown_connections 2024-04-20T21:19:46.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.393+0000 7f0cce600640 1 --2- 172.21.15.86:0/1239941802 >> v2:172.21.15.144:3300/0 conn(0x7f0cc806b980 0x7f0cc806be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:46.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.393+0000 7f0cce600640 1 --2- 172.21.15.86:0/1239941802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cc8155d30 0x7f0cc80cf500 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:46.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.393+0000 7f0cce600640 1 -- 172.21.15.86:0/1239941802 >> 172.21.15.86:0/1239941802 conn(0x7f0cc80b2110 msgr2=0x7f0cc80b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:46.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.393+0000 7f0cce600640 1 -- 172.21.15.86:0/1239941802 shutdown_connections 2024-04-20T21:19:46.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.393+0000 7f0cce600640 1 -- 172.21.15.86:0/1239941802 wait complete. 2024-04-20T21:19:46.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.395+0000 7f0cce600640 1 Processor -- start 2024-04-20T21:19:46.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.395+0000 7f0cce600640 1 -- start start 2024-04-20T21:19:46.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.396+0000 7f0cce600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cc806b980 0x7f0cc8145e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:46.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.396+0000 7f0cce600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0cc80b4ba0 0x7f0cc8148360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:46.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.396+0000 7f0cce600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cc8155d30 0x7f0cc8151c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:46.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.396+0000 7f0cce600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc80cefe0 con 0x7f0cc806b980 2024-04-20T21:19:46.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.396+0000 7f0cce600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc80ced00 con 0x7f0cc8155d30 2024-04-20T21:19:46.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.396+0000 7f0cce600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0cc806ddf0 con 0x7f0cc80b4ba0 2024-04-20T21:19:46.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.397+0000 7f0cc7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cc806b980 0x7f0cc8145e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:46.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.397+0000 7f0cc7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cc806b980 0x7f0cc8145e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39508/0 (socket says 172.21.15.86:39508) 2024-04-20T21:19:46.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.397+0000 7f0cc7400640 1 -- 172.21.15.86:0/3876433917 learned_addr learned my addr 172.21.15.86:0/3876433917 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:46.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.397+0000 7f0cc7400640 1 -- 172.21.15.86:0/3876433917 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0cc80b4ba0 msgr2=0x7f0cc8148360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:46.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.397+0000 7f0cc7400640 1 --2- 172.21.15.86:0/3876433917 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0cc80b4ba0 0x7f0cc8148360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:46.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.397+0000 7f0cc7400640 1 -- 172.21.15.86:0/3876433917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cc8155d30 msgr2=0x7f0cc8151c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:46.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.397+0000 7f0cc7400640 1 --2- 172.21.15.86:0/3876433917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cc8155d30 0x7f0cc8151c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:46.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.397+0000 7f0cc7400640 1 -- 172.21.15.86:0/3876433917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cc81f43f0 con 0x7f0cc806b980 2024-04-20T21:19:46.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.397+0000 7f0cc7400640 1 --2- 172.21.15.86:0/3876433917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cc806b980 0x7f0cc8145e20 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8009890 tx=0x7f0cb80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:46.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.398+0000 7f0cbf400640 1 -- 172.21.15.86:0/3876433917 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb8042030 con 0x7f0cc806b980 2024-04-20T21:19:46.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.398+0000 7f0cbf400640 1 -- 172.21.15.86:0/3876433917 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cb8031510 con 0x7f0cc806b980 2024-04-20T21:19:46.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.398+0000 7f0cbf400640 1 -- 172.21.15.86:0/3876433917 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb8038040 con 0x7f0cc806b980 2024-04-20T21:19:46.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.398+0000 7f0cce600640 1 -- 172.21.15.86:0/3876433917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cc81451c0 con 0x7f0cc806b980 2024-04-20T21:19:46.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.398+0000 7f0cce600640 1 -- 172.21.15.86:0/3876433917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cc813bd60 con 0x7f0cc806b980 2024-04-20T21:19:46.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.399+0000 7f0cce600640 1 -- 172.21.15.86:0/3876433917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cc80689e0 con 0x7f0cc806b980 2024-04-20T21:19:46.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.400+0000 7f0cbf400640 1 -- 172.21.15.86:0/3876433917 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0cb8045070 con 0x7f0cc806b980 2024-04-20T21:19:46.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.401+0000 7f0cbf400640 1 --2- 172.21.15.86:0/3876433917 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0ca406c8f0 0x7f0ca406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:46.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.401+0000 7f0cbf400640 1 -- 172.21.15.86:0/3876433917 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f0cb80ad240 con 0x7f0cc806b980 2024-04-20T21:19:46.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.401+0000 7f0cc6a00640 1 --2- 172.21.15.86:0/3876433917 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0ca406c8f0 0x7f0ca406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:46.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.402+0000 7f0cc6a00640 1 --2- 172.21.15.86:0/3876433917 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0ca406c8f0 0x7f0ca406edb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0cc81f7a40 tx=0x7f0cb0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:46.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.405+0000 7f0cbf400640 1 -- 172.21.15.86:0/3876433917 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0cb807b5d0 con 0x7f0cc806b980 2024-04-20T21:19:46.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.605+0000 7f0cce600640 1 -- 172.21.15.86:0/3876433917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0cc80b1a30 con 0x7f0cc806b980 2024-04-20T21:19:46.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.606+0000 7f0cbf400640 1 -- 172.21.15.86:0/3876433917 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) ==== 100+0+11 (secure 0 0 0) 0x7f0cb807aed0 con 0x7f0cc806b980 2024-04-20T21:19:46.605 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-20T21:19:46.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.608+0000 7f0cce600640 1 -- 172.21.15.86:0/3876433917 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0ca406c8f0 msgr2=0x7f0ca406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:46.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.608+0000 7f0cce600640 1 --2- 172.21.15.86:0/3876433917 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0ca406c8f0 0x7f0ca406edb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0cc81f7a40 tx=0x7f0cb0008040 comp rx=0 tx=0).stop 2024-04-20T21:19:46.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.608+0000 7f0cce600640 1 -- 172.21.15.86:0/3876433917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cc806b980 msgr2=0x7f0cc8145e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:46.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.608+0000 7f0cce600640 1 --2- 172.21.15.86:0/3876433917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cc806b980 0x7f0cc8145e20 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8009890 tx=0x7f0cb80046c0 comp rx=0 tx=0).stop 2024-04-20T21:19:46.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.608+0000 7f0cce600640 1 -- 172.21.15.86:0/3876433917 shutdown_connections 2024-04-20T21:19:46.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.608+0000 7f0cce600640 1 --2- 172.21.15.86:0/3876433917 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0ca406c8f0 0x7f0ca406edb0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:46.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.608+0000 7f0cce600640 1 --2- 172.21.15.86:0/3876433917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0cc8155d30 0x7f0cc8151c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:46.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.608+0000 7f0cce600640 1 --2- 172.21.15.86:0/3876433917 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0cc80b4ba0 0x7f0cc8148360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:46.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.608+0000 7f0cce600640 1 --2- 172.21.15.86:0/3876433917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0cc806b980 0x7f0cc8145e20 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:46.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.608+0000 7f0cce600640 1 -- 172.21.15.86:0/3876433917 >> 172.21.15.86:0/3876433917 conn(0x7f0cc80b2110 msgr2=0x7f0cc814d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:46.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.609+0000 7f0cce600640 1 -- 172.21.15.86:0/3876433917 shutdown_connections 2024-04-20T21:19:46.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.609+0000 7f0cce600640 1 -- 172.21.15.86:0/3876433917 wait complete. 2024-04-20T21:19:46.625 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:19:46.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.828+0000 7efc2fe00640 1 Processor -- start 2024-04-20T21:19:46.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.828+0000 7efc2fe00640 1 -- start start 2024-04-20T21:19:46.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.830+0000 7efc2fe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc3015c010 0x7efc3015c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:46.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.830+0000 7efc2fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7efc300af7b0 0x7efc300adda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:46.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.830+0000 7efc2fe00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc300ae710 con 0x7efc3006f4c0 2024-04-20T21:19:46.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.830+0000 7efc2fe00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7efc300af320 con 0x7efc300af7b0 2024-04-20T21:19:46.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.830+0000 7efc2fe00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7efc3015c930 con 0x7efc3015c010 2024-04-20T21:19:46.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.830+0000 7efc2e000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7efc300af7b0 0x7efc300adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:46.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.830+0000 7efc2e000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7efc300af7b0 0x7efc300adda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39512/0 (socket says 172.21.15.86:39512) 2024-04-20T21:19:46.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.830+0000 7efc2e000640 1 -- 172.21.15.86:0/3162361412 learned_addr learned my addr 172.21.15.86:0/3162361412 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:46.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.830+0000 7efc2ea00640 1 --2- 172.21.15.86:0/3162361412 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc3015c010 0x7efc3015c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:46.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.831+0000 7efc2e000640 1 -- 172.21.15.86:0/3162361412 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc3015c010 msgr2=0x7efc3015c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:46.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.831+0000 7efc2e000640 1 --2- 172.21.15.86:0/3162361412 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc3015c010 0x7efc3015c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:46.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.831+0000 7efc2e000640 1 -- 172.21.15.86:0/3162361412 >> v1:172.21.15.86:6789/0 conn(0x7efc3006f4c0 legacy=0x7efc3015ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:46.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.831+0000 7efc2e000640 1 -- 172.21.15.86:0/3162361412 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc300ae9f0 con 0x7efc300af7b0 2024-04-20T21:19:46.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.831+0000 7efc2ea00640 1 --2- 172.21.15.86:0/3162361412 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc3015c010 0x7efc3015c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:46.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.831+0000 7efc2d600640 1 -- 172.21.15.86:0/3162361412 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1115774615 0 0) 0x7efc300ae710 con 0x7efc3006f4c0 2024-04-20T21:19:46.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.832+0000 7efc2e000640 1 --2- 172.21.15.86:0/3162361412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc300af7b0 0x7efc300adda0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7efc1c009870 tx=0x7efc1c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6b0877a445df38f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:46.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.832+0000 7efc2d600640 1 -- 172.21.15.86:0/3162361412 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c034070 con 0x7efc300af7b0 2024-04-20T21:19:46.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.832+0000 7efc2d600640 1 -- 172.21.15.86:0/3162361412 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc1c030840 con 0x7efc300af7b0 2024-04-20T21:19:46.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.832+0000 7efc2d600640 1 -- 172.21.15.86:0/3162361412 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c030c10 con 0x7efc300af7b0 2024-04-20T21:19:46.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.833+0000 7efc2fe00640 1 -- 172.21.15.86:0/3162361412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc300af7b0 msgr2=0x7efc300adda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:46.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.833+0000 7efc2fe00640 1 --2- 172.21.15.86:0/3162361412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc300af7b0 0x7efc300adda0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7efc1c009870 tx=0x7efc1c02b980 comp rx=0 tx=0).stop 2024-04-20T21:19:46.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.833+0000 7efc2fe00640 1 -- 172.21.15.86:0/3162361412 shutdown_connections 2024-04-20T21:19:46.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.833+0000 7efc2fe00640 1 --2- 172.21.15.86:0/3162361412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc300af7b0 0x7efc300adda0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:46.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.833+0000 7efc2fe00640 1 --2- 172.21.15.86:0/3162361412 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc3015c010 0x7efc3015c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:46.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.833+0000 7efc2fe00640 1 -- 172.21.15.86:0/3162361412 >> 172.21.15.86:0/3162361412 conn(0x7efc300d0170 msgr2=0x7efc300d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:46.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.833+0000 7efc2fe00640 1 -- 172.21.15.86:0/3162361412 shutdown_connections 2024-04-20T21:19:46.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.833+0000 7efc2fe00640 1 -- 172.21.15.86:0/3162361412 wait complete. 2024-04-20T21:19:46.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.835+0000 7efc2fe00640 1 Processor -- start 2024-04-20T21:19:46.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.835+0000 7efc2fe00640 1 -- start start 2024-04-20T21:19:46.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.835+0000 7efc2fe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc3006f4c0 0x7efc3009cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:46.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.836+0000 7efc2fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc300af7b0 0x7efc300b44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:46.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.836+0000 7efc2fe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc3015c010 0x7efc300b4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:46.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.836+0000 7efc2fe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efc300ae2e0 con 0x7efc300af7b0 2024-04-20T21:19:46.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.836+0000 7efc2fe00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efc300aefb0 con 0x7efc3015c010 2024-04-20T21:19:46.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.836+0000 7efc2fe00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7efc300ae460 con 0x7efc3006f4c0 2024-04-20T21:19:46.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.836+0000 7efc2e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc300af7b0 0x7efc300b44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:46.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.836+0000 7efc2e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc300af7b0 0x7efc300b44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39520/0 (socket says 172.21.15.86:39520) 2024-04-20T21:19:46.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.836+0000 7efc2e000640 1 -- 172.21.15.86:0/691839699 learned_addr learned my addr 172.21.15.86:0/691839699 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:46.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.836+0000 7efc2e000640 1 -- 172.21.15.86:0/691839699 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc3006f4c0 msgr2=0x7efc3009cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:46.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.836+0000 7efc2ea00640 1 --2- 172.21.15.86:0/691839699 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc3006f4c0 0x7efc3009cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:46.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.836+0000 7efc2e000640 1 --2- 172.21.15.86:0/691839699 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc3006f4c0 0x7efc3009cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:46.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.836+0000 7efc2e000640 1 -- 172.21.15.86:0/691839699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc3015c010 msgr2=0x7efc300b4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:46.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.836+0000 7efc2f400640 1 --2- 172.21.15.86:0/691839699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc3015c010 0x7efc300b4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:46.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.836+0000 7efc2e000640 1 --2- 172.21.15.86:0/691839699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc3015c010 0x7efc300b4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:46.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.836+0000 7efc2e000640 1 -- 172.21.15.86:0/691839699 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc30152ec0 con 0x7efc300af7b0 2024-04-20T21:19:46.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.836+0000 7efc2f400640 1 --2- 172.21.15.86:0/691839699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc3015c010 0x7efc300b4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:46.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.837+0000 7efc2e000640 1 --2- 172.21.15.86:0/691839699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc300af7b0 0x7efc300b44e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7efc1c000c00 tx=0x7efc1c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:46.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.837+0000 7efc26a00640 1 -- 172.21.15.86:0/691839699 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c048070 con 0x7efc300af7b0 2024-04-20T21:19:46.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.837+0000 7efc26a00640 1 -- 172.21.15.86:0/691839699 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc1c030640 con 0x7efc300af7b0 2024-04-20T21:19:46.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.837+0000 7efc26a00640 1 -- 172.21.15.86:0/691839699 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c034040 con 0x7efc300af7b0 2024-04-20T21:19:46.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.837+0000 7efc2fe00640 1 -- 172.21.15.86:0/691839699 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc3009b1b0 con 0x7efc300af7b0 2024-04-20T21:19:46.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.837+0000 7efc2fe00640 1 -- 172.21.15.86:0/691839699 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc3009b770 con 0x7efc300af7b0 2024-04-20T21:19:46.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.838+0000 7efc2fe00640 1 -- 172.21.15.86:0/691839699 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc300b1230 con 0x7efc300af7b0 2024-04-20T21:19:46.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.839+0000 7efc26a00640 1 -- 172.21.15.86:0/691839699 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efc1c037070 con 0x7efc300af7b0 2024-04-20T21:19:46.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.840+0000 7efc26a00640 1 --2- 172.21.15.86:0/691839699 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efc1006c8f0 0x7efc1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:46.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.840+0000 7efc26a00640 1 -- 172.21.15.86:0/691839699 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7efc1c0ada40 con 0x7efc300af7b0 2024-04-20T21:19:46.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.840+0000 7efc2ea00640 1 --2- 172.21.15.86:0/691839699 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efc1006c8f0 0x7efc1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:46.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.841+0000 7efc2ea00640 1 --2- 172.21.15.86:0/691839699 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efc1006c8f0 0x7efc1006edb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7efc2000b840 tx=0x7efc20008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:46.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:46.845+0000 7efc26a00640 1 -- 172.21.15.86:0/691839699 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7efc300b1230 con 0x7efc300af7b0 2024-04-20T21:19:47.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.083+0000 7efc2fe00640 1 -- 172.21.15.86:0/691839699 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efc3004fcf0 con 0x7efc300af7b0 2024-04-20T21:19:47.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.085+0000 7efc26a00640 1 -- 172.21.15.86:0/691839699 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5205 (secure 0 0 0) 0x7efc1c0afe20 con 0x7efc300af7b0 2024-04-20T21:19:47.084 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:19:47.084 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4604,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/1957936003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":1957936003},{"type":"v1","addr":"172.21.15.144:6834","nonce":1957936003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4807,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/1510649389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":1510649389},{"type":"v1","addr":"172.21.15.144:6835","nonce":1510649389}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2180009156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2180009156},{"type":"v1","addr":"172.21.15.86:6836","nonce":2180009156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:19:09.979345+0000","modified":"2024-04-20T21:19:13.064114+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5226":{"gid":5226,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6837/3715801064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3715801064},{"type":"v1","addr":"172.21.15.86:6837","nonce":3715801064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5226,"qdb_cluster":[5226]},"id":2}]} 2024-04-20T21:19:47.085 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 16 2024-04-20T21:19:47.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.088+0000 7efc2fe00640 1 -- 172.21.15.86:0/691839699 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efc1006c8f0 msgr2=0x7efc1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:47.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.088+0000 7efc2fe00640 1 --2- 172.21.15.86:0/691839699 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efc1006c8f0 0x7efc1006edb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7efc2000b840 tx=0x7efc20008040 comp rx=0 tx=0).stop 2024-04-20T21:19:47.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.088+0000 7efc2fe00640 1 -- 172.21.15.86:0/691839699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc300af7b0 msgr2=0x7efc300b44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:47.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.088+0000 7efc2fe00640 1 --2- 172.21.15.86:0/691839699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc300af7b0 0x7efc300b44e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7efc1c000c00 tx=0x7efc1c004730 comp rx=0 tx=0).stop 2024-04-20T21:19:47.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.088+0000 7efc2fe00640 1 -- 172.21.15.86:0/691839699 shutdown_connections 2024-04-20T21:19:47.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.088+0000 7efc2fe00640 1 --2- 172.21.15.86:0/691839699 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efc1006c8f0 0x7efc1006edb0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:47.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.088+0000 7efc2fe00640 1 --2- 172.21.15.86:0/691839699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc3015c010 0x7efc300b4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:47.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.088+0000 7efc2fe00640 1 --2- 172.21.15.86:0/691839699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc300af7b0 0x7efc300b44e0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:47.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.088+0000 7efc2fe00640 1 --2- 172.21.15.86:0/691839699 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc3006f4c0 0x7efc3009cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:47.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.088+0000 7efc2fe00640 1 -- 172.21.15.86:0/691839699 >> 172.21.15.86:0/691839699 conn(0x7efc300d0170 msgr2=0x7efc3015e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:47.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.088+0000 7efc2fe00640 1 -- 172.21.15.86:0/691839699 shutdown_connections 2024-04-20T21:19:47.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.088+0000 7efc2fe00640 1 -- 172.21.15.86:0/691839699 wait complete. 2024-04-20T21:19:47.106 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:19:47.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.303+0000 7f8812000640 1 Processor -- start 2024-04-20T21:19:47.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.304+0000 7f8812000640 1 -- start start 2024-04-20T21:19:47.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.305+0000 7f8812000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f880c144e90 0x7f880c15ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:47.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.305+0000 7f8812000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f880c06f140 0x7f880c0aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:47.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.305+0000 7f8812000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f880c0af0b0 0x7f880c071d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:47.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.305+0000 7f8812000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f880c145270 con 0x7f880c144e90 2024-04-20T21:19:47.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.305+0000 7f8812000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f880c06dc30 con 0x7f880c06f140 2024-04-20T21:19:47.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.305+0000 7f8812000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f880c15f720 con 0x7f880c0af0b0 2024-04-20T21:19:47.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.306+0000 7f880b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f880c144e90 0x7f880c15ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:47.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.306+0000 7f880b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f880c144e90 0x7f880c15ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39522/0 (socket says 172.21.15.86:39522) 2024-04-20T21:19:47.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.306+0000 7f880b400640 1 -- 172.21.15.86:0/3268301812 learned_addr learned my addr 172.21.15.86:0/3268301812 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:47.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.306+0000 7f8802a00640 1 --2- 172.21.15.86:0/3268301812 >> v2:172.21.15.144:3300/0 conn(0x7f880c06f140 0x7f880c0aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:47.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.306+0000 7f880be00640 1 --2- 172.21.15.86:0/3268301812 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f880c0af0b0 0x7f880c071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:47.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.306+0000 7f880b400640 1 -- 172.21.15.86:0/3268301812 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f880c0af0b0 msgr2=0x7f880c071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:47.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.306+0000 7f880b400640 1 --2- 172.21.15.86:0/3268301812 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f880c0af0b0 0x7f880c071d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:47.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.306+0000 7f880b400640 1 -- 172.21.15.86:0/3268301812 >> v2:172.21.15.144:3300/0 conn(0x7f880c06f140 msgr2=0x7f880c0aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:47.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.306+0000 7f880b400640 1 --2- 172.21.15.86:0/3268301812 >> v2:172.21.15.144:3300/0 conn(0x7f880c06f140 0x7f880c0aeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:47.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.306+0000 7f880b400640 1 -- 172.21.15.86:0/3268301812 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f880c15fa00 con 0x7f880c144e90 2024-04-20T21:19:47.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.308+0000 7f880b400640 1 --2- 172.21.15.86:0/3268301812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f880c144e90 0x7f880c15ee10 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f880c0735c0 tx=0x7f87fc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d771bf407bc2b7a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:47.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.308+0000 7f880aa00640 1 -- 172.21.15.86:0/3268301812 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87fc035020 con 0x7f880c144e90 2024-04-20T21:19:47.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.308+0000 7f880aa00640 1 -- 172.21.15.86:0/3268301812 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87fc031870 con 0x7f880c144e90 2024-04-20T21:19:47.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.308+0000 7f880aa00640 1 -- 172.21.15.86:0/3268301812 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87fc031c40 con 0x7f880c144e90 2024-04-20T21:19:47.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.309+0000 7f8812000640 1 -- 172.21.15.86:0/3268301812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f880c144e90 msgr2=0x7f880c15ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:47.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.309+0000 7f8812000640 1 --2- 172.21.15.86:0/3268301812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f880c144e90 0x7f880c15ee10 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f880c0735c0 tx=0x7f87fc02b910 comp rx=0 tx=0).stop 2024-04-20T21:19:47.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.309+0000 7f8812000640 1 -- 172.21.15.86:0/3268301812 shutdown_connections 2024-04-20T21:19:47.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.309+0000 7f8812000640 1 --2- 172.21.15.86:0/3268301812 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f880c0af0b0 0x7f880c071d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:47.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.309+0000 7f8812000640 1 --2- 172.21.15.86:0/3268301812 >> v2:172.21.15.144:3300/0 conn(0x7f880c06f140 0x7f880c0aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:47.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.309+0000 7f8812000640 1 --2- 172.21.15.86:0/3268301812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f880c144e90 0x7f880c15ee10 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:47.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.309+0000 7f8812000640 1 -- 172.21.15.86:0/3268301812 >> 172.21.15.86:0/3268301812 conn(0x7f880c0b2360 msgr2=0x7f880c0b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:47.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.309+0000 7f8812000640 1 -- 172.21.15.86:0/3268301812 shutdown_connections 2024-04-20T21:19:47.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.309+0000 7f8812000640 1 -- 172.21.15.86:0/3268301812 wait complete. 2024-04-20T21:19:47.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.311+0000 7f8812000640 1 Processor -- start 2024-04-20T21:19:47.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.311+0000 7f8812000640 1 -- start start 2024-04-20T21:19:47.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.312+0000 7f8812000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f880c06f140 0x7f880c0cf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:47.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.312+0000 7f8812000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f880c0af0b0 0x7f880c0cfce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:47.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.312+0000 7f8812000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f880c144e90 0x7f880c133e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:47.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.312+0000 7f8812000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f880c15f350 con 0x7f880c06f140 2024-04-20T21:19:47.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.312+0000 7f8812000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f880c06ded0 con 0x7f880c0af0b0 2024-04-20T21:19:47.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.312+0000 7f8812000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f880c15f4d0 con 0x7f880c144e90 2024-04-20T21:19:47.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.312+0000 7f880b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f880c06f140 0x7f880c0cf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:47.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.312+0000 7f880b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f880c06f140 0x7f880c0cf7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39538/0 (socket says 172.21.15.86:39538) 2024-04-20T21:19:47.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.312+0000 7f880b400640 1 -- 172.21.15.86:0/3429930301 learned_addr learned my addr 172.21.15.86:0/3429930301 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:47.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.312+0000 7f880b400640 1 -- 172.21.15.86:0/3429930301 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f880c144e90 msgr2=0x7f880c133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:47.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.312+0000 7f880b400640 1 --2- 172.21.15.86:0/3429930301 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f880c144e90 0x7f880c133e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:47.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.312+0000 7f880b400640 1 -- 172.21.15.86:0/3429930301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f880c0af0b0 msgr2=0x7f880c0cfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:47.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.312+0000 7f880b400640 1 --2- 172.21.15.86:0/3429930301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f880c0af0b0 0x7f880c0cfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:47.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.312+0000 7f880b400640 1 -- 172.21.15.86:0/3429930301 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f880c147ff0 con 0x7f880c06f140 2024-04-20T21:19:47.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.313+0000 7f880b400640 1 --2- 172.21.15.86:0/3429930301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f880c06f140 0x7f880c0cf7a0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f87fc0090b0 tx=0x7f87fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:47.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.313+0000 7f8803e00640 1 -- 172.21.15.86:0/3429930301 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87fc040070 con 0x7f880c06f140 2024-04-20T21:19:47.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.313+0000 7f8803e00640 1 -- 172.21.15.86:0/3429930301 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87fc030d00 con 0x7f880c06f140 2024-04-20T21:19:47.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.313+0000 7f8803e00640 1 -- 172.21.15.86:0/3429930301 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87fc035040 con 0x7f880c06f140 2024-04-20T21:19:47.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.313+0000 7f8812000640 1 -- 172.21.15.86:0/3429930301 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f880c131da0 con 0x7f880c06f140 2024-04-20T21:19:47.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.313+0000 7f8812000640 1 -- 172.21.15.86:0/3429930301 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f880c132360 con 0x7f880c06f140 2024-04-20T21:19:47.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.314+0000 7f8812000640 1 -- 172.21.15.86:0/3429930301 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f880c0689e0 con 0x7f880c06f140 2024-04-20T21:19:47.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.315+0000 7f8803e00640 1 -- 172.21.15.86:0/3429930301 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f87fc04b060 con 0x7f880c06f140 2024-04-20T21:19:47.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.316+0000 7f8803e00640 1 --2- 172.21.15.86:0/3429930301 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f87e806c8f0 0x7f87e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:47.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.316+0000 7f8803e00640 1 -- 172.21.15.86:0/3429930301 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f87fc0ad8f0 con 0x7f880c06f140 2024-04-20T21:19:47.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.316+0000 7f8802a00640 1 --2- 172.21.15.86:0/3429930301 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f87e806c8f0 0x7f87e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:47.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.317+0000 7f8802a00640 1 --2- 172.21.15.86:0/3429930301 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f87e806c8f0 0x7f87e806edb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f87f4006fd0 tx=0x7f87f4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:47.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.320+0000 7f8803e00640 1 -- 172.21.15.86:0/3429930301 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f87fc07bd20 con 0x7f880c06f140 2024-04-20T21:19:47.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.513+0000 7f8812000640 1 -- 172.21.15.86:0/3429930301 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f880c144400 con 0x7f880c06f140 2024-04-20T21:19:47.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.515+0000 7f8803e00640 1 -- 172.21.15.86:0/3429930301 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16669 (secure 0 0 0) 0x7f87fc07b620 con 0x7f880c06f140 2024-04-20T21:19:47.515 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:19:47.515 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":33,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:19:11.559613+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:19:07.732422+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-20T21:19:08.960268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:19:47.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.518+0000 7f8812000640 1 -- 172.21.15.86:0/3429930301 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f87e806c8f0 msgr2=0x7f87e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:47.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.518+0000 7f8812000640 1 --2- 172.21.15.86:0/3429930301 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f87e806c8f0 0x7f87e806edb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f87f4006fd0 tx=0x7f87f4008040 comp rx=0 tx=0).stop 2024-04-20T21:19:47.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.518+0000 7f8812000640 1 -- 172.21.15.86:0/3429930301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f880c06f140 msgr2=0x7f880c0cf7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:47.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.518+0000 7f8812000640 1 --2- 172.21.15.86:0/3429930301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f880c06f140 0x7f880c0cf7a0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f87fc0090b0 tx=0x7f87fc0046c0 comp rx=0 tx=0).stop 2024-04-20T21:19:47.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.518+0000 7f8812000640 1 -- 172.21.15.86:0/3429930301 shutdown_connections 2024-04-20T21:19:47.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.518+0000 7f8812000640 1 --2- 172.21.15.86:0/3429930301 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f87e806c8f0 0x7f87e806edb0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:47.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.518+0000 7f8812000640 1 --2- 172.21.15.86:0/3429930301 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f880c144e90 0x7f880c133e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:47.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.518+0000 7f8812000640 1 --2- 172.21.15.86:0/3429930301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f880c0af0b0 0x7f880c0cfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:47.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.518+0000 7f8812000640 1 --2- 172.21.15.86:0/3429930301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f880c06f140 0x7f880c0cf7a0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:47.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.518+0000 7f8812000640 1 -- 172.21.15.86:0/3429930301 >> 172.21.15.86:0/3429930301 conn(0x7f880c0b2360 msgr2=0x7f880c141830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:47.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.518+0000 7f8812000640 1 -- 172.21.15.86:0/3429930301 shutdown_connections 2024-04-20T21:19:47.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.518+0000 7f8812000640 1 -- 172.21.15.86:0/3429930301 wait complete. 2024-04-20T21:19:47.536 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-20T21:19:47.536 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:19:47.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.729+0000 7f946f200640 1 Processor -- start 2024-04-20T21:19:47.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.729+0000 7f946f200640 1 -- start start 2024-04-20T21:19:47.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.731+0000 7f946f200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f947015ece0 0x7f947015f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:47.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.731+0000 7f946f200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f94700d50c0 0x7f94700d3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:47.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.731+0000 7f946f200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94700cf9c0 0x7f9470160000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:47.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.731+0000 7f946f200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f947015f600 con 0x7f94700d50c0 2024-04-20T21:19:47.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.731+0000 7f946f200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f94700d3950 con 0x7f947015ece0 2024-04-20T21:19:47.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.731+0000 7f946f200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f94700cec00 con 0x7f94700cf9c0 2024-04-20T21:19:47.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.731+0000 7f9465400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f94700d50c0 0x7f94700d3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:47.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.731+0000 7f9465400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f94700d50c0 0x7f94700d3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39550/0 (socket says 172.21.15.86:39550) 2024-04-20T21:19:47.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.731+0000 7f9465400640 1 -- 172.21.15.86:0/2757982923 learned_addr learned my addr 172.21.15.86:0/2757982923 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:47.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.731+0000 7f946e800640 1 --2- 172.21.15.86:0/2757982923 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94700cf9c0 0x7f9470160000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:47.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.731+0000 7f946de00640 1 --2- 172.21.15.86:0/2757982923 >> v2:172.21.15.144:3300/0 conn(0x7f947015ece0 0x7f947015f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:47.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.732+0000 7f9465400640 1 -- 172.21.15.86:0/2757982923 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94700cf9c0 msgr2=0x7f9470160000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:47.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.732+0000 7f9465400640 1 --2- 172.21.15.86:0/2757982923 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94700cf9c0 0x7f9470160000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:47.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.732+0000 7f9465400640 1 -- 172.21.15.86:0/2757982923 >> v2:172.21.15.144:3300/0 conn(0x7f947015ece0 msgr2=0x7f947015f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:47.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.732+0000 7f9465400640 1 --2- 172.21.15.86:0/2757982923 >> v2:172.21.15.144:3300/0 conn(0x7f947015ece0 0x7f947015f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:47.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.732+0000 7f9465400640 1 -- 172.21.15.86:0/2757982923 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94700ceee0 con 0x7f94700d50c0 2024-04-20T21:19:47.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.733+0000 7f9465400640 1 --2- 172.21.15.86:0/2757982923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94700d50c0 0x7f94700d3100 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f94700d2360 tx=0x7f945802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2da65c775368932b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:47.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.733+0000 7f946d400640 1 -- 172.21.15.86:0/2757982923 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945802e070 con 0x7f94700d50c0 2024-04-20T21:19:47.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.733+0000 7f946d400640 1 -- 172.21.15.86:0/2757982923 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9458037070 con 0x7f94700d50c0 2024-04-20T21:19:47.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.733+0000 7f946d400640 1 -- 172.21.15.86:0/2757982923 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9458031a00 con 0x7f94700d50c0 2024-04-20T21:19:47.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.734+0000 7f946f200640 1 -- 172.21.15.86:0/2757982923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94700d50c0 msgr2=0x7f94700d3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:47.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.734+0000 7f946f200640 1 --2- 172.21.15.86:0/2757982923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94700d50c0 0x7f94700d3100 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f94700d2360 tx=0x7f945802b880 comp rx=0 tx=0).stop 2024-04-20T21:19:47.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.734+0000 7f946f200640 1 -- 172.21.15.86:0/2757982923 shutdown_connections 2024-04-20T21:19:47.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.734+0000 7f946f200640 1 --2- 172.21.15.86:0/2757982923 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94700cf9c0 0x7f9470160000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:47.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.734+0000 7f946f200640 1 --2- 172.21.15.86:0/2757982923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94700d50c0 0x7f94700d3100 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:47.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.734+0000 7f946f200640 1 --2- 172.21.15.86:0/2757982923 >> v2:172.21.15.144:3300/0 conn(0x7f947015ece0 0x7f947015f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:47.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.734+0000 7f946f200640 1 -- 172.21.15.86:0/2757982923 >> 172.21.15.86:0/2757982923 conn(0x7f947008bf70 msgr2=0x7f94700d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:47.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.734+0000 7f946f200640 1 -- 172.21.15.86:0/2757982923 shutdown_connections 2024-04-20T21:19:47.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.734+0000 7f946f200640 1 -- 172.21.15.86:0/2757982923 wait complete. 2024-04-20T21:19:47.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.736+0000 7f946f200640 1 Processor -- start 2024-04-20T21:19:47.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.736+0000 7f946f200640 1 -- start start 2024-04-20T21:19:47.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.737+0000 7f946f200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94700cf9c0 0x7f94701ea350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:47.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.737+0000 7f946f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94700d50c0 0x7f94700c9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:47.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.737+0000 7f946f200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f947015ece0 0x7f94701e8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:47.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.737+0000 7f946f200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f94700d3640 con 0x7f94700d50c0 2024-04-20T21:19:47.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.737+0000 7f946f200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f94700cf4a0 con 0x7f94700cf9c0 2024-04-20T21:19:47.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.737+0000 7f946f200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f947015ea50 con 0x7f947015ece0 2024-04-20T21:19:47.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.738+0000 7f9465400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94700d50c0 0x7f94700c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:47.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.738+0000 7f9465400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94700d50c0 0x7f94700c9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39560/0 (socket says 172.21.15.86:39560) 2024-04-20T21:19:47.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.738+0000 7f9465400640 1 -- 172.21.15.86:0/3501695164 learned_addr learned my addr 172.21.15.86:0/3501695164 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:47.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.738+0000 7f946de00640 1 --2- 172.21.15.86:0/3501695164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94700cf9c0 0x7f94701ea350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:47.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.738+0000 7f946e800640 1 --2- 172.21.15.86:0/3501695164 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f947015ece0 0x7f94701e8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:47.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.738+0000 7f9465400640 1 -- 172.21.15.86:0/3501695164 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f947015ece0 msgr2=0x7f94701e8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:47.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.738+0000 7f9465400640 1 --2- 172.21.15.86:0/3501695164 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f947015ece0 0x7f94701e8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:47.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.738+0000 7f9465400640 1 -- 172.21.15.86:0/3501695164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94700cf9c0 msgr2=0x7f94701ea350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:47.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.738+0000 7f9465400640 1 --2- 172.21.15.86:0/3501695164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94700cf9c0 0x7f94701ea350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:47.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.738+0000 7f9465400640 1 -- 172.21.15.86:0/3501695164 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94700c83c0 con 0x7f94700d50c0 2024-04-20T21:19:47.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.739+0000 7f9465400640 1 --2- 172.21.15.86:0/3501695164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94700d50c0 0x7f94700c9460 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f9458000c00 tx=0x7f94580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:47.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.739+0000 7f9466a00640 1 -- 172.21.15.86:0/3501695164 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9458032670 con 0x7f94700d50c0 2024-04-20T21:19:47.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.739+0000 7f9466a00640 1 -- 172.21.15.86:0/3501695164 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9458032850 con 0x7f94700d50c0 2024-04-20T21:19:47.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.739+0000 7f9466a00640 1 -- 172.21.15.86:0/3501695164 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9458032c40 con 0x7f94700d50c0 2024-04-20T21:19:47.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.739+0000 7f946f200640 1 -- 172.21.15.86:0/3501695164 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94700c86a0 con 0x7f94700d50c0 2024-04-20T21:19:47.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.739+0000 7f946f200640 1 -- 172.21.15.86:0/3501695164 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94700c8c60 con 0x7f94700d50c0 2024-04-20T21:19:47.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.740+0000 7f946f200640 1 -- 172.21.15.86:0/3501695164 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9470161f70 con 0x7f94700d50c0 2024-04-20T21:19:47.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.741+0000 7f9466a00640 1 -- 172.21.15.86:0/3501695164 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f945804c060 con 0x7f94700d50c0 2024-04-20T21:19:47.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.742+0000 7f9466a00640 1 --2- 172.21.15.86:0/3501695164 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f944c06c940 0x7f944c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:47.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.742+0000 7f9466a00640 1 -- 172.21.15.86:0/3501695164 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f94580ad650 con 0x7f94700d50c0 2024-04-20T21:19:47.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.742+0000 7f946de00640 1 --2- 172.21.15.86:0/3501695164 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f944c06c940 0x7f944c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:47.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.743+0000 7f946de00640 1 --2- 172.21.15.86:0/3501695164 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f944c06c940 0x7f944c06ee00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9460006fd0 tx=0x7f9460008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:47.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.747+0000 7f9466a00640 1 -- 172.21.15.86:0/3501695164 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9470161f70 con 0x7f94700d50c0 2024-04-20T21:19:47.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.980+0000 7f946f200640 1 -- 172.21.15.86:0/3501695164 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f94700d0cc0 con 0x7f94700d50c0 2024-04-20T21:19:47.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.982+0000 7f9466a00640 1 -- 172.21.15.86:0/3501695164 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5205 (secure 0 0 0) 0x7f9458040070 con 0x7f94700d50c0 2024-04-20T21:19:47.981 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:19:47.981 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4604,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/1957936003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":1957936003},{"type":"v1","addr":"172.21.15.144:6834","nonce":1957936003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4807,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/1510649389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":1510649389},{"type":"v1","addr":"172.21.15.144:6835","nonce":1510649389}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2180009156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2180009156},{"type":"v1","addr":"172.21.15.86:6836","nonce":2180009156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:19:09.979345+0000","modified":"2024-04-20T21:19:13.064114+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5226":{"gid":5226,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6837/3715801064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3715801064},{"type":"v1","addr":"172.21.15.86:6837","nonce":3715801064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5226,"qdb_cluster":[5226]},"id":2}]} 2024-04-20T21:19:47.981 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 16 2024-04-20T21:19:47.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.984+0000 7f946f200640 1 -- 172.21.15.86:0/3501695164 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f944c06c940 msgr2=0x7f944c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:47.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.984+0000 7f946f200640 1 --2- 172.21.15.86:0/3501695164 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f944c06c940 0x7f944c06ee00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9460006fd0 tx=0x7f9460008040 comp rx=0 tx=0).stop 2024-04-20T21:19:47.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.984+0000 7f946f200640 1 -- 172.21.15.86:0/3501695164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94700d50c0 msgr2=0x7f94700c9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:47.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.984+0000 7f946f200640 1 --2- 172.21.15.86:0/3501695164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94700d50c0 0x7f94700c9460 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f9458000c00 tx=0x7f94580046c0 comp rx=0 tx=0).stop 2024-04-20T21:19:47.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.984+0000 7f946f200640 1 -- 172.21.15.86:0/3501695164 shutdown_connections 2024-04-20T21:19:47.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.984+0000 7f946f200640 1 --2- 172.21.15.86:0/3501695164 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f944c06c940 0x7f944c06ee00 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:47.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.984+0000 7f946f200640 1 --2- 172.21.15.86:0/3501695164 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f947015ece0 0x7f94701e8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:47.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.984+0000 7f946f200640 1 --2- 172.21.15.86:0/3501695164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94700d50c0 0x7f94700c9460 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:47.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.984+0000 7f946f200640 1 --2- 172.21.15.86:0/3501695164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94700cf9c0 0x7f94701ea350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:47.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.984+0000 7f946f200640 1 -- 172.21.15.86:0/3501695164 >> 172.21.15.86:0/3501695164 conn(0x7f947008bf70 msgr2=0x7f94700d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:47.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.985+0000 7f946f200640 1 -- 172.21.15.86:0/3501695164 shutdown_connections 2024-04-20T21:19:47.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:47.985+0000 7f946f200640 1 -- 172.21.15.86:0/3501695164 wait complete. 2024-04-20T21:19:48.002 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:19:48.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.175+0000 7fcc1c800640 1 Processor -- start 2024-04-20T21:19:48.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.176+0000 7fcc1c800640 1 -- start start 2024-04-20T21:19:48.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.177+0000 7fcc1c800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc14146130 0x7fcc1406e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:48.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.177+0000 7fcc1c800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fcc14072410 0x7fcc140cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:48.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.177+0000 7fcc1c800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc1406dc30 con 0x7fcc140ccdd0 2024-04-20T21:19:48.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.177+0000 7fcc1c800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fcc140cd8e0 con 0x7fcc14072410 2024-04-20T21:19:48.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.177+0000 7fcc1c800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fcc140cdbc0 con 0x7fcc14146130 2024-04-20T21:19:48.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.178+0000 7fcc19e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc14146130 0x7fcc1406e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:48.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.178+0000 7fcc19400640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fcc140ccdd0 0x7fcc14071d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.86:49942/0 (socket says 172.21.15.86:49942) 2024-04-20T21:19:48.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.178+0000 7fcc19400640 1 -- 172.21.15.86:0/1034798855 learned_addr learned my addr 172.21.15.86:0/1034798855 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:19:48.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.178+0000 7fcc1a800640 1 --2- 172.21.15.86:0/1034798855 >> v2:172.21.15.144:3300/0 conn(0x7fcc14072410 0x7fcc140cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:48.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.179+0000 7fcc1a800640 1 -- 172.21.15.86:0/1034798855 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc14146130 msgr2=0x7fcc1406e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:48.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.179+0000 7fcc1a800640 1 --2- 172.21.15.86:0/1034798855 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc14146130 0x7fcc1406e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:48.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.179+0000 7fcc1a800640 1 -- 172.21.15.86:0/1034798855 >> v1:172.21.15.144:6789/0 conn(0x7fcc140ccdd0 legacy=0x7fcc14071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:48.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.179+0000 7fcc1a800640 1 -- 172.21.15.86:0/1034798855 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc1406ded0 con 0x7fcc14072410 2024-04-20T21:19:48.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.180+0000 7fcc1a800640 1 --2- 172.21.15.86:0/1034798855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc14072410 0x7fcc140cd230 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcc10009d90 tx=0x7fcc1002bf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c68a0f2a89ae7ded server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:48.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.181+0000 7fcc18a00640 1 -- 172.21.15.86:0/1034798855 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc10039070 con 0x7fcc14072410 2024-04-20T21:19:48.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.181+0000 7fcc18a00640 1 -- 172.21.15.86:0/1034798855 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc1002cce0 con 0x7fcc14072410 2024-04-20T21:19:48.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.181+0000 7fcc18a00640 1 -- 172.21.15.86:0/1034798855 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc10034660 con 0x7fcc14072410 2024-04-20T21:19:48.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.182+0000 7fcc1c800640 1 -- 172.21.15.86:0/1034798855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc14072410 msgr2=0x7fcc140cd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:48.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.182+0000 7fcc1c800640 1 --2- 172.21.15.86:0/1034798855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc14072410 0x7fcc140cd230 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcc10009d90 tx=0x7fcc1002bf30 comp rx=0 tx=0).stop 2024-04-20T21:19:48.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.182+0000 7fcc1c800640 1 -- 172.21.15.86:0/1034798855 shutdown_connections 2024-04-20T21:19:48.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.182+0000 7fcc1c800640 1 --2- 172.21.15.86:0/1034798855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc14072410 0x7fcc140cd230 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:48.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.182+0000 7fcc1c800640 1 --2- 172.21.15.86:0/1034798855 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc14146130 0x7fcc1406e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:48.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.182+0000 7fcc1c800640 1 -- 172.21.15.86:0/1034798855 >> 172.21.15.86:0/1034798855 conn(0x7fcc140b2360 msgr2=0x7fcc140b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:48.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.182+0000 7fcc1c800640 1 -- 172.21.15.86:0/1034798855 shutdown_connections 2024-04-20T21:19:48.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.182+0000 7fcc1c800640 1 -- 172.21.15.86:0/1034798855 wait complete. 2024-04-20T21:19:48.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.183+0000 7fcc1c800640 1 Processor -- start 2024-04-20T21:19:48.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.183+0000 7fcc1c800640 1 -- start start 2024-04-20T21:19:48.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.184+0000 7fcc1c800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc14072410 0x7fcc1413d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:48.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.184+0000 7fcc1c800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc140ccdd0 0x7fcc1413a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:48.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.184+0000 7fcc1c800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc14146130 0x7fcc1415d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:48.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.184+0000 7fcc1c800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1415cc20 con 0x7fcc14072410 2024-04-20T21:19:48.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.184+0000 7fcc1c800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1406f150 con 0x7fcc140ccdd0 2024-04-20T21:19:48.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.184+0000 7fcc1c800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fcc1415cda0 con 0x7fcc14146130 2024-04-20T21:19:48.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.185+0000 7fcc19e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc14072410 0x7fcc1413d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:48.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.185+0000 7fcc19e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc14072410 0x7fcc1413d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39576/0 (socket says 172.21.15.86:39576) 2024-04-20T21:19:48.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.185+0000 7fcc19e00640 1 -- 172.21.15.86:0/1038783257 learned_addr learned my addr 172.21.15.86:0/1038783257 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:48.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.185+0000 7fcc19400640 1 --2- 172.21.15.86:0/1038783257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc140ccdd0 0x7fcc1413a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:48.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.185+0000 7fcc19e00640 1 -- 172.21.15.86:0/1038783257 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc14146130 msgr2=0x7fcc1415d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:48.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.185+0000 7fcc19e00640 1 --2- 172.21.15.86:0/1038783257 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc14146130 0x7fcc1415d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:48.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.185+0000 7fcc19e00640 1 -- 172.21.15.86:0/1038783257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc140ccdd0 msgr2=0x7fcc1413a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:48.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.185+0000 7fcc19e00640 1 --2- 172.21.15.86:0/1038783257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc140ccdd0 0x7fcc1413a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:48.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.185+0000 7fcc19e00640 1 -- 172.21.15.86:0/1038783257 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc1413ad50 con 0x7fcc14072410 2024-04-20T21:19:48.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.185+0000 7fcc19400640 1 --2- 172.21.15.86:0/1038783257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc140ccdd0 0x7fcc1413a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:48.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.185+0000 7fcc19e00640 1 --2- 172.21.15.86:0/1038783257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc14072410 0x7fcc1413d0c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fcc0400e820 tx=0x7fcc0400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:48.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.186+0000 7fcc0e000640 1 -- 172.21.15.86:0/1038783257 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc04017070 con 0x7fcc14072410 2024-04-20T21:19:48.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.186+0000 7fcc0e000640 1 -- 172.21.15.86:0/1038783257 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc0400c940 con 0x7fcc14072410 2024-04-20T21:19:48.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.186+0000 7fcc0e000640 1 -- 172.21.15.86:0/1038783257 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0400cd30 con 0x7fcc14072410 2024-04-20T21:19:48.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.186+0000 7fcc1c800640 1 -- 172.21.15.86:0/1038783257 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc1415d900 con 0x7fcc14072410 2024-04-20T21:19:48.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.186+0000 7fcc1c800640 1 -- 172.21.15.86:0/1038783257 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc1415dec0 con 0x7fcc14072410 2024-04-20T21:19:48.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.187+0000 7fcc1c800640 1 -- 172.21.15.86:0/1038783257 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc1415e5d0 con 0x7fcc14072410 2024-04-20T21:19:48.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.188+0000 7fcc0e000640 1 -- 172.21.15.86:0/1038783257 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcc0401a070 con 0x7fcc14072410 2024-04-20T21:19:48.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.189+0000 7fcc0e000640 1 --2- 172.21.15.86:0/1038783257 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcbf806c8f0 0x7fcbf806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:48.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.189+0000 7fcc0e000640 1 -- 172.21.15.86:0/1038783257 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fcc0408ba90 con 0x7fcc14072410 2024-04-20T21:19:48.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.189+0000 7fcc19400640 1 --2- 172.21.15.86:0/1038783257 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcbf806c8f0 0x7fcbf806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:48.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.190+0000 7fcc19400640 1 --2- 172.21.15.86:0/1038783257 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcbf806c8f0 0x7fcbf806edb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcc140b2a70 tx=0x7fcc08009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:48.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.193+0000 7fcc0e000640 1 -- 172.21.15.86:0/1038783257 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fcc1413ad50 con 0x7fcc14072410 2024-04-20T21:19:48.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.411+0000 7fcc1c800640 1 -- 172.21.15.86:0/1038783257 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcc140689e0 con 0x7fcc14072410 2024-04-20T21:19:48.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.413+0000 7fcc0e000640 1 -- 172.21.15.86:0/1038783257 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5205 (secure 0 0 0) 0x7fcc04059d90 con 0x7fcc14072410 2024-04-20T21:19:48.412 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:19:48.413 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4604,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/1957936003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":1957936003},{"type":"v1","addr":"172.21.15.144:6834","nonce":1957936003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4807,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/1510649389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":1510649389},{"type":"v1","addr":"172.21.15.144:6835","nonce":1510649389}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2180009156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2180009156},{"type":"v1","addr":"172.21.15.86:6836","nonce":2180009156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:19:09.979345+0000","modified":"2024-04-20T21:19:13.064114+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5226},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5226":{"gid":5226,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6837/3715801064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3715801064},{"type":"v1","addr":"172.21.15.86:6837","nonce":3715801064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5226,"qdb_cluster":[5226]},"id":2}]} 2024-04-20T21:19:48.413 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 16 2024-04-20T21:19:48.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.415+0000 7fcc1c800640 1 -- 172.21.15.86:0/1038783257 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcbf806c8f0 msgr2=0x7fcbf806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:48.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.415+0000 7fcc1c800640 1 --2- 172.21.15.86:0/1038783257 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcbf806c8f0 0x7fcbf806edb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcc140b2a70 tx=0x7fcc08009040 comp rx=0 tx=0).stop 2024-04-20T21:19:48.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.415+0000 7fcc1c800640 1 -- 172.21.15.86:0/1038783257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc14072410 msgr2=0x7fcc1413d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:48.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.415+0000 7fcc1c800640 1 --2- 172.21.15.86:0/1038783257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc14072410 0x7fcc1413d0c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fcc0400e820 tx=0x7fcc0400ece0 comp rx=0 tx=0).stop 2024-04-20T21:19:48.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.415+0000 7fcc1c800640 1 -- 172.21.15.86:0/1038783257 shutdown_connections 2024-04-20T21:19:48.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.415+0000 7fcc1c800640 1 --2- 172.21.15.86:0/1038783257 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcbf806c8f0 0x7fcbf806edb0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:48.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.415+0000 7fcc1c800640 1 --2- 172.21.15.86:0/1038783257 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc14146130 0x7fcc1415d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:48.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.415+0000 7fcc1c800640 1 --2- 172.21.15.86:0/1038783257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc140ccdd0 0x7fcc1413a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:48.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.415+0000 7fcc1c800640 1 --2- 172.21.15.86:0/1038783257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc14072410 0x7fcc1413d0c0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:48.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.415+0000 7fcc1c800640 1 -- 172.21.15.86:0/1038783257 >> 172.21.15.86:0/1038783257 conn(0x7fcc140b2360 msgr2=0x7fcc14071d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:48.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.415+0000 7fcc1c800640 1 -- 172.21.15.86:0/1038783257 shutdown_connections 2024-04-20T21:19:48.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.415+0000 7fcc1c800640 1 -- 172.21.15.86:0/1038783257 wait complete. 2024-04-20T21:19:48.432 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:19:48.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.631+0000 7f7397a00640 1 Processor -- start 2024-04-20T21:19:48.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.631+0000 7f7397a00640 1 -- start start 2024-04-20T21:19:48.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.633+0000 7f7397a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f739006e6d0 0x7f7390071d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:48.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.633+0000 7f7397a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7390072240 0x7f739015cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:48.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.633+0000 7f7397a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f739006eb30 con 0x7f7390146130 2024-04-20T21:19:48.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.633+0000 7f7397a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f739006dc30 con 0x7f7390072240 2024-04-20T21:19:48.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.633+0000 7f7397a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f73900cd8e0 con 0x7f739006e6d0 2024-04-20T21:19:48.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.634+0000 7f7387e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f739006e6d0 0x7f7390071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:48.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.634+0000 7f7395000640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f7390146130 0x7f73900ccdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.86:49954/0 (socket says 172.21.15.86:49954) 2024-04-20T21:19:48.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.634+0000 7f7395000640 1 -- 172.21.15.86:0/3436427335 learned_addr learned my addr 172.21.15.86:0/3436427335 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:19:48.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.634+0000 7f7395a00640 1 --2- 172.21.15.86:0/3436427335 >> v2:172.21.15.144:3300/0 conn(0x7f7390072240 0x7f739015cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:48.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.635+0000 7f7387e00640 1 -- 172.21.15.86:0/3436427335 >> v2:172.21.15.144:3300/0 conn(0x7f7390072240 msgr2=0x7f739015cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:48.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.635+0000 7f7387e00640 1 --2- 172.21.15.86:0/3436427335 >> v2:172.21.15.144:3300/0 conn(0x7f7390072240 0x7f739015cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:48.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.635+0000 7f7387e00640 1 -- 172.21.15.86:0/3436427335 >> v1:172.21.15.144:6789/0 conn(0x7f7390146130 legacy=0x7f73900ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:48.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.635+0000 7f738fe00640 1 -- 172.21.15.86:0/3436427335 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3462083280 0 0) 0x7f7380001490 con 0x7f7390146130 2024-04-20T21:19:48.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.635+0000 7f7387e00640 1 -- 172.21.15.86:0/3436427335 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73900cdbc0 con 0x7f739006e6d0 2024-04-20T21:19:48.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.636+0000 7f7387e00640 1 --2- 172.21.15.86:0/3436427335 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f739006e6d0 0x7f7390071d00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f73780097a0 tx=0x7f737802b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c94711d38cb29ff server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:48.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.636+0000 7f738fe00640 1 -- 172.21.15.86:0/3436427335 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7378034070 con 0x7f739006e6d0 2024-04-20T21:19:48.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.637+0000 7f738fe00640 1 -- 172.21.15.86:0/3436427335 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7378030610 con 0x7f739006e6d0 2024-04-20T21:19:48.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.637+0000 7f738fe00640 1 -- 172.21.15.86:0/3436427335 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73780309e0 con 0x7f739006e6d0 2024-04-20T21:19:48.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.637+0000 7f7397a00640 1 -- 172.21.15.86:0/3436427335 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f739006e6d0 msgr2=0x7f7390071d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:48.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.637+0000 7f7397a00640 1 --2- 172.21.15.86:0/3436427335 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f739006e6d0 0x7f7390071d00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f73780097a0 tx=0x7f737802b960 comp rx=0 tx=0).stop 2024-04-20T21:19:48.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.637+0000 7f7397a00640 1 -- 172.21.15.86:0/3436427335 shutdown_connections 2024-04-20T21:19:48.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.637+0000 7f7397a00640 1 --2- 172.21.15.86:0/3436427335 >> v2:172.21.15.144:3300/0 conn(0x7f7390072240 0x7f739015cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:48.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.637+0000 7f7397a00640 1 --2- 172.21.15.86:0/3436427335 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f739006e6d0 0x7f7390071d00 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:48.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.637+0000 7f7397a00640 1 -- 172.21.15.86:0/3436427335 >> 172.21.15.86:0/3436427335 conn(0x7f73900b2360 msgr2=0x7f73900b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:48.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.637+0000 7f7397a00640 1 -- 172.21.15.86:0/3436427335 shutdown_connections 2024-04-20T21:19:48.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.637+0000 7f7397a00640 1 -- 172.21.15.86:0/3436427335 wait complete. 2024-04-20T21:19:48.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.639+0000 7f7397a00640 1 Processor -- start 2024-04-20T21:19:48.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.640+0000 7f7397a00640 1 -- start start 2024-04-20T21:19:48.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.641+0000 7f7397a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f739006e6d0 0x7f73901f8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:48.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.641+0000 7f7397a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7390072240 0x7f73900adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:48.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.641+0000 7f7397a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7390146130 0x7f73900b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:48.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.641+0000 7f7397a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f739006f150 con 0x7f7390072240 2024-04-20T21:19:48.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.641+0000 7f7397a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f739006e1b0 con 0x7f7390146130 2024-04-20T21:19:48.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.641+0000 7f7397a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f73900cd4e0 con 0x7f739006e6d0 2024-04-20T21:19:48.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.641+0000 7f7387e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7390072240 0x7f73900adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:48.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.641+0000 7f7387e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7390072240 0x7f73900adaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39580/0 (socket says 172.21.15.86:39580) 2024-04-20T21:19:48.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.641+0000 7f7387e00640 1 -- 172.21.15.86:0/3379011997 learned_addr learned my addr 172.21.15.86:0/3379011997 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:48.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.641+0000 7f7395000640 1 --2- 172.21.15.86:0/3379011997 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f739006e6d0 0x7f73901f8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:48.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.642+0000 7f7395a00640 1 --2- 172.21.15.86:0/3379011997 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7390146130 0x7f73900b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:48.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.642+0000 7f7387e00640 1 -- 172.21.15.86:0/3379011997 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f739006e6d0 msgr2=0x7f73901f8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:48.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.642+0000 7f7387e00640 1 --2- 172.21.15.86:0/3379011997 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f739006e6d0 0x7f73901f8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:48.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.642+0000 7f7387e00640 1 -- 172.21.15.86:0/3379011997 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7390146130 msgr2=0x7f73900b47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:48.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.642+0000 7f7387e00640 1 --2- 172.21.15.86:0/3379011997 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7390146130 0x7f73900b47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:48.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.642+0000 7f7387e00640 1 -- 172.21.15.86:0/3379011997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73900b5560 con 0x7f7390072240 2024-04-20T21:19:48.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.642+0000 7f7387e00640 1 --2- 172.21.15.86:0/3379011997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7390072240 0x7f73900adaf0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f73780097a0 tx=0x7f73780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:48.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.642+0000 7f738d600640 1 -- 172.21.15.86:0/3379011997 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7378049070 con 0x7f7390072240 2024-04-20T21:19:48.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.642+0000 7f738d600640 1 -- 172.21.15.86:0/3379011997 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7378030410 con 0x7f7390072240 2024-04-20T21:19:48.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.642+0000 7f738d600640 1 -- 172.21.15.86:0/3379011997 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7378034040 con 0x7f7390072240 2024-04-20T21:19:48.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.642+0000 7f7397a00640 1 -- 172.21.15.86:0/3379011997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73900b5280 con 0x7f7390072240 2024-04-20T21:19:48.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.643+0000 7f7397a00640 1 -- 172.21.15.86:0/3379011997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73901f8440 con 0x7f7390072240 2024-04-20T21:19:48.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.644+0000 7f7397a00640 1 -- 172.21.15.86:0/3379011997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73900689e0 con 0x7f7390072240 2024-04-20T21:19:48.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.645+0000 7f738d600640 1 -- 172.21.15.86:0/3379011997 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f737804c070 con 0x7f7390072240 2024-04-20T21:19:48.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.646+0000 7f738d600640 1 --2- 172.21.15.86:0/3379011997 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f736c06c8f0 0x7f736c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:48.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.646+0000 7f738d600640 1 -- 172.21.15.86:0/3379011997 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f73780ad740 con 0x7f7390072240 2024-04-20T21:19:48.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.646+0000 7f7395000640 1 --2- 172.21.15.86:0/3379011997 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f736c06c8f0 0x7f736c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:48.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.647+0000 7f7395000640 1 --2- 172.21.15.86:0/3379011997 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f736c06c8f0 0x7f736c06edb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7380007ed0 tx=0x7f7380007e60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:48.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.650+0000 7f738d600640 1 -- 172.21.15.86:0/3379011997 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f737807bad0 con 0x7f7390072240 2024-04-20T21:19:48.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.852+0000 7f7397a00640 1 -- 172.21.15.86:0/3379011997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f739015ff50 con 0x7f7390072240 2024-04-20T21:19:48.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.853+0000 7f738d600640 1 -- 172.21.15.86:0/3379011997 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16669 (secure 0 0 0) 0x7f737807b3d0 con 0x7f7390072240 2024-04-20T21:19:48.853 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:19:48.853 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":33,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:19:11.559613+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:19:07.732422+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-20T21:19:08.960268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:19:48.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.856+0000 7f7397a00640 1 -- 172.21.15.86:0/3379011997 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f736c06c8f0 msgr2=0x7f736c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:48.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.856+0000 7f7397a00640 1 --2- 172.21.15.86:0/3379011997 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f736c06c8f0 0x7f736c06edb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7380007ed0 tx=0x7f7380007e60 comp rx=0 tx=0).stop 2024-04-20T21:19:48.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.856+0000 7f7397a00640 1 -- 172.21.15.86:0/3379011997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7390072240 msgr2=0x7f73900adaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:48.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.856+0000 7f7397a00640 1 --2- 172.21.15.86:0/3379011997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7390072240 0x7f73900adaf0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f73780097a0 tx=0x7f73780046c0 comp rx=0 tx=0).stop 2024-04-20T21:19:48.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.856+0000 7f7397a00640 1 -- 172.21.15.86:0/3379011997 shutdown_connections 2024-04-20T21:19:48.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.856+0000 7f7397a00640 1 --2- 172.21.15.86:0/3379011997 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f736c06c8f0 0x7f736c06edb0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:48.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.856+0000 7f7397a00640 1 --2- 172.21.15.86:0/3379011997 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7390146130 0x7f73900b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:48.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.856+0000 7f7397a00640 1 --2- 172.21.15.86:0/3379011997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7390072240 0x7f73900adaf0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:48.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.856+0000 7f7397a00640 1 --2- 172.21.15.86:0/3379011997 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f739006e6d0 0x7f73901f8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:48.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.856+0000 7f7397a00640 1 -- 172.21.15.86:0/3379011997 >> 172.21.15.86:0/3379011997 conn(0x7f73900b2360 msgr2=0x7f73901444b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:48.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.856+0000 7f7397a00640 1 -- 172.21.15.86:0/3379011997 shutdown_connections 2024-04-20T21:19:48.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:48.856+0000 7f7397a00640 1 -- 172.21.15.86:0/3379011997 wait complete. 2024-04-20T21:19:48.875 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-20T21:19:49.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.077+0000 7efe79a00640 1 Processor -- start 2024-04-20T21:19:49.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.077+0000 7efe79a00640 1 -- start start 2024-04-20T21:19:49.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.078+0000 7efe79a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7efe74145fe0 0x7efe740d02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:49.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.078+0000 7efe79a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7efe740d0810 0x7efe740cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:49.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.078+0000 7efe79a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe740cda40 con 0x7efe740cd610 2024-04-20T21:19:49.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.078+0000 7efe79a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7efe740d00d0 con 0x7efe740d0810 2024-04-20T21:19:49.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.078+0000 7efe79a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7efe740d0c70 con 0x7efe74145fe0 2024-04-20T21:19:49.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.079+0000 7efe72000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7efe740d0810 0x7efe740cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:49.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.079+0000 7efe73400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7efe740cd610 0x7efe74160350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:47250/0 (socket says 172.21.15.86:47250) 2024-04-20T21:19:49.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.079+0000 7efe73400640 1 -- 172.21.15.86:0/3947319564 learned_addr learned my addr 172.21.15.86:0/3947319564 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:19:49.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.079+0000 7efe72000640 1 -- 172.21.15.86:0/3947319564 >> v2:172.21.15.144:3300/0 conn(0x7efe74145fe0 msgr2=0x7efe740d02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:49.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.079+0000 7efe72a00640 1 --2- 172.21.15.86:0/3947319564 >> v2:172.21.15.144:3300/0 conn(0x7efe74145fe0 0x7efe740d02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:49.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.079+0000 7efe71600640 1 -- 172.21.15.86:0/3947319564 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2413791207 0 0) 0x7efe740cda40 con 0x7efe740cd610 2024-04-20T21:19:49.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.079+0000 7efe72000640 1 --2- 172.21.15.86:0/3947319564 >> v2:172.21.15.144:3300/0 conn(0x7efe74145fe0 0x7efe740d02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:49.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.079+0000 7efe72000640 1 -- 172.21.15.86:0/3947319564 >> v1:172.21.15.86:6789/0 conn(0x7efe740cd610 legacy=0x7efe74160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:49.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.079+0000 7efe72000640 1 -- 172.21.15.86:0/3947319564 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe7415dcc0 con 0x7efe740d0810 2024-04-20T21:19:49.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.079+0000 7efe72a00640 1 --2- 172.21.15.86:0/3947319564 >> v2:172.21.15.144:3300/0 conn(0x7efe74145fe0 0x7efe740d02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:49.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.080+0000 7efe72000640 1 --2- 172.21.15.86:0/3947319564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe740d0810 0x7efe740cd0d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7efe7406d5c0 tx=0x7efe6002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70f1d43fe585d699 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:49.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.081+0000 7efe71600640 1 -- 172.21.15.86:0/3947319564 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe60035020 con 0x7efe740d0810 2024-04-20T21:19:49.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.081+0000 7efe71600640 1 -- 172.21.15.86:0/3947319564 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe60031830 con 0x7efe740d0810 2024-04-20T21:19:49.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.081+0000 7efe71600640 1 -- 172.21.15.86:0/3947319564 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe60031c00 con 0x7efe740d0810 2024-04-20T21:19:49.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.081+0000 7efe79a00640 1 -- 172.21.15.86:0/3947319564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe740d0810 msgr2=0x7efe740cd0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:49.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.081+0000 7efe79a00640 1 --2- 172.21.15.86:0/3947319564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe740d0810 0x7efe740cd0d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7efe7406d5c0 tx=0x7efe6002b8d0 comp rx=0 tx=0).stop 2024-04-20T21:19:49.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.081+0000 7efe79a00640 1 -- 172.21.15.86:0/3947319564 shutdown_connections 2024-04-20T21:19:49.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.081+0000 7efe79a00640 1 --2- 172.21.15.86:0/3947319564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe740d0810 0x7efe740cd0d0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:49.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.081+0000 7efe79a00640 1 --2- 172.21.15.86:0/3947319564 >> v2:172.21.15.144:3300/0 conn(0x7efe74145fe0 0x7efe740d02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:49.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.081+0000 7efe79a00640 1 -- 172.21.15.86:0/3947319564 >> 172.21.15.86:0/3947319564 conn(0x7efe7406b980 msgr2=0x7efe7406bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:49.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.082+0000 7efe79a00640 1 -- 172.21.15.86:0/3947319564 shutdown_connections 2024-04-20T21:19:49.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.082+0000 7efe79a00640 1 -- 172.21.15.86:0/3947319564 wait complete. 2024-04-20T21:19:49.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.083+0000 7efe79a00640 1 Processor -- start 2024-04-20T21:19:49.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.083+0000 7efe79a00640 1 -- start start 2024-04-20T21:19:49.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.084+0000 7efe79a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe740cd610 0x7efe740affa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:49.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.084+0000 7efe79a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe740d0810 0x7efe741f5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:49.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.084+0000 7efe79a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe74145fe0 0x7efe740b1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:49.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.084+0000 7efe79a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efe7415d1e0 con 0x7efe740d0810 2024-04-20T21:19:49.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.084+0000 7efe79a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efe7415d4c0 con 0x7efe740cd610 2024-04-20T21:19:49.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.084+0000 7efe79a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7efe7415d7a0 con 0x7efe74145fe0 2024-04-20T21:19:49.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.085+0000 7efe72000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe740d0810 0x7efe741f5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:49.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.085+0000 7efe72000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe740d0810 0x7efe741f5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39592/0 (socket says 172.21.15.86:39592) 2024-04-20T21:19:49.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.085+0000 7efe72000640 1 -- 172.21.15.86:0/970163185 learned_addr learned my addr 172.21.15.86:0/970163185 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:49.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.085+0000 7efe72a00640 1 --2- 172.21.15.86:0/970163185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe740cd610 0x7efe740affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:49.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.085+0000 7efe72000640 1 -- 172.21.15.86:0/970163185 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe74145fe0 msgr2=0x7efe740b1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:49.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.085+0000 7efe72000640 1 --2- 172.21.15.86:0/970163185 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe74145fe0 0x7efe740b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:49.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.085+0000 7efe72000640 1 -- 172.21.15.86:0/970163185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe740cd610 msgr2=0x7efe740affa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:49.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.085+0000 7efe72000640 1 --2- 172.21.15.86:0/970163185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe740cd610 0x7efe740affa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:49.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.085+0000 7efe72000640 1 -- 172.21.15.86:0/970163185 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe741f5990 con 0x7efe740d0810 2024-04-20T21:19:49.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.085+0000 7efe72000640 1 --2- 172.21.15.86:0/970163185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe740d0810 0x7efe741f5450 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7efe60000c00 tx=0x7efe600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:49.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.086+0000 7efe6aa00640 1 -- 172.21.15.86:0/970163185 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe60035050 con 0x7efe740d0810 2024-04-20T21:19:49.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.086+0000 7efe6aa00640 1 -- 172.21.15.86:0/970163185 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe60004230 con 0x7efe740d0810 2024-04-20T21:19:49.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.086+0000 7efe79a00640 1 -- 172.21.15.86:0/970163185 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe740b05f0 con 0x7efe740d0810 2024-04-20T21:19:49.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.086+0000 7efe6aa00640 1 -- 172.21.15.86:0/970163185 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6003b800 con 0x7efe740d0810 2024-04-20T21:19:49.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.086+0000 7efe79a00640 1 -- 172.21.15.86:0/970163185 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe740b0bb0 con 0x7efe740d0810 2024-04-20T21:19:49.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.087+0000 7efe79a00640 1 -- 172.21.15.86:0/970163185 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe740689e0 con 0x7efe740d0810 2024-04-20T21:19:49.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.088+0000 7efe6aa00640 1 -- 172.21.15.86:0/970163185 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efe60038020 con 0x7efe740d0810 2024-04-20T21:19:49.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.089+0000 7efe6aa00640 1 --2- 172.21.15.86:0/970163185 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efe5406c8f0 0x7efe5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:49.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.089+0000 7efe6aa00640 1 -- 172.21.15.86:0/970163185 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7efe600ada20 con 0x7efe740d0810 2024-04-20T21:19:49.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.089+0000 7efe72a00640 1 --2- 172.21.15.86:0/970163185 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efe5406c8f0 0x7efe5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:49.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.090+0000 7efe72a00640 1 --2- 172.21.15.86:0/970163185 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efe5406c8f0 0x7efe5406edb0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7efe6400a950 tx=0x7efe64008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:49.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.093+0000 7efe6aa00640 1 -- 172.21.15.86:0/970163185 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7efe60040070 con 0x7efe740d0810 2024-04-20T21:19:49.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.332+0000 7efe79a00640 1 -- 172.21.15.86:0/970163185 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7efe741eab80 con 0x7efe740d0810 2024-04-20T21:19:49.332 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-04-20T21:19:49.333+0000 7f5bb9c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T21:19:49.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.337+0000 7efe6aa00640 1 -- 172.21.15.86:0/970163185 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) ==== 161+0+0 (secure 0 0 0) 0x7efe6007bed0 con 0x7efe740d0810 2024-04-20T21:19:49.336 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-20T21:19:49.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.341+0000 7efe79a00640 1 -- 172.21.15.86:0/970163185 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efe5406c8f0 msgr2=0x7efe5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:49.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.341+0000 7efe79a00640 1 --2- 172.21.15.86:0/970163185 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efe5406c8f0 0x7efe5406edb0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7efe6400a950 tx=0x7efe64008040 comp rx=0 tx=0).stop 2024-04-20T21:19:49.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.341+0000 7efe79a00640 1 -- 172.21.15.86:0/970163185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe740d0810 msgr2=0x7efe741f5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:49.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.341+0000 7efe79a00640 1 --2- 172.21.15.86:0/970163185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe740d0810 0x7efe741f5450 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7efe60000c00 tx=0x7efe600046c0 comp rx=0 tx=0).stop 2024-04-20T21:19:49.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.341+0000 7efe79a00640 1 -- 172.21.15.86:0/970163185 shutdown_connections 2024-04-20T21:19:49.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.341+0000 7efe79a00640 1 --2- 172.21.15.86:0/970163185 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efe5406c8f0 0x7efe5406edb0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:49.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.341+0000 7efe79a00640 1 --2- 172.21.15.86:0/970163185 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe74145fe0 0x7efe740b1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:49.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.341+0000 7efe79a00640 1 --2- 172.21.15.86:0/970163185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe740d0810 0x7efe741f5450 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:49.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.341+0000 7efe79a00640 1 --2- 172.21.15.86:0/970163185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe740cd610 0x7efe740affa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:49.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.341+0000 7efe79a00640 1 -- 172.21.15.86:0/970163185 >> 172.21.15.86:0/970163185 conn(0x7efe7406b980 msgr2=0x7efe7415e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:49.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.341+0000 7efe79a00640 1 -- 172.21.15.86:0/970163185 shutdown_connections 2024-04-20T21:19:49.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.342+0000 7efe79a00640 1 -- 172.21.15.86:0/970163185 wait complete. 2024-04-20T21:19:49.357 INFO:tasks.ceph.mds.a.smithi086.stderr: -6263> 2024-04-20T21:19:13.474+0000 7f35c0200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T21:19:49.366 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-20T21:19:49.412 INFO:tasks.ceph.mds.a.smithi086.stdout:starting mds.a at 2024-04-20T21:19:49.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.550+0000 7f7b1aa00640 1 Processor -- start 2024-04-20T21:19:49.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.551+0000 7f7b1aa00640 1 -- start start 2024-04-20T21:19:49.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.552+0000 7f7b1aa00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7b140cfc70 0x7f7b140cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:49.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.552+0000 7f7b1aa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b140cdbe0 0x7f7b1406eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:49.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.552+0000 7f7b1aa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7b140b3ba0 0x7f7b1406f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:49.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.552+0000 7f7b1aa00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f7b1415c8e0 con 0x7f7b140b3ba0 2024-04-20T21:19:49.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.552+0000 7f7b1aa00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f7b1415bcd0 con 0x7f7b140cfc70 2024-04-20T21:19:49.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.552+0000 7f7b1aa00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7b1415bfb0 con 0x7f7b140cdbe0 2024-04-20T21:19:49.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.552+0000 7f7b1a000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7b140b3ba0 0x7f7b1406f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:49.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.553+0000 7f7b1a000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7b140b3ba0 0x7f7b1406f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39606/0 (socket says 172.21.15.86:39606) 2024-04-20T21:19:49.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.553+0000 7f7b1a000640 1 -- 172.21.15.86:0/3323164406 learned_addr learned my addr 172.21.15.86:0/3323164406 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:49.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.553+0000 7f7b1a000640 1 -- 172.21.15.86:0/3323164406 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b140cdbe0 msgr2=0x7f7b1406eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:49.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.553+0000 7f7b1a000640 1 --2- 172.21.15.86:0/3323164406 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b140cdbe0 0x7f7b1406eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:49.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.553+0000 7f7b1a000640 1 -- 172.21.15.86:0/3323164406 >> v2:172.21.15.144:3300/0 conn(0x7f7b140cfc70 msgr2=0x7f7b140cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:49.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.553+0000 7f7b1a000640 1 --2- 172.21.15.86:0/3323164406 >> v2:172.21.15.144:3300/0 conn(0x7f7b140cfc70 0x7f7b140cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:49.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.553+0000 7f7b1a000640 1 -- 172.21.15.86:0/3323164406 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b1415c290 con 0x7f7b140b3ba0 2024-04-20T21:19:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.554+0000 7f7b1a000640 1 --2- 172.21.15.86:0/3323164406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b140b3ba0 0x7f7b1406f3f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f7b1406fac0 tx=0x7f7b0c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c601292154652221 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.555+0000 7f7b13e00640 1 -- 172.21.15.86:0/3323164406 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c02ca70 con 0x7f7b140b3ba0 2024-04-20T21:19:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.555+0000 7f7b13e00640 1 -- 172.21.15.86:0/3323164406 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b0c02cc10 con 0x7f7b140b3ba0 2024-04-20T21:19:49.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.555+0000 7f7b13e00640 1 -- 172.21.15.86:0/3323164406 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c0343f0 con 0x7f7b140b3ba0 2024-04-20T21:19:49.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.555+0000 7f7b1aa00640 1 -- 172.21.15.86:0/3323164406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b140b3ba0 msgr2=0x7f7b1406f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:49.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.555+0000 7f7b1aa00640 1 --2- 172.21.15.86:0/3323164406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b140b3ba0 0x7f7b1406f3f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f7b1406fac0 tx=0x7f7b0c02bed0 comp rx=0 tx=0).stop 2024-04-20T21:19:49.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.555+0000 7f7b1aa00640 1 -- 172.21.15.86:0/3323164406 shutdown_connections 2024-04-20T21:19:49.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.555+0000 7f7b1aa00640 1 --2- 172.21.15.86:0/3323164406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b140b3ba0 0x7f7b1406f3f0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:49.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.555+0000 7f7b1aa00640 1 --2- 172.21.15.86:0/3323164406 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b140cdbe0 0x7f7b1406eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:49.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.555+0000 7f7b1aa00640 1 --2- 172.21.15.86:0/3323164406 >> v2:172.21.15.144:3300/0 conn(0x7f7b140cfc70 0x7f7b140cd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:49.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.555+0000 7f7b1aa00640 1 -- 172.21.15.86:0/3323164406 >> 172.21.15.86:0/3323164406 conn(0x7f7b14067cf0 msgr2=0x7f7b1406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:49.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.555+0000 7f7b1aa00640 1 -- 172.21.15.86:0/3323164406 shutdown_connections 2024-04-20T21:19:49.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.556+0000 7f7b1aa00640 1 -- 172.21.15.86:0/3323164406 wait complete. 2024-04-20T21:19:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.557+0000 7f7b1aa00640 1 Processor -- start 2024-04-20T21:19:49.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.557+0000 7f7b1aa00640 1 -- start start 2024-04-20T21:19:49.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.558+0000 7f7b1aa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b140b3ba0 0x7f7b14157540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:49.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.558+0000 7f7b1aa00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b140cdbe0 0x7f7b141553a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:49.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.558+0000 7f7b1aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b140cfc70 0x7f7b141509d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:49.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.558+0000 7f7b1aa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7b140d0050 con 0x7f7b140cfc70 2024-04-20T21:19:49.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.558+0000 7f7b1aa00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7b140b3710 con 0x7f7b140cdbe0 2024-04-20T21:19:49.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.558+0000 7f7b1aa00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7b140d01d0 con 0x7f7b140b3ba0 2024-04-20T21:19:49.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.559+0000 7f7b1a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b140cfc70 0x7f7b141509d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:49.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.559+0000 7f7b1a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b140cfc70 0x7f7b141509d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39618/0 (socket says 172.21.15.86:39618) 2024-04-20T21:19:49.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.559+0000 7f7b1a000640 1 -- 172.21.15.86:0/791597648 learned_addr learned my addr 172.21.15.86:0/791597648 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.559+0000 7f7b19600640 1 --2- 172.21.15.86:0/791597648 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b140b3ba0 0x7f7b14157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.559+0000 7f7b18c00640 1 --2- 172.21.15.86:0/791597648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b140cdbe0 0x7f7b141553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.559+0000 7f7b1a000640 1 -- 172.21.15.86:0/791597648 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b140b3ba0 msgr2=0x7f7b14157540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.559+0000 7f7b1a000640 1 --2- 172.21.15.86:0/791597648 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b140b3ba0 0x7f7b14157540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.559+0000 7f7b1a000640 1 -- 172.21.15.86:0/791597648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b140cdbe0 msgr2=0x7f7b141553a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.559+0000 7f7b1a000640 1 --2- 172.21.15.86:0/791597648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b140cdbe0 0x7f7b141553a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.559+0000 7f7b1a000640 1 -- 172.21.15.86:0/791597648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b141567f0 con 0x7f7b140cfc70 2024-04-20T21:19:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.560+0000 7f7b1a000640 1 --2- 172.21.15.86:0/791597648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b140cfc70 0x7f7b141509d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c036040 tx=0x7f7b0c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.560+0000 7f7b11600640 1 -- 172.21.15.86:0/791597648 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c043020 con 0x7f7b140cfc70 2024-04-20T21:19:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.560+0000 7f7b11600640 1 -- 172.21.15.86:0/791597648 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b0c03e450 con 0x7f7b140cfc70 2024-04-20T21:19:49.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.560+0000 7f7b1aa00640 1 -- 172.21.15.86:0/791597648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b14156a60 con 0x7f7b140cfc70 2024-04-20T21:19:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.560+0000 7f7b11600640 1 -- 172.21.15.86:0/791597648 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c009510 con 0x7f7b140cfc70 2024-04-20T21:19:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.560+0000 7f7b1aa00640 1 -- 172.21.15.86:0/791597648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b141f4040 con 0x7f7b140cfc70 2024-04-20T21:19:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.561+0000 7f7b1aa00640 1 -- 172.21.15.86:0/791597648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b1404fd90 con 0x7f7b140cfc70 2024-04-20T21:19:49.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.562+0000 7f7b11600640 1 -- 172.21.15.86:0/791597648 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7b0c046020 con 0x7f7b140cfc70 2024-04-20T21:19:49.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.563+0000 7f7b11600640 1 --2- 172.21.15.86:0/791597648 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7af406c8f0 0x7f7af406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:49.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.563+0000 7f7b11600640 1 -- 172.21.15.86:0/791597648 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f7b0c0ae540 con 0x7f7b140cfc70 2024-04-20T21:19:49.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.563+0000 7f7b19600640 1 --2- 172.21.15.86:0/791597648 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7af406c8f0 0x7f7af406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:49.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.564+0000 7f7b19600640 1 --2- 172.21.15.86:0/791597648 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7af406c8f0 0x7f7af406edb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7b140afa80 tx=0x7f7b08008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:49.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.568+0000 7f7b11600640 1 -- 172.21.15.86:0/791597648 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7b0c04d030 con 0x7f7b140cfc70 2024-04-20T21:19:49.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:49.817+0000 7f7b1aa00640 1 -- 172.21.15.86:0/791597648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f7b14152390 con 0x7f7b140cfc70 2024-04-20T21:19:50.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.340+0000 7f7b11600640 1 -- 172.21.15.86:0/791597648 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) ==== 104+0+0 (secure 0 0 0) 0x7f7b0c07c910 con 0x7f7b140cfc70 2024-04-20T21:19:50.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.344+0000 7f7b1aa00640 1 -- 172.21.15.86:0/791597648 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7af406c8f0 msgr2=0x7f7af406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:50.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.344+0000 7f7b1aa00640 1 --2- 172.21.15.86:0/791597648 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7af406c8f0 0x7f7af406edb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7b140afa80 tx=0x7f7b08008040 comp rx=0 tx=0).stop 2024-04-20T21:19:50.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.344+0000 7f7b1aa00640 1 -- 172.21.15.86:0/791597648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b140cfc70 msgr2=0x7f7b141509d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:50.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.344+0000 7f7b1aa00640 1 --2- 172.21.15.86:0/791597648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b140cfc70 0x7f7b141509d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c036040 tx=0x7f7b0c004a40 comp rx=0 tx=0).stop 2024-04-20T21:19:50.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.344+0000 7f7b1aa00640 1 -- 172.21.15.86:0/791597648 shutdown_connections 2024-04-20T21:19:50.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.344+0000 7f7b1aa00640 1 --2- 172.21.15.86:0/791597648 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7af406c8f0 0x7f7af406edb0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:50.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.344+0000 7f7b1aa00640 1 --2- 172.21.15.86:0/791597648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b140cfc70 0x7f7b141509d0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:50.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.344+0000 7f7b1aa00640 1 --2- 172.21.15.86:0/791597648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b140cdbe0 0x7f7b141553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:50.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.344+0000 7f7b1aa00640 1 --2- 172.21.15.86:0/791597648 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b140b3ba0 0x7f7b14157540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:50.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.344+0000 7f7b1aa00640 1 -- 172.21.15.86:0/791597648 >> 172.21.15.86:0/791597648 conn(0x7f7b14067cf0 msgr2=0x7f7b1406e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:50.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.344+0000 7f7b1aa00640 1 -- 172.21.15.86:0/791597648 shutdown_connections 2024-04-20T21:19:50.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.344+0000 7f7b1aa00640 1 -- 172.21.15.86:0/791597648 wait complete. 2024-04-20T21:19:50.364 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-20T21:19:50.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.620+0000 7f6986a00640 1 Processor -- start 2024-04-20T21:19:50.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.620+0000 7f6986a00640 1 -- start start 2024-04-20T21:19:50.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.622+0000 7f6986a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6980167340 0x7f698006f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:50.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.622+0000 7f6986a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6980167790 con 0x7f6980072410 2024-04-20T21:19:50.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.622+0000 7f6986a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f698006f810 con 0x7f6980162bb0 2024-04-20T21:19:50.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.622+0000 7f6986a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f698006f5b0 con 0x7f6980167340 2024-04-20T21:19:50.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.622+0000 7f697f400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f6980072410 0x7f698006dc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:47282/0 (socket says 172.21.15.86:47282) 2024-04-20T21:19:50.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.622+0000 7f697f400640 1 -- 172.21.15.86:0/1867442499 learned_addr learned my addr 172.21.15.86:0/1867442499 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:19:50.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.623+0000 7f697ea00640 1 -- 172.21.15.86:0/1867442499 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4148989722 0 0) 0x7f6980167790 con 0x7f6980072410 2024-04-20T21:19:50.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.623+0000 7f697ea00640 1 -- 172.21.15.86:0/1867442499 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6960003520 con 0x7f6980072410 2024-04-20T21:19:50.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.623+0000 7f6984a00640 1 --2- 172.21.15.86:0/1867442499 >> v2:172.21.15.144:3300/0 conn(0x7f6980167340 0x7f698006f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:50.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.623+0000 7f697ea00640 1 -- 172.21.15.86:0/1867442499 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1978903923 0 0) 0x7f6960003520 con 0x7f6980072410 2024-04-20T21:19:50.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.623+0000 7f697ea00640 1 -- 172.21.15.86:0/1867442499 >> v2:172.21.15.144:3300/0 conn(0x7f6980167340 msgr2=0x7f698006f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:50.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.623+0000 7f697ea00640 1 --2- 172.21.15.86:0/1867442499 >> v2:172.21.15.144:3300/0 conn(0x7f6980167340 0x7f698006f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:50.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.623+0000 7f697ea00640 1 -- 172.21.15.86:0/1867442499 >> v1:172.21.15.144:6789/0 conn(0x7f6980162bb0 legacy=0x7f6980071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:50.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.623+0000 7f697ea00640 1 -- 172.21.15.86:0/1867442499 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69800b4840 con 0x7f6980072410 2024-04-20T21:19:50.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.623+0000 7f697ea00640 1 -- 172.21.15.86:0/1867442499 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f6968001b70 con 0x7f6980072410 2024-04-20T21:19:50.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.623+0000 7f697ea00640 1 -- 172.21.15.86:0/1867442499 >> v1:172.21.15.86:6789/0 conn(0x7f6980072410 legacy=0x7f698006dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:50.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.623+0000 7f697ea00640 1 --2- 172.21.15.86:0/1867442499 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69600039b0 0x7f6960003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:50.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.623+0000 7f697ea00640 1 --2- 172.21.15.86:0/1867442499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6960004800 0x7f6960004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:50.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.623+0000 7f697ea00640 1 --2- 172.21.15.86:0/1867442499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69600052e0 0x7f696000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:50.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.623+0000 7f697ea00640 1 -- 172.21.15.86:0/1867442499 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f698006f5b0 con 0x7f69600052e0 2024-04-20T21:19:50.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.623+0000 7f697ea00640 1 -- 172.21.15.86:0/1867442499 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f696000c1b0 con 0x7f6960004800 2024-04-20T21:19:50.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.623+0000 7f697ea00640 1 -- 172.21.15.86:0/1867442499 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f696000c3b0 con 0x7f69600039b0 2024-04-20T21:19:50.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.624+0000 7f6984a00640 1 --2- 172.21.15.86:0/1867442499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69600052e0 0x7f696000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:50.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.624+0000 7f697fe00640 1 --2- 172.21.15.86:0/1867442499 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69600039b0 0x7f6960003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:50.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.624+0000 7f697f400640 1 --2- 172.21.15.86:0/1867442499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6960004800 0x7f6960004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:50.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.624+0000 7f6984a00640 1 -- 172.21.15.86:0/1867442499 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69600039b0 msgr2=0x7f6960003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:50.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.624+0000 7f6984a00640 1 --2- 172.21.15.86:0/1867442499 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69600039b0 0x7f6960003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:50.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.624+0000 7f6984a00640 1 -- 172.21.15.86:0/1867442499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6960004800 msgr2=0x7f6960004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:50.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.624+0000 7f6984a00640 1 --2- 172.21.15.86:0/1867442499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6960004800 0x7f6960004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:50.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.624+0000 7f6984a00640 1 -- 172.21.15.86:0/1867442499 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6960003520 con 0x7f69600052e0 2024-04-20T21:19:50.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.624+0000 7f697f400640 1 --2- 172.21.15.86:0/1867442499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6960004800 0x7f6960004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:50.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.625+0000 7f6984a00640 1 -- 172.21.15.86:0/1867442499 reap_dead start 2024-04-20T21:19:50.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.625+0000 7f6984a00640 1 --2- 172.21.15.86:0/1867442499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69600052e0 0x7f696000b9c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f697800ba00 tx=0x7f6960005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=218e22096e811751 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:50.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.625+0000 7f697ea00640 1 -- 172.21.15.86:0/1867442499 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6978036030 con 0x7f69600052e0 2024-04-20T21:19:50.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.626+0000 7f697ea00640 1 -- 172.21.15.86:0/1867442499 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6978004020 con 0x7f69600052e0 2024-04-20T21:19:50.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.626+0000 7f697ea00640 1 -- 172.21.15.86:0/1867442499 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697800b600 con 0x7f69600052e0 2024-04-20T21:19:50.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.626+0000 7f6986a00640 1 -- 172.21.15.86:0/1867442499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69600052e0 msgr2=0x7f696000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:50.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.626+0000 7f6986a00640 1 --2- 172.21.15.86:0/1867442499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69600052e0 0x7f696000b9c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f697800ba00 tx=0x7f6960005830 comp rx=0 tx=0).stop 2024-04-20T21:19:50.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.626+0000 7f6986a00640 1 -- 172.21.15.86:0/1867442499 shutdown_connections 2024-04-20T21:19:50.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.626+0000 7f6986a00640 1 --2- 172.21.15.86:0/1867442499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69600052e0 0x7f696000b9c0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:50.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.626+0000 7f6986a00640 1 -- 172.21.15.86:0/1867442499 >> 172.21.15.86:0/1867442499 conn(0x7f69800b2180 msgr2=0x7f69800b2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:50.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.626+0000 7f6986a00640 1 -- 172.21.15.86:0/1867442499 shutdown_connections 2024-04-20T21:19:50.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.626+0000 7f6986a00640 1 -- 172.21.15.86:0/1867442499 wait complete. 2024-04-20T21:19:50.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.628+0000 7f6986a00640 1 Processor -- start 2024-04-20T21:19:50.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.628+0000 7f6986a00640 1 -- start start 2024-04-20T21:19:50.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.629+0000 7f6986a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69600052e0 0x7f6980165c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:50.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.629+0000 7f6986a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f698015a5a0 0x7f698015bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:50.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.629+0000 7f6986a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69800aecf0 0x7f69801602a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:50.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.629+0000 7f6986a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f69800b50e0 con 0x7f69600052e0 2024-04-20T21:19:50.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.629+0000 7f6986a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f69800b4e00 con 0x7f698015a5a0 2024-04-20T21:19:50.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.629+0000 7f6986a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f69800b5600 con 0x7f69800aecf0 2024-04-20T21:19:50.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.629+0000 7f697fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69600052e0 0x7f6980165c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:50.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.629+0000 7f697fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69600052e0 0x7f6980165c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39632/0 (socket says 172.21.15.86:39632) 2024-04-20T21:19:50.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.629+0000 7f697fe00640 1 -- 172.21.15.86:0/3444480209 learned_addr learned my addr 172.21.15.86:0/3444480209 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:50.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.630+0000 7f697fe00640 1 -- 172.21.15.86:0/3444480209 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69800aecf0 msgr2=0x7f69801602a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:50.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.630+0000 7f697fe00640 1 --2- 172.21.15.86:0/3444480209 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69800aecf0 0x7f69801602a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:50.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.630+0000 7f697fe00640 1 -- 172.21.15.86:0/3444480209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f698015a5a0 msgr2=0x7f698015bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:50.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.630+0000 7f697fe00640 1 --2- 172.21.15.86:0/3444480209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f698015a5a0 0x7f698015bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:50.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.630+0000 7f697fe00640 1 -- 172.21.15.86:0/3444480209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f698015ab60 con 0x7f69600052e0 2024-04-20T21:19:50.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.630+0000 7f697fe00640 1 --2- 172.21.15.86:0/3444480209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69600052e0 0x7f6980165c30 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f697000d520 tx=0x7f697000d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:50.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.630+0000 7f6977e00640 1 -- 172.21.15.86:0/3444480209 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6970005480 con 0x7f69600052e0 2024-04-20T21:19:50.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.630+0000 7f6977e00640 1 -- 172.21.15.86:0/3444480209 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6970017070 con 0x7f69600052e0 2024-04-20T21:19:50.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.630+0000 7f6977e00640 1 -- 172.21.15.86:0/3444480209 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6970003850 con 0x7f69600052e0 2024-04-20T21:19:50.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.630+0000 7f6986a00640 1 -- 172.21.15.86:0/3444480209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f698015ae40 con 0x7f69600052e0 2024-04-20T21:19:50.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.630+0000 7f6986a00640 1 -- 172.21.15.86:0/3444480209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f698015b400 con 0x7f69600052e0 2024-04-20T21:19:50.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.631+0000 7f6986a00640 1 -- 172.21.15.86:0/3444480209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69800689e0 con 0x7f69600052e0 2024-04-20T21:19:50.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.632+0000 7f6977e00640 1 -- 172.21.15.86:0/3444480209 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f69700039f0 con 0x7f69600052e0 2024-04-20T21:19:50.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.633+0000 7f6977e00640 1 --2- 172.21.15.86:0/3444480209 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f695c06c920 0x7f695c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:50.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.634+0000 7f6977e00640 1 -- 172.21.15.86:0/3444480209 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5795+0+0 (secure 0 0 0) 0x7f697008b930 con 0x7f69600052e0 2024-04-20T21:19:50.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.634+0000 7f697f400640 1 --2- 172.21.15.86:0/3444480209 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f695c06c920 0x7f695c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:50.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.634+0000 7f697f400640 1 --2- 172.21.15.86:0/3444480209 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f695c06c920 0x7f695c06ede0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6968001270 tx=0x7f6968009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:50.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.638+0000 7f6977e00640 1 -- 172.21.15.86:0/3444480209 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6970059c60 con 0x7f69600052e0 2024-04-20T21:19:50.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:50.833+0000 7f6986a00640 1 -- 172.21.15.86:0/3444480209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6980161bb0 con 0x7f69600052e0 2024-04-20T21:19:51.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:51.792+0000 7f6977e00640 1 -- 172.21.15.86:0/3444480209 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v35) ==== 181+0+0 (secure 0 0 0) 0x7f6970059560 con 0x7f69600052e0 2024-04-20T21:19:51.791 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' removed 2024-04-20T21:19:51.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:51.795+0000 7f6986a00640 1 -- 172.21.15.86:0/3444480209 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f695c06c920 msgr2=0x7f695c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:51.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:51.795+0000 7f6986a00640 1 --2- 172.21.15.86:0/3444480209 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f695c06c920 0x7f695c06ede0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6968001270 tx=0x7f6968009040 comp rx=0 tx=0).stop 2024-04-20T21:19:51.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:51.795+0000 7f6986a00640 1 -- 172.21.15.86:0/3444480209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69600052e0 msgr2=0x7f6980165c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:51.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:51.795+0000 7f6986a00640 1 --2- 172.21.15.86:0/3444480209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69600052e0 0x7f6980165c30 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f697000d520 tx=0x7f697000d9e0 comp rx=0 tx=0).stop 2024-04-20T21:19:51.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:51.796+0000 7f6986a00640 1 -- 172.21.15.86:0/3444480209 shutdown_connections 2024-04-20T21:19:51.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:51.796+0000 7f6986a00640 1 --2- 172.21.15.86:0/3444480209 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f695c06c920 0x7f695c06ede0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:51.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:51.796+0000 7f6986a00640 1 --2- 172.21.15.86:0/3444480209 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69800aecf0 0x7f69801602a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:51.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:51.796+0000 7f6986a00640 1 --2- 172.21.15.86:0/3444480209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f698015a5a0 0x7f698015bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:51.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:51.796+0000 7f6986a00640 1 --2- 172.21.15.86:0/3444480209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69600052e0 0x7f6980165c30 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:51.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:51.796+0000 7f6986a00640 1 -- 172.21.15.86:0/3444480209 >> 172.21.15.86:0/3444480209 conn(0x7f69800b2180 msgr2=0x7f69800b0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:51.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:51.796+0000 7f6986a00640 1 -- 172.21.15.86:0/3444480209 shutdown_connections 2024-04-20T21:19:51.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:51.796+0000 7f6986a00640 1 -- 172.21.15.86:0/3444480209 wait complete. 2024-04-20T21:19:51.821 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-20T21:19:52.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.015+0000 7f216f200640 1 Processor -- start 2024-04-20T21:19:52.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.016+0000 7f216f200640 1 -- start start 2024-04-20T21:19:52.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.017+0000 7f216f200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2170153590 0x7f21700d0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:52.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.017+0000 7f216f200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f21700af870 0x7f21700d0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:52.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.017+0000 7f216f200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f217015cbe0 con 0x7f2170071ad0 2024-04-20T21:19:52.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.017+0000 7f216f200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f2170153970 con 0x7f21700af870 2024-04-20T21:19:52.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.017+0000 7f216f200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f217015cea0 con 0x7f2170153590 2024-04-20T21:19:52.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.018+0000 7f216e800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f21700af870 0x7f21700d0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:52.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.018+0000 7f2167e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f2170071ad0 0x7f21700af160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:47296/0 (socket says 172.21.15.86:47296) 2024-04-20T21:19:52.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.018+0000 7f2167e00640 1 -- 172.21.15.86:0/1240404456 learned_addr learned my addr 172.21.15.86:0/1240404456 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:19:52.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.018+0000 7f216de00640 1 --2- 172.21.15.86:0/1240404456 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2170153590 0x7f21700d0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:52.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.018+0000 7f216e800640 1 -- 172.21.15.86:0/1240404456 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2170153590 msgr2=0x7f21700d0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:52.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.018+0000 7f216e800640 1 --2- 172.21.15.86:0/1240404456 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2170153590 0x7f21700d0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:52.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.018+0000 7f216d400640 1 -- 172.21.15.86:0/1240404456 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4287698368 0 0) 0x7f217015cbe0 con 0x7f2170071ad0 2024-04-20T21:19:52.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.018+0000 7f216e800640 1 -- 172.21.15.86:0/1240404456 >> v1:172.21.15.86:6789/0 conn(0x7f2170071ad0 legacy=0x7f21700af160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:52.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.018+0000 7f216e800640 1 -- 172.21.15.86:0/1240404456 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f217015d180 con 0x7f21700af870 2024-04-20T21:19:52.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.019+0000 7f216e800640 1 --2- 172.21.15.86:0/1240404456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21700af870 0x7f21700d0b50 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f2168009d90 tx=0x7f216802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e89f65164ed13b1b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:52.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.020+0000 7f216d400640 1 -- 172.21.15.86:0/1240404456 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2168039070 con 0x7f21700af870 2024-04-20T21:19:52.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.020+0000 7f216d400640 1 -- 172.21.15.86:0/1240404456 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f216802cc50 con 0x7f21700af870 2024-04-20T21:19:52.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.020+0000 7f216d400640 1 -- 172.21.15.86:0/1240404456 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2168034440 con 0x7f21700af870 2024-04-20T21:19:52.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.020+0000 7f216f200640 1 -- 172.21.15.86:0/1240404456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21700af870 msgr2=0x7f21700d0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:52.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.020+0000 7f216f200640 1 --2- 172.21.15.86:0/1240404456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21700af870 0x7f21700d0b50 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f2168009d90 tx=0x7f216802bea0 comp rx=0 tx=0).stop 2024-04-20T21:19:52.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.021+0000 7f216f200640 1 -- 172.21.15.86:0/1240404456 shutdown_connections 2024-04-20T21:19:52.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.021+0000 7f216f200640 1 --2- 172.21.15.86:0/1240404456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21700af870 0x7f21700d0b50 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:52.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.021+0000 7f216f200640 1 --2- 172.21.15.86:0/1240404456 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2170153590 0x7f21700d0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:52.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.021+0000 7f216f200640 1 -- 172.21.15.86:0/1240404456 >> 172.21.15.86:0/1240404456 conn(0x7f21700b2180 msgr2=0x7f21700b2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:52.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.021+0000 7f216f200640 1 -- 172.21.15.86:0/1240404456 shutdown_connections 2024-04-20T21:19:52.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.021+0000 7f216f200640 1 -- 172.21.15.86:0/1240404456 wait complete. 2024-04-20T21:19:52.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.022+0000 7f216f200640 1 Processor -- start 2024-04-20T21:19:52.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.023+0000 7f216f200640 1 -- start start 2024-04-20T21:19:52.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.023+0000 7f216f200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2170071ad0 0x7f2170146d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:52.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.023+0000 7f216f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21700af870 0x7f2170150140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:52.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.023+0000 7f216f200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2170153590 0x7f2170150680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:52.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.023+0000 7f216f200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f217006dce0 con 0x7f21700af870 2024-04-20T21:19:52.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.023+0000 7f216f200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f217015d740 con 0x7f2170153590 2024-04-20T21:19:52.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.023+0000 7f216f200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f217006de60 con 0x7f2170071ad0 2024-04-20T21:19:52.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.024+0000 7f2167e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21700af870 0x7f2170150140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:52.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.024+0000 7f2167e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21700af870 0x7f2170150140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39652/0 (socket says 172.21.15.86:39652) 2024-04-20T21:19:52.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.024+0000 7f2167e00640 1 -- 172.21.15.86:0/202728875 learned_addr learned my addr 172.21.15.86:0/202728875 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:52.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.024+0000 7f2167e00640 1 -- 172.21.15.86:0/202728875 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2170071ad0 msgr2=0x7f2170146d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:52.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.024+0000 7f2167e00640 1 --2- 172.21.15.86:0/202728875 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2170071ad0 0x7f2170146d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:52.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.024+0000 7f216e800640 1 --2- 172.21.15.86:0/202728875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2170153590 0x7f2170150680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:52.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.024+0000 7f2167e00640 1 -- 172.21.15.86:0/202728875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2170153590 msgr2=0x7f2170150680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:52.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.024+0000 7f2167e00640 1 --2- 172.21.15.86:0/202728875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2170153590 0x7f2170150680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:52.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.024+0000 7f2167e00640 1 -- 172.21.15.86:0/202728875 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21700af170 con 0x7f21700af870 2024-04-20T21:19:52.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.025+0000 7f2167e00640 1 --2- 172.21.15.86:0/202728875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21700af870 0x7f2170150140 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f215800c4d0 tx=0x7f215800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:52.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.025+0000 7f2166000640 1 -- 172.21.15.86:0/202728875 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2158018070 con 0x7f21700af870 2024-04-20T21:19:52.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.025+0000 7f2166000640 1 -- 172.21.15.86:0/202728875 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2158007500 con 0x7f21700af870 2024-04-20T21:19:52.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.025+0000 7f2166000640 1 -- 172.21.15.86:0/202728875 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2158004a20 con 0x7f21700af870 2024-04-20T21:19:52.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.025+0000 7f216f200640 1 -- 172.21.15.86:0/202728875 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2170155950 con 0x7f21700af870 2024-04-20T21:19:52.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.025+0000 7f216f200640 1 -- 172.21.15.86:0/202728875 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21700736c0 con 0x7f21700af870 2024-04-20T21:19:52.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.026+0000 7f216f200640 1 -- 172.21.15.86:0/202728875 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2170069470 con 0x7f21700af870 2024-04-20T21:19:52.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.027+0000 7f2166000640 1 -- 172.21.15.86:0/202728875 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2158004bc0 con 0x7f21700af870 2024-04-20T21:19:52.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.028+0000 7f2166000640 1 --2- 172.21.15.86:0/202728875 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f214c06c8f0 0x7f214c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:52.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.028+0000 7f2166000640 1 -- 172.21.15.86:0/202728875 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5357+0+0 (secure 0 0 0) 0x7f215808acf0 con 0x7f21700af870 2024-04-20T21:19:52.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.029+0000 7f216de00640 1 --2- 172.21.15.86:0/202728875 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f214c06c8f0 0x7f214c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:52.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.029+0000 7f216de00640 1 --2- 172.21.15.86:0/202728875 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f214c06c8f0 0x7f214c06edb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f216000a810 tx=0x7f2160008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:52.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.033+0000 7f2166000640 1 -- 172.21.15.86:0/202728875 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2170069470 con 0x7f21700af870 2024-04-20T21:19:52.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.237+0000 7f216f200640 1 -- 172.21.15.86:0/202728875 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f217004fcf0 con 0x7f21700af870 2024-04-20T21:19:52.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.805+0000 7f2166000640 1 -- 172.21.15.86:0/202728875 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v36) ==== 169+0+0 (secure 0 0 0) 0x7f2158059270 con 0x7f21700af870 2024-04-20T21:19:52.805 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' removed 2024-04-20T21:19:52.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.809+0000 7f216f200640 1 -- 172.21.15.86:0/202728875 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f214c06c8f0 msgr2=0x7f214c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:52.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.809+0000 7f216f200640 1 --2- 172.21.15.86:0/202728875 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f214c06c8f0 0x7f214c06edb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f216000a810 tx=0x7f2160008040 comp rx=0 tx=0).stop 2024-04-20T21:19:52.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.809+0000 7f216f200640 1 -- 172.21.15.86:0/202728875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21700af870 msgr2=0x7f2170150140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:52.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.809+0000 7f216f200640 1 --2- 172.21.15.86:0/202728875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21700af870 0x7f2170150140 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f215800c4d0 tx=0x7f215800c990 comp rx=0 tx=0).stop 2024-04-20T21:19:52.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.809+0000 7f216f200640 1 -- 172.21.15.86:0/202728875 shutdown_connections 2024-04-20T21:19:52.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.809+0000 7f216f200640 1 --2- 172.21.15.86:0/202728875 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f214c06c8f0 0x7f214c06edb0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:52.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.809+0000 7f216f200640 1 --2- 172.21.15.86:0/202728875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2170153590 0x7f2170150680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:52.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.809+0000 7f216f200640 1 --2- 172.21.15.86:0/202728875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21700af870 0x7f2170150140 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:52.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.809+0000 7f216f200640 1 --2- 172.21.15.86:0/202728875 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2170071ad0 0x7f2170146d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:52.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.809+0000 7f216f200640 1 -- 172.21.15.86:0/202728875 >> 172.21.15.86:0/202728875 conn(0x7f21700b2180 msgr2=0x7f217015fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:52.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.810+0000 7f216f200640 1 -- 172.21.15.86:0/202728875 shutdown_connections 2024-04-20T21:19:52.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:52.810+0000 7f216f200640 1 -- 172.21.15.86:0/202728875 wait complete. 2024-04-20T21:19:52.837 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-20T21:19:53.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.033+0000 7f074e200640 1 Processor -- start 2024-04-20T21:19:53.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.034+0000 7f074e200640 1 -- start start 2024-04-20T21:19:53.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.035+0000 7f074e200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f074806bbf0 0x7f074806e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:53.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.035+0000 7f074e200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0748146210 0x7f07480cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:53.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.035+0000 7f074e200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f074815cd40 con 0x7f074806bfd0 2024-04-20T21:19:53.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.035+0000 7f074e200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f074815d000 con 0x7f074806bbf0 2024-04-20T21:19:53.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.035+0000 7f074e200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f074815d2e0 con 0x7f0748146210 2024-04-20T21:19:53.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.036+0000 7f0747400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f074806bbf0 0x7f074806e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:53.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.036+0000 7f0747400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f074806bbf0 0x7f074806e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39666/0 (socket says 172.21.15.86:39666) 2024-04-20T21:19:53.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.036+0000 7f0747400640 1 -- 172.21.15.86:0/1645543140 learned_addr learned my addr 172.21.15.86:0/1645543140 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:53.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.036+0000 7f0747e00640 1 --2- 172.21.15.86:0/1645543140 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0748146210 0x7f07480cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:53.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.036+0000 7f0747400640 1 -- 172.21.15.86:0/1645543140 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0748146210 msgr2=0x7f07480cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:53.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.036+0000 7f0747400640 1 --2- 172.21.15.86:0/1645543140 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0748146210 0x7f07480cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:53.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.036+0000 7f0747400640 1 -- 172.21.15.86:0/1645543140 >> v1:172.21.15.144:6789/0 conn(0x7f074806bfd0 legacy=0x7f0748145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:53.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.036+0000 7f0747400640 1 -- 172.21.15.86:0/1645543140 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f074815d5c0 con 0x7f074806bbf0 2024-04-20T21:19:53.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.038+0000 7f0747400640 1 --2- 172.21.15.86:0/1645543140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f074806bbf0 0x7f074806e000 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f0738009e50 tx=0x7f073802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2d273dea226117 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:53.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.038+0000 7f0746a00640 1 -- 172.21.15.86:0/1645543140 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073802caf0 con 0x7f074806bbf0 2024-04-20T21:19:53.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.038+0000 7f0746a00640 1 -- 172.21.15.86:0/1645543140 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f073802cc90 con 0x7f074806bbf0 2024-04-20T21:19:53.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.038+0000 7f0746a00640 1 -- 172.21.15.86:0/1645543140 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0738034610 con 0x7f074806bbf0 2024-04-20T21:19:53.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.039+0000 7f074e200640 1 -- 172.21.15.86:0/1645543140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f074806bbf0 msgr2=0x7f074806e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:53.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.039+0000 7f074e200640 1 --2- 172.21.15.86:0/1645543140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f074806bbf0 0x7f074806e000 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f0738009e50 tx=0x7f073802bf50 comp rx=0 tx=0).stop 2024-04-20T21:19:53.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.039+0000 7f074e200640 1 -- 172.21.15.86:0/1645543140 shutdown_connections 2024-04-20T21:19:53.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.039+0000 7f074e200640 1 --2- 172.21.15.86:0/1645543140 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0748146210 0x7f07480cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:53.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.039+0000 7f074e200640 1 --2- 172.21.15.86:0/1645543140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f074806bbf0 0x7f074806e000 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:53.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.039+0000 7f074e200640 1 -- 172.21.15.86:0/1645543140 >> 172.21.15.86:0/1645543140 conn(0x7f07480b2110 msgr2=0x7f07480b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:53.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.039+0000 7f074e200640 1 -- 172.21.15.86:0/1645543140 shutdown_connections 2024-04-20T21:19:53.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.039+0000 7f074e200640 1 -- 172.21.15.86:0/1645543140 wait complete. 2024-04-20T21:19:53.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.041+0000 7f074e200640 1 Processor -- start 2024-04-20T21:19:53.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.041+0000 7f074e200640 1 -- start start 2024-04-20T21:19:53.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.042+0000 7f074e200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f074806bbf0 0x7f074814fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:53.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.042+0000 7f074e200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f074806bfd0 0x7f07481f7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:53.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.042+0000 7f074e200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0748146210 0x7f07481f81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:53.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.042+0000 7f074e200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f07480cdc00 con 0x7f074806bfd0 2024-04-20T21:19:53.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.042+0000 7f074e200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f07480cda80 con 0x7f074806bbf0 2024-04-20T21:19:53.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.042+0000 7f074e200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f07480cdd80 con 0x7f0748146210 2024-04-20T21:19:53.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.042+0000 7f073ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f074806bfd0 0x7f07481f7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:53.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.042+0000 7f073ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f074806bfd0 0x7f07481f7c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39672/0 (socket says 172.21.15.86:39672) 2024-04-20T21:19:53.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.042+0000 7f073ea00640 1 -- 172.21.15.86:0/2520862613 learned_addr learned my addr 172.21.15.86:0/2520862613 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:53.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.042+0000 7f0747400640 1 --2- 172.21.15.86:0/2520862613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f074806bbf0 0x7f074814fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:53.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.042+0000 7f073ea00640 1 -- 172.21.15.86:0/2520862613 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0748146210 msgr2=0x7f07481f81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:53.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.042+0000 7f073ea00640 1 --2- 172.21.15.86:0/2520862613 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0748146210 0x7f07481f81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:53.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.042+0000 7f073ea00640 1 -- 172.21.15.86:0/2520862613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f074806bbf0 msgr2=0x7f074814fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:53.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.042+0000 7f073ea00640 1 --2- 172.21.15.86:0/2520862613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f074806bbf0 0x7f074814fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:53.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.042+0000 7f073ea00640 1 -- 172.21.15.86:0/2520862613 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0748140d80 con 0x7f074806bfd0 2024-04-20T21:19:53.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.043+0000 7f073ea00640 1 --2- 172.21.15.86:0/2520862613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f074806bfd0 0x7f07481f7c80 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f073000c3c0 tx=0x7f073000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:53.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.043+0000 7f073fe00640 1 -- 172.21.15.86:0/2520862613 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07300052d0 con 0x7f074806bfd0 2024-04-20T21:19:53.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.043+0000 7f073fe00640 1 -- 172.21.15.86:0/2520862613 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0730005490 con 0x7f074806bfd0 2024-04-20T21:19:53.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.043+0000 7f074e200640 1 -- 172.21.15.86:0/2520862613 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0748141060 con 0x7f074806bfd0 2024-04-20T21:19:53.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.043+0000 7f073fe00640 1 -- 172.21.15.86:0/2520862613 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0730007260 con 0x7f074806bfd0 2024-04-20T21:19:53.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.043+0000 7f074e200640 1 -- 172.21.15.86:0/2520862613 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0748141620 con 0x7f074806bfd0 2024-04-20T21:19:53.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.044+0000 7f074e200640 1 -- 172.21.15.86:0/2520862613 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07480689e0 con 0x7f074806bfd0 2024-04-20T21:19:53.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.045+0000 7f073fe00640 1 -- 172.21.15.86:0/2520862613 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0730007400 con 0x7f074806bfd0 2024-04-20T21:19:53.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.046+0000 7f073fe00640 1 --2- 172.21.15.86:0/2520862613 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f072406c8f0 0x7f072406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:53.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.046+0000 7f073fe00640 1 -- 172.21.15.86:0/2520862613 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f073008ad00 con 0x7f074806bfd0 2024-04-20T21:19:53.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.046+0000 7f0747400640 1 --2- 172.21.15.86:0/2520862613 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f072406c8f0 0x7f072406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:53.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.047+0000 7f0747400640 1 --2- 172.21.15.86:0/2520862613 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f072406c8f0 0x7f072406edb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0738035000 tx=0x7f0738030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:53.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.051+0000 7f073fe00640 1 -- 172.21.15.86:0/2520862613 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0730059420 con 0x7f074806bfd0 2024-04-20T21:19:53.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.252+0000 7f074e200640 1 -- 172.21.15.86:0/2520862613 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f074806c780 con 0x7f074806bfd0 2024-04-20T21:19:53.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.258+0000 7f073fe00640 1 -- 172.21.15.86:0/2520862613 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f0730058d20 con 0x7f074806bfd0 2024-04-20T21:19:53.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.261+0000 7f074e200640 1 -- 172.21.15.86:0/2520862613 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f072406c8f0 msgr2=0x7f072406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:53.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.261+0000 7f074e200640 1 --2- 172.21.15.86:0/2520862613 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f072406c8f0 0x7f072406edb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0738035000 tx=0x7f0738030000 comp rx=0 tx=0).stop 2024-04-20T21:19:53.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.261+0000 7f074e200640 1 -- 172.21.15.86:0/2520862613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f074806bfd0 msgr2=0x7f07481f7c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:53.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.261+0000 7f074e200640 1 --2- 172.21.15.86:0/2520862613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f074806bfd0 0x7f07481f7c80 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f073000c3c0 tx=0x7f073000c880 comp rx=0 tx=0).stop 2024-04-20T21:19:53.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.261+0000 7f074e200640 1 -- 172.21.15.86:0/2520862613 shutdown_connections 2024-04-20T21:19:53.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.261+0000 7f074e200640 1 --2- 172.21.15.86:0/2520862613 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f072406c8f0 0x7f072406edb0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:53.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.262+0000 7f074e200640 1 --2- 172.21.15.86:0/2520862613 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0748146210 0x7f07481f81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:53.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.262+0000 7f074e200640 1 --2- 172.21.15.86:0/2520862613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f074806bfd0 0x7f07481f7c80 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:53.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.262+0000 7f074e200640 1 --2- 172.21.15.86:0/2520862613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f074806bbf0 0x7f074814fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:53.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.262+0000 7f074e200640 1 -- 172.21.15.86:0/2520862613 >> 172.21.15.86:0/2520862613 conn(0x7f07480b2110 msgr2=0x7f074815f660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:53.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.262+0000 7f074e200640 1 -- 172.21.15.86:0/2520862613 shutdown_connections 2024-04-20T21:19:53.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.262+0000 7f074e200640 1 -- 172.21.15.86:0/2520862613 wait complete. 2024-04-20T21:19:53.279 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-04-20T21:19:53.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.479+0000 7ff693000640 1 Processor -- start 2024-04-20T21:19:53.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.479+0000 7ff693000640 1 -- start start 2024-04-20T21:19:53.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.481+0000 7ff693000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff68c0d50c0 0x7ff68c0d3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:53.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.481+0000 7ff693000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff68c0d3950 con 0x7ff68c15ece0 2024-04-20T21:19:53.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.481+0000 7ff693000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff68c0cec00 con 0x7ff68c0cf9c0 2024-04-20T21:19:53.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.481+0000 7ff693000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff68c0ceee0 con 0x7ff68c0d50c0 2024-04-20T21:19:53.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.481+0000 7ff691c00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7ff68c15ece0 0x7ff68c15f0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:47298/0 (socket says 172.21.15.86:47298) 2024-04-20T21:19:53.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.481+0000 7ff691c00640 1 -- 172.21.15.86:0/886512252 learned_addr learned my addr 172.21.15.86:0/886512252 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:19:53.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.482+0000 7ff68be00640 1 -- 172.21.15.86:0/886512252 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2253840881 0 0) 0x7ff68c0d3950 con 0x7ff68c15ece0 2024-04-20T21:19:53.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.482+0000 7ff68be00640 1 -- 172.21.15.86:0/886512252 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff670003520 con 0x7ff68c15ece0 2024-04-20T21:19:53.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.482+0000 7ff691200640 1 --2- 172.21.15.86:0/886512252 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff68c0d50c0 0x7ff68c0d3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:53.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.482+0000 7ff68be00640 1 -- 172.21.15.86:0/886512252 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4272797210 0 0) 0x7ff670003520 con 0x7ff68c15ece0 2024-04-20T21:19:53.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.482+0000 7ff68be00640 1 -- 172.21.15.86:0/886512252 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff68c0d50c0 msgr2=0x7ff68c0d3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:53.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.482+0000 7ff68be00640 1 --2- 172.21.15.86:0/886512252 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff68c0d50c0 0x7ff68c0d3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:53.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.482+0000 7ff68be00640 1 -- 172.21.15.86:0/886512252 >> v1:172.21.15.144:6789/0 conn(0x7ff68c0cf9c0 legacy=0x7ff68c160000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:53.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.482+0000 7ff68be00640 1 -- 172.21.15.86:0/886512252 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff68c0cf1c0 con 0x7ff68c15ece0 2024-04-20T21:19:53.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.482+0000 7ff68be00640 1 -- 172.21.15.86:0/886512252 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7ff67c001670 con 0x7ff68c15ece0 2024-04-20T21:19:53.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.482+0000 7ff68be00640 1 -- 172.21.15.86:0/886512252 >> v1:172.21.15.86:6789/0 conn(0x7ff68c15ece0 legacy=0x7ff68c15f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:53.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.482+0000 7ff68be00640 1 --2- 172.21.15.86:0/886512252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6700039e0 0x7ff670003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:53.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.482+0000 7ff68be00640 1 --2- 172.21.15.86:0/886512252 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff68c0d50c0 0x7ff670004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:53.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.482+0000 7ff68be00640 1 --2- 172.21.15.86:0/886512252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff670004d40 0x7ff6700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:53.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.482+0000 7ff68be00640 1 -- 172.21.15.86:0/886512252 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff68c0ceee0 con 0x7ff670004d40 2024-04-20T21:19:53.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.482+0000 7ff68be00640 1 -- 172.21.15.86:0/886512252 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff67000bba0 con 0x7ff6700039e0 2024-04-20T21:19:53.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.482+0000 7ff68be00640 1 -- 172.21.15.86:0/886512252 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff67000bda0 con 0x7ff68c0d50c0 2024-04-20T21:19:53.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.483+0000 7ff692600640 1 --2- 172.21.15.86:0/886512252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff670004d40 0x7ff6700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:53.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.483+0000 7ff691200640 1 --2- 172.21.15.86:0/886512252 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff68c0d50c0 0x7ff670004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:53.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.483+0000 7ff691c00640 1 --2- 172.21.15.86:0/886512252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6700039e0 0x7ff670003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:53.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.483+0000 7ff692600640 1 -- 172.21.15.86:0/886512252 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff68c0d50c0 msgr2=0x7ff670004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:53.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.483+0000 7ff692600640 1 --2- 172.21.15.86:0/886512252 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff68c0d50c0 0x7ff670004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:53.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.483+0000 7ff692600640 1 -- 172.21.15.86:0/886512252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6700039e0 msgr2=0x7ff670003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:53.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.483+0000 7ff692600640 1 --2- 172.21.15.86:0/886512252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6700039e0 0x7ff670003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:53.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.483+0000 7ff692600640 1 -- 172.21.15.86:0/886512252 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff670003520 con 0x7ff670004d40 2024-04-20T21:19:53.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.483+0000 7ff691c00640 1 --2- 172.21.15.86:0/886512252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6700039e0 0x7ff670003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:53.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.484+0000 7ff692600640 1 --2- 172.21.15.86:0/886512252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff670004d40 0x7ff6700051c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff684001750 tx=0x7ff670005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a167a014aefce69 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:53.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.484+0000 7ff68be00640 1 -- 172.21.15.86:0/886512252 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff684001370 con 0x7ff670004d40 2024-04-20T21:19:53.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.485+0000 7ff68be00640 1 -- 172.21.15.86:0/886512252 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6840052d0 con 0x7ff670004d40 2024-04-20T21:19:53.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.485+0000 7ff68be00640 1 -- 172.21.15.86:0/886512252 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff684003830 con 0x7ff670004d40 2024-04-20T21:19:53.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.485+0000 7ff693000640 1 -- 172.21.15.86:0/886512252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff670004d40 msgr2=0x7ff6700051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:53.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.485+0000 7ff693000640 1 --2- 172.21.15.86:0/886512252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff670004d40 0x7ff6700051c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff684001750 tx=0x7ff670005830 comp rx=0 tx=0).stop 2024-04-20T21:19:53.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.485+0000 7ff692600640 1 -- 172.21.15.86:0/886512252 reap_dead start 2024-04-20T21:19:53.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.485+0000 7ff693000640 1 -- 172.21.15.86:0/886512252 shutdown_connections 2024-04-20T21:19:53.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.485+0000 7ff693000640 1 -- 172.21.15.86:0/886512252 >> 172.21.15.86:0/886512252 conn(0x7ff68c08bf70 msgr2=0x7ff68c0d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:53.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.485+0000 7ff693000640 1 -- 172.21.15.86:0/886512252 shutdown_connections 2024-04-20T21:19:53.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.485+0000 7ff693000640 1 -- 172.21.15.86:0/886512252 wait complete. 2024-04-20T21:19:53.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.487+0000 7ff693000640 1 Processor -- start 2024-04-20T21:19:53.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.487+0000 7ff693000640 1 -- start start 2024-04-20T21:19:53.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.488+0000 7ff693000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff670004d40 0x7ff68c0ce280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:53.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.488+0000 7ff693000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff68c15a9b0 0x7ff68c14d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:53.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.488+0000 7ff693000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff68c1ea770 0x7ff68c0cc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:53.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.488+0000 7ff693000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff68c0cfe40 con 0x7ff68c15a9b0 2024-04-20T21:19:53.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.488+0000 7ff693000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff68c15ea50 con 0x7ff670004d40 2024-04-20T21:19:53.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.488+0000 7ff693000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff68c0cffc0 con 0x7ff68c1ea770 2024-04-20T21:19:53.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.488+0000 7ff691200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff68c15a9b0 0x7ff68c14d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:53.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.488+0000 7ff691200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff68c15a9b0 0x7ff68c14d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39694/0 (socket says 172.21.15.86:39694) 2024-04-20T21:19:53.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.488+0000 7ff691200640 1 -- 172.21.15.86:0/625119274 learned_addr learned my addr 172.21.15.86:0/625119274 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:53.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.489+0000 7ff692600640 1 --2- 172.21.15.86:0/625119274 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff68c1ea770 0x7ff68c0cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:53.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.489+0000 7ff691c00640 1 --2- 172.21.15.86:0/625119274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff670004d40 0x7ff68c0ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:53.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.489+0000 7ff691200640 1 -- 172.21.15.86:0/625119274 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff68c1ea770 msgr2=0x7ff68c0cc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:53.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.489+0000 7ff691200640 1 --2- 172.21.15.86:0/625119274 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff68c1ea770 0x7ff68c0cc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:53.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.489+0000 7ff691200640 1 -- 172.21.15.86:0/625119274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff670004d40 msgr2=0x7ff68c0ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:53.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.489+0000 7ff691200640 1 --2- 172.21.15.86:0/625119274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff670004d40 0x7ff68c0ce280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:53.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.489+0000 7ff691200640 1 -- 172.21.15.86:0/625119274 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff68c14de60 con 0x7ff68c15a9b0 2024-04-20T21:19:53.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.489+0000 7ff691200640 1 --2- 172.21.15.86:0/625119274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff68c15a9b0 0x7ff68c14d920 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff678009c80 tx=0x7ff678007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:53.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.490+0000 7ff689600640 1 -- 172.21.15.86:0/625119274 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff678014610 con 0x7ff68c15a9b0 2024-04-20T21:19:53.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.490+0000 7ff689600640 1 -- 172.21.15.86:0/625119274 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff678019070 con 0x7ff68c15a9b0 2024-04-20T21:19:53.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.490+0000 7ff693000640 1 -- 172.21.15.86:0/625119274 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff68c1e93f0 con 0x7ff68c15a9b0 2024-04-20T21:19:53.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.490+0000 7ff689600640 1 -- 172.21.15.86:0/625119274 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6780149d0 con 0x7ff68c15a9b0 2024-04-20T21:19:53.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.490+0000 7ff693000640 1 -- 172.21.15.86:0/625119274 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff68c1e99b0 con 0x7ff68c15a9b0 2024-04-20T21:19:53.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.491+0000 7ff693000640 1 -- 172.21.15.86:0/625119274 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff68c04fcf0 con 0x7ff68c15a9b0 2024-04-20T21:19:53.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.492+0000 7ff689600640 1 -- 172.21.15.86:0/625119274 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff678014b70 con 0x7ff68c15a9b0 2024-04-20T21:19:53.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.493+0000 7ff689600640 1 --2- 172.21.15.86:0/625119274 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff66c06c8f0 0x7ff66c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:53.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.493+0000 7ff689600640 1 -- 172.21.15.86:0/625119274 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7ff67808c330 con 0x7ff68c15a9b0 2024-04-20T21:19:53.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.493+0000 7ff691c00640 1 --2- 172.21.15.86:0/625119274 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff66c06c8f0 0x7ff66c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:53.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.494+0000 7ff691c00640 1 --2- 172.21.15.86:0/625119274 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff66c06c8f0 0x7ff66c06edb0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff67c0017e0 tx=0x7ff67c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:53.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.497+0000 7ff689600640 1 -- 172.21.15.86:0/625119274 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff67805a930 con 0x7ff68c15a9b0 2024-04-20T21:19:53.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.789+0000 7ff693000640 1 -- 172.21.15.86:0/625119274 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) -- 0x7ff68c0d28a0 con 0x7ff68c15a9b0 2024-04-20T21:19:53.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.808+0000 7ff689600640 1 -- 172.21.15.86:0/625119274 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v135) ==== 142+0+0 (secure 0 0 0) 0x7ff67805a230 con 0x7ff68c15a9b0 2024-04-20T21:19:53.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.811+0000 7ff693000640 1 -- 172.21.15.86:0/625119274 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff66c06c8f0 msgr2=0x7ff66c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:53.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.811+0000 7ff693000640 1 --2- 172.21.15.86:0/625119274 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff66c06c8f0 0x7ff66c06edb0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff67c0017e0 tx=0x7ff67c009040 comp rx=0 tx=0).stop 2024-04-20T21:19:53.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.811+0000 7ff693000640 1 -- 172.21.15.86:0/625119274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff68c15a9b0 msgr2=0x7ff68c14d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:53.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.811+0000 7ff693000640 1 --2- 172.21.15.86:0/625119274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff68c15a9b0 0x7ff68c14d920 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff678009c80 tx=0x7ff678007700 comp rx=0 tx=0).stop 2024-04-20T21:19:53.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.811+0000 7ff693000640 1 -- 172.21.15.86:0/625119274 shutdown_connections 2024-04-20T21:19:53.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.811+0000 7ff693000640 1 --2- 172.21.15.86:0/625119274 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff66c06c8f0 0x7ff66c06edb0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:53.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.811+0000 7ff693000640 1 --2- 172.21.15.86:0/625119274 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff68c1ea770 0x7ff68c0cc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:53.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.811+0000 7ff693000640 1 --2- 172.21.15.86:0/625119274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff68c15a9b0 0x7ff68c14d920 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:53.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.811+0000 7ff693000640 1 --2- 172.21.15.86:0/625119274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff670004d40 0x7ff68c0ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:53.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.811+0000 7ff693000640 1 -- 172.21.15.86:0/625119274 >> 172.21.15.86:0/625119274 conn(0x7ff68c08bf70 msgr2=0x7ff68c0d3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:53.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.811+0000 7ff693000640 1 -- 172.21.15.86:0/625119274 shutdown_connections 2024-04-20T21:19:53.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:53.811+0000 7ff693000640 1 -- 172.21.15.86:0/625119274 wait complete. 2024-04-20T21:19:53.829 INFO:tasks.cephfs_test_runner:test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-04-20T21:19:53.829 INFO:tasks.cephfs_test_runner:Starting test: test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) That the MDS can repair an inodes backtrace in the data pool 2024-04-20T21:19:53.829 INFO:tasks.cephfs_test_runner:test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-20T21:19:53.829 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:19:54.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.023+0000 7fe6ef600640 1 Processor -- start 2024-04-20T21:19:54.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.024+0000 7fe6ef600640 1 -- start start 2024-04-20T21:19:54.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.025+0000 7fe6ef600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe6e806e6d0 0x7fe6e8071d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:54.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.025+0000 7fe6ef600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6e8072240 0x7fe6e815cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:54.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.025+0000 7fe6ef600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6e806eb30 con 0x7fe6e8146130 2024-04-20T21:19:54.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.025+0000 7fe6ef600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fe6e806dc30 con 0x7fe6e806e6d0 2024-04-20T21:19:54.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.025+0000 7fe6ef600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe6e80cd8e0 con 0x7fe6e8072240 2024-04-20T21:19:54.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.026+0000 7fe6ecc00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe6e8146130 0x7fe6e80ccdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:47308/0 (socket says 172.21.15.86:47308) 2024-04-20T21:19:54.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.026+0000 7fe6ecc00640 1 -- 172.21.15.86:0/2596762439 learned_addr learned my addr 172.21.15.86:0/2596762439 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:19:54.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.026+0000 7fe6ed600640 1 --2- 172.21.15.86:0/2596762439 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6e8072240 0x7fe6e815cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:54.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.026+0000 7fe6e7400640 1 -- 172.21.15.86:0/2596762439 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3048174722 0 0) 0x7fe6d8001490 con 0x7fe6e8146130 2024-04-20T21:19:54.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.026+0000 7fe6e7e00640 1 --2- 172.21.15.86:0/2596762439 >> v2:172.21.15.144:3300/0 conn(0x7fe6e806e6d0 0x7fe6e8071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:54.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.026+0000 7fe6e7400640 1 -- 172.21.15.86:0/2596762439 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe6cc003520 con 0x7fe6e8146130 2024-04-20T21:19:54.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.027+0000 7fe6e7400640 1 -- 172.21.15.86:0/2596762439 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2650053819 0 0) 0x7fe6cc003520 con 0x7fe6e8146130 2024-04-20T21:19:54.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.027+0000 7fe6e7400640 1 -- 172.21.15.86:0/2596762439 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6e8072240 msgr2=0x7fe6e815cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:54.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.027+0000 7fe6e7400640 1 --2- 172.21.15.86:0/2596762439 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6e8072240 0x7fe6e815cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:54.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.027+0000 7fe6e7400640 1 -- 172.21.15.86:0/2596762439 >> v2:172.21.15.144:3300/0 conn(0x7fe6e806e6d0 msgr2=0x7fe6e8071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:54.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.027+0000 7fe6e7400640 1 --2- 172.21.15.86:0/2596762439 >> v2:172.21.15.144:3300/0 conn(0x7fe6e806e6d0 0x7fe6e8071d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:54.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.027+0000 7fe6e7400640 1 -- 172.21.15.86:0/2596762439 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6e80cdbc0 con 0x7fe6e8146130 2024-04-20T21:19:54.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.027+0000 7fe6e7400640 1 -- 172.21.15.86:0/2596762439 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fe6d8001bc0 con 0x7fe6e8146130 2024-04-20T21:19:54.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.027+0000 7fe6e7e00640 1 --2- 172.21.15.86:0/2596762439 >> v2:172.21.15.144:3300/0 conn(0x7fe6e806e6d0 0x7fe6e8071d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:19:54.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.027+0000 7fe6e7400640 1 -- 172.21.15.86:0/2596762439 >> v1:172.21.15.86:6789/0 conn(0x7fe6e8146130 legacy=0x7fe6e80ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:54.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.027+0000 7fe6e7400640 1 --2- 172.21.15.86:0/2596762439 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6e8072240 0x7fe6cc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:54.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.027+0000 7fe6e7400640 1 --2- 172.21.15.86:0/2596762439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6cc003f80 0x7fe6cc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:54.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.027+0000 7fe6e7400640 1 --2- 172.21.15.86:0/2596762439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6cc004d40 0x7fe6cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:54.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.027+0000 7fe6e7400640 1 -- 172.21.15.86:0/2596762439 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e806dc30 con 0x7fe6cc003f80 2024-04-20T21:19:54.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.027+0000 7fe6e7400640 1 -- 172.21.15.86:0/2596762439 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e80cd8e0 con 0x7fe6cc004d40 2024-04-20T21:19:54.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.027+0000 7fe6e7400640 1 -- 172.21.15.86:0/2596762439 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe6cc00bb00 con 0x7fe6e8072240 2024-04-20T21:19:54.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.027+0000 7fe6e7e00640 1 --2- 172.21.15.86:0/2596762439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6cc003f80 0x7fe6cc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:54.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.027+0000 7fe6ecc00640 1 --2- 172.21.15.86:0/2596762439 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6e8072240 0x7fe6cc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:54.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.027+0000 7fe6e7e00640 1 -- 172.21.15.86:0/2596762439 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6e8072240 msgr2=0x7fe6cc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:54.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.027+0000 7fe6ed600640 1 --2- 172.21.15.86:0/2596762439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6cc004d40 0x7fe6cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:54.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.027+0000 7fe6e7e00640 1 --2- 172.21.15.86:0/2596762439 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6e8072240 0x7fe6cc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:54.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.027+0000 7fe6e7e00640 1 -- 172.21.15.86:0/2596762439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6cc004d40 msgr2=0x7fe6cc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:54.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.027+0000 7fe6e7e00640 1 --2- 172.21.15.86:0/2596762439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6cc004d40 0x7fe6cc0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:54.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.028+0000 7fe6e7e00640 1 -- 172.21.15.86:0/2596762439 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6cc003520 con 0x7fe6cc003f80 2024-04-20T21:19:54.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.028+0000 7fe6ecc00640 1 --2- 172.21.15.86:0/2596762439 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6e8072240 0x7fe6cc0039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:54.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.029+0000 7fe6e7e00640 1 --2- 172.21.15.86:0/2596762439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6cc003f80 0x7fe6cc004800 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe6d40098a0 tx=0x7fe6cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3104685b6ce0baa server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:54.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.029+0000 7fe6e7400640 1 -- 172.21.15.86:0/2596762439 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d400f040 con 0x7fe6cc003f80 2024-04-20T21:19:54.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.029+0000 7fe6e7400640 1 -- 172.21.15.86:0/2596762439 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6d4004230 con 0x7fe6cc003f80 2024-04-20T21:19:54.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.029+0000 7fe6e7400640 1 -- 172.21.15.86:0/2596762439 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d4003610 con 0x7fe6cc003f80 2024-04-20T21:19:54.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.029+0000 7fe6ef600640 1 -- 172.21.15.86:0/2596762439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6cc003f80 msgr2=0x7fe6cc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:54.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.029+0000 7fe6ef600640 1 --2- 172.21.15.86:0/2596762439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6cc003f80 0x7fe6cc004800 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe6d40098a0 tx=0x7fe6cc005830 comp rx=0 tx=0).stop 2024-04-20T21:19:54.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.029+0000 7fe6ed600640 1 -- 172.21.15.86:0/2596762439 reap_dead start 2024-04-20T21:19:54.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.029+0000 7fe6ef600640 1 -- 172.21.15.86:0/2596762439 shutdown_connections 2024-04-20T21:19:54.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.029+0000 7fe6ef600640 1 -- 172.21.15.86:0/2596762439 >> 172.21.15.86:0/2596762439 conn(0x7fe6e80b2360 msgr2=0x7fe6e80b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:54.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.030+0000 7fe6ef600640 1 -- 172.21.15.86:0/2596762439 shutdown_connections 2024-04-20T21:19:54.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.030+0000 7fe6ef600640 1 -- 172.21.15.86:0/2596762439 wait complete. 2024-04-20T21:19:54.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.031+0000 7fe6ef600640 1 Processor -- start 2024-04-20T21:19:54.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.031+0000 7fe6ef600640 1 -- start start 2024-04-20T21:19:54.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.032+0000 7fe6ef600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6cc003f80 0x7fe6e8140b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:54.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.032+0000 7fe6ef600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6e812fdd0 0x7fe6e81318c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:54.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.032+0000 7fe6ef600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6e8130250 0x7fe6e8131e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:54.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.032+0000 7fe6ef600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e806f150 con 0x7fe6e8130250 2024-04-20T21:19:54.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.032+0000 7fe6ef600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e806e1b0 con 0x7fe6cc003f80 2024-04-20T21:19:54.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.032+0000 7fe6ef600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe6e80cd4e0 con 0x7fe6e812fdd0 2024-04-20T21:19:54.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.032+0000 7fe6ed600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6e8130250 0x7fe6e8131e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:54.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.033+0000 7fe6ed600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6e8130250 0x7fe6e8131e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39722/0 (socket says 172.21.15.86:39722) 2024-04-20T21:19:54.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.033+0000 7fe6ed600640 1 -- 172.21.15.86:0/3974827700 learned_addr learned my addr 172.21.15.86:0/3974827700 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:54.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.033+0000 7fe6ed600640 1 -- 172.21.15.86:0/3974827700 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6e812fdd0 msgr2=0x7fe6e81318c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:54.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.033+0000 7fe6ed600640 1 --2- 172.21.15.86:0/3974827700 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6e812fdd0 0x7fe6e81318c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:54.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.033+0000 7fe6ed600640 1 -- 172.21.15.86:0/3974827700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6cc003f80 msgr2=0x7fe6e8140b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:54.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.033+0000 7fe6ed600640 1 --2- 172.21.15.86:0/3974827700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6cc003f80 0x7fe6e8140b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:54.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.033+0000 7fe6ed600640 1 -- 172.21.15.86:0/3974827700 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6e8130820 con 0x7fe6e8130250 2024-04-20T21:19:54.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.033+0000 7fe6ed600640 1 --2- 172.21.15.86:0/3974827700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6e8130250 0x7fe6e8131e00 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe6e806db60 tx=0x7fe6e0010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:54.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.033+0000 7fe6e4c00640 1 -- 172.21.15.86:0/3974827700 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e000f040 con 0x7fe6e8130250 2024-04-20T21:19:54.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.033+0000 7fe6e4c00640 1 -- 172.21.15.86:0/3974827700 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6e001c070 con 0x7fe6e8130250 2024-04-20T21:19:54.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.033+0000 7fe6e4c00640 1 -- 172.21.15.86:0/3974827700 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e000eb00 con 0x7fe6e8130250 2024-04-20T21:19:54.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.034+0000 7fe6ef600640 1 -- 172.21.15.86:0/3974827700 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6e8130b00 con 0x7fe6e8130250 2024-04-20T21:19:54.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.034+0000 7fe6ef600640 1 -- 172.21.15.86:0/3974827700 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6e81310c0 con 0x7fe6e8130250 2024-04-20T21:19:54.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.035+0000 7fe6ef600640 1 -- 172.21.15.86:0/3974827700 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6e80689e0 con 0x7fe6e8130250 2024-04-20T21:19:54.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.036+0000 7fe6e4c00640 1 -- 172.21.15.86:0/3974827700 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe6e000eca0 con 0x7fe6e8130250 2024-04-20T21:19:54.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.037+0000 7fe6e4c00640 1 --2- 172.21.15.86:0/3974827700 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe6c806c8f0 0x7fe6c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:54.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.037+0000 7fe6e4c00640 1 -- 172.21.15.86:0/3974827700 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7fe6e008b3a0 con 0x7fe6e8130250 2024-04-20T21:19:54.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.037+0000 7fe6ecc00640 1 --2- 172.21.15.86:0/3974827700 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe6c806c8f0 0x7fe6c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:54.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.037+0000 7fe6ecc00640 1 --2- 172.21.15.86:0/3974827700 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe6c806c8f0 0x7fe6c806edb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe6d80013d0 tx=0x7fe6d8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:54.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.041+0000 7fe6e4c00640 1 -- 172.21.15.86:0/3974827700 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe6e0026020 con 0x7fe6e8130250 2024-04-20T21:19:54.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.241+0000 7fe6ef600640 1 -- 172.21.15.86:0/3974827700 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe6e81423d0 con 0x7fe6e8130250 2024-04-20T21:19:54.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.243+0000 7fe6e4c00640 1 -- 172.21.15.86:0/3974827700 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+13020 (secure 0 0 0) 0x7fe6e00593e0 con 0x7fe6e8130250 2024-04-20T21:19:54.242 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:19:54.242 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":36,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:19:52.793516+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6837/3715801064":"2024-04-21T21:19:49.334103+0000","172.21.15.86:6835/3715801064":"2024-04-21T21:19:49.334103+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:19:54.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.245+0000 7fe6ef600640 1 -- 172.21.15.86:0/3974827700 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe6c806c8f0 msgr2=0x7fe6c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:54.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.245+0000 7fe6ef600640 1 --2- 172.21.15.86:0/3974827700 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe6c806c8f0 0x7fe6c806edb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe6d80013d0 tx=0x7fe6d8009040 comp rx=0 tx=0).stop 2024-04-20T21:19:54.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.245+0000 7fe6ef600640 1 -- 172.21.15.86:0/3974827700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6e8130250 msgr2=0x7fe6e8131e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:54.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.245+0000 7fe6ef600640 1 --2- 172.21.15.86:0/3974827700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6e8130250 0x7fe6e8131e00 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe6e806db60 tx=0x7fe6e0010c20 comp rx=0 tx=0).stop 2024-04-20T21:19:54.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.246+0000 7fe6ef600640 1 -- 172.21.15.86:0/3974827700 shutdown_connections 2024-04-20T21:19:54.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.246+0000 7fe6ef600640 1 --2- 172.21.15.86:0/3974827700 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe6c806c8f0 0x7fe6c806edb0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:54.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.246+0000 7fe6ef600640 1 --2- 172.21.15.86:0/3974827700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6e8130250 0x7fe6e8131e00 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:54.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.246+0000 7fe6ef600640 1 --2- 172.21.15.86:0/3974827700 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6e812fdd0 0x7fe6e81318c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:54.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.246+0000 7fe6ef600640 1 --2- 172.21.15.86:0/3974827700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6cc003f80 0x7fe6e8140b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:54.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.246+0000 7fe6ef600640 1 -- 172.21.15.86:0/3974827700 >> 172.21.15.86:0/3974827700 conn(0x7fe6e80b2360 msgr2=0x7fe6e80b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:54.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.246+0000 7fe6ef600640 1 -- 172.21.15.86:0/3974827700 shutdown_connections 2024-04-20T21:19:54.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.246+0000 7fe6ef600640 1 -- 172.21.15.86:0/3974827700 wait complete. 2024-04-20T21:19:54.265 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:19:54.265 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:19:54.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.468+0000 7eff15800640 1 Processor -- start 2024-04-20T21:19:54.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.468+0000 7eff15800640 1 -- start start 2024-04-20T21:19:54.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.470+0000 7eff15800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7eff100cf500 0x7eff100b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:54.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.470+0000 7eff15800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff10156110 con 0x7eff1006c090 2024-04-20T21:19:54.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.470+0000 7eff15800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff100ce460 con 0x7eff10155d30 2024-04-20T21:19:54.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.470+0000 7eff15800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7eff100ce740 con 0x7eff100cf500 2024-04-20T21:19:54.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.470+0000 7eff07e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7eff1006c090 0x7eff100b4ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:47320/0 (socket says 172.21.15.86:47320) 2024-04-20T21:19:54.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.470+0000 7eff07e00640 1 -- 172.21.15.86:0/2625725292 learned_addr learned my addr 172.21.15.86:0/2625725292 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:19:54.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.471+0000 7eff0f400640 1 --2- 172.21.15.86:0/2625725292 >> v2:172.21.15.144:3300/0 conn(0x7eff100cf500 0x7eff100b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:54.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.471+0000 7eff0e000640 1 -- 172.21.15.86:0/2625725292 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2522923224 0 0) 0x7eff10156110 con 0x7eff1006c090 2024-04-20T21:19:54.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.471+0000 7eff0e000640 1 -- 172.21.15.86:0/2625725292 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efef4005730 con 0x7eff1006c090 2024-04-20T21:19:54.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.471+0000 7eff0e000640 1 -- 172.21.15.86:0/2625725292 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1375846044 0 0) 0x7efef4005730 con 0x7eff1006c090 2024-04-20T21:19:54.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.471+0000 7eff0e000640 1 -- 172.21.15.86:0/2625725292 >> v2:172.21.15.144:3300/0 conn(0x7eff100cf500 msgr2=0x7eff100b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:54.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.471+0000 7eff0e000640 1 --2- 172.21.15.86:0/2625725292 >> v2:172.21.15.144:3300/0 conn(0x7eff100cf500 0x7eff100b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:54.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.471+0000 7eff0e000640 1 -- 172.21.15.86:0/2625725292 >> v1:172.21.15.144:6789/0 conn(0x7eff10155d30 legacy=0x7eff1006b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:54.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.471+0000 7eff0e000640 1 -- 172.21.15.86:0/2625725292 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff100cea20 con 0x7eff1006c090 2024-04-20T21:19:54.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.471+0000 7eff0e000640 1 -- 172.21.15.86:0/2625725292 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7efef8001670 con 0x7eff1006c090 2024-04-20T21:19:54.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.471+0000 7eff0e000640 1 -- 172.21.15.86:0/2625725292 >> v1:172.21.15.86:6789/0 conn(0x7eff1006c090 legacy=0x7eff100b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:54.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.471+0000 7eff0e000640 1 --2- 172.21.15.86:0/2625725292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efef4005bc0 0x7efef4005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:54.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.472+0000 7eff0e000640 1 --2- 172.21.15.86:0/2625725292 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efef4006a10 0x7efef4006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:54.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.472+0000 7eff0e000640 1 --2- 172.21.15.86:0/2625725292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efef40074f0 0x7efef400dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:54.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.472+0000 7eff0e000640 1 -- 172.21.15.86:0/2625725292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7eff100ce740 con 0x7efef4005bc0 2024-04-20T21:19:54.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.472+0000 7eff0e000640 1 -- 172.21.15.86:0/2625725292 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efef400e3c0 con 0x7efef40074f0 2024-04-20T21:19:54.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.472+0000 7eff0e000640 1 -- 172.21.15.86:0/2625725292 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7efef400e5c0 con 0x7efef4006a10 2024-04-20T21:19:54.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.472+0000 7eff0ea00640 1 --2- 172.21.15.86:0/2625725292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efef4005bc0 0x7efef4005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:54.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.472+0000 7eff07e00640 1 --2- 172.21.15.86:0/2625725292 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efef4006a10 0x7efef4006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:54.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.472+0000 7eff0f400640 1 --2- 172.21.15.86:0/2625725292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efef40074f0 0x7efef400dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:54.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.472+0000 7eff0ea00640 1 -- 172.21.15.86:0/2625725292 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efef4006a10 msgr2=0x7efef4006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:54.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.472+0000 7eff0ea00640 1 --2- 172.21.15.86:0/2625725292 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efef4006a10 0x7efef4006e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:54.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.472+0000 7eff0ea00640 1 -- 172.21.15.86:0/2625725292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efef40074f0 msgr2=0x7efef400dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:54.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.472+0000 7eff0ea00640 1 --2- 172.21.15.86:0/2625725292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efef40074f0 0x7efef400dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:54.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.472+0000 7eff0ea00640 1 -- 172.21.15.86:0/2625725292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efef4005730 con 0x7efef4005bc0 2024-04-20T21:19:54.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.472+0000 7eff0f400640 1 -- 172.21.15.86:0/2625725292 reap_dead start 2024-04-20T21:19:54.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.473+0000 7eff0ea00640 1 --2- 172.21.15.86:0/2625725292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efef4005bc0 0x7efef4005fe0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7eff00001720 tx=0x7efef4007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbb3c84d817b00c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:54.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.474+0000 7eff0e000640 1 -- 172.21.15.86:0/2625725292 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff00001370 con 0x7efef4005bc0 2024-04-20T21:19:54.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.474+0000 7eff0e000640 1 -- 172.21.15.86:0/2625725292 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff000052d0 con 0x7efef4005bc0 2024-04-20T21:19:54.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.474+0000 7eff0e000640 1 -- 172.21.15.86:0/2625725292 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff00003830 con 0x7efef4005bc0 2024-04-20T21:19:54.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.474+0000 7eff15800640 1 -- 172.21.15.86:0/2625725292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efef4005bc0 msgr2=0x7efef4005fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:54.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.474+0000 7eff15800640 1 --2- 172.21.15.86:0/2625725292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efef4005bc0 0x7efef4005fe0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7eff00001720 tx=0x7efef4007a40 comp rx=0 tx=0).stop 2024-04-20T21:19:54.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.474+0000 7eff15800640 1 -- 172.21.15.86:0/2625725292 shutdown_connections 2024-04-20T21:19:54.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.474+0000 7eff15800640 1 --2- 172.21.15.86:0/2625725292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efef4005bc0 0x7efef4005fe0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:54.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.474+0000 7eff15800640 1 -- 172.21.15.86:0/2625725292 >> 172.21.15.86:0/2625725292 conn(0x7eff100b2110 msgr2=0x7eff100b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:54.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.474+0000 7eff15800640 1 -- 172.21.15.86:0/2625725292 shutdown_connections 2024-04-20T21:19:54.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.474+0000 7eff15800640 1 -- 172.21.15.86:0/2625725292 wait complete. 2024-04-20T21:19:54.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.476+0000 7eff15800640 1 Processor -- start 2024-04-20T21:19:54.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.476+0000 7eff15800640 1 -- start start 2024-04-20T21:19:54.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.477+0000 7eff15800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efef4005bc0 0x7eff1015e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:54.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.477+0000 7eff15800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff101f8740 0x7eff101f7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:54.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.477+0000 7eff15800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff1014bc70 0x7eff1014c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:54.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.477+0000 7eff15800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7eff1006b980 con 0x7eff101f8740 2024-04-20T21:19:54.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.477+0000 7eff15800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7eff100cefe0 con 0x7eff1014bc70 2024-04-20T21:19:54.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.477+0000 7eff15800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7eff1006bb00 con 0x7efef4005bc0 2024-04-20T21:19:54.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.477+0000 7eff07e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff101f8740 0x7eff101f7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:54.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.478+0000 7eff07e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff101f8740 0x7eff101f7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39740/0 (socket says 172.21.15.86:39740) 2024-04-20T21:19:54.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.478+0000 7eff07e00640 1 -- 172.21.15.86:0/347165947 learned_addr learned my addr 172.21.15.86:0/347165947 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:54.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.478+0000 7eff0ea00640 1 --2- 172.21.15.86:0/347165947 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efef4005bc0 0x7eff1015e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:54.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.478+0000 7eff0f400640 1 --2- 172.21.15.86:0/347165947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff1014bc70 0x7eff1014c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:54.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.478+0000 7eff07e00640 1 -- 172.21.15.86:0/347165947 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efef4005bc0 msgr2=0x7eff1015e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:54.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.478+0000 7eff07e00640 1 --2- 172.21.15.86:0/347165947 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efef4005bc0 0x7eff1015e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:54.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.478+0000 7eff07e00640 1 -- 172.21.15.86:0/347165947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff1014bc70 msgr2=0x7eff1014c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:54.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.478+0000 7eff07e00640 1 --2- 172.21.15.86:0/347165947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff1014bc70 0x7eff1014c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:54.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.478+0000 7eff07e00640 1 -- 172.21.15.86:0/347165947 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff101518c0 con 0x7eff101f8740 2024-04-20T21:19:54.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.478+0000 7eff0ea00640 1 --2- 172.21.15.86:0/347165947 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efef4005bc0 0x7eff1015e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:54.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.478+0000 7eff07e00640 1 --2- 172.21.15.86:0/347165947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff101f8740 0x7eff101f7da0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7efef8000ff0 tx=0x7efef800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:54.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.478+0000 7eff06a00640 1 -- 172.21.15.86:0/347165947 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef8019070 con 0x7eff101f8740 2024-04-20T21:19:54.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.479+0000 7eff06a00640 1 -- 172.21.15.86:0/347165947 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efef80072d0 con 0x7eff101f8740 2024-04-20T21:19:54.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.479+0000 7eff15800640 1 -- 172.21.15.86:0/347165947 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff10151ba0 con 0x7eff101f8740 2024-04-20T21:19:54.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.479+0000 7eff06a00640 1 -- 172.21.15.86:0/347165947 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef80047e0 con 0x7eff101f8740 2024-04-20T21:19:54.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.479+0000 7eff15800640 1 -- 172.21.15.86:0/347165947 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff10152160 con 0x7eff101f8740 2024-04-20T21:19:54.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.480+0000 7eff15800640 1 -- 172.21.15.86:0/347165947 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff10152650 con 0x7eff101f8740 2024-04-20T21:19:54.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.481+0000 7eff06a00640 1 -- 172.21.15.86:0/347165947 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efef801c070 con 0x7eff101f8740 2024-04-20T21:19:54.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.482+0000 7eff06a00640 1 --2- 172.21.15.86:0/347165947 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efeec06c8f0 0x7efeec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:54.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.482+0000 7eff06a00640 1 -- 172.21.15.86:0/347165947 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7efef808b040 con 0x7eff101f8740 2024-04-20T21:19:54.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.482+0000 7eff0ea00640 1 --2- 172.21.15.86:0/347165947 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efeec06c8f0 0x7efeec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:54.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.483+0000 7eff0ea00640 1 --2- 172.21.15.86:0/347165947 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efeec06c8f0 0x7efeec06edb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7eff00033000 tx=0x7eff00009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:54.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.487+0000 7eff06a00640 1 -- 172.21.15.86:0/347165947 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7efef8059100 con 0x7eff101f8740 2024-04-20T21:19:54.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.687+0000 7eff15800640 1 -- 172.21.15.86:0/347165947 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7eff100689e0 con 0x7eff101f8740 2024-04-20T21:19:54.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.688+0000 7eff06a00640 1 -- 172.21.15.86:0/347165947 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) ==== 93+0+10 (secure 0 0 0) 0x7efef805d400 con 0x7eff101f8740 2024-04-20T21:19:54.687 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:19:54.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.690+0000 7eff15800640 1 -- 172.21.15.86:0/347165947 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efeec06c8f0 msgr2=0x7efeec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:54.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.690+0000 7eff15800640 1 --2- 172.21.15.86:0/347165947 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efeec06c8f0 0x7efeec06edb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7eff00033000 tx=0x7eff00009030 comp rx=0 tx=0).stop 2024-04-20T21:19:54.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.690+0000 7eff15800640 1 -- 172.21.15.86:0/347165947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff101f8740 msgr2=0x7eff101f7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:54.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.690+0000 7eff15800640 1 --2- 172.21.15.86:0/347165947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff101f8740 0x7eff101f7da0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7efef8000ff0 tx=0x7efef800dc00 comp rx=0 tx=0).stop 2024-04-20T21:19:54.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.691+0000 7eff15800640 1 -- 172.21.15.86:0/347165947 shutdown_connections 2024-04-20T21:19:54.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.691+0000 7eff15800640 1 --2- 172.21.15.86:0/347165947 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efeec06c8f0 0x7efeec06edb0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:54.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.691+0000 7eff15800640 1 --2- 172.21.15.86:0/347165947 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff1014bc70 0x7eff1014c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:54.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.691+0000 7eff15800640 1 --2- 172.21.15.86:0/347165947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff101f8740 0x7eff101f7da0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:54.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.691+0000 7eff15800640 1 --2- 172.21.15.86:0/347165947 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efef4005bc0 0x7eff1015e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:54.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.691+0000 7eff15800640 1 -- 172.21.15.86:0/347165947 >> 172.21.15.86:0/347165947 conn(0x7eff100b2110 msgr2=0x7eff100af6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:54.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.691+0000 7eff15800640 1 -- 172.21.15.86:0/347165947 shutdown_connections 2024-04-20T21:19:54.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.691+0000 7eff15800640 1 -- 172.21.15.86:0/347165947 wait complete. 2024-04-20T21:19:54.710 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-04-20T21:19:54.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.912+0000 7fc218c00640 1 Processor -- start 2024-04-20T21:19:54.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.912+0000 7fc218c00640 1 -- start start 2024-04-20T21:19:54.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.914+0000 7fc218c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc214146130 0x7fc21406e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:54.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.914+0000 7fc218c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc21406dc30 con 0x7fc2140ccdd0 2024-04-20T21:19:54.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.914+0000 7fc218c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2140cd8e0 con 0x7fc214072410 2024-04-20T21:19:54.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.914+0000 7fc218c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc2140cdbc0 con 0x7fc214146130 2024-04-20T21:19:54.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.914+0000 7fc211400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fc2140ccdd0 0x7fc214071d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:47336/0 (socket says 172.21.15.86:47336) 2024-04-20T21:19:54.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.914+0000 7fc211400640 1 -- 172.21.15.86:0/3605958050 learned_addr learned my addr 172.21.15.86:0/3605958050 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:19:54.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.914+0000 7fc211e00640 1 --2- 172.21.15.86:0/3605958050 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc214146130 0x7fc21406e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:54.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.915+0000 7fc210a00640 1 -- 172.21.15.86:0/3605958050 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 595549450 0 0) 0x7fc21406dc30 con 0x7fc2140ccdd0 2024-04-20T21:19:54.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.915+0000 7fc210a00640 1 -- 172.21.15.86:0/3605958050 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc1f8003520 con 0x7fc2140ccdd0 2024-04-20T21:19:54.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.915+0000 7fc210a00640 1 -- 172.21.15.86:0/3605958050 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3399670551 0 0) 0x7fc1f8003520 con 0x7fc2140ccdd0 2024-04-20T21:19:54.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.915+0000 7fc210a00640 1 -- 172.21.15.86:0/3605958050 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc214146130 msgr2=0x7fc21406e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:54.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.915+0000 7fc210a00640 1 --2- 172.21.15.86:0/3605958050 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc214146130 0x7fc21406e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:54.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.915+0000 7fc210a00640 1 -- 172.21.15.86:0/3605958050 >> v1:172.21.15.144:6789/0 conn(0x7fc214072410 legacy=0x7fc21415cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:54.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.915+0000 7fc210a00640 1 -- 172.21.15.86:0/3605958050 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc21406ded0 con 0x7fc2140ccdd0 2024-04-20T21:19:54.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.915+0000 7fc210a00640 1 -- 172.21.15.86:0/3605958050 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fc200001670 con 0x7fc2140ccdd0 2024-04-20T21:19:54.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.915+0000 7fc210a00640 1 -- 172.21.15.86:0/3605958050 >> v1:172.21.15.86:6789/0 conn(0x7fc2140ccdd0 legacy=0x7fc214071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:54.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.915+0000 7fc210a00640 1 --2- 172.21.15.86:0/3605958050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1f80039e0 0x7fc1f8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:54.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.915+0000 7fc210a00640 1 --2- 172.21.15.86:0/3605958050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1f8004800 0x7fc1f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:54.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.915+0000 7fc210a00640 1 --2- 172.21.15.86:0/3605958050 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc214146130 0x7fc1f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:54.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.915+0000 7fc210a00640 1 -- 172.21.15.86:0/3605958050 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc2140cdbc0 con 0x7fc1f80039e0 2024-04-20T21:19:54.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.915+0000 7fc210a00640 1 -- 172.21.15.86:0/3605958050 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f800bb00 con 0x7fc1f8004800 2024-04-20T21:19:54.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.915+0000 7fc210a00640 1 -- 172.21.15.86:0/3605958050 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc1f800bd00 con 0x7fc214146130 2024-04-20T21:19:54.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.915+0000 7fc211e00640 1 --2- 172.21.15.86:0/3605958050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1f80039e0 0x7fc1f8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:54.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.916+0000 7fc211400640 1 --2- 172.21.15.86:0/3605958050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1f8004800 0x7fc1f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:54.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.916+0000 7fc211e00640 1 -- 172.21.15.86:0/3605958050 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc214146130 msgr2=0x7fc1f80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:54.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.916+0000 7fc211e00640 1 --2- 172.21.15.86:0/3605958050 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc214146130 0x7fc1f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:54.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.916+0000 7fc211e00640 1 -- 172.21.15.86:0/3605958050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1f8004800 msgr2=0x7fc1f8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:54.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.916+0000 7fc211e00640 1 --2- 172.21.15.86:0/3605958050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1f8004800 0x7fc1f8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:54.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.916+0000 7fc211e00640 1 -- 172.21.15.86:0/3605958050 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1f8003520 con 0x7fc1f80039e0 2024-04-20T21:19:54.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.916+0000 7fc211400640 1 --2- 172.21.15.86:0/3605958050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1f8004800 0x7fc1f8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:54.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.917+0000 7fc211e00640 1 --2- 172.21.15.86:0/3605958050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1f80039e0 0x7fc1f8003e00 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc204009a90 tx=0x7fc1f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=863dc63fea5063be server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:54.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.917+0000 7fc210a00640 1 -- 172.21.15.86:0/3605958050 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc204039070 con 0x7fc1f80039e0 2024-04-20T21:19:54.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.917+0000 7fc210a00640 1 -- 172.21.15.86:0/3605958050 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc20402c6b0 con 0x7fc1f80039e0 2024-04-20T21:19:54.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.917+0000 7fc210a00640 1 -- 172.21.15.86:0/3605958050 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc204009690 con 0x7fc1f80039e0 2024-04-20T21:19:54.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.918+0000 7fc218c00640 1 -- 172.21.15.86:0/3605958050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1f80039e0 msgr2=0x7fc1f8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:54.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.918+0000 7fc218c00640 1 --2- 172.21.15.86:0/3605958050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1f80039e0 0x7fc1f8003e00 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc204009a90 tx=0x7fc1f8005830 comp rx=0 tx=0).stop 2024-04-20T21:19:54.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.918+0000 7fc212800640 1 -- 172.21.15.86:0/3605958050 reap_dead start 2024-04-20T21:19:54.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.918+0000 7fc218c00640 1 -- 172.21.15.86:0/3605958050 shutdown_connections 2024-04-20T21:19:54.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.918+0000 7fc218c00640 1 -- 172.21.15.86:0/3605958050 >> 172.21.15.86:0/3605958050 conn(0x7fc2140b2360 msgr2=0x7fc2140b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:54.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.918+0000 7fc218c00640 1 -- 172.21.15.86:0/3605958050 shutdown_connections 2024-04-20T21:19:54.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.918+0000 7fc218c00640 1 -- 172.21.15.86:0/3605958050 wait complete. 2024-04-20T21:19:54.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.920+0000 7fc218c00640 1 Processor -- start 2024-04-20T21:19:54.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.920+0000 7fc218c00640 1 -- start start 2024-04-20T21:19:54.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.921+0000 7fc218c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1f80039e0 0x7fc21415bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:54.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.921+0000 7fc218c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2141e7f40 0x7fc2141e83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:54.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.921+0000 7fc218c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2140cb390 0x7fc2140cb810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:54.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.921+0000 7fc218c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc2140cd230 con 0x7fc1f80039e0 2024-04-20T21:19:54.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.921+0000 7fc218c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc21406f150 con 0x7fc2140cb390 2024-04-20T21:19:54.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.921+0000 7fc218c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc2140cd3b0 con 0x7fc2141e7f40 2024-04-20T21:19:54.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.921+0000 7fc211e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1f80039e0 0x7fc21415bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:54.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.921+0000 7fc211e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1f80039e0 0x7fc21415bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39756/0 (socket says 172.21.15.86:39756) 2024-04-20T21:19:54.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.921+0000 7fc211e00640 1 -- 172.21.15.86:0/3609518639 learned_addr learned my addr 172.21.15.86:0/3609518639 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:54.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.921+0000 7fc211e00640 1 -- 172.21.15.86:0/3609518639 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2141e7f40 msgr2=0x7fc2141e83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:54.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.921+0000 7fc211e00640 1 --2- 172.21.15.86:0/3609518639 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2141e7f40 0x7fc2141e83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:54.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.921+0000 7fc211e00640 1 -- 172.21.15.86:0/3609518639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2140cb390 msgr2=0x7fc2140cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:54.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.921+0000 7fc211e00640 1 --2- 172.21.15.86:0/3609518639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2140cb390 0x7fc2140cb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:54.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.921+0000 7fc211e00640 1 -- 172.21.15.86:0/3609518639 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2140cc080 con 0x7fc1f80039e0 2024-04-20T21:19:54.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.922+0000 7fc211e00640 1 --2- 172.21.15.86:0/3609518639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1f80039e0 0x7fc21415bfc0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fc204009bb0 tx=0x7fc204004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:54.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.922+0000 7fc20a000640 1 -- 172.21.15.86:0/3609518639 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc204042020 con 0x7fc1f80039e0 2024-04-20T21:19:54.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.922+0000 7fc20a000640 1 -- 172.21.15.86:0/3609518639 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc20403a040 con 0x7fc1f80039e0 2024-04-20T21:19:54.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.922+0000 7fc20a000640 1 -- 172.21.15.86:0/3609518639 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc204039070 con 0x7fc1f80039e0 2024-04-20T21:19:54.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.922+0000 7fc218c00640 1 -- 172.21.15.86:0/3609518639 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2141e5810 con 0x7fc1f80039e0 2024-04-20T21:19:54.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.922+0000 7fc218c00640 1 -- 172.21.15.86:0/3609518639 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2140cbd50 con 0x7fc1f80039e0 2024-04-20T21:19:54.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.923+0000 7fc218c00640 1 -- 172.21.15.86:0/3609518639 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2140cc360 con 0x7fc1f80039e0 2024-04-20T21:19:54.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.924+0000 7fc20a000640 1 -- 172.21.15.86:0/3609518639 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc204045020 con 0x7fc1f80039e0 2024-04-20T21:19:54.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.925+0000 7fc20a000640 1 --2- 172.21.15.86:0/3609518639 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc1f406c920 0x7fc1f406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:54.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.925+0000 7fc20a000640 1 -- 172.21.15.86:0/3609518639 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7fc2040acb80 con 0x7fc1f80039e0 2024-04-20T21:19:54.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.926+0000 7fc211400640 1 --2- 172.21.15.86:0/3609518639 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc1f406c920 0x7fc1f406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:54.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.926+0000 7fc211400640 1 --2- 172.21.15.86:0/3609518639 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc1f406c920 0x7fc1f406ede0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc200001200 tx=0x7fc200009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:54.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:54.930+0000 7fc20a000640 1 -- 172.21.15.86:0/3609518639 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc20404c070 con 0x7fc1f80039e0 2024-04-20T21:19:55.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:55.202+0000 7fc218c00640 1 -- 172.21.15.86:0/3609518639 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) -- 0x7fc2140689e0 con 0x7fc1f80039e0 2024-04-20T21:19:55.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:55.822+0000 7fc20a000640 1 -- 172.21.15.86:0/3609518639 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v137) ==== 152+0+0 (secure 0 0 0) 0x7fc20407b2a0 con 0x7fc1f80039e0 2024-04-20T21:19:55.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:55.825+0000 7fc218c00640 1 -- 172.21.15.86:0/3609518639 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc1f406c920 msgr2=0x7fc1f406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:55.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:55.825+0000 7fc218c00640 1 --2- 172.21.15.86:0/3609518639 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc1f406c920 0x7fc1f406ede0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc200001200 tx=0x7fc200009040 comp rx=0 tx=0).stop 2024-04-20T21:19:55.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:55.825+0000 7fc218c00640 1 -- 172.21.15.86:0/3609518639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1f80039e0 msgr2=0x7fc21415bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:55.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:55.825+0000 7fc218c00640 1 --2- 172.21.15.86:0/3609518639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1f80039e0 0x7fc21415bfc0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fc204009bb0 tx=0x7fc204004730 comp rx=0 tx=0).stop 2024-04-20T21:19:55.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:55.825+0000 7fc218c00640 1 -- 172.21.15.86:0/3609518639 shutdown_connections 2024-04-20T21:19:55.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:55.825+0000 7fc218c00640 1 --2- 172.21.15.86:0/3609518639 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc1f406c920 0x7fc1f406ede0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:55.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:55.825+0000 7fc218c00640 1 --2- 172.21.15.86:0/3609518639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2140cb390 0x7fc2140cb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:55.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:55.825+0000 7fc218c00640 1 --2- 172.21.15.86:0/3609518639 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2141e7f40 0x7fc2141e83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:55.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:55.825+0000 7fc218c00640 1 --2- 172.21.15.86:0/3609518639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1f80039e0 0x7fc21415bfc0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:55.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:55.826+0000 7fc218c00640 1 -- 172.21.15.86:0/3609518639 >> 172.21.15.86:0/3609518639 conn(0x7fc2140b2360 msgr2=0x7fc21406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:55.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:55.826+0000 7fc218c00640 1 -- 172.21.15.86:0/3609518639 shutdown_connections 2024-04-20T21:19:55.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:55.826+0000 7fc218c00640 1 -- 172.21.15.86:0/3609518639 wait complete. 2024-04-20T21:19:55.846 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-20T21:19:56.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.044+0000 7fd092600640 1 Processor -- start 2024-04-20T21:19:56.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.044+0000 7fd092600640 1 -- start start 2024-04-20T21:19:56.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.046+0000 7fd092600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd08c145250 0x7fd08c06efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:56.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.046+0000 7fd092600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd08c145f00 0x7fd08c071d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:56.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.046+0000 7fd092600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd08c06f500 con 0x7fd08c072280 2024-04-20T21:19:56.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.046+0000 7fd092600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd08c0726b0 con 0x7fd08c145250 2024-04-20T21:19:56.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.046+0000 7fd092600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fd08c160410 con 0x7fd08c145f00 2024-04-20T21:19:56.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.046+0000 7fd091200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd08c145250 0x7fd08c06efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:56.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.046+0000 7fd091c00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd08c072280 0x7fd08c06dda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41534/0 (socket says 172.21.15.86:41534) 2024-04-20T21:19:56.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.046+0000 7fd091c00640 1 -- 172.21.15.86:0/2309666040 learned_addr learned my addr 172.21.15.86:0/2309666040 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:19:56.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.046+0000 7fd091200640 1 -- 172.21.15.86:0/2309666040 >> v2:172.21.15.144:3300/0 conn(0x7fd08c145f00 msgr2=0x7fd08c071d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:56.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.046+0000 7fd08be00640 1 --2- 172.21.15.86:0/2309666040 >> v2:172.21.15.144:3300/0 conn(0x7fd08c145f00 0x7fd08c071d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:56.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.047+0000 7fd091200640 1 --2- 172.21.15.86:0/2309666040 >> v2:172.21.15.144:3300/0 conn(0x7fd08c145f00 0x7fd08c071d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:56.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.047+0000 7fd091200640 1 -- 172.21.15.86:0/2309666040 >> v1:172.21.15.86:6789/0 conn(0x7fd08c072280 legacy=0x7fd08c06dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:56.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.047+0000 7fd08b400640 1 -- 172.21.15.86:0/2309666040 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2999007309 0 0) 0x7fd084001490 con 0x7fd08c072280 2024-04-20T21:19:56.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.047+0000 7fd091200640 1 -- 172.21.15.86:0/2309666040 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd08c1606f0 con 0x7fd08c145250 2024-04-20T21:19:56.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.047+0000 7fd08be00640 1 --2- 172.21.15.86:0/2309666040 >> v2:172.21.15.144:3300/0 conn(0x7fd08c145f00 0x7fd08c071d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:56.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.048+0000 7fd091200640 1 --2- 172.21.15.86:0/2309666040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd08c145250 0x7fd08c06efc0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd08c06c540 tx=0x7fd08002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9387860f7ec911da server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:56.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.048+0000 7fd08b400640 1 -- 172.21.15.86:0/2309666040 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0800315a0 con 0x7fd08c145250 2024-04-20T21:19:56.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.048+0000 7fd08b400640 1 -- 172.21.15.86:0/2309666040 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd080037070 con 0x7fd08c145250 2024-04-20T21:19:56.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.049+0000 7fd08b400640 1 -- 172.21.15.86:0/2309666040 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd080031960 con 0x7fd08c145250 2024-04-20T21:19:56.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.049+0000 7fd092600640 1 -- 172.21.15.86:0/2309666040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd08c145250 msgr2=0x7fd08c06efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:56.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.049+0000 7fd092600640 1 --2- 172.21.15.86:0/2309666040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd08c145250 0x7fd08c06efc0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd08c06c540 tx=0x7fd08002b840 comp rx=0 tx=0).stop 2024-04-20T21:19:56.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.049+0000 7fd092600640 1 -- 172.21.15.86:0/2309666040 shutdown_connections 2024-04-20T21:19:56.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.049+0000 7fd092600640 1 --2- 172.21.15.86:0/2309666040 >> v2:172.21.15.144:3300/0 conn(0x7fd08c145f00 0x7fd08c071d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:56.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.049+0000 7fd092600640 1 --2- 172.21.15.86:0/2309666040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd08c145250 0x7fd08c06efc0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:56.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.049+0000 7fd092600640 1 -- 172.21.15.86:0/2309666040 >> 172.21.15.86:0/2309666040 conn(0x7fd08c067cf0 msgr2=0x7fd08c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:56.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.049+0000 7fd092600640 1 -- 172.21.15.86:0/2309666040 shutdown_connections 2024-04-20T21:19:56.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.049+0000 7fd092600640 1 -- 172.21.15.86:0/2309666040 wait complete. 2024-04-20T21:19:56.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.051+0000 7fd092600640 1 Processor -- start 2024-04-20T21:19:56.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.051+0000 7fd092600640 1 -- start start 2024-04-20T21:19:56.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.052+0000 7fd092600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd08c072280 0x7fd08c0a9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:56.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.052+0000 7fd092600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd08c145250 0x7fd08c1f76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:56.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.052+0000 7fd092600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd08c145f00 0x7fd08c0cac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:56.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.052+0000 7fd092600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd08c145630 con 0x7fd08c145f00 2024-04-20T21:19:56.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.052+0000 7fd092600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd08c160cb0 con 0x7fd08c145250 2024-04-20T21:19:56.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.052+0000 7fd092600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd08c1457b0 con 0x7fd08c072280 2024-04-20T21:19:56.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.052+0000 7fd091c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd08c145f00 0x7fd08c0cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:56.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.052+0000 7fd091c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd08c145f00 0x7fd08c0cac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60558/0 (socket says 172.21.15.86:60558) 2024-04-20T21:19:56.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.052+0000 7fd091c00640 1 -- 172.21.15.86:0/2464951220 learned_addr learned my addr 172.21.15.86:0/2464951220 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:56.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.053+0000 7fd08be00640 1 --2- 172.21.15.86:0/2464951220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd08c145250 0x7fd08c1f76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:56.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.053+0000 7fd091200640 1 --2- 172.21.15.86:0/2464951220 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd08c072280 0x7fd08c0a9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:56.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.053+0000 7fd091c00640 1 -- 172.21.15.86:0/2464951220 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd08c072280 msgr2=0x7fd08c0a9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:56.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.053+0000 7fd091c00640 1 --2- 172.21.15.86:0/2464951220 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd08c072280 0x7fd08c0a9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:56.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.053+0000 7fd091c00640 1 -- 172.21.15.86:0/2464951220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd08c145250 msgr2=0x7fd08c1f76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:56.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.053+0000 7fd091c00640 1 --2- 172.21.15.86:0/2464951220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd08c145250 0x7fd08c1f76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:56.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.053+0000 7fd091c00640 1 -- 172.21.15.86:0/2464951220 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd08c0aa430 con 0x7fd08c145f00 2024-04-20T21:19:56.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.053+0000 7fd091200640 1 --2- 172.21.15.86:0/2464951220 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd08c072280 0x7fd08c0a9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:56.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.053+0000 7fd091c00640 1 --2- 172.21.15.86:0/2464951220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd08c145f00 0x7fd08c0cac60 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fd08400c7f0 tx=0x7fd08400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:56.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.054+0000 7fd088c00640 1 -- 172.21.15.86:0/2464951220 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd084018070 con 0x7fd08c145f00 2024-04-20T21:19:56.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.054+0000 7fd088c00640 1 -- 172.21.15.86:0/2464951220 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd08400a5e0 con 0x7fd08c145f00 2024-04-20T21:19:56.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.054+0000 7fd092600640 1 -- 172.21.15.86:0/2464951220 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd08c0aa710 con 0x7fd08c145f00 2024-04-20T21:19:56.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.054+0000 7fd088c00640 1 -- 172.21.15.86:0/2464951220 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08400a9d0 con 0x7fd08c145f00 2024-04-20T21:19:56.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.054+0000 7fd092600640 1 -- 172.21.15.86:0/2464951220 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd08c0aacd0 con 0x7fd08c145f00 2024-04-20T21:19:56.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.055+0000 7fd092600640 1 -- 172.21.15.86:0/2464951220 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd08c04fd90 con 0x7fd08c145f00 2024-04-20T21:19:56.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.056+0000 7fd088c00640 1 -- 172.21.15.86:0/2464951220 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd08401b070 con 0x7fd08c145f00 2024-04-20T21:19:56.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.057+0000 7fd088c00640 1 --2- 172.21.15.86:0/2464951220 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd06c06c8f0 0x7fd06c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:56.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.057+0000 7fd088c00640 1 -- 172.21.15.86:0/2464951220 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7fd08408af00 con 0x7fd08c145f00 2024-04-20T21:19:56.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.057+0000 7fd091200640 1 --2- 172.21.15.86:0/2464951220 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd06c06c8f0 0x7fd06c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:56.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.058+0000 7fd091200640 1 --2- 172.21.15.86:0/2464951220 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd06c06c8f0 0x7fd06c06edb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd080036000 tx=0x7fd080008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:56.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.061+0000 7fd088c00640 1 -- 172.21.15.86:0/2464951220 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd08c06f500 con 0x7fd08c145f00 2024-04-20T21:19:56.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.259+0000 7fd092600640 1 -- 172.21.15.86:0/2464951220 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fd08c072ea0 con 0x7fd08c145f00 2024-04-20T21:19:56.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.266+0000 7fd088c00640 1 -- 172.21.15.86:0/2464951220 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7fd084059590 con 0x7fd08c145f00 2024-04-20T21:19:56.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.268+0000 7fd092600640 1 -- 172.21.15.86:0/2464951220 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd06c06c8f0 msgr2=0x7fd06c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:56.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.268+0000 7fd092600640 1 --2- 172.21.15.86:0/2464951220 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd06c06c8f0 0x7fd06c06edb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd080036000 tx=0x7fd080008030 comp rx=0 tx=0).stop 2024-04-20T21:19:56.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.269+0000 7fd092600640 1 -- 172.21.15.86:0/2464951220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd08c145f00 msgr2=0x7fd08c0cac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:56.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.269+0000 7fd092600640 1 --2- 172.21.15.86:0/2464951220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd08c145f00 0x7fd08c0cac60 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fd08400c7f0 tx=0x7fd08400ccb0 comp rx=0 tx=0).stop 2024-04-20T21:19:56.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.269+0000 7fd092600640 1 -- 172.21.15.86:0/2464951220 shutdown_connections 2024-04-20T21:19:56.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.269+0000 7fd092600640 1 --2- 172.21.15.86:0/2464951220 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd06c06c8f0 0x7fd06c06edb0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:56.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.269+0000 7fd092600640 1 --2- 172.21.15.86:0/2464951220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd08c145f00 0x7fd08c0cac60 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:56.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.269+0000 7fd092600640 1 --2- 172.21.15.86:0/2464951220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd08c145250 0x7fd08c1f76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:56.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.269+0000 7fd092600640 1 --2- 172.21.15.86:0/2464951220 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd08c072280 0x7fd08c0a9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:56.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.269+0000 7fd092600640 1 -- 172.21.15.86:0/2464951220 >> 172.21.15.86:0/2464951220 conn(0x7fd08c067cf0 msgr2=0x7fd08c0b4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:56.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.269+0000 7fd092600640 1 -- 172.21.15.86:0/2464951220 shutdown_connections 2024-04-20T21:19:56.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.269+0000 7fd092600640 1 -- 172.21.15.86:0/2464951220 wait complete. 2024-04-20T21:19:56.285 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-20T21:19:56.286 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T21:19:56.386 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T21:19:56.489 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-20T21:19:56.489 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:19:56.489 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:19:56.546 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052731,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248996k,nr_inodes=812249,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T21:19:56.546 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:19:56.546 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:19:56.603 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T21:19:56.603 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:19:56.603 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:19:56.661 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052731,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248996k,nr_inodes=812249,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T21:19:56.661 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:19:56.662 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:19:56.717 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T21:19:56.717 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-20T21:19:56.717 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:19:56.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.925+0000 7f4fd5000640 1 Processor -- start 2024-04-20T21:19:56.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.925+0000 7f4fd5000640 1 -- start start 2024-04-20T21:19:56.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.927+0000 7f4fd5000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4fd006e6d0 0x7f4fd0071d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:56.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.927+0000 7f4fd5000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4fd0072240 0x7f4fd015cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:56.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.927+0000 7f4fd5000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fd006eb30 con 0x7f4fd0146130 2024-04-20T21:19:56.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.927+0000 7f4fd5000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f4fd006dc30 con 0x7f4fd006e6d0 2024-04-20T21:19:56.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.927+0000 7f4fd5000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f4fd00cd8e0 con 0x7f4fd0072240 2024-04-20T21:19:56.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.927+0000 7f4fcde00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f4fd0146130 0x7f4fd00ccdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41550/0 (socket says 172.21.15.86:41550) 2024-04-20T21:19:56.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.927+0000 7f4fcde00640 1 -- 172.21.15.86:0/3093228441 learned_addr learned my addr 172.21.15.86:0/3093228441 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:19:56.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.927+0000 7f4fce800640 1 --2- 172.21.15.86:0/3093228441 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4fd0072240 0x7f4fd015cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:56.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.927+0000 7f4fcca00640 1 -- 172.21.15.86:0/3093228441 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3394593378 0 0) 0x7f4fc0001110 con 0x7f4fd0146130 2024-04-20T21:19:56.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.928+0000 7f4fcd400640 1 --2- 172.21.15.86:0/3093228441 >> v2:172.21.15.144:3300/0 conn(0x7f4fd006e6d0 0x7f4fd0071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:56.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.928+0000 7f4fcca00640 1 -- 172.21.15.86:0/3093228441 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4fb4003520 con 0x7f4fd0146130 2024-04-20T21:19:56.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.928+0000 7f4fcca00640 1 -- 172.21.15.86:0/3093228441 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 591687515 0 0) 0x7f4fb4003520 con 0x7f4fd0146130 2024-04-20T21:19:56.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.928+0000 7f4fcca00640 1 -- 172.21.15.86:0/3093228441 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4fd0072240 msgr2=0x7f4fd015cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:56.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.928+0000 7f4fcca00640 1 --2- 172.21.15.86:0/3093228441 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4fd0072240 0x7f4fd015cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:56.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.928+0000 7f4fcca00640 1 -- 172.21.15.86:0/3093228441 >> v2:172.21.15.144:3300/0 conn(0x7f4fd006e6d0 msgr2=0x7f4fd0071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:56.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.928+0000 7f4fcca00640 1 --2- 172.21.15.86:0/3093228441 >> v2:172.21.15.144:3300/0 conn(0x7f4fd006e6d0 0x7f4fd0071d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:56.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.928+0000 7f4fcca00640 1 -- 172.21.15.86:0/3093228441 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fd00cdbc0 con 0x7f4fd0146130 2024-04-20T21:19:56.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.928+0000 7f4fcca00640 1 -- 172.21.15.86:0/3093228441 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f4fc0001840 con 0x7f4fd0146130 2024-04-20T21:19:56.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.928+0000 7f4fce800640 1 --2- 172.21.15.86:0/3093228441 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4fd0072240 0x7f4fd015cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:19:56.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.928+0000 7f4fcca00640 1 -- 172.21.15.86:0/3093228441 >> v1:172.21.15.86:6789/0 conn(0x7f4fd0146130 legacy=0x7f4fd00ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:56.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.928+0000 7f4fcca00640 1 --2- 172.21.15.86:0/3093228441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fb40039b0 0x7f4fb4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:56.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.928+0000 7f4fcca00640 1 --2- 172.21.15.86:0/3093228441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fb4004800 0x7f4fb4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:56.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.928+0000 7f4fcca00640 1 --2- 172.21.15.86:0/3093228441 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4fd0072240 0x7f4fb40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:56.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.928+0000 7f4fcca00640 1 -- 172.21.15.86:0/3093228441 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd006dc30 con 0x7f4fb40039b0 2024-04-20T21:19:56.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.928+0000 7f4fcca00640 1 -- 172.21.15.86:0/3093228441 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd00cd8e0 con 0x7f4fb4004800 2024-04-20T21:19:56.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.928+0000 7f4fcca00640 1 -- 172.21.15.86:0/3093228441 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f4fb400bb20 con 0x7f4fd0072240 2024-04-20T21:19:56.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.928+0000 7f4fcde00640 1 --2- 172.21.15.86:0/3093228441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fb40039b0 0x7f4fb4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:56.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.929+0000 7f4fcd400640 1 --2- 172.21.15.86:0/3093228441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fb4004800 0x7f4fb4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:56.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.929+0000 7f4fce800640 1 --2- 172.21.15.86:0/3093228441 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4fd0072240 0x7f4fb40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:56.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.929+0000 7f4fcde00640 1 -- 172.21.15.86:0/3093228441 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4fd0072240 msgr2=0x7f4fb40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:56.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.929+0000 7f4fcde00640 1 --2- 172.21.15.86:0/3093228441 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4fd0072240 0x7f4fb40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:56.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.929+0000 7f4fcde00640 1 -- 172.21.15.86:0/3093228441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fb4004800 msgr2=0x7f4fb4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:56.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.929+0000 7f4fcde00640 1 --2- 172.21.15.86:0/3093228441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fb4004800 0x7f4fb4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:56.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.929+0000 7f4fcde00640 1 -- 172.21.15.86:0/3093228441 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fb4003520 con 0x7f4fb40039b0 2024-04-20T21:19:56.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.930+0000 7f4fcde00640 1 --2- 172.21.15.86:0/3093228441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fb40039b0 0x7f4fb4003dd0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4fc000c2f0 tx=0x7f4fb4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9838b0c702db5635 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:56.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.930+0000 7f4fcca00640 1 -- 172.21.15.86:0/3093228441 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc002cc80 con 0x7f4fb40039b0 2024-04-20T21:19:56.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.930+0000 7f4fcca00640 1 -- 172.21.15.86:0/3093228441 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fc0030070 con 0x7f4fb40039b0 2024-04-20T21:19:56.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.930+0000 7f4fcca00640 1 -- 172.21.15.86:0/3093228441 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc0005b10 con 0x7f4fb40039b0 2024-04-20T21:19:56.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.931+0000 7f4fd5000640 1 -- 172.21.15.86:0/3093228441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fb40039b0 msgr2=0x7f4fb4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:56.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.931+0000 7f4fd5000640 1 --2- 172.21.15.86:0/3093228441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fb40039b0 0x7f4fb4003dd0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4fc000c2f0 tx=0x7f4fb4005830 comp rx=0 tx=0).stop 2024-04-20T21:19:56.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.931+0000 7f4fce800640 1 -- 172.21.15.86:0/3093228441 reap_dead start 2024-04-20T21:19:56.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.931+0000 7f4fd5000640 1 -- 172.21.15.86:0/3093228441 shutdown_connections 2024-04-20T21:19:56.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.931+0000 7f4fd5000640 1 -- 172.21.15.86:0/3093228441 >> 172.21.15.86:0/3093228441 conn(0x7f4fd00b2360 msgr2=0x7f4fd00b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:56.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.931+0000 7f4fd5000640 1 -- 172.21.15.86:0/3093228441 shutdown_connections 2024-04-20T21:19:56.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.931+0000 7f4fd5000640 1 -- 172.21.15.86:0/3093228441 wait complete. 2024-04-20T21:19:56.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.933+0000 7f4fd5000640 1 Processor -- start 2024-04-20T21:19:56.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.933+0000 7f4fd5000640 1 -- start start 2024-04-20T21:19:56.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.934+0000 7f4fd5000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4fb40039b0 0x7f4fd015c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:56.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.934+0000 7f4fd5000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fd00ae260 0x7f4fd00ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:56.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.934+0000 7f4fd5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fd00b43c0 0x7f4fd00b4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:56.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.934+0000 7f4fd5000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd006f150 con 0x7f4fd00b43c0 2024-04-20T21:19:56.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.934+0000 7f4fd5000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd006e1b0 con 0x7f4fd00ae260 2024-04-20T21:19:56.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.934+0000 7f4fd5000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f4fd00cd4e0 con 0x7f4fb40039b0 2024-04-20T21:19:56.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.934+0000 7f4fce800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fd00b43c0 0x7f4fd00b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:56.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.934+0000 7f4fce800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fd00b43c0 0x7f4fd00b4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60568/0 (socket says 172.21.15.86:60568) 2024-04-20T21:19:56.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.934+0000 7f4fce800640 1 -- 172.21.15.86:0/2686530809 learned_addr learned my addr 172.21.15.86:0/2686530809 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:56.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.935+0000 7f4fcde00640 1 --2- 172.21.15.86:0/2686530809 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4fb40039b0 0x7f4fd015c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:56.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.935+0000 7f4fce800640 1 -- 172.21.15.86:0/2686530809 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4fb40039b0 msgr2=0x7f4fd015c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:56.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.935+0000 7f4fce800640 1 --2- 172.21.15.86:0/2686530809 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4fb40039b0 0x7f4fd015c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:56.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.935+0000 7f4fce800640 1 -- 172.21.15.86:0/2686530809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fd00ae260 msgr2=0x7f4fd00ae6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:56.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.935+0000 7f4fce800640 1 --2- 172.21.15.86:0/2686530809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fd00ae260 0x7f4fd00ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:56.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.935+0000 7f4fce800640 1 -- 172.21.15.86:0/2686530809 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fd00aee20 con 0x7f4fd00b43c0 2024-04-20T21:19:56.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.935+0000 7f4fce800640 1 --2- 172.21.15.86:0/2686530809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fd00b43c0 0x7f4fd00b4840 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f4fd006db60 tx=0x7f4fc8004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:56.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.935+0000 7f4fc6000640 1 -- 172.21.15.86:0/2686530809 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc8007c90 con 0x7f4fd00b43c0 2024-04-20T21:19:56.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.935+0000 7f4fc6000640 1 -- 172.21.15.86:0/2686530809 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fc8019070 con 0x7f4fd00b43c0 2024-04-20T21:19:56.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.935+0000 7f4fc6000640 1 -- 172.21.15.86:0/2686530809 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc8014650 con 0x7f4fd00b43c0 2024-04-20T21:19:56.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.935+0000 7f4fd5000640 1 -- 172.21.15.86:0/2686530809 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fd00af100 con 0x7f4fd00b43c0 2024-04-20T21:19:56.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.935+0000 7f4fd5000640 1 -- 172.21.15.86:0/2686530809 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fd01e61b0 con 0x7f4fd00b43c0 2024-04-20T21:19:56.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.936+0000 7f4fd5000640 1 -- 172.21.15.86:0/2686530809 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fd00689e0 con 0x7f4fd00b43c0 2024-04-20T21:19:56.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.937+0000 7f4fc6000640 1 -- 172.21.15.86:0/2686530809 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4fc80147f0 con 0x7f4fd00b43c0 2024-04-20T21:19:56.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.939+0000 7f4fc6000640 1 --2- 172.21.15.86:0/2686530809 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4fb006c8f0 0x7f4fb006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:56.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.939+0000 7f4fc6000640 1 -- 172.21.15.86:0/2686530809 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f4fc808bdf0 con 0x7f4fd00b43c0 2024-04-20T21:19:56.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.939+0000 7f4fcde00640 1 --2- 172.21.15.86:0/2686530809 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4fb006c8f0 0x7f4fb006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:56.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.940+0000 7f4fcde00640 1 --2- 172.21.15.86:0/2686530809 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4fb006c8f0 0x7f4fb006edb0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0034000 tx=0x7f4fc0007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:56.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:56.943+0000 7f4fc6000640 1 -- 172.21.15.86:0/2686530809 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4fc805a480 con 0x7f4fd00b43c0 2024-04-20T21:19:57.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.176+0000 7f4fd5000640 1 -- 172.21.15.86:0/2686530809 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4fd015ff50 con 0x7f4fd00b43c0 2024-04-20T21:19:57.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.177+0000 7f4fc6000640 1 -- 172.21.15.86:0/2686530809 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+3875 (secure 0 0 0) 0x7f4fc8059d80 con 0x7f4fd00b43c0 2024-04-20T21:19:57.177 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:19:57.177 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4604,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/1957936003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":1957936003},{"type":"v1","addr":"172.21.15.144:6834","nonce":1957936003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4807,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/1510649389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":1510649389},{"type":"v1","addr":"172.21.15.144:6835","nonce":1510649389}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2180009156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2180009156},{"type":"v1","addr":"172.21.15.86:6836","nonce":2180009156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5607,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1576442046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":1576442046},{"type":"v1","addr":"172.21.15.86:6837","nonce":1576442046}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[]} 2024-04-20T21:19:57.177 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 18 2024-04-20T21:19:57.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.180+0000 7f4fd5000640 1 -- 172.21.15.86:0/2686530809 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4fb006c8f0 msgr2=0x7f4fb006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:57.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.180+0000 7f4fd5000640 1 --2- 172.21.15.86:0/2686530809 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4fb006c8f0 0x7f4fb006edb0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0034000 tx=0x7f4fc0007000 comp rx=0 tx=0).stop 2024-04-20T21:19:57.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.180+0000 7f4fd5000640 1 -- 172.21.15.86:0/2686530809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fd00b43c0 msgr2=0x7f4fd00b4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:57.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.180+0000 7f4fd5000640 1 --2- 172.21.15.86:0/2686530809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fd00b43c0 0x7f4fd00b4840 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f4fd006db60 tx=0x7f4fc8004700 comp rx=0 tx=0).stop 2024-04-20T21:19:57.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.180+0000 7f4fd5000640 1 -- 172.21.15.86:0/2686530809 shutdown_connections 2024-04-20T21:19:57.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.180+0000 7f4fd5000640 1 --2- 172.21.15.86:0/2686530809 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4fb006c8f0 0x7f4fb006edb0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:57.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.180+0000 7f4fd5000640 1 --2- 172.21.15.86:0/2686530809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fd00b43c0 0x7f4fd00b4840 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:57.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.180+0000 7f4fd5000640 1 --2- 172.21.15.86:0/2686530809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fd00ae260 0x7f4fd00ae6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:57.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.180+0000 7f4fd5000640 1 --2- 172.21.15.86:0/2686530809 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4fb40039b0 0x7f4fd015c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:57.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.180+0000 7f4fd5000640 1 -- 172.21.15.86:0/2686530809 >> 172.21.15.86:0/2686530809 conn(0x7f4fd00b2360 msgr2=0x7f4fd01445d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:57.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.180+0000 7f4fd5000640 1 -- 172.21.15.86:0/2686530809 shutdown_connections 2024-04-20T21:19:57.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.180+0000 7f4fd5000640 1 -- 172.21.15.86:0/2686530809 wait complete. 2024-04-20T21:19:57.198 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-20T21:19:57.198 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-20T21:19:57.199 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-20T21:19:57.199 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:19:57.199 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-20T21:19:57.199 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-20T21:19:57.199 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-20T21:19:57.199 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:19:57.199 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-20T21:19:57.199 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-20T21:19:57.200 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-20T21:19:57.200 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:19:57.200 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-20T21:19:57.200 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-20T21:19:57.200 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-20T21:19:57.200 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:19:57.216 INFO:tasks.ceph.mds.c:Stopped 2024-04-20T21:19:57.216 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-20T21:19:57.219 INFO:tasks.ceph.mds.a:Stopped 2024-04-20T21:19:57.220 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-20T21:19:57.223 INFO:tasks.ceph.mds.d:Stopped 2024-04-20T21:19:57.223 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-20T21:19:57.225 INFO:tasks.ceph.mds.b:Stopped 2024-04-20T21:19:57.225 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-20T21:19:57.264 INFO:tasks.ceph.mds.c:Started 2024-04-20T21:19:57.264 DEBUG:teuthology.parallel:result is None 2024-04-20T21:19:57.271 INFO:tasks.ceph.mds.d:Started 2024-04-20T21:19:57.271 DEBUG:teuthology.parallel:result is None 2024-04-20T21:19:57.272 INFO:tasks.ceph.mds.a:Started 2024-04-20T21:19:57.272 DEBUG:teuthology.parallel:result is None 2024-04-20T21:19:57.277 INFO:tasks.ceph.mds.b:Started 2024-04-20T21:19:57.277 DEBUG:teuthology.parallel:result is None 2024-04-20T21:19:57.278 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-20T21:19:57.404 INFO:tasks.ceph.mds.c.smithi086.stdout:starting mds.c at 2024-04-20T21:19:57.412 INFO:tasks.ceph.mds.a.smithi086.stdout:starting mds.a at 2024-04-20T21:19:57.422 INFO:tasks.ceph.mds.d.smithi144.stdout:starting mds.d at 2024-04-20T21:19:57.425 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2024-04-20T21:19:57.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.456+0000 7f1f56000640 1 Processor -- start 2024-04-20T21:19:57.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.456+0000 7f1f56000640 1 -- start start 2024-04-20T21:19:57.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.457+0000 7f1f56000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1f5015baf0 0x7f1f5015bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:57.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.457+0000 7f1f56000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f5015c4b0 0x7f1f500cbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:57.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.457+0000 7f1f56000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f500ced40 con 0x7f1f5015b710 2024-04-20T21:19:57.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.457+0000 7f1f56000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f1f500cdce0 con 0x7f1f5015baf0 2024-04-20T21:19:57.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.457+0000 7f1f56000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1f500cdf80 con 0x7f1f5015c4b0 2024-04-20T21:19:57.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.458+0000 7f1f46a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1f5015baf0 0x7f1f5015bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:57.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.458+0000 7f1f46a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1f5015baf0 0x7f1f5015bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60602/0 (socket says 172.21.15.86:60602) 2024-04-20T21:19:57.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.458+0000 7f1f46a00640 1 -- 172.21.15.86:0/2983761499 learned_addr learned my addr 172.21.15.86:0/2983761499 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:57.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.458+0000 7f1f4fe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f5015c4b0 0x7f1f500cbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:57.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.458+0000 7f1f46a00640 1 -- 172.21.15.86:0/2983761499 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f5015c4b0 msgr2=0x7f1f500cbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:57.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.458+0000 7f1f46a00640 1 --2- 172.21.15.86:0/2983761499 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f5015c4b0 0x7f1f500cbf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:57.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.458+0000 7f1f46a00640 1 -- 172.21.15.86:0/2983761499 >> v1:172.21.15.144:6789/0 conn(0x7f1f5015b710 legacy=0x7f1f500cfe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:57.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.458+0000 7f1f46a00640 1 -- 172.21.15.86:0/2983761499 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f500ce260 con 0x7f1f5015baf0 2024-04-20T21:19:57.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.458+0000 7f1f4ea00640 1 -- 172.21.15.86:0/2983761499 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2759863564 0 0) 0x7f1f500ced40 con 0x7f1f5015b710 2024-04-20T21:19:57.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.458+0000 7f1f4fe00640 1 --2- 172.21.15.86:0/2983761499 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f5015c4b0 0x7f1f500cbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:19:57.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.459+0000 7f1f46a00640 1 --2- 172.21.15.86:0/2983761499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f5015baf0 0x7f1f5015bf70 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f1f38009870 tx=0x7f1f3802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a31dc37161a0062b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:57.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.459+0000 7f1f4ea00640 1 -- 172.21.15.86:0/2983761499 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f38035020 con 0x7f1f5015baf0 2024-04-20T21:19:57.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.459+0000 7f1f4ea00640 1 -- 172.21.15.86:0/2983761499 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f38031640 con 0x7f1f5015baf0 2024-04-20T21:19:57.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.459+0000 7f1f4ea00640 1 -- 172.21.15.86:0/2983761499 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3802d040 con 0x7f1f5015baf0 2024-04-20T21:19:57.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.460+0000 7f1f56000640 1 -- 172.21.15.86:0/2983761499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f5015baf0 msgr2=0x7f1f5015bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:57.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.460+0000 7f1f56000640 1 --2- 172.21.15.86:0/2983761499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f5015baf0 0x7f1f5015bf70 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f1f38009870 tx=0x7f1f3802b8f0 comp rx=0 tx=0).stop 2024-04-20T21:19:57.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.460+0000 7f1f56000640 1 -- 172.21.15.86:0/2983761499 shutdown_connections 2024-04-20T21:19:57.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.460+0000 7f1f56000640 1 --2- 172.21.15.86:0/2983761499 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f5015c4b0 0x7f1f500cbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:57.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.460+0000 7f1f56000640 1 --2- 172.21.15.86:0/2983761499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f5015baf0 0x7f1f5015bf70 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:57.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.460+0000 7f1f56000640 1 -- 172.21.15.86:0/2983761499 >> 172.21.15.86:0/2983761499 conn(0x7f1f500b2fd0 msgr2=0x7f1f500b33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:57.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.460+0000 7f1f56000640 1 -- 172.21.15.86:0/2983761499 shutdown_connections 2024-04-20T21:19:57.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.460+0000 7f1f56000640 1 -- 172.21.15.86:0/2983761499 wait complete. 2024-04-20T21:19:57.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.462+0000 7f1f56000640 1 Processor -- start 2024-04-20T21:19:57.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.462+0000 7f1f56000640 1 -- start start 2024-04-20T21:19:57.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.463+0000 7f1f56000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f5015b710 0x7f1f5015a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:57.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.463+0000 7f1f56000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f5015baf0 0x7f1f50154ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:57.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.463+0000 7f1f56000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f5015c4b0 0x7f1f500ca4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:57.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.463+0000 7f1f56000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5015c940 con 0x7f1f5015c4b0 2024-04-20T21:19:57.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.463+0000 7f1f56000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1f500ce820 con 0x7f1f5015baf0 2024-04-20T21:19:57.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.463+0000 7f1f56000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1f5015cac0 con 0x7f1f5015b710 2024-04-20T21:19:57.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.463+0000 7f1f4fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f5015c4b0 0x7f1f500ca4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:57.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.463+0000 7f1f4fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f5015c4b0 0x7f1f500ca4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60604/0 (socket says 172.21.15.86:60604) 2024-04-20T21:19:57.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.463+0000 7f1f4fe00640 1 -- 172.21.15.86:0/1059507806 learned_addr learned my addr 172.21.15.86:0/1059507806 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:57.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.463+0000 7f1f4f400640 1 --2- 172.21.15.86:0/1059507806 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f5015b710 0x7f1f5015a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:57.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.463+0000 7f1f46a00640 1 --2- 172.21.15.86:0/1059507806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f5015baf0 0x7f1f50154ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:57.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.464+0000 7f1f4fe00640 1 -- 172.21.15.86:0/1059507806 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f5015b710 msgr2=0x7f1f5015a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:57.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.464+0000 7f1f4fe00640 1 --2- 172.21.15.86:0/1059507806 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f5015b710 0x7f1f5015a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:57.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.464+0000 7f1f4fe00640 1 -- 172.21.15.86:0/1059507806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f5015baf0 msgr2=0x7f1f50154ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:57.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.464+0000 7f1f4fe00640 1 --2- 172.21.15.86:0/1059507806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f5015baf0 0x7f1f50154ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:57.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.464+0000 7f1f4fe00640 1 -- 172.21.15.86:0/1059507806 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f501556b0 con 0x7f1f5015c4b0 2024-04-20T21:19:57.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.464+0000 7f1f4fe00640 1 --2- 172.21.15.86:0/1059507806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f5015c4b0 0x7f1f500ca4e0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f1f4800e880 tx=0x7f1f4800ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:57.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.464+0000 7f1f47e00640 1 -- 172.21.15.86:0/1059507806 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f48017070 con 0x7f1f5015c4b0 2024-04-20T21:19:57.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.464+0000 7f1f47e00640 1 -- 172.21.15.86:0/1059507806 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f48004020 con 0x7f1f5015c4b0 2024-04-20T21:19:57.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.464+0000 7f1f56000640 1 -- 172.21.15.86:0/1059507806 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f50155990 con 0x7f1f5015c4b0 2024-04-20T21:19:57.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.464+0000 7f1f47e00640 1 -- 172.21.15.86:0/1059507806 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f480043f0 con 0x7f1f5015c4b0 2024-04-20T21:19:57.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.464+0000 7f1f56000640 1 -- 172.21.15.86:0/1059507806 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f5015af00 con 0x7f1f5015c4b0 2024-04-20T21:19:57.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.466+0000 7f1f56000640 1 -- 172.21.15.86:0/1059507806 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f500690f0 con 0x7f1f5015c4b0 2024-04-20T21:19:57.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.467+0000 7f1f47e00640 1 -- 172.21.15.86:0/1059507806 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1f4800c940 con 0x7f1f5015c4b0 2024-04-20T21:19:57.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.467+0000 7f1f47e00640 1 --2- 172.21.15.86:0/1059507806 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1f2c06c8f0 0x7f1f2c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:57.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.467+0000 7f1f47e00640 1 -- 172.21.15.86:0/1059507806 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4943+0+0 (secure 0 0 0) 0x7f1f4808b8a0 con 0x7f1f5015c4b0 2024-04-20T21:19:57.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.468+0000 7f1f4f400640 1 --2- 172.21.15.86:0/1059507806 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1f2c06c8f0 0x7f1f2c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:57.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.468+0000 7f1f4f400640 1 --2- 172.21.15.86:0/1059507806 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1f2c06c8f0 0x7f1f2c06edb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1f400012e0 tx=0x7f1f40009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:57.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.472+0000 7f1f47e00640 1 -- 172.21.15.86:0/1059507806 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1f48059a10 con 0x7f1f5015c4b0 2024-04-20T21:19:57.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:57.681+0000 7f1f56000640 1 -- 172.21.15.86:0/1059507806 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f1f5006d8d0 con 0x7f1f5015c4b0 2024-04-20T21:19:58.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:58.861+0000 7f1f47e00640 1 -- 172.21.15.86:0/1059507806 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v41) ==== 97+0+0 (secure 0 0 0) 0x7f1f4805dd10 con 0x7f1f5015c4b0 2024-04-20T21:19:59.052 INFO:teuthology.orchestra.run.smithi086.stderr: removed all blocklist entries 2024-04-20T21:19:59.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:58.866+0000 7f1f56000640 1 -- 172.21.15.86:0/1059507806 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1f2c06c8f0 msgr2=0x7f1f2c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:59.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:58.866+0000 7f1f56000640 1 --2- 172.21.15.86:0/1059507806 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1f2c06c8f0 0x7f1f2c06edb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1f400012e0 tx=0x7f1f40009040 comp rx=0 tx=0).stop 2024-04-20T21:19:59.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:58.866+0000 7f1f56000640 1 -- 172.21.15.86:0/1059507806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f5015c4b0 msgr2=0x7f1f500ca4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:59.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:58.866+0000 7f1f56000640 1 --2- 172.21.15.86:0/1059507806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f5015c4b0 0x7f1f500ca4e0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f1f4800e880 tx=0x7f1f4800ed40 comp rx=0 tx=0).stop 2024-04-20T21:19:59.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:58.866+0000 7f1f56000640 1 -- 172.21.15.86:0/1059507806 shutdown_connections 2024-04-20T21:19:59.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:58.866+0000 7f1f56000640 1 --2- 172.21.15.86:0/1059507806 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1f2c06c8f0 0x7f1f2c06edb0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:59.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:58.866+0000 7f1f56000640 1 --2- 172.21.15.86:0/1059507806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f5015c4b0 0x7f1f500ca4e0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:59.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:58.866+0000 7f1f56000640 1 --2- 172.21.15.86:0/1059507806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f5015baf0 0x7f1f50154ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:59.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:58.866+0000 7f1f56000640 1 --2- 172.21.15.86:0/1059507806 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f5015b710 0x7f1f5015a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:59.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:58.866+0000 7f1f56000640 1 -- 172.21.15.86:0/1059507806 >> 172.21.15.86:0/1059507806 conn(0x7f1f500b2fd0 msgr2=0x7f1f500b1300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:59.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:58.866+0000 7f1f56000640 1 -- 172.21.15.86:0/1059507806 shutdown_connections 2024-04-20T21:19:59.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:58.866+0000 7f1f56000640 1 -- 172.21.15.86:0/1059507806 wait complete. 2024-04-20T21:19:59.055 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-20T21:19:59.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.261+0000 7f8d1e400640 1 Processor -- start 2024-04-20T21:19:59.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.261+0000 7f8d1e400640 1 -- start start 2024-04-20T21:19:59.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.262+0000 7f8d1e400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8d18071d40 0x7f8d180cead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:59.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.262+0000 7f8d1e400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d1806efc0 0x7f8d18072120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:59.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.263+0000 7f8d1e400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8d1806dda0 0x7f8d1806e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:59.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.263+0000 7f8d1e400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8d181457e0 con 0x7f8d1806dda0 2024-04-20T21:19:59.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.263+0000 7f8d1e400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f8d18160e10 con 0x7f8d18071d40 2024-04-20T21:19:59.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.263+0000 7f8d1e400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8d18072660 con 0x7f8d1806efc0 2024-04-20T21:19:59.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.263+0000 7f8d17e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8d1806dda0 0x7f8d1806e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:59.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.263+0000 7f8d17e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8d1806dda0 0x7f8d1806e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60620/0 (socket says 172.21.15.86:60620) 2024-04-20T21:19:59.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.263+0000 7f8d17e00640 1 -- 172.21.15.86:0/2928840003 learned_addr learned my addr 172.21.15.86:0/2928840003 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:59.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.263+0000 7f8d0ea00640 1 --2- 172.21.15.86:0/2928840003 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d1806efc0 0x7f8d18072120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:59.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.263+0000 7f8d17e00640 1 -- 172.21.15.86:0/2928840003 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d1806efc0 msgr2=0x7f8d18072120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:59.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.263+0000 7f8d17e00640 1 --2- 172.21.15.86:0/2928840003 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d1806efc0 0x7f8d18072120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:59.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.263+0000 7f8d17e00640 1 -- 172.21.15.86:0/2928840003 >> v2:172.21.15.144:3300/0 conn(0x7f8d18071d40 msgr2=0x7f8d180cead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:59.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.263+0000 7f8d17e00640 1 --2- 172.21.15.86:0/2928840003 >> v2:172.21.15.144:3300/0 conn(0x7f8d18071d40 0x7f8d180cead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:59.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.263+0000 7f8d17e00640 1 -- 172.21.15.86:0/2928840003 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d18160200 con 0x7f8d1806dda0 2024-04-20T21:19:59.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.264+0000 7f8d17e00640 1 --2- 172.21.15.86:0/2928840003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d1806dda0 0x7f8d1806e230 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f8d180b5a00 tx=0x7f8d1002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=188a38b7d2fae803 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:59.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.265+0000 7f8d16a00640 1 -- 172.21.15.86:0/2928840003 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1002cb00 con 0x7f8d1806dda0 2024-04-20T21:19:59.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.265+0000 7f8d16a00640 1 -- 172.21.15.86:0/2928840003 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d1002cca0 con 0x7f8d1806dda0 2024-04-20T21:19:59.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.265+0000 7f8d16a00640 1 -- 172.21.15.86:0/2928840003 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d10034610 con 0x7f8d1806dda0 2024-04-20T21:19:59.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.265+0000 7f8d1e400640 1 -- 172.21.15.86:0/2928840003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d1806dda0 msgr2=0x7f8d1806e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:59.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.265+0000 7f8d1e400640 1 --2- 172.21.15.86:0/2928840003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d1806dda0 0x7f8d1806e230 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f8d180b5a00 tx=0x7f8d1002bf60 comp rx=0 tx=0).stop 2024-04-20T21:19:59.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.266+0000 7f8d1e400640 1 -- 172.21.15.86:0/2928840003 shutdown_connections 2024-04-20T21:19:59.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.266+0000 7f8d1e400640 1 --2- 172.21.15.86:0/2928840003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d1806dda0 0x7f8d1806e230 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:59.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.266+0000 7f8d1e400640 1 --2- 172.21.15.86:0/2928840003 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d1806efc0 0x7f8d18072120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:59.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.266+0000 7f8d1e400640 1 --2- 172.21.15.86:0/2928840003 >> v2:172.21.15.144:3300/0 conn(0x7f8d18071d40 0x7f8d180cead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:59.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.266+0000 7f8d1e400640 1 -- 172.21.15.86:0/2928840003 >> 172.21.15.86:0/2928840003 conn(0x7f8d1806b980 msgr2=0x7f8d1806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:59.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.266+0000 7f8d1e400640 1 -- 172.21.15.86:0/2928840003 shutdown_connections 2024-04-20T21:19:59.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.266+0000 7f8d1e400640 1 -- 172.21.15.86:0/2928840003 wait complete. 2024-04-20T21:19:59.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.267+0000 7f8d1e400640 1 Processor -- start 2024-04-20T21:19:59.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.268+0000 7f8d1e400640 1 -- start start 2024-04-20T21:19:59.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.268+0000 7f8d1e400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d1806dda0 0x7f8d181f43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:59.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.268+0000 7f8d1e400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d1806efc0 0x7f8d181e79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:59.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.269+0000 7f8d1e400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d18071d40 0x7f8d181e82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:59.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.269+0000 7f8d1e400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8d18160aa0 con 0x7f8d18071d40 2024-04-20T21:19:59.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.269+0000 7f8d1e400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8d181607c0 con 0x7f8d1806efc0 2024-04-20T21:19:59.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.269+0000 7f8d1e400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8d1806f430 con 0x7f8d1806dda0 2024-04-20T21:19:59.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.269+0000 7f8d17e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d18071d40 0x7f8d181e82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:59.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.269+0000 7f8d17e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d18071d40 0x7f8d181e82e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60626/0 (socket says 172.21.15.86:60626) 2024-04-20T21:19:59.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.269+0000 7f8d17e00640 1 -- 172.21.15.86:0/944141034 learned_addr learned my addr 172.21.15.86:0/944141034 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:59.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.269+0000 7f8d17e00640 1 -- 172.21.15.86:0/944141034 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d1806dda0 msgr2=0x7f8d181f43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:59.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.269+0000 7f8d17400640 1 --2- 172.21.15.86:0/944141034 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d1806dda0 0x7f8d181f43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:59.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.269+0000 7f8d0ea00640 1 --2- 172.21.15.86:0/944141034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d1806efc0 0x7f8d181e79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:59.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.269+0000 7f8d17e00640 1 --2- 172.21.15.86:0/944141034 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d1806dda0 0x7f8d181f43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:59.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.269+0000 7f8d17e00640 1 -- 172.21.15.86:0/944141034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d1806efc0 msgr2=0x7f8d181e79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:59.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.269+0000 7f8d17e00640 1 --2- 172.21.15.86:0/944141034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d1806efc0 0x7f8d181e79c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:59.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.269+0000 7f8d17e00640 1 -- 172.21.15.86:0/944141034 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d181f53a0 con 0x7f8d18071d40 2024-04-20T21:19:59.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.269+0000 7f8d17400640 1 --2- 172.21.15.86:0/944141034 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d1806dda0 0x7f8d181f43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:59.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.270+0000 7f8d17e00640 1 --2- 172.21.15.86:0/944141034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d18071d40 0x7f8d181e82e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8d10036040 tx=0x7f8d10004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:59.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.270+0000 7f8d0fe00640 1 -- 172.21.15.86:0/944141034 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d10043020 con 0x7f8d18071d40 2024-04-20T21:19:59.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.270+0000 7f8d0fe00640 1 -- 172.21.15.86:0/944141034 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d1002ce50 con 0x7f8d18071d40 2024-04-20T21:19:59.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.270+0000 7f8d1e400640 1 -- 172.21.15.86:0/944141034 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d18142890 con 0x7f8d18071d40 2024-04-20T21:19:59.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.270+0000 7f8d0fe00640 1 -- 172.21.15.86:0/944141034 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1003e650 con 0x7f8d18071d40 2024-04-20T21:19:59.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.270+0000 7f8d1e400640 1 -- 172.21.15.86:0/944141034 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d181f4de0 con 0x7f8d18071d40 2024-04-20T21:19:59.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.271+0000 7f8d1e400640 1 -- 172.21.15.86:0/944141034 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d180689e0 con 0x7f8d18071d40 2024-04-20T21:19:59.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.272+0000 7f8d0fe00640 1 -- 172.21.15.86:0/944141034 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8d10046020 con 0x7f8d18071d40 2024-04-20T21:19:59.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.273+0000 7f8d0fe00640 1 --2- 172.21.15.86:0/944141034 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8cf406c8f0 0x7f8cf406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:59.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.274+0000 7f8d0fe00640 1 -- 172.21.15.86:0/944141034 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4857+0+0 (secure 0 0 0) 0x7f8d100adfe0 con 0x7f8d18071d40 2024-04-20T21:19:59.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.274+0000 7f8d17400640 1 --2- 172.21.15.86:0/944141034 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8cf406c8f0 0x7f8cf406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:59.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.274+0000 7f8d17400640 1 --2- 172.21.15.86:0/944141034 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8cf406c8f0 0x7f8cf406edb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8d08006fd0 tx=0x7f8d08008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:59.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.279+0000 7f8d0fe00640 1 -- 172.21.15.86:0/944141034 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8d1004d030 con 0x7f8d18071d40 2024-04-20T21:19:59.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.551+0000 7f8d1e400640 1 -- 172.21.15.86:0/944141034 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f8d18143580 con 0x7f8d18071d40 2024-04-20T21:19:59.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.552+0000 7f8d0fe00640 1 -- 172.21.15.86:0/944141034 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) ==== 80+0+5304 (secure 0 0 0) 0x7f8d1007c6e0 con 0x7f8d18071d40 2024-04-20T21:19:59.551 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:19:59.551 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:19:59.551 INFO:teuthology.orchestra.run.smithi086.stdout: "auth_dump": [ 2024-04-20T21:19:59.551 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:19:59.551 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.a", 2024-04-20T21:19:59.551 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDAMCRmOpS/LRAAPZUUQNf6TyeJ2tF7YPnIKw==", 2024-04-20T21:19:59.551 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:19:59.551 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:19:59.551 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-20T21:19:59.552 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-20T21:19:59.552 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:19:59.552 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:19:59.552 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:19:59.552 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:19:59.552 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.b", 2024-04-20T21:19:59.552 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDBMCRmCJLpDRAA/JiLRSAlQRNMpDC+qDSr0g==", 2024-04-20T21:19:59.552 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:19:59.552 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:19:59.552 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-20T21:19:59.552 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-20T21:19:59.552 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:19:59.552 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:19:59.552 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:19:59.552 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:19:59.553 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.c", 2024-04-20T21:19:59.553 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDAMCRmPHOFOxAA4tUsw/n6oSKrNmzPD+NKyw==", 2024-04-20T21:19:59.553 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:19:59.553 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:19:59.553 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-20T21:19:59.553 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-20T21:19:59.553 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:19:59.553 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:19:59.553 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:19:59.553 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:19:59.553 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.d", 2024-04-20T21:19:59.553 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDBMCRmZnPXHBAA0jXwBmGehaO7qens4boRIQ==", 2024-04-20T21:19:59.553 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:19:59.554 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:19:59.554 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-20T21:19:59.554 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-20T21:19:59.554 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:19:59.554 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:19:59.554 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:19:59.554 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:19:59.554 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.0", 2024-04-20T21:19:59.554 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDFMCRm4w9UHhAA56y4dJyDASI+IoZv2XrEoQ==", 2024-04-20T21:19:59.554 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:19:59.554 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:19:59.554 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:19:59.554 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:19:59.554 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:19:59.555 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:19:59.555 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:19:59.555 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.1", 2024-04-20T21:19:59.555 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDIMCRmFo7xKRAAou5B04+J1gZtTzW/Mo+nRQ==", 2024-04-20T21:19:59.555 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:19:59.555 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:19:59.555 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:19:59.555 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:19:59.555 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:19:59.555 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:19:59.555 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:19:59.555 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.2", 2024-04-20T21:19:59.555 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDLMCRm7/jFNRAA4KcRsapPFLaKf0NaH5MApg==", 2024-04-20T21:19:59.555 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:19:59.555 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:19:59.556 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:19:59.556 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:19:59.556 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:19:59.556 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:19:59.556 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:19:59.556 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.3", 2024-04-20T21:19:59.556 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDPMCRmUTPMExAA0/dubb4ehsWIwZPS/ao68w==", 2024-04-20T21:19:59.556 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:19:59.556 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:19:59.556 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:19:59.556 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:19:59.556 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:19:59.556 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:19:59.556 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:19:59.557 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.4", 2024-04-20T21:19:59.557 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDVMCRmB4F2KRAA8Xj0DfH5Ou8TUrO8X/BQyw==", 2024-04-20T21:19:59.557 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:19:59.557 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:19:59.557 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:19:59.557 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:19:59.557 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:19:59.557 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:19:59.557 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:19:59.557 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.5", 2024-04-20T21:19:59.557 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDYMCRmSOvYNRAAODYXmRHMGPrCAxcHwlS7Ww==", 2024-04-20T21:19:59.557 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:19:59.557 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:19:59.557 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:19:59.558 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:19:59.558 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:19:59.558 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:19:59.558 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:19:59.558 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.6", 2024-04-20T21:19:59.558 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDcMCRmENJ9BBAAi/KBUhbYGNXEyiSjSHavWQ==", 2024-04-20T21:19:59.558 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:19:59.558 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:19:59.558 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:19:59.558 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:19:59.558 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:19:59.558 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:19:59.559 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:19:59.559 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.7", 2024-04-20T21:19:59.559 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDfMCRmfTHTDhAALL9GZBSGjWYrhgFw/YI6IA==", 2024-04-20T21:19:59.559 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:19:59.559 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:19:59.559 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:19:59.559 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:19:59.559 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:19:59.559 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:19:59.559 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:19:59.559 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.0", 2024-04-20T21:19:59.559 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDBMCRmHWxyKRAANzhRi4eajp68h/jWkhFMnw==", 2024-04-20T21:19:59.560 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:19:59.560 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:19:59.560 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow r", 2024-04-20T21:19:59.560 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-20T21:19:59.560 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:19:59.560 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:19:59.560 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:19:59.560 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.1", 2024-04-20T21:19:59.560 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDBMCRmgBJAOxAADJLzO97tjEpKn6kUs8g2QA==", 2024-04-20T21:19:59.560 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:19:59.560 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:19:59.560 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow r", 2024-04-20T21:19:59.560 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow rw", 2024-04-20T21:19:59.560 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rwx" 2024-04-20T21:19:59.561 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:19:59.561 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:19:59.561 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:19:59.561 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.2", 2024-04-20T21:19:59.561 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDBMCRmQAdJMhAAiUmKhPEeuV+UPqQ3gynG9w==", 2024-04-20T21:19:59.561 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:19:59.561 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:19:59.561 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow r", 2024-04-20T21:19:59.561 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow rw", 2024-04-20T21:19:59.561 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rwx" 2024-04-20T21:19:59.561 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:19:59.561 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:19:59.561 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:19:59.561 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.3", 2024-04-20T21:19:59.561 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDCMCRm8yMSCBAAFgSVW0XYu7fa40YxHITljQ==", 2024-04-20T21:19:59.562 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:19:59.562 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:19:59.562 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow r", 2024-04-20T21:19:59.562 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow rw", 2024-04-20T21:19:59.562 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rwx" 2024-04-20T21:19:59.562 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:19:59.562 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:19:59.562 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:19:59.562 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.admin", 2024-04-20T21:19:59.562 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQC/MCRmyru0KBAApSqv1hZRcLvrRwPMnqylqQ==", 2024-04-20T21:19:59.562 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:19:59.562 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-04-20T21:19:59.562 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-20T21:19:59.562 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-20T21:19:59.563 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:19:59.563 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:19:59.563 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:19:59.563 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:19:59.563 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mgr.x", 2024-04-20T21:19:59.563 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDAMCRmvTgjIxAACeSV8gNkL5LV3xzPS6VPeg==", 2024-04-20T21:19:59.563 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:19:59.563 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-04-20T21:19:59.563 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile mgr", 2024-04-20T21:19:59.563 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:19:59.563 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:19:59.563 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:19:59.563 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:19:59.563 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mgr.y", 2024-04-20T21:19:59.563 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDAMCRmSn3VGBAA0eu3R53f3gFgscWFMRkDzg==", 2024-04-20T21:19:59.564 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:19:59.564 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-04-20T21:19:59.564 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile mgr", 2024-04-20T21:19:59.564 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:19:59.564 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:19:59.564 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:19:59.564 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-20T21:19:59.564 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:19:59.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.554+0000 7f8d1e400640 1 -- 172.21.15.86:0/944141034 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8cf406c8f0 msgr2=0x7f8cf406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:59.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.554+0000 7f8d1e400640 1 --2- 172.21.15.86:0/944141034 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8cf406c8f0 0x7f8cf406edb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8d08006fd0 tx=0x7f8d08008040 comp rx=0 tx=0).stop 2024-04-20T21:19:59.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.554+0000 7f8d1e400640 1 -- 172.21.15.86:0/944141034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d18071d40 msgr2=0x7f8d181e82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:59.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.554+0000 7f8d1e400640 1 --2- 172.21.15.86:0/944141034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d18071d40 0x7f8d181e82e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8d10036040 tx=0x7f8d10004a40 comp rx=0 tx=0).stop 2024-04-20T21:19:59.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.555+0000 7f8d1e400640 1 -- 172.21.15.86:0/944141034 shutdown_connections 2024-04-20T21:19:59.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.555+0000 7f8d1e400640 1 --2- 172.21.15.86:0/944141034 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8cf406c8f0 0x7f8cf406edb0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:59.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.555+0000 7f8d1e400640 1 --2- 172.21.15.86:0/944141034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d18071d40 0x7f8d181e82e0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:59.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.555+0000 7f8d1e400640 1 --2- 172.21.15.86:0/944141034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d1806efc0 0x7f8d181e79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:59.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.555+0000 7f8d1e400640 1 --2- 172.21.15.86:0/944141034 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8d1806dda0 0x7f8d181f43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:59.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.555+0000 7f8d1e400640 1 -- 172.21.15.86:0/944141034 >> 172.21.15.86:0/944141034 conn(0x7f8d1806b980 msgr2=0x7f8d180b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:59.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.555+0000 7f8d1e400640 1 -- 172.21.15.86:0/944141034 shutdown_connections 2024-04-20T21:19:59.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.555+0000 7f8d1e400640 1 -- 172.21.15.86:0/944141034 wait complete. 2024-04-20T21:19:59.572 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:19:59.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.761+0000 7fd19d000640 1 Processor -- start 2024-04-20T21:19:59.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.761+0000 7fd19d000640 1 -- start start 2024-04-20T21:19:59.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.762+0000 7fd19d000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd19815ece0 0x7fd19815f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:59.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.762+0000 7fd19d000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd1980d50c0 0x7fd1980d3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:59.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.762+0000 7fd19d000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd19815f600 con 0x7fd1980cf9c0 2024-04-20T21:19:59.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.762+0000 7fd19d000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fd1980d3950 con 0x7fd1980d50c0 2024-04-20T21:19:59.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.762+0000 7fd19d000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd1980cec00 con 0x7fd19815ece0 2024-04-20T21:19:59.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.763+0000 7fd197400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd19815ece0 0x7fd19815f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:59.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.763+0000 7fd18ea00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd1980d50c0 0x7fd1980d3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:59.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.763+0000 7fd197400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd19815ece0 0x7fd19815f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:51588/0 (socket says 172.21.15.86:51588) 2024-04-20T21:19:59.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.763+0000 7fd197400640 1 -- 172.21.15.86:0/4199478966 learned_addr learned my addr 172.21.15.86:0/4199478966 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:59.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.764+0000 7fd197400640 1 -- 172.21.15.86:0/4199478966 >> v2:172.21.15.144:3300/0 conn(0x7fd1980d50c0 msgr2=0x7fd1980d3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:19:59.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.764+0000 7fd197400640 1 --2- 172.21.15.86:0/4199478966 >> v2:172.21.15.144:3300/0 conn(0x7fd1980d50c0 0x7fd1980d3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:59.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.764+0000 7fd197400640 1 -- 172.21.15.86:0/4199478966 >> v1:172.21.15.144:6789/0 conn(0x7fd1980cf9c0 legacy=0x7fd198160000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:59.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.764+0000 7fd197400640 1 -- 172.21.15.86:0/4199478966 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1980ceee0 con 0x7fd19815ece0 2024-04-20T21:19:59.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.765+0000 7fd197400640 1 --2- 172.21.15.86:0/4199478966 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd19815ece0 0x7fd19815f0c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd188009e50 tx=0x7fd18802bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f1d4ee77dde031 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:59.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.766+0000 7fd196a00640 1 -- 172.21.15.86:0/4199478966 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18802ca70 con 0x7fd19815ece0 2024-04-20T21:19:59.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.766+0000 7fd196a00640 1 -- 172.21.15.86:0/4199478966 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd18802cc10 con 0x7fd19815ece0 2024-04-20T21:19:59.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.766+0000 7fd196a00640 1 -- 172.21.15.86:0/4199478966 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1880343f0 con 0x7fd19815ece0 2024-04-20T21:19:59.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.767+0000 7fd19d000640 1 -- 172.21.15.86:0/4199478966 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd19815ece0 msgr2=0x7fd19815f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:59.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.767+0000 7fd19d000640 1 --2- 172.21.15.86:0/4199478966 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd19815ece0 0x7fd19815f0c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd188009e50 tx=0x7fd18802bed0 comp rx=0 tx=0).stop 2024-04-20T21:19:59.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.767+0000 7fd19d000640 1 -- 172.21.15.86:0/4199478966 shutdown_connections 2024-04-20T21:19:59.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.767+0000 7fd19d000640 1 --2- 172.21.15.86:0/4199478966 >> v2:172.21.15.144:3300/0 conn(0x7fd1980d50c0 0x7fd1980d3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:59.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.767+0000 7fd19d000640 1 --2- 172.21.15.86:0/4199478966 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd19815ece0 0x7fd19815f0c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:59.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.767+0000 7fd19d000640 1 -- 172.21.15.86:0/4199478966 >> 172.21.15.86:0/4199478966 conn(0x7fd19808bf70 msgr2=0x7fd1980d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:59.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.767+0000 7fd19d000640 1 -- 172.21.15.86:0/4199478966 shutdown_connections 2024-04-20T21:19:59.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.767+0000 7fd19d000640 1 -- 172.21.15.86:0/4199478966 wait complete. 2024-04-20T21:19:59.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.769+0000 7fd19d000640 1 Processor -- start 2024-04-20T21:19:59.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.769+0000 7fd19d000640 1 -- start start 2024-04-20T21:19:59.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.770+0000 7fd19d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1980cf9c0 0x7fd198151a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:59.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.770+0000 7fd19d000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1980d50c0 0x7fd1981f77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:59.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.770+0000 7fd19d000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd19815ece0 0x7fd1980b9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:59.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.770+0000 7fd19d000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd1980d3640 con 0x7fd1980cf9c0 2024-04-20T21:19:59.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.770+0000 7fd19d000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd1980cf4a0 con 0x7fd1980d50c0 2024-04-20T21:19:59.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.770+0000 7fd19d000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd19815ea50 con 0x7fd19815ece0 2024-04-20T21:19:59.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.770+0000 7fd197400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1980cf9c0 0x7fd198151a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:59.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.770+0000 7fd197400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1980cf9c0 0x7fd198151a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60640/0 (socket says 172.21.15.86:60640) 2024-04-20T21:19:59.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.770+0000 7fd197400640 1 -- 172.21.15.86:0/2337388040 learned_addr learned my addr 172.21.15.86:0/2337388040 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:19:59.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.771+0000 7fd18ea00640 1 --2- 172.21.15.86:0/2337388040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1980d50c0 0x7fd1981f77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:59.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.771+0000 7fd197e00640 1 --2- 172.21.15.86:0/2337388040 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd19815ece0 0x7fd1980b9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:59.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.771+0000 7fd197400640 1 -- 172.21.15.86:0/2337388040 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd19815ece0 msgr2=0x7fd1980b9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:59.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.771+0000 7fd197400640 1 --2- 172.21.15.86:0/2337388040 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd19815ece0 0x7fd1980b9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:59.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.771+0000 7fd197400640 1 -- 172.21.15.86:0/2337388040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1980d50c0 msgr2=0x7fd1981f77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:59.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.771+0000 7fd197400640 1 --2- 172.21.15.86:0/2337388040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1980d50c0 0x7fd1981f77a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:59.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.771+0000 7fd197400640 1 -- 172.21.15.86:0/2337388040 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1981f6700 con 0x7fd1980cf9c0 2024-04-20T21:19:59.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.771+0000 7fd197e00640 1 --2- 172.21.15.86:0/2337388040 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd19815ece0 0x7fd1980b9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:19:59.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.771+0000 7fd197400640 1 --2- 172.21.15.86:0/2337388040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1980cf9c0 0x7fd198151a80 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fd188000f80 tx=0x7fd188004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:59.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.771+0000 7fd18fe00640 1 -- 172.21.15.86:0/2337388040 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd188044020 con 0x7fd1980cf9c0 2024-04-20T21:19:59.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.771+0000 7fd18fe00640 1 -- 172.21.15.86:0/2337388040 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd188040670 con 0x7fd1980cf9c0 2024-04-20T21:19:59.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.771+0000 7fd18fe00640 1 -- 172.21.15.86:0/2337388040 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd188009510 con 0x7fd1980cf9c0 2024-04-20T21:19:59.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.771+0000 7fd19d000640 1 -- 172.21.15.86:0/2337388040 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1981f69e0 con 0x7fd1980cf9c0 2024-04-20T21:19:59.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.772+0000 7fd19d000640 1 -- 172.21.15.86:0/2337388040 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1981f6fa0 con 0x7fd1980cf9c0 2024-04-20T21:19:59.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.773+0000 7fd19d000640 1 -- 172.21.15.86:0/2337388040 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd19804fcf0 con 0x7fd1980cf9c0 2024-04-20T21:19:59.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.774+0000 7fd18fe00640 1 -- 172.21.15.86:0/2337388040 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd188047020 con 0x7fd1980cf9c0 2024-04-20T21:19:59.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.775+0000 7fd18fe00640 1 --2- 172.21.15.86:0/2337388040 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd17406c8f0 0x7fd17406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:19:59.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.775+0000 7fd18fe00640 1 -- 172.21.15.86:0/2337388040 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4857+0+0 (secure 0 0 0) 0x7fd1880ae320 con 0x7fd1980cf9c0 2024-04-20T21:19:59.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.775+0000 7fd18ea00640 1 --2- 172.21.15.86:0/2337388040 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd17406c8f0 0x7fd17406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:19:59.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.776+0000 7fd18ea00640 1 --2- 172.21.15.86:0/2337388040 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd17406c8f0 0x7fd17406edb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd1980b6a10 tx=0x7fd180008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:19:59.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.779+0000 7fd18fe00640 1 -- 172.21.15.86:0/2337388040 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd18804e030 con 0x7fd1980cf9c0 2024-04-20T21:19:59.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.981+0000 7fd19d000640 1 -- 172.21.15.86:0/2337388040 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd19806e5d0 con 0x7fd1980cf9c0 2024-04-20T21:19:59.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.983+0000 7fd18fe00640 1 -- 172.21.15.86:0/2337388040 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+12891 (secure 0 0 0) 0x7fd18807cb30 con 0x7fd1980cf9c0 2024-04-20T21:19:59.983 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:19:59.983 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":41,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:19:58.858845+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:19:59.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.986+0000 7fd19d000640 1 -- 172.21.15.86:0/2337388040 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd17406c8f0 msgr2=0x7fd17406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:59.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.986+0000 7fd19d000640 1 --2- 172.21.15.86:0/2337388040 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd17406c8f0 0x7fd17406edb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd1980b6a10 tx=0x7fd180008040 comp rx=0 tx=0).stop 2024-04-20T21:19:59.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.986+0000 7fd19d000640 1 -- 172.21.15.86:0/2337388040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1980cf9c0 msgr2=0x7fd198151a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:19:59.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.986+0000 7fd19d000640 1 --2- 172.21.15.86:0/2337388040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1980cf9c0 0x7fd198151a80 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fd188000f80 tx=0x7fd188004a40 comp rx=0 tx=0).stop 2024-04-20T21:19:59.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.986+0000 7fd19d000640 1 -- 172.21.15.86:0/2337388040 shutdown_connections 2024-04-20T21:19:59.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.986+0000 7fd19d000640 1 --2- 172.21.15.86:0/2337388040 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd17406c8f0 0x7fd17406edb0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:59.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.986+0000 7fd19d000640 1 --2- 172.21.15.86:0/2337388040 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd19815ece0 0x7fd1980b9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:59.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.986+0000 7fd19d000640 1 --2- 172.21.15.86:0/2337388040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1980d50c0 0x7fd1981f77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:59.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.986+0000 7fd19d000640 1 --2- 172.21.15.86:0/2337388040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1980cf9c0 0x7fd198151a80 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:19:59.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.986+0000 7fd19d000640 1 -- 172.21.15.86:0/2337388040 >> 172.21.15.86:0/2337388040 conn(0x7fd19808bf70 msgr2=0x7fd1981621b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:19:59.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.987+0000 7fd19d000640 1 -- 172.21.15.86:0/2337388040 shutdown_connections 2024-04-20T21:19:59.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:19:59.987+0000 7fd19d000640 1 -- 172.21.15.86:0/2337388040 wait complete. 2024-04-20T21:20:00.004 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:20:00.005 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:20:00.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.212+0000 7f85f9200640 1 Processor -- start 2024-04-20T21:20:00.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.212+0000 7f85f9200640 1 -- start start 2024-04-20T21:20:00.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.214+0000 7f85f9200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f85f40d4310 0x7f85f40d4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:00.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.214+0000 7f85f9200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85f40cfb80 0x7f85f40d0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:00.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.214+0000 7f85f9200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85f40d4cb0 con 0x7f85f415e870 2024-04-20T21:20:00.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.214+0000 7f85f9200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f85f406d590 con 0x7f85f40d4310 2024-04-20T21:20:00.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.214+0000 7f85f9200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f85f40cf150 con 0x7f85f40cfb80 2024-04-20T21:20:00.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.215+0000 7f85f3400640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f85f415e870 0x7f85f406ce80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.86:53120/0 (socket says 172.21.15.86:53120) 2024-04-20T21:20:00.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.215+0000 7f85f3400640 1 -- 172.21.15.86:0/720365155 learned_addr learned my addr 172.21.15.86:0/720365155 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:20:00.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.215+0000 7f85f3e00640 1 --2- 172.21.15.86:0/720365155 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85f40cfb80 0x7f85f40d0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:00.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.215+0000 7f85f2a00640 1 --2- 172.21.15.86:0/720365155 >> v2:172.21.15.144:3300/0 conn(0x7f85f40d4310 0x7f85f40d4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:00.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.215+0000 7f85f2000640 1 -- 172.21.15.86:0/720365155 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1457194480 0 0) 0x7f85f40d4cb0 con 0x7f85f415e870 2024-04-20T21:20:00.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.215+0000 7f85f2000640 1 -- 172.21.15.86:0/720365155 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f85d4003520 con 0x7f85f415e870 2024-04-20T21:20:00.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.215+0000 7f85f3e00640 1 -- 172.21.15.86:0/720365155 >> v2:172.21.15.144:3300/0 conn(0x7f85f40d4310 msgr2=0x7f85f40d4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:00.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.215+0000 7f85f3e00640 1 --2- 172.21.15.86:0/720365155 >> v2:172.21.15.144:3300/0 conn(0x7f85f40d4310 0x7f85f40d4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:00.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.216+0000 7f85f3e00640 1 -- 172.21.15.86:0/720365155 >> v1:172.21.15.144:6789/0 conn(0x7f85f415e870 legacy=0x7f85f406ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:00.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.216+0000 7f85f3e00640 1 -- 172.21.15.86:0/720365155 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85f40d2ff0 con 0x7f85f40cfb80 2024-04-20T21:20:00.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.217+0000 7f85f3e00640 1 --2- 172.21.15.86:0/720365155 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85f40cfb80 0x7f85f40d0010 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f85ec0097a0 tx=0x7f85ec02b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=936403ecc8edcc3d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:00.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.217+0000 7f85f2000640 1 -- 172.21.15.86:0/720365155 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ec035020 con 0x7f85f40cfb80 2024-04-20T21:20:00.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.217+0000 7f85f2000640 1 -- 172.21.15.86:0/720365155 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85ec031810 con 0x7f85f40cfb80 2024-04-20T21:20:00.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.217+0000 7f85f2000640 1 -- 172.21.15.86:0/720365155 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ec031be0 con 0x7f85f40cfb80 2024-04-20T21:20:00.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.218+0000 7f85f9200640 1 -- 172.21.15.86:0/720365155 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85f40cfb80 msgr2=0x7f85f40d0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:00.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.218+0000 7f85f9200640 1 --2- 172.21.15.86:0/720365155 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85f40cfb80 0x7f85f40d0010 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f85ec0097a0 tx=0x7f85ec02b8b0 comp rx=0 tx=0).stop 2024-04-20T21:20:00.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.218+0000 7f85f9200640 1 -- 172.21.15.86:0/720365155 shutdown_connections 2024-04-20T21:20:00.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.218+0000 7f85f9200640 1 --2- 172.21.15.86:0/720365155 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85f40cfb80 0x7f85f40d0010 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:00.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.218+0000 7f85f9200640 1 --2- 172.21.15.86:0/720365155 >> v2:172.21.15.144:3300/0 conn(0x7f85f40d4310 0x7f85f40d4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:00.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.218+0000 7f85f9200640 1 -- 172.21.15.86:0/720365155 >> 172.21.15.86:0/720365155 conn(0x7f85f409d9c0 msgr2=0x7f85f408bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:00.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.218+0000 7f85f9200640 1 -- 172.21.15.86:0/720365155 shutdown_connections 2024-04-20T21:20:00.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.218+0000 7f85f9200640 1 -- 172.21.15.86:0/720365155 wait complete. 2024-04-20T21:20:00.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.220+0000 7f85f9200640 1 Processor -- start 2024-04-20T21:20:00.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.220+0000 7f85f9200640 1 -- start start 2024-04-20T21:20:00.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.221+0000 7f85f9200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85f40cfb80 0x7f85f415caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:00.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.221+0000 7f85f9200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85f40d4310 0x7f85f40b60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:00.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.221+0000 7f85f9200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85f415e870 0x7f85f40b6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:00.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.221+0000 7f85f9200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f85f40cf380 con 0x7f85f40cfb80 2024-04-20T21:20:00.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.221+0000 7f85f9200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f85f40d35b0 con 0x7f85f40d4310 2024-04-20T21:20:00.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.221+0000 7f85f9200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f85f40cf660 con 0x7f85f415e870 2024-04-20T21:20:00.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.221+0000 7f85f3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85f40cfb80 0x7f85f415caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:00.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.221+0000 7f85f3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85f40cfb80 0x7f85f415caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60642/0 (socket says 172.21.15.86:60642) 2024-04-20T21:20:00.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.221+0000 7f85f3400640 1 -- 172.21.15.86:0/774673935 learned_addr learned my addr 172.21.15.86:0/774673935 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:00.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.222+0000 7f85f2a00640 1 --2- 172.21.15.86:0/774673935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85f40d4310 0x7f85f40b60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:00.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.222+0000 7f85f3e00640 1 --2- 172.21.15.86:0/774673935 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85f415e870 0x7f85f40b6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:00.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.222+0000 7f85f3400640 1 -- 172.21.15.86:0/774673935 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85f415e870 msgr2=0x7f85f40b6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:00.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.222+0000 7f85f3400640 1 --2- 172.21.15.86:0/774673935 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85f415e870 0x7f85f40b6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:00.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.222+0000 7f85f3400640 1 -- 172.21.15.86:0/774673935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85f40d4310 msgr2=0x7f85f40b60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:00.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.222+0000 7f85f3400640 1 --2- 172.21.15.86:0/774673935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85f40d4310 0x7f85f40b60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:00.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.222+0000 7f85f3400640 1 -- 172.21.15.86:0/774673935 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85f40b8620 con 0x7f85f40cfb80 2024-04-20T21:20:00.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.222+0000 7f85f2a00640 1 --2- 172.21.15.86:0/774673935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85f40d4310 0x7f85f40b60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:20:00.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.222+0000 7f85f3400640 1 --2- 172.21.15.86:0/774673935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85f40cfb80 0x7f85f415caf0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f85e400c460 tx=0x7f85e400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:00.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.222+0000 7f85eb400640 1 -- 172.21.15.86:0/774673935 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e4017070 con 0x7f85f40cfb80 2024-04-20T21:20:00.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.222+0000 7f85eb400640 1 -- 172.21.15.86:0/774673935 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85e4005ce0 con 0x7f85f40cfb80 2024-04-20T21:20:00.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.222+0000 7f85f9200640 1 -- 172.21.15.86:0/774673935 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85f40b8900 con 0x7f85f40cfb80 2024-04-20T21:20:00.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.222+0000 7f85eb400640 1 -- 172.21.15.86:0/774673935 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e4007290 con 0x7f85f40cfb80 2024-04-20T21:20:00.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.222+0000 7f85f9200640 1 -- 172.21.15.86:0/774673935 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85f40b8ec0 con 0x7f85f40cfb80 2024-04-20T21:20:00.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.223+0000 7f85f9200640 1 -- 172.21.15.86:0/774673935 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85f404fd90 con 0x7f85f40cfb80 2024-04-20T21:20:00.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.224+0000 7f85eb400640 1 -- 172.21.15.86:0/774673935 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f85e401a070 con 0x7f85f40cfb80 2024-04-20T21:20:00.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.225+0000 7f85eb400640 1 --2- 172.21.15.86:0/774673935 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f85d006c8f0 0x7f85d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:00.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.225+0000 7f85eb400640 1 -- 172.21.15.86:0/774673935 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4857+0+0 (secure 0 0 0) 0x7f85e408ac30 con 0x7f85f40cfb80 2024-04-20T21:20:00.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.226+0000 7f85f2a00640 1 --2- 172.21.15.86:0/774673935 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f85d006c8f0 0x7f85d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:00.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.226+0000 7f85f2a00640 1 --2- 172.21.15.86:0/774673935 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f85d006c8f0 0x7f85d006edb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f85f415e380 tx=0x7f85dc00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:00.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.229+0000 7f85eb400640 1 -- 172.21.15.86:0/774673935 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f85e4091080 con 0x7f85f40cfb80 2024-04-20T21:20:00.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.427+0000 7f85f9200640 1 -- 172.21.15.86:0/774673935 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f85f406cc60 con 0x7f85f40cfb80 2024-04-20T21:20:00.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.428+0000 7f85eb400640 1 -- 172.21.15.86:0/774673935 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) ==== 93+0+10 (secure 0 0 0) 0x7f85e4059390 con 0x7f85f40cfb80 2024-04-20T21:20:00.427 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:20:00.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.430+0000 7f85f9200640 1 -- 172.21.15.86:0/774673935 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f85d006c8f0 msgr2=0x7f85d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:00.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.430+0000 7f85f9200640 1 --2- 172.21.15.86:0/774673935 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f85d006c8f0 0x7f85d006edb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f85f415e380 tx=0x7f85dc00b040 comp rx=0 tx=0).stop 2024-04-20T21:20:00.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.430+0000 7f85f9200640 1 -- 172.21.15.86:0/774673935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85f40cfb80 msgr2=0x7f85f415caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:00.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.430+0000 7f85f9200640 1 --2- 172.21.15.86:0/774673935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85f40cfb80 0x7f85f415caf0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f85e400c460 tx=0x7f85e400c920 comp rx=0 tx=0).stop 2024-04-20T21:20:00.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.430+0000 7f85f9200640 1 -- 172.21.15.86:0/774673935 shutdown_connections 2024-04-20T21:20:00.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.430+0000 7f85f9200640 1 --2- 172.21.15.86:0/774673935 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f85d006c8f0 0x7f85d006edb0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:00.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.430+0000 7f85f9200640 1 --2- 172.21.15.86:0/774673935 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f85f415e870 0x7f85f40b6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:00.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.430+0000 7f85f9200640 1 --2- 172.21.15.86:0/774673935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f85f40d4310 0x7f85f40b60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:00.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.430+0000 7f85f9200640 1 --2- 172.21.15.86:0/774673935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85f40cfb80 0x7f85f415caf0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:00.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.430+0000 7f85f9200640 1 -- 172.21.15.86:0/774673935 >> 172.21.15.86:0/774673935 conn(0x7f85f409d9c0 msgr2=0x7f85f415dad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:00.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.431+0000 7f85f9200640 1 -- 172.21.15.86:0/774673935 shutdown_connections 2024-04-20T21:20:00.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.431+0000 7f85f9200640 1 -- 172.21.15.86:0/774673935 wait complete. 2024-04-20T21:20:00.448 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-20T21:20:00.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.649+0000 7efe4e600640 1 Processor -- start 2024-04-20T21:20:00.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.649+0000 7efe4e600640 1 -- start start 2024-04-20T21:20:00.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.650+0000 7efe4e600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7efe4815d6a0 0x7efe480ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:00.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.650+0000 7efe4e600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe480ad620 0x7efe4806e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:00.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.650+0000 7efe4e600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe48160890 con 0x7efe48156ae0 2024-04-20T21:20:00.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.650+0000 7efe4e600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7efe480ada90 con 0x7efe4815d6a0 2024-04-20T21:20:00.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.650+0000 7efe4e600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7efe4806db20 con 0x7efe480ad620 2024-04-20T21:20:00.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.651+0000 7efe47e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7efe4815d6a0 0x7efe480ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:00.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.651+0000 7efe4dc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe480ad620 0x7efe4806e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:00.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.651+0000 7efe47e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7efe4815d6a0 0x7efe480ad0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.86:56182/0 (socket says 172.21.15.86:56182) 2024-04-20T21:20:00.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.651+0000 7efe47e00640 1 -- 172.21.15.86:0/2001412690 learned_addr learned my addr 172.21.15.86:0/2001412690 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:00.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.651+0000 7efe4dc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe480ad620 0x7efe4806e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:51640/0 (socket says 172.21.15.86:51640) 2024-04-20T21:20:00.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.652+0000 7efe47e00640 1 -- 172.21.15.86:0/2001412690 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe480ad620 msgr2=0x7efe4806e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:00.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.652+0000 7efe47e00640 1 --2- 172.21.15.86:0/2001412690 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe480ad620 0x7efe4806e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:00.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.652+0000 7efe47e00640 1 -- 172.21.15.86:0/2001412690 >> v1:172.21.15.144:6789/0 conn(0x7efe48156ae0 legacy=0x7efe48160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:00.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.652+0000 7efe47e00640 1 -- 172.21.15.86:0/2001412690 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe48160b70 con 0x7efe4815d6a0 2024-04-20T21:20:00.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.652+0000 7efe47400640 1 -- 172.21.15.86:0/2001412690 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1754223186 0 0) 0x7efe48160890 con 0x7efe48156ae0 2024-04-20T21:20:00.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.652+0000 7efe4dc00640 1 --2- 172.21.15.86:0/2001412690 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe480ad620 0x7efe4806e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:20:00.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.654+0000 7efe47e00640 1 --2- 172.21.15.86:0/2001412690 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe4815d6a0 0x7efe480ad0e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efe34009870 tx=0x7efe3402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdbe42ed3542b791 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:00.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.655+0000 7efe47400640 1 -- 172.21.15.86:0/2001412690 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe34034070 con 0x7efe4815d6a0 2024-04-20T21:20:00.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.655+0000 7efe47400640 1 -- 172.21.15.86:0/2001412690 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe34030840 con 0x7efe4815d6a0 2024-04-20T21:20:00.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.655+0000 7efe47400640 1 -- 172.21.15.86:0/2001412690 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe34030c10 con 0x7efe4815d6a0 2024-04-20T21:20:00.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.656+0000 7efe4e600640 1 -- 172.21.15.86:0/2001412690 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe4815d6a0 msgr2=0x7efe480ad0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:00.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.656+0000 7efe4e600640 1 --2- 172.21.15.86:0/2001412690 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe4815d6a0 0x7efe480ad0e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efe34009870 tx=0x7efe3402b980 comp rx=0 tx=0).stop 2024-04-20T21:20:00.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.656+0000 7efe4e600640 1 -- 172.21.15.86:0/2001412690 shutdown_connections 2024-04-20T21:20:00.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.656+0000 7efe4e600640 1 --2- 172.21.15.86:0/2001412690 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe480ad620 0x7efe4806e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:00.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.656+0000 7efe4e600640 1 --2- 172.21.15.86:0/2001412690 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe4815d6a0 0x7efe480ad0e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:00.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.656+0000 7efe4e600640 1 -- 172.21.15.86:0/2001412690 >> 172.21.15.86:0/2001412690 conn(0x7efe480d0170 msgr2=0x7efe480d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:00.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.656+0000 7efe4e600640 1 -- 172.21.15.86:0/2001412690 shutdown_connections 2024-04-20T21:20:00.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.656+0000 7efe4e600640 1 -- 172.21.15.86:0/2001412690 wait complete. 2024-04-20T21:20:00.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.658+0000 7efe4e600640 1 Processor -- start 2024-04-20T21:20:00.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.658+0000 7efe4e600640 1 -- start start 2024-04-20T21:20:00.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.659+0000 7efe4e600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe480ad620 0x7efe48151c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:00.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.660+0000 7efe4e600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe48156ae0 0x7efe4815a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:00.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.660+0000 7efe4e600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe4815d6a0 0x7efe4815abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:00.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.660+0000 7efe4e600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efe48156ec0 con 0x7efe480ad620 2024-04-20T21:20:00.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.660+0000 7efe4e600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efe4806dd70 con 0x7efe4815d6a0 2024-04-20T21:20:00.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.660+0000 7efe4e600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7efe4806e7d0 con 0x7efe48156ae0 2024-04-20T21:20:00.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.660+0000 7efe4d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe480ad620 0x7efe48151c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:00.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.660+0000 7efe4d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe480ad620 0x7efe48151c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60654/0 (socket says 172.21.15.86:60654) 2024-04-20T21:20:00.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.660+0000 7efe4d200640 1 -- 172.21.15.86:0/393936339 learned_addr learned my addr 172.21.15.86:0/393936339 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:00.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.660+0000 7efe4dc00640 1 --2- 172.21.15.86:0/393936339 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe4815d6a0 0x7efe4815abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:00.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.660+0000 7efe47e00640 1 --2- 172.21.15.86:0/393936339 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe48156ae0 0x7efe4815a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:00.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.660+0000 7efe4d200640 1 -- 172.21.15.86:0/393936339 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe48156ae0 msgr2=0x7efe4815a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:00.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.660+0000 7efe4d200640 1 --2- 172.21.15.86:0/393936339 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe48156ae0 0x7efe4815a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:00.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.660+0000 7efe4d200640 1 -- 172.21.15.86:0/393936339 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe4815d6a0 msgr2=0x7efe4815abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:00.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.660+0000 7efe4d200640 1 --2- 172.21.15.86:0/393936339 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe4815d6a0 0x7efe4815abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:00.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.660+0000 7efe4d200640 1 -- 172.21.15.86:0/393936339 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe48153f10 con 0x7efe480ad620 2024-04-20T21:20:00.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.661+0000 7efe4d200640 1 --2- 172.21.15.86:0/393936339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe480ad620 0x7efe48151c40 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7efe3c00c460 tx=0x7efe3c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:00.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.661+0000 7efe44c00640 1 -- 172.21.15.86:0/393936339 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3c005480 con 0x7efe480ad620 2024-04-20T21:20:00.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.661+0000 7efe44c00640 1 -- 172.21.15.86:0/393936339 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe3c0035f0 con 0x7efe480ad620 2024-04-20T21:20:00.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.661+0000 7efe4e600640 1 -- 172.21.15.86:0/393936339 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe481541f0 con 0x7efe480ad620 2024-04-20T21:20:00.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.661+0000 7efe44c00640 1 -- 172.21.15.86:0/393936339 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3c003a00 con 0x7efe480ad620 2024-04-20T21:20:00.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.661+0000 7efe4e600640 1 -- 172.21.15.86:0/393936339 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe481547b0 con 0x7efe480ad620 2024-04-20T21:20:00.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.662+0000 7efe4e600640 1 -- 172.21.15.86:0/393936339 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe4804fcf0 con 0x7efe480ad620 2024-04-20T21:20:00.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.663+0000 7efe44c00640 1 -- 172.21.15.86:0/393936339 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efe3c003ba0 con 0x7efe480ad620 2024-04-20T21:20:00.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.664+0000 7efe44c00640 1 --2- 172.21.15.86:0/393936339 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efe2806ca90 0x7efe2806ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:00.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.664+0000 7efe44c00640 1 -- 172.21.15.86:0/393936339 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4857+0+0 (secure 0 0 0) 0x7efe3c08b4e0 con 0x7efe480ad620 2024-04-20T21:20:00.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.664+0000 7efe47e00640 1 --2- 172.21.15.86:0/393936339 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efe2806ca90 0x7efe2806ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:00.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.665+0000 7efe47e00640 1 --2- 172.21.15.86:0/393936339 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efe2806ca90 0x7efe2806ef50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7efe34036000 tx=0x7efe3403b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:00.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.668+0000 7efe44c00640 1 -- 172.21.15.86:0/393936339 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7efe3c059c50 con 0x7efe480ad620 2024-04-20T21:20:00.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.856+0000 7efe4e600640 1 -- 172.21.15.86:0/393936339 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7efe480ae570 con 0x7efe480ad620 2024-04-20T21:20:00.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.857+0000 7efe44c00640 1 -- 172.21.15.86:0/393936339 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v41) ==== 84+0+65 (secure 0 0 0) 0x7efe3c059550 con 0x7efe480ad620 2024-04-20T21:20:00.856 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:20:00.856 INFO:teuthology.orchestra.run.smithi086.stdout:[ 2024-04-20T21:20:00.856 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:20:00.856 INFO:teuthology.orchestra.run.smithi086.stdout: "poolnum": 1, 2024-04-20T21:20:00.856 INFO:teuthology.orchestra.run.smithi086.stdout: "poolname": ".mgr" 2024-04-20T21:20:00.856 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:20:00.856 INFO:teuthology.orchestra.run.smithi086.stdout:] 2024-04-20T21:20:00.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.859+0000 7efe4e600640 1 -- 172.21.15.86:0/393936339 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efe2806ca90 msgr2=0x7efe2806ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:00.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.859+0000 7efe4e600640 1 --2- 172.21.15.86:0/393936339 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efe2806ca90 0x7efe2806ef50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7efe34036000 tx=0x7efe3403b040 comp rx=0 tx=0).stop 2024-04-20T21:20:00.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.859+0000 7efe4e600640 1 -- 172.21.15.86:0/393936339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe480ad620 msgr2=0x7efe48151c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:00.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.859+0000 7efe4e600640 1 --2- 172.21.15.86:0/393936339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe480ad620 0x7efe48151c40 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7efe3c00c460 tx=0x7efe3c00c920 comp rx=0 tx=0).stop 2024-04-20T21:20:00.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.859+0000 7efe4e600640 1 -- 172.21.15.86:0/393936339 shutdown_connections 2024-04-20T21:20:00.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.859+0000 7efe4e600640 1 --2- 172.21.15.86:0/393936339 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efe2806ca90 0x7efe2806ef50 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:00.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.859+0000 7efe4e600640 1 --2- 172.21.15.86:0/393936339 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe4815d6a0 0x7efe4815abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:00.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.859+0000 7efe4e600640 1 --2- 172.21.15.86:0/393936339 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe48156ae0 0x7efe4815a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:00.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.859+0000 7efe4e600640 1 --2- 172.21.15.86:0/393936339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe480ad620 0x7efe48151c40 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:00.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.859+0000 7efe4e600640 1 -- 172.21.15.86:0/393936339 >> 172.21.15.86:0/393936339 conn(0x7efe480d0170 msgr2=0x7efe480b13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:00.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.859+0000 7efe4e600640 1 -- 172.21.15.86:0/393936339 shutdown_connections 2024-04-20T21:20:00.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:00.859+0000 7efe4e600640 1 -- 172.21.15.86:0/393936339 wait complete. 2024-04-20T21:20:00.877 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-20T21:20:00.878 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-20T21:20:01.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.069+0000 7f1573600640 1 Processor -- start 2024-04-20T21:20:01.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.069+0000 7f1573600640 1 -- start start 2024-04-20T21:20:01.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.071+0000 7f1573600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f156c162bb0 0x7f156c167340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:01.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.071+0000 7f1573600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f156c072410 0x7f156c06dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:01.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.071+0000 7f1573600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f156c06f810 con 0x7f156c06f070 2024-04-20T21:20:01.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.071+0000 7f1573600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f156c0b4840 con 0x7f156c162bb0 2024-04-20T21:20:01.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.071+0000 7f1573600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f156c0b4b20 con 0x7f156c072410 2024-04-20T21:20:01.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.071+0000 7f1570c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f156c162bb0 0x7f156c167340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:01.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.071+0000 7f1571600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f156c072410 0x7f156c06dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:01.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.072+0000 7f1570c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f156c162bb0 0x7f156c167340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.86:56202/0 (socket says 172.21.15.86:56202) 2024-04-20T21:20:01.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.072+0000 7f1570c00640 1 -- 172.21.15.86:0/1155062941 learned_addr learned my addr 172.21.15.86:0/1155062941 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:01.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.072+0000 7f1571600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f156c072410 0x7f156c06dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:51658/0 (socket says 172.21.15.86:51658) 2024-04-20T21:20:01.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.072+0000 7f1571600640 1 -- 172.21.15.86:0/1155062941 >> v2:172.21.15.144:3300/0 conn(0x7f156c162bb0 msgr2=0x7f156c167340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:01.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.072+0000 7f1571600640 1 --2- 172.21.15.86:0/1155062941 >> v2:172.21.15.144:3300/0 conn(0x7f156c162bb0 0x7f156c167340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:01.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.072+0000 7f1571600640 1 -- 172.21.15.86:0/1155062941 >> v1:172.21.15.144:6789/0 conn(0x7f156c06f070 legacy=0x7f156c071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:01.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.072+0000 7f1571600640 1 -- 172.21.15.86:0/1155062941 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f156c0b4e00 con 0x7f156c072410 2024-04-20T21:20:01.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.074+0000 7f1571600640 1 --2- 172.21.15.86:0/1155062941 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f156c072410 0x7f156c06dc30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1564009d90 tx=0x7f156402beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c469c37a659d4939 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:01.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.074+0000 7f156b400640 1 -- 172.21.15.86:0/1155062941 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1564039070 con 0x7f156c072410 2024-04-20T21:20:01.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.074+0000 7f156b400640 1 -- 172.21.15.86:0/1155062941 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f156402cc60 con 0x7f156c072410 2024-04-20T21:20:01.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.075+0000 7f156b400640 1 -- 172.21.15.86:0/1155062941 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1564034660 con 0x7f156c072410 2024-04-20T21:20:01.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.075+0000 7f1573600640 1 -- 172.21.15.86:0/1155062941 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f156c072410 msgr2=0x7f156c06dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:01.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.075+0000 7f1573600640 1 --2- 172.21.15.86:0/1155062941 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f156c072410 0x7f156c06dc30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1564009d90 tx=0x7f156402beb0 comp rx=0 tx=0).stop 2024-04-20T21:20:01.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.075+0000 7f1573600640 1 -- 172.21.15.86:0/1155062941 shutdown_connections 2024-04-20T21:20:01.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.075+0000 7f1573600640 1 --2- 172.21.15.86:0/1155062941 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f156c072410 0x7f156c06dc30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:01.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.075+0000 7f1573600640 1 --2- 172.21.15.86:0/1155062941 >> v2:172.21.15.144:3300/0 conn(0x7f156c162bb0 0x7f156c167340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:01.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.075+0000 7f1573600640 1 -- 172.21.15.86:0/1155062941 >> 172.21.15.86:0/1155062941 conn(0x7f156c0b2180 msgr2=0x7f156c0b2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:01.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.075+0000 7f1573600640 1 -- 172.21.15.86:0/1155062941 shutdown_connections 2024-04-20T21:20:01.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.076+0000 7f1573600640 1 -- 172.21.15.86:0/1155062941 wait complete. 2024-04-20T21:20:01.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.078+0000 7f1573600640 1 Processor -- start 2024-04-20T21:20:01.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.078+0000 7f1573600640 1 -- start start 2024-04-20T21:20:01.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.079+0000 7f1573600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f156c06f070 0x7f156c150640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:01.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.079+0000 7f1573600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f156c072410 0x7f156c0ac270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:01.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.079+0000 7f1573600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f156c162bb0 0x7f156c0ac7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:01.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.079+0000 7f1573600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f156c0b5600 con 0x7f156c072410 2024-04-20T21:20:01.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.079+0000 7f1573600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f156c06f500 con 0x7f156c162bb0 2024-04-20T21:20:01.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.079+0000 7f1573600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f156c162f90 con 0x7f156c06f070 2024-04-20T21:20:01.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.079+0000 7f156be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f156c072410 0x7f156c0ac270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:01.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.080+0000 7f156be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f156c072410 0x7f156c0ac270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60658/0 (socket says 172.21.15.86:60658) 2024-04-20T21:20:01.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.080+0000 7f156be00640 1 -- 172.21.15.86:0/729111687 learned_addr learned my addr 172.21.15.86:0/729111687 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:01.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.080+0000 7f1570c00640 1 --2- 172.21.15.86:0/729111687 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f156c06f070 0x7f156c150640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:01.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.080+0000 7f1571600640 1 --2- 172.21.15.86:0/729111687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f156c162bb0 0x7f156c0ac7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:01.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.080+0000 7f156be00640 1 -- 172.21.15.86:0/729111687 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f156c06f070 msgr2=0x7f156c150640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:01.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.080+0000 7f156be00640 1 --2- 172.21.15.86:0/729111687 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f156c06f070 0x7f156c150640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:01.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.080+0000 7f156be00640 1 -- 172.21.15.86:0/729111687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f156c162bb0 msgr2=0x7f156c0ac7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:01.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.080+0000 7f156be00640 1 --2- 172.21.15.86:0/729111687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f156c162bb0 0x7f156c0ac7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:01.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.080+0000 7f156be00640 1 -- 172.21.15.86:0/729111687 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f156c160d30 con 0x7f156c072410 2024-04-20T21:20:01.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.080+0000 7f1570c00640 1 --2- 172.21.15.86:0/729111687 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f156c06f070 0x7f156c150640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:20:01.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.080+0000 7f156be00640 1 --2- 172.21.15.86:0/729111687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f156c072410 0x7f156c0ac270 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f155800c3c0 tx=0x7f155800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:01.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.081+0000 7f1568c00640 1 -- 172.21.15.86:0/729111687 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15580052d0 con 0x7f156c072410 2024-04-20T21:20:01.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.081+0000 7f1568c00640 1 -- 172.21.15.86:0/729111687 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15580054b0 con 0x7f156c072410 2024-04-20T21:20:01.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.081+0000 7f1568c00640 1 -- 172.21.15.86:0/729111687 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1558006a40 con 0x7f156c072410 2024-04-20T21:20:01.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.081+0000 7f1573600640 1 -- 172.21.15.86:0/729111687 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f156c161010 con 0x7f156c072410 2024-04-20T21:20:01.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.081+0000 7f1573600640 1 -- 172.21.15.86:0/729111687 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f156c1615d0 con 0x7f156c072410 2024-04-20T21:20:01.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.082+0000 7f1573600640 1 -- 172.21.15.86:0/729111687 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f156c0689e0 con 0x7f156c072410 2024-04-20T21:20:01.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.083+0000 7f1568c00640 1 -- 172.21.15.86:0/729111687 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1558006be0 con 0x7f156c072410 2024-04-20T21:20:01.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.084+0000 7f1568c00640 1 --2- 172.21.15.86:0/729111687 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f154c06c8f0 0x7f154c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:01.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.084+0000 7f1568c00640 1 -- 172.21.15.86:0/729111687 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4857+0+0 (secure 0 0 0) 0x7f155808acf0 con 0x7f156c072410 2024-04-20T21:20:01.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.084+0000 7f1570c00640 1 --2- 172.21.15.86:0/729111687 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f154c06c8f0 0x7f154c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:01.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.085+0000 7f1570c00640 1 --2- 172.21.15.86:0/729111687 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f154c06c8f0 0x7f154c06edb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1560009c20 tx=0x7f1560008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:01.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.088+0000 7f1568c00640 1 -- 172.21.15.86:0/729111687 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f15580593d0 con 0x7f156c072410 2024-04-20T21:20:01.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.290+0000 7f1573600640 1 -- 172.21.15.86:0/729111687 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f156c0b40a0 con 0x7f156c072410 2024-04-20T21:20:01.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.890+0000 7f1568c00640 1 -- 172.21.15.86:0/729111687 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v42) ==== 138+0+0 (secure 0 0 0) 0x7f1558058cd0 con 0x7f156c072410 2024-04-20T21:20:01.890 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' created 2024-04-20T21:20:01.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.894+0000 7f1573600640 1 -- 172.21.15.86:0/729111687 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f154c06c8f0 msgr2=0x7f154c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:01.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.894+0000 7f1573600640 1 --2- 172.21.15.86:0/729111687 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f154c06c8f0 0x7f154c06edb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1560009c20 tx=0x7f1560008040 comp rx=0 tx=0).stop 2024-04-20T21:20:01.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.894+0000 7f1573600640 1 -- 172.21.15.86:0/729111687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f156c072410 msgr2=0x7f156c0ac270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:01.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.894+0000 7f1573600640 1 --2- 172.21.15.86:0/729111687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f156c072410 0x7f156c0ac270 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f155800c3c0 tx=0x7f155800c880 comp rx=0 tx=0).stop 2024-04-20T21:20:01.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.894+0000 7f1573600640 1 -- 172.21.15.86:0/729111687 shutdown_connections 2024-04-20T21:20:01.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.894+0000 7f1573600640 1 --2- 172.21.15.86:0/729111687 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f154c06c8f0 0x7f154c06edb0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:01.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.894+0000 7f1573600640 1 --2- 172.21.15.86:0/729111687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f156c162bb0 0x7f156c0ac7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:01.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.894+0000 7f1573600640 1 --2- 172.21.15.86:0/729111687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f156c072410 0x7f156c0ac270 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:01.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.894+0000 7f1573600640 1 --2- 172.21.15.86:0/729111687 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f156c06f070 0x7f156c150640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:01.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.894+0000 7f1573600640 1 -- 172.21.15.86:0/729111687 >> 172.21.15.86:0/729111687 conn(0x7f156c0b2180 msgr2=0x7f156c0b31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:01.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.895+0000 7f1573600640 1 -- 172.21.15.86:0/729111687 shutdown_connections 2024-04-20T21:20:01.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:01.895+0000 7f1573600640 1 -- 172.21.15.86:0/729111687 wait complete. 2024-04-20T21:20:01.923 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-20T21:20:02.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.122+0000 7f2ab4000640 1 Processor -- start 2024-04-20T21:20:02.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.122+0000 7f2ab4000640 1 -- start start 2024-04-20T21:20:02.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.124+0000 7f2ab4000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2aac146210 0x7f2aac06e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:02.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.124+0000 7f2ab4000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2aac06bfd0 0x7f2aac0cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:02.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.124+0000 7f2ab4000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2aac15cd40 con 0x7f2aac06bbf0 2024-04-20T21:20:02.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.124+0000 7f2ab4000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f2aac15d000 con 0x7f2aac146210 2024-04-20T21:20:02.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.124+0000 7f2ab4000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2aac15d2e0 con 0x7f2aac06bfd0 2024-04-20T21:20:02.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.125+0000 7f2ab1600640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f2aac06bbf0 0x7f2aac145b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41578/0 (socket says 172.21.15.86:41578) 2024-04-20T21:20:02.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.125+0000 7f2ab1600640 1 -- 172.21.15.86:0/2793544123 learned_addr learned my addr 172.21.15.86:0/2793544123 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:20:02.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.125+0000 7f2ab0c00640 1 --2- 172.21.15.86:0/2793544123 >> v2:172.21.15.86:3300/0 conn(0x7f2aac146210 0x7f2aac06e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:02.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.125+0000 7f2ab2000640 1 --2- 172.21.15.86:0/2793544123 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2aac06bfd0 0x7f2aac0cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:02.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.125+0000 7f2aabe00640 1 -- 172.21.15.86:0/2793544123 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1380632784 0 0) 0x7f2aac15cd40 con 0x7f2aac06bbf0 2024-04-20T21:20:02.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.125+0000 7f2aabe00640 1 -- 172.21.15.86:0/2793544123 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a94003520 con 0x7f2aac06bbf0 2024-04-20T21:20:02.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.125+0000 7f2ab0c00640 1 -- 172.21.15.86:0/2793544123 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2aac06bfd0 msgr2=0x7f2aac0cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:02.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.125+0000 7f2ab0c00640 1 --2- 172.21.15.86:0/2793544123 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2aac06bfd0 0x7f2aac0cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:02.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.125+0000 7f2ab0c00640 1 -- 172.21.15.86:0/2793544123 >> v1:172.21.15.86:6789/0 conn(0x7f2aac06bbf0 legacy=0x7f2aac145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:02.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.125+0000 7f2ab0c00640 1 -- 172.21.15.86:0/2793544123 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aac15d5c0 con 0x7f2aac146210 2024-04-20T21:20:02.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.127+0000 7f2ab0c00640 1 --2- 172.21.15.86:0/2793544123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2aac146210 0x7f2aac06e000 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f2a980097a0 tx=0x7f2a9802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad78cef176d42570 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:02.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.127+0000 7f2aabe00640 1 -- 172.21.15.86:0/2793544123 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a98035020 con 0x7f2aac146210 2024-04-20T21:20:02.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.127+0000 7f2aabe00640 1 -- 172.21.15.86:0/2793544123 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a98031810 con 0x7f2aac146210 2024-04-20T21:20:02.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.127+0000 7f2aabe00640 1 -- 172.21.15.86:0/2793544123 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a98031be0 con 0x7f2aac146210 2024-04-20T21:20:02.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.127+0000 7f2ab4000640 1 -- 172.21.15.86:0/2793544123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2aac146210 msgr2=0x7f2aac06e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:02.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.127+0000 7f2ab4000640 1 --2- 172.21.15.86:0/2793544123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2aac146210 0x7f2aac06e000 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f2a980097a0 tx=0x7f2a9802b8b0 comp rx=0 tx=0).stop 2024-04-20T21:20:02.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.128+0000 7f2ab4000640 1 -- 172.21.15.86:0/2793544123 shutdown_connections 2024-04-20T21:20:02.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.128+0000 7f2ab4000640 1 --2- 172.21.15.86:0/2793544123 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2aac06bfd0 0x7f2aac0cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:02.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.128+0000 7f2ab4000640 1 --2- 172.21.15.86:0/2793544123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2aac146210 0x7f2aac06e000 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:02.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.128+0000 7f2ab4000640 1 -- 172.21.15.86:0/2793544123 >> 172.21.15.86:0/2793544123 conn(0x7f2aac0b2110 msgr2=0x7f2aac0b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:02.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.128+0000 7f2ab4000640 1 -- 172.21.15.86:0/2793544123 shutdown_connections 2024-04-20T21:20:02.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.128+0000 7f2ab4000640 1 -- 172.21.15.86:0/2793544123 wait complete. 2024-04-20T21:20:02.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.129+0000 7f2ab4000640 1 Processor -- start 2024-04-20T21:20:02.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.130+0000 7f2ab4000640 1 -- start start 2024-04-20T21:20:02.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.130+0000 7f2ab4000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2aac06bbf0 0x7f2aac0721d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:02.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.130+0000 7f2ab4000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2aac06bfd0 0x7f2aac1e4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:02.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.130+0000 7f2ab4000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2aac146210 0x7f2aac1e53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:02.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.130+0000 7f2ab4000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2aac0cdc00 con 0x7f2aac06bbf0 2024-04-20T21:20:02.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.130+0000 7f2ab4000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2aac0cda80 con 0x7f2aac06bfd0 2024-04-20T21:20:02.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.130+0000 7f2ab4000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2aac0cdd80 con 0x7f2aac146210 2024-04-20T21:20:02.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.131+0000 7f2ab1600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2aac06bbf0 0x7f2aac0721d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:02.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.131+0000 7f2ab1600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2aac06bbf0 0x7f2aac0721d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60674/0 (socket says 172.21.15.86:60674) 2024-04-20T21:20:02.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.131+0000 7f2ab1600640 1 -- 172.21.15.86:0/3309207306 learned_addr learned my addr 172.21.15.86:0/3309207306 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:02.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.131+0000 7f2ab0c00640 1 --2- 172.21.15.86:0/3309207306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2aac06bfd0 0x7f2aac1e4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:02.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.131+0000 7f2ab2000640 1 --2- 172.21.15.86:0/3309207306 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2aac146210 0x7f2aac1e53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:02.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.131+0000 7f2ab1600640 1 -- 172.21.15.86:0/3309207306 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2aac146210 msgr2=0x7f2aac1e53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:02.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.131+0000 7f2ab1600640 1 --2- 172.21.15.86:0/3309207306 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2aac146210 0x7f2aac1e53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:02.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.131+0000 7f2ab1600640 1 -- 172.21.15.86:0/3309207306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2aac06bfd0 msgr2=0x7f2aac1e4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:02.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.131+0000 7f2ab1600640 1 --2- 172.21.15.86:0/3309207306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2aac06bfd0 0x7f2aac1e4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:02.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.131+0000 7f2ab1600640 1 -- 172.21.15.86:0/3309207306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aac1e5a70 con 0x7f2aac06bbf0 2024-04-20T21:20:02.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.131+0000 7f2ab0c00640 1 --2- 172.21.15.86:0/3309207306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2aac06bfd0 0x7f2aac1e4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:20:02.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.132+0000 7f2ab1600640 1 --2- 172.21.15.86:0/3309207306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2aac06bbf0 0x7f2aac0721d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f2aa000cbb0 tx=0x7f2aa000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:02.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.132+0000 7f2aa9600640 1 -- 172.21.15.86:0/3309207306 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa0017070 con 0x7f2aac06bbf0 2024-04-20T21:20:02.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.132+0000 7f2aa9600640 1 -- 172.21.15.86:0/3309207306 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2aa000ad00 con 0x7f2aac06bbf0 2024-04-20T21:20:02.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.132+0000 7f2aa9600640 1 -- 172.21.15.86:0/3309207306 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa0012640 con 0x7f2aac06bbf0 2024-04-20T21:20:02.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.132+0000 7f2ab4000640 1 -- 172.21.15.86:0/3309207306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2aac131960 con 0x7f2aac06bbf0 2024-04-20T21:20:02.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.132+0000 7f2ab4000640 1 -- 172.21.15.86:0/3309207306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2aac1e5d50 con 0x7f2aac06bbf0 2024-04-20T21:20:02.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.133+0000 7f2ab4000640 1 -- 172.21.15.86:0/3309207306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2aac0689e0 con 0x7f2aac06bbf0 2024-04-20T21:20:02.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.136+0000 7f2aa9600640 1 -- 172.21.15.86:0/3309207306 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2aa001a070 con 0x7f2aac06bbf0 2024-04-20T21:20:02.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.137+0000 7f2aa9600640 1 --2- 172.21.15.86:0/3309207306 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2a8c06c8f0 0x7f2a8c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:02.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.137+0000 7f2aa9600640 1 -- 172.21.15.86:0/3309207306 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5227+0+0 (secure 0 0 0) 0x7f2aa008bc10 con 0x7f2aac06bbf0 2024-04-20T21:20:02.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.137+0000 7f2ab0c00640 1 --2- 172.21.15.86:0/3309207306 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2a8c06c8f0 0x7f2a8c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:02.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.138+0000 7f2ab0c00640 1 --2- 172.21.15.86:0/3309207306 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2a8c06c8f0 0x7f2a8c06edb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2a98036000 tx=0x7f2a98008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:02.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.141+0000 7f2aa9600640 1 -- 172.21.15.86:0/3309207306 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2aa0021020 con 0x7f2aac06bbf0 2024-04-20T21:20:02.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.343+0000 7f2ab4000640 1 -- 172.21.15.86:0/3309207306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f2aac06c780 con 0x7f2aac06bbf0 2024-04-20T21:20:02.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.905+0000 7f2aa9600640 1 -- 172.21.15.86:0/3309207306 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v43) ==== 170+0+0 (secure 0 0 0) 0x7f2aa005a290 con 0x7f2aac06bbf0 2024-04-20T21:20:02.904 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' created 2024-04-20T21:20:02.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.908+0000 7f2ab4000640 1 -- 172.21.15.86:0/3309207306 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2a8c06c8f0 msgr2=0x7f2a8c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:02.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.908+0000 7f2ab4000640 1 --2- 172.21.15.86:0/3309207306 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2a8c06c8f0 0x7f2a8c06edb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2a98036000 tx=0x7f2a98008030 comp rx=0 tx=0).stop 2024-04-20T21:20:02.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.908+0000 7f2ab4000640 1 -- 172.21.15.86:0/3309207306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2aac06bbf0 msgr2=0x7f2aac0721d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:02.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.908+0000 7f2ab4000640 1 --2- 172.21.15.86:0/3309207306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2aac06bbf0 0x7f2aac0721d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f2aa000cbb0 tx=0x7f2aa000a430 comp rx=0 tx=0).stop 2024-04-20T21:20:02.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.908+0000 7f2ab4000640 1 -- 172.21.15.86:0/3309207306 shutdown_connections 2024-04-20T21:20:02.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.908+0000 7f2ab4000640 1 --2- 172.21.15.86:0/3309207306 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2a8c06c8f0 0x7f2a8c06edb0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:02.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.908+0000 7f2ab4000640 1 --2- 172.21.15.86:0/3309207306 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2aac146210 0x7f2aac1e53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:02.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.908+0000 7f2ab4000640 1 --2- 172.21.15.86:0/3309207306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2aac06bfd0 0x7f2aac1e4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:02.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.908+0000 7f2ab4000640 1 --2- 172.21.15.86:0/3309207306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2aac06bbf0 0x7f2aac0721d0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:02.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.908+0000 7f2ab4000640 1 -- 172.21.15.86:0/3309207306 >> 172.21.15.86:0/3309207306 conn(0x7f2aac0b2110 msgr2=0x7f2aac15dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:02.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.908+0000 7f2ab4000640 1 -- 172.21.15.86:0/3309207306 shutdown_connections 2024-04-20T21:20:02.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:02.908+0000 7f2ab4000640 1 -- 172.21.15.86:0/3309207306 wait complete. 2024-04-20T21:20:02.943 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-20T21:20:03.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.115+0000 7fb7ac000640 1 Processor -- start 2024-04-20T21:20:03.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.115+0000 7fb7ac000640 1 -- start start 2024-04-20T21:20:03.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.117+0000 7fb7ac000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb7a40aee50 0x7fb7a406b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:03.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.117+0000 7fb7ac000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb7a40b4530 0x7fb7a406bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:03.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.117+0000 7fb7ac000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7a40adb10 con 0x7fb7a415eb30 2024-04-20T21:20:03.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.117+0000 7fb7ac000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb7a40af7d0 con 0x7fb7a40aee50 2024-04-20T21:20:03.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.117+0000 7fb7ac000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fb7a40addb0 con 0x7fb7a40b4530 2024-04-20T21:20:03.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.117+0000 7fb7a8c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb7a40aee50 0x7fb7a406b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:03.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.117+0000 7fb7a8c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb7a40aee50 0x7fb7a406b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60686/0 (socket says 172.21.15.86:60686) 2024-04-20T21:20:03.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.117+0000 7fb7a8c00640 1 -- 172.21.15.86:0/3089577309 learned_addr learned my addr 172.21.15.86:0/3089577309 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:03.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.117+0000 7fb7a8c00640 1 -- 172.21.15.86:0/3089577309 >> v2:172.21.15.144:3300/0 conn(0x7fb7a40b4530 msgr2=0x7fb7a406bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:03.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.117+0000 7fb7a8c00640 1 --2- 172.21.15.86:0/3089577309 >> v2:172.21.15.144:3300/0 conn(0x7fb7a40b4530 0x7fb7a406bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:03.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.117+0000 7fb7a8c00640 1 -- 172.21.15.86:0/3089577309 >> v1:172.21.15.144:6789/0 conn(0x7fb7a415eb30 legacy=0x7fb7a40b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:03.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.117+0000 7fb7a8c00640 1 -- 172.21.15.86:0/3089577309 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7a40ae090 con 0x7fb7a40aee50 2024-04-20T21:20:03.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.118+0000 7fb7a8c00640 1 --2- 172.21.15.86:0/3089577309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7a40aee50 0x7fb7a406b980 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb7a40d14a0 tx=0x7fb79002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81988122002c0a4e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:03.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.119+0000 7fb7a3e00640 1 -- 172.21.15.86:0/3089577309 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb790035020 con 0x7fb7a40aee50 2024-04-20T21:20:03.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.119+0000 7fb7a3e00640 1 -- 172.21.15.86:0/3089577309 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb790031870 con 0x7fb7a40aee50 2024-04-20T21:20:03.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.119+0000 7fb7a3e00640 1 -- 172.21.15.86:0/3089577309 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb790031c40 con 0x7fb7a40aee50 2024-04-20T21:20:03.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.119+0000 7fb7ac000640 1 -- 172.21.15.86:0/3089577309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7a40aee50 msgr2=0x7fb7a406b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:03.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.119+0000 7fb7ac000640 1 --2- 172.21.15.86:0/3089577309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7a40aee50 0x7fb7a406b980 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb7a40d14a0 tx=0x7fb79002b910 comp rx=0 tx=0).stop 2024-04-20T21:20:03.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.119+0000 7fb7ac000640 1 -- 172.21.15.86:0/3089577309 shutdown_connections 2024-04-20T21:20:03.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.119+0000 7fb7ac000640 1 --2- 172.21.15.86:0/3089577309 >> v2:172.21.15.144:3300/0 conn(0x7fb7a40b4530 0x7fb7a406bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:03.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.119+0000 7fb7ac000640 1 --2- 172.21.15.86:0/3089577309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7a40aee50 0x7fb7a406b980 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:03.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.119+0000 7fb7ac000640 1 -- 172.21.15.86:0/3089577309 >> 172.21.15.86:0/3089577309 conn(0x7fb7a40b2110 msgr2=0x7fb7a40b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:03.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.120+0000 7fb7ac000640 1 -- 172.21.15.86:0/3089577309 shutdown_connections 2024-04-20T21:20:03.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.120+0000 7fb7ac000640 1 -- 172.21.15.86:0/3089577309 wait complete. 2024-04-20T21:20:03.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.121+0000 7fb7ac000640 1 Processor -- start 2024-04-20T21:20:03.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.121+0000 7fb7ac000640 1 -- start start 2024-04-20T21:20:03.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.122+0000 7fb7ac000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb7a40aee50 0x7fb7a415b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:03.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.122+0000 7fb7ac000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7a40b4530 0x7fb7a415b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:03.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.122+0000 7fb7ac000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7a415eb30 0x7fb7a4156a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:03.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.122+0000 7fb7ac000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a40ae930 con 0x7fb7a415eb30 2024-04-20T21:20:03.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.122+0000 7fb7ac000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a40ae650 con 0x7fb7a40b4530 2024-04-20T21:20:03.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.122+0000 7fb7ac000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb7a406e160 con 0x7fb7a40aee50 2024-04-20T21:20:03.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.123+0000 7fb7aa000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7a415eb30 0x7fb7a4156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:03.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.123+0000 7fb7aa000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7a415eb30 0x7fb7a4156a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60702/0 (socket says 172.21.15.86:60702) 2024-04-20T21:20:03.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.123+0000 7fb7aa000640 1 -- 172.21.15.86:0/1494101757 learned_addr learned my addr 172.21.15.86:0/1494101757 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:03.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.123+0000 7fb7a9600640 1 --2- 172.21.15.86:0/1494101757 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb7a40aee50 0x7fb7a415b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:03.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.123+0000 7fb7aa000640 1 -- 172.21.15.86:0/1494101757 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb7a40aee50 msgr2=0x7fb7a415b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:03.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.123+0000 7fb7aa000640 1 --2- 172.21.15.86:0/1494101757 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb7a40aee50 0x7fb7a415b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:03.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.123+0000 7fb7aa000640 1 -- 172.21.15.86:0/1494101757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7a40b4530 msgr2=0x7fb7a415b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:03.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.123+0000 7fb7aa000640 1 --2- 172.21.15.86:0/1494101757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7a40b4530 0x7fb7a415b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:03.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.123+0000 7fb7aa000640 1 -- 172.21.15.86:0/1494101757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7a41f4cb0 con 0x7fb7a415eb30 2024-04-20T21:20:03.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.124+0000 7fb7aa000640 1 --2- 172.21.15.86:0/1494101757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7a415eb30 0x7fb7a4156a60 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fb79c00b9e0 tx=0x7fb79c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:03.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.124+0000 7fb7a1600640 1 -- 172.21.15.86:0/1494101757 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb79c019070 con 0x7fb7a415eb30 2024-04-20T21:20:03.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.124+0000 7fb7a1600640 1 -- 172.21.15.86:0/1494101757 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb79c00cb20 con 0x7fb7a415eb30 2024-04-20T21:20:03.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.124+0000 7fb7a1600640 1 -- 172.21.15.86:0/1494101757 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb79c014440 con 0x7fb7a415eb30 2024-04-20T21:20:03.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.124+0000 7fb7ac000640 1 -- 172.21.15.86:0/1494101757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7a41f4f90 con 0x7fb7a415eb30 2024-04-20T21:20:03.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.124+0000 7fb7ac000640 1 -- 172.21.15.86:0/1494101757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7a41f5550 con 0x7fb7a415eb30 2024-04-20T21:20:03.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.125+0000 7fb7ac000640 1 -- 172.21.15.86:0/1494101757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7a40689e0 con 0x7fb7a415eb30 2024-04-20T21:20:03.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.127+0000 7fb7a1600640 1 -- 172.21.15.86:0/1494101757 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb79c01c070 con 0x7fb7a415eb30 2024-04-20T21:20:03.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.127+0000 7fb7a1600640 1 --2- 172.21.15.86:0/1494101757 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb78406c8f0 0x7fb78406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:03.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.128+0000 7fb7a1600640 1 -- 172.21.15.86:0/1494101757 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5609+0+0 (secure 0 0 0) 0x7fb79c08b7a0 con 0x7fb7a415eb30 2024-04-20T21:20:03.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.128+0000 7fb7a9600640 1 --2- 172.21.15.86:0/1494101757 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb78406c8f0 0x7fb78406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:03.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.129+0000 7fb7a9600640 1 --2- 172.21.15.86:0/1494101757 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb78406c8f0 0x7fb78406edb0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb7a4152d30 tx=0x7fb798008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:03.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.133+0000 7fb7a1600640 1 -- 172.21.15.86:0/1494101757 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb79c059c20 con 0x7fb7a415eb30 2024-04-20T21:20:03.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.367+0000 7fb7ac000640 1 -- 172.21.15.86:0/1494101757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fb7a415c310 con 0x7fb7a415eb30 2024-04-20T21:20:03.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.933+0000 7fb7a1600640 1 -- 172.21.15.86:0/1494101757 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v20) ==== 172+0+0 (secure 0 0 0) 0x7fb79c059520 con 0x7fb7a415eb30 2024-04-20T21:20:03.940 INFO:teuthology.orchestra.run.smithi086.stderr:new fs with metadata pool 6 and data pool 7 2024-04-20T21:20:03.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.944+0000 7fb7ac000640 1 -- 172.21.15.86:0/1494101757 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb78406c8f0 msgr2=0x7fb78406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:03.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.944+0000 7fb7ac000640 1 --2- 172.21.15.86:0/1494101757 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb78406c8f0 0x7fb78406edb0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb7a4152d30 tx=0x7fb798008040 comp rx=0 tx=0).stop 2024-04-20T21:20:03.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.944+0000 7fb7ac000640 1 -- 172.21.15.86:0/1494101757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7a415eb30 msgr2=0x7fb7a4156a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:03.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.944+0000 7fb7ac000640 1 --2- 172.21.15.86:0/1494101757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7a415eb30 0x7fb7a4156a60 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fb79c00b9e0 tx=0x7fb79c00bea0 comp rx=0 tx=0).stop 2024-04-20T21:20:03.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.945+0000 7fb7ac000640 1 -- 172.21.15.86:0/1494101757 shutdown_connections 2024-04-20T21:20:03.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.945+0000 7fb7ac000640 1 --2- 172.21.15.86:0/1494101757 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb78406c8f0 0x7fb78406edb0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:03.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.945+0000 7fb7ac000640 1 --2- 172.21.15.86:0/1494101757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7a415eb30 0x7fb7a4156a60 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:03.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.945+0000 7fb7ac000640 1 --2- 172.21.15.86:0/1494101757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7a40b4530 0x7fb7a415b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:03.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.945+0000 7fb7ac000640 1 --2- 172.21.15.86:0/1494101757 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb7a40aee50 0x7fb7a415b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:03.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.945+0000 7fb7ac000640 1 -- 172.21.15.86:0/1494101757 >> 172.21.15.86:0/1494101757 conn(0x7fb7a40b2110 msgr2=0x7fb7a415dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:03.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.946+0000 7fb7ac000640 1 -- 172.21.15.86:0/1494101757 shutdown_connections 2024-04-20T21:20:03.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:03.946+0000 7fb7ac000640 1 -- 172.21.15.86:0/1494101757 wait complete. 2024-04-20T21:20:03.981 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:20:04.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.147+0000 7f1542a00640 1 Processor -- start 2024-04-20T21:20:04.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.147+0000 7f1542a00640 1 -- start start 2024-04-20T21:20:04.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.148+0000 7f1542a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f153c162180 0x7f153c0cfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:04.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.148+0000 7f1542a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f153c0d52c0 0x7f153c0d2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:04.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.148+0000 7f1542a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f153c0b34b0 0x7f153c0d0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:04.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.148+0000 7f1542a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f153c06dc50 con 0x7f153c162180 2024-04-20T21:20:04.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.148+0000 7f1542a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f153c06ea70 con 0x7f153c0d52c0 2024-04-20T21:20:04.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.148+0000 7f1542a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f153c0d1c80 con 0x7f153c0b34b0 2024-04-20T21:20:04.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.148+0000 7f1541600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f153c162180 0x7f153c0cfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:04.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.148+0000 7f1541600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f153c162180 0x7f153c0cfa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60708/0 (socket says 172.21.15.86:60708) 2024-04-20T21:20:04.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.148+0000 7f1541600640 1 -- 172.21.15.86:0/3565100876 learned_addr learned my addr 172.21.15.86:0/3565100876 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:04.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.149+0000 7f1541600640 1 -- 172.21.15.86:0/3565100876 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f153c0b34b0 msgr2=0x7f153c0d0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:04.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.149+0000 7f1541600640 1 --2- 172.21.15.86:0/3565100876 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f153c0b34b0 0x7f153c0d0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:04.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.149+0000 7f1541600640 1 -- 172.21.15.86:0/3565100876 >> v2:172.21.15.144:3300/0 conn(0x7f153c0d52c0 msgr2=0x7f153c0d2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:04.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.149+0000 7f1541600640 1 --2- 172.21.15.86:0/3565100876 >> v2:172.21.15.144:3300/0 conn(0x7f153c0d52c0 0x7f153c0d2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:04.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.149+0000 7f1541600640 1 -- 172.21.15.86:0/3565100876 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f153c06e4b0 con 0x7f153c162180 2024-04-20T21:20:04.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.150+0000 7f1541600640 1 --2- 172.21.15.86:0/3565100876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f153c162180 0x7f153c0cfa00 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f153c0b3fd0 tx=0x7f153002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e87294c5d34662b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:04.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.150+0000 7f153be00640 1 -- 172.21.15.86:0/3565100876 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1530035020 con 0x7f153c162180 2024-04-20T21:20:04.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.150+0000 7f153be00640 1 -- 172.21.15.86:0/3565100876 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1530031870 con 0x7f153c162180 2024-04-20T21:20:04.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.150+0000 7f153be00640 1 -- 172.21.15.86:0/3565100876 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1530031c40 con 0x7f153c162180 2024-04-20T21:20:04.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.151+0000 7f1542a00640 1 -- 172.21.15.86:0/3565100876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f153c162180 msgr2=0x7f153c0cfa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:04.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.151+0000 7f1542a00640 1 --2- 172.21.15.86:0/3565100876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f153c162180 0x7f153c0cfa00 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f153c0b3fd0 tx=0x7f153002b910 comp rx=0 tx=0).stop 2024-04-20T21:20:04.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.151+0000 7f1542a00640 1 -- 172.21.15.86:0/3565100876 shutdown_connections 2024-04-20T21:20:04.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.151+0000 7f1542a00640 1 --2- 172.21.15.86:0/3565100876 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f153c0b34b0 0x7f153c0d0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:04.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.151+0000 7f1542a00640 1 --2- 172.21.15.86:0/3565100876 >> v2:172.21.15.144:3300/0 conn(0x7f153c0d52c0 0x7f153c0d2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:04.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.151+0000 7f1542a00640 1 --2- 172.21.15.86:0/3565100876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f153c162180 0x7f153c0cfa00 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:04.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.151+0000 7f1542a00640 1 -- 172.21.15.86:0/3565100876 >> 172.21.15.86:0/3565100876 conn(0x7f153c08bf70 msgr2=0x7f153c0d45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:04.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.151+0000 7f1542a00640 1 -- 172.21.15.86:0/3565100876 shutdown_connections 2024-04-20T21:20:04.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.151+0000 7f1542a00640 1 -- 172.21.15.86:0/3565100876 wait complete. 2024-04-20T21:20:04.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.152+0000 7f1542a00640 1 Processor -- start 2024-04-20T21:20:04.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.153+0000 7f1542a00640 1 -- start start 2024-04-20T21:20:04.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.153+0000 7f1542a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f153c0b34b0 0x7f153c0ce3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:04.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.153+0000 7f1542a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f153c0d52c0 0x7f153c152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:04.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.153+0000 7f1542a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f153c162180 0x7f153c0b9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:04.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.153+0000 7f1542a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f153c06e260 con 0x7f153c162180 2024-04-20T21:20:04.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.153+0000 7f1542a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f153c06e0e0 con 0x7f153c0b34b0 2024-04-20T21:20:04.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.153+0000 7f1542a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f153c0cff40 con 0x7f153c0d52c0 2024-04-20T21:20:04.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.154+0000 7f1542000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f153c162180 0x7f153c0b9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:04.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.154+0000 7f1542000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f153c162180 0x7f153c0b9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60722/0 (socket says 172.21.15.86:60722) 2024-04-20T21:20:04.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.154+0000 7f1542000640 1 -- 172.21.15.86:0/3771231819 learned_addr learned my addr 172.21.15.86:0/3771231819 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:04.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.154+0000 7f1541600640 1 --2- 172.21.15.86:0/3771231819 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f153c0b34b0 0x7f153c0ce3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:04.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.154+0000 7f1542000640 1 -- 172.21.15.86:0/3771231819 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f153c0d52c0 msgr2=0x7f153c152be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:04.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.154+0000 7f1542000640 1 --2- 172.21.15.86:0/3771231819 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f153c0d52c0 0x7f153c152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:04.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.154+0000 7f1542000640 1 -- 172.21.15.86:0/3771231819 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f153c0b34b0 msgr2=0x7f153c0ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:04.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.154+0000 7f1542000640 1 --2- 172.21.15.86:0/3771231819 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f153c0b34b0 0x7f153c0ce3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:04.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.154+0000 7f1542000640 1 -- 172.21.15.86:0/3771231819 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f153c0ce900 con 0x7f153c162180 2024-04-20T21:20:04.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.154+0000 7f1542000640 1 --2- 172.21.15.86:0/3771231819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f153c162180 0x7f153c0b9b90 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f153400b970 tx=0x7f153400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:04.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.155+0000 7f1539600640 1 -- 172.21.15.86:0/3771231819 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1534019070 con 0x7f153c162180 2024-04-20T21:20:04.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.155+0000 7f1539600640 1 -- 172.21.15.86:0/3771231819 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f153400cb60 con 0x7f153c162180 2024-04-20T21:20:04.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.155+0000 7f1542a00640 1 -- 172.21.15.86:0/3771231819 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f153c0b8af0 con 0x7f153c162180 2024-04-20T21:20:04.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.155+0000 7f1539600640 1 -- 172.21.15.86:0/3771231819 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1534014440 con 0x7f153c162180 2024-04-20T21:20:04.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.155+0000 7f1542a00640 1 -- 172.21.15.86:0/3771231819 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f153c0b90b0 con 0x7f153c162180 2024-04-20T21:20:04.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.156+0000 7f1542a00640 1 -- 172.21.15.86:0/3771231819 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f153c04fcf0 con 0x7f153c162180 2024-04-20T21:20:04.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.157+0000 7f1539600640 1 -- 172.21.15.86:0/3771231819 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f153401c070 con 0x7f153c162180 2024-04-20T21:20:04.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.157+0000 7f1539600640 1 --2- 172.21.15.86:0/3771231819 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f151c06c8f0 0x7f151c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:04.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.158+0000 7f1539600640 1 -- 172.21.15.86:0/3771231819 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f153408b930 con 0x7f153c162180 2024-04-20T21:20:04.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.158+0000 7f1541600640 1 --2- 172.21.15.86:0/3771231819 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f151c06c8f0 0x7f151c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:04.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.158+0000 7f1541600640 1 --2- 172.21.15.86:0/3771231819 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f151c06c8f0 0x7f151c06edb0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1530036000 tx=0x7f153003c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:04.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.162+0000 7f1539600640 1 -- 172.21.15.86:0/3771231819 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1534059cc0 con 0x7f153c162180 2024-04-20T21:20:04.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.359+0000 7f1542a00640 1 -- 172.21.15.86:0/3771231819 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f153c0b16f0 con 0x7f153c162180 2024-04-20T21:20:04.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.361+0000 7f1539600640 1 -- 172.21.15.86:0/3771231819 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) ==== 74+0+16626 (secure 0 0 0) 0x7f15340595c0 con 0x7f153c162180 2024-04-20T21:20:04.360 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:20:04.360 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":44,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:20:03.369351+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:20:01.307757+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-20T21:20:02.359347+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:20:04.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.363+0000 7f1542a00640 1 -- 172.21.15.86:0/3771231819 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f151c06c8f0 msgr2=0x7f151c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:04.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.363+0000 7f1542a00640 1 --2- 172.21.15.86:0/3771231819 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f151c06c8f0 0x7f151c06edb0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1530036000 tx=0x7f153003c040 comp rx=0 tx=0).stop 2024-04-20T21:20:04.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.363+0000 7f1542a00640 1 -- 172.21.15.86:0/3771231819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f153c162180 msgr2=0x7f153c0b9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:04.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.363+0000 7f1542a00640 1 --2- 172.21.15.86:0/3771231819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f153c162180 0x7f153c0b9b90 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f153400b970 tx=0x7f153400be30 comp rx=0 tx=0).stop 2024-04-20T21:20:04.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.363+0000 7f1542a00640 1 -- 172.21.15.86:0/3771231819 shutdown_connections 2024-04-20T21:20:04.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.363+0000 7f1542a00640 1 --2- 172.21.15.86:0/3771231819 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f151c06c8f0 0x7f151c06edb0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:04.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.363+0000 7f1542a00640 1 --2- 172.21.15.86:0/3771231819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f153c162180 0x7f153c0b9b90 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:04.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.363+0000 7f1542a00640 1 --2- 172.21.15.86:0/3771231819 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f153c0d52c0 0x7f153c152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:04.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.363+0000 7f1542a00640 1 --2- 172.21.15.86:0/3771231819 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f153c0b34b0 0x7f153c0ce3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:04.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.363+0000 7f1542a00640 1 -- 172.21.15.86:0/3771231819 >> 172.21.15.86:0/3771231819 conn(0x7f153c08bf70 msgr2=0x7f153c15f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:04.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.363+0000 7f1542a00640 1 -- 172.21.15.86:0/3771231819 shutdown_connections 2024-04-20T21:20:04.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.363+0000 7f1542a00640 1 -- 172.21.15.86:0/3771231819 wait complete. 2024-04-20T21:20:04.380 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:20:04.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.574+0000 7f9692400640 1 Processor -- start 2024-04-20T21:20:04.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.574+0000 7f9692400640 1 -- start start 2024-04-20T21:20:04.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.576+0000 7f9692400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f968c06e6d0 0x7f968c071d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:04.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.576+0000 7f9692400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f968c06eb30 con 0x7f968c072240 2024-04-20T21:20:04.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.576+0000 7f9692400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f968c06dc30 con 0x7f968c146130 2024-04-20T21:20:04.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.576+0000 7f9692400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f968c0cd8e0 con 0x7f968c06e6d0 2024-04-20T21:20:04.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.576+0000 7f968be00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f968c072240 0x7f968c15cc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41586/0 (socket says 172.21.15.86:41586) 2024-04-20T21:20:04.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.576+0000 7f968be00640 1 -- 172.21.15.86:0/218870188 learned_addr learned my addr 172.21.15.86:0/218870188 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:20:04.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.576+0000 7f9683e00640 1 --2- 172.21.15.86:0/218870188 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f968c06e6d0 0x7f968c071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:04.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.577+0000 7f968aa00640 1 -- 172.21.15.86:0/218870188 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2538072759 0 0) 0x7f9684001490 con 0x7f968c072240 2024-04-20T21:20:04.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.577+0000 7f968aa00640 1 -- 172.21.15.86:0/218870188 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9670003520 con 0x7f968c072240 2024-04-20T21:20:04.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.577+0000 7f968aa00640 1 -- 172.21.15.86:0/218870188 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3585538777 0 0) 0x7f9670003520 con 0x7f968c072240 2024-04-20T21:20:04.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.577+0000 7f968aa00640 1 -- 172.21.15.86:0/218870188 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f968c06e6d0 msgr2=0x7f968c071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:04.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.577+0000 7f968aa00640 1 --2- 172.21.15.86:0/218870188 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f968c06e6d0 0x7f968c071d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:04.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.577+0000 7f968aa00640 1 -- 172.21.15.86:0/218870188 >> v1:172.21.15.144:6789/0 conn(0x7f968c146130 legacy=0x7f968c0ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:04.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.577+0000 7f968aa00640 1 -- 172.21.15.86:0/218870188 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f968c0cdbc0 con 0x7f968c072240 2024-04-20T21:20:04.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.577+0000 7f968aa00640 1 -- 172.21.15.86:0/218870188 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f9684001bc0 con 0x7f968c072240 2024-04-20T21:20:04.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.577+0000 7f968aa00640 1 -- 172.21.15.86:0/218870188 >> v1:172.21.15.86:6789/0 conn(0x7f968c072240 legacy=0x7f968c15cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:04.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.577+0000 7f968aa00640 1 --2- 172.21.15.86:0/218870188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96700039e0 0x7f9670003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:04.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.577+0000 7f968aa00640 1 --2- 172.21.15.86:0/218870188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9670004800 0x7f9670004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:04.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.577+0000 7f968aa00640 1 --2- 172.21.15.86:0/218870188 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f968c06e6d0 0x7f96700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:04.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.577+0000 7f968aa00640 1 -- 172.21.15.86:0/218870188 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f968c0cd8e0 con 0x7f9670004800 2024-04-20T21:20:04.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.577+0000 7f968aa00640 1 -- 172.21.15.86:0/218870188 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f967000bb00 con 0x7f96700039e0 2024-04-20T21:20:04.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.577+0000 7f968aa00640 1 -- 172.21.15.86:0/218870188 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f967000bd00 con 0x7f968c06e6d0 2024-04-20T21:20:04.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.578+0000 7f9683e00640 1 --2- 172.21.15.86:0/218870188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9670004800 0x7f9670004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:04.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.578+0000 7f968b400640 1 --2- 172.21.15.86:0/218870188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96700039e0 0x7f9670003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:04.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.578+0000 7f968be00640 1 --2- 172.21.15.86:0/218870188 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f968c06e6d0 0x7f96700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:04.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.578+0000 7f9683e00640 1 -- 172.21.15.86:0/218870188 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f968c06e6d0 msgr2=0x7f96700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:04.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.578+0000 7f9683e00640 1 --2- 172.21.15.86:0/218870188 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f968c06e6d0 0x7f96700051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:04.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.578+0000 7f9683e00640 1 -- 172.21.15.86:0/218870188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96700039e0 msgr2=0x7f9670003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:04.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.578+0000 7f9683e00640 1 --2- 172.21.15.86:0/218870188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96700039e0 0x7f9670003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:04.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.578+0000 7f9683e00640 1 -- 172.21.15.86:0/218870188 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9670003520 con 0x7f9670004800 2024-04-20T21:20:04.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.579+0000 7f9683e00640 1 --2- 172.21.15.86:0/218870188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9670004800 0x7f9670004c80 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9674009a90 tx=0x7f9670005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff8fc915f37a3aa3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:04.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.579+0000 7f968aa00640 1 -- 172.21.15.86:0/218870188 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9674039070 con 0x7f9670004800 2024-04-20T21:20:04.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.579+0000 7f968aa00640 1 -- 172.21.15.86:0/218870188 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f967402c680 con 0x7f9670004800 2024-04-20T21:20:04.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.579+0000 7f968aa00640 1 -- 172.21.15.86:0/218870188 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9674009690 con 0x7f9670004800 2024-04-20T21:20:04.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.580+0000 7f9692400640 1 -- 172.21.15.86:0/218870188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9670004800 msgr2=0x7f9670004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:04.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.580+0000 7f9692400640 1 --2- 172.21.15.86:0/218870188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9670004800 0x7f9670004c80 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9674009a90 tx=0x7f9670005830 comp rx=0 tx=0).stop 2024-04-20T21:20:04.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.580+0000 7f968be00640 1 -- 172.21.15.86:0/218870188 reap_dead start 2024-04-20T21:20:04.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.580+0000 7f9692400640 1 -- 172.21.15.86:0/218870188 shutdown_connections 2024-04-20T21:20:04.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.580+0000 7f9692400640 1 -- 172.21.15.86:0/218870188 >> 172.21.15.86:0/218870188 conn(0x7f968c0b2360 msgr2=0x7f968c0b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:04.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.580+0000 7f9692400640 1 -- 172.21.15.86:0/218870188 shutdown_connections 2024-04-20T21:20:04.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.580+0000 7f9692400640 1 -- 172.21.15.86:0/218870188 wait complete. 2024-04-20T21:20:04.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.582+0000 7f9692400640 1 Processor -- start 2024-04-20T21:20:04.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.582+0000 7f9692400640 1 -- start start 2024-04-20T21:20:04.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.583+0000 7f9692400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9670004800 0x7f968c1f2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:04.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.583+0000 7f9692400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f968c1f2bb0 0x7f968c0ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:04.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.583+0000 7f9692400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f968c15be10 0x7f968c1e67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:04.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.583+0000 7f9692400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f968c06f150 con 0x7f968c15be10 2024-04-20T21:20:04.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.583+0000 7f9692400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f968c06e1b0 con 0x7f9670004800 2024-04-20T21:20:04.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.583+0000 7f9692400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f968c0cd4e0 con 0x7f968c1f2bb0 2024-04-20T21:20:04.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.583+0000 7f968be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f968c15be10 0x7f968c1e67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:04.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.583+0000 7f968be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f968c15be10 0x7f968c1e67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60736/0 (socket says 172.21.15.86:60736) 2024-04-20T21:20:04.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.583+0000 7f968be00640 1 -- 172.21.15.86:0/883510961 learned_addr learned my addr 172.21.15.86:0/883510961 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:04.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.584+0000 7f968b400640 1 --2- 172.21.15.86:0/883510961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9670004800 0x7f968c1f2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:04.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.584+0000 7f9683e00640 1 --2- 172.21.15.86:0/883510961 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f968c1f2bb0 0x7f968c0ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:04.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.584+0000 7f968be00640 1 -- 172.21.15.86:0/883510961 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f968c1f2bb0 msgr2=0x7f968c0ae930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:04.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.584+0000 7f968be00640 1 --2- 172.21.15.86:0/883510961 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f968c1f2bb0 0x7f968c0ae930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:04.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.584+0000 7f968be00640 1 -- 172.21.15.86:0/883510961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9670004800 msgr2=0x7f968c1f2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:04.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.584+0000 7f968be00640 1 --2- 172.21.15.86:0/883510961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9670004800 0x7f968c1f2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:04.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.584+0000 7f968be00640 1 -- 172.21.15.86:0/883510961 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f968c1e5a30 con 0x7f968c15be10 2024-04-20T21:20:04.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.584+0000 7f9683e00640 1 --2- 172.21.15.86:0/883510961 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f968c1f2bb0 0x7f968c0ae930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:20:04.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.584+0000 7f968be00640 1 --2- 172.21.15.86:0/883510961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f968c15be10 0x7f968c1e67f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f9684001750 tx=0x7f968400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:04.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.584+0000 7f9683400640 1 -- 172.21.15.86:0/883510961 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9684004d60 con 0x7f968c15be10 2024-04-20T21:20:04.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.585+0000 7f9683400640 1 -- 172.21.15.86:0/883510961 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f968400bb80 con 0x7f968c15be10 2024-04-20T21:20:04.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.585+0000 7f9683400640 1 -- 172.21.15.86:0/883510961 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9684007110 con 0x7f968c15be10 2024-04-20T21:20:04.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.585+0000 7f9692400640 1 -- 172.21.15.86:0/883510961 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f968c1e5d10 con 0x7f968c15be10 2024-04-20T21:20:04.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.585+0000 7f9692400640 1 -- 172.21.15.86:0/883510961 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f968c1e62d0 con 0x7f968c15be10 2024-04-20T21:20:04.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.586+0000 7f9692400640 1 -- 172.21.15.86:0/883510961 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f968c0689e0 con 0x7f968c15be10 2024-04-20T21:20:04.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.587+0000 7f9683400640 1 -- 172.21.15.86:0/883510961 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f968401b060 con 0x7f968c15be10 2024-04-20T21:20:04.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.588+0000 7f9683400640 1 --2- 172.21.15.86:0/883510961 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9668070d50 0x7f9668073210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:04.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.588+0000 7f9683400640 1 -- 172.21.15.86:0/883510961 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f968408ba60 con 0x7f968c15be10 2024-04-20T21:20:04.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.588+0000 7f968b400640 1 --2- 172.21.15.86:0/883510961 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9668070d50 0x7f9668073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:04.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.589+0000 7f968b400640 1 --2- 172.21.15.86:0/883510961 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9668070d50 0x7f9668073210 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f968c13ec50 tx=0x7f967c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:04.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.592+0000 7f9683400640 1 -- 172.21.15.86:0/883510961 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f968c06eb30 con 0x7f968c15be10 2024-04-20T21:20:04.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.792+0000 7f9692400640 1 -- 172.21.15.86:0/883510961 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f968c141bf0 con 0x7f968c15be10 2024-04-20T21:20:04.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.794+0000 7f9683400640 1 -- 172.21.15.86:0/883510961 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) ==== 74+0+16626 (secure 0 0 0) 0x7f9684059f00 con 0x7f968c15be10 2024-04-20T21:20:04.793 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:20:04.794 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":44,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:20:03.369351+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:20:01.307757+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-20T21:20:02.359347+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:20:04.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.796+0000 7f9692400640 1 -- 172.21.15.86:0/883510961 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9668070d50 msgr2=0x7f9668073210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:04.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.796+0000 7f9692400640 1 --2- 172.21.15.86:0/883510961 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9668070d50 0x7f9668073210 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f968c13ec50 tx=0x7f967c009040 comp rx=0 tx=0).stop 2024-04-20T21:20:04.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.796+0000 7f9692400640 1 -- 172.21.15.86:0/883510961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f968c15be10 msgr2=0x7f968c1e67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:04.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.796+0000 7f9692400640 1 --2- 172.21.15.86:0/883510961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f968c15be10 0x7f968c1e67f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f9684001750 tx=0x7f968400b4a0 comp rx=0 tx=0).stop 2024-04-20T21:20:04.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.797+0000 7f9692400640 1 -- 172.21.15.86:0/883510961 shutdown_connections 2024-04-20T21:20:04.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.797+0000 7f9692400640 1 --2- 172.21.15.86:0/883510961 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9668070d50 0x7f9668073210 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:04.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.797+0000 7f9692400640 1 --2- 172.21.15.86:0/883510961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f968c15be10 0x7f968c1e67f0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:04.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.797+0000 7f9692400640 1 --2- 172.21.15.86:0/883510961 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f968c1f2bb0 0x7f968c0ae930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:04.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.797+0000 7f9692400640 1 --2- 172.21.15.86:0/883510961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9670004800 0x7f968c1f2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:04.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.797+0000 7f9692400640 1 -- 172.21.15.86:0/883510961 >> 172.21.15.86:0/883510961 conn(0x7f968c0b2360 msgr2=0x7f968c13fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:04.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.797+0000 7f9692400640 1 -- 172.21.15.86:0/883510961 shutdown_connections 2024-04-20T21:20:04.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:04.797+0000 7f9692400640 1 -- 172.21.15.86:0/883510961 wait complete. 2024-04-20T21:20:04.815 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-20T21:20:05.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.016+0000 7f2c42e00640 1 Processor -- start 2024-04-20T21:20:05.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.016+0000 7f2c42e00640 1 -- start start 2024-04-20T21:20:05.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.017+0000 7f2c42e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2c3c15eb30 0x7f2c3c0b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:05.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.017+0000 7f2c42e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2c3c0aee50 0x7f2c3c0b4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:05.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.018+0000 7f2c42e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c3c0adb10 con 0x7f2c3c06b980 2024-04-20T21:20:05.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.018+0000 7f2c42e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f2c3c0af7d0 con 0x7f2c3c0aee50 2024-04-20T21:20:05.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.018+0000 7f2c42e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f2c3c0addb0 con 0x7f2c3c15eb30 2024-04-20T21:20:05.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.018+0000 7f2c3b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2c3c0aee50 0x7f2c3c0b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:05.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.018+0000 7f2c3b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2c3c0aee50 0x7f2c3c0b4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60746/0 (socket says 172.21.15.86:60746) 2024-04-20T21:20:05.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.018+0000 7f2c3b400640 1 -- 172.21.15.86:0/4205319027 learned_addr learned my addr 172.21.15.86:0/4205319027 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:05.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.018+0000 7f2c3b400640 1 -- 172.21.15.86:0/4205319027 >> v2:172.21.15.144:3300/0 conn(0x7f2c3c15eb30 msgr2=0x7f2c3c0b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:05.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.018+0000 7f2c3be00640 1 --2- 172.21.15.86:0/4205319027 >> v2:172.21.15.144:3300/0 conn(0x7f2c3c15eb30 0x7f2c3c0b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:05.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.018+0000 7f2c3b400640 1 --2- 172.21.15.86:0/4205319027 >> v2:172.21.15.144:3300/0 conn(0x7f2c3c15eb30 0x7f2c3c0b3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:05.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.018+0000 7f2c3b400640 1 -- 172.21.15.86:0/4205319027 >> v1:172.21.15.144:6789/0 conn(0x7f2c3c06b980 legacy=0x7f2c3c071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:05.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.019+0000 7f2c3b400640 1 -- 172.21.15.86:0/4205319027 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c3c0ae090 con 0x7f2c3c0aee50 2024-04-20T21:20:05.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.020+0000 7f2c3b400640 1 --2- 172.21.15.86:0/4205319027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c3c0aee50 0x7f2c3c0b4360 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c070130 tx=0x7f2c2802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab249415305bc69 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:05.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.020+0000 7f2c3aa00640 1 -- 172.21.15.86:0/4205319027 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c28035020 con 0x7f2c3c0aee50 2024-04-20T21:20:05.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.020+0000 7f2c3aa00640 1 -- 172.21.15.86:0/4205319027 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c28031870 con 0x7f2c3c0aee50 2024-04-20T21:20:05.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.020+0000 7f2c3aa00640 1 -- 172.21.15.86:0/4205319027 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c28031c40 con 0x7f2c3c0aee50 2024-04-20T21:20:05.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.021+0000 7f2c42e00640 1 -- 172.21.15.86:0/4205319027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c3c0aee50 msgr2=0x7f2c3c0b4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:05.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.021+0000 7f2c42e00640 1 --2- 172.21.15.86:0/4205319027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c3c0aee50 0x7f2c3c0b4360 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c070130 tx=0x7f2c2802b910 comp rx=0 tx=0).stop 2024-04-20T21:20:05.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.021+0000 7f2c42e00640 1 -- 172.21.15.86:0/4205319027 shutdown_connections 2024-04-20T21:20:05.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.021+0000 7f2c42e00640 1 --2- 172.21.15.86:0/4205319027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c3c0aee50 0x7f2c3c0b4360 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:05.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.021+0000 7f2c42e00640 1 --2- 172.21.15.86:0/4205319027 >> v2:172.21.15.144:3300/0 conn(0x7f2c3c15eb30 0x7f2c3c0b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:05.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.021+0000 7f2c42e00640 1 -- 172.21.15.86:0/4205319027 >> 172.21.15.86:0/4205319027 conn(0x7f2c3c0b2110 msgr2=0x7f2c3c0b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:05.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.021+0000 7f2c42e00640 1 -- 172.21.15.86:0/4205319027 shutdown_connections 2024-04-20T21:20:05.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.021+0000 7f2c42e00640 1 -- 172.21.15.86:0/4205319027 wait complete. 2024-04-20T21:20:05.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.023+0000 7f2c42e00640 1 Processor -- start 2024-04-20T21:20:05.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.023+0000 7f2c42e00640 1 -- start start 2024-04-20T21:20:05.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.024+0000 7f2c42e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c3c06b980 0x7f2c3c15a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:05.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.024+0000 7f2c42e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c3c0aee50 0x7f2c3c1476a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:05.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.024+0000 7f2c42e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c3c15eb30 0x7f2c3c147be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:05.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.024+0000 7f2c42e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3c0ae930 con 0x7f2c3c0aee50 2024-04-20T21:20:05.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.024+0000 7f2c42e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3c0ae650 con 0x7f2c3c15eb30 2024-04-20T21:20:05.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.024+0000 7f2c42e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2c3c06bdf0 con 0x7f2c3c06b980 2024-04-20T21:20:05.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.025+0000 7f2c3b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c3c0aee50 0x7f2c3c1476a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:05.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.025+0000 7f2c3b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c3c0aee50 0x7f2c3c1476a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60752/0 (socket says 172.21.15.86:60752) 2024-04-20T21:20:05.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.025+0000 7f2c3b400640 1 -- 172.21.15.86:0/2185236611 learned_addr learned my addr 172.21.15.86:0/2185236611 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:05.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.025+0000 7f2c3be00640 1 --2- 172.21.15.86:0/2185236611 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c3c06b980 0x7f2c3c15a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:05.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.025+0000 7f2c40e00640 1 --2- 172.21.15.86:0/2185236611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c3c15eb30 0x7f2c3c147be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:05.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.025+0000 7f2c3b400640 1 -- 172.21.15.86:0/2185236611 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c3c06b980 msgr2=0x7f2c3c15a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:05.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.025+0000 7f2c3b400640 1 --2- 172.21.15.86:0/2185236611 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c3c06b980 0x7f2c3c15a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:05.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.025+0000 7f2c3b400640 1 -- 172.21.15.86:0/2185236611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c3c15eb30 msgr2=0x7f2c3c147be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:05.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.025+0000 7f2c3b400640 1 --2- 172.21.15.86:0/2185236611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c3c15eb30 0x7f2c3c147be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:05.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.025+0000 7f2c3b400640 1 -- 172.21.15.86:0/2185236611 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c3c15b080 con 0x7f2c3c0aee50 2024-04-20T21:20:05.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.025+0000 7f2c40e00640 1 --2- 172.21.15.86:0/2185236611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c3c15eb30 0x7f2c3c147be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:20:05.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.026+0000 7f2c3b400640 1 --2- 172.21.15.86:0/2185236611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c3c0aee50 0x7f2c3c1476a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f2c28000c00 tx=0x7f2c280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:05.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.026+0000 7f2c33e00640 1 -- 172.21.15.86:0/2185236611 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c28040070 con 0x7f2c3c0aee50 2024-04-20T21:20:05.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.026+0000 7f2c33e00640 1 -- 172.21.15.86:0/2185236611 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c2803a420 con 0x7f2c3c0aee50 2024-04-20T21:20:05.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.026+0000 7f2c33e00640 1 -- 172.21.15.86:0/2185236611 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c28035040 con 0x7f2c3c0aee50 2024-04-20T21:20:05.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.026+0000 7f2c42e00640 1 -- 172.21.15.86:0/2185236611 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c3c15b360 con 0x7f2c3c0aee50 2024-04-20T21:20:05.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.026+0000 7f2c42e00640 1 -- 172.21.15.86:0/2185236611 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c3c145900 con 0x7f2c3c0aee50 2024-04-20T21:20:05.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.027+0000 7f2c42e00640 1 -- 172.21.15.86:0/2185236611 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c3c0689e0 con 0x7f2c3c0aee50 2024-04-20T21:20:05.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.028+0000 7f2c33e00640 1 -- 172.21.15.86:0/2185236611 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2c28038020 con 0x7f2c3c0aee50 2024-04-20T21:20:05.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.029+0000 7f2c33e00640 1 --2- 172.21.15.86:0/2185236611 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2c1c06c8f0 0x7f2c1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:05.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.029+0000 7f2c33e00640 1 -- 172.21.15.86:0/2185236611 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f2c280adb50 con 0x7f2c3c0aee50 2024-04-20T21:20:05.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.029+0000 7f2c3be00640 1 --2- 172.21.15.86:0/2185236611 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2c1c06c8f0 0x7f2c1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:05.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.030+0000 7f2c3be00640 1 --2- 172.21.15.86:0/2185236611 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2c1c06c8f0 0x7f2c1c06edb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c00b7c0 tx=0x7f2c2c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:05.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.033+0000 7f2c33e00640 1 -- 172.21.15.86:0/2185236611 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2c2807bf70 con 0x7f2c3c0aee50 2024-04-20T21:20:05.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.276+0000 7f2c42e00640 1 -- 172.21.15.86:0/2185236611 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f2c3c0cccd0 con 0x7f2c3c0aee50 2024-04-20T21:20:05.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.951+0000 7f2c33e00640 1 -- 172.21.15.86:0/2185236611 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v23) ==== 118+0+0 (secure 0 0 0) 0x7f2c2807b870 con 0x7f2c3c0aee50 2024-04-20T21:20:05.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.954+0000 7f2c42e00640 1 -- 172.21.15.86:0/2185236611 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2c1c06c8f0 msgr2=0x7f2c1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:05.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.954+0000 7f2c42e00640 1 --2- 172.21.15.86:0/2185236611 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2c1c06c8f0 0x7f2c1c06edb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c00b7c0 tx=0x7f2c2c008040 comp rx=0 tx=0).stop 2024-04-20T21:20:05.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.954+0000 7f2c42e00640 1 -- 172.21.15.86:0/2185236611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c3c0aee50 msgr2=0x7f2c3c1476a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:05.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.954+0000 7f2c42e00640 1 --2- 172.21.15.86:0/2185236611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c3c0aee50 0x7f2c3c1476a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f2c28000c00 tx=0x7f2c280046c0 comp rx=0 tx=0).stop 2024-04-20T21:20:05.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.954+0000 7f2c42e00640 1 -- 172.21.15.86:0/2185236611 shutdown_connections 2024-04-20T21:20:05.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.955+0000 7f2c42e00640 1 --2- 172.21.15.86:0/2185236611 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2c1c06c8f0 0x7f2c1c06edb0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:05.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.955+0000 7f2c42e00640 1 --2- 172.21.15.86:0/2185236611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c3c15eb30 0x7f2c3c147be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:05.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.955+0000 7f2c42e00640 1 --2- 172.21.15.86:0/2185236611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c3c0aee50 0x7f2c3c1476a0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:05.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.955+0000 7f2c42e00640 1 --2- 172.21.15.86:0/2185236611 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c3c06b980 0x7f2c3c15a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:05.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.955+0000 7f2c42e00640 1 -- 172.21.15.86:0/2185236611 >> 172.21.15.86:0/2185236611 conn(0x7f2c3c0b2110 msgr2=0x7f2c3c0afb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:05.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.955+0000 7f2c42e00640 1 -- 172.21.15.86:0/2185236611 shutdown_connections 2024-04-20T21:20:05.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:05.955+0000 7f2c42e00640 1 -- 172.21.15.86:0/2185236611 wait complete. 2024-04-20T21:20:05.976 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-20T21:20:05.976 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-20T21:20:06.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.179+0000 7f5ca0a00640 1 Processor -- start 2024-04-20T21:20:06.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.179+0000 7f5ca0a00640 1 -- start start 2024-04-20T21:20:06.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.181+0000 7f5ca0a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5c9c145250 0x7f5c9c06efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:06.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.181+0000 7f5ca0a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5c9c072450 0x7f5c9c15cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:06.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.181+0000 7f5ca0a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c9c06f500 con 0x7f5c9c145f00 2024-04-20T21:20:06.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.181+0000 7f5ca0a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f5c9c160410 con 0x7f5c9c145250 2024-04-20T21:20:06.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.181+0000 7f5ca0a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f5c9c1606f0 con 0x7f5c9c072450 2024-04-20T21:20:06.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.181+0000 7f5c9b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5c9c145250 0x7f5c9c06efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:06.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.181+0000 7f5c93e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f5c9c145f00 0x7f5c9c071d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39640/0 (socket says 172.21.15.86:39640) 2024-04-20T21:20:06.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.181+0000 7f5c93e00640 1 -- 172.21.15.86:0/1100467818 learned_addr learned my addr 172.21.15.86:0/1100467818 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:20:06.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.182+0000 7f5c9be00640 1 --2- 172.21.15.86:0/1100467818 >> v2:172.21.15.144:3300/0 conn(0x7f5c9c072450 0x7f5c9c15cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:06.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.182+0000 7f5c9b400640 1 -- 172.21.15.86:0/1100467818 >> v2:172.21.15.144:3300/0 conn(0x7f5c9c072450 msgr2=0x7f5c9c15cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:06.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.182+0000 7f5c9b400640 1 --2- 172.21.15.86:0/1100467818 >> v2:172.21.15.144:3300/0 conn(0x7f5c9c072450 0x7f5c9c15cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:06.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.182+0000 7f5c9b400640 1 -- 172.21.15.86:0/1100467818 >> v1:172.21.15.86:6789/0 conn(0x7f5c9c145f00 legacy=0x7f5c9c071d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:06.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.182+0000 7f5c9b400640 1 -- 172.21.15.86:0/1100467818 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c9c1609d0 con 0x7f5c9c145250 2024-04-20T21:20:06.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.182+0000 7f5c9aa00640 1 -- 172.21.15.86:0/1100467818 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 889203166 0 0) 0x7f5c84001110 con 0x7f5c9c145f00 2024-04-20T21:20:06.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.184+0000 7f5c9b400640 1 --2- 172.21.15.86:0/1100467818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c9c145250 0x7f5c9c06efc0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c145e90 tx=0x7f5c8c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1155ec42fabce7cd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:06.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.184+0000 7f5c9aa00640 1 -- 172.21.15.86:0/1100467818 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c0315a0 con 0x7f5c9c145250 2024-04-20T21:20:06.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.184+0000 7f5c9aa00640 1 -- 172.21.15.86:0/1100467818 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c8c037070 con 0x7f5c9c145250 2024-04-20T21:20:06.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.184+0000 7f5c9aa00640 1 -- 172.21.15.86:0/1100467818 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c031960 con 0x7f5c9c145250 2024-04-20T21:20:06.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.185+0000 7f5ca0a00640 1 -- 172.21.15.86:0/1100467818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c9c145250 msgr2=0x7f5c9c06efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:06.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.185+0000 7f5ca0a00640 1 --2- 172.21.15.86:0/1100467818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c9c145250 0x7f5c9c06efc0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c145e90 tx=0x7f5c8c02b840 comp rx=0 tx=0).stop 2024-04-20T21:20:06.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.185+0000 7f5ca0a00640 1 -- 172.21.15.86:0/1100467818 shutdown_connections 2024-04-20T21:20:06.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.185+0000 7f5ca0a00640 1 --2- 172.21.15.86:0/1100467818 >> v2:172.21.15.144:3300/0 conn(0x7f5c9c072450 0x7f5c9c15cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:06.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.185+0000 7f5ca0a00640 1 --2- 172.21.15.86:0/1100467818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c9c145250 0x7f5c9c06efc0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:06.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.185+0000 7f5ca0a00640 1 -- 172.21.15.86:0/1100467818 >> 172.21.15.86:0/1100467818 conn(0x7f5c9c067cf0 msgr2=0x7f5c9c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:06.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.185+0000 7f5ca0a00640 1 -- 172.21.15.86:0/1100467818 shutdown_connections 2024-04-20T21:20:06.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.185+0000 7f5ca0a00640 1 -- 172.21.15.86:0/1100467818 wait complete. 2024-04-20T21:20:06.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.187+0000 7f5ca0a00640 1 Processor -- start 2024-04-20T21:20:06.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.187+0000 7f5ca0a00640 1 -- start start 2024-04-20T21:20:06.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.188+0000 7f5ca0a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c9c072450 0x7f5c9c15bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:06.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.188+0000 7f5ca0a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c9c145250 0x7f5c9c12f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:06.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.188+0000 7f5ca0a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c9c145f00 0x7f5c9c12fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:06.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.188+0000 7f5ca0a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9c1457b0 con 0x7f5c9c072450 2024-04-20T21:20:06.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.188+0000 7f5ca0a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9c145630 con 0x7f5c9c145f00 2024-04-20T21:20:06.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.188+0000 7f5ca0a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5c9c15d330 con 0x7f5c9c145250 2024-04-20T21:20:06.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.188+0000 7f5c9b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c9c072450 0x7f5c9c15bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:06.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.188+0000 7f5c9b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c9c072450 0x7f5c9c15bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45516/0 (socket says 172.21.15.86:45516) 2024-04-20T21:20:06.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.188+0000 7f5c9b400640 1 -- 172.21.15.86:0/3594768533 learned_addr learned my addr 172.21.15.86:0/3594768533 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:06.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.189+0000 7f5c9b400640 1 -- 172.21.15.86:0/3594768533 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c9c145250 msgr2=0x7f5c9c12f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:06.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.189+0000 7f5c9b400640 1 --2- 172.21.15.86:0/3594768533 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c9c145250 0x7f5c9c12f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:06.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.189+0000 7f5c9be00640 1 --2- 172.21.15.86:0/3594768533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c9c145f00 0x7f5c9c12fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:06.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.189+0000 7f5c9b400640 1 -- 172.21.15.86:0/3594768533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c9c145f00 msgr2=0x7f5c9c12fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:06.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.189+0000 7f5c9b400640 1 --2- 172.21.15.86:0/3594768533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c9c145f00 0x7f5c9c12fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:06.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.189+0000 7f5c9b400640 1 -- 172.21.15.86:0/3594768533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c9c1e6080 con 0x7f5c9c072450 2024-04-20T21:20:06.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.189+0000 7f5c9b400640 1 --2- 172.21.15.86:0/3594768533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c9c072450 0x7f5c9c15bfc0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c000c00 tx=0x7f5c8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:06.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.189+0000 7f5c93400640 1 -- 172.21.15.86:0/3594768533 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c032840 con 0x7f5c9c072450 2024-04-20T21:20:06.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.189+0000 7f5c93400640 1 -- 172.21.15.86:0/3594768533 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c8c0329e0 con 0x7f5c9c072450 2024-04-20T21:20:06.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.189+0000 7f5ca0a00640 1 -- 172.21.15.86:0/3594768533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c9c0caf00 con 0x7f5c9c072450 2024-04-20T21:20:06.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.189+0000 7f5c93400640 1 -- 172.21.15.86:0/3594768533 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c032dd0 con 0x7f5c9c072450 2024-04-20T21:20:06.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.189+0000 7f5ca0a00640 1 -- 172.21.15.86:0/3594768533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c9c1e6320 con 0x7f5c9c072450 2024-04-20T21:20:06.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.191+0000 7f5ca0a00640 1 -- 172.21.15.86:0/3594768533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c9c04fd90 con 0x7f5c9c072450 2024-04-20T21:20:06.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.192+0000 7f5c93400640 1 -- 172.21.15.86:0/3594768533 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5c8c04d3f0 con 0x7f5c9c072450 2024-04-20T21:20:06.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.193+0000 7f5c93400640 1 --2- 172.21.15.86:0/3594768533 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5c7806c8f0 0x7f5c7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:06.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.193+0000 7f5c93400640 1 -- 172.21.15.86:0/3594768533 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f5c8c02f030 con 0x7f5c9c072450 2024-04-20T21:20:06.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.193+0000 7f5c93e00640 1 --2- 172.21.15.86:0/3594768533 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5c7806c8f0 0x7f5c7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:06.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.194+0000 7f5c93e00640 1 --2- 172.21.15.86:0/3594768533 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5c7806c8f0 0x7f5c7806edb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5c84007fd0 tx=0x7f5c84007960 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:06.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.197+0000 7f5c93400640 1 -- 172.21.15.86:0/3594768533 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5c8c040070 con 0x7f5c9c072450 2024-04-20T21:20:06.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.437+0000 7f5ca0a00640 1 -- 172.21.15.86:0/3594768533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f5c9c072ea0 con 0x7f5c9c072450 2024-04-20T21:20:06.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.963+0000 7f5c93400640 1 -- 172.21.15.86:0/3594768533 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v24) ==== 122+0+0 (secure 0 0 0) 0x7f5c8c07bb30 con 0x7f5c9c072450 2024-04-20T21:20:06.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.967+0000 7f5ca0a00640 1 -- 172.21.15.86:0/3594768533 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5c7806c8f0 msgr2=0x7f5c7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:06.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.967+0000 7f5ca0a00640 1 --2- 172.21.15.86:0/3594768533 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5c7806c8f0 0x7f5c7806edb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5c84007fd0 tx=0x7f5c84007960 comp rx=0 tx=0).stop 2024-04-20T21:20:06.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.968+0000 7f5ca0a00640 1 -- 172.21.15.86:0/3594768533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c9c072450 msgr2=0x7f5c9c15bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:06.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.968+0000 7f5ca0a00640 1 --2- 172.21.15.86:0/3594768533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c9c072450 0x7f5c9c15bfc0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c000c00 tx=0x7f5c8c0046c0 comp rx=0 tx=0).stop 2024-04-20T21:20:06.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.968+0000 7f5ca0a00640 1 -- 172.21.15.86:0/3594768533 shutdown_connections 2024-04-20T21:20:06.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.968+0000 7f5ca0a00640 1 --2- 172.21.15.86:0/3594768533 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5c7806c8f0 0x7f5c7806edb0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:06.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.968+0000 7f5ca0a00640 1 --2- 172.21.15.86:0/3594768533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c9c145f00 0x7f5c9c12fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:06.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.968+0000 7f5ca0a00640 1 --2- 172.21.15.86:0/3594768533 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c9c145250 0x7f5c9c12f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:06.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.968+0000 7f5ca0a00640 1 --2- 172.21.15.86:0/3594768533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c9c072450 0x7f5c9c15bfc0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:06.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.968+0000 7f5ca0a00640 1 -- 172.21.15.86:0/3594768533 >> 172.21.15.86:0/3594768533 conn(0x7f5c9c067cf0 msgr2=0x7f5c9c0b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:06.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.968+0000 7f5ca0a00640 1 -- 172.21.15.86:0/3594768533 shutdown_connections 2024-04-20T21:20:06.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:06.968+0000 7f5ca0a00640 1 -- 172.21.15.86:0/3594768533 wait complete. 2024-04-20T21:20:06.990 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-20T21:20:06.990 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:20:07.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.189+0000 7f5f42600640 1 Processor -- start 2024-04-20T21:20:07.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.189+0000 7f5f42600640 1 -- start start 2024-04-20T21:20:07.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.191+0000 7f5f42600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f3c0d0d20 0x7f5f3c0d1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:07.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.191+0000 7f5f42600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f3c163100 con 0x7f5f3c0d5150 2024-04-20T21:20:07.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.191+0000 7f5f42600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f3c0d1640 con 0x7f5f3c0d4cf0 2024-04-20T21:20:07.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.191+0000 7f5f42600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5f3c0cfea0 con 0x7f5f3c0d0d20 2024-04-20T21:20:07.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.191+0000 7f5f41c00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f5f3c0d5150 0x7f5f3c1629f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39656/0 (socket says 172.21.15.86:39656) 2024-04-20T21:20:07.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.191+0000 7f5f41c00640 1 -- 172.21.15.86:0/1129459672 learned_addr learned my addr 172.21.15.86:0/1129459672 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:20:07.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.192+0000 7f5f41200640 1 --2- 172.21.15.86:0/1129459672 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f3c0d0d20 0x7f5f3c0d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:07.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.192+0000 7f5f3be00640 1 -- 172.21.15.86:0/1129459672 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 87066251 0 0) 0x7f5f3c163100 con 0x7f5f3c0d5150 2024-04-20T21:20:07.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.192+0000 7f5f3be00640 1 -- 172.21.15.86:0/1129459672 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f1c003520 con 0x7f5f3c0d5150 2024-04-20T21:20:07.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.192+0000 7f5f3be00640 1 -- 172.21.15.86:0/1129459672 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2093328889 0 0) 0x7f5f1c003520 con 0x7f5f3c0d5150 2024-04-20T21:20:07.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.192+0000 7f5f3be00640 1 -- 172.21.15.86:0/1129459672 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f3c0d0d20 msgr2=0x7f5f3c0d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:07.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.192+0000 7f5f3be00640 1 --2- 172.21.15.86:0/1129459672 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f3c0d0d20 0x7f5f3c0d1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:07.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.192+0000 7f5f3be00640 1 -- 172.21.15.86:0/1129459672 >> v1:172.21.15.144:6789/0 conn(0x7f5f3c0d4cf0 legacy=0x7f5f3c1622e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:07.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.192+0000 7f5f3be00640 1 -- 172.21.15.86:0/1129459672 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f3c0cf290 con 0x7f5f3c0d5150 2024-04-20T21:20:07.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.192+0000 7f5f3be00640 1 -- 172.21.15.86:0/1129459672 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f5f340019f0 con 0x7f5f3c0d5150 2024-04-20T21:20:07.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.192+0000 7f5f3be00640 1 -- 172.21.15.86:0/1129459672 >> v1:172.21.15.86:6789/0 conn(0x7f5f3c0d5150 legacy=0x7f5f3c1629f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:07.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.192+0000 7f5f3be00640 1 --2- 172.21.15.86:0/1129459672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f1c0039e0 0x7f5f1c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:07.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.192+0000 7f5f3be00640 1 --2- 172.21.15.86:0/1129459672 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f3c0d0d20 0x7f5f1c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:07.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.192+0000 7f5f3be00640 1 --2- 172.21.15.86:0/1129459672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f1c004d40 0x7f5f1c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:07.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.192+0000 7f5f3be00640 1 -- 172.21.15.86:0/1129459672 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5f3c0cfea0 con 0x7f5f1c0039e0 2024-04-20T21:20:07.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.192+0000 7f5f3be00640 1 -- 172.21.15.86:0/1129459672 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5f1c00baa0 con 0x7f5f1c004d40 2024-04-20T21:20:07.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.192+0000 7f5f3be00640 1 -- 172.21.15.86:0/1129459672 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5f1c00bca0 con 0x7f5f3c0d0d20 2024-04-20T21:20:07.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.193+0000 7f5f41200640 1 --2- 172.21.15.86:0/1129459672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f1c0039e0 0x7f5f1c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:07.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.193+0000 7f5f41c00640 1 --2- 172.21.15.86:0/1129459672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f1c004d40 0x7f5f1c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:07.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.193+0000 7f5f41200640 1 -- 172.21.15.86:0/1129459672 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f3c0d0d20 msgr2=0x7f5f1c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:07.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.193+0000 7f5f41200640 1 --2- 172.21.15.86:0/1129459672 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f3c0d0d20 0x7f5f1c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:07.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.193+0000 7f5f41200640 1 -- 172.21.15.86:0/1129459672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f1c004d40 msgr2=0x7f5f1c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:07.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.193+0000 7f5f41200640 1 --2- 172.21.15.86:0/1129459672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f1c004d40 0x7f5f1c0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:07.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.193+0000 7f5f41200640 1 -- 172.21.15.86:0/1129459672 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f1c003520 con 0x7f5f1c0039e0 2024-04-20T21:20:07.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.194+0000 7f5f41200640 1 --2- 172.21.15.86:0/1129459672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f1c0039e0 0x7f5f1c003e00 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009a90 tx=0x7f5f1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3c2bbc13c616214 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:07.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.194+0000 7f5f3be00640 1 -- 172.21.15.86:0/1129459672 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f24039070 con 0x7f5f1c0039e0 2024-04-20T21:20:07.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.194+0000 7f5f3be00640 1 -- 172.21.15.86:0/1129459672 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f2402c6b0 con 0x7f5f1c0039e0 2024-04-20T21:20:07.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.194+0000 7f5f3be00640 1 -- 172.21.15.86:0/1129459672 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f24009690 con 0x7f5f1c0039e0 2024-04-20T21:20:07.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.195+0000 7f5f42600640 1 -- 172.21.15.86:0/1129459672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f1c0039e0 msgr2=0x7f5f1c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:07.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.195+0000 7f5f42600640 1 --2- 172.21.15.86:0/1129459672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f1c0039e0 0x7f5f1c003e00 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009a90 tx=0x7f5f1c005830 comp rx=0 tx=0).stop 2024-04-20T21:20:07.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.195+0000 7f5f41c00640 1 -- 172.21.15.86:0/1129459672 reap_dead start 2024-04-20T21:20:07.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.195+0000 7f5f42600640 1 -- 172.21.15.86:0/1129459672 shutdown_connections 2024-04-20T21:20:07.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.195+0000 7f5f42600640 1 -- 172.21.15.86:0/1129459672 >> 172.21.15.86:0/1129459672 conn(0x7f5f3c08bf70 msgr2=0x7f5f3c0d45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:07.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.195+0000 7f5f42600640 1 -- 172.21.15.86:0/1129459672 shutdown_connections 2024-04-20T21:20:07.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.195+0000 7f5f42600640 1 -- 172.21.15.86:0/1129459672 wait complete. 2024-04-20T21:20:07.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.197+0000 7f5f42600640 1 Processor -- start 2024-04-20T21:20:07.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.197+0000 7f5f42600640 1 -- start start 2024-04-20T21:20:07.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.198+0000 7f5f42600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f1c0039e0 0x7f5f3c1523e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:07.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.198+0000 7f5f42600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f3c1e8070 0x7f5f3c0b7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:07.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.198+0000 7f5f42600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f3c0b94f0 0x7f5f3c0b9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:07.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.198+0000 7f5f42600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5f3c0cf850 con 0x7f5f3c0b94f0 2024-04-20T21:20:07.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.198+0000 7f5f42600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5f3c0d31c0 con 0x7f5f3c1e8070 2024-04-20T21:20:07.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.198+0000 7f5f42600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5f3c0cfb30 con 0x7f5f1c0039e0 2024-04-20T21:20:07.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.198+0000 7f5f41c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f3c0b94f0 0x7f5f3c0b9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:07.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.198+0000 7f5f41c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f3c0b94f0 0x7f5f3c0b9970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45530/0 (socket says 172.21.15.86:45530) 2024-04-20T21:20:07.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.198+0000 7f5f41c00640 1 -- 172.21.15.86:0/912658312 learned_addr learned my addr 172.21.15.86:0/912658312 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:07.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.199+0000 7f5f33e00640 1 --2- 172.21.15.86:0/912658312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f3c1e8070 0x7f5f3c0b7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:07.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.199+0000 7f5f41200640 1 --2- 172.21.15.86:0/912658312 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f1c0039e0 0x7f5f3c1523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:07.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.199+0000 7f5f41c00640 1 -- 172.21.15.86:0/912658312 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f1c0039e0 msgr2=0x7f5f3c1523e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:07.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.199+0000 7f5f41c00640 1 --2- 172.21.15.86:0/912658312 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f1c0039e0 0x7f5f3c1523e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:07.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.199+0000 7f5f41c00640 1 -- 172.21.15.86:0/912658312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f3c1e8070 msgr2=0x7f5f3c0b7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:07.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.199+0000 7f5f41c00640 1 --2- 172.21.15.86:0/912658312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f3c1e8070 0x7f5f3c0b7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:07.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.199+0000 7f5f41c00640 1 -- 172.21.15.86:0/912658312 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f3c1f46f0 con 0x7f5f3c0b94f0 2024-04-20T21:20:07.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.199+0000 7f5f41c00640 1 --2- 172.21.15.86:0/912658312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f3c0b94f0 0x7f5f3c0b9970 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f5f34001370 tx=0x7f5f3400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:07.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.199+0000 7f5f39600640 1 -- 172.21.15.86:0/912658312 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3400de90 con 0x7f5f3c0b94f0 2024-04-20T21:20:07.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.199+0000 7f5f39600640 1 -- 172.21.15.86:0/912658312 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f3400bb80 con 0x7f5f3c0b94f0 2024-04-20T21:20:07.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.199+0000 7f5f42600640 1 -- 172.21.15.86:0/912658312 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f3c0b8720 con 0x7f5f3c0b94f0 2024-04-20T21:20:07.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.200+0000 7f5f39600640 1 -- 172.21.15.86:0/912658312 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f34007070 con 0x7f5f3c0b94f0 2024-04-20T21:20:07.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.200+0000 7f5f42600640 1 -- 172.21.15.86:0/912658312 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f3c0cbb30 con 0x7f5f3c0b94f0 2024-04-20T21:20:07.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.201+0000 7f5f42600640 1 -- 172.21.15.86:0/912658312 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f3c0cb920 con 0x7f5f3c0b94f0 2024-04-20T21:20:07.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.202+0000 7f5f39600640 1 -- 172.21.15.86:0/912658312 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5f3401b020 con 0x7f5f3c0b94f0 2024-04-20T21:20:07.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.203+0000 7f5f39600640 1 --2- 172.21.15.86:0/912658312 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5f1806ca90 0x7f5f1806ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:07.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.203+0000 7f5f39600640 1 -- 172.21.15.86:0/912658312 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f5f3408bdd0 con 0x7f5f3c0b94f0 2024-04-20T21:20:07.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.203+0000 7f5f41200640 1 --2- 172.21.15.86:0/912658312 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5f1806ca90 0x7f5f1806ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:07.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.204+0000 7f5f41200640 1 --2- 172.21.15.86:0/912658312 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5f1806ca90 0x7f5f1806ef50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5f24036000 tx=0x7f5f24030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:07.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.207+0000 7f5f39600640 1 -- 172.21.15.86:0/912658312 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5f3405a160 con 0x7f5f3c0b94f0 2024-04-20T21:20:07.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.444+0000 7f5f42600640 1 -- 172.21.15.86:0/912658312 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f3c04fcf0 con 0x7f5f3c0b94f0 2024-04-20T21:20:07.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.446+0000 7f5f39600640 1 -- 172.21.15.86:0/912658312 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5199 (secure 0 0 0) 0x7f5f34059a60 con 0x7f5f3c0b94f0 2024-04-20T21:20:07.445 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:20:07.445 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5044,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/1013106052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":1013106052},{"type":"v1","addr":"172.21.15.144:6833","nonce":1013106052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5047,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/1073576945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1073576945},{"type":"v1","addr":"172.21.15.144:6835","nonce":1073576945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/821548012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":821548012},{"type":"v1","addr":"172.21.15.86:6835","nonce":821548012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:20:03.369281+0000","modified":"2024-04-20T21:20:06.438592+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5691},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5691":{"gid":5691,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/712628984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":712628984},{"type":"v1","addr":"172.21.15.86:6837","nonce":712628984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5691,"qdb_cluster":[5691]},"id":3}]} 2024-04-20T21:20:07.445 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 24 2024-04-20T21:20:07.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.448+0000 7f5f42600640 1 -- 172.21.15.86:0/912658312 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5f1806ca90 msgr2=0x7f5f1806ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:07.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.448+0000 7f5f42600640 1 --2- 172.21.15.86:0/912658312 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5f1806ca90 0x7f5f1806ef50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5f24036000 tx=0x7f5f24030000 comp rx=0 tx=0).stop 2024-04-20T21:20:07.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.448+0000 7f5f42600640 1 -- 172.21.15.86:0/912658312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f3c0b94f0 msgr2=0x7f5f3c0b9970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:07.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.448+0000 7f5f42600640 1 --2- 172.21.15.86:0/912658312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f3c0b94f0 0x7f5f3c0b9970 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f5f34001370 tx=0x7f5f3400b4a0 comp rx=0 tx=0).stop 2024-04-20T21:20:07.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.449+0000 7f5f42600640 1 -- 172.21.15.86:0/912658312 shutdown_connections 2024-04-20T21:20:07.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.449+0000 7f5f42600640 1 --2- 172.21.15.86:0/912658312 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5f1806ca90 0x7f5f1806ef50 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:07.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.449+0000 7f5f42600640 1 --2- 172.21.15.86:0/912658312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5f3c0b94f0 0x7f5f3c0b9970 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:07.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.449+0000 7f5f42600640 1 --2- 172.21.15.86:0/912658312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f3c1e8070 0x7f5f3c0b7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:07.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.449+0000 7f5f42600640 1 --2- 172.21.15.86:0/912658312 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5f1c0039e0 0x7f5f3c1523e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:07.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.449+0000 7f5f42600640 1 -- 172.21.15.86:0/912658312 >> 172.21.15.86:0/912658312 conn(0x7f5f3c08bf70 msgr2=0x7f5f3c15ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:07.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.449+0000 7f5f42600640 1 -- 172.21.15.86:0/912658312 shutdown_connections 2024-04-20T21:20:07.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.449+0000 7f5f42600640 1 -- 172.21.15.86:0/912658312 wait complete. 2024-04-20T21:20:07.466 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:20:07.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.653+0000 7ff543e00640 1 Processor -- start 2024-04-20T21:20:07.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.653+0000 7ff543e00640 1 -- start start 2024-04-20T21:20:07.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.655+0000 7ff543e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff54415c010 0x7ff54415c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:07.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.655+0000 7ff543e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff5440af7b0 0x7ff5440adda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:07.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.655+0000 7ff543e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5440ae710 con 0x7ff54406f4c0 2024-04-20T21:20:07.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.655+0000 7ff543e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff5440af320 con 0x7ff5440af7b0 2024-04-20T21:20:07.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.655+0000 7ff543e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff54415c930 con 0x7ff54415c010 2024-04-20T21:20:07.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.655+0000 7ff542000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff5440af7b0 0x7ff5440adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:07.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.655+0000 7ff542000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff5440af7b0 0x7ff5440adda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45542/0 (socket says 172.21.15.86:45542) 2024-04-20T21:20:07.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.655+0000 7ff542000640 1 -- 172.21.15.86:0/3740246914 learned_addr learned my addr 172.21.15.86:0/3740246914 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:07.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.655+0000 7ff542a00640 1 --2- 172.21.15.86:0/3740246914 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff54415c010 0x7ff54415c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:07.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.656+0000 7ff542000640 1 -- 172.21.15.86:0/3740246914 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff54415c010 msgr2=0x7ff54415c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:07.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.656+0000 7ff542000640 1 --2- 172.21.15.86:0/3740246914 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff54415c010 0x7ff54415c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:07.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.656+0000 7ff542000640 1 -- 172.21.15.86:0/3740246914 >> v1:172.21.15.144:6789/0 conn(0x7ff54406f4c0 legacy=0x7ff54415ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:07.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.656+0000 7ff542000640 1 -- 172.21.15.86:0/3740246914 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5440ae9f0 con 0x7ff5440af7b0 2024-04-20T21:20:07.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.657+0000 7ff542000640 1 --2- 172.21.15.86:0/3740246914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5440af7b0 0x7ff5440adda0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7ff530009870 tx=0x7ff53002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a55013e7c37b101 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:07.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.657+0000 7ff541600640 1 -- 172.21.15.86:0/3740246914 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff530035020 con 0x7ff5440af7b0 2024-04-20T21:20:07.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.657+0000 7ff541600640 1 -- 172.21.15.86:0/3740246914 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff530031640 con 0x7ff5440af7b0 2024-04-20T21:20:07.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.657+0000 7ff541600640 1 -- 172.21.15.86:0/3740246914 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53002d040 con 0x7ff5440af7b0 2024-04-20T21:20:07.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.658+0000 7ff543e00640 1 -- 172.21.15.86:0/3740246914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5440af7b0 msgr2=0x7ff5440adda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:07.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.658+0000 7ff543e00640 1 --2- 172.21.15.86:0/3740246914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5440af7b0 0x7ff5440adda0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7ff530009870 tx=0x7ff53002b8f0 comp rx=0 tx=0).stop 2024-04-20T21:20:07.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.658+0000 7ff543e00640 1 -- 172.21.15.86:0/3740246914 shutdown_connections 2024-04-20T21:20:07.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.658+0000 7ff543e00640 1 --2- 172.21.15.86:0/3740246914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5440af7b0 0x7ff5440adda0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:07.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.658+0000 7ff543e00640 1 --2- 172.21.15.86:0/3740246914 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff54415c010 0x7ff54415c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:07.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.658+0000 7ff543e00640 1 -- 172.21.15.86:0/3740246914 >> 172.21.15.86:0/3740246914 conn(0x7ff5440d0170 msgr2=0x7ff5440d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:07.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.658+0000 7ff543e00640 1 -- 172.21.15.86:0/3740246914 shutdown_connections 2024-04-20T21:20:07.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.658+0000 7ff543e00640 1 -- 172.21.15.86:0/3740246914 wait complete. 2024-04-20T21:20:07.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.660+0000 7ff543e00640 1 Processor -- start 2024-04-20T21:20:07.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.660+0000 7ff543e00640 1 -- start start 2024-04-20T21:20:07.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.661+0000 7ff543e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff54406f4c0 0x7ff5441f3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:07.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.661+0000 7ff543e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5440af7b0 0x7ff54414a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:07.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.661+0000 7ff543e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff54415c010 0x7ff544159d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:07.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.661+0000 7ff543e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff5440ae2e0 con 0x7ff54406f4c0 2024-04-20T21:20:07.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.661+0000 7ff543e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff5440aefb0 con 0x7ff5440af7b0 2024-04-20T21:20:07.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.661+0000 7ff543e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff5440ae460 con 0x7ff54415c010 2024-04-20T21:20:07.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.661+0000 7ff542a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff54406f4c0 0x7ff5441f3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:07.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.661+0000 7ff542a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff54406f4c0 0x7ff5441f3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45558/0 (socket says 172.21.15.86:45558) 2024-04-20T21:20:07.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.661+0000 7ff542a00640 1 -- 172.21.15.86:0/2165343848 learned_addr learned my addr 172.21.15.86:0/2165343848 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:07.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.662+0000 7ff542000640 1 --2- 172.21.15.86:0/2165343848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5440af7b0 0x7ff54414a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:07.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.662+0000 7ff542a00640 1 -- 172.21.15.86:0/2165343848 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff54415c010 msgr2=0x7ff544159d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:07.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.662+0000 7ff543400640 1 --2- 172.21.15.86:0/2165343848 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff54415c010 0x7ff544159d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:07.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.662+0000 7ff542a00640 1 --2- 172.21.15.86:0/2165343848 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff54415c010 0x7ff544159d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:07.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.662+0000 7ff542a00640 1 -- 172.21.15.86:0/2165343848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5440af7b0 msgr2=0x7ff54414a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:07.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.662+0000 7ff542a00640 1 --2- 172.21.15.86:0/2165343848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5440af7b0 0x7ff54414a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:07.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.662+0000 7ff542a00640 1 -- 172.21.15.86:0/2165343848 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff54414b0b0 con 0x7ff54406f4c0 2024-04-20T21:20:07.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.662+0000 7ff542000640 1 --2- 172.21.15.86:0/2165343848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5440af7b0 0x7ff54414a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:20:07.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.662+0000 7ff542a00640 1 --2- 172.21.15.86:0/2165343848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff54406f4c0 0x7ff5441f3fb0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff53400d5f0 tx=0x7ff53400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:07.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.662+0000 7ff53b400640 1 -- 172.21.15.86:0/2165343848 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5340125f0 con 0x7ff54406f4c0 2024-04-20T21:20:07.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.662+0000 7ff53b400640 1 -- 172.21.15.86:0/2165343848 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff534019070 con 0x7ff54406f4c0 2024-04-20T21:20:07.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.662+0000 7ff543e00640 1 -- 172.21.15.86:0/2165343848 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff54414b390 con 0x7ff54406f4c0 2024-04-20T21:20:07.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.662+0000 7ff53b400640 1 -- 172.21.15.86:0/2165343848 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5340129d0 con 0x7ff54406f4c0 2024-04-20T21:20:07.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.663+0000 7ff543e00640 1 -- 172.21.15.86:0/2165343848 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff54414bc30 con 0x7ff54406f4c0 2024-04-20T21:20:07.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.664+0000 7ff543e00640 1 -- 172.21.15.86:0/2165343848 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff54414b560 con 0x7ff54406f4c0 2024-04-20T21:20:07.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.665+0000 7ff53b400640 1 -- 172.21.15.86:0/2165343848 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff534012b70 con 0x7ff54406f4c0 2024-04-20T21:20:07.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.666+0000 7ff53b400640 1 --2- 172.21.15.86:0/2165343848 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff52406c8f0 0x7ff52406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:07.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.666+0000 7ff53b400640 1 -- 172.21.15.86:0/2165343848 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7ff53408bff0 con 0x7ff54406f4c0 2024-04-20T21:20:07.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.666+0000 7ff542000640 1 --2- 172.21.15.86:0/2165343848 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff52406c8f0 0x7ff52406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:07.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.667+0000 7ff542000640 1 --2- 172.21.15.86:0/2165343848 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff52406c8f0 0x7ff52406edb0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff530036000 tx=0x7ff530008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:07.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.671+0000 7ff53b400640 1 -- 172.21.15.86:0/2165343848 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff53405a380 con 0x7ff54406f4c0 2024-04-20T21:20:07.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.869+0000 7ff543e00640 1 -- 172.21.15.86:0/2165343848 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff544073bb0 con 0x7ff54406f4c0 2024-04-20T21:20:07.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.871+0000 7ff53b400640 1 -- 172.21.15.86:0/2165343848 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16613 (secure 0 0 0) 0x7ff544073bb0 con 0x7ff54406f4c0 2024-04-20T21:20:07.870 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:20:07.871 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":45,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:20:04.934190+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:20:01.307757+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-20T21:20:02.359347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:20:07.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.874+0000 7ff543e00640 1 -- 172.21.15.86:0/2165343848 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff52406c8f0 msgr2=0x7ff52406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:07.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.874+0000 7ff543e00640 1 --2- 172.21.15.86:0/2165343848 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff52406c8f0 0x7ff52406edb0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff530036000 tx=0x7ff530008030 comp rx=0 tx=0).stop 2024-04-20T21:20:07.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.874+0000 7ff543e00640 1 -- 172.21.15.86:0/2165343848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff54406f4c0 msgr2=0x7ff5441f3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:07.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.874+0000 7ff543e00640 1 --2- 172.21.15.86:0/2165343848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff54406f4c0 0x7ff5441f3fb0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff53400d5f0 tx=0x7ff53400dab0 comp rx=0 tx=0).stop 2024-04-20T21:20:07.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.874+0000 7ff543e00640 1 -- 172.21.15.86:0/2165343848 shutdown_connections 2024-04-20T21:20:07.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.874+0000 7ff543e00640 1 --2- 172.21.15.86:0/2165343848 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff52406c8f0 0x7ff52406edb0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:07.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.874+0000 7ff543e00640 1 --2- 172.21.15.86:0/2165343848 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff54415c010 0x7ff544159d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:07.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.874+0000 7ff543e00640 1 --2- 172.21.15.86:0/2165343848 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5440af7b0 0x7ff54414a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:07.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.874+0000 7ff543e00640 1 --2- 172.21.15.86:0/2165343848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff54406f4c0 0x7ff5441f3fb0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:07.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.874+0000 7ff543e00640 1 -- 172.21.15.86:0/2165343848 >> 172.21.15.86:0/2165343848 conn(0x7ff5440d0170 msgr2=0x7ff5440b0660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:07.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.874+0000 7ff543e00640 1 -- 172.21.15.86:0/2165343848 shutdown_connections 2024-04-20T21:20:07.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:07.874+0000 7ff543e00640 1 -- 172.21.15.86:0/2165343848 wait complete. 2024-04-20T21:20:07.892 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-20T21:20:07.892 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T21:20:08.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.094+0000 7feefd600640 1 Processor -- start 2024-04-20T21:20:08.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.094+0000 7feefd600640 1 -- start start 2024-04-20T21:20:08.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.096+0000 7feefd600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feef80af8d0 0x7feef8141f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:08.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.096+0000 7feefd600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7feef8162c40 0x7feef80b3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:08.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.096+0000 7feefd600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feef8071c10 con 0x7feef8148050 2024-04-20T21:20:08.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.096+0000 7feefd600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7feef8142490 con 0x7feef8162c40 2024-04-20T21:20:08.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.096+0000 7feefd600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7feef815eaf0 con 0x7feef80af8d0 2024-04-20T21:20:08.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.096+0000 7feef7400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7feef8162c40 0x7feef80b3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:08.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.096+0000 7feef7e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feef80af8d0 0x7feef8141f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:08.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.097+0000 7feef7400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7feef8162c40 0x7feef80b3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.86:45414/0 (socket says 172.21.15.86:45414) 2024-04-20T21:20:08.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.097+0000 7feef7400640 1 -- 172.21.15.86:0/3825933906 learned_addr learned my addr 172.21.15.86:0/3825933906 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:08.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.097+0000 7feef7e00640 1 -- 172.21.15.86:0/3825933906 >> v2:172.21.15.144:3300/0 conn(0x7feef8162c40 msgr2=0x7feef80b3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:08.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.097+0000 7feef7e00640 1 --2- 172.21.15.86:0/3825933906 >> v2:172.21.15.144:3300/0 conn(0x7feef8162c40 0x7feef80b3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:08.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.097+0000 7feef7e00640 1 -- 172.21.15.86:0/3825933906 >> v1:172.21.15.144:6789/0 conn(0x7feef8148050 legacy=0x7feef8072990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:08.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.097+0000 7feef7e00640 1 -- 172.21.15.86:0/3825933906 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feef8071eb0 con 0x7feef80af8d0 2024-04-20T21:20:08.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.099+0000 7feef7e00640 1 --2- 172.21.15.86:0/3825933906 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feef80af8d0 0x7feef8141f50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7feee00097a0 tx=0x7feee002b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=beb028e2c3dacb68 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:08.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.099+0000 7feef6a00640 1 -- 172.21.15.86:0/3825933906 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee00315a0 con 0x7feef80af8d0 2024-04-20T21:20:08.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.099+0000 7feef6a00640 1 -- 172.21.15.86:0/3825933906 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feee0037070 con 0x7feef80af8d0 2024-04-20T21:20:08.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.099+0000 7feef6a00640 1 -- 172.21.15.86:0/3825933906 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee0031960 con 0x7feef80af8d0 2024-04-20T21:20:08.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.100+0000 7feefd600640 1 -- 172.21.15.86:0/3825933906 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feef80af8d0 msgr2=0x7feef8141f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:08.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.100+0000 7feefd600640 1 --2- 172.21.15.86:0/3825933906 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feef80af8d0 0x7feef8141f50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7feee00097a0 tx=0x7feee002b840 comp rx=0 tx=0).stop 2024-04-20T21:20:08.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.100+0000 7feefd600640 1 -- 172.21.15.86:0/3825933906 shutdown_connections 2024-04-20T21:20:08.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.100+0000 7feefd600640 1 --2- 172.21.15.86:0/3825933906 >> v2:172.21.15.144:3300/0 conn(0x7feef8162c40 0x7feef80b3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:08.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.100+0000 7feefd600640 1 --2- 172.21.15.86:0/3825933906 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feef80af8d0 0x7feef8141f50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:08.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.100+0000 7feefd600640 1 -- 172.21.15.86:0/3825933906 >> 172.21.15.86:0/3825933906 conn(0x7feef80d0170 msgr2=0x7feef80d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:08.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.100+0000 7feefd600640 1 -- 172.21.15.86:0/3825933906 shutdown_connections 2024-04-20T21:20:08.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.100+0000 7feefd600640 1 -- 172.21.15.86:0/3825933906 wait complete. 2024-04-20T21:20:08.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.102+0000 7feefd600640 1 Processor -- start 2024-04-20T21:20:08.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.102+0000 7feefd600640 1 -- start start 2024-04-20T21:20:08.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.103+0000 7feefd600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feef80af8d0 0x7feef80992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:08.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.103+0000 7feefd600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feef8148050 0x7feef8146780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:08.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.103+0000 7feefd600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feef8162c40 0x7feef80b01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:08.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.103+0000 7feefd600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feef8163100 con 0x7feef8148050 2024-04-20T21:20:08.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.103+0000 7feefd600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7feef8072470 con 0x7feef8162c40 2024-04-20T21:20:08.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.103+0000 7feefd600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7feef8147a00 con 0x7feef80af8d0 2024-04-20T21:20:08.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.103+0000 7feef7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feef8148050 0x7feef8146780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:08.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.103+0000 7feef7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feef8148050 0x7feef8146780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45566/0 (socket says 172.21.15.86:45566) 2024-04-20T21:20:08.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.103+0000 7feef7400640 1 -- 172.21.15.86:0/3688540850 learned_addr learned my addr 172.21.15.86:0/3688540850 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:08.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.103+0000 7feef7400640 1 -- 172.21.15.86:0/3688540850 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feef80af8d0 msgr2=0x7feef80992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:08.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.103+0000 7feef7400640 1 --2- 172.21.15.86:0/3688540850 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feef80af8d0 0x7feef80992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:08.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.103+0000 7feef7400640 1 -- 172.21.15.86:0/3688540850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feef8162c40 msgr2=0x7feef80b01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:08.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.103+0000 7feef7400640 1 --2- 172.21.15.86:0/3688540850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feef8162c40 0x7feef80b01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:08.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.103+0000 7feef7400640 1 -- 172.21.15.86:0/3688540850 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feef80997f0 con 0x7feef8148050 2024-04-20T21:20:08.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.104+0000 7feef7400640 1 --2- 172.21.15.86:0/3688540850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feef8148050 0x7feef8146780 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7feee800efc0 tx=0x7feee800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:08.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.104+0000 7feeefe00640 1 -- 172.21.15.86:0/3688540850 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee800cda0 con 0x7feef8148050 2024-04-20T21:20:08.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.104+0000 7feeefe00640 1 -- 172.21.15.86:0/3688540850 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feee8019070 con 0x7feef8148050 2024-04-20T21:20:08.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.104+0000 7feefd600640 1 -- 172.21.15.86:0/3688540850 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feef815f180 con 0x7feef8148050 2024-04-20T21:20:08.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.104+0000 7feeefe00640 1 -- 172.21.15.86:0/3688540850 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee80146a0 con 0x7feef8148050 2024-04-20T21:20:08.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.104+0000 7feefd600640 1 -- 172.21.15.86:0/3688540850 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feef815f740 con 0x7feef8148050 2024-04-20T21:20:08.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.105+0000 7feefd600640 1 -- 172.21.15.86:0/3688540850 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feef8147690 con 0x7feef8148050 2024-04-20T21:20:08.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.106+0000 7feeefe00640 1 -- 172.21.15.86:0/3688540850 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feee8005d10 con 0x7feef8148050 2024-04-20T21:20:08.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.108+0000 7feeefe00640 1 --2- 172.21.15.86:0/3688540850 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7feed406c8f0 0x7feed406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:08.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.108+0000 7feeefe00640 1 -- 172.21.15.86:0/3688540850 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7feee808bff0 con 0x7feef8148050 2024-04-20T21:20:08.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.108+0000 7feef7e00640 1 --2- 172.21.15.86:0/3688540850 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7feed406c8f0 0x7feed406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:08.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.109+0000 7feef7e00640 1 --2- 172.21.15.86:0/3688540850 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7feed406c8f0 0x7feed406edb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7feee0036000 tx=0x7feee0030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:08.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.112+0000 7feeefe00640 1 -- 172.21.15.86:0/3688540850 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7feef8147690 con 0x7feef8148050 2024-04-20T21:20:08.282 INFO:tasks.ceph.mds.a.smithi086.stderr:2024-04-20T21:20:08.282+0000 7f1e0fc00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T21:20:08.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.294+0000 7feefd600640 1 -- 172.21.15.86:0/3688540850 --> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7feef81624e0 con 0x7feed406c8f0 2024-04-20T21:20:08.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.302+0000 7feeefe00640 1 -- 172.21.15.86:0/3688540850 <== mgr.4107 v2:172.21.15.86:6832/302045240 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251931 (secure 0 0 0) 0x7feef81624e0 con 0x7feed406c8f0 2024-04-20T21:20:08.304 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:20:08.306 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-20T21:20:08.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.307+0000 7feefd600640 1 -- 172.21.15.86:0/3688540850 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7feed406c8f0 msgr2=0x7feed406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:08.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.307+0000 7feefd600640 1 --2- 172.21.15.86:0/3688540850 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7feed406c8f0 0x7feed406edb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7feee0036000 tx=0x7feee0030000 comp rx=0 tx=0).stop 2024-04-20T21:20:08.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.307+0000 7feefd600640 1 -- 172.21.15.86:0/3688540850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feef8148050 msgr2=0x7feef8146780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:08.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.307+0000 7feefd600640 1 --2- 172.21.15.86:0/3688540850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feef8148050 0x7feef8146780 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7feee800efc0 tx=0x7feee800c490 comp rx=0 tx=0).stop 2024-04-20T21:20:08.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.307+0000 7feefd600640 1 -- 172.21.15.86:0/3688540850 shutdown_connections 2024-04-20T21:20:08.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.307+0000 7feefd600640 1 --2- 172.21.15.86:0/3688540850 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7feed406c8f0 0x7feed406edb0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:08.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.307+0000 7feefd600640 1 --2- 172.21.15.86:0/3688540850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feef8162c40 0x7feef80b01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:08.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.307+0000 7feefd600640 1 --2- 172.21.15.86:0/3688540850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feef8148050 0x7feef8146780 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:08.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.307+0000 7feefd600640 1 --2- 172.21.15.86:0/3688540850 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feef80af8d0 0x7feef80992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:08.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.307+0000 7feefd600640 1 -- 172.21.15.86:0/3688540850 >> 172.21.15.86:0/3688540850 conn(0x7feef80d0170 msgr2=0x7feef80b3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:08.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.307+0000 7feefd600640 1 -- 172.21.15.86:0/3688540850 shutdown_connections 2024-04-20T21:20:08.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.307+0000 7feefd600640 1 -- 172.21.15.86:0/3688540850 wait complete. 2024-04-20T21:20:08.323 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2024-04-20T21:20:06.671815+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":114,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":225624,"kb_used_data":5280,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749096,"statfs":{"total":773094113280,"available":772863074304,"internally_reserved":0,"allocated":5406720,"data_stored":2868676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.776399"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.957076+0000","last_change":"2024-04-20T21:20:03.953844+0000","last_active":"2024-04-20T21:20:03.957076+0000","last_peered":"2024-04-20T21:20:03.957076+0000","last_clean":"2024-04-20T21:20:03.957076+0000","last_became_active":"2024-04-20T21:20:03.952822+0000","last_became_peered":"2024-04-20T21:20:03.952822+0000","last_unstale":"2024-04-20T21:20:03.957076+0000","last_undegraded":"2024-04-20T21:20:03.957076+0000","last_fullsized":"2024-04-20T21:20:03.957076+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:27:08.974286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.966513+0000","last_change":"2024-04-20T21:20:03.963223+0000","last_active":"2024-04-20T21:20:03.966513+0000","last_peered":"2024-04-20T21:20:03.966513+0000","last_clean":"2024-04-20T21:20:03.966513+0000","last_became_active":"2024-04-20T21:20:03.962841+0000","last_became_peered":"2024-04-20T21:20:03.962841+0000","last_unstale":"2024-04-20T21:20:03.966513+0000","last_undegraded":"2024-04-20T21:20:03.966513+0000","last_fullsized":"2024-04-20T21:20:03.966513+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:40:06.872797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.957056+0000","last_change":"2024-04-20T21:20:03.934734+0000","last_active":"2024-04-20T21:20:04.957056+0000","last_peered":"2024-04-20T21:20:04.957056+0000","last_clean":"2024-04-20T21:20:04.957056+0000","last_became_active":"2024-04-20T21:20:03.934290+0000","last_became_peered":"2024-04-20T21:20:03.934290+0000","last_unstale":"2024-04-20T21:20:04.957056+0000","last_undegraded":"2024-04-20T21:20:04.957056+0000","last_fullsized":"2024-04-20T21:20:04.957056+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:32:53.349182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.962140+0000","last_change":"2024-04-20T21:20:03.959935+0000","last_active":"2024-04-20T21:20:03.962140+0000","last_peered":"2024-04-20T21:20:03.962140+0000","last_clean":"2024-04-20T21:20:03.962140+0000","last_became_active":"2024-04-20T21:20:03.958987+0000","last_became_peered":"2024-04-20T21:20:03.958987+0000","last_unstale":"2024-04-20T21:20:03.962140+0000","last_undegraded":"2024-04-20T21:20:03.962140+0000","last_fullsized":"2024-04-20T21:20:03.962140+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:47:26.349120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.963864+0000","last_change":"2024-04-20T21:20:03.959137+0000","last_active":"2024-04-20T21:20:03.963864+0000","last_peered":"2024-04-20T21:20:03.963864+0000","last_clean":"2024-04-20T21:20:03.963864+0000","last_became_active":"2024-04-20T21:20:03.958830+0000","last_became_peered":"2024-04-20T21:20:03.958830+0000","last_unstale":"2024-04-20T21:20:03.963864+0000","last_undegraded":"2024-04-20T21:20:03.963864+0000","last_fullsized":"2024-04-20T21:20:03.963864+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:56:29.421282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.966803+0000","last_change":"2024-04-20T21:20:03.950648+0000","last_active":"2024-04-20T21:20:04.966803+0000","last_peered":"2024-04-20T21:20:04.966803+0000","last_clean":"2024-04-20T21:20:04.966803+0000","last_became_active":"2024-04-20T21:20:03.950288+0000","last_became_peered":"2024-04-20T21:20:03.950288+0000","last_unstale":"2024-04-20T21:20:04.966803+0000","last_undegraded":"2024-04-20T21:20:04.966803+0000","last_fullsized":"2024-04-20T21:20:04.966803+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:11:08.517771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.955972+0000","last_change":"2024-04-20T21:20:03.949963+0000","last_active":"2024-04-20T21:20:03.955972+0000","last_peered":"2024-04-20T21:20:03.955972+0000","last_clean":"2024-04-20T21:20:03.955972+0000","last_became_active":"2024-04-20T21:20:03.949607+0000","last_became_peered":"2024-04-20T21:20:03.949607+0000","last_unstale":"2024-04-20T21:20:03.955972+0000","last_undegraded":"2024-04-20T21:20:03.955972+0000","last_fullsized":"2024-04-20T21:20:03.955972+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:56:29.421282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.963869+0000","last_change":"2024-04-20T21:20:03.960633+0000","last_active":"2024-04-20T21:20:03.963869+0000","last_peered":"2024-04-20T21:20:03.963869+0000","last_clean":"2024-04-20T21:20:03.963869+0000","last_became_active":"2024-04-20T21:20:03.960018+0000","last_became_peered":"2024-04-20T21:20:03.960018+0000","last_unstale":"2024-04-20T21:20:03.963869+0000","last_undegraded":"2024-04-20T21:20:03.963869+0000","last_fullsized":"2024-04-20T21:20:03.963869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:10:13.986163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.964377+0000","last_change":"2024-04-20T21:20:03.962654+0000","last_active":"2024-04-20T21:20:03.964377+0000","last_peered":"2024-04-20T21:20:03.964377+0000","last_clean":"2024-04-20T21:20:03.964377+0000","last_became_active":"2024-04-20T21:20:03.962366+0000","last_became_peered":"2024-04-20T21:20:03.962366+0000","last_unstale":"2024-04-20T21:20:03.964377+0000","last_undegraded":"2024-04-20T21:20:03.964377+0000","last_fullsized":"2024-04-20T21:20:03.964377+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:57:58.076361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.964257+0000","last_change":"2024-04-20T21:20:03.961204+0000","last_active":"2024-04-20T21:20:03.964257+0000","last_peered":"2024-04-20T21:20:03.964257+0000","last_clean":"2024-04-20T21:20:03.964257+0000","last_became_active":"2024-04-20T21:20:03.960258+0000","last_became_peered":"2024-04-20T21:20:03.960258+0000","last_unstale":"2024-04-20T21:20:03.964257+0000","last_undegraded":"2024-04-20T21:20:03.964257+0000","last_fullsized":"2024-04-20T21:20:03.964257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:06:28.924456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.963894+0000","last_change":"2024-04-20T21:20:03.960942+0000","last_active":"2024-04-20T21:20:03.963894+0000","last_peered":"2024-04-20T21:20:03.963894+0000","last_clean":"2024-04-20T21:20:03.963894+0000","last_became_active":"2024-04-20T21:20:03.960584+0000","last_became_peered":"2024-04-20T21:20:03.960584+0000","last_unstale":"2024-04-20T21:20:03.963894+0000","last_undegraded":"2024-04-20T21:20:03.963894+0000","last_fullsized":"2024-04-20T21:20:03.963894+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:40:01.156667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.949162+0000","last_change":"2024-04-20T21:20:03.944336+0000","last_active":"2024-04-20T21:20:03.949162+0000","last_peered":"2024-04-20T21:20:03.949162+0000","last_clean":"2024-04-20T21:20:03.949162+0000","last_became_active":"2024-04-20T21:20:03.943825+0000","last_became_peered":"2024-04-20T21:20:03.943825+0000","last_unstale":"2024-04-20T21:20:03.949162+0000","last_undegraded":"2024-04-20T21:20:03.949162+0000","last_fullsized":"2024-04-20T21:20:03.949162+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:57:58.076361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.952927+0000","last_change":"2024-04-20T21:20:03.949248+0000","last_active":"2024-04-20T21:20:04.952927+0000","last_peered":"2024-04-20T21:20:04.952927+0000","last_clean":"2024-04-20T21:20:04.952927+0000","last_became_active":"2024-04-20T21:20:03.948888+0000","last_became_peered":"2024-04-20T21:20:03.948888+0000","last_unstale":"2024-04-20T21:20:04.952927+0000","last_undegraded":"2024-04-20T21:20:04.952927+0000","last_fullsized":"2024-04-20T21:20:04.952927+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:31:29.580997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.960620+0000","last_change":"2024-04-20T21:20:03.951776+0000","last_active":"2024-04-20T21:20:03.960620+0000","last_peered":"2024-04-20T21:20:03.960620+0000","last_clean":"2024-04-20T21:20:03.960620+0000","last_became_active":"2024-04-20T21:20:03.951426+0000","last_became_peered":"2024-04-20T21:20:03.951426+0000","last_unstale":"2024-04-20T21:20:03.960620+0000","last_undegraded":"2024-04-20T21:20:03.960620+0000","last_fullsized":"2024-04-20T21:20:03.960620+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:42:14.328950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.965560+0000","last_change":"2024-04-20T21:20:03.941753+0000","last_active":"2024-04-20T21:20:04.965560+0000","last_peered":"2024-04-20T21:20:04.965560+0000","last_clean":"2024-04-20T21:20:04.965560+0000","last_became_active":"2024-04-20T21:20:03.937663+0000","last_became_peered":"2024-04-20T21:20:03.937663+0000","last_unstale":"2024-04-20T21:20:04.965560+0000","last_undegraded":"2024-04-20T21:20:04.965560+0000","last_fullsized":"2024-04-20T21:20:04.965560+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:31:02.145378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.957536+0000","last_change":"2024-04-20T21:20:03.957153+0000","last_active":"2024-04-20T21:20:04.957536+0000","last_peered":"2024-04-20T21:20:04.957536+0000","last_clean":"2024-04-20T21:20:04.957536+0000","last_became_active":"2024-04-20T21:20:03.956660+0000","last_became_peered":"2024-04-20T21:20:03.956660+0000","last_unstale":"2024-04-20T21:20:04.957536+0000","last_undegraded":"2024-04-20T21:20:04.957536+0000","last_fullsized":"2024-04-20T21:20:04.957536+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:00:13.844466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.957548+0000","last_change":"2024-04-20T21:20:03.940211+0000","last_active":"2024-04-20T21:20:04.957548+0000","last_peered":"2024-04-20T21:20:04.957548+0000","last_clean":"2024-04-20T21:20:04.957548+0000","last_became_active":"2024-04-20T21:20:03.939697+0000","last_became_peered":"2024-04-20T21:20:03.939697+0000","last_unstale":"2024-04-20T21:20:04.957548+0000","last_undegraded":"2024-04-20T21:20:04.957548+0000","last_fullsized":"2024-04-20T21:20:04.957548+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:56:29.421282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.957584+0000","last_change":"2024-04-20T21:20:03.962713+0000","last_active":"2024-04-20T21:20:04.957584+0000","last_peered":"2024-04-20T21:20:04.957584+0000","last_clean":"2024-04-20T21:20:04.957584+0000","last_became_active":"2024-04-20T21:20:03.962360+0000","last_became_peered":"2024-04-20T21:20:03.962360+0000","last_unstale":"2024-04-20T21:20:04.957584+0000","last_undegraded":"2024-04-20T21:20:04.957584+0000","last_fullsized":"2024-04-20T21:20:04.957584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:40:06.872797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.959566+0000","last_change":"2024-04-20T21:20:03.957492+0000","last_active":"2024-04-20T21:20:04.959566+0000","last_peered":"2024-04-20T21:20:04.959566+0000","last_clean":"2024-04-20T21:20:04.959566+0000","last_became_active":"2024-04-20T21:20:03.957216+0000","last_became_peered":"2024-04-20T21:20:03.957216+0000","last_unstale":"2024-04-20T21:20:04.959566+0000","last_undegraded":"2024-04-20T21:20:04.959566+0000","last_fullsized":"2024-04-20T21:20:04.959566+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:10:13.986163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.953672+0000","last_change":"2024-04-20T21:20:03.957221+0000","last_active":"2024-04-20T21:20:04.953672+0000","last_peered":"2024-04-20T21:20:04.953672+0000","last_clean":"2024-04-20T21:20:04.953672+0000","last_became_active":"2024-04-20T21:20:03.956891+0000","last_became_peered":"2024-04-20T21:20:03.956891+0000","last_unstale":"2024-04-20T21:20:04.953672+0000","last_undegraded":"2024-04-20T21:20:04.953672+0000","last_fullsized":"2024-04-20T21:20:04.953672+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:14:19.125873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.965745+0000","last_change":"2024-04-20T21:20:03.940435+0000","last_active":"2024-04-20T21:20:04.965745+0000","last_peered":"2024-04-20T21:20:04.965745+0000","last_clean":"2024-04-20T21:20:04.965745+0000","last_became_active":"2024-04-20T21:20:03.940160+0000","last_became_peered":"2024-04-20T21:20:03.940160+0000","last_unstale":"2024-04-20T21:20:04.965745+0000","last_undegraded":"2024-04-20T21:20:04.965745+0000","last_fullsized":"2024-04-20T21:20:04.965745+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:41:38.773748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.962569+0000","last_change":"2024-04-20T21:20:03.956429+0000","last_active":"2024-04-20T21:20:03.962569+0000","last_peered":"2024-04-20T21:20:03.962569+0000","last_clean":"2024-04-20T21:20:03.962569+0000","last_became_active":"2024-04-20T21:20:03.956167+0000","last_became_peered":"2024-04-20T21:20:03.956167+0000","last_unstale":"2024-04-20T21:20:03.962569+0000","last_undegraded":"2024-04-20T21:20:03.962569+0000","last_fullsized":"2024-04-20T21:20:03.962569+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:07:48.221573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.958021+0000","last_change":"2024-04-20T21:20:03.949152+0000","last_active":"2024-04-20T21:20:04.958021+0000","last_peered":"2024-04-20T21:20:04.958021+0000","last_clean":"2024-04-20T21:20:04.958021+0000","last_became_active":"2024-04-20T21:20:03.948736+0000","last_became_peered":"2024-04-20T21:20:03.948736+0000","last_unstale":"2024-04-20T21:20:04.958021+0000","last_undegraded":"2024-04-20T21:20:04.958021+0000","last_fullsized":"2024-04-20T21:20:04.958021+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:20:06.592684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.957535+0000","last_change":"2024-04-20T21:20:03.951534+0000","last_active":"2024-04-20T21:20:04.957535+0000","last_peered":"2024-04-20T21:20:04.957535+0000","last_clean":"2024-04-20T21:20:04.957535+0000","last_became_active":"2024-04-20T21:20:03.951227+0000","last_became_peered":"2024-04-20T21:20:03.951227+0000","last_unstale":"2024-04-20T21:20:04.957535+0000","last_undegraded":"2024-04-20T21:20:04.957535+0000","last_fullsized":"2024-04-20T21:20:04.957535+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:31:54.378016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.959322+0000","last_change":"2024-04-20T21:20:03.947006+0000","last_active":"2024-04-20T21:20:04.959322+0000","last_peered":"2024-04-20T21:20:04.959322+0000","last_clean":"2024-04-20T21:20:04.959322+0000","last_became_active":"2024-04-20T21:20:03.946596+0000","last_became_peered":"2024-04-20T21:20:03.946596+0000","last_unstale":"2024-04-20T21:20:04.959322+0000","last_undegraded":"2024-04-20T21:20:04.959322+0000","last_fullsized":"2024-04-20T21:20:04.959322+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:03:41.090616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.953365+0000","last_change":"2024-04-20T21:20:03.936009+0000","last_active":"2024-04-20T21:20:04.953365+0000","last_peered":"2024-04-20T21:20:04.953365+0000","last_clean":"2024-04-20T21:20:04.953365+0000","last_became_active":"2024-04-20T21:20:03.935496+0000","last_became_peered":"2024-04-20T21:20:03.935496+0000","last_unstale":"2024-04-20T21:20:04.953365+0000","last_undegraded":"2024-04-20T21:20:04.953365+0000","last_fullsized":"2024-04-20T21:20:04.953365+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:22:18.347547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.957099+0000","last_change":"2024-04-20T21:20:03.943653+0000","last_active":"2024-04-20T21:20:04.957099+0000","last_peered":"2024-04-20T21:20:04.957099+0000","last_clean":"2024-04-20T21:20:04.957099+0000","last_became_active":"2024-04-20T21:20:03.943348+0000","last_became_peered":"2024-04-20T21:20:03.943348+0000","last_unstale":"2024-04-20T21:20:04.957099+0000","last_undegraded":"2024-04-20T21:20:04.957099+0000","last_fullsized":"2024-04-20T21:20:04.957099+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:18:01.186077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.944891+0000","last_change":"2024-04-20T21:20:03.942646+0000","last_active":"2024-04-20T21:20:03.944891+0000","last_peered":"2024-04-20T21:20:03.944891+0000","last_clean":"2024-04-20T21:20:03.944891+0000","last_became_active":"2024-04-20T21:20:03.941871+0000","last_became_peered":"2024-04-20T21:20:03.941871+0000","last_unstale":"2024-04-20T21:20:03.944891+0000","last_undegraded":"2024-04-20T21:20:03.944891+0000","last_fullsized":"2024-04-20T21:20:03.944891+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:08:22.030102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.958769+0000","last_change":"2024-04-20T21:20:03.947375+0000","last_active":"2024-04-20T21:20:04.958769+0000","last_peered":"2024-04-20T21:20:04.958769+0000","last_clean":"2024-04-20T21:20:04.958769+0000","last_became_active":"2024-04-20T21:20:03.946107+0000","last_became_peered":"2024-04-20T21:20:03.946107+0000","last_unstale":"2024-04-20T21:20:04.958769+0000","last_undegraded":"2024-04-20T21:20:04.958769+0000","last_fullsized":"2024-04-20T21:20:04.958769+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:39:45.020282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.956167+0000","last_change":"2024-04-20T21:20:03.952087+0000","last_active":"2024-04-20T21:20:04.956167+0000","last_peered":"2024-04-20T21:20:04.956167+0000","last_clean":"2024-04-20T21:20:04.956167+0000","last_became_active":"2024-04-20T21:20:03.951745+0000","last_became_peered":"2024-04-20T21:20:03.951745+0000","last_unstale":"2024-04-20T21:20:04.956167+0000","last_undegraded":"2024-04-20T21:20:04.956167+0000","last_fullsized":"2024-04-20T21:20:04.956167+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:08:22.030102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.966617+0000","last_change":"2024-04-20T21:20:03.942401+0000","last_active":"2024-04-20T21:20:04.966617+0000","last_peered":"2024-04-20T21:20:04.966617+0000","last_clean":"2024-04-20T21:20:04.966617+0000","last_became_active":"2024-04-20T21:20:03.942083+0000","last_became_peered":"2024-04-20T21:20:03.942083+0000","last_unstale":"2024-04-20T21:20:04.966617+0000","last_undegraded":"2024-04-20T21:20:04.966617+0000","last_fullsized":"2024-04-20T21:20:04.966617+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:46:20.994921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.952682+0000","last_change":"2024-04-20T21:20:03.937678+0000","last_active":"2024-04-20T21:20:04.952682+0000","last_peered":"2024-04-20T21:20:04.952682+0000","last_clean":"2024-04-20T21:20:04.952682+0000","last_became_active":"2024-04-20T21:20:03.936838+0000","last_became_peered":"2024-04-20T21:20:03.936838+0000","last_unstale":"2024-04-20T21:20:04.952682+0000","last_undegraded":"2024-04-20T21:20:04.952682+0000","last_fullsized":"2024-04-20T21:20:04.952682+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:28:04.120963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:05.232158+0000","last_change":"2024-04-20T21:20:03.943607+0000","last_active":"2024-04-20T21:20:05.232158+0000","last_peered":"2024-04-20T21:20:05.232158+0000","last_clean":"2024-04-20T21:20:05.232158+0000","last_became_active":"2024-04-20T21:20:03.939432+0000","last_became_peered":"2024-04-20T21:20:03.939432+0000","last_unstale":"2024-04-20T21:20:05.232158+0000","last_undegraded":"2024-04-20T21:20:05.232158+0000","last_fullsized":"2024-04-20T21:20:05.232158+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:01:47.306517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"44'4","reported_seq":18,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.966652+0000","last_change":"2024-04-20T21:20:02.929219+0000","last_active":"2024-04-20T21:20:04.966652+0000","last_peered":"2024-04-20T21:20:04.966652+0000","last_clean":"2024-04-20T21:20:04.966652+0000","last_became_active":"2024-04-20T21:20:02.928868+0000","last_became_peered":"2024-04-20T21:20:02.928868+0000","last_unstale":"2024-04-20T21:20:04.966652+0000","last_undegraded":"2024-04-20T21:20:04.966652+0000","last_fullsized":"2024-04-20T21:20:04.966652+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:08:21.013641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.959520+0000","last_change":"2024-04-20T21:20:03.940579+0000","last_active":"2024-04-20T21:20:04.959520+0000","last_peered":"2024-04-20T21:20:04.959520+0000","last_clean":"2024-04-20T21:20:04.959520+0000","last_became_active":"2024-04-20T21:20:03.940123+0000","last_became_peered":"2024-04-20T21:20:03.940123+0000","last_unstale":"2024-04-20T21:20:04.959520+0000","last_undegraded":"2024-04-20T21:20:04.959520+0000","last_fullsized":"2024-04-20T21:20:04.959520+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:47:26.349120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.957950+0000","last_change":"2024-04-20T21:20:02.939856+0000","last_active":"2024-04-20T21:20:04.957950+0000","last_peered":"2024-04-20T21:20:04.957950+0000","last_clean":"2024-04-20T21:20:04.957950+0000","last_became_active":"2024-04-20T21:20:02.939491+0000","last_became_peered":"2024-04-20T21:20:02.939491+0000","last_unstale":"2024-04-20T21:20:04.957950+0000","last_undegraded":"2024-04-20T21:20:04.957950+0000","last_fullsized":"2024-04-20T21:20:04.957950+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:57:21.228630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.953239+0000","last_change":"2024-04-20T21:20:03.949355+0000","last_active":"2024-04-20T21:20:03.953239+0000","last_peered":"2024-04-20T21:20:03.953239+0000","last_clean":"2024-04-20T21:20:03.953239+0000","last_became_active":"2024-04-20T21:20:03.948932+0000","last_became_peered":"2024-04-20T21:20:03.948932+0000","last_unstale":"2024-04-20T21:20:03.953239+0000","last_undegraded":"2024-04-20T21:20:03.953239+0000","last_fullsized":"2024-04-20T21:20:03.953239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:18:01.186077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:05.232133+0000","last_change":"2024-04-20T21:20:02.941932+0000","last_active":"2024-04-20T21:20:05.232133+0000","last_peered":"2024-04-20T21:20:05.232133+0000","last_clean":"2024-04-20T21:20:05.232133+0000","last_became_active":"2024-04-20T21:20:02.941631+0000","last_became_peered":"2024-04-20T21:20:02.941631+0000","last_unstale":"2024-04-20T21:20:05.232133+0000","last_undegraded":"2024-04-20T21:20:05.232133+0000","last_fullsized":"2024-04-20T21:20:05.232133+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:31:53.361555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:05.232163+0000","last_change":"2024-04-20T21:20:03.937038+0000","last_active":"2024-04-20T21:20:05.232163+0000","last_peered":"2024-04-20T21:20:05.232163+0000","last_clean":"2024-04-20T21:20:05.232163+0000","last_became_active":"2024-04-20T21:20:03.936747+0000","last_became_peered":"2024-04-20T21:20:03.936747+0000","last_unstale":"2024-04-20T21:20:05.232163+0000","last_undegraded":"2024-04-20T21:20:05.232163+0000","last_fullsized":"2024-04-20T21:20:05.232163+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:47:26.349120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.975160+0000","last_change":"2024-04-20T21:20:02.949111+0000","last_active":"2024-04-20T21:20:03.975160+0000","last_peered":"2024-04-20T21:20:03.975160+0000","last_clean":"2024-04-20T21:20:03.975160+0000","last_became_active":"2024-04-20T21:20:02.948782+0000","last_became_peered":"2024-04-20T21:20:02.948782+0000","last_unstale":"2024-04-20T21:20:03.975160+0000","last_undegraded":"2024-04-20T21:20:03.975160+0000","last_fullsized":"2024-04-20T21:20:03.975160+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:57:57.059900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.956306+0000","last_change":"2024-04-20T21:20:03.937277+0000","last_active":"2024-04-20T21:20:04.956306+0000","last_peered":"2024-04-20T21:20:04.956306+0000","last_clean":"2024-04-20T21:20:04.956306+0000","last_became_active":"2024-04-20T21:20:03.936929+0000","last_became_peered":"2024-04-20T21:20:03.936929+0000","last_unstale":"2024-04-20T21:20:04.956306+0000","last_undegraded":"2024-04-20T21:20:04.956306+0000","last_fullsized":"2024-04-20T21:20:04.956306+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:57:58.076361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.965883+0000","last_change":"2024-04-20T21:20:02.939114+0000","last_active":"2024-04-20T21:20:04.965883+0000","last_peered":"2024-04-20T21:20:04.965883+0000","last_clean":"2024-04-20T21:20:04.965883+0000","last_became_active":"2024-04-20T21:20:02.938882+0000","last_became_peered":"2024-04-20T21:20:02.938882+0000","last_unstale":"2024-04-20T21:20:04.965883+0000","last_undegraded":"2024-04-20T21:20:04.965883+0000","last_fullsized":"2024-04-20T21:20:04.965883+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:56:28.404820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.962020+0000","last_change":"2024-04-20T21:20:03.957466+0000","last_active":"2024-04-20T21:20:03.962020+0000","last_peered":"2024-04-20T21:20:03.962020+0000","last_clean":"2024-04-20T21:20:03.962020+0000","last_became_active":"2024-04-20T21:20:03.956529+0000","last_became_peered":"2024-04-20T21:20:03.956529+0000","last_unstale":"2024-04-20T21:20:03.962020+0000","last_undegraded":"2024-04-20T21:20:03.962020+0000","last_fullsized":"2024-04-20T21:20:03.962020+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:03:41.090616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.956529+0000","last_change":"2024-04-20T21:20:02.945473+0000","last_active":"2024-04-20T21:20:04.956529+0000","last_peered":"2024-04-20T21:20:04.956529+0000","last_clean":"2024-04-20T21:20:04.956529+0000","last_became_active":"2024-04-20T21:20:02.945167+0000","last_became_peered":"2024-04-20T21:20:02.945167+0000","last_unstale":"2024-04-20T21:20:04.956529+0000","last_undegraded":"2024-04-20T21:20:04.956529+0000","last_fullsized":"2024-04-20T21:20:04.956529+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:31:53.361555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.964535+0000","last_change":"2024-04-20T21:20:03.961697+0000","last_active":"2024-04-20T21:20:03.964535+0000","last_peered":"2024-04-20T21:20:03.964535+0000","last_clean":"2024-04-20T21:20:03.964535+0000","last_became_active":"2024-04-20T21:20:03.958756+0000","last_became_peered":"2024-04-20T21:20:03.958756+0000","last_unstale":"2024-04-20T21:20:03.964535+0000","last_undegraded":"2024-04-20T21:20:03.964535+0000","last_fullsized":"2024-04-20T21:20:03.964535+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:41:17.519520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.937576+0000","last_change":"2024-04-20T21:20:02.938583+0000","last_active":"2024-04-20T21:20:03.937576+0000","last_peered":"2024-04-20T21:20:03.937576+0000","last_clean":"2024-04-20T21:20:03.937576+0000","last_became_active":"2024-04-20T21:20:02.938138+0000","last_became_peered":"2024-04-20T21:20:02.938138+0000","last_unstale":"2024-04-20T21:20:03.937576+0000","last_undegraded":"2024-04-20T21:20:03.937576+0000","last_fullsized":"2024-04-20T21:20:03.937576+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:40:00.140206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.957564+0000","last_change":"2024-04-20T21:20:03.950608+0000","last_active":"2024-04-20T21:20:04.957564+0000","last_peered":"2024-04-20T21:20:04.957564+0000","last_clean":"2024-04-20T21:20:04.957564+0000","last_became_active":"2024-04-20T21:20:03.950353+0000","last_became_peered":"2024-04-20T21:20:03.950353+0000","last_unstale":"2024-04-20T21:20:04.957564+0000","last_undegraded":"2024-04-20T21:20:04.957564+0000","last_fullsized":"2024-04-20T21:20:04.957564+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:57:22.245091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.953632+0000","last_change":"2024-04-20T21:20:02.931504+0000","last_active":"2024-04-20T21:20:03.953632+0000","last_peered":"2024-04-20T21:20:03.953632+0000","last_clean":"2024-04-20T21:20:03.953632+0000","last_became_active":"2024-04-20T21:20:02.931096+0000","last_became_peered":"2024-04-20T21:20:02.931096+0000","last_unstale":"2024-04-20T21:20:03.953632+0000","last_undegraded":"2024-04-20T21:20:03.953632+0000","last_fullsized":"2024-04-20T21:20:03.953632+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:57:21.228630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.957977+0000","last_change":"2024-04-20T21:20:03.961306+0000","last_active":"2024-04-20T21:20:04.957977+0000","last_peered":"2024-04-20T21:20:04.957977+0000","last_clean":"2024-04-20T21:20:04.957977+0000","last_became_active":"2024-04-20T21:20:03.960825+0000","last_became_peered":"2024-04-20T21:20:03.960825+0000","last_unstale":"2024-04-20T21:20:04.957977+0000","last_undegraded":"2024-04-20T21:20:04.957977+0000","last_fullsized":"2024-04-20T21:20:04.957977+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:18:23.071607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.958510+0000","last_change":"2024-04-20T21:20:02.941406+0000","last_active":"2024-04-20T21:20:04.958510+0000","last_peered":"2024-04-20T21:20:04.958510+0000","last_clean":"2024-04-20T21:20:04.958510+0000","last_became_active":"2024-04-20T21:20:02.941110+0000","last_became_peered":"2024-04-20T21:20:02.941110+0000","last_unstale":"2024-04-20T21:20:04.958510+0000","last_undegraded":"2024-04-20T21:20:04.958510+0000","last_fullsized":"2024-04-20T21:20:04.958510+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:51:42.526533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.956077+0000","last_change":"2024-04-20T21:20:03.950015+0000","last_active":"2024-04-20T21:20:03.956077+0000","last_peered":"2024-04-20T21:20:03.956077+0000","last_clean":"2024-04-20T21:20:03.956077+0000","last_became_active":"2024-04-20T21:20:03.949750+0000","last_became_peered":"2024-04-20T21:20:03.949750+0000","last_unstale":"2024-04-20T21:20:03.956077+0000","last_undegraded":"2024-04-20T21:20:03.956077+0000","last_fullsized":"2024-04-20T21:20:03.956077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:40:06.872797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.952116+0000","last_change":"2024-04-20T21:20:02.937766+0000","last_active":"2024-04-20T21:20:03.952116+0000","last_peered":"2024-04-20T21:20:03.952116+0000","last_clean":"2024-04-20T21:20:03.952116+0000","last_became_active":"2024-04-20T21:20:02.937439+0000","last_became_peered":"2024-04-20T21:20:02.937439+0000","last_unstale":"2024-04-20T21:20:03.952116+0000","last_undegraded":"2024-04-20T21:20:03.952116+0000","last_fullsized":"2024-04-20T21:20:03.952116+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:19:00.618007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.966232+0000","last_change":"2024-04-20T21:20:03.943716+0000","last_active":"2024-04-20T21:20:04.966232+0000","last_peered":"2024-04-20T21:20:04.966232+0000","last_clean":"2024-04-20T21:20:04.966232+0000","last_became_active":"2024-04-20T21:20:03.943453+0000","last_became_peered":"2024-04-20T21:20:03.943453+0000","last_unstale":"2024-04-20T21:20:04.966232+0000","last_undegraded":"2024-04-20T21:20:04.966232+0000","last_fullsized":"2024-04-20T21:20:04.966232+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:02:42.796359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"44'3","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.975202+0000","last_change":"2024-04-20T21:20:02.947542+0000","last_active":"2024-04-20T21:20:03.975202+0000","last_peered":"2024-04-20T21:20:03.975202+0000","last_clean":"2024-04-20T21:20:03.975202+0000","last_became_active":"2024-04-20T21:20:02.947222+0000","last_became_peered":"2024-04-20T21:20:02.947222+0000","last_unstale":"2024-04-20T21:20:03.975202+0000","last_undegraded":"2024-04-20T21:20:03.975202+0000","last_fullsized":"2024-04-20T21:20:03.975202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:32:52.332721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.956955+0000","last_change":"2024-04-20T21:20:03.953218+0000","last_active":"2024-04-20T21:20:03.956955+0000","last_peered":"2024-04-20T21:20:03.956955+0000","last_clean":"2024-04-20T21:20:03.956955+0000","last_became_active":"2024-04-20T21:20:03.952840+0000","last_became_peered":"2024-04-20T21:20:03.952840+0000","last_unstale":"2024-04-20T21:20:03.956955+0000","last_undegraded":"2024-04-20T21:20:03.956955+0000","last_fullsized":"2024-04-20T21:20:03.956955+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:18:23.071607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"44'6","reported_seq":20,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.953369+0000","last_change":"2024-04-20T21:20:02.938292+0000","last_active":"2024-04-20T21:20:04.953369+0000","last_peered":"2024-04-20T21:20:04.953369+0000","last_clean":"2024-04-20T21:20:04.953369+0000","last_became_active":"2024-04-20T21:20:02.937716+0000","last_became_peered":"2024-04-20T21:20:02.937716+0000","last_unstale":"2024-04-20T21:20:04.953369+0000","last_undegraded":"2024-04-20T21:20:04.953369+0000","last_fullsized":"2024-04-20T21:20:04.953369+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:11:37.705235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.958164+0000","last_change":"2024-04-20T21:20:03.953844+0000","last_active":"2024-04-20T21:20:04.958164+0000","last_peered":"2024-04-20T21:20:04.958164+0000","last_clean":"2024-04-20T21:20:04.958164+0000","last_became_active":"2024-04-20T21:20:03.953385+0000","last_became_peered":"2024-04-20T21:20:03.953385+0000","last_unstale":"2024-04-20T21:20:04.958164+0000","last_undegraded":"2024-04-20T21:20:04.958164+0000","last_fullsized":"2024-04-20T21:20:04.958164+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:51:43.542994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.965992+0000","last_change":"2024-04-20T21:20:02.943078+0000","last_active":"2024-04-20T21:20:04.965992+0000","last_peered":"2024-04-20T21:20:04.965992+0000","last_clean":"2024-04-20T21:20:04.965992+0000","last_became_active":"2024-04-20T21:20:02.942796+0000","last_became_peered":"2024-04-20T21:20:02.942796+0000","last_unstale":"2024-04-20T21:20:04.965992+0000","last_undegraded":"2024-04-20T21:20:04.965992+0000","last_fullsized":"2024-04-20T21:20:04.965992+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:18:22.055146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.961992+0000","last_change":"2024-04-20T21:20:03.955471+0000","last_active":"2024-04-20T21:20:03.961992+0000","last_peered":"2024-04-20T21:20:03.961992+0000","last_clean":"2024-04-20T21:20:03.961992+0000","last_became_active":"2024-04-20T21:20:03.954995+0000","last_became_peered":"2024-04-20T21:20:03.954995+0000","last_unstale":"2024-04-20T21:20:03.961992+0000","last_undegraded":"2024-04-20T21:20:03.961992+0000","last_fullsized":"2024-04-20T21:20:03.961992+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:51:43.542994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.944771+0000","last_change":"2024-04-20T21:20:02.944818+0000","last_active":"2024-04-20T21:20:03.944771+0000","last_peered":"2024-04-20T21:20:03.944771+0000","last_clean":"2024-04-20T21:20:03.944771+0000","last_became_active":"2024-04-20T21:20:02.944545+0000","last_became_peered":"2024-04-20T21:20:02.944545+0000","last_unstale":"2024-04-20T21:20:03.944771+0000","last_undegraded":"2024-04-20T21:20:03.944771+0000","last_fullsized":"2024-04-20T21:20:03.944771+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:06:27.907994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.965533+0000","last_change":"2024-04-20T21:20:03.939784+0000","last_active":"2024-04-20T21:20:04.965533+0000","last_peered":"2024-04-20T21:20:04.965533+0000","last_clean":"2024-04-20T21:20:04.965533+0000","last_became_active":"2024-04-20T21:20:03.939468+0000","last_became_peered":"2024-04-20T21:20:03.939468+0000","last_unstale":"2024-04-20T21:20:04.965533+0000","last_undegraded":"2024-04-20T21:20:04.965533+0000","last_fullsized":"2024-04-20T21:20:04.965533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:21:53.879172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.965752+0000","last_change":"2024-04-20T21:20:02.935626+0000","last_active":"2024-04-20T21:20:04.965752+0000","last_peered":"2024-04-20T21:20:04.965752+0000","last_clean":"2024-04-20T21:20:04.965752+0000","last_became_active":"2024-04-20T21:20:02.935287+0000","last_became_peered":"2024-04-20T21:20:02.935287+0000","last_unstale":"2024-04-20T21:20:04.965752+0000","last_undegraded":"2024-04-20T21:20:04.965752+0000","last_fullsized":"2024-04-20T21:20:04.965752+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:32:52.332721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.957158+0000","last_change":"2024-04-20T21:20:03.955305+0000","last_active":"2024-04-20T21:20:03.957158+0000","last_peered":"2024-04-20T21:20:03.957158+0000","last_clean":"2024-04-20T21:20:03.957158+0000","last_became_active":"2024-04-20T21:20:03.954925+0000","last_became_peered":"2024-04-20T21:20:03.954925+0000","last_unstale":"2024-04-20T21:20:03.957158+0000","last_undegraded":"2024-04-20T21:20:03.957158+0000","last_fullsized":"2024-04-20T21:20:03.957158+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:26:24.300536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"44'8","reported_seq":20,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.977133+0000","last_change":"2024-04-20T21:20:02.943948+0000","last_active":"2024-04-20T21:20:03.977133+0000","last_peered":"2024-04-20T21:20:03.977133+0000","last_clean":"2024-04-20T21:20:03.977133+0000","last_became_active":"2024-04-20T21:20:02.943624+0000","last_became_peered":"2024-04-20T21:20:02.943624+0000","last_unstale":"2024-04-20T21:20:03.977133+0000","last_undegraded":"2024-04-20T21:20:03.977133+0000","last_fullsized":"2024-04-20T21:20:03.977133+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:31:53.361555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.953529+0000","last_change":"2024-04-20T21:20:03.941174+0000","last_active":"2024-04-20T21:20:04.953529+0000","last_peered":"2024-04-20T21:20:04.953529+0000","last_clean":"2024-04-20T21:20:04.953529+0000","last_became_active":"2024-04-20T21:20:03.940872+0000","last_became_peered":"2024-04-20T21:20:03.940872+0000","last_unstale":"2024-04-20T21:20:04.953529+0000","last_undegraded":"2024-04-20T21:20:04.953529+0000","last_fullsized":"2024-04-20T21:20:04.953529+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:46:47.907080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.953425+0000","last_change":"2024-04-20T21:20:02.926662+0000","last_active":"2024-04-20T21:20:04.953425+0000","last_peered":"2024-04-20T21:20:04.953425+0000","last_clean":"2024-04-20T21:20:04.953425+0000","last_became_active":"2024-04-20T21:20:02.926377+0000","last_became_peered":"2024-04-20T21:20:02.926377+0000","last_unstale":"2024-04-20T21:20:04.953425+0000","last_undegraded":"2024-04-20T21:20:04.953425+0000","last_fullsized":"2024-04-20T21:20:04.953425+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:11:31.027478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.964353+0000","last_change":"2024-04-20T21:20:03.962320+0000","last_active":"2024-04-20T21:20:03.964353+0000","last_peered":"2024-04-20T21:20:03.964353+0000","last_clean":"2024-04-20T21:20:03.964353+0000","last_became_active":"2024-04-20T21:20:03.961988+0000","last_became_peered":"2024-04-20T21:20:03.961988+0000","last_unstale":"2024-04-20T21:20:03.964353+0000","last_undegraded":"2024-04-20T21:20:03.964353+0000","last_fullsized":"2024-04-20T21:20:03.964353+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:08:22.030102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.958146+0000","last_change":"2024-04-20T21:20:02.942589+0000","last_active":"2024-04-20T21:20:04.958146+0000","last_peered":"2024-04-20T21:20:04.958146+0000","last_clean":"2024-04-20T21:20:04.958146+0000","last_became_active":"2024-04-20T21:20:02.942306+0000","last_became_peered":"2024-04-20T21:20:02.942306+0000","last_unstale":"2024-04-20T21:20:04.958146+0000","last_undegraded":"2024-04-20T21:20:04.958146+0000","last_fullsized":"2024-04-20T21:20:04.958146+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:20:05.576223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.958686+0000","last_change":"2024-04-20T21:20:03.942731+0000","last_active":"2024-04-20T21:20:04.958686+0000","last_peered":"2024-04-20T21:20:04.958686+0000","last_clean":"2024-04-20T21:20:04.958686+0000","last_became_active":"2024-04-20T21:20:03.942281+0000","last_became_peered":"2024-04-20T21:20:03.942281+0000","last_unstale":"2024-04-20T21:20:04.958686+0000","last_undegraded":"2024-04-20T21:20:04.958686+0000","last_fullsized":"2024-04-20T21:20:04.958686+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:07:48.221573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"44'1","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.973061+0000","last_change":"2024-04-20T21:20:02.941701+0000","last_active":"2024-04-20T21:20:03.973061+0000","last_peered":"2024-04-20T21:20:03.973061+0000","last_clean":"2024-04-20T21:20:03.973061+0000","last_became_active":"2024-04-20T21:20:02.941400+0000","last_became_peered":"2024-04-20T21:20:02.941400+0000","last_unstale":"2024-04-20T21:20:03.973061+0000","last_undegraded":"2024-04-20T21:20:03.973061+0000","last_fullsized":"2024-04-20T21:20:03.973061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:06:27.907994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.959371+0000","last_change":"2024-04-20T21:20:03.947103+0000","last_active":"2024-04-20T21:20:04.959371+0000","last_peered":"2024-04-20T21:20:04.959371+0000","last_clean":"2024-04-20T21:20:04.959371+0000","last_became_active":"2024-04-20T21:20:03.946087+0000","last_became_peered":"2024-04-20T21:20:03.946087+0000","last_unstale":"2024-04-20T21:20:04.959371+0000","last_undegraded":"2024-04-20T21:20:04.959371+0000","last_fullsized":"2024-04-20T21:20:04.959371+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:01:47.306517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.966332+0000","last_change":"2024-04-20T21:20:02.936129+0000","last_active":"2024-04-20T21:20:04.966332+0000","last_peered":"2024-04-20T21:20:04.966332+0000","last_clean":"2024-04-20T21:20:04.966332+0000","last_became_active":"2024-04-20T21:20:02.935380+0000","last_became_peered":"2024-04-20T21:20:02.935380+0000","last_unstale":"2024-04-20T21:20:04.966332+0000","last_undegraded":"2024-04-20T21:20:04.966332+0000","last_fullsized":"2024-04-20T21:20:04.966332+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:18:00.169616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.966033+0000","last_change":"2024-04-20T21:20:03.936823+0000","last_active":"2024-04-20T21:20:04.966033+0000","last_peered":"2024-04-20T21:20:04.966033+0000","last_clean":"2024-04-20T21:20:04.966033+0000","last_became_active":"2024-04-20T21:20:03.936483+0000","last_became_peered":"2024-04-20T21:20:03.936483+0000","last_unstale":"2024-04-20T21:20:04.966033+0000","last_undegraded":"2024-04-20T21:20:04.966033+0000","last_fullsized":"2024-04-20T21:20:04.966033+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:21:57.579803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.966192+0000","last_change":"2024-04-20T21:20:02.939226+0000","last_active":"2024-04-20T21:20:04.966192+0000","last_peered":"2024-04-20T21:20:04.966192+0000","last_clean":"2024-04-20T21:20:04.966192+0000","last_became_active":"2024-04-20T21:20:02.938943+0000","last_became_peered":"2024-04-20T21:20:02.938943+0000","last_unstale":"2024-04-20T21:20:04.966192+0000","last_undegraded":"2024-04-20T21:20:04.966192+0000","last_fullsized":"2024-04-20T21:20:04.966192+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:40:05.856336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:05.231753+0000","last_change":"2024-04-20T21:20:03.945415+0000","last_active":"2024-04-20T21:20:05.231753+0000","last_peered":"2024-04-20T21:20:05.231753+0000","last_clean":"2024-04-20T21:20:05.231753+0000","last_became_active":"2024-04-20T21:20:03.944297+0000","last_became_peered":"2024-04-20T21:20:03.944297+0000","last_unstale":"2024-04-20T21:20:05.231753+0000","last_undegraded":"2024-04-20T21:20:05.231753+0000","last_fullsized":"2024-04-20T21:20:05.231753+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:03:41.090616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.936097+0000","last_change":"2024-04-20T21:20:02.941304+0000","last_active":"2024-04-20T21:20:03.936097+0000","last_peered":"2024-04-20T21:20:03.936097+0000","last_clean":"2024-04-20T21:20:03.936097+0000","last_became_active":"2024-04-20T21:20:02.940993+0000","last_became_peered":"2024-04-20T21:20:02.940993+0000","last_unstale":"2024-04-20T21:20:03.936097+0000","last_undegraded":"2024-04-20T21:20:03.936097+0000","last_fullsized":"2024-04-20T21:20:03.936097+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:10:12.969702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.953078+0000","last_change":"2024-04-20T21:20:03.948314+0000","last_active":"2024-04-20T21:20:04.953078+0000","last_peered":"2024-04-20T21:20:04.953078+0000","last_clean":"2024-04-20T21:20:04.953078+0000","last_became_active":"2024-04-20T21:20:03.948018+0000","last_became_peered":"2024-04-20T21:20:03.948018+0000","last_unstale":"2024-04-20T21:20:04.953078+0000","last_undegraded":"2024-04-20T21:20:04.953078+0000","last_fullsized":"2024-04-20T21:20:04.953078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:38:51.311160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.944505+0000","last_change":"2024-04-20T21:20:02.938238+0000","last_active":"2024-04-20T21:20:03.944505+0000","last_peered":"2024-04-20T21:20:03.944505+0000","last_clean":"2024-04-20T21:20:03.944505+0000","last_became_active":"2024-04-20T21:20:02.937929+0000","last_became_peered":"2024-04-20T21:20:02.937929+0000","last_unstale":"2024-04-20T21:20:03.944505+0000","last_undegraded":"2024-04-20T21:20:03.944505+0000","last_fullsized":"2024-04-20T21:20:03.944505+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:14:18.109412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.960775+0000","last_change":"2024-04-20T21:20:03.955409+0000","last_active":"2024-04-20T21:20:03.960775+0000","last_peered":"2024-04-20T21:20:03.960775+0000","last_clean":"2024-04-20T21:20:03.960775+0000","last_became_active":"2024-04-20T21:20:03.955047+0000","last_became_peered":"2024-04-20T21:20:03.955047+0000","last_unstale":"2024-04-20T21:20:03.960775+0000","last_undegraded":"2024-04-20T21:20:03.960775+0000","last_fullsized":"2024-04-20T21:20:03.960775+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:20:06.592684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.945304+0000","last_change":"2024-04-20T21:20:02.941315+0000","last_active":"2024-04-20T21:20:03.945304+0000","last_peered":"2024-04-20T21:20:03.945304+0000","last_clean":"2024-04-20T21:20:03.945304+0000","last_became_active":"2024-04-20T21:20:02.940992+0000","last_became_peered":"2024-04-20T21:20:02.940992+0000","last_unstale":"2024-04-20T21:20:03.945304+0000","last_undegraded":"2024-04-20T21:20:03.945304+0000","last_fullsized":"2024-04-20T21:20:03.945304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:41:16.503059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.959584+0000","last_change":"2024-04-20T21:20:03.960412+0000","last_active":"2024-04-20T21:20:04.959584+0000","last_peered":"2024-04-20T21:20:04.959584+0000","last_clean":"2024-04-20T21:20:04.959584+0000","last_became_active":"2024-04-20T21:20:03.960094+0000","last_became_peered":"2024-04-20T21:20:03.960094+0000","last_unstale":"2024-04-20T21:20:04.959584+0000","last_undegraded":"2024-04-20T21:20:04.959584+0000","last_fullsized":"2024-04-20T21:20:04.959584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:40:01.156667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"45'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.968965+0000","last_change":"2024-04-20T21:20:02.941534+0000","last_active":"2024-04-20T21:20:04.968965+0000","last_peered":"2024-04-20T21:20:04.968965+0000","last_clean":"2024-04-20T21:20:04.968965+0000","last_became_active":"2024-04-20T21:20:02.941313+0000","last_became_peered":"2024-04-20T21:20:02.941313+0000","last_unstale":"2024-04-20T21:20:04.968965+0000","last_undegraded":"2024-04-20T21:20:04.968965+0000","last_fullsized":"2024-04-20T21:20:04.968965+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:57:57.059900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.962119+0000","last_change":"2024-04-20T21:20:03.956916+0000","last_active":"2024-04-20T21:20:03.962119+0000","last_peered":"2024-04-20T21:20:03.962119+0000","last_clean":"2024-04-20T21:20:03.962119+0000","last_became_active":"2024-04-20T21:20:03.956585+0000","last_became_peered":"2024-04-20T21:20:03.956585+0000","last_unstale":"2024-04-20T21:20:03.962119+0000","last_undegraded":"2024-04-20T21:20:03.962119+0000","last_fullsized":"2024-04-20T21:20:03.962119+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:39:45.020282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.947101+0000","last_change":"2024-04-20T21:20:02.948112+0000","last_active":"2024-04-20T21:20:03.947101+0000","last_peered":"2024-04-20T21:20:03.947101+0000","last_clean":"2024-04-20T21:20:03.947101+0000","last_became_active":"2024-04-20T21:20:02.947665+0000","last_became_peered":"2024-04-20T21:20:02.947665+0000","last_unstale":"2024-04-20T21:20:03.947101+0000","last_undegraded":"2024-04-20T21:20:03.947101+0000","last_fullsized":"2024-04-20T21:20:03.947101+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:18:00.169616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.957408+0000","last_change":"2024-04-20T21:20:03.954130+0000","last_active":"2024-04-20T21:20:03.957408+0000","last_peered":"2024-04-20T21:20:03.957408+0000","last_clean":"2024-04-20T21:20:03.957408+0000","last_became_active":"2024-04-20T21:20:03.953236+0000","last_became_peered":"2024-04-20T21:20:03.953236+0000","last_unstale":"2024-04-20T21:20:03.957408+0000","last_undegraded":"2024-04-20T21:20:03.957408+0000","last_fullsized":"2024-04-20T21:20:03.957408+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:56:57.740183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:05.231627+0000","last_change":"2024-04-20T21:20:02.941938+0000","last_active":"2024-04-20T21:20:05.231627+0000","last_peered":"2024-04-20T21:20:05.231627+0000","last_clean":"2024-04-20T21:20:05.231627+0000","last_became_active":"2024-04-20T21:20:02.941603+0000","last_became_peered":"2024-04-20T21:20:02.941603+0000","last_unstale":"2024-04-20T21:20:05.231627+0000","last_undegraded":"2024-04-20T21:20:05.231627+0000","last_fullsized":"2024-04-20T21:20:05.231627+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:00:12.828005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.966414+0000","last_change":"2024-04-20T21:20:03.943596+0000","last_active":"2024-04-20T21:20:04.966414+0000","last_peered":"2024-04-20T21:20:04.966414+0000","last_clean":"2024-04-20T21:20:04.966414+0000","last_became_active":"2024-04-20T21:20:03.943345+0000","last_became_peered":"2024-04-20T21:20:03.943345+0000","last_unstale":"2024-04-20T21:20:04.966414+0000","last_undegraded":"2024-04-20T21:20:04.966414+0000","last_fullsized":"2024-04-20T21:20:04.966414+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:41:39.708030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.946605+0000","last_change":"2024-04-20T21:20:02.946571+0000","last_active":"2024-04-20T21:20:03.946605+0000","last_peered":"2024-04-20T21:20:03.946605+0000","last_clean":"2024-04-20T21:20:03.946605+0000","last_became_active":"2024-04-20T21:20:02.946286+0000","last_became_peered":"2024-04-20T21:20:02.946286+0000","last_unstale":"2024-04-20T21:20:03.946605+0000","last_undegraded":"2024-04-20T21:20:03.946605+0000","last_fullsized":"2024-04-20T21:20:03.946605+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:08:21.013641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.966394+0000","last_change":"2024-04-20T21:20:03.946626+0000","last_active":"2024-04-20T21:20:04.966394+0000","last_peered":"2024-04-20T21:20:04.966394+0000","last_clean":"2024-04-20T21:20:04.966394+0000","last_became_active":"2024-04-20T21:20:03.946366+0000","last_became_peered":"2024-04-20T21:20:03.946366+0000","last_unstale":"2024-04-20T21:20:04.966394+0000","last_undegraded":"2024-04-20T21:20:04.966394+0000","last_fullsized":"2024-04-20T21:20:04.966394+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:29:24.103308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.956687+0000","last_change":"2024-04-20T21:20:02.938506+0000","last_active":"2024-04-20T21:20:04.956687+0000","last_peered":"2024-04-20T21:20:04.956687+0000","last_clean":"2024-04-20T21:20:04.956687+0000","last_became_active":"2024-04-20T21:20:02.938164+0000","last_became_peered":"2024-04-20T21:20:02.938164+0000","last_unstale":"2024-04-20T21:20:04.956687+0000","last_undegraded":"2024-04-20T21:20:04.956687+0000","last_fullsized":"2024-04-20T21:20:04.956687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:00:12.828005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.949531+0000","last_change":"2024-04-20T21:20:03.947582+0000","last_active":"2024-04-20T21:20:03.949531+0000","last_peered":"2024-04-20T21:20:03.949531+0000","last_clean":"2024-04-20T21:20:03.949531+0000","last_became_active":"2024-04-20T21:20:03.947299+0000","last_became_peered":"2024-04-20T21:20:03.947299+0000","last_unstale":"2024-04-20T21:20:03.949531+0000","last_undegraded":"2024-04-20T21:20:03.949531+0000","last_fullsized":"2024-04-20T21:20:03.949531+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:32:53.349182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:05.231300+0000","last_change":"2024-04-20T21:20:02.928908+0000","last_active":"2024-04-20T21:20:05.231300+0000","last_peered":"2024-04-20T21:20:05.231300+0000","last_clean":"2024-04-20T21:20:05.231300+0000","last_became_active":"2024-04-20T21:20:02.928634+0000","last_became_peered":"2024-04-20T21:20:02.928634+0000","last_unstale":"2024-04-20T21:20:05.231300+0000","last_undegraded":"2024-04-20T21:20:05.231300+0000","last_fullsized":"2024-04-20T21:20:05.231300+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:57:21.228630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.956095+0000","last_change":"2024-04-20T21:20:03.934008+0000","last_active":"2024-04-20T21:20:04.956095+0000","last_peered":"2024-04-20T21:20:04.956095+0000","last_clean":"2024-04-20T21:20:04.956095+0000","last_became_active":"2024-04-20T21:20:03.933452+0000","last_became_peered":"2024-04-20T21:20:03.933452+0000","last_unstale":"2024-04-20T21:20:04.956095+0000","last_undegraded":"2024-04-20T21:20:04.956095+0000","last_fullsized":"2024-04-20T21:20:04.956095+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:41:17.519520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.953353+0000","last_change":"2024-04-20T21:20:02.932961+0000","last_active":"2024-04-20T21:20:03.953353+0000","last_peered":"2024-04-20T21:20:03.953353+0000","last_clean":"2024-04-20T21:20:03.953353+0000","last_became_active":"2024-04-20T21:20:02.932624+0000","last_became_peered":"2024-04-20T21:20:02.932624+0000","last_unstale":"2024-04-20T21:20:03.953353+0000","last_undegraded":"2024-04-20T21:20:03.953353+0000","last_fullsized":"2024-04-20T21:20:03.953353+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:00:12.828005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.962152+0000","last_change":"2024-04-20T21:20:03.958692+0000","last_active":"2024-04-20T21:20:03.962152+0000","last_peered":"2024-04-20T21:20:03.962152+0000","last_clean":"2024-04-20T21:20:03.962152+0000","last_became_active":"2024-04-20T21:20:03.958420+0000","last_became_peered":"2024-04-20T21:20:03.958420+0000","last_unstale":"2024-04-20T21:20:03.962152+0000","last_undegraded":"2024-04-20T21:20:03.962152+0000","last_fullsized":"2024-04-20T21:20:03.962152+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:01:47.306517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.975131+0000","last_change":"2024-04-20T21:20:02.943225+0000","last_active":"2024-04-20T21:20:03.975131+0000","last_peered":"2024-04-20T21:20:03.975131+0000","last_clean":"2024-04-20T21:20:03.975131+0000","last_became_active":"2024-04-20T21:20:02.942904+0000","last_became_peered":"2024-04-20T21:20:02.942904+0000","last_unstale":"2024-04-20T21:20:03.975131+0000","last_undegraded":"2024-04-20T21:20:03.975131+0000","last_fullsized":"2024-04-20T21:20:03.975131+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:41:16.503059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":211,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.945359+0000","last_change":"2024-04-20T21:17:43.059637+0000","last_active":"2024-04-20T21:20:03.945359+0000","last_peered":"2024-04-20T21:20:03.945359+0000","last_clean":"2024-04-20T21:20:03.945359+0000","last_became_active":"2024-04-20T21:17:43.056207+0000","last_became_peered":"2024-04-20T21:17:43.056207+0000","last_unstale":"2024-04-20T21:20:03.945359+0000","last_undegraded":"2024-04-20T21:20:03.945359+0000","last_fullsized":"2024-04-20T21:20:03.945359+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:41.036290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:41.036290+0000","last_clean_scrub_stamp":"2024-04-20T21:17:41.036290+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:58:51.091337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574879,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1114112,"data_stored":799273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27540,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":524288,"data_stored":209487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574882,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":520192,"data_stored":208885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":540672,"data_stored":216727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27544,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":528384,"data_stored":209453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":544768,"data_stored":216749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574880,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":520192,"data_stored":208885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28108,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343732,"statfs":{"total":96636764160,"available":96607981568,"internally_reserved":0,"allocated":1114112,"data_stored":799217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T21:20:08.327 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T21:20:08.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.563+0000 7f0f70e00640 1 Processor -- start 2024-04-20T21:20:08.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.563+0000 7f0f70e00640 1 -- start start 2024-04-20T21:20:08.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.564+0000 7f0f70e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f6c156ae0 0x7f0f6c15d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:08.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.564+0000 7f0f70e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0f6c160080 0x7f0f6c0ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:08.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.564+0000 7f0f70e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0f6c0ad620 0x7f0f6c141cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:08.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.564+0000 7f0f70e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f0f6c06db20 con 0x7f0f6c0ad620 2024-04-20T21:20:08.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.564+0000 7f0f70e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f0f6c160890 con 0x7f0f6c160080 2024-04-20T21:20:08.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.564+0000 7f0f70e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0f6c160b70 con 0x7f0f6c156ae0 2024-04-20T21:20:08.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.564+0000 7f0f6be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0f6c0ad620 0x7f0f6c141cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:08.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.565+0000 7f0f6be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0f6c0ad620 0x7f0f6c141cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45580/0 (socket says 172.21.15.86:45580) 2024-04-20T21:20:08.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.565+0000 7f0f6be00640 1 -- 172.21.15.86:0/336454867 learned_addr learned my addr 172.21.15.86:0/336454867 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:08.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.565+0000 7f0f6b400640 1 --2- 172.21.15.86:0/336454867 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f6c156ae0 0x7f0f6c15d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:08.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.565+0000 7f0f6be00640 1 -- 172.21.15.86:0/336454867 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f6c156ae0 msgr2=0x7f0f6c15d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:08.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.565+0000 7f0f6be00640 1 --2- 172.21.15.86:0/336454867 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f6c156ae0 0x7f0f6c15d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:08.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.565+0000 7f0f6be00640 1 -- 172.21.15.86:0/336454867 >> v2:172.21.15.144:3300/0 conn(0x7f0f6c160080 msgr2=0x7f0f6c0ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:08.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.565+0000 7f0f62a00640 1 --2- 172.21.15.86:0/336454867 >> v2:172.21.15.144:3300/0 conn(0x7f0f6c160080 0x7f0f6c0ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:08.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.565+0000 7f0f6be00640 1 --2- 172.21.15.86:0/336454867 >> v2:172.21.15.144:3300/0 conn(0x7f0f6c160080 0x7f0f6c0ad0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:08.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.565+0000 7f0f6be00640 1 -- 172.21.15.86:0/336454867 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f6c160e50 con 0x7f0f6c0ad620 2024-04-20T21:20:08.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.566+0000 7f0f6be00640 1 --2- 172.21.15.86:0/336454867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f6c0ad620 0x7f0f6c141cb0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c06d490 tx=0x7f0f6402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6980fa1b7f78561 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:08.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.566+0000 7f0f6aa00640 1 -- 172.21.15.86:0/336454867 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6402ca70 con 0x7f0f6c0ad620 2024-04-20T21:20:08.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.566+0000 7f0f6aa00640 1 -- 172.21.15.86:0/336454867 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f6402cc10 con 0x7f0f6c0ad620 2024-04-20T21:20:08.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.566+0000 7f0f6aa00640 1 -- 172.21.15.86:0/336454867 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f640343f0 con 0x7f0f6c0ad620 2024-04-20T21:20:08.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.567+0000 7f0f70e00640 1 -- 172.21.15.86:0/336454867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f6c0ad620 msgr2=0x7f0f6c141cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:08.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.567+0000 7f0f70e00640 1 --2- 172.21.15.86:0/336454867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f6c0ad620 0x7f0f6c141cb0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c06d490 tx=0x7f0f6402bed0 comp rx=0 tx=0).stop 2024-04-20T21:20:08.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.567+0000 7f0f70e00640 1 -- 172.21.15.86:0/336454867 shutdown_connections 2024-04-20T21:20:08.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.567+0000 7f0f70e00640 1 --2- 172.21.15.86:0/336454867 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f6c0ad620 0x7f0f6c141cb0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:08.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.567+0000 7f0f70e00640 1 --2- 172.21.15.86:0/336454867 >> v2:172.21.15.144:3300/0 conn(0x7f0f6c160080 0x7f0f6c0ad0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:08.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.567+0000 7f0f70e00640 1 --2- 172.21.15.86:0/336454867 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f6c156ae0 0x7f0f6c15d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:08.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.567+0000 7f0f70e00640 1 -- 172.21.15.86:0/336454867 >> 172.21.15.86:0/336454867 conn(0x7f0f6c0d0170 msgr2=0x7f0f6c0d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:08.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.567+0000 7f0f70e00640 1 -- 172.21.15.86:0/336454867 shutdown_connections 2024-04-20T21:20:08.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.567+0000 7f0f70e00640 1 -- 172.21.15.86:0/336454867 wait complete. 2024-04-20T21:20:08.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.569+0000 7f0f70e00640 1 Processor -- start 2024-04-20T21:20:08.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.569+0000 7f0f70e00640 1 -- start start 2024-04-20T21:20:08.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.570+0000 7f0f70e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f6c0ad620 0x7f0f6c15c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:08.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.570+0000 7f0f70e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f6c156ae0 0x7f0f6c142f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:08.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.570+0000 7f0f70e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f6c160080 0x7f0f6c1f4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:08.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.570+0000 7f0f70e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6c160500 con 0x7f0f6c156ae0 2024-04-20T21:20:08.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.570+0000 7f0f70e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6c156ec0 con 0x7f0f6c0ad620 2024-04-20T21:20:08.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.570+0000 7f0f70e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0f6c1421f0 con 0x7f0f6c160080 2024-04-20T21:20:08.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.570+0000 7f0f62a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f6c156ae0 0x7f0f6c142f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:08.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.570+0000 7f0f62a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f6c156ae0 0x7f0f6c142f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45588/0 (socket says 172.21.15.86:45588) 2024-04-20T21:20:08.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.570+0000 7f0f62a00640 1 -- 172.21.15.86:0/919158943 learned_addr learned my addr 172.21.15.86:0/919158943 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:08.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.570+0000 7f0f6be00640 1 --2- 172.21.15.86:0/919158943 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f6c160080 0x7f0f6c1f4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:08.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.571+0000 7f0f6b400640 1 --2- 172.21.15.86:0/919158943 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f6c0ad620 0x7f0f6c15c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:08.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.571+0000 7f0f62a00640 1 -- 172.21.15.86:0/919158943 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f6c160080 msgr2=0x7f0f6c1f4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:08.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.571+0000 7f0f62a00640 1 --2- 172.21.15.86:0/919158943 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f6c160080 0x7f0f6c1f4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:08.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.571+0000 7f0f62a00640 1 -- 172.21.15.86:0/919158943 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f6c0ad620 msgr2=0x7f0f6c15c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:08.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.571+0000 7f0f62a00640 1 --2- 172.21.15.86:0/919158943 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f6c0ad620 0x7f0f6c15c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:08.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.571+0000 7f0f62a00640 1 -- 172.21.15.86:0/919158943 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f6c15c880 con 0x7f0f6c156ae0 2024-04-20T21:20:08.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.571+0000 7f0f62a00640 1 --2- 172.21.15.86:0/919158943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f6c156ae0 0x7f0f6c142f00 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f0f5400c7f0 tx=0x7f0f5400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:08.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.571+0000 7f0f63e00640 1 -- 172.21.15.86:0/919158943 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f54015070 con 0x7f0f6c156ae0 2024-04-20T21:20:08.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.571+0000 7f0f63e00640 1 -- 172.21.15.86:0/919158943 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f54004220 con 0x7f0f6c156ae0 2024-04-20T21:20:08.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.571+0000 7f0f63e00640 1 -- 172.21.15.86:0/919158943 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f54007620 con 0x7f0f6c156ae0 2024-04-20T21:20:08.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.571+0000 7f0f70e00640 1 -- 172.21.15.86:0/919158943 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f6c1f49c0 con 0x7f0f6c156ae0 2024-04-20T21:20:08.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.571+0000 7f0f70e00640 1 -- 172.21.15.86:0/919158943 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f6c144d70 con 0x7f0f6c156ae0 2024-04-20T21:20:08.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.572+0000 7f0f70e00640 1 -- 172.21.15.86:0/919158943 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f6c070510 con 0x7f0f6c156ae0 2024-04-20T21:20:08.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.573+0000 7f0f63e00640 1 -- 172.21.15.86:0/919158943 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0f540077c0 con 0x7f0f6c156ae0 2024-04-20T21:20:08.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.574+0000 7f0f63e00640 1 --2- 172.21.15.86:0/919158943 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0f4806c920 0x7f0f4806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:08.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.574+0000 7f0f63e00640 1 -- 172.21.15.86:0/919158943 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f0f5408b7d0 con 0x7f0f6c156ae0 2024-04-20T21:20:08.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.574+0000 7f0f6b400640 1 --2- 172.21.15.86:0/919158943 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0f4806c920 0x7f0f4806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:08.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.575+0000 7f0f6b400640 1 --2- 172.21.15.86:0/919158943 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0f4806c920 0x7f0f4806ede0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c006fd0 tx=0x7f0f5c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:08.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.579+0000 7f0f63e00640 1 -- 172.21.15.86:0/919158943 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0f6c070510 con 0x7f0f6c156ae0 2024-04-20T21:20:08.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.769+0000 7f0f70e00640 1 -- 172.21.15.86:0/919158943 --> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0f6c04fcf0 con 0x7f0f4806c920 2024-04-20T21:20:08.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.780+0000 7f0f63e00640 1 -- 172.21.15.86:0/919158943 <== mgr.4107 v2:172.21.15.86:6832/302045240 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251931 (secure 0 0 0) 0x7f0f6c04fcf0 con 0x7f0f4806c920 2024-04-20T21:20:08.782 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:20:08.785 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-20T21:20:08.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.784+0000 7f0f70e00640 1 -- 172.21.15.86:0/919158943 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0f4806c920 msgr2=0x7f0f4806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:08.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.784+0000 7f0f70e00640 1 --2- 172.21.15.86:0/919158943 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0f4806c920 0x7f0f4806ede0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c006fd0 tx=0x7f0f5c008040 comp rx=0 tx=0).stop 2024-04-20T21:20:08.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.784+0000 7f0f70e00640 1 -- 172.21.15.86:0/919158943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f6c156ae0 msgr2=0x7f0f6c142f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:08.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.784+0000 7f0f70e00640 1 --2- 172.21.15.86:0/919158943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f6c156ae0 0x7f0f6c142f00 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f0f5400c7f0 tx=0x7f0f5400ccb0 comp rx=0 tx=0).stop 2024-04-20T21:20:08.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.785+0000 7f0f70e00640 1 -- 172.21.15.86:0/919158943 shutdown_connections 2024-04-20T21:20:08.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.785+0000 7f0f70e00640 1 --2- 172.21.15.86:0/919158943 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0f4806c920 0x7f0f4806ede0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:08.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.785+0000 7f0f70e00640 1 --2- 172.21.15.86:0/919158943 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f6c160080 0x7f0f6c1f4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:08.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.785+0000 7f0f70e00640 1 --2- 172.21.15.86:0/919158943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f6c156ae0 0x7f0f6c142f00 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:08.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.785+0000 7f0f70e00640 1 --2- 172.21.15.86:0/919158943 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f6c0ad620 0x7f0f6c15c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:08.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.785+0000 7f0f70e00640 1 -- 172.21.15.86:0/919158943 >> 172.21.15.86:0/919158943 conn(0x7f0f6c0d0170 msgr2=0x7f0f6c0afd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:08.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.785+0000 7f0f70e00640 1 -- 172.21.15.86:0/919158943 shutdown_connections 2024-04-20T21:20:08.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:08.785+0000 7f0f70e00640 1 -- 172.21.15.86:0/919158943 wait complete. 2024-04-20T21:20:08.802 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":112,"stamp":"2024-04-20T21:20:08.672631+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":114,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":225624,"kb_used_data":5280,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749096,"statfs":{"total":773094113280,"available":772863074304,"internally_reserved":0,"allocated":5406720,"data_stored":2868676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002186"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.957076+0000","last_change":"2024-04-20T21:20:03.953844+0000","last_active":"2024-04-20T21:20:03.957076+0000","last_peered":"2024-04-20T21:20:03.957076+0000","last_clean":"2024-04-20T21:20:03.957076+0000","last_became_active":"2024-04-20T21:20:03.952822+0000","last_became_peered":"2024-04-20T21:20:03.952822+0000","last_unstale":"2024-04-20T21:20:03.957076+0000","last_undegraded":"2024-04-20T21:20:03.957076+0000","last_fullsized":"2024-04-20T21:20:03.957076+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:27:08.974286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.966513+0000","last_change":"2024-04-20T21:20:03.963223+0000","last_active":"2024-04-20T21:20:03.966513+0000","last_peered":"2024-04-20T21:20:03.966513+0000","last_clean":"2024-04-20T21:20:03.966513+0000","last_became_active":"2024-04-20T21:20:03.962841+0000","last_became_peered":"2024-04-20T21:20:03.962841+0000","last_unstale":"2024-04-20T21:20:03.966513+0000","last_undegraded":"2024-04-20T21:20:03.966513+0000","last_fullsized":"2024-04-20T21:20:03.966513+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:40:06.872797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.957056+0000","last_change":"2024-04-20T21:20:03.934734+0000","last_active":"2024-04-20T21:20:04.957056+0000","last_peered":"2024-04-20T21:20:04.957056+0000","last_clean":"2024-04-20T21:20:04.957056+0000","last_became_active":"2024-04-20T21:20:03.934290+0000","last_became_peered":"2024-04-20T21:20:03.934290+0000","last_unstale":"2024-04-20T21:20:04.957056+0000","last_undegraded":"2024-04-20T21:20:04.957056+0000","last_fullsized":"2024-04-20T21:20:04.957056+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:32:53.349182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.962140+0000","last_change":"2024-04-20T21:20:03.959935+0000","last_active":"2024-04-20T21:20:03.962140+0000","last_peered":"2024-04-20T21:20:03.962140+0000","last_clean":"2024-04-20T21:20:03.962140+0000","last_became_active":"2024-04-20T21:20:03.958987+0000","last_became_peered":"2024-04-20T21:20:03.958987+0000","last_unstale":"2024-04-20T21:20:03.962140+0000","last_undegraded":"2024-04-20T21:20:03.962140+0000","last_fullsized":"2024-04-20T21:20:03.962140+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:47:26.349120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.963864+0000","last_change":"2024-04-20T21:20:03.959137+0000","last_active":"2024-04-20T21:20:03.963864+0000","last_peered":"2024-04-20T21:20:03.963864+0000","last_clean":"2024-04-20T21:20:03.963864+0000","last_became_active":"2024-04-20T21:20:03.958830+0000","last_became_peered":"2024-04-20T21:20:03.958830+0000","last_unstale":"2024-04-20T21:20:03.963864+0000","last_undegraded":"2024-04-20T21:20:03.963864+0000","last_fullsized":"2024-04-20T21:20:03.963864+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:56:29.421282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.966803+0000","last_change":"2024-04-20T21:20:03.950648+0000","last_active":"2024-04-20T21:20:04.966803+0000","last_peered":"2024-04-20T21:20:04.966803+0000","last_clean":"2024-04-20T21:20:04.966803+0000","last_became_active":"2024-04-20T21:20:03.950288+0000","last_became_peered":"2024-04-20T21:20:03.950288+0000","last_unstale":"2024-04-20T21:20:04.966803+0000","last_undegraded":"2024-04-20T21:20:04.966803+0000","last_fullsized":"2024-04-20T21:20:04.966803+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:11:08.517771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.955972+0000","last_change":"2024-04-20T21:20:03.949963+0000","last_active":"2024-04-20T21:20:03.955972+0000","last_peered":"2024-04-20T21:20:03.955972+0000","last_clean":"2024-04-20T21:20:03.955972+0000","last_became_active":"2024-04-20T21:20:03.949607+0000","last_became_peered":"2024-04-20T21:20:03.949607+0000","last_unstale":"2024-04-20T21:20:03.955972+0000","last_undegraded":"2024-04-20T21:20:03.955972+0000","last_fullsized":"2024-04-20T21:20:03.955972+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:56:29.421282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.963869+0000","last_change":"2024-04-20T21:20:03.960633+0000","last_active":"2024-04-20T21:20:03.963869+0000","last_peered":"2024-04-20T21:20:03.963869+0000","last_clean":"2024-04-20T21:20:03.963869+0000","last_became_active":"2024-04-20T21:20:03.960018+0000","last_became_peered":"2024-04-20T21:20:03.960018+0000","last_unstale":"2024-04-20T21:20:03.963869+0000","last_undegraded":"2024-04-20T21:20:03.963869+0000","last_fullsized":"2024-04-20T21:20:03.963869+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:10:13.986163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.964377+0000","last_change":"2024-04-20T21:20:03.962654+0000","last_active":"2024-04-20T21:20:03.964377+0000","last_peered":"2024-04-20T21:20:03.964377+0000","last_clean":"2024-04-20T21:20:03.964377+0000","last_became_active":"2024-04-20T21:20:03.962366+0000","last_became_peered":"2024-04-20T21:20:03.962366+0000","last_unstale":"2024-04-20T21:20:03.964377+0000","last_undegraded":"2024-04-20T21:20:03.964377+0000","last_fullsized":"2024-04-20T21:20:03.964377+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:57:58.076361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.964257+0000","last_change":"2024-04-20T21:20:03.961204+0000","last_active":"2024-04-20T21:20:03.964257+0000","last_peered":"2024-04-20T21:20:03.964257+0000","last_clean":"2024-04-20T21:20:03.964257+0000","last_became_active":"2024-04-20T21:20:03.960258+0000","last_became_peered":"2024-04-20T21:20:03.960258+0000","last_unstale":"2024-04-20T21:20:03.964257+0000","last_undegraded":"2024-04-20T21:20:03.964257+0000","last_fullsized":"2024-04-20T21:20:03.964257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:06:28.924456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.963894+0000","last_change":"2024-04-20T21:20:03.960942+0000","last_active":"2024-04-20T21:20:03.963894+0000","last_peered":"2024-04-20T21:20:03.963894+0000","last_clean":"2024-04-20T21:20:03.963894+0000","last_became_active":"2024-04-20T21:20:03.960584+0000","last_became_peered":"2024-04-20T21:20:03.960584+0000","last_unstale":"2024-04-20T21:20:03.963894+0000","last_undegraded":"2024-04-20T21:20:03.963894+0000","last_fullsized":"2024-04-20T21:20:03.963894+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:40:01.156667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.949162+0000","last_change":"2024-04-20T21:20:03.944336+0000","last_active":"2024-04-20T21:20:03.949162+0000","last_peered":"2024-04-20T21:20:03.949162+0000","last_clean":"2024-04-20T21:20:03.949162+0000","last_became_active":"2024-04-20T21:20:03.943825+0000","last_became_peered":"2024-04-20T21:20:03.943825+0000","last_unstale":"2024-04-20T21:20:03.949162+0000","last_undegraded":"2024-04-20T21:20:03.949162+0000","last_fullsized":"2024-04-20T21:20:03.949162+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:57:58.076361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.952927+0000","last_change":"2024-04-20T21:20:03.949248+0000","last_active":"2024-04-20T21:20:04.952927+0000","last_peered":"2024-04-20T21:20:04.952927+0000","last_clean":"2024-04-20T21:20:04.952927+0000","last_became_active":"2024-04-20T21:20:03.948888+0000","last_became_peered":"2024-04-20T21:20:03.948888+0000","last_unstale":"2024-04-20T21:20:04.952927+0000","last_undegraded":"2024-04-20T21:20:04.952927+0000","last_fullsized":"2024-04-20T21:20:04.952927+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:31:29.580997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.960620+0000","last_change":"2024-04-20T21:20:03.951776+0000","last_active":"2024-04-20T21:20:03.960620+0000","last_peered":"2024-04-20T21:20:03.960620+0000","last_clean":"2024-04-20T21:20:03.960620+0000","last_became_active":"2024-04-20T21:20:03.951426+0000","last_became_peered":"2024-04-20T21:20:03.951426+0000","last_unstale":"2024-04-20T21:20:03.960620+0000","last_undegraded":"2024-04-20T21:20:03.960620+0000","last_fullsized":"2024-04-20T21:20:03.960620+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:42:14.328950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.965560+0000","last_change":"2024-04-20T21:20:03.941753+0000","last_active":"2024-04-20T21:20:04.965560+0000","last_peered":"2024-04-20T21:20:04.965560+0000","last_clean":"2024-04-20T21:20:04.965560+0000","last_became_active":"2024-04-20T21:20:03.937663+0000","last_became_peered":"2024-04-20T21:20:03.937663+0000","last_unstale":"2024-04-20T21:20:04.965560+0000","last_undegraded":"2024-04-20T21:20:04.965560+0000","last_fullsized":"2024-04-20T21:20:04.965560+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:31:02.145378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.957536+0000","last_change":"2024-04-20T21:20:03.957153+0000","last_active":"2024-04-20T21:20:04.957536+0000","last_peered":"2024-04-20T21:20:04.957536+0000","last_clean":"2024-04-20T21:20:04.957536+0000","last_became_active":"2024-04-20T21:20:03.956660+0000","last_became_peered":"2024-04-20T21:20:03.956660+0000","last_unstale":"2024-04-20T21:20:04.957536+0000","last_undegraded":"2024-04-20T21:20:04.957536+0000","last_fullsized":"2024-04-20T21:20:04.957536+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:00:13.844466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.957548+0000","last_change":"2024-04-20T21:20:03.940211+0000","last_active":"2024-04-20T21:20:04.957548+0000","last_peered":"2024-04-20T21:20:04.957548+0000","last_clean":"2024-04-20T21:20:04.957548+0000","last_became_active":"2024-04-20T21:20:03.939697+0000","last_became_peered":"2024-04-20T21:20:03.939697+0000","last_unstale":"2024-04-20T21:20:04.957548+0000","last_undegraded":"2024-04-20T21:20:04.957548+0000","last_fullsized":"2024-04-20T21:20:04.957548+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:56:29.421282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.957584+0000","last_change":"2024-04-20T21:20:03.962713+0000","last_active":"2024-04-20T21:20:04.957584+0000","last_peered":"2024-04-20T21:20:04.957584+0000","last_clean":"2024-04-20T21:20:04.957584+0000","last_became_active":"2024-04-20T21:20:03.962360+0000","last_became_peered":"2024-04-20T21:20:03.962360+0000","last_unstale":"2024-04-20T21:20:04.957584+0000","last_undegraded":"2024-04-20T21:20:04.957584+0000","last_fullsized":"2024-04-20T21:20:04.957584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:40:06.872797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.959566+0000","last_change":"2024-04-20T21:20:03.957492+0000","last_active":"2024-04-20T21:20:04.959566+0000","last_peered":"2024-04-20T21:20:04.959566+0000","last_clean":"2024-04-20T21:20:04.959566+0000","last_became_active":"2024-04-20T21:20:03.957216+0000","last_became_peered":"2024-04-20T21:20:03.957216+0000","last_unstale":"2024-04-20T21:20:04.959566+0000","last_undegraded":"2024-04-20T21:20:04.959566+0000","last_fullsized":"2024-04-20T21:20:04.959566+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:10:13.986163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.953672+0000","last_change":"2024-04-20T21:20:03.957221+0000","last_active":"2024-04-20T21:20:04.953672+0000","last_peered":"2024-04-20T21:20:04.953672+0000","last_clean":"2024-04-20T21:20:04.953672+0000","last_became_active":"2024-04-20T21:20:03.956891+0000","last_became_peered":"2024-04-20T21:20:03.956891+0000","last_unstale":"2024-04-20T21:20:04.953672+0000","last_undegraded":"2024-04-20T21:20:04.953672+0000","last_fullsized":"2024-04-20T21:20:04.953672+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:14:19.125873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.965745+0000","last_change":"2024-04-20T21:20:03.940435+0000","last_active":"2024-04-20T21:20:04.965745+0000","last_peered":"2024-04-20T21:20:04.965745+0000","last_clean":"2024-04-20T21:20:04.965745+0000","last_became_active":"2024-04-20T21:20:03.940160+0000","last_became_peered":"2024-04-20T21:20:03.940160+0000","last_unstale":"2024-04-20T21:20:04.965745+0000","last_undegraded":"2024-04-20T21:20:04.965745+0000","last_fullsized":"2024-04-20T21:20:04.965745+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:41:38.773748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.962569+0000","last_change":"2024-04-20T21:20:03.956429+0000","last_active":"2024-04-20T21:20:03.962569+0000","last_peered":"2024-04-20T21:20:03.962569+0000","last_clean":"2024-04-20T21:20:03.962569+0000","last_became_active":"2024-04-20T21:20:03.956167+0000","last_became_peered":"2024-04-20T21:20:03.956167+0000","last_unstale":"2024-04-20T21:20:03.962569+0000","last_undegraded":"2024-04-20T21:20:03.962569+0000","last_fullsized":"2024-04-20T21:20:03.962569+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:07:48.221573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.958021+0000","last_change":"2024-04-20T21:20:03.949152+0000","last_active":"2024-04-20T21:20:04.958021+0000","last_peered":"2024-04-20T21:20:04.958021+0000","last_clean":"2024-04-20T21:20:04.958021+0000","last_became_active":"2024-04-20T21:20:03.948736+0000","last_became_peered":"2024-04-20T21:20:03.948736+0000","last_unstale":"2024-04-20T21:20:04.958021+0000","last_undegraded":"2024-04-20T21:20:04.958021+0000","last_fullsized":"2024-04-20T21:20:04.958021+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:20:06.592684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.957535+0000","last_change":"2024-04-20T21:20:03.951534+0000","last_active":"2024-04-20T21:20:04.957535+0000","last_peered":"2024-04-20T21:20:04.957535+0000","last_clean":"2024-04-20T21:20:04.957535+0000","last_became_active":"2024-04-20T21:20:03.951227+0000","last_became_peered":"2024-04-20T21:20:03.951227+0000","last_unstale":"2024-04-20T21:20:04.957535+0000","last_undegraded":"2024-04-20T21:20:04.957535+0000","last_fullsized":"2024-04-20T21:20:04.957535+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:31:54.378016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.959322+0000","last_change":"2024-04-20T21:20:03.947006+0000","last_active":"2024-04-20T21:20:04.959322+0000","last_peered":"2024-04-20T21:20:04.959322+0000","last_clean":"2024-04-20T21:20:04.959322+0000","last_became_active":"2024-04-20T21:20:03.946596+0000","last_became_peered":"2024-04-20T21:20:03.946596+0000","last_unstale":"2024-04-20T21:20:04.959322+0000","last_undegraded":"2024-04-20T21:20:04.959322+0000","last_fullsized":"2024-04-20T21:20:04.959322+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:03:41.090616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.953365+0000","last_change":"2024-04-20T21:20:03.936009+0000","last_active":"2024-04-20T21:20:04.953365+0000","last_peered":"2024-04-20T21:20:04.953365+0000","last_clean":"2024-04-20T21:20:04.953365+0000","last_became_active":"2024-04-20T21:20:03.935496+0000","last_became_peered":"2024-04-20T21:20:03.935496+0000","last_unstale":"2024-04-20T21:20:04.953365+0000","last_undegraded":"2024-04-20T21:20:04.953365+0000","last_fullsized":"2024-04-20T21:20:04.953365+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:22:18.347547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.957099+0000","last_change":"2024-04-20T21:20:03.943653+0000","last_active":"2024-04-20T21:20:04.957099+0000","last_peered":"2024-04-20T21:20:04.957099+0000","last_clean":"2024-04-20T21:20:04.957099+0000","last_became_active":"2024-04-20T21:20:03.943348+0000","last_became_peered":"2024-04-20T21:20:03.943348+0000","last_unstale":"2024-04-20T21:20:04.957099+0000","last_undegraded":"2024-04-20T21:20:04.957099+0000","last_fullsized":"2024-04-20T21:20:04.957099+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:18:01.186077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.944891+0000","last_change":"2024-04-20T21:20:03.942646+0000","last_active":"2024-04-20T21:20:03.944891+0000","last_peered":"2024-04-20T21:20:03.944891+0000","last_clean":"2024-04-20T21:20:03.944891+0000","last_became_active":"2024-04-20T21:20:03.941871+0000","last_became_peered":"2024-04-20T21:20:03.941871+0000","last_unstale":"2024-04-20T21:20:03.944891+0000","last_undegraded":"2024-04-20T21:20:03.944891+0000","last_fullsized":"2024-04-20T21:20:03.944891+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:08:22.030102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.958769+0000","last_change":"2024-04-20T21:20:03.947375+0000","last_active":"2024-04-20T21:20:04.958769+0000","last_peered":"2024-04-20T21:20:04.958769+0000","last_clean":"2024-04-20T21:20:04.958769+0000","last_became_active":"2024-04-20T21:20:03.946107+0000","last_became_peered":"2024-04-20T21:20:03.946107+0000","last_unstale":"2024-04-20T21:20:04.958769+0000","last_undegraded":"2024-04-20T21:20:04.958769+0000","last_fullsized":"2024-04-20T21:20:04.958769+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:39:45.020282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.956167+0000","last_change":"2024-04-20T21:20:03.952087+0000","last_active":"2024-04-20T21:20:04.956167+0000","last_peered":"2024-04-20T21:20:04.956167+0000","last_clean":"2024-04-20T21:20:04.956167+0000","last_became_active":"2024-04-20T21:20:03.951745+0000","last_became_peered":"2024-04-20T21:20:03.951745+0000","last_unstale":"2024-04-20T21:20:04.956167+0000","last_undegraded":"2024-04-20T21:20:04.956167+0000","last_fullsized":"2024-04-20T21:20:04.956167+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:08:22.030102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.966617+0000","last_change":"2024-04-20T21:20:03.942401+0000","last_active":"2024-04-20T21:20:04.966617+0000","last_peered":"2024-04-20T21:20:04.966617+0000","last_clean":"2024-04-20T21:20:04.966617+0000","last_became_active":"2024-04-20T21:20:03.942083+0000","last_became_peered":"2024-04-20T21:20:03.942083+0000","last_unstale":"2024-04-20T21:20:04.966617+0000","last_undegraded":"2024-04-20T21:20:04.966617+0000","last_fullsized":"2024-04-20T21:20:04.966617+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:46:20.994921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.952682+0000","last_change":"2024-04-20T21:20:03.937678+0000","last_active":"2024-04-20T21:20:04.952682+0000","last_peered":"2024-04-20T21:20:04.952682+0000","last_clean":"2024-04-20T21:20:04.952682+0000","last_became_active":"2024-04-20T21:20:03.936838+0000","last_became_peered":"2024-04-20T21:20:03.936838+0000","last_unstale":"2024-04-20T21:20:04.952682+0000","last_undegraded":"2024-04-20T21:20:04.952682+0000","last_fullsized":"2024-04-20T21:20:04.952682+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:28:04.120963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:05.232158+0000","last_change":"2024-04-20T21:20:03.943607+0000","last_active":"2024-04-20T21:20:05.232158+0000","last_peered":"2024-04-20T21:20:05.232158+0000","last_clean":"2024-04-20T21:20:05.232158+0000","last_became_active":"2024-04-20T21:20:03.939432+0000","last_became_peered":"2024-04-20T21:20:03.939432+0000","last_unstale":"2024-04-20T21:20:05.232158+0000","last_undegraded":"2024-04-20T21:20:05.232158+0000","last_fullsized":"2024-04-20T21:20:05.232158+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:01:47.306517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"44'4","reported_seq":18,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.966652+0000","last_change":"2024-04-20T21:20:02.929219+0000","last_active":"2024-04-20T21:20:04.966652+0000","last_peered":"2024-04-20T21:20:04.966652+0000","last_clean":"2024-04-20T21:20:04.966652+0000","last_became_active":"2024-04-20T21:20:02.928868+0000","last_became_peered":"2024-04-20T21:20:02.928868+0000","last_unstale":"2024-04-20T21:20:04.966652+0000","last_undegraded":"2024-04-20T21:20:04.966652+0000","last_fullsized":"2024-04-20T21:20:04.966652+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:08:21.013641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.959520+0000","last_change":"2024-04-20T21:20:03.940579+0000","last_active":"2024-04-20T21:20:04.959520+0000","last_peered":"2024-04-20T21:20:04.959520+0000","last_clean":"2024-04-20T21:20:04.959520+0000","last_became_active":"2024-04-20T21:20:03.940123+0000","last_became_peered":"2024-04-20T21:20:03.940123+0000","last_unstale":"2024-04-20T21:20:04.959520+0000","last_undegraded":"2024-04-20T21:20:04.959520+0000","last_fullsized":"2024-04-20T21:20:04.959520+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:47:26.349120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.957950+0000","last_change":"2024-04-20T21:20:02.939856+0000","last_active":"2024-04-20T21:20:04.957950+0000","last_peered":"2024-04-20T21:20:04.957950+0000","last_clean":"2024-04-20T21:20:04.957950+0000","last_became_active":"2024-04-20T21:20:02.939491+0000","last_became_peered":"2024-04-20T21:20:02.939491+0000","last_unstale":"2024-04-20T21:20:04.957950+0000","last_undegraded":"2024-04-20T21:20:04.957950+0000","last_fullsized":"2024-04-20T21:20:04.957950+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:57:21.228630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.953239+0000","last_change":"2024-04-20T21:20:03.949355+0000","last_active":"2024-04-20T21:20:03.953239+0000","last_peered":"2024-04-20T21:20:03.953239+0000","last_clean":"2024-04-20T21:20:03.953239+0000","last_became_active":"2024-04-20T21:20:03.948932+0000","last_became_peered":"2024-04-20T21:20:03.948932+0000","last_unstale":"2024-04-20T21:20:03.953239+0000","last_undegraded":"2024-04-20T21:20:03.953239+0000","last_fullsized":"2024-04-20T21:20:03.953239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:18:01.186077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:05.232133+0000","last_change":"2024-04-20T21:20:02.941932+0000","last_active":"2024-04-20T21:20:05.232133+0000","last_peered":"2024-04-20T21:20:05.232133+0000","last_clean":"2024-04-20T21:20:05.232133+0000","last_became_active":"2024-04-20T21:20:02.941631+0000","last_became_peered":"2024-04-20T21:20:02.941631+0000","last_unstale":"2024-04-20T21:20:05.232133+0000","last_undegraded":"2024-04-20T21:20:05.232133+0000","last_fullsized":"2024-04-20T21:20:05.232133+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:31:53.361555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:05.232163+0000","last_change":"2024-04-20T21:20:03.937038+0000","last_active":"2024-04-20T21:20:05.232163+0000","last_peered":"2024-04-20T21:20:05.232163+0000","last_clean":"2024-04-20T21:20:05.232163+0000","last_became_active":"2024-04-20T21:20:03.936747+0000","last_became_peered":"2024-04-20T21:20:03.936747+0000","last_unstale":"2024-04-20T21:20:05.232163+0000","last_undegraded":"2024-04-20T21:20:05.232163+0000","last_fullsized":"2024-04-20T21:20:05.232163+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:47:26.349120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.975160+0000","last_change":"2024-04-20T21:20:02.949111+0000","last_active":"2024-04-20T21:20:03.975160+0000","last_peered":"2024-04-20T21:20:03.975160+0000","last_clean":"2024-04-20T21:20:03.975160+0000","last_became_active":"2024-04-20T21:20:02.948782+0000","last_became_peered":"2024-04-20T21:20:02.948782+0000","last_unstale":"2024-04-20T21:20:03.975160+0000","last_undegraded":"2024-04-20T21:20:03.975160+0000","last_fullsized":"2024-04-20T21:20:03.975160+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:57:57.059900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.956306+0000","last_change":"2024-04-20T21:20:03.937277+0000","last_active":"2024-04-20T21:20:04.956306+0000","last_peered":"2024-04-20T21:20:04.956306+0000","last_clean":"2024-04-20T21:20:04.956306+0000","last_became_active":"2024-04-20T21:20:03.936929+0000","last_became_peered":"2024-04-20T21:20:03.936929+0000","last_unstale":"2024-04-20T21:20:04.956306+0000","last_undegraded":"2024-04-20T21:20:04.956306+0000","last_fullsized":"2024-04-20T21:20:04.956306+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:57:58.076361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.965883+0000","last_change":"2024-04-20T21:20:02.939114+0000","last_active":"2024-04-20T21:20:04.965883+0000","last_peered":"2024-04-20T21:20:04.965883+0000","last_clean":"2024-04-20T21:20:04.965883+0000","last_became_active":"2024-04-20T21:20:02.938882+0000","last_became_peered":"2024-04-20T21:20:02.938882+0000","last_unstale":"2024-04-20T21:20:04.965883+0000","last_undegraded":"2024-04-20T21:20:04.965883+0000","last_fullsized":"2024-04-20T21:20:04.965883+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:56:28.404820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.962020+0000","last_change":"2024-04-20T21:20:03.957466+0000","last_active":"2024-04-20T21:20:03.962020+0000","last_peered":"2024-04-20T21:20:03.962020+0000","last_clean":"2024-04-20T21:20:03.962020+0000","last_became_active":"2024-04-20T21:20:03.956529+0000","last_became_peered":"2024-04-20T21:20:03.956529+0000","last_unstale":"2024-04-20T21:20:03.962020+0000","last_undegraded":"2024-04-20T21:20:03.962020+0000","last_fullsized":"2024-04-20T21:20:03.962020+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:03:41.090616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.956529+0000","last_change":"2024-04-20T21:20:02.945473+0000","last_active":"2024-04-20T21:20:04.956529+0000","last_peered":"2024-04-20T21:20:04.956529+0000","last_clean":"2024-04-20T21:20:04.956529+0000","last_became_active":"2024-04-20T21:20:02.945167+0000","last_became_peered":"2024-04-20T21:20:02.945167+0000","last_unstale":"2024-04-20T21:20:04.956529+0000","last_undegraded":"2024-04-20T21:20:04.956529+0000","last_fullsized":"2024-04-20T21:20:04.956529+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:31:53.361555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.964535+0000","last_change":"2024-04-20T21:20:03.961697+0000","last_active":"2024-04-20T21:20:03.964535+0000","last_peered":"2024-04-20T21:20:03.964535+0000","last_clean":"2024-04-20T21:20:03.964535+0000","last_became_active":"2024-04-20T21:20:03.958756+0000","last_became_peered":"2024-04-20T21:20:03.958756+0000","last_unstale":"2024-04-20T21:20:03.964535+0000","last_undegraded":"2024-04-20T21:20:03.964535+0000","last_fullsized":"2024-04-20T21:20:03.964535+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:41:17.519520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.937576+0000","last_change":"2024-04-20T21:20:02.938583+0000","last_active":"2024-04-20T21:20:03.937576+0000","last_peered":"2024-04-20T21:20:03.937576+0000","last_clean":"2024-04-20T21:20:03.937576+0000","last_became_active":"2024-04-20T21:20:02.938138+0000","last_became_peered":"2024-04-20T21:20:02.938138+0000","last_unstale":"2024-04-20T21:20:03.937576+0000","last_undegraded":"2024-04-20T21:20:03.937576+0000","last_fullsized":"2024-04-20T21:20:03.937576+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:40:00.140206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.957564+0000","last_change":"2024-04-20T21:20:03.950608+0000","last_active":"2024-04-20T21:20:04.957564+0000","last_peered":"2024-04-20T21:20:04.957564+0000","last_clean":"2024-04-20T21:20:04.957564+0000","last_became_active":"2024-04-20T21:20:03.950353+0000","last_became_peered":"2024-04-20T21:20:03.950353+0000","last_unstale":"2024-04-20T21:20:04.957564+0000","last_undegraded":"2024-04-20T21:20:04.957564+0000","last_fullsized":"2024-04-20T21:20:04.957564+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:57:22.245091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.953632+0000","last_change":"2024-04-20T21:20:02.931504+0000","last_active":"2024-04-20T21:20:03.953632+0000","last_peered":"2024-04-20T21:20:03.953632+0000","last_clean":"2024-04-20T21:20:03.953632+0000","last_became_active":"2024-04-20T21:20:02.931096+0000","last_became_peered":"2024-04-20T21:20:02.931096+0000","last_unstale":"2024-04-20T21:20:03.953632+0000","last_undegraded":"2024-04-20T21:20:03.953632+0000","last_fullsized":"2024-04-20T21:20:03.953632+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:57:21.228630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.957977+0000","last_change":"2024-04-20T21:20:03.961306+0000","last_active":"2024-04-20T21:20:04.957977+0000","last_peered":"2024-04-20T21:20:04.957977+0000","last_clean":"2024-04-20T21:20:04.957977+0000","last_became_active":"2024-04-20T21:20:03.960825+0000","last_became_peered":"2024-04-20T21:20:03.960825+0000","last_unstale":"2024-04-20T21:20:04.957977+0000","last_undegraded":"2024-04-20T21:20:04.957977+0000","last_fullsized":"2024-04-20T21:20:04.957977+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:18:23.071607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.958510+0000","last_change":"2024-04-20T21:20:02.941406+0000","last_active":"2024-04-20T21:20:04.958510+0000","last_peered":"2024-04-20T21:20:04.958510+0000","last_clean":"2024-04-20T21:20:04.958510+0000","last_became_active":"2024-04-20T21:20:02.941110+0000","last_became_peered":"2024-04-20T21:20:02.941110+0000","last_unstale":"2024-04-20T21:20:04.958510+0000","last_undegraded":"2024-04-20T21:20:04.958510+0000","last_fullsized":"2024-04-20T21:20:04.958510+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:51:42.526533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.956077+0000","last_change":"2024-04-20T21:20:03.950015+0000","last_active":"2024-04-20T21:20:03.956077+0000","last_peered":"2024-04-20T21:20:03.956077+0000","last_clean":"2024-04-20T21:20:03.956077+0000","last_became_active":"2024-04-20T21:20:03.949750+0000","last_became_peered":"2024-04-20T21:20:03.949750+0000","last_unstale":"2024-04-20T21:20:03.956077+0000","last_undegraded":"2024-04-20T21:20:03.956077+0000","last_fullsized":"2024-04-20T21:20:03.956077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:40:06.872797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.952116+0000","last_change":"2024-04-20T21:20:02.937766+0000","last_active":"2024-04-20T21:20:03.952116+0000","last_peered":"2024-04-20T21:20:03.952116+0000","last_clean":"2024-04-20T21:20:03.952116+0000","last_became_active":"2024-04-20T21:20:02.937439+0000","last_became_peered":"2024-04-20T21:20:02.937439+0000","last_unstale":"2024-04-20T21:20:03.952116+0000","last_undegraded":"2024-04-20T21:20:03.952116+0000","last_fullsized":"2024-04-20T21:20:03.952116+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:19:00.618007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.966232+0000","last_change":"2024-04-20T21:20:03.943716+0000","last_active":"2024-04-20T21:20:04.966232+0000","last_peered":"2024-04-20T21:20:04.966232+0000","last_clean":"2024-04-20T21:20:04.966232+0000","last_became_active":"2024-04-20T21:20:03.943453+0000","last_became_peered":"2024-04-20T21:20:03.943453+0000","last_unstale":"2024-04-20T21:20:04.966232+0000","last_undegraded":"2024-04-20T21:20:04.966232+0000","last_fullsized":"2024-04-20T21:20:04.966232+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:02:42.796359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"44'3","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.975202+0000","last_change":"2024-04-20T21:20:02.947542+0000","last_active":"2024-04-20T21:20:03.975202+0000","last_peered":"2024-04-20T21:20:03.975202+0000","last_clean":"2024-04-20T21:20:03.975202+0000","last_became_active":"2024-04-20T21:20:02.947222+0000","last_became_peered":"2024-04-20T21:20:02.947222+0000","last_unstale":"2024-04-20T21:20:03.975202+0000","last_undegraded":"2024-04-20T21:20:03.975202+0000","last_fullsized":"2024-04-20T21:20:03.975202+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:32:52.332721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.956955+0000","last_change":"2024-04-20T21:20:03.953218+0000","last_active":"2024-04-20T21:20:03.956955+0000","last_peered":"2024-04-20T21:20:03.956955+0000","last_clean":"2024-04-20T21:20:03.956955+0000","last_became_active":"2024-04-20T21:20:03.952840+0000","last_became_peered":"2024-04-20T21:20:03.952840+0000","last_unstale":"2024-04-20T21:20:03.956955+0000","last_undegraded":"2024-04-20T21:20:03.956955+0000","last_fullsized":"2024-04-20T21:20:03.956955+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:18:23.071607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"44'6","reported_seq":20,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.953369+0000","last_change":"2024-04-20T21:20:02.938292+0000","last_active":"2024-04-20T21:20:04.953369+0000","last_peered":"2024-04-20T21:20:04.953369+0000","last_clean":"2024-04-20T21:20:04.953369+0000","last_became_active":"2024-04-20T21:20:02.937716+0000","last_became_peered":"2024-04-20T21:20:02.937716+0000","last_unstale":"2024-04-20T21:20:04.953369+0000","last_undegraded":"2024-04-20T21:20:04.953369+0000","last_fullsized":"2024-04-20T21:20:04.953369+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:11:37.705235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.958164+0000","last_change":"2024-04-20T21:20:03.953844+0000","last_active":"2024-04-20T21:20:04.958164+0000","last_peered":"2024-04-20T21:20:04.958164+0000","last_clean":"2024-04-20T21:20:04.958164+0000","last_became_active":"2024-04-20T21:20:03.953385+0000","last_became_peered":"2024-04-20T21:20:03.953385+0000","last_unstale":"2024-04-20T21:20:04.958164+0000","last_undegraded":"2024-04-20T21:20:04.958164+0000","last_fullsized":"2024-04-20T21:20:04.958164+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:51:43.542994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.965992+0000","last_change":"2024-04-20T21:20:02.943078+0000","last_active":"2024-04-20T21:20:04.965992+0000","last_peered":"2024-04-20T21:20:04.965992+0000","last_clean":"2024-04-20T21:20:04.965992+0000","last_became_active":"2024-04-20T21:20:02.942796+0000","last_became_peered":"2024-04-20T21:20:02.942796+0000","last_unstale":"2024-04-20T21:20:04.965992+0000","last_undegraded":"2024-04-20T21:20:04.965992+0000","last_fullsized":"2024-04-20T21:20:04.965992+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:18:22.055146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.961992+0000","last_change":"2024-04-20T21:20:03.955471+0000","last_active":"2024-04-20T21:20:03.961992+0000","last_peered":"2024-04-20T21:20:03.961992+0000","last_clean":"2024-04-20T21:20:03.961992+0000","last_became_active":"2024-04-20T21:20:03.954995+0000","last_became_peered":"2024-04-20T21:20:03.954995+0000","last_unstale":"2024-04-20T21:20:03.961992+0000","last_undegraded":"2024-04-20T21:20:03.961992+0000","last_fullsized":"2024-04-20T21:20:03.961992+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:51:43.542994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.944771+0000","last_change":"2024-04-20T21:20:02.944818+0000","last_active":"2024-04-20T21:20:03.944771+0000","last_peered":"2024-04-20T21:20:03.944771+0000","last_clean":"2024-04-20T21:20:03.944771+0000","last_became_active":"2024-04-20T21:20:02.944545+0000","last_became_peered":"2024-04-20T21:20:02.944545+0000","last_unstale":"2024-04-20T21:20:03.944771+0000","last_undegraded":"2024-04-20T21:20:03.944771+0000","last_fullsized":"2024-04-20T21:20:03.944771+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:06:27.907994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.965533+0000","last_change":"2024-04-20T21:20:03.939784+0000","last_active":"2024-04-20T21:20:04.965533+0000","last_peered":"2024-04-20T21:20:04.965533+0000","last_clean":"2024-04-20T21:20:04.965533+0000","last_became_active":"2024-04-20T21:20:03.939468+0000","last_became_peered":"2024-04-20T21:20:03.939468+0000","last_unstale":"2024-04-20T21:20:04.965533+0000","last_undegraded":"2024-04-20T21:20:04.965533+0000","last_fullsized":"2024-04-20T21:20:04.965533+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:21:53.879172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.965752+0000","last_change":"2024-04-20T21:20:02.935626+0000","last_active":"2024-04-20T21:20:04.965752+0000","last_peered":"2024-04-20T21:20:04.965752+0000","last_clean":"2024-04-20T21:20:04.965752+0000","last_became_active":"2024-04-20T21:20:02.935287+0000","last_became_peered":"2024-04-20T21:20:02.935287+0000","last_unstale":"2024-04-20T21:20:04.965752+0000","last_undegraded":"2024-04-20T21:20:04.965752+0000","last_fullsized":"2024-04-20T21:20:04.965752+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:32:52.332721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.957158+0000","last_change":"2024-04-20T21:20:03.955305+0000","last_active":"2024-04-20T21:20:03.957158+0000","last_peered":"2024-04-20T21:20:03.957158+0000","last_clean":"2024-04-20T21:20:03.957158+0000","last_became_active":"2024-04-20T21:20:03.954925+0000","last_became_peered":"2024-04-20T21:20:03.954925+0000","last_unstale":"2024-04-20T21:20:03.957158+0000","last_undegraded":"2024-04-20T21:20:03.957158+0000","last_fullsized":"2024-04-20T21:20:03.957158+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:26:24.300536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"44'8","reported_seq":20,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.977133+0000","last_change":"2024-04-20T21:20:02.943948+0000","last_active":"2024-04-20T21:20:03.977133+0000","last_peered":"2024-04-20T21:20:03.977133+0000","last_clean":"2024-04-20T21:20:03.977133+0000","last_became_active":"2024-04-20T21:20:02.943624+0000","last_became_peered":"2024-04-20T21:20:02.943624+0000","last_unstale":"2024-04-20T21:20:03.977133+0000","last_undegraded":"2024-04-20T21:20:03.977133+0000","last_fullsized":"2024-04-20T21:20:03.977133+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:31:53.361555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.953529+0000","last_change":"2024-04-20T21:20:03.941174+0000","last_active":"2024-04-20T21:20:04.953529+0000","last_peered":"2024-04-20T21:20:04.953529+0000","last_clean":"2024-04-20T21:20:04.953529+0000","last_became_active":"2024-04-20T21:20:03.940872+0000","last_became_peered":"2024-04-20T21:20:03.940872+0000","last_unstale":"2024-04-20T21:20:04.953529+0000","last_undegraded":"2024-04-20T21:20:04.953529+0000","last_fullsized":"2024-04-20T21:20:04.953529+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:46:47.907080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.953425+0000","last_change":"2024-04-20T21:20:02.926662+0000","last_active":"2024-04-20T21:20:04.953425+0000","last_peered":"2024-04-20T21:20:04.953425+0000","last_clean":"2024-04-20T21:20:04.953425+0000","last_became_active":"2024-04-20T21:20:02.926377+0000","last_became_peered":"2024-04-20T21:20:02.926377+0000","last_unstale":"2024-04-20T21:20:04.953425+0000","last_undegraded":"2024-04-20T21:20:04.953425+0000","last_fullsized":"2024-04-20T21:20:04.953425+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:11:31.027478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.964353+0000","last_change":"2024-04-20T21:20:03.962320+0000","last_active":"2024-04-20T21:20:03.964353+0000","last_peered":"2024-04-20T21:20:03.964353+0000","last_clean":"2024-04-20T21:20:03.964353+0000","last_became_active":"2024-04-20T21:20:03.961988+0000","last_became_peered":"2024-04-20T21:20:03.961988+0000","last_unstale":"2024-04-20T21:20:03.964353+0000","last_undegraded":"2024-04-20T21:20:03.964353+0000","last_fullsized":"2024-04-20T21:20:03.964353+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:08:22.030102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.958146+0000","last_change":"2024-04-20T21:20:02.942589+0000","last_active":"2024-04-20T21:20:04.958146+0000","last_peered":"2024-04-20T21:20:04.958146+0000","last_clean":"2024-04-20T21:20:04.958146+0000","last_became_active":"2024-04-20T21:20:02.942306+0000","last_became_peered":"2024-04-20T21:20:02.942306+0000","last_unstale":"2024-04-20T21:20:04.958146+0000","last_undegraded":"2024-04-20T21:20:04.958146+0000","last_fullsized":"2024-04-20T21:20:04.958146+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:20:05.576223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.958686+0000","last_change":"2024-04-20T21:20:03.942731+0000","last_active":"2024-04-20T21:20:04.958686+0000","last_peered":"2024-04-20T21:20:04.958686+0000","last_clean":"2024-04-20T21:20:04.958686+0000","last_became_active":"2024-04-20T21:20:03.942281+0000","last_became_peered":"2024-04-20T21:20:03.942281+0000","last_unstale":"2024-04-20T21:20:04.958686+0000","last_undegraded":"2024-04-20T21:20:04.958686+0000","last_fullsized":"2024-04-20T21:20:04.958686+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:07:48.221573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"44'1","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.973061+0000","last_change":"2024-04-20T21:20:02.941701+0000","last_active":"2024-04-20T21:20:03.973061+0000","last_peered":"2024-04-20T21:20:03.973061+0000","last_clean":"2024-04-20T21:20:03.973061+0000","last_became_active":"2024-04-20T21:20:02.941400+0000","last_became_peered":"2024-04-20T21:20:02.941400+0000","last_unstale":"2024-04-20T21:20:03.973061+0000","last_undegraded":"2024-04-20T21:20:03.973061+0000","last_fullsized":"2024-04-20T21:20:03.973061+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:06:27.907994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.959371+0000","last_change":"2024-04-20T21:20:03.947103+0000","last_active":"2024-04-20T21:20:04.959371+0000","last_peered":"2024-04-20T21:20:04.959371+0000","last_clean":"2024-04-20T21:20:04.959371+0000","last_became_active":"2024-04-20T21:20:03.946087+0000","last_became_peered":"2024-04-20T21:20:03.946087+0000","last_unstale":"2024-04-20T21:20:04.959371+0000","last_undegraded":"2024-04-20T21:20:04.959371+0000","last_fullsized":"2024-04-20T21:20:04.959371+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:01:47.306517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.966332+0000","last_change":"2024-04-20T21:20:02.936129+0000","last_active":"2024-04-20T21:20:04.966332+0000","last_peered":"2024-04-20T21:20:04.966332+0000","last_clean":"2024-04-20T21:20:04.966332+0000","last_became_active":"2024-04-20T21:20:02.935380+0000","last_became_peered":"2024-04-20T21:20:02.935380+0000","last_unstale":"2024-04-20T21:20:04.966332+0000","last_undegraded":"2024-04-20T21:20:04.966332+0000","last_fullsized":"2024-04-20T21:20:04.966332+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:18:00.169616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.966033+0000","last_change":"2024-04-20T21:20:03.936823+0000","last_active":"2024-04-20T21:20:04.966033+0000","last_peered":"2024-04-20T21:20:04.966033+0000","last_clean":"2024-04-20T21:20:04.966033+0000","last_became_active":"2024-04-20T21:20:03.936483+0000","last_became_peered":"2024-04-20T21:20:03.936483+0000","last_unstale":"2024-04-20T21:20:04.966033+0000","last_undegraded":"2024-04-20T21:20:04.966033+0000","last_fullsized":"2024-04-20T21:20:04.966033+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:21:57.579803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.966192+0000","last_change":"2024-04-20T21:20:02.939226+0000","last_active":"2024-04-20T21:20:04.966192+0000","last_peered":"2024-04-20T21:20:04.966192+0000","last_clean":"2024-04-20T21:20:04.966192+0000","last_became_active":"2024-04-20T21:20:02.938943+0000","last_became_peered":"2024-04-20T21:20:02.938943+0000","last_unstale":"2024-04-20T21:20:04.966192+0000","last_undegraded":"2024-04-20T21:20:04.966192+0000","last_fullsized":"2024-04-20T21:20:04.966192+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:40:05.856336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:05.231753+0000","last_change":"2024-04-20T21:20:03.945415+0000","last_active":"2024-04-20T21:20:05.231753+0000","last_peered":"2024-04-20T21:20:05.231753+0000","last_clean":"2024-04-20T21:20:05.231753+0000","last_became_active":"2024-04-20T21:20:03.944297+0000","last_became_peered":"2024-04-20T21:20:03.944297+0000","last_unstale":"2024-04-20T21:20:05.231753+0000","last_undegraded":"2024-04-20T21:20:05.231753+0000","last_fullsized":"2024-04-20T21:20:05.231753+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:03:41.090616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.936097+0000","last_change":"2024-04-20T21:20:02.941304+0000","last_active":"2024-04-20T21:20:03.936097+0000","last_peered":"2024-04-20T21:20:03.936097+0000","last_clean":"2024-04-20T21:20:03.936097+0000","last_became_active":"2024-04-20T21:20:02.940993+0000","last_became_peered":"2024-04-20T21:20:02.940993+0000","last_unstale":"2024-04-20T21:20:03.936097+0000","last_undegraded":"2024-04-20T21:20:03.936097+0000","last_fullsized":"2024-04-20T21:20:03.936097+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:10:12.969702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.953078+0000","last_change":"2024-04-20T21:20:03.948314+0000","last_active":"2024-04-20T21:20:04.953078+0000","last_peered":"2024-04-20T21:20:04.953078+0000","last_clean":"2024-04-20T21:20:04.953078+0000","last_became_active":"2024-04-20T21:20:03.948018+0000","last_became_peered":"2024-04-20T21:20:03.948018+0000","last_unstale":"2024-04-20T21:20:04.953078+0000","last_undegraded":"2024-04-20T21:20:04.953078+0000","last_fullsized":"2024-04-20T21:20:04.953078+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:38:51.311160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.944505+0000","last_change":"2024-04-20T21:20:02.938238+0000","last_active":"2024-04-20T21:20:03.944505+0000","last_peered":"2024-04-20T21:20:03.944505+0000","last_clean":"2024-04-20T21:20:03.944505+0000","last_became_active":"2024-04-20T21:20:02.937929+0000","last_became_peered":"2024-04-20T21:20:02.937929+0000","last_unstale":"2024-04-20T21:20:03.944505+0000","last_undegraded":"2024-04-20T21:20:03.944505+0000","last_fullsized":"2024-04-20T21:20:03.944505+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:14:18.109412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.960775+0000","last_change":"2024-04-20T21:20:03.955409+0000","last_active":"2024-04-20T21:20:03.960775+0000","last_peered":"2024-04-20T21:20:03.960775+0000","last_clean":"2024-04-20T21:20:03.960775+0000","last_became_active":"2024-04-20T21:20:03.955047+0000","last_became_peered":"2024-04-20T21:20:03.955047+0000","last_unstale":"2024-04-20T21:20:03.960775+0000","last_undegraded":"2024-04-20T21:20:03.960775+0000","last_fullsized":"2024-04-20T21:20:03.960775+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:20:06.592684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.945304+0000","last_change":"2024-04-20T21:20:02.941315+0000","last_active":"2024-04-20T21:20:03.945304+0000","last_peered":"2024-04-20T21:20:03.945304+0000","last_clean":"2024-04-20T21:20:03.945304+0000","last_became_active":"2024-04-20T21:20:02.940992+0000","last_became_peered":"2024-04-20T21:20:02.940992+0000","last_unstale":"2024-04-20T21:20:03.945304+0000","last_undegraded":"2024-04-20T21:20:03.945304+0000","last_fullsized":"2024-04-20T21:20:03.945304+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:41:16.503059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.959584+0000","last_change":"2024-04-20T21:20:03.960412+0000","last_active":"2024-04-20T21:20:04.959584+0000","last_peered":"2024-04-20T21:20:04.959584+0000","last_clean":"2024-04-20T21:20:04.959584+0000","last_became_active":"2024-04-20T21:20:03.960094+0000","last_became_peered":"2024-04-20T21:20:03.960094+0000","last_unstale":"2024-04-20T21:20:04.959584+0000","last_undegraded":"2024-04-20T21:20:04.959584+0000","last_fullsized":"2024-04-20T21:20:04.959584+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:40:01.156667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"45'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.968965+0000","last_change":"2024-04-20T21:20:02.941534+0000","last_active":"2024-04-20T21:20:04.968965+0000","last_peered":"2024-04-20T21:20:04.968965+0000","last_clean":"2024-04-20T21:20:04.968965+0000","last_became_active":"2024-04-20T21:20:02.941313+0000","last_became_peered":"2024-04-20T21:20:02.941313+0000","last_unstale":"2024-04-20T21:20:04.968965+0000","last_undegraded":"2024-04-20T21:20:04.968965+0000","last_fullsized":"2024-04-20T21:20:04.968965+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:57:57.059900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.962119+0000","last_change":"2024-04-20T21:20:03.956916+0000","last_active":"2024-04-20T21:20:03.962119+0000","last_peered":"2024-04-20T21:20:03.962119+0000","last_clean":"2024-04-20T21:20:03.962119+0000","last_became_active":"2024-04-20T21:20:03.956585+0000","last_became_peered":"2024-04-20T21:20:03.956585+0000","last_unstale":"2024-04-20T21:20:03.962119+0000","last_undegraded":"2024-04-20T21:20:03.962119+0000","last_fullsized":"2024-04-20T21:20:03.962119+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:39:45.020282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.947101+0000","last_change":"2024-04-20T21:20:02.948112+0000","last_active":"2024-04-20T21:20:03.947101+0000","last_peered":"2024-04-20T21:20:03.947101+0000","last_clean":"2024-04-20T21:20:03.947101+0000","last_became_active":"2024-04-20T21:20:02.947665+0000","last_became_peered":"2024-04-20T21:20:02.947665+0000","last_unstale":"2024-04-20T21:20:03.947101+0000","last_undegraded":"2024-04-20T21:20:03.947101+0000","last_fullsized":"2024-04-20T21:20:03.947101+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:18:00.169616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.957408+0000","last_change":"2024-04-20T21:20:03.954130+0000","last_active":"2024-04-20T21:20:03.957408+0000","last_peered":"2024-04-20T21:20:03.957408+0000","last_clean":"2024-04-20T21:20:03.957408+0000","last_became_active":"2024-04-20T21:20:03.953236+0000","last_became_peered":"2024-04-20T21:20:03.953236+0000","last_unstale":"2024-04-20T21:20:03.957408+0000","last_undegraded":"2024-04-20T21:20:03.957408+0000","last_fullsized":"2024-04-20T21:20:03.957408+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:56:57.740183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:05.231627+0000","last_change":"2024-04-20T21:20:02.941938+0000","last_active":"2024-04-20T21:20:05.231627+0000","last_peered":"2024-04-20T21:20:05.231627+0000","last_clean":"2024-04-20T21:20:05.231627+0000","last_became_active":"2024-04-20T21:20:02.941603+0000","last_became_peered":"2024-04-20T21:20:02.941603+0000","last_unstale":"2024-04-20T21:20:05.231627+0000","last_undegraded":"2024-04-20T21:20:05.231627+0000","last_fullsized":"2024-04-20T21:20:05.231627+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:00:12.828005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.966414+0000","last_change":"2024-04-20T21:20:03.943596+0000","last_active":"2024-04-20T21:20:04.966414+0000","last_peered":"2024-04-20T21:20:04.966414+0000","last_clean":"2024-04-20T21:20:04.966414+0000","last_became_active":"2024-04-20T21:20:03.943345+0000","last_became_peered":"2024-04-20T21:20:03.943345+0000","last_unstale":"2024-04-20T21:20:04.966414+0000","last_undegraded":"2024-04-20T21:20:04.966414+0000","last_fullsized":"2024-04-20T21:20:04.966414+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:41:39.708030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.946605+0000","last_change":"2024-04-20T21:20:02.946571+0000","last_active":"2024-04-20T21:20:03.946605+0000","last_peered":"2024-04-20T21:20:03.946605+0000","last_clean":"2024-04-20T21:20:03.946605+0000","last_became_active":"2024-04-20T21:20:02.946286+0000","last_became_peered":"2024-04-20T21:20:02.946286+0000","last_unstale":"2024-04-20T21:20:03.946605+0000","last_undegraded":"2024-04-20T21:20:03.946605+0000","last_fullsized":"2024-04-20T21:20:03.946605+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:08:21.013641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.966394+0000","last_change":"2024-04-20T21:20:03.946626+0000","last_active":"2024-04-20T21:20:04.966394+0000","last_peered":"2024-04-20T21:20:04.966394+0000","last_clean":"2024-04-20T21:20:04.966394+0000","last_became_active":"2024-04-20T21:20:03.946366+0000","last_became_peered":"2024-04-20T21:20:03.946366+0000","last_unstale":"2024-04-20T21:20:04.966394+0000","last_undegraded":"2024-04-20T21:20:04.966394+0000","last_fullsized":"2024-04-20T21:20:04.966394+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:29:24.103308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.956687+0000","last_change":"2024-04-20T21:20:02.938506+0000","last_active":"2024-04-20T21:20:04.956687+0000","last_peered":"2024-04-20T21:20:04.956687+0000","last_clean":"2024-04-20T21:20:04.956687+0000","last_became_active":"2024-04-20T21:20:02.938164+0000","last_became_peered":"2024-04-20T21:20:02.938164+0000","last_unstale":"2024-04-20T21:20:04.956687+0000","last_undegraded":"2024-04-20T21:20:04.956687+0000","last_fullsized":"2024-04-20T21:20:04.956687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:00:12.828005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.949531+0000","last_change":"2024-04-20T21:20:03.947582+0000","last_active":"2024-04-20T21:20:03.949531+0000","last_peered":"2024-04-20T21:20:03.949531+0000","last_clean":"2024-04-20T21:20:03.949531+0000","last_became_active":"2024-04-20T21:20:03.947299+0000","last_became_peered":"2024-04-20T21:20:03.947299+0000","last_unstale":"2024-04-20T21:20:03.949531+0000","last_undegraded":"2024-04-20T21:20:03.949531+0000","last_fullsized":"2024-04-20T21:20:03.949531+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:32:53.349182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:05.231300+0000","last_change":"2024-04-20T21:20:02.928908+0000","last_active":"2024-04-20T21:20:05.231300+0000","last_peered":"2024-04-20T21:20:05.231300+0000","last_clean":"2024-04-20T21:20:05.231300+0000","last_became_active":"2024-04-20T21:20:02.928634+0000","last_became_peered":"2024-04-20T21:20:02.928634+0000","last_unstale":"2024-04-20T21:20:05.231300+0000","last_undegraded":"2024-04-20T21:20:05.231300+0000","last_fullsized":"2024-04-20T21:20:05.231300+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:57:21.228630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-20T21:20:04.956095+0000","last_change":"2024-04-20T21:20:03.934008+0000","last_active":"2024-04-20T21:20:04.956095+0000","last_peered":"2024-04-20T21:20:04.956095+0000","last_clean":"2024-04-20T21:20:04.956095+0000","last_became_active":"2024-04-20T21:20:03.933452+0000","last_became_peered":"2024-04-20T21:20:03.933452+0000","last_unstale":"2024-04-20T21:20:04.956095+0000","last_undegraded":"2024-04-20T21:20:04.956095+0000","last_fullsized":"2024-04-20T21:20:04.956095+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:41:17.519520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.953353+0000","last_change":"2024-04-20T21:20:02.932961+0000","last_active":"2024-04-20T21:20:03.953353+0000","last_peered":"2024-04-20T21:20:03.953353+0000","last_clean":"2024-04-20T21:20:03.953353+0000","last_became_active":"2024-04-20T21:20:02.932624+0000","last_became_peered":"2024-04-20T21:20:02.932624+0000","last_unstale":"2024-04-20T21:20:03.953353+0000","last_undegraded":"2024-04-20T21:20:03.953353+0000","last_fullsized":"2024-04-20T21:20:03.953353+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:00:12.828005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.962152+0000","last_change":"2024-04-20T21:20:03.958692+0000","last_active":"2024-04-20T21:20:03.962152+0000","last_peered":"2024-04-20T21:20:03.962152+0000","last_clean":"2024-04-20T21:20:03.962152+0000","last_became_active":"2024-04-20T21:20:03.958420+0000","last_became_peered":"2024-04-20T21:20:03.958420+0000","last_unstale":"2024-04-20T21:20:03.962152+0000","last_undegraded":"2024-04-20T21:20:03.962152+0000","last_fullsized":"2024-04-20T21:20:03.962152+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:02.892034+0000","last_clean_scrub_stamp":"2024-04-20T21:20:02.892034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:01:47.306517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"44'2","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.975131+0000","last_change":"2024-04-20T21:20:02.943225+0000","last_active":"2024-04-20T21:20:03.975131+0000","last_peered":"2024-04-20T21:20:03.975131+0000","last_clean":"2024-04-20T21:20:03.975131+0000","last_became_active":"2024-04-20T21:20:02.942904+0000","last_became_peered":"2024-04-20T21:20:02.942904+0000","last_unstale":"2024-04-20T21:20:03.975131+0000","last_undegraded":"2024-04-20T21:20:03.975131+0000","last_fullsized":"2024-04-20T21:20:03.975131+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:20:01.875572+0000","last_clean_scrub_stamp":"2024-04-20T21:20:01.875572+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:41:16.503059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":211,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-20T21:20:03.945359+0000","last_change":"2024-04-20T21:17:43.059637+0000","last_active":"2024-04-20T21:20:03.945359+0000","last_peered":"2024-04-20T21:20:03.945359+0000","last_clean":"2024-04-20T21:20:03.945359+0000","last_became_active":"2024-04-20T21:17:43.056207+0000","last_became_peered":"2024-04-20T21:17:43.056207+0000","last_unstale":"2024-04-20T21:20:03.945359+0000","last_undegraded":"2024-04-20T21:20:03.945359+0000","last_fullsized":"2024-04-20T21:20:03.945359+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:41.036290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:41.036290+0000","last_clean_scrub_stamp":"2024-04-20T21:17:41.036290+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:58:51.091337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574879,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1114112,"data_stored":799273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27540,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":524288,"data_stored":209487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574882,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":520192,"data_stored":208885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":540672,"data_stored":216727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27544,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":528384,"data_stored":209453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":544768,"data_stored":216749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574880,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":520192,"data_stored":208885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28108,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343732,"statfs":{"total":96636764160,"available":96607981568,"internally_reserved":0,"allocated":1114112,"data_stored":799217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T21:20:08.806 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-20T21:20:08.806 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-20T21:20:09.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.052+0000 7f7182c00640 1 Processor -- start 2024-04-20T21:20:09.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.053+0000 7f7182c00640 1 -- start start 2024-04-20T21:20:09.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.054+0000 7f7182c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f717c0aee50 0x7f717c06b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:09.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.054+0000 7f7182c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f717c0b4530 0x7f717c06bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:09.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.054+0000 7f7182c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f717c0adb10 con 0x7f717c15eb30 2024-04-20T21:20:09.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.054+0000 7f7182c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f717c0af7d0 con 0x7f717c0b4530 2024-04-20T21:20:09.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.054+0000 7f7182c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f717c0addb0 con 0x7f717c0aee50 2024-04-20T21:20:09.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.055+0000 7f7180c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f717c0b4530 0x7f717c06bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:09.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.055+0000 7f7180c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f717c0b4530 0x7f717c06bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45596/0 (socket says 172.21.15.86:45596) 2024-04-20T21:20:09.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.055+0000 7f7180c00640 1 -- 172.21.15.86:0/1175415719 learned_addr learned my addr 172.21.15.86:0/1175415719 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:09.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.055+0000 7f7173e00640 1 --2- 172.21.15.86:0/1175415719 >> v2:172.21.15.144:3300/0 conn(0x7f717c0aee50 0x7f717c06b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:09.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.055+0000 7f7180c00640 1 -- 172.21.15.86:0/1175415719 >> v2:172.21.15.144:3300/0 conn(0x7f717c0aee50 msgr2=0x7f717c06b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:09.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.055+0000 7f7180c00640 1 --2- 172.21.15.86:0/1175415719 >> v2:172.21.15.144:3300/0 conn(0x7f717c0aee50 0x7f717c06b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:09.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.055+0000 7f7180c00640 1 -- 172.21.15.86:0/1175415719 >> v1:172.21.15.144:6789/0 conn(0x7f717c15eb30 legacy=0x7f717c0b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:09.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.055+0000 7f7180c00640 1 -- 172.21.15.86:0/1175415719 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f717c0ae090 con 0x7f717c0b4530 2024-04-20T21:20:09.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.057+0000 7f7180c00640 1 --2- 172.21.15.86:0/1175415719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f717c0b4530 0x7f717c06bec0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f717c070040 tx=0x7f717402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7b5cf5f63af3ddc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:09.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.057+0000 7f717b400640 1 -- 172.21.15.86:0/1175415719 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717402ca80 con 0x7f717c0b4530 2024-04-20T21:20:09.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.057+0000 7f717b400640 1 -- 172.21.15.86:0/1175415719 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f717402cc20 con 0x7f717c0b4530 2024-04-20T21:20:09.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.057+0000 7f717b400640 1 -- 172.21.15.86:0/1175415719 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71740343f0 con 0x7f717c0b4530 2024-04-20T21:20:09.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.058+0000 7f7182c00640 1 -- 172.21.15.86:0/1175415719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f717c0b4530 msgr2=0x7f717c06bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:09.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.058+0000 7f7182c00640 1 --2- 172.21.15.86:0/1175415719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f717c0b4530 0x7f717c06bec0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f717c070040 tx=0x7f717402bee0 comp rx=0 tx=0).stop 2024-04-20T21:20:09.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.058+0000 7f7182c00640 1 -- 172.21.15.86:0/1175415719 shutdown_connections 2024-04-20T21:20:09.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.058+0000 7f7182c00640 1 --2- 172.21.15.86:0/1175415719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f717c0b4530 0x7f717c06bec0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:09.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.058+0000 7f7182c00640 1 --2- 172.21.15.86:0/1175415719 >> v2:172.21.15.144:3300/0 conn(0x7f717c0aee50 0x7f717c06b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:09.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.058+0000 7f7182c00640 1 -- 172.21.15.86:0/1175415719 >> 172.21.15.86:0/1175415719 conn(0x7f717c0b2110 msgr2=0x7f717c0b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:09.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.058+0000 7f7182c00640 1 -- 172.21.15.86:0/1175415719 shutdown_connections 2024-04-20T21:20:09.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.058+0000 7f7182c00640 1 -- 172.21.15.86:0/1175415719 wait complete. 2024-04-20T21:20:09.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.060+0000 7f7182c00640 1 Processor -- start 2024-04-20T21:20:09.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.060+0000 7f7182c00640 1 -- start start 2024-04-20T21:20:09.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.061+0000 7f7182c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f717c0aee50 0x7f717c141dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:09.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.061+0000 7f7182c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f717c0b4530 0x7f717c0ceee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:09.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.061+0000 7f7182c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f717c15eb30 0x7f717c0cf420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:09.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.061+0000 7f7182c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f717c0ae930 con 0x7f717c15eb30 2024-04-20T21:20:09.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.061+0000 7f7182c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f717c0ae650 con 0x7f717c0b4530 2024-04-20T21:20:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.061+0000 7f7182c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f717c06e160 con 0x7f717c0aee50 2024-04-20T21:20:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.061+0000 7f7180c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f717c15eb30 0x7f717c0cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.061+0000 7f7180c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f717c15eb30 0x7f717c0cf420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45610/0 (socket says 172.21.15.86:45610) 2024-04-20T21:20:09.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.061+0000 7f7180c00640 1 -- 172.21.15.86:0/2351056959 learned_addr learned my addr 172.21.15.86:0/2351056959 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:09.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.062+0000 7f7173e00640 1 --2- 172.21.15.86:0/2351056959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f717c0b4530 0x7f717c0ceee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:09.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.062+0000 7f717be00640 1 --2- 172.21.15.86:0/2351056959 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f717c0aee50 0x7f717c141dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:09.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.062+0000 7f7180c00640 1 -- 172.21.15.86:0/2351056959 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f717c0aee50 msgr2=0x7f717c141dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:09.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.062+0000 7f7180c00640 1 --2- 172.21.15.86:0/2351056959 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f717c0aee50 0x7f717c141dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:09.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.062+0000 7f7180c00640 1 -- 172.21.15.86:0/2351056959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f717c0b4530 msgr2=0x7f717c0ceee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:09.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.062+0000 7f7180c00640 1 --2- 172.21.15.86:0/2351056959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f717c0b4530 0x7f717c0ceee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:09.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.062+0000 7f7180c00640 1 -- 172.21.15.86:0/2351056959 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f717c0cd710 con 0x7f717c15eb30 2024-04-20T21:20:09.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.062+0000 7f7173e00640 1 --2- 172.21.15.86:0/2351056959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f717c0b4530 0x7f717c0ceee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:20:09.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.062+0000 7f717be00640 1 --2- 172.21.15.86:0/2351056959 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f717c0aee50 0x7f717c141dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:20:09.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.062+0000 7f7180c00640 1 --2- 172.21.15.86:0/2351056959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f717c15eb30 0x7f717c0cf420 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f7174036040 tx=0x7f7174004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:09.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.062+0000 7f7178c00640 1 -- 172.21.15.86:0/2351056959 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7174043020 con 0x7f717c15eb30 2024-04-20T21:20:09.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.062+0000 7f7178c00640 1 -- 172.21.15.86:0/2351056959 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f717403e450 con 0x7f717c15eb30 2024-04-20T21:20:09.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.062+0000 7f7178c00640 1 -- 172.21.15.86:0/2351056959 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7174009510 con 0x7f717c15eb30 2024-04-20T21:20:09.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.062+0000 7f7182c00640 1 -- 172.21.15.86:0/2351056959 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f717c0cd9f0 con 0x7f717c15eb30 2024-04-20T21:20:09.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.063+0000 7f7182c00640 1 -- 172.21.15.86:0/2351056959 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f717c0cdfb0 con 0x7f717c15eb30 2024-04-20T21:20:09.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.064+0000 7f7182c00640 1 -- 172.21.15.86:0/2351056959 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f717c0b4350 con 0x7f717c15eb30 2024-04-20T21:20:09.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.065+0000 7f7178c00640 1 -- 172.21.15.86:0/2351056959 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7174046020 con 0x7f717c15eb30 2024-04-20T21:20:09.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.066+0000 7f7178c00640 1 --2- 172.21.15.86:0/2351056959 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f715806c920 0x7f715806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:09.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.066+0000 7f7178c00640 1 -- 172.21.15.86:0/2351056959 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f71740ae340 con 0x7f717c15eb30 2024-04-20T21:20:09.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.066+0000 7f717be00640 1 --2- 172.21.15.86:0/2351056959 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f715806c920 0x7f715806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:09.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.066+0000 7f717be00640 1 --2- 172.21.15.86:0/2351056959 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f715806c920 0x7f715806ede0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f716c00a830 tx=0x7f716c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:09.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.070+0000 7f7178c00640 1 -- 172.21.15.86:0/2351056959 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f717c0b4350 con 0x7f717c15eb30 2024-04-20T21:20:09.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.336+0000 7f7182c00640 1 -- 172.21.15.86:0/2351056959 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f717c0689e0 con 0x7f717c15eb30 2024-04-20T21:20:09.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.340+0000 7f7178c00640 1 -- 172.21.15.86:0/2351056959 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) ==== 190+0+147 (secure 0 0 0) 0x7f717404d030 con 0x7f717c15eb30 2024-04-20T21:20:09.339 INFO:teuthology.orchestra.run.smithi086.stdout:[client.0] 2024-04-20T21:20:09.339 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQDBMCRmHWxyKRAANzhRi4eajp68h/jWkhFMnw== 2024-04-20T21:20:09.339 INFO:teuthology.orchestra.run.smithi086.stdout: caps mds = "allow" 2024-04-20T21:20:09.339 INFO:teuthology.orchestra.run.smithi086.stdout: caps mon = "allow r" 2024-04-20T21:20:09.339 INFO:teuthology.orchestra.run.smithi086.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-20T21:20:09.340 INFO:teuthology.orchestra.run.smithi086.stderr:updated caps for client.0 2024-04-20T21:20:09.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.342+0000 7f7182c00640 1 -- 172.21.15.86:0/2351056959 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f715806c920 msgr2=0x7f715806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:09.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.342+0000 7f7182c00640 1 --2- 172.21.15.86:0/2351056959 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f715806c920 0x7f715806ede0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f716c00a830 tx=0x7f716c008040 comp rx=0 tx=0).stop 2024-04-20T21:20:09.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.342+0000 7f7182c00640 1 -- 172.21.15.86:0/2351056959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f717c15eb30 msgr2=0x7f717c0cf420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:09.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.342+0000 7f7182c00640 1 --2- 172.21.15.86:0/2351056959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f717c15eb30 0x7f717c0cf420 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f7174036040 tx=0x7f7174004a40 comp rx=0 tx=0).stop 2024-04-20T21:20:09.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.342+0000 7f7182c00640 1 -- 172.21.15.86:0/2351056959 shutdown_connections 2024-04-20T21:20:09.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.342+0000 7f7182c00640 1 --2- 172.21.15.86:0/2351056959 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f715806c920 0x7f715806ede0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:09.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.342+0000 7f7182c00640 1 --2- 172.21.15.86:0/2351056959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f717c15eb30 0x7f717c0cf420 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:09.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.342+0000 7f7182c00640 1 --2- 172.21.15.86:0/2351056959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f717c0b4530 0x7f717c0ceee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:09.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.342+0000 7f7182c00640 1 --2- 172.21.15.86:0/2351056959 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f717c0aee50 0x7f717c141dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:09.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.342+0000 7f7182c00640 1 -- 172.21.15.86:0/2351056959 >> 172.21.15.86:0/2351056959 conn(0x7f717c0b2110 msgr2=0x7f717c0b3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:09.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.343+0000 7f7182c00640 1 -- 172.21.15.86:0/2351056959 shutdown_connections 2024-04-20T21:20:09.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.343+0000 7f7182c00640 1 -- 172.21.15.86:0/2351056959 wait complete. 2024-04-20T21:20:09.360 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:20:09.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.557+0000 7f8523600640 1 Processor -- start 2024-04-20T21:20:09.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.557+0000 7f8523600640 1 -- start start 2024-04-20T21:20:09.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.558+0000 7f8523600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f851c0cd170 0x7f851c06ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:09.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.558+0000 7f8523600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f851c0cd600 0x7f851c06f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:09.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.558+0000 7f8523600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f851c0cdb50 con 0x7f851c144e90 2024-04-20T21:20:09.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.558+0000 7f8523600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f851c145270 con 0x7f851c0cd170 2024-04-20T21:20:09.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.558+0000 7f8523600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f851c0b3e50 con 0x7f851c0cd600 2024-04-20T21:20:09.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.559+0000 7f8520c00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f851c144e90 0x7f851c0b4930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39664/0 (socket says 172.21.15.86:39664) 2024-04-20T21:20:09.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.559+0000 7f8520c00640 1 -- 172.21.15.86:0/1613668378 learned_addr learned my addr 172.21.15.86:0/1613668378 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:20:09.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.559+0000 7f8513e00640 1 --2- 172.21.15.86:0/1613668378 >> v2:172.21.15.86:3300/0 conn(0x7f851c0cd170 0x7f851c06ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:09.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.559+0000 7f8521600640 1 --2- 172.21.15.86:0/1613668378 >> v2:172.21.15.144:3300/0 conn(0x7f851c0cd600 0x7f851c06f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:09.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.559+0000 7f851be00640 1 -- 172.21.15.86:0/1613668378 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1151159240 0 0) 0x7f851c0cdb50 con 0x7f851c144e90 2024-04-20T21:20:09.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.559+0000 7f851be00640 1 -- 172.21.15.86:0/1613668378 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8500003520 con 0x7f851c144e90 2024-04-20T21:20:09.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.559+0000 7f8513e00640 1 -- 172.21.15.86:0/1613668378 >> v2:172.21.15.144:3300/0 conn(0x7f851c0cd600 msgr2=0x7f851c06f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:09.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.559+0000 7f8513e00640 1 --2- 172.21.15.86:0/1613668378 >> v2:172.21.15.144:3300/0 conn(0x7f851c0cd600 0x7f851c06f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:09.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.559+0000 7f8513e00640 1 -- 172.21.15.86:0/1613668378 >> v1:172.21.15.86:6789/0 conn(0x7f851c144e90 legacy=0x7f851c0b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:09.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.559+0000 7f8513e00640 1 -- 172.21.15.86:0/1613668378 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f851c0b4130 con 0x7f851c0cd170 2024-04-20T21:20:09.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.560+0000 7f8513e00640 1 --2- 172.21.15.86:0/1613668378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f851c0cd170 0x7f851c06ed40 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f851c0b5320 tx=0x7f850402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=198b5b4731fe5506 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:09.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.561+0000 7f851be00640 1 -- 172.21.15.86:0/1613668378 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85040315a0 con 0x7f851c0cd170 2024-04-20T21:20:09.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.561+0000 7f851be00640 1 -- 172.21.15.86:0/1613668378 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8504037070 con 0x7f851c0cd170 2024-04-20T21:20:09.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.561+0000 7f851be00640 1 -- 172.21.15.86:0/1613668378 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8504031960 con 0x7f851c0cd170 2024-04-20T21:20:09.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.561+0000 7f8523600640 1 -- 172.21.15.86:0/1613668378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f851c0cd170 msgr2=0x7f851c06ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:09.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.561+0000 7f8523600640 1 --2- 172.21.15.86:0/1613668378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f851c0cd170 0x7f851c06ed40 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f851c0b5320 tx=0x7f850402b840 comp rx=0 tx=0).stop 2024-04-20T21:20:09.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.561+0000 7f8523600640 1 -- 172.21.15.86:0/1613668378 shutdown_connections 2024-04-20T21:20:09.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.561+0000 7f8523600640 1 --2- 172.21.15.86:0/1613668378 >> v2:172.21.15.144:3300/0 conn(0x7f851c0cd600 0x7f851c06f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:09.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.561+0000 7f8523600640 1 --2- 172.21.15.86:0/1613668378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f851c0cd170 0x7f851c06ed40 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:09.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.561+0000 7f8523600640 1 -- 172.21.15.86:0/1613668378 >> 172.21.15.86:0/1613668378 conn(0x7f851c0b2360 msgr2=0x7f851c0b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:09.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.562+0000 7f8523600640 1 -- 172.21.15.86:0/1613668378 shutdown_connections 2024-04-20T21:20:09.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.562+0000 7f8523600640 1 -- 172.21.15.86:0/1613668378 wait complete. 2024-04-20T21:20:09.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.563+0000 7f8523600640 1 Processor -- start 2024-04-20T21:20:09.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.563+0000 7f8523600640 1 -- start start 2024-04-20T21:20:09.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.564+0000 7f8523600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f851c0cd170 0x7f851c0caa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:09.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.564+0000 7f8523600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f851c0cd600 0x7f851c0aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:09.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.564+0000 7f8523600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f851c144e90 0x7f851c0c8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:09.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.564+0000 7f8523600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f851c0d0190 con 0x7f851c0cd170 2024-04-20T21:20:09.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.564+0000 7f8523600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f851c0d0010 con 0x7f851c144e90 2024-04-20T21:20:09.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.564+0000 7f8523600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f851c15bb50 con 0x7f851c0cd600 2024-04-20T21:20:09.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.565+0000 7f8520c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f851c0cd170 0x7f851c0caa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:09.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.565+0000 7f8520c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f851c0cd170 0x7f851c0caa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45638/0 (socket says 172.21.15.86:45638) 2024-04-20T21:20:09.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.565+0000 7f8520c00640 1 -- 172.21.15.86:0/2402118194 learned_addr learned my addr 172.21.15.86:0/2402118194 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:09.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.565+0000 7f8520c00640 1 -- 172.21.15.86:0/2402118194 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f851c0cd600 msgr2=0x7f851c0aeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:09.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.565+0000 7f8520c00640 1 --2- 172.21.15.86:0/2402118194 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f851c0cd600 0x7f851c0aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:09.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.565+0000 7f8520c00640 1 -- 172.21.15.86:0/2402118194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f851c144e90 msgr2=0x7f851c0c8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:09.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.565+0000 7f8520c00640 1 --2- 172.21.15.86:0/2402118194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f851c144e90 0x7f851c0c8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:09.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.565+0000 7f8520c00640 1 -- 172.21.15.86:0/2402118194 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f851c0cc3a0 con 0x7f851c0cd170 2024-04-20T21:20:09.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.565+0000 7f8520c00640 1 --2- 172.21.15.86:0/2402118194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f851c0cd170 0x7f851c0caa40 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f850c00c390 tx=0x7f850c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:09.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.566+0000 7f8519600640 1 -- 172.21.15.86:0/2402118194 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850c017070 con 0x7f851c0cd170 2024-04-20T21:20:09.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.566+0000 7f8519600640 1 -- 172.21.15.86:0/2402118194 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f850c005ce0 con 0x7f851c0cd170 2024-04-20T21:20:09.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.566+0000 7f8519600640 1 -- 172.21.15.86:0/2402118194 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850c0072f0 con 0x7f851c0cd170 2024-04-20T21:20:09.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.566+0000 7f8523600640 1 -- 172.21.15.86:0/2402118194 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f851c0cb300 con 0x7f851c0cd170 2024-04-20T21:20:09.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.566+0000 7f8523600640 1 -- 172.21.15.86:0/2402118194 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f851c0cb8c0 con 0x7f851c0cd170 2024-04-20T21:20:09.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.567+0000 7f8523600640 1 -- 172.21.15.86:0/2402118194 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f851c13f4d0 con 0x7f851c0cd170 2024-04-20T21:20:09.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.568+0000 7f8519600640 1 -- 172.21.15.86:0/2402118194 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f850c01a070 con 0x7f851c0cd170 2024-04-20T21:20:09.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.569+0000 7f8519600640 1 --2- 172.21.15.86:0/2402118194 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f84f806c8f0 0x7f84f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:09.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.569+0000 7f8519600640 1 -- 172.21.15.86:0/2402118194 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f850c08b080 con 0x7f851c0cd170 2024-04-20T21:20:09.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.569+0000 7f8513e00640 1 --2- 172.21.15.86:0/2402118194 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f84f806c8f0 0x7f84f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:09.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.570+0000 7f8513e00640 1 --2- 172.21.15.86:0/2402118194 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f84f806c8f0 0x7f84f806edb0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8504036000 tx=0x7f8504030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:09.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.573+0000 7f8519600640 1 -- 172.21.15.86:0/2402118194 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f850c059440 con 0x7f851c0cd170 2024-04-20T21:20:09.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.811+0000 7f8523600640 1 -- 172.21.15.86:0/2402118194 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f851c0689e0 con 0x7f851c0cd170 2024-04-20T21:20:09.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.812+0000 7f8519600640 1 -- 172.21.15.86:0/2402118194 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5199 (secure 0 0 0) 0x7f850c058d40 con 0x7f851c0cd170 2024-04-20T21:20:09.813 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:20:09.813 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5044,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/1013106052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":1013106052},{"type":"v1","addr":"172.21.15.144:6833","nonce":1013106052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5047,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/1073576945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1073576945},{"type":"v1","addr":"172.21.15.144:6835","nonce":1073576945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/821548012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":821548012},{"type":"v1","addr":"172.21.15.86:6835","nonce":821548012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:20:03.369281+0000","modified":"2024-04-20T21:20:06.438592+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5691},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5691":{"gid":5691,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/712628984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":712628984},{"type":"v1","addr":"172.21.15.86:6837","nonce":712628984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5691,"qdb_cluster":[5691]},"id":3}]} 2024-04-20T21:20:09.813 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 24 2024-04-20T21:20:09.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.815+0000 7f8523600640 1 -- 172.21.15.86:0/2402118194 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f84f806c8f0 msgr2=0x7f84f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:09.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.815+0000 7f8523600640 1 --2- 172.21.15.86:0/2402118194 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f84f806c8f0 0x7f84f806edb0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8504036000 tx=0x7f8504030000 comp rx=0 tx=0).stop 2024-04-20T21:20:09.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.815+0000 7f8523600640 1 -- 172.21.15.86:0/2402118194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f851c0cd170 msgr2=0x7f851c0caa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:09.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.815+0000 7f8523600640 1 --2- 172.21.15.86:0/2402118194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f851c0cd170 0x7f851c0caa40 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f850c00c390 tx=0x7f850c00c850 comp rx=0 tx=0).stop 2024-04-20T21:20:09.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.815+0000 7f8523600640 1 -- 172.21.15.86:0/2402118194 shutdown_connections 2024-04-20T21:20:09.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.815+0000 7f8523600640 1 --2- 172.21.15.86:0/2402118194 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f84f806c8f0 0x7f84f806edb0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:09.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.815+0000 7f8523600640 1 --2- 172.21.15.86:0/2402118194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f851c144e90 0x7f851c0c8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:09.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.815+0000 7f8523600640 1 --2- 172.21.15.86:0/2402118194 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f851c0cd600 0x7f851c0aeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:09.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.815+0000 7f8523600640 1 --2- 172.21.15.86:0/2402118194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f851c0cd170 0x7f851c0caa40 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:09.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.815+0000 7f8523600640 1 -- 172.21.15.86:0/2402118194 >> 172.21.15.86:0/2402118194 conn(0x7f851c0b2360 msgr2=0x7f851c142d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:09.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.815+0000 7f8523600640 1 -- 172.21.15.86:0/2402118194 shutdown_connections 2024-04-20T21:20:09.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:09.815+0000 7f8523600640 1 -- 172.21.15.86:0/2402118194 wait complete. 2024-04-20T21:20:10.236 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T21:20:03.369281+0000', 'modified': '2024-04-20T21:20:06.438592+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5691}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5691': {'gid': 5691, 'name': 'a', 'rank': 0, 'incarnation': 20, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/712628984', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6836', 'nonce': 712628984}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 712628984}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5691, 'qdb_cluster': [5691]} 2024-04-20T21:20:10.236 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-20T21:20:10.236 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-20T21:20:10.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.440+0000 7f1f31600640 1 Processor -- start 2024-04-20T21:20:10.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.441+0000 7f1f31600640 1 -- start start 2024-04-20T21:20:10.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.442+0000 7f1f31600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1f2c0cd690 0x7f1f2c0b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:10.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.442+0000 7f1f31600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f2c0b44b0 0x7f1f2c0ce530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:10.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.442+0000 7f1f31600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f2c1632d0 con 0x7f1f2c162ef0 2024-04-20T21:20:10.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.442+0000 7f1f31600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f1f2c06e5e0 con 0x7f1f2c0cd690 2024-04-20T21:20:10.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.442+0000 7f1f31600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1f2c06e8c0 con 0x7f1f2c0b44b0 2024-04-20T21:20:10.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.443+0000 7f1f23e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1f2c0cd690 0x7f1f2c0b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:10.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.443+0000 7f1f23e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1f2c0cd690 0x7f1f2c0b3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.86:45510/0 (socket says 172.21.15.86:45510) 2024-04-20T21:20:10.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.443+0000 7f1f23e00640 1 -- 172.21.15.86:0/2383754338 learned_addr learned my addr 172.21.15.86:0/2383754338 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:10.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.443+0000 7f1f30c00640 1 --2- 172.21.15.86:0/2383754338 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f2c0b44b0 0x7f1f2c0ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:10.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.443+0000 7f1f23e00640 1 -- 172.21.15.86:0/2383754338 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f2c0b44b0 msgr2=0x7f1f2c0ce530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:10.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.443+0000 7f1f23e00640 1 --2- 172.21.15.86:0/2383754338 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f2c0b44b0 0x7f1f2c0ce530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:10.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.443+0000 7f1f23e00640 1 -- 172.21.15.86:0/2383754338 >> v1:172.21.15.144:6789/0 conn(0x7f1f2c162ef0 legacy=0x7f1f2c06dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:10.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.444+0000 7f1f23e00640 1 -- 172.21.15.86:0/2383754338 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f2c0b3770 con 0x7f1f2c0cd690 2024-04-20T21:20:10.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.445+0000 7f1f23e00640 1 --2- 172.21.15.86:0/2383754338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f2c0cd690 0x7f1f2c0b3f70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1f14009870 tx=0x7f1f1402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54d9e86188e86e51 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:10.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.446+0000 7f1f2b400640 1 -- 172.21.15.86:0/2383754338 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f14034070 con 0x7f1f2c0cd690 2024-04-20T21:20:10.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.446+0000 7f1f2b400640 1 -- 172.21.15.86:0/2383754338 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f14030840 con 0x7f1f2c0cd690 2024-04-20T21:20:10.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.446+0000 7f1f2b400640 1 -- 172.21.15.86:0/2383754338 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f14030c10 con 0x7f1f2c0cd690 2024-04-20T21:20:10.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.446+0000 7f1f31600640 1 -- 172.21.15.86:0/2383754338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f2c0cd690 msgr2=0x7f1f2c0b3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:10.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.446+0000 7f1f31600640 1 --2- 172.21.15.86:0/2383754338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f2c0cd690 0x7f1f2c0b3f70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1f14009870 tx=0x7f1f1402b980 comp rx=0 tx=0).stop 2024-04-20T21:20:10.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.446+0000 7f1f31600640 1 -- 172.21.15.86:0/2383754338 shutdown_connections 2024-04-20T21:20:10.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.446+0000 7f1f31600640 1 --2- 172.21.15.86:0/2383754338 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f2c0b44b0 0x7f1f2c0ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:10.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.446+0000 7f1f31600640 1 --2- 172.21.15.86:0/2383754338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f2c0cd690 0x7f1f2c0b3f70 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:10.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.446+0000 7f1f31600640 1 -- 172.21.15.86:0/2383754338 >> 172.21.15.86:0/2383754338 conn(0x7f1f2c06bb60 msgr2=0x7f1f2c06bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:10.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.447+0000 7f1f31600640 1 -- 172.21.15.86:0/2383754338 shutdown_connections 2024-04-20T21:20:10.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.447+0000 7f1f31600640 1 -- 172.21.15.86:0/2383754338 wait complete. 2024-04-20T21:20:10.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.449+0000 7f1f31600640 1 Processor -- start 2024-04-20T21:20:10.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.449+0000 7f1f31600640 1 -- start start 2024-04-20T21:20:10.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.450+0000 7f1f31600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f2c0b44b0 0x7f1f2c159c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:10.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.450+0000 7f1f31600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f2c0cd690 0x7f1f2c0c9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:10.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.450+0000 7f1f31600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f2c162ef0 0x7f1f2c0c9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:10.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.450+0000 7f1f31600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1f2c0b3550 con 0x7f1f2c162ef0 2024-04-20T21:20:10.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.450+0000 7f1f31600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1f2c0b33d0 con 0x7f1f2c0b44b0 2024-04-20T21:20:10.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.450+0000 7f1f31600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1f2c0cea70 con 0x7f1f2c0cd690 2024-04-20T21:20:10.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.450+0000 7f1f30c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f2c162ef0 0x7f1f2c0c9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:10.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.450+0000 7f1f30c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f2c162ef0 0x7f1f2c0c9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45652/0 (socket says 172.21.15.86:45652) 2024-04-20T21:20:10.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.450+0000 7f1f30c00640 1 -- 172.21.15.86:0/2811186840 learned_addr learned my addr 172.21.15.86:0/2811186840 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:10.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.451+0000 7f1f23e00640 1 --2- 172.21.15.86:0/2811186840 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f2c0cd690 0x7f1f2c0c9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:10.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.451+0000 7f1f2be00640 1 --2- 172.21.15.86:0/2811186840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f2c0b44b0 0x7f1f2c159c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:10.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.451+0000 7f1f30c00640 1 -- 172.21.15.86:0/2811186840 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f2c0cd690 msgr2=0x7f1f2c0c9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:10.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.451+0000 7f1f30c00640 1 --2- 172.21.15.86:0/2811186840 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f2c0cd690 0x7f1f2c0c9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:10.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.451+0000 7f1f30c00640 1 -- 172.21.15.86:0/2811186840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f2c0b44b0 msgr2=0x7f1f2c159c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:10.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.451+0000 7f1f30c00640 1 --2- 172.21.15.86:0/2811186840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f2c0b44b0 0x7f1f2c159c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:10.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.451+0000 7f1f30c00640 1 -- 172.21.15.86:0/2811186840 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f2c0c7390 con 0x7f1f2c162ef0 2024-04-20T21:20:10.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.451+0000 7f1f30c00640 1 --2- 172.21.15.86:0/2811186840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f2c162ef0 0x7f1f2c0c9aa0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f1f24009dd0 tx=0x7f1f2400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:10.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.452+0000 7f1f28c00640 1 -- 172.21.15.86:0/2811186840 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f24018020 con 0x7f1f2c162ef0 2024-04-20T21:20:10.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.452+0000 7f1f28c00640 1 -- 172.21.15.86:0/2811186840 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f24014480 con 0x7f1f2c162ef0 2024-04-20T21:20:10.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.452+0000 7f1f28c00640 1 -- 172.21.15.86:0/2811186840 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f24014870 con 0x7f1f2c162ef0 2024-04-20T21:20:10.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.452+0000 7f1f31600640 1 -- 172.21.15.86:0/2811186840 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f2c0c7670 con 0x7f1f2c162ef0 2024-04-20T21:20:10.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.452+0000 7f1f31600640 1 -- 172.21.15.86:0/2811186840 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f2c15a480 con 0x7f1f2c162ef0 2024-04-20T21:20:10.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.454+0000 7f1f28c00640 1 -- 172.21.15.86:0/2811186840 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1f24005d10 con 0x7f1f2c162ef0 2024-04-20T21:20:10.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.455+0000 7f1f28c00640 1 --2- 172.21.15.86:0/2811186840 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1f0806c8f0 0x7f1f0806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:10.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.455+0000 7f1f28c00640 1 -- 172.21.15.86:0/2811186840 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f1f2408c6d0 con 0x7f1f2c162ef0 2024-04-20T21:20:10.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.456+0000 7f1f2be00640 1 --2- 172.21.15.86:0/2811186840 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1f0806c8f0 0x7f1f0806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:10.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.456+0000 7f1f35842740 1 Processor -- start 2024-04-20T21:20:10.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.456+0000 7f1f35842740 1 -- start start 2024-04-20T21:20:10.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.456+0000 7f1f2be00640 1 --2- 172.21.15.86:0/2811186840 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1f0806c8f0 0x7f1f0806edb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c007fd0 tx=0x7f1f1c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:10.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.457+0000 7f1f35842740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecd4c10670 0x55ecd4bc7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:10.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.457+0000 7f1f35842740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecd4bc66f0 0x55ecd4bc81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:10.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.457+0000 7f1f35842740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ecd4bc2b90 0x55ecd4bf2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:10.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.457+0000 7f1f35842740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ecd4b2e9d0 con 0x55ecd4c10670 2024-04-20T21:20:10.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.457+0000 7f1f35842740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ecd4ad5270 con 0x55ecd4bc66f0 2024-04-20T21:20:10.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.457+0000 7f1f35842740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55ecd4b2a990 con 0x55ecd4bc2b90 2024-04-20T21:20:10.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.457+0000 7f1f2be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecd4c10670 0x55ecd4bc7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:10.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.457+0000 7f1f2be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecd4c10670 0x55ecd4bc7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45660/0 (socket says 172.21.15.86:45660) 2024-04-20T21:20:10.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.457+0000 7f1f2be00640 1 -- 172.21.15.86:0/654687286 learned_addr learned my addr 172.21.15.86:0/654687286 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:10.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.458+0000 7f1f23e00640 1 --2- 172.21.15.86:0/654687286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecd4bc66f0 0x55ecd4bc81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:10.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.458+0000 7f1f30c00640 1 --2- 172.21.15.86:0/654687286 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ecd4bc2b90 0x55ecd4bf2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:10.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.458+0000 7f1f2be00640 1 -- 172.21.15.86:0/654687286 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ecd4bc2b90 msgr2=0x55ecd4bf2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:10.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.458+0000 7f1f2be00640 1 --2- 172.21.15.86:0/654687286 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ecd4bc2b90 0x55ecd4bf2980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:10.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.458+0000 7f1f2be00640 1 -- 172.21.15.86:0/654687286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecd4bc66f0 msgr2=0x55ecd4bc81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:10.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.458+0000 7f1f2be00640 1 --2- 172.21.15.86:0/654687286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecd4bc66f0 0x55ecd4bc81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:10.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.458+0000 7f1f2be00640 1 -- 172.21.15.86:0/654687286 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecd48412d0 con 0x55ecd4c10670 2024-04-20T21:20:10.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.458+0000 7f1f23e00640 1 --2- 172.21.15.86:0/654687286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecd4bc66f0 0x55ecd4bc81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:20:10.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.458+0000 7f1f2be00640 1 --2- 172.21.15.86:0/654687286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecd4c10670 0x55ecd4bc7ca0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c0052b0 tx=0x7f1f1c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c08dece2c941b6f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:10.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.458+0000 7f1f1be00640 1 -- 172.21.15.86:0/654687286 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c021450 con 0x55ecd4c10670 2024-04-20T21:20:10.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.458+0000 7f1f1be00640 1 -- 172.21.15.86:0/654687286 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f1c0215f0 con 0x55ecd4c10670 2024-04-20T21:20:10.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.458+0000 7f1f1be00640 1 -- 172.21.15.86:0/654687286 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c0219a0 con 0x55ecd4c10670 2024-04-20T21:20:10.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.459+0000 7f1f35842740 1 -- 172.21.15.86:0/654687286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecd4c10670 msgr2=0x55ecd4bc7ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:10.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.459+0000 7f1f35842740 1 --2- 172.21.15.86:0/654687286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecd4c10670 0x55ecd4bc7ca0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c0052b0 tx=0x7f1f1c011000 comp rx=0 tx=0).stop 2024-04-20T21:20:10.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.459+0000 7f1f35842740 1 -- 172.21.15.86:0/654687286 shutdown_connections 2024-04-20T21:20:10.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.459+0000 7f1f35842740 1 --2- 172.21.15.86:0/654687286 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ecd4bc2b90 0x55ecd4bf2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:10.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.459+0000 7f1f35842740 1 --2- 172.21.15.86:0/654687286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecd4bc66f0 0x55ecd4bc81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:10.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.459+0000 7f1f35842740 1 --2- 172.21.15.86:0/654687286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecd4c10670 0x55ecd4bc7ca0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:10.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.459+0000 7f1f35842740 1 -- 172.21.15.86:0/654687286 >> 172.21.15.86:0/654687286 conn(0x55ecd4bc4e00 msgr2=0x55ecd4c0d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:10.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.459+0000 7f1f35842740 1 -- 172.21.15.86:0/654687286 shutdown_connections 2024-04-20T21:20:10.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.459+0000 7f1f35842740 1 -- 172.21.15.86:0/654687286 wait complete. 2024-04-20T21:20:10.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.460+0000 7f1f35842740 1 -- start start 2024-04-20T21:20:10.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.460+0000 7f1f35842740 1 Processor -- start 2024-04-20T21:20:10.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.461+0000 7f1f35842740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecd4bc66f0 0x55ecd4be9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:10.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.461+0000 7f1f35842740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ecd4c10670 0x55ecd4c13300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:10.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.461+0000 7f1f35842740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecd4bedc80 0x55ecd4bee0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:10.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.461+0000 7f1f35842740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ecd4bc6c00 con 0x55ecd4bedc80 2024-04-20T21:20:10.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.461+0000 7f1f35842740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ecd4ae85e0 con 0x55ecd4bc66f0 2024-04-20T21:20:10.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.461+0000 7f1f35842740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55ecd4bc6ee0 con 0x55ecd4c10670 2024-04-20T21:20:10.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.461+0000 7f1f35842740 10 client.0 _ll_register_callbacks cb 0x55ecd4840ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:20:10.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.462+0000 7f1f30c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecd4bedc80 0x55ecd4bee0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:10.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.462+0000 7f1f30c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecd4bedc80 0x55ecd4bee0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45676/0 (socket says 172.21.15.86:45676) 2024-04-20T21:20:10.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.462+0000 7f1f30c00640 1 -- 172.21.15.86:0/2291911685 learned_addr learned my addr 172.21.15.86:0/2291911685 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:10.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.462+0000 7f1f23e00640 1 --2- 172.21.15.86:0/2291911685 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ecd4c10670 0x55ecd4c13300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:10.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.462+0000 7f1f2be00640 1 --2- 172.21.15.86:0/2291911685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecd4bc66f0 0x55ecd4be9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:10.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.462+0000 7f1f30c00640 1 -- 172.21.15.86:0/2291911685 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ecd4c10670 msgr2=0x55ecd4c13300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:10.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.462+0000 7f1f30c00640 1 --2- 172.21.15.86:0/2291911685 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ecd4c10670 0x55ecd4c13300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:10.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.462+0000 7f1f30c00640 1 -- 172.21.15.86:0/2291911685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecd4bc66f0 msgr2=0x55ecd4be9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:10.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.462+0000 7f1f30c00640 1 --2- 172.21.15.86:0/2291911685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecd4bc66f0 0x55ecd4be9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:10.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.462+0000 7f1f30c00640 1 -- 172.21.15.86:0/2291911685 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecd490b6b0 con 0x55ecd4bedc80 2024-04-20T21:20:10.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.462+0000 7f1f30c00640 1 --2- 172.21.15.86:0/2291911685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecd4bedc80 0x55ecd4bee0e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f1f24056850 tx=0x7f1f2405a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3bb7a6fe3db79ed server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:10.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.462+0000 7f1f18c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:20:10.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.462+0000 7f1f18c00640 1 -- 172.21.15.86:0/2291911685 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f24062fd0 con 0x55ecd4bedc80 2024-04-20T21:20:10.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.462+0000 7f1f18c00640 1 -- 172.21.15.86:0/2291911685 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f1f2401ada0 con 0x55ecd4bedc80 2024-04-20T21:20:10.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.463+0000 7f1f35842740 1 -- 172.21.15.86:0/2291911685 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f1c0215f0 con 0x55ecd4bedc80 2024-04-20T21:20:10.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.463+0000 7f1f18c00640 1 -- 172.21.15.86:0/2291911685 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1f24072bd0 con 0x55ecd4bedc80 2024-04-20T21:20:10.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.463+0000 7f1f35842740 10 client.5814 fetch_fsmap learned FSMap version 24 2024-04-20T21:20:10.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.463+0000 7f1f35842740 1 -- 172.21.15.86:0/2291911685 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x55ecd4c147d0 con 0x55ecd4bedc80 2024-04-20T21:20:10.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.463+0000 7f1f18c00640 1 -- 172.21.15.86:0/2291911685 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f1c0215f0 con 0x55ecd4bedc80 2024-04-20T21:20:10.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.463+0000 7f1f18c00640 1 -- 172.21.15.86:0/2291911685 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 24) ==== 3495+0+0 (secure 0 0 0) 0x7f1f24062b40 con 0x55ecd4bedc80 2024-04-20T21:20:10.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.463+0000 7f1f35842740 10 client.5814 fetch_fsmap finished waiting for FSMap version 24 2024-04-20T21:20:10.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.463+0000 7f1f35842740 10 client.5814 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-20T21:20:10.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.463+0000 7f1f35842740 20 client.5814 populate_metadata read hostname 'smithi086' 2024-04-20T21:20:10.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.463+0000 7f1f35842740 1 --2- 172.21.15.86:0/2291911685 >> [v2:172.21.15.86:6836/712628984,v1:172.21.15.86:6837/712628984] conn(0x55ecd4c1ef60 0x55ecd4c1f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:10.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.463+0000 7f1f35842740 4 client.5814 mds_command: new command op to 5691 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:20:10.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.463+0000 7f1f35842740 1 -- 172.21.15.86:0/2291911685 --> [v2:172.21.15.86:6836/712628984,v1:172.21.15.86:6837/712628984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ecd4b3c3f0 con 0x55ecd4c1ef60 2024-04-20T21:20:10.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.464+0000 7f1f2be00640 1 --2- 172.21.15.86:0/2291911685 >> [v2:172.21.15.86:6836/712628984,v1:172.21.15.86:6837/712628984] conn(0x55ecd4c1ef60 0x55ecd4c1f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:10.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.464+0000 7f1f2be00640 1 --2- 172.21.15.86:0/2291911685 >> [v2:172.21.15.86:6836/712628984,v1:172.21.15.86:6837/712628984] conn(0x55ecd4c1ef60 0x55ecd4c1f340 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e749ad758f15946 server_cookie=bb3aa400a0e2b888 in_seq=0 out_seq=0 2024-04-20T21:20:10.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.464+0000 7f1f18c00640 10 client.5814 ms_handle_connect on v2:172.21.15.86:6836/712628984 2024-04-20T21:20:10.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.465+0000 7f1f18c00640 1 -- 172.21.15.86:0/2291911685 <== mds.0 v2:172.21.15.86:6836/712628984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ecd4b3c3f0 con 0x55ecd4c1ef60 2024-04-20T21:20:10.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.465+0000 7f1f18c00640 10 client.5814 handle_command_reply: tid=0 2024-04-20T21:20:10.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.465+0000 7f1f35842740 1 client.5814 shutdown 2024-04-20T21:20:10.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.465+0000 7f1f35842740 1 -- 172.21.15.86:0/2291911685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecd4bedc80 msgr2=0x55ecd4bee0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:10.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.465+0000 7f1f35842740 1 --2- 172.21.15.86:0/2291911685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecd4bedc80 0x55ecd4bee0e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f1f24056850 tx=0x7f1f2405a3f0 comp rx=0 tx=0).stop 2024-04-20T21:20:10.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.465+0000 7f1f35842740 1 -- 172.21.15.86:0/2291911685 shutdown_connections 2024-04-20T21:20:10.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.465+0000 7f1f35842740 1 --2- 172.21.15.86:0/2291911685 >> [v2:172.21.15.86:6836/712628984,v1:172.21.15.86:6837/712628984] conn(0x55ecd4c1ef60 0x55ecd4c1f340 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:10.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.465+0000 7f1f35842740 1 --2- 172.21.15.86:0/2291911685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecd4bedc80 0x55ecd4bee0e0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:10.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.465+0000 7f1f35842740 1 --2- 172.21.15.86:0/2291911685 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ecd4c10670 0x55ecd4c13300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:10.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.465+0000 7f1f35842740 1 --2- 172.21.15.86:0/2291911685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecd4bc66f0 0x55ecd4be9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:10.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.465+0000 7f1f18c00640 0 client.5814 ms_handle_reset on v2:172.21.15.86:6836/712628984 2024-04-20T21:20:10.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.465+0000 7f1f35842740 1 -- 172.21.15.86:0/2291911685 >> 172.21.15.86:0/2291911685 conn(0x55ecd4bc2b90 msgr2=0x55ecd4c0d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:10.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.466+0000 7f1f35842740 1 -- 172.21.15.86:0/2291911685 shutdown_connections 2024-04-20T21:20:10.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.466+0000 7f1f35842740 1 -- 172.21.15.86:0/2291911685 wait complete. 2024-04-20T21:20:10.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.466+0000 7f1f35842740 20 client.5814 trim_cache size 0 max 16384 2024-04-20T21:20:10.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.485+0000 7f1f35842740 1 Processor -- start 2024-04-20T21:20:10.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.486+0000 7f1f35842740 1 -- start start 2024-04-20T21:20:10.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.486+0000 7f1f35842740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecd4bc66f0 0x55ecd4bc8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:10.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.486+0000 7f1f35842740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecd4bedc80 0x55ecd4c123e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:10.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.486+0000 7f1f35842740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ecd4c10670 0x55ecd4c12920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:10.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.486+0000 7f1f35842740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1f2c2050c0 con 0x55ecd4bedc80 2024-04-20T21:20:10.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.486+0000 7f1f35842740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1f2c2040d0 con 0x55ecd4bc66f0 2024-04-20T21:20:10.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.486+0000 7f1f35842740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1f2c203490 con 0x55ecd4c10670 2024-04-20T21:20:10.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.487+0000 7f1f23e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecd4bedc80 0x55ecd4c123e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:10.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.487+0000 7f1f23e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecd4bedc80 0x55ecd4c123e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45680/0 (socket says 172.21.15.86:45680) 2024-04-20T21:20:10.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.487+0000 7f1f23e00640 1 -- 172.21.15.86:0/1317516385 learned_addr learned my addr 172.21.15.86:0/1317516385 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:10.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.487+0000 7f1f23e00640 1 -- 172.21.15.86:0/1317516385 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ecd4c10670 msgr2=0x55ecd4c12920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:10.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.487+0000 7f1f2be00640 1 --2- 172.21.15.86:0/1317516385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecd4bc66f0 0x55ecd4bc8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:10.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.487+0000 7f1f23e00640 1 --2- 172.21.15.86:0/1317516385 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ecd4c10670 0x55ecd4c12920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:10.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.487+0000 7f1f23e00640 1 -- 172.21.15.86:0/1317516385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecd4bc66f0 msgr2=0x55ecd4bc8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:10.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.487+0000 7f1f23e00640 1 --2- 172.21.15.86:0/1317516385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecd4bc66f0 0x55ecd4bc8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:10.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.487+0000 7f1f23e00640 1 -- 172.21.15.86:0/1317516385 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecd4bc3ac0 con 0x55ecd4bedc80 2024-04-20T21:20:10.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.487+0000 7f1f2be00640 1 --2- 172.21.15.86:0/1317516385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecd4bc66f0 0x55ecd4bc8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:20:10.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.488+0000 7f1f23e00640 1 --2- 172.21.15.86:0/1317516385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecd4bedc80 0x55ecd4c123e0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f1f1402bf90 tx=0x7f1f14030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c10b7eca2f375ae5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:10.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.488+0000 7f1f1b400640 1 -- 172.21.15.86:0/1317516385 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f14048070 con 0x55ecd4bedc80 2024-04-20T21:20:10.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.488+0000 7f1f1b400640 1 -- 172.21.15.86:0/1317516385 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f14030900 con 0x55ecd4bedc80 2024-04-20T21:20:10.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.488+0000 7f1f1b400640 1 -- 172.21.15.86:0/1317516385 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f14034040 con 0x55ecd4bedc80 2024-04-20T21:20:10.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.488+0000 7f1f35842740 1 -- 172.21.15.86:0/1317516385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecd4bedc80 msgr2=0x55ecd4c123e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:10.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.488+0000 7f1f35842740 1 --2- 172.21.15.86:0/1317516385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecd4bedc80 0x55ecd4c123e0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f1f1402bf90 tx=0x7f1f14030000 comp rx=0 tx=0).stop 2024-04-20T21:20:10.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.489+0000 7f1f35842740 1 -- 172.21.15.86:0/1317516385 shutdown_connections 2024-04-20T21:20:10.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.489+0000 7f1f35842740 1 --2- 172.21.15.86:0/1317516385 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ecd4c10670 0x55ecd4c12920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:10.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.489+0000 7f1f35842740 1 --2- 172.21.15.86:0/1317516385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecd4bedc80 0x55ecd4c123e0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:10.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.489+0000 7f1f35842740 1 --2- 172.21.15.86:0/1317516385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecd4bc66f0 0x55ecd4bc8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:10.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.489+0000 7f1f35842740 1 -- 172.21.15.86:0/1317516385 >> 172.21.15.86:0/1317516385 conn(0x55ecd4bc2b90 msgr2=0x55ecd4bc3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:10.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.489+0000 7f1f35842740 1 -- 172.21.15.86:0/1317516385 shutdown_connections 2024-04-20T21:20:10.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.489+0000 7f1f35842740 1 -- 172.21.15.86:0/1317516385 wait complete. 2024-04-20T21:20:10.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.489+0000 7f1f35842740 1 -- start start 2024-04-20T21:20:10.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.490+0000 7f1f35842740 1 Processor -- start 2024-04-20T21:20:10.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.491+0000 7f1f35842740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecd4bc66f0 0x55ecd4c123e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:10.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.491+0000 7f1f35842740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ecd4bedc80 0x55ecd4c12920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:10.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.491+0000 7f1f35842740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecd4c10670 0x55ecd4c31010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:10.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.491+0000 7f1f35842740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ecd4c21a20 con 0x55ecd4bc66f0 2024-04-20T21:20:10.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.491+0000 7f1f35842740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1f2c203ab0 con 0x55ecd4c10670 2024-04-20T21:20:10.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.491+0000 7f1f35842740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55ecd4c14d90 con 0x55ecd4bedc80 2024-04-20T21:20:10.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.491+0000 7f1f35842740 10 client.0 _ll_register_callbacks cb 0x7f1f24056710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:20:10.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.491+0000 7f1f2be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecd4bc66f0 0x55ecd4c123e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:10.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.491+0000 7f1f2be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecd4bc66f0 0x55ecd4c123e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45688/0 (socket says 172.21.15.86:45688) 2024-04-20T21:20:10.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.491+0000 7f1f2be00640 1 -- 172.21.15.86:0/2258140288 learned_addr learned my addr 172.21.15.86:0/2258140288 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:10.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.491+0000 7f1f2be00640 1 -- 172.21.15.86:0/2258140288 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ecd4bedc80 msgr2=0x55ecd4c12920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:10.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.491+0000 7f1f2be00640 1 --2- 172.21.15.86:0/2258140288 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ecd4bedc80 0x55ecd4c12920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:10.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.491+0000 7f1f2be00640 1 -- 172.21.15.86:0/2258140288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecd4c10670 msgr2=0x55ecd4c31010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:10.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.491+0000 7f1f2be00640 1 --2- 172.21.15.86:0/2258140288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecd4c10670 0x55ecd4c31010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:10.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.491+0000 7f1f2be00640 1 -- 172.21.15.86:0/2258140288 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecd4c21040 con 0x55ecd4bc66f0 2024-04-20T21:20:10.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.491+0000 7f1f2be00640 1 --2- 172.21.15.86:0/2258140288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecd4bc66f0 0x55ecd4c123e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c014000 tx=0x7f1f1c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1360e0cf2d0b3e5d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:10.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.491+0000 7f1f18c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:20:10.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.492+0000 7f1f18c00640 1 -- 172.21.15.86:0/2258140288 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c032060 con 0x55ecd4bc66f0 2024-04-20T21:20:10.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.492+0000 7f1f18c00640 1 -- 172.21.15.86:0/2258140288 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f1f1c021bc0 con 0x55ecd4bc66f0 2024-04-20T21:20:10.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.492+0000 7f1f35842740 1 -- 172.21.15.86:0/2258140288 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f14030900 con 0x55ecd4bc66f0 2024-04-20T21:20:10.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.492+0000 7f1f18c00640 1 -- 172.21.15.86:0/2258140288 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1f1c021660 con 0x55ecd4bc66f0 2024-04-20T21:20:10.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.492+0000 7f1f35842740 10 client.5820 fetch_fsmap learned FSMap version 24 2024-04-20T21:20:10.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.492+0000 7f1f35842740 1 -- 172.21.15.86:0/2258140288 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x55ecd4bc7f20 con 0x55ecd4bc66f0 2024-04-20T21:20:10.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.492+0000 7f1f18c00640 1 -- 172.21.15.86:0/2258140288 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f14030900 con 0x55ecd4bc66f0 2024-04-20T21:20:10.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.492+0000 7f1f18c00640 1 -- 172.21.15.86:0/2258140288 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 24) ==== 3495+0+0 (secure 0 0 0) 0x7f1f1c02d420 con 0x55ecd4bc66f0 2024-04-20T21:20:10.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.492+0000 7f1f35842740 10 client.5820 fetch_fsmap finished waiting for FSMap version 24 2024-04-20T21:20:10.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.492+0000 7f1f35842740 10 client.5820 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-20T21:20:10.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.492+0000 7f1f35842740 20 client.5820 populate_metadata read hostname 'smithi086' 2024-04-20T21:20:10.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.492+0000 7f1f35842740 1 --2- 172.21.15.86:0/2258140288 >> [v2:172.21.15.86:6836/712628984,v1:172.21.15.86:6837/712628984] conn(0x55ecd4c31aa0 0x55ecd4c1fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:10.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.492+0000 7f1f35842740 4 client.5820 mds_command: new command op to 5691 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T21:20:10.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.492+0000 7f1f35842740 1 -- 172.21.15.86:0/2258140288 --> [v2:172.21.15.86:6836/712628984,v1:172.21.15.86:6837/712628984] -- command(tid 0: {"prefix": "status"}) -- 0x55ecd4b36260 con 0x55ecd4c31aa0 2024-04-20T21:20:10.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.493+0000 7f1f23e00640 1 --2- 172.21.15.86:0/2258140288 >> [v2:172.21.15.86:6836/712628984,v1:172.21.15.86:6837/712628984] conn(0x55ecd4c31aa0 0x55ecd4c1fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:10.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.493+0000 7f1f23e00640 1 --2- 172.21.15.86:0/2258140288 >> [v2:172.21.15.86:6836/712628984,v1:172.21.15.86:6837/712628984] conn(0x55ecd4c31aa0 0x55ecd4c1fad0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3c833abea52b952 server_cookie=47f409ea2d9eec0b in_seq=0 out_seq=0 2024-04-20T21:20:10.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.493+0000 7f1f18c00640 10 client.5820 ms_handle_connect on v2:172.21.15.86:6836/712628984 2024-04-20T21:20:10.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.493+0000 7f1f18c00640 1 -- 172.21.15.86:0/2258140288 <== mds.0 v2:172.21.15.86:6836/712628984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55ecd4b36260 con 0x55ecd4c31aa0 2024-04-20T21:20:10.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.493+0000 7f1f18c00640 10 client.5820 handle_command_reply: tid=0 2024-04-20T21:20:10.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.493+0000 7f1f35842740 1 client.5820 shutdown 2024-04-20T21:20:10.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.494+0000 7f1f35842740 1 -- 172.21.15.86:0/2258140288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecd4bc66f0 msgr2=0x55ecd4c123e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:10.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.494+0000 7f1f35842740 1 --2- 172.21.15.86:0/2258140288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecd4bc66f0 0x55ecd4c123e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c014000 tx=0x7f1f1c013000 comp rx=0 tx=0).stop 2024-04-20T21:20:10.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.494+0000 7f1f35842740 1 -- 172.21.15.86:0/2258140288 shutdown_connections 2024-04-20T21:20:10.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.494+0000 7f1f35842740 1 --2- 172.21.15.86:0/2258140288 >> [v2:172.21.15.86:6836/712628984,v1:172.21.15.86:6837/712628984] conn(0x55ecd4c31aa0 0x55ecd4c1fad0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:10.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.494+0000 7f1f35842740 1 --2- 172.21.15.86:0/2258140288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecd4c10670 0x55ecd4c31010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:10.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.494+0000 7f1f35842740 1 --2- 172.21.15.86:0/2258140288 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ecd4bedc80 0x55ecd4c12920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:10.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.494+0000 7f1f35842740 1 --2- 172.21.15.86:0/2258140288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecd4bc66f0 0x55ecd4c123e0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:10.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.494+0000 7f1f18c00640 0 client.5820 ms_handle_reset on v2:172.21.15.86:6836/712628984 2024-04-20T21:20:10.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.494+0000 7f1f35842740 1 -- 172.21.15.86:0/2258140288 >> 172.21.15.86:0/2258140288 conn(0x55ecd4bc2b90 msgr2=0x55ecd4bc8d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:10.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.494+0000 7f1f35842740 1 -- 172.21.15.86:0/2258140288 shutdown_connections 2024-04-20T21:20:10.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.494+0000 7f1f35842740 1 -- 172.21.15.86:0/2258140288 wait complete. 2024-04-20T21:20:10.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.494+0000 7f1f35842740 20 client.5820 trim_cache size 0 max 16384 2024-04-20T21:20:10.497 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:20:10.497 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "9c2dec1d-d7ae-4a83-a812-79922808e278", 2024-04-20T21:20:10.498 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-04-20T21:20:10.498 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 5691, 2024-04-20T21:20:10.498 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-20T21:20:10.498 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-20T21:20:10.498 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-20T21:20:10.498 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 6.5586210100000004, 2024-04-20T21:20:10.498 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 24, 2024-04-20T21:20:10.498 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 45, 2024-04-20T21:20:10.498 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 45, 2024-04-20T21:20:10.498 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 13.077500086000001 2024-04-20T21:20:10.498 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:20:10.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.495+0000 7f1f31600640 1 -- 172.21.15.86:0/2811186840 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1f0806c8f0 msgr2=0x7f1f0806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:10.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.495+0000 7f1f31600640 1 --2- 172.21.15.86:0/2811186840 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1f0806c8f0 0x7f1f0806edb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c007fd0 tx=0x7f1f1c009040 comp rx=0 tx=0).stop 2024-04-20T21:20:10.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.495+0000 7f1f31600640 1 -- 172.21.15.86:0/2811186840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f2c162ef0 msgr2=0x7f1f2c0c9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:10.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.495+0000 7f1f31600640 1 --2- 172.21.15.86:0/2811186840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f2c162ef0 0x7f1f2c0c9aa0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f1f24009dd0 tx=0x7f1f2400c490 comp rx=0 tx=0).stop 2024-04-20T21:20:10.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.495+0000 7f1f31600640 1 -- 172.21.15.86:0/2811186840 shutdown_connections 2024-04-20T21:20:10.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.495+0000 7f1f31600640 1 --2- 172.21.15.86:0/2811186840 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1f0806c8f0 0x7f1f0806edb0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:10.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.495+0000 7f1f31600640 1 --2- 172.21.15.86:0/2811186840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f2c162ef0 0x7f1f2c0c9aa0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:10.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.495+0000 7f1f31600640 1 --2- 172.21.15.86:0/2811186840 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f2c0cd690 0x7f1f2c0c9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:10.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.495+0000 7f1f31600640 1 --2- 172.21.15.86:0/2811186840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f2c0b44b0 0x7f1f2c159c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:10.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.495+0000 7f1f31600640 1 -- 172.21.15.86:0/2811186840 >> 172.21.15.86:0/2811186840 conn(0x7f1f2c06bb60 msgr2=0x7f1f2c06bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:10.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.495+0000 7f1f31600640 1 -- 172.21.15.86:0/2811186840 shutdown_connections 2024-04-20T21:20:10.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.495+0000 7f1f31600640 1 -- 172.21.15.86:0/2811186840 wait complete. 2024-04-20T21:20:10.512 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T21:20:10.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.698+0000 7f820a400640 1 Processor -- start 2024-04-20T21:20:10.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.698+0000 7f820a400640 1 -- start start 2024-04-20T21:20:10.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.700+0000 7f820a400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8204146130 0x7f820406e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:10.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.700+0000 7f820a400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82040ccdd0 0x7f82040cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:10.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.700+0000 7f820a400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8204071d00 0x7f8204072190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:10.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.700+0000 7f820a400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f820406dc30 con 0x7f8204146130 2024-04-20T21:20:10.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.700+0000 7f820a400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f82040cd8e0 con 0x7f8204071d00 2024-04-20T21:20:10.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.700+0000 7f820a400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f82040cdbc0 con 0x7f82040ccdd0 2024-04-20T21:20:10.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.700+0000 7f8203400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8204146130 0x7f820406e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:10.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.700+0000 7f8203400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8204146130 0x7f820406e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45702/0 (socket says 172.21.15.86:45702) 2024-04-20T21:20:10.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.700+0000 7f8203400640 1 -- 172.21.15.86:0/2222318449 learned_addr learned my addr 172.21.15.86:0/2222318449 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:10.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.701+0000 7f8202a00640 1 --2- 172.21.15.86:0/2222318449 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82040ccdd0 0x7f82040cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:10.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.701+0000 7f8203400640 1 -- 172.21.15.86:0/2222318449 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82040ccdd0 msgr2=0x7f82040cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:10.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.701+0000 7f8203e00640 1 --2- 172.21.15.86:0/2222318449 >> v2:172.21.15.144:3300/0 conn(0x7f8204071d00 0x7f8204072190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:10.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.701+0000 7f8203400640 1 --2- 172.21.15.86:0/2222318449 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82040ccdd0 0x7f82040cd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:10.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.701+0000 7f8203400640 1 -- 172.21.15.86:0/2222318449 >> v2:172.21.15.144:3300/0 conn(0x7f8204071d00 msgr2=0x7f8204072190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:10.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.701+0000 7f8203400640 1 --2- 172.21.15.86:0/2222318449 >> v2:172.21.15.144:3300/0 conn(0x7f8204071d00 0x7f8204072190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:10.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.701+0000 7f8203400640 1 -- 172.21.15.86:0/2222318449 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f820406ded0 con 0x7f8204146130 2024-04-20T21:20:10.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.702+0000 7f8203400640 1 --2- 172.21.15.86:0/2222318449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8204146130 0x7f820406e6d0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f8204070070 tx=0x7f81f402b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c36bf760bccf3c56 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:10.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.703+0000 7f8202000640 1 -- 172.21.15.86:0/2222318449 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f4034070 con 0x7f8204146130 2024-04-20T21:20:10.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.703+0000 7f8202000640 1 -- 172.21.15.86:0/2222318449 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81f4030850 con 0x7f8204146130 2024-04-20T21:20:10.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.703+0000 7f8202000640 1 -- 172.21.15.86:0/2222318449 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f4030c20 con 0x7f8204146130 2024-04-20T21:20:10.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.703+0000 7f820a400640 1 -- 172.21.15.86:0/2222318449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8204146130 msgr2=0x7f820406e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:10.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.703+0000 7f820a400640 1 --2- 172.21.15.86:0/2222318449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8204146130 0x7f820406e6d0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f8204070070 tx=0x7f81f402b990 comp rx=0 tx=0).stop 2024-04-20T21:20:10.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.703+0000 7f820a400640 1 -- 172.21.15.86:0/2222318449 shutdown_connections 2024-04-20T21:20:10.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.703+0000 7f820a400640 1 --2- 172.21.15.86:0/2222318449 >> v2:172.21.15.144:3300/0 conn(0x7f8204071d00 0x7f8204072190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:10.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.703+0000 7f820a400640 1 --2- 172.21.15.86:0/2222318449 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82040ccdd0 0x7f82040cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:10.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.703+0000 7f820a400640 1 --2- 172.21.15.86:0/2222318449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8204146130 0x7f820406e6d0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:10.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.703+0000 7f820a400640 1 -- 172.21.15.86:0/2222318449 >> 172.21.15.86:0/2222318449 conn(0x7f82040b2360 msgr2=0x7f82040b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:10.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.704+0000 7f820a400640 1 -- 172.21.15.86:0/2222318449 shutdown_connections 2024-04-20T21:20:10.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.704+0000 7f820a400640 1 -- 172.21.15.86:0/2222318449 wait complete. 2024-04-20T21:20:10.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.706+0000 7f820a400640 1 Processor -- start 2024-04-20T21:20:10.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.706+0000 7f820a400640 1 -- start start 2024-04-20T21:20:10.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.707+0000 7f820a400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8204071d00 0x7f8204133af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:10.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.707+0000 7f820a400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82040ccdd0 0x7f82041e4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:10.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.707+0000 7f820a400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8204146130 0x7f82040cc200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:10.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.707+0000 7f820a400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f820415cc20 con 0x7f8204071d00 2024-04-20T21:20:10.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.707+0000 7f820a400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f820406f150 con 0x7f82040ccdd0 2024-04-20T21:20:10.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.707+0000 7f820a400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f820415cda0 con 0x7f8204146130 2024-04-20T21:20:10.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.707+0000 7f8203400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8204071d00 0x7f8204133af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:10.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.707+0000 7f8203400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8204071d00 0x7f8204133af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45718/0 (socket says 172.21.15.86:45718) 2024-04-20T21:20:10.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.707+0000 7f8203400640 1 -- 172.21.15.86:0/2406380625 learned_addr learned my addr 172.21.15.86:0/2406380625 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:10.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.707+0000 7f8202a00640 1 --2- 172.21.15.86:0/2406380625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82040ccdd0 0x7f82041e4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:10.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.707+0000 7f8203e00640 1 --2- 172.21.15.86:0/2406380625 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8204146130 0x7f82040cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:10.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.707+0000 7f8203400640 1 -- 172.21.15.86:0/2406380625 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8204146130 msgr2=0x7f82040cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:10.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.707+0000 7f8203400640 1 --2- 172.21.15.86:0/2406380625 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8204146130 0x7f82040cc200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:10.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.707+0000 7f8203400640 1 -- 172.21.15.86:0/2406380625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82040ccdd0 msgr2=0x7f82041e4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:10.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.707+0000 7f8203400640 1 --2- 172.21.15.86:0/2406380625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82040ccdd0 0x7f82041e4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:10.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.708+0000 7f8203400640 1 -- 172.21.15.86:0/2406380625 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8204134030 con 0x7f8204071d00 2024-04-20T21:20:10.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.708+0000 7f8202a00640 1 --2- 172.21.15.86:0/2406380625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82040ccdd0 0x7f82041e4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:20:10.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.708+0000 7f8203400640 1 --2- 172.21.15.86:0/2406380625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8204071d00 0x7f8204133af0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f81f4000c00 tx=0x7f81f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:10.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.708+0000 7f81fb400640 1 -- 172.21.15.86:0/2406380625 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f4034040 con 0x7f8204071d00 2024-04-20T21:20:10.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.708+0000 7f81fb400640 1 -- 172.21.15.86:0/2406380625 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81f40433f0 con 0x7f8204071d00 2024-04-20T21:20:10.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.708+0000 7f81fb400640 1 -- 172.21.15.86:0/2406380625 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f40437e0 con 0x7f8204071d00 2024-04-20T21:20:10.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.708+0000 7f820a400640 1 -- 172.21.15.86:0/2406380625 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82041e53c0 con 0x7f8204071d00 2024-04-20T21:20:10.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.708+0000 7f820a400640 1 -- 172.21.15.86:0/2406380625 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82040cb440 con 0x7f8204071d00 2024-04-20T21:20:10.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.710+0000 7f820a400640 1 -- 172.21.15.86:0/2406380625 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82040689e0 con 0x7f8204071d00 2024-04-20T21:20:10.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.711+0000 7f81fb400640 1 -- 172.21.15.86:0/2406380625 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f81f404a060 con 0x7f8204071d00 2024-04-20T21:20:10.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.712+0000 7f81fb400640 1 --2- 172.21.15.86:0/2406380625 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f81e406c8f0 0x7f81e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:10.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.712+0000 7f81fb400640 1 -- 172.21.15.86:0/2406380625 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f81f40ade20 con 0x7f8204071d00 2024-04-20T21:20:10.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.712+0000 7f8202a00640 1 --2- 172.21.15.86:0/2406380625 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f81e406c8f0 0x7f81e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:10.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.713+0000 7f8202a00640 1 --2- 172.21.15.86:0/2406380625 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f81e406c8f0 0x7f81e406edb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f820415d560 tx=0x7f81f0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:10.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.716+0000 7f81fb400640 1 -- 172.21.15.86:0/2406380625 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f81f403e070 con 0x7f8204071d00 2024-04-20T21:20:10.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.967+0000 7f820a400640 1 -- 172.21.15.86:0/2406380625 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f820415ff50 con 0x7f8204071d00 2024-04-20T21:20:10.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.968+0000 7f81fb400640 1 -- 172.21.15.86:0/2406380625 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+73 (secure 0 0 0) 0x7f81f404be90 con 0x7f8204071d00 2024-04-20T21:20:10.968 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T21:20:10.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.970+0000 7f820a400640 1 -- 172.21.15.86:0/2406380625 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f81e406c8f0 msgr2=0x7f81e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:10.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.970+0000 7f820a400640 1 --2- 172.21.15.86:0/2406380625 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f81e406c8f0 0x7f81e406edb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f820415d560 tx=0x7f81f0008040 comp rx=0 tx=0).stop 2024-04-20T21:20:10.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.971+0000 7f820a400640 1 -- 172.21.15.86:0/2406380625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8204071d00 msgr2=0x7f8204133af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:10.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.971+0000 7f820a400640 1 --2- 172.21.15.86:0/2406380625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8204071d00 0x7f8204133af0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f81f4000c00 tx=0x7f81f40046c0 comp rx=0 tx=0).stop 2024-04-20T21:20:10.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.971+0000 7f820a400640 1 -- 172.21.15.86:0/2406380625 shutdown_connections 2024-04-20T21:20:10.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.971+0000 7f820a400640 1 --2- 172.21.15.86:0/2406380625 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f81e406c8f0 0x7f81e406edb0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:10.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.971+0000 7f820a400640 1 --2- 172.21.15.86:0/2406380625 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8204146130 0x7f82040cc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:10.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.971+0000 7f820a400640 1 --2- 172.21.15.86:0/2406380625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82040ccdd0 0x7f82041e4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:10.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.971+0000 7f820a400640 1 --2- 172.21.15.86:0/2406380625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8204071d00 0x7f8204133af0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:10.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.971+0000 7f820a400640 1 -- 172.21.15.86:0/2406380625 >> 172.21.15.86:0/2406380625 conn(0x7f82040b2360 msgr2=0x7f82040b4b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:10.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.971+0000 7f820a400640 1 -- 172.21.15.86:0/2406380625 shutdown_connections 2024-04-20T21:20:10.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:10.971+0000 7f820a400640 1 -- 172.21.15.86:0/2406380625 wait complete. 2024-04-20T21:20:10.990 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T21:20:10.990 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T21:20:10.990 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-04-20T21:20:10.990 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-20T21:20:10.990 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T21:20:10.990 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T21:20:10.991 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T21:20:10.991 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T21:20:10.991 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:20:10.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:20:10.991 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-04-20T21:20:11.043 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T21:20:11.043 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T21:20:11.043 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T21:20:11.044 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:20:11.044 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-20T21:20:11.044 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:20:11.044 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:20:11.044 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:20:11.044 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-04-20T21:20:11.044 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:20:11.044 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:20:11.044 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-04-20T21:20:11.044 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:20:11.044 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:20:11.044 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T21:20:11.044 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:20:11.044 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T21:20:11.045 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42545sec preferred_lft 42545sec 2024-04-20T21:20:11.045 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-04-20T21:20:11.045 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:20:11.045 INFO:teuthology.orchestra.run.smithi086.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T21:20:11.045 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 86:25:97:2c:9c:bb brd ff:ff:ff:ff:ff:ff 2024-04-20T21:20:11.045 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-20T21:20:11.045 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:20:11.045 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::a4fb:30ff:fe90:2d6a/64 scope link 2024-04-20T21:20:11.045 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:20:11.045 INFO:teuthology.orchestra.run.smithi086.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T21:20:11.045 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 86:25:97:2c:9c:bb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:20:11.045 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::8425:97ff:fe2c:9cbb/64 scope link 2024-04-20T21:20:11.045 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:20:11.045 INFO:teuthology.orchestra.run.smithi086.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T21:20:11.046 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c2:9c:8b:55:ca:84 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:20:11.046 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c09c:8bff:fe55:ca84/64 scope link 2024-04-20T21:20:11.046 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:20:11.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:20:11.046 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T21:20:11.125 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-20T21:20:11.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:20:11.128 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-04-20T21:20:11.175 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-04-20T21:20:11.175 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T21:20:11.175 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T21:20:11.175 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T21:20:11.175 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T21:20:11.176 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:20:11.176 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-20T21:20:11.176 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T21:20:11.176 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T21:20:11.176 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T21:20:11.176 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-20T21:20:11.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:20:11.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:20:11.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:20:11.383 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-20T21:20:11.432 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-20T21:20:11.433 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-20T21:20:11.433 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:20:11.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:20:11.434 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-04-20T21:20:11.515 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-20T21:20:11.516 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T21:20:11.516 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-20T21:20:11.569 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:20:11.570 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T21:20:11.570 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-20T21:20:11.616 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-04-20T21:20:11.689 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T21:20:11.741 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-04-20T21:20:11.742 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-04-20T21:20:11.742 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-04-20T21:20:11.742 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-04-20T21:20:11.742 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-04-20T21:20:11.742 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-04-20T21:20:11.742 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-04-20T21:20:11.742 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-04-20T21:20:11.742 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-04-20T21:20:11.742 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-04-20T21:20:11.742 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-04-20T21:20:11.742 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/selinux 2024-04-20T21:20:11.742 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T21:20:11.742 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-04-20T21:20:11.742 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-04-20T21:20:11.743 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-04-20T21:20:11.743 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-04-20T21:20:11.743 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-04-20T21:20:11.743 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-04-20T21:20:11.743 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T21:20:11.743 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T21:20:11.743 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T21:20:11.743 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T21:20:11.743 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-04-20T21:20:11.743 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-04-20T21:20:11.743 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T21:20:11.743 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T21:20:11.743 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T21:20:11.743 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T21:20:11.744 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-04-20T21:20:11.744 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:20:11.744 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:20:11.744 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:20:11.744 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:20:11.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:20:11.745 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-04-20T21:20:11.797 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T21:20:11.797 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-20T21:20:11.802 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-04-20T21:20:11.871 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T21:20:11.911 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-04-20T21:20:11.911 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-04-20T21:20:11.912 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-04-20T21:20:11.912 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-04-20T21:20:11.912 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-04-20T21:20:11.912 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-04-20T21:20:11.912 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-04-20T21:20:11.912 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-04-20T21:20:11.912 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-04-20T21:20:11.912 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-04-20T21:20:11.912 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-04-20T21:20:11.912 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/selinux 2024-04-20T21:20:11.912 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T21:20:11.912 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-04-20T21:20:11.912 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-04-20T21:20:11.912 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-04-20T21:20:11.913 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-04-20T21:20:11.913 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-04-20T21:20:11.913 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-04-20T21:20:11.913 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T21:20:11.913 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T21:20:11.913 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T21:20:11.913 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T21:20:11.913 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-04-20T21:20:11.913 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-04-20T21:20:11.913 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T21:20:11.913 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T21:20:11.913 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T21:20:11.913 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T21:20:11.913 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-04-20T21:20:11.914 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:20:11.914 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:20:11.914 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:20:11.914 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:20:11.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:20:11.915 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-04-20T21:20:11.970 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:2024-04-20T21:20:11.970+0000 7efc9776e480 -1 init, newargv = 0x561bd89663e0 newargc=15 2024-04-20T21:20:11.970 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[40289]: starting ceph client 2024-04-20T21:20:11.979 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[40289]: starting fuse 2024-04-20T21:20:11.999 INFO:teuthology.orchestra.run.smithi086.stdout:41 2024-04-20T21:20:12.000 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-04-20T21:20:12.000 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer -- python3 -c ' 2024-04-20T21:20:12.000 DEBUG:teuthology.orchestra.run.smithi086:> import glob 2024-04-20T21:20:12.000 DEBUG:teuthology.orchestra.run.smithi086:> import re 2024-04-20T21:20:12.000 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:20:12.000 DEBUG:teuthology.orchestra.run.smithi086:> import subprocess 2024-04-20T21:20:12.000 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:20:12.000 DEBUG:teuthology.orchestra.run.smithi086:> def _find_admin_socket(client_name): 2024-04-20T21:20:12.001 DEBUG:teuthology.orchestra.run.smithi086:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-20T21:20:12.001 DEBUG:teuthology.orchestra.run.smithi086:> files = glob.glob(asok_path) 2024-04-20T21:20:12.001 DEBUG:teuthology.orchestra.run.smithi086:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-20T21:20:12.001 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:20:12.001 DEBUG:teuthology.orchestra.run.smithi086:> # Given a non-glob path, it better be there 2024-04-20T21:20:12.001 DEBUG:teuthology.orchestra.run.smithi086:> if "*" not in asok_path: 2024-04-20T21:20:12.001 DEBUG:teuthology.orchestra.run.smithi086:> assert(len(files) == 1) 2024-04-20T21:20:12.001 DEBUG:teuthology.orchestra.run.smithi086:> return files[0] 2024-04-20T21:20:12.001 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:20:12.001 DEBUG:teuthology.orchestra.run.smithi086:> for f in files: 2024-04-20T21:20:12.001 DEBUG:teuthology.orchestra.run.smithi086:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T21:20:12.001 DEBUG:teuthology.orchestra.run.smithi086:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T21:20:12.001 DEBUG:teuthology.orchestra.run.smithi086:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T21:20:12.002 DEBUG:teuthology.orchestra.run.smithi086:> contents = proc_f.read() 2024-04-20T21:20:12.002 DEBUG:teuthology.orchestra.run.smithi086:> if mountpoint in contents: 2024-04-20T21:20:12.002 DEBUG:teuthology.orchestra.run.smithi086:> return f 2024-04-20T21:20:12.002 DEBUG:teuthology.orchestra.run.smithi086:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T21:20:12.002 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:20:12.002 DEBUG:teuthology.orchestra.run.smithi086:> print(_find_admin_socket("client.0")) 2024-04-20T21:20:12.002 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:20:12.185 INFO:teuthology.orchestra.run.smithi086.stdout:/var/run/ceph/ceph-client.0.40289.asok 2024-04-20T21:20:12.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:20:12.202 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.40289.asok 2024-04-20T21:20:12.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:20:12.202 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.40289.asok status 2024-04-20T21:20:12.424 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:20:12.425 INFO:teuthology.orchestra.run.smithi086.stdout: "metadata": { 2024-04-20T21:20:12.425 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T21:20:12.425 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T21:20:12.425 INFO:teuthology.orchestra.run.smithi086.stdout: "entity_id": "0", 2024-04-20T21:20:12.425 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": "smithi086", 2024-04-20T21:20:12.425 INFO:teuthology.orchestra.run.smithi086.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-20T21:20:12.425 INFO:teuthology.orchestra.run.smithi086.stdout: "pid": "40289", 2024-04-20T21:20:12.425 INFO:teuthology.orchestra.run.smithi086.stdout: "root": "/" 2024-04-20T21:20:12.425 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:20:12.425 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_count": 0, 2024-04-20T21:20:12.425 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_pinned_count": 0, 2024-04-20T21:20:12.425 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 5829, 2024-04-20T21:20:12.425 INFO:teuthology.orchestra.run.smithi086.stdout: "inst": { 2024-04-20T21:20:12.425 INFO:teuthology.orchestra.run.smithi086.stdout: "name": { 2024-04-20T21:20:12.426 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "client", 2024-04-20T21:20:12.426 INFO:teuthology.orchestra.run.smithi086.stdout: "num": 5829 2024-04-20T21:20:12.426 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:20:12.426 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2024-04-20T21:20:12.426 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-04-20T21:20:12.426 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2024-04-20T21:20:12.426 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 1182196237 2024-04-20T21:20:12.426 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:20:12.426 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:20:12.426 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2024-04-20T21:20:12.426 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-04-20T21:20:12.426 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2024-04-20T21:20:12.427 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 1182196237 2024-04-20T21:20:12.427 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:20:12.427 INFO:teuthology.orchestra.run.smithi086.stdout: "inst_str": "client.5829 192.168.0.1:0/1182196237", 2024-04-20T21:20:12.427 INFO:teuthology.orchestra.run.smithi086.stdout: "addr_str": "192.168.0.1:0/1182196237", 2024-04-20T21:20:12.427 INFO:teuthology.orchestra.run.smithi086.stdout: "inode_count": 1, 2024-04-20T21:20:12.427 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_epoch": 24, 2024-04-20T21:20:12.427 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch": 45, 2024-04-20T21:20:12.427 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch_barrier": 0, 2024-04-20T21:20:12.427 INFO:teuthology.orchestra.run.smithi086.stdout: "blocklisted": false, 2024-04-20T21:20:12.427 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs" 2024-04-20T21:20:12.427 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:20:12.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:20:12.439 DEBUG:teuthology.orchestra.run.smithi086:> stat --file-system '--printf=%T 2024-04-20T21:20:12.439 DEBUG:teuthology.orchestra.run.smithi086:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-20T21:20:12.495 INFO:teuthology.orchestra.run.smithi086.stdout:fuseblk 2024-04-20T21:20:12.495 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-20T21:20:12.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:20:12.495 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-20T21:20:12.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:20:12.585 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent_a) 2024-04-20T21:20:12.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:20:12.647 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parent_a/alpha) 2024-04-20T21:20:12.702 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-04-20T21:20:12.702 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:20:12.702 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-04-20T21:20:12.703 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:20:12.703 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/parent_a/alpha").st_ino) 2024-04-20T21:20:12.703 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:20:12.840 INFO:teuthology.orchestra.run.smithi086.stdout:1099511627777 2024-04-20T21:20:12.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:20:12.853 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:20:13.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.104+0000 7f28cb000640 1 Processor -- start 2024-04-20T21:20:13.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.104+0000 7f28cb000640 1 -- start start 2024-04-20T21:20:13.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.106+0000 7f28cb000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f28c4146130 0x7f28c406e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:13.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.106+0000 7f28cb000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f28c4072410 0x7f28c40cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:13.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.106+0000 7f28cb000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28c406dc30 con 0x7f28c40ccdd0 2024-04-20T21:20:13.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.106+0000 7f28cb000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f28c40cd8e0 con 0x7f28c4146130 2024-04-20T21:20:13.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.106+0000 7f28cb000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f28c40cdbc0 con 0x7f28c4072410 2024-04-20T21:20:13.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.106+0000 7f28c3e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f28c4146130 0x7f28c406e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:13.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.106+0000 7f28c3400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f28c40ccdd0 0x7f28c4071d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39676/0 (socket says 172.21.15.86:39676) 2024-04-20T21:20:13.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.106+0000 7f28c3400640 1 -- 172.21.15.86:0/3682846657 learned_addr learned my addr 172.21.15.86:0/3682846657 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:20:13.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.107+0000 7f28c9000640 1 --2- 172.21.15.86:0/3682846657 >> v2:172.21.15.144:3300/0 conn(0x7f28c4072410 0x7f28c40cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:13.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.107+0000 7f28c2a00640 1 -- 172.21.15.86:0/3682846657 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3443817625 0 0) 0x7f28c406dc30 con 0x7f28c40ccdd0 2024-04-20T21:20:13.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.107+0000 7f28c2a00640 1 -- 172.21.15.86:0/3682846657 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f28a8003520 con 0x7f28c40ccdd0 2024-04-20T21:20:13.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.107+0000 7f28c3e00640 1 -- 172.21.15.86:0/3682846657 >> v2:172.21.15.144:3300/0 conn(0x7f28c4072410 msgr2=0x7f28c40cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:13.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.107+0000 7f28c3e00640 1 --2- 172.21.15.86:0/3682846657 >> v2:172.21.15.144:3300/0 conn(0x7f28c4072410 0x7f28c40cd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:13.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.107+0000 7f28c3e00640 1 -- 172.21.15.86:0/3682846657 >> v1:172.21.15.86:6789/0 conn(0x7f28c40ccdd0 legacy=0x7f28c4071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:13.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.107+0000 7f28c3e00640 1 -- 172.21.15.86:0/3682846657 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28c406ded0 con 0x7f28c4146130 2024-04-20T21:20:13.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.108+0000 7f28c3e00640 1 --2- 172.21.15.86:0/3682846657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28c4146130 0x7f28c406e6d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f28c4071c90 tx=0x7f28b402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf39b65ecf0425cd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:13.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.109+0000 7f28c2a00640 1 -- 172.21.15.86:0/3682846657 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b4039070 con 0x7f28c4146130 2024-04-20T21:20:13.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.109+0000 7f28c2a00640 1 -- 172.21.15.86:0/3682846657 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28b402cd00 con 0x7f28c4146130 2024-04-20T21:20:13.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.109+0000 7f28c2a00640 1 -- 172.21.15.86:0/3682846657 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b4034640 con 0x7f28c4146130 2024-04-20T21:20:13.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.109+0000 7f28cb000640 1 -- 172.21.15.86:0/3682846657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28c4146130 msgr2=0x7f28c406e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:13.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.109+0000 7f28cb000640 1 --2- 172.21.15.86:0/3682846657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28c4146130 0x7f28c406e6d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f28c4071c90 tx=0x7f28b402bf50 comp rx=0 tx=0).stop 2024-04-20T21:20:13.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.109+0000 7f28cb000640 1 -- 172.21.15.86:0/3682846657 shutdown_connections 2024-04-20T21:20:13.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.109+0000 7f28cb000640 1 --2- 172.21.15.86:0/3682846657 >> v2:172.21.15.144:3300/0 conn(0x7f28c4072410 0x7f28c40cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:13.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.109+0000 7f28cb000640 1 --2- 172.21.15.86:0/3682846657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28c4146130 0x7f28c406e6d0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:13.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.109+0000 7f28cb000640 1 -- 172.21.15.86:0/3682846657 >> 172.21.15.86:0/3682846657 conn(0x7f28c40b2360 msgr2=0x7f28c40b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:13.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.109+0000 7f28cb000640 1 -- 172.21.15.86:0/3682846657 shutdown_connections 2024-04-20T21:20:13.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.109+0000 7f28cb000640 1 -- 172.21.15.86:0/3682846657 wait complete. 2024-04-20T21:20:13.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.111+0000 7f28cb000640 1 Processor -- start 2024-04-20T21:20:13.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.111+0000 7f28cb000640 1 -- start start 2024-04-20T21:20:13.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.112+0000 7f28cb000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28c4072410 0x7f28c41f8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:13.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.112+0000 7f28cb000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28c40ccdd0 0x7f28c4142480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:13.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.112+0000 7f28cb000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28c4146130 0x7f28c415bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:13.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.112+0000 7f28cb000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f28c415cc20 con 0x7f28c4146130 2024-04-20T21:20:13.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.112+0000 7f28cb000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f28c406f150 con 0x7f28c4072410 2024-04-20T21:20:13.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.112+0000 7f28cb000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f28c415cda0 con 0x7f28c40ccdd0 2024-04-20T21:20:13.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.113+0000 7f28c9000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28c4146130 0x7f28c415bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:13.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.113+0000 7f28c9000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28c4146130 0x7f28c415bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45732/0 (socket says 172.21.15.86:45732) 2024-04-20T21:20:13.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.113+0000 7f28c9000640 1 -- 172.21.15.86:0/3010972078 learned_addr learned my addr 172.21.15.86:0/3010972078 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:13.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.113+0000 7f28c9000640 1 -- 172.21.15.86:0/3010972078 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28c40ccdd0 msgr2=0x7f28c4142480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:13.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.113+0000 7f28c9000640 1 --2- 172.21.15.86:0/3010972078 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28c40ccdd0 0x7f28c4142480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:13.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.113+0000 7f28c9000640 1 -- 172.21.15.86:0/3010972078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28c4072410 msgr2=0x7f28c41f8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:13.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.113+0000 7f28c9000640 1 --2- 172.21.15.86:0/3010972078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28c4072410 0x7f28c41f8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:13.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.113+0000 7f28c9000640 1 -- 172.21.15.86:0/3010972078 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28c41429c0 con 0x7f28c4146130 2024-04-20T21:20:13.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.113+0000 7f28c9000640 1 --2- 172.21.15.86:0/3010972078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28c4146130 0x7f28c415bfa0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f28bc00d5d0 tx=0x7f28bc00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:13.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.113+0000 7f28bbe00640 1 -- 172.21.15.86:0/3010972078 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28bc0125f0 con 0x7f28c4146130 2024-04-20T21:20:13.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.114+0000 7f28bbe00640 1 -- 172.21.15.86:0/3010972078 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28bc019070 con 0x7f28c4146130 2024-04-20T21:20:13.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.114+0000 7f28bbe00640 1 -- 172.21.15.86:0/3010972078 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28bc0129d0 con 0x7f28c4146130 2024-04-20T21:20:13.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.114+0000 7f28cb000640 1 -- 172.21.15.86:0/3010972078 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28c4142c90 con 0x7f28c4146130 2024-04-20T21:20:13.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.114+0000 7f28cb000640 1 -- 172.21.15.86:0/3010972078 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28c4143250 con 0x7f28c4146130 2024-04-20T21:20:13.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.115+0000 7f28cb000640 1 -- 172.21.15.86:0/3010972078 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28c40689e0 con 0x7f28c4146130 2024-04-20T21:20:13.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.116+0000 7f28bbe00640 1 -- 172.21.15.86:0/3010972078 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f28bc012b70 con 0x7f28c4146130 2024-04-20T21:20:13.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.117+0000 7f28bbe00640 1 --2- 172.21.15.86:0/3010972078 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f28a406c8f0 0x7f28a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:13.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.118+0000 7f28bbe00640 1 -- 172.21.15.86:0/3010972078 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f28bc08bff0 con 0x7f28c4146130 2024-04-20T21:20:13.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.118+0000 7f28c3e00640 1 --2- 172.21.15.86:0/3010972078 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f28a406c8f0 0x7f28a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:13.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.119+0000 7f28c3e00640 1 --2- 172.21.15.86:0/3010972078 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f28a406c8f0 0x7f28a406edb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f28b4036000 tx=0x7f28b402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:13.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.122+0000 7f28bbe00640 1 -- 172.21.15.86:0/3010972078 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f28bc05a3e0 con 0x7f28c4146130 2024-04-20T21:20:13.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.362+0000 7f28cb000640 1 -- 172.21.15.86:0/3010972078 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28c40b52a0 con 0x7f28c4146130 2024-04-20T21:20:13.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.363+0000 7f28bbe00640 1 -- 172.21.15.86:0/3010972078 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5199 (secure 0 0 0) 0x7f28bc059ce0 con 0x7f28c4146130 2024-04-20T21:20:13.363 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:20:13.363 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5044,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/1013106052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":1013106052},{"type":"v1","addr":"172.21.15.144:6833","nonce":1013106052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5047,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/1073576945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1073576945},{"type":"v1","addr":"172.21.15.144:6835","nonce":1073576945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/821548012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":821548012},{"type":"v1","addr":"172.21.15.86:6835","nonce":821548012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:20:03.369281+0000","modified":"2024-04-20T21:20:06.438592+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5691},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5691":{"gid":5691,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/712628984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":712628984},{"type":"v1","addr":"172.21.15.86:6837","nonce":712628984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5691,"qdb_cluster":[5691]},"id":3}]} 2024-04-20T21:20:13.363 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 24 2024-04-20T21:20:13.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.366+0000 7f28cb000640 1 -- 172.21.15.86:0/3010972078 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f28a406c8f0 msgr2=0x7f28a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:13.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.366+0000 7f28cb000640 1 --2- 172.21.15.86:0/3010972078 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f28a406c8f0 0x7f28a406edb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f28b4036000 tx=0x7f28b402e000 comp rx=0 tx=0).stop 2024-04-20T21:20:13.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.366+0000 7f28cb000640 1 -- 172.21.15.86:0/3010972078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28c4146130 msgr2=0x7f28c415bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:13.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.366+0000 7f28cb000640 1 --2- 172.21.15.86:0/3010972078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28c4146130 0x7f28c415bfa0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f28bc00d5d0 tx=0x7f28bc00da90 comp rx=0 tx=0).stop 2024-04-20T21:20:13.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.366+0000 7f28cb000640 1 -- 172.21.15.86:0/3010972078 shutdown_connections 2024-04-20T21:20:13.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.366+0000 7f28cb000640 1 --2- 172.21.15.86:0/3010972078 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f28a406c8f0 0x7f28a406edb0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:13.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.366+0000 7f28cb000640 1 --2- 172.21.15.86:0/3010972078 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28c4146130 0x7f28c415bfa0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:13.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.366+0000 7f28cb000640 1 --2- 172.21.15.86:0/3010972078 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28c40ccdd0 0x7f28c4142480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:13.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.366+0000 7f28cb000640 1 --2- 172.21.15.86:0/3010972078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f28c4072410 0x7f28c41f8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:13.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.366+0000 7f28cb000640 1 -- 172.21.15.86:0/3010972078 >> 172.21.15.86:0/3010972078 conn(0x7f28c40b2360 msgr2=0x7f28c4071d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:13.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.366+0000 7f28cb000640 1 -- 172.21.15.86:0/3010972078 shutdown_connections 2024-04-20T21:20:13.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.366+0000 7f28cb000640 1 -- 172.21.15.86:0/3010972078 wait complete. 2024-04-20T21:20:13.385 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-20T21:20:13.624 INFO:teuthology.orchestra.run.smithi086.stdout:{"message":"","return_code":0} 2024-04-20T21:20:13.625 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-20T21:20:13.625 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-20T21:20:13.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.720+0000 7f73c3c41f40 1 Processor -- start 2024-04-20T21:20:13.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.720+0000 7f73c3c41f40 1 -- start start 2024-04-20T21:20:13.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.721+0000 7f73c3c41f40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558000073290 0x558000103550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:13.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.721+0000 7f73c3c41f40 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557ffff9ab20 0x557ffff9af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:13.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.721+0000 7f73c3c41f40 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55800005b800 0x55800005bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:13.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.721+0000 7f73c3c41f40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557ffffe8a50 con 0x558000073290 2024-04-20T21:20:13.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.721+0000 7f73c3c41f40 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557ffff43cc0 con 0x55800005b800 2024-04-20T21:20:13.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.721+0000 7f73c3c41f40 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x557ffffe8cf0 con 0x557ffff9ab20 2024-04-20T21:20:13.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.721+0000 7f73c0600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558000073290 0x558000103550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:13.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.721+0000 7f73c0600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558000073290 0x558000103550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45758/0 (socket says 172.21.15.86:45758) 2024-04-20T21:20:13.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.721+0000 7f73c0600640 1 -- 172.21.15.86:0/3935814424 learned_addr learned my addr 172.21.15.86:0/3935814424 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:13.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.721+0000 7f73c0600640 1 -- 172.21.15.86:0/3935814424 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557ffff9ab20 msgr2=0x557ffff9af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:13.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.721+0000 7f73c0600640 1 --2- 172.21.15.86:0/3935814424 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557ffff9ab20 0x557ffff9af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:13.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.721+0000 7f73c0600640 1 -- 172.21.15.86:0/3935814424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55800005b800 msgr2=0x55800005bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:13.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.721+0000 7f73c0600640 1 --2- 172.21.15.86:0/3935814424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55800005b800 0x55800005bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:13.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.721+0000 7f73c0600640 1 -- 172.21.15.86:0/3935814424 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558000058d70 con 0x558000073290 2024-04-20T21:20:13.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.722+0000 7f73c0600640 1 --2- 172.21.15.86:0/3935814424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558000073290 0x558000103550 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f73b0009790 tx=0x7f73b0009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=370ab27e626f0fd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:13.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.722+0000 7f73be800640 1 -- 172.21.15.86:0/3935814424 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b0019070 con 0x558000073290 2024-04-20T21:20:13.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.722+0000 7f73be800640 1 -- 172.21.15.86:0/3935814424 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73b00088b0 con 0x558000073290 2024-04-20T21:20:13.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.722+0000 7f73be800640 1 -- 172.21.15.86:0/3935814424 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b0008ca0 con 0x558000073290 2024-04-20T21:20:13.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.722+0000 7f73c3c41f40 1 -- 172.21.15.86:0/3935814424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558000073290 msgr2=0x558000103550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:13.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.722+0000 7f73c3c41f40 1 --2- 172.21.15.86:0/3935814424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558000073290 0x558000103550 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f73b0009790 tx=0x7f73b0009880 comp rx=0 tx=0).stop 2024-04-20T21:20:13.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.723+0000 7f73c3c41f40 1 -- 172.21.15.86:0/3935814424 shutdown_connections 2024-04-20T21:20:13.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.723+0000 7f73c3c41f40 1 --2- 172.21.15.86:0/3935814424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55800005b800 0x55800005bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:13.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.723+0000 7f73c3c41f40 1 --2- 172.21.15.86:0/3935814424 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557ffff9ab20 0x557ffff9af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:13.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.723+0000 7f73c3c41f40 1 --2- 172.21.15.86:0/3935814424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558000073290 0x558000103550 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:13.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.723+0000 7f73c3c41f40 1 -- 172.21.15.86:0/3935814424 >> 172.21.15.86:0/3935814424 conn(0x557ffffa1950 msgr2=0x557ffffe6270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:13.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.723+0000 7f73c3c41f40 1 -- 172.21.15.86:0/3935814424 shutdown_connections 2024-04-20T21:20:13.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.723+0000 7f73c3c41f40 1 -- 172.21.15.86:0/3935814424 wait complete. 2024-04-20T21:20:13.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.723+0000 7f73c3c41f40 1 Processor -- start 2024-04-20T21:20:13.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.724+0000 7f73c3c41f40 1 -- start start 2024-04-20T21:20:13.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.724+0000 7f73c3c41f40 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557ffff9ab20 0x558000104f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:13.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.724+0000 7f73c3c41f40 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55800005b800 0x557ffffe6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:13.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.724+0000 7f73c3c41f40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558000073290 0x55800010ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:13.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.724+0000 7f73c3c41f40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557ffffcb330 con 0x558000073290 2024-04-20T21:20:13.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.724+0000 7f73c3c41f40 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557ffffcad20 con 0x55800005b800 2024-04-20T21:20:13.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.724+0000 7f73c3c41f40 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x557ffff9b440 con 0x557ffff9ab20 2024-04-20T21:20:13.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.725+0000 7f73c1000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558000073290 0x55800010ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:13.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.725+0000 7f73c1000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558000073290 0x55800010ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45764/0 (socket says 172.21.15.86:45764) 2024-04-20T21:20:13.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.725+0000 7f73c1000640 1 -- 172.21.15.86:0/1839736279 learned_addr learned my addr 172.21.15.86:0/1839736279 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:13.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.725+0000 7f73c1000640 1 -- 172.21.15.86:0/1839736279 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557ffff9ab20 msgr2=0x558000104f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:13.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.725+0000 7f73c1000640 1 --2- 172.21.15.86:0/1839736279 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557ffff9ab20 0x558000104f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:13.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.725+0000 7f73c1000640 1 -- 172.21.15.86:0/1839736279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55800005b800 msgr2=0x557ffffe6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:13.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.725+0000 7f73c1000640 1 --2- 172.21.15.86:0/1839736279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55800005b800 0x557ffffe6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:13.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.725+0000 7f73c1000640 1 -- 172.21.15.86:0/1839736279 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55800005b1e0 con 0x558000073290 2024-04-20T21:20:13.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.725+0000 7f73c1000640 1 --2- 172.21.15.86:0/1839736279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558000073290 0x55800010ab10 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f73b801fa80 tx=0x7f73b8001310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:13.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.726+0000 7f73bd400640 1 -- 172.21.15.86:0/1839736279 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b805b070 con 0x558000073290 2024-04-20T21:20:13.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.726+0000 7f73bd400640 1 -- 172.21.15.86:0/1839736279 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73b8006270 con 0x558000073290 2024-04-20T21:20:13.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.726+0000 7f73bd400640 1 -- 172.21.15.86:0/1839736279 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b80068f0 con 0x558000073290 2024-04-20T21:20:13.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.726+0000 7f73c3c41f40 1 -- 172.21.15.86:0/1839736279 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55800005b4c0 con 0x558000073290 2024-04-20T21:20:13.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.726+0000 7f73c3c41f40 1 -- 172.21.15.86:0/1839736279 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558000102ba0 con 0x558000073290 2024-04-20T21:20:13.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.728+0000 7f73bd400640 1 -- 172.21.15.86:0/1839736279 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f73b804dc90 con 0x558000073290 2024-04-20T21:20:13.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.729+0000 7f73bd400640 1 --2- 172.21.15.86:0/1839736279 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f73a006b2d0 0x7f73a006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:13.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.729+0000 7f73bd400640 1 -- 172.21.15.86:0/1839736279 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f73b80ccf30 con 0x558000073290 2024-04-20T21:20:13.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.729+0000 7f73c0600640 1 --2- 172.21.15.86:0/1839736279 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f73a006b2d0 0x7f73a006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:13.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.729+0000 7f73c3c41f40 1 --2- 172.21.15.86:0/1839736279 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x55800010b050 0x558000075d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:13.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.729+0000 7f73c3c41f40 1 -- 172.21.15.86:0/1839736279 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:1 7.7 7:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45) -- 0x55800005c2d0 con 0x55800010b050 2024-04-20T21:20:13.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.730+0000 7f73c0600640 1 --2- 172.21.15.86:0/1839736279 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f73a006b2d0 0x7f73a006d790 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f73b00098c0 tx=0x7f73b0011000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:13.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.730+0000 7f73bfc00640 1 --2- 172.21.15.86:0/1839736279 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x55800010b050 0x558000075d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:13.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.731+0000 7f73bfc00640 1 --2- 172.21.15.86:0/1839736279 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x55800010b050 0x558000075d70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:13.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.732+0000 7f73bfc00640 1 -- 172.21.15.86:0/1839736279 <== osd.6 v2:172.21.15.144:6824/1893803079 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv1 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x7f73ac00a6c0 con 0x55800010b050 2024-04-20T21:20:13.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.732+0000 7f73c3c41f40 1 -- 172.21.15.86:0/1839736279 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x55800010b050 msgr2=0x558000075d70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:13.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.732+0000 7f73c3c41f40 1 --2- 172.21.15.86:0/1839736279 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x55800010b050 0x558000075d70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:13.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.732+0000 7f73c3c41f40 1 -- 172.21.15.86:0/1839736279 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f73a006b2d0 msgr2=0x7f73a006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:13.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.732+0000 7f73c3c41f40 1 --2- 172.21.15.86:0/1839736279 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f73a006b2d0 0x7f73a006d790 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f73b00098c0 tx=0x7f73b0011000 comp rx=0 tx=0).stop 2024-04-20T21:20:13.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.732+0000 7f73c3c41f40 1 -- 172.21.15.86:0/1839736279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558000073290 msgr2=0x55800010ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:13.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.732+0000 7f73c3c41f40 1 --2- 172.21.15.86:0/1839736279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558000073290 0x55800010ab10 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f73b801fa80 tx=0x7f73b8001310 comp rx=0 tx=0).stop 2024-04-20T21:20:13.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.732+0000 7f73c1000640 1 -- 172.21.15.86:0/1839736279 reap_dead start 2024-04-20T21:20:13.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.732+0000 7f73c3c41f40 1 -- 172.21.15.86:0/1839736279 shutdown_connections 2024-04-20T21:20:13.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.732+0000 7f73c3c41f40 1 -- 172.21.15.86:0/1839736279 >> 172.21.15.86:0/1839736279 conn(0x557ffffa1950 msgr2=0x558000059e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:13.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.732+0000 7f73c3c41f40 1 -- 172.21.15.86:0/1839736279 shutdown_connections 2024-04-20T21:20:13.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.732+0000 7f73c3c41f40 1 -- 172.21.15.86:0/1839736279 wait complete. 2024-04-20T21:20:13.736 INFO:teuthology.orchestra.run.smithi086.stdout:Yalphaparent_a 2024-04-20T21:20:13.737 DEBUG:teuthology.orchestra.run.smithi086:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-20T21:20:13.883 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:20:13.883 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 1099511627777, 2024-04-20T21:20:13.883 INFO:teuthology.orchestra.run.smithi086.stdout: "ancestors": [ 2024-04-20T21:20:13.883 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:20:13.883 INFO:teuthology.orchestra.run.smithi086.stdout: "dirino": 1099511627776, 2024-04-20T21:20:13.884 INFO:teuthology.orchestra.run.smithi086.stdout: "dname": "alpha", 2024-04-20T21:20:13.884 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 2 2024-04-20T21:20:13.884 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:20:13.884 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:20:13.884 INFO:teuthology.orchestra.run.smithi086.stdout: "dirino": 1, 2024-04-20T21:20:13.884 INFO:teuthology.orchestra.run.smithi086.stdout: "dname": "parent_a", 2024-04-20T21:20:13.884 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 4 2024-04-20T21:20:13.884 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:20:13.884 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-20T21:20:13.884 INFO:teuthology.orchestra.run.smithi086.stdout: "pool": 7, 2024-04-20T21:20:13.884 INFO:teuthology.orchestra.run.smithi086.stdout: "old_pools": [] 2024-04-20T21:20:13.884 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:20:13.884 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:20:13.893 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'oh i'"'"'m sorry did i overwrite your xattr?' 2024-04-20T21:20:13.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.999+0000 7f80f0648f40 1 Processor -- start 2024-04-20T21:20:14.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:13.999+0000 7f80f0648f40 1 -- start start 2024-04-20T21:20:14.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.000+0000 7f80f0648f40 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5570376dbe30 0x55703774dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.000+0000 7f80f0648f40 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55703774e610 0x55703774fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.000+0000 7f80f0648f40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557037751ed0 0x557037750140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.000+0000 7f80f0648f40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5570376dda50 con 0x557037751ed0 2024-04-20T21:20:14.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.000+0000 7f80f0648f40 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557037638cc0 con 0x55703774e610 2024-04-20T21:20:14.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.000+0000 7f80f0648f40 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5570376ddcf0 con 0x5570376dbe30 2024-04-20T21:20:14.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.000+0000 7f80edc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557037751ed0 0x557037750140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:14.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.001+0000 7f80edc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557037751ed0 0x557037750140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45778/0 (socket says 172.21.15.86:45778) 2024-04-20T21:20:14.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.001+0000 7f80edc00640 1 -- 172.21.15.86:0/2688330280 learned_addr learned my addr 172.21.15.86:0/2688330280 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:14.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.001+0000 7f80edc00640 1 -- 172.21.15.86:0/2688330280 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5570376dbe30 msgr2=0x55703774dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:14.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.001+0000 7f80edc00640 1 --2- 172.21.15.86:0/2688330280 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5570376dbe30 0x55703774dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.001+0000 7f80edc00640 1 -- 172.21.15.86:0/2688330280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55703774e610 msgr2=0x55703774fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:14.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.001+0000 7f80edc00640 1 --2- 172.21.15.86:0/2688330280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55703774e610 0x55703774fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.001+0000 7f80edc00640 1 -- 172.21.15.86:0/2688330280 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570376c2d80 con 0x557037751ed0 2024-04-20T21:20:14.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.001+0000 7f80edc00640 1 --2- 172.21.15.86:0/2688330280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557037751ed0 0x557037750140 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f80e4001690 tx=0x7f80e4001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3688975a28efcae6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:14.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.002+0000 7f80eb400640 1 -- 172.21.15.86:0/2688330280 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e4003650 con 0x557037751ed0 2024-04-20T21:20:14.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.002+0000 7f80eb400640 1 -- 172.21.15.86:0/2688330280 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80e4003810 con 0x557037751ed0 2024-04-20T21:20:14.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.002+0000 7f80eb400640 1 -- 172.21.15.86:0/2688330280 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e4003c00 con 0x557037751ed0 2024-04-20T21:20:14.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.002+0000 7f80f0648f40 1 -- 172.21.15.86:0/2688330280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557037751ed0 msgr2=0x557037750140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:14.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.002+0000 7f80f0648f40 1 --2- 172.21.15.86:0/2688330280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557037751ed0 0x557037750140 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f80e4001690 tx=0x7f80e4001b50 comp rx=0 tx=0).stop 2024-04-20T21:20:14.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.002+0000 7f80f0648f40 1 -- 172.21.15.86:0/2688330280 shutdown_connections 2024-04-20T21:20:14.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.002+0000 7f80f0648f40 1 --2- 172.21.15.86:0/2688330280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557037751ed0 0x557037750140 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.002+0000 7f80f0648f40 1 --2- 172.21.15.86:0/2688330280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55703774e610 0x55703774fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.002+0000 7f80f0648f40 1 --2- 172.21.15.86:0/2688330280 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5570376dbe30 0x55703774dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.002+0000 7f80f0648f40 1 -- 172.21.15.86:0/2688330280 >> 172.21.15.86:0/2688330280 conn(0x557037696950 msgr2=0x55703776af00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:14.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.002+0000 7f80f0648f40 1 -- 172.21.15.86:0/2688330280 shutdown_connections 2024-04-20T21:20:14.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.002+0000 7f80f0648f40 1 -- 172.21.15.86:0/2688330280 wait complete. 2024-04-20T21:20:14.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.003+0000 7f80f0648f40 1 Processor -- start 2024-04-20T21:20:14.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.003+0000 7f80f0648f40 1 -- start start 2024-04-20T21:20:14.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.004+0000 7f80f0648f40 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5570376dbe30 0x55703774c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.004+0000 7f80f0648f40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55703774e610 0x55703776c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.004+0000 7f80f0648f40 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557037751ed0 0x55703776c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.004+0000 7f80f0648f40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55703774d510 con 0x55703774e610 2024-04-20T21:20:14.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.004+0000 7f80f0648f40 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5570376c3020 con 0x557037751ed0 2024-04-20T21:20:14.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.004+0000 7f80f0648f40 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5570376c3560 con 0x5570376dbe30 2024-04-20T21:20:14.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.004+0000 7f80ec800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55703774e610 0x55703776c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:14.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.004+0000 7f80ec800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55703774e610 0x55703776c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45794/0 (socket says 172.21.15.86:45794) 2024-04-20T21:20:14.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.004+0000 7f80ec800640 1 -- 172.21.15.86:0/111634926 learned_addr learned my addr 172.21.15.86:0/111634926 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:14.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.005+0000 7f80ed200640 1 --2- 172.21.15.86:0/111634926 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5570376dbe30 0x55703774c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:14.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.005+0000 7f80edc00640 1 --2- 172.21.15.86:0/111634926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557037751ed0 0x55703776c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:14.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.005+0000 7f80ec800640 1 -- 172.21.15.86:0/111634926 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5570376dbe30 msgr2=0x55703774c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:14.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.005+0000 7f80ec800640 1 --2- 172.21.15.86:0/111634926 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5570376dbe30 0x55703774c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.005+0000 7f80ec800640 1 -- 172.21.15.86:0/111634926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557037751ed0 msgr2=0x55703776c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:14.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.005+0000 7f80ec800640 1 --2- 172.21.15.86:0/111634926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557037751ed0 0x55703776c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.005+0000 7f80ec800640 1 -- 172.21.15.86:0/111634926 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570376c20a0 con 0x55703774e610 2024-04-20T21:20:14.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.005+0000 7f80edc00640 1 --2- 172.21.15.86:0/111634926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557037751ed0 0x55703776c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:20:14.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.005+0000 7f80ec800640 1 --2- 172.21.15.86:0/111634926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55703774e610 0x55703776c330 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f80d800adb0 tx=0x7f80d800aea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:14.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.005+0000 7f80ea000640 1 -- 172.21.15.86:0/111634926 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d801a070 con 0x55703774e610 2024-04-20T21:20:14.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.006+0000 7f80ea000640 1 -- 172.21.15.86:0/111634926 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80d8008a70 con 0x55703774e610 2024-04-20T21:20:14.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.006+0000 7f80f0648f40 1 -- 172.21.15.86:0/111634926 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5570376c2380 con 0x55703774e610 2024-04-20T21:20:14.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.006+0000 7f80ea000640 1 -- 172.21.15.86:0/111634926 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d8015420 con 0x55703774e610 2024-04-20T21:20:14.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.006+0000 7f80f0648f40 1 -- 172.21.15.86:0/111634926 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570377704c0 con 0x55703774e610 2024-04-20T21:20:14.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.008+0000 7f80ea000640 1 -- 172.21.15.86:0/111634926 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f80d801d070 con 0x55703774e610 2024-04-20T21:20:14.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.009+0000 7f80ea000640 1 --2- 172.21.15.86:0/111634926 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f80cc06b2d0 0x7f80cc06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.009+0000 7f80ea000640 1 -- 172.21.15.86:0/111634926 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f80d8089b10 con 0x55703774e610 2024-04-20T21:20:14.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.009+0000 7f80ed200640 1 --2- 172.21.15.86:0/111634926 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f80cc06b2d0 0x7f80cc06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:14.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.009+0000 7f80f0648f40 1 --2- 172.21.15.86:0/111634926 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x55703776bab0 0x5570378016f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.009+0000 7f80f0648f40 1 -- 172.21.15.86:0/111634926 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:1 7.7 7:e2e07930:::10000000001.00000000:head [setxattr parent (40) in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45) -- 0x557037801c30 con 0x55703776bab0 2024-04-20T21:20:14.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.010+0000 7f80ed200640 1 --2- 172.21.15.86:0/111634926 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f80cc06b2d0 0x7f80cc06d790 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f80e0054000 tx=0x7f80e004c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:14.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.010+0000 7f80edc00640 1 --2- 172.21.15.86:0/111634926 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x55703776bab0 0x5570378016f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:14.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.011+0000 7f80edc00640 1 --2- 172.21.15.86:0/111634926 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x55703776bab0 0x5570378016f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:14.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.013+0000 7f80edc00640 1 -- 172.21.15.86:0/111634926 <== osd.6 v2:172.21.15.144:6824/1893803079 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (40)] v45'2 uv2 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f80e400f030 con 0x55703776bab0 2024-04-20T21:20:14.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.013+0000 7f80f0648f40 1 -- 172.21.15.86:0/111634926 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x55703776bab0 msgr2=0x5570378016f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:14.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.013+0000 7f80f0648f40 1 --2- 172.21.15.86:0/111634926 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x55703776bab0 0x5570378016f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.013+0000 7f80f0648f40 1 -- 172.21.15.86:0/111634926 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f80cc06b2d0 msgr2=0x7f80cc06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:14.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.013+0000 7f80f0648f40 1 --2- 172.21.15.86:0/111634926 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f80cc06b2d0 0x7f80cc06d790 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f80e0054000 tx=0x7f80e004c000 comp rx=0 tx=0).stop 2024-04-20T21:20:14.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.014+0000 7f80f0648f40 1 -- 172.21.15.86:0/111634926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55703774e610 msgr2=0x55703776c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:14.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.014+0000 7f80f0648f40 1 --2- 172.21.15.86:0/111634926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55703774e610 0x55703776c330 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f80d800adb0 tx=0x7f80d800aea0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.014+0000 7f80edc00640 1 -- 172.21.15.86:0/111634926 reap_dead start 2024-04-20T21:20:14.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.014+0000 7f80f0648f40 1 -- 172.21.15.86:0/111634926 shutdown_connections 2024-04-20T21:20:14.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.014+0000 7f80f0648f40 1 -- 172.21.15.86:0/111634926 >> 172.21.15.86:0/111634926 conn(0x557037696950 msgr2=0x55703774dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:14.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.014+0000 7f80f0648f40 1 -- 172.21.15.86:0/111634926 shutdown_connections 2024-04-20T21:20:14.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.014+0000 7f80f0648f40 1 -- 172.21.15.86:0/111634926 wait complete. 2024-04-20T21:20:14.019 DEBUG:teuthology.orchestra.run.smithi086:> sudo daemon-helper kill ceph --cluster ceph -w 2024-04-20T21:20:14.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:20:14.024 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub start / repair,recursive 2024-04-20T21:20:14.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.201+0000 7f3289c00640 1 Processor -- start 2024-04-20T21:20:14.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.201+0000 7f3289c00640 1 -- start start 2024-04-20T21:20:14.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.202+0000 7f3289c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f328415ec20 0x7f32840c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.202+0000 7f3289c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3284163080 0x7f32840d4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.202+0000 7f3289c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32841634d0 con 0x7f328415e840 2024-04-20T21:20:14.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.202+0000 7f3289c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f32840d3510 con 0x7f328415ec20 2024-04-20T21:20:14.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.202+0000 7f3289c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f32840cf5b0 con 0x7f3284163080 2024-04-20T21:20:14.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.203+0000 7f3282000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f328415ec20 0x7f32840c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:14.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.203+0000 7f3282000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f328415ec20 0x7f32840c82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45808/0 (socket says 172.21.15.86:45808) 2024-04-20T21:20:14.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.203+0000 7f3282000640 1 -- 172.21.15.86:0/1268561824 learned_addr learned my addr 172.21.15.86:0/1268561824 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:14.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.203+0000 7f3283400640 1 --2- 172.21.15.86:0/1268561824 >> v2:172.21.15.144:3300/0 conn(0x7f3284163080 0x7f32840d4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:14.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.203+0000 7f3282000640 1 -- 172.21.15.86:0/1268561824 >> v2:172.21.15.144:3300/0 conn(0x7f3284163080 msgr2=0x7f32840d4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:14.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.203+0000 7f3282000640 1 --2- 172.21.15.86:0/1268561824 >> v2:172.21.15.144:3300/0 conn(0x7f3284163080 0x7f32840d4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.203+0000 7f3282000640 1 -- 172.21.15.86:0/1268561824 >> v1:172.21.15.144:6789/0 conn(0x7f328415e840 legacy=0x7f32840d2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:14.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.203+0000 7f3282000640 1 -- 172.21.15.86:0/1268561824 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32840cf890 con 0x7f328415ec20 2024-04-20T21:20:14.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.205+0000 7f3282000640 1 --2- 172.21.15.86:0/1268561824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f328415ec20 0x7f32840c82a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f3284162500 tx=0x7f327002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=142f7d5922df1d15 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:14.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.205+0000 7f3281600640 1 -- 172.21.15.86:0/1268561824 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3270035020 con 0x7f328415ec20 2024-04-20T21:20:14.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.205+0000 7f3281600640 1 -- 172.21.15.86:0/1268561824 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3270031870 con 0x7f328415ec20 2024-04-20T21:20:14.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.205+0000 7f3281600640 1 -- 172.21.15.86:0/1268561824 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3270031c40 con 0x7f328415ec20 2024-04-20T21:20:14.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.206+0000 7f3289c00640 1 -- 172.21.15.86:0/1268561824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f328415ec20 msgr2=0x7f32840c82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:14.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.206+0000 7f3289c00640 1 --2- 172.21.15.86:0/1268561824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f328415ec20 0x7f32840c82a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f3284162500 tx=0x7f327002b910 comp rx=0 tx=0).stop 2024-04-20T21:20:14.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.206+0000 7f3289c00640 1 -- 172.21.15.86:0/1268561824 shutdown_connections 2024-04-20T21:20:14.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.206+0000 7f3289c00640 1 --2- 172.21.15.86:0/1268561824 >> v2:172.21.15.144:3300/0 conn(0x7f3284163080 0x7f32840d4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.206+0000 7f3289c00640 1 --2- 172.21.15.86:0/1268561824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f328415ec20 0x7f32840c82a0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.206+0000 7f3289c00640 1 -- 172.21.15.86:0/1268561824 >> 172.21.15.86:0/1268561824 conn(0x7f328409da40 msgr2=0x7f328408bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:14.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.206+0000 7f3289c00640 1 -- 172.21.15.86:0/1268561824 shutdown_connections 2024-04-20T21:20:14.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.206+0000 7f3289c00640 1 -- 172.21.15.86:0/1268561824 wait complete. 2024-04-20T21:20:14.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.209+0000 7f3289c00640 1 Processor -- start 2024-04-20T21:20:14.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.209+0000 7f3289c00640 1 -- start start 2024-04-20T21:20:14.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.211+0000 7f3289c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f328415e840 0x7f328414dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.211+0000 7f3289c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f328415ec20 0x7f3284161220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.211+0000 7f3289c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3284163080 0x7f3284161760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.211+0000 7f3289c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f32840d4940 con 0x7f328415e840 2024-04-20T21:20:14.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.211+0000 7f3289c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f32840cf260 con 0x7f3284163080 2024-04-20T21:20:14.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.211+0000 7f3289c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f32840d4ac0 con 0x7f328415ec20 2024-04-20T21:20:14.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.211+0000 7f3282a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f328415e840 0x7f328414dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:14.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.211+0000 7f3282a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f328415e840 0x7f328414dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45812/0 (socket says 172.21.15.86:45812) 2024-04-20T21:20:14.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.211+0000 7f3282a00640 1 -- 172.21.15.86:0/1032989186 learned_addr learned my addr 172.21.15.86:0/1032989186 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:14.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.212+0000 7f3282a00640 1 -- 172.21.15.86:0/1032989186 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f328415ec20 msgr2=0x7f3284161220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:14.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.212+0000 7f3282a00640 1 --2- 172.21.15.86:0/1032989186 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f328415ec20 0x7f3284161220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.212+0000 7f3283400640 1 --2- 172.21.15.86:0/1032989186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3284163080 0x7f3284161760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:14.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.212+0000 7f3282a00640 1 -- 172.21.15.86:0/1032989186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3284163080 msgr2=0x7f3284161760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:14.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.212+0000 7f3282a00640 1 --2- 172.21.15.86:0/1032989186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3284163080 0x7f3284161760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.212+0000 7f3282a00640 1 -- 172.21.15.86:0/1032989186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32840cc8c0 con 0x7f328415e840 2024-04-20T21:20:14.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.212+0000 7f3282a00640 1 --2- 172.21.15.86:0/1032989186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f328415e840 0x7f328414dc80 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f327400c460 tx=0x7f327400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:14.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.212+0000 7f327aa00640 1 -- 172.21.15.86:0/1032989186 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3274016020 con 0x7f328415e840 2024-04-20T21:20:14.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.212+0000 7f327aa00640 1 -- 172.21.15.86:0/1032989186 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3274012870 con 0x7f328415e840 2024-04-20T21:20:14.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.212+0000 7f327aa00640 1 -- 172.21.15.86:0/1032989186 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3274012c60 con 0x7f328415e840 2024-04-20T21:20:14.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.212+0000 7f3289c00640 1 -- 172.21.15.86:0/1032989186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32840cb820 con 0x7f328415e840 2024-04-20T21:20:14.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.213+0000 7f3289c00640 1 -- 172.21.15.86:0/1032989186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32840cbde0 con 0x7f328415e840 2024-04-20T21:20:14.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.215+0000 7f327aa00640 1 -- 172.21.15.86:0/1032989186 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3274019020 con 0x7f328415e840 2024-04-20T21:20:14.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.217+0000 7f327aa00640 1 --2- 172.21.15.86:0/1032989186 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f326406c8f0 0x7f326406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.217+0000 7f327aa00640 1 -- 172.21.15.86:0/1032989186 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f327408b5d0 con 0x7f328415e840 2024-04-20T21:20:14.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.217+0000 7f3282000640 1 --2- 172.21.15.86:0/1032989186 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f326406c8f0 0x7f326406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:14.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.217+0000 7f328ca01740 1 Processor -- start 2024-04-20T21:20:14.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.217+0000 7f328ca01740 1 -- start start 2024-04-20T21:20:14.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.218+0000 7f3282000640 1 --2- 172.21.15.86:0/1032989186 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f326406c8f0 0x7f326406edb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3270036000 tx=0x7f3270030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:14.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.218+0000 7f328ca01740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560ab3323fe0 0x560ab32f6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.218+0000 7f328ca01740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ab33243c0 0x560ab3326d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.218+0000 7f328ca01740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ab32f74c0 0x560ab33272b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.218+0000 7f328ca01740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560ab3261d20 con 0x560ab33243c0 2024-04-20T21:20:14.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.218+0000 7f328ca01740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560ab32083e0 con 0x560ab32f74c0 2024-04-20T21:20:14.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.218+0000 7f328ca01740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x560ab325dce0 con 0x560ab3323fe0 2024-04-20T21:20:14.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.218+0000 7f3282000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ab33243c0 0x560ab3326d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:14.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.218+0000 7f3282000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ab33243c0 0x560ab3326d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45818/0 (socket says 172.21.15.86:45818) 2024-04-20T21:20:14.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.219+0000 7f3282000640 1 -- 172.21.15.86:0/2884333364 learned_addr learned my addr 172.21.15.86:0/2884333364 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:14.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.219+0000 7f3283400640 1 --2- 172.21.15.86:0/2884333364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ab32f74c0 0x560ab33272b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:14.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.219+0000 7f3282a00640 1 --2- 172.21.15.86:0/2884333364 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560ab3323fe0 0x560ab32f6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:14.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.219+0000 7f3282000640 1 -- 172.21.15.86:0/2884333364 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560ab3323fe0 msgr2=0x560ab32f6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:14.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.219+0000 7f3282000640 1 --2- 172.21.15.86:0/2884333364 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560ab3323fe0 0x560ab32f6f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.219+0000 7f3282000640 1 -- 172.21.15.86:0/2884333364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ab32f74c0 msgr2=0x560ab33272b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:14.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.219+0000 7f3282000640 1 --2- 172.21.15.86:0/2884333364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ab32f74c0 0x560ab33272b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.219+0000 7f3282000640 1 -- 172.21.15.86:0/2884333364 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ab3227980 con 0x560ab33243c0 2024-04-20T21:20:14.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.219+0000 7f3282000640 1 --2- 172.21.15.86:0/2884333364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ab33243c0 0x560ab3326d70 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f327002b950 tx=0x7f3270031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=345c9d1087259147 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:14.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.220+0000 7f326f400640 1 -- 172.21.15.86:0/2884333364 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3270040070 con 0x560ab33243c0 2024-04-20T21:20:14.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.220+0000 7f326f400640 1 -- 172.21.15.86:0/2884333364 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f327003abb0 con 0x560ab33243c0 2024-04-20T21:20:14.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.220+0000 7f326f400640 1 -- 172.21.15.86:0/2884333364 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3270035040 con 0x560ab33243c0 2024-04-20T21:20:14.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.220+0000 7f328ca01740 1 -- 172.21.15.86:0/2884333364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ab33243c0 msgr2=0x560ab3326d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:14.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.220+0000 7f328ca01740 1 --2- 172.21.15.86:0/2884333364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ab33243c0 0x560ab3326d70 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f327002b950 tx=0x7f3270031000 comp rx=0 tx=0).stop 2024-04-20T21:20:14.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.220+0000 7f328ca01740 1 -- 172.21.15.86:0/2884333364 shutdown_connections 2024-04-20T21:20:14.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.220+0000 7f328ca01740 1 --2- 172.21.15.86:0/2884333364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ab32f74c0 0x560ab33272b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.220+0000 7f328ca01740 1 --2- 172.21.15.86:0/2884333364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ab33243c0 0x560ab3326d70 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.220+0000 7f328ca01740 1 --2- 172.21.15.86:0/2884333364 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560ab3323fe0 0x560ab32f6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.220+0000 7f328ca01740 1 -- 172.21.15.86:0/2884333364 >> 172.21.15.86:0/2884333364 conn(0x560ab32f91f0 msgr2=0x560ab3341d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:14.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.221+0000 7f328ca01740 1 -- 172.21.15.86:0/2884333364 shutdown_connections 2024-04-20T21:20:14.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.221+0000 7f328ca01740 1 -- 172.21.15.86:0/2884333364 wait complete. 2024-04-20T21:20:14.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.221+0000 7f328ca01740 1 -- start start 2024-04-20T21:20:14.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.222+0000 7f328ca01740 1 Processor -- start 2024-04-20T21:20:14.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.222+0000 7f328ca01740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560ab3323fe0 0x560ab3329130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.223+0000 7f328ca01740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ab33243c0 0x560ab3342480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.223+0000 7f328ca01740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ab33225d0 0x560ab3320240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.223+0000 7f328ca01740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560ab32fb890 con 0x560ab33225d0 2024-04-20T21:20:14.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.223+0000 7f328ca01740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560ab321b7a0 con 0x560ab33243c0 2024-04-20T21:20:14.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.223+0000 7f328ca01740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x560ab32fe0d0 con 0x560ab3323fe0 2024-04-20T21:20:14.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.223+0000 7f328ca01740 10 client.0 _ll_register_callbacks cb 0x560ab321db80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:20:14.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.223+0000 7f3283400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ab33225d0 0x560ab3320240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:14.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.223+0000 7f3283400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ab33225d0 0x560ab3320240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45824/0 (socket says 172.21.15.86:45824) 2024-04-20T21:20:14.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.223+0000 7f3283400640 1 -- 172.21.15.86:0/2410593654 learned_addr learned my addr 172.21.15.86:0/2410593654 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:14.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.223+0000 7f3282a00640 1 --2- 172.21.15.86:0/2410593654 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560ab3323fe0 0x560ab3329130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:14.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.223+0000 7f3282000640 1 --2- 172.21.15.86:0/2410593654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ab33243c0 0x560ab3342480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:14.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.223+0000 7f3283400640 1 -- 172.21.15.86:0/2410593654 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560ab3323fe0 msgr2=0x560ab3329130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:14.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.223+0000 7f3283400640 1 --2- 172.21.15.86:0/2410593654 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560ab3323fe0 0x560ab3329130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.223+0000 7f3283400640 1 -- 172.21.15.86:0/2410593654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ab33243c0 msgr2=0x560ab3342480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:14.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.223+0000 7f3283400640 1 --2- 172.21.15.86:0/2410593654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ab33243c0 0x560ab3342480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.223+0000 7f3283400640 1 -- 172.21.15.86:0/2410593654 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ab303c7e0 con 0x560ab33225d0 2024-04-20T21:20:14.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.224+0000 7f3283400640 1 --2- 172.21.15.86:0/2410593654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ab33225d0 0x560ab3320240 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f327c007a00 tx=0x7f327c007ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c0cce3d493e8bb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:14.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.224+0000 7f3263e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:20:14.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.224+0000 7f3263e00640 1 -- 172.21.15.86:0/2410593654 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327c019020 con 0x560ab33225d0 2024-04-20T21:20:14.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.224+0000 7f3263e00640 1 -- 172.21.15.86:0/2410593654 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f327c010b50 con 0x560ab33225d0 2024-04-20T21:20:14.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.224+0000 7f328ca01740 1 -- 172.21.15.86:0/2410593654 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f327003abb0 con 0x560ab33225d0 2024-04-20T21:20:14.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.224+0000 7f063dc00640 1 Processor -- start 2024-04-20T21:20:14.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.224+0000 7f3263e00640 1 -- 172.21.15.86:0/2410593654 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f327c023050 con 0x560ab33225d0 2024-04-20T21:20:14.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.224+0000 7f328ca01740 10 client.5871 fetch_fsmap learned FSMap version 24 2024-04-20T21:20:14.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.224+0000 7f328ca01740 1 -- 172.21.15.86:0/2410593654 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x560ab309cf70 con 0x560ab33225d0 2024-04-20T21:20:14.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.224+0000 7f063dc00640 1 -- start start 2024-04-20T21:20:14.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.225+0000 7f3263e00640 1 -- 172.21.15.86:0/2410593654 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f327003abb0 con 0x560ab33225d0 2024-04-20T21:20:14.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.225+0000 7f3263e00640 1 -- 172.21.15.86:0/2410593654 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 24) ==== 3495+0+0 (secure 0 0 0) 0x7f327c026020 con 0x560ab33225d0 2024-04-20T21:20:14.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.225+0000 7f328ca01740 10 client.5871 fetch_fsmap finished waiting for FSMap version 24 2024-04-20T21:20:14.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.225+0000 7f328ca01740 10 client.5871 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-20T21:20:14.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.225+0000 7f328ca01740 20 client.5871 populate_metadata read hostname 'smithi086' 2024-04-20T21:20:14.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.225+0000 7f328ca01740 1 --2- 172.21.15.86:0/2410593654 >> [v2:172.21.15.86:6836/712628984,v1:172.21.15.86:6837/712628984] conn(0x560ab331e330 0x560ab3355730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.225+0000 7f328ca01740 4 client.5871 mds_command: new command op to 5691 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:20:14.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.225+0000 7f328ca01740 1 -- 172.21.15.86:0/2410593654 --> [v2:172.21.15.86:6836/712628984,v1:172.21.15.86:6837/712628984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ab326f740 con 0x560ab331e330 2024-04-20T21:20:14.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.225+0000 7f3282a00640 1 --2- 172.21.15.86:0/2410593654 >> [v2:172.21.15.86:6836/712628984,v1:172.21.15.86:6837/712628984] conn(0x560ab331e330 0x560ab3355730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:14.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.226+0000 7f3282a00640 1 --2- 172.21.15.86:0/2410593654 >> [v2:172.21.15.86:6836/712628984,v1:172.21.15.86:6837/712628984] conn(0x560ab331e330 0x560ab3355730 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9acf66d161935766 server_cookie=92e6435dcb0af566 in_seq=0 out_seq=0 2024-04-20T21:20:14.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.226+0000 7f3263e00640 10 client.5871 ms_handle_connect on v2:172.21.15.86:6836/712628984 2024-04-20T21:20:14.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.226+0000 7f063dc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f063815ecd0 0x7f06380d5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.226+0000 7f063dc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f06381630d0 0x7f06380d2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.226+0000 7f063dc00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0638163530 con 0x7f06380d3120 2024-04-20T21:20:14.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.226+0000 7f063dc00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f06380d5600 con 0x7f06381630d0 2024-04-20T21:20:14.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.226+0000 7f063dc00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f063815f260 con 0x7f063815ecd0 2024-04-20T21:20:14.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.226+0000 7f0636000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f06381630d0 0x7f06380d2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:14.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.226+0000 7f0637400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f06380d3120 0x7f06380d0ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39698/0 (socket says 172.21.15.86:39698) 2024-04-20T21:20:14.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.226+0000 7f0637400640 1 -- 172.21.15.86:0/4072379885 learned_addr learned my addr 172.21.15.86:0/4072379885 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:20:14.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.226+0000 7f0636a00640 1 --2- 172.21.15.86:0/4072379885 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f063815ecd0 0x7f06380d5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:14.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.226+0000 7f3263e00640 1 -- 172.21.15.86:0/2410593654 <== mds.0 v2:172.21.15.86:6836/712628984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560ab326f740 con 0x560ab331e330 2024-04-20T21:20:14.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.226+0000 7f3263e00640 10 client.5871 handle_command_reply: tid=0 2024-04-20T21:20:14.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.227+0000 7f0636000640 1 -- 172.21.15.86:0/4072379885 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f063815ecd0 msgr2=0x7f06380d5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:14.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.227+0000 7f0636000640 1 --2- 172.21.15.86:0/4072379885 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f063815ecd0 0x7f06380d5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.227+0000 7f0636000640 1 -- 172.21.15.86:0/4072379885 >> v1:172.21.15.86:6789/0 conn(0x7f06380d3120 legacy=0x7f06380d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:14.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.227+0000 7f328ca01740 1 client.5871 shutdown 2024-04-20T21:20:14.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.227+0000 7f0636000640 1 -- 172.21.15.86:0/4072379885 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f063815f540 con 0x7f06381630d0 2024-04-20T21:20:14.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.227+0000 7f0635600640 1 -- 172.21.15.86:0/4072379885 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 813255887 0 0) 0x7f0638163530 con 0x7f06380d3120 2024-04-20T21:20:14.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.227+0000 7f328ca01740 1 -- 172.21.15.86:0/2410593654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ab33225d0 msgr2=0x560ab3320240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:14.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.227+0000 7f328ca01740 1 --2- 172.21.15.86:0/2410593654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ab33225d0 0x560ab3320240 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f327c007a00 tx=0x7f327c007ec0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.227+0000 7f328ca01740 1 -- 172.21.15.86:0/2410593654 shutdown_connections 2024-04-20T21:20:14.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.227+0000 7f328ca01740 1 --2- 172.21.15.86:0/2410593654 >> [v2:172.21.15.86:6836/712628984,v1:172.21.15.86:6837/712628984] conn(0x560ab331e330 0x560ab3355730 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.227+0000 7f328ca01740 1 --2- 172.21.15.86:0/2410593654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ab33225d0 0x560ab3320240 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.227+0000 7f328ca01740 1 --2- 172.21.15.86:0/2410593654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ab33243c0 0x560ab3342480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.227+0000 7f328ca01740 1 --2- 172.21.15.86:0/2410593654 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560ab3323fe0 0x560ab3329130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.227+0000 7f328ca01740 1 -- 172.21.15.86:0/2410593654 >> 172.21.15.86:0/2410593654 conn(0x560ab32f74c0 msgr2=0x560ab3326810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:14.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.227+0000 7f3263e00640 0 client.5871 ms_handle_reset on v2:172.21.15.86:6836/712628984 2024-04-20T21:20:14.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.227+0000 7f328ca01740 1 -- 172.21.15.86:0/2410593654 shutdown_connections 2024-04-20T21:20:14.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.227+0000 7f328ca01740 1 -- 172.21.15.86:0/2410593654 wait complete. 2024-04-20T21:20:14.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.228+0000 7f328ca01740 20 client.5871 trim_cache size 0 max 16384 2024-04-20T21:20:14.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.228+0000 7f0636000640 1 --2- 172.21.15.86:0/4072379885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06381630d0 0x7f06380d2be0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f06240097a0 tx=0x7f062402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb03995a0686c2bf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:14.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.229+0000 7f0635600640 1 -- 172.21.15.86:0/4072379885 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0624035020 con 0x7f06381630d0 2024-04-20T21:20:14.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.229+0000 7f0635600640 1 -- 172.21.15.86:0/4072379885 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0624031830 con 0x7f06381630d0 2024-04-20T21:20:14.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.229+0000 7f0635600640 1 -- 172.21.15.86:0/4072379885 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0624031c00 con 0x7f06381630d0 2024-04-20T21:20:14.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.229+0000 7f063dc00640 1 -- 172.21.15.86:0/4072379885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06381630d0 msgr2=0x7f06380d2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:14.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.229+0000 7f063dc00640 1 --2- 172.21.15.86:0/4072379885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06381630d0 0x7f06380d2be0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f06240097a0 tx=0x7f062402b8d0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.229+0000 7f063dc00640 1 -- 172.21.15.86:0/4072379885 shutdown_connections 2024-04-20T21:20:14.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.229+0000 7f063dc00640 1 --2- 172.21.15.86:0/4072379885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06381630d0 0x7f06380d2be0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.229+0000 7f063dc00640 1 --2- 172.21.15.86:0/4072379885 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f063815ecd0 0x7f06380d5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.229+0000 7f063dc00640 1 -- 172.21.15.86:0/4072379885 >> 172.21.15.86:0/4072379885 conn(0x7f063808bf70 msgr2=0x7f06380d4700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:14.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.229+0000 7f063dc00640 1 -- 172.21.15.86:0/4072379885 shutdown_connections 2024-04-20T21:20:14.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.229+0000 7f063dc00640 1 -- 172.21.15.86:0/4072379885 wait complete. 2024-04-20T21:20:14.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.231+0000 7f063dc00640 1 Processor -- start 2024-04-20T21:20:14.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.231+0000 7f063dc00640 1 -- start start 2024-04-20T21:20:14.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.232+0000 7f063dc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06380d3120 0x7f06380ca380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.232+0000 7f063dc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f063815ecd0 0x7f06380b6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.232+0000 7f063dc00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06381630d0 0x7f06380b6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.232+0000 7f063dc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f063815fb00 con 0x7f06380d3120 2024-04-20T21:20:14.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.232+0000 7f063dc00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f063815f820 con 0x7f06381630d0 2024-04-20T21:20:14.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.232+0000 7f063dc00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0638160020 con 0x7f063815ecd0 2024-04-20T21:20:14.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.232+0000 7f0636a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06380d3120 0x7f06380ca380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:14.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.232+0000 7f0636a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06380d3120 0x7f06380ca380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45842/0 (socket says 172.21.15.86:45842) 2024-04-20T21:20:14.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.232+0000 7f0636a00640 1 -- 172.21.15.86:0/2326961910 learned_addr learned my addr 172.21.15.86:0/2326961910 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:14.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.233+0000 7f0637400640 1 --2- 172.21.15.86:0/2326961910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06381630d0 0x7f06380b6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:14.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.233+0000 7f0636000640 1 --2- 172.21.15.86:0/2326961910 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f063815ecd0 0x7f06380b6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:14.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.233+0000 7f0636a00640 1 -- 172.21.15.86:0/2326961910 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f063815ecd0 msgr2=0x7f06380b6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:14.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.233+0000 7f0636a00640 1 --2- 172.21.15.86:0/2326961910 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f063815ecd0 0x7f06380b6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.233+0000 7f0636a00640 1 -- 172.21.15.86:0/2326961910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06381630d0 msgr2=0x7f06380b6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:14.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.233+0000 7f0636a00640 1 --2- 172.21.15.86:0/2326961910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06381630d0 0x7f06380b6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.233+0000 7f0636a00640 1 -- 172.21.15.86:0/2326961910 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06381f7bd0 con 0x7f06380d3120 2024-04-20T21:20:14.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.233+0000 7f0636a00640 1 --2- 172.21.15.86:0/2326961910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06380d3120 0x7f06380ca380 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f062800d800 tx=0x7f062800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:14.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.233+0000 7f062ea00640 1 -- 172.21.15.86:0/2326961910 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062800b6c0 con 0x7f06380d3120 2024-04-20T21:20:14.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.233+0000 7f062ea00640 1 -- 172.21.15.86:0/2326961910 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0628019070 con 0x7f06380d3120 2024-04-20T21:20:14.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.233+0000 7f062ea00640 1 -- 172.21.15.86:0/2326961910 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062800ba80 con 0x7f06380d3120 2024-04-20T21:20:14.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.234+0000 7f063dc00640 1 -- 172.21.15.86:0/2326961910 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06381f7e70 con 0x7f06380d3120 2024-04-20T21:20:14.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.234+0000 7f063dc00640 1 -- 172.21.15.86:0/2326961910 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06381f8430 con 0x7f06380d3120 2024-04-20T21:20:14.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.234+0000 7f063dc00640 1 -- 172.21.15.86:0/2326961910 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f063806c3c0 con 0x7f06380d3120 2024-04-20T21:20:14.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.236+0000 7f062ea00640 1 -- 172.21.15.86:0/2326961910 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f062800bc20 con 0x7f06380d3120 2024-04-20T21:20:14.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.237+0000 7f062ea00640 1 --2- 172.21.15.86:0/2326961910 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f061806c8f0 0x7f061806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.237+0000 7f062ea00640 1 -- 172.21.15.86:0/2326961910 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f062808cad0 con 0x7f06380d3120 2024-04-20T21:20:14.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.237+0000 7f0636000640 1 --2- 172.21.15.86:0/2326961910 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f061806c8f0 0x7f061806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:14.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.237+0000 7f062ea00640 1 -- 172.21.15.86:0/2326961910 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+501 (secure 0 0 0) 0x7f063806c3c0 con 0x7f06380d3120 2024-04-20T21:20:14.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.237+0000 7f063dc00640 1 -- 172.21.15.86:0/2326961910 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7f06380690f0 con 0x7f06380d3120 2024-04-20T21:20:14.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.238+0000 7f0636000640 1 --2- 172.21.15.86:0/2326961910 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f061806c8f0 0x7f061806edb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0624036000 tx=0x7f0624008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:14.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.248+0000 7f328ca01740 1 Processor -- start 2024-04-20T21:20:14.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.249+0000 7f328ca01740 1 -- start start 2024-04-20T21:20:14.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.249+0000 7f328ca01740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ab33225d0 0x560ab3322ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.249+0000 7f328ca01740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ab3323fe0 0x560ab3327d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.249+0000 7f328ca01740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560ab33243c0 0x560ab33282b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.249+0000 7f328ca01740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3284200e20 con 0x560ab33225d0 2024-04-20T21:20:14.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.249+0000 7f328ca01740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f32841ffe80 con 0x560ab3323fe0 2024-04-20T21:20:14.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.249+0000 7f328ca01740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f32841ff2a0 con 0x560ab33243c0 2024-04-20T21:20:14.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.250+0000 7f3282a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ab33225d0 0x560ab3322ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:14.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.250+0000 7f3282a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ab33225d0 0x560ab3322ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45846/0 (socket says 172.21.15.86:45846) 2024-04-20T21:20:14.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.250+0000 7f3282a00640 1 -- 172.21.15.86:0/2840552201 learned_addr learned my addr 172.21.15.86:0/2840552201 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:14.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.250+0000 7f3282a00640 1 -- 172.21.15.86:0/2840552201 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560ab33243c0 msgr2=0x560ab33282b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:14.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.250+0000 7f3282a00640 1 --2- 172.21.15.86:0/2840552201 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560ab33243c0 0x560ab33282b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.250+0000 7f3282a00640 1 -- 172.21.15.86:0/2840552201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ab3323fe0 msgr2=0x560ab3327d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:14.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.250+0000 7f3282a00640 1 --2- 172.21.15.86:0/2840552201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ab3323fe0 0x560ab3327d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.250+0000 7f3282a00640 1 -- 172.21.15.86:0/2840552201 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ab2f34170 con 0x560ab33225d0 2024-04-20T21:20:14.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.250+0000 7f3282a00640 1 --2- 172.21.15.86:0/2840552201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ab33225d0 0x560ab3322ca0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f3274059290 tx=0x7f32740899d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d806e08af6e77742 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:14.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.251+0000 7f326ea00640 1 -- 172.21.15.86:0/2840552201 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3274076070 con 0x560ab33225d0 2024-04-20T21:20:14.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.251+0000 7f326ea00640 1 -- 172.21.15.86:0/2840552201 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3274089c10 con 0x560ab33225d0 2024-04-20T21:20:14.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.251+0000 7f326ea00640 1 -- 172.21.15.86:0/2840552201 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3274089070 con 0x560ab33225d0 2024-04-20T21:20:14.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.251+0000 7f328ca01740 1 -- 172.21.15.86:0/2840552201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ab33225d0 msgr2=0x560ab3322ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:14.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.251+0000 7f328ca01740 1 --2- 172.21.15.86:0/2840552201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ab33225d0 0x560ab3322ca0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f3274059290 tx=0x7f32740899d0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.252+0000 7f328ca01740 1 -- 172.21.15.86:0/2840552201 shutdown_connections 2024-04-20T21:20:14.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.252+0000 7f328ca01740 1 --2- 172.21.15.86:0/2840552201 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560ab33243c0 0x560ab33282b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.252+0000 7f328ca01740 1 --2- 172.21.15.86:0/2840552201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ab3323fe0 0x560ab3327d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.252+0000 7f328ca01740 1 --2- 172.21.15.86:0/2840552201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ab33225d0 0x560ab3322ca0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.252+0000 7f328ca01740 1 -- 172.21.15.86:0/2840552201 >> 172.21.15.86:0/2840552201 conn(0x560ab32f74c0 msgr2=0x560ab3326eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:14.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.252+0000 7f328ca01740 1 -- 172.21.15.86:0/2840552201 shutdown_connections 2024-04-20T21:20:14.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.252+0000 7f328ca01740 1 -- 172.21.15.86:0/2840552201 wait complete. 2024-04-20T21:20:14.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.252+0000 7f328ca01740 1 -- start start 2024-04-20T21:20:14.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.253+0000 7f328ca01740 1 Processor -- start 2024-04-20T21:20:14.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.253+0000 7f328ca01740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ab33225d0 0x560ab33c4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.253+0000 7f328ca01740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560ab3323fe0 0x560ab33bad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.253+0000 7f328ca01740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ab33243c0 0x560ab3342f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.253+0000 7f328ca01740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f32841ff830 con 0x560ab33225d0 2024-04-20T21:20:14.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.253+0000 7f328ca01740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560ab33417f0 con 0x560ab33243c0 2024-04-20T21:20:14.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.253+0000 7f328ca01740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x560ab33c4690 con 0x560ab3323fe0 2024-04-20T21:20:14.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.254+0000 7f328ca01740 10 client.0 _ll_register_callbacks cb 0x7f327c009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:20:14.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.254+0000 7f3282a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ab33225d0 0x560ab33c4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:14.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.254+0000 7f3282a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ab33225d0 0x560ab33c4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45860/0 (socket says 172.21.15.86:45860) 2024-04-20T21:20:14.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.254+0000 7f3282a00640 1 -- 172.21.15.86:0/1105489909 learned_addr learned my addr 172.21.15.86:0/1105489909 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:14.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.254+0000 7f3282a00640 1 -- 172.21.15.86:0/1105489909 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560ab3323fe0 msgr2=0x560ab33bad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:14.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.254+0000 7f3282a00640 1 --2- 172.21.15.86:0/1105489909 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560ab3323fe0 0x560ab33bad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.254+0000 7f3282a00640 1 -- 172.21.15.86:0/1105489909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ab33243c0 msgr2=0x560ab3342f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:14.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.254+0000 7f3282a00640 1 --2- 172.21.15.86:0/1105489909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ab33243c0 0x560ab3342f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.254+0000 7f3282a00640 1 -- 172.21.15.86:0/1105489909 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ab3344aa0 con 0x560ab33225d0 2024-04-20T21:20:14.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.254+0000 7f3282a00640 1 --2- 172.21.15.86:0/1105489909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ab33225d0 0x560ab33c4130 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3274089b50 tx=0x7f327c009e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1347232ef9918e03 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:14.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.254+0000 7f3263e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:20:14.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.255+0000 7f3263e00640 1 -- 172.21.15.86:0/1105489909 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3274076030 con 0x560ab33225d0 2024-04-20T21:20:14.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.255+0000 7f3263e00640 1 -- 172.21.15.86:0/1105489909 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f3274012e00 con 0x560ab33225d0 2024-04-20T21:20:14.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.255+0000 7f328ca01740 1 -- 172.21.15.86:0/1105489909 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3274089c10 con 0x560ab33225d0 2024-04-20T21:20:14.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.255+0000 7f3263e00640 1 -- 172.21.15.86:0/1105489909 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f327407f050 con 0x560ab33225d0 2024-04-20T21:20:14.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.255+0000 7f328ca01740 10 client.5886 fetch_fsmap learned FSMap version 24 2024-04-20T21:20:14.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.255+0000 7f328ca01740 1 -- 172.21.15.86:0/1105489909 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x560ab335c230 con 0x560ab33225d0 2024-04-20T21:20:14.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.255+0000 7f3263e00640 1 -- 172.21.15.86:0/1105489909 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3274089c10 con 0x560ab33225d0 2024-04-20T21:20:14.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.255+0000 7f3263e00640 1 -- 172.21.15.86:0/1105489909 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 24) ==== 3495+0+0 (secure 0 0 0) 0x7f3274082020 con 0x560ab33225d0 2024-04-20T21:20:14.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.255+0000 7f328ca01740 10 client.5886 fetch_fsmap finished waiting for FSMap version 24 2024-04-20T21:20:14.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.255+0000 7f328ca01740 10 client.5886 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-20T21:20:14.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.255+0000 7f328ca01740 20 client.5886 populate_metadata read hostname 'smithi086' 2024-04-20T21:20:14.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.255+0000 7f328ca01740 1 --2- 172.21.15.86:0/1105489909 >> [v2:172.21.15.86:6836/712628984,v1:172.21.15.86:6837/712628984] conn(0x560ab3343470 0x560ab330ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.255+0000 7f328ca01740 4 client.5886 mds_command: new command op to 5691 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-04-20T21:20:14.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.255+0000 7f328ca01740 1 -- 172.21.15.86:0/1105489909 --> [v2:172.21.15.86:6836/712628984,v1:172.21.15.86:6837/712628984] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x560ab32695b0 con 0x560ab3343470 2024-04-20T21:20:14.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.256+0000 7f3282000640 1 --2- 172.21.15.86:0/1105489909 >> [v2:172.21.15.86:6836/712628984,v1:172.21.15.86:6837/712628984] conn(0x560ab3343470 0x560ab330ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:14.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.256+0000 7f3282000640 1 --2- 172.21.15.86:0/1105489909 >> [v2:172.21.15.86:6836/712628984,v1:172.21.15.86:6837/712628984] conn(0x560ab3343470 0x560ab330ac70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db1bfeb9a9744102 server_cookie=8d05cc8b3b5a9d60 in_seq=0 out_seq=0 2024-04-20T21:20:14.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.256+0000 7f3263e00640 10 client.5886 ms_handle_connect on v2:172.21.15.86:6836/712628984 2024-04-20T21:20:14.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.257+0000 7f3263e00640 1 -- 172.21.15.86:0/1105489909 <== mds.0 v2:172.21.15.86:6836/712628984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560ab32695b0 con 0x560ab3343470 2024-04-20T21:20:14.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.257+0000 7f3263e00640 10 client.5886 handle_command_reply: tid=0 2024-04-20T21:20:14.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.257+0000 7f328ca01740 1 client.5886 shutdown 2024-04-20T21:20:14.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.257+0000 7f328ca01740 1 -- 172.21.15.86:0/1105489909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ab33225d0 msgr2=0x560ab33c4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:14.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.257+0000 7f328ca01740 1 --2- 172.21.15.86:0/1105489909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ab33225d0 0x560ab33c4130 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3274089b50 tx=0x7f327c009e10 comp rx=0 tx=0).stop 2024-04-20T21:20:14.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.257+0000 7f328ca01740 1 -- 172.21.15.86:0/1105489909 shutdown_connections 2024-04-20T21:20:14.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.257+0000 7f328ca01740 1 --2- 172.21.15.86:0/1105489909 >> [v2:172.21.15.86:6836/712628984,v1:172.21.15.86:6837/712628984] conn(0x560ab3343470 0x560ab330ac70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.257+0000 7f328ca01740 1 --2- 172.21.15.86:0/1105489909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ab33243c0 0x560ab3342f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.257+0000 7f328ca01740 1 --2- 172.21.15.86:0/1105489909 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560ab3323fe0 0x560ab33bad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.257+0000 7f328ca01740 1 --2- 172.21.15.86:0/1105489909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ab33225d0 0x560ab33c4130 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.257+0000 7f3263e00640 0 client.5886 ms_handle_reset on v2:172.21.15.86:6836/712628984 2024-04-20T21:20:14.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.257+0000 7f328ca01740 1 -- 172.21.15.86:0/1105489909 >> 172.21.15.86:0/1105489909 conn(0x560ab32f74c0 msgr2=0x560ab331e330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:14.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.258+0000 7f328ca01740 1 -- 172.21.15.86:0/1105489909 shutdown_connections 2024-04-20T21:20:14.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.258+0000 7f328ca01740 1 -- 172.21.15.86:0/1105489909 wait complete. 2024-04-20T21:20:14.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.258+0000 7f328ca01740 20 client.5886 trim_cache size 0 max 16384 2024-04-20T21:20:14.261 INFO:tasks.ceph.mds.a.smithi086.stderr:2024-04-20T21:20:14.258+0000 7f1e0ac00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={5829=0-4194304@1} caps={5829=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5829 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x55b78368cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-20T21:20:14.261 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:20:14.261 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-20T21:20:14.261 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "2c66b459-b923-4c8e-9bb7-8e5706f27b8f", 2024-04-20T21:20:14.261 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-20T21:20:14.261 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:20:14.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.258+0000 7f3289c00640 1 -- 172.21.15.86:0/1032989186 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f326406c8f0 msgr2=0x7f326406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:14.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.258+0000 7f3289c00640 1 --2- 172.21.15.86:0/1032989186 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f326406c8f0 0x7f326406edb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3270036000 tx=0x7f3270030000 comp rx=0 tx=0).stop 2024-04-20T21:20:14.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.258+0000 7f3289c00640 1 -- 172.21.15.86:0/1032989186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f328415e840 msgr2=0x7f328414dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:14.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.258+0000 7f3289c00640 1 --2- 172.21.15.86:0/1032989186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f328415e840 0x7f328414dc80 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f327400c460 tx=0x7f327400c920 comp rx=0 tx=0).stop 2024-04-20T21:20:14.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.259+0000 7f3289c00640 1 -- 172.21.15.86:0/1032989186 shutdown_connections 2024-04-20T21:20:14.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.259+0000 7f3289c00640 1 --2- 172.21.15.86:0/1032989186 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f326406c8f0 0x7f326406edb0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.259+0000 7f3289c00640 1 --2- 172.21.15.86:0/1032989186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3284163080 0x7f3284161760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.259+0000 7f3289c00640 1 --2- 172.21.15.86:0/1032989186 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f328415ec20 0x7f3284161220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.259+0000 7f3289c00640 1 --2- 172.21.15.86:0/1032989186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f328415e840 0x7f328414dc80 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.259+0000 7f3289c00640 1 -- 172.21.15.86:0/1032989186 >> 172.21.15.86:0/1032989186 conn(0x7f328409da40 msgr2=0x7f32840c8b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:14.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.259+0000 7f3289c00640 1 -- 172.21.15.86:0/1032989186 shutdown_connections 2024-04-20T21:20:14.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.259+0000 7f3289c00640 1 -- 172.21.15.86:0/1032989186 wait complete. 2024-04-20T21:20:14.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:20:14.276 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-20T21:20:14.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.457+0000 7f7c3bc00640 1 Processor -- start 2024-04-20T21:20:14.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.458+0000 7f7c3bc00640 1 -- start start 2024-04-20T21:20:14.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.459+0000 7f7c3bc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c3406bbf0 0x7f7c3406e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.459+0000 7f7c3bc00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c3415cd40 con 0x7f7c3406bfd0 2024-04-20T21:20:14.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.459+0000 7f7c3bc00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c3415d000 con 0x7f7c34146210 2024-04-20T21:20:14.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.459+0000 7f7c3bc00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7c3415d2e0 con 0x7f7c3406bbf0 2024-04-20T21:20:14.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.460+0000 7f7c2be00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f7c3406bfd0 0x7f7c34145b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39708/0 (socket says 172.21.15.86:39708) 2024-04-20T21:20:14.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.460+0000 7f7c2be00640 1 -- 172.21.15.86:0/2949019048 learned_addr learned my addr 172.21.15.86:0/2949019048 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:20:14.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.460+0000 7f7c39200640 1 --2- 172.21.15.86:0/2949019048 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c3406bbf0 0x7f7c3406e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:14.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.460+0000 7f7c33e00640 1 -- 172.21.15.86:0/2949019048 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1879117733 0 0) 0x7f7c3415cd40 con 0x7f7c3406bfd0 2024-04-20T21:20:14.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.460+0000 7f7c33e00640 1 -- 172.21.15.86:0/2949019048 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c18003520 con 0x7f7c3406bfd0 2024-04-20T21:20:14.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.461+0000 7f7c33e00640 1 -- 172.21.15.86:0/2949019048 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 699482571 0 0) 0x7f7c18003520 con 0x7f7c3406bfd0 2024-04-20T21:20:14.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.461+0000 7f7c33e00640 1 -- 172.21.15.86:0/2949019048 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c3406bbf0 msgr2=0x7f7c3406e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:14.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.461+0000 7f7c33e00640 1 --2- 172.21.15.86:0/2949019048 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c3406bbf0 0x7f7c3406e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.461+0000 7f7c33e00640 1 -- 172.21.15.86:0/2949019048 >> v1:172.21.15.144:6789/0 conn(0x7f7c34146210 legacy=0x7f7c340cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:14.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.461+0000 7f7c33e00640 1 -- 172.21.15.86:0/2949019048 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c3415d5c0 con 0x7f7c3406bfd0 2024-04-20T21:20:14.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.461+0000 7f7c33e00640 1 -- 172.21.15.86:0/2949019048 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f7c1c001670 con 0x7f7c3406bfd0 2024-04-20T21:20:14.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.461+0000 7f7c33e00640 1 -- 172.21.15.86:0/2949019048 >> v1:172.21.15.86:6789/0 conn(0x7f7c3406bfd0 legacy=0x7f7c34145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:14.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.461+0000 7f7c33e00640 1 --2- 172.21.15.86:0/2949019048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c180039e0 0x7f7c18003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.461+0000 7f7c33e00640 1 --2- 172.21.15.86:0/2949019048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c18004800 0x7f7c18004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.461+0000 7f7c33e00640 1 --2- 172.21.15.86:0/2949019048 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c3406bbf0 0x7f7c180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.461+0000 7f7c33e00640 1 -- 172.21.15.86:0/2949019048 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7c3415d2e0 con 0x7f7c18004800 2024-04-20T21:20:14.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.461+0000 7f7c33e00640 1 -- 172.21.15.86:0/2949019048 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1800ba70 con 0x7f7c180039e0 2024-04-20T21:20:14.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.461+0000 7f7c33e00640 1 -- 172.21.15.86:0/2949019048 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7c1800bc70 con 0x7f7c3406bbf0 2024-04-20T21:20:14.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.461+0000 7f7c2be00640 1 --2- 172.21.15.86:0/2949019048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c18004800 0x7f7c18004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:14.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.461+0000 7f7c2be00640 1 -- 172.21.15.86:0/2949019048 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c3406bbf0 msgr2=0x7f7c180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:14.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.461+0000 7f7c2be00640 1 --2- 172.21.15.86:0/2949019048 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c3406bbf0 0x7f7c180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.461+0000 7f7c2be00640 1 -- 172.21.15.86:0/2949019048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c180039e0 msgr2=0x7f7c18003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:14.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.461+0000 7f7c2be00640 1 --2- 172.21.15.86:0/2949019048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c180039e0 0x7f7c18003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.462+0000 7f7c2be00640 1 -- 172.21.15.86:0/2949019048 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c18003520 con 0x7f7c18004800 2024-04-20T21:20:14.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.463+0000 7f7c2be00640 1 --2- 172.21.15.86:0/2949019048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c18004800 0x7f7c18004c80 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c001eb0 tx=0x7f7c18005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a46293cdf647c03 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:14.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.463+0000 7f7c33e00640 1 -- 172.21.15.86:0/2949019048 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c007ce0 con 0x7f7c18004800 2024-04-20T21:20:14.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.463+0000 7f7c33e00640 1 -- 172.21.15.86:0/2949019048 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c1c030070 con 0x7f7c18004800 2024-04-20T21:20:14.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.463+0000 7f7c33e00640 1 -- 172.21.15.86:0/2949019048 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c005d10 con 0x7f7c18004800 2024-04-20T21:20:14.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.463+0000 7f7c3bc00640 1 -- 172.21.15.86:0/2949019048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c18004800 msgr2=0x7f7c18004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:14.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.463+0000 7f7c3bc00640 1 --2- 172.21.15.86:0/2949019048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c18004800 0x7f7c18004c80 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c001eb0 tx=0x7f7c18005830 comp rx=0 tx=0).stop 2024-04-20T21:20:14.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.463+0000 7f7c39c00640 1 -- 172.21.15.86:0/2949019048 reap_dead start 2024-04-20T21:20:14.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.464+0000 7f7c3bc00640 1 -- 172.21.15.86:0/2949019048 shutdown_connections 2024-04-20T21:20:14.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.464+0000 7f7c3bc00640 1 -- 172.21.15.86:0/2949019048 >> 172.21.15.86:0/2949019048 conn(0x7f7c340b2110 msgr2=0x7f7c340b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:14.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.464+0000 7f7c3bc00640 1 -- 172.21.15.86:0/2949019048 shutdown_connections 2024-04-20T21:20:14.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.464+0000 7f7c3bc00640 1 -- 172.21.15.86:0/2949019048 wait complete. 2024-04-20T21:20:14.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.465+0000 7f7c3bc00640 1 Processor -- start 2024-04-20T21:20:14.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.466+0000 7f7c3bc00640 1 -- start start 2024-04-20T21:20:14.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.467+0000 7f7c3bc00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c18004800 0x7f7c3413bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.467+0000 7f7c3bc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c3413e860 0x7f7c3415b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.467+0000 7f7c3bc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c341f2770 0x7f7c34143fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.467+0000 7f7c3bc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7c340cddd0 con 0x7f7c341f2770 2024-04-20T21:20:14.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.467+0000 7f7c3bc00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7c340cdc50 con 0x7f7c18004800 2024-04-20T21:20:14.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.467+0000 7f7c3bc00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7c340cdf50 con 0x7f7c3413e860 2024-04-20T21:20:14.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.467+0000 7f7c39c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c341f2770 0x7f7c34143fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:14.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.467+0000 7f7c39c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c341f2770 0x7f7c34143fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45872/0 (socket says 172.21.15.86:45872) 2024-04-20T21:20:14.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.467+0000 7f7c39c00640 1 -- 172.21.15.86:0/1361704140 learned_addr learned my addr 172.21.15.86:0/1361704140 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:14.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.467+0000 7f7c39200640 1 --2- 172.21.15.86:0/1361704140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c18004800 0x7f7c3413bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:14.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.467+0000 7f7c2be00640 1 --2- 172.21.15.86:0/1361704140 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c3413e860 0x7f7c3415b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:14.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.467+0000 7f7c39c00640 1 -- 172.21.15.86:0/1361704140 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c3413e860 msgr2=0x7f7c3415b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:14.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.468+0000 7f7c39c00640 1 --2- 172.21.15.86:0/1361704140 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c3413e860 0x7f7c3415b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.468+0000 7f7c39c00640 1 -- 172.21.15.86:0/1361704140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c18004800 msgr2=0x7f7c3413bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:14.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.468+0000 7f7c39c00640 1 --2- 172.21.15.86:0/1361704140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c18004800 0x7f7c3413bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.468+0000 7f7c39c00640 1 -- 172.21.15.86:0/1361704140 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c3413c2a0 con 0x7f7c341f2770 2024-04-20T21:20:14.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.468+0000 7f7c39c00640 1 --2- 172.21.15.86:0/1361704140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c341f2770 0x7f7c34143fc0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f7c34073250 tx=0x7f7c2c00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:14.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.468+0000 7f7c31600640 1 -- 172.21.15.86:0/1361704140 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c017070 con 0x7f7c341f2770 2024-04-20T21:20:14.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.468+0000 7f7c31600640 1 -- 172.21.15.86:0/1361704140 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c2c005d40 con 0x7f7c341f2770 2024-04-20T21:20:14.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.468+0000 7f7c3bc00640 1 -- 172.21.15.86:0/1361704140 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c3413c540 con 0x7f7c341f2770 2024-04-20T21:20:14.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.468+0000 7f7c31600640 1 -- 172.21.15.86:0/1361704140 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c00a850 con 0x7f7c341f2770 2024-04-20T21:20:14.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.468+0000 7f7c3bc00640 1 -- 172.21.15.86:0/1361704140 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c340c6ad0 con 0x7f7c341f2770 2024-04-20T21:20:14.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.470+0000 7f7c31600640 1 -- 172.21.15.86:0/1361704140 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7c2c01a070 con 0x7f7c341f2770 2024-04-20T21:20:14.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.471+0000 7f7c31600640 1 --2- 172.21.15.86:0/1361704140 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7c1006c920 0x7f7c1006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.471+0000 7f7c31600640 1 -- 172.21.15.86:0/1361704140 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f7c2c08ae10 con 0x7f7c341f2770 2024-04-20T21:20:14.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.472+0000 7f7c39200640 1 --2- 172.21.15.86:0/1361704140 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7c1006c920 0x7f7c1006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:14.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.472+0000 7f7c3e991740 1 Processor -- start 2024-04-20T21:20:14.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.472+0000 7f7c3e991740 1 -- start start 2024-04-20T21:20:14.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.472+0000 7f7c39200640 1 --2- 172.21.15.86:0/1361704140 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7c1006c920 0x7f7c1006ede0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7c240047b0 tx=0x7f7c24008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:14.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.473+0000 7f7c3e991740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e66c8081f0 0x55e66c8085d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.473+0000 7f7c3e991740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e66c7d8210 0x55e66c805460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.473+0000 7f7c3e991740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e66c7dbd70 0x55e66c8059a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.473+0000 7f7c3e991740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e66c744150 con 0x55e66c7d8210 2024-04-20T21:20:14.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.473+0000 7f7c3e991740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e66c6ea370 con 0x55e66c8081f0 2024-04-20T21:20:14.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.473+0000 7f7c3e991740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55e66c740110 con 0x55e66c7dbd70 2024-04-20T21:20:14.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.473+0000 7f7c2be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e66c7d8210 0x55e66c805460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:14.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.473+0000 7f7c2be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e66c7d8210 0x55e66c805460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45880/0 (socket says 172.21.15.86:45880) 2024-04-20T21:20:14.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.473+0000 7f7c2be00640 1 -- 172.21.15.86:0/3159962746 learned_addr learned my addr 172.21.15.86:0/3159962746 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:14.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.474+0000 7f7c39200640 1 --2- 172.21.15.86:0/3159962746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e66c8081f0 0x55e66c8085d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:14.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.474+0000 7f7c39c00640 1 --2- 172.21.15.86:0/3159962746 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e66c7dbd70 0x55e66c8059a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:14.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.474+0000 7f7c2be00640 1 -- 172.21.15.86:0/3159962746 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e66c7dbd70 msgr2=0x55e66c8059a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:14.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.474+0000 7f7c2be00640 1 --2- 172.21.15.86:0/3159962746 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e66c7dbd70 0x55e66c8059a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.474+0000 7f7c2be00640 1 -- 172.21.15.86:0/3159962746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e66c8081f0 msgr2=0x55e66c8085d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:14.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.474+0000 7f7c2be00640 1 --2- 172.21.15.86:0/3159962746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e66c8081f0 0x55e66c8085d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.474+0000 7f7c2be00640 1 -- 172.21.15.86:0/3159962746 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e66c5207d0 con 0x55e66c7d8210 2024-04-20T21:20:14.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.474+0000 7f7c39c00640 1 --2- 172.21.15.86:0/3159962746 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e66c7dbd70 0x55e66c8059a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:20:14.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.474+0000 7f7c2be00640 1 --2- 172.21.15.86:0/3159962746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e66c7d8210 0x55e66c805460 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c001eb0 tx=0x7f7c1c02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9bae804b4c520af server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:14.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.474+0000 7f7c29600640 1 -- 172.21.15.86:0/3159962746 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c02c830 con 0x55e66c7d8210 2024-04-20T21:20:14.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.474+0000 7f7c29600640 1 -- 172.21.15.86:0/3159962746 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c1c030040 con 0x55e66c7d8210 2024-04-20T21:20:14.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.474+0000 7f7c29600640 1 -- 172.21.15.86:0/3159962746 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c02cc10 con 0x55e66c7d8210 2024-04-20T21:20:14.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.475+0000 7f7c3e991740 1 -- 172.21.15.86:0/3159962746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e66c7d8210 msgr2=0x55e66c805460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:14.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.475+0000 7f7c3e991740 1 --2- 172.21.15.86:0/3159962746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e66c7d8210 0x55e66c805460 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c001eb0 tx=0x7f7c1c02e000 comp rx=0 tx=0).stop 2024-04-20T21:20:14.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.475+0000 7f7c3e991740 1 -- 172.21.15.86:0/3159962746 shutdown_connections 2024-04-20T21:20:14.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.475+0000 7f7c3e991740 1 --2- 172.21.15.86:0/3159962746 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e66c7dbd70 0x55e66c8059a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.475+0000 7f7c3e991740 1 --2- 172.21.15.86:0/3159962746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e66c7d8210 0x55e66c805460 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.475+0000 7f7c3e991740 1 --2- 172.21.15.86:0/3159962746 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e66c8081f0 0x55e66c8085d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.475+0000 7f7c3e991740 1 -- 172.21.15.86:0/3159962746 >> 172.21.15.86:0/3159962746 conn(0x55e66c7da480 msgr2=0x55e66c823000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:14.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.475+0000 7f7c3e991740 1 -- 172.21.15.86:0/3159962746 shutdown_connections 2024-04-20T21:20:14.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.475+0000 7f7c3e991740 1 -- 172.21.15.86:0/3159962746 wait complete. 2024-04-20T21:20:14.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.476+0000 7f7c3e991740 1 -- start start 2024-04-20T21:20:14.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.476+0000 7f7c3e991740 1 Processor -- start 2024-04-20T21:20:14.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.477+0000 7f7c3e991740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e66c7dbd70 0x55e66c803900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.477+0000 7f7c3e991740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e66c8081f0 0x55e66c837a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.477+0000 7f7c3e991740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e66c804570 0x55e66c828f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.477+0000 7f7c3e991740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e66c7dcf20 con 0x55e66c7dbd70 2024-04-20T21:20:14.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.477+0000 7f7c3e991740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e66c6fd740 con 0x55e66c804570 2024-04-20T21:20:14.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.477+0000 7f7c3e991740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55e66c7dc310 con 0x55e66c8081f0 2024-04-20T21:20:14.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.477+0000 7f7c3e991740 10 client.0 _ll_register_callbacks cb 0x55e66c6f8b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:20:14.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.478+0000 7f7c39200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e66c7dbd70 0x55e66c803900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:14.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.478+0000 7f7c39200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e66c7dbd70 0x55e66c803900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45888/0 (socket says 172.21.15.86:45888) 2024-04-20T21:20:14.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.478+0000 7f7c39200640 1 -- 172.21.15.86:0/113925868 learned_addr learned my addr 172.21.15.86:0/113925868 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:14.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.478+0000 7f7c2be00640 1 --2- 172.21.15.86:0/113925868 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e66c8081f0 0x55e66c837a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:14.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.478+0000 7f7c39200640 1 -- 172.21.15.86:0/113925868 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e66c8081f0 msgr2=0x55e66c837a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:14.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.478+0000 7f7c39200640 1 --2- 172.21.15.86:0/113925868 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e66c8081f0 0x55e66c837a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.478+0000 7f7c39200640 1 -- 172.21.15.86:0/113925868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e66c804570 msgr2=0x55e66c828f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:14.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.478+0000 7f7c39200640 1 --2- 172.21.15.86:0/113925868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e66c804570 0x55e66c828f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.478+0000 7f7c39200640 1 -- 172.21.15.86:0/113925868 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e66c4563f0 con 0x55e66c7dbd70 2024-04-20T21:20:14.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.478+0000 7f7c39200640 1 --2- 172.21.15.86:0/113925868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e66c7dbd70 0x55e66c803900 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f7c24015720 tx=0x7f7c24009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3228503fa0b8679a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:14.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.478+0000 7f7c22000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:20:14.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.479+0000 7f7c22000640 1 -- 172.21.15.86:0/113925868 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c24026070 con 0x55e66c7dbd70 2024-04-20T21:20:14.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.479+0000 7f7c22000640 1 -- 172.21.15.86:0/113925868 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f7c24014730 con 0x55e66c7dbd70 2024-04-20T21:20:14.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.479+0000 7f7c3e991740 1 -- 172.21.15.86:0/113925868 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c1c030040 con 0x55e66c7dbd70 2024-04-20T21:20:14.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.479+0000 7f7c22000640 1 -- 172.21.15.86:0/113925868 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7c24031050 con 0x55e66c7dbd70 2024-04-20T21:20:14.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.479+0000 7f7c3e991740 10 client.5898 fetch_fsmap learned FSMap version 24 2024-04-20T21:20:14.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.479+0000 7f7c3e991740 1 -- 172.21.15.86:0/113925868 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x55e66c4160f0 con 0x55e66c7dbd70 2024-04-20T21:20:14.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.479+0000 7f7c22000640 1 -- 172.21.15.86:0/113925868 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c1c030040 con 0x55e66c7dbd70 2024-04-20T21:20:14.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.479+0000 7f7c22000640 1 -- 172.21.15.86:0/113925868 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 24) ==== 3495+0+0 (secure 0 0 0) 0x7f7c24034020 con 0x55e66c7dbd70 2024-04-20T21:20:14.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.479+0000 7f7c3e991740 10 client.5898 fetch_fsmap finished waiting for FSMap version 24 2024-04-20T21:20:14.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.479+0000 7f7c3e991740 10 client.5898 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-20T21:20:14.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.479+0000 7f7c3e991740 20 client.5898 populate_metadata read hostname 'smithi086' 2024-04-20T21:20:14.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.479+0000 7f7c3e991740 1 --2- 172.21.15.86:0/113925868 >> [v2:172.21.15.86:6836/712628984,v1:172.21.15.86:6837/712628984] conn(0x55e66c80a450 0x55e66c80a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.479+0000 7f7c3e991740 4 client.5898 mds_command: new command op to 5691 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:20:14.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.479+0000 7f7c3e991740 1 -- 172.21.15.86:0/113925868 --> [v2:172.21.15.86:6836/712628984,v1:172.21.15.86:6837/712628984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e66c751b70 con 0x55e66c80a450 2024-04-20T21:20:14.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.480+0000 7f7c2be00640 1 --2- 172.21.15.86:0/113925868 >> [v2:172.21.15.86:6836/712628984,v1:172.21.15.86:6837/712628984] conn(0x55e66c80a450 0x55e66c80a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:14.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.480+0000 7f7c2be00640 1 --2- 172.21.15.86:0/113925868 >> [v2:172.21.15.86:6836/712628984,v1:172.21.15.86:6837/712628984] conn(0x55e66c80a450 0x55e66c80a8d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4316fb57a743b935 server_cookie=3c658ef16d8479c in_seq=0 out_seq=0 2024-04-20T21:20:14.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.480+0000 7f7c22000640 10 client.5898 ms_handle_connect on v2:172.21.15.86:6836/712628984 2024-04-20T21:20:14.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.481+0000 7f7c22000640 1 -- 172.21.15.86:0/113925868 <== mds.0 v2:172.21.15.86:6836/712628984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e66c751b70 con 0x55e66c80a450 2024-04-20T21:20:14.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.481+0000 7f7c22000640 10 client.5898 handle_command_reply: tid=0 2024-04-20T21:20:14.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.481+0000 7f7c3e991740 1 client.5898 shutdown 2024-04-20T21:20:14.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.481+0000 7f7c3e991740 1 -- 172.21.15.86:0/113925868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e66c7dbd70 msgr2=0x55e66c803900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:14.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.481+0000 7f7c3e991740 1 --2- 172.21.15.86:0/113925868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e66c7dbd70 0x55e66c803900 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f7c24015720 tx=0x7f7c24009190 comp rx=0 tx=0).stop 2024-04-20T21:20:14.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.481+0000 7f7c3e991740 1 -- 172.21.15.86:0/113925868 shutdown_connections 2024-04-20T21:20:14.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.481+0000 7f7c3e991740 1 --2- 172.21.15.86:0/113925868 >> [v2:172.21.15.86:6836/712628984,v1:172.21.15.86:6837/712628984] conn(0x55e66c80a450 0x55e66c80a8d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.481+0000 7f7c3e991740 1 --2- 172.21.15.86:0/113925868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e66c804570 0x55e66c828f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.481+0000 7f7c3e991740 1 --2- 172.21.15.86:0/113925868 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e66c8081f0 0x55e66c837a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.481+0000 7f7c3e991740 1 --2- 172.21.15.86:0/113925868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e66c7dbd70 0x55e66c803900 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.481+0000 7f7c22000640 0 client.5898 ms_handle_reset on v2:172.21.15.86:6836/712628984 2024-04-20T21:20:14.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.481+0000 7f7c3e991740 1 -- 172.21.15.86:0/113925868 >> 172.21.15.86:0/113925868 conn(0x55e66c7d8210 msgr2=0x55e66c826510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:14.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.481+0000 7f7c3e991740 1 -- 172.21.15.86:0/113925868 shutdown_connections 2024-04-20T21:20:14.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.481+0000 7f7c3e991740 1 -- 172.21.15.86:0/113925868 wait complete. 2024-04-20T21:20:14.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.482+0000 7f7c3e991740 20 client.5898 trim_cache size 0 max 16384 2024-04-20T21:20:14.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.500+0000 7f7c3e991740 1 Processor -- start 2024-04-20T21:20:14.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.500+0000 7f7c3e991740 1 -- start start 2024-04-20T21:20:14.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.501+0000 7f7c3e991740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e66c7dbd70 0x55e66c84b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.501+0000 7f7c3e991740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e66c804570 0x55e66c8370d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.501+0000 7f7c3e991740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e66c8081f0 0x55e66c837610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.501+0000 7f7c3e991740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7c341fd3d0 con 0x55e66c8081f0 2024-04-20T21:20:14.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.501+0000 7f7c3e991740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7c340b4d40 con 0x55e66c804570 2024-04-20T21:20:14.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.501+0000 7f7c3e991740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7c340c61a0 con 0x55e66c7dbd70 2024-04-20T21:20:14.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.501+0000 7f7c39c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e66c8081f0 0x55e66c837610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:14.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.502+0000 7f7c39c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e66c8081f0 0x55e66c837610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45892/0 (socket says 172.21.15.86:45892) 2024-04-20T21:20:14.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.502+0000 7f7c39c00640 1 -- 172.21.15.86:0/2949825720 learned_addr learned my addr 172.21.15.86:0/2949825720 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:14.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.502+0000 7f7c2be00640 1 --2- 172.21.15.86:0/2949825720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e66c804570 0x55e66c8370d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:14.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.502+0000 7f7c39200640 1 --2- 172.21.15.86:0/2949825720 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e66c7dbd70 0x55e66c84b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:14.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.502+0000 7f7c39c00640 1 -- 172.21.15.86:0/2949825720 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e66c7dbd70 msgr2=0x55e66c84b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:14.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.502+0000 7f7c39c00640 1 --2- 172.21.15.86:0/2949825720 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e66c7dbd70 0x55e66c84b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.502+0000 7f7c39c00640 1 -- 172.21.15.86:0/2949825720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e66c804570 msgr2=0x55e66c8370d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:14.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.502+0000 7f7c39c00640 1 --2- 172.21.15.86:0/2949825720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e66c804570 0x55e66c8370d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.502+0000 7f7c39c00640 1 -- 172.21.15.86:0/2949825720 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e66c822dd0 con 0x55e66c8081f0 2024-04-20T21:20:14.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.502+0000 7f7c39c00640 1 --2- 172.21.15.86:0/2949825720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e66c8081f0 0x55e66c837610 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c061150 tx=0x7f7c2c0591a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7921aa4b3de1a0d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:14.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.502+0000 7f7c28c00640 1 -- 172.21.15.86:0/2949825720 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c05ce50 con 0x55e66c8081f0 2024-04-20T21:20:14.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.503+0000 7f7c28c00640 1 -- 172.21.15.86:0/2949825720 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c2c061680 con 0x55e66c8081f0 2024-04-20T21:20:14.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.503+0000 7f7c28c00640 1 -- 172.21.15.86:0/2949825720 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c055110 con 0x55e66c8081f0 2024-04-20T21:20:14.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.503+0000 7f7c3e991740 1 -- 172.21.15.86:0/2949825720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e66c8081f0 msgr2=0x55e66c837610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:14.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.503+0000 7f7c3e991740 1 --2- 172.21.15.86:0/2949825720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e66c8081f0 0x55e66c837610 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c061150 tx=0x7f7c2c0591a0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.503+0000 7f7c3e991740 1 -- 172.21.15.86:0/2949825720 shutdown_connections 2024-04-20T21:20:14.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.503+0000 7f7c3e991740 1 --2- 172.21.15.86:0/2949825720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e66c8081f0 0x55e66c837610 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.503+0000 7f7c3e991740 1 --2- 172.21.15.86:0/2949825720 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e66c804570 0x55e66c8370d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.503+0000 7f7c3e991740 1 --2- 172.21.15.86:0/2949825720 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e66c7dbd70 0x55e66c84b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.503+0000 7f7c3e991740 1 -- 172.21.15.86:0/2949825720 >> 172.21.15.86:0/2949825720 conn(0x55e66c7d8210 msgr2=0x55e66c803710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:14.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.503+0000 7f7c3e991740 1 -- 172.21.15.86:0/2949825720 shutdown_connections 2024-04-20T21:20:14.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.503+0000 7f7c3e991740 1 -- 172.21.15.86:0/2949825720 wait complete. 2024-04-20T21:20:14.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.504+0000 7f7c3e991740 1 -- start start 2024-04-20T21:20:14.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.504+0000 7f7c3e991740 1 Processor -- start 2024-04-20T21:20:14.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.505+0000 7f7c3e991740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e66c7dbd70 0x55e66c8a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.505+0000 7f7c3e991740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e66c804570 0x55e66c825410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.505+0000 7f7c3e991740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e66c8081f0 0x55e66c837090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.505+0000 7f7c3e991740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e66c829100 con 0x55e66c804570 2024-04-20T21:20:14.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.505+0000 7f7c3e991740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7c340b4720 con 0x55e66c7dbd70 2024-04-20T21:20:14.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.505+0000 7f7c3e991740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55e66c802a70 con 0x55e66c8081f0 2024-04-20T21:20:14.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.505+0000 7f7c3e991740 10 client.0 _ll_register_callbacks cb 0x7f7c24011860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:20:14.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.505+0000 7f7c2be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e66c804570 0x55e66c825410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:14.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.505+0000 7f7c2be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e66c804570 0x55e66c825410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45900/0 (socket says 172.21.15.86:45900) 2024-04-20T21:20:14.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.505+0000 7f7c2be00640 1 -- 172.21.15.86:0/641144478 learned_addr learned my addr 172.21.15.86:0/641144478 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:14.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.506+0000 7f7c2be00640 1 -- 172.21.15.86:0/641144478 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e66c8081f0 msgr2=0x55e66c837090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:14.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.506+0000 7f7c2be00640 1 --2- 172.21.15.86:0/641144478 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e66c8081f0 0x55e66c837090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.506+0000 7f7c2be00640 1 -- 172.21.15.86:0/641144478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e66c7dbd70 msgr2=0x55e66c8a4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:14.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.506+0000 7f7c2be00640 1 --2- 172.21.15.86:0/641144478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e66c7dbd70 0x55e66c8a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.506+0000 7f7c2be00640 1 -- 172.21.15.86:0/641144478 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e66c841bc0 con 0x55e66c804570 2024-04-20T21:20:14.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.506+0000 7f7c2be00640 1 --2- 172.21.15.86:0/641144478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e66c804570 0x55e66c825410 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c001eb0 tx=0x7f7c3415d1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c833a5dc4d9d87 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:14.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.506+0000 7f7c22000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:20:14.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.506+0000 7f7c22000640 1 -- 172.21.15.86:0/641144478 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c02c830 con 0x55e66c804570 2024-04-20T21:20:14.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.506+0000 7f7c22000640 1 -- 172.21.15.86:0/641144478 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f7c1c02c9d0 con 0x55e66c804570 2024-04-20T21:20:14.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.506+0000 7f7c3e991740 1 -- 172.21.15.86:0/641144478 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c2c061680 con 0x55e66c804570 2024-04-20T21:20:14.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.507+0000 7f7c22000640 1 -- 172.21.15.86:0/641144478 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7c1c055050 con 0x55e66c804570 2024-04-20T21:20:14.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.507+0000 7f7c3e991740 10 client.5904 fetch_fsmap learned FSMap version 24 2024-04-20T21:20:14.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.507+0000 7f7c3e991740 1 -- 172.21.15.86:0/641144478 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x55e66c7db8c0 con 0x55e66c804570 2024-04-20T21:20:14.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.507+0000 7f7c22000640 1 -- 172.21.15.86:0/641144478 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c2c061680 con 0x55e66c804570 2024-04-20T21:20:14.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.507+0000 7f7c22000640 1 -- 172.21.15.86:0/641144478 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 24) ==== 3495+0+0 (secure 0 0 0) 0x7f7c1c058020 con 0x55e66c804570 2024-04-20T21:20:14.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.507+0000 7f7c3e991740 10 client.5904 fetch_fsmap finished waiting for FSMap version 24 2024-04-20T21:20:14.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.507+0000 7f7c3e991740 10 client.5904 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-20T21:20:14.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.507+0000 7f7c3e991740 20 client.5904 populate_metadata read hostname 'smithi086' 2024-04-20T21:20:14.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.507+0000 7f7c3e991740 1 --2- 172.21.15.86:0/641144478 >> [v2:172.21.15.86:6836/712628984,v1:172.21.15.86:6837/712628984] conn(0x55e66c8375d0 0x55e66c801d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:14.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.507+0000 7f7c3e991740 4 client.5904 mds_command: new command op to 5691 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:20:14.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.507+0000 7f7c3e991740 1 -- 172.21.15.86:0/641144478 --> [v2:172.21.15.86:6836/712628984,v1:172.21.15.86:6837/712628984] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e66c74b9e0 con 0x55e66c8375d0 2024-04-20T21:20:14.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.507+0000 7f7c39200640 1 --2- 172.21.15.86:0/641144478 >> [v2:172.21.15.86:6836/712628984,v1:172.21.15.86:6837/712628984] conn(0x55e66c8375d0 0x55e66c801d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:14.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.508+0000 7f7c39200640 1 --2- 172.21.15.86:0/641144478 >> [v2:172.21.15.86:6836/712628984,v1:172.21.15.86:6837/712628984] conn(0x55e66c8375d0 0x55e66c801d70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62e84237f107383d server_cookie=5cf0fa602dc92754 in_seq=0 out_seq=0 2024-04-20T21:20:14.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.508+0000 7f7c22000640 10 client.5904 ms_handle_connect on v2:172.21.15.86:6836/712628984 2024-04-20T21:20:14.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.508+0000 7f7c22000640 1 -- 172.21.15.86:0/641144478 <== mds.0 v2:172.21.15.86:6836/712628984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+260 (crc 0 0 0) 0x55e66c74b9e0 con 0x55e66c8375d0 2024-04-20T21:20:14.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.508+0000 7f7c22000640 10 client.5904 handle_command_reply: tid=0 2024-04-20T21:20:14.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.508+0000 7f7c3e991740 1 client.5904 shutdown 2024-04-20T21:20:14.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.508+0000 7f7c3e991740 1 -- 172.21.15.86:0/641144478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e66c804570 msgr2=0x55e66c825410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:14.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.508+0000 7f7c3e991740 1 --2- 172.21.15.86:0/641144478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e66c804570 0x55e66c825410 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c001eb0 tx=0x7f7c3415d1d0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.508+0000 7f7c3e991740 1 -- 172.21.15.86:0/641144478 shutdown_connections 2024-04-20T21:20:14.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.508+0000 7f7c3e991740 1 --2- 172.21.15.86:0/641144478 >> [v2:172.21.15.86:6836/712628984,v1:172.21.15.86:6837/712628984] conn(0x55e66c8375d0 0x55e66c801d70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.508+0000 7f7c3e991740 1 --2- 172.21.15.86:0/641144478 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e66c8081f0 0x55e66c837090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.508+0000 7f7c3e991740 1 --2- 172.21.15.86:0/641144478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e66c804570 0x55e66c825410 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.508+0000 7f7c3e991740 1 --2- 172.21.15.86:0/641144478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e66c7dbd70 0x55e66c8a4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.508+0000 7f7c22000640 0 client.5904 ms_handle_reset on v2:172.21.15.86:6836/712628984 2024-04-20T21:20:14.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.508+0000 7f7c3e991740 1 -- 172.21.15.86:0/641144478 >> 172.21.15.86:0/641144478 conn(0x55e66c7d8210 msgr2=0x55e66c826510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:14.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.508+0000 7f7c3e991740 1 -- 172.21.15.86:0/641144478 shutdown_connections 2024-04-20T21:20:14.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.508+0000 7f7c3e991740 1 -- 172.21.15.86:0/641144478 wait complete. 2024-04-20T21:20:14.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.509+0000 7f7c3e991740 20 client.5904 trim_cache size 0 max 16384 2024-04-20T21:20:14.514 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:20:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T21:20:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-20T21:20:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: "2c66b459-b923-4c8e-9bb7-8e5706f27b8f": { 2024-04-20T21:20:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-20T21:20:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "2c66b459-b923-4c8e-9bb7-8e5706f27b8f", 2024-04-20T21:20:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,repair" 2024-04-20T21:20:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:20:14.514 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:20:14.514 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:20:14.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.509+0000 7f7c3bc00640 1 -- 172.21.15.86:0/1361704140 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7c1006c920 msgr2=0x7f7c1006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:14.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.509+0000 7f7c3bc00640 1 --2- 172.21.15.86:0/1361704140 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7c1006c920 0x7f7c1006ede0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7c240047b0 tx=0x7f7c24008040 comp rx=0 tx=0).stop 2024-04-20T21:20:14.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.509+0000 7f7c3bc00640 1 -- 172.21.15.86:0/1361704140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c341f2770 msgr2=0x7f7c34143fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:14.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.509+0000 7f7c3bc00640 1 --2- 172.21.15.86:0/1361704140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c341f2770 0x7f7c34143fc0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f7c34073250 tx=0x7f7c2c00cb90 comp rx=0 tx=0).stop 2024-04-20T21:20:14.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.509+0000 7f7c3bc00640 1 -- 172.21.15.86:0/1361704140 shutdown_connections 2024-04-20T21:20:14.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.509+0000 7f7c3bc00640 1 --2- 172.21.15.86:0/1361704140 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7c1006c920 0x7f7c1006ede0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.509+0000 7f7c3bc00640 1 --2- 172.21.15.86:0/1361704140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c341f2770 0x7f7c34143fc0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.509+0000 7f7c3bc00640 1 --2- 172.21.15.86:0/1361704140 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c3413e860 0x7f7c3415b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.509+0000 7f7c3bc00640 1 --2- 172.21.15.86:0/1361704140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c18004800 0x7f7c3413bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:14.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.509+0000 7f7c3bc00640 1 -- 172.21.15.86:0/1361704140 >> 172.21.15.86:0/1361704140 conn(0x7f7c340b2110 msgr2=0x7f7c3415f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:14.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.509+0000 7f7c3bc00640 1 -- 172.21.15.86:0/1361704140 shutdown_connections 2024-04-20T21:20:14.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.509+0000 7f7c3bc00640 1 -- 172.21.15.86:0/1361704140 wait complete. 2024-04-20T21:20:14.526 INFO:tasks.cephfs.filesystem:scrub status for tag:2c66b459-b923-4c8e-9bb7-8e5706f27b8f - {'path': '/', 'tag': '2c66b459-b923-4c8e-9bb7-8e5706f27b8f', 'options': 'recursive,repair'} 2024-04-20T21:20:14.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:14.759+0000 7f062ea00640 1 -- 172.21.15.86:0/2326961910 <== mon.0 v2:172.21.15.86:3300/0 7 ==== log(6 entries from seq 1 at 2024-04-20T21:20:14.258479+0000) ==== 1078+0+0 (secure 0 0 0) 0x7f062808ce00 con 0x7f06380d3120 2024-04-20T21:20:14.760 INFO:teuthology.orchestra.run.smithi086.stdout: cluster: 2024-04-20T21:20:14.760 INFO:teuthology.orchestra.run.smithi086.stdout: id: 9c2dec1d-d7ae-4a83-a812-79922808e278 2024-04-20T21:20:14.760 INFO:teuthology.orchestra.run.smithi086.stdout: health: HEALTH_OK 2024-04-20T21:20:14.761 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:20:14.761 INFO:teuthology.orchestra.run.smithi086.stdout: services: 2024-04-20T21:20:14.761 INFO:teuthology.orchestra.run.smithi086.stdout: mon: 3 daemons, quorum a,b,c (age 2m) 2024-04-20T21:20:14.761 INFO:teuthology.orchestra.run.smithi086.stdout: mgr: y(active, since 2m), standbys: x 2024-04-20T21:20:14.761 INFO:teuthology.orchestra.run.smithi086.stdout: mds: 1/1 daemons up, 3 standby 2024-04-20T21:20:14.761 INFO:teuthology.orchestra.run.smithi086.stdout: osd: 8 osds: 8 up (since 2m), 8 in (since 2m) 2024-04-20T21:20:14.761 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:20:14.761 INFO:teuthology.orchestra.run.smithi086.stdout: data: 2024-04-20T21:20:14.761 INFO:teuthology.orchestra.run.smithi086.stdout: volumes: 1/1 healthy 2024-04-20T21:20:14.761 INFO:teuthology.orchestra.run.smithi086.stdout: pools: 3 pools, 97 pgs 2024-04-20T21:20:14.761 INFO:teuthology.orchestra.run.smithi086.stdout: objects: 24 objects, 579 KiB 2024-04-20T21:20:14.761 INFO:teuthology.orchestra.run.smithi086.stdout: usage: 220 MiB used, 720 GiB / 720 GiB avail 2024-04-20T21:20:14.762 INFO:teuthology.orchestra.run.smithi086.stdout: pgs: 97 active+clean 2024-04-20T21:20:14.762 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:20:14.762 INFO:teuthology.orchestra.run.smithi086.stdout: io: 2024-04-20T21:20:14.762 INFO:teuthology.orchestra.run.smithi086.stdout: client: 1.6 KiB/s wr, 0 op/s rd, 5 op/s wr 2024-04-20T21:20:14.762 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:20:14.762 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:20:14.762 INFO:teuthology.orchestra.run.smithi086.stdout:2024-04-20T21:20:14.258479+0000 mds.a [INF] scrub queued for path: / 2024-04-20T21:20:14.762 INFO:teuthology.orchestra.run.smithi086.stdout:2024-04-20T21:20:14.258483+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-04-20T21:20:14.762 INFO:teuthology.orchestra.run.smithi086.stdout:2024-04-20T21:20:14.258488+0000 mds.a [INF] scrub summary: active paths [/] 2024-04-20T21:20:14.762 INFO:teuthology.orchestra.run.smithi086.stdout:2024-04-20T21:20:14.260017+0000 mds.a [WRN] bad backtrace on inode 0x10000000001(/parent_a/alpha), rewriting it 2024-04-20T21:20:14.762 INFO:teuthology.orchestra.run.smithi086.stdout:2024-04-20T21:20:14.260029+0000 mds.a [INF] Scrub repaired inode 0x10000000001 (/parent_a/alpha) 2024-04-20T21:20:14.762 INFO:teuthology.orchestra.run.smithi086.stdout:2024-04-20T21:20:14.260069+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-04-20T21:20:15.827 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:20:15.836 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:20:18.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:18.475+0000 7f062ea00640 1 -- 172.21.15.86:0/2326961910 <== mon.0 v2:172.21.15.86:3300/0 8 ==== log(1 entries from seq 7 at 2024-04-20T21:20:17.421543+0000) ==== 189+0+0 (secure 0 0 0) 0x7f062808d000 con 0x7f06380d3120 2024-04-20T21:20:18.475 INFO:teuthology.orchestra.run.smithi086.stdout:2024-04-20T21:20:17.421543+0000 mds.a [INF] scrub summary: idle 2024-04-20T21:20:33.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:33.763+0000 7f062ea00640 1 -- 172.21.15.86:0/2326961910 <== mon.0 v2:172.21.15.86:3300/0 9 ==== log(2 entries from seq 561 at 2024-04-20T21:20:32.813837+0000) ==== 603+0+0 (secure 0 0 0) 0x7f0628058800 con 0x7f06380d3120 2024-04-20T21:20:34.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:34.783+0000 7f062ea00640 1 -- 172.21.15.86:0/2326961910 <== mon.0 v2:172.21.15.86:3300/0 10 ==== log(2 entries from seq 563 at 2024-04-20T21:20:33.771391+0000) ==== 607+0+0 (secure 0 0 0) 0x7f0628058a60 con 0x7f06380d3120 2024-04-20T21:20:44.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:20:44.528 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-20T21:20:44.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.740+0000 7f2d6be00640 1 Processor -- start 2024-04-20T21:20:44.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.740+0000 7f2d6be00640 1 -- start start 2024-04-20T21:20:44.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.741+0000 7f2d6be00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d6c15ecc0 0x7f2d6c0d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:44.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.741+0000 7f2d6be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2d6c0d4ba0 0x7f2d6c0cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:44.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.741+0000 7f2d6be00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d6c0d1f60 con 0x7f2d6c15e8e0 2024-04-20T21:20:44.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.741+0000 7f2d6be00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f2d6c0d5010 con 0x7f2d6c0d4ba0 2024-04-20T21:20:44.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.741+0000 7f2d6be00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2d6c0cf7f0 con 0x7f2d6c15ecc0 2024-04-20T21:20:44.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.742+0000 7f2d6b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2d6c0d4ba0 0x7f2d6c0cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:44.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.742+0000 7f2d6b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2d6c0d4ba0 0x7f2d6c0cfc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54260/0 (socket says 172.21.15.86:54260) 2024-04-20T21:20:44.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.742+0000 7f2d6b400640 1 -- 172.21.15.86:0/931850399 learned_addr learned my addr 172.21.15.86:0/931850399 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:44.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.742+0000 7f2d6a000640 1 --2- 172.21.15.86:0/931850399 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d6c15ecc0 0x7f2d6c0d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:44.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.742+0000 7f2d6b400640 1 -- 172.21.15.86:0/931850399 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d6c15ecc0 msgr2=0x7f2d6c0d4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:44.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.742+0000 7f2d6b400640 1 --2- 172.21.15.86:0/931850399 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d6c15ecc0 0x7f2d6c0d4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:44.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.742+0000 7f2d6b400640 1 -- 172.21.15.86:0/931850399 >> v1:172.21.15.86:6789/0 conn(0x7f2d6c15e8e0 legacy=0x7f2d6c06d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:44.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.742+0000 7f2d6b400640 1 -- 172.21.15.86:0/931850399 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d6c0d19a0 con 0x7f2d6c0d4ba0 2024-04-20T21:20:44.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.742+0000 7f2d6a000640 1 --2- 172.21.15.86:0/931850399 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d6c15ecc0 0x7f2d6c0d4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:20:44.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.744+0000 7f2d6b400640 1 --2- 172.21.15.86:0/931850399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d6c0d4ba0 0x7f2d6c0cfc80 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f2d64009e50 tx=0x7f2d6402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3016d4a1956ba20 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:44.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.744+0000 7f2d69600640 1 -- 172.21.15.86:0/931850399 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6402cb00 con 0x7f2d6c0d4ba0 2024-04-20T21:20:44.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.744+0000 7f2d69600640 1 -- 172.21.15.86:0/931850399 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d6402cca0 con 0x7f2d6c0d4ba0 2024-04-20T21:20:44.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.744+0000 7f2d69600640 1 -- 172.21.15.86:0/931850399 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d64034610 con 0x7f2d6c0d4ba0 2024-04-20T21:20:44.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.745+0000 7f2d6be00640 1 -- 172.21.15.86:0/931850399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d6c0d4ba0 msgr2=0x7f2d6c0cfc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:44.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.745+0000 7f2d6be00640 1 --2- 172.21.15.86:0/931850399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d6c0d4ba0 0x7f2d6c0cfc80 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f2d64009e50 tx=0x7f2d6402bf60 comp rx=0 tx=0).stop 2024-04-20T21:20:44.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.745+0000 7f2d6be00640 1 -- 172.21.15.86:0/931850399 shutdown_connections 2024-04-20T21:20:44.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.745+0000 7f2d6be00640 1 --2- 172.21.15.86:0/931850399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d6c0d4ba0 0x7f2d6c0cfc80 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:44.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.745+0000 7f2d6be00640 1 --2- 172.21.15.86:0/931850399 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d6c15ecc0 0x7f2d6c0d4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:44.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.745+0000 7f2d6be00640 1 -- 172.21.15.86:0/931850399 >> 172.21.15.86:0/931850399 conn(0x7f2d6c09d9d0 msgr2=0x7f2d6c08bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:44.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.745+0000 7f2d6be00640 1 -- 172.21.15.86:0/931850399 shutdown_connections 2024-04-20T21:20:44.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.745+0000 7f2d6be00640 1 -- 172.21.15.86:0/931850399 wait complete. 2024-04-20T21:20:44.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.747+0000 7f2d6be00640 1 Processor -- start 2024-04-20T21:20:44.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.747+0000 7f2d6be00640 1 -- start start 2024-04-20T21:20:44.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.748+0000 7f2d6be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d6c0d4ba0 0x7f2d6c152d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:44.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.748+0000 7f2d6be00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d6c15e8e0 0x7f2d6c0c9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:44.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.748+0000 7f2d6be00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d6c0c9c80 0x7f2d6c160ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:44.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.748+0000 7f2d6be00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6c06d910 con 0x7f2d6c0d4ba0 2024-04-20T21:20:44.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.748+0000 7f2d6be00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6c0cf480 con 0x7f2d6c0c9c80 2024-04-20T21:20:44.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.748+0000 7f2d6be00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2d6c0d01c0 con 0x7f2d6c15e8e0 2024-04-20T21:20:44.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.748+0000 7f2d6aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d6c0d4ba0 0x7f2d6c152d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:44.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.748+0000 7f2d6aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d6c0d4ba0 0x7f2d6c152d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54272/0 (socket says 172.21.15.86:54272) 2024-04-20T21:20:44.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.748+0000 7f2d6aa00640 1 -- 172.21.15.86:0/2520109362 learned_addr learned my addr 172.21.15.86:0/2520109362 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:44.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.749+0000 7f2d6a000640 1 --2- 172.21.15.86:0/2520109362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d6c15e8e0 0x7f2d6c0c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:44.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.749+0000 7f2d6aa00640 1 -- 172.21.15.86:0/2520109362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d6c15e8e0 msgr2=0x7f2d6c0c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:44.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.749+0000 7f2d6aa00640 1 --2- 172.21.15.86:0/2520109362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d6c15e8e0 0x7f2d6c0c9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:44.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.749+0000 7f2d6aa00640 1 -- 172.21.15.86:0/2520109362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d6c0c9c80 msgr2=0x7f2d6c160ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:44.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.749+0000 7f2d6aa00640 1 --2- 172.21.15.86:0/2520109362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d6c0c9c80 0x7f2d6c160ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:44.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.749+0000 7f2d6aa00640 1 -- 172.21.15.86:0/2520109362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d6c15fc60 con 0x7f2d6c0d4ba0 2024-04-20T21:20:44.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.749+0000 7f2d6aa00640 1 --2- 172.21.15.86:0/2520109362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d6c0d4ba0 0x7f2d6c152d50 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c00c430 tx=0x7f2d5c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:44.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.749+0000 7f2d62a00640 1 -- 172.21.15.86:0/2520109362 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c017070 con 0x7f2d6c0d4ba0 2024-04-20T21:20:44.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.749+0000 7f2d62a00640 1 -- 172.21.15.86:0/2520109362 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d5c0052d0 con 0x7f2d6c0d4ba0 2024-04-20T21:20:44.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.749+0000 7f2d62a00640 1 -- 172.21.15.86:0/2520109362 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c003850 con 0x7f2d6c0d4ba0 2024-04-20T21:20:44.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.749+0000 7f2d6be00640 1 -- 172.21.15.86:0/2520109362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d6c15ff40 con 0x7f2d6c0d4ba0 2024-04-20T21:20:44.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.749+0000 7f2d6be00640 1 -- 172.21.15.86:0/2520109362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d6c160500 con 0x7f2d6c0d4ba0 2024-04-20T21:20:44.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.751+0000 7f2d62a00640 1 -- 172.21.15.86:0/2520109362 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2d5c01a070 con 0x7f2d6c0d4ba0 2024-04-20T21:20:44.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.752+0000 7f2d62a00640 1 --2- 172.21.15.86:0/2520109362 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2d4c06c8f0 0x7f2d4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:44.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.753+0000 7f2d62a00640 1 -- 172.21.15.86:0/2520109362 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f2d5c08afd0 con 0x7f2d6c0d4ba0 2024-04-20T21:20:44.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.753+0000 7f2d6a000640 1 --2- 172.21.15.86:0/2520109362 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2d4c06c8f0 0x7f2d4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:44.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.754+0000 7f2d745c5740 1 Processor -- start 2024-04-20T21:20:44.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.754+0000 7f2d6a000640 1 --2- 172.21.15.86:0/2520109362 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2d4c06c8f0 0x7f2d4c06edb0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2d580096d0 tx=0x7f2d58008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:44.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.754+0000 7f2d745c5740 1 -- start start 2024-04-20T21:20:44.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.755+0000 7f2d745c5740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555a2550a3d0 0x555a2550a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:44.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.755+0000 7f2d745c5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a254da3f0 0x555a25507640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:44.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.755+0000 7f2d745c5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a254ddf50 0x555a25507b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:44.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.755+0000 7f2d745c5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555a25446150 con 0x555a254ddf50 2024-04-20T21:20:44.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.755+0000 7f2d745c5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555a253ec370 con 0x555a254da3f0 2024-04-20T21:20:44.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.755+0000 7f2d745c5740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x555a25442110 con 0x555a2550a3d0 2024-04-20T21:20:44.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.755+0000 7f2d6b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a254ddf50 0x555a25507b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:44.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.755+0000 7f2d6b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a254ddf50 0x555a25507b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54288/0 (socket says 172.21.15.86:54288) 2024-04-20T21:20:44.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.755+0000 7f2d6b400640 1 -- 172.21.15.86:0/4039778489 learned_addr learned my addr 172.21.15.86:0/4039778489 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:44.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.755+0000 7f2d6aa00640 1 --2- 172.21.15.86:0/4039778489 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555a2550a3d0 0x555a2550a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:44.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.755+0000 7f2d6b400640 1 -- 172.21.15.86:0/4039778489 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555a2550a3d0 msgr2=0x555a2550a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:44.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.755+0000 7f2d6b400640 1 --2- 172.21.15.86:0/4039778489 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555a2550a3d0 0x555a2550a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:44.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.755+0000 7f2d6b400640 1 -- 172.21.15.86:0/4039778489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a254da3f0 msgr2=0x555a25507640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:44.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.755+0000 7f2d6b400640 1 --2- 172.21.15.86:0/4039778489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a254da3f0 0x555a25507640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:44.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.756+0000 7f2d6b400640 1 -- 172.21.15.86:0/4039778489 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a252227d0 con 0x555a254ddf50 2024-04-20T21:20:44.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.756+0000 7f2d6b400640 1 --2- 172.21.15.86:0/4039778489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a254ddf50 0x555a25507b80 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f2d64009f70 tx=0x7f2d6402e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6964c50507f4788 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:44.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.756+0000 7f2d57400640 1 -- 172.21.15.86:0/4039778489 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d64009510 con 0x555a254ddf50 2024-04-20T21:20:44.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.756+0000 7f2d57400640 1 -- 172.21.15.86:0/4039778489 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d64044070 con 0x555a254ddf50 2024-04-20T21:20:44.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.756+0000 7f2d57400640 1 -- 172.21.15.86:0/4039778489 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6402cb20 con 0x555a254ddf50 2024-04-20T21:20:44.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.757+0000 7f2d745c5740 1 -- 172.21.15.86:0/4039778489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a254ddf50 msgr2=0x555a25507b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:44.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.757+0000 7f2d745c5740 1 --2- 172.21.15.86:0/4039778489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a254ddf50 0x555a25507b80 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f2d64009f70 tx=0x7f2d6402e070 comp rx=0 tx=0).stop 2024-04-20T21:20:44.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.757+0000 7f2d745c5740 1 -- 172.21.15.86:0/4039778489 shutdown_connections 2024-04-20T21:20:44.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.757+0000 7f2d745c5740 1 --2- 172.21.15.86:0/4039778489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a254ddf50 0x555a25507b80 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:44.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.757+0000 7f2d745c5740 1 --2- 172.21.15.86:0/4039778489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a254da3f0 0x555a25507640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:44.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.757+0000 7f2d745c5740 1 --2- 172.21.15.86:0/4039778489 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555a2550a3d0 0x555a2550a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:44.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.757+0000 7f2d745c5740 1 -- 172.21.15.86:0/4039778489 >> 172.21.15.86:0/4039778489 conn(0x555a254dc660 msgr2=0x555a255251e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:44.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.757+0000 7f2d745c5740 1 -- 172.21.15.86:0/4039778489 shutdown_connections 2024-04-20T21:20:44.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.757+0000 7f2d745c5740 1 -- 172.21.15.86:0/4039778489 wait complete. 2024-04-20T21:20:44.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.758+0000 7f2d745c5740 1 -- start start 2024-04-20T21:20:44.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.758+0000 7f2d745c5740 1 Processor -- start 2024-04-20T21:20:44.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.759+0000 7f2d745c5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a254ddf50 0x555a25505630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:44.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.759+0000 7f2d745c5740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555a2550a3d0 0x555a25527000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:44.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.759+0000 7f2d745c5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a254ffc80 0x555a25527540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:44.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.759+0000 7f2d745c5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555a254df100 con 0x555a254ffc80 2024-04-20T21:20:44.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.759+0000 7f2d745c5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555a253ff740 con 0x555a254ddf50 2024-04-20T21:20:44.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.759+0000 7f2d745c5740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x555a254de4f0 con 0x555a2550a3d0 2024-04-20T21:20:44.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.759+0000 7f2d745c5740 10 client.0 _ll_register_callbacks cb 0x555a253fab20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:20:44.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.759+0000 7f2d6b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a254ffc80 0x555a25527540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:44.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.759+0000 7f2d6b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a254ffc80 0x555a25527540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54294/0 (socket says 172.21.15.86:54294) 2024-04-20T21:20:44.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.759+0000 7f2d6b400640 1 -- 172.21.15.86:0/2899379270 learned_addr learned my addr 172.21.15.86:0/2899379270 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:44.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.760+0000 7f2d6a000640 1 --2- 172.21.15.86:0/2899379270 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555a2550a3d0 0x555a25527000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:44.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.760+0000 7f2d6aa00640 1 --2- 172.21.15.86:0/2899379270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a254ddf50 0x555a25505630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:44.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.760+0000 7f2d6b400640 1 -- 172.21.15.86:0/2899379270 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555a2550a3d0 msgr2=0x555a25527000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:44.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.760+0000 7f2d6b400640 1 --2- 172.21.15.86:0/2899379270 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555a2550a3d0 0x555a25527000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:44.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.760+0000 7f2d6b400640 1 -- 172.21.15.86:0/2899379270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a254ddf50 msgr2=0x555a25505630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:44.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.760+0000 7f2d6b400640 1 --2- 172.21.15.86:0/2899379270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a254ddf50 0x555a25505630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:44.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.760+0000 7f2d6b400640 1 -- 172.21.15.86:0/2899379270 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a251583f0 con 0x555a254ffc80 2024-04-20T21:20:44.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.760+0000 7f2d6aa00640 1 --2- 172.21.15.86:0/2899379270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a254ddf50 0x555a25505630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:20:44.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.760+0000 7f2d6b400640 1 --2- 172.21.15.86:0/2899379270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a254ffc80 0x555a25527540 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f2d64041040 tx=0x7f2d64004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bbc66e1ad1c23e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:44.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.760+0000 7f2d4be00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:20:44.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.760+0000 7f2d4be00640 1 -- 172.21.15.86:0/2899379270 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d640040f0 con 0x555a254ffc80 2024-04-20T21:20:44.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.760+0000 7f2d4be00640 1 -- 172.21.15.86:0/2899379270 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f2d64004290 con 0x555a254ffc80 2024-04-20T21:20:44.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.760+0000 7f2d745c5740 1 -- 172.21.15.86:0/2899379270 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d64044070 con 0x555a254ffc80 2024-04-20T21:20:44.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.761+0000 7f2d4be00640 1 -- 172.21.15.86:0/2899379270 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2d64058050 con 0x555a254ffc80 2024-04-20T21:20:44.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.761+0000 7f2d745c5740 10 client.5919 fetch_fsmap learned FSMap version 24 2024-04-20T21:20:44.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.761+0000 7f2d745c5740 1 -- 172.21.15.86:0/2899379270 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x555a251180f0 con 0x555a254ffc80 2024-04-20T21:20:44.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.765+0000 7f2d4be00640 1 -- 172.21.15.86:0/2899379270 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d64044070 con 0x555a254ffc80 2024-04-20T21:20:44.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.766+0000 7f2d4be00640 1 -- 172.21.15.86:0/2899379270 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 24) ==== 3495+0+0 (secure 0 0 0) 0x7f2d6405b020 con 0x555a254ffc80 2024-04-20T21:20:44.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.766+0000 7f2d745c5740 10 client.5919 fetch_fsmap finished waiting for FSMap version 24 2024-04-20T21:20:44.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.766+0000 7f2d745c5740 10 client.5919 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-20T21:20:44.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.766+0000 7f2d745c5740 20 client.5919 populate_metadata read hostname 'smithi086' 2024-04-20T21:20:44.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.766+0000 7f2d745c5740 1 --2- 172.21.15.86:0/2899379270 >> [v2:172.21.15.86:6836/712628984,v1:172.21.15.86:6837/712628984] conn(0x555a25536710 0x555a25500830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:44.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.766+0000 7f2d745c5740 4 client.5919 mds_command: new command op to 5691 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:20:44.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.766+0000 7f2d745c5740 1 -- 172.21.15.86:0/2899379270 --> [v2:172.21.15.86:6836/712628984,v1:172.21.15.86:6837/712628984] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a25453b70 con 0x555a25536710 2024-04-20T21:20:44.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.766+0000 7f2d6aa00640 1 --2- 172.21.15.86:0/2899379270 >> [v2:172.21.15.86:6836/712628984,v1:172.21.15.86:6837/712628984] conn(0x555a25536710 0x555a25500830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:44.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.767+0000 7f2d6aa00640 1 --2- 172.21.15.86:0/2899379270 >> [v2:172.21.15.86:6836/712628984,v1:172.21.15.86:6837/712628984] conn(0x555a25536710 0x555a25500830 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=813f2e7c310f33f6 server_cookie=a322054b108d9d34 in_seq=0 out_seq=0 2024-04-20T21:20:44.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.767+0000 7f2d4be00640 10 client.5919 ms_handle_connect on v2:172.21.15.86:6836/712628984 2024-04-20T21:20:44.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.767+0000 7f2d4be00640 1 -- 172.21.15.86:0/2899379270 <== mds.0 v2:172.21.15.86:6836/712628984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555a25453b70 con 0x555a25536710 2024-04-20T21:20:44.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.767+0000 7f2d4be00640 10 client.5919 handle_command_reply: tid=0 2024-04-20T21:20:44.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.767+0000 7f2d745c5740 1 client.5919 shutdown 2024-04-20T21:20:44.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.768+0000 7f2d745c5740 1 -- 172.21.15.86:0/2899379270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a254ffc80 msgr2=0x555a25527540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:44.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.768+0000 7f2d745c5740 1 --2- 172.21.15.86:0/2899379270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a254ffc80 0x555a25527540 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f2d64041040 tx=0x7f2d64004ab0 comp rx=0 tx=0).stop 2024-04-20T21:20:44.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.768+0000 7f2d745c5740 1 -- 172.21.15.86:0/2899379270 shutdown_connections 2024-04-20T21:20:44.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.768+0000 7f2d745c5740 1 --2- 172.21.15.86:0/2899379270 >> [v2:172.21.15.86:6836/712628984,v1:172.21.15.86:6837/712628984] conn(0x555a25536710 0x555a25500830 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:44.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.768+0000 7f2d745c5740 1 --2- 172.21.15.86:0/2899379270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a254ffc80 0x555a25527540 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:44.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.768+0000 7f2d745c5740 1 --2- 172.21.15.86:0/2899379270 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555a2550a3d0 0x555a25527000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:44.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.768+0000 7f2d745c5740 1 --2- 172.21.15.86:0/2899379270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a254ddf50 0x555a25505630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:44.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.768+0000 7f2d4be00640 0 client.5919 ms_handle_reset on v2:172.21.15.86:6836/712628984 2024-04-20T21:20:44.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.768+0000 7f2d745c5740 1 -- 172.21.15.86:0/2899379270 >> 172.21.15.86:0/2899379270 conn(0x555a254da3f0 msgr2=0x555a254e0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:44.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.768+0000 7f2d745c5740 1 -- 172.21.15.86:0/2899379270 shutdown_connections 2024-04-20T21:20:44.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.768+0000 7f2d745c5740 1 -- 172.21.15.86:0/2899379270 wait complete. 2024-04-20T21:20:44.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.768+0000 7f2d745c5740 20 client.5919 trim_cache size 0 max 16384 2024-04-20T21:20:44.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.788+0000 7f2d745c5740 1 Processor -- start 2024-04-20T21:20:44.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.788+0000 7f2d745c5740 1 -- start start 2024-04-20T21:20:44.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.789+0000 7f2d745c5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a254ddf50 0x555a25527540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:44.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.789+0000 7f2d745c5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a254ffc80 0x555a255286f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:44.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.789+0000 7f2d745c5740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555a2550a3d0 0x555a25528c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:44.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.789+0000 7f2d745c5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555a254e13c0 con 0x555a254ffc80 2024-04-20T21:20:44.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.789+0000 7f2d745c5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6c200d10 con 0x555a254ddf50 2024-04-20T21:20:44.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.789+0000 7f2d745c5740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2d6c1ffce0 con 0x555a2550a3d0 2024-04-20T21:20:44.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.789+0000 7f2d6a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a254ffc80 0x555a255286f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:44.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.789+0000 7f2d6a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a254ffc80 0x555a255286f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54302/0 (socket says 172.21.15.86:54302) 2024-04-20T21:20:44.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.789+0000 7f2d6a000640 1 -- 172.21.15.86:0/2067467535 learned_addr learned my addr 172.21.15.86:0/2067467535 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:44.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.789+0000 7f2d6a000640 1 -- 172.21.15.86:0/2067467535 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555a2550a3d0 msgr2=0x555a25528c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:44.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.789+0000 7f2d6a000640 1 --2- 172.21.15.86:0/2067467535 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555a2550a3d0 0x555a25528c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:44.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.789+0000 7f2d6a000640 1 -- 172.21.15.86:0/2067467535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a254ddf50 msgr2=0x555a25527540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:44.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.789+0000 7f2d6a000640 1 --2- 172.21.15.86:0/2067467535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a254ddf50 0x555a25527540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:44.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.789+0000 7f2d6a000640 1 -- 172.21.15.86:0/2067467535 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a25502600 con 0x555a254ffc80 2024-04-20T21:20:44.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.790+0000 7f2d6a000640 1 --2- 172.21.15.86:0/2067467535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a254ffc80 0x555a255286f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f2d5800e6f0 tx=0x7f2d58011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e525cb869429e2e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:44.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.790+0000 7f2d56a00640 1 -- 172.21.15.86:0/2067467535 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d58026070 con 0x555a254ffc80 2024-04-20T21:20:44.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.790+0000 7f2d56a00640 1 -- 172.21.15.86:0/2067467535 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d5800ede0 con 0x555a254ffc80 2024-04-20T21:20:44.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.790+0000 7f2d56a00640 1 -- 172.21.15.86:0/2067467535 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5800f040 con 0x555a254ffc80 2024-04-20T21:20:44.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.791+0000 7f2d745c5740 1 -- 172.21.15.86:0/2067467535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a254ffc80 msgr2=0x555a255286f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:44.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.791+0000 7f2d745c5740 1 --2- 172.21.15.86:0/2067467535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a254ffc80 0x555a255286f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f2d5800e6f0 tx=0x7f2d58011000 comp rx=0 tx=0).stop 2024-04-20T21:20:44.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.791+0000 7f2d745c5740 1 -- 172.21.15.86:0/2067467535 shutdown_connections 2024-04-20T21:20:44.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.791+0000 7f2d745c5740 1 --2- 172.21.15.86:0/2067467535 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555a2550a3d0 0x555a25528c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:44.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.791+0000 7f2d745c5740 1 --2- 172.21.15.86:0/2067467535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a254ffc80 0x555a255286f0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:44.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.791+0000 7f2d745c5740 1 --2- 172.21.15.86:0/2067467535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a254ddf50 0x555a25527540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:44.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.791+0000 7f2d745c5740 1 -- 172.21.15.86:0/2067467535 >> 172.21.15.86:0/2067467535 conn(0x555a254da3f0 msgr2=0x555a25526570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:44.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.791+0000 7f2d745c5740 1 -- 172.21.15.86:0/2067467535 shutdown_connections 2024-04-20T21:20:44.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.791+0000 7f2d745c5740 1 -- 172.21.15.86:0/2067467535 wait complete. 2024-04-20T21:20:44.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.792+0000 7f2d745c5740 1 -- start start 2024-04-20T21:20:44.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.792+0000 7f2d745c5740 1 Processor -- start 2024-04-20T21:20:44.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.793+0000 7f2d745c5740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555a254ddf50 0x555a255488c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:44.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.793+0000 7f2d745c5740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a2550a3d0 0x555a255ac0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:44.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.793+0000 7f2d745c5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a255a7cc0 0x555a2551ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:44.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.793+0000 7f2d745c5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6c0b8cf0 con 0x555a255a7cc0 2024-04-20T21:20:44.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.793+0000 7f2d745c5740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6c0b86a0 con 0x555a2550a3d0 2024-04-20T21:20:44.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.793+0000 7f2d745c5740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x555a254ffa90 con 0x555a254ddf50 2024-04-20T21:20:44.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.793+0000 7f2d745c5740 10 client.0 _ll_register_callbacks cb 0x7f2d640092e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:20:44.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.793+0000 7f2d6b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a255a7cc0 0x555a2551ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:44.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.793+0000 7f2d6b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a255a7cc0 0x555a2551ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54316/0 (socket says 172.21.15.86:54316) 2024-04-20T21:20:44.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.793+0000 7f2d6b400640 1 -- 172.21.15.86:0/2958833478 learned_addr learned my addr 172.21.15.86:0/2958833478 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:44.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.793+0000 7f2d6b400640 1 -- 172.21.15.86:0/2958833478 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555a254ddf50 msgr2=0x555a255488c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:44.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.793+0000 7f2d6aa00640 1 --2- 172.21.15.86:0/2958833478 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555a254ddf50 0x555a255488c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:44.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.793+0000 7f2d6b400640 1 --2- 172.21.15.86:0/2958833478 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555a254ddf50 0x555a255488c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:44.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.793+0000 7f2d6b400640 1 -- 172.21.15.86:0/2958833478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a2550a3d0 msgr2=0x555a255ac0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:44.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.793+0000 7f2d6a000640 1 --2- 172.21.15.86:0/2958833478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a2550a3d0 0x555a255ac0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:44.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.793+0000 7f2d6b400640 1 --2- 172.21.15.86:0/2958833478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a2550a3d0 0x555a255ac0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:44.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.793+0000 7f2d6b400640 1 -- 172.21.15.86:0/2958833478 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a254de6a0 con 0x555a255a7cc0 2024-04-20T21:20:44.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.794+0000 7f2d6b400640 1 --2- 172.21.15.86:0/2958833478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a255a7cc0 0x555a2551ab70 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f2d64004af0 tx=0x7f2d64004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53418c3e9bde9a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:44.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.794+0000 7f2d4be00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:20:44.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.794+0000 7f2d4be00640 1 -- 172.21.15.86:0/2958833478 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6402db20 con 0x555a255a7cc0 2024-04-20T21:20:44.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.794+0000 7f2d4be00640 1 -- 172.21.15.86:0/2958833478 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f2d6405dae0 con 0x555a255a7cc0 2024-04-20T21:20:44.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.794+0000 7f2d745c5740 1 -- 172.21.15.86:0/2958833478 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d5800ede0 con 0x555a255a7cc0 2024-04-20T21:20:44.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.794+0000 7f2d4be00640 1 -- 172.21.15.86:0/2958833478 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2d64009af0 con 0x555a255a7cc0 2024-04-20T21:20:44.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.794+0000 7f2d745c5740 10 client.5925 fetch_fsmap learned FSMap version 24 2024-04-20T21:20:44.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.794+0000 7f2d745c5740 1 -- 172.21.15.86:0/2958833478 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x555a25527710 con 0x555a255a7cc0 2024-04-20T21:20:44.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.795+0000 7f2d4be00640 1 -- 172.21.15.86:0/2958833478 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d5800ede0 con 0x555a255a7cc0 2024-04-20T21:20:44.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.795+0000 7f2d4be00640 1 -- 172.21.15.86:0/2958833478 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 24) ==== 3495+0+0 (secure 0 0 0) 0x7f2d6405b040 con 0x555a255a7cc0 2024-04-20T21:20:44.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.795+0000 7f2d745c5740 10 client.5925 fetch_fsmap finished waiting for FSMap version 24 2024-04-20T21:20:44.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.795+0000 7f2d745c5740 10 client.5925 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-20T21:20:44.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.795+0000 7f2d745c5740 20 client.5925 populate_metadata read hostname 'smithi086' 2024-04-20T21:20:44.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.795+0000 7f2d745c5740 1 --2- 172.21.15.86:0/2958833478 >> [v2:172.21.15.86:6836/712628984,v1:172.21.15.86:6837/712628984] conn(0x555a2551b100 0x555a255286f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:44.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.795+0000 7f2d745c5740 4 client.5925 mds_command: new command op to 5691 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:20:44.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.795+0000 7f2d745c5740 1 -- 172.21.15.86:0/2958833478 --> [v2:172.21.15.86:6836/712628984,v1:172.21.15.86:6837/712628984] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555a2544d9e0 con 0x555a2551b100 2024-04-20T21:20:44.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.795+0000 7f2d6aa00640 1 --2- 172.21.15.86:0/2958833478 >> [v2:172.21.15.86:6836/712628984,v1:172.21.15.86:6837/712628984] conn(0x555a2551b100 0x555a255286f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:44.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.796+0000 7f2d4be00640 10 client.5925 ms_handle_connect on v2:172.21.15.86:6836/712628984 2024-04-20T21:20:44.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.796+0000 7f2d6aa00640 1 --2- 172.21.15.86:0/2958833478 >> [v2:172.21.15.86:6836/712628984,v1:172.21.15.86:6837/712628984] conn(0x555a2551b100 0x555a255286f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0996dcb0f8b6004 server_cookie=401a9ad4a4ecc747 in_seq=0 out_seq=0 2024-04-20T21:20:44.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.796+0000 7f2d4be00640 1 -- 172.21.15.86:0/2958833478 <== mds.0 v2:172.21.15.86:6836/712628984 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555a2544d9e0 con 0x555a2551b100 2024-04-20T21:20:44.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.796+0000 7f2d4be00640 10 client.5925 handle_command_reply: tid=0 2024-04-20T21:20:44.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.796+0000 7f2d745c5740 1 client.5925 shutdown 2024-04-20T21:20:44.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.796+0000 7f2d745c5740 1 -- 172.21.15.86:0/2958833478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a255a7cc0 msgr2=0x555a2551ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:44.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.796+0000 7f2d745c5740 1 --2- 172.21.15.86:0/2958833478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a255a7cc0 0x555a2551ab70 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f2d64004af0 tx=0x7f2d64004a40 comp rx=0 tx=0).stop 2024-04-20T21:20:44.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.796+0000 7f2d745c5740 1 -- 172.21.15.86:0/2958833478 shutdown_connections 2024-04-20T21:20:44.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.796+0000 7f2d745c5740 1 --2- 172.21.15.86:0/2958833478 >> [v2:172.21.15.86:6836/712628984,v1:172.21.15.86:6837/712628984] conn(0x555a2551b100 0x555a255286f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:44.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.796+0000 7f2d745c5740 1 --2- 172.21.15.86:0/2958833478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555a255a7cc0 0x555a2551ab70 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:44.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.796+0000 7f2d745c5740 1 --2- 172.21.15.86:0/2958833478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555a2550a3d0 0x555a255ac0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:44.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.796+0000 7f2d745c5740 1 --2- 172.21.15.86:0/2958833478 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555a254ddf50 0x555a255488c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:44.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.796+0000 7f2d4be00640 0 client.5925 ms_handle_reset on v2:172.21.15.86:6836/712628984 2024-04-20T21:20:44.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.796+0000 7f2d745c5740 1 -- 172.21.15.86:0/2958833478 >> 172.21.15.86:0/2958833478 conn(0x555a254da3f0 msgr2=0x555a254da880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:44.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.796+0000 7f2d745c5740 1 -- 172.21.15.86:0/2958833478 shutdown_connections 2024-04-20T21:20:44.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.796+0000 7f2d745c5740 1 -- 172.21.15.86:0/2958833478 wait complete. 2024-04-20T21:20:44.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.796+0000 7f2d745c5740 20 client.5925 trim_cache size 0 max 16384 2024-04-20T21:20:44.802 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:20:44.802 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-20T21:20:44.802 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-20T21:20:44.803 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:20:44.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.797+0000 7f2d6be00640 1 -- 172.21.15.86:0/2520109362 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2d4c06c8f0 msgr2=0x7f2d4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:44.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.797+0000 7f2d6be00640 1 --2- 172.21.15.86:0/2520109362 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2d4c06c8f0 0x7f2d4c06edb0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2d580096d0 tx=0x7f2d58008040 comp rx=0 tx=0).stop 2024-04-20T21:20:44.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.797+0000 7f2d6be00640 1 -- 172.21.15.86:0/2520109362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d6c0d4ba0 msgr2=0x7f2d6c152d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:44.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.797+0000 7f2d6be00640 1 --2- 172.21.15.86:0/2520109362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d6c0d4ba0 0x7f2d6c152d50 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c00c430 tx=0x7f2d5c00c8f0 comp rx=0 tx=0).stop 2024-04-20T21:20:44.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.797+0000 7f2d6be00640 1 -- 172.21.15.86:0/2520109362 shutdown_connections 2024-04-20T21:20:44.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.797+0000 7f2d6be00640 1 --2- 172.21.15.86:0/2520109362 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2d4c06c8f0 0x7f2d4c06edb0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:44.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.797+0000 7f2d6be00640 1 --2- 172.21.15.86:0/2520109362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d6c0c9c80 0x7f2d6c160ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:44.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.797+0000 7f2d6be00640 1 --2- 172.21.15.86:0/2520109362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d6c15e8e0 0x7f2d6c0c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:44.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.797+0000 7f2d6be00640 1 --2- 172.21.15.86:0/2520109362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d6c0d4ba0 0x7f2d6c152d50 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:44.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.797+0000 7f2d6be00640 1 -- 172.21.15.86:0/2520109362 >> 172.21.15.86:0/2520109362 conn(0x7f2d6c09d9d0 msgr2=0x7f2d6c0b4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:44.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.797+0000 7f2d6be00640 1 -- 172.21.15.86:0/2520109362 shutdown_connections 2024-04-20T21:20:44.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:44.797+0000 7f2d6be00640 1 -- 172.21.15.86:0/2520109362 wait complete. 2024-04-20T21:20:44.815 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T21:20:44.916 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:20:45.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.111+0000 7fd236800640 1 Processor -- start 2024-04-20T21:20:45.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.111+0000 7fd236800640 1 -- start start 2024-04-20T21:20:45.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.112+0000 7fd236800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd230156ae0 0x7fd23015d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:45.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.112+0000 7fd236800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd230160080 0x7fd2300ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:45.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.112+0000 7fd236800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2300ad620 0x7fd230141cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:45.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.112+0000 7fd236800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd2300ada90 con 0x7fd230160080 2024-04-20T21:20:45.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.112+0000 7fd236800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fd23006db20 con 0x7fd230156ae0 2024-04-20T21:20:45.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.112+0000 7fd236800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd230160890 con 0x7fd2300ad620 2024-04-20T21:20:45.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.113+0000 7fd234a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd230160080 0x7fd2300ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:45.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.113+0000 7fd234a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd230160080 0x7fd2300ad0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54320/0 (socket says 172.21.15.86:54320) 2024-04-20T21:20:45.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.113+0000 7fd234a00640 1 -- 172.21.15.86:0/3403239629 learned_addr learned my addr 172.21.15.86:0/3403239629 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:45.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.113+0000 7fd235400640 1 --2- 172.21.15.86:0/3403239629 >> v2:172.21.15.144:3300/0 conn(0x7fd230156ae0 0x7fd23015d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:45.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.113+0000 7fd234a00640 1 -- 172.21.15.86:0/3403239629 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2300ad620 msgr2=0x7fd230141cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:45.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.113+0000 7fd234a00640 1 --2- 172.21.15.86:0/3403239629 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2300ad620 0x7fd230141cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:45.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.113+0000 7fd234a00640 1 -- 172.21.15.86:0/3403239629 >> v2:172.21.15.144:3300/0 conn(0x7fd230156ae0 msgr2=0x7fd23015d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:45.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.113+0000 7fd234a00640 1 --2- 172.21.15.86:0/3403239629 >> v2:172.21.15.144:3300/0 conn(0x7fd230156ae0 0x7fd23015d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:45.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.113+0000 7fd234a00640 1 -- 172.21.15.86:0/3403239629 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd230160b70 con 0x7fd230160080 2024-04-20T21:20:45.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.114+0000 7fd234a00640 1 --2- 172.21.15.86:0/3403239629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd230160080 0x7fd2300ad0e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fd23006d460 tx=0x7fd21c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb680d5aeaec3aba server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:45.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.115+0000 7fd22fe00640 1 -- 172.21.15.86:0/3403239629 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd21c035020 con 0x7fd230160080 2024-04-20T21:20:45.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.115+0000 7fd22fe00640 1 -- 172.21.15.86:0/3403239629 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd21c031870 con 0x7fd230160080 2024-04-20T21:20:45.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.115+0000 7fd22fe00640 1 -- 172.21.15.86:0/3403239629 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd21c031c40 con 0x7fd230160080 2024-04-20T21:20:45.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.115+0000 7fd236800640 1 -- 172.21.15.86:0/3403239629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd230160080 msgr2=0x7fd2300ad0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:45.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.115+0000 7fd236800640 1 --2- 172.21.15.86:0/3403239629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd230160080 0x7fd2300ad0e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fd23006d460 tx=0x7fd21c02b910 comp rx=0 tx=0).stop 2024-04-20T21:20:45.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.115+0000 7fd236800640 1 -- 172.21.15.86:0/3403239629 shutdown_connections 2024-04-20T21:20:45.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.115+0000 7fd236800640 1 --2- 172.21.15.86:0/3403239629 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd2300ad620 0x7fd230141cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:45.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.115+0000 7fd236800640 1 --2- 172.21.15.86:0/3403239629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd230160080 0x7fd2300ad0e0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:45.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.115+0000 7fd236800640 1 --2- 172.21.15.86:0/3403239629 >> v2:172.21.15.144:3300/0 conn(0x7fd230156ae0 0x7fd23015d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:45.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.115+0000 7fd236800640 1 -- 172.21.15.86:0/3403239629 >> 172.21.15.86:0/3403239629 conn(0x7fd2300d0170 msgr2=0x7fd2300d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:45.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.115+0000 7fd236800640 1 -- 172.21.15.86:0/3403239629 shutdown_connections 2024-04-20T21:20:45.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.115+0000 7fd236800640 1 -- 172.21.15.86:0/3403239629 wait complete. 2024-04-20T21:20:45.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.117+0000 7fd236800640 1 Processor -- start 2024-04-20T21:20:45.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.117+0000 7fd236800640 1 -- start start 2024-04-20T21:20:45.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.118+0000 7fd236800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2300ad620 0x7fd230099ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:45.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.118+0000 7fd236800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd230156ae0 0x7fd2301f2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:45.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.118+0000 7fd236800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd230160080 0x7fd2300b3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:45.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.118+0000 7fd236800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd230156ec0 con 0x7fd2300ad620 2024-04-20T21:20:45.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.118+0000 7fd236800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd23006dd70 con 0x7fd230160080 2024-04-20T21:20:45.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.118+0000 7fd236800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd230160500 con 0x7fd230156ae0 2024-04-20T21:20:45.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.119+0000 7fd235400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2300ad620 0x7fd230099ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:45.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.119+0000 7fd235400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2300ad620 0x7fd230099ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54332/0 (socket says 172.21.15.86:54332) 2024-04-20T21:20:45.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.119+0000 7fd235400640 1 -- 172.21.15.86:0/2242094663 learned_addr learned my addr 172.21.15.86:0/2242094663 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:45.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.119+0000 7fd235e00640 1 --2- 172.21.15.86:0/2242094663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd230160080 0x7fd2300b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:45.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.119+0000 7fd234a00640 1 --2- 172.21.15.86:0/2242094663 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd230156ae0 0x7fd2301f2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:45.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.119+0000 7fd235400640 1 -- 172.21.15.86:0/2242094663 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd230156ae0 msgr2=0x7fd2301f2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:45.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.119+0000 7fd235400640 1 --2- 172.21.15.86:0/2242094663 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd230156ae0 0x7fd2301f2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:45.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.119+0000 7fd235400640 1 -- 172.21.15.86:0/2242094663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd230160080 msgr2=0x7fd2300b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:45.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.119+0000 7fd235400640 1 --2- 172.21.15.86:0/2242094663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd230160080 0x7fd2300b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:45.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.119+0000 7fd235400640 1 -- 172.21.15.86:0/2242094663 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd23015ab90 con 0x7fd2300ad620 2024-04-20T21:20:45.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.119+0000 7fd235400640 1 --2- 172.21.15.86:0/2242094663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2300ad620 0x7fd230099ab0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fd220007ef0 tx=0x7fd22000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:45.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.120+0000 7fd22d600640 1 -- 172.21.15.86:0/2242094663 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd220014620 con 0x7fd2300ad620 2024-04-20T21:20:45.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.120+0000 7fd22d600640 1 -- 172.21.15.86:0/2242094663 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd220019070 con 0x7fd2300ad620 2024-04-20T21:20:45.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.120+0000 7fd22d600640 1 -- 172.21.15.86:0/2242094663 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd220014a00 con 0x7fd2300ad620 2024-04-20T21:20:45.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.120+0000 7fd236800640 1 -- 172.21.15.86:0/2242094663 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd23015ae70 con 0x7fd2300ad620 2024-04-20T21:20:45.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.120+0000 7fd236800640 1 -- 172.21.15.86:0/2242094663 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd23015b430 con 0x7fd2300ad620 2024-04-20T21:20:45.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.121+0000 7fd236800640 1 -- 172.21.15.86:0/2242094663 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd23004fcf0 con 0x7fd2300ad620 2024-04-20T21:20:45.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.122+0000 7fd22d600640 1 -- 172.21.15.86:0/2242094663 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd220014ba0 con 0x7fd2300ad620 2024-04-20T21:20:45.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.123+0000 7fd22d600640 1 --2- 172.21.15.86:0/2242094663 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd21006c8f0 0x7fd21006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:45.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.123+0000 7fd22d600640 1 -- 172.21.15.86:0/2242094663 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fd22008c570 con 0x7fd2300ad620 2024-04-20T21:20:45.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.123+0000 7fd234a00640 1 --2- 172.21.15.86:0/2242094663 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd21006c8f0 0x7fd21006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:45.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.124+0000 7fd234a00640 1 --2- 172.21.15.86:0/2242094663 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd21006c8f0 0x7fd21006edb0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd21c036000 tx=0x7fd21c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:45.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.128+0000 7fd22d600640 1 -- 172.21.15.86:0/2242094663 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd22005a8c0 con 0x7fd2300ad620 2024-04-20T21:20:45.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.367+0000 7fd236800640 1 -- 172.21.15.86:0/2242094663 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd23006fb40 con 0x7fd2300ad620 2024-04-20T21:20:45.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.368+0000 7fd22d600640 1 -- 172.21.15.86:0/2242094663 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5199 (secure 0 0 0) 0x7fd22005a1c0 con 0x7fd2300ad620 2024-04-20T21:20:45.368 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:20:45.369 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5044,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/1013106052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":1013106052},{"type":"v1","addr":"172.21.15.144:6833","nonce":1013106052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5047,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/1073576945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1073576945},{"type":"v1","addr":"172.21.15.144:6835","nonce":1073576945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/821548012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":821548012},{"type":"v1","addr":"172.21.15.86:6835","nonce":821548012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:20:03.369281+0000","modified":"2024-04-20T21:20:06.438592+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5691},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5691":{"gid":5691,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/712628984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":712628984},{"type":"v1","addr":"172.21.15.86:6837","nonce":712628984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5691,"qdb_cluster":[5691]},"id":3}]} 2024-04-20T21:20:45.369 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 24 2024-04-20T21:20:45.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.371+0000 7fd236800640 1 -- 172.21.15.86:0/2242094663 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd21006c8f0 msgr2=0x7fd21006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:45.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.371+0000 7fd236800640 1 --2- 172.21.15.86:0/2242094663 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd21006c8f0 0x7fd21006edb0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd21c036000 tx=0x7fd21c008030 comp rx=0 tx=0).stop 2024-04-20T21:20:45.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.371+0000 7fd236800640 1 -- 172.21.15.86:0/2242094663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2300ad620 msgr2=0x7fd230099ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:45.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.371+0000 7fd236800640 1 --2- 172.21.15.86:0/2242094663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2300ad620 0x7fd230099ab0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fd220007ef0 tx=0x7fd22000e4b0 comp rx=0 tx=0).stop 2024-04-20T21:20:45.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.371+0000 7fd236800640 1 -- 172.21.15.86:0/2242094663 shutdown_connections 2024-04-20T21:20:45.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.371+0000 7fd236800640 1 --2- 172.21.15.86:0/2242094663 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd21006c8f0 0x7fd21006edb0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:45.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.371+0000 7fd236800640 1 --2- 172.21.15.86:0/2242094663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd230160080 0x7fd2300b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:45.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.371+0000 7fd236800640 1 --2- 172.21.15.86:0/2242094663 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd230156ae0 0x7fd2301f2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:45.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.371+0000 7fd236800640 1 --2- 172.21.15.86:0/2242094663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2300ad620 0x7fd230099ab0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:45.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.371+0000 7fd236800640 1 -- 172.21.15.86:0/2242094663 >> 172.21.15.86:0/2242094663 conn(0x7fd2300d0170 msgr2=0x7fd2300b0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:45.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.371+0000 7fd236800640 1 -- 172.21.15.86:0/2242094663 shutdown_connections 2024-04-20T21:20:45.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.371+0000 7fd236800640 1 -- 172.21.15.86:0/2242094663 wait complete. 2024-04-20T21:20:45.389 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-20T21:20:45.617 INFO:teuthology.orchestra.run.smithi086.stdout:{"message":"","return_code":0} 2024-04-20T21:20:45.617 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-20T21:20:45.618 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-20T21:20:45.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.715+0000 7faf406d9f40 1 Processor -- start 2024-04-20T21:20:45.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.715+0000 7faf406d9f40 1 -- start start 2024-04-20T21:20:45.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.716+0000 7faf406d9f40 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db4691e30 0x564db46ffa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:45.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.716+0000 7faf406d9f40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564db47047e0 0x564db47010e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:45.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.716+0000 7faf406d9f40 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564db4707ed0 0x564db4701620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:45.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.716+0000 7faf406d9f40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564db45eecc0 con 0x564db47047e0 2024-04-20T21:20:45.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.716+0000 7faf406d9f40 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564db45f2d00 con 0x564db4691e30 2024-04-20T21:20:45.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.716+0000 7faf406d9f40 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x564db4693cf0 con 0x564db4707ed0 2024-04-20T21:20:45.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.716+0000 7faf3ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564db47047e0 0x564db47010e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:45.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.716+0000 7faf3ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564db47047e0 0x564db47010e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54342/0 (socket says 172.21.15.86:54342) 2024-04-20T21:20:45.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.716+0000 7faf3ce00640 1 -- 172.21.15.86:0/3546623287 learned_addr learned my addr 172.21.15.86:0/3546623287 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:45.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.717+0000 7faf3d800640 1 --2- 172.21.15.86:0/3546623287 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db4691e30 0x564db46ffa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:45.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.717+0000 7faf3ce00640 1 -- 172.21.15.86:0/3546623287 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564db4707ed0 msgr2=0x564db4701620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:45.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.717+0000 7faf3e200640 1 --2- 172.21.15.86:0/3546623287 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564db4707ed0 0x564db4701620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:45.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.717+0000 7faf3ce00640 1 --2- 172.21.15.86:0/3546623287 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564db4707ed0 0x564db4701620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:45.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.717+0000 7faf3ce00640 1 -- 172.21.15.86:0/3546623287 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db4691e30 msgr2=0x564db46ffa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:45.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.717+0000 7faf3ce00640 1 --2- 172.21.15.86:0/3546623287 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db4691e30 0x564db46ffa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:45.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.717+0000 7faf3ce00640 1 -- 172.21.15.86:0/3546623287 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564db4705eb0 con 0x564db47047e0 2024-04-20T21:20:45.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.717+0000 7faf3ce00640 1 --2- 172.21.15.86:0/3546623287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564db47047e0 0x564db47010e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7faf2c00a460 tx=0x7faf2c00a920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5279076dc601288 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:45.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.717+0000 7faf37400640 1 -- 172.21.15.86:0/3546623287 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2c00b920 con 0x564db47047e0 2024-04-20T21:20:45.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.717+0000 7faf37400640 1 -- 172.21.15.86:0/3546623287 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf2c00f070 con 0x564db47047e0 2024-04-20T21:20:45.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.717+0000 7faf37400640 1 -- 172.21.15.86:0/3546623287 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2c00bd20 con 0x564db47047e0 2024-04-20T21:20:45.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.718+0000 7faf406d9f40 1 -- 172.21.15.86:0/3546623287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564db47047e0 msgr2=0x564db47010e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:45.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.718+0000 7faf406d9f40 1 --2- 172.21.15.86:0/3546623287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564db47047e0 0x564db47010e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7faf2c00a460 tx=0x7faf2c00a920 comp rx=0 tx=0).stop 2024-04-20T21:20:45.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.718+0000 7faf406d9f40 1 -- 172.21.15.86:0/3546623287 shutdown_connections 2024-04-20T21:20:45.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.718+0000 7faf406d9f40 1 --2- 172.21.15.86:0/3546623287 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564db4707ed0 0x564db4701620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:45.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.718+0000 7faf406d9f40 1 --2- 172.21.15.86:0/3546623287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564db47047e0 0x564db47010e0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:45.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.718+0000 7faf406d9f40 1 --2- 172.21.15.86:0/3546623287 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db4691e30 0x564db46ffa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:45.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.718+0000 7faf406d9f40 1 -- 172.21.15.86:0/3546623287 >> 172.21.15.86:0/3546623287 conn(0x564db464c950 msgr2=0x564db4704e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:45.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.718+0000 7faf406d9f40 1 -- 172.21.15.86:0/3546623287 shutdown_connections 2024-04-20T21:20:45.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.718+0000 7faf406d9f40 1 -- 172.21.15.86:0/3546623287 wait complete. 2024-04-20T21:20:45.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.719+0000 7faf406d9f40 1 Processor -- start 2024-04-20T21:20:45.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.719+0000 7faf406d9f40 1 -- start start 2024-04-20T21:20:45.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.720+0000 7faf406d9f40 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564db4691e30 0x564db46f8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:45.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.720+0000 7faf406d9f40 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db47047e0 0x564db47210a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:45.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.720+0000 7faf406d9f40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564db4707ed0 0x564db46fe030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:45.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.720+0000 7faf406d9f40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564db4706150 con 0x564db4707ed0 2024-04-20T21:20:45.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.720+0000 7faf406d9f40 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x564db47063f0 con 0x564db47047e0 2024-04-20T21:20:45.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.720+0000 7faf406d9f40 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x564db46fc500 con 0x564db4691e30 2024-04-20T21:20:45.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.720+0000 7faf3e200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564db4707ed0 0x564db46fe030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:45.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.720+0000 7faf3e200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564db4707ed0 0x564db46fe030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54348/0 (socket says 172.21.15.86:54348) 2024-04-20T21:20:45.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.720+0000 7faf3e200640 1 -- 172.21.15.86:0/405456821 learned_addr learned my addr 172.21.15.86:0/405456821 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:45.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.720+0000 7faf3d800640 1 --2- 172.21.15.86:0/405456821 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564db4691e30 0x564db46f8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:45.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.720+0000 7faf3ce00640 1 --2- 172.21.15.86:0/405456821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db47047e0 0x564db47210a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:45.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.720+0000 7faf3e200640 1 -- 172.21.15.86:0/405456821 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564db4691e30 msgr2=0x564db46f8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:45.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.720+0000 7faf3e200640 1 --2- 172.21.15.86:0/405456821 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x564db4691e30 0x564db46f8440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:45.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.720+0000 7faf3e200640 1 -- 172.21.15.86:0/405456821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db47047e0 msgr2=0x564db47210a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:45.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.720+0000 7faf3e200640 1 --2- 172.21.15.86:0/405456821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x564db47047e0 0x564db47210a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:45.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.720+0000 7faf3e200640 1 -- 172.21.15.86:0/405456821 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564db47215e0 con 0x564db4707ed0 2024-04-20T21:20:45.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.721+0000 7faf3e200640 1 --2- 172.21.15.86:0/405456821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564db4707ed0 0x564db46fe030 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7faf380021c0 tx=0x7faf3800c5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:45.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.721+0000 7faf36000640 1 -- 172.21.15.86:0/405456821 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf380062d0 con 0x564db4707ed0 2024-04-20T21:20:45.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.721+0000 7faf36000640 1 -- 172.21.15.86:0/405456821 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf38003450 con 0x564db4707ed0 2024-04-20T21:20:45.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.721+0000 7faf36000640 1 -- 172.21.15.86:0/405456821 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf38003860 con 0x564db4707ed0 2024-04-20T21:20:45.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.721+0000 7faf406d9f40 1 -- 172.21.15.86:0/405456821 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564db4679d40 con 0x564db4707ed0 2024-04-20T21:20:45.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.721+0000 7faf406d9f40 1 -- 172.21.15.86:0/405456821 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564db4677ef0 con 0x564db4707ed0 2024-04-20T21:20:45.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.723+0000 7faf36000640 1 -- 172.21.15.86:0/405456821 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faf3801a020 con 0x564db4707ed0 2024-04-20T21:20:45.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.724+0000 7faf36000640 1 --2- 172.21.15.86:0/405456821 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7faf2006b470 0x7faf2006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:45.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.724+0000 7faf36000640 1 -- 172.21.15.86:0/405456821 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7faf3808a5c0 con 0x564db4707ed0 2024-04-20T21:20:45.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.724+0000 7faf3d800640 1 --2- 172.21.15.86:0/405456821 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7faf2006b470 0x7faf2006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:45.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.724+0000 7faf406d9f40 1 --2- 172.21.15.86:0/405456821 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x564db46fe570 0x564db471d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:45.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.724+0000 7faf406d9f40 1 -- 172.21.15.86:0/405456821 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:1 7.7 7:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47) -- 0x564db46790b0 con 0x564db46fe570 2024-04-20T21:20:45.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.725+0000 7faf3d800640 1 --2- 172.21.15.86:0/405456821 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7faf2006b470 0x7faf2006d930 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7faf30054000 tx=0x7faf3004e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:45.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.725+0000 7faf3ce00640 1 --2- 172.21.15.86:0/405456821 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x564db46fe570 0x564db471d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:45.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.726+0000 7faf3ce00640 1 --2- 172.21.15.86:0/405456821 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x564db46fe570 0x564db471d200 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:45.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.727+0000 7faf3ce00640 1 -- 172.21.15.86:0/405456821 <== osd.6 v2:172.21.15.144:6824/1893803079 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv3 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x7faf2c00c950 con 0x564db46fe570 2024-04-20T21:20:45.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.727+0000 7faf406d9f40 1 -- 172.21.15.86:0/405456821 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x564db46fe570 msgr2=0x564db471d200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:45.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.727+0000 7faf406d9f40 1 --2- 172.21.15.86:0/405456821 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x564db46fe570 0x564db471d200 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:45.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.727+0000 7faf406d9f40 1 -- 172.21.15.86:0/405456821 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7faf2006b470 msgr2=0x7faf2006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:45.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.727+0000 7faf406d9f40 1 --2- 172.21.15.86:0/405456821 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7faf2006b470 0x7faf2006d930 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7faf30054000 tx=0x7faf3004e000 comp rx=0 tx=0).stop 2024-04-20T21:20:45.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.727+0000 7faf406d9f40 1 -- 172.21.15.86:0/405456821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564db4707ed0 msgr2=0x564db46fe030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:45.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.727+0000 7faf406d9f40 1 --2- 172.21.15.86:0/405456821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564db4707ed0 0x564db46fe030 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7faf380021c0 tx=0x7faf3800c5e0 comp rx=0 tx=0).stop 2024-04-20T21:20:45.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.727+0000 7faf3e200640 1 -- 172.21.15.86:0/405456821 reap_dead start 2024-04-20T21:20:45.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.728+0000 7faf406d9f40 1 -- 172.21.15.86:0/405456821 shutdown_connections 2024-04-20T21:20:45.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.728+0000 7faf406d9f40 1 -- 172.21.15.86:0/405456821 >> 172.21.15.86:0/405456821 conn(0x564db464c950 msgr2=0x564db4704e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:45.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.728+0000 7faf406d9f40 1 -- 172.21.15.86:0/405456821 shutdown_connections 2024-04-20T21:20:45.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:45.728+0000 7faf406d9f40 1 -- 172.21.15.86:0/405456821 wait complete. 2024-04-20T21:20:45.731 INFO:teuthology.orchestra.run.smithi086.stdout:Yalphaparent_a 2024-04-20T21:20:45.731 DEBUG:teuthology.orchestra.run.smithi086:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-20T21:20:45.820 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:20:45.820 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 1099511627777, 2024-04-20T21:20:45.820 INFO:teuthology.orchestra.run.smithi086.stdout: "ancestors": [ 2024-04-20T21:20:45.820 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:20:45.820 INFO:teuthology.orchestra.run.smithi086.stdout: "dirino": 1099511627776, 2024-04-20T21:20:45.820 INFO:teuthology.orchestra.run.smithi086.stdout: "dname": "alpha", 2024-04-20T21:20:45.820 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 4 2024-04-20T21:20:45.820 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:20:45.820 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:20:45.820 INFO:teuthology.orchestra.run.smithi086.stdout: "dirino": 1, 2024-04-20T21:20:45.820 INFO:teuthology.orchestra.run.smithi086.stdout: "dname": "parent_a", 2024-04-20T21:20:45.820 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 7 2024-04-20T21:20:45.820 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:20:45.821 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-20T21:20:45.821 INFO:teuthology.orchestra.run.smithi086.stdout: "pool": 7, 2024-04-20T21:20:45.821 INFO:teuthology.orchestra.run.smithi086.stdout: "old_pools": [] 2024-04-20T21:20:45.821 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:20:45.821 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:20:45.827 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-20T21:20:45.827 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T21:20:45.930 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T21:20:45.966 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:20:45.974 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:20:46.043 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-20T21:20:46.043 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T21:20:46.043 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:20:46.043 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:20:46.094 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:20:46.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.341+0000 7f1385a00640 1 Processor -- start 2024-04-20T21:20:46.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.341+0000 7f1385a00640 1 -- start start 2024-04-20T21:20:46.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.343+0000 7f1385a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1380144e90 0x7f138015ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:46.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.343+0000 7f1385a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1380071d00 0x7f1380072160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:46.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.343+0000 7f1385a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f138006dc30 con 0x7f138006f140 2024-04-20T21:20:46.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.343+0000 7f1385a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f13800726a0 con 0x7f1380071d00 2024-04-20T21:20:46.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.343+0000 7f1385a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1380145270 con 0x7f1380144e90 2024-04-20T21:20:46.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.343+0000 7f1376000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f138006f140 0x7f13800aeb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40388/0 (socket says 172.21.15.86:40388) 2024-04-20T21:20:46.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.343+0000 7f1376000640 1 -- 172.21.15.86:0/3443915434 learned_addr learned my addr 172.21.15.86:0/3443915434 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:20:46.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.344+0000 7f137f400640 1 --2- 172.21.15.86:0/3443915434 >> v2:172.21.15.144:3300/0 conn(0x7f1380071d00 0x7f1380072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:46.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.344+0000 7f137ea00640 1 --2- 172.21.15.86:0/3443915434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1380144e90 0x7f138015ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:46.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.344+0000 7f137e000640 1 -- 172.21.15.86:0/3443915434 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4285688365 0 0) 0x7f138006dc30 con 0x7f138006f140 2024-04-20T21:20:46.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.344+0000 7f137e000640 1 -- 172.21.15.86:0/3443915434 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1364003520 con 0x7f138006f140 2024-04-20T21:20:46.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.344+0000 7f137e000640 1 -- 172.21.15.86:0/3443915434 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3086738924 0 0) 0x7f1364003520 con 0x7f138006f140 2024-04-20T21:20:46.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.344+0000 7f137e000640 1 -- 172.21.15.86:0/3443915434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1380144e90 msgr2=0x7f138015ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:46.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.344+0000 7f137e000640 1 --2- 172.21.15.86:0/3443915434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1380144e90 0x7f138015ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:46.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.344+0000 7f137e000640 1 -- 172.21.15.86:0/3443915434 >> v2:172.21.15.144:3300/0 conn(0x7f1380071d00 msgr2=0x7f1380072160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:46.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.344+0000 7f137e000640 1 --2- 172.21.15.86:0/3443915434 >> v2:172.21.15.144:3300/0 conn(0x7f1380071d00 0x7f1380072160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:46.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.344+0000 7f137e000640 1 -- 172.21.15.86:0/3443915434 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f138015f720 con 0x7f138006f140 2024-04-20T21:20:46.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.344+0000 7f137e000640 1 -- 172.21.15.86:0/3443915434 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f1368001670 con 0x7f138006f140 2024-04-20T21:20:46.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.344+0000 7f137e000640 1 -- 172.21.15.86:0/3443915434 >> v1:172.21.15.86:6789/0 conn(0x7f138006f140 legacy=0x7f13800aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:46.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.344+0000 7f137e000640 1 --2- 172.21.15.86:0/3443915434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13640039b0 0x7f1364003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:46.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.345+0000 7f137e000640 1 --2- 172.21.15.86:0/3443915434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1364004800 0x7f1364004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:46.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.345+0000 7f137e000640 1 --2- 172.21.15.86:0/3443915434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1380144e90 0x7f13640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:46.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.345+0000 7f137e000640 1 -- 172.21.15.86:0/3443915434 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1380145270 con 0x7f1364004800 2024-04-20T21:20:46.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.345+0000 7f137e000640 1 -- 172.21.15.86:0/3443915434 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f136400b9c0 con 0x7f13640039b0 2024-04-20T21:20:46.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.345+0000 7f137e000640 1 -- 172.21.15.86:0/3443915434 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f136400bb40 con 0x7f1380144e90 2024-04-20T21:20:46.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.345+0000 7f1376000640 1 --2- 172.21.15.86:0/3443915434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1364004800 0x7f1364004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:46.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.345+0000 7f137ea00640 1 --2- 172.21.15.86:0/3443915434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13640039b0 0x7f1364003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:46.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.345+0000 7f137f400640 1 --2- 172.21.15.86:0/3443915434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1380144e90 0x7f13640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:46.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.345+0000 7f1376000640 1 -- 172.21.15.86:0/3443915434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1380144e90 msgr2=0x7f13640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:46.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.345+0000 7f1376000640 1 --2- 172.21.15.86:0/3443915434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1380144e90 0x7f13640051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:46.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.345+0000 7f1376000640 1 -- 172.21.15.86:0/3443915434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13640039b0 msgr2=0x7f1364003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:46.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.345+0000 7f1376000640 1 --2- 172.21.15.86:0/3443915434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13640039b0 0x7f1364003df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:46.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.345+0000 7f1376000640 1 -- 172.21.15.86:0/3443915434 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1364003520 con 0x7f1364004800 2024-04-20T21:20:46.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.346+0000 7f1376000640 1 --2- 172.21.15.86:0/3443915434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1364004800 0x7f1364004c80 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f136800c2c0 tx=0x7f1364005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c185b3e15bdc73a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:46.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.347+0000 7f137e000640 1 -- 172.21.15.86:0/3443915434 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1368007d50 con 0x7f1364004800 2024-04-20T21:20:46.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.347+0000 7f137e000640 1 -- 172.21.15.86:0/3443915434 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1368030070 con 0x7f1364004800 2024-04-20T21:20:46.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.347+0000 7f137e000640 1 -- 172.21.15.86:0/3443915434 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136802cd60 con 0x7f1364004800 2024-04-20T21:20:46.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.347+0000 7f1385a00640 1 -- 172.21.15.86:0/3443915434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1364004800 msgr2=0x7f1364004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:46.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.347+0000 7f1385a00640 1 --2- 172.21.15.86:0/3443915434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1364004800 0x7f1364004c80 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f136800c2c0 tx=0x7f1364005830 comp rx=0 tx=0).stop 2024-04-20T21:20:46.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.347+0000 7f137f400640 1 -- 172.21.15.86:0/3443915434 reap_dead start 2024-04-20T21:20:46.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.347+0000 7f1385a00640 1 -- 172.21.15.86:0/3443915434 shutdown_connections 2024-04-20T21:20:46.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.347+0000 7f1385a00640 1 -- 172.21.15.86:0/3443915434 >> 172.21.15.86:0/3443915434 conn(0x7f13800b2360 msgr2=0x7f13800b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:46.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.347+0000 7f1385a00640 1 -- 172.21.15.86:0/3443915434 shutdown_connections 2024-04-20T21:20:46.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.347+0000 7f1385a00640 1 -- 172.21.15.86:0/3443915434 wait complete. 2024-04-20T21:20:46.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.349+0000 7f1385a00640 1 Processor -- start 2024-04-20T21:20:46.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.349+0000 7f1385a00640 1 -- start start 2024-04-20T21:20:46.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.350+0000 7f1385a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1364004800 0x7f138013e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:46.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.350+0000 7f1385a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13801475a0 0x7f13801f4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:46.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.350+0000 7f1385a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13800cd5e0 0x7f1380160960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:46.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.350+0000 7f1385a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f138006ded0 con 0x7f1364004800 2024-04-20T21:20:46.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.350+0000 7f1385a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f138015fce0 con 0x7f13801475a0 2024-04-20T21:20:46.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.350+0000 7f1385a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f13800af280 con 0x7f13800cd5e0 2024-04-20T21:20:46.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.351+0000 7f137ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1364004800 0x7f138013e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:46.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.351+0000 7f137ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1364004800 0x7f138013e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39498/0 (socket says 172.21.15.86:39498) 2024-04-20T21:20:46.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.351+0000 7f137ea00640 1 -- 172.21.15.86:0/1784087407 learned_addr learned my addr 172.21.15.86:0/1784087407 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:46.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.351+0000 7f1376000640 1 --2- 172.21.15.86:0/1784087407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13801475a0 0x7f13801f4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:46.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.351+0000 7f137ea00640 1 -- 172.21.15.86:0/1784087407 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13800cd5e0 msgr2=0x7f1380160960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:46.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.351+0000 7f137ea00640 1 --2- 172.21.15.86:0/1784087407 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13800cd5e0 0x7f1380160960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:46.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.351+0000 7f137ea00640 1 -- 172.21.15.86:0/1784087407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13801475a0 msgr2=0x7f13801f4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:46.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.351+0000 7f137ea00640 1 --2- 172.21.15.86:0/1784087407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13801475a0 0x7f13801f4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:46.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.351+0000 7f137ea00640 1 -- 172.21.15.86:0/1784087407 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1380147a80 con 0x7f1364004800 2024-04-20T21:20:46.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.351+0000 7f137ea00640 1 --2- 172.21.15.86:0/1784087407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1364004800 0x7f138013e950 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f1370010860 tx=0x7f1370010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:46.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.352+0000 7f1377400640 1 -- 172.21.15.86:0/1784087407 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137000f040 con 0x7f1364004800 2024-04-20T21:20:46.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.352+0000 7f1377400640 1 -- 172.21.15.86:0/1784087407 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f137000eae0 con 0x7f1364004800 2024-04-20T21:20:46.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.352+0000 7f1377400640 1 -- 172.21.15.86:0/1784087407 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137001c400 con 0x7f1364004800 2024-04-20T21:20:46.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.352+0000 7f1385a00640 1 -- 172.21.15.86:0/1784087407 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1380147d60 con 0x7f1364004800 2024-04-20T21:20:46.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.352+0000 7f1385a00640 1 -- 172.21.15.86:0/1784087407 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1380148320 con 0x7f1364004800 2024-04-20T21:20:46.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.353+0000 7f1385a00640 1 -- 172.21.15.86:0/1784087407 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13800689e0 con 0x7f1364004800 2024-04-20T21:20:46.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.354+0000 7f1377400640 1 -- 172.21.15.86:0/1784087407 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f137001c5a0 con 0x7f1364004800 2024-04-20T21:20:46.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.355+0000 7f1377400640 1 --2- 172.21.15.86:0/1784087407 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f135c06c8f0 0x7f135c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:46.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.355+0000 7f1377400640 1 -- 172.21.15.86:0/1784087407 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f137008c030 con 0x7f1364004800 2024-04-20T21:20:46.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.355+0000 7f1376000640 1 --2- 172.21.15.86:0/1784087407 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f135c06c8f0 0x7f135c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:46.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.356+0000 7f1376000640 1 --2- 172.21.15.86:0/1784087407 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f135c06c8f0 0x7f135c06edb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1368034000 tx=0x7f1368007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:46.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.360+0000 7f1377400640 1 -- 172.21.15.86:0/1784087407 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1370018030 con 0x7f1364004800 2024-04-20T21:20:46.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.566+0000 7f1385a00640 1 -- 172.21.15.86:0/1784087407 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f13800add90 con 0x7f1364004800 2024-04-20T21:20:46.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.568+0000 7f1377400640 1 -- 172.21.15.86:0/1784087407 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16704 (secure 0 0 0) 0x7f137005a4b0 con 0x7f1364004800 2024-04-20T21:20:46.568 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:20:46.568 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":47,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:20:34.772220+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:20:01.307757+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-20T21:20:02.359347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.6","mappings":[{"from":0,"to":4}]},{"pgid":"7.36","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:20:46.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.571+0000 7f1385a00640 1 -- 172.21.15.86:0/1784087407 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f135c06c8f0 msgr2=0x7f135c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:46.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.571+0000 7f1385a00640 1 --2- 172.21.15.86:0/1784087407 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f135c06c8f0 0x7f135c06edb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1368034000 tx=0x7f1368007000 comp rx=0 tx=0).stop 2024-04-20T21:20:46.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.571+0000 7f1385a00640 1 -- 172.21.15.86:0/1784087407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1364004800 msgr2=0x7f138013e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:46.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.571+0000 7f1385a00640 1 --2- 172.21.15.86:0/1784087407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1364004800 0x7f138013e950 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f1370010860 tx=0x7f1370010d20 comp rx=0 tx=0).stop 2024-04-20T21:20:46.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.571+0000 7f1385a00640 1 -- 172.21.15.86:0/1784087407 shutdown_connections 2024-04-20T21:20:46.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.571+0000 7f1385a00640 1 --2- 172.21.15.86:0/1784087407 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f135c06c8f0 0x7f135c06edb0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:46.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.571+0000 7f1385a00640 1 --2- 172.21.15.86:0/1784087407 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13800cd5e0 0x7f1380160960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:46.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.571+0000 7f1385a00640 1 --2- 172.21.15.86:0/1784087407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13801475a0 0x7f13801f4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:46.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.572+0000 7f1385a00640 1 --2- 172.21.15.86:0/1784087407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1364004800 0x7f138013e950 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:46.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.572+0000 7f1385a00640 1 -- 172.21.15.86:0/1784087407 >> 172.21.15.86:0/1784087407 conn(0x7f13800b2360 msgr2=0x7f1380160000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:46.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.572+0000 7f1385a00640 1 -- 172.21.15.86:0/1784087407 shutdown_connections 2024-04-20T21:20:46.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.572+0000 7f1385a00640 1 -- 172.21.15.86:0/1784087407 wait complete. 2024-04-20T21:20:46.590 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:20:01.307757+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:20:02.359347+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:20:46.591 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:20:46.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.796+0000 7ff90ce00640 1 Processor -- start 2024-04-20T21:20:46.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.796+0000 7ff90ce00640 1 -- start start 2024-04-20T21:20:46.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.798+0000 7ff90ce00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff908071db0 0x7ff908072210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:46.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.798+0000 7ff90ce00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9080b4bf0 con 0x7ff9080b3ec0 2024-04-20T21:20:46.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.798+0000 7ff90ce00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9080b4ed0 con 0x7ff908158950 2024-04-20T21:20:46.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.798+0000 7ff90ce00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff90806c090 con 0x7ff908071db0 2024-04-20T21:20:46.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.799+0000 7ff906800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7ff9080b3ec0 0x7ff9080b4330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40394/0 (socket says 172.21.15.86:40394) 2024-04-20T21:20:46.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.799+0000 7ff906800640 1 -- 172.21.15.86:0/1785430166 learned_addr learned my addr 172.21.15.86:0/1785430166 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:20:46.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.799+0000 7ff905400640 1 --2- 172.21.15.86:0/1785430166 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff908071db0 0x7ff908072210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:46.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.799+0000 7ff904a00640 1 -- 172.21.15.86:0/1785430166 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 728802749 0 0) 0x7ff9080b4bf0 con 0x7ff9080b3ec0 2024-04-20T21:20:46.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.799+0000 7ff904a00640 1 -- 172.21.15.86:0/1785430166 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff8ec003520 con 0x7ff9080b3ec0 2024-04-20T21:20:46.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.799+0000 7ff904a00640 1 -- 172.21.15.86:0/1785430166 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1611420448 0 0) 0x7ff8ec003520 con 0x7ff9080b3ec0 2024-04-20T21:20:46.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.799+0000 7ff904a00640 1 -- 172.21.15.86:0/1785430166 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff908071db0 msgr2=0x7ff908072210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:46.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.799+0000 7ff904a00640 1 --2- 172.21.15.86:0/1785430166 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff908071db0 0x7ff908072210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:46.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.800+0000 7ff904a00640 1 -- 172.21.15.86:0/1785430166 >> v1:172.21.15.144:6789/0 conn(0x7ff908158950 legacy=0x7ff90806b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:46.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.800+0000 7ff904a00640 1 -- 172.21.15.86:0/1785430166 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff90806de90 con 0x7ff9080b3ec0 2024-04-20T21:20:46.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.800+0000 7ff904a00640 1 -- 172.21.15.86:0/1785430166 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7ff9000019f0 con 0x7ff9080b3ec0 2024-04-20T21:20:46.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.800+0000 7ff904a00640 1 -- 172.21.15.86:0/1785430166 >> v1:172.21.15.86:6789/0 conn(0x7ff9080b3ec0 legacy=0x7ff9080b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:46.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.800+0000 7ff904a00640 1 --2- 172.21.15.86:0/1785430166 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff908071db0 0x7ff8ec0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:46.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.800+0000 7ff904a00640 1 --2- 172.21.15.86:0/1785430166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8ec003fb0 0x7ff8ec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:46.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.800+0000 7ff904a00640 1 --2- 172.21.15.86:0/1785430166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8ec004d40 0x7ff8ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:46.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.800+0000 7ff904a00640 1 -- 172.21.15.86:0/1785430166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff90806c090 con 0x7ff8ec003fb0 2024-04-20T21:20:46.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.800+0000 7ff904a00640 1 -- 172.21.15.86:0/1785430166 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec00dd10 con 0x7ff8ec004d40 2024-04-20T21:20:46.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.800+0000 7ff904a00640 1 -- 172.21.15.86:0/1785430166 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff8ec00df10 con 0x7ff908071db0 2024-04-20T21:20:46.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.800+0000 7ff905400640 1 --2- 172.21.15.86:0/1785430166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8ec003fb0 0x7ff8ec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:46.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.800+0000 7ff905e00640 1 --2- 172.21.15.86:0/1785430166 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff908071db0 0x7ff8ec0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:46.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.800+0000 7ff905400640 1 -- 172.21.15.86:0/1785430166 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff908071db0 msgr2=0x7ff8ec0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:46.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.800+0000 7ff905400640 1 --2- 172.21.15.86:0/1785430166 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff908071db0 0x7ff8ec0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:46.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.800+0000 7ff905400640 1 -- 172.21.15.86:0/1785430166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8ec004d40 msgr2=0x7ff8ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:46.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.800+0000 7ff906800640 1 --2- 172.21.15.86:0/1785430166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8ec004d40 0x7ff8ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:46.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.801+0000 7ff905400640 1 --2- 172.21.15.86:0/1785430166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8ec004d40 0x7ff8ec0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:46.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.801+0000 7ff905400640 1 -- 172.21.15.86:0/1785430166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8ec003520 con 0x7ff8ec003fb0 2024-04-20T21:20:46.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.801+0000 7ff905e00640 1 --2- 172.21.15.86:0/1785430166 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff908071db0 0x7ff8ec0039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:20:46.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.802+0000 7ff905400640 1 --2- 172.21.15.86:0/1785430166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8ec003fb0 0x7ff8ec004800 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4009ac0 tx=0x7ff8ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c21b283c0250ba server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:46.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.802+0000 7ff904a00640 1 -- 172.21.15.86:0/1785430166 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f4039070 con 0x7ff8ec003fb0 2024-04-20T21:20:46.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.802+0000 7ff904a00640 1 -- 172.21.15.86:0/1785430166 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8f402c600 con 0x7ff8ec003fb0 2024-04-20T21:20:46.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.802+0000 7ff904a00640 1 -- 172.21.15.86:0/1785430166 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f4009690 con 0x7ff8ec003fb0 2024-04-20T21:20:46.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.802+0000 7ff90ce00640 1 -- 172.21.15.86:0/1785430166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8ec003fb0 msgr2=0x7ff8ec004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:46.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.802+0000 7ff90ce00640 1 --2- 172.21.15.86:0/1785430166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8ec003fb0 0x7ff8ec004800 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4009ac0 tx=0x7ff8ec005830 comp rx=0 tx=0).stop 2024-04-20T21:20:46.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.803+0000 7ff906800640 1 -- 172.21.15.86:0/1785430166 reap_dead start 2024-04-20T21:20:46.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.803+0000 7ff90ce00640 1 -- 172.21.15.86:0/1785430166 shutdown_connections 2024-04-20T21:20:46.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.803+0000 7ff90ce00640 1 -- 172.21.15.86:0/1785430166 >> 172.21.15.86:0/1785430166 conn(0x7ff9080b2110 msgr2=0x7ff9080b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:46.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.803+0000 7ff90ce00640 1 -- 172.21.15.86:0/1785430166 shutdown_connections 2024-04-20T21:20:46.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.803+0000 7ff90ce00640 1 -- 172.21.15.86:0/1785430166 wait complete. 2024-04-20T21:20:46.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.804+0000 7ff90ce00640 1 Processor -- start 2024-04-20T21:20:46.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.805+0000 7ff90ce00640 1 -- start start 2024-04-20T21:20:46.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.805+0000 7ff90ce00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8ec003fb0 0x7ff90814b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:46.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.806+0000 7ff90ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff908148da0 0x7ff9081496f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:46.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.806+0000 7ff90ce00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff908149c30 0x7ff9080ae010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:46.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.806+0000 7ff90ce00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff908158d30 con 0x7ff908148da0 2024-04-20T21:20:46.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.806+0000 7ff90ce00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff90806e450 con 0x7ff908149c30 2024-04-20T21:20:46.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.806+0000 7ff90ce00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff908158eb0 con 0x7ff8ec003fb0 2024-04-20T21:20:46.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.806+0000 7ff905400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff908148da0 0x7ff9081496f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:46.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.806+0000 7ff905400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff908148da0 0x7ff9081496f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39524/0 (socket says 172.21.15.86:39524) 2024-04-20T21:20:46.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.806+0000 7ff905400640 1 -- 172.21.15.86:0/1224808058 learned_addr learned my addr 172.21.15.86:0/1224808058 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:46.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.806+0000 7ff905e00640 1 --2- 172.21.15.86:0/1224808058 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8ec003fb0 0x7ff90814b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:46.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.806+0000 7ff906800640 1 --2- 172.21.15.86:0/1224808058 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff908149c30 0x7ff9080ae010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:46.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.806+0000 7ff905400640 1 -- 172.21.15.86:0/1224808058 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8ec003fb0 msgr2=0x7ff90814b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:46.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.806+0000 7ff905400640 1 --2- 172.21.15.86:0/1224808058 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8ec003fb0 0x7ff90814b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:46.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.806+0000 7ff905400640 1 -- 172.21.15.86:0/1224808058 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff908149c30 msgr2=0x7ff9080ae010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:46.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.806+0000 7ff905400640 1 --2- 172.21.15.86:0/1224808058 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff908149c30 0x7ff9080ae010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:46.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.806+0000 7ff905400640 1 -- 172.21.15.86:0/1224808058 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff90814a0d0 con 0x7ff908148da0 2024-04-20T21:20:46.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.806+0000 7ff905e00640 1 --2- 172.21.15.86:0/1224808058 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8ec003fb0 0x7ff90814b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:20:46.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.807+0000 7ff905400640 1 --2- 172.21.15.86:0/1224808058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff908148da0 0x7ff9081496f0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4000c00 tx=0x7ff8f4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:46.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.807+0000 7ff8fe000640 1 -- 172.21.15.86:0/1224808058 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f4042020 con 0x7ff908148da0 2024-04-20T21:20:46.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.807+0000 7ff8fe000640 1 -- 172.21.15.86:0/1224808058 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8f403a040 con 0x7ff908148da0 2024-04-20T21:20:46.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.807+0000 7ff8fe000640 1 -- 172.21.15.86:0/1224808058 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f4039070 con 0x7ff908148da0 2024-04-20T21:20:46.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.807+0000 7ff90ce00640 1 -- 172.21.15.86:0/1224808058 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff90814a3b0 con 0x7ff908148da0 2024-04-20T21:20:46.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.807+0000 7ff90ce00640 1 -- 172.21.15.86:0/1224808058 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff90814a970 con 0x7ff908148da0 2024-04-20T21:20:46.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.808+0000 7ff90ce00640 1 -- 172.21.15.86:0/1224808058 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9080689e0 con 0x7ff908148da0 2024-04-20T21:20:46.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.809+0000 7ff8fe000640 1 -- 172.21.15.86:0/1224808058 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff8f4045020 con 0x7ff908148da0 2024-04-20T21:20:46.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.810+0000 7ff8fe000640 1 --2- 172.21.15.86:0/1224808058 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff8e806c8f0 0x7ff8e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:46.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.810+0000 7ff8fe000640 1 -- 172.21.15.86:0/1224808058 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7ff8f40ad100 con 0x7ff908148da0 2024-04-20T21:20:46.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.810+0000 7ff905e00640 1 --2- 172.21.15.86:0/1224808058 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff8e806c8f0 0x7ff8e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:46.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.811+0000 7ff905e00640 1 --2- 172.21.15.86:0/1224808058 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff8e806c8f0 0x7ff8e806edb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff8f80016f0 tx=0x7ff8f8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:46.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:46.815+0000 7ff8fe000640 1 -- 172.21.15.86:0/1224808058 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff8f404c070 con 0x7ff908148da0 2024-04-20T21:20:47.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.015+0000 7ff90ce00640 1 -- 172.21.15.86:0/1224808058 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff9080729b0 con 0x7ff908148da0 2024-04-20T21:20:47.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.016+0000 7ff8fe000640 1 -- 172.21.15.86:0/1224808058 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) ==== 93+0+10 (secure 0 0 0) 0x7ff8f407b450 con 0x7ff908148da0 2024-04-20T21:20:47.015 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:20:47.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.018+0000 7ff90ce00640 1 -- 172.21.15.86:0/1224808058 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff8e806c8f0 msgr2=0x7ff8e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:47.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.018+0000 7ff90ce00640 1 --2- 172.21.15.86:0/1224808058 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff8e806c8f0 0x7ff8e806edb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff8f80016f0 tx=0x7ff8f8009040 comp rx=0 tx=0).stop 2024-04-20T21:20:47.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.018+0000 7ff90ce00640 1 -- 172.21.15.86:0/1224808058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff908148da0 msgr2=0x7ff9081496f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:47.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.018+0000 7ff90ce00640 1 --2- 172.21.15.86:0/1224808058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff908148da0 0x7ff9081496f0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4000c00 tx=0x7ff8f4004730 comp rx=0 tx=0).stop 2024-04-20T21:20:47.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.018+0000 7ff90ce00640 1 -- 172.21.15.86:0/1224808058 shutdown_connections 2024-04-20T21:20:47.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.018+0000 7ff90ce00640 1 --2- 172.21.15.86:0/1224808058 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff8e806c8f0 0x7ff8e806edb0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:47.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.018+0000 7ff90ce00640 1 --2- 172.21.15.86:0/1224808058 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff908149c30 0x7ff9080ae010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:47.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.018+0000 7ff90ce00640 1 --2- 172.21.15.86:0/1224808058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff908148da0 0x7ff9081496f0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:47.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.018+0000 7ff90ce00640 1 --2- 172.21.15.86:0/1224808058 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8ec003fb0 0x7ff90814b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:47.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.018+0000 7ff90ce00640 1 -- 172.21.15.86:0/1224808058 >> 172.21.15.86:0/1224808058 conn(0x7ff9080b2110 msgr2=0x7ff9080b3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:47.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.019+0000 7ff90ce00640 1 -- 172.21.15.86:0/1224808058 shutdown_connections 2024-04-20T21:20:47.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.019+0000 7ff90ce00640 1 -- 172.21.15.86:0/1224808058 wait complete. 2024-04-20T21:20:47.039 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:20:47.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.239+0000 7f5120200640 1 Processor -- start 2024-04-20T21:20:47.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.239+0000 7f5120200640 1 -- start start 2024-04-20T21:20:47.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.240+0000 7f5120200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f51180b4530 0x7f5118169550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:47.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.240+0000 7f5120200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51180adb10 con 0x7f511815eb30 2024-04-20T21:20:47.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.240+0000 7f5120200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51180addb0 con 0x7f51180aee50 2024-04-20T21:20:47.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.240+0000 7f5120200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f51180af7d0 con 0x7f51180b4530 2024-04-20T21:20:47.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.241+0000 7f511e200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f51180b4530 0x7f5118169550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:47.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.241+0000 7f511e200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f51180b4530 0x7f5118169550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39540/0 (socket says 172.21.15.86:39540) 2024-04-20T21:20:47.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.241+0000 7f511e200640 1 -- 172.21.15.86:0/1248717778 learned_addr learned my addr 172.21.15.86:0/1248717778 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:47.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.241+0000 7f511d800640 1 --1- 172.21.15.86:0/1248717778 >> v1:172.21.15.86:6789/0 conn(0x7f511815eb30 0x7f51180b3e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40408/0 (socket says 172.21.15.86:40408) 2024-04-20T21:20:47.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.241+0000 7f511e200640 1 -- 172.21.15.86:0/1248717778 >> v1:172.21.15.144:6789/0 conn(0x7f51180aee50 legacy=0x7f511806b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:47.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.241+0000 7f511e200640 1 -- 172.21.15.86:0/1248717778 >> v1:172.21.15.86:6789/0 conn(0x7f511815eb30 legacy=0x7f51180b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:47.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.241+0000 7f511e200640 1 -- 172.21.15.86:0/1248717778 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51180ae090 con 0x7f51180b4530 2024-04-20T21:20:47.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.242+0000 7f511e200640 1 --2- 172.21.15.86:0/1248717778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51180b4530 0x7f5118169550 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f5110009e50 tx=0x7f511002bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4659554c0c7ae16 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:47.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.243+0000 7f5117e00640 1 -- 172.21.15.86:0/1248717778 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511002cb10 con 0x7f51180b4530 2024-04-20T21:20:47.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.243+0000 7f5117e00640 1 -- 172.21.15.86:0/1248717778 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f511002ccb0 con 0x7f51180b4530 2024-04-20T21:20:47.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.243+0000 7f5117e00640 1 -- 172.21.15.86:0/1248717778 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511002ce60 con 0x7f51180b4530 2024-04-20T21:20:47.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.243+0000 7f5120200640 1 -- 172.21.15.86:0/1248717778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51180b4530 msgr2=0x7f5118169550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:47.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.243+0000 7f5120200640 1 --2- 172.21.15.86:0/1248717778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51180b4530 0x7f5118169550 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f5110009e50 tx=0x7f511002bf70 comp rx=0 tx=0).stop 2024-04-20T21:20:47.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.243+0000 7f5120200640 1 -- 172.21.15.86:0/1248717778 shutdown_connections 2024-04-20T21:20:47.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.243+0000 7f5120200640 1 --2- 172.21.15.86:0/1248717778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51180b4530 0x7f5118169550 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:47.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.243+0000 7f5120200640 1 -- 172.21.15.86:0/1248717778 >> 172.21.15.86:0/1248717778 conn(0x7f51180b2110 msgr2=0x7f51180b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:47.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.243+0000 7f5120200640 1 -- 172.21.15.86:0/1248717778 shutdown_connections 2024-04-20T21:20:47.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.243+0000 7f5120200640 1 -- 172.21.15.86:0/1248717778 wait complete. 2024-04-20T21:20:47.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.245+0000 7f5120200640 1 Processor -- start 2024-04-20T21:20:47.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.245+0000 7f5120200640 1 -- start start 2024-04-20T21:20:47.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.246+0000 7f5120200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f51180aee50 0x7f5118145250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:47.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.246+0000 7f5120200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51180b4530 0x7f51181f8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:47.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.246+0000 7f5120200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f511815eb30 0x7f5118157e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:47.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.246+0000 7f5120200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f51180ae930 con 0x7f51180b4530 2024-04-20T21:20:47.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.246+0000 7f5120200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f51180ae650 con 0x7f511815eb30 2024-04-20T21:20:47.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.246+0000 7f5120200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f511806c090 con 0x7f51180aee50 2024-04-20T21:20:47.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.246+0000 7f511ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51180b4530 0x7f51181f8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:47.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.246+0000 7f511ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51180b4530 0x7f51181f8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39552/0 (socket says 172.21.15.86:39552) 2024-04-20T21:20:47.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.246+0000 7f511ce00640 1 -- 172.21.15.86:0/377759722 learned_addr learned my addr 172.21.15.86:0/377759722 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:47.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.247+0000 7f511d800640 1 --2- 172.21.15.86:0/377759722 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f51180aee50 0x7f5118145250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:47.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.247+0000 7f511e200640 1 --2- 172.21.15.86:0/377759722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f511815eb30 0x7f5118157e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:47.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.247+0000 7f511ce00640 1 -- 172.21.15.86:0/377759722 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f51180aee50 msgr2=0x7f5118145250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:47.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.247+0000 7f511ce00640 1 --2- 172.21.15.86:0/377759722 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f51180aee50 0x7f5118145250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:47.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.247+0000 7f511ce00640 1 -- 172.21.15.86:0/377759722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f511815eb30 msgr2=0x7f5118157e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:47.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.247+0000 7f511ce00640 1 --2- 172.21.15.86:0/377759722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f511815eb30 0x7f5118157e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:47.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.247+0000 7f511ce00640 1 -- 172.21.15.86:0/377759722 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5118143fa0 con 0x7f51180b4530 2024-04-20T21:20:47.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.247+0000 7f511e200640 1 --2- 172.21.15.86:0/377759722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f511815eb30 0x7f5118157e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:20:47.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.247+0000 7f511ce00640 1 --2- 172.21.15.86:0/377759722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51180b4530 0x7f51181f8300 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f510800c430 tx=0x7f510800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:47.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.247+0000 7f5115600640 1 -- 172.21.15.86:0/377759722 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5108011790 con 0x7f51180b4530 2024-04-20T21:20:47.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.247+0000 7f5115600640 1 -- 172.21.15.86:0/377759722 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5108017070 con 0x7f51180b4530 2024-04-20T21:20:47.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.247+0000 7f5115600640 1 -- 172.21.15.86:0/377759722 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5108011bb0 con 0x7f51180b4530 2024-04-20T21:20:47.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.248+0000 7f5120200640 1 -- 172.21.15.86:0/377759722 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5118142f00 con 0x7f51180b4530 2024-04-20T21:20:47.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.248+0000 7f5120200640 1 -- 172.21.15.86:0/377759722 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51181434c0 con 0x7f51180b4530 2024-04-20T21:20:47.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.249+0000 7f5120200640 1 -- 172.21.15.86:0/377759722 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51180689e0 con 0x7f51180b4530 2024-04-20T21:20:47.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.250+0000 7f5115600640 1 -- 172.21.15.86:0/377759722 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f510801c460 con 0x7f51180b4530 2024-04-20T21:20:47.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.251+0000 7f5115600640 1 --2- 172.21.15.86:0/377759722 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f50fc06c8f0 0x7f50fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:47.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.251+0000 7f5115600640 1 -- 172.21.15.86:0/377759722 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f510808bad0 con 0x7f51180b4530 2024-04-20T21:20:47.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.251+0000 7f511d800640 1 --2- 172.21.15.86:0/377759722 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f50fc06c8f0 0x7f50fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:47.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.252+0000 7f511d800640 1 --2- 172.21.15.86:0/377759722 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f50fc06c8f0 0x7f50fc06edb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f510c000f60 tx=0x7f510c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:47.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.256+0000 7f5115600640 1 -- 172.21.15.86:0/377759722 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5108059e20 con 0x7f51180b4530 2024-04-20T21:20:47.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.455+0000 7f5120200640 1 -- 172.21.15.86:0/377759722 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5118141a40 con 0x7f51180b4530 2024-04-20T21:20:47.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.456+0000 7f5115600640 1 -- 172.21.15.86:0/377759722 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v47) ==== 104+0+11 (secure 0 0 0) 0x7f5108059720 con 0x7f51180b4530 2024-04-20T21:20:47.456 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-20T21:20:47.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.459+0000 7f5120200640 1 -- 172.21.15.86:0/377759722 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f50fc06c8f0 msgr2=0x7f50fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:47.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.459+0000 7f5120200640 1 --2- 172.21.15.86:0/377759722 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f50fc06c8f0 0x7f50fc06edb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f510c000f60 tx=0x7f510c009040 comp rx=0 tx=0).stop 2024-04-20T21:20:47.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.459+0000 7f5120200640 1 -- 172.21.15.86:0/377759722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51180b4530 msgr2=0x7f51181f8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:47.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.459+0000 7f5120200640 1 --2- 172.21.15.86:0/377759722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51180b4530 0x7f51181f8300 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f510800c430 tx=0x7f510800c8f0 comp rx=0 tx=0).stop 2024-04-20T21:20:47.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.459+0000 7f5120200640 1 -- 172.21.15.86:0/377759722 shutdown_connections 2024-04-20T21:20:47.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.459+0000 7f5120200640 1 --2- 172.21.15.86:0/377759722 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f50fc06c8f0 0x7f50fc06edb0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:47.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.459+0000 7f5120200640 1 --2- 172.21.15.86:0/377759722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f511815eb30 0x7f5118157e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:47.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.459+0000 7f5120200640 1 --2- 172.21.15.86:0/377759722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51180b4530 0x7f51181f8300 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:47.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.459+0000 7f5120200640 1 --2- 172.21.15.86:0/377759722 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f51180aee50 0x7f5118145250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:47.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.459+0000 7f5120200640 1 -- 172.21.15.86:0/377759722 >> 172.21.15.86:0/377759722 conn(0x7f51180b2110 msgr2=0x7f5118141f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:47.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.460+0000 7f5120200640 1 -- 172.21.15.86:0/377759722 shutdown_connections 2024-04-20T21:20:47.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.460+0000 7f5120200640 1 -- 172.21.15.86:0/377759722 wait complete. 2024-04-20T21:20:47.476 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:20:47.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.679+0000 7f3726600640 1 Processor -- start 2024-04-20T21:20:47.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.680+0000 7f3726600640 1 -- start start 2024-04-20T21:20:47.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.681+0000 7f3726600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3720155d30 0x7f37200cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:47.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.681+0000 7f3726600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f37200b52b0 0x7f372006be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:47.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.681+0000 7f3726600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3720156110 con 0x7f372006b980 2024-04-20T21:20:47.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.681+0000 7f3726600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f37200b56e0 con 0x7f3720155d30 2024-04-20T21:20:47.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.681+0000 7f3726600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f37200ce460 con 0x7f37200b52b0 2024-04-20T21:20:47.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.682+0000 7f371f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3720155d30 0x7f37200cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:47.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.682+0000 7f371f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3720155d30 0x7f37200cf500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39558/0 (socket says 172.21.15.86:39558) 2024-04-20T21:20:47.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.682+0000 7f371f400640 1 -- 172.21.15.86:0/1646091962 learned_addr learned my addr 172.21.15.86:0/1646091962 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:47.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.682+0000 7f371fe00640 1 --2- 172.21.15.86:0/1646091962 >> v2:172.21.15.144:3300/0 conn(0x7f37200b52b0 0x7f372006be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:47.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.682+0000 7f371f400640 1 -- 172.21.15.86:0/1646091962 >> v2:172.21.15.144:3300/0 conn(0x7f37200b52b0 msgr2=0x7f372006be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:47.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.682+0000 7f371f400640 1 --2- 172.21.15.86:0/1646091962 >> v2:172.21.15.144:3300/0 conn(0x7f37200b52b0 0x7f372006be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:47.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.682+0000 7f371f400640 1 -- 172.21.15.86:0/1646091962 >> v1:172.21.15.144:6789/0 conn(0x7f372006b980 legacy=0x7f37200b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:47.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.682+0000 7f371f400640 1 -- 172.21.15.86:0/1646091962 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37200ce740 con 0x7f3720155d30 2024-04-20T21:20:47.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.683+0000 7f371f400640 1 --2- 172.21.15.86:0/1646091962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3720155d30 0x7f37200cf500 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f37200b3070 tx=0x7f371002b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8be8fdf8ab53fba server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:47.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.684+0000 7f371ea00640 1 -- 172.21.15.86:0/1646091962 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3710038070 con 0x7f3720155d30 2024-04-20T21:20:47.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.684+0000 7f371ea00640 1 -- 172.21.15.86:0/1646091962 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f371002f050 con 0x7f3720155d30 2024-04-20T21:20:47.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.684+0000 7f371ea00640 1 -- 172.21.15.86:0/1646091962 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3710031970 con 0x7f3720155d30 2024-04-20T21:20:47.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.684+0000 7f3726600640 1 -- 172.21.15.86:0/1646091962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3720155d30 msgr2=0x7f37200cf500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:47.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.684+0000 7f3726600640 1 --2- 172.21.15.86:0/1646091962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3720155d30 0x7f37200cf500 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f37200b3070 tx=0x7f371002b7f0 comp rx=0 tx=0).stop 2024-04-20T21:20:47.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.685+0000 7f3726600640 1 -- 172.21.15.86:0/1646091962 shutdown_connections 2024-04-20T21:20:47.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.685+0000 7f3726600640 1 --2- 172.21.15.86:0/1646091962 >> v2:172.21.15.144:3300/0 conn(0x7f37200b52b0 0x7f372006be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:47.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.685+0000 7f3726600640 1 --2- 172.21.15.86:0/1646091962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3720155d30 0x7f37200cf500 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:47.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.685+0000 7f3726600640 1 -- 172.21.15.86:0/1646091962 >> 172.21.15.86:0/1646091962 conn(0x7f37200b2110 msgr2=0x7f37200b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:47.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.685+0000 7f3726600640 1 -- 172.21.15.86:0/1646091962 shutdown_connections 2024-04-20T21:20:47.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.685+0000 7f3726600640 1 -- 172.21.15.86:0/1646091962 wait complete. 2024-04-20T21:20:47.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.686+0000 7f3726600640 1 Processor -- start 2024-04-20T21:20:47.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.687+0000 7f3726600640 1 -- start start 2024-04-20T21:20:47.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.688+0000 7f3726600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f372006b980 0x7f372015cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:47.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.688+0000 7f3726600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37200b52b0 0x7f37201f4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:47.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.688+0000 7f3726600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3720155d30 0x7f3720148ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:47.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.688+0000 7f3726600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f37200cefe0 con 0x7f3720155d30 2024-04-20T21:20:47.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.688+0000 7f3726600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f37200ced00 con 0x7f37200b52b0 2024-04-20T21:20:47.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.688+0000 7f3726600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f372006ddf0 con 0x7f372006b980 2024-04-20T21:20:47.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.688+0000 7f371fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3720155d30 0x7f3720148ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:47.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.688+0000 7f371fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3720155d30 0x7f3720148ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39566/0 (socket says 172.21.15.86:39566) 2024-04-20T21:20:47.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.688+0000 7f371fe00640 1 -- 172.21.15.86:0/92016297 learned_addr learned my addr 172.21.15.86:0/92016297 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:47.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.688+0000 7f371f400640 1 --2- 172.21.15.86:0/92016297 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f372006b980 0x7f372015cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:47.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.688+0000 7f3717e00640 1 --2- 172.21.15.86:0/92016297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37200b52b0 0x7f37201f4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:47.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.688+0000 7f371fe00640 1 -- 172.21.15.86:0/92016297 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f372006b980 msgr2=0x7f372015cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:47.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.688+0000 7f371fe00640 1 --2- 172.21.15.86:0/92016297 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f372006b980 0x7f372015cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:47.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.688+0000 7f371fe00640 1 -- 172.21.15.86:0/92016297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37200b52b0 msgr2=0x7f37201f4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:47.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.688+0000 7f371fe00640 1 --2- 172.21.15.86:0/92016297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37200b52b0 0x7f37201f4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:47.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.688+0000 7f371fe00640 1 -- 172.21.15.86:0/92016297 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37201592f0 con 0x7f3720155d30 2024-04-20T21:20:47.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.688+0000 7f371f400640 1 --2- 172.21.15.86:0/92016297 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f372006b980 0x7f372015cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:20:47.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.689+0000 7f371fe00640 1 --2- 172.21.15.86:0/92016297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3720155d30 0x7f3720148ed0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f371800cd70 tx=0x7f371800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:47.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.689+0000 7f3717400640 1 -- 172.21.15.86:0/92016297 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3718003b90 con 0x7f3720155d30 2024-04-20T21:20:47.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.689+0000 7f3717400640 1 -- 172.21.15.86:0/92016297 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3718003d30 con 0x7f3720155d30 2024-04-20T21:20:47.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.689+0000 7f3717400640 1 -- 172.21.15.86:0/92016297 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3718004240 con 0x7f3720155d30 2024-04-20T21:20:47.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.689+0000 7f3726600640 1 -- 172.21.15.86:0/92016297 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37201f4b70 con 0x7f3720155d30 2024-04-20T21:20:47.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.689+0000 7f3726600640 1 -- 172.21.15.86:0/92016297 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3720149410 con 0x7f3720155d30 2024-04-20T21:20:47.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.690+0000 7f3726600640 1 -- 172.21.15.86:0/92016297 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37200689e0 con 0x7f3720155d30 2024-04-20T21:20:47.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.691+0000 7f3717400640 1 -- 172.21.15.86:0/92016297 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f37180043e0 con 0x7f3720155d30 2024-04-20T21:20:47.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.692+0000 7f3717400640 1 --2- 172.21.15.86:0/92016297 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f36fc06c8f0 0x7f36fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:47.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.692+0000 7f3717400640 1 -- 172.21.15.86:0/92016297 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f371808c630 con 0x7f3720155d30 2024-04-20T21:20:47.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.692+0000 7f371f400640 1 --2- 172.21.15.86:0/92016297 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f36fc06c8f0 0x7f36fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:47.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.693+0000 7f371f400640 1 --2- 172.21.15.86:0/92016297 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f36fc06c8f0 0x7f36fc06edb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3710035000 tx=0x7f3710008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:47.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.697+0000 7f3717400640 1 -- 172.21.15.86:0/92016297 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f371805aa20 con 0x7f3720155d30 2024-04-20T21:20:47.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.897+0000 7f3726600640 1 -- 172.21.15.86:0/92016297 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f37200b35f0 con 0x7f3720155d30 2024-04-20T21:20:47.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.898+0000 7f3717400640 1 -- 172.21.15.86:0/92016297 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v47) ==== 100+0+11 (secure 0 0 0) 0x7f371805a320 con 0x7f3720155d30 2024-04-20T21:20:47.898 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-20T21:20:47.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.901+0000 7f3726600640 1 -- 172.21.15.86:0/92016297 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f36fc06c8f0 msgr2=0x7f36fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:47.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.901+0000 7f3726600640 1 --2- 172.21.15.86:0/92016297 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f36fc06c8f0 0x7f36fc06edb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3710035000 tx=0x7f3710008030 comp rx=0 tx=0).stop 2024-04-20T21:20:47.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.901+0000 7f3726600640 1 -- 172.21.15.86:0/92016297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3720155d30 msgr2=0x7f3720148ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:47.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.901+0000 7f3726600640 1 --2- 172.21.15.86:0/92016297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3720155d30 0x7f3720148ed0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f371800cd70 tx=0x7f371800a660 comp rx=0 tx=0).stop 2024-04-20T21:20:47.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.901+0000 7f3726600640 1 -- 172.21.15.86:0/92016297 shutdown_connections 2024-04-20T21:20:47.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.901+0000 7f3726600640 1 --2- 172.21.15.86:0/92016297 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f36fc06c8f0 0x7f36fc06edb0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:47.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.901+0000 7f3726600640 1 --2- 172.21.15.86:0/92016297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3720155d30 0x7f3720148ed0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:47.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.901+0000 7f3726600640 1 --2- 172.21.15.86:0/92016297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37200b52b0 0x7f37201f4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:47.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.901+0000 7f3726600640 1 --2- 172.21.15.86:0/92016297 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f372006b980 0x7f372015cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:47.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.901+0000 7f3726600640 1 -- 172.21.15.86:0/92016297 >> 172.21.15.86:0/92016297 conn(0x7f37200b2110 msgr2=0x7f3720153290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:47.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.901+0000 7f3726600640 1 -- 172.21.15.86:0/92016297 shutdown_connections 2024-04-20T21:20:47.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:47.902+0000 7f3726600640 1 -- 172.21.15.86:0/92016297 wait complete. 2024-04-20T21:20:47.920 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T21:20:48.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.120+0000 7f8e43c00640 1 Processor -- start 2024-04-20T21:20:48.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.120+0000 7f8e43c00640 1 -- start start 2024-04-20T21:20:48.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.122+0000 7f8e43c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8e3c145fe0 0x7f8e3c0d02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:48.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.122+0000 7f8e43c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8e3c0cd7e0 0x7f8e3c071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:48.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.122+0000 7f8e43c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e3c15dcc0 con 0x7f8e3c0d0810 2024-04-20T21:20:48.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.122+0000 7f8e43c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8e3c0d00d0 con 0x7f8e3c0cd7e0 2024-04-20T21:20:48.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.122+0000 7f8e43c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8e3c0d0c70 con 0x7f8e3c145fe0 2024-04-20T21:20:48.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.122+0000 7f8e41c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8e3c0cd7e0 0x7f8e3c071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:48.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.122+0000 7f8e41c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8e3c0cd7e0 0x7f8e3c071cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39582/0 (socket says 172.21.15.86:39582) 2024-04-20T21:20:48.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.122+0000 7f8e41c00640 1 -- 172.21.15.86:0/625696824 learned_addr learned my addr 172.21.15.86:0/625696824 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:48.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.123+0000 7f8e41200640 1 --2- 172.21.15.86:0/625696824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8e3c145fe0 0x7f8e3c0d02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:48.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.123+0000 7f8e41c00640 1 -- 172.21.15.86:0/625696824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8e3c145fe0 msgr2=0x7f8e3c0d02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:48.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.123+0000 7f8e41c00640 1 --2- 172.21.15.86:0/625696824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8e3c145fe0 0x7f8e3c0d02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:48.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.123+0000 7f8e41c00640 1 -- 172.21.15.86:0/625696824 >> v1:172.21.15.144:6789/0 conn(0x7f8e3c0d0810 legacy=0x7f8e3c0cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:48.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.123+0000 7f8e41c00640 1 -- 172.21.15.86:0/625696824 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e3c15cf00 con 0x7f8e3c0cd7e0 2024-04-20T21:20:48.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.124+0000 7f8e41c00640 1 --2- 172.21.15.86:0/625696824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e3c0cd7e0 0x7f8e3c071cd0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f8e34009e50 tx=0x7f8e3402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37443bfe646d4f0e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:48.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.124+0000 7f8e3be00640 1 -- 172.21.15.86:0/625696824 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3402ca70 con 0x7f8e3c0cd7e0 2024-04-20T21:20:48.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.124+0000 7f8e3be00640 1 -- 172.21.15.86:0/625696824 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e3402cc10 con 0x7f8e3c0cd7e0 2024-04-20T21:20:48.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.124+0000 7f8e3be00640 1 -- 172.21.15.86:0/625696824 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e340343f0 con 0x7f8e3c0cd7e0 2024-04-20T21:20:48.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.125+0000 7f8e43c00640 1 -- 172.21.15.86:0/625696824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e3c0cd7e0 msgr2=0x7f8e3c071cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:48.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.125+0000 7f8e43c00640 1 --2- 172.21.15.86:0/625696824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e3c0cd7e0 0x7f8e3c071cd0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f8e34009e50 tx=0x7f8e3402bed0 comp rx=0 tx=0).stop 2024-04-20T21:20:48.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.125+0000 7f8e43c00640 1 -- 172.21.15.86:0/625696824 shutdown_connections 2024-04-20T21:20:48.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.125+0000 7f8e43c00640 1 --2- 172.21.15.86:0/625696824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e3c0cd7e0 0x7f8e3c071cd0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:48.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.125+0000 7f8e43c00640 1 --2- 172.21.15.86:0/625696824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8e3c145fe0 0x7f8e3c0d02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:48.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.125+0000 7f8e43c00640 1 -- 172.21.15.86:0/625696824 >> 172.21.15.86:0/625696824 conn(0x7f8e3c06b980 msgr2=0x7f8e3c06bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:48.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.125+0000 7f8e43c00640 1 -- 172.21.15.86:0/625696824 shutdown_connections 2024-04-20T21:20:48.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.125+0000 7f8e43c00640 1 -- 172.21.15.86:0/625696824 wait complete. 2024-04-20T21:20:48.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.127+0000 7f8e43c00640 1 Processor -- start 2024-04-20T21:20:48.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.127+0000 7f8e43c00640 1 -- start start 2024-04-20T21:20:48.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.128+0000 7f8e43c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e3c0cd7e0 0x7f8e3c0aec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:48.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.128+0000 7f8e43c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8e3c0d0810 0x7f8e3c15b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:48.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.129+0000 7f8e43c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e3c145fe0 0x7f8e3c13e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:48.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.129+0000 7f8e43c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3c15d7a0 con 0x7f8e3c0cd7e0 2024-04-20T21:20:48.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.129+0000 7f8e43c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3c15d4c0 con 0x7f8e3c145fe0 2024-04-20T21:20:48.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.129+0000 7f8e43c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8e3c1463c0 con 0x7f8e3c0d0810 2024-04-20T21:20:48.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.129+0000 7f8e41200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e3c0cd7e0 0x7f8e3c0aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:48.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.129+0000 7f8e41200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e3c0cd7e0 0x7f8e3c0aec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39590/0 (socket says 172.21.15.86:39590) 2024-04-20T21:20:48.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.129+0000 7f8e41200640 1 -- 172.21.15.86:0/1033490921 learned_addr learned my addr 172.21.15.86:0/1033490921 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:48.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.129+0000 7f8e33e00640 1 --2- 172.21.15.86:0/1033490921 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8e3c0d0810 0x7f8e3c15b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:48.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.129+0000 7f8e41200640 1 -- 172.21.15.86:0/1033490921 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8e3c0d0810 msgr2=0x7f8e3c15b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:48.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.129+0000 7f8e41200640 1 --2- 172.21.15.86:0/1033490921 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8e3c0d0810 0x7f8e3c15b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:48.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.129+0000 7f8e41c00640 1 --2- 172.21.15.86:0/1033490921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e3c145fe0 0x7f8e3c13e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:48.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.129+0000 7f8e41200640 1 -- 172.21.15.86:0/1033490921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e3c145fe0 msgr2=0x7f8e3c13e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:48.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.129+0000 7f8e41200640 1 --2- 172.21.15.86:0/1033490921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e3c145fe0 0x7f8e3c13e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:48.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.129+0000 7f8e41200640 1 -- 172.21.15.86:0/1033490921 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e3c0af140 con 0x7f8e3c0cd7e0 2024-04-20T21:20:48.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.129+0000 7f8e33e00640 1 --2- 172.21.15.86:0/1033490921 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8e3c0d0810 0x7f8e3c15b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:20:48.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.130+0000 7f8e41200640 1 --2- 172.21.15.86:0/1033490921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e3c0cd7e0 0x7f8e3c0aec00 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c00d620 tx=0x7f8e2c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:48.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.130+0000 7f8e39600640 1 -- 172.21.15.86:0/1033490921 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c00b620 con 0x7f8e3c0cd7e0 2024-04-20T21:20:48.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.130+0000 7f8e39600640 1 -- 172.21.15.86:0/1033490921 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e2c019070 con 0x7f8e3c0cd7e0 2024-04-20T21:20:48.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.130+0000 7f8e39600640 1 -- 172.21.15.86:0/1033490921 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c00ba00 con 0x7f8e3c0cd7e0 2024-04-20T21:20:48.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.130+0000 7f8e43c00640 1 -- 172.21.15.86:0/1033490921 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e3c0adb60 con 0x7f8e3c0cd7e0 2024-04-20T21:20:48.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.130+0000 7f8e43c00640 1 -- 172.21.15.86:0/1033490921 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e3c0ae120 con 0x7f8e3c0cd7e0 2024-04-20T21:20:48.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.132+0000 7f8e43c00640 1 -- 172.21.15.86:0/1033490921 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e3c0689e0 con 0x7f8e3c0cd7e0 2024-04-20T21:20:48.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.132+0000 7f8e39600640 1 -- 172.21.15.86:0/1033490921 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8e2c00bba0 con 0x7f8e3c0cd7e0 2024-04-20T21:20:48.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.134+0000 7f8e39600640 1 --2- 172.21.15.86:0/1033490921 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8e1806c920 0x7f8e1806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:48.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.134+0000 7f8e39600640 1 -- 172.21.15.86:0/1033490921 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f8e2c08bff0 con 0x7f8e3c0cd7e0 2024-04-20T21:20:48.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.134+0000 7f8e33e00640 1 --2- 172.21.15.86:0/1033490921 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8e1806c920 0x7f8e1806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:48.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.135+0000 7f8e33e00640 1 --2- 172.21.15.86:0/1033490921 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8e1806c920 0x7f8e1806ede0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c0cc4e0 tx=0x7f8e24009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:48.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.139+0000 7f8e39600640 1 -- 172.21.15.86:0/1033490921 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8e2c05a340 con 0x7f8e3c0cd7e0 2024-04-20T21:20:48.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.354+0000 7f8e43c00640 1 -- 172.21.15.86:0/1033490921 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8e3c144850 con 0x7f8e3c0cd7e0 2024-04-20T21:20:48.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.355+0000 7f8e39600640 1 -- 172.21.15.86:0/1033490921 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v47) ==== 80+0+0 (secure 0 0 0) 0x7f8e2c059c40 con 0x7f8e3c0cd7e0 2024-04-20T21:20:48.354 INFO:teuthology.orchestra.run.smithi086.stderr:listed 0 entries 2024-04-20T21:20:48.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.358+0000 7f8e43c00640 1 -- 172.21.15.86:0/1033490921 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8e1806c920 msgr2=0x7f8e1806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:48.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.358+0000 7f8e43c00640 1 --2- 172.21.15.86:0/1033490921 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8e1806c920 0x7f8e1806ede0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c0cc4e0 tx=0x7f8e24009040 comp rx=0 tx=0).stop 2024-04-20T21:20:48.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.358+0000 7f8e43c00640 1 -- 172.21.15.86:0/1033490921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e3c0cd7e0 msgr2=0x7f8e3c0aec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:48.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.358+0000 7f8e43c00640 1 --2- 172.21.15.86:0/1033490921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e3c0cd7e0 0x7f8e3c0aec00 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c00d620 tx=0x7f8e2c00dae0 comp rx=0 tx=0).stop 2024-04-20T21:20:48.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.358+0000 7f8e43c00640 1 -- 172.21.15.86:0/1033490921 shutdown_connections 2024-04-20T21:20:48.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.358+0000 7f8e43c00640 1 --2- 172.21.15.86:0/1033490921 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8e1806c920 0x7f8e1806ede0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:48.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.358+0000 7f8e43c00640 1 --2- 172.21.15.86:0/1033490921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8e3c145fe0 0x7f8e3c13e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:48.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.358+0000 7f8e43c00640 1 --2- 172.21.15.86:0/1033490921 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8e3c0d0810 0x7f8e3c15b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:48.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.358+0000 7f8e43c00640 1 --2- 172.21.15.86:0/1033490921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e3c0cd7e0 0x7f8e3c0aec00 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:48.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.358+0000 7f8e43c00640 1 -- 172.21.15.86:0/1033490921 >> 172.21.15.86:0/1033490921 conn(0x7f8e3c06b980 msgr2=0x7f8e3c0cd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:48.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.358+0000 7f8e43c00640 1 -- 172.21.15.86:0/1033490921 shutdown_connections 2024-04-20T21:20:48.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:48.358+0000 7f8e43c00640 1 -- 172.21.15.86:0/1033490921 wait complete. 2024-04-20T21:20:48.375 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi086.front.sepia.ceph.com... 2024-04-20T21:20:48.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:20:48.376 DEBUG:teuthology.orchestra.run.smithi086:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-20T21:20:48.454 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[40289]: fuse finished with error 0 and tester_r 0 2024-04-20T21:20:48.458 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-20T21:20:48.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:20:48.459 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-20T21:20:48.603 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T21:20:48.604 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:20:48.604 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:20:48.649 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T21:20:48.649 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2024-04-20T21:20:48.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:20:48.649 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-20T21:20:48.740 INFO:teuthology.orchestra.run.smithi144.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-20T21:20:48.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:20:48.741 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T21:20:48.741 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:20:48.741 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:20:48.797 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052731,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248996k,nr_inodes=812249,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T21:20:48.797 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-20T21:20:48.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:20:48.797 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-20T21:20:48.845 INFO:teuthology.orchestra.run.smithi086.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-20T21:20:48.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:20:48.847 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T21:20:48.847 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:20:48.847 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:20:48.903 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T21:20:48.903 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2024-04-20T21:20:48.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:20:48.903 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-20T21:20:48.949 INFO:teuthology.orchestra.run.smithi144.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-20T21:20:48.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:20:48.951 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:20:49.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.155+0000 7fd6a5a00640 1 Processor -- start 2024-04-20T21:20:49.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.156+0000 7fd6a5a00640 1 -- start start 2024-04-20T21:20:49.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.157+0000 7fd6a5a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6a015c6a0 0x7fd6a00d0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:49.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.157+0000 7fd6a5a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd6a00cce30 0x7fd6a0160200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:49.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.157+0000 7fd6a5a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6a00cd310 con 0x7fd6a00d1000 2024-04-20T21:20:49.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.157+0000 7fd6a5a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd6a00cd5f0 con 0x7fd6a00cce30 2024-04-20T21:20:49.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.157+0000 7fd6a5a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd6a00cd8d0 con 0x7fd6a015c6a0 2024-04-20T21:20:49.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.158+0000 7fd69f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd6a00cce30 0x7fd6a0160200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:49.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.158+0000 7fd69f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd6a00cce30 0x7fd6a0160200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39596/0 (socket says 172.21.15.86:39596) 2024-04-20T21:20:49.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.158+0000 7fd69f400640 1 -- 172.21.15.86:0/200656239 learned_addr learned my addr 172.21.15.86:0/200656239 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:49.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.158+0000 7fd69fe00640 1 --2- 172.21.15.86:0/200656239 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6a015c6a0 0x7fd6a00d0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:49.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.158+0000 7fd69f400640 1 -- 172.21.15.86:0/200656239 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6a015c6a0 msgr2=0x7fd6a00d0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:49.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.158+0000 7fd69f400640 1 --2- 172.21.15.86:0/200656239 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6a015c6a0 0x7fd6a00d0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:49.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.158+0000 7fd69f400640 1 -- 172.21.15.86:0/200656239 >> v1:172.21.15.144:6789/0 conn(0x7fd6a00d1000 legacy=0x7fd6a015ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:49.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.158+0000 7fd69f400640 1 -- 172.21.15.86:0/200656239 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6a00cdbb0 con 0x7fd6a00cce30 2024-04-20T21:20:49.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.158+0000 7fd69fe00640 1 --2- 172.21.15.86:0/200656239 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6a015c6a0 0x7fd6a00d0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:20:49.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.159+0000 7fd69f400640 1 --2- 172.21.15.86:0/200656239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6a00cce30 0x7fd6a0160200 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fd68c009e50 tx=0x7fd68c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71c24cb74f6dd07a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:49.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.160+0000 7fd69ea00640 1 -- 172.21.15.86:0/200656239 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c02ca70 con 0x7fd6a00cce30 2024-04-20T21:20:49.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.160+0000 7fd69ea00640 1 -- 172.21.15.86:0/200656239 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd68c02cc10 con 0x7fd6a00cce30 2024-04-20T21:20:49.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.160+0000 7fd69ea00640 1 -- 172.21.15.86:0/200656239 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c0343f0 con 0x7fd6a00cce30 2024-04-20T21:20:49.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.160+0000 7fd6a5a00640 1 -- 172.21.15.86:0/200656239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6a00cce30 msgr2=0x7fd6a0160200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:49.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.160+0000 7fd6a5a00640 1 --2- 172.21.15.86:0/200656239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6a00cce30 0x7fd6a0160200 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fd68c009e50 tx=0x7fd68c02bed0 comp rx=0 tx=0).stop 2024-04-20T21:20:49.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.160+0000 7fd6a5a00640 1 -- 172.21.15.86:0/200656239 shutdown_connections 2024-04-20T21:20:49.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.160+0000 7fd6a5a00640 1 --2- 172.21.15.86:0/200656239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6a00cce30 0x7fd6a0160200 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:49.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.160+0000 7fd6a5a00640 1 --2- 172.21.15.86:0/200656239 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6a015c6a0 0x7fd6a00d0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:49.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.160+0000 7fd6a5a00640 1 -- 172.21.15.86:0/200656239 >> 172.21.15.86:0/200656239 conn(0x7fd6a00b3e10 msgr2=0x7fd6a00b4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:49.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.161+0000 7fd6a5a00640 1 -- 172.21.15.86:0/200656239 shutdown_connections 2024-04-20T21:20:49.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.161+0000 7fd6a5a00640 1 -- 172.21.15.86:0/200656239 wait complete. 2024-04-20T21:20:49.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.162+0000 7fd6a5a00640 1 Processor -- start 2024-04-20T21:20:49.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.163+0000 7fd6a5a00640 1 -- start start 2024-04-20T21:20:49.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.163+0000 7fd6a5a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6a00cce30 0x7fd6a01f5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:49.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.163+0000 7fd6a5a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6a00d1000 0x7fd6a01f7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:49.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.163+0000 7fd6a5a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6a015c6a0 0x7fd6a00b6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:49.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.164+0000 7fd6a5a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a009bb70 con 0x7fd6a015c6a0 2024-04-20T21:20:49.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.164+0000 7fd6a5a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a009b9f0 con 0x7fd6a00cce30 2024-04-20T21:20:49.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.164+0000 7fd6a5a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd6a0160740 con 0x7fd6a00d1000 2024-04-20T21:20:49.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.164+0000 7fd6a5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6a015c6a0 0x7fd6a00b6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:49.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.164+0000 7fd6a5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6a015c6a0 0x7fd6a00b6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39602/0 (socket says 172.21.15.86:39602) 2024-04-20T21:20:49.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.164+0000 7fd6a5000640 1 -- 172.21.15.86:0/2881948790 learned_addr learned my addr 172.21.15.86:0/2881948790 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:49.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.164+0000 7fd69f400640 1 --2- 172.21.15.86:0/2881948790 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6a00d1000 0x7fd6a01f7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:49.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.164+0000 7fd6a5000640 1 -- 172.21.15.86:0/2881948790 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6a00d1000 msgr2=0x7fd6a01f7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:49.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.164+0000 7fd6a5000640 1 --2- 172.21.15.86:0/2881948790 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6a00d1000 0x7fd6a01f7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:49.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.164+0000 7fd69fe00640 1 --2- 172.21.15.86:0/2881948790 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6a00cce30 0x7fd6a01f5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:49.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.164+0000 7fd6a5000640 1 -- 172.21.15.86:0/2881948790 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6a00cce30 msgr2=0x7fd6a01f5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:49.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.164+0000 7fd6a5000640 1 --2- 172.21.15.86:0/2881948790 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6a00cce30 0x7fd6a01f5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:49.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.164+0000 7fd6a5000640 1 -- 172.21.15.86:0/2881948790 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6a00b4fa0 con 0x7fd6a015c6a0 2024-04-20T21:20:49.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.164+0000 7fd69f400640 1 --2- 172.21.15.86:0/2881948790 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6a00d1000 0x7fd6a01f7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:20:49.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.165+0000 7fd6a5000640 1 --2- 172.21.15.86:0/2881948790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6a015c6a0 0x7fd6a00b6fc0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fd69800c410 tx=0x7fd69800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:49.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.165+0000 7fd697e00640 1 -- 172.21.15.86:0/2881948790 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd698011790 con 0x7fd6a015c6a0 2024-04-20T21:20:49.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.165+0000 7fd697e00640 1 -- 172.21.15.86:0/2881948790 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd698017070 con 0x7fd6a015c6a0 2024-04-20T21:20:49.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.165+0000 7fd697e00640 1 -- 172.21.15.86:0/2881948790 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd698011bb0 con 0x7fd6a015c6a0 2024-04-20T21:20:49.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.165+0000 7fd6a5a00640 1 -- 172.21.15.86:0/2881948790 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6a00b5280 con 0x7fd6a015c6a0 2024-04-20T21:20:49.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.165+0000 7fd6a5a00640 1 -- 172.21.15.86:0/2881948790 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6a00b5840 con 0x7fd6a015c6a0 2024-04-20T21:20:49.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.166+0000 7fd6a5a00640 1 -- 172.21.15.86:0/2881948790 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6a004fcf0 con 0x7fd6a015c6a0 2024-04-20T21:20:49.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.167+0000 7fd697e00640 1 -- 172.21.15.86:0/2881948790 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd69801c460 con 0x7fd6a015c6a0 2024-04-20T21:20:49.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.168+0000 7fd697e00640 1 --2- 172.21.15.86:0/2881948790 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd68006c8f0 0x7fd68006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:49.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.169+0000 7fd697e00640 1 -- 172.21.15.86:0/2881948790 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fd69808b810 con 0x7fd6a015c6a0 2024-04-20T21:20:49.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.169+0000 7fd69fe00640 1 --2- 172.21.15.86:0/2881948790 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd68006c8f0 0x7fd68006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:49.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.169+0000 7fd69fe00640 1 --2- 172.21.15.86:0/2881948790 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd68006c8f0 0x7fd68006edb0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0071a20 tx=0x7fd690008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:49.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.173+0000 7fd697e00640 1 -- 172.21.15.86:0/2881948790 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd698021020 con 0x7fd6a015c6a0 2024-04-20T21:20:49.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.411+0000 7fd6a5a00640 1 -- 172.21.15.86:0/2881948790 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd6a00ce7c0 con 0x7fd6a015c6a0 2024-04-20T21:20:49.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.412+0000 7fd697e00640 1 -- 172.21.15.86:0/2881948790 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5199 (secure 0 0 0) 0x7fd698059b60 con 0x7fd6a015c6a0 2024-04-20T21:20:49.411 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:20:49.411 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5044,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/1013106052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":1013106052},{"type":"v1","addr":"172.21.15.144:6833","nonce":1013106052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5047,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/1073576945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1073576945},{"type":"v1","addr":"172.21.15.144:6835","nonce":1073576945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/821548012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":821548012},{"type":"v1","addr":"172.21.15.86:6835","nonce":821548012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:20:03.369281+0000","modified":"2024-04-20T21:20:06.438592+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5691},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5691":{"gid":5691,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/712628984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":712628984},{"type":"v1","addr":"172.21.15.86:6837","nonce":712628984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5691,"qdb_cluster":[5691]},"id":3}]} 2024-04-20T21:20:49.411 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 24 2024-04-20T21:20:49.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.414+0000 7fd6a5a00640 1 -- 172.21.15.86:0/2881948790 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd68006c8f0 msgr2=0x7fd68006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:49.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.414+0000 7fd6a5a00640 1 --2- 172.21.15.86:0/2881948790 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd68006c8f0 0x7fd68006edb0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0071a20 tx=0x7fd690008040 comp rx=0 tx=0).stop 2024-04-20T21:20:49.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.415+0000 7fd6a5a00640 1 -- 172.21.15.86:0/2881948790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6a015c6a0 msgr2=0x7fd6a00b6fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:49.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.415+0000 7fd6a5a00640 1 --2- 172.21.15.86:0/2881948790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6a015c6a0 0x7fd6a00b6fc0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fd69800c410 tx=0x7fd69800c8d0 comp rx=0 tx=0).stop 2024-04-20T21:20:49.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.415+0000 7fd6a5a00640 1 -- 172.21.15.86:0/2881948790 shutdown_connections 2024-04-20T21:20:49.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.415+0000 7fd6a5a00640 1 --2- 172.21.15.86:0/2881948790 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd68006c8f0 0x7fd68006edb0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:49.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.415+0000 7fd6a5a00640 1 --2- 172.21.15.86:0/2881948790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6a015c6a0 0x7fd6a00b6fc0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:49.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.415+0000 7fd6a5a00640 1 --2- 172.21.15.86:0/2881948790 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd6a00d1000 0x7fd6a01f7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:49.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.415+0000 7fd6a5a00640 1 --2- 172.21.15.86:0/2881948790 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd6a00cce30 0x7fd6a01f5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:49.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.415+0000 7fd6a5a00640 1 -- 172.21.15.86:0/2881948790 >> 172.21.15.86:0/2881948790 conn(0x7fd6a00b3e10 msgr2=0x7fd6a00b30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:49.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.415+0000 7fd6a5a00640 1 -- 172.21.15.86:0/2881948790 shutdown_connections 2024-04-20T21:20:49.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.415+0000 7fd6a5a00640 1 -- 172.21.15.86:0/2881948790 wait complete. 2024-04-20T21:20:49.433 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:20:49.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.635+0000 7f15f2200640 1 Processor -- start 2024-04-20T21:20:49.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.635+0000 7f15f2200640 1 -- start start 2024-04-20T21:20:49.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.637+0000 7f15f2200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15ec0ce8e0 0x7f15ec06b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:49.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.637+0000 7f15f2200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f15ec06bec0 0x7f15ec0cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:49.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.637+0000 7f15f2200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15ec0cd840 con 0x7f15ec15e950 2024-04-20T21:20:49.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.637+0000 7f15f2200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f15ec0cd610 con 0x7f15ec06bec0 2024-04-20T21:20:49.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.637+0000 7f15f2200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f15ec06c320 con 0x7f15ec0ce8e0 2024-04-20T21:20:49.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.637+0000 7f15ebe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f15ec06bec0 0x7f15ec0cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:49.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.637+0000 7f15eb400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f15ec15e950 0x7f15ec0b24d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40418/0 (socket says 172.21.15.86:40418) 2024-04-20T21:20:49.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.637+0000 7f15eb400640 1 -- 172.21.15.86:0/2279270350 learned_addr learned my addr 172.21.15.86:0/2279270350 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:20:49.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.638+0000 7f15e3e00640 1 --2- 172.21.15.86:0/2279270350 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15ec0ce8e0 0x7f15ec06b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:49.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.638+0000 7f15ebe00640 1 -- 172.21.15.86:0/2279270350 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15ec0ce8e0 msgr2=0x7f15ec06b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:49.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.638+0000 7f15ebe00640 1 --2- 172.21.15.86:0/2279270350 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15ec0ce8e0 0x7f15ec06b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:49.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.638+0000 7f15ebe00640 1 -- 172.21.15.86:0/2279270350 >> v1:172.21.15.86:6789/0 conn(0x7f15ec15e950 legacy=0x7f15ec0b24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:49.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.638+0000 7f15eaa00640 1 -- 172.21.15.86:0/2279270350 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3463170944 0 0) 0x7f15ec0cd840 con 0x7f15ec15e950 2024-04-20T21:20:49.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.638+0000 7f15ebe00640 1 -- 172.21.15.86:0/2279270350 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15ec0cdb20 con 0x7f15ec06bec0 2024-04-20T21:20:49.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.639+0000 7f15ebe00640 1 --2- 172.21.15.86:0/2279270350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15ec06bec0 0x7f15ec0cbaa0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f15e4009d90 tx=0x7f15e402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2142a446b754354b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:49.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.639+0000 7f15eaa00640 1 -- 172.21.15.86:0/2279270350 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e4039070 con 0x7f15ec06bec0 2024-04-20T21:20:49.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.639+0000 7f15eaa00640 1 -- 172.21.15.86:0/2279270350 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15e402cc50 con 0x7f15ec06bec0 2024-04-20T21:20:49.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.639+0000 7f15eaa00640 1 -- 172.21.15.86:0/2279270350 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e4034440 con 0x7f15ec06bec0 2024-04-20T21:20:49.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.640+0000 7f15f2200640 1 -- 172.21.15.86:0/2279270350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15ec06bec0 msgr2=0x7f15ec0cbaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:49.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.640+0000 7f15f2200640 1 --2- 172.21.15.86:0/2279270350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15ec06bec0 0x7f15ec0cbaa0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f15e4009d90 tx=0x7f15e402bea0 comp rx=0 tx=0).stop 2024-04-20T21:20:49.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.640+0000 7f15f2200640 1 -- 172.21.15.86:0/2279270350 shutdown_connections 2024-04-20T21:20:49.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.640+0000 7f15f2200640 1 --2- 172.21.15.86:0/2279270350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15ec06bec0 0x7f15ec0cbaa0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:49.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.640+0000 7f15f2200640 1 --2- 172.21.15.86:0/2279270350 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15ec0ce8e0 0x7f15ec06b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:49.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.640+0000 7f15f2200640 1 -- 172.21.15.86:0/2279270350 >> 172.21.15.86:0/2279270350 conn(0x7f15ec09a9e0 msgr2=0x7f15ec0b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:49.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.640+0000 7f15f2200640 1 -- 172.21.15.86:0/2279270350 shutdown_connections 2024-04-20T21:20:49.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.640+0000 7f15f2200640 1 -- 172.21.15.86:0/2279270350 wait complete. 2024-04-20T21:20:49.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.642+0000 7f15f2200640 1 Processor -- start 2024-04-20T21:20:49.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.642+0000 7f15f2200640 1 -- start start 2024-04-20T21:20:49.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.643+0000 7f15f2200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15ec06bec0 0x7f15ec157fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:49.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.643+0000 7f15f2200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15ec0ce8e0 0x7f15ec0cae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:49.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.643+0000 7f15f2200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15ec15e950 0x7f15ec0c86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:49.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.643+0000 7f15f2200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f15ec0cf350 con 0x7f15ec15e950 2024-04-20T21:20:49.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.643+0000 7f15f2200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f15ec0cece0 con 0x7f15ec06bec0 2024-04-20T21:20:49.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.643+0000 7f15f2200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f15ec0ce0e0 con 0x7f15ec0ce8e0 2024-04-20T21:20:49.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.643+0000 7f15ebe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15ec15e950 0x7f15ec0c86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:49.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.643+0000 7f15ebe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15ec15e950 0x7f15ec0c86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39626/0 (socket says 172.21.15.86:39626) 2024-04-20T21:20:49.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.643+0000 7f15ebe00640 1 -- 172.21.15.86:0/4015102100 learned_addr learned my addr 172.21.15.86:0/4015102100 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:49.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.644+0000 7f15ebe00640 1 -- 172.21.15.86:0/4015102100 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15ec0ce8e0 msgr2=0x7f15ec0cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:49.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.644+0000 7f15e3e00640 1 --2- 172.21.15.86:0/4015102100 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15ec0ce8e0 0x7f15ec0cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:49.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.644+0000 7f15eb400640 1 --2- 172.21.15.86:0/4015102100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15ec06bec0 0x7f15ec157fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:49.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.644+0000 7f15ebe00640 1 --2- 172.21.15.86:0/4015102100 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15ec0ce8e0 0x7f15ec0cae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:49.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.644+0000 7f15ebe00640 1 -- 172.21.15.86:0/4015102100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15ec06bec0 msgr2=0x7f15ec157fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:49.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.644+0000 7f15ebe00640 1 --2- 172.21.15.86:0/4015102100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15ec06bec0 0x7f15ec157fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:49.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.644+0000 7f15ebe00640 1 -- 172.21.15.86:0/4015102100 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15ec15b240 con 0x7f15ec15e950 2024-04-20T21:20:49.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.644+0000 7f15ebe00640 1 --2- 172.21.15.86:0/4015102100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15ec15e950 0x7f15ec0c86d0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f15e4009d90 tx=0x7f15e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:49.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.644+0000 7f15e3400640 1 -- 172.21.15.86:0/4015102100 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e4044020 con 0x7f15ec15e950 2024-04-20T21:20:49.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.644+0000 7f15e3400640 1 -- 172.21.15.86:0/4015102100 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15e403a040 con 0x7f15ec15e950 2024-04-20T21:20:49.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.644+0000 7f15e3400640 1 -- 172.21.15.86:0/4015102100 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e4039050 con 0x7f15ec15e950 2024-04-20T21:20:49.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.644+0000 7f15f2200640 1 -- 172.21.15.86:0/4015102100 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15ec15b520 con 0x7f15ec15e950 2024-04-20T21:20:49.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.645+0000 7f15f2200640 1 -- 172.21.15.86:0/4015102100 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15ec15bae0 con 0x7f15ec15e950 2024-04-20T21:20:49.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.646+0000 7f15f2200640 1 -- 172.21.15.86:0/4015102100 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15ec0689e0 con 0x7f15ec15e950 2024-04-20T21:20:49.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.646+0000 7f15e3400640 1 -- 172.21.15.86:0/4015102100 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f15e4003990 con 0x7f15ec15e950 2024-04-20T21:20:49.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.647+0000 7f15e3400640 1 --2- 172.21.15.86:0/4015102100 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f15c806c8f0 0x7f15c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:49.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.647+0000 7f15e3400640 1 -- 172.21.15.86:0/4015102100 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f15e40adfd0 con 0x7f15ec15e950 2024-04-20T21:20:49.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.648+0000 7f15eb400640 1 --2- 172.21.15.86:0/4015102100 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f15c806c8f0 0x7f15c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:49.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.648+0000 7f15eb400640 1 --2- 172.21.15.86:0/4015102100 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f15c806c8f0 0x7f15c806edb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f15dc000f60 tx=0x7f15dc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:49.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.652+0000 7f15e3400640 1 -- 172.21.15.86:0/4015102100 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f15e404e060 con 0x7f15ec15e950 2024-04-20T21:20:49.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.853+0000 7f15f2200640 1 -- 172.21.15.86:0/4015102100 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f15ec15bce0 con 0x7f15ec15e950 2024-04-20T21:20:49.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.855+0000 7f15e3400640 1 -- 172.21.15.86:0/4015102100 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16704 (secure 0 0 0) 0x7f15ec15bce0 con 0x7f15ec15e950 2024-04-20T21:20:49.855 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:20:49.855 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":47,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:20:34.772220+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:20:01.307757+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-20T21:20:02.359347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.6","mappings":[{"from":0,"to":4}]},{"pgid":"7.36","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:20:49.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.858+0000 7f15f2200640 1 -- 172.21.15.86:0/4015102100 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f15c806c8f0 msgr2=0x7f15c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:49.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.858+0000 7f15f2200640 1 --2- 172.21.15.86:0/4015102100 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f15c806c8f0 0x7f15c806edb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f15dc000f60 tx=0x7f15dc009040 comp rx=0 tx=0).stop 2024-04-20T21:20:49.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.858+0000 7f15f2200640 1 -- 172.21.15.86:0/4015102100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15ec15e950 msgr2=0x7f15ec0c86d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:49.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.858+0000 7f15f2200640 1 --2- 172.21.15.86:0/4015102100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15ec15e950 0x7f15ec0c86d0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f15e4009d90 tx=0x7f15e4004a40 comp rx=0 tx=0).stop 2024-04-20T21:20:49.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.858+0000 7f15f2200640 1 -- 172.21.15.86:0/4015102100 shutdown_connections 2024-04-20T21:20:49.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.858+0000 7f15f2200640 1 --2- 172.21.15.86:0/4015102100 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f15c806c8f0 0x7f15c806edb0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:49.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.858+0000 7f15f2200640 1 --2- 172.21.15.86:0/4015102100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15ec15e950 0x7f15ec0c86d0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:49.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.858+0000 7f15f2200640 1 --2- 172.21.15.86:0/4015102100 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15ec0ce8e0 0x7f15ec0cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:49.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.858+0000 7f15f2200640 1 --2- 172.21.15.86:0/4015102100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15ec06bec0 0x7f15ec157fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:49.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.858+0000 7f15f2200640 1 -- 172.21.15.86:0/4015102100 >> 172.21.15.86:0/4015102100 conn(0x7f15ec09a9e0 msgr2=0x7f15ec1eb190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:49.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.858+0000 7f15f2200640 1 -- 172.21.15.86:0/4015102100 shutdown_connections 2024-04-20T21:20:49.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:49.858+0000 7f15f2200640 1 -- 172.21.15.86:0/4015102100 wait complete. 2024-04-20T21:20:49.875 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:20:01.307757+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:20:02.359347+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:20:49.875 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:20:50.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.067+0000 7fd413600640 1 Processor -- start 2024-04-20T21:20:50.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.068+0000 7fd413600640 1 -- start start 2024-04-20T21:20:50.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.069+0000 7fd413600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd40c09a9e0 0x7fd40c0b3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:50.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.069+0000 7fd413600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd40c09adc0 0x7fd40c0d1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:50.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.069+0000 7fd413600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd40c06d830 0x7fd40c160500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:50.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.069+0000 7fd413600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd40c06ca90 con 0x7fd40c09adc0 2024-04-20T21:20:50.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.069+0000 7fd413600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fd40c0d1ab0 con 0x7fd40c09a9e0 2024-04-20T21:20:50.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.069+0000 7fd413600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd40c06cd50 con 0x7fd40c06d830 2024-04-20T21:20:50.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.069+0000 7fd40be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd40c09adc0 0x7fd40c0d1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:50.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.069+0000 7fd40be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd40c09adc0 0x7fd40c0d1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39638/0 (socket says 172.21.15.86:39638) 2024-04-20T21:20:50.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.069+0000 7fd40be00640 1 -- 172.21.15.86:0/2917272386 learned_addr learned my addr 172.21.15.86:0/2917272386 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:50.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.070+0000 7fd40be00640 1 -- 172.21.15.86:0/2917272386 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd40c06d830 msgr2=0x7fd40c160500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:50.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.070+0000 7fd411600640 1 --2- 172.21.15.86:0/2917272386 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd40c06d830 0x7fd40c160500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:50.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.070+0000 7fd410c00640 1 --2- 172.21.15.86:0/2917272386 >> v2:172.21.15.144:3300/0 conn(0x7fd40c09a9e0 0x7fd40c0b3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:50.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.070+0000 7fd40be00640 1 --2- 172.21.15.86:0/2917272386 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd40c06d830 0x7fd40c160500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:50.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.070+0000 7fd40be00640 1 -- 172.21.15.86:0/2917272386 >> v2:172.21.15.144:3300/0 conn(0x7fd40c09a9e0 msgr2=0x7fd40c0b3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:50.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.070+0000 7fd40be00640 1 --2- 172.21.15.86:0/2917272386 >> v2:172.21.15.144:3300/0 conn(0x7fd40c09a9e0 0x7fd40c0b3980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:50.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.070+0000 7fd40be00640 1 -- 172.21.15.86:0/2917272386 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd40c06d030 con 0x7fd40c09adc0 2024-04-20T21:20:50.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.071+0000 7fd40be00640 1 --2- 172.21.15.86:0/2917272386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd40c09adc0 0x7fd40c0d1230 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fd40c06ee00 tx=0x7fd3fc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21e8774c423f060c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:50.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.071+0000 7fd40b400640 1 -- 172.21.15.86:0/2917272386 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc02ca70 con 0x7fd40c09adc0 2024-04-20T21:20:50.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.071+0000 7fd40b400640 1 -- 172.21.15.86:0/2917272386 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3fc02cc10 con 0x7fd40c09adc0 2024-04-20T21:20:50.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.071+0000 7fd40b400640 1 -- 172.21.15.86:0/2917272386 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc0343f0 con 0x7fd40c09adc0 2024-04-20T21:20:50.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.072+0000 7fd413600640 1 -- 172.21.15.86:0/2917272386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd40c09adc0 msgr2=0x7fd40c0d1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:50.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.072+0000 7fd413600640 1 --2- 172.21.15.86:0/2917272386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd40c09adc0 0x7fd40c0d1230 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fd40c06ee00 tx=0x7fd3fc02bed0 comp rx=0 tx=0).stop 2024-04-20T21:20:50.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.072+0000 7fd413600640 1 -- 172.21.15.86:0/2917272386 shutdown_connections 2024-04-20T21:20:50.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.072+0000 7fd413600640 1 --2- 172.21.15.86:0/2917272386 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd40c06d830 0x7fd40c160500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:50.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.072+0000 7fd413600640 1 --2- 172.21.15.86:0/2917272386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd40c09adc0 0x7fd40c0d1230 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:50.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.072+0000 7fd413600640 1 --2- 172.21.15.86:0/2917272386 >> v2:172.21.15.144:3300/0 conn(0x7fd40c09a9e0 0x7fd40c0b3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:50.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.072+0000 7fd413600640 1 -- 172.21.15.86:0/2917272386 >> 172.21.15.86:0/2917272386 conn(0x7fd40c0b4070 msgr2=0x7fd40c0b4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:50.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.072+0000 7fd413600640 1 -- 172.21.15.86:0/2917272386 shutdown_connections 2024-04-20T21:20:50.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.072+0000 7fd413600640 1 -- 172.21.15.86:0/2917272386 wait complete. 2024-04-20T21:20:50.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.073+0000 7fd413600640 1 Processor -- start 2024-04-20T21:20:50.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.074+0000 7fd413600640 1 -- start start 2024-04-20T21:20:50.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.074+0000 7fd413600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd40c06d830 0x7fd40c0b7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:50.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.074+0000 7fd413600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd40c09a9e0 0x7fd40c0b5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:50.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.074+0000 7fd413600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd40c09adc0 0x7fd40c0af2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:50.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.074+0000 7fd413600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd40c09b210 con 0x7fd40c09a9e0 2024-04-20T21:20:50.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.074+0000 7fd413600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd40c0d1770 con 0x7fd40c09adc0 2024-04-20T21:20:50.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.074+0000 7fd413600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd40c09b390 con 0x7fd40c06d830 2024-04-20T21:20:50.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.075+0000 7fd40be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd40c09a9e0 0x7fd40c0b5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:50.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.075+0000 7fd40be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd40c09a9e0 0x7fd40c0b5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39648/0 (socket says 172.21.15.86:39648) 2024-04-20T21:20:50.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.075+0000 7fd40be00640 1 -- 172.21.15.86:0/4004984185 learned_addr learned my addr 172.21.15.86:0/4004984185 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:50.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.075+0000 7fd40be00640 1 -- 172.21.15.86:0/4004984185 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd40c06d830 msgr2=0x7fd40c0b7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:50.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.075+0000 7fd40be00640 1 --2- 172.21.15.86:0/4004984185 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd40c06d830 0x7fd40c0b7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:50.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.075+0000 7fd40be00640 1 -- 172.21.15.86:0/4004984185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd40c09adc0 msgr2=0x7fd40c0af2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:50.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.075+0000 7fd40be00640 1 --2- 172.21.15.86:0/4004984185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd40c09adc0 0x7fd40c0af2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:50.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.075+0000 7fd40be00640 1 -- 172.21.15.86:0/4004984185 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd40c0b66b0 con 0x7fd40c09a9e0 2024-04-20T21:20:50.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.075+0000 7fd40be00640 1 --2- 172.21.15.86:0/4004984185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd40c09a9e0 0x7fd40c0b5c30 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc009e20 tx=0x7fd3fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:50.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.076+0000 7fd408c00640 1 -- 172.21.15.86:0/4004984185 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc044020 con 0x7fd40c09a9e0 2024-04-20T21:20:50.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.076+0000 7fd408c00640 1 -- 172.21.15.86:0/4004984185 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3fc0406b0 con 0x7fd40c09a9e0 2024-04-20T21:20:50.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.076+0000 7fd408c00640 1 -- 172.21.15.86:0/4004984185 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc040aa0 con 0x7fd40c09a9e0 2024-04-20T21:20:50.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.076+0000 7fd413600640 1 -- 172.21.15.86:0/4004984185 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd40c0b6990 con 0x7fd40c09a9e0 2024-04-20T21:20:50.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.076+0000 7fd413600640 1 -- 172.21.15.86:0/4004984185 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd40c0b6f50 con 0x7fd40c09a9e0 2024-04-20T21:20:50.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.077+0000 7fd413600640 1 -- 172.21.15.86:0/4004984185 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd40c0718a0 con 0x7fd40c09a9e0 2024-04-20T21:20:50.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.078+0000 7fd408c00640 1 -- 172.21.15.86:0/4004984185 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd3fc047020 con 0x7fd40c09a9e0 2024-04-20T21:20:50.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.079+0000 7fd408c00640 1 --2- 172.21.15.86:0/4004984185 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd3e806c8f0 0x7fd3e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:50.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.079+0000 7fd408c00640 1 -- 172.21.15.86:0/4004984185 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fd3fc0ae780 con 0x7fd40c09a9e0 2024-04-20T21:20:50.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.079+0000 7fd410c00640 1 --2- 172.21.15.86:0/4004984185 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd3e806c8f0 0x7fd3e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:50.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.080+0000 7fd410c00640 1 --2- 172.21.15.86:0/4004984185 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd3e806c8f0 0x7fd3e806edb0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd3f400a770 tx=0x7fd3f4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:50.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.083+0000 7fd408c00640 1 -- 172.21.15.86:0/4004984185 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd40c0718a0 con 0x7fd40c09a9e0 2024-04-20T21:20:50.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.283+0000 7fd413600640 1 -- 172.21.15.86:0/4004984185 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd40c0689e0 con 0x7fd40c09a9e0 2024-04-20T21:20:50.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.284+0000 7fd408c00640 1 -- 172.21.15.86:0/4004984185 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) ==== 93+0+10 (secure 0 0 0) 0x7fd3fc04e030 con 0x7fd40c09a9e0 2024-04-20T21:20:50.284 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:20:50.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.287+0000 7fd413600640 1 -- 172.21.15.86:0/4004984185 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd3e806c8f0 msgr2=0x7fd3e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:50.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.287+0000 7fd413600640 1 --2- 172.21.15.86:0/4004984185 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd3e806c8f0 0x7fd3e806edb0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd3f400a770 tx=0x7fd3f4008040 comp rx=0 tx=0).stop 2024-04-20T21:20:50.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.287+0000 7fd413600640 1 -- 172.21.15.86:0/4004984185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd40c09a9e0 msgr2=0x7fd40c0b5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:50.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.287+0000 7fd413600640 1 --2- 172.21.15.86:0/4004984185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd40c09a9e0 0x7fd40c0b5c30 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc009e20 tx=0x7fd3fc004a40 comp rx=0 tx=0).stop 2024-04-20T21:20:50.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.287+0000 7fd413600640 1 -- 172.21.15.86:0/4004984185 shutdown_connections 2024-04-20T21:20:50.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.287+0000 7fd413600640 1 --2- 172.21.15.86:0/4004984185 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd3e806c8f0 0x7fd3e806edb0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:50.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.287+0000 7fd413600640 1 --2- 172.21.15.86:0/4004984185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd40c09adc0 0x7fd40c0af2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:50.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.287+0000 7fd413600640 1 --2- 172.21.15.86:0/4004984185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd40c09a9e0 0x7fd40c0b5c30 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:50.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.287+0000 7fd413600640 1 --2- 172.21.15.86:0/4004984185 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd40c06d830 0x7fd40c0b7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:50.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.287+0000 7fd413600640 1 -- 172.21.15.86:0/4004984185 >> 172.21.15.86:0/4004984185 conn(0x7fd40c0b4070 msgr2=0x7fd40c06ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:50.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.287+0000 7fd413600640 1 -- 172.21.15.86:0/4004984185 shutdown_connections 2024-04-20T21:20:50.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.288+0000 7fd413600640 1 -- 172.21.15.86:0/4004984185 wait complete. 2024-04-20T21:20:50.304 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:20:50.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.504+0000 7f0f6b000640 1 Processor -- start 2024-04-20T21:20:50.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.504+0000 7f0f6b000640 1 -- start start 2024-04-20T21:20:50.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.505+0000 7f0f6b000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0f6406bbf0 0x7f0f6406e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:50.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.505+0000 7f0f6b000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f6406bfd0 0x7f0f64145b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:50.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.505+0000 7f0f6b000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f6415cd40 con 0x7f0f64146040 2024-04-20T21:20:50.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.505+0000 7f0f6b000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f0f6415d000 con 0x7f0f6406bbf0 2024-04-20T21:20:50.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.505+0000 7f0f6b000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0f6415d2e0 con 0x7f0f6406bfd0 2024-04-20T21:20:50.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.506+0000 7f0f63e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0f6406bbf0 0x7f0f6406e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:50.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.506+0000 7f0f69000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f0f64146040 0x7f0f640cd540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40424/0 (socket says 172.21.15.86:40424) 2024-04-20T21:20:50.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.506+0000 7f0f69000640 1 -- 172.21.15.86:0/1181162950 learned_addr learned my addr 172.21.15.86:0/1181162950 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:20:50.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.506+0000 7f0f63400640 1 --2- 172.21.15.86:0/1181162950 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f6406bfd0 0x7f0f64145b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:50.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.506+0000 7f0f63e00640 1 -- 172.21.15.86:0/1181162950 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f6406bfd0 msgr2=0x7f0f64145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:50.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.506+0000 7f0f63e00640 1 --2- 172.21.15.86:0/1181162950 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f6406bfd0 0x7f0f64145b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:50.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.506+0000 7f0f63e00640 1 -- 172.21.15.86:0/1181162950 >> v1:172.21.15.86:6789/0 conn(0x7f0f64146040 legacy=0x7f0f640cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:50.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.506+0000 7f0f63e00640 1 -- 172.21.15.86:0/1181162950 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f6415d5c0 con 0x7f0f6406bbf0 2024-04-20T21:20:50.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.506+0000 7f0f62a00640 1 -- 172.21.15.86:0/1181162950 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3012316549 0 0) 0x7f0f6415cd40 con 0x7f0f64146040 2024-04-20T21:20:50.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.507+0000 7f0f63e00640 1 --2- 172.21.15.86:0/1181162950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f6406bbf0 0x7f0f6406e000 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f0f540097a0 tx=0x7f0f5402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40324339c6ef2391 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:50.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.508+0000 7f0f62a00640 1 -- 172.21.15.86:0/1181162950 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f54035020 con 0x7f0f6406bbf0 2024-04-20T21:20:50.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.508+0000 7f0f62a00640 1 -- 172.21.15.86:0/1181162950 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f54031830 con 0x7f0f6406bbf0 2024-04-20T21:20:50.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.508+0000 7f0f62a00640 1 -- 172.21.15.86:0/1181162950 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f54031c00 con 0x7f0f6406bbf0 2024-04-20T21:20:50.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.508+0000 7f0f6b000640 1 -- 172.21.15.86:0/1181162950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f6406bbf0 msgr2=0x7f0f6406e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:50.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.508+0000 7f0f6b000640 1 --2- 172.21.15.86:0/1181162950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f6406bbf0 0x7f0f6406e000 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f0f540097a0 tx=0x7f0f5402b8d0 comp rx=0 tx=0).stop 2024-04-20T21:20:50.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.508+0000 7f0f6b000640 1 -- 172.21.15.86:0/1181162950 shutdown_connections 2024-04-20T21:20:50.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.508+0000 7f0f6b000640 1 --2- 172.21.15.86:0/1181162950 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f6406bfd0 0x7f0f64145b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:50.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.508+0000 7f0f6b000640 1 --2- 172.21.15.86:0/1181162950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f6406bbf0 0x7f0f6406e000 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:50.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.508+0000 7f0f6b000640 1 -- 172.21.15.86:0/1181162950 >> 172.21.15.86:0/1181162950 conn(0x7f0f640b2110 msgr2=0x7f0f640b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:50.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.508+0000 7f0f6b000640 1 -- 172.21.15.86:0/1181162950 shutdown_connections 2024-04-20T21:20:50.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.508+0000 7f0f6b000640 1 -- 172.21.15.86:0/1181162950 wait complete. 2024-04-20T21:20:50.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.510+0000 7f0f6b000640 1 Processor -- start 2024-04-20T21:20:50.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.510+0000 7f0f6b000640 1 -- start start 2024-04-20T21:20:50.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.511+0000 7f0f6b000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f6406bbf0 0x7f0f640c8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:50.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.511+0000 7f0f6b000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f6406bfd0 0x7f0f641e6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:50.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.511+0000 7f0f6b000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f64146040 0x7f0f6413f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:50.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.511+0000 7f0f6b000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0f640cddd0 con 0x7f0f6406bbf0 2024-04-20T21:20:50.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.511+0000 7f0f6b000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0f640cdc50 con 0x7f0f64146040 2024-04-20T21:20:50.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.511+0000 7f0f6b000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0f640cdf50 con 0x7f0f6406bfd0 2024-04-20T21:20:50.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.511+0000 7f0f63e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f6406bbf0 0x7f0f640c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:50.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.511+0000 7f0f63e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f6406bbf0 0x7f0f640c8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39668/0 (socket says 172.21.15.86:39668) 2024-04-20T21:20:50.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.511+0000 7f0f63e00640 1 -- 172.21.15.86:0/236971963 learned_addr learned my addr 172.21.15.86:0/236971963 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:50.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.512+0000 7f0f63400640 1 --2- 172.21.15.86:0/236971963 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f6406bfd0 0x7f0f641e6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:50.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.512+0000 7f0f63e00640 1 -- 172.21.15.86:0/236971963 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f6406bfd0 msgr2=0x7f0f641e6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:50.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.512+0000 7f0f63e00640 1 --2- 172.21.15.86:0/236971963 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f6406bfd0 0x7f0f641e6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:50.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.512+0000 7f0f63e00640 1 -- 172.21.15.86:0/236971963 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f64146040 msgr2=0x7f0f6413f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:50.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.512+0000 7f0f69000640 1 --2- 172.21.15.86:0/236971963 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f64146040 0x7f0f6413f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:50.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.512+0000 7f0f63e00640 1 --2- 172.21.15.86:0/236971963 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f64146040 0x7f0f6413f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:50.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.512+0000 7f0f63e00640 1 -- 172.21.15.86:0/236971963 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f641e79c0 con 0x7f0f6406bbf0 2024-04-20T21:20:50.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.512+0000 7f0f63e00640 1 --2- 172.21.15.86:0/236971963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f6406bbf0 0x7f0f640c8130 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f0f5402bee0 tx=0x7f0f540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:50.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.512+0000 7f0f5be00640 1 -- 172.21.15.86:0/236971963 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f54041070 con 0x7f0f6406bbf0 2024-04-20T21:20:50.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.512+0000 7f0f5be00640 1 -- 172.21.15.86:0/236971963 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f5403a950 con 0x7f0f6406bbf0 2024-04-20T21:20:50.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.512+0000 7f0f5be00640 1 -- 172.21.15.86:0/236971963 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f54035040 con 0x7f0f6406bbf0 2024-04-20T21:20:50.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.512+0000 7f0f6b000640 1 -- 172.21.15.86:0/236971963 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f641e6e50 con 0x7f0f6406bbf0 2024-04-20T21:20:50.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.513+0000 7f0f6b000640 1 -- 172.21.15.86:0/236971963 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f641e7400 con 0x7f0f6406bbf0 2024-04-20T21:20:50.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.514+0000 7f0f6b000640 1 -- 172.21.15.86:0/236971963 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f640689e0 con 0x7f0f6406bbf0 2024-04-20T21:20:50.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.515+0000 7f0f5be00640 1 -- 172.21.15.86:0/236971963 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0f54038020 con 0x7f0f6406bbf0 2024-04-20T21:20:50.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.516+0000 7f0f5be00640 1 --2- 172.21.15.86:0/236971963 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0f4406c8f0 0x7f0f4406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:50.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.516+0000 7f0f5be00640 1 -- 172.21.15.86:0/236971963 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f0f540ad710 con 0x7f0f6406bbf0 2024-04-20T21:20:50.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.516+0000 7f0f63400640 1 --2- 172.21.15.86:0/236971963 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0f4406c8f0 0x7f0f4406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:50.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.516+0000 7f0f63400640 1 --2- 172.21.15.86:0/236971963 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0f4406c8f0 0x7f0f4406edb0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0f50006fd0 tx=0x7f0f50008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:50.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.520+0000 7f0f5be00640 1 -- 172.21.15.86:0/236971963 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0f5407ba60 con 0x7f0f6406bbf0 2024-04-20T21:20:50.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.722+0000 7f0f6b000640 1 -- 172.21.15.86:0/236971963 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0f6406c780 con 0x7f0f6406bbf0 2024-04-20T21:20:50.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.723+0000 7f0f5be00640 1 -- 172.21.15.86:0/236971963 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v47) ==== 104+0+11 (secure 0 0 0) 0x7f0f5407b360 con 0x7f0f6406bbf0 2024-04-20T21:20:50.722 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-20T21:20:50.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.725+0000 7f0f6b000640 1 -- 172.21.15.86:0/236971963 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0f4406c8f0 msgr2=0x7f0f4406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:50.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.725+0000 7f0f6b000640 1 --2- 172.21.15.86:0/236971963 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0f4406c8f0 0x7f0f4406edb0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0f50006fd0 tx=0x7f0f50008040 comp rx=0 tx=0).stop 2024-04-20T21:20:50.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.725+0000 7f0f6b000640 1 -- 172.21.15.86:0/236971963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f6406bbf0 msgr2=0x7f0f640c8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:50.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.725+0000 7f0f6b000640 1 --2- 172.21.15.86:0/236971963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f6406bbf0 0x7f0f640c8130 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f0f5402bee0 tx=0x7f0f540046c0 comp rx=0 tx=0).stop 2024-04-20T21:20:50.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.725+0000 7f0f6b000640 1 -- 172.21.15.86:0/236971963 shutdown_connections 2024-04-20T21:20:50.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.725+0000 7f0f6b000640 1 --2- 172.21.15.86:0/236971963 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0f4406c8f0 0x7f0f4406edb0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:50.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.725+0000 7f0f6b000640 1 --2- 172.21.15.86:0/236971963 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f64146040 0x7f0f6413f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:50.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.725+0000 7f0f6b000640 1 --2- 172.21.15.86:0/236971963 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0f6406bfd0 0x7f0f641e6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:50.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.725+0000 7f0f6b000640 1 --2- 172.21.15.86:0/236971963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f6406bbf0 0x7f0f640c8130 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:50.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.725+0000 7f0f6b000640 1 -- 172.21.15.86:0/236971963 >> 172.21.15.86:0/236971963 conn(0x7f0f640b2110 msgr2=0x7f0f6406f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:50.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.725+0000 7f0f6b000640 1 -- 172.21.15.86:0/236971963 shutdown_connections 2024-04-20T21:20:50.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.725+0000 7f0f6b000640 1 -- 172.21.15.86:0/236971963 wait complete. 2024-04-20T21:20:50.744 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:20:50.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.952+0000 7f48c2c00640 1 Processor -- start 2024-04-20T21:20:50.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.952+0000 7f48c2c00640 1 -- start start 2024-04-20T21:20:50.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.954+0000 7f48c2c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f48bc146210 0x7f48bc06e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:50.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.954+0000 7f48c2c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48bc15cd40 con 0x7f48bc06bbf0 2024-04-20T21:20:50.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.954+0000 7f48c2c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48bc15d000 con 0x7f48bc06bfd0 2024-04-20T21:20:50.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.954+0000 7f48c2c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f48bc15d2e0 con 0x7f48bc146210 2024-04-20T21:20:50.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.954+0000 7f48bbe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f48bc06bbf0 0x7f48bc145b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40440/0 (socket says 172.21.15.86:40440) 2024-04-20T21:20:50.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.954+0000 7f48bbe00640 1 -- 172.21.15.86:0/1879332991 learned_addr learned my addr 172.21.15.86:0/1879332991 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:20:50.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.954+0000 7f48bb400640 1 --2- 172.21.15.86:0/1879332991 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f48bc146210 0x7f48bc06e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:50.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.955+0000 7f48baa00640 1 -- 172.21.15.86:0/1879332991 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2005697083 0 0) 0x7f48bc15cd40 con 0x7f48bc06bbf0 2024-04-20T21:20:50.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.955+0000 7f48baa00640 1 -- 172.21.15.86:0/1879332991 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f48a0003520 con 0x7f48bc06bbf0 2024-04-20T21:20:50.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.955+0000 7f48baa00640 1 -- 172.21.15.86:0/1879332991 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 759398035 0 0) 0x7f48a0003520 con 0x7f48bc06bbf0 2024-04-20T21:20:50.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.955+0000 7f48baa00640 1 -- 172.21.15.86:0/1879332991 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f48bc146210 msgr2=0x7f48bc06e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:50.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.955+0000 7f48baa00640 1 --2- 172.21.15.86:0/1879332991 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f48bc146210 0x7f48bc06e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:50.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.955+0000 7f48baa00640 1 -- 172.21.15.86:0/1879332991 >> v1:172.21.15.144:6789/0 conn(0x7f48bc06bfd0 legacy=0x7f48bc0cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:50.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.955+0000 7f48baa00640 1 -- 172.21.15.86:0/1879332991 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48bc15d5c0 con 0x7f48bc06bbf0 2024-04-20T21:20:50.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.955+0000 7f48baa00640 1 -- 172.21.15.86:0/1879332991 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f48ac001670 con 0x7f48bc06bbf0 2024-04-20T21:20:50.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.955+0000 7f48baa00640 1 -- 172.21.15.86:0/1879332991 >> v1:172.21.15.86:6789/0 conn(0x7f48bc06bbf0 legacy=0x7f48bc145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:50.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.955+0000 7f48baa00640 1 --2- 172.21.15.86:0/1879332991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48a00039e0 0x7f48a0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:50.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.955+0000 7f48baa00640 1 --2- 172.21.15.86:0/1879332991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48a0004800 0x7f48a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:50.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.955+0000 7f48baa00640 1 --2- 172.21.15.86:0/1879332991 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f48bc146210 0x7f48a00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:50.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.955+0000 7f48baa00640 1 -- 172.21.15.86:0/1879332991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f48bc15d2e0 con 0x7f48a00039e0 2024-04-20T21:20:50.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.955+0000 7f48baa00640 1 -- 172.21.15.86:0/1879332991 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f48a000ba70 con 0x7f48a0004800 2024-04-20T21:20:50.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.955+0000 7f48baa00640 1 -- 172.21.15.86:0/1879332991 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f48a000bc70 con 0x7f48bc146210 2024-04-20T21:20:50.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.955+0000 7f48bbe00640 1 --2- 172.21.15.86:0/1879332991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48a00039e0 0x7f48a0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:50.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.956+0000 7f48bb400640 1 --2- 172.21.15.86:0/1879332991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48a0004800 0x7f48a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:50.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.956+0000 7f48c0c00640 1 --2- 172.21.15.86:0/1879332991 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f48bc146210 0x7f48a00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:50.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.956+0000 7f48bbe00640 1 -- 172.21.15.86:0/1879332991 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f48bc146210 msgr2=0x7f48a00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:50.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.956+0000 7f48bbe00640 1 --2- 172.21.15.86:0/1879332991 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f48bc146210 0x7f48a00051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:50.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.956+0000 7f48bbe00640 1 -- 172.21.15.86:0/1879332991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48a0004800 msgr2=0x7f48a0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:50.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.956+0000 7f48bbe00640 1 --2- 172.21.15.86:0/1879332991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48a0004800 0x7f48a0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:50.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.956+0000 7f48bbe00640 1 -- 172.21.15.86:0/1879332991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48a0003520 con 0x7f48a00039e0 2024-04-20T21:20:50.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.957+0000 7f48bbe00640 1 --2- 172.21.15.86:0/1879332991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48a00039e0 0x7f48a0003e00 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f48ac00c210 tx=0x7f48a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66eaa47fd205910 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:50.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.957+0000 7f48baa00640 1 -- 172.21.15.86:0/1879332991 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48ac007cf0 con 0x7f48a00039e0 2024-04-20T21:20:50.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.957+0000 7f48baa00640 1 -- 172.21.15.86:0/1879332991 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48ac030070 con 0x7f48a00039e0 2024-04-20T21:20:50.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.957+0000 7f48baa00640 1 -- 172.21.15.86:0/1879332991 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48ac005d30 con 0x7f48a00039e0 2024-04-20T21:20:50.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.958+0000 7f48c2c00640 1 -- 172.21.15.86:0/1879332991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48a00039e0 msgr2=0x7f48a0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:50.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.958+0000 7f48c2c00640 1 --2- 172.21.15.86:0/1879332991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48a00039e0 0x7f48a0003e00 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f48ac00c210 tx=0x7f48a0005830 comp rx=0 tx=0).stop 2024-04-20T21:20:50.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.958+0000 7f48c0c00640 1 -- 172.21.15.86:0/1879332991 reap_dead start 2024-04-20T21:20:50.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.958+0000 7f48c2c00640 1 -- 172.21.15.86:0/1879332991 shutdown_connections 2024-04-20T21:20:50.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.958+0000 7f48c2c00640 1 -- 172.21.15.86:0/1879332991 >> 172.21.15.86:0/1879332991 conn(0x7f48bc0b2110 msgr2=0x7f48bc0b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:50.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.958+0000 7f48c2c00640 1 -- 172.21.15.86:0/1879332991 shutdown_connections 2024-04-20T21:20:50.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.958+0000 7f48c2c00640 1 -- 172.21.15.86:0/1879332991 wait complete. 2024-04-20T21:20:50.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.960+0000 7f48c2c00640 1 Processor -- start 2024-04-20T21:20:50.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.960+0000 7f48c2c00640 1 -- start start 2024-04-20T21:20:50.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.961+0000 7f48c2c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48a00039e0 0x7f48bc0ad850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:50.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.961+0000 7f48c2c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48bc0b5090 0x7f48bc1e7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:50.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.961+0000 7f48c2c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f48bc1f4b60 0x7f48bc1f4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:50.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.961+0000 7f48c2c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f48bc0cddd0 con 0x7f48a00039e0 2024-04-20T21:20:50.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.961+0000 7f48c2c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f48bc0cdc50 con 0x7f48bc0b5090 2024-04-20T21:20:50.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.961+0000 7f48c2c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f48bc0cdf50 con 0x7f48bc1f4b60 2024-04-20T21:20:50.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.961+0000 7f48bbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48a00039e0 0x7f48bc0ad850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:50.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.961+0000 7f48bbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48a00039e0 0x7f48bc0ad850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39684/0 (socket says 172.21.15.86:39684) 2024-04-20T21:20:50.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.961+0000 7f48bbe00640 1 -- 172.21.15.86:0/3527625327 learned_addr learned my addr 172.21.15.86:0/3527625327 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:50.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.961+0000 7f48bbe00640 1 -- 172.21.15.86:0/3527625327 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f48bc1f4b60 msgr2=0x7f48bc1f4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:50.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.961+0000 7f48bbe00640 1 --2- 172.21.15.86:0/3527625327 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f48bc1f4b60 0x7f48bc1f4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:50.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.961+0000 7f48bbe00640 1 -- 172.21.15.86:0/3527625327 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48bc0b5090 msgr2=0x7f48bc1e7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:50.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.961+0000 7f48bbe00640 1 --2- 172.21.15.86:0/3527625327 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48bc0b5090 0x7f48bc1e7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:50.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.961+0000 7f48bbe00640 1 -- 172.21.15.86:0/3527625327 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48bc1f5520 con 0x7f48a00039e0 2024-04-20T21:20:50.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.962+0000 7f48bbe00640 1 --2- 172.21.15.86:0/3527625327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48a00039e0 0x7f48bc0ad850 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f48ac00c210 tx=0x7f48bc15d1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:50.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.962+0000 7f48b3e00640 1 -- 172.21.15.86:0/3527625327 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48ac045070 con 0x7f48a00039e0 2024-04-20T21:20:50.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.962+0000 7f48b3e00640 1 -- 172.21.15.86:0/3527625327 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48ac004bf0 con 0x7f48a00039e0 2024-04-20T21:20:50.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.962+0000 7f48c2c00640 1 -- 172.21.15.86:0/3527625327 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48bc0b54f0 con 0x7f48a00039e0 2024-04-20T21:20:50.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.962+0000 7f48b3e00640 1 -- 172.21.15.86:0/3527625327 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48ac001c30 con 0x7f48a00039e0 2024-04-20T21:20:50.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.962+0000 7f48c2c00640 1 -- 172.21.15.86:0/3527625327 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48bc0ae230 con 0x7f48a00039e0 2024-04-20T21:20:50.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.963+0000 7f48c2c00640 1 -- 172.21.15.86:0/3527625327 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48bc0689e0 con 0x7f48a00039e0 2024-04-20T21:20:50.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.964+0000 7f48b3e00640 1 -- 172.21.15.86:0/3527625327 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f48ac007070 con 0x7f48a00039e0 2024-04-20T21:20:50.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.965+0000 7f48b3e00640 1 --2- 172.21.15.86:0/3527625327 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f489c06c920 0x7f489c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:50.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.965+0000 7f48b3e00640 1 -- 172.21.15.86:0/3527625327 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f48ac0acb30 con 0x7f48a00039e0 2024-04-20T21:20:50.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.966+0000 7f48bb400640 1 --2- 172.21.15.86:0/3527625327 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f489c06c920 0x7f489c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:50.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.966+0000 7f48bb400640 1 --2- 172.21.15.86:0/3527625327 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f489c06c920 0x7f489c06ede0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f48bc1f8090 tx=0x7f48a8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:50.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:50.970+0000 7f48b3e00640 1 -- 172.21.15.86:0/3527625327 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f48bc1f5520 con 0x7f48a00039e0 2024-04-20T21:20:51.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.155+0000 7f48c2c00640 1 -- 172.21.15.86:0/3527625327 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f48bc0d0a10 con 0x7f48a00039e0 2024-04-20T21:20:51.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.156+0000 7f48b3e00640 1 -- 172.21.15.86:0/3527625327 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v47) ==== 100+0+11 (secure 0 0 0) 0x7f48ac07af90 con 0x7f48a00039e0 2024-04-20T21:20:51.155 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-20T21:20:51.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.159+0000 7f48c2c00640 1 -- 172.21.15.86:0/3527625327 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f489c06c920 msgr2=0x7f489c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:51.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.159+0000 7f48c2c00640 1 --2- 172.21.15.86:0/3527625327 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f489c06c920 0x7f489c06ede0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f48bc1f8090 tx=0x7f48a8008040 comp rx=0 tx=0).stop 2024-04-20T21:20:51.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.159+0000 7f48c2c00640 1 -- 172.21.15.86:0/3527625327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48a00039e0 msgr2=0x7f48bc0ad850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:51.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.159+0000 7f48c2c00640 1 --2- 172.21.15.86:0/3527625327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48a00039e0 0x7f48bc0ad850 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f48ac00c210 tx=0x7f48bc15d1d0 comp rx=0 tx=0).stop 2024-04-20T21:20:51.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.159+0000 7f48c2c00640 1 -- 172.21.15.86:0/3527625327 shutdown_connections 2024-04-20T21:20:51.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.159+0000 7f48c2c00640 1 --2- 172.21.15.86:0/3527625327 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f489c06c920 0x7f489c06ede0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:51.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.159+0000 7f48c2c00640 1 --2- 172.21.15.86:0/3527625327 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f48bc1f4b60 0x7f48bc1f4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:51.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.159+0000 7f48c2c00640 1 --2- 172.21.15.86:0/3527625327 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48bc0b5090 0x7f48bc1e7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:51.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.159+0000 7f48c2c00640 1 --2- 172.21.15.86:0/3527625327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f48a00039e0 0x7f48bc0ad850 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:51.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.159+0000 7f48c2c00640 1 -- 172.21.15.86:0/3527625327 >> 172.21.15.86:0/3527625327 conn(0x7f48bc0b2110 msgr2=0x7f48bc0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:51.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.159+0000 7f48c2c00640 1 -- 172.21.15.86:0/3527625327 shutdown_connections 2024-04-20T21:20:51.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.159+0000 7f48c2c00640 1 -- 172.21.15.86:0/3527625327 wait complete. 2024-04-20T21:20:51.177 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:20:51.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.376+0000 7f5752600640 1 Processor -- start 2024-04-20T21:20:51.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.376+0000 7f5752600640 1 -- start start 2024-04-20T21:20:51.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.378+0000 7f5752600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f574c15c010 0x7f574c15c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:51.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.378+0000 7f5752600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f574c0af7b0 0x7f574c0adda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:51.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.378+0000 7f5752600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f574c0ae710 con 0x7f574c06f4c0 2024-04-20T21:20:51.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.378+0000 7f5752600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f574c0af320 con 0x7f574c15c010 2024-04-20T21:20:51.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.378+0000 7f5752600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f574c15c930 con 0x7f574c0af7b0 2024-04-20T21:20:51.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.378+0000 7f5751200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f574c15c010 0x7f574c15c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:51.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.378+0000 7f5751200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f574c15c010 0x7f574c15c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39690/0 (socket says 172.21.15.86:39690) 2024-04-20T21:20:51.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.378+0000 7f5751200640 1 -- 172.21.15.86:0/1608988547 learned_addr learned my addr 172.21.15.86:0/1608988547 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:51.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.378+0000 7f574be00640 1 --2- 172.21.15.86:0/1608988547 >> v2:172.21.15.144:3300/0 conn(0x7f574c0af7b0 0x7f574c0adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:51.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.379+0000 7f5751200640 1 -- 172.21.15.86:0/1608988547 >> v2:172.21.15.144:3300/0 conn(0x7f574c0af7b0 msgr2=0x7f574c0adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:51.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.379+0000 7f5751200640 1 --2- 172.21.15.86:0/1608988547 >> v2:172.21.15.144:3300/0 conn(0x7f574c0af7b0 0x7f574c0adda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:51.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.379+0000 7f5751200640 1 -- 172.21.15.86:0/1608988547 >> v1:172.21.15.144:6789/0 conn(0x7f574c06f4c0 legacy=0x7f574c15ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:51.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.379+0000 7f5751200640 1 -- 172.21.15.86:0/1608988547 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f574c0ae9f0 con 0x7f574c15c010 2024-04-20T21:20:51.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.380+0000 7f5751200640 1 --2- 172.21.15.86:0/1608988547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f574c15c010 0x7f574c15c3f0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f574c145230 tx=0x7f574002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20cf9d38b97800c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:51.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.380+0000 7f574b400640 1 -- 172.21.15.86:0/1608988547 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5740035020 con 0x7f574c15c010 2024-04-20T21:20:51.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.380+0000 7f574b400640 1 -- 172.21.15.86:0/1608988547 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5740031870 con 0x7f574c15c010 2024-04-20T21:20:51.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.380+0000 7f574b400640 1 -- 172.21.15.86:0/1608988547 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5740031c40 con 0x7f574c15c010 2024-04-20T21:20:51.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.381+0000 7f5752600640 1 -- 172.21.15.86:0/1608988547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f574c15c010 msgr2=0x7f574c15c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:51.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.381+0000 7f5752600640 1 --2- 172.21.15.86:0/1608988547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f574c15c010 0x7f574c15c3f0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f574c145230 tx=0x7f574002b910 comp rx=0 tx=0).stop 2024-04-20T21:20:51.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.381+0000 7f5752600640 1 -- 172.21.15.86:0/1608988547 shutdown_connections 2024-04-20T21:20:51.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.381+0000 7f5752600640 1 --2- 172.21.15.86:0/1608988547 >> v2:172.21.15.144:3300/0 conn(0x7f574c0af7b0 0x7f574c0adda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:51.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.381+0000 7f5752600640 1 --2- 172.21.15.86:0/1608988547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f574c15c010 0x7f574c15c3f0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:51.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.381+0000 7f5752600640 1 -- 172.21.15.86:0/1608988547 >> 172.21.15.86:0/1608988547 conn(0x7f574c0d0170 msgr2=0x7f574c0d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:51.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.381+0000 7f5752600640 1 -- 172.21.15.86:0/1608988547 shutdown_connections 2024-04-20T21:20:51.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.381+0000 7f5752600640 1 -- 172.21.15.86:0/1608988547 wait complete. 2024-04-20T21:20:51.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.383+0000 7f5752600640 1 Processor -- start 2024-04-20T21:20:51.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.383+0000 7f5752600640 1 -- start start 2024-04-20T21:20:51.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.384+0000 7f5752600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f574c06f4c0 0x7f574c09b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:51.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.384+0000 7f5752600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f574c0af7b0 0x7f574c09d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:51.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.384+0000 7f5752600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f574c15c010 0x7f574c09dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:51.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.384+0000 7f5752600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f574c0ae2e0 con 0x7f574c15c010 2024-04-20T21:20:51.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.384+0000 7f5752600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f574c0aefb0 con 0x7f574c0af7b0 2024-04-20T21:20:51.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.384+0000 7f5752600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f574c0ae460 con 0x7f574c06f4c0 2024-04-20T21:20:51.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.384+0000 7f5751c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f574c15c010 0x7f574c09dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:51.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.384+0000 7f5751c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f574c15c010 0x7f574c09dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39698/0 (socket says 172.21.15.86:39698) 2024-04-20T21:20:51.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.384+0000 7f5751c00640 1 -- 172.21.15.86:0/4193697806 learned_addr learned my addr 172.21.15.86:0/4193697806 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:51.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.384+0000 7f5751200640 1 --2- 172.21.15.86:0/4193697806 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f574c06f4c0 0x7f574c09b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:51.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.384+0000 7f574be00640 1 --2- 172.21.15.86:0/4193697806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f574c0af7b0 0x7f574c09d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:51.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.384+0000 7f5751c00640 1 -- 172.21.15.86:0/4193697806 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f574c06f4c0 msgr2=0x7f574c09b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:51.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.384+0000 7f5751c00640 1 --2- 172.21.15.86:0/4193697806 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f574c06f4c0 0x7f574c09b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:51.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.384+0000 7f5751c00640 1 -- 172.21.15.86:0/4193697806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f574c0af7b0 msgr2=0x7f574c09d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:51.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.384+0000 7f5751c00640 1 --2- 172.21.15.86:0/4193697806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f574c0af7b0 0x7f574c09d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:51.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.384+0000 7f5751c00640 1 -- 172.21.15.86:0/4193697806 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f574c09a4b0 con 0x7f574c15c010 2024-04-20T21:20:51.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.385+0000 7f5751c00640 1 --2- 172.21.15.86:0/4193697806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f574c15c010 0x7f574c09dc80 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f574400c780 tx=0x7f574400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:51.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.385+0000 7f5748c00640 1 -- 172.21.15.86:0/4193697806 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5744017070 con 0x7f574c15c010 2024-04-20T21:20:51.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.385+0000 7f5748c00640 1 -- 172.21.15.86:0/4193697806 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5744003da0 con 0x7f574c15c010 2024-04-20T21:20:51.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.385+0000 7f5748c00640 1 -- 172.21.15.86:0/4193697806 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57440052b0 con 0x7f574c15c010 2024-04-20T21:20:51.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.385+0000 7f5752600640 1 -- 172.21.15.86:0/4193697806 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f574c09a790 con 0x7f574c15c010 2024-04-20T21:20:51.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.385+0000 7f5752600640 1 -- 172.21.15.86:0/4193697806 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f574c09ad50 con 0x7f574c15c010 2024-04-20T21:20:51.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.386+0000 7f5752600640 1 -- 172.21.15.86:0/4193697806 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f574c0694e0 con 0x7f574c15c010 2024-04-20T21:20:51.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.387+0000 7f5748c00640 1 -- 172.21.15.86:0/4193697806 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5744005450 con 0x7f574c15c010 2024-04-20T21:20:51.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.389+0000 7f5748c00640 1 --2- 172.21.15.86:0/4193697806 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f572c06c8f0 0x7f572c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:51.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.389+0000 7f5748c00640 1 -- 172.21.15.86:0/4193697806 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f574408b1b0 con 0x7f574c15c010 2024-04-20T21:20:51.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.389+0000 7f5751200640 1 --2- 172.21.15.86:0/4193697806 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f572c06c8f0 0x7f572c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:51.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.390+0000 7f5751200640 1 --2- 172.21.15.86:0/4193697806 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f572c06c8f0 0x7f572c06edb0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5740036000 tx=0x7f5740008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:51.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.393+0000 7f5748c00640 1 -- 172.21.15.86:0/4193697806 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f574c0694e0 con 0x7f574c15c010 2024-04-20T21:20:51.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.629+0000 7f5752600640 1 -- 172.21.15.86:0/4193697806 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f574c15c480 con 0x7f574c15c010 2024-04-20T21:20:51.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.631+0000 7f5748c00640 1 -- 172.21.15.86:0/4193697806 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5199 (secure 0 0 0) 0x7f5744059500 con 0x7f574c15c010 2024-04-20T21:20:51.630 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:20:51.630 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5044,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/1013106052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":1013106052},{"type":"v1","addr":"172.21.15.144:6833","nonce":1013106052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5047,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/1073576945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1073576945},{"type":"v1","addr":"172.21.15.144:6835","nonce":1073576945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/821548012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":821548012},{"type":"v1","addr":"172.21.15.86:6835","nonce":821548012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:20:03.369281+0000","modified":"2024-04-20T21:20:06.438592+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5691},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5691":{"gid":5691,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/712628984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":712628984},{"type":"v1","addr":"172.21.15.86:6837","nonce":712628984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5691,"qdb_cluster":[5691]},"id":3}]} 2024-04-20T21:20:51.630 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 24 2024-04-20T21:20:51.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.633+0000 7f5752600640 1 -- 172.21.15.86:0/4193697806 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f572c06c8f0 msgr2=0x7f572c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:51.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.633+0000 7f5752600640 1 --2- 172.21.15.86:0/4193697806 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f572c06c8f0 0x7f572c06edb0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5740036000 tx=0x7f5740008030 comp rx=0 tx=0).stop 2024-04-20T21:20:51.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.633+0000 7f5752600640 1 -- 172.21.15.86:0/4193697806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f574c15c010 msgr2=0x7f574c09dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:51.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.633+0000 7f5752600640 1 --2- 172.21.15.86:0/4193697806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f574c15c010 0x7f574c09dc80 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f574400c780 tx=0x7f574400cc40 comp rx=0 tx=0).stop 2024-04-20T21:20:51.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.634+0000 7f5752600640 1 -- 172.21.15.86:0/4193697806 shutdown_connections 2024-04-20T21:20:51.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.634+0000 7f5752600640 1 --2- 172.21.15.86:0/4193697806 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f572c06c8f0 0x7f572c06edb0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:51.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.634+0000 7f5752600640 1 --2- 172.21.15.86:0/4193697806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f574c15c010 0x7f574c09dc80 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:51.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.634+0000 7f5752600640 1 --2- 172.21.15.86:0/4193697806 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f574c0af7b0 0x7f574c09d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:51.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.634+0000 7f5752600640 1 --2- 172.21.15.86:0/4193697806 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f574c06f4c0 0x7f574c09b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:51.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.634+0000 7f5752600640 1 -- 172.21.15.86:0/4193697806 >> 172.21.15.86:0/4193697806 conn(0x7f574c0d0170 msgr2=0x7f574c073720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:51.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.634+0000 7f5752600640 1 -- 172.21.15.86:0/4193697806 shutdown_connections 2024-04-20T21:20:51.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.634+0000 7f5752600640 1 -- 172.21.15.86:0/4193697806 wait complete. 2024-04-20T21:20:51.650 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:20:51.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.848+0000 7f1dce000640 1 Processor -- start 2024-04-20T21:20:51.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.849+0000 7f1dce000640 1 -- start start 2024-04-20T21:20:51.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.850+0000 7f1dce000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1dc80d17e0 0x7f1dc80d4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:51.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.850+0000 7f1dce000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dc80d5480 con 0x7f1dc80d1400 2024-04-20T21:20:51.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.850+0000 7f1dce000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dc80cf580 con 0x7f1dc80d0060 2024-04-20T21:20:51.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.850+0000 7f1dce000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f1dc80cf860 con 0x7f1dc80d17e0 2024-04-20T21:20:51.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.851+0000 7f1dc7400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f1dc80d1400 0x7f1dc8162ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40448/0 (socket says 172.21.15.86:40448) 2024-04-20T21:20:51.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.851+0000 7f1dc7400640 1 -- 172.21.15.86:0/3438050494 learned_addr learned my addr 172.21.15.86:0/3438050494 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:20:51.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.851+0000 7f1dc6a00640 1 --2- 172.21.15.86:0/3438050494 >> v2:172.21.15.144:3300/0 conn(0x7f1dc80d17e0 0x7f1dc80d4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:51.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.851+0000 7f1dc6000640 1 -- 172.21.15.86:0/3438050494 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1514993963 0 0) 0x7f1dc80d5480 con 0x7f1dc80d1400 2024-04-20T21:20:51.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.851+0000 7f1dc6000640 1 -- 172.21.15.86:0/3438050494 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1dac003520 con 0x7f1dc80d1400 2024-04-20T21:20:51.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.852+0000 7f1dc6000640 1 -- 172.21.15.86:0/3438050494 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 312608965 0 0) 0x7f1dac003520 con 0x7f1dc80d1400 2024-04-20T21:20:51.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.852+0000 7f1dc6000640 1 -- 172.21.15.86:0/3438050494 >> v2:172.21.15.144:3300/0 conn(0x7f1dc80d17e0 msgr2=0x7f1dc80d4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:51.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.852+0000 7f1dc6000640 1 --2- 172.21.15.86:0/3438050494 >> v2:172.21.15.144:3300/0 conn(0x7f1dc80d17e0 0x7f1dc80d4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:51.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.852+0000 7f1dc6000640 1 -- 172.21.15.86:0/3438050494 >> v1:172.21.15.144:6789/0 conn(0x7f1dc80d0060 legacy=0x7f1dc8161a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:51.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.852+0000 7f1dc6000640 1 -- 172.21.15.86:0/3438050494 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc80cfb40 con 0x7f1dc80d1400 2024-04-20T21:20:51.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.852+0000 7f1dc6000640 1 -- 172.21.15.86:0/3438050494 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f1db80019f0 con 0x7f1dc80d1400 2024-04-20T21:20:51.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.852+0000 7f1dc6000640 1 -- 172.21.15.86:0/3438050494 >> v1:172.21.15.86:6789/0 conn(0x7f1dc80d1400 legacy=0x7f1dc8162ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:51.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.852+0000 7f1dc6000640 1 --2- 172.21.15.86:0/3438050494 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1dac003a00 0x7f1dac003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:51.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.852+0000 7f1dc6000640 1 --2- 172.21.15.86:0/3438050494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1dac004800 0x7f1dac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:51.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.852+0000 7f1dc6000640 1 --2- 172.21.15.86:0/3438050494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1dac0051c0 0x7f1dac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:51.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.852+0000 7f1dc6000640 1 -- 172.21.15.86:0/3438050494 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc80cf860 con 0x7f1dac004800 2024-04-20T21:20:51.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.852+0000 7f1dc6000640 1 -- 172.21.15.86:0/3438050494 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac00c0c0 con 0x7f1dac0051c0 2024-04-20T21:20:51.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.852+0000 7f1dc6000640 1 -- 172.21.15.86:0/3438050494 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1dac00c2c0 con 0x7f1dac003a00 2024-04-20T21:20:51.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.852+0000 7f1dc6a00640 1 --2- 172.21.15.86:0/3438050494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1dac004800 0x7f1dac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:51.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.852+0000 7f1dc7400640 1 --2- 172.21.15.86:0/3438050494 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1dac003a00 0x7f1dac003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:51.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.852+0000 7f1dc7e00640 1 --2- 172.21.15.86:0/3438050494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1dac0051c0 0x7f1dac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:51.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.853+0000 7f1dc6a00640 1 -- 172.21.15.86:0/3438050494 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1dac003a00 msgr2=0x7f1dac003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:51.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.853+0000 7f1dc6a00640 1 --2- 172.21.15.86:0/3438050494 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1dac003a00 0x7f1dac003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:51.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.853+0000 7f1dc6a00640 1 -- 172.21.15.86:0/3438050494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1dac0051c0 msgr2=0x7f1dac00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:51.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.853+0000 7f1dc6a00640 1 --2- 172.21.15.86:0/3438050494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1dac0051c0 0x7f1dac00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:51.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.853+0000 7f1dc6a00640 1 -- 172.21.15.86:0/3438050494 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dac003520 con 0x7f1dac004800 2024-04-20T21:20:51.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.853+0000 7f1dc7e00640 1 --2- 172.21.15.86:0/3438050494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1dac0051c0 0x7f1dac00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:20:51.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.853+0000 7f1dc7e00640 1 -- 172.21.15.86:0/3438050494 reap_dead start 2024-04-20T21:20:51.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.854+0000 7f1dc6a00640 1 --2- 172.21.15.86:0/3438050494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1dac004800 0x7f1dac004c80 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f1db0009a80 tx=0x7f1dac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93ab9a62ff9daca2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:51.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.854+0000 7f1dc6000640 1 -- 172.21.15.86:0/3438050494 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db0009680 con 0x7f1dac004800 2024-04-20T21:20:51.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.854+0000 7f1dc6000640 1 -- 172.21.15.86:0/3438050494 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1db0005990 con 0x7f1dac004800 2024-04-20T21:20:51.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.854+0000 7f1dc6000640 1 -- 172.21.15.86:0/3438050494 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db0005d80 con 0x7f1dac004800 2024-04-20T21:20:51.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.855+0000 7f1dce000640 1 -- 172.21.15.86:0/3438050494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1dac004800 msgr2=0x7f1dac004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:51.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.855+0000 7f1dce000640 1 --2- 172.21.15.86:0/3438050494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1dac004800 0x7f1dac004c80 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f1db0009a80 tx=0x7f1dac005830 comp rx=0 tx=0).stop 2024-04-20T21:20:51.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.855+0000 7f1dce000640 1 -- 172.21.15.86:0/3438050494 shutdown_connections 2024-04-20T21:20:51.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.855+0000 7f1dce000640 1 --2- 172.21.15.86:0/3438050494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1dac004800 0x7f1dac004c80 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:51.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.855+0000 7f1dce000640 1 -- 172.21.15.86:0/3438050494 >> 172.21.15.86:0/3438050494 conn(0x7f1dc808bf70 msgr2=0x7f1dc80d4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:51.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.855+0000 7f1dce000640 1 -- 172.21.15.86:0/3438050494 shutdown_connections 2024-04-20T21:20:51.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.855+0000 7f1dce000640 1 -- 172.21.15.86:0/3438050494 wait complete. 2024-04-20T21:20:51.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.857+0000 7f1dce000640 1 Processor -- start 2024-04-20T21:20:51.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.857+0000 7f1dce000640 1 -- start start 2024-04-20T21:20:51.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.858+0000 7f1dce000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1dac004800 0x7f1dc8154f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:51.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.858+0000 7f1dce000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1dc81549d0 0x7f1dc8154220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:51.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.858+0000 7f1dce000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1dc80cdcf0 0x7f1dc80ce170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:51.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.858+0000 7f1dce000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc80d4e30 con 0x7f1dac004800 2024-04-20T21:20:51.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.858+0000 7f1dce000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc80d0510 con 0x7f1dc81549d0 2024-04-20T21:20:51.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.858+0000 7f1dce000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1dc80d4fb0 con 0x7f1dc80cdcf0 2024-04-20T21:20:51.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.858+0000 7f1dc7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1dac004800 0x7f1dc8154f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:51.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.858+0000 7f1dc7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1dac004800 0x7f1dc8154f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39720/0 (socket says 172.21.15.86:39720) 2024-04-20T21:20:51.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.858+0000 7f1dc7400640 1 -- 172.21.15.86:0/2520333973 learned_addr learned my addr 172.21.15.86:0/2520333973 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:51.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.858+0000 7f1dc6a00640 1 --2- 172.21.15.86:0/2520333973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1dc81549d0 0x7f1dc8154220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:51.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.858+0000 7f1dc7e00640 1 --2- 172.21.15.86:0/2520333973 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1dc80cdcf0 0x7f1dc80ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:51.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.858+0000 7f1dc7400640 1 -- 172.21.15.86:0/2520333973 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1dc80cdcf0 msgr2=0x7f1dc80ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:51.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.858+0000 7f1dc7400640 1 --2- 172.21.15.86:0/2520333973 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1dc80cdcf0 0x7f1dc80ce170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:51.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.858+0000 7f1dc7400640 1 -- 172.21.15.86:0/2520333973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1dc81549d0 msgr2=0x7f1dc8154220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:51.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.858+0000 7f1dc7400640 1 --2- 172.21.15.86:0/2520333973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1dc81549d0 0x7f1dc8154220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:51.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.858+0000 7f1dc7400640 1 -- 172.21.15.86:0/2520333973 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc80cc200 con 0x7f1dac004800 2024-04-20T21:20:51.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.858+0000 7f1dc6a00640 1 --2- 172.21.15.86:0/2520333973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1dc81549d0 0x7f1dc8154220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:20:51.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.859+0000 7f1dc7400640 1 --2- 172.21.15.86:0/2520333973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1dac004800 0x7f1dc8154f00 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f1db8007a30 tx=0x7f1db800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:51.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.859+0000 7f1dbf400640 1 -- 172.21.15.86:0/2520333973 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db800ee50 con 0x7f1dac004800 2024-04-20T21:20:51.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.859+0000 7f1dbf400640 1 -- 172.21.15.86:0/2520333973 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1db8018070 con 0x7f1dac004800 2024-04-20T21:20:51.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.859+0000 7f1dbf400640 1 -- 172.21.15.86:0/2520333973 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db8013660 con 0x7f1dac004800 2024-04-20T21:20:51.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.859+0000 7f1dce000640 1 -- 172.21.15.86:0/2520333973 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dc80cc4e0 con 0x7f1dac004800 2024-04-20T21:20:51.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.859+0000 7f1dce000640 1 -- 172.21.15.86:0/2520333973 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dc80cbc00 con 0x7f1dac004800 2024-04-20T21:20:51.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.860+0000 7f1dce000640 1 -- 172.21.15.86:0/2520333973 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dc80c8890 con 0x7f1dac004800 2024-04-20T21:20:51.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.861+0000 7f1dbf400640 1 -- 172.21.15.86:0/2520333973 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1db8013800 con 0x7f1dac004800 2024-04-20T21:20:51.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.863+0000 7f1dbf400640 1 --2- 172.21.15.86:0/2520333973 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1da406c920 0x7f1da406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:51.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.863+0000 7f1dbf400640 1 -- 172.21.15.86:0/2520333973 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f1db808c280 con 0x7f1dac004800 2024-04-20T21:20:51.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.863+0000 7f1dc6a00640 1 --2- 172.21.15.86:0/2520333973 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1da406c920 0x7f1da406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:51.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.864+0000 7f1dc6a00640 1 --2- 172.21.15.86:0/2520333973 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1da406c920 0x7f1da406ede0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1db0035000 tx=0x7f1db0030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:51.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:51.867+0000 7f1dbf400640 1 -- 172.21.15.86:0/2520333973 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1dc80c8890 con 0x7f1dac004800 2024-04-20T21:20:52.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.071+0000 7f1dce000640 1 -- 172.21.15.86:0/2520333973 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1dc80689e0 con 0x7f1dac004800 2024-04-20T21:20:52.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.074+0000 7f1dbf400640 1 -- 172.21.15.86:0/2520333973 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16704 (secure 0 0 0) 0x7f1db805a660 con 0x7f1dac004800 2024-04-20T21:20:52.073 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:20:52.073 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":47,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:20:34.772220+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:20:01.307757+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-20T21:20:02.359347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.6","mappings":[{"from":0,"to":4}]},{"pgid":"7.36","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:20:52.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.076+0000 7f1dce000640 1 -- 172.21.15.86:0/2520333973 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1da406c920 msgr2=0x7f1da406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:52.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.076+0000 7f1dce000640 1 --2- 172.21.15.86:0/2520333973 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1da406c920 0x7f1da406ede0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1db0035000 tx=0x7f1db0030000 comp rx=0 tx=0).stop 2024-04-20T21:20:52.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.076+0000 7f1dce000640 1 -- 172.21.15.86:0/2520333973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1dac004800 msgr2=0x7f1dc8154f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:52.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.076+0000 7f1dce000640 1 --2- 172.21.15.86:0/2520333973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1dac004800 0x7f1dc8154f00 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f1db8007a30 tx=0x7f1db800e530 comp rx=0 tx=0).stop 2024-04-20T21:20:52.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.076+0000 7f1dce000640 1 -- 172.21.15.86:0/2520333973 shutdown_connections 2024-04-20T21:20:52.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.076+0000 7f1dce000640 1 --2- 172.21.15.86:0/2520333973 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1da406c920 0x7f1da406ede0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:52.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.076+0000 7f1dce000640 1 --2- 172.21.15.86:0/2520333973 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1dc80cdcf0 0x7f1dc80ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:52.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.076+0000 7f1dce000640 1 --2- 172.21.15.86:0/2520333973 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1dc81549d0 0x7f1dc8154220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:52.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.076+0000 7f1dce000640 1 --2- 172.21.15.86:0/2520333973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1dac004800 0x7f1dc8154f00 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:52.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.076+0000 7f1dce000640 1 -- 172.21.15.86:0/2520333973 >> 172.21.15.86:0/2520333973 conn(0x7f1dc808bf70 msgr2=0x7f1dc8160c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:52.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.077+0000 7f1dce000640 1 -- 172.21.15.86:0/2520333973 shutdown_connections 2024-04-20T21:20:52.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.077+0000 7f1dce000640 1 -- 172.21.15.86:0/2520333973 wait complete. 2024-04-20T21:20:52.095 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-20T21:20:52.095 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:20:52.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.284+0000 7f19b1000640 1 Processor -- start 2024-04-20T21:20:52.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.284+0000 7f19b1000640 1 -- start start 2024-04-20T21:20:52.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.285+0000 7f19b1000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f19ac15d6a0 0x7f19ac0ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:52.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.285+0000 7f19b1000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f19ac0ad620 0x7f19ac06e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:52.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.285+0000 7f19b1000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19ac160890 con 0x7f19ac156ae0 2024-04-20T21:20:52.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.285+0000 7f19b1000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f19ac0ada90 con 0x7f19ac15d6a0 2024-04-20T21:20:52.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.285+0000 7f19b1000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f19ac06db20 con 0x7f19ac0ad620 2024-04-20T21:20:52.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.285+0000 7f19aaa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f19ac15d6a0 0x7f19ac0ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:52.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.286+0000 7f19aaa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f19ac15d6a0 0x7f19ac0ad0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39734/0 (socket says 172.21.15.86:39734) 2024-04-20T21:20:52.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.286+0000 7f19aaa00640 1 -- 172.21.15.86:0/2635108340 learned_addr learned my addr 172.21.15.86:0/2635108340 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:52.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.286+0000 7f19aaa00640 1 -- 172.21.15.86:0/2635108340 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f19ac0ad620 msgr2=0x7f19ac06e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:52.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.286+0000 7f19abe00640 1 --2- 172.21.15.86:0/2635108340 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f19ac0ad620 0x7f19ac06e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:52.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.286+0000 7f19aaa00640 1 --2- 172.21.15.86:0/2635108340 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f19ac0ad620 0x7f19ac06e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:52.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.286+0000 7f19aaa00640 1 -- 172.21.15.86:0/2635108340 >> v1:172.21.15.144:6789/0 conn(0x7f19ac156ae0 legacy=0x7f19ac160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:52.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.286+0000 7f19aaa00640 1 -- 172.21.15.86:0/2635108340 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19ac160b70 con 0x7f19ac15d6a0 2024-04-20T21:20:52.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.287+0000 7f19aaa00640 1 --2- 172.21.15.86:0/2635108340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19ac15d6a0 0x7f19ac0ad0e0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f1998009870 tx=0x7f199802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6267c9c38e690b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:52.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.287+0000 7f19aa000640 1 -- 172.21.15.86:0/2635108340 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1998035020 con 0x7f19ac15d6a0 2024-04-20T21:20:52.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.287+0000 7f19aa000640 1 -- 172.21.15.86:0/2635108340 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1998031640 con 0x7f19ac15d6a0 2024-04-20T21:20:52.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.287+0000 7f19aa000640 1 -- 172.21.15.86:0/2635108340 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199802d040 con 0x7f19ac15d6a0 2024-04-20T21:20:52.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.288+0000 7f19b1000640 1 -- 172.21.15.86:0/2635108340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19ac15d6a0 msgr2=0x7f19ac0ad0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:52.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.288+0000 7f19b1000640 1 --2- 172.21.15.86:0/2635108340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19ac15d6a0 0x7f19ac0ad0e0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f1998009870 tx=0x7f199802b8f0 comp rx=0 tx=0).stop 2024-04-20T21:20:52.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.288+0000 7f19b1000640 1 -- 172.21.15.86:0/2635108340 shutdown_connections 2024-04-20T21:20:52.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.288+0000 7f19b1000640 1 --2- 172.21.15.86:0/2635108340 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f19ac0ad620 0x7f19ac06e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:52.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.288+0000 7f19b1000640 1 --2- 172.21.15.86:0/2635108340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19ac15d6a0 0x7f19ac0ad0e0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:52.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.288+0000 7f19b1000640 1 -- 172.21.15.86:0/2635108340 >> 172.21.15.86:0/2635108340 conn(0x7f19ac0d0170 msgr2=0x7f19ac0d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:52.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.288+0000 7f19b1000640 1 -- 172.21.15.86:0/2635108340 shutdown_connections 2024-04-20T21:20:52.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.288+0000 7f19b1000640 1 -- 172.21.15.86:0/2635108340 wait complete. 2024-04-20T21:20:52.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.290+0000 7f19b1000640 1 Processor -- start 2024-04-20T21:20:52.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.290+0000 7f19b1000640 1 -- start start 2024-04-20T21:20:52.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.291+0000 7f19b1000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19ac0ad620 0x7f19ac15ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:52.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.291+0000 7f19b1000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19ac156ae0 0x7f19ac1f3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:52.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.291+0000 7f19b1000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f19ac15d6a0 0x7f19ac1f44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:52.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.291+0000 7f19b1000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f19ac156ec0 con 0x7f19ac0ad620 2024-04-20T21:20:52.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.291+0000 7f19b1000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f19ac06dd70 con 0x7f19ac156ae0 2024-04-20T21:20:52.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.291+0000 7f19b1000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f19ac06e7d0 con 0x7f19ac15d6a0 2024-04-20T21:20:52.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.291+0000 7f19ab400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19ac0ad620 0x7f19ac15ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:52.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.291+0000 7f19ab400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19ac0ad620 0x7f19ac15ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39738/0 (socket says 172.21.15.86:39738) 2024-04-20T21:20:52.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.291+0000 7f19ab400640 1 -- 172.21.15.86:0/450068137 learned_addr learned my addr 172.21.15.86:0/450068137 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:52.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.291+0000 7f19abe00640 1 --2- 172.21.15.86:0/450068137 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f19ac15d6a0 0x7f19ac1f44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:52.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.292+0000 7f19ab400640 1 -- 172.21.15.86:0/450068137 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f19ac15d6a0 msgr2=0x7f19ac1f44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:52.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.292+0000 7f19aaa00640 1 --2- 172.21.15.86:0/450068137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19ac156ae0 0x7f19ac1f3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:52.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.292+0000 7f19ab400640 1 --2- 172.21.15.86:0/450068137 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f19ac15d6a0 0x7f19ac1f44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:52.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.292+0000 7f19ab400640 1 -- 172.21.15.86:0/450068137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19ac156ae0 msgr2=0x7f19ac1f3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:52.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.292+0000 7f19ab400640 1 --2- 172.21.15.86:0/450068137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19ac156ae0 0x7f19ac1f3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:52.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.292+0000 7f19ab400640 1 -- 172.21.15.86:0/450068137 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19ac157040 con 0x7f19ac0ad620 2024-04-20T21:20:52.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.292+0000 7f19ab400640 1 --2- 172.21.15.86:0/450068137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19ac0ad620 0x7f19ac15ed90 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f199c00c460 tx=0x7f199c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:52.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.292+0000 7f19a3400640 1 -- 172.21.15.86:0/450068137 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199c016020 con 0x7f19ac0ad620 2024-04-20T21:20:52.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.292+0000 7f19a3400640 1 -- 172.21.15.86:0/450068137 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f199c012870 con 0x7f19ac0ad620 2024-04-20T21:20:52.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.292+0000 7f19b1000640 1 -- 172.21.15.86:0/450068137 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19ac157320 con 0x7f19ac0ad620 2024-04-20T21:20:52.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.292+0000 7f19a3400640 1 -- 172.21.15.86:0/450068137 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199c012c60 con 0x7f19ac0ad620 2024-04-20T21:20:52.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.292+0000 7f19b1000640 1 -- 172.21.15.86:0/450068137 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19ac1578e0 con 0x7f19ac0ad620 2024-04-20T21:20:52.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.293+0000 7f19b1000640 1 -- 172.21.15.86:0/450068137 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19ac0d0fd0 con 0x7f19ac0ad620 2024-04-20T21:20:52.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.294+0000 7f19a3400640 1 -- 172.21.15.86:0/450068137 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f199c019020 con 0x7f19ac0ad620 2024-04-20T21:20:52.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.295+0000 7f19a3400640 1 --2- 172.21.15.86:0/450068137 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f198c06c8f0 0x7f198c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:52.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.295+0000 7f19a3400640 1 -- 172.21.15.86:0/450068137 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f199c08b6b0 con 0x7f19ac0ad620 2024-04-20T21:20:52.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.295+0000 7f19aaa00640 1 --2- 172.21.15.86:0/450068137 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f198c06c8f0 0x7f198c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:52.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.296+0000 7f19aaa00640 1 --2- 172.21.15.86:0/450068137 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f198c06c8f0 0x7f198c06edb0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1998036000 tx=0x7f1998008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:52.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.299+0000 7f19a3400640 1 -- 172.21.15.86:0/450068137 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f19ac0d0fd0 con 0x7f19ac0ad620 2024-04-20T21:20:52.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.519+0000 7f19b1000640 1 -- 172.21.15.86:0/450068137 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f19ac06fb40 con 0x7f19ac0ad620 2024-04-20T21:20:52.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.521+0000 7f19a3400640 1 -- 172.21.15.86:0/450068137 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5199 (secure 0 0 0) 0x7f199c020070 con 0x7f19ac0ad620 2024-04-20T21:20:52.521 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:20:52.521 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5044,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/1013106052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":1013106052},{"type":"v1","addr":"172.21.15.144:6833","nonce":1013106052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5047,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/1073576945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1073576945},{"type":"v1","addr":"172.21.15.144:6835","nonce":1073576945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/821548012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":821548012},{"type":"v1","addr":"172.21.15.86:6835","nonce":821548012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:20:03.369281+0000","modified":"2024-04-20T21:20:06.438592+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5691},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5691":{"gid":5691,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/712628984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":712628984},{"type":"v1","addr":"172.21.15.86:6837","nonce":712628984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5691,"qdb_cluster":[5691]},"id":3}]} 2024-04-20T21:20:52.521 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 24 2024-04-20T21:20:52.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.524+0000 7f19b1000640 1 -- 172.21.15.86:0/450068137 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f198c06c8f0 msgr2=0x7f198c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:52.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.524+0000 7f19b1000640 1 --2- 172.21.15.86:0/450068137 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f198c06c8f0 0x7f198c06edb0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1998036000 tx=0x7f1998008030 comp rx=0 tx=0).stop 2024-04-20T21:20:52.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.524+0000 7f19b1000640 1 -- 172.21.15.86:0/450068137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19ac0ad620 msgr2=0x7f19ac15ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:52.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.524+0000 7f19b1000640 1 --2- 172.21.15.86:0/450068137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19ac0ad620 0x7f19ac15ed90 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f199c00c460 tx=0x7f199c00c920 comp rx=0 tx=0).stop 2024-04-20T21:20:52.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.524+0000 7f19b1000640 1 -- 172.21.15.86:0/450068137 shutdown_connections 2024-04-20T21:20:52.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.524+0000 7f19b1000640 1 --2- 172.21.15.86:0/450068137 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f198c06c8f0 0x7f198c06edb0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:52.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.524+0000 7f19b1000640 1 --2- 172.21.15.86:0/450068137 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f19ac15d6a0 0x7f19ac1f44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:52.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.524+0000 7f19b1000640 1 --2- 172.21.15.86:0/450068137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19ac156ae0 0x7f19ac1f3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:52.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.524+0000 7f19b1000640 1 --2- 172.21.15.86:0/450068137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19ac0ad620 0x7f19ac15ed90 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:52.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.524+0000 7f19b1000640 1 -- 172.21.15.86:0/450068137 >> 172.21.15.86:0/450068137 conn(0x7f19ac0d0170 msgr2=0x7f19ac15cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:52.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.524+0000 7f19b1000640 1 -- 172.21.15.86:0/450068137 shutdown_connections 2024-04-20T21:20:52.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.524+0000 7f19b1000640 1 -- 172.21.15.86:0/450068137 wait complete. 2024-04-20T21:20:52.540 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:20:52.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.737+0000 7f5b7a000640 1 Processor -- start 2024-04-20T21:20:52.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.737+0000 7f5b7a000640 1 -- start start 2024-04-20T21:20:52.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.739+0000 7f5b7a000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5b7415eb10 0x7f5b7409a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:52.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.739+0000 7f5b7a000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b7409b330 con 0x7f5b7409af20 2024-04-20T21:20:52.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.739+0000 7f5b7a000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b7406eaa0 con 0x7f5b7415e730 2024-04-20T21:20:52.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.739+0000 7f5b7a000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f5b740713c0 con 0x7f5b7415eb10 2024-04-20T21:20:52.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.739+0000 7f5b73e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f5b7409af20 0x7f5b741687d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40460/0 (socket says 172.21.15.86:40460) 2024-04-20T21:20:52.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.739+0000 7f5b73e00640 1 -- 172.21.15.86:0/298127839 learned_addr learned my addr 172.21.15.86:0/298127839 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:20:52.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.739+0000 7f5b72a00640 1 --2- 172.21.15.86:0/298127839 >> v2:172.21.15.144:3300/0 conn(0x7f5b7415eb10 0x7f5b7409a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:52.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.740+0000 7f5b72000640 1 -- 172.21.15.86:0/298127839 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4277485439 0 0) 0x7f5b6c001110 con 0x7f5b7409af20 2024-04-20T21:20:52.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.740+0000 7f5b72000640 1 -- 172.21.15.86:0/298127839 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b58003520 con 0x7f5b7409af20 2024-04-20T21:20:52.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.740+0000 7f5b72000640 1 -- 172.21.15.86:0/298127839 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1595855894 0 0) 0x7f5b58003520 con 0x7f5b7409af20 2024-04-20T21:20:52.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.740+0000 7f5b72000640 1 -- 172.21.15.86:0/298127839 >> v2:172.21.15.144:3300/0 conn(0x7f5b7415eb10 msgr2=0x7f5b7409a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:52.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.740+0000 7f5b72000640 1 --2- 172.21.15.86:0/298127839 >> v2:172.21.15.144:3300/0 conn(0x7f5b7415eb10 0x7f5b7409a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:52.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.740+0000 7f5b72000640 1 -- 172.21.15.86:0/298127839 >> v1:172.21.15.144:6789/0 conn(0x7f5b7415e730 legacy=0x7f5b740d1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:52.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.740+0000 7f5b72000640 1 -- 172.21.15.86:0/298127839 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b74070320 con 0x7f5b7409af20 2024-04-20T21:20:52.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.740+0000 7f5b72000640 1 -- 172.21.15.86:0/298127839 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f5b6c001840 con 0x7f5b7409af20 2024-04-20T21:20:52.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.740+0000 7f5b72000640 1 -- 172.21.15.86:0/298127839 >> v1:172.21.15.86:6789/0 conn(0x7f5b7409af20 legacy=0x7f5b741687d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:52.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.740+0000 7f5b72000640 1 --2- 172.21.15.86:0/298127839 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b580039b0 0x7f5b58003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:52.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.740+0000 7f5b72000640 1 --2- 172.21.15.86:0/298127839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b58004800 0x7f5b58004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:52.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.740+0000 7f5b72000640 1 --2- 172.21.15.86:0/298127839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b580051c0 0x7f5b5800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:52.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.740+0000 7f5b72000640 1 -- 172.21.15.86:0/298127839 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5b740713c0 con 0x7f5b580051c0 2024-04-20T21:20:52.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.740+0000 7f5b72000640 1 -- 172.21.15.86:0/298127839 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5800c070 con 0x7f5b58004800 2024-04-20T21:20:52.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.740+0000 7f5b72000640 1 -- 172.21.15.86:0/298127839 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5b5800c270 con 0x7f5b580039b0 2024-04-20T21:20:52.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.740+0000 7f5b73e00640 1 --2- 172.21.15.86:0/298127839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b580051c0 0x7f5b5800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:52.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.741+0000 7f5b72a00640 1 --2- 172.21.15.86:0/298127839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b58004800 0x7f5b58004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:52.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.741+0000 7f5b73400640 1 --2- 172.21.15.86:0/298127839 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b580039b0 0x7f5b58003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:52.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.741+0000 7f5b73e00640 1 -- 172.21.15.86:0/298127839 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b580039b0 msgr2=0x7f5b58003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:52.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.741+0000 7f5b73e00640 1 --2- 172.21.15.86:0/298127839 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b580039b0 0x7f5b58003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:52.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.741+0000 7f5b73e00640 1 -- 172.21.15.86:0/298127839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b58004800 msgr2=0x7f5b58004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:52.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.741+0000 7f5b73e00640 1 --2- 172.21.15.86:0/298127839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b58004800 0x7f5b58004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:52.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.741+0000 7f5b73e00640 1 -- 172.21.15.86:0/298127839 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b58003520 con 0x7f5b580051c0 2024-04-20T21:20:52.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.742+0000 7f5b73e00640 1 -- 172.21.15.86:0/298127839 reap_dead start 2024-04-20T21:20:52.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.742+0000 7f5b73e00640 1 --2- 172.21.15.86:0/298127839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b580051c0 0x7f5b5800b9c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c00c4c0 tx=0x7f5b58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7611057a56bb0643 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:52.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.742+0000 7f5b72000640 1 -- 172.21.15.86:0/298127839 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c00c0e0 con 0x7f5b580051c0 2024-04-20T21:20:52.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.743+0000 7f5b72000640 1 -- 172.21.15.86:0/298127839 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b6c030070 con 0x7f5b580051c0 2024-04-20T21:20:52.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.743+0000 7f5b72000640 1 -- 172.21.15.86:0/298127839 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c0058d0 con 0x7f5b580051c0 2024-04-20T21:20:52.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.743+0000 7f5b7a000640 1 -- 172.21.15.86:0/298127839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b580051c0 msgr2=0x7f5b5800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:52.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.743+0000 7f5b7a000640 1 --2- 172.21.15.86:0/298127839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b580051c0 0x7f5b5800b9c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c00c4c0 tx=0x7f5b58005830 comp rx=0 tx=0).stop 2024-04-20T21:20:52.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.743+0000 7f5b7a000640 1 -- 172.21.15.86:0/298127839 shutdown_connections 2024-04-20T21:20:52.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.743+0000 7f5b7a000640 1 --2- 172.21.15.86:0/298127839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b580051c0 0x7f5b5800b9c0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:52.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.743+0000 7f5b7a000640 1 -- 172.21.15.86:0/298127839 >> 172.21.15.86:0/298127839 conn(0x7f5b740b4080 msgr2=0x7f5b740b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:52.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.743+0000 7f5b7a000640 1 -- 172.21.15.86:0/298127839 shutdown_connections 2024-04-20T21:20:52.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.743+0000 7f5b7a000640 1 -- 172.21.15.86:0/298127839 wait complete. 2024-04-20T21:20:52.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.745+0000 7f5b7a000640 1 Processor -- start 2024-04-20T21:20:52.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.745+0000 7f5b7a000640 1 -- start start 2024-04-20T21:20:52.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.746+0000 7f5b7a000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b580051c0 0x7f5b74154c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:52.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.746+0000 7f5b7a000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b740c71e0 0x7f5b7409ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:52.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.746+0000 7f5b7a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b7409f140 0x7f5b74151130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:52.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.746+0000 7f5b7a000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5b74070bc0 con 0x7f5b7409f140 2024-04-20T21:20:52.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.746+0000 7f5b7a000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5b740708e0 con 0x7f5b740c71e0 2024-04-20T21:20:52.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.746+0000 7f5b7a000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5b74070ea0 con 0x7f5b580051c0 2024-04-20T21:20:52.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.746+0000 7f5b73e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b7409f140 0x7f5b74151130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:52.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.746+0000 7f5b73e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b7409f140 0x7f5b74151130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39768/0 (socket says 172.21.15.86:39768) 2024-04-20T21:20:52.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.746+0000 7f5b73e00640 1 -- 172.21.15.86:0/43024419 learned_addr learned my addr 172.21.15.86:0/43024419 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:52.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.747+0000 7f5b73400640 1 --2- 172.21.15.86:0/43024419 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b580051c0 0x7f5b74154c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:52.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.747+0000 7f5b72a00640 1 --2- 172.21.15.86:0/43024419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b740c71e0 0x7f5b7409ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:52.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.747+0000 7f5b73e00640 1 -- 172.21.15.86:0/43024419 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b580051c0 msgr2=0x7f5b74154c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:52.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.747+0000 7f5b73e00640 1 --2- 172.21.15.86:0/43024419 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b580051c0 0x7f5b74154c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:52.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.747+0000 7f5b73e00640 1 -- 172.21.15.86:0/43024419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b740c71e0 msgr2=0x7f5b7409ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:52.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.747+0000 7f5b73e00640 1 --2- 172.21.15.86:0/43024419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b740c71e0 0x7f5b7409ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:52.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.747+0000 7f5b73e00640 1 -- 172.21.15.86:0/43024419 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b740c7660 con 0x7f5b7409f140 2024-04-20T21:20:52.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.747+0000 7f5b72a00640 1 --2- 172.21.15.86:0/43024419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b740c71e0 0x7f5b7409ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:20:52.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.747+0000 7f5b73e00640 1 --2- 172.21.15.86:0/43024419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b7409f140 0x7f5b74151130 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c00c5e0 tx=0x7f5b6c001c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:52.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.747+0000 7f5b6b400640 1 -- 172.21.15.86:0/43024419 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c039030 con 0x7f5b7409f140 2024-04-20T21:20:52.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.747+0000 7f5b6b400640 1 -- 172.21.15.86:0/43024419 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b6c004bf0 con 0x7f5b7409f140 2024-04-20T21:20:52.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.747+0000 7f5b6b400640 1 -- 172.21.15.86:0/43024419 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c001110 con 0x7f5b7409f140 2024-04-20T21:20:52.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.748+0000 7f5b7a000640 1 -- 172.21.15.86:0/43024419 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b7409f5c0 con 0x7f5b7409f140 2024-04-20T21:20:52.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.748+0000 7f5b7a000640 1 -- 172.21.15.86:0/43024419 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b740c78e0 con 0x7f5b7409f140 2024-04-20T21:20:52.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.749+0000 7f5b7a000640 1 -- 172.21.15.86:0/43024419 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b740d0c60 con 0x7f5b7409f140 2024-04-20T21:20:52.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.750+0000 7f5b6b400640 1 -- 172.21.15.86:0/43024419 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5b6c007070 con 0x7f5b7409f140 2024-04-20T21:20:52.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.751+0000 7f5b6b400640 1 --2- 172.21.15.86:0/43024419 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5b5006c8f0 0x7f5b5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:52.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.751+0000 7f5b6b400640 1 -- 172.21.15.86:0/43024419 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f5b6c0acde0 con 0x7f5b7409f140 2024-04-20T21:20:52.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.751+0000 7f5b73400640 1 --2- 172.21.15.86:0/43024419 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5b5006c8f0 0x7f5b5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:52.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.752+0000 7f5b73400640 1 --2- 172.21.15.86:0/43024419 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5b5006c8f0 0x7f5b5006edb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5b64001c90 tx=0x7f5b64009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:52.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.755+0000 7f5b6b400640 1 -- 172.21.15.86:0/43024419 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5b6c047020 con 0x7f5b7409f140 2024-04-20T21:20:52.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.995+0000 7f5b7a000640 1 -- 172.21.15.86:0/43024419 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b740689e0 con 0x7f5b7409f140 2024-04-20T21:20:52.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.996+0000 7f5b6b400640 1 -- 172.21.15.86:0/43024419 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5199 (secure 0 0 0) 0x7f5b7409f5c0 con 0x7f5b7409f140 2024-04-20T21:20:52.995 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:20:52.995 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5044,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/1013106052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":1013106052},{"type":"v1","addr":"172.21.15.144:6833","nonce":1013106052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5047,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/1073576945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1073576945},{"type":"v1","addr":"172.21.15.144:6835","nonce":1073576945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/821548012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":821548012},{"type":"v1","addr":"172.21.15.86:6835","nonce":821548012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:20:03.369281+0000","modified":"2024-04-20T21:20:06.438592+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5691},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5691":{"gid":5691,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/712628984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":712628984},{"type":"v1","addr":"172.21.15.86:6837","nonce":712628984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5691,"qdb_cluster":[5691]},"id":3}]} 2024-04-20T21:20:52.995 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 24 2024-04-20T21:20:52.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.998+0000 7f5b7a000640 1 -- 172.21.15.86:0/43024419 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5b5006c8f0 msgr2=0x7f5b5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:52.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.998+0000 7f5b7a000640 1 --2- 172.21.15.86:0/43024419 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5b5006c8f0 0x7f5b5006edb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5b64001c90 tx=0x7f5b64009040 comp rx=0 tx=0).stop 2024-04-20T21:20:52.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.998+0000 7f5b7a000640 1 -- 172.21.15.86:0/43024419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b7409f140 msgr2=0x7f5b74151130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:52.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.998+0000 7f5b7a000640 1 --2- 172.21.15.86:0/43024419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b7409f140 0x7f5b74151130 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c00c5e0 tx=0x7f5b6c001c10 comp rx=0 tx=0).stop 2024-04-20T21:20:52.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.999+0000 7f5b7a000640 1 -- 172.21.15.86:0/43024419 shutdown_connections 2024-04-20T21:20:52.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.999+0000 7f5b7a000640 1 --2- 172.21.15.86:0/43024419 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5b5006c8f0 0x7f5b5006edb0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:52.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.999+0000 7f5b7a000640 1 --2- 172.21.15.86:0/43024419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b7409f140 0x7f5b74151130 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:52.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.999+0000 7f5b7a000640 1 --2- 172.21.15.86:0/43024419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b740c71e0 0x7f5b7409ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:52.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.999+0000 7f5b7a000640 1 --2- 172.21.15.86:0/43024419 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b580051c0 0x7f5b74154c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:52.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.999+0000 7f5b7a000640 1 -- 172.21.15.86:0/43024419 >> 172.21.15.86:0/43024419 conn(0x7f5b740b4080 msgr2=0x7f5b740b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:52.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.999+0000 7f5b7a000640 1 -- 172.21.15.86:0/43024419 shutdown_connections 2024-04-20T21:20:52.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:52.999+0000 7f5b7a000640 1 -- 172.21.15.86:0/43024419 wait complete. 2024-04-20T21:20:53.016 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:20:53.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.217+0000 7f7714200640 1 Processor -- start 2024-04-20T21:20:53.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.217+0000 7f7714200640 1 -- start start 2024-04-20T21:20:53.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.218+0000 7f7714200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f770c0cd170 0x7f770c06ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:53.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.218+0000 7f7714200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f770c0cd5d0 0x7f770c06f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:53.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.219+0000 7f7714200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f770c0cdb50 con 0x7f770c144e90 2024-04-20T21:20:53.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.219+0000 7f7714200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f770c145270 con 0x7f770c0cd5d0 2024-04-20T21:20:53.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.219+0000 7f7714200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f770c0b3e50 con 0x7f770c0cd170 2024-04-20T21:20:53.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.219+0000 7f7711800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f770c144e90 0x7f770c0b4930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40466/0 (socket says 172.21.15.86:40466) 2024-04-20T21:20:53.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.219+0000 7f7711800640 1 -- 172.21.15.86:0/1414374444 learned_addr learned my addr 172.21.15.86:0/1414374444 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:20:53.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.219+0000 7f770be00640 1 -- 172.21.15.86:0/1414374444 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1866516845 0 0) 0x7f770c0cdb50 con 0x7f770c144e90 2024-04-20T21:20:53.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.219+0000 7f7712200640 1 --2- 172.21.15.86:0/1414374444 >> v2:172.21.15.144:3300/0 conn(0x7f770c0cd5d0 0x7f770c06f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:53.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.219+0000 7f770be00640 1 -- 172.21.15.86:0/1414374444 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76f4003520 con 0x7f770c144e90 2024-04-20T21:20:53.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.219+0000 7f7710e00640 1 --2- 172.21.15.86:0/1414374444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f770c0cd170 0x7f770c06ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:53.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.220+0000 7f770be00640 1 -- 172.21.15.86:0/1414374444 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1659839666 0 0) 0x7f76f4003520 con 0x7f770c144e90 2024-04-20T21:20:53.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.220+0000 7f770be00640 1 -- 172.21.15.86:0/1414374444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f770c0cd170 msgr2=0x7f770c06ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:53.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.220+0000 7f770be00640 1 --2- 172.21.15.86:0/1414374444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f770c0cd170 0x7f770c06ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:53.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.220+0000 7f770be00640 1 -- 172.21.15.86:0/1414374444 >> v2:172.21.15.144:3300/0 conn(0x7f770c0cd5d0 msgr2=0x7f770c06f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:53.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.220+0000 7f770be00640 1 --2- 172.21.15.86:0/1414374444 >> v2:172.21.15.144:3300/0 conn(0x7f770c0cd5d0 0x7f770c06f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:53.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.220+0000 7f770be00640 1 -- 172.21.15.86:0/1414374444 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f770c0b4130 con 0x7f770c144e90 2024-04-20T21:20:53.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.220+0000 7f770be00640 1 -- 172.21.15.86:0/1414374444 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f7700001670 con 0x7f770c144e90 2024-04-20T21:20:53.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.220+0000 7f7710e00640 1 --2- 172.21.15.86:0/1414374444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f770c0cd170 0x7f770c06ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:20:53.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.220+0000 7f770be00640 1 -- 172.21.15.86:0/1414374444 >> v1:172.21.15.86:6789/0 conn(0x7f770c144e90 legacy=0x7f770c0b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:53.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.220+0000 7f770be00640 1 --2- 172.21.15.86:0/1414374444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f770c0cd170 0x7f76f40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:53.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.220+0000 7f770be00640 1 --2- 172.21.15.86:0/1414374444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76f4003f40 0x7f76f4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:53.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.220+0000 7f770be00640 1 --2- 172.21.15.86:0/1414374444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76f4004d40 0x7f76f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:53.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.220+0000 7f770be00640 1 -- 172.21.15.86:0/1414374444 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f770c145270 con 0x7f76f4004d40 2024-04-20T21:20:53.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.220+0000 7f770be00640 1 -- 172.21.15.86:0/1414374444 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f770c0b3e50 con 0x7f76f4003f40 2024-04-20T21:20:53.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.220+0000 7f770be00640 1 -- 172.21.15.86:0/1414374444 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f76f400bb80 con 0x7f770c0cd170 2024-04-20T21:20:53.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.220+0000 7f7712200640 1 --2- 172.21.15.86:0/1414374444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76f4004d40 0x7f76f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:53.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.220+0000 7f7712200640 1 -- 172.21.15.86:0/1414374444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f770c0cd170 msgr2=0x7f76f40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:53.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.220+0000 7f7712200640 1 --2- 172.21.15.86:0/1414374444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f770c0cd170 0x7f76f40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:53.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.220+0000 7f7710e00640 1 --2- 172.21.15.86:0/1414374444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76f4003f40 0x7f76f4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:53.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.220+0000 7f7712200640 1 -- 172.21.15.86:0/1414374444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76f4003f40 msgr2=0x7f76f4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:53.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.221+0000 7f7712200640 1 --2- 172.21.15.86:0/1414374444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76f4003f40 0x7f76f4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:53.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.221+0000 7f7712200640 1 -- 172.21.15.86:0/1414374444 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76f4003520 con 0x7f76f4004d40 2024-04-20T21:20:53.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.221+0000 7f7710e00640 1 --2- 172.21.15.86:0/1414374444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76f4003f40 0x7f76f4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:20:53.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.222+0000 7f7712200640 1 --2- 172.21.15.86:0/1414374444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76f4004d40 0x7f76f40051c0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f7704010350 tx=0x7f76f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9fb55fdbfab55eb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:53.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.222+0000 7f770be00640 1 -- 172.21.15.86:0/1414374444 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770400be20 con 0x7f76f4004d40 2024-04-20T21:20:53.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.222+0000 7f770be00640 1 -- 172.21.15.86:0/1414374444 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f770402ee50 con 0x7f76f4004d40 2024-04-20T21:20:53.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.222+0000 7f770be00640 1 -- 172.21.15.86:0/1414374444 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770400b310 con 0x7f76f4004d40 2024-04-20T21:20:53.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.223+0000 7f7714200640 1 -- 172.21.15.86:0/1414374444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76f4004d40 msgr2=0x7f76f40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:53.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.223+0000 7f7714200640 1 --2- 172.21.15.86:0/1414374444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76f4004d40 0x7f76f40051c0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f7704010350 tx=0x7f76f4005830 comp rx=0 tx=0).stop 2024-04-20T21:20:53.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.223+0000 7f7712200640 1 -- 172.21.15.86:0/1414374444 reap_dead start 2024-04-20T21:20:53.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.223+0000 7f7714200640 1 -- 172.21.15.86:0/1414374444 shutdown_connections 2024-04-20T21:20:53.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.223+0000 7f7714200640 1 -- 172.21.15.86:0/1414374444 >> 172.21.15.86:0/1414374444 conn(0x7f770c0b2360 msgr2=0x7f770c0b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:53.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.223+0000 7f7714200640 1 -- 172.21.15.86:0/1414374444 shutdown_connections 2024-04-20T21:20:53.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.223+0000 7f7714200640 1 -- 172.21.15.86:0/1414374444 wait complete. 2024-04-20T21:20:53.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.225+0000 7f7714200640 1 Processor -- start 2024-04-20T21:20:53.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.225+0000 7f7714200640 1 -- start start 2024-04-20T21:20:53.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.226+0000 7f7714200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76f4004d40 0x7f770c15adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:53.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.226+0000 7f7714200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f770c1e1940 0x7f770c1e9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:53.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.226+0000 7f7714200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f770c1e0be0 0x7f770c1e1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:53.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.226+0000 7f7714200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f770c0d0190 con 0x7f76f4004d40 2024-04-20T21:20:53.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.226+0000 7f7714200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f770c0d0010 con 0x7f770c1e1940 2024-04-20T21:20:53.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.226+0000 7f7714200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f770c15bb50 con 0x7f770c1e0be0 2024-04-20T21:20:53.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.226+0000 7f7711800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76f4004d40 0x7f770c15adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:53.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.226+0000 7f7711800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76f4004d40 0x7f770c15adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39780/0 (socket says 172.21.15.86:39780) 2024-04-20T21:20:53.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.226+0000 7f7711800640 1 -- 172.21.15.86:0/2520724319 learned_addr learned my addr 172.21.15.86:0/2520724319 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:53.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.226+0000 7f7710e00640 1 --2- 172.21.15.86:0/2520724319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f770c1e1940 0x7f770c1e9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:53.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.226+0000 7f7711800640 1 -- 172.21.15.86:0/2520724319 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f770c1e0be0 msgr2=0x7f770c1e1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:53.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.226+0000 7f7711800640 1 --2- 172.21.15.86:0/2520724319 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f770c1e0be0 0x7f770c1e1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:53.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.226+0000 7f7711800640 1 -- 172.21.15.86:0/2520724319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f770c1e1940 msgr2=0x7f770c1e9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:53.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.226+0000 7f7711800640 1 --2- 172.21.15.86:0/2520724319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f770c1e1940 0x7f770c1e9ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:53.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.226+0000 7f7711800640 1 -- 172.21.15.86:0/2520724319 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f770c1e1e70 con 0x7f76f4004d40 2024-04-20T21:20:53.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.227+0000 7f7711800640 1 --2- 172.21.15.86:0/2520724319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76f4004d40 0x7f770c15adc0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f770000c640 tx=0x7f770000cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:53.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.227+0000 7f7709600640 1 -- 172.21.15.86:0/2520724319 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7700018070 con 0x7f76f4004d40 2024-04-20T21:20:53.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.227+0000 7f7709600640 1 -- 172.21.15.86:0/2520724319 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7700007500 con 0x7f76f4004d40 2024-04-20T21:20:53.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.227+0000 7f7709600640 1 -- 172.21.15.86:0/2520724319 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7700004a20 con 0x7f76f4004d40 2024-04-20T21:20:53.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.227+0000 7f7714200640 1 -- 172.21.15.86:0/2520724319 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f770c1e2150 con 0x7f76f4004d40 2024-04-20T21:20:53.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.227+0000 7f7714200640 1 -- 172.21.15.86:0/2520724319 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f770c1e6510 con 0x7f76f4004d40 2024-04-20T21:20:53.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.228+0000 7f7714200640 1 -- 172.21.15.86:0/2520724319 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f770c1e6710 con 0x7f76f4004d40 2024-04-20T21:20:53.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.229+0000 7f7709600640 1 -- 172.21.15.86:0/2520724319 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7700004bc0 con 0x7f76f4004d40 2024-04-20T21:20:53.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.231+0000 7f7709600640 1 --2- 172.21.15.86:0/2520724319 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f76f006c920 0x7f76f006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:53.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.231+0000 7f7709600640 1 -- 172.21.15.86:0/2520724319 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f770008b0d0 con 0x7f76f4004d40 2024-04-20T21:20:53.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.232+0000 7f7710e00640 1 --2- 172.21.15.86:0/2520724319 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f76f006c920 0x7f76f006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:53.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.232+0000 7f7710e00640 1 --2- 172.21.15.86:0/2520724319 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f76f006c920 0x7f76f006ede0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f76fc006fd0 tx=0x7f76fc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:53.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.235+0000 7f7709600640 1 -- 172.21.15.86:0/2520724319 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f770c1e6710 con 0x7f76f4004d40 2024-04-20T21:20:53.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.434+0000 7f7714200640 1 -- 172.21.15.86:0/2520724319 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f770c0689e0 con 0x7f76f4004d40 2024-04-20T21:20:53.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.437+0000 7f7709600640 1 -- 172.21.15.86:0/2520724319 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16704 (secure 0 0 0) 0x7f77000594b0 con 0x7f76f4004d40 2024-04-20T21:20:53.436 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:20:53.437 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":47,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:20:34.772220+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:20:01.307757+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-20T21:20:02.359347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.6","mappings":[{"from":0,"to":4}]},{"pgid":"7.36","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:20:53.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.440+0000 7f7714200640 1 -- 172.21.15.86:0/2520724319 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f76f006c920 msgr2=0x7f76f006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:53.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.440+0000 7f7714200640 1 --2- 172.21.15.86:0/2520724319 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f76f006c920 0x7f76f006ede0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f76fc006fd0 tx=0x7f76fc008040 comp rx=0 tx=0).stop 2024-04-20T21:20:53.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.440+0000 7f7714200640 1 -- 172.21.15.86:0/2520724319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76f4004d40 msgr2=0x7f770c15adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:53.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.440+0000 7f7714200640 1 --2- 172.21.15.86:0/2520724319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76f4004d40 0x7f770c15adc0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f770000c640 tx=0x7f770000cb00 comp rx=0 tx=0).stop 2024-04-20T21:20:53.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.440+0000 7f7714200640 1 -- 172.21.15.86:0/2520724319 shutdown_connections 2024-04-20T21:20:53.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.440+0000 7f7714200640 1 --2- 172.21.15.86:0/2520724319 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f76f006c920 0x7f76f006ede0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:53.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.440+0000 7f7714200640 1 --2- 172.21.15.86:0/2520724319 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f770c1e0be0 0x7f770c1e1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:53.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.440+0000 7f7714200640 1 --2- 172.21.15.86:0/2520724319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f770c1e1940 0x7f770c1e9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:53.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.440+0000 7f7714200640 1 --2- 172.21.15.86:0/2520724319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76f4004d40 0x7f770c15adc0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:53.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.440+0000 7f7714200640 1 -- 172.21.15.86:0/2520724319 >> 172.21.15.86:0/2520724319 conn(0x7f770c0b2360 msgr2=0x7f770c144010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:53.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.440+0000 7f7714200640 1 -- 172.21.15.86:0/2520724319 shutdown_connections 2024-04-20T21:20:53.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.440+0000 7f7714200640 1 -- 172.21.15.86:0/2520724319 wait complete. 2024-04-20T21:20:53.458 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-20T21:20:53.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.654+0000 7f3066a00640 1 Processor -- start 2024-04-20T21:20:53.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.654+0000 7f3066a00640 1 -- start start 2024-04-20T21:20:53.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.656+0000 7f3066a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f306009b3f0 0x7f30600d1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:53.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.656+0000 7f3066a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f306006d800 0x7f30600b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:53.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.656+0000 7f3066a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f306009ba10 con 0x7f306006d420 2024-04-20T21:20:53.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.656+0000 7f3066a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f30600d0810 con 0x7f306009b3f0 2024-04-20T21:20:53.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.656+0000 7f3066a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f30600b3c70 con 0x7f306006d800 2024-04-20T21:20:53.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.656+0000 7f305fe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f306006d420 0x7f306006ca90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40480/0 (socket says 172.21.15.86:40480) 2024-04-20T21:20:53.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.656+0000 7f305fe00640 1 -- 172.21.15.86:0/73147137 learned_addr learned my addr 172.21.15.86:0/73147137 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:20:53.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.657+0000 7f3064a00640 1 --2- 172.21.15.86:0/73147137 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f306006d800 0x7f30600b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:53.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.657+0000 7f305f400640 1 --2- 172.21.15.86:0/73147137 >> v2:172.21.15.144:3300/0 conn(0x7f306009b3f0 0x7f30600d1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:53.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.657+0000 7f305ea00640 1 -- 172.21.15.86:0/73147137 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3408006117 0 0) 0x7f306009ba10 con 0x7f306006d420 2024-04-20T21:20:53.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.657+0000 7f305ea00640 1 -- 172.21.15.86:0/73147137 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3044003520 con 0x7f306006d420 2024-04-20T21:20:53.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.657+0000 7f305ea00640 1 -- 172.21.15.86:0/73147137 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2333923675 0 0) 0x7f3044003520 con 0x7f306006d420 2024-04-20T21:20:53.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.657+0000 7f305ea00640 1 -- 172.21.15.86:0/73147137 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f306006d800 msgr2=0x7f30600b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:53.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.657+0000 7f305ea00640 1 --2- 172.21.15.86:0/73147137 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f306006d800 0x7f30600b1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:53.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.657+0000 7f305ea00640 1 -- 172.21.15.86:0/73147137 >> v2:172.21.15.144:3300/0 conn(0x7f306009b3f0 msgr2=0x7f30600d1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:53.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.657+0000 7f305ea00640 1 --2- 172.21.15.86:0/73147137 >> v2:172.21.15.144:3300/0 conn(0x7f306009b3f0 0x7f30600d1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:53.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.657+0000 7f305ea00640 1 -- 172.21.15.86:0/73147137 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30600d0a40 con 0x7f306006d420 2024-04-20T21:20:53.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.657+0000 7f305ea00640 1 -- 172.21.15.86:0/73147137 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f3050001670 con 0x7f306006d420 2024-04-20T21:20:53.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.657+0000 7f305ea00640 1 -- 172.21.15.86:0/73147137 >> v1:172.21.15.86:6789/0 conn(0x7f306006d420 legacy=0x7f306006ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:53.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.657+0000 7f305ea00640 1 --2- 172.21.15.86:0/73147137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30440039b0 0x7f3044003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:53.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.657+0000 7f305ea00640 1 --2- 172.21.15.86:0/73147137 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f306006d800 0x7f3044004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:53.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.657+0000 7f305ea00640 1 --2- 172.21.15.86:0/73147137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3044004d40 0x7f30440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:53.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.657+0000 7f305ea00640 1 -- 172.21.15.86:0/73147137 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f30600b3c70 con 0x7f3044004d40 2024-04-20T21:20:53.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.657+0000 7f305ea00640 1 -- 172.21.15.86:0/73147137 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f304400b9c0 con 0x7f30440039b0 2024-04-20T21:20:53.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.657+0000 7f305ea00640 1 -- 172.21.15.86:0/73147137 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f304400bb40 con 0x7f306006d800 2024-04-20T21:20:53.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.658+0000 7f3064a00640 1 --2- 172.21.15.86:0/73147137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3044004d40 0x7f30440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:53.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.658+0000 7f3064a00640 1 -- 172.21.15.86:0/73147137 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f306006d800 msgr2=0x7f3044004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:53.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.658+0000 7f305f400640 1 --2- 172.21.15.86:0/73147137 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f306006d800 0x7f3044004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:53.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.658+0000 7f305fe00640 1 --2- 172.21.15.86:0/73147137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30440039b0 0x7f3044003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:53.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.658+0000 7f3064a00640 1 --2- 172.21.15.86:0/73147137 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f306006d800 0x7f3044004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:53.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.658+0000 7f3064a00640 1 -- 172.21.15.86:0/73147137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30440039b0 msgr2=0x7f3044003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:53.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.658+0000 7f3064a00640 1 --2- 172.21.15.86:0/73147137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30440039b0 0x7f3044003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:53.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.658+0000 7f3064a00640 1 -- 172.21.15.86:0/73147137 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3044003520 con 0x7f3044004d40 2024-04-20T21:20:53.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.658+0000 7f305f400640 1 --2- 172.21.15.86:0/73147137 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f306006d800 0x7f3044004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:20:53.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.658+0000 7f305fe00640 1 --2- 172.21.15.86:0/73147137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30440039b0 0x7f3044003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:20:53.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.659+0000 7f3064a00640 1 --2- 172.21.15.86:0/73147137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3044004d40 0x7f30440051c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f305800e170 tx=0x7f3044005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f35ca5104142f409 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:53.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.659+0000 7f305ea00640 1 -- 172.21.15.86:0/73147137 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3058039070 con 0x7f3044004d40 2024-04-20T21:20:53.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.659+0000 7f305ea00640 1 -- 172.21.15.86:0/73147137 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f305802cbf0 con 0x7f3044004d40 2024-04-20T21:20:53.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.659+0000 7f305ea00640 1 -- 172.21.15.86:0/73147137 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3058009c10 con 0x7f3044004d40 2024-04-20T21:20:53.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.660+0000 7f3066a00640 1 -- 172.21.15.86:0/73147137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3044004d40 msgr2=0x7f30440051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:53.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.660+0000 7f3066a00640 1 --2- 172.21.15.86:0/73147137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3044004d40 0x7f30440051c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f305800e170 tx=0x7f3044005830 comp rx=0 tx=0).stop 2024-04-20T21:20:53.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.660+0000 7f3064a00640 1 -- 172.21.15.86:0/73147137 reap_dead start 2024-04-20T21:20:53.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.660+0000 7f3066a00640 1 -- 172.21.15.86:0/73147137 shutdown_connections 2024-04-20T21:20:53.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.660+0000 7f3066a00640 1 -- 172.21.15.86:0/73147137 >> 172.21.15.86:0/73147137 conn(0x7f30600b4030 msgr2=0x7f30600b4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:53.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.660+0000 7f3066a00640 1 -- 172.21.15.86:0/73147137 shutdown_connections 2024-04-20T21:20:53.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.660+0000 7f3066a00640 1 -- 172.21.15.86:0/73147137 wait complete. 2024-04-20T21:20:53.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.662+0000 7f3066a00640 1 Processor -- start 2024-04-20T21:20:53.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.662+0000 7f3066a00640 1 -- start start 2024-04-20T21:20:53.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.663+0000 7f3066a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30601f4bc0 0x7f30601f4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:53.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.663+0000 7f3066a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f30601f54e0 0x7f30600b5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:53.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.663+0000 7f3066a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30600b60d0 0x7f30600af2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:53.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.663+0000 7f3066a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f30600d1000 con 0x7f30601f4bc0 2024-04-20T21:20:53.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.663+0000 7f3066a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f306006d270 con 0x7f30600b60d0 2024-04-20T21:20:53.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.663+0000 7f3066a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f30600d12e0 con 0x7f30601f54e0 2024-04-20T21:20:53.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.664+0000 7f305fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30601f4bc0 0x7f30601f4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:53.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.664+0000 7f305fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30601f4bc0 0x7f30601f4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39804/0 (socket says 172.21.15.86:39804) 2024-04-20T21:20:53.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.664+0000 7f305fe00640 1 -- 172.21.15.86:0/949305297 learned_addr learned my addr 172.21.15.86:0/949305297 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:53.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.664+0000 7f3064a00640 1 --2- 172.21.15.86:0/949305297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30600b60d0 0x7f30600af2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:53.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.664+0000 7f305f400640 1 --2- 172.21.15.86:0/949305297 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f30601f54e0 0x7f30600b5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:53.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.664+0000 7f305fe00640 1 -- 172.21.15.86:0/949305297 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f30601f54e0 msgr2=0x7f30600b5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:53.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.664+0000 7f305fe00640 1 --2- 172.21.15.86:0/949305297 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f30601f54e0 0x7f30600b5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:53.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.664+0000 7f305fe00640 1 -- 172.21.15.86:0/949305297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30600b60d0 msgr2=0x7f30600af2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:53.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.664+0000 7f305fe00640 1 --2- 172.21.15.86:0/949305297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30600b60d0 0x7f30600af2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:53.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.664+0000 7f305fe00640 1 -- 172.21.15.86:0/949305297 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30601e4230 con 0x7f30601f4bc0 2024-04-20T21:20:53.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.664+0000 7f305f400640 1 --2- 172.21.15.86:0/949305297 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f30601f54e0 0x7f30600b5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:20:53.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.664+0000 7f305fe00640 1 --2- 172.21.15.86:0/949305297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30601f4bc0 0x7f30601f4fa0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f305000daf0 tx=0x7f3050007e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:53.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.665+0000 7f3057e00640 1 -- 172.21.15.86:0/949305297 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3050019070 con 0x7f30601f4bc0 2024-04-20T21:20:53.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.665+0000 7f3057e00640 1 -- 172.21.15.86:0/949305297 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3050004cb0 con 0x7f30601f4bc0 2024-04-20T21:20:53.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.665+0000 7f3057e00640 1 -- 172.21.15.86:0/949305297 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3050004e60 con 0x7f30601f4bc0 2024-04-20T21:20:53.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.665+0000 7f3066a00640 1 -- 172.21.15.86:0/949305297 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30601e4510 con 0x7f30601f4bc0 2024-04-20T21:20:53.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.665+0000 7f3066a00640 1 -- 172.21.15.86:0/949305297 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30601e4ad0 con 0x7f30601f4bc0 2024-04-20T21:20:53.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.666+0000 7f3066a00640 1 -- 172.21.15.86:0/949305297 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30600689e0 con 0x7f30601f4bc0 2024-04-20T21:20:53.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.667+0000 7f3057e00640 1 -- 172.21.15.86:0/949305297 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f30500072d0 con 0x7f30601f4bc0 2024-04-20T21:20:53.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.668+0000 7f3057e00640 1 --2- 172.21.15.86:0/949305297 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f304006c8f0 0x7f304006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:53.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.668+0000 7f3057e00640 1 -- 172.21.15.86:0/949305297 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f305008b7a0 con 0x7f30601f4bc0 2024-04-20T21:20:53.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.668+0000 7f305f400640 1 --2- 172.21.15.86:0/949305297 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f304006c8f0 0x7f304006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:53.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.669+0000 7f305f400640 1 --2- 172.21.15.86:0/949305297 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f304006c8f0 0x7f304006edb0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f30601e98a0 tx=0x7f304c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:53.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.673+0000 7f3057e00640 1 -- 172.21.15.86:0/949305297 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3050059bb0 con 0x7f30601f4bc0 2024-04-20T21:20:53.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.909+0000 7f3066a00640 1 -- 172.21.15.86:0/949305297 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f30600d1ee0 con 0x7f30601f4bc0 2024-04-20T21:20:53.909 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-04-20T21:20:53.910+0000 7f5bb9c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T21:20:53.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.913+0000 7f3057e00640 1 -- 172.21.15.86:0/949305297 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v25) ==== 161+0+0 (secure 0 0 0) 0x7f30500594b0 con 0x7f30601f4bc0 2024-04-20T21:20:53.913 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-20T21:20:53.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.917+0000 7f3066a00640 1 -- 172.21.15.86:0/949305297 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f304006c8f0 msgr2=0x7f304006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:53.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.917+0000 7f3066a00640 1 --2- 172.21.15.86:0/949305297 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f304006c8f0 0x7f304006edb0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f30601e98a0 tx=0x7f304c008040 comp rx=0 tx=0).stop 2024-04-20T21:20:53.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.917+0000 7f3066a00640 1 -- 172.21.15.86:0/949305297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30601f4bc0 msgr2=0x7f30601f4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:53.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.917+0000 7f3066a00640 1 --2- 172.21.15.86:0/949305297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30601f4bc0 0x7f30601f4fa0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f305000daf0 tx=0x7f3050007e30 comp rx=0 tx=0).stop 2024-04-20T21:20:53.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.917+0000 7f3066a00640 1 -- 172.21.15.86:0/949305297 shutdown_connections 2024-04-20T21:20:53.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.917+0000 7f3066a00640 1 --2- 172.21.15.86:0/949305297 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f304006c8f0 0x7f304006edb0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:53.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.917+0000 7f3066a00640 1 --2- 172.21.15.86:0/949305297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30600b60d0 0x7f30600af2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:53.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.917+0000 7f3066a00640 1 --2- 172.21.15.86:0/949305297 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f30601f54e0 0x7f30600b5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:53.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.917+0000 7f3066a00640 1 --2- 172.21.15.86:0/949305297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30601f4bc0 0x7f30601f4fa0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:53.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.917+0000 7f3066a00640 1 -- 172.21.15.86:0/949305297 >> 172.21.15.86:0/949305297 conn(0x7f30600b4030 msgr2=0x7f30601e5f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:53.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.918+0000 7f3066a00640 1 -- 172.21.15.86:0/949305297 shutdown_connections 2024-04-20T21:20:53.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:53.918+0000 7f3066a00640 1 -- 172.21.15.86:0/949305297 wait complete. 2024-04-20T21:20:53.932 INFO:tasks.ceph.mds.a.smithi086.stderr: -5010> 2024-04-20T21:20:08.282+0000 7f1e0fc00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T21:20:53.936 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-20T21:20:53.940 INFO:tasks.ceph.mds.a.smithi086.stderr: -3219> 2024-04-20T21:20:14.258+0000 7f1e0ac00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={5829=0-4194304@1} caps={5829=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5829 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x55b78368cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-20T21:20:53.997 INFO:tasks.ceph.mds.a.smithi086.stdout:starting mds.a at 2024-04-20T21:20:54.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.113+0000 7fed00200640 1 Processor -- start 2024-04-20T21:20:54.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.113+0000 7fed00200640 1 -- start start 2024-04-20T21:20:54.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.114+0000 7fed00200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fecf806bbf0 0x7fecf806e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:54.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.114+0000 7fed00200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fecf8146210 0x7fecf80cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:54.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.114+0000 7fed00200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecf815cd40 con 0x7fecf806bfd0 2024-04-20T21:20:54.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.114+0000 7fed00200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fecf815d000 con 0x7fecf8146210 2024-04-20T21:20:54.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.114+0000 7fed00200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fecf815d2e0 con 0x7fecf806bbf0 2024-04-20T21:20:54.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.115+0000 7fecfe200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fecf8146210 0x7fecf80cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:54.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.115+0000 7fecfe200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fecf8146210 0x7fecf80cd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39838/0 (socket says 172.21.15.86:39838) 2024-04-20T21:20:54.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.115+0000 7fecfe200640 1 -- 172.21.15.86:0/171661573 learned_addr learned my addr 172.21.15.86:0/171661573 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:54.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.115+0000 7fecfd800640 1 --2- 172.21.15.86:0/171661573 >> v2:172.21.15.144:3300/0 conn(0x7fecf806bbf0 0x7fecf806e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:54.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.115+0000 7fecfe200640 1 -- 172.21.15.86:0/171661573 >> v2:172.21.15.144:3300/0 conn(0x7fecf806bbf0 msgr2=0x7fecf806e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:54.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.115+0000 7fecfe200640 1 --2- 172.21.15.86:0/171661573 >> v2:172.21.15.144:3300/0 conn(0x7fecf806bbf0 0x7fecf806e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:54.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.115+0000 7fecfe200640 1 -- 172.21.15.86:0/171661573 >> v1:172.21.15.144:6789/0 conn(0x7fecf806bfd0 legacy=0x7fecf8145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:54.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.115+0000 7fecfe200640 1 -- 172.21.15.86:0/171661573 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecf815d5c0 con 0x7fecf8146210 2024-04-20T21:20:54.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.117+0000 7fecfe200640 1 --2- 172.21.15.86:0/171661573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecf8146210 0x7fecf80cd540 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fecf8073150 tx=0x7fecf002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2ecb9545b4694a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:54.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.117+0000 7fecf7e00640 1 -- 172.21.15.86:0/171661573 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf0035020 con 0x7fecf8146210 2024-04-20T21:20:54.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.117+0000 7fecf7e00640 1 -- 172.21.15.86:0/171661573 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecf0031870 con 0x7fecf8146210 2024-04-20T21:20:54.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.117+0000 7fecf7e00640 1 -- 172.21.15.86:0/171661573 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf0031c40 con 0x7fecf8146210 2024-04-20T21:20:54.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.117+0000 7fed00200640 1 -- 172.21.15.86:0/171661573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecf8146210 msgr2=0x7fecf80cd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:54.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.117+0000 7fed00200640 1 --2- 172.21.15.86:0/171661573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecf8146210 0x7fecf80cd540 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fecf8073150 tx=0x7fecf002b910 comp rx=0 tx=0).stop 2024-04-20T21:20:54.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.118+0000 7fed00200640 1 -- 172.21.15.86:0/171661573 shutdown_connections 2024-04-20T21:20:54.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.118+0000 7fed00200640 1 --2- 172.21.15.86:0/171661573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecf8146210 0x7fecf80cd540 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:54.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.118+0000 7fed00200640 1 --2- 172.21.15.86:0/171661573 >> v2:172.21.15.144:3300/0 conn(0x7fecf806bbf0 0x7fecf806e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:54.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.118+0000 7fed00200640 1 -- 172.21.15.86:0/171661573 >> 172.21.15.86:0/171661573 conn(0x7fecf80b2110 msgr2=0x7fecf80b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:54.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.118+0000 7fed00200640 1 -- 172.21.15.86:0/171661573 shutdown_connections 2024-04-20T21:20:54.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.118+0000 7fed00200640 1 -- 172.21.15.86:0/171661573 wait complete. 2024-04-20T21:20:54.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.119+0000 7fed00200640 1 Processor -- start 2024-04-20T21:20:54.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.119+0000 7fed00200640 1 -- start start 2024-04-20T21:20:54.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.120+0000 7fed00200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecf806bbf0 0x7fecf8144890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:54.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.120+0000 7fed00200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fecf806bfd0 0x7fecf81ea2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:54.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.120+0000 7fed00200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecf8146210 0x7fecf81ea810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:54.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.120+0000 7fed00200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fecf80cdc00 con 0x7fecf8146210 2024-04-20T21:20:54.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.120+0000 7fed00200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fecf80cda80 con 0x7fecf806bbf0 2024-04-20T21:20:54.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.120+0000 7fed00200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fecf80cdd80 con 0x7fecf806bfd0 2024-04-20T21:20:54.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.121+0000 7fecfe200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecf8146210 0x7fecf81ea810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:54.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.121+0000 7fecfe200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecf8146210 0x7fecf81ea810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39844/0 (socket says 172.21.15.86:39844) 2024-04-20T21:20:54.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.121+0000 7fecfe200640 1 -- 172.21.15.86:0/2899347390 learned_addr learned my addr 172.21.15.86:0/2899347390 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:54.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.121+0000 7fecfd800640 1 --2- 172.21.15.86:0/2899347390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecf806bbf0 0x7fecf8144890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:54.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.121+0000 7fecfce00640 1 --2- 172.21.15.86:0/2899347390 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fecf806bfd0 0x7fecf81ea2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:54.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.121+0000 7fecfe200640 1 -- 172.21.15.86:0/2899347390 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fecf806bfd0 msgr2=0x7fecf81ea2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:54.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.121+0000 7fecfe200640 1 --2- 172.21.15.86:0/2899347390 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fecf806bfd0 0x7fecf81ea2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:54.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.121+0000 7fecfe200640 1 -- 172.21.15.86:0/2899347390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecf806bbf0 msgr2=0x7fecf8144890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:54.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.121+0000 7fecfe200640 1 --2- 172.21.15.86:0/2899347390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecf806bbf0 0x7fecf8144890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:54.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.121+0000 7fecfe200640 1 -- 172.21.15.86:0/2899347390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecf81e64d0 con 0x7fecf8146210 2024-04-20T21:20:54.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.121+0000 7fecfe200640 1 --2- 172.21.15.86:0/2899347390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecf8146210 0x7fecf81ea810 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fecf0009960 tx=0x7fecf00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:54.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.122+0000 7fecf5600640 1 -- 172.21.15.86:0/2899347390 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf0040070 con 0x7fecf8146210 2024-04-20T21:20:54.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.122+0000 7fecf5600640 1 -- 172.21.15.86:0/2899347390 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecf0030d00 con 0x7fecf8146210 2024-04-20T21:20:54.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.122+0000 7fecf5600640 1 -- 172.21.15.86:0/2899347390 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf0035040 con 0x7fecf8146210 2024-04-20T21:20:54.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.122+0000 7fed00200640 1 -- 172.21.15.86:0/2899347390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecf81e67b0 con 0x7fecf8146210 2024-04-20T21:20:54.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.122+0000 7fed00200640 1 -- 172.21.15.86:0/2899347390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecf81e7f30 con 0x7fecf8146210 2024-04-20T21:20:54.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.123+0000 7fed00200640 1 -- 172.21.15.86:0/2899347390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecf80689e0 con 0x7fecf8146210 2024-04-20T21:20:54.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.124+0000 7fecf5600640 1 -- 172.21.15.86:0/2899347390 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fecf0038020 con 0x7fecf8146210 2024-04-20T21:20:54.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.125+0000 7fecf5600640 1 --2- 172.21.15.86:0/2899347390 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fecd806c8f0 0x7fecd806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:54.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.126+0000 7fecf5600640 1 -- 172.21.15.86:0/2899347390 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5853+0+0 (secure 0 0 0) 0x7fecf00adad0 con 0x7fecf8146210 2024-04-20T21:20:54.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.126+0000 7fecfd800640 1 --2- 172.21.15.86:0/2899347390 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fecd806c8f0 0x7fecd806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:54.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.126+0000 7fecfd800640 1 --2- 172.21.15.86:0/2899347390 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fecd806c8f0 0x7fecd806edb0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fecf81e6af0 tx=0x7fecec0039b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:54.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.130+0000 7fecf5600640 1 -- 172.21.15.86:0/2899347390 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fecf007bdd0 con 0x7fecf8146210 2024-04-20T21:20:54.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.375+0000 7fed00200640 1 -- 172.21.15.86:0/2899347390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fecf806c780 con 0x7fecf8146210 2024-04-20T21:20:54.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.917+0000 7fecf5600640 1 -- 172.21.15.86:0/2899347390 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v26) ==== 104+0+0 (secure 0 0 0) 0x7fecf007b6d0 con 0x7fecf8146210 2024-04-20T21:20:54.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.921+0000 7fed00200640 1 -- 172.21.15.86:0/2899347390 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fecd806c8f0 msgr2=0x7fecd806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:54.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.921+0000 7fed00200640 1 --2- 172.21.15.86:0/2899347390 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fecd806c8f0 0x7fecd806edb0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fecf81e6af0 tx=0x7fecec0039b0 comp rx=0 tx=0).stop 2024-04-20T21:20:54.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.921+0000 7fed00200640 1 -- 172.21.15.86:0/2899347390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecf8146210 msgr2=0x7fecf81ea810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:54.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.921+0000 7fed00200640 1 --2- 172.21.15.86:0/2899347390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecf8146210 0x7fecf81ea810 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fecf0009960 tx=0x7fecf00046c0 comp rx=0 tx=0).stop 2024-04-20T21:20:54.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.921+0000 7fed00200640 1 -- 172.21.15.86:0/2899347390 shutdown_connections 2024-04-20T21:20:54.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.921+0000 7fed00200640 1 --2- 172.21.15.86:0/2899347390 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fecd806c8f0 0x7fecd806edb0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:54.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.921+0000 7fed00200640 1 --2- 172.21.15.86:0/2899347390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecf8146210 0x7fecf81ea810 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:54.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.921+0000 7fed00200640 1 --2- 172.21.15.86:0/2899347390 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fecf806bfd0 0x7fecf81ea2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:54.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.921+0000 7fed00200640 1 --2- 172.21.15.86:0/2899347390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fecf806bbf0 0x7fecf8144890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:54.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.921+0000 7fed00200640 1 -- 172.21.15.86:0/2899347390 >> 172.21.15.86:0/2899347390 conn(0x7fecf80b2110 msgr2=0x7fecf815f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:54.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.921+0000 7fed00200640 1 -- 172.21.15.86:0/2899347390 shutdown_connections 2024-04-20T21:20:54.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:54.921+0000 7fed00200640 1 -- 172.21.15.86:0/2899347390 wait complete. 2024-04-20T21:20:54.944 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-20T21:20:55.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.147+0000 7f69b5400640 1 Processor -- start 2024-04-20T21:20:55.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.147+0000 7f69b5400640 1 -- start start 2024-04-20T21:20:55.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.148+0000 7f69b5400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69b0071d00 0x7f69b006f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:55.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.148+0000 7f69b5400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f69b015cc70 0x7f69b015d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:55.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.149+0000 7f69b5400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69b006e010 con 0x7f69b00720e0 2024-04-20T21:20:55.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.149+0000 7f69b5400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f69b006f5f0 con 0x7f69b015cc70 2024-04-20T21:20:55.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.149+0000 7f69b5400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f69b006e2f0 con 0x7f69b0071d00 2024-04-20T21:20:55.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.149+0000 7f69a5e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f69b00720e0 0x7f69b0145a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40500/0 (socket says 172.21.15.86:40500) 2024-04-20T21:20:55.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.149+0000 7f69af200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f69b015cc70 0x7f69b015d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:55.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.149+0000 7f69a5e00640 1 -- 172.21.15.86:0/435705650 learned_addr learned my addr 172.21.15.86:0/435705650 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:20:55.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.149+0000 7f69ae800640 1 --2- 172.21.15.86:0/435705650 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69b0071d00 0x7f69b006f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:55.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.149+0000 7f69ade00640 1 -- 172.21.15.86:0/435705650 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1301373573 0 0) 0x7f69b006e010 con 0x7f69b00720e0 2024-04-20T21:20:55.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.150+0000 7f69ade00640 1 -- 172.21.15.86:0/435705650 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6994003520 con 0x7f69b00720e0 2024-04-20T21:20:55.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.150+0000 7f69af200640 1 -- 172.21.15.86:0/435705650 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69b0071d00 msgr2=0x7f69b006f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:55.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.150+0000 7f69af200640 1 --2- 172.21.15.86:0/435705650 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69b0071d00 0x7f69b006f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:55.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.150+0000 7f69af200640 1 -- 172.21.15.86:0/435705650 >> v1:172.21.15.86:6789/0 conn(0x7f69b00720e0 legacy=0x7f69b0145a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:55.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.150+0000 7f69af200640 1 -- 172.21.15.86:0/435705650 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69b006e5d0 con 0x7f69b015cc70 2024-04-20T21:20:55.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.150+0000 7f69ae800640 1 --2- 172.21.15.86:0/435705650 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69b0071d00 0x7f69b006f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:20:55.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.151+0000 7f69af200640 1 --2- 172.21.15.86:0/435705650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69b015cc70 0x7f69b015d100 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f69a8009d90 tx=0x7f69a802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfe4441586d66682 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:55.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.151+0000 7f69ade00640 1 -- 172.21.15.86:0/435705650 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a8039070 con 0x7f69b015cc70 2024-04-20T21:20:55.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.151+0000 7f69ade00640 1 -- 172.21.15.86:0/435705650 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69a802cc50 con 0x7f69b015cc70 2024-04-20T21:20:55.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.151+0000 7f69ade00640 1 -- 172.21.15.86:0/435705650 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a8034440 con 0x7f69b015cc70 2024-04-20T21:20:55.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.151+0000 7f69b5400640 1 -- 172.21.15.86:0/435705650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69b015cc70 msgr2=0x7f69b015d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:55.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.151+0000 7f69b5400640 1 --2- 172.21.15.86:0/435705650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69b015cc70 0x7f69b015d100 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f69a8009d90 tx=0x7f69a802bea0 comp rx=0 tx=0).stop 2024-04-20T21:20:55.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.152+0000 7f69b5400640 1 -- 172.21.15.86:0/435705650 shutdown_connections 2024-04-20T21:20:55.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.152+0000 7f69b5400640 1 --2- 172.21.15.86:0/435705650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69b015cc70 0x7f69b015d100 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:55.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.152+0000 7f69b5400640 1 --2- 172.21.15.86:0/435705650 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69b0071d00 0x7f69b006f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:55.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.152+0000 7f69b5400640 1 -- 172.21.15.86:0/435705650 >> 172.21.15.86:0/435705650 conn(0x7f69b00b2180 msgr2=0x7f69b00b2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:55.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.152+0000 7f69b5400640 1 -- 172.21.15.86:0/435705650 shutdown_connections 2024-04-20T21:20:55.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.152+0000 7f69b5400640 1 -- 172.21.15.86:0/435705650 wait complete. 2024-04-20T21:20:55.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.153+0000 7f69b5400640 1 Processor -- start 2024-04-20T21:20:55.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.154+0000 7f69b5400640 1 -- start start 2024-04-20T21:20:55.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.154+0000 7f69b5400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69b0071d00 0x7f69b013d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:55.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.154+0000 7f69b5400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69b00720e0 0x7f69b014e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:55.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.154+0000 7f69b5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69b015cc70 0x7f69b00b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:55.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.154+0000 7f69b5400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f69b01461a0 con 0x7f69b015cc70 2024-04-20T21:20:55.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.154+0000 7f69b5400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f69b006eb90 con 0x7f69b0071d00 2024-04-20T21:20:55.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.154+0000 7f69b5400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f69b0072540 con 0x7f69b00720e0 2024-04-20T21:20:55.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.155+0000 7f69af200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69b015cc70 0x7f69b00b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:55.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.155+0000 7f69af200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69b015cc70 0x7f69b00b4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39858/0 (socket says 172.21.15.86:39858) 2024-04-20T21:20:55.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.155+0000 7f69af200640 1 -- 172.21.15.86:0/633166350 learned_addr learned my addr 172.21.15.86:0/633166350 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:55.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.155+0000 7f69a5e00640 1 --2- 172.21.15.86:0/633166350 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69b00720e0 0x7f69b014e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:55.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.155+0000 7f69af200640 1 -- 172.21.15.86:0/633166350 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69b00720e0 msgr2=0x7f69b014e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:55.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.155+0000 7f69ae800640 1 --2- 172.21.15.86:0/633166350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69b0071d00 0x7f69b013d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:55.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.155+0000 7f69af200640 1 --2- 172.21.15.86:0/633166350 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69b00720e0 0x7f69b014e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:55.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.155+0000 7f69af200640 1 -- 172.21.15.86:0/633166350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69b0071d00 msgr2=0x7f69b013d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:55.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.155+0000 7f69af200640 1 --2- 172.21.15.86:0/633166350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69b0071d00 0x7f69b013d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:55.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.155+0000 7f69af200640 1 -- 172.21.15.86:0/633166350 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69b0141780 con 0x7f69b015cc70 2024-04-20T21:20:55.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.156+0000 7f69af200640 1 --2- 172.21.15.86:0/633166350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69b015cc70 0x7f69b00b4e20 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f69a8009eb0 tx=0x7f69a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:55.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.156+0000 7f69a7400640 1 -- 172.21.15.86:0/633166350 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a8039050 con 0x7f69b015cc70 2024-04-20T21:20:55.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.156+0000 7f69a7400640 1 -- 172.21.15.86:0/633166350 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69a8045070 con 0x7f69b015cc70 2024-04-20T21:20:55.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.156+0000 7f69a7400640 1 -- 172.21.15.86:0/633166350 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a8009450 con 0x7f69b015cc70 2024-04-20T21:20:55.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.156+0000 7f69b5400640 1 -- 172.21.15.86:0/633166350 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69b013dd50 con 0x7f69b015cc70 2024-04-20T21:20:55.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.156+0000 7f69b5400640 1 -- 172.21.15.86:0/633166350 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69b013e310 con 0x7f69b015cc70 2024-04-20T21:20:55.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.157+0000 7f69b5400640 1 -- 172.21.15.86:0/633166350 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69b013e550 con 0x7f69b015cc70 2024-04-20T21:20:55.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.158+0000 7f69a7400640 1 -- 172.21.15.86:0/633166350 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f69a803e610 con 0x7f69b015cc70 2024-04-20T21:20:55.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.159+0000 7f69a7400640 1 --2- 172.21.15.86:0/633166350 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f698c06c8f0 0x7f698c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:55.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.159+0000 7f69a7400640 1 -- 172.21.15.86:0/633166350 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5853+0+0 (secure 0 0 0) 0x7f69a80ae6e0 con 0x7f69b015cc70 2024-04-20T21:20:55.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.159+0000 7f69ae800640 1 --2- 172.21.15.86:0/633166350 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f698c06c8f0 0x7f698c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:55.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.160+0000 7f69ae800640 1 --2- 172.21.15.86:0/633166350 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f698c06c8f0 0x7f698c06edb0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f69a00096d0 tx=0x7f69a0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:55.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.164+0000 7f69a7400640 1 -- 172.21.15.86:0/633166350 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f69a807c9e0 con 0x7f69b015cc70 2024-04-20T21:20:55.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.361+0000 7f69b5400640 1 -- 172.21.15.86:0/633166350 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f69b013da40 con 0x7f69b015cc70 2024-04-20T21:20:55.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.930+0000 7f69a7400640 1 -- 172.21.15.86:0/633166350 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v49) ==== 181+0+0 (secure 0 0 0) 0x7f69a807c2e0 con 0x7f69b015cc70 2024-04-20T21:20:55.930 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' removed 2024-04-20T21:20:55.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.934+0000 7f69b5400640 1 -- 172.21.15.86:0/633166350 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f698c06c8f0 msgr2=0x7f698c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:55.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.934+0000 7f69b5400640 1 --2- 172.21.15.86:0/633166350 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f698c06c8f0 0x7f698c06edb0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f69a00096d0 tx=0x7f69a0008040 comp rx=0 tx=0).stop 2024-04-20T21:20:55.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.934+0000 7f69b5400640 1 -- 172.21.15.86:0/633166350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69b015cc70 msgr2=0x7f69b00b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:55.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.934+0000 7f69b5400640 1 --2- 172.21.15.86:0/633166350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69b015cc70 0x7f69b00b4e20 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f69a8009eb0 tx=0x7f69a8004a40 comp rx=0 tx=0).stop 2024-04-20T21:20:55.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.934+0000 7f69b5400640 1 -- 172.21.15.86:0/633166350 shutdown_connections 2024-04-20T21:20:55.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.934+0000 7f69b5400640 1 --2- 172.21.15.86:0/633166350 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f698c06c8f0 0x7f698c06edb0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:55.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.934+0000 7f69b5400640 1 --2- 172.21.15.86:0/633166350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69b015cc70 0x7f69b00b4e20 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:55.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.934+0000 7f69b5400640 1 --2- 172.21.15.86:0/633166350 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f69b00720e0 0x7f69b014e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:55.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.934+0000 7f69b5400640 1 --2- 172.21.15.86:0/633166350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69b0071d00 0x7f69b013d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:55.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.934+0000 7f69b5400640 1 -- 172.21.15.86:0/633166350 >> 172.21.15.86:0/633166350 conn(0x7f69b00b2180 msgr2=0x7f69b00ae2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:55.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.935+0000 7f69b5400640 1 -- 172.21.15.86:0/633166350 shutdown_connections 2024-04-20T21:20:55.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:55.935+0000 7f69b5400640 1 -- 172.21.15.86:0/633166350 wait complete. 2024-04-20T21:20:55.961 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-20T21:20:56.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.159+0000 7f4d8f600640 1 Processor -- start 2024-04-20T21:20:56.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.159+0000 7f4d8f600640 1 -- start start 2024-04-20T21:20:56.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.161+0000 7f4d8f600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4d880afb60 0x7f4d8806ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:56.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.161+0000 7f4d8f600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d8815ca50 con 0x7f4d8806f280 2024-04-20T21:20:56.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.161+0000 7f4d8f600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d880cdef0 con 0x7f4d880cd8d0 2024-04-20T21:20:56.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.161+0000 7f4d8f600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f4d8815bc90 con 0x7f4d880afb60 2024-04-20T21:20:56.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.161+0000 7f4d87e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f4d8806f280 0x7f4d880cd1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54158/0 (socket says 172.21.15.86:54158) 2024-04-20T21:20:56.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.161+0000 7f4d87e00640 1 -- 172.21.15.86:0/450867996 learned_addr learned my addr 172.21.15.86:0/450867996 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:20:56.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.162+0000 7f4d8cc00640 1 --2- 172.21.15.86:0/450867996 >> v2:172.21.15.144:3300/0 conn(0x7f4d880afb60 0x7f4d8806ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:56.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.162+0000 7f4d87400640 1 -- 172.21.15.86:0/450867996 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3467999917 0 0) 0x7f4d8815ca50 con 0x7f4d8806f280 2024-04-20T21:20:56.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.162+0000 7f4d87400640 1 -- 172.21.15.86:0/450867996 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d70003520 con 0x7f4d8806f280 2024-04-20T21:20:56.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.162+0000 7f4d87400640 1 -- 172.21.15.86:0/450867996 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2578192612 0 0) 0x7f4d70003520 con 0x7f4d8806f280 2024-04-20T21:20:56.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.162+0000 7f4d87400640 1 -- 172.21.15.86:0/450867996 >> v2:172.21.15.144:3300/0 conn(0x7f4d880afb60 msgr2=0x7f4d8806ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:56.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.162+0000 7f4d87400640 1 --2- 172.21.15.86:0/450867996 >> v2:172.21.15.144:3300/0 conn(0x7f4d880afb60 0x7f4d8806ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:56.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.162+0000 7f4d87400640 1 -- 172.21.15.86:0/450867996 >> v1:172.21.15.144:6789/0 conn(0x7f4d880cd8d0 legacy=0x7f4d880b39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:56.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.162+0000 7f4d87400640 1 -- 172.21.15.86:0/450867996 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d8815bf70 con 0x7f4d8806f280 2024-04-20T21:20:56.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.162+0000 7f4d87400640 1 -- 172.21.15.86:0/450867996 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f4d740017f0 con 0x7f4d8806f280 2024-04-20T21:20:56.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.162+0000 7f4d87400640 1 -- 172.21.15.86:0/450867996 >> v1:172.21.15.86:6789/0 conn(0x7f4d8806f280 legacy=0x7f4d880cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:56.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.162+0000 7f4d87400640 1 --2- 172.21.15.86:0/450867996 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d70003a00 0x7f4d70003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:56.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.162+0000 7f4d87400640 1 --2- 172.21.15.86:0/450867996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d70004800 0x7f4d70004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:56.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.162+0000 7f4d87400640 1 --2- 172.21.15.86:0/450867996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d700051c0 0x7f4d7000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:56.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.162+0000 7f4d87400640 1 -- 172.21.15.86:0/450867996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8815bc90 con 0x7f4d700051c0 2024-04-20T21:20:56.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.162+0000 7f4d87400640 1 -- 172.21.15.86:0/450867996 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4d7000c0c0 con 0x7f4d70004800 2024-04-20T21:20:56.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.162+0000 7f4d87400640 1 -- 172.21.15.86:0/450867996 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f4d7000c2c0 con 0x7f4d70003a00 2024-04-20T21:20:56.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.163+0000 7f4d8d600640 1 --2- 172.21.15.86:0/450867996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d700051c0 0x7f4d7000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:56.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.163+0000 7f4d87e00640 1 --2- 172.21.15.86:0/450867996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d70004800 0x7f4d70004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:56.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.163+0000 7f4d8cc00640 1 --2- 172.21.15.86:0/450867996 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d70003a00 0x7f4d70003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:56.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.163+0000 7f4d8d600640 1 -- 172.21.15.86:0/450867996 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d70003a00 msgr2=0x7f4d70003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:56.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.163+0000 7f4d8d600640 1 --2- 172.21.15.86:0/450867996 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d70003a00 0x7f4d70003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:56.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.163+0000 7f4d8d600640 1 -- 172.21.15.86:0/450867996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d70004800 msgr2=0x7f4d70004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:56.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.163+0000 7f4d8d600640 1 --2- 172.21.15.86:0/450867996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d70004800 0x7f4d70004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:56.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.163+0000 7f4d8d600640 1 -- 172.21.15.86:0/450867996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d70003520 con 0x7f4d700051c0 2024-04-20T21:20:56.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.164+0000 7f4d8d600640 1 -- 172.21.15.86:0/450867996 reap_dead start 2024-04-20T21:20:56.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.164+0000 7f4d8d600640 1 --2- 172.21.15.86:0/450867996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d700051c0 0x7f4d7000b9c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f4d80001e50 tx=0x7f4d70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9e86195c71f154f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:56.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.165+0000 7f4d87400640 1 -- 172.21.15.86:0/450867996 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d80001a70 con 0x7f4d700051c0 2024-04-20T21:20:56.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.165+0000 7f4d87400640 1 -- 172.21.15.86:0/450867996 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d80003b90 con 0x7f4d700051c0 2024-04-20T21:20:56.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.165+0000 7f4d87400640 1 -- 172.21.15.86:0/450867996 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d800050d0 con 0x7f4d700051c0 2024-04-20T21:20:56.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.165+0000 7f4d8f600640 1 -- 172.21.15.86:0/450867996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d700051c0 msgr2=0x7f4d7000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:56.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.165+0000 7f4d8f600640 1 --2- 172.21.15.86:0/450867996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d700051c0 0x7f4d7000b9c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f4d80001e50 tx=0x7f4d70005830 comp rx=0 tx=0).stop 2024-04-20T21:20:56.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.165+0000 7f4d8f600640 1 -- 172.21.15.86:0/450867996 shutdown_connections 2024-04-20T21:20:56.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.165+0000 7f4d8f600640 1 --2- 172.21.15.86:0/450867996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d700051c0 0x7f4d7000b9c0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:56.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.165+0000 7f4d8f600640 1 -- 172.21.15.86:0/450867996 >> 172.21.15.86:0/450867996 conn(0x7f4d880b2180 msgr2=0x7f4d880b2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:56.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.165+0000 7f4d8f600640 1 -- 172.21.15.86:0/450867996 shutdown_connections 2024-04-20T21:20:56.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.165+0000 7f4d8f600640 1 -- 172.21.15.86:0/450867996 wait complete. 2024-04-20T21:20:56.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.167+0000 7f4d8f600640 1 Processor -- start 2024-04-20T21:20:56.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.167+0000 7f4d8f600640 1 -- start start 2024-04-20T21:20:56.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.168+0000 7f4d8f600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d700051c0 0x7f4d88142f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:56.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.168+0000 7f4d8f600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d88148a10 0x7f4d8815d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:56.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.168+0000 7f4d8f600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d88148e90 0x7f4d88145620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:56.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.168+0000 7f4d8f600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4d880cd1c0 con 0x7f4d700051c0 2024-04-20T21:20:56.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.168+0000 7f4d8f600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8815c530 con 0x7f4d88148a10 2024-04-20T21:20:56.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.168+0000 7f4d8f600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f4d880cd340 con 0x7f4d88148e90 2024-04-20T21:20:56.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.168+0000 7f4d8cc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d700051c0 0x7f4d88142f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:56.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.168+0000 7f4d8cc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d700051c0 0x7f4d88142f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55608/0 (socket says 172.21.15.86:55608) 2024-04-20T21:20:56.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.168+0000 7f4d8cc00640 1 -- 172.21.15.86:0/3450968315 learned_addr learned my addr 172.21.15.86:0/3450968315 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:56.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.169+0000 7f4d87e00640 1 --2- 172.21.15.86:0/3450968315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d88148a10 0x7f4d8815d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:56.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.169+0000 7f4d8d600640 1 --2- 172.21.15.86:0/3450968315 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d88148e90 0x7f4d88145620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:56.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.169+0000 7f4d8cc00640 1 -- 172.21.15.86:0/3450968315 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d88148e90 msgr2=0x7f4d88145620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:56.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.169+0000 7f4d8cc00640 1 --2- 172.21.15.86:0/3450968315 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d88148e90 0x7f4d88145620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:56.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.169+0000 7f4d8cc00640 1 -- 172.21.15.86:0/3450968315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d88148a10 msgr2=0x7f4d8815d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:56.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.169+0000 7f4d8cc00640 1 --2- 172.21.15.86:0/3450968315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d88148a10 0x7f4d8815d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:56.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.169+0000 7f4d8cc00640 1 -- 172.21.15.86:0/3450968315 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d88143780 con 0x7f4d700051c0 2024-04-20T21:20:56.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.169+0000 7f4d8cc00640 1 --2- 172.21.15.86:0/3450968315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d700051c0 0x7f4d88142f70 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c011570 tx=0x7f4d7c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:56.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.169+0000 7f4d84c00640 1 -- 172.21.15.86:0/3450968315 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c015930 con 0x7f4d700051c0 2024-04-20T21:20:56.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.169+0000 7f4d84c00640 1 -- 172.21.15.86:0/3450968315 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d7c015ad0 con 0x7f4d700051c0 2024-04-20T21:20:56.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.169+0000 7f4d84c00640 1 -- 172.21.15.86:0/3450968315 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c00f460 con 0x7f4d700051c0 2024-04-20T21:20:56.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.170+0000 7f4d8f600640 1 -- 172.21.15.86:0/3450968315 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d88144580 con 0x7f4d700051c0 2024-04-20T21:20:56.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.170+0000 7f4d8f600640 1 -- 172.21.15.86:0/3450968315 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d88144b40 con 0x7f4d700051c0 2024-04-20T21:20:56.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.171+0000 7f4d8f600640 1 -- 172.21.15.86:0/3450968315 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d880689e0 con 0x7f4d700051c0 2024-04-20T21:20:56.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.172+0000 7f4d84c00640 1 -- 172.21.15.86:0/3450968315 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4d7c00f600 con 0x7f4d700051c0 2024-04-20T21:20:56.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.173+0000 7f4d84c00640 1 --2- 172.21.15.86:0/3450968315 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4d6806c8f0 0x7f4d6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:56.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.173+0000 7f4d84c00640 1 -- 172.21.15.86:0/3450968315 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5415+0+0 (secure 0 0 0) 0x7f4d7c08c290 con 0x7f4d700051c0 2024-04-20T21:20:56.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.173+0000 7f4d87e00640 1 --2- 172.21.15.86:0/3450968315 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4d6806c8f0 0x7f4d6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:56.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.174+0000 7f4d87e00640 1 --2- 172.21.15.86:0/3450968315 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4d6806c8f0 0x7f4d6806edb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4d880ce850 tx=0x7f4d74009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:56.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.178+0000 7f4d84c00640 1 -- 172.21.15.86:0/3450968315 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4d7c019030 con 0x7f4d700051c0 2024-04-20T21:20:56.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.378+0000 7f4d8f600640 1 -- 172.21.15.86:0/3450968315 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4d880cc670 con 0x7f4d700051c0 2024-04-20T21:20:56.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.940+0000 7f4d84c00640 1 -- 172.21.15.86:0/3450968315 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v50) ==== 169+0+0 (secure 0 0 0) 0x7f4d7c05a810 con 0x7f4d700051c0 2024-04-20T21:20:56.939 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' removed 2024-04-20T21:20:56.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.944+0000 7f4d8f600640 1 -- 172.21.15.86:0/3450968315 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4d6806c8f0 msgr2=0x7f4d6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:56.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.944+0000 7f4d8f600640 1 --2- 172.21.15.86:0/3450968315 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4d6806c8f0 0x7f4d6806edb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4d880ce850 tx=0x7f4d74009040 comp rx=0 tx=0).stop 2024-04-20T21:20:56.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.944+0000 7f4d8f600640 1 -- 172.21.15.86:0/3450968315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d700051c0 msgr2=0x7f4d88142f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:56.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.944+0000 7f4d8f600640 1 --2- 172.21.15.86:0/3450968315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d700051c0 0x7f4d88142f70 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c011570 tx=0x7f4d7c011a30 comp rx=0 tx=0).stop 2024-04-20T21:20:56.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.946+0000 7f4d8f600640 1 -- 172.21.15.86:0/3450968315 shutdown_connections 2024-04-20T21:20:56.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.946+0000 7f4d8f600640 1 --2- 172.21.15.86:0/3450968315 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4d6806c8f0 0x7f4d6806edb0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:56.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.946+0000 7f4d8f600640 1 --2- 172.21.15.86:0/3450968315 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d88148e90 0x7f4d88145620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:56.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.946+0000 7f4d8f600640 1 --2- 172.21.15.86:0/3450968315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d88148a10 0x7f4d8815d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:56.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.946+0000 7f4d8f600640 1 --2- 172.21.15.86:0/3450968315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d700051c0 0x7f4d88142f70 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:56.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.946+0000 7f4d8f600640 1 -- 172.21.15.86:0/3450968315 >> 172.21.15.86:0/3450968315 conn(0x7f4d880b2180 msgr2=0x7f4d8806e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:56.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.947+0000 7f4d8f600640 1 -- 172.21.15.86:0/3450968315 shutdown_connections 2024-04-20T21:20:56.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:56.947+0000 7f4d8f600640 1 -- 172.21.15.86:0/3450968315 wait complete. 2024-04-20T21:20:56.971 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-20T21:20:57.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.161+0000 7f3315e00640 1 Processor -- start 2024-04-20T21:20:57.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.162+0000 7f3315e00640 1 -- start start 2024-04-20T21:20:57.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.163+0000 7f3315e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3310072450 0x7f3310167670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:57.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.163+0000 7f3315e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3310160410 con 0x7f3310145250 2024-04-20T21:20:57.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.163+0000 7f3315e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33101606f0 con 0x7f3310145f00 2024-04-20T21:20:57.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.163+0000 7f3315e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f33101609d0 con 0x7f3310072450 2024-04-20T21:20:57.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.164+0000 7f3314a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f3310145250 0x7f331006efc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54170/0 (socket says 172.21.15.86:54170) 2024-04-20T21:20:57.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.164+0000 7f3314a00640 1 -- 172.21.15.86:0/49161148 learned_addr learned my addr 172.21.15.86:0/49161148 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:20:57.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.164+0000 7f330f400640 1 -- 172.21.15.86:0/49161148 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2371651676 0 0) 0x7f3310160410 con 0x7f3310145250 2024-04-20T21:20:57.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.164+0000 7f330f400640 1 -- 172.21.15.86:0/49161148 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32f8003520 con 0x7f3310145250 2024-04-20T21:20:57.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.164+0000 7f3315400640 1 --2- 172.21.15.86:0/49161148 >> v2:172.21.15.144:3300/0 conn(0x7f3310072450 0x7f3310167670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:57.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.164+0000 7f330f400640 1 -- 172.21.15.86:0/49161148 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4137182379 0 0) 0x7f32f8003520 con 0x7f3310145250 2024-04-20T21:20:57.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.164+0000 7f330f400640 1 -- 172.21.15.86:0/49161148 >> v2:172.21.15.144:3300/0 conn(0x7f3310072450 msgr2=0x7f3310167670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:57.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.164+0000 7f330f400640 1 --2- 172.21.15.86:0/49161148 >> v2:172.21.15.144:3300/0 conn(0x7f3310072450 0x7f3310167670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:57.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.164+0000 7f330f400640 1 -- 172.21.15.86:0/49161148 >> v1:172.21.15.144:6789/0 conn(0x7f3310145f00 legacy=0x7f3310071d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:57.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.164+0000 7f330f400640 1 -- 172.21.15.86:0/49161148 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3310160cb0 con 0x7f3310145250 2024-04-20T21:20:57.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.164+0000 7f330f400640 1 -- 172.21.15.86:0/49161148 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f3304001670 con 0x7f3310145250 2024-04-20T21:20:57.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.164+0000 7f330f400640 1 -- 172.21.15.86:0/49161148 >> v1:172.21.15.86:6789/0 conn(0x7f3310145250 legacy=0x7f331006efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:57.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.165+0000 7f330f400640 1 --2- 172.21.15.86:0/49161148 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32f8003a70 0x7f32f8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:57.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.165+0000 7f330f400640 1 --2- 172.21.15.86:0/49161148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32f8004800 0x7f32f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:57.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.165+0000 7f330f400640 1 --2- 172.21.15.86:0/49161148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32f80053c0 0x7f32f800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:57.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.165+0000 7f330f400640 1 -- 172.21.15.86:0/49161148 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f33101609d0 con 0x7f32f8004800 2024-04-20T21:20:57.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.165+0000 7f330f400640 1 -- 172.21.15.86:0/49161148 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f32f800c250 con 0x7f32f80053c0 2024-04-20T21:20:57.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.165+0000 7f330f400640 1 -- 172.21.15.86:0/49161148 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f32f800c450 con 0x7f32f8003a70 2024-04-20T21:20:57.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.165+0000 7f330fe00640 1 --2- 172.21.15.86:0/49161148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32f8004800 0x7f32f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:57.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.165+0000 7f3315400640 1 --2- 172.21.15.86:0/49161148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32f80053c0 0x7f32f800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:57.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.165+0000 7f3314a00640 1 --2- 172.21.15.86:0/49161148 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32f8003a70 0x7f32f8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:57.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.165+0000 7f330fe00640 1 -- 172.21.15.86:0/49161148 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32f8003a70 msgr2=0x7f32f8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:57.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.165+0000 7f330fe00640 1 --2- 172.21.15.86:0/49161148 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32f8003a70 0x7f32f8003e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:57.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.165+0000 7f330fe00640 1 -- 172.21.15.86:0/49161148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32f80053c0 msgr2=0x7f32f800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:57.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.165+0000 7f330fe00640 1 --2- 172.21.15.86:0/49161148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32f80053c0 0x7f32f800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:57.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.165+0000 7f330fe00640 1 -- 172.21.15.86:0/49161148 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32f8003520 con 0x7f32f8004800 2024-04-20T21:20:57.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.165+0000 7f3315400640 1 -- 172.21.15.86:0/49161148 reap_dead start 2024-04-20T21:20:57.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.166+0000 7f330fe00640 1 --2- 172.21.15.86:0/49161148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32f8004800 0x7f32f8004c80 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f3300003db0 tx=0x7f32f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c0103721d11022a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:57.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.167+0000 7f330f400640 1 -- 172.21.15.86:0/49161148 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3300037070 con 0x7f32f8004800 2024-04-20T21:20:57.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.167+0000 7f330f400640 1 -- 172.21.15.86:0/49161148 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f330002be50 con 0x7f32f8004800 2024-04-20T21:20:57.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.167+0000 7f330f400640 1 -- 172.21.15.86:0/49161148 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3300003980 con 0x7f32f8004800 2024-04-20T21:20:57.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.167+0000 7f3315e00640 1 -- 172.21.15.86:0/49161148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32f8004800 msgr2=0x7f32f8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:57.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.167+0000 7f3315e00640 1 --2- 172.21.15.86:0/49161148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32f8004800 0x7f32f8004c80 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f3300003db0 tx=0x7f32f8005830 comp rx=0 tx=0).stop 2024-04-20T21:20:57.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.167+0000 7f3315e00640 1 -- 172.21.15.86:0/49161148 shutdown_connections 2024-04-20T21:20:57.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.167+0000 7f3315e00640 1 --2- 172.21.15.86:0/49161148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32f8004800 0x7f32f8004c80 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:57.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.167+0000 7f3315e00640 1 -- 172.21.15.86:0/49161148 >> 172.21.15.86:0/49161148 conn(0x7f3310067cf0 msgr2=0x7f331006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:57.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.167+0000 7f3315e00640 1 -- 172.21.15.86:0/49161148 shutdown_connections 2024-04-20T21:20:57.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.167+0000 7f3315e00640 1 -- 172.21.15.86:0/49161148 wait complete. 2024-04-20T21:20:57.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.169+0000 7f3315e00640 1 Processor -- start 2024-04-20T21:20:57.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.169+0000 7f3315e00640 1 -- start start 2024-04-20T21:20:57.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.170+0000 7f3315e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32f8004800 0x7f33100b3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:57.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.170+0000 7f3315e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3310141a40 0x7f3310141ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:57.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.170+0000 7f3315e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33101f87b0 0x7f33101f8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:57.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.170+0000 7f3315e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3310071ec0 con 0x7f33101f87b0 2024-04-20T21:20:57.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.170+0000 7f3315e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3310071d40 con 0x7f32f8004800 2024-04-20T21:20:57.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.170+0000 7f3315e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3310072040 con 0x7f3310141a40 2024-04-20T21:20:57.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.170+0000 7f3315400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33101f87b0 0x7f33101f8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:57.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.171+0000 7f3315400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33101f87b0 0x7f33101f8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55620/0 (socket says 172.21.15.86:55620) 2024-04-20T21:20:57.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.171+0000 7f3315400640 1 -- 172.21.15.86:0/1637153864 learned_addr learned my addr 172.21.15.86:0/1637153864 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:57.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.171+0000 7f3314a00640 1 --2- 172.21.15.86:0/1637153864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32f8004800 0x7f33100b3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:57.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.171+0000 7f330fe00640 1 --2- 172.21.15.86:0/1637153864 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3310141a40 0x7f3310141ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:57.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.171+0000 7f3315400640 1 -- 172.21.15.86:0/1637153864 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3310141a40 msgr2=0x7f3310141ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:57.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.171+0000 7f3315400640 1 --2- 172.21.15.86:0/1637153864 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3310141a40 0x7f3310141ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:57.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.171+0000 7f3315400640 1 -- 172.21.15.86:0/1637153864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32f8004800 msgr2=0x7f33100b3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:57.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.171+0000 7f3315400640 1 --2- 172.21.15.86:0/1637153864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32f8004800 0x7f33100b3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:57.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.171+0000 7f3315400640 1 -- 172.21.15.86:0/1637153864 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3310142400 con 0x7f33101f87b0 2024-04-20T21:20:57.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.171+0000 7f3315400640 1 --2- 172.21.15.86:0/1637153864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33101f87b0 0x7f33101f8c30 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f33100681c0 tx=0x7f330800e510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:57.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.171+0000 7f330cc00640 1 -- 172.21.15.86:0/1637153864 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330801b070 con 0x7f33101f87b0 2024-04-20T21:20:57.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.172+0000 7f330cc00640 1 -- 172.21.15.86:0/1637153864 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3308007e40 con 0x7f33101f87b0 2024-04-20T21:20:57.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.172+0000 7f330cc00640 1 -- 172.21.15.86:0/1637153864 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3308003bd0 con 0x7f33101f87b0 2024-04-20T21:20:57.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.172+0000 7f3315e00640 1 -- 172.21.15.86:0/1637153864 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33100b4700 con 0x7f33101f87b0 2024-04-20T21:20:57.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.172+0000 7f3315e00640 1 -- 172.21.15.86:0/1637153864 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33100b4cc0 con 0x7f33101f87b0 2024-04-20T21:20:57.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.173+0000 7f3315e00640 1 -- 172.21.15.86:0/1637153864 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3310160000 con 0x7f33101f87b0 2024-04-20T21:20:57.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.174+0000 7f330cc00640 1 -- 172.21.15.86:0/1637153864 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f330801e070 con 0x7f33101f87b0 2024-04-20T21:20:57.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.175+0000 7f330cc00640 1 --2- 172.21.15.86:0/1637153864 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f32f406c920 0x7f32f406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:57.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.175+0000 7f330cc00640 1 -- 172.21.15.86:0/1637153864 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f330808bd50 con 0x7f33101f87b0 2024-04-20T21:20:57.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.175+0000 7f3314a00640 1 --2- 172.21.15.86:0/1637153864 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f32f406c920 0x7f32f406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:57.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.176+0000 7f3314a00640 1 --2- 172.21.15.86:0/1637153864 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f32f406c920 0x7f32f406ede0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f331002db60 tx=0x7f3304009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:57.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.180+0000 7f330cc00640 1 -- 172.21.15.86:0/1637153864 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3310160000 con 0x7f33101f87b0 2024-04-20T21:20:57.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.380+0000 7f3315e00640 1 -- 172.21.15.86:0/1637153864 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f33100b3060 con 0x7f33101f87b0 2024-04-20T21:20:57.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.387+0000 7f330cc00640 1 -- 172.21.15.86:0/1637153864 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) ==== 104+0+0 (secure 0 0 0) 0x7f3308010040 con 0x7f33101f87b0 2024-04-20T21:20:57.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.390+0000 7f3315e00640 1 -- 172.21.15.86:0/1637153864 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f32f406c920 msgr2=0x7f32f406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:57.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.390+0000 7f3315e00640 1 --2- 172.21.15.86:0/1637153864 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f32f406c920 0x7f32f406ede0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f331002db60 tx=0x7f3304009040 comp rx=0 tx=0).stop 2024-04-20T21:20:57.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.390+0000 7f3315e00640 1 -- 172.21.15.86:0/1637153864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33101f87b0 msgr2=0x7f33101f8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:57.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.390+0000 7f3315e00640 1 --2- 172.21.15.86:0/1637153864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33101f87b0 0x7f33101f8c30 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f33100681c0 tx=0x7f330800e510 comp rx=0 tx=0).stop 2024-04-20T21:20:57.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.390+0000 7f3315e00640 1 -- 172.21.15.86:0/1637153864 shutdown_connections 2024-04-20T21:20:57.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.390+0000 7f3315e00640 1 --2- 172.21.15.86:0/1637153864 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f32f406c920 0x7f32f406ede0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:57.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.390+0000 7f3315e00640 1 --2- 172.21.15.86:0/1637153864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33101f87b0 0x7f33101f8c30 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:57.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.390+0000 7f3315e00640 1 --2- 172.21.15.86:0/1637153864 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3310141a40 0x7f3310141ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:57.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.390+0000 7f3315e00640 1 --2- 172.21.15.86:0/1637153864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32f8004800 0x7f33100b3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:57.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.390+0000 7f3315e00640 1 -- 172.21.15.86:0/1637153864 >> 172.21.15.86:0/1637153864 conn(0x7f3310067cf0 msgr2=0x7f3310144480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:57.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.390+0000 7f3315e00640 1 -- 172.21.15.86:0/1637153864 shutdown_connections 2024-04-20T21:20:57.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.390+0000 7f3315e00640 1 -- 172.21.15.86:0/1637153864 wait complete. 2024-04-20T21:20:57.409 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-04-20T21:20:57.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.605+0000 7f5c3be00640 1 Processor -- start 2024-04-20T21:20:57.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.605+0000 7f5c3be00640 1 -- start start 2024-04-20T21:20:57.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.607+0000 7f5c3be00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c3c15ece0 0x7f5c3c15f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:57.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.607+0000 7f5c3be00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c3c15f600 con 0x7f5c3c0cf9c0 2024-04-20T21:20:57.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.607+0000 7f5c3be00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c3c0d3950 con 0x7f5c3c0d50c0 2024-04-20T21:20:57.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.607+0000 7f5c3be00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5c3c0cec00 con 0x7f5c3c15ece0 2024-04-20T21:20:57.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.607+0000 7f5c3b400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f5c3c0cf9c0 0x7f5c3c160000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54184/0 (socket says 172.21.15.86:54184) 2024-04-20T21:20:57.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.607+0000 7f5c3b400640 1 -- 172.21.15.86:0/3951573302 learned_addr learned my addr 172.21.15.86:0/3951573302 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:20:57.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.607+0000 7f5c3aa00640 1 --2- 172.21.15.86:0/3951573302 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c3c15ece0 0x7f5c3c15f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:57.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.607+0000 7f5c3a000640 1 -- 172.21.15.86:0/3951573302 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3782862333 0 0) 0x7f5c3c15f600 con 0x7f5c3c0cf9c0 2024-04-20T21:20:57.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.607+0000 7f5c3a000640 1 -- 172.21.15.86:0/3951573302 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5c20003520 con 0x7f5c3c0cf9c0 2024-04-20T21:20:57.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.608+0000 7f5c3a000640 1 -- 172.21.15.86:0/3951573302 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1132533966 0 0) 0x7f5c20003520 con 0x7f5c3c0cf9c0 2024-04-20T21:20:57.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.608+0000 7f5c3a000640 1 -- 172.21.15.86:0/3951573302 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c3c15ece0 msgr2=0x7f5c3c15f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:57.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.608+0000 7f5c3a000640 1 --2- 172.21.15.86:0/3951573302 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c3c15ece0 0x7f5c3c15f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:57.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.608+0000 7f5c3a000640 1 -- 172.21.15.86:0/3951573302 >> v1:172.21.15.144:6789/0 conn(0x7f5c3c0d50c0 legacy=0x7f5c3c0d3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:57.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.608+0000 7f5c3a000640 1 -- 172.21.15.86:0/3951573302 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c3c0ceee0 con 0x7f5c3c0cf9c0 2024-04-20T21:20:57.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.608+0000 7f5c3a000640 1 -- 172.21.15.86:0/3951573302 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f5c340019f0 con 0x7f5c3c0cf9c0 2024-04-20T21:20:57.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.608+0000 7f5c3a000640 1 -- 172.21.15.86:0/3951573302 >> v1:172.21.15.86:6789/0 conn(0x7f5c3c0cf9c0 legacy=0x7f5c3c160000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:57.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.608+0000 7f5c3a000640 1 --2- 172.21.15.86:0/3951573302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c200039e0 0x7f5c20003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:57.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.608+0000 7f5c3a000640 1 --2- 172.21.15.86:0/3951573302 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c3c15ece0 0x7f5c20004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:57.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.608+0000 7f5c3a000640 1 --2- 172.21.15.86:0/3951573302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c20004d40 0x7f5c200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:57.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.608+0000 7f5c3a000640 1 -- 172.21.15.86:0/3951573302 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5c3c0cec00 con 0x7f5c200039e0 2024-04-20T21:20:57.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.608+0000 7f5c3a000640 1 -- 172.21.15.86:0/3951573302 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2000bba0 con 0x7f5c20004d40 2024-04-20T21:20:57.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.608+0000 7f5c3a000640 1 -- 172.21.15.86:0/3951573302 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5c2000bda0 con 0x7f5c3c15ece0 2024-04-20T21:20:57.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.608+0000 7f5c3aa00640 1 --2- 172.21.15.86:0/3951573302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c200039e0 0x7f5c20003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:57.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.609+0000 7f5c3b400640 1 --2- 172.21.15.86:0/3951573302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c20004d40 0x7f5c200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:57.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.609+0000 7f5c32000640 1 --2- 172.21.15.86:0/3951573302 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c3c15ece0 0x7f5c20004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:57.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.609+0000 7f5c3aa00640 1 -- 172.21.15.86:0/3951573302 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c3c15ece0 msgr2=0x7f5c20004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:57.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.609+0000 7f5c3aa00640 1 --2- 172.21.15.86:0/3951573302 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c3c15ece0 0x7f5c20004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:57.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.609+0000 7f5c3aa00640 1 -- 172.21.15.86:0/3951573302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c20004d40 msgr2=0x7f5c200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:57.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.609+0000 7f5c3aa00640 1 --2- 172.21.15.86:0/3951573302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c20004d40 0x7f5c200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:57.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.609+0000 7f5c3aa00640 1 -- 172.21.15.86:0/3951573302 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c20003520 con 0x7f5c200039e0 2024-04-20T21:20:57.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.609+0000 7f5c3b400640 1 --2- 172.21.15.86:0/3951573302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c20004d40 0x7f5c200051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:20:57.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.610+0000 7f5c3aa00640 1 --2- 172.21.15.86:0/3951573302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c200039e0 0x7f5c20003e00 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c00ba30 tx=0x7f5c20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c53f632474277d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:57.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.610+0000 7f5c3a000640 1 -- 172.21.15.86:0/3951573302 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c036020 con 0x7f5c200039e0 2024-04-20T21:20:57.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.610+0000 7f5c3a000640 1 -- 172.21.15.86:0/3951573302 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c2c032630 con 0x7f5c200039e0 2024-04-20T21:20:57.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.610+0000 7f5c3a000640 1 -- 172.21.15.86:0/3951573302 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c00b600 con 0x7f5c200039e0 2024-04-20T21:20:57.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.611+0000 7f5c3be00640 1 -- 172.21.15.86:0/3951573302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c200039e0 msgr2=0x7f5c20003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:57.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.611+0000 7f5c3be00640 1 --2- 172.21.15.86:0/3951573302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c200039e0 0x7f5c20003e00 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c00ba30 tx=0x7f5c20005830 comp rx=0 tx=0).stop 2024-04-20T21:20:57.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.611+0000 7f5c3b400640 1 -- 172.21.15.86:0/3951573302 reap_dead start 2024-04-20T21:20:57.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.611+0000 7f5c3be00640 1 -- 172.21.15.86:0/3951573302 shutdown_connections 2024-04-20T21:20:57.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.611+0000 7f5c3be00640 1 -- 172.21.15.86:0/3951573302 >> 172.21.15.86:0/3951573302 conn(0x7f5c3c08bf70 msgr2=0x7f5c3c0d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:57.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.611+0000 7f5c3be00640 1 -- 172.21.15.86:0/3951573302 shutdown_connections 2024-04-20T21:20:57.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.611+0000 7f5c3be00640 1 -- 172.21.15.86:0/3951573302 wait complete. 2024-04-20T21:20:57.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.613+0000 7f5c3be00640 1 Processor -- start 2024-04-20T21:20:57.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.613+0000 7f5c3be00640 1 -- start start 2024-04-20T21:20:57.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.614+0000 7f5c3be00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c200039e0 0x7f5c3c157180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:57.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.614+0000 7f5c3be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c3c156350 0x7f5c3c0b9b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:57.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.614+0000 7f5c3be00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c3c0ba0b0 0x7f5c3c0cc760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:57.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.614+0000 7f5c3be00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5c3c15ea50 con 0x7f5c3c156350 2024-04-20T21:20:57.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.614+0000 7f5c3be00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5c3c0cf4a0 con 0x7f5c3c0ba0b0 2024-04-20T21:20:57.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.614+0000 7f5c3be00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5c3c0cfe20 con 0x7f5c200039e0 2024-04-20T21:20:57.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.614+0000 7f5c32000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c3c156350 0x7f5c3c0b9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:57.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.614+0000 7f5c32000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c3c156350 0x7f5c3c0b9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55632/0 (socket says 172.21.15.86:55632) 2024-04-20T21:20:57.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.614+0000 7f5c32000640 1 -- 172.21.15.86:0/3748999833 learned_addr learned my addr 172.21.15.86:0/3748999833 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:57.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.614+0000 7f5c3aa00640 1 --2- 172.21.15.86:0/3748999833 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c200039e0 0x7f5c3c157180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:57.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.614+0000 7f5c32000640 1 -- 172.21.15.86:0/3748999833 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c200039e0 msgr2=0x7f5c3c157180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:57.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.614+0000 7f5c32000640 1 --2- 172.21.15.86:0/3748999833 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c200039e0 0x7f5c3c157180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:57.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.614+0000 7f5c32000640 1 -- 172.21.15.86:0/3748999833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c3c0ba0b0 msgr2=0x7f5c3c0cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:57.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.615+0000 7f5c3b400640 1 --2- 172.21.15.86:0/3748999833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c3c0ba0b0 0x7f5c3c0cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:57.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.615+0000 7f5c32000640 1 --2- 172.21.15.86:0/3748999833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c3c0ba0b0 0x7f5c3c0cc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:57.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.615+0000 7f5c32000640 1 -- 172.21.15.86:0/3748999833 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c3c15a080 con 0x7f5c3c156350 2024-04-20T21:20:57.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.615+0000 7f5c32000640 1 --2- 172.21.15.86:0/3748999833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c3c156350 0x7f5c3c0b9b70 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f5c2400d5a0 tx=0x7f5c2400da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:57.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.615+0000 7f5c33400640 1 -- 172.21.15.86:0/3748999833 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c24005480 con 0x7f5c3c156350 2024-04-20T21:20:57.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.615+0000 7f5c33400640 1 -- 172.21.15.86:0/3748999833 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c24017070 con 0x7f5c3c156350 2024-04-20T21:20:57.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.615+0000 7f5c33400640 1 -- 172.21.15.86:0/3748999833 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c24003810 con 0x7f5c3c156350 2024-04-20T21:20:57.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.615+0000 7f5c3be00640 1 -- 172.21.15.86:0/3748999833 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c3c0c9d20 con 0x7f5c3c156350 2024-04-20T21:20:57.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.615+0000 7f5c3be00640 1 -- 172.21.15.86:0/3748999833 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c3c0ca2e0 con 0x7f5c3c156350 2024-04-20T21:20:57.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.616+0000 7f5c3be00640 1 -- 172.21.15.86:0/3748999833 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c3c04fcf0 con 0x7f5c3c156350 2024-04-20T21:20:57.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.617+0000 7f5c33400640 1 -- 172.21.15.86:0/3748999833 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5c240039b0 con 0x7f5c3c156350 2024-04-20T21:20:57.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.618+0000 7f5c33400640 1 --2- 172.21.15.86:0/3748999833 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5c1806c920 0x7f5c1806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:57.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.618+0000 7f5c33400640 1 -- 172.21.15.86:0/3748999833 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f5c2408b610 con 0x7f5c3c156350 2024-04-20T21:20:57.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.619+0000 7f5c3aa00640 1 --2- 172.21.15.86:0/3748999833 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5c1806c920 0x7f5c1806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:57.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.619+0000 7f5c3aa00640 1 --2- 172.21.15.86:0/3748999833 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5c1806c920 0x7f5c1806ede0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c00ba00 tx=0x7f5c2c030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:57.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.623+0000 7f5c33400640 1 -- 172.21.15.86:0/3748999833 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5c24059d30 con 0x7f5c3c156350 2024-04-20T21:20:57.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.900+0000 7f5c3be00640 1 -- 172.21.15.86:0/3748999833 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) -- 0x7f5c3c0d29a0 con 0x7f5c3c156350 2024-04-20T21:20:57.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.942+0000 7f5c33400640 1 -- 172.21.15.86:0/3748999833 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v186) ==== 149+0+0 (secure 0 0 0) 0x7f5c2401ae40 con 0x7f5c3c156350 2024-04-20T21:20:57.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.946+0000 7f5c3be00640 1 -- 172.21.15.86:0/3748999833 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5c1806c920 msgr2=0x7f5c1806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:57.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.946+0000 7f5c3be00640 1 --2- 172.21.15.86:0/3748999833 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5c1806c920 0x7f5c1806ede0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c00ba00 tx=0x7f5c2c030000 comp rx=0 tx=0).stop 2024-04-20T21:20:57.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.946+0000 7f5c3be00640 1 -- 172.21.15.86:0/3748999833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c3c156350 msgr2=0x7f5c3c0b9b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:57.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.946+0000 7f5c3be00640 1 --2- 172.21.15.86:0/3748999833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c3c156350 0x7f5c3c0b9b70 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f5c2400d5a0 tx=0x7f5c2400da60 comp rx=0 tx=0).stop 2024-04-20T21:20:57.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.946+0000 7f5c3be00640 1 -- 172.21.15.86:0/3748999833 shutdown_connections 2024-04-20T21:20:57.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.946+0000 7f5c3be00640 1 --2- 172.21.15.86:0/3748999833 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5c1806c920 0x7f5c1806ede0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:57.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.946+0000 7f5c3be00640 1 --2- 172.21.15.86:0/3748999833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c3c0ba0b0 0x7f5c3c0cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:57.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.946+0000 7f5c3be00640 1 --2- 172.21.15.86:0/3748999833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c3c156350 0x7f5c3c0b9b70 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:57.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.946+0000 7f5c3be00640 1 --2- 172.21.15.86:0/3748999833 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c200039e0 0x7f5c3c157180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:57.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.946+0000 7f5c3be00640 1 -- 172.21.15.86:0/3748999833 >> 172.21.15.86:0/3748999833 conn(0x7f5c3c08bf70 msgr2=0x7f5c3c06dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:57.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.946+0000 7f5c3be00640 1 -- 172.21.15.86:0/3748999833 shutdown_connections 2024-04-20T21:20:57.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:57.946+0000 7f5c3be00640 1 -- 172.21.15.86:0/3748999833 wait complete. 2024-04-20T21:20:57.964 INFO:tasks.cephfs_test_runner:That the MDS can repair an inodes backtrace in the data pool ... ok 2024-04-20T21:20:57.964 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-04-20T21:20:57.964 INFO:tasks.cephfs_test_runner:test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-20T21:20:57.965 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:20:58.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.147+0000 7f844be00640 1 Processor -- start 2024-04-20T21:20:58.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.147+0000 7f844be00640 1 -- start start 2024-04-20T21:20:58.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.149+0000 7f844be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f844c15ece0 0x7f844c15f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:58.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.149+0000 7f844be00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f844c0d50c0 0x7f844c0d3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:58.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.149+0000 7f844be00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f844c15f600 con 0x7f844c0cf9c0 2024-04-20T21:20:58.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.149+0000 7f844be00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f844c0d3950 con 0x7f844c15ece0 2024-04-20T21:20:58.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.149+0000 7f844be00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f844c0cec00 con 0x7f844c0d50c0 2024-04-20T21:20:58.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.149+0000 7f844aa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f844c15ece0 0x7f844c15f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:58.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.150+0000 7f844aa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f844c15ece0 0x7f844c15f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55640/0 (socket says 172.21.15.86:55640) 2024-04-20T21:20:58.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.150+0000 7f844aa00640 1 -- 172.21.15.86:0/4090710546 learned_addr learned my addr 172.21.15.86:0/4090710546 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:58.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.150+0000 7f844aa00640 1 -- 172.21.15.86:0/4090710546 >> v2:172.21.15.144:3300/0 conn(0x7f844c0d50c0 msgr2=0x7f844c0d3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:58.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.150+0000 7f8442000640 1 --2- 172.21.15.86:0/4090710546 >> v2:172.21.15.144:3300/0 conn(0x7f844c0d50c0 0x7f844c0d3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:58.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.150+0000 7f844aa00640 1 --2- 172.21.15.86:0/4090710546 >> v2:172.21.15.144:3300/0 conn(0x7f844c0d50c0 0x7f844c0d3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:58.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.150+0000 7f844aa00640 1 -- 172.21.15.86:0/4090710546 >> v1:172.21.15.144:6789/0 conn(0x7f844c0cf9c0 legacy=0x7f844c160000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:58.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.150+0000 7f844aa00640 1 -- 172.21.15.86:0/4090710546 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f844c0ceee0 con 0x7f844c15ece0 2024-04-20T21:20:58.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.150+0000 7f8442000640 1 --2- 172.21.15.86:0/4090710546 >> v2:172.21.15.144:3300/0 conn(0x7f844c0d50c0 0x7f844c0d3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:20:58.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.151+0000 7f844aa00640 1 --2- 172.21.15.86:0/4090710546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f844c15ece0 0x7f844c15f0c0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f844c0ceac0 tx=0x7f843c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65a2484bebe97a77 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:58.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.151+0000 7f844a000640 1 -- 172.21.15.86:0/4090710546 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c02e070 con 0x7f844c15ece0 2024-04-20T21:20:58.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.151+0000 7f844a000640 1 -- 172.21.15.86:0/4090710546 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f843c037070 con 0x7f844c15ece0 2024-04-20T21:20:58.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.151+0000 7f844a000640 1 -- 172.21.15.86:0/4090710546 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c031a00 con 0x7f844c15ece0 2024-04-20T21:20:58.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.152+0000 7f844be00640 1 -- 172.21.15.86:0/4090710546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f844c15ece0 msgr2=0x7f844c15f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:58.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.152+0000 7f844be00640 1 --2- 172.21.15.86:0/4090710546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f844c15ece0 0x7f844c15f0c0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f844c0ceac0 tx=0x7f843c02b880 comp rx=0 tx=0).stop 2024-04-20T21:20:58.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.152+0000 7f844be00640 1 -- 172.21.15.86:0/4090710546 shutdown_connections 2024-04-20T21:20:58.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.152+0000 7f844be00640 1 --2- 172.21.15.86:0/4090710546 >> v2:172.21.15.144:3300/0 conn(0x7f844c0d50c0 0x7f844c0d3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:58.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.152+0000 7f844be00640 1 --2- 172.21.15.86:0/4090710546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f844c15ece0 0x7f844c15f0c0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:58.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.152+0000 7f844be00640 1 -- 172.21.15.86:0/4090710546 >> 172.21.15.86:0/4090710546 conn(0x7f844c08bf70 msgr2=0x7f844c0d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:58.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.152+0000 7f844be00640 1 -- 172.21.15.86:0/4090710546 shutdown_connections 2024-04-20T21:20:58.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.152+0000 7f844be00640 1 -- 172.21.15.86:0/4090710546 wait complete. 2024-04-20T21:20:58.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.154+0000 7f844be00640 1 Processor -- start 2024-04-20T21:20:58.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.154+0000 7f844be00640 1 -- start start 2024-04-20T21:20:58.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.155+0000 7f844be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f844c0cf9c0 0x7f844c0cda20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:58.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.155+0000 7f844be00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f844c0d50c0 0x7f844c1e8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:58.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.155+0000 7f844be00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f844c15ece0 0x7f844c0b9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:58.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.155+0000 7f844be00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f844c0d3640 con 0x7f844c0cf9c0 2024-04-20T21:20:58.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.155+0000 7f844be00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f844c0cf4a0 con 0x7f844c0d50c0 2024-04-20T21:20:58.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.155+0000 7f844be00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f844c15ea50 con 0x7f844c15ece0 2024-04-20T21:20:58.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.155+0000 7f844aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f844c0cf9c0 0x7f844c0cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:58.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.155+0000 7f844aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f844c0cf9c0 0x7f844c0cda20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55648/0 (socket says 172.21.15.86:55648) 2024-04-20T21:20:58.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.155+0000 7f844aa00640 1 -- 172.21.15.86:0/1156367940 learned_addr learned my addr 172.21.15.86:0/1156367940 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:58.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.155+0000 7f844aa00640 1 -- 172.21.15.86:0/1156367940 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f844c15ece0 msgr2=0x7f844c0b9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:58.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.155+0000 7f844aa00640 1 --2- 172.21.15.86:0/1156367940 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f844c15ece0 0x7f844c0b9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:58.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.155+0000 7f844aa00640 1 -- 172.21.15.86:0/1156367940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f844c0d50c0 msgr2=0x7f844c1e8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:58.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.155+0000 7f844aa00640 1 --2- 172.21.15.86:0/1156367940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f844c0d50c0 0x7f844c1e8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:58.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.155+0000 7f844aa00640 1 -- 172.21.15.86:0/1156367940 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f844c0b7160 con 0x7f844c0cf9c0 2024-04-20T21:20:58.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.156+0000 7f844aa00640 1 --2- 172.21.15.86:0/1156367940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f844c0cf9c0 0x7f844c0cda20 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f843c02b8c0 tx=0x7f843c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:58.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.156+0000 7f8443400640 1 -- 172.21.15.86:0/1156367940 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c032880 con 0x7f844c0cf9c0 2024-04-20T21:20:58.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.156+0000 7f8443400640 1 -- 172.21.15.86:0/1156367940 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f843c032a40 con 0x7f844c0cf9c0 2024-04-20T21:20:58.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.156+0000 7f844be00640 1 -- 172.21.15.86:0/1156367940 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f844c0b7440 con 0x7f844c0cf9c0 2024-04-20T21:20:58.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.156+0000 7f8443400640 1 -- 172.21.15.86:0/1156367940 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c032e30 con 0x7f844c0cf9c0 2024-04-20T21:20:58.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.156+0000 7f844be00640 1 -- 172.21.15.86:0/1156367940 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f844c1e8060 con 0x7f844c0cf9c0 2024-04-20T21:20:58.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.157+0000 7f844be00640 1 -- 172.21.15.86:0/1156367940 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f844c04fcf0 con 0x7f844c0cf9c0 2024-04-20T21:20:58.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.158+0000 7f8443400640 1 -- 172.21.15.86:0/1156367940 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f843c04c060 con 0x7f844c0cf9c0 2024-04-20T21:20:58.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.159+0000 7f8443400640 1 --2- 172.21.15.86:0/1156367940 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f842806c8f0 0x7f842806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:58.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.159+0000 7f8443400640 1 -- 172.21.15.86:0/1156367940 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f843c0ad330 con 0x7f844c0cf9c0 2024-04-20T21:20:58.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.159+0000 7f8442000640 1 --2- 172.21.15.86:0/1156367940 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f842806c8f0 0x7f842806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:58.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.160+0000 7f8442000640 1 --2- 172.21.15.86:0/1156367940 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f842806c8f0 0x7f842806edb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8434006fd0 tx=0x7f8434008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:58.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.164+0000 7f8443400640 1 -- 172.21.15.86:0/1156367940 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f843c040070 con 0x7f844c0cf9c0 2024-04-20T21:20:58.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.362+0000 7f844be00640 1 -- 172.21.15.86:0/1156367940 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f844c06e2a0 con 0x7f844c0cf9c0 2024-04-20T21:20:58.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.364+0000 7f8443400640 1 -- 172.21.15.86:0/1156367940 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) ==== 74+0+13018 (secure 0 0 0) 0x7f843c07b370 con 0x7f844c0cf9c0 2024-04-20T21:20:58.365 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:20:58.365 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":50,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:20:56.931068+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6837/712628984":"2024-04-21T21:20:53.910445+0000","172.21.15.86:6836/712628984":"2024-04-21T21:20:53.910445+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:20:58.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.367+0000 7f844be00640 1 -- 172.21.15.86:0/1156367940 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f842806c8f0 msgr2=0x7f842806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:58.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.367+0000 7f844be00640 1 --2- 172.21.15.86:0/1156367940 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f842806c8f0 0x7f842806edb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8434006fd0 tx=0x7f8434008040 comp rx=0 tx=0).stop 2024-04-20T21:20:58.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.367+0000 7f844be00640 1 -- 172.21.15.86:0/1156367940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f844c0cf9c0 msgr2=0x7f844c0cda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:58.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.367+0000 7f844be00640 1 --2- 172.21.15.86:0/1156367940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f844c0cf9c0 0x7f844c0cda20 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f843c02b8c0 tx=0x7f843c0046c0 comp rx=0 tx=0).stop 2024-04-20T21:20:58.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.367+0000 7f844be00640 1 -- 172.21.15.86:0/1156367940 shutdown_connections 2024-04-20T21:20:58.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.367+0000 7f844be00640 1 --2- 172.21.15.86:0/1156367940 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f842806c8f0 0x7f842806edb0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:58.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.367+0000 7f844be00640 1 --2- 172.21.15.86:0/1156367940 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f844c15ece0 0x7f844c0b9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:58.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.367+0000 7f844be00640 1 --2- 172.21.15.86:0/1156367940 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f844c0d50c0 0x7f844c1e8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:58.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.367+0000 7f844be00640 1 --2- 172.21.15.86:0/1156367940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f844c0cf9c0 0x7f844c0cda20 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:58.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.367+0000 7f844be00640 1 -- 172.21.15.86:0/1156367940 >> 172.21.15.86:0/1156367940 conn(0x7f844c08bf70 msgr2=0x7f844c1621b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:58.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.368+0000 7f844be00640 1 -- 172.21.15.86:0/1156367940 shutdown_connections 2024-04-20T21:20:58.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.368+0000 7f844be00640 1 -- 172.21.15.86:0/1156367940 wait complete. 2024-04-20T21:20:58.387 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:20:58.387 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:20:58.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.588+0000 7f7faf200640 1 Processor -- start 2024-04-20T21:20:58.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.588+0000 7f7faf200640 1 -- start start 2024-04-20T21:20:58.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.590+0000 7f7faf200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7fa815b4b0 0x7f7fa80cc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:58.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.590+0000 7f7faf200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7fa815ef50 0x7f7fa815f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:58.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.590+0000 7f7faf200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fa80729f0 con 0x7f7fa815b890 2024-04-20T21:20:58.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.590+0000 7f7faf200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f7fa815fe10 con 0x7f7fa815b4b0 2024-04-20T21:20:58.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.590+0000 7f7faf200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f7fa815f900 con 0x7f7fa815ef50 2024-04-20T21:20:58.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.590+0000 7f7fade00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7fa815b4b0 0x7f7fa80cc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:58.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.590+0000 7f7fae800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f7fa815b890 0x7f7fa815bcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54190/0 (socket says 172.21.15.86:54190) 2024-04-20T21:20:58.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.590+0000 7f7fae800640 1 -- 172.21.15.86:0/3978905463 learned_addr learned my addr 172.21.15.86:0/3978905463 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:20:58.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.590+0000 7f7fad400640 1 --2- 172.21.15.86:0/3978905463 >> v2:172.21.15.144:3300/0 conn(0x7f7fa815ef50 0x7f7fa815f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:58.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.590+0000 7f7fade00640 1 -- 172.21.15.86:0/3978905463 >> v2:172.21.15.144:3300/0 conn(0x7f7fa815ef50 msgr2=0x7f7fa815f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:58.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.591+0000 7f7fade00640 1 --2- 172.21.15.86:0/3978905463 >> v2:172.21.15.144:3300/0 conn(0x7f7fa815ef50 0x7f7fa815f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:58.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.591+0000 7f7fade00640 1 -- 172.21.15.86:0/3978905463 >> v1:172.21.15.86:6789/0 conn(0x7f7fa815b890 legacy=0x7f7fa815bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:58.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.591+0000 7f7faca00640 1 -- 172.21.15.86:0/3978905463 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1066582437 0 0) 0x7f7fa80729f0 con 0x7f7fa815b890 2024-04-20T21:20:58.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.591+0000 7f7fade00640 1 -- 172.21.15.86:0/3978905463 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fa8072cd0 con 0x7f7fa815b4b0 2024-04-20T21:20:58.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.591+0000 7f7fad400640 1 --2- 172.21.15.86:0/3978905463 >> v2:172.21.15.144:3300/0 conn(0x7f7fa815ef50 0x7f7fa815f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:20:58.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.592+0000 7f7fade00640 1 --2- 172.21.15.86:0/3978905463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fa815b4b0 0x7f7fa80cc3f0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f7fa806fc30 tx=0x7f7f9c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53d5a7c5109daa89 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:58.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.592+0000 7f7faca00640 1 -- 172.21.15.86:0/3978905463 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c035020 con 0x7f7fa815b4b0 2024-04-20T21:20:58.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.592+0000 7f7faca00640 1 -- 172.21.15.86:0/3978905463 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f9c031830 con 0x7f7fa815b4b0 2024-04-20T21:20:58.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.592+0000 7f7faca00640 1 -- 172.21.15.86:0/3978905463 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c031c00 con 0x7f7fa815b4b0 2024-04-20T21:20:58.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.592+0000 7f7faf200640 1 -- 172.21.15.86:0/3978905463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fa815b4b0 msgr2=0x7f7fa80cc3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:58.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.593+0000 7f7faf200640 1 --2- 172.21.15.86:0/3978905463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fa815b4b0 0x7f7fa80cc3f0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f7fa806fc30 tx=0x7f7f9c02b8d0 comp rx=0 tx=0).stop 2024-04-20T21:20:58.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.593+0000 7f7faf200640 1 -- 172.21.15.86:0/3978905463 shutdown_connections 2024-04-20T21:20:58.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.593+0000 7f7faf200640 1 --2- 172.21.15.86:0/3978905463 >> v2:172.21.15.144:3300/0 conn(0x7f7fa815ef50 0x7f7fa815f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:58.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.593+0000 7f7faf200640 1 --2- 172.21.15.86:0/3978905463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fa815b4b0 0x7f7fa80cc3f0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:58.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.593+0000 7f7faf200640 1 -- 172.21.15.86:0/3978905463 >> 172.21.15.86:0/3978905463 conn(0x7f7fa80b1f00 msgr2=0x7f7fa80b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:58.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.593+0000 7f7faf200640 1 -- 172.21.15.86:0/3978905463 shutdown_connections 2024-04-20T21:20:58.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.593+0000 7f7faf200640 1 -- 172.21.15.86:0/3978905463 wait complete. 2024-04-20T21:20:58.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.594+0000 7f7faf200640 1 Processor -- start 2024-04-20T21:20:58.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.595+0000 7f7faf200640 1 -- start start 2024-04-20T21:20:58.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.595+0000 7f7faf200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7fa815b4b0 0x7f7fa814d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:58.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.595+0000 7f7faf200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fa815b890 0x7f7fa81f2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:58.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.595+0000 7f7faf200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fa815ef50 0x7f7fa8158ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:58.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.595+0000 7f7faf200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa8073570 con 0x7f7fa815ef50 2024-04-20T21:20:58.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.595+0000 7f7faf200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa8073290 con 0x7f7fa815b890 2024-04-20T21:20:58.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.595+0000 7f7faf200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7fa8072650 con 0x7f7fa815b4b0 2024-04-20T21:20:58.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.596+0000 7f7fae800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fa815ef50 0x7f7fa8158ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:58.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.596+0000 7f7fae800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fa815ef50 0x7f7fa8158ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55674/0 (socket says 172.21.15.86:55674) 2024-04-20T21:20:58.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.596+0000 7f7fae800640 1 -- 172.21.15.86:0/90268006 learned_addr learned my addr 172.21.15.86:0/90268006 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:58.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.596+0000 7f7fade00640 1 --2- 172.21.15.86:0/90268006 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7fa815b4b0 0x7f7fa814d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:58.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.596+0000 7f7fad400640 1 --2- 172.21.15.86:0/90268006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fa815b890 0x7f7fa81f2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:58.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.596+0000 7f7fae800640 1 -- 172.21.15.86:0/90268006 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7fa815b4b0 msgr2=0x7f7fa814d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:58.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.596+0000 7f7fae800640 1 --2- 172.21.15.86:0/90268006 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7fa815b4b0 0x7f7fa814d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:58.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.596+0000 7f7fae800640 1 -- 172.21.15.86:0/90268006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fa815b890 msgr2=0x7f7fa81f2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:58.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.596+0000 7f7fae800640 1 --2- 172.21.15.86:0/90268006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fa815b890 0x7f7fa81f2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:58.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.596+0000 7f7fae800640 1 -- 172.21.15.86:0/90268006 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fa814d7c0 con 0x7f7fa815ef50 2024-04-20T21:20:58.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.596+0000 7f7fad400640 1 --2- 172.21.15.86:0/90268006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fa815b890 0x7f7fa81f2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:20:58.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.597+0000 7f7fae800640 1 --2- 172.21.15.86:0/90268006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fa815ef50 0x7f7fa8158ee0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f7fa400c7d0 tx=0x7f7fa400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:58.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.597+0000 7f7fa2000640 1 -- 172.21.15.86:0/90268006 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa4018070 con 0x7f7fa815ef50 2024-04-20T21:20:58.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.597+0000 7f7fa2000640 1 -- 172.21.15.86:0/90268006 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fa400a640 con 0x7f7fa815ef50 2024-04-20T21:20:58.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.597+0000 7f7fa2000640 1 -- 172.21.15.86:0/90268006 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa400aa70 con 0x7f7fa815ef50 2024-04-20T21:20:58.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.597+0000 7f7faf200640 1 -- 172.21.15.86:0/90268006 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fa814c1e0 con 0x7f7fa815ef50 2024-04-20T21:20:58.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.597+0000 7f7faf200640 1 -- 172.21.15.86:0/90268006 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fa814c7a0 con 0x7f7fa815ef50 2024-04-20T21:20:58.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.598+0000 7f7faf200640 1 -- 172.21.15.86:0/90268006 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fa80681f0 con 0x7f7fa815ef50 2024-04-20T21:20:58.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.599+0000 7f7fa2000640 1 -- 172.21.15.86:0/90268006 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7fa400ac10 con 0x7f7fa815ef50 2024-04-20T21:20:58.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.601+0000 7f7fa2000640 1 --2- 172.21.15.86:0/90268006 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7f8806c8f0 0x7f7f8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:58.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.601+0000 7f7fa2000640 1 -- 172.21.15.86:0/90268006 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f7fa408aeb0 con 0x7f7fa815ef50 2024-04-20T21:20:58.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.601+0000 7f7fade00640 1 --2- 172.21.15.86:0/90268006 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7f8806c8f0 0x7f7f8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:58.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.602+0000 7f7fade00640 1 --2- 172.21.15.86:0/90268006 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7f8806c8f0 0x7f7f8806edb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c036000 tx=0x7f7f9c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:58.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.605+0000 7f7fa2000640 1 -- 172.21.15.86:0/90268006 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7fa80681f0 con 0x7f7fa815ef50 2024-04-20T21:20:58.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.805+0000 7f7faf200640 1 -- 172.21.15.86:0/90268006 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7fa815fbf0 con 0x7f7fa815ef50 2024-04-20T21:20:58.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.806+0000 7f7fa2000640 1 -- 172.21.15.86:0/90268006 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) ==== 93+0+10 (secure 0 0 0) 0x7f7fa40595d0 con 0x7f7fa815ef50 2024-04-20T21:20:58.806 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:20:58.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.809+0000 7f7faf200640 1 -- 172.21.15.86:0/90268006 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7f8806c8f0 msgr2=0x7f7f8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:58.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.809+0000 7f7faf200640 1 --2- 172.21.15.86:0/90268006 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7f8806c8f0 0x7f7f8806edb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c036000 tx=0x7f7f9c008030 comp rx=0 tx=0).stop 2024-04-20T21:20:58.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.809+0000 7f7faf200640 1 -- 172.21.15.86:0/90268006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fa815ef50 msgr2=0x7f7fa8158ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:58.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.809+0000 7f7faf200640 1 --2- 172.21.15.86:0/90268006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fa815ef50 0x7f7fa8158ee0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f7fa400c7d0 tx=0x7f7fa400cc90 comp rx=0 tx=0).stop 2024-04-20T21:20:58.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.809+0000 7f7faf200640 1 -- 172.21.15.86:0/90268006 shutdown_connections 2024-04-20T21:20:58.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.809+0000 7f7faf200640 1 --2- 172.21.15.86:0/90268006 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7f8806c8f0 0x7f7f8806edb0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:58.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.809+0000 7f7faf200640 1 --2- 172.21.15.86:0/90268006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fa815ef50 0x7f7fa8158ee0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:58.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.809+0000 7f7faf200640 1 --2- 172.21.15.86:0/90268006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fa815b890 0x7f7fa81f2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:58.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.809+0000 7f7faf200640 1 --2- 172.21.15.86:0/90268006 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7fa815b4b0 0x7f7fa814d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:58.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.809+0000 7f7faf200640 1 -- 172.21.15.86:0/90268006 >> 172.21.15.86:0/90268006 conn(0x7f7fa80b1f00 msgr2=0x7f7fa815de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:58.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.809+0000 7f7faf200640 1 -- 172.21.15.86:0/90268006 shutdown_connections 2024-04-20T21:20:58.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:58.810+0000 7f7faf200640 1 -- 172.21.15.86:0/90268006 wait complete. 2024-04-20T21:20:58.829 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-04-20T21:20:59.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.018+0000 7f15b6a00640 1 Processor -- start 2024-04-20T21:20:59.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.018+0000 7f15b6a00640 1 -- start start 2024-04-20T21:20:59.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.020+0000 7f15b6a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f15b006e6d0 0x7f15b0071d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:59.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.020+0000 7f15b6a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f15b0072240 0x7f15b015cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:59.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.020+0000 7f15b6a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15b006eb30 con 0x7f15b0146130 2024-04-20T21:20:59.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.020+0000 7f15b6a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f15b0072690 con 0x7f15b006e6d0 2024-04-20T21:20:59.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.020+0000 7f15b6a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f15b006dc30 con 0x7f15b0072240 2024-04-20T21:20:59.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.020+0000 7f15a7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f15b006e6d0 0x7f15b0071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:59.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.020+0000 7f15a7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f15b006e6d0 0x7f15b0071d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55678/0 (socket says 172.21.15.86:55678) 2024-04-20T21:20:59.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.020+0000 7f15a7e00640 1 -- 172.21.15.86:0/180098063 learned_addr learned my addr 172.21.15.86:0/180098063 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:59.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.021+0000 7f15b4a00640 1 --2- 172.21.15.86:0/180098063 >> v2:172.21.15.144:3300/0 conn(0x7f15b0072240 0x7f15b015cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:59.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.021+0000 7f15a7e00640 1 -- 172.21.15.86:0/180098063 >> v2:172.21.15.144:3300/0 conn(0x7f15b0072240 msgr2=0x7f15b015cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:59.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.021+0000 7f15a7e00640 1 --2- 172.21.15.86:0/180098063 >> v2:172.21.15.144:3300/0 conn(0x7f15b0072240 0x7f15b015cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:59.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.021+0000 7f15a7e00640 1 -- 172.21.15.86:0/180098063 >> v1:172.21.15.144:6789/0 conn(0x7f15b0146130 legacy=0x7f15b00ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:20:59.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.021+0000 7f15a7e00640 1 -- 172.21.15.86:0/180098063 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b00cd8e0 con 0x7f15b006e6d0 2024-04-20T21:20:59.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.022+0000 7f15a7e00640 1 --2- 172.21.15.86:0/180098063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15b006e6d0 0x7f15b0071d00 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f15b0071c90 tx=0x7f159802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e468be3d9fccb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:59.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.022+0000 7f15af400640 1 -- 172.21.15.86:0/180098063 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1598034070 con 0x7f15b006e6d0 2024-04-20T21:20:59.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.022+0000 7f15af400640 1 -- 172.21.15.86:0/180098063 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1598030860 con 0x7f15b006e6d0 2024-04-20T21:20:59.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.022+0000 7f15af400640 1 -- 172.21.15.86:0/180098063 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1598030c30 con 0x7f15b006e6d0 2024-04-20T21:20:59.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.023+0000 7f15b6a00640 1 -- 172.21.15.86:0/180098063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15b006e6d0 msgr2=0x7f15b0071d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:59.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.023+0000 7f15b6a00640 1 --2- 172.21.15.86:0/180098063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15b006e6d0 0x7f15b0071d00 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f15b0071c90 tx=0x7f159802b9a0 comp rx=0 tx=0).stop 2024-04-20T21:20:59.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.023+0000 7f15b6a00640 1 -- 172.21.15.86:0/180098063 shutdown_connections 2024-04-20T21:20:59.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.023+0000 7f15b6a00640 1 --2- 172.21.15.86:0/180098063 >> v2:172.21.15.144:3300/0 conn(0x7f15b0072240 0x7f15b015cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:59.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.023+0000 7f15b6a00640 1 --2- 172.21.15.86:0/180098063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15b006e6d0 0x7f15b0071d00 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:59.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.023+0000 7f15b6a00640 1 -- 172.21.15.86:0/180098063 >> 172.21.15.86:0/180098063 conn(0x7f15b00b2360 msgr2=0x7f15b00b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:59.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.023+0000 7f15b6a00640 1 -- 172.21.15.86:0/180098063 shutdown_connections 2024-04-20T21:20:59.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.023+0000 7f15b6a00640 1 -- 172.21.15.86:0/180098063 wait complete. 2024-04-20T21:20:59.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.025+0000 7f15b6a00640 1 Processor -- start 2024-04-20T21:20:59.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.025+0000 7f15b6a00640 1 -- start start 2024-04-20T21:20:59.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.026+0000 7f15b6a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15b006e6d0 0x7f15b00ab900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:59.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.026+0000 7f15b6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15b0072240 0x7f15b015f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:59.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.026+0000 7f15b6a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15b0146130 0x7f15b015bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:59.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.026+0000 7f15b6a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f15b006e1b0 con 0x7f15b0072240 2024-04-20T21:20:59.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.026+0000 7f15b6a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f15b006ded0 con 0x7f15b0146130 2024-04-20T21:20:59.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.026+0000 7f15b6a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f15b006f150 con 0x7f15b006e6d0 2024-04-20T21:20:59.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.026+0000 7f15a7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15b0072240 0x7f15b015f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:59.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.026+0000 7f15a7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15b0072240 0x7f15b015f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55694/0 (socket says 172.21.15.86:55694) 2024-04-20T21:20:59.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.026+0000 7f15a7e00640 1 -- 172.21.15.86:0/877474172 learned_addr learned my addr 172.21.15.86:0/877474172 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:20:59.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.026+0000 7f15a7e00640 1 -- 172.21.15.86:0/877474172 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15b006e6d0 msgr2=0x7f15b00ab900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:59.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.026+0000 7f15a7e00640 1 --2- 172.21.15.86:0/877474172 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15b006e6d0 0x7f15b00ab900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:59.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.026+0000 7f15a7e00640 1 -- 172.21.15.86:0/877474172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15b0146130 msgr2=0x7f15b015bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:59.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.026+0000 7f15a7e00640 1 --2- 172.21.15.86:0/877474172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15b0146130 0x7f15b015bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:59.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.026+0000 7f15a7e00640 1 -- 172.21.15.86:0/877474172 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b00aa160 con 0x7f15b0072240 2024-04-20T21:20:59.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.027+0000 7f15a7e00640 1 --2- 172.21.15.86:0/877474172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15b0072240 0x7f15b015f1c0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f1598000c00 tx=0x7f15980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:59.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.027+0000 7f15acc00640 1 -- 172.21.15.86:0/877474172 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1598034040 con 0x7f15b0072240 2024-04-20T21:20:59.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.027+0000 7f15acc00640 1 -- 172.21.15.86:0/877474172 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15980433f0 con 0x7f15b0072240 2024-04-20T21:20:59.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.027+0000 7f15acc00640 1 -- 172.21.15.86:0/877474172 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15980437e0 con 0x7f15b0072240 2024-04-20T21:20:59.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.027+0000 7f15b6a00640 1 -- 172.21.15.86:0/877474172 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15b01f4070 con 0x7f15b0072240 2024-04-20T21:20:59.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.027+0000 7f15b6a00640 1 -- 172.21.15.86:0/877474172 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15b01f4630 con 0x7f15b0072240 2024-04-20T21:20:59.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.028+0000 7f15b6a00640 1 -- 172.21.15.86:0/877474172 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15b01443d0 con 0x7f15b0072240 2024-04-20T21:20:59.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.029+0000 7f15acc00640 1 -- 172.21.15.86:0/877474172 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f159804a060 con 0x7f15b0072240 2024-04-20T21:20:59.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.030+0000 7f15acc00640 1 --2- 172.21.15.86:0/877474172 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f158c06c8f0 0x7f158c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:20:59.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.030+0000 7f15acc00640 1 -- 172.21.15.86:0/877474172 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f15980adc50 con 0x7f15b0072240 2024-04-20T21:20:59.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.030+0000 7f15afe00640 1 --2- 172.21.15.86:0/877474172 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f158c06c8f0 0x7f158c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:20:59.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.031+0000 7f15afe00640 1 --2- 172.21.15.86:0/877474172 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f158c06c8f0 0x7f158c06edb0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f15b013f730 tx=0x7f15a0007990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:20:59.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.034+0000 7f15acc00640 1 -- 172.21.15.86:0/877474172 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f15b01443d0 con 0x7f15b0072240 2024-04-20T21:20:59.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.303+0000 7f15b6a00640 1 -- 172.21.15.86:0/877474172 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) -- 0x7f15b00689e0 con 0x7f15b0072240 2024-04-20T21:20:59.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.955+0000 7f15acc00640 1 -- 172.21.15.86:0/877474172 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v188) ==== 172+0+0 (secure 0 0 0) 0x7f159803e070 con 0x7f15b0072240 2024-04-20T21:20:59.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.960+0000 7f15b6a00640 1 -- 172.21.15.86:0/877474172 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f158c06c8f0 msgr2=0x7f158c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:59.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.960+0000 7f15b6a00640 1 --2- 172.21.15.86:0/877474172 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f158c06c8f0 0x7f158c06edb0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f15b013f730 tx=0x7f15a0007990 comp rx=0 tx=0).stop 2024-04-20T21:20:59.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.960+0000 7f15b6a00640 1 -- 172.21.15.86:0/877474172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15b0072240 msgr2=0x7f15b015f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:20:59.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.960+0000 7f15b6a00640 1 --2- 172.21.15.86:0/877474172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15b0072240 0x7f15b015f1c0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f1598000c00 tx=0x7f15980046c0 comp rx=0 tx=0).stop 2024-04-20T21:20:59.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.960+0000 7f15b6a00640 1 -- 172.21.15.86:0/877474172 shutdown_connections 2024-04-20T21:20:59.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.960+0000 7f15b6a00640 1 --2- 172.21.15.86:0/877474172 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f158c06c8f0 0x7f158c06edb0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:59.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.960+0000 7f15b6a00640 1 --2- 172.21.15.86:0/877474172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f15b0146130 0x7f15b015bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:59.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.960+0000 7f15b6a00640 1 --2- 172.21.15.86:0/877474172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15b0072240 0x7f15b015f1c0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:59.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.960+0000 7f15b6a00640 1 --2- 172.21.15.86:0/877474172 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f15b006e6d0 0x7f15b00ab900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:20:59.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.960+0000 7f15b6a00640 1 -- 172.21.15.86:0/877474172 >> 172.21.15.86:0/877474172 conn(0x7f15b00b2360 msgr2=0x7f15b0143720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:20:59.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.960+0000 7f15b6a00640 1 -- 172.21.15.86:0/877474172 shutdown_connections 2024-04-20T21:20:59.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:20:59.960+0000 7f15b6a00640 1 -- 172.21.15.86:0/877474172 wait complete. 2024-04-20T21:20:59.980 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-20T21:21:00.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.181+0000 7f3d20c00640 1 Processor -- start 2024-04-20T21:21:00.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.181+0000 7f3d20c00640 1 -- start start 2024-04-20T21:21:00.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.183+0000 7f3d20c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d1c15ee00 0x7f3d1c1634c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:00.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.183+0000 7f3d20c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3d1c0cf830 0x7f3d1c15fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:00.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.183+0000 7f3d20c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3d1c0d1260 0x7f3d1c0d16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:00.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.183+0000 7f3d20c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f3d1c0d0550 con 0x7f3d1c0d1260 2024-04-20T21:21:00.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.183+0000 7f3d20c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f3d1c0cff90 con 0x7f3d1c0cf830 2024-04-20T21:21:00.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.183+0000 7f3d20c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3d1c0d0270 con 0x7f3d1c15ee00 2024-04-20T21:21:00.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.183+0000 7f3d1a800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3d1c0d1260 0x7f3d1c0d16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:00.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.183+0000 7f3d1a800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3d1c0d1260 0x7f3d1c0d16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55710/0 (socket says 172.21.15.86:55710) 2024-04-20T21:21:00.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.183+0000 7f3d1a800640 1 -- 172.21.15.86:0/2272021671 learned_addr learned my addr 172.21.15.86:0/2272021671 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:00.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.184+0000 7f3d13e00640 1 --2- 172.21.15.86:0/2272021671 >> v2:172.21.15.144:3300/0 conn(0x7f3d1c0cf830 0x7f3d1c15fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:00.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.184+0000 7f3d1a800640 1 -- 172.21.15.86:0/2272021671 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d1c15ee00 msgr2=0x7f3d1c1634c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:00.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.184+0000 7f3d1a800640 1 --2- 172.21.15.86:0/2272021671 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d1c15ee00 0x7f3d1c1634c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:00.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.184+0000 7f3d1a800640 1 -- 172.21.15.86:0/2272021671 >> v2:172.21.15.144:3300/0 conn(0x7f3d1c0cf830 msgr2=0x7f3d1c15fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:00.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.184+0000 7f3d1a800640 1 --2- 172.21.15.86:0/2272021671 >> v2:172.21.15.144:3300/0 conn(0x7f3d1c0cf830 0x7f3d1c15fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:00.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.184+0000 7f3d1a800640 1 -- 172.21.15.86:0/2272021671 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d1c15f640 con 0x7f3d1c0d1260 2024-04-20T21:21:00.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.185+0000 7f3d1a800640 1 --2- 172.21.15.86:0/2272021671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d1c0d1260 0x7f3d1c0d16f0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c08ccb0 tx=0x7f3d1402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c60b123a698a900b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:00.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.185+0000 7f3d19400640 1 -- 172.21.15.86:0/2272021671 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1402ca70 con 0x7f3d1c0d1260 2024-04-20T21:21:00.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.185+0000 7f3d19400640 1 -- 172.21.15.86:0/2272021671 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d1402cc10 con 0x7f3d1c0d1260 2024-04-20T21:21:00.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.185+0000 7f3d19400640 1 -- 172.21.15.86:0/2272021671 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d140343f0 con 0x7f3d1c0d1260 2024-04-20T21:21:00.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.186+0000 7f3d20c00640 1 -- 172.21.15.86:0/2272021671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d1c0d1260 msgr2=0x7f3d1c0d16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:00.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.186+0000 7f3d20c00640 1 --2- 172.21.15.86:0/2272021671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d1c0d1260 0x7f3d1c0d16f0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c08ccb0 tx=0x7f3d1402bed0 comp rx=0 tx=0).stop 2024-04-20T21:21:00.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.186+0000 7f3d20c00640 1 -- 172.21.15.86:0/2272021671 shutdown_connections 2024-04-20T21:21:00.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.186+0000 7f3d20c00640 1 --2- 172.21.15.86:0/2272021671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d1c0d1260 0x7f3d1c0d16f0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:00.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.186+0000 7f3d20c00640 1 --2- 172.21.15.86:0/2272021671 >> v2:172.21.15.144:3300/0 conn(0x7f3d1c0cf830 0x7f3d1c15fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:00.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.186+0000 7f3d20c00640 1 --2- 172.21.15.86:0/2272021671 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d1c15ee00 0x7f3d1c1634c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:00.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.186+0000 7f3d20c00640 1 -- 172.21.15.86:0/2272021671 >> 172.21.15.86:0/2272021671 conn(0x7f3d1c06cc20 msgr2=0x7f3d1c06d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:00.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.186+0000 7f3d20c00640 1 -- 172.21.15.86:0/2272021671 shutdown_connections 2024-04-20T21:21:00.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.186+0000 7f3d20c00640 1 -- 172.21.15.86:0/2272021671 wait complete. 2024-04-20T21:21:00.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.188+0000 7f3d20c00640 1 Processor -- start 2024-04-20T21:21:00.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.188+0000 7f3d20c00640 1 -- start start 2024-04-20T21:21:00.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.189+0000 7f3d20c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d1c0cf830 0x7f3d1c1584f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:00.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.189+0000 7f3d20c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d1c0d1260 0x7f3d1c0ca3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:00.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.189+0000 7f3d20c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d1c15ee00 0x7f3d1c149710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:00.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.189+0000 7f3d20c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1c15f1e0 con 0x7f3d1c0d1260 2024-04-20T21:21:00.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.189+0000 7f3d20c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1c160380 con 0x7f3d1c15ee00 2024-04-20T21:21:00.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.189+0000 7f3d20c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3d1c15f360 con 0x7f3d1c0cf830 2024-04-20T21:21:00.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.189+0000 7f3d13e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d1c0d1260 0x7f3d1c0ca3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:00.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.189+0000 7f3d13e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d1c0d1260 0x7f3d1c0ca3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55712/0 (socket says 172.21.15.86:55712) 2024-04-20T21:21:00.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.189+0000 7f3d13e00640 1 -- 172.21.15.86:0/416610881 learned_addr learned my addr 172.21.15.86:0/416610881 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:00.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.189+0000 7f3d19e00640 1 --2- 172.21.15.86:0/416610881 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d1c0cf830 0x7f3d1c1584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:00.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.189+0000 7f3d1a800640 1 --2- 172.21.15.86:0/416610881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d1c15ee00 0x7f3d1c149710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:00.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.190+0000 7f3d13e00640 1 -- 172.21.15.86:0/416610881 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d1c0cf830 msgr2=0x7f3d1c1584f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:00.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.190+0000 7f3d13e00640 1 --2- 172.21.15.86:0/416610881 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d1c0cf830 0x7f3d1c1584f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:00.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.190+0000 7f3d13e00640 1 -- 172.21.15.86:0/416610881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d1c15ee00 msgr2=0x7f3d1c149710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:00.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.190+0000 7f3d13e00640 1 --2- 172.21.15.86:0/416610881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d1c15ee00 0x7f3d1c149710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:00.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.190+0000 7f3d13e00640 1 -- 172.21.15.86:0/416610881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d1c158a30 con 0x7f3d1c0d1260 2024-04-20T21:21:00.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.190+0000 7f3d1a800640 1 --2- 172.21.15.86:0/416610881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d1c15ee00 0x7f3d1c149710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:21:00.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.190+0000 7f3d13e00640 1 --2- 172.21.15.86:0/416610881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d1c0d1260 0x7f3d1c0ca3d0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f3d0400c820 tx=0x7f3d0400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:00.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.190+0000 7f3d12000640 1 -- 172.21.15.86:0/416610881 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d04015070 con 0x7f3d1c0d1260 2024-04-20T21:21:00.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.190+0000 7f3d12000640 1 -- 172.21.15.86:0/416610881 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d04004020 con 0x7f3d1c0d1260 2024-04-20T21:21:00.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.190+0000 7f3d12000640 1 -- 172.21.15.86:0/416610881 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d040043f0 con 0x7f3d1c0d1260 2024-04-20T21:21:00.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.190+0000 7f3d20c00640 1 -- 172.21.15.86:0/416610881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d1c0ca910 con 0x7f3d1c0d1260 2024-04-20T21:21:00.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.190+0000 7f3d20c00640 1 -- 172.21.15.86:0/416610881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d1c0cbfe0 con 0x7f3d1c0d1260 2024-04-20T21:21:00.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.192+0000 7f3d20c00640 1 -- 172.21.15.86:0/416610881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d1c0689e0 con 0x7f3d1c0d1260 2024-04-20T21:21:00.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.192+0000 7f3d12000640 1 -- 172.21.15.86:0/416610881 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3d04007830 con 0x7f3d1c0d1260 2024-04-20T21:21:00.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.193+0000 7f3d12000640 1 --2- 172.21.15.86:0/416610881 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3cf806c8f0 0x7f3cf806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:00.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.194+0000 7f3d12000640 1 -- 172.21.15.86:0/416610881 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f3d0408b610 con 0x7f3d1c0d1260 2024-04-20T21:21:00.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.194+0000 7f3d19e00640 1 --2- 172.21.15.86:0/416610881 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3cf806c8f0 0x7f3cf806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:00.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.194+0000 7f3d19e00640 1 --2- 172.21.15.86:0/416610881 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3cf806c8f0 0x7f3cf806edb0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c009890 tx=0x7f3d0c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:00.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.198+0000 7f3d12000640 1 -- 172.21.15.86:0/416610881 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3d1c0ca910 con 0x7f3d1c0d1260 2024-04-20T21:21:00.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.398+0000 7f3d20c00640 1 -- 172.21.15.86:0/416610881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f3d1c0d29c0 con 0x7f3d1c0d1260 2024-04-20T21:21:00.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.404+0000 7f3d12000640 1 -- 172.21.15.86:0/416610881 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7f3d04059d30 con 0x7f3d1c0d1260 2024-04-20T21:21:00.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.406+0000 7f3d20c00640 1 -- 172.21.15.86:0/416610881 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3cf806c8f0 msgr2=0x7f3cf806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:00.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.406+0000 7f3d20c00640 1 --2- 172.21.15.86:0/416610881 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3cf806c8f0 0x7f3cf806edb0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c009890 tx=0x7f3d0c008040 comp rx=0 tx=0).stop 2024-04-20T21:21:00.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.406+0000 7f3d20c00640 1 -- 172.21.15.86:0/416610881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d1c0d1260 msgr2=0x7f3d1c0ca3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:00.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.406+0000 7f3d20c00640 1 --2- 172.21.15.86:0/416610881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d1c0d1260 0x7f3d1c0ca3d0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f3d0400c820 tx=0x7f3d0400cce0 comp rx=0 tx=0).stop 2024-04-20T21:21:00.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.406+0000 7f3d20c00640 1 -- 172.21.15.86:0/416610881 shutdown_connections 2024-04-20T21:21:00.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.406+0000 7f3d20c00640 1 --2- 172.21.15.86:0/416610881 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3cf806c8f0 0x7f3cf806edb0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:00.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.406+0000 7f3d20c00640 1 --2- 172.21.15.86:0/416610881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d1c15ee00 0x7f3d1c149710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:00.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.406+0000 7f3d20c00640 1 --2- 172.21.15.86:0/416610881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d1c0d1260 0x7f3d1c0ca3d0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:00.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.406+0000 7f3d20c00640 1 --2- 172.21.15.86:0/416610881 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3d1c0cf830 0x7f3d1c1584f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:00.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.406+0000 7f3d20c00640 1 -- 172.21.15.86:0/416610881 >> 172.21.15.86:0/416610881 conn(0x7f3d1c06cc20 msgr2=0x7f3d1c160830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:00.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.407+0000 7f3d20c00640 1 -- 172.21.15.86:0/416610881 shutdown_connections 2024-04-20T21:21:00.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:00.407+0000 7f3d20c00640 1 -- 172.21.15.86:0/416610881 wait complete. 2024-04-20T21:21:00.425 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-20T21:21:00.425 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T21:21:00.520 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T21:21:00.636 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-20T21:21:00.636 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:21:00.636 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:21:00.692 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052731,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248996k,nr_inodes=812249,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T21:21:00.692 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:21:00.692 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:21:00.749 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T21:21:00.749 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:21:00.749 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:21:00.807 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052731,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248996k,nr_inodes=812249,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T21:21:00.808 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:21:00.808 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:21:00.864 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T21:21:00.865 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-20T21:21:00.865 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:21:01.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.072+0000 7fd833200640 1 Processor -- start 2024-04-20T21:21:01.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.073+0000 7fd833200640 1 -- start start 2024-04-20T21:21:01.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.074+0000 7fd833200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd834161f30 0x7fd8340cf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:01.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.074+0000 7fd833200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd8340d3a80 0x7fd8340d3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:01.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.074+0000 7fd833200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd83406d000 con 0x7fd8340d0e50 2024-04-20T21:21:01.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.074+0000 7fd833200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd83406dc80 con 0x7fd8340d3a80 2024-04-20T21:21:01.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.074+0000 7fd833200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd83406cdb0 con 0x7fd834161f30 2024-04-20T21:21:01.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.074+0000 7fd831400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd8340d3a80 0x7fd8340d3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:01.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.074+0000 7fd831400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd8340d3a80 0x7fd8340d3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55720/0 (socket says 172.21.15.86:55720) 2024-04-20T21:21:01.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.074+0000 7fd831400640 1 -- 172.21.15.86:0/1925103607 learned_addr learned my addr 172.21.15.86:0/1925103607 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:01.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.075+0000 7fd831e00640 1 --2- 172.21.15.86:0/1925103607 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd834161f30 0x7fd8340cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:01.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.075+0000 7fd831400640 1 -- 172.21.15.86:0/1925103607 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd834161f30 msgr2=0x7fd8340cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:01.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.075+0000 7fd831400640 1 --2- 172.21.15.86:0/1925103607 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd834161f30 0x7fd8340cf000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:01.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.075+0000 7fd831400640 1 -- 172.21.15.86:0/1925103607 >> v1:172.21.15.144:6789/0 conn(0x7fd8340d0e50 legacy=0x7fd834162940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:01.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.075+0000 7fd831400640 1 -- 172.21.15.86:0/1925103607 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd83406d2e0 con 0x7fd8340d3a80 2024-04-20T21:21:01.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.075+0000 7fd831e00640 1 --2- 172.21.15.86:0/1925103607 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd834161f30 0x7fd8340cf000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:21:01.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.076+0000 7fd831400640 1 --2- 172.21.15.86:0/1925103607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8340d3a80 0x7fd8340d3ee0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fd824009e50 tx=0x7fd82402bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cd88d75e6fe0e6a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:01.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.076+0000 7fd830a00640 1 -- 172.21.15.86:0/1925103607 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82402c960 con 0x7fd8340d3a80 2024-04-20T21:21:01.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.076+0000 7fd830a00640 1 -- 172.21.15.86:0/1925103607 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd82402cb00 con 0x7fd8340d3a80 2024-04-20T21:21:01.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.076+0000 7fd830a00640 1 -- 172.21.15.86:0/1925103607 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8240343f0 con 0x7fd8340d3a80 2024-04-20T21:21:01.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.077+0000 7fd833200640 1 -- 172.21.15.86:0/1925103607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8340d3a80 msgr2=0x7fd8340d3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:01.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.077+0000 7fd833200640 1 --2- 172.21.15.86:0/1925103607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8340d3a80 0x7fd8340d3ee0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fd824009e50 tx=0x7fd82402bdc0 comp rx=0 tx=0).stop 2024-04-20T21:21:01.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.077+0000 7fd833200640 1 -- 172.21.15.86:0/1925103607 shutdown_connections 2024-04-20T21:21:01.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.077+0000 7fd833200640 1 --2- 172.21.15.86:0/1925103607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8340d3a80 0x7fd8340d3ee0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:01.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.077+0000 7fd833200640 1 --2- 172.21.15.86:0/1925103607 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd834161f30 0x7fd8340cf000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:01.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.077+0000 7fd833200640 1 -- 172.21.15.86:0/1925103607 >> 172.21.15.86:0/1925103607 conn(0x7fd83408bf70 msgr2=0x7fd8340d4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:01.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.077+0000 7fd833200640 1 -- 172.21.15.86:0/1925103607 shutdown_connections 2024-04-20T21:21:01.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.077+0000 7fd833200640 1 -- 172.21.15.86:0/1925103607 wait complete. 2024-04-20T21:21:01.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.079+0000 7fd833200640 1 Processor -- start 2024-04-20T21:21:01.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.079+0000 7fd833200640 1 -- start start 2024-04-20T21:21:01.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.080+0000 7fd833200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8340d0e50 0x7fd8341f3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:01.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.080+0000 7fd833200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd8340d3a80 0x7fd8341f3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:01.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.080+0000 7fd833200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd834161f30 0x7fd8341f7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:01.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.080+0000 7fd833200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd834162310 con 0x7fd834161f30 2024-04-20T21:21:01.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.080+0000 7fd833200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd83406d8a0 con 0x7fd8340d0e50 2024-04-20T21:21:01.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.080+0000 7fd833200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd8340cf540 con 0x7fd8340d3a80 2024-04-20T21:21:01.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.080+0000 7fd832800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd834161f30 0x7fd8341f7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:01.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.081+0000 7fd832800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd834161f30 0x7fd8341f7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55728/0 (socket says 172.21.15.86:55728) 2024-04-20T21:21:01.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.081+0000 7fd832800640 1 -- 172.21.15.86:0/4207435138 learned_addr learned my addr 172.21.15.86:0/4207435138 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:01.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.081+0000 7fd831400640 1 --2- 172.21.15.86:0/4207435138 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd8340d3a80 0x7fd8341f3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:01.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.081+0000 7fd831e00640 1 --2- 172.21.15.86:0/4207435138 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8340d0e50 0x7fd8341f3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:01.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.081+0000 7fd832800640 1 -- 172.21.15.86:0/4207435138 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd8340d3a80 msgr2=0x7fd8341f3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:01.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.081+0000 7fd832800640 1 --2- 172.21.15.86:0/4207435138 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd8340d3a80 0x7fd8341f3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:01.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.081+0000 7fd832800640 1 -- 172.21.15.86:0/4207435138 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8340d0e50 msgr2=0x7fd8341f3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:01.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.081+0000 7fd832800640 1 --2- 172.21.15.86:0/4207435138 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8340d0e50 0x7fd8341f3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:01.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.081+0000 7fd832800640 1 -- 172.21.15.86:0/4207435138 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8340b9640 con 0x7fd834161f30 2024-04-20T21:21:01.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.081+0000 7fd832800640 1 --2- 172.21.15.86:0/4207435138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd834161f30 0x7fd8341f7d20 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fd82c00b410 tx=0x7fd82c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:01.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.082+0000 7fd82a000640 1 -- 172.21.15.86:0/4207435138 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82c015030 con 0x7fd834161f30 2024-04-20T21:21:01.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.082+0000 7fd82a000640 1 -- 172.21.15.86:0/4207435138 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd82c004020 con 0x7fd834161f30 2024-04-20T21:21:01.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.082+0000 7fd833200640 1 -- 172.21.15.86:0/4207435138 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8340b9920 con 0x7fd834161f30 2024-04-20T21:21:01.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.082+0000 7fd82a000640 1 -- 172.21.15.86:0/4207435138 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82c004410 con 0x7fd834161f30 2024-04-20T21:21:01.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.082+0000 7fd833200640 1 -- 172.21.15.86:0/4207435138 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8340cbdf0 con 0x7fd834161f30 2024-04-20T21:21:01.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.083+0000 7fd833200640 1 -- 172.21.15.86:0/4207435138 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8340cb900 con 0x7fd834161f30 2024-04-20T21:21:01.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.084+0000 7fd82a000640 1 -- 172.21.15.86:0/4207435138 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd82c018070 con 0x7fd834161f30 2024-04-20T21:21:01.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.085+0000 7fd82a000640 1 --2- 172.21.15.86:0/4207435138 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd81006c8f0 0x7fd81006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:01.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.085+0000 7fd82a000640 1 -- 172.21.15.86:0/4207435138 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7fd82c08b440 con 0x7fd834161f30 2024-04-20T21:21:01.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.085+0000 7fd831e00640 1 --2- 172.21.15.86:0/4207435138 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd81006c8f0 0x7fd81006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:01.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.086+0000 7fd831e00640 1 --2- 172.21.15.86:0/4207435138 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd81006c8f0 0x7fd81006edb0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd81c00a990 tx=0x7fd81c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:01.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.089+0000 7fd82a000640 1 -- 172.21.15.86:0/4207435138 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd82c059ad0 con 0x7fd834161f30 2024-04-20T21:21:01.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.329+0000 7fd833200640 1 -- 172.21.15.86:0/4207435138 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd83404fcf0 con 0x7fd834161f30 2024-04-20T21:21:01.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.329+0000 7fd82a000640 1 -- 172.21.15.86:0/4207435138 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+3869 (secure 0 0 0) 0x7fd82c0593d0 con 0x7fd834161f30 2024-04-20T21:21:01.330 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:21:01.330 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":26,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5044,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/1013106052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":1013106052},{"type":"v1","addr":"172.21.15.144:6833","nonce":1013106052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5047,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/1073576945","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1073576945},{"type":"v1","addr":"172.21.15.144:6835","nonce":1073576945}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/821548012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":821548012},{"type":"v1","addr":"172.21.15.86:6835","nonce":821548012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":6057,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/955835182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":955835182},{"type":"v1","addr":"172.21.15.86:6837","nonce":955835182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[]} 2024-04-20T21:21:01.330 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 26 2024-04-20T21:21:01.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.332+0000 7fd833200640 1 -- 172.21.15.86:0/4207435138 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd81006c8f0 msgr2=0x7fd81006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:01.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.332+0000 7fd833200640 1 --2- 172.21.15.86:0/4207435138 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd81006c8f0 0x7fd81006edb0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd81c00a990 tx=0x7fd81c008040 comp rx=0 tx=0).stop 2024-04-20T21:21:01.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.332+0000 7fd833200640 1 -- 172.21.15.86:0/4207435138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd834161f30 msgr2=0x7fd8341f7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:01.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.332+0000 7fd833200640 1 --2- 172.21.15.86:0/4207435138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd834161f30 0x7fd8341f7d20 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fd82c00b410 tx=0x7fd82c00b8d0 comp rx=0 tx=0).stop 2024-04-20T21:21:01.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.332+0000 7fd833200640 1 -- 172.21.15.86:0/4207435138 shutdown_connections 2024-04-20T21:21:01.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.332+0000 7fd833200640 1 --2- 172.21.15.86:0/4207435138 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd81006c8f0 0x7fd81006edb0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:01.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.332+0000 7fd833200640 1 --2- 172.21.15.86:0/4207435138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd834161f30 0x7fd8341f7d20 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:01.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.332+0000 7fd833200640 1 --2- 172.21.15.86:0/4207435138 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd8340d3a80 0x7fd8341f3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:01.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.332+0000 7fd833200640 1 --2- 172.21.15.86:0/4207435138 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8340d0e50 0x7fd8341f3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:01.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.332+0000 7fd833200640 1 -- 172.21.15.86:0/4207435138 >> 172.21.15.86:0/4207435138 conn(0x7fd83408bf70 msgr2=0x7fd8340b5700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:01.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.332+0000 7fd833200640 1 -- 172.21.15.86:0/4207435138 shutdown_connections 2024-04-20T21:21:01.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.333+0000 7fd833200640 1 -- 172.21.15.86:0/4207435138 wait complete. 2024-04-20T21:21:01.350 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-20T21:21:01.350 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-20T21:21:01.350 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-20T21:21:01.351 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:21:01.351 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-20T21:21:01.351 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-20T21:21:01.351 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-20T21:21:01.351 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:21:01.351 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-20T21:21:01.351 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-20T21:21:01.352 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-20T21:21:01.352 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:21:01.352 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-20T21:21:01.352 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-20T21:21:01.352 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-20T21:21:01.352 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:21:01.369 INFO:tasks.ceph.mds.a:Stopped 2024-04-20T21:21:01.369 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-20T21:21:01.370 INFO:tasks.ceph.mds.c:Stopped 2024-04-20T21:21:01.370 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-20T21:21:01.373 INFO:tasks.ceph.mds.d:Stopped 2024-04-20T21:21:01.373 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-20T21:21:01.375 INFO:tasks.ceph.mds.b:Stopped 2024-04-20T21:21:01.375 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-20T21:21:01.417 INFO:tasks.ceph.mds.a:Started 2024-04-20T21:21:01.418 DEBUG:teuthology.parallel:result is None 2024-04-20T21:21:01.422 INFO:tasks.ceph.mds.d:Started 2024-04-20T21:21:01.423 DEBUG:teuthology.parallel:result is None 2024-04-20T21:21:01.423 INFO:tasks.ceph.mds.c:Started 2024-04-20T21:21:01.423 DEBUG:teuthology.parallel:result is None 2024-04-20T21:21:01.429 INFO:tasks.ceph.mds.b:Started 2024-04-20T21:21:01.429 DEBUG:teuthology.parallel:result is None 2024-04-20T21:21:01.429 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-20T21:21:01.565 INFO:tasks.ceph.mds.c.smithi086.stdout:starting mds.c at 2024-04-20T21:21:01.566 INFO:tasks.ceph.mds.a.smithi086.stdout:starting mds.a at 2024-04-20T21:21:01.581 INFO:tasks.ceph.mds.d.smithi144.stdout:starting mds.d at 2024-04-20T21:21:01.583 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2024-04-20T21:21:01.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.609+0000 7ffb3e200640 1 Processor -- start 2024-04-20T21:21:01.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.609+0000 7ffb3e200640 1 -- start start 2024-04-20T21:21:01.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.610+0000 7ffb3e200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ffb380d0ee0 0x7ffb380d1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:01.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.610+0000 7ffb3e200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb380d3100 con 0x7ffb3815e690 2024-04-20T21:21:01.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.610+0000 7ffb3e200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb380d33b0 con 0x7ffb380d25a0 2024-04-20T21:21:01.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.610+0000 7ffb3e200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ffb380d18b0 con 0x7ffb380d0ee0 2024-04-20T21:21:01.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.611+0000 7ffb37e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ffb380d0ee0 0x7ffb380d1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:01.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.611+0000 7ffb37e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ffb380d0ee0 0x7ffb380d1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55764/0 (socket says 172.21.15.86:55764) 2024-04-20T21:21:01.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.611+0000 7ffb37e00640 1 -- 172.21.15.86:0/1138079818 learned_addr learned my addr 172.21.15.86:0/1138079818 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:01.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.611+0000 7ffb37e00640 1 -- 172.21.15.86:0/1138079818 >> v1:172.21.15.144:6789/0 conn(0x7ffb380d25a0 legacy=0x7ffb380d29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:01.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.611+0000 7ffb37e00640 1 -- 172.21.15.86:0/1138079818 >> v1:172.21.15.86:6789/0 conn(0x7ffb3815e690 legacy=0x7ffb3806ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:01.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.611+0000 7ffb37e00640 1 -- 172.21.15.86:0/1138079818 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb380d3690 con 0x7ffb380d0ee0 2024-04-20T21:21:01.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.612+0000 7ffb37e00640 1 --2- 172.21.15.86:0/1138079818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb380d0ee0 0x7ffb380d1370 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7ffb24009850 tx=0x7ffb2402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c66c3094b84a938 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:01.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.612+0000 7ffb37400640 1 -- 172.21.15.86:0/1138079818 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb24004230 con 0x7ffb380d0ee0 2024-04-20T21:21:01.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.612+0000 7ffb37400640 1 -- 172.21.15.86:0/1138079818 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb240043d0 con 0x7ffb380d0ee0 2024-04-20T21:21:01.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.612+0000 7ffb37400640 1 -- 172.21.15.86:0/1138079818 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb24030840 con 0x7ffb380d0ee0 2024-04-20T21:21:01.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.613+0000 7ffb3e200640 1 -- 172.21.15.86:0/1138079818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb380d0ee0 msgr2=0x7ffb380d1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:01.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.613+0000 7ffb3e200640 1 --2- 172.21.15.86:0/1138079818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb380d0ee0 0x7ffb380d1370 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7ffb24009850 tx=0x7ffb2402b980 comp rx=0 tx=0).stop 2024-04-20T21:21:01.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.613+0000 7ffb3e200640 1 -- 172.21.15.86:0/1138079818 shutdown_connections 2024-04-20T21:21:01.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.613+0000 7ffb3e200640 1 --2- 172.21.15.86:0/1138079818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb380d0ee0 0x7ffb380d1370 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:01.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.613+0000 7ffb3e200640 1 -- 172.21.15.86:0/1138079818 >> 172.21.15.86:0/1138079818 conn(0x7ffb3809d8a0 msgr2=0x7ffb3808bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:01.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.613+0000 7ffb3e200640 1 -- 172.21.15.86:0/1138079818 shutdown_connections 2024-04-20T21:21:01.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.613+0000 7ffb3e200640 1 -- 172.21.15.86:0/1138079818 wait complete. 2024-04-20T21:21:01.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.615+0000 7ffb3e200640 1 Processor -- start 2024-04-20T21:21:01.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.615+0000 7ffb3e200640 1 -- start start 2024-04-20T21:21:01.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.616+0000 7ffb3e200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb380d0ee0 0x7ffb38144a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:01.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.616+0000 7ffb3e200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb380d25a0 0x7ffb380c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:01.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.616+0000 7ffb3e200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb3815e690 0x7ffb381419e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:01.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.616+0000 7ffb3e200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffb380cef00 con 0x7ffb3815e690 2024-04-20T21:21:01.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.616+0000 7ffb3e200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ffb380cec20 con 0x7ffb380d0ee0 2024-04-20T21:21:01.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.616+0000 7ffb3e200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ffb3815ea70 con 0x7ffb380d25a0 2024-04-20T21:21:01.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.616+0000 7ffb3d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb3815e690 0x7ffb381419e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:01.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.616+0000 7ffb3d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb3815e690 0x7ffb381419e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55768/0 (socket says 172.21.15.86:55768) 2024-04-20T21:21:01.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.616+0000 7ffb3d800640 1 -- 172.21.15.86:0/1735058250 learned_addr learned my addr 172.21.15.86:0/1735058250 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:01.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.616+0000 7ffb37e00640 1 --2- 172.21.15.86:0/1735058250 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb380d25a0 0x7ffb380c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:01.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.616+0000 7ffb3ce00640 1 --2- 172.21.15.86:0/1735058250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb380d0ee0 0x7ffb38144a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:01.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.617+0000 7ffb3d800640 1 -- 172.21.15.86:0/1735058250 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb380d25a0 msgr2=0x7ffb380c8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:01.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.617+0000 7ffb3d800640 1 --2- 172.21.15.86:0/1735058250 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb380d25a0 0x7ffb380c8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:01.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.617+0000 7ffb3d800640 1 -- 172.21.15.86:0/1735058250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb380d0ee0 msgr2=0x7ffb38144a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:01.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.617+0000 7ffb3d800640 1 --2- 172.21.15.86:0/1735058250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb380d0ee0 0x7ffb38144a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:01.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.617+0000 7ffb3d800640 1 -- 172.21.15.86:0/1735058250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb380b5650 con 0x7ffb3815e690 2024-04-20T21:21:01.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.617+0000 7ffb3d800640 1 --2- 172.21.15.86:0/1735058250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb3815e690 0x7ffb381419e0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7ffb3000c780 tx=0x7ffb3000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:01.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.617+0000 7ffb34c00640 1 -- 172.21.15.86:0/1735058250 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb30017070 con 0x7ffb3815e690 2024-04-20T21:21:01.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.617+0000 7ffb34c00640 1 -- 172.21.15.86:0/1735058250 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb30003da0 con 0x7ffb3815e690 2024-04-20T21:21:01.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.617+0000 7ffb3e200640 1 -- 172.21.15.86:0/1735058250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb38141f20 con 0x7ffb3815e690 2024-04-20T21:21:01.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.617+0000 7ffb34c00640 1 -- 172.21.15.86:0/1735058250 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb300052b0 con 0x7ffb3815e690 2024-04-20T21:21:01.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.617+0000 7ffb3e200640 1 -- 172.21.15.86:0/1735058250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb380cac70 con 0x7ffb3815e690 2024-04-20T21:21:01.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.619+0000 7ffb3e200640 1 -- 172.21.15.86:0/1735058250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb380caeb0 con 0x7ffb3815e690 2024-04-20T21:21:01.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.619+0000 7ffb34c00640 1 -- 172.21.15.86:0/1735058250 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffb3001a070 con 0x7ffb3815e690 2024-04-20T21:21:01.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.620+0000 7ffb34c00640 1 --2- 172.21.15.86:0/1735058250 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ffb1806c8f0 0x7ffb1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:01.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.620+0000 7ffb34c00640 1 -- 172.21.15.86:0/1735058250 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7ffb3008adf0 con 0x7ffb3815e690 2024-04-20T21:21:01.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.620+0000 7ffb3ce00640 1 --2- 172.21.15.86:0/1735058250 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ffb1806c8f0 0x7ffb1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:01.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.621+0000 7ffb3ce00640 1 --2- 172.21.15.86:0/1735058250 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ffb1806c8f0 0x7ffb1806edb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c0016e0 tx=0x7ffb2c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:01.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.625+0000 7ffb34c00640 1 -- 172.21.15.86:0/1735058250 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ffb380b5650 con 0x7ffb3815e690 2024-04-20T21:21:01.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:01.833+0000 7ffb3e200640 1 -- 172.21.15.86:0/1735058250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7ffb3804fcf0 con 0x7ffb3815e690 2024-04-20T21:21:02.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:02.992+0000 7ffb34c00640 1 -- 172.21.15.86:0/1735058250 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v55) ==== 97+0+0 (secure 0 0 0) 0x7ffb30058ec0 con 0x7ffb3815e690 2024-04-20T21:21:02.993 INFO:teuthology.orchestra.run.smithi086.stderr: removed all blocklist entries 2024-04-20T21:21:02.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:02.996+0000 7ffb3e200640 1 -- 172.21.15.86:0/1735058250 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ffb1806c8f0 msgr2=0x7ffb1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:02.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:02.996+0000 7ffb3e200640 1 --2- 172.21.15.86:0/1735058250 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ffb1806c8f0 0x7ffb1806edb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c0016e0 tx=0x7ffb2c009040 comp rx=0 tx=0).stop 2024-04-20T21:21:02.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:02.996+0000 7ffb3e200640 1 -- 172.21.15.86:0/1735058250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb3815e690 msgr2=0x7ffb381419e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:02.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:02.996+0000 7ffb3e200640 1 --2- 172.21.15.86:0/1735058250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb3815e690 0x7ffb381419e0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7ffb3000c780 tx=0x7ffb3000cc40 comp rx=0 tx=0).stop 2024-04-20T21:21:02.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:02.996+0000 7ffb3e200640 1 -- 172.21.15.86:0/1735058250 shutdown_connections 2024-04-20T21:21:02.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:02.996+0000 7ffb3e200640 1 --2- 172.21.15.86:0/1735058250 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ffb1806c8f0 0x7ffb1806edb0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:02.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:02.996+0000 7ffb3e200640 1 --2- 172.21.15.86:0/1735058250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb3815e690 0x7ffb381419e0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:02.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:02.996+0000 7ffb3e200640 1 --2- 172.21.15.86:0/1735058250 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb380d25a0 0x7ffb380c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:02.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:02.996+0000 7ffb3e200640 1 --2- 172.21.15.86:0/1735058250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb380d0ee0 0x7ffb38144a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:02.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:02.996+0000 7ffb3e200640 1 -- 172.21.15.86:0/1735058250 >> 172.21.15.86:0/1735058250 conn(0x7ffb3809d8a0 msgr2=0x7ffb3806ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:02.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:02.997+0000 7ffb3e200640 1 -- 172.21.15.86:0/1735058250 shutdown_connections 2024-04-20T21:21:02.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:02.997+0000 7ffb3e200640 1 -- 172.21.15.86:0/1735058250 wait complete. 2024-04-20T21:21:03.018 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-20T21:21:03.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.212+0000 7f7cfa400640 1 Processor -- start 2024-04-20T21:21:03.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.212+0000 7f7cfa400640 1 -- start start 2024-04-20T21:21:03.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.214+0000 7f7cfa400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7cf406eeb0 0x7f7cf40b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:03.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.214+0000 7f7cfa400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7cf406f290 0x7f7cf40cd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:03.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.214+0000 7f7cfa400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cf415f160 con 0x7f7cf40cdb70 2024-04-20T21:21:03.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.214+0000 7f7cfa400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f7cf415f440 con 0x7f7cf406f290 2024-04-20T21:21:03.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.214+0000 7f7cfa400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f7cf415f720 con 0x7f7cf406eeb0 2024-04-20T21:21:03.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.214+0000 7f7cf2a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7cf406f290 0x7f7cf40cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:03.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.214+0000 7f7cf3e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f7cf40cdb70 0x7f7cf415bbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54222/0 (socket says 172.21.15.86:54222) 2024-04-20T21:21:03.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.214+0000 7f7cf3e00640 1 -- 172.21.15.86:0/389670611 learned_addr learned my addr 172.21.15.86:0/389670611 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:21:03.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.215+0000 7f7cf3400640 1 --2- 172.21.15.86:0/389670611 >> v2:172.21.15.144:3300/0 conn(0x7f7cf406eeb0 0x7f7cf40b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:03.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.215+0000 7f7cf2a00640 1 -- 172.21.15.86:0/389670611 >> v2:172.21.15.144:3300/0 conn(0x7f7cf406eeb0 msgr2=0x7f7cf40b3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:03.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.215+0000 7f7cf2a00640 1 --2- 172.21.15.86:0/389670611 >> v2:172.21.15.144:3300/0 conn(0x7f7cf406eeb0 0x7f7cf40b3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:03.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.215+0000 7f7cf2a00640 1 -- 172.21.15.86:0/389670611 >> v1:172.21.15.86:6789/0 conn(0x7f7cf40cdb70 legacy=0x7f7cf415bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:03.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.215+0000 7f7cf2a00640 1 -- 172.21.15.86:0/389670611 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cf415fa00 con 0x7f7cf406f290 2024-04-20T21:21:03.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.215+0000 7f7cf2000640 1 -- 172.21.15.86:0/389670611 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 304949842 0 0) 0x7f7cf415f160 con 0x7f7cf40cdb70 2024-04-20T21:21:03.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.216+0000 7f7cf2a00640 1 --2- 172.21.15.86:0/389670611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cf406f290 0x7f7cf40cd630 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f7cf406c720 tx=0x7f7cdc02bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70f07ab1344dc641 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:03.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.216+0000 7f7cf2000640 1 -- 172.21.15.86:0/389670611 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc039070 con 0x7f7cf406f290 2024-04-20T21:21:03.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.216+0000 7f7cf2000640 1 -- 172.21.15.86:0/389670611 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cdc02cb60 con 0x7f7cf406f290 2024-04-20T21:21:03.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.216+0000 7f7cf2000640 1 -- 172.21.15.86:0/389670611 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc034440 con 0x7f7cf406f290 2024-04-20T21:21:03.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.217+0000 7f7cfa400640 1 -- 172.21.15.86:0/389670611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cf406f290 msgr2=0x7f7cf40cd630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:03.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.217+0000 7f7cfa400640 1 --2- 172.21.15.86:0/389670611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cf406f290 0x7f7cf40cd630 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f7cf406c720 tx=0x7f7cdc02bdb0 comp rx=0 tx=0).stop 2024-04-20T21:21:03.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.217+0000 7f7cfa400640 1 -- 172.21.15.86:0/389670611 shutdown_connections 2024-04-20T21:21:03.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.217+0000 7f7cfa400640 1 --2- 172.21.15.86:0/389670611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cf406f290 0x7f7cf40cd630 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:03.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.217+0000 7f7cfa400640 1 --2- 172.21.15.86:0/389670611 >> v2:172.21.15.144:3300/0 conn(0x7f7cf406eeb0 0x7f7cf40b3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:03.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.217+0000 7f7cfa400640 1 -- 172.21.15.86:0/389670611 >> 172.21.15.86:0/389670611 conn(0x7f7cf406b980 msgr2=0x7f7cf406bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:03.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.217+0000 7f7cfa400640 1 -- 172.21.15.86:0/389670611 shutdown_connections 2024-04-20T21:21:03.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.217+0000 7f7cfa400640 1 -- 172.21.15.86:0/389670611 wait complete. 2024-04-20T21:21:03.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.219+0000 7f7cfa400640 1 Processor -- start 2024-04-20T21:21:03.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.219+0000 7f7cfa400640 1 -- start start 2024-04-20T21:21:03.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.220+0000 7f7cfa400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7cf406eeb0 0x7f7cf41686c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:03.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.220+0000 7f7cfa400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cf406f290 0x7f7cf4141a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:03.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.220+0000 7f7cfa400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cf40cdb70 0x7f7cf4167bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:03.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.220+0000 7f7cfa400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf40cff90 con 0x7f7cf40cdb70 2024-04-20T21:21:03.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.220+0000 7f7cfa400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf406f700 con 0x7f7cf406f290 2024-04-20T21:21:03.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.220+0000 7f7cfa400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7cf40d0110 con 0x7f7cf406eeb0 2024-04-20T21:21:03.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.220+0000 7f7cf3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cf40cdb70 0x7f7cf4167bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:03.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.220+0000 7f7cf3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cf40cdb70 0x7f7cf4167bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55798/0 (socket says 172.21.15.86:55798) 2024-04-20T21:21:03.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.220+0000 7f7cf3e00640 1 -- 172.21.15.86:0/39808419 learned_addr learned my addr 172.21.15.86:0/39808419 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:03.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.221+0000 7f7cf2a00640 1 --2- 172.21.15.86:0/39808419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cf406f290 0x7f7cf4141a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:03.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.221+0000 7f7cf3e00640 1 -- 172.21.15.86:0/39808419 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7cf406eeb0 msgr2=0x7f7cf41686c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:03.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.221+0000 7f7cf3e00640 1 --2- 172.21.15.86:0/39808419 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7cf406eeb0 0x7f7cf41686c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:03.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.221+0000 7f7cf3e00640 1 -- 172.21.15.86:0/39808419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cf406f290 msgr2=0x7f7cf4141a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:03.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.221+0000 7f7cf3e00640 1 --2- 172.21.15.86:0/39808419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cf406f290 0x7f7cf4141a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:03.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.221+0000 7f7cf3e00640 1 -- 172.21.15.86:0/39808419 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cf4149180 con 0x7f7cf40cdb70 2024-04-20T21:21:03.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.221+0000 7f7cf3e00640 1 --2- 172.21.15.86:0/39808419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cf40cdb70 0x7f7cf4167bb0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f7cec00c450 tx=0x7f7cec00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:03.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.221+0000 7f7ceb400640 1 -- 172.21.15.86:0/39808419 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cec018070 con 0x7f7cf40cdb70 2024-04-20T21:21:03.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.221+0000 7f7ceb400640 1 -- 172.21.15.86:0/39808419 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cec007500 con 0x7f7cf40cdb70 2024-04-20T21:21:03.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.221+0000 7f7ceb400640 1 -- 172.21.15.86:0/39808419 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cec004a20 con 0x7f7cf40cdb70 2024-04-20T21:21:03.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.221+0000 7f7cfa400640 1 -- 172.21.15.86:0/39808419 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cf4149460 con 0x7f7cf40cdb70 2024-04-20T21:21:03.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.221+0000 7f7cfa400640 1 -- 172.21.15.86:0/39808419 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cf4149a20 con 0x7f7cf40cdb70 2024-04-20T21:21:03.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.222+0000 7f7cfa400640 1 -- 172.21.15.86:0/39808419 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cf40689e0 con 0x7f7cf40cdb70 2024-04-20T21:21:03.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.223+0000 7f7ceb400640 1 -- 172.21.15.86:0/39808419 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7cec004bc0 con 0x7f7cf40cdb70 2024-04-20T21:21:03.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.224+0000 7f7ceb400640 1 --2- 172.21.15.86:0/39808419 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7cd006c8f0 0x7f7cd006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:03.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.224+0000 7f7ceb400640 1 -- 172.21.15.86:0/39808419 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4857+0+0 (secure 0 0 0) 0x7f7cec08aa90 con 0x7f7cf40cdb70 2024-04-20T21:21:03.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.225+0000 7f7cf3400640 1 --2- 172.21.15.86:0/39808419 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7cd006c8f0 0x7f7cd006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:03.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.225+0000 7f7cf3400640 1 --2- 172.21.15.86:0/39808419 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7cd006c8f0 0x7f7cd006edb0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4006fd0 tx=0x7f7ce4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:03.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.230+0000 7f7ceb400640 1 -- 172.21.15.86:0/39808419 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7cec059210 con 0x7f7cf40cdb70 2024-04-20T21:21:03.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.494+0000 7f7cfa400640 1 -- 172.21.15.86:0/39808419 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f7cf415d750 con 0x7f7cf40cdb70 2024-04-20T21:21:03.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.495+0000 7f7ceb400640 1 -- 172.21.15.86:0/39808419 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) ==== 80+0+5304 (secure 0 0 0) 0x7f7cec058b10 con 0x7f7cf40cdb70 2024-04-20T21:21:03.496 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:21:03.496 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:21:03.496 INFO:teuthology.orchestra.run.smithi086.stdout: "auth_dump": [ 2024-04-20T21:21:03.496 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:21:03.496 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.a", 2024-04-20T21:21:03.496 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDAMCRmOpS/LRAAPZUUQNf6TyeJ2tF7YPnIKw==", 2024-04-20T21:21:03.496 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:21:03.496 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:21:03.496 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-20T21:21:03.496 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-20T21:21:03.496 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:21:03.497 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:21:03.497 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:21:03.497 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:21:03.497 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.b", 2024-04-20T21:21:03.497 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDBMCRmCJLpDRAA/JiLRSAlQRNMpDC+qDSr0g==", 2024-04-20T21:21:03.497 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:21:03.497 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:21:03.497 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-20T21:21:03.497 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-20T21:21:03.497 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:21:03.497 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:21:03.497 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:21:03.497 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:21:03.497 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.c", 2024-04-20T21:21:03.497 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDAMCRmPHOFOxAA4tUsw/n6oSKrNmzPD+NKyw==", 2024-04-20T21:21:03.497 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:21:03.498 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:21:03.498 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-20T21:21:03.498 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-20T21:21:03.498 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:21:03.498 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:21:03.498 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:21:03.498 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:21:03.498 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.d", 2024-04-20T21:21:03.498 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDBMCRmZnPXHBAA0jXwBmGehaO7qens4boRIQ==", 2024-04-20T21:21:03.498 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:21:03.498 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:21:03.498 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-20T21:21:03.498 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-20T21:21:03.498 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:21:03.499 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:21:03.499 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:21:03.499 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:21:03.499 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.0", 2024-04-20T21:21:03.499 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDFMCRm4w9UHhAA56y4dJyDASI+IoZv2XrEoQ==", 2024-04-20T21:21:03.499 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:21:03.499 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:21:03.499 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:21:03.499 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:21:03.499 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:21:03.499 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:21:03.499 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:21:03.499 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.1", 2024-04-20T21:21:03.499 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDIMCRmFo7xKRAAou5B04+J1gZtTzW/Mo+nRQ==", 2024-04-20T21:21:03.499 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:21:03.499 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:21:03.499 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:21:03.500 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:21:03.500 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:21:03.500 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:21:03.500 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:21:03.500 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.2", 2024-04-20T21:21:03.500 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDLMCRm7/jFNRAA4KcRsapPFLaKf0NaH5MApg==", 2024-04-20T21:21:03.500 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:21:03.500 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:21:03.500 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:21:03.500 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:21:03.500 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:21:03.500 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:21:03.500 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:21:03.500 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.3", 2024-04-20T21:21:03.500 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDPMCRmUTPMExAA0/dubb4ehsWIwZPS/ao68w==", 2024-04-20T21:21:03.500 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:21:03.501 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:21:03.501 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:21:03.501 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:21:03.501 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:21:03.501 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:21:03.501 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:21:03.501 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.4", 2024-04-20T21:21:03.501 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDVMCRmB4F2KRAA8Xj0DfH5Ou8TUrO8X/BQyw==", 2024-04-20T21:21:03.501 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:21:03.501 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:21:03.501 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:21:03.501 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:21:03.501 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:21:03.501 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:21:03.501 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:21:03.502 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.5", 2024-04-20T21:21:03.502 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDYMCRmSOvYNRAAODYXmRHMGPrCAxcHwlS7Ww==", 2024-04-20T21:21:03.502 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:21:03.502 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:21:03.502 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:21:03.502 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:21:03.502 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:21:03.502 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:21:03.502 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:21:03.502 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.6", 2024-04-20T21:21:03.502 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDcMCRmENJ9BBAAi/KBUhbYGNXEyiSjSHavWQ==", 2024-04-20T21:21:03.502 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:21:03.502 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:21:03.502 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:21:03.502 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:21:03.503 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:21:03.503 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:21:03.503 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:21:03.503 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.7", 2024-04-20T21:21:03.503 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDfMCRmfTHTDhAALL9GZBSGjWYrhgFw/YI6IA==", 2024-04-20T21:21:03.503 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:21:03.503 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:21:03.503 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:21:03.503 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:21:03.503 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:21:03.503 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:21:03.503 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:21:03.503 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.0", 2024-04-20T21:21:03.503 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDBMCRmHWxyKRAANzhRi4eajp68h/jWkhFMnw==", 2024-04-20T21:21:03.503 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:21:03.503 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:21:03.503 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow r", 2024-04-20T21:21:03.504 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-20T21:21:03.504 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:21:03.504 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:21:03.504 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:21:03.504 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.1", 2024-04-20T21:21:03.504 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDBMCRmgBJAOxAADJLzO97tjEpKn6kUs8g2QA==", 2024-04-20T21:21:03.504 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:21:03.504 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:21:03.504 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow r", 2024-04-20T21:21:03.504 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow rw", 2024-04-20T21:21:03.504 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rwx" 2024-04-20T21:21:03.504 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:21:03.504 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:21:03.504 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:21:03.504 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.2", 2024-04-20T21:21:03.505 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDBMCRmQAdJMhAAiUmKhPEeuV+UPqQ3gynG9w==", 2024-04-20T21:21:03.505 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:21:03.505 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:21:03.505 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow r", 2024-04-20T21:21:03.505 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow rw", 2024-04-20T21:21:03.505 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rwx" 2024-04-20T21:21:03.505 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:21:03.505 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:21:03.505 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:21:03.505 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.3", 2024-04-20T21:21:03.505 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDCMCRm8yMSCBAAFgSVW0XYu7fa40YxHITljQ==", 2024-04-20T21:21:03.505 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:21:03.505 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:21:03.505 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow r", 2024-04-20T21:21:03.505 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow rw", 2024-04-20T21:21:03.505 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rwx" 2024-04-20T21:21:03.506 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:21:03.506 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:21:03.506 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:21:03.506 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.admin", 2024-04-20T21:21:03.506 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQC/MCRmyru0KBAApSqv1hZRcLvrRwPMnqylqQ==", 2024-04-20T21:21:03.506 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:21:03.506 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-04-20T21:21:03.506 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-20T21:21:03.506 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-20T21:21:03.506 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:21:03.506 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:21:03.506 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:21:03.506 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:21:03.506 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mgr.x", 2024-04-20T21:21:03.507 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDAMCRmvTgjIxAACeSV8gNkL5LV3xzPS6VPeg==", 2024-04-20T21:21:03.507 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:21:03.507 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-04-20T21:21:03.507 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile mgr", 2024-04-20T21:21:03.507 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:21:03.507 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:21:03.507 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:21:03.507 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:21:03.507 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mgr.y", 2024-04-20T21:21:03.507 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDAMCRmSn3VGBAA0eu3R53f3gFgscWFMRkDzg==", 2024-04-20T21:21:03.507 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:21:03.507 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-04-20T21:21:03.507 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile mgr", 2024-04-20T21:21:03.507 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:21:03.507 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:21:03.507 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:21:03.508 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-20T21:21:03.508 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:21:03.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.498+0000 7f7cfa400640 1 -- 172.21.15.86:0/39808419 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7cd006c8f0 msgr2=0x7f7cd006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:03.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.498+0000 7f7cfa400640 1 --2- 172.21.15.86:0/39808419 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7cd006c8f0 0x7f7cd006edb0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4006fd0 tx=0x7f7ce4008040 comp rx=0 tx=0).stop 2024-04-20T21:21:03.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.498+0000 7f7cfa400640 1 -- 172.21.15.86:0/39808419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cf40cdb70 msgr2=0x7f7cf4167bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:03.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.498+0000 7f7cfa400640 1 --2- 172.21.15.86:0/39808419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cf40cdb70 0x7f7cf4167bb0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f7cec00c450 tx=0x7f7cec00c910 comp rx=0 tx=0).stop 2024-04-20T21:21:03.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.498+0000 7f7cfa400640 1 -- 172.21.15.86:0/39808419 shutdown_connections 2024-04-20T21:21:03.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.498+0000 7f7cfa400640 1 --2- 172.21.15.86:0/39808419 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7cd006c8f0 0x7f7cd006edb0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:03.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.498+0000 7f7cfa400640 1 --2- 172.21.15.86:0/39808419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cf40cdb70 0x7f7cf4167bb0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:03.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.498+0000 7f7cfa400640 1 --2- 172.21.15.86:0/39808419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cf406f290 0x7f7cf4141a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:03.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.498+0000 7f7cfa400640 1 --2- 172.21.15.86:0/39808419 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7cf406eeb0 0x7f7cf41686c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:03.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.498+0000 7f7cfa400640 1 -- 172.21.15.86:0/39808419 >> 172.21.15.86:0/39808419 conn(0x7f7cf406b980 msgr2=0x7f7cf415d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:03.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.498+0000 7f7cfa400640 1 -- 172.21.15.86:0/39808419 shutdown_connections 2024-04-20T21:21:03.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.498+0000 7f7cfa400640 1 -- 172.21.15.86:0/39808419 wait complete. 2024-04-20T21:21:03.517 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:21:03.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.718+0000 7f4d37200640 1 Processor -- start 2024-04-20T21:21:03.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.718+0000 7f4d37200640 1 -- start start 2024-04-20T21:21:03.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.720+0000 7f4d37200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4d300d50c0 0x7f4d300d3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:03.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.720+0000 7f4d37200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d300d3950 con 0x7f4d3015ece0 2024-04-20T21:21:03.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.720+0000 7f4d37200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d300cec00 con 0x7f4d300cf9c0 2024-04-20T21:21:03.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.720+0000 7f4d37200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f4d300ceee0 con 0x7f4d300d50c0 2024-04-20T21:21:03.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.720+0000 7f4d35400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4d300d50c0 0x7f4d300d3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:03.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.720+0000 7f4d35e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f4d3015ece0 0x7f4d3015f0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54238/0 (socket says 172.21.15.86:54238) 2024-04-20T21:21:03.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.720+0000 7f4d35e00640 1 -- 172.21.15.86:0/3016229190 learned_addr learned my addr 172.21.15.86:0/3016229190 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:21:03.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.721+0000 7f4d35400640 1 -- 172.21.15.86:0/3016229190 >> v1:172.21.15.144:6789/0 conn(0x7f4d300cf9c0 legacy=0x7f4d30160000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:03.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.721+0000 7f4d35400640 1 -- 172.21.15.86:0/3016229190 >> v1:172.21.15.86:6789/0 conn(0x7f4d3015ece0 legacy=0x7f4d3015f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:03.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.721+0000 7f4d35400640 1 -- 172.21.15.86:0/3016229190 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d300cf1c0 con 0x7f4d300d50c0 2024-04-20T21:21:03.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.721+0000 7f4d34a00640 1 -- 172.21.15.86:0/3016229190 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3864158673 0 0) 0x7f4d300d3950 con 0x7f4d3015ece0 2024-04-20T21:21:03.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.722+0000 7f4d35400640 1 --2- 172.21.15.86:0/3016229190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d300d50c0 0x7f4d300d3100 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f4d200097d0 tx=0x7f4d2002b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e66ac9c3c0211b7f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:03.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.722+0000 7f4d34a00640 1 -- 172.21.15.86:0/3016229190 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d20031780 con 0x7f4d300d50c0 2024-04-20T21:21:03.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.722+0000 7f4d34a00640 1 -- 172.21.15.86:0/3016229190 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d20037070 con 0x7f4d300d50c0 2024-04-20T21:21:03.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.722+0000 7f4d34a00640 1 -- 172.21.15.86:0/3016229190 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d20031b40 con 0x7f4d300d50c0 2024-04-20T21:21:03.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.723+0000 7f4d37200640 1 -- 172.21.15.86:0/3016229190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d300d50c0 msgr2=0x7f4d300d3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:03.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.723+0000 7f4d37200640 1 --2- 172.21.15.86:0/3016229190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d300d50c0 0x7f4d300d3100 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f4d200097d0 tx=0x7f4d2002b820 comp rx=0 tx=0).stop 2024-04-20T21:21:03.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.723+0000 7f4d37200640 1 -- 172.21.15.86:0/3016229190 shutdown_connections 2024-04-20T21:21:03.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.723+0000 7f4d37200640 1 --2- 172.21.15.86:0/3016229190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d300d50c0 0x7f4d300d3100 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:03.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.723+0000 7f4d37200640 1 -- 172.21.15.86:0/3016229190 >> 172.21.15.86:0/3016229190 conn(0x7f4d3008bf70 msgr2=0x7f4d300d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:03.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.723+0000 7f4d37200640 1 -- 172.21.15.86:0/3016229190 shutdown_connections 2024-04-20T21:21:03.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.723+0000 7f4d37200640 1 -- 172.21.15.86:0/3016229190 wait complete. 2024-04-20T21:21:03.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.725+0000 7f4d37200640 1 Processor -- start 2024-04-20T21:21:03.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.725+0000 7f4d37200640 1 -- start start 2024-04-20T21:21:03.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.726+0000 7f4d37200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d300cf9c0 0x7f4d300b9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:03.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.726+0000 7f4d37200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d300d50c0 0x7f4d301e8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:03.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.726+0000 7f4d37200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d3015ece0 0x7f4d30156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:03.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.726+0000 7f4d37200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3015ea50 con 0x7f4d300cf9c0 2024-04-20T21:21:03.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.726+0000 7f4d37200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4d300d3640 con 0x7f4d300d50c0 2024-04-20T21:21:03.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.726+0000 7f4d37200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f4d300cfe70 con 0x7f4d3015ece0 2024-04-20T21:21:03.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.726+0000 7f4d35e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d300cf9c0 0x7f4d300b9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:03.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.726+0000 7f4d35e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d300cf9c0 0x7f4d300b9f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55818/0 (socket says 172.21.15.86:55818) 2024-04-20T21:21:03.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.726+0000 7f4d35e00640 1 -- 172.21.15.86:0/1315023533 learned_addr learned my addr 172.21.15.86:0/1315023533 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:03.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.726+0000 7f4d35e00640 1 -- 172.21.15.86:0/1315023533 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d3015ece0 msgr2=0x7f4d30156d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:03.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.726+0000 7f4d35e00640 1 --2- 172.21.15.86:0/1315023533 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d3015ece0 0x7f4d30156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:03.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.726+0000 7f4d35e00640 1 -- 172.21.15.86:0/1315023533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d300d50c0 msgr2=0x7f4d301e8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:03.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.726+0000 7f4d35400640 1 --2- 172.21.15.86:0/1315023533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d300d50c0 0x7f4d301e8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:03.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.727+0000 7f4d35e00640 1 --2- 172.21.15.86:0/1315023533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d300d50c0 0x7f4d301e8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:03.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.727+0000 7f4d35e00640 1 -- 172.21.15.86:0/1315023533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d30158f90 con 0x7f4d300cf9c0 2024-04-20T21:21:03.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.727+0000 7f4d35400640 1 --2- 172.21.15.86:0/1315023533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d300d50c0 0x7f4d301e8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:21:03.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.727+0000 7f4d35e00640 1 --2- 172.21.15.86:0/1315023533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d300cf9c0 0x7f4d300b9f00 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f4d2800c890 tx=0x7f4d2800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:03.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.727+0000 7f4d26000640 1 -- 172.21.15.86:0/1315023533 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d28018070 con 0x7f4d300cf9c0 2024-04-20T21:21:03.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.727+0000 7f4d26000640 1 -- 172.21.15.86:0/1315023533 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d2800a640 con 0x7f4d300cf9c0 2024-04-20T21:21:03.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.727+0000 7f4d26000640 1 -- 172.21.15.86:0/1315023533 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2800aa30 con 0x7f4d300cf9c0 2024-04-20T21:21:03.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.727+0000 7f4d37200640 1 -- 172.21.15.86:0/1315023533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d30159270 con 0x7f4d300cf9c0 2024-04-20T21:21:03.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.727+0000 7f4d37200640 1 -- 172.21.15.86:0/1315023533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d30159830 con 0x7f4d300cf9c0 2024-04-20T21:21:03.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.728+0000 7f4d37200640 1 -- 172.21.15.86:0/1315023533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d3004fcf0 con 0x7f4d300cf9c0 2024-04-20T21:21:03.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.729+0000 7f4d26000640 1 -- 172.21.15.86:0/1315023533 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4d2801b070 con 0x7f4d300cf9c0 2024-04-20T21:21:03.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.730+0000 7f4d26000640 1 --2- 172.21.15.86:0/1315023533 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4d1406c8f0 0x7f4d1406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:03.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.730+0000 7f4d26000640 1 -- 172.21.15.86:0/1315023533 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4857+0+0 (secure 0 0 0) 0x7f4d2808afb0 con 0x7f4d300cf9c0 2024-04-20T21:21:03.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.731+0000 7f4d35400640 1 --2- 172.21.15.86:0/1315023533 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4d1406c8f0 0x7f4d1406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:03.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.731+0000 7f4d35400640 1 --2- 172.21.15.86:0/1315023533 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4d1406c8f0 0x7f4d1406edb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4d20036000 tx=0x7f4d20008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:03.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.735+0000 7f4d26000640 1 -- 172.21.15.86:0/1315023533 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4d28059690 con 0x7f4d300cf9c0 2024-04-20T21:21:03.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.924+0000 7f4d37200640 1 -- 172.21.15.86:0/1315023533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4d3006d020 con 0x7f4d300cf9c0 2024-04-20T21:21:03.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.926+0000 7f4d26000640 1 -- 172.21.15.86:0/1315023533 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) ==== 74+0+12891 (secure 0 0 0) 0x7f4d28058f90 con 0x7f4d300cf9c0 2024-04-20T21:21:03.926 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:21:03.927 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":55,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:21:02.990782+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:21:03.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.929+0000 7f4d37200640 1 -- 172.21.15.86:0/1315023533 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4d1406c8f0 msgr2=0x7f4d1406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:03.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.929+0000 7f4d37200640 1 --2- 172.21.15.86:0/1315023533 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4d1406c8f0 0x7f4d1406edb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4d20036000 tx=0x7f4d20008030 comp rx=0 tx=0).stop 2024-04-20T21:21:03.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.929+0000 7f4d37200640 1 -- 172.21.15.86:0/1315023533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d300cf9c0 msgr2=0x7f4d300b9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:03.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.929+0000 7f4d37200640 1 --2- 172.21.15.86:0/1315023533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d300cf9c0 0x7f4d300b9f00 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f4d2800c890 tx=0x7f4d2800cd50 comp rx=0 tx=0).stop 2024-04-20T21:21:03.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.929+0000 7f4d37200640 1 -- 172.21.15.86:0/1315023533 shutdown_connections 2024-04-20T21:21:03.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.929+0000 7f4d37200640 1 --2- 172.21.15.86:0/1315023533 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4d1406c8f0 0x7f4d1406edb0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:03.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.929+0000 7f4d37200640 1 --2- 172.21.15.86:0/1315023533 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d3015ece0 0x7f4d30156d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:03.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.929+0000 7f4d37200640 1 --2- 172.21.15.86:0/1315023533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d300d50c0 0x7f4d301e8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:03.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.929+0000 7f4d37200640 1 --2- 172.21.15.86:0/1315023533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d300cf9c0 0x7f4d300b9f00 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:03.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.929+0000 7f4d37200640 1 -- 172.21.15.86:0/1315023533 >> 172.21.15.86:0/1315023533 conn(0x7f4d3008bf70 msgr2=0x7f4d301611b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:03.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.929+0000 7f4d37200640 1 -- 172.21.15.86:0/1315023533 shutdown_connections 2024-04-20T21:21:03.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:03.929+0000 7f4d37200640 1 -- 172.21.15.86:0/1315023533 wait complete. 2024-04-20T21:21:03.947 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:21:03.947 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:21:04.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.141+0000 7fa83e800640 1 Processor -- start 2024-04-20T21:21:04.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.141+0000 7fa83e800640 1 -- start start 2024-04-20T21:21:04.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.142+0000 7fa83e800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa83815b4b0 0x7fa8380cc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:04.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.142+0000 7fa83e800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa8380af070 0x7fa83815f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:04.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.142+0000 7fa83e800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8380729f0 con 0x7fa83815ef50 2024-04-20T21:21:04.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.142+0000 7fa83e800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fa83815fe10 con 0x7fa8380af070 2024-04-20T21:21:04.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.142+0000 7fa83e800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa83815f8d0 con 0x7fa83815b4b0 2024-04-20T21:21:04.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.143+0000 7fa83ca00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fa83815ef50 0x7fa83815b890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54252/0 (socket says 172.21.15.86:54252) 2024-04-20T21:21:04.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.143+0000 7fa83ca00640 1 -- 172.21.15.86:0/587932563 learned_addr learned my addr 172.21.15.86:0/587932563 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:21:04.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.143+0000 7fa83de00640 1 --2- 172.21.15.86:0/587932563 >> v2:172.21.15.144:3300/0 conn(0x7fa8380af070 0x7fa83815f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:04.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.143+0000 7fa837e00640 1 -- 172.21.15.86:0/587932563 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 15327577 0 0) 0x7fa8380729f0 con 0x7fa83815ef50 2024-04-20T21:21:04.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.143+0000 7fa83d400640 1 --2- 172.21.15.86:0/587932563 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa83815b4b0 0x7fa8380cc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:04.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.143+0000 7fa837e00640 1 -- 172.21.15.86:0/587932563 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa820003520 con 0x7fa83815ef50 2024-04-20T21:21:04.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.144+0000 7fa837e00640 1 -- 172.21.15.86:0/587932563 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1005376286 0 0) 0x7fa820003520 con 0x7fa83815ef50 2024-04-20T21:21:04.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.144+0000 7fa837e00640 1 -- 172.21.15.86:0/587932563 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa83815b4b0 msgr2=0x7fa8380cc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:04.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.144+0000 7fa837e00640 1 --2- 172.21.15.86:0/587932563 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa83815b4b0 0x7fa8380cc3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:04.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.144+0000 7fa837e00640 1 -- 172.21.15.86:0/587932563 >> v2:172.21.15.144:3300/0 conn(0x7fa8380af070 msgr2=0x7fa83815f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:04.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.144+0000 7fa837e00640 1 --2- 172.21.15.86:0/587932563 >> v2:172.21.15.144:3300/0 conn(0x7fa8380af070 0x7fa83815f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:04.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.144+0000 7fa837e00640 1 -- 172.21.15.86:0/587932563 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa838072cd0 con 0x7fa83815ef50 2024-04-20T21:21:04.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.144+0000 7fa837e00640 1 -- 172.21.15.86:0/587932563 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fa824001670 con 0x7fa83815ef50 2024-04-20T21:21:04.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.144+0000 7fa83d400640 1 --2- 172.21.15.86:0/587932563 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa83815b4b0 0x7fa8380cc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:21:04.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.144+0000 7fa837e00640 1 -- 172.21.15.86:0/587932563 >> v1:172.21.15.86:6789/0 conn(0x7fa83815ef50 legacy=0x7fa83815b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:04.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.144+0000 7fa837e00640 1 --2- 172.21.15.86:0/587932563 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa83815b4b0 0x7fa8200039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:04.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.144+0000 7fa837e00640 1 --2- 172.21.15.86:0/587932563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa820003f40 0x7fa820004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:04.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.144+0000 7fa837e00640 1 --2- 172.21.15.86:0/587932563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa820004d40 0x7fa8200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:04.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.144+0000 7fa837e00640 1 -- 172.21.15.86:0/587932563 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa83815fe10 con 0x7fa820003f40 2024-04-20T21:21:04.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.144+0000 7fa837e00640 1 -- 172.21.15.86:0/587932563 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa83815f8d0 con 0x7fa820004d40 2024-04-20T21:21:04.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.144+0000 7fa837e00640 1 -- 172.21.15.86:0/587932563 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa82000ba40 con 0x7fa83815b4b0 2024-04-20T21:21:04.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.144+0000 7fa83ca00640 1 --2- 172.21.15.86:0/587932563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa820003f40 0x7fa820004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:04.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.145+0000 7fa83d400640 1 --2- 172.21.15.86:0/587932563 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa83815b4b0 0x7fa8200039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:04.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.145+0000 7fa83de00640 1 --2- 172.21.15.86:0/587932563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa820004d40 0x7fa8200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:04.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.145+0000 7fa83ca00640 1 -- 172.21.15.86:0/587932563 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa83815b4b0 msgr2=0x7fa8200039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:04.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.145+0000 7fa83ca00640 1 --2- 172.21.15.86:0/587932563 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa83815b4b0 0x7fa8200039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:04.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.145+0000 7fa83ca00640 1 -- 172.21.15.86:0/587932563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa820004d40 msgr2=0x7fa8200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:04.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.145+0000 7fa83ca00640 1 --2- 172.21.15.86:0/587932563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa820004d40 0x7fa8200051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:04.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.145+0000 7fa83ca00640 1 -- 172.21.15.86:0/587932563 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa820003520 con 0x7fa820003f40 2024-04-20T21:21:04.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.146+0000 7fa83ca00640 1 --2- 172.21.15.86:0/587932563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa820003f40 0x7fa820004800 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fa82400c160 tx=0x7fa820005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92203a00bb03745 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:04.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.146+0000 7fa837e00640 1 -- 172.21.15.86:0/587932563 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa824007c80 con 0x7fa820003f40 2024-04-20T21:21:04.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.146+0000 7fa837e00640 1 -- 172.21.15.86:0/587932563 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa824030070 con 0x7fa820003f40 2024-04-20T21:21:04.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.146+0000 7fa837e00640 1 -- 172.21.15.86:0/587932563 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa824005d70 con 0x7fa820003f40 2024-04-20T21:21:04.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.147+0000 7fa83e800640 1 -- 172.21.15.86:0/587932563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa820003f40 msgr2=0x7fa820004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:04.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.147+0000 7fa83e800640 1 --2- 172.21.15.86:0/587932563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa820003f40 0x7fa820004800 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fa82400c160 tx=0x7fa820005830 comp rx=0 tx=0).stop 2024-04-20T21:21:04.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.147+0000 7fa83de00640 1 -- 172.21.15.86:0/587932563 reap_dead start 2024-04-20T21:21:04.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.147+0000 7fa83e800640 1 -- 172.21.15.86:0/587932563 shutdown_connections 2024-04-20T21:21:04.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.147+0000 7fa83e800640 1 -- 172.21.15.86:0/587932563 >> 172.21.15.86:0/587932563 conn(0x7fa8380b1f00 msgr2=0x7fa8380b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:04.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.147+0000 7fa83e800640 1 -- 172.21.15.86:0/587932563 shutdown_connections 2024-04-20T21:21:04.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.147+0000 7fa83e800640 1 -- 172.21.15.86:0/587932563 wait complete. 2024-04-20T21:21:04.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.149+0000 7fa83e800640 1 Processor -- start 2024-04-20T21:21:04.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.149+0000 7fa83e800640 1 -- start start 2024-04-20T21:21:04.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.150+0000 7fa83e800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa820003f40 0x7fa83809d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:04.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.150+0000 7fa83e800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa83815cef0 0x7fa83814cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:04.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.150+0000 7fa83e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa83815b890 0x7fa83815bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:04.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.150+0000 7fa83e800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa838073570 con 0x7fa83815b890 2024-04-20T21:21:04.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.150+0000 7fa83e800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa838073290 con 0x7fa83815cef0 2024-04-20T21:21:04.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.150+0000 7fa83e800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa838072650 con 0x7fa820003f40 2024-04-20T21:21:04.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.150+0000 7fa83de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa83815b890 0x7fa83815bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:04.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.150+0000 7fa83de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa83815b890 0x7fa83815bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55844/0 (socket says 172.21.15.86:55844) 2024-04-20T21:21:04.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.150+0000 7fa83de00640 1 -- 172.21.15.86:0/1806890385 learned_addr learned my addr 172.21.15.86:0/1806890385 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:04.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.150+0000 7fa83de00640 1 -- 172.21.15.86:0/1806890385 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa820003f40 msgr2=0x7fa83809d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:04.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.150+0000 7fa83de00640 1 --2- 172.21.15.86:0/1806890385 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa820003f40 0x7fa83809d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:04.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.150+0000 7fa83de00640 1 -- 172.21.15.86:0/1806890385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa83815cef0 msgr2=0x7fa83814cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:04.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.150+0000 7fa83de00640 1 --2- 172.21.15.86:0/1806890385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa83815cef0 0x7fa83814cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:04.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.150+0000 7fa83de00640 1 -- 172.21.15.86:0/1806890385 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa83815d6b0 con 0x7fa83815b890 2024-04-20T21:21:04.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.151+0000 7fa83de00640 1 --2- 172.21.15.86:0/1806890385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa83815b890 0x7fa83815bd40 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fa8380af500 tx=0x7fa83000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:04.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.151+0000 7fa835600640 1 -- 172.21.15.86:0/1806890385 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83001c070 con 0x7fa83815b890 2024-04-20T21:21:04.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.151+0000 7fa835600640 1 -- 172.21.15.86:0/1806890385 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa830017400 con 0x7fa83815b890 2024-04-20T21:21:04.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.151+0000 7fa835600640 1 -- 172.21.15.86:0/1806890385 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8300177f0 con 0x7fa83815b890 2024-04-20T21:21:04.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.151+0000 7fa83e800640 1 -- 172.21.15.86:0/1806890385 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa83815d3d0 con 0x7fa83815b890 2024-04-20T21:21:04.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.151+0000 7fa83e800640 1 -- 172.21.15.86:0/1806890385 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8380b33e0 con 0x7fa83815b890 2024-04-20T21:21:04.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.152+0000 7fa83e800640 1 -- 172.21.15.86:0/1806890385 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa83804fd90 con 0x7fa83815b890 2024-04-20T21:21:04.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.153+0000 7fa835600640 1 -- 172.21.15.86:0/1806890385 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa830005d10 con 0x7fa83815b890 2024-04-20T21:21:04.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.154+0000 7fa835600640 1 --2- 172.21.15.86:0/1806890385 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa81806c920 0x7fa81806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:04.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.154+0000 7fa835600640 1 -- 172.21.15.86:0/1806890385 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4857+0+0 (secure 0 0 0) 0x7fa83008bb40 con 0x7fa83815b890 2024-04-20T21:21:04.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.154+0000 7fa83d400640 1 --2- 172.21.15.86:0/1806890385 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa81806c920 0x7fa81806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:04.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.155+0000 7fa83d400640 1 --2- 172.21.15.86:0/1806890385 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa81806c920 0x7fa81806ede0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa82c006d40 tx=0x7fa82c006c30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:04.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.158+0000 7fa835600640 1 -- 172.21.15.86:0/1806890385 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa830012070 con 0x7fa83815b890 2024-04-20T21:21:04.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.357+0000 7fa83e800640 1 -- 172.21.15.86:0/1806890385 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa8380b1ce0 con 0x7fa83815b890 2024-04-20T21:21:04.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.358+0000 7fa835600640 1 -- 172.21.15.86:0/1806890385 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v55) ==== 93+0+10 (secure 0 0 0) 0x7fa83005a2b0 con 0x7fa83815b890 2024-04-20T21:21:04.358 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:21:04.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.360+0000 7fa83e800640 1 -- 172.21.15.86:0/1806890385 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa81806c920 msgr2=0x7fa81806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:04.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.360+0000 7fa83e800640 1 --2- 172.21.15.86:0/1806890385 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa81806c920 0x7fa81806ede0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa82c006d40 tx=0x7fa82c006c30 comp rx=0 tx=0).stop 2024-04-20T21:21:04.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.360+0000 7fa83e800640 1 -- 172.21.15.86:0/1806890385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa83815b890 msgr2=0x7fa83815bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:04.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.360+0000 7fa83e800640 1 --2- 172.21.15.86:0/1806890385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa83815b890 0x7fa83815bd40 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fa8380af500 tx=0x7fa83000f4a0 comp rx=0 tx=0).stop 2024-04-20T21:21:04.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.361+0000 7fa83e800640 1 -- 172.21.15.86:0/1806890385 shutdown_connections 2024-04-20T21:21:04.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.361+0000 7fa83e800640 1 --2- 172.21.15.86:0/1806890385 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa81806c920 0x7fa81806ede0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:04.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.361+0000 7fa83e800640 1 --2- 172.21.15.86:0/1806890385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa83815b890 0x7fa83815bd40 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:04.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.361+0000 7fa83e800640 1 --2- 172.21.15.86:0/1806890385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa83815cef0 0x7fa83814cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:04.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.361+0000 7fa83e800640 1 --2- 172.21.15.86:0/1806890385 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa820003f40 0x7fa83809d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:04.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.361+0000 7fa83e800640 1 -- 172.21.15.86:0/1806890385 >> 172.21.15.86:0/1806890385 conn(0x7fa8380b1f00 msgr2=0x7fa838150290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:04.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.361+0000 7fa83e800640 1 -- 172.21.15.86:0/1806890385 shutdown_connections 2024-04-20T21:21:04.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.361+0000 7fa83e800640 1 -- 172.21.15.86:0/1806890385 wait complete. 2024-04-20T21:21:04.380 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-20T21:21:04.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.578+0000 7f9ea2a00640 1 Processor -- start 2024-04-20T21:21:04.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.578+0000 7f9ea2a00640 1 -- start start 2024-04-20T21:21:04.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.579+0000 7f9ea2a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9e9c06e6d0 0x7f9e9c071d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:04.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.579+0000 7f9ea2a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e9c06eb30 con 0x7f9e9c072240 2024-04-20T21:21:04.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.579+0000 7f9ea2a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e9c06dc30 con 0x7f9e9c146130 2024-04-20T21:21:04.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.579+0000 7f9ea2a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f9e9c072690 con 0x7f9e9c06e6d0 2024-04-20T21:21:04.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.580+0000 7f9ea0a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f9e9c072240 0x7f9e9c15cc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54256/0 (socket says 172.21.15.86:54256) 2024-04-20T21:21:04.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.580+0000 7f9ea0a00640 1 -- 172.21.15.86:0/918430022 learned_addr learned my addr 172.21.15.86:0/918430022 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:21:04.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.580+0000 7f9e93400640 1 --2- 172.21.15.86:0/918430022 >> v2:172.21.15.144:3300/0 conn(0x7f9e9c06e6d0 0x7f9e9c071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:04.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.580+0000 7f9e9b400640 1 -- 172.21.15.86:0/918430022 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3001308943 0 0) 0x7f9e94001110 con 0x7f9e9c072240 2024-04-20T21:21:04.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.580+0000 7f9e9b400640 1 -- 172.21.15.86:0/918430022 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9e80003520 con 0x7f9e9c072240 2024-04-20T21:21:04.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.580+0000 7f9e9b400640 1 -- 172.21.15.86:0/918430022 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 82014665 0 0) 0x7f9e80003520 con 0x7f9e9c072240 2024-04-20T21:21:04.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.580+0000 7f9e9b400640 1 -- 172.21.15.86:0/918430022 >> v2:172.21.15.144:3300/0 conn(0x7f9e9c06e6d0 msgr2=0x7f9e9c071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:04.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.580+0000 7f9e9b400640 1 --2- 172.21.15.86:0/918430022 >> v2:172.21.15.144:3300/0 conn(0x7f9e9c06e6d0 0x7f9e9c071d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:04.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.580+0000 7f9e9b400640 1 -- 172.21.15.86:0/918430022 >> v1:172.21.15.144:6789/0 conn(0x7f9e9c146130 legacy=0x7f9e9c0ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:04.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.580+0000 7f9e9b400640 1 -- 172.21.15.86:0/918430022 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e9c0cd8e0 con 0x7f9e9c072240 2024-04-20T21:21:04.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.580+0000 7f9e9b400640 1 -- 172.21.15.86:0/918430022 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f9e94001840 con 0x7f9e9c072240 2024-04-20T21:21:04.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.581+0000 7f9e9b400640 1 -- 172.21.15.86:0/918430022 >> v1:172.21.15.86:6789/0 conn(0x7f9e9c072240 legacy=0x7f9e9c15cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:04.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.581+0000 7f9e9b400640 1 --2- 172.21.15.86:0/918430022 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e800039b0 0x7f9e80003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:04.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.581+0000 7f9e9b400640 1 --2- 172.21.15.86:0/918430022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e80004800 0x7f9e80004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:04.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.581+0000 7f9e9b400640 1 --2- 172.21.15.86:0/918430022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e800051c0 0x7f9e8000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:04.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.581+0000 7f9e9b400640 1 -- 172.21.15.86:0/918430022 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9c072690 con 0x7f9e800051c0 2024-04-20T21:21:04.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.581+0000 7f9e9b400640 1 -- 172.21.15.86:0/918430022 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9e8000c070 con 0x7f9e80004800 2024-04-20T21:21:04.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.581+0000 7f9e9b400640 1 -- 172.21.15.86:0/918430022 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9e8000c270 con 0x7f9e800039b0 2024-04-20T21:21:04.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.581+0000 7f9ea0a00640 1 --2- 172.21.15.86:0/918430022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e800051c0 0x7f9e8000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:04.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.581+0000 7f9ea0a00640 1 -- 172.21.15.86:0/918430022 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e800039b0 msgr2=0x7f9e80003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:04.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.581+0000 7f9e9be00640 1 --2- 172.21.15.86:0/918430022 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e800039b0 0x7f9e80003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:04.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.581+0000 7f9e93400640 1 --2- 172.21.15.86:0/918430022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e80004800 0x7f9e80004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:04.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.581+0000 7f9ea0a00640 1 --2- 172.21.15.86:0/918430022 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e800039b0 0x7f9e80003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:04.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.581+0000 7f9ea0a00640 1 -- 172.21.15.86:0/918430022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e80004800 msgr2=0x7f9e80004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:04.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.581+0000 7f9ea0a00640 1 --2- 172.21.15.86:0/918430022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e80004800 0x7f9e80004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:04.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.581+0000 7f9ea0a00640 1 -- 172.21.15.86:0/918430022 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e80003520 con 0x7f9e800051c0 2024-04-20T21:21:04.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.581+0000 7f9e9be00640 1 --2- 172.21.15.86:0/918430022 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e800039b0 0x7f9e80003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:21:04.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.582+0000 7f9ea0a00640 1 -- 172.21.15.86:0/918430022 reap_dead start 2024-04-20T21:21:04.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.582+0000 7f9ea0a00640 1 --2- 172.21.15.86:0/918430022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e800051c0 0x7f9e8000b9c0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f9e9400c490 tx=0x7f9e80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ed9bedd579dec6b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:04.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.583+0000 7f9e9b400640 1 -- 172.21.15.86:0/918430022 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9400c0e0 con 0x7f9e800051c0 2024-04-20T21:21:04.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.583+0000 7f9e9b400640 1 -- 172.21.15.86:0/918430022 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e94030070 con 0x7f9e800051c0 2024-04-20T21:21:04.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.583+0000 7f9e9b400640 1 -- 172.21.15.86:0/918430022 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e940058d0 con 0x7f9e800051c0 2024-04-20T21:21:04.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.583+0000 7f9ea2a00640 1 -- 172.21.15.86:0/918430022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e800051c0 msgr2=0x7f9e8000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:04.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.583+0000 7f9ea2a00640 1 --2- 172.21.15.86:0/918430022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e800051c0 0x7f9e8000b9c0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f9e9400c490 tx=0x7f9e80005830 comp rx=0 tx=0).stop 2024-04-20T21:21:04.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.583+0000 7f9ea2a00640 1 -- 172.21.15.86:0/918430022 shutdown_connections 2024-04-20T21:21:04.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.583+0000 7f9ea2a00640 1 --2- 172.21.15.86:0/918430022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e800051c0 0x7f9e8000b9c0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:04.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.583+0000 7f9ea2a00640 1 -- 172.21.15.86:0/918430022 >> 172.21.15.86:0/918430022 conn(0x7f9e9c0b2360 msgr2=0x7f9e9c0b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:04.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.584+0000 7f9ea2a00640 1 -- 172.21.15.86:0/918430022 shutdown_connections 2024-04-20T21:21:04.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.584+0000 7f9ea2a00640 1 -- 172.21.15.86:0/918430022 wait complete. 2024-04-20T21:21:04.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.585+0000 7f9ea2a00640 1 Processor -- start 2024-04-20T21:21:04.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.585+0000 7f9ea2a00640 1 -- start start 2024-04-20T21:21:04.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.586+0000 7f9ea2a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e800051c0 0x7f9e9c13b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:04.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.586+0000 7f9ea2a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e9c143610 0x7f9e9c1f81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:04.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.586+0000 7f9ea2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e9c15b920 0x7f9e9c15bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:04.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.586+0000 7f9ea2a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9c06e1b0 con 0x7f9e9c15b920 2024-04-20T21:21:04.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.586+0000 7f9ea2a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9e9c06ded0 con 0x7f9e800051c0 2024-04-20T21:21:04.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.586+0000 7f9ea2a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9e9c06f150 con 0x7f9e9c143610 2024-04-20T21:21:04.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.587+0000 7f9ea0a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e9c15b920 0x7f9e9c15bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:04.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.587+0000 7f9ea0a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e9c15b920 0x7f9e9c15bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55866/0 (socket says 172.21.15.86:55866) 2024-04-20T21:21:04.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.587+0000 7f9ea0a00640 1 -- 172.21.15.86:0/3001461015 learned_addr learned my addr 172.21.15.86:0/3001461015 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:04.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.587+0000 7f9e9be00640 1 --2- 172.21.15.86:0/3001461015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e800051c0 0x7f9e9c13b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:04.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.587+0000 7f9e93400640 1 --2- 172.21.15.86:0/3001461015 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e9c143610 0x7f9e9c1f81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:04.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.587+0000 7f9ea0a00640 1 -- 172.21.15.86:0/3001461015 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e9c143610 msgr2=0x7f9e9c1f81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:04.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.587+0000 7f9ea0a00640 1 --2- 172.21.15.86:0/3001461015 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e9c143610 0x7f9e9c1f81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:04.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.587+0000 7f9ea0a00640 1 -- 172.21.15.86:0/3001461015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e800051c0 msgr2=0x7f9e9c13b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:04.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.587+0000 7f9ea0a00640 1 --2- 172.21.15.86:0/3001461015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e800051c0 0x7f9e9c13b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:04.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.587+0000 7f9ea0a00640 1 -- 172.21.15.86:0/3001461015 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e9c13c040 con 0x7f9e9c15b920 2024-04-20T21:21:04.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.587+0000 7f9e9be00640 1 --2- 172.21.15.86:0/3001461015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e800051c0 0x7f9e9c13b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:21:04.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.587+0000 7f9ea0a00640 1 --2- 172.21.15.86:0/3001461015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e9c15b920 0x7f9e9c15bdd0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f9e94000c00 tx=0x7f9e9c0cdab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:04.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.588+0000 7f9e98c00640 1 -- 172.21.15.86:0/3001461015 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e94004bf0 con 0x7f9e9c15b920 2024-04-20T21:21:04.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.588+0000 7f9e98c00640 1 -- 172.21.15.86:0/3001461015 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e94004d90 con 0x7f9e9c15b920 2024-04-20T21:21:04.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.588+0000 7f9e98c00640 1 -- 172.21.15.86:0/3001461015 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e94007270 con 0x7f9e9c15b920 2024-04-20T21:21:04.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.588+0000 7f9ea2a00640 1 -- 172.21.15.86:0/3001461015 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e9c13e990 con 0x7f9e9c15b920 2024-04-20T21:21:04.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.588+0000 7f9ea2a00640 1 -- 172.21.15.86:0/3001461015 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e9c13c320 con 0x7f9e9c15b920 2024-04-20T21:21:04.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.589+0000 7f9ea2a00640 1 -- 172.21.15.86:0/3001461015 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e9c0689e0 con 0x7f9e9c15b920 2024-04-20T21:21:04.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.590+0000 7f9e98c00640 1 -- 172.21.15.86:0/3001461015 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9e94007410 con 0x7f9e9c15b920 2024-04-20T21:21:04.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.591+0000 7f9e98c00640 1 --2- 172.21.15.86:0/3001461015 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9e7806c8f0 0x7f9e7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:04.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.591+0000 7f9e98c00640 1 -- 172.21.15.86:0/3001461015 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4857+0+0 (secure 0 0 0) 0x7f9e940ac830 con 0x7f9e9c15b920 2024-04-20T21:21:04.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.591+0000 7f9e9be00640 1 --2- 172.21.15.86:0/3001461015 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9e7806c8f0 0x7f9e7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:04.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.592+0000 7f9e9be00640 1 --2- 172.21.15.86:0/3001461015 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9e7806c8f0 0x7f9e7806edb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c06de00 tx=0x7f9e8c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:04.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.595+0000 7f9e98c00640 1 -- 172.21.15.86:0/3001461015 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9e94048030 con 0x7f9e9c15b920 2024-04-20T21:21:04.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.795+0000 7f9ea2a00640 1 -- 172.21.15.86:0/3001461015 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f9e9c15ff50 con 0x7f9e9c15b920 2024-04-20T21:21:04.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.796+0000 7f9e98c00640 1 -- 172.21.15.86:0/3001461015 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v55) ==== 84+0+65 (secure 0 0 0) 0x7f9e9c06eb30 con 0x7f9e9c15b920 2024-04-20T21:21:04.796 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:21:04.796 INFO:teuthology.orchestra.run.smithi086.stdout:[ 2024-04-20T21:21:04.797 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:21:04.797 INFO:teuthology.orchestra.run.smithi086.stdout: "poolnum": 1, 2024-04-20T21:21:04.797 INFO:teuthology.orchestra.run.smithi086.stdout: "poolname": ".mgr" 2024-04-20T21:21:04.797 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:21:04.797 INFO:teuthology.orchestra.run.smithi086.stdout:] 2024-04-20T21:21:04.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.799+0000 7f9ea2a00640 1 -- 172.21.15.86:0/3001461015 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9e7806c8f0 msgr2=0x7f9e7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:04.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.799+0000 7f9ea2a00640 1 --2- 172.21.15.86:0/3001461015 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9e7806c8f0 0x7f9e7806edb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c06de00 tx=0x7f9e8c009040 comp rx=0 tx=0).stop 2024-04-20T21:21:04.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.799+0000 7f9ea2a00640 1 -- 172.21.15.86:0/3001461015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e9c15b920 msgr2=0x7f9e9c15bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:04.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.799+0000 7f9ea2a00640 1 --2- 172.21.15.86:0/3001461015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e9c15b920 0x7f9e9c15bdd0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f9e94000c00 tx=0x7f9e9c0cdab0 comp rx=0 tx=0).stop 2024-04-20T21:21:04.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.799+0000 7f9ea2a00640 1 -- 172.21.15.86:0/3001461015 shutdown_connections 2024-04-20T21:21:04.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.799+0000 7f9ea2a00640 1 --2- 172.21.15.86:0/3001461015 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9e7806c8f0 0x7f9e7806edb0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:04.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.799+0000 7f9ea2a00640 1 --2- 172.21.15.86:0/3001461015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e9c15b920 0x7f9e9c15bdd0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:04.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.799+0000 7f9ea2a00640 1 --2- 172.21.15.86:0/3001461015 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e9c143610 0x7f9e9c1f81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:04.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.799+0000 7f9ea2a00640 1 --2- 172.21.15.86:0/3001461015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e800051c0 0x7f9e9c13b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:04.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.799+0000 7f9ea2a00640 1 -- 172.21.15.86:0/3001461015 >> 172.21.15.86:0/3001461015 conn(0x7f9e9c0b2360 msgr2=0x7f9e9c0b4df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:04.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.799+0000 7f9ea2a00640 1 -- 172.21.15.86:0/3001461015 shutdown_connections 2024-04-20T21:21:04.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:04.799+0000 7f9ea2a00640 1 -- 172.21.15.86:0/3001461015 wait complete. 2024-04-20T21:21:04.817 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-20T21:21:04.817 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-20T21:21:05.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.013+0000 7f04a6e00640 1 Processor -- start 2024-04-20T21:21:05.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.013+0000 7f04a6e00640 1 -- start start 2024-04-20T21:21:05.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.014+0000 7f04a6e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04a00af870 0x7f04a006dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:05.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.014+0000 7f04a6e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04a015cbe0 con 0x7f04a0153590 2024-04-20T21:21:05.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.014+0000 7f04a6e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04a015cea0 con 0x7f04a00d0d20 2024-04-20T21:21:05.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.014+0000 7f04a6e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f04a0153970 con 0x7f04a00af870 2024-04-20T21:21:05.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.015+0000 7f04a5a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f04a0153590 0x7f04a00d0610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54262/0 (socket says 172.21.15.86:54262) 2024-04-20T21:21:05.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.015+0000 7f04a5a00640 1 -- 172.21.15.86:0/3781053084 learned_addr learned my addr 172.21.15.86:0/3781053084 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:21:05.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.015+0000 7f04a6400640 1 --2- 172.21.15.86:0/3781053084 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04a00af870 0x7f04a006dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:05.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.015+0000 7f049fe00640 1 -- 172.21.15.86:0/3781053084 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3009856124 0 0) 0x7f04a015cbe0 con 0x7f04a0153590 2024-04-20T21:21:05.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.015+0000 7f049fe00640 1 -- 172.21.15.86:0/3781053084 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0488003520 con 0x7f04a0153590 2024-04-20T21:21:05.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.015+0000 7f049fe00640 1 -- 172.21.15.86:0/3781053084 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4026975663 0 0) 0x7f0488003520 con 0x7f04a0153590 2024-04-20T21:21:05.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.015+0000 7f049fe00640 1 -- 172.21.15.86:0/3781053084 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04a00af870 msgr2=0x7f04a006dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:05.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.015+0000 7f049fe00640 1 --2- 172.21.15.86:0/3781053084 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04a00af870 0x7f04a006dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:05.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.015+0000 7f049fe00640 1 -- 172.21.15.86:0/3781053084 >> v1:172.21.15.144:6789/0 conn(0x7f04a00d0d20 legacy=0x7f04a00af160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:05.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.015+0000 7f049fe00640 1 -- 172.21.15.86:0/3781053084 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04a015d180 con 0x7f04a0153590 2024-04-20T21:21:05.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.015+0000 7f049fe00640 1 -- 172.21.15.86:0/3781053084 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f0494001670 con 0x7f04a0153590 2024-04-20T21:21:05.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.015+0000 7f049fe00640 1 -- 172.21.15.86:0/3781053084 >> v1:172.21.15.86:6789/0 conn(0x7f04a0153590 legacy=0x7f04a00d0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:05.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.015+0000 7f049fe00640 1 --2- 172.21.15.86:0/3781053084 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04a00af870 0x7f04880039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:05.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.015+0000 7f049fe00640 1 --2- 172.21.15.86:0/3781053084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0488004010 0x7f0488004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:05.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.015+0000 7f049fe00640 1 --2- 172.21.15.86:0/3781053084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0488004d40 0x7f04880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:05.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.016+0000 7f049fe00640 1 -- 172.21.15.86:0/3781053084 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f04a0153970 con 0x7f0488004d40 2024-04-20T21:21:05.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.016+0000 7f049fe00640 1 -- 172.21.15.86:0/3781053084 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f048800bb80 con 0x7f0488004010 2024-04-20T21:21:05.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.016+0000 7f049fe00640 1 -- 172.21.15.86:0/3781053084 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f048800bd80 con 0x7f04a00af870 2024-04-20T21:21:05.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.016+0000 7f04a6400640 1 --2- 172.21.15.86:0/3781053084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0488004d40 0x7f04880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:05.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.016+0000 7f04a5000640 1 --2- 172.21.15.86:0/3781053084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0488004010 0x7f0488004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:05.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.016+0000 7f04a6400640 1 -- 172.21.15.86:0/3781053084 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04a00af870 msgr2=0x7f04880039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:05.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.016+0000 7f04a6400640 1 --2- 172.21.15.86:0/3781053084 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04a00af870 0x7f04880039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:05.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.016+0000 7f04a6400640 1 -- 172.21.15.86:0/3781053084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0488004010 msgr2=0x7f0488004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:05.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.016+0000 7f04a6400640 1 --2- 172.21.15.86:0/3781053084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0488004010 0x7f0488004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:05.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.016+0000 7f04a6400640 1 -- 172.21.15.86:0/3781053084 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0488003520 con 0x7f0488004d40 2024-04-20T21:21:05.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.017+0000 7f04a6400640 1 --2- 172.21.15.86:0/3781053084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0488004d40 0x7f04880051c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f0498007360 tx=0x7f0488005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5dc2727f9b0356b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:05.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.017+0000 7f049fe00640 1 -- 172.21.15.86:0/3781053084 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0498039070 con 0x7f0488004d40 2024-04-20T21:21:05.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.017+0000 7f049fe00640 1 -- 172.21.15.86:0/3781053084 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f049802ccf0 con 0x7f0488004d40 2024-04-20T21:21:05.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.017+0000 7f049fe00640 1 -- 172.21.15.86:0/3781053084 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0498009be0 con 0x7f0488004d40 2024-04-20T21:21:05.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.018+0000 7f04a6e00640 1 -- 172.21.15.86:0/3781053084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0488004d40 msgr2=0x7f04880051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:05.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.018+0000 7f04a6e00640 1 --2- 172.21.15.86:0/3781053084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0488004d40 0x7f04880051c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f0498007360 tx=0x7f0488005830 comp rx=0 tx=0).stop 2024-04-20T21:21:05.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.018+0000 7f04a6400640 1 -- 172.21.15.86:0/3781053084 reap_dead start 2024-04-20T21:21:05.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.018+0000 7f04a6e00640 1 -- 172.21.15.86:0/3781053084 shutdown_connections 2024-04-20T21:21:05.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.018+0000 7f04a6e00640 1 -- 172.21.15.86:0/3781053084 >> 172.21.15.86:0/3781053084 conn(0x7f04a00b2180 msgr2=0x7f04a00b2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:05.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.018+0000 7f04a6e00640 1 -- 172.21.15.86:0/3781053084 shutdown_connections 2024-04-20T21:21:05.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.018+0000 7f04a6e00640 1 -- 172.21.15.86:0/3781053084 wait complete. 2024-04-20T21:21:05.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.020+0000 7f04a6e00640 1 Processor -- start 2024-04-20T21:21:05.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.020+0000 7f04a6e00640 1 -- start start 2024-04-20T21:21:05.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.021+0000 7f04a6e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0488004d40 0x7f04a014d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:05.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.021+0000 7f04a6e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04a014fd00 0x7f04a00afd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:05.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.021+0000 7f04a6e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04a0150180 0x7f04a00b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:05.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.021+0000 7f04a6e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f04a0071ad0 con 0x7f0488004d40 2024-04-20T21:21:05.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.021+0000 7f04a6e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f04a015d740 con 0x7f04a0150180 2024-04-20T21:21:05.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.021+0000 7f04a6e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f04a0071c50 con 0x7f04a014fd00 2024-04-20T21:21:05.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.021+0000 7f04a5a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0488004d40 0x7f04a014d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:05.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.021+0000 7f04a5a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0488004d40 0x7f04a014d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55880/0 (socket says 172.21.15.86:55880) 2024-04-20T21:21:05.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.021+0000 7f04a5a00640 1 -- 172.21.15.86:0/989314568 learned_addr learned my addr 172.21.15.86:0/989314568 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:05.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.021+0000 7f04a5a00640 1 -- 172.21.15.86:0/989314568 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04a014fd00 msgr2=0x7f04a00afd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:05.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.021+0000 7f04a5a00640 1 --2- 172.21.15.86:0/989314568 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04a014fd00 0x7f04a00afd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:05.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.021+0000 7f04a5a00640 1 -- 172.21.15.86:0/989314568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04a0150180 msgr2=0x7f04a00b0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:05.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.021+0000 7f04a5a00640 1 --2- 172.21.15.86:0/989314568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04a0150180 0x7f04a00b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:05.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.021+0000 7f04a5a00640 1 -- 172.21.15.86:0/989314568 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04a00b4ac0 con 0x7f0488004d40 2024-04-20T21:21:05.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.022+0000 7f04a5a00640 1 --2- 172.21.15.86:0/989314568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0488004d40 0x7f04a014d980 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f049400c840 tx=0x7f049400cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:05.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.022+0000 7f049d600640 1 -- 172.21.15.86:0/989314568 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0494018070 con 0x7f0488004d40 2024-04-20T21:21:05.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.022+0000 7f049d600640 1 -- 172.21.15.86:0/989314568 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04940074e0 con 0x7f0488004d40 2024-04-20T21:21:05.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.022+0000 7f04a6e00640 1 -- 172.21.15.86:0/989314568 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04a0155c40 con 0x7f0488004d40 2024-04-20T21:21:05.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.022+0000 7f049d600640 1 -- 172.21.15.86:0/989314568 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0494004a20 con 0x7f0488004d40 2024-04-20T21:21:05.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.022+0000 7f04a6e00640 1 -- 172.21.15.86:0/989314568 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04a0156200 con 0x7f0488004d40 2024-04-20T21:21:05.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.023+0000 7f04a6e00640 1 -- 172.21.15.86:0/989314568 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04a004fcf0 con 0x7f0488004d40 2024-04-20T21:21:05.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.024+0000 7f049d600640 1 -- 172.21.15.86:0/989314568 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f049401b070 con 0x7f0488004d40 2024-04-20T21:21:05.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.025+0000 7f049d600640 1 --2- 172.21.15.86:0/989314568 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f048006c920 0x7f048006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:05.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.025+0000 7f049d600640 1 -- 172.21.15.86:0/989314568 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4857+0+0 (secure 0 0 0) 0x7f049408aca0 con 0x7f0488004d40 2024-04-20T21:21:05.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.025+0000 7f04a5000640 1 --2- 172.21.15.86:0/989314568 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f048006c920 0x7f048006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:05.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.026+0000 7f04a5000640 1 --2- 172.21.15.86:0/989314568 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f048006c920 0x7f048006ede0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f04a00ae2e0 tx=0x7f048c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:05.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.029+0000 7f049d600640 1 -- 172.21.15.86:0/989314568 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f04940594b0 con 0x7f0488004d40 2024-04-20T21:21:05.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:05.229+0000 7f04a6e00640 1 -- 172.21.15.86:0/989314568 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f04a00add40 con 0x7f0488004d40 2024-04-20T21:21:06.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.019+0000 7f049d600640 1 -- 172.21.15.86:0/989314568 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v56) ==== 138+0+0 (secure 0 0 0) 0x7f0494058db0 con 0x7f0488004d40 2024-04-20T21:21:06.020 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' created 2024-04-20T21:21:06.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.023+0000 7f04a6e00640 1 -- 172.21.15.86:0/989314568 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f048006c920 msgr2=0x7f048006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:06.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.023+0000 7f04a6e00640 1 --2- 172.21.15.86:0/989314568 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f048006c920 0x7f048006ede0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f04a00ae2e0 tx=0x7f048c009040 comp rx=0 tx=0).stop 2024-04-20T21:21:06.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.023+0000 7f04a6e00640 1 -- 172.21.15.86:0/989314568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0488004d40 msgr2=0x7f04a014d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:06.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.023+0000 7f04a6e00640 1 --2- 172.21.15.86:0/989314568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0488004d40 0x7f04a014d980 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f049400c840 tx=0x7f049400cd00 comp rx=0 tx=0).stop 2024-04-20T21:21:06.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.023+0000 7f04a6e00640 1 -- 172.21.15.86:0/989314568 shutdown_connections 2024-04-20T21:21:06.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.023+0000 7f04a6e00640 1 --2- 172.21.15.86:0/989314568 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f048006c920 0x7f048006ede0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:06.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.023+0000 7f04a6e00640 1 --2- 172.21.15.86:0/989314568 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f04a0150180 0x7f04a00b0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:06.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.023+0000 7f04a6e00640 1 --2- 172.21.15.86:0/989314568 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f04a014fd00 0x7f04a00afd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:06.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.023+0000 7f04a6e00640 1 --2- 172.21.15.86:0/989314568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0488004d40 0x7f04a014d980 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:06.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.023+0000 7f04a6e00640 1 -- 172.21.15.86:0/989314568 >> 172.21.15.86:0/989314568 conn(0x7f04a00b2180 msgr2=0x7f04a015db20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:06.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.024+0000 7f04a6e00640 1 -- 172.21.15.86:0/989314568 shutdown_connections 2024-04-20T21:21:06.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.024+0000 7f04a6e00640 1 -- 172.21.15.86:0/989314568 wait complete. 2024-04-20T21:21:06.050 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-20T21:21:06.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.241+0000 7f5882600640 1 Processor -- start 2024-04-20T21:21:06.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.242+0000 7f5882600640 1 -- start start 2024-04-20T21:21:06.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.243+0000 7f5882600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f587c15ee00 0x7f587c1634c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:06.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.243+0000 7f5882600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f587c0d1260 0x7f587c0d16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:06.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.243+0000 7f5882600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f587c0cff90 con 0x7f587c0cf830 2024-04-20T21:21:06.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.243+0000 7f5882600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f587c0d0550 con 0x7f587c0d1260 2024-04-20T21:21:06.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.243+0000 7f5882600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f587c0d0270 con 0x7f587c15ee00 2024-04-20T21:21:06.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.243+0000 7f587aa00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f587c0cf830 0x7f587c15fe40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:48428/0 (socket says 172.21.15.86:48428) 2024-04-20T21:21:06.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.243+0000 7f587aa00640 1 -- 172.21.15.86:0/3909980602 learned_addr learned my addr 172.21.15.86:0/3909980602 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:21:06.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.244+0000 7f587b400640 1 --2- 172.21.15.86:0/3909980602 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f587c15ee00 0x7f587c1634c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:06.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.244+0000 7f587a000640 1 -- 172.21.15.86:0/3909980602 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3480250641 0 0) 0x7f587c0cff90 con 0x7f587c0cf830 2024-04-20T21:21:06.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.244+0000 7f587be00640 1 --2- 172.21.15.86:0/3909980602 >> v2:172.21.15.144:3300/0 conn(0x7f587c0d1260 0x7f587c0d16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:06.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.244+0000 7f587a000640 1 -- 172.21.15.86:0/3909980602 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5860003520 con 0x7f587c0cf830 2024-04-20T21:21:06.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.244+0000 7f587a000640 1 -- 172.21.15.86:0/3909980602 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3973726695 0 0) 0x7f5860003520 con 0x7f587c0cf830 2024-04-20T21:21:06.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.244+0000 7f587a000640 1 -- 172.21.15.86:0/3909980602 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f587c15ee00 msgr2=0x7f587c1634c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:06.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.244+0000 7f587a000640 1 --2- 172.21.15.86:0/3909980602 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f587c15ee00 0x7f587c1634c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:06.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.244+0000 7f587a000640 1 -- 172.21.15.86:0/3909980602 >> v2:172.21.15.144:3300/0 conn(0x7f587c0d1260 msgr2=0x7f587c0d16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:06.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.244+0000 7f587a000640 1 --2- 172.21.15.86:0/3909980602 >> v2:172.21.15.144:3300/0 conn(0x7f587c0d1260 0x7f587c0d16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:06.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.244+0000 7f587a000640 1 -- 172.21.15.86:0/3909980602 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f587c15f640 con 0x7f587c0cf830 2024-04-20T21:21:06.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.244+0000 7f587a000640 1 -- 172.21.15.86:0/3909980602 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f5864001670 con 0x7f587c0cf830 2024-04-20T21:21:06.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.244+0000 7f587be00640 1 --2- 172.21.15.86:0/3909980602 >> v2:172.21.15.144:3300/0 conn(0x7f587c0d1260 0x7f587c0d16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:21:06.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.244+0000 7f587a000640 1 -- 172.21.15.86:0/3909980602 >> v1:172.21.15.86:6789/0 conn(0x7f587c0cf830 legacy=0x7f587c15fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:06.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.244+0000 7f587a000640 1 --2- 172.21.15.86:0/3909980602 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f587c15ee00 0x7f58600039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:06.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.244+0000 7f587a000640 1 --2- 172.21.15.86:0/3909980602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5860003f30 0x7f5860004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:06.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.244+0000 7f587a000640 1 --2- 172.21.15.86:0/3909980602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5860004d40 0x7f58600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:06.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.244+0000 7f587a000640 1 -- 172.21.15.86:0/3909980602 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f587c0d0550 con 0x7f5860003f30 2024-04-20T21:21:06.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.245+0000 7f587a000640 1 -- 172.21.15.86:0/3909980602 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f587c0d0270 con 0x7f5860004d40 2024-04-20T21:21:06.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.245+0000 7f587a000640 1 -- 172.21.15.86:0/3909980602 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f586000ba10 con 0x7f587c15ee00 2024-04-20T21:21:06.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.245+0000 7f587aa00640 1 --2- 172.21.15.86:0/3909980602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5860003f30 0x7f5860004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:06.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.245+0000 7f587be00640 1 --2- 172.21.15.86:0/3909980602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5860004d40 0x7f58600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:06.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.245+0000 7f587aa00640 1 -- 172.21.15.86:0/3909980602 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f587c15ee00 msgr2=0x7f58600039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:06.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.245+0000 7f587aa00640 1 --2- 172.21.15.86:0/3909980602 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f587c15ee00 0x7f58600039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:06.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.245+0000 7f587aa00640 1 -- 172.21.15.86:0/3909980602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5860004d40 msgr2=0x7f58600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:06.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.245+0000 7f587aa00640 1 --2- 172.21.15.86:0/3909980602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5860004d40 0x7f58600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:06.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.245+0000 7f587aa00640 1 -- 172.21.15.86:0/3909980602 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5860003520 con 0x7f5860003f30 2024-04-20T21:21:06.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.245+0000 7f587be00640 1 --2- 172.21.15.86:0/3909980602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5860004d40 0x7f58600051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:21:06.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.246+0000 7f587aa00640 1 --2- 172.21.15.86:0/3909980602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5860003f30 0x7f5860004800 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f586400c290 tx=0x7f5860005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=368bb52d421962e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:06.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.247+0000 7f587a000640 1 -- 172.21.15.86:0/3909980602 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5864007d50 con 0x7f5860003f30 2024-04-20T21:21:06.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.247+0000 7f587a000640 1 -- 172.21.15.86:0/3909980602 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5864030070 con 0x7f5860003f30 2024-04-20T21:21:06.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.247+0000 7f587a000640 1 -- 172.21.15.86:0/3909980602 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586402cdc0 con 0x7f5860003f30 2024-04-20T21:21:06.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.247+0000 7f5882600640 1 -- 172.21.15.86:0/3909980602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5860003f30 msgr2=0x7f5860004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:06.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.247+0000 7f5882600640 1 --2- 172.21.15.86:0/3909980602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5860003f30 0x7f5860004800 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f586400c290 tx=0x7f5860005830 comp rx=0 tx=0).stop 2024-04-20T21:21:06.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.247+0000 7f587be00640 1 -- 172.21.15.86:0/3909980602 reap_dead start 2024-04-20T21:21:06.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.247+0000 7f5882600640 1 -- 172.21.15.86:0/3909980602 shutdown_connections 2024-04-20T21:21:06.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.247+0000 7f5882600640 1 -- 172.21.15.86:0/3909980602 >> 172.21.15.86:0/3909980602 conn(0x7f587c06cc20 msgr2=0x7f587c06d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:06.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.247+0000 7f5882600640 1 -- 172.21.15.86:0/3909980602 shutdown_connections 2024-04-20T21:21:06.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.247+0000 7f5882600640 1 -- 172.21.15.86:0/3909980602 wait complete. 2024-04-20T21:21:06.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.249+0000 7f5882600640 1 Processor -- start 2024-04-20T21:21:06.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.249+0000 7f5882600640 1 -- start start 2024-04-20T21:21:06.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.250+0000 7f5882600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5860003f30 0x7f587c1525c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:06.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.250+0000 7f5882600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f587c0cbae0 0x7f587c1558c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:06.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.250+0000 7f5882600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f587c0cd380 0x7f587c0cd800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:06.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.250+0000 7f5882600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f587c15f360 con 0x7f587c0cd380 2024-04-20T21:21:06.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.250+0000 7f5882600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f587c15f1e0 con 0x7f5860003f30 2024-04-20T21:21:06.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.250+0000 7f5882600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f587c1634c0 con 0x7f587c0cbae0 2024-04-20T21:21:06.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.251+0000 7f587be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f587c0cd380 0x7f587c0cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:06.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.251+0000 7f587be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f587c0cd380 0x7f587c0cd800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47674/0 (socket says 172.21.15.86:47674) 2024-04-20T21:21:06.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.251+0000 7f587be00640 1 -- 172.21.15.86:0/464844625 learned_addr learned my addr 172.21.15.86:0/464844625 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:06.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.251+0000 7f587b400640 1 --2- 172.21.15.86:0/464844625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5860003f30 0x7f587c1525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:06.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.251+0000 7f587aa00640 1 --2- 172.21.15.86:0/464844625 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f587c0cbae0 0x7f587c1558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:06.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.251+0000 7f587be00640 1 -- 172.21.15.86:0/464844625 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f587c0cbae0 msgr2=0x7f587c1558c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:06.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.251+0000 7f587be00640 1 --2- 172.21.15.86:0/464844625 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f587c0cbae0 0x7f587c1558c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:06.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.251+0000 7f587be00640 1 -- 172.21.15.86:0/464844625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5860003f30 msgr2=0x7f587c1525c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:06.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.251+0000 7f587be00640 1 --2- 172.21.15.86:0/464844625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5860003f30 0x7f587c1525c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:06.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.251+0000 7f587be00640 1 -- 172.21.15.86:0/464844625 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f587c158ff0 con 0x7f587c0cd380 2024-04-20T21:21:06.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.251+0000 7f587be00640 1 --2- 172.21.15.86:0/464844625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f587c0cd380 0x7f587c0cd800 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f587c0d2990 tx=0x7f587400f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:06.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.252+0000 7f5873400640 1 -- 172.21.15.86:0/464844625 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587400fea0 con 0x7f587c0cd380 2024-04-20T21:21:06.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.252+0000 7f5873400640 1 -- 172.21.15.86:0/464844625 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5874019070 con 0x7f587c0cd380 2024-04-20T21:21:06.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.252+0000 7f5873400640 1 -- 172.21.15.86:0/464844625 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5874004490 con 0x7f587c0cd380 2024-04-20T21:21:06.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.252+0000 7f5882600640 1 -- 172.21.15.86:0/464844625 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f587c155100 con 0x7f587c0cd380 2024-04-20T21:21:06.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.252+0000 7f5882600640 1 -- 172.21.15.86:0/464844625 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f587c158d10 con 0x7f587c0cd380 2024-04-20T21:21:06.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.253+0000 7f5882600640 1 -- 172.21.15.86:0/464844625 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f587c0689e0 con 0x7f587c0cd380 2024-04-20T21:21:06.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.254+0000 7f5873400640 1 -- 172.21.15.86:0/464844625 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5874012070 con 0x7f587c0cd380 2024-04-20T21:21:06.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.255+0000 7f5873400640 1 --2- 172.21.15.86:0/464844625 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f585806c920 0x7f585806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:06.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.255+0000 7f5873400640 1 -- 172.21.15.86:0/464844625 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5227+0+0 (secure 0 0 0) 0x7f587408b390 con 0x7f587c0cd380 2024-04-20T21:21:06.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.256+0000 7f587b400640 1 --2- 172.21.15.86:0/464844625 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f585806c920 0x7f585806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:06.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.256+0000 7f587b400640 1 --2- 172.21.15.86:0/464844625 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f585806c920 0x7f585806ede0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f586c009f90 tx=0x7f586c00d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:06.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.260+0000 7f5873400640 1 -- 172.21.15.86:0/464844625 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5874059990 con 0x7f587c0cd380 2024-04-20T21:21:06.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:06.461+0000 7f5882600640 1 -- 172.21.15.86:0/464844625 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f587c0d29c0 con 0x7f587c0cd380 2024-04-20T21:21:07.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.026+0000 7f5873400640 1 -- 172.21.15.86:0/464844625 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v57) ==== 170+0+0 (secure 0 0 0) 0x7f5874089e20 con 0x7f587c0cd380 2024-04-20T21:21:07.026 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' created 2024-04-20T21:21:07.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.029+0000 7f5882600640 1 -- 172.21.15.86:0/464844625 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f585806c920 msgr2=0x7f585806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:07.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.029+0000 7f5882600640 1 --2- 172.21.15.86:0/464844625 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f585806c920 0x7f585806ede0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f586c009f90 tx=0x7f586c00d040 comp rx=0 tx=0).stop 2024-04-20T21:21:07.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.030+0000 7f5882600640 1 -- 172.21.15.86:0/464844625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f587c0cd380 msgr2=0x7f587c0cd800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:07.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.030+0000 7f5882600640 1 --2- 172.21.15.86:0/464844625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f587c0cd380 0x7f587c0cd800 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f587c0d2990 tx=0x7f587400f670 comp rx=0 tx=0).stop 2024-04-20T21:21:07.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.030+0000 7f5882600640 1 -- 172.21.15.86:0/464844625 shutdown_connections 2024-04-20T21:21:07.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.030+0000 7f5882600640 1 --2- 172.21.15.86:0/464844625 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f585806c920 0x7f585806ede0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:07.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.030+0000 7f5882600640 1 --2- 172.21.15.86:0/464844625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f587c0cd380 0x7f587c0cd800 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:07.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.030+0000 7f5882600640 1 --2- 172.21.15.86:0/464844625 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f587c0cbae0 0x7f587c1558c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:07.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.030+0000 7f5882600640 1 --2- 172.21.15.86:0/464844625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5860003f30 0x7f587c1525c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:07.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.030+0000 7f5882600640 1 -- 172.21.15.86:0/464844625 >> 172.21.15.86:0/464844625 conn(0x7f587c06cc20 msgr2=0x7f587c0b8180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:07.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.030+0000 7f5882600640 1 -- 172.21.15.86:0/464844625 shutdown_connections 2024-04-20T21:21:07.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.030+0000 7f5882600640 1 -- 172.21.15.86:0/464844625 wait complete. 2024-04-20T21:21:07.064 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-20T21:21:07.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.240+0000 7f3903a00640 1 Processor -- start 2024-04-20T21:21:07.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.240+0000 7f3903a00640 1 -- start start 2024-04-20T21:21:07.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.242+0000 7f3903a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f38fc06c090 0x7f38fc0cd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:07.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.242+0000 7f3903a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f38fc0b3410 0x7f38fc0cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:07.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.242+0000 7f3903a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38fc0b3910 con 0x7f38fc162ed0 2024-04-20T21:21:07.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.242+0000 7f3903a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f38fc0b3bf0 con 0x7f38fc0b3410 2024-04-20T21:21:07.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.242+0000 7f3903a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f38fc0b3ed0 con 0x7f38fc06c090 2024-04-20T21:21:07.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.242+0000 7f3901000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f38fc162ed0 0x7f38fc06b980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:48430/0 (socket says 172.21.15.86:48430) 2024-04-20T21:21:07.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.242+0000 7f3901a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f38fc0b3410 0x7f38fc0cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:07.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.242+0000 7f3901000640 1 -- 172.21.15.86:0/3232907273 learned_addr learned my addr 172.21.15.86:0/3232907273 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:21:07.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.243+0000 7f38f3e00640 1 --2- 172.21.15.86:0/3232907273 >> v2:172.21.15.144:3300/0 conn(0x7f38fc06c090 0x7f38fc0cd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:07.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.243+0000 7f3901a00640 1 -- 172.21.15.86:0/3232907273 >> v2:172.21.15.144:3300/0 conn(0x7f38fc06c090 msgr2=0x7f38fc0cd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:07.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.243+0000 7f3901a00640 1 --2- 172.21.15.86:0/3232907273 >> v2:172.21.15.144:3300/0 conn(0x7f38fc06c090 0x7f38fc0cd360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:07.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.243+0000 7f3901a00640 1 -- 172.21.15.86:0/3232907273 >> v1:172.21.15.86:6789/0 conn(0x7f38fc162ed0 legacy=0x7f38fc06b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:07.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.243+0000 7f3901a00640 1 -- 172.21.15.86:0/3232907273 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38fc0b41b0 con 0x7f38fc0b3410 2024-04-20T21:21:07.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.243+0000 7f38fbe00640 1 -- 172.21.15.86:0/3232907273 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2236756910 0 0) 0x7f38fc0b3910 con 0x7f38fc162ed0 2024-04-20T21:21:07.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.244+0000 7f3901a00640 1 --2- 172.21.15.86:0/3232907273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38fc0b3410 0x7f38fc0cd8a0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f38fc071b70 tx=0x7f38f402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8de0d1658e25d4ea server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:07.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.244+0000 7f38fbe00640 1 -- 172.21.15.86:0/3232907273 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f4039070 con 0x7f38fc0b3410 2024-04-20T21:21:07.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.244+0000 7f38fbe00640 1 -- 172.21.15.86:0/3232907273 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38f402cc80 con 0x7f38fc0b3410 2024-04-20T21:21:07.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.244+0000 7f38fbe00640 1 -- 172.21.15.86:0/3232907273 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f4034660 con 0x7f38fc0b3410 2024-04-20T21:21:07.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.245+0000 7f3903a00640 1 -- 172.21.15.86:0/3232907273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38fc0b3410 msgr2=0x7f38fc0cd8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:07.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.245+0000 7f3903a00640 1 --2- 172.21.15.86:0/3232907273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38fc0b3410 0x7f38fc0cd8a0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f38fc071b70 tx=0x7f38f402bed0 comp rx=0 tx=0).stop 2024-04-20T21:21:07.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.245+0000 7f3903a00640 1 -- 172.21.15.86:0/3232907273 shutdown_connections 2024-04-20T21:21:07.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.245+0000 7f3903a00640 1 --2- 172.21.15.86:0/3232907273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38fc0b3410 0x7f38fc0cd8a0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:07.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.245+0000 7f3903a00640 1 --2- 172.21.15.86:0/3232907273 >> v2:172.21.15.144:3300/0 conn(0x7f38fc06c090 0x7f38fc0cd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:07.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.245+0000 7f3903a00640 1 -- 172.21.15.86:0/3232907273 >> 172.21.15.86:0/3232907273 conn(0x7f38fc0b2110 msgr2=0x7f38fc0b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:07.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.245+0000 7f3903a00640 1 -- 172.21.15.86:0/3232907273 shutdown_connections 2024-04-20T21:21:07.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.245+0000 7f3903a00640 1 -- 172.21.15.86:0/3232907273 wait complete. 2024-04-20T21:21:07.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.247+0000 7f3903a00640 1 Processor -- start 2024-04-20T21:21:07.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.247+0000 7f3903a00640 1 -- start start 2024-04-20T21:21:07.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.248+0000 7f3903a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38fc06c090 0x7f38fc147fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:07.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.248+0000 7f3903a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38fc0b3410 0x7f38fc15df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:07.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.248+0000 7f3903a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38fc162ed0 0x7f38fc15e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:07.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.248+0000 7f3903a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f38fc06fca0 con 0x7f38fc06c090 2024-04-20T21:21:07.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.248+0000 7f3903a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f38fc06fb20 con 0x7f38fc162ed0 2024-04-20T21:21:07.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.248+0000 7f3903a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f38fc06e160 con 0x7f38fc0b3410 2024-04-20T21:21:07.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.248+0000 7f3901000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38fc06c090 0x7f38fc147fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:07.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.248+0000 7f3901000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38fc06c090 0x7f38fc147fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47696/0 (socket says 172.21.15.86:47696) 2024-04-20T21:21:07.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.248+0000 7f3901000640 1 -- 172.21.15.86:0/4115757458 learned_addr learned my addr 172.21.15.86:0/4115757458 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:07.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.249+0000 7f3901a00640 1 --2- 172.21.15.86:0/4115757458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38fc162ed0 0x7f38fc15e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:07.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.249+0000 7f3901000640 1 -- 172.21.15.86:0/4115757458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38fc0b3410 msgr2=0x7f38fc15df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:07.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.249+0000 7f3901000640 1 --2- 172.21.15.86:0/4115757458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38fc0b3410 0x7f38fc15df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:07.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.249+0000 7f3901000640 1 -- 172.21.15.86:0/4115757458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38fc162ed0 msgr2=0x7f38fc15e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:07.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.249+0000 7f3901000640 1 --2- 172.21.15.86:0/4115757458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38fc162ed0 0x7f38fc15e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:07.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.249+0000 7f3901000640 1 -- 172.21.15.86:0/4115757458 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38fc15f410 con 0x7f38fc06c090 2024-04-20T21:21:07.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.249+0000 7f3901000640 1 --2- 172.21.15.86:0/4115757458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38fc06c090 0x7f38fc147fd0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f38e400c430 tx=0x7f38e400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:07.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.249+0000 7f38f9600640 1 -- 172.21.15.86:0/4115757458 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e40074e0 con 0x7f38fc06c090 2024-04-20T21:21:07.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.249+0000 7f38f9600640 1 -- 172.21.15.86:0/4115757458 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38e40047e0 con 0x7f38fc06c090 2024-04-20T21:21:07.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.249+0000 7f3903a00640 1 -- 172.21.15.86:0/4115757458 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38fc148510 con 0x7f38fc06c090 2024-04-20T21:21:07.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.249+0000 7f38f9600640 1 -- 172.21.15.86:0/4115757458 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e4004bd0 con 0x7f38fc06c090 2024-04-20T21:21:07.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.249+0000 7f3903a00640 1 -- 172.21.15.86:0/4115757458 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38fc15b080 con 0x7f38fc06c090 2024-04-20T21:21:07.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.251+0000 7f3903a00640 1 -- 172.21.15.86:0/4115757458 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38fc0689e0 con 0x7f38fc06c090 2024-04-20T21:21:07.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.252+0000 7f38f9600640 1 -- 172.21.15.86:0/4115757458 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f38e4011070 con 0x7f38fc06c090 2024-04-20T21:21:07.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.253+0000 7f38f9600640 1 --2- 172.21.15.86:0/4115757458 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f38d806c8f0 0x7f38d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:07.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.253+0000 7f38f9600640 1 -- 172.21.15.86:0/4115757458 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5609+0+0 (secure 0 0 0) 0x7f38e408b7a0 con 0x7f38fc06c090 2024-04-20T21:21:07.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.253+0000 7f38f3e00640 1 --2- 172.21.15.86:0/4115757458 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f38d806c8f0 0x7f38d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:07.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.254+0000 7f38f3e00640 1 --2- 172.21.15.86:0/4115757458 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f38d806c8f0 0x7f38d806edb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f38fc069210 tx=0x7f38ec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:07.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.258+0000 7f38f9600640 1 -- 172.21.15.86:0/4115757458 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f38e400e280 con 0x7f38fc06c090 2024-04-20T21:21:07.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:07.500+0000 7f3903a00640 1 -- 172.21.15.86:0/4115757458 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f38fc15d980 con 0x7f38fc06c090 2024-04-20T21:21:08.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.075+0000 7f38f9600640 1 -- 172.21.15.86:0/4115757458 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v28) ==== 172+0+0 (secure 0 0 0) 0x7f38e4059b90 con 0x7f38fc06c090 2024-04-20T21:21:08.078 INFO:teuthology.orchestra.run.smithi086.stderr:new fs with metadata pool 8 and data pool 9 2024-04-20T21:21:08.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.082+0000 7f3903a00640 1 -- 172.21.15.86:0/4115757458 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f38d806c8f0 msgr2=0x7f38d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:08.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.082+0000 7f3903a00640 1 --2- 172.21.15.86:0/4115757458 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f38d806c8f0 0x7f38d806edb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f38fc069210 tx=0x7f38ec008040 comp rx=0 tx=0).stop 2024-04-20T21:21:08.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.082+0000 7f3903a00640 1 -- 172.21.15.86:0/4115757458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38fc06c090 msgr2=0x7f38fc147fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:08.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.082+0000 7f3903a00640 1 --2- 172.21.15.86:0/4115757458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38fc06c090 0x7f38fc147fd0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f38e400c430 tx=0x7f38e400c8f0 comp rx=0 tx=0).stop 2024-04-20T21:21:08.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.084+0000 7f3903a00640 1 -- 172.21.15.86:0/4115757458 shutdown_connections 2024-04-20T21:21:08.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.084+0000 7f3903a00640 1 --2- 172.21.15.86:0/4115757458 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f38d806c8f0 0x7f38d806edb0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:08.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.084+0000 7f3903a00640 1 --2- 172.21.15.86:0/4115757458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38fc162ed0 0x7f38fc15e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:08.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.084+0000 7f3903a00640 1 --2- 172.21.15.86:0/4115757458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38fc0b3410 0x7f38fc15df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:08.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.084+0000 7f3903a00640 1 --2- 172.21.15.86:0/4115757458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38fc06c090 0x7f38fc147fd0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:08.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.084+0000 7f3903a00640 1 -- 172.21.15.86:0/4115757458 >> 172.21.15.86:0/4115757458 conn(0x7f38fc0b2110 msgr2=0x7f38fc161cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:08.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.085+0000 7f3903a00640 1 -- 172.21.15.86:0/4115757458 shutdown_connections 2024-04-20T21:21:08.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.085+0000 7f3903a00640 1 -- 172.21.15.86:0/4115757458 wait complete. 2024-04-20T21:21:08.116 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:21:08.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.283+0000 7f592f000640 1 Processor -- start 2024-04-20T21:21:08.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.283+0000 7f592f000640 1 -- start start 2024-04-20T21:21:08.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.285+0000 7f592f000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f59280d1ed0 0x7f5928169900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:08.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.285+0000 7f592f000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f592815eed0 con 0x7f59280b4030 2024-04-20T21:21:08.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.285+0000 7f592f000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59281627e0 con 0x7f592815eaf0 2024-04-20T21:21:08.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.285+0000 7f592f000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f5928163110 con 0x7f59280d1ed0 2024-04-20T21:21:08.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.285+0000 7f5927400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f59280b4030 0x7f59280d17c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:48436/0 (socket says 172.21.15.86:48436) 2024-04-20T21:21:08.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.285+0000 7f5927400640 1 -- 172.21.15.86:0/957254674 learned_addr learned my addr 172.21.15.86:0/957254674 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:21:08.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.285+0000 7f5926a00640 1 -- 172.21.15.86:0/957254674 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 78683517 0 0) 0x7f592815eed0 con 0x7f59280b4030 2024-04-20T21:21:08.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.285+0000 7f5926a00640 1 -- 172.21.15.86:0/957254674 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f590c003520 con 0x7f59280b4030 2024-04-20T21:21:08.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.285+0000 7f592d000640 1 --2- 172.21.15.86:0/957254674 >> v2:172.21.15.144:3300/0 conn(0x7f59280d1ed0 0x7f5928169900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:08.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.286+0000 7f5926a00640 1 -- 172.21.15.86:0/957254674 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1086156660 0 0) 0x7f590c003520 con 0x7f59280b4030 2024-04-20T21:21:08.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.286+0000 7f5926a00640 1 -- 172.21.15.86:0/957254674 >> v2:172.21.15.144:3300/0 conn(0x7f59280d1ed0 msgr2=0x7f5928169900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:08.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.286+0000 7f5926a00640 1 --2- 172.21.15.86:0/957254674 >> v2:172.21.15.144:3300/0 conn(0x7f59280d1ed0 0x7f5928169900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:08.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.286+0000 7f5926a00640 1 -- 172.21.15.86:0/957254674 >> v1:172.21.15.144:6789/0 conn(0x7f592815eaf0 legacy=0x7f59280ba1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:08.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.286+0000 7f5926a00640 1 -- 172.21.15.86:0/957254674 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5928162ac0 con 0x7f59280b4030 2024-04-20T21:21:08.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.286+0000 7f5926a00640 1 -- 172.21.15.86:0/957254674 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f5910001670 con 0x7f59280b4030 2024-04-20T21:21:08.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.286+0000 7f592d000640 1 --2- 172.21.15.86:0/957254674 >> v2:172.21.15.144:3300/0 conn(0x7f59280d1ed0 0x7f5928169900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:21:08.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.286+0000 7f5926a00640 1 -- 172.21.15.86:0/957254674 >> v1:172.21.15.86:6789/0 conn(0x7f59280b4030 legacy=0x7f59280d17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:08.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.286+0000 7f5926a00640 1 --2- 172.21.15.86:0/957254674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f590c003a00 0x7f590c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:08.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.286+0000 7f5926a00640 1 --2- 172.21.15.86:0/957254674 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f590c004800 0x7f590c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:08.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.286+0000 7f5926a00640 1 --2- 172.21.15.86:0/957254674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f590c005210 0x7f590c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:08.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.286+0000 7f5926a00640 1 -- 172.21.15.86:0/957254674 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5928163110 con 0x7f590c005210 2024-04-20T21:21:08.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.286+0000 7f5926a00640 1 -- 172.21.15.86:0/957254674 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f590c00c150 con 0x7f590c003a00 2024-04-20T21:21:08.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.286+0000 7f5926a00640 1 -- 172.21.15.86:0/957254674 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f590c00c350 con 0x7f590c004800 2024-04-20T21:21:08.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.286+0000 7f592d000640 1 --2- 172.21.15.86:0/957254674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f590c005210 0x7f590c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:08.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.287+0000 7f5927e00640 1 --2- 172.21.15.86:0/957254674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f590c003a00 0x7f590c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:08.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.287+0000 7f5927400640 1 --2- 172.21.15.86:0/957254674 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f590c004800 0x7f590c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:08.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.287+0000 7f592d000640 1 -- 172.21.15.86:0/957254674 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f590c004800 msgr2=0x7f590c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:08.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.287+0000 7f592d000640 1 --2- 172.21.15.86:0/957254674 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f590c004800 0x7f590c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:08.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.287+0000 7f592d000640 1 -- 172.21.15.86:0/957254674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f590c003a00 msgr2=0x7f590c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:08.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.287+0000 7f592d000640 1 --2- 172.21.15.86:0/957254674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f590c003a00 0x7f590c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:08.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.287+0000 7f592d000640 1 -- 172.21.15.86:0/957254674 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f590c003520 con 0x7f590c005210 2024-04-20T21:21:08.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.287+0000 7f5927400640 1 --2- 172.21.15.86:0/957254674 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f590c004800 0x7f590c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:21:08.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.288+0000 7f592d000640 1 -- 172.21.15.86:0/957254674 reap_dead start 2024-04-20T21:21:08.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.288+0000 7f592d000640 1 --2- 172.21.15.86:0/957254674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f590c005210 0x7f590c00b9c0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f59200098a0 tx=0x7f590c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=350486e12454d9b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:08.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.288+0000 7f5926a00640 1 -- 172.21.15.86:0/957254674 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592000f040 con 0x7f590c005210 2024-04-20T21:21:08.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.288+0000 7f5926a00640 1 -- 172.21.15.86:0/957254674 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5920004260 con 0x7f590c005210 2024-04-20T21:21:08.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.288+0000 7f5926a00640 1 -- 172.21.15.86:0/957254674 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5920006df0 con 0x7f590c005210 2024-04-20T21:21:08.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.289+0000 7f592f000640 1 -- 172.21.15.86:0/957254674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f590c005210 msgr2=0x7f590c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:08.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.289+0000 7f592f000640 1 --2- 172.21.15.86:0/957254674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f590c005210 0x7f590c00b9c0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f59200098a0 tx=0x7f590c005830 comp rx=0 tx=0).stop 2024-04-20T21:21:08.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.289+0000 7f592f000640 1 -- 172.21.15.86:0/957254674 shutdown_connections 2024-04-20T21:21:08.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.289+0000 7f592f000640 1 --2- 172.21.15.86:0/957254674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f590c005210 0x7f590c00b9c0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:08.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.289+0000 7f592f000640 1 -- 172.21.15.86:0/957254674 >> 172.21.15.86:0/957254674 conn(0x7f59280678a0 msgr2=0x7f5928067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:08.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.289+0000 7f592f000640 1 -- 172.21.15.86:0/957254674 shutdown_connections 2024-04-20T21:21:08.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.289+0000 7f592f000640 1 -- 172.21.15.86:0/957254674 wait complete. 2024-04-20T21:21:08.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.290+0000 7f592f000640 1 Processor -- start 2024-04-20T21:21:08.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.291+0000 7f592f000640 1 -- start start 2024-04-20T21:21:08.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.291+0000 7f592f000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f590c005210 0x7f59280cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:08.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.291+0000 7f592f000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5928159a90 0x7f59281ea520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:08.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.292+0000 7f592f000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59280b67a0 0x7f5928158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:08.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.292+0000 7f592f000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f59280d17c0 con 0x7f59280b67a0 2024-04-20T21:21:08.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.292+0000 7f592f000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f59280b7a00 con 0x7f590c005210 2024-04-20T21:21:08.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.292+0000 7f592f000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f59280d1940 con 0x7f5928159a90 2024-04-20T21:21:08.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.292+0000 7f592d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59280b67a0 0x7f5928158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:08.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.292+0000 7f592d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59280b67a0 0x7f5928158e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47710/0 (socket says 172.21.15.86:47710) 2024-04-20T21:21:08.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.292+0000 7f592d000640 1 -- 172.21.15.86:0/3467898717 learned_addr learned my addr 172.21.15.86:0/3467898717 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:08.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.292+0000 7f5927400640 1 --2- 172.21.15.86:0/3467898717 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5928159a90 0x7f59281ea520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:08.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.292+0000 7f5927e00640 1 --2- 172.21.15.86:0/3467898717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f590c005210 0x7f59280cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:08.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.292+0000 7f592d000640 1 -- 172.21.15.86:0/3467898717 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5928159a90 msgr2=0x7f59281ea520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:08.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.292+0000 7f592d000640 1 --2- 172.21.15.86:0/3467898717 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5928159a90 0x7f59281ea520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:08.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.292+0000 7f592d000640 1 -- 172.21.15.86:0/3467898717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f590c005210 msgr2=0x7f59280cc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:08.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.292+0000 7f592d000640 1 --2- 172.21.15.86:0/3467898717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f590c005210 0x7f59280cc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:08.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.292+0000 7f592d000640 1 -- 172.21.15.86:0/3467898717 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59280cca20 con 0x7f59280b67a0 2024-04-20T21:21:08.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.292+0000 7f5927400640 1 --2- 172.21.15.86:0/3467898717 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5928159a90 0x7f59281ea520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:21:08.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.292+0000 7f5927e00640 1 --2- 172.21.15.86:0/3467898717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f590c005210 0x7f59280cc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:21:08.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.293+0000 7f592d000640 1 --2- 172.21.15.86:0/3467898717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59280b67a0 0x7f5928158e00 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f59200099c0 tx=0x7f5920004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:08.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.293+0000 7f591fe00640 1 -- 172.21.15.86:0/3467898717 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5920047070 con 0x7f59280b67a0 2024-04-20T21:21:08.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.293+0000 7f591fe00640 1 -- 172.21.15.86:0/3467898717 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5920042600 con 0x7f59280b67a0 2024-04-20T21:21:08.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.293+0000 7f591fe00640 1 -- 172.21.15.86:0/3467898717 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59200429f0 con 0x7f59280b67a0 2024-04-20T21:21:08.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.293+0000 7f592f000640 1 -- 172.21.15.86:0/3467898717 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59281eaa60 con 0x7f59280b67a0 2024-04-20T21:21:08.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.293+0000 7f592f000640 1 -- 172.21.15.86:0/3467898717 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59281e9760 con 0x7f59280b67a0 2024-04-20T21:21:08.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.294+0000 7f592f000640 1 -- 172.21.15.86:0/3467898717 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59280689e0 con 0x7f59280b67a0 2024-04-20T21:21:08.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.295+0000 7f591fe00640 1 -- 172.21.15.86:0/3467898717 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f592004a070 con 0x7f59280b67a0 2024-04-20T21:21:08.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.296+0000 7f591fe00640 1 --2- 172.21.15.86:0/3467898717 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f590406c920 0x7f590406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:08.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.296+0000 7f591fe00640 1 -- 172.21.15.86:0/3467898717 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f59200ad140 con 0x7f59280b67a0 2024-04-20T21:21:08.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.296+0000 7f5927e00640 1 --2- 172.21.15.86:0/3467898717 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f590406c920 0x7f590406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:08.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.297+0000 7f5927e00640 1 --2- 172.21.15.86:0/3467898717 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f590406c920 0x7f590406ede0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5918007fd0 tx=0x7f5918009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:08.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.301+0000 7f591fe00640 1 -- 172.21.15.86:0/3467898717 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f59280cca20 con 0x7f59280b67a0 2024-04-20T21:21:08.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.501+0000 7f592f000640 1 -- 172.21.15.86:0/3467898717 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f592806dda0 con 0x7f59280b67a0 2024-04-20T21:21:08.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.502+0000 7f591fe00640 1 -- 172.21.15.86:0/3467898717 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16678 (secure 0 0 0) 0x7f5920033070 con 0x7f59280b67a0 2024-04-20T21:21:08.503 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:21:08.503 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":58,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:21:07.502893+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:21:05.248922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-20T21:21:06.479064+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:21:08.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.505+0000 7f592f000640 1 -- 172.21.15.86:0/3467898717 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f590406c920 msgr2=0x7f590406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:08.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.505+0000 7f592f000640 1 --2- 172.21.15.86:0/3467898717 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f590406c920 0x7f590406ede0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5918007fd0 tx=0x7f5918009040 comp rx=0 tx=0).stop 2024-04-20T21:21:08.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.505+0000 7f592f000640 1 -- 172.21.15.86:0/3467898717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59280b67a0 msgr2=0x7f5928158e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:08.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.505+0000 7f592f000640 1 --2- 172.21.15.86:0/3467898717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59280b67a0 0x7f5928158e00 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f59200099c0 tx=0x7f5920004730 comp rx=0 tx=0).stop 2024-04-20T21:21:08.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.505+0000 7f592f000640 1 -- 172.21.15.86:0/3467898717 shutdown_connections 2024-04-20T21:21:08.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.505+0000 7f592f000640 1 --2- 172.21.15.86:0/3467898717 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f590406c920 0x7f590406ede0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:08.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.505+0000 7f592f000640 1 --2- 172.21.15.86:0/3467898717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59280b67a0 0x7f5928158e00 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:08.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.505+0000 7f592f000640 1 --2- 172.21.15.86:0/3467898717 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5928159a90 0x7f59281ea520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:08.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.505+0000 7f592f000640 1 --2- 172.21.15.86:0/3467898717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f590c005210 0x7f59280cc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:08.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.505+0000 7f592f000640 1 -- 172.21.15.86:0/3467898717 >> 172.21.15.86:0/3467898717 conn(0x7f59280678a0 msgr2=0x7f59280b95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:08.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.506+0000 7f592f000640 1 -- 172.21.15.86:0/3467898717 shutdown_connections 2024-04-20T21:21:08.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.506+0000 7f592f000640 1 -- 172.21.15.86:0/3467898717 wait complete. 2024-04-20T21:21:08.523 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:21:08.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.706+0000 7f8965000640 1 Processor -- start 2024-04-20T21:21:08.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.706+0000 7f8965000640 1 -- start start 2024-04-20T21:21:08.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.707+0000 7f8965000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f896006e6d0 0x7f8960071d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:08.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.708+0000 7f8965000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8960072240 0x7f896015cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:08.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.708+0000 7f8965000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f896006eb30 con 0x7f8960146130 2024-04-20T21:21:08.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.708+0000 7f8965000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f896006dc30 con 0x7f8960072240 2024-04-20T21:21:08.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.708+0000 7f8965000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f89600cd8e0 con 0x7f896006e6d0 2024-04-20T21:21:08.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.708+0000 7f895e800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8960072240 0x7f896015cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:08.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.708+0000 7f895e800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8960072240 0x7f896015cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47726/0 (socket says 172.21.15.86:47726) 2024-04-20T21:21:08.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.708+0000 7f895e800640 1 -- 172.21.15.86:0/1946854974 learned_addr learned my addr 172.21.15.86:0/1946854974 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:08.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.708+0000 7f895d400640 1 --2- 172.21.15.86:0/1946854974 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f896006e6d0 0x7f8960071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:08.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.708+0000 7f895e800640 1 -- 172.21.15.86:0/1946854974 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f896006e6d0 msgr2=0x7f8960071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:08.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.708+0000 7f895e800640 1 --2- 172.21.15.86:0/1946854974 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f896006e6d0 0x7f8960071d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:08.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.708+0000 7f895e800640 1 -- 172.21.15.86:0/1946854974 >> v1:172.21.15.144:6789/0 conn(0x7f8960146130 legacy=0x7f89600ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:08.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.708+0000 7f895e800640 1 -- 172.21.15.86:0/1946854974 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89600cdbc0 con 0x7f8960072240 2024-04-20T21:21:08.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.708+0000 7f895d400640 1 --2- 172.21.15.86:0/1946854974 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f896006e6d0 0x7f8960071d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:21:08.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.710+0000 7f895e800640 1 --2- 172.21.15.86:0/1946854974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8960072240 0x7f896015cc20 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f8958009e50 tx=0x7f895802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0ae4d92e417b9fa server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:08.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.710+0000 7f895ca00640 1 -- 172.21.15.86:0/1946854974 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895802caf0 con 0x7f8960072240 2024-04-20T21:21:08.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.710+0000 7f895ca00640 1 -- 172.21.15.86:0/1946854974 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f895802cc90 con 0x7f8960072240 2024-04-20T21:21:08.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.710+0000 7f895ca00640 1 -- 172.21.15.86:0/1946854974 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8958034610 con 0x7f8960072240 2024-04-20T21:21:08.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.710+0000 7f8965000640 1 -- 172.21.15.86:0/1946854974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8960072240 msgr2=0x7f896015cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:08.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.710+0000 7f8965000640 1 --2- 172.21.15.86:0/1946854974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8960072240 0x7f896015cc20 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f8958009e50 tx=0x7f895802bf50 comp rx=0 tx=0).stop 2024-04-20T21:21:08.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.711+0000 7f8965000640 1 -- 172.21.15.86:0/1946854974 shutdown_connections 2024-04-20T21:21:08.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.711+0000 7f8965000640 1 --2- 172.21.15.86:0/1946854974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8960072240 0x7f896015cc20 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:08.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.711+0000 7f8965000640 1 --2- 172.21.15.86:0/1946854974 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f896006e6d0 0x7f8960071d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:08.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.711+0000 7f8965000640 1 -- 172.21.15.86:0/1946854974 >> 172.21.15.86:0/1946854974 conn(0x7f89600b2360 msgr2=0x7f89600b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:08.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.711+0000 7f8965000640 1 -- 172.21.15.86:0/1946854974 shutdown_connections 2024-04-20T21:21:08.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.711+0000 7f8965000640 1 -- 172.21.15.86:0/1946854974 wait complete. 2024-04-20T21:21:08.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.712+0000 7f8965000640 1 Processor -- start 2024-04-20T21:21:08.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.712+0000 7f8965000640 1 -- start start 2024-04-20T21:21:08.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.713+0000 7f8965000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f896006e6d0 0x7f896015c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:08.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.713+0000 7f8965000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8960072240 0x7f89600ca650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:08.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.713+0000 7f8965000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8960146130 0x7f89601e7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:08.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.713+0000 7f8965000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f896006f150 con 0x7f896006e6d0 2024-04-20T21:21:08.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.713+0000 7f8965000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f896006e1b0 con 0x7f8960146130 2024-04-20T21:21:08.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.713+0000 7f8965000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f89600cd4e0 con 0x7f8960072240 2024-04-20T21:21:08.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.714+0000 7f895de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f896006e6d0 0x7f896015c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:08.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.714+0000 7f895de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f896006e6d0 0x7f896015c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47736/0 (socket says 172.21.15.86:47736) 2024-04-20T21:21:08.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.714+0000 7f895de00640 1 -- 172.21.15.86:0/3499232413 learned_addr learned my addr 172.21.15.86:0/3499232413 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:08.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.714+0000 7f895de00640 1 -- 172.21.15.86:0/3499232413 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8960072240 msgr2=0x7f89600ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:08.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.714+0000 7f895de00640 1 --2- 172.21.15.86:0/3499232413 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8960072240 0x7f89600ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:08.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.714+0000 7f895de00640 1 -- 172.21.15.86:0/3499232413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8960146130 msgr2=0x7f89601e7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:08.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.714+0000 7f895de00640 1 --2- 172.21.15.86:0/3499232413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8960146130 0x7f89601e7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:08.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.714+0000 7f895de00640 1 -- 172.21.15.86:0/3499232413 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89601e4f20 con 0x7f896006e6d0 2024-04-20T21:21:08.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.714+0000 7f895de00640 1 --2- 172.21.15.86:0/3499232413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f896006e6d0 0x7f896015c050 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f895000b3e0 tx=0x7f895000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:08.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.714+0000 7f8956000640 1 -- 172.21.15.86:0/3499232413 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8950015020 con 0x7f896006e6d0 2024-04-20T21:21:08.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.715+0000 7f8956000640 1 -- 172.21.15.86:0/3499232413 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8950011850 con 0x7f896006e6d0 2024-04-20T21:21:08.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.715+0000 7f8956000640 1 -- 172.21.15.86:0/3499232413 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8950011c40 con 0x7f896006e6d0 2024-04-20T21:21:08.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.715+0000 7f8965000640 1 -- 172.21.15.86:0/3499232413 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89600cafc0 con 0x7f896006e6d0 2024-04-20T21:21:08.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.715+0000 7f8965000640 1 -- 172.21.15.86:0/3499232413 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89600cb580 con 0x7f896006e6d0 2024-04-20T21:21:08.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.716+0000 7f8965000640 1 -- 172.21.15.86:0/3499232413 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89601e52f0 con 0x7f896006e6d0 2024-04-20T21:21:08.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.717+0000 7f8956000640 1 -- 172.21.15.86:0/3499232413 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8950018020 con 0x7f896006e6d0 2024-04-20T21:21:08.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.717+0000 7f8956000640 1 --2- 172.21.15.86:0/3499232413 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f894006c8f0 0x7f894006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:08.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.718+0000 7f8956000640 1 -- 172.21.15.86:0/3499232413 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f895008baa0 con 0x7f896006e6d0 2024-04-20T21:21:08.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.718+0000 7f895d400640 1 --2- 172.21.15.86:0/3499232413 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f894006c8f0 0x7f894006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:08.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.718+0000 7f895d400640 1 --2- 172.21.15.86:0/3499232413 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f894006c8f0 0x7f894006edb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f894c006fd0 tx=0x7f894c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:08.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.722+0000 7f8956000640 1 -- 172.21.15.86:0/3499232413 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f895001f020 con 0x7f896006e6d0 2024-04-20T21:21:08.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.920+0000 7f8965000640 1 -- 172.21.15.86:0/3499232413 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f89600689e0 con 0x7f896006e6d0 2024-04-20T21:21:08.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.923+0000 7f8956000640 1 -- 172.21.15.86:0/3499232413 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16678 (secure 0 0 0) 0x7f89500172c0 con 0x7f896006e6d0 2024-04-20T21:21:08.924 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:21:08.924 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":58,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:21:07.502893+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:21:05.248922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-20T21:21:06.479064+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:21:08.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.926+0000 7f8965000640 1 -- 172.21.15.86:0/3499232413 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f894006c8f0 msgr2=0x7f894006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:08.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.926+0000 7f8965000640 1 --2- 172.21.15.86:0/3499232413 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f894006c8f0 0x7f894006edb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f894c006fd0 tx=0x7f894c008040 comp rx=0 tx=0).stop 2024-04-20T21:21:08.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.926+0000 7f8965000640 1 -- 172.21.15.86:0/3499232413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f896006e6d0 msgr2=0x7f896015c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:08.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.926+0000 7f8965000640 1 --2- 172.21.15.86:0/3499232413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f896006e6d0 0x7f896015c050 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f895000b3e0 tx=0x7f895000b8a0 comp rx=0 tx=0).stop 2024-04-20T21:21:08.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.926+0000 7f8965000640 1 -- 172.21.15.86:0/3499232413 shutdown_connections 2024-04-20T21:21:08.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.926+0000 7f8965000640 1 --2- 172.21.15.86:0/3499232413 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f894006c8f0 0x7f894006edb0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:08.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.926+0000 7f8965000640 1 --2- 172.21.15.86:0/3499232413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8960146130 0x7f89601e7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:08.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.926+0000 7f8965000640 1 --2- 172.21.15.86:0/3499232413 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8960072240 0x7f89600ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:08.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.926+0000 7f8965000640 1 --2- 172.21.15.86:0/3499232413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f896006e6d0 0x7f896015c050 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:08.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.926+0000 7f8965000640 1 -- 172.21.15.86:0/3499232413 >> 172.21.15.86:0/3499232413 conn(0x7f89600b2360 msgr2=0x7f896006c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:08.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.926+0000 7f8965000640 1 -- 172.21.15.86:0/3499232413 shutdown_connections 2024-04-20T21:21:08.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:08.926+0000 7f8965000640 1 -- 172.21.15.86:0/3499232413 wait complete. 2024-04-20T21:21:08.945 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-20T21:21:09.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.137+0000 7fd749800640 1 Processor -- start 2024-04-20T21:21:09.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.137+0000 7fd749800640 1 -- start start 2024-04-20T21:21:09.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.138+0000 7fd749800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd7440b5060 0x7fd7440b54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:09.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.138+0000 7fd749800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd74415ecb0 con 0x7fd74406c990 2024-04-20T21:21:09.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.138+0000 7fd749800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd744163380 con 0x7fd74415e8d0 2024-04-20T21:21:09.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.139+0000 7fd749800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fd7440b59e0 con 0x7fd7440b5060 2024-04-20T21:21:09.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.139+0000 7fd743400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd74406c990 0x7fd7440aeb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:48440/0 (socket says 172.21.15.86:48440) 2024-04-20T21:21:09.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.139+0000 7fd743400640 1 -- 172.21.15.86:0/601801300 learned_addr learned my addr 172.21.15.86:0/601801300 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:21:09.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.139+0000 7fd741600640 1 -- 172.21.15.86:0/601801300 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3760174350 0 0) 0x7fd74415ecb0 con 0x7fd74406c990 2024-04-20T21:21:09.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.139+0000 7fd741600640 1 -- 172.21.15.86:0/601801300 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd728003520 con 0x7fd74406c990 2024-04-20T21:21:09.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.139+0000 7fd742000640 1 --2- 172.21.15.86:0/601801300 >> v2:172.21.15.144:3300/0 conn(0x7fd7440b5060 0x7fd7440b54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:09.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.139+0000 7fd741600640 1 -- 172.21.15.86:0/601801300 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3311036574 0 0) 0x7fd728003520 con 0x7fd74406c990 2024-04-20T21:21:09.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.140+0000 7fd741600640 1 -- 172.21.15.86:0/601801300 >> v2:172.21.15.144:3300/0 conn(0x7fd7440b5060 msgr2=0x7fd7440b54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:09.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.140+0000 7fd741600640 1 --2- 172.21.15.86:0/601801300 >> v2:172.21.15.144:3300/0 conn(0x7fd7440b5060 0x7fd7440b54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:09.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.140+0000 7fd741600640 1 -- 172.21.15.86:0/601801300 >> v1:172.21.15.144:6789/0 conn(0x7fd74415e8d0 legacy=0x7fd74406f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:09.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.140+0000 7fd741600640 1 -- 172.21.15.86:0/601801300 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7440b5c20 con 0x7fd74406c990 2024-04-20T21:21:09.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.140+0000 7fd741600640 1 -- 172.21.15.86:0/601801300 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fd73c001670 con 0x7fd74406c990 2024-04-20T21:21:09.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.140+0000 7fd741600640 1 -- 172.21.15.86:0/601801300 >> v1:172.21.15.86:6789/0 conn(0x7fd74406c990 legacy=0x7fd7440aeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:09.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.140+0000 7fd741600640 1 --2- 172.21.15.86:0/601801300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7280039b0 0x7fd728003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:09.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.140+0000 7fd741600640 1 --2- 172.21.15.86:0/601801300 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd728004800 0x7fd728004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:09.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.140+0000 7fd741600640 1 --2- 172.21.15.86:0/601801300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7280051c0 0x7fd72800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:09.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.140+0000 7fd741600640 1 -- 172.21.15.86:0/601801300 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd7440b59e0 con 0x7fd7280051c0 2024-04-20T21:21:09.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.140+0000 7fd741600640 1 -- 172.21.15.86:0/601801300 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd72800c0f0 con 0x7fd7280039b0 2024-04-20T21:21:09.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.140+0000 7fd741600640 1 -- 172.21.15.86:0/601801300 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd72800c2f0 con 0x7fd728004800 2024-04-20T21:21:09.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.140+0000 7fd743400640 1 --2- 172.21.15.86:0/601801300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7280051c0 0x7fd72800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:09.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.140+0000 7fd742000640 1 --2- 172.21.15.86:0/601801300 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd728004800 0x7fd728004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:09.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.140+0000 7fd742a00640 1 --2- 172.21.15.86:0/601801300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7280039b0 0x7fd728003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:09.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.140+0000 7fd743400640 1 -- 172.21.15.86:0/601801300 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd728004800 msgr2=0x7fd728004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:09.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.140+0000 7fd743400640 1 --2- 172.21.15.86:0/601801300 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd728004800 0x7fd728004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:09.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.141+0000 7fd743400640 1 -- 172.21.15.86:0/601801300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7280039b0 msgr2=0x7fd728003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:09.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.141+0000 7fd743400640 1 --2- 172.21.15.86:0/601801300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7280039b0 0x7fd728003db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:09.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.141+0000 7fd743400640 1 -- 172.21.15.86:0/601801300 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd728003520 con 0x7fd7280051c0 2024-04-20T21:21:09.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.141+0000 7fd743400640 1 -- 172.21.15.86:0/601801300 reap_dead start 2024-04-20T21:21:09.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.141+0000 7fd743400640 1 --2- 172.21.15.86:0/601801300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7280051c0 0x7fd72800b9c0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fd73c007fd0 tx=0x7fd728005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1c1141bcf1d39ae server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:09.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.142+0000 7fd741600640 1 -- 172.21.15.86:0/601801300 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73c007d50 con 0x7fd7280051c0 2024-04-20T21:21:09.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.142+0000 7fd741600640 1 -- 172.21.15.86:0/601801300 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd73c030070 con 0x7fd7280051c0 2024-04-20T21:21:09.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.142+0000 7fd741600640 1 -- 172.21.15.86:0/601801300 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73c005d30 con 0x7fd7280051c0 2024-04-20T21:21:09.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.142+0000 7fd749800640 1 -- 172.21.15.86:0/601801300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7280051c0 msgr2=0x7fd72800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:09.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.142+0000 7fd749800640 1 --2- 172.21.15.86:0/601801300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7280051c0 0x7fd72800b9c0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fd73c007fd0 tx=0x7fd728005830 comp rx=0 tx=0).stop 2024-04-20T21:21:09.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.142+0000 7fd749800640 1 -- 172.21.15.86:0/601801300 shutdown_connections 2024-04-20T21:21:09.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.142+0000 7fd749800640 1 --2- 172.21.15.86:0/601801300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7280051c0 0x7fd72800b9c0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:09.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.142+0000 7fd749800640 1 -- 172.21.15.86:0/601801300 >> 172.21.15.86:0/601801300 conn(0x7fd7440725c0 msgr2=0x7fd7440729d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:09.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.142+0000 7fd749800640 1 -- 172.21.15.86:0/601801300 shutdown_connections 2024-04-20T21:21:09.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.142+0000 7fd749800640 1 -- 172.21.15.86:0/601801300 wait complete. 2024-04-20T21:21:09.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.144+0000 7fd749800640 1 Processor -- start 2024-04-20T21:21:09.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.144+0000 7fd749800640 1 -- start start 2024-04-20T21:21:09.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.145+0000 7fd749800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7280051c0 0x7fd7441f4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:09.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.145+0000 7fd749800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd74415ecb0 0x7fd74415f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:09.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.145+0000 7fd749800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7440cfa80 0x7fd7440cfec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:09.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.145+0000 7fd749800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd74406e990 con 0x7fd7280051c0 2024-04-20T21:21:09.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.145+0000 7fd749800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd74406e460 con 0x7fd7440cfa80 2024-04-20T21:21:09.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.145+0000 7fd749800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd74406ec70 con 0x7fd74415ecb0 2024-04-20T21:21:09.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.145+0000 7fd742a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7280051c0 0x7fd7441f4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:09.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.145+0000 7fd742a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7280051c0 0x7fd7441f4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47752/0 (socket says 172.21.15.86:47752) 2024-04-20T21:21:09.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.145+0000 7fd742a00640 1 -- 172.21.15.86:0/2161480096 learned_addr learned my addr 172.21.15.86:0/2161480096 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:09.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.146+0000 7fd742a00640 1 -- 172.21.15.86:0/2161480096 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd74415ecb0 msgr2=0x7fd74415f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:09.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.146+0000 7fd742a00640 1 --2- 172.21.15.86:0/2161480096 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd74415ecb0 0x7fd74415f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:09.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.146+0000 7fd742a00640 1 -- 172.21.15.86:0/2161480096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7440cfa80 msgr2=0x7fd7440cfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:09.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.146+0000 7fd743400640 1 --2- 172.21.15.86:0/2161480096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7440cfa80 0x7fd7440cfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:09.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.146+0000 7fd742a00640 1 --2- 172.21.15.86:0/2161480096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7440cfa80 0x7fd7440cfec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:09.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.146+0000 7fd742a00640 1 -- 172.21.15.86:0/2161480096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7440d0400 con 0x7fd7280051c0 2024-04-20T21:21:09.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.146+0000 7fd742a00640 1 --2- 172.21.15.86:0/2161480096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7280051c0 0x7fd7441f4eb0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fd72c00d540 tx=0x7fd72c00da00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:09.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.146+0000 7fd73aa00640 1 -- 172.21.15.86:0/2161480096 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72c016070 con 0x7fd7280051c0 2024-04-20T21:21:09.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.146+0000 7fd73aa00640 1 -- 172.21.15.86:0/2161480096 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd72c012800 con 0x7fd7280051c0 2024-04-20T21:21:09.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.146+0000 7fd73aa00640 1 -- 172.21.15.86:0/2161480096 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72c012bf0 con 0x7fd7280051c0 2024-04-20T21:21:09.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.146+0000 7fd749800640 1 -- 172.21.15.86:0/2161480096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd74415f730 con 0x7fd7280051c0 2024-04-20T21:21:09.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.146+0000 7fd749800640 1 -- 172.21.15.86:0/2161480096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd74415fcf0 con 0x7fd7280051c0 2024-04-20T21:21:09.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.147+0000 7fd749800640 1 -- 172.21.15.86:0/2161480096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7440689e0 con 0x7fd7280051c0 2024-04-20T21:21:09.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.148+0000 7fd73aa00640 1 -- 172.21.15.86:0/2161480096 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd72c019070 con 0x7fd7280051c0 2024-04-20T21:21:09.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.149+0000 7fd73aa00640 1 --2- 172.21.15.86:0/2161480096 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd72006c920 0x7fd72006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:09.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.149+0000 7fd73aa00640 1 -- 172.21.15.86:0/2161480096 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fd72c08b610 con 0x7fd7280051c0 2024-04-20T21:21:09.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.150+0000 7fd742000640 1 --2- 172.21.15.86:0/2161480096 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd72006c920 0x7fd72006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:09.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.150+0000 7fd742000640 1 --2- 172.21.15.86:0/2161480096 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd72006c920 0x7fd72006ede0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd734007130 tx=0x7fd734007080 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:09.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.154+0000 7fd73aa00640 1 -- 172.21.15.86:0/2161480096 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd72c059a30 con 0x7fd7280051c0 2024-04-20T21:21:09.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:09.379+0000 7fd749800640 1 -- 172.21.15.86:0/2161480096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fd7441593d0 con 0x7fd7280051c0 2024-04-20T21:21:10.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.094+0000 7fd73aa00640 1 -- 172.21.15.86:0/2161480096 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v31) ==== 118+0+0 (secure 0 0 0) 0x7fd72c059330 con 0x7fd7280051c0 2024-04-20T21:21:10.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.098+0000 7fd749800640 1 -- 172.21.15.86:0/2161480096 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd72006c920 msgr2=0x7fd72006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:10.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.098+0000 7fd749800640 1 --2- 172.21.15.86:0/2161480096 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd72006c920 0x7fd72006ede0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd734007130 tx=0x7fd734007080 comp rx=0 tx=0).stop 2024-04-20T21:21:10.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.098+0000 7fd749800640 1 -- 172.21.15.86:0/2161480096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7280051c0 msgr2=0x7fd7441f4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:10.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.098+0000 7fd749800640 1 --2- 172.21.15.86:0/2161480096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7280051c0 0x7fd7441f4eb0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fd72c00d540 tx=0x7fd72c00da00 comp rx=0 tx=0).stop 2024-04-20T21:21:10.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.098+0000 7fd749800640 1 -- 172.21.15.86:0/2161480096 shutdown_connections 2024-04-20T21:21:10.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.098+0000 7fd749800640 1 --2- 172.21.15.86:0/2161480096 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd72006c920 0x7fd72006ede0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:10.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.098+0000 7fd749800640 1 --2- 172.21.15.86:0/2161480096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7440cfa80 0x7fd7440cfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:10.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.098+0000 7fd749800640 1 --2- 172.21.15.86:0/2161480096 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd74415ecb0 0x7fd74415f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:10.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.098+0000 7fd749800640 1 --2- 172.21.15.86:0/2161480096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7280051c0 0x7fd7441f4eb0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:10.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.098+0000 7fd749800640 1 -- 172.21.15.86:0/2161480096 >> 172.21.15.86:0/2161480096 conn(0x7fd7440725c0 msgr2=0x7fd74406fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:10.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.098+0000 7fd749800640 1 -- 172.21.15.86:0/2161480096 shutdown_connections 2024-04-20T21:21:10.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.099+0000 7fd749800640 1 -- 172.21.15.86:0/2161480096 wait complete. 2024-04-20T21:21:10.119 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-20T21:21:10.119 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-20T21:21:10.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.309+0000 7f556b200640 1 Processor -- start 2024-04-20T21:21:10.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.310+0000 7f556b200640 1 -- start start 2024-04-20T21:21:10.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.312+0000 7f556b200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f556415e8f0 0x7f5564162640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:10.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.312+0000 7f556b200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f556415ecd0 0x7f5564162b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:10.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.312+0000 7f556b200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55640d29a0 con 0x7f55640cf680 2024-04-20T21:21:10.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.312+0000 7f556b200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f55640d2fd0 con 0x7f556415ecd0 2024-04-20T21:21:10.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.312+0000 7f556b200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f55641631b0 con 0x7f556415e8f0 2024-04-20T21:21:10.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.312+0000 7f5569200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f55640cf680 0x7f55640b6330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:48448/0 (socket says 172.21.15.86:48448) 2024-04-20T21:21:10.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.312+0000 7f5569200640 1 -- 172.21.15.86:0/3068821855 learned_addr learned my addr 172.21.15.86:0/3068821855 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:21:10.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.312+0000 7f5563e00640 1 --2- 172.21.15.86:0/3068821855 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f556415e8f0 0x7f5564162640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:10.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.312+0000 7f555b400640 1 --2- 172.21.15.86:0/3068821855 >> v2:172.21.15.144:3300/0 conn(0x7f556415ecd0 0x7f5564162b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:10.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.312+0000 7f5563400640 1 -- 172.21.15.86:0/3068821855 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4149562901 0 0) 0x7f55640d29a0 con 0x7f55640cf680 2024-04-20T21:21:10.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.312+0000 7f5563400640 1 -- 172.21.15.86:0/3068821855 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5548003520 con 0x7f55640cf680 2024-04-20T21:21:10.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.312+0000 7f5563400640 1 -- 172.21.15.86:0/3068821855 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3558558799 0 0) 0x7f5548003520 con 0x7f55640cf680 2024-04-20T21:21:10.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.312+0000 7f5563400640 1 -- 172.21.15.86:0/3068821855 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f556415e8f0 msgr2=0x7f5564162640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:10.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.312+0000 7f5563400640 1 --2- 172.21.15.86:0/3068821855 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f556415e8f0 0x7f5564162640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:10.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.312+0000 7f5563400640 1 -- 172.21.15.86:0/3068821855 >> v2:172.21.15.144:3300/0 conn(0x7f556415ecd0 msgr2=0x7f5564162b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:10.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.312+0000 7f5563400640 1 --2- 172.21.15.86:0/3068821855 >> v2:172.21.15.144:3300/0 conn(0x7f556415ecd0 0x7f5564162b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:10.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.312+0000 7f5563400640 1 -- 172.21.15.86:0/3068821855 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5564163490 con 0x7f55640cf680 2024-04-20T21:21:10.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.312+0000 7f5563400640 1 -- 172.21.15.86:0/3068821855 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f555c0019f0 con 0x7f55640cf680 2024-04-20T21:21:10.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.312+0000 7f5563400640 1 -- 172.21.15.86:0/3068821855 >> v1:172.21.15.86:6789/0 conn(0x7f55640cf680 legacy=0x7f55640b6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:10.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.312+0000 7f5563400640 1 --2- 172.21.15.86:0/3068821855 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f556415e8f0 0x7f55480039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:10.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.312+0000 7f5563400640 1 --2- 172.21.15.86:0/3068821855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5548003f30 0x7f5548004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:10.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.313+0000 7f5563400640 1 --2- 172.21.15.86:0/3068821855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5548004d40 0x7f55480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:10.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.313+0000 7f5563400640 1 -- 172.21.15.86:0/3068821855 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f55640d2fd0 con 0x7f5548003f30 2024-04-20T21:21:10.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.313+0000 7f5563400640 1 -- 172.21.15.86:0/3068821855 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f55641631b0 con 0x7f5548004d40 2024-04-20T21:21:10.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.313+0000 7f5563400640 1 -- 172.21.15.86:0/3068821855 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f554800ba10 con 0x7f556415e8f0 2024-04-20T21:21:10.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.313+0000 7f555b400640 1 --2- 172.21.15.86:0/3068821855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5548003f30 0x7f5548004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:10.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.313+0000 7f555b400640 1 -- 172.21.15.86:0/3068821855 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f556415e8f0 msgr2=0x7f55480039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:10.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.313+0000 7f555b400640 1 --2- 172.21.15.86:0/3068821855 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f556415e8f0 0x7f55480039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:10.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.313+0000 7f555b400640 1 -- 172.21.15.86:0/3068821855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5548004d40 msgr2=0x7f55480051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:10.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.313+0000 7f555b400640 1 --2- 172.21.15.86:0/3068821855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5548004d40 0x7f55480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:10.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.313+0000 7f555b400640 1 -- 172.21.15.86:0/3068821855 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5548003520 con 0x7f5548003f30 2024-04-20T21:21:10.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.314+0000 7f555b400640 1 --2- 172.21.15.86:0/3068821855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5548003f30 0x7f5548004800 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f554c009a90 tx=0x7f5548005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=323ee660b612b03b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:10.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.314+0000 7f5563400640 1 -- 172.21.15.86:0/3068821855 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554c039070 con 0x7f5548003f30 2024-04-20T21:21:10.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.314+0000 7f5563400640 1 -- 172.21.15.86:0/3068821855 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f554c02c6d0 con 0x7f5548003f30 2024-04-20T21:21:10.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.314+0000 7f5563400640 1 -- 172.21.15.86:0/3068821855 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554c009690 con 0x7f5548003f30 2024-04-20T21:21:10.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.315+0000 7f556b200640 1 -- 172.21.15.86:0/3068821855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5548003f30 msgr2=0x7f5548004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:10.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.315+0000 7f556b200640 1 --2- 172.21.15.86:0/3068821855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5548003f30 0x7f5548004800 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f554c009a90 tx=0x7f5548005830 comp rx=0 tx=0).stop 2024-04-20T21:21:10.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.315+0000 7f5569200640 1 -- 172.21.15.86:0/3068821855 reap_dead start 2024-04-20T21:21:10.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.315+0000 7f556b200640 1 -- 172.21.15.86:0/3068821855 shutdown_connections 2024-04-20T21:21:10.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.315+0000 7f556b200640 1 -- 172.21.15.86:0/3068821855 >> 172.21.15.86:0/3068821855 conn(0x7f556406cc70 msgr2=0x7f556406d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:10.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.315+0000 7f556b200640 1 -- 172.21.15.86:0/3068821855 shutdown_connections 2024-04-20T21:21:10.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.315+0000 7f556b200640 1 -- 172.21.15.86:0/3068821855 wait complete. 2024-04-20T21:21:10.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.317+0000 7f556b200640 1 Processor -- start 2024-04-20T21:21:10.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.317+0000 7f556b200640 1 -- start start 2024-04-20T21:21:10.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.318+0000 7f556b200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5548003f30 0x7f5564148c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:10.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.318+0000 7f556b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55640cdeb0 0x7f55640ce310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:10.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.318+0000 7f556b200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f556415c580 0x7f556415ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:10.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.318+0000 7f556b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f55640ba7f0 con 0x7f55640cdeb0 2024-04-20T21:21:10.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.318+0000 7f556b200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f55640b7a50 con 0x7f556415c580 2024-04-20T21:21:10.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.318+0000 7f556b200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f55640d4470 con 0x7f5548003f30 2024-04-20T21:21:10.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.318+0000 7f555b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55640cdeb0 0x7f55640ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:10.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.318+0000 7f555b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55640cdeb0 0x7f55640ce310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47768/0 (socket says 172.21.15.86:47768) 2024-04-20T21:21:10.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.318+0000 7f555b400640 1 -- 172.21.15.86:0/3138744845 learned_addr learned my addr 172.21.15.86:0/3138744845 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:10.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.318+0000 7f5563e00640 1 --2- 172.21.15.86:0/3138744845 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5548003f30 0x7f5564148c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:10.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.318+0000 7f5569200640 1 --2- 172.21.15.86:0/3138744845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f556415c580 0x7f556415ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:10.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.318+0000 7f555b400640 1 -- 172.21.15.86:0/3138744845 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5548003f30 msgr2=0x7f5564148c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:10.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.318+0000 7f555b400640 1 --2- 172.21.15.86:0/3138744845 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5548003f30 0x7f5564148c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:10.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.318+0000 7f555b400640 1 -- 172.21.15.86:0/3138744845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f556415c580 msgr2=0x7f556415ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:10.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.318+0000 7f555b400640 1 --2- 172.21.15.86:0/3138744845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f556415c580 0x7f556415ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:10.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.318+0000 7f555b400640 1 -- 172.21.15.86:0/3138744845 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f556415cf40 con 0x7f55640cdeb0 2024-04-20T21:21:10.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.318+0000 7f5563e00640 1 --2- 172.21.15.86:0/3138744845 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5548003f30 0x7f5564148c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:21:10.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.319+0000 7f555b400640 1 --2- 172.21.15.86:0/3138744845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55640cdeb0 0x7f55640ce310 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f554c037040 tx=0x7f554c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:10.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.319+0000 7f5560c00640 1 -- 172.21.15.86:0/3138744845 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554c042020 con 0x7f55640cdeb0 2024-04-20T21:21:10.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.319+0000 7f5560c00640 1 -- 172.21.15.86:0/3138744845 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f554c03a040 con 0x7f55640cdeb0 2024-04-20T21:21:10.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.319+0000 7f5560c00640 1 -- 172.21.15.86:0/3138744845 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554c039070 con 0x7f55640cdeb0 2024-04-20T21:21:10.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.319+0000 7f556b200640 1 -- 172.21.15.86:0/3138744845 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55640cc5a0 con 0x7f55640cdeb0 2024-04-20T21:21:10.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.319+0000 7f556b200640 1 -- 172.21.15.86:0/3138744845 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55640ccb60 con 0x7f55640cdeb0 2024-04-20T21:21:10.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.320+0000 7f556b200640 1 -- 172.21.15.86:0/3138744845 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55640689e0 con 0x7f55640cdeb0 2024-04-20T21:21:10.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.321+0000 7f5560c00640 1 -- 172.21.15.86:0/3138744845 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f554c045020 con 0x7f55640cdeb0 2024-04-20T21:21:10.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.323+0000 7f5560c00640 1 --2- 172.21.15.86:0/3138744845 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f554006c920 0x7f554006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:10.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.323+0000 7f5560c00640 1 -- 172.21.15.86:0/3138744845 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f554c0ad1b0 con 0x7f55640cdeb0 2024-04-20T21:21:10.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.323+0000 7f5563e00640 1 --2- 172.21.15.86:0/3138744845 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f554006c920 0x7f554006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:10.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.324+0000 7f5563e00640 1 --2- 172.21.15.86:0/3138744845 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f554006c920 0x7f554006ede0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5554006a90 tx=0x7f5554008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:10.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.328+0000 7f5560c00640 1 -- 172.21.15.86:0/3138744845 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f554c04c070 con 0x7f55640cdeb0 2024-04-20T21:21:10.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:10.551+0000 7f556b200640 1 -- 172.21.15.86:0/3138744845 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f55640d2550 con 0x7f55640cdeb0 2024-04-20T21:21:11.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.107+0000 7f5560c00640 1 -- 172.21.15.86:0/3138744845 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v32) ==== 122+0+0 (secure 0 0 0) 0x7f554c07b4b0 con 0x7f55640cdeb0 2024-04-20T21:21:11.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.111+0000 7f556b200640 1 -- 172.21.15.86:0/3138744845 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f554006c920 msgr2=0x7f554006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:11.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.111+0000 7f556b200640 1 --2- 172.21.15.86:0/3138744845 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f554006c920 0x7f554006ede0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5554006a90 tx=0x7f5554008040 comp rx=0 tx=0).stop 2024-04-20T21:21:11.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.111+0000 7f556b200640 1 -- 172.21.15.86:0/3138744845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55640cdeb0 msgr2=0x7f55640ce310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:11.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.111+0000 7f556b200640 1 --2- 172.21.15.86:0/3138744845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55640cdeb0 0x7f55640ce310 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f554c037040 tx=0x7f554c004730 comp rx=0 tx=0).stop 2024-04-20T21:21:11.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.112+0000 7f556b200640 1 -- 172.21.15.86:0/3138744845 shutdown_connections 2024-04-20T21:21:11.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.112+0000 7f556b200640 1 --2- 172.21.15.86:0/3138744845 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f554006c920 0x7f554006ede0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:11.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.112+0000 7f556b200640 1 --2- 172.21.15.86:0/3138744845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f556415c580 0x7f556415ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:11.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.112+0000 7f556b200640 1 --2- 172.21.15.86:0/3138744845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55640cdeb0 0x7f55640ce310 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:11.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.112+0000 7f556b200640 1 --2- 172.21.15.86:0/3138744845 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5548003f30 0x7f5564148c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:11.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.112+0000 7f556b200640 1 -- 172.21.15.86:0/3138744845 >> 172.21.15.86:0/3138744845 conn(0x7f556406cc70 msgr2=0x7f556415d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:11.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.112+0000 7f556b200640 1 -- 172.21.15.86:0/3138744845 shutdown_connections 2024-04-20T21:21:11.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.112+0000 7f556b200640 1 -- 172.21.15.86:0/3138744845 wait complete. 2024-04-20T21:21:11.131 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-20T21:21:11.131 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:21:11.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.327+0000 7fad9f200640 1 Processor -- start 2024-04-20T21:21:11.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.328+0000 7fad9f200640 1 -- start start 2024-04-20T21:21:11.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.329+0000 7fad9f200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fad9815ece0 0x7fad9815f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:11.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.329+0000 7fad9f200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fad980d50c0 0x7fad980d3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:11.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.329+0000 7fad9f200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad9815f600 con 0x7fad980cf9c0 2024-04-20T21:21:11.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.329+0000 7fad9f200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fad980d3950 con 0x7fad9815ece0 2024-04-20T21:21:11.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.329+0000 7fad9f200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fad980cec00 con 0x7fad980d50c0 2024-04-20T21:21:11.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.329+0000 7fad9e800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fad980cf9c0 0x7fad98160000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:48454/0 (socket says 172.21.15.86:48454) 2024-04-20T21:21:11.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.329+0000 7fad9de00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fad9815ece0 0x7fad9815f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:11.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.329+0000 7fad9e800640 1 -- 172.21.15.86:0/1059890113 learned_addr learned my addr 172.21.15.86:0/1059890113 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:21:11.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.330+0000 7fad9d400640 1 --2- 172.21.15.86:0/1059890113 >> v2:172.21.15.144:3300/0 conn(0x7fad980d50c0 0x7fad980d3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:11.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.330+0000 7fad9de00640 1 -- 172.21.15.86:0/1059890113 >> v2:172.21.15.144:3300/0 conn(0x7fad980d50c0 msgr2=0x7fad980d3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:11.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.330+0000 7fad9de00640 1 --2- 172.21.15.86:0/1059890113 >> v2:172.21.15.144:3300/0 conn(0x7fad980d50c0 0x7fad980d3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:11.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.330+0000 7fad9de00640 1 -- 172.21.15.86:0/1059890113 >> v1:172.21.15.86:6789/0 conn(0x7fad980cf9c0 legacy=0x7fad98160000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:11.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.330+0000 7fad9ca00640 1 -- 172.21.15.86:0/1059890113 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2472103106 0 0) 0x7fad9815f600 con 0x7fad980cf9c0 2024-04-20T21:21:11.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.330+0000 7fad9de00640 1 -- 172.21.15.86:0/1059890113 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad980ceee0 con 0x7fad9815ece0 2024-04-20T21:21:11.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.330+0000 7fad9d400640 1 --2- 172.21.15.86:0/1059890113 >> v2:172.21.15.144:3300/0 conn(0x7fad980d50c0 0x7fad980d3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:21:11.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.331+0000 7fad9de00640 1 --2- 172.21.15.86:0/1059890113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad9815ece0 0x7fad9815f0c0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fad980ceac0 tx=0x7fad9002be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=342b490b3ae4b8a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:11.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.331+0000 7fad9ca00640 1 -- 172.21.15.86:0/1059890113 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad90039070 con 0x7fad9815ece0 2024-04-20T21:21:11.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.331+0000 7fad9ca00640 1 -- 172.21.15.86:0/1059890113 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad9002cbe0 con 0x7fad9815ece0 2024-04-20T21:21:11.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.331+0000 7fad9ca00640 1 -- 172.21.15.86:0/1059890113 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad90034440 con 0x7fad9815ece0 2024-04-20T21:21:11.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.332+0000 7fad9f200640 1 -- 172.21.15.86:0/1059890113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad9815ece0 msgr2=0x7fad9815f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:11.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.332+0000 7fad9f200640 1 --2- 172.21.15.86:0/1059890113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad9815ece0 0x7fad9815f0c0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fad980ceac0 tx=0x7fad9002be30 comp rx=0 tx=0).stop 2024-04-20T21:21:11.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.332+0000 7fad9f200640 1 -- 172.21.15.86:0/1059890113 shutdown_connections 2024-04-20T21:21:11.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.332+0000 7fad9f200640 1 --2- 172.21.15.86:0/1059890113 >> v2:172.21.15.144:3300/0 conn(0x7fad980d50c0 0x7fad980d3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:11.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.332+0000 7fad9f200640 1 --2- 172.21.15.86:0/1059890113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad9815ece0 0x7fad9815f0c0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:11.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.332+0000 7fad9f200640 1 -- 172.21.15.86:0/1059890113 >> 172.21.15.86:0/1059890113 conn(0x7fad9808bf70 msgr2=0x7fad980d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:11.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.332+0000 7fad9f200640 1 -- 172.21.15.86:0/1059890113 shutdown_connections 2024-04-20T21:21:11.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.332+0000 7fad9f200640 1 -- 172.21.15.86:0/1059890113 wait complete. 2024-04-20T21:21:11.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.334+0000 7fad9f200640 1 Processor -- start 2024-04-20T21:21:11.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.334+0000 7fad9f200640 1 -- start start 2024-04-20T21:21:11.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.335+0000 7fad9f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad980cf9c0 0x7fad9815d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:11.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.335+0000 7fad9f200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad980d50c0 0x7fad9809ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:11.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.335+0000 7fad9f200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad9815ece0 0x7fad9809f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:11.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.335+0000 7fad9f200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fad980d3640 con 0x7fad980cf9c0 2024-04-20T21:21:11.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.335+0000 7fad9f200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fad980cf4a0 con 0x7fad9815ece0 2024-04-20T21:21:11.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.335+0000 7fad9f200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fad9815ea50 con 0x7fad980d50c0 2024-04-20T21:21:11.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.335+0000 7fad9de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad980cf9c0 0x7fad9815d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:11.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.335+0000 7fad9de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad980cf9c0 0x7fad9815d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47784/0 (socket says 172.21.15.86:47784) 2024-04-20T21:21:11.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.335+0000 7fad9de00640 1 -- 172.21.15.86:0/27995965 learned_addr learned my addr 172.21.15.86:0/27995965 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:11.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.335+0000 7fad9e800640 1 --2- 172.21.15.86:0/27995965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad9815ece0 0x7fad9809f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:11.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.335+0000 7fad9de00640 1 -- 172.21.15.86:0/27995965 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad980d50c0 msgr2=0x7fad9809ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:11.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.335+0000 7fad9de00640 1 --2- 172.21.15.86:0/27995965 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad980d50c0 0x7fad9809ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:11.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.335+0000 7fad9de00640 1 -- 172.21.15.86:0/27995965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad9815ece0 msgr2=0x7fad9809f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:11.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.335+0000 7fad9de00640 1 --2- 172.21.15.86:0/27995965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad9815ece0 0x7fad9809f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:11.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.335+0000 7fad9de00640 1 -- 172.21.15.86:0/27995965 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad9815d900 con 0x7fad980cf9c0 2024-04-20T21:21:11.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.336+0000 7fad9de00640 1 --2- 172.21.15.86:0/27995965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad980cf9c0 0x7fad9815d3c0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fad90037040 tx=0x7fad90004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:11.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.336+0000 7fad8e000640 1 -- 172.21.15.86:0/27995965 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad90039050 con 0x7fad980cf9c0 2024-04-20T21:21:11.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.336+0000 7fad8e000640 1 -- 172.21.15.86:0/27995965 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad9003a040 con 0x7fad980cf9c0 2024-04-20T21:21:11.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.336+0000 7fad9f200640 1 -- 172.21.15.86:0/27995965 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad9809f730 con 0x7fad980cf9c0 2024-04-20T21:21:11.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.336+0000 7fad8e000640 1 -- 172.21.15.86:0/27995965 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9003e610 con 0x7fad980cf9c0 2024-04-20T21:21:11.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.336+0000 7fad9f200640 1 -- 172.21.15.86:0/27995965 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad981f8340 con 0x7fad980cf9c0 2024-04-20T21:21:11.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.337+0000 7fad9f200640 1 -- 172.21.15.86:0/27995965 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad9804fcf0 con 0x7fad980cf9c0 2024-04-20T21:21:11.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.338+0000 7fad8e000640 1 -- 172.21.15.86:0/27995965 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fad90047020 con 0x7fad980cf9c0 2024-04-20T21:21:11.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.339+0000 7fad8e000640 1 --2- 172.21.15.86:0/27995965 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fad7c06c8f0 0x7fad7c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:11.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.339+0000 7fad8e000640 1 -- 172.21.15.86:0/27995965 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fad900adf90 con 0x7fad980cf9c0 2024-04-20T21:21:11.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.339+0000 7fad9d400640 1 --2- 172.21.15.86:0/27995965 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fad7c06c8f0 0x7fad7c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:11.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.340+0000 7fad9d400640 1 --2- 172.21.15.86:0/27995965 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fad7c06c8f0 0x7fad7c06edb0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fad88006fd0 tx=0x7fad88008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:11.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.344+0000 7fad8e000640 1 -- 172.21.15.86:0/27995965 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fad9004e020 con 0x7fad980cf9c0 2024-04-20T21:21:11.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.585+0000 7fad9f200640 1 -- 172.21.15.86:0/27995965 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad98161130 con 0x7fad980cf9c0 2024-04-20T21:21:11.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.586+0000 7fad8e000640 1 -- 172.21.15.86:0/27995965 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+5205 (secure 0 0 0) 0x7fad9007c3b0 con 0x7fad980cf9c0 2024-04-20T21:21:11.585 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:21:11.585 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4913,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3933408964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3933408964},{"type":"v1","addr":"172.21.15.144:6833","nonce":3933408964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5221,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/1522070692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1522070692},{"type":"v1","addr":"172.21.15.144:6835","nonce":1522070692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":6132,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/1352073341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1352073341},{"type":"v1","addr":"172.21.15.86:6836","nonce":1352073341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:21:07.502800+0000","modified":"2024-04-20T21:21:10.553495+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6135},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6135":{"gid":6135,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3008383261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3008383261},{"type":"v1","addr":"172.21.15.86:6837","nonce":3008383261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6135,"qdb_cluster":[6135]},"id":4}]} 2024-04-20T21:21:11.585 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 32 2024-04-20T21:21:11.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.588+0000 7fad9f200640 1 -- 172.21.15.86:0/27995965 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fad7c06c8f0 msgr2=0x7fad7c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:11.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.588+0000 7fad9f200640 1 --2- 172.21.15.86:0/27995965 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fad7c06c8f0 0x7fad7c06edb0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fad88006fd0 tx=0x7fad88008040 comp rx=0 tx=0).stop 2024-04-20T21:21:11.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.588+0000 7fad9f200640 1 -- 172.21.15.86:0/27995965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad980cf9c0 msgr2=0x7fad9815d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:11.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.588+0000 7fad9f200640 1 --2- 172.21.15.86:0/27995965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad980cf9c0 0x7fad9815d3c0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fad90037040 tx=0x7fad90004a40 comp rx=0 tx=0).stop 2024-04-20T21:21:11.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.589+0000 7fad9f200640 1 -- 172.21.15.86:0/27995965 shutdown_connections 2024-04-20T21:21:11.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.589+0000 7fad9f200640 1 --2- 172.21.15.86:0/27995965 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fad7c06c8f0 0x7fad7c06edb0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:11.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.589+0000 7fad9f200640 1 --2- 172.21.15.86:0/27995965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad9815ece0 0x7fad9809f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:11.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.589+0000 7fad9f200640 1 --2- 172.21.15.86:0/27995965 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad980d50c0 0x7fad9809ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:11.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.589+0000 7fad9f200640 1 --2- 172.21.15.86:0/27995965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad980cf9c0 0x7fad9815d3c0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:11.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.589+0000 7fad9f200640 1 -- 172.21.15.86:0/27995965 >> 172.21.15.86:0/27995965 conn(0x7fad9808bf70 msgr2=0x7fad981621b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:11.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.589+0000 7fad9f200640 1 -- 172.21.15.86:0/27995965 shutdown_connections 2024-04-20T21:21:11.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.589+0000 7fad9f200640 1 -- 172.21.15.86:0/27995965 wait complete. 2024-04-20T21:21:11.607 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:21:11.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.807+0000 7f0c03200640 1 Processor -- start 2024-04-20T21:21:11.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.807+0000 7f0c03200640 1 -- start start 2024-04-20T21:21:11.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.809+0000 7f0c03200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c0415d6a0 0x7f0c040ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:11.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.809+0000 7f0c03200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c04160890 con 0x7f0c040ad620 2024-04-20T21:21:11.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.809+0000 7f0c03200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c04160b70 con 0x7f0c04156ae0 2024-04-20T21:21:11.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.809+0000 7f0c03200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0c0406db20 con 0x7f0c0415d6a0 2024-04-20T21:21:11.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.809+0000 7f0c02800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f0c040ad620 0x7f0c0406e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:48458/0 (socket says 172.21.15.86:48458) 2024-04-20T21:21:11.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.809+0000 7f0c02800640 1 -- 172.21.15.86:0/1240186553 learned_addr learned my addr 172.21.15.86:0/1240186553 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:21:11.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.810+0000 7f0c01400640 1 --2- 172.21.15.86:0/1240186553 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c0415d6a0 0x7f0c040ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:11.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.810+0000 7f0c00a00640 1 -- 172.21.15.86:0/1240186553 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1651998010 0 0) 0x7f0c04160890 con 0x7f0c040ad620 2024-04-20T21:21:11.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.810+0000 7f0c00a00640 1 -- 172.21.15.86:0/1240186553 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0be8003520 con 0x7f0c040ad620 2024-04-20T21:21:11.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.810+0000 7f0c00a00640 1 -- 172.21.15.86:0/1240186553 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1197122274 0 0) 0x7f0be8003520 con 0x7f0c040ad620 2024-04-20T21:21:11.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.810+0000 7f0c00a00640 1 -- 172.21.15.86:0/1240186553 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c0415d6a0 msgr2=0x7f0c040ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:11.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.810+0000 7f0c00a00640 1 --2- 172.21.15.86:0/1240186553 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c0415d6a0 0x7f0c040ad0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:11.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.810+0000 7f0c00a00640 1 -- 172.21.15.86:0/1240186553 >> v1:172.21.15.144:6789/0 conn(0x7f0c04156ae0 legacy=0x7f0c04160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:11.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.810+0000 7f0c00a00640 1 -- 172.21.15.86:0/1240186553 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c04160e50 con 0x7f0c040ad620 2024-04-20T21:21:11.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.810+0000 7f0c00a00640 1 -- 172.21.15.86:0/1240186553 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f0bfc001670 con 0x7f0c040ad620 2024-04-20T21:21:11.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.810+0000 7f0c00a00640 1 -- 172.21.15.86:0/1240186553 >> v1:172.21.15.86:6789/0 conn(0x7f0c040ad620 legacy=0x7f0c0406e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:11.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.810+0000 7f0c00a00640 1 --2- 172.21.15.86:0/1240186553 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c0415d6a0 0x7f0be80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:11.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.810+0000 7f0c00a00640 1 --2- 172.21.15.86:0/1240186553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0be8003f60 0x7f0be8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:11.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.810+0000 7f0c00a00640 1 --2- 172.21.15.86:0/1240186553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0be8004d40 0x7f0be80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:11.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.810+0000 7f0c00a00640 1 -- 172.21.15.86:0/1240186553 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0406db20 con 0x7f0be8004d40 2024-04-20T21:21:11.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.810+0000 7f0c00a00640 1 -- 172.21.15.86:0/1240186553 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0be800baa0 con 0x7f0be8003f60 2024-04-20T21:21:11.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.810+0000 7f0c00a00640 1 -- 172.21.15.86:0/1240186553 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0be800bca0 con 0x7f0c0415d6a0 2024-04-20T21:21:11.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.811+0000 7f0c02800640 1 --2- 172.21.15.86:0/1240186553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0be8004d40 0x7f0be80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:11.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.811+0000 7f0c01400640 1 --2- 172.21.15.86:0/1240186553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0be8003f60 0x7f0be8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:11.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.811+0000 7f0c02800640 1 -- 172.21.15.86:0/1240186553 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c0415d6a0 msgr2=0x7f0be80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:11.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.811+0000 7f0c01e00640 1 --2- 172.21.15.86:0/1240186553 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c0415d6a0 0x7f0be80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:11.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.811+0000 7f0c02800640 1 --2- 172.21.15.86:0/1240186553 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c0415d6a0 0x7f0be80039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:11.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.811+0000 7f0c02800640 1 -- 172.21.15.86:0/1240186553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0be8003f60 msgr2=0x7f0be8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:11.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.811+0000 7f0c02800640 1 --2- 172.21.15.86:0/1240186553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0be8003f60 0x7f0be8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:11.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.811+0000 7f0c02800640 1 -- 172.21.15.86:0/1240186553 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0be8003520 con 0x7f0be8004d40 2024-04-20T21:21:11.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.811+0000 7f0c01e00640 1 --2- 172.21.15.86:0/1240186553 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c0415d6a0 0x7f0be80039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:21:11.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.811+0000 7f0c01400640 1 --2- 172.21.15.86:0/1240186553 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0be8003f60 0x7f0be8004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:21:11.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.812+0000 7f0c02800640 1 --2- 172.21.15.86:0/1240186553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0be8004d40 0x7f0be80051c0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc00c210 tx=0x7f0be8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88413e5cc5c7b38b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:11.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.812+0000 7f0c00a00640 1 -- 172.21.15.86:0/1240186553 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc007d30 con 0x7f0be8004d40 2024-04-20T21:21:11.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.813+0000 7f0c00a00640 1 -- 172.21.15.86:0/1240186553 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bfc030070 con 0x7f0be8004d40 2024-04-20T21:21:11.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.813+0000 7f0c00a00640 1 -- 172.21.15.86:0/1240186553 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc02cda0 con 0x7f0be8004d40 2024-04-20T21:21:11.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.813+0000 7f0c03200640 1 -- 172.21.15.86:0/1240186553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0be8004d40 msgr2=0x7f0be80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:11.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.813+0000 7f0c03200640 1 --2- 172.21.15.86:0/1240186553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0be8004d40 0x7f0be80051c0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc00c210 tx=0x7f0be8005830 comp rx=0 tx=0).stop 2024-04-20T21:21:11.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.813+0000 7f0c02800640 1 -- 172.21.15.86:0/1240186553 reap_dead start 2024-04-20T21:21:11.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.813+0000 7f0c03200640 1 -- 172.21.15.86:0/1240186553 shutdown_connections 2024-04-20T21:21:11.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.813+0000 7f0c03200640 1 -- 172.21.15.86:0/1240186553 >> 172.21.15.86:0/1240186553 conn(0x7f0c040d0170 msgr2=0x7f0c040d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:11.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.813+0000 7f0c03200640 1 -- 172.21.15.86:0/1240186553 shutdown_connections 2024-04-20T21:21:11.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.813+0000 7f0c03200640 1 -- 172.21.15.86:0/1240186553 wait complete. 2024-04-20T21:21:11.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.815+0000 7f0c03200640 1 Processor -- start 2024-04-20T21:21:11.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.815+0000 7f0c03200640 1 -- start start 2024-04-20T21:21:11.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.816+0000 7f0c03200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c041f2fd0 0x7f0c040b4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:11.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.816+0000 7f0c03200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c041f33b0 0x7f0c040b4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:11.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.816+0000 7f0c03200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c0408bf70 0x7f0c0408c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:11.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.816+0000 7f0c03200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0c040ad0e0 con 0x7f0c041f33b0 2024-04-20T21:21:11.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.816+0000 7f0c03200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0c04156ec0 con 0x7f0c041f2fd0 2024-04-20T21:21:11.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.816+0000 7f0c03200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0c040ad260 con 0x7f0c0408bf70 2024-04-20T21:21:11.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.816+0000 7f0c01400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c041f33b0 0x7f0c040b4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:11.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.816+0000 7f0c01400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c041f33b0 0x7f0c040b4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47800/0 (socket says 172.21.15.86:47800) 2024-04-20T21:21:11.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.816+0000 7f0c01400640 1 -- 172.21.15.86:0/870717498 learned_addr learned my addr 172.21.15.86:0/870717498 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:11.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.817+0000 7f0c01e00640 1 --2- 172.21.15.86:0/870717498 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c041f2fd0 0x7f0c040b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:11.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.817+0000 7f0c02800640 1 --2- 172.21.15.86:0/870717498 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c0408bf70 0x7f0c0408c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:11.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.817+0000 7f0c01400640 1 -- 172.21.15.86:0/870717498 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c0408bf70 msgr2=0x7f0c0408c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:11.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.817+0000 7f0c01400640 1 --2- 172.21.15.86:0/870717498 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c0408bf70 0x7f0c0408c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:11.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.817+0000 7f0c01400640 1 -- 172.21.15.86:0/870717498 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c041f2fd0 msgr2=0x7f0c040b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:11.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.817+0000 7f0c01400640 1 --2- 172.21.15.86:0/870717498 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c041f2fd0 0x7f0c040b4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:11.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.817+0000 7f0c01400640 1 -- 172.21.15.86:0/870717498 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c0408c930 con 0x7f0c041f33b0 2024-04-20T21:21:11.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.817+0000 7f0c01400640 1 --2- 172.21.15.86:0/870717498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c041f33b0 0x7f0c040b4570 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f0bf00108b0 tx=0x7f0bf0010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:11.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.817+0000 7f0bfa000640 1 -- 172.21.15.86:0/870717498 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf000f040 con 0x7f0c041f33b0 2024-04-20T21:21:11.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.817+0000 7f0bfa000640 1 -- 172.21.15.86:0/870717498 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bf000eb60 con 0x7f0c041f33b0 2024-04-20T21:21:11.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.817+0000 7f0c03200640 1 -- 172.21.15.86:0/870717498 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c0408dc40 con 0x7f0c041f33b0 2024-04-20T21:21:11.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.817+0000 7f0bfa000640 1 -- 172.21.15.86:0/870717498 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf001c400 con 0x7f0c041f33b0 2024-04-20T21:21:11.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.818+0000 7f0c03200640 1 -- 172.21.15.86:0/870717498 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c0408ce80 con 0x7f0c041f33b0 2024-04-20T21:21:11.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.819+0000 7f0c03200640 1 -- 172.21.15.86:0/870717498 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c0404fcf0 con 0x7f0c041f33b0 2024-04-20T21:21:11.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.820+0000 7f0bfa000640 1 -- 172.21.15.86:0/870717498 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0bf001c5a0 con 0x7f0c041f33b0 2024-04-20T21:21:11.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.821+0000 7f0bfa000640 1 --2- 172.21.15.86:0/870717498 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0be406c920 0x7f0be406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:11.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.821+0000 7f0bfa000640 1 -- 172.21.15.86:0/870717498 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f0bf008c3a0 con 0x7f0c041f33b0 2024-04-20T21:21:11.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.821+0000 7f0c01e00640 1 --2- 172.21.15.86:0/870717498 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0be406c920 0x7f0be406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:11.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.822+0000 7f0c01e00640 1 --2- 172.21.15.86:0/870717498 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0be406c920 0x7f0be406ede0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4001e40 tx=0x7f0bf4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:11.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:11.825+0000 7f0bfa000640 1 -- 172.21.15.86:0/870717498 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0c0408dc40 con 0x7f0c041f33b0 2024-04-20T21:21:12.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.024+0000 7f0c03200640 1 -- 172.21.15.86:0/870717498 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0c0406fb40 con 0x7f0c041f33b0 2024-04-20T21:21:12.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.027+0000 7f0bfa000640 1 -- 172.21.15.86:0/870717498 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+16665 (secure 0 0 0) 0x7f0bf0018030 con 0x7f0c041f33b0 2024-04-20T21:21:12.026 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:21:12.027 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":59,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:21:09.072894+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:21:05.248922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-20T21:21:06.479064+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:21:12.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.029+0000 7f0c03200640 1 -- 172.21.15.86:0/870717498 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0be406c920 msgr2=0x7f0be406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:12.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.029+0000 7f0c03200640 1 --2- 172.21.15.86:0/870717498 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0be406c920 0x7f0be406ede0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4001e40 tx=0x7f0bf4009040 comp rx=0 tx=0).stop 2024-04-20T21:21:12.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.029+0000 7f0c03200640 1 -- 172.21.15.86:0/870717498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c041f33b0 msgr2=0x7f0c040b4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:12.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.029+0000 7f0c03200640 1 --2- 172.21.15.86:0/870717498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c041f33b0 0x7f0c040b4570 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f0bf00108b0 tx=0x7f0bf0010d70 comp rx=0 tx=0).stop 2024-04-20T21:21:12.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.029+0000 7f0c03200640 1 -- 172.21.15.86:0/870717498 shutdown_connections 2024-04-20T21:21:12.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.029+0000 7f0c03200640 1 --2- 172.21.15.86:0/870717498 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0be406c920 0x7f0be406ede0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:12.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.029+0000 7f0c03200640 1 --2- 172.21.15.86:0/870717498 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c0408bf70 0x7f0c0408c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:12.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.029+0000 7f0c03200640 1 --2- 172.21.15.86:0/870717498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c041f33b0 0x7f0c040b4570 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:12.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.030+0000 7f0c03200640 1 --2- 172.21.15.86:0/870717498 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c041f2fd0 0x7f0c040b4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:12.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.030+0000 7f0c03200640 1 -- 172.21.15.86:0/870717498 >> 172.21.15.86:0/870717498 conn(0x7f0c040d0170 msgr2=0x7f0c040b3510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:12.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.030+0000 7f0c03200640 1 -- 172.21.15.86:0/870717498 shutdown_connections 2024-04-20T21:21:12.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.030+0000 7f0c03200640 1 -- 172.21.15.86:0/870717498 wait complete. 2024-04-20T21:21:12.046 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-20T21:21:12.046 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T21:21:12.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.239+0000 7fde8ac00640 1 Processor -- start 2024-04-20T21:21:12.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.239+0000 7fde8ac00640 1 -- start start 2024-04-20T21:21:12.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.241+0000 7fde8ac00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde84156ae0 0x7fde8415d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:12.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.241+0000 7fde8ac00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fde840ad7f0 0x7fde8406e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:12.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.241+0000 7fde8ac00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde84160890 con 0x7fde84160080 2024-04-20T21:21:12.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.241+0000 7fde8ac00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fde8406db20 con 0x7fde840ad7f0 2024-04-20T21:21:12.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.241+0000 7fde8ac00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fde84160b70 con 0x7fde84156ae0 2024-04-20T21:21:12.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.241+0000 7fde8a200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fde840ad7f0 0x7fde8406e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:12.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.241+0000 7fde8a200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fde840ad7f0 0x7fde8406e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47802/0 (socket says 172.21.15.86:47802) 2024-04-20T21:21:12.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.241+0000 7fde8a200640 1 -- 172.21.15.86:0/2420557683 learned_addr learned my addr 172.21.15.86:0/2420557683 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:12.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.241+0000 7fde89800640 1 --2- 172.21.15.86:0/2420557683 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde84156ae0 0x7fde8415d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:12.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.242+0000 7fde8a200640 1 -- 172.21.15.86:0/2420557683 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde84156ae0 msgr2=0x7fde8415d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:12.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.242+0000 7fde8a200640 1 --2- 172.21.15.86:0/2420557683 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde84156ae0 0x7fde8415d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:12.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.242+0000 7fde8a200640 1 -- 172.21.15.86:0/2420557683 >> v1:172.21.15.144:6789/0 conn(0x7fde84160080 legacy=0x7fde840ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:12.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.242+0000 7fde8a200640 1 -- 172.21.15.86:0/2420557683 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde84160e50 con 0x7fde840ad7f0 2024-04-20T21:21:12.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.243+0000 7fde8a200640 1 --2- 172.21.15.86:0/2420557683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde840ad7f0 0x7fde8406e290 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fde7c009e50 tx=0x7fde7c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=117cb475de64dfb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:12.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.243+0000 7fde83e00640 1 -- 172.21.15.86:0/2420557683 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde7c02cb00 con 0x7fde840ad7f0 2024-04-20T21:21:12.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.243+0000 7fde83e00640 1 -- 172.21.15.86:0/2420557683 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde7c02cca0 con 0x7fde840ad7f0 2024-04-20T21:21:12.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.243+0000 7fde83e00640 1 -- 172.21.15.86:0/2420557683 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde7c034610 con 0x7fde840ad7f0 2024-04-20T21:21:12.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.244+0000 7fde8ac00640 1 -- 172.21.15.86:0/2420557683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde840ad7f0 msgr2=0x7fde8406e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:12.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.244+0000 7fde8ac00640 1 --2- 172.21.15.86:0/2420557683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde840ad7f0 0x7fde8406e290 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fde7c009e50 tx=0x7fde7c02bf60 comp rx=0 tx=0).stop 2024-04-20T21:21:12.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.244+0000 7fde8ac00640 1 -- 172.21.15.86:0/2420557683 shutdown_connections 2024-04-20T21:21:12.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.244+0000 7fde8ac00640 1 --2- 172.21.15.86:0/2420557683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde840ad7f0 0x7fde8406e290 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:12.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.244+0000 7fde8ac00640 1 --2- 172.21.15.86:0/2420557683 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde84156ae0 0x7fde8415d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:12.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.244+0000 7fde8ac00640 1 -- 172.21.15.86:0/2420557683 >> 172.21.15.86:0/2420557683 conn(0x7fde840d0170 msgr2=0x7fde840d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:12.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.244+0000 7fde8ac00640 1 -- 172.21.15.86:0/2420557683 shutdown_connections 2024-04-20T21:21:12.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.244+0000 7fde8ac00640 1 -- 172.21.15.86:0/2420557683 wait complete. 2024-04-20T21:21:12.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.246+0000 7fde8ac00640 1 Processor -- start 2024-04-20T21:21:12.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.246+0000 7fde8ac00640 1 -- start start 2024-04-20T21:21:12.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.247+0000 7fde8ac00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde840ad7f0 0x7fde8409a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:12.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.247+0000 7fde8ac00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde84156ae0 0x7fde84155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:12.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.247+0000 7fde8ac00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde84160080 0x7fde8409d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:12.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.247+0000 7fde8ac00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fde84160500 con 0x7fde84160080 2024-04-20T21:21:12.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.247+0000 7fde8ac00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fde84156ec0 con 0x7fde84156ae0 2024-04-20T21:21:12.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.247+0000 7fde8ac00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fde8406e7d0 con 0x7fde840ad7f0 2024-04-20T21:21:12.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.247+0000 7fde8a200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde84160080 0x7fde8409d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:12.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.247+0000 7fde8a200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde84160080 0x7fde8409d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47812/0 (socket says 172.21.15.86:47812) 2024-04-20T21:21:12.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.247+0000 7fde8a200640 1 -- 172.21.15.86:0/1477602195 learned_addr learned my addr 172.21.15.86:0/1477602195 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:12.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.248+0000 7fde88e00640 1 --2- 172.21.15.86:0/1477602195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde84156ae0 0x7fde84155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:12.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.248+0000 7fde8a200640 1 -- 172.21.15.86:0/1477602195 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde840ad7f0 msgr2=0x7fde8409a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:12.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.248+0000 7fde89800640 1 --2- 172.21.15.86:0/1477602195 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde840ad7f0 0x7fde8409a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:12.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.248+0000 7fde8a200640 1 --2- 172.21.15.86:0/1477602195 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde840ad7f0 0x7fde8409a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:12.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.248+0000 7fde8a200640 1 -- 172.21.15.86:0/1477602195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde84156ae0 msgr2=0x7fde84155390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:12.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.248+0000 7fde8a200640 1 --2- 172.21.15.86:0/1477602195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde84156ae0 0x7fde84155390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:12.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.248+0000 7fde8a200640 1 -- 172.21.15.86:0/1477602195 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde841558d0 con 0x7fde84160080 2024-04-20T21:21:12.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.248+0000 7fde88e00640 1 --2- 172.21.15.86:0/1477602195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde84156ae0 0x7fde84155390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:21:12.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.248+0000 7fde89800640 1 --2- 172.21.15.86:0/1477602195 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde840ad7f0 0x7fde8409a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:21:12.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.248+0000 7fde8a200640 1 --2- 172.21.15.86:0/1477602195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde84160080 0x7fde8409d730 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fde7c036040 tx=0x7fde7c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:12.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.248+0000 7fde81600640 1 -- 172.21.15.86:0/1477602195 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde7c043020 con 0x7fde84160080 2024-04-20T21:21:12.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.248+0000 7fde81600640 1 -- 172.21.15.86:0/1477602195 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde7c02ce50 con 0x7fde84160080 2024-04-20T21:21:12.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.248+0000 7fde81600640 1 -- 172.21.15.86:0/1477602195 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde7c009510 con 0x7fde84160080 2024-04-20T21:21:12.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.248+0000 7fde8ac00640 1 -- 172.21.15.86:0/1477602195 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde841f1ea0 con 0x7fde84160080 2024-04-20T21:21:12.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.249+0000 7fde8ac00640 1 -- 172.21.15.86:0/1477602195 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde84158a20 con 0x7fde84160080 2024-04-20T21:21:12.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.250+0000 7fde8ac00640 1 -- 172.21.15.86:0/1477602195 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde8404fcf0 con 0x7fde84160080 2024-04-20T21:21:12.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.251+0000 7fde81600640 1 -- 172.21.15.86:0/1477602195 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fde7c046020 con 0x7fde84160080 2024-04-20T21:21:12.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.252+0000 7fde81600640 1 --2- 172.21.15.86:0/1477602195 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fde6406c8f0 0x7fde6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:12.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.252+0000 7fde81600640 1 -- 172.21.15.86:0/1477602195 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fde7c0ae220 con 0x7fde84160080 2024-04-20T21:21:12.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.252+0000 7fde89800640 1 --2- 172.21.15.86:0/1477602195 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fde6406c8f0 0x7fde6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:12.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.252+0000 7fde89800640 1 --2- 172.21.15.86:0/1477602195 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fde6406c8f0 0x7fde6406edb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fde78006fd0 tx=0x7fde78008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:12.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.256+0000 7fde81600640 1 -- 172.21.15.86:0/1477602195 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fde7c04d030 con 0x7fde84160080 2024-04-20T21:21:12.431 INFO:tasks.ceph.mds.c.smithi086.stderr:2024-04-20T21:21:12.432+0000 7fb614600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T21:21:12.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.436+0000 7fde8ac00640 1 -- 172.21.15.86:0/1477602195 --> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fde8406fb40 con 0x7fde6406c8f0 2024-04-20T21:21:12.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.444+0000 7fde81600640 1 -- 172.21.15.86:0/1477602195 <== mgr.4107 v2:172.21.15.86:6832/302045240 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+237202 (secure 0 0 0) 0x7fde8406fb40 con 0x7fde6406c8f0 2024-04-20T21:21:12.446 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:21:12.448 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-20T21:21:12.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.449+0000 7fde8ac00640 1 -- 172.21.15.86:0/1477602195 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fde6406c8f0 msgr2=0x7fde6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:12.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.449+0000 7fde8ac00640 1 --2- 172.21.15.86:0/1477602195 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fde6406c8f0 0x7fde6406edb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fde78006fd0 tx=0x7fde78008040 comp rx=0 tx=0).stop 2024-04-20T21:21:12.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.449+0000 7fde8ac00640 1 -- 172.21.15.86:0/1477602195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde84160080 msgr2=0x7fde8409d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:12.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.449+0000 7fde8ac00640 1 --2- 172.21.15.86:0/1477602195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde84160080 0x7fde8409d730 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fde7c036040 tx=0x7fde7c004a40 comp rx=0 tx=0).stop 2024-04-20T21:21:12.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.450+0000 7fde8ac00640 1 -- 172.21.15.86:0/1477602195 shutdown_connections 2024-04-20T21:21:12.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.450+0000 7fde8ac00640 1 --2- 172.21.15.86:0/1477602195 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fde6406c8f0 0x7fde6406edb0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:12.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.450+0000 7fde8ac00640 1 --2- 172.21.15.86:0/1477602195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde84160080 0x7fde8409d730 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:12.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.450+0000 7fde8ac00640 1 --2- 172.21.15.86:0/1477602195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde84156ae0 0x7fde84155390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:12.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.450+0000 7fde8ac00640 1 --2- 172.21.15.86:0/1477602195 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde840ad7f0 0x7fde8409a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:12.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.450+0000 7fde8ac00640 1 -- 172.21.15.86:0/1477602195 >> 172.21.15.86:0/1477602195 conn(0x7fde840d0170 msgr2=0x7fde8406ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:12.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.450+0000 7fde8ac00640 1 -- 172.21.15.86:0/1477602195 shutdown_connections 2024-04-20T21:21:12.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.450+0000 7fde8ac00640 1 -- 172.21.15.86:0/1477602195 wait complete. 2024-04-20T21:21:12.467 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":157,"stamp":"2024-04-20T21:21:10.694867+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":234984,"kb_used_data":6312,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754739736,"statfs":{"total":773094113280,"available":772853489664,"internally_reserved":0,"allocated":6463488,"data_stored":3345704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":234016304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.672747"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:06.006425+0000","last_change":"2024-04-20T21:21:06.006425+0000","last_active":"2024-04-20T21:21:06.006425+0000","last_peered":"2024-04-20T21:21:06.006425+0000","last_clean":"2024-04-20T21:21:06.006425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:06.006425+0000","last_undegraded":"2024-04-20T21:21:06.006425+0000","last_fullsized":"2024-04-20T21:21:06.006425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:06.006425+0000","last_change":"2024-04-20T21:21:06.006425+0000","last_active":"2024-04-20T21:21:06.006425+0000","last_peered":"2024-04-20T21:21:06.006425+0000","last_clean":"2024-04-20T21:21:06.006425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:06.006425+0000","last_undegraded":"2024-04-20T21:21:06.006425+0000","last_fullsized":"2024-04-20T21:21:06.006425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:06.006425+0000","last_change":"2024-04-20T21:21:06.006425+0000","last_active":"2024-04-20T21:21:06.006425+0000","last_peered":"2024-04-20T21:21:06.006425+0000","last_clean":"2024-04-20T21:21:06.006425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:06.006425+0000","last_undegraded":"2024-04-20T21:21:06.006425+0000","last_fullsized":"2024-04-20T21:21:06.006425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:06.006425+0000","last_change":"2024-04-20T21:21:06.006425+0000","last_active":"2024-04-20T21:21:06.006425+0000","last_peered":"2024-04-20T21:21:06.006425+0000","last_clean":"2024-04-20T21:21:06.006425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:06.006425+0000","last_undegraded":"2024-04-20T21:21:06.006425+0000","last_fullsized":"2024-04-20T21:21:06.006425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:06.006425+0000","last_change":"2024-04-20T21:21:06.006425+0000","last_active":"2024-04-20T21:21:06.006425+0000","last_peered":"2024-04-20T21:21:06.006425+0000","last_clean":"2024-04-20T21:21:06.006425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:06.006425+0000","last_undegraded":"2024-04-20T21:21:06.006425+0000","last_fullsized":"2024-04-20T21:21:06.006425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:06.006425+0000","last_change":"2024-04-20T21:21:06.006425+0000","last_active":"2024-04-20T21:21:06.006425+0000","last_peered":"2024-04-20T21:21:06.006425+0000","last_clean":"2024-04-20T21:21:06.006425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:06.006425+0000","last_undegraded":"2024-04-20T21:21:06.006425+0000","last_fullsized":"2024-04-20T21:21:06.006425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:06.006425+0000","last_change":"2024-04-20T21:21:06.006425+0000","last_active":"2024-04-20T21:21:06.006425+0000","last_peered":"2024-04-20T21:21:06.006425+0000","last_clean":"2024-04-20T21:21:06.006425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:06.006425+0000","last_undegraded":"2024-04-20T21:21:06.006425+0000","last_fullsized":"2024-04-20T21:21:06.006425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:06.006425+0000","last_change":"2024-04-20T21:21:06.006425+0000","last_active":"2024-04-20T21:21:06.006425+0000","last_peered":"2024-04-20T21:21:06.006425+0000","last_clean":"2024-04-20T21:21:06.006425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:06.006425+0000","last_undegraded":"2024-04-20T21:21:06.006425+0000","last_fullsized":"2024-04-20T21:21:06.006425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:06.006425+0000","last_change":"2024-04-20T21:21:06.006425+0000","last_active":"2024-04-20T21:21:06.006425+0000","last_peered":"2024-04-20T21:21:06.006425+0000","last_clean":"2024-04-20T21:21:06.006425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:06.006425+0000","last_undegraded":"2024-04-20T21:21:06.006425+0000","last_fullsized":"2024-04-20T21:21:06.006425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:06.006425+0000","last_change":"2024-04-20T21:21:06.006425+0000","last_active":"2024-04-20T21:21:06.006425+0000","last_peered":"2024-04-20T21:21:06.006425+0000","last_clean":"2024-04-20T21:21:06.006425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:06.006425+0000","last_undegraded":"2024-04-20T21:21:06.006425+0000","last_fullsized":"2024-04-20T21:21:06.006425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:06.006425+0000","last_change":"2024-04-20T21:21:06.006425+0000","last_active":"2024-04-20T21:21:06.006425+0000","last_peered":"2024-04-20T21:21:06.006425+0000","last_clean":"2024-04-20T21:21:06.006425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:06.006425+0000","last_undegraded":"2024-04-20T21:21:06.006425+0000","last_fullsized":"2024-04-20T21:21:06.006425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:06.006425+0000","last_change":"2024-04-20T21:21:06.006425+0000","last_active":"2024-04-20T21:21:06.006425+0000","last_peered":"2024-04-20T21:21:06.006425+0000","last_clean":"2024-04-20T21:21:06.006425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:06.006425+0000","last_undegraded":"2024-04-20T21:21:06.006425+0000","last_fullsized":"2024-04-20T21:21:06.006425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:06.006425+0000","last_change":"2024-04-20T21:21:06.006425+0000","last_active":"2024-04-20T21:21:06.006425+0000","last_peered":"2024-04-20T21:21:06.006425+0000","last_clean":"2024-04-20T21:21:06.006425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:06.006425+0000","last_undegraded":"2024-04-20T21:21:06.006425+0000","last_fullsized":"2024-04-20T21:21:06.006425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:06.006425+0000","last_change":"2024-04-20T21:21:06.006425+0000","last_active":"2024-04-20T21:21:06.006425+0000","last_peered":"2024-04-20T21:21:06.006425+0000","last_clean":"2024-04-20T21:21:06.006425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:06.006425+0000","last_undegraded":"2024-04-20T21:21:06.006425+0000","last_fullsized":"2024-04-20T21:21:06.006425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:06.006425+0000","last_change":"2024-04-20T21:21:06.006425+0000","last_active":"2024-04-20T21:21:06.006425+0000","last_peered":"2024-04-20T21:21:06.006425+0000","last_clean":"2024-04-20T21:21:06.006425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:06.006425+0000","last_undegraded":"2024-04-20T21:21:06.006425+0000","last_fullsized":"2024-04-20T21:21:06.006425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:06.006425+0000","last_change":"2024-04-20T21:21:06.006425+0000","last_active":"2024-04-20T21:21:06.006425+0000","last_peered":"2024-04-20T21:21:06.006425+0000","last_clean":"2024-04-20T21:21:06.006425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:06.006425+0000","last_undegraded":"2024-04-20T21:21:06.006425+0000","last_fullsized":"2024-04-20T21:21:06.006425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:06.006425+0000","last_change":"2024-04-20T21:21:06.006425+0000","last_active":"2024-04-20T21:21:06.006425+0000","last_peered":"2024-04-20T21:21:06.006425+0000","last_clean":"2024-04-20T21:21:06.006425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:06.006425+0000","last_undegraded":"2024-04-20T21:21:06.006425+0000","last_fullsized":"2024-04-20T21:21:06.006425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:06.006425+0000","last_change":"2024-04-20T21:21:06.006425+0000","last_active":"2024-04-20T21:21:06.006425+0000","last_peered":"2024-04-20T21:21:06.006425+0000","last_clean":"2024-04-20T21:21:06.006425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:06.006425+0000","last_undegraded":"2024-04-20T21:21:06.006425+0000","last_fullsized":"2024-04-20T21:21:06.006425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:06.006425+0000","last_change":"2024-04-20T21:21:06.006425+0000","last_active":"2024-04-20T21:21:06.006425+0000","last_peered":"2024-04-20T21:21:06.006425+0000","last_clean":"2024-04-20T21:21:06.006425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:06.006425+0000","last_undegraded":"2024-04-20T21:21:06.006425+0000","last_fullsized":"2024-04-20T21:21:06.006425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:06.006425+0000","last_change":"2024-04-20T21:21:06.006425+0000","last_active":"2024-04-20T21:21:06.006425+0000","last_peered":"2024-04-20T21:21:06.006425+0000","last_clean":"2024-04-20T21:21:06.006425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:06.006425+0000","last_undegraded":"2024-04-20T21:21:06.006425+0000","last_fullsized":"2024-04-20T21:21:06.006425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:06.006425+0000","last_change":"2024-04-20T21:21:06.006425+0000","last_active":"2024-04-20T21:21:06.006425+0000","last_peered":"2024-04-20T21:21:06.006425+0000","last_clean":"2024-04-20T21:21:06.006425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:06.006425+0000","last_undegraded":"2024-04-20T21:21:06.006425+0000","last_fullsized":"2024-04-20T21:21:06.006425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:06.006425+0000","last_change":"2024-04-20T21:21:06.006425+0000","last_active":"2024-04-20T21:21:06.006425+0000","last_peered":"2024-04-20T21:21:06.006425+0000","last_clean":"2024-04-20T21:21:06.006425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:06.006425+0000","last_undegraded":"2024-04-20T21:21:06.006425+0000","last_fullsized":"2024-04-20T21:21:06.006425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:06.006425+0000","last_change":"2024-04-20T21:21:06.006425+0000","last_active":"2024-04-20T21:21:06.006425+0000","last_peered":"2024-04-20T21:21:06.006425+0000","last_clean":"2024-04-20T21:21:06.006425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:06.006425+0000","last_undegraded":"2024-04-20T21:21:06.006425+0000","last_fullsized":"2024-04-20T21:21:06.006425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:06.006425+0000","last_change":"2024-04-20T21:21:06.006425+0000","last_active":"2024-04-20T21:21:06.006425+0000","last_peered":"2024-04-20T21:21:06.006425+0000","last_clean":"2024-04-20T21:21:06.006425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:06.006425+0000","last_undegraded":"2024-04-20T21:21:06.006425+0000","last_fullsized":"2024-04-20T21:21:06.006425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:06.006425+0000","last_change":"2024-04-20T21:21:06.006425+0000","last_active":"2024-04-20T21:21:06.006425+0000","last_peered":"2024-04-20T21:21:06.006425+0000","last_clean":"2024-04-20T21:21:06.006425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:06.006425+0000","last_undegraded":"2024-04-20T21:21:06.006425+0000","last_fullsized":"2024-04-20T21:21:06.006425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:06.006425+0000","last_change":"2024-04-20T21:21:06.006425+0000","last_active":"2024-04-20T21:21:06.006425+0000","last_peered":"2024-04-20T21:21:06.006425+0000","last_clean":"2024-04-20T21:21:06.006425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:06.006425+0000","last_undegraded":"2024-04-20T21:21:06.006425+0000","last_fullsized":"2024-04-20T21:21:06.006425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:06.006425+0000","last_change":"2024-04-20T21:21:06.006425+0000","last_active":"2024-04-20T21:21:06.006425+0000","last_peered":"2024-04-20T21:21:06.006425+0000","last_clean":"2024-04-20T21:21:06.006425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:06.006425+0000","last_undegraded":"2024-04-20T21:21:06.006425+0000","last_fullsized":"2024-04-20T21:21:06.006425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:06.006425+0000","last_change":"2024-04-20T21:21:06.006425+0000","last_active":"2024-04-20T21:21:06.006425+0000","last_peered":"2024-04-20T21:21:06.006425+0000","last_clean":"2024-04-20T21:21:06.006425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:06.006425+0000","last_undegraded":"2024-04-20T21:21:06.006425+0000","last_fullsized":"2024-04-20T21:21:06.006425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:06.006425+0000","last_change":"2024-04-20T21:21:06.006425+0000","last_active":"2024-04-20T21:21:06.006425+0000","last_peered":"2024-04-20T21:21:06.006425+0000","last_clean":"2024-04-20T21:21:06.006425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:06.006425+0000","last_undegraded":"2024-04-20T21:21:06.006425+0000","last_fullsized":"2024-04-20T21:21:06.006425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:06.006425+0000","last_change":"2024-04-20T21:21:06.006425+0000","last_active":"2024-04-20T21:21:06.006425+0000","last_peered":"2024-04-20T21:21:06.006425+0000","last_clean":"2024-04-20T21:21:06.006425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:06.006425+0000","last_undegraded":"2024-04-20T21:21:06.006425+0000","last_fullsized":"2024-04-20T21:21:06.006425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:06.006425+0000","last_change":"2024-04-20T21:21:06.006425+0000","last_active":"2024-04-20T21:21:06.006425+0000","last_peered":"2024-04-20T21:21:06.006425+0000","last_clean":"2024-04-20T21:21:06.006425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:06.006425+0000","last_undegraded":"2024-04-20T21:21:06.006425+0000","last_fullsized":"2024-04-20T21:21:06.006425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:07.020801+0000","last_change":"2024-04-20T21:21:07.020801+0000","last_active":"2024-04-20T21:21:07.020801+0000","last_peered":"2024-04-20T21:21:07.020801+0000","last_clean":"2024-04-20T21:21:07.020801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:07.020801+0000","last_undegraded":"2024-04-20T21:21:07.020801+0000","last_fullsized":"2024-04-20T21:21:07.020801+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T21:21:06.006425+0000","last_change":"2024-04-20T21:21:06.006425+0000","last_active":"2024-04-20T21:21:06.006425+0000","last_peered":"2024-04-20T21:21:06.006425+0000","last_clean":"2024-04-20T21:21:06.006425+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:21:06.006425+0000","last_undegraded":"2024-04-20T21:21:06.006425+0000","last_fullsized":"2024-04-20T21:21:06.006425+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":230,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-20T21:21:03.003189+0000","last_change":"2024-04-20T21:17:43.059637+0000","last_active":"2024-04-20T21:21:03.003189+0000","last_peered":"2024-04-20T21:21:03.003189+0000","last_clean":"2024-04-20T21:21:03.003189+0000","last_became_active":"2024-04-20T21:17:43.056207+0000","last_became_peered":"2024-04-20T21:17:43.056207+0000","last_unstale":"2024-04-20T21:21:03.003189+0000","last_undegraded":"2024-04-20T21:21:03.003189+0000","last_fullsized":"2024-04-20T21:21:03.003189+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:41.036290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:41.036290+0000","last_clean_scrub_stamp":"2024-04-20T21:17:41.036290+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:58:51.091337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574891,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32408,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339432,"statfs":{"total":96636764160,"available":96603578368,"internally_reserved":0,"allocated":1253376,"data_stored":860989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27680,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":659456,"data_stored":270621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27664,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":659456,"data_stored":270621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27664,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":659456,"data_stored":270621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27668,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":659456,"data_stored":270621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27672,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":659456,"data_stored":270621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31824,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340016,"statfs":{"total":96636764160,"available":96604176384,"internally_reserved":0,"allocated":659456,"data_stored":270621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574893,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32404,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339436,"statfs":{"total":96636764160,"available":96603582464,"internally_reserved":0,"allocated":1253376,"data_stored":860989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T21:21:12.472 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T21:21:12.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.712+0000 7f2d1da00640 1 Processor -- start 2024-04-20T21:21:12.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.712+0000 7f2d1da00640 1 -- start start 2024-04-20T21:21:12.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.714+0000 7f2d1da00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2d180af7b0 0x7f2d1815c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:12.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.714+0000 7f2d1da00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d180ae710 con 0x7f2d1815c010 2024-04-20T21:21:12.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.714+0000 7f2d1da00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d180ae9f0 con 0x7f2d1806f4c0 2024-04-20T21:21:12.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.714+0000 7f2d1da00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f2d180af320 con 0x7f2d180af7b0 2024-04-20T21:21:12.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.715+0000 7f2d17e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f2d1815c010 0x7f2d180adda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:48466/0 (socket says 172.21.15.86:48466) 2024-04-20T21:21:12.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.715+0000 7f2d17e00640 1 -- 172.21.15.86:0/2849806431 learned_addr learned my addr 172.21.15.86:0/2849806431 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:21:12.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.715+0000 7f2d17400640 1 --2- 172.21.15.86:0/2849806431 >> v2:172.21.15.144:3300/0 conn(0x7f2d180af7b0 0x7f2d1815c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:12.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.715+0000 7f2d16a00640 1 -- 172.21.15.86:0/2849806431 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4266253475 0 0) 0x7f2d180ae710 con 0x7f2d1815c010 2024-04-20T21:21:12.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.715+0000 7f2d16a00640 1 -- 172.21.15.86:0/2849806431 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2cfc003520 con 0x7f2d1815c010 2024-04-20T21:21:12.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.715+0000 7f2d16a00640 1 -- 172.21.15.86:0/2849806431 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3939956966 0 0) 0x7f2cfc003520 con 0x7f2d1815c010 2024-04-20T21:21:12.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.715+0000 7f2d16a00640 1 -- 172.21.15.86:0/2849806431 >> v2:172.21.15.144:3300/0 conn(0x7f2d180af7b0 msgr2=0x7f2d1815c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:12.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.715+0000 7f2d16a00640 1 --2- 172.21.15.86:0/2849806431 >> v2:172.21.15.144:3300/0 conn(0x7f2d180af7b0 0x7f2d1815c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:12.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.715+0000 7f2d16a00640 1 -- 172.21.15.86:0/2849806431 >> v1:172.21.15.144:6789/0 conn(0x7f2d1806f4c0 legacy=0x7f2d18071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:12.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.715+0000 7f2d16a00640 1 -- 172.21.15.86:0/2849806431 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d180aecd0 con 0x7f2d1815c010 2024-04-20T21:21:12.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.715+0000 7f2d16a00640 1 -- 172.21.15.86:0/2849806431 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f2d08001670 con 0x7f2d1815c010 2024-04-20T21:21:12.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.715+0000 7f2d16a00640 1 -- 172.21.15.86:0/2849806431 >> v1:172.21.15.86:6789/0 conn(0x7f2d1815c010 legacy=0x7f2d180adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:12.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.716+0000 7f2d16a00640 1 --2- 172.21.15.86:0/2849806431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cfc0039b0 0x7f2cfc003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:12.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.716+0000 7f2d16a00640 1 --2- 172.21.15.86:0/2849806431 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2cfc004800 0x7f2cfc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:12.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.716+0000 7f2d16a00640 1 --2- 172.21.15.86:0/2849806431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cfc0051c0 0x7f2cfc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:12.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.716+0000 7f2d16a00640 1 -- 172.21.15.86:0/2849806431 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2d180af320 con 0x7f2cfc0039b0 2024-04-20T21:21:12.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.716+0000 7f2d16a00640 1 -- 172.21.15.86:0/2849806431 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2cfc00c040 con 0x7f2cfc0051c0 2024-04-20T21:21:12.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.716+0000 7f2d16a00640 1 -- 172.21.15.86:0/2849806431 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2cfc00c240 con 0x7f2cfc004800 2024-04-20T21:21:12.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.716+0000 7f2d17e00640 1 --2- 172.21.15.86:0/2849806431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cfc0039b0 0x7f2cfc003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:12.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.716+0000 7f2d17400640 1 --2- 172.21.15.86:0/2849806431 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2cfc004800 0x7f2cfc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:12.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.716+0000 7f2d17e00640 1 -- 172.21.15.86:0/2849806431 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2cfc004800 msgr2=0x7f2cfc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:12.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.716+0000 7f2d17e00640 1 --2- 172.21.15.86:0/2849806431 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2cfc004800 0x7f2cfc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:12.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.716+0000 7f2d1d000640 1 --2- 172.21.15.86:0/2849806431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cfc0051c0 0x7f2cfc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:12.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.716+0000 7f2d17e00640 1 -- 172.21.15.86:0/2849806431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cfc0051c0 msgr2=0x7f2cfc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:12.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.716+0000 7f2d17e00640 1 --2- 172.21.15.86:0/2849806431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cfc0051c0 0x7f2cfc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:12.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.717+0000 7f2d17e00640 1 -- 172.21.15.86:0/2849806431 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cfc003520 con 0x7f2cfc0039b0 2024-04-20T21:21:12.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.717+0000 7f2d1d000640 1 --2- 172.21.15.86:0/2849806431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cfc0051c0 0x7f2cfc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:21:12.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.717+0000 7f2d1d000640 1 -- 172.21.15.86:0/2849806431 reap_dead start 2024-04-20T21:21:12.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.718+0000 7f2d17e00640 1 --2- 172.21.15.86:0/2849806431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cfc0039b0 0x7f2cfc003db0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f2d08007cf0 tx=0x7f2cfc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=109bd235899c76a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:12.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.718+0000 7f2d16a00640 1 -- 172.21.15.86:0/2849806431 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d08007d30 con 0x7f2cfc0039b0 2024-04-20T21:21:12.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.718+0000 7f2d16a00640 1 -- 172.21.15.86:0/2849806431 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d08032070 con 0x7f2cfc0039b0 2024-04-20T21:21:12.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.718+0000 7f2d16a00640 1 -- 172.21.15.86:0/2849806431 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d080058d0 con 0x7f2cfc0039b0 2024-04-20T21:21:12.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.719+0000 7f2d1da00640 1 -- 172.21.15.86:0/2849806431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cfc0039b0 msgr2=0x7f2cfc003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:12.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.719+0000 7f2d1da00640 1 --2- 172.21.15.86:0/2849806431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cfc0039b0 0x7f2cfc003db0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f2d08007cf0 tx=0x7f2cfc005830 comp rx=0 tx=0).stop 2024-04-20T21:21:12.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.719+0000 7f2d1da00640 1 -- 172.21.15.86:0/2849806431 shutdown_connections 2024-04-20T21:21:12.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.719+0000 7f2d1da00640 1 --2- 172.21.15.86:0/2849806431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2cfc0039b0 0x7f2cfc003db0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:12.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.719+0000 7f2d1da00640 1 -- 172.21.15.86:0/2849806431 >> 172.21.15.86:0/2849806431 conn(0x7f2d180d0170 msgr2=0x7f2d180d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:12.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.719+0000 7f2d1da00640 1 -- 172.21.15.86:0/2849806431 shutdown_connections 2024-04-20T21:21:12.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.719+0000 7f2d1da00640 1 -- 172.21.15.86:0/2849806431 wait complete. 2024-04-20T21:21:12.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.721+0000 7f2d1da00640 1 Processor -- start 2024-04-20T21:21:12.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.722+0000 7f2d1da00640 1 -- start start 2024-04-20T21:21:12.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.723+0000 7f2d1da00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cfc0039b0 0x7f2d181f3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:12.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.723+0000 7f2d1da00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d1814b980 0x7f2d1814bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:12.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.723+0000 7f2d1da00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d181577b0 0x7f2d18157c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:12.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.723+0000 7f2d1da00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1815c570 con 0x7f2d1814b980 2024-04-20T21:21:12.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.723+0000 7f2d1da00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1815c3f0 con 0x7f2cfc0039b0 2024-04-20T21:21:12.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.723+0000 7f2d1da00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2d1815c6f0 con 0x7f2d181577b0 2024-04-20T21:21:12.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.723+0000 7f2d17400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d1814b980 0x7f2d1814bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:12.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.723+0000 7f2d17400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d1814b980 0x7f2d1814bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47820/0 (socket says 172.21.15.86:47820) 2024-04-20T21:21:12.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.723+0000 7f2d17400640 1 -- 172.21.15.86:0/2625120005 learned_addr learned my addr 172.21.15.86:0/2625120005 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:12.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.724+0000 7f2d17e00640 1 --2- 172.21.15.86:0/2625120005 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cfc0039b0 0x7f2d181f3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:12.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.724+0000 7f2d1d000640 1 --2- 172.21.15.86:0/2625120005 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d181577b0 0x7f2d18157c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:12.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.724+0000 7f2d17400640 1 -- 172.21.15.86:0/2625120005 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d181577b0 msgr2=0x7f2d18157c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:12.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.724+0000 7f2d17400640 1 --2- 172.21.15.86:0/2625120005 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d181577b0 0x7f2d18157c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:12.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.724+0000 7f2d17400640 1 -- 172.21.15.86:0/2625120005 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cfc0039b0 msgr2=0x7f2d181f3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:12.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.724+0000 7f2d17400640 1 --2- 172.21.15.86:0/2625120005 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cfc0039b0 0x7f2d181f3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:12.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.724+0000 7f2d17400640 1 -- 172.21.15.86:0/2625120005 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d181584c0 con 0x7f2d1814b980 2024-04-20T21:21:12.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.724+0000 7f2d17e00640 1 --2- 172.21.15.86:0/2625120005 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cfc0039b0 0x7f2d181f3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:21:12.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.724+0000 7f2d17400640 1 --2- 172.21.15.86:0/2625120005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d1814b980 0x7f2d1814bde0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f2d040116d0 tx=0x7f2d04011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:12.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.724+0000 7f2d0fe00640 1 -- 172.21.15.86:0/2625120005 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0400f9c0 con 0x7f2d1814b980 2024-04-20T21:21:12.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.724+0000 7f2d0fe00640 1 -- 172.21.15.86:0/2625120005 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d0400fb60 con 0x7f2d1814b980 2024-04-20T21:21:12.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.724+0000 7f2d0fe00640 1 -- 172.21.15.86:0/2625120005 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d04015460 con 0x7f2d1814b980 2024-04-20T21:21:12.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.724+0000 7f2d1da00640 1 -- 172.21.15.86:0/2625120005 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d18149990 con 0x7f2d1814b980 2024-04-20T21:21:12.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.725+0000 7f2d1da00640 1 -- 172.21.15.86:0/2625120005 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d18149f50 con 0x7f2d1814b980 2024-04-20T21:21:12.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.726+0000 7f2d1da00640 1 -- 172.21.15.86:0/2625120005 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d1815e980 con 0x7f2d1814b980 2024-04-20T21:21:12.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.727+0000 7f2d0fe00640 1 -- 172.21.15.86:0/2625120005 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2d04015600 con 0x7f2d1814b980 2024-04-20T21:21:12.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.728+0000 7f2d0fe00640 1 --2- 172.21.15.86:0/2625120005 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2cf806c8f0 0x7f2cf806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:12.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.729+0000 7f2d0fe00640 1 -- 172.21.15.86:0/2625120005 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f2d0408c560 con 0x7f2d1814b980 2024-04-20T21:21:12.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.729+0000 7f2d17e00640 1 --2- 172.21.15.86:0/2625120005 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2cf806c8f0 0x7f2cf806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:12.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.729+0000 7f2d17e00640 1 --2- 172.21.15.86:0/2625120005 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2cf806c8f0 0x7f2cf806edb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2d08036000 tx=0x7f2d08009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:12.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.735+0000 7f2d0fe00640 1 -- 172.21.15.86:0/2625120005 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2d1815e980 con 0x7f2d1814b980 2024-04-20T21:21:12.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.932+0000 7f2d1da00640 1 -- 172.21.15.86:0/2625120005 --> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2d1806ee50 con 0x7f2cf806c8f0 2024-04-20T21:21:12.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.940+0000 7f2d0fe00640 1 -- 172.21.15.86:0/2625120005 <== mgr.4107 v2:172.21.15.86:6832/302045240 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251953 (secure 0 0 0) 0x7f2d08082850 con 0x7f2cf806c8f0 2024-04-20T21:21:12.943 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:21:12.945 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-20T21:21:12.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.946+0000 7f2d1da00640 1 -- 172.21.15.86:0/2625120005 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2cf806c8f0 msgr2=0x7f2cf806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:12.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.946+0000 7f2d1da00640 1 --2- 172.21.15.86:0/2625120005 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2cf806c8f0 0x7f2cf806edb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2d08036000 tx=0x7f2d08009030 comp rx=0 tx=0).stop 2024-04-20T21:21:12.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.946+0000 7f2d1da00640 1 -- 172.21.15.86:0/2625120005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d1814b980 msgr2=0x7f2d1814bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:12.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.946+0000 7f2d1da00640 1 --2- 172.21.15.86:0/2625120005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d1814b980 0x7f2d1814bde0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f2d040116d0 tx=0x7f2d04011b90 comp rx=0 tx=0).stop 2024-04-20T21:21:12.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.946+0000 7f2d1da00640 1 -- 172.21.15.86:0/2625120005 shutdown_connections 2024-04-20T21:21:12.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.946+0000 7f2d1da00640 1 --2- 172.21.15.86:0/2625120005 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2cf806c8f0 0x7f2cf806edb0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:12.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.946+0000 7f2d1da00640 1 --2- 172.21.15.86:0/2625120005 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d181577b0 0x7f2d18157c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:12.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.946+0000 7f2d1da00640 1 --2- 172.21.15.86:0/2625120005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d1814b980 0x7f2d1814bde0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:12.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.946+0000 7f2d1da00640 1 --2- 172.21.15.86:0/2625120005 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2cfc0039b0 0x7f2d181f3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:12.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.946+0000 7f2d1da00640 1 -- 172.21.15.86:0/2625120005 >> 172.21.15.86:0/2625120005 conn(0x7f2d180d0170 msgr2=0x7f2d180b13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:12.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.946+0000 7f2d1da00640 1 -- 172.21.15.86:0/2625120005 shutdown_connections 2024-04-20T21:21:12.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:12.947+0000 7f2d1da00640 1 -- 172.21.15.86:0/2625120005 wait complete. 2024-04-20T21:21:12.964 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":158,"stamp":"2024-04-20T21:21:12.695423+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":136,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":243744,"kb_used_data":6760,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754730976,"statfs":{"total":773094113280,"available":772844519424,"internally_reserved":0,"allocated":6922240,"data_stored":3551796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001775"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.345404+0000","last_change":"2024-04-20T21:21:08.091733+0000","last_active":"2024-04-20T21:21:09.345404+0000","last_peered":"2024-04-20T21:21:09.345404+0000","last_clean":"2024-04-20T21:21:09.345404+0000","last_became_active":"2024-04-20T21:21:08.088482+0000","last_became_peered":"2024-04-20T21:21:08.088482+0000","last_unstale":"2024-04-20T21:21:09.345404+0000","last_undegraded":"2024-04-20T21:21:09.345404+0000","last_fullsized":"2024-04-20T21:21:09.345404+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:11:06.521937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.346411+0000","last_change":"2024-04-20T21:21:08.101785+0000","last_active":"2024-04-20T21:21:09.346411+0000","last_peered":"2024-04-20T21:21:09.346411+0000","last_clean":"2024-04-20T21:21:09.346411+0000","last_became_active":"2024-04-20T21:21:08.101467+0000","last_became_peered":"2024-04-20T21:21:08.101467+0000","last_unstale":"2024-04-20T21:21:09.346411+0000","last_undegraded":"2024-04-20T21:21:09.346411+0000","last_fullsized":"2024-04-20T21:21:09.346411+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:33:43.580782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.348834+0000","last_change":"2024-04-20T21:21:08.094866+0000","last_active":"2024-04-20T21:21:09.348834+0000","last_peered":"2024-04-20T21:21:09.348834+0000","last_clean":"2024-04-20T21:21:09.348834+0000","last_became_active":"2024-04-20T21:21:08.094540+0000","last_became_peered":"2024-04-20T21:21:08.094540+0000","last_unstale":"2024-04-20T21:21:09.348834+0000","last_undegraded":"2024-04-20T21:21:09.348834+0000","last_fullsized":"2024-04-20T21:21:09.348834+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:39:01.553821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.346824+0000","last_change":"2024-04-20T21:21:08.095975+0000","last_active":"2024-04-20T21:21:09.346824+0000","last_peered":"2024-04-20T21:21:09.346824+0000","last_clean":"2024-04-20T21:21:09.346824+0000","last_became_active":"2024-04-20T21:21:08.095191+0000","last_became_peered":"2024-04-20T21:21:08.095191+0000","last_unstale":"2024-04-20T21:21:09.346824+0000","last_undegraded":"2024-04-20T21:21:09.346824+0000","last_fullsized":"2024-04-20T21:21:09.346824+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:08:45.663725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.344204+0000","last_change":"2024-04-20T21:21:08.091105+0000","last_active":"2024-04-20T21:21:09.344204+0000","last_peered":"2024-04-20T21:21:09.344204+0000","last_clean":"2024-04-20T21:21:09.344204+0000","last_became_active":"2024-04-20T21:21:08.088046+0000","last_became_peered":"2024-04-20T21:21:08.088046+0000","last_unstale":"2024-04-20T21:21:09.344204+0000","last_undegraded":"2024-04-20T21:21:09.344204+0000","last_fullsized":"2024-04-20T21:21:09.344204+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:39:39.792017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.104949+0000","last_change":"2024-04-20T21:21:08.099227+0000","last_active":"2024-04-20T21:21:09.104949+0000","last_peered":"2024-04-20T21:21:09.104949+0000","last_clean":"2024-04-20T21:21:09.104949+0000","last_became_active":"2024-04-20T21:21:08.098966+0000","last_became_peered":"2024-04-20T21:21:08.098966+0000","last_unstale":"2024-04-20T21:21:09.104949+0000","last_undegraded":"2024-04-20T21:21:09.104949+0000","last_fullsized":"2024-04-20T21:21:09.104949+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:39:14.817307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.091467+0000","last_change":"2024-04-20T21:21:08.093459+0000","last_active":"2024-04-20T21:21:09.091467+0000","last_peered":"2024-04-20T21:21:09.091467+0000","last_clean":"2024-04-20T21:21:09.091467+0000","last_became_active":"2024-04-20T21:21:08.093134+0000","last_became_peered":"2024-04-20T21:21:08.093134+0000","last_unstale":"2024-04-20T21:21:09.091467+0000","last_undegraded":"2024-04-20T21:21:09.091467+0000","last_fullsized":"2024-04-20T21:21:09.091467+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:21:51.679572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.346231+0000","last_change":"2024-04-20T21:21:08.083880+0000","last_active":"2024-04-20T21:21:09.346231+0000","last_peered":"2024-04-20T21:21:09.346231+0000","last_clean":"2024-04-20T21:21:09.346231+0000","last_became_active":"2024-04-20T21:21:08.083561+0000","last_became_peered":"2024-04-20T21:21:08.083561+0000","last_unstale":"2024-04-20T21:21:09.346231+0000","last_undegraded":"2024-04-20T21:21:09.346231+0000","last_fullsized":"2024-04-20T21:21:09.346231+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:05:31.032938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.092905+0000","last_change":"2024-04-20T21:21:08.095752+0000","last_active":"2024-04-20T21:21:09.092905+0000","last_peered":"2024-04-20T21:21:09.092905+0000","last_clean":"2024-04-20T21:21:09.092905+0000","last_became_active":"2024-04-20T21:21:08.094059+0000","last_became_peered":"2024-04-20T21:21:08.094059+0000","last_unstale":"2024-04-20T21:21:09.092905+0000","last_undegraded":"2024-04-20T21:21:09.092905+0000","last_fullsized":"2024-04-20T21:21:09.092905+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:13:34.349612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.086401+0000","last_change":"2024-04-20T21:21:08.094799+0000","last_active":"2024-04-20T21:21:09.086401+0000","last_peered":"2024-04-20T21:21:09.086401+0000","last_clean":"2024-04-20T21:21:09.086401+0000","last_became_active":"2024-04-20T21:21:08.094444+0000","last_became_peered":"2024-04-20T21:21:08.094444+0000","last_unstale":"2024-04-20T21:21:09.086401+0000","last_undegraded":"2024-04-20T21:21:09.086401+0000","last_fullsized":"2024-04-20T21:21:09.086401+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:34:28.510613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.086335+0000","last_change":"2024-04-20T21:21:08.093253+0000","last_active":"2024-04-20T21:21:09.086335+0000","last_peered":"2024-04-20T21:21:09.086335+0000","last_clean":"2024-04-20T21:21:09.086335+0000","last_became_active":"2024-04-20T21:21:08.087765+0000","last_became_peered":"2024-04-20T21:21:08.087765+0000","last_unstale":"2024-04-20T21:21:09.086335+0000","last_undegraded":"2024-04-20T21:21:09.086335+0000","last_fullsized":"2024-04-20T21:21:09.086335+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:20:30.049240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.347049+0000","last_change":"2024-04-20T21:21:08.092902+0000","last_active":"2024-04-20T21:21:09.347049+0000","last_peered":"2024-04-20T21:21:09.347049+0000","last_clean":"2024-04-20T21:21:09.347049+0000","last_became_active":"2024-04-20T21:21:08.092597+0000","last_became_peered":"2024-04-20T21:21:08.092597+0000","last_unstale":"2024-04-20T21:21:09.347049+0000","last_undegraded":"2024-04-20T21:21:09.347049+0000","last_fullsized":"2024-04-20T21:21:09.347049+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:39:14.817307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.092501+0000","last_change":"2024-04-20T21:21:08.099763+0000","last_active":"2024-04-20T21:21:09.092501+0000","last_peered":"2024-04-20T21:21:09.092501+0000","last_clean":"2024-04-20T21:21:09.092501+0000","last_became_active":"2024-04-20T21:21:08.099342+0000","last_became_peered":"2024-04-20T21:21:08.099342+0000","last_unstale":"2024-04-20T21:21:09.092501+0000","last_undegraded":"2024-04-20T21:21:09.092501+0000","last_fullsized":"2024-04-20T21:21:09.092501+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:32:10.911172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.091705+0000","last_change":"2024-04-20T21:21:08.071307+0000","last_active":"2024-04-20T21:21:09.091705+0000","last_peered":"2024-04-20T21:21:09.091705+0000","last_clean":"2024-04-20T21:21:09.091705+0000","last_became_active":"2024-04-20T21:21:08.070199+0000","last_became_peered":"2024-04-20T21:21:08.070199+0000","last_unstale":"2024-04-20T21:21:09.091705+0000","last_undegraded":"2024-04-20T21:21:09.091705+0000","last_fullsized":"2024-04-20T21:21:09.091705+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:54:22.899778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.092561+0000","last_change":"2024-04-20T21:21:08.089275+0000","last_active":"2024-04-20T21:21:09.092561+0000","last_peered":"2024-04-20T21:21:09.092561+0000","last_clean":"2024-04-20T21:21:09.092561+0000","last_became_active":"2024-04-20T21:21:08.088811+0000","last_became_peered":"2024-04-20T21:21:08.088811+0000","last_unstale":"2024-04-20T21:21:09.092561+0000","last_undegraded":"2024-04-20T21:21:09.092561+0000","last_fullsized":"2024-04-20T21:21:09.092561+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:06:08.208229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.104786+0000","last_change":"2024-04-20T21:21:08.101764+0000","last_active":"2024-04-20T21:21:09.104786+0000","last_peered":"2024-04-20T21:21:09.104786+0000","last_clean":"2024-04-20T21:21:09.104786+0000","last_became_active":"2024-04-20T21:21:08.101359+0000","last_became_peered":"2024-04-20T21:21:08.101359+0000","last_unstale":"2024-04-20T21:21:09.104786+0000","last_undegraded":"2024-04-20T21:21:09.104786+0000","last_fullsized":"2024-04-20T21:21:09.104786+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:39:01.553821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.347775+0000","last_change":"2024-04-20T21:21:08.094699+0000","last_active":"2024-04-20T21:21:09.347775+0000","last_peered":"2024-04-20T21:21:09.347775+0000","last_clean":"2024-04-20T21:21:09.347775+0000","last_became_active":"2024-04-20T21:21:08.093743+0000","last_became_peered":"2024-04-20T21:21:08.093743+0000","last_unstale":"2024-04-20T21:21:09.347775+0000","last_undegraded":"2024-04-20T21:21:09.347775+0000","last_fullsized":"2024-04-20T21:21:09.347775+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:52:05.367270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.348803+0000","last_change":"2024-04-20T21:21:08.101101+0000","last_active":"2024-04-20T21:21:09.348803+0000","last_peered":"2024-04-20T21:21:09.348803+0000","last_clean":"2024-04-20T21:21:09.348803+0000","last_became_active":"2024-04-20T21:21:08.100752+0000","last_became_peered":"2024-04-20T21:21:08.100752+0000","last_unstale":"2024-04-20T21:21:09.348803+0000","last_undegraded":"2024-04-20T21:21:09.348803+0000","last_fullsized":"2024-04-20T21:21:09.348803+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:16:45.378325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.344638+0000","last_change":"2024-04-20T21:21:08.088359+0000","last_active":"2024-04-20T21:21:09.344638+0000","last_peered":"2024-04-20T21:21:09.344638+0000","last_clean":"2024-04-20T21:21:09.344638+0000","last_became_active":"2024-04-20T21:21:08.087812+0000","last_became_peered":"2024-04-20T21:21:08.087812+0000","last_unstale":"2024-04-20T21:21:09.344638+0000","last_undegraded":"2024-04-20T21:21:09.344638+0000","last_fullsized":"2024-04-20T21:21:09.344638+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:16:50.288097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.347217+0000","last_change":"2024-04-20T21:21:08.094374+0000","last_active":"2024-04-20T21:21:09.347217+0000","last_peered":"2024-04-20T21:21:09.347217+0000","last_clean":"2024-04-20T21:21:09.347217+0000","last_became_active":"2024-04-20T21:21:08.094075+0000","last_became_peered":"2024-04-20T21:21:08.094075+0000","last_unstale":"2024-04-20T21:21:09.347217+0000","last_undegraded":"2024-04-20T21:21:09.347217+0000","last_fullsized":"2024-04-20T21:21:09.347217+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:45:28.298126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.348002+0000","last_change":"2024-04-20T21:21:08.090489+0000","last_active":"2024-04-20T21:21:09.348002+0000","last_peered":"2024-04-20T21:21:09.348002+0000","last_clean":"2024-04-20T21:21:09.348002+0000","last_became_active":"2024-04-20T21:21:08.090087+0000","last_became_peered":"2024-04-20T21:21:08.090087+0000","last_unstale":"2024-04-20T21:21:09.348002+0000","last_undegraded":"2024-04-20T21:21:09.348002+0000","last_fullsized":"2024-04-20T21:21:09.348002+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:28:10.926935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.346554+0000","last_change":"2024-04-20T21:21:08.088964+0000","last_active":"2024-04-20T21:21:09.346554+0000","last_peered":"2024-04-20T21:21:09.346554+0000","last_clean":"2024-04-20T21:21:09.346554+0000","last_became_active":"2024-04-20T21:21:08.088663+0000","last_became_peered":"2024-04-20T21:21:08.088663+0000","last_unstale":"2024-04-20T21:21:09.346554+0000","last_undegraded":"2024-04-20T21:21:09.346554+0000","last_fullsized":"2024-04-20T21:21:09.346554+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:48:47.410759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.347502+0000","last_change":"2024-04-20T21:21:08.093005+0000","last_active":"2024-04-20T21:21:09.347502+0000","last_peered":"2024-04-20T21:21:09.347502+0000","last_clean":"2024-04-20T21:21:09.347502+0000","last_became_active":"2024-04-20T21:21:08.092450+0000","last_became_peered":"2024-04-20T21:21:08.092450+0000","last_unstale":"2024-04-20T21:21:09.347502+0000","last_undegraded":"2024-04-20T21:21:09.347502+0000","last_fullsized":"2024-04-20T21:21:09.347502+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:37:05.301218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.091433+0000","last_change":"2024-04-20T21:21:08.099964+0000","last_active":"2024-04-20T21:21:09.091433+0000","last_peered":"2024-04-20T21:21:09.091433+0000","last_clean":"2024-04-20T21:21:09.091433+0000","last_became_active":"2024-04-20T21:21:08.099532+0000","last_became_peered":"2024-04-20T21:21:08.099532+0000","last_unstale":"2024-04-20T21:21:09.091433+0000","last_undegraded":"2024-04-20T21:21:09.091433+0000","last_fullsized":"2024-04-20T21:21:09.091433+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:37:36.217818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.345373+0000","last_change":"2024-04-20T21:21:08.095678+0000","last_active":"2024-04-20T21:21:09.345373+0000","last_peered":"2024-04-20T21:21:09.345373+0000","last_clean":"2024-04-20T21:21:09.345373+0000","last_became_active":"2024-04-20T21:21:08.095384+0000","last_became_peered":"2024-04-20T21:21:08.095384+0000","last_unstale":"2024-04-20T21:21:09.345373+0000","last_undegraded":"2024-04-20T21:21:09.345373+0000","last_fullsized":"2024-04-20T21:21:09.345373+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:06:24.807043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.346613+0000","last_change":"2024-04-20T21:21:08.088161+0000","last_active":"2024-04-20T21:21:09.346613+0000","last_peered":"2024-04-20T21:21:09.346613+0000","last_clean":"2024-04-20T21:21:09.346613+0000","last_became_active":"2024-04-20T21:21:08.087818+0000","last_became_peered":"2024-04-20T21:21:08.087818+0000","last_unstale":"2024-04-20T21:21:09.346613+0000","last_undegraded":"2024-04-20T21:21:09.346613+0000","last_fullsized":"2024-04-20T21:21:09.346613+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:28:10.926935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.105045+0000","last_change":"2024-04-20T21:21:08.105444+0000","last_active":"2024-04-20T21:21:09.105045+0000","last_peered":"2024-04-20T21:21:09.105045+0000","last_clean":"2024-04-20T21:21:09.105045+0000","last_became_active":"2024-04-20T21:21:08.105119+0000","last_became_peered":"2024-04-20T21:21:08.105119+0000","last_unstale":"2024-04-20T21:21:09.105045+0000","last_undegraded":"2024-04-20T21:21:09.105045+0000","last_fullsized":"2024-04-20T21:21:09.105045+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:44:11.969230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.344974+0000","last_change":"2024-04-20T21:21:08.093283+0000","last_active":"2024-04-20T21:21:09.344974+0000","last_peered":"2024-04-20T21:21:09.344974+0000","last_clean":"2024-04-20T21:21:09.344974+0000","last_became_active":"2024-04-20T21:21:08.093021+0000","last_became_peered":"2024-04-20T21:21:08.093021+0000","last_unstale":"2024-04-20T21:21:09.344974+0000","last_undegraded":"2024-04-20T21:21:09.344974+0000","last_fullsized":"2024-04-20T21:21:09.344974+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:29:43.884817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.347560+0000","last_change":"2024-04-20T21:21:08.090010+0000","last_active":"2024-04-20T21:21:09.347560+0000","last_peered":"2024-04-20T21:21:09.347560+0000","last_clean":"2024-04-20T21:21:09.347560+0000","last_became_active":"2024-04-20T21:21:08.089745+0000","last_became_peered":"2024-04-20T21:21:08.089745+0000","last_unstale":"2024-04-20T21:21:09.347560+0000","last_undegraded":"2024-04-20T21:21:09.347560+0000","last_fullsized":"2024-04-20T21:21:09.347560+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:18:18.759575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.347644+0000","last_change":"2024-04-20T21:21:08.094934+0000","last_active":"2024-04-20T21:21:09.347644+0000","last_peered":"2024-04-20T21:21:09.347644+0000","last_clean":"2024-04-20T21:21:09.347644+0000","last_became_active":"2024-04-20T21:21:08.094307+0000","last_became_peered":"2024-04-20T21:21:08.094307+0000","last_unstale":"2024-04-20T21:21:09.347644+0000","last_undegraded":"2024-04-20T21:21:09.347644+0000","last_fullsized":"2024-04-20T21:21:09.347644+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:19:45.390643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.344021+0000","last_change":"2024-04-20T21:21:08.089016+0000","last_active":"2024-04-20T21:21:09.344021+0000","last_peered":"2024-04-20T21:21:09.344021+0000","last_clean":"2024-04-20T21:21:09.344021+0000","last_became_active":"2024-04-20T21:21:08.088521+0000","last_became_peered":"2024-04-20T21:21:08.088521+0000","last_unstale":"2024-04-20T21:21:09.344021+0000","last_undegraded":"2024-04-20T21:21:09.344021+0000","last_fullsized":"2024-04-20T21:21:09.344021+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:27:28.429303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.105097+0000","last_change":"2024-04-20T21:21:08.094207+0000","last_active":"2024-04-20T21:21:09.105097+0000","last_peered":"2024-04-20T21:21:09.105097+0000","last_clean":"2024-04-20T21:21:09.105097+0000","last_became_active":"2024-04-20T21:21:08.093841+0000","last_became_peered":"2024-04-20T21:21:08.093841+0000","last_unstale":"2024-04-20T21:21:09.105097+0000","last_undegraded":"2024-04-20T21:21:09.105097+0000","last_fullsized":"2024-04-20T21:21:09.105097+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:38:37.845726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.105593+0000","last_change":"2024-04-20T21:21:08.105496+0000","last_active":"2024-04-20T21:21:09.105593+0000","last_peered":"2024-04-20T21:21:09.105593+0000","last_clean":"2024-04-20T21:21:09.105593+0000","last_became_active":"2024-04-20T21:21:08.105185+0000","last_became_peered":"2024-04-20T21:21:08.105185+0000","last_unstale":"2024-04-20T21:21:09.105593+0000","last_undegraded":"2024-04-20T21:21:09.105593+0000","last_fullsized":"2024-04-20T21:21:09.105593+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:17:57.922373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"58'4","reported_seq":18,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.105753+0000","last_change":"2024-04-20T21:21:07.064736+0000","last_active":"2024-04-20T21:21:09.105753+0000","last_peered":"2024-04-20T21:21:09.105753+0000","last_clean":"2024-04-20T21:21:09.105753+0000","last_became_active":"2024-04-20T21:21:07.064434+0000","last_became_peered":"2024-04-20T21:21:07.064434+0000","last_unstale":"2024-04-20T21:21:09.105753+0000","last_undegraded":"2024-04-20T21:21:09.105753+0000","last_fullsized":"2024-04-20T21:21:09.105753+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:02:34.081872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.086593+0000","last_change":"2024-04-20T21:21:08.097275+0000","last_active":"2024-04-20T21:21:09.086593+0000","last_peered":"2024-04-20T21:21:09.086593+0000","last_clean":"2024-04-20T21:21:09.086593+0000","last_became_active":"2024-04-20T21:21:08.090690+0000","last_became_peered":"2024-04-20T21:21:08.090690+0000","last_unstale":"2024-04-20T21:21:09.086593+0000","last_undegraded":"2024-04-20T21:21:09.086593+0000","last_fullsized":"2024-04-20T21:21:09.086593+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:11:41.302453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.348005+0000","last_change":"2024-04-20T21:21:07.062354+0000","last_active":"2024-04-20T21:21:09.348005+0000","last_peered":"2024-04-20T21:21:09.348005+0000","last_clean":"2024-04-20T21:21:09.348005+0000","last_became_active":"2024-04-20T21:21:07.062071+0000","last_became_peered":"2024-04-20T21:21:07.062071+0000","last_unstale":"2024-04-20T21:21:09.348005+0000","last_undegraded":"2024-04-20T21:21:09.348005+0000","last_fullsized":"2024-04-20T21:21:09.348005+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:37:04.286843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.105596+0000","last_change":"2024-04-20T21:21:08.105870+0000","last_active":"2024-04-20T21:21:09.105596+0000","last_peered":"2024-04-20T21:21:09.105596+0000","last_clean":"2024-04-20T21:21:09.105596+0000","last_became_active":"2024-04-20T21:21:08.105547+0000","last_became_peered":"2024-04-20T21:21:08.105547+0000","last_unstale":"2024-04-20T21:21:09.105596+0000","last_undegraded":"2024-04-20T21:21:09.105596+0000","last_fullsized":"2024-04-20T21:21:09.105596+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:47:21.478103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.105442+0000","last_change":"2024-04-20T21:21:07.058758+0000","last_active":"2024-04-20T21:21:09.105442+0000","last_peered":"2024-04-20T21:21:09.105442+0000","last_clean":"2024-04-20T21:21:09.105442+0000","last_became_active":"2024-04-20T21:21:07.058447+0000","last_became_peered":"2024-04-20T21:21:07.058447+0000","last_unstale":"2024-04-20T21:21:09.105442+0000","last_undegraded":"2024-04-20T21:21:09.105442+0000","last_fullsized":"2024-04-20T21:21:09.105442+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:31:45.780980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.105057+0000","last_change":"2024-04-20T21:21:08.094438+0000","last_active":"2024-04-20T21:21:09.105057+0000","last_peered":"2024-04-20T21:21:09.105057+0000","last_clean":"2024-04-20T21:21:09.105057+0000","last_became_active":"2024-04-20T21:21:08.094150+0000","last_became_peered":"2024-04-20T21:21:08.094150+0000","last_unstale":"2024-04-20T21:21:09.105057+0000","last_undegraded":"2024-04-20T21:21:09.105057+0000","last_fullsized":"2024-04-20T21:21:09.105057+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:16:45.378325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.347010+0000","last_change":"2024-04-20T21:21:07.066367+0000","last_active":"2024-04-20T21:21:09.347010+0000","last_peered":"2024-04-20T21:21:09.347010+0000","last_clean":"2024-04-20T21:21:09.347010+0000","last_became_active":"2024-04-20T21:21:07.065939+0000","last_became_peered":"2024-04-20T21:21:07.065939+0000","last_unstale":"2024-04-20T21:21:09.347010+0000","last_undegraded":"2024-04-20T21:21:09.347010+0000","last_fullsized":"2024-04-20T21:21:09.347010+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:31:45.780980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.346943+0000","last_change":"2024-04-20T21:21:08.096754+0000","last_active":"2024-04-20T21:21:09.346943+0000","last_peered":"2024-04-20T21:21:09.346943+0000","last_clean":"2024-04-20T21:21:09.346943+0000","last_became_active":"2024-04-20T21:21:08.096428+0000","last_became_peered":"2024-04-20T21:21:08.096428+0000","last_unstale":"2024-04-20T21:21:09.346943+0000","last_undegraded":"2024-04-20T21:21:09.346943+0000","last_fullsized":"2024-04-20T21:21:09.346943+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:00:27.110698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.091554+0000","last_change":"2024-04-20T21:21:07.064779+0000","last_active":"2024-04-20T21:21:09.091554+0000","last_peered":"2024-04-20T21:21:09.091554+0000","last_clean":"2024-04-20T21:21:09.091554+0000","last_became_active":"2024-04-20T21:21:07.064315+0000","last_became_peered":"2024-04-20T21:21:07.064315+0000","last_unstale":"2024-04-20T21:21:09.091554+0000","last_undegraded":"2024-04-20T21:21:09.091554+0000","last_fullsized":"2024-04-20T21:21:09.091554+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:10:54.025506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.347160+0000","last_change":"2024-04-20T21:21:08.110606+0000","last_active":"2024-04-20T21:21:09.347160+0000","last_peered":"2024-04-20T21:21:09.347160+0000","last_clean":"2024-04-20T21:21:09.347160+0000","last_became_active":"2024-04-20T21:21:08.110339+0000","last_became_peered":"2024-04-20T21:21:08.110339+0000","last_unstale":"2024-04-20T21:21:09.347160+0000","last_undegraded":"2024-04-20T21:21:09.347160+0000","last_fullsized":"2024-04-20T21:21:09.347160+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:25:51.430783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.105204+0000","last_change":"2024-04-20T21:21:07.060337+0000","last_active":"2024-04-20T21:21:09.105204+0000","last_peered":"2024-04-20T21:21:09.105204+0000","last_clean":"2024-04-20T21:21:09.105204+0000","last_became_active":"2024-04-20T21:21:07.059107+0000","last_became_peered":"2024-04-20T21:21:07.059107+0000","last_unstale":"2024-04-20T21:21:09.105204+0000","last_undegraded":"2024-04-20T21:21:09.105204+0000","last_fullsized":"2024-04-20T21:21:09.105204+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:04:22.384267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.091296+0000","last_change":"2024-04-20T21:21:08.090949+0000","last_active":"2024-04-20T21:21:09.091296+0000","last_peered":"2024-04-20T21:21:09.091296+0000","last_clean":"2024-04-20T21:21:09.091296+0000","last_became_active":"2024-04-20T21:21:08.089972+0000","last_became_peered":"2024-04-20T21:21:08.089972+0000","last_unstale":"2024-04-20T21:21:09.091296+0000","last_undegraded":"2024-04-20T21:21:09.091296+0000","last_fullsized":"2024-04-20T21:21:09.091296+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:10:07.293884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.345872+0000","last_change":"2024-04-20T21:21:07.058643+0000","last_active":"2024-04-20T21:21:09.345872+0000","last_peered":"2024-04-20T21:21:09.345872+0000","last_clean":"2024-04-20T21:21:09.345872+0000","last_became_active":"2024-04-20T21:21:07.058357+0000","last_became_peered":"2024-04-20T21:21:07.058357+0000","last_unstale":"2024-04-20T21:21:09.345872+0000","last_undegraded":"2024-04-20T21:21:09.345872+0000","last_fullsized":"2024-04-20T21:21:09.345872+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:23:08.916964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.346092+0000","last_change":"2024-04-20T21:21:08.100627+0000","last_active":"2024-04-20T21:21:09.346092+0000","last_peered":"2024-04-20T21:21:09.346092+0000","last_clean":"2024-04-20T21:21:09.346092+0000","last_became_active":"2024-04-20T21:21:08.095155+0000","last_became_peered":"2024-04-20T21:21:08.095155+0000","last_unstale":"2024-04-20T21:21:09.346092+0000","last_undegraded":"2024-04-20T21:21:09.346092+0000","last_fullsized":"2024-04-20T21:21:09.346092+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:44:15.579437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.091002+0000","last_change":"2024-04-20T21:21:07.061079+0000","last_active":"2024-04-20T21:21:09.091002+0000","last_peered":"2024-04-20T21:21:09.091002+0000","last_clean":"2024-04-20T21:21:09.091002+0000","last_became_active":"2024-04-20T21:21:07.060595+0000","last_became_peered":"2024-04-20T21:21:07.060595+0000","last_unstale":"2024-04-20T21:21:09.091002+0000","last_undegraded":"2024-04-20T21:21:09.091002+0000","last_fullsized":"2024-04-20T21:21:09.091002+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:16:47.508481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.092604+0000","last_change":"2024-04-20T21:21:08.090064+0000","last_active":"2024-04-20T21:21:09.092604+0000","last_peered":"2024-04-20T21:21:09.092604+0000","last_clean":"2024-04-20T21:21:09.092604+0000","last_became_active":"2024-04-20T21:21:08.087014+0000","last_became_peered":"2024-04-20T21:21:08.087014+0000","last_unstale":"2024-04-20T21:21:09.092604+0000","last_undegraded":"2024-04-20T21:21:09.092604+0000","last_fullsized":"2024-04-20T21:21:09.092604+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:56:13.886897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.348954+0000","last_change":"2024-04-20T21:21:07.068719+0000","last_active":"2024-04-20T21:21:09.348954+0000","last_peered":"2024-04-20T21:21:09.348954+0000","last_clean":"2024-04-20T21:21:09.348954+0000","last_became_active":"2024-04-20T21:21:07.068426+0000","last_became_peered":"2024-04-20T21:21:07.068426+0000","last_unstale":"2024-04-20T21:21:09.348954+0000","last_undegraded":"2024-04-20T21:21:09.348954+0000","last_fullsized":"2024-04-20T21:21:09.348954+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:08:39.116971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.348972+0000","last_change":"2024-04-20T21:21:08.104441+0000","last_active":"2024-04-20T21:21:09.348972+0000","last_peered":"2024-04-20T21:21:09.348972+0000","last_clean":"2024-04-20T21:21:09.348972+0000","last_became_active":"2024-04-20T21:21:08.104163+0000","last_became_peered":"2024-04-20T21:21:08.104163+0000","last_unstale":"2024-04-20T21:21:09.348972+0000","last_undegraded":"2024-04-20T21:21:09.348972+0000","last_fullsized":"2024-04-20T21:21:09.348972+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:57:33.459043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.348018+0000","last_change":"2024-04-20T21:21:07.068441+0000","last_active":"2024-04-20T21:21:09.348018+0000","last_peered":"2024-04-20T21:21:09.348018+0000","last_clean":"2024-04-20T21:21:09.348018+0000","last_became_active":"2024-04-20T21:21:07.068086+0000","last_became_peered":"2024-04-20T21:21:07.068086+0000","last_unstale":"2024-04-20T21:21:09.348018+0000","last_undegraded":"2024-04-20T21:21:09.348018+0000","last_fullsized":"2024-04-20T21:21:09.348018+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:19:44.376267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.086127+0000","last_change":"2024-04-20T21:21:08.100649+0000","last_active":"2024-04-20T21:21:09.086127+0000","last_peered":"2024-04-20T21:21:09.086127+0000","last_clean":"2024-04-20T21:21:09.086127+0000","last_became_active":"2024-04-20T21:21:08.099136+0000","last_became_peered":"2024-04-20T21:21:08.099136+0000","last_unstale":"2024-04-20T21:21:09.086127+0000","last_undegraded":"2024-04-20T21:21:09.086127+0000","last_fullsized":"2024-04-20T21:21:09.086127+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:56:33.326306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"58'3","reported_seq":17,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.349171+0000","last_change":"2024-04-20T21:21:07.068225+0000","last_active":"2024-04-20T21:21:09.349171+0000","last_peered":"2024-04-20T21:21:09.349171+0000","last_clean":"2024-04-20T21:21:09.349171+0000","last_became_active":"2024-04-20T21:21:07.067908+0000","last_became_peered":"2024-04-20T21:21:07.067908+0000","last_unstale":"2024-04-20T21:21:09.349171+0000","last_undegraded":"2024-04-20T21:21:09.349171+0000","last_fullsized":"2024-04-20T21:21:09.349171+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:18:00.107759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.348801+0000","last_change":"2024-04-20T21:21:08.104022+0000","last_active":"2024-04-20T21:21:09.348801+0000","last_peered":"2024-04-20T21:21:09.348801+0000","last_clean":"2024-04-20T21:21:09.348801+0000","last_became_active":"2024-04-20T21:21:08.103549+0000","last_became_peered":"2024-04-20T21:21:08.103549+0000","last_unstale":"2024-04-20T21:21:09.348801+0000","last_undegraded":"2024-04-20T21:21:09.348801+0000","last_fullsized":"2024-04-20T21:21:09.348801+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:38:37.845726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"58'6","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.092756+0000","last_change":"2024-04-20T21:21:07.065162+0000","last_active":"2024-04-20T21:21:09.092756+0000","last_peered":"2024-04-20T21:21:09.092756+0000","last_clean":"2024-04-20T21:21:09.092756+0000","last_became_active":"2024-04-20T21:21:07.064629+0000","last_became_peered":"2024-04-20T21:21:07.064629+0000","last_unstale":"2024-04-20T21:21:09.092756+0000","last_undegraded":"2024-04-20T21:21:09.092756+0000","last_fullsized":"2024-04-20T21:21:09.092756+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:31:44.268841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.346075+0000","last_change":"2024-04-20T21:21:08.105498+0000","last_active":"2024-04-20T21:21:09.346075+0000","last_peered":"2024-04-20T21:21:09.346075+0000","last_clean":"2024-04-20T21:21:09.346075+0000","last_became_active":"2024-04-20T21:21:08.105195+0000","last_became_peered":"2024-04-20T21:21:08.105195+0000","last_unstale":"2024-04-20T21:21:09.346075+0000","last_undegraded":"2024-04-20T21:21:09.346075+0000","last_fullsized":"2024-04-20T21:21:09.346075+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:19:45.390643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.347168+0000","last_change":"2024-04-20T21:21:07.064794+0000","last_active":"2024-04-20T21:21:09.347168+0000","last_peered":"2024-04-20T21:21:09.347168+0000","last_clean":"2024-04-20T21:21:09.347168+0000","last_became_active":"2024-04-20T21:21:07.064512+0000","last_became_peered":"2024-04-20T21:21:07.064512+0000","last_unstale":"2024-04-20T21:21:09.347168+0000","last_undegraded":"2024-04-20T21:21:09.347168+0000","last_fullsized":"2024-04-20T21:21:09.347168+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:58:07.560291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.348052+0000","last_change":"2024-04-20T21:21:08.095302+0000","last_active":"2024-04-20T21:21:09.348052+0000","last_peered":"2024-04-20T21:21:09.348052+0000","last_clean":"2024-04-20T21:21:09.348052+0000","last_became_active":"2024-04-20T21:21:08.094987+0000","last_became_peered":"2024-04-20T21:21:08.094987+0000","last_unstale":"2024-04-20T21:21:09.348052+0000","last_undegraded":"2024-04-20T21:21:09.348052+0000","last_fullsized":"2024-04-20T21:21:09.348052+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:08:45.663725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.090640+0000","last_change":"2024-04-20T21:21:07.061926+0000","last_active":"2024-04-20T21:21:09.090640+0000","last_peered":"2024-04-20T21:21:09.090640+0000","last_clean":"2024-04-20T21:21:09.090640+0000","last_became_active":"2024-04-20T21:21:07.061645+0000","last_became_peered":"2024-04-20T21:21:07.061645+0000","last_unstale":"2024-04-20T21:21:09.090640+0000","last_undegraded":"2024-04-20T21:21:09.090640+0000","last_fullsized":"2024-04-20T21:21:09.090640+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:03:45.910750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.091824+0000","last_change":"2024-04-20T21:21:08.085990+0000","last_active":"2024-04-20T21:21:09.091824+0000","last_peered":"2024-04-20T21:21:09.091824+0000","last_clean":"2024-04-20T21:21:09.091824+0000","last_became_active":"2024-04-20T21:21:08.085391+0000","last_became_peered":"2024-04-20T21:21:08.085391+0000","last_unstale":"2024-04-20T21:21:09.091824+0000","last_undegraded":"2024-04-20T21:21:09.091824+0000","last_fullsized":"2024-04-20T21:21:09.091824+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:02:35.096247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.345920+0000","last_change":"2024-04-20T21:21:07.068906+0000","last_active":"2024-04-20T21:21:09.345920+0000","last_peered":"2024-04-20T21:21:09.345920+0000","last_clean":"2024-04-20T21:21:09.345920+0000","last_became_active":"2024-04-20T21:21:07.068320+0000","last_became_peered":"2024-04-20T21:21:07.068320+0000","last_unstale":"2024-04-20T21:21:09.345920+0000","last_undegraded":"2024-04-20T21:21:09.345920+0000","last_fullsized":"2024-04-20T21:21:09.345920+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:04:22.384267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.347929+0000","last_change":"2024-04-20T21:21:08.094417+0000","last_active":"2024-04-20T21:21:09.347929+0000","last_peered":"2024-04-20T21:21:09.347929+0000","last_clean":"2024-04-20T21:21:09.347929+0000","last_became_active":"2024-04-20T21:21:08.093011+0000","last_became_peered":"2024-04-20T21:21:08.093011+0000","last_unstale":"2024-04-20T21:21:09.347929+0000","last_undegraded":"2024-04-20T21:21:09.347929+0000","last_fullsized":"2024-04-20T21:21:09.347929+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:22:32.356804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"58'8","reported_seq":22,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.346837+0000","last_change":"2024-04-20T21:21:07.066171+0000","last_active":"2024-04-20T21:21:09.346837+0000","last_peered":"2024-04-20T21:21:09.346837+0000","last_clean":"2024-04-20T21:21:09.346837+0000","last_became_active":"2024-04-20T21:21:07.065377+0000","last_became_peered":"2024-04-20T21:21:07.065377+0000","last_unstale":"2024-04-20T21:21:09.346837+0000","last_undegraded":"2024-04-20T21:21:09.346837+0000","last_fullsized":"2024-04-20T21:21:09.346837+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:23:08.916964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.348747+0000","last_change":"2024-04-20T21:21:08.095768+0000","last_active":"2024-04-20T21:21:09.348747+0000","last_peered":"2024-04-20T21:21:09.348747+0000","last_clean":"2024-04-20T21:21:09.348747+0000","last_became_active":"2024-04-20T21:21:08.095495+0000","last_became_peered":"2024-04-20T21:21:08.095495+0000","last_unstale":"2024-04-20T21:21:09.348747+0000","last_undegraded":"2024-04-20T21:21:09.348747+0000","last_fullsized":"2024-04-20T21:21:09.348747+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:44:11.969230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.348051+0000","last_change":"2024-04-20T21:21:07.065157+0000","last_active":"2024-04-20T21:21:09.348051+0000","last_peered":"2024-04-20T21:21:09.348051+0000","last_clean":"2024-04-20T21:21:09.348051+0000","last_became_active":"2024-04-20T21:21:07.064864+0000","last_became_peered":"2024-04-20T21:21:07.064864+0000","last_unstale":"2024-04-20T21:21:09.348051+0000","last_undegraded":"2024-04-20T21:21:09.348051+0000","last_fullsized":"2024-04-20T21:21:09.348051+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:28:09.912560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.347245+0000","last_change":"2024-04-20T21:21:08.108105+0000","last_active":"2024-04-20T21:21:09.347245+0000","last_peered":"2024-04-20T21:21:09.347245+0000","last_clean":"2024-04-20T21:21:09.347245+0000","last_became_active":"2024-04-20T21:21:08.104645+0000","last_became_peered":"2024-04-20T21:21:08.104645+0000","last_unstale":"2024-04-20T21:21:09.347245+0000","last_undegraded":"2024-04-20T21:21:09.347245+0000","last_fullsized":"2024-04-20T21:21:09.347245+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:16:45.378325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.105576+0000","last_change":"2024-04-20T21:21:07.068253+0000","last_active":"2024-04-20T21:21:09.105576+0000","last_peered":"2024-04-20T21:21:09.105576+0000","last_clean":"2024-04-20T21:21:09.105576+0000","last_became_active":"2024-04-20T21:21:07.067929+0000","last_became_peered":"2024-04-20T21:21:07.067929+0000","last_unstale":"2024-04-20T21:21:09.105576+0000","last_undegraded":"2024-04-20T21:21:09.105576+0000","last_fullsized":"2024-04-20T21:21:09.105576+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:21:50.665197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.347357+0000","last_change":"2024-04-20T21:21:08.090271+0000","last_active":"2024-04-20T21:21:09.347357+0000","last_peered":"2024-04-20T21:21:09.347357+0000","last_clean":"2024-04-20T21:21:09.347357+0000","last_became_active":"2024-04-20T21:21:08.089639+0000","last_became_peered":"2024-04-20T21:21:08.089639+0000","last_unstale":"2024-04-20T21:21:09.347357+0000","last_undegraded":"2024-04-20T21:21:09.347357+0000","last_fullsized":"2024-04-20T21:21:09.347357+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:57:10.155813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"58'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.346688+0000","last_change":"2024-04-20T21:21:07.068572+0000","last_active":"2024-04-20T21:21:09.346688+0000","last_peered":"2024-04-20T21:21:09.346688+0000","last_clean":"2024-04-20T21:21:09.346688+0000","last_became_active":"2024-04-20T21:21:07.068225+0000","last_became_peered":"2024-04-20T21:21:07.068225+0000","last_unstale":"2024-04-20T21:21:09.346688+0000","last_undegraded":"2024-04-20T21:21:09.346688+0000","last_fullsized":"2024-04-20T21:21:09.346688+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:54:21.885403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.349323+0000","last_change":"2024-04-20T21:21:08.104360+0000","last_active":"2024-04-20T21:21:09.349323+0000","last_peered":"2024-04-20T21:21:09.349323+0000","last_clean":"2024-04-20T21:21:09.349323+0000","last_became_active":"2024-04-20T21:21:08.103989+0000","last_became_peered":"2024-04-20T21:21:08.103989+0000","last_unstale":"2024-04-20T21:21:09.349323+0000","last_undegraded":"2024-04-20T21:21:09.349323+0000","last_fullsized":"2024-04-20T21:21:09.349323+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:07:19.659957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.105433+0000","last_change":"2024-04-20T21:21:07.061618+0000","last_active":"2024-04-20T21:21:09.105433+0000","last_peered":"2024-04-20T21:21:09.105433+0000","last_clean":"2024-04-20T21:21:09.105433+0000","last_became_active":"2024-04-20T21:21:07.061318+0000","last_became_peered":"2024-04-20T21:21:07.061318+0000","last_unstale":"2024-04-20T21:21:09.105433+0000","last_undegraded":"2024-04-20T21:21:09.105433+0000","last_fullsized":"2024-04-20T21:21:09.105433+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:06:07.193854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.346125+0000","last_change":"2024-04-20T21:21:08.088980+0000","last_active":"2024-04-20T21:21:09.346125+0000","last_peered":"2024-04-20T21:21:09.346125+0000","last_clean":"2024-04-20T21:21:09.346125+0000","last_became_active":"2024-04-20T21:21:08.088594+0000","last_became_peered":"2024-04-20T21:21:08.088594+0000","last_unstale":"2024-04-20T21:21:09.346125+0000","last_undegraded":"2024-04-20T21:21:09.346125+0000","last_fullsized":"2024-04-20T21:21:09.346125+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:22:32.356804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.347511+0000","last_change":"2024-04-20T21:21:07.053114+0000","last_active":"2024-04-20T21:21:09.347511+0000","last_peered":"2024-04-20T21:21:09.347511+0000","last_clean":"2024-04-20T21:21:09.347511+0000","last_became_active":"2024-04-20T21:21:07.052293+0000","last_became_peered":"2024-04-20T21:21:07.052293+0000","last_unstale":"2024-04-20T21:21:09.347511+0000","last_undegraded":"2024-04-20T21:21:09.347511+0000","last_fullsized":"2024-04-20T21:21:09.347511+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:56:13.239542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.105238+0000","last_change":"2024-04-20T21:21:08.106845+0000","last_active":"2024-04-20T21:21:09.105238+0000","last_peered":"2024-04-20T21:21:09.105238+0000","last_clean":"2024-04-20T21:21:09.105238+0000","last_became_active":"2024-04-20T21:21:08.105015+0000","last_became_peered":"2024-04-20T21:21:08.105015+0000","last_unstale":"2024-04-20T21:21:09.105238+0000","last_undegraded":"2024-04-20T21:21:09.105238+0000","last_fullsized":"2024-04-20T21:21:09.105238+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:57:33.459043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.085898+0000","last_change":"2024-04-20T21:21:07.068471+0000","last_active":"2024-04-20T21:21:09.085898+0000","last_peered":"2024-04-20T21:21:09.085898+0000","last_clean":"2024-04-20T21:21:09.085898+0000","last_became_active":"2024-04-20T21:21:07.068159+0000","last_became_peered":"2024-04-20T21:21:07.068159+0000","last_unstale":"2024-04-20T21:21:09.085898+0000","last_undegraded":"2024-04-20T21:21:09.085898+0000","last_fullsized":"2024-04-20T21:21:09.085898+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:03:45.910750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.105574+0000","last_change":"2024-04-20T21:21:08.101246+0000","last_active":"2024-04-20T21:21:09.105574+0000","last_peered":"2024-04-20T21:21:09.105574+0000","last_clean":"2024-04-20T21:21:09.105574+0000","last_became_active":"2024-04-20T21:21:08.100632+0000","last_became_peered":"2024-04-20T21:21:08.100632+0000","last_unstale":"2024-04-20T21:21:09.105574+0000","last_undegraded":"2024-04-20T21:21:09.105574+0000","last_fullsized":"2024-04-20T21:21:09.105574+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:35:59.168208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.345802+0000","last_change":"2024-04-20T21:21:07.062605+0000","last_active":"2024-04-20T21:21:09.345802+0000","last_peered":"2024-04-20T21:21:09.345802+0000","last_clean":"2024-04-20T21:21:09.345802+0000","last_became_active":"2024-04-20T21:21:07.060511+0000","last_became_peered":"2024-04-20T21:21:07.060511+0000","last_unstale":"2024-04-20T21:21:09.345802+0000","last_undegraded":"2024-04-20T21:21:09.345802+0000","last_fullsized":"2024-04-20T21:21:09.345802+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:36:32.878955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.346835+0000","last_change":"2024-04-20T21:21:08.096531+0000","last_active":"2024-04-20T21:21:09.346835+0000","last_peered":"2024-04-20T21:21:09.346835+0000","last_clean":"2024-04-20T21:21:09.346835+0000","last_became_active":"2024-04-20T21:21:08.096211+0000","last_became_peered":"2024-04-20T21:21:08.096211+0000","last_unstale":"2024-04-20T21:21:09.346835+0000","last_undegraded":"2024-04-20T21:21:09.346835+0000","last_fullsized":"2024-04-20T21:21:09.346835+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:39:01.553821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"58'2","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.086646+0000","last_change":"2024-04-20T21:21:07.054308+0000","last_active":"2024-04-20T21:21:09.086646+0000","last_peered":"2024-04-20T21:21:09.086646+0000","last_clean":"2024-04-20T21:21:09.086646+0000","last_became_active":"2024-04-20T21:21:07.052501+0000","last_became_peered":"2024-04-20T21:21:07.052501+0000","last_unstale":"2024-04-20T21:21:09.086646+0000","last_undegraded":"2024-04-20T21:21:09.086646+0000","last_fullsized":"2024-04-20T21:21:09.086646+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:32:05.259770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.347099+0000","last_change":"2024-04-20T21:21:08.092611+0000","last_active":"2024-04-20T21:21:09.347099+0000","last_peered":"2024-04-20T21:21:09.347099+0000","last_clean":"2024-04-20T21:21:09.347099+0000","last_became_active":"2024-04-20T21:21:08.091516+0000","last_became_peered":"2024-04-20T21:21:08.091516+0000","last_unstale":"2024-04-20T21:21:09.347099+0000","last_undegraded":"2024-04-20T21:21:09.347099+0000","last_fullsized":"2024-04-20T21:21:09.347099+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:44:11.969230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"59'1","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.109189+0000","last_change":"2024-04-20T21:21:07.060124+0000","last_active":"2024-04-20T21:21:09.109189+0000","last_peered":"2024-04-20T21:21:09.109189+0000","last_clean":"2024-04-20T21:21:09.109189+0000","last_became_active":"2024-04-20T21:21:07.059226+0000","last_became_peered":"2024-04-20T21:21:07.059226+0000","last_unstale":"2024-04-20T21:21:09.109189+0000","last_undegraded":"2024-04-20T21:21:09.109189+0000","last_fullsized":"2024-04-20T21:21:09.109189+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:54:21.885403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.086173+0000","last_change":"2024-04-20T21:21:08.091843+0000","last_active":"2024-04-20T21:21:09.086173+0000","last_peered":"2024-04-20T21:21:09.086173+0000","last_clean":"2024-04-20T21:21:09.086173+0000","last_became_active":"2024-04-20T21:21:08.088675+0000","last_became_peered":"2024-04-20T21:21:08.088675+0000","last_unstale":"2024-04-20T21:21:09.086173+0000","last_undegraded":"2024-04-20T21:21:09.086173+0000","last_fullsized":"2024-04-20T21:21:09.086173+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:58:50.845304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.093098+0000","last_change":"2024-04-20T21:21:07.054484+0000","last_active":"2024-04-20T21:21:09.093098+0000","last_peered":"2024-04-20T21:21:09.093098+0000","last_clean":"2024-04-20T21:21:09.093098+0000","last_became_active":"2024-04-20T21:21:07.052103+0000","last_became_peered":"2024-04-20T21:21:07.052103+0000","last_unstale":"2024-04-20T21:21:09.093098+0000","last_undegraded":"2024-04-20T21:21:09.093098+0000","last_fullsized":"2024-04-20T21:21:09.093098+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:23:00.694194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.105272+0000","last_change":"2024-04-20T21:21:08.096244+0000","last_active":"2024-04-20T21:21:09.105272+0000","last_peered":"2024-04-20T21:21:09.105272+0000","last_clean":"2024-04-20T21:21:09.105272+0000","last_became_active":"2024-04-20T21:21:08.093495+0000","last_became_peered":"2024-04-20T21:21:08.093495+0000","last_unstale":"2024-04-20T21:21:09.105272+0000","last_undegraded":"2024-04-20T21:21:09.105272+0000","last_fullsized":"2024-04-20T21:21:09.105272+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:07:19.659957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.347141+0000","last_change":"2024-04-20T21:21:07.062344+0000","last_active":"2024-04-20T21:21:09.347141+0000","last_peered":"2024-04-20T21:21:09.347141+0000","last_clean":"2024-04-20T21:21:09.347141+0000","last_became_active":"2024-04-20T21:21:07.061985+0000","last_became_peered":"2024-04-20T21:21:07.061985+0000","last_unstale":"2024-04-20T21:21:09.347141+0000","last_undegraded":"2024-04-20T21:21:09.347141+0000","last_fullsized":"2024-04-20T21:21:09.347141+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:21:50.665197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.346964+0000","last_change":"2024-04-20T21:21:08.107889+0000","last_active":"2024-04-20T21:21:09.346964+0000","last_peered":"2024-04-20T21:21:09.346964+0000","last_clean":"2024-04-20T21:21:09.346964+0000","last_became_active":"2024-04-20T21:21:08.107578+0000","last_became_peered":"2024-04-20T21:21:08.107578+0000","last_unstale":"2024-04-20T21:21:09.346964+0000","last_undegraded":"2024-04-20T21:21:09.346964+0000","last_fullsized":"2024-04-20T21:21:09.346964+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:22:03.289717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.085900+0000","last_change":"2024-04-20T21:21:07.058822+0000","last_active":"2024-04-20T21:21:09.085900+0000","last_peered":"2024-04-20T21:21:09.085900+0000","last_clean":"2024-04-20T21:21:09.085900+0000","last_became_active":"2024-04-20T21:21:07.058499+0000","last_became_peered":"2024-04-20T21:21:07.058499+0000","last_unstale":"2024-04-20T21:21:09.085900+0000","last_undegraded":"2024-04-20T21:21:09.085900+0000","last_fullsized":"2024-04-20T21:21:09.085900+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:23:00.694194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.348404+0000","last_change":"2024-04-20T21:21:08.095104+0000","last_active":"2024-04-20T21:21:09.348404+0000","last_peered":"2024-04-20T21:21:09.348404+0000","last_clean":"2024-04-20T21:21:09.348404+0000","last_became_active":"2024-04-20T21:21:08.093719+0000","last_became_peered":"2024-04-20T21:21:08.093719+0000","last_unstale":"2024-04-20T21:21:09.348404+0000","last_undegraded":"2024-04-20T21:21:09.348404+0000","last_fullsized":"2024-04-20T21:21:09.348404+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:39:14.817307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.091830+0000","last_change":"2024-04-20T21:21:07.063722+0000","last_active":"2024-04-20T21:21:09.091830+0000","last_peered":"2024-04-20T21:21:09.091830+0000","last_clean":"2024-04-20T21:21:09.091830+0000","last_became_active":"2024-04-20T21:21:07.063416+0000","last_became_peered":"2024-04-20T21:21:07.063416+0000","last_unstale":"2024-04-20T21:21:09.091830+0000","last_undegraded":"2024-04-20T21:21:09.091830+0000","last_fullsized":"2024-04-20T21:21:09.091830+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:18:27.328908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.085837+0000","last_change":"2024-04-20T21:21:08.085903+0000","last_active":"2024-04-20T21:21:09.085837+0000","last_peered":"2024-04-20T21:21:09.085837+0000","last_clean":"2024-04-20T21:21:09.085837+0000","last_became_active":"2024-04-20T21:21:08.085552+0000","last_became_peered":"2024-04-20T21:21:08.085552+0000","last_unstale":"2024-04-20T21:21:09.085837+0000","last_undegraded":"2024-04-20T21:21:09.085837+0000","last_fullsized":"2024-04-20T21:21:09.085837+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:32:10.911172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.086178+0000","last_change":"2024-04-20T21:21:07.057886+0000","last_active":"2024-04-20T21:21:09.086178+0000","last_peered":"2024-04-20T21:21:09.086178+0000","last_clean":"2024-04-20T21:21:09.086178+0000","last_became_active":"2024-04-20T21:21:07.057214+0000","last_became_peered":"2024-04-20T21:21:07.057214+0000","last_unstale":"2024-04-20T21:21:09.086178+0000","last_undegraded":"2024-04-20T21:21:09.086178+0000","last_fullsized":"2024-04-20T21:21:09.086178+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:42:41.888140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.346444+0000","last_change":"2024-04-20T21:21:08.099788+0000","last_active":"2024-04-20T21:21:09.346444+0000","last_peered":"2024-04-20T21:21:09.346444+0000","last_clean":"2024-04-20T21:21:09.346444+0000","last_became_active":"2024-04-20T21:21:08.094821+0000","last_became_peered":"2024-04-20T21:21:08.094821+0000","last_unstale":"2024-04-20T21:21:09.346444+0000","last_undegraded":"2024-04-20T21:21:09.346444+0000","last_fullsized":"2024-04-20T21:21:09.346444+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:59:42.451946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.346484+0000","last_change":"2024-04-20T21:21:07.061965+0000","last_active":"2024-04-20T21:21:09.346484+0000","last_peered":"2024-04-20T21:21:09.346484+0000","last_clean":"2024-04-20T21:21:09.346484+0000","last_became_active":"2024-04-20T21:21:07.060801+0000","last_became_peered":"2024-04-20T21:21:07.060801+0000","last_unstale":"2024-04-20T21:21:09.346484+0000","last_undegraded":"2024-04-20T21:21:09.346484+0000","last_fullsized":"2024-04-20T21:21:09.346484+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:27:43.075589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.346631+0000","last_change":"2024-04-20T21:21:08.097808+0000","last_active":"2024-04-20T21:21:09.346631+0000","last_peered":"2024-04-20T21:21:09.346631+0000","last_clean":"2024-04-20T21:21:09.346631+0000","last_became_active":"2024-04-20T21:21:08.097518+0000","last_became_peered":"2024-04-20T21:21:08.097518+0000","last_unstale":"2024-04-20T21:21:09.346631+0000","last_undegraded":"2024-04-20T21:21:09.346631+0000","last_fullsized":"2024-04-20T21:21:09.346631+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:07.020801+0000","last_clean_scrub_stamp":"2024-04-20T21:21:07.020801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:44:23.355317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.091256+0000","last_change":"2024-04-20T21:21:07.062806+0000","last_active":"2024-04-20T21:21:09.091256+0000","last_peered":"2024-04-20T21:21:09.091256+0000","last_clean":"2024-04-20T21:21:09.091256+0000","last_became_active":"2024-04-20T21:21:07.062536+0000","last_became_peered":"2024-04-20T21:21:07.062536+0000","last_unstale":"2024-04-20T21:21:09.091256+0000","last_undegraded":"2024-04-20T21:21:09.091256+0000","last_fullsized":"2024-04-20T21:21:09.091256+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:21:06.006425+0000","last_clean_scrub_stamp":"2024-04-20T21:21:06.006425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:42:41.888140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":238,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T21:21:09.346984+0000","last_change":"2024-04-20T21:17:43.059637+0000","last_active":"2024-04-20T21:21:09.346984+0000","last_peered":"2024-04-20T21:21:09.346984+0000","last_clean":"2024-04-20T21:21:09.346984+0000","last_became_active":"2024-04-20T21:17:43.056207+0000","last_became_peered":"2024-04-20T21:17:43.056207+0000","last_unstale":"2024-04-20T21:21:09.346984+0000","last_undegraded":"2024-04-20T21:21:09.346984+0000","last_fullsized":"2024-04-20T21:21:09.346984+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:41.036290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:41.036290+0000","last_clean_scrub_stamp":"2024-04-20T21:17:41.036290+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:58:51.091337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574892,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32464,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339376,"statfs":{"total":96636764160,"available":96603521024,"internally_reserved":0,"allocated":1310720,"data_stored":886308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574893,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":708608,"data_stored":295780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574895,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27720,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344120,"statfs":{"total":96636764160,"available":96608378880,"internally_reserved":0,"allocated":716800,"data_stored":296432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574894,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27724,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344116,"statfs":{"total":96636764160,"available":96608374784,"internally_reserved":0,"allocated":720896,"data_stored":297392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31884,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339956,"statfs":{"total":96636764160,"available":96604114944,"internally_reserved":0,"allocated":716800,"data_stored":295836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574894,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31884,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339956,"statfs":{"total":96636764160,"available":96604114944,"internally_reserved":0,"allocated":716800,"data_stored":295836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574893,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31876,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339964,"statfs":{"total":96636764160,"available":96604123136,"internally_reserved":0,"allocated":712704,"data_stored":295870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574894,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32476,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1318912,"data_stored":888342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T21:21:12.968 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-20T21:21:12.968 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-20T21:21:13.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.221+0000 7f75ad800640 1 Processor -- start 2024-04-20T21:21:13.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.221+0000 7f75ad800640 1 -- start start 2024-04-20T21:21:13.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.222+0000 7f75ad800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f75a80d45a0 0x7f75a80d3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:13.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.222+0000 7f75ad800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f75a815ea70 0x7f75a80d4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:13.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.222+0000 7f75ad800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f75a806d570 0x7f75a80d1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:13.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.222+0000 7f75ad800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f75a80cf120 con 0x7f75a806d570 2024-04-20T21:21:13.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.222+0000 7f75ad800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f75a806de70 con 0x7f75a80d45a0 2024-04-20T21:21:13.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.222+0000 7f75ad800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f75a806e150 con 0x7f75a815ea70 2024-04-20T21:21:13.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.223+0000 7f75ace00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f75a806d570 0x7f75a80d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:13.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.223+0000 7f75ace00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f75a806d570 0x7f75a80d1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47828/0 (socket says 172.21.15.86:47828) 2024-04-20T21:21:13.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.223+0000 7f75ace00640 1 -- 172.21.15.86:0/4006515381 learned_addr learned my addr 172.21.15.86:0/4006515381 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:13.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.223+0000 7f75a7e00640 1 --2- 172.21.15.86:0/4006515381 >> v2:172.21.15.144:3300/0 conn(0x7f75a80d45a0 0x7f75a80d3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:13.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.223+0000 7f759fe00640 1 --2- 172.21.15.86:0/4006515381 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f75a815ea70 0x7f75a80d4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:13.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.223+0000 7f75ace00640 1 -- 172.21.15.86:0/4006515381 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f75a815ea70 msgr2=0x7f75a80d4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:13.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.223+0000 7f75ace00640 1 --2- 172.21.15.86:0/4006515381 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f75a815ea70 0x7f75a80d4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:13.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.223+0000 7f75ace00640 1 -- 172.21.15.86:0/4006515381 >> v2:172.21.15.144:3300/0 conn(0x7f75a80d45a0 msgr2=0x7f75a80d3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:13.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.223+0000 7f75ace00640 1 --2- 172.21.15.86:0/4006515381 >> v2:172.21.15.144:3300/0 conn(0x7f75a80d45a0 0x7f75a80d3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:13.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.223+0000 7f75ace00640 1 -- 172.21.15.86:0/4006515381 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75a806e430 con 0x7f75a806d570 2024-04-20T21:21:13.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.224+0000 7f75ace00640 1 --2- 172.21.15.86:0/4006515381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75a806d570 0x7f75a80d1100 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f75a808e910 tx=0x7f75a002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a67537fc1996235 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:13.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.224+0000 7f75a7400640 1 -- 172.21.15.86:0/4006515381 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a002c9e0 con 0x7f75a806d570 2024-04-20T21:21:13.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.224+0000 7f75a7400640 1 -- 172.21.15.86:0/4006515381 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75a002cb80 con 0x7f75a806d570 2024-04-20T21:21:13.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.224+0000 7f75a7400640 1 -- 172.21.15.86:0/4006515381 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a00343f0 con 0x7f75a806d570 2024-04-20T21:21:13.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.225+0000 7f75ad800640 1 -- 172.21.15.86:0/4006515381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75a806d570 msgr2=0x7f75a80d1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:13.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.225+0000 7f75ad800640 1 --2- 172.21.15.86:0/4006515381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75a806d570 0x7f75a80d1100 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f75a808e910 tx=0x7f75a002be40 comp rx=0 tx=0).stop 2024-04-20T21:21:13.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.225+0000 7f75ad800640 1 -- 172.21.15.86:0/4006515381 shutdown_connections 2024-04-20T21:21:13.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.225+0000 7f75ad800640 1 --2- 172.21.15.86:0/4006515381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75a806d570 0x7f75a80d1100 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:13.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.225+0000 7f75ad800640 1 --2- 172.21.15.86:0/4006515381 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f75a815ea70 0x7f75a80d4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:13.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.225+0000 7f75ad800640 1 --2- 172.21.15.86:0/4006515381 >> v2:172.21.15.144:3300/0 conn(0x7f75a80d45a0 0x7f75a80d3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:13.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.225+0000 7f75ad800640 1 -- 172.21.15.86:0/4006515381 >> 172.21.15.86:0/4006515381 conn(0x7f75a809d9d0 msgr2=0x7f75a808bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:13.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.225+0000 7f75ad800640 1 -- 172.21.15.86:0/4006515381 shutdown_connections 2024-04-20T21:21:13.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.225+0000 7f75ad800640 1 -- 172.21.15.86:0/4006515381 wait complete. 2024-04-20T21:21:13.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.227+0000 7f75ad800640 1 Processor -- start 2024-04-20T21:21:13.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.227+0000 7f75ad800640 1 -- start start 2024-04-20T21:21:13.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.228+0000 7f75ad800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75a806d570 0x7f75a80cd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:13.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.228+0000 7f75ad800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75a80d45a0 0x7f75a815fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:13.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.228+0000 7f75ad800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f75a815ea70 0x7f75a8155f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:13.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.228+0000 7f75ad800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f75a806da00 con 0x7f75a80d45a0 2024-04-20T21:21:13.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.228+0000 7f75ad800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f75a80d4ec0 con 0x7f75a806d570 2024-04-20T21:21:13.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.228+0000 7f75ad800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f75a806db80 con 0x7f75a815ea70 2024-04-20T21:21:13.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.228+0000 7f759fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75a80d45a0 0x7f75a815fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:13.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.228+0000 7f759fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75a80d45a0 0x7f75a815fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47844/0 (socket says 172.21.15.86:47844) 2024-04-20T21:21:13.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.228+0000 7f759fe00640 1 -- 172.21.15.86:0/1699792957 learned_addr learned my addr 172.21.15.86:0/1699792957 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:13.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.229+0000 7f75ace00640 1 --2- 172.21.15.86:0/1699792957 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f75a815ea70 0x7f75a8155f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:13.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.229+0000 7f75a7e00640 1 --2- 172.21.15.86:0/1699792957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75a806d570 0x7f75a80cd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:13.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.229+0000 7f759fe00640 1 -- 172.21.15.86:0/1699792957 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f75a815ea70 msgr2=0x7f75a8155f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:13.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.229+0000 7f759fe00640 1 --2- 172.21.15.86:0/1699792957 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f75a815ea70 0x7f75a8155f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:13.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.229+0000 7f759fe00640 1 -- 172.21.15.86:0/1699792957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75a806d570 msgr2=0x7f75a80cd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:13.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.229+0000 7f759fe00640 1 --2- 172.21.15.86:0/1699792957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75a806d570 0x7f75a80cd7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:13.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.229+0000 7f759fe00640 1 -- 172.21.15.86:0/1699792957 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75a80cdff0 con 0x7f75a80d45a0 2024-04-20T21:21:13.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.229+0000 7f759fe00640 1 --2- 172.21.15.86:0/1699792957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75a80d45a0 0x7f75a815fea0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f759000c7f0 tx=0x7f759000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:13.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.229+0000 7f75a4c00640 1 -- 172.21.15.86:0/1699792957 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7590015070 con 0x7f75a80d45a0 2024-04-20T21:21:13.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.229+0000 7f75a4c00640 1 -- 172.21.15.86:0/1699792957 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7590004220 con 0x7f75a80d45a0 2024-04-20T21:21:13.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.229+0000 7f75a4c00640 1 -- 172.21.15.86:0/1699792957 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7590007620 con 0x7f75a80d45a0 2024-04-20T21:21:13.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.229+0000 7f75ad800640 1 -- 172.21.15.86:0/1699792957 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75a80c9a80 con 0x7f75a80d45a0 2024-04-20T21:21:13.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.230+0000 7f75ad800640 1 -- 172.21.15.86:0/1699792957 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75a80cdce0 con 0x7f75a80d45a0 2024-04-20T21:21:13.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.231+0000 7f75ad800640 1 -- 172.21.15.86:0/1699792957 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75a804fd90 con 0x7f75a80d45a0 2024-04-20T21:21:13.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.232+0000 7f75a4c00640 1 -- 172.21.15.86:0/1699792957 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f75900079d0 con 0x7f75a80d45a0 2024-04-20T21:21:13.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.233+0000 7f75a4c00640 1 --2- 172.21.15.86:0/1699792957 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f758406c8f0 0x7f758406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:13.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.233+0000 7f75a4c00640 1 -- 172.21.15.86:0/1699792957 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f759008be70 con 0x7f75a80d45a0 2024-04-20T21:21:13.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.233+0000 7f75a7e00640 1 --2- 172.21.15.86:0/1699792957 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f758406c8f0 0x7f758406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:13.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.233+0000 7f75a7e00640 1 --2- 172.21.15.86:0/1699792957 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f758406c8f0 0x7f758406edb0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7598006fd0 tx=0x7f7598008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:13.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.237+0000 7f75a4c00640 1 -- 172.21.15.86:0/1699792957 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f759005a290 con 0x7f75a80d45a0 2024-04-20T21:21:13.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.507+0000 7f75ad800640 1 -- 172.21.15.86:0/1699792957 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f75a815e020 con 0x7f75a80d45a0 2024-04-20T21:21:13.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.511+0000 7f75a4c00640 1 -- 172.21.15.86:0/1699792957 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7f7590059b90 con 0x7f75a80d45a0 2024-04-20T21:21:13.511 INFO:teuthology.orchestra.run.smithi086.stdout:[client.0] 2024-04-20T21:21:13.511 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQDBMCRmHWxyKRAANzhRi4eajp68h/jWkhFMnw== 2024-04-20T21:21:13.511 INFO:teuthology.orchestra.run.smithi086.stdout: caps mds = "allow" 2024-04-20T21:21:13.511 INFO:teuthology.orchestra.run.smithi086.stdout: caps mon = "allow r" 2024-04-20T21:21:13.511 INFO:teuthology.orchestra.run.smithi086.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-20T21:21:13.511 INFO:teuthology.orchestra.run.smithi086.stderr:updated caps for client.0 2024-04-20T21:21:13.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.514+0000 7f75ad800640 1 -- 172.21.15.86:0/1699792957 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f758406c8f0 msgr2=0x7f758406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:13.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.514+0000 7f75ad800640 1 --2- 172.21.15.86:0/1699792957 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f758406c8f0 0x7f758406edb0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7598006fd0 tx=0x7f7598008040 comp rx=0 tx=0).stop 2024-04-20T21:21:13.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.514+0000 7f75ad800640 1 -- 172.21.15.86:0/1699792957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75a80d45a0 msgr2=0x7f75a815fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:13.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.514+0000 7f75ad800640 1 --2- 172.21.15.86:0/1699792957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75a80d45a0 0x7f75a815fea0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f759000c7f0 tx=0x7f759000ccb0 comp rx=0 tx=0).stop 2024-04-20T21:21:13.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.514+0000 7f75ad800640 1 -- 172.21.15.86:0/1699792957 shutdown_connections 2024-04-20T21:21:13.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.514+0000 7f75ad800640 1 --2- 172.21.15.86:0/1699792957 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f758406c8f0 0x7f758406edb0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:13.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.514+0000 7f75ad800640 1 --2- 172.21.15.86:0/1699792957 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f75a815ea70 0x7f75a8155f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:13.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.514+0000 7f75ad800640 1 --2- 172.21.15.86:0/1699792957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75a80d45a0 0x7f75a815fea0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:13.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.514+0000 7f75ad800640 1 --2- 172.21.15.86:0/1699792957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75a806d570 0x7f75a80cd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:13.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.514+0000 7f75ad800640 1 -- 172.21.15.86:0/1699792957 >> 172.21.15.86:0/1699792957 conn(0x7f75a809d9d0 msgr2=0x7f75a808bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:13.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.514+0000 7f75ad800640 1 -- 172.21.15.86:0/1699792957 shutdown_connections 2024-04-20T21:21:13.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.514+0000 7f75ad800640 1 -- 172.21.15.86:0/1699792957 wait complete. 2024-04-20T21:21:13.532 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:21:13.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.730+0000 7f2d1d000640 1 Processor -- start 2024-04-20T21:21:13.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.730+0000 7f2d1d000640 1 -- start start 2024-04-20T21:21:13.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.731+0000 7f2d1d000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2d18156ae0 0x7f2d1815d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:13.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.731+0000 7f2d1d000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d18160890 con 0x7f2d18160080 2024-04-20T21:21:13.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.731+0000 7f2d1d000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d18160b70 con 0x7f2d180ad7f0 2024-04-20T21:21:13.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.732+0000 7f2d1d000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f2d1806db20 con 0x7f2d18156ae0 2024-04-20T21:21:13.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.732+0000 7f2d17400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2d18156ae0 0x7f2d1815d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:13.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.732+0000 7f2d0fe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f2d18160080 0x7f2d180ad0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:48472/0 (socket says 172.21.15.86:48472) 2024-04-20T21:21:13.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.732+0000 7f2d0fe00640 1 -- 172.21.15.86:0/3237357523 learned_addr learned my addr 172.21.15.86:0/3237357523 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:21:13.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.732+0000 7f2d17400640 1 -- 172.21.15.86:0/3237357523 >> v1:172.21.15.144:6789/0 conn(0x7f2d180ad7f0 legacy=0x7f2d1806e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:13.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.732+0000 7f2d17400640 1 -- 172.21.15.86:0/3237357523 >> v1:172.21.15.86:6789/0 conn(0x7f2d18160080 legacy=0x7f2d180ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:13.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.732+0000 7f2d16a00640 1 -- 172.21.15.86:0/3237357523 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 600201789 0 0) 0x7f2d18160890 con 0x7f2d18160080 2024-04-20T21:21:13.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.732+0000 7f2d17400640 1 -- 172.21.15.86:0/3237357523 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d18160e50 con 0x7f2d18156ae0 2024-04-20T21:21:13.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.733+0000 7f2d17400640 1 --2- 172.21.15.86:0/3237357523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d18156ae0 0x7f2d1815d6a0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f2d080097a0 tx=0x7f2d0802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86040182c9f192e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:13.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.734+0000 7f2d16a00640 1 -- 172.21.15.86:0/3237357523 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d08034070 con 0x7f2d18156ae0 2024-04-20T21:21:13.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.734+0000 7f2d16a00640 1 -- 172.21.15.86:0/3237357523 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d08030610 con 0x7f2d18156ae0 2024-04-20T21:21:13.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.734+0000 7f2d16a00640 1 -- 172.21.15.86:0/3237357523 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d080309e0 con 0x7f2d18156ae0 2024-04-20T21:21:13.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.734+0000 7f2d1d000640 1 -- 172.21.15.86:0/3237357523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d18156ae0 msgr2=0x7f2d1815d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:13.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.734+0000 7f2d1d000640 1 --2- 172.21.15.86:0/3237357523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d18156ae0 0x7f2d1815d6a0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f2d080097a0 tx=0x7f2d0802b960 comp rx=0 tx=0).stop 2024-04-20T21:21:13.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.734+0000 7f2d1d000640 1 -- 172.21.15.86:0/3237357523 shutdown_connections 2024-04-20T21:21:13.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.734+0000 7f2d1d000640 1 --2- 172.21.15.86:0/3237357523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d18156ae0 0x7f2d1815d6a0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:13.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.734+0000 7f2d1d000640 1 -- 172.21.15.86:0/3237357523 >> 172.21.15.86:0/3237357523 conn(0x7f2d180d0170 msgr2=0x7f2d180d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:13.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.734+0000 7f2d1d000640 1 -- 172.21.15.86:0/3237357523 shutdown_connections 2024-04-20T21:21:13.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.734+0000 7f2d1d000640 1 -- 172.21.15.86:0/3237357523 wait complete. 2024-04-20T21:21:13.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.736+0000 7f2d1d000640 1 Processor -- start 2024-04-20T21:21:13.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.736+0000 7f2d1d000640 1 -- start start 2024-04-20T21:21:13.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.737+0000 7f2d1d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d180ad7f0 0x7f2d18157040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:13.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.737+0000 7f2d1d000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d18156ae0 0x7f2d180999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:13.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.737+0000 7f2d1d000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d18160080 0x7f2d181f62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:13.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.737+0000 7f2d1d000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2d18160500 con 0x7f2d180ad7f0 2024-04-20T21:21:13.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.737+0000 7f2d1d000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2d18156ec0 con 0x7f2d18156ae0 2024-04-20T21:21:13.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.737+0000 7f2d1d000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2d1806e9a0 con 0x7f2d18160080 2024-04-20T21:21:13.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.737+0000 7f2d17400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d180ad7f0 0x7f2d18157040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:13.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.737+0000 7f2d17400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d180ad7f0 0x7f2d18157040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47858/0 (socket says 172.21.15.86:47858) 2024-04-20T21:21:13.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.737+0000 7f2d17400640 1 -- 172.21.15.86:0/1440696250 learned_addr learned my addr 172.21.15.86:0/1440696250 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:13.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.738+0000 7f2d0fe00640 1 --2- 172.21.15.86:0/1440696250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d18156ae0 0x7f2d180999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:13.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.738+0000 7f2d17400640 1 -- 172.21.15.86:0/1440696250 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d18160080 msgr2=0x7f2d181f62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:13.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.738+0000 7f2d17e00640 1 --2- 172.21.15.86:0/1440696250 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d18160080 0x7f2d181f62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:13.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.738+0000 7f2d17400640 1 --2- 172.21.15.86:0/1440696250 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d18160080 0x7f2d181f62e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:13.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.738+0000 7f2d17400640 1 -- 172.21.15.86:0/1440696250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d18156ae0 msgr2=0x7f2d180999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:13.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.738+0000 7f2d17400640 1 --2- 172.21.15.86:0/1440696250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d18156ae0 0x7f2d180999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:13.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.738+0000 7f2d17400640 1 -- 172.21.15.86:0/1440696250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d18159b30 con 0x7f2d180ad7f0 2024-04-20T21:21:13.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.738+0000 7f2d0fe00640 1 --2- 172.21.15.86:0/1440696250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d18156ae0 0x7f2d180999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:21:13.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.738+0000 7f2d17400640 1 --2- 172.21.15.86:0/1440696250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d180ad7f0 0x7f2d18157040 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f2d0802bf70 tx=0x7f2d080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:13.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.738+0000 7f2d0f400640 1 -- 172.21.15.86:0/1440696250 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d08034040 con 0x7f2d180ad7f0 2024-04-20T21:21:13.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.738+0000 7f2d0f400640 1 -- 172.21.15.86:0/1440696250 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d08039a10 con 0x7f2d180ad7f0 2024-04-20T21:21:13.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.738+0000 7f2d0f400640 1 -- 172.21.15.86:0/1440696250 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d08039de0 con 0x7f2d180ad7f0 2024-04-20T21:21:13.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.738+0000 7f2d1d000640 1 -- 172.21.15.86:0/1440696250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d18159e10 con 0x7f2d180ad7f0 2024-04-20T21:21:13.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.739+0000 7f2d1d000640 1 -- 172.21.15.86:0/1440696250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d1815ccc0 con 0x7f2d180ad7f0 2024-04-20T21:21:13.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.740+0000 7f2d1d000640 1 -- 172.21.15.86:0/1440696250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d180b5610 con 0x7f2d180ad7f0 2024-04-20T21:21:13.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.740+0000 7f2d0f400640 1 -- 172.21.15.86:0/1440696250 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2d08037070 con 0x7f2d180ad7f0 2024-04-20T21:21:13.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.741+0000 7f2d0f400640 1 --2- 172.21.15.86:0/1440696250 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2cf406c8f0 0x7f2cf406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:13.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.741+0000 7f2d0f400640 1 -- 172.21.15.86:0/1440696250 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f2d080adcd0 con 0x7f2d180ad7f0 2024-04-20T21:21:13.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.742+0000 7f2d0fe00640 1 --2- 172.21.15.86:0/1440696250 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2cf406c8f0 0x7f2cf406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:13.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.742+0000 7f2d0fe00640 1 --2- 172.21.15.86:0/1440696250 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2cf406c8f0 0x7f2cf406edb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2d18159710 tx=0x7f2d00009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:13.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.748+0000 7f2d0f400640 1 -- 172.21.15.86:0/1440696250 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2d180b5610 con 0x7f2d180ad7f0 2024-04-20T21:21:13.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.987+0000 7f2d1d000640 1 -- 172.21.15.86:0/1440696250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d1804fcf0 con 0x7f2d180ad7f0 2024-04-20T21:21:13.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.988+0000 7f2d0f400640 1 -- 172.21.15.86:0/1440696250 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+5205 (secure 0 0 0) 0x7f2d0807c200 con 0x7f2d180ad7f0 2024-04-20T21:21:13.987 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:21:13.987 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4913,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3933408964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3933408964},{"type":"v1","addr":"172.21.15.144:6833","nonce":3933408964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5221,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/1522070692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1522070692},{"type":"v1","addr":"172.21.15.144:6835","nonce":1522070692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":6132,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/1352073341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1352073341},{"type":"v1","addr":"172.21.15.86:6836","nonce":1352073341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:21:07.502800+0000","modified":"2024-04-20T21:21:10.553495+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6135},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6135":{"gid":6135,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3008383261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3008383261},{"type":"v1","addr":"172.21.15.86:6837","nonce":3008383261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6135,"qdb_cluster":[6135]},"id":4}]} 2024-04-20T21:21:13.988 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 32 2024-04-20T21:21:13.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.991+0000 7f2d1d000640 1 -- 172.21.15.86:0/1440696250 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2cf406c8f0 msgr2=0x7f2cf406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:13.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.991+0000 7f2d1d000640 1 --2- 172.21.15.86:0/1440696250 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2cf406c8f0 0x7f2cf406edb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2d18159710 tx=0x7f2d00009040 comp rx=0 tx=0).stop 2024-04-20T21:21:13.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.991+0000 7f2d1d000640 1 -- 172.21.15.86:0/1440696250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d180ad7f0 msgr2=0x7f2d18157040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:13.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.991+0000 7f2d1d000640 1 --2- 172.21.15.86:0/1440696250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d180ad7f0 0x7f2d18157040 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f2d0802bf70 tx=0x7f2d080046c0 comp rx=0 tx=0).stop 2024-04-20T21:21:13.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.991+0000 7f2d1d000640 1 -- 172.21.15.86:0/1440696250 shutdown_connections 2024-04-20T21:21:13.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.991+0000 7f2d1d000640 1 --2- 172.21.15.86:0/1440696250 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2cf406c8f0 0x7f2cf406edb0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:13.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.991+0000 7f2d1d000640 1 --2- 172.21.15.86:0/1440696250 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2d18160080 0x7f2d181f62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:13.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.991+0000 7f2d1d000640 1 --2- 172.21.15.86:0/1440696250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d18156ae0 0x7f2d180999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:13.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.991+0000 7f2d1d000640 1 --2- 172.21.15.86:0/1440696250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d180ad7f0 0x7f2d18157040 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:13.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.991+0000 7f2d1d000640 1 -- 172.21.15.86:0/1440696250 >> 172.21.15.86:0/1440696250 conn(0x7f2d180d0170 msgr2=0x7f2d180b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:13.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.991+0000 7f2d1d000640 1 -- 172.21.15.86:0/1440696250 shutdown_connections 2024-04-20T21:21:13.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:13.991+0000 7f2d1d000640 1 -- 172.21.15.86:0/1440696250 wait complete. 2024-04-20T21:21:14.410 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 32, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T21:21:07.502800+0000', 'modified': '2024-04-20T21:21:10.553495+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6135}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6135': {'gid': 6135, 'name': 'c', 'rank': 0, 'incarnation': 28, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/3008383261', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6835', 'nonce': 3008383261}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 3008383261}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6135, 'qdb_cluster': [6135]} 2024-04-20T21:21:14.410 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-20T21:21:14.411 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-20T21:21:14.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.618+0000 7f2fdca00640 1 Processor -- start 2024-04-20T21:21:14.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.618+0000 7f2fdca00640 1 -- start start 2024-04-20T21:21:14.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.620+0000 7f2fdca00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2fd8073540 0x7f2fd80d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:14.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.620+0000 7f2fdca00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fd80af870 0x7f2fd80d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:14.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.620+0000 7f2fdca00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2fd80cfa00 0x7f2fd815f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:14.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.620+0000 7f2fdca00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f2fd80ce1c0 con 0x7f2fd8073540 2024-04-20T21:21:14.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.620+0000 7f2fdca00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f2fd80ce4a0 con 0x7f2fd80cfa00 2024-04-20T21:21:14.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.620+0000 7f2fdca00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2fd80ce780 con 0x7f2fd80af870 2024-04-20T21:21:14.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.620+0000 7f2fd6800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2fd8073540 0x7f2fd80d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:14.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.620+0000 7f2fd6800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2fd8073540 0x7f2fd80d00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47872/0 (socket says 172.21.15.86:47872) 2024-04-20T21:21:14.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.620+0000 7f2fd6800640 1 -- 172.21.15.86:0/2537430889 learned_addr learned my addr 172.21.15.86:0/2537430889 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:14.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.620+0000 7f2fd6800640 1 -- 172.21.15.86:0/2537430889 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fd80af870 msgr2=0x7f2fd80d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:14.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.620+0000 7f2fd6800640 1 --2- 172.21.15.86:0/2537430889 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fd80af870 0x7f2fd80d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:14.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.620+0000 7f2fd6800640 1 -- 172.21.15.86:0/2537430889 >> v2:172.21.15.144:3300/0 conn(0x7f2fd80cfa00 msgr2=0x7f2fd815f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:14.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.620+0000 7f2fd7200640 1 --2- 172.21.15.86:0/2537430889 >> v2:172.21.15.144:3300/0 conn(0x7f2fd80cfa00 0x7f2fd815f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:14.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.621+0000 7f2fd6800640 1 --2- 172.21.15.86:0/2537430889 >> v2:172.21.15.144:3300/0 conn(0x7f2fd80cfa00 0x7f2fd815f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:14.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.621+0000 7f2fd6800640 1 -- 172.21.15.86:0/2537430889 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fd80cea60 con 0x7f2fd8073540 2024-04-20T21:21:14.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.621+0000 7f2fd7200640 1 --2- 172.21.15.86:0/2537430889 >> v2:172.21.15.144:3300/0 conn(0x7f2fd80cfa00 0x7f2fd815f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:21:14.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.622+0000 7f2fd6800640 1 --2- 172.21.15.86:0/2537430889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fd8073540 0x7f2fd80d00b0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f2fd80b4d20 tx=0x7f2fc802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e327ae16f7a407e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:14.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.622+0000 7f2fd5400640 1 -- 172.21.15.86:0/2537430889 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc8035020 con 0x7f2fd8073540 2024-04-20T21:21:14.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.622+0000 7f2fd5400640 1 -- 172.21.15.86:0/2537430889 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fc802d040 con 0x7f2fd8073540 2024-04-20T21:21:14.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.622+0000 7f2fd5400640 1 -- 172.21.15.86:0/2537430889 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc8031a80 con 0x7f2fd8073540 2024-04-20T21:21:14.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.622+0000 7f2fdca00640 1 -- 172.21.15.86:0/2537430889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fd8073540 msgr2=0x7f2fd80d00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:14.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.622+0000 7f2fdca00640 1 --2- 172.21.15.86:0/2537430889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fd8073540 0x7f2fd80d00b0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f2fd80b4d20 tx=0x7f2fc802b900 comp rx=0 tx=0).stop 2024-04-20T21:21:14.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.623+0000 7f2fdca00640 1 -- 172.21.15.86:0/2537430889 shutdown_connections 2024-04-20T21:21:14.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.623+0000 7f2fdca00640 1 --2- 172.21.15.86:0/2537430889 >> v2:172.21.15.144:3300/0 conn(0x7f2fd80cfa00 0x7f2fd815f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:14.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.623+0000 7f2fdca00640 1 --2- 172.21.15.86:0/2537430889 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fd80af870 0x7f2fd80d05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:14.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.623+0000 7f2fdca00640 1 --2- 172.21.15.86:0/2537430889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fd8073540 0x7f2fd80d00b0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:14.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.623+0000 7f2fdca00640 1 -- 172.21.15.86:0/2537430889 >> 172.21.15.86:0/2537430889 conn(0x7f2fd806bb60 msgr2=0x7f2fd806bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:14.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.623+0000 7f2fdca00640 1 -- 172.21.15.86:0/2537430889 shutdown_connections 2024-04-20T21:21:14.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.623+0000 7f2fdca00640 1 -- 172.21.15.86:0/2537430889 wait complete. 2024-04-20T21:21:14.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.624+0000 7f2fdca00640 1 Processor -- start 2024-04-20T21:21:14.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.624+0000 7f2fdca00640 1 -- start start 2024-04-20T21:21:14.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.625+0000 7f2fdca00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fd8073540 0x7f2fd815c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:14.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.625+0000 7f2fdca00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fd80cfa00 0x7f2fd815e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:14.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.625+0000 7f2fdca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fd815af10 0x7f2fd80c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:14.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.625+0000 7f2fdca00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd8099830 con 0x7f2fd815af10 2024-04-20T21:21:14.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.625+0000 7f2fdca00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd8073920 con 0x7f2fd80cfa00 2024-04-20T21:21:14.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.625+0000 7f2fdca00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2fd80999b0 con 0x7f2fd8073540 2024-04-20T21:21:14.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.626+0000 7f2fd7200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fd815af10 0x7f2fd80c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:14.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.626+0000 7f2fd7200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fd815af10 0x7f2fd80c54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47880/0 (socket says 172.21.15.86:47880) 2024-04-20T21:21:14.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.626+0000 7f2fd7200640 1 -- 172.21.15.86:0/3977139906 learned_addr learned my addr 172.21.15.86:0/3977139906 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:14.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.626+0000 7f2fd6800640 1 --2- 172.21.15.86:0/3977139906 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fd8073540 0x7f2fd815c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:14.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.626+0000 7f2fd7200640 1 -- 172.21.15.86:0/3977139906 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fd8073540 msgr2=0x7f2fd815c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:14.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.626+0000 7f2fd7200640 1 --2- 172.21.15.86:0/3977139906 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fd8073540 0x7f2fd815c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:14.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.626+0000 7f2fd7200640 1 -- 172.21.15.86:0/3977139906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fd80cfa00 msgr2=0x7f2fd815e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:14.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.626+0000 7f2fd7200640 1 --2- 172.21.15.86:0/3977139906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fd80cfa00 0x7f2fd815e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:14.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.626+0000 7f2fd7200640 1 -- 172.21.15.86:0/3977139906 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fd80c7c70 con 0x7f2fd815af10 2024-04-20T21:21:14.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.626+0000 7f2fd7200640 1 --2- 172.21.15.86:0/3977139906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fd815af10 0x7f2fd80c54a0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f2fd000cd50 tx=0x7f2fd000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:14.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.626+0000 7f2fcea00640 1 -- 172.21.15.86:0/3977139906 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd000f680 con 0x7f2fd815af10 2024-04-20T21:21:14.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.626+0000 7f2fcea00640 1 -- 172.21.15.86:0/3977139906 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fd000f840 con 0x7f2fd815af10 2024-04-20T21:21:14.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.627+0000 7f2fdca00640 1 -- 172.21.15.86:0/3977139906 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fd815e7a0 con 0x7f2fd815af10 2024-04-20T21:21:14.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.627+0000 7f2fcea00640 1 -- 172.21.15.86:0/3977139906 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd000fc30 con 0x7f2fd815af10 2024-04-20T21:21:14.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.627+0000 7f2fdca00640 1 -- 172.21.15.86:0/3977139906 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fd815cc90 con 0x7f2fd815af10 2024-04-20T21:21:14.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.629+0000 7f2fcea00640 1 -- 172.21.15.86:0/3977139906 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2fd00213f0 con 0x7f2fd815af10 2024-04-20T21:21:14.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.629+0000 7f2fcea00640 1 --2- 172.21.15.86:0/3977139906 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2fb806c8f0 0x7f2fb806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:14.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.630+0000 7f2fcea00640 1 -- 172.21.15.86:0/3977139906 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f2fd008c930 con 0x7f2fd815af10 2024-04-20T21:21:14.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.630+0000 7f2fd6800640 1 --2- 172.21.15.86:0/3977139906 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2fb806c8f0 0x7f2fb806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:14.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.631+0000 7f2fd6800640 1 --2- 172.21.15.86:0/3977139906 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2fb806c8f0 0x7f2fb806edb0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8036000 tx=0x7f2fc8008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:14.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.631+0000 7f2fe019f740 1 Processor -- start 2024-04-20T21:21:14.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.631+0000 7f2fe019f740 1 -- start start 2024-04-20T21:21:14.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.632+0000 7f2fe019f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563387e14670 0x563387dcbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:14.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.632+0000 7f2fe019f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563387dca6f0 0x563387dcc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:14.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.632+0000 7f2fe019f740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563387dc6b90 0x563387df6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:14.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.632+0000 7f2fe019f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563387d329d0 con 0x563387dca6f0 2024-04-20T21:21:14.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.632+0000 7f2fe019f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563387cd9270 con 0x563387e14670 2024-04-20T21:21:14.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.632+0000 7f2fe019f740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x563387d2e990 con 0x563387dc6b90 2024-04-20T21:21:14.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.632+0000 7f2fd5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563387dca6f0 0x563387dcc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:14.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.632+0000 7f2fd5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563387dca6f0 0x563387dcc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47888/0 (socket says 172.21.15.86:47888) 2024-04-20T21:21:14.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.632+0000 7f2fd5e00640 1 -- 172.21.15.86:0/4263017328 learned_addr learned my addr 172.21.15.86:0/4263017328 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:14.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.632+0000 7f2fd5e00640 1 -- 172.21.15.86:0/4263017328 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563387dc6b90 msgr2=0x563387df6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:14.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.632+0000 7f2fd5e00640 1 --2- 172.21.15.86:0/4263017328 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563387dc6b90 0x563387df6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:14.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.632+0000 7f2fd5e00640 1 -- 172.21.15.86:0/4263017328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563387e14670 msgr2=0x563387dcbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:14.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.632+0000 7f2fd5e00640 1 --2- 172.21.15.86:0/4263017328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563387e14670 0x563387dcbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:14.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.632+0000 7f2fd5e00640 1 -- 172.21.15.86:0/4263017328 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563387a452d0 con 0x563387dca6f0 2024-04-20T21:21:14.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.633+0000 7f2fd5e00640 1 --2- 172.21.15.86:0/4263017328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563387dca6f0 0x563387dcc1e0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4009750 tx=0x7f2fc4009c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99acd778de3b9d27 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:14.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.633+0000 7f2fc3000640 1 -- 172.21.15.86:0/4263017328 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc4015020 con 0x563387dca6f0 2024-04-20T21:21:14.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.633+0000 7f2fc3000640 1 -- 172.21.15.86:0/4263017328 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fc400bad0 con 0x563387dca6f0 2024-04-20T21:21:14.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.633+0000 7f2fc3000640 1 -- 172.21.15.86:0/4263017328 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc4009e60 con 0x563387dca6f0 2024-04-20T21:21:14.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.633+0000 7f2fe019f740 1 -- 172.21.15.86:0/4263017328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563387dca6f0 msgr2=0x563387dcc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:14.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.633+0000 7f2fe019f740 1 --2- 172.21.15.86:0/4263017328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563387dca6f0 0x563387dcc1e0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4009750 tx=0x7f2fc4009c10 comp rx=0 tx=0).stop 2024-04-20T21:21:14.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.634+0000 7f2fe019f740 1 -- 172.21.15.86:0/4263017328 shutdown_connections 2024-04-20T21:21:14.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.634+0000 7f2fe019f740 1 --2- 172.21.15.86:0/4263017328 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563387dc6b90 0x563387df6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:14.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.634+0000 7f2fe019f740 1 --2- 172.21.15.86:0/4263017328 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563387dca6f0 0x563387dcc1e0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:14.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.634+0000 7f2fe019f740 1 --2- 172.21.15.86:0/4263017328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563387e14670 0x563387dcbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:14.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.634+0000 7f2fe019f740 1 -- 172.21.15.86:0/4263017328 >> 172.21.15.86:0/4263017328 conn(0x563387dc8e00 msgr2=0x563387e11980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:14.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.634+0000 7f2fe019f740 1 -- 172.21.15.86:0/4263017328 shutdown_connections 2024-04-20T21:21:14.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.634+0000 7f2fe019f740 1 -- 172.21.15.86:0/4263017328 wait complete. 2024-04-20T21:21:14.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.634+0000 7f2fe019f740 1 -- start start 2024-04-20T21:21:14.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.634+0000 7f2fe019f740 1 Processor -- start 2024-04-20T21:21:14.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.635+0000 7f2fe019f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563387dca6f0 0x563387e10e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:14.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.635+0000 7f2fe019f740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563387e14670 0x563387dee8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:14.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.635+0000 7f2fe019f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563387e180d0 0x563387deee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:14.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.635+0000 7f2fe019f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563387dcac00 con 0x563387e180d0 2024-04-20T21:21:14.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.635+0000 7f2fe019f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563387cec5e0 con 0x563387dca6f0 2024-04-20T21:21:14.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.635+0000 7f2fe019f740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x563387dcaee0 con 0x563387e14670 2024-04-20T21:21:14.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.635+0000 7f2fe019f740 10 client.0 _ll_register_callbacks cb 0x563387a44ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:21:14.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.636+0000 7f2fd7200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563387e180d0 0x563387deee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:14.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.636+0000 7f2fd7200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563387e180d0 0x563387deee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47890/0 (socket says 172.21.15.86:47890) 2024-04-20T21:21:14.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.636+0000 7f2fd7200640 1 -- 172.21.15.86:0/1243791989 learned_addr learned my addr 172.21.15.86:0/1243791989 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:14.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.636+0000 7f2fd5e00640 1 --2- 172.21.15.86:0/1243791989 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563387e14670 0x563387dee8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:14.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.636+0000 7f2fd7200640 1 -- 172.21.15.86:0/1243791989 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563387e14670 msgr2=0x563387dee8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:14.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.636+0000 7f2fd7200640 1 --2- 172.21.15.86:0/1243791989 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563387e14670 0x563387dee8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:14.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.636+0000 7f2fd7200640 1 -- 172.21.15.86:0/1243791989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563387dca6f0 msgr2=0x563387e10e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:14.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.636+0000 7f2fd7200640 1 --2- 172.21.15.86:0/1243791989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563387dca6f0 0x563387e10e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:14.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.636+0000 7f2fd7200640 1 -- 172.21.15.86:0/1243791989 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563387b0f6b0 con 0x563387e180d0 2024-04-20T21:21:14.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.636+0000 7f2fd7200640 1 --2- 172.21.15.86:0/1243791989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563387e180d0 0x563387deee30 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0062d00 tx=0x7f2fd005a790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8a94b5a22823455 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:14.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.636+0000 7f2fb7400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:21:14.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.637+0000 7f2fb7400640 1 -- 172.21.15.86:0/1243791989 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd0062da0 con 0x563387e180d0 2024-04-20T21:21:14.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.637+0000 7f2fb7400640 1 -- 172.21.15.86:0/1243791989 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f2fd0062f40 con 0x563387e180d0 2024-04-20T21:21:14.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.637+0000 7f2fe019f740 1 -- 172.21.15.86:0/1243791989 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fc400bad0 con 0x563387e180d0 2024-04-20T21:21:14.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.637+0000 7f2fb7400640 1 -- 172.21.15.86:0/1243791989 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2fd005fb90 con 0x563387e180d0 2024-04-20T21:21:14.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.637+0000 7f2fe019f740 10 client.6285 fetch_fsmap learned FSMap version 32 2024-04-20T21:21:14.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.637+0000 7f2fe019f740 1 -- 172.21.15.86:0/1243791989 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) -- 0x563387e19150 con 0x563387e180d0 2024-04-20T21:21:14.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.637+0000 7f2fb7400640 1 -- 172.21.15.86:0/1243791989 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fc400bad0 con 0x563387e180d0 2024-04-20T21:21:14.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.637+0000 7f2fb7400640 1 -- 172.21.15.86:0/1243791989 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 3495+0+0 (secure 0 0 0) 0x7f2fd0059aa0 con 0x563387e180d0 2024-04-20T21:21:14.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.637+0000 7f2fe019f740 10 client.6285 fetch_fsmap finished waiting for FSMap version 32 2024-04-20T21:21:14.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.637+0000 7f2fe019f740 10 client.6285 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-20T21:21:14.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.637+0000 7f2fe019f740 20 client.6285 populate_metadata read hostname 'smithi086' 2024-04-20T21:21:14.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.637+0000 7f2fe019f740 1 --2- 172.21.15.86:0/1243791989 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x563387df0b10 0x563387dedf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:14.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.637+0000 7f2fe019f740 4 client.6285 mds_command: new command op to 6135 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:21:14.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.637+0000 7f2fe019f740 1 -- 172.21.15.86:0/1243791989 --> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563387d403f0 con 0x563387df0b10 2024-04-20T21:21:14.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.638+0000 7f2fd6800640 1 --2- 172.21.15.86:0/1243791989 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x563387df0b10 0x563387dedf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:14.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.638+0000 7f2fd6800640 1 --2- 172.21.15.86:0/1243791989 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x563387df0b10 0x563387dedf10 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=171196c5ab3d408d server_cookie=1aeb904ed2f0811a in_seq=0 out_seq=0 2024-04-20T21:21:14.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.638+0000 7f2fb7400640 10 client.6285 ms_handle_connect on v2:172.21.15.86:6835/3008383261 2024-04-20T21:21:14.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.639+0000 7f2fb7400640 1 -- 172.21.15.86:0/1243791989 <== mds.0 v2:172.21.15.86:6835/3008383261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563387d403f0 con 0x563387df0b10 2024-04-20T21:21:14.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.639+0000 7f2fb7400640 10 client.6285 handle_command_reply: tid=0 2024-04-20T21:21:14.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.639+0000 7f2fe019f740 1 client.6285 shutdown 2024-04-20T21:21:14.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.639+0000 7f2fe019f740 1 -- 172.21.15.86:0/1243791989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563387e180d0 msgr2=0x563387deee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:14.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.639+0000 7f2fe019f740 1 --2- 172.21.15.86:0/1243791989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563387e180d0 0x563387deee30 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0062d00 tx=0x7f2fd005a790 comp rx=0 tx=0).stop 2024-04-20T21:21:14.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.639+0000 7f2fe019f740 1 -- 172.21.15.86:0/1243791989 shutdown_connections 2024-04-20T21:21:14.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.639+0000 7f2fe019f740 1 --2- 172.21.15.86:0/1243791989 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x563387df0b10 0x563387dedf10 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:14.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.639+0000 7f2fe019f740 1 --2- 172.21.15.86:0/1243791989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563387e180d0 0x563387deee30 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:14.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.639+0000 7f2fe019f740 1 --2- 172.21.15.86:0/1243791989 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563387e14670 0x563387dee8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:14.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.639+0000 7f2fe019f740 1 --2- 172.21.15.86:0/1243791989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563387dca6f0 0x563387e10e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:14.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.639+0000 7f2fb7400640 0 client.6285 ms_handle_reset on v2:172.21.15.86:6835/3008383261 2024-04-20T21:21:14.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.639+0000 7f2fe019f740 1 -- 172.21.15.86:0/1243791989 >> 172.21.15.86:0/1243791989 conn(0x563387dc6b90 msgr2=0x563387dcc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:14.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.639+0000 7f2fe019f740 1 -- 172.21.15.86:0/1243791989 shutdown_connections 2024-04-20T21:21:14.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.639+0000 7f2fe019f740 1 -- 172.21.15.86:0/1243791989 wait complete. 2024-04-20T21:21:14.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.640+0000 7f2fe019f740 20 client.6285 trim_cache size 0 max 16384 2024-04-20T21:21:14.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.659+0000 7f2fe019f740 1 Processor -- start 2024-04-20T21:21:14.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.659+0000 7f2fe019f740 1 -- start start 2024-04-20T21:21:14.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.660+0000 7f2fe019f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563387dca6f0 0x563387e11780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:14.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.660+0000 7f2fe019f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563387e14670 0x563387e11cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:14.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.660+0000 7f2fe019f740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563387e180d0 0x563387de8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:14.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.660+0000 7f2fe019f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd81ffc40 con 0x563387e14670 2024-04-20T21:21:14.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.660+0000 7f2fe019f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd80c8b10 con 0x563387dca6f0 2024-04-20T21:21:14.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.660+0000 7f2fe019f740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2fd80c9160 con 0x563387e180d0 2024-04-20T21:21:14.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.660+0000 7f2fd5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563387e14670 0x563387e11cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:14.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.660+0000 7f2fd5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563387e14670 0x563387e11cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47892/0 (socket says 172.21.15.86:47892) 2024-04-20T21:21:14.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.660+0000 7f2fd5e00640 1 -- 172.21.15.86:0/1747879073 learned_addr learned my addr 172.21.15.86:0/1747879073 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:14.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.660+0000 7f2fd5e00640 1 -- 172.21.15.86:0/1747879073 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563387e180d0 msgr2=0x563387de8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:14.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.660+0000 7f2fd5e00640 1 --2- 172.21.15.86:0/1747879073 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563387e180d0 0x563387de8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:14.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.660+0000 7f2fd5e00640 1 -- 172.21.15.86:0/1747879073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563387dca6f0 msgr2=0x563387e11780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:14.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.660+0000 7f2fd5e00640 1 --2- 172.21.15.86:0/1747879073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563387dca6f0 0x563387e11780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:14.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.660+0000 7f2fd5e00640 1 -- 172.21.15.86:0/1747879073 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563387dcd2f0 con 0x563387e14670 2024-04-20T21:21:14.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.661+0000 7f2fd5e00640 1 --2- 172.21.15.86:0/1747879073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563387e14670 0x563387e11cc0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4000c00 tx=0x7f2fc4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac2ca01c9b08967 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:14.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.661+0000 7f2fc2600640 1 -- 172.21.15.86:0/1747879073 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc4015040 con 0x563387e14670 2024-04-20T21:21:14.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.661+0000 7f2fc2600640 1 -- 172.21.15.86:0/1747879073 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fc400bc10 con 0x563387e14670 2024-04-20T21:21:14.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.661+0000 7f2fc2600640 1 -- 172.21.15.86:0/1747879073 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc4024430 con 0x563387e14670 2024-04-20T21:21:14.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.661+0000 7f2fe019f740 1 -- 172.21.15.86:0/1747879073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563387e14670 msgr2=0x563387e11cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:14.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.661+0000 7f2fe019f740 1 --2- 172.21.15.86:0/1747879073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563387e14670 0x563387e11cc0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4000c00 tx=0x7f2fc4010000 comp rx=0 tx=0).stop 2024-04-20T21:21:14.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.662+0000 7f2fe019f740 1 -- 172.21.15.86:0/1747879073 shutdown_connections 2024-04-20T21:21:14.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.662+0000 7f2fe019f740 1 --2- 172.21.15.86:0/1747879073 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563387e180d0 0x563387de8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:14.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.662+0000 7f2fe019f740 1 --2- 172.21.15.86:0/1747879073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563387e14670 0x563387e11cc0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:14.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.662+0000 7f2fe019f740 1 --2- 172.21.15.86:0/1747879073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563387dca6f0 0x563387e11780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:14.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.662+0000 7f2fe019f740 1 -- 172.21.15.86:0/1747879073 >> 172.21.15.86:0/1747879073 conn(0x563387dc6b90 msgr2=0x563387dc7070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:14.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.662+0000 7f2fe019f740 1 -- 172.21.15.86:0/1747879073 shutdown_connections 2024-04-20T21:21:14.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.662+0000 7f2fe019f740 1 -- 172.21.15.86:0/1747879073 wait complete. 2024-04-20T21:21:14.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.662+0000 7f2fe019f740 1 -- start start 2024-04-20T21:21:14.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.662+0000 7f2fe019f740 1 Processor -- start 2024-04-20T21:21:14.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.663+0000 7f2fe019f740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563387dca6f0 0x563387de8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:14.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.663+0000 7f2fe019f740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563387e14670 0x563387de8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:14.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.663+0000 7f2fe019f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563387e180d0 0x563387e2b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:14.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.663+0000 7f2fe019f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563387df15b0 con 0x563387e180d0 2024-04-20T21:21:14.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.663+0000 7f2fe019f740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563387e19490 con 0x563387dca6f0 2024-04-20T21:21:14.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.663+0000 7f2fe019f740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x563387df0fe0 con 0x563387e14670 2024-04-20T21:21:14.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.663+0000 7f2fe019f740 10 client.0 _ll_register_callbacks cb 0x7f2fd00568f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:21:14.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.664+0000 7f2fd7200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563387e180d0 0x563387e2b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:14.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.664+0000 7f2fd7200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563387e180d0 0x563387e2b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47894/0 (socket says 172.21.15.86:47894) 2024-04-20T21:21:14.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.664+0000 7f2fd7200640 1 -- 172.21.15.86:0/3818314345 learned_addr learned my addr 172.21.15.86:0/3818314345 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:14.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.664+0000 7f2fd7200640 1 -- 172.21.15.86:0/3818314345 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563387e14670 msgr2=0x563387de8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:14.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.664+0000 7f2fd7200640 1 --2- 172.21.15.86:0/3818314345 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563387e14670 0x563387de8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:14.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.664+0000 7f2fd7200640 1 -- 172.21.15.86:0/3818314345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563387dca6f0 msgr2=0x563387de8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:14.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.664+0000 7f2fd7200640 1 --2- 172.21.15.86:0/3818314345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563387dca6f0 0x563387de8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:14.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.664+0000 7f2fd7200640 1 -- 172.21.15.86:0/3818314345 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563387dcc120 con 0x563387e180d0 2024-04-20T21:21:14.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.664+0000 7f2fd7200640 1 --2- 172.21.15.86:0/3818314345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563387e180d0 0x563387e2b4f0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f2fd005a5e0 tx=0x7f2fd000af70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9d2ec6a85f69844 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:14.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.664+0000 7f2fb7400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:21:14.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.664+0000 7f2fb7400640 1 -- 172.21.15.86:0/3818314345 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd00565a0 con 0x563387e180d0 2024-04-20T21:21:14.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.665+0000 7f2fb7400640 1 -- 172.21.15.86:0/3818314345 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f2fd0059510 con 0x563387e180d0 2024-04-20T21:21:14.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.665+0000 7f2fe019f740 1 -- 172.21.15.86:0/3818314345 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fc400bc10 con 0x563387e180d0 2024-04-20T21:21:14.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.665+0000 7f2fb7400640 1 -- 172.21.15.86:0/3818314345 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2fd0081050 con 0x563387e180d0 2024-04-20T21:21:14.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.665+0000 7f2fe019f740 10 client.6291 fetch_fsmap learned FSMap version 32 2024-04-20T21:21:14.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.665+0000 7f2fe019f740 1 -- 172.21.15.86:0/3818314345 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) -- 0x563387dcb9a0 con 0x563387e180d0 2024-04-20T21:21:14.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.665+0000 7f2fb7400640 1 -- 172.21.15.86:0/3818314345 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fc400bc10 con 0x563387e180d0 2024-04-20T21:21:14.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.665+0000 7f2fb7400640 1 -- 172.21.15.86:0/3818314345 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 3495+0+0 (secure 0 0 0) 0x7f2fd0084020 con 0x563387e180d0 2024-04-20T21:21:14.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.665+0000 7f2fe019f740 10 client.6291 fetch_fsmap finished waiting for FSMap version 32 2024-04-20T21:21:14.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.665+0000 7f2fe019f740 10 client.6291 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-20T21:21:14.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.665+0000 7f2fe019f740 20 client.6291 populate_metadata read hostname 'smithi086' 2024-04-20T21:21:14.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.665+0000 7f2fe019f740 1 --2- 172.21.15.86:0/3818314345 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x563387e0f3e0 0x563387e0f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:14.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.665+0000 7f2fe019f740 4 client.6291 mds_command: new command op to 6135 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T21:21:14.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.665+0000 7f2fe019f740 1 -- 172.21.15.86:0/3818314345 --> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] -- command(tid 0: {"prefix": "status"}) -- 0x563387e17c40 con 0x563387e0f3e0 2024-04-20T21:21:14.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.665+0000 7f2fd6800640 1 --2- 172.21.15.86:0/3818314345 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x563387e0f3e0 0x563387e0f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:14.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.666+0000 7f2fd6800640 1 --2- 172.21.15.86:0/3818314345 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x563387e0f3e0 0x563387e0f7c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e889f79cd19a6966 server_cookie=63ce8abe8ab54530 in_seq=0 out_seq=0 2024-04-20T21:21:14.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.666+0000 7f2fb7400640 10 client.6291 ms_handle_connect on v2:172.21.15.86:6835/3008383261 2024-04-20T21:21:14.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.666+0000 7f2fb7400640 1 -- 172.21.15.86:0/3818314345 <== mds.0 v2:172.21.15.86:6835/3008383261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x563387e17c40 con 0x563387e0f3e0 2024-04-20T21:21:14.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.666+0000 7f2fb7400640 10 client.6291 handle_command_reply: tid=0 2024-04-20T21:21:14.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.666+0000 7f2fe019f740 1 client.6291 shutdown 2024-04-20T21:21:14.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.666+0000 7f2fe019f740 1 -- 172.21.15.86:0/3818314345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563387e180d0 msgr2=0x563387e2b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:14.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.666+0000 7f2fe019f740 1 --2- 172.21.15.86:0/3818314345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563387e180d0 0x563387e2b4f0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f2fd005a5e0 tx=0x7f2fd000af70 comp rx=0 tx=0).stop 2024-04-20T21:21:14.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.666+0000 7f2fe019f740 1 -- 172.21.15.86:0/3818314345 shutdown_connections 2024-04-20T21:21:14.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.666+0000 7f2fe019f740 1 --2- 172.21.15.86:0/3818314345 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x563387e0f3e0 0x563387e0f7c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:14.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.666+0000 7f2fe019f740 1 --2- 172.21.15.86:0/3818314345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563387e180d0 0x563387e2b4f0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:14.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.666+0000 7f2fe019f740 1 --2- 172.21.15.86:0/3818314345 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563387e14670 0x563387de8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:14.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.666+0000 7f2fe019f740 1 --2- 172.21.15.86:0/3818314345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563387dca6f0 0x563387de8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:14.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.666+0000 7f2fb7400640 0 client.6291 ms_handle_reset on v2:172.21.15.86:6835/3008383261 2024-04-20T21:21:14.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.666+0000 7f2fe019f740 1 -- 172.21.15.86:0/3818314345 >> 172.21.15.86:0/3818314345 conn(0x563387dc6b90 msgr2=0x563387e10c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:14.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.667+0000 7f2fe019f740 1 -- 172.21.15.86:0/3818314345 shutdown_connections 2024-04-20T21:21:14.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.667+0000 7f2fe019f740 1 -- 172.21.15.86:0/3818314345 wait complete. 2024-04-20T21:21:14.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.667+0000 7f2fe019f740 20 client.6291 trim_cache size 0 max 16384 2024-04-20T21:21:14.678 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:21:14.678 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "9c2dec1d-d7ae-4a83-a812-79922808e278", 2024-04-20T21:21:14.678 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-04-20T21:21:14.678 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 6135, 2024-04-20T21:21:14.678 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-20T21:21:14.679 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-20T21:21:14.679 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-20T21:21:14.679 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 6.5904676589999998, 2024-04-20T21:21:14.679 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 32, 2024-04-20T21:21:14.679 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 59, 2024-04-20T21:21:14.679 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 59, 2024-04-20T21:21:14.679 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 13.093618948 2024-04-20T21:21:14.679 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:21:14.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.667+0000 7f2fdca00640 1 -- 172.21.15.86:0/3977139906 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2fb806c8f0 msgr2=0x7f2fb806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:14.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.667+0000 7f2fdca00640 1 --2- 172.21.15.86:0/3977139906 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2fb806c8f0 0x7f2fb806edb0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8036000 tx=0x7f2fc8008030 comp rx=0 tx=0).stop 2024-04-20T21:21:14.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.667+0000 7f2fdca00640 1 -- 172.21.15.86:0/3977139906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fd815af10 msgr2=0x7f2fd80c54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:14.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.667+0000 7f2fdca00640 1 --2- 172.21.15.86:0/3977139906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fd815af10 0x7f2fd80c54a0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f2fd000cd50 tx=0x7f2fd000a650 comp rx=0 tx=0).stop 2024-04-20T21:21:14.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.667+0000 7f2fdca00640 1 -- 172.21.15.86:0/3977139906 shutdown_connections 2024-04-20T21:21:14.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.667+0000 7f2fdca00640 1 --2- 172.21.15.86:0/3977139906 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2fb806c8f0 0x7f2fb806edb0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:14.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.667+0000 7f2fdca00640 1 --2- 172.21.15.86:0/3977139906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fd815af10 0x7f2fd80c54a0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:14.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.667+0000 7f2fdca00640 1 --2- 172.21.15.86:0/3977139906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2fd80cfa00 0x7f2fd815e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:14.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.667+0000 7f2fdca00640 1 --2- 172.21.15.86:0/3977139906 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2fd8073540 0x7f2fd815c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:14.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.667+0000 7f2fdca00640 1 -- 172.21.15.86:0/3977139906 >> 172.21.15.86:0/3977139906 conn(0x7f2fd806bb60 msgr2=0x7f2fd80cb0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:14.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.668+0000 7f2fdca00640 1 -- 172.21.15.86:0/3977139906 shutdown_connections 2024-04-20T21:21:14.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.668+0000 7f2fdca00640 1 -- 172.21.15.86:0/3977139906 wait complete. 2024-04-20T21:21:14.683 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T21:21:14.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.866+0000 7f5e8a800640 1 Processor -- start 2024-04-20T21:21:14.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.866+0000 7f5e8a800640 1 -- start start 2024-04-20T21:21:14.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.868+0000 7f5e8a800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5e84146130 0x7f5e8406e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:14.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.868+0000 7f5e8a800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5e840ccdd0 0x7f5e840cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:14.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.868+0000 7f5e8a800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e84071d00 0x7f5e84072190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:14.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.868+0000 7f5e8a800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f5e8406dc30 con 0x7f5e840ccdd0 2024-04-20T21:21:14.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.868+0000 7f5e8a800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f5e840cd8e0 con 0x7f5e84146130 2024-04-20T21:21:14.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.868+0000 7f5e8a800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5e840cdbc0 con 0x7f5e84071d00 2024-04-20T21:21:14.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.868+0000 7f5e7be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5e840ccdd0 0x7f5e840cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:14.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.869+0000 7f5e7be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5e840ccdd0 0x7f5e840cd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47898/0 (socket says 172.21.15.86:47898) 2024-04-20T21:21:14.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.869+0000 7f5e7be00640 1 -- 172.21.15.86:0/2461880389 learned_addr learned my addr 172.21.15.86:0/2461880389 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:14.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.869+0000 7f5e83e00640 1 --2- 172.21.15.86:0/2461880389 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e84071d00 0x7f5e84072190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:14.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.869+0000 7f5e83400640 1 --2- 172.21.15.86:0/2461880389 >> v2:172.21.15.144:3300/0 conn(0x7f5e84146130 0x7f5e8406e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:14.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.869+0000 7f5e7be00640 1 -- 172.21.15.86:0/2461880389 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e84071d00 msgr2=0x7f5e84072190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:14.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.869+0000 7f5e7be00640 1 --2- 172.21.15.86:0/2461880389 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e84071d00 0x7f5e84072190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:14.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.869+0000 7f5e7be00640 1 -- 172.21.15.86:0/2461880389 >> v2:172.21.15.144:3300/0 conn(0x7f5e84146130 msgr2=0x7f5e8406e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:14.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.869+0000 7f5e7be00640 1 --2- 172.21.15.86:0/2461880389 >> v2:172.21.15.144:3300/0 conn(0x7f5e84146130 0x7f5e8406e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:14.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.869+0000 7f5e7be00640 1 -- 172.21.15.86:0/2461880389 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e8406ded0 con 0x7f5e840ccdd0 2024-04-20T21:21:14.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.870+0000 7f5e7be00640 1 --2- 172.21.15.86:0/2461880389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e840ccdd0 0x7f5e840cd230 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f5e840b5950 tx=0x7f5e6c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96336a8c3a6517ba server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:14.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.870+0000 7f5e82a00640 1 -- 172.21.15.86:0/2461880389 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c034070 con 0x7f5e840ccdd0 2024-04-20T21:21:14.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.871+0000 7f5e82a00640 1 -- 172.21.15.86:0/2461880389 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e6c030860 con 0x7f5e840ccdd0 2024-04-20T21:21:14.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.871+0000 7f5e82a00640 1 -- 172.21.15.86:0/2461880389 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c030c30 con 0x7f5e840ccdd0 2024-04-20T21:21:14.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.871+0000 7f5e8a800640 1 -- 172.21.15.86:0/2461880389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e840ccdd0 msgr2=0x7f5e840cd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:14.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.871+0000 7f5e8a800640 1 --2- 172.21.15.86:0/2461880389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e840ccdd0 0x7f5e840cd230 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f5e840b5950 tx=0x7f5e6c02b9a0 comp rx=0 tx=0).stop 2024-04-20T21:21:14.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.871+0000 7f5e8a800640 1 -- 172.21.15.86:0/2461880389 shutdown_connections 2024-04-20T21:21:14.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.871+0000 7f5e8a800640 1 --2- 172.21.15.86:0/2461880389 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e84071d00 0x7f5e84072190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:14.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.871+0000 7f5e8a800640 1 --2- 172.21.15.86:0/2461880389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e840ccdd0 0x7f5e840cd230 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:14.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.871+0000 7f5e8a800640 1 --2- 172.21.15.86:0/2461880389 >> v2:172.21.15.144:3300/0 conn(0x7f5e84146130 0x7f5e8406e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:14.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.871+0000 7f5e8a800640 1 -- 172.21.15.86:0/2461880389 >> 172.21.15.86:0/2461880389 conn(0x7f5e840b2360 msgr2=0x7f5e840b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:14.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.871+0000 7f5e8a800640 1 -- 172.21.15.86:0/2461880389 shutdown_connections 2024-04-20T21:21:14.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.871+0000 7f5e8a800640 1 -- 172.21.15.86:0/2461880389 wait complete. 2024-04-20T21:21:14.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.873+0000 7f5e8a800640 1 Processor -- start 2024-04-20T21:21:14.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.873+0000 7f5e8a800640 1 -- start start 2024-04-20T21:21:14.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.874+0000 7f5e8a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e84071d00 0x7f5e841f5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:14.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.874+0000 7f5e8a800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e840ccdd0 0x7f5e841e78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:14.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.874+0000 7f5e8a800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e84146130 0x7f5e841e7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:14.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.874+0000 7f5e8a800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8415cc20 con 0x7f5e84071d00 2024-04-20T21:21:14.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.874+0000 7f5e8a800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8406f150 con 0x7f5e84146130 2024-04-20T21:21:14.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.874+0000 7f5e8a800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5e8415cda0 con 0x7f5e840ccdd0 2024-04-20T21:21:14.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.875+0000 7f5e83400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e84071d00 0x7f5e841f5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:14.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.875+0000 7f5e83400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e84071d00 0x7f5e841f5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47908/0 (socket says 172.21.15.86:47908) 2024-04-20T21:21:14.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.875+0000 7f5e83400640 1 -- 172.21.15.86:0/529873302 learned_addr learned my addr 172.21.15.86:0/529873302 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:14.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.875+0000 7f5e83e00640 1 --2- 172.21.15.86:0/529873302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e84146130 0x7f5e841e7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:14.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.875+0000 7f5e7be00640 1 --2- 172.21.15.86:0/529873302 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e840ccdd0 0x7f5e841e78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:14.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.875+0000 7f5e83400640 1 -- 172.21.15.86:0/529873302 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e840ccdd0 msgr2=0x7f5e841e78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:14.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.875+0000 7f5e83400640 1 --2- 172.21.15.86:0/529873302 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e840ccdd0 0x7f5e841e78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:14.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.875+0000 7f5e83400640 1 -- 172.21.15.86:0/529873302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e84146130 msgr2=0x7f5e841e7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:14.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.875+0000 7f5e83400640 1 --2- 172.21.15.86:0/529873302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e84146130 0x7f5e841e7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:14.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.875+0000 7f5e83400640 1 -- 172.21.15.86:0/529873302 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e840af0b0 con 0x7f5e84071d00 2024-04-20T21:21:14.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.875+0000 7f5e83400640 1 --2- 172.21.15.86:0/529873302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e84071d00 0x7f5e841f5ac0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f5e7400d5f0 tx=0x7f5e7400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:14.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.876+0000 7f5e7b400640 1 -- 172.21.15.86:0/529873302 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e740125f0 con 0x7f5e84071d00 2024-04-20T21:21:14.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.876+0000 7f5e7b400640 1 -- 172.21.15.86:0/529873302 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e74019070 con 0x7f5e84071d00 2024-04-20T21:21:14.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.876+0000 7f5e7b400640 1 -- 172.21.15.86:0/529873302 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e740129d0 con 0x7f5e84071d00 2024-04-20T21:21:14.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.876+0000 7f5e8a800640 1 -- 172.21.15.86:0/529873302 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e841e58c0 con 0x7f5e84071d00 2024-04-20T21:21:14.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.876+0000 7f5e8a800640 1 -- 172.21.15.86:0/529873302 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e841e5e80 con 0x7f5e84071d00 2024-04-20T21:21:14.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.877+0000 7f5e8a800640 1 -- 172.21.15.86:0/529873302 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e840689e0 con 0x7f5e84071d00 2024-04-20T21:21:14.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.878+0000 7f5e7b400640 1 -- 172.21.15.86:0/529873302 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5e74012b70 con 0x7f5e84071d00 2024-04-20T21:21:14.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.879+0000 7f5e7b400640 1 --2- 172.21.15.86:0/529873302 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5e6006c8f0 0x7f5e6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:14.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.879+0000 7f5e7b400640 1 -- 172.21.15.86:0/529873302 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f5e7408bfe0 con 0x7f5e84071d00 2024-04-20T21:21:14.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.879+0000 7f5e7be00640 1 --2- 172.21.15.86:0/529873302 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5e6006c8f0 0x7f5e6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:14.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.880+0000 7f5e7be00640 1 --2- 172.21.15.86:0/529873302 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5e6006c8f0 0x7f5e6006edb0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c036000 tx=0x7f5e6c03b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:14.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:14.883+0000 7f5e7b400640 1 -- 172.21.15.86:0/529873302 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5e7405a390 con 0x7f5e84071d00 2024-04-20T21:21:15.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:15.128+0000 7f5e8a800640 1 -- 172.21.15.86:0/529873302 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5e8415ff50 con 0x7f5e84071d00 2024-04-20T21:21:15.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:15.132+0000 7f5e7b400640 1 -- 172.21.15.86:0/529873302 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v32) ==== 53+0+73 (secure 0 0 0) 0x7f5e74059c90 con 0x7f5e84071d00 2024-04-20T21:21:15.131 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T21:21:15.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:15.135+0000 7f5e8a800640 1 -- 172.21.15.86:0/529873302 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5e6006c8f0 msgr2=0x7f5e6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:15.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:15.135+0000 7f5e8a800640 1 --2- 172.21.15.86:0/529873302 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5e6006c8f0 0x7f5e6006edb0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c036000 tx=0x7f5e6c03b040 comp rx=0 tx=0).stop 2024-04-20T21:21:15.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:15.135+0000 7f5e8a800640 1 -- 172.21.15.86:0/529873302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e84071d00 msgr2=0x7f5e841f5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:15.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:15.135+0000 7f5e8a800640 1 --2- 172.21.15.86:0/529873302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e84071d00 0x7f5e841f5ac0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f5e7400d5f0 tx=0x7f5e7400dab0 comp rx=0 tx=0).stop 2024-04-20T21:21:15.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:15.135+0000 7f5e8a800640 1 -- 172.21.15.86:0/529873302 shutdown_connections 2024-04-20T21:21:15.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:15.135+0000 7f5e8a800640 1 --2- 172.21.15.86:0/529873302 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5e6006c8f0 0x7f5e6006edb0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:15.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:15.135+0000 7f5e8a800640 1 --2- 172.21.15.86:0/529873302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e84146130 0x7f5e841e7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:15.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:15.135+0000 7f5e8a800640 1 --2- 172.21.15.86:0/529873302 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e840ccdd0 0x7f5e841e78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:15.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:15.135+0000 7f5e8a800640 1 --2- 172.21.15.86:0/529873302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e84071d00 0x7f5e841f5ac0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:15.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:15.135+0000 7f5e8a800640 1 -- 172.21.15.86:0/529873302 >> 172.21.15.86:0/529873302 conn(0x7f5e840b2360 msgr2=0x7f5e840b3730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:15.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:15.135+0000 7f5e8a800640 1 -- 172.21.15.86:0/529873302 shutdown_connections 2024-04-20T21:21:15.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:15.135+0000 7f5e8a800640 1 -- 172.21.15.86:0/529873302 wait complete. 2024-04-20T21:21:15.153 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T21:21:15.153 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T21:21:15.154 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-04-20T21:21:15.154 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-20T21:21:15.154 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T21:21:15.154 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T21:21:15.154 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T21:21:15.154 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T21:21:15.154 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:21:15.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:21:15.154 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-04-20T21:21:15.203 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T21:21:15.203 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T21:21:15.203 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T21:21:15.203 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:21:15.203 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-20T21:21:15.203 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:21:15.204 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:21:15.204 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:21:15.204 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-04-20T21:21:15.204 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:21:15.204 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:21:15.204 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-04-20T21:21:15.204 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:21:15.204 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:21:15.204 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T21:21:15.204 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:21:15.204 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T21:21:15.204 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42481sec preferred_lft 42481sec 2024-04-20T21:21:15.205 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-04-20T21:21:15.205 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:21:15.205 INFO:teuthology.orchestra.run.smithi086.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T21:21:15.205 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 86:25:97:2c:9c:bb brd ff:ff:ff:ff:ff:ff 2024-04-20T21:21:15.205 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-20T21:21:15.205 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:21:15.205 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::a4fb:30ff:fe90:2d6a/64 scope link 2024-04-20T21:21:15.205 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:21:15.205 INFO:teuthology.orchestra.run.smithi086.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T21:21:15.205 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 86:25:97:2c:9c:bb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:21:15.205 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::8425:97ff:fe2c:9cbb/64 scope link 2024-04-20T21:21:15.205 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:21:15.205 INFO:teuthology.orchestra.run.smithi086.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T21:21:15.205 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c2:9c:8b:55:ca:84 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:21:15.206 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c09c:8bff:fe55:ca84/64 scope link 2024-04-20T21:21:15.206 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:21:15.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:21:15.206 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T21:21:15.284 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-20T21:21:15.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:21:15.288 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-04-20T21:21:15.339 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-04-20T21:21:15.339 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T21:21:15.339 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T21:21:15.340 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T21:21:15.340 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T21:21:15.341 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:21:15.341 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-20T21:21:15.341 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T21:21:15.341 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T21:21:15.341 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T21:21:15.341 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-20T21:21:15.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:21:15.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:21:15.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:21:15.552 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-20T21:21:15.595 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-20T21:21:15.596 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-20T21:21:15.596 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:21:15.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:21:15.597 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-04-20T21:21:15.720 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-20T21:21:15.721 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T21:21:15.721 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-20T21:21:15.772 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:21:15.773 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T21:21:15.773 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-20T21:21:15.825 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-04-20T21:21:15.902 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T21:21:15.954 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-04-20T21:21:15.955 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-04-20T21:21:15.955 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-04-20T21:21:15.955 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-04-20T21:21:15.955 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-04-20T21:21:15.955 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-04-20T21:21:15.955 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-04-20T21:21:15.955 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-04-20T21:21:15.955 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-04-20T21:21:15.955 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-04-20T21:21:15.955 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-04-20T21:21:15.955 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/selinux 2024-04-20T21:21:15.955 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T21:21:15.955 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-04-20T21:21:15.955 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-04-20T21:21:15.956 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-04-20T21:21:15.956 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-04-20T21:21:15.956 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-04-20T21:21:15.956 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-04-20T21:21:15.956 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T21:21:15.956 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T21:21:15.956 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T21:21:15.956 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T21:21:15.956 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-04-20T21:21:15.956 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-04-20T21:21:15.956 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T21:21:15.956 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T21:21:15.956 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T21:21:15.956 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T21:21:15.956 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-04-20T21:21:15.957 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:21:15.957 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:21:15.957 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:21:15.957 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:21:15.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:21:15.958 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-04-20T21:21:16.004 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T21:21:16.004 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-20T21:21:16.011 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-04-20T21:21:16.080 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T21:21:16.093 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:21:16.125 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-04-20T21:21:16.125 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-04-20T21:21:16.125 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-04-20T21:21:16.126 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-04-20T21:21:16.126 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-04-20T21:21:16.126 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-04-20T21:21:16.126 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-04-20T21:21:16.126 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-04-20T21:21:16.126 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-04-20T21:21:16.126 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-04-20T21:21:16.126 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-04-20T21:21:16.126 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/selinux 2024-04-20T21:21:16.126 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T21:21:16.126 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-04-20T21:21:16.126 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-04-20T21:21:16.126 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-04-20T21:21:16.127 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-04-20T21:21:16.127 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-04-20T21:21:16.127 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-04-20T21:21:16.127 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T21:21:16.127 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T21:21:16.127 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T21:21:16.127 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T21:21:16.127 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-04-20T21:21:16.127 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-04-20T21:21:16.127 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T21:21:16.128 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T21:21:16.128 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T21:21:16.128 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T21:21:16.128 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-04-20T21:21:16.128 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:21:16.128 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:21:16.128 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:21:16.128 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:21:16.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:21:16.129 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-04-20T21:21:16.132 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:21:16.182 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:2024-04-20T21:21:16.183+0000 7f18f244b480 -1 init, newargv = 0x557d72c47020 newargc=15 2024-04-20T21:21:16.182 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[43494]: starting ceph client 2024-04-20T21:21:16.191 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[43494]: starting fuse 2024-04-20T21:21:17.273 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-04-20T21:21:17.363 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T21:21:17.418 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-04-20T21:21:17.418 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-04-20T21:21:17.418 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-04-20T21:21:17.418 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-04-20T21:21:17.418 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-04-20T21:21:17.418 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-04-20T21:21:17.418 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-04-20T21:21:17.418 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-04-20T21:21:17.418 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-04-20T21:21:17.418 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-04-20T21:21:17.419 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-04-20T21:21:17.419 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/selinux 2024-04-20T21:21:17.419 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T21:21:17.419 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-04-20T21:21:17.419 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-04-20T21:21:17.419 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-04-20T21:21:17.419 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-04-20T21:21:17.419 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-04-20T21:21:17.419 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-04-20T21:21:17.419 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T21:21:17.419 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T21:21:17.419 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T21:21:17.419 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T21:21:17.420 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-04-20T21:21:17.420 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-04-20T21:21:17.420 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T21:21:17.420 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T21:21:17.420 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T21:21:17.420 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T21:21:17.420 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-04-20T21:21:17.420 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:21:17.420 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:21:17.420 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:21:17.420 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:21:17.420 INFO:teuthology.orchestra.run.smithi086.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-20T21:21:17.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:21:17.421 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-04-20T21:21:17.474 INFO:teuthology.orchestra.run.smithi086.stdout:41 2024-04-20T21:21:17.475 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-04-20T21:21:17.475 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer -- python3 -c ' 2024-04-20T21:21:17.475 DEBUG:teuthology.orchestra.run.smithi086:> import glob 2024-04-20T21:21:17.475 DEBUG:teuthology.orchestra.run.smithi086:> import re 2024-04-20T21:21:17.475 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:21:17.475 DEBUG:teuthology.orchestra.run.smithi086:> import subprocess 2024-04-20T21:21:17.475 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:21:17.475 DEBUG:teuthology.orchestra.run.smithi086:> def _find_admin_socket(client_name): 2024-04-20T21:21:17.475 DEBUG:teuthology.orchestra.run.smithi086:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-20T21:21:17.475 DEBUG:teuthology.orchestra.run.smithi086:> files = glob.glob(asok_path) 2024-04-20T21:21:17.476 DEBUG:teuthology.orchestra.run.smithi086:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-20T21:21:17.476 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:21:17.476 DEBUG:teuthology.orchestra.run.smithi086:> # Given a non-glob path, it better be there 2024-04-20T21:21:17.476 DEBUG:teuthology.orchestra.run.smithi086:> if "*" not in asok_path: 2024-04-20T21:21:17.476 DEBUG:teuthology.orchestra.run.smithi086:> assert(len(files) == 1) 2024-04-20T21:21:17.476 DEBUG:teuthology.orchestra.run.smithi086:> return files[0] 2024-04-20T21:21:17.476 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:21:17.476 DEBUG:teuthology.orchestra.run.smithi086:> for f in files: 2024-04-20T21:21:17.476 DEBUG:teuthology.orchestra.run.smithi086:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T21:21:17.476 DEBUG:teuthology.orchestra.run.smithi086:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T21:21:17.476 DEBUG:teuthology.orchestra.run.smithi086:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T21:21:17.476 DEBUG:teuthology.orchestra.run.smithi086:> contents = proc_f.read() 2024-04-20T21:21:17.476 DEBUG:teuthology.orchestra.run.smithi086:> if mountpoint in contents: 2024-04-20T21:21:17.476 DEBUG:teuthology.orchestra.run.smithi086:> return f 2024-04-20T21:21:17.477 DEBUG:teuthology.orchestra.run.smithi086:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T21:21:17.477 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:21:17.477 DEBUG:teuthology.orchestra.run.smithi086:> print(_find_admin_socket("client.0")) 2024-04-20T21:21:17.477 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:21:17.643 INFO:teuthology.orchestra.run.smithi086.stdout:/var/run/ceph/ceph-client.0.43494.asok 2024-04-20T21:21:17.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:21:17.662 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.43494.asok 2024-04-20T21:21:17.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:21:17.662 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.43494.asok status 2024-04-20T21:21:17.889 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:21:17.890 INFO:teuthology.orchestra.run.smithi086.stdout: "metadata": { 2024-04-20T21:21:17.890 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T21:21:17.890 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T21:21:17.890 INFO:teuthology.orchestra.run.smithi086.stdout: "entity_id": "0", 2024-04-20T21:21:17.890 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": "smithi086", 2024-04-20T21:21:17.890 INFO:teuthology.orchestra.run.smithi086.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-20T21:21:17.890 INFO:teuthology.orchestra.run.smithi086.stdout: "pid": "43494", 2024-04-20T21:21:17.890 INFO:teuthology.orchestra.run.smithi086.stdout: "root": "/" 2024-04-20T21:21:17.890 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:21:17.890 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_count": 0, 2024-04-20T21:21:17.890 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_pinned_count": 0, 2024-04-20T21:21:17.890 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 6306, 2024-04-20T21:21:17.890 INFO:teuthology.orchestra.run.smithi086.stdout: "inst": { 2024-04-20T21:21:17.891 INFO:teuthology.orchestra.run.smithi086.stdout: "name": { 2024-04-20T21:21:17.891 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "client", 2024-04-20T21:21:17.891 INFO:teuthology.orchestra.run.smithi086.stdout: "num": 6306 2024-04-20T21:21:17.891 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:21:17.891 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2024-04-20T21:21:17.891 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-04-20T21:21:17.891 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2024-04-20T21:21:17.891 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 1725633253 2024-04-20T21:21:17.891 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:21:17.891 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:21:17.891 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2024-04-20T21:21:17.891 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-04-20T21:21:17.891 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2024-04-20T21:21:17.891 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 1725633253 2024-04-20T21:21:17.892 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:21:17.892 INFO:teuthology.orchestra.run.smithi086.stdout: "inst_str": "client.6306 192.168.0.1:0/1725633253", 2024-04-20T21:21:17.892 INFO:teuthology.orchestra.run.smithi086.stdout: "addr_str": "192.168.0.1:0/1725633253", 2024-04-20T21:21:17.892 INFO:teuthology.orchestra.run.smithi086.stdout: "inode_count": 1, 2024-04-20T21:21:17.892 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_epoch": 32, 2024-04-20T21:21:17.892 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch": 59, 2024-04-20T21:21:17.892 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch_barrier": 0, 2024-04-20T21:21:17.892 INFO:teuthology.orchestra.run.smithi086.stdout: "blocklisted": false, 2024-04-20T21:21:17.892 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs" 2024-04-20T21:21:17.892 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:21:17.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:21:17.905 DEBUG:teuthology.orchestra.run.smithi086:> stat --file-system '--printf=%T 2024-04-20T21:21:17.905 DEBUG:teuthology.orchestra.run.smithi086:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-20T21:21:17.957 INFO:teuthology.orchestra.run.smithi086.stdout:fuseblk 2024-04-20T21:21:17.958 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-20T21:21:17.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:21:17.958 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-20T21:21:18.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:21:18.043 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir_test) 2024-04-20T21:21:18.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:21:18.111 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir_test/file) 2024-04-20T21:21:18.167 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-04-20T21:21:18.168 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:21:18.168 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-04-20T21:21:18.168 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:21:18.168 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir_test/file").st_ino) 2024-04-20T21:21:18.168 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:21:18.301 INFO:teuthology.orchestra.run.smithi086.stdout:1099511627777 2024-04-20T21:21:18.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:21:18.316 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:21:18.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.564+0000 7f527ae00640 1 Processor -- start 2024-04-20T21:21:18.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.564+0000 7f527ae00640 1 -- start start 2024-04-20T21:21:18.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.565+0000 7f527ae00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5274146130 0x7f527406e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:18.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.565+0000 7f527ae00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5274072410 0x7f52740cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:18.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.566+0000 7f527ae00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f527406dc30 con 0x7f52740ccdd0 2024-04-20T21:21:18.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.566+0000 7f527ae00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f52740cd8e0 con 0x7f5274072410 2024-04-20T21:21:18.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.566+0000 7f527ae00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f52740cdbc0 con 0x7f5274146130 2024-04-20T21:21:18.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.566+0000 7f5278e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5274072410 0x7f52740cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:18.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.566+0000 7f5278e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5274072410 0x7f52740cd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58636/0 (socket says 172.21.15.86:58636) 2024-04-20T21:21:18.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.566+0000 7f5278e00640 1 -- 172.21.15.86:0/3718978409 learned_addr learned my addr 172.21.15.86:0/3718978409 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:18.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.566+0000 7f526b400640 1 --1- 172.21.15.86:0/3718978409 >> v1:172.21.15.86:6789/0 conn(0x7f52740ccdd0 0x7f5274071d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42444/0 (socket says 172.21.15.86:42444) 2024-04-20T21:21:18.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.566+0000 7f5273e00640 1 --2- 172.21.15.86:0/3718978409 >> v2:172.21.15.144:3300/0 conn(0x7f5274146130 0x7f527406e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:18.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.566+0000 7f5278e00640 1 -- 172.21.15.86:0/3718978409 >> v2:172.21.15.144:3300/0 conn(0x7f5274146130 msgr2=0x7f527406e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:18.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.566+0000 7f5278e00640 1 --2- 172.21.15.86:0/3718978409 >> v2:172.21.15.144:3300/0 conn(0x7f5274146130 0x7f527406e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:18.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.566+0000 7f5278e00640 1 -- 172.21.15.86:0/3718978409 >> v1:172.21.15.86:6789/0 conn(0x7f52740ccdd0 legacy=0x7f5274071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:18.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.566+0000 7f5278e00640 1 -- 172.21.15.86:0/3718978409 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f527406ded0 con 0x7f5274072410 2024-04-20T21:21:18.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.566+0000 7f5273400640 1 -- 172.21.15.86:0/3718978409 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1201829605 0 0) 0x7f527406dc30 con 0x7f52740ccdd0 2024-04-20T21:21:18.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.566+0000 7f5273e00640 1 --2- 172.21.15.86:0/3718978409 >> v2:172.21.15.144:3300/0 conn(0x7f5274146130 0x7f527406e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:21:18.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.568+0000 7f5278e00640 1 --2- 172.21.15.86:0/3718978409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5274072410 0x7f52740cd230 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f52740b5950 tx=0x7f526c02c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e7ef9418a429d14 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:18.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.568+0000 7f5273400640 1 -- 172.21.15.86:0/3718978409 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f526c02cbb0 con 0x7f5274072410 2024-04-20T21:21:18.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.568+0000 7f5273400640 1 -- 172.21.15.86:0/3718978409 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f526c02cd50 con 0x7f5274072410 2024-04-20T21:21:18.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.568+0000 7f5273400640 1 -- 172.21.15.86:0/3718978409 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f526c034610 con 0x7f5274072410 2024-04-20T21:21:18.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.569+0000 7f527ae00640 1 -- 172.21.15.86:0/3718978409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5274072410 msgr2=0x7f52740cd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:18.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.569+0000 7f527ae00640 1 --2- 172.21.15.86:0/3718978409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5274072410 0x7f52740cd230 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f52740b5950 tx=0x7f526c02c010 comp rx=0 tx=0).stop 2024-04-20T21:21:18.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.569+0000 7f527ae00640 1 -- 172.21.15.86:0/3718978409 shutdown_connections 2024-04-20T21:21:18.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.569+0000 7f527ae00640 1 --2- 172.21.15.86:0/3718978409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5274072410 0x7f52740cd230 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:18.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.569+0000 7f527ae00640 1 --2- 172.21.15.86:0/3718978409 >> v2:172.21.15.144:3300/0 conn(0x7f5274146130 0x7f527406e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:18.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.569+0000 7f527ae00640 1 -- 172.21.15.86:0/3718978409 >> 172.21.15.86:0/3718978409 conn(0x7f52740b2360 msgr2=0x7f52740b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:18.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.569+0000 7f527ae00640 1 -- 172.21.15.86:0/3718978409 shutdown_connections 2024-04-20T21:21:18.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.569+0000 7f527ae00640 1 -- 172.21.15.86:0/3718978409 wait complete. 2024-04-20T21:21:18.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.570+0000 7f527ae00640 1 Processor -- start 2024-04-20T21:21:18.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.571+0000 7f527ae00640 1 -- start start 2024-04-20T21:21:18.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.571+0000 7f527ae00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5274072410 0x7f52741e5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:18.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.571+0000 7f527ae00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52740ccdd0 0x7f52741f7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:18.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.572+0000 7f527ae00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5274146130 0x7f52741f41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:18.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.572+0000 7f527ae00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f527415cc20 con 0x7f52740ccdd0 2024-04-20T21:21:18.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.572+0000 7f527ae00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f527406f150 con 0x7f5274072410 2024-04-20T21:21:18.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.572+0000 7f527ae00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f527415cda0 con 0x7f5274146130 2024-04-20T21:21:18.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.572+0000 7f526b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52740ccdd0 0x7f52741f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:18.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.572+0000 7f526b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52740ccdd0 0x7f52741f7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58638/0 (socket says 172.21.15.86:58638) 2024-04-20T21:21:18.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.572+0000 7f526b400640 1 -- 172.21.15.86:0/932348427 learned_addr learned my addr 172.21.15.86:0/932348427 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:18.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.572+0000 7f5273e00640 1 --2- 172.21.15.86:0/932348427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5274072410 0x7f52741e5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:18.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.572+0000 7f526b400640 1 -- 172.21.15.86:0/932348427 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5274146130 msgr2=0x7f52741f41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:18.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.572+0000 7f526b400640 1 --2- 172.21.15.86:0/932348427 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5274146130 0x7f52741f41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:18.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.572+0000 7f526b400640 1 -- 172.21.15.86:0/932348427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5274072410 msgr2=0x7f52741e5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:18.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.572+0000 7f526b400640 1 --2- 172.21.15.86:0/932348427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5274072410 0x7f52741e5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:18.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.572+0000 7f526b400640 1 -- 172.21.15.86:0/932348427 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52741f46f0 con 0x7f52740ccdd0 2024-04-20T21:21:18.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.572+0000 7f5273e00640 1 --2- 172.21.15.86:0/932348427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5274072410 0x7f52741e5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:21:18.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.573+0000 7f526b400640 1 --2- 172.21.15.86:0/932348427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52740ccdd0 0x7f52741f7150 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f525c00c4e0 tx=0x7f525c00c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:18.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.573+0000 7f5270c00640 1 -- 172.21.15.86:0/932348427 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525c018070 con 0x7f52740ccdd0 2024-04-20T21:21:18.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.573+0000 7f5270c00640 1 -- 172.21.15.86:0/932348427 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f525c0074c0 con 0x7f52740ccdd0 2024-04-20T21:21:18.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.573+0000 7f5270c00640 1 -- 172.21.15.86:0/932348427 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525c004a20 con 0x7f52740ccdd0 2024-04-20T21:21:18.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.573+0000 7f527ae00640 1 -- 172.21.15.86:0/932348427 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52740b3a70 con 0x7f52740ccdd0 2024-04-20T21:21:18.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.573+0000 7f527ae00640 1 -- 172.21.15.86:0/932348427 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52740b4030 con 0x7f52740ccdd0 2024-04-20T21:21:18.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.574+0000 7f527ae00640 1 -- 172.21.15.86:0/932348427 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52740689e0 con 0x7f52740ccdd0 2024-04-20T21:21:18.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.575+0000 7f5270c00640 1 -- 172.21.15.86:0/932348427 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f525c01b070 con 0x7f52740ccdd0 2024-04-20T21:21:18.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.576+0000 7f5270c00640 1 --2- 172.21.15.86:0/932348427 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f525006c8f0 0x7f525006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:18.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.576+0000 7f5270c00640 1 -- 172.21.15.86:0/932348427 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f525c08aee0 con 0x7f52740ccdd0 2024-04-20T21:21:18.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.577+0000 7f5273e00640 1 --2- 172.21.15.86:0/932348427 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f525006c8f0 0x7f525006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:18.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.577+0000 7f5273e00640 1 --2- 172.21.15.86:0/932348427 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f525006c8f0 0x7f525006edb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5264006fd0 tx=0x7f5264008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:18.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.581+0000 7f5270c00640 1 -- 172.21.15.86:0/932348427 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f52741f46f0 con 0x7f52740ccdd0 2024-04-20T21:21:18.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.824+0000 7f527ae00640 1 -- 172.21.15.86:0/932348427 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f527415ff50 con 0x7f52740ccdd0 2024-04-20T21:21:18.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.825+0000 7f5270c00640 1 -- 172.21.15.86:0/932348427 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+5205 (secure 0 0 0) 0x7f525c0593c0 con 0x7f52740ccdd0 2024-04-20T21:21:18.825 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:21:18.825 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4913,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3933408964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3933408964},{"type":"v1","addr":"172.21.15.144:6833","nonce":3933408964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5221,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/1522070692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1522070692},{"type":"v1","addr":"172.21.15.144:6835","nonce":1522070692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":6132,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/1352073341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1352073341},{"type":"v1","addr":"172.21.15.86:6836","nonce":1352073341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:21:07.502800+0000","modified":"2024-04-20T21:21:10.553495+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6135},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6135":{"gid":6135,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/3008383261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3008383261},{"type":"v1","addr":"172.21.15.86:6837","nonce":3008383261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6135,"qdb_cluster":[6135]},"id":4}]} 2024-04-20T21:21:18.825 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 32 2024-04-20T21:21:18.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.828+0000 7f527ae00640 1 -- 172.21.15.86:0/932348427 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f525006c8f0 msgr2=0x7f525006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:18.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.828+0000 7f527ae00640 1 --2- 172.21.15.86:0/932348427 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f525006c8f0 0x7f525006edb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5264006fd0 tx=0x7f5264008040 comp rx=0 tx=0).stop 2024-04-20T21:21:18.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.828+0000 7f527ae00640 1 -- 172.21.15.86:0/932348427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52740ccdd0 msgr2=0x7f52741f7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:18.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.828+0000 7f527ae00640 1 --2- 172.21.15.86:0/932348427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52740ccdd0 0x7f52741f7150 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f525c00c4e0 tx=0x7f525c00c9a0 comp rx=0 tx=0).stop 2024-04-20T21:21:18.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.829+0000 7f527ae00640 1 -- 172.21.15.86:0/932348427 shutdown_connections 2024-04-20T21:21:18.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.829+0000 7f527ae00640 1 --2- 172.21.15.86:0/932348427 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f525006c8f0 0x7f525006edb0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:18.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.829+0000 7f527ae00640 1 --2- 172.21.15.86:0/932348427 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5274146130 0x7f52741f41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:18.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.829+0000 7f527ae00640 1 --2- 172.21.15.86:0/932348427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52740ccdd0 0x7f52741f7150 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:18.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.829+0000 7f527ae00640 1 --2- 172.21.15.86:0/932348427 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5274072410 0x7f52741e5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:18.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.829+0000 7f527ae00640 1 -- 172.21.15.86:0/932348427 >> 172.21.15.86:0/932348427 conn(0x7f52740b2360 msgr2=0x7f527406e520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:18.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.829+0000 7f527ae00640 1 -- 172.21.15.86:0/932348427 shutdown_connections 2024-04-20T21:21:18.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:18.829+0000 7f527ae00640 1 -- 172.21.15.86:0/932348427 wait complete. 2024-04-20T21:21:18.846 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-20T21:21:19.089 INFO:teuthology.orchestra.run.smithi086.stdout:{"message":"","return_code":0} 2024-04-20T21:21:19.089 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-20T21:21:19.089 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-20T21:21:19.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.196+0000 7f5e64fb3f40 1 Processor -- start 2024-04-20T21:21:19.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.196+0000 7f5e64fb3f40 1 -- start start 2024-04-20T21:21:19.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.197+0000 7f5e64fb3f40 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5562a155ecf0 0x5562a15fd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.197+0000 7f5e64fb3f40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562a1573f60 0x5562a155bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.197+0000 7f5e64fb3f40 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5562a1579720 0x5562a155aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.197+0000 7f5e64fb3f40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5562a14d6cc0 con 0x5562a1573f60 2024-04-20T21:21:19.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.197+0000 7f5e64fb3f40 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5562a1601150 con 0x5562a155ecf0 2024-04-20T21:21:19.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.197+0000 7f5e64fb3f40 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5562a152db20 con 0x5562a1579720 2024-04-20T21:21:19.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.197+0000 7f5e61800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562a1573f60 0x5562a155bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:19.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.197+0000 7f5e61800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562a1573f60 0x5562a155bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58650/0 (socket says 172.21.15.86:58650) 2024-04-20T21:21:19.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.197+0000 7f5e61800640 1 -- 172.21.15.86:0/117889009 learned_addr learned my addr 172.21.15.86:0/117889009 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:19.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.197+0000 7f5e61800640 1 -- 172.21.15.86:0/117889009 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5562a1579720 msgr2=0x5562a155aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:19.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.197+0000 7f5e61800640 1 --2- 172.21.15.86:0/117889009 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5562a1579720 0x5562a155aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.197+0000 7f5e61800640 1 -- 172.21.15.86:0/117889009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5562a155ecf0 msgr2=0x5562a15fd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:19.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.197+0000 7f5e61800640 1 --2- 172.21.15.86:0/117889009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5562a155ecf0 0x5562a15fd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.197+0000 7f5e61800640 1 -- 172.21.15.86:0/117889009 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562a155fbc0 con 0x5562a1573f60 2024-04-20T21:21:19.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.198+0000 7f5e61800640 1 --2- 172.21.15.86:0/117889009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562a1573f60 0x5562a155bb40 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f5e5000b270 tx=0x7f5e5000b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b24b7f142be2e37 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:19.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.198+0000 7f5e5be00640 1 -- 172.21.15.86:0/117889009 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e50008070 con 0x5562a1573f60 2024-04-20T21:21:19.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.198+0000 7f5e5be00640 1 -- 172.21.15.86:0/117889009 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e5000d500 con 0x5562a1573f60 2024-04-20T21:21:19.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.198+0000 7f5e5be00640 1 -- 172.21.15.86:0/117889009 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5000d8b0 con 0x5562a1573f60 2024-04-20T21:21:19.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.198+0000 7f5e64fb3f40 1 -- 172.21.15.86:0/117889009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562a1573f60 msgr2=0x5562a155bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:19.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.198+0000 7f5e64fb3f40 1 --2- 172.21.15.86:0/117889009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562a1573f60 0x5562a155bb40 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f5e5000b270 tx=0x7f5e5000b730 comp rx=0 tx=0).stop 2024-04-20T21:21:19.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.198+0000 7f5e64fb3f40 1 -- 172.21.15.86:0/117889009 shutdown_connections 2024-04-20T21:21:19.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.198+0000 7f5e64fb3f40 1 --2- 172.21.15.86:0/117889009 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5562a1579720 0x5562a155aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.198+0000 7f5e64fb3f40 1 --2- 172.21.15.86:0/117889009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562a1573f60 0x5562a155bb40 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.198+0000 7f5e64fb3f40 1 --2- 172.21.15.86:0/117889009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5562a155ecf0 0x5562a15fd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.198+0000 7f5e64fb3f40 1 -- 172.21.15.86:0/117889009 >> 172.21.15.86:0/117889009 conn(0x5562a152d220 msgr2=0x5562a1577870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:19.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.199+0000 7f5e64fb3f40 1 -- 172.21.15.86:0/117889009 shutdown_connections 2024-04-20T21:21:19.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.199+0000 7f5e64fb3f40 1 -- 172.21.15.86:0/117889009 wait complete. 2024-04-20T21:21:19.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.199+0000 7f5e64fb3f40 1 Processor -- start 2024-04-20T21:21:19.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.200+0000 7f5e64fb3f40 1 -- start start 2024-04-20T21:21:19.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.200+0000 7f5e64fb3f40 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5562a155ecf0 0x5562a15fd850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.200+0000 7f5e64fb3f40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562a1573f60 0x5562a15f8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.200+0000 7f5e64fb3f40 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5562a1579720 0x5562a15fba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.201+0000 7f5e64fb3f40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5562a15f93b0 con 0x5562a1573f60 2024-04-20T21:21:19.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.201+0000 7f5e64fb3f40 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5562a15f9c60 con 0x5562a155ecf0 2024-04-20T21:21:19.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.201+0000 7f5e64fb3f40 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5562a152d600 con 0x5562a1579720 2024-04-20T21:21:19.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.201+0000 7f5e61800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562a1573f60 0x5562a15f8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:19.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.201+0000 7f5e61800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562a1573f60 0x5562a15f8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58658/0 (socket says 172.21.15.86:58658) 2024-04-20T21:21:19.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.201+0000 7f5e61800640 1 -- 172.21.15.86:0/728986782 learned_addr learned my addr 172.21.15.86:0/728986782 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:19.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.201+0000 7f5e62200640 1 --2- 172.21.15.86:0/728986782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5562a155ecf0 0x5562a15fd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:19.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.201+0000 7f5e62c00640 1 --2- 172.21.15.86:0/728986782 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5562a1579720 0x5562a15fba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:19.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.201+0000 7f5e61800640 1 -- 172.21.15.86:0/728986782 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5562a1579720 msgr2=0x5562a15fba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:19.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.201+0000 7f5e61800640 1 --2- 172.21.15.86:0/728986782 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5562a1579720 0x5562a15fba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.201+0000 7f5e61800640 1 -- 172.21.15.86:0/728986782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5562a155ecf0 msgr2=0x5562a15fd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:19.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.201+0000 7f5e61800640 1 --2- 172.21.15.86:0/728986782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5562a155ecf0 0x5562a15fd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.201+0000 7f5e61800640 1 -- 172.21.15.86:0/728986782 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562a15f5b70 con 0x5562a1573f60 2024-04-20T21:21:19.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.201+0000 7f5e62200640 1 --2- 172.21.15.86:0/728986782 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5562a155ecf0 0x5562a15fd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:21:19.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.202+0000 7f5e61800640 1 --2- 172.21.15.86:0/728986782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562a1573f60 0x5562a15f8cf0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f5e5000b770 tx=0x7f5e50000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:19.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.202+0000 7f5e5aa00640 1 -- 172.21.15.86:0/728986782 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e50021030 con 0x5562a1573f60 2024-04-20T21:21:19.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.202+0000 7f5e5aa00640 1 -- 172.21.15.86:0/728986782 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e5001c440 con 0x5562a1573f60 2024-04-20T21:21:19.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.202+0000 7f5e64fb3f40 1 -- 172.21.15.86:0/728986782 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5562a155c6b0 con 0x5562a1573f60 2024-04-20T21:21:19.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.202+0000 7f5e5aa00640 1 -- 172.21.15.86:0/728986782 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5001c6d0 con 0x5562a1573f60 2024-04-20T21:21:19.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.202+0000 7f5e64fb3f40 1 -- 172.21.15.86:0/728986782 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562a15f8210 con 0x5562a1573f60 2024-04-20T21:21:19.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.204+0000 7f5e5aa00640 1 -- 172.21.15.86:0/728986782 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5e50024070 con 0x5562a1573f60 2024-04-20T21:21:19.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.205+0000 7f5e5aa00640 1 --2- 172.21.15.86:0/728986782 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5e4406b300 0x7f5e4406d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.205+0000 7f5e5aa00640 1 -- 172.21.15.86:0/728986782 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f5e5008ba30 con 0x5562a1573f60 2024-04-20T21:21:19.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.205+0000 7f5e62200640 1 --2- 172.21.15.86:0/728986782 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5e4406b300 0x7f5e4406d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:19.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.205+0000 7f5e64fb3f40 1 --2- 172.21.15.86:0/728986782 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x5562a15fbf90 0x5562a169e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.205+0000 7f5e64fb3f40 1 -- 172.21.15.86:0/728986782 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:1 9.7 9:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59) -- 0x5562a155a750 con 0x5562a15fbf90 2024-04-20T21:21:19.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.206+0000 7f5e62200640 1 --2- 172.21.15.86:0/728986782 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5e4406b300 0x7f5e4406d7c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5e54057000 tx=0x7f5e54051000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:19.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.206+0000 7f5e62c00640 1 --2- 172.21.15.86:0/728986782 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x5562a15fbf90 0x5562a169e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:19.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.207+0000 7f5e62c00640 1 --2- 172.21.15.86:0/728986782 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x5562a15fbf90 0x5562a169e150 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:19.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.208+0000 7f5e62c00640 1 -- 172.21.15.86:0/728986782 <== osd.5 v2:172.21.15.144:6808/1635974704 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (94) out=94b] v0'0 uv1 ondisk = 0) ==== 164+0+94 (crc 0 0 0) 0x7f5e5c00c620 con 0x5562a15fbf90 2024-04-20T21:21:19.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.208+0000 7f5e64fb3f40 1 -- 172.21.15.86:0/728986782 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x5562a15fbf90 msgr2=0x5562a169e150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:19.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.208+0000 7f5e64fb3f40 1 --2- 172.21.15.86:0/728986782 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x5562a15fbf90 0x5562a169e150 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.208+0000 7f5e64fb3f40 1 -- 172.21.15.86:0/728986782 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5e4406b300 msgr2=0x7f5e4406d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:19.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.208+0000 7f5e64fb3f40 1 --2- 172.21.15.86:0/728986782 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5e4406b300 0x7f5e4406d7c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5e54057000 tx=0x7f5e54051000 comp rx=0 tx=0).stop 2024-04-20T21:21:19.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.208+0000 7f5e64fb3f40 1 -- 172.21.15.86:0/728986782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562a1573f60 msgr2=0x5562a15f8cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:19.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.208+0000 7f5e64fb3f40 1 --2- 172.21.15.86:0/728986782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5562a1573f60 0x5562a15f8cf0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f5e5000b770 tx=0x7f5e50000ed0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.208+0000 7f5e62c00640 1 -- 172.21.15.86:0/728986782 reap_dead start 2024-04-20T21:21:19.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.209+0000 7f5e64fb3f40 1 -- 172.21.15.86:0/728986782 shutdown_connections 2024-04-20T21:21:19.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.209+0000 7f5e64fb3f40 1 -- 172.21.15.86:0/728986782 >> 172.21.15.86:0/728986782 conn(0x5562a152d220 msgr2=0x5562a155ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:19.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.209+0000 7f5e64fb3f40 1 -- 172.21.15.86:0/728986782 shutdown_connections 2024-04-20T21:21:19.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.209+0000 7f5e64fb3f40 1 -- 172.21.15.86:0/728986782 wait complete. 2024-04-20T21:21:19.212 INFO:teuthology.orchestra.run.smithi086.stdout:Xfiledir_test 2024-04-20T21:21:19.212 DEBUG:teuthology.orchestra.run.smithi086:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-20T21:21:19.309 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:21:19.309 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 1099511627777, 2024-04-20T21:21:19.310 INFO:teuthology.orchestra.run.smithi086.stdout: "ancestors": [ 2024-04-20T21:21:19.310 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:21:19.310 INFO:teuthology.orchestra.run.smithi086.stdout: "dirino": 1099511627776, 2024-04-20T21:21:19.310 INFO:teuthology.orchestra.run.smithi086.stdout: "dname": "file", 2024-04-20T21:21:19.310 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 2 2024-04-20T21:21:19.310 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:21:19.310 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:21:19.310 INFO:teuthology.orchestra.run.smithi086.stdout: "dirino": 1, 2024-04-20T21:21:19.310 INFO:teuthology.orchestra.run.smithi086.stdout: "dname": "dir_test", 2024-04-20T21:21:19.310 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 4 2024-04-20T21:21:19.310 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:21:19.310 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-20T21:21:19.310 INFO:teuthology.orchestra.run.smithi086.stdout: "pool": 9, 2024-04-20T21:21:19.311 INFO:teuthology.orchestra.run.smithi086.stdout: "old_pools": [] 2024-04-20T21:21:19.311 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:21:19.311 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:21:19.315 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'The backtrace is corrupted' 2024-04-20T21:21:19.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.424+0000 7f73521fef40 1 Processor -- start 2024-04-20T21:21:19.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.424+0000 7f73521fef40 1 -- start start 2024-04-20T21:21:19.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.424+0000 7f73521fef40 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ecf294ccf0 0x55ecf291bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.425+0000 7f73521fef40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecf29f89d0 0x55ecf291c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.425+0000 7f73521fef40 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecf29f8db0 0x55ecf29ec9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.425+0000 7f73521fef40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ecf28c4cc0 con 0x55ecf29f89d0 2024-04-20T21:21:19.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.425+0000 7f73521fef40 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ecf294d2b0 con 0x55ecf29f8db0 2024-04-20T21:21:19.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.425+0000 7f73521fef40 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55ecf2a88030 con 0x55ecf294ccf0 2024-04-20T21:21:19.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.425+0000 7f734ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecf29f89d0 0x55ecf291c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:19.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.425+0000 7f734ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecf29f89d0 0x55ecf291c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58662/0 (socket says 172.21.15.86:58662) 2024-04-20T21:21:19.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.425+0000 7f734ea00640 1 -- 172.21.15.86:0/3001803573 learned_addr learned my addr 172.21.15.86:0/3001803573 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:19.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.425+0000 7f734fe00640 1 --2- 172.21.15.86:0/3001803573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecf29f8db0 0x55ecf29ec9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:19.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.425+0000 7f734f400640 1 --2- 172.21.15.86:0/3001803573 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ecf294ccf0 0x55ecf291bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:19.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.425+0000 7f734ea00640 1 -- 172.21.15.86:0/3001803573 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ecf294ccf0 msgr2=0x55ecf291bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:19.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.425+0000 7f734ea00640 1 --2- 172.21.15.86:0/3001803573 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ecf294ccf0 0x55ecf291bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.425+0000 7f734ea00640 1 -- 172.21.15.86:0/3001803573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecf29f8db0 msgr2=0x55ecf29ec9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:19.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.425+0000 7f734ea00640 1 --2- 172.21.15.86:0/3001803573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecf29f8db0 0x55ecf29ec9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.425+0000 7f734ea00640 1 -- 172.21.15.86:0/3001803573 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecf2a87780 con 0x55ecf29f89d0 2024-04-20T21:21:19.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.426+0000 7f734ea00640 1 --2- 172.21.15.86:0/3001803573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecf29f89d0 0x55ecf291c440 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f733c009750 tx=0x7f733c005ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a5f659a3094bde server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:19.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.426+0000 7f734d600640 1 -- 172.21.15.86:0/3001803573 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c019070 con 0x55ecf29f89d0 2024-04-20T21:21:19.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.426+0000 7f734d600640 1 -- 172.21.15.86:0/3001803573 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f733c008890 con 0x55ecf29f89d0 2024-04-20T21:21:19.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.426+0000 7f734d600640 1 -- 172.21.15.86:0/3001803573 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c008c40 con 0x55ecf29f89d0 2024-04-20T21:21:19.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.426+0000 7f73521fef40 1 -- 172.21.15.86:0/3001803573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecf29f89d0 msgr2=0x55ecf291c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:19.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.427+0000 7f73521fef40 1 --2- 172.21.15.86:0/3001803573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecf29f89d0 0x55ecf291c440 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f733c009750 tx=0x7f733c005ac0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.427+0000 7f73521fef40 1 -- 172.21.15.86:0/3001803573 shutdown_connections 2024-04-20T21:21:19.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.427+0000 7f73521fef40 1 --2- 172.21.15.86:0/3001803573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecf29f8db0 0x55ecf29ec9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.427+0000 7f73521fef40 1 --2- 172.21.15.86:0/3001803573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecf29f89d0 0x55ecf291c440 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.427+0000 7f73521fef40 1 --2- 172.21.15.86:0/3001803573 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ecf294ccf0 0x55ecf291bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.427+0000 7f73521fef40 1 -- 172.21.15.86:0/3001803573 >> 172.21.15.86:0/3001803573 conn(0x55ecf291b220 msgr2=0x55ecf2966f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:19.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.427+0000 7f73521fef40 1 -- 172.21.15.86:0/3001803573 shutdown_connections 2024-04-20T21:21:19.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.427+0000 7f73521fef40 1 -- 172.21.15.86:0/3001803573 wait complete. 2024-04-20T21:21:19.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.427+0000 7f73521fef40 1 Processor -- start 2024-04-20T21:21:19.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.428+0000 7f73521fef40 1 -- start start 2024-04-20T21:21:19.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.428+0000 7f73521fef40 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecf294ccf0 0x55ecf29f0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.428+0000 7f73521fef40 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ecf29f89d0 0x55ecf29e7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.428+0000 7f73521fef40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecf29f8db0 0x55ecf29f5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.428+0000 7f73521fef40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ecf2a87cc0 con 0x55ecf29f8db0 2024-04-20T21:21:19.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.428+0000 7f73521fef40 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ecf2a87a20 con 0x55ecf294ccf0 2024-04-20T21:21:19.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.428+0000 7f73521fef40 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55ecf2a884c0 con 0x55ecf29f89d0 2024-04-20T21:21:19.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.429+0000 7f734fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecf29f8db0 0x55ecf29f5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:19.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.429+0000 7f734fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecf29f8db0 0x55ecf29f5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58666/0 (socket says 172.21.15.86:58666) 2024-04-20T21:21:19.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.429+0000 7f734fe00640 1 -- 172.21.15.86:0/3502993367 learned_addr learned my addr 172.21.15.86:0/3502993367 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:19.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.429+0000 7f734ea00640 1 --2- 172.21.15.86:0/3502993367 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ecf29f89d0 0x55ecf29e7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:19.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.429+0000 7f734f400640 1 --2- 172.21.15.86:0/3502993367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecf294ccf0 0x55ecf29f0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:19.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.429+0000 7f734fe00640 1 -- 172.21.15.86:0/3502993367 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ecf29f89d0 msgr2=0x55ecf29e7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:19.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.429+0000 7f734fe00640 1 --2- 172.21.15.86:0/3502993367 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ecf29f89d0 0x55ecf29e7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.429+0000 7f734fe00640 1 -- 172.21.15.86:0/3502993367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecf294ccf0 msgr2=0x55ecf29f0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:19.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.429+0000 7f734fe00640 1 --2- 172.21.15.86:0/3502993367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecf294ccf0 0x55ecf29f0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.429+0000 7f734fe00640 1 -- 172.21.15.86:0/3502993367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecf29e8580 con 0x55ecf29f8db0 2024-04-20T21:21:19.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.429+0000 7f734ea00640 1 --2- 172.21.15.86:0/3502993367 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ecf29f89d0 0x55ecf29e7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:21:19.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.429+0000 7f734f400640 1 --2- 172.21.15.86:0/3502993367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ecf294ccf0 0x55ecf29f0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:21:19.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.430+0000 7f734fe00640 1 --2- 172.21.15.86:0/3502993367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecf29f8db0 0x55ecf29f5e30 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f734800cd00 tx=0x7f734800a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:19.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.430+0000 7f7347e00640 1 -- 172.21.15.86:0/3502993367 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f734800f620 con 0x55ecf29f8db0 2024-04-20T21:21:19.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.430+0000 7f7347e00640 1 -- 172.21.15.86:0/3502993367 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7348006070 con 0x55ecf29f8db0 2024-04-20T21:21:19.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.430+0000 7f7347e00640 1 -- 172.21.15.86:0/3502993367 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f734800f9e0 con 0x55ecf29f8db0 2024-04-20T21:21:19.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.430+0000 7f73521fef40 1 -- 172.21.15.86:0/3502993367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ecf29e8860 con 0x55ecf29f8db0 2024-04-20T21:21:19.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.430+0000 7f73521fef40 1 -- 172.21.15.86:0/3502993367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecf29e80f0 con 0x55ecf29f8db0 2024-04-20T21:21:19.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.432+0000 7f7347e00640 1 -- 172.21.15.86:0/3502993367 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f734800fb80 con 0x55ecf29f8db0 2024-04-20T21:21:19.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.433+0000 7f7347e00640 1 --2- 172.21.15.86:0/3502993367 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f733006b2d0 0x7f733006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.433+0000 7f7347e00640 1 -- 172.21.15.86:0/3502993367 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f734808a980 con 0x55ecf29f8db0 2024-04-20T21:21:19.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.433+0000 7f734f400640 1 --2- 172.21.15.86:0/3502993367 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f733006b2d0 0x7f733006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:19.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.434+0000 7f73521fef40 1 --2- 172.21.15.86:0/3502993367 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x55ecf29fb860 0x55ecf29f6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.434+0000 7f73521fef40 1 -- 172.21.15.86:0/3502993367 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:1 9.7 9:e2e07930:::10000000001.00000000:head [setxattr parent (26) in=32b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59) -- 0x55ecf29ebcd0 con 0x55ecf29fb860 2024-04-20T21:21:19.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.434+0000 7f734f400640 1 --2- 172.21.15.86:0/3502993367 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f733006b2d0 0x7f733006d790 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7340056000 tx=0x7f7340007030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:19.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.434+0000 7f734ea00640 1 --2- 172.21.15.86:0/3502993367 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x55ecf29fb860 0x55ecf29f6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:19.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.435+0000 7f734ea00640 1 --2- 172.21.15.86:0/3502993367 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x55ecf29fb860 0x55ecf29f6880 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:19.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.439+0000 7f734ea00640 1 -- 172.21.15.86:0/3502993367 <== osd.5 v2:172.21.15.144:6808/1635974704 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (26)] v59'2 uv2 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f733c008a40 con 0x55ecf29fb860 2024-04-20T21:21:19.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.439+0000 7f73521fef40 1 -- 172.21.15.86:0/3502993367 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x55ecf29fb860 msgr2=0x55ecf29f6880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:19.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.439+0000 7f73521fef40 1 --2- 172.21.15.86:0/3502993367 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x55ecf29fb860 0x55ecf29f6880 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.439+0000 7f73521fef40 1 -- 172.21.15.86:0/3502993367 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f733006b2d0 msgr2=0x7f733006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:19.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.439+0000 7f73521fef40 1 --2- 172.21.15.86:0/3502993367 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f733006b2d0 0x7f733006d790 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7340056000 tx=0x7f7340007030 comp rx=0 tx=0).stop 2024-04-20T21:21:19.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.439+0000 7f73521fef40 1 -- 172.21.15.86:0/3502993367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecf29f8db0 msgr2=0x55ecf29f5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:19.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.439+0000 7f73521fef40 1 --2- 172.21.15.86:0/3502993367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ecf29f8db0 0x55ecf29f5e30 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f734800cd00 tx=0x7f734800a5f0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.439+0000 7f734fe00640 1 -- 172.21.15.86:0/3502993367 reap_dead start 2024-04-20T21:21:19.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.439+0000 7f73521fef40 1 -- 172.21.15.86:0/3502993367 shutdown_connections 2024-04-20T21:21:19.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.439+0000 7f73521fef40 1 -- 172.21.15.86:0/3502993367 >> 172.21.15.86:0/3502993367 conn(0x55ecf291b220 msgr2=0x55ecf29f9430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:19.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.439+0000 7f73521fef40 1 -- 172.21.15.86:0/3502993367 shutdown_connections 2024-04-20T21:21:19.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.439+0000 7f73521fef40 1 -- 172.21.15.86:0/3502993367 wait complete. 2024-04-20T21:21:19.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:21:19.448 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / recursive 2024-04-20T21:21:19.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.651+0000 7f7e92e00640 1 Processor -- start 2024-04-20T21:21:19.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.652+0000 7f7e92e00640 1 -- start start 2024-04-20T21:21:19.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.653+0000 7f7e92e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e8c1569d0 0x7f7e8c0d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.653+0000 7f7e92e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7e8c06e700 0x7f7e8c0d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.653+0000 7f7e92e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e8c15d6b0 con 0x7f7e8c15c9f0 2024-04-20T21:21:19.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.653+0000 7f7e92e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f7e8c06dc20 con 0x7f7e8c06e700 2024-04-20T21:21:19.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.653+0000 7f7e92e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7e8c06df00 con 0x7f7e8c1569d0 2024-04-20T21:21:19.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.654+0000 7f7e91000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7e8c06e700 0x7f7e8c0d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:19.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.654+0000 7f7e91a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e8c1569d0 0x7f7e8c0d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:19.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.654+0000 7f7e91000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7e8c06e700 0x7f7e8c0d0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.86:51948/0 (socket says 172.21.15.86:51948) 2024-04-20T21:21:19.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.654+0000 7f7e91000640 1 -- 172.21.15.86:0/4063581985 learned_addr learned my addr 172.21.15.86:0/4063581985 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:19.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.654+0000 7f7e92400640 1 --1- 172.21.15.86:0/4063581985 >> v1:172.21.15.144:6789/0 conn(0x7f7e8c15c9f0 0x7f7e8c15ce40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.86:39734/0 (socket says 172.21.15.86:39734) 2024-04-20T21:21:19.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.655+0000 7f7e91000640 1 -- 172.21.15.86:0/4063581985 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e8c1569d0 msgr2=0x7f7e8c0d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:19.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.655+0000 7f7e91000640 1 --2- 172.21.15.86:0/4063581985 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e8c1569d0 0x7f7e8c0d05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.655+0000 7f7e91000640 1 -- 172.21.15.86:0/4063581985 >> v1:172.21.15.144:6789/0 conn(0x7f7e8c15c9f0 legacy=0x7f7e8c15ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:19.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.655+0000 7f7e91000640 1 -- 172.21.15.86:0/4063581985 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e8c06e1e0 con 0x7f7e8c06e700 2024-04-20T21:21:19.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.656+0000 7f7e91000640 1 --2- 172.21.15.86:0/4063581985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e8c06e700 0x7f7e8c0d0b30 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7e78009870 tx=0x7f7e7802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=701ec23926490aba server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:19.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.657+0000 7f7e8be00640 1 -- 172.21.15.86:0/4063581985 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e78034070 con 0x7f7e8c06e700 2024-04-20T21:21:19.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.657+0000 7f7e8be00640 1 -- 172.21.15.86:0/4063581985 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e78030840 con 0x7f7e8c06e700 2024-04-20T21:21:19.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.657+0000 7f7e8be00640 1 -- 172.21.15.86:0/4063581985 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e78030c10 con 0x7f7e8c06e700 2024-04-20T21:21:19.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.658+0000 7f7e92e00640 1 -- 172.21.15.86:0/4063581985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e8c06e700 msgr2=0x7f7e8c0d0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:19.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.658+0000 7f7e92e00640 1 --2- 172.21.15.86:0/4063581985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e8c06e700 0x7f7e8c0d0b30 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7e78009870 tx=0x7f7e7802b980 comp rx=0 tx=0).stop 2024-04-20T21:21:19.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.658+0000 7f7e92e00640 1 -- 172.21.15.86:0/4063581985 shutdown_connections 2024-04-20T21:21:19.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.658+0000 7f7e92e00640 1 --2- 172.21.15.86:0/4063581985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e8c06e700 0x7f7e8c0d0b30 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.658+0000 7f7e92e00640 1 --2- 172.21.15.86:0/4063581985 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e8c1569d0 0x7f7e8c0d05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.658+0000 7f7e92e00640 1 -- 172.21.15.86:0/4063581985 >> 172.21.15.86:0/4063581985 conn(0x7f7e8c0b2220 msgr2=0x7f7e8c0b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:19.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.658+0000 7f7e92e00640 1 -- 172.21.15.86:0/4063581985 shutdown_connections 2024-04-20T21:21:19.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.658+0000 7f7e92e00640 1 -- 172.21.15.86:0/4063581985 wait complete. 2024-04-20T21:21:19.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.660+0000 7f7e92e00640 1 Processor -- start 2024-04-20T21:21:19.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.660+0000 7f7e92e00640 1 -- start start 2024-04-20T21:21:19.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.662+0000 7f7e92e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e8c06e700 0x7f7e8c1464a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.662+0000 7f7e92e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e8c1569d0 0x7f7e8c152380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.662+0000 7f7e92e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e8c15c9f0 0x7f7e8c1537d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.662+0000 7f7e92e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c15dc80 con 0x7f7e8c15c9f0 2024-04-20T21:21:19.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.662+0000 7f7e92e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c15db00 con 0x7f7e8c1569d0 2024-04-20T21:21:19.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.662+0000 7f7e92e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7e8c15de00 con 0x7f7e8c06e700 2024-04-20T21:21:19.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.662+0000 7f7e92400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e8c15c9f0 0x7f7e8c1537d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:19.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.662+0000 7f7e92400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e8c15c9f0 0x7f7e8c1537d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58672/0 (socket says 172.21.15.86:58672) 2024-04-20T21:21:19.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.662+0000 7f7e92400640 1 -- 172.21.15.86:0/1519941358 learned_addr learned my addr 172.21.15.86:0/1519941358 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:19.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.662+0000 7f7e91000640 1 --2- 172.21.15.86:0/1519941358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e8c1569d0 0x7f7e8c152380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:19.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.662+0000 7f7e91a00640 1 --2- 172.21.15.86:0/1519941358 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e8c06e700 0x7f7e8c1464a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:19.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.663+0000 7f7e92400640 1 -- 172.21.15.86:0/1519941358 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e8c06e700 msgr2=0x7f7e8c1464a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:19.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.663+0000 7f7e92400640 1 --2- 172.21.15.86:0/1519941358 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e8c06e700 0x7f7e8c1464a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.663+0000 7f7e92400640 1 -- 172.21.15.86:0/1519941358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e8c1569d0 msgr2=0x7f7e8c152380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:19.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.663+0000 7f7e92400640 1 --2- 172.21.15.86:0/1519941358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e8c1569d0 0x7f7e8c152380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.663+0000 7f7e92400640 1 -- 172.21.15.86:0/1519941358 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e8c152c80 con 0x7f7e8c15c9f0 2024-04-20T21:21:19.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.663+0000 7f7e92400640 1 --2- 172.21.15.86:0/1519941358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e8c15c9f0 0x7f7e8c1537d0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f7e8400c730 tx=0x7f7e8400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:19.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.663+0000 7f7e89600640 1 -- 172.21.15.86:0/1519941358 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e84017070 con 0x7f7e8c15c9f0 2024-04-20T21:21:19.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.663+0000 7f7e89600640 1 -- 172.21.15.86:0/1519941358 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e8400ce40 con 0x7f7e8c15c9f0 2024-04-20T21:21:19.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.663+0000 7f7e89600640 1 -- 172.21.15.86:0/1519941358 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8400a840 con 0x7f7e8c15c9f0 2024-04-20T21:21:19.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.663+0000 7f7e92e00640 1 -- 172.21.15.86:0/1519941358 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e8c152f60 con 0x7f7e8c15c9f0 2024-04-20T21:21:19.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.663+0000 7f7e92e00640 1 -- 172.21.15.86:0/1519941358 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e8c1469e0 con 0x7f7e8c15c9f0 2024-04-20T21:21:19.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.667+0000 7f7e89600640 1 -- 172.21.15.86:0/1519941358 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7e8401a070 con 0x7f7e8c15c9f0 2024-04-20T21:21:19.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.667+0000 7f7e89600640 1 --2- 172.21.15.86:0/1519941358 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7e6c06c8f0 0x7f7e6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.668+0000 7f7e89600640 1 -- 172.21.15.86:0/1519941358 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f7e8408afe0 con 0x7f7e8c15c9f0 2024-04-20T21:21:19.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.668+0000 7f7e97220740 1 Processor -- start 2024-04-20T21:21:19.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.668+0000 7f7e91a00640 1 --2- 172.21.15.86:0/1519941358 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7e6c06c8f0 0x7f7e6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:19.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.668+0000 7f7e97220740 1 -- start start 2024-04-20T21:21:19.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.669+0000 7f7e91a00640 1 --2- 172.21.15.86:0/1519941358 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7e6c06c8f0 0x7f7e6c06edb0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7e80009a10 tx=0x7f7e80008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:19.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.669+0000 7f7e97220740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c5bdecf460 0x55c5bdea2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.669+0000 7f7e97220740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bdecf840 0x55c5bded21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.669+0000 7f7e97220740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c5bdea2940 0x55c5bded2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.669+0000 7f7e97220740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c5bde0cd50 con 0x55c5bdecf840 2024-04-20T21:21:19.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.669+0000 7f7e97220740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c5bddb32b0 con 0x55c5bdecf460 2024-04-20T21:21:19.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.669+0000 7f7e97220740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55c5bde08d10 con 0x55c5bdea2940 2024-04-20T21:21:19.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.669+0000 7f7e91000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bdecf840 0x55c5bded21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:19.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.669+0000 7f7e91000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bdecf840 0x55c5bded21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58688/0 (socket says 172.21.15.86:58688) 2024-04-20T21:21:19.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.669+0000 7f7e91000640 1 -- 172.21.15.86:0/3951625278 learned_addr learned my addr 172.21.15.86:0/3951625278 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:19.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.670+0000 7f7e91000640 1 -- 172.21.15.86:0/3951625278 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c5bdea2940 msgr2=0x55c5bded2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:19.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.670+0000 7f7e92400640 1 --2- 172.21.15.86:0/3951625278 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c5bdea2940 0x55c5bded2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:19.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.670+0000 7f7e91a00640 1 --2- 172.21.15.86:0/3951625278 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c5bdecf460 0x55c5bdea2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:19.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.670+0000 7f7e91000640 1 --2- 172.21.15.86:0/3951625278 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c5bdea2940 0x55c5bded2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.670+0000 7f7e91000640 1 -- 172.21.15.86:0/3951625278 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c5bdecf460 msgr2=0x55c5bdea2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:19.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.670+0000 7f7e91000640 1 --2- 172.21.15.86:0/3951625278 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c5bdecf460 0x55c5bdea2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.670+0000 7f7e91000640 1 -- 172.21.15.86:0/3951625278 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5bddca230 con 0x55c5bdecf840 2024-04-20T21:21:19.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.670+0000 7f7e91000640 1 --2- 172.21.15.86:0/3951625278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bdecf840 0x55c5bded21f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f7e78000c00 tx=0x7f7e78008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b160250878c0d117 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:19.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.670+0000 7f7e7e000640 1 -- 172.21.15.86:0/3951625278 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e78034040 con 0x55c5bdecf840 2024-04-20T21:21:19.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.670+0000 7f7e7e000640 1 -- 172.21.15.86:0/3951625278 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e78049070 con 0x55c5bdecf840 2024-04-20T21:21:19.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.670+0000 7f7e7e000640 1 -- 172.21.15.86:0/3951625278 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e78030e10 con 0x55c5bdecf840 2024-04-20T21:21:19.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.671+0000 7f7e97220740 1 -- 172.21.15.86:0/3951625278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bdecf840 msgr2=0x55c5bded21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:19.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.671+0000 7f7e97220740 1 --2- 172.21.15.86:0/3951625278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bdecf840 0x55c5bded21f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f7e78000c00 tx=0x7f7e78008030 comp rx=0 tx=0).stop 2024-04-20T21:21:19.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.671+0000 7f7e97220740 1 -- 172.21.15.86:0/3951625278 shutdown_connections 2024-04-20T21:21:19.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.671+0000 7f7e97220740 1 --2- 172.21.15.86:0/3951625278 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c5bdea2940 0x55c5bded2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.671+0000 7f7e97220740 1 --2- 172.21.15.86:0/3951625278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bdecf840 0x55c5bded21f0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.671+0000 7f7e97220740 1 --2- 172.21.15.86:0/3951625278 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c5bdecf460 0x55c5bdea2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.671+0000 7f7e97220740 1 -- 172.21.15.86:0/3951625278 >> 172.21.15.86:0/3951625278 conn(0x55c5bdea4670 msgr2=0x55c5bdeed1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:19.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.671+0000 7f7e97220740 1 -- 172.21.15.86:0/3951625278 shutdown_connections 2024-04-20T21:21:19.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.671+0000 7f7e97220740 1 -- 172.21.15.86:0/3951625278 wait complete. 2024-04-20T21:21:19.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.672+0000 7f7e97220740 1 -- start start 2024-04-20T21:21:19.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.672+0000 7f7e97220740 1 Processor -- start 2024-04-20T21:21:19.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.673+0000 7f7e97220740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c5bdecf460 0x55c5bdee7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.673+0000 7f7e97220740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bdecf840 0x55c5bded4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.673+0000 7f7e97220740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c5bdec3100 0x55c5bdec3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.673+0000 7f7e97220740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c5bdea6c10 con 0x55c5bdecf840 2024-04-20T21:21:19.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.673+0000 7f7e97220740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c5bddc65e0 con 0x55c5bdec3100 2024-04-20T21:21:19.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.673+0000 7f7e97220740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55c5bdea6000 con 0x55c5bdecf460 2024-04-20T21:21:19.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.673+0000 7f7e97220740 10 client.0 _ll_register_callbacks cb 0x55c5bdd94fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:21:19.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.673+0000 7f7e91000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bdecf840 0x55c5bded4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:19.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.673+0000 7f7e91000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bdecf840 0x55c5bded4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58700/0 (socket says 172.21.15.86:58700) 2024-04-20T21:21:19.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.673+0000 7f7e91000640 1 -- 172.21.15.86:0/4125966363 learned_addr learned my addr 172.21.15.86:0/4125966363 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:19.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.674+0000 7f7e91000640 1 -- 172.21.15.86:0/4125966363 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c5bdecf460 msgr2=0x55c5bdee7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:19.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.674+0000 7f7e91000640 1 --2- 172.21.15.86:0/4125966363 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c5bdecf460 0x55c5bdee7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.674+0000 7f7e91000640 1 -- 172.21.15.86:0/4125966363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c5bdec3100 msgr2=0x55c5bdec3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:19.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.674+0000 7f7e91000640 1 --2- 172.21.15.86:0/4125966363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c5bdec3100 0x55c5bdec3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.674+0000 7f7e91000640 1 -- 172.21.15.86:0/4125966363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5bdbeb7b0 con 0x55c5bdecf840 2024-04-20T21:21:19.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.674+0000 7f7e91000640 1 --2- 172.21.15.86:0/4125966363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bdecf840 0x55c5bded4ad0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f7e78009990 tx=0x7f7e78004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55cb1e8abd5953c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:19.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.674+0000 7f7e72a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:21:19.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.674+0000 7f7e72a00640 1 -- 172.21.15.86:0/4125966363 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e78034050 con 0x55c5bdecf840 2024-04-20T21:21:19.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.675+0000 7f7e97220740 1 -- 172.21.15.86:0/4125966363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e78049070 con 0x55c5bdecf840 2024-04-20T21:21:19.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.675+0000 7f7e72a00640 1 -- 172.21.15.86:0/4125966363 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f7e78003810 con 0x55c5bdecf840 2024-04-20T21:21:19.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.675+0000 7f7e72a00640 1 -- 172.21.15.86:0/4125966363 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7e78009490 con 0x55c5bdecf840 2024-04-20T21:21:19.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.675+0000 7f7e97220740 10 client.6342 fetch_fsmap learned FSMap version 32 2024-04-20T21:21:19.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.675+0000 7f7e97220740 1 -- 172.21.15.86:0/4125966363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) -- 0x55c5bdc4bf40 con 0x55c5bdecf840 2024-04-20T21:21:19.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.675+0000 7f7e72a00640 1 -- 172.21.15.86:0/4125966363 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e78049070 con 0x55c5bdecf840 2024-04-20T21:21:19.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.675+0000 7f7e72a00640 1 -- 172.21.15.86:0/4125966363 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 3495+0+0 (secure 0 0 0) 0x7f7e7805e080 con 0x55c5bdecf840 2024-04-20T21:21:19.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.675+0000 7f7e97220740 10 client.6342 fetch_fsmap finished waiting for FSMap version 32 2024-04-20T21:21:19.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.675+0000 7f7e97220740 10 client.6342 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-20T21:21:19.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.675+0000 7f7e97220740 20 client.6342 populate_metadata read hostname 'smithi086' 2024-04-20T21:21:19.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.675+0000 7f7e97220740 1 --2- 172.21.15.86:0/4125966363 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x55c5bdebd240 0x55c5bdebfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.675+0000 7f7e97220740 4 client.6342 mds_command: new command op to 6135 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:21:19.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.675+0000 7f7e97220740 1 -- 172.21.15.86:0/4125966363 --> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5bde1a770 con 0x55c5bdebd240 2024-04-20T21:21:19.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.676+0000 7f7e91a00640 1 --2- 172.21.15.86:0/4125966363 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x55c5bdebd240 0x55c5bdebfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:19.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.676+0000 7f7e91a00640 1 --2- 172.21.15.86:0/4125966363 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x55c5bdebd240 0x55c5bdebfc80 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ced9db2f4628e11 server_cookie=19c1732eefdcb4a8 in_seq=0 out_seq=0 2024-04-20T21:21:19.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.676+0000 7f7e72a00640 10 client.6342 ms_handle_connect on v2:172.21.15.86:6835/3008383261 2024-04-20T21:21:19.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.677+0000 7f7e72a00640 1 -- 172.21.15.86:0/4125966363 <== mds.0 v2:172.21.15.86:6835/3008383261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c5bde1a770 con 0x55c5bdebd240 2024-04-20T21:21:19.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.677+0000 7f7e72a00640 10 client.6342 handle_command_reply: tid=0 2024-04-20T21:21:19.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.677+0000 7f7e97220740 1 client.6342 shutdown 2024-04-20T21:21:19.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.677+0000 7f7e97220740 1 -- 172.21.15.86:0/4125966363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bdecf840 msgr2=0x55c5bded4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:19.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.677+0000 7f7e97220740 1 --2- 172.21.15.86:0/4125966363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bdecf840 0x55c5bded4ad0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f7e78009990 tx=0x7f7e78004730 comp rx=0 tx=0).stop 2024-04-20T21:21:19.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.677+0000 7f7e97220740 1 -- 172.21.15.86:0/4125966363 shutdown_connections 2024-04-20T21:21:19.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.677+0000 7f7e97220740 1 --2- 172.21.15.86:0/4125966363 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x55c5bdebd240 0x55c5bdebfc80 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.677+0000 7f7e97220740 1 --2- 172.21.15.86:0/4125966363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c5bdec3100 0x55c5bdec3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.677+0000 7f7e97220740 1 --2- 172.21.15.86:0/4125966363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bdecf840 0x55c5bded4ad0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.677+0000 7f7e97220740 1 --2- 172.21.15.86:0/4125966363 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c5bdecf460 0x55c5bdee7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.677+0000 7f7e72a00640 0 client.6342 ms_handle_reset on v2:172.21.15.86:6835/3008383261 2024-04-20T21:21:19.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.677+0000 7f7e97220740 1 -- 172.21.15.86:0/4125966363 >> 172.21.15.86:0/4125966363 conn(0x55c5bdea2940 msgr2=0x55c5bded21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:19.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.678+0000 7f7e97220740 1 -- 172.21.15.86:0/4125966363 shutdown_connections 2024-04-20T21:21:19.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.678+0000 7f7e97220740 1 -- 172.21.15.86:0/4125966363 wait complete. 2024-04-20T21:21:19.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.678+0000 7f7e97220740 20 client.6342 trim_cache size 0 max 16384 2024-04-20T21:21:19.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.697+0000 7f7e97220740 1 Processor -- start 2024-04-20T21:21:19.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.697+0000 7f7e97220740 1 -- start start 2024-04-20T21:21:19.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.698+0000 7f7e97220740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c5bdea2940 0x55c5bdecdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.698+0000 7f7e97220740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c5bdec3100 0x55c5bded48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.698+0000 7f7e97220740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bdecf460 0x55c5bded4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.698+0000 7f7e97220740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c200bf0 con 0x55c5bdecf460 2024-04-20T21:21:19.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.698+0000 7f7e97220740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c1ffd40 con 0x55c5bdec3100 2024-04-20T21:21:19.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.698+0000 7f7e97220740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7e8c1f7c60 con 0x55c5bdea2940 2024-04-20T21:21:19.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.698+0000 7f7e92400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bdecf460 0x55c5bded4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:19.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.698+0000 7f7e92400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bdecf460 0x55c5bded4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58702/0 (socket says 172.21.15.86:58702) 2024-04-20T21:21:19.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.698+0000 7f7e92400640 1 -- 172.21.15.86:0/1602891654 learned_addr learned my addr 172.21.15.86:0/1602891654 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:19.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.699+0000 7f7e92400640 1 -- 172.21.15.86:0/1602891654 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c5bdea2940 msgr2=0x55c5bdecdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:19.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.699+0000 7f7e92400640 1 --2- 172.21.15.86:0/1602891654 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c5bdea2940 0x55c5bdecdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.699+0000 7f7e92400640 1 -- 172.21.15.86:0/1602891654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c5bdec3100 msgr2=0x55c5bded48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:19.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.699+0000 7f7e92400640 1 --2- 172.21.15.86:0/1602891654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c5bdec3100 0x55c5bded48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.699+0000 7f7e92400640 1 -- 172.21.15.86:0/1602891654 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5bdec38f0 con 0x55c5bdecf460 2024-04-20T21:21:19.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.699+0000 7f7e92400640 1 --2- 172.21.15.86:0/1602891654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bdecf460 0x55c5bded4de0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f7e84058b10 tx=0x7f7e84058ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b8cf689b8e85037 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:19.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.699+0000 7f7e7d600640 1 -- 172.21.15.86:0/1602891654 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e84087070 con 0x55c5bdecf460 2024-04-20T21:21:19.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.699+0000 7f7e7d600640 1 -- 172.21.15.86:0/1602891654 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e84061940 con 0x55c5bdecf460 2024-04-20T21:21:19.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.699+0000 7f7e7d600640 1 -- 172.21.15.86:0/1602891654 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e84071070 con 0x55c5bdecf460 2024-04-20T21:21:19.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.700+0000 7f7e97220740 1 -- 172.21.15.86:0/1602891654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bdecf460 msgr2=0x55c5bded4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:19.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.700+0000 7f7e97220740 1 --2- 172.21.15.86:0/1602891654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bdecf460 0x55c5bded4de0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f7e84058b10 tx=0x7f7e84058ed0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.700+0000 7f7e97220740 1 -- 172.21.15.86:0/1602891654 shutdown_connections 2024-04-20T21:21:19.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.700+0000 7f7e97220740 1 --2- 172.21.15.86:0/1602891654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bdecf460 0x55c5bded4de0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.700+0000 7f7e97220740 1 --2- 172.21.15.86:0/1602891654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c5bdec3100 0x55c5bded48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.700+0000 7f7e97220740 1 --2- 172.21.15.86:0/1602891654 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c5bdea2940 0x55c5bdecdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.700+0000 7f7e97220740 1 -- 172.21.15.86:0/1602891654 >> 172.21.15.86:0/1602891654 conn(0x55c5bdee8610 msgr2=0x55c5bdee8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:19.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.700+0000 7f7e97220740 1 -- 172.21.15.86:0/1602891654 shutdown_connections 2024-04-20T21:21:19.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.700+0000 7f7e97220740 1 -- 172.21.15.86:0/1602891654 wait complete. 2024-04-20T21:21:19.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.701+0000 7f7e97220740 1 -- start start 2024-04-20T21:21:19.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.701+0000 7f7e97220740 1 Processor -- start 2024-04-20T21:21:19.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.702+0000 7f7e97220740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c5bdea2940 0x55c5bdebbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.702+0000 7f7e97220740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bdec3100 0x55c5bdebc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.702+0000 7f7e97220740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c5bdecf460 0x55c5bded1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.702+0000 7f7e97220740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c5bdea9830 con 0x55c5bdec3100 2024-04-20T21:21:19.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.702+0000 7f7e97220740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c1f82b0 con 0x55c5bdecf460 2024-04-20T21:21:19.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.702+0000 7f7e97220740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55c5bdecef90 con 0x55c5bdea2940 2024-04-20T21:21:19.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.702+0000 7f7e97220740 10 client.0 _ll_register_callbacks cb 0x7f7e78030dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:21:19.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.702+0000 7f7e91000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bdec3100 0x55c5bdebc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:19.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.702+0000 7f7e91000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bdec3100 0x55c5bdebc420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58708/0 (socket says 172.21.15.86:58708) 2024-04-20T21:21:19.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.702+0000 7f7e91000640 1 -- 172.21.15.86:0/805655259 learned_addr learned my addr 172.21.15.86:0/805655259 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:19.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.703+0000 7f7e92400640 1 --2- 172.21.15.86:0/805655259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c5bdecf460 0x55c5bded1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:19.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.703+0000 7f7e91a00640 1 --2- 172.21.15.86:0/805655259 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c5bdea2940 0x55c5bdebbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:19.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.703+0000 7f7e91000640 1 -- 172.21.15.86:0/805655259 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c5bdea2940 msgr2=0x55c5bdebbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:19.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.703+0000 7f7e91000640 1 --2- 172.21.15.86:0/805655259 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c5bdea2940 0x55c5bdebbee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.703+0000 7f7e91000640 1 -- 172.21.15.86:0/805655259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c5bdecf460 msgr2=0x55c5bded1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:19.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.703+0000 7f7e91000640 1 --2- 172.21.15.86:0/805655259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c5bdecf460 0x55c5bded1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.703+0000 7f7e91000640 1 -- 172.21.15.86:0/805655259 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5bdadf0a0 con 0x55c5bdec3100 2024-04-20T21:21:19.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.703+0000 7f7e91000640 1 --2- 172.21.15.86:0/805655259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bdec3100 0x55c5bdebc420 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f7e78009870 tx=0x7f7e780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=681729ee7128dd75 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:19.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.703+0000 7f7e72a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:21:19.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.703+0000 7f7e72a00640 1 -- 172.21.15.86:0/805655259 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e78000e60 con 0x55c5bdec3100 2024-04-20T21:21:19.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.703+0000 7f7e72a00640 1 -- 172.21.15.86:0/805655259 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f7e78003cf0 con 0x55c5bdec3100 2024-04-20T21:21:19.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.703+0000 7f7e97220740 1 -- 172.21.15.86:0/805655259 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e84061940 con 0x55c5bdec3100 2024-04-20T21:21:19.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.703+0000 7f7e72a00640 1 -- 172.21.15.86:0/805655259 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7e78069050 con 0x55c5bdec3100 2024-04-20T21:21:19.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.704+0000 7f7e97220740 10 client.6348 fetch_fsmap learned FSMap version 32 2024-04-20T21:21:19.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.704+0000 7f7e97220740 1 -- 172.21.15.86:0/805655259 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) -- 0x55c5bdefceb0 con 0x55c5bdec3100 2024-04-20T21:21:19.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.704+0000 7f7e72a00640 1 -- 172.21.15.86:0/805655259 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e84061940 con 0x55c5bdec3100 2024-04-20T21:21:19.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.704+0000 7f7e72a00640 1 -- 172.21.15.86:0/805655259 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 3495+0+0 (secure 0 0 0) 0x7f7e7806c020 con 0x55c5bdec3100 2024-04-20T21:21:19.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.704+0000 7f7e97220740 10 client.6348 fetch_fsmap finished waiting for FSMap version 32 2024-04-20T21:21:19.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.704+0000 7f7e97220740 10 client.6348 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-20T21:21:19.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.704+0000 7f7e97220740 20 client.6348 populate_metadata read hostname 'smithi086' 2024-04-20T21:21:19.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.704+0000 7f7e97220740 1 --2- 172.21.15.86:0/805655259 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x55c5bdecf840 0x55c5bded2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.704+0000 7f7e97220740 4 client.6348 mds_command: new command op to 6135 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}] 2024-04-20T21:21:19.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.704+0000 7f7e97220740 1 -- 172.21.15.86:0/805655259 --> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}) -- 0x55c5bdee9640 con 0x55c5bdecf840 2024-04-20T21:21:19.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.704+0000 7f7e91a00640 1 --2- 172.21.15.86:0/805655259 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x55c5bdecf840 0x55c5bded2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:19.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.705+0000 7f7e91a00640 1 --2- 172.21.15.86:0/805655259 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x55c5bdecf840 0x55c5bded2360 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c8b803182eebe84 server_cookie=48c918fef1ed07f8 in_seq=0 out_seq=0 2024-04-20T21:21:19.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.705+0000 7f7e72a00640 10 client.6348 ms_handle_connect on v2:172.21.15.86:6835/3008383261 2024-04-20T21:21:19.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.705+0000 7f7e72a00640 1 -- 172.21.15.86:0/805655259 <== mds.0 v2:172.21.15.86:6835/3008383261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c5bdee9640 con 0x55c5bdecf840 2024-04-20T21:21:19.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.705+0000 7f7e72a00640 10 client.6348 handle_command_reply: tid=0 2024-04-20T21:21:19.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.705+0000 7f7e97220740 1 client.6348 shutdown 2024-04-20T21:21:19.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.706+0000 7f7e97220740 1 -- 172.21.15.86:0/805655259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bdec3100 msgr2=0x55c5bdebc420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:19.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.706+0000 7f7e97220740 1 --2- 172.21.15.86:0/805655259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bdec3100 0x55c5bdebc420 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f7e78009870 tx=0x7f7e780046c0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.706+0000 7f7e97220740 1 -- 172.21.15.86:0/805655259 shutdown_connections 2024-04-20T21:21:19.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.706+0000 7f7e97220740 1 --2- 172.21.15.86:0/805655259 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x55c5bdecf840 0x55c5bded2360 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.706+0000 7f7e97220740 1 --2- 172.21.15.86:0/805655259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c5bdecf460 0x55c5bded1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.706+0000 7f7e97220740 1 --2- 172.21.15.86:0/805655259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c5bdec3100 0x55c5bdebc420 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.706+0000 7f7e97220740 1 --2- 172.21.15.86:0/805655259 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c5bdea2940 0x55c5bdebbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.706+0000 7f7e97220740 1 -- 172.21.15.86:0/805655259 >> 172.21.15.86:0/805655259 conn(0x55c5bdee8610 msgr2=0x55c5bdea8240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:19.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.706+0000 7f7e72a00640 0 client.6348 ms_handle_reset on v2:172.21.15.86:6835/3008383261 2024-04-20T21:21:19.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.706+0000 7f7e97220740 1 -- 172.21.15.86:0/805655259 shutdown_connections 2024-04-20T21:21:19.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.706+0000 7f7e97220740 1 -- 172.21.15.86:0/805655259 wait complete. 2024-04-20T21:21:19.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.706+0000 7f7e97220740 20 client.6348 trim_cache size 0 max 16384 2024-04-20T21:21:19.709 INFO:tasks.ceph.mds.c.smithi086.stderr:2024-04-20T21:21:19.706+0000 7fb60f600640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={6306=0-4194304@1} caps={6306=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6306 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x559d2d306b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-20T21:21:19.709 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:21:19.709 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-20T21:21:19.709 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "9671ecfe-2c5d-450c-a0be-b56b83d19bde", 2024-04-20T21:21:19.709 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-20T21:21:19.709 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:21:19.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.707+0000 7f7e92e00640 1 -- 172.21.15.86:0/1519941358 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7e6c06c8f0 msgr2=0x7f7e6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:19.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.707+0000 7f7e92e00640 1 --2- 172.21.15.86:0/1519941358 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7e6c06c8f0 0x7f7e6c06edb0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7e80009a10 tx=0x7f7e80008040 comp rx=0 tx=0).stop 2024-04-20T21:21:19.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.707+0000 7f7e92e00640 1 -- 172.21.15.86:0/1519941358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e8c15c9f0 msgr2=0x7f7e8c1537d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:19.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.707+0000 7f7e92e00640 1 --2- 172.21.15.86:0/1519941358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e8c15c9f0 0x7f7e8c1537d0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f7e8400c730 tx=0x7f7e8400cbf0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.707+0000 7f7e92e00640 1 -- 172.21.15.86:0/1519941358 shutdown_connections 2024-04-20T21:21:19.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.707+0000 7f7e92e00640 1 --2- 172.21.15.86:0/1519941358 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7e6c06c8f0 0x7f7e6c06edb0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.707+0000 7f7e92e00640 1 --2- 172.21.15.86:0/1519941358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e8c15c9f0 0x7f7e8c1537d0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.707+0000 7f7e92e00640 1 --2- 172.21.15.86:0/1519941358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e8c1569d0 0x7f7e8c152380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.707+0000 7f7e92e00640 1 --2- 172.21.15.86:0/1519941358 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e8c06e700 0x7f7e8c1464a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.707+0000 7f7e92e00640 1 -- 172.21.15.86:0/1519941358 >> 172.21.15.86:0/1519941358 conn(0x7f7e8c0b2220 msgr2=0x7f7e8c0ad370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:19.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.707+0000 7f7e92e00640 1 -- 172.21.15.86:0/1519941358 shutdown_connections 2024-04-20T21:21:19.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.707+0000 7f7e92e00640 1 -- 172.21.15.86:0/1519941358 wait complete. 2024-04-20T21:21:19.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:21:19.723 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-20T21:21:19.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.914+0000 7fd0aa600640 1 Processor -- start 2024-04-20T21:21:19.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.914+0000 7fd0aa600640 1 -- start start 2024-04-20T21:21:19.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.916+0000 7fd0aa600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd0a415f990 0x7fd0a406dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.916+0000 7fd0aa600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd0a4071d40 0x7fd0a4072190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.916+0000 7fd0aa600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0a40aecd0 con 0x7fd0a4161c60 2024-04-20T21:21:19.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.916+0000 7fd0aa600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd0a40726d0 con 0x7fd0a415f990 2024-04-20T21:21:19.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.916+0000 7fd0aa600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fd0a406e2e0 con 0x7fd0a4071d40 2024-04-20T21:21:19.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.916+0000 7fd0a9200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd0a4161c60 0x7fd0a415ff10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:42472/0 (socket says 172.21.15.86:42472) 2024-04-20T21:21:19.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.916+0000 7fd0a9200640 1 -- 172.21.15.86:0/106387587 learned_addr learned my addr 172.21.15.86:0/106387587 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:21:19.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.917+0000 7fd09be00640 1 --2- 172.21.15.86:0/106387587 >> v2:172.21.15.86:3300/0 conn(0x7fd0a415f990 0x7fd0a406dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:19.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.917+0000 7fd0a9c00640 1 --2- 172.21.15.86:0/106387587 >> v2:172.21.15.144:3300/0 conn(0x7fd0a4071d40 0x7fd0a4072190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:19.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.917+0000 7fd0a3e00640 1 -- 172.21.15.86:0/106387587 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3918271782 0 0) 0x7fd0a40aecd0 con 0x7fd0a4161c60 2024-04-20T21:21:19.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.917+0000 7fd0a3e00640 1 -- 172.21.15.86:0/106387587 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd084003520 con 0x7fd0a4161c60 2024-04-20T21:21:19.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.917+0000 7fd09be00640 1 -- 172.21.15.86:0/106387587 >> v2:172.21.15.144:3300/0 conn(0x7fd0a4071d40 msgr2=0x7fd0a4072190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:19.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.917+0000 7fd09be00640 1 --2- 172.21.15.86:0/106387587 >> v2:172.21.15.144:3300/0 conn(0x7fd0a4071d40 0x7fd0a4072190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.917+0000 7fd09be00640 1 -- 172.21.15.86:0/106387587 >> v1:172.21.15.86:6789/0 conn(0x7fd0a4161c60 legacy=0x7fd0a415ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:19.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.917+0000 7fd09be00640 1 -- 172.21.15.86:0/106387587 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a406e580 con 0x7fd0a415f990 2024-04-20T21:21:19.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.919+0000 7fd09be00640 1 --2- 172.21.15.86:0/106387587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0a415f990 0x7fd0a406dda0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fd0a4071b90 tx=0x7fd09402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40079144618711ee server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:19.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.919+0000 7fd0a3e00640 1 -- 172.21.15.86:0/106387587 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd094035020 con 0x7fd0a415f990 2024-04-20T21:21:19.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.919+0000 7fd0a3e00640 1 -- 172.21.15.86:0/106387587 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd094031830 con 0x7fd0a415f990 2024-04-20T21:21:19.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.919+0000 7fd0a3e00640 1 -- 172.21.15.86:0/106387587 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd094031be0 con 0x7fd0a415f990 2024-04-20T21:21:19.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.919+0000 7fd0aa600640 1 -- 172.21.15.86:0/106387587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0a415f990 msgr2=0x7fd0a406dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:19.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.919+0000 7fd0aa600640 1 --2- 172.21.15.86:0/106387587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0a415f990 0x7fd0a406dda0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fd0a4071b90 tx=0x7fd09402b8d0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.920+0000 7fd0aa600640 1 -- 172.21.15.86:0/106387587 shutdown_connections 2024-04-20T21:21:19.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.920+0000 7fd0aa600640 1 --2- 172.21.15.86:0/106387587 >> v2:172.21.15.144:3300/0 conn(0x7fd0a4071d40 0x7fd0a4072190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.920+0000 7fd0aa600640 1 --2- 172.21.15.86:0/106387587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0a415f990 0x7fd0a406dda0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.920+0000 7fd0aa600640 1 -- 172.21.15.86:0/106387587 >> 172.21.15.86:0/106387587 conn(0x7fd0a4067cf0 msgr2=0x7fd0a406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:19.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.920+0000 7fd0aa600640 1 -- 172.21.15.86:0/106387587 shutdown_connections 2024-04-20T21:21:19.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.920+0000 7fd0aa600640 1 -- 172.21.15.86:0/106387587 wait complete. 2024-04-20T21:21:19.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.921+0000 7fd0aa600640 1 Processor -- start 2024-04-20T21:21:19.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.921+0000 7fd0aa600640 1 -- start start 2024-04-20T21:21:19.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.922+0000 7fd0aa600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0a4071d40 0x7fd0a415dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.922+0000 7fd0aa600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0a415f990 0x7fd0a413bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.922+0000 7fd0aa600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0a4161c60 0x7fd0a415b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.922+0000 7fd0aa600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a406ee20 con 0x7fd0a4161c60 2024-04-20T21:21:19.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.922+0000 7fd0aa600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a406eb40 con 0x7fd0a4071d40 2024-04-20T21:21:19.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.922+0000 7fd0aa600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd0a4162040 con 0x7fd0a415f990 2024-04-20T21:21:19.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.923+0000 7fd0a9c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0a4161c60 0x7fd0a415b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:19.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.923+0000 7fd0a9c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0a4161c60 0x7fd0a415b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58728/0 (socket says 172.21.15.86:58728) 2024-04-20T21:21:19.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.923+0000 7fd0a9c00640 1 -- 172.21.15.86:0/2474414912 learned_addr learned my addr 172.21.15.86:0/2474414912 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:19.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.923+0000 7fd0a9200640 1 --2- 172.21.15.86:0/2474414912 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0a4071d40 0x7fd0a415dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:19.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.923+0000 7fd0a9c00640 1 -- 172.21.15.86:0/2474414912 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0a415f990 msgr2=0x7fd0a413bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:19.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.923+0000 7fd09be00640 1 --2- 172.21.15.86:0/2474414912 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0a415f990 0x7fd0a413bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:19.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.923+0000 7fd0a9c00640 1 --2- 172.21.15.86:0/2474414912 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0a415f990 0x7fd0a413bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.923+0000 7fd0a9c00640 1 -- 172.21.15.86:0/2474414912 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0a4071d40 msgr2=0x7fd0a415dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:19.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.923+0000 7fd0a9c00640 1 --2- 172.21.15.86:0/2474414912 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0a4071d40 0x7fd0a415dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.923+0000 7fd0a9c00640 1 -- 172.21.15.86:0/2474414912 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a40cf4c0 con 0x7fd0a4161c60 2024-04-20T21:21:19.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.923+0000 7fd0a9200640 1 --2- 172.21.15.86:0/2474414912 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0a4071d40 0x7fd0a415dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:21:19.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.924+0000 7fd0a9c00640 1 --2- 172.21.15.86:0/2474414912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0a4161c60 0x7fd0a415b150 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fd09c00efc0 tx=0x7fd09c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:19.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.924+0000 7fd0a1600640 1 -- 172.21.15.86:0/2474414912 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09c018020 con 0x7fd0a4161c60 2024-04-20T21:21:19.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.924+0000 7fd0a1600640 1 -- 172.21.15.86:0/2474414912 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd09c014480 con 0x7fd0a4161c60 2024-04-20T21:21:19.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.924+0000 7fd0a1600640 1 -- 172.21.15.86:0/2474414912 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09c014870 con 0x7fd0a4161c60 2024-04-20T21:21:19.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.924+0000 7fd0aa600640 1 -- 172.21.15.86:0/2474414912 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0a415a670 con 0x7fd0a4161c60 2024-04-20T21:21:19.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.924+0000 7fd0aa600640 1 -- 172.21.15.86:0/2474414912 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0a415ac30 con 0x7fd0a4161c60 2024-04-20T21:21:19.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.927+0000 7fd0a1600640 1 -- 172.21.15.86:0/2474414912 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd09c005d10 con 0x7fd0a4161c60 2024-04-20T21:21:19.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.928+0000 7fd0a1600640 1 --2- 172.21.15.86:0/2474414912 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd08006c8f0 0x7fd08006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.928+0000 7fd0a1600640 1 -- 172.21.15.86:0/2474414912 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fd09c08c660 con 0x7fd0a4161c60 2024-04-20T21:21:19.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.929+0000 7fd0a9200640 1 --2- 172.21.15.86:0/2474414912 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd08006c8f0 0x7fd08006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:19.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.929+0000 7fd0ae84a740 1 Processor -- start 2024-04-20T21:21:19.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.929+0000 7fd0ae84a740 1 -- start start 2024-04-20T21:21:19.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.929+0000 7fd0a9200640 1 --2- 172.21.15.86:0/2474414912 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd08006c8f0 0x7fd08006edb0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd08c000f60 tx=0x7fd08c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:19.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.930+0000 7fd0ae84a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ab5d743d0 0x555ab5d747b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.930+0000 7fd0ae84a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ab5d443f0 0x555ab5d71640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.930+0000 7fd0ae84a740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555ab5d47f50 0x555ab5d71b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.930+0000 7fd0ae84a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555ab5cb0150 con 0x555ab5d443f0 2024-04-20T21:21:19.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.930+0000 7fd0ae84a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555ab5c56370 con 0x555ab5d743d0 2024-04-20T21:21:19.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.930+0000 7fd0ae84a740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x555ab5cac110 con 0x555ab5d47f50 2024-04-20T21:21:19.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.930+0000 7fd09be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ab5d443f0 0x555ab5d71640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:19.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.930+0000 7fd09be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ab5d443f0 0x555ab5d71640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58736/0 (socket says 172.21.15.86:58736) 2024-04-20T21:21:19.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.930+0000 7fd09be00640 1 -- 172.21.15.86:0/2493056015 learned_addr learned my addr 172.21.15.86:0/2493056015 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:19.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.931+0000 7fd0a9200640 1 --2- 172.21.15.86:0/2493056015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ab5d743d0 0x555ab5d747b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:19.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.931+0000 7fd0a9c00640 1 --2- 172.21.15.86:0/2493056015 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555ab5d47f50 0x555ab5d71b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:19.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.931+0000 7fd09be00640 1 -- 172.21.15.86:0/2493056015 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555ab5d47f50 msgr2=0x555ab5d71b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:19.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.931+0000 7fd09be00640 1 --2- 172.21.15.86:0/2493056015 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555ab5d47f50 0x555ab5d71b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.931+0000 7fd09be00640 1 -- 172.21.15.86:0/2493056015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ab5d743d0 msgr2=0x555ab5d747b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:19.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.931+0000 7fd09be00640 1 --2- 172.21.15.86:0/2493056015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ab5d743d0 0x555ab5d747b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.931+0000 7fd09be00640 1 -- 172.21.15.86:0/2493056015 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ab5a8c7d0 con 0x555ab5d443f0 2024-04-20T21:21:19.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.931+0000 7fd09be00640 1 --2- 172.21.15.86:0/2493056015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ab5d443f0 0x555ab5d71640 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fd09402b910 tx=0x7fd094030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23b3d2b5d6ce70e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:19.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.931+0000 7fd099600640 1 -- 172.21.15.86:0/2493056015 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd094035040 con 0x555ab5d443f0 2024-04-20T21:21:19.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.932+0000 7fd099600640 1 -- 172.21.15.86:0/2493056015 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd094031730 con 0x555ab5d443f0 2024-04-20T21:21:19.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.932+0000 7fd099600640 1 -- 172.21.15.86:0/2493056015 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd094031b20 con 0x555ab5d443f0 2024-04-20T21:21:19.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.932+0000 7fd0ae84a740 1 -- 172.21.15.86:0/2493056015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ab5d443f0 msgr2=0x555ab5d71640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:19.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.932+0000 7fd0ae84a740 1 --2- 172.21.15.86:0/2493056015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ab5d443f0 0x555ab5d71640 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fd09402b910 tx=0x7fd094030000 comp rx=0 tx=0).stop 2024-04-20T21:21:19.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.932+0000 7fd0ae84a740 1 -- 172.21.15.86:0/2493056015 shutdown_connections 2024-04-20T21:21:19.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.932+0000 7fd0ae84a740 1 --2- 172.21.15.86:0/2493056015 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555ab5d47f50 0x555ab5d71b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.932+0000 7fd0ae84a740 1 --2- 172.21.15.86:0/2493056015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ab5d443f0 0x555ab5d71640 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.932+0000 7fd0ae84a740 1 --2- 172.21.15.86:0/2493056015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ab5d743d0 0x555ab5d747b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.932+0000 7fd0ae84a740 1 -- 172.21.15.86:0/2493056015 >> 172.21.15.86:0/2493056015 conn(0x555ab5d46660 msgr2=0x555ab5d8f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:19.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.932+0000 7fd0ae84a740 1 -- 172.21.15.86:0/2493056015 shutdown_connections 2024-04-20T21:21:19.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.932+0000 7fd0ae84a740 1 -- 172.21.15.86:0/2493056015 wait complete. 2024-04-20T21:21:19.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.933+0000 7fd0ae84a740 1 -- start start 2024-04-20T21:21:19.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.933+0000 7fd0ae84a740 1 Processor -- start 2024-04-20T21:21:19.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.934+0000 7fd0ae84a740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555ab5d47f50 0x555ab5d75130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.934+0000 7fd0ae84a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ab5d743d0 0x555ab5d74840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.934+0000 7fd0ae84a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ab5d5b9f0 0x555ab5d76c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.934+0000 7fd0ae84a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555ab5d49100 con 0x555ab5d743d0 2024-04-20T21:21:19.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.934+0000 7fd0ae84a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555ab5c69740 con 0x555ab5d5b9f0 2024-04-20T21:21:19.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.934+0000 7fd0ae84a740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x555ab5d484f0 con 0x555ab5d47f50 2024-04-20T21:21:19.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.934+0000 7fd0ae84a740 10 client.0 _ll_register_callbacks cb 0x555ab5c64b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:21:19.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.935+0000 7fd09be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ab5d743d0 0x555ab5d74840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:19.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.935+0000 7fd09be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ab5d743d0 0x555ab5d74840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58750/0 (socket says 172.21.15.86:58750) 2024-04-20T21:21:19.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.935+0000 7fd09be00640 1 -- 172.21.15.86:0/2357032649 learned_addr learned my addr 172.21.15.86:0/2357032649 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:19.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.935+0000 7fd0a9200640 1 --2- 172.21.15.86:0/2357032649 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555ab5d47f50 0x555ab5d75130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:19.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.935+0000 7fd09be00640 1 -- 172.21.15.86:0/2357032649 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555ab5d47f50 msgr2=0x555ab5d75130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:19.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.935+0000 7fd09be00640 1 --2- 172.21.15.86:0/2357032649 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555ab5d47f50 0x555ab5d75130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.935+0000 7fd09be00640 1 -- 172.21.15.86:0/2357032649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ab5d5b9f0 msgr2=0x555ab5d76c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:19.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.935+0000 7fd0a9c00640 1 --2- 172.21.15.86:0/2357032649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ab5d5b9f0 0x555ab5d76c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:19.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.935+0000 7fd09be00640 1 --2- 172.21.15.86:0/2357032649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ab5d5b9f0 0x555ab5d76c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.935+0000 7fd09be00640 1 -- 172.21.15.86:0/2357032649 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ab59c23f0 con 0x555ab5d743d0 2024-04-20T21:21:19.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.935+0000 7fd09be00640 1 --2- 172.21.15.86:0/2357032649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ab5d743d0 0x555ab5d74840 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fd094009890 tx=0x7fd094004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7404471c39a1e2fd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:19.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.935+0000 7fd092000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:21:19.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.936+0000 7fd092000640 1 -- 172.21.15.86:0/2357032649 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd094052060 con 0x555ab5d743d0 2024-04-20T21:21:19.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.936+0000 7fd092000640 1 -- 172.21.15.86:0/2357032649 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fd0940318e0 con 0x555ab5d743d0 2024-04-20T21:21:19.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.936+0000 7fd0ae84a740 1 -- 172.21.15.86:0/2357032649 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd094031730 con 0x555ab5d743d0 2024-04-20T21:21:19.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.936+0000 7fd092000640 1 -- 172.21.15.86:0/2357032649 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd094035040 con 0x555ab5d743d0 2024-04-20T21:21:19.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.936+0000 7fd0ae84a740 10 client.6363 fetch_fsmap learned FSMap version 32 2024-04-20T21:21:19.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.936+0000 7fd0ae84a740 1 -- 172.21.15.86:0/2357032649 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) -- 0x555ab59820f0 con 0x555ab5d743d0 2024-04-20T21:21:19.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.936+0000 7fd092000640 1 -- 172.21.15.86:0/2357032649 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd094031730 con 0x555ab5d743d0 2024-04-20T21:21:19.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.936+0000 7fd092000640 1 -- 172.21.15.86:0/2357032649 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 3495+0+0 (secure 0 0 0) 0x7fd094031ae0 con 0x555ab5d743d0 2024-04-20T21:21:19.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.936+0000 7fd0ae84a740 10 client.6363 fetch_fsmap finished waiting for FSMap version 32 2024-04-20T21:21:19.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.936+0000 7fd0ae84a740 10 client.6363 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-20T21:21:19.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.936+0000 7fd0ae84a740 20 client.6363 populate_metadata read hostname 'smithi086' 2024-04-20T21:21:19.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.936+0000 7fd0ae84a740 1 --2- 172.21.15.86:0/2357032649 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x555ab5d6b690 0x555ab5d6baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.936+0000 7fd0ae84a740 4 client.6363 mds_command: new command op to 6135 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:21:19.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.936+0000 7fd0ae84a740 1 -- 172.21.15.86:0/2357032649 --> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ab5cbdb70 con 0x555ab5d6b690 2024-04-20T21:21:19.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.937+0000 7fd0a9200640 1 --2- 172.21.15.86:0/2357032649 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x555ab5d6b690 0x555ab5d6baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:19.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.937+0000 7fd0a9200640 1 --2- 172.21.15.86:0/2357032649 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x555ab5d6b690 0x555ab5d6baa0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9e35496100307d1 server_cookie=45f92263369de7fd in_seq=0 out_seq=0 2024-04-20T21:21:19.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.937+0000 7fd092000640 10 client.6363 ms_handle_connect on v2:172.21.15.86:6835/3008383261 2024-04-20T21:21:19.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.938+0000 7fd092000640 1 -- 172.21.15.86:0/2357032649 <== mds.0 v2:172.21.15.86:6835/3008383261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555ab5cbdb70 con 0x555ab5d6b690 2024-04-20T21:21:19.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.938+0000 7fd092000640 10 client.6363 handle_command_reply: tid=0 2024-04-20T21:21:19.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.938+0000 7fd0ae84a740 1 client.6363 shutdown 2024-04-20T21:21:19.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.938+0000 7fd0ae84a740 1 -- 172.21.15.86:0/2357032649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ab5d743d0 msgr2=0x555ab5d74840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:19.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.938+0000 7fd0ae84a740 1 --2- 172.21.15.86:0/2357032649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ab5d743d0 0x555ab5d74840 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fd094009890 tx=0x7fd094004730 comp rx=0 tx=0).stop 2024-04-20T21:21:19.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.938+0000 7fd0ae84a740 1 -- 172.21.15.86:0/2357032649 shutdown_connections 2024-04-20T21:21:19.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.938+0000 7fd0ae84a740 1 --2- 172.21.15.86:0/2357032649 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x555ab5d6b690 0x555ab5d6baa0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.938+0000 7fd0ae84a740 1 --2- 172.21.15.86:0/2357032649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ab5d5b9f0 0x555ab5d76c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.938+0000 7fd0ae84a740 1 --2- 172.21.15.86:0/2357032649 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ab5d743d0 0x555ab5d74840 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.938+0000 7fd0ae84a740 1 --2- 172.21.15.86:0/2357032649 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555ab5d47f50 0x555ab5d75130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.938+0000 7fd092000640 0 client.6363 ms_handle_reset on v2:172.21.15.86:6835/3008383261 2024-04-20T21:21:19.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.938+0000 7fd0ae84a740 1 -- 172.21.15.86:0/2357032649 >> 172.21.15.86:0/2357032649 conn(0x555ab5d443f0 msgr2=0x555ab5d926f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:19.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.938+0000 7fd0ae84a740 1 -- 172.21.15.86:0/2357032649 shutdown_connections 2024-04-20T21:21:19.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.939+0000 7fd0ae84a740 1 -- 172.21.15.86:0/2357032649 wait complete. 2024-04-20T21:21:19.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.939+0000 7fd0ae84a740 20 client.6363 trim_cache size 0 max 16384 2024-04-20T21:21:19.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.958+0000 7fd0ae84a740 1 Processor -- start 2024-04-20T21:21:19.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.958+0000 7fd0ae84a740 1 -- start start 2024-04-20T21:21:19.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.959+0000 7fd0ae84a740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555ab5d47f50 0x555ab5d6a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.959+0000 7fd0ae84a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ab5d5b9f0 0x555ab5d6ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.959+0000 7fd0ae84a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ab5d743d0 0x555ab5d91480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.959+0000 7fd0ae84a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a42009f0 con 0x555ab5d5b9f0 2024-04-20T21:21:19.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.959+0000 7fd0ae84a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a41ff9e0 con 0x555ab5d743d0 2024-04-20T21:21:19.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.959+0000 7fd0ae84a740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd0a41fed40 con 0x555ab5d47f50 2024-04-20T21:21:19.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.959+0000 7fd09be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ab5d5b9f0 0x555ab5d6ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:19.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.959+0000 7fd09be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ab5d5b9f0 0x555ab5d6ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58758/0 (socket says 172.21.15.86:58758) 2024-04-20T21:21:19.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.959+0000 7fd09be00640 1 -- 172.21.15.86:0/80697097 learned_addr learned my addr 172.21.15.86:0/80697097 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:19.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.960+0000 7fd09be00640 1 -- 172.21.15.86:0/80697097 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555ab5d47f50 msgr2=0x555ab5d6a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:19.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.960+0000 7fd09be00640 1 --2- 172.21.15.86:0/80697097 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555ab5d47f50 0x555ab5d6a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.960+0000 7fd09be00640 1 -- 172.21.15.86:0/80697097 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ab5d743d0 msgr2=0x555ab5d91480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:19.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.960+0000 7fd09be00640 1 --2- 172.21.15.86:0/80697097 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ab5d743d0 0x555ab5d91480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.960+0000 7fd09be00640 1 -- 172.21.15.86:0/80697097 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ab5d75700 con 0x555ab5d5b9f0 2024-04-20T21:21:19.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.960+0000 7fd09be00640 1 --2- 172.21.15.86:0/80697097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ab5d5b9f0 0x555ab5d6ab00 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fd094009860 tx=0x7fd09404b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70091c438c1a712d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:19.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.960+0000 7fd098c00640 1 -- 172.21.15.86:0/80697097 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd094031270 con 0x555ab5d5b9f0 2024-04-20T21:21:19.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.960+0000 7fd098c00640 1 -- 172.21.15.86:0/80697097 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd094031730 con 0x555ab5d5b9f0 2024-04-20T21:21:19.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.960+0000 7fd098c00640 1 -- 172.21.15.86:0/80697097 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd094031b20 con 0x555ab5d5b9f0 2024-04-20T21:21:19.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.961+0000 7fd0ae84a740 1 -- 172.21.15.86:0/80697097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ab5d5b9f0 msgr2=0x555ab5d6ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:19.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.961+0000 7fd0ae84a740 1 --2- 172.21.15.86:0/80697097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ab5d5b9f0 0x555ab5d6ab00 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fd094009860 tx=0x7fd09404b000 comp rx=0 tx=0).stop 2024-04-20T21:21:19.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.961+0000 7fd0ae84a740 1 -- 172.21.15.86:0/80697097 shutdown_connections 2024-04-20T21:21:19.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.961+0000 7fd0ae84a740 1 --2- 172.21.15.86:0/80697097 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ab5d743d0 0x555ab5d91480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.961+0000 7fd0ae84a740 1 --2- 172.21.15.86:0/80697097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ab5d5b9f0 0x555ab5d6ab00 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.961+0000 7fd0ae84a740 1 --2- 172.21.15.86:0/80697097 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555ab5d47f50 0x555ab5d6a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.961+0000 7fd0ae84a740 1 -- 172.21.15.86:0/80697097 >> 172.21.15.86:0/80697097 conn(0x555ab5d443f0 msgr2=0x555ab5d3f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:19.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.961+0000 7fd0ae84a740 1 -- 172.21.15.86:0/80697097 shutdown_connections 2024-04-20T21:21:19.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.961+0000 7fd0ae84a740 1 -- 172.21.15.86:0/80697097 wait complete. 2024-04-20T21:21:19.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.962+0000 7fd0ae84a740 1 -- start start 2024-04-20T21:21:19.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.962+0000 7fd0ae84a740 1 Processor -- start 2024-04-20T21:21:19.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.963+0000 7fd0ae84a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ab5d47f50 0x555ab5e10ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.963+0000 7fd0ae84a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ab5d743d0 0x555ab5d6cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.963+0000 7fd0ae84a740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555ab5d6d770 0x555ab5d8f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.963+0000 7fd0ae84a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a41ff390 con 0x555ab5d47f50 2024-04-20T21:21:19.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.963+0000 7fd0ae84a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555ab5d6d560 con 0x555ab5d743d0 2024-04-20T21:21:19.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.963+0000 7fd0ae84a740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x555ab5d6fc50 con 0x555ab5d6d770 2024-04-20T21:21:19.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.963+0000 7fd0ae84a740 10 client.0 _ll_register_callbacks cb 0x7fd09c0095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:21:19.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.963+0000 7fd0a9200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ab5d47f50 0x555ab5e10ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:19.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.963+0000 7fd0a9200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ab5d47f50 0x555ab5e10ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58764/0 (socket says 172.21.15.86:58764) 2024-04-20T21:21:19.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.963+0000 7fd0a9200640 1 -- 172.21.15.86:0/3103665595 learned_addr learned my addr 172.21.15.86:0/3103665595 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:19.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.964+0000 7fd09be00640 1 --2- 172.21.15.86:0/3103665595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ab5d743d0 0x555ab5d6cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:19.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.964+0000 7fd0a9200640 1 -- 172.21.15.86:0/3103665595 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555ab5d6d770 msgr2=0x555ab5d8f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:19.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.964+0000 7fd0a9200640 1 --2- 172.21.15.86:0/3103665595 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555ab5d6d770 0x555ab5d8f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.964+0000 7fd0a9200640 1 -- 172.21.15.86:0/3103665595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ab5d743d0 msgr2=0x555ab5d6cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:19.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.964+0000 7fd0a9200640 1 --2- 172.21.15.86:0/3103665595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ab5d743d0 0x555ab5d6cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.964+0000 7fd0a9200640 1 -- 172.21.15.86:0/3103665595 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ab5da1760 con 0x555ab5d47f50 2024-04-20T21:21:19.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.964+0000 7fd0a9200640 1 --2- 172.21.15.86:0/3103665595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ab5d47f50 0x555ab5e10ae0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fd08c001550 tx=0x7fd08c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c402a0bd8106c8d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:19.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.964+0000 7fd092000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:21:19.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.964+0000 7fd092000640 1 -- 172.21.15.86:0/3103665595 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08c01d040 con 0x555ab5d47f50 2024-04-20T21:21:19.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.964+0000 7fd092000640 1 -- 172.21.15.86:0/3103665595 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fd08c00d9f0 con 0x555ab5d47f50 2024-04-20T21:21:19.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.964+0000 7fd0ae84a740 1 -- 172.21.15.86:0/3103665595 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd094031730 con 0x555ab5d47f50 2024-04-20T21:21:19.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.965+0000 7fd092000640 1 -- 172.21.15.86:0/3103665595 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd08c031050 con 0x555ab5d47f50 2024-04-20T21:21:19.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.965+0000 7fd0ae84a740 10 client.6369 fetch_fsmap learned FSMap version 32 2024-04-20T21:21:19.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.965+0000 7fd0ae84a740 1 -- 172.21.15.86:0/3103665595 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) -- 0x555ab5da7e30 con 0x555ab5d47f50 2024-04-20T21:21:19.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.965+0000 7fd092000640 1 -- 172.21.15.86:0/3103665595 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd094031730 con 0x555ab5d47f50 2024-04-20T21:21:19.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.965+0000 7fd092000640 1 -- 172.21.15.86:0/3103665595 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 32) ==== 3495+0+0 (secure 0 0 0) 0x7fd08c034020 con 0x555ab5d47f50 2024-04-20T21:21:19.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.965+0000 7fd0ae84a740 10 client.6369 fetch_fsmap finished waiting for FSMap version 32 2024-04-20T21:21:19.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.965+0000 7fd0ae84a740 10 client.6369 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-20T21:21:19.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.965+0000 7fd0ae84a740 20 client.6369 populate_metadata read hostname 'smithi086' 2024-04-20T21:21:19.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.965+0000 7fd0ae84a740 1 --2- 172.21.15.86:0/3103665595 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x555ab5d6acd0 0x555ab5d6dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:19.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.965+0000 7fd0ae84a740 4 client.6369 mds_command: new command op to 6135 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:21:19.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.965+0000 7fd0ae84a740 1 -- 172.21.15.86:0/3103665595 --> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555ab5cb79e0 con 0x555ab5d6acd0 2024-04-20T21:21:19.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.965+0000 7fd09be00640 1 --2- 172.21.15.86:0/3103665595 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x555ab5d6acd0 0x555ab5d6dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:19.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.966+0000 7fd09be00640 1 --2- 172.21.15.86:0/3103665595 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x555ab5d6acd0 0x555ab5d6dc00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a16998d6b1bea443 server_cookie=de5c460efe8961e9 in_seq=0 out_seq=0 2024-04-20T21:21:19.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.966+0000 7fd092000640 10 client.6369 ms_handle_connect on v2:172.21.15.86:6835/3008383261 2024-04-20T21:21:19.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.966+0000 7fd092000640 1 -- 172.21.15.86:0/3103665595 <== mds.0 v2:172.21.15.86:6835/3008383261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+253 (crc 0 0 0) 0x555ab5cb79e0 con 0x555ab5d6acd0 2024-04-20T21:21:19.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.966+0000 7fd092000640 10 client.6369 handle_command_reply: tid=0 2024-04-20T21:21:19.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.966+0000 7fd0ae84a740 1 client.6369 shutdown 2024-04-20T21:21:19.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.966+0000 7fd0ae84a740 1 -- 172.21.15.86:0/3103665595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ab5d47f50 msgr2=0x555ab5e10ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:19.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.966+0000 7fd0ae84a740 1 --2- 172.21.15.86:0/3103665595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ab5d47f50 0x555ab5e10ae0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fd08c001550 tx=0x7fd08c000bc0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.966+0000 7fd0ae84a740 1 -- 172.21.15.86:0/3103665595 shutdown_connections 2024-04-20T21:21:19.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.966+0000 7fd0ae84a740 1 --2- 172.21.15.86:0/3103665595 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x555ab5d6acd0 0x555ab5d6dc00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.966+0000 7fd0ae84a740 1 --2- 172.21.15.86:0/3103665595 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555ab5d6d770 0x555ab5d8f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.966+0000 7fd0ae84a740 1 --2- 172.21.15.86:0/3103665595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ab5d743d0 0x555ab5d6cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.966+0000 7fd0ae84a740 1 --2- 172.21.15.86:0/3103665595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ab5d47f50 0x555ab5e10ae0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.966+0000 7fd092000640 0 client.6369 ms_handle_reset on v2:172.21.15.86:6835/3008383261 2024-04-20T21:21:19.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.966+0000 7fd0ae84a740 1 -- 172.21.15.86:0/3103665595 >> 172.21.15.86:0/3103665595 conn(0x555ab5d443f0 msgr2=0x555ab5d6a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:19.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.967+0000 7fd0ae84a740 1 -- 172.21.15.86:0/3103665595 shutdown_connections 2024-04-20T21:21:19.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.967+0000 7fd0ae84a740 1 -- 172.21.15.86:0/3103665595 wait complete. 2024-04-20T21:21:19.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.967+0000 7fd0ae84a740 20 client.6369 trim_cache size 0 max 16384 2024-04-20T21:21:19.969 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:21:19.970 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T21:21:19.970 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-20T21:21:19.970 INFO:teuthology.orchestra.run.smithi086.stdout: "9671ecfe-2c5d-450c-a0be-b56b83d19bde": { 2024-04-20T21:21:19.970 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-20T21:21:19.970 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "9671ecfe-2c5d-450c-a0be-b56b83d19bde", 2024-04-20T21:21:19.970 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive" 2024-04-20T21:21:19.970 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:21:19.970 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:21:19.970 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:21:19.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.967+0000 7fd0aa600640 1 -- 172.21.15.86:0/2474414912 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd08006c8f0 msgr2=0x7fd08006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:19.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.967+0000 7fd0aa600640 1 --2- 172.21.15.86:0/2474414912 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd08006c8f0 0x7fd08006edb0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd08c000f60 tx=0x7fd08c009040 comp rx=0 tx=0).stop 2024-04-20T21:21:19.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.967+0000 7fd0aa600640 1 -- 172.21.15.86:0/2474414912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0a4161c60 msgr2=0x7fd0a415b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:19.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.967+0000 7fd0aa600640 1 --2- 172.21.15.86:0/2474414912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0a4161c60 0x7fd0a415b150 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fd09c00efc0 tx=0x7fd09c00c490 comp rx=0 tx=0).stop 2024-04-20T21:21:19.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.967+0000 7fd0aa600640 1 -- 172.21.15.86:0/2474414912 shutdown_connections 2024-04-20T21:21:19.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.967+0000 7fd0aa600640 1 --2- 172.21.15.86:0/2474414912 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd08006c8f0 0x7fd08006edb0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.967+0000 7fd0aa600640 1 --2- 172.21.15.86:0/2474414912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0a4161c60 0x7fd0a415b150 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.967+0000 7fd0aa600640 1 --2- 172.21.15.86:0/2474414912 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0a415f990 0x7fd0a413bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.967+0000 7fd0aa600640 1 --2- 172.21.15.86:0/2474414912 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0a4071d40 0x7fd0a415dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:19.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.967+0000 7fd0aa600640 1 -- 172.21.15.86:0/2474414912 >> 172.21.15.86:0/2474414912 conn(0x7fd0a4067cf0 msgr2=0x7fd0a40b1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:19.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.968+0000 7fd0aa600640 1 -- 172.21.15.86:0/2474414912 shutdown_connections 2024-04-20T21:21:19.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:19.968+0000 7fd0aa600640 1 -- 172.21.15.86:0/2474414912 wait complete. 2024-04-20T21:21:19.983 INFO:tasks.cephfs.filesystem:scrub status for tag:9671ecfe-2c5d-450c-a0be-b56b83d19bde - {'path': '/', 'tag': '9671ecfe-2c5d-450c-a0be-b56b83d19bde', 'options': 'recursive'} 2024-04-20T21:21:24.641 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-04-20T21:21:24.642+0000 7f5bbce00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-20T21:21:46.241 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:21:46.250 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:21:49.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:21:49.985 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-20T21:21:50.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.198+0000 7fea73c00640 1 Processor -- start 2024-04-20T21:21:50.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.199+0000 7fea73c00640 1 -- start start 2024-04-20T21:21:50.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.200+0000 7fea73c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea6c15eb30 0x7fea6c0b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.200+0000 7fea73c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea6c0adb10 con 0x7fea6c0aee50 2024-04-20T21:21:50.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.200+0000 7fea73c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea6c0addb0 con 0x7fea6c06c090 2024-04-20T21:21:50.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.200+0000 7fea73c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fea6c0af7d0 con 0x7fea6c15eb30 2024-04-20T21:21:50.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.201+0000 7fea63e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fea6c0aee50 0x7fea6c06b980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:49300/0 (socket says 172.21.15.86:49300) 2024-04-20T21:21:50.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.201+0000 7fea63e00640 1 -- 172.21.15.86:0/2339122901 learned_addr learned my addr 172.21.15.86:0/2339122901 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:21:50.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.201+0000 7fea71200640 1 --2- 172.21.15.86:0/2339122901 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea6c15eb30 0x7fea6c0b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:50.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.201+0000 7fea6be00640 1 -- 172.21.15.86:0/2339122901 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 482434981 0 0) 0x7fea6c0adb10 con 0x7fea6c0aee50 2024-04-20T21:21:50.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.201+0000 7fea6be00640 1 -- 172.21.15.86:0/2339122901 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fea50003520 con 0x7fea6c0aee50 2024-04-20T21:21:50.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.202+0000 7fea6be00640 1 -- 172.21.15.86:0/2339122901 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2394999552 0 0) 0x7fea50003520 con 0x7fea6c0aee50 2024-04-20T21:21:50.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.202+0000 7fea6be00640 1 -- 172.21.15.86:0/2339122901 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea6c15eb30 msgr2=0x7fea6c0b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:50.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.202+0000 7fea6be00640 1 --2- 172.21.15.86:0/2339122901 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea6c15eb30 0x7fea6c0b3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.202+0000 7fea6be00640 1 -- 172.21.15.86:0/2339122901 >> v1:172.21.15.144:6789/0 conn(0x7fea6c06c090 legacy=0x7fea6c06e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:50.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.202+0000 7fea6be00640 1 -- 172.21.15.86:0/2339122901 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea6c0ae090 con 0x7fea6c0aee50 2024-04-20T21:21:50.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.202+0000 7fea6be00640 1 -- 172.21.15.86:0/2339122901 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fea54001670 con 0x7fea6c0aee50 2024-04-20T21:21:50.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.202+0000 7fea6be00640 1 -- 172.21.15.86:0/2339122901 >> v1:172.21.15.86:6789/0 conn(0x7fea6c0aee50 legacy=0x7fea6c06b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:50.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.202+0000 7fea6be00640 1 --2- 172.21.15.86:0/2339122901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea500039e0 0x7fea50003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.202+0000 7fea6be00640 1 --2- 172.21.15.86:0/2339122901 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea6c15eb30 0x7fea50004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.202+0000 7fea6be00640 1 --2- 172.21.15.86:0/2339122901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea50004d40 0x7fea500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.202+0000 7fea6be00640 1 -- 172.21.15.86:0/2339122901 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fea6c0af7d0 con 0x7fea500039e0 2024-04-20T21:21:50.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.202+0000 7fea6be00640 1 -- 172.21.15.86:0/2339122901 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fea5000ba70 con 0x7fea50004d40 2024-04-20T21:21:50.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.202+0000 7fea6be00640 1 -- 172.21.15.86:0/2339122901 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fea5000bc70 con 0x7fea6c15eb30 2024-04-20T21:21:50.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.202+0000 7fea71200640 1 --2- 172.21.15.86:0/2339122901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea500039e0 0x7fea50003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:50.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.203+0000 7fea71c00640 1 --2- 172.21.15.86:0/2339122901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea50004d40 0x7fea500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:50.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.203+0000 7fea63e00640 1 --2- 172.21.15.86:0/2339122901 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea6c15eb30 0x7fea50004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:50.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.203+0000 7fea71200640 1 -- 172.21.15.86:0/2339122901 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea6c15eb30 msgr2=0x7fea50004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:50.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.203+0000 7fea71200640 1 --2- 172.21.15.86:0/2339122901 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea6c15eb30 0x7fea50004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.203+0000 7fea71200640 1 -- 172.21.15.86:0/2339122901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea50004d40 msgr2=0x7fea500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:50.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.203+0000 7fea71200640 1 --2- 172.21.15.86:0/2339122901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea50004d40 0x7fea500051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.203+0000 7fea71200640 1 -- 172.21.15.86:0/2339122901 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea50003520 con 0x7fea500039e0 2024-04-20T21:21:50.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.203+0000 7fea71c00640 1 --2- 172.21.15.86:0/2339122901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea50004d40 0x7fea500051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:21:50.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.204+0000 7fea71200640 1 --2- 172.21.15.86:0/2339122901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea500039e0 0x7fea50003e00 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fea5c009a90 tx=0x7fea50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de955655425c2a05 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:50.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.204+0000 7fea6be00640 1 -- 172.21.15.86:0/2339122901 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5c039070 con 0x7fea500039e0 2024-04-20T21:21:50.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.204+0000 7fea6be00640 1 -- 172.21.15.86:0/2339122901 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea5c02c680 con 0x7fea500039e0 2024-04-20T21:21:50.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.204+0000 7fea6be00640 1 -- 172.21.15.86:0/2339122901 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5c009690 con 0x7fea500039e0 2024-04-20T21:21:50.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.205+0000 7fea73c00640 1 -- 172.21.15.86:0/2339122901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea500039e0 msgr2=0x7fea50003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:50.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.205+0000 7fea73c00640 1 --2- 172.21.15.86:0/2339122901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea500039e0 0x7fea50003e00 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fea5c009a90 tx=0x7fea50005830 comp rx=0 tx=0).stop 2024-04-20T21:21:50.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.205+0000 7fea71c00640 1 -- 172.21.15.86:0/2339122901 reap_dead start 2024-04-20T21:21:50.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.205+0000 7fea73c00640 1 -- 172.21.15.86:0/2339122901 shutdown_connections 2024-04-20T21:21:50.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.205+0000 7fea73c00640 1 -- 172.21.15.86:0/2339122901 >> 172.21.15.86:0/2339122901 conn(0x7fea6c0b2110 msgr2=0x7fea6c0b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:50.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.205+0000 7fea73c00640 1 -- 172.21.15.86:0/2339122901 shutdown_connections 2024-04-20T21:21:50.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.205+0000 7fea73c00640 1 -- 172.21.15.86:0/2339122901 wait complete. 2024-04-20T21:21:50.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.207+0000 7fea73c00640 1 Processor -- start 2024-04-20T21:21:50.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.207+0000 7fea73c00640 1 -- start start 2024-04-20T21:21:50.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.208+0000 7fea73c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea500039e0 0x7fea6c148da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.208+0000 7fea73c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea6c156b10 0x7fea6c15d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.208+0000 7fea73c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea6c156f90 0x7fea6c15de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.208+0000 7fea73c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fea6c0ae930 con 0x7fea500039e0 2024-04-20T21:21:50.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.208+0000 7fea73c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fea6c0ae650 con 0x7fea6c156b10 2024-04-20T21:21:50.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.208+0000 7fea73c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fea6c06e870 con 0x7fea6c156f90 2024-04-20T21:21:50.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.208+0000 7fea71200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea500039e0 0x7fea6c148da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:50.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.208+0000 7fea71200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea500039e0 0x7fea6c148da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58136/0 (socket says 172.21.15.86:58136) 2024-04-20T21:21:50.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.208+0000 7fea71200640 1 -- 172.21.15.86:0/3619393220 learned_addr learned my addr 172.21.15.86:0/3619393220 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:50.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.208+0000 7fea71200640 1 -- 172.21.15.86:0/3619393220 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea6c156f90 msgr2=0x7fea6c15de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:50.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.208+0000 7fea71200640 1 --2- 172.21.15.86:0/3619393220 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea6c156f90 0x7fea6c15de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.208+0000 7fea71200640 1 -- 172.21.15.86:0/3619393220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea6c156b10 msgr2=0x7fea6c15d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:50.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.208+0000 7fea71200640 1 --2- 172.21.15.86:0/3619393220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea6c156b10 0x7fea6c15d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.208+0000 7fea71200640 1 -- 172.21.15.86:0/3619393220 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea6c1575d0 con 0x7fea500039e0 2024-04-20T21:21:50.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.209+0000 7fea71200640 1 --2- 172.21.15.86:0/3619393220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea500039e0 0x7fea6c148da0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fea5c037040 tx=0x7fea5c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:50.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.209+0000 7fea69600640 1 -- 172.21.15.86:0/3619393220 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5c042020 con 0x7fea500039e0 2024-04-20T21:21:50.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.209+0000 7fea69600640 1 -- 172.21.15.86:0/3619393220 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea5c03a040 con 0x7fea500039e0 2024-04-20T21:21:50.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.209+0000 7fea69600640 1 -- 172.21.15.86:0/3619393220 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5c039070 con 0x7fea500039e0 2024-04-20T21:21:50.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.209+0000 7fea73c00640 1 -- 172.21.15.86:0/3619393220 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea6c1578b0 con 0x7fea500039e0 2024-04-20T21:21:50.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.209+0000 7fea73c00640 1 -- 172.21.15.86:0/3619393220 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea6c1f44e0 con 0x7fea500039e0 2024-04-20T21:21:50.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.211+0000 7fea69600640 1 -- 172.21.15.86:0/3619393220 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fea5c045020 con 0x7fea500039e0 2024-04-20T21:21:50.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.212+0000 7fea69600640 1 --2- 172.21.15.86:0/3619393220 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fea4806c920 0x7fea4806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.212+0000 7fea69600640 1 -- 172.21.15.86:0/3619393220 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fea5c0acbe0 con 0x7fea500039e0 2024-04-20T21:21:50.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.212+0000 7fea63e00640 1 --2- 172.21.15.86:0/3619393220 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fea4806c920 0x7fea4806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:50.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.213+0000 7fea76995740 1 Processor -- start 2024-04-20T21:21:50.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.213+0000 7fea76995740 1 -- start start 2024-04-20T21:21:50.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.213+0000 7fea63e00640 1 --2- 172.21.15.86:0/3619393220 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fea4806c920 0x7fea4806ede0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fea540017e0 tx=0x7fea54009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:50.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.214+0000 7fea76995740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55782cdd41f0 0x55782cdd45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.214+0000 7fea76995740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55782cda4210 0x55782cdd1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.214+0000 7fea76995740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55782cda7d70 0x55782cdd19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.214+0000 7fea76995740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55782cd10150 con 0x55782cda4210 2024-04-20T21:21:50.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.214+0000 7fea76995740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55782ccb6370 con 0x55782cdd41f0 2024-04-20T21:21:50.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.214+0000 7fea76995740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55782cd0c110 con 0x55782cda7d70 2024-04-20T21:21:50.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.214+0000 7fea63e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55782cda4210 0x55782cdd1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:50.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.214+0000 7fea63e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55782cda4210 0x55782cdd1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58146/0 (socket says 172.21.15.86:58146) 2024-04-20T21:21:50.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.214+0000 7fea63e00640 1 -- 172.21.15.86:0/1078051103 learned_addr learned my addr 172.21.15.86:0/1078051103 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:50.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.215+0000 7fea63e00640 1 -- 172.21.15.86:0/1078051103 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55782cda7d70 msgr2=0x55782cdd19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:50.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.215+0000 7fea63e00640 1 --2- 172.21.15.86:0/1078051103 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55782cda7d70 0x55782cdd19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.215+0000 7fea71200640 1 --2- 172.21.15.86:0/1078051103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55782cdd41f0 0x55782cdd45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:50.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.215+0000 7fea63e00640 1 -- 172.21.15.86:0/1078051103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55782cdd41f0 msgr2=0x55782cdd45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:50.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.215+0000 7fea63e00640 1 --2- 172.21.15.86:0/1078051103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55782cdd41f0 0x55782cdd45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.215+0000 7fea63e00640 1 -- 172.21.15.86:0/1078051103 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55782caec7d0 con 0x55782cda4210 2024-04-20T21:21:50.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.215+0000 7fea63e00640 1 --2- 172.21.15.86:0/1078051103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55782cda4210 0x55782cdd1460 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fea54005c10 tx=0x7fea54010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3be537b39bfce4c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:50.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.215+0000 7fea61600640 1 -- 172.21.15.86:0/1078051103 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea540076e0 con 0x55782cda4210 2024-04-20T21:21:50.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.215+0000 7fea61600640 1 -- 172.21.15.86:0/1078051103 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea54005c40 con 0x55782cda4210 2024-04-20T21:21:50.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.215+0000 7fea61600640 1 -- 172.21.15.86:0/1078051103 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea54020450 con 0x55782cda4210 2024-04-20T21:21:50.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.216+0000 7fea76995740 1 -- 172.21.15.86:0/1078051103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55782cda4210 msgr2=0x55782cdd1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:50.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.216+0000 7fea76995740 1 --2- 172.21.15.86:0/1078051103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55782cda4210 0x55782cdd1460 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fea54005c10 tx=0x7fea54010000 comp rx=0 tx=0).stop 2024-04-20T21:21:50.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.216+0000 7fea76995740 1 -- 172.21.15.86:0/1078051103 shutdown_connections 2024-04-20T21:21:50.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.216+0000 7fea76995740 1 --2- 172.21.15.86:0/1078051103 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55782cda7d70 0x55782cdd19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.216+0000 7fea76995740 1 --2- 172.21.15.86:0/1078051103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55782cda4210 0x55782cdd1460 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.216+0000 7fea76995740 1 --2- 172.21.15.86:0/1078051103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55782cdd41f0 0x55782cdd45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.216+0000 7fea76995740 1 -- 172.21.15.86:0/1078051103 >> 172.21.15.86:0/1078051103 conn(0x55782cda6480 msgr2=0x55782cdef000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:50.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.216+0000 7fea76995740 1 -- 172.21.15.86:0/1078051103 shutdown_connections 2024-04-20T21:21:50.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.216+0000 7fea76995740 1 -- 172.21.15.86:0/1078051103 wait complete. 2024-04-20T21:21:50.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.217+0000 7fea76995740 1 -- start start 2024-04-20T21:21:50.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.217+0000 7fea76995740 1 Processor -- start 2024-04-20T21:21:50.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.218+0000 7fea76995740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55782cda7d70 0x55782cdbd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.218+0000 7fea76995740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55782cdd41f0 0x55782cdbc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.218+0000 7fea76995740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55782cdd4b30 0x55782cdcff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.218+0000 7fea76995740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55782cda8f20 con 0x55782cdd41f0 2024-04-20T21:21:50.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.218+0000 7fea76995740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55782ccc9740 con 0x55782cda7d70 2024-04-20T21:21:50.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.218+0000 7fea76995740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55782cda8310 con 0x55782cdd4b30 2024-04-20T21:21:50.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.218+0000 7fea76995740 10 client.0 _ll_register_callbacks cb 0x55782ccc4b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:21:50.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.218+0000 7fea63e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55782cdd41f0 0x55782cdbc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:50.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.219+0000 7fea63e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55782cdd41f0 0x55782cdbc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58148/0 (socket says 172.21.15.86:58148) 2024-04-20T21:21:50.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.219+0000 7fea63e00640 1 -- 172.21.15.86:0/3775727194 learned_addr learned my addr 172.21.15.86:0/3775727194 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:50.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.219+0000 7fea71200640 1 --2- 172.21.15.86:0/3775727194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55782cda7d70 0x55782cdbd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:50.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.219+0000 7fea71c00640 1 --2- 172.21.15.86:0/3775727194 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55782cdd4b30 0x55782cdcff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:50.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.219+0000 7fea63e00640 1 -- 172.21.15.86:0/3775727194 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55782cdd4b30 msgr2=0x55782cdcff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:50.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.219+0000 7fea63e00640 1 --2- 172.21.15.86:0/3775727194 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55782cdd4b30 0x55782cdcff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.219+0000 7fea63e00640 1 -- 172.21.15.86:0/3775727194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55782cda7d70 msgr2=0x55782cdbd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:50.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.219+0000 7fea63e00640 1 --2- 172.21.15.86:0/3775727194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55782cda7d70 0x55782cdbd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.219+0000 7fea63e00640 1 -- 172.21.15.86:0/3775727194 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55782ca223f0 con 0x55782cdd41f0 2024-04-20T21:21:50.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.219+0000 7fea71200640 1 --2- 172.21.15.86:0/3775727194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55782cda7d70 0x55782cdbd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:21:50.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.219+0000 7fea63e00640 1 --2- 172.21.15.86:0/3775727194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55782cdd41f0 0x55782cdbc6d0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fea54009010 tx=0x7fea54001150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df54425c2c53f9cf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:50.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.219+0000 7fea5a000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:21:50.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.220+0000 7fea5a000640 1 -- 172.21.15.86:0/3775727194 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea54030060 con 0x55782cdd41f0 2024-04-20T21:21:50.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.220+0000 7fea5a000640 1 -- 172.21.15.86:0/3775727194 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fea54021a20 con 0x55782cdd41f0 2024-04-20T21:21:50.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.220+0000 7fea76995740 1 -- 172.21.15.86:0/3775727194 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea54005c40 con 0x55782cdd41f0 2024-04-20T21:21:50.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.220+0000 7fea5a000640 1 -- 172.21.15.86:0/3775727194 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fea54036050 con 0x55782cdd41f0 2024-04-20T21:21:50.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.220+0000 7fea76995740 10 client.6381 fetch_fsmap learned FSMap version 33 2024-04-20T21:21:50.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.220+0000 7fea76995740 1 -- 172.21.15.86:0/3775727194 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x55782c9e20f0 con 0x55782cdd41f0 2024-04-20T21:21:50.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.220+0000 7fea5a000640 1 -- 172.21.15.86:0/3775727194 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea54005c40 con 0x55782cdd41f0 2024-04-20T21:21:50.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.220+0000 7fea5a000640 1 -- 172.21.15.86:0/3775727194 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 33) ==== 3495+0+0 (secure 0 0 0) 0x7fea54039020 con 0x55782cdd41f0 2024-04-20T21:21:50.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.220+0000 7fea76995740 10 client.6381 fetch_fsmap finished waiting for FSMap version 33 2024-04-20T21:21:50.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.220+0000 7fea76995740 10 client.6381 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-20T21:21:50.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.220+0000 7fea76995740 20 client.6381 populate_metadata read hostname 'smithi086' 2024-04-20T21:21:50.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.220+0000 7fea76995740 1 --2- 172.21.15.86:0/3775727194 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x55782cdd62d0 0x55782cdd66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.221+0000 7fea76995740 4 client.6381 mds_command: new command op to 6135 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:21:50.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.221+0000 7fea76995740 1 -- 172.21.15.86:0/3775727194 --> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55782cd1db70 con 0x55782cdd62d0 2024-04-20T21:21:50.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.221+0000 7fea71200640 1 --2- 172.21.15.86:0/3775727194 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x55782cdd62d0 0x55782cdd66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:50.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.221+0000 7fea5a000640 10 client.6381 ms_handle_connect on v2:172.21.15.86:6835/3008383261 2024-04-20T21:21:50.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.221+0000 7fea71200640 1 --2- 172.21.15.86:0/3775727194 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x55782cdd62d0 0x55782cdd66d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d101e074607fb23 server_cookie=fb9d1fada2a50687 in_seq=0 out_seq=0 2024-04-20T21:21:50.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.222+0000 7fea5a000640 1 -- 172.21.15.86:0/3775727194 <== mds.0 v2:172.21.15.86:6835/3008383261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55782cd1db70 con 0x55782cdd62d0 2024-04-20T21:21:50.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.222+0000 7fea5a000640 10 client.6381 handle_command_reply: tid=0 2024-04-20T21:21:50.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.222+0000 7fea76995740 1 client.6381 shutdown 2024-04-20T21:21:50.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.222+0000 7fea76995740 1 -- 172.21.15.86:0/3775727194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55782cdd41f0 msgr2=0x55782cdbc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:50.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.222+0000 7fea76995740 1 --2- 172.21.15.86:0/3775727194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55782cdd41f0 0x55782cdbc6d0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fea54009010 tx=0x7fea54001150 comp rx=0 tx=0).stop 2024-04-20T21:21:50.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.222+0000 7fea76995740 1 -- 172.21.15.86:0/3775727194 shutdown_connections 2024-04-20T21:21:50.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.222+0000 7fea76995740 1 --2- 172.21.15.86:0/3775727194 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x55782cdd62d0 0x55782cdd66d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.222+0000 7fea76995740 1 --2- 172.21.15.86:0/3775727194 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55782cdd4b30 0x55782cdcff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.222+0000 7fea76995740 1 --2- 172.21.15.86:0/3775727194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55782cdd41f0 0x55782cdbc6d0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.222+0000 7fea76995740 1 --2- 172.21.15.86:0/3775727194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55782cda7d70 0x55782cdbd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.222+0000 7fea5a000640 0 client.6381 ms_handle_reset on v2:172.21.15.86:6835/3008383261 2024-04-20T21:21:50.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.222+0000 7fea76995740 1 -- 172.21.15.86:0/3775727194 >> 172.21.15.86:0/3775727194 conn(0x55782cda4210 msgr2=0x55782cdf05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:50.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.223+0000 7fea76995740 1 -- 172.21.15.86:0/3775727194 shutdown_connections 2024-04-20T21:21:50.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.223+0000 7fea76995740 1 -- 172.21.15.86:0/3775727194 wait complete. 2024-04-20T21:21:50.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.223+0000 7fea76995740 20 client.6381 trim_cache size 0 max 16384 2024-04-20T21:21:50.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.242+0000 7fea76995740 1 Processor -- start 2024-04-20T21:21:50.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.242+0000 7fea76995740 1 -- start start 2024-04-20T21:21:50.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.243+0000 7fea76995740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55782cda7d70 0x55782cda77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.243+0000 7fea76995740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55782cdd41f0 0x55782cdff660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.243+0000 7fea76995740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55782cdd4b30 0x55782cdbd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.243+0000 7fea76995740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55782cdbbf00 con 0x55782cdd41f0 2024-04-20T21:21:50.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.243+0000 7fea76995740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fea6c2017e0 con 0x55782cdd4b30 2024-04-20T21:21:50.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.243+0000 7fea76995740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fea6c2007b0 con 0x55782cda7d70 2024-04-20T21:21:50.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.243+0000 7fea63e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55782cdd41f0 0x55782cdff660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:50.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.243+0000 7fea63e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55782cdd41f0 0x55782cdff660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58156/0 (socket says 172.21.15.86:58156) 2024-04-20T21:21:50.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.243+0000 7fea63e00640 1 -- 172.21.15.86:0/1451919638 learned_addr learned my addr 172.21.15.86:0/1451919638 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:50.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.244+0000 7fea71c00640 1 --2- 172.21.15.86:0/1451919638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55782cdd4b30 0x55782cdbd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:50.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.244+0000 7fea71200640 1 --2- 172.21.15.86:0/1451919638 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55782cda7d70 0x55782cda77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:50.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.244+0000 7fea63e00640 1 -- 172.21.15.86:0/1451919638 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55782cda7d70 msgr2=0x55782cda77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:50.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.244+0000 7fea63e00640 1 --2- 172.21.15.86:0/1451919638 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55782cda7d70 0x55782cda77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.244+0000 7fea63e00640 1 -- 172.21.15.86:0/1451919638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55782cdd4b30 msgr2=0x55782cdbd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:50.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.244+0000 7fea63e00640 1 --2- 172.21.15.86:0/1451919638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55782cdd4b30 0x55782cdbd0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.244+0000 7fea63e00640 1 -- 172.21.15.86:0/1451919638 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55782ce13d90 con 0x55782cdd41f0 2024-04-20T21:21:50.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.244+0000 7fea71200640 1 --2- 172.21.15.86:0/1451919638 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55782cda7d70 0x55782cda77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:21:50.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.244+0000 7fea63e00640 1 --2- 172.21.15.86:0/1451919638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55782cdd41f0 0x55782cdff660 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fea54005ae0 tx=0x7fea54023070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c30504f5f14a02 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:50.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.244+0000 7fea60c00640 1 -- 172.21.15.86:0/1451919638 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea54005bc0 con 0x55782cdd41f0 2024-04-20T21:21:50.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.244+0000 7fea60c00640 1 -- 172.21.15.86:0/1451919638 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea54021e50 con 0x55782cdd41f0 2024-04-20T21:21:50.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.244+0000 7fea60c00640 1 -- 172.21.15.86:0/1451919638 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea540047e0 con 0x55782cdd41f0 2024-04-20T21:21:50.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.245+0000 7fea76995740 1 -- 172.21.15.86:0/1451919638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55782cdd41f0 msgr2=0x55782cdff660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:50.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.245+0000 7fea76995740 1 --2- 172.21.15.86:0/1451919638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55782cdd41f0 0x55782cdff660 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fea54005ae0 tx=0x7fea54023070 comp rx=0 tx=0).stop 2024-04-20T21:21:50.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.245+0000 7fea76995740 1 -- 172.21.15.86:0/1451919638 shutdown_connections 2024-04-20T21:21:50.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.245+0000 7fea76995740 1 --2- 172.21.15.86:0/1451919638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55782cdd4b30 0x55782cdbd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.245+0000 7fea76995740 1 --2- 172.21.15.86:0/1451919638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55782cdd41f0 0x55782cdff660 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.245+0000 7fea76995740 1 --2- 172.21.15.86:0/1451919638 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55782cda7d70 0x55782cda77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.245+0000 7fea76995740 1 -- 172.21.15.86:0/1451919638 >> 172.21.15.86:0/1451919638 conn(0x55782cda4210 msgr2=0x55782cdd76c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:50.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.245+0000 7fea76995740 1 -- 172.21.15.86:0/1451919638 shutdown_connections 2024-04-20T21:21:50.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.245+0000 7fea76995740 1 -- 172.21.15.86:0/1451919638 wait complete. 2024-04-20T21:21:50.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.246+0000 7fea76995740 1 -- start start 2024-04-20T21:21:50.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.246+0000 7fea76995740 1 Processor -- start 2024-04-20T21:21:50.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.247+0000 7fea76995740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55782cda7d70 0x55782ce748c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.247+0000 7fea76995740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55782cdd41f0 0x55782cdc8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.247+0000 7fea76995740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55782cdd4b30 0x55782cdc8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.247+0000 7fea76995740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fea6c200160 con 0x55782cdd41f0 2024-04-20T21:21:50.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.247+0000 7fea76995740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fea6c1ffb90 con 0x55782cda7d70 2024-04-20T21:21:50.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.247+0000 7fea76995740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55782cdffba0 con 0x55782cdd4b30 2024-04-20T21:21:50.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.247+0000 7fea76995740 10 client.0 _ll_register_callbacks cb 0x7fea54005b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:21:50.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.247+0000 7fea63e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55782cdd41f0 0x55782cdc8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:50.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.247+0000 7fea63e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55782cdd41f0 0x55782cdc8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58172/0 (socket says 172.21.15.86:58172) 2024-04-20T21:21:50.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.247+0000 7fea63e00640 1 -- 172.21.15.86:0/3303272615 learned_addr learned my addr 172.21.15.86:0/3303272615 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:50.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.248+0000 7fea71200640 1 --2- 172.21.15.86:0/3303272615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55782cda7d70 0x55782ce748c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:50.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.248+0000 7fea71c00640 1 --2- 172.21.15.86:0/3303272615 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55782cdd4b30 0x55782cdc8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:50.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.248+0000 7fea63e00640 1 -- 172.21.15.86:0/3303272615 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55782cdd4b30 msgr2=0x55782cdc8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:50.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.248+0000 7fea63e00640 1 --2- 172.21.15.86:0/3303272615 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55782cdd4b30 0x55782cdc8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.248+0000 7fea63e00640 1 -- 172.21.15.86:0/3303272615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55782cda7d70 msgr2=0x55782ce748c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:50.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.248+0000 7fea63e00640 1 --2- 172.21.15.86:0/3303272615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55782cda7d70 0x55782ce748c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.248+0000 7fea63e00640 1 -- 172.21.15.86:0/3303272615 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55782cdbd670 con 0x55782cdd41f0 2024-04-20T21:21:50.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.248+0000 7fea63e00640 1 --2- 172.21.15.86:0/3303272615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55782cdd41f0 0x55782cdc8120 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fea5402d040 tx=0x7fea5400b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c069361d7520e51 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:50.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.248+0000 7fea5a000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:21:50.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.248+0000 7fea5a000640 1 -- 172.21.15.86:0/3303272615 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea54021a20 con 0x55782cdd41f0 2024-04-20T21:21:50.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.248+0000 7fea76995740 1 -- 172.21.15.86:0/3303272615 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea54021e50 con 0x55782cdd41f0 2024-04-20T21:21:50.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.248+0000 7fea5a000640 1 -- 172.21.15.86:0/3303272615 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fea54045ad0 con 0x55782cdd41f0 2024-04-20T21:21:50.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.248+0000 7fea5a000640 1 -- 172.21.15.86:0/3303272615 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fea54045cd0 con 0x55782cdd41f0 2024-04-20T21:21:50.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.249+0000 7fea76995740 10 client.6387 fetch_fsmap learned FSMap version 33 2024-04-20T21:21:50.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.249+0000 7fea76995740 1 -- 172.21.15.86:0/3303272615 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x55782cda3e30 con 0x55782cdd41f0 2024-04-20T21:21:50.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.249+0000 7fea5a000640 1 -- 172.21.15.86:0/3303272615 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea54021e50 con 0x55782cdd41f0 2024-04-20T21:21:50.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.249+0000 7fea5a000640 1 -- 172.21.15.86:0/3303272615 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 33) ==== 3495+0+0 (secure 0 0 0) 0x7fea54039070 con 0x55782cdd41f0 2024-04-20T21:21:50.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.249+0000 7fea76995740 10 client.6387 fetch_fsmap finished waiting for FSMap version 33 2024-04-20T21:21:50.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.249+0000 7fea76995740 10 client.6387 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-20T21:21:50.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.249+0000 7fea76995740 20 client.6387 populate_metadata read hostname 'smithi086' 2024-04-20T21:21:50.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.249+0000 7fea76995740 1 --2- 172.21.15.86:0/3303272615 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x55782ce70f20 0x55782cdf15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.249+0000 7fea76995740 4 client.6387 mds_command: new command op to 6135 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:21:50.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.249+0000 7fea76995740 1 -- 172.21.15.86:0/3303272615 --> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55782cd179e0 con 0x55782ce70f20 2024-04-20T21:21:50.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.249+0000 7fea71200640 1 --2- 172.21.15.86:0/3303272615 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x55782ce70f20 0x55782cdf15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:50.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.250+0000 7fea71200640 1 --2- 172.21.15.86:0/3303272615 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x55782ce70f20 0x55782cdf15f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=620aeee3ef692512 server_cookie=50ebdf561778ea76 in_seq=0 out_seq=0 2024-04-20T21:21:50.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.250+0000 7fea5a000640 10 client.6387 ms_handle_connect on v2:172.21.15.86:6835/3008383261 2024-04-20T21:21:50.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.250+0000 7fea5a000640 1 -- 172.21.15.86:0/3303272615 <== mds.0 v2:172.21.15.86:6835/3008383261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55782cd179e0 con 0x55782ce70f20 2024-04-20T21:21:50.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.250+0000 7fea5a000640 10 client.6387 handle_command_reply: tid=0 2024-04-20T21:21:50.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.250+0000 7fea76995740 1 client.6387 shutdown 2024-04-20T21:21:50.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.250+0000 7fea76995740 1 -- 172.21.15.86:0/3303272615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55782cdd41f0 msgr2=0x55782cdc8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:50.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.250+0000 7fea76995740 1 --2- 172.21.15.86:0/3303272615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55782cdd41f0 0x55782cdc8120 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fea5402d040 tx=0x7fea5400b000 comp rx=0 tx=0).stop 2024-04-20T21:21:50.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.250+0000 7fea76995740 1 -- 172.21.15.86:0/3303272615 shutdown_connections 2024-04-20T21:21:50.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.250+0000 7fea76995740 1 --2- 172.21.15.86:0/3303272615 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x55782ce70f20 0x55782cdf15f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.250+0000 7fea76995740 1 --2- 172.21.15.86:0/3303272615 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55782cdd4b30 0x55782cdc8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.250+0000 7fea76995740 1 --2- 172.21.15.86:0/3303272615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55782cdd41f0 0x55782cdc8120 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.250+0000 7fea76995740 1 --2- 172.21.15.86:0/3303272615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55782cda7d70 0x55782ce748c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.250+0000 7fea5a000640 0 client.6387 ms_handle_reset on v2:172.21.15.86:6835/3008383261 2024-04-20T21:21:50.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.250+0000 7fea76995740 1 -- 172.21.15.86:0/3303272615 >> 172.21.15.86:0/3303272615 conn(0x55782cda4210 msgr2=0x55782cdf2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:50.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.250+0000 7fea76995740 1 -- 172.21.15.86:0/3303272615 shutdown_connections 2024-04-20T21:21:50.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.250+0000 7fea76995740 1 -- 172.21.15.86:0/3303272615 wait complete. 2024-04-20T21:21:50.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.251+0000 7fea76995740 20 client.6387 trim_cache size 0 max 16384 2024-04-20T21:21:50.255 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:21:50.255 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-20T21:21:50.255 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-20T21:21:50.256 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:21:50.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.251+0000 7fea73c00640 1 -- 172.21.15.86:0/3619393220 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fea4806c920 msgr2=0x7fea4806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:50.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.251+0000 7fea73c00640 1 --2- 172.21.15.86:0/3619393220 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fea4806c920 0x7fea4806ede0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fea540017e0 tx=0x7fea54009040 comp rx=0 tx=0).stop 2024-04-20T21:21:50.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.251+0000 7fea73c00640 1 -- 172.21.15.86:0/3619393220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea500039e0 msgr2=0x7fea6c148da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:50.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.251+0000 7fea73c00640 1 --2- 172.21.15.86:0/3619393220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea500039e0 0x7fea6c148da0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fea5c037040 tx=0x7fea5c004730 comp rx=0 tx=0).stop 2024-04-20T21:21:50.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.251+0000 7fea73c00640 1 -- 172.21.15.86:0/3619393220 shutdown_connections 2024-04-20T21:21:50.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.251+0000 7fea73c00640 1 --2- 172.21.15.86:0/3619393220 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fea4806c920 0x7fea4806ede0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.251+0000 7fea73c00640 1 --2- 172.21.15.86:0/3619393220 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fea6c156f90 0x7fea6c15de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.251+0000 7fea73c00640 1 --2- 172.21.15.86:0/3619393220 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea6c156b10 0x7fea6c15d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.251+0000 7fea73c00640 1 --2- 172.21.15.86:0/3619393220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea500039e0 0x7fea6c148da0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.251+0000 7fea73c00640 1 -- 172.21.15.86:0/3619393220 >> 172.21.15.86:0/3619393220 conn(0x7fea6c0b2110 msgr2=0x7fea6c0d0630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:50.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.251+0000 7fea73c00640 1 -- 172.21.15.86:0/3619393220 shutdown_connections 2024-04-20T21:21:50.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.251+0000 7fea73c00640 1 -- 172.21.15.86:0/3619393220 wait complete. 2024-04-20T21:21:50.269 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T21:21:50.270 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:21:50.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.466+0000 7f139be00640 1 Processor -- start 2024-04-20T21:21:50.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.466+0000 7f139be00640 1 -- start start 2024-04-20T21:21:50.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.468+0000 7f139be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f139c15ece0 0x7f139c15f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.468+0000 7f139be00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f139c0d50c0 0x7f139c0d3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.468+0000 7f139be00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f139c0cf9c0 0x7f139c160000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.468+0000 7f139be00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f139c15f600 con 0x7f139c15ece0 2024-04-20T21:21:50.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.468+0000 7f139be00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f139c0d3950 con 0x7f139c0cf9c0 2024-04-20T21:21:50.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.468+0000 7f139be00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f139c0cec00 con 0x7f139c0d50c0 2024-04-20T21:21:50.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.468+0000 7f139aa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f139c15ece0 0x7f139c15f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:50.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.468+0000 7f139aa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f139c15ece0 0x7f139c15f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58188/0 (socket says 172.21.15.86:58188) 2024-04-20T21:21:50.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.468+0000 7f139aa00640 1 -- 172.21.15.86:0/3235334986 learned_addr learned my addr 172.21.15.86:0/3235334986 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:50.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.468+0000 7f139aa00640 1 -- 172.21.15.86:0/3235334986 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f139c0d50c0 msgr2=0x7f139c0d3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:50.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.468+0000 7f139aa00640 1 --2- 172.21.15.86:0/3235334986 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f139c0d50c0 0x7f139c0d3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.468+0000 7f139aa00640 1 -- 172.21.15.86:0/3235334986 >> v2:172.21.15.144:3300/0 conn(0x7f139c0cf9c0 msgr2=0x7f139c160000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:50.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.468+0000 7f139aa00640 1 --2- 172.21.15.86:0/3235334986 >> v2:172.21.15.144:3300/0 conn(0x7f139c0cf9c0 0x7f139c160000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.468+0000 7f139aa00640 1 -- 172.21.15.86:0/3235334986 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f139c0ceee0 con 0x7f139c15ece0 2024-04-20T21:21:50.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.469+0000 7f139aa00640 1 --2- 172.21.15.86:0/3235334986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f139c15ece0 0x7f139c15f0c0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f139c0ba5c0 tx=0x7f138c02b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f534b8dadbd94df6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:50.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.470+0000 7f139a000640 1 -- 172.21.15.86:0/3235334986 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138c0317f0 con 0x7f139c15ece0 2024-04-20T21:21:50.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.470+0000 7f139a000640 1 -- 172.21.15.86:0/3235334986 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f138c037070 con 0x7f139c15ece0 2024-04-20T21:21:50.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.470+0000 7f139a000640 1 -- 172.21.15.86:0/3235334986 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138c031bb0 con 0x7f139c15ece0 2024-04-20T21:21:50.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.470+0000 7f139be00640 1 -- 172.21.15.86:0/3235334986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f139c15ece0 msgr2=0x7f139c15f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:50.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.470+0000 7f139be00640 1 --2- 172.21.15.86:0/3235334986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f139c15ece0 0x7f139c15f0c0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f139c0ba5c0 tx=0x7f138c02b890 comp rx=0 tx=0).stop 2024-04-20T21:21:50.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.470+0000 7f139be00640 1 -- 172.21.15.86:0/3235334986 shutdown_connections 2024-04-20T21:21:50.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.470+0000 7f139be00640 1 --2- 172.21.15.86:0/3235334986 >> v2:172.21.15.144:3300/0 conn(0x7f139c0cf9c0 0x7f139c160000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.470+0000 7f139be00640 1 --2- 172.21.15.86:0/3235334986 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f139c0d50c0 0x7f139c0d3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.470+0000 7f139be00640 1 --2- 172.21.15.86:0/3235334986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f139c15ece0 0x7f139c15f0c0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.470+0000 7f139be00640 1 -- 172.21.15.86:0/3235334986 >> 172.21.15.86:0/3235334986 conn(0x7f139c08bf70 msgr2=0x7f139c0d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:50.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.471+0000 7f139be00640 1 -- 172.21.15.86:0/3235334986 shutdown_connections 2024-04-20T21:21:50.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.471+0000 7f139be00640 1 -- 172.21.15.86:0/3235334986 wait complete. 2024-04-20T21:21:50.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.472+0000 7f139be00640 1 Processor -- start 2024-04-20T21:21:50.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.472+0000 7f139be00640 1 -- start start 2024-04-20T21:21:50.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.473+0000 7f139be00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f139c0cf9c0 0x7f139c09ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.473+0000 7f139be00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f139c0d50c0 0x7f139c0a04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.473+0000 7f139be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f139c15ece0 0x7f139c0ca210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.473+0000 7f139be00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f139c0d3640 con 0x7f139c15ece0 2024-04-20T21:21:50.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.473+0000 7f139be00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f139c0cf4a0 con 0x7f139c0d50c0 2024-04-20T21:21:50.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.473+0000 7f139be00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f139c15ea50 con 0x7f139c0cf9c0 2024-04-20T21:21:50.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.474+0000 7f139b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f139c15ece0 0x7f139c0ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:50.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.474+0000 7f139b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f139c15ece0 0x7f139c0ca210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58194/0 (socket says 172.21.15.86:58194) 2024-04-20T21:21:50.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.474+0000 7f139b400640 1 -- 172.21.15.86:0/2134991034 learned_addr learned my addr 172.21.15.86:0/2134991034 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:50.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.474+0000 7f139aa00640 1 --2- 172.21.15.86:0/2134991034 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f139c0cf9c0 0x7f139c09ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:50.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.474+0000 7f1392000640 1 --2- 172.21.15.86:0/2134991034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f139c0d50c0 0x7f139c0a04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:50.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.474+0000 7f139b400640 1 -- 172.21.15.86:0/2134991034 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f139c0cf9c0 msgr2=0x7f139c09ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:50.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.474+0000 7f139b400640 1 --2- 172.21.15.86:0/2134991034 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f139c0cf9c0 0x7f139c09ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.474+0000 7f139b400640 1 -- 172.21.15.86:0/2134991034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f139c0d50c0 msgr2=0x7f139c0a04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:50.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.474+0000 7f139b400640 1 --2- 172.21.15.86:0/2134991034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f139c0d50c0 0x7f139c0a04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.474+0000 7f139b400640 1 -- 172.21.15.86:0/2134991034 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f139c0a0b20 con 0x7f139c15ece0 2024-04-20T21:21:50.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.474+0000 7f139b400640 1 --2- 172.21.15.86:0/2134991034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f139c15ece0 0x7f139c0ca210 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f1394009fd0 tx=0x7f139400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:50.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.475+0000 7f1393400640 1 -- 172.21.15.86:0/2134991034 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1394011830 con 0x7f139c15ece0 2024-04-20T21:21:50.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.475+0000 7f1393400640 1 -- 172.21.15.86:0/2134991034 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1394016070 con 0x7f139c15ece0 2024-04-20T21:21:50.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.475+0000 7f1393400640 1 -- 172.21.15.86:0/2134991034 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1394011bf0 con 0x7f139c15ece0 2024-04-20T21:21:50.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.475+0000 7f139be00640 1 -- 172.21.15.86:0/2134991034 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f139c0a0e00 con 0x7f139c15ece0 2024-04-20T21:21:50.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.475+0000 7f139be00640 1 -- 172.21.15.86:0/2134991034 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f139c0a13c0 con 0x7f139c15ece0 2024-04-20T21:21:50.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.476+0000 7f139be00640 1 -- 172.21.15.86:0/2134991034 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f139c04fcf0 con 0x7f139c15ece0 2024-04-20T21:21:50.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.477+0000 7f1393400640 1 -- 172.21.15.86:0/2134991034 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f139401b460 con 0x7f139c15ece0 2024-04-20T21:21:50.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.478+0000 7f1393400640 1 --2- 172.21.15.86:0/2134991034 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f137806c8f0 0x7f137806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.478+0000 7f1393400640 1 -- 172.21.15.86:0/2134991034 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f139408b890 con 0x7f139c15ece0 2024-04-20T21:21:50.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.478+0000 7f139aa00640 1 --2- 172.21.15.86:0/2134991034 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f137806c8f0 0x7f137806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:50.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.479+0000 7f139aa00640 1 --2- 172.21.15.86:0/2134991034 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f137806c8f0 0x7f137806edb0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f138c036000 tx=0x7f138c03d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:50.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.483+0000 7f1393400640 1 -- 172.21.15.86:0/2134991034 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1394059cb0 con 0x7f139c15ece0 2024-04-20T21:21:50.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.718+0000 7f139be00640 1 -- 172.21.15.86:0/2134991034 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f139c08e320 con 0x7f139c15ece0 2024-04-20T21:21:50.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.720+0000 7f1393400640 1 -- 172.21.15.86:0/2134991034 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+5205 (secure 0 0 0) 0x7f13940595b0 con 0x7f139c15ece0 2024-04-20T21:21:50.720 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:21:50.720 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4913,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3933408964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3933408964},{"type":"v1","addr":"172.21.15.144:6833","nonce":3933408964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5221,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/1522070692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1522070692},{"type":"v1","addr":"172.21.15.144:6835","nonce":1522070692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":6132,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/1352073341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1352073341},{"type":"v1","addr":"172.21.15.86:6836","nonce":1352073341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:21:07.502800+0000","modified":"2024-04-20T21:21:24.118743+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6135},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6135":{"gid":6135,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":7,"addr":"172.21.15.86:6837/3008383261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3008383261},{"type":"v1","addr":"172.21.15.86:6837","nonce":3008383261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6135,"qdb_cluster":[6135]},"id":4}]} 2024-04-20T21:21:50.721 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 33 2024-04-20T21:21:50.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.723+0000 7f139be00640 1 -- 172.21.15.86:0/2134991034 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f137806c8f0 msgr2=0x7f137806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:50.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.723+0000 7f139be00640 1 --2- 172.21.15.86:0/2134991034 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f137806c8f0 0x7f137806edb0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f138c036000 tx=0x7f138c03d040 comp rx=0 tx=0).stop 2024-04-20T21:21:50.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.723+0000 7f139be00640 1 -- 172.21.15.86:0/2134991034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f139c15ece0 msgr2=0x7f139c0ca210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:50.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.723+0000 7f139be00640 1 --2- 172.21.15.86:0/2134991034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f139c15ece0 0x7f139c0ca210 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f1394009fd0 tx=0x7f139400b8e0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.723+0000 7f139be00640 1 -- 172.21.15.86:0/2134991034 shutdown_connections 2024-04-20T21:21:50.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.723+0000 7f139be00640 1 --2- 172.21.15.86:0/2134991034 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f137806c8f0 0x7f137806edb0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.723+0000 7f139be00640 1 --2- 172.21.15.86:0/2134991034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f139c15ece0 0x7f139c0ca210 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.723+0000 7f139be00640 1 --2- 172.21.15.86:0/2134991034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f139c0d50c0 0x7f139c0a04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.723+0000 7f139be00640 1 --2- 172.21.15.86:0/2134991034 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f139c0cf9c0 0x7f139c09ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.723+0000 7f139be00640 1 -- 172.21.15.86:0/2134991034 >> 172.21.15.86:0/2134991034 conn(0x7f139c08bf70 msgr2=0x7f139c15d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:50.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.723+0000 7f139be00640 1 -- 172.21.15.86:0/2134991034 shutdown_connections 2024-04-20T21:21:50.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.723+0000 7f139be00640 1 -- 172.21.15.86:0/2134991034 wait complete. 2024-04-20T21:21:50.741 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-20T21:21:50.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.918+0000 7fee90e00640 1 Processor -- start 2024-04-20T21:21:50.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.918+0000 7fee90e00640 1 -- start start 2024-04-20T21:21:50.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.920+0000 7fee90e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fee8c145250 0x7fee8c06efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.920+0000 7fee90e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee8c072450 0x7fee8c15cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.920+0000 7fee90e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee8c06f500 con 0x7fee8c145f00 2024-04-20T21:21:50.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.920+0000 7fee90e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fee8c160410 con 0x7fee8c145250 2024-04-20T21:21:50.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.920+0000 7fee90e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fee8c1606f0 con 0x7fee8c072450 2024-04-20T21:21:50.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.920+0000 7fee82a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fee8c145f00 0x7fee8c071d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:49308/0 (socket says 172.21.15.86:49308) 2024-04-20T21:21:50.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.920+0000 7fee82a00640 1 -- 172.21.15.86:0/3741496528 learned_addr learned my addr 172.21.15.86:0/3741496528 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:21:50.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.920+0000 7fee8b400640 1 --2- 172.21.15.86:0/3741496528 >> v2:172.21.15.144:3300/0 conn(0x7fee8c145250 0x7fee8c06efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:50.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.920+0000 7fee8be00640 1 --2- 172.21.15.86:0/3741496528 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee8c072450 0x7fee8c15cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:50.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.921+0000 7fee8aa00640 1 -- 172.21.15.86:0/3741496528 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 904640471 0 0) 0x7fee74001110 con 0x7fee8c145f00 2024-04-20T21:21:50.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.921+0000 7fee8aa00640 1 -- 172.21.15.86:0/3741496528 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee70003520 con 0x7fee8c145f00 2024-04-20T21:21:50.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.921+0000 7fee8aa00640 1 -- 172.21.15.86:0/3741496528 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1390218573 0 0) 0x7fee70003520 con 0x7fee8c145f00 2024-04-20T21:21:50.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.921+0000 7fee8aa00640 1 -- 172.21.15.86:0/3741496528 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee8c072450 msgr2=0x7fee8c15cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:50.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.921+0000 7fee8aa00640 1 --2- 172.21.15.86:0/3741496528 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee8c072450 0x7fee8c15cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.921+0000 7fee8aa00640 1 -- 172.21.15.86:0/3741496528 >> v2:172.21.15.144:3300/0 conn(0x7fee8c145250 msgr2=0x7fee8c06efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:50.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.921+0000 7fee8aa00640 1 --2- 172.21.15.86:0/3741496528 >> v2:172.21.15.144:3300/0 conn(0x7fee8c145250 0x7fee8c06efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.921+0000 7fee8aa00640 1 -- 172.21.15.86:0/3741496528 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee8c1609d0 con 0x7fee8c145f00 2024-04-20T21:21:50.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.921+0000 7fee8aa00640 1 -- 172.21.15.86:0/3741496528 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fee74001840 con 0x7fee8c145f00 2024-04-20T21:21:50.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.921+0000 7fee8aa00640 1 -- 172.21.15.86:0/3741496528 >> v1:172.21.15.86:6789/0 conn(0x7fee8c145f00 legacy=0x7fee8c071d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:50.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.921+0000 7fee8aa00640 1 --2- 172.21.15.86:0/3741496528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee70003a20 0x7fee70003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.921+0000 7fee8aa00640 1 --2- 172.21.15.86:0/3741496528 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee8c072450 0x7fee70004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.921+0000 7fee8aa00640 1 --2- 172.21.15.86:0/3741496528 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee70004d40 0x7fee700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.921+0000 7fee8aa00640 1 -- 172.21.15.86:0/3741496528 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fee8c160410 con 0x7fee70003a20 2024-04-20T21:21:50.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.921+0000 7fee8aa00640 1 -- 172.21.15.86:0/3741496528 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fee8c1606f0 con 0x7fee70004d40 2024-04-20T21:21:50.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.921+0000 7fee8aa00640 1 -- 172.21.15.86:0/3741496528 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fee7000baa0 con 0x7fee8c072450 2024-04-20T21:21:50.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.921+0000 7fee8b400640 1 --2- 172.21.15.86:0/3741496528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee70003a20 0x7fee70003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:50.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.922+0000 7fee8be00640 1 --2- 172.21.15.86:0/3741496528 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee70004d40 0x7fee700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:50.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.922+0000 7fee82a00640 1 --2- 172.21.15.86:0/3741496528 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee8c072450 0x7fee70004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:50.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.922+0000 7fee8b400640 1 -- 172.21.15.86:0/3741496528 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee8c072450 msgr2=0x7fee70004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:50.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.922+0000 7fee8b400640 1 --2- 172.21.15.86:0/3741496528 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee8c072450 0x7fee70004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.922+0000 7fee8b400640 1 -- 172.21.15.86:0/3741496528 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee70004d40 msgr2=0x7fee700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:50.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.922+0000 7fee8b400640 1 --2- 172.21.15.86:0/3741496528 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee70004d40 0x7fee700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.922+0000 7fee8b400640 1 -- 172.21.15.86:0/3741496528 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee70003520 con 0x7fee70003a20 2024-04-20T21:21:50.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.922+0000 7fee82a00640 1 --2- 172.21.15.86:0/3741496528 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee8c072450 0x7fee70004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:21:50.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.923+0000 7fee8b400640 1 --2- 172.21.15.86:0/3741496528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee70003a20 0x7fee70003e40 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fee7c009a90 tx=0x7fee70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a0b05f58b3dac03 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:50.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.923+0000 7fee8aa00640 1 -- 172.21.15.86:0/3741496528 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7c039070 con 0x7fee70003a20 2024-04-20T21:21:50.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.923+0000 7fee8aa00640 1 -- 172.21.15.86:0/3741496528 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee7c02c6d0 con 0x7fee70003a20 2024-04-20T21:21:50.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.923+0000 7fee8aa00640 1 -- 172.21.15.86:0/3741496528 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7c009690 con 0x7fee70003a20 2024-04-20T21:21:50.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.924+0000 7fee90e00640 1 -- 172.21.15.86:0/3741496528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee70003a20 msgr2=0x7fee70003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:50.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.924+0000 7fee90e00640 1 --2- 172.21.15.86:0/3741496528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee70003a20 0x7fee70003e40 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fee7c009a90 tx=0x7fee70005830 comp rx=0 tx=0).stop 2024-04-20T21:21:50.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.924+0000 7fee8be00640 1 -- 172.21.15.86:0/3741496528 reap_dead start 2024-04-20T21:21:50.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.924+0000 7fee90e00640 1 -- 172.21.15.86:0/3741496528 shutdown_connections 2024-04-20T21:21:50.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.924+0000 7fee90e00640 1 -- 172.21.15.86:0/3741496528 >> 172.21.15.86:0/3741496528 conn(0x7fee8c067cf0 msgr2=0x7fee8c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:50.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.924+0000 7fee90e00640 1 -- 172.21.15.86:0/3741496528 shutdown_connections 2024-04-20T21:21:50.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.924+0000 7fee90e00640 1 -- 172.21.15.86:0/3741496528 wait complete. 2024-04-20T21:21:50.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.926+0000 7fee90e00640 1 Processor -- start 2024-04-20T21:21:50.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.926+0000 7fee90e00640 1 -- start start 2024-04-20T21:21:50.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.927+0000 7fee90e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee70003a20 0x7fee8c140450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.927+0000 7fee90e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee8c15bc20 0x7fee8c1f8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.927+0000 7fee90e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee8c15c0a0 0x7fee8c1f87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.927+0000 7fee90e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fee8c1457b0 con 0x7fee70003a20 2024-04-20T21:21:50.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.927+0000 7fee90e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fee8c145630 con 0x7fee8c15c0a0 2024-04-20T21:21:50.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.927+0000 7fee90e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fee8c15cdf0 con 0x7fee8c15bc20 2024-04-20T21:21:50.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.927+0000 7fee8b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee70003a20 0x7fee8c140450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:50.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.927+0000 7fee8b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee70003a20 0x7fee8c140450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58218/0 (socket says 172.21.15.86:58218) 2024-04-20T21:21:50.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.927+0000 7fee8b400640 1 -- 172.21.15.86:0/947798367 learned_addr learned my addr 172.21.15.86:0/947798367 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:50.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.927+0000 7fee8b400640 1 -- 172.21.15.86:0/947798367 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee8c15bc20 msgr2=0x7fee8c1f8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:50.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.927+0000 7fee8b400640 1 --2- 172.21.15.86:0/947798367 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee8c15bc20 0x7fee8c1f8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.927+0000 7fee8b400640 1 -- 172.21.15.86:0/947798367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee8c15c0a0 msgr2=0x7fee8c1f87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:50.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.927+0000 7fee8b400640 1 --2- 172.21.15.86:0/947798367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee8c15c0a0 0x7fee8c1f87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.927+0000 7fee8b400640 1 -- 172.21.15.86:0/947798367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee8c1f90c0 con 0x7fee70003a20 2024-04-20T21:21:50.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.928+0000 7fee8b400640 1 --2- 172.21.15.86:0/947798367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee70003a20 0x7fee8c140450 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fee7c009bb0 tx=0x7fee7c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:50.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.928+0000 7fee83e00640 1 -- 172.21.15.86:0/947798367 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7c039050 con 0x7fee70003a20 2024-04-20T21:21:50.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.928+0000 7fee83e00640 1 -- 172.21.15.86:0/947798367 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee7c045070 con 0x7fee70003a20 2024-04-20T21:21:50.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.928+0000 7fee83e00640 1 -- 172.21.15.86:0/947798367 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7c003850 con 0x7fee70003a20 2024-04-20T21:21:50.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.928+0000 7fee90e00640 1 -- 172.21.15.86:0/947798367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee8c0ca4d0 con 0x7fee70003a20 2024-04-20T21:21:50.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.928+0000 7fee90e00640 1 -- 172.21.15.86:0/947798367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee8c0caa90 con 0x7fee70003a20 2024-04-20T21:21:50.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.930+0000 7fee83e00640 1 -- 172.21.15.86:0/947798367 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fee7c0039f0 con 0x7fee70003a20 2024-04-20T21:21:50.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.931+0000 7fee83e00640 1 --2- 172.21.15.86:0/947798367 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fee6806c8f0 0x7fee6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.931+0000 7fee83e00640 1 -- 172.21.15.86:0/947798367 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fee7c0ad620 con 0x7fee70003a20 2024-04-20T21:21:50.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.931+0000 7fee82a00640 1 --2- 172.21.15.86:0/947798367 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fee6806c8f0 0x7fee6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:50.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.932+0000 7fee951b3740 1 Processor -- start 2024-04-20T21:21:50.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.932+0000 7fee951b3740 1 -- start start 2024-04-20T21:21:50.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.932+0000 7fee82a00640 1 --2- 172.21.15.86:0/947798367 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fee6806c8f0 0x7fee6806edb0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fee74000c00 tx=0x7fee74009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:50.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.933+0000 7fee951b3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ec13bd0b0 0x560ec1390050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.933+0000 7fee951b3740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560ec13bd490 0x560ec13bfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.933+0000 7fee951b3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ec1390590 0x560ec13c0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.933+0000 7fee951b3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560ec12fb060 con 0x560ec1390590 2024-04-20T21:21:50.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.933+0000 7fee951b3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560ec12a0f00 con 0x560ec13bd0b0 2024-04-20T21:21:50.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.933+0000 7fee951b3740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x560ec12f7020 con 0x560ec13bd490 2024-04-20T21:21:50.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.933+0000 7fee8be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ec1390590 0x560ec13c0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:50.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.933+0000 7fee8be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ec1390590 0x560ec13c0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58222/0 (socket says 172.21.15.86:58222) 2024-04-20T21:21:50.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.933+0000 7fee8be00640 1 -- 172.21.15.86:0/367644769 learned_addr learned my addr 172.21.15.86:0/367644769 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:50.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.933+0000 7fee8be00640 1 -- 172.21.15.86:0/367644769 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560ec13bd490 msgr2=0x560ec13bfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:50.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.933+0000 7fee8be00640 1 --2- 172.21.15.86:0/367644769 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560ec13bd490 0x560ec13bfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.933+0000 7fee8be00640 1 -- 172.21.15.86:0/367644769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ec13bd0b0 msgr2=0x560ec1390050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:50.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.933+0000 7fee8be00640 1 --2- 172.21.15.86:0/367644769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ec13bd0b0 0x560ec1390050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.933+0000 7fee8be00640 1 -- 172.21.15.86:0/367644769 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ec1289a30 con 0x560ec1390590 2024-04-20T21:21:50.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.934+0000 7fee8be00640 1 --2- 172.21.15.86:0/367644769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ec1390590 0x560ec13c0380 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fee8c145eb0 tx=0x7fee84007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ac8f88055435cd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:50.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.934+0000 7fee7be00640 1 -- 172.21.15.86:0/367644769 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee84007e50 con 0x560ec1390590 2024-04-20T21:21:50.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.934+0000 7fee7be00640 1 -- 172.21.15.86:0/367644769 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee8401a070 con 0x560ec1390590 2024-04-20T21:21:50.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.934+0000 7fee7be00640 1 -- 172.21.15.86:0/367644769 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee84003ba0 con 0x560ec1390590 2024-04-20T21:21:50.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.935+0000 7fee951b3740 1 -- 172.21.15.86:0/367644769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ec1390590 msgr2=0x560ec13c0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:50.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.935+0000 7fee951b3740 1 --2- 172.21.15.86:0/367644769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ec1390590 0x560ec13c0380 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fee8c145eb0 tx=0x7fee84007700 comp rx=0 tx=0).stop 2024-04-20T21:21:50.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.935+0000 7fee951b3740 1 -- 172.21.15.86:0/367644769 shutdown_connections 2024-04-20T21:21:50.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.935+0000 7fee951b3740 1 --2- 172.21.15.86:0/367644769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ec1390590 0x560ec13c0380 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.935+0000 7fee951b3740 1 --2- 172.21.15.86:0/367644769 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560ec13bd490 0x560ec13bfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.935+0000 7fee951b3740 1 --2- 172.21.15.86:0/367644769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ec13bd0b0 0x560ec1390050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.935+0000 7fee951b3740 1 -- 172.21.15.86:0/367644769 >> 172.21.15.86:0/367644769 conn(0x560ec13922c0 msgr2=0x560ec13dae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:50.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.935+0000 7fee951b3740 1 -- 172.21.15.86:0/367644769 shutdown_connections 2024-04-20T21:21:50.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.935+0000 7fee951b3740 1 -- 172.21.15.86:0/367644769 wait complete. 2024-04-20T21:21:50.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.935+0000 7fee951b3740 1 -- start start 2024-04-20T21:21:50.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.936+0000 7fee951b3740 1 Processor -- start 2024-04-20T21:21:50.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.937+0000 7fee951b3740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560ec13bd0b0 0x560ec13bc830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.937+0000 7fee951b3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ec13bd490 0x560ec13d5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.937+0000 7fee951b3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ec13d6350 0x560ec13d2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.937+0000 7fee951b3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560ec1394860 con 0x560ec13bd490 2024-04-20T21:21:50.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.937+0000 7fee951b3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560ec12b43d0 con 0x560ec13d6350 2024-04-20T21:21:50.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.937+0000 7fee951b3740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x560ec1393c50 con 0x560ec13bd0b0 2024-04-20T21:21:50.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.937+0000 7fee951b3740 10 client.0 _ll_register_callbacks cb 0x560ec1282960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:21:50.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.937+0000 7fee82a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ec13bd490 0x560ec13d5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:50.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.937+0000 7fee82a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ec13bd490 0x560ec13d5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58238/0 (socket says 172.21.15.86:58238) 2024-04-20T21:21:50.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.937+0000 7fee82a00640 1 -- 172.21.15.86:0/811659153 learned_addr learned my addr 172.21.15.86:0/811659153 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:50.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.937+0000 7fee82a00640 1 -- 172.21.15.86:0/811659153 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560ec13bd0b0 msgr2=0x560ec13bc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:50.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.937+0000 7fee82a00640 1 --2- 172.21.15.86:0/811659153 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560ec13bd0b0 0x560ec13bc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.937+0000 7fee82a00640 1 -- 172.21.15.86:0/811659153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ec13d6350 msgr2=0x560ec13d2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:50.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.937+0000 7fee82a00640 1 --2- 172.21.15.86:0/811659153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ec13d6350 0x560ec13d2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.937+0000 7fee82a00640 1 -- 172.21.15.86:0/811659153 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ec10d97c0 con 0x560ec13bd490 2024-04-20T21:21:50.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.938+0000 7fee82a00640 1 --2- 172.21.15.86:0/811659153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ec13bd490 0x560ec13d5e10 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fee74005a70 tx=0x7fee74005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=832b2344fff23 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:50.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.938+0000 7fee78c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:21:50.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.938+0000 7fee78c00640 1 -- 172.21.15.86:0/811659153 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee74021620 con 0x560ec13bd490 2024-04-20T21:21:50.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.938+0000 7fee78c00640 1 -- 172.21.15.86:0/811659153 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fee74012410 con 0x560ec13bd490 2024-04-20T21:21:50.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.938+0000 7fee951b3740 1 -- 172.21.15.86:0/811659153 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee8401a070 con 0x560ec13bd490 2024-04-20T21:21:50.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.938+0000 7fee78c00640 1 -- 172.21.15.86:0/811659153 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fee74012610 con 0x560ec13bd490 2024-04-20T21:21:50.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.938+0000 7fee951b3740 10 client.6405 fetch_fsmap learned FSMap version 33 2024-04-20T21:21:50.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.938+0000 7fee951b3740 1 -- 172.21.15.86:0/811659153 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x560ec1139f50 con 0x560ec13bd490 2024-04-20T21:21:50.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.938+0000 7fee78c00640 1 -- 172.21.15.86:0/811659153 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee8401a070 con 0x560ec13bd490 2024-04-20T21:21:50.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.938+0000 7fee78c00640 1 -- 172.21.15.86:0/811659153 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 33) ==== 3495+0+0 (secure 0 0 0) 0x7fee74029050 con 0x560ec13bd490 2024-04-20T21:21:50.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.938+0000 7fee951b3740 10 client.6405 fetch_fsmap finished waiting for FSMap version 33 2024-04-20T21:21:50.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.939+0000 7fee951b3740 10 client.6405 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-20T21:21:50.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.939+0000 7fee951b3740 20 client.6405 populate_metadata read hostname 'smithi086' 2024-04-20T21:21:50.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.939+0000 7fee951b3740 1 --2- 172.21.15.86:0/811659153 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x560ec13eb640 0x560ec13f9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.939+0000 7fee951b3740 4 client.6405 mds_command: new command op to 6135 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:21:50.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.939+0000 7fee951b3740 1 -- 172.21.15.86:0/811659153 --> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ec1308a80 con 0x560ec13eb640 2024-04-20T21:21:50.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.939+0000 7fee8b400640 1 --2- 172.21.15.86:0/811659153 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x560ec13eb640 0x560ec13f9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:50.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.939+0000 7fee8b400640 1 --2- 172.21.15.86:0/811659153 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x560ec13eb640 0x560ec13f9070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b4ccbc8307e764 server_cookie=b70c77944ace5761 in_seq=0 out_seq=0 2024-04-20T21:21:50.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.939+0000 7fee78c00640 10 client.6405 ms_handle_connect on v2:172.21.15.86:6835/3008383261 2024-04-20T21:21:50.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.940+0000 7fee78c00640 1 -- 172.21.15.86:0/811659153 <== mds.0 v2:172.21.15.86:6835/3008383261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560ec1308a80 con 0x560ec13eb640 2024-04-20T21:21:50.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.940+0000 7fee78c00640 10 client.6405 handle_command_reply: tid=0 2024-04-20T21:21:50.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.940+0000 7fee951b3740 1 client.6405 shutdown 2024-04-20T21:21:50.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.940+0000 7fee951b3740 1 -- 172.21.15.86:0/811659153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ec13bd490 msgr2=0x560ec13d5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:50.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.940+0000 7fee951b3740 1 --2- 172.21.15.86:0/811659153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ec13bd490 0x560ec13d5e10 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fee74005a70 tx=0x7fee74005e70 comp rx=0 tx=0).stop 2024-04-20T21:21:50.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.940+0000 7fee951b3740 1 -- 172.21.15.86:0/811659153 shutdown_connections 2024-04-20T21:21:50.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.940+0000 7fee951b3740 1 --2- 172.21.15.86:0/811659153 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x560ec13eb640 0x560ec13f9070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.940+0000 7fee951b3740 1 --2- 172.21.15.86:0/811659153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ec13d6350 0x560ec13d2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.940+0000 7fee951b3740 1 --2- 172.21.15.86:0/811659153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ec13bd490 0x560ec13d5e10 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.940+0000 7fee78c00640 0 client.6405 ms_handle_reset on v2:172.21.15.86:6835/3008383261 2024-04-20T21:21:50.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.940+0000 7fee951b3740 1 --2- 172.21.15.86:0/811659153 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560ec13bd0b0 0x560ec13bc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.940+0000 7fee951b3740 1 -- 172.21.15.86:0/811659153 >> 172.21.15.86:0/811659153 conn(0x560ec1390590 msgr2=0x560ec13d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:50.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.941+0000 7fee951b3740 1 -- 172.21.15.86:0/811659153 shutdown_connections 2024-04-20T21:21:50.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.941+0000 7fee951b3740 1 -- 172.21.15.86:0/811659153 wait complete. 2024-04-20T21:21:50.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.941+0000 7fee951b3740 20 client.6405 trim_cache size 0 max 16384 2024-04-20T21:21:50.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.955+0000 7fee951b3740 1 Processor -- start 2024-04-20T21:21:50.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.955+0000 7fee951b3740 1 -- start start 2024-04-20T21:21:50.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.956+0000 7fee951b3740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560ec13bd0b0 0x560ec1394770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.956+0000 7fee951b3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ec13bd490 0x560ec13d51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.956+0000 7fee951b3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ec13d6350 0x560ec13ea5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.956+0000 7fee951b3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fee8c200620 con 0x560ec13bd490 2024-04-20T21:21:50.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.956+0000 7fee951b3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fee8c13bf30 con 0x560ec13d6350 2024-04-20T21:21:50.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.956+0000 7fee951b3740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fee8c13c580 con 0x560ec13bd0b0 2024-04-20T21:21:50.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.956+0000 7fee82a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ec13bd490 0x560ec13d51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:50.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.956+0000 7fee82a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ec13bd490 0x560ec13d51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58240/0 (socket says 172.21.15.86:58240) 2024-04-20T21:21:50.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.956+0000 7fee82a00640 1 -- 172.21.15.86:0/2703809797 learned_addr learned my addr 172.21.15.86:0/2703809797 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:50.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.956+0000 7fee82a00640 1 -- 172.21.15.86:0/2703809797 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560ec13bd0b0 msgr2=0x560ec1394770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:50.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.956+0000 7fee82a00640 1 --2- 172.21.15.86:0/2703809797 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560ec13bd0b0 0x560ec1394770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.956+0000 7fee82a00640 1 -- 172.21.15.86:0/2703809797 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ec13d6350 msgr2=0x560ec13ea5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:50.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.956+0000 7fee82a00640 1 --2- 172.21.15.86:0/2703809797 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ec13d6350 0x560ec13ea5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.956+0000 7fee82a00640 1 -- 172.21.15.86:0/2703809797 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ec0fcd0b0 con 0x560ec13bd490 2024-04-20T21:21:50.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.957+0000 7fee82a00640 1 --2- 172.21.15.86:0/2703809797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ec13bd490 0x560ec13d51c0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x560ec13d24f0 tx=0x7fee7401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9723b34ef515a869 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:50.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.957+0000 7fee7b400640 1 -- 172.21.15.86:0/2703809797 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee74012cd0 con 0x560ec13bd490 2024-04-20T21:21:50.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.957+0000 7fee7b400640 1 -- 172.21.15.86:0/2703809797 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee7402b470 con 0x560ec13bd490 2024-04-20T21:21:50.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.957+0000 7fee7b400640 1 -- 172.21.15.86:0/2703809797 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7402b820 con 0x560ec13bd490 2024-04-20T21:21:50.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.958+0000 7fee951b3740 1 -- 172.21.15.86:0/2703809797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ec13bd490 msgr2=0x560ec13d51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:50.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.958+0000 7fee951b3740 1 --2- 172.21.15.86:0/2703809797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ec13bd490 0x560ec13d51c0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x560ec13d24f0 tx=0x7fee7401c000 comp rx=0 tx=0).stop 2024-04-20T21:21:50.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.958+0000 7fee951b3740 1 -- 172.21.15.86:0/2703809797 shutdown_connections 2024-04-20T21:21:50.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.958+0000 7fee951b3740 1 --2- 172.21.15.86:0/2703809797 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ec13d6350 0x560ec13ea5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.958+0000 7fee951b3740 1 --2- 172.21.15.86:0/2703809797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ec13bd490 0x560ec13d51c0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.958+0000 7fee951b3740 1 --2- 172.21.15.86:0/2703809797 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560ec13bd0b0 0x560ec1394770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.958+0000 7fee951b3740 1 -- 172.21.15.86:0/2703809797 >> 172.21.15.86:0/2703809797 conn(0x560ec1390590 msgr2=0x560ec138b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:50.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.958+0000 7fee951b3740 1 -- 172.21.15.86:0/2703809797 shutdown_connections 2024-04-20T21:21:50.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.958+0000 7fee951b3740 1 -- 172.21.15.86:0/2703809797 wait complete. 2024-04-20T21:21:50.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.958+0000 7fee951b3740 1 -- start start 2024-04-20T21:21:50.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.959+0000 7fee951b3740 1 Processor -- start 2024-04-20T21:21:50.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.960+0000 7fee951b3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ec13bd0b0 0x560ec1394540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.960+0000 7fee951b3740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ec13bd490 0x560ec14551a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.960+0000 7fee951b3740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560ec13d6350 0x560ec13fd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.960+0000 7fee951b3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560ec13eb4c0 con 0x560ec13bd0b0 2024-04-20T21:21:50.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.960+0000 7fee951b3740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560ec13eafd0 con 0x560ec13bd490 2024-04-20T21:21:50.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.960+0000 7fee951b3740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x560ec13ebb30 con 0x560ec13d6350 2024-04-20T21:21:50.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.960+0000 7fee951b3740 10 client.0 _ll_register_callbacks cb 0x560ec1282960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:21:50.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.960+0000 7fee8b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ec13bd0b0 0x560ec1394540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:50.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.960+0000 7fee8b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ec13bd0b0 0x560ec1394540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58250/0 (socket says 172.21.15.86:58250) 2024-04-20T21:21:50.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.960+0000 7fee8b400640 1 -- 172.21.15.86:0/2805209374 learned_addr learned my addr 172.21.15.86:0/2805209374 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:50.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.960+0000 7fee8b400640 1 -- 172.21.15.86:0/2805209374 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560ec13d6350 msgr2=0x560ec13fd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:50.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.960+0000 7fee8b400640 1 --2- 172.21.15.86:0/2805209374 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560ec13d6350 0x560ec13fd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.960+0000 7fee8b400640 1 -- 172.21.15.86:0/2805209374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ec13bd490 msgr2=0x560ec14551a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:50.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.960+0000 7fee8b400640 1 --2- 172.21.15.86:0/2805209374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ec13bd490 0x560ec14551a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.960+0000 7fee8b400640 1 -- 172.21.15.86:0/2805209374 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ec1394a80 con 0x560ec13bd0b0 2024-04-20T21:21:50.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.960+0000 7fee8b400640 1 --2- 172.21.15.86:0/2805209374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ec13bd0b0 0x560ec1394540 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fee7c0abb40 tx=0x7fee7c07bb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46689d53e2c1cbc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:50.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.960+0000 7fee78c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:21:50.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.961+0000 7fee78c00640 1 -- 172.21.15.86:0/2805209374 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7c0aa060 con 0x560ec13bd0b0 2024-04-20T21:21:50.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.961+0000 7fee78c00640 1 -- 172.21.15.86:0/2805209374 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fee7c003ce0 con 0x560ec13bd0b0 2024-04-20T21:21:50.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.961+0000 7fee951b3740 1 -- 172.21.15.86:0/2805209374 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee7402b470 con 0x560ec13bd0b0 2024-04-20T21:21:50.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.961+0000 7fee78c00640 1 -- 172.21.15.86:0/2805209374 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fee7c098050 con 0x560ec13bd0b0 2024-04-20T21:21:50.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.961+0000 7fee951b3740 10 client.6411 fetch_fsmap learned FSMap version 33 2024-04-20T21:21:50.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.961+0000 7fee951b3740 1 -- 172.21.15.86:0/2805209374 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x560ec1455b20 con 0x560ec13bd0b0 2024-04-20T21:21:50.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.961+0000 7fee78c00640 1 -- 172.21.15.86:0/2805209374 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee7402b470 con 0x560ec13bd0b0 2024-04-20T21:21:50.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.961+0000 7fee78c00640 1 -- 172.21.15.86:0/2805209374 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 33) ==== 3495+0+0 (secure 0 0 0) 0x7fee7c0779b0 con 0x560ec13bd0b0 2024-04-20T21:21:50.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.961+0000 7fee951b3740 10 client.6411 fetch_fsmap finished waiting for FSMap version 33 2024-04-20T21:21:50.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.961+0000 7fee951b3740 10 client.6411 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-20T21:21:50.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.961+0000 7fee951b3740 20 client.6411 populate_metadata read hostname 'smithi086' 2024-04-20T21:21:50.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.961+0000 7fee951b3740 1 --2- 172.21.15.86:0/2805209374 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x560ec13fd5d0 0x560ec13c0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:50.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.961+0000 7fee951b3740 4 client.6411 mds_command: new command op to 6135 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-20T21:21:50.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.961+0000 7fee951b3740 1 -- 172.21.15.86:0/2805209374 --> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x560ec13028f0 con 0x560ec13fd5d0 2024-04-20T21:21:50.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.962+0000 7fee82a00640 1 --2- 172.21.15.86:0/2805209374 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x560ec13fd5d0 0x560ec13c0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:50.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.962+0000 7fee82a00640 1 --2- 172.21.15.86:0/2805209374 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x560ec13fd5d0 0x560ec13c0b80 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2beb968ec025210a server_cookie=6588f7aaf055c8d3 in_seq=0 out_seq=0 2024-04-20T21:21:50.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.962+0000 7fee78c00640 10 client.6411 ms_handle_connect on v2:172.21.15.86:6835/3008383261 2024-04-20T21:21:50.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.962+0000 7fee78c00640 1 -- 172.21.15.86:0/2805209374 <== mds.0 v2:172.21.15.86:6835/3008383261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+141 (crc 0 0 0) 0x560ec13028f0 con 0x560ec13fd5d0 2024-04-20T21:21:50.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.962+0000 7fee78c00640 10 client.6411 handle_command_reply: tid=0 2024-04-20T21:21:50.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.962+0000 7fee951b3740 1 client.6411 shutdown 2024-04-20T21:21:50.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.962+0000 7fee951b3740 1 -- 172.21.15.86:0/2805209374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ec13bd0b0 msgr2=0x560ec1394540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:50.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.962+0000 7fee951b3740 1 --2- 172.21.15.86:0/2805209374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ec13bd0b0 0x560ec1394540 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fee7c0abb40 tx=0x7fee7c07bb00 comp rx=0 tx=0).stop 2024-04-20T21:21:50.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.962+0000 7fee951b3740 1 -- 172.21.15.86:0/2805209374 shutdown_connections 2024-04-20T21:21:50.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.962+0000 7fee951b3740 1 --2- 172.21.15.86:0/2805209374 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x560ec13fd5d0 0x560ec13c0b80 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.963+0000 7fee951b3740 1 --2- 172.21.15.86:0/2805209374 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560ec13d6350 0x560ec13fd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.963+0000 7fee951b3740 1 --2- 172.21.15.86:0/2805209374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ec13bd490 0x560ec14551a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.963+0000 7fee951b3740 1 --2- 172.21.15.86:0/2805209374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ec13bd0b0 0x560ec1394540 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.963+0000 7fee78c00640 0 client.6411 ms_handle_reset on v2:172.21.15.86:6835/3008383261 2024-04-20T21:21:50.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.963+0000 7fee951b3740 1 -- 172.21.15.86:0/2805209374 >> 172.21.15.86:0/2805209374 conn(0x560ec1390590 msgr2=0x560ec138b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:50.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.963+0000 7fee951b3740 1 -- 172.21.15.86:0/2805209374 shutdown_connections 2024-04-20T21:21:50.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.963+0000 7fee951b3740 1 -- 172.21.15.86:0/2805209374 wait complete. 2024-04-20T21:21:50.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.963+0000 7fee951b3740 20 client.6411 trim_cache size 0 max 16384 2024-04-20T21:21:50.967 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:21:50.967 INFO:teuthology.orchestra.run.smithi086.stdout:[ 2024-04-20T21:21:50.967 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:21:50.967 INFO:teuthology.orchestra.run.smithi086.stdout: "damage_type": "backtrace", 2024-04-20T21:21:50.967 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 3174509567, 2024-04-20T21:21:50.967 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 1099511627777, 2024-04-20T21:21:50.967 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/dir_test/file" 2024-04-20T21:21:50.967 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:21:50.967 INFO:teuthology.orchestra.run.smithi086.stdout:] 2024-04-20T21:21:50.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.963+0000 7fee90e00640 1 -- 172.21.15.86:0/947798367 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fee6806c8f0 msgr2=0x7fee6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:50.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.963+0000 7fee90e00640 1 --2- 172.21.15.86:0/947798367 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fee6806c8f0 0x7fee6806edb0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fee74000c00 tx=0x7fee74009040 comp rx=0 tx=0).stop 2024-04-20T21:21:50.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.963+0000 7fee90e00640 1 -- 172.21.15.86:0/947798367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee70003a20 msgr2=0x7fee8c140450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:50.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.963+0000 7fee90e00640 1 --2- 172.21.15.86:0/947798367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee70003a20 0x7fee8c140450 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fee7c009bb0 tx=0x7fee7c004730 comp rx=0 tx=0).stop 2024-04-20T21:21:50.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.963+0000 7fee90e00640 1 -- 172.21.15.86:0/947798367 shutdown_connections 2024-04-20T21:21:50.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.963+0000 7fee90e00640 1 --2- 172.21.15.86:0/947798367 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fee6806c8f0 0x7fee6806edb0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.964+0000 7fee90e00640 1 --2- 172.21.15.86:0/947798367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee8c15c0a0 0x7fee8c1f87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.964+0000 7fee90e00640 1 --2- 172.21.15.86:0/947798367 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee8c15bc20 0x7fee8c1f8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.964+0000 7fee90e00640 1 --2- 172.21.15.86:0/947798367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee70003a20 0x7fee8c140450 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:50.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.964+0000 7fee90e00640 1 -- 172.21.15.86:0/947798367 >> 172.21.15.86:0/947798367 conn(0x7fee8c067cf0 msgr2=0x7fee8c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:50.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.964+0000 7fee90e00640 1 -- 172.21.15.86:0/947798367 shutdown_connections 2024-04-20T21:21:50.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:50.964+0000 7fee90e00640 1 -- 172.21.15.86:0/947798367 wait complete. 2024-04-20T21:21:50.979 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-20T21:21:51.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.170+0000 7f0fcda00640 1 Processor -- start 2024-04-20T21:21:51.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.170+0000 7f0fcda00640 1 -- start start 2024-04-20T21:21:51.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.172+0000 7f0fcda00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0fc815d2c0 0x7f0fc806f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:51.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.172+0000 7f0fcda00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fc806f5f0 0x7f0fc8071d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:51.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.172+0000 7f0fcda00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fc806fb70 con 0x7f0fc8072240 2024-04-20T21:21:51.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.172+0000 7f0fcda00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f0fc80726b0 con 0x7f0fc815d2c0 2024-04-20T21:21:51.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.172+0000 7f0fcda00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0fc8164210 con 0x7f0fc806f5f0 2024-04-20T21:21:51.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.173+0000 7f0fbfe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fc806f5f0 0x7f0fc8071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:51.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.173+0000 7f0fc6a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0fc815d2c0 0x7f0fc806f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:51.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.173+0000 7f0fbfe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fc806f5f0 0x7f0fc8071d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:52638/0 (socket says 172.21.15.86:52638) 2024-04-20T21:21:51.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.173+0000 7f0fbfe00640 1 -- 172.21.15.86:0/2030220132 learned_addr learned my addr 172.21.15.86:0/2030220132 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:51.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.174+0000 7f0fbfe00640 1 -- 172.21.15.86:0/2030220132 >> v2:172.21.15.144:3300/0 conn(0x7f0fc815d2c0 msgr2=0x7f0fc806f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:51.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.174+0000 7f0fbfe00640 1 --2- 172.21.15.86:0/2030220132 >> v2:172.21.15.144:3300/0 conn(0x7f0fc815d2c0 0x7f0fc806f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.174+0000 7f0fbfe00640 1 -- 172.21.15.86:0/2030220132 >> v1:172.21.15.144:6789/0 conn(0x7f0fc8072240 legacy=0x7f0fc806dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:51.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.174+0000 7f0fbfe00640 1 -- 172.21.15.86:0/2030220132 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fc815d6a0 con 0x7f0fc806f5f0 2024-04-20T21:21:51.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.175+0000 7f0fbfe00640 1 --2- 172.21.15.86:0/2030220132 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fc806f5f0 0x7f0fc8071d00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0009870 tx=0x7f0fb002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1f4e82fe39ec3ab server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:51.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.176+0000 7f0fc6000640 1 -- 172.21.15.86:0/2030220132 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb0034070 con 0x7f0fc806f5f0 2024-04-20T21:21:51.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.176+0000 7f0fc6000640 1 -- 172.21.15.86:0/2030220132 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fb0030840 con 0x7f0fc806f5f0 2024-04-20T21:21:51.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.176+0000 7f0fc6000640 1 -- 172.21.15.86:0/2030220132 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb0030c10 con 0x7f0fc806f5f0 2024-04-20T21:21:51.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.176+0000 7f0fcda00640 1 -- 172.21.15.86:0/2030220132 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fc806f5f0 msgr2=0x7f0fc8071d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:51.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.176+0000 7f0fcda00640 1 --2- 172.21.15.86:0/2030220132 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fc806f5f0 0x7f0fc8071d00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0009870 tx=0x7f0fb002b980 comp rx=0 tx=0).stop 2024-04-20T21:21:51.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.177+0000 7f0fcda00640 1 -- 172.21.15.86:0/2030220132 shutdown_connections 2024-04-20T21:21:51.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.177+0000 7f0fcda00640 1 --2- 172.21.15.86:0/2030220132 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fc806f5f0 0x7f0fc8071d00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.177+0000 7f0fcda00640 1 --2- 172.21.15.86:0/2030220132 >> v2:172.21.15.144:3300/0 conn(0x7f0fc815d2c0 0x7f0fc806f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.177+0000 7f0fcda00640 1 -- 172.21.15.86:0/2030220132 >> 172.21.15.86:0/2030220132 conn(0x7f0fc80b2360 msgr2=0x7f0fc80b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:51.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.177+0000 7f0fcda00640 1 -- 172.21.15.86:0/2030220132 shutdown_connections 2024-04-20T21:21:51.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.177+0000 7f0fcda00640 1 -- 172.21.15.86:0/2030220132 wait complete. 2024-04-20T21:21:51.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.178+0000 7f0fcda00640 1 Processor -- start 2024-04-20T21:21:51.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.178+0000 7f0fcda00640 1 -- start start 2024-04-20T21:21:51.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.179+0000 7f0fcda00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fc806f5f0 0x7f0fc815a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:51.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.179+0000 7f0fcda00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fc8072240 0x7f0fc81582d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:51.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.179+0000 7f0fcda00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fc815d2c0 0x7f0fc814bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:51.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.179+0000 7f0fcda00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc8166ec0 con 0x7f0fc8072240 2024-04-20T21:21:51.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.179+0000 7f0fcda00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc8166be0 con 0x7f0fc806f5f0 2024-04-20T21:21:51.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.179+0000 7f0fcda00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0fc81671a0 con 0x7f0fc815d2c0 2024-04-20T21:21:51.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.180+0000 7f0fbfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fc8072240 0x7f0fc81582d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:51.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.180+0000 7f0fbfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fc8072240 0x7f0fc81582d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58260/0 (socket says 172.21.15.86:58260) 2024-04-20T21:21:51.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.180+0000 7f0fbfe00640 1 -- 172.21.15.86:0/1406697024 learned_addr learned my addr 172.21.15.86:0/1406697024 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:51.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.180+0000 7f0fc6a00640 1 --2- 172.21.15.86:0/1406697024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fc806f5f0 0x7f0fc815a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:51.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.180+0000 7f0fc7400640 1 --2- 172.21.15.86:0/1406697024 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fc815d2c0 0x7f0fc814bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:51.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.180+0000 7f0fbfe00640 1 -- 172.21.15.86:0/1406697024 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fc815d2c0 msgr2=0x7f0fc814bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:51.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.180+0000 7f0fbfe00640 1 --2- 172.21.15.86:0/1406697024 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fc815d2c0 0x7f0fc814bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.180+0000 7f0fbfe00640 1 -- 172.21.15.86:0/1406697024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fc806f5f0 msgr2=0x7f0fc815a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:51.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.180+0000 7f0fbfe00640 1 --2- 172.21.15.86:0/1406697024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fc806f5f0 0x7f0fc815a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.180+0000 7f0fbfe00640 1 -- 172.21.15.86:0/1406697024 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fc8158810 con 0x7f0fc8072240 2024-04-20T21:21:51.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.180+0000 7f0fc7400640 1 --2- 172.21.15.86:0/1406697024 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fc815d2c0 0x7f0fc814bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:21:51.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.181+0000 7f0fbfe00640 1 --2- 172.21.15.86:0/1406697024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fc8072240 0x7f0fc81582d0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0000c00 tx=0x7f0fb0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:51.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.181+0000 7f0fbea00640 1 -- 172.21.15.86:0/1406697024 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb0048070 con 0x7f0fc8072240 2024-04-20T21:21:51.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.181+0000 7f0fbea00640 1 -- 172.21.15.86:0/1406697024 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fb0030640 con 0x7f0fc8072240 2024-04-20T21:21:51.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.181+0000 7f0fbea00640 1 -- 172.21.15.86:0/1406697024 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb0034040 con 0x7f0fc8072240 2024-04-20T21:21:51.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.181+0000 7f0fcda00640 1 -- 172.21.15.86:0/1406697024 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fc815ae90 con 0x7f0fc8072240 2024-04-20T21:21:51.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.181+0000 7f0fcda00640 1 -- 172.21.15.86:0/1406697024 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fc815b450 con 0x7f0fc8072240 2024-04-20T21:21:51.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.182+0000 7f0fcda00640 1 -- 172.21.15.86:0/1406697024 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fc815ce00 con 0x7f0fc8072240 2024-04-20T21:21:51.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.183+0000 7f0fbea00640 1 -- 172.21.15.86:0/1406697024 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0fb004b070 con 0x7f0fc8072240 2024-04-20T21:21:51.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.184+0000 7f0fbea00640 1 --2- 172.21.15.86:0/1406697024 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0fa406c920 0x7f0fa406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:51.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.184+0000 7f0fbea00640 1 -- 172.21.15.86:0/1406697024 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f0fb00ad9d0 con 0x7f0fc8072240 2024-04-20T21:21:51.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.184+0000 7f0fc6a00640 1 --2- 172.21.15.86:0/1406697024 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0fa406c920 0x7f0fa406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:51.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.185+0000 7f0fc6a00640 1 --2- 172.21.15.86:0/1406697024 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0fa406c920 0x7f0fa406ede0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0fc80b42f0 tx=0x7f0fb8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:51.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.189+0000 7f0fbea00640 1 -- 172.21.15.86:0/1406697024 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0fb007bdf0 con 0x7f0fc8072240 2024-04-20T21:21:51.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.429+0000 7f0fcda00640 1 -- 172.21.15.86:0/1406697024 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f0fc80689e0 con 0x7f0fc8072240 2024-04-20T21:21:51.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.430+0000 7f0fbea00640 1 -- 172.21.15.86:0/1406697024 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7f0fb0030810 con 0x7f0fc8072240 2024-04-20T21:21:51.430 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:21:51.430 INFO:teuthology.orchestra.run.smithi086.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-04-20T21:21:51.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.433+0000 7f0fcda00640 1 -- 172.21.15.86:0/1406697024 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0fa406c920 msgr2=0x7f0fa406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:51.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.433+0000 7f0fcda00640 1 --2- 172.21.15.86:0/1406697024 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0fa406c920 0x7f0fa406ede0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0fc80b42f0 tx=0x7f0fb8008040 comp rx=0 tx=0).stop 2024-04-20T21:21:51.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.433+0000 7f0fcda00640 1 -- 172.21.15.86:0/1406697024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fc8072240 msgr2=0x7f0fc81582d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:51.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.433+0000 7f0fcda00640 1 --2- 172.21.15.86:0/1406697024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fc8072240 0x7f0fc81582d0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0000c00 tx=0x7f0fb0004730 comp rx=0 tx=0).stop 2024-04-20T21:21:51.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.433+0000 7f0fcda00640 1 -- 172.21.15.86:0/1406697024 shutdown_connections 2024-04-20T21:21:51.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.433+0000 7f0fcda00640 1 --2- 172.21.15.86:0/1406697024 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0fa406c920 0x7f0fa406ede0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.433+0000 7f0fcda00640 1 --2- 172.21.15.86:0/1406697024 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fc815d2c0 0x7f0fc814bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.433+0000 7f0fcda00640 1 --2- 172.21.15.86:0/1406697024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fc8072240 0x7f0fc81582d0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.433+0000 7f0fcda00640 1 --2- 172.21.15.86:0/1406697024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fc806f5f0 0x7f0fc815a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.433+0000 7f0fcda00640 1 -- 172.21.15.86:0/1406697024 >> 172.21.15.86:0/1406697024 conn(0x7f0fc80b2360 msgr2=0x7f0fc80b4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:51.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.433+0000 7f0fcda00640 1 -- 172.21.15.86:0/1406697024 shutdown_connections 2024-04-20T21:21:51.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.433+0000 7f0fcda00640 1 -- 172.21.15.86:0/1406697024 wait complete. 2024-04-20T21:21:51.452 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-20T21:21:51.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:21:51.452 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / repair,recursive,force 2024-04-20T21:21:51.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.658+0000 7f8414200640 1 Processor -- start 2024-04-20T21:21:51.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.658+0000 7f8414200640 1 -- start start 2024-04-20T21:21:51.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.659+0000 7f8414200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f840c06f0b0 0x7f840c145a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:51.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.659+0000 7f8414200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f840c145fd0 0x7f840c15cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:51.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.659+0000 7f8414200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f840c06e010 con 0x7f840c071d00 2024-04-20T21:21:51.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.659+0000 7f8414200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f840c06e2f0 con 0x7f840c145fd0 2024-04-20T21:21:51.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.659+0000 7f8414200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f840c06e5d0 con 0x7f840c06f0b0 2024-04-20T21:21:51.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.660+0000 7f8410e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f840c06f0b0 0x7f840c145a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:51.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.660+0000 7f8411800640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f840c071d00 0x7f840c0720e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.86:43744/0 (socket says 172.21.15.86:43744) 2024-04-20T21:21:51.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.660+0000 7f8411800640 1 -- 172.21.15.86:0/3664254976 learned_addr learned my addr 172.21.15.86:0/3664254976 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:21:51.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.660+0000 7f8412200640 1 --2- 172.21.15.86:0/3664254976 >> v2:172.21.15.144:3300/0 conn(0x7f840c145fd0 0x7f840c15cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:51.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.661+0000 7f8410e00640 1 -- 172.21.15.86:0/3664254976 >> v2:172.21.15.144:3300/0 conn(0x7f840c145fd0 msgr2=0x7f840c15cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:51.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.661+0000 7f8410e00640 1 --2- 172.21.15.86:0/3664254976 >> v2:172.21.15.144:3300/0 conn(0x7f840c145fd0 0x7f840c15cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.661+0000 7f8410e00640 1 -- 172.21.15.86:0/3664254976 >> v1:172.21.15.144:6789/0 conn(0x7f840c071d00 legacy=0x7f840c0720e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:51.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.661+0000 7f8410e00640 1 -- 172.21.15.86:0/3664254976 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f840c06e8b0 con 0x7f840c06f0b0 2024-04-20T21:21:51.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.662+0000 7f8410e00640 1 --2- 172.21.15.86:0/3664254976 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f840c06f0b0 0x7f840c145a90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f83f8009790 tx=0x7f83f802b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86612f33934154c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:51.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.663+0000 7f840be00640 1 -- 172.21.15.86:0/3664254976 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f8035020 con 0x7f840c06f0b0 2024-04-20T21:21:51.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.663+0000 7f840be00640 1 -- 172.21.15.86:0/3664254976 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83f8031810 con 0x7f840c06f0b0 2024-04-20T21:21:51.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.663+0000 7f840be00640 1 -- 172.21.15.86:0/3664254976 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f8031be0 con 0x7f840c06f0b0 2024-04-20T21:21:51.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.664+0000 7f8414200640 1 -- 172.21.15.86:0/3664254976 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f840c06f0b0 msgr2=0x7f840c145a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:51.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.664+0000 7f8414200640 1 --2- 172.21.15.86:0/3664254976 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f840c06f0b0 0x7f840c145a90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f83f8009790 tx=0x7f83f802b8b0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.664+0000 7f8414200640 1 -- 172.21.15.86:0/3664254976 shutdown_connections 2024-04-20T21:21:51.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.664+0000 7f8414200640 1 --2- 172.21.15.86:0/3664254976 >> v2:172.21.15.144:3300/0 conn(0x7f840c145fd0 0x7f840c15cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.664+0000 7f8414200640 1 --2- 172.21.15.86:0/3664254976 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f840c06f0b0 0x7f840c145a90 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.664+0000 7f8414200640 1 -- 172.21.15.86:0/3664254976 >> 172.21.15.86:0/3664254976 conn(0x7f840c0b2180 msgr2=0x7f840c0b2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:51.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.664+0000 7f8414200640 1 -- 172.21.15.86:0/3664254976 shutdown_connections 2024-04-20T21:21:51.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.664+0000 7f8414200640 1 -- 172.21.15.86:0/3664254976 wait complete. 2024-04-20T21:21:51.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.666+0000 7f8414200640 1 Processor -- start 2024-04-20T21:21:51.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.666+0000 7f8414200640 1 -- start start 2024-04-20T21:21:51.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.667+0000 7f8414200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f840c06f0b0 0x7f840c15a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:51.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.667+0000 7f8414200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f840c071d00 0x7f840c13c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:51.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.668+0000 7f8414200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f840c145fd0 0x7f840c13cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:51.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.668+0000 7f8414200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f840c06dc30 con 0x7f840c145fd0 2024-04-20T21:21:51.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.668+0000 7f8414200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f840c06f510 con 0x7f840c06f0b0 2024-04-20T21:21:51.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.668+0000 7f8414200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f840c06ddb0 con 0x7f840c071d00 2024-04-20T21:21:51.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.668+0000 7f8412200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f840c145fd0 0x7f840c13cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:51.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.668+0000 7f8412200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f840c145fd0 0x7f840c13cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58268/0 (socket says 172.21.15.86:58268) 2024-04-20T21:21:51.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.668+0000 7f8412200640 1 -- 172.21.15.86:0/3030429962 learned_addr learned my addr 172.21.15.86:0/3030429962 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:51.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.668+0000 7f8411800640 1 --2- 172.21.15.86:0/3030429962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f840c06f0b0 0x7f840c15a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:51.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.668+0000 7f8410e00640 1 --2- 172.21.15.86:0/3030429962 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f840c071d00 0x7f840c13c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:51.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.668+0000 7f8412200640 1 -- 172.21.15.86:0/3030429962 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f840c071d00 msgr2=0x7f840c13c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:51.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.668+0000 7f8412200640 1 --2- 172.21.15.86:0/3030429962 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f840c071d00 0x7f840c13c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.668+0000 7f8412200640 1 -- 172.21.15.86:0/3030429962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f840c06f0b0 msgr2=0x7f840c15a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:51.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.668+0000 7f8412200640 1 --2- 172.21.15.86:0/3030429962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f840c06f0b0 0x7f840c15a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.668+0000 7f8412200640 1 -- 172.21.15.86:0/3030429962 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f840c15c630 con 0x7f840c145fd0 2024-04-20T21:21:51.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.668+0000 7f8410e00640 1 --2- 172.21.15.86:0/3030429962 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f840c071d00 0x7f840c13c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:21:51.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.669+0000 7f8412200640 1 --2- 172.21.15.86:0/3030429962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f840c145fd0 0x7f840c13cec0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f840400efc0 tx=0x7f840400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:51.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.669+0000 7f8409600640 1 -- 172.21.15.86:0/3030429962 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8404018020 con 0x7f840c145fd0 2024-04-20T21:21:51.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.669+0000 7f8409600640 1 -- 172.21.15.86:0/3030429962 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8404014480 con 0x7f840c145fd0 2024-04-20T21:21:51.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.669+0000 7f8409600640 1 -- 172.21.15.86:0/3030429962 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8404014870 con 0x7f840c145fd0 2024-04-20T21:21:51.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.669+0000 7f8414200640 1 -- 172.21.15.86:0/3030429962 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f840c13db20 con 0x7f840c145fd0 2024-04-20T21:21:51.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.669+0000 7f8414200640 1 -- 172.21.15.86:0/3030429962 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f840c13b4a0 con 0x7f840c145fd0 2024-04-20T21:21:51.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.671+0000 7f8409600640 1 -- 172.21.15.86:0/3030429962 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8404005d10 con 0x7f840c145fd0 2024-04-20T21:21:51.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.672+0000 7f8409600640 1 --2- 172.21.15.86:0/3030429962 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f83ec06c8f0 0x7f83ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:51.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.673+0000 7f8409600640 1 -- 172.21.15.86:0/3030429962 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f840408ca20 con 0x7f840c145fd0 2024-04-20T21:21:51.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.673+0000 7f8411800640 1 --2- 172.21.15.86:0/3030429962 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f83ec06c8f0 0x7f83ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:51.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.673+0000 7f8416fad740 1 Processor -- start 2024-04-20T21:21:51.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.673+0000 7f8416fad740 1 -- start start 2024-04-20T21:21:51.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.673+0000 7f8411800640 1 --2- 172.21.15.86:0/3030429962 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f83ec06c8f0 0x7f83ec06edb0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8400000f60 tx=0x7f8400009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:51.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.674+0000 7f8416fad740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f045d98380 0x55f045d6b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:51.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.674+0000 7f8416fad740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f045d98760 0x55f045d9b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:51.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.674+0000 7f8416fad740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f045d6b860 0x55f045d9b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:51.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.674+0000 7f8416fad740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f045cd6ea0 con 0x55f045d98760 2024-04-20T21:21:51.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.674+0000 7f8416fad740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f045c74220 con 0x55f045d98380 2024-04-20T21:21:51.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.674+0000 7f8416fad740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55f045cd2e60 con 0x55f045d6b860 2024-04-20T21:21:51.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.674+0000 7f8410e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f045d98760 0x55f045d9b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:51.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.674+0000 7f8410e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f045d98760 0x55f045d9b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58276/0 (socket says 172.21.15.86:58276) 2024-04-20T21:21:51.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.674+0000 7f8410e00640 1 -- 172.21.15.86:0/3796911984 learned_addr learned my addr 172.21.15.86:0/3796911984 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:51.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.675+0000 7f8411800640 1 --2- 172.21.15.86:0/3796911984 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f045d98380 0x55f045d6b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:51.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.675+0000 7f8410e00640 1 -- 172.21.15.86:0/3796911984 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f045d6b860 msgr2=0x55f045d9b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:51.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.675+0000 7f8412200640 1 --2- 172.21.15.86:0/3796911984 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f045d6b860 0x55f045d9b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:51.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.675+0000 7f8410e00640 1 --2- 172.21.15.86:0/3796911984 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f045d6b860 0x55f045d9b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.675+0000 7f8410e00640 1 -- 172.21.15.86:0/3796911984 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f045d98380 msgr2=0x55f045d6b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:51.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.675+0000 7f8410e00640 1 --2- 172.21.15.86:0/3796911984 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f045d98380 0x55f045d6b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.675+0000 7f8410e00640 1 -- 172.21.15.86:0/3796911984 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f045c8d840 con 0x55f045d98760 2024-04-20T21:21:51.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.675+0000 7f8411800640 1 --2- 172.21.15.86:0/3796911984 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f045d98380 0x55f045d6b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:21:51.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.675+0000 7f8410e00640 1 --2- 172.21.15.86:0/3796911984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f045d98760 0x55f045d9b110 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f83f8034040 tx=0x7f83f8030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=108287a71125fddc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:51.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.675+0000 7f83fe000640 1 -- 172.21.15.86:0/3796911984 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f8035040 con 0x55f045d98760 2024-04-20T21:21:51.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.675+0000 7f83fe000640 1 -- 172.21.15.86:0/3796911984 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83f8031a50 con 0x55f045d98760 2024-04-20T21:21:51.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.675+0000 7f83fe000640 1 -- 172.21.15.86:0/3796911984 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f8031e20 con 0x55f045d98760 2024-04-20T21:21:51.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.676+0000 7f8416fad740 1 -- 172.21.15.86:0/3796911984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f045d98760 msgr2=0x55f045d9b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:51.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.676+0000 7f8416fad740 1 --2- 172.21.15.86:0/3796911984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f045d98760 0x55f045d9b110 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f83f8034040 tx=0x7f83f8030000 comp rx=0 tx=0).stop 2024-04-20T21:21:51.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.676+0000 7f8416fad740 1 -- 172.21.15.86:0/3796911984 shutdown_connections 2024-04-20T21:21:51.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.676+0000 7f8416fad740 1 --2- 172.21.15.86:0/3796911984 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f045d6b860 0x55f045d9b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.676+0000 7f8416fad740 1 --2- 172.21.15.86:0/3796911984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f045d98760 0x55f045d9b110 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.676+0000 7f8416fad740 1 --2- 172.21.15.86:0/3796911984 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f045d98380 0x55f045d6b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.676+0000 7f8416fad740 1 -- 172.21.15.86:0/3796911984 >> 172.21.15.86:0/3796911984 conn(0x55f045d6d590 msgr2=0x55f045db6110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:51.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.676+0000 7f8416fad740 1 -- 172.21.15.86:0/3796911984 shutdown_connections 2024-04-20T21:21:51.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.676+0000 7f8416fad740 1 -- 172.21.15.86:0/3796911984 wait complete. 2024-04-20T21:21:51.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.677+0000 7f8416fad740 1 -- start start 2024-04-20T21:21:51.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.677+0000 7f8416fad740 1 Processor -- start 2024-04-20T21:21:51.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.678+0000 7f8416fad740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f045d98380 0x55f045d952b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:51.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.678+0000 7f8416fad740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f045d98760 0x55f045d994e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:51.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.678+0000 7f8416fad740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f045d8d880 0x55f045d99a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:51.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.678+0000 7f8416fad740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f045d6fc30 con 0x55f045d8d880 2024-04-20T21:21:51.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.678+0000 7f8416fad740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f045c5f180 con 0x55f045d98380 2024-04-20T21:21:51.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.678+0000 7f8416fad740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55f045d72470 con 0x55f045d98760 2024-04-20T21:21:51.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.678+0000 7f8416fad740 10 client.0 _ll_register_callbacks cb 0x55f0459e8d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:21:51.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.678+0000 7f8412200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f045d8d880 0x55f045d99a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:51.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.678+0000 7f8412200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f045d8d880 0x55f045d99a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58292/0 (socket says 172.21.15.86:58292) 2024-04-20T21:21:51.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.678+0000 7f8412200640 1 -- 172.21.15.86:0/1465492274 learned_addr learned my addr 172.21.15.86:0/1465492274 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:51.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.679+0000 7f8410e00640 1 --2- 172.21.15.86:0/1465492274 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f045d98760 0x55f045d994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:51.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.679+0000 7f8411800640 1 --2- 172.21.15.86:0/1465492274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f045d98380 0x55f045d952b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:51.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.679+0000 7f8412200640 1 -- 172.21.15.86:0/1465492274 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f045d98760 msgr2=0x55f045d994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:51.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.679+0000 7f8412200640 1 --2- 172.21.15.86:0/1465492274 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f045d98760 0x55f045d994e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.679+0000 7f8412200640 1 -- 172.21.15.86:0/1465492274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f045d98380 msgr2=0x55f045d952b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:51.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.679+0000 7f8412200640 1 --2- 172.21.15.86:0/1465492274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f045d98380 0x55f045d952b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.679+0000 7f8412200640 1 -- 172.21.15.86:0/1465492274 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f045ab5940 con 0x55f045d8d880 2024-04-20T21:21:51.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.679+0000 7f8412200640 1 --2- 172.21.15.86:0/1465492274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f045d8d880 0x55f045d99a20 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f8404062e00 tx=0x7f840405af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd9eb30d2582cc29 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:51.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.679+0000 7f83f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:21:51.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.679+0000 7f83f2a00640 1 -- 172.21.15.86:0/1465492274 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8404089060 con 0x55f045d8d880 2024-04-20T21:21:51.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.679+0000 7f83f2a00640 1 -- 172.21.15.86:0/1465492274 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f8404059ba0 con 0x55f045d8d880 2024-04-20T21:21:51.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.679+0000 7f8416fad740 1 -- 172.21.15.86:0/1465492274 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83f8031a50 con 0x55f045d8d880 2024-04-20T21:21:51.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.679+0000 7f83f2a00640 1 -- 172.21.15.86:0/1465492274 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8404077050 con 0x55f045d8d880 2024-04-20T21:21:51.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.679+0000 7f8416fad740 10 client.6423 fetch_fsmap learned FSMap version 33 2024-04-20T21:21:51.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.679+0000 7f8416fad740 1 -- 172.21.15.86:0/1465492274 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x55f045b160d0 con 0x55f045d8d880 2024-04-20T21:21:51.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.680+0000 7f83f2a00640 1 -- 172.21.15.86:0/1465492274 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83f8031a50 con 0x55f045d8d880 2024-04-20T21:21:51.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.680+0000 7f83f2a00640 1 -- 172.21.15.86:0/1465492274 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 33) ==== 3495+0+0 (secure 0 0 0) 0x7f84040564a0 con 0x55f045d8d880 2024-04-20T21:21:51.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.680+0000 7f8416fad740 10 client.6423 fetch_fsmap finished waiting for FSMap version 33 2024-04-20T21:21:51.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.680+0000 7f8416fad740 10 client.6423 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-20T21:21:51.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.680+0000 7f8416fad740 20 client.6423 populate_metadata read hostname 'smithi086' 2024-04-20T21:21:51.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.680+0000 7f8416fad740 1 --2- 172.21.15.86:0/1465492274 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x55f045dc82f0 0x55f045dc8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:51.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.680+0000 7f8416fad740 4 client.6423 mds_command: new command op to 6135 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:21:51.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.680+0000 7f8416fad740 1 -- 172.21.15.86:0/1465492274 --> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f045ce48c0 con 0x55f045dc82f0 2024-04-20T21:21:51.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.680+0000 7f8411800640 1 --2- 172.21.15.86:0/1465492274 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x55f045dc82f0 0x55f045dc8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:51.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.681+0000 7f8411800640 1 --2- 172.21.15.86:0/1465492274 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x55f045dc82f0 0x55f045dc8760 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0904b5300449501 server_cookie=414170c0599e2108 in_seq=0 out_seq=0 2024-04-20T21:21:51.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.681+0000 7f83f2a00640 10 client.6423 ms_handle_connect on v2:172.21.15.86:6835/3008383261 2024-04-20T21:21:51.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.681+0000 7f83f2a00640 1 -- 172.21.15.86:0/1465492274 <== mds.0 v2:172.21.15.86:6835/3008383261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f045ce48c0 con 0x55f045dc82f0 2024-04-20T21:21:51.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.681+0000 7f83f2a00640 10 client.6423 handle_command_reply: tid=0 2024-04-20T21:21:51.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.681+0000 7f8416fad740 1 client.6423 shutdown 2024-04-20T21:21:51.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.682+0000 7f8416fad740 1 -- 172.21.15.86:0/1465492274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f045d8d880 msgr2=0x55f045d99a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:51.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.682+0000 7f8416fad740 1 --2- 172.21.15.86:0/1465492274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f045d8d880 0x55f045d99a20 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f8404062e00 tx=0x7f840405af90 comp rx=0 tx=0).stop 2024-04-20T21:21:51.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.682+0000 7f8416fad740 1 -- 172.21.15.86:0/1465492274 shutdown_connections 2024-04-20T21:21:51.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.682+0000 7f8416fad740 1 --2- 172.21.15.86:0/1465492274 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x55f045dc82f0 0x55f045dc8760 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.682+0000 7f8416fad740 1 --2- 172.21.15.86:0/1465492274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f045d8d880 0x55f045d99a20 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.682+0000 7f8416fad740 1 --2- 172.21.15.86:0/1465492274 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f045d98760 0x55f045d994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.682+0000 7f8416fad740 1 --2- 172.21.15.86:0/1465492274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f045d98380 0x55f045d952b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.682+0000 7f83f2a00640 0 client.6423 ms_handle_reset on v2:172.21.15.86:6835/3008383261 2024-04-20T21:21:51.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.682+0000 7f8416fad740 1 -- 172.21.15.86:0/1465492274 >> 172.21.15.86:0/1465492274 conn(0x55f045d6b860 msgr2=0x55f045d9b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:51.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.682+0000 7f8416fad740 1 -- 172.21.15.86:0/1465492274 shutdown_connections 2024-04-20T21:21:51.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.682+0000 7f8416fad740 1 -- 172.21.15.86:0/1465492274 wait complete. 2024-04-20T21:21:51.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.682+0000 7f8416fad740 20 client.6423 trim_cache size 0 max 16384 2024-04-20T21:21:51.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.700+0000 7f8416fad740 1 Processor -- start 2024-04-20T21:21:51.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.701+0000 7f8416fad740 1 -- start start 2024-04-20T21:21:51.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.701+0000 7f8416fad740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f045d8d880 0x55f045db4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:51.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.701+0000 7f8416fad740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f045d98380 0x55f045db9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:51.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.701+0000 7f8416fad740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f045d98760 0x55f045d9af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:51.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.701+0000 7f8416fad740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f840c200f50 con 0x55f045d8d880 2024-04-20T21:21:51.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.701+0000 7f8416fad740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f840c1fffb0 con 0x55f045d98380 2024-04-20T21:21:51.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.701+0000 7f8416fad740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f840c1ff4f0 con 0x55f045d98760 2024-04-20T21:21:51.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.702+0000 7f8411800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f045d8d880 0x55f045db4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:51.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.702+0000 7f8411800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f045d8d880 0x55f045db4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58308/0 (socket says 172.21.15.86:58308) 2024-04-20T21:21:51.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.702+0000 7f8411800640 1 -- 172.21.15.86:0/626265092 learned_addr learned my addr 172.21.15.86:0/626265092 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:51.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.702+0000 7f8411800640 1 -- 172.21.15.86:0/626265092 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f045d98760 msgr2=0x55f045d9af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:51.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.702+0000 7f8411800640 1 --2- 172.21.15.86:0/626265092 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f045d98760 0x55f045d9af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.702+0000 7f8411800640 1 -- 172.21.15.86:0/626265092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f045d98380 msgr2=0x55f045db9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:51.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.702+0000 7f8411800640 1 --2- 172.21.15.86:0/626265092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f045d98380 0x55f045db9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.702+0000 7f8411800640 1 -- 172.21.15.86:0/626265092 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f045d95620 con 0x55f045d8d880 2024-04-20T21:21:51.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.702+0000 7f8411800640 1 --2- 172.21.15.86:0/626265092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f045d8d880 0x55f045db4050 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x55f045db5020 tx=0x7f8400012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f1dad1cae2c5024 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:51.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.703+0000 7f83fd600640 1 -- 172.21.15.86:0/626265092 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8400005d00 con 0x55f045d8d880 2024-04-20T21:21:51.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.703+0000 7f83fd600640 1 -- 172.21.15.86:0/626265092 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8400029070 con 0x55f045d8d880 2024-04-20T21:21:51.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.703+0000 7f83fd600640 1 -- 172.21.15.86:0/626265092 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840000e980 con 0x55f045d8d880 2024-04-20T21:21:51.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.703+0000 7f8416fad740 1 -- 172.21.15.86:0/626265092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f045d8d880 msgr2=0x55f045db4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:51.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.703+0000 7f8416fad740 1 --2- 172.21.15.86:0/626265092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f045d8d880 0x55f045db4050 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x55f045db5020 tx=0x7f8400012000 comp rx=0 tx=0).stop 2024-04-20T21:21:51.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.703+0000 7f8416fad740 1 -- 172.21.15.86:0/626265092 shutdown_connections 2024-04-20T21:21:51.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.703+0000 7f8416fad740 1 --2- 172.21.15.86:0/626265092 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f045d98760 0x55f045d9af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.703+0000 7f8416fad740 1 --2- 172.21.15.86:0/626265092 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f045d98380 0x55f045db9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.703+0000 7f8416fad740 1 --2- 172.21.15.86:0/626265092 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f045d8d880 0x55f045db4050 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.703+0000 7f8416fad740 1 -- 172.21.15.86:0/626265092 >> 172.21.15.86:0/626265092 conn(0x55f045d6b860 msgr2=0x55f045d6f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:51.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.703+0000 7f8416fad740 1 -- 172.21.15.86:0/626265092 shutdown_connections 2024-04-20T21:21:51.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.703+0000 7f8416fad740 1 -- 172.21.15.86:0/626265092 wait complete. 2024-04-20T21:21:51.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.704+0000 7f8416fad740 1 -- start start 2024-04-20T21:21:51.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.704+0000 7f8416fad740 1 Processor -- start 2024-04-20T21:21:51.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.705+0000 7f8416fad740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f045d8d880 0x55f045d9b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:51.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.705+0000 7f8416fad740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f045d98380 0x55f045e26b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:51.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.705+0000 7f8416fad740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f045d98760 0x55f045e27050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:51.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.705+0000 7f8416fad740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f045db6570 con 0x55f045d8d880 2024-04-20T21:21:51.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.705+0000 7f8416fad740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f840c1ff990 con 0x55f045d98380 2024-04-20T21:21:51.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.705+0000 7f8416fad740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55f045db6ae0 con 0x55f045d98760 2024-04-20T21:21:51.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.705+0000 7f8416fad740 10 client.0 _ll_register_callbacks cb 0x7f8404009ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:21:51.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.705+0000 7f8411800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f045d8d880 0x55f045d9b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:51.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.705+0000 7f8411800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f045d8d880 0x55f045d9b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58312/0 (socket says 172.21.15.86:58312) 2024-04-20T21:21:51.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.705+0000 7f8411800640 1 -- 172.21.15.86:0/3907806547 learned_addr learned my addr 172.21.15.86:0/3907806547 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:51.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.706+0000 7f8412200640 1 --2- 172.21.15.86:0/3907806547 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f045d98760 0x55f045e27050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:51.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.706+0000 7f8410e00640 1 --2- 172.21.15.86:0/3907806547 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f045d98380 0x55f045e26b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:51.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.706+0000 7f8411800640 1 -- 172.21.15.86:0/3907806547 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f045d98760 msgr2=0x55f045e27050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:51.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.706+0000 7f8411800640 1 --2- 172.21.15.86:0/3907806547 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f045d98760 0x55f045e27050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.706+0000 7f8411800640 1 -- 172.21.15.86:0/3907806547 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f045d98380 msgr2=0x55f045e26b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:51.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.706+0000 7f8411800640 1 --2- 172.21.15.86:0/3907806547 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f045d98380 0x55f045e26b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.706+0000 7f8411800640 1 -- 172.21.15.86:0/3907806547 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0459a9230 con 0x55f045d8d880 2024-04-20T21:21:51.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.706+0000 7f8411800640 1 --2- 172.21.15.86:0/3907806547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f045d8d880 0x55f045d9b230 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f8400026040 tx=0x7f8400000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9436f199f5860817 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:51.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.706+0000 7f83f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:21:51.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.706+0000 7f83f2a00640 1 -- 172.21.15.86:0/3907806547 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840000eb80 con 0x55f045d8d880 2024-04-20T21:21:51.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.706+0000 7f83f2a00640 1 -- 172.21.15.86:0/3907806547 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f840000ed20 con 0x55f045d8d880 2024-04-20T21:21:51.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.706+0000 7f8416fad740 1 -- 172.21.15.86:0/3907806547 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8400029070 con 0x55f045d8d880 2024-04-20T21:21:51.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.706+0000 7f83f2a00640 1 -- 172.21.15.86:0/3907806547 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8400024070 con 0x55f045d8d880 2024-04-20T21:21:51.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.707+0000 7f8416fad740 10 client.6429 fetch_fsmap learned FSMap version 33 2024-04-20T21:21:51.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.707+0000 7f8416fad740 1 -- 172.21.15.86:0/3907806547 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x55f045d8ced0 con 0x55f045d8d880 2024-04-20T21:21:51.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.707+0000 7f83f2a00640 1 -- 172.21.15.86:0/3907806547 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8400029070 con 0x55f045d8d880 2024-04-20T21:21:51.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.707+0000 7f83f2a00640 1 -- 172.21.15.86:0/3907806547 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 33) ==== 3495+0+0 (secure 0 0 0) 0x7f8400024210 con 0x55f045d8d880 2024-04-20T21:21:51.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.707+0000 7f8416fad740 10 client.6429 fetch_fsmap finished waiting for FSMap version 33 2024-04-20T21:21:51.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.707+0000 7f8416fad740 10 client.6429 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-20T21:21:51.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.707+0000 7f8416fad740 20 client.6429 populate_metadata read hostname 'smithi086' 2024-04-20T21:21:51.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.707+0000 7f8416fad740 1 --2- 172.21.15.86:0/3907806547 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x55f045dd9df0 0x55f045db73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:51.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.707+0000 7f8416fad740 4 client.6429 mds_command: new command op to 6135 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}] 2024-04-20T21:21:51.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.707+0000 7f8416fad740 1 -- 172.21.15.86:0/3907806547 --> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}) -- 0x55f045cde730 con 0x55f045dd9df0 2024-04-20T21:21:51.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.707+0000 7f8410e00640 1 --2- 172.21.15.86:0/3907806547 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x55f045dd9df0 0x55f045db73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:51.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.708+0000 7f8410e00640 1 --2- 172.21.15.86:0/3907806547 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x55f045dd9df0 0x55f045db73b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=696db4329ceda27c server_cookie=d437a4aad15c849a in_seq=0 out_seq=0 2024-04-20T21:21:51.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.708+0000 7f83f2a00640 10 client.6429 ms_handle_connect on v2:172.21.15.86:6835/3008383261 2024-04-20T21:21:51.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.709+0000 7f83f2a00640 1 -- 172.21.15.86:0/3907806547 <== mds.0 v2:172.21.15.86:6835/3008383261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f045cde730 con 0x55f045dd9df0 2024-04-20T21:21:51.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.709+0000 7f83f2a00640 10 client.6429 handle_command_reply: tid=0 2024-04-20T21:21:51.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.709+0000 7f8416fad740 1 client.6429 shutdown 2024-04-20T21:21:51.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.709+0000 7f8416fad740 1 -- 172.21.15.86:0/3907806547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f045d8d880 msgr2=0x55f045d9b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:51.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.709+0000 7f8416fad740 1 --2- 172.21.15.86:0/3907806547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f045d8d880 0x55f045d9b230 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f8400026040 tx=0x7f8400000bc0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.709+0000 7f8416fad740 1 -- 172.21.15.86:0/3907806547 shutdown_connections 2024-04-20T21:21:51.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.709+0000 7f8416fad740 1 --2- 172.21.15.86:0/3907806547 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x55f045dd9df0 0x55f045db73b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.709+0000 7f8416fad740 1 --2- 172.21.15.86:0/3907806547 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f045d98760 0x55f045e27050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.709+0000 7f8416fad740 1 --2- 172.21.15.86:0/3907806547 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f045d98380 0x55f045e26b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.709+0000 7f8416fad740 1 --2- 172.21.15.86:0/3907806547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f045d8d880 0x55f045d9b230 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.709+0000 7f83f2a00640 0 client.6429 ms_handle_reset on v2:172.21.15.86:6835/3008383261 2024-04-20T21:21:51.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.709+0000 7f8416fad740 1 -- 172.21.15.86:0/3907806547 >> 172.21.15.86:0/3907806547 conn(0x55f045d6b860 msgr2=0x55f045d9a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:51.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.709+0000 7f8416fad740 1 -- 172.21.15.86:0/3907806547 shutdown_connections 2024-04-20T21:21:51.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.709+0000 7f8416fad740 1 -- 172.21.15.86:0/3907806547 wait complete. 2024-04-20T21:21:51.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.709+0000 7f8416fad740 20 client.6429 trim_cache size 0 max 16384 2024-04-20T21:21:51.713 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:21:51.714 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-20T21:21:51.714 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "52d9d507-79f2-48e8-97e8-45786fa716db", 2024-04-20T21:21:51.714 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-20T21:21:51.714 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:21:51.714 INFO:tasks.ceph.mds.c.smithi086.stderr:2024-04-20T21:21:51.710+0000 7fb60f600640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v4 ap=1 DIRTYPARENT s=0 n(v0 rc2024-04-20T21:21:18.168279+0000 1=1+0) caps={6306=pAsLsXsFs/-@2},l=6306(-1) | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=1 authpin=1 scrubqueue=0 0x559d2d306b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v4>,<0x1/dir_test v7>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-20T21:21:51.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.710+0000 7f8414200640 1 -- 172.21.15.86:0/3030429962 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f83ec06c8f0 msgr2=0x7f83ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:51.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.710+0000 7f8414200640 1 --2- 172.21.15.86:0/3030429962 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f83ec06c8f0 0x7f83ec06edb0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8400000f60 tx=0x7f8400009040 comp rx=0 tx=0).stop 2024-04-20T21:21:51.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.710+0000 7f8414200640 1 -- 172.21.15.86:0/3030429962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f840c145fd0 msgr2=0x7f840c13cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:51.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.710+0000 7f8414200640 1 --2- 172.21.15.86:0/3030429962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f840c145fd0 0x7f840c13cec0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f840400efc0 tx=0x7f840400c490 comp rx=0 tx=0).stop 2024-04-20T21:21:51.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.710+0000 7f8414200640 1 -- 172.21.15.86:0/3030429962 shutdown_connections 2024-04-20T21:21:51.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.710+0000 7f8414200640 1 --2- 172.21.15.86:0/3030429962 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f83ec06c8f0 0x7f83ec06edb0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.710+0000 7f8414200640 1 --2- 172.21.15.86:0/3030429962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f840c145fd0 0x7f840c13cec0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.710+0000 7f8414200640 1 --2- 172.21.15.86:0/3030429962 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f840c071d00 0x7f840c13c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.710+0000 7f8414200640 1 --2- 172.21.15.86:0/3030429962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f840c06f0b0 0x7f840c15a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.710+0000 7f8414200640 1 -- 172.21.15.86:0/3030429962 >> 172.21.15.86:0/3030429962 conn(0x7f840c0b2180 msgr2=0x7f840c15ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:51.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.710+0000 7f8414200640 1 -- 172.21.15.86:0/3030429962 shutdown_connections 2024-04-20T21:21:51.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.710+0000 7f8414200640 1 -- 172.21.15.86:0/3030429962 wait complete. 2024-04-20T21:21:51.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:21:51.726 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-20T21:21:51.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.913+0000 7f6367e00640 1 Processor -- start 2024-04-20T21:21:51.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.914+0000 7f6367e00640 1 -- start start 2024-04-20T21:21:51.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.915+0000 7f6367e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6368145250 0x7f636806efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:51.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.915+0000 7f6367e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6368072450 0x7f636815cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:51.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.915+0000 7f6367e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f636806f500 con 0x7f6368145f00 2024-04-20T21:21:51.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.915+0000 7f6367e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f6368160410 con 0x7f6368072450 2024-04-20T21:21:51.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.915+0000 7f6367e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f63681606f0 con 0x7f6368145250 2024-04-20T21:21:51.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.915+0000 7f6367400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6368072450 0x7f636815cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:51.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.915+0000 7f6367400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6368072450 0x7f636815cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58320/0 (socket says 172.21.15.86:58320) 2024-04-20T21:21:51.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.915+0000 7f6367400640 1 -- 172.21.15.86:0/2091401441 learned_addr learned my addr 172.21.15.86:0/2091401441 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:51.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.916+0000 7f6366a00640 1 --2- 172.21.15.86:0/2091401441 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6368145250 0x7f636806efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:51.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.916+0000 7f6367400640 1 -- 172.21.15.86:0/2091401441 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6368145250 msgr2=0x7f636806efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:51.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.916+0000 7f6367400640 1 --2- 172.21.15.86:0/2091401441 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6368145250 0x7f636806efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.916+0000 7f6367400640 1 -- 172.21.15.86:0/2091401441 >> v1:172.21.15.144:6789/0 conn(0x7f6368145f00 legacy=0x7f6368071d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:51.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.916+0000 7f6367400640 1 -- 172.21.15.86:0/2091401441 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63681609d0 con 0x7f6368072450 2024-04-20T21:21:51.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.917+0000 7f6367400640 1 --2- 172.21.15.86:0/2091401441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6368072450 0x7f636815cdf0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f6360009e50 tx=0x7f636002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cc0826a5262916e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:51.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.917+0000 7f6366000640 1 -- 172.21.15.86:0/2091401441 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636002c9e0 con 0x7f6368072450 2024-04-20T21:21:51.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.917+0000 7f6366000640 1 -- 172.21.15.86:0/2091401441 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f636002cb80 con 0x7f6368072450 2024-04-20T21:21:51.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.917+0000 7f6366000640 1 -- 172.21.15.86:0/2091401441 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63600343f0 con 0x7f6368072450 2024-04-20T21:21:51.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.918+0000 7f6367e00640 1 -- 172.21.15.86:0/2091401441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6368072450 msgr2=0x7f636815cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:51.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.918+0000 7f6367e00640 1 --2- 172.21.15.86:0/2091401441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6368072450 0x7f636815cdf0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f6360009e50 tx=0x7f636002be40 comp rx=0 tx=0).stop 2024-04-20T21:21:51.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.918+0000 7f6367e00640 1 -- 172.21.15.86:0/2091401441 shutdown_connections 2024-04-20T21:21:51.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.918+0000 7f6367e00640 1 --2- 172.21.15.86:0/2091401441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6368072450 0x7f636815cdf0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.918+0000 7f6367e00640 1 --2- 172.21.15.86:0/2091401441 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6368145250 0x7f636806efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.918+0000 7f6367e00640 1 -- 172.21.15.86:0/2091401441 >> 172.21.15.86:0/2091401441 conn(0x7f6368067cf0 msgr2=0x7f636806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:51.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.918+0000 7f6367e00640 1 -- 172.21.15.86:0/2091401441 shutdown_connections 2024-04-20T21:21:51.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.918+0000 7f6367e00640 1 -- 172.21.15.86:0/2091401441 wait complete. 2024-04-20T21:21:51.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.920+0000 7f6367e00640 1 Processor -- start 2024-04-20T21:21:51.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.920+0000 7f6367e00640 1 -- start start 2024-04-20T21:21:51.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.921+0000 7f6367e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6368072450 0x7f63681f42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:51.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.921+0000 7f6367e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6368145250 0x7f63680b2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:51.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.921+0000 7f6367e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6368145f00 0x7f63680ab720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:51.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.921+0000 7f6367e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f63681457b0 con 0x7f6368145250 2024-04-20T21:21:51.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.921+0000 7f6367e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6368145630 con 0x7f6368145f00 2024-04-20T21:21:51.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.921+0000 7f6367e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f636815d330 con 0x7f6368072450 2024-04-20T21:21:51.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.921+0000 7f635e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6368145250 0x7f63680b2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:51.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.921+0000 7f635e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6368145250 0x7f63680b2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58322/0 (socket says 172.21.15.86:58322) 2024-04-20T21:21:51.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.921+0000 7f635e000640 1 -- 172.21.15.86:0/284176180 learned_addr learned my addr 172.21.15.86:0/284176180 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:51.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.922+0000 7f6366a00640 1 --2- 172.21.15.86:0/284176180 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6368072450 0x7f63681f42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:51.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.922+0000 7f635e000640 1 -- 172.21.15.86:0/284176180 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6368072450 msgr2=0x7f63681f42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:51.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.922+0000 7f6367400640 1 --2- 172.21.15.86:0/284176180 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6368145f00 0x7f63680ab720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:51.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.922+0000 7f635e000640 1 --2- 172.21.15.86:0/284176180 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6368072450 0x7f63681f42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.922+0000 7f635e000640 1 -- 172.21.15.86:0/284176180 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6368145f00 msgr2=0x7f63680ab720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:51.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.922+0000 7f635e000640 1 --2- 172.21.15.86:0/284176180 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6368145f00 0x7f63680ab720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.922+0000 7f635e000640 1 -- 172.21.15.86:0/284176180 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63680abc60 con 0x7f6368145250 2024-04-20T21:21:51.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.922+0000 7f635e000640 1 --2- 172.21.15.86:0/284176180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6368145250 0x7f63680b2b10 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f635000b420 tx=0x7f635000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:51.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.922+0000 7f635f400640 1 -- 172.21.15.86:0/284176180 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6350011830 con 0x7f6368145250 2024-04-20T21:21:51.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.922+0000 7f635f400640 1 -- 172.21.15.86:0/284176180 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6350016070 con 0x7f6368145250 2024-04-20T21:21:51.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.922+0000 7f635f400640 1 -- 172.21.15.86:0/284176180 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6350011c10 con 0x7f6368145250 2024-04-20T21:21:51.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.922+0000 7f6367e00640 1 -- 172.21.15.86:0/284176180 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63681f49b0 con 0x7f6368145250 2024-04-20T21:21:51.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.922+0000 7f6367e00640 1 -- 172.21.15.86:0/284176180 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63681f4f70 con 0x7f6368145250 2024-04-20T21:21:51.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.924+0000 7f635f400640 1 -- 172.21.15.86:0/284176180 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f635001b460 con 0x7f6368145250 2024-04-20T21:21:51.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.925+0000 7f635f400640 1 --2- 172.21.15.86:0/284176180 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f634406c8f0 0x7f634406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:51.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.926+0000 7f635f400640 1 -- 172.21.15.86:0/284176180 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f635008b5d0 con 0x7f6368145250 2024-04-20T21:21:51.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.926+0000 7f6366a00640 1 --2- 172.21.15.86:0/284176180 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f634406c8f0 0x7f634406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:51.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.926+0000 7f6370835740 1 Processor -- start 2024-04-20T21:21:51.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.926+0000 7f6366a00640 1 --2- 172.21.15.86:0/284176180 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f634406c8f0 0x7f634406edb0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6358006fd0 tx=0x7f6358008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:51.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.926+0000 7f6370835740 1 -- start start 2024-04-20T21:21:51.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.927+0000 7f6370835740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afea98e3d0 0x55afea98e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:51.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.927+0000 7f6370835740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afea95e3f0 0x55afea98b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:51.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.927+0000 7f6370835740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55afea961f50 0x55afea98bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:51.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.927+0000 7f6370835740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55afea8ca150 con 0x55afea95e3f0 2024-04-20T21:21:51.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.927+0000 7f6370835740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55afea870370 con 0x55afea98e3d0 2024-04-20T21:21:51.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.927+0000 7f6370835740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55afea8c6110 con 0x55afea961f50 2024-04-20T21:21:51.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.927+0000 7f635e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afea95e3f0 0x55afea98b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:51.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.928+0000 7f635e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afea95e3f0 0x55afea98b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58328/0 (socket says 172.21.15.86:58328) 2024-04-20T21:21:51.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.928+0000 7f635e000640 1 -- 172.21.15.86:0/2601402608 learned_addr learned my addr 172.21.15.86:0/2601402608 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:51.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.928+0000 7f6366a00640 1 --2- 172.21.15.86:0/2601402608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afea98e3d0 0x55afea98e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:51.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.928+0000 7f635e000640 1 -- 172.21.15.86:0/2601402608 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55afea961f50 msgr2=0x55afea98bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:51.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.928+0000 7f6367400640 1 --2- 172.21.15.86:0/2601402608 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55afea961f50 0x55afea98bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:51.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.928+0000 7f635e000640 1 --2- 172.21.15.86:0/2601402608 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55afea961f50 0x55afea98bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.928+0000 7f635e000640 1 -- 172.21.15.86:0/2601402608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afea98e3d0 msgr2=0x55afea98e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:51.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.928+0000 7f635e000640 1 --2- 172.21.15.86:0/2601402608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afea98e3d0 0x55afea98e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.928+0000 7f635e000640 1 -- 172.21.15.86:0/2601402608 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afea6a67d0 con 0x55afea95e3f0 2024-04-20T21:21:51.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.928+0000 7f635e000640 1 --2- 172.21.15.86:0/2601402608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afea95e3f0 0x55afea98b640 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f63500619a0 tx=0x7f63500593b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=471250d46aa96a6d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:51.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.928+0000 7f6357400640 1 -- 172.21.15.86:0/2601402608 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63500575f0 con 0x55afea95e3f0 2024-04-20T21:21:51.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.928+0000 7f6357400640 1 -- 172.21.15.86:0/2601402608 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6350055960 con 0x55afea95e3f0 2024-04-20T21:21:51.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.928+0000 7f6357400640 1 -- 172.21.15.86:0/2601402608 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635005d390 con 0x55afea95e3f0 2024-04-20T21:21:51.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.929+0000 7f6370835740 1 -- 172.21.15.86:0/2601402608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afea95e3f0 msgr2=0x55afea98b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:51.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.929+0000 7f6370835740 1 --2- 172.21.15.86:0/2601402608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afea95e3f0 0x55afea98b640 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f63500619a0 tx=0x7f63500593b0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.929+0000 7f6370835740 1 -- 172.21.15.86:0/2601402608 shutdown_connections 2024-04-20T21:21:51.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.929+0000 7f6370835740 1 --2- 172.21.15.86:0/2601402608 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55afea961f50 0x55afea98bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.929+0000 7f6370835740 1 --2- 172.21.15.86:0/2601402608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afea95e3f0 0x55afea98b640 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.929+0000 7f6370835740 1 --2- 172.21.15.86:0/2601402608 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afea98e3d0 0x55afea98e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.929+0000 7f6370835740 1 -- 172.21.15.86:0/2601402608 >> 172.21.15.86:0/2601402608 conn(0x55afea960660 msgr2=0x55afea9a91e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:51.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.929+0000 7f6370835740 1 -- 172.21.15.86:0/2601402608 shutdown_connections 2024-04-20T21:21:51.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.929+0000 7f6370835740 1 -- 172.21.15.86:0/2601402608 wait complete. 2024-04-20T21:21:51.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.930+0000 7f6370835740 1 -- start start 2024-04-20T21:21:51.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.930+0000 7f6370835740 1 Processor -- start 2024-04-20T21:21:51.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.931+0000 7f6370835740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afea961f50 0x55afea979790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:51.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.931+0000 7f6370835740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55afea98e3d0 0x55afea9868e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:51.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.931+0000 7f6370835740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afea9b9da0 0x55afea989e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:51.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.931+0000 7f6370835740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55afea963100 con 0x55afea961f50 2024-04-20T21:21:51.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.931+0000 7f6370835740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55afea883740 con 0x55afea9b9da0 2024-04-20T21:21:51.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.931+0000 7f6370835740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55afea9624f0 con 0x55afea98e3d0 2024-04-20T21:21:51.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.931+0000 7f6370835740 10 client.0 _ll_register_callbacks cb 0x55afea87eb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:21:51.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.932+0000 7f6366a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afea961f50 0x55afea979790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:51.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.932+0000 7f6366a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afea961f50 0x55afea979790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58340/0 (socket says 172.21.15.86:58340) 2024-04-20T21:21:51.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.932+0000 7f6366a00640 1 -- 172.21.15.86:0/3242751327 learned_addr learned my addr 172.21.15.86:0/3242751327 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:51.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.932+0000 7f6367400640 1 --2- 172.21.15.86:0/3242751327 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afea9b9da0 0x55afea989e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:51.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.932+0000 7f6366a00640 1 -- 172.21.15.86:0/3242751327 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55afea98e3d0 msgr2=0x55afea9868e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:51.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.932+0000 7f635e000640 1 --2- 172.21.15.86:0/3242751327 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55afea98e3d0 0x55afea9868e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:51.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.932+0000 7f6366a00640 1 --2- 172.21.15.86:0/3242751327 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55afea98e3d0 0x55afea9868e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.932+0000 7f6366a00640 1 -- 172.21.15.86:0/3242751327 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afea9b9da0 msgr2=0x55afea989e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:51.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.932+0000 7f6366a00640 1 --2- 172.21.15.86:0/3242751327 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afea9b9da0 0x55afea989e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.932+0000 7f6366a00640 1 -- 172.21.15.86:0/3242751327 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afea5dc3f0 con 0x55afea961f50 2024-04-20T21:21:51.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.932+0000 7f6366a00640 1 --2- 172.21.15.86:0/3242751327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afea961f50 0x55afea979790 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f635800efe0 tx=0x7f6358004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5225d7391c47896 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:51.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.932+0000 7f634be00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:21:51.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.933+0000 7f634be00640 1 -- 172.21.15.86:0/3242751327 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635800f040 con 0x55afea961f50 2024-04-20T21:21:51.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.933+0000 7f634be00640 1 -- 172.21.15.86:0/3242751327 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f6358015da0 con 0x55afea961f50 2024-04-20T21:21:51.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.933+0000 7f6370835740 1 -- 172.21.15.86:0/3242751327 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6350055960 con 0x55afea961f50 2024-04-20T21:21:51.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.933+0000 7f634be00640 1 -- 172.21.15.86:0/3242751327 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f635802d050 con 0x55afea961f50 2024-04-20T21:21:51.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.933+0000 7f6370835740 10 client.6441 fetch_fsmap learned FSMap version 33 2024-04-20T21:21:51.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.933+0000 7f6370835740 1 -- 172.21.15.86:0/3242751327 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x55afea59c0f0 con 0x55afea961f50 2024-04-20T21:21:51.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.933+0000 7f634be00640 1 -- 172.21.15.86:0/3242751327 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6350055960 con 0x55afea961f50 2024-04-20T21:21:51.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.933+0000 7f634be00640 1 -- 172.21.15.86:0/3242751327 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 33) ==== 3495+0+0 (secure 0 0 0) 0x7f6358030020 con 0x55afea961f50 2024-04-20T21:21:51.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.933+0000 7f6370835740 10 client.6441 fetch_fsmap finished waiting for FSMap version 33 2024-04-20T21:21:51.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.933+0000 7f6370835740 10 client.6441 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-20T21:21:51.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.933+0000 7f6370835740 20 client.6441 populate_metadata read hostname 'smithi086' 2024-04-20T21:21:51.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.933+0000 7f6370835740 1 --2- 172.21.15.86:0/3242751327 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x55afea984690 0x55afea984aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:51.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.934+0000 7f6370835740 4 client.6441 mds_command: new command op to 6135 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:21:51.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.934+0000 7f6370835740 1 -- 172.21.15.86:0/3242751327 --> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55afea8d7b70 con 0x55afea984690 2024-04-20T21:21:51.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.934+0000 7f635e000640 1 --2- 172.21.15.86:0/3242751327 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x55afea984690 0x55afea984aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:51.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.934+0000 7f635e000640 1 --2- 172.21.15.86:0/3242751327 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x55afea984690 0x55afea984aa0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bdfdc98dc99d689 server_cookie=dffb1ace3d50e16e in_seq=0 out_seq=0 2024-04-20T21:21:51.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.934+0000 7f634be00640 10 client.6441 ms_handle_connect on v2:172.21.15.86:6835/3008383261 2024-04-20T21:21:51.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.935+0000 7f634be00640 1 -- 172.21.15.86:0/3242751327 <== mds.0 v2:172.21.15.86:6835/3008383261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55afea8d7b70 con 0x55afea984690 2024-04-20T21:21:51.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.935+0000 7f634be00640 10 client.6441 handle_command_reply: tid=0 2024-04-20T21:21:51.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.935+0000 7f6370835740 1 client.6441 shutdown 2024-04-20T21:21:51.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.935+0000 7f6370835740 1 -- 172.21.15.86:0/3242751327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afea961f50 msgr2=0x55afea979790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:51.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.935+0000 7f6370835740 1 --2- 172.21.15.86:0/3242751327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afea961f50 0x55afea979790 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f635800efe0 tx=0x7f6358004730 comp rx=0 tx=0).stop 2024-04-20T21:21:51.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.935+0000 7f6370835740 1 -- 172.21.15.86:0/3242751327 shutdown_connections 2024-04-20T21:21:51.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.935+0000 7f6370835740 1 --2- 172.21.15.86:0/3242751327 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x55afea984690 0x55afea984aa0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.935+0000 7f6370835740 1 --2- 172.21.15.86:0/3242751327 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afea9b9da0 0x55afea989e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.935+0000 7f6370835740 1 --2- 172.21.15.86:0/3242751327 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55afea98e3d0 0x55afea9868e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.935+0000 7f6370835740 1 --2- 172.21.15.86:0/3242751327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afea961f50 0x55afea979790 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.935+0000 7f634be00640 0 client.6441 ms_handle_reset on v2:172.21.15.86:6835/3008383261 2024-04-20T21:21:51.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.935+0000 7f6370835740 1 -- 172.21.15.86:0/3242751327 >> 172.21.15.86:0/3242751327 conn(0x55afea95e3f0 msgr2=0x55afea9aa780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:51.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.936+0000 7f6370835740 1 -- 172.21.15.86:0/3242751327 shutdown_connections 2024-04-20T21:21:51.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.936+0000 7f6370835740 1 -- 172.21.15.86:0/3242751327 wait complete. 2024-04-20T21:21:51.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.936+0000 7f6370835740 20 client.6441 trim_cache size 0 max 16384 2024-04-20T21:21:51.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.955+0000 7f6370835740 1 Processor -- start 2024-04-20T21:21:51.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.955+0000 7f6370835740 1 -- start start 2024-04-20T21:21:51.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.956+0000 7f6370835740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55afea961f50 0x55afea9850d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:51.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.956+0000 7f6370835740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afea98e3d0 0x55afea985610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:51.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.956+0000 7f6370835740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afea9b9da0 0x55afea9aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:51.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.956+0000 7f6370835740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6368200cf0 con 0x55afea9b9da0 2024-04-20T21:21:51.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.956+0000 7f6370835740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f63681ffd20 con 0x55afea98e3d0 2024-04-20T21:21:51.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.956+0000 7f6370835740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f63681f7d40 con 0x55afea961f50 2024-04-20T21:21:51.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.956+0000 7f6367400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afea9b9da0 0x55afea9aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:51.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.956+0000 7f6367400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afea9b9da0 0x55afea9aa470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58348/0 (socket says 172.21.15.86:58348) 2024-04-20T21:21:51.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.956+0000 7f6367400640 1 -- 172.21.15.86:0/1508767515 learned_addr learned my addr 172.21.15.86:0/1508767515 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:51.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.956+0000 7f635e000640 1 --2- 172.21.15.86:0/1508767515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afea98e3d0 0x55afea985610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:51.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.956+0000 7f6366a00640 1 --2- 172.21.15.86:0/1508767515 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55afea961f50 0x55afea9850d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:51.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.956+0000 7f6367400640 1 -- 172.21.15.86:0/1508767515 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55afea961f50 msgr2=0x55afea9850d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:51.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.956+0000 7f6367400640 1 --2- 172.21.15.86:0/1508767515 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55afea961f50 0x55afea9850d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.956+0000 7f6367400640 1 -- 172.21.15.86:0/1508767515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afea98e3d0 msgr2=0x55afea985610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:51.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.956+0000 7f6367400640 1 --2- 172.21.15.86:0/1508767515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afea98e3d0 0x55afea985610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.957+0000 7f6367400640 1 -- 172.21.15.86:0/1508767515 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afea964b50 con 0x55afea9b9da0 2024-04-20T21:21:51.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.957+0000 7f635e000640 1 --2- 172.21.15.86:0/1508767515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afea98e3d0 0x55afea985610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:21:51.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.957+0000 7f6367400640 1 --2- 172.21.15.86:0/1508767515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afea9b9da0 0x55afea9aa470 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f636002c450 tx=0x7f636002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee42a842fb62b2e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:51.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.957+0000 7f6356a00640 1 -- 172.21.15.86:0/1508767515 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6360009510 con 0x55afea9b9da0 2024-04-20T21:21:51.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.957+0000 7f6356a00640 1 -- 172.21.15.86:0/1508767515 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f636002c9e0 con 0x55afea9b9da0 2024-04-20T21:21:51.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.957+0000 7f6356a00640 1 -- 172.21.15.86:0/1508767515 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636002cb90 con 0x55afea9b9da0 2024-04-20T21:21:51.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.958+0000 7f6370835740 1 -- 172.21.15.86:0/1508767515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afea9b9da0 msgr2=0x55afea9aa470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:51.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.958+0000 7f6370835740 1 --2- 172.21.15.86:0/1508767515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afea9b9da0 0x55afea9aa470 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f636002c450 tx=0x7f636002e000 comp rx=0 tx=0).stop 2024-04-20T21:21:51.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.958+0000 7f6370835740 1 -- 172.21.15.86:0/1508767515 shutdown_connections 2024-04-20T21:21:51.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.958+0000 7f6370835740 1 --2- 172.21.15.86:0/1508767515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afea9b9da0 0x55afea9aa470 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.958+0000 7f6370835740 1 --2- 172.21.15.86:0/1508767515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afea98e3d0 0x55afea985610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.958+0000 7f6370835740 1 --2- 172.21.15.86:0/1508767515 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55afea961f50 0x55afea9850d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.958+0000 7f6370835740 1 -- 172.21.15.86:0/1508767515 >> 172.21.15.86:0/1508767515 conn(0x55afea95e3f0 msgr2=0x55afea964440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:51.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.958+0000 7f6370835740 1 -- 172.21.15.86:0/1508767515 shutdown_connections 2024-04-20T21:21:51.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.958+0000 7f6370835740 1 -- 172.21.15.86:0/1508767515 wait complete. 2024-04-20T21:21:51.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.959+0000 7f6370835740 1 -- start start 2024-04-20T21:21:51.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.959+0000 7f6370835740 1 Processor -- start 2024-04-20T21:21:51.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.960+0000 7f6370835740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55afea961f50 0x55afeaa1f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:51.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.960+0000 7f6370835740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afea98e3d0 0x55afea98fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:51.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.960+0000 7f6370835740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afea9b9da0 0x55afea98ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:51.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.960+0000 7f6370835740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f63681f8390 con 0x55afea9b9da0 2024-04-20T21:21:51.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.960+0000 7f6370835740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55afea9aa240 con 0x55afea98e3d0 2024-04-20T21:21:51.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.960+0000 7f6370835740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55afea9a9b50 con 0x55afea961f50 2024-04-20T21:21:51.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.960+0000 7f6370835740 10 client.0 _ll_register_callbacks cb 0x7f6358009540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:21:51.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.960+0000 7f6367400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afea9b9da0 0x55afea98ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:51.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.960+0000 7f6367400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afea9b9da0 0x55afea98ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58360/0 (socket says 172.21.15.86:58360) 2024-04-20T21:21:51.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.960+0000 7f6367400640 1 -- 172.21.15.86:0/1525787383 learned_addr learned my addr 172.21.15.86:0/1525787383 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:21:51.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.961+0000 7f6367400640 1 -- 172.21.15.86:0/1525787383 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55afea961f50 msgr2=0x55afeaa1f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:51.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.961+0000 7f6367400640 1 --2- 172.21.15.86:0/1525787383 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55afea961f50 0x55afeaa1f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.961+0000 7f6367400640 1 -- 172.21.15.86:0/1525787383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afea98e3d0 msgr2=0x55afea98fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:21:51.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.961+0000 7f635e000640 1 --2- 172.21.15.86:0/1525787383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afea98e3d0 0x55afea98fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:51.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.961+0000 7f6367400640 1 --2- 172.21.15.86:0/1525787383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afea98e3d0 0x55afea98fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.961+0000 7f6367400640 1 -- 172.21.15.86:0/1525787383 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afea981680 con 0x55afea9b9da0 2024-04-20T21:21:51.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.961+0000 7f6367400640 1 --2- 172.21.15.86:0/1525787383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afea9b9da0 0x55afea98ff90 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f6360044000 tx=0x7f6360004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a7235b15680eb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:21:51.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.961+0000 7f634be00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:21:51.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.961+0000 7f634be00640 1 -- 172.21.15.86:0/1525787383 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6360009510 con 0x55afea9b9da0 2024-04-20T21:21:51.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.961+0000 7f634be00640 1 -- 172.21.15.86:0/1525787383 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f6360004250 con 0x55afea9b9da0 2024-04-20T21:21:51.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.961+0000 7f6370835740 1 -- 172.21.15.86:0/1525787383 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f636002c9e0 con 0x55afea9b9da0 2024-04-20T21:21:51.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.962+0000 7f634be00640 1 -- 172.21.15.86:0/1525787383 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6360004450 con 0x55afea9b9da0 2024-04-20T21:21:51.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.962+0000 7f6370835740 10 client.6447 fetch_fsmap learned FSMap version 33 2024-04-20T21:21:51.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.962+0000 7f6370835740 1 -- 172.21.15.86:0/1525787383 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x55afea981c60 con 0x55afea9b9da0 2024-04-20T21:21:51.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.962+0000 7f634be00640 1 -- 172.21.15.86:0/1525787383 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f636002c9e0 con 0x55afea9b9da0 2024-04-20T21:21:51.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.962+0000 7f634be00640 1 -- 172.21.15.86:0/1525787383 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 33) ==== 3495+0+0 (secure 0 0 0) 0x7f6360008040 con 0x55afea9b9da0 2024-04-20T21:21:51.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.962+0000 7f6370835740 10 client.6447 fetch_fsmap finished waiting for FSMap version 33 2024-04-20T21:21:51.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.962+0000 7f6370835740 10 client.6447 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-20T21:21:51.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.962+0000 7f6370835740 20 client.6447 populate_metadata read hostname 'smithi086' 2024-04-20T21:21:51.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.962+0000 7f6370835740 1 --2- 172.21.15.86:0/1525787383 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x55afea982540 0x55afea9a8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:21:51.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.962+0000 7f6370835740 4 client.6447 mds_command: new command op to 6135 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:21:51.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.962+0000 7f6370835740 1 -- 172.21.15.86:0/1525787383 --> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55afea95ebf0 con 0x55afea982540 2024-04-20T21:21:51.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.962+0000 7f6366a00640 1 --2- 172.21.15.86:0/1525787383 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x55afea982540 0x55afea9a8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:21:51.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.963+0000 7f6366a00640 1 --2- 172.21.15.86:0/1525787383 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x55afea982540 0x55afea9a8bc0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87065891c72a649a server_cookie=384e12db3a388301 in_seq=0 out_seq=0 2024-04-20T21:21:51.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.963+0000 7f634be00640 10 client.6447 ms_handle_connect on v2:172.21.15.86:6835/3008383261 2024-04-20T21:21:51.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.963+0000 7f634be00640 1 -- 172.21.15.86:0/1525787383 <== mds.0 v2:172.21.15.86:6835/3008383261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x55afea95ebf0 con 0x55afea982540 2024-04-20T21:21:51.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.963+0000 7f634be00640 10 client.6447 handle_command_reply: tid=0 2024-04-20T21:21:51.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.963+0000 7f6370835740 1 client.6447 shutdown 2024-04-20T21:21:51.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.963+0000 7f6370835740 1 -- 172.21.15.86:0/1525787383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afea9b9da0 msgr2=0x55afea98ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:51.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.963+0000 7f6370835740 1 --2- 172.21.15.86:0/1525787383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afea9b9da0 0x55afea98ff90 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f6360044000 tx=0x7f6360004ab0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.963+0000 7f6370835740 1 -- 172.21.15.86:0/1525787383 shutdown_connections 2024-04-20T21:21:51.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.963+0000 7f6370835740 1 --2- 172.21.15.86:0/1525787383 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x55afea982540 0x55afea9a8bc0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.963+0000 7f6370835740 1 --2- 172.21.15.86:0/1525787383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afea9b9da0 0x55afea98ff90 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.963+0000 7f6370835740 1 --2- 172.21.15.86:0/1525787383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55afea98e3d0 0x55afea98fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.963+0000 7f6370835740 1 --2- 172.21.15.86:0/1525787383 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55afea961f50 0x55afeaa1f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.963+0000 7f634be00640 0 client.6447 ms_handle_reset on v2:172.21.15.86:6835/3008383261 2024-04-20T21:21:51.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.963+0000 7f6370835740 1 -- 172.21.15.86:0/1525787383 >> 172.21.15.86:0/1525787383 conn(0x55afea95e3f0 msgr2=0x55afea9850d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:51.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.964+0000 7f6370835740 1 -- 172.21.15.86:0/1525787383 shutdown_connections 2024-04-20T21:21:51.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.964+0000 7f6370835740 1 -- 172.21.15.86:0/1525787383 wait complete. 2024-04-20T21:21:51.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.964+0000 7f6370835740 20 client.6447 trim_cache size 0 max 16384 2024-04-20T21:21:51.968 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:21:51.968 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T21:21:51.968 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-20T21:21:51.968 INFO:teuthology.orchestra.run.smithi086.stdout: "52d9d507-79f2-48e8-97e8-45786fa716db": { 2024-04-20T21:21:51.968 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-20T21:21:51.968 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "52d9d507-79f2-48e8-97e8-45786fa716db", 2024-04-20T21:21:51.968 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,repair,force" 2024-04-20T21:21:51.969 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:21:51.969 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:21:51.969 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:21:51.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.964+0000 7f6367e00640 1 -- 172.21.15.86:0/284176180 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f634406c8f0 msgr2=0x7f634406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:51.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.964+0000 7f6367e00640 1 --2- 172.21.15.86:0/284176180 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f634406c8f0 0x7f634406edb0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6358006fd0 tx=0x7f6358008040 comp rx=0 tx=0).stop 2024-04-20T21:21:51.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.964+0000 7f6367e00640 1 -- 172.21.15.86:0/284176180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6368145250 msgr2=0x7f63680b2b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:21:51.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.964+0000 7f6367e00640 1 --2- 172.21.15.86:0/284176180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6368145250 0x7f63680b2b10 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f635000b420 tx=0x7f635000b8e0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.964+0000 7f6367e00640 1 -- 172.21.15.86:0/284176180 shutdown_connections 2024-04-20T21:21:51.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.964+0000 7f6367e00640 1 --2- 172.21.15.86:0/284176180 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f634406c8f0 0x7f634406edb0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.964+0000 7f6367e00640 1 --2- 172.21.15.86:0/284176180 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6368145f00 0x7f63680ab720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.964+0000 7f6367e00640 1 --2- 172.21.15.86:0/284176180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6368145250 0x7f63680b2b10 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.964+0000 7f6367e00640 1 --2- 172.21.15.86:0/284176180 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6368072450 0x7f63681f42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:21:51.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.964+0000 7f6367e00640 1 -- 172.21.15.86:0/284176180 >> 172.21.15.86:0/284176180 conn(0x7f6368067cf0 msgr2=0x7f6368144460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:21:51.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.965+0000 7f6367e00640 1 -- 172.21.15.86:0/284176180 shutdown_connections 2024-04-20T21:21:51.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:21:51.965+0000 7f6367e00640 1 -- 172.21.15.86:0/284176180 wait complete. 2024-04-20T21:21:51.982 INFO:tasks.cephfs.filesystem:scrub status for tag:52d9d507-79f2-48e8-97e8-45786fa716db - {'path': '/', 'tag': '52d9d507-79f2-48e8-97e8-45786fa716db', 'options': 'recursive,repair,force'} 2024-04-20T21:22:16.370 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:22:16.379 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:22:21.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:22:21.983 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-20T21:22:22.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.195+0000 7fb4d7e00640 1 Processor -- start 2024-04-20T21:22:22.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.195+0000 7fb4d7e00640 1 -- start start 2024-04-20T21:22:22.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.197+0000 7fb4d7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb4d8145250 0x7fb4d806efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:22.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.197+0000 7fb4d7e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb4d8145f00 0x7fb4d8071d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:22.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.197+0000 7fb4d7e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4d8072280 0x7fb4d806dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:22.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.197+0000 7fb4d7e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb4d80726f0 con 0x7fb4d8145250 2024-04-20T21:22:22.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.197+0000 7fb4d7e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fb4d806f500 con 0x7fb4d8145f00 2024-04-20T21:22:22.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.197+0000 7fb4d7e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb4d8160410 con 0x7fb4d8072280 2024-04-20T21:22:22.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.197+0000 7fb4d6a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb4d8145250 0x7fb4d806efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:22.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.197+0000 7fb4d6a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb4d8145250 0x7fb4d806efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47834/0 (socket says 172.21.15.86:47834) 2024-04-20T21:22:22.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.197+0000 7fb4d6a00640 1 -- 172.21.15.86:0/197631110 learned_addr learned my addr 172.21.15.86:0/197631110 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:22.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.197+0000 7fb4d6a00640 1 -- 172.21.15.86:0/197631110 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4d8072280 msgr2=0x7fb4d806dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:22.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.197+0000 7fb4d6a00640 1 --2- 172.21.15.86:0/197631110 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4d8072280 0x7fb4d806dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.197+0000 7fb4d6a00640 1 -- 172.21.15.86:0/197631110 >> v2:172.21.15.144:3300/0 conn(0x7fb4d8145f00 msgr2=0x7fb4d8071d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:22.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.197+0000 7fb4d6a00640 1 --2- 172.21.15.86:0/197631110 >> v2:172.21.15.144:3300/0 conn(0x7fb4d8145f00 0x7fb4d8071d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.197+0000 7fb4d6a00640 1 -- 172.21.15.86:0/197631110 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4d81606f0 con 0x7fb4d8145250 2024-04-20T21:22:22.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.199+0000 7fb4d6a00640 1 --2- 172.21.15.86:0/197631110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4d8145250 0x7fb4d806efc0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fb4d806c540 tx=0x7fb4c802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e69c41449e94a9fc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:22.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.199+0000 7fb4d6000640 1 -- 172.21.15.86:0/197631110 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c802c9f0 con 0x7fb4d8145250 2024-04-20T21:22:22.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.199+0000 7fb4d6000640 1 -- 172.21.15.86:0/197631110 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4c802cb90 con 0x7fb4d8145250 2024-04-20T21:22:22.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.199+0000 7fb4d6000640 1 -- 172.21.15.86:0/197631110 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c80343f0 con 0x7fb4d8145250 2024-04-20T21:22:22.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.199+0000 7fb4d7e00640 1 -- 172.21.15.86:0/197631110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4d8145250 msgr2=0x7fb4d806efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:22.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.199+0000 7fb4d7e00640 1 --2- 172.21.15.86:0/197631110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4d8145250 0x7fb4d806efc0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fb4d806c540 tx=0x7fb4c802be50 comp rx=0 tx=0).stop 2024-04-20T21:22:22.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.200+0000 7fb4d7e00640 1 -- 172.21.15.86:0/197631110 shutdown_connections 2024-04-20T21:22:22.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.200+0000 7fb4d7e00640 1 --2- 172.21.15.86:0/197631110 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4d8072280 0x7fb4d806dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.200+0000 7fb4d7e00640 1 --2- 172.21.15.86:0/197631110 >> v2:172.21.15.144:3300/0 conn(0x7fb4d8145f00 0x7fb4d8071d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.200+0000 7fb4d7e00640 1 --2- 172.21.15.86:0/197631110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4d8145250 0x7fb4d806efc0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.200+0000 7fb4d7e00640 1 -- 172.21.15.86:0/197631110 >> 172.21.15.86:0/197631110 conn(0x7fb4d8067cf0 msgr2=0x7fb4d806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:22.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.200+0000 7fb4d7e00640 1 -- 172.21.15.86:0/197631110 shutdown_connections 2024-04-20T21:22:22.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.200+0000 7fb4d7e00640 1 -- 172.21.15.86:0/197631110 wait complete. 2024-04-20T21:22:22.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.201+0000 7fb4d7e00640 1 Processor -- start 2024-04-20T21:22:22.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.202+0000 7fb4d7e00640 1 -- start start 2024-04-20T21:22:22.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.202+0000 7fb4d7e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4d8072280 0x7fb4d80ad8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:22.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.203+0000 7fb4d7e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4d8145250 0x7fb4d80b3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:22.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.203+0000 7fb4d7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4d8145f00 0x7fb4d80b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:22.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.203+0000 7fb4d7e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d8145630 con 0x7fb4d8145f00 2024-04-20T21:22:22.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.203+0000 7fb4d7e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d8160cb0 con 0x7fb4d8072280 2024-04-20T21:22:22.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.203+0000 7fb4d7e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb4d81457b0 con 0x7fb4d8145250 2024-04-20T21:22:22.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.203+0000 7fb4d7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4d8145f00 0x7fb4d80b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:22.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.203+0000 7fb4d7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4d8145f00 0x7fb4d80b3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47848/0 (socket says 172.21.15.86:47848) 2024-04-20T21:22:22.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.203+0000 7fb4d7400640 1 -- 172.21.15.86:0/3847660920 learned_addr learned my addr 172.21.15.86:0/3847660920 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:22.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.203+0000 7fb4d6a00640 1 --2- 172.21.15.86:0/3847660920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4d8072280 0x7fb4d80ad8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:22.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.203+0000 7fb4ce000640 1 --2- 172.21.15.86:0/3847660920 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4d8145250 0x7fb4d80b3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:22.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.203+0000 7fb4d7400640 1 -- 172.21.15.86:0/3847660920 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4d8145250 msgr2=0x7fb4d80b3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:22.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.203+0000 7fb4d7400640 1 --2- 172.21.15.86:0/3847660920 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4d8145250 0x7fb4d80b3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.203+0000 7fb4d7400640 1 -- 172.21.15.86:0/3847660920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4d8072280 msgr2=0x7fb4d80ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:22.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.203+0000 7fb4d7400640 1 --2- 172.21.15.86:0/3847660920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4d8072280 0x7fb4d80ad8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.203+0000 7fb4d7400640 1 -- 172.21.15.86:0/3847660920 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4d80cb2c0 con 0x7fb4d8145f00 2024-04-20T21:22:22.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.203+0000 7fb4d6a00640 1 --2- 172.21.15.86:0/3847660920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4d8072280 0x7fb4d80ad8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:22:22.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.204+0000 7fb4d7400640 1 --2- 172.21.15.86:0/3847660920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4d8145f00 0x7fb4d80b3ac0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0009fd0 tx=0x7fb4d000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:22.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.204+0000 7fb4cf400640 1 -- 172.21.15.86:0/3847660920 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d0011830 con 0x7fb4d8145f00 2024-04-20T21:22:22.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.204+0000 7fb4cf400640 1 -- 172.21.15.86:0/3847660920 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4d0016070 con 0x7fb4d8145f00 2024-04-20T21:22:22.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.204+0000 7fb4d7e00640 1 -- 172.21.15.86:0/3847660920 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4d80cb5a0 con 0x7fb4d8145f00 2024-04-20T21:22:22.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.204+0000 7fb4cf400640 1 -- 172.21.15.86:0/3847660920 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d0011bf0 con 0x7fb4d8145f00 2024-04-20T21:22:22.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.204+0000 7fb4d7e00640 1 -- 172.21.15.86:0/3847660920 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4d80cbb60 con 0x7fb4d8145f00 2024-04-20T21:22:22.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.206+0000 7fb4cf400640 1 -- 172.21.15.86:0/3847660920 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb4d001b460 con 0x7fb4d8145f00 2024-04-20T21:22:22.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.207+0000 7fb4cf400640 1 --2- 172.21.15.86:0/3847660920 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb4b406c920 0x7fb4b406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:22.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.207+0000 7fb4cf400640 1 -- 172.21.15.86:0/3847660920 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fb4d008b870 con 0x7fb4d8145f00 2024-04-20T21:22:22.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.208+0000 7fb4d6a00640 1 --2- 172.21.15.86:0/3847660920 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb4b406c920 0x7fb4b406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:22.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.208+0000 7fb4e05a4740 1 Processor -- start 2024-04-20T21:22:22.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.208+0000 7fb4d6a00640 1 --2- 172.21.15.86:0/3847660920 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb4b406c920 0x7fb4b406ede0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8035000 tx=0x7fb4c8030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:22.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.208+0000 7fb4e05a4740 1 -- start start 2024-04-20T21:22:22.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.209+0000 7fb4e05a4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55df8b0383d0 0x55df8b0387b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:22.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.209+0000 7fb4e05a4740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55df8b0083f0 0x55df8b035640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:22.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.209+0000 7fb4e05a4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df8b00bf50 0x55df8b035b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:22.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.209+0000 7fb4e05a4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55df8af74150 con 0x55df8b00bf50 2024-04-20T21:22:22.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.209+0000 7fb4e05a4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55df8af1a370 con 0x55df8b0383d0 2024-04-20T21:22:22.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.209+0000 7fb4e05a4740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55df8af70110 con 0x55df8b0083f0 2024-04-20T21:22:22.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.210+0000 7fb4d7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df8b00bf50 0x55df8b035b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:22.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.210+0000 7fb4d7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df8b00bf50 0x55df8b035b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47856/0 (socket says 172.21.15.86:47856) 2024-04-20T21:22:22.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.210+0000 7fb4d7400640 1 -- 172.21.15.86:0/3492379984 learned_addr learned my addr 172.21.15.86:0/3492379984 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:22.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.210+0000 7fb4d6a00640 1 --2- 172.21.15.86:0/3492379984 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55df8b0383d0 0x55df8b0387b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:22.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.210+0000 7fb4ce000640 1 --2- 172.21.15.86:0/3492379984 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55df8b0083f0 0x55df8b035640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:22.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.210+0000 7fb4d7400640 1 -- 172.21.15.86:0/3492379984 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55df8b0083f0 msgr2=0x55df8b035640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:22.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.210+0000 7fb4d7400640 1 --2- 172.21.15.86:0/3492379984 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55df8b0083f0 0x55df8b035640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.210+0000 7fb4d7400640 1 -- 172.21.15.86:0/3492379984 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55df8b0383d0 msgr2=0x55df8b0387b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:22.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.210+0000 7fb4d7400640 1 --2- 172.21.15.86:0/3492379984 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55df8b0383d0 0x55df8b0387b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.210+0000 7fb4d7400640 1 -- 172.21.15.86:0/3492379984 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df8ad507d0 con 0x55df8b00bf50 2024-04-20T21:22:22.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.210+0000 7fb4d7400640 1 --2- 172.21.15.86:0/3492379984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df8b00bf50 0x55df8b035b80 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0061c40 tx=0x7fb4d000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d7b713c90c3a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:22.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.211+0000 7fb4c7400640 1 -- 172.21.15.86:0/3492379984 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d0057890 con 0x55df8b00bf50 2024-04-20T21:22:22.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.211+0000 7fb4c7400640 1 -- 172.21.15.86:0/3492379984 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4d0055c00 con 0x55df8b00bf50 2024-04-20T21:22:22.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.211+0000 7fb4c7400640 1 -- 172.21.15.86:0/3492379984 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d0061f20 con 0x55df8b00bf50 2024-04-20T21:22:22.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.211+0000 7fb4e05a4740 1 -- 172.21.15.86:0/3492379984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df8b00bf50 msgr2=0x55df8b035b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:22.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.211+0000 7fb4e05a4740 1 --2- 172.21.15.86:0/3492379984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df8b00bf50 0x55df8b035b80 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0061c40 tx=0x7fb4d000bf90 comp rx=0 tx=0).stop 2024-04-20T21:22:22.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.211+0000 7fb4e05a4740 1 -- 172.21.15.86:0/3492379984 shutdown_connections 2024-04-20T21:22:22.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.211+0000 7fb4e05a4740 1 --2- 172.21.15.86:0/3492379984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df8b00bf50 0x55df8b035b80 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.211+0000 7fb4e05a4740 1 --2- 172.21.15.86:0/3492379984 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55df8b0083f0 0x55df8b035640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.211+0000 7fb4e05a4740 1 --2- 172.21.15.86:0/3492379984 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55df8b0383d0 0x55df8b0387b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.211+0000 7fb4e05a4740 1 -- 172.21.15.86:0/3492379984 >> 172.21.15.86:0/3492379984 conn(0x55df8b00a660 msgr2=0x55df8b0531e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:22.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.212+0000 7fb4e05a4740 1 -- 172.21.15.86:0/3492379984 shutdown_connections 2024-04-20T21:22:22.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.212+0000 7fb4e05a4740 1 -- 172.21.15.86:0/3492379984 wait complete. 2024-04-20T21:22:22.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.212+0000 7fb4e05a4740 1 -- start start 2024-04-20T21:22:22.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.213+0000 7fb4e05a4740 1 Processor -- start 2024-04-20T21:22:22.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.213+0000 7fb4e05a4740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55df8b00bf50 0x55df8b067360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:22.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.213+0000 7fb4e05a4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55df8b0383d0 0x55df8b039970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:22.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.214+0000 7fb4e05a4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df8b039eb0 0x55df8b022080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:22.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.214+0000 7fb4e05a4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55df8b00d100 con 0x55df8b039eb0 2024-04-20T21:22:22.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.214+0000 7fb4e05a4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55df8af2d740 con 0x55df8b0383d0 2024-04-20T21:22:22.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.214+0000 7fb4e05a4740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55df8b00c4f0 con 0x55df8b00bf50 2024-04-20T21:22:22.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.214+0000 7fb4e05a4740 10 client.0 _ll_register_callbacks cb 0x55df8af28b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:22:22.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.214+0000 7fb4d7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df8b039eb0 0x55df8b022080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:22.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.214+0000 7fb4d7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df8b039eb0 0x55df8b022080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47858/0 (socket says 172.21.15.86:47858) 2024-04-20T21:22:22.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.214+0000 7fb4d7400640 1 -- 172.21.15.86:0/1130852741 learned_addr learned my addr 172.21.15.86:0/1130852741 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:22.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.214+0000 7fb4d7400640 1 -- 172.21.15.86:0/1130852741 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55df8b00bf50 msgr2=0x55df8b067360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:22.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.214+0000 7fb4d7400640 1 --2- 172.21.15.86:0/1130852741 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55df8b00bf50 0x55df8b067360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.214+0000 7fb4ce000640 1 --2- 172.21.15.86:0/1130852741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55df8b0383d0 0x55df8b039970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:22.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.214+0000 7fb4d7400640 1 -- 172.21.15.86:0/1130852741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55df8b0383d0 msgr2=0x55df8b039970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:22.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.214+0000 7fb4d7400640 1 --2- 172.21.15.86:0/1130852741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55df8b0383d0 0x55df8b039970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.214+0000 7fb4d7400640 1 -- 172.21.15.86:0/1130852741 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df8ac863f0 con 0x55df8b039eb0 2024-04-20T21:22:22.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.214+0000 7fb4ce000640 1 --2- 172.21.15.86:0/1130852741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55df8b0383d0 0x55df8b039970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:22:22.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.215+0000 7fb4d7400640 1 --2- 172.21.15.86:0/1130852741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df8b039eb0 0x55df8b022080 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0013010 tx=0x7fb4d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46610ed082a4488f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:22.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.215+0000 7fb4bbe00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:22:22.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.215+0000 7fb4bbe00640 1 -- 172.21.15.86:0/1130852741 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d0089050 con 0x55df8b039eb0 2024-04-20T21:22:22.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.215+0000 7fb4bbe00640 1 -- 172.21.15.86:0/1130852741 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fb4d00589e0 con 0x55df8b039eb0 2024-04-20T21:22:22.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.215+0000 7fb4e05a4740 1 -- 172.21.15.86:0/1130852741 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4d0055c00 con 0x55df8b039eb0 2024-04-20T21:22:22.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.215+0000 7fb4bbe00640 1 -- 172.21.15.86:0/1130852741 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb4d007f050 con 0x55df8b039eb0 2024-04-20T21:22:22.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.215+0000 7fb4e05a4740 10 client.6459 fetch_fsmap learned FSMap version 34 2024-04-20T21:22:22.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.215+0000 7fb4e05a4740 1 -- 172.21.15.86:0/1130852741 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x55df8ac460f0 con 0x55df8b039eb0 2024-04-20T21:22:22.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.215+0000 7fb4bbe00640 1 -- 172.21.15.86:0/1130852741 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4d0055c00 con 0x55df8b039eb0 2024-04-20T21:22:22.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.215+0000 7fb4bbe00640 1 -- 172.21.15.86:0/1130852741 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 34) ==== 3495+0+0 (secure 0 0 0) 0x7fb4d0082020 con 0x55df8b039eb0 2024-04-20T21:22:22.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.215+0000 7fb4e05a4740 10 client.6459 fetch_fsmap finished waiting for FSMap version 34 2024-04-20T21:22:22.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.215+0000 7fb4e05a4740 10 client.6459 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-20T21:22:22.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.215+0000 7fb4e05a4740 20 client.6459 populate_metadata read hostname 'smithi086' 2024-04-20T21:22:22.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.215+0000 7fb4e05a4740 1 --2- 172.21.15.86:0/1130852741 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x55df8b023450 0x55df8b023830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:22.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.215+0000 7fb4e05a4740 4 client.6459 mds_command: new command op to 6135 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:22:22.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.215+0000 7fb4e05a4740 1 -- 172.21.15.86:0/1130852741 --> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df8af81b70 con 0x55df8b023450 2024-04-20T21:22:22.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.216+0000 7fb4d6a00640 1 --2- 172.21.15.86:0/1130852741 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x55df8b023450 0x55df8b023830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:22.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.216+0000 7fb4d6a00640 1 --2- 172.21.15.86:0/1130852741 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x55df8b023450 0x55df8b023830 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50772b7ebdb5a733 server_cookie=504fcb500c1f21f in_seq=0 out_seq=0 2024-04-20T21:22:22.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.216+0000 7fb4bbe00640 10 client.6459 ms_handle_connect on v2:172.21.15.86:6835/3008383261 2024-04-20T21:22:22.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.217+0000 7fb4bbe00640 1 -- 172.21.15.86:0/1130852741 <== mds.0 v2:172.21.15.86:6835/3008383261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55df8af81b70 con 0x55df8b023450 2024-04-20T21:22:22.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.217+0000 7fb4bbe00640 10 client.6459 handle_command_reply: tid=0 2024-04-20T21:22:22.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.217+0000 7fb4e05a4740 1 client.6459 shutdown 2024-04-20T21:22:22.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.217+0000 7fb4e05a4740 1 -- 172.21.15.86:0/1130852741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df8b039eb0 msgr2=0x55df8b022080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:22.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.217+0000 7fb4e05a4740 1 --2- 172.21.15.86:0/1130852741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df8b039eb0 0x55df8b022080 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0013010 tx=0x7fb4d00046c0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.217+0000 7fb4e05a4740 1 -- 172.21.15.86:0/1130852741 shutdown_connections 2024-04-20T21:22:22.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.217+0000 7fb4e05a4740 1 --2- 172.21.15.86:0/1130852741 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x55df8b023450 0x55df8b023830 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.217+0000 7fb4e05a4740 1 --2- 172.21.15.86:0/1130852741 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df8b039eb0 0x55df8b022080 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.217+0000 7fb4e05a4740 1 --2- 172.21.15.86:0/1130852741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55df8b0383d0 0x55df8b039970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.217+0000 7fb4bbe00640 0 client.6459 ms_handle_reset on v2:172.21.15.86:6835/3008383261 2024-04-20T21:22:22.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.217+0000 7fb4e05a4740 1 --2- 172.21.15.86:0/1130852741 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55df8b00bf50 0x55df8b067360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.217+0000 7fb4e05a4740 1 -- 172.21.15.86:0/1130852741 >> 172.21.15.86:0/1130852741 conn(0x55df8b0083f0 msgr2=0x55df8b0566f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:22.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.218+0000 7fb4e05a4740 1 -- 172.21.15.86:0/1130852741 shutdown_connections 2024-04-20T21:22:22.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.218+0000 7fb4e05a4740 1 -- 172.21.15.86:0/1130852741 wait complete. 2024-04-20T21:22:22.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.218+0000 7fb4e05a4740 20 client.6459 trim_cache size 0 max 16384 2024-04-20T21:22:22.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.237+0000 7fb4e05a4740 1 Processor -- start 2024-04-20T21:22:22.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.237+0000 7fb4e05a4740 1 -- start start 2024-04-20T21:22:22.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.238+0000 7fb4e05a4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55df8b00bf50 0x55df8b0566f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:22.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.238+0000 7fb4e05a4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df8b0383d0 0x55df8b056c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:22.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.238+0000 7fb4e05a4740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55df8b039eb0 0x55df8b065670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:22.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.238+0000 7fb4e05a4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d8200da0 con 0x55df8b0383d0 2024-04-20T21:22:22.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.238+0000 7fb4e05a4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d81ffd90 con 0x55df8b00bf50 2024-04-20T21:22:22.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.238+0000 7fb4e05a4740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb4d81ff0f0 con 0x55df8b039eb0 2024-04-20T21:22:22.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.238+0000 7fb4ce000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df8b0383d0 0x55df8b056c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:22.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.238+0000 7fb4ce000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df8b0383d0 0x55df8b056c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47860/0 (socket says 172.21.15.86:47860) 2024-04-20T21:22:22.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.238+0000 7fb4ce000640 1 -- 172.21.15.86:0/2592508452 learned_addr learned my addr 172.21.15.86:0/2592508452 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:22.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.238+0000 7fb4ce000640 1 -- 172.21.15.86:0/2592508452 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55df8b039eb0 msgr2=0x55df8b065670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:22.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.238+0000 7fb4ce000640 1 --2- 172.21.15.86:0/2592508452 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55df8b039eb0 0x55df8b065670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.238+0000 7fb4ce000640 1 -- 172.21.15.86:0/2592508452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55df8b00bf50 msgr2=0x55df8b0566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:22.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.238+0000 7fb4ce000640 1 --2- 172.21.15.86:0/2592508452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55df8b00bf50 0x55df8b0566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.238+0000 7fb4ce000640 1 -- 172.21.15.86:0/2592508452 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df8b067dc0 con 0x55df8b0383d0 2024-04-20T21:22:22.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.239+0000 7fb4ce000640 1 --2- 172.21.15.86:0/2592508452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df8b0383d0 0x55df8b056c30 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fb4c000c770 tx=0x7fb4c000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cfe24726e1f8d2d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:22.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.239+0000 7fb4c6a00640 1 -- 172.21.15.86:0/2592508452 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c000ec20 con 0x55df8b0383d0 2024-04-20T21:22:22.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.239+0000 7fb4c6a00640 1 -- 172.21.15.86:0/2592508452 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4c000edc0 con 0x55df8b0383d0 2024-04-20T21:22:22.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.239+0000 7fb4c6a00640 1 -- 172.21.15.86:0/2592508452 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c0012630 con 0x55df8b0383d0 2024-04-20T21:22:22.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.240+0000 7fb4e05a4740 1 -- 172.21.15.86:0/2592508452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df8b0383d0 msgr2=0x55df8b056c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:22.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.240+0000 7fb4e05a4740 1 --2- 172.21.15.86:0/2592508452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df8b0383d0 0x55df8b056c30 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fb4c000c770 tx=0x7fb4c000cc30 comp rx=0 tx=0).stop 2024-04-20T21:22:22.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.240+0000 7fb4e05a4740 1 -- 172.21.15.86:0/2592508452 shutdown_connections 2024-04-20T21:22:22.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.240+0000 7fb4e05a4740 1 --2- 172.21.15.86:0/2592508452 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55df8b039eb0 0x55df8b065670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.240+0000 7fb4e05a4740 1 --2- 172.21.15.86:0/2592508452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df8b0383d0 0x55df8b056c30 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.240+0000 7fb4e05a4740 1 --2- 172.21.15.86:0/2592508452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55df8b00bf50 0x55df8b0566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.240+0000 7fb4e05a4740 1 -- 172.21.15.86:0/2592508452 >> 172.21.15.86:0/2592508452 conn(0x55df8b0083f0 msgr2=0x55df8b003b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:22.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.240+0000 7fb4e05a4740 1 -- 172.21.15.86:0/2592508452 shutdown_connections 2024-04-20T21:22:22.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.240+0000 7fb4e05a4740 1 -- 172.21.15.86:0/2592508452 wait complete. 2024-04-20T21:22:22.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.240+0000 7fb4e05a4740 1 -- start start 2024-04-20T21:22:22.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.241+0000 7fb4e05a4740 1 Processor -- start 2024-04-20T21:22:22.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.242+0000 7fb4e05a4740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55df8b00bf50 0x55df8b0d4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:22.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.242+0000 7fb4e05a4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df8b0383d0 0x55df8b0566f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:22.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.242+0000 7fb4e05a4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55df8b039eb0 0x55df8b056c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:22.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.242+0000 7fb4e05a4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55df8b034db0 con 0x55df8b0383d0 2024-04-20T21:22:22.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.242+0000 7fb4e05a4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d81ff740 con 0x55df8b039eb0 2024-04-20T21:22:22.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.242+0000 7fb4e05a4740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55df8b00f390 con 0x55df8b00bf50 2024-04-20T21:22:22.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.242+0000 7fb4e05a4740 10 client.0 _ll_register_callbacks cb 0x7fb4d0059d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:22:22.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.242+0000 7fb4ce000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df8b0383d0 0x55df8b0566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:22.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.242+0000 7fb4ce000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df8b0383d0 0x55df8b0566f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47862/0 (socket says 172.21.15.86:47862) 2024-04-20T21:22:22.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.242+0000 7fb4ce000640 1 -- 172.21.15.86:0/3410917706 learned_addr learned my addr 172.21.15.86:0/3410917706 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:22.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.242+0000 7fb4ce000640 1 -- 172.21.15.86:0/3410917706 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55df8b00bf50 msgr2=0x55df8b0d4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:22.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.242+0000 7fb4ce000640 1 --2- 172.21.15.86:0/3410917706 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55df8b00bf50 0x55df8b0d4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.242+0000 7fb4ce000640 1 -- 172.21.15.86:0/3410917706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55df8b039eb0 msgr2=0x55df8b056c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:22.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.242+0000 7fb4ce000640 1 --2- 172.21.15.86:0/3410917706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55df8b039eb0 0x55df8b056c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.242+0000 7fb4ce000640 1 -- 172.21.15.86:0/3410917706 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df8b0683a0 con 0x55df8b0383d0 2024-04-20T21:22:22.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.243+0000 7fb4ce000640 1 --2- 172.21.15.86:0/3410917706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df8b0383d0 0x55df8b0566f0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0016000 tx=0x7fb4c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36bcf86885be0327 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:22.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.243+0000 7fb4bbe00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:22:22.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.243+0000 7fb4bbe00640 1 -- 172.21.15.86:0/3410917706 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c0012b70 con 0x55df8b0383d0 2024-04-20T21:22:22.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.243+0000 7fb4bbe00640 1 -- 172.21.15.86:0/3410917706 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fb4c0012d10 con 0x55df8b0383d0 2024-04-20T21:22:22.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.243+0000 7fb4e05a4740 1 -- 172.21.15.86:0/3410917706 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4c000edc0 con 0x55df8b0383d0 2024-04-20T21:22:22.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.243+0000 7fb4bbe00640 1 -- 172.21.15.86:0/3410917706 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb4c002e050 con 0x55df8b0383d0 2024-04-20T21:22:22.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.243+0000 7fb4e05a4740 10 client.6465 fetch_fsmap learned FSMap version 34 2024-04-20T21:22:22.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.243+0000 7fb4e05a4740 1 -- 172.21.15.86:0/3410917706 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x55df8b066650 con 0x55df8b0383d0 2024-04-20T21:22:22.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.243+0000 7fb4bbe00640 1 -- 172.21.15.86:0/3410917706 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4c000edc0 con 0x55df8b0383d0 2024-04-20T21:22:22.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.243+0000 7fb4bbe00640 1 -- 172.21.15.86:0/3410917706 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 34) ==== 3495+0+0 (secure 0 0 0) 0x7fb4c0031020 con 0x55df8b0383d0 2024-04-20T21:22:22.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.243+0000 7fb4e05a4740 10 client.6465 fetch_fsmap finished waiting for FSMap version 34 2024-04-20T21:22:22.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.243+0000 7fb4e05a4740 10 client.6465 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-20T21:22:22.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.243+0000 7fb4e05a4740 20 client.6465 populate_metadata read hostname 'smithi086' 2024-04-20T21:22:22.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.244+0000 7fb4e05a4740 1 --2- 172.21.15.86:0/3410917706 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x55df8b04df60 0x55df8b065670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:22.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.244+0000 7fb4e05a4740 4 client.6465 mds_command: new command op to 6135 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:22:22.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.244+0000 7fb4e05a4740 1 -- 172.21.15.86:0/3410917706 --> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55df8af7b9e0 con 0x55df8b04df60 2024-04-20T21:22:22.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.244+0000 7fb4d6a00640 1 --2- 172.21.15.86:0/3410917706 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x55df8b04df60 0x55df8b065670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:22.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.244+0000 7fb4d6a00640 1 --2- 172.21.15.86:0/3410917706 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x55df8b04df60 0x55df8b065670 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbe68d8f6b18bb9f server_cookie=b4982c01128401b9 in_seq=0 out_seq=0 2024-04-20T21:22:22.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.244+0000 7fb4bbe00640 10 client.6465 ms_handle_connect on v2:172.21.15.86:6835/3008383261 2024-04-20T21:22:22.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.244+0000 7fb4bbe00640 1 -- 172.21.15.86:0/3410917706 <== mds.0 v2:172.21.15.86:6835/3008383261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55df8af7b9e0 con 0x55df8b04df60 2024-04-20T21:22:22.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.244+0000 7fb4bbe00640 10 client.6465 handle_command_reply: tid=0 2024-04-20T21:22:22.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.245+0000 7fb4e05a4740 1 client.6465 shutdown 2024-04-20T21:22:22.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.245+0000 7fb4e05a4740 1 -- 172.21.15.86:0/3410917706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df8b0383d0 msgr2=0x55df8b0566f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:22.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.245+0000 7fb4e05a4740 1 --2- 172.21.15.86:0/3410917706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df8b0383d0 0x55df8b0566f0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0016000 tx=0x7fb4c00046c0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.245+0000 7fb4e05a4740 1 -- 172.21.15.86:0/3410917706 shutdown_connections 2024-04-20T21:22:22.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.245+0000 7fb4e05a4740 1 --2- 172.21.15.86:0/3410917706 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x55df8b04df60 0x55df8b065670 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.245+0000 7fb4e05a4740 1 --2- 172.21.15.86:0/3410917706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55df8b039eb0 0x55df8b056c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.245+0000 7fb4e05a4740 1 --2- 172.21.15.86:0/3410917706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55df8b0383d0 0x55df8b0566f0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.245+0000 7fb4bbe00640 0 client.6465 ms_handle_reset on v2:172.21.15.86:6835/3008383261 2024-04-20T21:22:22.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.245+0000 7fb4e05a4740 1 --2- 172.21.15.86:0/3410917706 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55df8b00bf50 0x55df8b0d4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.245+0000 7fb4e05a4740 1 -- 172.21.15.86:0/3410917706 >> 172.21.15.86:0/3410917706 conn(0x55df8b0083f0 msgr2=0x55df8b0344d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:22.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.245+0000 7fb4e05a4740 1 -- 172.21.15.86:0/3410917706 shutdown_connections 2024-04-20T21:22:22.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.245+0000 7fb4e05a4740 1 -- 172.21.15.86:0/3410917706 wait complete. 2024-04-20T21:22:22.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.245+0000 7fb4e05a4740 20 client.6465 trim_cache size 0 max 16384 2024-04-20T21:22:22.248 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:22:22.248 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-20T21:22:22.248 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-20T21:22:22.248 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:22:22.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.246+0000 7fb4d7e00640 1 -- 172.21.15.86:0/3847660920 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb4b406c920 msgr2=0x7fb4b406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:22.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.246+0000 7fb4d7e00640 1 --2- 172.21.15.86:0/3847660920 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb4b406c920 0x7fb4b406ede0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8035000 tx=0x7fb4c8030000 comp rx=0 tx=0).stop 2024-04-20T21:22:22.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.246+0000 7fb4d7e00640 1 -- 172.21.15.86:0/3847660920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4d8145f00 msgr2=0x7fb4d80b3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:22.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.246+0000 7fb4d7e00640 1 --2- 172.21.15.86:0/3847660920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4d8145f00 0x7fb4d80b3ac0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0009fd0 tx=0x7fb4d000b8e0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.246+0000 7fb4d7e00640 1 -- 172.21.15.86:0/3847660920 shutdown_connections 2024-04-20T21:22:22.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.246+0000 7fb4d7e00640 1 --2- 172.21.15.86:0/3847660920 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb4b406c920 0x7fb4b406ede0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.246+0000 7fb4d7e00640 1 --2- 172.21.15.86:0/3847660920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4d8145f00 0x7fb4d80b3ac0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.246+0000 7fb4d7e00640 1 --2- 172.21.15.86:0/3847660920 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4d8145250 0x7fb4d80b3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.246+0000 7fb4d7e00640 1 --2- 172.21.15.86:0/3847660920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4d8072280 0x7fb4d80ad8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.246+0000 7fb4d7e00640 1 -- 172.21.15.86:0/3847660920 >> 172.21.15.86:0/3847660920 conn(0x7fb4d8067cf0 msgr2=0x7fb4d815f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:22.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.246+0000 7fb4d7e00640 1 -- 172.21.15.86:0/3847660920 shutdown_connections 2024-04-20T21:22:22.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.246+0000 7fb4d7e00640 1 -- 172.21.15.86:0/3847660920 wait complete. 2024-04-20T21:22:22.262 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T21:22:22.262 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:22:22.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.453+0000 7f2601e00640 1 Processor -- start 2024-04-20T21:22:22.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.453+0000 7f2601e00640 1 -- start start 2024-04-20T21:22:22.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.455+0000 7f2601e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f25fc162000 0x7f25fc0b76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:22.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.455+0000 7f2601e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25fc0cff70 con 0x7f25fc0cfb20 2024-04-20T21:22:22.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.455+0000 7f2601e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25fc0d01e0 con 0x7f25fc0d11d0 2024-04-20T21:22:22.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.455+0000 7f2601e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f25fc0d04a0 con 0x7f25fc162000 2024-04-20T21:22:22.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.455+0000 7f25faa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f25fc162000 0x7f25fc0b76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:22.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.455+0000 7f25fb400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f25fc0cfb20 0x7f25fc0d2c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51180/0 (socket says 172.21.15.86:51180) 2024-04-20T21:22:22.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.455+0000 7f25fb400640 1 -- 172.21.15.86:0/1583116450 learned_addr learned my addr 172.21.15.86:0/1583116450 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:22:22.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.455+0000 7f25faa00640 1 --2- 172.21.15.86:0/1583116450 >> v2:172.21.15.86:3300/0 conn(0x7f25fc162000 0x7f25fc0b76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47872/0 (socket says 172.21.15.86:47872) 2024-04-20T21:22:22.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.456+0000 7f25faa00640 1 -- 172.21.15.86:0/1583116450 >> v1:172.21.15.144:6789/0 conn(0x7f25fc0d11d0 legacy=0x7f25fc0cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:22.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.456+0000 7f25faa00640 1 -- 172.21.15.86:0/1583116450 >> v1:172.21.15.86:6789/0 conn(0x7f25fc0cfb20 legacy=0x7f25fc0d2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:22.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.456+0000 7f25faa00640 1 -- 172.21.15.86:0/1583116450 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25fc0d0780 con 0x7f25fc162000 2024-04-20T21:22:22.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.456+0000 7f25f9600640 1 -- 172.21.15.86:0/1583116450 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3392320759 0 0) 0x7f25fc0cff70 con 0x7f25fc0cfb20 2024-04-20T21:22:22.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.457+0000 7f25faa00640 1 --2- 172.21.15.86:0/1583116450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25fc162000 0x7f25fc0b76c0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f25ec0097a0 tx=0x7f25ec02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=892945e044b22afc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:22.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.458+0000 7f25f9600640 1 -- 172.21.15.86:0/1583116450 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25ec0315a0 con 0x7f25fc162000 2024-04-20T21:22:22.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.458+0000 7f25f9600640 1 -- 172.21.15.86:0/1583116450 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25ec037070 con 0x7f25fc162000 2024-04-20T21:22:22.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.458+0000 7f25f9600640 1 -- 172.21.15.86:0/1583116450 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25ec031960 con 0x7f25fc162000 2024-04-20T21:22:22.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.458+0000 7f2601e00640 1 -- 172.21.15.86:0/1583116450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25fc162000 msgr2=0x7f25fc0b76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:22.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.458+0000 7f2601e00640 1 --2- 172.21.15.86:0/1583116450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25fc162000 0x7f25fc0b76c0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f25ec0097a0 tx=0x7f25ec02b840 comp rx=0 tx=0).stop 2024-04-20T21:22:22.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.458+0000 7f2601e00640 1 -- 172.21.15.86:0/1583116450 shutdown_connections 2024-04-20T21:22:22.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.458+0000 7f2601e00640 1 --2- 172.21.15.86:0/1583116450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25fc162000 0x7f25fc0b76c0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.458+0000 7f2601e00640 1 -- 172.21.15.86:0/1583116450 >> 172.21.15.86:0/1583116450 conn(0x7f25fc08bf70 msgr2=0x7f25fc0d4870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:22.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.458+0000 7f2601e00640 1 -- 172.21.15.86:0/1583116450 shutdown_connections 2024-04-20T21:22:22.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.458+0000 7f2601e00640 1 -- 172.21.15.86:0/1583116450 wait complete. 2024-04-20T21:22:22.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.460+0000 7f2601e00640 1 Processor -- start 2024-04-20T21:22:22.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.460+0000 7f2601e00640 1 -- start start 2024-04-20T21:22:22.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.461+0000 7f2601e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25fc0cfb20 0x7f25fc1f83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:22.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.461+0000 7f2601e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25fc0d11d0 0x7f25fc0cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:22.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.461+0000 7f2601e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25fc162000 0x7f25fc158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:22.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.461+0000 7f2601e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f25fc1623e0 con 0x7f25fc162000 2024-04-20T21:22:22.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.461+0000 7f2601e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f25fc0b7c00 con 0x7f25fc0cfb20 2024-04-20T21:22:22.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.461+0000 7f2601e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f25fc162560 con 0x7f25fc0d11d0 2024-04-20T21:22:22.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.461+0000 7f25fb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25fc162000 0x7f25fc158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:22.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.461+0000 7f25fb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25fc162000 0x7f25fc158ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47878/0 (socket says 172.21.15.86:47878) 2024-04-20T21:22:22.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.461+0000 7f25fb400640 1 -- 172.21.15.86:0/3642202502 learned_addr learned my addr 172.21.15.86:0/3642202502 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:22.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.462+0000 7f25faa00640 1 --2- 172.21.15.86:0/3642202502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25fc0cfb20 0x7f25fc1f83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:22.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.462+0000 7f25fa000640 1 --2- 172.21.15.86:0/3642202502 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25fc0d11d0 0x7f25fc0cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:22.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.462+0000 7f25fb400640 1 -- 172.21.15.86:0/3642202502 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25fc0d11d0 msgr2=0x7f25fc0cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:22.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.462+0000 7f25fb400640 1 --2- 172.21.15.86:0/3642202502 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25fc0d11d0 0x7f25fc0cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.462+0000 7f25fb400640 1 -- 172.21.15.86:0/3642202502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25fc0cfb20 msgr2=0x7f25fc1f83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:22.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.462+0000 7f25fb400640 1 --2- 172.21.15.86:0/3642202502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25fc0cfb20 0x7f25fc1f83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.462+0000 7f25fb400640 1 -- 172.21.15.86:0/3642202502 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25fc0ce4c0 con 0x7f25fc162000 2024-04-20T21:22:22.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.462+0000 7f25fb400640 1 --2- 172.21.15.86:0/3642202502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25fc162000 0x7f25fc158ce0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f25f400c730 tx=0x7f25f400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:22.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.463+0000 7f25f3400640 1 -- 172.21.15.86:0/3642202502 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f4018070 con 0x7f25fc162000 2024-04-20T21:22:22.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.463+0000 7f25f3400640 1 -- 172.21.15.86:0/3642202502 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25f400ce40 con 0x7f25fc162000 2024-04-20T21:22:22.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.463+0000 7f25f3400640 1 -- 172.21.15.86:0/3642202502 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f400a860 con 0x7f25fc162000 2024-04-20T21:22:22.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.463+0000 7f2601e00640 1 -- 172.21.15.86:0/3642202502 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25fc0ce740 con 0x7f25fc162000 2024-04-20T21:22:22.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.463+0000 7f2601e00640 1 -- 172.21.15.86:0/3642202502 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25fc0cbc20 con 0x7f25fc162000 2024-04-20T21:22:22.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.464+0000 7f2601e00640 1 -- 172.21.15.86:0/3642202502 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25fc0689e0 con 0x7f25fc162000 2024-04-20T21:22:22.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.465+0000 7f25f3400640 1 -- 172.21.15.86:0/3642202502 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f25f400aa00 con 0x7f25fc162000 2024-04-20T21:22:22.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.466+0000 7f25f3400640 1 --2- 172.21.15.86:0/3642202502 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f25dc06c8f0 0x7f25dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:22.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.466+0000 7f25f3400640 1 -- 172.21.15.86:0/3642202502 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f25f408b140 con 0x7f25fc162000 2024-04-20T21:22:22.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.466+0000 7f25faa00640 1 --2- 172.21.15.86:0/3642202502 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f25dc06c8f0 0x7f25dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:22.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.467+0000 7f25faa00640 1 --2- 172.21.15.86:0/3642202502 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f25dc06c8f0 0x7f25dc06edb0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f25ec036000 tx=0x7f25ec008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:22.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.471+0000 7f25f3400640 1 -- 172.21.15.86:0/3642202502 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f25fc0ce4c0 con 0x7f25fc162000 2024-04-20T21:22:22.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.711+0000 7f2601e00640 1 -- 172.21.15.86:0/3642202502 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f25fc0d1ef0 con 0x7f25fc162000 2024-04-20T21:22:22.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.712+0000 7f25f3400640 1 -- 172.21.15.86:0/3642202502 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5206 (secure 0 0 0) 0x7f25f4058e80 con 0x7f25fc162000 2024-04-20T21:22:22.712 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:22:22.712 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4913,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3933408964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3933408964},{"type":"v1","addr":"172.21.15.144:6833","nonce":3933408964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5221,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/1522070692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1522070692},{"type":"v1","addr":"172.21.15.144:6835","nonce":1522070692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":6132,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/1352073341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1352073341},{"type":"v1","addr":"172.21.15.86:6836","nonce":1352073341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:21:07.502800+0000","modified":"2024-04-20T21:22:00.123314+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6135},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6135":{"gid":6135,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":16,"addr":"172.21.15.86:6837/3008383261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3008383261},{"type":"v1","addr":"172.21.15.86:6837","nonce":3008383261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6135,"qdb_cluster":[6135]},"id":4}]} 2024-04-20T21:22:22.712 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 34 2024-04-20T21:22:22.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.715+0000 7f2601e00640 1 -- 172.21.15.86:0/3642202502 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f25dc06c8f0 msgr2=0x7f25dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:22.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.715+0000 7f2601e00640 1 --2- 172.21.15.86:0/3642202502 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f25dc06c8f0 0x7f25dc06edb0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f25ec036000 tx=0x7f25ec008030 comp rx=0 tx=0).stop 2024-04-20T21:22:22.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.715+0000 7f2601e00640 1 -- 172.21.15.86:0/3642202502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25fc162000 msgr2=0x7f25fc158ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:22.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.715+0000 7f2601e00640 1 --2- 172.21.15.86:0/3642202502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25fc162000 0x7f25fc158ce0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f25f400c730 tx=0x7f25f400cbf0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.715+0000 7f2601e00640 1 -- 172.21.15.86:0/3642202502 shutdown_connections 2024-04-20T21:22:22.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.715+0000 7f2601e00640 1 --2- 172.21.15.86:0/3642202502 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f25dc06c8f0 0x7f25dc06edb0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.715+0000 7f2601e00640 1 --2- 172.21.15.86:0/3642202502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25fc162000 0x7f25fc158ce0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.715+0000 7f2601e00640 1 --2- 172.21.15.86:0/3642202502 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25fc0d11d0 0x7f25fc0cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.715+0000 7f2601e00640 1 --2- 172.21.15.86:0/3642202502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25fc0cfb20 0x7f25fc1f83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.715+0000 7f2601e00640 1 -- 172.21.15.86:0/3642202502 >> 172.21.15.86:0/3642202502 conn(0x7f25fc08bf70 msgr2=0x7f25fc0b9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:22.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.716+0000 7f2601e00640 1 -- 172.21.15.86:0/3642202502 shutdown_connections 2024-04-20T21:22:22.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.716+0000 7f2601e00640 1 -- 172.21.15.86:0/3642202502 wait complete. 2024-04-20T21:22:22.733 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-20T21:22:22.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.934+0000 7f5491800640 1 Processor -- start 2024-04-20T21:22:22.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.935+0000 7f5491800640 1 -- start start 2024-04-20T21:22:22.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.936+0000 7f5491800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f548c162ed0 0x7f548c06b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:22.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.936+0000 7f5491800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f548c072bd0 con 0x7f548c0b3410 2024-04-20T21:22:22.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.936+0000 7f5491800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f548c0b3910 con 0x7f548c0ce370 2024-04-20T21:22:22.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.936+0000 7f5491800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f548c0b3bf0 con 0x7f548c162ed0 2024-04-20T21:22:22.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.936+0000 7f548aa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f548c162ed0 0x7f548c06b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:22.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.937+0000 7f5482000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f548c0b3410 0x7f548c071db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51188/0 (socket says 172.21.15.86:51188) 2024-04-20T21:22:22.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.937+0000 7f5482000640 1 -- 172.21.15.86:0/3183388869 learned_addr learned my addr 172.21.15.86:0/3183388869 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:22:22.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.937+0000 7f548aa00640 1 -- 172.21.15.86:0/3183388869 >> v1:172.21.15.144:6789/0 conn(0x7f548c0ce370 legacy=0x7f548c0724c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:22.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.937+0000 7f548aa00640 1 -- 172.21.15.86:0/3183388869 >> v1:172.21.15.86:6789/0 conn(0x7f548c0b3410 legacy=0x7f548c071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:22.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.937+0000 7f548aa00640 1 -- 172.21.15.86:0/3183388869 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f548c0b3ed0 con 0x7f548c162ed0 2024-04-20T21:22:22.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.938+0000 7f548aa00640 1 --2- 172.21.15.86:0/3183388869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f548c162ed0 0x7f548c06b980 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f547c009d90 tx=0x7f547c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ee4a2dbed6e08ef server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:22.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.938+0000 7f548a000640 1 -- 172.21.15.86:0/3183388869 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547c039070 con 0x7f548c162ed0 2024-04-20T21:22:22.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.938+0000 7f548a000640 1 -- 172.21.15.86:0/3183388869 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f547c02cc60 con 0x7f548c162ed0 2024-04-20T21:22:22.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.938+0000 7f548a000640 1 -- 172.21.15.86:0/3183388869 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547c034660 con 0x7f548c162ed0 2024-04-20T21:22:22.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.939+0000 7f5491800640 1 -- 172.21.15.86:0/3183388869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f548c162ed0 msgr2=0x7f548c06b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:22.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.939+0000 7f5491800640 1 --2- 172.21.15.86:0/3183388869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f548c162ed0 0x7f548c06b980 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f547c009d90 tx=0x7f547c02beb0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.939+0000 7f5491800640 1 -- 172.21.15.86:0/3183388869 shutdown_connections 2024-04-20T21:22:22.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.939+0000 7f5491800640 1 --2- 172.21.15.86:0/3183388869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f548c162ed0 0x7f548c06b980 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.939+0000 7f5491800640 1 -- 172.21.15.86:0/3183388869 >> 172.21.15.86:0/3183388869 conn(0x7f548c0b2110 msgr2=0x7f548c0b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:22.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.939+0000 7f5491800640 1 -- 172.21.15.86:0/3183388869 shutdown_connections 2024-04-20T21:22:22.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.939+0000 7f5491800640 1 -- 172.21.15.86:0/3183388869 wait complete. 2024-04-20T21:22:22.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.941+0000 7f5491800640 1 Processor -- start 2024-04-20T21:22:22.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.941+0000 7f5491800640 1 -- start start 2024-04-20T21:22:22.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.942+0000 7f5491800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f548c0b3410 0x7f548c148a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:22.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.942+0000 7f5491800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f548c0ce370 0x7f548c1610b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:22.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.942+0000 7f5491800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f548c162ed0 0x7f548c1615f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:22.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.942+0000 7f5491800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f548c06fb20 con 0x7f548c162ed0 2024-04-20T21:22:22.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.942+0000 7f5491800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f548c0b4490 con 0x7f548c0ce370 2024-04-20T21:22:22.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.942+0000 7f5491800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f548c06fca0 con 0x7f548c0b3410 2024-04-20T21:22:22.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.942+0000 7f548b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f548c162ed0 0x7f548c1615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:22.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.942+0000 7f548b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f548c162ed0 0x7f548c1615f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47904/0 (socket says 172.21.15.86:47904) 2024-04-20T21:22:22.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.942+0000 7f548b400640 1 -- 172.21.15.86:0/4221015602 learned_addr learned my addr 172.21.15.86:0/4221015602 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:22.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.943+0000 7f5482000640 1 --2- 172.21.15.86:0/4221015602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f548c0ce370 0x7f548c1610b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:22.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.943+0000 7f548b400640 1 -- 172.21.15.86:0/4221015602 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f548c0b3410 msgr2=0x7f548c148a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:22.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.943+0000 7f548b400640 1 --2- 172.21.15.86:0/4221015602 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f548c0b3410 0x7f548c148a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.943+0000 7f548b400640 1 -- 172.21.15.86:0/4221015602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f548c0ce370 msgr2=0x7f548c1610b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:22.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.943+0000 7f548b400640 1 --2- 172.21.15.86:0/4221015602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f548c0ce370 0x7f548c1610b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.943+0000 7f548b400640 1 -- 172.21.15.86:0/4221015602 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f548c160bd0 con 0x7f548c162ed0 2024-04-20T21:22:22.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.943+0000 7f5482000640 1 --2- 172.21.15.86:0/4221015602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f548c0ce370 0x7f548c1610b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:22:22.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.943+0000 7f548b400640 1 --2- 172.21.15.86:0/4221015602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f548c162ed0 0x7f548c1615f0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f548400c410 tx=0x7f548400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:22.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.943+0000 7f5483400640 1 -- 172.21.15.86:0/4221015602 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5484011790 con 0x7f548c162ed0 2024-04-20T21:22:22.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.943+0000 7f5483400640 1 -- 172.21.15.86:0/4221015602 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5484017070 con 0x7f548c162ed0 2024-04-20T21:22:22.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.943+0000 7f5483400640 1 -- 172.21.15.86:0/4221015602 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5484011bb0 con 0x7f548c162ed0 2024-04-20T21:22:22.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.943+0000 7f5491800640 1 -- 172.21.15.86:0/4221015602 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f548c160910 con 0x7f548c162ed0 2024-04-20T21:22:22.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.943+0000 7f5491800640 1 -- 172.21.15.86:0/4221015602 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f548c158090 con 0x7f548c162ed0 2024-04-20T21:22:22.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.946+0000 7f5483400640 1 -- 172.21.15.86:0/4221015602 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f548401c460 con 0x7f548c162ed0 2024-04-20T21:22:22.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.946+0000 7f5483400640 1 --2- 172.21.15.86:0/4221015602 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f546806c920 0x7f546806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:22.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.947+0000 7f5483400640 1 -- 172.21.15.86:0/4221015602 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f548408b780 con 0x7f548c162ed0 2024-04-20T21:22:22.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.947+0000 7f548aa00640 1 --2- 172.21.15.86:0/4221015602 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f546806c920 0x7f546806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:22.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.947+0000 7f54945c6740 1 Processor -- start 2024-04-20T21:22:22.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.947+0000 7f54945c6740 1 -- start start 2024-04-20T21:22:22.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.948+0000 7f548aa00640 1 --2- 172.21.15.86:0/4221015602 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f546806c920 0x7f546806ede0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f547c036000 tx=0x7f547c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:22.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.948+0000 7f54945c6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8bafaeb0 0x55fa8bacde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:22.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.948+0000 7f54945c6740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fa8bafb290 0x55fa8bafdc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:22.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.948+0000 7f54945c6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa8bace390 0x55fa8bafe180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:22.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.948+0000 7f54945c6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fa8ba39060 con 0x55fa8bafaeb0 2024-04-20T21:22:22.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.948+0000 7f54945c6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fa8b9def00 con 0x55fa8bace390 2024-04-20T21:22:22.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.948+0000 7f54945c6740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55fa8ba35020 con 0x55fa8bafb290 2024-04-20T21:22:22.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.949+0000 7f548aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8bafaeb0 0x55fa8bacde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:22.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.949+0000 7f548aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8bafaeb0 0x55fa8bacde50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47918/0 (socket says 172.21.15.86:47918) 2024-04-20T21:22:22.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.949+0000 7f548aa00640 1 -- 172.21.15.86:0/587329392 learned_addr learned my addr 172.21.15.86:0/587329392 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:22.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.949+0000 7f5482000640 1 --2- 172.21.15.86:0/587329392 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fa8bafb290 0x55fa8bafdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:22.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.949+0000 7f548aa00640 1 -- 172.21.15.86:0/587329392 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fa8bafb290 msgr2=0x55fa8bafdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:22.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.949+0000 7f548aa00640 1 --2- 172.21.15.86:0/587329392 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fa8bafb290 0x55fa8bafdc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.949+0000 7f548b400640 1 --2- 172.21.15.86:0/587329392 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa8bace390 0x55fa8bafe180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:22.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.949+0000 7f548aa00640 1 -- 172.21.15.86:0/587329392 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa8bace390 msgr2=0x55fa8bafe180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:22.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.949+0000 7f548aa00640 1 --2- 172.21.15.86:0/587329392 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa8bace390 0x55fa8bafe180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.949+0000 7f548aa00640 1 -- 172.21.15.86:0/587329392 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa8b9c7a30 con 0x55fa8bafaeb0 2024-04-20T21:22:22.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.950+0000 7f548aa00640 1 --2- 172.21.15.86:0/587329392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8bafaeb0 0x55fa8bacde50 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f547c009e80 tx=0x7f547c031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb3dac634f0dde94 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:22.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.950+0000 7f547b400640 1 -- 172.21.15.86:0/587329392 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547c04e070 con 0x55fa8bafaeb0 2024-04-20T21:22:22.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.950+0000 7f547b400640 1 -- 172.21.15.86:0/587329392 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f547c044070 con 0x55fa8bafaeb0 2024-04-20T21:22:22.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.950+0000 7f547b400640 1 -- 172.21.15.86:0/587329392 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547c039040 con 0x55fa8bafaeb0 2024-04-20T21:22:22.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.950+0000 7f54945c6740 1 -- 172.21.15.86:0/587329392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8bafaeb0 msgr2=0x55fa8bacde50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:22.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.951+0000 7f54945c6740 1 --2- 172.21.15.86:0/587329392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8bafaeb0 0x55fa8bacde50 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f547c009e80 tx=0x7f547c031000 comp rx=0 tx=0).stop 2024-04-20T21:22:22.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.951+0000 7f54945c6740 1 -- 172.21.15.86:0/587329392 shutdown_connections 2024-04-20T21:22:22.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.951+0000 7f54945c6740 1 --2- 172.21.15.86:0/587329392 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa8bace390 0x55fa8bafe180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.951+0000 7f54945c6740 1 --2- 172.21.15.86:0/587329392 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fa8bafb290 0x55fa8bafdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.951+0000 7f54945c6740 1 --2- 172.21.15.86:0/587329392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8bafaeb0 0x55fa8bacde50 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.951+0000 7f54945c6740 1 -- 172.21.15.86:0/587329392 >> 172.21.15.86:0/587329392 conn(0x55fa8bad00c0 msgr2=0x55fa8bb18c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:22.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.951+0000 7f54945c6740 1 -- 172.21.15.86:0/587329392 shutdown_connections 2024-04-20T21:22:22.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.951+0000 7f54945c6740 1 -- 172.21.15.86:0/587329392 wait complete. 2024-04-20T21:22:22.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.951+0000 7f54945c6740 1 -- start start 2024-04-20T21:22:22.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.952+0000 7f54945c6740 1 Processor -- start 2024-04-20T21:22:22.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.953+0000 7f54945c6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa8bafaeb0 0x55fa8bb01600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:22.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.953+0000 7f54945c6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8bafb290 0x55fa8baf3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:22.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.953+0000 7f54945c6740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fa8baf62c0 0x55fa8bb288a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:22.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.953+0000 7f54945c6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fa8bad2660 con 0x55fa8bafb290 2024-04-20T21:22:22.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.953+0000 7f54945c6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fa8b9f23d0 con 0x55fa8bafaeb0 2024-04-20T21:22:22.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.953+0000 7f54945c6740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55fa8bad1a50 con 0x55fa8baf62c0 2024-04-20T21:22:22.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.953+0000 7f54945c6740 10 client.0 _ll_register_callbacks cb 0x55fa8b9c0960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:22:22.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.953+0000 7f5482000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8bafb290 0x55fa8baf3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:22.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.953+0000 7f5482000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8bafb290 0x55fa8baf3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47930/0 (socket says 172.21.15.86:47930) 2024-04-20T21:22:22.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.953+0000 7f5482000640 1 -- 172.21.15.86:0/3551731337 learned_addr learned my addr 172.21.15.86:0/3551731337 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:22.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.953+0000 7f548b400640 1 --2- 172.21.15.86:0/3551731337 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fa8baf62c0 0x55fa8bb288a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:22.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.953+0000 7f548aa00640 1 --2- 172.21.15.86:0/3551731337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa8bafaeb0 0x55fa8bb01600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:22.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.953+0000 7f5482000640 1 -- 172.21.15.86:0/3551731337 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fa8baf62c0 msgr2=0x55fa8bb288a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:22.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.953+0000 7f5482000640 1 --2- 172.21.15.86:0/3551731337 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fa8baf62c0 0x55fa8bb288a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.953+0000 7f5482000640 1 -- 172.21.15.86:0/3551731337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa8bafaeb0 msgr2=0x55fa8bb01600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:22.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.953+0000 7f5482000640 1 --2- 172.21.15.86:0/3551731337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa8bafaeb0 0x55fa8bb01600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.953+0000 7f5482000640 1 -- 172.21.15.86:0/3551731337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa8b8177c0 con 0x55fa8bafb290 2024-04-20T21:22:22.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.954+0000 7f5482000640 1 --2- 172.21.15.86:0/3551731337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8bafb290 0x55fa8baf3dc0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f5474001cf0 tx=0x7f547400e590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbd084d29e417de6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:22.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.954+0000 7f546fe00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:22:22.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.954+0000 7f546fe00640 1 -- 172.21.15.86:0/3551731337 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5474019060 con 0x55fa8bafb290 2024-04-20T21:22:22.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.954+0000 7f546fe00640 1 -- 172.21.15.86:0/3551731337 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f5474014b30 con 0x55fa8bafb290 2024-04-20T21:22:22.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.954+0000 7f54945c6740 1 -- 172.21.15.86:0/3551731337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f547c044070 con 0x55fa8bafb290 2024-04-20T21:22:22.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.954+0000 7f546fe00640 1 -- 172.21.15.86:0/3551731337 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5474014d70 con 0x55fa8bafb290 2024-04-20T21:22:22.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.954+0000 7f54945c6740 10 client.6489 fetch_fsmap learned FSMap version 34 2024-04-20T21:22:22.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.954+0000 7f54945c6740 1 -- 172.21.15.86:0/3551731337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x55fa8b877f50 con 0x55fa8bafb290 2024-04-20T21:22:22.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.955+0000 7f546fe00640 1 -- 172.21.15.86:0/3551731337 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f547c044070 con 0x55fa8bafb290 2024-04-20T21:22:22.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.955+0000 7f546fe00640 1 -- 172.21.15.86:0/3551731337 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 34) ==== 3495+0+0 (secure 0 0 0) 0x7f5474005ad0 con 0x55fa8bafb290 2024-04-20T21:22:22.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.955+0000 7f54945c6740 10 client.6489 fetch_fsmap finished waiting for FSMap version 34 2024-04-20T21:22:22.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.955+0000 7f54945c6740 10 client.6489 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-20T21:22:22.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.955+0000 7f54945c6740 20 client.6489 populate_metadata read hostname 'smithi086' 2024-04-20T21:22:22.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.955+0000 7f54945c6740 1 --2- 172.21.15.86:0/3551731337 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x55fa8bae4940 0x55fa8bb37090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:22.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.955+0000 7f54945c6740 4 client.6489 mds_command: new command op to 6135 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:22:22.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.955+0000 7f54945c6740 1 -- 172.21.15.86:0/3551731337 --> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa8ba46a80 con 0x55fa8bae4940 2024-04-20T21:22:22.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.955+0000 7f548aa00640 1 --2- 172.21.15.86:0/3551731337 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x55fa8bae4940 0x55fa8bb37090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:22.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.956+0000 7f548aa00640 1 --2- 172.21.15.86:0/3551731337 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x55fa8bae4940 0x55fa8bb37090 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f06ea1661c97f12 server_cookie=f298f2a7e465e78b in_seq=0 out_seq=0 2024-04-20T21:22:22.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.956+0000 7f546fe00640 10 client.6489 ms_handle_connect on v2:172.21.15.86:6835/3008383261 2024-04-20T21:22:22.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.956+0000 7f546fe00640 1 -- 172.21.15.86:0/3551731337 <== mds.0 v2:172.21.15.86:6835/3008383261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fa8ba46a80 con 0x55fa8bae4940 2024-04-20T21:22:22.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.956+0000 7f546fe00640 10 client.6489 handle_command_reply: tid=0 2024-04-20T21:22:22.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.956+0000 7f54945c6740 1 client.6489 shutdown 2024-04-20T21:22:22.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.957+0000 7f54945c6740 1 -- 172.21.15.86:0/3551731337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8bafb290 msgr2=0x55fa8baf3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:22.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.957+0000 7f54945c6740 1 --2- 172.21.15.86:0/3551731337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8bafb290 0x55fa8baf3dc0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f5474001cf0 tx=0x7f547400e590 comp rx=0 tx=0).stop 2024-04-20T21:22:22.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.957+0000 7f54945c6740 1 -- 172.21.15.86:0/3551731337 shutdown_connections 2024-04-20T21:22:22.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.957+0000 7f54945c6740 1 --2- 172.21.15.86:0/3551731337 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x55fa8bae4940 0x55fa8bb37090 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.957+0000 7f54945c6740 1 --2- 172.21.15.86:0/3551731337 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fa8baf62c0 0x55fa8bb288a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.957+0000 7f54945c6740 1 --2- 172.21.15.86:0/3551731337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8bafb290 0x55fa8baf3dc0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.957+0000 7f54945c6740 1 --2- 172.21.15.86:0/3551731337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa8bafaeb0 0x55fa8bb01600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.957+0000 7f546fe00640 0 client.6489 ms_handle_reset on v2:172.21.15.86:6835/3008383261 2024-04-20T21:22:22.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.957+0000 7f54945c6740 1 -- 172.21.15.86:0/3551731337 >> 172.21.15.86:0/3551731337 conn(0x55fa8bace390 msgr2=0x55fa8bb1c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:22.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.957+0000 7f54945c6740 1 -- 172.21.15.86:0/3551731337 shutdown_connections 2024-04-20T21:22:22.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.957+0000 7f54945c6740 1 -- 172.21.15.86:0/3551731337 wait complete. 2024-04-20T21:22:22.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.957+0000 7f54945c6740 20 client.6489 trim_cache size 0 max 16384 2024-04-20T21:22:22.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.972+0000 7f54945c6740 1 Processor -- start 2024-04-20T21:22:22.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.972+0000 7f54945c6740 1 -- start start 2024-04-20T21:22:22.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.972+0000 7f54945c6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8baf62c0 0x55fa8baf2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:22.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.972+0000 7f54945c6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa8bafaeb0 0x55fa8bae16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:22.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.972+0000 7f54945c6740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fa8bafb290 0x55fa8bae1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:22.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.973+0000 7f54945c6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f548c14b9e0 con 0x55fa8baf62c0 2024-04-20T21:22:22.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.973+0000 7f54945c6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f548c15ad00 con 0x55fa8bafaeb0 2024-04-20T21:22:22.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.973+0000 7f54945c6740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f548c14b390 con 0x55fa8bafb290 2024-04-20T21:22:22.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.973+0000 7f548aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8baf62c0 0x55fa8baf2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:22.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.973+0000 7f548aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8baf62c0 0x55fa8baf2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47944/0 (socket says 172.21.15.86:47944) 2024-04-20T21:22:22.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.973+0000 7f548aa00640 1 -- 172.21.15.86:0/2002362288 learned_addr learned my addr 172.21.15.86:0/2002362288 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:22.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.973+0000 7f548aa00640 1 -- 172.21.15.86:0/2002362288 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fa8bafb290 msgr2=0x55fa8bae1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:22.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.973+0000 7f548aa00640 1 --2- 172.21.15.86:0/2002362288 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fa8bafb290 0x55fa8bae1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.973+0000 7f548aa00640 1 -- 172.21.15.86:0/2002362288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa8bafaeb0 msgr2=0x55fa8bae16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:22.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.973+0000 7f548aa00640 1 --2- 172.21.15.86:0/2002362288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa8bafaeb0 0x55fa8bae16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.973+0000 7f548aa00640 1 -- 172.21.15.86:0/2002362288 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa8b70b0b0 con 0x55fa8baf62c0 2024-04-20T21:22:22.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.973+0000 7f548aa00640 1 --2- 172.21.15.86:0/2002362288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8baf62c0 0x55fa8baf2670 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f547c009500 tx=0x7f547c041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2492c92e2f615a1c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:22.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.974+0000 7f547aa00640 1 -- 172.21.15.86:0/2002362288 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547c04e070 con 0x55fa8baf62c0 2024-04-20T21:22:22.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.974+0000 7f547aa00640 1 -- 172.21.15.86:0/2002362288 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f547c044040 con 0x55fa8baf62c0 2024-04-20T21:22:22.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.974+0000 7f547aa00640 1 -- 172.21.15.86:0/2002362288 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547c039040 con 0x55fa8baf62c0 2024-04-20T21:22:22.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.974+0000 7f54945c6740 1 -- 172.21.15.86:0/2002362288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8baf62c0 msgr2=0x55fa8baf2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:22.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.974+0000 7f54945c6740 1 --2- 172.21.15.86:0/2002362288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8baf62c0 0x55fa8baf2670 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f547c009500 tx=0x7f547c041000 comp rx=0 tx=0).stop 2024-04-20T21:22:22.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.974+0000 7f54945c6740 1 -- 172.21.15.86:0/2002362288 shutdown_connections 2024-04-20T21:22:22.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.974+0000 7f54945c6740 1 --2- 172.21.15.86:0/2002362288 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fa8bafb290 0x55fa8bae1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.974+0000 7f54945c6740 1 --2- 172.21.15.86:0/2002362288 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa8bafaeb0 0x55fa8bae16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.974+0000 7f54945c6740 1 --2- 172.21.15.86:0/2002362288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8baf62c0 0x55fa8baf2670 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.974+0000 7f54945c6740 1 -- 172.21.15.86:0/2002362288 >> 172.21.15.86:0/2002362288 conn(0x55fa8bace390 msgr2=0x55fa8bb01410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:22.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.974+0000 7f54945c6740 1 -- 172.21.15.86:0/2002362288 shutdown_connections 2024-04-20T21:22:22.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.974+0000 7f54945c6740 1 -- 172.21.15.86:0/2002362288 wait complete. 2024-04-20T21:22:22.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.975+0000 7f54945c6740 1 -- start start 2024-04-20T21:22:22.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.975+0000 7f54945c6740 1 Processor -- start 2024-04-20T21:22:22.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.976+0000 7f54945c6740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fa8baf62c0 0x55fa8bb3a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:22.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.976+0000 7f54945c6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8bafaeb0 0x55fa8bae16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:22.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.976+0000 7f54945c6740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa8bafb290 0x55fa8bae1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:22.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.976+0000 7f54945c6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fa8bafbeb0 con 0x55fa8bafaeb0 2024-04-20T21:22:22.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.976+0000 7f54945c6740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fa8baf69a0 con 0x55fa8bafb290 2024-04-20T21:22:22.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.976+0000 7f54945c6740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55fa8bafcdb0 con 0x55fa8baf62c0 2024-04-20T21:22:22.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.976+0000 7f54945c6740 10 client.0 _ll_register_callbacks cb 0x7f54740018d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:22:22.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.977+0000 7f5482000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8bafaeb0 0x55fa8bae16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:22.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.977+0000 7f5482000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8bafaeb0 0x55fa8bae16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47958/0 (socket says 172.21.15.86:47958) 2024-04-20T21:22:22.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.977+0000 7f5482000640 1 -- 172.21.15.86:0/422945630 learned_addr learned my addr 172.21.15.86:0/422945630 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:22.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.977+0000 7f548aa00640 1 --2- 172.21.15.86:0/422945630 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fa8baf62c0 0x55fa8bb3a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:22.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.977+0000 7f548b400640 1 --2- 172.21.15.86:0/422945630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa8bafb290 0x55fa8bae1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:22.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.977+0000 7f5482000640 1 -- 172.21.15.86:0/422945630 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fa8baf62c0 msgr2=0x55fa8bb3a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:22.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.977+0000 7f5482000640 1 --2- 172.21.15.86:0/422945630 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fa8baf62c0 0x55fa8bb3a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.977+0000 7f5482000640 1 -- 172.21.15.86:0/422945630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa8bafb290 msgr2=0x55fa8bae1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:22.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.977+0000 7f5482000640 1 --2- 172.21.15.86:0/422945630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa8bafb290 0x55fa8bae1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.977+0000 7f5482000640 1 -- 172.21.15.86:0/422945630 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa8bb015a0 con 0x55fa8bafaeb0 2024-04-20T21:22:22.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.977+0000 7f5482000640 1 --2- 172.21.15.86:0/422945630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8bafaeb0 0x55fa8bae16f0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f5474017040 tx=0x7f5474000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51b1ad870929986b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:22.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.977+0000 7f546fe00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:22:22.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.978+0000 7f546fe00640 1 -- 172.21.15.86:0/422945630 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5474019020 con 0x55fa8bafaeb0 2024-04-20T21:22:22.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.978+0000 7f546fe00640 1 -- 172.21.15.86:0/422945630 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f5474024040 con 0x55fa8bafaeb0 2024-04-20T21:22:22.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.978+0000 7f54945c6740 1 -- 172.21.15.86:0/422945630 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f547c044040 con 0x55fa8bafaeb0 2024-04-20T21:22:22.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.978+0000 7f546fe00640 1 -- 172.21.15.86:0/422945630 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f547400ea90 con 0x55fa8bafaeb0 2024-04-20T21:22:22.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.978+0000 7f54945c6740 10 client.6495 fetch_fsmap learned FSMap version 34 2024-04-20T21:22:22.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.978+0000 7f54945c6740 1 -- 172.21.15.86:0/422945630 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x55fa8bb926f0 con 0x55fa8bafaeb0 2024-04-20T21:22:22.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.978+0000 7f546fe00640 1 -- 172.21.15.86:0/422945630 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f547c044040 con 0x55fa8bafaeb0 2024-04-20T21:22:22.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.978+0000 7f546fe00640 1 -- 172.21.15.86:0/422945630 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 34) ==== 3495+0+0 (secure 0 0 0) 0x7f5474007280 con 0x55fa8bafaeb0 2024-04-20T21:22:22.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.978+0000 7f54945c6740 10 client.6495 fetch_fsmap finished waiting for FSMap version 34 2024-04-20T21:22:22.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.978+0000 7f54945c6740 10 client.6495 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-20T21:22:22.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.978+0000 7f54945c6740 20 client.6495 populate_metadata read hostname 'smithi086' 2024-04-20T21:22:22.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.979+0000 7f54945c6740 1 --2- 172.21.15.86:0/422945630 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x55fa8baf0d80 0x55fa8baf1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:22.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.979+0000 7f54945c6740 4 client.6495 mds_command: new command op to 6135 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-20T21:22:22.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.979+0000 7f54945c6740 1 -- 172.21.15.86:0/422945630 --> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55fa8ba408f0 con 0x55fa8baf0d80 2024-04-20T21:22:22.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.979+0000 7f548aa00640 1 --2- 172.21.15.86:0/422945630 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x55fa8baf0d80 0x55fa8baf1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:22.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.979+0000 7f548aa00640 1 --2- 172.21.15.86:0/422945630 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x55fa8baf0d80 0x55fa8baf1160 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bd3aef5fdba2bf5 server_cookie=a8079b5376b15a2f in_seq=0 out_seq=0 2024-04-20T21:22:22.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.979+0000 7f546fe00640 10 client.6495 ms_handle_connect on v2:172.21.15.86:6835/3008383261 2024-04-20T21:22:22.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.980+0000 7f546fe00640 1 -- 172.21.15.86:0/422945630 <== mds.0 v2:172.21.15.86:6835/3008383261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fa8ba408f0 con 0x55fa8baf0d80 2024-04-20T21:22:22.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.980+0000 7f546fe00640 10 client.6495 handle_command_reply: tid=0 2024-04-20T21:22:22.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.980+0000 7f54945c6740 1 client.6495 shutdown 2024-04-20T21:22:22.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.980+0000 7f54945c6740 1 -- 172.21.15.86:0/422945630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8bafaeb0 msgr2=0x55fa8bae16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:22.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.980+0000 7f54945c6740 1 --2- 172.21.15.86:0/422945630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8bafaeb0 0x55fa8bae16f0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f5474017040 tx=0x7f5474000bc0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.980+0000 7f54945c6740 1 -- 172.21.15.86:0/422945630 shutdown_connections 2024-04-20T21:22:22.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.980+0000 7f54945c6740 1 --2- 172.21.15.86:0/422945630 >> [v2:172.21.15.86:6835/3008383261,v1:172.21.15.86:6837/3008383261] conn(0x55fa8baf0d80 0x55fa8baf1160 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.980+0000 7f54945c6740 1 --2- 172.21.15.86:0/422945630 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fa8bafb290 0x55fa8bae1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.980+0000 7f54945c6740 1 --2- 172.21.15.86:0/422945630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa8bafaeb0 0x55fa8bae16f0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.980+0000 7f54945c6740 1 --2- 172.21.15.86:0/422945630 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fa8baf62c0 0x55fa8bb3a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.980+0000 7f546fe00640 0 client.6495 ms_handle_reset on v2:172.21.15.86:6835/3008383261 2024-04-20T21:22:22.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.980+0000 7f54945c6740 1 -- 172.21.15.86:0/422945630 >> 172.21.15.86:0/422945630 conn(0x55fa8bace390 msgr2=0x55fa8bad4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:22.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.980+0000 7f54945c6740 1 -- 172.21.15.86:0/422945630 shutdown_connections 2024-04-20T21:22:22.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.980+0000 7f54945c6740 1 -- 172.21.15.86:0/422945630 wait complete. 2024-04-20T21:22:22.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.980+0000 7f54945c6740 20 client.6495 trim_cache size 0 max 16384 2024-04-20T21:22:22.982 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:22:22.983 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-20T21:22:22.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.981+0000 7f5491800640 1 -- 172.21.15.86:0/4221015602 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f546806c920 msgr2=0x7f546806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:22.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.981+0000 7f5491800640 1 --2- 172.21.15.86:0/4221015602 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f546806c920 0x7f546806ede0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f547c036000 tx=0x7f547c02e000 comp rx=0 tx=0).stop 2024-04-20T21:22:22.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.981+0000 7f5491800640 1 -- 172.21.15.86:0/4221015602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f548c162ed0 msgr2=0x7f548c1615f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:22.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.981+0000 7f5491800640 1 --2- 172.21.15.86:0/4221015602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f548c162ed0 0x7f548c1615f0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f548400c410 tx=0x7f548400c8d0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.981+0000 7f5491800640 1 -- 172.21.15.86:0/4221015602 shutdown_connections 2024-04-20T21:22:22.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.981+0000 7f5491800640 1 --2- 172.21.15.86:0/4221015602 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f546806c920 0x7f546806ede0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.981+0000 7f5491800640 1 --2- 172.21.15.86:0/4221015602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f548c162ed0 0x7f548c1615f0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.981+0000 7f5491800640 1 --2- 172.21.15.86:0/4221015602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f548c0ce370 0x7f548c1610b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.981+0000 7f5491800640 1 --2- 172.21.15.86:0/4221015602 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f548c0b3410 0x7f548c148a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:22.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.981+0000 7f5491800640 1 -- 172.21.15.86:0/4221015602 >> 172.21.15.86:0/4221015602 conn(0x7f548c0b2110 msgr2=0x7f548c071db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:22.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.981+0000 7f5491800640 1 -- 172.21.15.86:0/4221015602 shutdown_connections 2024-04-20T21:22:22.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:22.981+0000 7f5491800640 1 -- 172.21.15.86:0/4221015602 wait complete. 2024-04-20T21:22:22.997 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-20T21:22:23.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.188+0000 7f6273600640 1 Processor -- start 2024-04-20T21:22:23.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.188+0000 7f6273600640 1 -- start start 2024-04-20T21:22:23.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.189+0000 7f6273600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f626c1620b0 0x7f626c06cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:23.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.189+0000 7f6273600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f626c0d29b0 0x7f626c0d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:23.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.189+0000 7f6273600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f626c0b3900 con 0x7f626c0d1110 2024-04-20T21:22:23.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.189+0000 7f6273600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f626c0d37c0 con 0x7f626c1620b0 2024-04-20T21:22:23.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.189+0000 7f6273600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f626c0d2f20 con 0x7f626c0d29b0 2024-04-20T21:22:23.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.190+0000 7f6270c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f626c1620b0 0x7f626c06cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:23.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.190+0000 7f6270c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f626c1620b0 0x7f626c06cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47974/0 (socket says 172.21.15.86:47974) 2024-04-20T21:22:23.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.190+0000 7f6270c00640 1 -- 172.21.15.86:0/2548076430 learned_addr learned my addr 172.21.15.86:0/2548076430 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:23.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.190+0000 7f626be00640 1 --2- 172.21.15.86:0/2548076430 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f626c0d29b0 0x7f626c0d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:23.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.190+0000 7f6270c00640 1 -- 172.21.15.86:0/2548076430 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f626c0d29b0 msgr2=0x7f626c0d22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:23.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.190+0000 7f6270c00640 1 --2- 172.21.15.86:0/2548076430 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f626c0d29b0 0x7f626c0d22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:23.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.190+0000 7f6270c00640 1 -- 172.21.15.86:0/2548076430 >> v1:172.21.15.144:6789/0 conn(0x7f626c0d1110 legacy=0x7f626c0cfb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:23.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.190+0000 7f6270c00640 1 -- 172.21.15.86:0/2548076430 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f626c0d3200 con 0x7f626c1620b0 2024-04-20T21:22:23.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.191+0000 7f6270c00640 1 --2- 172.21.15.86:0/2548076430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f626c1620b0 0x7f626c06cb40 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f625c009870 tx=0x7f625c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b02ee38eec0b6e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:23.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.192+0000 7f626b400640 1 -- 172.21.15.86:0/2548076430 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c035020 con 0x7f626c1620b0 2024-04-20T21:22:23.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.192+0000 7f626b400640 1 -- 172.21.15.86:0/2548076430 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f625c031640 con 0x7f626c1620b0 2024-04-20T21:22:23.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.192+0000 7f626b400640 1 -- 172.21.15.86:0/2548076430 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c02d040 con 0x7f626c1620b0 2024-04-20T21:22:23.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.192+0000 7f6273600640 1 -- 172.21.15.86:0/2548076430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f626c1620b0 msgr2=0x7f626c06cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:23.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.192+0000 7f6273600640 1 --2- 172.21.15.86:0/2548076430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f626c1620b0 0x7f626c06cb40 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f625c009870 tx=0x7f625c02b8f0 comp rx=0 tx=0).stop 2024-04-20T21:22:23.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.192+0000 7f6273600640 1 -- 172.21.15.86:0/2548076430 shutdown_connections 2024-04-20T21:22:23.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.192+0000 7f6273600640 1 --2- 172.21.15.86:0/2548076430 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f626c0d29b0 0x7f626c0d22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:23.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.192+0000 7f6273600640 1 --2- 172.21.15.86:0/2548076430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f626c1620b0 0x7f626c06cb40 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:23.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.192+0000 7f6273600640 1 -- 172.21.15.86:0/2548076430 >> 172.21.15.86:0/2548076430 conn(0x7f626c08bf70 msgr2=0x7f626c0d4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:23.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.192+0000 7f6273600640 1 -- 172.21.15.86:0/2548076430 shutdown_connections 2024-04-20T21:22:23.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.193+0000 7f6273600640 1 -- 172.21.15.86:0/2548076430 wait complete. 2024-04-20T21:22:23.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.194+0000 7f6273600640 1 Processor -- start 2024-04-20T21:22:23.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.194+0000 7f6273600640 1 -- start start 2024-04-20T21:22:23.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.195+0000 7f6273600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f626c0d1110 0x7f626c152750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:23.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.195+0000 7f6273600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f626c0d29b0 0x7f626c1560d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:23.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.195+0000 7f6273600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f626c1620b0 0x7f626c0cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:23.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.195+0000 7f6273600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f626c0cf610 con 0x7f626c0d1110 2024-04-20T21:22:23.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.195+0000 7f6273600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f626c0cf330 con 0x7f626c0d29b0 2024-04-20T21:22:23.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.195+0000 7f6273600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f626c0d15a0 con 0x7f626c1620b0 2024-04-20T21:22:23.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.195+0000 7f6270c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f626c0d1110 0x7f626c152750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:23.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.196+0000 7f6270c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f626c0d1110 0x7f626c152750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47978/0 (socket says 172.21.15.86:47978) 2024-04-20T21:22:23.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.196+0000 7f6270c00640 1 -- 172.21.15.86:0/559694617 learned_addr learned my addr 172.21.15.86:0/559694617 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:23.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.196+0000 7f6271600640 1 --2- 172.21.15.86:0/559694617 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f626c1620b0 0x7f626c0cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:23.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.196+0000 7f6270c00640 1 -- 172.21.15.86:0/559694617 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f626c1620b0 msgr2=0x7f626c0cb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:23.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.196+0000 7f626be00640 1 --2- 172.21.15.86:0/559694617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f626c0d29b0 0x7f626c1560d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:23.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.196+0000 7f6270c00640 1 --2- 172.21.15.86:0/559694617 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f626c1620b0 0x7f626c0cb7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:23.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.196+0000 7f6270c00640 1 -- 172.21.15.86:0/559694617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f626c0d29b0 msgr2=0x7f626c1560d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:23.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.196+0000 7f6270c00640 1 --2- 172.21.15.86:0/559694617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f626c0d29b0 0x7f626c1560d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:23.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.196+0000 7f6270c00640 1 -- 172.21.15.86:0/559694617 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f626c155030 con 0x7f626c0d1110 2024-04-20T21:22:23.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.196+0000 7f6270c00640 1 --2- 172.21.15.86:0/559694617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f626c0d1110 0x7f626c152750 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f625c000c00 tx=0x7f625c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:23.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.197+0000 7f6268c00640 1 -- 172.21.15.86:0/559694617 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c035040 con 0x7f626c0d1110 2024-04-20T21:22:23.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.197+0000 7f6268c00640 1 -- 172.21.15.86:0/559694617 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f625c039c10 con 0x7f626c0d1110 2024-04-20T21:22:23.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.197+0000 7f6268c00640 1 -- 172.21.15.86:0/559694617 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625c045480 con 0x7f626c0d1110 2024-04-20T21:22:23.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.197+0000 7f6273600640 1 -- 172.21.15.86:0/559694617 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f626c155310 con 0x7f626c0d1110 2024-04-20T21:22:23.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.197+0000 7f6273600640 1 -- 172.21.15.86:0/559694617 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f626c1558d0 con 0x7f626c0d1110 2024-04-20T21:22:23.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.198+0000 7f6273600640 1 -- 172.21.15.86:0/559694617 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f626c0690f0 con 0x7f626c0d1110 2024-04-20T21:22:23.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.199+0000 7f6268c00640 1 -- 172.21.15.86:0/559694617 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f625c038020 con 0x7f626c0d1110 2024-04-20T21:22:23.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.200+0000 7f6268c00640 1 --2- 172.21.15.86:0/559694617 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f624c06c8f0 0x7f624c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:23.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.200+0000 7f6268c00640 1 -- 172.21.15.86:0/559694617 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f625c0ad9e0 con 0x7f626c0d1110 2024-04-20T21:22:23.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.200+0000 7f626be00640 1 --2- 172.21.15.86:0/559694617 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f624c06c8f0 0x7f624c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:23.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.201+0000 7f626be00640 1 --2- 172.21.15.86:0/559694617 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f624c06c8f0 0x7f624c06edb0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f626c0ca690 tx=0x7f6258008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:23.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.205+0000 7f6268c00640 1 -- 172.21.15.86:0/559694617 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f626c0b3900 con 0x7f626c0d1110 2024-04-20T21:22:23.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.443+0000 7f6273600640 1 -- 172.21.15.86:0/559694617 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f626c0d3b20 con 0x7f626c0d1110 2024-04-20T21:22:23.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.444+0000 7f6268c00640 1 -- 172.21.15.86:0/559694617 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f625c07bd70 con 0x7f626c0d1110 2024-04-20T21:22:23.443 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:22:23.443 INFO:teuthology.orchestra.run.smithi086.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-20T21:22:23.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.446+0000 7f6273600640 1 -- 172.21.15.86:0/559694617 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f624c06c8f0 msgr2=0x7f624c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:23.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.446+0000 7f6273600640 1 --2- 172.21.15.86:0/559694617 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f624c06c8f0 0x7f624c06edb0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f626c0ca690 tx=0x7f6258008040 comp rx=0 tx=0).stop 2024-04-20T21:22:23.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.446+0000 7f6273600640 1 -- 172.21.15.86:0/559694617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f626c0d1110 msgr2=0x7f626c152750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:23.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.446+0000 7f6273600640 1 --2- 172.21.15.86:0/559694617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f626c0d1110 0x7f626c152750 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f625c000c00 tx=0x7f625c0046c0 comp rx=0 tx=0).stop 2024-04-20T21:22:23.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.446+0000 7f6273600640 1 -- 172.21.15.86:0/559694617 shutdown_connections 2024-04-20T21:22:23.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.446+0000 7f6273600640 1 --2- 172.21.15.86:0/559694617 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f624c06c8f0 0x7f624c06edb0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:23.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.447+0000 7f6273600640 1 --2- 172.21.15.86:0/559694617 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f626c1620b0 0x7f626c0cb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:23.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.447+0000 7f6273600640 1 --2- 172.21.15.86:0/559694617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f626c0d29b0 0x7f626c1560d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:23.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.447+0000 7f6273600640 1 --2- 172.21.15.86:0/559694617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f626c0d1110 0x7f626c152750 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:23.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.447+0000 7f6273600640 1 -- 172.21.15.86:0/559694617 >> 172.21.15.86:0/559694617 conn(0x7f626c08bf70 msgr2=0x7f626c0cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:23.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.447+0000 7f6273600640 1 -- 172.21.15.86:0/559694617 shutdown_connections 2024-04-20T21:22:23.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.447+0000 7f6273600640 1 -- 172.21.15.86:0/559694617 wait complete. 2024-04-20T21:22:23.464 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-20T21:22:23.464 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-20T21:22:23.465 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T21:22:23.571 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T21:22:23.676 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-20T21:22:23.676 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T21:22:23.677 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:22:23.677 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:22:23.737 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:22:23.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.943+0000 7f219c800640 1 Processor -- start 2024-04-20T21:22:23.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.943+0000 7f219c800640 1 -- start start 2024-04-20T21:22:23.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.944+0000 7f219c800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2194144e90 0x7f21940b4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:23.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.944+0000 7f219c800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21940cdb50 con 0x7f21940cd170 2024-04-20T21:22:23.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.944+0000 7f219c800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21940b3e50 con 0x7f219406f450 2024-04-20T21:22:23.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.944+0000 7f219c800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2194145270 con 0x7f2194144e90 2024-04-20T21:22:23.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.945+0000 7f2199400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f21940cd170 0x7f219406ed40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51202/0 (socket says 172.21.15.86:51202) 2024-04-20T21:22:23.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.945+0000 7f2199400640 1 -- 172.21.15.86:0/2433529682 learned_addr learned my addr 172.21.15.86:0/2433529682 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:22:23.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.945+0000 7f2198a00640 1 -- 172.21.15.86:0/2433529682 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2419264195 0 0) 0x7f21940cdb50 con 0x7f21940cd170 2024-04-20T21:22:23.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.945+0000 7f2198a00640 1 -- 172.21.15.86:0/2433529682 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f217c003520 con 0x7f21940cd170 2024-04-20T21:22:23.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.945+0000 7f2199e00640 1 --2- 172.21.15.86:0/2433529682 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2194144e90 0x7f21940b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:23.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.946+0000 7f2198a00640 1 -- 172.21.15.86:0/2433529682 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 491533738 0 0) 0x7f217c003520 con 0x7f21940cd170 2024-04-20T21:22:23.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.946+0000 7f2198a00640 1 -- 172.21.15.86:0/2433529682 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2194144e90 msgr2=0x7f21940b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:23.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.946+0000 7f2198a00640 1 --2- 172.21.15.86:0/2433529682 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2194144e90 0x7f21940b4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:23.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.946+0000 7f2198a00640 1 -- 172.21.15.86:0/2433529682 >> v1:172.21.15.144:6789/0 conn(0x7f219406f450 legacy=0x7f219415bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:23.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.946+0000 7f2198a00640 1 -- 172.21.15.86:0/2433529682 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21940b4130 con 0x7f21940cd170 2024-04-20T21:22:23.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.946+0000 7f2198a00640 1 -- 172.21.15.86:0/2433529682 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f2184001670 con 0x7f21940cd170 2024-04-20T21:22:23.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.946+0000 7f2198a00640 1 -- 172.21.15.86:0/2433529682 >> v1:172.21.15.86:6789/0 conn(0x7f21940cd170 legacy=0x7f219406ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:23.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.946+0000 7f2198a00640 1 --2- 172.21.15.86:0/2433529682 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f217c0039e0 0x7f217c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:23.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.946+0000 7f2198a00640 1 --2- 172.21.15.86:0/2433529682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f217c004800 0x7f217c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:23.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.946+0000 7f2198a00640 1 --2- 172.21.15.86:0/2433529682 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f217c0051c0 0x7f217c00dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:23.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.946+0000 7f2198a00640 1 -- 172.21.15.86:0/2433529682 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2194145270 con 0x7f217c004800 2024-04-20T21:22:23.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.946+0000 7f2198a00640 1 -- 172.21.15.86:0/2433529682 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f217c00e2d0 con 0x7f217c0051c0 2024-04-20T21:22:23.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.946+0000 7f2198a00640 1 -- 172.21.15.86:0/2433529682 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f217c00e4d0 con 0x7f217c0039e0 2024-04-20T21:22:23.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.946+0000 7f2199400640 1 --2- 172.21.15.86:0/2433529682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f217c004800 0x7f217c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:23.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.947+0000 7f2199400640 1 -- 172.21.15.86:0/2433529682 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f217c0039e0 msgr2=0x7f217c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:23.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.947+0000 7f2199e00640 1 --2- 172.21.15.86:0/2433529682 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f217c0039e0 0x7f217c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:23.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.947+0000 7f219a800640 1 --2- 172.21.15.86:0/2433529682 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f217c0051c0 0x7f217c00dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:23.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.947+0000 7f2199400640 1 --2- 172.21.15.86:0/2433529682 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f217c0039e0 0x7f217c003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:23.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.947+0000 7f2199400640 1 -- 172.21.15.86:0/2433529682 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f217c0051c0 msgr2=0x7f217c00dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:23.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.947+0000 7f2199400640 1 --2- 172.21.15.86:0/2433529682 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f217c0051c0 0x7f217c00dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:23.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.947+0000 7f2199400640 1 -- 172.21.15.86:0/2433529682 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f217c003520 con 0x7f217c004800 2024-04-20T21:22:23.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.947+0000 7f219a800640 1 --2- 172.21.15.86:0/2433529682 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f217c0051c0 0x7f217c00dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:22:23.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.948+0000 7f2199400640 1 --2- 172.21.15.86:0/2433529682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f217c004800 0x7f217c004c80 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f218400c0e0 tx=0x7f217c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=450637fea274b877 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:23.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.948+0000 7f2198a00640 1 -- 172.21.15.86:0/2433529682 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2184007c20 con 0x7f217c004800 2024-04-20T21:22:23.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.948+0000 7f2198a00640 1 -- 172.21.15.86:0/2433529682 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2184030070 con 0x7f217c004800 2024-04-20T21:22:23.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.948+0000 7f2198a00640 1 -- 172.21.15.86:0/2433529682 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2184005d70 con 0x7f217c004800 2024-04-20T21:22:23.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.949+0000 7f219c800640 1 -- 172.21.15.86:0/2433529682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f217c004800 msgr2=0x7f217c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:23.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.949+0000 7f219c800640 1 --2- 172.21.15.86:0/2433529682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f217c004800 0x7f217c004c80 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f218400c0e0 tx=0x7f217c005830 comp rx=0 tx=0).stop 2024-04-20T21:22:23.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.949+0000 7f219a800640 1 -- 172.21.15.86:0/2433529682 reap_dead start 2024-04-20T21:22:23.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.949+0000 7f219c800640 1 -- 172.21.15.86:0/2433529682 shutdown_connections 2024-04-20T21:22:23.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.949+0000 7f219c800640 1 -- 172.21.15.86:0/2433529682 >> 172.21.15.86:0/2433529682 conn(0x7f21940b2360 msgr2=0x7f21940b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:23.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.949+0000 7f219c800640 1 -- 172.21.15.86:0/2433529682 shutdown_connections 2024-04-20T21:22:23.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.949+0000 7f219c800640 1 -- 172.21.15.86:0/2433529682 wait complete. 2024-04-20T21:22:23.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.951+0000 7f219c800640 1 Processor -- start 2024-04-20T21:22:23.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.951+0000 7f219c800640 1 -- start start 2024-04-20T21:22:23.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.952+0000 7f219c800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f217c004800 0x7f219413e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:23.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.952+0000 7f219c800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2194140b70 0x7f2194149c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:23.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.952+0000 7f219c800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2194140fd0 0x7f219414a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:23.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.952+0000 7f219c800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f21940d0190 con 0x7f2194140fd0 2024-04-20T21:22:23.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.952+0000 7f219c800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f21940d0010 con 0x7f2194140b70 2024-04-20T21:22:23.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.952+0000 7f219c800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f21940cd5d0 con 0x7f217c004800 2024-04-20T21:22:23.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.952+0000 7f219a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2194140fd0 0x7f219414a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:23.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.952+0000 7f219a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2194140fd0 0x7f219414a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47988/0 (socket says 172.21.15.86:47988) 2024-04-20T21:22:23.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.952+0000 7f219a800640 1 -- 172.21.15.86:0/4091288820 learned_addr learned my addr 172.21.15.86:0/4091288820 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:23.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.952+0000 7f2199400640 1 --2- 172.21.15.86:0/4091288820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2194140b70 0x7f2194149c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:23.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.952+0000 7f2199e00640 1 --2- 172.21.15.86:0/4091288820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f217c004800 0x7f219413e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:23.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.952+0000 7f219a800640 1 -- 172.21.15.86:0/4091288820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f217c004800 msgr2=0x7f219413e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:23.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.952+0000 7f219a800640 1 --2- 172.21.15.86:0/4091288820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f217c004800 0x7f219413e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:23.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.952+0000 7f219a800640 1 -- 172.21.15.86:0/4091288820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2194140b70 msgr2=0x7f2194149c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:23.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.952+0000 7f219a800640 1 --2- 172.21.15.86:0/4091288820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2194140b70 0x7f2194149c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:23.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.952+0000 7f219a800640 1 -- 172.21.15.86:0/4091288820 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f219413ec30 con 0x7f2194140fd0 2024-04-20T21:22:23.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.952+0000 7f2199400640 1 --2- 172.21.15.86:0/4091288820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2194140b70 0x7f2194149c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:22:23.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.953+0000 7f219a800640 1 --2- 172.21.15.86:0/4091288820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2194140fd0 0x7f219414a140 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f21940cdfc0 tx=0x7f219000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:23.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.953+0000 7f218e000640 1 -- 172.21.15.86:0/4091288820 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2190006d50 con 0x7f2194140fd0 2024-04-20T21:22:23.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.953+0000 7f218e000640 1 -- 172.21.15.86:0/4091288820 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2190003970 con 0x7f2194140fd0 2024-04-20T21:22:23.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.953+0000 7f219c800640 1 -- 172.21.15.86:0/4091288820 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f219413ef10 con 0x7f2194140fd0 2024-04-20T21:22:23.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.953+0000 7f218e000640 1 -- 172.21.15.86:0/4091288820 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2190003da0 con 0x7f2194140fd0 2024-04-20T21:22:23.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.953+0000 7f219c800640 1 -- 172.21.15.86:0/4091288820 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f219413f4d0 con 0x7f2194140fd0 2024-04-20T21:22:23.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.954+0000 7f219c800640 1 -- 172.21.15.86:0/4091288820 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21940689e0 con 0x7f2194140fd0 2024-04-20T21:22:23.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.955+0000 7f218e000640 1 -- 172.21.15.86:0/4091288820 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f219001b020 con 0x7f2194140fd0 2024-04-20T21:22:23.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.957+0000 7f218e000640 1 --2- 172.21.15.86:0/4091288820 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f217806c920 0x7f217806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:23.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.957+0000 7f218e000640 1 -- 172.21.15.86:0/4091288820 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f219008b860 con 0x7f2194140fd0 2024-04-20T21:22:23.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.957+0000 7f2199e00640 1 --2- 172.21.15.86:0/4091288820 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f217806c920 0x7f217806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:23.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.958+0000 7f2199e00640 1 --2- 172.21.15.86:0/4091288820 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f217806c920 0x7f217806ede0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2188006fd0 tx=0x7f2188008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:23.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:23.962+0000 7f218e000640 1 -- 172.21.15.86:0/4091288820 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2190059bf0 con 0x7f2194140fd0 2024-04-20T21:22:24.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.162+0000 7f219c800640 1 -- 172.21.15.86:0/4091288820 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f21940727b0 con 0x7f2194140fd0 2024-04-20T21:22:24.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.164+0000 7f218e000640 1 -- 172.21.15.86:0/4091288820 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+16665 (secure 0 0 0) 0x7f21900594f0 con 0x7f2194140fd0 2024-04-20T21:22:24.163 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:22:24.163 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":59,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:21:09.072894+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:21:05.248922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-20T21:21:06.479064+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:22:24.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.166+0000 7f219c800640 1 -- 172.21.15.86:0/4091288820 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f217806c920 msgr2=0x7f217806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:24.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.166+0000 7f219c800640 1 --2- 172.21.15.86:0/4091288820 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f217806c920 0x7f217806ede0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2188006fd0 tx=0x7f2188008040 comp rx=0 tx=0).stop 2024-04-20T21:22:24.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.166+0000 7f219c800640 1 -- 172.21.15.86:0/4091288820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2194140fd0 msgr2=0x7f219414a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:24.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.166+0000 7f219c800640 1 --2- 172.21.15.86:0/4091288820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2194140fd0 0x7f219414a140 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f21940cdfc0 tx=0x7f219000dc00 comp rx=0 tx=0).stop 2024-04-20T21:22:24.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.167+0000 7f219c800640 1 -- 172.21.15.86:0/4091288820 shutdown_connections 2024-04-20T21:22:24.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.167+0000 7f219c800640 1 --2- 172.21.15.86:0/4091288820 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f217806c920 0x7f217806ede0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:24.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.167+0000 7f219c800640 1 --2- 172.21.15.86:0/4091288820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2194140fd0 0x7f219414a140 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:24.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.167+0000 7f219c800640 1 --2- 172.21.15.86:0/4091288820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2194140b70 0x7f2194149c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:24.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.167+0000 7f219c800640 1 --2- 172.21.15.86:0/4091288820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f217c004800 0x7f219413e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:24.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.167+0000 7f219c800640 1 -- 172.21.15.86:0/4091288820 >> 172.21.15.86:0/4091288820 conn(0x7f21940b2360 msgr2=0x7f2194071d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:24.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.167+0000 7f219c800640 1 -- 172.21.15.86:0/4091288820 shutdown_connections 2024-04-20T21:22:24.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.167+0000 7f219c800640 1 -- 172.21.15.86:0/4091288820 wait complete. 2024-04-20T21:22:24.184 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:21:05.248922+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:21:06.479064+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:22:24.185 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:22:24.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.388+0000 7f68f3400640 1 Processor -- start 2024-04-20T21:22:24.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.389+0000 7f68f3400640 1 -- start start 2024-04-20T21:22:24.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.390+0000 7f68f3400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f68ec09a9e0 0x7f68ec0b3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:24.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.390+0000 7f68f3400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68ec06ca90 con 0x7f68ec06d830 2024-04-20T21:22:24.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.390+0000 7f68f3400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68ec0d1ab0 con 0x7f68ec09adc0 2024-04-20T21:22:24.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.390+0000 7f68f3400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f68ec06cd50 con 0x7f68ec09a9e0 2024-04-20T21:22:24.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.390+0000 7f68f0a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f68ec09a9e0 0x7f68ec0b3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:24.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.390+0000 7f68f1400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f68ec06d830 0x7f68ec160500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51216/0 (socket says 172.21.15.86:51216) 2024-04-20T21:22:24.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.390+0000 7f68f0a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f68ec09a9e0 0x7f68ec0b3980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47990/0 (socket says 172.21.15.86:47990) 2024-04-20T21:22:24.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.391+0000 7f68f0a00640 1 -- 172.21.15.86:0/4239466515 learned_addr learned my addr 172.21.15.86:0/4239466515 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:24.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.391+0000 7f68f0a00640 1 -- 172.21.15.86:0/4239466515 >> v1:172.21.15.144:6789/0 conn(0x7f68ec09adc0 legacy=0x7f68ec0d1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:24.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.391+0000 7f68f0a00640 1 -- 172.21.15.86:0/4239466515 >> v1:172.21.15.86:6789/0 conn(0x7f68ec06d830 legacy=0x7f68ec160500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:24.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.391+0000 7f68f0a00640 1 -- 172.21.15.86:0/4239466515 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68ec06d030 con 0x7f68ec09a9e0 2024-04-20T21:22:24.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.391+0000 7f68eb400640 1 -- 172.21.15.86:0/4239466515 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1413717059 0 0) 0x7f68ec06ca90 con 0x7f68ec06d830 2024-04-20T21:22:24.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.392+0000 7f68f0a00640 1 --2- 172.21.15.86:0/4239466515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68ec09a9e0 0x7f68ec0b3980 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f68dc009870 tx=0x7f68dc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22d2b890655bb0a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:24.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.392+0000 7f68eb400640 1 -- 172.21.15.86:0/4239466515 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68dc034070 con 0x7f68ec09a9e0 2024-04-20T21:22:24.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.392+0000 7f68eb400640 1 -- 172.21.15.86:0/4239466515 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68dc030860 con 0x7f68ec09a9e0 2024-04-20T21:22:24.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.392+0000 7f68eb400640 1 -- 172.21.15.86:0/4239466515 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68dc030c30 con 0x7f68ec09a9e0 2024-04-20T21:22:24.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.393+0000 7f68f3400640 1 -- 172.21.15.86:0/4239466515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68ec09a9e0 msgr2=0x7f68ec0b3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:24.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.393+0000 7f68f3400640 1 --2- 172.21.15.86:0/4239466515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68ec09a9e0 0x7f68ec0b3980 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f68dc009870 tx=0x7f68dc02b9a0 comp rx=0 tx=0).stop 2024-04-20T21:22:24.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.393+0000 7f68f3400640 1 -- 172.21.15.86:0/4239466515 shutdown_connections 2024-04-20T21:22:24.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.393+0000 7f68f3400640 1 --2- 172.21.15.86:0/4239466515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68ec09a9e0 0x7f68ec0b3980 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:24.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.393+0000 7f68f3400640 1 -- 172.21.15.86:0/4239466515 >> 172.21.15.86:0/4239466515 conn(0x7f68ec0b4070 msgr2=0x7f68ec0b4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:24.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.393+0000 7f68f3400640 1 -- 172.21.15.86:0/4239466515 shutdown_connections 2024-04-20T21:22:24.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.393+0000 7f68f3400640 1 -- 172.21.15.86:0/4239466515 wait complete. 2024-04-20T21:22:24.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.395+0000 7f68f3400640 1 Processor -- start 2024-04-20T21:22:24.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.395+0000 7f68f3400640 1 -- start start 2024-04-20T21:22:24.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.396+0000 7f68f3400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f68ec06d830 0x7f68ec1515c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:24.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.396+0000 7f68f3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68ec09a9e0 0x7f68ec0b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:24.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.396+0000 7f68f3400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68ec09adc0 0x7f68ec1e9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:24.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.396+0000 7f68f3400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f68ec09b390 con 0x7f68ec09a9e0 2024-04-20T21:22:24.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.396+0000 7f68f3400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f68ec09b210 con 0x7f68ec09adc0 2024-04-20T21:22:24.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.396+0000 7f68f3400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f68ec15ffd0 con 0x7f68ec06d830 2024-04-20T21:22:24.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.396+0000 7f68ebe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68ec09a9e0 0x7f68ec0b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:24.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.396+0000 7f68ebe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68ec09a9e0 0x7f68ec0b4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48004/0 (socket says 172.21.15.86:48004) 2024-04-20T21:22:24.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.396+0000 7f68ebe00640 1 -- 172.21.15.86:0/76898738 learned_addr learned my addr 172.21.15.86:0/76898738 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:24.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.397+0000 7f68ebe00640 1 -- 172.21.15.86:0/76898738 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f68ec06d830 msgr2=0x7f68ec1515c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:24.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.397+0000 7f68ebe00640 1 --2- 172.21.15.86:0/76898738 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f68ec06d830 0x7f68ec1515c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:24.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.397+0000 7f68ebe00640 1 -- 172.21.15.86:0/76898738 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68ec09adc0 msgr2=0x7f68ec1e9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:24.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.397+0000 7f68ebe00640 1 --2- 172.21.15.86:0/76898738 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68ec09adc0 0x7f68ec1e9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:24.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.397+0000 7f68ebe00640 1 -- 172.21.15.86:0/76898738 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68ec0c7550 con 0x7f68ec09a9e0 2024-04-20T21:22:24.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.397+0000 7f68ebe00640 1 --2- 172.21.15.86:0/76898738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68ec09a9e0 0x7f68ec0b4ff0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f68d400c7c0 tx=0x7f68d400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:24.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.397+0000 7f68e8c00640 1 -- 172.21.15.86:0/76898738 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d4016020 con 0x7f68ec09a9e0 2024-04-20T21:22:24.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.397+0000 7f68e8c00640 1 -- 172.21.15.86:0/76898738 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68d400a9f0 con 0x7f68ec09a9e0 2024-04-20T21:22:24.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.397+0000 7f68e8c00640 1 -- 172.21.15.86:0/76898738 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d400ae20 con 0x7f68ec09a9e0 2024-04-20T21:22:24.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.397+0000 7f68f3400640 1 -- 172.21.15.86:0/76898738 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68ec0c7830 con 0x7f68ec09a9e0 2024-04-20T21:22:24.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.398+0000 7f68f3400640 1 -- 172.21.15.86:0/76898738 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68ec0c80d0 con 0x7f68ec09a9e0 2024-04-20T21:22:24.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.399+0000 7f68f3400640 1 -- 172.21.15.86:0/76898738 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68ec0c82d0 con 0x7f68ec09a9e0 2024-04-20T21:22:24.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.400+0000 7f68e8c00640 1 -- 172.21.15.86:0/76898738 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f68d4019020 con 0x7f68ec09a9e0 2024-04-20T21:22:24.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.401+0000 7f68e8c00640 1 --2- 172.21.15.86:0/76898738 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f68c806c920 0x7f68c806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:24.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.401+0000 7f68e8c00640 1 -- 172.21.15.86:0/76898738 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f68d408b7f0 con 0x7f68ec09a9e0 2024-04-20T21:22:24.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.401+0000 7f68f0a00640 1 --2- 172.21.15.86:0/76898738 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f68c806c920 0x7f68c806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:24.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.402+0000 7f68f0a00640 1 --2- 172.21.15.86:0/76898738 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f68c806c920 0x7f68c806ede0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f68dc036000 tx=0x7f68dc030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:24.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.405+0000 7f68e8c00640 1 -- 172.21.15.86:0/76898738 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f68d4020070 con 0x7f68ec09a9e0 2024-04-20T21:22:24.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.606+0000 7f68f3400640 1 -- 172.21.15.86:0/76898738 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f68ec0689e0 con 0x7f68ec09a9e0 2024-04-20T21:22:24.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.607+0000 7f68e8c00640 1 -- 172.21.15.86:0/76898738 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) ==== 93+0+10 (secure 0 0 0) 0x7f68d4059c20 con 0x7f68ec09a9e0 2024-04-20T21:22:24.606 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:22:24.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.610+0000 7f68f3400640 1 -- 172.21.15.86:0/76898738 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f68c806c920 msgr2=0x7f68c806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:24.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.610+0000 7f68f3400640 1 --2- 172.21.15.86:0/76898738 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f68c806c920 0x7f68c806ede0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f68dc036000 tx=0x7f68dc030000 comp rx=0 tx=0).stop 2024-04-20T21:22:24.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.610+0000 7f68f3400640 1 -- 172.21.15.86:0/76898738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68ec09a9e0 msgr2=0x7f68ec0b4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:24.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.610+0000 7f68f3400640 1 --2- 172.21.15.86:0/76898738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68ec09a9e0 0x7f68ec0b4ff0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f68d400c7c0 tx=0x7f68d400cc80 comp rx=0 tx=0).stop 2024-04-20T21:22:24.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.610+0000 7f68f3400640 1 -- 172.21.15.86:0/76898738 shutdown_connections 2024-04-20T21:22:24.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.610+0000 7f68f3400640 1 --2- 172.21.15.86:0/76898738 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f68c806c920 0x7f68c806ede0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:24.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.610+0000 7f68f3400640 1 --2- 172.21.15.86:0/76898738 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68ec09adc0 0x7f68ec1e9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:24.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.610+0000 7f68f3400640 1 --2- 172.21.15.86:0/76898738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68ec09a9e0 0x7f68ec0b4ff0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:24.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.610+0000 7f68f3400640 1 --2- 172.21.15.86:0/76898738 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f68ec06d830 0x7f68ec1515c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:24.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.610+0000 7f68f3400640 1 -- 172.21.15.86:0/76898738 >> 172.21.15.86:0/76898738 conn(0x7f68ec0b4070 msgr2=0x7f68ec160900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:24.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.611+0000 7f68f3400640 1 -- 172.21.15.86:0/76898738 shutdown_connections 2024-04-20T21:22:24.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.611+0000 7f68f3400640 1 -- 172.21.15.86:0/76898738 wait complete. 2024-04-20T21:22:24.629 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:22:24.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.833+0000 7f6040a00640 1 Processor -- start 2024-04-20T21:22:24.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.833+0000 7f6040a00640 1 -- start start 2024-04-20T21:22:24.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.835+0000 7f6040a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f603c0d45a0 0x7f603c0d3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:24.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.835+0000 7f6040a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f603c15ea70 0x7f603c0d4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:24.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.835+0000 7f6040a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f603c06d570 0x7f603c0d1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:24.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.835+0000 7f6040a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f603c0cf120 con 0x7f603c06d570 2024-04-20T21:22:24.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.835+0000 7f6040a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f603c06de70 con 0x7f603c0d45a0 2024-04-20T21:22:24.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.835+0000 7f6040a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f603c06e150 con 0x7f603c15ea70 2024-04-20T21:22:24.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.835+0000 7f603be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f603c06d570 0x7f603c0d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:24.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.835+0000 7f603be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f603c06d570 0x7f603c0d1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48006/0 (socket says 172.21.15.86:48006) 2024-04-20T21:22:24.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.835+0000 7f603be00640 1 -- 172.21.15.86:0/2946997104 learned_addr learned my addr 172.21.15.86:0/2946997104 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:24.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.835+0000 7f603be00640 1 -- 172.21.15.86:0/2946997104 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f603c15ea70 msgr2=0x7f603c0d4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:24.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.835+0000 7f6033e00640 1 --2- 172.21.15.86:0/2946997104 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f603c15ea70 0x7f603c0d4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:24.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.835+0000 7f603b400640 1 --2- 172.21.15.86:0/2946997104 >> v2:172.21.15.144:3300/0 conn(0x7f603c0d45a0 0x7f603c0d3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:24.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.835+0000 7f603be00640 1 --2- 172.21.15.86:0/2946997104 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f603c15ea70 0x7f603c0d4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:24.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.836+0000 7f603be00640 1 -- 172.21.15.86:0/2946997104 >> v2:172.21.15.144:3300/0 conn(0x7f603c0d45a0 msgr2=0x7f603c0d3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:24.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.836+0000 7f603be00640 1 --2- 172.21.15.86:0/2946997104 >> v2:172.21.15.144:3300/0 conn(0x7f603c0d45a0 0x7f603c0d3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:24.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.836+0000 7f603be00640 1 -- 172.21.15.86:0/2946997104 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f603c06e430 con 0x7f603c06d570 2024-04-20T21:22:24.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.836+0000 7f603b400640 1 --2- 172.21.15.86:0/2946997104 >> v2:172.21.15.144:3300/0 conn(0x7f603c0d45a0 0x7f603c0d3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:22:24.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.837+0000 7f603be00640 1 --2- 172.21.15.86:0/2946997104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f603c06d570 0x7f603c0d1100 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f603c08e910 tx=0x7f603402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e9bebf562d198fc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:24.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.837+0000 7f603aa00640 1 -- 172.21.15.86:0/2946997104 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603402c9f0 con 0x7f603c06d570 2024-04-20T21:22:24.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.837+0000 7f603aa00640 1 -- 172.21.15.86:0/2946997104 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f603402cb90 con 0x7f603c06d570 2024-04-20T21:22:24.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.837+0000 7f603aa00640 1 -- 172.21.15.86:0/2946997104 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60340343f0 con 0x7f603c06d570 2024-04-20T21:22:24.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.838+0000 7f6040a00640 1 -- 172.21.15.86:0/2946997104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f603c06d570 msgr2=0x7f603c0d1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:24.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.838+0000 7f6040a00640 1 --2- 172.21.15.86:0/2946997104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f603c06d570 0x7f603c0d1100 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f603c08e910 tx=0x7f603402be50 comp rx=0 tx=0).stop 2024-04-20T21:22:24.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.838+0000 7f6040a00640 1 -- 172.21.15.86:0/2946997104 shutdown_connections 2024-04-20T21:22:24.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.838+0000 7f6040a00640 1 --2- 172.21.15.86:0/2946997104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f603c06d570 0x7f603c0d1100 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:24.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.838+0000 7f6040a00640 1 --2- 172.21.15.86:0/2946997104 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f603c15ea70 0x7f603c0d4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:24.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.838+0000 7f6040a00640 1 --2- 172.21.15.86:0/2946997104 >> v2:172.21.15.144:3300/0 conn(0x7f603c0d45a0 0x7f603c0d3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:24.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.838+0000 7f6040a00640 1 -- 172.21.15.86:0/2946997104 >> 172.21.15.86:0/2946997104 conn(0x7f603c09d9d0 msgr2=0x7f603c08bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:24.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.838+0000 7f6040a00640 1 -- 172.21.15.86:0/2946997104 shutdown_connections 2024-04-20T21:22:24.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.838+0000 7f6040a00640 1 -- 172.21.15.86:0/2946997104 wait complete. 2024-04-20T21:22:24.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.840+0000 7f6040a00640 1 Processor -- start 2024-04-20T21:22:24.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.840+0000 7f6040a00640 1 -- start start 2024-04-20T21:22:24.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.841+0000 7f6040a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f603c06d570 0x7f603c15a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:24.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.841+0000 7f6040a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f603c0d45a0 0x7f603c0ce180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:24.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.841+0000 7f6040a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f603c15ea70 0x7f603c15cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:24.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.841+0000 7f6040a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f603c06da00 con 0x7f603c06d570 2024-04-20T21:22:24.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.841+0000 7f6040a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f603c0d4ec0 con 0x7f603c0d45a0 2024-04-20T21:22:24.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.841+0000 7f6040a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f603c06db80 con 0x7f603c15ea70 2024-04-20T21:22:24.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.841+0000 7f603b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f603c06d570 0x7f603c15a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:24.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.841+0000 7f603b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f603c06d570 0x7f603c15a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48018/0 (socket says 172.21.15.86:48018) 2024-04-20T21:22:24.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.841+0000 7f603b400640 1 -- 172.21.15.86:0/505662282 learned_addr learned my addr 172.21.15.86:0/505662282 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:24.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.841+0000 7f603b400640 1 -- 172.21.15.86:0/505662282 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f603c15ea70 msgr2=0x7f603c15cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:24.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.841+0000 7f603b400640 1 --2- 172.21.15.86:0/505662282 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f603c15ea70 0x7f603c15cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:24.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.841+0000 7f603b400640 1 -- 172.21.15.86:0/505662282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f603c0d45a0 msgr2=0x7f603c0ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:24.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.841+0000 7f603b400640 1 --2- 172.21.15.86:0/505662282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f603c0d45a0 0x7f603c0ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:24.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.841+0000 7f603b400640 1 -- 172.21.15.86:0/505662282 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f603c1f3cc0 con 0x7f603c06d570 2024-04-20T21:22:24.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.842+0000 7f603b400640 1 --2- 172.21.15.86:0/505662282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f603c06d570 0x7f603c15a7a0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f602c00c7f0 tx=0x7f602c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:24.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.842+0000 7f6033400640 1 -- 172.21.15.86:0/505662282 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c015070 con 0x7f603c06d570 2024-04-20T21:22:24.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.842+0000 7f6033400640 1 -- 172.21.15.86:0/505662282 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f602c007a30 con 0x7f603c06d570 2024-04-20T21:22:24.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.842+0000 7f6033400640 1 -- 172.21.15.86:0/505662282 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c007e00 con 0x7f603c06d570 2024-04-20T21:22:24.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.842+0000 7f6040a00640 1 -- 172.21.15.86:0/505662282 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f603c0c8640 con 0x7f603c06d570 2024-04-20T21:22:24.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.842+0000 7f6040a00640 1 -- 172.21.15.86:0/505662282 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f603c0c8e80 con 0x7f603c06d570 2024-04-20T21:22:24.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.843+0000 7f6040a00640 1 -- 172.21.15.86:0/505662282 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f603c04fd90 con 0x7f603c06d570 2024-04-20T21:22:24.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.844+0000 7f6033400640 1 -- 172.21.15.86:0/505662282 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f602c018070 con 0x7f603c06d570 2024-04-20T21:22:24.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.845+0000 7f6033400640 1 --2- 172.21.15.86:0/505662282 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f601806c8f0 0x7f601806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:24.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.845+0000 7f6033400640 1 -- 172.21.15.86:0/505662282 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f602c08bb20 con 0x7f603c06d570 2024-04-20T21:22:24.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.845+0000 7f6033e00640 1 --2- 172.21.15.86:0/505662282 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f601806c8f0 0x7f601806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:24.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.846+0000 7f6033e00640 1 --2- 172.21.15.86:0/505662282 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f601806c8f0 0x7f601806edb0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f603c15de10 tx=0x7f6024008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:24.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:24.850+0000 7f6033400640 1 -- 172.21.15.86:0/505662282 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f603c0c8640 con 0x7f603c06d570 2024-04-20T21:22:25.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.051+0000 7f6040a00640 1 -- 172.21.15.86:0/505662282 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f603c15ad10 con 0x7f603c06d570 2024-04-20T21:22:25.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.052+0000 7f6033400640 1 -- 172.21.15.86:0/505662282 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v59) ==== 104+0+11 (secure 0 0 0) 0x7f602c059f40 con 0x7f603c06d570 2024-04-20T21:22:25.051 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-20T21:22:25.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.054+0000 7f6040a00640 1 -- 172.21.15.86:0/505662282 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f601806c8f0 msgr2=0x7f601806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:25.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.054+0000 7f6040a00640 1 --2- 172.21.15.86:0/505662282 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f601806c8f0 0x7f601806edb0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f603c15de10 tx=0x7f6024008040 comp rx=0 tx=0).stop 2024-04-20T21:22:25.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.055+0000 7f6040a00640 1 -- 172.21.15.86:0/505662282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f603c06d570 msgr2=0x7f603c15a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:25.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.055+0000 7f6040a00640 1 --2- 172.21.15.86:0/505662282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f603c06d570 0x7f603c15a7a0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f602c00c7f0 tx=0x7f602c00cbf0 comp rx=0 tx=0).stop 2024-04-20T21:22:25.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.055+0000 7f6040a00640 1 -- 172.21.15.86:0/505662282 shutdown_connections 2024-04-20T21:22:25.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.055+0000 7f6040a00640 1 --2- 172.21.15.86:0/505662282 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f601806c8f0 0x7f601806edb0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:25.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.055+0000 7f6040a00640 1 --2- 172.21.15.86:0/505662282 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f603c15ea70 0x7f603c15cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:25.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.055+0000 7f6040a00640 1 --2- 172.21.15.86:0/505662282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f603c0d45a0 0x7f603c0ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:25.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.055+0000 7f6040a00640 1 --2- 172.21.15.86:0/505662282 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f603c06d570 0x7f603c15a7a0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:25.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.055+0000 7f6040a00640 1 -- 172.21.15.86:0/505662282 >> 172.21.15.86:0/505662282 conn(0x7f603c09d9d0 msgr2=0x7f603c08bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:25.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.055+0000 7f6040a00640 1 -- 172.21.15.86:0/505662282 shutdown_connections 2024-04-20T21:22:25.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.055+0000 7f6040a00640 1 -- 172.21.15.86:0/505662282 wait complete. 2024-04-20T21:22:25.072 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:22:25.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.269+0000 7f414b200640 1 Processor -- start 2024-04-20T21:22:25.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.270+0000 7f414b200640 1 -- start start 2024-04-20T21:22:25.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.271+0000 7f414b200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f414415e850 0x7f41440b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:25.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.271+0000 7f414b200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f41440b7d30 0x7f41440d1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:25.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.271+0000 7f414b200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41440d3870 con 0x7f41440d1c70 2024-04-20T21:22:25.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.271+0000 7f414b200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f41440d3d40 con 0x7f414415e850 2024-04-20T21:22:25.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.271+0000 7f414b200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f41440d4020 con 0x7f41440b7d30 2024-04-20T21:22:25.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.271+0000 7f4143e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f414415e850 0x7f41440b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:25.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.271+0000 7f4143e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f414415e850 0x7f41440b77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48034/0 (socket says 172.21.15.86:48034) 2024-04-20T21:22:25.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.271+0000 7f4143e00640 1 -- 172.21.15.86:0/1988956005 learned_addr learned my addr 172.21.15.86:0/1988956005 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:25.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.272+0000 7f4143400640 1 --2- 172.21.15.86:0/1988956005 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f41440b7d30 0x7f41440d1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:25.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.272+0000 7f4143e00640 1 -- 172.21.15.86:0/1988956005 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f41440b7d30 msgr2=0x7f41440d1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:25.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.272+0000 7f4143e00640 1 --2- 172.21.15.86:0/1988956005 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f41440b7d30 0x7f41440d1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:25.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.272+0000 7f4143e00640 1 -- 172.21.15.86:0/1988956005 >> v1:172.21.15.144:6789/0 conn(0x7f41440d1c70 legacy=0x7f41440d4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:25.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.272+0000 7f4143e00640 1 -- 172.21.15.86:0/1988956005 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41440d4300 con 0x7f414415e850 2024-04-20T21:22:25.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.273+0000 7f4143e00640 1 --2- 172.21.15.86:0/1988956005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f414415e850 0x7f41440b77f0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f4134009870 tx=0x7f413402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a2fba18de08ef84 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:25.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.273+0000 7f4142a00640 1 -- 172.21.15.86:0/1988956005 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4134035020 con 0x7f414415e850 2024-04-20T21:22:25.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.273+0000 7f4142a00640 1 -- 172.21.15.86:0/1988956005 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4134031640 con 0x7f414415e850 2024-04-20T21:22:25.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.274+0000 7f4142a00640 1 -- 172.21.15.86:0/1988956005 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413402d040 con 0x7f414415e850 2024-04-20T21:22:25.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.274+0000 7f414b200640 1 -- 172.21.15.86:0/1988956005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f414415e850 msgr2=0x7f41440b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:25.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.274+0000 7f414b200640 1 --2- 172.21.15.86:0/1988956005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f414415e850 0x7f41440b77f0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f4134009870 tx=0x7f413402b8f0 comp rx=0 tx=0).stop 2024-04-20T21:22:25.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.274+0000 7f414b200640 1 -- 172.21.15.86:0/1988956005 shutdown_connections 2024-04-20T21:22:25.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.274+0000 7f414b200640 1 --2- 172.21.15.86:0/1988956005 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f41440b7d30 0x7f41440d1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:25.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.274+0000 7f414b200640 1 --2- 172.21.15.86:0/1988956005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f414415e850 0x7f41440b77f0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:25.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.274+0000 7f414b200640 1 -- 172.21.15.86:0/1988956005 >> 172.21.15.86:0/1988956005 conn(0x7f4144067ae0 msgr2=0x7f4144067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:25.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.274+0000 7f414b200640 1 -- 172.21.15.86:0/1988956005 shutdown_connections 2024-04-20T21:22:25.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.274+0000 7f414b200640 1 -- 172.21.15.86:0/1988956005 wait complete. 2024-04-20T21:22:25.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.276+0000 7f414b200640 1 Processor -- start 2024-04-20T21:22:25.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.276+0000 7f414b200640 1 -- start start 2024-04-20T21:22:25.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.277+0000 7f414b200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41440b7d30 0x7f41441f3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:25.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.277+0000 7f414b200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f41440d1c70 0x7f41441e7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:25.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.277+0000 7f414b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f414415e850 0x7f41441e75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:25.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.277+0000 7f414b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f41440d5210 con 0x7f414415e850 2024-04-20T21:22:25.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.277+0000 7f414b200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f414415ec30 con 0x7f41440b7d30 2024-04-20T21:22:25.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.277+0000 7f414b200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f41440d5390 con 0x7f41440d1c70 2024-04-20T21:22:25.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.277+0000 7f4149200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f414415e850 0x7f41441e75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:25.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.277+0000 7f4149200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f414415e850 0x7f41441e75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48044/0 (socket says 172.21.15.86:48044) 2024-04-20T21:22:25.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.277+0000 7f4149200640 1 -- 172.21.15.86:0/3555569972 learned_addr learned my addr 172.21.15.86:0/3555569972 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:25.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.277+0000 7f4143400640 1 --2- 172.21.15.86:0/3555569972 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f41440d1c70 0x7f41441e7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:25.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.277+0000 7f4149200640 1 -- 172.21.15.86:0/3555569972 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f41440d1c70 msgr2=0x7f41441e7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:25.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.277+0000 7f4143e00640 1 --2- 172.21.15.86:0/3555569972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41440b7d30 0x7f41441f3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:25.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.278+0000 7f4149200640 1 --2- 172.21.15.86:0/3555569972 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f41440d1c70 0x7f41441e7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:25.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.278+0000 7f4149200640 1 -- 172.21.15.86:0/3555569972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41440b7d30 msgr2=0x7f41441f3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:25.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.278+0000 7f4149200640 1 --2- 172.21.15.86:0/3555569972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41440b7d30 0x7f41441f3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:25.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.278+0000 7f4149200640 1 -- 172.21.15.86:0/3555569972 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41440ce190 con 0x7f414415e850 2024-04-20T21:22:25.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.278+0000 7f4143e00640 1 --2- 172.21.15.86:0/3555569972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41440b7d30 0x7f41441f3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:22:25.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.278+0000 7f4149200640 1 --2- 172.21.15.86:0/3555569972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f414415e850 0x7f41441e75a0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f413c00c410 tx=0x7f413c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:25.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.278+0000 7f413be00640 1 -- 172.21.15.86:0/3555569972 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413c011790 con 0x7f414415e850 2024-04-20T21:22:25.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.278+0000 7f413be00640 1 -- 172.21.15.86:0/3555569972 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f413c017070 con 0x7f414415e850 2024-04-20T21:22:25.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.278+0000 7f413be00640 1 -- 172.21.15.86:0/3555569972 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413c011bb0 con 0x7f414415e850 2024-04-20T21:22:25.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.278+0000 7f414b200640 1 -- 172.21.15.86:0/3555569972 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41441464d0 con 0x7f414415e850 2024-04-20T21:22:25.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.278+0000 7f414b200640 1 -- 172.21.15.86:0/3555569972 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4144146a90 con 0x7f414415e850 2024-04-20T21:22:25.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.279+0000 7f414b200640 1 -- 172.21.15.86:0/3555569972 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4144146c90 con 0x7f414415e850 2024-04-20T21:22:25.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.280+0000 7f413be00640 1 -- 172.21.15.86:0/3555569972 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f413c01c460 con 0x7f414415e850 2024-04-20T21:22:25.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.281+0000 7f413be00640 1 --2- 172.21.15.86:0/3555569972 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f412006c8f0 0x7f412006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:25.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.281+0000 7f413be00640 1 -- 172.21.15.86:0/3555569972 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f413c08b7d0 con 0x7f414415e850 2024-04-20T21:22:25.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.282+0000 7f4143e00640 1 --2- 172.21.15.86:0/3555569972 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f412006c8f0 0x7f412006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:25.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.282+0000 7f4143e00640 1 --2- 172.21.15.86:0/3555569972 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f412006c8f0 0x7f412006edb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4134036000 tx=0x7f4134008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:25.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.286+0000 7f413be00640 1 -- 172.21.15.86:0/3555569972 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f413c021020 con 0x7f414415e850 2024-04-20T21:22:25.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.468+0000 7f414b200640 1 -- 172.21.15.86:0/3555569972 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f41440690f0 con 0x7f414415e850 2024-04-20T21:22:25.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.469+0000 7f413be00640 1 -- 172.21.15.86:0/3555569972 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v59) ==== 100+0+11 (secure 0 0 0) 0x7f413c059b60 con 0x7f414415e850 2024-04-20T21:22:25.468 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-20T21:22:25.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.471+0000 7f414b200640 1 -- 172.21.15.86:0/3555569972 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f412006c8f0 msgr2=0x7f412006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:25.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.471+0000 7f414b200640 1 --2- 172.21.15.86:0/3555569972 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f412006c8f0 0x7f412006edb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4134036000 tx=0x7f4134008030 comp rx=0 tx=0).stop 2024-04-20T21:22:25.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.471+0000 7f414b200640 1 -- 172.21.15.86:0/3555569972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f414415e850 msgr2=0x7f41441e75a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:25.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.471+0000 7f414b200640 1 --2- 172.21.15.86:0/3555569972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f414415e850 0x7f41441e75a0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f413c00c410 tx=0x7f413c00c8d0 comp rx=0 tx=0).stop 2024-04-20T21:22:25.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.472+0000 7f414b200640 1 -- 172.21.15.86:0/3555569972 shutdown_connections 2024-04-20T21:22:25.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.472+0000 7f414b200640 1 --2- 172.21.15.86:0/3555569972 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f412006c8f0 0x7f412006edb0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:25.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.472+0000 7f414b200640 1 --2- 172.21.15.86:0/3555569972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f414415e850 0x7f41441e75a0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:25.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.472+0000 7f414b200640 1 --2- 172.21.15.86:0/3555569972 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f41440d1c70 0x7f41441e7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:25.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.472+0000 7f414b200640 1 --2- 172.21.15.86:0/3555569972 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41440b7d30 0x7f41441f3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:25.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.472+0000 7f414b200640 1 -- 172.21.15.86:0/3555569972 >> 172.21.15.86:0/3555569972 conn(0x7f4144067ae0 msgr2=0x7f41440cac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:25.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.472+0000 7f414b200640 1 -- 172.21.15.86:0/3555569972 shutdown_connections 2024-04-20T21:22:25.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.472+0000 7f414b200640 1 -- 172.21.15.86:0/3555569972 wait complete. 2024-04-20T21:22:25.488 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T21:22:25.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.674+0000 7f06f8c00640 1 Processor -- start 2024-04-20T21:22:25.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.675+0000 7f06f8c00640 1 -- start start 2024-04-20T21:22:25.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.676+0000 7f06f8c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f06f406f070 0x7f06f40b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:25.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.676+0000 7f06f8c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06f415b9d0 con 0x7f06f40cff80 2024-04-20T21:22:25.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.676+0000 7f06f8c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06f40af850 con 0x7f06f415b5f0 2024-04-20T21:22:25.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.676+0000 7f06f8c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f06f40b4650 con 0x7f06f406f070 2024-04-20T21:22:25.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.676+0000 7f06e9400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f06f406f070 0x7f06f40b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:25.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.676+0000 7f06e9400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f06f406f070 0x7f06f40b3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48048/0 (socket says 172.21.15.86:48048) 2024-04-20T21:22:25.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.676+0000 7f06f2800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f06f40cff80 0x7f06f40ce130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51230/0 (socket says 172.21.15.86:51230) 2024-04-20T21:22:25.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.676+0000 7f06e9400640 1 -- 172.21.15.86:0/2719248523 learned_addr learned my addr 172.21.15.86:0/2719248523 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:25.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.677+0000 7f06e9400640 1 -- 172.21.15.86:0/2719248523 >> v1:172.21.15.144:6789/0 conn(0x7f06f415b5f0 legacy=0x7f06f40b50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:25.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.677+0000 7f06e9400640 1 -- 172.21.15.86:0/2719248523 >> v1:172.21.15.86:6789/0 conn(0x7f06f40cff80 legacy=0x7f06f40ce130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:25.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.677+0000 7f06e9400640 1 -- 172.21.15.86:0/2719248523 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06f40afb30 con 0x7f06f406f070 2024-04-20T21:22:25.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.677+0000 7f06f1400640 1 -- 172.21.15.86:0/2719248523 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 518197555 0 0) 0x7f06f415b9d0 con 0x7f06f40cff80 2024-04-20T21:22:25.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.678+0000 7f06e9400640 1 --2- 172.21.15.86:0/2719248523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06f406f070 0x7f06f40b3d40 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f06dc009870 tx=0x7f06dc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b358762f9c0f3d62 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:25.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.678+0000 7f06f1400640 1 -- 172.21.15.86:0/2719248523 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06dc034070 con 0x7f06f406f070 2024-04-20T21:22:25.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.678+0000 7f06f1400640 1 -- 172.21.15.86:0/2719248523 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06dc030860 con 0x7f06f406f070 2024-04-20T21:22:25.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.678+0000 7f06f1400640 1 -- 172.21.15.86:0/2719248523 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06dc030c30 con 0x7f06f406f070 2024-04-20T21:22:25.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.679+0000 7f06f8c00640 1 -- 172.21.15.86:0/2719248523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06f406f070 msgr2=0x7f06f40b3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:25.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.679+0000 7f06f8c00640 1 --2- 172.21.15.86:0/2719248523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06f406f070 0x7f06f40b3d40 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f06dc009870 tx=0x7f06dc02b9a0 comp rx=0 tx=0).stop 2024-04-20T21:22:25.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.679+0000 7f06f8c00640 1 -- 172.21.15.86:0/2719248523 shutdown_connections 2024-04-20T21:22:25.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.679+0000 7f06f8c00640 1 --2- 172.21.15.86:0/2719248523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06f406f070 0x7f06f40b3d40 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:25.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.679+0000 7f06f8c00640 1 -- 172.21.15.86:0/2719248523 >> 172.21.15.86:0/2719248523 conn(0x7f06f406b980 msgr2=0x7f06f406bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:25.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.679+0000 7f06f8c00640 1 -- 172.21.15.86:0/2719248523 shutdown_connections 2024-04-20T21:22:25.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.679+0000 7f06f8c00640 1 -- 172.21.15.86:0/2719248523 wait complete. 2024-04-20T21:22:25.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.680+0000 7f06f8c00640 1 Processor -- start 2024-04-20T21:22:25.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.681+0000 7f06f8c00640 1 -- start start 2024-04-20T21:22:25.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.682+0000 7f06f8c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06f406f070 0x7f06f4158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:25.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.682+0000 7f06f8c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06f40cff80 0x7f06f414daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:25.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.682+0000 7f06f8c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06f415b5f0 0x7f06f414e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:25.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.682+0000 7f06f8c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f06f40d03d0 con 0x7f06f40cff80 2024-04-20T21:22:25.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.682+0000 7f06f8c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f06f40b4b80 con 0x7f06f415b5f0 2024-04-20T21:22:25.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.682+0000 7f06f8c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f06f40d0550 con 0x7f06f406f070 2024-04-20T21:22:25.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.682+0000 7f06e9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06f40cff80 0x7f06f414daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:25.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.682+0000 7f06e9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06f40cff80 0x7f06f414daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48054/0 (socket says 172.21.15.86:48054) 2024-04-20T21:22:25.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.682+0000 7f06e9400640 1 -- 172.21.15.86:0/703004401 learned_addr learned my addr 172.21.15.86:0/703004401 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:25.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.682+0000 7f06e9400640 1 -- 172.21.15.86:0/703004401 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06f406f070 msgr2=0x7f06f4158100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:25.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.682+0000 7f06e9400640 1 --2- 172.21.15.86:0/703004401 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06f406f070 0x7f06f4158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:25.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.682+0000 7f06e9400640 1 -- 172.21.15.86:0/703004401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06f415b5f0 msgr2=0x7f06f414e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:25.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.682+0000 7f06e9400640 1 --2- 172.21.15.86:0/703004401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06f415b5f0 0x7f06f414e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:25.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.682+0000 7f06e9400640 1 -- 172.21.15.86:0/703004401 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06f4152b20 con 0x7f06f40cff80 2024-04-20T21:22:25.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.683+0000 7f06e9400640 1 --2- 172.21.15.86:0/703004401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06f40cff80 0x7f06f414daa0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f06dc000c00 tx=0x7f06dc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:25.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.683+0000 7f06eaa00640 1 -- 172.21.15.86:0/703004401 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06dc048070 con 0x7f06f40cff80 2024-04-20T21:22:25.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.683+0000 7f06eaa00640 1 -- 172.21.15.86:0/703004401 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06dc030660 con 0x7f06f40cff80 2024-04-20T21:22:25.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.683+0000 7f06eaa00640 1 -- 172.21.15.86:0/703004401 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06dc034040 con 0x7f06f40cff80 2024-04-20T21:22:25.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.683+0000 7f06f8c00640 1 -- 172.21.15.86:0/703004401 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06f4152e00 con 0x7f06f40cff80 2024-04-20T21:22:25.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.683+0000 7f06f8c00640 1 -- 172.21.15.86:0/703004401 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06f41533c0 con 0x7f06f40cff80 2024-04-20T21:22:25.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.684+0000 7f06f8c00640 1 -- 172.21.15.86:0/703004401 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06f40689e0 con 0x7f06f40cff80 2024-04-20T21:22:25.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.685+0000 7f06eaa00640 1 -- 172.21.15.86:0/703004401 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f06dc037070 con 0x7f06f40cff80 2024-04-20T21:22:25.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.686+0000 7f06eaa00640 1 --2- 172.21.15.86:0/703004401 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f06d006c8f0 0x7f06d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:25.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.686+0000 7f06eaa00640 1 -- 172.21.15.86:0/703004401 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f06dc0adb10 con 0x7f06f40cff80 2024-04-20T21:22:25.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.686+0000 7f06f1e00640 1 --2- 172.21.15.86:0/703004401 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f06d006c8f0 0x7f06d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:25.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.687+0000 7f06f1e00640 1 --2- 172.21.15.86:0/703004401 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f06d006c8f0 0x7f06d006edb0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f06e4006fd0 tx=0x7f06e4006bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:25.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.691+0000 7f06eaa00640 1 -- 172.21.15.86:0/703004401 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f06dc07c060 con 0x7f06f40cff80 2024-04-20T21:22:25.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.905+0000 7f06f8c00640 1 -- 172.21.15.86:0/703004401 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f06f4158640 con 0x7f06f40cff80 2024-04-20T21:22:25.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.906+0000 7f06eaa00640 1 -- 172.21.15.86:0/703004401 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v59) ==== 80+0+0 (secure 0 0 0) 0x7f06dc07b960 con 0x7f06f40cff80 2024-04-20T21:22:25.905 INFO:teuthology.orchestra.run.smithi086.stderr:listed 0 entries 2024-04-20T21:22:25.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.910+0000 7f06f8c00640 1 -- 172.21.15.86:0/703004401 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f06d006c8f0 msgr2=0x7f06d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:25.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.910+0000 7f06f8c00640 1 --2- 172.21.15.86:0/703004401 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f06d006c8f0 0x7f06d006edb0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f06e4006fd0 tx=0x7f06e4006bc0 comp rx=0 tx=0).stop 2024-04-20T21:22:25.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.910+0000 7f06f8c00640 1 -- 172.21.15.86:0/703004401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06f40cff80 msgr2=0x7f06f414daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:25.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.910+0000 7f06f8c00640 1 --2- 172.21.15.86:0/703004401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06f40cff80 0x7f06f414daa0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f06dc000c00 tx=0x7f06dc004730 comp rx=0 tx=0).stop 2024-04-20T21:22:25.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.910+0000 7f06f8c00640 1 -- 172.21.15.86:0/703004401 shutdown_connections 2024-04-20T21:22:25.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.910+0000 7f06f8c00640 1 --2- 172.21.15.86:0/703004401 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f06d006c8f0 0x7f06d006edb0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:25.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.910+0000 7f06f8c00640 1 --2- 172.21.15.86:0/703004401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06f415b5f0 0x7f06f414e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:25.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.910+0000 7f06f8c00640 1 --2- 172.21.15.86:0/703004401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06f40cff80 0x7f06f414daa0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:25.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.910+0000 7f06f8c00640 1 --2- 172.21.15.86:0/703004401 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06f406f070 0x7f06f4158100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:25.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.910+0000 7f06f8c00640 1 -- 172.21.15.86:0/703004401 >> 172.21.15.86:0/703004401 conn(0x7f06f406b980 msgr2=0x7f06f415dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:25.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.910+0000 7f06f8c00640 1 -- 172.21.15.86:0/703004401 shutdown_connections 2024-04-20T21:22:25.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:25.910+0000 7f06f8c00640 1 -- 172.21.15.86:0/703004401 wait complete. 2024-04-20T21:22:25.927 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi086.front.sepia.ceph.com... 2024-04-20T21:22:25.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:22:25.927 DEBUG:teuthology.orchestra.run.smithi086:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-20T21:22:26.002 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[43494]: fuse finished with error 0 and tester_r 0 2024-04-20T21:22:26.008 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-20T21:22:26.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:22:26.008 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-20T21:22:26.086 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T21:22:26.086 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:22:26.086 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:22:26.125 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T21:22:26.125 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2024-04-20T21:22:26.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:22:26.125 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-20T21:22:26.209 INFO:teuthology.orchestra.run.smithi144.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-20T21:22:26.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:22:26.210 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T21:22:26.210 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:22:26.210 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:22:26.256 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052731,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248996k,nr_inodes=812249,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T21:22:26.256 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-20T21:22:26.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:22:26.256 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-20T21:22:26.348 INFO:teuthology.orchestra.run.smithi086.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-20T21:22:26.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:22:26.349 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T21:22:26.349 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:22:26.349 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:22:26.404 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T21:22:26.405 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2024-04-20T21:22:26.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:22:26.405 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-20T21:22:26.457 INFO:teuthology.orchestra.run.smithi144.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-20T21:22:26.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:22:26.459 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:22:26.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.669+0000 7fa27e400640 1 Processor -- start 2024-04-20T21:22:26.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.669+0000 7fa27e400640 1 -- start start 2024-04-20T21:22:26.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.671+0000 7fa27e400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa2780ce5d0 0x7fa278071d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:26.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.671+0000 7fa27e400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa278072240 0x7fa27815ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:26.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.671+0000 7fa27e400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2780b5690 con 0x7fa2780ccdb0 2024-04-20T21:22:26.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.671+0000 7fa27e400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fa27815efa0 con 0x7fa2780ce5d0 2024-04-20T21:22:26.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.671+0000 7fa27e400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa2780cd190 con 0x7fa278072240 2024-04-20T21:22:26.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.671+0000 7fa277400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fa2780ccdb0 0x7fa2780b4f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:49562/0 (socket says 172.21.15.86:49562) 2024-04-20T21:22:26.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.671+0000 7fa277400640 1 -- 172.21.15.86:0/2339207333 learned_addr learned my addr 172.21.15.86:0/2339207333 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:22:26.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.671+0000 7fa276a00640 1 -- 172.21.15.86:0/2339207333 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2790659606 0 0) 0x7fa2780b5690 con 0x7fa2780ccdb0 2024-04-20T21:22:26.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.671+0000 7fa276a00640 1 -- 172.21.15.86:0/2339207333 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa25c004630 con 0x7fa2780ccdb0 2024-04-20T21:22:26.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.671+0000 7fa277e00640 1 --2- 172.21.15.86:0/2339207333 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa278072240 0x7fa27815ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:26.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.672+0000 7fa26ea00640 1 --2- 172.21.15.86:0/2339207333 >> v2:172.21.15.144:3300/0 conn(0x7fa2780ce5d0 0x7fa278071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:26.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.672+0000 7fa276a00640 1 -- 172.21.15.86:0/2339207333 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3464459532 0 0) 0x7fa25c004630 con 0x7fa2780ccdb0 2024-04-20T21:22:26.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.672+0000 7fa276a00640 1 -- 172.21.15.86:0/2339207333 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa278072240 msgr2=0x7fa27815ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:26.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.672+0000 7fa276a00640 1 --2- 172.21.15.86:0/2339207333 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa278072240 0x7fa27815ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:26.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.672+0000 7fa276a00640 1 -- 172.21.15.86:0/2339207333 >> v2:172.21.15.144:3300/0 conn(0x7fa2780ce5d0 msgr2=0x7fa278071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:26.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.672+0000 7fa276a00640 1 --2- 172.21.15.86:0/2339207333 >> v2:172.21.15.144:3300/0 conn(0x7fa2780ce5d0 0x7fa278071d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:26.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.672+0000 7fa276a00640 1 -- 172.21.15.86:0/2339207333 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa27815e390 con 0x7fa2780ccdb0 2024-04-20T21:22:26.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.672+0000 7fa276a00640 1 -- 172.21.15.86:0/2339207333 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fa2680019f0 con 0x7fa2780ccdb0 2024-04-20T21:22:26.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.672+0000 7fa276a00640 1 -- 172.21.15.86:0/2339207333 >> v1:172.21.15.86:6789/0 conn(0x7fa2780ccdb0 legacy=0x7fa2780b4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:26.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.672+0000 7fa276a00640 1 --2- 172.21.15.86:0/2339207333 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa278072240 0x7fa25c004ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:26.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.672+0000 7fa276a00640 1 --2- 172.21.15.86:0/2339207333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa25c0050f0 0x7fa25c005910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:26.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.672+0000 7fa276a00640 1 --2- 172.21.15.86:0/2339207333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa25c005e50 0x7fa25c0062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:26.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.672+0000 7fa276a00640 1 -- 172.21.15.86:0/2339207333 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa27815efa0 con 0x7fa25c0050f0 2024-04-20T21:22:26.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.672+0000 7fa276a00640 1 -- 172.21.15.86:0/2339207333 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa2780cd190 con 0x7fa25c005e50 2024-04-20T21:22:26.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.672+0000 7fa276a00640 1 -- 172.21.15.86:0/2339207333 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa25c003630 con 0x7fa278072240 2024-04-20T21:22:26.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.672+0000 7fa26ea00640 1 --2- 172.21.15.86:0/2339207333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa25c0050f0 0x7fa25c005910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:26.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.673+0000 7fa277400640 1 --2- 172.21.15.86:0/2339207333 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa278072240 0x7fa25c004ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:26.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.673+0000 7fa277e00640 1 --2- 172.21.15.86:0/2339207333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa25c005e50 0x7fa25c0062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:26.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.673+0000 7fa26ea00640 1 -- 172.21.15.86:0/2339207333 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa278072240 msgr2=0x7fa25c004ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:26.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.673+0000 7fa26ea00640 1 --2- 172.21.15.86:0/2339207333 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa278072240 0x7fa25c004ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:26.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.673+0000 7fa26ea00640 1 -- 172.21.15.86:0/2339207333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa25c005e50 msgr2=0x7fa25c0062d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:26.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.673+0000 7fa26ea00640 1 --2- 172.21.15.86:0/2339207333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa25c005e50 0x7fa25c0062d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:26.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.673+0000 7fa26ea00640 1 -- 172.21.15.86:0/2339207333 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa25c004630 con 0x7fa25c0050f0 2024-04-20T21:22:26.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.673+0000 7fa277e00640 1 --2- 172.21.15.86:0/2339207333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa25c005e50 0x7fa25c0062d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:22:26.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.674+0000 7fa26ea00640 1 --2- 172.21.15.86:0/2339207333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa25c0050f0 0x7fa25c005910 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fa26000ba00 tx=0x7fa25c006940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=683971238582989 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:26.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.674+0000 7fa276a00640 1 -- 172.21.15.86:0/2339207333 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa260036020 con 0x7fa25c0050f0 2024-04-20T21:22:26.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.674+0000 7fa276a00640 1 -- 172.21.15.86:0/2339207333 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa260032640 con 0x7fa25c0050f0 2024-04-20T21:22:26.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.674+0000 7fa276a00640 1 -- 172.21.15.86:0/2339207333 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26000b600 con 0x7fa25c0050f0 2024-04-20T21:22:26.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.675+0000 7fa27e400640 1 -- 172.21.15.86:0/2339207333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa25c0050f0 msgr2=0x7fa25c005910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:26.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.675+0000 7fa27e400640 1 --2- 172.21.15.86:0/2339207333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa25c0050f0 0x7fa25c005910 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fa26000ba00 tx=0x7fa25c006940 comp rx=0 tx=0).stop 2024-04-20T21:22:26.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.675+0000 7fa277e00640 1 -- 172.21.15.86:0/2339207333 reap_dead start 2024-04-20T21:22:26.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.675+0000 7fa27e400640 1 -- 172.21.15.86:0/2339207333 shutdown_connections 2024-04-20T21:22:26.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.675+0000 7fa27e400640 1 -- 172.21.15.86:0/2339207333 >> 172.21.15.86:0/2339207333 conn(0x7fa2780b2360 msgr2=0x7fa2780b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:26.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.675+0000 7fa27e400640 1 -- 172.21.15.86:0/2339207333 shutdown_connections 2024-04-20T21:22:26.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.675+0000 7fa27e400640 1 -- 172.21.15.86:0/2339207333 wait complete. 2024-04-20T21:22:26.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.677+0000 7fa27e400640 1 Processor -- start 2024-04-20T21:22:26.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.677+0000 7fa27e400640 1 -- start start 2024-04-20T21:22:26.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.678+0000 7fa27e400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa25c0050f0 0x7fa2780cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:26.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.678+0000 7fa27e400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa27815c700 0x7fa27815ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:26.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.678+0000 7fa27e400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa27815b150 0x7fa2781579e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:26.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.678+0000 7fa27e400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa27815ec30 con 0x7fa27815c700 2024-04-20T21:22:26.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.678+0000 7fa27e400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa27815e950 con 0x7fa25c0050f0 2024-04-20T21:22:26.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.678+0000 7fa27e400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa2780b41e0 con 0x7fa27815b150 2024-04-20T21:22:26.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.678+0000 7fa26ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa27815c700 0x7fa27815ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:26.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.678+0000 7fa26ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa27815c700 0x7fa27815ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38742/0 (socket says 172.21.15.86:38742) 2024-04-20T21:22:26.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.678+0000 7fa26ea00640 1 -- 172.21.15.86:0/2144944147 learned_addr learned my addr 172.21.15.86:0/2144944147 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:26.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.679+0000 7fa277400640 1 --2- 172.21.15.86:0/2144944147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa25c0050f0 0x7fa2780cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:26.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.679+0000 7fa277e00640 1 --2- 172.21.15.86:0/2144944147 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa27815b150 0x7fa2781579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:26.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.679+0000 7fa26ea00640 1 -- 172.21.15.86:0/2144944147 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa27815b150 msgr2=0x7fa2781579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:26.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.679+0000 7fa26ea00640 1 --2- 172.21.15.86:0/2144944147 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa27815b150 0x7fa2781579e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:26.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.679+0000 7fa26ea00640 1 -- 172.21.15.86:0/2144944147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa25c0050f0 msgr2=0x7fa2780cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:26.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.679+0000 7fa26ea00640 1 --2- 172.21.15.86:0/2144944147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa25c0050f0 0x7fa2780cbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:26.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.679+0000 7fa26ea00640 1 -- 172.21.15.86:0/2144944147 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa27815b660 con 0x7fa27815c700 2024-04-20T21:22:26.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.679+0000 7fa277400640 1 --2- 172.21.15.86:0/2144944147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa25c0050f0 0x7fa2780cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:22:26.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.679+0000 7fa26ea00640 1 --2- 172.21.15.86:0/2144944147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa27815c700 0x7fa27815ac10 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fa26002bf90 tx=0x7fa260004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:26.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.679+0000 7fa26fe00640 1 -- 172.21.15.86:0/2144944147 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa260041070 con 0x7fa27815c700 2024-04-20T21:22:26.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.679+0000 7fa26fe00640 1 -- 172.21.15.86:0/2144944147 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa260030a50 con 0x7fa27815c700 2024-04-20T21:22:26.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.679+0000 7fa27e400640 1 -- 172.21.15.86:0/2144944147 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa27815b940 con 0x7fa27815c700 2024-04-20T21:22:26.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.679+0000 7fa26fe00640 1 -- 172.21.15.86:0/2144944147 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa260036040 con 0x7fa27815c700 2024-04-20T21:22:26.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.680+0000 7fa27e400640 1 -- 172.21.15.86:0/2144944147 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa27815bf00 con 0x7fa27815c700 2024-04-20T21:22:26.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.681+0000 7fa27e400640 1 -- 172.21.15.86:0/2144944147 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa278072fc0 con 0x7fa27815c700 2024-04-20T21:22:26.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.682+0000 7fa26fe00640 1 -- 172.21.15.86:0/2144944147 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa260039020 con 0x7fa27815c700 2024-04-20T21:22:26.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.683+0000 7fa26fe00640 1 --2- 172.21.15.86:0/2144944147 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa25406c8f0 0x7fa25406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:26.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.683+0000 7fa26fe00640 1 -- 172.21.15.86:0/2144944147 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fa2600aded0 con 0x7fa27815c700 2024-04-20T21:22:26.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.683+0000 7fa277400640 1 --2- 172.21.15.86:0/2144944147 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa25406c8f0 0x7fa25406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:26.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.684+0000 7fa277400640 1 --2- 172.21.15.86:0/2144944147 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa25406c8f0 0x7fa25406edb0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa268005d10 tx=0x7fa268002150 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:26.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.688+0000 7fa26fe00640 1 -- 172.21.15.86:0/2144944147 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa278072fc0 con 0x7fa27815c700 2024-04-20T21:22:26.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.924+0000 7fa27e400640 1 -- 172.21.15.86:0/2144944147 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2780689e0 con 0x7fa27815c700 2024-04-20T21:22:26.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.925+0000 7fa26fe00640 1 -- 172.21.15.86:0/2144944147 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5206 (secure 0 0 0) 0x7fa26003ae60 con 0x7fa27815c700 2024-04-20T21:22:26.925 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:22:26.925 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4913,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3933408964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3933408964},{"type":"v1","addr":"172.21.15.144:6833","nonce":3933408964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5221,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/1522070692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1522070692},{"type":"v1","addr":"172.21.15.144:6835","nonce":1522070692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":6132,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/1352073341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1352073341},{"type":"v1","addr":"172.21.15.86:6836","nonce":1352073341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:21:07.502800+0000","modified":"2024-04-20T21:22:00.123314+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6135},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6135":{"gid":6135,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":16,"addr":"172.21.15.86:6837/3008383261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3008383261},{"type":"v1","addr":"172.21.15.86:6837","nonce":3008383261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6135,"qdb_cluster":[6135]},"id":4}]} 2024-04-20T21:22:26.925 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 34 2024-04-20T21:22:26.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.928+0000 7fa27e400640 1 -- 172.21.15.86:0/2144944147 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa25406c8f0 msgr2=0x7fa25406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:26.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.928+0000 7fa27e400640 1 --2- 172.21.15.86:0/2144944147 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa25406c8f0 0x7fa25406edb0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa268005d10 tx=0x7fa268002150 comp rx=0 tx=0).stop 2024-04-20T21:22:26.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.928+0000 7fa27e400640 1 -- 172.21.15.86:0/2144944147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa27815c700 msgr2=0x7fa27815ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:26.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.928+0000 7fa27e400640 1 --2- 172.21.15.86:0/2144944147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa27815c700 0x7fa27815ac10 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fa26002bf90 tx=0x7fa260004730 comp rx=0 tx=0).stop 2024-04-20T21:22:26.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.928+0000 7fa27e400640 1 -- 172.21.15.86:0/2144944147 shutdown_connections 2024-04-20T21:22:26.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.928+0000 7fa27e400640 1 --2- 172.21.15.86:0/2144944147 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa25406c8f0 0x7fa25406edb0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:26.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.928+0000 7fa27e400640 1 --2- 172.21.15.86:0/2144944147 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa27815b150 0x7fa2781579e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:26.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.928+0000 7fa27e400640 1 --2- 172.21.15.86:0/2144944147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa27815c700 0x7fa27815ac10 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:26.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.928+0000 7fa27e400640 1 --2- 172.21.15.86:0/2144944147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa25c0050f0 0x7fa2780cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:26.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.928+0000 7fa27e400640 1 -- 172.21.15.86:0/2144944147 >> 172.21.15.86:0/2144944147 conn(0x7fa2780b2360 msgr2=0x7fa2780b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:26.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.928+0000 7fa27e400640 1 -- 172.21.15.86:0/2144944147 shutdown_connections 2024-04-20T21:22:26.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:26.928+0000 7fa27e400640 1 -- 172.21.15.86:0/2144944147 wait complete. 2024-04-20T21:22:26.946 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:22:27.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.149+0000 7f7c5c200640 1 Processor -- start 2024-04-20T21:22:27.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.149+0000 7f7c5c200640 1 -- start start 2024-04-20T21:22:27.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.151+0000 7f7c5c200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c54144e90 0x7f7c540b4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:27.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.151+0000 7f7c5c200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7c540cd170 0x7f7c5406ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:27.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.151+0000 7f7c5c200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c540cdb50 con 0x7f7c540cd5d0 2024-04-20T21:22:27.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.151+0000 7f7c5c200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f7c54145270 con 0x7f7c540cd170 2024-04-20T21:22:27.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.151+0000 7f7c5c200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7c540b3e50 con 0x7f7c54144e90 2024-04-20T21:22:27.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.151+0000 7f7c58e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7c540cd170 0x7f7c5406ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:27.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.151+0000 7f7c5a200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f7c540cd5d0 0x7f7c5415bb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:49578/0 (socket says 172.21.15.86:49578) 2024-04-20T21:22:27.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.151+0000 7f7c5a200640 1 -- 172.21.15.86:0/563507357 learned_addr learned my addr 172.21.15.86:0/563507357 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:22:27.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.152+0000 7f7c59800640 1 --2- 172.21.15.86:0/563507357 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c54144e90 0x7f7c540b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:27.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.152+0000 7f7c58e00640 1 -- 172.21.15.86:0/563507357 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c54144e90 msgr2=0x7f7c540b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:27.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.152+0000 7f7c58e00640 1 --2- 172.21.15.86:0/563507357 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c54144e90 0x7f7c540b4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:27.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.152+0000 7f7c58e00640 1 -- 172.21.15.86:0/563507357 >> v1:172.21.15.86:6789/0 conn(0x7f7c540cd5d0 legacy=0x7f7c5415bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:27.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.152+0000 7f7c58e00640 1 -- 172.21.15.86:0/563507357 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c540b4130 con 0x7f7c540cd170 2024-04-20T21:22:27.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.152+0000 7f7c53e00640 1 -- 172.21.15.86:0/563507357 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 887322531 0 0) 0x7f7c540cdb50 con 0x7f7c540cd5d0 2024-04-20T21:22:27.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.153+0000 7f7c58e00640 1 --2- 172.21.15.86:0/563507357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c540cd170 0x7f7c5406ed40 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f7c400097a0 tx=0x7f7c4002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=935547e43a731839 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:27.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.153+0000 7f7c53e00640 1 -- 172.21.15.86:0/563507357 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c400315a0 con 0x7f7c540cd170 2024-04-20T21:22:27.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.153+0000 7f7c53e00640 1 -- 172.21.15.86:0/563507357 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c40037070 con 0x7f7c540cd170 2024-04-20T21:22:27.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.153+0000 7f7c53e00640 1 -- 172.21.15.86:0/563507357 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c40031960 con 0x7f7c540cd170 2024-04-20T21:22:27.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.154+0000 7f7c5c200640 1 -- 172.21.15.86:0/563507357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c540cd170 msgr2=0x7f7c5406ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:27.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.154+0000 7f7c5c200640 1 --2- 172.21.15.86:0/563507357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c540cd170 0x7f7c5406ed40 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f7c400097a0 tx=0x7f7c4002b840 comp rx=0 tx=0).stop 2024-04-20T21:22:27.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.154+0000 7f7c5c200640 1 -- 172.21.15.86:0/563507357 shutdown_connections 2024-04-20T21:22:27.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.154+0000 7f7c5c200640 1 --2- 172.21.15.86:0/563507357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c540cd170 0x7f7c5406ed40 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:27.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.154+0000 7f7c5c200640 1 --2- 172.21.15.86:0/563507357 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c54144e90 0x7f7c540b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:27.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.154+0000 7f7c5c200640 1 -- 172.21.15.86:0/563507357 >> 172.21.15.86:0/563507357 conn(0x7f7c540b2360 msgr2=0x7f7c540b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:27.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.154+0000 7f7c5c200640 1 -- 172.21.15.86:0/563507357 shutdown_connections 2024-04-20T21:22:27.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.154+0000 7f7c5c200640 1 -- 172.21.15.86:0/563507357 wait complete. 2024-04-20T21:22:27.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.156+0000 7f7c5c200640 1 Processor -- start 2024-04-20T21:22:27.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.156+0000 7f7c5c200640 1 -- start start 2024-04-20T21:22:27.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.157+0000 7f7c5c200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c540cd170 0x7f7c5413f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:27.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.157+0000 7f7c5c200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c540cd5d0 0x7f7c541441f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:27.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.157+0000 7f7c5c200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c54144e90 0x7f7c540c91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:27.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.157+0000 7f7c5c200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7c540d0010 con 0x7f7c540cd170 2024-04-20T21:22:27.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.157+0000 7f7c5c200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7c540b4e70 con 0x7f7c540cd5d0 2024-04-20T21:22:27.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.157+0000 7f7c5c200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7c540d0190 con 0x7f7c54144e90 2024-04-20T21:22:27.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.157+0000 7f7c59800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c540cd170 0x7f7c5413f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:27.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.157+0000 7f7c59800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c540cd170 0x7f7c5413f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38772/0 (socket says 172.21.15.86:38772) 2024-04-20T21:22:27.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.157+0000 7f7c59800640 1 -- 172.21.15.86:0/1002406945 learned_addr learned my addr 172.21.15.86:0/1002406945 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:27.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.158+0000 7f7c58e00640 1 --2- 172.21.15.86:0/1002406945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c540cd5d0 0x7f7c541441f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:27.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.158+0000 7f7c5a200640 1 --2- 172.21.15.86:0/1002406945 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c54144e90 0x7f7c540c91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:27.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.158+0000 7f7c59800640 1 -- 172.21.15.86:0/1002406945 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c54144e90 msgr2=0x7f7c540c91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:27.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.158+0000 7f7c59800640 1 --2- 172.21.15.86:0/1002406945 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c54144e90 0x7f7c540c91c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:27.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.158+0000 7f7c59800640 1 -- 172.21.15.86:0/1002406945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c540cd5d0 msgr2=0x7f7c541441f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:27.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.158+0000 7f7c59800640 1 --2- 172.21.15.86:0/1002406945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c540cd5d0 0x7f7c541441f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:27.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.158+0000 7f7c59800640 1 -- 172.21.15.86:0/1002406945 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c540cc150 con 0x7f7c540cd170 2024-04-20T21:22:27.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.158+0000 7f7c59800640 1 --2- 172.21.15.86:0/1002406945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c540cd170 0x7f7c5413f070 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f7c4400d800 tx=0x7f7c4400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:27.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.158+0000 7f7c51600640 1 -- 172.21.15.86:0/1002406945 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4400b660 con 0x7f7c540cd170 2024-04-20T21:22:27.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.158+0000 7f7c51600640 1 -- 172.21.15.86:0/1002406945 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c44019070 con 0x7f7c540cd170 2024-04-20T21:22:27.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.158+0000 7f7c51600640 1 -- 172.21.15.86:0/1002406945 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4400ba20 con 0x7f7c540cd170 2024-04-20T21:22:27.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.158+0000 7f7c5c200640 1 -- 172.21.15.86:0/1002406945 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c540cc430 con 0x7f7c540cd170 2024-04-20T21:22:27.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.159+0000 7f7c5c200640 1 -- 172.21.15.86:0/1002406945 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c541397c0 con 0x7f7c540cd170 2024-04-20T21:22:27.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.160+0000 7f7c5c200640 1 -- 172.21.15.86:0/1002406945 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c540689e0 con 0x7f7c540cd170 2024-04-20T21:22:27.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.160+0000 7f7c51600640 1 -- 172.21.15.86:0/1002406945 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7c4400bbc0 con 0x7f7c540cd170 2024-04-20T21:22:27.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.161+0000 7f7c51600640 1 --2- 172.21.15.86:0/1002406945 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7c3406c8f0 0x7f7c3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:27.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.161+0000 7f7c51600640 1 -- 172.21.15.86:0/1002406945 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f7c4408bf90 con 0x7f7c540cd170 2024-04-20T21:22:27.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.162+0000 7f7c58e00640 1 --2- 172.21.15.86:0/1002406945 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7c3406c8f0 0x7f7c3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:27.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.162+0000 7f7c58e00640 1 --2- 172.21.15.86:0/1002406945 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7c3406c8f0 0x7f7c3406edb0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7c40036000 tx=0x7f7c40008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:27.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.166+0000 7f7c51600640 1 -- 172.21.15.86:0/1002406945 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7c4401c360 con 0x7f7c540cd170 2024-04-20T21:22:27.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.366+0000 7f7c5c200640 1 -- 172.21.15.86:0/1002406945 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7c540cf7a0 con 0x7f7c540cd170 2024-04-20T21:22:27.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.369+0000 7f7c51600640 1 -- 172.21.15.86:0/1002406945 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+16665 (secure 0 0 0) 0x7f7c4405a3b0 con 0x7f7c540cd170 2024-04-20T21:22:27.377 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:22:27.377 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":59,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:21:09.072894+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:21:05.248922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-20T21:21:06.479064+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:22:27.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.371+0000 7f7c5c200640 1 -- 172.21.15.86:0/1002406945 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7c3406c8f0 msgr2=0x7f7c3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:27.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.371+0000 7f7c5c200640 1 --2- 172.21.15.86:0/1002406945 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7c3406c8f0 0x7f7c3406edb0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7c40036000 tx=0x7f7c40008030 comp rx=0 tx=0).stop 2024-04-20T21:22:27.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.371+0000 7f7c5c200640 1 -- 172.21.15.86:0/1002406945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c540cd170 msgr2=0x7f7c5413f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:27.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.371+0000 7f7c5c200640 1 --2- 172.21.15.86:0/1002406945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c540cd170 0x7f7c5413f070 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f7c4400d800 tx=0x7f7c4400dcc0 comp rx=0 tx=0).stop 2024-04-20T21:22:27.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.372+0000 7f7c5c200640 1 -- 172.21.15.86:0/1002406945 shutdown_connections 2024-04-20T21:22:27.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.372+0000 7f7c5c200640 1 --2- 172.21.15.86:0/1002406945 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7c3406c8f0 0x7f7c3406edb0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:27.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.372+0000 7f7c5c200640 1 --2- 172.21.15.86:0/1002406945 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c54144e90 0x7f7c540c91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:27.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.372+0000 7f7c5c200640 1 --2- 172.21.15.86:0/1002406945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c540cd5d0 0x7f7c541441f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:27.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.372+0000 7f7c5c200640 1 --2- 172.21.15.86:0/1002406945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c540cd170 0x7f7c5413f070 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:27.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.372+0000 7f7c5c200640 1 -- 172.21.15.86:0/1002406945 >> 172.21.15.86:0/1002406945 conn(0x7f7c540b2360 msgr2=0x7f7c5406e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:27.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.372+0000 7f7c5c200640 1 -- 172.21.15.86:0/1002406945 shutdown_connections 2024-04-20T21:22:27.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.372+0000 7f7c5c200640 1 -- 172.21.15.86:0/1002406945 wait complete. 2024-04-20T21:22:27.392 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:21:05.248922+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:21:06.479064+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:22:27.393 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:22:27.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.604+0000 7f7c19800640 1 Processor -- start 2024-04-20T21:22:27.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.605+0000 7f7c19800640 1 -- start start 2024-04-20T21:22:27.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.606+0000 7f7c19800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c14146210 0x7f7c1406e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:27.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.606+0000 7f7c19800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7c1406bfd0 0x7f7c140cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:27.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.606+0000 7f7c19800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c1415cd40 con 0x7f7c1406bbf0 2024-04-20T21:22:27.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.606+0000 7f7c19800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f7c1415d000 con 0x7f7c1406bfd0 2024-04-20T21:22:27.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.606+0000 7f7c19800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7c1415d2e0 con 0x7f7c14146210 2024-04-20T21:22:27.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.607+0000 7f7c13400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7c1406bfd0 0x7f7c140cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:27.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.607+0000 7f7c13400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7c1406bfd0 0x7f7c140cd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38782/0 (socket says 172.21.15.86:38782) 2024-04-20T21:22:27.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.607+0000 7f7c13400640 1 -- 172.21.15.86:0/381944151 learned_addr learned my addr 172.21.15.86:0/381944151 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:27.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.607+0000 7f7c0a000640 1 --2- 172.21.15.86:0/381944151 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c14146210 0x7f7c1406e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:27.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.607+0000 7f7c13400640 1 -- 172.21.15.86:0/381944151 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c14146210 msgr2=0x7f7c1406e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:27.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.607+0000 7f7c13400640 1 --2- 172.21.15.86:0/381944151 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c14146210 0x7f7c1406e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:27.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.607+0000 7f7c13400640 1 -- 172.21.15.86:0/381944151 >> v1:172.21.15.144:6789/0 conn(0x7f7c1406bbf0 legacy=0x7f7c14145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:27.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.607+0000 7f7c13400640 1 -- 172.21.15.86:0/381944151 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c1415d5c0 con 0x7f7c1406bfd0 2024-04-20T21:22:27.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.608+0000 7f7c13400640 1 --2- 172.21.15.86:0/381944151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c1406bfd0 0x7f7c140cd540 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c009870 tx=0x7f7c0c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2268beda8bf1a141 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:27.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.608+0000 7f7c12000640 1 -- 172.21.15.86:0/381944151 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c035020 con 0x7f7c1406bfd0 2024-04-20T21:22:27.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.609+0000 7f7c12000640 1 -- 172.21.15.86:0/381944151 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c0c031640 con 0x7f7c1406bfd0 2024-04-20T21:22:27.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.609+0000 7f7c12000640 1 -- 172.21.15.86:0/381944151 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c02d040 con 0x7f7c1406bfd0 2024-04-20T21:22:27.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.609+0000 7f7c19800640 1 -- 172.21.15.86:0/381944151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c1406bfd0 msgr2=0x7f7c140cd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:27.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.609+0000 7f7c19800640 1 --2- 172.21.15.86:0/381944151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c1406bfd0 0x7f7c140cd540 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c009870 tx=0x7f7c0c02b8f0 comp rx=0 tx=0).stop 2024-04-20T21:22:27.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.609+0000 7f7c19800640 1 -- 172.21.15.86:0/381944151 shutdown_connections 2024-04-20T21:22:27.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.609+0000 7f7c19800640 1 --2- 172.21.15.86:0/381944151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c1406bfd0 0x7f7c140cd540 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:27.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.609+0000 7f7c19800640 1 --2- 172.21.15.86:0/381944151 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c14146210 0x7f7c1406e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:27.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.609+0000 7f7c19800640 1 -- 172.21.15.86:0/381944151 >> 172.21.15.86:0/381944151 conn(0x7f7c140b2110 msgr2=0x7f7c140b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:27.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.609+0000 7f7c19800640 1 -- 172.21.15.86:0/381944151 shutdown_connections 2024-04-20T21:22:27.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.610+0000 7f7c19800640 1 -- 172.21.15.86:0/381944151 wait complete. 2024-04-20T21:22:27.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.611+0000 7f7c19800640 1 Processor -- start 2024-04-20T21:22:27.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.611+0000 7f7c19800640 1 -- start start 2024-04-20T21:22:27.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.612+0000 7f7c19800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c1406bbf0 0x7f7c141e6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:27.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.612+0000 7f7c19800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c1406bfd0 0x7f7c140b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:27.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.612+0000 7f7c19800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c14146210 0x7f7c1415c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:27.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.612+0000 7f7c19800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7c140cdc00 con 0x7f7c1406bfd0 2024-04-20T21:22:27.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.612+0000 7f7c19800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7c140cda80 con 0x7f7c14146210 2024-04-20T21:22:27.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.612+0000 7f7c19800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7c140cdd80 con 0x7f7c1406bbf0 2024-04-20T21:22:27.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.612+0000 7f7c0a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c1406bfd0 0x7f7c140b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:27.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.612+0000 7f7c0a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c1406bfd0 0x7f7c140b4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38786/0 (socket says 172.21.15.86:38786) 2024-04-20T21:22:27.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.612+0000 7f7c0a000640 1 -- 172.21.15.86:0/3695411773 learned_addr learned my addr 172.21.15.86:0/3695411773 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:27.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.613+0000 7f7c12a00640 1 --2- 172.21.15.86:0/3695411773 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c1406bbf0 0x7f7c141e6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:27.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.613+0000 7f7c13400640 1 --2- 172.21.15.86:0/3695411773 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c14146210 0x7f7c1415c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:27.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.613+0000 7f7c0a000640 1 -- 172.21.15.86:0/3695411773 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c1406bbf0 msgr2=0x7f7c141e6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:27.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.613+0000 7f7c0a000640 1 --2- 172.21.15.86:0/3695411773 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c1406bbf0 0x7f7c141e6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:27.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.613+0000 7f7c0a000640 1 -- 172.21.15.86:0/3695411773 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c14146210 msgr2=0x7f7c1415c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:27.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.613+0000 7f7c0a000640 1 --2- 172.21.15.86:0/3695411773 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c14146210 0x7f7c1415c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:27.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.613+0000 7f7c0a000640 1 -- 172.21.15.86:0/3695411773 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c141e65c0 con 0x7f7c1406bfd0 2024-04-20T21:22:27.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.613+0000 7f7c0a000640 1 --2- 172.21.15.86:0/3695411773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c1406bfd0 0x7f7c140b4e10 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc00c7f0 tx=0x7f7bfc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:27.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.613+0000 7f7c0b400640 1 -- 172.21.15.86:0/3695411773 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc015070 con 0x7f7c1406bfd0 2024-04-20T21:22:27.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.614+0000 7f7c0b400640 1 -- 172.21.15.86:0/3695411773 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bfc004220 con 0x7f7c1406bfd0 2024-04-20T21:22:27.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.614+0000 7f7c0b400640 1 -- 172.21.15.86:0/3695411773 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc003610 con 0x7f7c1406bfd0 2024-04-20T21:22:27.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.614+0000 7f7c19800640 1 -- 172.21.15.86:0/3695411773 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c141e4fe0 con 0x7f7c1406bfd0 2024-04-20T21:22:27.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.614+0000 7f7c19800640 1 -- 172.21.15.86:0/3695411773 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c141e55a0 con 0x7f7c1406bfd0 2024-04-20T21:22:27.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.615+0000 7f7c19800640 1 -- 172.21.15.86:0/3695411773 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c140689e0 con 0x7f7c1406bfd0 2024-04-20T21:22:27.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.616+0000 7f7c0b400640 1 -- 172.21.15.86:0/3695411773 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7bfc0039c0 con 0x7f7c1406bfd0 2024-04-20T21:22:27.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.617+0000 7f7c0b400640 1 --2- 172.21.15.86:0/3695411773 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7bf006c8f0 0x7f7bf006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:27.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.617+0000 7f7c0b400640 1 -- 172.21.15.86:0/3695411773 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f7bfc08bdc0 con 0x7f7c1406bfd0 2024-04-20T21:22:27.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.617+0000 7f7c12a00640 1 --2- 172.21.15.86:0/3695411773 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7bf006c8f0 0x7f7bf006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:27.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.618+0000 7f7c12a00640 1 --2- 172.21.15.86:0/3695411773 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7bf006c8f0 0x7f7bf006edb0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7c04007fd0 tx=0x7f7c04009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:27.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.622+0000 7f7c0b400640 1 -- 172.21.15.86:0/3695411773 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7bfc05a260 con 0x7f7c1406bfd0 2024-04-20T21:22:27.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.823+0000 7f7c19800640 1 -- 172.21.15.86:0/3695411773 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7c1406c780 con 0x7f7c1406bfd0 2024-04-20T21:22:27.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.824+0000 7f7c0b400640 1 -- 172.21.15.86:0/3695411773 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) ==== 93+0+10 (secure 0 0 0) 0x7f7bfc059b60 con 0x7f7c1406bfd0 2024-04-20T21:22:27.823 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:22:27.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.826+0000 7f7c19800640 1 -- 172.21.15.86:0/3695411773 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7bf006c8f0 msgr2=0x7f7bf006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:27.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.826+0000 7f7c19800640 1 --2- 172.21.15.86:0/3695411773 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7bf006c8f0 0x7f7bf006edb0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7c04007fd0 tx=0x7f7c04009040 comp rx=0 tx=0).stop 2024-04-20T21:22:27.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.826+0000 7f7c19800640 1 -- 172.21.15.86:0/3695411773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c1406bfd0 msgr2=0x7f7c140b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:27.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.826+0000 7f7c19800640 1 --2- 172.21.15.86:0/3695411773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c1406bfd0 0x7f7c140b4e10 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc00c7f0 tx=0x7f7bfc00ccb0 comp rx=0 tx=0).stop 2024-04-20T21:22:27.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.827+0000 7f7c19800640 1 -- 172.21.15.86:0/3695411773 shutdown_connections 2024-04-20T21:22:27.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.827+0000 7f7c19800640 1 --2- 172.21.15.86:0/3695411773 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7bf006c8f0 0x7f7bf006edb0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:27.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.827+0000 7f7c19800640 1 --2- 172.21.15.86:0/3695411773 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c14146210 0x7f7c1415c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:27.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.827+0000 7f7c19800640 1 --2- 172.21.15.86:0/3695411773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c1406bfd0 0x7f7c140b4e10 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:27.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.827+0000 7f7c19800640 1 --2- 172.21.15.86:0/3695411773 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c1406bbf0 0x7f7c141e6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:27.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.827+0000 7f7c19800640 1 -- 172.21.15.86:0/3695411773 >> 172.21.15.86:0/3695411773 conn(0x7f7c140b2110 msgr2=0x7f7c1406e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:27.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.827+0000 7f7c19800640 1 -- 172.21.15.86:0/3695411773 shutdown_connections 2024-04-20T21:22:27.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:27.827+0000 7f7c19800640 1 -- 172.21.15.86:0/3695411773 wait complete. 2024-04-20T21:22:27.845 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:22:28.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.043+0000 7f6f9f200640 1 Processor -- start 2024-04-20T21:22:28.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.043+0000 7f6f9f200640 1 -- start start 2024-04-20T21:22:28.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.045+0000 7f6f9f200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6fa0144e90 0x7f6fa00af780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:28.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.045+0000 7f6f9f200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6fa0072450 0x7f6fa0160150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:28.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.045+0000 7f6f9f200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fa006e2d0 con 0x7f6fa006dda0 2024-04-20T21:22:28.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.045+0000 7f6f9f200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f6fa0145270 con 0x7f6fa0072450 2024-04-20T21:22:28.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.045+0000 7f6f9f200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f6fa006e5b0 con 0x7f6fa0144e90 2024-04-20T21:22:28.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.045+0000 7f6f9e800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6fa0072450 0x7f6fa0160150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:28.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.045+0000 7f6f9e800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6fa0072450 0x7f6fa0160150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38794/0 (socket says 172.21.15.86:38794) 2024-04-20T21:22:28.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.045+0000 7f6f9e800640 1 -- 172.21.15.86:0/2857516687 learned_addr learned my addr 172.21.15.86:0/2857516687 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:28.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.046+0000 7f6f9de00640 1 --2- 172.21.15.86:0/2857516687 >> v2:172.21.15.144:3300/0 conn(0x7f6fa0144e90 0x7f6fa00af780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:28.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.046+0000 7f6f9e800640 1 -- 172.21.15.86:0/2857516687 >> v2:172.21.15.144:3300/0 conn(0x7f6fa0144e90 msgr2=0x7f6fa00af780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:28.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.046+0000 7f6f9e800640 1 --2- 172.21.15.86:0/2857516687 >> v2:172.21.15.144:3300/0 conn(0x7f6fa0144e90 0x7f6fa00af780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:28.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.046+0000 7f6f9e800640 1 -- 172.21.15.86:0/2857516687 >> v1:172.21.15.144:6789/0 conn(0x7f6fa006dda0 legacy=0x7f6fa0071d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:28.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.046+0000 7f6f9e800640 1 -- 172.21.15.86:0/2857516687 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fa006e890 con 0x7f6fa0072450 2024-04-20T21:22:28.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.048+0000 7f6f9e800640 1 --2- 172.21.15.86:0/2857516687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fa0072450 0x7f6fa0160150 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f6fa006fe90 tx=0x7f6f9802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0066bdbeb102777 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:28.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.048+0000 7f6f9ca00640 1 -- 172.21.15.86:0/2857516687 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9802ca80 con 0x7f6fa0072450 2024-04-20T21:22:28.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.048+0000 7f6f9ca00640 1 -- 172.21.15.86:0/2857516687 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f9802cc20 con 0x7f6fa0072450 2024-04-20T21:22:28.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.048+0000 7f6f9ca00640 1 -- 172.21.15.86:0/2857516687 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f980343f0 con 0x7f6fa0072450 2024-04-20T21:22:28.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.049+0000 7f6f9f200640 1 -- 172.21.15.86:0/2857516687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fa0072450 msgr2=0x7f6fa0160150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:28.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.049+0000 7f6f9f200640 1 --2- 172.21.15.86:0/2857516687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fa0072450 0x7f6fa0160150 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f6fa006fe90 tx=0x7f6f9802bee0 comp rx=0 tx=0).stop 2024-04-20T21:22:28.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.049+0000 7f6f9f200640 1 -- 172.21.15.86:0/2857516687 shutdown_connections 2024-04-20T21:22:28.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.049+0000 7f6f9f200640 1 --2- 172.21.15.86:0/2857516687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fa0072450 0x7f6fa0160150 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:28.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.049+0000 7f6f9f200640 1 --2- 172.21.15.86:0/2857516687 >> v2:172.21.15.144:3300/0 conn(0x7f6fa0144e90 0x7f6fa00af780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:28.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.049+0000 7f6f9f200640 1 -- 172.21.15.86:0/2857516687 >> 172.21.15.86:0/2857516687 conn(0x7f6fa0067cf0 msgr2=0x7f6fa006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:28.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.049+0000 7f6f9f200640 1 -- 172.21.15.86:0/2857516687 shutdown_connections 2024-04-20T21:22:28.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.049+0000 7f6f9f200640 1 -- 172.21.15.86:0/2857516687 wait complete. 2024-04-20T21:22:28.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.050+0000 7f6f9f200640 1 Processor -- start 2024-04-20T21:22:28.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.051+0000 7f6f9f200640 1 -- start start 2024-04-20T21:22:28.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.051+0000 7f6f9f200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6fa006dda0 0x7f6fa013e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:28.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.051+0000 7f6f9f200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fa0072450 0x7f6fa00b3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:28.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.052+0000 7f6f9f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fa0144e90 0x7f6fa015e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:28.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.052+0000 7f6f9f200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa00afcc0 con 0x7f6fa0144e90 2024-04-20T21:22:28.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.052+0000 7f6f9f200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa006ee50 con 0x7f6fa0072450 2024-04-20T21:22:28.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.052+0000 7f6f9f200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6fa00afe40 con 0x7f6fa006dda0 2024-04-20T21:22:28.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.052+0000 7f6f9e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fa0144e90 0x7f6fa015e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:28.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.052+0000 7f6f9e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fa0144e90 0x7f6fa015e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38802/0 (socket says 172.21.15.86:38802) 2024-04-20T21:22:28.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.052+0000 7f6f9e800640 1 -- 172.21.15.86:0/3107641363 learned_addr learned my addr 172.21.15.86:0/3107641363 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:28.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.052+0000 7f6f9de00640 1 --2- 172.21.15.86:0/3107641363 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6fa006dda0 0x7f6fa013e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:28.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.052+0000 7f6f9d400640 1 --2- 172.21.15.86:0/3107641363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fa0072450 0x7f6fa00b3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:28.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.052+0000 7f6f9e800640 1 -- 172.21.15.86:0/3107641363 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6fa006dda0 msgr2=0x7f6fa013e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:28.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.052+0000 7f6f9e800640 1 --2- 172.21.15.86:0/3107641363 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6fa006dda0 0x7f6fa013e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:28.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.052+0000 7f6f9e800640 1 -- 172.21.15.86:0/3107641363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fa0072450 msgr2=0x7f6fa00b3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:28.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.052+0000 7f6f9e800640 1 --2- 172.21.15.86:0/3107641363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fa0072450 0x7f6fa00b3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:28.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.052+0000 7f6f9e800640 1 -- 172.21.15.86:0/3107641363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fa015e6e0 con 0x7f6fa0144e90 2024-04-20T21:22:28.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.053+0000 7f6f9e800640 1 --2- 172.21.15.86:0/3107641363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fa0144e90 0x7f6fa015e1a0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f6f98036040 tx=0x7f6f98004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:28.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.053+0000 7f6f96000640 1 -- 172.21.15.86:0/3107641363 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f98043020 con 0x7f6fa0144e90 2024-04-20T21:22:28.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.053+0000 7f6f96000640 1 -- 172.21.15.86:0/3107641363 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f9803e450 con 0x7f6fa0144e90 2024-04-20T21:22:28.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.053+0000 7f6f96000640 1 -- 172.21.15.86:0/3107641363 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f98009510 con 0x7f6fa0144e90 2024-04-20T21:22:28.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.053+0000 7f6f9f200640 1 -- 172.21.15.86:0/3107641363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fa00b29b0 con 0x7f6fa0144e90 2024-04-20T21:22:28.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.053+0000 7f6f9f200640 1 -- 172.21.15.86:0/3107641363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fa00b2f70 con 0x7f6fa0144e90 2024-04-20T21:22:28.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.054+0000 7f6f9f200640 1 -- 172.21.15.86:0/3107641363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fa004fd90 con 0x7f6fa0144e90 2024-04-20T21:22:28.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.055+0000 7f6f96000640 1 -- 172.21.15.86:0/3107641363 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6f98046020 con 0x7f6fa0144e90 2024-04-20T21:22:28.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.056+0000 7f6f96000640 1 --2- 172.21.15.86:0/3107641363 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6f8006c8f0 0x7f6f8006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:28.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.056+0000 7f6f96000640 1 -- 172.21.15.86:0/3107641363 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f6f980ae380 con 0x7f6fa0144e90 2024-04-20T21:22:28.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.057+0000 7f6f9de00640 1 --2- 172.21.15.86:0/3107641363 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6f8006c8f0 0x7f6f8006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:28.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.057+0000 7f6f9de00640 1 --2- 172.21.15.86:0/3107641363 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6f8006c8f0 0x7f6f8006edb0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6f90006fd0 tx=0x7f6f90008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:28.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.061+0000 7f6f96000640 1 -- 172.21.15.86:0/3107641363 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6f9804d030 con 0x7f6fa0144e90 2024-04-20T21:22:28.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.253+0000 7f6f9f200640 1 -- 172.21.15.86:0/3107641363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6fa0144500 con 0x7f6fa0144e90 2024-04-20T21:22:28.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.253+0000 7f6f96000640 1 -- 172.21.15.86:0/3107641363 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v59) ==== 104+0+11 (secure 0 0 0) 0x7f6f9807c7b0 con 0x7f6fa0144e90 2024-04-20T21:22:28.252 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-20T21:22:28.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.256+0000 7f6f9f200640 1 -- 172.21.15.86:0/3107641363 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6f8006c8f0 msgr2=0x7f6f8006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:28.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.256+0000 7f6f9f200640 1 --2- 172.21.15.86:0/3107641363 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6f8006c8f0 0x7f6f8006edb0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6f90006fd0 tx=0x7f6f90008040 comp rx=0 tx=0).stop 2024-04-20T21:22:28.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.256+0000 7f6f9f200640 1 -- 172.21.15.86:0/3107641363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fa0144e90 msgr2=0x7f6fa015e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:28.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.256+0000 7f6f9f200640 1 --2- 172.21.15.86:0/3107641363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fa0144e90 0x7f6fa015e1a0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f6f98036040 tx=0x7f6f98004a40 comp rx=0 tx=0).stop 2024-04-20T21:22:28.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.256+0000 7f6f9f200640 1 -- 172.21.15.86:0/3107641363 shutdown_connections 2024-04-20T21:22:28.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.256+0000 7f6f9f200640 1 --2- 172.21.15.86:0/3107641363 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6f8006c8f0 0x7f6f8006edb0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:28.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.256+0000 7f6f9f200640 1 --2- 172.21.15.86:0/3107641363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6fa0144e90 0x7f6fa015e1a0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:28.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.256+0000 7f6f9f200640 1 --2- 172.21.15.86:0/3107641363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fa0072450 0x7f6fa00b3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:28.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.256+0000 7f6f9f200640 1 --2- 172.21.15.86:0/3107641363 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6fa006dda0 0x7f6fa013e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:28.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.256+0000 7f6f9f200640 1 -- 172.21.15.86:0/3107641363 >> 172.21.15.86:0/3107641363 conn(0x7f6fa0067cf0 msgr2=0x7f6fa00affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:28.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.256+0000 7f6f9f200640 1 -- 172.21.15.86:0/3107641363 shutdown_connections 2024-04-20T21:22:28.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.256+0000 7f6f9f200640 1 -- 172.21.15.86:0/3107641363 wait complete. 2024-04-20T21:22:28.273 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:22:28.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.475+0000 7fd3bfa00640 1 Processor -- start 2024-04-20T21:22:28.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.475+0000 7fd3bfa00640 1 -- start start 2024-04-20T21:22:28.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.477+0000 7fd3bfa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd3b80cdc50 0x7fd3b806e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:28.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.477+0000 7fd3bfa00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3b815cd40 con 0x7fd3b8146210 2024-04-20T21:22:28.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.477+0000 7fd3bfa00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3b815d000 con 0x7fd3b806bbf0 2024-04-20T21:22:28.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.477+0000 7fd3bfa00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd3b815d2e0 con 0x7fd3b80cdc50 2024-04-20T21:22:28.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.477+0000 7fd3b7e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd3b8146210 0x7fd3b80cd540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:49586/0 (socket says 172.21.15.86:49586) 2024-04-20T21:22:28.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.477+0000 7fd3b7e00640 1 -- 172.21.15.86:0/3802512625 learned_addr learned my addr 172.21.15.86:0/3802512625 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:22:28.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.477+0000 7fd3bda00640 1 --2- 172.21.15.86:0/3802512625 >> v2:172.21.15.86:3300/0 conn(0x7fd3b80cdc50 0x7fd3b806e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:28.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.478+0000 7fd3b7400640 1 -- 172.21.15.86:0/3802512625 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2932499767 0 0) 0x7fd3b815cd40 con 0x7fd3b8146210 2024-04-20T21:22:28.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.478+0000 7fd3b7400640 1 -- 172.21.15.86:0/3802512625 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd3a0003520 con 0x7fd3b8146210 2024-04-20T21:22:28.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.478+0000 7fd3bda00640 1 -- 172.21.15.86:0/3802512625 >> v1:172.21.15.144:6789/0 conn(0x7fd3b806bbf0 legacy=0x7fd3b8145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:28.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.478+0000 7fd3bda00640 1 -- 172.21.15.86:0/3802512625 >> v1:172.21.15.86:6789/0 conn(0x7fd3b8146210 legacy=0x7fd3b80cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:28.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.478+0000 7fd3bda00640 1 -- 172.21.15.86:0/3802512625 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3b815d5c0 con 0x7fd3b80cdc50 2024-04-20T21:22:28.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.479+0000 7fd3bda00640 1 --2- 172.21.15.86:0/3802512625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3b80cdc50 0x7fd3b806e000 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0009d90 tx=0x7fd3b002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afa8f81f3dd25755 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:28.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.479+0000 7fd3b7400640 1 -- 172.21.15.86:0/3802512625 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b0039070 con 0x7fd3b80cdc50 2024-04-20T21:22:28.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.479+0000 7fd3b7400640 1 -- 172.21.15.86:0/3802512625 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3b002cd00 con 0x7fd3b80cdc50 2024-04-20T21:22:28.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.479+0000 7fd3b7400640 1 -- 172.21.15.86:0/3802512625 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b0034660 con 0x7fd3b80cdc50 2024-04-20T21:22:28.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.480+0000 7fd3bfa00640 1 -- 172.21.15.86:0/3802512625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3b80cdc50 msgr2=0x7fd3b806e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:28.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.480+0000 7fd3bfa00640 1 --2- 172.21.15.86:0/3802512625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3b80cdc50 0x7fd3b806e000 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0009d90 tx=0x7fd3b002bf50 comp rx=0 tx=0).stop 2024-04-20T21:22:28.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.480+0000 7fd3bfa00640 1 -- 172.21.15.86:0/3802512625 shutdown_connections 2024-04-20T21:22:28.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.480+0000 7fd3bfa00640 1 --2- 172.21.15.86:0/3802512625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3b80cdc50 0x7fd3b806e000 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:28.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.480+0000 7fd3bfa00640 1 -- 172.21.15.86:0/3802512625 >> 172.21.15.86:0/3802512625 conn(0x7fd3b80b2110 msgr2=0x7fd3b80b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:28.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.480+0000 7fd3bfa00640 1 -- 172.21.15.86:0/3802512625 shutdown_connections 2024-04-20T21:22:28.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.480+0000 7fd3bfa00640 1 -- 172.21.15.86:0/3802512625 wait complete. 2024-04-20T21:22:28.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.482+0000 7fd3bfa00640 1 Processor -- start 2024-04-20T21:22:28.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.482+0000 7fd3bfa00640 1 -- start start 2024-04-20T21:22:28.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.483+0000 7fd3bfa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3b806bbf0 0x7fd3b81e5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:28.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.483+0000 7fd3bfa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd3b80cdc50 0x7fd3b814ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:28.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.483+0000 7fd3bfa00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3b8146210 0x7fd3b814ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:28.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.483+0000 7fd3bfa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b806c150 con 0x7fd3b806bbf0 2024-04-20T21:22:28.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.483+0000 7fd3bfa00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b806bfd0 con 0x7fd3b8146210 2024-04-20T21:22:28.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.483+0000 7fd3bfa00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd3b806c2d0 con 0x7fd3b80cdc50 2024-04-20T21:22:28.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.483+0000 7fd3bd000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3b806bbf0 0x7fd3b81e5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:28.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.483+0000 7fd3bd000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3b806bbf0 0x7fd3b81e5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38828/0 (socket says 172.21.15.86:38828) 2024-04-20T21:22:28.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.483+0000 7fd3bd000640 1 -- 172.21.15.86:0/3452624011 learned_addr learned my addr 172.21.15.86:0/3452624011 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:28.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.484+0000 7fd3b7e00640 1 --2- 172.21.15.86:0/3452624011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd3b80cdc50 0x7fd3b814ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:28.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.484+0000 7fd3bd000640 1 -- 172.21.15.86:0/3452624011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd3b80cdc50 msgr2=0x7fd3b814ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:28.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.484+0000 7fd3bd000640 1 --2- 172.21.15.86:0/3452624011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd3b80cdc50 0x7fd3b814ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:28.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.484+0000 7fd3bd000640 1 -- 172.21.15.86:0/3452624011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3b8146210 msgr2=0x7fd3b814ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:28.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.484+0000 7fd3bd000640 1 --2- 172.21.15.86:0/3452624011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3b8146210 0x7fd3b814ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:28.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.484+0000 7fd3bd000640 1 -- 172.21.15.86:0/3452624011 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3b8149dc0 con 0x7fd3b806bbf0 2024-04-20T21:22:28.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.484+0000 7fd3b7e00640 1 --2- 172.21.15.86:0/3452624011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd3b80cdc50 0x7fd3b814ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:22:28.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.484+0000 7fd3bd000640 1 --2- 172.21.15.86:0/3452624011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3b806bbf0 0x7fd3b81e5120 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac00c400 tx=0x7fd3ac00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:28.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.484+0000 7fd3b4c00640 1 -- 172.21.15.86:0/3452624011 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac0052d0 con 0x7fd3b806bbf0 2024-04-20T21:22:28.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.484+0000 7fd3b4c00640 1 -- 172.21.15.86:0/3452624011 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3ac005490 con 0x7fd3b806bbf0 2024-04-20T21:22:28.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.484+0000 7fd3b4c00640 1 -- 172.21.15.86:0/3452624011 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac007260 con 0x7fd3b806bbf0 2024-04-20T21:22:28.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.484+0000 7fd3bfa00640 1 -- 172.21.15.86:0/3452624011 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3b814a0a0 con 0x7fd3b806bbf0 2024-04-20T21:22:28.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.485+0000 7fd3bfa00640 1 -- 172.21.15.86:0/3452624011 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3b814a660 con 0x7fd3b806bbf0 2024-04-20T21:22:28.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.486+0000 7fd3bfa00640 1 -- 172.21.15.86:0/3452624011 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3b80689e0 con 0x7fd3b806bbf0 2024-04-20T21:22:28.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.486+0000 7fd3b4c00640 1 -- 172.21.15.86:0/3452624011 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd3ac007400 con 0x7fd3b806bbf0 2024-04-20T21:22:28.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.487+0000 7fd3b4c00640 1 --2- 172.21.15.86:0/3452624011 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd39806c8f0 0x7fd39806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:28.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.488+0000 7fd3b4c00640 1 -- 172.21.15.86:0/3452624011 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fd3ac08b180 con 0x7fd3b806bbf0 2024-04-20T21:22:28.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.488+0000 7fd3b7e00640 1 --2- 172.21.15.86:0/3452624011 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd39806c8f0 0x7fd39806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:28.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.488+0000 7fd3b7e00640 1 --2- 172.21.15.86:0/3452624011 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd39806c8f0 0x7fd39806edb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8145db0 tx=0x7fd3a4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:28.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.492+0000 7fd3b4c00640 1 -- 172.21.15.86:0/3452624011 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd3ac059510 con 0x7fd3b806bbf0 2024-04-20T21:22:28.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.692+0000 7fd3bfa00640 1 -- 172.21.15.86:0/3452624011 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd3b80b16f0 con 0x7fd3b806bbf0 2024-04-20T21:22:28.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.693+0000 7fd3b4c00640 1 -- 172.21.15.86:0/3452624011 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v59) ==== 100+0+11 (secure 0 0 0) 0x7fd3ac058e10 con 0x7fd3b806bbf0 2024-04-20T21:22:28.692 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-20T21:22:28.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.695+0000 7fd3bfa00640 1 -- 172.21.15.86:0/3452624011 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd39806c8f0 msgr2=0x7fd39806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:28.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.695+0000 7fd3bfa00640 1 --2- 172.21.15.86:0/3452624011 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd39806c8f0 0x7fd39806edb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8145db0 tx=0x7fd3a4009040 comp rx=0 tx=0).stop 2024-04-20T21:22:28.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.695+0000 7fd3bfa00640 1 -- 172.21.15.86:0/3452624011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3b806bbf0 msgr2=0x7fd3b81e5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:28.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.695+0000 7fd3bfa00640 1 --2- 172.21.15.86:0/3452624011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3b806bbf0 0x7fd3b81e5120 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac00c400 tx=0x7fd3ac00c8c0 comp rx=0 tx=0).stop 2024-04-20T21:22:28.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.695+0000 7fd3bfa00640 1 -- 172.21.15.86:0/3452624011 shutdown_connections 2024-04-20T21:22:28.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.695+0000 7fd3bfa00640 1 --2- 172.21.15.86:0/3452624011 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd39806c8f0 0x7fd39806edb0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:28.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.695+0000 7fd3bfa00640 1 --2- 172.21.15.86:0/3452624011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3b8146210 0x7fd3b814ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:28.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.695+0000 7fd3bfa00640 1 --2- 172.21.15.86:0/3452624011 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd3b80cdc50 0x7fd3b814ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:28.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.695+0000 7fd3bfa00640 1 --2- 172.21.15.86:0/3452624011 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd3b806bbf0 0x7fd3b81e5120 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:28.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.695+0000 7fd3bfa00640 1 -- 172.21.15.86:0/3452624011 >> 172.21.15.86:0/3452624011 conn(0x7fd3b80b2110 msgr2=0x7fd3b815dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:28.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.696+0000 7fd3bfa00640 1 -- 172.21.15.86:0/3452624011 shutdown_connections 2024-04-20T21:22:28.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.696+0000 7fd3bfa00640 1 -- 172.21.15.86:0/3452624011 wait complete. 2024-04-20T21:22:28.712 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:22:28.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.915+0000 7fde5a400640 1 Processor -- start 2024-04-20T21:22:28.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.915+0000 7fde5a400640 1 -- start start 2024-04-20T21:22:28.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.917+0000 7fde5a400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fde5406f2a0 0x7fde54167410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:28.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.917+0000 7fde5a400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde54160890 con 0x7fde5415d6a0 2024-04-20T21:22:28.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.917+0000 7fde5a400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde54160b70 con 0x7fde54156ae0 2024-04-20T21:22:28.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.917+0000 7fde5a400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fde5406db20 con 0x7fde5406f2a0 2024-04-20T21:22:28.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.917+0000 7fde4be00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fde5415d6a0 0x7fde540ad0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:49596/0 (socket says 172.21.15.86:49596) 2024-04-20T21:22:28.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.917+0000 7fde4be00640 1 -- 172.21.15.86:0/3872008632 learned_addr learned my addr 172.21.15.86:0/3872008632 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:22:28.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.917+0000 7fde59a00640 1 --2- 172.21.15.86:0/3872008632 >> v2:172.21.15.144:3300/0 conn(0x7fde5406f2a0 0x7fde54167410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:28.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.918+0000 7fde53e00640 1 -- 172.21.15.86:0/3872008632 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4136956452 0 0) 0x7fde54160890 con 0x7fde5415d6a0 2024-04-20T21:22:28.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.918+0000 7fde53e00640 1 -- 172.21.15.86:0/3872008632 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde38003520 con 0x7fde5415d6a0 2024-04-20T21:22:28.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.918+0000 7fde53e00640 1 -- 172.21.15.86:0/3872008632 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 475813260 0 0) 0x7fde38003520 con 0x7fde5415d6a0 2024-04-20T21:22:28.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.918+0000 7fde53e00640 1 -- 172.21.15.86:0/3872008632 >> v2:172.21.15.144:3300/0 conn(0x7fde5406f2a0 msgr2=0x7fde54167410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:28.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.918+0000 7fde53e00640 1 --2- 172.21.15.86:0/3872008632 >> v2:172.21.15.144:3300/0 conn(0x7fde5406f2a0 0x7fde54167410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:28.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.918+0000 7fde53e00640 1 -- 172.21.15.86:0/3872008632 >> v1:172.21.15.144:6789/0 conn(0x7fde54156ae0 legacy=0x7fde54160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:28.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.918+0000 7fde53e00640 1 -- 172.21.15.86:0/3872008632 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde54160e50 con 0x7fde5415d6a0 2024-04-20T21:22:28.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.918+0000 7fde53e00640 1 -- 172.21.15.86:0/3872008632 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fde3c001670 con 0x7fde5415d6a0 2024-04-20T21:22:28.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.918+0000 7fde53e00640 1 -- 172.21.15.86:0/3872008632 >> v1:172.21.15.86:6789/0 conn(0x7fde5415d6a0 legacy=0x7fde540ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:28.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.918+0000 7fde53e00640 1 --2- 172.21.15.86:0/3872008632 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde380039b0 0x7fde38003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:28.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.918+0000 7fde53e00640 1 --2- 172.21.15.86:0/3872008632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde38004800 0x7fde38004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:28.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.918+0000 7fde53e00640 1 --2- 172.21.15.86:0/3872008632 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde380051c0 0x7fde3800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:28.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.918+0000 7fde53e00640 1 -- 172.21.15.86:0/3872008632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fde5406db20 con 0x7fde38004800 2024-04-20T21:22:28.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.918+0000 7fde53e00640 1 -- 172.21.15.86:0/3872008632 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fde3800c040 con 0x7fde380039b0 2024-04-20T21:22:28.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.918+0000 7fde53e00640 1 -- 172.21.15.86:0/3872008632 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fde3800c240 con 0x7fde380051c0 2024-04-20T21:22:28.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.918+0000 7fde4be00640 1 --2- 172.21.15.86:0/3872008632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde38004800 0x7fde38004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:28.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.919+0000 7fde59a00640 1 --2- 172.21.15.86:0/3872008632 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde380051c0 0x7fde3800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:28.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.919+0000 7fde59000640 1 --2- 172.21.15.86:0/3872008632 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde380039b0 0x7fde38003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:28.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.919+0000 7fde4be00640 1 -- 172.21.15.86:0/3872008632 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde380051c0 msgr2=0x7fde3800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:28.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.919+0000 7fde4be00640 1 --2- 172.21.15.86:0/3872008632 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde380051c0 0x7fde3800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:28.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.919+0000 7fde4be00640 1 -- 172.21.15.86:0/3872008632 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde380039b0 msgr2=0x7fde38003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:28.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.919+0000 7fde4be00640 1 --2- 172.21.15.86:0/3872008632 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde380039b0 0x7fde38003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:28.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.919+0000 7fde4be00640 1 -- 172.21.15.86:0/3872008632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde38003520 con 0x7fde38004800 2024-04-20T21:22:28.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.919+0000 7fde59000640 1 --2- 172.21.15.86:0/3872008632 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde380039b0 0x7fde38003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:22:28.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.919+0000 7fde59a00640 1 -- 172.21.15.86:0/3872008632 reap_dead start 2024-04-20T21:22:28.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.920+0000 7fde4be00640 1 --2- 172.21.15.86:0/3872008632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde38004800 0x7fde38004c80 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fde3c00c290 tx=0x7fde38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be446b8982d1429a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:28.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.920+0000 7fde53e00640 1 -- 172.21.15.86:0/3872008632 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3c007d50 con 0x7fde38004800 2024-04-20T21:22:28.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.920+0000 7fde53e00640 1 -- 172.21.15.86:0/3872008632 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde3c030070 con 0x7fde38004800 2024-04-20T21:22:28.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.920+0000 7fde53e00640 1 -- 172.21.15.86:0/3872008632 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3c005d70 con 0x7fde38004800 2024-04-20T21:22:28.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.921+0000 7fde5a400640 1 -- 172.21.15.86:0/3872008632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde38004800 msgr2=0x7fde38004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:28.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.921+0000 7fde5a400640 1 --2- 172.21.15.86:0/3872008632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde38004800 0x7fde38004c80 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fde3c00c290 tx=0x7fde38005830 comp rx=0 tx=0).stop 2024-04-20T21:22:28.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.921+0000 7fde5a400640 1 -- 172.21.15.86:0/3872008632 shutdown_connections 2024-04-20T21:22:28.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.921+0000 7fde5a400640 1 --2- 172.21.15.86:0/3872008632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde38004800 0x7fde38004c80 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:28.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.921+0000 7fde5a400640 1 -- 172.21.15.86:0/3872008632 >> 172.21.15.86:0/3872008632 conn(0x7fde540d0170 msgr2=0x7fde540d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:28.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.921+0000 7fde5a400640 1 -- 172.21.15.86:0/3872008632 shutdown_connections 2024-04-20T21:22:28.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.921+0000 7fde5a400640 1 -- 172.21.15.86:0/3872008632 wait complete. 2024-04-20T21:22:28.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.923+0000 7fde5a400640 1 Processor -- start 2024-04-20T21:22:28.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.923+0000 7fde5a400640 1 -- start start 2024-04-20T21:22:28.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.924+0000 7fde5a400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde38004800 0x7fde541f3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:28.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.924+0000 7fde5a400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde541583f0 0x7fde54144600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:28.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.924+0000 7fde5a400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde54158870 0x7fde54144b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:28.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.924+0000 7fde5a400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fde54160080 con 0x7fde541583f0 2024-04-20T21:22:28.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.924+0000 7fde5a400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fde54156ec0 con 0x7fde38004800 2024-04-20T21:22:28.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.924+0000 7fde5a400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fde54160200 con 0x7fde54158870 2024-04-20T21:22:28.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.924+0000 7fde4be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde541583f0 0x7fde54144600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:28.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.924+0000 7fde4be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde541583f0 0x7fde54144600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38850/0 (socket says 172.21.15.86:38850) 2024-04-20T21:22:28.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.924+0000 7fde4be00640 1 -- 172.21.15.86:0/790825082 learned_addr learned my addr 172.21.15.86:0/790825082 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:28.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.924+0000 7fde4be00640 1 -- 172.21.15.86:0/790825082 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde54158870 msgr2=0x7fde54144b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:28.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.924+0000 7fde59000640 1 --2- 172.21.15.86:0/790825082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde38004800 0x7fde541f3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:28.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.924+0000 7fde4be00640 1 --2- 172.21.15.86:0/790825082 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde54158870 0x7fde54144b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:28.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.924+0000 7fde4be00640 1 -- 172.21.15.86:0/790825082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde38004800 msgr2=0x7fde541f3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:28.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.925+0000 7fde4be00640 1 --2- 172.21.15.86:0/790825082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde38004800 0x7fde541f3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:28.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.925+0000 7fde4be00640 1 -- 172.21.15.86:0/790825082 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde54158e40 con 0x7fde541583f0 2024-04-20T21:22:28.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.925+0000 7fde59000640 1 --2- 172.21.15.86:0/790825082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde38004800 0x7fde541f3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:22:28.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.925+0000 7fde4be00640 1 --2- 172.21.15.86:0/790825082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde541583f0 0x7fde54144600 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fde3c02c480 tx=0x7fde3c02e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:28.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.925+0000 7fde51600640 1 -- 172.21.15.86:0/790825082 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3c004bf0 con 0x7fde541583f0 2024-04-20T21:22:28.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.925+0000 7fde51600640 1 -- 172.21.15.86:0/790825082 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde3c004d90 con 0x7fde541583f0 2024-04-20T21:22:28.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.925+0000 7fde51600640 1 -- 172.21.15.86:0/790825082 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3c0072b0 con 0x7fde541583f0 2024-04-20T21:22:28.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.925+0000 7fde5a400640 1 -- 172.21.15.86:0/790825082 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde54159120 con 0x7fde541583f0 2024-04-20T21:22:28.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.925+0000 7fde5a400640 1 -- 172.21.15.86:0/790825082 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde541596e0 con 0x7fde541583f0 2024-04-20T21:22:28.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.927+0000 7fde5a400640 1 -- 172.21.15.86:0/790825082 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde5404fcf0 con 0x7fde541583f0 2024-04-20T21:22:28.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.927+0000 7fde51600640 1 -- 172.21.15.86:0/790825082 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fde3c048020 con 0x7fde541583f0 2024-04-20T21:22:28.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.928+0000 7fde51600640 1 --2- 172.21.15.86:0/790825082 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fde3006c920 0x7fde3006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:28.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.928+0000 7fde51600640 1 -- 172.21.15.86:0/790825082 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7fde3c0ace50 con 0x7fde541583f0 2024-04-20T21:22:28.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.929+0000 7fde59000640 1 --2- 172.21.15.86:0/790825082 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fde3006c920 0x7fde3006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:28.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.929+0000 7fde59000640 1 --2- 172.21.15.86:0/790825082 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fde3006c920 0x7fde3006ede0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fde54069090 tx=0x7fde44009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:28.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:28.934+0000 7fde51600640 1 -- 172.21.15.86:0/790825082 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fde3c02de20 con 0x7fde541583f0 2024-04-20T21:22:29.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.172+0000 7fde5a400640 1 -- 172.21.15.86:0/790825082 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fde540b0230 con 0x7fde541583f0 2024-04-20T21:22:29.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.173+0000 7fde51600640 1 -- 172.21.15.86:0/790825082 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5206 (secure 0 0 0) 0x7fde3c07b2d0 con 0x7fde541583f0 2024-04-20T21:22:29.173 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:22:29.173 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4913,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3933408964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3933408964},{"type":"v1","addr":"172.21.15.144:6833","nonce":3933408964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5221,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/1522070692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1522070692},{"type":"v1","addr":"172.21.15.144:6835","nonce":1522070692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":6132,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/1352073341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1352073341},{"type":"v1","addr":"172.21.15.86:6836","nonce":1352073341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:21:07.502800+0000","modified":"2024-04-20T21:22:00.123314+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6135},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6135":{"gid":6135,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":16,"addr":"172.21.15.86:6837/3008383261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3008383261},{"type":"v1","addr":"172.21.15.86:6837","nonce":3008383261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6135,"qdb_cluster":[6135]},"id":4}]} 2024-04-20T21:22:29.173 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 34 2024-04-20T21:22:29.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.176+0000 7fde5a400640 1 -- 172.21.15.86:0/790825082 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fde3006c920 msgr2=0x7fde3006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:29.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.176+0000 7fde5a400640 1 --2- 172.21.15.86:0/790825082 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fde3006c920 0x7fde3006ede0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fde54069090 tx=0x7fde44009040 comp rx=0 tx=0).stop 2024-04-20T21:22:29.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.176+0000 7fde5a400640 1 -- 172.21.15.86:0/790825082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde541583f0 msgr2=0x7fde54144600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:29.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.176+0000 7fde5a400640 1 --2- 172.21.15.86:0/790825082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde541583f0 0x7fde54144600 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fde3c02c480 tx=0x7fde3c02e070 comp rx=0 tx=0).stop 2024-04-20T21:22:29.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.176+0000 7fde5a400640 1 -- 172.21.15.86:0/790825082 shutdown_connections 2024-04-20T21:22:29.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.176+0000 7fde5a400640 1 --2- 172.21.15.86:0/790825082 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fde3006c920 0x7fde3006ede0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:29.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.176+0000 7fde5a400640 1 --2- 172.21.15.86:0/790825082 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fde54158870 0x7fde54144b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:29.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.176+0000 7fde5a400640 1 --2- 172.21.15.86:0/790825082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde541583f0 0x7fde54144600 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:29.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.176+0000 7fde5a400640 1 --2- 172.21.15.86:0/790825082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde38004800 0x7fde541f3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:29.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.176+0000 7fde5a400640 1 -- 172.21.15.86:0/790825082 >> 172.21.15.86:0/790825082 conn(0x7fde540d0170 msgr2=0x7fde540ad0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:29.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.176+0000 7fde5a400640 1 -- 172.21.15.86:0/790825082 shutdown_connections 2024-04-20T21:22:29.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.176+0000 7fde5a400640 1 -- 172.21.15.86:0/790825082 wait complete. 2024-04-20T21:22:29.195 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:22:29.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.393+0000 7f201a600640 1 Processor -- start 2024-04-20T21:22:29.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.393+0000 7f201a600640 1 -- start start 2024-04-20T21:22:29.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.395+0000 7f201a600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f201409a9e0 0x7f201409ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:29.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.395+0000 7f201a600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f20140b2e60 0x7f20140b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:29.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.395+0000 7f201a600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f201415efd0 con 0x7f201415ebf0 2024-04-20T21:22:29.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.395+0000 7f201a600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f201409b380 con 0x7f20140b2e60 2024-04-20T21:22:29.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.395+0000 7f201a600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f20140b1dc0 con 0x7f201409a9e0 2024-04-20T21:22:29.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.395+0000 7f2013400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f201415ebf0 0x7f20140d12e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:49608/0 (socket says 172.21.15.86:49608) 2024-04-20T21:22:29.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.395+0000 7f2013400640 1 -- 172.21.15.86:0/3527330694 learned_addr learned my addr 172.21.15.86:0/3527330694 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:22:29.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.395+0000 7f2013e00640 1 --2- 172.21.15.86:0/3527330694 >> v2:172.21.15.86:3300/0 conn(0x7f20140b2e60 0x7f20140b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:29.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.396+0000 7f2012a00640 1 --2- 172.21.15.86:0/3527330694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f201409a9e0 0x7f201409ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:29.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.396+0000 7f2012000640 1 -- 172.21.15.86:0/3527330694 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4198248902 0 0) 0x7f201415efd0 con 0x7f201415ebf0 2024-04-20T21:22:29.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.396+0000 7f2012000640 1 -- 172.21.15.86:0/3527330694 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ff4003520 con 0x7f201415ebf0 2024-04-20T21:22:29.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.396+0000 7f2013e00640 1 -- 172.21.15.86:0/3527330694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f201409a9e0 msgr2=0x7f201409ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:29.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.396+0000 7f2013e00640 1 --2- 172.21.15.86:0/3527330694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f201409a9e0 0x7f201409ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:29.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.396+0000 7f2013e00640 1 -- 172.21.15.86:0/3527330694 >> v1:172.21.15.86:6789/0 conn(0x7f201415ebf0 legacy=0x7f20140d12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:29.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.396+0000 7f2013e00640 1 -- 172.21.15.86:0/3527330694 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20140b20a0 con 0x7f20140b2e60 2024-04-20T21:22:29.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.397+0000 7f2013e00640 1 --2- 172.21.15.86:0/3527330694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20140b2e60 0x7f20140b32f0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f200c0097a0 tx=0x7f200c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af3efd2163c01122 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:29.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.397+0000 7f2012000640 1 -- 172.21.15.86:0/3527330694 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c035020 con 0x7f20140b2e60 2024-04-20T21:22:29.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.397+0000 7f2012000640 1 -- 172.21.15.86:0/3527330694 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f200c031810 con 0x7f20140b2e60 2024-04-20T21:22:29.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.397+0000 7f2012000640 1 -- 172.21.15.86:0/3527330694 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c031be0 con 0x7f20140b2e60 2024-04-20T21:22:29.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.398+0000 7f201a600640 1 -- 172.21.15.86:0/3527330694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20140b2e60 msgr2=0x7f20140b32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:29.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.398+0000 7f201a600640 1 --2- 172.21.15.86:0/3527330694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20140b2e60 0x7f20140b32f0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f200c0097a0 tx=0x7f200c02b8b0 comp rx=0 tx=0).stop 2024-04-20T21:22:29.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.398+0000 7f201a600640 1 -- 172.21.15.86:0/3527330694 shutdown_connections 2024-04-20T21:22:29.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.398+0000 7f201a600640 1 --2- 172.21.15.86:0/3527330694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20140b2e60 0x7f20140b32f0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:29.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.398+0000 7f201a600640 1 --2- 172.21.15.86:0/3527330694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f201409a9e0 0x7f201409ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:29.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.398+0000 7f201a600640 1 -- 172.21.15.86:0/3527330694 >> 172.21.15.86:0/3527330694 conn(0x7f20140b3ff0 msgr2=0x7f20140b4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:29.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.398+0000 7f201a600640 1 -- 172.21.15.86:0/3527330694 shutdown_connections 2024-04-20T21:22:29.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.398+0000 7f201a600640 1 -- 172.21.15.86:0/3527330694 wait complete. 2024-04-20T21:22:29.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.400+0000 7f201a600640 1 Processor -- start 2024-04-20T21:22:29.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.400+0000 7f201a600640 1 -- start start 2024-04-20T21:22:29.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.401+0000 7f201a600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f201409a9e0 0x7f20140b7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:29.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.401+0000 7f201a600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20140b2e60 0x7f20141f7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:29.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.401+0000 7f201a600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f201415ebf0 0x7f20140cc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:29.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.401+0000 7f201a600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f20140b2940 con 0x7f201409a9e0 2024-04-20T21:22:29.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.401+0000 7f201a600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f20140b2660 con 0x7f20140b2e60 2024-04-20T21:22:29.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.401+0000 7f201a600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f20140ccb50 con 0x7f201415ebf0 2024-04-20T21:22:29.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.401+0000 7f2013400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f201409a9e0 0x7f20140b7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:29.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.402+0000 7f2013400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f201409a9e0 0x7f20140b7520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38866/0 (socket says 172.21.15.86:38866) 2024-04-20T21:22:29.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.402+0000 7f2013400640 1 -- 172.21.15.86:0/2709016684 learned_addr learned my addr 172.21.15.86:0/2709016684 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:29.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.402+0000 7f2013e00640 1 --2- 172.21.15.86:0/2709016684 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f201415ebf0 0x7f20140cc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:29.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.402+0000 7f2012a00640 1 --2- 172.21.15.86:0/2709016684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20140b2e60 0x7f20141f7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:29.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.402+0000 7f2013400640 1 -- 172.21.15.86:0/2709016684 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f201415ebf0 msgr2=0x7f20140cc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:29.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.402+0000 7f2013400640 1 --2- 172.21.15.86:0/2709016684 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f201415ebf0 0x7f20140cc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:29.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.402+0000 7f2013400640 1 -- 172.21.15.86:0/2709016684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20140b2e60 msgr2=0x7f20141f7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:29.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.402+0000 7f2013400640 1 --2- 172.21.15.86:0/2709016684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20140b2e60 0x7f20141f7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:29.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.402+0000 7f2013400640 1 -- 172.21.15.86:0/2709016684 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20140b64a0 con 0x7f201409a9e0 2024-04-20T21:22:29.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.402+0000 7f2013400640 1 --2- 172.21.15.86:0/2709016684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f201409a9e0 0x7f20140b7520 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f200400c430 tx=0x7f200400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:29.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.403+0000 7f200b400640 1 -- 172.21.15.86:0/2709016684 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2004017070 con 0x7f201409a9e0 2024-04-20T21:22:29.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.403+0000 7f200b400640 1 -- 172.21.15.86:0/2709016684 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2004005ce0 con 0x7f201409a9e0 2024-04-20T21:22:29.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.403+0000 7f201a600640 1 -- 172.21.15.86:0/2709016684 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20140b6780 con 0x7f201409a9e0 2024-04-20T21:22:29.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.403+0000 7f200b400640 1 -- 172.21.15.86:0/2709016684 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20040072f0 con 0x7f201409a9e0 2024-04-20T21:22:29.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.403+0000 7f201a600640 1 -- 172.21.15.86:0/2709016684 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20140b6d40 con 0x7f201409a9e0 2024-04-20T21:22:29.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.404+0000 7f201a600640 1 -- 172.21.15.86:0/2709016684 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20140689e0 con 0x7f201409a9e0 2024-04-20T21:22:29.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.405+0000 7f200b400640 1 -- 172.21.15.86:0/2709016684 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f200401a070 con 0x7f201409a9e0 2024-04-20T21:22:29.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.406+0000 7f200b400640 1 --2- 172.21.15.86:0/2709016684 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1ff006c940 0x7f1ff006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:29.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.406+0000 7f200b400640 1 -- 172.21.15.86:0/2709016684 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f200408af60 con 0x7f201409a9e0 2024-04-20T21:22:29.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.406+0000 7f2012a00640 1 --2- 172.21.15.86:0/2709016684 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1ff006c940 0x7f1ff006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:29.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.407+0000 7f2012a00640 1 --2- 172.21.15.86:0/2709016684 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1ff006c940 0x7f1ff006ee00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2014071cd0 tx=0x7f1ffc003990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:29.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.411+0000 7f200b400640 1 -- 172.21.15.86:0/2709016684 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2004058db0 con 0x7f201409a9e0 2024-04-20T21:22:29.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.614+0000 7f201a600640 1 -- 172.21.15.86:0/2709016684 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2014071310 con 0x7f201409a9e0 2024-04-20T21:22:29.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.617+0000 7f200b400640 1 -- 172.21.15.86:0/2709016684 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+16665 (secure 0 0 0) 0x7f200405d0b0 con 0x7f201409a9e0 2024-04-20T21:22:29.616 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:22:29.617 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":59,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:21:09.072894+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:21:05.248922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-20T21:21:06.479064+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:22:29.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.620+0000 7f201a600640 1 -- 172.21.15.86:0/2709016684 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1ff006c940 msgr2=0x7f1ff006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:29.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.620+0000 7f201a600640 1 --2- 172.21.15.86:0/2709016684 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1ff006c940 0x7f1ff006ee00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2014071cd0 tx=0x7f1ffc003990 comp rx=0 tx=0).stop 2024-04-20T21:22:29.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.620+0000 7f201a600640 1 -- 172.21.15.86:0/2709016684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f201409a9e0 msgr2=0x7f20140b7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:29.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.620+0000 7f201a600640 1 --2- 172.21.15.86:0/2709016684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f201409a9e0 0x7f20140b7520 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f200400c430 tx=0x7f200400c8f0 comp rx=0 tx=0).stop 2024-04-20T21:22:29.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.620+0000 7f201a600640 1 -- 172.21.15.86:0/2709016684 shutdown_connections 2024-04-20T21:22:29.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.620+0000 7f201a600640 1 --2- 172.21.15.86:0/2709016684 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1ff006c940 0x7f1ff006ee00 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:29.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.620+0000 7f201a600640 1 --2- 172.21.15.86:0/2709016684 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f201415ebf0 0x7f20140cc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:29.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.620+0000 7f201a600640 1 --2- 172.21.15.86:0/2709016684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20140b2e60 0x7f20141f7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:29.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.620+0000 7f201a600640 1 --2- 172.21.15.86:0/2709016684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f201409a9e0 0x7f20140b7520 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:29.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.620+0000 7f201a600640 1 -- 172.21.15.86:0/2709016684 >> 172.21.15.86:0/2709016684 conn(0x7f20140b3ff0 msgr2=0x7f201415dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:29.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.620+0000 7f201a600640 1 -- 172.21.15.86:0/2709016684 shutdown_connections 2024-04-20T21:22:29.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.620+0000 7f201a600640 1 -- 172.21.15.86:0/2709016684 wait complete. 2024-04-20T21:22:29.638 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-20T21:22:29.638 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:22:29.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.840+0000 7f84aa200640 1 Processor -- start 2024-04-20T21:22:29.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.840+0000 7f84aa200640 1 -- start start 2024-04-20T21:22:29.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.841+0000 7f84aa200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f84a4144e90 0x7f84a40b4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:29.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.841+0000 7f84aa200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f84a40cd170 0x7f84a406ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:29.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.841+0000 7f84aa200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84a40cd600 0x7f84a406f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:29.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.841+0000 7f84aa200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f84a4145270 con 0x7f84a40cd170 2024-04-20T21:22:29.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.841+0000 7f84aa200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f84a40cdb50 con 0x7f84a4144e90 2024-04-20T21:22:29.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.841+0000 7f84aa200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f84a40b3e50 con 0x7f84a40cd600 2024-04-20T21:22:29.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.842+0000 7f84a2a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f84a40cd170 0x7f84a406ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:29.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.842+0000 7f84a2a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f84a40cd170 0x7f84a406ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38882/0 (socket says 172.21.15.86:38882) 2024-04-20T21:22:29.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.842+0000 7f84a2a00640 1 -- 172.21.15.86:0/1276512025 learned_addr learned my addr 172.21.15.86:0/1276512025 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:29.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.842+0000 7f84a3400640 1 --2- 172.21.15.86:0/1276512025 >> v2:172.21.15.144:3300/0 conn(0x7f84a4144e90 0x7f84a40b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:29.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.842+0000 7f84a3e00640 1 --2- 172.21.15.86:0/1276512025 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84a40cd600 0x7f84a406f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:29.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.842+0000 7f84a2a00640 1 -- 172.21.15.86:0/1276512025 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84a40cd600 msgr2=0x7f84a406f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:29.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.842+0000 7f84a2a00640 1 --2- 172.21.15.86:0/1276512025 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84a40cd600 0x7f84a406f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:29.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.842+0000 7f84a2a00640 1 -- 172.21.15.86:0/1276512025 >> v2:172.21.15.144:3300/0 conn(0x7f84a4144e90 msgr2=0x7f84a40b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:29.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.842+0000 7f84a2a00640 1 --2- 172.21.15.86:0/1276512025 >> v2:172.21.15.144:3300/0 conn(0x7f84a4144e90 0x7f84a40b4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:29.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.842+0000 7f84a2a00640 1 -- 172.21.15.86:0/1276512025 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84a40b4130 con 0x7f84a40cd170 2024-04-20T21:22:29.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.842+0000 7f84a3400640 1 --2- 172.21.15.86:0/1276512025 >> v2:172.21.15.144:3300/0 conn(0x7f84a4144e90 0x7f84a40b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:22:29.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.843+0000 7f84a2a00640 1 --2- 172.21.15.86:0/1276512025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84a40cd170 0x7f84a406ed40 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f84a406f920 tx=0x7f848c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5f76ac5352bbc07 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:29.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.844+0000 7f84a2000640 1 -- 172.21.15.86:0/1276512025 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848c035020 con 0x7f84a40cd170 2024-04-20T21:22:29.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.844+0000 7f84a2000640 1 -- 172.21.15.86:0/1276512025 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f848c031870 con 0x7f84a40cd170 2024-04-20T21:22:29.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.844+0000 7f84a2000640 1 -- 172.21.15.86:0/1276512025 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848c031c40 con 0x7f84a40cd170 2024-04-20T21:22:29.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.844+0000 7f84aa200640 1 -- 172.21.15.86:0/1276512025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84a40cd170 msgr2=0x7f84a406ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:29.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.844+0000 7f84aa200640 1 --2- 172.21.15.86:0/1276512025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84a40cd170 0x7f84a406ed40 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f84a406f920 tx=0x7f848c02b910 comp rx=0 tx=0).stop 2024-04-20T21:22:29.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.844+0000 7f84aa200640 1 -- 172.21.15.86:0/1276512025 shutdown_connections 2024-04-20T21:22:29.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.844+0000 7f84aa200640 1 --2- 172.21.15.86:0/1276512025 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84a40cd600 0x7f84a406f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:29.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.844+0000 7f84aa200640 1 --2- 172.21.15.86:0/1276512025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84a40cd170 0x7f84a406ed40 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:29.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.844+0000 7f84aa200640 1 --2- 172.21.15.86:0/1276512025 >> v2:172.21.15.144:3300/0 conn(0x7f84a4144e90 0x7f84a40b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:29.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.844+0000 7f84aa200640 1 -- 172.21.15.86:0/1276512025 >> 172.21.15.86:0/1276512025 conn(0x7f84a40b2360 msgr2=0x7f84a40b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:29.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.844+0000 7f84aa200640 1 -- 172.21.15.86:0/1276512025 shutdown_connections 2024-04-20T21:22:29.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.845+0000 7f84aa200640 1 -- 172.21.15.86:0/1276512025 wait complete. 2024-04-20T21:22:29.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.846+0000 7f84aa200640 1 Processor -- start 2024-04-20T21:22:29.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.846+0000 7f84aa200640 1 -- start start 2024-04-20T21:22:29.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.847+0000 7f84aa200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84a40cd170 0x7f84a4142060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:29.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.847+0000 7f84aa200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84a40cd600 0x7f84a40aee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:29.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.847+0000 7f84aa200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84a4144e90 0x7f84a40c60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:29.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.847+0000 7f84aa200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f84a40d0010 con 0x7f84a40cd600 2024-04-20T21:22:29.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.847+0000 7f84aa200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f84a40b4e70 con 0x7f84a40cd170 2024-04-20T21:22:29.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.847+0000 7f84aa200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f84a40d0190 con 0x7f84a4144e90 2024-04-20T21:22:29.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.848+0000 7f84a2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84a40cd600 0x7f84a40aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:29.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.848+0000 7f84a2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84a40cd600 0x7f84a40aee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38894/0 (socket says 172.21.15.86:38894) 2024-04-20T21:22:29.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.848+0000 7f84a2a00640 1 -- 172.21.15.86:0/291008551 learned_addr learned my addr 172.21.15.86:0/291008551 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:29.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.848+0000 7f84a3400640 1 --2- 172.21.15.86:0/291008551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84a40cd170 0x7f84a4142060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:29.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.848+0000 7f84a3e00640 1 --2- 172.21.15.86:0/291008551 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84a4144e90 0x7f84a40c60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:29.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.848+0000 7f84a2a00640 1 -- 172.21.15.86:0/291008551 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84a4144e90 msgr2=0x7f84a40c60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:29.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.848+0000 7f84a2a00640 1 --2- 172.21.15.86:0/291008551 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84a4144e90 0x7f84a40c60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:29.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.848+0000 7f84a2a00640 1 -- 172.21.15.86:0/291008551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84a40cd170 msgr2=0x7f84a4142060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:29.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.848+0000 7f84a2a00640 1 --2- 172.21.15.86:0/291008551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84a40cd170 0x7f84a4142060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:29.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.848+0000 7f84a2a00640 1 -- 172.21.15.86:0/291008551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84a41e5940 con 0x7f84a40cd600 2024-04-20T21:22:29.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.849+0000 7f84a2a00640 1 --2- 172.21.15.86:0/291008551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84a40cd600 0x7f84a40aee30 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f848c02b950 tx=0x7f848c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:29.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.849+0000 7f849b400640 1 -- 172.21.15.86:0/291008551 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848c040070 con 0x7f84a40cd600 2024-04-20T21:22:29.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.849+0000 7f849b400640 1 -- 172.21.15.86:0/291008551 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f848c030d00 con 0x7f84a40cd600 2024-04-20T21:22:29.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.849+0000 7f849b400640 1 -- 172.21.15.86:0/291008551 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848c035040 con 0x7f84a40cd600 2024-04-20T21:22:29.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.849+0000 7f84aa200640 1 -- 172.21.15.86:0/291008551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84a41e5c20 con 0x7f84a40cd600 2024-04-20T21:22:29.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.849+0000 7f84aa200640 1 -- 172.21.15.86:0/291008551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84a41e61e0 con 0x7f84a40cd600 2024-04-20T21:22:29.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.850+0000 7f84aa200640 1 -- 172.21.15.86:0/291008551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84a40689e0 con 0x7f84a40cd600 2024-04-20T21:22:29.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.851+0000 7f849b400640 1 -- 172.21.15.86:0/291008551 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f848c04b060 con 0x7f84a40cd600 2024-04-20T21:22:29.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.852+0000 7f849b400640 1 --2- 172.21.15.86:0/291008551 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f848006c920 0x7f848006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:29.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.852+0000 7f849b400640 1 -- 172.21.15.86:0/291008551 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f848c0ad950 con 0x7f84a40cd600 2024-04-20T21:22:29.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.852+0000 7f84a3400640 1 --2- 172.21.15.86:0/291008551 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f848006c920 0x7f848006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:29.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.853+0000 7f84a3400640 1 --2- 172.21.15.86:0/291008551 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f848006c920 0x7f848006ede0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8494008850 tx=0x7f8494008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:29.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:29.856+0000 7f849b400640 1 -- 172.21.15.86:0/291008551 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f848c07bcf0 con 0x7f84a40cd600 2024-04-20T21:22:30.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.096+0000 7f84aa200640 1 -- 172.21.15.86:0/291008551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84a4070630 con 0x7f84a40cd600 2024-04-20T21:22:30.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.098+0000 7f849b400640 1 -- 172.21.15.86:0/291008551 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5206 (secure 0 0 0) 0x7f848c07b5f0 con 0x7f84a40cd600 2024-04-20T21:22:30.097 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:22:30.097 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4913,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3933408964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3933408964},{"type":"v1","addr":"172.21.15.144:6833","nonce":3933408964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5221,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/1522070692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1522070692},{"type":"v1","addr":"172.21.15.144:6835","nonce":1522070692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":6132,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/1352073341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1352073341},{"type":"v1","addr":"172.21.15.86:6836","nonce":1352073341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:21:07.502800+0000","modified":"2024-04-20T21:22:00.123314+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6135},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6135":{"gid":6135,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":16,"addr":"172.21.15.86:6837/3008383261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3008383261},{"type":"v1","addr":"172.21.15.86:6837","nonce":3008383261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6135,"qdb_cluster":[6135]},"id":4}]} 2024-04-20T21:22:30.097 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 34 2024-04-20T21:22:30.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.100+0000 7f84aa200640 1 -- 172.21.15.86:0/291008551 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f848006c920 msgr2=0x7f848006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:30.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.100+0000 7f84aa200640 1 --2- 172.21.15.86:0/291008551 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f848006c920 0x7f848006ede0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8494008850 tx=0x7f8494008040 comp rx=0 tx=0).stop 2024-04-20T21:22:30.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.100+0000 7f84aa200640 1 -- 172.21.15.86:0/291008551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84a40cd600 msgr2=0x7f84a40aee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:30.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.100+0000 7f84aa200640 1 --2- 172.21.15.86:0/291008551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84a40cd600 0x7f84a40aee30 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f848c02b950 tx=0x7f848c0046c0 comp rx=0 tx=0).stop 2024-04-20T21:22:30.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.101+0000 7f84aa200640 1 -- 172.21.15.86:0/291008551 shutdown_connections 2024-04-20T21:22:30.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.101+0000 7f84aa200640 1 --2- 172.21.15.86:0/291008551 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f848006c920 0x7f848006ede0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:30.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.101+0000 7f84aa200640 1 --2- 172.21.15.86:0/291008551 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84a4144e90 0x7f84a40c60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:30.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.101+0000 7f84aa200640 1 --2- 172.21.15.86:0/291008551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84a40cd600 0x7f84a40aee30 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:30.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.101+0000 7f84aa200640 1 --2- 172.21.15.86:0/291008551 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84a40cd170 0x7f84a4142060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:30.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.101+0000 7f84aa200640 1 -- 172.21.15.86:0/291008551 >> 172.21.15.86:0/291008551 conn(0x7f84a40b2360 msgr2=0x7f84a4072590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:30.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.101+0000 7f84aa200640 1 -- 172.21.15.86:0/291008551 shutdown_connections 2024-04-20T21:22:30.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.101+0000 7f84aa200640 1 -- 172.21.15.86:0/291008551 wait complete. 2024-04-20T21:22:30.120 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:22:30.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.316+0000 7f656ce00640 1 Processor -- start 2024-04-20T21:22:30.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.316+0000 7f656ce00640 1 -- start start 2024-04-20T21:22:30.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.318+0000 7f656ce00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f65680cfb20 0x7f65680d1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:30.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.318+0000 7f656ce00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65680cff70 con 0x7f6568162000 2024-04-20T21:22:30.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.318+0000 7f656ce00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65680d01e0 con 0x7f65680d11d0 2024-04-20T21:22:30.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.318+0000 7f656ce00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f65680d1b40 con 0x7f65680cfb20 2024-04-20T21:22:30.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.318+0000 7f6565e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f6568162000 0x7f65680b76c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:49618/0 (socket says 172.21.15.86:49618) 2024-04-20T21:22:30.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.318+0000 7f6565e00640 1 -- 172.21.15.86:0/1717486319 learned_addr learned my addr 172.21.15.86:0/1717486319 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:22:30.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.319+0000 7f6566800640 1 --2- 172.21.15.86:0/1717486319 >> v2:172.21.15.144:3300/0 conn(0x7f65680cfb20 0x7f65680d1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:30.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.319+0000 7f6565400640 1 -- 172.21.15.86:0/1717486319 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2949003615 0 0) 0x7f65680cff70 con 0x7f6568162000 2024-04-20T21:22:30.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.319+0000 7f6565400640 1 -- 172.21.15.86:0/1717486319 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f654c003520 con 0x7f6568162000 2024-04-20T21:22:30.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.319+0000 7f6565400640 1 -- 172.21.15.86:0/1717486319 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1084550186 0 0) 0x7f654c003520 con 0x7f6568162000 2024-04-20T21:22:30.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.319+0000 7f6565400640 1 -- 172.21.15.86:0/1717486319 >> v2:172.21.15.144:3300/0 conn(0x7f65680cfb20 msgr2=0x7f65680d1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:30.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.319+0000 7f6565400640 1 --2- 172.21.15.86:0/1717486319 >> v2:172.21.15.144:3300/0 conn(0x7f65680cfb20 0x7f65680d1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:30.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.319+0000 7f6565400640 1 -- 172.21.15.86:0/1717486319 >> v1:172.21.15.144:6789/0 conn(0x7f65680d11d0 legacy=0x7f65680cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:30.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.319+0000 7f6565400640 1 -- 172.21.15.86:0/1717486319 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65680d04a0 con 0x7f6568162000 2024-04-20T21:22:30.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.319+0000 7f6565400640 1 -- 172.21.15.86:0/1717486319 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f6558001670 con 0x7f6568162000 2024-04-20T21:22:30.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.319+0000 7f6565400640 1 -- 172.21.15.86:0/1717486319 >> v1:172.21.15.86:6789/0 conn(0x7f6568162000 legacy=0x7f65680b76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:30.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.319+0000 7f6565400640 1 --2- 172.21.15.86:0/1717486319 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f654c0039b0 0x7f654c003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:30.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.319+0000 7f6565400640 1 --2- 172.21.15.86:0/1717486319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f654c004800 0x7f654c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:30.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.319+0000 7f6565400640 1 --2- 172.21.15.86:0/1717486319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f654c0052e0 0x7f654c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:30.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.319+0000 7f6565400640 1 -- 172.21.15.86:0/1717486319 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f65680d1b40 con 0x7f654c0052e0 2024-04-20T21:22:30.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.319+0000 7f6565400640 1 -- 172.21.15.86:0/1717486319 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f654c00c230 con 0x7f654c004800 2024-04-20T21:22:30.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.319+0000 7f6565400640 1 -- 172.21.15.86:0/1717486319 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f654c00c430 con 0x7f654c0039b0 2024-04-20T21:22:30.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.320+0000 7f6566800640 1 --2- 172.21.15.86:0/1717486319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f654c0052e0 0x7f654c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:30.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.320+0000 7f6565e00640 1 --2- 172.21.15.86:0/1717486319 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f654c0039b0 0x7f654c003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:30.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.320+0000 7f655d400640 1 --2- 172.21.15.86:0/1717486319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f654c004800 0x7f654c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:30.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.320+0000 7f6566800640 1 -- 172.21.15.86:0/1717486319 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f654c0039b0 msgr2=0x7f654c003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:30.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.320+0000 7f6566800640 1 --2- 172.21.15.86:0/1717486319 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f654c0039b0 0x7f654c003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:30.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.320+0000 7f6566800640 1 -- 172.21.15.86:0/1717486319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f654c004800 msgr2=0x7f654c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:30.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.320+0000 7f6566800640 1 --2- 172.21.15.86:0/1717486319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f654c004800 0x7f654c004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:30.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.320+0000 7f6566800640 1 -- 172.21.15.86:0/1717486319 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f654c003520 con 0x7f654c0052e0 2024-04-20T21:22:30.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.321+0000 7f6566800640 1 -- 172.21.15.86:0/1717486319 reap_dead start 2024-04-20T21:22:30.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.321+0000 7f6566800640 1 --2- 172.21.15.86:0/1717486319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f654c0052e0 0x7f654c00b9c0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f6560007330 tx=0x7f654c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a793b8da084db619 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:30.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.321+0000 7f6565400640 1 -- 172.21.15.86:0/1717486319 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6560009be0 con 0x7f654c0052e0 2024-04-20T21:22:30.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.321+0000 7f6565400640 1 -- 172.21.15.86:0/1717486319 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f656002ccd0 con 0x7f654c0052e0 2024-04-20T21:22:30.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.321+0000 7f6565400640 1 -- 172.21.15.86:0/1717486319 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6560003bd0 con 0x7f654c0052e0 2024-04-20T21:22:30.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.322+0000 7f656ce00640 1 -- 172.21.15.86:0/1717486319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f654c0052e0 msgr2=0x7f654c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:30.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.322+0000 7f656ce00640 1 --2- 172.21.15.86:0/1717486319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f654c0052e0 0x7f654c00b9c0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f6560007330 tx=0x7f654c005830 comp rx=0 tx=0).stop 2024-04-20T21:22:30.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.322+0000 7f656ce00640 1 -- 172.21.15.86:0/1717486319 shutdown_connections 2024-04-20T21:22:30.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.322+0000 7f656ce00640 1 --2- 172.21.15.86:0/1717486319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f654c0052e0 0x7f654c00b9c0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:30.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.322+0000 7f656ce00640 1 -- 172.21.15.86:0/1717486319 >> 172.21.15.86:0/1717486319 conn(0x7f656808bf70 msgr2=0x7f65680d4870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:30.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.322+0000 7f656ce00640 1 -- 172.21.15.86:0/1717486319 shutdown_connections 2024-04-20T21:22:30.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.322+0000 7f656ce00640 1 -- 172.21.15.86:0/1717486319 wait complete. 2024-04-20T21:22:30.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.324+0000 7f656ce00640 1 Processor -- start 2024-04-20T21:22:30.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.324+0000 7f656ce00640 1 -- start start 2024-04-20T21:22:30.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.325+0000 7f656ce00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f654c0052e0 0x7f65681527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:30.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.325+0000 7f656ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65680b84b0 0x7f65680b8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:30.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.325+0000 7f656ce00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6568156a90 0x7f6568156f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:30.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.325+0000 7f656ce00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f65681623e0 con 0x7f65680b84b0 2024-04-20T21:22:30.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.325+0000 7f656ce00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f65680d2440 con 0x7f654c0052e0 2024-04-20T21:22:30.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.325+0000 7f656ce00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6568162560 con 0x7f6568156a90 2024-04-20T21:22:30.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.325+0000 7f655d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65680b84b0 0x7f65680b8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:30.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.325+0000 7f655d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65680b84b0 0x7f65680b8910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38902/0 (socket says 172.21.15.86:38902) 2024-04-20T21:22:30.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.325+0000 7f655d400640 1 -- 172.21.15.86:0/4128206552 learned_addr learned my addr 172.21.15.86:0/4128206552 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:30.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.326+0000 7f6565e00640 1 --2- 172.21.15.86:0/4128206552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f654c0052e0 0x7f65681527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:30.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.326+0000 7f6566800640 1 --2- 172.21.15.86:0/4128206552 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6568156a90 0x7f6568156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:30.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.326+0000 7f655d400640 1 -- 172.21.15.86:0/4128206552 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6568156a90 msgr2=0x7f6568156f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:30.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.326+0000 7f655d400640 1 --2- 172.21.15.86:0/4128206552 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6568156a90 0x7f6568156f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:30.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.326+0000 7f655d400640 1 -- 172.21.15.86:0/4128206552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f654c0052e0 msgr2=0x7f65681527f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:30.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.326+0000 7f655d400640 1 --2- 172.21.15.86:0/4128206552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f654c0052e0 0x7f65681527f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:30.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.326+0000 7f655d400640 1 -- 172.21.15.86:0/4128206552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65681f7a80 con 0x7f65680b84b0 2024-04-20T21:22:30.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.326+0000 7f655d400640 1 --2- 172.21.15.86:0/4128206552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65680b84b0 0x7f65680b8910 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f655000d510 tx=0x7f655000d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:30.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.326+0000 7f655ea00640 1 -- 172.21.15.86:0/4128206552 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6550016070 con 0x7f65680b84b0 2024-04-20T21:22:30.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.326+0000 7f655ea00640 1 -- 172.21.15.86:0/4128206552 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6550012800 con 0x7f65680b84b0 2024-04-20T21:22:30.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.326+0000 7f655ea00640 1 -- 172.21.15.86:0/4128206552 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6550012bf0 con 0x7f65680b84b0 2024-04-20T21:22:30.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.326+0000 7f656ce00640 1 -- 172.21.15.86:0/4128206552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65681f7d60 con 0x7f65680b84b0 2024-04-20T21:22:30.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.326+0000 7f656ce00640 1 -- 172.21.15.86:0/4128206552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65681f8320 con 0x7f65680b84b0 2024-04-20T21:22:30.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.328+0000 7f656ce00640 1 -- 172.21.15.86:0/4128206552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65680689e0 con 0x7f65680b84b0 2024-04-20T21:22:30.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.328+0000 7f655ea00640 1 -- 172.21.15.86:0/4128206552 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6550019070 con 0x7f65680b84b0 2024-04-20T21:22:30.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.329+0000 7f655ea00640 1 --2- 172.21.15.86:0/4128206552 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f654406c920 0x7f654406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:30.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.330+0000 7f655ea00640 1 -- 172.21.15.86:0/4128206552 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f655008b830 con 0x7f65680b84b0 2024-04-20T21:22:30.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.330+0000 7f6565e00640 1 --2- 172.21.15.86:0/4128206552 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f654406c920 0x7f654406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:30.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.330+0000 7f6565e00640 1 --2- 172.21.15.86:0/4128206552 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f654406c920 0x7f654406ede0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6558001200 tx=0x7f6558009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:30.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.334+0000 7f655ea00640 1 -- 172.21.15.86:0/4128206552 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6550059cd0 con 0x7f65680b84b0 2024-04-20T21:22:30.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.575+0000 7f656ce00640 1 -- 172.21.15.86:0/4128206552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f65680d01e0 con 0x7f65680b84b0 2024-04-20T21:22:30.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.576+0000 7f655ea00640 1 -- 172.21.15.86:0/4128206552 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5206 (secure 0 0 0) 0x7f65500595d0 con 0x7f65680b84b0 2024-04-20T21:22:30.575 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:22:30.575 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4913,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3933408964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3933408964},{"type":"v1","addr":"172.21.15.144:6833","nonce":3933408964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5221,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/1522070692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1522070692},{"type":"v1","addr":"172.21.15.144:6835","nonce":1522070692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":6132,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/1352073341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1352073341},{"type":"v1","addr":"172.21.15.86:6836","nonce":1352073341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:21:07.502800+0000","modified":"2024-04-20T21:22:00.123314+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6135},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6135":{"gid":6135,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":16,"addr":"172.21.15.86:6837/3008383261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":3008383261},{"type":"v1","addr":"172.21.15.86:6837","nonce":3008383261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6135,"qdb_cluster":[6135]},"id":4}]} 2024-04-20T21:22:30.575 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 34 2024-04-20T21:22:30.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.578+0000 7f656ce00640 1 -- 172.21.15.86:0/4128206552 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f654406c920 msgr2=0x7f654406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:30.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.578+0000 7f656ce00640 1 --2- 172.21.15.86:0/4128206552 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f654406c920 0x7f654406ede0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6558001200 tx=0x7f6558009040 comp rx=0 tx=0).stop 2024-04-20T21:22:30.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.578+0000 7f656ce00640 1 -- 172.21.15.86:0/4128206552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65680b84b0 msgr2=0x7f65680b8910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:30.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.578+0000 7f656ce00640 1 --2- 172.21.15.86:0/4128206552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65680b84b0 0x7f65680b8910 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f655000d510 tx=0x7f655000d9d0 comp rx=0 tx=0).stop 2024-04-20T21:22:30.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.579+0000 7f656ce00640 1 -- 172.21.15.86:0/4128206552 shutdown_connections 2024-04-20T21:22:30.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.579+0000 7f656ce00640 1 --2- 172.21.15.86:0/4128206552 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f654406c920 0x7f654406ede0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:30.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.579+0000 7f656ce00640 1 --2- 172.21.15.86:0/4128206552 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6568156a90 0x7f6568156f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:30.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.579+0000 7f656ce00640 1 --2- 172.21.15.86:0/4128206552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65680b84b0 0x7f65680b8910 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:30.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.579+0000 7f656ce00640 1 --2- 172.21.15.86:0/4128206552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f654c0052e0 0x7f65681527f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:30.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.579+0000 7f656ce00640 1 -- 172.21.15.86:0/4128206552 >> 172.21.15.86:0/4128206552 conn(0x7f656808bf70 msgr2=0x7f656815fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:30.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.579+0000 7f656ce00640 1 -- 172.21.15.86:0/4128206552 shutdown_connections 2024-04-20T21:22:30.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.579+0000 7f656ce00640 1 -- 172.21.15.86:0/4128206552 wait complete. 2024-04-20T21:22:30.597 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:22:30.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.789+0000 7f77ed200640 1 Processor -- start 2024-04-20T21:22:30.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.789+0000 7f77ed200640 1 -- start start 2024-04-20T21:22:30.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.791+0000 7f77ed200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f77e8144e90 0x7f77e815ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:30.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.791+0000 7f77ed200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f77e806f140 0x7f77e80aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:30.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.791+0000 7f77ed200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77e806dc30 con 0x7f77e80af0b0 2024-04-20T21:22:30.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.791+0000 7f77ed200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f77e8145270 con 0x7f77e806f140 2024-04-20T21:22:30.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.791+0000 7f77ed200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f77e815f720 con 0x7f77e8144e90 2024-04-20T21:22:30.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.791+0000 7f77e6800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f77e80af0b0 0x7f77e8071d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:49624/0 (socket says 172.21.15.86:49624) 2024-04-20T21:22:30.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.791+0000 7f77e6800640 1 -- 172.21.15.86:0/2514438250 learned_addr learned my addr 172.21.15.86:0/2514438250 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:22:30.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.791+0000 7f77e5400640 1 --2- 172.21.15.86:0/2514438250 >> v2:172.21.15.144:3300/0 conn(0x7f77e806f140 0x7f77e80aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:30.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.791+0000 7f77e5e00640 1 --2- 172.21.15.86:0/2514438250 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f77e8144e90 0x7f77e815ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:30.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.792+0000 7f77e4a00640 1 -- 172.21.15.86:0/2514438250 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3425106059 0 0) 0x7f77e806dc30 con 0x7f77e80af0b0 2024-04-20T21:22:30.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.792+0000 7f77e4a00640 1 -- 172.21.15.86:0/2514438250 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f77cc003520 con 0x7f77e80af0b0 2024-04-20T21:22:30.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.792+0000 7f77e4a00640 1 -- 172.21.15.86:0/2514438250 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1679962405 0 0) 0x7f77cc003520 con 0x7f77e80af0b0 2024-04-20T21:22:30.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.792+0000 7f77e4a00640 1 -- 172.21.15.86:0/2514438250 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f77e8144e90 msgr2=0x7f77e815ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:30.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.792+0000 7f77e4a00640 1 --2- 172.21.15.86:0/2514438250 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f77e8144e90 0x7f77e815ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:30.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.792+0000 7f77e4a00640 1 -- 172.21.15.86:0/2514438250 >> v2:172.21.15.144:3300/0 conn(0x7f77e806f140 msgr2=0x7f77e80aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:30.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.792+0000 7f77e4a00640 1 --2- 172.21.15.86:0/2514438250 >> v2:172.21.15.144:3300/0 conn(0x7f77e806f140 0x7f77e80aeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:30.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.792+0000 7f77e4a00640 1 -- 172.21.15.86:0/2514438250 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77e815fa00 con 0x7f77e80af0b0 2024-04-20T21:22:30.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.792+0000 7f77e4a00640 1 -- 172.21.15.86:0/2514438250 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f77e00019f0 con 0x7f77e80af0b0 2024-04-20T21:22:30.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.792+0000 7f77e4a00640 1 -- 172.21.15.86:0/2514438250 >> v1:172.21.15.86:6789/0 conn(0x7f77e80af0b0 legacy=0x7f77e8071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:30.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.792+0000 7f77e4a00640 1 --2- 172.21.15.86:0/2514438250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77cc0039b0 0x7f77cc003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:30.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.792+0000 7f77e4a00640 1 --2- 172.21.15.86:0/2514438250 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f77e8144e90 0x7f77cc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:30.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.792+0000 7f77e4a00640 1 --2- 172.21.15.86:0/2514438250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77cc004d40 0x7f77cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:30.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.792+0000 7f77e4a00640 1 -- 172.21.15.86:0/2514438250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f77e8145270 con 0x7f77cc004d40 2024-04-20T21:22:30.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.792+0000 7f77e4a00640 1 -- 172.21.15.86:0/2514438250 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f77e815f720 con 0x7f77cc0039b0 2024-04-20T21:22:30.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.792+0000 7f77e4a00640 1 -- 172.21.15.86:0/2514438250 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f77cc00b9c0 con 0x7f77e8144e90 2024-04-20T21:22:30.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.792+0000 7f77e6800640 1 --2- 172.21.15.86:0/2514438250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77cc004d40 0x7f77cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:30.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.793+0000 7f77e5e00640 1 --2- 172.21.15.86:0/2514438250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77cc0039b0 0x7f77cc003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:30.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.793+0000 7f77e5400640 1 --2- 172.21.15.86:0/2514438250 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f77e8144e90 0x7f77cc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:30.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.793+0000 7f77e6800640 1 -- 172.21.15.86:0/2514438250 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f77e8144e90 msgr2=0x7f77cc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:30.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.793+0000 7f77e6800640 1 --2- 172.21.15.86:0/2514438250 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f77e8144e90 0x7f77cc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:30.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.793+0000 7f77e6800640 1 -- 172.21.15.86:0/2514438250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77cc0039b0 msgr2=0x7f77cc003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:30.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.793+0000 7f77e6800640 1 --2- 172.21.15.86:0/2514438250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77cc0039b0 0x7f77cc003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:30.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.793+0000 7f77e6800640 1 -- 172.21.15.86:0/2514438250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77cc003520 con 0x7f77cc004d40 2024-04-20T21:22:30.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.794+0000 7f77e6800640 1 --2- 172.21.15.86:0/2514438250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77cc004d40 0x7f77cc0051c0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f77e000a7a0 tx=0x7f77cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=133253fd4d87551f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:30.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.795+0000 7f77e4a00640 1 -- 172.21.15.86:0/2514438250 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e000a3f0 con 0x7f77cc004d40 2024-04-20T21:22:30.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.795+0000 7f77e4a00640 1 -- 172.21.15.86:0/2514438250 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77e0034630 con 0x7f77cc004d40 2024-04-20T21:22:30.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.795+0000 7f77e4a00640 1 -- 172.21.15.86:0/2514438250 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e0034a40 con 0x7f77cc004d40 2024-04-20T21:22:30.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.795+0000 7f77ed200640 1 -- 172.21.15.86:0/2514438250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77cc004d40 msgr2=0x7f77cc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:30.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.795+0000 7f77ed200640 1 --2- 172.21.15.86:0/2514438250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77cc004d40 0x7f77cc0051c0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f77e000a7a0 tx=0x7f77cc005830 comp rx=0 tx=0).stop 2024-04-20T21:22:30.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.795+0000 7f77e6800640 1 -- 172.21.15.86:0/2514438250 reap_dead start 2024-04-20T21:22:30.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.795+0000 7f77ed200640 1 -- 172.21.15.86:0/2514438250 shutdown_connections 2024-04-20T21:22:30.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.795+0000 7f77ed200640 1 -- 172.21.15.86:0/2514438250 >> 172.21.15.86:0/2514438250 conn(0x7f77e80b2360 msgr2=0x7f77e80b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:30.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.795+0000 7f77ed200640 1 -- 172.21.15.86:0/2514438250 shutdown_connections 2024-04-20T21:22:30.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.796+0000 7f77ed200640 1 -- 172.21.15.86:0/2514438250 wait complete. 2024-04-20T21:22:30.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.797+0000 7f77ed200640 1 Processor -- start 2024-04-20T21:22:30.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.797+0000 7f77ed200640 1 -- start start 2024-04-20T21:22:30.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.798+0000 7f77ed200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77e8133f80 0x7f77e8131c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:30.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.798+0000 7f77ed200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f77e8134360 0x7f77e81321c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:30.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.798+0000 7f77ed200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77e81373e0 0x7f77e8137860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:30.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.798+0000 7f77ed200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f77e8072410 con 0x7f77e8133f80 2024-04-20T21:22:30.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.798+0000 7f77ed200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f77e806ded0 con 0x7f77e81373e0 2024-04-20T21:22:30.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.798+0000 7f77ed200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f77e8072590 con 0x7f77e8134360 2024-04-20T21:22:30.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.798+0000 7f77e5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77e8133f80 0x7f77e8131c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:30.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.799+0000 7f77e5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77e8133f80 0x7f77e8131c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38928/0 (socket says 172.21.15.86:38928) 2024-04-20T21:22:30.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.799+0000 7f77e5e00640 1 -- 172.21.15.86:0/3739903452 learned_addr learned my addr 172.21.15.86:0/3739903452 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:30.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.799+0000 7f77e5e00640 1 -- 172.21.15.86:0/3739903452 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f77e8134360 msgr2=0x7f77e81321c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:30.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.799+0000 7f77e5e00640 1 --2- 172.21.15.86:0/3739903452 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f77e8134360 0x7f77e81321c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:30.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.799+0000 7f77e5e00640 1 -- 172.21.15.86:0/3739903452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77e81373e0 msgr2=0x7f77e8137860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:30.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.799+0000 7f77e5e00640 1 --2- 172.21.15.86:0/3739903452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77e81373e0 0x7f77e8137860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:30.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.799+0000 7f77e5e00640 1 -- 172.21.15.86:0/3739903452 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77e8132700 con 0x7f77e8133f80 2024-04-20T21:22:30.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.799+0000 7f77e5e00640 1 --2- 172.21.15.86:0/3739903452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77e8133f80 0x7f77e8131c80 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f77d8011570 tx=0x7f77d8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:30.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.799+0000 7f77de000640 1 -- 172.21.15.86:0/3739903452 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d8015930 con 0x7f77e8133f80 2024-04-20T21:22:30.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.799+0000 7f77de000640 1 -- 172.21.15.86:0/3739903452 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77d8015ad0 con 0x7f77e8133f80 2024-04-20T21:22:30.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.799+0000 7f77ed200640 1 -- 172.21.15.86:0/3739903452 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77e8132990 con 0x7f77e8133f80 2024-04-20T21:22:30.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.799+0000 7f77de000640 1 -- 172.21.15.86:0/3739903452 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d801d400 con 0x7f77e8133f80 2024-04-20T21:22:30.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.800+0000 7f77ed200640 1 -- 172.21.15.86:0/3739903452 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77e8132f50 con 0x7f77e8133f80 2024-04-20T21:22:30.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.801+0000 7f77ed200640 1 -- 172.21.15.86:0/3739903452 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77e80689e0 con 0x7f77e8133f80 2024-04-20T21:22:30.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.802+0000 7f77de000640 1 -- 172.21.15.86:0/3739903452 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f77d801d5a0 con 0x7f77e8133f80 2024-04-20T21:22:30.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.803+0000 7f77de000640 1 --2- 172.21.15.86:0/3739903452 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f77c806c8f0 0x7f77c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:30.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.803+0000 7f77de000640 1 -- 172.21.15.86:0/3739903452 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f77d808c3b0 con 0x7f77e8133f80 2024-04-20T21:22:30.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.803+0000 7f77e5400640 1 --2- 172.21.15.86:0/3739903452 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f77c806c8f0 0x7f77c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:30.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.803+0000 7f77e5400640 1 --2- 172.21.15.86:0/3739903452 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f77c806c8f0 0x7f77c806edb0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f77e80ccc10 tx=0x7f77d4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:30.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:30.807+0000 7f77de000640 1 -- 172.21.15.86:0/3739903452 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f77d8019030 con 0x7f77e8133f80 2024-04-20T21:22:31.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.005+0000 7f77ed200640 1 -- 172.21.15.86:0/3739903452 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f77e815f400 con 0x7f77e8133f80 2024-04-20T21:22:31.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.006+0000 7f77de000640 1 -- 172.21.15.86:0/3739903452 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+16665 (secure 0 0 0) 0x7f77d805a7d0 con 0x7f77e8133f80 2024-04-20T21:22:31.006 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:22:31.006 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":59,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:21:09.072894+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:21:05.248922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-20T21:21:06.479064+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:22:31.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.009+0000 7f77ed200640 1 -- 172.21.15.86:0/3739903452 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f77c806c8f0 msgr2=0x7f77c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:31.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.009+0000 7f77ed200640 1 --2- 172.21.15.86:0/3739903452 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f77c806c8f0 0x7f77c806edb0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f77e80ccc10 tx=0x7f77d4008040 comp rx=0 tx=0).stop 2024-04-20T21:22:31.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.009+0000 7f77ed200640 1 -- 172.21.15.86:0/3739903452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77e8133f80 msgr2=0x7f77e8131c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:31.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.009+0000 7f77ed200640 1 --2- 172.21.15.86:0/3739903452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77e8133f80 0x7f77e8131c80 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f77d8011570 tx=0x7f77d8011a30 comp rx=0 tx=0).stop 2024-04-20T21:22:31.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.009+0000 7f77ed200640 1 -- 172.21.15.86:0/3739903452 shutdown_connections 2024-04-20T21:22:31.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.009+0000 7f77ed200640 1 --2- 172.21.15.86:0/3739903452 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f77c806c8f0 0x7f77c806edb0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:31.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.009+0000 7f77ed200640 1 --2- 172.21.15.86:0/3739903452 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77e81373e0 0x7f77e8137860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:31.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.009+0000 7f77ed200640 1 --2- 172.21.15.86:0/3739903452 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f77e8134360 0x7f77e81321c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:31.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.009+0000 7f77ed200640 1 --2- 172.21.15.86:0/3739903452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77e8133f80 0x7f77e8131c80 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:31.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.009+0000 7f77ed200640 1 -- 172.21.15.86:0/3739903452 >> 172.21.15.86:0/3739903452 conn(0x7f77e80b2360 msgr2=0x7f77e8160420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:31.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.009+0000 7f77ed200640 1 -- 172.21.15.86:0/3739903452 shutdown_connections 2024-04-20T21:22:31.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.009+0000 7f77ed200640 1 -- 172.21.15.86:0/3739903452 wait complete. 2024-04-20T21:22:31.027 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-20T21:22:31.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.222+0000 7f9873e00640 1 Processor -- start 2024-04-20T21:22:31.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.222+0000 7f9873e00640 1 -- start start 2024-04-20T21:22:31.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.223+0000 7f9873e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f986c06f070 0x7f986c0b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:31.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.223+0000 7f9873e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f986c0cff80 0x7f986c0d0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:31.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.223+0000 7f9873e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f986c15b9d0 con 0x7f986c15b5f0 2024-04-20T21:22:31.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.223+0000 7f9873e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f986c0b4650 con 0x7f986c0cff80 2024-04-20T21:22:31.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.223+0000 7f9873e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f986c0af850 con 0x7f986c06f070 2024-04-20T21:22:31.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.224+0000 7f9871e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f986c0cff80 0x7f986c0d0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:31.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.224+0000 7f9871e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f986c0cff80 0x7f986c0d0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38944/0 (socket says 172.21.15.86:38944) 2024-04-20T21:22:31.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.224+0000 7f9871e00640 1 -- 172.21.15.86:0/3956541572 learned_addr learned my addr 172.21.15.86:0/3956541572 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:31.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.224+0000 7f9870a00640 1 --2- 172.21.15.86:0/3956541572 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f986c06f070 0x7f986c0b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:31.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.224+0000 7f9871e00640 1 -- 172.21.15.86:0/3956541572 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f986c06f070 msgr2=0x7f986c0b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:31.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.224+0000 7f9871e00640 1 --2- 172.21.15.86:0/3956541572 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f986c06f070 0x7f986c0b3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:31.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.224+0000 7f9871e00640 1 -- 172.21.15.86:0/3956541572 >> v1:172.21.15.144:6789/0 conn(0x7f986c15b5f0 legacy=0x7f986c0b50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:31.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.224+0000 7f9871e00640 1 -- 172.21.15.86:0/3956541572 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f986c0afb30 con 0x7f986c0cff80 2024-04-20T21:22:31.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.225+0000 7f9871e00640 1 --2- 172.21.15.86:0/3956541572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f986c0cff80 0x7f986c0d0410 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f9864009e50 tx=0x7f986402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2286d193f837f77f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:31.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.225+0000 7f986be00640 1 -- 172.21.15.86:0/3956541572 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986402ca70 con 0x7f986c0cff80 2024-04-20T21:22:31.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.225+0000 7f986be00640 1 -- 172.21.15.86:0/3956541572 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f986402cc10 con 0x7f986c0cff80 2024-04-20T21:22:31.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.225+0000 7f986be00640 1 -- 172.21.15.86:0/3956541572 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98640343f0 con 0x7f986c0cff80 2024-04-20T21:22:31.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.226+0000 7f9873e00640 1 -- 172.21.15.86:0/3956541572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f986c0cff80 msgr2=0x7f986c0d0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:31.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.226+0000 7f9873e00640 1 --2- 172.21.15.86:0/3956541572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f986c0cff80 0x7f986c0d0410 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f9864009e50 tx=0x7f986402bed0 comp rx=0 tx=0).stop 2024-04-20T21:22:31.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.226+0000 7f9873e00640 1 -- 172.21.15.86:0/3956541572 shutdown_connections 2024-04-20T21:22:31.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.226+0000 7f9873e00640 1 --2- 172.21.15.86:0/3956541572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f986c0cff80 0x7f986c0d0410 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:31.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.226+0000 7f9873e00640 1 --2- 172.21.15.86:0/3956541572 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f986c06f070 0x7f986c0b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:31.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.226+0000 7f9873e00640 1 -- 172.21.15.86:0/3956541572 >> 172.21.15.86:0/3956541572 conn(0x7f986c06b980 msgr2=0x7f986c06bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:31.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.226+0000 7f9873e00640 1 -- 172.21.15.86:0/3956541572 shutdown_connections 2024-04-20T21:22:31.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.226+0000 7f9873e00640 1 -- 172.21.15.86:0/3956541572 wait complete. 2024-04-20T21:22:31.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.228+0000 7f9873e00640 1 Processor -- start 2024-04-20T21:22:31.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.228+0000 7f9873e00640 1 -- start start 2024-04-20T21:22:31.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.229+0000 7f9873e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f986c06f070 0x7f986c1f5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:31.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.229+0000 7f9873e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f986c0cff80 0x7f986c14a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:31.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.229+0000 7f9873e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f986c15b5f0 0x7f986c141930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:31.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.229+0000 7f9873e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f986c0ce130 con 0x7f986c0cff80 2024-04-20T21:22:31.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.229+0000 7f9873e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f986c0b4b80 con 0x7f986c06f070 2024-04-20T21:22:31.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.229+0000 7f9873e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f986c0ce2b0 con 0x7f986c15b5f0 2024-04-20T21:22:31.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.229+0000 7f9870a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f986c0cff80 0x7f986c14a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:31.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.229+0000 7f9870a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f986c0cff80 0x7f986c14a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38952/0 (socket says 172.21.15.86:38952) 2024-04-20T21:22:31.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.229+0000 7f9870a00640 1 -- 172.21.15.86:0/1574212965 learned_addr learned my addr 172.21.15.86:0/1574212965 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:31.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.229+0000 7f9871e00640 1 --2- 172.21.15.86:0/1574212965 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f986c15b5f0 0x7f986c141930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:31.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.229+0000 7f9870a00640 1 -- 172.21.15.86:0/1574212965 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f986c15b5f0 msgr2=0x7f986c141930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:31.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.229+0000 7f9870a00640 1 --2- 172.21.15.86:0/1574212965 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f986c15b5f0 0x7f986c141930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:31.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.229+0000 7f9870a00640 1 -- 172.21.15.86:0/1574212965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f986c06f070 msgr2=0x7f986c1f5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:31.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.229+0000 7f9871400640 1 --2- 172.21.15.86:0/1574212965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f986c06f070 0x7f986c1f5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:31.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.229+0000 7f9870a00640 1 --2- 172.21.15.86:0/1574212965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f986c06f070 0x7f986c1f5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:31.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.229+0000 7f9870a00640 1 -- 172.21.15.86:0/1574212965 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f986c1f61a0 con 0x7f986c0cff80 2024-04-20T21:22:31.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.230+0000 7f9870a00640 1 --2- 172.21.15.86:0/1574212965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f986c0cff80 0x7f986c14a1d0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f985800c7f0 tx=0x7f985800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:31.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.230+0000 7f9869600640 1 -- 172.21.15.86:0/1574212965 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9858015070 con 0x7f986c0cff80 2024-04-20T21:22:31.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.230+0000 7f9869600640 1 -- 172.21.15.86:0/1574212965 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9858004240 con 0x7f986c0cff80 2024-04-20T21:22:31.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.230+0000 7f9869600640 1 -- 172.21.15.86:0/1574212965 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9858007620 con 0x7f986c0cff80 2024-04-20T21:22:31.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.230+0000 7f9873e00640 1 -- 172.21.15.86:0/1574212965 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f986c1f6480 con 0x7f986c0cff80 2024-04-20T21:22:31.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.230+0000 7f9873e00640 1 -- 172.21.15.86:0/1574212965 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f986c142390 con 0x7f986c0cff80 2024-04-20T21:22:31.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.231+0000 7f9873e00640 1 -- 172.21.15.86:0/1574212965 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f986c0689e0 con 0x7f986c0cff80 2024-04-20T21:22:31.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.232+0000 7f9869600640 1 -- 172.21.15.86:0/1574212965 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f98580079d0 con 0x7f986c0cff80 2024-04-20T21:22:31.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.233+0000 7f9869600640 1 --2- 172.21.15.86:0/1574212965 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f984c06c8f0 0x7f984c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:31.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.233+0000 7f9869600640 1 -- 172.21.15.86:0/1574212965 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f985808bc20 con 0x7f986c0cff80 2024-04-20T21:22:31.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.233+0000 7f9871400640 1 --2- 172.21.15.86:0/1574212965 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f984c06c8f0 0x7f984c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:31.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.234+0000 7f9871400640 1 --2- 172.21.15.86:0/1574212965 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f984c06c8f0 0x7f984c06edb0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f985c00a750 tx=0x7f985c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:31.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.237+0000 7f9869600640 1 -- 172.21.15.86:0/1574212965 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f985805a040 con 0x7f986c0cff80 2024-04-20T21:22:31.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.472+0000 7f9873e00640 1 -- 172.21.15.86:0/1574212965 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f986c142560 con 0x7f986c0cff80 2024-04-20T21:22:31.472 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-04-20T21:22:31.473+0000 7f5bb9c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T21:22:31.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.477+0000 7f9869600640 1 -- 172.21.15.86:0/1574212965 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v35) ==== 161+0+0 (secure 0 0 0) 0x7f9858059940 con 0x7f986c0cff80 2024-04-20T21:22:31.476 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-20T21:22:31.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.480+0000 7f9873e00640 1 -- 172.21.15.86:0/1574212965 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f984c06c8f0 msgr2=0x7f984c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:31.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.480+0000 7f9873e00640 1 --2- 172.21.15.86:0/1574212965 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f984c06c8f0 0x7f984c06edb0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f985c00a750 tx=0x7f985c008040 comp rx=0 tx=0).stop 2024-04-20T21:22:31.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.480+0000 7f9873e00640 1 -- 172.21.15.86:0/1574212965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f986c0cff80 msgr2=0x7f986c14a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:31.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.480+0000 7f9873e00640 1 --2- 172.21.15.86:0/1574212965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f986c0cff80 0x7f986c14a1d0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f985800c7f0 tx=0x7f985800ccb0 comp rx=0 tx=0).stop 2024-04-20T21:22:31.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.480+0000 7f9873e00640 1 -- 172.21.15.86:0/1574212965 shutdown_connections 2024-04-20T21:22:31.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.480+0000 7f9873e00640 1 --2- 172.21.15.86:0/1574212965 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f984c06c8f0 0x7f984c06edb0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:31.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.480+0000 7f9873e00640 1 --2- 172.21.15.86:0/1574212965 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f986c15b5f0 0x7f986c141930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:31.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.480+0000 7f9873e00640 1 --2- 172.21.15.86:0/1574212965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f986c0cff80 0x7f986c14a1d0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:31.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.480+0000 7f9873e00640 1 --2- 172.21.15.86:0/1574212965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f986c06f070 0x7f986c1f5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:31.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.480+0000 7f9873e00640 1 -- 172.21.15.86:0/1574212965 >> 172.21.15.86:0/1574212965 conn(0x7f986c06b980 msgr2=0x7f986c0d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:31.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.480+0000 7f9873e00640 1 -- 172.21.15.86:0/1574212965 shutdown_connections 2024-04-20T21:22:31.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.480+0000 7f9873e00640 1 -- 172.21.15.86:0/1574212965 wait complete. 2024-04-20T21:22:31.497 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-20T21:22:31.509 INFO:tasks.ceph.mds.c.smithi086.stderr: -6723> 2024-04-20T21:21:12.432+0000 7fb614600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T21:22:31.516 INFO:tasks.ceph.mds.c.smithi086.stderr: -4943> 2024-04-20T21:21:19.706+0000 7fb60f600640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={6306=0-4194304@1} caps={6306=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6306 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x559d2d306b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-20T21:22:31.521 INFO:tasks.ceph.mds.c.smithi086.stderr: -2976> 2024-04-20T21:21:51.710+0000 7fb60f600640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v4 ap=1 DIRTYPARENT s=0 n(v0 rc2024-04-20T21:21:18.168279+0000 1=1+0) caps={6306=pAsLsXsFs/-@2},l=6306(-1) | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=1 authpin=1 scrubqueue=0 0x559d2d306b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v4>,<0x1/dir_test v7>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-20T21:22:31.583 INFO:tasks.ceph.mds.c.smithi086.stdout:starting mds.c at 2024-04-20T21:22:31.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.668+0000 7fcc4a000640 1 Processor -- start 2024-04-20T21:22:31.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.668+0000 7fcc4a000640 1 -- start start 2024-04-20T21:22:31.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.669+0000 7fcc4a000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc4406c090 0x7fcc440cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:31.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.669+0000 7fcc4a000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc44156110 con 0x7fcc44155d30 2024-04-20T21:22:31.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.670+0000 7fcc4a000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc440ce460 con 0x7fcc440b4ba0 2024-04-20T21:22:31.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.670+0000 7fcc4a000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fcc440b56f0 con 0x7fcc4406c090 2024-04-20T21:22:31.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.670+0000 7fcc43400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fcc44155d30 0x7fcc4406b980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:49626/0 (socket says 172.21.15.86:49626) 2024-04-20T21:22:31.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.670+0000 7fcc43400640 1 -- 172.21.15.86:0/1322186824 learned_addr learned my addr 172.21.15.86:0/1322186824 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:22:31.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.670+0000 7fcc42a00640 1 -- 172.21.15.86:0/1322186824 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1704126626 0 0) 0x7fcc44156110 con 0x7fcc44155d30 2024-04-20T21:22:31.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.670+0000 7fcc42a00640 1 -- 172.21.15.86:0/1322186824 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcc28003520 con 0x7fcc44155d30 2024-04-20T21:22:31.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.670+0000 7fcc3aa00640 1 --2- 172.21.15.86:0/1322186824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc4406c090 0x7fcc440cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:31.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.670+0000 7fcc42a00640 1 -- 172.21.15.86:0/1322186824 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2194542019 0 0) 0x7fcc28003520 con 0x7fcc44155d30 2024-04-20T21:22:31.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.670+0000 7fcc42a00640 1 -- 172.21.15.86:0/1322186824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc4406c090 msgr2=0x7fcc440cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:31.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.670+0000 7fcc42a00640 1 --2- 172.21.15.86:0/1322186824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc4406c090 0x7fcc440cf500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:31.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.670+0000 7fcc42a00640 1 -- 172.21.15.86:0/1322186824 >> v1:172.21.15.144:6789/0 conn(0x7fcc440b4ba0 legacy=0x7fcc440b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:31.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.670+0000 7fcc42a00640 1 -- 172.21.15.86:0/1322186824 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc440ce740 con 0x7fcc44155d30 2024-04-20T21:22:31.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.670+0000 7fcc42a00640 1 -- 172.21.15.86:0/1322186824 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fcc340019f0 con 0x7fcc44155d30 2024-04-20T21:22:31.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.670+0000 7fcc42a00640 1 -- 172.21.15.86:0/1322186824 >> v1:172.21.15.86:6789/0 conn(0x7fcc44155d30 legacy=0x7fcc4406b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:31.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.670+0000 7fcc42a00640 1 --2- 172.21.15.86:0/1322186824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc4406c090 0x7fcc280039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:31.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.670+0000 7fcc42a00640 1 --2- 172.21.15.86:0/1322186824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc28004040 0x7fcc28004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:31.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.671+0000 7fcc42a00640 1 --2- 172.21.15.86:0/1322186824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc28004db0 0x7fcc28005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:31.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.671+0000 7fcc42a00640 1 -- 172.21.15.86:0/1322186824 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcc440b56f0 con 0x7fcc28004db0 2024-04-20T21:22:31.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.671+0000 7fcc42a00640 1 -- 172.21.15.86:0/1322186824 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2800bcc0 con 0x7fcc28004040 2024-04-20T21:22:31.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.671+0000 7fcc42a00640 1 -- 172.21.15.86:0/1322186824 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fcc2800bec0 con 0x7fcc4406c090 2024-04-20T21:22:31.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.671+0000 7fcc43e00640 1 --2- 172.21.15.86:0/1322186824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc28004db0 0x7fcc28005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:31.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.671+0000 7fcc3aa00640 1 --2- 172.21.15.86:0/1322186824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc28004040 0x7fcc28004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:31.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.671+0000 7fcc43400640 1 --2- 172.21.15.86:0/1322186824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc4406c090 0x7fcc280039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:31.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.671+0000 7fcc43e00640 1 -- 172.21.15.86:0/1322186824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc4406c090 msgr2=0x7fcc280039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:31.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.671+0000 7fcc43e00640 1 --2- 172.21.15.86:0/1322186824 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc4406c090 0x7fcc280039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:31.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.671+0000 7fcc43e00640 1 -- 172.21.15.86:0/1322186824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc28004040 msgr2=0x7fcc28004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:31.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.671+0000 7fcc43e00640 1 --2- 172.21.15.86:0/1322186824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc28004040 0x7fcc28004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:31.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.671+0000 7fcc43e00640 1 -- 172.21.15.86:0/1322186824 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc28003520 con 0x7fcc28004db0 2024-04-20T21:22:31.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.673+0000 7fcc43e00640 1 --2- 172.21.15.86:0/1322186824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc28004db0 0x7fcc28005230 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c003db0 tx=0x7fcc28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3c9f512b0cc040b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:31.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.673+0000 7fcc42a00640 1 -- 172.21.15.86:0/1322186824 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c037070 con 0x7fcc28004db0 2024-04-20T21:22:31.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.673+0000 7fcc42a00640 1 -- 172.21.15.86:0/1322186824 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc3c032600 con 0x7fcc28004db0 2024-04-20T21:22:31.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.673+0000 7fcc42a00640 1 -- 172.21.15.86:0/1322186824 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c003980 con 0x7fcc28004db0 2024-04-20T21:22:31.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.673+0000 7fcc4a000640 1 -- 172.21.15.86:0/1322186824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc28004db0 msgr2=0x7fcc28005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:31.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.673+0000 7fcc4a000640 1 --2- 172.21.15.86:0/1322186824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc28004db0 0x7fcc28005230 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c003db0 tx=0x7fcc28005830 comp rx=0 tx=0).stop 2024-04-20T21:22:31.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.673+0000 7fcc43e00640 1 -- 172.21.15.86:0/1322186824 reap_dead start 2024-04-20T21:22:31.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.673+0000 7fcc4a000640 1 -- 172.21.15.86:0/1322186824 shutdown_connections 2024-04-20T21:22:31.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.673+0000 7fcc4a000640 1 -- 172.21.15.86:0/1322186824 >> 172.21.15.86:0/1322186824 conn(0x7fcc440b2110 msgr2=0x7fcc440b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:31.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.673+0000 7fcc4a000640 1 -- 172.21.15.86:0/1322186824 shutdown_connections 2024-04-20T21:22:31.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.674+0000 7fcc4a000640 1 -- 172.21.15.86:0/1322186824 wait complete. 2024-04-20T21:22:31.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.675+0000 7fcc4a000640 1 Processor -- start 2024-04-20T21:22:31.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.675+0000 7fcc4a000640 1 -- start start 2024-04-20T21:22:31.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.676+0000 7fcc4a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc44150040 0x7fcc441f4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:31.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.676+0000 7fcc4a000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc44152480 0x7fcc4415d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:31.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.676+0000 7fcc4a000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc4415da80 0x7fcc4414ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:31.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.676+0000 7fcc4a000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcc440cefe0 con 0x7fcc44150040 2024-04-20T21:22:31.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.676+0000 7fcc4a000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcc440ced00 con 0x7fcc44152480 2024-04-20T21:22:31.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.676+0000 7fcc4a000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fcc440cf500 con 0x7fcc4415da80 2024-04-20T21:22:31.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.677+0000 7fcc43400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc44150040 0x7fcc441f4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:31.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.677+0000 7fcc43400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc44150040 0x7fcc441f4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38974/0 (socket says 172.21.15.86:38974) 2024-04-20T21:22:31.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.677+0000 7fcc43400640 1 -- 172.21.15.86:0/3424150808 learned_addr learned my addr 172.21.15.86:0/3424150808 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:31.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.677+0000 7fcc43400640 1 -- 172.21.15.86:0/3424150808 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc4415da80 msgr2=0x7fcc4414ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:31.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.677+0000 7fcc43400640 1 --2- 172.21.15.86:0/3424150808 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc4415da80 0x7fcc4414ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:31.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.677+0000 7fcc43400640 1 -- 172.21.15.86:0/3424150808 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc44152480 msgr2=0x7fcc4415d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:31.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.677+0000 7fcc43400640 1 --2- 172.21.15.86:0/3424150808 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc44152480 0x7fcc4415d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:31.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.677+0000 7fcc43400640 1 -- 172.21.15.86:0/3424150808 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc441f4e20 con 0x7fcc44150040 2024-04-20T21:22:31.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.677+0000 7fcc43400640 1 --2- 172.21.15.86:0/3424150808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc44150040 0x7fcc441f4580 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fcc3400e440 tx=0x7fcc3400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:31.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.677+0000 7fcc3be00640 1 -- 172.21.15.86:0/3424150808 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3400ee30 con 0x7fcc44150040 2024-04-20T21:22:31.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.677+0000 7fcc3be00640 1 -- 172.21.15.86:0/3424150808 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc34018070 con 0x7fcc44150040 2024-04-20T21:22:31.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.677+0000 7fcc4a000640 1 -- 172.21.15.86:0/3424150808 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc441f5100 con 0x7fcc44150040 2024-04-20T21:22:31.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.678+0000 7fcc3be00640 1 -- 172.21.15.86:0/3424150808 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc34013660 con 0x7fcc44150040 2024-04-20T21:22:31.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.678+0000 7fcc4a000640 1 -- 172.21.15.86:0/3424150808 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc4415cd40 con 0x7fcc44150040 2024-04-20T21:22:31.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.679+0000 7fcc4a000640 1 -- 172.21.15.86:0/3424150808 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc44152d40 con 0x7fcc44150040 2024-04-20T21:22:31.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.680+0000 7fcc3be00640 1 -- 172.21.15.86:0/3424150808 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcc34013800 con 0x7fcc44150040 2024-04-20T21:22:31.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.680+0000 7fcc3be00640 1 --2- 172.21.15.86:0/3424150808 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcc2006c920 0x7fcc2006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:31.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.681+0000 7fcc3be00640 1 -- 172.21.15.86:0/3424150808 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5795+0+0 (secure 0 0 0) 0x7fcc3408c310 con 0x7fcc44150040 2024-04-20T21:22:31.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.681+0000 7fcc3aa00640 1 --2- 172.21.15.86:0/3424150808 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcc2006c920 0x7fcc2006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:31.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.681+0000 7fcc3aa00640 1 --2- 172.21.15.86:0/3424150808 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcc2006c920 0x7fcc2006ede0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c006fd0 tx=0x7fcc2c00b2e0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:31.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.685+0000 7fcc3be00640 1 -- 172.21.15.86:0/3424150808 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fcc44152d40 con 0x7fcc44150040 2024-04-20T21:22:31.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:31.932+0000 7fcc4a000640 1 -- 172.21.15.86:0/3424150808 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fcc440689e0 con 0x7fcc44150040 2024-04-20T21:22:32.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.480+0000 7fcc3be00640 1 -- 172.21.15.86:0/3424150808 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v36) ==== 104+0+0 (secure 0 0 0) 0x7fcc3405a640 con 0x7fcc44150040 2024-04-20T21:22:32.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.483+0000 7fcc4a000640 1 -- 172.21.15.86:0/3424150808 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcc2006c920 msgr2=0x7fcc2006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:32.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.483+0000 7fcc4a000640 1 --2- 172.21.15.86:0/3424150808 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcc2006c920 0x7fcc2006ede0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c006fd0 tx=0x7fcc2c00b2e0 comp rx=0 tx=0).stop 2024-04-20T21:22:32.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.483+0000 7fcc4a000640 1 -- 172.21.15.86:0/3424150808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc44150040 msgr2=0x7fcc441f4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:32.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.483+0000 7fcc4a000640 1 --2- 172.21.15.86:0/3424150808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc44150040 0x7fcc441f4580 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fcc3400e440 tx=0x7fcc3400e530 comp rx=0 tx=0).stop 2024-04-20T21:22:32.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.484+0000 7fcc4a000640 1 -- 172.21.15.86:0/3424150808 shutdown_connections 2024-04-20T21:22:32.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.484+0000 7fcc4a000640 1 --2- 172.21.15.86:0/3424150808 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcc2006c920 0x7fcc2006ede0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:32.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.484+0000 7fcc4a000640 1 --2- 172.21.15.86:0/3424150808 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc4415da80 0x7fcc4414ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:32.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.484+0000 7fcc4a000640 1 --2- 172.21.15.86:0/3424150808 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc44152480 0x7fcc4415d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:32.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.484+0000 7fcc4a000640 1 --2- 172.21.15.86:0/3424150808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc44150040 0x7fcc441f4580 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:32.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.484+0000 7fcc4a000640 1 -- 172.21.15.86:0/3424150808 >> 172.21.15.86:0/3424150808 conn(0x7fcc440b2110 msgr2=0x7fcc44154c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:32.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.484+0000 7fcc4a000640 1 -- 172.21.15.86:0/3424150808 shutdown_connections 2024-04-20T21:22:32.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.484+0000 7fcc4a000640 1 -- 172.21.15.86:0/3424150808 wait complete. 2024-04-20T21:22:32.506 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-20T21:22:32.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.759+0000 7ff10fe00640 1 Processor -- start 2024-04-20T21:22:32.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.759+0000 7ff10fe00640 1 -- start start 2024-04-20T21:22:32.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.760+0000 7ff10fe00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff1101569d0 0x7ff1100d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:32.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.760+0000 7ff10fe00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff11015d6b0 con 0x7ff11015d100 2024-04-20T21:22:32.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.760+0000 7ff10fe00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff11006dc20 con 0x7ff11006e700 2024-04-20T21:22:32.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.760+0000 7ff10fe00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7ff11006df00 con 0x7ff1101569d0 2024-04-20T21:22:32.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.761+0000 7ff10f400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7ff11015d100 0x7ff11015db00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:49630/0 (socket says 172.21.15.86:49630) 2024-04-20T21:22:32.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.761+0000 7ff10f400640 1 -- 172.21.15.86:0/1585698622 learned_addr learned my addr 172.21.15.86:0/1585698622 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:22:32.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.761+0000 7ff10e000640 1 -- 172.21.15.86:0/1585698622 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1445327472 0 0) 0x7ff11015d6b0 con 0x7ff11015d100 2024-04-20T21:22:32.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.761+0000 7ff10e000640 1 -- 172.21.15.86:0/1585698622 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0f4003520 con 0x7ff11015d100 2024-04-20T21:22:32.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.761+0000 7ff10ea00640 1 --2- 172.21.15.86:0/1585698622 >> v2:172.21.15.144:3300/0 conn(0x7ff1101569d0 0x7ff1100d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:32.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.761+0000 7ff10e000640 1 -- 172.21.15.86:0/1585698622 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2128892311 0 0) 0x7ff0f4003520 con 0x7ff11015d100 2024-04-20T21:22:32.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.761+0000 7ff10e000640 1 -- 172.21.15.86:0/1585698622 >> v2:172.21.15.144:3300/0 conn(0x7ff1101569d0 msgr2=0x7ff1100d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:32.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.761+0000 7ff10e000640 1 --2- 172.21.15.86:0/1585698622 >> v2:172.21.15.144:3300/0 conn(0x7ff1101569d0 0x7ff1100d05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:32.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.761+0000 7ff10e000640 1 -- 172.21.15.86:0/1585698622 >> v1:172.21.15.144:6789/0 conn(0x7ff11006e700 legacy=0x7ff11015c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:32.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.761+0000 7ff10e000640 1 -- 172.21.15.86:0/1585698622 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff11006e1e0 con 0x7ff11015d100 2024-04-20T21:22:32.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.761+0000 7ff10e000640 1 -- 172.21.15.86:0/1585698622 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7ff1080019f0 con 0x7ff11015d100 2024-04-20T21:22:32.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.761+0000 7ff10ea00640 1 --2- 172.21.15.86:0/1585698622 >> v2:172.21.15.144:3300/0 conn(0x7ff1101569d0 0x7ff1100d05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:22:32.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.761+0000 7ff10e000640 1 -- 172.21.15.86:0/1585698622 >> v1:172.21.15.86:6789/0 conn(0x7ff11015d100 legacy=0x7ff11015db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:32.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.761+0000 7ff10e000640 1 --2- 172.21.15.86:0/1585698622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0f4003a00 0x7ff0f4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:32.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.761+0000 7ff10e000640 1 --2- 172.21.15.86:0/1585698622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0f4004800 0x7ff0f4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:32.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.762+0000 7ff10e000640 1 --2- 172.21.15.86:0/1585698622 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0f40052e0 0x7ff0f400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:32.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.762+0000 7ff10e000640 1 -- 172.21.15.86:0/1585698622 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff11006df00 con 0x7ff0f4004800 2024-04-20T21:22:32.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.762+0000 7ff10e000640 1 -- 172.21.15.86:0/1585698622 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f400c230 con 0x7ff0f4003a00 2024-04-20T21:22:32.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.762+0000 7ff10e000640 1 -- 172.21.15.86:0/1585698622 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff0f400c430 con 0x7ff0f40052e0 2024-04-20T21:22:32.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.762+0000 7ff106000640 1 --2- 172.21.15.86:0/1585698622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0f4004800 0x7ff0f4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:32.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.762+0000 7ff106000640 1 -- 172.21.15.86:0/1585698622 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0f40052e0 msgr2=0x7ff0f400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:32.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.762+0000 7ff106000640 1 --2- 172.21.15.86:0/1585698622 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff0f40052e0 0x7ff0f400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:32.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.762+0000 7ff106000640 1 -- 172.21.15.86:0/1585698622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0f4003a00 msgr2=0x7ff0f4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:32.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.762+0000 7ff106000640 1 --2- 172.21.15.86:0/1585698622 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0f4003a00 0x7ff0f4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:32.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.762+0000 7ff106000640 1 -- 172.21.15.86:0/1585698622 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0f4003520 con 0x7ff0f4004800 2024-04-20T21:22:32.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.762+0000 7ff10f400640 1 -- 172.21.15.86:0/1585698622 reap_dead start 2024-04-20T21:22:32.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.763+0000 7ff106000640 1 --2- 172.21.15.86:0/1585698622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0f4004800 0x7ff0f4004c80 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8001620 tx=0x7ff0f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbb2c9b85eeb1a44 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:32.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.763+0000 7ff10e000640 1 -- 172.21.15.86:0/1585698622 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8001220 con 0x7ff0f4004800 2024-04-20T21:22:32.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.763+0000 7ff10e000640 1 -- 172.21.15.86:0/1585698622 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0f8005290 con 0x7ff0f4004800 2024-04-20T21:22:32.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.763+0000 7ff10e000640 1 -- 172.21.15.86:0/1585698622 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8003850 con 0x7ff0f4004800 2024-04-20T21:22:32.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.764+0000 7ff10fe00640 1 -- 172.21.15.86:0/1585698622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0f4004800 msgr2=0x7ff0f4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:32.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.764+0000 7ff10fe00640 1 --2- 172.21.15.86:0/1585698622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0f4004800 0x7ff0f4004c80 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8001620 tx=0x7ff0f4005830 comp rx=0 tx=0).stop 2024-04-20T21:22:32.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.764+0000 7ff10fe00640 1 -- 172.21.15.86:0/1585698622 shutdown_connections 2024-04-20T21:22:32.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.764+0000 7ff10fe00640 1 --2- 172.21.15.86:0/1585698622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0f4004800 0x7ff0f4004c80 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:32.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.764+0000 7ff10fe00640 1 -- 172.21.15.86:0/1585698622 >> 172.21.15.86:0/1585698622 conn(0x7ff1100b2220 msgr2=0x7ff1100b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:32.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.764+0000 7ff10fe00640 1 -- 172.21.15.86:0/1585698622 shutdown_connections 2024-04-20T21:22:32.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.764+0000 7ff10fe00640 1 -- 172.21.15.86:0/1585698622 wait complete. 2024-04-20T21:22:32.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.766+0000 7ff10fe00640 1 Processor -- start 2024-04-20T21:22:32.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.766+0000 7ff10fe00640 1 -- start start 2024-04-20T21:22:32.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.767+0000 7ff10fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0f4004800 0x7ff1101465d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:32.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.767+0000 7ff10fe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1100b04d0 0x7ff1100b0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:32.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.767+0000 7ff10fe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff11009ceb0 0x7ff11009d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:32.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.767+0000 7ff10fe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff11015cb70 con 0x7ff0f4004800 2024-04-20T21:22:32.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.767+0000 7ff10fe00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff11015c9f0 con 0x7ff11009ceb0 2024-04-20T21:22:32.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.767+0000 7ff10fe00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff11015ccf0 con 0x7ff1100b04d0 2024-04-20T21:22:32.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.767+0000 7ff10ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0f4004800 0x7ff1101465d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:32.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.767+0000 7ff10ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0f4004800 0x7ff1101465d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38992/0 (socket says 172.21.15.86:38992) 2024-04-20T21:22:32.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.767+0000 7ff10ea00640 1 -- 172.21.15.86:0/3011853532 learned_addr learned my addr 172.21.15.86:0/3011853532 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:32.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.767+0000 7ff106000640 1 --2- 172.21.15.86:0/3011853532 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1100b04d0 0x7ff1100b0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:32.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.768+0000 7ff10f400640 1 --2- 172.21.15.86:0/3011853532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff11009ceb0 0x7ff11009d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:32.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.768+0000 7ff10ea00640 1 -- 172.21.15.86:0/3011853532 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1100b04d0 msgr2=0x7ff1100b0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:32.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.768+0000 7ff10ea00640 1 --2- 172.21.15.86:0/3011853532 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1100b04d0 0x7ff1100b0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:32.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.768+0000 7ff10ea00640 1 -- 172.21.15.86:0/3011853532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff11009ceb0 msgr2=0x7ff11009d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:32.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.768+0000 7ff10ea00640 1 --2- 172.21.15.86:0/3011853532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff11009ceb0 0x7ff11009d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:32.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.768+0000 7ff10ea00640 1 -- 172.21.15.86:0/3011853532 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff11009e1a0 con 0x7ff0f4004800 2024-04-20T21:22:32.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.768+0000 7ff10f400640 1 --2- 172.21.15.86:0/3011853532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff11009ceb0 0x7ff11009d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:22:32.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.768+0000 7ff10ea00640 1 --2- 172.21.15.86:0/3011853532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0f4004800 0x7ff1101465d0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7ff10000c7a0 tx=0x7ff10000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:32.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.768+0000 7ff107400640 1 -- 172.21.15.86:0/3011853532 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff100015070 con 0x7ff0f4004800 2024-04-20T21:22:32.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.768+0000 7ff107400640 1 -- 172.21.15.86:0/3011853532 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff100007820 con 0x7ff0f4004800 2024-04-20T21:22:32.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.768+0000 7ff107400640 1 -- 172.21.15.86:0/3011853532 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff100007bd0 con 0x7ff0f4004800 2024-04-20T21:22:32.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.768+0000 7ff10fe00640 1 -- 172.21.15.86:0/3011853532 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff11009dbe0 con 0x7ff0f4004800 2024-04-20T21:22:32.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.768+0000 7ff10fe00640 1 -- 172.21.15.86:0/3011853532 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1100b3350 con 0x7ff0f4004800 2024-04-20T21:22:32.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.770+0000 7ff10fe00640 1 -- 172.21.15.86:0/3011853532 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff11004fcf0 con 0x7ff0f4004800 2024-04-20T21:22:32.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.770+0000 7ff107400640 1 -- 172.21.15.86:0/3011853532 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff100018070 con 0x7ff0f4004800 2024-04-20T21:22:32.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.771+0000 7ff107400640 1 --2- 172.21.15.86:0/3011853532 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff0ec06c920 0x7ff0ec06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:32.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.772+0000 7ff107400640 1 -- 172.21.15.86:0/3011853532 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5795+0+0 (secure 0 0 0) 0x7ff10008bad0 con 0x7ff0f4004800 2024-04-20T21:22:32.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.772+0000 7ff106000640 1 --2- 172.21.15.86:0/3011853532 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff0ec06c920 0x7ff0ec06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:32.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.772+0000 7ff106000640 1 --2- 172.21.15.86:0/3011853532 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff0ec06c920 0x7ff0ec06ede0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8033000 tx=0x7ff0f8009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:32.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.776+0000 7ff107400640 1 -- 172.21.15.86:0/3011853532 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff100059e00 con 0x7ff0f4004800 2024-04-20T21:22:32.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:32.983+0000 7ff10fe00640 1 -- 172.21.15.86:0/3011853532 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff110071fe0 con 0x7ff0f4004800 2024-04-20T21:22:33.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:33.847+0000 7ff107400640 1 -- 172.21.15.86:0/3011853532 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v61) ==== 181+0+0 (secure 0 0 0) 0x7ff100059700 con 0x7ff0f4004800 2024-04-20T21:22:33.847 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' removed 2024-04-20T21:22:33.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:33.852+0000 7ff10fe00640 1 -- 172.21.15.86:0/3011853532 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff0ec06c920 msgr2=0x7ff0ec06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:33.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:33.852+0000 7ff10fe00640 1 --2- 172.21.15.86:0/3011853532 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff0ec06c920 0x7ff0ec06ede0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8033000 tx=0x7ff0f8009030 comp rx=0 tx=0).stop 2024-04-20T21:22:33.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:33.852+0000 7ff10fe00640 1 -- 172.21.15.86:0/3011853532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0f4004800 msgr2=0x7ff1101465d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:33.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:33.852+0000 7ff10fe00640 1 --2- 172.21.15.86:0/3011853532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0f4004800 0x7ff1101465d0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7ff10000c7a0 tx=0x7ff10000cc60 comp rx=0 tx=0).stop 2024-04-20T21:22:33.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:33.853+0000 7ff10fe00640 1 -- 172.21.15.86:0/3011853532 shutdown_connections 2024-04-20T21:22:33.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:33.853+0000 7ff10fe00640 1 --2- 172.21.15.86:0/3011853532 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff0ec06c920 0x7ff0ec06ede0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:33.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:33.853+0000 7ff10fe00640 1 --2- 172.21.15.86:0/3011853532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff11009ceb0 0x7ff11009d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:33.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:33.853+0000 7ff10fe00640 1 --2- 172.21.15.86:0/3011853532 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1100b04d0 0x7ff1100b0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:33.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:33.853+0000 7ff10fe00640 1 --2- 172.21.15.86:0/3011853532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0f4004800 0x7ff1101465d0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:33.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:33.853+0000 7ff10fe00640 1 -- 172.21.15.86:0/3011853532 >> 172.21.15.86:0/3011853532 conn(0x7ff1100b2220 msgr2=0x7ff1100d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:33.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:33.853+0000 7ff10fe00640 1 -- 172.21.15.86:0/3011853532 shutdown_connections 2024-04-20T21:22:33.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:33.853+0000 7ff10fe00640 1 -- 172.21.15.86:0/3011853532 wait complete. 2024-04-20T21:22:33.874 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-20T21:22:34.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.069+0000 7fc4c9000640 1 Processor -- start 2024-04-20T21:22:34.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.069+0000 7fc4c9000640 1 -- start start 2024-04-20T21:22:34.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.071+0000 7fc4c9000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc4c4153590 0x7fc4c40d0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:34.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.071+0000 7fc4c9000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4c4071ad0 0x7fc4c40d0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:34.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.071+0000 7fc4c9000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4c415cbe0 con 0x7fc4c40af160 2024-04-20T21:22:34.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.071+0000 7fc4c9000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc4c4153970 con 0x7fc4c4153590 2024-04-20T21:22:34.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.071+0000 7fc4c9000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc4c415cea0 con 0x7fc4c4071ad0 2024-04-20T21:22:34.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.071+0000 7fc4c3400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc4c4153590 0x7fc4c40d0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:34.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.071+0000 7fc4c3e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fc4c40af160 0x7fc4c406dce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:49646/0 (socket says 172.21.15.86:49646) 2024-04-20T21:22:34.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.071+0000 7fc4c3e00640 1 -- 172.21.15.86:0/1777022997 learned_addr learned my addr 172.21.15.86:0/1777022997 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:22:34.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.072+0000 7fc4bbe00640 1 --2- 172.21.15.86:0/1777022997 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4c4071ad0 0x7fc4c40d0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:34.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.072+0000 7fc4c3400640 1 -- 172.21.15.86:0/1777022997 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4c4071ad0 msgr2=0x7fc4c40d0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:34.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.072+0000 7fc4c3400640 1 --2- 172.21.15.86:0/1777022997 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4c4071ad0 0x7fc4c40d0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:34.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.072+0000 7fc4c3400640 1 -- 172.21.15.86:0/1777022997 >> v1:172.21.15.86:6789/0 conn(0x7fc4c40af160 legacy=0x7fc4c406dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:34.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.072+0000 7fc4c3400640 1 -- 172.21.15.86:0/1777022997 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c415d180 con 0x7fc4c4153590 2024-04-20T21:22:34.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.072+0000 7fc4c2a00640 1 -- 172.21.15.86:0/1777022997 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 860208652 0 0) 0x7fc4c415cbe0 con 0x7fc4c40af160 2024-04-20T21:22:34.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.073+0000 7fc4c3400640 1 --2- 172.21.15.86:0/1777022997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4c4153590 0x7fc4c40d0610 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fc4b40097a0 tx=0x7fc4b402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c33b945fec291ada server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:34.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.073+0000 7fc4c2a00640 1 -- 172.21.15.86:0/1777022997 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b4035020 con 0x7fc4c4153590 2024-04-20T21:22:34.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.073+0000 7fc4c2a00640 1 -- 172.21.15.86:0/1777022997 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4b4031830 con 0x7fc4c4153590 2024-04-20T21:22:34.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.073+0000 7fc4c2a00640 1 -- 172.21.15.86:0/1777022997 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b4031c00 con 0x7fc4c4153590 2024-04-20T21:22:34.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.074+0000 7fc4c9000640 1 -- 172.21.15.86:0/1777022997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4c4153590 msgr2=0x7fc4c40d0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:34.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.074+0000 7fc4c9000640 1 --2- 172.21.15.86:0/1777022997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4c4153590 0x7fc4c40d0610 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fc4b40097a0 tx=0x7fc4b402b8d0 comp rx=0 tx=0).stop 2024-04-20T21:22:34.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.074+0000 7fc4c9000640 1 -- 172.21.15.86:0/1777022997 shutdown_connections 2024-04-20T21:22:34.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.074+0000 7fc4c9000640 1 --2- 172.21.15.86:0/1777022997 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4c4071ad0 0x7fc4c40d0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:34.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.074+0000 7fc4c9000640 1 --2- 172.21.15.86:0/1777022997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4c4153590 0x7fc4c40d0610 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:34.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.074+0000 7fc4c9000640 1 -- 172.21.15.86:0/1777022997 >> 172.21.15.86:0/1777022997 conn(0x7fc4c40b2180 msgr2=0x7fc4c40b2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:34.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.074+0000 7fc4c9000640 1 -- 172.21.15.86:0/1777022997 shutdown_connections 2024-04-20T21:22:34.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.074+0000 7fc4c9000640 1 -- 172.21.15.86:0/1777022997 wait complete. 2024-04-20T21:22:34.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.076+0000 7fc4c9000640 1 Processor -- start 2024-04-20T21:22:34.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.076+0000 7fc4c9000640 1 -- start start 2024-04-20T21:22:34.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.077+0000 7fc4c9000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4c4071ad0 0x7fc4c414ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:34.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.077+0000 7fc4c9000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4c40af160 0x7fc4c40ceb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:34.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.077+0000 7fc4c9000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4c4153590 0x7fc4c41512c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:34.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.078+0000 7fc4c9000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c40af5f0 con 0x7fc4c4071ad0 2024-04-20T21:22:34.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.078+0000 7fc4c9000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c415d740 con 0x7fc4c40af160 2024-04-20T21:22:34.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.078+0000 7fc4c9000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc4c40af770 con 0x7fc4c4153590 2024-04-20T21:22:34.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.078+0000 7fc4c3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4c4071ad0 0x7fc4c414ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:34.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.078+0000 7fc4c3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4c4071ad0 0x7fc4c414ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39004/0 (socket says 172.21.15.86:39004) 2024-04-20T21:22:34.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.078+0000 7fc4c3400640 1 -- 172.21.15.86:0/3998805103 learned_addr learned my addr 172.21.15.86:0/3998805103 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:34.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.078+0000 7fc4c3400640 1 -- 172.21.15.86:0/3998805103 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4c4153590 msgr2=0x7fc4c41512c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:34.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.078+0000 7fc4c3400640 1 --2- 172.21.15.86:0/3998805103 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4c4153590 0x7fc4c41512c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:34.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.078+0000 7fc4c3400640 1 -- 172.21.15.86:0/3998805103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4c40af160 msgr2=0x7fc4c40ceb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:34.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.078+0000 7fc4c3400640 1 --2- 172.21.15.86:0/3998805103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4c40af160 0x7fc4c40ceb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:34.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.078+0000 7fc4c3400640 1 -- 172.21.15.86:0/3998805103 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c414f2e0 con 0x7fc4c4071ad0 2024-04-20T21:22:34.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.078+0000 7fc4c3400640 1 --2- 172.21.15.86:0/3998805103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4c4071ad0 0x7fc4c414ec00 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4034040 tx=0x7fc4b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:34.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.079+0000 7fc4bb400640 1 -- 172.21.15.86:0/3998805103 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b4040070 con 0x7fc4c4071ad0 2024-04-20T21:22:34.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.079+0000 7fc4bb400640 1 -- 172.21.15.86:0/3998805103 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4b40399a0 con 0x7fc4c4071ad0 2024-04-20T21:22:34.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.079+0000 7fc4bb400640 1 -- 172.21.15.86:0/3998805103 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b4035040 con 0x7fc4c4071ad0 2024-04-20T21:22:34.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.079+0000 7fc4c9000640 1 -- 172.21.15.86:0/3998805103 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4c414f5c0 con 0x7fc4c4071ad0 2024-04-20T21:22:34.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.079+0000 7fc4c9000640 1 -- 172.21.15.86:0/3998805103 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4c40b3890 con 0x7fc4c4071ad0 2024-04-20T21:22:34.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.080+0000 7fc4c9000640 1 -- 172.21.15.86:0/3998805103 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4c404fcf0 con 0x7fc4c4071ad0 2024-04-20T21:22:34.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.081+0000 7fc4bb400640 1 -- 172.21.15.86:0/3998805103 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc4b4038020 con 0x7fc4c4071ad0 2024-04-20T21:22:34.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.082+0000 7fc4bb400640 1 --2- 172.21.15.86:0/3998805103 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc4a006c8f0 0x7fc4a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:34.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.082+0000 7fc4bb400640 1 -- 172.21.15.86:0/3998805103 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5357+0+0 (secure 0 0 0) 0x7fc4b40ad4b0 con 0x7fc4c4071ad0 2024-04-20T21:22:34.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.082+0000 7fc4bbe00640 1 --2- 172.21.15.86:0/3998805103 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc4a006c8f0 0x7fc4a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:34.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.083+0000 7fc4bbe00640 1 --2- 172.21.15.86:0/3998805103 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc4a006c8f0 0x7fc4a006edb0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc4c415a0c0 tx=0x7fc4ac008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:34.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.086+0000 7fc4bb400640 1 -- 172.21.15.86:0/3998805103 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc4b407b9b0 con 0x7fc4c4071ad0 2024-04-20T21:22:34.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.290+0000 7fc4c9000640 1 -- 172.21.15.86:0/3998805103 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc4c4072550 con 0x7fc4c4071ad0 2024-04-20T21:22:34.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.859+0000 7fc4bb400640 1 -- 172.21.15.86:0/3998805103 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v62) ==== 169+0+0 (secure 0 0 0) 0x7fc4b407b2b0 con 0x7fc4c4071ad0 2024-04-20T21:22:34.860 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' removed 2024-04-20T21:22:34.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.863+0000 7fc4c9000640 1 -- 172.21.15.86:0/3998805103 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc4a006c8f0 msgr2=0x7fc4a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:34.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.863+0000 7fc4c9000640 1 --2- 172.21.15.86:0/3998805103 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc4a006c8f0 0x7fc4a006edb0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc4c415a0c0 tx=0x7fc4ac008040 comp rx=0 tx=0).stop 2024-04-20T21:22:34.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.863+0000 7fc4c9000640 1 -- 172.21.15.86:0/3998805103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4c4071ad0 msgr2=0x7fc4c414ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:34.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.863+0000 7fc4c9000640 1 --2- 172.21.15.86:0/3998805103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4c4071ad0 0x7fc4c414ec00 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4034040 tx=0x7fc4b40046c0 comp rx=0 tx=0).stop 2024-04-20T21:22:34.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.865+0000 7fc4c9000640 1 -- 172.21.15.86:0/3998805103 shutdown_connections 2024-04-20T21:22:34.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.865+0000 7fc4c9000640 1 --2- 172.21.15.86:0/3998805103 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc4a006c8f0 0x7fc4a006edb0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:34.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.865+0000 7fc4c9000640 1 --2- 172.21.15.86:0/3998805103 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4c4153590 0x7fc4c41512c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:34.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.865+0000 7fc4c9000640 1 --2- 172.21.15.86:0/3998805103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4c40af160 0x7fc4c40ceb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:34.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.865+0000 7fc4c9000640 1 --2- 172.21.15.86:0/3998805103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4c4071ad0 0x7fc4c414ec00 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:34.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.865+0000 7fc4c9000640 1 -- 172.21.15.86:0/3998805103 >> 172.21.15.86:0/3998805103 conn(0x7fc4c40b2180 msgr2=0x7fc4c4072770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:34.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.866+0000 7fc4c9000640 1 -- 172.21.15.86:0/3998805103 shutdown_connections 2024-04-20T21:22:34.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:34.866+0000 7fc4c9000640 1 -- 172.21.15.86:0/3998805103 wait complete. 2024-04-20T21:22:34.889 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-20T21:22:35.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.080+0000 7f1145e00640 1 Processor -- start 2024-04-20T21:22:35.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.080+0000 7f1145e00640 1 -- start start 2024-04-20T21:22:35.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.082+0000 7f1145e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1140152450 0x7f11400aff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:35.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.082+0000 7f1145e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f11400b5e20 0x7f114006c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:35.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.082+0000 7f1145e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11400b0650 con 0x7f114006ced0 2024-04-20T21:22:35.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.082+0000 7f1145e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f114006d340 con 0x7f1140152450 2024-04-20T21:22:35.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.082+0000 7f1145e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f114006f570 con 0x7f11400b5e20 2024-04-20T21:22:35.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.082+0000 7f113ea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1140152450 0x7f11400aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:35.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.082+0000 7f113ea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1140152450 0x7f11400aff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39012/0 (socket says 172.21.15.86:39012) 2024-04-20T21:22:35.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.082+0000 7f113ea00640 1 -- 172.21.15.86:0/2502970259 learned_addr learned my addr 172.21.15.86:0/2502970259 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:35.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.083+0000 7f113ea00640 1 -- 172.21.15.86:0/2502970259 >> v2:172.21.15.144:3300/0 conn(0x7f11400b5e20 msgr2=0x7f114006c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:35.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.083+0000 7f113ea00640 1 --2- 172.21.15.86:0/2502970259 >> v2:172.21.15.144:3300/0 conn(0x7f11400b5e20 0x7f114006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:35.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.083+0000 7f113ea00640 1 -- 172.21.15.86:0/2502970259 >> v1:172.21.15.144:6789/0 conn(0x7f114006ced0 legacy=0x7f11400b4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:35.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.083+0000 7f113ea00640 1 -- 172.21.15.86:0/2502970259 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f114006f7c0 con 0x7f1140152450 2024-04-20T21:22:35.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.084+0000 7f113ea00640 1 --2- 172.21.15.86:0/2502970259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1140152450 0x7f11400aff00 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f1140070620 tx=0x7f113002e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67a7b2e8470821a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:35.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.084+0000 7f113e000640 1 -- 172.21.15.86:0/2502970259 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113002ea80 con 0x7f1140152450 2024-04-20T21:22:35.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.084+0000 7f113e000640 1 -- 172.21.15.86:0/2502970259 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f113002ec20 con 0x7f1140152450 2024-04-20T21:22:35.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.084+0000 7f113e000640 1 -- 172.21.15.86:0/2502970259 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1130009600 con 0x7f1140152450 2024-04-20T21:22:35.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.085+0000 7f1145e00640 1 -- 172.21.15.86:0/2502970259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1140152450 msgr2=0x7f11400aff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:35.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.085+0000 7f1145e00640 1 --2- 172.21.15.86:0/2502970259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1140152450 0x7f11400aff00 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f1140070620 tx=0x7f113002e0e0 comp rx=0 tx=0).stop 2024-04-20T21:22:35.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.085+0000 7f1145e00640 1 -- 172.21.15.86:0/2502970259 shutdown_connections 2024-04-20T21:22:35.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.085+0000 7f1145e00640 1 --2- 172.21.15.86:0/2502970259 >> v2:172.21.15.144:3300/0 conn(0x7f11400b5e20 0x7f114006c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:35.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.085+0000 7f1145e00640 1 --2- 172.21.15.86:0/2502970259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1140152450 0x7f11400aff00 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:35.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.085+0000 7f1145e00640 1 -- 172.21.15.86:0/2502970259 >> 172.21.15.86:0/2502970259 conn(0x7f11400725c0 msgr2=0x7f11400729d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:35.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.085+0000 7f1145e00640 1 -- 172.21.15.86:0/2502970259 shutdown_connections 2024-04-20T21:22:35.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.085+0000 7f1145e00640 1 -- 172.21.15.86:0/2502970259 wait complete. 2024-04-20T21:22:35.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.087+0000 7f1145e00640 1 Processor -- start 2024-04-20T21:22:35.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.088+0000 7f1145e00640 1 -- start start 2024-04-20T21:22:35.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.089+0000 7f1145e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f114006ced0 0x7f1140150510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:35.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.089+0000 7f1145e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11400b5e20 0x7f114009a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:35.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.089+0000 7f1145e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1140152450 0x7f114009da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:35.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.089+0000 7f1145e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f11400702a0 con 0x7f1140152450 2024-04-20T21:22:35.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.089+0000 7f1145e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f114006fe10 con 0x7f114006ced0 2024-04-20T21:22:35.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.089+0000 7f1145e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f11400ae960 con 0x7f11400b5e20 2024-04-20T21:22:35.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.089+0000 7f113f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1140152450 0x7f114009da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:35.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.089+0000 7f113f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1140152450 0x7f114009da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39014/0 (socket says 172.21.15.86:39014) 2024-04-20T21:22:35.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.089+0000 7f113f400640 1 -- 172.21.15.86:0/1566513733 learned_addr learned my addr 172.21.15.86:0/1566513733 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:35.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.089+0000 7f113ea00640 1 --2- 172.21.15.86:0/1566513733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f114006ced0 0x7f1140150510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:35.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.089+0000 7f113f400640 1 -- 172.21.15.86:0/1566513733 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11400b5e20 msgr2=0x7f114009a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:35.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.089+0000 7f113f400640 1 --2- 172.21.15.86:0/1566513733 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11400b5e20 0x7f114009a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:35.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.089+0000 7f113f400640 1 -- 172.21.15.86:0/1566513733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f114006ced0 msgr2=0x7f1140150510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:35.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.089+0000 7f113f400640 1 --2- 172.21.15.86:0/1566513733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f114006ced0 0x7f1140150510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:35.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.089+0000 7f113f400640 1 -- 172.21.15.86:0/1566513733 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f114015e6d0 con 0x7f1140152450 2024-04-20T21:22:35.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.090+0000 7f113f400640 1 --2- 172.21.15.86:0/1566513733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1140152450 0x7f114009da10 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f113800b490 tx=0x7f113800b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:35.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.090+0000 7f1137e00640 1 -- 172.21.15.86:0/1566513733 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1138015020 con 0x7f1140152450 2024-04-20T21:22:35.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.090+0000 7f1137e00640 1 -- 172.21.15.86:0/1566513733 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1138011850 con 0x7f1140152450 2024-04-20T21:22:35.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.090+0000 7f1137e00640 1 -- 172.21.15.86:0/1566513733 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1138011c40 con 0x7f1140152450 2024-04-20T21:22:35.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.090+0000 7f1145e00640 1 -- 172.21.15.86:0/1566513733 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f114015e9b0 con 0x7f1140152450 2024-04-20T21:22:35.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.090+0000 7f1145e00640 1 -- 172.21.15.86:0/1566513733 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f114009d4f0 con 0x7f1140152450 2024-04-20T21:22:35.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.091+0000 7f1145e00640 1 -- 172.21.15.86:0/1566513733 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f114009d6f0 con 0x7f1140152450 2024-04-20T21:22:35.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.092+0000 7f1137e00640 1 -- 172.21.15.86:0/1566513733 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1138018020 con 0x7f1140152450 2024-04-20T21:22:35.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.094+0000 7f1137e00640 1 --2- 172.21.15.86:0/1566513733 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f111c06c8f0 0x7f111c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:35.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.094+0000 7f1137e00640 1 -- 172.21.15.86:0/1566513733 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 4943+0+0 (secure 0 0 0) 0x7f113808b6f0 con 0x7f1140152450 2024-04-20T21:22:35.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.094+0000 7f113ea00640 1 --2- 172.21.15.86:0/1566513733 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f111c06c8f0 0x7f111c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:35.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.095+0000 7f113ea00640 1 --2- 172.21.15.86:0/1566513733 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f111c06c8f0 0x7f111c06edb0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1130037000 tx=0x7f1130032000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:35.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.098+0000 7f1137e00640 1 -- 172.21.15.86:0/1566513733 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f114009d6f0 con 0x7f1140152450 2024-04-20T21:22:35.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.298+0000 7f1145e00640 1 -- 172.21.15.86:0/1566513733 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f11400b3dd0 con 0x7f1140152450 2024-04-20T21:22:35.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.304+0000 7f1137e00640 1 -- 172.21.15.86:0/1566513733 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) ==== 104+0+0 (secure 0 0 0) 0x7f113801f020 con 0x7f1140152450 2024-04-20T21:22:35.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.307+0000 7f1145e00640 1 -- 172.21.15.86:0/1566513733 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f111c06c8f0 msgr2=0x7f111c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:35.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.307+0000 7f1145e00640 1 --2- 172.21.15.86:0/1566513733 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f111c06c8f0 0x7f111c06edb0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1130037000 tx=0x7f1130032000 comp rx=0 tx=0).stop 2024-04-20T21:22:35.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.307+0000 7f1145e00640 1 -- 172.21.15.86:0/1566513733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1140152450 msgr2=0x7f114009da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:35.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.307+0000 7f1145e00640 1 --2- 172.21.15.86:0/1566513733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1140152450 0x7f114009da10 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f113800b490 tx=0x7f113800b950 comp rx=0 tx=0).stop 2024-04-20T21:22:35.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.307+0000 7f1145e00640 1 -- 172.21.15.86:0/1566513733 shutdown_connections 2024-04-20T21:22:35.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.307+0000 7f1145e00640 1 --2- 172.21.15.86:0/1566513733 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f111c06c8f0 0x7f111c06edb0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:35.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.307+0000 7f1145e00640 1 --2- 172.21.15.86:0/1566513733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1140152450 0x7f114009da10 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:35.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.307+0000 7f1145e00640 1 --2- 172.21.15.86:0/1566513733 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11400b5e20 0x7f114009a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:35.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.307+0000 7f1145e00640 1 --2- 172.21.15.86:0/1566513733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f114006ced0 0x7f1140150510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:35.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.307+0000 7f1145e00640 1 -- 172.21.15.86:0/1566513733 >> 172.21.15.86:0/1566513733 conn(0x7f11400725c0 msgr2=0x7f11400b0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:35.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.307+0000 7f1145e00640 1 -- 172.21.15.86:0/1566513733 shutdown_connections 2024-04-20T21:22:35.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.307+0000 7f1145e00640 1 -- 172.21.15.86:0/1566513733 wait complete. 2024-04-20T21:22:35.326 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-04-20T21:22:35.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.528+0000 7ff5e5c00640 1 Processor -- start 2024-04-20T21:22:35.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.529+0000 7ff5e5c00640 1 -- start start 2024-04-20T21:22:35.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.530+0000 7ff5e5c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff5e015ece0 0x7ff5e015f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:35.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.530+0000 7ff5e5c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff5e00d50c0 0x7ff5e00d3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:35.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.530+0000 7ff5e5c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5e00cf9c0 0x7ff5e0160000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:35.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.530+0000 7ff5e5c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff5e015f600 con 0x7ff5e015ece0 2024-04-20T21:22:35.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.530+0000 7ff5e5c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7ff5e00d3950 con 0x7ff5e00d50c0 2024-04-20T21:22:35.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.530+0000 7ff5e5c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff5e00cec00 con 0x7ff5e00cf9c0 2024-04-20T21:22:35.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.530+0000 7ff5dfe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff5e015ece0 0x7ff5e015f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:35.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.530+0000 7ff5dfe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff5e015ece0 0x7ff5e015f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39022/0 (socket says 172.21.15.86:39022) 2024-04-20T21:22:35.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.530+0000 7ff5dfe00640 1 -- 172.21.15.86:0/2763121710 learned_addr learned my addr 172.21.15.86:0/2763121710 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:35.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.530+0000 7ff5dfe00640 1 -- 172.21.15.86:0/2763121710 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5e00cf9c0 msgr2=0x7ff5e0160000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:35.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.530+0000 7ff5dfe00640 1 --2- 172.21.15.86:0/2763121710 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5e00cf9c0 0x7ff5e0160000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:35.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.530+0000 7ff5dfe00640 1 -- 172.21.15.86:0/2763121710 >> v2:172.21.15.144:3300/0 conn(0x7ff5e00d50c0 msgr2=0x7ff5e00d3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:35.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.530+0000 7ff5dfe00640 1 --2- 172.21.15.86:0/2763121710 >> v2:172.21.15.144:3300/0 conn(0x7ff5e00d50c0 0x7ff5e00d3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:35.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.530+0000 7ff5dfe00640 1 -- 172.21.15.86:0/2763121710 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e00ceee0 con 0x7ff5e015ece0 2024-04-20T21:22:35.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.532+0000 7ff5dfe00640 1 --2- 172.21.15.86:0/2763121710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5e015ece0 0x7ff5e015f0c0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7ff5e00ceac0 tx=0x7ff5d002b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c0582cc0b87ea04 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:35.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.532+0000 7ff5dea00640 1 -- 172.21.15.86:0/2763121710 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d00317d0 con 0x7ff5e015ece0 2024-04-20T21:22:35.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.532+0000 7ff5dea00640 1 -- 172.21.15.86:0/2763121710 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5d0037070 con 0x7ff5e015ece0 2024-04-20T21:22:35.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.532+0000 7ff5dea00640 1 -- 172.21.15.86:0/2763121710 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d0031b90 con 0x7ff5e015ece0 2024-04-20T21:22:35.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.532+0000 7ff5e5c00640 1 -- 172.21.15.86:0/2763121710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5e015ece0 msgr2=0x7ff5e015f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:35.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.532+0000 7ff5e5c00640 1 --2- 172.21.15.86:0/2763121710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5e015ece0 0x7ff5e015f0c0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7ff5e00ceac0 tx=0x7ff5d002b870 comp rx=0 tx=0).stop 2024-04-20T21:22:35.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.533+0000 7ff5e5c00640 1 -- 172.21.15.86:0/2763121710 shutdown_connections 2024-04-20T21:22:35.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.533+0000 7ff5e5c00640 1 --2- 172.21.15.86:0/2763121710 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5e00cf9c0 0x7ff5e0160000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:35.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.533+0000 7ff5e5c00640 1 --2- 172.21.15.86:0/2763121710 >> v2:172.21.15.144:3300/0 conn(0x7ff5e00d50c0 0x7ff5e00d3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:35.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.533+0000 7ff5e5c00640 1 --2- 172.21.15.86:0/2763121710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5e015ece0 0x7ff5e015f0c0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:35.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.533+0000 7ff5e5c00640 1 -- 172.21.15.86:0/2763121710 >> 172.21.15.86:0/2763121710 conn(0x7ff5e008bf70 msgr2=0x7ff5e00d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:35.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.533+0000 7ff5e5c00640 1 -- 172.21.15.86:0/2763121710 shutdown_connections 2024-04-20T21:22:35.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.533+0000 7ff5e5c00640 1 -- 172.21.15.86:0/2763121710 wait complete. 2024-04-20T21:22:35.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.534+0000 7ff5e5c00640 1 Processor -- start 2024-04-20T21:22:35.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.534+0000 7ff5e5c00640 1 -- start start 2024-04-20T21:22:35.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.535+0000 7ff5e5c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5e00cf9c0 0x7ff5e015cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:35.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.535+0000 7ff5e5c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5e00d50c0 0x7ff5e00ca480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:35.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.535+0000 7ff5e5c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5e015ece0 0x7ff5e015be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:35.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.535+0000 7ff5e5c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e00d3640 con 0x7ff5e00d50c0 2024-04-20T21:22:35.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.535+0000 7ff5e5c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e00cf4a0 con 0x7ff5e00cf9c0 2024-04-20T21:22:35.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.535+0000 7ff5e5c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff5e015ea50 con 0x7ff5e015ece0 2024-04-20T21:22:35.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.536+0000 7ff5df400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5e00d50c0 0x7ff5e00ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:35.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.536+0000 7ff5df400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5e00d50c0 0x7ff5e00ca480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39026/0 (socket says 172.21.15.86:39026) 2024-04-20T21:22:35.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.536+0000 7ff5df400640 1 -- 172.21.15.86:0/3515393423 learned_addr learned my addr 172.21.15.86:0/3515393423 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:35.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.536+0000 7ff5dfe00640 1 --2- 172.21.15.86:0/3515393423 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5e00cf9c0 0x7ff5e015cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:35.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.536+0000 7ff5df400640 1 -- 172.21.15.86:0/3515393423 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5e015ece0 msgr2=0x7ff5e015be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:35.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.536+0000 7ff5e5200640 1 --2- 172.21.15.86:0/3515393423 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5e015ece0 0x7ff5e015be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:35.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.536+0000 7ff5df400640 1 --2- 172.21.15.86:0/3515393423 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5e015ece0 0x7ff5e015be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:35.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.536+0000 7ff5df400640 1 -- 172.21.15.86:0/3515393423 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5e00cf9c0 msgr2=0x7ff5e015cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:35.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.536+0000 7ff5df400640 1 --2- 172.21.15.86:0/3515393423 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5e00cf9c0 0x7ff5e015cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:35.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.536+0000 7ff5df400640 1 -- 172.21.15.86:0/3515393423 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e015d0b0 con 0x7ff5e00d50c0 2024-04-20T21:22:35.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.536+0000 7ff5dfe00640 1 --2- 172.21.15.86:0/3515393423 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5e00cf9c0 0x7ff5e015cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:22:35.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.537+0000 7ff5df400640 1 --2- 172.21.15.86:0/3515393423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5e00d50c0 0x7ff5e00ca480 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7ff5c800b480 tx=0x7ff5c800b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:35.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.537+0000 7ff5d7e00640 1 -- 172.21.15.86:0/3515393423 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c8014070 con 0x7ff5e00d50c0 2024-04-20T21:22:35.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.537+0000 7ff5d7e00640 1 -- 172.21.15.86:0/3515393423 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5c8010870 con 0x7ff5e00d50c0 2024-04-20T21:22:35.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.537+0000 7ff5d7e00640 1 -- 172.21.15.86:0/3515393423 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c8010c60 con 0x7ff5e00d50c0 2024-04-20T21:22:35.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.537+0000 7ff5e5c00640 1 -- 172.21.15.86:0/3515393423 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5e00a0bd0 con 0x7ff5e00d50c0 2024-04-20T21:22:35.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.537+0000 7ff5e5c00640 1 -- 172.21.15.86:0/3515393423 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5e00a1190 con 0x7ff5e00d50c0 2024-04-20T21:22:35.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.538+0000 7ff5e5c00640 1 -- 172.21.15.86:0/3515393423 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5e015f420 con 0x7ff5e00d50c0 2024-04-20T21:22:35.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.539+0000 7ff5d7e00640 1 -- 172.21.15.86:0/3515393423 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff5c8017070 con 0x7ff5e00d50c0 2024-04-20T21:22:35.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.540+0000 7ff5d7e00640 1 --2- 172.21.15.86:0/3515393423 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff5bc06c8f0 0x7ff5bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:35.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.540+0000 7ff5d7e00640 1 -- 172.21.15.86:0/3515393423 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 4943+0+0 (secure 0 0 0) 0x7ff5c808b680 con 0x7ff5e00d50c0 2024-04-20T21:22:35.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.540+0000 7ff5dfe00640 1 --2- 172.21.15.86:0/3515393423 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff5bc06c8f0 0x7ff5bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:35.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.541+0000 7ff5dfe00640 1 --2- 172.21.15.86:0/3515393423 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff5bc06c8f0 0x7ff5bc06edb0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0036000 tx=0x7ff5d0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:35.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.544+0000 7ff5d7e00640 1 -- 172.21.15.86:0/3515393423 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff5e015f420 con 0x7ff5e00d50c0 2024-04-20T21:22:35.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.827+0000 7ff5e5c00640 1 -- 172.21.15.86:0/3515393423 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) -- 0x7ff5e004fcf0 con 0x7ff5e00d50c0 2024-04-20T21:22:35.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.862+0000 7ff5d7e00640 1 -- 172.21.15.86:0/3515393423 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v257) ==== 169+0+0 (secure 0 0 0) 0x7ff5c8059da0 con 0x7ff5e00d50c0 2024-04-20T21:22:35.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.865+0000 7ff5e5c00640 1 -- 172.21.15.86:0/3515393423 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff5bc06c8f0 msgr2=0x7ff5bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:35.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.865+0000 7ff5e5c00640 1 --2- 172.21.15.86:0/3515393423 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff5bc06c8f0 0x7ff5bc06edb0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0036000 tx=0x7ff5d0008030 comp rx=0 tx=0).stop 2024-04-20T21:22:35.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.865+0000 7ff5e5c00640 1 -- 172.21.15.86:0/3515393423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5e00d50c0 msgr2=0x7ff5e00ca480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:35.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.865+0000 7ff5e5c00640 1 --2- 172.21.15.86:0/3515393423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5e00d50c0 0x7ff5e00ca480 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7ff5c800b480 tx=0x7ff5c800b940 comp rx=0 tx=0).stop 2024-04-20T21:22:35.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.865+0000 7ff5e5c00640 1 -- 172.21.15.86:0/3515393423 shutdown_connections 2024-04-20T21:22:35.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.865+0000 7ff5e5c00640 1 --2- 172.21.15.86:0/3515393423 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff5bc06c8f0 0x7ff5bc06edb0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:35.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.865+0000 7ff5e5c00640 1 --2- 172.21.15.86:0/3515393423 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5e015ece0 0x7ff5e015be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:35.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.865+0000 7ff5e5c00640 1 --2- 172.21.15.86:0/3515393423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5e00d50c0 0x7ff5e00ca480 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:35.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.865+0000 7ff5e5c00640 1 --2- 172.21.15.86:0/3515393423 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5e00cf9c0 0x7ff5e015cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:35.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.865+0000 7ff5e5c00640 1 -- 172.21.15.86:0/3515393423 >> 172.21.15.86:0/3515393423 conn(0x7ff5e008bf70 msgr2=0x7ff5e015d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:35.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.866+0000 7ff5e5c00640 1 -- 172.21.15.86:0/3515393423 shutdown_connections 2024-04-20T21:22:35.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:35.866+0000 7ff5e5c00640 1 -- 172.21.15.86:0/3515393423 wait complete. 2024-04-20T21:22:35.885 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-04-20T21:22:35.885 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-04-20T21:22:35.885 INFO:tasks.cephfs_test_runner:test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-20T21:22:35.886 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:22:36.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.083+0000 7f2bd5e00640 1 Processor -- start 2024-04-20T21:22:36.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.083+0000 7f2bd5e00640 1 -- start start 2024-04-20T21:22:36.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.085+0000 7f2bd5e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2bd0144e90 0x7f2bd00b4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:36.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.085+0000 7f2bd5e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2bd00cd170 0x7f2bd006ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:36.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.085+0000 7f2bd5e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bd00cdb50 con 0x7f2bd00cd5d0 2024-04-20T21:22:36.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.085+0000 7f2bd5e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f2bd0145270 con 0x7f2bd0144e90 2024-04-20T21:22:36.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.085+0000 7f2bd5e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2bd00b3e50 con 0x7f2bd00cd170 2024-04-20T21:22:36.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.086+0000 7f2bc6000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2bd00cd170 0x7f2bd006ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:36.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.086+0000 7f2bcea00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2bd0144e90 0x7f2bd00b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:36.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.086+0000 7f2bc6000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2bd00cd170 0x7f2bd006ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:59664/0 (socket says 172.21.15.86:59664) 2024-04-20T21:22:36.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.086+0000 7f2bc6000640 1 -- 172.21.15.86:0/3715908005 learned_addr learned my addr 172.21.15.86:0/3715908005 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:36.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.087+0000 7f2bc6000640 1 -- 172.21.15.86:0/3715908005 >> v2:172.21.15.144:3300/0 conn(0x7f2bd0144e90 msgr2=0x7f2bd00b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:36.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.087+0000 7f2bc6000640 1 --2- 172.21.15.86:0/3715908005 >> v2:172.21.15.144:3300/0 conn(0x7f2bd0144e90 0x7f2bd00b4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:36.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.087+0000 7f2bc6000640 1 -- 172.21.15.86:0/3715908005 >> v1:172.21.15.144:6789/0 conn(0x7f2bd00cd5d0 legacy=0x7f2bd015bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:36.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.087+0000 7f2bc6000640 1 -- 172.21.15.86:0/3715908005 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bd00b4130 con 0x7f2bd00cd170 2024-04-20T21:22:36.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.088+0000 7f2bc6000640 1 --2- 172.21.15.86:0/3715908005 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2bd00cd170 0x7f2bd006ed40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8009870 tx=0x7f2bb802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd0bd4a2533e014a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:36.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.088+0000 7f2bce000640 1 -- 172.21.15.86:0/3715908005 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8035020 con 0x7f2bd00cd170 2024-04-20T21:22:36.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.088+0000 7f2bce000640 1 -- 172.21.15.86:0/3715908005 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bb8031640 con 0x7f2bd00cd170 2024-04-20T21:22:36.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.088+0000 7f2bce000640 1 -- 172.21.15.86:0/3715908005 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb802d040 con 0x7f2bd00cd170 2024-04-20T21:22:36.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.089+0000 7f2bd5e00640 1 -- 172.21.15.86:0/3715908005 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2bd00cd170 msgr2=0x7f2bd006ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:36.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.089+0000 7f2bd5e00640 1 --2- 172.21.15.86:0/3715908005 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2bd00cd170 0x7f2bd006ed40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8009870 tx=0x7f2bb802b8f0 comp rx=0 tx=0).stop 2024-04-20T21:22:36.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.089+0000 7f2bd5e00640 1 -- 172.21.15.86:0/3715908005 shutdown_connections 2024-04-20T21:22:36.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.089+0000 7f2bd5e00640 1 --2- 172.21.15.86:0/3715908005 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2bd00cd170 0x7f2bd006ed40 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:36.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.089+0000 7f2bd5e00640 1 --2- 172.21.15.86:0/3715908005 >> v2:172.21.15.144:3300/0 conn(0x7f2bd0144e90 0x7f2bd00b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:36.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.089+0000 7f2bd5e00640 1 -- 172.21.15.86:0/3715908005 >> 172.21.15.86:0/3715908005 conn(0x7f2bd00b2360 msgr2=0x7f2bd00b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:36.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.089+0000 7f2bd5e00640 1 -- 172.21.15.86:0/3715908005 shutdown_connections 2024-04-20T21:22:36.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.089+0000 7f2bd5e00640 1 -- 172.21.15.86:0/3715908005 wait complete. 2024-04-20T21:22:36.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.091+0000 7f2bd5e00640 1 Processor -- start 2024-04-20T21:22:36.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.091+0000 7f2bd5e00640 1 -- start start 2024-04-20T21:22:36.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.092+0000 7f2bd5e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bd00cd170 0x7f2bd0142a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:36.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.092+0000 7f2bd5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bd00cd5d0 0x7f2bd01398b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:36.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.092+0000 7f2bd5e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2bd0144e90 0x7f2bd0139df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:36.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.092+0000 7f2bd5e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd00d0010 con 0x7f2bd00cd5d0 2024-04-20T21:22:36.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.092+0000 7f2bd5e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd00b4e70 con 0x7f2bd00cd170 2024-04-20T21:22:36.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.092+0000 7f2bd5e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2bd00d0190 con 0x7f2bd0144e90 2024-04-20T21:22:36.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.093+0000 7f2bc6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bd00cd5d0 0x7f2bd01398b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:36.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.093+0000 7f2bc6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bd00cd5d0 0x7f2bd01398b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53858/0 (socket says 172.21.15.86:53858) 2024-04-20T21:22:36.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.093+0000 7f2bc6000640 1 -- 172.21.15.86:0/2738992964 learned_addr learned my addr 172.21.15.86:0/2738992964 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:36.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.093+0000 7f2bcea00640 1 --2- 172.21.15.86:0/2738992964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bd00cd170 0x7f2bd0142a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:36.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.093+0000 7f2bcf400640 1 --2- 172.21.15.86:0/2738992964 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2bd0144e90 0x7f2bd0139df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:36.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.093+0000 7f2bc6000640 1 -- 172.21.15.86:0/2738992964 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2bd0144e90 msgr2=0x7f2bd0139df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:36.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.093+0000 7f2bc6000640 1 --2- 172.21.15.86:0/2738992964 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2bd0144e90 0x7f2bd0139df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:36.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.093+0000 7f2bc6000640 1 -- 172.21.15.86:0/2738992964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bd00cd170 msgr2=0x7f2bd0142a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:36.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.093+0000 7f2bc6000640 1 --2- 172.21.15.86:0/2738992964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bd00cd170 0x7f2bd0142a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:36.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.093+0000 7f2bc6000640 1 -- 172.21.15.86:0/2738992964 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bd013a6e0 con 0x7f2bd00cd5d0 2024-04-20T21:22:36.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.094+0000 7f2bc6000640 1 --2- 172.21.15.86:0/2738992964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bd00cd5d0 0x7f2bd01398b0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8000c00 tx=0x7f2bb80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:36.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.094+0000 7f2bc7e00640 1 -- 172.21.15.86:0/2738992964 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8040070 con 0x7f2bd00cd5d0 2024-04-20T21:22:36.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.094+0000 7f2bd5e00640 1 -- 172.21.15.86:0/2738992964 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bd013a9c0 con 0x7f2bd00cd5d0 2024-04-20T21:22:36.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.094+0000 7f2bc7e00640 1 -- 172.21.15.86:0/2738992964 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bb80399f0 con 0x7f2bd00cd5d0 2024-04-20T21:22:36.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.094+0000 7f2bc7e00640 1 -- 172.21.15.86:0/2738992964 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8035040 con 0x7f2bd00cd5d0 2024-04-20T21:22:36.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.094+0000 7f2bd5e00640 1 -- 172.21.15.86:0/2738992964 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bd013af80 con 0x7f2bd00cd5d0 2024-04-20T21:22:36.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.095+0000 7f2bd5e00640 1 -- 172.21.15.86:0/2738992964 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bd00689e0 con 0x7f2bd00cd5d0 2024-04-20T21:22:36.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.096+0000 7f2bc7e00640 1 -- 172.21.15.86:0/2738992964 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2bb804b060 con 0x7f2bd00cd5d0 2024-04-20T21:22:36.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.098+0000 7f2bc7e00640 1 --2- 172.21.15.86:0/2738992964 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2bac06c8f0 0x7f2bac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:36.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.098+0000 7f2bc7e00640 1 -- 172.21.15.86:0/2738992964 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 4943+0+0 (secure 0 0 0) 0x7f2bb80ad050 con 0x7f2bd00cd5d0 2024-04-20T21:22:36.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.098+0000 7f2bcea00640 1 --2- 172.21.15.86:0/2738992964 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2bac06c8f0 0x7f2bac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:36.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.099+0000 7f2bcea00640 1 --2- 172.21.15.86:0/2738992964 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2bac06c8f0 0x7f2bac06edb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0009a30 tx=0x7f2bc0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:36.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.103+0000 7f2bc7e00640 1 -- 172.21.15.86:0/2738992964 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2bb807b810 con 0x7f2bd00cd5d0 2024-04-20T21:22:36.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.316+0000 7f2bd5e00640 1 -- 172.21.15.86:0/2738992964 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2bd006cce0 con 0x7f2bd00cd5d0 2024-04-20T21:22:36.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.318+0000 7f2bc7e00640 1 -- 172.21.15.86:0/2738992964 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) ==== 74+0+13020 (secure 0 0 0) 0x7f2bb807b110 con 0x7f2bd00cd5d0 2024-04-20T21:22:36.318 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:22:36.319 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":62,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:22:34.849731+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6837/3008383261":"2024-04-21T21:22:31.474289+0000","172.21.15.86:6835/3008383261":"2024-04-21T21:22:31.474289+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:22:36.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.321+0000 7f2bd5e00640 1 -- 172.21.15.86:0/2738992964 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2bac06c8f0 msgr2=0x7f2bac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:36.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.321+0000 7f2bd5e00640 1 --2- 172.21.15.86:0/2738992964 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2bac06c8f0 0x7f2bac06edb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0009a30 tx=0x7f2bc0008040 comp rx=0 tx=0).stop 2024-04-20T21:22:36.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.321+0000 7f2bd5e00640 1 -- 172.21.15.86:0/2738992964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bd00cd5d0 msgr2=0x7f2bd01398b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:36.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.321+0000 7f2bd5e00640 1 --2- 172.21.15.86:0/2738992964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bd00cd5d0 0x7f2bd01398b0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8000c00 tx=0x7f2bb80046c0 comp rx=0 tx=0).stop 2024-04-20T21:22:36.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.321+0000 7f2bd5e00640 1 -- 172.21.15.86:0/2738992964 shutdown_connections 2024-04-20T21:22:36.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.321+0000 7f2bd5e00640 1 --2- 172.21.15.86:0/2738992964 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2bac06c8f0 0x7f2bac06edb0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:36.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.321+0000 7f2bd5e00640 1 --2- 172.21.15.86:0/2738992964 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2bd0144e90 0x7f2bd0139df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:36.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.321+0000 7f2bd5e00640 1 --2- 172.21.15.86:0/2738992964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bd00cd5d0 0x7f2bd01398b0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:36.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.321+0000 7f2bd5e00640 1 --2- 172.21.15.86:0/2738992964 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bd00cd170 0x7f2bd0142a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:36.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.321+0000 7f2bd5e00640 1 -- 172.21.15.86:0/2738992964 >> 172.21.15.86:0/2738992964 conn(0x7f2bd00b2360 msgr2=0x7f2bd00b00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:36.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.321+0000 7f2bd5e00640 1 -- 172.21.15.86:0/2738992964 shutdown_connections 2024-04-20T21:22:36.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.321+0000 7f2bd5e00640 1 -- 172.21.15.86:0/2738992964 wait complete. 2024-04-20T21:22:36.340 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:22:36.340 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:22:36.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.529+0000 7ff707000640 1 Processor -- start 2024-04-20T21:22:36.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.529+0000 7ff707000640 1 -- start start 2024-04-20T21:22:36.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.530+0000 7ff707000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7000af1f0 0x7ff70006ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:36.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.530+0000 7ff707000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff70006d9c0 0x7ff70015d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:36.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.530+0000 7ff707000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff70015da60 con 0x7ff7000af5d0 2024-04-20T21:22:36.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.530+0000 7ff707000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7ff70006f460 con 0x7ff70006d9c0 2024-04-20T21:22:36.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.530+0000 7ff707000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff70006de80 con 0x7ff7000af1f0 2024-04-20T21:22:36.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.531+0000 7ff706600640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7ff7000af5d0 0x7ff70015de70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:49752/0 (socket says 172.21.15.86:49752) 2024-04-20T21:22:36.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.531+0000 7ff706600640 1 -- 172.21.15.86:0/587639876 learned_addr learned my addr 172.21.15.86:0/587639876 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:22:36.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.531+0000 7ff705c00640 1 --2- 172.21.15.86:0/587639876 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7000af1f0 0x7ff70006ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:36.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.531+0000 7ff6ffe00640 1 -- 172.21.15.86:0/587639876 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2236655543 0 0) 0x7ff6f8001490 con 0x7ff7000af5d0 2024-04-20T21:22:36.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.531+0000 7ff6ffe00640 1 -- 172.21.15.86:0/587639876 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff6e8003520 con 0x7ff7000af5d0 2024-04-20T21:22:36.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.531+0000 7ff705200640 1 --2- 172.21.15.86:0/587639876 >> v2:172.21.15.144:3300/0 conn(0x7ff70006d9c0 0x7ff70015d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:36.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.532+0000 7ff6ffe00640 1 -- 172.21.15.86:0/587639876 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 697494891 0 0) 0x7ff6e8003520 con 0x7ff7000af5d0 2024-04-20T21:22:36.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.532+0000 7ff6ffe00640 1 -- 172.21.15.86:0/587639876 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7000af1f0 msgr2=0x7ff70006ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:36.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.532+0000 7ff6ffe00640 1 --2- 172.21.15.86:0/587639876 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7000af1f0 0x7ff70006ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:36.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.532+0000 7ff6ffe00640 1 -- 172.21.15.86:0/587639876 >> v2:172.21.15.144:3300/0 conn(0x7ff70006d9c0 msgr2=0x7ff70015d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:36.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.532+0000 7ff6ffe00640 1 --2- 172.21.15.86:0/587639876 >> v2:172.21.15.144:3300/0 conn(0x7ff70006d9c0 0x7ff70015d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:36.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.532+0000 7ff6ffe00640 1 -- 172.21.15.86:0/587639876 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff70006e160 con 0x7ff7000af5d0 2024-04-20T21:22:36.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.532+0000 7ff6ffe00640 1 -- 172.21.15.86:0/587639876 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7ff6f8001bc0 con 0x7ff7000af5d0 2024-04-20T21:22:36.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.532+0000 7ff705200640 1 --2- 172.21.15.86:0/587639876 >> v2:172.21.15.144:3300/0 conn(0x7ff70006d9c0 0x7ff70015d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:22:36.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.532+0000 7ff6ffe00640 1 -- 172.21.15.86:0/587639876 >> v1:172.21.15.86:6789/0 conn(0x7ff7000af5d0 legacy=0x7ff70015de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:36.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.532+0000 7ff6ffe00640 1 --2- 172.21.15.86:0/587639876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6e80039b0 0x7ff6e8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:36.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.532+0000 7ff6ffe00640 1 --2- 172.21.15.86:0/587639876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6e8004800 0x7ff6e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:36.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.532+0000 7ff6ffe00640 1 --2- 172.21.15.86:0/587639876 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7000af1f0 0x7ff6e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:36.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.532+0000 7ff6ffe00640 1 -- 172.21.15.86:0/587639876 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff70006f460 con 0x7ff6e80039b0 2024-04-20T21:22:36.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.532+0000 7ff6ffe00640 1 -- 172.21.15.86:0/587639876 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff70006de80 con 0x7ff6e8004800 2024-04-20T21:22:36.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.532+0000 7ff6ffe00640 1 -- 172.21.15.86:0/587639876 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff6e800ba70 con 0x7ff7000af1f0 2024-04-20T21:22:36.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.532+0000 7ff705c00640 1 --2- 172.21.15.86:0/587639876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6e80039b0 0x7ff6e8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:36.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.532+0000 7ff706600640 1 --2- 172.21.15.86:0/587639876 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7000af1f0 0x7ff6e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:36.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.533+0000 7ff705200640 1 --2- 172.21.15.86:0/587639876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6e8004800 0x7ff6e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:36.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.533+0000 7ff705c00640 1 -- 172.21.15.86:0/587639876 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7000af1f0 msgr2=0x7ff6e80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:36.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.533+0000 7ff705c00640 1 --2- 172.21.15.86:0/587639876 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7000af1f0 0x7ff6e80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:36.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.533+0000 7ff705c00640 1 -- 172.21.15.86:0/587639876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6e8004800 msgr2=0x7ff6e8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:36.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.533+0000 7ff705c00640 1 --2- 172.21.15.86:0/587639876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff6e8004800 0x7ff6e8004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:36.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.533+0000 7ff705c00640 1 -- 172.21.15.86:0/587639876 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6e8003520 con 0x7ff6e80039b0 2024-04-20T21:22:36.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.533+0000 7ff706600640 1 --2- 172.21.15.86:0/587639876 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff7000af1f0 0x7ff6e80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:22:36.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.534+0000 7ff705c00640 1 --2- 172.21.15.86:0/587639876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6e80039b0 0x7ff6e8003dd0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7ff6f400ba00 tx=0x7ff6e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea3dd707daed02b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:36.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.534+0000 7ff6ffe00640 1 -- 172.21.15.86:0/587639876 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f400b600 con 0x7ff6e80039b0 2024-04-20T21:22:36.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.534+0000 7ff6ffe00640 1 -- 172.21.15.86:0/587639876 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6f4038070 con 0x7ff6e80039b0 2024-04-20T21:22:36.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.534+0000 7ff6ffe00640 1 -- 172.21.15.86:0/587639876 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f40338a0 con 0x7ff6e80039b0 2024-04-20T21:22:36.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.535+0000 7ff707000640 1 -- 172.21.15.86:0/587639876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6e80039b0 msgr2=0x7ff6e8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:36.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.535+0000 7ff707000640 1 --2- 172.21.15.86:0/587639876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6e80039b0 0x7ff6e8003dd0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7ff6f400ba00 tx=0x7ff6e8005830 comp rx=0 tx=0).stop 2024-04-20T21:22:36.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.535+0000 7ff706600640 1 -- 172.21.15.86:0/587639876 reap_dead start 2024-04-20T21:22:36.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.535+0000 7ff707000640 1 -- 172.21.15.86:0/587639876 shutdown_connections 2024-04-20T21:22:36.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.535+0000 7ff707000640 1 -- 172.21.15.86:0/587639876 >> 172.21.15.86:0/587639876 conn(0x7ff7000b1f00 msgr2=0x7ff7000b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:36.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.535+0000 7ff707000640 1 -- 172.21.15.86:0/587639876 shutdown_connections 2024-04-20T21:22:36.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.535+0000 7ff707000640 1 -- 172.21.15.86:0/587639876 wait complete. 2024-04-20T21:22:36.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.537+0000 7ff707000640 1 Processor -- start 2024-04-20T21:22:36.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.537+0000 7ff707000640 1 -- start start 2024-04-20T21:22:36.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.538+0000 7ff707000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6e80039b0 0x7ff7001f4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:36.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.538+0000 7ff707000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff70014bb60 0x7ff7000b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:36.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.538+0000 7ff707000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7000b4be0 0x7ff70014dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:36.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.538+0000 7ff707000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff70006ea00 con 0x7ff70014bb60 2024-04-20T21:22:36.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.538+0000 7ff707000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff70006e720 con 0x7ff7000b4be0 2024-04-20T21:22:36.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.538+0000 7ff707000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff7000afa80 con 0x7ff6e80039b0 2024-04-20T21:22:36.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.538+0000 7ff705200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff70014bb60 0x7ff7000b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:36.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.538+0000 7ff705200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff70014bb60 0x7ff7000b46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53876/0 (socket says 172.21.15.86:53876) 2024-04-20T21:22:36.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.538+0000 7ff705200640 1 -- 172.21.15.86:0/2708309447 learned_addr learned my addr 172.21.15.86:0/2708309447 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:36.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.538+0000 7ff706600640 1 --2- 172.21.15.86:0/2708309447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7000b4be0 0x7ff70014dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:36.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.538+0000 7ff705c00640 1 --2- 172.21.15.86:0/2708309447 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6e80039b0 0x7ff7001f4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:36.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.538+0000 7ff705200640 1 -- 172.21.15.86:0/2708309447 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6e80039b0 msgr2=0x7ff7001f4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:36.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.538+0000 7ff705200640 1 --2- 172.21.15.86:0/2708309447 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6e80039b0 0x7ff7001f4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:36.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.538+0000 7ff705200640 1 -- 172.21.15.86:0/2708309447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7000b4be0 msgr2=0x7ff70014dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:36.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.538+0000 7ff705200640 1 --2- 172.21.15.86:0/2708309447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7000b4be0 0x7ff70014dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:36.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.539+0000 7ff705200640 1 -- 172.21.15.86:0/2708309447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff70014ea00 con 0x7ff70014bb60 2024-04-20T21:22:36.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.539+0000 7ff706600640 1 --2- 172.21.15.86:0/2708309447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7000b4be0 0x7ff70014dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:22:36.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.539+0000 7ff705200640 1 --2- 172.21.15.86:0/2708309447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff70014bb60 0x7ff7000b46a0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec00f5d0 tx=0x7ff6ec00fa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:36.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.539+0000 7ff6fd600640 1 -- 172.21.15.86:0/2708309447 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec018070 con 0x7ff70014bb60 2024-04-20T21:22:36.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.539+0000 7ff6fd600640 1 -- 172.21.15.86:0/2708309447 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6ec014890 con 0x7ff70014bb60 2024-04-20T21:22:36.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.539+0000 7ff707000640 1 -- 172.21.15.86:0/2708309447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff70009c9d0 con 0x7ff70014bb60 2024-04-20T21:22:36.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.539+0000 7ff6fd600640 1 -- 172.21.15.86:0/2708309447 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec014c80 con 0x7ff70014bb60 2024-04-20T21:22:36.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.539+0000 7ff707000640 1 -- 172.21.15.86:0/2708309447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff70009c630 con 0x7ff70014bb60 2024-04-20T21:22:36.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.540+0000 7ff707000640 1 -- 172.21.15.86:0/2708309447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff70004fd90 con 0x7ff70014bb60 2024-04-20T21:22:36.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.541+0000 7ff6fd600640 1 -- 172.21.15.86:0/2708309447 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff6ec01b070 con 0x7ff70014bb60 2024-04-20T21:22:36.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.542+0000 7ff6fd600640 1 --2- 172.21.15.86:0/2708309447 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff6e006c920 0x7ff6e006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:36.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.542+0000 7ff6fd600640 1 -- 172.21.15.86:0/2708309447 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 4943+0+0 (secure 0 0 0) 0x7ff6ec08bb10 con 0x7ff70014bb60 2024-04-20T21:22:36.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.543+0000 7ff705c00640 1 --2- 172.21.15.86:0/2708309447 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff6e006c920 0x7ff6e006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:36.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.543+0000 7ff705c00640 1 --2- 172.21.15.86:0/2708309447 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff6e006c920 0x7ff6e006ede0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff6e80057e0 tx=0x7ff6f4030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:36.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.547+0000 7ff6fd600640 1 -- 172.21.15.86:0/2708309447 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff6ec05a1a0 con 0x7ff70014bb60 2024-04-20T21:22:36.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.745+0000 7ff707000640 1 -- 172.21.15.86:0/2708309447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff70006f880 con 0x7ff70014bb60 2024-04-20T21:22:36.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.746+0000 7ff6fd600640 1 -- 172.21.15.86:0/2708309447 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v62) ==== 93+0+10 (secure 0 0 0) 0x7ff6ec014e20 con 0x7ff70014bb60 2024-04-20T21:22:36.746 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:22:36.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.749+0000 7ff707000640 1 -- 172.21.15.86:0/2708309447 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff6e006c920 msgr2=0x7ff6e006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:36.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.749+0000 7ff707000640 1 --2- 172.21.15.86:0/2708309447 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff6e006c920 0x7ff6e006ede0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff6e80057e0 tx=0x7ff6f4030000 comp rx=0 tx=0).stop 2024-04-20T21:22:36.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.749+0000 7ff707000640 1 -- 172.21.15.86:0/2708309447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff70014bb60 msgr2=0x7ff7000b46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:36.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.749+0000 7ff707000640 1 --2- 172.21.15.86:0/2708309447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff70014bb60 0x7ff7000b46a0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec00f5d0 tx=0x7ff6ec00fa90 comp rx=0 tx=0).stop 2024-04-20T21:22:36.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.749+0000 7ff707000640 1 -- 172.21.15.86:0/2708309447 shutdown_connections 2024-04-20T21:22:36.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.749+0000 7ff707000640 1 --2- 172.21.15.86:0/2708309447 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff6e006c920 0x7ff6e006ede0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:36.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.749+0000 7ff707000640 1 --2- 172.21.15.86:0/2708309447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7000b4be0 0x7ff70014dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:36.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.749+0000 7ff707000640 1 --2- 172.21.15.86:0/2708309447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff70014bb60 0x7ff7000b46a0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:36.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.749+0000 7ff707000640 1 --2- 172.21.15.86:0/2708309447 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6e80039b0 0x7ff7001f4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:36.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.749+0000 7ff707000640 1 -- 172.21.15.86:0/2708309447 >> 172.21.15.86:0/2708309447 conn(0x7ff7000b1f00 msgr2=0x7ff7000cecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:36.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.749+0000 7ff707000640 1 -- 172.21.15.86:0/2708309447 shutdown_connections 2024-04-20T21:22:36.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.749+0000 7ff707000640 1 -- 172.21.15.86:0/2708309447 wait complete. 2024-04-20T21:22:36.768 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-04-20T21:22:36.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.962+0000 7fa741400640 1 Processor -- start 2024-04-20T21:22:36.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.963+0000 7fa741400640 1 -- start start 2024-04-20T21:22:36.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.964+0000 7fa741400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa73c072410 0x7fa73c15cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:36.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.964+0000 7fa741400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa73c06eb30 con 0x7fa73c06e6d0 2024-04-20T21:22:36.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.964+0000 7fa741400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa73c06dc30 con 0x7fa73c146130 2024-04-20T21:22:36.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.964+0000 7fa741400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fa73c0cd8e0 con 0x7fa73c072410 2024-04-20T21:22:36.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.964+0000 7fa731e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fa73c06e6d0 0x7fa73c071d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:49762/0 (socket says 172.21.15.86:49762) 2024-04-20T21:22:36.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.964+0000 7fa731e00640 1 -- 172.21.15.86:0/23842828 learned_addr learned my addr 172.21.15.86:0/23842828 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:22:36.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.965+0000 7fa739e00640 1 -- 172.21.15.86:0/23842828 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 716271878 0 0) 0x7fa724001110 con 0x7fa73c06e6d0 2024-04-20T21:22:36.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.965+0000 7fa739e00640 1 -- 172.21.15.86:0/23842828 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa720003520 con 0x7fa73c06e6d0 2024-04-20T21:22:36.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.965+0000 7fa73b200640 1 --2- 172.21.15.86:0/23842828 >> v2:172.21.15.144:3300/0 conn(0x7fa73c072410 0x7fa73c15cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:36.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.965+0000 7fa739e00640 1 -- 172.21.15.86:0/23842828 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 303322158 0 0) 0x7fa720003520 con 0x7fa73c06e6d0 2024-04-20T21:22:36.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.965+0000 7fa739e00640 1 -- 172.21.15.86:0/23842828 >> v2:172.21.15.144:3300/0 conn(0x7fa73c072410 msgr2=0x7fa73c15cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:36.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.965+0000 7fa739e00640 1 --2- 172.21.15.86:0/23842828 >> v2:172.21.15.144:3300/0 conn(0x7fa73c072410 0x7fa73c15cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:36.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.965+0000 7fa739e00640 1 -- 172.21.15.86:0/23842828 >> v1:172.21.15.144:6789/0 conn(0x7fa73c146130 legacy=0x7fa73c0ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:36.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.965+0000 7fa739e00640 1 -- 172.21.15.86:0/23842828 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa73c0cdbc0 con 0x7fa73c06e6d0 2024-04-20T21:22:36.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.965+0000 7fa739e00640 1 -- 172.21.15.86:0/23842828 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fa724001840 con 0x7fa73c06e6d0 2024-04-20T21:22:36.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.965+0000 7fa739e00640 1 -- 172.21.15.86:0/23842828 >> v1:172.21.15.86:6789/0 conn(0x7fa73c06e6d0 legacy=0x7fa73c071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:36.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.965+0000 7fa739e00640 1 --2- 172.21.15.86:0/23842828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7200039b0 0x7fa720003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:36.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.965+0000 7fa739e00640 1 --2- 172.21.15.86:0/23842828 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa720004800 0x7fa720004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:36.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.966+0000 7fa739e00640 1 --2- 172.21.15.86:0/23842828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7200051c0 0x7fa72000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:36.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.966+0000 7fa739e00640 1 -- 172.21.15.86:0/23842828 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa73c0cd8e0 con 0x7fa7200039b0 2024-04-20T21:22:36.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.966+0000 7fa739e00640 1 -- 172.21.15.86:0/23842828 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa72000c0f0 con 0x7fa7200051c0 2024-04-20T21:22:36.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.966+0000 7fa739e00640 1 -- 172.21.15.86:0/23842828 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa72000c2f0 con 0x7fa720004800 2024-04-20T21:22:36.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.966+0000 7fa73a800640 1 --2- 172.21.15.86:0/23842828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7200039b0 0x7fa720003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:36.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.966+0000 7fa731e00640 1 --2- 172.21.15.86:0/23842828 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa720004800 0x7fa720004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:36.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.966+0000 7fa73a800640 1 -- 172.21.15.86:0/23842828 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa720004800 msgr2=0x7fa720004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:36.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.966+0000 7fa73b200640 1 --2- 172.21.15.86:0/23842828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7200051c0 0x7fa72000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:36.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.966+0000 7fa73a800640 1 --2- 172.21.15.86:0/23842828 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa720004800 0x7fa720004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:36.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.966+0000 7fa73a800640 1 -- 172.21.15.86:0/23842828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7200051c0 msgr2=0x7fa72000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:36.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.966+0000 7fa73a800640 1 --2- 172.21.15.86:0/23842828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7200051c0 0x7fa72000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:36.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.966+0000 7fa73b200640 1 -- 172.21.15.86:0/23842828 reap_dead start 2024-04-20T21:22:36.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.966+0000 7fa73a800640 1 -- 172.21.15.86:0/23842828 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa720003520 con 0x7fa7200039b0 2024-04-20T21:22:36.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.967+0000 7fa73a800640 1 --2- 172.21.15.86:0/23842828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7200039b0 0x7fa720003dd0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fa72c001750 tx=0x7fa720005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc2a6d4e9a35d40a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:36.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.968+0000 7fa739e00640 1 -- 172.21.15.86:0/23842828 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72c001370 con 0x7fa7200039b0 2024-04-20T21:22:36.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.968+0000 7fa739e00640 1 -- 172.21.15.86:0/23842828 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa72c0052d0 con 0x7fa7200039b0 2024-04-20T21:22:36.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.968+0000 7fa739e00640 1 -- 172.21.15.86:0/23842828 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72c003830 con 0x7fa7200039b0 2024-04-20T21:22:36.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.968+0000 7fa741400640 1 -- 172.21.15.86:0/23842828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7200039b0 msgr2=0x7fa720003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:36.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.968+0000 7fa741400640 1 --2- 172.21.15.86:0/23842828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7200039b0 0x7fa720003dd0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fa72c001750 tx=0x7fa720005830 comp rx=0 tx=0).stop 2024-04-20T21:22:36.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.968+0000 7fa741400640 1 -- 172.21.15.86:0/23842828 shutdown_connections 2024-04-20T21:22:36.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.968+0000 7fa741400640 1 --2- 172.21.15.86:0/23842828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7200039b0 0x7fa720003dd0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:36.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.968+0000 7fa741400640 1 -- 172.21.15.86:0/23842828 >> 172.21.15.86:0/23842828 conn(0x7fa73c0b2360 msgr2=0x7fa73c0b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:36.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.968+0000 7fa741400640 1 -- 172.21.15.86:0/23842828 shutdown_connections 2024-04-20T21:22:36.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.968+0000 7fa741400640 1 -- 172.21.15.86:0/23842828 wait complete. 2024-04-20T21:22:36.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.970+0000 7fa741400640 1 Processor -- start 2024-04-20T21:22:36.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.970+0000 7fa741400640 1 -- start start 2024-04-20T21:22:36.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.971+0000 7fa741400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7200039b0 0x7fa73c1f5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:36.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.971+0000 7fa741400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa73c15bf50 0x7fa73c144780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:36.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.971+0000 7fa741400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa73c144cc0 0x7fa73c1f80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:36.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.971+0000 7fa741400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa73c06f150 con 0x7fa73c15bf50 2024-04-20T21:22:36.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.971+0000 7fa741400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa73c06e1b0 con 0x7fa73c144cc0 2024-04-20T21:22:36.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.971+0000 7fa741400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa73c071d00 con 0x7fa7200039b0 2024-04-20T21:22:36.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.971+0000 7fa731e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa73c15bf50 0x7fa73c144780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:36.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.971+0000 7fa731e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa73c15bf50 0x7fa73c144780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53900/0 (socket says 172.21.15.86:53900) 2024-04-20T21:22:36.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.971+0000 7fa731e00640 1 -- 172.21.15.86:0/379949007 learned_addr learned my addr 172.21.15.86:0/379949007 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:36.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.972+0000 7fa73a800640 1 --2- 172.21.15.86:0/379949007 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7200039b0 0x7fa73c1f5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:36.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.972+0000 7fa73b200640 1 --2- 172.21.15.86:0/379949007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa73c144cc0 0x7fa73c1f80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:36.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.972+0000 7fa731e00640 1 -- 172.21.15.86:0/379949007 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7200039b0 msgr2=0x7fa73c1f5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:36.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.972+0000 7fa731e00640 1 --2- 172.21.15.86:0/379949007 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7200039b0 0x7fa73c1f5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:36.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.972+0000 7fa731e00640 1 -- 172.21.15.86:0/379949007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa73c144cc0 msgr2=0x7fa73c1f80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:36.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.972+0000 7fa731e00640 1 --2- 172.21.15.86:0/379949007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa73c144cc0 0x7fa73c1f80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:36.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.972+0000 7fa731e00640 1 -- 172.21.15.86:0/379949007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa73c131c70 con 0x7fa73c15bf50 2024-04-20T21:22:36.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.972+0000 7fa731e00640 1 --2- 172.21.15.86:0/379949007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa73c15bf50 0x7fa73c144780 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fa7240013d0 tx=0x7fa72400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:36.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.972+0000 7fa733400640 1 -- 172.21.15.86:0/379949007 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa724019070 con 0x7fa73c15bf50 2024-04-20T21:22:36.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.973+0000 7fa733400640 1 -- 172.21.15.86:0/379949007 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa72400de50 con 0x7fa73c15bf50 2024-04-20T21:22:36.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.973+0000 7fa733400640 1 -- 172.21.15.86:0/379949007 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7240047e0 con 0x7fa73c15bf50 2024-04-20T21:22:36.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.973+0000 7fa741400640 1 -- 172.21.15.86:0/379949007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa73c131f50 con 0x7fa73c15bf50 2024-04-20T21:22:36.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.973+0000 7fa741400640 1 -- 172.21.15.86:0/379949007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa73c132510 con 0x7fa73c15bf50 2024-04-20T21:22:36.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.974+0000 7fa741400640 1 -- 172.21.15.86:0/379949007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa73c0689e0 con 0x7fa73c15bf50 2024-04-20T21:22:36.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.975+0000 7fa733400640 1 -- 172.21.15.86:0/379949007 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa724004980 con 0x7fa73c15bf50 2024-04-20T21:22:36.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.976+0000 7fa733400640 1 --2- 172.21.15.86:0/379949007 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa71806c920 0x7fa71806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:36.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.976+0000 7fa733400640 1 -- 172.21.15.86:0/379949007 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 4943+0+0 (secure 0 0 0) 0x7fa72408b130 con 0x7fa73c15bf50 2024-04-20T21:22:36.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.976+0000 7fa73a800640 1 --2- 172.21.15.86:0/379949007 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa71806c920 0x7fa71806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:36.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.977+0000 7fa73a800640 1 --2- 172.21.15.86:0/379949007 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa71806c920 0x7fa71806ede0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa72c033000 tx=0x7fa72c009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:36.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:36.980+0000 7fa733400640 1 -- 172.21.15.86:0/379949007 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa73c06eb30 con 0x7fa73c15bf50 2024-04-20T21:22:37.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:37.257+0000 7fa741400640 1 -- 172.21.15.86:0/379949007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) -- 0x7fa73c15ff50 con 0x7fa73c15bf50 2024-04-20T21:22:37.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:37.874+0000 7fa733400640 1 -- 172.21.15.86:0/379949007 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v259) ==== 169+0+0 (secure 0 0 0) 0x7fa724004c70 con 0x7fa73c15bf50 2024-04-20T21:22:37.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:37.878+0000 7fa741400640 1 -- 172.21.15.86:0/379949007 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa71806c920 msgr2=0x7fa71806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:37.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:37.878+0000 7fa741400640 1 --2- 172.21.15.86:0/379949007 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa71806c920 0x7fa71806ede0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa72c033000 tx=0x7fa72c009030 comp rx=0 tx=0).stop 2024-04-20T21:22:37.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:37.878+0000 7fa741400640 1 -- 172.21.15.86:0/379949007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa73c15bf50 msgr2=0x7fa73c144780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:37.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:37.878+0000 7fa741400640 1 --2- 172.21.15.86:0/379949007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa73c15bf50 0x7fa73c144780 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fa7240013d0 tx=0x7fa72400dc00 comp rx=0 tx=0).stop 2024-04-20T21:22:37.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:37.879+0000 7fa741400640 1 -- 172.21.15.86:0/379949007 shutdown_connections 2024-04-20T21:22:37.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:37.879+0000 7fa741400640 1 --2- 172.21.15.86:0/379949007 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa71806c920 0x7fa71806ede0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:37.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:37.879+0000 7fa741400640 1 --2- 172.21.15.86:0/379949007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa73c144cc0 0x7fa73c1f80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:37.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:37.879+0000 7fa741400640 1 --2- 172.21.15.86:0/379949007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa73c15bf50 0x7fa73c144780 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:37.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:37.879+0000 7fa741400640 1 --2- 172.21.15.86:0/379949007 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7200039b0 0x7fa73c1f5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:37.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:37.879+0000 7fa741400640 1 -- 172.21.15.86:0/379949007 >> 172.21.15.86:0/379949007 conn(0x7fa73c0b2360 msgr2=0x7fa73c15eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:37.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:37.879+0000 7fa741400640 1 -- 172.21.15.86:0/379949007 shutdown_connections 2024-04-20T21:22:37.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:37.879+0000 7fa741400640 1 -- 172.21.15.86:0/379949007 wait complete. 2024-04-20T21:22:37.900 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-20T21:22:38.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.101+0000 7fc723200640 1 Processor -- start 2024-04-20T21:22:38.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.101+0000 7fc723200640 1 -- start start 2024-04-20T21:22:38.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.102+0000 7fc723200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc71c0b6730 0x7fc71c0707e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:38.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.103+0000 7fc723200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc71c0d03b0 0x7fc71c15d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:38.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.103+0000 7fc723200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc71c0ce960 con 0x7fc71c15cc50 2024-04-20T21:22:38.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.103+0000 7fc723200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc71c155fc0 con 0x7fc71c0d03b0 2024-04-20T21:22:38.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.103+0000 7fc723200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fc71c0b6b10 con 0x7fc71c0b6730 2024-04-20T21:22:38.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.103+0000 7fc722800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc71c0d03b0 0x7fc71c15d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:38.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.103+0000 7fc721400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fc71c15cc50 0x7fc71c0cfca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:49776/0 (socket says 172.21.15.86:49776) 2024-04-20T21:22:38.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.103+0000 7fc721400640 1 -- 172.21.15.86:0/915731472 learned_addr learned my addr 172.21.15.86:0/915731472 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:22:38.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.103+0000 7fc721e00640 1 --2- 172.21.15.86:0/915731472 >> v2:172.21.15.144:3300/0 conn(0x7fc71c0b6730 0x7fc71c0707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:38.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.103+0000 7fc722800640 1 -- 172.21.15.86:0/915731472 >> v2:172.21.15.144:3300/0 conn(0x7fc71c0b6730 msgr2=0x7fc71c0707e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:38.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.103+0000 7fc722800640 1 --2- 172.21.15.86:0/915731472 >> v2:172.21.15.144:3300/0 conn(0x7fc71c0b6730 0x7fc71c0707e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:38.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.103+0000 7fc722800640 1 -- 172.21.15.86:0/915731472 >> v1:172.21.15.86:6789/0 conn(0x7fc71c15cc50 legacy=0x7fc71c0cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:38.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.103+0000 7fc722800640 1 -- 172.21.15.86:0/915731472 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc71c0cec00 con 0x7fc71c0d03b0 2024-04-20T21:22:38.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.105+0000 7fc722800640 1 --2- 172.21.15.86:0/915731472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc71c0d03b0 0x7fc71c15d100 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fc71c068060 tx=0x7fc71802be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cae137c1e4b1908 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:38.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.105+0000 7fc720a00640 1 -- 172.21.15.86:0/915731472 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc718039070 con 0x7fc71c0d03b0 2024-04-20T21:22:38.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.105+0000 7fc720a00640 1 -- 172.21.15.86:0/915731472 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc71802cbe0 con 0x7fc71c0d03b0 2024-04-20T21:22:38.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.105+0000 7fc720a00640 1 -- 172.21.15.86:0/915731472 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc718034440 con 0x7fc71c0d03b0 2024-04-20T21:22:38.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.106+0000 7fc723200640 1 -- 172.21.15.86:0/915731472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc71c0d03b0 msgr2=0x7fc71c15d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:38.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.106+0000 7fc723200640 1 --2- 172.21.15.86:0/915731472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc71c0d03b0 0x7fc71c15d100 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fc71c068060 tx=0x7fc71802be30 comp rx=0 tx=0).stop 2024-04-20T21:22:38.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.106+0000 7fc723200640 1 -- 172.21.15.86:0/915731472 shutdown_connections 2024-04-20T21:22:38.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.106+0000 7fc723200640 1 --2- 172.21.15.86:0/915731472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc71c0d03b0 0x7fc71c15d100 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:38.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.106+0000 7fc723200640 1 --2- 172.21.15.86:0/915731472 >> v2:172.21.15.144:3300/0 conn(0x7fc71c0b6730 0x7fc71c0707e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:38.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.106+0000 7fc723200640 1 -- 172.21.15.86:0/915731472 >> 172.21.15.86:0/915731472 conn(0x7fc71c0733d0 msgr2=0x7fc71c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:38.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.106+0000 7fc723200640 1 -- 172.21.15.86:0/915731472 shutdown_connections 2024-04-20T21:22:38.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.106+0000 7fc723200640 1 -- 172.21.15.86:0/915731472 wait complete. 2024-04-20T21:22:38.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.108+0000 7fc723200640 1 Processor -- start 2024-04-20T21:22:38.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.108+0000 7fc723200640 1 -- start start 2024-04-20T21:22:38.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.108+0000 7fc723200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc71c0b6730 0x7fc71c0b1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:38.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.109+0000 7fc723200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc71c0d03b0 0x7fc71c0aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:38.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.109+0000 7fc723200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc71c15cc50 0x7fc71c153690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:38.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.109+0000 7fc723200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc71c0cf4a0 con 0x7fc71c15cc50 2024-04-20T21:22:38.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.109+0000 7fc723200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc71c0cf1c0 con 0x7fc71c0b6730 2024-04-20T21:22:38.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.109+0000 7fc723200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc71c15c8f0 con 0x7fc71c0d03b0 2024-04-20T21:22:38.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.109+0000 7fc722800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc71c15cc50 0x7fc71c153690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:38.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.109+0000 7fc722800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc71c15cc50 0x7fc71c153690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53916/0 (socket says 172.21.15.86:53916) 2024-04-20T21:22:38.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.109+0000 7fc722800640 1 -- 172.21.15.86:0/2933929834 learned_addr learned my addr 172.21.15.86:0/2933929834 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:38.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.109+0000 7fc721e00640 1 --2- 172.21.15.86:0/2933929834 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc71c0b6730 0x7fc71c0b1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:38.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.109+0000 7fc721400640 1 --2- 172.21.15.86:0/2933929834 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc71c0d03b0 0x7fc71c0aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:38.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.109+0000 7fc722800640 1 -- 172.21.15.86:0/2933929834 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc71c0d03b0 msgr2=0x7fc71c0aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:38.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.109+0000 7fc722800640 1 --2- 172.21.15.86:0/2933929834 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc71c0d03b0 0x7fc71c0aeb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:38.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.109+0000 7fc722800640 1 -- 172.21.15.86:0/2933929834 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc71c0b6730 msgr2=0x7fc71c0b1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:38.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.109+0000 7fc722800640 1 --2- 172.21.15.86:0/2933929834 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc71c0b6730 0x7fc71c0b1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:38.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.109+0000 7fc722800640 1 -- 172.21.15.86:0/2933929834 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc71c0af3a0 con 0x7fc71c15cc50 2024-04-20T21:22:38.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.110+0000 7fc722800640 1 --2- 172.21.15.86:0/2933929834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc71c15cc50 0x7fc71c153690 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fc71802c440 tx=0x7fc718004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:38.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.110+0000 7fc716000640 1 -- 172.21.15.86:0/2933929834 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc718039050 con 0x7fc71c15cc50 2024-04-20T21:22:38.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.110+0000 7fc716000640 1 -- 172.21.15.86:0/2933929834 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc718045070 con 0x7fc71c15cc50 2024-04-20T21:22:38.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.110+0000 7fc723200640 1 -- 172.21.15.86:0/2933929834 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc71c0af680 con 0x7fc71c15cc50 2024-04-20T21:22:38.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.110+0000 7fc716000640 1 -- 172.21.15.86:0/2933929834 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71803e830 con 0x7fc71c15cc50 2024-04-20T21:22:38.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.110+0000 7fc723200640 1 -- 172.21.15.86:0/2933929834 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc71c0afc40 con 0x7fc71c15cc50 2024-04-20T21:22:38.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.111+0000 7fc723200640 1 -- 172.21.15.86:0/2933929834 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc71c04fd90 con 0x7fc71c15cc50 2024-04-20T21:22:38.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.113+0000 7fc716000640 1 -- 172.21.15.86:0/2933929834 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc71803e9d0 con 0x7fc71c15cc50 2024-04-20T21:22:38.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.114+0000 7fc716000640 1 --2- 172.21.15.86:0/2933929834 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc70006c8f0 0x7fc70006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:38.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.114+0000 7fc716000640 1 -- 172.21.15.86:0/2933929834 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 4943+0+0 (secure 0 0 0) 0x7fc7180af0b0 con 0x7fc71c15cc50 2024-04-20T21:22:38.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.114+0000 7fc721e00640 1 --2- 172.21.15.86:0/2933929834 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc70006c8f0 0x7fc70006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:38.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.115+0000 7fc721e00640 1 --2- 172.21.15.86:0/2933929834 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc70006c8f0 0x7fc70006edb0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc710006fd0 tx=0x7fc710008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:38.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.118+0000 7fc716000640 1 -- 172.21.15.86:0/2933929834 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc71807d7c0 con 0x7fc71c15cc50 2024-04-20T21:22:38.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.319+0000 7fc723200640 1 -- 172.21.15.86:0/2933929834 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fc71c152ce0 con 0x7fc71c15cc50 2024-04-20T21:22:38.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.325+0000 7fc716000640 1 -- 172.21.15.86:0/2933929834 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7fc71807d0c0 con 0x7fc71c15cc50 2024-04-20T21:22:38.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.327+0000 7fc723200640 1 -- 172.21.15.86:0/2933929834 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc70006c8f0 msgr2=0x7fc70006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:38.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.327+0000 7fc723200640 1 --2- 172.21.15.86:0/2933929834 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc70006c8f0 0x7fc70006edb0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc710006fd0 tx=0x7fc710008040 comp rx=0 tx=0).stop 2024-04-20T21:22:38.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.327+0000 7fc723200640 1 -- 172.21.15.86:0/2933929834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc71c15cc50 msgr2=0x7fc71c153690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:38.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.327+0000 7fc723200640 1 --2- 172.21.15.86:0/2933929834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc71c15cc50 0x7fc71c153690 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fc71802c440 tx=0x7fc718004a40 comp rx=0 tx=0).stop 2024-04-20T21:22:38.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.327+0000 7fc723200640 1 -- 172.21.15.86:0/2933929834 shutdown_connections 2024-04-20T21:22:38.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.327+0000 7fc723200640 1 --2- 172.21.15.86:0/2933929834 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc70006c8f0 0x7fc70006edb0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:38.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.327+0000 7fc723200640 1 --2- 172.21.15.86:0/2933929834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc71c15cc50 0x7fc71c153690 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:38.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.327+0000 7fc723200640 1 --2- 172.21.15.86:0/2933929834 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc71c0d03b0 0x7fc71c0aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:38.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.328+0000 7fc723200640 1 --2- 172.21.15.86:0/2933929834 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc71c0b6730 0x7fc71c0b1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:38.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.328+0000 7fc723200640 1 -- 172.21.15.86:0/2933929834 >> 172.21.15.86:0/2933929834 conn(0x7fc71c0733d0 msgr2=0x7fc71c159f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:38.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.328+0000 7fc723200640 1 -- 172.21.15.86:0/2933929834 shutdown_connections 2024-04-20T21:22:38.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:38.328+0000 7fc723200640 1 -- 172.21.15.86:0/2933929834 wait complete. 2024-04-20T21:22:38.347 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-20T21:22:38.347 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T21:22:38.445 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T21:22:38.565 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-20T21:22:38.566 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:22:38.566 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:22:38.622 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052731,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248996k,nr_inodes=812249,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T21:22:38.622 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:22:38.622 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:22:38.679 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T21:22:38.679 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:22:38.680 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:22:38.735 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052731,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248996k,nr_inodes=812249,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T21:22:38.736 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:22:38.736 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:22:38.793 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T21:22:38.793 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-20T21:22:38.793 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:22:39.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.005+0000 7f9e8d600640 1 Processor -- start 2024-04-20T21:22:39.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.006+0000 7f9e8d600640 1 -- start start 2024-04-20T21:22:39.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.007+0000 7f9e8d600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9e88071d00 0x7f9e88072180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:39.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.007+0000 7f9e8d600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9e88163200 0x7f9e88163650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:39.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.007+0000 7f9e8d600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e8806fb70 con 0x7f9e8815d2c0 2024-04-20T21:22:39.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.007+0000 7f9e8d600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9e8815d6a0 con 0x7f9e88163200 2024-04-20T21:22:39.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.007+0000 7f9e8d600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f9e88166900 con 0x7f9e88071d00 2024-04-20T21:22:39.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.008+0000 7f9e87400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9e88163200 0x7f9e88163650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:39.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.008+0000 7f9e87400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9e88163200 0x7f9e88163650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53926/0 (socket says 172.21.15.86:53926) 2024-04-20T21:22:39.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.008+0000 7f9e87400640 1 -- 172.21.15.86:0/3208621804 learned_addr learned my addr 172.21.15.86:0/3208621804 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:39.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.008+0000 7f9e86000640 1 --2- 172.21.15.86:0/3208621804 >> v2:172.21.15.144:3300/0 conn(0x7f9e88071d00 0x7f9e88072180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:39.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.008+0000 7f9e87400640 1 -- 172.21.15.86:0/3208621804 >> v2:172.21.15.144:3300/0 conn(0x7f9e88071d00 msgr2=0x7f9e88072180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:39.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.008+0000 7f9e87400640 1 --2- 172.21.15.86:0/3208621804 >> v2:172.21.15.144:3300/0 conn(0x7f9e88071d00 0x7f9e88072180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:39.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.008+0000 7f9e87400640 1 -- 172.21.15.86:0/3208621804 >> v1:172.21.15.144:6789/0 conn(0x7f9e8815d2c0 legacy=0x7f9e8806f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:39.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.008+0000 7f9e87400640 1 -- 172.21.15.86:0/3208621804 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e88166be0 con 0x7f9e88163200 2024-04-20T21:22:39.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.009+0000 7f9e87400640 1 --2- 172.21.15.86:0/3208621804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e88163200 0x7f9e88163650 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f9e880735c0 tx=0x7f9e8002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d52cab7d7ac2783 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:39.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.009+0000 7f9e85600640 1 -- 172.21.15.86:0/3208621804 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e8002ca80 con 0x7f9e88163200 2024-04-20T21:22:39.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.010+0000 7f9e85600640 1 -- 172.21.15.86:0/3208621804 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e8002cc20 con 0x7f9e88163200 2024-04-20T21:22:39.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.010+0000 7f9e85600640 1 -- 172.21.15.86:0/3208621804 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e800343f0 con 0x7f9e88163200 2024-04-20T21:22:39.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.010+0000 7f9e8d600640 1 -- 172.21.15.86:0/3208621804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e88163200 msgr2=0x7f9e88163650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:39.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.010+0000 7f9e8d600640 1 --2- 172.21.15.86:0/3208621804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e88163200 0x7f9e88163650 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f9e880735c0 tx=0x7f9e8002bee0 comp rx=0 tx=0).stop 2024-04-20T21:22:39.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.010+0000 7f9e8d600640 1 -- 172.21.15.86:0/3208621804 shutdown_connections 2024-04-20T21:22:39.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.010+0000 7f9e8d600640 1 --2- 172.21.15.86:0/3208621804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e88163200 0x7f9e88163650 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:39.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.010+0000 7f9e8d600640 1 --2- 172.21.15.86:0/3208621804 >> v2:172.21.15.144:3300/0 conn(0x7f9e88071d00 0x7f9e88072180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:39.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.010+0000 7f9e8d600640 1 -- 172.21.15.86:0/3208621804 >> 172.21.15.86:0/3208621804 conn(0x7f9e880b2360 msgr2=0x7f9e880b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:39.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.010+0000 7f9e8d600640 1 -- 172.21.15.86:0/3208621804 shutdown_connections 2024-04-20T21:22:39.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.010+0000 7f9e8d600640 1 -- 172.21.15.86:0/3208621804 wait complete. 2024-04-20T21:22:39.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.012+0000 7f9e8d600640 1 Processor -- start 2024-04-20T21:22:39.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.012+0000 7f9e8d600640 1 -- start start 2024-04-20T21:22:39.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.013+0000 7f9e8d600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e88071d00 0x7f9e881ec550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:39.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.013+0000 7f9e8d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e8815d2c0 0x7f9e88156660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:39.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.013+0000 7f9e8d600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e88163200 0x7f9e8815bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:39.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.013+0000 7f9e8d600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9e881676c0 con 0x7f9e8815d2c0 2024-04-20T21:22:39.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.013+0000 7f9e8d600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9e881671a0 con 0x7f9e88163200 2024-04-20T21:22:39.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.013+0000 7f9e8d600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9e880b42c0 con 0x7f9e88071d00 2024-04-20T21:22:39.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.013+0000 7f9e86000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e8815d2c0 0x7f9e88156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:39.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.014+0000 7f9e86000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e8815d2c0 0x7f9e88156660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53928/0 (socket says 172.21.15.86:53928) 2024-04-20T21:22:39.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.014+0000 7f9e86000640 1 -- 172.21.15.86:0/1507040059 learned_addr learned my addr 172.21.15.86:0/1507040059 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:39.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.014+0000 7f9e86a00640 1 --2- 172.21.15.86:0/1507040059 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e88071d00 0x7f9e881ec550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:39.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.014+0000 7f9e87400640 1 --2- 172.21.15.86:0/1507040059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e88163200 0x7f9e8815bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:39.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.014+0000 7f9e86000640 1 -- 172.21.15.86:0/1507040059 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e88071d00 msgr2=0x7f9e881ec550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:39.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.014+0000 7f9e86000640 1 --2- 172.21.15.86:0/1507040059 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e88071d00 0x7f9e881ec550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:39.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.014+0000 7f9e86000640 1 -- 172.21.15.86:0/1507040059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e88163200 msgr2=0x7f9e8815bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:39.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.014+0000 7f9e86000640 1 --2- 172.21.15.86:0/1507040059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e88163200 0x7f9e8815bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:39.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.014+0000 7f9e86000640 1 -- 172.21.15.86:0/1507040059 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e881ecc50 con 0x7f9e8815d2c0 2024-04-20T21:22:39.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.014+0000 7f9e86000640 1 --2- 172.21.15.86:0/1507040059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e8815d2c0 0x7f9e88156660 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f9e7400d800 tx=0x7f9e7400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:39.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.015+0000 7f9e7ea00640 1 -- 172.21.15.86:0/1507040059 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7400b6c0 con 0x7f9e8815d2c0 2024-04-20T21:22:39.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.015+0000 7f9e7ea00640 1 -- 172.21.15.86:0/1507040059 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e74019070 con 0x7f9e8815d2c0 2024-04-20T21:22:39.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.015+0000 7f9e7ea00640 1 -- 172.21.15.86:0/1507040059 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7400ba80 con 0x7f9e8815d2c0 2024-04-20T21:22:39.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.015+0000 7f9e8d600640 1 -- 172.21.15.86:0/1507040059 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e881ecf30 con 0x7f9e8815d2c0 2024-04-20T21:22:39.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.015+0000 7f9e8d600640 1 -- 172.21.15.86:0/1507040059 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e881ed4f0 con 0x7f9e8815d2c0 2024-04-20T21:22:39.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.016+0000 7f9e8d600640 1 -- 172.21.15.86:0/1507040059 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e88158040 con 0x7f9e8815d2c0 2024-04-20T21:22:39.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.017+0000 7f9e7ea00640 1 -- 172.21.15.86:0/1507040059 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9e7400bc20 con 0x7f9e8815d2c0 2024-04-20T21:22:39.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.018+0000 7f9e7ea00640 1 --2- 172.21.15.86:0/1507040059 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9e6806c8f0 0x7f9e6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:39.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.018+0000 7f9e7ea00640 1 -- 172.21.15.86:0/1507040059 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 4943+0+0 (secure 0 0 0) 0x7f9e7408bde0 con 0x7f9e8815d2c0 2024-04-20T21:22:39.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.018+0000 7f9e86a00640 1 --2- 172.21.15.86:0/1507040059 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9e6806c8f0 0x7f9e6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:39.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.019+0000 7f9e86a00640 1 --2- 172.21.15.86:0/1507040059 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9e6806c8f0 0x7f9e6806edb0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9e78007fd0 tx=0x7f9e78009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:39.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.022+0000 7f9e7ea00640 1 -- 172.21.15.86:0/1507040059 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9e7401c360 con 0x7f9e8815d2c0 2024-04-20T21:22:39.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.262+0000 7f9e8d600640 1 -- 172.21.15.86:0/1507040059 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9e880689e0 con 0x7f9e8815d2c0 2024-04-20T21:22:39.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.263+0000 7f9e7ea00640 1 -- 172.21.15.86:0/1507040059 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+3875 (secure 0 0 0) 0x7f9e7405a470 con 0x7f9e8815d2c0 2024-04-20T21:22:39.263 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:22:39.263 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":36,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4913,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3933408964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3933408964},{"type":"v1","addr":"172.21.15.144:6833","nonce":3933408964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5221,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/1522070692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":1522070692},{"type":"v1","addr":"172.21.15.144:6835","nonce":1522070692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":6132,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/1352073341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1352073341},{"type":"v1","addr":"172.21.15.86:6836","nonce":1352073341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":6615,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2854702374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2854702374},{"type":"v1","addr":"172.21.15.86:6837","nonce":2854702374}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36}],"filesystems":[]} 2024-04-20T21:22:39.264 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 36 2024-04-20T21:22:39.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.266+0000 7f9e8d600640 1 -- 172.21.15.86:0/1507040059 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9e6806c8f0 msgr2=0x7f9e6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:39.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.266+0000 7f9e8d600640 1 --2- 172.21.15.86:0/1507040059 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9e6806c8f0 0x7f9e6806edb0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9e78007fd0 tx=0x7f9e78009040 comp rx=0 tx=0).stop 2024-04-20T21:22:39.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.266+0000 7f9e8d600640 1 -- 172.21.15.86:0/1507040059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e8815d2c0 msgr2=0x7f9e88156660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:39.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.266+0000 7f9e8d600640 1 --2- 172.21.15.86:0/1507040059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e8815d2c0 0x7f9e88156660 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f9e7400d800 tx=0x7f9e7400dcc0 comp rx=0 tx=0).stop 2024-04-20T21:22:39.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.266+0000 7f9e8d600640 1 -- 172.21.15.86:0/1507040059 shutdown_connections 2024-04-20T21:22:39.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.266+0000 7f9e8d600640 1 --2- 172.21.15.86:0/1507040059 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9e6806c8f0 0x7f9e6806edb0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:39.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.266+0000 7f9e8d600640 1 --2- 172.21.15.86:0/1507040059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e88163200 0x7f9e8815bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:39.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.266+0000 7f9e8d600640 1 --2- 172.21.15.86:0/1507040059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e8815d2c0 0x7f9e88156660 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:39.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.266+0000 7f9e8d600640 1 --2- 172.21.15.86:0/1507040059 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e88071d00 0x7f9e881ec550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:39.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.266+0000 7f9e8d600640 1 -- 172.21.15.86:0/1507040059 >> 172.21.15.86:0/1507040059 conn(0x7f9e880b2360 msgr2=0x7f9e880ce1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:39.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.266+0000 7f9e8d600640 1 -- 172.21.15.86:0/1507040059 shutdown_connections 2024-04-20T21:22:39.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.266+0000 7f9e8d600640 1 -- 172.21.15.86:0/1507040059 wait complete. 2024-04-20T21:22:39.285 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-20T21:22:39.285 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-20T21:22:39.285 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-20T21:22:39.285 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:22:39.285 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-20T21:22:39.286 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-20T21:22:39.286 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-20T21:22:39.286 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:22:39.286 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-20T21:22:39.286 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-20T21:22:39.286 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-20T21:22:39.286 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:22:39.287 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-20T21:22:39.287 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-20T21:22:39.287 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-20T21:22:39.287 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:22:39.303 INFO:tasks.ceph.mds.c:Stopped 2024-04-20T21:22:39.304 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-20T21:22:39.305 INFO:tasks.ceph.mds.a:Stopped 2024-04-20T21:22:39.305 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-20T21:22:39.306 INFO:tasks.ceph.mds.d:Stopped 2024-04-20T21:22:39.306 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-20T21:22:39.316 INFO:tasks.ceph.mds.b:Stopped 2024-04-20T21:22:39.316 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-20T21:22:39.351 INFO:tasks.ceph.mds.c:Started 2024-04-20T21:22:39.352 DEBUG:teuthology.parallel:result is None 2024-04-20T21:22:39.357 INFO:tasks.ceph.mds.a:Started 2024-04-20T21:22:39.358 DEBUG:teuthology.parallel:result is None 2024-04-20T21:22:39.363 INFO:tasks.ceph.mds.d:Started 2024-04-20T21:22:39.363 DEBUG:teuthology.parallel:result is None 2024-04-20T21:22:39.369 INFO:tasks.ceph.mds.b:Started 2024-04-20T21:22:39.369 DEBUG:teuthology.parallel:result is None 2024-04-20T21:22:39.369 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-20T21:22:39.501 INFO:tasks.ceph.mds.c.smithi086.stdout:starting mds.c at 2024-04-20T21:22:39.502 INFO:tasks.ceph.mds.a.smithi086.stdout:starting mds.a at 2024-04-20T21:22:39.508 INFO:tasks.ceph.mds.d.smithi144.stdout:starting mds.d at 2024-04-20T21:22:39.517 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2024-04-20T21:22:39.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.545+0000 7f32f5200640 1 Processor -- start 2024-04-20T21:22:39.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.545+0000 7f32f5200640 1 -- start start 2024-04-20T21:22:39.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.546+0000 7f32f5200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f32f015e690 0x7f32f006ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:39.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.546+0000 7f32f5200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f32f00d2cb0 0x7f32f00d1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:39.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.546+0000 7f32f5200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32f00d3100 con 0x7f32f00d0ee0 2024-04-20T21:22:39.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.546+0000 7f32f5200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f32f00d18b0 con 0x7f32f00d2cb0 2024-04-20T21:22:39.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.546+0000 7f32f5200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f32f00d33b0 con 0x7f32f015e690 2024-04-20T21:22:39.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.546+0000 7f32efe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f32f00d2cb0 0x7f32f00d1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:39.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.547+0000 7f32eea00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f32f00d0ee0 0x7f32f00d25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:49806/0 (socket says 172.21.15.86:49806) 2024-04-20T21:22:39.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.547+0000 7f32eea00640 1 -- 172.21.15.86:0/4282734330 learned_addr learned my addr 172.21.15.86:0/4282734330 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:22:39.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.547+0000 7f32ef400640 1 --2- 172.21.15.86:0/4282734330 >> v2:172.21.15.144:3300/0 conn(0x7f32f015e690 0x7f32f006ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:39.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.547+0000 7f32efe00640 1 -- 172.21.15.86:0/4282734330 >> v2:172.21.15.144:3300/0 conn(0x7f32f015e690 msgr2=0x7f32f006ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:39.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.547+0000 7f32efe00640 1 --2- 172.21.15.86:0/4282734330 >> v2:172.21.15.144:3300/0 conn(0x7f32f015e690 0x7f32f006ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:39.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.547+0000 7f32efe00640 1 -- 172.21.15.86:0/4282734330 >> v1:172.21.15.86:6789/0 conn(0x7f32f00d0ee0 legacy=0x7f32f00d25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:39.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.547+0000 7f32efe00640 1 -- 172.21.15.86:0/4282734330 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32f00d3690 con 0x7f32f00d2cb0 2024-04-20T21:22:39.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.548+0000 7f32efe00640 1 --2- 172.21.15.86:0/4282734330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32f00d2cb0 0x7f32f00d1370 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f32f00a2470 tx=0x7f32e802bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce2e4ffa4c53b1e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:39.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.548+0000 7f32ee000640 1 -- 172.21.15.86:0/4282734330 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e8039070 con 0x7f32f00d2cb0 2024-04-20T21:22:39.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.548+0000 7f32ee000640 1 -- 172.21.15.86:0/4282734330 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32e802cc70 con 0x7f32f00d2cb0 2024-04-20T21:22:39.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.548+0000 7f32ee000640 1 -- 172.21.15.86:0/4282734330 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e8034660 con 0x7f32f00d2cb0 2024-04-20T21:22:39.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.549+0000 7f32f5200640 1 -- 172.21.15.86:0/4282734330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32f00d2cb0 msgr2=0x7f32f00d1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:39.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.549+0000 7f32f5200640 1 --2- 172.21.15.86:0/4282734330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32f00d2cb0 0x7f32f00d1370 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f32f00a2470 tx=0x7f32e802bec0 comp rx=0 tx=0).stop 2024-04-20T21:22:39.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.549+0000 7f32f5200640 1 -- 172.21.15.86:0/4282734330 shutdown_connections 2024-04-20T21:22:39.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.549+0000 7f32f5200640 1 --2- 172.21.15.86:0/4282734330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32f00d2cb0 0x7f32f00d1370 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:39.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.549+0000 7f32f5200640 1 --2- 172.21.15.86:0/4282734330 >> v2:172.21.15.144:3300/0 conn(0x7f32f015e690 0x7f32f006ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:39.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.549+0000 7f32f5200640 1 -- 172.21.15.86:0/4282734330 >> 172.21.15.86:0/4282734330 conn(0x7f32f009d8a0 msgr2=0x7f32f008bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:39.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.549+0000 7f32f5200640 1 -- 172.21.15.86:0/4282734330 shutdown_connections 2024-04-20T21:22:39.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.549+0000 7f32f5200640 1 -- 172.21.15.86:0/4282734330 wait complete. 2024-04-20T21:22:39.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.551+0000 7f32f5200640 1 Processor -- start 2024-04-20T21:22:39.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.551+0000 7f32f5200640 1 -- start start 2024-04-20T21:22:39.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.552+0000 7f32f5200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32f00d0ee0 0x7f32f01502d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:39.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.552+0000 7f32f5200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32f00d2cb0 0x7f32f00cd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:39.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.552+0000 7f32f5200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32f015e690 0x7f32f0155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:39.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.552+0000 7f32f5200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f32f00cef00 con 0x7f32f00d0ee0 2024-04-20T21:22:39.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.552+0000 7f32f5200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f32f00cec20 con 0x7f32f00d2cb0 2024-04-20T21:22:39.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.552+0000 7f32f5200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f32f006d360 con 0x7f32f015e690 2024-04-20T21:22:39.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.552+0000 7f32ef400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32f00d0ee0 0x7f32f01502d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:39.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.552+0000 7f32ef400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32f00d0ee0 0x7f32f01502d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53968/0 (socket says 172.21.15.86:53968) 2024-04-20T21:22:39.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.552+0000 7f32ef400640 1 -- 172.21.15.86:0/2254451688 learned_addr learned my addr 172.21.15.86:0/2254451688 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:39.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.552+0000 7f32efe00640 1 --2- 172.21.15.86:0/2254451688 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32f015e690 0x7f32f0155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:39.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.552+0000 7f32eea00640 1 --2- 172.21.15.86:0/2254451688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32f00d2cb0 0x7f32f00cd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:39.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.552+0000 7f32ef400640 1 -- 172.21.15.86:0/2254451688 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32f015e690 msgr2=0x7f32f0155cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:39.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.552+0000 7f32ef400640 1 --2- 172.21.15.86:0/2254451688 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32f015e690 0x7f32f0155cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:39.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.552+0000 7f32ef400640 1 -- 172.21.15.86:0/2254451688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32f00d2cb0 msgr2=0x7f32f00cd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:39.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.552+0000 7f32ef400640 1 --2- 172.21.15.86:0/2254451688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32f00d2cb0 0x7f32f00cd4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:39.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.552+0000 7f32ef400640 1 -- 172.21.15.86:0/2254451688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32f00cb850 con 0x7f32f00d0ee0 2024-04-20T21:22:39.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.553+0000 7f32ef400640 1 --2- 172.21.15.86:0/2254451688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32f00d0ee0 0x7f32f01502d0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f32e000e9b0 tx=0x7f32e000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:39.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.553+0000 7f32e7400640 1 -- 172.21.15.86:0/2254451688 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e0017070 con 0x7f32f00d0ee0 2024-04-20T21:22:39.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.553+0000 7f32e7400640 1 -- 172.21.15.86:0/2254451688 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32e000c9a0 con 0x7f32f00d0ee0 2024-04-20T21:22:39.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.553+0000 7f32e7400640 1 -- 172.21.15.86:0/2254451688 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e000cd90 con 0x7f32f00d0ee0 2024-04-20T21:22:39.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.553+0000 7f32f5200640 1 -- 172.21.15.86:0/2254451688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32f00cbb30 con 0x7f32f00d0ee0 2024-04-20T21:22:39.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.553+0000 7f32f5200640 1 -- 172.21.15.86:0/2254451688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32f0150810 con 0x7f32f00d0ee0 2024-04-20T21:22:39.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.554+0000 7f32f5200640 1 -- 172.21.15.86:0/2254451688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32f004fcf0 con 0x7f32f00d0ee0 2024-04-20T21:22:39.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.555+0000 7f32e7400640 1 -- 172.21.15.86:0/2254451688 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f32e001a070 con 0x7f32f00d0ee0 2024-04-20T21:22:39.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.557+0000 7f32e7400640 1 --2- 172.21.15.86:0/2254451688 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f32d006c8f0 0x7f32d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:39.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.557+0000 7f32e7400640 1 -- 172.21.15.86:0/2254451688 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 4943+0+0 (secure 0 0 0) 0x7f32e008bb40 con 0x7f32f00d0ee0 2024-04-20T21:22:39.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.557+0000 7f32eea00640 1 --2- 172.21.15.86:0/2254451688 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f32d006c8f0 0x7f32d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:39.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.558+0000 7f32eea00640 1 --2- 172.21.15.86:0/2254451688 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f32d006c8f0 0x7f32d006edb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f32f00c97f0 tx=0x7f32dc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:39.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.561+0000 7f32e7400640 1 -- 172.21.15.86:0/2254451688 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f32e0059c10 con 0x7f32f00d0ee0 2024-04-20T21:22:39.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:39.766+0000 7f32f5200640 1 -- 172.21.15.86:0/2254451688 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f32f006e7a0 con 0x7f32f00d0ee0 2024-04-20T21:22:40.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:40.913+0000 7f32e7400640 1 -- 172.21.15.86:0/2254451688 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v67) ==== 97+0+0 (secure 0 0 0) 0x7f32e005df10 con 0x7f32f00d0ee0 2024-04-20T21:22:40.914 INFO:teuthology.orchestra.run.smithi086.stderr: removed all blocklist entries 2024-04-20T21:22:40.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:40.918+0000 7f32f5200640 1 -- 172.21.15.86:0/2254451688 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f32d006c8f0 msgr2=0x7f32d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:40.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:40.918+0000 7f32f5200640 1 --2- 172.21.15.86:0/2254451688 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f32d006c8f0 0x7f32d006edb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f32f00c97f0 tx=0x7f32dc009040 comp rx=0 tx=0).stop 2024-04-20T21:22:40.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:40.918+0000 7f32f5200640 1 -- 172.21.15.86:0/2254451688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32f00d0ee0 msgr2=0x7f32f01502d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:40.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:40.918+0000 7f32f5200640 1 --2- 172.21.15.86:0/2254451688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32f00d0ee0 0x7f32f01502d0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f32e000e9b0 tx=0x7f32e000ee70 comp rx=0 tx=0).stop 2024-04-20T21:22:40.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:40.918+0000 7f32f5200640 1 -- 172.21.15.86:0/2254451688 shutdown_connections 2024-04-20T21:22:40.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:40.918+0000 7f32f5200640 1 --2- 172.21.15.86:0/2254451688 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f32d006c8f0 0x7f32d006edb0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:40.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:40.918+0000 7f32f5200640 1 --2- 172.21.15.86:0/2254451688 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f32f015e690 0x7f32f0155cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:40.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:40.918+0000 7f32f5200640 1 --2- 172.21.15.86:0/2254451688 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32f00d2cb0 0x7f32f00cd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:40.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:40.918+0000 7f32f5200640 1 --2- 172.21.15.86:0/2254451688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32f00d0ee0 0x7f32f01502d0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:40.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:40.918+0000 7f32f5200640 1 -- 172.21.15.86:0/2254451688 >> 172.21.15.86:0/2254451688 conn(0x7f32f009d8a0 msgr2=0x7f32f0159160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:40.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:40.918+0000 7f32f5200640 1 -- 172.21.15.86:0/2254451688 shutdown_connections 2024-04-20T21:22:40.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:40.918+0000 7f32f5200640 1 -- 172.21.15.86:0/2254451688 wait complete. 2024-04-20T21:22:40.937 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-20T21:22:41.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.136+0000 7f7135800640 1 Processor -- start 2024-04-20T21:22:41.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.137+0000 7f7135800640 1 -- start start 2024-04-20T21:22:41.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.138+0000 7f7135800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f713015e870 0x7f71300b3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:41.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.138+0000 7f7135800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f71300b45b0 0x7f71300d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:41.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.138+0000 7f7135800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f71300cf550 0x7f71300cf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:41.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.138+0000 7f7135800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f71300b5c90 con 0x7f71300b45b0 2024-04-20T21:22:41.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.138+0000 7f7135800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f71300cfef0 con 0x7f71300cf550 2024-04-20T21:22:41.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.138+0000 7f7135800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f71300d2be0 con 0x7f713015e870 2024-04-20T21:22:41.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.138+0000 7f712f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f71300b45b0 0x7f71300d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:41.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.138+0000 7f712f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f71300b45b0 0x7f71300d26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53982/0 (socket says 172.21.15.86:53982) 2024-04-20T21:22:41.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.138+0000 7f712f400640 1 -- 172.21.15.86:0/1042344141 learned_addr learned my addr 172.21.15.86:0/1042344141 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:41.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.139+0000 7f712f400640 1 -- 172.21.15.86:0/1042344141 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f713015e870 msgr2=0x7f71300b3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:41.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.139+0000 7f712f400640 1 --2- 172.21.15.86:0/1042344141 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f713015e870 0x7f71300b3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:41.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.139+0000 7f712f400640 1 -- 172.21.15.86:0/1042344141 >> v2:172.21.15.144:3300/0 conn(0x7f71300cf550 msgr2=0x7f71300cf9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:41.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.139+0000 7f7134e00640 1 --2- 172.21.15.86:0/1042344141 >> v2:172.21.15.144:3300/0 conn(0x7f71300cf550 0x7f71300cf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:41.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.139+0000 7f712f400640 1 --2- 172.21.15.86:0/1042344141 >> v2:172.21.15.144:3300/0 conn(0x7f71300cf550 0x7f71300cf9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:41.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.139+0000 7f712f400640 1 -- 172.21.15.86:0/1042344141 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71300b4b50 con 0x7f71300b45b0 2024-04-20T21:22:41.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.139+0000 7f7134e00640 1 --2- 172.21.15.86:0/1042344141 >> v2:172.21.15.144:3300/0 conn(0x7f71300cf550 0x7f71300cf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:22:41.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.140+0000 7f712f400640 1 --2- 172.21.15.86:0/1042344141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71300b45b0 0x7f71300d26a0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f71301623a0 tx=0x7f711802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61ebb2f4eab684dc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:41.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.140+0000 7f712ea00640 1 -- 172.21.15.86:0/1042344141 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711802ca80 con 0x7f71300b45b0 2024-04-20T21:22:41.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.140+0000 7f712ea00640 1 -- 172.21.15.86:0/1042344141 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f711802cc20 con 0x7f71300b45b0 2024-04-20T21:22:41.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.140+0000 7f712ea00640 1 -- 172.21.15.86:0/1042344141 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71180343f0 con 0x7f71300b45b0 2024-04-20T21:22:41.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.141+0000 7f7135800640 1 -- 172.21.15.86:0/1042344141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71300b45b0 msgr2=0x7f71300d26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:41.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.141+0000 7f7135800640 1 --2- 172.21.15.86:0/1042344141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71300b45b0 0x7f71300d26a0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f71301623a0 tx=0x7f711802bee0 comp rx=0 tx=0).stop 2024-04-20T21:22:41.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.141+0000 7f7135800640 1 -- 172.21.15.86:0/1042344141 shutdown_connections 2024-04-20T21:22:41.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.141+0000 7f7135800640 1 --2- 172.21.15.86:0/1042344141 >> v2:172.21.15.144:3300/0 conn(0x7f71300cf550 0x7f71300cf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:41.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.141+0000 7f7135800640 1 --2- 172.21.15.86:0/1042344141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71300b45b0 0x7f71300d26a0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:41.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.141+0000 7f7135800640 1 --2- 172.21.15.86:0/1042344141 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f713015e870 0x7f71300b3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:41.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.141+0000 7f7135800640 1 -- 172.21.15.86:0/1042344141 >> 172.21.15.86:0/1042344141 conn(0x7f713009d880 msgr2=0x7f713008bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:41.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.141+0000 7f7135800640 1 -- 172.21.15.86:0/1042344141 shutdown_connections 2024-04-20T21:22:41.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.141+0000 7f7135800640 1 -- 172.21.15.86:0/1042344141 wait complete. 2024-04-20T21:22:41.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.143+0000 7f7135800640 1 Processor -- start 2024-04-20T21:22:41.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.143+0000 7f7135800640 1 -- start start 2024-04-20T21:22:41.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.144+0000 7f7135800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71300b45b0 0x7f71300b70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:41.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.144+0000 7f7135800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71300cf550 0x7f713015d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:41.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.144+0000 7f7135800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f713015e870 0x7f71300caa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:41.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.144+0000 7f7135800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f71300b53f0 con 0x7f71300cf550 2024-04-20T21:22:41.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.144+0000 7f7135800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f71300b5110 con 0x7f71300b45b0 2024-04-20T21:22:41.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.144+0000 7f7135800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f713015ec50 con 0x7f713015e870 2024-04-20T21:22:41.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.144+0000 7f712f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71300cf550 0x7f713015d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:41.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.144+0000 7f712f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71300cf550 0x7f713015d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53986/0 (socket says 172.21.15.86:53986) 2024-04-20T21:22:41.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.144+0000 7f712f400640 1 -- 172.21.15.86:0/1793210873 learned_addr learned my addr 172.21.15.86:0/1793210873 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:41.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.145+0000 7f712f400640 1 -- 172.21.15.86:0/1793210873 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f713015e870 msgr2=0x7f71300caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:41.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.145+0000 7f712f400640 1 --2- 172.21.15.86:0/1793210873 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f713015e870 0x7f71300caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:41.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.145+0000 7f712f400640 1 -- 172.21.15.86:0/1793210873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71300b45b0 msgr2=0x7f71300b70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:41.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.145+0000 7f712f400640 1 --2- 172.21.15.86:0/1793210873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71300b45b0 0x7f71300b70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:41.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.145+0000 7f712f400640 1 -- 172.21.15.86:0/1793210873 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71300cd960 con 0x7f71300cf550 2024-04-20T21:22:41.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.145+0000 7f712f400640 1 --2- 172.21.15.86:0/1793210873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71300cf550 0x7f713015d520 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f7118036040 tx=0x7f7118004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:41.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.145+0000 7f7127e00640 1 -- 172.21.15.86:0/1793210873 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7118043020 con 0x7f71300cf550 2024-04-20T21:22:41.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.145+0000 7f7127e00640 1 -- 172.21.15.86:0/1793210873 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f711803e450 con 0x7f71300cf550 2024-04-20T21:22:41.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.145+0000 7f7135800640 1 -- 172.21.15.86:0/1793210873 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71300c7ef0 con 0x7f71300cf550 2024-04-20T21:22:41.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.145+0000 7f7127e00640 1 -- 172.21.15.86:0/1793210873 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711803e840 con 0x7f71300cf550 2024-04-20T21:22:41.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.145+0000 7f7135800640 1 -- 172.21.15.86:0/1793210873 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71300c8190 con 0x7f71300cf550 2024-04-20T21:22:41.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.147+0000 7f7135800640 1 -- 172.21.15.86:0/1793210873 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f713004fcf0 con 0x7f71300cf550 2024-04-20T21:22:41.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.147+0000 7f7127e00640 1 -- 172.21.15.86:0/1793210873 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7118046020 con 0x7f71300cf550 2024-04-20T21:22:41.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.148+0000 7f7127e00640 1 --2- 172.21.15.86:0/1793210873 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f710c06c8f0 0x7f710c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:41.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.148+0000 7f7127e00640 1 -- 172.21.15.86:0/1793210873 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 4857+0+0 (secure 0 0 0) 0x7f71180ae210 con 0x7f71300cf550 2024-04-20T21:22:41.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.149+0000 7f712fe00640 1 --2- 172.21.15.86:0/1793210873 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f710c06c8f0 0x7f710c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:41.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.149+0000 7f712fe00640 1 --2- 172.21.15.86:0/1793210873 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f710c06c8f0 0x7f710c06edb0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7120009670 tx=0x7f7120006bf0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:41.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.152+0000 7f7127e00640 1 -- 172.21.15.86:0/1793210873 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f711804d030 con 0x7f71300cf550 2024-04-20T21:22:41.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.424+0000 7f7135800640 1 -- 172.21.15.86:0/1793210873 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f71300b4020 con 0x7f71300cf550 2024-04-20T21:22:41.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.425+0000 7f7127e00640 1 -- 172.21.15.86:0/1793210873 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) ==== 80+0+5304 (secure 0 0 0) 0x7f711807c980 con 0x7f71300cf550 2024-04-20T21:22:41.426 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:22:41.426 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:22:41.426 INFO:teuthology.orchestra.run.smithi086.stdout: "auth_dump": [ 2024-04-20T21:22:41.426 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:22:41.426 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.a", 2024-04-20T21:22:41.426 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDAMCRmOpS/LRAAPZUUQNf6TyeJ2tF7YPnIKw==", 2024-04-20T21:22:41.426 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:22:41.426 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:22:41.426 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-20T21:22:41.426 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-20T21:22:41.426 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:22:41.426 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:22:41.426 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:22:41.427 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:22:41.427 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.b", 2024-04-20T21:22:41.427 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDBMCRmCJLpDRAA/JiLRSAlQRNMpDC+qDSr0g==", 2024-04-20T21:22:41.427 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:22:41.427 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:22:41.427 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-20T21:22:41.427 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-20T21:22:41.427 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:22:41.427 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:22:41.427 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:22:41.427 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:22:41.427 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.c", 2024-04-20T21:22:41.427 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDAMCRmPHOFOxAA4tUsw/n6oSKrNmzPD+NKyw==", 2024-04-20T21:22:41.427 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:22:41.428 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:22:41.428 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-20T21:22:41.428 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-20T21:22:41.428 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:22:41.428 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:22:41.428 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:22:41.428 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:22:41.428 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.d", 2024-04-20T21:22:41.428 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDBMCRmZnPXHBAA0jXwBmGehaO7qens4boRIQ==", 2024-04-20T21:22:41.428 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:22:41.428 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:22:41.428 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-20T21:22:41.428 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-20T21:22:41.428 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:22:41.428 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:22:41.429 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:22:41.429 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:22:41.429 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.0", 2024-04-20T21:22:41.429 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDFMCRm4w9UHhAA56y4dJyDASI+IoZv2XrEoQ==", 2024-04-20T21:22:41.429 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:22:41.429 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:22:41.429 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:22:41.429 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:22:41.429 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:22:41.429 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:22:41.429 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:22:41.429 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.1", 2024-04-20T21:22:41.429 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDIMCRmFo7xKRAAou5B04+J1gZtTzW/Mo+nRQ==", 2024-04-20T21:22:41.429 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:22:41.430 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:22:41.430 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:22:41.430 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:22:41.430 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:22:41.430 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:22:41.430 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:22:41.430 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.2", 2024-04-20T21:22:41.430 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDLMCRm7/jFNRAA4KcRsapPFLaKf0NaH5MApg==", 2024-04-20T21:22:41.430 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:22:41.430 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:22:41.430 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:22:41.430 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:22:41.430 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:22:41.430 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:22:41.431 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:22:41.431 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.3", 2024-04-20T21:22:41.431 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDPMCRmUTPMExAA0/dubb4ehsWIwZPS/ao68w==", 2024-04-20T21:22:41.431 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:22:41.431 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:22:41.431 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:22:41.431 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:22:41.431 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:22:41.431 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:22:41.431 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:22:41.431 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.4", 2024-04-20T21:22:41.431 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDVMCRmB4F2KRAA8Xj0DfH5Ou8TUrO8X/BQyw==", 2024-04-20T21:22:41.431 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:22:41.431 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:22:41.431 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:22:41.432 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:22:41.432 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:22:41.432 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:22:41.432 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:22:41.432 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.5", 2024-04-20T21:22:41.432 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDYMCRmSOvYNRAAODYXmRHMGPrCAxcHwlS7Ww==", 2024-04-20T21:22:41.432 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:22:41.432 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:22:41.432 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:22:41.432 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:22:41.432 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:22:41.432 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:22:41.432 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:22:41.433 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.6", 2024-04-20T21:22:41.433 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDcMCRmENJ9BBAAi/KBUhbYGNXEyiSjSHavWQ==", 2024-04-20T21:22:41.433 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:22:41.433 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:22:41.433 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:22:41.433 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:22:41.433 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:22:41.433 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:22:41.433 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:22:41.433 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.7", 2024-04-20T21:22:41.433 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDfMCRmfTHTDhAALL9GZBSGjWYrhgFw/YI6IA==", 2024-04-20T21:22:41.433 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:22:41.433 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:22:41.433 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:22:41.433 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:22:41.434 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:22:41.434 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:22:41.434 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:22:41.434 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.0", 2024-04-20T21:22:41.434 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDBMCRmHWxyKRAANzhRi4eajp68h/jWkhFMnw==", 2024-04-20T21:22:41.434 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:22:41.434 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:22:41.434 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow r", 2024-04-20T21:22:41.434 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-20T21:22:41.434 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:22:41.434 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:22:41.434 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:22:41.434 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.1", 2024-04-20T21:22:41.434 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDBMCRmgBJAOxAADJLzO97tjEpKn6kUs8g2QA==", 2024-04-20T21:22:41.434 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:22:41.435 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:22:41.435 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow r", 2024-04-20T21:22:41.435 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow rw", 2024-04-20T21:22:41.435 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rwx" 2024-04-20T21:22:41.435 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:22:41.435 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:22:41.435 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:22:41.435 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.2", 2024-04-20T21:22:41.435 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDBMCRmQAdJMhAAiUmKhPEeuV+UPqQ3gynG9w==", 2024-04-20T21:22:41.435 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:22:41.435 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:22:41.435 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow r", 2024-04-20T21:22:41.435 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow rw", 2024-04-20T21:22:41.436 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rwx" 2024-04-20T21:22:41.436 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:22:41.436 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:22:41.436 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:22:41.436 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.3", 2024-04-20T21:22:41.436 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDCMCRm8yMSCBAAFgSVW0XYu7fa40YxHITljQ==", 2024-04-20T21:22:41.436 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:22:41.436 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:22:41.436 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow r", 2024-04-20T21:22:41.436 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow rw", 2024-04-20T21:22:41.436 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rwx" 2024-04-20T21:22:41.436 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:22:41.436 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:22:41.436 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:22:41.436 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.admin", 2024-04-20T21:22:41.437 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQC/MCRmyru0KBAApSqv1hZRcLvrRwPMnqylqQ==", 2024-04-20T21:22:41.437 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:22:41.437 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-04-20T21:22:41.437 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-20T21:22:41.437 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-20T21:22:41.437 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:22:41.437 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:22:41.437 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:22:41.437 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:22:41.437 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mgr.x", 2024-04-20T21:22:41.437 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDAMCRmvTgjIxAACeSV8gNkL5LV3xzPS6VPeg==", 2024-04-20T21:22:41.437 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:22:41.437 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-04-20T21:22:41.437 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile mgr", 2024-04-20T21:22:41.437 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:22:41.438 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:22:41.438 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:22:41.438 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:22:41.438 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mgr.y", 2024-04-20T21:22:41.438 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDAMCRmSn3VGBAA0eu3R53f3gFgscWFMRkDzg==", 2024-04-20T21:22:41.438 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:22:41.438 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-04-20T21:22:41.438 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile mgr", 2024-04-20T21:22:41.438 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:22:41.438 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:22:41.438 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:22:41.438 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-20T21:22:41.438 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:22:41.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.428+0000 7f7135800640 1 -- 172.21.15.86:0/1793210873 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f710c06c8f0 msgr2=0x7f710c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:41.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.428+0000 7f7135800640 1 --2- 172.21.15.86:0/1793210873 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f710c06c8f0 0x7f710c06edb0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7120009670 tx=0x7f7120006bf0 comp rx=0 tx=0).stop 2024-04-20T21:22:41.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.428+0000 7f7135800640 1 -- 172.21.15.86:0/1793210873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71300cf550 msgr2=0x7f713015d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:41.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.428+0000 7f7135800640 1 --2- 172.21.15.86:0/1793210873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71300cf550 0x7f713015d520 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f7118036040 tx=0x7f7118004a40 comp rx=0 tx=0).stop 2024-04-20T21:22:41.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.428+0000 7f7135800640 1 -- 172.21.15.86:0/1793210873 shutdown_connections 2024-04-20T21:22:41.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.428+0000 7f7135800640 1 --2- 172.21.15.86:0/1793210873 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f710c06c8f0 0x7f710c06edb0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:41.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.428+0000 7f7135800640 1 --2- 172.21.15.86:0/1793210873 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f713015e870 0x7f71300caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:41.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.428+0000 7f7135800640 1 --2- 172.21.15.86:0/1793210873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f71300cf550 0x7f713015d520 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:41.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.428+0000 7f7135800640 1 --2- 172.21.15.86:0/1793210873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f71300b45b0 0x7f71300b70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:41.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.428+0000 7f7135800640 1 -- 172.21.15.86:0/1793210873 >> 172.21.15.86:0/1793210873 conn(0x7f713009d880 msgr2=0x7f713006d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:41.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.428+0000 7f7135800640 1 -- 172.21.15.86:0/1793210873 shutdown_connections 2024-04-20T21:22:41.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.428+0000 7f7135800640 1 -- 172.21.15.86:0/1793210873 wait complete. 2024-04-20T21:22:41.448 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:22:41.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.632+0000 7f2b0d800640 1 Processor -- start 2024-04-20T21:22:41.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.633+0000 7f2b0d800640 1 -- start start 2024-04-20T21:22:41.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.634+0000 7f2b0d800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2b0815cc70 0x7f2b080d06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:41.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.634+0000 7f2b0d800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2b080cddd0 0x7f2b080d0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:41.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.634+0000 7f2b0d800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b080ce2e0 con 0x7f2b080d1270 2024-04-20T21:22:41.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.634+0000 7f2b0d800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f2b080ce5c0 con 0x7f2b0815cc70 2024-04-20T21:22:41.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.634+0000 7f2b0d800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f2b080c6270 con 0x7f2b080cddd0 2024-04-20T21:22:41.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.634+0000 7f2b06a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2b0815cc70 0x7f2b080d06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:41.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.635+0000 7f2b06a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2b0815cc70 0x7f2b080d06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53998/0 (socket says 172.21.15.86:53998) 2024-04-20T21:22:41.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.635+0000 7f2b06a00640 1 -- 172.21.15.86:0/4338271 learned_addr learned my addr 172.21.15.86:0/4338271 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:41.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.635+0000 7f2b06a00640 1 -- 172.21.15.86:0/4338271 >> v2:172.21.15.144:3300/0 conn(0x7f2b080cddd0 msgr2=0x7f2b080d0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:41.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.635+0000 7f2b06a00640 1 --2- 172.21.15.86:0/4338271 >> v2:172.21.15.144:3300/0 conn(0x7f2b080cddd0 0x7f2b080d0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:41.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.635+0000 7f2b06a00640 1 -- 172.21.15.86:0/4338271 >> v1:172.21.15.144:6789/0 conn(0x7f2b080d1270 legacy=0x7f2b080c6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:41.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.635+0000 7f2b06a00640 1 -- 172.21.15.86:0/4338271 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b080c6550 con 0x7f2b0815cc70 2024-04-20T21:22:41.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.636+0000 7f2b06a00640 1 --2- 172.21.15.86:0/4338271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b0815cc70 0x7f2b080d06b0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f2b081607d0 tx=0x7f2af802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a27f97a7c37f40ca server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:41.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.637+0000 7f2b05600640 1 -- 172.21.15.86:0/4338271 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af802e070 con 0x7f2b0815cc70 2024-04-20T21:22:41.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.637+0000 7f2b05600640 1 -- 172.21.15.86:0/4338271 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2af8037070 con 0x7f2b0815cc70 2024-04-20T21:22:41.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.637+0000 7f2b05600640 1 -- 172.21.15.86:0/4338271 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af8031a00 con 0x7f2b0815cc70 2024-04-20T21:22:41.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.637+0000 7f2b0d800640 1 -- 172.21.15.86:0/4338271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b0815cc70 msgr2=0x7f2b080d06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:41.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.637+0000 7f2b0d800640 1 --2- 172.21.15.86:0/4338271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b0815cc70 0x7f2b080d06b0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f2b081607d0 tx=0x7f2af802b880 comp rx=0 tx=0).stop 2024-04-20T21:22:41.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.637+0000 7f2b0d800640 1 -- 172.21.15.86:0/4338271 shutdown_connections 2024-04-20T21:22:41.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.637+0000 7f2b0d800640 1 --2- 172.21.15.86:0/4338271 >> v2:172.21.15.144:3300/0 conn(0x7f2b080cddd0 0x7f2b080d0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:41.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.637+0000 7f2b0d800640 1 --2- 172.21.15.86:0/4338271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b0815cc70 0x7f2b080d06b0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:41.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.637+0000 7f2b0d800640 1 -- 172.21.15.86:0/4338271 >> 172.21.15.86:0/4338271 conn(0x7f2b080b4090 msgr2=0x7f2b080b44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:41.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.638+0000 7f2b0d800640 1 -- 172.21.15.86:0/4338271 shutdown_connections 2024-04-20T21:22:41.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.638+0000 7f2b0d800640 1 -- 172.21.15.86:0/4338271 wait complete. 2024-04-20T21:22:41.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.640+0000 7f2b0d800640 1 Processor -- start 2024-04-20T21:22:41.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.640+0000 7f2b0d800640 1 -- start start 2024-04-20T21:22:41.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.641+0000 7f2b0d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b080cddd0 0x7f2b0815b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:41.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.641+0000 7f2b0d800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2b080d1270 0x7f2b080b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:41.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.641+0000 7f2b0d800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b0815cc70 0x7f2b080b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:41.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.641+0000 7f2b0d800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0809bb70 con 0x7f2b080cddd0 2024-04-20T21:22:41.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.641+0000 7f2b0d800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0809b9f0 con 0x7f2b0815cc70 2024-04-20T21:22:41.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.641+0000 7f2b0d800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2b0809bcf0 con 0x7f2b080d1270 2024-04-20T21:22:41.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.641+0000 7f2b06a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b080cddd0 0x7f2b0815b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:41.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.642+0000 7f2b06a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b080cddd0 0x7f2b0815b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54004/0 (socket says 172.21.15.86:54004) 2024-04-20T21:22:41.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.642+0000 7f2b06a00640 1 -- 172.21.15.86:0/4082622068 learned_addr learned my addr 172.21.15.86:0/4082622068 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:41.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.642+0000 7f2b06000640 1 --2- 172.21.15.86:0/4082622068 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2b080d1270 0x7f2b080b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:41.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.642+0000 7f2b07400640 1 --2- 172.21.15.86:0/4082622068 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b0815cc70 0x7f2b080b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:41.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.642+0000 7f2b06a00640 1 -- 172.21.15.86:0/4082622068 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2b080d1270 msgr2=0x7f2b080b72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:41.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.642+0000 7f2b06a00640 1 --2- 172.21.15.86:0/4082622068 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2b080d1270 0x7f2b080b72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:41.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.642+0000 7f2b06a00640 1 -- 172.21.15.86:0/4082622068 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b0815cc70 msgr2=0x7f2b080b77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:41.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.642+0000 7f2b06a00640 1 --2- 172.21.15.86:0/4082622068 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b0815cc70 0x7f2b080b77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:41.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.642+0000 7f2b06a00640 1 -- 172.21.15.86:0/4082622068 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b080c9710 con 0x7f2b080cddd0 2024-04-20T21:22:41.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.642+0000 7f2b06000640 1 --2- 172.21.15.86:0/4082622068 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2b080d1270 0x7f2b080b72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:22:41.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.642+0000 7f2b07400640 1 --2- 172.21.15.86:0/4082622068 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b0815cc70 0x7f2b080b77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:22:41.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.642+0000 7f2b06a00640 1 --2- 172.21.15.86:0/4082622068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b080cddd0 0x7f2b0815b210 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f2af8000c00 tx=0x7f2af80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:41.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.643+0000 7f2afea00640 1 -- 172.21.15.86:0/4082622068 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af8032880 con 0x7f2b080cddd0 2024-04-20T21:22:41.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.643+0000 7f2afea00640 1 -- 172.21.15.86:0/4082622068 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2af8032a20 con 0x7f2b080cddd0 2024-04-20T21:22:41.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.643+0000 7f2afea00640 1 -- 172.21.15.86:0/4082622068 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af8032e10 con 0x7f2b080cddd0 2024-04-20T21:22:41.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.643+0000 7f2b0d800640 1 -- 172.21.15.86:0/4082622068 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b080c99b0 con 0x7f2b080cddd0 2024-04-20T21:22:41.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.643+0000 7f2b0d800640 1 -- 172.21.15.86:0/4082622068 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b080ca490 con 0x7f2b080cddd0 2024-04-20T21:22:41.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.644+0000 7f2b0d800640 1 -- 172.21.15.86:0/4082622068 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b080d04a0 con 0x7f2b080cddd0 2024-04-20T21:22:41.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.645+0000 7f2afea00640 1 -- 172.21.15.86:0/4082622068 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2af80453f0 con 0x7f2b080cddd0 2024-04-20T21:22:41.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.646+0000 7f2afea00640 1 --2- 172.21.15.86:0/4082622068 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2ae406c8f0 0x7f2ae406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:41.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.646+0000 7f2afea00640 1 -- 172.21.15.86:0/4082622068 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 4857+0+0 (secure 0 0 0) 0x7f2af80ad740 con 0x7f2b080cddd0 2024-04-20T21:22:41.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.646+0000 7f2b06000640 1 --2- 172.21.15.86:0/4082622068 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2ae406c8f0 0x7f2ae406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:41.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.647+0000 7f2b06000640 1 --2- 172.21.15.86:0/4082622068 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2ae406c8f0 0x7f2ae406edb0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2b080af830 tx=0x7f2af00073c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:41.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.651+0000 7f2afea00640 1 -- 172.21.15.86:0/4082622068 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2b080d04a0 con 0x7f2b080cddd0 2024-04-20T21:22:41.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.843+0000 7f2b0d800640 1 -- 172.21.15.86:0/4082622068 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2b080690f0 con 0x7f2b080cddd0 2024-04-20T21:22:41.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.844+0000 7f2afea00640 1 -- 172.21.15.86:0/4082622068 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) ==== 74+0+12891 (secure 0 0 0) 0x7f2af8040070 con 0x7f2b080cddd0 2024-04-20T21:22:41.845 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:22:41.845 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":67,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:22:40.911216+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:22:41.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.847+0000 7f2b0d800640 1 -- 172.21.15.86:0/4082622068 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2ae406c8f0 msgr2=0x7f2ae406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:41.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.847+0000 7f2b0d800640 1 --2- 172.21.15.86:0/4082622068 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2ae406c8f0 0x7f2ae406edb0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2b080af830 tx=0x7f2af00073c0 comp rx=0 tx=0).stop 2024-04-20T21:22:41.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.847+0000 7f2b0d800640 1 -- 172.21.15.86:0/4082622068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b080cddd0 msgr2=0x7f2b0815b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:41.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.847+0000 7f2b0d800640 1 --2- 172.21.15.86:0/4082622068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b080cddd0 0x7f2b0815b210 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f2af8000c00 tx=0x7f2af80046c0 comp rx=0 tx=0).stop 2024-04-20T21:22:41.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.847+0000 7f2b0d800640 1 -- 172.21.15.86:0/4082622068 shutdown_connections 2024-04-20T21:22:41.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.847+0000 7f2b0d800640 1 --2- 172.21.15.86:0/4082622068 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2ae406c8f0 0x7f2ae406edb0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:41.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.847+0000 7f2b0d800640 1 --2- 172.21.15.86:0/4082622068 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b0815cc70 0x7f2b080b77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:41.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.847+0000 7f2b0d800640 1 --2- 172.21.15.86:0/4082622068 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2b080d1270 0x7f2b080b72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:41.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.847+0000 7f2b0d800640 1 --2- 172.21.15.86:0/4082622068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b080cddd0 0x7f2b0815b210 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:41.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.847+0000 7f2b0d800640 1 -- 172.21.15.86:0/4082622068 >> 172.21.15.86:0/4082622068 conn(0x7f2b080b4090 msgr2=0x7f2b0806d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:41.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.847+0000 7f2b0d800640 1 -- 172.21.15.86:0/4082622068 shutdown_connections 2024-04-20T21:22:41.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:41.847+0000 7f2b0d800640 1 -- 172.21.15.86:0/4082622068 wait complete. 2024-04-20T21:22:41.864 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:22:41.865 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:22:42.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.058+0000 7f9582c00640 1 Processor -- start 2024-04-20T21:22:42.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.058+0000 7f9582c00640 1 -- start start 2024-04-20T21:22:42.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.060+0000 7f9582c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f957c06bbf0 0x7f957c06e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:42.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.060+0000 7f9582c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f957c06bfd0 0x7f957c145b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:42.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.060+0000 7f9582c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f957c146040 0x7f957c0cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:42.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.060+0000 7f9582c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f957c15cd40 con 0x7f957c146040 2024-04-20T21:22:42.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.060+0000 7f9582c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f957c15d000 con 0x7f957c06bbf0 2024-04-20T21:22:42.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.060+0000 7f9582c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f957c15d2e0 con 0x7f957c06bfd0 2024-04-20T21:22:42.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.060+0000 7f9580c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f957c146040 0x7f957c0cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:42.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.060+0000 7f9580c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f957c146040 0x7f957c0cd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54008/0 (socket says 172.21.15.86:54008) 2024-04-20T21:22:42.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.060+0000 7f9580c00640 1 -- 172.21.15.86:0/1154808458 learned_addr learned my addr 172.21.15.86:0/1154808458 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:42.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.060+0000 7f957be00640 1 --2- 172.21.15.86:0/1154808458 >> v2:172.21.15.144:3300/0 conn(0x7f957c06bbf0 0x7f957c06e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:42.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.061+0000 7f957b400640 1 --2- 172.21.15.86:0/1154808458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f957c06bfd0 0x7f957c145b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:42.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.061+0000 7f9580c00640 1 -- 172.21.15.86:0/1154808458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f957c06bfd0 msgr2=0x7f957c145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:42.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.061+0000 7f9580c00640 1 --2- 172.21.15.86:0/1154808458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f957c06bfd0 0x7f957c145b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:42.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.061+0000 7f9580c00640 1 -- 172.21.15.86:0/1154808458 >> v2:172.21.15.144:3300/0 conn(0x7f957c06bbf0 msgr2=0x7f957c06e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:42.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.061+0000 7f9580c00640 1 --2- 172.21.15.86:0/1154808458 >> v2:172.21.15.144:3300/0 conn(0x7f957c06bbf0 0x7f957c06e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:42.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.061+0000 7f9580c00640 1 -- 172.21.15.86:0/1154808458 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f957c15d5c0 con 0x7f957c146040 2024-04-20T21:22:42.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.062+0000 7f9580c00640 1 --2- 172.21.15.86:0/1154808458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f957c146040 0x7f957c0cd540 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f957c073150 tx=0x7f957402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=949dd597b9eee9ed server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:42.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.062+0000 7f957aa00640 1 -- 172.21.15.86:0/1154808458 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957402ca70 con 0x7f957c146040 2024-04-20T21:22:42.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.062+0000 7f957aa00640 1 -- 172.21.15.86:0/1154808458 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f957402cc10 con 0x7f957c146040 2024-04-20T21:22:42.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.062+0000 7f957aa00640 1 -- 172.21.15.86:0/1154808458 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95740343f0 con 0x7f957c146040 2024-04-20T21:22:42.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.063+0000 7f9582c00640 1 -- 172.21.15.86:0/1154808458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f957c146040 msgr2=0x7f957c0cd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:42.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.063+0000 7f9582c00640 1 --2- 172.21.15.86:0/1154808458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f957c146040 0x7f957c0cd540 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f957c073150 tx=0x7f957402bed0 comp rx=0 tx=0).stop 2024-04-20T21:22:42.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.063+0000 7f9582c00640 1 -- 172.21.15.86:0/1154808458 shutdown_connections 2024-04-20T21:22:42.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.063+0000 7f9582c00640 1 --2- 172.21.15.86:0/1154808458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f957c146040 0x7f957c0cd540 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:42.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.063+0000 7f9582c00640 1 --2- 172.21.15.86:0/1154808458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f957c06bfd0 0x7f957c145b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:42.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.063+0000 7f9582c00640 1 --2- 172.21.15.86:0/1154808458 >> v2:172.21.15.144:3300/0 conn(0x7f957c06bbf0 0x7f957c06e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:42.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.063+0000 7f9582c00640 1 -- 172.21.15.86:0/1154808458 >> 172.21.15.86:0/1154808458 conn(0x7f957c0b2110 msgr2=0x7f957c0b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:42.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.063+0000 7f9582c00640 1 -- 172.21.15.86:0/1154808458 shutdown_connections 2024-04-20T21:22:42.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.063+0000 7f9582c00640 1 -- 172.21.15.86:0/1154808458 wait complete. 2024-04-20T21:22:42.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.065+0000 7f9582c00640 1 Processor -- start 2024-04-20T21:22:42.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.066+0000 7f9582c00640 1 -- start start 2024-04-20T21:22:42.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.067+0000 7f9582c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f957c06bbf0 0x7f957c1f8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:42.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.067+0000 7f9582c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f957c06bfd0 0x7f957c143600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:42.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.067+0000 7f9582c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f957c146040 0x7f957c152490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:42.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.067+0000 7f9582c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f957c0cdc00 con 0x7f957c06bfd0 2024-04-20T21:22:42.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.067+0000 7f9582c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f957c0cda80 con 0x7f957c146040 2024-04-20T21:22:42.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.067+0000 7f9582c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f957c0cdd80 con 0x7f957c06bbf0 2024-04-20T21:22:42.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.067+0000 7f957b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f957c06bfd0 0x7f957c143600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:42.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.067+0000 7f957b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f957c06bfd0 0x7f957c143600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54020/0 (socket says 172.21.15.86:54020) 2024-04-20T21:22:42.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.067+0000 7f957b400640 1 -- 172.21.15.86:0/2486292819 learned_addr learned my addr 172.21.15.86:0/2486292819 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:42.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.067+0000 7f957b400640 1 -- 172.21.15.86:0/2486292819 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f957c06bbf0 msgr2=0x7f957c1f8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:42.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.067+0000 7f957b400640 1 --2- 172.21.15.86:0/2486292819 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f957c06bbf0 0x7f957c1f8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:42.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.067+0000 7f957b400640 1 -- 172.21.15.86:0/2486292819 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f957c146040 msgr2=0x7f957c152490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:42.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.067+0000 7f957b400640 1 --2- 172.21.15.86:0/2486292819 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f957c146040 0x7f957c152490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:42.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.067+0000 7f957b400640 1 -- 172.21.15.86:0/2486292819 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f957c143b40 con 0x7f957c06bfd0 2024-04-20T21:22:42.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.068+0000 7f957b400640 1 --2- 172.21.15.86:0/2486292819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f957c06bfd0 0x7f957c143600 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f956800d620 tx=0x7f956800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:42.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.068+0000 7f9573e00640 1 -- 172.21.15.86:0/2486292819 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956800b620 con 0x7f957c06bfd0 2024-04-20T21:22:42.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.068+0000 7f9573e00640 1 -- 172.21.15.86:0/2486292819 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9568019070 con 0x7f957c06bfd0 2024-04-20T21:22:42.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.068+0000 7f9573e00640 1 -- 172.21.15.86:0/2486292819 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956800ba00 con 0x7f957c06bfd0 2024-04-20T21:22:42.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.068+0000 7f9582c00640 1 -- 172.21.15.86:0/2486292819 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f957c143dc0 con 0x7f957c06bfd0 2024-04-20T21:22:42.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.068+0000 7f9582c00640 1 -- 172.21.15.86:0/2486292819 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f957c144380 con 0x7f957c06bfd0 2024-04-20T21:22:42.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.069+0000 7f9582c00640 1 -- 172.21.15.86:0/2486292819 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f957c06f6f0 con 0x7f957c06bfd0 2024-04-20T21:22:42.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.070+0000 7f9573e00640 1 -- 172.21.15.86:0/2486292819 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f956800bba0 con 0x7f957c06bfd0 2024-04-20T21:22:42.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.072+0000 7f9573e00640 1 --2- 172.21.15.86:0/2486292819 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f955c06c8f0 0x7f955c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:42.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.072+0000 7f9573e00640 1 -- 172.21.15.86:0/2486292819 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 4857+0+0 (secure 0 0 0) 0x7f956808bc90 con 0x7f957c06bfd0 2024-04-20T21:22:42.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.072+0000 7f957be00640 1 --2- 172.21.15.86:0/2486292819 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f955c06c8f0 0x7f955c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:42.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.073+0000 7f957be00640 1 --2- 172.21.15.86:0/2486292819 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f955c06c8f0 0x7f955c06edb0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f956c00b7f0 tx=0x7f956c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:42.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.076+0000 7f9573e00640 1 -- 172.21.15.86:0/2486292819 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f957c06f6f0 con 0x7f957c06bfd0 2024-04-20T21:22:42.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.258+0000 7f9582c00640 1 -- 172.21.15.86:0/2486292819 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f957c14cd50 con 0x7f957c06bfd0 2024-04-20T21:22:42.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.259+0000 7f9573e00640 1 -- 172.21.15.86:0/2486292819 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v67) ==== 93+0+10 (secure 0 0 0) 0x7f9568060070 con 0x7f957c06bfd0 2024-04-20T21:22:42.259 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:22:42.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.261+0000 7f9582c00640 1 -- 172.21.15.86:0/2486292819 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f955c06c8f0 msgr2=0x7f955c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:42.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.261+0000 7f9582c00640 1 --2- 172.21.15.86:0/2486292819 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f955c06c8f0 0x7f955c06edb0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f956c00b7f0 tx=0x7f956c008040 comp rx=0 tx=0).stop 2024-04-20T21:22:42.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.261+0000 7f9582c00640 1 -- 172.21.15.86:0/2486292819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f957c06bfd0 msgr2=0x7f957c143600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:42.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.261+0000 7f9582c00640 1 --2- 172.21.15.86:0/2486292819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f957c06bfd0 0x7f957c143600 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f956800d620 tx=0x7f956800dae0 comp rx=0 tx=0).stop 2024-04-20T21:22:42.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.261+0000 7f9582c00640 1 -- 172.21.15.86:0/2486292819 shutdown_connections 2024-04-20T21:22:42.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.261+0000 7f9582c00640 1 --2- 172.21.15.86:0/2486292819 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f955c06c8f0 0x7f955c06edb0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:42.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.261+0000 7f9582c00640 1 --2- 172.21.15.86:0/2486292819 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f957c146040 0x7f957c152490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:42.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.261+0000 7f9582c00640 1 --2- 172.21.15.86:0/2486292819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f957c06bfd0 0x7f957c143600 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:42.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.261+0000 7f9582c00640 1 --2- 172.21.15.86:0/2486292819 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f957c06bbf0 0x7f957c1f8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:42.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.261+0000 7f9582c00640 1 -- 172.21.15.86:0/2486292819 >> 172.21.15.86:0/2486292819 conn(0x7f957c0b2110 msgr2=0x7f957c15f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:42.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.261+0000 7f9582c00640 1 -- 172.21.15.86:0/2486292819 shutdown_connections 2024-04-20T21:22:42.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.261+0000 7f9582c00640 1 -- 172.21.15.86:0/2486292819 wait complete. 2024-04-20T21:22:42.280 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-20T21:22:42.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.481+0000 7ff48de00640 1 Processor -- start 2024-04-20T21:22:42.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.481+0000 7ff48de00640 1 -- start start 2024-04-20T21:22:42.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.483+0000 7ff48de00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff488156ae0 0x7ff48815d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:42.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.483+0000 7ff48de00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff4880ad7f0 0x7ff48806e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:42.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.483+0000 7ff48de00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff488160890 con 0x7ff488160080 2024-04-20T21:22:42.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.483+0000 7ff48de00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff48806db20 con 0x7ff4880ad7f0 2024-04-20T21:22:42.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.483+0000 7ff48de00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff488160b70 con 0x7ff488156ae0 2024-04-20T21:22:42.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.483+0000 7ff48d400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff4880ad7f0 0x7ff48806e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:42.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.483+0000 7ff48d400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff4880ad7f0 0x7ff48806e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54036/0 (socket says 172.21.15.86:54036) 2024-04-20T21:22:42.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.483+0000 7ff48d400640 1 -- 172.21.15.86:0/947742450 learned_addr learned my addr 172.21.15.86:0/947742450 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:42.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.483+0000 7ff487e00640 1 --1- 172.21.15.86:0/947742450 >> v1:172.21.15.86:6789/0 conn(0x7ff488160080 0x7ff4880ad0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:49822/0 (socket says 172.21.15.86:49822) 2024-04-20T21:22:42.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.483+0000 7ff48ca00640 1 --2- 172.21.15.86:0/947742450 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff488156ae0 0x7ff48815d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:42.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.484+0000 7ff48d400640 1 -- 172.21.15.86:0/947742450 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff488156ae0 msgr2=0x7ff48815d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:42.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.484+0000 7ff48d400640 1 --2- 172.21.15.86:0/947742450 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff488156ae0 0x7ff48815d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:42.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.484+0000 7ff48d400640 1 -- 172.21.15.86:0/947742450 >> v1:172.21.15.86:6789/0 conn(0x7ff488160080 legacy=0x7ff4880ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:42.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.484+0000 7ff487400640 1 -- 172.21.15.86:0/947742450 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3259447673 0 0) 0x7ff488160890 con 0x7ff488160080 2024-04-20T21:22:42.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.484+0000 7ff48d400640 1 -- 172.21.15.86:0/947742450 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff488160e50 con 0x7ff4880ad7f0 2024-04-20T21:22:42.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.485+0000 7ff48d400640 1 --2- 172.21.15.86:0/947742450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4880ad7f0 0x7ff48806e290 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7ff480009e50 tx=0x7ff48002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1301335d9ba206ac server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:42.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.485+0000 7ff487400640 1 -- 172.21.15.86:0/947742450 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48002cb00 con 0x7ff4880ad7f0 2024-04-20T21:22:42.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.486+0000 7ff487400640 1 -- 172.21.15.86:0/947742450 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff48002cca0 con 0x7ff4880ad7f0 2024-04-20T21:22:42.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.486+0000 7ff487400640 1 -- 172.21.15.86:0/947742450 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff480034610 con 0x7ff4880ad7f0 2024-04-20T21:22:42.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.486+0000 7ff48de00640 1 -- 172.21.15.86:0/947742450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4880ad7f0 msgr2=0x7ff48806e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:42.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.486+0000 7ff48de00640 1 --2- 172.21.15.86:0/947742450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4880ad7f0 0x7ff48806e290 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7ff480009e50 tx=0x7ff48002bf60 comp rx=0 tx=0).stop 2024-04-20T21:22:42.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.486+0000 7ff48de00640 1 -- 172.21.15.86:0/947742450 shutdown_connections 2024-04-20T21:22:42.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.486+0000 7ff48de00640 1 --2- 172.21.15.86:0/947742450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4880ad7f0 0x7ff48806e290 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:42.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.486+0000 7ff48de00640 1 --2- 172.21.15.86:0/947742450 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff488156ae0 0x7ff48815d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:42.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.486+0000 7ff48de00640 1 -- 172.21.15.86:0/947742450 >> 172.21.15.86:0/947742450 conn(0x7ff4880d0170 msgr2=0x7ff4880d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:42.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.487+0000 7ff48de00640 1 -- 172.21.15.86:0/947742450 shutdown_connections 2024-04-20T21:22:42.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.487+0000 7ff48de00640 1 -- 172.21.15.86:0/947742450 wait complete. 2024-04-20T21:22:42.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.488+0000 7ff48de00640 1 Processor -- start 2024-04-20T21:22:42.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.488+0000 7ff48de00640 1 -- start start 2024-04-20T21:22:42.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.489+0000 7ff48de00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4880ad7f0 0x7ff4881f73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:42.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.489+0000 7ff48de00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff488156ae0 0x7ff48809a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:42.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.489+0000 7ff48de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff488160080 0x7ff48809a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:42.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.489+0000 7ff48de00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff488160500 con 0x7ff488160080 2024-04-20T21:22:42.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.489+0000 7ff48de00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff488156ec0 con 0x7ff4880ad7f0 2024-04-20T21:22:42.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.489+0000 7ff48de00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff48806e7d0 con 0x7ff488156ae0 2024-04-20T21:22:42.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.490+0000 7ff48d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff488160080 0x7ff48809a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:42.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.490+0000 7ff48d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff488160080 0x7ff48809a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54044/0 (socket says 172.21.15.86:54044) 2024-04-20T21:22:42.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.490+0000 7ff48d400640 1 -- 172.21.15.86:0/1528333318 learned_addr learned my addr 172.21.15.86:0/1528333318 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:42.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.490+0000 7ff487e00640 1 --2- 172.21.15.86:0/1528333318 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff488156ae0 0x7ff48809a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:42.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.490+0000 7ff48ca00640 1 --2- 172.21.15.86:0/1528333318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4880ad7f0 0x7ff4881f73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:42.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.490+0000 7ff48d400640 1 -- 172.21.15.86:0/1528333318 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff488156ae0 msgr2=0x7ff48809a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:42.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.490+0000 7ff48d400640 1 --2- 172.21.15.86:0/1528333318 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff488156ae0 0x7ff48809a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:42.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.490+0000 7ff48d400640 1 -- 172.21.15.86:0/1528333318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4880ad7f0 msgr2=0x7ff4881f73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:42.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.490+0000 7ff48d400640 1 --2- 172.21.15.86:0/1528333318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4880ad7f0 0x7ff4881f73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:42.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.490+0000 7ff48d400640 1 -- 172.21.15.86:0/1528333318 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff48809c5a0 con 0x7ff488160080 2024-04-20T21:22:42.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.490+0000 7ff48d400640 1 --2- 172.21.15.86:0/1528333318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff488160080 0x7ff48809a7a0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7ff480036040 tx=0x7ff480004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:42.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.491+0000 7ff484c00640 1 -- 172.21.15.86:0/1528333318 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff480043030 con 0x7ff488160080 2024-04-20T21:22:42.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.491+0000 7ff484c00640 1 -- 172.21.15.86:0/1528333318 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff48002ce50 con 0x7ff488160080 2024-04-20T21:22:42.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.491+0000 7ff484c00640 1 -- 172.21.15.86:0/1528333318 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff480009510 con 0x7ff488160080 2024-04-20T21:22:42.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.491+0000 7ff48de00640 1 -- 172.21.15.86:0/1528333318 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff488147190 con 0x7ff488160080 2024-04-20T21:22:42.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.491+0000 7ff48de00640 1 -- 172.21.15.86:0/1528333318 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff48809caf0 con 0x7ff488160080 2024-04-20T21:22:42.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.492+0000 7ff48de00640 1 -- 172.21.15.86:0/1528333318 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff48804fcf0 con 0x7ff488160080 2024-04-20T21:22:42.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.493+0000 7ff484c00640 1 -- 172.21.15.86:0/1528333318 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff480046070 con 0x7ff488160080 2024-04-20T21:22:42.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.494+0000 7ff484c00640 1 --2- 172.21.15.86:0/1528333318 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff46806c920 0x7ff46806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:42.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.494+0000 7ff484c00640 1 -- 172.21.15.86:0/1528333318 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 4857+0+0 (secure 0 0 0) 0x7ff4800adb20 con 0x7ff488160080 2024-04-20T21:22:42.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.494+0000 7ff48ca00640 1 --2- 172.21.15.86:0/1528333318 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff46806c920 0x7ff46806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:42.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.495+0000 7ff48ca00640 1 --2- 172.21.15.86:0/1528333318 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff46806c920 0x7ff46806ede0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff47c006fd0 tx=0x7ff47c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:42.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.499+0000 7ff484c00640 1 -- 172.21.15.86:0/1528333318 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff48004d070 con 0x7ff488160080 2024-04-20T21:22:42.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.688+0000 7ff48de00640 1 -- 172.21.15.86:0/1528333318 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ff4880b0b90 con 0x7ff488160080 2024-04-20T21:22:42.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.688+0000 7ff484c00640 1 -- 172.21.15.86:0/1528333318 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v67) ==== 84+0+65 (secure 0 0 0) 0x7ff4800333a0 con 0x7ff488160080 2024-04-20T21:22:42.688 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:22:42.689 INFO:teuthology.orchestra.run.smithi086.stdout:[ 2024-04-20T21:22:42.689 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:22:42.689 INFO:teuthology.orchestra.run.smithi086.stdout: "poolnum": 1, 2024-04-20T21:22:42.689 INFO:teuthology.orchestra.run.smithi086.stdout: "poolname": ".mgr" 2024-04-20T21:22:42.689 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:22:42.689 INFO:teuthology.orchestra.run.smithi086.stdout:] 2024-04-20T21:22:42.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.691+0000 7ff48de00640 1 -- 172.21.15.86:0/1528333318 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff46806c920 msgr2=0x7ff46806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:42.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.691+0000 7ff48de00640 1 --2- 172.21.15.86:0/1528333318 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff46806c920 0x7ff46806ede0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff47c006fd0 tx=0x7ff47c008040 comp rx=0 tx=0).stop 2024-04-20T21:22:42.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.691+0000 7ff48de00640 1 -- 172.21.15.86:0/1528333318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff488160080 msgr2=0x7ff48809a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:42.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.691+0000 7ff48de00640 1 --2- 172.21.15.86:0/1528333318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff488160080 0x7ff48809a7a0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7ff480036040 tx=0x7ff480004a40 comp rx=0 tx=0).stop 2024-04-20T21:22:42.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.691+0000 7ff48de00640 1 -- 172.21.15.86:0/1528333318 shutdown_connections 2024-04-20T21:22:42.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.691+0000 7ff48de00640 1 --2- 172.21.15.86:0/1528333318 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff46806c920 0x7ff46806ede0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:42.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.691+0000 7ff48de00640 1 --2- 172.21.15.86:0/1528333318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff488160080 0x7ff48809a7a0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:42.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.691+0000 7ff48de00640 1 --2- 172.21.15.86:0/1528333318 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff488156ae0 0x7ff48809a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:42.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.691+0000 7ff48de00640 1 --2- 172.21.15.86:0/1528333318 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4880ad7f0 0x7ff4881f73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:42.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.691+0000 7ff48de00640 1 -- 172.21.15.86:0/1528333318 >> 172.21.15.86:0/1528333318 conn(0x7ff4880d0170 msgr2=0x7ff4880afd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:42.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.691+0000 7ff48de00640 1 -- 172.21.15.86:0/1528333318 shutdown_connections 2024-04-20T21:22:42.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.691+0000 7ff48de00640 1 -- 172.21.15.86:0/1528333318 wait complete. 2024-04-20T21:22:42.710 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-20T21:22:42.710 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-20T21:22:42.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.903+0000 7f4d97e00640 1 Processor -- start 2024-04-20T21:22:42.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.904+0000 7f4d97e00640 1 -- start start 2024-04-20T21:22:42.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.905+0000 7f4d97e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d98153590 0x7f4d980d0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:42.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.905+0000 7f4d97e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4d980af870 0x7f4d980d0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:42.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.905+0000 7f4d97e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d9815cbe0 con 0x7f4d98071ad0 2024-04-20T21:22:42.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.905+0000 7f4d97e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f4d98153970 con 0x7f4d980af870 2024-04-20T21:22:42.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.905+0000 7f4d97e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f4d9815cea0 con 0x7f4d98153590 2024-04-20T21:22:42.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.905+0000 7f4d8fe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f4d98071ad0 0x7f4d980af160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:49834/0 (socket says 172.21.15.86:49834) 2024-04-20T21:22:42.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.905+0000 7f4d8fe00640 1 -- 172.21.15.86:0/1300698130 learned_addr learned my addr 172.21.15.86:0/1300698130 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:22:42.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.906+0000 7f4d97400640 1 --2- 172.21.15.86:0/1300698130 >> v2:172.21.15.86:3300/0 conn(0x7f4d980af870 0x7f4d980d0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:42.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.906+0000 7f4d96a00640 1 --2- 172.21.15.86:0/1300698130 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d98153590 0x7f4d980d0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:42.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.906+0000 7f4d96000640 1 -- 172.21.15.86:0/1300698130 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1787146279 0 0) 0x7f4d9815cbe0 con 0x7f4d98071ad0 2024-04-20T21:22:42.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.906+0000 7f4d96000640 1 -- 172.21.15.86:0/1300698130 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d7c003520 con 0x7f4d98071ad0 2024-04-20T21:22:42.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.906+0000 7f4d97400640 1 -- 172.21.15.86:0/1300698130 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d98153590 msgr2=0x7f4d980d0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:42.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.906+0000 7f4d97400640 1 --2- 172.21.15.86:0/1300698130 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d98153590 0x7f4d980d0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:42.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.906+0000 7f4d97400640 1 -- 172.21.15.86:0/1300698130 >> v1:172.21.15.86:6789/0 conn(0x7f4d98071ad0 legacy=0x7f4d980af160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:42.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.906+0000 7f4d97400640 1 -- 172.21.15.86:0/1300698130 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d9815d180 con 0x7f4d980af870 2024-04-20T21:22:42.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.908+0000 7f4d97400640 1 --2- 172.21.15.86:0/1300698130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d980af870 0x7f4d980d0b50 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f4d90009d90 tx=0x7f4d9002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8185145a89e4820f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:42.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.908+0000 7f4d96000640 1 -- 172.21.15.86:0/1300698130 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d90039070 con 0x7f4d980af870 2024-04-20T21:22:42.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.908+0000 7f4d96000640 1 -- 172.21.15.86:0/1300698130 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d9002cc50 con 0x7f4d980af870 2024-04-20T21:22:42.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.908+0000 7f4d96000640 1 -- 172.21.15.86:0/1300698130 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d90034440 con 0x7f4d980af870 2024-04-20T21:22:42.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.908+0000 7f4d97e00640 1 -- 172.21.15.86:0/1300698130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d980af870 msgr2=0x7f4d980d0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:42.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.908+0000 7f4d97e00640 1 --2- 172.21.15.86:0/1300698130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d980af870 0x7f4d980d0b50 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f4d90009d90 tx=0x7f4d9002bea0 comp rx=0 tx=0).stop 2024-04-20T21:22:42.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.909+0000 7f4d97e00640 1 -- 172.21.15.86:0/1300698130 shutdown_connections 2024-04-20T21:22:42.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.909+0000 7f4d97e00640 1 --2- 172.21.15.86:0/1300698130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d980af870 0x7f4d980d0b50 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:42.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.909+0000 7f4d97e00640 1 --2- 172.21.15.86:0/1300698130 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d98153590 0x7f4d980d0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:42.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.909+0000 7f4d97e00640 1 -- 172.21.15.86:0/1300698130 >> 172.21.15.86:0/1300698130 conn(0x7f4d980b2180 msgr2=0x7f4d980b2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:42.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.909+0000 7f4d97e00640 1 -- 172.21.15.86:0/1300698130 shutdown_connections 2024-04-20T21:22:42.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.909+0000 7f4d97e00640 1 -- 172.21.15.86:0/1300698130 wait complete. 2024-04-20T21:22:42.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.910+0000 7f4d97e00640 1 Processor -- start 2024-04-20T21:22:42.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.910+0000 7f4d97e00640 1 -- start start 2024-04-20T21:22:42.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.911+0000 7f4d97e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d98071ad0 0x7f4d98141510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:42.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.911+0000 7f4d97e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d980af870 0x7f4d9814e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:42.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.911+0000 7f4d97e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d98153590 0x7f4d9814ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:42.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.911+0000 7f4d97e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4d9806dce0 con 0x7f4d98071ad0 2024-04-20T21:22:42.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.911+0000 7f4d97e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4d9815d740 con 0x7f4d98153590 2024-04-20T21:22:42.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.911+0000 7f4d97e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f4d9806de60 con 0x7f4d980af870 2024-04-20T21:22:42.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.912+0000 7f4d96a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d98071ad0 0x7f4d98141510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:42.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.912+0000 7f4d96a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d98071ad0 0x7f4d98141510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54060/0 (socket says 172.21.15.86:54060) 2024-04-20T21:22:42.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.912+0000 7f4d96a00640 1 -- 172.21.15.86:0/88205736 learned_addr learned my addr 172.21.15.86:0/88205736 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:42.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.912+0000 7f4d96a00640 1 -- 172.21.15.86:0/88205736 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d980af870 msgr2=0x7f4d9814e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:42.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.912+0000 7f4d8fe00640 1 --2- 172.21.15.86:0/88205736 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d980af870 0x7f4d9814e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:42.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.912+0000 7f4d96a00640 1 --2- 172.21.15.86:0/88205736 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d980af870 0x7f4d9814e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:42.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.912+0000 7f4d96a00640 1 -- 172.21.15.86:0/88205736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d98153590 msgr2=0x7f4d9814ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:42.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.912+0000 7f4d96a00640 1 --2- 172.21.15.86:0/88205736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d98153590 0x7f4d9814ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:42.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.912+0000 7f4d96a00640 1 -- 172.21.15.86:0/88205736 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d98154a00 con 0x7f4d98071ad0 2024-04-20T21:22:42.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.912+0000 7f4d96a00640 1 --2- 172.21.15.86:0/88205736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d98071ad0 0x7f4d98141510 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f4d8800d5f0 tx=0x7f4d8800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:42.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.913+0000 7f4d8ea00640 1 -- 172.21.15.86:0/88205736 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d880125f0 con 0x7f4d98071ad0 2024-04-20T21:22:42.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.913+0000 7f4d8ea00640 1 -- 172.21.15.86:0/88205736 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d88019070 con 0x7f4d98071ad0 2024-04-20T21:22:42.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.913+0000 7f4d8ea00640 1 -- 172.21.15.86:0/88205736 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d880129d0 con 0x7f4d98071ad0 2024-04-20T21:22:42.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.913+0000 7f4d97e00640 1 -- 172.21.15.86:0/88205736 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d98141a50 con 0x7f4d98071ad0 2024-04-20T21:22:42.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.913+0000 7f4d97e00640 1 -- 172.21.15.86:0/88205736 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d98155230 con 0x7f4d98071ad0 2024-04-20T21:22:42.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.914+0000 7f4d97e00640 1 -- 172.21.15.86:0/88205736 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d980adf70 con 0x7f4d98071ad0 2024-04-20T21:22:42.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.915+0000 7f4d8ea00640 1 -- 172.21.15.86:0/88205736 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4d88012b70 con 0x7f4d98071ad0 2024-04-20T21:22:42.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.916+0000 7f4d8ea00640 1 --2- 172.21.15.86:0/88205736 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4d7406c8f0 0x7f4d7406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:42.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.916+0000 7f4d8ea00640 1 -- 172.21.15.86:0/88205736 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 4857+0+0 (secure 0 0 0) 0x7f4d8808bc60 con 0x7f4d98071ad0 2024-04-20T21:22:42.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.917+0000 7f4d8fe00640 1 --2- 172.21.15.86:0/88205736 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4d7406c8f0 0x7f4d7406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:42.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.917+0000 7f4d8fe00640 1 --2- 172.21.15.86:0/88205736 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4d7406c8f0 0x7f4d7406edb0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4d98151640 tx=0x7f4d80009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:42.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:42.921+0000 7f4d8ea00640 1 -- 172.21.15.86:0/88205736 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4d980adf70 con 0x7f4d98071ad0 2024-04-20T21:22:43.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:43.108+0000 7f4d97e00640 1 -- 172.21.15.86:0/88205736 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f4d9804fcf0 con 0x7f4d98071ad0 2024-04-20T21:22:43.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:43.944+0000 7f4d8ea00640 1 -- 172.21.15.86:0/88205736 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v68) ==== 138+0+0 (secure 0 0 0) 0x7f4d8805a3d0 con 0x7f4d98071ad0 2024-04-20T21:22:43.944 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' created 2024-04-20T21:22:43.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:43.947+0000 7f4d97e00640 1 -- 172.21.15.86:0/88205736 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4d7406c8f0 msgr2=0x7f4d7406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:43.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:43.947+0000 7f4d97e00640 1 --2- 172.21.15.86:0/88205736 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4d7406c8f0 0x7f4d7406edb0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4d98151640 tx=0x7f4d80009040 comp rx=0 tx=0).stop 2024-04-20T21:22:43.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:43.947+0000 7f4d97e00640 1 -- 172.21.15.86:0/88205736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d98071ad0 msgr2=0x7f4d98141510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:43.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:43.947+0000 7f4d97e00640 1 --2- 172.21.15.86:0/88205736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d98071ad0 0x7f4d98141510 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f4d8800d5f0 tx=0x7f4d8800dab0 comp rx=0 tx=0).stop 2024-04-20T21:22:43.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:43.947+0000 7f4d97e00640 1 -- 172.21.15.86:0/88205736 shutdown_connections 2024-04-20T21:22:43.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:43.947+0000 7f4d97e00640 1 --2- 172.21.15.86:0/88205736 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4d7406c8f0 0x7f4d7406edb0 secure :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4d98151640 tx=0x7f4d80009040 comp rx=0 tx=0).stop 2024-04-20T21:22:43.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:43.947+0000 7f4d97e00640 1 --2- 172.21.15.86:0/88205736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d98153590 0x7f4d9814ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:43.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:43.947+0000 7f4d97e00640 1 --2- 172.21.15.86:0/88205736 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d980af870 0x7f4d9814e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:43.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:43.947+0000 7f4d97e00640 1 --2- 172.21.15.86:0/88205736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d98071ad0 0x7f4d98141510 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:43.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:43.947+0000 7f4d97e00640 1 -- 172.21.15.86:0/88205736 >> 172.21.15.86:0/88205736 conn(0x7f4d980b2180 msgr2=0x7f4d98072290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:43.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:43.948+0000 7f4d97e00640 1 -- 172.21.15.86:0/88205736 shutdown_connections 2024-04-20T21:22:43.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:43.948+0000 7f4d97e00640 1 -- 172.21.15.86:0/88205736 wait complete. 2024-04-20T21:22:43.978 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-20T21:22:44.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.177+0000 7fc741c00640 1 Processor -- start 2024-04-20T21:22:44.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.178+0000 7fc741c00640 1 -- start start 2024-04-20T21:22:44.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.179+0000 7fc741c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc73c0d45a0 0x7fc73c0d3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:44.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.179+0000 7fc741c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc73c15ea70 0x7fc73c0d4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:44.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.179+0000 7fc741c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc73c0cf120 con 0x7fc73c06d570 2024-04-20T21:22:44.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.179+0000 7fc741c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc73c06de70 con 0x7fc73c0d45a0 2024-04-20T21:22:44.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.179+0000 7fc741c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc73c06e150 con 0x7fc73c15ea70 2024-04-20T21:22:44.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.179+0000 7fc73be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc73c0d45a0 0x7fc73c0d3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:44.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.179+0000 7fc73be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc73c0d45a0 0x7fc73c0d3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54076/0 (socket says 172.21.15.86:54076) 2024-04-20T21:22:44.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.179+0000 7fc73be00640 1 -- 172.21.15.86:0/3900017295 learned_addr learned my addr 172.21.15.86:0/3900017295 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:44.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.180+0000 7fc73be00640 1 -- 172.21.15.86:0/3900017295 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc73c15ea70 msgr2=0x7fc73c0d4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:44.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.180+0000 7fc73be00640 1 --2- 172.21.15.86:0/3900017295 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc73c15ea70 0x7fc73c0d4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:44.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.180+0000 7fc73be00640 1 -- 172.21.15.86:0/3900017295 >> v1:172.21.15.144:6789/0 conn(0x7fc73c06d570 legacy=0x7fc73c0d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:44.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.180+0000 7fc73be00640 1 -- 172.21.15.86:0/3900017295 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc73c06e430 con 0x7fc73c0d45a0 2024-04-20T21:22:44.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.181+0000 7fc73be00640 1 --2- 172.21.15.86:0/3900017295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc73c0d45a0 0x7fc73c0d3260 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fc72c009870 tx=0x7fc72c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b66088909dcc3388 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:44.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.181+0000 7fc73aa00640 1 -- 172.21.15.86:0/3900017295 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c0317c0 con 0x7fc73c0d45a0 2024-04-20T21:22:44.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.181+0000 7fc73aa00640 1 -- 172.21.15.86:0/3900017295 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc72c037070 con 0x7fc73c0d45a0 2024-04-20T21:22:44.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.181+0000 7fc73aa00640 1 -- 172.21.15.86:0/3900017295 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c031b80 con 0x7fc73c0d45a0 2024-04-20T21:22:44.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.182+0000 7fc741c00640 1 -- 172.21.15.86:0/3900017295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc73c0d45a0 msgr2=0x7fc73c0d3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:44.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.182+0000 7fc741c00640 1 --2- 172.21.15.86:0/3900017295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc73c0d45a0 0x7fc73c0d3260 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fc72c009870 tx=0x7fc72c02b860 comp rx=0 tx=0).stop 2024-04-20T21:22:44.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.182+0000 7fc741c00640 1 -- 172.21.15.86:0/3900017295 shutdown_connections 2024-04-20T21:22:44.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.182+0000 7fc741c00640 1 --2- 172.21.15.86:0/3900017295 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc73c15ea70 0x7fc73c0d4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:44.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.182+0000 7fc741c00640 1 --2- 172.21.15.86:0/3900017295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc73c0d45a0 0x7fc73c0d3260 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:44.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.182+0000 7fc741c00640 1 -- 172.21.15.86:0/3900017295 >> 172.21.15.86:0/3900017295 conn(0x7fc73c09d9d0 msgr2=0x7fc73c08bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:44.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.182+0000 7fc741c00640 1 -- 172.21.15.86:0/3900017295 shutdown_connections 2024-04-20T21:22:44.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.182+0000 7fc741c00640 1 -- 172.21.15.86:0/3900017295 wait complete. 2024-04-20T21:22:44.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.183+0000 7fc741c00640 1 Processor -- start 2024-04-20T21:22:44.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.184+0000 7fc741c00640 1 -- start start 2024-04-20T21:22:44.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.184+0000 7fc741c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc73c06d570 0x7fc73c15fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:44.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.185+0000 7fc741c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc73c0d45a0 0x7fc73c1575b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:44.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.185+0000 7fc741c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc73c15ea70 0x7fc73c161550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:44.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.185+0000 7fc741c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc73c06da00 con 0x7fc73c0d45a0 2024-04-20T21:22:44.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.185+0000 7fc741c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc73c0d4ec0 con 0x7fc73c06d570 2024-04-20T21:22:44.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.185+0000 7fc741c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc73c06db80 con 0x7fc73c15ea70 2024-04-20T21:22:44.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.185+0000 7fc73b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc73c0d45a0 0x7fc73c1575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:44.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.185+0000 7fc73b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc73c0d45a0 0x7fc73c1575b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54086/0 (socket says 172.21.15.86:54086) 2024-04-20T21:22:44.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.185+0000 7fc73b400640 1 -- 172.21.15.86:0/832064754 learned_addr learned my addr 172.21.15.86:0/832064754 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:44.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.185+0000 7fc73b400640 1 -- 172.21.15.86:0/832064754 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc73c15ea70 msgr2=0x7fc73c161550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:44.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.185+0000 7fc73b400640 1 --2- 172.21.15.86:0/832064754 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc73c15ea70 0x7fc73c161550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:44.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.185+0000 7fc73b400640 1 -- 172.21.15.86:0/832064754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc73c06d570 msgr2=0x7fc73c15fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:44.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.185+0000 7fc73b400640 1 --2- 172.21.15.86:0/832064754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc73c06d570 0x7fc73c15fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:44.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.185+0000 7fc73b400640 1 -- 172.21.15.86:0/832064754 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc73c157af0 con 0x7fc73c0d45a0 2024-04-20T21:22:44.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.185+0000 7fc73b400640 1 --2- 172.21.15.86:0/832064754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc73c0d45a0 0x7fc73c1575b0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fc72400b410 tx=0x7fc72400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:44.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.186+0000 7fc733e00640 1 -- 172.21.15.86:0/832064754 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc724015020 con 0x7fc73c0d45a0 2024-04-20T21:22:44.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.186+0000 7fc733e00640 1 -- 172.21.15.86:0/832064754 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc724011850 con 0x7fc73c0d45a0 2024-04-20T21:22:44.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.186+0000 7fc733e00640 1 -- 172.21.15.86:0/832064754 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc724011c40 con 0x7fc73c0d45a0 2024-04-20T21:22:44.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.186+0000 7fc741c00640 1 -- 172.21.15.86:0/832064754 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc73c1604b0 con 0x7fc73c0d45a0 2024-04-20T21:22:44.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.186+0000 7fc741c00640 1 -- 172.21.15.86:0/832064754 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc73c160a70 con 0x7fc73c0d45a0 2024-04-20T21:22:44.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.187+0000 7fc741c00640 1 -- 172.21.15.86:0/832064754 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc73c04fd90 con 0x7fc73c0d45a0 2024-04-20T21:22:44.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.188+0000 7fc733e00640 1 -- 172.21.15.86:0/832064754 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc724018020 con 0x7fc73c0d45a0 2024-04-20T21:22:44.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.189+0000 7fc733e00640 1 --2- 172.21.15.86:0/832064754 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc71806c8f0 0x7fc71806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:44.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.189+0000 7fc733e00640 1 -- 172.21.15.86:0/832064754 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5227+0+0 (secure 0 0 0) 0x7fc72408b900 con 0x7fc73c0d45a0 2024-04-20T21:22:44.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.189+0000 7fc73be00640 1 --2- 172.21.15.86:0/832064754 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc71806c8f0 0x7fc71806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:44.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.190+0000 7fc73be00640 1 --2- 172.21.15.86:0/832064754 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc71806c8f0 0x7fc71806edb0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc72c036000 tx=0x7fc72c030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:44.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.194+0000 7fc733e00640 1 -- 172.21.15.86:0/832064754 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc72401f020 con 0x7fc73c0d45a0 2024-04-20T21:22:44.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.394+0000 7fc741c00640 1 -- 172.21.15.86:0/832064754 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fc73c0b5e90 con 0x7fc73c0d45a0 2024-04-20T21:22:44.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.959+0000 7fc733e00640 1 -- 172.21.15.86:0/832064754 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v69) ==== 170+0+0 (secure 0 0 0) 0x7fc7240172c0 con 0x7fc73c0d45a0 2024-04-20T21:22:44.960 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' created 2024-04-20T21:22:44.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.963+0000 7fc741c00640 1 -- 172.21.15.86:0/832064754 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc71806c8f0 msgr2=0x7fc71806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:44.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.963+0000 7fc741c00640 1 --2- 172.21.15.86:0/832064754 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc71806c8f0 0x7fc71806edb0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc72c036000 tx=0x7fc72c030000 comp rx=0 tx=0).stop 2024-04-20T21:22:44.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.963+0000 7fc741c00640 1 -- 172.21.15.86:0/832064754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc73c0d45a0 msgr2=0x7fc73c1575b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:44.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.963+0000 7fc741c00640 1 --2- 172.21.15.86:0/832064754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc73c0d45a0 0x7fc73c1575b0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fc72400b410 tx=0x7fc72400b8d0 comp rx=0 tx=0).stop 2024-04-20T21:22:44.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.964+0000 7fc741c00640 1 -- 172.21.15.86:0/832064754 shutdown_connections 2024-04-20T21:22:44.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.964+0000 7fc741c00640 1 --2- 172.21.15.86:0/832064754 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc71806c8f0 0x7fc71806edb0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:44.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.964+0000 7fc741c00640 1 --2- 172.21.15.86:0/832064754 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc73c15ea70 0x7fc73c161550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:44.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.964+0000 7fc741c00640 1 --2- 172.21.15.86:0/832064754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc73c0d45a0 0x7fc73c1575b0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:44.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.964+0000 7fc741c00640 1 --2- 172.21.15.86:0/832064754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc73c06d570 0x7fc73c15fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:44.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.964+0000 7fc741c00640 1 -- 172.21.15.86:0/832064754 >> 172.21.15.86:0/832064754 conn(0x7fc73c09d9d0 msgr2=0x7fc73c0b27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:44.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.965+0000 7fc741c00640 1 -- 172.21.15.86:0/832064754 shutdown_connections 2024-04-20T21:22:44.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:44.965+0000 7fc741c00640 1 -- 172.21.15.86:0/832064754 wait complete. 2024-04-20T21:22:44.996 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-20T21:22:45.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.170+0000 7f95ca600640 1 Processor -- start 2024-04-20T21:22:45.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.171+0000 7f95ca600640 1 -- start start 2024-04-20T21:22:45.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.172+0000 7f95ca600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95c4145250 0x7f95c406efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:45.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.172+0000 7f95ca600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f95c4072450 0x7f95c415cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:45.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.172+0000 7f95ca600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95c406f500 con 0x7f95c4145f00 2024-04-20T21:22:45.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.172+0000 7f95ca600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f95c4160410 con 0x7f95c4072450 2024-04-20T21:22:45.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.172+0000 7f95ca600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f95c41606f0 con 0x7f95c4145250 2024-04-20T21:22:45.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.173+0000 7f95c9200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95c4145250 0x7f95c406efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:45.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.173+0000 7f95c9200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95c4145250 0x7f95c406efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:59838/0 (socket says 172.21.15.86:59838) 2024-04-20T21:22:45.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.173+0000 7f95c9200640 1 -- 172.21.15.86:0/2844947110 learned_addr learned my addr 172.21.15.86:0/2844947110 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:45.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.173+0000 7f95c9c00640 1 --2- 172.21.15.86:0/2844947110 >> v2:172.21.15.144:3300/0 conn(0x7f95c4072450 0x7f95c415cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:45.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.175+0000 7f95c9200640 1 -- 172.21.15.86:0/2844947110 >> v2:172.21.15.144:3300/0 conn(0x7f95c4072450 msgr2=0x7f95c415cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:45.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.175+0000 7f95c9200640 1 --2- 172.21.15.86:0/2844947110 >> v2:172.21.15.144:3300/0 conn(0x7f95c4072450 0x7f95c415cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:45.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.175+0000 7f95c9200640 1 -- 172.21.15.86:0/2844947110 >> v1:172.21.15.144:6789/0 conn(0x7f95c4145f00 legacy=0x7f95c4071d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:45.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.175+0000 7f95c9200640 1 -- 172.21.15.86:0/2844947110 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95c41609d0 con 0x7f95c4145250 2024-04-20T21:22:45.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.176+0000 7f95c9200640 1 --2- 172.21.15.86:0/2844947110 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95c4145250 0x7f95c406efc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f95b4009870 tx=0x7f95b402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebdc2430542b0b97 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:45.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.176+0000 7f95c3e00640 1 -- 172.21.15.86:0/2844947110 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b4035020 con 0x7f95c4145250 2024-04-20T21:22:45.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.176+0000 7f95c3e00640 1 -- 172.21.15.86:0/2844947110 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95b4031640 con 0x7f95c4145250 2024-04-20T21:22:45.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.176+0000 7f95c3e00640 1 -- 172.21.15.86:0/2844947110 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b402d040 con 0x7f95c4145250 2024-04-20T21:22:45.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.176+0000 7f95ca600640 1 -- 172.21.15.86:0/2844947110 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95c4145250 msgr2=0x7f95c406efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:45.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.176+0000 7f95ca600640 1 --2- 172.21.15.86:0/2844947110 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95c4145250 0x7f95c406efc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f95b4009870 tx=0x7f95b402b8f0 comp rx=0 tx=0).stop 2024-04-20T21:22:45.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.176+0000 7f95ca600640 1 -- 172.21.15.86:0/2844947110 shutdown_connections 2024-04-20T21:22:45.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.176+0000 7f95ca600640 1 --2- 172.21.15.86:0/2844947110 >> v2:172.21.15.144:3300/0 conn(0x7f95c4072450 0x7f95c415cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:45.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.177+0000 7f95ca600640 1 --2- 172.21.15.86:0/2844947110 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95c4145250 0x7f95c406efc0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:45.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.177+0000 7f95ca600640 1 -- 172.21.15.86:0/2844947110 >> 172.21.15.86:0/2844947110 conn(0x7f95c4067cf0 msgr2=0x7f95c406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:45.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.177+0000 7f95ca600640 1 -- 172.21.15.86:0/2844947110 shutdown_connections 2024-04-20T21:22:45.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.177+0000 7f95ca600640 1 -- 172.21.15.86:0/2844947110 wait complete. 2024-04-20T21:22:45.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.178+0000 7f95ca600640 1 Processor -- start 2024-04-20T21:22:45.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.178+0000 7f95ca600640 1 -- start start 2024-04-20T21:22:45.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.179+0000 7f95ca600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95c4072450 0x7f95c41438a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:45.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.179+0000 7f95ca600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95c4145250 0x7f95c40b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:45.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.179+0000 7f95ca600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95c4145f00 0x7f95c40aec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:45.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.179+0000 7f95ca600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f95c41457b0 con 0x7f95c4145250 2024-04-20T21:22:45.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.179+0000 7f95ca600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f95c4145630 con 0x7f95c4145f00 2024-04-20T21:22:45.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.179+0000 7f95ca600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f95c415d330 con 0x7f95c4072450 2024-04-20T21:22:45.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.180+0000 7f95bbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95c4145250 0x7f95c40b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:45.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.180+0000 7f95bbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95c4145250 0x7f95c40b1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54096/0 (socket says 172.21.15.86:54096) 2024-04-20T21:22:45.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.180+0000 7f95bbe00640 1 -- 172.21.15.86:0/3558288809 learned_addr learned my addr 172.21.15.86:0/3558288809 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:45.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.180+0000 7f95c9200640 1 --2- 172.21.15.86:0/3558288809 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95c4072450 0x7f95c41438a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:45.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.180+0000 7f95bbe00640 1 -- 172.21.15.86:0/3558288809 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95c4072450 msgr2=0x7f95c41438a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:45.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.180+0000 7f95bbe00640 1 --2- 172.21.15.86:0/3558288809 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95c4072450 0x7f95c41438a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:45.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.180+0000 7f95bbe00640 1 -- 172.21.15.86:0/3558288809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95c4145f00 msgr2=0x7f95c40aec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:45.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.180+0000 7f95bbe00640 1 --2- 172.21.15.86:0/3558288809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95c4145f00 0x7f95c40aec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:45.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.180+0000 7f95bbe00640 1 -- 172.21.15.86:0/3558288809 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95c40b0de0 con 0x7f95c4145250 2024-04-20T21:22:45.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.181+0000 7f95bbe00640 1 --2- 172.21.15.86:0/3558288809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95c4145250 0x7f95c40b1e40 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f95ac00c3c0 tx=0x7f95ac00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:45.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.181+0000 7f95c1600640 1 -- 172.21.15.86:0/3558288809 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ac0052d0 con 0x7f95c4145250 2024-04-20T21:22:45.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.181+0000 7f95c1600640 1 -- 172.21.15.86:0/3558288809 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95ac005490 con 0x7f95c4145250 2024-04-20T21:22:45.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.181+0000 7f95c1600640 1 -- 172.21.15.86:0/3558288809 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ac007260 con 0x7f95c4145250 2024-04-20T21:22:45.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.181+0000 7f95ca600640 1 -- 172.21.15.86:0/3558288809 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95c40b1080 con 0x7f95c4145250 2024-04-20T21:22:45.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.181+0000 7f95ca600640 1 -- 172.21.15.86:0/3558288809 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95c40b1640 con 0x7f95c4145250 2024-04-20T21:22:45.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.182+0000 7f95ca600640 1 -- 172.21.15.86:0/3558288809 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95c404fd90 con 0x7f95c4145250 2024-04-20T21:22:45.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.184+0000 7f95c1600640 1 -- 172.21.15.86:0/3558288809 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f95ac007400 con 0x7f95c4145250 2024-04-20T21:22:45.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.184+0000 7f95c1600640 1 --2- 172.21.15.86:0/3558288809 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f95a006c8f0 0x7f95a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:45.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.185+0000 7f95c1600640 1 -- 172.21.15.86:0/3558288809 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5609+0+0 (secure 0 0 0) 0x7f95ac08af10 con 0x7f95c4145250 2024-04-20T21:22:45.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.185+0000 7f95c9200640 1 --2- 172.21.15.86:0/3558288809 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f95a006c8f0 0x7f95a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:45.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.185+0000 7f95c9200640 1 --2- 172.21.15.86:0/3558288809 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f95a006c8f0 0x7f95a006edb0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f95b4036000 tx=0x7f95b4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:45.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.190+0000 7f95c1600640 1 -- 172.21.15.86:0/3558288809 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f95c406f500 con 0x7f95c4145250 2024-04-20T21:22:45.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.428+0000 7f95ca600640 1 -- 172.21.15.86:0/3558288809 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f95c4072ea0 con 0x7f95c4145250 2024-04-20T21:22:45.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.982+0000 7f95c1600640 1 -- 172.21.15.86:0/3558288809 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v38) ==== 174+0+0 (secure 0 0 0) 0x7f95ac059390 con 0x7f95c4145250 2024-04-20T21:22:45.983 INFO:teuthology.orchestra.run.smithi086.stderr:new fs with metadata pool 10 and data pool 11 2024-04-20T21:22:45.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.988+0000 7f95ca600640 1 -- 172.21.15.86:0/3558288809 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f95a006c8f0 msgr2=0x7f95a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:45.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.988+0000 7f95ca600640 1 --2- 172.21.15.86:0/3558288809 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f95a006c8f0 0x7f95a006edb0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f95b4036000 tx=0x7f95b4008030 comp rx=0 tx=0).stop 2024-04-20T21:22:45.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.988+0000 7f95ca600640 1 -- 172.21.15.86:0/3558288809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95c4145250 msgr2=0x7f95c40b1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:45.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.988+0000 7f95ca600640 1 --2- 172.21.15.86:0/3558288809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95c4145250 0x7f95c40b1e40 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f95ac00c3c0 tx=0x7f95ac00c880 comp rx=0 tx=0).stop 2024-04-20T21:22:45.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.992+0000 7f95ca600640 1 -- 172.21.15.86:0/3558288809 shutdown_connections 2024-04-20T21:22:45.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.992+0000 7f95ca600640 1 --2- 172.21.15.86:0/3558288809 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f95a006c8f0 0x7f95a006edb0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:45.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.992+0000 7f95ca600640 1 --2- 172.21.15.86:0/3558288809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95c4145f00 0x7f95c40aec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:45.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.993+0000 7f95ca600640 1 --2- 172.21.15.86:0/3558288809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95c4145250 0x7f95c40b1e40 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:45.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.993+0000 7f95ca600640 1 --2- 172.21.15.86:0/3558288809 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f95c4072450 0x7f95c41438a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:45.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.993+0000 7f95ca600640 1 -- 172.21.15.86:0/3558288809 >> 172.21.15.86:0/3558288809 conn(0x7f95c4067cf0 msgr2=0x7f95c40b3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:45.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.994+0000 7f95ca600640 1 -- 172.21.15.86:0/3558288809 shutdown_connections 2024-04-20T21:22:45.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:45.994+0000 7f95ca600640 1 -- 172.21.15.86:0/3558288809 wait complete. 2024-04-20T21:22:46.028 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:22:46.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.197+0000 7f2c8ac00640 1 Processor -- start 2024-04-20T21:22:46.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.197+0000 7f2c8ac00640 1 -- start start 2024-04-20T21:22:46.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.199+0000 7f2c8ac00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2c84146130 0x7f2c8406e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:46.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.199+0000 7f2c8ac00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c840ccdd0 0x7f2c840cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:46.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.199+0000 7f2c8ac00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c8406dc30 con 0x7f2c84071d00 2024-04-20T21:22:46.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.199+0000 7f2c8ac00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f2c840cd8e0 con 0x7f2c84146130 2024-04-20T21:22:46.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.199+0000 7f2c8ac00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2c840cdbc0 con 0x7f2c840ccdd0 2024-04-20T21:22:46.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.199+0000 7f2c83e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2c84146130 0x7f2c8406e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:46.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.199+0000 7f2c83e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2c84146130 0x7f2c8406e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51946/0 (socket says 172.21.15.86:51946) 2024-04-20T21:22:46.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.199+0000 7f2c83e00640 1 -- 172.21.15.86:0/343202406 learned_addr learned my addr 172.21.15.86:0/343202406 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:46.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.199+0000 7f2c83400640 1 --2- 172.21.15.86:0/343202406 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c840ccdd0 0x7f2c840cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:46.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.199+0000 7f2c83e00640 1 -- 172.21.15.86:0/343202406 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c840ccdd0 msgr2=0x7f2c840cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:46.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.199+0000 7f2c83e00640 1 --2- 172.21.15.86:0/343202406 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c840ccdd0 0x7f2c840cd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:46.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.199+0000 7f2c83e00640 1 -- 172.21.15.86:0/343202406 >> v1:172.21.15.86:6789/0 conn(0x7f2c84071d00 legacy=0x7f2c8415cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:46.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.200+0000 7f2c83e00640 1 -- 172.21.15.86:0/343202406 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c8406ded0 con 0x7f2c84146130 2024-04-20T21:22:46.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.201+0000 7f2c83e00640 1 --2- 172.21.15.86:0/343202406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c84146130 0x7f2c8406e6d0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f2c74009e50 tx=0x7f2c7402bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c89f42e5fd4866c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:46.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.201+0000 7f2c82a00640 1 -- 172.21.15.86:0/343202406 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7402cb90 con 0x7f2c84146130 2024-04-20T21:22:46.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.201+0000 7f2c82a00640 1 -- 172.21.15.86:0/343202406 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c7402cd30 con 0x7f2c84146130 2024-04-20T21:22:46.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.201+0000 7f2c82a00640 1 -- 172.21.15.86:0/343202406 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c74034610 con 0x7f2c84146130 2024-04-20T21:22:46.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.201+0000 7f2c8ac00640 1 -- 172.21.15.86:0/343202406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c84146130 msgr2=0x7f2c8406e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:46.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.201+0000 7f2c8ac00640 1 --2- 172.21.15.86:0/343202406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c84146130 0x7f2c8406e6d0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f2c74009e50 tx=0x7f2c7402bff0 comp rx=0 tx=0).stop 2024-04-20T21:22:46.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.202+0000 7f2c8ac00640 1 -- 172.21.15.86:0/343202406 shutdown_connections 2024-04-20T21:22:46.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.202+0000 7f2c8ac00640 1 --2- 172.21.15.86:0/343202406 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c840ccdd0 0x7f2c840cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:46.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.202+0000 7f2c8ac00640 1 --2- 172.21.15.86:0/343202406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c84146130 0x7f2c8406e6d0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:46.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.202+0000 7f2c8ac00640 1 -- 172.21.15.86:0/343202406 >> 172.21.15.86:0/343202406 conn(0x7f2c840b2360 msgr2=0x7f2c840b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:46.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.202+0000 7f2c8ac00640 1 -- 172.21.15.86:0/343202406 shutdown_connections 2024-04-20T21:22:46.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.202+0000 7f2c8ac00640 1 -- 172.21.15.86:0/343202406 wait complete. 2024-04-20T21:22:46.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.203+0000 7f2c8ac00640 1 Processor -- start 2024-04-20T21:22:46.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.203+0000 7f2c8ac00640 1 -- start start 2024-04-20T21:22:46.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.204+0000 7f2c8ac00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c84071d00 0x7f2c8415c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:46.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.204+0000 7f2c8ac00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c840ccdd0 0x7f2c84139290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:46.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.204+0000 7f2c8ac00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c84146130 0x7f2c841526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:46.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.204+0000 7f2c8ac00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2c84072190 con 0x7f2c84071d00 2024-04-20T21:22:46.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.204+0000 7f2c8ac00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8406f150 con 0x7f2c84146130 2024-04-20T21:22:46.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.204+0000 7f2c8ac00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2c84072310 con 0x7f2c840ccdd0 2024-04-20T21:22:46.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.204+0000 7f2c83e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c84071d00 0x7f2c8415c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:46.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.205+0000 7f2c83e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c84071d00 0x7f2c8415c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51962/0 (socket says 172.21.15.86:51962) 2024-04-20T21:22:46.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.205+0000 7f2c83e00640 1 -- 172.21.15.86:0/3786642259 learned_addr learned my addr 172.21.15.86:0/3786642259 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:46.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.205+0000 7f2c83e00640 1 -- 172.21.15.86:0/3786642259 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c840ccdd0 msgr2=0x7f2c84139290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:46.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.205+0000 7f2c83e00640 1 --2- 172.21.15.86:0/3786642259 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c840ccdd0 0x7f2c84139290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:46.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.205+0000 7f2c83e00640 1 -- 172.21.15.86:0/3786642259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c84146130 msgr2=0x7f2c841526d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:46.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.205+0000 7f2c83e00640 1 --2- 172.21.15.86:0/3786642259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c84146130 0x7f2c841526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:46.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.205+0000 7f2c83e00640 1 -- 172.21.15.86:0/3786642259 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c84152c10 con 0x7f2c84071d00 2024-04-20T21:22:46.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.205+0000 7f2c83e00640 1 --2- 172.21.15.86:0/3786642259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c84071d00 0x7f2c8415c280 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f2c74009e50 tx=0x7f2c74004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:46.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.206+0000 7f2c7be00640 1 -- 172.21.15.86:0/3786642259 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c74043020 con 0x7f2c84071d00 2024-04-20T21:22:46.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.206+0000 7f2c7be00640 1 -- 172.21.15.86:0/3786642259 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c7403e450 con 0x7f2c84071d00 2024-04-20T21:22:46.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.206+0000 7f2c8ac00640 1 -- 172.21.15.86:0/3786642259 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c841555a0 con 0x7f2c84071d00 2024-04-20T21:22:46.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.206+0000 7f2c7be00640 1 -- 172.21.15.86:0/3786642259 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7403e840 con 0x7f2c84071d00 2024-04-20T21:22:46.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.206+0000 7f2c8ac00640 1 -- 172.21.15.86:0/3786642259 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c84140610 con 0x7f2c84071d00 2024-04-20T21:22:46.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.207+0000 7f2c8ac00640 1 -- 172.21.15.86:0/3786642259 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c840689e0 con 0x7f2c84071d00 2024-04-20T21:22:46.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.208+0000 7f2c7be00640 1 -- 172.21.15.86:0/3786642259 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2c74046020 con 0x7f2c84071d00 2024-04-20T21:22:46.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.209+0000 7f2c7be00640 1 --2- 172.21.15.86:0/3786642259 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2c6406c8f0 0x7f2c6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:46.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.209+0000 7f2c7be00640 1 -- 172.21.15.86:0/3786642259 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f2c740ae4d0 con 0x7f2c84071d00 2024-04-20T21:22:46.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.209+0000 7f2c83400640 1 --2- 172.21.15.86:0/3786642259 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2c6406c8f0 0x7f2c6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:46.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.210+0000 7f2c83400640 1 --2- 172.21.15.86:0/3786642259 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2c6406c8f0 0x7f2c6406edb0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2c841f7d00 tx=0x7f2c70008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:46.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.214+0000 7f2c7be00640 1 -- 172.21.15.86:0/3786642259 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2c7404d030 con 0x7f2c84071d00 2024-04-20T21:22:46.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.399+0000 7f2c8ac00640 1 -- 172.21.15.86:0/3786642259 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2c8415ff50 con 0x7f2c84071d00 2024-04-20T21:22:46.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.401+0000 7f2c7be00640 1 -- 172.21.15.86:0/3786642259 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+16681 (secure 0 0 0) 0x7f2c7407c860 con 0x7f2c84071d00 2024-04-20T21:22:46.401 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:22:46.401 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":70,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:22:45.431170+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:22:43.125767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-20T21:22:44.410549+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:22:46.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.404+0000 7f2c8ac00640 1 -- 172.21.15.86:0/3786642259 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2c6406c8f0 msgr2=0x7f2c6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:46.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.404+0000 7f2c8ac00640 1 --2- 172.21.15.86:0/3786642259 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2c6406c8f0 0x7f2c6406edb0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2c841f7d00 tx=0x7f2c70008040 comp rx=0 tx=0).stop 2024-04-20T21:22:46.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.404+0000 7f2c8ac00640 1 -- 172.21.15.86:0/3786642259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c84071d00 msgr2=0x7f2c8415c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:46.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.404+0000 7f2c8ac00640 1 --2- 172.21.15.86:0/3786642259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c84071d00 0x7f2c8415c280 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f2c74009e50 tx=0x7f2c74004a40 comp rx=0 tx=0).stop 2024-04-20T21:22:46.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.404+0000 7f2c8ac00640 1 -- 172.21.15.86:0/3786642259 shutdown_connections 2024-04-20T21:22:46.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.404+0000 7f2c8ac00640 1 --2- 172.21.15.86:0/3786642259 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2c6406c8f0 0x7f2c6406edb0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:46.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.404+0000 7f2c8ac00640 1 --2- 172.21.15.86:0/3786642259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c84146130 0x7f2c841526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:46.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.404+0000 7f2c8ac00640 1 --2- 172.21.15.86:0/3786642259 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c840ccdd0 0x7f2c84139290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:46.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.404+0000 7f2c8ac00640 1 --2- 172.21.15.86:0/3786642259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c84071d00 0x7f2c8415c280 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:46.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.404+0000 7f2c8ac00640 1 -- 172.21.15.86:0/3786642259 >> 172.21.15.86:0/3786642259 conn(0x7f2c840b2360 msgr2=0x7f2c84142a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:46.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.404+0000 7f2c8ac00640 1 -- 172.21.15.86:0/3786642259 shutdown_connections 2024-04-20T21:22:46.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.404+0000 7f2c8ac00640 1 -- 172.21.15.86:0/3786642259 wait complete. 2024-04-20T21:22:46.419 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:22:46.500 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:22:46.507 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:22:46.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.612+0000 7fc7dc400640 1 Processor -- start 2024-04-20T21:22:46.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.612+0000 7fc7dc400640 1 -- start start 2024-04-20T21:22:46.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.613+0000 7fc7dc400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc7d406e6d0 0x7fc7d4071d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:46.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.613+0000 7fc7dc400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc7d4072240 0x7fc7d415cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:46.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.613+0000 7fc7dc400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7d406eb30 con 0x7fc7d4146130 2024-04-20T21:22:46.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.613+0000 7fc7dc400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc7d4072690 con 0x7fc7d4072240 2024-04-20T21:22:46.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.613+0000 7fc7dc400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fc7d406dc30 con 0x7fc7d406e6d0 2024-04-20T21:22:46.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.614+0000 7fc7da400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc7d4072240 0x7fc7d415cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:46.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.614+0000 7fc7d9a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fc7d4146130 0x7fc7d40ccdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44698/0 (socket says 172.21.15.86:44698) 2024-04-20T21:22:46.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.614+0000 7fc7d9a00640 1 -- 172.21.15.86:0/3493550451 learned_addr learned my addr 172.21.15.86:0/3493550451 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:22:46.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.614+0000 7fc7da400640 1 -- 172.21.15.86:0/3493550451 >> v2:172.21.15.144:3300/0 conn(0x7fc7d406e6d0 msgr2=0x7fc7d4071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:46.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.614+0000 7fc7da400640 1 --2- 172.21.15.86:0/3493550451 >> v2:172.21.15.144:3300/0 conn(0x7fc7d406e6d0 0x7fc7d4071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:46.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.614+0000 7fc7da400640 1 -- 172.21.15.86:0/3493550451 >> v1:172.21.15.86:6789/0 conn(0x7fc7d4146130 legacy=0x7fc7d40ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:46.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.614+0000 7fc7d3e00640 1 -- 172.21.15.86:0/3493550451 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2976285806 0 0) 0x7fc7c4001490 con 0x7fc7d4146130 2024-04-20T21:22:46.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.614+0000 7fc7da400640 1 -- 172.21.15.86:0/3493550451 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7d40cd8e0 con 0x7fc7d4072240 2024-04-20T21:22:46.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.615+0000 7fc7da400640 1 --2- 172.21.15.86:0/3493550451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7d4072240 0x7fc7d415cc20 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4073120 tx=0x7fc7cc02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7cd6e4960d4db2f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:46.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.616+0000 7fc7d3e00640 1 -- 172.21.15.86:0/3493550451 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc034070 con 0x7fc7d4072240 2024-04-20T21:22:46.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.616+0000 7fc7d3e00640 1 -- 172.21.15.86:0/3493550451 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7cc030610 con 0x7fc7d4072240 2024-04-20T21:22:46.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.616+0000 7fc7d3e00640 1 -- 172.21.15.86:0/3493550451 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc0309e0 con 0x7fc7d4072240 2024-04-20T21:22:46.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.616+0000 7fc7dc400640 1 -- 172.21.15.86:0/3493550451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7d4072240 msgr2=0x7fc7d415cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:46.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.616+0000 7fc7dc400640 1 --2- 172.21.15.86:0/3493550451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7d4072240 0x7fc7d415cc20 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4073120 tx=0x7fc7cc02b960 comp rx=0 tx=0).stop 2024-04-20T21:22:46.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.616+0000 7fc7dc400640 1 -- 172.21.15.86:0/3493550451 shutdown_connections 2024-04-20T21:22:46.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.616+0000 7fc7dc400640 1 --2- 172.21.15.86:0/3493550451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7d4072240 0x7fc7d415cc20 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:46.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.616+0000 7fc7dc400640 1 --2- 172.21.15.86:0/3493550451 >> v2:172.21.15.144:3300/0 conn(0x7fc7d406e6d0 0x7fc7d4071d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:46.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.616+0000 7fc7dc400640 1 -- 172.21.15.86:0/3493550451 >> 172.21.15.86:0/3493550451 conn(0x7fc7d40b2360 msgr2=0x7fc7d40b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:46.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.616+0000 7fc7dc400640 1 -- 172.21.15.86:0/3493550451 shutdown_connections 2024-04-20T21:22:46.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.616+0000 7fc7dc400640 1 -- 172.21.15.86:0/3493550451 wait complete. 2024-04-20T21:22:46.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.618+0000 7fc7dc400640 1 Processor -- start 2024-04-20T21:22:46.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.618+0000 7fc7dc400640 1 -- start start 2024-04-20T21:22:46.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.619+0000 7fc7dc400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7d406e6d0 0x7fc7d41f73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:46.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.619+0000 7fc7dc400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7d4072240 0x7fc7d41e7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:46.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.619+0000 7fc7dc400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7d4146130 0x7fc7d41e78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:46.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.619+0000 7fc7dc400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d406e1b0 con 0x7fc7d4146130 2024-04-20T21:22:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.619+0000 7fc7dc400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d406ded0 con 0x7fc7d406e6d0 2024-04-20T21:22:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.619+0000 7fc7dc400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc7d406f150 con 0x7fc7d4072240 2024-04-20T21:22:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.619+0000 7fc7da400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7d4146130 0x7fc7d41e78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.619+0000 7fc7da400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7d4146130 0x7fc7d41e78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51986/0 (socket says 172.21.15.86:51986) 2024-04-20T21:22:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.619+0000 7fc7da400640 1 -- 172.21.15.86:0/2383787067 learned_addr learned my addr 172.21.15.86:0/2383787067 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.619+0000 7fc7d9000640 1 --2- 172.21.15.86:0/2383787067 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7d4072240 0x7fc7d41e7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.620+0000 7fc7da400640 1 -- 172.21.15.86:0/2383787067 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7d4072240 msgr2=0x7fc7d41e7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.620+0000 7fc7da400640 1 --2- 172.21.15.86:0/2383787067 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7d4072240 0x7fc7d41e7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.620+0000 7fc7d9a00640 1 --2- 172.21.15.86:0/2383787067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7d406e6d0 0x7fc7d41f73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.620+0000 7fc7da400640 1 -- 172.21.15.86:0/2383787067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7d406e6d0 msgr2=0x7fc7d41f73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.620+0000 7fc7da400640 1 --2- 172.21.15.86:0/2383787067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7d406e6d0 0x7fc7d41f73a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.620+0000 7fc7da400640 1 -- 172.21.15.86:0/2383787067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7d415d850 con 0x7fc7d4146130 2024-04-20T21:22:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.620+0000 7fc7d9000640 1 --2- 172.21.15.86:0/2383787067 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7d4072240 0x7fc7d41e7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:22:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.620+0000 7fc7da400640 1 --2- 172.21.15.86:0/2383787067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7d4146130 0x7fc7d41e78c0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc009860 tx=0x7fc7cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.620+0000 7fc7d1600640 1 -- 172.21.15.86:0/2383787067 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc034040 con 0x7fc7d4146130 2024-04-20T21:22:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.620+0000 7fc7d1600640 1 -- 172.21.15.86:0/2383787067 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7cc004030 con 0x7fc7d4146130 2024-04-20T21:22:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.620+0000 7fc7d1600640 1 -- 172.21.15.86:0/2383787067 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc0043e0 con 0x7fc7d4146130 2024-04-20T21:22:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.620+0000 7fc7dc400640 1 -- 172.21.15.86:0/2383787067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7d415db30 con 0x7fc7d4146130 2024-04-20T21:22:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.620+0000 7fc7dc400640 1 -- 172.21.15.86:0/2383787067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7d415e0f0 con 0x7fc7d4146130 2024-04-20T21:22:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.621+0000 7fc7dc400640 1 -- 172.21.15.86:0/2383787067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7d40689e0 con 0x7fc7d4146130 2024-04-20T21:22:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.623+0000 7fc7d1600640 1 -- 172.21.15.86:0/2383787067 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc7cc037070 con 0x7fc7d4146130 2024-04-20T21:22:46.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.624+0000 7fc7d1600640 1 --2- 172.21.15.86:0/2383787067 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc7b406c8f0 0x7fc7b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:46.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.624+0000 7fc7d1600640 1 -- 172.21.15.86:0/2383787067 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7fc7cc0ad7d0 con 0x7fc7d4146130 2024-04-20T21:22:46.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.624+0000 7fc7d9a00640 1 --2- 172.21.15.86:0/2383787067 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc7b406c8f0 0x7fc7b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:46.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.625+0000 7fc7d9a00640 1 --2- 172.21.15.86:0/2383787067 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc7b406c8f0 0x7fc7b406edb0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc7c40012e0 tx=0x7fc7c4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:46.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.628+0000 7fc7d1600640 1 -- 172.21.15.86:0/2383787067 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc7cc07bc90 con 0x7fc7d4146130 2024-04-20T21:22:46.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.818+0000 7fc7dc400640 1 -- 172.21.15.86:0/2383787067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc7d4142360 con 0x7fc7d4146130 2024-04-20T21:22:46.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.821+0000 7fc7d1600640 1 -- 172.21.15.86:0/2383787067 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+16681 (secure 0 0 0) 0x7fc7cc07b590 con 0x7fc7d4146130 2024-04-20T21:22:46.820 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:22:46.820 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":70,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:22:45.431170+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:22:43.125767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-20T21:22:44.410549+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:22:46.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.823+0000 7fc7dc400640 1 -- 172.21.15.86:0/2383787067 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc7b406c8f0 msgr2=0x7fc7b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:46.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.823+0000 7fc7dc400640 1 --2- 172.21.15.86:0/2383787067 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc7b406c8f0 0x7fc7b406edb0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc7c40012e0 tx=0x7fc7c4009040 comp rx=0 tx=0).stop 2024-04-20T21:22:46.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.823+0000 7fc7dc400640 1 -- 172.21.15.86:0/2383787067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7d4146130 msgr2=0x7fc7d41e78c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:46.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.823+0000 7fc7dc400640 1 --2- 172.21.15.86:0/2383787067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7d4146130 0x7fc7d41e78c0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc009860 tx=0x7fc7cc0046c0 comp rx=0 tx=0).stop 2024-04-20T21:22:46.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.824+0000 7fc7dc400640 1 -- 172.21.15.86:0/2383787067 shutdown_connections 2024-04-20T21:22:46.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.824+0000 7fc7dc400640 1 --2- 172.21.15.86:0/2383787067 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc7b406c8f0 0x7fc7b406edb0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:46.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.824+0000 7fc7dc400640 1 --2- 172.21.15.86:0/2383787067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc7d4146130 0x7fc7d41e78c0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:46.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.824+0000 7fc7dc400640 1 --2- 172.21.15.86:0/2383787067 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc7d4072240 0x7fc7d41e7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:46.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.824+0000 7fc7dc400640 1 --2- 172.21.15.86:0/2383787067 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7d406e6d0 0x7fc7d41f73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:46.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.824+0000 7fc7dc400640 1 -- 172.21.15.86:0/2383787067 >> 172.21.15.86:0/2383787067 conn(0x7fc7d40b2360 msgr2=0x7fc7d40b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:46.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.824+0000 7fc7dc400640 1 -- 172.21.15.86:0/2383787067 shutdown_connections 2024-04-20T21:22:46.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:46.824+0000 7fc7dc400640 1 -- 172.21.15.86:0/2383787067 wait complete. 2024-04-20T21:22:46.842 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-20T21:22:47.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.048+0000 7febe2c00640 1 Processor -- start 2024-04-20T21:22:47.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.048+0000 7febe2c00640 1 -- start start 2024-04-20T21:22:47.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.049+0000 7febe2c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7febdc155d30 0x7febdc0cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:47.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.049+0000 7febe2c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7febdc0b52b0 0x7febdc06be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:47.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.049+0000 7febe2c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febdc156110 con 0x7febdc06b980 2024-04-20T21:22:47.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.049+0000 7febe2c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7febdc0b56e0 con 0x7febdc0b52b0 2024-04-20T21:22:47.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.049+0000 7febe2c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7febdc0ce460 con 0x7febdc155d30 2024-04-20T21:22:47.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.050+0000 7febe0c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7febdc0b52b0 0x7febdc06be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:47.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.050+0000 7febdb400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7febdc06b980 0x7febdc0b4ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44704/0 (socket says 172.21.15.86:44704) 2024-04-20T21:22:47.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.050+0000 7febe0c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7febdc0b52b0 0x7febdc06be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51988/0 (socket says 172.21.15.86:51988) 2024-04-20T21:22:47.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.050+0000 7febdb400640 1 -- 172.21.15.86:0/3875991999 learned_addr learned my addr 172.21.15.86:0/3875991999 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:22:47.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.050+0000 7febdbe00640 1 --2- 172.21.15.86:0/3875991999 >> v2:172.21.15.144:3300/0 conn(0x7febdc155d30 0x7febdc0cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:47.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.050+0000 7febe0c00640 1 -- 172.21.15.86:0/3875991999 >> v2:172.21.15.144:3300/0 conn(0x7febdc155d30 msgr2=0x7febdc0cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:47.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.050+0000 7febe0c00640 1 --2- 172.21.15.86:0/3875991999 >> v2:172.21.15.144:3300/0 conn(0x7febdc155d30 0x7febdc0cf500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:47.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.050+0000 7febe0c00640 1 -- 172.21.15.86:0/3875991999 >> v1:172.21.15.86:6789/0 conn(0x7febdc06b980 legacy=0x7febdc0b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:47.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.050+0000 7febe0c00640 1 -- 172.21.15.86:0/3875991999 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febdc0ce740 con 0x7febdc0b52b0 2024-04-20T21:22:47.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.050+0000 7febdbe00640 1 --2- 172.21.15.86:0/3875991999 >> v2:172.21.15.144:3300/0 conn(0x7febdc155d30 0x7febdc0cf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:22:47.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.051+0000 7febe0c00640 1 --2- 172.21.15.86:0/3875991999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febdc0b52b0 0x7febdc06be30 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7febdc0b3150 tx=0x7febd402bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36d0711e0223c832 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:47.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.052+0000 7febdaa00640 1 -- 172.21.15.86:0/3875991999 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd4039070 con 0x7febdc0b52b0 2024-04-20T21:22:47.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.052+0000 7febdaa00640 1 -- 172.21.15.86:0/3875991999 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febd402cb50 con 0x7febdc0b52b0 2024-04-20T21:22:47.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.052+0000 7febdaa00640 1 -- 172.21.15.86:0/3875991999 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd4034440 con 0x7febdc0b52b0 2024-04-20T21:22:47.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.052+0000 7febe2c00640 1 -- 172.21.15.86:0/3875991999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febdc0b52b0 msgr2=0x7febdc06be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:47.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.052+0000 7febe2c00640 1 --2- 172.21.15.86:0/3875991999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febdc0b52b0 0x7febdc06be30 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7febdc0b3150 tx=0x7febd402bda0 comp rx=0 tx=0).stop 2024-04-20T21:22:47.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.052+0000 7febe2c00640 1 -- 172.21.15.86:0/3875991999 shutdown_connections 2024-04-20T21:22:47.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.052+0000 7febe2c00640 1 --2- 172.21.15.86:0/3875991999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febdc0b52b0 0x7febdc06be30 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:47.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.052+0000 7febe2c00640 1 --2- 172.21.15.86:0/3875991999 >> v2:172.21.15.144:3300/0 conn(0x7febdc155d30 0x7febdc0cf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:47.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.052+0000 7febe2c00640 1 -- 172.21.15.86:0/3875991999 >> 172.21.15.86:0/3875991999 conn(0x7febdc0b2110 msgr2=0x7febdc0b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:47.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.052+0000 7febe2c00640 1 -- 172.21.15.86:0/3875991999 shutdown_connections 2024-04-20T21:22:47.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.052+0000 7febe2c00640 1 -- 172.21.15.86:0/3875991999 wait complete. 2024-04-20T21:22:47.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.054+0000 7febe2c00640 1 Processor -- start 2024-04-20T21:22:47.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.054+0000 7febe2c00640 1 -- start start 2024-04-20T21:22:47.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.055+0000 7febe2c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febdc06b980 0x7febdc14cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:47.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.055+0000 7febe2c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febdc0b52b0 0x7febdc150110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:47.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.055+0000 7febe2c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7febdc155d30 0x7febdc150650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:47.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.055+0000 7febe2c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7febdc0cefe0 con 0x7febdc06b980 2024-04-20T21:22:47.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.055+0000 7febe2c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7febdc0ced00 con 0x7febdc0b52b0 2024-04-20T21:22:47.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.055+0000 7febe2c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7febdc06ddf0 con 0x7febdc155d30 2024-04-20T21:22:47.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.056+0000 7febdbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febdc06b980 0x7febdc14cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:47.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.056+0000 7febdbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febdc06b980 0x7febdc14cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52000/0 (socket says 172.21.15.86:52000) 2024-04-20T21:22:47.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.056+0000 7febdbe00640 1 -- 172.21.15.86:0/2592752826 learned_addr learned my addr 172.21.15.86:0/2592752826 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:47.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.056+0000 7febdb400640 1 --2- 172.21.15.86:0/2592752826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febdc0b52b0 0x7febdc150110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:47.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.056+0000 7febe0c00640 1 --2- 172.21.15.86:0/2592752826 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7febdc155d30 0x7febdc150650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:47.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.056+0000 7febdbe00640 1 -- 172.21.15.86:0/2592752826 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7febdc155d30 msgr2=0x7febdc150650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:47.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.056+0000 7febdbe00640 1 --2- 172.21.15.86:0/2592752826 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7febdc155d30 0x7febdc150650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:47.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.056+0000 7febdbe00640 1 -- 172.21.15.86:0/2592752826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febdc0b52b0 msgr2=0x7febdc150110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:47.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.056+0000 7febdbe00640 1 --2- 172.21.15.86:0/2592752826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febdc0b52b0 0x7febdc150110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:47.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.056+0000 7febdbe00640 1 -- 172.21.15.86:0/2592752826 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febdc14d010 con 0x7febdc06b980 2024-04-20T21:22:47.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.056+0000 7febdb400640 1 --2- 172.21.15.86:0/2592752826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febdc0b52b0 0x7febdc150110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:22:47.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.056+0000 7febdbe00640 1 --2- 172.21.15.86:0/2592752826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febdc06b980 0x7febdc14cad0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7febcc00c7f0 tx=0x7febcc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:47.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.057+0000 7febd3e00640 1 -- 172.21.15.86:0/2592752826 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febcc015070 con 0x7febdc06b980 2024-04-20T21:22:47.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.057+0000 7febd3e00640 1 -- 172.21.15.86:0/2592752826 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febcc007820 con 0x7febdc06b980 2024-04-20T21:22:47.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.057+0000 7febd3e00640 1 -- 172.21.15.86:0/2592752826 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febcc007bd0 con 0x7febdc06b980 2024-04-20T21:22:47.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.057+0000 7febe2c00640 1 -- 172.21.15.86:0/2592752826 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febdc15da90 con 0x7febdc06b980 2024-04-20T21:22:47.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.057+0000 7febe2c00640 1 -- 172.21.15.86:0/2592752826 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febdc1f5dd0 con 0x7febdc06b980 2024-04-20T21:22:47.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.058+0000 7febe2c00640 1 -- 172.21.15.86:0/2592752826 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febdc0689e0 con 0x7febdc06b980 2024-04-20T21:22:47.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.059+0000 7febd3e00640 1 -- 172.21.15.86:0/2592752826 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7febcc018070 con 0x7febdc06b980 2024-04-20T21:22:47.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.060+0000 7febd3e00640 1 --2- 172.21.15.86:0/2592752826 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7febbc06c8f0 0x7febbc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:47.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.060+0000 7febd3e00640 1 -- 172.21.15.86:0/2592752826 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7febcc08bd90 con 0x7febdc06b980 2024-04-20T21:22:47.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.060+0000 7febdb400640 1 --2- 172.21.15.86:0/2592752826 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7febbc06c8f0 0x7febbc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:47.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.061+0000 7febdb400640 1 --2- 172.21.15.86:0/2592752826 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7febbc06c8f0 0x7febbc06edb0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7febdc14d400 tx=0x7febc8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:47.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.065+0000 7febd3e00640 1 -- 172.21.15.86:0/2592752826 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7febcc05a090 con 0x7febdc06b980 2024-04-20T21:22:47.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.306+0000 7febe2c00640 1 -- 172.21.15.86:0/2592752826 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7febdc1f58c0 con 0x7febdc06b980 2024-04-20T21:22:47.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:47.995+0000 7febd3e00640 1 -- 172.21.15.86:0/2592752826 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v41) ==== 118+0+0 (secure 0 0 0) 0x7febcc059990 con 0x7febdc06b980 2024-04-20T21:22:48.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.000+0000 7febe2c00640 1 -- 172.21.15.86:0/2592752826 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7febbc06c8f0 msgr2=0x7febbc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:48.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.000+0000 7febe2c00640 1 --2- 172.21.15.86:0/2592752826 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7febbc06c8f0 0x7febbc06edb0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7febdc14d400 tx=0x7febc8009040 comp rx=0 tx=0).stop 2024-04-20T21:22:48.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.001+0000 7febe2c00640 1 -- 172.21.15.86:0/2592752826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febdc06b980 msgr2=0x7febdc14cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:48.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.001+0000 7febe2c00640 1 --2- 172.21.15.86:0/2592752826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febdc06b980 0x7febdc14cad0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7febcc00c7f0 tx=0x7febcc00ccb0 comp rx=0 tx=0).stop 2024-04-20T21:22:48.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.001+0000 7febe2c00640 1 -- 172.21.15.86:0/2592752826 shutdown_connections 2024-04-20T21:22:48.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.001+0000 7febe2c00640 1 --2- 172.21.15.86:0/2592752826 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7febbc06c8f0 0x7febbc06edb0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:48.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.001+0000 7febe2c00640 1 --2- 172.21.15.86:0/2592752826 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7febdc155d30 0x7febdc150650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:48.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.001+0000 7febe2c00640 1 --2- 172.21.15.86:0/2592752826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febdc0b52b0 0x7febdc150110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:48.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.001+0000 7febe2c00640 1 --2- 172.21.15.86:0/2592752826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febdc06b980 0x7febdc14cad0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:48.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.001+0000 7febe2c00640 1 -- 172.21.15.86:0/2592752826 >> 172.21.15.86:0/2592752826 conn(0x7febdc0b2110 msgr2=0x7febdc0b18a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:48.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.001+0000 7febe2c00640 1 -- 172.21.15.86:0/2592752826 shutdown_connections 2024-04-20T21:22:48.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.001+0000 7febe2c00640 1 -- 172.21.15.86:0/2592752826 wait complete. 2024-04-20T21:22:48.023 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-20T21:22:48.023 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-20T21:22:48.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.232+0000 7f6da1400640 1 Processor -- start 2024-04-20T21:22:48.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.233+0000 7f6da1400640 1 -- start start 2024-04-20T21:22:48.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.234+0000 7f6da1400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6d9c15eb30 0x7f6d9c0b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:48.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.234+0000 7f6da1400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d9c0adb10 con 0x7f6d9c0aee50 2024-04-20T21:22:48.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.234+0000 7f6da1400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d9c0addb0 con 0x7f6d9c06c090 2024-04-20T21:22:48.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.234+0000 7f6da1400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f6d9c0af7d0 con 0x7f6d9c15eb30 2024-04-20T21:22:48.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.235+0000 7f6d91e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f6d9c0aee50 0x7f6d9c06b980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44712/0 (socket says 172.21.15.86:44712) 2024-04-20T21:22:48.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.235+0000 7f6d91e00640 1 -- 172.21.15.86:0/3634439166 learned_addr learned my addr 172.21.15.86:0/3634439166 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:22:48.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.235+0000 7f6d9a800640 1 --2- 172.21.15.86:0/3634439166 >> v2:172.21.15.144:3300/0 conn(0x7f6d9c15eb30 0x7f6d9c0b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:48.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.235+0000 7f6d99e00640 1 -- 172.21.15.86:0/3634439166 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3978855271 0 0) 0x7f6d9c0adb10 con 0x7f6d9c0aee50 2024-04-20T21:22:48.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.235+0000 7f6d99e00640 1 -- 172.21.15.86:0/3634439166 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d80003520 con 0x7f6d9c0aee50 2024-04-20T21:22:48.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.235+0000 7f6d99e00640 1 -- 172.21.15.86:0/3634439166 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4201715469 0 0) 0x7f6d80003520 con 0x7f6d9c0aee50 2024-04-20T21:22:48.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.235+0000 7f6d99e00640 1 -- 172.21.15.86:0/3634439166 >> v2:172.21.15.144:3300/0 conn(0x7f6d9c15eb30 msgr2=0x7f6d9c0b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:48.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.235+0000 7f6d99e00640 1 --2- 172.21.15.86:0/3634439166 >> v2:172.21.15.144:3300/0 conn(0x7f6d9c15eb30 0x7f6d9c0b3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:48.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.235+0000 7f6d99e00640 1 -- 172.21.15.86:0/3634439166 >> v1:172.21.15.144:6789/0 conn(0x7f6d9c06c090 legacy=0x7f6d9c06e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:48.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.235+0000 7f6d99e00640 1 -- 172.21.15.86:0/3634439166 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d9c0ae090 con 0x7f6d9c0aee50 2024-04-20T21:22:48.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.236+0000 7f6d99e00640 1 -- 172.21.15.86:0/3634439166 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f6d84001670 con 0x7f6d9c0aee50 2024-04-20T21:22:48.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.236+0000 7f6d99e00640 1 -- 172.21.15.86:0/3634439166 >> v1:172.21.15.86:6789/0 conn(0x7f6d9c0aee50 legacy=0x7f6d9c06b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:48.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.236+0000 7f6d99e00640 1 --2- 172.21.15.86:0/3634439166 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6d80003a00 0x7f6d80003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:48.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.236+0000 7f6d99e00640 1 --2- 172.21.15.86:0/3634439166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d80004800 0x7f6d80004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:48.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.236+0000 7f6d99e00640 1 --2- 172.21.15.86:0/3634439166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d800051c0 0x7f6d8000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:48.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.236+0000 7f6d99e00640 1 -- 172.21.15.86:0/3634439166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9c0af7d0 con 0x7f6d800051c0 2024-04-20T21:22:48.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.236+0000 7f6d99e00640 1 -- 172.21.15.86:0/3634439166 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8000c0c0 con 0x7f6d80004800 2024-04-20T21:22:48.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.236+0000 7f6d99e00640 1 -- 172.21.15.86:0/3634439166 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6d8000c2c0 con 0x7f6d80003a00 2024-04-20T21:22:48.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.236+0000 7f6d9b200640 1 --2- 172.21.15.86:0/3634439166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d800051c0 0x7f6d8000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:48.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.236+0000 7f6d9a800640 1 --2- 172.21.15.86:0/3634439166 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6d80003a00 0x7f6d80003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:48.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.236+0000 7f6d9b200640 1 -- 172.21.15.86:0/3634439166 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6d80003a00 msgr2=0x7f6d80003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:48.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.236+0000 7f6d9b200640 1 --2- 172.21.15.86:0/3634439166 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6d80003a00 0x7f6d80003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:48.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.236+0000 7f6d9b200640 1 -- 172.21.15.86:0/3634439166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d80004800 msgr2=0x7f6d80004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:48.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.236+0000 7f6d9b200640 1 --2- 172.21.15.86:0/3634439166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d80004800 0x7f6d80004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:48.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.236+0000 7f6d9b200640 1 -- 172.21.15.86:0/3634439166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d80003520 con 0x7f6d800051c0 2024-04-20T21:22:48.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.237+0000 7f6d9b200640 1 -- 172.21.15.86:0/3634439166 reap_dead start 2024-04-20T21:22:48.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.238+0000 7f6d9b200640 1 --2- 172.21.15.86:0/3634439166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d800051c0 0x7f6d8000b9c0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c070100 tx=0x7f6d80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12c82b17dbea11db server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:48.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.238+0000 7f6d99e00640 1 -- 172.21.15.86:0/3634439166 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d94001a70 con 0x7f6d800051c0 2024-04-20T21:22:48.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.238+0000 7f6d99e00640 1 -- 172.21.15.86:0/3634439166 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d94003b70 con 0x7f6d800051c0 2024-04-20T21:22:48.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.238+0000 7f6d99e00640 1 -- 172.21.15.86:0/3634439166 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d940050d0 con 0x7f6d800051c0 2024-04-20T21:22:48.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.238+0000 7f6da1400640 1 -- 172.21.15.86:0/3634439166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d800051c0 msgr2=0x7f6d8000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:48.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.238+0000 7f6da1400640 1 --2- 172.21.15.86:0/3634439166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d800051c0 0x7f6d8000b9c0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c070100 tx=0x7f6d80005830 comp rx=0 tx=0).stop 2024-04-20T21:22:48.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.238+0000 7f6da1400640 1 -- 172.21.15.86:0/3634439166 shutdown_connections 2024-04-20T21:22:48.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.238+0000 7f6da1400640 1 --2- 172.21.15.86:0/3634439166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d800051c0 0x7f6d8000b9c0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:48.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.238+0000 7f6da1400640 1 -- 172.21.15.86:0/3634439166 >> 172.21.15.86:0/3634439166 conn(0x7f6d9c0b2110 msgr2=0x7f6d9c0b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:48.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.239+0000 7f6da1400640 1 -- 172.21.15.86:0/3634439166 shutdown_connections 2024-04-20T21:22:48.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.239+0000 7f6da1400640 1 -- 172.21.15.86:0/3634439166 wait complete. 2024-04-20T21:22:48.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.240+0000 7f6da1400640 1 Processor -- start 2024-04-20T21:22:48.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.241+0000 7f6da1400640 1 -- start start 2024-04-20T21:22:48.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.241+0000 7f6da1400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6d800051c0 0x7f6d9c0cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:48.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.241+0000 7f6da1400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d9c158530 0x7f6d9c149db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:48.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.241+0000 7f6da1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d9c0cf4c0 0x7f6d9c15cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:48.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.241+0000 7f6da1400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9c0ae930 con 0x7f6d9c0cf4c0 2024-04-20T21:22:48.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.241+0000 7f6da1400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9c0ae650 con 0x7f6d9c158530 2024-04-20T21:22:48.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.242+0000 7f6da1400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6d9c06b980 con 0x7f6d800051c0 2024-04-20T21:22:48.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.242+0000 7f6d9b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d9c0cf4c0 0x7f6d9c15cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:48.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.242+0000 7f6d9b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d9c0cf4c0 0x7f6d9c15cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52018/0 (socket says 172.21.15.86:52018) 2024-04-20T21:22:48.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.242+0000 7f6d9b200640 1 -- 172.21.15.86:0/1480900470 learned_addr learned my addr 172.21.15.86:0/1480900470 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:48.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.242+0000 7f6d9a800640 1 --2- 172.21.15.86:0/1480900470 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6d800051c0 0x7f6d9c0cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:48.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.242+0000 7f6d91e00640 1 --2- 172.21.15.86:0/1480900470 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d9c158530 0x7f6d9c149db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:48.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.242+0000 7f6d9b200640 1 -- 172.21.15.86:0/1480900470 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6d800051c0 msgr2=0x7f6d9c0cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:48.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.242+0000 7f6d9b200640 1 --2- 172.21.15.86:0/1480900470 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6d800051c0 0x7f6d9c0cdd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:48.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.242+0000 7f6d9b200640 1 -- 172.21.15.86:0/1480900470 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d9c158530 msgr2=0x7f6d9c149db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:48.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.242+0000 7f6d9b200640 1 --2- 172.21.15.86:0/1480900470 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d9c158530 0x7f6d9c149db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:48.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.242+0000 7f6d9b200640 1 -- 172.21.15.86:0/1480900470 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d9c158990 con 0x7f6d9c0cf4c0 2024-04-20T21:22:48.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.242+0000 7f6d91e00640 1 --2- 172.21.15.86:0/1480900470 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d9c158530 0x7f6d9c149db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:22:48.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.243+0000 7f6d9b200640 1 --2- 172.21.15.86:0/1480900470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d9c0cf4c0 0x7f6d9c15cc50 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f6d9402c020 tx=0x7f6d940014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:48.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.243+0000 7f6d93400640 1 -- 172.21.15.86:0/1480900470 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d94006b50 con 0x7f6d9c0cf4c0 2024-04-20T21:22:48.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.243+0000 7f6d93400640 1 -- 172.21.15.86:0/1480900470 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d94006cf0 con 0x7f6d9c0cf4c0 2024-04-20T21:22:48.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.243+0000 7f6d93400640 1 -- 172.21.15.86:0/1480900470 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d94007250 con 0x7f6d9c0cf4c0 2024-04-20T21:22:48.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.243+0000 7f6da1400640 1 -- 172.21.15.86:0/1480900470 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d9c0ce420 con 0x7f6d9c0cf4c0 2024-04-20T21:22:48.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.243+0000 7f6da1400640 1 -- 172.21.15.86:0/1480900470 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d9c0ce9e0 con 0x7f6d9c0cf4c0 2024-04-20T21:22:48.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.244+0000 7f6da1400640 1 -- 172.21.15.86:0/1480900470 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d9c0689e0 con 0x7f6d9c0cf4c0 2024-04-20T21:22:48.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.245+0000 7f6d93400640 1 -- 172.21.15.86:0/1480900470 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6d940073f0 con 0x7f6d9c0cf4c0 2024-04-20T21:22:48.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.247+0000 7f6d93400640 1 --2- 172.21.15.86:0/1480900470 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6d7806c8f0 0x7f6d7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:48.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.247+0000 7f6d93400640 1 -- 172.21.15.86:0/1480900470 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f6d940ace50 con 0x7f6d9c0cf4c0 2024-04-20T21:22:48.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.247+0000 7f6d9a800640 1 --2- 172.21.15.86:0/1480900470 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6d7806c8f0 0x7f6d7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:48.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.248+0000 7f6d9a800640 1 --2- 172.21.15.86:0/1480900470 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6d7806c8f0 0x7f6d7806edb0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c009990 tx=0x7f6d8c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:48.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.252+0000 7f6d93400640 1 -- 172.21.15.86:0/1480900470 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6d94049020 con 0x7f6d9c0cf4c0 2024-04-20T21:22:48.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.491+0000 7f6da1400640 1 -- 172.21.15.86:0/1480900470 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f6d9c0b4370 con 0x7f6d9c0cf4c0 2024-04-20T21:22:48.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:48.998+0000 7f6d93400640 1 -- 172.21.15.86:0/1480900470 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v42) ==== 122+0+0 (secure 0 0 0) 0x7f6d9403f070 con 0x7f6d9c0cf4c0 2024-04-20T21:22:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.001+0000 7f6da1400640 1 -- 172.21.15.86:0/1480900470 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6d7806c8f0 msgr2=0x7f6d7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.001+0000 7f6da1400640 1 --2- 172.21.15.86:0/1480900470 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6d7806c8f0 0x7f6d7806edb0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c009990 tx=0x7f6d8c008040 comp rx=0 tx=0).stop 2024-04-20T21:22:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.001+0000 7f6da1400640 1 -- 172.21.15.86:0/1480900470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d9c0cf4c0 msgr2=0x7f6d9c15cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.001+0000 7f6da1400640 1 --2- 172.21.15.86:0/1480900470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d9c0cf4c0 0x7f6d9c15cc50 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f6d9402c020 tx=0x7f6d940014a0 comp rx=0 tx=0).stop 2024-04-20T21:22:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.002+0000 7f6da1400640 1 -- 172.21.15.86:0/1480900470 shutdown_connections 2024-04-20T21:22:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.002+0000 7f6da1400640 1 --2- 172.21.15.86:0/1480900470 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6d7806c8f0 0x7f6d7806edb0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.002+0000 7f6da1400640 1 --2- 172.21.15.86:0/1480900470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d9c0cf4c0 0x7f6d9c15cc50 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.002+0000 7f6da1400640 1 --2- 172.21.15.86:0/1480900470 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d9c158530 0x7f6d9c149db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:49.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.002+0000 7f6da1400640 1 --2- 172.21.15.86:0/1480900470 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6d800051c0 0x7f6d9c0cdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:49.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.002+0000 7f6da1400640 1 -- 172.21.15.86:0/1480900470 >> 172.21.15.86:0/1480900470 conn(0x7f6d9c0b2110 msgr2=0x7f6d9c0b0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:49.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.002+0000 7f6da1400640 1 -- 172.21.15.86:0/1480900470 shutdown_connections 2024-04-20T21:22:49.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.002+0000 7f6da1400640 1 -- 172.21.15.86:0/1480900470 wait complete. 2024-04-20T21:22:49.022 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-20T21:22:49.022 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:22:49.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.218+0000 7f8884a00640 1 Processor -- start 2024-04-20T21:22:49.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.218+0000 7f8884a00640 1 -- start start 2024-04-20T21:22:49.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.220+0000 7f8884a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f888006e6d0 0x7f8880071d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:49.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.220+0000 7f8884a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8880072240 0x7f888015cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:49.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.220+0000 7f8884a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f888006eb30 con 0x7f8880146130 2024-04-20T21:22:49.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.220+0000 7f8884a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f888006dc30 con 0x7f888006e6d0 2024-04-20T21:22:49.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.220+0000 7f8884a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f88800cd8e0 con 0x7f8880072240 2024-04-20T21:22:49.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.220+0000 7f887d400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f888006e6d0 0x7f8880071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:49.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.220+0000 7f887d400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f888006e6d0 0x7f8880071d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52030/0 (socket says 172.21.15.86:52030) 2024-04-20T21:22:49.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.220+0000 7f887d400640 1 -- 172.21.15.86:0/2921626356 learned_addr learned my addr 172.21.15.86:0/2921626356 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:49.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.221+0000 7f887e800640 1 --2- 172.21.15.86:0/2921626356 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8880072240 0x7f888015cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:49.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.221+0000 7f887d400640 1 -- 172.21.15.86:0/2921626356 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8880072240 msgr2=0x7f888015cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:49.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.221+0000 7f887d400640 1 --2- 172.21.15.86:0/2921626356 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8880072240 0x7f888015cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:49.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.221+0000 7f887d400640 1 -- 172.21.15.86:0/2921626356 >> v1:172.21.15.86:6789/0 conn(0x7f8880146130 legacy=0x7f88800ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:49.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.221+0000 7f887d400640 1 -- 172.21.15.86:0/2921626356 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88800cdbc0 con 0x7f888006e6d0 2024-04-20T21:22:49.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.221+0000 7f887e800640 1 --2- 172.21.15.86:0/2921626356 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8880072240 0x7f888015cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:22:49.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.221+0000 7f887ca00640 1 -- 172.21.15.86:0/2921626356 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3098187062 0 0) 0x7f8870001110 con 0x7f8880146130 2024-04-20T21:22:49.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.222+0000 7f887d400640 1 --2- 172.21.15.86:0/2921626356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f888006e6d0 0x7f8880071d00 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f886c009870 tx=0x7f886c02ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52fdc017d6a72aaa server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:49.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.222+0000 7f887ca00640 1 -- 172.21.15.86:0/2921626356 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f886c02f840 con 0x7f888006e6d0 2024-04-20T21:22:49.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.222+0000 7f887ca00640 1 -- 172.21.15.86:0/2921626356 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f886c02f9e0 con 0x7f888006e6d0 2024-04-20T21:22:49.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.222+0000 7f887ca00640 1 -- 172.21.15.86:0/2921626356 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f886c02fdb0 con 0x7f888006e6d0 2024-04-20T21:22:49.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.223+0000 7f8884a00640 1 -- 172.21.15.86:0/2921626356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f888006e6d0 msgr2=0x7f8880071d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:49.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.223+0000 7f8884a00640 1 --2- 172.21.15.86:0/2921626356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f888006e6d0 0x7f8880071d00 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f886c009870 tx=0x7f886c02ba20 comp rx=0 tx=0).stop 2024-04-20T21:22:49.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.223+0000 7f8884a00640 1 -- 172.21.15.86:0/2921626356 shutdown_connections 2024-04-20T21:22:49.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.223+0000 7f8884a00640 1 --2- 172.21.15.86:0/2921626356 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8880072240 0x7f888015cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:49.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.223+0000 7f8884a00640 1 --2- 172.21.15.86:0/2921626356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f888006e6d0 0x7f8880071d00 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:49.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.223+0000 7f8884a00640 1 -- 172.21.15.86:0/2921626356 >> 172.21.15.86:0/2921626356 conn(0x7f88800b2360 msgr2=0x7f88800b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:49.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.223+0000 7f8884a00640 1 -- 172.21.15.86:0/2921626356 shutdown_connections 2024-04-20T21:22:49.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.223+0000 7f8884a00640 1 -- 172.21.15.86:0/2921626356 wait complete. 2024-04-20T21:22:49.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.225+0000 7f8884a00640 1 Processor -- start 2024-04-20T21:22:49.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.225+0000 7f8884a00640 1 -- start start 2024-04-20T21:22:49.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.226+0000 7f8884a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f888006e6d0 0x7f888013da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:49.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.226+0000 7f8884a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8880072240 0x7f888013a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:49.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.226+0000 7f8884a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8880146130 0x7f888013b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:49.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.226+0000 7f8884a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f888006f150 con 0x7f8880146130 2024-04-20T21:22:49.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.226+0000 7f8884a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f888006e1b0 con 0x7f8880072240 2024-04-20T21:22:49.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.226+0000 7f8884a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f88800cd4e0 con 0x7f888006e6d0 2024-04-20T21:22:49.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.226+0000 7f887e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8880146130 0x7f888013b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:49.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.226+0000 7f887e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8880146130 0x7f888013b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52032/0 (socket says 172.21.15.86:52032) 2024-04-20T21:22:49.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.226+0000 7f887e800640 1 -- 172.21.15.86:0/3227297769 learned_addr learned my addr 172.21.15.86:0/3227297769 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:49.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.227+0000 7f887d400640 1 --2- 172.21.15.86:0/3227297769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8880072240 0x7f888013a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:49.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.227+0000 7f887e800640 1 -- 172.21.15.86:0/3227297769 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f888006e6d0 msgr2=0x7f888013da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:49.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.227+0000 7f887de00640 1 --2- 172.21.15.86:0/3227297769 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f888006e6d0 0x7f888013da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:49.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.227+0000 7f887e800640 1 --2- 172.21.15.86:0/3227297769 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f888006e6d0 0x7f888013da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:49.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.227+0000 7f887e800640 1 -- 172.21.15.86:0/3227297769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8880072240 msgr2=0x7f888013a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:49.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.227+0000 7f887e800640 1 --2- 172.21.15.86:0/3227297769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8880072240 0x7f888013a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:49.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.227+0000 7f887e800640 1 -- 172.21.15.86:0/3227297769 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f888013df40 con 0x7f8880146130 2024-04-20T21:22:49.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.227+0000 7f887d400640 1 --2- 172.21.15.86:0/3227297769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8880072240 0x7f888013a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:22:49.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.227+0000 7f887de00640 1 --2- 172.21.15.86:0/3227297769 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f888006e6d0 0x7f888013da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:22:49.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.227+0000 7f887e800640 1 --2- 172.21.15.86:0/3227297769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8880146130 0x7f888013b4d0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f887800cf20 tx=0x7f887800a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:49.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.227+0000 7f8876000640 1 -- 172.21.15.86:0/3227297769 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887801a070 con 0x7f8880146130 2024-04-20T21:22:49.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.227+0000 7f8876000640 1 -- 172.21.15.86:0/3227297769 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f887800d840 con 0x7f8880146130 2024-04-20T21:22:49.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.227+0000 7f8884a00640 1 -- 172.21.15.86:0/3227297769 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8880139160 con 0x7f8880146130 2024-04-20T21:22:49.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.227+0000 7f8876000640 1 -- 172.21.15.86:0/3227297769 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887800dc30 con 0x7f8880146130 2024-04-20T21:22:49.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.227+0000 7f8884a00640 1 -- 172.21.15.86:0/3227297769 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8880139720 con 0x7f8880146130 2024-04-20T21:22:49.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.228+0000 7f8884a00640 1 -- 172.21.15.86:0/3227297769 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88800689e0 con 0x7f8880146130 2024-04-20T21:22:49.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.229+0000 7f8876000640 1 -- 172.21.15.86:0/3227297769 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f887801d070 con 0x7f8880146130 2024-04-20T21:22:49.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.231+0000 7f8876000640 1 --2- 172.21.15.86:0/3227297769 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f886006c8f0 0x7f886006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:49.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.231+0000 7f8876000640 1 -- 172.21.15.86:0/3227297769 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f887808c4b0 con 0x7f8880146130 2024-04-20T21:22:49.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.231+0000 7f887de00640 1 --2- 172.21.15.86:0/3227297769 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f886006c8f0 0x7f886006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:49.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.232+0000 7f887de00640 1 --2- 172.21.15.86:0/3227297769 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f886006c8f0 0x7f886006edb0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8870000f60 tx=0x7f8870009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:49.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.236+0000 7f8876000640 1 -- 172.21.15.86:0/3227297769 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f887805a840 con 0x7f8880146130 2024-04-20T21:22:49.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.478+0000 7f8884a00640 1 -- 172.21.15.86:0/3227297769 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f888015ff50 con 0x7f8880146130 2024-04-20T21:22:49.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.479+0000 7f8876000640 1 -- 172.21.15.86:0/3227297769 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+5201 (secure 0 0 0) 0x7f887805a140 con 0x7f8880146130 2024-04-20T21:22:49.479 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:22:49.479 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":42,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2740872549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2740872549},{"type":"v1","addr":"172.21.15.86:6836","nonce":2740872549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5048,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/301899138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":301899138},{"type":"v1","addr":"172.21.15.144:6833","nonce":301899138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5051,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2816343379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2816343379},{"type":"v1","addr":"172.21.15.144:6835","nonce":2816343379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:22:45.431100+0000","modified":"2024-04-20T21:22:48.492788+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5407":{"gid":5407,"name":"a","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/202143875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":202143875},{"type":"v1","addr":"172.21.15.86:6837","nonce":202143875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5407,"qdb_cluster":[5407]},"id":5}]} 2024-04-20T21:22:49.479 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 42 2024-04-20T21:22:49.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.482+0000 7f8884a00640 1 -- 172.21.15.86:0/3227297769 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f886006c8f0 msgr2=0x7f886006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:49.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.482+0000 7f8884a00640 1 --2- 172.21.15.86:0/3227297769 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f886006c8f0 0x7f886006edb0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8870000f60 tx=0x7f8870009040 comp rx=0 tx=0).stop 2024-04-20T21:22:49.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.482+0000 7f8884a00640 1 -- 172.21.15.86:0/3227297769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8880146130 msgr2=0x7f888013b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:49.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.482+0000 7f8884a00640 1 --2- 172.21.15.86:0/3227297769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8880146130 0x7f888013b4d0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f887800cf20 tx=0x7f887800a7d0 comp rx=0 tx=0).stop 2024-04-20T21:22:49.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.482+0000 7f8884a00640 1 -- 172.21.15.86:0/3227297769 shutdown_connections 2024-04-20T21:22:49.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.482+0000 7f8884a00640 1 --2- 172.21.15.86:0/3227297769 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f886006c8f0 0x7f886006edb0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:49.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.482+0000 7f8884a00640 1 --2- 172.21.15.86:0/3227297769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8880146130 0x7f888013b4d0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:49.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.482+0000 7f8884a00640 1 --2- 172.21.15.86:0/3227297769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8880072240 0x7f888013a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:49.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.482+0000 7f8884a00640 1 --2- 172.21.15.86:0/3227297769 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f888006e6d0 0x7f888013da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:49.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.483+0000 7f8884a00640 1 -- 172.21.15.86:0/3227297769 >> 172.21.15.86:0/3227297769 conn(0x7f88800b2360 msgr2=0x7f88800b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:49.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.483+0000 7f8884a00640 1 -- 172.21.15.86:0/3227297769 shutdown_connections 2024-04-20T21:22:49.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.483+0000 7f8884a00640 1 -- 172.21.15.86:0/3227297769 wait complete. 2024-04-20T21:22:49.500 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:22:49.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.699+0000 7fe589200640 1 Processor -- start 2024-04-20T21:22:49.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.699+0000 7fe589200640 1 -- start start 2024-04-20T21:22:49.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.700+0000 7fe589200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe58415ac10 0x7fe5840b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:49.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.700+0000 7fe589200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe58415f210 0x7fe584072590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:49.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.700+0000 7fe589200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe58406dcd0 con 0x7fe584072ad0 2024-04-20T21:22:49.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.700+0000 7fe589200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fe58406e8e0 con 0x7fe58415ac10 2024-04-20T21:22:49.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.700+0000 7fe589200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe58406dfb0 con 0x7fe58415f210 2024-04-20T21:22:49.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.700+0000 7fe582800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe584072ad0 0x7fe5840b3480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44724/0 (socket says 172.21.15.86:44724) 2024-04-20T21:22:49.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.700+0000 7fe582800640 1 -- 172.21.15.86:0/2997012036 learned_addr learned my addr 172.21.15.86:0/2997012036 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:22:49.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.701+0000 7fe581e00640 1 --2- 172.21.15.86:0/2997012036 >> v2:172.21.15.144:3300/0 conn(0x7fe58415ac10 0x7fe5840b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:49.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.701+0000 7fe579400640 1 --2- 172.21.15.86:0/2997012036 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe58415f210 0x7fe584072590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:49.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.701+0000 7fe581400640 1 -- 172.21.15.86:0/2997012036 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2271434251 0 0) 0x7fe58406dcd0 con 0x7fe584072ad0 2024-04-20T21:22:49.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.701+0000 7fe581400640 1 -- 172.21.15.86:0/2997012036 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe568003520 con 0x7fe584072ad0 2024-04-20T21:22:49.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.701+0000 7fe581400640 1 -- 172.21.15.86:0/2997012036 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4052879818 0 0) 0x7fe568003520 con 0x7fe584072ad0 2024-04-20T21:22:49.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.701+0000 7fe581400640 1 -- 172.21.15.86:0/2997012036 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe58415f210 msgr2=0x7fe584072590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:49.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.701+0000 7fe581400640 1 --2- 172.21.15.86:0/2997012036 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe58415f210 0x7fe584072590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:49.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.701+0000 7fe581400640 1 -- 172.21.15.86:0/2997012036 >> v2:172.21.15.144:3300/0 conn(0x7fe58415ac10 msgr2=0x7fe5840b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:49.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.701+0000 7fe581400640 1 --2- 172.21.15.86:0/2997012036 >> v2:172.21.15.144:3300/0 conn(0x7fe58415ac10 0x7fe5840b53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:49.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.701+0000 7fe581400640 1 -- 172.21.15.86:0/2997012036 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe58406e290 con 0x7fe584072ad0 2024-04-20T21:22:49.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.701+0000 7fe581400640 1 -- 172.21.15.86:0/2997012036 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fe57c0019f0 con 0x7fe584072ad0 2024-04-20T21:22:49.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.701+0000 7fe579400640 1 --2- 172.21.15.86:0/2997012036 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe58415f210 0x7fe584072590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:22:49.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.701+0000 7fe581400640 1 -- 172.21.15.86:0/2997012036 >> v1:172.21.15.86:6789/0 conn(0x7fe584072ad0 legacy=0x7fe5840b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:49.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.701+0000 7fe581400640 1 --2- 172.21.15.86:0/2997012036 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe58415f210 0x7fe5680039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:49.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.701+0000 7fe581400640 1 --2- 172.21.15.86:0/2997012036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe568003ef0 0x7fe568004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:49.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.701+0000 7fe581400640 1 --2- 172.21.15.86:0/2997012036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe568004d40 0x7fe5680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:49.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.702+0000 7fe581400640 1 -- 172.21.15.86:0/2997012036 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe58406dfb0 con 0x7fe568003ef0 2024-04-20T21:22:49.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.702+0000 7fe581400640 1 -- 172.21.15.86:0/2997012036 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe56800dc00 con 0x7fe568004d40 2024-04-20T21:22:49.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.702+0000 7fe581400640 1 -- 172.21.15.86:0/2997012036 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe56800de00 con 0x7fe58415f210 2024-04-20T21:22:49.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.702+0000 7fe579400640 1 --2- 172.21.15.86:0/2997012036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe568003ef0 0x7fe568004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:49.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.702+0000 7fe581e00640 1 --2- 172.21.15.86:0/2997012036 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe58415f210 0x7fe5680039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:49.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.702+0000 7fe579400640 1 -- 172.21.15.86:0/2997012036 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe58415f210 msgr2=0x7fe5680039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:49.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.702+0000 7fe582800640 1 --2- 172.21.15.86:0/2997012036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe568004d40 0x7fe5680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:49.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.702+0000 7fe579400640 1 --2- 172.21.15.86:0/2997012036 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe58415f210 0x7fe5680039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:49.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.702+0000 7fe579400640 1 -- 172.21.15.86:0/2997012036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe568004d40 msgr2=0x7fe5680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:49.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.702+0000 7fe579400640 1 --2- 172.21.15.86:0/2997012036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe568004d40 0x7fe5680051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:49.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.702+0000 7fe579400640 1 -- 172.21.15.86:0/2997012036 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe568003520 con 0x7fe568003ef0 2024-04-20T21:22:49.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.702+0000 7fe582800640 1 --2- 172.21.15.86:0/2997012036 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe568004d40 0x7fe5680051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:22:49.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.703+0000 7fe579400640 1 --2- 172.21.15.86:0/2997012036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe568003ef0 0x7fe568004800 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fe56c0098a0 tx=0x7fe568005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd771677d4ef49ef server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:49.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.703+0000 7fe581400640 1 -- 172.21.15.86:0/2997012036 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c00f040 con 0x7fe568003ef0 2024-04-20T21:22:49.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.704+0000 7fe581400640 1 -- 172.21.15.86:0/2997012036 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe56c004230 con 0x7fe568003ef0 2024-04-20T21:22:49.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.704+0000 7fe581400640 1 -- 172.21.15.86:0/2997012036 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c003610 con 0x7fe568003ef0 2024-04-20T21:22:49.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.704+0000 7fe589200640 1 -- 172.21.15.86:0/2997012036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe568003ef0 msgr2=0x7fe568004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:49.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.704+0000 7fe589200640 1 --2- 172.21.15.86:0/2997012036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe568003ef0 0x7fe568004800 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fe56c0098a0 tx=0x7fe568005830 comp rx=0 tx=0).stop 2024-04-20T21:22:49.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.704+0000 7fe582800640 1 -- 172.21.15.86:0/2997012036 reap_dead start 2024-04-20T21:22:49.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.704+0000 7fe589200640 1 -- 172.21.15.86:0/2997012036 shutdown_connections 2024-04-20T21:22:49.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.704+0000 7fe589200640 1 -- 172.21.15.86:0/2997012036 >> 172.21.15.86:0/2997012036 conn(0x7fe5840b2360 msgr2=0x7fe5840b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:49.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.704+0000 7fe589200640 1 -- 172.21.15.86:0/2997012036 shutdown_connections 2024-04-20T21:22:49.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.704+0000 7fe589200640 1 -- 172.21.15.86:0/2997012036 wait complete. 2024-04-20T21:22:49.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.706+0000 7fe589200640 1 Processor -- start 2024-04-20T21:22:49.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.706+0000 7fe589200640 1 -- start start 2024-04-20T21:22:49.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.707+0000 7fe589200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe568003ef0 0x7fe5841f3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:49.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.707+0000 7fe589200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe58414fc50 0x7fe58415be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:49.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.707+0000 7fe589200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe584150b40 0x7fe58415c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:49.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.707+0000 7fe589200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe58415f650 con 0x7fe58414fc50 2024-04-20T21:22:49.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.707+0000 7fe589200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe58415aff0 con 0x7fe584150b40 2024-04-20T21:22:49.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.707+0000 7fe589200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe58415f7d0 con 0x7fe568003ef0 2024-04-20T21:22:49.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.707+0000 7fe579400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe58414fc50 0x7fe58415be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:49.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.707+0000 7fe579400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe58414fc50 0x7fe58415be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52052/0 (socket says 172.21.15.86:52052) 2024-04-20T21:22:49.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.707+0000 7fe579400640 1 -- 172.21.15.86:0/1610811796 learned_addr learned my addr 172.21.15.86:0/1610811796 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:49.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.708+0000 7fe582800640 1 --2- 172.21.15.86:0/1610811796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe584150b40 0x7fe58415c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:49.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.708+0000 7fe581e00640 1 --2- 172.21.15.86:0/1610811796 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe568003ef0 0x7fe5841f3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:49.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.708+0000 7fe579400640 1 -- 172.21.15.86:0/1610811796 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe568003ef0 msgr2=0x7fe5841f3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:49.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.708+0000 7fe579400640 1 --2- 172.21.15.86:0/1610811796 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe568003ef0 0x7fe5841f3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:49.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.708+0000 7fe579400640 1 -- 172.21.15.86:0/1610811796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe584150b40 msgr2=0x7fe58415c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:49.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.708+0000 7fe579400640 1 --2- 172.21.15.86:0/1610811796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe584150b40 0x7fe58415c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:49.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.708+0000 7fe579400640 1 -- 172.21.15.86:0/1610811796 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5841510e0 con 0x7fe58414fc50 2024-04-20T21:22:49.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.708+0000 7fe582800640 1 --2- 172.21.15.86:0/1610811796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe584150b40 0x7fe58415c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:22:49.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.708+0000 7fe579400640 1 --2- 172.21.15.86:0/1610811796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe58414fc50 0x7fe58415be20 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fe56c009990 tx=0x7fe56c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:49.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.708+0000 7fe57aa00640 1 -- 172.21.15.86:0/1610811796 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c048070 con 0x7fe58414fc50 2024-04-20T21:22:49.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.708+0000 7fe57aa00640 1 -- 172.21.15.86:0/1610811796 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe56c030430 con 0x7fe58414fc50 2024-04-20T21:22:49.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.708+0000 7fe57aa00640 1 -- 172.21.15.86:0/1610811796 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c030800 con 0x7fe58414fc50 2024-04-20T21:22:49.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.709+0000 7fe589200640 1 -- 172.21.15.86:0/1610811796 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5841513c0 con 0x7fe58414fc50 2024-04-20T21:22:49.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.709+0000 7fe589200640 1 -- 172.21.15.86:0/1610811796 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5840adcd0 con 0x7fe58414fc50 2024-04-20T21:22:49.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.710+0000 7fe589200640 1 -- 172.21.15.86:0/1610811796 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5840689e0 con 0x7fe58414fc50 2024-04-20T21:22:49.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.711+0000 7fe57aa00640 1 -- 172.21.15.86:0/1610811796 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe56c04b070 con 0x7fe58414fc50 2024-04-20T21:22:49.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.712+0000 7fe57aa00640 1 --2- 172.21.15.86:0/1610811796 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe56006c8f0 0x7fe56006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:49.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.712+0000 7fe57aa00640 1 -- 172.21.15.86:0/1610811796 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7fe56c0ad3b0 con 0x7fe58414fc50 2024-04-20T21:22:49.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.712+0000 7fe581e00640 1 --2- 172.21.15.86:0/1610811796 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe56006c8f0 0x7fe56006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:49.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.713+0000 7fe581e00640 1 --2- 172.21.15.86:0/1610811796 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe56006c8f0 0x7fe56006edb0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe57400aa10 tx=0x7fe574008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:49.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.716+0000 7fe57aa00640 1 -- 172.21.15.86:0/1610811796 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe56c034020 con 0x7fe58414fc50 2024-04-20T21:22:49.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.911+0000 7fe589200640 1 -- 172.21.15.86:0/1610811796 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe5840b56a0 con 0x7fe58414fc50 2024-04-20T21:22:49.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.913+0000 7fe57aa00640 1 -- 172.21.15.86:0/1610811796 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+16668 (secure 0 0 0) 0x7fe56c07b870 con 0x7fe58414fc50 2024-04-20T21:22:49.913 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:22:49.913 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":71,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:22:46.983093+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:22:43.125767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-20T21:22:44.410549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:22:49.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.916+0000 7fe589200640 1 -- 172.21.15.86:0/1610811796 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe56006c8f0 msgr2=0x7fe56006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:49.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.916+0000 7fe589200640 1 --2- 172.21.15.86:0/1610811796 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe56006c8f0 0x7fe56006edb0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe57400aa10 tx=0x7fe574008040 comp rx=0 tx=0).stop 2024-04-20T21:22:49.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.916+0000 7fe589200640 1 -- 172.21.15.86:0/1610811796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe58414fc50 msgr2=0x7fe58415be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:49.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.916+0000 7fe589200640 1 --2- 172.21.15.86:0/1610811796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe58414fc50 0x7fe58415be20 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fe56c009990 tx=0x7fe56c004730 comp rx=0 tx=0).stop 2024-04-20T21:22:49.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.916+0000 7fe589200640 1 -- 172.21.15.86:0/1610811796 shutdown_connections 2024-04-20T21:22:49.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.916+0000 7fe589200640 1 --2- 172.21.15.86:0/1610811796 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe56006c8f0 0x7fe56006edb0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:49.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.916+0000 7fe589200640 1 --2- 172.21.15.86:0/1610811796 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe584150b40 0x7fe58415c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:49.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.916+0000 7fe589200640 1 --2- 172.21.15.86:0/1610811796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe58414fc50 0x7fe58415be20 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:49.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.916+0000 7fe589200640 1 --2- 172.21.15.86:0/1610811796 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe568003ef0 0x7fe5841f3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:49.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.916+0000 7fe589200640 1 -- 172.21.15.86:0/1610811796 >> 172.21.15.86:0/1610811796 conn(0x7fe5840b2360 msgr2=0x7fe5841549d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:49.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.916+0000 7fe589200640 1 -- 172.21.15.86:0/1610811796 shutdown_connections 2024-04-20T21:22:49.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:49.917+0000 7fe589200640 1 -- 172.21.15.86:0/1610811796 wait complete. 2024-04-20T21:22:49.934 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-20T21:22:49.934 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T21:22:50.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.129+0000 7f1455200640 1 Processor -- start 2024-04-20T21:22:50.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.129+0000 7f1455200640 1 -- start start 2024-04-20T21:22:50.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.130+0000 7f1455200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f145015d6a0 0x7f14500ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:50.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.130+0000 7f1455200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f14500ad620 0x7f145006e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:50.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.130+0000 7f1455200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1450160890 con 0x7f1450156ae0 2024-04-20T21:22:50.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.130+0000 7f1455200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f145006db20 con 0x7f14500ad620 2024-04-20T21:22:50.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.130+0000 7f1455200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f14500ada50 con 0x7f145015d6a0 2024-04-20T21:22:50.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.131+0000 7f144fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f14500ad620 0x7f145006e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:50.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.131+0000 7f144f400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f1450156ae0 0x7f1450160080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44732/0 (socket says 172.21.15.86:44732) 2024-04-20T21:22:50.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.131+0000 7f144f400640 1 -- 172.21.15.86:0/3574923685 learned_addr learned my addr 172.21.15.86:0/3574923685 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:22:50.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.131+0000 7f144ea00640 1 --2- 172.21.15.86:0/3574923685 >> v2:172.21.15.144:3300/0 conn(0x7f145015d6a0 0x7f14500ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:50.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.131+0000 7f144fe00640 1 -- 172.21.15.86:0/3574923685 >> v2:172.21.15.144:3300/0 conn(0x7f145015d6a0 msgr2=0x7f14500ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:50.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.131+0000 7f144fe00640 1 --2- 172.21.15.86:0/3574923685 >> v2:172.21.15.144:3300/0 conn(0x7f145015d6a0 0x7f14500ad0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:50.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.131+0000 7f144fe00640 1 -- 172.21.15.86:0/3574923685 >> v1:172.21.15.86:6789/0 conn(0x7f1450156ae0 legacy=0x7f1450160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:50.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.131+0000 7f144e000640 1 -- 172.21.15.86:0/3574923685 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2082325254 0 0) 0x7f1450160890 con 0x7f1450156ae0 2024-04-20T21:22:50.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.132+0000 7f144fe00640 1 -- 172.21.15.86:0/3574923685 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1450160b70 con 0x7f14500ad620 2024-04-20T21:22:50.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.133+0000 7f144fe00640 1 --2- 172.21.15.86:0/3574923685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14500ad620 0x7f145006e290 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f1450074020 tx=0x7f144802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b61dec64f4a0cc31 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:50.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.133+0000 7f144e000640 1 -- 172.21.15.86:0/3574923685 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1448039070 con 0x7f14500ad620 2024-04-20T21:22:50.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.133+0000 7f144e000640 1 -- 172.21.15.86:0/3574923685 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f144802cc80 con 0x7f14500ad620 2024-04-20T21:22:50.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.133+0000 7f144e000640 1 -- 172.21.15.86:0/3574923685 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1448034660 con 0x7f14500ad620 2024-04-20T21:22:50.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.134+0000 7f1455200640 1 -- 172.21.15.86:0/3574923685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14500ad620 msgr2=0x7f145006e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:50.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.134+0000 7f1455200640 1 --2- 172.21.15.86:0/3574923685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14500ad620 0x7f145006e290 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f1450074020 tx=0x7f144802bed0 comp rx=0 tx=0).stop 2024-04-20T21:22:50.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.134+0000 7f1455200640 1 -- 172.21.15.86:0/3574923685 shutdown_connections 2024-04-20T21:22:50.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.134+0000 7f1455200640 1 --2- 172.21.15.86:0/3574923685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14500ad620 0x7f145006e290 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:50.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.134+0000 7f1455200640 1 --2- 172.21.15.86:0/3574923685 >> v2:172.21.15.144:3300/0 conn(0x7f145015d6a0 0x7f14500ad0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:50.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.134+0000 7f1455200640 1 -- 172.21.15.86:0/3574923685 >> 172.21.15.86:0/3574923685 conn(0x7f14500d0170 msgr2=0x7f14500d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:50.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.134+0000 7f1455200640 1 -- 172.21.15.86:0/3574923685 shutdown_connections 2024-04-20T21:22:50.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.134+0000 7f1455200640 1 -- 172.21.15.86:0/3574923685 wait complete. 2024-04-20T21:22:50.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.136+0000 7f1455200640 1 Processor -- start 2024-04-20T21:22:50.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.136+0000 7f1455200640 1 -- start start 2024-04-20T21:22:50.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.137+0000 7f1455200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14500ad620 0x7f14500afd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:50.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.137+0000 7f1455200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1450156ae0 0x7f14500b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:50.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.137+0000 7f1455200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f145015d6a0 0x7f1450143750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:50.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.137+0000 7f1455200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1450156ec0 con 0x7f1450156ae0 2024-04-20T21:22:50.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.137+0000 7f1455200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f145006dd70 con 0x7f14500ad620 2024-04-20T21:22:50.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.137+0000 7f1455200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f145006e7d0 con 0x7f145015d6a0 2024-04-20T21:22:50.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.137+0000 7f144ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1450156ae0 0x7f14500b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:50.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.137+0000 7f144ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1450156ae0 0x7f14500b02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52072/0 (socket says 172.21.15.86:52072) 2024-04-20T21:22:50.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.137+0000 7f144ea00640 1 -- 172.21.15.86:0/3254343822 learned_addr learned my addr 172.21.15.86:0/3254343822 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:50.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.137+0000 7f144ea00640 1 -- 172.21.15.86:0/3254343822 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f145015d6a0 msgr2=0x7f1450143750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:50.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.137+0000 7f144ea00640 1 --2- 172.21.15.86:0/3254343822 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f145015d6a0 0x7f1450143750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:50.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.137+0000 7f144ea00640 1 -- 172.21.15.86:0/3254343822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14500ad620 msgr2=0x7f14500afd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:50.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.137+0000 7f144ea00640 1 --2- 172.21.15.86:0/3254343822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14500ad620 0x7f14500afd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:50.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.137+0000 7f144ea00640 1 -- 172.21.15.86:0/3254343822 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14501f42e0 con 0x7f1450156ae0 2024-04-20T21:22:50.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.138+0000 7f144ea00640 1 --2- 172.21.15.86:0/3254343822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1450156ae0 0x7f14500b02b0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f143c00d620 tx=0x7f143c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:50.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.138+0000 7f1447400640 1 -- 172.21.15.86:0/3254343822 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143c00b620 con 0x7f1450156ae0 2024-04-20T21:22:50.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.138+0000 7f1447400640 1 -- 172.21.15.86:0/3254343822 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f143c019070 con 0x7f1450156ae0 2024-04-20T21:22:50.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.138+0000 7f1455200640 1 -- 172.21.15.86:0/3254343822 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1450144cb0 con 0x7f1450156ae0 2024-04-20T21:22:50.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.138+0000 7f1447400640 1 -- 172.21.15.86:0/3254343822 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143c00ba00 con 0x7f1450156ae0 2024-04-20T21:22:50.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.138+0000 7f1455200640 1 -- 172.21.15.86:0/3254343822 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1450071ef0 con 0x7f1450156ae0 2024-04-20T21:22:50.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.139+0000 7f1455200640 1 -- 172.21.15.86:0/3254343822 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f145004fcf0 con 0x7f1450156ae0 2024-04-20T21:22:50.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.140+0000 7f1447400640 1 -- 172.21.15.86:0/3254343822 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f143c00bba0 con 0x7f1450156ae0 2024-04-20T21:22:50.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.141+0000 7f1447400640 1 --2- 172.21.15.86:0/3254343822 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f143006c8f0 0x7f143006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:50.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.141+0000 7f1447400640 1 -- 172.21.15.86:0/3254343822 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f143c08bfa0 con 0x7f1450156ae0 2024-04-20T21:22:50.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.141+0000 7f144f400640 1 --2- 172.21.15.86:0/3254343822 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f143006c8f0 0x7f143006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:50.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.142+0000 7f144f400640 1 --2- 172.21.15.86:0/3254343822 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f143006c8f0 0x7f143006edb0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1440007fd0 tx=0x7f1440007920 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:50.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.147+0000 7f1447400640 1 -- 172.21.15.86:0/3254343822 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f143c05a370 con 0x7f1450156ae0 2024-04-20T21:22:50.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.342+0000 7f1455200640 1 -- 172.21.15.86:0/3254343822 --> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f14500b1020 con 0x7f143006c8f0 2024-04-20T21:22:50.344 INFO:tasks.ceph.mds.a.smithi086.stderr:2024-04-20T21:22:50.345+0000 7fae43a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T21:22:50.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.354+0000 7f1447400640 1 -- 172.21.15.86:0/3254343822 <== mgr.4107 v2:172.21.15.86:6832/302045240 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246752 (secure 0 0 0) 0x7f14500b1020 con 0x7f143006c8f0 2024-04-20T21:22:50.357 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:22:50.358 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-20T21:22:50.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.360+0000 7f1455200640 1 -- 172.21.15.86:0/3254343822 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f143006c8f0 msgr2=0x7f143006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:50.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.361+0000 7f1455200640 1 --2- 172.21.15.86:0/3254343822 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f143006c8f0 0x7f143006edb0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1440007fd0 tx=0x7f1440007920 comp rx=0 tx=0).stop 2024-04-20T21:22:50.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.361+0000 7f1455200640 1 -- 172.21.15.86:0/3254343822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1450156ae0 msgr2=0x7f14500b02b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:50.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.361+0000 7f1455200640 1 --2- 172.21.15.86:0/3254343822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1450156ae0 0x7f14500b02b0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f143c00d620 tx=0x7f143c00dae0 comp rx=0 tx=0).stop 2024-04-20T21:22:50.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.361+0000 7f1455200640 1 -- 172.21.15.86:0/3254343822 shutdown_connections 2024-04-20T21:22:50.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.361+0000 7f1455200640 1 --2- 172.21.15.86:0/3254343822 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f143006c8f0 0x7f143006edb0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:50.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.361+0000 7f1455200640 1 --2- 172.21.15.86:0/3254343822 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f145015d6a0 0x7f1450143750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:50.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.361+0000 7f1455200640 1 --2- 172.21.15.86:0/3254343822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1450156ae0 0x7f14500b02b0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:50.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.361+0000 7f1455200640 1 --2- 172.21.15.86:0/3254343822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f14500ad620 0x7f14500afd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:50.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.361+0000 7f1455200640 1 -- 172.21.15.86:0/3254343822 >> 172.21.15.86:0/3254343822 conn(0x7f14500d0170 msgr2=0x7f14500b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:50.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.361+0000 7f1455200640 1 -- 172.21.15.86:0/3254343822 shutdown_connections 2024-04-20T21:22:50.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.361+0000 7f1455200640 1 -- 172.21.15.86:0/3254343822 wait complete. 2024-04-20T21:22:50.377 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":218,"stamp":"2024-04-20T21:22:48.731851+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257176,"kb_used_data":7720,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754717544,"statfs":{"total":773094113280,"available":772830765056,"internally_reserved":0,"allocated":7905280,"data_stored":3990496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.783886"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.989723+0000","last_change":"2024-04-20T21:22:44.989723+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.989723+0000","last_undegraded":"2024-04-20T21:22:44.989723+0000","last_fullsized":"2024-04-20T21:22:44.989723+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.972721+0000","last_change":"2024-04-20T21:22:44.972721+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.972721+0000","last_undegraded":"2024-04-20T21:22:44.972721+0000","last_fullsized":"2024-04-20T21:22:44.972721+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.975196+0000","last_change":"2024-04-20T21:22:44.975196+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.975196+0000","last_undegraded":"2024-04-20T21:22:44.975196+0000","last_fullsized":"2024-04-20T21:22:44.975196+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.978626+0000","last_change":"2024-04-20T21:22:44.978626+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.978626+0000","last_undegraded":"2024-04-20T21:22:44.978626+0000","last_fullsized":"2024-04-20T21:22:44.978626+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.971269+0000","last_change":"2024-04-20T21:22:44.971269+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.971269+0000","last_undegraded":"2024-04-20T21:22:44.971269+0000","last_fullsized":"2024-04-20T21:22:44.971269+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.976258+0000","last_change":"2024-04-20T21:22:44.976258+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.976258+0000","last_undegraded":"2024-04-20T21:22:44.976258+0000","last_fullsized":"2024-04-20T21:22:44.976258+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.986473+0000","last_change":"2024-04-20T21:22:44.986473+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.986473+0000","last_undegraded":"2024-04-20T21:22:44.986473+0000","last_fullsized":"2024-04-20T21:22:44.986473+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.971878+0000","last_change":"2024-04-20T21:22:44.971878+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.971878+0000","last_undegraded":"2024-04-20T21:22:44.971878+0000","last_fullsized":"2024-04-20T21:22:44.971878+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.971801+0000","last_change":"2024-04-20T21:22:44.971801+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.971801+0000","last_undegraded":"2024-04-20T21:22:44.971801+0000","last_fullsized":"2024-04-20T21:22:44.971801+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.976430+0000","last_change":"2024-04-20T21:22:44.976430+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.976430+0000","last_undegraded":"2024-04-20T21:22:44.976430+0000","last_fullsized":"2024-04-20T21:22:44.976430+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.989195+0000","last_change":"2024-04-20T21:22:44.989195+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.989195+0000","last_undegraded":"2024-04-20T21:22:44.989195+0000","last_fullsized":"2024-04-20T21:22:44.989195+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.978036+0000","last_change":"2024-04-20T21:22:44.978036+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.978036+0000","last_undegraded":"2024-04-20T21:22:44.978036+0000","last_fullsized":"2024-04-20T21:22:44.978036+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.983752+0000","last_change":"2024-04-20T21:22:44.983752+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.983752+0000","last_undegraded":"2024-04-20T21:22:44.983752+0000","last_fullsized":"2024-04-20T21:22:44.983752+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.971036+0000","last_change":"2024-04-20T21:22:44.971036+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.971036+0000","last_undegraded":"2024-04-20T21:22:44.971036+0000","last_fullsized":"2024-04-20T21:22:44.971036+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.984365+0000","last_change":"2024-04-20T21:22:44.984365+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.984365+0000","last_undegraded":"2024-04-20T21:22:44.984365+0000","last_fullsized":"2024-04-20T21:22:44.984365+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.977907+0000","last_change":"2024-04-20T21:22:44.977907+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.977907+0000","last_undegraded":"2024-04-20T21:22:44.977907+0000","last_fullsized":"2024-04-20T21:22:44.977907+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.971170+0000","last_change":"2024-04-20T21:22:44.971170+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.971170+0000","last_undegraded":"2024-04-20T21:22:44.971170+0000","last_fullsized":"2024-04-20T21:22:44.971170+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.980798+0000","last_change":"2024-04-20T21:22:44.980798+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.980798+0000","last_undegraded":"2024-04-20T21:22:44.980798+0000","last_fullsized":"2024-04-20T21:22:44.980798+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.978703+0000","last_change":"2024-04-20T21:22:44.978703+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.978703+0000","last_undegraded":"2024-04-20T21:22:44.978703+0000","last_fullsized":"2024-04-20T21:22:44.978703+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.986040+0000","last_change":"2024-04-20T21:22:44.986040+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.986040+0000","last_undegraded":"2024-04-20T21:22:44.986040+0000","last_fullsized":"2024-04-20T21:22:44.986040+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.978512+0000","last_change":"2024-04-20T21:22:44.978512+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.978512+0000","last_undegraded":"2024-04-20T21:22:44.978512+0000","last_fullsized":"2024-04-20T21:22:44.978512+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.972626+0000","last_change":"2024-04-20T21:22:44.972626+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.972626+0000","last_undegraded":"2024-04-20T21:22:44.972626+0000","last_fullsized":"2024-04-20T21:22:44.972626+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.985232+0000","last_change":"2024-04-20T21:22:44.985232+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.985232+0000","last_undegraded":"2024-04-20T21:22:44.985232+0000","last_fullsized":"2024-04-20T21:22:44.985232+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.983826+0000","last_change":"2024-04-20T21:22:44.983826+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.983826+0000","last_undegraded":"2024-04-20T21:22:44.983826+0000","last_fullsized":"2024-04-20T21:22:44.983826+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.977590+0000","last_change":"2024-04-20T21:22:44.977590+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.977590+0000","last_undegraded":"2024-04-20T21:22:44.977590+0000","last_fullsized":"2024-04-20T21:22:44.977590+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.984137+0000","last_change":"2024-04-20T21:22:44.984137+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.984137+0000","last_undegraded":"2024-04-20T21:22:44.984137+0000","last_fullsized":"2024-04-20T21:22:44.984137+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.973277+0000","last_change":"2024-04-20T21:22:44.973277+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.973277+0000","last_undegraded":"2024-04-20T21:22:44.973277+0000","last_fullsized":"2024-04-20T21:22:44.973277+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.974424+0000","last_change":"2024-04-20T21:22:44.974424+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.974424+0000","last_undegraded":"2024-04-20T21:22:44.974424+0000","last_fullsized":"2024-04-20T21:22:44.974424+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.975174+0000","last_change":"2024-04-20T21:22:44.975174+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.975174+0000","last_undegraded":"2024-04-20T21:22:44.975174+0000","last_fullsized":"2024-04-20T21:22:44.975174+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.983576+0000","last_change":"2024-04-20T21:22:44.983576+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.983576+0000","last_undegraded":"2024-04-20T21:22:44.983576+0000","last_fullsized":"2024-04-20T21:22:44.983576+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.983334+0000","last_change":"2024-04-20T21:22:44.983334+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.983334+0000","last_undegraded":"2024-04-20T21:22:44.983334+0000","last_fullsized":"2024-04-20T21:22:44.983334+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.970617+0000","last_change":"2024-04-20T21:22:44.970617+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.970617+0000","last_undegraded":"2024-04-20T21:22:44.970617+0000","last_fullsized":"2024-04-20T21:22:44.970617+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.982545+0000","last_change":"2024-04-20T21:22:44.982545+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.982545+0000","last_undegraded":"2024-04-20T21:22:44.982545+0000","last_fullsized":"2024-04-20T21:22:44.982545+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:45.004990+0000","last_change":"2024-04-20T21:22:45.000877+0000","last_active":"2024-04-20T21:22:45.004990+0000","last_peered":"2024-04-20T21:22:45.004990+0000","last_clean":"2024-04-20T21:22:45.004990+0000","last_became_active":"2024-04-20T21:22:45.000402+0000","last_became_peered":"2024-04-20T21:22:45.000402+0000","last_unstale":"2024-04-20T21:22:45.004990+0000","last_undegraded":"2024-04-20T21:22:45.004990+0000","last_fullsized":"2024-04-20T21:22:45.004990+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:56:13.768402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.970495+0000","last_change":"2024-04-20T21:22:44.970495+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.970495+0000","last_undegraded":"2024-04-20T21:22:44.970495+0000","last_fullsized":"2024-04-20T21:22:44.970495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.999915+0000","last_change":"2024-04-20T21:22:44.998432+0000","last_active":"2024-04-20T21:22:44.999915+0000","last_peered":"2024-04-20T21:22:44.999915+0000","last_clean":"2024-04-20T21:22:44.999915+0000","last_became_active":"2024-04-20T21:22:44.998006+0000","last_became_peered":"2024-04-20T21:22:44.998006+0000","last_unstale":"2024-04-20T21:22:44.999915+0000","last_undegraded":"2024-04-20T21:22:44.999915+0000","last_fullsized":"2024-04-20T21:22:44.999915+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:53:04.669954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.979534+0000","last_change":"2024-04-20T21:22:44.979534+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.979534+0000","last_undegraded":"2024-04-20T21:22:44.979534+0000","last_fullsized":"2024-04-20T21:22:44.979534+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:45.002180+0000","last_change":"2024-04-20T21:22:44.997985+0000","last_active":"2024-04-20T21:22:45.002180+0000","last_peered":"2024-04-20T21:22:45.002180+0000","last_clean":"2024-04-20T21:22:45.002180+0000","last_became_active":"2024-04-20T21:22:44.997672+0000","last_became_peered":"2024-04-20T21:22:44.997672+0000","last_unstale":"2024-04-20T21:22:45.002180+0000","last_undegraded":"2024-04-20T21:22:45.002180+0000","last_fullsized":"2024-04-20T21:22:45.002180+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:16:44.463868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.988718+0000","last_change":"2024-04-20T21:22:44.988718+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.988718+0000","last_undegraded":"2024-04-20T21:22:44.988718+0000","last_fullsized":"2024-04-20T21:22:44.988718+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.996650+0000","last_change":"2024-04-20T21:22:44.989343+0000","last_active":"2024-04-20T21:22:44.996650+0000","last_peered":"2024-04-20T21:22:44.996650+0000","last_clean":"2024-04-20T21:22:44.996650+0000","last_became_active":"2024-04-20T21:22:44.989022+0000","last_became_peered":"2024-04-20T21:22:44.989022+0000","last_unstale":"2024-04-20T21:22:44.996650+0000","last_undegraded":"2024-04-20T21:22:44.996650+0000","last_fullsized":"2024-04-20T21:22:44.996650+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:45:48.879276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.981773+0000","last_change":"2024-04-20T21:22:44.981773+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.981773+0000","last_undegraded":"2024-04-20T21:22:44.981773+0000","last_fullsized":"2024-04-20T21:22:44.981773+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.998018+0000","last_change":"2024-04-20T21:22:44.994866+0000","last_active":"2024-04-20T21:22:44.998018+0000","last_peered":"2024-04-20T21:22:44.998018+0000","last_clean":"2024-04-20T21:22:44.998018+0000","last_became_active":"2024-04-20T21:22:44.994089+0000","last_became_peered":"2024-04-20T21:22:44.994089+0000","last_unstale":"2024-04-20T21:22:44.998018+0000","last_undegraded":"2024-04-20T21:22:44.998018+0000","last_fullsized":"2024-04-20T21:22:44.998018+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:53:16.833450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.975250+0000","last_change":"2024-04-20T21:22:44.975250+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.975250+0000","last_undegraded":"2024-04-20T21:22:44.975250+0000","last_fullsized":"2024-04-20T21:22:44.975250+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.999221+0000","last_change":"2024-04-20T21:22:44.995756+0000","last_active":"2024-04-20T21:22:44.999221+0000","last_peered":"2024-04-20T21:22:44.999221+0000","last_clean":"2024-04-20T21:22:44.999221+0000","last_became_active":"2024-04-20T21:22:44.995451+0000","last_became_peered":"2024-04-20T21:22:44.995451+0000","last_unstale":"2024-04-20T21:22:44.999221+0000","last_undegraded":"2024-04-20T21:22:44.999221+0000","last_fullsized":"2024-04-20T21:22:44.999221+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:37:36.078254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.970247+0000","last_change":"2024-04-20T21:22:44.970247+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.970247+0000","last_undegraded":"2024-04-20T21:22:44.970247+0000","last_fullsized":"2024-04-20T21:22:44.970247+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.997961+0000","last_change":"2024-04-20T21:22:44.996824+0000","last_active":"2024-04-20T21:22:44.997961+0000","last_peered":"2024-04-20T21:22:44.997961+0000","last_clean":"2024-04-20T21:22:44.997961+0000","last_became_active":"2024-04-20T21:22:44.996531+0000","last_became_peered":"2024-04-20T21:22:44.996531+0000","last_unstale":"2024-04-20T21:22:44.997961+0000","last_undegraded":"2024-04-20T21:22:44.997961+0000","last_fullsized":"2024-04-20T21:22:44.997961+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:45:52.489483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.983784+0000","last_change":"2024-04-20T21:22:44.983784+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.983784+0000","last_undegraded":"2024-04-20T21:22:44.983784+0000","last_fullsized":"2024-04-20T21:22:44.983784+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.986417+0000","last_change":"2024-04-20T21:22:44.982668+0000","last_active":"2024-04-20T21:22:44.986417+0000","last_peered":"2024-04-20T21:22:44.986417+0000","last_clean":"2024-04-20T21:22:44.986417+0000","last_became_active":"2024-04-20T21:22:44.982035+0000","last_became_peered":"2024-04-20T21:22:44.982035+0000","last_unstale":"2024-04-20T21:22:44.986417+0000","last_undegraded":"2024-04-20T21:22:44.986417+0000","last_fullsized":"2024-04-20T21:22:44.986417+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:33:23.705401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.983607+0000","last_change":"2024-04-20T21:22:44.983607+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.983607+0000","last_undegraded":"2024-04-20T21:22:44.983607+0000","last_fullsized":"2024-04-20T21:22:44.983607+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.999461+0000","last_change":"2024-04-20T21:22:44.994181+0000","last_active":"2024-04-20T21:22:44.999461+0000","last_peered":"2024-04-20T21:22:44.999461+0000","last_clean":"2024-04-20T21:22:44.999461+0000","last_became_active":"2024-04-20T21:22:44.992861+0000","last_became_peered":"2024-04-20T21:22:44.992861+0000","last_unstale":"2024-04-20T21:22:44.999461+0000","last_undegraded":"2024-04-20T21:22:44.999461+0000","last_fullsized":"2024-04-20T21:22:44.999461+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:09:40.557733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.988271+0000","last_change":"2024-04-20T21:22:44.988271+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.988271+0000","last_undegraded":"2024-04-20T21:22:44.988271+0000","last_fullsized":"2024-04-20T21:22:44.988271+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.999321+0000","last_change":"2024-04-20T21:22:44.992156+0000","last_active":"2024-04-20T21:22:44.999321+0000","last_peered":"2024-04-20T21:22:44.999321+0000","last_clean":"2024-04-20T21:22:44.999321+0000","last_became_active":"2024-04-20T21:22:44.991749+0000","last_became_peered":"2024-04-20T21:22:44.991749+0000","last_unstale":"2024-04-20T21:22:44.999321+0000","last_undegraded":"2024-04-20T21:22:44.999321+0000","last_fullsized":"2024-04-20T21:22:44.999321+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:59:10.369089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.977417+0000","last_change":"2024-04-20T21:22:44.977417+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.977417+0000","last_undegraded":"2024-04-20T21:22:44.977417+0000","last_fullsized":"2024-04-20T21:22:44.977417+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.992559+0000","last_change":"2024-04-20T21:22:44.990563+0000","last_active":"2024-04-20T21:22:44.992559+0000","last_peered":"2024-04-20T21:22:44.992559+0000","last_clean":"2024-04-20T21:22:44.992559+0000","last_became_active":"2024-04-20T21:22:44.990292+0000","last_became_peered":"2024-04-20T21:22:44.990292+0000","last_unstale":"2024-04-20T21:22:44.992559+0000","last_undegraded":"2024-04-20T21:22:44.992559+0000","last_fullsized":"2024-04-20T21:22:44.992559+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:58:26.186590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.971007+0000","last_change":"2024-04-20T21:22:44.971007+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.971007+0000","last_undegraded":"2024-04-20T21:22:44.971007+0000","last_fullsized":"2024-04-20T21:22:44.971007+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.993386+0000","last_change":"2024-04-20T21:22:44.988227+0000","last_active":"2024-04-20T21:22:44.993386+0000","last_peered":"2024-04-20T21:22:44.993386+0000","last_clean":"2024-04-20T21:22:44.993386+0000","last_became_active":"2024-04-20T21:22:44.987939+0000","last_became_peered":"2024-04-20T21:22:44.987939+0000","last_unstale":"2024-04-20T21:22:44.993386+0000","last_undegraded":"2024-04-20T21:22:44.993386+0000","last_fullsized":"2024-04-20T21:22:44.993386+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:19:34.832419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.985271+0000","last_change":"2024-04-20T21:22:44.985271+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.985271+0000","last_undegraded":"2024-04-20T21:22:44.985271+0000","last_fullsized":"2024-04-20T21:22:44.985271+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.997222+0000","last_change":"2024-04-20T21:22:44.992317+0000","last_active":"2024-04-20T21:22:44.997222+0000","last_peered":"2024-04-20T21:22:44.997222+0000","last_clean":"2024-04-20T21:22:44.997222+0000","last_became_active":"2024-04-20T21:22:44.992040+0000","last_became_peered":"2024-04-20T21:22:44.992040+0000","last_unstale":"2024-04-20T21:22:44.997222+0000","last_undegraded":"2024-04-20T21:22:44.997222+0000","last_fullsized":"2024-04-20T21:22:44.997222+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:33:17.580290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.983001+0000","last_change":"2024-04-20T21:22:44.983001+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.983001+0000","last_undegraded":"2024-04-20T21:22:44.983001+0000","last_fullsized":"2024-04-20T21:22:44.983001+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.992803+0000","last_change":"2024-04-20T21:22:44.987324+0000","last_active":"2024-04-20T21:22:44.992803+0000","last_peered":"2024-04-20T21:22:44.992803+0000","last_clean":"2024-04-20T21:22:44.992803+0000","last_became_active":"2024-04-20T21:22:44.986974+0000","last_became_peered":"2024-04-20T21:22:44.986974+0000","last_unstale":"2024-04-20T21:22:44.992803+0000","last_undegraded":"2024-04-20T21:22:44.992803+0000","last_fullsized":"2024-04-20T21:22:44.992803+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:40:38.463867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.983547+0000","last_change":"2024-04-20T21:22:44.983547+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.983547+0000","last_undegraded":"2024-04-20T21:22:44.983547+0000","last_fullsized":"2024-04-20T21:22:44.983547+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:45.000030+0000","last_change":"2024-04-20T21:22:44.997932+0000","last_active":"2024-04-20T21:22:45.000030+0000","last_peered":"2024-04-20T21:22:45.000030+0000","last_clean":"2024-04-20T21:22:45.000030+0000","last_became_active":"2024-04-20T21:22:44.997397+0000","last_became_peered":"2024-04-20T21:22:44.997397+0000","last_unstale":"2024-04-20T21:22:45.000030+0000","last_undegraded":"2024-04-20T21:22:45.000030+0000","last_fullsized":"2024-04-20T21:22:45.000030+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:21:37.019143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.970173+0000","last_change":"2024-04-20T21:22:44.970173+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.970173+0000","last_undegraded":"2024-04-20T21:22:44.970173+0000","last_fullsized":"2024-04-20T21:22:44.970173+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.999123+0000","last_change":"2024-04-20T21:22:44.995360+0000","last_active":"2024-04-20T21:22:44.999123+0000","last_peered":"2024-04-20T21:22:44.999123+0000","last_clean":"2024-04-20T21:22:44.999123+0000","last_became_active":"2024-04-20T21:22:44.995055+0000","last_became_peered":"2024-04-20T21:22:44.995055+0000","last_unstale":"2024-04-20T21:22:44.999123+0000","last_undegraded":"2024-04-20T21:22:44.999123+0000","last_fullsized":"2024-04-20T21:22:44.999123+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:56:28.238720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.975298+0000","last_change":"2024-04-20T21:22:44.975298+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.975298+0000","last_undegraded":"2024-04-20T21:22:44.975298+0000","last_fullsized":"2024-04-20T21:22:44.975298+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.999385+0000","last_change":"2024-04-20T21:22:44.993745+0000","last_active":"2024-04-20T21:22:44.999385+0000","last_peered":"2024-04-20T21:22:44.999385+0000","last_clean":"2024-04-20T21:22:44.999385+0000","last_became_active":"2024-04-20T21:22:44.992981+0000","last_became_peered":"2024-04-20T21:22:44.992981+0000","last_unstale":"2024-04-20T21:22:44.999385+0000","last_undegraded":"2024-04-20T21:22:44.999385+0000","last_fullsized":"2024-04-20T21:22:44.999385+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:08:56.570003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.987280+0000","last_change":"2024-04-20T21:22:44.987280+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.987280+0000","last_undegraded":"2024-04-20T21:22:44.987280+0000","last_fullsized":"2024-04-20T21:22:44.987280+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:45.005043+0000","last_change":"2024-04-20T21:22:44.998796+0000","last_active":"2024-04-20T21:22:45.005043+0000","last_peered":"2024-04-20T21:22:45.005043+0000","last_clean":"2024-04-20T21:22:45.005043+0000","last_became_active":"2024-04-20T21:22:44.998424+0000","last_became_peered":"2024-04-20T21:22:44.998424+0000","last_unstale":"2024-04-20T21:22:45.005043+0000","last_undegraded":"2024-04-20T21:22:45.005043+0000","last_fullsized":"2024-04-20T21:22:45.005043+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:22:05.632471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.977478+0000","last_change":"2024-04-20T21:22:44.977478+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.977478+0000","last_undegraded":"2024-04-20T21:22:44.977478+0000","last_fullsized":"2024-04-20T21:22:44.977478+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:45.003213+0000","last_change":"2024-04-20T21:22:45.000092+0000","last_active":"2024-04-20T21:22:45.003213+0000","last_peered":"2024-04-20T21:22:45.003213+0000","last_clean":"2024-04-20T21:22:45.003213+0000","last_became_active":"2024-04-20T21:22:44.998998+0000","last_became_peered":"2024-04-20T21:22:44.998998+0000","last_unstale":"2024-04-20T21:22:45.003213+0000","last_undegraded":"2024-04-20T21:22:45.003213+0000","last_fullsized":"2024-04-20T21:22:45.003213+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:19:34.832419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.970404+0000","last_change":"2024-04-20T21:22:44.970404+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.970404+0000","last_undegraded":"2024-04-20T21:22:44.970404+0000","last_fullsized":"2024-04-20T21:22:44.970404+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:45.007767+0000","last_change":"2024-04-20T21:22:45.002908+0000","last_active":"2024-04-20T21:22:45.007767+0000","last_peered":"2024-04-20T21:22:45.007767+0000","last_clean":"2024-04-20T21:22:45.007767+0000","last_became_active":"2024-04-20T21:22:45.002596+0000","last_became_peered":"2024-04-20T21:22:45.002596+0000","last_unstale":"2024-04-20T21:22:45.007767+0000","last_undegraded":"2024-04-20T21:22:45.007767+0000","last_fullsized":"2024-04-20T21:22:45.007767+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:07:15.232666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.973416+0000","last_change":"2024-04-20T21:22:44.973416+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.973416+0000","last_undegraded":"2024-04-20T21:22:44.973416+0000","last_fullsized":"2024-04-20T21:22:44.973416+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:45.000938+0000","last_change":"2024-04-20T21:22:44.996534+0000","last_active":"2024-04-20T21:22:45.000938+0000","last_peered":"2024-04-20T21:22:45.000938+0000","last_clean":"2024-04-20T21:22:45.000938+0000","last_became_active":"2024-04-20T21:22:44.996190+0000","last_became_peered":"2024-04-20T21:22:44.996190+0000","last_unstale":"2024-04-20T21:22:45.000938+0000","last_undegraded":"2024-04-20T21:22:45.000938+0000","last_fullsized":"2024-04-20T21:22:45.000938+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:48:58.388149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.972592+0000","last_change":"2024-04-20T21:22:44.972592+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.972592+0000","last_undegraded":"2024-04-20T21:22:44.972592+0000","last_fullsized":"2024-04-20T21:22:44.972592+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.998579+0000","last_change":"2024-04-20T21:22:44.993852+0000","last_active":"2024-04-20T21:22:44.998579+0000","last_peered":"2024-04-20T21:22:44.998579+0000","last_clean":"2024-04-20T21:22:44.998579+0000","last_became_active":"2024-04-20T21:22:44.993491+0000","last_became_peered":"2024-04-20T21:22:44.993491+0000","last_unstale":"2024-04-20T21:22:44.998579+0000","last_undegraded":"2024-04-20T21:22:44.998579+0000","last_fullsized":"2024-04-20T21:22:44.998579+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:10:01.129225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.973391+0000","last_change":"2024-04-20T21:22:44.973391+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.973391+0000","last_undegraded":"2024-04-20T21:22:44.973391+0000","last_fullsized":"2024-04-20T21:22:44.973391+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.994517+0000","last_change":"2024-04-20T21:22:44.992558+0000","last_active":"2024-04-20T21:22:44.994517+0000","last_peered":"2024-04-20T21:22:44.994517+0000","last_clean":"2024-04-20T21:22:44.994517+0000","last_became_active":"2024-04-20T21:22:44.991402+0000","last_became_peered":"2024-04-20T21:22:44.991402+0000","last_unstale":"2024-04-20T21:22:44.994517+0000","last_undegraded":"2024-04-20T21:22:44.994517+0000","last_fullsized":"2024-04-20T21:22:44.994517+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:47:51.476465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.985660+0000","last_change":"2024-04-20T21:22:44.985660+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.985660+0000","last_undegraded":"2024-04-20T21:22:44.985660+0000","last_fullsized":"2024-04-20T21:22:44.985660+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.996911+0000","last_change":"2024-04-20T21:22:44.990230+0000","last_active":"2024-04-20T21:22:44.996911+0000","last_peered":"2024-04-20T21:22:44.996911+0000","last_clean":"2024-04-20T21:22:44.996911+0000","last_became_active":"2024-04-20T21:22:44.989958+0000","last_became_peered":"2024-04-20T21:22:44.989958+0000","last_unstale":"2024-04-20T21:22:44.996911+0000","last_undegraded":"2024-04-20T21:22:44.996911+0000","last_fullsized":"2024-04-20T21:22:44.996911+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:18:22.288371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.985832+0000","last_change":"2024-04-20T21:22:44.985832+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.985832+0000","last_undegraded":"2024-04-20T21:22:44.985832+0000","last_fullsized":"2024-04-20T21:22:44.985832+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.999992+0000","last_change":"2024-04-20T21:22:44.997554+0000","last_active":"2024-04-20T21:22:44.999992+0000","last_peered":"2024-04-20T21:22:44.999992+0000","last_clean":"2024-04-20T21:22:44.999992+0000","last_became_active":"2024-04-20T21:22:44.997234+0000","last_became_peered":"2024-04-20T21:22:44.997234+0000","last_unstale":"2024-04-20T21:22:44.999992+0000","last_undegraded":"2024-04-20T21:22:44.999992+0000","last_fullsized":"2024-04-20T21:22:44.999992+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:52:23.085259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.986315+0000","last_change":"2024-04-20T21:22:44.986315+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.986315+0000","last_undegraded":"2024-04-20T21:22:44.986315+0000","last_fullsized":"2024-04-20T21:22:44.986315+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.999554+0000","last_change":"2024-04-20T21:22:44.998361+0000","last_active":"2024-04-20T21:22:44.999554+0000","last_peered":"2024-04-20T21:22:44.999554+0000","last_clean":"2024-04-20T21:22:44.999554+0000","last_became_active":"2024-04-20T21:22:44.996313+0000","last_became_peered":"2024-04-20T21:22:44.996313+0000","last_unstale":"2024-04-20T21:22:44.999554+0000","last_undegraded":"2024-04-20T21:22:44.999554+0000","last_fullsized":"2024-04-20T21:22:44.999554+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:55:59.809824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.986997+0000","last_change":"2024-04-20T21:22:44.986997+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.986997+0000","last_undegraded":"2024-04-20T21:22:44.986997+0000","last_fullsized":"2024-04-20T21:22:44.986997+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:45.004224+0000","last_change":"2024-04-20T21:22:45.003069+0000","last_active":"2024-04-20T21:22:45.004224+0000","last_peered":"2024-04-20T21:22:45.004224+0000","last_clean":"2024-04-20T21:22:45.004224+0000","last_became_active":"2024-04-20T21:22:45.002662+0000","last_became_peered":"2024-04-20T21:22:45.002662+0000","last_unstale":"2024-04-20T21:22:45.004224+0000","last_undegraded":"2024-04-20T21:22:45.004224+0000","last_fullsized":"2024-04-20T21:22:45.004224+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:17:48.882110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.970101+0000","last_change":"2024-04-20T21:22:44.970101+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.970101+0000","last_undegraded":"2024-04-20T21:22:44.970101+0000","last_fullsized":"2024-04-20T21:22:44.970101+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.993408+0000","last_change":"2024-04-20T21:22:44.990525+0000","last_active":"2024-04-20T21:22:44.993408+0000","last_peered":"2024-04-20T21:22:44.993408+0000","last_clean":"2024-04-20T21:22:44.993408+0000","last_became_active":"2024-04-20T21:22:44.990259+0000","last_became_peered":"2024-04-20T21:22:44.990259+0000","last_unstale":"2024-04-20T21:22:44.993408+0000","last_undegraded":"2024-04-20T21:22:44.993408+0000","last_fullsized":"2024-04-20T21:22:44.993408+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:23:28.589618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.971783+0000","last_change":"2024-04-20T21:22:44.971783+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.971783+0000","last_undegraded":"2024-04-20T21:22:44.971783+0000","last_fullsized":"2024-04-20T21:22:44.971783+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.998441+0000","last_change":"2024-04-20T21:22:44.996969+0000","last_active":"2024-04-20T21:22:44.998441+0000","last_peered":"2024-04-20T21:22:44.998441+0000","last_clean":"2024-04-20T21:22:44.998441+0000","last_became_active":"2024-04-20T21:22:44.995514+0000","last_became_peered":"2024-04-20T21:22:44.995514+0000","last_unstale":"2024-04-20T21:22:44.998441+0000","last_undegraded":"2024-04-20T21:22:44.998441+0000","last_fullsized":"2024-04-20T21:22:44.998441+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:07:07.942984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.977076+0000","last_change":"2024-04-20T21:22:44.977076+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.977076+0000","last_undegraded":"2024-04-20T21:22:44.977076+0000","last_fullsized":"2024-04-20T21:22:44.977076+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.998862+0000","last_change":"2024-04-20T21:22:44.994701+0000","last_active":"2024-04-20T21:22:44.998862+0000","last_peered":"2024-04-20T21:22:44.998862+0000","last_clean":"2024-04-20T21:22:44.998862+0000","last_became_active":"2024-04-20T21:22:44.994359+0000","last_became_peered":"2024-04-20T21:22:44.994359+0000","last_unstale":"2024-04-20T21:22:44.998862+0000","last_undegraded":"2024-04-20T21:22:44.998862+0000","last_fullsized":"2024-04-20T21:22:44.998862+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:34:26.989546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.973852+0000","last_change":"2024-04-20T21:22:44.973852+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.973852+0000","last_undegraded":"2024-04-20T21:22:44.973852+0000","last_fullsized":"2024-04-20T21:22:44.973852+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.996863+0000","last_change":"2024-04-20T21:22:44.990936+0000","last_active":"2024-04-20T21:22:44.996863+0000","last_peered":"2024-04-20T21:22:44.996863+0000","last_clean":"2024-04-20T21:22:44.996863+0000","last_became_active":"2024-04-20T21:22:44.988494+0000","last_became_peered":"2024-04-20T21:22:44.988494+0000","last_unstale":"2024-04-20T21:22:44.996863+0000","last_undegraded":"2024-04-20T21:22:44.996863+0000","last_fullsized":"2024-04-20T21:22:44.996863+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:40:14.755772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.972577+0000","last_change":"2024-04-20T21:22:44.972577+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.972577+0000","last_undegraded":"2024-04-20T21:22:44.972577+0000","last_fullsized":"2024-04-20T21:22:44.972577+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.993111+0000","last_change":"2024-04-20T21:22:44.988535+0000","last_active":"2024-04-20T21:22:44.993111+0000","last_peered":"2024-04-20T21:22:44.993111+0000","last_clean":"2024-04-20T21:22:44.993111+0000","last_became_active":"2024-04-20T21:22:44.988183+0000","last_became_peered":"2024-04-20T21:22:44.988183+0000","last_unstale":"2024-04-20T21:22:44.993111+0000","last_undegraded":"2024-04-20T21:22:44.993111+0000","last_fullsized":"2024-04-20T21:22:44.993111+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:40:51.727352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":256,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.990279+0000","last_change":"2024-04-20T21:17:43.059637+0000","last_active":"2024-04-20T21:22:44.990279+0000","last_peered":"2024-04-20T21:22:44.990279+0000","last_clean":"2024-04-20T21:22:44.990279+0000","last_became_active":"2024-04-20T21:17:43.056207+0000","last_became_peered":"2024-04-20T21:17:43.056207+0000","last_unstale":"2024-04-20T21:22:44.990279+0000","last_undegraded":"2024-04-20T21:22:44.990279+0000","last_fullsized":"2024-04-20T21:22:44.990279+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:41.036290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:41.036290+0000","last_clean_scrub_stamp":"2024-04-20T21:17:41.036290+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:58:51.091337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574911,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32584,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339256,"statfs":{"total":96636764160,"available":96603398144,"internally_reserved":0,"allocated":1433600,"data_stored":941588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574912,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":839680,"data_stored":351220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574914,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32008,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339832,"statfs":{"total":96636764160,"available":96603987968,"internally_reserved":0,"allocated":839680,"data_stored":351220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":839680,"data_stored":351220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574913,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":839680,"data_stored":351220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":839680,"data_stored":351220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":839680,"data_stored":351220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574913,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32584,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339256,"statfs":{"total":96636764160,"available":96603398144,"internally_reserved":0,"allocated":1433600,"data_stored":941588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T21:22:50.382 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T21:22:50.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.624+0000 7f080b000640 1 Processor -- start 2024-04-20T21:22:50.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.624+0000 7f080b000640 1 -- start start 2024-04-20T21:22:50.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.626+0000 7f080b000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0804144e90 0x7f080415ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:50.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.626+0000 7f080b000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0804071d00 0x7f0804072160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:50.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.626+0000 7f080b000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f080406dc30 con 0x7f080406f140 2024-04-20T21:22:50.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.626+0000 7f080b000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f08040726a0 con 0x7f0804071d00 2024-04-20T21:22:50.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.626+0000 7f080b000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0804145270 con 0x7f0804144e90 2024-04-20T21:22:50.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.626+0000 7f0809000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0804071d00 0x7f0804072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:50.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.626+0000 7f0803400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f080406f140 0x7f08040aeb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44748/0 (socket says 172.21.15.86:44748) 2024-04-20T21:22:50.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.626+0000 7f0803400640 1 -- 172.21.15.86:0/1140326544 learned_addr learned my addr 172.21.15.86:0/1140326544 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:22:50.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.627+0000 7f0803e00640 1 --2- 172.21.15.86:0/1140326544 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0804144e90 0x7f080415ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:50.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.627+0000 7f0809000640 1 -- 172.21.15.86:0/1140326544 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0804144e90 msgr2=0x7f080415ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:50.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.627+0000 7f0809000640 1 --2- 172.21.15.86:0/1140326544 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0804144e90 0x7f080415ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:50.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.627+0000 7f0809000640 1 -- 172.21.15.86:0/1140326544 >> v1:172.21.15.86:6789/0 conn(0x7f080406f140 legacy=0x7f08040aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:50.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.627+0000 7f0809000640 1 -- 172.21.15.86:0/1140326544 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f080415f720 con 0x7f0804071d00 2024-04-20T21:22:50.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.627+0000 7f0802a00640 1 -- 172.21.15.86:0/1140326544 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2997095454 0 0) 0x7f080406dc30 con 0x7f080406f140 2024-04-20T21:22:50.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.628+0000 7f0809000640 1 --2- 172.21.15.86:0/1140326544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0804071d00 0x7f0804072160 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f08041453f0 tx=0x7f07fc02be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9088dca3d0b1b8f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:50.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.628+0000 7f0802a00640 1 -- 172.21.15.86:0/1140326544 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07fc039070 con 0x7f0804071d00 2024-04-20T21:22:50.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.628+0000 7f0802a00640 1 -- 172.21.15.86:0/1140326544 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07fc02cc40 con 0x7f0804071d00 2024-04-20T21:22:50.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.628+0000 7f0802a00640 1 -- 172.21.15.86:0/1140326544 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07fc034440 con 0x7f0804071d00 2024-04-20T21:22:50.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.629+0000 7f080b000640 1 -- 172.21.15.86:0/1140326544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0804071d00 msgr2=0x7f0804072160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:50.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.629+0000 7f080b000640 1 --2- 172.21.15.86:0/1140326544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0804071d00 0x7f0804072160 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f08041453f0 tx=0x7f07fc02be90 comp rx=0 tx=0).stop 2024-04-20T21:22:50.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.629+0000 7f080b000640 1 -- 172.21.15.86:0/1140326544 shutdown_connections 2024-04-20T21:22:50.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.629+0000 7f080b000640 1 --2- 172.21.15.86:0/1140326544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0804071d00 0x7f0804072160 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:50.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.629+0000 7f080b000640 1 --2- 172.21.15.86:0/1140326544 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0804144e90 0x7f080415ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:50.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.629+0000 7f080b000640 1 -- 172.21.15.86:0/1140326544 >> 172.21.15.86:0/1140326544 conn(0x7f08040b2360 msgr2=0x7f08040b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:50.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.629+0000 7f080b000640 1 -- 172.21.15.86:0/1140326544 shutdown_connections 2024-04-20T21:22:50.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.629+0000 7f080b000640 1 -- 172.21.15.86:0/1140326544 wait complete. 2024-04-20T21:22:50.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.630+0000 7f080b000640 1 Processor -- start 2024-04-20T21:22:50.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.631+0000 7f080b000640 1 -- start start 2024-04-20T21:22:50.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.631+0000 7f080b000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f080406f140 0x7f080415e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:50.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.631+0000 7f080b000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0804071d00 0x7f08041f44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:50.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.631+0000 7f080b000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0804144e90 0x7f08040b1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:50.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.631+0000 7f080b000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f080406ded0 con 0x7f080406f140 2024-04-20T21:22:50.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.632+0000 7f080b000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f080415fce0 con 0x7f0804144e90 2024-04-20T21:22:50.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.632+0000 7f080b000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f080415f350 con 0x7f0804071d00 2024-04-20T21:22:50.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.632+0000 7f0803e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f080406f140 0x7f080415e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:50.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.632+0000 7f0803e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f080406f140 0x7f080415e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52076/0 (socket says 172.21.15.86:52076) 2024-04-20T21:22:50.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.632+0000 7f0803e00640 1 -- 172.21.15.86:0/2476127685 learned_addr learned my addr 172.21.15.86:0/2476127685 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:50.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.632+0000 7f0803e00640 1 -- 172.21.15.86:0/2476127685 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0804071d00 msgr2=0x7f08041f44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:50.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.632+0000 7f0803e00640 1 --2- 172.21.15.86:0/2476127685 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0804071d00 0x7f08041f44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:50.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.632+0000 7f0803e00640 1 -- 172.21.15.86:0/2476127685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0804144e90 msgr2=0x7f08040b1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:50.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.632+0000 7f0803e00640 1 --2- 172.21.15.86:0/2476127685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0804144e90 0x7f08040b1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:50.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.632+0000 7f0803e00640 1 -- 172.21.15.86:0/2476127685 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08041f4fb0 con 0x7f080406f140 2024-04-20T21:22:50.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.632+0000 7f0803e00640 1 --2- 172.21.15.86:0/2476127685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f080406f140 0x7f080415e0e0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f07f400d800 tx=0x7f07f400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:50.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.633+0000 7f07fbe00640 1 -- 172.21.15.86:0/2476127685 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f400b6c0 con 0x7f080406f140 2024-04-20T21:22:50.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.633+0000 7f07fbe00640 1 -- 172.21.15.86:0/2476127685 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07f4019070 con 0x7f080406f140 2024-04-20T21:22:50.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.633+0000 7f080b000640 1 -- 172.21.15.86:0/2476127685 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08041f5290 con 0x7f080406f140 2024-04-20T21:22:50.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.633+0000 7f07fbe00640 1 -- 172.21.15.86:0/2476127685 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f400ba80 con 0x7f080406f140 2024-04-20T21:22:50.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.633+0000 7f080b000640 1 -- 172.21.15.86:0/2476127685 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f080415e620 con 0x7f080406f140 2024-04-20T21:22:50.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.634+0000 7f080b000640 1 -- 172.21.15.86:0/2476127685 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08040689e0 con 0x7f080406f140 2024-04-20T21:22:50.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.635+0000 7f07fbe00640 1 -- 172.21.15.86:0/2476127685 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f07f400bc20 con 0x7f080406f140 2024-04-20T21:22:50.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.636+0000 7f07fbe00640 1 --2- 172.21.15.86:0/2476127685 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f07e406c8f0 0x7f07e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:50.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.636+0000 7f07fbe00640 1 -- 172.21.15.86:0/2476127685 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f07f408bf70 con 0x7f080406f140 2024-04-20T21:22:50.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.636+0000 7f0803400640 1 --2- 172.21.15.86:0/2476127685 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f07e406c8f0 0x7f07e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:50.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.637+0000 7f0803400640 1 --2- 172.21.15.86:0/2476127685 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f07e406c8f0 0x7f07e406edb0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f07f0007fd0 tx=0x7f07f0007960 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:50.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.641+0000 7f07fbe00640 1 -- 172.21.15.86:0/2476127685 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f07f4011340 con 0x7f080406f140 2024-04-20T21:22:50.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.831+0000 7f080b000640 1 -- 172.21.15.86:0/2476127685 --> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f08040ccc20 con 0x7f07e406c8f0 2024-04-20T21:22:50.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.850+0000 7f07fbe00640 1 -- 172.21.15.86:0/2476127685 <== mgr.4107 v2:172.21.15.86:6832/302045240 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249925 (secure 0 0 0) 0x7f08040ccc20 con 0x7f07e406c8f0 2024-04-20T21:22:50.852 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:22:50.854 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-20T21:22:50.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.856+0000 7f080b000640 1 -- 172.21.15.86:0/2476127685 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f07e406c8f0 msgr2=0x7f07e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:50.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.856+0000 7f080b000640 1 --2- 172.21.15.86:0/2476127685 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f07e406c8f0 0x7f07e406edb0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f07f0007fd0 tx=0x7f07f0007960 comp rx=0 tx=0).stop 2024-04-20T21:22:50.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.856+0000 7f080b000640 1 -- 172.21.15.86:0/2476127685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f080406f140 msgr2=0x7f080415e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:50.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.856+0000 7f080b000640 1 --2- 172.21.15.86:0/2476127685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f080406f140 0x7f080415e0e0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f07f400d800 tx=0x7f07f400dcc0 comp rx=0 tx=0).stop 2024-04-20T21:22:50.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.856+0000 7f080b000640 1 -- 172.21.15.86:0/2476127685 shutdown_connections 2024-04-20T21:22:50.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.856+0000 7f080b000640 1 --2- 172.21.15.86:0/2476127685 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f07e406c8f0 0x7f07e406edb0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:50.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.856+0000 7f080b000640 1 --2- 172.21.15.86:0/2476127685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0804144e90 0x7f08040b1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:50.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.856+0000 7f080b000640 1 --2- 172.21.15.86:0/2476127685 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0804071d00 0x7f08041f44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:50.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.856+0000 7f080b000640 1 --2- 172.21.15.86:0/2476127685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f080406f140 0x7f080415e0e0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:50.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.857+0000 7f080b000640 1 -- 172.21.15.86:0/2476127685 >> 172.21.15.86:0/2476127685 conn(0x7f08040b2360 msgr2=0x7f0804141a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:50.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.857+0000 7f080b000640 1 -- 172.21.15.86:0/2476127685 shutdown_connections 2024-04-20T21:22:50.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:50.857+0000 7f080b000640 1 -- 172.21.15.86:0/2476127685 wait complete. 2024-04-20T21:22:50.872 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":219,"stamp":"2024-04-20T21:22:50.732372+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592104,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":191,"num_write_kb":2427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":128,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257356,"kb_used_data":7888,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754717364,"statfs":{"total":773094113280,"available":772830580736,"internally_reserved":0,"allocated":8077312,"data_stored":4059983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1736,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001508"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.989723+0000","last_change":"2024-04-20T21:22:44.989723+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.989723+0000","last_undegraded":"2024-04-20T21:22:44.989723+0000","last_fullsized":"2024-04-20T21:22:44.989723+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.972721+0000","last_change":"2024-04-20T21:22:44.972721+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.972721+0000","last_undegraded":"2024-04-20T21:22:44.972721+0000","last_fullsized":"2024-04-20T21:22:44.972721+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.358114+0000","last_change":"2024-04-20T21:22:46.000461+0000","last_active":"2024-04-20T21:22:47.358114+0000","last_peered":"2024-04-20T21:22:47.358114+0000","last_clean":"2024-04-20T21:22:47.358114+0000","last_became_active":"2024-04-20T21:22:46.000188+0000","last_became_peered":"2024-04-20T21:22:46.000188+0000","last_unstale":"2024-04-20T21:22:47.358114+0000","last_undegraded":"2024-04-20T21:22:47.358114+0000","last_fullsized":"2024-04-20T21:22:47.358114+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:27:38.679422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.996119+0000","last_change":"2024-04-20T21:22:45.996176+0000","last_active":"2024-04-20T21:22:46.996119+0000","last_peered":"2024-04-20T21:22:46.996119+0000","last_clean":"2024-04-20T21:22:46.996119+0000","last_became_active":"2024-04-20T21:22:45.995860+0000","last_became_peered":"2024-04-20T21:22:45.995860+0000","last_unstale":"2024-04-20T21:22:46.996119+0000","last_undegraded":"2024-04-20T21:22:46.996119+0000","last_fullsized":"2024-04-20T21:22:46.996119+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:19:56.685552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.357267+0000","last_change":"2024-04-20T21:22:46.003219+0000","last_active":"2024-04-20T21:22:47.357267+0000","last_peered":"2024-04-20T21:22:47.357267+0000","last_clean":"2024-04-20T21:22:47.357267+0000","last_became_active":"2024-04-20T21:22:46.002927+0000","last_became_peered":"2024-04-20T21:22:46.002927+0000","last_unstale":"2024-04-20T21:22:47.357267+0000","last_undegraded":"2024-04-20T21:22:47.357267+0000","last_fullsized":"2024-04-20T21:22:47.357267+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:18:23.304303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.976258+0000","last_change":"2024-04-20T21:22:44.976258+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.976258+0000","last_undegraded":"2024-04-20T21:22:44.976258+0000","last_fullsized":"2024-04-20T21:22:44.976258+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.014778+0000","last_change":"2024-04-20T21:22:45.994406+0000","last_active":"2024-04-20T21:22:47.014778+0000","last_peered":"2024-04-20T21:22:47.014778+0000","last_clean":"2024-04-20T21:22:47.014778+0000","last_became_active":"2024-04-20T21:22:45.994109+0000","last_became_peered":"2024-04-20T21:22:45.994109+0000","last_unstale":"2024-04-20T21:22:47.014778+0000","last_undegraded":"2024-04-20T21:22:47.014778+0000","last_fullsized":"2024-04-20T21:22:47.014778+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:06:06.356878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.971878+0000","last_change":"2024-04-20T21:22:44.971878+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.971878+0000","last_undegraded":"2024-04-20T21:22:44.971878+0000","last_fullsized":"2024-04-20T21:22:44.971878+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.358759+0000","last_change":"2024-04-20T21:22:46.009487+0000","last_active":"2024-04-20T21:22:47.358759+0000","last_peered":"2024-04-20T21:22:47.358759+0000","last_clean":"2024-04-20T21:22:47.358759+0000","last_became_active":"2024-04-20T21:22:46.008835+0000","last_became_peered":"2024-04-20T21:22:46.008835+0000","last_unstale":"2024-04-20T21:22:47.358759+0000","last_undegraded":"2024-04-20T21:22:47.358759+0000","last_fullsized":"2024-04-20T21:22:47.358759+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:19:35.848351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.001975+0000","last_change":"2024-04-20T21:22:46.008121+0000","last_active":"2024-04-20T21:22:47.001975+0000","last_peered":"2024-04-20T21:22:47.001975+0000","last_clean":"2024-04-20T21:22:47.001975+0000","last_became_active":"2024-04-20T21:22:46.007835+0000","last_became_peered":"2024-04-20T21:22:46.007835+0000","last_unstale":"2024-04-20T21:22:47.001975+0000","last_undegraded":"2024-04-20T21:22:47.001975+0000","last_fullsized":"2024-04-20T21:22:47.001975+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:42:09.491911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.989195+0000","last_change":"2024-04-20T21:22:44.989195+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.989195+0000","last_undegraded":"2024-04-20T21:22:44.989195+0000","last_fullsized":"2024-04-20T21:22:44.989195+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.996166+0000","last_change":"2024-04-20T21:22:46.005658+0000","last_active":"2024-04-20T21:22:46.996166+0000","last_peered":"2024-04-20T21:22:46.996166+0000","last_clean":"2024-04-20T21:22:46.996166+0000","last_became_active":"2024-04-20T21:22:46.005346+0000","last_became_peered":"2024-04-20T21:22:46.005346+0000","last_unstale":"2024-04-20T21:22:46.996166+0000","last_undegraded":"2024-04-20T21:22:46.996166+0000","last_fullsized":"2024-04-20T21:22:46.996166+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:35:49.635636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.983752+0000","last_change":"2024-04-20T21:22:44.983752+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.983752+0000","last_undegraded":"2024-04-20T21:22:44.983752+0000","last_fullsized":"2024-04-20T21:22:44.983752+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.357690+0000","last_change":"2024-04-20T21:22:46.001211+0000","last_active":"2024-04-20T21:22:47.357690+0000","last_peered":"2024-04-20T21:22:47.357690+0000","last_clean":"2024-04-20T21:22:47.357690+0000","last_became_active":"2024-04-20T21:22:46.000899+0000","last_became_peered":"2024-04-20T21:22:46.000899+0000","last_unstale":"2024-04-20T21:22:47.357690+0000","last_undegraded":"2024-04-20T21:22:47.357690+0000","last_fullsized":"2024-04-20T21:22:47.357690+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:08:57.585935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.984365+0000","last_change":"2024-04-20T21:22:44.984365+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.984365+0000","last_undegraded":"2024-04-20T21:22:44.984365+0000","last_fullsized":"2024-04-20T21:22:44.984365+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.999077+0000","last_change":"2024-04-20T21:22:45.998551+0000","last_active":"2024-04-20T21:22:46.999077+0000","last_peered":"2024-04-20T21:22:46.999077+0000","last_clean":"2024-04-20T21:22:46.999077+0000","last_became_active":"2024-04-20T21:22:45.996213+0000","last_became_peered":"2024-04-20T21:22:45.996213+0000","last_unstale":"2024-04-20T21:22:46.999077+0000","last_undegraded":"2024-04-20T21:22:46.999077+0000","last_fullsized":"2024-04-20T21:22:46.999077+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:49:36.482159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.358766+0000","last_change":"2024-04-20T21:22:46.013340+0000","last_active":"2024-04-20T21:22:47.358766+0000","last_peered":"2024-04-20T21:22:47.358766+0000","last_clean":"2024-04-20T21:22:47.358766+0000","last_became_active":"2024-04-20T21:22:46.009224+0000","last_became_peered":"2024-04-20T21:22:46.009224+0000","last_unstale":"2024-04-20T21:22:47.358766+0000","last_undegraded":"2024-04-20T21:22:47.358766+0000","last_fullsized":"2024-04-20T21:22:47.358766+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:33:18.596222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.996803+0000","last_change":"2024-04-20T21:22:45.987458+0000","last_active":"2024-04-20T21:22:46.996803+0000","last_peered":"2024-04-20T21:22:46.996803+0000","last_clean":"2024-04-20T21:22:46.996803+0000","last_became_active":"2024-04-20T21:22:45.987046+0000","last_became_peered":"2024-04-20T21:22:45.987046+0000","last_unstale":"2024-04-20T21:22:46.996803+0000","last_undegraded":"2024-04-20T21:22:46.996803+0000","last_fullsized":"2024-04-20T21:22:46.996803+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:09:41.573665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.978703+0000","last_change":"2024-04-20T21:22:44.978703+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.978703+0000","last_undegraded":"2024-04-20T21:22:44.978703+0000","last_fullsized":"2024-04-20T21:22:44.978703+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.986040+0000","last_change":"2024-04-20T21:22:44.986040+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.986040+0000","last_undegraded":"2024-04-20T21:22:44.986040+0000","last_fullsized":"2024-04-20T21:22:44.986040+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.000249+0000","last_change":"2024-04-20T21:22:45.998328+0000","last_active":"2024-04-20T21:22:47.000249+0000","last_peered":"2024-04-20T21:22:47.000249+0000","last_clean":"2024-04-20T21:22:47.000249+0000","last_became_active":"2024-04-20T21:22:45.998016+0000","last_became_peered":"2024-04-20T21:22:45.998016+0000","last_unstale":"2024-04-20T21:22:47.000249+0000","last_undegraded":"2024-04-20T21:22:47.000249+0000","last_fullsized":"2024-04-20T21:22:47.000249+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:53:42.792540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.999681+0000","last_change":"2024-04-20T21:22:46.006835+0000","last_active":"2024-04-20T21:22:46.999681+0000","last_peered":"2024-04-20T21:22:46.999681+0000","last_clean":"2024-04-20T21:22:46.999681+0000","last_became_active":"2024-04-20T21:22:46.006544+0000","last_became_peered":"2024-04-20T21:22:46.006544+0000","last_unstale":"2024-04-20T21:22:46.999681+0000","last_undegraded":"2024-04-20T21:22:46.999681+0000","last_fullsized":"2024-04-20T21:22:46.999681+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:05:10.087962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.985232+0000","last_change":"2024-04-20T21:22:44.985232+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.985232+0000","last_undegraded":"2024-04-20T21:22:44.985232+0000","last_fullsized":"2024-04-20T21:22:44.985232+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.983826+0000","last_change":"2024-04-20T21:22:44.983826+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.983826+0000","last_undegraded":"2024-04-20T21:22:44.983826+0000","last_fullsized":"2024-04-20T21:22:44.983826+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.001458+0000","last_change":"2024-04-20T21:22:46.009435+0000","last_active":"2024-04-20T21:22:47.001458+0000","last_peered":"2024-04-20T21:22:47.001458+0000","last_clean":"2024-04-20T21:22:47.001458+0000","last_became_active":"2024-04-20T21:22:46.008653+0000","last_became_peered":"2024-04-20T21:22:46.008653+0000","last_unstale":"2024-04-20T21:22:47.001458+0000","last_undegraded":"2024-04-20T21:22:47.001458+0000","last_fullsized":"2024-04-20T21:22:47.001458+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:58:27.202522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.015568+0000","last_change":"2024-04-20T21:22:45.999532+0000","last_active":"2024-04-20T21:22:47.015568+0000","last_peered":"2024-04-20T21:22:47.015568+0000","last_clean":"2024-04-20T21:22:47.015568+0000","last_became_active":"2024-04-20T21:22:45.999223+0000","last_became_peered":"2024-04-20T21:22:45.999223+0000","last_unstale":"2024-04-20T21:22:47.015568+0000","last_undegraded":"2024-04-20T21:22:47.015568+0000","last_fullsized":"2024-04-20T21:22:47.015568+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:55:14.180769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.358026+0000","last_change":"2024-04-20T21:22:46.002322+0000","last_active":"2024-04-20T21:22:47.358026+0000","last_peered":"2024-04-20T21:22:47.358026+0000","last_clean":"2024-04-20T21:22:47.358026+0000","last_became_active":"2024-04-20T21:22:46.002035+0000","last_became_peered":"2024-04-20T21:22:46.002035+0000","last_unstale":"2024-04-20T21:22:47.358026+0000","last_undegraded":"2024-04-20T21:22:47.358026+0000","last_fullsized":"2024-04-20T21:22:47.358026+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:47:00.839639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.001277+0000","last_change":"2024-04-20T21:22:46.006624+0000","last_active":"2024-04-20T21:22:47.001277+0000","last_peered":"2024-04-20T21:22:47.001277+0000","last_clean":"2024-04-20T21:22:47.001277+0000","last_became_active":"2024-04-20T21:22:46.006273+0000","last_became_peered":"2024-04-20T21:22:46.006273+0000","last_unstale":"2024-04-20T21:22:47.001277+0000","last_undegraded":"2024-04-20T21:22:47.001277+0000","last_fullsized":"2024-04-20T21:22:47.001277+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:06:06.356878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.357002+0000","last_change":"2024-04-20T21:22:45.998542+0000","last_active":"2024-04-20T21:22:47.357002+0000","last_peered":"2024-04-20T21:22:47.357002+0000","last_clean":"2024-04-20T21:22:47.357002+0000","last_became_active":"2024-04-20T21:22:45.998274+0000","last_became_peered":"2024-04-20T21:22:45.998274+0000","last_unstale":"2024-04-20T21:22:47.357002+0000","last_undegraded":"2024-04-20T21:22:47.357002+0000","last_fullsized":"2024-04-20T21:22:47.357002+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:17:21.685206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.014934+0000","last_change":"2024-04-20T21:22:45.995816+0000","last_active":"2024-04-20T21:22:47.014934+0000","last_peered":"2024-04-20T21:22:47.014934+0000","last_clean":"2024-04-20T21:22:47.014934+0000","last_became_active":"2024-04-20T21:22:45.994917+0000","last_became_peered":"2024-04-20T21:22:45.994917+0000","last_unstale":"2024-04-20T21:22:47.014934+0000","last_undegraded":"2024-04-20T21:22:47.014934+0000","last_fullsized":"2024-04-20T21:22:47.014934+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:53:17.849382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.015118+0000","last_change":"2024-04-20T21:22:46.001790+0000","last_active":"2024-04-20T21:22:47.015118+0000","last_peered":"2024-04-20T21:22:47.015118+0000","last_clean":"2024-04-20T21:22:47.015118+0000","last_became_active":"2024-04-20T21:22:46.001494+0000","last_became_peered":"2024-04-20T21:22:46.001494+0000","last_unstale":"2024-04-20T21:22:47.015118+0000","last_undegraded":"2024-04-20T21:22:47.015118+0000","last_fullsized":"2024-04-20T21:22:47.015118+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:42:09.491911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.970617+0000","last_change":"2024-04-20T21:22:44.970617+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.970617+0000","last_undegraded":"2024-04-20T21:22:44.970617+0000","last_fullsized":"2024-04-20T21:22:44.970617+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.982545+0000","last_change":"2024-04-20T21:22:44.982545+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.982545+0000","last_undegraded":"2024-04-20T21:22:44.982545+0000","last_fullsized":"2024-04-20T21:22:44.982545+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:45.004990+0000","last_change":"2024-04-20T21:22:45.000877+0000","last_active":"2024-04-20T21:22:45.004990+0000","last_peered":"2024-04-20T21:22:45.004990+0000","last_clean":"2024-04-20T21:22:45.004990+0000","last_became_active":"2024-04-20T21:22:45.000402+0000","last_became_peered":"2024-04-20T21:22:45.000402+0000","last_unstale":"2024-04-20T21:22:45.004990+0000","last_undegraded":"2024-04-20T21:22:45.004990+0000","last_fullsized":"2024-04-20T21:22:45.004990+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:56:13.768402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.358957+0000","last_change":"2024-04-20T21:22:46.007646+0000","last_active":"2024-04-20T21:22:47.358957+0000","last_peered":"2024-04-20T21:22:47.358957+0000","last_clean":"2024-04-20T21:22:47.358957+0000","last_became_active":"2024-04-20T21:22:46.007304+0000","last_became_peered":"2024-04-20T21:22:46.007304+0000","last_unstale":"2024-04-20T21:22:47.358957+0000","last_undegraded":"2024-04-20T21:22:47.358957+0000","last_fullsized":"2024-04-20T21:22:47.358957+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:45:53.505415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.999915+0000","last_change":"2024-04-20T21:22:44.998432+0000","last_active":"2024-04-20T21:22:44.999915+0000","last_peered":"2024-04-20T21:22:44.999915+0000","last_clean":"2024-04-20T21:22:44.999915+0000","last_became_active":"2024-04-20T21:22:44.998006+0000","last_became_peered":"2024-04-20T21:22:44.998006+0000","last_unstale":"2024-04-20T21:22:44.999915+0000","last_undegraded":"2024-04-20T21:22:44.999915+0000","last_fullsized":"2024-04-20T21:22:44.999915+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:53:04.669954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.979534+0000","last_change":"2024-04-20T21:22:44.979534+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.979534+0000","last_undegraded":"2024-04-20T21:22:44.979534+0000","last_fullsized":"2024-04-20T21:22:44.979534+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"70'2","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.001646+0000","last_change":"2024-04-20T21:22:44.997985+0000","last_active":"2024-04-20T21:22:47.001646+0000","last_peered":"2024-04-20T21:22:47.001646+0000","last_clean":"2024-04-20T21:22:47.001646+0000","last_became_active":"2024-04-20T21:22:44.997672+0000","last_became_peered":"2024-04-20T21:22:44.997672+0000","last_unstale":"2024-04-20T21:22:47.001646+0000","last_undegraded":"2024-04-20T21:22:47.001646+0000","last_fullsized":"2024-04-20T21:22:47.001646+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:10:01.129225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.988718+0000","last_change":"2024-04-20T21:22:44.988718+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.988718+0000","last_undegraded":"2024-04-20T21:22:44.988718+0000","last_fullsized":"2024-04-20T21:22:44.988718+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.996650+0000","last_change":"2024-04-20T21:22:44.989343+0000","last_active":"2024-04-20T21:22:44.996650+0000","last_peered":"2024-04-20T21:22:44.996650+0000","last_clean":"2024-04-20T21:22:44.996650+0000","last_became_active":"2024-04-20T21:22:44.989022+0000","last_became_peered":"2024-04-20T21:22:44.989022+0000","last_unstale":"2024-04-20T21:22:44.996650+0000","last_undegraded":"2024-04-20T21:22:44.996650+0000","last_fullsized":"2024-04-20T21:22:44.996650+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:45:48.879276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.014909+0000","last_change":"2024-04-20T21:22:46.004052+0000","last_active":"2024-04-20T21:22:47.014909+0000","last_peered":"2024-04-20T21:22:47.014909+0000","last_clean":"2024-04-20T21:22:47.014909+0000","last_became_active":"2024-04-20T21:22:46.002637+0000","last_became_peered":"2024-04-20T21:22:46.002637+0000","last_unstale":"2024-04-20T21:22:47.014909+0000","last_undegraded":"2024-04-20T21:22:47.014909+0000","last_fullsized":"2024-04-20T21:22:47.014909+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:58:27.202522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"70'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.357559+0000","last_change":"2024-04-20T21:22:44.994866+0000","last_active":"2024-04-20T21:22:47.357559+0000","last_peered":"2024-04-20T21:22:47.357559+0000","last_clean":"2024-04-20T21:22:47.357559+0000","last_became_active":"2024-04-20T21:22:44.994089+0000","last_became_peered":"2024-04-20T21:22:44.994089+0000","last_unstale":"2024-04-20T21:22:47.357559+0000","last_undegraded":"2024-04-20T21:22:47.357559+0000","last_fullsized":"2024-04-20T21:22:47.357559+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:42:08.475979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.000407+0000","last_change":"2024-04-20T21:22:46.008256+0000","last_active":"2024-04-20T21:22:47.000407+0000","last_peered":"2024-04-20T21:22:47.000407+0000","last_clean":"2024-04-20T21:22:47.000407+0000","last_became_active":"2024-04-20T21:22:46.007960+0000","last_became_peered":"2024-04-20T21:22:46.007960+0000","last_unstale":"2024-04-20T21:22:47.000407+0000","last_undegraded":"2024-04-20T21:22:47.000407+0000","last_fullsized":"2024-04-20T21:22:47.000407+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:28:56.572056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.999221+0000","last_change":"2024-04-20T21:22:44.995756+0000","last_active":"2024-04-20T21:22:44.999221+0000","last_peered":"2024-04-20T21:22:44.999221+0000","last_clean":"2024-04-20T21:22:44.999221+0000","last_became_active":"2024-04-20T21:22:44.995451+0000","last_became_peered":"2024-04-20T21:22:44.995451+0000","last_unstale":"2024-04-20T21:22:44.999221+0000","last_undegraded":"2024-04-20T21:22:44.999221+0000","last_fullsized":"2024-04-20T21:22:44.999221+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:37:36.078254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.356960+0000","last_change":"2024-04-20T21:22:46.004215+0000","last_active":"2024-04-20T21:22:47.356960+0000","last_peered":"2024-04-20T21:22:47.356960+0000","last_clean":"2024-04-20T21:22:47.356960+0000","last_became_active":"2024-04-20T21:22:46.003804+0000","last_became_peered":"2024-04-20T21:22:46.003804+0000","last_unstale":"2024-04-20T21:22:47.356960+0000","last_undegraded":"2024-04-20T21:22:47.356960+0000","last_fullsized":"2024-04-20T21:22:47.356960+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:45:49.895208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.995960+0000","last_change":"2024-04-20T21:22:44.996824+0000","last_active":"2024-04-20T21:22:46.995960+0000","last_peered":"2024-04-20T21:22:46.995960+0000","last_clean":"2024-04-20T21:22:46.995960+0000","last_became_active":"2024-04-20T21:22:44.996531+0000","last_became_peered":"2024-04-20T21:22:44.996531+0000","last_unstale":"2024-04-20T21:22:46.995960+0000","last_undegraded":"2024-04-20T21:22:46.995960+0000","last_fullsized":"2024-04-20T21:22:46.995960+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:01:19.361992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.983784+0000","last_change":"2024-04-20T21:22:44.983784+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.983784+0000","last_undegraded":"2024-04-20T21:22:44.983784+0000","last_fullsized":"2024-04-20T21:22:44.983784+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.356944+0000","last_change":"2024-04-20T21:22:44.982668+0000","last_active":"2024-04-20T21:22:47.356944+0000","last_peered":"2024-04-20T21:22:47.356944+0000","last_clean":"2024-04-20T21:22:47.356944+0000","last_became_active":"2024-04-20T21:22:44.982035+0000","last_became_peered":"2024-04-20T21:22:44.982035+0000","last_unstale":"2024-04-20T21:22:47.356944+0000","last_undegraded":"2024-04-20T21:22:47.356944+0000","last_fullsized":"2024-04-20T21:22:47.356944+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:04:12.006293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.015640+0000","last_change":"2024-04-20T21:22:46.005753+0000","last_active":"2024-04-20T21:22:47.015640+0000","last_peered":"2024-04-20T21:22:47.015640+0000","last_clean":"2024-04-20T21:22:47.015640+0000","last_became_active":"2024-04-20T21:22:46.005305+0000","last_became_peered":"2024-04-20T21:22:46.005305+0000","last_unstale":"2024-04-20T21:22:47.015640+0000","last_undegraded":"2024-04-20T21:22:47.015640+0000","last_fullsized":"2024-04-20T21:22:47.015640+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:27:57.913146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.001557+0000","last_change":"2024-04-20T21:22:44.994181+0000","last_active":"2024-04-20T21:22:47.001557+0000","last_peered":"2024-04-20T21:22:47.001557+0000","last_clean":"2024-04-20T21:22:47.001557+0000","last_became_active":"2024-04-20T21:22:44.992861+0000","last_became_peered":"2024-04-20T21:22:44.992861+0000","last_unstale":"2024-04-20T21:22:47.001557+0000","last_undegraded":"2024-04-20T21:22:47.001557+0000","last_fullsized":"2024-04-20T21:22:47.001557+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:52:23.085259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.988271+0000","last_change":"2024-04-20T21:22:44.988271+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.988271+0000","last_undegraded":"2024-04-20T21:22:44.988271+0000","last_fullsized":"2024-04-20T21:22:44.988271+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.999321+0000","last_change":"2024-04-20T21:22:44.992156+0000","last_active":"2024-04-20T21:22:44.999321+0000","last_peered":"2024-04-20T21:22:44.999321+0000","last_clean":"2024-04-20T21:22:44.999321+0000","last_became_active":"2024-04-20T21:22:44.991749+0000","last_became_peered":"2024-04-20T21:22:44.991749+0000","last_unstale":"2024-04-20T21:22:44.999321+0000","last_undegraded":"2024-04-20T21:22:44.999321+0000","last_fullsized":"2024-04-20T21:22:44.999321+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:59:10.369089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.977417+0000","last_change":"2024-04-20T21:22:44.977417+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.977417+0000","last_undegraded":"2024-04-20T21:22:44.977417+0000","last_fullsized":"2024-04-20T21:22:44.977417+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"70'3","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.358192+0000","last_change":"2024-04-20T21:22:44.990563+0000","last_active":"2024-04-20T21:22:47.358192+0000","last_peered":"2024-04-20T21:22:47.358192+0000","last_clean":"2024-04-20T21:22:47.358192+0000","last_became_active":"2024-04-20T21:22:44.990292+0000","last_became_peered":"2024-04-20T21:22:44.990292+0000","last_unstale":"2024-04-20T21:22:47.358192+0000","last_undegraded":"2024-04-20T21:22:47.358192+0000","last_fullsized":"2024-04-20T21:22:47.358192+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:55:37.925344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.357584+0000","last_change":"2024-04-20T21:22:45.997098+0000","last_active":"2024-04-20T21:22:47.357584+0000","last_peered":"2024-04-20T21:22:47.357584+0000","last_clean":"2024-04-20T21:22:47.357584+0000","last_became_active":"2024-04-20T21:22:45.996820+0000","last_became_peered":"2024-04-20T21:22:45.996820+0000","last_unstale":"2024-04-20T21:22:47.357584+0000","last_undegraded":"2024-04-20T21:22:47.357584+0000","last_fullsized":"2024-04-20T21:22:47.357584+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:59:11.385021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"70'6","reported_seq":20,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.996263+0000","last_change":"2024-04-20T21:22:44.988227+0000","last_active":"2024-04-20T21:22:46.996263+0000","last_peered":"2024-04-20T21:22:46.996263+0000","last_clean":"2024-04-20T21:22:46.996263+0000","last_became_active":"2024-04-20T21:22:44.987939+0000","last_became_peered":"2024-04-20T21:22:44.987939+0000","last_unstale":"2024-04-20T21:22:46.996263+0000","last_undegraded":"2024-04-20T21:22:46.996263+0000","last_fullsized":"2024-04-20T21:22:46.996263+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:46:00.265362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.985271+0000","last_change":"2024-04-20T21:22:44.985271+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.985271+0000","last_undegraded":"2024-04-20T21:22:44.985271+0000","last_fullsized":"2024-04-20T21:22:44.985271+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"70'2","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.996842+0000","last_change":"2024-04-20T21:22:44.992317+0000","last_active":"2024-04-20T21:22:46.996842+0000","last_peered":"2024-04-20T21:22:46.996842+0000","last_clean":"2024-04-20T21:22:46.996842+0000","last_became_active":"2024-04-20T21:22:44.992040+0000","last_became_peered":"2024-04-20T21:22:44.992040+0000","last_unstale":"2024-04-20T21:22:46.996842+0000","last_undegraded":"2024-04-20T21:22:46.996842+0000","last_fullsized":"2024-04-20T21:22:46.996842+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:50:24.320805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.015238+0000","last_change":"2024-04-20T21:22:45.998870+0000","last_active":"2024-04-20T21:22:47.015238+0000","last_peered":"2024-04-20T21:22:47.015238+0000","last_clean":"2024-04-20T21:22:47.015238+0000","last_became_active":"2024-04-20T21:22:45.998537+0000","last_became_peered":"2024-04-20T21:22:45.998537+0000","last_unstale":"2024-04-20T21:22:47.015238+0000","last_undegraded":"2024-04-20T21:22:47.015238+0000","last_fullsized":"2024-04-20T21:22:47.015238+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:55:38.941276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.992803+0000","last_change":"2024-04-20T21:22:44.987324+0000","last_active":"2024-04-20T21:22:44.992803+0000","last_peered":"2024-04-20T21:22:44.992803+0000","last_clean":"2024-04-20T21:22:44.992803+0000","last_became_active":"2024-04-20T21:22:44.986974+0000","last_became_peered":"2024-04-20T21:22:44.986974+0000","last_unstale":"2024-04-20T21:22:44.992803+0000","last_undegraded":"2024-04-20T21:22:44.992803+0000","last_fullsized":"2024-04-20T21:22:44.992803+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:40:38.463867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.983547+0000","last_change":"2024-04-20T21:22:44.983547+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.983547+0000","last_undegraded":"2024-04-20T21:22:44.983547+0000","last_fullsized":"2024-04-20T21:22:44.983547+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.014700+0000","last_change":"2024-04-20T21:22:44.997932+0000","last_active":"2024-04-20T21:22:47.014700+0000","last_peered":"2024-04-20T21:22:47.014700+0000","last_clean":"2024-04-20T21:22:47.014700+0000","last_became_active":"2024-04-20T21:22:44.997397+0000","last_became_peered":"2024-04-20T21:22:44.997397+0000","last_unstale":"2024-04-20T21:22:47.014700+0000","last_undegraded":"2024-04-20T21:22:47.014700+0000","last_fullsized":"2024-04-20T21:22:47.014700+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:56:13.768402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.357344+0000","last_change":"2024-04-20T21:22:45.997195+0000","last_active":"2024-04-20T21:22:47.357344+0000","last_peered":"2024-04-20T21:22:47.357344+0000","last_clean":"2024-04-20T21:22:47.357344+0000","last_became_active":"2024-04-20T21:22:45.996696+0000","last_became_peered":"2024-04-20T21:22:45.996696+0000","last_unstale":"2024-04-20T21:22:47.357344+0000","last_undegraded":"2024-04-20T21:22:47.357344+0000","last_fullsized":"2024-04-20T21:22:47.357344+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:40:15.771704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"70'8","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.998939+0000","last_change":"2024-04-20T21:22:44.995360+0000","last_active":"2024-04-20T21:22:46.998939+0000","last_peered":"2024-04-20T21:22:46.998939+0000","last_clean":"2024-04-20T21:22:46.998939+0000","last_became_active":"2024-04-20T21:22:44.995055+0000","last_became_peered":"2024-04-20T21:22:44.995055+0000","last_unstale":"2024-04-20T21:22:46.998939+0000","last_undegraded":"2024-04-20T21:22:46.998939+0000","last_fullsized":"2024-04-20T21:22:46.998939+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:14:22.771721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.001952+0000","last_change":"2024-04-20T21:22:45.993981+0000","last_active":"2024-04-20T21:22:47.001952+0000","last_peered":"2024-04-20T21:22:47.001952+0000","last_clean":"2024-04-20T21:22:47.001952+0000","last_became_active":"2024-04-20T21:22:45.993203+0000","last_became_peered":"2024-04-20T21:22:45.993203+0000","last_unstale":"2024-04-20T21:22:47.001952+0000","last_undegraded":"2024-04-20T21:22:47.001952+0000","last_fullsized":"2024-04-20T21:22:47.001952+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:53:17.849382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.999385+0000","last_change":"2024-04-20T21:22:44.993745+0000","last_active":"2024-04-20T21:22:44.999385+0000","last_peered":"2024-04-20T21:22:44.999385+0000","last_clean":"2024-04-20T21:22:44.999385+0000","last_became_active":"2024-04-20T21:22:44.992981+0000","last_became_peered":"2024-04-20T21:22:44.992981+0000","last_unstale":"2024-04-20T21:22:44.999385+0000","last_undegraded":"2024-04-20T21:22:44.999385+0000","last_fullsized":"2024-04-20T21:22:44.999385+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:08:56.570003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.987280+0000","last_change":"2024-04-20T21:22:44.987280+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.987280+0000","last_undegraded":"2024-04-20T21:22:44.987280+0000","last_fullsized":"2024-04-20T21:22:44.987280+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"70'2","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.002056+0000","last_change":"2024-04-20T21:22:44.998796+0000","last_active":"2024-04-20T21:22:47.002056+0000","last_peered":"2024-04-20T21:22:47.002056+0000","last_clean":"2024-04-20T21:22:47.002056+0000","last_became_active":"2024-04-20T21:22:44.998424+0000","last_became_peered":"2024-04-20T21:22:44.998424+0000","last_unstale":"2024-04-20T21:22:47.002056+0000","last_undegraded":"2024-04-20T21:22:47.002056+0000","last_fullsized":"2024-04-20T21:22:47.002056+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:45:10.580900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.996286+0000","last_change":"2024-04-20T21:22:45.999490+0000","last_active":"2024-04-20T21:22:46.996286+0000","last_peered":"2024-04-20T21:22:46.996286+0000","last_clean":"2024-04-20T21:22:46.996286+0000","last_became_active":"2024-04-20T21:22:45.999118+0000","last_became_peered":"2024-04-20T21:22:45.999118+0000","last_unstale":"2024-04-20T21:22:46.996286+0000","last_undegraded":"2024-04-20T21:22:46.996286+0000","last_fullsized":"2024-04-20T21:22:46.996286+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:55:03.918649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:45.003213+0000","last_change":"2024-04-20T21:22:45.000092+0000","last_active":"2024-04-20T21:22:45.003213+0000","last_peered":"2024-04-20T21:22:45.003213+0000","last_clean":"2024-04-20T21:22:45.003213+0000","last_became_active":"2024-04-20T21:22:44.998998+0000","last_became_peered":"2024-04-20T21:22:44.998998+0000","last_unstale":"2024-04-20T21:22:45.003213+0000","last_undegraded":"2024-04-20T21:22:45.003213+0000","last_fullsized":"2024-04-20T21:22:45.003213+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:19:34.832419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.358172+0000","last_change":"2024-04-20T21:22:46.001345+0000","last_active":"2024-04-20T21:22:47.358172+0000","last_peered":"2024-04-20T21:22:47.358172+0000","last_clean":"2024-04-20T21:22:47.358172+0000","last_became_active":"2024-04-20T21:22:46.001055+0000","last_became_peered":"2024-04-20T21:22:46.001055+0000","last_unstale":"2024-04-20T21:22:47.358172+0000","last_undegraded":"2024-04-20T21:22:47.358172+0000","last_fullsized":"2024-04-20T21:22:47.358172+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:48:59.404081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:45.007767+0000","last_change":"2024-04-20T21:22:45.002908+0000","last_active":"2024-04-20T21:22:45.007767+0000","last_peered":"2024-04-20T21:22:45.007767+0000","last_clean":"2024-04-20T21:22:45.007767+0000","last_became_active":"2024-04-20T21:22:45.002596+0000","last_became_peered":"2024-04-20T21:22:45.002596+0000","last_unstale":"2024-04-20T21:22:45.007767+0000","last_undegraded":"2024-04-20T21:22:45.007767+0000","last_fullsized":"2024-04-20T21:22:45.007767+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:07:15.232666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.357387+0000","last_change":"2024-04-20T21:22:46.004184+0000","last_active":"2024-04-20T21:22:47.357387+0000","last_peered":"2024-04-20T21:22:47.357387+0000","last_clean":"2024-04-20T21:22:47.357387+0000","last_became_active":"2024-04-20T21:22:46.003915+0000","last_became_peered":"2024-04-20T21:22:46.003915+0000","last_unstale":"2024-04-20T21:22:47.357387+0000","last_undegraded":"2024-04-20T21:22:47.357387+0000","last_fullsized":"2024-04-20T21:22:47.357387+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:40:38.260253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:45.000938+0000","last_change":"2024-04-20T21:22:44.996534+0000","last_active":"2024-04-20T21:22:45.000938+0000","last_peered":"2024-04-20T21:22:45.000938+0000","last_clean":"2024-04-20T21:22:45.000938+0000","last_became_active":"2024-04-20T21:22:44.996190+0000","last_became_peered":"2024-04-20T21:22:44.996190+0000","last_unstale":"2024-04-20T21:22:45.000938+0000","last_undegraded":"2024-04-20T21:22:45.000938+0000","last_fullsized":"2024-04-20T21:22:45.000938+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:48:58.388149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.356519+0000","last_change":"2024-04-20T21:22:46.005627+0000","last_active":"2024-04-20T21:22:47.356519+0000","last_peered":"2024-04-20T21:22:47.356519+0000","last_clean":"2024-04-20T21:22:47.356519+0000","last_became_active":"2024-04-20T21:22:46.005377+0000","last_became_peered":"2024-04-20T21:22:46.005377+0000","last_unstale":"2024-04-20T21:22:47.356519+0000","last_undegraded":"2024-04-20T21:22:47.356519+0000","last_fullsized":"2024-04-20T21:22:47.356519+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:06:18.941092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.015474+0000","last_change":"2024-04-20T21:22:44.993852+0000","last_active":"2024-04-20T21:22:47.015474+0000","last_peered":"2024-04-20T21:22:47.015474+0000","last_clean":"2024-04-20T21:22:47.015474+0000","last_became_active":"2024-04-20T21:22:44.993491+0000","last_became_peered":"2024-04-20T21:22:44.993491+0000","last_unstale":"2024-04-20T21:22:47.015474+0000","last_undegraded":"2024-04-20T21:22:47.015474+0000","last_fullsized":"2024-04-20T21:22:47.015474+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:07:15.232666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.356646+0000","last_change":"2024-04-20T21:22:46.001218+0000","last_active":"2024-04-20T21:22:47.356646+0000","last_peered":"2024-04-20T21:22:47.356646+0000","last_clean":"2024-04-20T21:22:47.356646+0000","last_became_active":"2024-04-20T21:22:46.000900+0000","last_became_peered":"2024-04-20T21:22:46.000900+0000","last_unstale":"2024-04-20T21:22:47.356646+0000","last_undegraded":"2024-04-20T21:22:47.356646+0000","last_fullsized":"2024-04-20T21:22:47.356646+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:53:21.061854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.994517+0000","last_change":"2024-04-20T21:22:44.992558+0000","last_active":"2024-04-20T21:22:44.994517+0000","last_peered":"2024-04-20T21:22:44.994517+0000","last_clean":"2024-04-20T21:22:44.994517+0000","last_became_active":"2024-04-20T21:22:44.991402+0000","last_became_peered":"2024-04-20T21:22:44.991402+0000","last_unstale":"2024-04-20T21:22:44.994517+0000","last_undegraded":"2024-04-20T21:22:44.994517+0000","last_fullsized":"2024-04-20T21:22:44.994517+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:47:51.476465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.985660+0000","last_change":"2024-04-20T21:22:44.985660+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.985660+0000","last_undegraded":"2024-04-20T21:22:44.985660+0000","last_fullsized":"2024-04-20T21:22:44.985660+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.996911+0000","last_change":"2024-04-20T21:22:44.990230+0000","last_active":"2024-04-20T21:22:44.996911+0000","last_peered":"2024-04-20T21:22:44.996911+0000","last_clean":"2024-04-20T21:22:44.996911+0000","last_became_active":"2024-04-20T21:22:44.989958+0000","last_became_peered":"2024-04-20T21:22:44.989958+0000","last_unstale":"2024-04-20T21:22:44.996911+0000","last_undegraded":"2024-04-20T21:22:44.996911+0000","last_fullsized":"2024-04-20T21:22:44.996911+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:18:22.288371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.015250+0000","last_change":"2024-04-20T21:22:45.993591+0000","last_active":"2024-04-20T21:22:47.015250+0000","last_peered":"2024-04-20T21:22:47.015250+0000","last_clean":"2024-04-20T21:22:47.015250+0000","last_became_active":"2024-04-20T21:22:45.993320+0000","last_became_peered":"2024-04-20T21:22:45.993320+0000","last_unstale":"2024-04-20T21:22:47.015250+0000","last_undegraded":"2024-04-20T21:22:47.015250+0000","last_fullsized":"2024-04-20T21:22:47.015250+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:24:16.195831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"71'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.015880+0000","last_change":"2024-04-20T21:22:44.997554+0000","last_active":"2024-04-20T21:22:47.015880+0000","last_peered":"2024-04-20T21:22:47.015880+0000","last_clean":"2024-04-20T21:22:47.015880+0000","last_became_active":"2024-04-20T21:22:44.997234+0000","last_became_peered":"2024-04-20T21:22:44.997234+0000","last_unstale":"2024-04-20T21:22:47.015880+0000","last_undegraded":"2024-04-20T21:22:47.015880+0000","last_fullsized":"2024-04-20T21:22:47.015880+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:47:51.476465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.986315+0000","last_change":"2024-04-20T21:22:44.986315+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.986315+0000","last_undegraded":"2024-04-20T21:22:44.986315+0000","last_fullsized":"2024-04-20T21:22:44.986315+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.358437+0000","last_change":"2024-04-20T21:22:44.998361+0000","last_active":"2024-04-20T21:22:47.358437+0000","last_peered":"2024-04-20T21:22:47.358437+0000","last_clean":"2024-04-20T21:22:47.358437+0000","last_became_active":"2024-04-20T21:22:44.996313+0000","last_became_peered":"2024-04-20T21:22:44.996313+0000","last_unstale":"2024-04-20T21:22:47.358437+0000","last_undegraded":"2024-04-20T21:22:47.358437+0000","last_fullsized":"2024-04-20T21:22:47.358437+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:57:50.796943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.986997+0000","last_change":"2024-04-20T21:22:44.986997+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.986997+0000","last_undegraded":"2024-04-20T21:22:44.986997+0000","last_fullsized":"2024-04-20T21:22:44.986997+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:45.004224+0000","last_change":"2024-04-20T21:22:45.003069+0000","last_active":"2024-04-20T21:22:45.004224+0000","last_peered":"2024-04-20T21:22:45.004224+0000","last_clean":"2024-04-20T21:22:45.004224+0000","last_became_active":"2024-04-20T21:22:45.002662+0000","last_became_peered":"2024-04-20T21:22:45.002662+0000","last_unstale":"2024-04-20T21:22:45.004224+0000","last_undegraded":"2024-04-20T21:22:45.004224+0000","last_fullsized":"2024-04-20T21:22:45.004224+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:17:48.882110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.357975+0000","last_change":"2024-04-20T21:22:45.998814+0000","last_active":"2024-04-20T21:22:47.357975+0000","last_peered":"2024-04-20T21:22:47.357975+0000","last_clean":"2024-04-20T21:22:47.357975+0000","last_became_active":"2024-04-20T21:22:45.998463+0000","last_became_peered":"2024-04-20T21:22:45.998463+0000","last_unstale":"2024-04-20T21:22:47.357975+0000","last_undegraded":"2024-04-20T21:22:47.357975+0000","last_fullsized":"2024-04-20T21:22:47.357975+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:37:37.094186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.357407+0000","last_change":"2024-04-20T21:22:44.990525+0000","last_active":"2024-04-20T21:22:47.357407+0000","last_peered":"2024-04-20T21:22:47.357407+0000","last_clean":"2024-04-20T21:22:47.357407+0000","last_became_active":"2024-04-20T21:22:44.990259+0000","last_became_peered":"2024-04-20T21:22:44.990259+0000","last_unstale":"2024-04-20T21:22:47.357407+0000","last_undegraded":"2024-04-20T21:22:47.357407+0000","last_fullsized":"2024-04-20T21:22:47.357407+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:07:45.118275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.971783+0000","last_change":"2024-04-20T21:22:44.971783+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.971783+0000","last_undegraded":"2024-04-20T21:22:44.971783+0000","last_fullsized":"2024-04-20T21:22:44.971783+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.995914+0000","last_change":"2024-04-20T21:22:44.996969+0000","last_active":"2024-04-20T21:22:46.995914+0000","last_peered":"2024-04-20T21:22:46.995914+0000","last_clean":"2024-04-20T21:22:46.995914+0000","last_became_active":"2024-04-20T21:22:44.995514+0000","last_became_peered":"2024-04-20T21:22:44.995514+0000","last_unstale":"2024-04-20T21:22:46.995914+0000","last_undegraded":"2024-04-20T21:22:46.995914+0000","last_fullsized":"2024-04-20T21:22:46.995914+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:35:20.490828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.996108+0000","last_change":"2024-04-20T21:22:46.000298+0000","last_active":"2024-04-20T21:22:46.996108+0000","last_peered":"2024-04-20T21:22:46.996108+0000","last_clean":"2024-04-20T21:22:46.996108+0000","last_became_active":"2024-04-20T21:22:46.000027+0000","last_became_peered":"2024-04-20T21:22:46.000027+0000","last_unstale":"2024-04-20T21:22:46.996108+0000","last_undegraded":"2024-04-20T21:22:46.996108+0000","last_fullsized":"2024-04-20T21:22:46.996108+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:53:43.293248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.999123+0000","last_change":"2024-04-20T21:22:44.994701+0000","last_active":"2024-04-20T21:22:46.999123+0000","last_peered":"2024-04-20T21:22:46.999123+0000","last_clean":"2024-04-20T21:22:46.999123+0000","last_became_active":"2024-04-20T21:22:44.994359+0000","last_became_peered":"2024-04-20T21:22:44.994359+0000","last_unstale":"2024-04-20T21:22:46.999123+0000","last_undegraded":"2024-04-20T21:22:46.999123+0000","last_fullsized":"2024-04-20T21:22:46.999123+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:34:52.260354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.973852+0000","last_change":"2024-04-20T21:22:44.973852+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.973852+0000","last_undegraded":"2024-04-20T21:22:44.973852+0000","last_fullsized":"2024-04-20T21:22:44.973852+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.996863+0000","last_change":"2024-04-20T21:22:44.990936+0000","last_active":"2024-04-20T21:22:44.996863+0000","last_peered":"2024-04-20T21:22:44.996863+0000","last_clean":"2024-04-20T21:22:44.996863+0000","last_became_active":"2024-04-20T21:22:44.988494+0000","last_became_peered":"2024-04-20T21:22:44.988494+0000","last_unstale":"2024-04-20T21:22:44.996863+0000","last_undegraded":"2024-04-20T21:22:44.996863+0000","last_fullsized":"2024-04-20T21:22:44.996863+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:40:14.755772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.357272+0000","last_change":"2024-04-20T21:22:46.007192+0000","last_active":"2024-04-20T21:22:47.357272+0000","last_peered":"2024-04-20T21:22:47.357272+0000","last_clean":"2024-04-20T21:22:47.357272+0000","last_became_active":"2024-04-20T21:22:46.006583+0000","last_became_peered":"2024-04-20T21:22:46.006583+0000","last_unstale":"2024-04-20T21:22:47.357272+0000","last_undegraded":"2024-04-20T21:22:47.357272+0000","last_fullsized":"2024-04-20T21:22:47.357272+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:28:45.591146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.993111+0000","last_change":"2024-04-20T21:22:44.988535+0000","last_active":"2024-04-20T21:22:44.993111+0000","last_peered":"2024-04-20T21:22:44.993111+0000","last_clean":"2024-04-20T21:22:44.993111+0000","last_became_active":"2024-04-20T21:22:44.988183+0000","last_became_peered":"2024-04-20T21:22:44.988183+0000","last_unstale":"2024-04-20T21:22:44.993111+0000","last_undegraded":"2024-04-20T21:22:44.993111+0000","last_fullsized":"2024-04-20T21:22:44.993111+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:40:51.727352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":256,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.990279+0000","last_change":"2024-04-20T21:17:43.059637+0000","last_active":"2024-04-20T21:22:44.990279+0000","last_peered":"2024-04-20T21:22:44.990279+0000","last_clean":"2024-04-20T21:22:44.990279+0000","last_became_active":"2024-04-20T21:17:43.056207+0000","last_became_peered":"2024-04-20T21:17:43.056207+0000","last_unstale":"2024-04-20T21:22:44.990279+0000","last_undegraded":"2024-04-20T21:22:44.990279+0000","last_fullsized":"2024-04-20T21:22:44.990279+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:41.036290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:41.036290+0000","last_clean_scrub_stamp":"2024-04-20T21:17:41.036290+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:58:51.091337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":1736,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25,"ondisk_log_size":25,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574911,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32584,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339256,"statfs":{"total":96636764160,"available":96603398144,"internally_reserved":0,"allocated":1433600,"data_stored":941588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574912,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":839680,"data_stored":351220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574915,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":868352,"data_stored":364385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574914,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32036,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339804,"statfs":{"total":96636764160,"available":96603959296,"internally_reserved":0,"allocated":868352,"data_stored":364365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574914,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":872448,"data_stored":364407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574914,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32036,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339804,"statfs":{"total":96636764160,"available":96603959296,"internally_reserved":0,"allocated":876544,"data_stored":365907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32052,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339788,"statfs":{"total":96636764160,"available":96603942912,"internally_reserved":0,"allocated":884736,"data_stored":366523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574913,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32584,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339256,"statfs":{"total":96636764160,"available":96603398144,"internally_reserved":0,"allocated":1433600,"data_stored":941588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T21:22:50.876 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T21:22:51.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.111+0000 7f5d68c00640 1 Processor -- start 2024-04-20T21:22:51.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.112+0000 7f5d68c00640 1 -- start start 2024-04-20T21:22:51.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.113+0000 7f5d68c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5d6406e6d0 0x7f5d64071d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:51.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.113+0000 7f5d68c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5d64072240 0x7f5d6415cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:51.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.113+0000 7f5d68c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d6406eb30 con 0x7f5d64146130 2024-04-20T21:22:51.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.113+0000 7f5d68c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f5d64072690 con 0x7f5d64072240 2024-04-20T21:22:51.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.113+0000 7f5d68c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f5d6406dc30 con 0x7f5d6406e6d0 2024-04-20T21:22:51.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.113+0000 7f5d62800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5d64072240 0x7f5d6415cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:51.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.113+0000 7f5d62800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5d64072240 0x7f5d6415cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52080/0 (socket says 172.21.15.86:52080) 2024-04-20T21:22:51.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.113+0000 7f5d62800640 1 -- 172.21.15.86:0/2053497701 learned_addr learned my addr 172.21.15.86:0/2053497701 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:51.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.114+0000 7f5d62800640 1 -- 172.21.15.86:0/2053497701 >> v2:172.21.15.144:3300/0 conn(0x7f5d6406e6d0 msgr2=0x7f5d64071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:51.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.114+0000 7f5d61400640 1 --2- 172.21.15.86:0/2053497701 >> v2:172.21.15.144:3300/0 conn(0x7f5d6406e6d0 0x7f5d64071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:51.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.114+0000 7f5d62800640 1 --2- 172.21.15.86:0/2053497701 >> v2:172.21.15.144:3300/0 conn(0x7f5d6406e6d0 0x7f5d64071d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:51.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.114+0000 7f5d62800640 1 -- 172.21.15.86:0/2053497701 >> v1:172.21.15.144:6789/0 conn(0x7f5d64146130 legacy=0x7f5d640ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:51.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.114+0000 7f5d62800640 1 -- 172.21.15.86:0/2053497701 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d640cd8e0 con 0x7f5d64072240 2024-04-20T21:22:51.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.114+0000 7f5d61400640 1 --2- 172.21.15.86:0/2053497701 >> v2:172.21.15.144:3300/0 conn(0x7f5d6406e6d0 0x7f5d64071d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:22:51.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.115+0000 7f5d62800640 1 --2- 172.21.15.86:0/2053497701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d64072240 0x7f5d6415cc20 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f5d64073120 tx=0x7f5d5c02bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90e698a0e2c4e1b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:51.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.115+0000 7f5d60a00640 1 -- 172.21.15.86:0/2053497701 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c02cb10 con 0x7f5d64072240 2024-04-20T21:22:51.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.115+0000 7f5d60a00640 1 -- 172.21.15.86:0/2053497701 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d5c02ccb0 con 0x7f5d64072240 2024-04-20T21:22:51.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.115+0000 7f5d60a00640 1 -- 172.21.15.86:0/2053497701 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c02ce60 con 0x7f5d64072240 2024-04-20T21:22:51.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.116+0000 7f5d68c00640 1 -- 172.21.15.86:0/2053497701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d64072240 msgr2=0x7f5d6415cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:51.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.116+0000 7f5d68c00640 1 --2- 172.21.15.86:0/2053497701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d64072240 0x7f5d6415cc20 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f5d64073120 tx=0x7f5d5c02bf70 comp rx=0 tx=0).stop 2024-04-20T21:22:51.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.116+0000 7f5d68c00640 1 -- 172.21.15.86:0/2053497701 shutdown_connections 2024-04-20T21:22:51.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.116+0000 7f5d68c00640 1 --2- 172.21.15.86:0/2053497701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d64072240 0x7f5d6415cc20 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:51.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.116+0000 7f5d68c00640 1 --2- 172.21.15.86:0/2053497701 >> v2:172.21.15.144:3300/0 conn(0x7f5d6406e6d0 0x7f5d64071d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:51.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.116+0000 7f5d68c00640 1 -- 172.21.15.86:0/2053497701 >> 172.21.15.86:0/2053497701 conn(0x7f5d640b2360 msgr2=0x7f5d640b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:51.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.116+0000 7f5d68c00640 1 -- 172.21.15.86:0/2053497701 shutdown_connections 2024-04-20T21:22:51.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.116+0000 7f5d68c00640 1 -- 172.21.15.86:0/2053497701 wait complete. 2024-04-20T21:22:51.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.118+0000 7f5d68c00640 1 Processor -- start 2024-04-20T21:22:51.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.118+0000 7f5d68c00640 1 -- start start 2024-04-20T21:22:51.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.119+0000 7f5d68c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d6406e6d0 0x7f5d6415bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:51.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.119+0000 7f5d68c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d64072240 0x7f5d641e7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:51.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.119+0000 7f5d68c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d64146130 0x7f5d641e6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:51.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.119+0000 7f5d68c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6406e1b0 con 0x7f5d64146130 2024-04-20T21:22:51.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.119+0000 7f5d68c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6406ded0 con 0x7f5d64072240 2024-04-20T21:22:51.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.119+0000 7f5d68c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5d6406f150 con 0x7f5d6406e6d0 2024-04-20T21:22:51.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.119+0000 7f5d62800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d64146130 0x7f5d641e6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:51.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.119+0000 7f5d62800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d64146130 0x7f5d641e6ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52090/0 (socket says 172.21.15.86:52090) 2024-04-20T21:22:51.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.119+0000 7f5d62800640 1 -- 172.21.15.86:0/1458584775 learned_addr learned my addr 172.21.15.86:0/1458584775 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:51.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.119+0000 7f5d61400640 1 --2- 172.21.15.86:0/1458584775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d64072240 0x7f5d641e7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:51.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.120+0000 7f5d62800640 1 -- 172.21.15.86:0/1458584775 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d6406e6d0 msgr2=0x7f5d6415bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:51.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.120+0000 7f5d61e00640 1 --2- 172.21.15.86:0/1458584775 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d6406e6d0 0x7f5d6415bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:51.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.120+0000 7f5d62800640 1 --2- 172.21.15.86:0/1458584775 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d6406e6d0 0x7f5d6415bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:51.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.120+0000 7f5d62800640 1 -- 172.21.15.86:0/1458584775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d64072240 msgr2=0x7f5d641e7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:51.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.120+0000 7f5d62800640 1 --2- 172.21.15.86:0/1458584775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d64072240 0x7f5d641e7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:51.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.120+0000 7f5d62800640 1 -- 172.21.15.86:0/1458584775 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d6415c2d0 con 0x7f5d64146130 2024-04-20T21:22:51.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.120+0000 7f5d62800640 1 --2- 172.21.15.86:0/1458584775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d64146130 0x7f5d641e6ea0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c02c580 tx=0x7f5d5c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:51.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.120+0000 7f5d5a000640 1 -- 172.21.15.86:0/1458584775 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c043020 con 0x7f5d64146130 2024-04-20T21:22:51.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.120+0000 7f5d5a000640 1 -- 172.21.15.86:0/1458584775 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d5c03e450 con 0x7f5d64146130 2024-04-20T21:22:51.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.120+0000 7f5d5a000640 1 -- 172.21.15.86:0/1458584775 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c02ce60 con 0x7f5d64146130 2024-04-20T21:22:51.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.120+0000 7f5d68c00640 1 -- 172.21.15.86:0/1458584775 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d641e5e00 con 0x7f5d64146130 2024-04-20T21:22:51.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.121+0000 7f5d68c00640 1 -- 172.21.15.86:0/1458584775 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d641e63c0 con 0x7f5d64146130 2024-04-20T21:22:51.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.122+0000 7f5d68c00640 1 -- 172.21.15.86:0/1458584775 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d640689e0 con 0x7f5d64146130 2024-04-20T21:22:51.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.123+0000 7f5d5a000640 1 -- 172.21.15.86:0/1458584775 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5d5c005d10 con 0x7f5d64146130 2024-04-20T21:22:51.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.124+0000 7f5d5a000640 1 --2- 172.21.15.86:0/1458584775 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5d4006c8f0 0x7f5d4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:51.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.124+0000 7f5d5a000640 1 -- 172.21.15.86:0/1458584775 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f5d5c0ae1e0 con 0x7f5d64146130 2024-04-20T21:22:51.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.124+0000 7f5d61e00640 1 --2- 172.21.15.86:0/1458584775 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5d4006c8f0 0x7f5d4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:51.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.124+0000 7f5d61e00640 1 --2- 172.21.15.86:0/1458584775 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5d4006c8f0 0x7f5d4006edb0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f5d54003d90 tx=0x7f5d54009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:51.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.128+0000 7f5d5a000640 1 -- 172.21.15.86:0/1458584775 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5d6406eb30 con 0x7f5d64146130 2024-04-20T21:22:51.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.324+0000 7f5d68c00640 1 -- 172.21.15.86:0/1458584775 --> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5d6415ff50 con 0x7f5d4006c8f0 2024-04-20T21:22:51.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.341+0000 7f5d5a000640 1 -- 172.21.15.86:0/1458584775 <== mgr.4107 v2:172.21.15.86:6832/302045240 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249925 (secure 0 0 0) 0x7f5d6415ff50 con 0x7f5d4006c8f0 2024-04-20T21:22:51.344 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:22:51.345 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-20T21:22:51.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.347+0000 7f5d68c00640 1 -- 172.21.15.86:0/1458584775 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5d4006c8f0 msgr2=0x7f5d4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:51.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.347+0000 7f5d68c00640 1 --2- 172.21.15.86:0/1458584775 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5d4006c8f0 0x7f5d4006edb0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f5d54003d90 tx=0x7f5d54009040 comp rx=0 tx=0).stop 2024-04-20T21:22:51.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.347+0000 7f5d68c00640 1 -- 172.21.15.86:0/1458584775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d64146130 msgr2=0x7f5d641e6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:51.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.347+0000 7f5d68c00640 1 --2- 172.21.15.86:0/1458584775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d64146130 0x7f5d641e6ea0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c02c580 tx=0x7f5d5c004a40 comp rx=0 tx=0).stop 2024-04-20T21:22:51.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.347+0000 7f5d68c00640 1 -- 172.21.15.86:0/1458584775 shutdown_connections 2024-04-20T21:22:51.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.347+0000 7f5d68c00640 1 --2- 172.21.15.86:0/1458584775 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5d4006c8f0 0x7f5d4006edb0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:51.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.347+0000 7f5d68c00640 1 --2- 172.21.15.86:0/1458584775 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d64146130 0x7f5d641e6ea0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:51.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.347+0000 7f5d68c00640 1 --2- 172.21.15.86:0/1458584775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d64072240 0x7f5d641e7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:51.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.347+0000 7f5d68c00640 1 --2- 172.21.15.86:0/1458584775 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d6406e6d0 0x7f5d6415bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:51.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.347+0000 7f5d68c00640 1 -- 172.21.15.86:0/1458584775 >> 172.21.15.86:0/1458584775 conn(0x7f5d640b2360 msgr2=0x7f5d640b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:51.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.347+0000 7f5d68c00640 1 -- 172.21.15.86:0/1458584775 shutdown_connections 2024-04-20T21:22:51.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.347+0000 7f5d68c00640 1 -- 172.21.15.86:0/1458584775 wait complete. 2024-04-20T21:22:51.365 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":219,"stamp":"2024-04-20T21:22:50.732372+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592104,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":191,"num_write_kb":2427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":128,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257356,"kb_used_data":7888,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754717364,"statfs":{"total":773094113280,"available":772830580736,"internally_reserved":0,"allocated":8077312,"data_stored":4059983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1736,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001508"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.989723+0000","last_change":"2024-04-20T21:22:44.989723+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.989723+0000","last_undegraded":"2024-04-20T21:22:44.989723+0000","last_fullsized":"2024-04-20T21:22:44.989723+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.972721+0000","last_change":"2024-04-20T21:22:44.972721+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.972721+0000","last_undegraded":"2024-04-20T21:22:44.972721+0000","last_fullsized":"2024-04-20T21:22:44.972721+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.358114+0000","last_change":"2024-04-20T21:22:46.000461+0000","last_active":"2024-04-20T21:22:47.358114+0000","last_peered":"2024-04-20T21:22:47.358114+0000","last_clean":"2024-04-20T21:22:47.358114+0000","last_became_active":"2024-04-20T21:22:46.000188+0000","last_became_peered":"2024-04-20T21:22:46.000188+0000","last_unstale":"2024-04-20T21:22:47.358114+0000","last_undegraded":"2024-04-20T21:22:47.358114+0000","last_fullsized":"2024-04-20T21:22:47.358114+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:27:38.679422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.996119+0000","last_change":"2024-04-20T21:22:45.996176+0000","last_active":"2024-04-20T21:22:46.996119+0000","last_peered":"2024-04-20T21:22:46.996119+0000","last_clean":"2024-04-20T21:22:46.996119+0000","last_became_active":"2024-04-20T21:22:45.995860+0000","last_became_peered":"2024-04-20T21:22:45.995860+0000","last_unstale":"2024-04-20T21:22:46.996119+0000","last_undegraded":"2024-04-20T21:22:46.996119+0000","last_fullsized":"2024-04-20T21:22:46.996119+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:19:56.685552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.357267+0000","last_change":"2024-04-20T21:22:46.003219+0000","last_active":"2024-04-20T21:22:47.357267+0000","last_peered":"2024-04-20T21:22:47.357267+0000","last_clean":"2024-04-20T21:22:47.357267+0000","last_became_active":"2024-04-20T21:22:46.002927+0000","last_became_peered":"2024-04-20T21:22:46.002927+0000","last_unstale":"2024-04-20T21:22:47.357267+0000","last_undegraded":"2024-04-20T21:22:47.357267+0000","last_fullsized":"2024-04-20T21:22:47.357267+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:18:23.304303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.976258+0000","last_change":"2024-04-20T21:22:44.976258+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.976258+0000","last_undegraded":"2024-04-20T21:22:44.976258+0000","last_fullsized":"2024-04-20T21:22:44.976258+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.014778+0000","last_change":"2024-04-20T21:22:45.994406+0000","last_active":"2024-04-20T21:22:47.014778+0000","last_peered":"2024-04-20T21:22:47.014778+0000","last_clean":"2024-04-20T21:22:47.014778+0000","last_became_active":"2024-04-20T21:22:45.994109+0000","last_became_peered":"2024-04-20T21:22:45.994109+0000","last_unstale":"2024-04-20T21:22:47.014778+0000","last_undegraded":"2024-04-20T21:22:47.014778+0000","last_fullsized":"2024-04-20T21:22:47.014778+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:06:06.356878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.971878+0000","last_change":"2024-04-20T21:22:44.971878+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.971878+0000","last_undegraded":"2024-04-20T21:22:44.971878+0000","last_fullsized":"2024-04-20T21:22:44.971878+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.358759+0000","last_change":"2024-04-20T21:22:46.009487+0000","last_active":"2024-04-20T21:22:47.358759+0000","last_peered":"2024-04-20T21:22:47.358759+0000","last_clean":"2024-04-20T21:22:47.358759+0000","last_became_active":"2024-04-20T21:22:46.008835+0000","last_became_peered":"2024-04-20T21:22:46.008835+0000","last_unstale":"2024-04-20T21:22:47.358759+0000","last_undegraded":"2024-04-20T21:22:47.358759+0000","last_fullsized":"2024-04-20T21:22:47.358759+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:19:35.848351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.001975+0000","last_change":"2024-04-20T21:22:46.008121+0000","last_active":"2024-04-20T21:22:47.001975+0000","last_peered":"2024-04-20T21:22:47.001975+0000","last_clean":"2024-04-20T21:22:47.001975+0000","last_became_active":"2024-04-20T21:22:46.007835+0000","last_became_peered":"2024-04-20T21:22:46.007835+0000","last_unstale":"2024-04-20T21:22:47.001975+0000","last_undegraded":"2024-04-20T21:22:47.001975+0000","last_fullsized":"2024-04-20T21:22:47.001975+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:42:09.491911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.989195+0000","last_change":"2024-04-20T21:22:44.989195+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.989195+0000","last_undegraded":"2024-04-20T21:22:44.989195+0000","last_fullsized":"2024-04-20T21:22:44.989195+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.996166+0000","last_change":"2024-04-20T21:22:46.005658+0000","last_active":"2024-04-20T21:22:46.996166+0000","last_peered":"2024-04-20T21:22:46.996166+0000","last_clean":"2024-04-20T21:22:46.996166+0000","last_became_active":"2024-04-20T21:22:46.005346+0000","last_became_peered":"2024-04-20T21:22:46.005346+0000","last_unstale":"2024-04-20T21:22:46.996166+0000","last_undegraded":"2024-04-20T21:22:46.996166+0000","last_fullsized":"2024-04-20T21:22:46.996166+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:35:49.635636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.983752+0000","last_change":"2024-04-20T21:22:44.983752+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.983752+0000","last_undegraded":"2024-04-20T21:22:44.983752+0000","last_fullsized":"2024-04-20T21:22:44.983752+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.357690+0000","last_change":"2024-04-20T21:22:46.001211+0000","last_active":"2024-04-20T21:22:47.357690+0000","last_peered":"2024-04-20T21:22:47.357690+0000","last_clean":"2024-04-20T21:22:47.357690+0000","last_became_active":"2024-04-20T21:22:46.000899+0000","last_became_peered":"2024-04-20T21:22:46.000899+0000","last_unstale":"2024-04-20T21:22:47.357690+0000","last_undegraded":"2024-04-20T21:22:47.357690+0000","last_fullsized":"2024-04-20T21:22:47.357690+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:08:57.585935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.984365+0000","last_change":"2024-04-20T21:22:44.984365+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.984365+0000","last_undegraded":"2024-04-20T21:22:44.984365+0000","last_fullsized":"2024-04-20T21:22:44.984365+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.999077+0000","last_change":"2024-04-20T21:22:45.998551+0000","last_active":"2024-04-20T21:22:46.999077+0000","last_peered":"2024-04-20T21:22:46.999077+0000","last_clean":"2024-04-20T21:22:46.999077+0000","last_became_active":"2024-04-20T21:22:45.996213+0000","last_became_peered":"2024-04-20T21:22:45.996213+0000","last_unstale":"2024-04-20T21:22:46.999077+0000","last_undegraded":"2024-04-20T21:22:46.999077+0000","last_fullsized":"2024-04-20T21:22:46.999077+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:49:36.482159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.358766+0000","last_change":"2024-04-20T21:22:46.013340+0000","last_active":"2024-04-20T21:22:47.358766+0000","last_peered":"2024-04-20T21:22:47.358766+0000","last_clean":"2024-04-20T21:22:47.358766+0000","last_became_active":"2024-04-20T21:22:46.009224+0000","last_became_peered":"2024-04-20T21:22:46.009224+0000","last_unstale":"2024-04-20T21:22:47.358766+0000","last_undegraded":"2024-04-20T21:22:47.358766+0000","last_fullsized":"2024-04-20T21:22:47.358766+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:33:18.596222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.996803+0000","last_change":"2024-04-20T21:22:45.987458+0000","last_active":"2024-04-20T21:22:46.996803+0000","last_peered":"2024-04-20T21:22:46.996803+0000","last_clean":"2024-04-20T21:22:46.996803+0000","last_became_active":"2024-04-20T21:22:45.987046+0000","last_became_peered":"2024-04-20T21:22:45.987046+0000","last_unstale":"2024-04-20T21:22:46.996803+0000","last_undegraded":"2024-04-20T21:22:46.996803+0000","last_fullsized":"2024-04-20T21:22:46.996803+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:09:41.573665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.978703+0000","last_change":"2024-04-20T21:22:44.978703+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.978703+0000","last_undegraded":"2024-04-20T21:22:44.978703+0000","last_fullsized":"2024-04-20T21:22:44.978703+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.986040+0000","last_change":"2024-04-20T21:22:44.986040+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.986040+0000","last_undegraded":"2024-04-20T21:22:44.986040+0000","last_fullsized":"2024-04-20T21:22:44.986040+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.000249+0000","last_change":"2024-04-20T21:22:45.998328+0000","last_active":"2024-04-20T21:22:47.000249+0000","last_peered":"2024-04-20T21:22:47.000249+0000","last_clean":"2024-04-20T21:22:47.000249+0000","last_became_active":"2024-04-20T21:22:45.998016+0000","last_became_peered":"2024-04-20T21:22:45.998016+0000","last_unstale":"2024-04-20T21:22:47.000249+0000","last_undegraded":"2024-04-20T21:22:47.000249+0000","last_fullsized":"2024-04-20T21:22:47.000249+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:53:42.792540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.999681+0000","last_change":"2024-04-20T21:22:46.006835+0000","last_active":"2024-04-20T21:22:46.999681+0000","last_peered":"2024-04-20T21:22:46.999681+0000","last_clean":"2024-04-20T21:22:46.999681+0000","last_became_active":"2024-04-20T21:22:46.006544+0000","last_became_peered":"2024-04-20T21:22:46.006544+0000","last_unstale":"2024-04-20T21:22:46.999681+0000","last_undegraded":"2024-04-20T21:22:46.999681+0000","last_fullsized":"2024-04-20T21:22:46.999681+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:05:10.087962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.985232+0000","last_change":"2024-04-20T21:22:44.985232+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.985232+0000","last_undegraded":"2024-04-20T21:22:44.985232+0000","last_fullsized":"2024-04-20T21:22:44.985232+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.983826+0000","last_change":"2024-04-20T21:22:44.983826+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.983826+0000","last_undegraded":"2024-04-20T21:22:44.983826+0000","last_fullsized":"2024-04-20T21:22:44.983826+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.001458+0000","last_change":"2024-04-20T21:22:46.009435+0000","last_active":"2024-04-20T21:22:47.001458+0000","last_peered":"2024-04-20T21:22:47.001458+0000","last_clean":"2024-04-20T21:22:47.001458+0000","last_became_active":"2024-04-20T21:22:46.008653+0000","last_became_peered":"2024-04-20T21:22:46.008653+0000","last_unstale":"2024-04-20T21:22:47.001458+0000","last_undegraded":"2024-04-20T21:22:47.001458+0000","last_fullsized":"2024-04-20T21:22:47.001458+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:58:27.202522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.015568+0000","last_change":"2024-04-20T21:22:45.999532+0000","last_active":"2024-04-20T21:22:47.015568+0000","last_peered":"2024-04-20T21:22:47.015568+0000","last_clean":"2024-04-20T21:22:47.015568+0000","last_became_active":"2024-04-20T21:22:45.999223+0000","last_became_peered":"2024-04-20T21:22:45.999223+0000","last_unstale":"2024-04-20T21:22:47.015568+0000","last_undegraded":"2024-04-20T21:22:47.015568+0000","last_fullsized":"2024-04-20T21:22:47.015568+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:55:14.180769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.358026+0000","last_change":"2024-04-20T21:22:46.002322+0000","last_active":"2024-04-20T21:22:47.358026+0000","last_peered":"2024-04-20T21:22:47.358026+0000","last_clean":"2024-04-20T21:22:47.358026+0000","last_became_active":"2024-04-20T21:22:46.002035+0000","last_became_peered":"2024-04-20T21:22:46.002035+0000","last_unstale":"2024-04-20T21:22:47.358026+0000","last_undegraded":"2024-04-20T21:22:47.358026+0000","last_fullsized":"2024-04-20T21:22:47.358026+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:47:00.839639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.001277+0000","last_change":"2024-04-20T21:22:46.006624+0000","last_active":"2024-04-20T21:22:47.001277+0000","last_peered":"2024-04-20T21:22:47.001277+0000","last_clean":"2024-04-20T21:22:47.001277+0000","last_became_active":"2024-04-20T21:22:46.006273+0000","last_became_peered":"2024-04-20T21:22:46.006273+0000","last_unstale":"2024-04-20T21:22:47.001277+0000","last_undegraded":"2024-04-20T21:22:47.001277+0000","last_fullsized":"2024-04-20T21:22:47.001277+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:06:06.356878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.357002+0000","last_change":"2024-04-20T21:22:45.998542+0000","last_active":"2024-04-20T21:22:47.357002+0000","last_peered":"2024-04-20T21:22:47.357002+0000","last_clean":"2024-04-20T21:22:47.357002+0000","last_became_active":"2024-04-20T21:22:45.998274+0000","last_became_peered":"2024-04-20T21:22:45.998274+0000","last_unstale":"2024-04-20T21:22:47.357002+0000","last_undegraded":"2024-04-20T21:22:47.357002+0000","last_fullsized":"2024-04-20T21:22:47.357002+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:17:21.685206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.014934+0000","last_change":"2024-04-20T21:22:45.995816+0000","last_active":"2024-04-20T21:22:47.014934+0000","last_peered":"2024-04-20T21:22:47.014934+0000","last_clean":"2024-04-20T21:22:47.014934+0000","last_became_active":"2024-04-20T21:22:45.994917+0000","last_became_peered":"2024-04-20T21:22:45.994917+0000","last_unstale":"2024-04-20T21:22:47.014934+0000","last_undegraded":"2024-04-20T21:22:47.014934+0000","last_fullsized":"2024-04-20T21:22:47.014934+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:53:17.849382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.015118+0000","last_change":"2024-04-20T21:22:46.001790+0000","last_active":"2024-04-20T21:22:47.015118+0000","last_peered":"2024-04-20T21:22:47.015118+0000","last_clean":"2024-04-20T21:22:47.015118+0000","last_became_active":"2024-04-20T21:22:46.001494+0000","last_became_peered":"2024-04-20T21:22:46.001494+0000","last_unstale":"2024-04-20T21:22:47.015118+0000","last_undegraded":"2024-04-20T21:22:47.015118+0000","last_fullsized":"2024-04-20T21:22:47.015118+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:42:09.491911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.970617+0000","last_change":"2024-04-20T21:22:44.970617+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.970617+0000","last_undegraded":"2024-04-20T21:22:44.970617+0000","last_fullsized":"2024-04-20T21:22:44.970617+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.982545+0000","last_change":"2024-04-20T21:22:44.982545+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.982545+0000","last_undegraded":"2024-04-20T21:22:44.982545+0000","last_fullsized":"2024-04-20T21:22:44.982545+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:45.004990+0000","last_change":"2024-04-20T21:22:45.000877+0000","last_active":"2024-04-20T21:22:45.004990+0000","last_peered":"2024-04-20T21:22:45.004990+0000","last_clean":"2024-04-20T21:22:45.004990+0000","last_became_active":"2024-04-20T21:22:45.000402+0000","last_became_peered":"2024-04-20T21:22:45.000402+0000","last_unstale":"2024-04-20T21:22:45.004990+0000","last_undegraded":"2024-04-20T21:22:45.004990+0000","last_fullsized":"2024-04-20T21:22:45.004990+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:56:13.768402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.358957+0000","last_change":"2024-04-20T21:22:46.007646+0000","last_active":"2024-04-20T21:22:47.358957+0000","last_peered":"2024-04-20T21:22:47.358957+0000","last_clean":"2024-04-20T21:22:47.358957+0000","last_became_active":"2024-04-20T21:22:46.007304+0000","last_became_peered":"2024-04-20T21:22:46.007304+0000","last_unstale":"2024-04-20T21:22:47.358957+0000","last_undegraded":"2024-04-20T21:22:47.358957+0000","last_fullsized":"2024-04-20T21:22:47.358957+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:45:53.505415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.999915+0000","last_change":"2024-04-20T21:22:44.998432+0000","last_active":"2024-04-20T21:22:44.999915+0000","last_peered":"2024-04-20T21:22:44.999915+0000","last_clean":"2024-04-20T21:22:44.999915+0000","last_became_active":"2024-04-20T21:22:44.998006+0000","last_became_peered":"2024-04-20T21:22:44.998006+0000","last_unstale":"2024-04-20T21:22:44.999915+0000","last_undegraded":"2024-04-20T21:22:44.999915+0000","last_fullsized":"2024-04-20T21:22:44.999915+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:53:04.669954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.979534+0000","last_change":"2024-04-20T21:22:44.979534+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.979534+0000","last_undegraded":"2024-04-20T21:22:44.979534+0000","last_fullsized":"2024-04-20T21:22:44.979534+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"70'2","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.001646+0000","last_change":"2024-04-20T21:22:44.997985+0000","last_active":"2024-04-20T21:22:47.001646+0000","last_peered":"2024-04-20T21:22:47.001646+0000","last_clean":"2024-04-20T21:22:47.001646+0000","last_became_active":"2024-04-20T21:22:44.997672+0000","last_became_peered":"2024-04-20T21:22:44.997672+0000","last_unstale":"2024-04-20T21:22:47.001646+0000","last_undegraded":"2024-04-20T21:22:47.001646+0000","last_fullsized":"2024-04-20T21:22:47.001646+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:10:01.129225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.988718+0000","last_change":"2024-04-20T21:22:44.988718+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.988718+0000","last_undegraded":"2024-04-20T21:22:44.988718+0000","last_fullsized":"2024-04-20T21:22:44.988718+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.996650+0000","last_change":"2024-04-20T21:22:44.989343+0000","last_active":"2024-04-20T21:22:44.996650+0000","last_peered":"2024-04-20T21:22:44.996650+0000","last_clean":"2024-04-20T21:22:44.996650+0000","last_became_active":"2024-04-20T21:22:44.989022+0000","last_became_peered":"2024-04-20T21:22:44.989022+0000","last_unstale":"2024-04-20T21:22:44.996650+0000","last_undegraded":"2024-04-20T21:22:44.996650+0000","last_fullsized":"2024-04-20T21:22:44.996650+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:45:48.879276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.014909+0000","last_change":"2024-04-20T21:22:46.004052+0000","last_active":"2024-04-20T21:22:47.014909+0000","last_peered":"2024-04-20T21:22:47.014909+0000","last_clean":"2024-04-20T21:22:47.014909+0000","last_became_active":"2024-04-20T21:22:46.002637+0000","last_became_peered":"2024-04-20T21:22:46.002637+0000","last_unstale":"2024-04-20T21:22:47.014909+0000","last_undegraded":"2024-04-20T21:22:47.014909+0000","last_fullsized":"2024-04-20T21:22:47.014909+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:58:27.202522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"70'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.357559+0000","last_change":"2024-04-20T21:22:44.994866+0000","last_active":"2024-04-20T21:22:47.357559+0000","last_peered":"2024-04-20T21:22:47.357559+0000","last_clean":"2024-04-20T21:22:47.357559+0000","last_became_active":"2024-04-20T21:22:44.994089+0000","last_became_peered":"2024-04-20T21:22:44.994089+0000","last_unstale":"2024-04-20T21:22:47.357559+0000","last_undegraded":"2024-04-20T21:22:47.357559+0000","last_fullsized":"2024-04-20T21:22:47.357559+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:42:08.475979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.000407+0000","last_change":"2024-04-20T21:22:46.008256+0000","last_active":"2024-04-20T21:22:47.000407+0000","last_peered":"2024-04-20T21:22:47.000407+0000","last_clean":"2024-04-20T21:22:47.000407+0000","last_became_active":"2024-04-20T21:22:46.007960+0000","last_became_peered":"2024-04-20T21:22:46.007960+0000","last_unstale":"2024-04-20T21:22:47.000407+0000","last_undegraded":"2024-04-20T21:22:47.000407+0000","last_fullsized":"2024-04-20T21:22:47.000407+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:28:56.572056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.999221+0000","last_change":"2024-04-20T21:22:44.995756+0000","last_active":"2024-04-20T21:22:44.999221+0000","last_peered":"2024-04-20T21:22:44.999221+0000","last_clean":"2024-04-20T21:22:44.999221+0000","last_became_active":"2024-04-20T21:22:44.995451+0000","last_became_peered":"2024-04-20T21:22:44.995451+0000","last_unstale":"2024-04-20T21:22:44.999221+0000","last_undegraded":"2024-04-20T21:22:44.999221+0000","last_fullsized":"2024-04-20T21:22:44.999221+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:37:36.078254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.356960+0000","last_change":"2024-04-20T21:22:46.004215+0000","last_active":"2024-04-20T21:22:47.356960+0000","last_peered":"2024-04-20T21:22:47.356960+0000","last_clean":"2024-04-20T21:22:47.356960+0000","last_became_active":"2024-04-20T21:22:46.003804+0000","last_became_peered":"2024-04-20T21:22:46.003804+0000","last_unstale":"2024-04-20T21:22:47.356960+0000","last_undegraded":"2024-04-20T21:22:47.356960+0000","last_fullsized":"2024-04-20T21:22:47.356960+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:45:49.895208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.995960+0000","last_change":"2024-04-20T21:22:44.996824+0000","last_active":"2024-04-20T21:22:46.995960+0000","last_peered":"2024-04-20T21:22:46.995960+0000","last_clean":"2024-04-20T21:22:46.995960+0000","last_became_active":"2024-04-20T21:22:44.996531+0000","last_became_peered":"2024-04-20T21:22:44.996531+0000","last_unstale":"2024-04-20T21:22:46.995960+0000","last_undegraded":"2024-04-20T21:22:46.995960+0000","last_fullsized":"2024-04-20T21:22:46.995960+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:01:19.361992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.983784+0000","last_change":"2024-04-20T21:22:44.983784+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.983784+0000","last_undegraded":"2024-04-20T21:22:44.983784+0000","last_fullsized":"2024-04-20T21:22:44.983784+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.356944+0000","last_change":"2024-04-20T21:22:44.982668+0000","last_active":"2024-04-20T21:22:47.356944+0000","last_peered":"2024-04-20T21:22:47.356944+0000","last_clean":"2024-04-20T21:22:47.356944+0000","last_became_active":"2024-04-20T21:22:44.982035+0000","last_became_peered":"2024-04-20T21:22:44.982035+0000","last_unstale":"2024-04-20T21:22:47.356944+0000","last_undegraded":"2024-04-20T21:22:47.356944+0000","last_fullsized":"2024-04-20T21:22:47.356944+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:04:12.006293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.015640+0000","last_change":"2024-04-20T21:22:46.005753+0000","last_active":"2024-04-20T21:22:47.015640+0000","last_peered":"2024-04-20T21:22:47.015640+0000","last_clean":"2024-04-20T21:22:47.015640+0000","last_became_active":"2024-04-20T21:22:46.005305+0000","last_became_peered":"2024-04-20T21:22:46.005305+0000","last_unstale":"2024-04-20T21:22:47.015640+0000","last_undegraded":"2024-04-20T21:22:47.015640+0000","last_fullsized":"2024-04-20T21:22:47.015640+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:27:57.913146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.001557+0000","last_change":"2024-04-20T21:22:44.994181+0000","last_active":"2024-04-20T21:22:47.001557+0000","last_peered":"2024-04-20T21:22:47.001557+0000","last_clean":"2024-04-20T21:22:47.001557+0000","last_became_active":"2024-04-20T21:22:44.992861+0000","last_became_peered":"2024-04-20T21:22:44.992861+0000","last_unstale":"2024-04-20T21:22:47.001557+0000","last_undegraded":"2024-04-20T21:22:47.001557+0000","last_fullsized":"2024-04-20T21:22:47.001557+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:52:23.085259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.988271+0000","last_change":"2024-04-20T21:22:44.988271+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.988271+0000","last_undegraded":"2024-04-20T21:22:44.988271+0000","last_fullsized":"2024-04-20T21:22:44.988271+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.999321+0000","last_change":"2024-04-20T21:22:44.992156+0000","last_active":"2024-04-20T21:22:44.999321+0000","last_peered":"2024-04-20T21:22:44.999321+0000","last_clean":"2024-04-20T21:22:44.999321+0000","last_became_active":"2024-04-20T21:22:44.991749+0000","last_became_peered":"2024-04-20T21:22:44.991749+0000","last_unstale":"2024-04-20T21:22:44.999321+0000","last_undegraded":"2024-04-20T21:22:44.999321+0000","last_fullsized":"2024-04-20T21:22:44.999321+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:59:10.369089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.977417+0000","last_change":"2024-04-20T21:22:44.977417+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.977417+0000","last_undegraded":"2024-04-20T21:22:44.977417+0000","last_fullsized":"2024-04-20T21:22:44.977417+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"70'3","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.358192+0000","last_change":"2024-04-20T21:22:44.990563+0000","last_active":"2024-04-20T21:22:47.358192+0000","last_peered":"2024-04-20T21:22:47.358192+0000","last_clean":"2024-04-20T21:22:47.358192+0000","last_became_active":"2024-04-20T21:22:44.990292+0000","last_became_peered":"2024-04-20T21:22:44.990292+0000","last_unstale":"2024-04-20T21:22:47.358192+0000","last_undegraded":"2024-04-20T21:22:47.358192+0000","last_fullsized":"2024-04-20T21:22:47.358192+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:55:37.925344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.357584+0000","last_change":"2024-04-20T21:22:45.997098+0000","last_active":"2024-04-20T21:22:47.357584+0000","last_peered":"2024-04-20T21:22:47.357584+0000","last_clean":"2024-04-20T21:22:47.357584+0000","last_became_active":"2024-04-20T21:22:45.996820+0000","last_became_peered":"2024-04-20T21:22:45.996820+0000","last_unstale":"2024-04-20T21:22:47.357584+0000","last_undegraded":"2024-04-20T21:22:47.357584+0000","last_fullsized":"2024-04-20T21:22:47.357584+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:59:11.385021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"70'6","reported_seq":20,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.996263+0000","last_change":"2024-04-20T21:22:44.988227+0000","last_active":"2024-04-20T21:22:46.996263+0000","last_peered":"2024-04-20T21:22:46.996263+0000","last_clean":"2024-04-20T21:22:46.996263+0000","last_became_active":"2024-04-20T21:22:44.987939+0000","last_became_peered":"2024-04-20T21:22:44.987939+0000","last_unstale":"2024-04-20T21:22:46.996263+0000","last_undegraded":"2024-04-20T21:22:46.996263+0000","last_fullsized":"2024-04-20T21:22:46.996263+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:46:00.265362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.985271+0000","last_change":"2024-04-20T21:22:44.985271+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.985271+0000","last_undegraded":"2024-04-20T21:22:44.985271+0000","last_fullsized":"2024-04-20T21:22:44.985271+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"70'2","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.996842+0000","last_change":"2024-04-20T21:22:44.992317+0000","last_active":"2024-04-20T21:22:46.996842+0000","last_peered":"2024-04-20T21:22:46.996842+0000","last_clean":"2024-04-20T21:22:46.996842+0000","last_became_active":"2024-04-20T21:22:44.992040+0000","last_became_peered":"2024-04-20T21:22:44.992040+0000","last_unstale":"2024-04-20T21:22:46.996842+0000","last_undegraded":"2024-04-20T21:22:46.996842+0000","last_fullsized":"2024-04-20T21:22:46.996842+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:50:24.320805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.015238+0000","last_change":"2024-04-20T21:22:45.998870+0000","last_active":"2024-04-20T21:22:47.015238+0000","last_peered":"2024-04-20T21:22:47.015238+0000","last_clean":"2024-04-20T21:22:47.015238+0000","last_became_active":"2024-04-20T21:22:45.998537+0000","last_became_peered":"2024-04-20T21:22:45.998537+0000","last_unstale":"2024-04-20T21:22:47.015238+0000","last_undegraded":"2024-04-20T21:22:47.015238+0000","last_fullsized":"2024-04-20T21:22:47.015238+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:55:38.941276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.992803+0000","last_change":"2024-04-20T21:22:44.987324+0000","last_active":"2024-04-20T21:22:44.992803+0000","last_peered":"2024-04-20T21:22:44.992803+0000","last_clean":"2024-04-20T21:22:44.992803+0000","last_became_active":"2024-04-20T21:22:44.986974+0000","last_became_peered":"2024-04-20T21:22:44.986974+0000","last_unstale":"2024-04-20T21:22:44.992803+0000","last_undegraded":"2024-04-20T21:22:44.992803+0000","last_fullsized":"2024-04-20T21:22:44.992803+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:40:38.463867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.983547+0000","last_change":"2024-04-20T21:22:44.983547+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.983547+0000","last_undegraded":"2024-04-20T21:22:44.983547+0000","last_fullsized":"2024-04-20T21:22:44.983547+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.014700+0000","last_change":"2024-04-20T21:22:44.997932+0000","last_active":"2024-04-20T21:22:47.014700+0000","last_peered":"2024-04-20T21:22:47.014700+0000","last_clean":"2024-04-20T21:22:47.014700+0000","last_became_active":"2024-04-20T21:22:44.997397+0000","last_became_peered":"2024-04-20T21:22:44.997397+0000","last_unstale":"2024-04-20T21:22:47.014700+0000","last_undegraded":"2024-04-20T21:22:47.014700+0000","last_fullsized":"2024-04-20T21:22:47.014700+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:56:13.768402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.357344+0000","last_change":"2024-04-20T21:22:45.997195+0000","last_active":"2024-04-20T21:22:47.357344+0000","last_peered":"2024-04-20T21:22:47.357344+0000","last_clean":"2024-04-20T21:22:47.357344+0000","last_became_active":"2024-04-20T21:22:45.996696+0000","last_became_peered":"2024-04-20T21:22:45.996696+0000","last_unstale":"2024-04-20T21:22:47.357344+0000","last_undegraded":"2024-04-20T21:22:47.357344+0000","last_fullsized":"2024-04-20T21:22:47.357344+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:40:15.771704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"70'8","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.998939+0000","last_change":"2024-04-20T21:22:44.995360+0000","last_active":"2024-04-20T21:22:46.998939+0000","last_peered":"2024-04-20T21:22:46.998939+0000","last_clean":"2024-04-20T21:22:46.998939+0000","last_became_active":"2024-04-20T21:22:44.995055+0000","last_became_peered":"2024-04-20T21:22:44.995055+0000","last_unstale":"2024-04-20T21:22:46.998939+0000","last_undegraded":"2024-04-20T21:22:46.998939+0000","last_fullsized":"2024-04-20T21:22:46.998939+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:14:22.771721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.001952+0000","last_change":"2024-04-20T21:22:45.993981+0000","last_active":"2024-04-20T21:22:47.001952+0000","last_peered":"2024-04-20T21:22:47.001952+0000","last_clean":"2024-04-20T21:22:47.001952+0000","last_became_active":"2024-04-20T21:22:45.993203+0000","last_became_peered":"2024-04-20T21:22:45.993203+0000","last_unstale":"2024-04-20T21:22:47.001952+0000","last_undegraded":"2024-04-20T21:22:47.001952+0000","last_fullsized":"2024-04-20T21:22:47.001952+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:53:17.849382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.999385+0000","last_change":"2024-04-20T21:22:44.993745+0000","last_active":"2024-04-20T21:22:44.999385+0000","last_peered":"2024-04-20T21:22:44.999385+0000","last_clean":"2024-04-20T21:22:44.999385+0000","last_became_active":"2024-04-20T21:22:44.992981+0000","last_became_peered":"2024-04-20T21:22:44.992981+0000","last_unstale":"2024-04-20T21:22:44.999385+0000","last_undegraded":"2024-04-20T21:22:44.999385+0000","last_fullsized":"2024-04-20T21:22:44.999385+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:08:56.570003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.987280+0000","last_change":"2024-04-20T21:22:44.987280+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.987280+0000","last_undegraded":"2024-04-20T21:22:44.987280+0000","last_fullsized":"2024-04-20T21:22:44.987280+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"70'2","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.002056+0000","last_change":"2024-04-20T21:22:44.998796+0000","last_active":"2024-04-20T21:22:47.002056+0000","last_peered":"2024-04-20T21:22:47.002056+0000","last_clean":"2024-04-20T21:22:47.002056+0000","last_became_active":"2024-04-20T21:22:44.998424+0000","last_became_peered":"2024-04-20T21:22:44.998424+0000","last_unstale":"2024-04-20T21:22:47.002056+0000","last_undegraded":"2024-04-20T21:22:47.002056+0000","last_fullsized":"2024-04-20T21:22:47.002056+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:45:10.580900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.996286+0000","last_change":"2024-04-20T21:22:45.999490+0000","last_active":"2024-04-20T21:22:46.996286+0000","last_peered":"2024-04-20T21:22:46.996286+0000","last_clean":"2024-04-20T21:22:46.996286+0000","last_became_active":"2024-04-20T21:22:45.999118+0000","last_became_peered":"2024-04-20T21:22:45.999118+0000","last_unstale":"2024-04-20T21:22:46.996286+0000","last_undegraded":"2024-04-20T21:22:46.996286+0000","last_fullsized":"2024-04-20T21:22:46.996286+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:55:03.918649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:45.003213+0000","last_change":"2024-04-20T21:22:45.000092+0000","last_active":"2024-04-20T21:22:45.003213+0000","last_peered":"2024-04-20T21:22:45.003213+0000","last_clean":"2024-04-20T21:22:45.003213+0000","last_became_active":"2024-04-20T21:22:44.998998+0000","last_became_peered":"2024-04-20T21:22:44.998998+0000","last_unstale":"2024-04-20T21:22:45.003213+0000","last_undegraded":"2024-04-20T21:22:45.003213+0000","last_fullsized":"2024-04-20T21:22:45.003213+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:19:34.832419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.358172+0000","last_change":"2024-04-20T21:22:46.001345+0000","last_active":"2024-04-20T21:22:47.358172+0000","last_peered":"2024-04-20T21:22:47.358172+0000","last_clean":"2024-04-20T21:22:47.358172+0000","last_became_active":"2024-04-20T21:22:46.001055+0000","last_became_peered":"2024-04-20T21:22:46.001055+0000","last_unstale":"2024-04-20T21:22:47.358172+0000","last_undegraded":"2024-04-20T21:22:47.358172+0000","last_fullsized":"2024-04-20T21:22:47.358172+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:48:59.404081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:45.007767+0000","last_change":"2024-04-20T21:22:45.002908+0000","last_active":"2024-04-20T21:22:45.007767+0000","last_peered":"2024-04-20T21:22:45.007767+0000","last_clean":"2024-04-20T21:22:45.007767+0000","last_became_active":"2024-04-20T21:22:45.002596+0000","last_became_peered":"2024-04-20T21:22:45.002596+0000","last_unstale":"2024-04-20T21:22:45.007767+0000","last_undegraded":"2024-04-20T21:22:45.007767+0000","last_fullsized":"2024-04-20T21:22:45.007767+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:07:15.232666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.357387+0000","last_change":"2024-04-20T21:22:46.004184+0000","last_active":"2024-04-20T21:22:47.357387+0000","last_peered":"2024-04-20T21:22:47.357387+0000","last_clean":"2024-04-20T21:22:47.357387+0000","last_became_active":"2024-04-20T21:22:46.003915+0000","last_became_peered":"2024-04-20T21:22:46.003915+0000","last_unstale":"2024-04-20T21:22:47.357387+0000","last_undegraded":"2024-04-20T21:22:47.357387+0000","last_fullsized":"2024-04-20T21:22:47.357387+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:40:38.260253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:45.000938+0000","last_change":"2024-04-20T21:22:44.996534+0000","last_active":"2024-04-20T21:22:45.000938+0000","last_peered":"2024-04-20T21:22:45.000938+0000","last_clean":"2024-04-20T21:22:45.000938+0000","last_became_active":"2024-04-20T21:22:44.996190+0000","last_became_peered":"2024-04-20T21:22:44.996190+0000","last_unstale":"2024-04-20T21:22:45.000938+0000","last_undegraded":"2024-04-20T21:22:45.000938+0000","last_fullsized":"2024-04-20T21:22:45.000938+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:48:58.388149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.356519+0000","last_change":"2024-04-20T21:22:46.005627+0000","last_active":"2024-04-20T21:22:47.356519+0000","last_peered":"2024-04-20T21:22:47.356519+0000","last_clean":"2024-04-20T21:22:47.356519+0000","last_became_active":"2024-04-20T21:22:46.005377+0000","last_became_peered":"2024-04-20T21:22:46.005377+0000","last_unstale":"2024-04-20T21:22:47.356519+0000","last_undegraded":"2024-04-20T21:22:47.356519+0000","last_fullsized":"2024-04-20T21:22:47.356519+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:06:18.941092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.015474+0000","last_change":"2024-04-20T21:22:44.993852+0000","last_active":"2024-04-20T21:22:47.015474+0000","last_peered":"2024-04-20T21:22:47.015474+0000","last_clean":"2024-04-20T21:22:47.015474+0000","last_became_active":"2024-04-20T21:22:44.993491+0000","last_became_peered":"2024-04-20T21:22:44.993491+0000","last_unstale":"2024-04-20T21:22:47.015474+0000","last_undegraded":"2024-04-20T21:22:47.015474+0000","last_fullsized":"2024-04-20T21:22:47.015474+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:07:15.232666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.356646+0000","last_change":"2024-04-20T21:22:46.001218+0000","last_active":"2024-04-20T21:22:47.356646+0000","last_peered":"2024-04-20T21:22:47.356646+0000","last_clean":"2024-04-20T21:22:47.356646+0000","last_became_active":"2024-04-20T21:22:46.000900+0000","last_became_peered":"2024-04-20T21:22:46.000900+0000","last_unstale":"2024-04-20T21:22:47.356646+0000","last_undegraded":"2024-04-20T21:22:47.356646+0000","last_fullsized":"2024-04-20T21:22:47.356646+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:53:21.061854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.994517+0000","last_change":"2024-04-20T21:22:44.992558+0000","last_active":"2024-04-20T21:22:44.994517+0000","last_peered":"2024-04-20T21:22:44.994517+0000","last_clean":"2024-04-20T21:22:44.994517+0000","last_became_active":"2024-04-20T21:22:44.991402+0000","last_became_peered":"2024-04-20T21:22:44.991402+0000","last_unstale":"2024-04-20T21:22:44.994517+0000","last_undegraded":"2024-04-20T21:22:44.994517+0000","last_fullsized":"2024-04-20T21:22:44.994517+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:47:51.476465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.985660+0000","last_change":"2024-04-20T21:22:44.985660+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.985660+0000","last_undegraded":"2024-04-20T21:22:44.985660+0000","last_fullsized":"2024-04-20T21:22:44.985660+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.996911+0000","last_change":"2024-04-20T21:22:44.990230+0000","last_active":"2024-04-20T21:22:44.996911+0000","last_peered":"2024-04-20T21:22:44.996911+0000","last_clean":"2024-04-20T21:22:44.996911+0000","last_became_active":"2024-04-20T21:22:44.989958+0000","last_became_peered":"2024-04-20T21:22:44.989958+0000","last_unstale":"2024-04-20T21:22:44.996911+0000","last_undegraded":"2024-04-20T21:22:44.996911+0000","last_fullsized":"2024-04-20T21:22:44.996911+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:18:22.288371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.015250+0000","last_change":"2024-04-20T21:22:45.993591+0000","last_active":"2024-04-20T21:22:47.015250+0000","last_peered":"2024-04-20T21:22:47.015250+0000","last_clean":"2024-04-20T21:22:47.015250+0000","last_became_active":"2024-04-20T21:22:45.993320+0000","last_became_peered":"2024-04-20T21:22:45.993320+0000","last_unstale":"2024-04-20T21:22:47.015250+0000","last_undegraded":"2024-04-20T21:22:47.015250+0000","last_fullsized":"2024-04-20T21:22:47.015250+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:24:16.195831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"71'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.015880+0000","last_change":"2024-04-20T21:22:44.997554+0000","last_active":"2024-04-20T21:22:47.015880+0000","last_peered":"2024-04-20T21:22:47.015880+0000","last_clean":"2024-04-20T21:22:47.015880+0000","last_became_active":"2024-04-20T21:22:44.997234+0000","last_became_peered":"2024-04-20T21:22:44.997234+0000","last_unstale":"2024-04-20T21:22:47.015880+0000","last_undegraded":"2024-04-20T21:22:47.015880+0000","last_fullsized":"2024-04-20T21:22:47.015880+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:47:51.476465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.986315+0000","last_change":"2024-04-20T21:22:44.986315+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.986315+0000","last_undegraded":"2024-04-20T21:22:44.986315+0000","last_fullsized":"2024-04-20T21:22:44.986315+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.358437+0000","last_change":"2024-04-20T21:22:44.998361+0000","last_active":"2024-04-20T21:22:47.358437+0000","last_peered":"2024-04-20T21:22:47.358437+0000","last_clean":"2024-04-20T21:22:47.358437+0000","last_became_active":"2024-04-20T21:22:44.996313+0000","last_became_peered":"2024-04-20T21:22:44.996313+0000","last_unstale":"2024-04-20T21:22:47.358437+0000","last_undegraded":"2024-04-20T21:22:47.358437+0000","last_fullsized":"2024-04-20T21:22:47.358437+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:57:50.796943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.986997+0000","last_change":"2024-04-20T21:22:44.986997+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.986997+0000","last_undegraded":"2024-04-20T21:22:44.986997+0000","last_fullsized":"2024-04-20T21:22:44.986997+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:45.004224+0000","last_change":"2024-04-20T21:22:45.003069+0000","last_active":"2024-04-20T21:22:45.004224+0000","last_peered":"2024-04-20T21:22:45.004224+0000","last_clean":"2024-04-20T21:22:45.004224+0000","last_became_active":"2024-04-20T21:22:45.002662+0000","last_became_peered":"2024-04-20T21:22:45.002662+0000","last_unstale":"2024-04-20T21:22:45.004224+0000","last_undegraded":"2024-04-20T21:22:45.004224+0000","last_fullsized":"2024-04-20T21:22:45.004224+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:17:48.882110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.357975+0000","last_change":"2024-04-20T21:22:45.998814+0000","last_active":"2024-04-20T21:22:47.357975+0000","last_peered":"2024-04-20T21:22:47.357975+0000","last_clean":"2024-04-20T21:22:47.357975+0000","last_became_active":"2024-04-20T21:22:45.998463+0000","last_became_peered":"2024-04-20T21:22:45.998463+0000","last_unstale":"2024-04-20T21:22:47.357975+0000","last_undegraded":"2024-04-20T21:22:47.357975+0000","last_fullsized":"2024-04-20T21:22:47.357975+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:37:37.094186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.357407+0000","last_change":"2024-04-20T21:22:44.990525+0000","last_active":"2024-04-20T21:22:47.357407+0000","last_peered":"2024-04-20T21:22:47.357407+0000","last_clean":"2024-04-20T21:22:47.357407+0000","last_became_active":"2024-04-20T21:22:44.990259+0000","last_became_peered":"2024-04-20T21:22:44.990259+0000","last_unstale":"2024-04-20T21:22:47.357407+0000","last_undegraded":"2024-04-20T21:22:47.357407+0000","last_fullsized":"2024-04-20T21:22:47.357407+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:07:45.118275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.971783+0000","last_change":"2024-04-20T21:22:44.971783+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.971783+0000","last_undegraded":"2024-04-20T21:22:44.971783+0000","last_fullsized":"2024-04-20T21:22:44.971783+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.995914+0000","last_change":"2024-04-20T21:22:44.996969+0000","last_active":"2024-04-20T21:22:46.995914+0000","last_peered":"2024-04-20T21:22:46.995914+0000","last_clean":"2024-04-20T21:22:46.995914+0000","last_became_active":"2024-04-20T21:22:44.995514+0000","last_became_peered":"2024-04-20T21:22:44.995514+0000","last_unstale":"2024-04-20T21:22:46.995914+0000","last_undegraded":"2024-04-20T21:22:46.995914+0000","last_fullsized":"2024-04-20T21:22:46.995914+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:35:20.490828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.996108+0000","last_change":"2024-04-20T21:22:46.000298+0000","last_active":"2024-04-20T21:22:46.996108+0000","last_peered":"2024-04-20T21:22:46.996108+0000","last_clean":"2024-04-20T21:22:46.996108+0000","last_became_active":"2024-04-20T21:22:46.000027+0000","last_became_peered":"2024-04-20T21:22:46.000027+0000","last_unstale":"2024-04-20T21:22:46.996108+0000","last_undegraded":"2024-04-20T21:22:46.996108+0000","last_fullsized":"2024-04-20T21:22:46.996108+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:53:43.293248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.999123+0000","last_change":"2024-04-20T21:22:44.994701+0000","last_active":"2024-04-20T21:22:46.999123+0000","last_peered":"2024-04-20T21:22:46.999123+0000","last_clean":"2024-04-20T21:22:46.999123+0000","last_became_active":"2024-04-20T21:22:44.994359+0000","last_became_peered":"2024-04-20T21:22:44.994359+0000","last_unstale":"2024-04-20T21:22:46.999123+0000","last_undegraded":"2024-04-20T21:22:46.999123+0000","last_fullsized":"2024-04-20T21:22:46.999123+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:34:52.260354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.973852+0000","last_change":"2024-04-20T21:22:44.973852+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.973852+0000","last_undegraded":"2024-04-20T21:22:44.973852+0000","last_fullsized":"2024-04-20T21:22:44.973852+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.996863+0000","last_change":"2024-04-20T21:22:44.990936+0000","last_active":"2024-04-20T21:22:44.996863+0000","last_peered":"2024-04-20T21:22:44.996863+0000","last_clean":"2024-04-20T21:22:44.996863+0000","last_became_active":"2024-04-20T21:22:44.988494+0000","last_became_peered":"2024-04-20T21:22:44.988494+0000","last_unstale":"2024-04-20T21:22:44.996863+0000","last_undegraded":"2024-04-20T21:22:44.996863+0000","last_fullsized":"2024-04-20T21:22:44.996863+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:40:14.755772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.357272+0000","last_change":"2024-04-20T21:22:46.007192+0000","last_active":"2024-04-20T21:22:47.357272+0000","last_peered":"2024-04-20T21:22:47.357272+0000","last_clean":"2024-04-20T21:22:47.357272+0000","last_became_active":"2024-04-20T21:22:46.006583+0000","last_became_peered":"2024-04-20T21:22:46.006583+0000","last_unstale":"2024-04-20T21:22:47.357272+0000","last_undegraded":"2024-04-20T21:22:47.357272+0000","last_fullsized":"2024-04-20T21:22:47.357272+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:28:45.591146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.993111+0000","last_change":"2024-04-20T21:22:44.988535+0000","last_active":"2024-04-20T21:22:44.993111+0000","last_peered":"2024-04-20T21:22:44.993111+0000","last_clean":"2024-04-20T21:22:44.993111+0000","last_became_active":"2024-04-20T21:22:44.988183+0000","last_became_peered":"2024-04-20T21:22:44.988183+0000","last_unstale":"2024-04-20T21:22:44.993111+0000","last_undegraded":"2024-04-20T21:22:44.993111+0000","last_fullsized":"2024-04-20T21:22:44.993111+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:40:51.727352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":256,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.990279+0000","last_change":"2024-04-20T21:17:43.059637+0000","last_active":"2024-04-20T21:22:44.990279+0000","last_peered":"2024-04-20T21:22:44.990279+0000","last_clean":"2024-04-20T21:22:44.990279+0000","last_became_active":"2024-04-20T21:17:43.056207+0000","last_became_peered":"2024-04-20T21:17:43.056207+0000","last_unstale":"2024-04-20T21:22:44.990279+0000","last_undegraded":"2024-04-20T21:22:44.990279+0000","last_fullsized":"2024-04-20T21:22:44.990279+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:41.036290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:41.036290+0000","last_clean_scrub_stamp":"2024-04-20T21:17:41.036290+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:58:51.091337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":1736,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25,"ondisk_log_size":25,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574911,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32584,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339256,"statfs":{"total":96636764160,"available":96603398144,"internally_reserved":0,"allocated":1433600,"data_stored":941588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574912,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":839680,"data_stored":351220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574915,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":868352,"data_stored":364385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574914,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32036,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339804,"statfs":{"total":96636764160,"available":96603959296,"internally_reserved":0,"allocated":868352,"data_stored":364365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574914,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":872448,"data_stored":364407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574914,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32036,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339804,"statfs":{"total":96636764160,"available":96603959296,"internally_reserved":0,"allocated":876544,"data_stored":365907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32052,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339788,"statfs":{"total":96636764160,"available":96603942912,"internally_reserved":0,"allocated":884736,"data_stored":366523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574913,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32584,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339256,"statfs":{"total":96636764160,"available":96603398144,"internally_reserved":0,"allocated":1433600,"data_stored":941588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T21:22:51.370 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3f is not active+clean 2024-04-20T21:22:51.370 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2024-04-20T21:22:44.989723+0000', 'last_change': '2024-04-20T21:22:44.989723+0000', 'last_active': '2024-04-20T21:22:44.946779+0000', 'last_peered': '2024-04-20T21:22:44.946779+0000', 'last_clean': '2024-04-20T21:22:44.946779+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T21:22:44.989723+0000', 'last_undegraded': '2024-04-20T21:22:44.989723+0000', 'last_fullsized': '2024-04-20T21:22:44.989723+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_clean_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-20T21:22:51.370 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3e is not active+clean 2024-04-20T21:22:51.370 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2024-04-20T21:22:44.972721+0000', 'last_change': '2024-04-20T21:22:44.972721+0000', 'last_active': '2024-04-20T21:22:44.946779+0000', 'last_peered': '2024-04-20T21:22:44.946779+0000', 'last_clean': '2024-04-20T21:22:44.946779+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T21:22:44.972721+0000', 'last_undegraded': '2024-04-20T21:22:44.972721+0000', 'last_fullsized': '2024-04-20T21:22:44.972721+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_clean_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-20T21:22:51.370 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3a is not active+clean 2024-04-20T21:22:51.370 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2024-04-20T21:22:44.976258+0000', 'last_change': '2024-04-20T21:22:44.976258+0000', 'last_active': '2024-04-20T21:22:44.946779+0000', 'last_peered': '2024-04-20T21:22:44.946779+0000', 'last_clean': '2024-04-20T21:22:44.946779+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T21:22:44.976258+0000', 'last_undegraded': '2024-04-20T21:22:44.976258+0000', 'last_fullsized': '2024-04-20T21:22:44.976258+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_clean_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-20T21:22:51.370 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.38 is not active+clean 2024-04-20T21:22:51.370 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2024-04-20T21:22:44.971878+0000', 'last_change': '2024-04-20T21:22:44.971878+0000', 'last_active': '2024-04-20T21:22:44.946779+0000', 'last_peered': '2024-04-20T21:22:44.946779+0000', 'last_clean': '2024-04-20T21:22:44.946779+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T21:22:44.971878+0000', 'last_undegraded': '2024-04-20T21:22:44.971878+0000', 'last_fullsized': '2024-04-20T21:22:44.971878+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_clean_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-20T21:22:51.371 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.35 is not active+clean 2024-04-20T21:22:51.371 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2024-04-20T21:22:44.989195+0000', 'last_change': '2024-04-20T21:22:44.989195+0000', 'last_active': '2024-04-20T21:22:44.946779+0000', 'last_peered': '2024-04-20T21:22:44.946779+0000', 'last_clean': '2024-04-20T21:22:44.946779+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T21:22:44.989195+0000', 'last_undegraded': '2024-04-20T21:22:44.989195+0000', 'last_fullsized': '2024-04-20T21:22:44.989195+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_clean_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-20T21:22:51.371 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.33 is not active+clean 2024-04-20T21:22:51.371 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2024-04-20T21:22:44.983752+0000', 'last_change': '2024-04-20T21:22:44.983752+0000', 'last_active': '2024-04-20T21:22:44.946779+0000', 'last_peered': '2024-04-20T21:22:44.946779+0000', 'last_clean': '2024-04-20T21:22:44.946779+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T21:22:44.983752+0000', 'last_undegraded': '2024-04-20T21:22:44.983752+0000', 'last_fullsized': '2024-04-20T21:22:44.983752+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_clean_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-20T21:22:51.371 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.31 is not active+clean 2024-04-20T21:22:51.371 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2024-04-20T21:22:44.984365+0000', 'last_change': '2024-04-20T21:22:44.984365+0000', 'last_active': '2024-04-20T21:22:44.946779+0000', 'last_peered': '2024-04-20T21:22:44.946779+0000', 'last_clean': '2024-04-20T21:22:44.946779+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T21:22:44.984365+0000', 'last_undegraded': '2024-04-20T21:22:44.984365+0000', 'last_fullsized': '2024-04-20T21:22:44.984365+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_clean_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-20T21:22:51.371 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2d is not active+clean 2024-04-20T21:22:51.371 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2024-04-20T21:22:44.978703+0000', 'last_change': '2024-04-20T21:22:44.978703+0000', 'last_active': '2024-04-20T21:22:44.946779+0000', 'last_peered': '2024-04-20T21:22:44.946779+0000', 'last_clean': '2024-04-20T21:22:44.946779+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T21:22:44.978703+0000', 'last_undegraded': '2024-04-20T21:22:44.978703+0000', 'last_fullsized': '2024-04-20T21:22:44.978703+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_clean_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-20T21:22:51.371 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2c is not active+clean 2024-04-20T21:22:51.371 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2024-04-20T21:22:44.986040+0000', 'last_change': '2024-04-20T21:22:44.986040+0000', 'last_active': '2024-04-20T21:22:44.946779+0000', 'last_peered': '2024-04-20T21:22:44.946779+0000', 'last_clean': '2024-04-20T21:22:44.946779+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T21:22:44.986040+0000', 'last_undegraded': '2024-04-20T21:22:44.986040+0000', 'last_fullsized': '2024-04-20T21:22:44.986040+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_clean_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-20T21:22:51.371 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.29 is not active+clean 2024-04-20T21:22:51.372 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2024-04-20T21:22:44.985232+0000', 'last_change': '2024-04-20T21:22:44.985232+0000', 'last_active': '2024-04-20T21:22:44.946779+0000', 'last_peered': '2024-04-20T21:22:44.946779+0000', 'last_clean': '2024-04-20T21:22:44.946779+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T21:22:44.985232+0000', 'last_undegraded': '2024-04-20T21:22:44.985232+0000', 'last_fullsized': '2024-04-20T21:22:44.985232+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_clean_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-20T21:22:51.372 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.28 is not active+clean 2024-04-20T21:22:51.372 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2024-04-20T21:22:44.983826+0000', 'last_change': '2024-04-20T21:22:44.983826+0000', 'last_active': '2024-04-20T21:22:44.946779+0000', 'last_peered': '2024-04-20T21:22:44.946779+0000', 'last_clean': '2024-04-20T21:22:44.946779+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T21:22:44.983826+0000', 'last_undegraded': '2024-04-20T21:22:44.983826+0000', 'last_fullsized': '2024-04-20T21:22:44.983826+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_clean_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-20T21:22:51.372 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.20 is not active+clean 2024-04-20T21:22:51.372 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2024-04-20T21:22:44.970617+0000', 'last_change': '2024-04-20T21:22:44.970617+0000', 'last_active': '2024-04-20T21:22:44.946779+0000', 'last_peered': '2024-04-20T21:22:44.946779+0000', 'last_clean': '2024-04-20T21:22:44.946779+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T21:22:44.970617+0000', 'last_undegraded': '2024-04-20T21:22:44.970617+0000', 'last_fullsized': '2024-04-20T21:22:44.970617+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_clean_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-20T21:22:51.372 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1e is not active+clean 2024-04-20T21:22:51.372 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2024-04-20T21:22:44.982545+0000', 'last_change': '2024-04-20T21:22:44.982545+0000', 'last_active': '2024-04-20T21:22:44.946779+0000', 'last_peered': '2024-04-20T21:22:44.946779+0000', 'last_clean': '2024-04-20T21:22:44.946779+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T21:22:44.982545+0000', 'last_undegraded': '2024-04-20T21:22:44.982545+0000', 'last_fullsized': '2024-04-20T21:22:44.982545+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_clean_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-20T21:22:51.372 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1c is not active+clean 2024-04-20T21:22:51.372 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2024-04-20T21:22:44.979534+0000', 'last_change': '2024-04-20T21:22:44.979534+0000', 'last_active': '2024-04-20T21:22:44.946779+0000', 'last_peered': '2024-04-20T21:22:44.946779+0000', 'last_clean': '2024-04-20T21:22:44.946779+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T21:22:44.979534+0000', 'last_undegraded': '2024-04-20T21:22:44.979534+0000', 'last_fullsized': '2024-04-20T21:22:44.979534+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_clean_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-20T21:22:51.372 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1d is not active+clean 2024-04-20T21:22:51.373 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2024-04-20T21:22:44.988718+0000', 'last_change': '2024-04-20T21:22:44.988718+0000', 'last_active': '2024-04-20T21:22:44.946779+0000', 'last_peered': '2024-04-20T21:22:44.946779+0000', 'last_clean': '2024-04-20T21:22:44.946779+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T21:22:44.988718+0000', 'last_undegraded': '2024-04-20T21:22:44.988718+0000', 'last_fullsized': '2024-04-20T21:22:44.988718+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_clean_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-20T21:22:51.373 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.19 is not active+clean 2024-04-20T21:22:51.373 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2024-04-20T21:22:44.983784+0000', 'last_change': '2024-04-20T21:22:44.983784+0000', 'last_active': '2024-04-20T21:22:44.946779+0000', 'last_peered': '2024-04-20T21:22:44.946779+0000', 'last_clean': '2024-04-20T21:22:44.946779+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T21:22:44.983784+0000', 'last_undegraded': '2024-04-20T21:22:44.983784+0000', 'last_fullsized': '2024-04-20T21:22:44.983784+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_clean_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-20T21:22:51.373 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.17 is not active+clean 2024-04-20T21:22:51.373 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2024-04-20T21:22:44.988271+0000', 'last_change': '2024-04-20T21:22:44.988271+0000', 'last_active': '2024-04-20T21:22:44.946779+0000', 'last_peered': '2024-04-20T21:22:44.946779+0000', 'last_clean': '2024-04-20T21:22:44.946779+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T21:22:44.988271+0000', 'last_undegraded': '2024-04-20T21:22:44.988271+0000', 'last_fullsized': '2024-04-20T21:22:44.988271+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_clean_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-20T21:22:51.373 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.14 is not active+clean 2024-04-20T21:22:51.373 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2024-04-20T21:22:44.977417+0000', 'last_change': '2024-04-20T21:22:44.977417+0000', 'last_active': '2024-04-20T21:22:44.946779+0000', 'last_peered': '2024-04-20T21:22:44.946779+0000', 'last_clean': '2024-04-20T21:22:44.946779+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T21:22:44.977417+0000', 'last_undegraded': '2024-04-20T21:22:44.977417+0000', 'last_fullsized': '2024-04-20T21:22:44.977417+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_clean_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-20T21:22:51.373 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.12 is not active+clean 2024-04-20T21:22:51.373 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2024-04-20T21:22:44.985271+0000', 'last_change': '2024-04-20T21:22:44.985271+0000', 'last_active': '2024-04-20T21:22:44.946779+0000', 'last_peered': '2024-04-20T21:22:44.946779+0000', 'last_clean': '2024-04-20T21:22:44.946779+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T21:22:44.985271+0000', 'last_undegraded': '2024-04-20T21:22:44.985271+0000', 'last_fullsized': '2024-04-20T21:22:44.985271+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_clean_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-20T21:22:51.373 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.10 is not active+clean 2024-04-20T21:22:51.373 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2024-04-20T21:22:44.983547+0000', 'last_change': '2024-04-20T21:22:44.983547+0000', 'last_active': '2024-04-20T21:22:44.946779+0000', 'last_peered': '2024-04-20T21:22:44.946779+0000', 'last_clean': '2024-04-20T21:22:44.946779+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T21:22:44.983547+0000', 'last_undegraded': '2024-04-20T21:22:44.983547+0000', 'last_fullsized': '2024-04-20T21:22:44.983547+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_clean_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-20T21:22:51.374 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.f is not active+clean 2024-04-20T21:22:51.374 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2024-04-20T21:22:44.987280+0000', 'last_change': '2024-04-20T21:22:44.987280+0000', 'last_active': '2024-04-20T21:22:44.946779+0000', 'last_peered': '2024-04-20T21:22:44.946779+0000', 'last_clean': '2024-04-20T21:22:44.946779+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T21:22:44.987280+0000', 'last_undegraded': '2024-04-20T21:22:44.987280+0000', 'last_fullsized': '2024-04-20T21:22:44.987280+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_clean_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-20T21:22:51.374 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.6 is not active+clean 2024-04-20T21:22:51.374 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2024-04-20T21:22:44.985660+0000', 'last_change': '2024-04-20T21:22:44.985660+0000', 'last_active': '2024-04-20T21:22:44.946779+0000', 'last_peered': '2024-04-20T21:22:44.946779+0000', 'last_clean': '2024-04-20T21:22:44.946779+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T21:22:44.985660+0000', 'last_undegraded': '2024-04-20T21:22:44.985660+0000', 'last_fullsized': '2024-04-20T21:22:44.985660+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_clean_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-20T21:22:51.374 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.4 is not active+clean 2024-04-20T21:22:51.374 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2024-04-20T21:22:44.986315+0000', 'last_change': '2024-04-20T21:22:44.986315+0000', 'last_active': '2024-04-20T21:22:44.946779+0000', 'last_peered': '2024-04-20T21:22:44.946779+0000', 'last_clean': '2024-04-20T21:22:44.946779+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T21:22:44.986315+0000', 'last_undegraded': '2024-04-20T21:22:44.986315+0000', 'last_fullsized': '2024-04-20T21:22:44.986315+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_clean_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-20T21:22:51.374 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.5 is not active+clean 2024-04-20T21:22:51.374 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2024-04-20T21:22:44.986997+0000', 'last_change': '2024-04-20T21:22:44.986997+0000', 'last_active': '2024-04-20T21:22:44.946779+0000', 'last_peered': '2024-04-20T21:22:44.946779+0000', 'last_clean': '2024-04-20T21:22:44.946779+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T21:22:44.986997+0000', 'last_undegraded': '2024-04-20T21:22:44.986997+0000', 'last_fullsized': '2024-04-20T21:22:44.986997+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_clean_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-20T21:22:51.374 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3 is not active+clean 2024-04-20T21:22:51.374 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2024-04-20T21:22:44.971783+0000', 'last_change': '2024-04-20T21:22:44.971783+0000', 'last_active': '2024-04-20T21:22:44.946779+0000', 'last_peered': '2024-04-20T21:22:44.946779+0000', 'last_clean': '2024-04-20T21:22:44.946779+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T21:22:44.971783+0000', 'last_undegraded': '2024-04-20T21:22:44.971783+0000', 'last_fullsized': '2024-04-20T21:22:44.971783+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_clean_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-20T21:22:51.375 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1 is not active+clean 2024-04-20T21:22:51.375 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 69, 'state': 'creating+peering', 'last_fresh': '2024-04-20T21:22:44.973852+0000', 'last_change': '2024-04-20T21:22:44.973852+0000', 'last_active': '2024-04-20T21:22:44.946779+0000', 'last_peered': '2024-04-20T21:22:44.946779+0000', 'last_clean': '2024-04-20T21:22:44.946779+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T21:22:44.973852+0000', 'last_undegraded': '2024-04-20T21:22:44.973852+0000', 'last_fullsized': '2024-04-20T21:22:44.973852+0000', 'mapping_epoch': 69, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 69, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'last_clean_scrub_stamp': '2024-04-20T21:22:44.946779+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-20T21:22:51.375 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-20T21:22:51.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.607+0000 7f1af4a00640 1 Processor -- start 2024-04-20T21:22:51.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.607+0000 7f1af4a00640 1 -- start start 2024-04-20T21:22:51.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.609+0000 7f1af4a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1af0144e90 0x7f1af00b4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:51.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.609+0000 7f1af4a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1af00cdb50 con 0x7f1af00cd170 2024-04-20T21:22:51.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.609+0000 7f1af4a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1af00b3e50 con 0x7f1af006f450 2024-04-20T21:22:51.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.609+0000 7f1af4a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f1af0145270 con 0x7f1af0144e90 2024-04-20T21:22:51.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.609+0000 7f1ae5400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f1af00cd170 0x7f1af006ed40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44756/0 (socket says 172.21.15.86:44756) 2024-04-20T21:22:51.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.609+0000 7f1ae5400640 1 -- 172.21.15.86:0/2697342911 learned_addr learned my addr 172.21.15.86:0/2697342911 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:22:51.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.610+0000 7f1aede00640 1 --2- 172.21.15.86:0/2697342911 >> v2:172.21.15.144:3300/0 conn(0x7f1af0144e90 0x7f1af00b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:51.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.610+0000 7f1aed400640 1 -- 172.21.15.86:0/2697342911 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2093184177 0 0) 0x7f1af00cdb50 con 0x7f1af00cd170 2024-04-20T21:22:51.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.610+0000 7f1aed400640 1 -- 172.21.15.86:0/2697342911 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ad4003520 con 0x7f1af00cd170 2024-04-20T21:22:51.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.610+0000 7f1aed400640 1 -- 172.21.15.86:0/2697342911 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2952217473 0 0) 0x7f1ad4003520 con 0x7f1af00cd170 2024-04-20T21:22:51.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.610+0000 7f1aed400640 1 -- 172.21.15.86:0/2697342911 >> v2:172.21.15.144:3300/0 conn(0x7f1af0144e90 msgr2=0x7f1af00b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:51.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.610+0000 7f1aed400640 1 --2- 172.21.15.86:0/2697342911 >> v2:172.21.15.144:3300/0 conn(0x7f1af0144e90 0x7f1af00b4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:51.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.610+0000 7f1aed400640 1 -- 172.21.15.86:0/2697342911 >> v1:172.21.15.144:6789/0 conn(0x7f1af006f450 legacy=0x7f1af015bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:51.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.610+0000 7f1aed400640 1 -- 172.21.15.86:0/2697342911 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1af00b4130 con 0x7f1af00cd170 2024-04-20T21:22:51.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.610+0000 7f1aed400640 1 -- 172.21.15.86:0/2697342911 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f1ad8001670 con 0x7f1af00cd170 2024-04-20T21:22:51.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.610+0000 7f1aed400640 1 -- 172.21.15.86:0/2697342911 >> v1:172.21.15.86:6789/0 conn(0x7f1af00cd170 legacy=0x7f1af006ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:51.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.610+0000 7f1aed400640 1 --2- 172.21.15.86:0/2697342911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ad40039b0 0x7f1ad4003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:51.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.610+0000 7f1aed400640 1 --2- 172.21.15.86:0/2697342911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ad4004800 0x7f1ad4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:51.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.610+0000 7f1aed400640 1 --2- 172.21.15.86:0/2697342911 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ad40051c0 0x7f1ad400dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:51.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.610+0000 7f1aed400640 1 -- 172.21.15.86:0/2697342911 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1af0145270 con 0x7f1ad40039b0 2024-04-20T21:22:51.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.610+0000 7f1aed400640 1 -- 172.21.15.86:0/2697342911 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad400e2a0 con 0x7f1ad4004800 2024-04-20T21:22:51.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.610+0000 7f1aed400640 1 -- 172.21.15.86:0/2697342911 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1ad400e4a0 con 0x7f1ad40051c0 2024-04-20T21:22:51.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.611+0000 7f1aede00640 1 --2- 172.21.15.86:0/2697342911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ad40039b0 0x7f1ad4003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:51.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.611+0000 7f1aede00640 1 -- 172.21.15.86:0/2697342911 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ad40051c0 msgr2=0x7f1ad400dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:51.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.611+0000 7f1aede00640 1 --2- 172.21.15.86:0/2697342911 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ad40051c0 0x7f1ad400dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:51.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.611+0000 7f1aede00640 1 -- 172.21.15.86:0/2697342911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ad4004800 msgr2=0x7f1ad4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:51.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.611+0000 7f1aede00640 1 --2- 172.21.15.86:0/2697342911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ad4004800 0x7f1ad4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:51.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.611+0000 7f1aede00640 1 -- 172.21.15.86:0/2697342911 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ad4003520 con 0x7f1ad40039b0 2024-04-20T21:22:51.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.611+0000 7f1aee800640 1 -- 172.21.15.86:0/2697342911 reap_dead start 2024-04-20T21:22:51.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.612+0000 7f1aede00640 1 --2- 172.21.15.86:0/2697342911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ad40039b0 0x7f1ad4003df0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0009fc0 tx=0x7f1ad4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fc42f5bf2f4239b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:51.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.612+0000 7f1aed400640 1 -- 172.21.15.86:0/2697342911 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae0039070 con 0x7f1ad40039b0 2024-04-20T21:22:51.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.612+0000 7f1aed400640 1 -- 172.21.15.86:0/2697342911 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ae002cc60 con 0x7f1ad40039b0 2024-04-20T21:22:51.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.612+0000 7f1aed400640 1 -- 172.21.15.86:0/2697342911 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae0009c10 con 0x7f1ad40039b0 2024-04-20T21:22:51.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.613+0000 7f1af4a00640 1 -- 172.21.15.86:0/2697342911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ad40039b0 msgr2=0x7f1ad4003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:51.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.613+0000 7f1af4a00640 1 --2- 172.21.15.86:0/2697342911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ad40039b0 0x7f1ad4003df0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0009fc0 tx=0x7f1ad4005830 comp rx=0 tx=0).stop 2024-04-20T21:22:51.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.613+0000 7f1af4a00640 1 -- 172.21.15.86:0/2697342911 shutdown_connections 2024-04-20T21:22:51.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.613+0000 7f1af4a00640 1 --2- 172.21.15.86:0/2697342911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ad40039b0 0x7f1ad4003df0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:51.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.613+0000 7f1af4a00640 1 -- 172.21.15.86:0/2697342911 >> 172.21.15.86:0/2697342911 conn(0x7f1af00b2360 msgr2=0x7f1af00b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:51.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.613+0000 7f1af4a00640 1 -- 172.21.15.86:0/2697342911 shutdown_connections 2024-04-20T21:22:51.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.613+0000 7f1af4a00640 1 -- 172.21.15.86:0/2697342911 wait complete. 2024-04-20T21:22:51.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.615+0000 7f1af4a00640 1 Processor -- start 2024-04-20T21:22:51.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.615+0000 7f1af4a00640 1 -- start start 2024-04-20T21:22:51.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.616+0000 7f1af4a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ad40039b0 0x7f1af013fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:51.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.616+0000 7f1af4a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1af015b120 0x7f1af00abcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:51.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.616+0000 7f1af4a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1af00ac1f0 0x7f1af015a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:51.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.616+0000 7f1af4a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1af00d0190 con 0x7f1ad40039b0 2024-04-20T21:22:51.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.616+0000 7f1af4a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1af00d0010 con 0x7f1af015b120 2024-04-20T21:22:51.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.616+0000 7f1af4a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1af00cd600 con 0x7f1af00ac1f0 2024-04-20T21:22:51.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.616+0000 7f1aede00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ad40039b0 0x7f1af013fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:51.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.616+0000 7f1aede00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ad40039b0 0x7f1af013fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52108/0 (socket says 172.21.15.86:52108) 2024-04-20T21:22:51.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.616+0000 7f1aede00640 1 -- 172.21.15.86:0/149954211 learned_addr learned my addr 172.21.15.86:0/149954211 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:51.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.616+0000 7f1aede00640 1 -- 172.21.15.86:0/149954211 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1af00ac1f0 msgr2=0x7f1af015a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:51.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.616+0000 7f1aede00640 1 --2- 172.21.15.86:0/149954211 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1af00ac1f0 0x7f1af015a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:51.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.616+0000 7f1aede00640 1 -- 172.21.15.86:0/149954211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1af015b120 msgr2=0x7f1af00abcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:51.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.616+0000 7f1aede00640 1 --2- 172.21.15.86:0/149954211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1af015b120 0x7f1af00abcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:51.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.616+0000 7f1aede00640 1 -- 172.21.15.86:0/149954211 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1af00a9690 con 0x7f1ad40039b0 2024-04-20T21:22:51.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.617+0000 7f1aede00640 1 --2- 172.21.15.86:0/149954211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ad40039b0 0x7f1af013fd30 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0037040 tx=0x7f1ae0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:51.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.617+0000 7f1ae6a00640 1 -- 172.21.15.86:0/149954211 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae0039050 con 0x7f1ad40039b0 2024-04-20T21:22:51.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.617+0000 7f1ae6a00640 1 -- 172.21.15.86:0/149954211 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ae003a040 con 0x7f1ad40039b0 2024-04-20T21:22:51.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.617+0000 7f1ae6a00640 1 -- 172.21.15.86:0/149954211 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae0009c10 con 0x7f1ad40039b0 2024-04-20T21:22:51.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.617+0000 7f1af4a00640 1 -- 172.21.15.86:0/149954211 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1af00a9930 con 0x7f1ad40039b0 2024-04-20T21:22:51.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.617+0000 7f1af4a00640 1 -- 172.21.15.86:0/149954211 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1af00aa1d0 con 0x7f1ad40039b0 2024-04-20T21:22:51.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.618+0000 7f1af4a00640 1 -- 172.21.15.86:0/149954211 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1af00a9b00 con 0x7f1ad40039b0 2024-04-20T21:22:51.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.619+0000 7f1ae6a00640 1 -- 172.21.15.86:0/149954211 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1ae0047020 con 0x7f1ad40039b0 2024-04-20T21:22:51.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.620+0000 7f1ae6a00640 1 --2- 172.21.15.86:0/149954211 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1acc06c8f0 0x7f1acc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:51.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.620+0000 7f1ae6a00640 1 -- 172.21.15.86:0/149954211 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f1ae00add40 con 0x7f1ad40039b0 2024-04-20T21:22:51.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.621+0000 7f1ae5400640 1 --2- 172.21.15.86:0/149954211 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1acc06c8f0 0x7f1acc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:51.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.621+0000 7f1ae5400640 1 --2- 172.21.15.86:0/149954211 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1acc06c8f0 0x7f1acc06edb0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1af015b520 tx=0x7f1ad8001520 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:51.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.625+0000 7f1ae6a00640 1 -- 172.21.15.86:0/149954211 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1ae004e020 con 0x7f1ad40039b0 2024-04-20T21:22:51.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.900+0000 7f1af4a00640 1 -- 172.21.15.86:0/149954211 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f1af00689e0 con 0x7f1ad40039b0 2024-04-20T21:22:51.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.901+0000 7f1ae6a00640 1 -- 172.21.15.86:0/149954211 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1260 (secure 0 0 0) 0x7f1ae007c0d0 con 0x7f1ad40039b0 2024-04-20T21:22:51.900 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:22:51.900 INFO:teuthology.orchestra.run.smithi086.stdout:{"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":324,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":71,"num_osds":8,"num_up_osds":8,"osd_up_since":1713647860,"num_in_osds":8,"osd_in_since":1713647850,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":71},{"state_name":"creating+peering","count":26}],"num_pgs":97,"num_pools":3,"num_objects":17,"data_bytes":592104,"bytes_used":263532544,"bytes_avail":772830580736,"bytes_total":773094113280,"inactive_pgs_ratio":0.2680412232875824,"write_bytes_sec":1876,"read_op_per_sec":0,"write_op_per_sec":5},"fsmap":{"epoch":42,"id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"a","status":"up:active","gid":5407}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2024-04-20T21:22:02.716058+0000","services":{"mds":{"daemons":{"summary":"","c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-20T21:22:51.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.904+0000 7f1af4a00640 1 -- 172.21.15.86:0/149954211 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1acc06c8f0 msgr2=0x7f1acc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:51.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.904+0000 7f1af4a00640 1 --2- 172.21.15.86:0/149954211 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1acc06c8f0 0x7f1acc06edb0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1af015b520 tx=0x7f1ad8001520 comp rx=0 tx=0).stop 2024-04-20T21:22:51.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.904+0000 7f1af4a00640 1 -- 172.21.15.86:0/149954211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ad40039b0 msgr2=0x7f1af013fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:51.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.904+0000 7f1af4a00640 1 --2- 172.21.15.86:0/149954211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ad40039b0 0x7f1af013fd30 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0037040 tx=0x7f1ae0009750 comp rx=0 tx=0).stop 2024-04-20T21:22:51.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.904+0000 7f1af4a00640 1 -- 172.21.15.86:0/149954211 shutdown_connections 2024-04-20T21:22:51.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.904+0000 7f1af4a00640 1 --2- 172.21.15.86:0/149954211 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1acc06c8f0 0x7f1acc06edb0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:51.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.904+0000 7f1af4a00640 1 --2- 172.21.15.86:0/149954211 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1af00ac1f0 0x7f1af015a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:51.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.904+0000 7f1af4a00640 1 --2- 172.21.15.86:0/149954211 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1af015b120 0x7f1af00abcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:51.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.904+0000 7f1af4a00640 1 --2- 172.21.15.86:0/149954211 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ad40039b0 0x7f1af013fd30 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:51.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.904+0000 7f1af4a00640 1 -- 172.21.15.86:0/149954211 >> 172.21.15.86:0/149954211 conn(0x7f1af00b2360 msgr2=0x7f1af00cc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:51.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.904+0000 7f1af4a00640 1 -- 172.21.15.86:0/149954211 shutdown_connections 2024-04-20T21:22:51.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:51.904+0000 7f1af4a00640 1 -- 172.21.15.86:0/149954211 wait complete. 2024-04-20T21:22:51.922 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-20T21:22:51.922 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T21:22:52.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.121+0000 7f65c6e00640 1 Processor -- start 2024-04-20T21:22:52.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.121+0000 7f65c6e00640 1 -- start start 2024-04-20T21:22:52.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.123+0000 7f65c6e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f65c0144e90 0x7f65c00b4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:52.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.123+0000 7f65c6e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f65c006f450 0x7f65c015bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:52.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.123+0000 7f65c6e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65c00cdb50 con 0x7f65c00cd170 2024-04-20T21:22:52.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.123+0000 7f65c6e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f65c0145270 con 0x7f65c0144e90 2024-04-20T21:22:52.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.123+0000 7f65c6e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f65c00b3e50 con 0x7f65c006f450 2024-04-20T21:22:52.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.123+0000 7f65bfe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f65c0144e90 0x7f65c00b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:52.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.123+0000 7f65bfe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f65c0144e90 0x7f65c00b4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52110/0 (socket says 172.21.15.86:52110) 2024-04-20T21:22:52.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.123+0000 7f65bfe00640 1 -- 172.21.15.86:0/1563438038 learned_addr learned my addr 172.21.15.86:0/1563438038 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:52.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.123+0000 7f65c4e00640 1 --2- 172.21.15.86:0/1563438038 >> v2:172.21.15.144:3300/0 conn(0x7f65c006f450 0x7f65c015bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:52.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.124+0000 7f65bfe00640 1 -- 172.21.15.86:0/1563438038 >> v2:172.21.15.144:3300/0 conn(0x7f65c006f450 msgr2=0x7f65c015bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:52.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.124+0000 7f65bfe00640 1 --2- 172.21.15.86:0/1563438038 >> v2:172.21.15.144:3300/0 conn(0x7f65c006f450 0x7f65c015bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:52.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.124+0000 7f65bfe00640 1 -- 172.21.15.86:0/1563438038 >> v1:172.21.15.86:6789/0 conn(0x7f65c00cd170 legacy=0x7f65c006ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:52.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.124+0000 7f65bfe00640 1 -- 172.21.15.86:0/1563438038 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c00b4130 con 0x7f65c0144e90 2024-04-20T21:22:52.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.124+0000 7f65c4e00640 1 --2- 172.21.15.86:0/1563438038 >> v2:172.21.15.144:3300/0 conn(0x7f65c006f450 0x7f65c015bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:22:52.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.124+0000 7f65bea00640 1 -- 172.21.15.86:0/1563438038 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1197704579 0 0) 0x7f65c00cdb50 con 0x7f65c00cd170 2024-04-20T21:22:52.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.125+0000 7f65bfe00640 1 --2- 172.21.15.86:0/1563438038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65c0144e90 0x7f65c00b4930 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f65c00b5320 tx=0x7f65b002b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17efe51c2d9629f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:52.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.125+0000 7f65bea00640 1 -- 172.21.15.86:0/1563438038 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b0034070 con 0x7f65c0144e90 2024-04-20T21:22:52.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.125+0000 7f65bea00640 1 -- 172.21.15.86:0/1563438038 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65b0030880 con 0x7f65c0144e90 2024-04-20T21:22:52.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.125+0000 7f65bea00640 1 -- 172.21.15.86:0/1563438038 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b0030c50 con 0x7f65c0144e90 2024-04-20T21:22:52.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.126+0000 7f65c6e00640 1 -- 172.21.15.86:0/1563438038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65c0144e90 msgr2=0x7f65c00b4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:52.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.126+0000 7f65c6e00640 1 --2- 172.21.15.86:0/1563438038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65c0144e90 0x7f65c00b4930 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f65c00b5320 tx=0x7f65b002b9c0 comp rx=0 tx=0).stop 2024-04-20T21:22:52.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.126+0000 7f65c6e00640 1 -- 172.21.15.86:0/1563438038 shutdown_connections 2024-04-20T21:22:52.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.126+0000 7f65c6e00640 1 --2- 172.21.15.86:0/1563438038 >> v2:172.21.15.144:3300/0 conn(0x7f65c006f450 0x7f65c015bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:52.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.126+0000 7f65c6e00640 1 --2- 172.21.15.86:0/1563438038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65c0144e90 0x7f65c00b4930 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:52.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.126+0000 7f65c6e00640 1 -- 172.21.15.86:0/1563438038 >> 172.21.15.86:0/1563438038 conn(0x7f65c00b2360 msgr2=0x7f65c00b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:52.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.126+0000 7f65c6e00640 1 -- 172.21.15.86:0/1563438038 shutdown_connections 2024-04-20T21:22:52.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.126+0000 7f65c6e00640 1 -- 172.21.15.86:0/1563438038 wait complete. 2024-04-20T21:22:52.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.127+0000 7f65c6e00640 1 Processor -- start 2024-04-20T21:22:52.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.128+0000 7f65c6e00640 1 -- start start 2024-04-20T21:22:52.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.129+0000 7f65c6e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65c006f450 0x7f65c01f4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:52.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.129+0000 7f65c6e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65c00cd170 0x7f65c00c6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:52.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.129+0000 7f65c6e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65c0144e90 0x7f65c00c6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:52.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.129+0000 7f65c6e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f65c00d0010 con 0x7f65c00cd170 2024-04-20T21:22:52.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.129+0000 7f65c6e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f65c00b4e70 con 0x7f65c0144e90 2024-04-20T21:22:52.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.129+0000 7f65c6e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f65c00d0190 con 0x7f65c006f450 2024-04-20T21:22:52.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.129+0000 7f65bf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65c00cd170 0x7f65c00c6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:52.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.129+0000 7f65bf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65c00cd170 0x7f65c00c6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52114/0 (socket says 172.21.15.86:52114) 2024-04-20T21:22:52.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.129+0000 7f65bf400640 1 -- 172.21.15.86:0/1935588591 learned_addr learned my addr 172.21.15.86:0/1935588591 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:52.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.129+0000 7f65bf400640 1 -- 172.21.15.86:0/1935588591 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65c006f450 msgr2=0x7f65c01f4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:52.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.129+0000 7f65bf400640 1 --2- 172.21.15.86:0/1935588591 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65c006f450 0x7f65c01f4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:52.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.129+0000 7f65bf400640 1 -- 172.21.15.86:0/1935588591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65c0144e90 msgr2=0x7f65c00c6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:52.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.129+0000 7f65bf400640 1 --2- 172.21.15.86:0/1935588591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65c0144e90 0x7f65c00c6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:52.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.129+0000 7f65bf400640 1 -- 172.21.15.86:0/1935588591 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c00ae620 con 0x7f65c00cd170 2024-04-20T21:22:52.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.130+0000 7f65bf400640 1 --2- 172.21.15.86:0/1935588591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65c00cd170 0x7f65c00c6250 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f65ac00c4b0 tx=0x7f65ac00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:52.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.130+0000 7f65b7e00640 1 -- 172.21.15.86:0/1935588591 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ac017070 con 0x7f65c00cd170 2024-04-20T21:22:52.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.130+0000 7f65b7e00640 1 -- 172.21.15.86:0/1935588591 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65ac005480 con 0x7f65c00cd170 2024-04-20T21:22:52.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.130+0000 7f65b7e00640 1 -- 172.21.15.86:0/1935588591 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ac003830 con 0x7f65c00cd170 2024-04-20T21:22:52.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.130+0000 7f65c6e00640 1 -- 172.21.15.86:0/1935588591 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65c00ae900 con 0x7f65c00cd170 2024-04-20T21:22:52.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.130+0000 7f65c6e00640 1 -- 172.21.15.86:0/1935588591 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65c00af1a0 con 0x7f65c00cd170 2024-04-20T21:22:52.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.131+0000 7f65c6e00640 1 -- 172.21.15.86:0/1935588591 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65c00af3a0 con 0x7f65c00cd170 2024-04-20T21:22:52.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.132+0000 7f65b7e00640 1 -- 172.21.15.86:0/1935588591 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f65ac01a070 con 0x7f65c00cd170 2024-04-20T21:22:52.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.133+0000 7f65b7e00640 1 --2- 172.21.15.86:0/1935588591 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f65a006c8f0 0x7f65a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:52.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.133+0000 7f65b7e00640 1 -- 172.21.15.86:0/1935588591 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f65ac08aff0 con 0x7f65c00cd170 2024-04-20T21:22:52.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.133+0000 7f65bfe00640 1 --2- 172.21.15.86:0/1935588591 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f65a006c8f0 0x7f65a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:52.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.134+0000 7f65bfe00640 1 --2- 172.21.15.86:0/1935588591 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f65a006c8f0 0x7f65a006edb0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f65b0036000 tx=0x7f65b0030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:52.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.138+0000 7f65b7e00640 1 -- 172.21.15.86:0/1935588591 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f65c00af3a0 con 0x7f65c00cd170 2024-04-20T21:22:52.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.338+0000 7f65c6e00640 1 -- 172.21.15.86:0/1935588591 --> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f65c00689e0 con 0x7f65a006c8f0 2024-04-20T21:22:52.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.355+0000 7f65b7e00640 1 -- 172.21.15.86:0/1935588591 <== mgr.4107 v2:172.21.15.86:6832/302045240 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249925 (secure 0 0 0) 0x7f65c00689e0 con 0x7f65a006c8f0 2024-04-20T21:22:52.358 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:22:52.360 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-20T21:22:52.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.361+0000 7f65c6e00640 1 -- 172.21.15.86:0/1935588591 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f65a006c8f0 msgr2=0x7f65a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:52.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.361+0000 7f65c6e00640 1 --2- 172.21.15.86:0/1935588591 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f65a006c8f0 0x7f65a006edb0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f65b0036000 tx=0x7f65b0030000 comp rx=0 tx=0).stop 2024-04-20T21:22:52.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.361+0000 7f65c6e00640 1 -- 172.21.15.86:0/1935588591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65c00cd170 msgr2=0x7f65c00c6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:52.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.361+0000 7f65c6e00640 1 --2- 172.21.15.86:0/1935588591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65c00cd170 0x7f65c00c6250 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f65ac00c4b0 tx=0x7f65ac00c970 comp rx=0 tx=0).stop 2024-04-20T21:22:52.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.362+0000 7f65c6e00640 1 -- 172.21.15.86:0/1935588591 shutdown_connections 2024-04-20T21:22:52.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.362+0000 7f65c6e00640 1 --2- 172.21.15.86:0/1935588591 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f65a006c8f0 0x7f65a006edb0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:52.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.362+0000 7f65c6e00640 1 --2- 172.21.15.86:0/1935588591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65c0144e90 0x7f65c00c6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:52.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.362+0000 7f65c6e00640 1 --2- 172.21.15.86:0/1935588591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65c00cd170 0x7f65c00c6250 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:52.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.362+0000 7f65c6e00640 1 --2- 172.21.15.86:0/1935588591 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f65c006f450 0x7f65c01f4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:52.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.362+0000 7f65c6e00640 1 -- 172.21.15.86:0/1935588591 >> 172.21.15.86:0/1935588591 conn(0x7f65c00b2360 msgr2=0x7f65c00b00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:52.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.362+0000 7f65c6e00640 1 -- 172.21.15.86:0/1935588591 shutdown_connections 2024-04-20T21:22:52.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:52.362+0000 7f65c6e00640 1 -- 172.21.15.86:0/1935588591 wait complete. 2024-04-20T21:22:52.381 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":219,"stamp":"2024-04-20T21:22:50.732372+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592104,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":191,"num_write_kb":2427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":128,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257356,"kb_used_data":7888,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754717364,"statfs":{"total":773094113280,"available":772830580736,"internally_reserved":0,"allocated":8077312,"data_stored":4059983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1736,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001508"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.989723+0000","last_change":"2024-04-20T21:22:44.989723+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.989723+0000","last_undegraded":"2024-04-20T21:22:44.989723+0000","last_fullsized":"2024-04-20T21:22:44.989723+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.972721+0000","last_change":"2024-04-20T21:22:44.972721+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.972721+0000","last_undegraded":"2024-04-20T21:22:44.972721+0000","last_fullsized":"2024-04-20T21:22:44.972721+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.358114+0000","last_change":"2024-04-20T21:22:46.000461+0000","last_active":"2024-04-20T21:22:47.358114+0000","last_peered":"2024-04-20T21:22:47.358114+0000","last_clean":"2024-04-20T21:22:47.358114+0000","last_became_active":"2024-04-20T21:22:46.000188+0000","last_became_peered":"2024-04-20T21:22:46.000188+0000","last_unstale":"2024-04-20T21:22:47.358114+0000","last_undegraded":"2024-04-20T21:22:47.358114+0000","last_fullsized":"2024-04-20T21:22:47.358114+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:27:38.679422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.996119+0000","last_change":"2024-04-20T21:22:45.996176+0000","last_active":"2024-04-20T21:22:46.996119+0000","last_peered":"2024-04-20T21:22:46.996119+0000","last_clean":"2024-04-20T21:22:46.996119+0000","last_became_active":"2024-04-20T21:22:45.995860+0000","last_became_peered":"2024-04-20T21:22:45.995860+0000","last_unstale":"2024-04-20T21:22:46.996119+0000","last_undegraded":"2024-04-20T21:22:46.996119+0000","last_fullsized":"2024-04-20T21:22:46.996119+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:19:56.685552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.357267+0000","last_change":"2024-04-20T21:22:46.003219+0000","last_active":"2024-04-20T21:22:47.357267+0000","last_peered":"2024-04-20T21:22:47.357267+0000","last_clean":"2024-04-20T21:22:47.357267+0000","last_became_active":"2024-04-20T21:22:46.002927+0000","last_became_peered":"2024-04-20T21:22:46.002927+0000","last_unstale":"2024-04-20T21:22:47.357267+0000","last_undegraded":"2024-04-20T21:22:47.357267+0000","last_fullsized":"2024-04-20T21:22:47.357267+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:18:23.304303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.976258+0000","last_change":"2024-04-20T21:22:44.976258+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.976258+0000","last_undegraded":"2024-04-20T21:22:44.976258+0000","last_fullsized":"2024-04-20T21:22:44.976258+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.014778+0000","last_change":"2024-04-20T21:22:45.994406+0000","last_active":"2024-04-20T21:22:47.014778+0000","last_peered":"2024-04-20T21:22:47.014778+0000","last_clean":"2024-04-20T21:22:47.014778+0000","last_became_active":"2024-04-20T21:22:45.994109+0000","last_became_peered":"2024-04-20T21:22:45.994109+0000","last_unstale":"2024-04-20T21:22:47.014778+0000","last_undegraded":"2024-04-20T21:22:47.014778+0000","last_fullsized":"2024-04-20T21:22:47.014778+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:06:06.356878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.971878+0000","last_change":"2024-04-20T21:22:44.971878+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.971878+0000","last_undegraded":"2024-04-20T21:22:44.971878+0000","last_fullsized":"2024-04-20T21:22:44.971878+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.358759+0000","last_change":"2024-04-20T21:22:46.009487+0000","last_active":"2024-04-20T21:22:47.358759+0000","last_peered":"2024-04-20T21:22:47.358759+0000","last_clean":"2024-04-20T21:22:47.358759+0000","last_became_active":"2024-04-20T21:22:46.008835+0000","last_became_peered":"2024-04-20T21:22:46.008835+0000","last_unstale":"2024-04-20T21:22:47.358759+0000","last_undegraded":"2024-04-20T21:22:47.358759+0000","last_fullsized":"2024-04-20T21:22:47.358759+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:19:35.848351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.001975+0000","last_change":"2024-04-20T21:22:46.008121+0000","last_active":"2024-04-20T21:22:47.001975+0000","last_peered":"2024-04-20T21:22:47.001975+0000","last_clean":"2024-04-20T21:22:47.001975+0000","last_became_active":"2024-04-20T21:22:46.007835+0000","last_became_peered":"2024-04-20T21:22:46.007835+0000","last_unstale":"2024-04-20T21:22:47.001975+0000","last_undegraded":"2024-04-20T21:22:47.001975+0000","last_fullsized":"2024-04-20T21:22:47.001975+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:42:09.491911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.989195+0000","last_change":"2024-04-20T21:22:44.989195+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.989195+0000","last_undegraded":"2024-04-20T21:22:44.989195+0000","last_fullsized":"2024-04-20T21:22:44.989195+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.996166+0000","last_change":"2024-04-20T21:22:46.005658+0000","last_active":"2024-04-20T21:22:46.996166+0000","last_peered":"2024-04-20T21:22:46.996166+0000","last_clean":"2024-04-20T21:22:46.996166+0000","last_became_active":"2024-04-20T21:22:46.005346+0000","last_became_peered":"2024-04-20T21:22:46.005346+0000","last_unstale":"2024-04-20T21:22:46.996166+0000","last_undegraded":"2024-04-20T21:22:46.996166+0000","last_fullsized":"2024-04-20T21:22:46.996166+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:35:49.635636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.983752+0000","last_change":"2024-04-20T21:22:44.983752+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.983752+0000","last_undegraded":"2024-04-20T21:22:44.983752+0000","last_fullsized":"2024-04-20T21:22:44.983752+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.357690+0000","last_change":"2024-04-20T21:22:46.001211+0000","last_active":"2024-04-20T21:22:47.357690+0000","last_peered":"2024-04-20T21:22:47.357690+0000","last_clean":"2024-04-20T21:22:47.357690+0000","last_became_active":"2024-04-20T21:22:46.000899+0000","last_became_peered":"2024-04-20T21:22:46.000899+0000","last_unstale":"2024-04-20T21:22:47.357690+0000","last_undegraded":"2024-04-20T21:22:47.357690+0000","last_fullsized":"2024-04-20T21:22:47.357690+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:08:57.585935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.984365+0000","last_change":"2024-04-20T21:22:44.984365+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.984365+0000","last_undegraded":"2024-04-20T21:22:44.984365+0000","last_fullsized":"2024-04-20T21:22:44.984365+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.999077+0000","last_change":"2024-04-20T21:22:45.998551+0000","last_active":"2024-04-20T21:22:46.999077+0000","last_peered":"2024-04-20T21:22:46.999077+0000","last_clean":"2024-04-20T21:22:46.999077+0000","last_became_active":"2024-04-20T21:22:45.996213+0000","last_became_peered":"2024-04-20T21:22:45.996213+0000","last_unstale":"2024-04-20T21:22:46.999077+0000","last_undegraded":"2024-04-20T21:22:46.999077+0000","last_fullsized":"2024-04-20T21:22:46.999077+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:49:36.482159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.358766+0000","last_change":"2024-04-20T21:22:46.013340+0000","last_active":"2024-04-20T21:22:47.358766+0000","last_peered":"2024-04-20T21:22:47.358766+0000","last_clean":"2024-04-20T21:22:47.358766+0000","last_became_active":"2024-04-20T21:22:46.009224+0000","last_became_peered":"2024-04-20T21:22:46.009224+0000","last_unstale":"2024-04-20T21:22:47.358766+0000","last_undegraded":"2024-04-20T21:22:47.358766+0000","last_fullsized":"2024-04-20T21:22:47.358766+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:33:18.596222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.996803+0000","last_change":"2024-04-20T21:22:45.987458+0000","last_active":"2024-04-20T21:22:46.996803+0000","last_peered":"2024-04-20T21:22:46.996803+0000","last_clean":"2024-04-20T21:22:46.996803+0000","last_became_active":"2024-04-20T21:22:45.987046+0000","last_became_peered":"2024-04-20T21:22:45.987046+0000","last_unstale":"2024-04-20T21:22:46.996803+0000","last_undegraded":"2024-04-20T21:22:46.996803+0000","last_fullsized":"2024-04-20T21:22:46.996803+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:09:41.573665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.978703+0000","last_change":"2024-04-20T21:22:44.978703+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.978703+0000","last_undegraded":"2024-04-20T21:22:44.978703+0000","last_fullsized":"2024-04-20T21:22:44.978703+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.986040+0000","last_change":"2024-04-20T21:22:44.986040+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.986040+0000","last_undegraded":"2024-04-20T21:22:44.986040+0000","last_fullsized":"2024-04-20T21:22:44.986040+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.000249+0000","last_change":"2024-04-20T21:22:45.998328+0000","last_active":"2024-04-20T21:22:47.000249+0000","last_peered":"2024-04-20T21:22:47.000249+0000","last_clean":"2024-04-20T21:22:47.000249+0000","last_became_active":"2024-04-20T21:22:45.998016+0000","last_became_peered":"2024-04-20T21:22:45.998016+0000","last_unstale":"2024-04-20T21:22:47.000249+0000","last_undegraded":"2024-04-20T21:22:47.000249+0000","last_fullsized":"2024-04-20T21:22:47.000249+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:53:42.792540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.999681+0000","last_change":"2024-04-20T21:22:46.006835+0000","last_active":"2024-04-20T21:22:46.999681+0000","last_peered":"2024-04-20T21:22:46.999681+0000","last_clean":"2024-04-20T21:22:46.999681+0000","last_became_active":"2024-04-20T21:22:46.006544+0000","last_became_peered":"2024-04-20T21:22:46.006544+0000","last_unstale":"2024-04-20T21:22:46.999681+0000","last_undegraded":"2024-04-20T21:22:46.999681+0000","last_fullsized":"2024-04-20T21:22:46.999681+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:05:10.087962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.985232+0000","last_change":"2024-04-20T21:22:44.985232+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.985232+0000","last_undegraded":"2024-04-20T21:22:44.985232+0000","last_fullsized":"2024-04-20T21:22:44.985232+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.983826+0000","last_change":"2024-04-20T21:22:44.983826+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.983826+0000","last_undegraded":"2024-04-20T21:22:44.983826+0000","last_fullsized":"2024-04-20T21:22:44.983826+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.001458+0000","last_change":"2024-04-20T21:22:46.009435+0000","last_active":"2024-04-20T21:22:47.001458+0000","last_peered":"2024-04-20T21:22:47.001458+0000","last_clean":"2024-04-20T21:22:47.001458+0000","last_became_active":"2024-04-20T21:22:46.008653+0000","last_became_peered":"2024-04-20T21:22:46.008653+0000","last_unstale":"2024-04-20T21:22:47.001458+0000","last_undegraded":"2024-04-20T21:22:47.001458+0000","last_fullsized":"2024-04-20T21:22:47.001458+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:58:27.202522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.015568+0000","last_change":"2024-04-20T21:22:45.999532+0000","last_active":"2024-04-20T21:22:47.015568+0000","last_peered":"2024-04-20T21:22:47.015568+0000","last_clean":"2024-04-20T21:22:47.015568+0000","last_became_active":"2024-04-20T21:22:45.999223+0000","last_became_peered":"2024-04-20T21:22:45.999223+0000","last_unstale":"2024-04-20T21:22:47.015568+0000","last_undegraded":"2024-04-20T21:22:47.015568+0000","last_fullsized":"2024-04-20T21:22:47.015568+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:55:14.180769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.358026+0000","last_change":"2024-04-20T21:22:46.002322+0000","last_active":"2024-04-20T21:22:47.358026+0000","last_peered":"2024-04-20T21:22:47.358026+0000","last_clean":"2024-04-20T21:22:47.358026+0000","last_became_active":"2024-04-20T21:22:46.002035+0000","last_became_peered":"2024-04-20T21:22:46.002035+0000","last_unstale":"2024-04-20T21:22:47.358026+0000","last_undegraded":"2024-04-20T21:22:47.358026+0000","last_fullsized":"2024-04-20T21:22:47.358026+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:47:00.839639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.001277+0000","last_change":"2024-04-20T21:22:46.006624+0000","last_active":"2024-04-20T21:22:47.001277+0000","last_peered":"2024-04-20T21:22:47.001277+0000","last_clean":"2024-04-20T21:22:47.001277+0000","last_became_active":"2024-04-20T21:22:46.006273+0000","last_became_peered":"2024-04-20T21:22:46.006273+0000","last_unstale":"2024-04-20T21:22:47.001277+0000","last_undegraded":"2024-04-20T21:22:47.001277+0000","last_fullsized":"2024-04-20T21:22:47.001277+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:06:06.356878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.357002+0000","last_change":"2024-04-20T21:22:45.998542+0000","last_active":"2024-04-20T21:22:47.357002+0000","last_peered":"2024-04-20T21:22:47.357002+0000","last_clean":"2024-04-20T21:22:47.357002+0000","last_became_active":"2024-04-20T21:22:45.998274+0000","last_became_peered":"2024-04-20T21:22:45.998274+0000","last_unstale":"2024-04-20T21:22:47.357002+0000","last_undegraded":"2024-04-20T21:22:47.357002+0000","last_fullsized":"2024-04-20T21:22:47.357002+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:17:21.685206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.014934+0000","last_change":"2024-04-20T21:22:45.995816+0000","last_active":"2024-04-20T21:22:47.014934+0000","last_peered":"2024-04-20T21:22:47.014934+0000","last_clean":"2024-04-20T21:22:47.014934+0000","last_became_active":"2024-04-20T21:22:45.994917+0000","last_became_peered":"2024-04-20T21:22:45.994917+0000","last_unstale":"2024-04-20T21:22:47.014934+0000","last_undegraded":"2024-04-20T21:22:47.014934+0000","last_fullsized":"2024-04-20T21:22:47.014934+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:53:17.849382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.015118+0000","last_change":"2024-04-20T21:22:46.001790+0000","last_active":"2024-04-20T21:22:47.015118+0000","last_peered":"2024-04-20T21:22:47.015118+0000","last_clean":"2024-04-20T21:22:47.015118+0000","last_became_active":"2024-04-20T21:22:46.001494+0000","last_became_peered":"2024-04-20T21:22:46.001494+0000","last_unstale":"2024-04-20T21:22:47.015118+0000","last_undegraded":"2024-04-20T21:22:47.015118+0000","last_fullsized":"2024-04-20T21:22:47.015118+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:42:09.491911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.970617+0000","last_change":"2024-04-20T21:22:44.970617+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.970617+0000","last_undegraded":"2024-04-20T21:22:44.970617+0000","last_fullsized":"2024-04-20T21:22:44.970617+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.982545+0000","last_change":"2024-04-20T21:22:44.982545+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.982545+0000","last_undegraded":"2024-04-20T21:22:44.982545+0000","last_fullsized":"2024-04-20T21:22:44.982545+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:45.004990+0000","last_change":"2024-04-20T21:22:45.000877+0000","last_active":"2024-04-20T21:22:45.004990+0000","last_peered":"2024-04-20T21:22:45.004990+0000","last_clean":"2024-04-20T21:22:45.004990+0000","last_became_active":"2024-04-20T21:22:45.000402+0000","last_became_peered":"2024-04-20T21:22:45.000402+0000","last_unstale":"2024-04-20T21:22:45.004990+0000","last_undegraded":"2024-04-20T21:22:45.004990+0000","last_fullsized":"2024-04-20T21:22:45.004990+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:56:13.768402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.358957+0000","last_change":"2024-04-20T21:22:46.007646+0000","last_active":"2024-04-20T21:22:47.358957+0000","last_peered":"2024-04-20T21:22:47.358957+0000","last_clean":"2024-04-20T21:22:47.358957+0000","last_became_active":"2024-04-20T21:22:46.007304+0000","last_became_peered":"2024-04-20T21:22:46.007304+0000","last_unstale":"2024-04-20T21:22:47.358957+0000","last_undegraded":"2024-04-20T21:22:47.358957+0000","last_fullsized":"2024-04-20T21:22:47.358957+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:45:53.505415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.999915+0000","last_change":"2024-04-20T21:22:44.998432+0000","last_active":"2024-04-20T21:22:44.999915+0000","last_peered":"2024-04-20T21:22:44.999915+0000","last_clean":"2024-04-20T21:22:44.999915+0000","last_became_active":"2024-04-20T21:22:44.998006+0000","last_became_peered":"2024-04-20T21:22:44.998006+0000","last_unstale":"2024-04-20T21:22:44.999915+0000","last_undegraded":"2024-04-20T21:22:44.999915+0000","last_fullsized":"2024-04-20T21:22:44.999915+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:53:04.669954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.979534+0000","last_change":"2024-04-20T21:22:44.979534+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.979534+0000","last_undegraded":"2024-04-20T21:22:44.979534+0000","last_fullsized":"2024-04-20T21:22:44.979534+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"70'2","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.001646+0000","last_change":"2024-04-20T21:22:44.997985+0000","last_active":"2024-04-20T21:22:47.001646+0000","last_peered":"2024-04-20T21:22:47.001646+0000","last_clean":"2024-04-20T21:22:47.001646+0000","last_became_active":"2024-04-20T21:22:44.997672+0000","last_became_peered":"2024-04-20T21:22:44.997672+0000","last_unstale":"2024-04-20T21:22:47.001646+0000","last_undegraded":"2024-04-20T21:22:47.001646+0000","last_fullsized":"2024-04-20T21:22:47.001646+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:10:01.129225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.988718+0000","last_change":"2024-04-20T21:22:44.988718+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.988718+0000","last_undegraded":"2024-04-20T21:22:44.988718+0000","last_fullsized":"2024-04-20T21:22:44.988718+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.996650+0000","last_change":"2024-04-20T21:22:44.989343+0000","last_active":"2024-04-20T21:22:44.996650+0000","last_peered":"2024-04-20T21:22:44.996650+0000","last_clean":"2024-04-20T21:22:44.996650+0000","last_became_active":"2024-04-20T21:22:44.989022+0000","last_became_peered":"2024-04-20T21:22:44.989022+0000","last_unstale":"2024-04-20T21:22:44.996650+0000","last_undegraded":"2024-04-20T21:22:44.996650+0000","last_fullsized":"2024-04-20T21:22:44.996650+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:45:48.879276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.014909+0000","last_change":"2024-04-20T21:22:46.004052+0000","last_active":"2024-04-20T21:22:47.014909+0000","last_peered":"2024-04-20T21:22:47.014909+0000","last_clean":"2024-04-20T21:22:47.014909+0000","last_became_active":"2024-04-20T21:22:46.002637+0000","last_became_peered":"2024-04-20T21:22:46.002637+0000","last_unstale":"2024-04-20T21:22:47.014909+0000","last_undegraded":"2024-04-20T21:22:47.014909+0000","last_fullsized":"2024-04-20T21:22:47.014909+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:58:27.202522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"70'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.357559+0000","last_change":"2024-04-20T21:22:44.994866+0000","last_active":"2024-04-20T21:22:47.357559+0000","last_peered":"2024-04-20T21:22:47.357559+0000","last_clean":"2024-04-20T21:22:47.357559+0000","last_became_active":"2024-04-20T21:22:44.994089+0000","last_became_peered":"2024-04-20T21:22:44.994089+0000","last_unstale":"2024-04-20T21:22:47.357559+0000","last_undegraded":"2024-04-20T21:22:47.357559+0000","last_fullsized":"2024-04-20T21:22:47.357559+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:42:08.475979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.000407+0000","last_change":"2024-04-20T21:22:46.008256+0000","last_active":"2024-04-20T21:22:47.000407+0000","last_peered":"2024-04-20T21:22:47.000407+0000","last_clean":"2024-04-20T21:22:47.000407+0000","last_became_active":"2024-04-20T21:22:46.007960+0000","last_became_peered":"2024-04-20T21:22:46.007960+0000","last_unstale":"2024-04-20T21:22:47.000407+0000","last_undegraded":"2024-04-20T21:22:47.000407+0000","last_fullsized":"2024-04-20T21:22:47.000407+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:28:56.572056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.999221+0000","last_change":"2024-04-20T21:22:44.995756+0000","last_active":"2024-04-20T21:22:44.999221+0000","last_peered":"2024-04-20T21:22:44.999221+0000","last_clean":"2024-04-20T21:22:44.999221+0000","last_became_active":"2024-04-20T21:22:44.995451+0000","last_became_peered":"2024-04-20T21:22:44.995451+0000","last_unstale":"2024-04-20T21:22:44.999221+0000","last_undegraded":"2024-04-20T21:22:44.999221+0000","last_fullsized":"2024-04-20T21:22:44.999221+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:37:36.078254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.356960+0000","last_change":"2024-04-20T21:22:46.004215+0000","last_active":"2024-04-20T21:22:47.356960+0000","last_peered":"2024-04-20T21:22:47.356960+0000","last_clean":"2024-04-20T21:22:47.356960+0000","last_became_active":"2024-04-20T21:22:46.003804+0000","last_became_peered":"2024-04-20T21:22:46.003804+0000","last_unstale":"2024-04-20T21:22:47.356960+0000","last_undegraded":"2024-04-20T21:22:47.356960+0000","last_fullsized":"2024-04-20T21:22:47.356960+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:45:49.895208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.995960+0000","last_change":"2024-04-20T21:22:44.996824+0000","last_active":"2024-04-20T21:22:46.995960+0000","last_peered":"2024-04-20T21:22:46.995960+0000","last_clean":"2024-04-20T21:22:46.995960+0000","last_became_active":"2024-04-20T21:22:44.996531+0000","last_became_peered":"2024-04-20T21:22:44.996531+0000","last_unstale":"2024-04-20T21:22:46.995960+0000","last_undegraded":"2024-04-20T21:22:46.995960+0000","last_fullsized":"2024-04-20T21:22:46.995960+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:01:19.361992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.983784+0000","last_change":"2024-04-20T21:22:44.983784+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.983784+0000","last_undegraded":"2024-04-20T21:22:44.983784+0000","last_fullsized":"2024-04-20T21:22:44.983784+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.356944+0000","last_change":"2024-04-20T21:22:44.982668+0000","last_active":"2024-04-20T21:22:47.356944+0000","last_peered":"2024-04-20T21:22:47.356944+0000","last_clean":"2024-04-20T21:22:47.356944+0000","last_became_active":"2024-04-20T21:22:44.982035+0000","last_became_peered":"2024-04-20T21:22:44.982035+0000","last_unstale":"2024-04-20T21:22:47.356944+0000","last_undegraded":"2024-04-20T21:22:47.356944+0000","last_fullsized":"2024-04-20T21:22:47.356944+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:04:12.006293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.015640+0000","last_change":"2024-04-20T21:22:46.005753+0000","last_active":"2024-04-20T21:22:47.015640+0000","last_peered":"2024-04-20T21:22:47.015640+0000","last_clean":"2024-04-20T21:22:47.015640+0000","last_became_active":"2024-04-20T21:22:46.005305+0000","last_became_peered":"2024-04-20T21:22:46.005305+0000","last_unstale":"2024-04-20T21:22:47.015640+0000","last_undegraded":"2024-04-20T21:22:47.015640+0000","last_fullsized":"2024-04-20T21:22:47.015640+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:27:57.913146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.001557+0000","last_change":"2024-04-20T21:22:44.994181+0000","last_active":"2024-04-20T21:22:47.001557+0000","last_peered":"2024-04-20T21:22:47.001557+0000","last_clean":"2024-04-20T21:22:47.001557+0000","last_became_active":"2024-04-20T21:22:44.992861+0000","last_became_peered":"2024-04-20T21:22:44.992861+0000","last_unstale":"2024-04-20T21:22:47.001557+0000","last_undegraded":"2024-04-20T21:22:47.001557+0000","last_fullsized":"2024-04-20T21:22:47.001557+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:52:23.085259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.988271+0000","last_change":"2024-04-20T21:22:44.988271+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.988271+0000","last_undegraded":"2024-04-20T21:22:44.988271+0000","last_fullsized":"2024-04-20T21:22:44.988271+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.999321+0000","last_change":"2024-04-20T21:22:44.992156+0000","last_active":"2024-04-20T21:22:44.999321+0000","last_peered":"2024-04-20T21:22:44.999321+0000","last_clean":"2024-04-20T21:22:44.999321+0000","last_became_active":"2024-04-20T21:22:44.991749+0000","last_became_peered":"2024-04-20T21:22:44.991749+0000","last_unstale":"2024-04-20T21:22:44.999321+0000","last_undegraded":"2024-04-20T21:22:44.999321+0000","last_fullsized":"2024-04-20T21:22:44.999321+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:59:10.369089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.977417+0000","last_change":"2024-04-20T21:22:44.977417+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.977417+0000","last_undegraded":"2024-04-20T21:22:44.977417+0000","last_fullsized":"2024-04-20T21:22:44.977417+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"70'3","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.358192+0000","last_change":"2024-04-20T21:22:44.990563+0000","last_active":"2024-04-20T21:22:47.358192+0000","last_peered":"2024-04-20T21:22:47.358192+0000","last_clean":"2024-04-20T21:22:47.358192+0000","last_became_active":"2024-04-20T21:22:44.990292+0000","last_became_peered":"2024-04-20T21:22:44.990292+0000","last_unstale":"2024-04-20T21:22:47.358192+0000","last_undegraded":"2024-04-20T21:22:47.358192+0000","last_fullsized":"2024-04-20T21:22:47.358192+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:55:37.925344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.357584+0000","last_change":"2024-04-20T21:22:45.997098+0000","last_active":"2024-04-20T21:22:47.357584+0000","last_peered":"2024-04-20T21:22:47.357584+0000","last_clean":"2024-04-20T21:22:47.357584+0000","last_became_active":"2024-04-20T21:22:45.996820+0000","last_became_peered":"2024-04-20T21:22:45.996820+0000","last_unstale":"2024-04-20T21:22:47.357584+0000","last_undegraded":"2024-04-20T21:22:47.357584+0000","last_fullsized":"2024-04-20T21:22:47.357584+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:59:11.385021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"70'6","reported_seq":20,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.996263+0000","last_change":"2024-04-20T21:22:44.988227+0000","last_active":"2024-04-20T21:22:46.996263+0000","last_peered":"2024-04-20T21:22:46.996263+0000","last_clean":"2024-04-20T21:22:46.996263+0000","last_became_active":"2024-04-20T21:22:44.987939+0000","last_became_peered":"2024-04-20T21:22:44.987939+0000","last_unstale":"2024-04-20T21:22:46.996263+0000","last_undegraded":"2024-04-20T21:22:46.996263+0000","last_fullsized":"2024-04-20T21:22:46.996263+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:46:00.265362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.985271+0000","last_change":"2024-04-20T21:22:44.985271+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.985271+0000","last_undegraded":"2024-04-20T21:22:44.985271+0000","last_fullsized":"2024-04-20T21:22:44.985271+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"70'2","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.996842+0000","last_change":"2024-04-20T21:22:44.992317+0000","last_active":"2024-04-20T21:22:46.996842+0000","last_peered":"2024-04-20T21:22:46.996842+0000","last_clean":"2024-04-20T21:22:46.996842+0000","last_became_active":"2024-04-20T21:22:44.992040+0000","last_became_peered":"2024-04-20T21:22:44.992040+0000","last_unstale":"2024-04-20T21:22:46.996842+0000","last_undegraded":"2024-04-20T21:22:46.996842+0000","last_fullsized":"2024-04-20T21:22:46.996842+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:50:24.320805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.015238+0000","last_change":"2024-04-20T21:22:45.998870+0000","last_active":"2024-04-20T21:22:47.015238+0000","last_peered":"2024-04-20T21:22:47.015238+0000","last_clean":"2024-04-20T21:22:47.015238+0000","last_became_active":"2024-04-20T21:22:45.998537+0000","last_became_peered":"2024-04-20T21:22:45.998537+0000","last_unstale":"2024-04-20T21:22:47.015238+0000","last_undegraded":"2024-04-20T21:22:47.015238+0000","last_fullsized":"2024-04-20T21:22:47.015238+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:55:38.941276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.992803+0000","last_change":"2024-04-20T21:22:44.987324+0000","last_active":"2024-04-20T21:22:44.992803+0000","last_peered":"2024-04-20T21:22:44.992803+0000","last_clean":"2024-04-20T21:22:44.992803+0000","last_became_active":"2024-04-20T21:22:44.986974+0000","last_became_peered":"2024-04-20T21:22:44.986974+0000","last_unstale":"2024-04-20T21:22:44.992803+0000","last_undegraded":"2024-04-20T21:22:44.992803+0000","last_fullsized":"2024-04-20T21:22:44.992803+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:40:38.463867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.983547+0000","last_change":"2024-04-20T21:22:44.983547+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.983547+0000","last_undegraded":"2024-04-20T21:22:44.983547+0000","last_fullsized":"2024-04-20T21:22:44.983547+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.014700+0000","last_change":"2024-04-20T21:22:44.997932+0000","last_active":"2024-04-20T21:22:47.014700+0000","last_peered":"2024-04-20T21:22:47.014700+0000","last_clean":"2024-04-20T21:22:47.014700+0000","last_became_active":"2024-04-20T21:22:44.997397+0000","last_became_peered":"2024-04-20T21:22:44.997397+0000","last_unstale":"2024-04-20T21:22:47.014700+0000","last_undegraded":"2024-04-20T21:22:47.014700+0000","last_fullsized":"2024-04-20T21:22:47.014700+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:56:13.768402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.357344+0000","last_change":"2024-04-20T21:22:45.997195+0000","last_active":"2024-04-20T21:22:47.357344+0000","last_peered":"2024-04-20T21:22:47.357344+0000","last_clean":"2024-04-20T21:22:47.357344+0000","last_became_active":"2024-04-20T21:22:45.996696+0000","last_became_peered":"2024-04-20T21:22:45.996696+0000","last_unstale":"2024-04-20T21:22:47.357344+0000","last_undegraded":"2024-04-20T21:22:47.357344+0000","last_fullsized":"2024-04-20T21:22:47.357344+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:40:15.771704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"70'8","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.998939+0000","last_change":"2024-04-20T21:22:44.995360+0000","last_active":"2024-04-20T21:22:46.998939+0000","last_peered":"2024-04-20T21:22:46.998939+0000","last_clean":"2024-04-20T21:22:46.998939+0000","last_became_active":"2024-04-20T21:22:44.995055+0000","last_became_peered":"2024-04-20T21:22:44.995055+0000","last_unstale":"2024-04-20T21:22:46.998939+0000","last_undegraded":"2024-04-20T21:22:46.998939+0000","last_fullsized":"2024-04-20T21:22:46.998939+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:14:22.771721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.001952+0000","last_change":"2024-04-20T21:22:45.993981+0000","last_active":"2024-04-20T21:22:47.001952+0000","last_peered":"2024-04-20T21:22:47.001952+0000","last_clean":"2024-04-20T21:22:47.001952+0000","last_became_active":"2024-04-20T21:22:45.993203+0000","last_became_peered":"2024-04-20T21:22:45.993203+0000","last_unstale":"2024-04-20T21:22:47.001952+0000","last_undegraded":"2024-04-20T21:22:47.001952+0000","last_fullsized":"2024-04-20T21:22:47.001952+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:53:17.849382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.999385+0000","last_change":"2024-04-20T21:22:44.993745+0000","last_active":"2024-04-20T21:22:44.999385+0000","last_peered":"2024-04-20T21:22:44.999385+0000","last_clean":"2024-04-20T21:22:44.999385+0000","last_became_active":"2024-04-20T21:22:44.992981+0000","last_became_peered":"2024-04-20T21:22:44.992981+0000","last_unstale":"2024-04-20T21:22:44.999385+0000","last_undegraded":"2024-04-20T21:22:44.999385+0000","last_fullsized":"2024-04-20T21:22:44.999385+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:08:56.570003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.987280+0000","last_change":"2024-04-20T21:22:44.987280+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.987280+0000","last_undegraded":"2024-04-20T21:22:44.987280+0000","last_fullsized":"2024-04-20T21:22:44.987280+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"70'2","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.002056+0000","last_change":"2024-04-20T21:22:44.998796+0000","last_active":"2024-04-20T21:22:47.002056+0000","last_peered":"2024-04-20T21:22:47.002056+0000","last_clean":"2024-04-20T21:22:47.002056+0000","last_became_active":"2024-04-20T21:22:44.998424+0000","last_became_peered":"2024-04-20T21:22:44.998424+0000","last_unstale":"2024-04-20T21:22:47.002056+0000","last_undegraded":"2024-04-20T21:22:47.002056+0000","last_fullsized":"2024-04-20T21:22:47.002056+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:45:10.580900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.996286+0000","last_change":"2024-04-20T21:22:45.999490+0000","last_active":"2024-04-20T21:22:46.996286+0000","last_peered":"2024-04-20T21:22:46.996286+0000","last_clean":"2024-04-20T21:22:46.996286+0000","last_became_active":"2024-04-20T21:22:45.999118+0000","last_became_peered":"2024-04-20T21:22:45.999118+0000","last_unstale":"2024-04-20T21:22:46.996286+0000","last_undegraded":"2024-04-20T21:22:46.996286+0000","last_fullsized":"2024-04-20T21:22:46.996286+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:55:03.918649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:45.003213+0000","last_change":"2024-04-20T21:22:45.000092+0000","last_active":"2024-04-20T21:22:45.003213+0000","last_peered":"2024-04-20T21:22:45.003213+0000","last_clean":"2024-04-20T21:22:45.003213+0000","last_became_active":"2024-04-20T21:22:44.998998+0000","last_became_peered":"2024-04-20T21:22:44.998998+0000","last_unstale":"2024-04-20T21:22:45.003213+0000","last_undegraded":"2024-04-20T21:22:45.003213+0000","last_fullsized":"2024-04-20T21:22:45.003213+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:19:34.832419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.358172+0000","last_change":"2024-04-20T21:22:46.001345+0000","last_active":"2024-04-20T21:22:47.358172+0000","last_peered":"2024-04-20T21:22:47.358172+0000","last_clean":"2024-04-20T21:22:47.358172+0000","last_became_active":"2024-04-20T21:22:46.001055+0000","last_became_peered":"2024-04-20T21:22:46.001055+0000","last_unstale":"2024-04-20T21:22:47.358172+0000","last_undegraded":"2024-04-20T21:22:47.358172+0000","last_fullsized":"2024-04-20T21:22:47.358172+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:48:59.404081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:45.007767+0000","last_change":"2024-04-20T21:22:45.002908+0000","last_active":"2024-04-20T21:22:45.007767+0000","last_peered":"2024-04-20T21:22:45.007767+0000","last_clean":"2024-04-20T21:22:45.007767+0000","last_became_active":"2024-04-20T21:22:45.002596+0000","last_became_peered":"2024-04-20T21:22:45.002596+0000","last_unstale":"2024-04-20T21:22:45.007767+0000","last_undegraded":"2024-04-20T21:22:45.007767+0000","last_fullsized":"2024-04-20T21:22:45.007767+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:07:15.232666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.357387+0000","last_change":"2024-04-20T21:22:46.004184+0000","last_active":"2024-04-20T21:22:47.357387+0000","last_peered":"2024-04-20T21:22:47.357387+0000","last_clean":"2024-04-20T21:22:47.357387+0000","last_became_active":"2024-04-20T21:22:46.003915+0000","last_became_peered":"2024-04-20T21:22:46.003915+0000","last_unstale":"2024-04-20T21:22:47.357387+0000","last_undegraded":"2024-04-20T21:22:47.357387+0000","last_fullsized":"2024-04-20T21:22:47.357387+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:40:38.260253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:45.000938+0000","last_change":"2024-04-20T21:22:44.996534+0000","last_active":"2024-04-20T21:22:45.000938+0000","last_peered":"2024-04-20T21:22:45.000938+0000","last_clean":"2024-04-20T21:22:45.000938+0000","last_became_active":"2024-04-20T21:22:44.996190+0000","last_became_peered":"2024-04-20T21:22:44.996190+0000","last_unstale":"2024-04-20T21:22:45.000938+0000","last_undegraded":"2024-04-20T21:22:45.000938+0000","last_fullsized":"2024-04-20T21:22:45.000938+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:48:58.388149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.356519+0000","last_change":"2024-04-20T21:22:46.005627+0000","last_active":"2024-04-20T21:22:47.356519+0000","last_peered":"2024-04-20T21:22:47.356519+0000","last_clean":"2024-04-20T21:22:47.356519+0000","last_became_active":"2024-04-20T21:22:46.005377+0000","last_became_peered":"2024-04-20T21:22:46.005377+0000","last_unstale":"2024-04-20T21:22:47.356519+0000","last_undegraded":"2024-04-20T21:22:47.356519+0000","last_fullsized":"2024-04-20T21:22:47.356519+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:06:18.941092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.015474+0000","last_change":"2024-04-20T21:22:44.993852+0000","last_active":"2024-04-20T21:22:47.015474+0000","last_peered":"2024-04-20T21:22:47.015474+0000","last_clean":"2024-04-20T21:22:47.015474+0000","last_became_active":"2024-04-20T21:22:44.993491+0000","last_became_peered":"2024-04-20T21:22:44.993491+0000","last_unstale":"2024-04-20T21:22:47.015474+0000","last_undegraded":"2024-04-20T21:22:47.015474+0000","last_fullsized":"2024-04-20T21:22:47.015474+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:07:15.232666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.356646+0000","last_change":"2024-04-20T21:22:46.001218+0000","last_active":"2024-04-20T21:22:47.356646+0000","last_peered":"2024-04-20T21:22:47.356646+0000","last_clean":"2024-04-20T21:22:47.356646+0000","last_became_active":"2024-04-20T21:22:46.000900+0000","last_became_peered":"2024-04-20T21:22:46.000900+0000","last_unstale":"2024-04-20T21:22:47.356646+0000","last_undegraded":"2024-04-20T21:22:47.356646+0000","last_fullsized":"2024-04-20T21:22:47.356646+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:53:21.061854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.994517+0000","last_change":"2024-04-20T21:22:44.992558+0000","last_active":"2024-04-20T21:22:44.994517+0000","last_peered":"2024-04-20T21:22:44.994517+0000","last_clean":"2024-04-20T21:22:44.994517+0000","last_became_active":"2024-04-20T21:22:44.991402+0000","last_became_peered":"2024-04-20T21:22:44.991402+0000","last_unstale":"2024-04-20T21:22:44.994517+0000","last_undegraded":"2024-04-20T21:22:44.994517+0000","last_fullsized":"2024-04-20T21:22:44.994517+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:47:51.476465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.985660+0000","last_change":"2024-04-20T21:22:44.985660+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.985660+0000","last_undegraded":"2024-04-20T21:22:44.985660+0000","last_fullsized":"2024-04-20T21:22:44.985660+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.996911+0000","last_change":"2024-04-20T21:22:44.990230+0000","last_active":"2024-04-20T21:22:44.996911+0000","last_peered":"2024-04-20T21:22:44.996911+0000","last_clean":"2024-04-20T21:22:44.996911+0000","last_became_active":"2024-04-20T21:22:44.989958+0000","last_became_peered":"2024-04-20T21:22:44.989958+0000","last_unstale":"2024-04-20T21:22:44.996911+0000","last_undegraded":"2024-04-20T21:22:44.996911+0000","last_fullsized":"2024-04-20T21:22:44.996911+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:18:22.288371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.015250+0000","last_change":"2024-04-20T21:22:45.993591+0000","last_active":"2024-04-20T21:22:47.015250+0000","last_peered":"2024-04-20T21:22:47.015250+0000","last_clean":"2024-04-20T21:22:47.015250+0000","last_became_active":"2024-04-20T21:22:45.993320+0000","last_became_peered":"2024-04-20T21:22:45.993320+0000","last_unstale":"2024-04-20T21:22:47.015250+0000","last_undegraded":"2024-04-20T21:22:47.015250+0000","last_fullsized":"2024-04-20T21:22:47.015250+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:24:16.195831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"71'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.015880+0000","last_change":"2024-04-20T21:22:44.997554+0000","last_active":"2024-04-20T21:22:47.015880+0000","last_peered":"2024-04-20T21:22:47.015880+0000","last_clean":"2024-04-20T21:22:47.015880+0000","last_became_active":"2024-04-20T21:22:44.997234+0000","last_became_peered":"2024-04-20T21:22:44.997234+0000","last_unstale":"2024-04-20T21:22:47.015880+0000","last_undegraded":"2024-04-20T21:22:47.015880+0000","last_fullsized":"2024-04-20T21:22:47.015880+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:47:51.476465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.986315+0000","last_change":"2024-04-20T21:22:44.986315+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.986315+0000","last_undegraded":"2024-04-20T21:22:44.986315+0000","last_fullsized":"2024-04-20T21:22:44.986315+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.358437+0000","last_change":"2024-04-20T21:22:44.998361+0000","last_active":"2024-04-20T21:22:47.358437+0000","last_peered":"2024-04-20T21:22:47.358437+0000","last_clean":"2024-04-20T21:22:47.358437+0000","last_became_active":"2024-04-20T21:22:44.996313+0000","last_became_peered":"2024-04-20T21:22:44.996313+0000","last_unstale":"2024-04-20T21:22:47.358437+0000","last_undegraded":"2024-04-20T21:22:47.358437+0000","last_fullsized":"2024-04-20T21:22:47.358437+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:57:50.796943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.986997+0000","last_change":"2024-04-20T21:22:44.986997+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.986997+0000","last_undegraded":"2024-04-20T21:22:44.986997+0000","last_fullsized":"2024-04-20T21:22:44.986997+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:45.004224+0000","last_change":"2024-04-20T21:22:45.003069+0000","last_active":"2024-04-20T21:22:45.004224+0000","last_peered":"2024-04-20T21:22:45.004224+0000","last_clean":"2024-04-20T21:22:45.004224+0000","last_became_active":"2024-04-20T21:22:45.002662+0000","last_became_peered":"2024-04-20T21:22:45.002662+0000","last_unstale":"2024-04-20T21:22:45.004224+0000","last_undegraded":"2024-04-20T21:22:45.004224+0000","last_fullsized":"2024-04-20T21:22:45.004224+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:17:48.882110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.357975+0000","last_change":"2024-04-20T21:22:45.998814+0000","last_active":"2024-04-20T21:22:47.357975+0000","last_peered":"2024-04-20T21:22:47.357975+0000","last_clean":"2024-04-20T21:22:47.357975+0000","last_became_active":"2024-04-20T21:22:45.998463+0000","last_became_peered":"2024-04-20T21:22:45.998463+0000","last_unstale":"2024-04-20T21:22:47.357975+0000","last_undegraded":"2024-04-20T21:22:47.357975+0000","last_fullsized":"2024-04-20T21:22:47.357975+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:37:37.094186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.357407+0000","last_change":"2024-04-20T21:22:44.990525+0000","last_active":"2024-04-20T21:22:47.357407+0000","last_peered":"2024-04-20T21:22:47.357407+0000","last_clean":"2024-04-20T21:22:47.357407+0000","last_became_active":"2024-04-20T21:22:44.990259+0000","last_became_peered":"2024-04-20T21:22:44.990259+0000","last_unstale":"2024-04-20T21:22:47.357407+0000","last_undegraded":"2024-04-20T21:22:47.357407+0000","last_fullsized":"2024-04-20T21:22:47.357407+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:07:45.118275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.971783+0000","last_change":"2024-04-20T21:22:44.971783+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.971783+0000","last_undegraded":"2024-04-20T21:22:44.971783+0000","last_fullsized":"2024-04-20T21:22:44.971783+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.995914+0000","last_change":"2024-04-20T21:22:44.996969+0000","last_active":"2024-04-20T21:22:46.995914+0000","last_peered":"2024-04-20T21:22:46.995914+0000","last_clean":"2024-04-20T21:22:46.995914+0000","last_became_active":"2024-04-20T21:22:44.995514+0000","last_became_peered":"2024-04-20T21:22:44.995514+0000","last_unstale":"2024-04-20T21:22:46.995914+0000","last_undegraded":"2024-04-20T21:22:46.995914+0000","last_fullsized":"2024-04-20T21:22:46.995914+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:35:20.490828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.996108+0000","last_change":"2024-04-20T21:22:46.000298+0000","last_active":"2024-04-20T21:22:46.996108+0000","last_peered":"2024-04-20T21:22:46.996108+0000","last_clean":"2024-04-20T21:22:46.996108+0000","last_became_active":"2024-04-20T21:22:46.000027+0000","last_became_peered":"2024-04-20T21:22:46.000027+0000","last_unstale":"2024-04-20T21:22:46.996108+0000","last_undegraded":"2024-04-20T21:22:46.996108+0000","last_fullsized":"2024-04-20T21:22:46.996108+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:53:43.293248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.999123+0000","last_change":"2024-04-20T21:22:44.994701+0000","last_active":"2024-04-20T21:22:46.999123+0000","last_peered":"2024-04-20T21:22:46.999123+0000","last_clean":"2024-04-20T21:22:46.999123+0000","last_became_active":"2024-04-20T21:22:44.994359+0000","last_became_peered":"2024-04-20T21:22:44.994359+0000","last_unstale":"2024-04-20T21:22:46.999123+0000","last_undegraded":"2024-04-20T21:22:46.999123+0000","last_fullsized":"2024-04-20T21:22:46.999123+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:34:52.260354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":69,"state":"creating+peering","last_fresh":"2024-04-20T21:22:44.973852+0000","last_change":"2024-04-20T21:22:44.973852+0000","last_active":"2024-04-20T21:22:44.946779+0000","last_peered":"2024-04-20T21:22:44.946779+0000","last_clean":"2024-04-20T21:22:44.946779+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:22:44.973852+0000","last_undegraded":"2024-04-20T21:22:44.973852+0000","last_fullsized":"2024-04-20T21:22:44.973852+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.996863+0000","last_change":"2024-04-20T21:22:44.990936+0000","last_active":"2024-04-20T21:22:44.996863+0000","last_peered":"2024-04-20T21:22:44.996863+0000","last_clean":"2024-04-20T21:22:44.996863+0000","last_became_active":"2024-04-20T21:22:44.988494+0000","last_became_peered":"2024-04-20T21:22:44.988494+0000","last_unstale":"2024-04-20T21:22:44.996863+0000","last_undegraded":"2024-04-20T21:22:44.996863+0000","last_fullsized":"2024-04-20T21:22:44.996863+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:40:14.755772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.357272+0000","last_change":"2024-04-20T21:22:46.007192+0000","last_active":"2024-04-20T21:22:47.357272+0000","last_peered":"2024-04-20T21:22:47.357272+0000","last_clean":"2024-04-20T21:22:47.357272+0000","last_became_active":"2024-04-20T21:22:46.006583+0000","last_became_peered":"2024-04-20T21:22:46.006583+0000","last_unstale":"2024-04-20T21:22:47.357272+0000","last_undegraded":"2024-04-20T21:22:47.357272+0000","last_fullsized":"2024-04-20T21:22:47.357272+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:28:45.591146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.993111+0000","last_change":"2024-04-20T21:22:44.988535+0000","last_active":"2024-04-20T21:22:44.993111+0000","last_peered":"2024-04-20T21:22:44.993111+0000","last_clean":"2024-04-20T21:22:44.993111+0000","last_became_active":"2024-04-20T21:22:44.988183+0000","last_became_peered":"2024-04-20T21:22:44.988183+0000","last_unstale":"2024-04-20T21:22:44.993111+0000","last_undegraded":"2024-04-20T21:22:44.993111+0000","last_fullsized":"2024-04-20T21:22:44.993111+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:40:51.727352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":256,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-20T21:22:44.990279+0000","last_change":"2024-04-20T21:17:43.059637+0000","last_active":"2024-04-20T21:22:44.990279+0000","last_peered":"2024-04-20T21:22:44.990279+0000","last_clean":"2024-04-20T21:22:44.990279+0000","last_became_active":"2024-04-20T21:17:43.056207+0000","last_became_peered":"2024-04-20T21:17:43.056207+0000","last_unstale":"2024-04-20T21:22:44.990279+0000","last_undegraded":"2024-04-20T21:22:44.990279+0000","last_fullsized":"2024-04-20T21:22:44.990279+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:41.036290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:41.036290+0000","last_clean_scrub_stamp":"2024-04-20T21:17:41.036290+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:58:51.091337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":1736,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25,"ondisk_log_size":25,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574911,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32584,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339256,"statfs":{"total":96636764160,"available":96603398144,"internally_reserved":0,"allocated":1433600,"data_stored":941588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574912,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":839680,"data_stored":351220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574915,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":868352,"data_stored":364385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574914,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32036,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339804,"statfs":{"total":96636764160,"available":96603959296,"internally_reserved":0,"allocated":868352,"data_stored":364365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574914,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":872448,"data_stored":364407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574914,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32036,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339804,"statfs":{"total":96636764160,"available":96603959296,"internally_reserved":0,"allocated":876544,"data_stored":365907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32052,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339788,"statfs":{"total":96636764160,"available":96603942912,"internally_reserved":0,"allocated":884736,"data_stored":366523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574913,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32584,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339256,"statfs":{"total":96636764160,"available":96603398144,"internally_reserved":0,"allocated":1433600,"data_stored":941588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T21:22:55.386 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T21:22:55.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.598+0000 7f0447a00640 1 Processor -- start 2024-04-20T21:22:55.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.598+0000 7f0447a00640 1 -- start start 2024-04-20T21:22:55.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.600+0000 7f0447a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f044006e6d0 0x7f0440071d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:55.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.600+0000 7f0447a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f044006eb30 con 0x7f0440072240 2024-04-20T21:22:55.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.600+0000 7f0447a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f044006dc30 con 0x7f0440146130 2024-04-20T21:22:55.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.600+0000 7f0447a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f0440072690 con 0x7f044006e6d0 2024-04-20T21:22:55.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.600+0000 7f043fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f044006e6d0 0x7f0440071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:55.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.600+0000 7f0445a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f0440072240 0x7f044015cc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44766/0 (socket says 172.21.15.86:44766) 2024-04-20T21:22:55.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.600+0000 7f0445a00640 1 -- 172.21.15.86:0/2511776176 learned_addr learned my addr 172.21.15.86:0/2511776176 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:22:55.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.601+0000 7f043fe00640 1 -- 172.21.15.86:0/2511776176 >> v1:172.21.15.144:6789/0 conn(0x7f0440146130 legacy=0x7f04400ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:55.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.601+0000 7f043fe00640 1 -- 172.21.15.86:0/2511776176 >> v1:172.21.15.86:6789/0 conn(0x7f0440072240 legacy=0x7f044015cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:55.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.601+0000 7f043fe00640 1 -- 172.21.15.86:0/2511776176 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04400cd8e0 con 0x7f044006e6d0 2024-04-20T21:22:55.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.602+0000 7f043fe00640 1 --2- 172.21.15.86:0/2511776176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f044006e6d0 0x7f0440071d00 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f0440073120 tx=0x7f042c02b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8920ba78e2e42a2e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:55.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.602+0000 7f043f400640 1 -- 172.21.15.86:0/2511776176 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f042c035020 con 0x7f044006e6d0 2024-04-20T21:22:55.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.602+0000 7f043f400640 1 -- 172.21.15.86:0/2511776176 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f042c031890 con 0x7f044006e6d0 2024-04-20T21:22:55.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.602+0000 7f043f400640 1 -- 172.21.15.86:0/2511776176 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f042c031c60 con 0x7f044006e6d0 2024-04-20T21:22:55.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.603+0000 7f0447a00640 1 -- 172.21.15.86:0/2511776176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f044006e6d0 msgr2=0x7f0440071d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:55.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.603+0000 7f0447a00640 1 --2- 172.21.15.86:0/2511776176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f044006e6d0 0x7f0440071d00 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f0440073120 tx=0x7f042c02b930 comp rx=0 tx=0).stop 2024-04-20T21:22:55.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.603+0000 7f0447a00640 1 -- 172.21.15.86:0/2511776176 shutdown_connections 2024-04-20T21:22:55.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.603+0000 7f0447a00640 1 --2- 172.21.15.86:0/2511776176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f044006e6d0 0x7f0440071d00 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:55.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.603+0000 7f0447a00640 1 -- 172.21.15.86:0/2511776176 >> 172.21.15.86:0/2511776176 conn(0x7f04400b2360 msgr2=0x7f04400b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:55.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.603+0000 7f0447a00640 1 -- 172.21.15.86:0/2511776176 shutdown_connections 2024-04-20T21:22:55.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.603+0000 7f0447a00640 1 -- 172.21.15.86:0/2511776176 wait complete. 2024-04-20T21:22:55.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.605+0000 7f0447a00640 1 Processor -- start 2024-04-20T21:22:55.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.605+0000 7f0447a00640 1 -- start start 2024-04-20T21:22:55.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.606+0000 7f0447a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f044006e6d0 0x7f044013d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:55.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.606+0000 7f0447a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0440072240 0x7f044013f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:55.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.606+0000 7f0447a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0440146130 0x7f04401f61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:55.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.606+0000 7f0447a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f044006e1b0 con 0x7f0440072240 2024-04-20T21:22:55.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.606+0000 7f0447a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f044006ded0 con 0x7f0440146130 2024-04-20T21:22:55.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.606+0000 7f0447a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f044006f150 con 0x7f044006e6d0 2024-04-20T21:22:55.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.606+0000 7f043fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0440072240 0x7f044013f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:55.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.606+0000 7f043fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0440072240 0x7f044013f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52130/0 (socket says 172.21.15.86:52130) 2024-04-20T21:22:55.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.606+0000 7f043fe00640 1 -- 172.21.15.86:0/3059698047 learned_addr learned my addr 172.21.15.86:0/3059698047 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:55.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.607+0000 7f043fe00640 1 -- 172.21.15.86:0/3059698047 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f044006e6d0 msgr2=0x7f044013d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:55.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.607+0000 7f0445a00640 1 --2- 172.21.15.86:0/3059698047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0440146130 0x7f04401f61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:55.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.607+0000 7f0445000640 1 --2- 172.21.15.86:0/3059698047 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f044006e6d0 0x7f044013d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:55.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.607+0000 7f043fe00640 1 --2- 172.21.15.86:0/3059698047 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f044006e6d0 0x7f044013d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:55.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.607+0000 7f043fe00640 1 -- 172.21.15.86:0/3059698047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0440146130 msgr2=0x7f04401f61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:55.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.607+0000 7f043fe00640 1 --2- 172.21.15.86:0/3059698047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0440146130 0x7f04401f61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:55.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.607+0000 7f043fe00640 1 -- 172.21.15.86:0/3059698047 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f044015aa50 con 0x7f0440072240 2024-04-20T21:22:55.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.607+0000 7f0445000640 1 --2- 172.21.15.86:0/3059698047 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f044006e6d0 0x7f044013d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:22:55.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.607+0000 7f043fe00640 1 --2- 172.21.15.86:0/3059698047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0440072240 0x7f044013f230 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f042c009890 tx=0x7f042c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:55.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.607+0000 7f043cc00640 1 -- 172.21.15.86:0/3059698047 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f042c0316b0 con 0x7f0440072240 2024-04-20T21:22:55.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.608+0000 7f043cc00640 1 -- 172.21.15.86:0/3059698047 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f042c0453f0 con 0x7f0440072240 2024-04-20T21:22:55.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.608+0000 7f0447a00640 1 -- 172.21.15.86:0/3059698047 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04401f6dd0 con 0x7f0440072240 2024-04-20T21:22:55.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.608+0000 7f043cc00640 1 -- 172.21.15.86:0/3059698047 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f042c0457e0 con 0x7f0440072240 2024-04-20T21:22:55.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.608+0000 7f0447a00640 1 -- 172.21.15.86:0/3059698047 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f044013c550 con 0x7f0440072240 2024-04-20T21:22:55.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.609+0000 7f0447a00640 1 -- 172.21.15.86:0/3059698047 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04400681f0 con 0x7f0440072240 2024-04-20T21:22:55.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.610+0000 7f043cc00640 1 -- 172.21.15.86:0/3059698047 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f042c038020 con 0x7f0440072240 2024-04-20T21:22:55.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.611+0000 7f043cc00640 1 --2- 172.21.15.86:0/3059698047 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f042006c920 0x7f042006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:55.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.611+0000 7f043cc00640 1 -- 172.21.15.86:0/3059698047 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f042c0adfa0 con 0x7f0440072240 2024-04-20T21:22:55.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.612+0000 7f0445000640 1 --2- 172.21.15.86:0/3059698047 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f042006c920 0x7f042006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:55.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.612+0000 7f0445000640 1 --2- 172.21.15.86:0/3059698047 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f042006c920 0x7f042006ede0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f043400a830 tx=0x7f0434008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:55.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.617+0000 7f043cc00640 1 -- 172.21.15.86:0/3059698047 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f04400681f0 con 0x7f0440072240 2024-04-20T21:22:55.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.813+0000 7f0447a00640 1 -- 172.21.15.86:0/3059698047 --> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f04400689e0 con 0x7f042006c920 2024-04-20T21:22:55.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.824+0000 7f043cc00640 1 -- 172.21.15.86:0/3059698047 <== mgr.4107 v2:172.21.15.86:6832/302045240 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252069 (secure 0 0 0) 0x7f04400689e0 con 0x7f042006c920 2024-04-20T21:22:55.826 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:22:55.828 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-20T21:22:55.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.829+0000 7f0447a00640 1 -- 172.21.15.86:0/3059698047 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f042006c920 msgr2=0x7f042006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:55.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.829+0000 7f0447a00640 1 --2- 172.21.15.86:0/3059698047 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f042006c920 0x7f042006ede0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f043400a830 tx=0x7f0434008040 comp rx=0 tx=0).stop 2024-04-20T21:22:55.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.829+0000 7f0447a00640 1 -- 172.21.15.86:0/3059698047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0440072240 msgr2=0x7f044013f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:55.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.829+0000 7f0447a00640 1 --2- 172.21.15.86:0/3059698047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0440072240 0x7f044013f230 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f042c009890 tx=0x7f042c0046c0 comp rx=0 tx=0).stop 2024-04-20T21:22:55.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.830+0000 7f0447a00640 1 -- 172.21.15.86:0/3059698047 shutdown_connections 2024-04-20T21:22:55.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.830+0000 7f0447a00640 1 --2- 172.21.15.86:0/3059698047 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f042006c920 0x7f042006ede0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:55.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.830+0000 7f0447a00640 1 --2- 172.21.15.86:0/3059698047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0440146130 0x7f04401f61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:55.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.830+0000 7f0447a00640 1 --2- 172.21.15.86:0/3059698047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0440072240 0x7f044013f230 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:55.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.830+0000 7f0447a00640 1 --2- 172.21.15.86:0/3059698047 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f044006e6d0 0x7f044013d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:55.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.830+0000 7f0447a00640 1 -- 172.21.15.86:0/3059698047 >> 172.21.15.86:0/3059698047 conn(0x7f04400b2360 msgr2=0x7f0440143720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:55.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.830+0000 7f0447a00640 1 -- 172.21.15.86:0/3059698047 shutdown_connections 2024-04-20T21:22:55.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:55.830+0000 7f0447a00640 1 -- 172.21.15.86:0/3059698047 wait complete. 2024-04-20T21:22:55.849 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2024-04-20T21:22:54.734266+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257444,"kb_used_data":7976,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754717276,"statfs":{"total":773094113280,"available":772830490624,"internally_reserved":0,"allocated":8167424,"data_stored":4099916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.767289"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.295485+0000","last_change":"2024-04-20T21:22:46.006048+0000","last_active":"2024-04-20T21:22:47.295485+0000","last_peered":"2024-04-20T21:22:47.295485+0000","last_clean":"2024-04-20T21:22:47.295485+0000","last_became_active":"2024-04-20T21:22:46.005701+0000","last_became_peered":"2024-04-20T21:22:46.005701+0000","last_unstale":"2024-04-20T21:22:47.295485+0000","last_undegraded":"2024-04-20T21:22:47.295485+0000","last_fullsized":"2024-04-20T21:22:47.295485+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:06:18.941092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.001003+0000","last_change":"2024-04-20T21:22:45.991592+0000","last_active":"2024-04-20T21:22:47.001003+0000","last_peered":"2024-04-20T21:22:47.001003+0000","last_clean":"2024-04-20T21:22:47.001003+0000","last_became_active":"2024-04-20T21:22:45.990410+0000","last_became_peered":"2024-04-20T21:22:45.990410+0000","last_unstale":"2024-04-20T21:22:47.001003+0000","last_undegraded":"2024-04-20T21:22:47.001003+0000","last_fullsized":"2024-04-20T21:22:47.001003+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:52:24.101191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.358114+0000","last_change":"2024-04-20T21:22:46.000461+0000","last_active":"2024-04-20T21:22:47.358114+0000","last_peered":"2024-04-20T21:22:47.358114+0000","last_clean":"2024-04-20T21:22:47.358114+0000","last_became_active":"2024-04-20T21:22:46.000188+0000","last_became_peered":"2024-04-20T21:22:46.000188+0000","last_unstale":"2024-04-20T21:22:47.358114+0000","last_undegraded":"2024-04-20T21:22:47.358114+0000","last_fullsized":"2024-04-20T21:22:47.358114+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:27:38.679422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.996119+0000","last_change":"2024-04-20T21:22:45.996176+0000","last_active":"2024-04-20T21:22:46.996119+0000","last_peered":"2024-04-20T21:22:46.996119+0000","last_clean":"2024-04-20T21:22:46.996119+0000","last_became_active":"2024-04-20T21:22:45.995860+0000","last_became_peered":"2024-04-20T21:22:45.995860+0000","last_unstale":"2024-04-20T21:22:46.996119+0000","last_undegraded":"2024-04-20T21:22:46.996119+0000","last_fullsized":"2024-04-20T21:22:46.996119+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:19:56.685552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.357267+0000","last_change":"2024-04-20T21:22:46.003219+0000","last_active":"2024-04-20T21:22:47.357267+0000","last_peered":"2024-04-20T21:22:47.357267+0000","last_clean":"2024-04-20T21:22:47.357267+0000","last_became_active":"2024-04-20T21:22:46.002927+0000","last_became_peered":"2024-04-20T21:22:46.002927+0000","last_unstale":"2024-04-20T21:22:47.357267+0000","last_undegraded":"2024-04-20T21:22:47.357267+0000","last_fullsized":"2024-04-20T21:22:47.357267+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:18:23.304303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.000229+0000","last_change":"2024-04-20T21:22:45.998217+0000","last_active":"2024-04-20T21:22:47.000229+0000","last_peered":"2024-04-20T21:22:47.000229+0000","last_clean":"2024-04-20T21:22:47.000229+0000","last_became_active":"2024-04-20T21:22:45.997927+0000","last_became_peered":"2024-04-20T21:22:45.997927+0000","last_unstale":"2024-04-20T21:22:47.000229+0000","last_undegraded":"2024-04-20T21:22:47.000229+0000","last_fullsized":"2024-04-20T21:22:47.000229+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:34:53.276286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.014778+0000","last_change":"2024-04-20T21:22:45.994406+0000","last_active":"2024-04-20T21:22:47.014778+0000","last_peered":"2024-04-20T21:22:47.014778+0000","last_clean":"2024-04-20T21:22:47.014778+0000","last_became_active":"2024-04-20T21:22:45.994109+0000","last_became_peered":"2024-04-20T21:22:45.994109+0000","last_unstale":"2024-04-20T21:22:47.014778+0000","last_undegraded":"2024-04-20T21:22:47.014778+0000","last_fullsized":"2024-04-20T21:22:47.014778+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:06:06.356878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.999796+0000","last_change":"2024-04-20T21:22:45.990144+0000","last_active":"2024-04-20T21:22:46.999796+0000","last_peered":"2024-04-20T21:22:46.999796+0000","last_clean":"2024-04-20T21:22:46.999796+0000","last_became_active":"2024-04-20T21:22:45.987854+0000","last_became_peered":"2024-04-20T21:22:45.987854+0000","last_unstale":"2024-04-20T21:22:46.999796+0000","last_undegraded":"2024-04-20T21:22:46.999796+0000","last_fullsized":"2024-04-20T21:22:46.999796+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:14:23.787653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.358759+0000","last_change":"2024-04-20T21:22:46.009487+0000","last_active":"2024-04-20T21:22:47.358759+0000","last_peered":"2024-04-20T21:22:47.358759+0000","last_clean":"2024-04-20T21:22:47.358759+0000","last_became_active":"2024-04-20T21:22:46.008835+0000","last_became_peered":"2024-04-20T21:22:46.008835+0000","last_unstale":"2024-04-20T21:22:47.358759+0000","last_undegraded":"2024-04-20T21:22:47.358759+0000","last_fullsized":"2024-04-20T21:22:47.358759+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:19:35.848351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.001975+0000","last_change":"2024-04-20T21:22:46.008121+0000","last_active":"2024-04-20T21:22:47.001975+0000","last_peered":"2024-04-20T21:22:47.001975+0000","last_clean":"2024-04-20T21:22:47.001975+0000","last_became_active":"2024-04-20T21:22:46.007835+0000","last_became_peered":"2024-04-20T21:22:46.007835+0000","last_unstale":"2024-04-20T21:22:47.001975+0000","last_undegraded":"2024-04-20T21:22:47.001975+0000","last_fullsized":"2024-04-20T21:22:47.001975+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:42:09.491911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.295181+0000","last_change":"2024-04-20T21:22:46.008417+0000","last_active":"2024-04-20T21:22:47.295181+0000","last_peered":"2024-04-20T21:22:47.295181+0000","last_clean":"2024-04-20T21:22:47.295181+0000","last_became_active":"2024-04-20T21:22:46.008127+0000","last_became_peered":"2024-04-20T21:22:46.008127+0000","last_unstale":"2024-04-20T21:22:47.295181+0000","last_undegraded":"2024-04-20T21:22:47.295181+0000","last_fullsized":"2024-04-20T21:22:47.295181+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:11:12.822718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.996166+0000","last_change":"2024-04-20T21:22:46.005658+0000","last_active":"2024-04-20T21:22:46.996166+0000","last_peered":"2024-04-20T21:22:46.996166+0000","last_clean":"2024-04-20T21:22:46.996166+0000","last_became_active":"2024-04-20T21:22:46.005346+0000","last_became_peered":"2024-04-20T21:22:46.005346+0000","last_unstale":"2024-04-20T21:22:46.996166+0000","last_undegraded":"2024-04-20T21:22:46.996166+0000","last_fullsized":"2024-04-20T21:22:46.996166+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:35:49.635636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.020614+0000","last_change":"2024-04-20T21:22:46.007780+0000","last_active":"2024-04-20T21:22:47.020614+0000","last_peered":"2024-04-20T21:22:47.020614+0000","last_clean":"2024-04-20T21:22:47.020614+0000","last_became_active":"2024-04-20T21:22:46.007331+0000","last_became_peered":"2024-04-20T21:22:46.007331+0000","last_unstale":"2024-04-20T21:22:47.020614+0000","last_undegraded":"2024-04-20T21:22:47.020614+0000","last_fullsized":"2024-04-20T21:22:47.020614+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:55:14.180769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.357690+0000","last_change":"2024-04-20T21:22:46.001211+0000","last_active":"2024-04-20T21:22:47.357690+0000","last_peered":"2024-04-20T21:22:47.357690+0000","last_clean":"2024-04-20T21:22:47.357690+0000","last_became_active":"2024-04-20T21:22:46.000899+0000","last_became_peered":"2024-04-20T21:22:46.000899+0000","last_unstale":"2024-04-20T21:22:47.357690+0000","last_undegraded":"2024-04-20T21:22:47.357690+0000","last_fullsized":"2024-04-20T21:22:47.357690+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:08:57.585935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.020346+0000","last_change":"2024-04-20T21:22:46.015980+0000","last_active":"2024-04-20T21:22:47.020346+0000","last_peered":"2024-04-20T21:22:47.020346+0000","last_clean":"2024-04-20T21:22:47.020346+0000","last_became_active":"2024-04-20T21:22:46.015477+0000","last_became_peered":"2024-04-20T21:22:46.015477+0000","last_unstale":"2024-04-20T21:22:47.020346+0000","last_undegraded":"2024-04-20T21:22:47.020346+0000","last_fullsized":"2024-04-20T21:22:47.020346+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:24:16.195831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.999077+0000","last_change":"2024-04-20T21:22:45.998551+0000","last_active":"2024-04-20T21:22:46.999077+0000","last_peered":"2024-04-20T21:22:46.999077+0000","last_clean":"2024-04-20T21:22:46.999077+0000","last_became_active":"2024-04-20T21:22:45.996213+0000","last_became_peered":"2024-04-20T21:22:45.996213+0000","last_unstale":"2024-04-20T21:22:46.999077+0000","last_undegraded":"2024-04-20T21:22:46.999077+0000","last_fullsized":"2024-04-20T21:22:46.999077+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:49:36.482159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.358766+0000","last_change":"2024-04-20T21:22:46.013340+0000","last_active":"2024-04-20T21:22:47.358766+0000","last_peered":"2024-04-20T21:22:47.358766+0000","last_clean":"2024-04-20T21:22:47.358766+0000","last_became_active":"2024-04-20T21:22:46.009224+0000","last_became_peered":"2024-04-20T21:22:46.009224+0000","last_unstale":"2024-04-20T21:22:47.358766+0000","last_undegraded":"2024-04-20T21:22:47.358766+0000","last_fullsized":"2024-04-20T21:22:47.358766+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:33:18.596222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.996803+0000","last_change":"2024-04-20T21:22:45.987458+0000","last_active":"2024-04-20T21:22:46.996803+0000","last_peered":"2024-04-20T21:22:46.996803+0000","last_clean":"2024-04-20T21:22:46.996803+0000","last_became_active":"2024-04-20T21:22:45.987046+0000","last_became_peered":"2024-04-20T21:22:45.987046+0000","last_unstale":"2024-04-20T21:22:46.996803+0000","last_undegraded":"2024-04-20T21:22:46.996803+0000","last_fullsized":"2024-04-20T21:22:46.996803+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:09:41.573665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.001399+0000","last_change":"2024-04-20T21:22:45.995689+0000","last_active":"2024-04-20T21:22:47.001399+0000","last_peered":"2024-04-20T21:22:47.001399+0000","last_clean":"2024-04-20T21:22:47.001399+0000","last_became_active":"2024-04-20T21:22:45.995323+0000","last_became_peered":"2024-04-20T21:22:45.995323+0000","last_unstale":"2024-04-20T21:22:47.001399+0000","last_undegraded":"2024-04-20T21:22:47.001399+0000","last_fullsized":"2024-04-20T21:22:47.001399+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:56:14.784334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.020743+0000","last_change":"2024-04-20T21:22:46.005639+0000","last_active":"2024-04-20T21:22:47.020743+0000","last_peered":"2024-04-20T21:22:47.020743+0000","last_clean":"2024-04-20T21:22:47.020743+0000","last_became_active":"2024-04-20T21:22:46.005338+0000","last_became_peered":"2024-04-20T21:22:46.005338+0000","last_unstale":"2024-04-20T21:22:47.020743+0000","last_undegraded":"2024-04-20T21:22:47.020743+0000","last_fullsized":"2024-04-20T21:22:47.020743+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:27:57.913146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.000249+0000","last_change":"2024-04-20T21:22:45.998328+0000","last_active":"2024-04-20T21:22:47.000249+0000","last_peered":"2024-04-20T21:22:47.000249+0000","last_clean":"2024-04-20T21:22:47.000249+0000","last_became_active":"2024-04-20T21:22:45.998016+0000","last_became_peered":"2024-04-20T21:22:45.998016+0000","last_unstale":"2024-04-20T21:22:47.000249+0000","last_undegraded":"2024-04-20T21:22:47.000249+0000","last_fullsized":"2024-04-20T21:22:47.000249+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:53:42.792540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.999681+0000","last_change":"2024-04-20T21:22:46.006835+0000","last_active":"2024-04-20T21:22:46.999681+0000","last_peered":"2024-04-20T21:22:46.999681+0000","last_clean":"2024-04-20T21:22:46.999681+0000","last_became_active":"2024-04-20T21:22:46.006544+0000","last_became_peered":"2024-04-20T21:22:46.006544+0000","last_unstale":"2024-04-20T21:22:46.999681+0000","last_undegraded":"2024-04-20T21:22:46.999681+0000","last_fullsized":"2024-04-20T21:22:46.999681+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:05:10.087962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.294440+0000","last_change":"2024-04-20T21:22:46.002176+0000","last_active":"2024-04-20T21:22:47.294440+0000","last_peered":"2024-04-20T21:22:47.294440+0000","last_clean":"2024-04-20T21:22:47.294440+0000","last_became_active":"2024-04-20T21:22:46.001901+0000","last_became_peered":"2024-04-20T21:22:46.001901+0000","last_unstale":"2024-04-20T21:22:47.294440+0000","last_undegraded":"2024-04-20T21:22:47.294440+0000","last_fullsized":"2024-04-20T21:22:47.294440+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:55:38.941276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.294481+0000","last_change":"2024-04-20T21:22:46.012772+0000","last_active":"2024-04-20T21:22:47.294481+0000","last_peered":"2024-04-20T21:22:47.294481+0000","last_clean":"2024-04-20T21:22:47.294481+0000","last_became_active":"2024-04-20T21:22:46.011133+0000","last_became_peered":"2024-04-20T21:22:46.011133+0000","last_unstale":"2024-04-20T21:22:47.294481+0000","last_undegraded":"2024-04-20T21:22:47.294481+0000","last_fullsized":"2024-04-20T21:22:47.294481+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:24:16.195831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.001458+0000","last_change":"2024-04-20T21:22:46.009435+0000","last_active":"2024-04-20T21:22:47.001458+0000","last_peered":"2024-04-20T21:22:47.001458+0000","last_clean":"2024-04-20T21:22:47.001458+0000","last_became_active":"2024-04-20T21:22:46.008653+0000","last_became_peered":"2024-04-20T21:22:46.008653+0000","last_unstale":"2024-04-20T21:22:47.001458+0000","last_undegraded":"2024-04-20T21:22:47.001458+0000","last_fullsized":"2024-04-20T21:22:47.001458+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:58:27.202522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.015568+0000","last_change":"2024-04-20T21:22:45.999532+0000","last_active":"2024-04-20T21:22:47.015568+0000","last_peered":"2024-04-20T21:22:47.015568+0000","last_clean":"2024-04-20T21:22:47.015568+0000","last_became_active":"2024-04-20T21:22:45.999223+0000","last_became_peered":"2024-04-20T21:22:45.999223+0000","last_unstale":"2024-04-20T21:22:47.015568+0000","last_undegraded":"2024-04-20T21:22:47.015568+0000","last_fullsized":"2024-04-20T21:22:47.015568+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:55:14.180769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.358026+0000","last_change":"2024-04-20T21:22:46.002322+0000","last_active":"2024-04-20T21:22:47.358026+0000","last_peered":"2024-04-20T21:22:47.358026+0000","last_clean":"2024-04-20T21:22:47.358026+0000","last_became_active":"2024-04-20T21:22:46.002035+0000","last_became_peered":"2024-04-20T21:22:46.002035+0000","last_unstale":"2024-04-20T21:22:47.358026+0000","last_undegraded":"2024-04-20T21:22:47.358026+0000","last_fullsized":"2024-04-20T21:22:47.358026+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:47:00.839639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.001277+0000","last_change":"2024-04-20T21:22:46.006624+0000","last_active":"2024-04-20T21:22:47.001277+0000","last_peered":"2024-04-20T21:22:47.001277+0000","last_clean":"2024-04-20T21:22:47.001277+0000","last_became_active":"2024-04-20T21:22:46.006273+0000","last_became_peered":"2024-04-20T21:22:46.006273+0000","last_unstale":"2024-04-20T21:22:47.001277+0000","last_undegraded":"2024-04-20T21:22:47.001277+0000","last_fullsized":"2024-04-20T21:22:47.001277+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:06:06.356878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.357002+0000","last_change":"2024-04-20T21:22:45.998542+0000","last_active":"2024-04-20T21:22:47.357002+0000","last_peered":"2024-04-20T21:22:47.357002+0000","last_clean":"2024-04-20T21:22:47.357002+0000","last_became_active":"2024-04-20T21:22:45.998274+0000","last_became_peered":"2024-04-20T21:22:45.998274+0000","last_unstale":"2024-04-20T21:22:47.357002+0000","last_undegraded":"2024-04-20T21:22:47.357002+0000","last_fullsized":"2024-04-20T21:22:47.357002+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:17:21.685206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.014934+0000","last_change":"2024-04-20T21:22:45.995816+0000","last_active":"2024-04-20T21:22:47.014934+0000","last_peered":"2024-04-20T21:22:47.014934+0000","last_clean":"2024-04-20T21:22:47.014934+0000","last_became_active":"2024-04-20T21:22:45.994917+0000","last_became_peered":"2024-04-20T21:22:45.994917+0000","last_unstale":"2024-04-20T21:22:47.014934+0000","last_undegraded":"2024-04-20T21:22:47.014934+0000","last_fullsized":"2024-04-20T21:22:47.014934+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:53:17.849382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.015118+0000","last_change":"2024-04-20T21:22:46.001790+0000","last_active":"2024-04-20T21:22:47.015118+0000","last_peered":"2024-04-20T21:22:47.015118+0000","last_clean":"2024-04-20T21:22:47.015118+0000","last_became_active":"2024-04-20T21:22:46.001494+0000","last_became_peered":"2024-04-20T21:22:46.001494+0000","last_unstale":"2024-04-20T21:22:47.015118+0000","last_undegraded":"2024-04-20T21:22:47.015118+0000","last_fullsized":"2024-04-20T21:22:47.015118+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:42:09.491911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.999590+0000","last_change":"2024-04-20T21:22:46.002543+0000","last_active":"2024-04-20T21:22:46.999590+0000","last_peered":"2024-04-20T21:22:46.999590+0000","last_clean":"2024-04-20T21:22:46.999590+0000","last_became_active":"2024-04-20T21:22:46.001680+0000","last_became_peered":"2024-04-20T21:22:46.001680+0000","last_unstale":"2024-04-20T21:22:46.999590+0000","last_undegraded":"2024-04-20T21:22:46.999590+0000","last_fullsized":"2024-04-20T21:22:46.999590+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:22:06.648403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.295439+0000","last_change":"2024-04-20T21:22:46.011516+0000","last_active":"2024-04-20T21:22:47.295439+0000","last_peered":"2024-04-20T21:22:47.295439+0000","last_clean":"2024-04-20T21:22:47.295439+0000","last_became_active":"2024-04-20T21:22:46.010968+0000","last_became_peered":"2024-04-20T21:22:46.010968+0000","last_unstale":"2024-04-20T21:22:47.295439+0000","last_undegraded":"2024-04-20T21:22:47.295439+0000","last_fullsized":"2024-04-20T21:22:47.295439+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:01:42.221000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"70'4","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.295725+0000","last_change":"2024-04-20T21:22:45.000877+0000","last_active":"2024-04-20T21:22:47.295725+0000","last_peered":"2024-04-20T21:22:47.295725+0000","last_clean":"2024-04-20T21:22:47.295725+0000","last_became_active":"2024-04-20T21:22:45.000402+0000","last_became_peered":"2024-04-20T21:22:45.000402+0000","last_unstale":"2024-04-20T21:22:47.295725+0000","last_undegraded":"2024-04-20T21:22:47.295725+0000","last_fullsized":"2024-04-20T21:22:47.295725+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:47:51.476465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.358957+0000","last_change":"2024-04-20T21:22:46.007646+0000","last_active":"2024-04-20T21:22:47.358957+0000","last_peered":"2024-04-20T21:22:47.358957+0000","last_clean":"2024-04-20T21:22:47.358957+0000","last_became_active":"2024-04-20T21:22:46.007304+0000","last_became_peered":"2024-04-20T21:22:46.007304+0000","last_unstale":"2024-04-20T21:22:47.358957+0000","last_undegraded":"2024-04-20T21:22:47.358957+0000","last_fullsized":"2024-04-20T21:22:47.358957+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:45:53.505415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"70'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.020979+0000","last_change":"2024-04-20T21:22:44.998432+0000","last_active":"2024-04-20T21:22:47.020979+0000","last_peered":"2024-04-20T21:22:47.020979+0000","last_clean":"2024-04-20T21:22:47.020979+0000","last_became_active":"2024-04-20T21:22:44.998006+0000","last_became_peered":"2024-04-20T21:22:44.998006+0000","last_unstale":"2024-04-20T21:22:47.020979+0000","last_undegraded":"2024-04-20T21:22:47.020979+0000","last_fullsized":"2024-04-20T21:22:47.020979+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:30:25.442091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.000767+0000","last_change":"2024-04-20T21:22:46.013298+0000","last_active":"2024-04-20T21:22:47.000767+0000","last_peered":"2024-04-20T21:22:47.000767+0000","last_clean":"2024-04-20T21:22:47.000767+0000","last_became_active":"2024-04-20T21:22:46.011869+0000","last_became_peered":"2024-04-20T21:22:46.011869+0000","last_unstale":"2024-04-20T21:22:47.000767+0000","last_undegraded":"2024-04-20T21:22:47.000767+0000","last_fullsized":"2024-04-20T21:22:47.000767+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:10:02.145157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"70'2","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.001646+0000","last_change":"2024-04-20T21:22:44.997985+0000","last_active":"2024-04-20T21:22:47.001646+0000","last_peered":"2024-04-20T21:22:47.001646+0000","last_clean":"2024-04-20T21:22:47.001646+0000","last_became_active":"2024-04-20T21:22:44.997672+0000","last_became_peered":"2024-04-20T21:22:44.997672+0000","last_unstale":"2024-04-20T21:22:47.001646+0000","last_undegraded":"2024-04-20T21:22:47.001646+0000","last_fullsized":"2024-04-20T21:22:47.001646+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:10:01.129225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.295193+0000","last_change":"2024-04-20T21:22:46.012757+0000","last_active":"2024-04-20T21:22:47.295193+0000","last_peered":"2024-04-20T21:22:47.295193+0000","last_clean":"2024-04-20T21:22:47.295193+0000","last_became_active":"2024-04-20T21:22:46.011583+0000","last_became_peered":"2024-04-20T21:22:46.011583+0000","last_unstale":"2024-04-20T21:22:47.295193+0000","last_undegraded":"2024-04-20T21:22:47.295193+0000","last_fullsized":"2024-04-20T21:22:47.295193+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:06:57.239468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"70'2","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.000874+0000","last_change":"2024-04-20T21:22:44.989343+0000","last_active":"2024-04-20T21:22:47.000874+0000","last_peered":"2024-04-20T21:22:47.000874+0000","last_clean":"2024-04-20T21:22:47.000874+0000","last_became_active":"2024-04-20T21:22:44.989022+0000","last_became_peered":"2024-04-20T21:22:44.989022+0000","last_unstale":"2024-04-20T21:22:47.000874+0000","last_undegraded":"2024-04-20T21:22:47.000874+0000","last_fullsized":"2024-04-20T21:22:47.000874+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:50:24.320805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.014909+0000","last_change":"2024-04-20T21:22:46.004052+0000","last_active":"2024-04-20T21:22:47.014909+0000","last_peered":"2024-04-20T21:22:47.014909+0000","last_clean":"2024-04-20T21:22:47.014909+0000","last_became_active":"2024-04-20T21:22:46.002637+0000","last_became_peered":"2024-04-20T21:22:46.002637+0000","last_unstale":"2024-04-20T21:22:47.014909+0000","last_undegraded":"2024-04-20T21:22:47.014909+0000","last_fullsized":"2024-04-20T21:22:47.014909+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:58:27.202522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"70'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.357559+0000","last_change":"2024-04-20T21:22:44.994866+0000","last_active":"2024-04-20T21:22:47.357559+0000","last_peered":"2024-04-20T21:22:47.357559+0000","last_clean":"2024-04-20T21:22:47.357559+0000","last_became_active":"2024-04-20T21:22:44.994089+0000","last_became_peered":"2024-04-20T21:22:44.994089+0000","last_unstale":"2024-04-20T21:22:47.357559+0000","last_undegraded":"2024-04-20T21:22:47.357559+0000","last_fullsized":"2024-04-20T21:22:47.357559+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:42:08.475979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.000407+0000","last_change":"2024-04-20T21:22:46.008256+0000","last_active":"2024-04-20T21:22:47.000407+0000","last_peered":"2024-04-20T21:22:47.000407+0000","last_clean":"2024-04-20T21:22:47.000407+0000","last_became_active":"2024-04-20T21:22:46.007960+0000","last_became_peered":"2024-04-20T21:22:46.007960+0000","last_unstale":"2024-04-20T21:22:47.000407+0000","last_undegraded":"2024-04-20T21:22:47.000407+0000","last_fullsized":"2024-04-20T21:22:47.000407+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:28:56.572056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"70'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.000308+0000","last_change":"2024-04-20T21:22:44.995756+0000","last_active":"2024-04-20T21:22:47.000308+0000","last_peered":"2024-04-20T21:22:47.000308+0000","last_clean":"2024-04-20T21:22:47.000308+0000","last_became_active":"2024-04-20T21:22:44.995451+0000","last_became_peered":"2024-04-20T21:22:44.995451+0000","last_unstale":"2024-04-20T21:22:47.000308+0000","last_undegraded":"2024-04-20T21:22:47.000308+0000","last_fullsized":"2024-04-20T21:22:47.000308+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:07:07.942984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.356960+0000","last_change":"2024-04-20T21:22:46.004215+0000","last_active":"2024-04-20T21:22:47.356960+0000","last_peered":"2024-04-20T21:22:47.356960+0000","last_clean":"2024-04-20T21:22:47.356960+0000","last_became_active":"2024-04-20T21:22:46.003804+0000","last_became_peered":"2024-04-20T21:22:46.003804+0000","last_unstale":"2024-04-20T21:22:47.356960+0000","last_undegraded":"2024-04-20T21:22:47.356960+0000","last_fullsized":"2024-04-20T21:22:47.356960+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:45:49.895208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.995960+0000","last_change":"2024-04-20T21:22:44.996824+0000","last_active":"2024-04-20T21:22:46.995960+0000","last_peered":"2024-04-20T21:22:46.995960+0000","last_clean":"2024-04-20T21:22:46.995960+0000","last_became_active":"2024-04-20T21:22:44.996531+0000","last_became_peered":"2024-04-20T21:22:44.996531+0000","last_unstale":"2024-04-20T21:22:46.995960+0000","last_undegraded":"2024-04-20T21:22:46.995960+0000","last_fullsized":"2024-04-20T21:22:46.995960+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:01:19.361992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.020299+0000","last_change":"2024-04-20T21:22:46.012496+0000","last_active":"2024-04-20T21:22:47.020299+0000","last_peered":"2024-04-20T21:22:47.020299+0000","last_clean":"2024-04-20T21:22:47.020299+0000","last_became_active":"2024-04-20T21:22:46.011693+0000","last_became_peered":"2024-04-20T21:22:46.011693+0000","last_unstale":"2024-04-20T21:22:47.020299+0000","last_undegraded":"2024-04-20T21:22:47.020299+0000","last_fullsized":"2024-04-20T21:22:47.020299+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:55:38.941276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.356944+0000","last_change":"2024-04-20T21:22:44.982668+0000","last_active":"2024-04-20T21:22:47.356944+0000","last_peered":"2024-04-20T21:22:47.356944+0000","last_clean":"2024-04-20T21:22:47.356944+0000","last_became_active":"2024-04-20T21:22:44.982035+0000","last_became_peered":"2024-04-20T21:22:44.982035+0000","last_unstale":"2024-04-20T21:22:47.356944+0000","last_undegraded":"2024-04-20T21:22:47.356944+0000","last_fullsized":"2024-04-20T21:22:47.356944+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:04:12.006293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.015640+0000","last_change":"2024-04-20T21:22:46.005753+0000","last_active":"2024-04-20T21:22:47.015640+0000","last_peered":"2024-04-20T21:22:47.015640+0000","last_clean":"2024-04-20T21:22:47.015640+0000","last_became_active":"2024-04-20T21:22:46.005305+0000","last_became_peered":"2024-04-20T21:22:46.005305+0000","last_unstale":"2024-04-20T21:22:47.015640+0000","last_undegraded":"2024-04-20T21:22:47.015640+0000","last_fullsized":"2024-04-20T21:22:47.015640+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:27:57.913146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.001557+0000","last_change":"2024-04-20T21:22:44.994181+0000","last_active":"2024-04-20T21:22:47.001557+0000","last_peered":"2024-04-20T21:22:47.001557+0000","last_clean":"2024-04-20T21:22:47.001557+0000","last_became_active":"2024-04-20T21:22:44.992861+0000","last_became_peered":"2024-04-20T21:22:44.992861+0000","last_unstale":"2024-04-20T21:22:47.001557+0000","last_undegraded":"2024-04-20T21:22:47.001557+0000","last_fullsized":"2024-04-20T21:22:47.001557+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:52:23.085259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.295586+0000","last_change":"2024-04-20T21:22:46.006964+0000","last_active":"2024-04-20T21:22:47.295586+0000","last_peered":"2024-04-20T21:22:47.295586+0000","last_clean":"2024-04-20T21:22:47.295586+0000","last_became_active":"2024-04-20T21:22:46.006690+0000","last_became_peered":"2024-04-20T21:22:46.006690+0000","last_unstale":"2024-04-20T21:22:47.295586+0000","last_undegraded":"2024-04-20T21:22:47.295586+0000","last_fullsized":"2024-04-20T21:22:47.295586+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:53:21.061854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.001840+0000","last_change":"2024-04-20T21:22:44.992156+0000","last_active":"2024-04-20T21:22:47.001840+0000","last_peered":"2024-04-20T21:22:47.001840+0000","last_clean":"2024-04-20T21:22:47.001840+0000","last_became_active":"2024-04-20T21:22:44.991749+0000","last_became_peered":"2024-04-20T21:22:44.991749+0000","last_unstale":"2024-04-20T21:22:47.001840+0000","last_undegraded":"2024-04-20T21:22:47.001840+0000","last_fullsized":"2024-04-20T21:22:47.001840+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:02:04.020744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.001028+0000","last_change":"2024-04-20T21:22:45.999681+0000","last_active":"2024-04-20T21:22:47.001028+0000","last_peered":"2024-04-20T21:22:47.001028+0000","last_clean":"2024-04-20T21:22:47.001028+0000","last_became_active":"2024-04-20T21:22:45.998534+0000","last_became_peered":"2024-04-20T21:22:45.998534+0000","last_unstale":"2024-04-20T21:22:47.001028+0000","last_undegraded":"2024-04-20T21:22:47.001028+0000","last_fullsized":"2024-04-20T21:22:47.001028+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:21:38.035075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"70'3","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.358192+0000","last_change":"2024-04-20T21:22:44.990563+0000","last_active":"2024-04-20T21:22:47.358192+0000","last_peered":"2024-04-20T21:22:47.358192+0000","last_clean":"2024-04-20T21:22:47.358192+0000","last_became_active":"2024-04-20T21:22:44.990292+0000","last_became_peered":"2024-04-20T21:22:44.990292+0000","last_unstale":"2024-04-20T21:22:47.358192+0000","last_undegraded":"2024-04-20T21:22:47.358192+0000","last_fullsized":"2024-04-20T21:22:47.358192+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:55:37.925344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.357584+0000","last_change":"2024-04-20T21:22:45.997098+0000","last_active":"2024-04-20T21:22:47.357584+0000","last_peered":"2024-04-20T21:22:47.357584+0000","last_clean":"2024-04-20T21:22:47.357584+0000","last_became_active":"2024-04-20T21:22:45.996820+0000","last_became_peered":"2024-04-20T21:22:45.996820+0000","last_unstale":"2024-04-20T21:22:47.357584+0000","last_undegraded":"2024-04-20T21:22:47.357584+0000","last_fullsized":"2024-04-20T21:22:47.357584+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:59:11.385021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"70'6","reported_seq":20,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.996263+0000","last_change":"2024-04-20T21:22:44.988227+0000","last_active":"2024-04-20T21:22:46.996263+0000","last_peered":"2024-04-20T21:22:46.996263+0000","last_clean":"2024-04-20T21:22:46.996263+0000","last_became_active":"2024-04-20T21:22:44.987939+0000","last_became_peered":"2024-04-20T21:22:44.987939+0000","last_unstale":"2024-04-20T21:22:46.996263+0000","last_undegraded":"2024-04-20T21:22:46.996263+0000","last_fullsized":"2024-04-20T21:22:46.996263+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:46:00.265362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.294726+0000","last_change":"2024-04-20T21:22:46.003843+0000","last_active":"2024-04-20T21:22:47.294726+0000","last_peered":"2024-04-20T21:22:47.294726+0000","last_clean":"2024-04-20T21:22:47.294726+0000","last_became_active":"2024-04-20T21:22:46.003508+0000","last_became_peered":"2024-04-20T21:22:46.003508+0000","last_unstale":"2024-04-20T21:22:47.294726+0000","last_undegraded":"2024-04-20T21:22:47.294726+0000","last_fullsized":"2024-04-20T21:22:47.294726+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:55:14.180769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"70'2","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.996842+0000","last_change":"2024-04-20T21:22:44.992317+0000","last_active":"2024-04-20T21:22:46.996842+0000","last_peered":"2024-04-20T21:22:46.996842+0000","last_clean":"2024-04-20T21:22:46.996842+0000","last_became_active":"2024-04-20T21:22:44.992040+0000","last_became_peered":"2024-04-20T21:22:44.992040+0000","last_unstale":"2024-04-20T21:22:46.996842+0000","last_undegraded":"2024-04-20T21:22:46.996842+0000","last_fullsized":"2024-04-20T21:22:46.996842+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:50:24.320805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.015238+0000","last_change":"2024-04-20T21:22:45.998870+0000","last_active":"2024-04-20T21:22:47.015238+0000","last_peered":"2024-04-20T21:22:47.015238+0000","last_clean":"2024-04-20T21:22:47.015238+0000","last_became_active":"2024-04-20T21:22:45.998537+0000","last_became_peered":"2024-04-20T21:22:45.998537+0000","last_unstale":"2024-04-20T21:22:47.015238+0000","last_undegraded":"2024-04-20T21:22:47.015238+0000","last_fullsized":"2024-04-20T21:22:47.015238+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:55:38.941276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.000501+0000","last_change":"2024-04-20T21:22:44.987324+0000","last_active":"2024-04-20T21:22:47.000501+0000","last_peered":"2024-04-20T21:22:47.000501+0000","last_clean":"2024-04-20T21:22:47.000501+0000","last_became_active":"2024-04-20T21:22:44.986974+0000","last_became_peered":"2024-04-20T21:22:44.986974+0000","last_unstale":"2024-04-20T21:22:47.000501+0000","last_undegraded":"2024-04-20T21:22:47.000501+0000","last_fullsized":"2024-04-20T21:22:47.000501+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:35:20.490828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.020104+0000","last_change":"2024-04-20T21:22:46.012070+0000","last_active":"2024-04-20T21:22:47.020104+0000","last_peered":"2024-04-20T21:22:47.020104+0000","last_clean":"2024-04-20T21:22:47.020104+0000","last_became_active":"2024-04-20T21:22:46.010808+0000","last_became_peered":"2024-04-20T21:22:46.010808+0000","last_unstale":"2024-04-20T21:22:47.020104+0000","last_undegraded":"2024-04-20T21:22:47.020104+0000","last_fullsized":"2024-04-20T21:22:47.020104+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:42:09.491911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.014700+0000","last_change":"2024-04-20T21:22:44.997932+0000","last_active":"2024-04-20T21:22:47.014700+0000","last_peered":"2024-04-20T21:22:47.014700+0000","last_clean":"2024-04-20T21:22:47.014700+0000","last_became_active":"2024-04-20T21:22:44.997397+0000","last_became_peered":"2024-04-20T21:22:44.997397+0000","last_unstale":"2024-04-20T21:22:47.014700+0000","last_undegraded":"2024-04-20T21:22:47.014700+0000","last_fullsized":"2024-04-20T21:22:47.014700+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:56:13.768402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.357344+0000","last_change":"2024-04-20T21:22:45.997195+0000","last_active":"2024-04-20T21:22:47.357344+0000","last_peered":"2024-04-20T21:22:47.357344+0000","last_clean":"2024-04-20T21:22:47.357344+0000","last_became_active":"2024-04-20T21:22:45.996696+0000","last_became_peered":"2024-04-20T21:22:45.996696+0000","last_unstale":"2024-04-20T21:22:47.357344+0000","last_undegraded":"2024-04-20T21:22:47.357344+0000","last_fullsized":"2024-04-20T21:22:47.357344+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:40:15.771704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"70'8","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.998939+0000","last_change":"2024-04-20T21:22:44.995360+0000","last_active":"2024-04-20T21:22:46.998939+0000","last_peered":"2024-04-20T21:22:46.998939+0000","last_clean":"2024-04-20T21:22:46.998939+0000","last_became_active":"2024-04-20T21:22:44.995055+0000","last_became_peered":"2024-04-20T21:22:44.995055+0000","last_unstale":"2024-04-20T21:22:46.998939+0000","last_undegraded":"2024-04-20T21:22:46.998939+0000","last_fullsized":"2024-04-20T21:22:46.998939+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:14:22.771721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.001952+0000","last_change":"2024-04-20T21:22:45.993981+0000","last_active":"2024-04-20T21:22:47.001952+0000","last_peered":"2024-04-20T21:22:47.001952+0000","last_clean":"2024-04-20T21:22:47.001952+0000","last_became_active":"2024-04-20T21:22:45.993203+0000","last_became_peered":"2024-04-20T21:22:45.993203+0000","last_unstale":"2024-04-20T21:22:47.001952+0000","last_undegraded":"2024-04-20T21:22:47.001952+0000","last_fullsized":"2024-04-20T21:22:47.001952+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:53:17.849382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"70'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.001990+0000","last_change":"2024-04-20T21:22:44.993745+0000","last_active":"2024-04-20T21:22:47.001990+0000","last_peered":"2024-04-20T21:22:47.001990+0000","last_clean":"2024-04-20T21:22:47.001990+0000","last_became_active":"2024-04-20T21:22:44.992981+0000","last_became_peered":"2024-04-20T21:22:44.992981+0000","last_unstale":"2024-04-20T21:22:47.001990+0000","last_undegraded":"2024-04-20T21:22:47.001990+0000","last_fullsized":"2024-04-20T21:22:47.001990+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:23:40.199763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.295754+0000","last_change":"2024-04-20T21:22:46.006870+0000","last_active":"2024-04-20T21:22:47.295754+0000","last_peered":"2024-04-20T21:22:47.295754+0000","last_clean":"2024-04-20T21:22:47.295754+0000","last_became_active":"2024-04-20T21:22:46.006093+0000","last_became_peered":"2024-04-20T21:22:46.006093+0000","last_unstale":"2024-04-20T21:22:47.295754+0000","last_undegraded":"2024-04-20T21:22:47.295754+0000","last_fullsized":"2024-04-20T21:22:47.295754+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:17:21.685206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"70'2","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.002056+0000","last_change":"2024-04-20T21:22:44.998796+0000","last_active":"2024-04-20T21:22:47.002056+0000","last_peered":"2024-04-20T21:22:47.002056+0000","last_clean":"2024-04-20T21:22:47.002056+0000","last_became_active":"2024-04-20T21:22:44.998424+0000","last_became_peered":"2024-04-20T21:22:44.998424+0000","last_unstale":"2024-04-20T21:22:47.002056+0000","last_undegraded":"2024-04-20T21:22:47.002056+0000","last_fullsized":"2024-04-20T21:22:47.002056+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:45:10.580900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.996286+0000","last_change":"2024-04-20T21:22:45.999490+0000","last_active":"2024-04-20T21:22:46.996286+0000","last_peered":"2024-04-20T21:22:46.996286+0000","last_clean":"2024-04-20T21:22:46.996286+0000","last_became_active":"2024-04-20T21:22:45.999118+0000","last_became_peered":"2024-04-20T21:22:45.999118+0000","last_unstale":"2024-04-20T21:22:46.996286+0000","last_undegraded":"2024-04-20T21:22:46.996286+0000","last_fullsized":"2024-04-20T21:22:46.996286+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:55:03.918649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"70'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.001641+0000","last_change":"2024-04-20T21:22:45.000092+0000","last_active":"2024-04-20T21:22:47.001641+0000","last_peered":"2024-04-20T21:22:47.001641+0000","last_clean":"2024-04-20T21:22:47.001641+0000","last_became_active":"2024-04-20T21:22:44.998998+0000","last_became_peered":"2024-04-20T21:22:44.998998+0000","last_unstale":"2024-04-20T21:22:47.001641+0000","last_undegraded":"2024-04-20T21:22:47.001641+0000","last_fullsized":"2024-04-20T21:22:47.001641+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:46:00.265362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.358172+0000","last_change":"2024-04-20T21:22:46.001345+0000","last_active":"2024-04-20T21:22:47.358172+0000","last_peered":"2024-04-20T21:22:47.358172+0000","last_clean":"2024-04-20T21:22:47.358172+0000","last_became_active":"2024-04-20T21:22:46.001055+0000","last_became_peered":"2024-04-20T21:22:46.001055+0000","last_unstale":"2024-04-20T21:22:47.358172+0000","last_undegraded":"2024-04-20T21:22:47.358172+0000","last_fullsized":"2024-04-20T21:22:47.358172+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:48:59.404081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.294957+0000","last_change":"2024-04-20T21:22:45.002908+0000","last_active":"2024-04-20T21:22:47.294957+0000","last_peered":"2024-04-20T21:22:47.294957+0000","last_clean":"2024-04-20T21:22:47.294957+0000","last_became_active":"2024-04-20T21:22:45.002596+0000","last_became_peered":"2024-04-20T21:22:45.002596+0000","last_unstale":"2024-04-20T21:22:47.294957+0000","last_undegraded":"2024-04-20T21:22:47.294957+0000","last_fullsized":"2024-04-20T21:22:47.294957+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:53:04.669954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.357387+0000","last_change":"2024-04-20T21:22:46.004184+0000","last_active":"2024-04-20T21:22:47.357387+0000","last_peered":"2024-04-20T21:22:47.357387+0000","last_clean":"2024-04-20T21:22:47.357387+0000","last_became_active":"2024-04-20T21:22:46.003915+0000","last_became_peered":"2024-04-20T21:22:46.003915+0000","last_unstale":"2024-04-20T21:22:47.357387+0000","last_undegraded":"2024-04-20T21:22:47.357387+0000","last_fullsized":"2024-04-20T21:22:47.357387+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:40:38.260253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.000565+0000","last_change":"2024-04-20T21:22:44.996534+0000","last_active":"2024-04-20T21:22:47.000565+0000","last_peered":"2024-04-20T21:22:47.000565+0000","last_clean":"2024-04-20T21:22:47.000565+0000","last_became_active":"2024-04-20T21:22:44.996190+0000","last_became_peered":"2024-04-20T21:22:44.996190+0000","last_unstale":"2024-04-20T21:22:47.000565+0000","last_undegraded":"2024-04-20T21:22:47.000565+0000","last_fullsized":"2024-04-20T21:22:47.000565+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:01:19.361992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.356519+0000","last_change":"2024-04-20T21:22:46.005627+0000","last_active":"2024-04-20T21:22:47.356519+0000","last_peered":"2024-04-20T21:22:47.356519+0000","last_clean":"2024-04-20T21:22:47.356519+0000","last_became_active":"2024-04-20T21:22:46.005377+0000","last_became_peered":"2024-04-20T21:22:46.005377+0000","last_unstale":"2024-04-20T21:22:47.356519+0000","last_undegraded":"2024-04-20T21:22:47.356519+0000","last_fullsized":"2024-04-20T21:22:47.356519+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:06:18.941092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.015474+0000","last_change":"2024-04-20T21:22:44.993852+0000","last_active":"2024-04-20T21:22:47.015474+0000","last_peered":"2024-04-20T21:22:47.015474+0000","last_clean":"2024-04-20T21:22:47.015474+0000","last_became_active":"2024-04-20T21:22:44.993491+0000","last_became_peered":"2024-04-20T21:22:44.993491+0000","last_unstale":"2024-04-20T21:22:47.015474+0000","last_undegraded":"2024-04-20T21:22:47.015474+0000","last_fullsized":"2024-04-20T21:22:47.015474+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:07:15.232666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.356646+0000","last_change":"2024-04-20T21:22:46.001218+0000","last_active":"2024-04-20T21:22:47.356646+0000","last_peered":"2024-04-20T21:22:47.356646+0000","last_clean":"2024-04-20T21:22:47.356646+0000","last_became_active":"2024-04-20T21:22:46.000900+0000","last_became_peered":"2024-04-20T21:22:46.000900+0000","last_unstale":"2024-04-20T21:22:47.356646+0000","last_undegraded":"2024-04-20T21:22:47.356646+0000","last_fullsized":"2024-04-20T21:22:47.356646+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:53:21.061854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.019997+0000","last_change":"2024-04-20T21:22:44.992558+0000","last_active":"2024-04-20T21:22:47.019997+0000","last_peered":"2024-04-20T21:22:47.019997+0000","last_clean":"2024-04-20T21:22:47.019997+0000","last_became_active":"2024-04-20T21:22:44.991402+0000","last_became_peered":"2024-04-20T21:22:44.991402+0000","last_unstale":"2024-04-20T21:22:47.019997+0000","last_undegraded":"2024-04-20T21:22:47.019997+0000","last_fullsized":"2024-04-20T21:22:47.019997+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:11:00.035082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.020853+0000","last_change":"2024-04-20T21:22:46.012268+0000","last_active":"2024-04-20T21:22:47.020853+0000","last_peered":"2024-04-20T21:22:47.020853+0000","last_clean":"2024-04-20T21:22:47.020853+0000","last_became_active":"2024-04-20T21:22:46.011977+0000","last_became_peered":"2024-04-20T21:22:46.011977+0000","last_unstale":"2024-04-20T21:22:47.020853+0000","last_undegraded":"2024-04-20T21:22:47.020853+0000","last_fullsized":"2024-04-20T21:22:47.020853+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:11:12.822718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"70'2","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.001977+0000","last_change":"2024-04-20T21:22:44.990230+0000","last_active":"2024-04-20T21:22:47.001977+0000","last_peered":"2024-04-20T21:22:47.001977+0000","last_clean":"2024-04-20T21:22:47.001977+0000","last_became_active":"2024-04-20T21:22:44.989958+0000","last_became_peered":"2024-04-20T21:22:44.989958+0000","last_unstale":"2024-04-20T21:22:47.001977+0000","last_undegraded":"2024-04-20T21:22:47.001977+0000","last_fullsized":"2024-04-20T21:22:47.001977+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:27:28.340829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.015250+0000","last_change":"2024-04-20T21:22:45.993591+0000","last_active":"2024-04-20T21:22:47.015250+0000","last_peered":"2024-04-20T21:22:47.015250+0000","last_clean":"2024-04-20T21:22:47.015250+0000","last_became_active":"2024-04-20T21:22:45.993320+0000","last_became_peered":"2024-04-20T21:22:45.993320+0000","last_unstale":"2024-04-20T21:22:47.015250+0000","last_undegraded":"2024-04-20T21:22:47.015250+0000","last_fullsized":"2024-04-20T21:22:47.015250+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:24:16.195831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"71'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.015880+0000","last_change":"2024-04-20T21:22:44.997554+0000","last_active":"2024-04-20T21:22:47.015880+0000","last_peered":"2024-04-20T21:22:47.015880+0000","last_clean":"2024-04-20T21:22:47.015880+0000","last_became_active":"2024-04-20T21:22:44.997234+0000","last_became_peered":"2024-04-20T21:22:44.997234+0000","last_unstale":"2024-04-20T21:22:47.015880+0000","last_undegraded":"2024-04-20T21:22:47.015880+0000","last_fullsized":"2024-04-20T21:22:47.015880+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:47:51.476465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.294970+0000","last_change":"2024-04-20T21:22:46.011351+0000","last_active":"2024-04-20T21:22:47.294970+0000","last_peered":"2024-04-20T21:22:47.294970+0000","last_clean":"2024-04-20T21:22:47.294970+0000","last_became_active":"2024-04-20T21:22:46.011027+0000","last_became_peered":"2024-04-20T21:22:46.011027+0000","last_unstale":"2024-04-20T21:22:47.294970+0000","last_undegraded":"2024-04-20T21:22:47.294970+0000","last_fullsized":"2024-04-20T21:22:47.294970+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:27:57.913146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.358437+0000","last_change":"2024-04-20T21:22:44.998361+0000","last_active":"2024-04-20T21:22:47.358437+0000","last_peered":"2024-04-20T21:22:47.358437+0000","last_clean":"2024-04-20T21:22:47.358437+0000","last_became_active":"2024-04-20T21:22:44.996313+0000","last_became_peered":"2024-04-20T21:22:44.996313+0000","last_unstale":"2024-04-20T21:22:47.358437+0000","last_undegraded":"2024-04-20T21:22:47.358437+0000","last_fullsized":"2024-04-20T21:22:47.358437+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:57:50.796943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.295515+0000","last_change":"2024-04-20T21:22:46.006166+0000","last_active":"2024-04-20T21:22:47.295515+0000","last_peered":"2024-04-20T21:22:47.295515+0000","last_clean":"2024-04-20T21:22:47.295515+0000","last_became_active":"2024-04-20T21:22:46.005816+0000","last_became_peered":"2024-04-20T21:22:46.005816+0000","last_unstale":"2024-04-20T21:22:47.295515+0000","last_undegraded":"2024-04-20T21:22:47.295515+0000","last_fullsized":"2024-04-20T21:22:47.295515+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:05:13.355719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.020619+0000","last_change":"2024-04-20T21:22:45.003069+0000","last_active":"2024-04-20T21:22:47.020619+0000","last_peered":"2024-04-20T21:22:47.020619+0000","last_clean":"2024-04-20T21:22:47.020619+0000","last_became_active":"2024-04-20T21:22:45.002662+0000","last_became_peered":"2024-04-20T21:22:45.002662+0000","last_unstale":"2024-04-20T21:22:47.020619+0000","last_undegraded":"2024-04-20T21:22:47.020619+0000","last_fullsized":"2024-04-20T21:22:47.020619+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:37:28.682071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.357975+0000","last_change":"2024-04-20T21:22:45.998814+0000","last_active":"2024-04-20T21:22:47.357975+0000","last_peered":"2024-04-20T21:22:47.357975+0000","last_clean":"2024-04-20T21:22:47.357975+0000","last_became_active":"2024-04-20T21:22:45.998463+0000","last_became_peered":"2024-04-20T21:22:45.998463+0000","last_unstale":"2024-04-20T21:22:47.357975+0000","last_undegraded":"2024-04-20T21:22:47.357975+0000","last_fullsized":"2024-04-20T21:22:47.357975+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:37:37.094186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.357407+0000","last_change":"2024-04-20T21:22:44.990525+0000","last_active":"2024-04-20T21:22:47.357407+0000","last_peered":"2024-04-20T21:22:47.357407+0000","last_clean":"2024-04-20T21:22:47.357407+0000","last_became_active":"2024-04-20T21:22:44.990259+0000","last_became_peered":"2024-04-20T21:22:44.990259+0000","last_unstale":"2024-04-20T21:22:47.357407+0000","last_undegraded":"2024-04-20T21:22:47.357407+0000","last_fullsized":"2024-04-20T21:22:47.357407+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:07:45.118275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.000495+0000","last_change":"2024-04-20T21:22:46.003962+0000","last_active":"2024-04-20T21:22:47.000495+0000","last_peered":"2024-04-20T21:22:47.000495+0000","last_clean":"2024-04-20T21:22:47.000495+0000","last_became_active":"2024-04-20T21:22:46.001643+0000","last_became_peered":"2024-04-20T21:22:46.001643+0000","last_unstale":"2024-04-20T21:22:47.000495+0000","last_undegraded":"2024-04-20T21:22:47.000495+0000","last_fullsized":"2024-04-20T21:22:47.000495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:45:11.596833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.995914+0000","last_change":"2024-04-20T21:22:44.996969+0000","last_active":"2024-04-20T21:22:46.995914+0000","last_peered":"2024-04-20T21:22:46.995914+0000","last_clean":"2024-04-20T21:22:46.995914+0000","last_became_active":"2024-04-20T21:22:44.995514+0000","last_became_peered":"2024-04-20T21:22:44.995514+0000","last_unstale":"2024-04-20T21:22:46.995914+0000","last_undegraded":"2024-04-20T21:22:46.995914+0000","last_fullsized":"2024-04-20T21:22:46.995914+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:35:20.490828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.996108+0000","last_change":"2024-04-20T21:22:46.000298+0000","last_active":"2024-04-20T21:22:46.996108+0000","last_peered":"2024-04-20T21:22:46.996108+0000","last_clean":"2024-04-20T21:22:46.996108+0000","last_became_active":"2024-04-20T21:22:46.000027+0000","last_became_peered":"2024-04-20T21:22:46.000027+0000","last_unstale":"2024-04-20T21:22:46.996108+0000","last_undegraded":"2024-04-20T21:22:46.996108+0000","last_fullsized":"2024-04-20T21:22:46.996108+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:53:43.293248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.999123+0000","last_change":"2024-04-20T21:22:44.994701+0000","last_active":"2024-04-20T21:22:46.999123+0000","last_peered":"2024-04-20T21:22:46.999123+0000","last_clean":"2024-04-20T21:22:46.999123+0000","last_became_active":"2024-04-20T21:22:44.994359+0000","last_became_peered":"2024-04-20T21:22:44.994359+0000","last_unstale":"2024-04-20T21:22:46.999123+0000","last_undegraded":"2024-04-20T21:22:46.999123+0000","last_fullsized":"2024-04-20T21:22:46.999123+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:34:52.260354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.999207+0000","last_change":"2024-04-20T21:22:45.999332+0000","last_active":"2024-04-20T21:22:46.999207+0000","last_peered":"2024-04-20T21:22:46.999207+0000","last_clean":"2024-04-20T21:22:46.999207+0000","last_became_active":"2024-04-20T21:22:45.998564+0000","last_became_peered":"2024-04-20T21:22:45.998564+0000","last_unstale":"2024-04-20T21:22:46.999207+0000","last_undegraded":"2024-04-20T21:22:46.999207+0000","last_fullsized":"2024-04-20T21:22:46.999207+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:16:45.479800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:46.999825+0000","last_change":"2024-04-20T21:22:44.990936+0000","last_active":"2024-04-20T21:22:46.999825+0000","last_peered":"2024-04-20T21:22:46.999825+0000","last_clean":"2024-04-20T21:22:46.999825+0000","last_became_active":"2024-04-20T21:22:44.988494+0000","last_became_peered":"2024-04-20T21:22:44.988494+0000","last_unstale":"2024-04-20T21:22:46.999825+0000","last_undegraded":"2024-04-20T21:22:46.999825+0000","last_fullsized":"2024-04-20T21:22:46.999825+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:33:17.580290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.357272+0000","last_change":"2024-04-20T21:22:46.007192+0000","last_active":"2024-04-20T21:22:47.357272+0000","last_peered":"2024-04-20T21:22:47.357272+0000","last_clean":"2024-04-20T21:22:47.357272+0000","last_became_active":"2024-04-20T21:22:46.006583+0000","last_became_peered":"2024-04-20T21:22:46.006583+0000","last_unstale":"2024-04-20T21:22:47.357272+0000","last_undegraded":"2024-04-20T21:22:47.357272+0000","last_fullsized":"2024-04-20T21:22:47.357272+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:44.946779+0000","last_clean_scrub_stamp":"2024-04-20T21:22:44.946779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:28:45.591146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"70'2","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.000747+0000","last_change":"2024-04-20T21:22:44.988535+0000","last_active":"2024-04-20T21:22:47.000747+0000","last_peered":"2024-04-20T21:22:47.000747+0000","last_clean":"2024-04-20T21:22:47.000747+0000","last_became_active":"2024-04-20T21:22:44.988183+0000","last_became_peered":"2024-04-20T21:22:44.988183+0000","last_unstale":"2024-04-20T21:22:47.000747+0000","last_undegraded":"2024-04-20T21:22:47.000747+0000","last_fullsized":"2024-04-20T21:22:47.000747+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:22:43.930846+0000","last_clean_scrub_stamp":"2024-04-20T21:22:43.930846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:45:52.489483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":260,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-04-20T21:22:47.293973+0000","last_change":"2024-04-20T21:17:43.059637+0000","last_active":"2024-04-20T21:22:47.293973+0000","last_peered":"2024-04-20T21:22:47.293973+0000","last_clean":"2024-04-20T21:22:47.293973+0000","last_became_active":"2024-04-20T21:17:43.056207+0000","last_became_peered":"2024-04-20T21:17:43.056207+0000","last_unstale":"2024-04-20T21:22:47.293973+0000","last_undegraded":"2024-04-20T21:22:47.293973+0000","last_fullsized":"2024-04-20T21:22:47.293973+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:41.036290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:41.036290+0000","last_clean_scrub_stamp":"2024-04-20T21:17:41.036290+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:58:51.091337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574912,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32612,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339228,"statfs":{"total":96636764160,"available":96603369472,"internally_reserved":0,"allocated":1466368,"data_stored":955279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574913,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":872448,"data_stored":364387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574915,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":868352,"data_stored":364385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574914,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32036,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339804,"statfs":{"total":96636764160,"available":96603959296,"internally_reserved":0,"allocated":868352,"data_stored":364365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574914,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":872448,"data_stored":364407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574914,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32036,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339804,"statfs":{"total":96636764160,"available":96603959296,"internally_reserved":0,"allocated":876544,"data_stored":365907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32052,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339788,"statfs":{"total":96636764160,"available":96603942912,"internally_reserved":0,"allocated":884736,"data_stored":366523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574914,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32604,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1458176,"data_stored":954663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T21:22:55.853 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-20T21:22:55.853 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-20T21:22:56.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.106+0000 7f7c45c00640 1 Processor -- start 2024-04-20T21:22:56.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.106+0000 7f7c45c00640 1 -- start start 2024-04-20T21:22:56.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.108+0000 7f7c45c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7c400d1740 0x7f7c400cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:56.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.108+0000 7f7c45c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c400cf970 con 0x7f7c40161d40 2024-04-20T21:22:56.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.108+0000 7f7c45c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c400cfc50 con 0x7f7c4006dca0 2024-04-20T21:22:56.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.108+0000 7f7c45c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f7c400d0580 con 0x7f7c400d1740 2024-04-20T21:22:56.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.108+0000 7f7c37e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7c400d1740 0x7f7c400cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:56.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.108+0000 7f7c3fe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f7c40161d40 0x7f7c400d1030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54838/0 (socket says 172.21.15.86:54838) 2024-04-20T21:22:56.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.108+0000 7f7c3fe00640 1 -- 172.21.15.86:0/1014719899 learned_addr learned my addr 172.21.15.86:0/1014719899 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:22:56.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.109+0000 7f7c37e00640 1 -- 172.21.15.86:0/1014719899 >> v1:172.21.15.144:6789/0 conn(0x7f7c4006dca0 legacy=0x7f7c400d22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:56.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.109+0000 7f7c37e00640 1 -- 172.21.15.86:0/1014719899 >> v1:172.21.15.86:6789/0 conn(0x7f7c40161d40 legacy=0x7f7c400d1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:56.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.109+0000 7f7c37e00640 1 -- 172.21.15.86:0/1014719899 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c400cff30 con 0x7f7c400d1740 2024-04-20T21:22:56.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.109+0000 7f7c3f400640 1 -- 172.21.15.86:0/1014719899 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1437008107 0 0) 0x7f7c400cf970 con 0x7f7c40161d40 2024-04-20T21:22:56.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.110+0000 7f7c37e00640 1 --2- 172.21.15.86:0/1014719899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c400d1740 0x7f7c400cf1e0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f7c280097a0 tx=0x7f7c2802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5daf83bb65266d65 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:56.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.110+0000 7f7c3f400640 1 -- 172.21.15.86:0/1014719899 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c280315a0 con 0x7f7c400d1740 2024-04-20T21:22:56.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.110+0000 7f7c3f400640 1 -- 172.21.15.86:0/1014719899 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c28037070 con 0x7f7c400d1740 2024-04-20T21:22:56.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.110+0000 7f7c3f400640 1 -- 172.21.15.86:0/1014719899 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c28031960 con 0x7f7c400d1740 2024-04-20T21:22:56.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.111+0000 7f7c45c00640 1 -- 172.21.15.86:0/1014719899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c400d1740 msgr2=0x7f7c400cf1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:56.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.111+0000 7f7c45c00640 1 --2- 172.21.15.86:0/1014719899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c400d1740 0x7f7c400cf1e0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f7c280097a0 tx=0x7f7c2802b840 comp rx=0 tx=0).stop 2024-04-20T21:22:56.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.111+0000 7f7c45c00640 1 -- 172.21.15.86:0/1014719899 shutdown_connections 2024-04-20T21:22:56.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.111+0000 7f7c45c00640 1 --2- 172.21.15.86:0/1014719899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c400d1740 0x7f7c400cf1e0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:56.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.111+0000 7f7c45c00640 1 -- 172.21.15.86:0/1014719899 >> 172.21.15.86:0/1014719899 conn(0x7f7c4008bf70 msgr2=0x7f7c400d46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:56.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.111+0000 7f7c45c00640 1 -- 172.21.15.86:0/1014719899 shutdown_connections 2024-04-20T21:22:56.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.111+0000 7f7c45c00640 1 -- 172.21.15.86:0/1014719899 wait complete. 2024-04-20T21:22:56.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.113+0000 7f7c45c00640 1 Processor -- start 2024-04-20T21:22:56.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.113+0000 7f7c45c00640 1 -- start start 2024-04-20T21:22:56.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.114+0000 7f7c45c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c4006dca0 0x7f7c400cb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:56.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.114+0000 7f7c45c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c400d1740 0x7f7c400cd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:56.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.114+0000 7f7c45c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c40161d40 0x7f7c400cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:56.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.114+0000 7f7c45c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7c400b3ce0 con 0x7f7c40161d40 2024-04-20T21:22:56.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.114+0000 7f7c45c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7c400cf720 con 0x7f7c4006dca0 2024-04-20T21:22:56.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.114+0000 7f7c45c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7c40162120 con 0x7f7c400d1740 2024-04-20T21:22:56.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.114+0000 7f7c45200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c40161d40 0x7f7c400cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:56.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.114+0000 7f7c45200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c40161d40 0x7f7c400cd8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53060/0 (socket says 172.21.15.86:53060) 2024-04-20T21:22:56.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.114+0000 7f7c45200640 1 -- 172.21.15.86:0/3282699478 learned_addr learned my addr 172.21.15.86:0/3282699478 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:56.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.115+0000 7f7c3fe00640 1 --2- 172.21.15.86:0/3282699478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c4006dca0 0x7f7c400cb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:56.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.115+0000 7f7c45200640 1 -- 172.21.15.86:0/3282699478 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c400d1740 msgr2=0x7f7c400cd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:56.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.115+0000 7f7c45200640 1 --2- 172.21.15.86:0/3282699478 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c400d1740 0x7f7c400cd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:56.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.115+0000 7f7c45200640 1 -- 172.21.15.86:0/3282699478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c4006dca0 msgr2=0x7f7c400cb7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:56.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.115+0000 7f7c45200640 1 --2- 172.21.15.86:0/3282699478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c4006dca0 0x7f7c400cb7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:56.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.115+0000 7f7c45200640 1 -- 172.21.15.86:0/3282699478 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c400cb060 con 0x7f7c40161d40 2024-04-20T21:22:56.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.115+0000 7f7c45200640 1 --2- 172.21.15.86:0/3282699478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c40161d40 0x7f7c400cd8e0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f7c3800b9e0 tx=0x7f7c3800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:56.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.115+0000 7f7c3cc00640 1 -- 172.21.15.86:0/3282699478 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c38003b70 con 0x7f7c40161d40 2024-04-20T21:22:56.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.115+0000 7f7c3cc00640 1 -- 172.21.15.86:0/3282699478 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c38003d10 con 0x7f7c40161d40 2024-04-20T21:22:56.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.115+0000 7f7c3cc00640 1 -- 172.21.15.86:0/3282699478 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3800cb60 con 0x7f7c40161d40 2024-04-20T21:22:56.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.115+0000 7f7c45c00640 1 -- 172.21.15.86:0/3282699478 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c400b8e80 con 0x7f7c40161d40 2024-04-20T21:22:56.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.115+0000 7f7c45c00640 1 -- 172.21.15.86:0/3282699478 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c400b9440 con 0x7f7c40161d40 2024-04-20T21:22:56.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.117+0000 7f7c45c00640 1 -- 172.21.15.86:0/3282699478 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c4004fd90 con 0x7f7c40161d40 2024-04-20T21:22:56.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.118+0000 7f7c3cc00640 1 -- 172.21.15.86:0/3282699478 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7c3800cd00 con 0x7f7c40161d40 2024-04-20T21:22:56.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.119+0000 7f7c3cc00640 1 --2- 172.21.15.86:0/3282699478 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7c1c06c8f0 0x7f7c1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:56.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.119+0000 7f7c3cc00640 1 -- 172.21.15.86:0/3282699478 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f7c3808b9d0 con 0x7f7c40161d40 2024-04-20T21:22:56.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.119+0000 7f7c3fe00640 1 --2- 172.21.15.86:0/3282699478 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7c1c06c8f0 0x7f7c1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:56.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.120+0000 7f7c3fe00640 1 --2- 172.21.15.86:0/3282699478 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7c1c06c8f0 0x7f7c1c06edb0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7c30007fd0 tx=0x7f7c30009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:56.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.123+0000 7f7c3cc00640 1 -- 172.21.15.86:0/3282699478 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7c400cb060 con 0x7f7c40161d40 2024-04-20T21:22:56.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.398+0000 7f7c45c00640 1 -- 172.21.15.86:0/3282699478 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f7c400b6c40 con 0x7f7c40161d40 2024-04-20T21:22:56.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.402+0000 7f7c3cc00640 1 -- 172.21.15.86:0/3282699478 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) ==== 190+0+147 (secure 0 0 0) 0x7f7c38059e00 con 0x7f7c40161d40 2024-04-20T21:22:56.402 INFO:teuthology.orchestra.run.smithi086.stdout:[client.0] 2024-04-20T21:22:56.402 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQDBMCRmHWxyKRAANzhRi4eajp68h/jWkhFMnw== 2024-04-20T21:22:56.402 INFO:teuthology.orchestra.run.smithi086.stdout: caps mds = "allow" 2024-04-20T21:22:56.402 INFO:teuthology.orchestra.run.smithi086.stdout: caps mon = "allow r" 2024-04-20T21:22:56.402 INFO:teuthology.orchestra.run.smithi086.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-20T21:22:56.402 INFO:teuthology.orchestra.run.smithi086.stderr:updated caps for client.0 2024-04-20T21:22:56.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.405+0000 7f7c45c00640 1 -- 172.21.15.86:0/3282699478 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7c1c06c8f0 msgr2=0x7f7c1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:56.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.405+0000 7f7c45c00640 1 --2- 172.21.15.86:0/3282699478 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7c1c06c8f0 0x7f7c1c06edb0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7c30007fd0 tx=0x7f7c30009040 comp rx=0 tx=0).stop 2024-04-20T21:22:56.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.405+0000 7f7c45c00640 1 -- 172.21.15.86:0/3282699478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c40161d40 msgr2=0x7f7c400cd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:56.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.405+0000 7f7c45c00640 1 --2- 172.21.15.86:0/3282699478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c40161d40 0x7f7c400cd8e0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f7c3800b9e0 tx=0x7f7c3800bea0 comp rx=0 tx=0).stop 2024-04-20T21:22:56.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.405+0000 7f7c45c00640 1 -- 172.21.15.86:0/3282699478 shutdown_connections 2024-04-20T21:22:56.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.405+0000 7f7c45c00640 1 --2- 172.21.15.86:0/3282699478 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7c1c06c8f0 0x7f7c1c06edb0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:56.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.405+0000 7f7c45c00640 1 --2- 172.21.15.86:0/3282699478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c40161d40 0x7f7c400cd8e0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:56.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.405+0000 7f7c45c00640 1 --2- 172.21.15.86:0/3282699478 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7c400d1740 0x7f7c400cd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:56.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.405+0000 7f7c45c00640 1 --2- 172.21.15.86:0/3282699478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c4006dca0 0x7f7c400cb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:56.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.405+0000 7f7c45c00640 1 -- 172.21.15.86:0/3282699478 >> 172.21.15.86:0/3282699478 conn(0x7f7c4008bf70 msgr2=0x7f7c400b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:56.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.405+0000 7f7c45c00640 1 -- 172.21.15.86:0/3282699478 shutdown_connections 2024-04-20T21:22:56.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.405+0000 7f7c45c00640 1 -- 172.21.15.86:0/3282699478 wait complete. 2024-04-20T21:22:56.423 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:22:56.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.612+0000 7f06dc000640 1 Processor -- start 2024-04-20T21:22:56.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.612+0000 7f06dc000640 1 -- start start 2024-04-20T21:22:56.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.614+0000 7f06dc000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f06d415e730 0x7f06d40d1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:56.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.614+0000 7f06dc000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06d40d1990 0x7f06d409a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:56.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.614+0000 7f06dc000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f06d415eb10 0x7f06d409af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:56.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.614+0000 7f06dc000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f06d40713c0 con 0x7f06d415eb10 2024-04-20T21:22:56.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.614+0000 7f06dc000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f06d406eaa0 con 0x7f06d415e730 2024-04-20T21:22:56.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.614+0000 7f06dc000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f06d4070320 con 0x7f06d40d1990 2024-04-20T21:22:56.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.614+0000 7f06da000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f06d415eb10 0x7f06d409af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:56.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.614+0000 7f06da000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f06d415eb10 0x7f06d409af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53062/0 (socket says 172.21.15.86:53062) 2024-04-20T21:22:56.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.614+0000 7f06da000640 1 -- 172.21.15.86:0/4146805892 learned_addr learned my addr 172.21.15.86:0/4146805892 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:56.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.615+0000 7f06d9600640 1 --2- 172.21.15.86:0/4146805892 >> v2:172.21.15.144:3300/0 conn(0x7f06d415e730 0x7f06d40d1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:56.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.615+0000 7f06d8c00640 1 --2- 172.21.15.86:0/4146805892 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06d40d1990 0x7f06d409a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:56.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.615+0000 7f06da000640 1 -- 172.21.15.86:0/4146805892 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06d40d1990 msgr2=0x7f06d409a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:56.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.615+0000 7f06da000640 1 --2- 172.21.15.86:0/4146805892 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06d40d1990 0x7f06d409a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:56.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.615+0000 7f06da000640 1 -- 172.21.15.86:0/4146805892 >> v2:172.21.15.144:3300/0 conn(0x7f06d415e730 msgr2=0x7f06d40d1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:56.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.615+0000 7f06da000640 1 --2- 172.21.15.86:0/4146805892 >> v2:172.21.15.144:3300/0 conn(0x7f06d415e730 0x7f06d40d1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:56.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.615+0000 7f06da000640 1 -- 172.21.15.86:0/4146805892 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06d4070600 con 0x7f06d415eb10 2024-04-20T21:22:56.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.615+0000 7f06d9600640 1 --2- 172.21.15.86:0/4146805892 >> v2:172.21.15.144:3300/0 conn(0x7f06d415e730 0x7f06d40d1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:22:56.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.616+0000 7f06da000640 1 --2- 172.21.15.86:0/4146805892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06d415eb10 0x7f06d409af20 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f06d40734b0 tx=0x7f06cc02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35ecc63f3fe48778 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:56.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.616+0000 7f06d3e00640 1 -- 172.21.15.86:0/4146805892 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06cc02cb00 con 0x7f06d415eb10 2024-04-20T21:22:56.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.616+0000 7f06d3e00640 1 -- 172.21.15.86:0/4146805892 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06cc02cca0 con 0x7f06d415eb10 2024-04-20T21:22:56.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.616+0000 7f06d3e00640 1 -- 172.21.15.86:0/4146805892 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06cc034610 con 0x7f06d415eb10 2024-04-20T21:22:56.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.617+0000 7f06dc000640 1 -- 172.21.15.86:0/4146805892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06d415eb10 msgr2=0x7f06d409af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:56.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.617+0000 7f06dc000640 1 --2- 172.21.15.86:0/4146805892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06d415eb10 0x7f06d409af20 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f06d40734b0 tx=0x7f06cc02bf60 comp rx=0 tx=0).stop 2024-04-20T21:22:56.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.617+0000 7f06dc000640 1 -- 172.21.15.86:0/4146805892 shutdown_connections 2024-04-20T21:22:56.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.617+0000 7f06dc000640 1 --2- 172.21.15.86:0/4146805892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06d415eb10 0x7f06d409af20 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:56.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.617+0000 7f06dc000640 1 --2- 172.21.15.86:0/4146805892 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06d40d1990 0x7f06d409a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:56.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.617+0000 7f06dc000640 1 --2- 172.21.15.86:0/4146805892 >> v2:172.21.15.144:3300/0 conn(0x7f06d415e730 0x7f06d40d1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:56.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.617+0000 7f06dc000640 1 -- 172.21.15.86:0/4146805892 >> 172.21.15.86:0/4146805892 conn(0x7f06d40b4080 msgr2=0x7f06d40b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:56.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.617+0000 7f06dc000640 1 -- 172.21.15.86:0/4146805892 shutdown_connections 2024-04-20T21:22:56.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.617+0000 7f06dc000640 1 -- 172.21.15.86:0/4146805892 wait complete. 2024-04-20T21:22:56.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.619+0000 7f06dc000640 1 Processor -- start 2024-04-20T21:22:56.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.620+0000 7f06dc000640 1 -- start start 2024-04-20T21:22:56.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.621+0000 7f06dc000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06d40d1990 0x7f06d40b0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:56.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.621+0000 7f06dc000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06d415e730 0x7f06d40b5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:56.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.621+0000 7f06dc000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06d415eb10 0x7f06d40b5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:56.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.621+0000 7f06dc000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f06d4070ea0 con 0x7f06d415eb10 2024-04-20T21:22:56.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.621+0000 7f06dc000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f06d4070bc0 con 0x7f06d415e730 2024-04-20T21:22:56.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.621+0000 7f06dc000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f06d406fef0 con 0x7f06d40d1990 2024-04-20T21:22:56.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.621+0000 7f06da000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06d415eb10 0x7f06d40b5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:56.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.621+0000 7f06da000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06d415eb10 0x7f06d40b5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53066/0 (socket says 172.21.15.86:53066) 2024-04-20T21:22:56.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.621+0000 7f06da000640 1 -- 172.21.15.86:0/2276503573 learned_addr learned my addr 172.21.15.86:0/2276503573 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:56.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.621+0000 7f06d9600640 1 --2- 172.21.15.86:0/2276503573 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06d40d1990 0x7f06d40b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:56.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.621+0000 7f06d8c00640 1 --2- 172.21.15.86:0/2276503573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06d415e730 0x7f06d40b5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:56.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.622+0000 7f06da000640 1 -- 172.21.15.86:0/2276503573 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06d40d1990 msgr2=0x7f06d40b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:56.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.622+0000 7f06da000640 1 --2- 172.21.15.86:0/2276503573 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06d40d1990 0x7f06d40b0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:56.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.622+0000 7f06da000640 1 -- 172.21.15.86:0/2276503573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06d415e730 msgr2=0x7f06d40b5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:56.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.622+0000 7f06da000640 1 --2- 172.21.15.86:0/2276503573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06d415e730 0x7f06d40b5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:56.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.622+0000 7f06da000640 1 -- 172.21.15.86:0/2276503573 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06d40b6ca0 con 0x7f06d415eb10 2024-04-20T21:22:56.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.622+0000 7f06da000640 1 --2- 172.21.15.86:0/2276503573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06d415eb10 0x7f06d40b5cd0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f06cc036040 tx=0x7f06cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:56.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.622+0000 7f06d1600640 1 -- 172.21.15.86:0/2276503573 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06cc043020 con 0x7f06d415eb10 2024-04-20T21:22:56.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.622+0000 7f06d1600640 1 -- 172.21.15.86:0/2276503573 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06cc02ce50 con 0x7f06d415eb10 2024-04-20T21:22:56.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.622+0000 7f06dc000640 1 -- 172.21.15.86:0/2276503573 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06d40b6f80 con 0x7f06d415eb10 2024-04-20T21:22:56.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.622+0000 7f06d1600640 1 -- 172.21.15.86:0/2276503573 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06cc009510 con 0x7f06d415eb10 2024-04-20T21:22:56.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.623+0000 7f06dc000640 1 -- 172.21.15.86:0/2276503573 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06d40b7540 con 0x7f06d415eb10 2024-04-20T21:22:56.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.624+0000 7f06dc000640 1 -- 172.21.15.86:0/2276503573 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06d40689e0 con 0x7f06d415eb10 2024-04-20T21:22:56.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.625+0000 7f06d1600640 1 -- 172.21.15.86:0/2276503573 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f06cc046020 con 0x7f06d415eb10 2024-04-20T21:22:56.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.625+0000 7f06d1600640 1 --2- 172.21.15.86:0/2276503573 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f06b406c8f0 0x7f06b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:56.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.626+0000 7f06d1600640 1 -- 172.21.15.86:0/2276503573 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f06cc0ae1d0 con 0x7f06d415eb10 2024-04-20T21:22:56.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.626+0000 7f06d9600640 1 --2- 172.21.15.86:0/2276503573 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f06b406c8f0 0x7f06b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:56.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.627+0000 7f06d9600640 1 --2- 172.21.15.86:0/2276503573 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f06b406c8f0 0x7f06b406edb0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f06d415d5b0 tx=0x7f06c4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:56.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.630+0000 7f06d1600640 1 -- 172.21.15.86:0/2276503573 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f06cc04d030 con 0x7f06d415eb10 2024-04-20T21:22:56.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.871+0000 7f06dc000640 1 -- 172.21.15.86:0/2276503573 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f06d40ceb70 con 0x7f06d415eb10 2024-04-20T21:22:56.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.872+0000 7f06d1600640 1 -- 172.21.15.86:0/2276503573 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+5201 (secure 0 0 0) 0x7f06cc07c690 con 0x7f06d415eb10 2024-04-20T21:22:56.871 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:22:56.871 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":42,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2740872549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2740872549},{"type":"v1","addr":"172.21.15.86:6836","nonce":2740872549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5048,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/301899138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":301899138},{"type":"v1","addr":"172.21.15.144:6833","nonce":301899138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5051,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2816343379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2816343379},{"type":"v1","addr":"172.21.15.144:6835","nonce":2816343379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:22:45.431100+0000","modified":"2024-04-20T21:22:48.492788+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5407":{"gid":5407,"name":"a","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/202143875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":202143875},{"type":"v1","addr":"172.21.15.86:6837","nonce":202143875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5407,"qdb_cluster":[5407]},"id":5}]} 2024-04-20T21:22:56.872 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 42 2024-04-20T21:22:56.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.874+0000 7f06dc000640 1 -- 172.21.15.86:0/2276503573 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f06b406c8f0 msgr2=0x7f06b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:56.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.875+0000 7f06dc000640 1 --2- 172.21.15.86:0/2276503573 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f06b406c8f0 0x7f06b406edb0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f06d415d5b0 tx=0x7f06c4008040 comp rx=0 tx=0).stop 2024-04-20T21:22:56.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.875+0000 7f06dc000640 1 -- 172.21.15.86:0/2276503573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06d415eb10 msgr2=0x7f06d40b5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:56.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.875+0000 7f06dc000640 1 --2- 172.21.15.86:0/2276503573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06d415eb10 0x7f06d40b5cd0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f06cc036040 tx=0x7f06cc004a40 comp rx=0 tx=0).stop 2024-04-20T21:22:56.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.875+0000 7f06dc000640 1 -- 172.21.15.86:0/2276503573 shutdown_connections 2024-04-20T21:22:56.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.875+0000 7f06dc000640 1 --2- 172.21.15.86:0/2276503573 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f06b406c8f0 0x7f06b406edb0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:56.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.875+0000 7f06dc000640 1 --2- 172.21.15.86:0/2276503573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06d415eb10 0x7f06d40b5cd0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:56.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.875+0000 7f06dc000640 1 --2- 172.21.15.86:0/2276503573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06d415e730 0x7f06d40b5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:56.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.875+0000 7f06dc000640 1 --2- 172.21.15.86:0/2276503573 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06d40d1990 0x7f06d40b0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:56.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.875+0000 7f06dc000640 1 -- 172.21.15.86:0/2276503573 >> 172.21.15.86:0/2276503573 conn(0x7f06d40b4080 msgr2=0x7f06d406ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:56.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.875+0000 7f06dc000640 1 -- 172.21.15.86:0/2276503573 shutdown_connections 2024-04-20T21:22:56.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:56.875+0000 7f06dc000640 1 -- 172.21.15.86:0/2276503573 wait complete. 2024-04-20T21:22:57.294 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 42, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T21:22:45.431100+0000', 'modified': '2024-04-20T21:22:48.492788+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5407}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5407': {'gid': 5407, 'name': 'a', 'rank': 0, 'incarnation': 38, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/202143875', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6835', 'nonce': 202143875}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 202143875}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5407, 'qdb_cluster': [5407]} 2024-04-20T21:22:57.295 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-20T21:22:57.295 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-20T21:22:57.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.510+0000 7f7042e00640 1 Processor -- start 2024-04-20T21:22:57.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.511+0000 7f7042e00640 1 -- start start 2024-04-20T21:22:57.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.512+0000 7f7042e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f703c0ce790 0x7f703c0d13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:57.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.512+0000 7f7042e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f703c06dbb0 0x7f703c15e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:57.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.512+0000 7f7042e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f703c09ae10 con 0x7f703c0d1910 2024-04-20T21:22:57.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.512+0000 7f7042e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f703c09b0d0 con 0x7f703c0ce790 2024-04-20T21:22:57.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.512+0000 7f7042e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f703c09b3b0 con 0x7f703c06dbb0 2024-04-20T21:22:57.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.512+0000 7f703be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f703c0ce790 0x7f703c0d13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:57.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.512+0000 7f703be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f703c0ce790 0x7f703c0d13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53068/0 (socket says 172.21.15.86:53068) 2024-04-20T21:22:57.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.512+0000 7f703be00640 1 -- 172.21.15.86:0/3271487959 learned_addr learned my addr 172.21.15.86:0/3271487959 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:57.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.513+0000 7f703be00640 1 -- 172.21.15.86:0/3271487959 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f703c06dbb0 msgr2=0x7f703c15e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:57.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.513+0000 7f703be00640 1 --2- 172.21.15.86:0/3271487959 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f703c06dbb0 0x7f703c15e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:57.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.513+0000 7f703be00640 1 -- 172.21.15.86:0/3271487959 >> v1:172.21.15.86:6789/0 conn(0x7f703c0d1910 legacy=0x7f703c06d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:57.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.513+0000 7f703be00640 1 -- 172.21.15.86:0/3271487959 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f703c09b690 con 0x7f703c0ce790 2024-04-20T21:22:57.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.514+0000 7f703be00640 1 --2- 172.21.15.86:0/3271487959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f703c0ce790 0x7f703c0d13d0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f702c009870 tx=0x7f702c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66fb0f47bd65b30b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:57.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.514+0000 7f703aa00640 1 -- 172.21.15.86:0/3271487959 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702c034070 con 0x7f703c0ce790 2024-04-20T21:22:57.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.514+0000 7f703aa00640 1 -- 172.21.15.86:0/3271487959 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f702c030840 con 0x7f703c0ce790 2024-04-20T21:22:57.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.514+0000 7f703aa00640 1 -- 172.21.15.86:0/3271487959 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702c030c10 con 0x7f703c0ce790 2024-04-20T21:22:57.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.515+0000 7f7042e00640 1 -- 172.21.15.86:0/3271487959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f703c0ce790 msgr2=0x7f703c0d13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:57.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.515+0000 7f7042e00640 1 --2- 172.21.15.86:0/3271487959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f703c0ce790 0x7f703c0d13d0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f702c009870 tx=0x7f702c02b980 comp rx=0 tx=0).stop 2024-04-20T21:22:57.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.515+0000 7f7042e00640 1 -- 172.21.15.86:0/3271487959 shutdown_connections 2024-04-20T21:22:57.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.515+0000 7f7042e00640 1 --2- 172.21.15.86:0/3271487959 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f703c06dbb0 0x7f703c15e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:57.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.515+0000 7f7042e00640 1 --2- 172.21.15.86:0/3271487959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f703c0ce790 0x7f703c0d13d0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:57.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.515+0000 7f7042e00640 1 -- 172.21.15.86:0/3271487959 >> 172.21.15.86:0/3271487959 conn(0x7f703c0b3fe0 msgr2=0x7f703c0b43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:57.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.515+0000 7f7042e00640 1 -- 172.21.15.86:0/3271487959 shutdown_connections 2024-04-20T21:22:57.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.515+0000 7f7042e00640 1 -- 172.21.15.86:0/3271487959 wait complete. 2024-04-20T21:22:57.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.517+0000 7f7042e00640 1 Processor -- start 2024-04-20T21:22:57.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.517+0000 7f7042e00640 1 -- start start 2024-04-20T21:22:57.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.518+0000 7f7042e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f703c06dbb0 0x7f703c0b7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:57.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.518+0000 7f7042e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f703c0ce790 0x7f703c14faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:57.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.518+0000 7f7042e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f703c0b6b30 0x7f703c0cd450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:57.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.518+0000 7f7042e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f703c0cecf0 con 0x7f703c0ce790 2024-04-20T21:22:57.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.518+0000 7f7042e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f703c0ceb70 con 0x7f703c06dbb0 2024-04-20T21:22:57.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.518+0000 7f7042e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f703c09a9e0 con 0x7f703c0b6b30 2024-04-20T21:22:57.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.518+0000 7f703b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f703c0ce790 0x7f703c14faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:57.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.518+0000 7f703b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f703c0ce790 0x7f703c14faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53078/0 (socket says 172.21.15.86:53078) 2024-04-20T21:22:57.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.518+0000 7f703b400640 1 -- 172.21.15.86:0/12258074 learned_addr learned my addr 172.21.15.86:0/12258074 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:57.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.518+0000 7f7040e00640 1 --2- 172.21.15.86:0/12258074 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f703c0b6b30 0x7f703c0cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:57.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.518+0000 7f703be00640 1 --2- 172.21.15.86:0/12258074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f703c06dbb0 0x7f703c0b7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:57.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.518+0000 7f703b400640 1 -- 172.21.15.86:0/12258074 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f703c0b6b30 msgr2=0x7f703c0cd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:57.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.519+0000 7f703b400640 1 --2- 172.21.15.86:0/12258074 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f703c0b6b30 0x7f703c0cd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:57.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.519+0000 7f703b400640 1 -- 172.21.15.86:0/12258074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f703c06dbb0 msgr2=0x7f703c0b7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:57.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.519+0000 7f703b400640 1 --2- 172.21.15.86:0/12258074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f703c06dbb0 0x7f703c0b7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:57.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.519+0000 7f703b400640 1 -- 172.21.15.86:0/12258074 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f703c09d850 con 0x7f703c0ce790 2024-04-20T21:22:57.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.519+0000 7f7040e00640 1 --2- 172.21.15.86:0/12258074 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f703c0b6b30 0x7f703c0cd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:22:57.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.519+0000 7f703be00640 1 --2- 172.21.15.86:0/12258074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f703c06dbb0 0x7f703c0b7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:22:57.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.519+0000 7f703b400640 1 --2- 172.21.15.86:0/12258074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f703c0ce790 0x7f703c14faf0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f702400c400 tx=0x7f702400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:57.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.519+0000 7f7033e00640 1 -- 172.21.15.86:0/12258074 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7024017070 con 0x7f703c0ce790 2024-04-20T21:22:57.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.519+0000 7f7033e00640 1 -- 172.21.15.86:0/12258074 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70240052d0 con 0x7f703c0ce790 2024-04-20T21:22:57.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.519+0000 7f7033e00640 1 -- 172.21.15.86:0/12258074 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7024003830 con 0x7f703c0ce790 2024-04-20T21:22:57.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.519+0000 7f7042e00640 1 -- 172.21.15.86:0/12258074 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f703c0b6fe0 con 0x7f703c0ce790 2024-04-20T21:22:57.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.519+0000 7f7042e00640 1 -- 172.21.15.86:0/12258074 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f703c0b7580 con 0x7f703c0ce790 2024-04-20T21:22:57.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.521+0000 7f7033e00640 1 -- 172.21.15.86:0/12258074 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f702401a070 con 0x7f703c0ce790 2024-04-20T21:22:57.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.522+0000 7f7033e00640 1 --2- 172.21.15.86:0/12258074 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f701806c8f0 0x7f701806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:57.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.523+0000 7f7033e00640 1 -- 172.21.15.86:0/12258074 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f702408ae80 con 0x7f703c0ce790 2024-04-20T21:22:57.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.523+0000 7f703be00640 1 --2- 172.21.15.86:0/12258074 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f701806c8f0 0x7f701806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:57.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.523+0000 7f7045b9b740 1 Processor -- start 2024-04-20T21:22:57.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.523+0000 7f7045b9b740 1 -- start start 2024-04-20T21:22:57.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.523+0000 7f703be00640 1 --2- 172.21.15.86:0/12258074 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f701806c8f0 0x7f701806edb0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f702c036000 tx=0x7f702c03b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:57.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.524+0000 7f7045b9b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8185c8460 0x55e81857fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:57.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.524+0000 7f7045b9b740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e81857e4e0 0x55e81857ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:57.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.524+0000 7f7045b9b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e81857a930 0x55e8185aa770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:57.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.524+0000 7f7045b9b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e8184e69d0 con 0x55e81857a930 2024-04-20T21:22:57.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.524+0000 7f7045b9b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e81848d270 con 0x55e8185c8460 2024-04-20T21:22:57.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.524+0000 7f7045b9b740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55e8184e2990 con 0x55e81857e4e0 2024-04-20T21:22:57.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.524+0000 7f7040e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e81857a930 0x55e8185aa770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:57.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.524+0000 7f7040e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e81857a930 0x55e8185aa770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53082/0 (socket says 172.21.15.86:53082) 2024-04-20T21:22:57.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.524+0000 7f7040e00640 1 -- 172.21.15.86:0/1143784652 learned_addr learned my addr 172.21.15.86:0/1143784652 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:57.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.525+0000 7f703be00640 1 --2- 172.21.15.86:0/1143784652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8185c8460 0x55e81857fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:57.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.525+0000 7f703b400640 1 --2- 172.21.15.86:0/1143784652 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e81857e4e0 0x55e81857ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:57.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.525+0000 7f7040e00640 1 -- 172.21.15.86:0/1143784652 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e81857e4e0 msgr2=0x55e81857ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:57.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.525+0000 7f7040e00640 1 --2- 172.21.15.86:0/1143784652 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e81857e4e0 0x55e81857ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:57.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.525+0000 7f7040e00640 1 -- 172.21.15.86:0/1143784652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8185c8460 msgr2=0x55e81857fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:57.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.525+0000 7f7040e00640 1 --2- 172.21.15.86:0/1143784652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8185c8460 0x55e81857fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:57.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.525+0000 7f7040e00640 1 -- 172.21.15.86:0/1143784652 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8181f92d0 con 0x55e81857a930 2024-04-20T21:22:57.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.525+0000 7f7040e00640 1 --2- 172.21.15.86:0/1143784652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e81857a930 0x55e8185aa770 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f7034007f00 tx=0x7f703400c690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=639394114da6d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:57.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.525+0000 7f7030c00640 1 -- 172.21.15.86:0/1143784652 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7034014070 con 0x55e81857a930 2024-04-20T21:22:57.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.526+0000 7f7030c00640 1 -- 172.21.15.86:0/1143784652 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f703400f850 con 0x55e81857a930 2024-04-20T21:22:57.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.526+0000 7f7030c00640 1 -- 172.21.15.86:0/1143784652 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703400fc20 con 0x55e81857a930 2024-04-20T21:22:57.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.526+0000 7f7045b9b740 1 -- 172.21.15.86:0/1143784652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e81857a930 msgr2=0x55e8185aa770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:57.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.526+0000 7f7045b9b740 1 --2- 172.21.15.86:0/1143784652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e81857a930 0x55e8185aa770 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f7034007f00 tx=0x7f703400c690 comp rx=0 tx=0).stop 2024-04-20T21:22:57.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.526+0000 7f7045b9b740 1 -- 172.21.15.86:0/1143784652 shutdown_connections 2024-04-20T21:22:57.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.526+0000 7f7045b9b740 1 --2- 172.21.15.86:0/1143784652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e81857a930 0x55e8185aa770 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:57.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.526+0000 7f7045b9b740 1 --2- 172.21.15.86:0/1143784652 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e81857e4e0 0x55e81857ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:57.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.526+0000 7f7045b9b740 1 --2- 172.21.15.86:0/1143784652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8185c8460 0x55e81857fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:57.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.526+0000 7f7045b9b740 1 -- 172.21.15.86:0/1143784652 >> 172.21.15.86:0/1143784652 conn(0x55e81857cbf0 msgr2=0x55e8185c5770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:57.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.527+0000 7f7045b9b740 1 -- 172.21.15.86:0/1143784652 shutdown_connections 2024-04-20T21:22:57.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.527+0000 7f7045b9b740 1 -- 172.21.15.86:0/1143784652 wait complete. 2024-04-20T21:22:57.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.527+0000 7f7045b9b740 1 -- start start 2024-04-20T21:22:57.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.528+0000 7f7045b9b740 1 Processor -- start 2024-04-20T21:22:57.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.529+0000 7f7045b9b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e81857e4e0 0x55e8185a1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:57.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.529+0000 7f7045b9b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8185c8460 0x55e8185cad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:57.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.529+0000 7f7045b9b740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8185a5a70 0x55e8185a5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:57.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.529+0000 7f7045b9b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e81857e9f0 con 0x55e8185c8460 2024-04-20T21:22:57.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.529+0000 7f7045b9b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e8184a05e0 con 0x55e81857e4e0 2024-04-20T21:22:57.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.529+0000 7f7045b9b740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55e81857ecd0 con 0x55e8185a5a70 2024-04-20T21:22:57.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.529+0000 7f7045b9b740 10 client.0 _ll_register_callbacks cb 0x55e8181f8ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:22:57.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.529+0000 7f703b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8185c8460 0x55e8185cad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:57.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.529+0000 7f703b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8185c8460 0x55e8185cad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53086/0 (socket says 172.21.15.86:53086) 2024-04-20T21:22:57.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.529+0000 7f703b400640 1 -- 172.21.15.86:0/1960238073 learned_addr learned my addr 172.21.15.86:0/1960238073 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:57.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.529+0000 7f703b400640 1 -- 172.21.15.86:0/1960238073 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8185a5a70 msgr2=0x55e8185a5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:57.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.529+0000 7f703b400640 1 --2- 172.21.15.86:0/1960238073 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8185a5a70 0x55e8185a5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:57.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.529+0000 7f703b400640 1 -- 172.21.15.86:0/1960238073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e81857e4e0 msgr2=0x55e8185a1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:57.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.529+0000 7f703b400640 1 --2- 172.21.15.86:0/1960238073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e81857e4e0 0x55e8185a1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:57.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.529+0000 7f703b400640 1 -- 172.21.15.86:0/1960238073 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8182c36b0 con 0x55e8185c8460 2024-04-20T21:22:57.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.530+0000 7f703b400640 1 --2- 172.21.15.86:0/1960238073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8185c8460 0x55e8185cad60 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f70240588a0 tx=0x7f7024058ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f990e91f4b34e44 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:57.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.530+0000 7f7029600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:22:57.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.530+0000 7f7029600640 1 -- 172.21.15.86:0/1960238073 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7024071180 con 0x55e8185c8460 2024-04-20T21:22:57.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.530+0000 7f7029600640 1 -- 172.21.15.86:0/1960238073 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f7024057ff0 con 0x55e8185c8460 2024-04-20T21:22:57.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.530+0000 7f7045b9b740 1 -- 172.21.15.86:0/1960238073 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f703400f850 con 0x55e8185c8460 2024-04-20T21:22:57.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.530+0000 7f7029600640 1 -- 172.21.15.86:0/1960238073 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f70240581f0 con 0x55e8185c8460 2024-04-20T21:22:57.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.530+0000 7f7045b9b740 10 client.6879 fetch_fsmap learned FSMap version 42 2024-04-20T21:22:57.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.530+0000 7f7045b9b740 1 -- 172.21.15.86:0/1960238073 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) -- 0x55e8185a9970 con 0x55e8185c8460 2024-04-20T21:22:57.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.530+0000 7f7029600640 1 -- 172.21.15.86:0/1960238073 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f703400f850 con 0x55e8185c8460 2024-04-20T21:22:57.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.530+0000 7f7029600640 1 -- 172.21.15.86:0/1960238073 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 42) ==== 3495+0+0 (secure 0 0 0) 0x7f702405f150 con 0x55e8185c8460 2024-04-20T21:22:57.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.530+0000 7f7045b9b740 10 client.6879 fetch_fsmap finished waiting for FSMap version 42 2024-04-20T21:22:57.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.530+0000 7f7045b9b740 10 client.6879 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-20T21:22:57.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.530+0000 7f7045b9b740 20 client.6879 populate_metadata read hostname 'smithi086' 2024-04-20T21:22:57.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.531+0000 7f7045b9b740 1 --2- 172.21.15.86:0/1960238073 >> [v2:172.21.15.86:6835/202143875,v1:172.21.15.86:6837/202143875] conn(0x55e8185ccb00 0x55e8185aab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:57.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.531+0000 7f7045b9b740 4 client.6879 mds_command: new command op to 5407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:22:57.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.531+0000 7f7045b9b740 1 -- 172.21.15.86:0/1960238073 --> [v2:172.21.15.86:6835/202143875,v1:172.21.15.86:6837/202143875] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8184f43f0 con 0x55e8185ccb00 2024-04-20T21:22:57.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.531+0000 7f703be00640 1 --2- 172.21.15.86:0/1960238073 >> [v2:172.21.15.86:6835/202143875,v1:172.21.15.86:6837/202143875] conn(0x55e8185ccb00 0x55e8185aab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:57.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.531+0000 7f703be00640 1 --2- 172.21.15.86:0/1960238073 >> [v2:172.21.15.86:6835/202143875,v1:172.21.15.86:6837/202143875] conn(0x55e8185ccb00 0x55e8185aab40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34cbbf3085bbcef9 server_cookie=acf18a4c17999c8e in_seq=0 out_seq=0 2024-04-20T21:22:57.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.531+0000 7f7029600640 10 client.6879 ms_handle_connect on v2:172.21.15.86:6835/202143875 2024-04-20T21:22:57.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.532+0000 7f7029600640 1 -- 172.21.15.86:0/1960238073 <== mds.0 v2:172.21.15.86:6835/202143875 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e8184f43f0 con 0x55e8185ccb00 2024-04-20T21:22:57.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.532+0000 7f7029600640 10 client.6879 handle_command_reply: tid=0 2024-04-20T21:22:57.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.532+0000 7f7045b9b740 1 client.6879 shutdown 2024-04-20T21:22:57.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.532+0000 7f7045b9b740 1 -- 172.21.15.86:0/1960238073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8185c8460 msgr2=0x55e8185cad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:57.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.532+0000 7f7045b9b740 1 --2- 172.21.15.86:0/1960238073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8185c8460 0x55e8185cad60 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f70240588a0 tx=0x7f7024058ce0 comp rx=0 tx=0).stop 2024-04-20T21:22:57.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.532+0000 7f7045b9b740 1 -- 172.21.15.86:0/1960238073 shutdown_connections 2024-04-20T21:22:57.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.532+0000 7f7045b9b740 1 --2- 172.21.15.86:0/1960238073 >> [v2:172.21.15.86:6835/202143875,v1:172.21.15.86:6837/202143875] conn(0x55e8185ccb00 0x55e8185aab40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:57.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.532+0000 7f7045b9b740 1 --2- 172.21.15.86:0/1960238073 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8185a5a70 0x55e8185a5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:57.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.532+0000 7f7045b9b740 1 --2- 172.21.15.86:0/1960238073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8185c8460 0x55e8185cad60 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:57.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.532+0000 7f7045b9b740 1 --2- 172.21.15.86:0/1960238073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e81857e4e0 0x55e8185a1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:57.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.532+0000 7f7029600640 0 client.6879 ms_handle_reset on v2:172.21.15.86:6835/202143875 2024-04-20T21:22:57.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.532+0000 7f7045b9b740 1 -- 172.21.15.86:0/1960238073 >> 172.21.15.86:0/1960238073 conn(0x55e81857a930 msgr2=0x55e8185805a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:57.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.533+0000 7f7045b9b740 1 -- 172.21.15.86:0/1960238073 shutdown_connections 2024-04-20T21:22:57.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.533+0000 7f7045b9b740 1 -- 172.21.15.86:0/1960238073 wait complete. 2024-04-20T21:22:57.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.533+0000 7f7045b9b740 20 client.6879 trim_cache size 0 max 16384 2024-04-20T21:22:57.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.553+0000 7f7045b9b740 1 Processor -- start 2024-04-20T21:22:57.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.553+0000 7f7045b9b740 1 -- start start 2024-04-20T21:22:57.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.554+0000 7f7045b9b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e81857a930 0x55e8185d89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:57.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.554+0000 7f7045b9b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e81857e4e0 0x55e8185ca730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:57.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.554+0000 7f7045b9b740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8185a5a70 0x55e8185cac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:57.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.554+0000 7f7045b9b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f703c1fffb0 con 0x55e81857a930 2024-04-20T21:22:57.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.554+0000 7f7045b9b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f703c0b5f70 con 0x55e81857e4e0 2024-04-20T21:22:57.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.554+0000 7f7045b9b740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f703c09f0a0 con 0x55e8185a5a70 2024-04-20T21:22:57.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.554+0000 7f703be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e81857a930 0x55e8185d89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:57.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.554+0000 7f703be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e81857a930 0x55e8185d89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53102/0 (socket says 172.21.15.86:53102) 2024-04-20T21:22:57.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.554+0000 7f703be00640 1 -- 172.21.15.86:0/2468879387 learned_addr learned my addr 172.21.15.86:0/2468879387 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:57.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.554+0000 7f703be00640 1 -- 172.21.15.86:0/2468879387 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8185a5a70 msgr2=0x55e8185cac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:57.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.554+0000 7f703be00640 1 --2- 172.21.15.86:0/2468879387 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8185a5a70 0x55e8185cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:57.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.554+0000 7f703be00640 1 -- 172.21.15.86:0/2468879387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e81857e4e0 msgr2=0x55e8185ca730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:57.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.554+0000 7f703b400640 1 --2- 172.21.15.86:0/2468879387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e81857e4e0 0x55e8185ca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:57.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.555+0000 7f703be00640 1 --2- 172.21.15.86:0/2468879387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e81857e4e0 0x55e8185ca730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:57.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.555+0000 7f703be00640 1 -- 172.21.15.86:0/2468879387 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e81835c820 con 0x55e81857a930 2024-04-20T21:22:57.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.555+0000 7f703b400640 1 --2- 172.21.15.86:0/2468879387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e81857e4e0 0x55e8185ca730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:22:57.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.555+0000 7f703be00640 1 --2- 172.21.15.86:0/2468879387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e81857a930 0x55e8185d89b0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f702c009960 tx=0x7f702c009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70b5eb462bfc9017 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:57.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.555+0000 7f702be00640 1 -- 172.21.15.86:0/2468879387 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702c034040 con 0x55e81857a930 2024-04-20T21:22:57.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.555+0000 7f702be00640 1 -- 172.21.15.86:0/2468879387 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f702c056070 con 0x55e81857a930 2024-04-20T21:22:57.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.555+0000 7f702be00640 1 -- 172.21.15.86:0/2468879387 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702c050680 con 0x55e81857a930 2024-04-20T21:22:57.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.556+0000 7f7045b9b740 1 -- 172.21.15.86:0/2468879387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e81857a930 msgr2=0x55e8185d89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:57.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.556+0000 7f7045b9b740 1 --2- 172.21.15.86:0/2468879387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e81857a930 0x55e8185d89b0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f702c009960 tx=0x7f702c009240 comp rx=0 tx=0).stop 2024-04-20T21:22:57.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.556+0000 7f7045b9b740 1 -- 172.21.15.86:0/2468879387 shutdown_connections 2024-04-20T21:22:57.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.556+0000 7f7045b9b740 1 --2- 172.21.15.86:0/2468879387 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8185a5a70 0x55e8185cac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:57.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.556+0000 7f7045b9b740 1 --2- 172.21.15.86:0/2468879387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e81857e4e0 0x55e8185ca730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:57.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.556+0000 7f7045b9b740 1 --2- 172.21.15.86:0/2468879387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e81857a930 0x55e8185d89b0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:57.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.556+0000 7f7045b9b740 1 -- 172.21.15.86:0/2468879387 >> 172.21.15.86:0/2468879387 conn(0x55e8185c4f80 msgr2=0x55e8185805a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:57.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.556+0000 7f7045b9b740 1 -- 172.21.15.86:0/2468879387 shutdown_connections 2024-04-20T21:22:57.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.556+0000 7f7045b9b740 1 -- 172.21.15.86:0/2468879387 wait complete. 2024-04-20T21:22:57.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.557+0000 7f7045b9b740 1 -- start start 2024-04-20T21:22:57.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.557+0000 7f7045b9b740 1 Processor -- start 2024-04-20T21:22:57.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.558+0000 7f7045b9b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e81857a930 0x55e8185adce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:57.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.558+0000 7f7045b9b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e81857e4e0 0x55e8186408d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:57.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.558+0000 7f7045b9b740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8185a5a70 0x55e8185a0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:57.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.558+0000 7f7045b9b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e8185cc980 con 0x55e81857e4e0 2024-04-20T21:22:57.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.558+0000 7f7045b9b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e8185ab1f0 con 0x55e81857a930 2024-04-20T21:22:57.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.558+0000 7f7045b9b740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55e8185ccfe0 con 0x55e8185a5a70 2024-04-20T21:22:57.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.558+0000 7f7045b9b740 10 client.0 _ll_register_callbacks cb 0x7f7024058760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:22:57.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.558+0000 7f703b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e81857e4e0 0x55e8186408d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:57.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.558+0000 7f703b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e81857e4e0 0x55e8186408d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53106/0 (socket says 172.21.15.86:53106) 2024-04-20T21:22:57.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.558+0000 7f703b400640 1 -- 172.21.15.86:0/4117881714 learned_addr learned my addr 172.21.15.86:0/4117881714 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:57.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.559+0000 7f7040e00640 1 --2- 172.21.15.86:0/4117881714 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8185a5a70 0x55e8185a0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:57.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.559+0000 7f703be00640 1 --2- 172.21.15.86:0/4117881714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e81857a930 0x55e8185adce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:57.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.559+0000 7f703b400640 1 -- 172.21.15.86:0/4117881714 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8185a5a70 msgr2=0x55e8185a0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:57.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.559+0000 7f703b400640 1 --2- 172.21.15.86:0/4117881714 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8185a5a70 0x55e8185a0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:57.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.559+0000 7f703b400640 1 -- 172.21.15.86:0/4117881714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e81857a930 msgr2=0x55e8185adce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:57.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.559+0000 7f703b400640 1 --2- 172.21.15.86:0/4117881714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e81857a930 0x55e8185adce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:57.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.559+0000 7f703b400640 1 -- 172.21.15.86:0/4117881714 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8185ad520 con 0x55e81857e4e0 2024-04-20T21:22:57.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.559+0000 7f703be00640 1 --2- 172.21.15.86:0/4117881714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e81857a930 0x55e8185adce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:22:57.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.559+0000 7f703b400640 1 --2- 172.21.15.86:0/4117881714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e81857e4e0 0x55e8186408d0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f702400cdc0 tx=0x7f7024089270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10df7bf03d0637fb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:57.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.559+0000 7f7029600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:22:57.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.559+0000 7f7029600640 1 -- 172.21.15.86:0/4117881714 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7024071180 con 0x55e81857e4e0 2024-04-20T21:22:57.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.559+0000 7f7029600640 1 -- 172.21.15.86:0/4117881714 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f7024057ff0 con 0x55e81857e4e0 2024-04-20T21:22:57.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.559+0000 7f7045b9b740 1 -- 172.21.15.86:0/4117881714 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f702c056070 con 0x55e81857e4e0 2024-04-20T21:22:57.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.559+0000 7f7029600640 1 -- 172.21.15.86:0/4117881714 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f702407e050 con 0x55e81857e4e0 2024-04-20T21:22:57.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.559+0000 7f7045b9b740 10 client.6885 fetch_fsmap learned FSMap version 42 2024-04-20T21:22:57.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.559+0000 7f7045b9b740 1 -- 172.21.15.86:0/4117881714 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) -- 0x55e8186412c0 con 0x55e81857e4e0 2024-04-20T21:22:57.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.560+0000 7f7029600640 1 -- 172.21.15.86:0/4117881714 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f702c056070 con 0x55e81857e4e0 2024-04-20T21:22:57.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.560+0000 7f7029600640 1 -- 172.21.15.86:0/4117881714 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 42) ==== 3495+0+0 (secure 0 0 0) 0x7f7024081020 con 0x55e81857e4e0 2024-04-20T21:22:57.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.560+0000 7f7045b9b740 10 client.6885 fetch_fsmap finished waiting for FSMap version 42 2024-04-20T21:22:57.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.560+0000 7f7045b9b740 10 client.6885 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-20T21:22:57.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.560+0000 7f7045b9b740 20 client.6885 populate_metadata read hostname 'smithi086' 2024-04-20T21:22:57.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.560+0000 7f7045b9b740 1 --2- 172.21.15.86:0/4117881714 >> [v2:172.21.15.86:6835/202143875,v1:172.21.15.86:6837/202143875] conn(0x55e8185c8460 0x55e8185a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:57.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.560+0000 7f7045b9b740 4 client.6885 mds_command: new command op to 5407 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T21:22:57.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.560+0000 7f7045b9b740 1 -- 172.21.15.86:0/4117881714 --> [v2:172.21.15.86:6835/202143875,v1:172.21.15.86:6837/202143875] -- command(tid 0: {"prefix": "status"}) -- 0x55e8184ee260 con 0x55e8185c8460 2024-04-20T21:22:57.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.560+0000 7f703be00640 1 --2- 172.21.15.86:0/4117881714 >> [v2:172.21.15.86:6835/202143875,v1:172.21.15.86:6837/202143875] conn(0x55e8185c8460 0x55e8185a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:57.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.561+0000 7f703be00640 1 --2- 172.21.15.86:0/4117881714 >> [v2:172.21.15.86:6835/202143875,v1:172.21.15.86:6837/202143875] conn(0x55e8185c8460 0x55e8185a06c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a46d332168d681f server_cookie=d1faea7c9888e55d in_seq=0 out_seq=0 2024-04-20T21:22:57.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.561+0000 7f7029600640 10 client.6885 ms_handle_connect on v2:172.21.15.86:6835/202143875 2024-04-20T21:22:57.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.561+0000 7f7029600640 1 -- 172.21.15.86:0/4117881714 <== mds.0 v2:172.21.15.86:6835/202143875 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55e8184ee260 con 0x55e8185c8460 2024-04-20T21:22:57.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.561+0000 7f7029600640 10 client.6885 handle_command_reply: tid=0 2024-04-20T21:22:57.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.561+0000 7f7045b9b740 1 client.6885 shutdown 2024-04-20T21:22:57.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.561+0000 7f7045b9b740 1 -- 172.21.15.86:0/4117881714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e81857e4e0 msgr2=0x55e8186408d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:57.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.561+0000 7f7045b9b740 1 --2- 172.21.15.86:0/4117881714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e81857e4e0 0x55e8186408d0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f702400cdc0 tx=0x7f7024089270 comp rx=0 tx=0).stop 2024-04-20T21:22:57.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.561+0000 7f7045b9b740 1 -- 172.21.15.86:0/4117881714 shutdown_connections 2024-04-20T21:22:57.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.561+0000 7f7045b9b740 1 --2- 172.21.15.86:0/4117881714 >> [v2:172.21.15.86:6835/202143875,v1:172.21.15.86:6837/202143875] conn(0x55e8185c8460 0x55e8185a06c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:57.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.561+0000 7f7045b9b740 1 --2- 172.21.15.86:0/4117881714 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8185a5a70 0x55e8185a0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:57.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.561+0000 7f7045b9b740 1 --2- 172.21.15.86:0/4117881714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e81857e4e0 0x55e8186408d0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:57.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.561+0000 7f7045b9b740 1 --2- 172.21.15.86:0/4117881714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e81857a930 0x55e8185adce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:57.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.561+0000 7f7045b9b740 1 -- 172.21.15.86:0/4117881714 >> 172.21.15.86:0/4117881714 conn(0x55e8185c4f80 msgr2=0x55e8185805a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:57.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.561+0000 7f7029600640 0 client.6885 ms_handle_reset on v2:172.21.15.86:6835/202143875 2024-04-20T21:22:57.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.561+0000 7f7045b9b740 1 -- 172.21.15.86:0/4117881714 shutdown_connections 2024-04-20T21:22:57.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.561+0000 7f7045b9b740 1 -- 172.21.15.86:0/4117881714 wait complete. 2024-04-20T21:22:57.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.561+0000 7f7045b9b740 20 client.6885 trim_cache size 0 max 16384 2024-04-20T21:22:57.567 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:22:57.567 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "9c2dec1d-d7ae-4a83-a812-79922808e278", 2024-04-20T21:22:57.568 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-04-20T21:22:57.568 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 5407, 2024-04-20T21:22:57.568 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-20T21:22:57.568 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-20T21:22:57.568 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-20T21:22:57.568 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 11.570325326000001, 2024-04-20T21:22:57.568 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 42, 2024-04-20T21:22:57.568 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 71, 2024-04-20T21:22:57.568 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 71, 2024-04-20T21:22:57.568 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 18.053145310000001 2024-04-20T21:22:57.568 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:22:57.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.562+0000 7f7042e00640 1 -- 172.21.15.86:0/12258074 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f701806c8f0 msgr2=0x7f701806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:57.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.562+0000 7f7042e00640 1 --2- 172.21.15.86:0/12258074 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f701806c8f0 0x7f701806edb0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f702c036000 tx=0x7f702c03b040 comp rx=0 tx=0).stop 2024-04-20T21:22:57.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.562+0000 7f7042e00640 1 -- 172.21.15.86:0/12258074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f703c0ce790 msgr2=0x7f703c14faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:57.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.562+0000 7f7042e00640 1 --2- 172.21.15.86:0/12258074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f703c0ce790 0x7f703c14faf0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f702400c400 tx=0x7f702400c8c0 comp rx=0 tx=0).stop 2024-04-20T21:22:57.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.562+0000 7f7042e00640 1 -- 172.21.15.86:0/12258074 shutdown_connections 2024-04-20T21:22:57.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.562+0000 7f7042e00640 1 --2- 172.21.15.86:0/12258074 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f701806c8f0 0x7f701806edb0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:57.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.562+0000 7f7042e00640 1 --2- 172.21.15.86:0/12258074 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f703c0b6b30 0x7f703c0cd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:57.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.562+0000 7f7042e00640 1 --2- 172.21.15.86:0/12258074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f703c0ce790 0x7f703c14faf0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:57.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.562+0000 7f7042e00640 1 --2- 172.21.15.86:0/12258074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f703c06dbb0 0x7f703c0b7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:57.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.562+0000 7f7042e00640 1 -- 172.21.15.86:0/12258074 >> 172.21.15.86:0/12258074 conn(0x7f703c0b3fe0 msgr2=0x7f703c0cb5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:57.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.562+0000 7f7042e00640 1 -- 172.21.15.86:0/12258074 shutdown_connections 2024-04-20T21:22:57.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.562+0000 7f7042e00640 1 -- 172.21.15.86:0/12258074 wait complete. 2024-04-20T21:22:57.579 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T21:22:57.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.768+0000 7fe866000640 1 Processor -- start 2024-04-20T21:22:57.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.768+0000 7fe866000640 1 -- start start 2024-04-20T21:22:57.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.770+0000 7fe866000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe86006e6d0 0x7fe860071d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:57.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.770+0000 7fe866000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe86006eb30 con 0x7fe860146130 2024-04-20T21:22:57.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.770+0000 7fe866000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe86006dc30 con 0x7fe860072240 2024-04-20T21:22:57.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.770+0000 7fe866000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fe860072690 con 0x7fe86006e6d0 2024-04-20T21:22:57.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.770+0000 7fe85f400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe860146130 0x7fe8600ccdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54852/0 (socket says 172.21.15.86:54852) 2024-04-20T21:22:57.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.770+0000 7fe85f400640 1 -- 172.21.15.86:0/3971393227 learned_addr learned my addr 172.21.15.86:0/3971393227 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:22:57.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.770+0000 7fe85e000640 1 -- 172.21.15.86:0/3971393227 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1244655562 0 0) 0x7fe850001110 con 0x7fe860146130 2024-04-20T21:22:57.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.770+0000 7fe85e000640 1 -- 172.21.15.86:0/3971393227 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe844003520 con 0x7fe860146130 2024-04-20T21:22:57.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.770+0000 7fe85ea00640 1 --2- 172.21.15.86:0/3971393227 >> v2:172.21.15.144:3300/0 conn(0x7fe86006e6d0 0x7fe860071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:57.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.771+0000 7fe85e000640 1 -- 172.21.15.86:0/3971393227 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2764341732 0 0) 0x7fe844003520 con 0x7fe860146130 2024-04-20T21:22:57.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.771+0000 7fe85e000640 1 -- 172.21.15.86:0/3971393227 >> v2:172.21.15.144:3300/0 conn(0x7fe86006e6d0 msgr2=0x7fe860071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:57.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.771+0000 7fe85e000640 1 --2- 172.21.15.86:0/3971393227 >> v2:172.21.15.144:3300/0 conn(0x7fe86006e6d0 0x7fe860071d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:57.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.771+0000 7fe85e000640 1 -- 172.21.15.86:0/3971393227 >> v1:172.21.15.144:6789/0 conn(0x7fe860072240 legacy=0x7fe86015cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:57.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.771+0000 7fe85e000640 1 -- 172.21.15.86:0/3971393227 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8600cd8e0 con 0x7fe860146130 2024-04-20T21:22:57.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.771+0000 7fe85e000640 1 -- 172.21.15.86:0/3971393227 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fe850001840 con 0x7fe860146130 2024-04-20T21:22:57.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.771+0000 7fe85e000640 1 -- 172.21.15.86:0/3971393227 >> v1:172.21.15.86:6789/0 conn(0x7fe860146130 legacy=0x7fe8600ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:57.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.771+0000 7fe85e000640 1 --2- 172.21.15.86:0/3971393227 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8440039b0 0x7fe844003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:57.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.771+0000 7fe85e000640 1 --2- 172.21.15.86:0/3971393227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe844004800 0x7fe844004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:57.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.771+0000 7fe85e000640 1 --2- 172.21.15.86:0/3971393227 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe8440051c0 0x7fe84400fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:57.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.771+0000 7fe85e000640 1 -- 172.21.15.86:0/3971393227 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe860072690 con 0x7fe844004800 2024-04-20T21:22:57.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.771+0000 7fe85e000640 1 -- 172.21.15.86:0/3971393227 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe844010500 con 0x7fe8440039b0 2024-04-20T21:22:57.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.771+0000 7fe85e000640 1 -- 172.21.15.86:0/3971393227 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe844010700 con 0x7fe8440051c0 2024-04-20T21:22:57.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.771+0000 7fe85ea00640 1 --2- 172.21.15.86:0/3971393227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe844004800 0x7fe844004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:57.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.772+0000 7fe85ea00640 1 -- 172.21.15.86:0/3971393227 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe8440051c0 msgr2=0x7fe84400fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:57.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.772+0000 7fe85ea00640 1 --2- 172.21.15.86:0/3971393227 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe8440051c0 0x7fe84400fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:57.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.772+0000 7fe85ea00640 1 -- 172.21.15.86:0/3971393227 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8440039b0 msgr2=0x7fe844003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:22:57.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.772+0000 7fe85ea00640 1 --2- 172.21.15.86:0/3971393227 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe8440039b0 0x7fe844003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:57.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.772+0000 7fe85ea00640 1 -- 172.21.15.86:0/3971393227 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe844003520 con 0x7fe844004800 2024-04-20T21:22:57.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.772+0000 7fe85fe00640 1 -- 172.21.15.86:0/3971393227 reap_dead start 2024-04-20T21:22:57.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.773+0000 7fe85ea00640 1 --2- 172.21.15.86:0/3971393227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe844004800 0x7fe844004c80 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fe84c00ba30 tx=0x7fe844005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7c1e90d3852c49e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:57.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.773+0000 7fe85e000640 1 -- 172.21.15.86:0/3971393227 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84c036020 con 0x7fe844004800 2024-04-20T21:22:57.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.773+0000 7fe85e000640 1 -- 172.21.15.86:0/3971393227 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe84c032630 con 0x7fe844004800 2024-04-20T21:22:57.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.773+0000 7fe85e000640 1 -- 172.21.15.86:0/3971393227 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84c00b600 con 0x7fe844004800 2024-04-20T21:22:57.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.774+0000 7fe866000640 1 -- 172.21.15.86:0/3971393227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe844004800 msgr2=0x7fe844004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:57.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.774+0000 7fe866000640 1 --2- 172.21.15.86:0/3971393227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe844004800 0x7fe844004c80 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fe84c00ba30 tx=0x7fe844005830 comp rx=0 tx=0).stop 2024-04-20T21:22:57.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.774+0000 7fe866000640 1 -- 172.21.15.86:0/3971393227 shutdown_connections 2024-04-20T21:22:57.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.774+0000 7fe866000640 1 --2- 172.21.15.86:0/3971393227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe844004800 0x7fe844004c80 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:57.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.774+0000 7fe866000640 1 -- 172.21.15.86:0/3971393227 >> 172.21.15.86:0/3971393227 conn(0x7fe8600b2360 msgr2=0x7fe8600b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:57.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.774+0000 7fe866000640 1 -- 172.21.15.86:0/3971393227 shutdown_connections 2024-04-20T21:22:57.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.774+0000 7fe866000640 1 -- 172.21.15.86:0/3971393227 wait complete. 2024-04-20T21:22:57.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.775+0000 7fe866000640 1 Processor -- start 2024-04-20T21:22:57.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.776+0000 7fe866000640 1 -- start start 2024-04-20T21:22:57.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.776+0000 7fe866000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe844004800 0x7fe86015b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:57.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.776+0000 7fe866000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe86015bc20 0x7fe8600b4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:57.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.776+0000 7fe866000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe86015e060 0x7fe8600b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:57.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.776+0000 7fe866000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe86006e1b0 con 0x7fe844004800 2024-04-20T21:22:57.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.776+0000 7fe866000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe86006ded0 con 0x7fe86015e060 2024-04-20T21:22:57.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.776+0000 7fe866000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe86006f150 con 0x7fe86015bc20 2024-04-20T21:22:57.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.777+0000 7fe85f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe844004800 0x7fe86015b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:57.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.777+0000 7fe85f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe844004800 0x7fe86015b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53132/0 (socket says 172.21.15.86:53132) 2024-04-20T21:22:57.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.777+0000 7fe85f400640 1 -- 172.21.15.86:0/3877636393 learned_addr learned my addr 172.21.15.86:0/3877636393 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:22:57.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.777+0000 7fe85f400640 1 -- 172.21.15.86:0/3877636393 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe86015bc20 msgr2=0x7fe8600b4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:57.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.777+0000 7fe85ea00640 1 --2- 172.21.15.86:0/3877636393 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe86015bc20 0x7fe8600b4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:57.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.777+0000 7fe85f400640 1 --2- 172.21.15.86:0/3877636393 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe86015bc20 0x7fe8600b4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:57.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.777+0000 7fe85fe00640 1 --2- 172.21.15.86:0/3877636393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe86015e060 0x7fe8600b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:57.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.777+0000 7fe85f400640 1 -- 172.21.15.86:0/3877636393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe86015e060 msgr2=0x7fe8600b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:57.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.777+0000 7fe85f400640 1 --2- 172.21.15.86:0/3877636393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe86015e060 0x7fe8600b47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:57.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.777+0000 7fe85f400640 1 -- 172.21.15.86:0/3877636393 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe86015e600 con 0x7fe844004800 2024-04-20T21:22:57.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.778+0000 7fe85f400640 1 --2- 172.21.15.86:0/3877636393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe844004800 0x7fe86015b440 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fe85000c730 tx=0x7fe85000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:57.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.778+0000 7fe857400640 1 -- 172.21.15.86:0/3877636393 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe850018070 con 0x7fe844004800 2024-04-20T21:22:57.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.778+0000 7fe857400640 1 -- 172.21.15.86:0/3877636393 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8500072f0 con 0x7fe844004800 2024-04-20T21:22:57.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.778+0000 7fe866000640 1 -- 172.21.15.86:0/3877636393 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe86015e8e0 con 0x7fe844004800 2024-04-20T21:22:57.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.778+0000 7fe857400640 1 -- 172.21.15.86:0/3877636393 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8500047e0 con 0x7fe844004800 2024-04-20T21:22:57.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.778+0000 7fe866000640 1 -- 172.21.15.86:0/3877636393 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe86015eea0 con 0x7fe844004800 2024-04-20T21:22:57.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.779+0000 7fe866000640 1 -- 172.21.15.86:0/3877636393 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8600689e0 con 0x7fe844004800 2024-04-20T21:22:57.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.780+0000 7fe857400640 1 -- 172.21.15.86:0/3877636393 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe85001b070 con 0x7fe844004800 2024-04-20T21:22:57.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.781+0000 7fe857400640 1 --2- 172.21.15.86:0/3877636393 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe84006c920 0x7fe84006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:22:57.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.781+0000 7fe857400640 1 -- 172.21.15.86:0/3877636393 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7fe85008b060 con 0x7fe844004800 2024-04-20T21:22:57.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.781+0000 7fe85ea00640 1 --2- 172.21.15.86:0/3877636393 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe84006c920 0x7fe84006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:22:57.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.782+0000 7fe85ea00640 1 --2- 172.21.15.86:0/3877636393 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe84006c920 0x7fe84006ede0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe84c00ba00 tx=0x7fe84c030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:22:57.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:57.786+0000 7fe857400640 1 -- 172.21.15.86:0/3877636393 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe86006eb30 con 0x7fe844004800 2024-04-20T21:22:58.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:58.027+0000 7fe866000640 1 -- 172.21.15.86:0/3877636393 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe86015ff50 con 0x7fe844004800 2024-04-20T21:22:58.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:58.027+0000 7fe857400640 1 -- 172.21.15.86:0/3877636393 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v42) ==== 53+0+73 (secure 0 0 0) 0x7fe8500593f0 con 0x7fe844004800 2024-04-20T21:22:58.026 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T21:22:58.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:58.030+0000 7fe866000640 1 -- 172.21.15.86:0/3877636393 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe84006c920 msgr2=0x7fe84006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:58.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:58.030+0000 7fe866000640 1 --2- 172.21.15.86:0/3877636393 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe84006c920 0x7fe84006ede0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe84c00ba00 tx=0x7fe84c030000 comp rx=0 tx=0).stop 2024-04-20T21:22:58.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:58.030+0000 7fe866000640 1 -- 172.21.15.86:0/3877636393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe844004800 msgr2=0x7fe86015b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:22:58.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:58.030+0000 7fe866000640 1 --2- 172.21.15.86:0/3877636393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe844004800 0x7fe86015b440 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fe85000c730 tx=0x7fe85000cbf0 comp rx=0 tx=0).stop 2024-04-20T21:22:58.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:58.030+0000 7fe866000640 1 -- 172.21.15.86:0/3877636393 shutdown_connections 2024-04-20T21:22:58.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:58.030+0000 7fe866000640 1 --2- 172.21.15.86:0/3877636393 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe84006c920 0x7fe84006ede0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:58.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:58.030+0000 7fe866000640 1 --2- 172.21.15.86:0/3877636393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe86015e060 0x7fe8600b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:58.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:58.030+0000 7fe866000640 1 --2- 172.21.15.86:0/3877636393 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe86015bc20 0x7fe8600b4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:58.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:58.030+0000 7fe866000640 1 --2- 172.21.15.86:0/3877636393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe844004800 0x7fe86015b440 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:22:58.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:58.030+0000 7fe866000640 1 -- 172.21.15.86:0/3877636393 >> 172.21.15.86:0/3877636393 conn(0x7fe8600b2360 msgr2=0x7fe8601422d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:22:58.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:58.030+0000 7fe866000640 1 -- 172.21.15.86:0/3877636393 shutdown_connections 2024-04-20T21:22:58.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:58.030+0000 7fe866000640 1 -- 172.21.15.86:0/3877636393 wait complete. 2024-04-20T21:22:58.047 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T21:22:58.047 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T21:22:58.047 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-04-20T21:22:58.047 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-20T21:22:58.047 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T21:22:58.047 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T21:22:58.047 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T21:22:58.047 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T21:22:58.047 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:22:58.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:22:58.048 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-04-20T21:22:58.096 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T21:22:58.096 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T21:22:58.096 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T21:22:58.096 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:22:58.096 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-20T21:22:58.096 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:22:58.096 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:22:58.096 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:22:58.097 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-04-20T21:22:58.097 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:22:58.097 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:22:58.097 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-04-20T21:22:58.097 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:22:58.097 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:22:58.097 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T21:22:58.097 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:22:58.097 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T21:22:58.097 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42378sec preferred_lft 42378sec 2024-04-20T21:22:58.097 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-04-20T21:22:58.097 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:22:58.097 INFO:teuthology.orchestra.run.smithi086.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T21:22:58.097 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 86:25:97:2c:9c:bb brd ff:ff:ff:ff:ff:ff 2024-04-20T21:22:58.097 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-20T21:22:58.097 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:22:58.098 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::a4fb:30ff:fe90:2d6a/64 scope link 2024-04-20T21:22:58.098 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:22:58.098 INFO:teuthology.orchestra.run.smithi086.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T21:22:58.098 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 86:25:97:2c:9c:bb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:22:58.098 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::8425:97ff:fe2c:9cbb/64 scope link 2024-04-20T21:22:58.098 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:22:58.098 INFO:teuthology.orchestra.run.smithi086.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T21:22:58.098 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c2:9c:8b:55:ca:84 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:22:58.098 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c09c:8bff:fe55:ca84/64 scope link 2024-04-20T21:22:58.098 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:22:58.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:22:58.099 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T21:22:58.219 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-20T21:22:58.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:22:58.224 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-04-20T21:22:58.276 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-04-20T21:22:58.276 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T21:22:58.276 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T21:22:58.276 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T21:22:58.276 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T21:22:58.277 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:22:58.277 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-20T21:22:58.277 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T21:22:58.278 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T21:22:58.278 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T21:22:58.278 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-20T21:22:58.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:22:58.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:22:58.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:22:58.485 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-20T21:22:58.537 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-20T21:22:58.537 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-20T21:22:58.538 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:22:58.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:22:58.538 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-04-20T21:22:58.616 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-20T21:22:58.616 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T21:22:58.616 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-20T21:22:58.666 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:22:58.667 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T21:22:58.667 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-20T21:22:58.720 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-04-20T21:22:58.804 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T21:22:58.860 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-04-20T21:22:58.861 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-04-20T21:22:58.861 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-04-20T21:22:58.861 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-04-20T21:22:58.861 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-04-20T21:22:58.861 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-04-20T21:22:58.861 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-04-20T21:22:58.861 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-04-20T21:22:58.861 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-04-20T21:22:58.861 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-04-20T21:22:58.861 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-04-20T21:22:58.861 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/selinux 2024-04-20T21:22:58.861 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T21:22:58.861 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-04-20T21:22:58.862 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-04-20T21:22:58.862 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-04-20T21:22:58.862 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-04-20T21:22:58.862 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-04-20T21:22:58.862 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-04-20T21:22:58.862 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T21:22:58.862 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T21:22:58.862 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T21:22:58.862 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T21:22:58.862 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-04-20T21:22:58.862 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-04-20T21:22:58.862 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T21:22:58.862 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T21:22:58.862 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T21:22:58.863 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T21:22:58.863 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-04-20T21:22:58.863 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:22:58.863 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:22:58.863 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:22:58.863 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:22:58.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:22:58.864 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-04-20T21:22:58.915 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T21:22:58.915 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-20T21:22:58.922 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-04-20T21:22:58.994 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T21:22:59.038 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-04-20T21:22:59.038 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-04-20T21:22:59.038 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-04-20T21:22:59.038 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-04-20T21:22:59.039 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-04-20T21:22:59.039 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-04-20T21:22:59.039 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-04-20T21:22:59.039 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-04-20T21:22:59.039 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-04-20T21:22:59.039 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-04-20T21:22:59.039 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-04-20T21:22:59.039 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/selinux 2024-04-20T21:22:59.039 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T21:22:59.039 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-04-20T21:22:59.039 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-04-20T21:22:59.039 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-04-20T21:22:59.039 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-04-20T21:22:59.039 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-04-20T21:22:59.040 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-04-20T21:22:59.040 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T21:22:59.040 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T21:22:59.040 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T21:22:59.040 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T21:22:59.040 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-04-20T21:22:59.040 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-04-20T21:22:59.040 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T21:22:59.040 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T21:22:59.040 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T21:22:59.040 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T21:22:59.040 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-04-20T21:22:59.040 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:22:59.040 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:22:59.041 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:22:59.041 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:22:59.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:22:59.042 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-04-20T21:22:59.088 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:2024-04-20T21:22:59.089+0000 7f1a46bb4480 -1 init, newargv = 0x55fa8416a2a0 newargc=15 2024-04-20T21:22:59.088 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[47282]: starting ceph client 2024-04-20T21:22:59.095 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[47282]: starting fuse 2024-04-20T21:22:59.123 INFO:teuthology.orchestra.run.smithi086.stdout:41 2024-04-20T21:22:59.124 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-04-20T21:22:59.124 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer -- python3 -c ' 2024-04-20T21:22:59.124 DEBUG:teuthology.orchestra.run.smithi086:> import glob 2024-04-20T21:22:59.124 DEBUG:teuthology.orchestra.run.smithi086:> import re 2024-04-20T21:22:59.124 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:22:59.124 DEBUG:teuthology.orchestra.run.smithi086:> import subprocess 2024-04-20T21:22:59.124 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:22:59.124 DEBUG:teuthology.orchestra.run.smithi086:> def _find_admin_socket(client_name): 2024-04-20T21:22:59.124 DEBUG:teuthology.orchestra.run.smithi086:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-20T21:22:59.124 DEBUG:teuthology.orchestra.run.smithi086:> files = glob.glob(asok_path) 2024-04-20T21:22:59.124 DEBUG:teuthology.orchestra.run.smithi086:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-20T21:22:59.125 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:22:59.125 DEBUG:teuthology.orchestra.run.smithi086:> # Given a non-glob path, it better be there 2024-04-20T21:22:59.125 DEBUG:teuthology.orchestra.run.smithi086:> if "*" not in asok_path: 2024-04-20T21:22:59.125 DEBUG:teuthology.orchestra.run.smithi086:> assert(len(files) == 1) 2024-04-20T21:22:59.125 DEBUG:teuthology.orchestra.run.smithi086:> return files[0] 2024-04-20T21:22:59.125 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:22:59.125 DEBUG:teuthology.orchestra.run.smithi086:> for f in files: 2024-04-20T21:22:59.125 DEBUG:teuthology.orchestra.run.smithi086:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T21:22:59.125 DEBUG:teuthology.orchestra.run.smithi086:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T21:22:59.125 DEBUG:teuthology.orchestra.run.smithi086:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T21:22:59.125 DEBUG:teuthology.orchestra.run.smithi086:> contents = proc_f.read() 2024-04-20T21:22:59.125 DEBUG:teuthology.orchestra.run.smithi086:> if mountpoint in contents: 2024-04-20T21:22:59.125 DEBUG:teuthology.orchestra.run.smithi086:> return f 2024-04-20T21:22:59.125 DEBUG:teuthology.orchestra.run.smithi086:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T21:22:59.125 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:22:59.125 DEBUG:teuthology.orchestra.run.smithi086:> print(_find_admin_socket("client.0")) 2024-04-20T21:22:59.125 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:22:59.321 INFO:teuthology.orchestra.run.smithi086.stdout:/var/run/ceph/ceph-client.0.47282.asok 2024-04-20T21:22:59.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:22:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:22:59.340 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.47282.asok 2024-04-20T21:22:59.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:22:59.340 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.47282.asok status 2024-04-20T21:22:59.568 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:22:59.568 INFO:teuthology.orchestra.run.smithi086.stdout: "metadata": { 2024-04-20T21:22:59.568 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T21:22:59.568 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T21:22:59.568 INFO:teuthology.orchestra.run.smithi086.stdout: "entity_id": "0", 2024-04-20T21:22:59.568 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": "smithi086", 2024-04-20T21:22:59.568 INFO:teuthology.orchestra.run.smithi086.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-20T21:22:59.569 INFO:teuthology.orchestra.run.smithi086.stdout: "pid": "47282", 2024-04-20T21:22:59.569 INFO:teuthology.orchestra.run.smithi086.stdout: "root": "/" 2024-04-20T21:22:59.569 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:22:59.569 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_count": 0, 2024-04-20T21:22:59.569 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_pinned_count": 0, 2024-04-20T21:22:59.569 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 6894, 2024-04-20T21:22:59.569 INFO:teuthology.orchestra.run.smithi086.stdout: "inst": { 2024-04-20T21:22:59.569 INFO:teuthology.orchestra.run.smithi086.stdout: "name": { 2024-04-20T21:22:59.569 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "client", 2024-04-20T21:22:59.569 INFO:teuthology.orchestra.run.smithi086.stdout: "num": 6894 2024-04-20T21:22:59.569 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:22:59.569 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2024-04-20T21:22:59.570 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-04-20T21:22:59.570 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2024-04-20T21:22:59.570 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 1933391013 2024-04-20T21:22:59.570 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:22:59.570 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:22:59.570 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2024-04-20T21:22:59.570 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-04-20T21:22:59.570 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2024-04-20T21:22:59.570 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 1933391013 2024-04-20T21:22:59.570 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:22:59.570 INFO:teuthology.orchestra.run.smithi086.stdout: "inst_str": "client.6894 192.168.0.1:0/1933391013", 2024-04-20T21:22:59.570 INFO:teuthology.orchestra.run.smithi086.stdout: "addr_str": "192.168.0.1:0/1933391013", 2024-04-20T21:22:59.570 INFO:teuthology.orchestra.run.smithi086.stdout: "inode_count": 1, 2024-04-20T21:22:59.571 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_epoch": 42, 2024-04-20T21:22:59.571 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch": 71, 2024-04-20T21:22:59.571 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch_barrier": 0, 2024-04-20T21:22:59.571 INFO:teuthology.orchestra.run.smithi086.stdout: "blocklisted": false, 2024-04-20T21:22:59.571 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs" 2024-04-20T21:22:59.571 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:22:59.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:22:59.583 DEBUG:teuthology.orchestra.run.smithi086:> stat --file-system '--printf=%T 2024-04-20T21:22:59.583 DEBUG:teuthology.orchestra.run.smithi086:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-20T21:22:59.633 INFO:teuthology.orchestra.run.smithi086.stdout:fuseblk 2024-04-20T21:22:59.633 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-20T21:22:59.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:22:59.634 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-20T21:22:59.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:22:59.708 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir/) 2024-04-20T21:22:59.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:22:59.771 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_undamaged) 2024-04-20T21:22:59.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:22:59.832 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_to_be_damaged) 2024-04-20T21:22:59.891 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-04-20T21:22:59.891 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:22:59.891 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-04-20T21:22:59.891 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:22:59.891 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/subdir").st_ino) 2024-04-20T21:22:59.891 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:23:00.030 INFO:teuthology.orchestra.run.smithi086.stdout:1099511627776 2024-04-20T21:23:00.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:23:00.044 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:23:00.044 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:23:00.141 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:23:00.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.339+0000 7f4499c00640 1 Processor -- start 2024-04-20T21:23:00.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.339+0000 7f4499c00640 1 -- start start 2024-04-20T21:23:00.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.340+0000 7f4499c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f449415d6a0 0x7f44940ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:00.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.340+0000 7f4499c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f44940ad620 0x7f449406e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:00.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.340+0000 7f4499c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4494160890 con 0x7f4494156ae0 2024-04-20T21:23:00.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.341+0000 7f4499c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f449406db20 con 0x7f44940ad620 2024-04-20T21:23:00.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.341+0000 7f4499c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f4494160b70 con 0x7f449415d6a0 2024-04-20T21:23:00.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.341+0000 7f4493e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f4494156ae0 0x7f4494160080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54864/0 (socket says 172.21.15.86:54864) 2024-04-20T21:23:00.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.341+0000 7f4493e00640 1 -- 172.21.15.86:0/4230335984 learned_addr learned my addr 172.21.15.86:0/4230335984 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:23:00.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.341+0000 7f4499200640 1 --2- 172.21.15.86:0/4230335984 >> v2:172.21.15.86:3300/0 conn(0x7f44940ad620 0x7f449406e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:00.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.341+0000 7f4499200640 1 -- 172.21.15.86:0/4230335984 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f449415d6a0 msgr2=0x7f44940ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:00.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.341+0000 7f4493400640 1 --2- 172.21.15.86:0/4230335984 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f449415d6a0 0x7f44940ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:00.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.341+0000 7f4492a00640 1 -- 172.21.15.86:0/4230335984 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2744143676 0 0) 0x7f4494160890 con 0x7f4494156ae0 2024-04-20T21:23:00.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.341+0000 7f4499200640 1 --2- 172.21.15.86:0/4230335984 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f449415d6a0 0x7f44940ad0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:00.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.341+0000 7f4499200640 1 -- 172.21.15.86:0/4230335984 >> v1:172.21.15.86:6789/0 conn(0x7f4494156ae0 legacy=0x7f4494160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:00.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.341+0000 7f4499200640 1 -- 172.21.15.86:0/4230335984 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4494160e50 con 0x7f44940ad620 2024-04-20T21:23:00.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.343+0000 7f4499200640 1 --2- 172.21.15.86:0/4230335984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44940ad620 0x7f449406e290 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f448c009d90 tx=0x7f448c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d75808fa4c346af server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:00.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.343+0000 7f4492a00640 1 -- 172.21.15.86:0/4230335984 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f448c039070 con 0x7f44940ad620 2024-04-20T21:23:00.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.343+0000 7f4492a00640 1 -- 172.21.15.86:0/4230335984 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f448c02cc50 con 0x7f44940ad620 2024-04-20T21:23:00.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.343+0000 7f4492a00640 1 -- 172.21.15.86:0/4230335984 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f448c034440 con 0x7f44940ad620 2024-04-20T21:23:00.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.344+0000 7f4499c00640 1 -- 172.21.15.86:0/4230335984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44940ad620 msgr2=0x7f449406e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:00.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.344+0000 7f4499c00640 1 --2- 172.21.15.86:0/4230335984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44940ad620 0x7f449406e290 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f448c009d90 tx=0x7f448c02bea0 comp rx=0 tx=0).stop 2024-04-20T21:23:00.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.344+0000 7f4499c00640 1 -- 172.21.15.86:0/4230335984 shutdown_connections 2024-04-20T21:23:00.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.344+0000 7f4499c00640 1 --2- 172.21.15.86:0/4230335984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44940ad620 0x7f449406e290 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:00.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.344+0000 7f4499c00640 1 --2- 172.21.15.86:0/4230335984 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f449415d6a0 0x7f44940ad0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:00.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.344+0000 7f4499c00640 1 -- 172.21.15.86:0/4230335984 >> 172.21.15.86:0/4230335984 conn(0x7f44940d0170 msgr2=0x7f44940d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:00.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.344+0000 7f4499c00640 1 -- 172.21.15.86:0/4230335984 shutdown_connections 2024-04-20T21:23:00.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.344+0000 7f4499c00640 1 -- 172.21.15.86:0/4230335984 wait complete. 2024-04-20T21:23:00.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.346+0000 7f4499c00640 1 Processor -- start 2024-04-20T21:23:00.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.346+0000 7f4499c00640 1 -- start start 2024-04-20T21:23:00.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.347+0000 7f4499c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44940ad620 0x7f44941f6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:00.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.347+0000 7f4499c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4494156ae0 0x7f44941f1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:00.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.347+0000 7f4499c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f449415d6a0 0x7f44941546d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:00.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.347+0000 7f4499c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f449406e7d0 con 0x7f449415d6a0 2024-04-20T21:23:00.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.347+0000 7f4499c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4494156ec0 con 0x7f4494156ae0 2024-04-20T21:23:00.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.347+0000 7f4499c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f449406e950 con 0x7f44940ad620 2024-04-20T21:23:00.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.347+0000 7f4499200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f449415d6a0 0x7f44941546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:00.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.347+0000 7f4499200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f449415d6a0 0x7f44941546d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53150/0 (socket says 172.21.15.86:53150) 2024-04-20T21:23:00.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.347+0000 7f4499200640 1 -- 172.21.15.86:0/2349084642 learned_addr learned my addr 172.21.15.86:0/2349084642 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:00.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.347+0000 7f4493e00640 1 --2- 172.21.15.86:0/2349084642 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44940ad620 0x7f44941f6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:00.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.348+0000 7f4493400640 1 --2- 172.21.15.86:0/2349084642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4494156ae0 0x7f44941f1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:00.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.348+0000 7f4499200640 1 -- 172.21.15.86:0/2349084642 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44940ad620 msgr2=0x7f44941f6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:00.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.348+0000 7f4499200640 1 --2- 172.21.15.86:0/2349084642 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44940ad620 0x7f44941f6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:00.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.348+0000 7f4499200640 1 -- 172.21.15.86:0/2349084642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4494156ae0 msgr2=0x7f44941f1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:00.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.348+0000 7f4499200640 1 --2- 172.21.15.86:0/2349084642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4494156ae0 0x7f44941f1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:00.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.348+0000 7f4499200640 1 -- 172.21.15.86:0/2349084642 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44941444c0 con 0x7f449415d6a0 2024-04-20T21:23:00.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.348+0000 7f4493e00640 1 --2- 172.21.15.86:0/2349084642 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44940ad620 0x7f44941f6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:23:00.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.348+0000 7f4493400640 1 --2- 172.21.15.86:0/2349084642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4494156ae0 0x7f44941f1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:23:00.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.348+0000 7f4499200640 1 --2- 172.21.15.86:0/2349084642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f449415d6a0 0x7f44941546d0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f448c009d90 tx=0x7f448c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:00.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.348+0000 7f448be00640 1 -- 172.21.15.86:0/2349084642 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f448c039050 con 0x7f449415d6a0 2024-04-20T21:23:00.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.348+0000 7f448be00640 1 -- 172.21.15.86:0/2349084642 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f448c045070 con 0x7f449415d6a0 2024-04-20T21:23:00.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.348+0000 7f448be00640 1 -- 172.21.15.86:0/2349084642 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f448c03e610 con 0x7f449415d6a0 2024-04-20T21:23:00.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.348+0000 7f4499c00640 1 -- 172.21.15.86:0/2349084642 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44941447a0 con 0x7f449415d6a0 2024-04-20T21:23:00.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.349+0000 7f4499c00640 1 -- 172.21.15.86:0/2349084642 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4494144d60 con 0x7f449415d6a0 2024-04-20T21:23:00.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.350+0000 7f4499c00640 1 -- 172.21.15.86:0/2349084642 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f449404fcf0 con 0x7f449415d6a0 2024-04-20T21:23:00.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.351+0000 7f448be00640 1 -- 172.21.15.86:0/2349084642 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f448c03e7b0 con 0x7f449415d6a0 2024-04-20T21:23:00.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.352+0000 7f448be00640 1 --2- 172.21.15.86:0/2349084642 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f447406c8f0 0x7f447406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:00.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.352+0000 7f448be00640 1 -- 172.21.15.86:0/2349084642 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f448c0ae700 con 0x7f449415d6a0 2024-04-20T21:23:00.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.352+0000 7f4493e00640 1 --2- 172.21.15.86:0/2349084642 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f447406c8f0 0x7f447406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:00.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.353+0000 7f4493e00640 1 --2- 172.21.15.86:0/2349084642 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f447406c8f0 0x7f447406edb0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4484000f60 tx=0x7f4484009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:00.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.356+0000 7f448be00640 1 -- 172.21.15.86:0/2349084642 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f448c032e20 con 0x7f449415d6a0 2024-04-20T21:23:00.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.558+0000 7f4499c00640 1 -- 172.21.15.86:0/2349084642 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f449415c6d0 con 0x7f449415d6a0 2024-04-20T21:23:00.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.561+0000 7f448be00640 1 -- 172.21.15.86:0/2349084642 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+16668 (secure 0 0 0) 0x7f448c07cb20 con 0x7f449415d6a0 2024-04-20T21:23:00.563 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:23:00.563 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":71,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:22:46.983093+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:22:43.125767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-20T21:22:44.410549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:23:00.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.563+0000 7f4499c00640 1 -- 172.21.15.86:0/2349084642 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f447406c8f0 msgr2=0x7f447406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:00.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.563+0000 7f4499c00640 1 --2- 172.21.15.86:0/2349084642 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f447406c8f0 0x7f447406edb0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4484000f60 tx=0x7f4484009040 comp rx=0 tx=0).stop 2024-04-20T21:23:00.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.563+0000 7f4499c00640 1 -- 172.21.15.86:0/2349084642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f449415d6a0 msgr2=0x7f44941546d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:00.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.563+0000 7f4499c00640 1 --2- 172.21.15.86:0/2349084642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f449415d6a0 0x7f44941546d0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f448c009d90 tx=0x7f448c004a40 comp rx=0 tx=0).stop 2024-04-20T21:23:00.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.564+0000 7f4499c00640 1 -- 172.21.15.86:0/2349084642 shutdown_connections 2024-04-20T21:23:00.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.564+0000 7f4499c00640 1 --2- 172.21.15.86:0/2349084642 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f447406c8f0 0x7f447406edb0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:00.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.564+0000 7f4499c00640 1 --2- 172.21.15.86:0/2349084642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f449415d6a0 0x7f44941546d0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:00.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.564+0000 7f4499c00640 1 --2- 172.21.15.86:0/2349084642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4494156ae0 0x7f44941f1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:00.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.564+0000 7f4499c00640 1 --2- 172.21.15.86:0/2349084642 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44940ad620 0x7f44941f6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:00.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.564+0000 7f4499c00640 1 -- 172.21.15.86:0/2349084642 >> 172.21.15.86:0/2349084642 conn(0x7f44940d0170 msgr2=0x7f44940b1490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:00.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.564+0000 7f4499c00640 1 -- 172.21.15.86:0/2349084642 shutdown_connections 2024-04-20T21:23:00.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.564+0000 7f4499c00640 1 -- 172.21.15.86:0/2349084642 wait complete. 2024-04-20T21:23:00.583 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:22:43.125767+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:22:44.410549+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:23:00.583 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:23:00.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.789+0000 7f5909400640 1 Processor -- start 2024-04-20T21:23:00.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.789+0000 7f5909400640 1 -- start start 2024-04-20T21:23:00.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.790+0000 7f5909400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5904155d30 0x7f59040cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:00.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.791+0000 7f5909400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f59040b52b0 0x7f590406be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:00.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.791+0000 7f5909400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5904156110 con 0x7f590406b980 2024-04-20T21:23:00.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.791+0000 7f5909400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f59040b56e0 con 0x7f59040b52b0 2024-04-20T21:23:00.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.791+0000 7f5909400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f59040ce460 con 0x7f5904155d30 2024-04-20T21:23:00.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.791+0000 7f5903200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f59040b52b0 0x7f590406be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:00.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.791+0000 7f58f9e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f590406b980 0x7f59040b4ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54874/0 (socket says 172.21.15.86:54874) 2024-04-20T21:23:00.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.791+0000 7f5903200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f59040b52b0 0x7f590406be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53154/0 (socket says 172.21.15.86:53154) 2024-04-20T21:23:00.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.791+0000 7f58f9e00640 1 -- 172.21.15.86:0/3656436189 learned_addr learned my addr 172.21.15.86:0/3656436189 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:23:00.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.791+0000 7f5902800640 1 --2- 172.21.15.86:0/3656436189 >> v2:172.21.15.144:3300/0 conn(0x7f5904155d30 0x7f59040cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:00.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.791+0000 7f5903200640 1 -- 172.21.15.86:0/3656436189 >> v2:172.21.15.144:3300/0 conn(0x7f5904155d30 msgr2=0x7f59040cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:00.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.791+0000 7f5903200640 1 --2- 172.21.15.86:0/3656436189 >> v2:172.21.15.144:3300/0 conn(0x7f5904155d30 0x7f59040cf500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:00.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.791+0000 7f5903200640 1 -- 172.21.15.86:0/3656436189 >> v1:172.21.15.86:6789/0 conn(0x7f590406b980 legacy=0x7f59040b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:00.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.792+0000 7f5903200640 1 -- 172.21.15.86:0/3656436189 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59040ce740 con 0x7f59040b52b0 2024-04-20T21:23:00.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.793+0000 7f5903200640 1 --2- 172.21.15.86:0/3656436189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59040b52b0 0x7f590406be30 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f59040b3150 tx=0x7f58fc02b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5381cad30e4ff63f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:00.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.793+0000 7f5901e00640 1 -- 172.21.15.86:0/3656436189 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58fc038070 con 0x7f59040b52b0 2024-04-20T21:23:00.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.793+0000 7f5901e00640 1 -- 172.21.15.86:0/3656436189 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58fc031520 con 0x7f59040b52b0 2024-04-20T21:23:00.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.793+0000 7f5901e00640 1 -- 172.21.15.86:0/3656436189 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58fc0318f0 con 0x7f59040b52b0 2024-04-20T21:23:00.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.793+0000 7f5909400640 1 -- 172.21.15.86:0/3656436189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59040b52b0 msgr2=0x7f590406be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:00.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.793+0000 7f5909400640 1 --2- 172.21.15.86:0/3656436189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59040b52b0 0x7f590406be30 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f59040b3150 tx=0x7f58fc02b7c0 comp rx=0 tx=0).stop 2024-04-20T21:23:00.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.794+0000 7f5909400640 1 -- 172.21.15.86:0/3656436189 shutdown_connections 2024-04-20T21:23:00.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.794+0000 7f5909400640 1 --2- 172.21.15.86:0/3656436189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59040b52b0 0x7f590406be30 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:00.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.794+0000 7f5909400640 1 --2- 172.21.15.86:0/3656436189 >> v2:172.21.15.144:3300/0 conn(0x7f5904155d30 0x7f59040cf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:00.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.794+0000 7f5909400640 1 -- 172.21.15.86:0/3656436189 >> 172.21.15.86:0/3656436189 conn(0x7f59040b2110 msgr2=0x7f59040b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:00.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.794+0000 7f5909400640 1 -- 172.21.15.86:0/3656436189 shutdown_connections 2024-04-20T21:23:00.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.794+0000 7f5909400640 1 -- 172.21.15.86:0/3656436189 wait complete. 2024-04-20T21:23:00.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.795+0000 7f5909400640 1 Processor -- start 2024-04-20T21:23:00.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.796+0000 7f5909400640 1 -- start start 2024-04-20T21:23:00.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.797+0000 7f5909400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f590406b980 0x7f5904146e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:00.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.797+0000 7f5909400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59040b52b0 0x7f59041e82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:00.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.797+0000 7f5909400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5904155d30 0x7f59041e87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:00.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.797+0000 7f5909400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f59040cefe0 con 0x7f5904155d30 2024-04-20T21:23:00.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.797+0000 7f5909400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f59040ced00 con 0x7f590406b980 2024-04-20T21:23:00.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.797+0000 7f5909400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f590406ddf0 con 0x7f59040b52b0 2024-04-20T21:23:00.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.797+0000 7f5903200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5904155d30 0x7f59041e87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:00.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.797+0000 7f5903200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5904155d30 0x7f59041e87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53156/0 (socket says 172.21.15.86:53156) 2024-04-20T21:23:00.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.797+0000 7f5903200640 1 -- 172.21.15.86:0/2354608236 learned_addr learned my addr 172.21.15.86:0/2354608236 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:00.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.797+0000 7f5902800640 1 --2- 172.21.15.86:0/2354608236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f590406b980 0x7f5904146e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:00.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.797+0000 7f5903200640 1 -- 172.21.15.86:0/2354608236 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59040b52b0 msgr2=0x7f59041e82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:00.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.797+0000 7f58f9e00640 1 --2- 172.21.15.86:0/2354608236 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59040b52b0 0x7f59041e82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:00.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.797+0000 7f5903200640 1 --2- 172.21.15.86:0/2354608236 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59040b52b0 0x7f59041e82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:00.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.797+0000 7f5903200640 1 -- 172.21.15.86:0/2354608236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f590406b980 msgr2=0x7f5904146e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:00.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.797+0000 7f5903200640 1 --2- 172.21.15.86:0/2354608236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f590406b980 0x7f5904146e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:00.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.797+0000 7f5903200640 1 -- 172.21.15.86:0/2354608236 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f590414d8d0 con 0x7f5904155d30 2024-04-20T21:23:00.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.797+0000 7f5902800640 1 --2- 172.21.15.86:0/2354608236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f590406b980 0x7f5904146e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:23:00.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.798+0000 7f5903200640 1 --2- 172.21.15.86:0/2354608236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5904155d30 0x7f59041e87f0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f58fc009890 tx=0x7f58fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:00.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.798+0000 7f58fb400640 1 -- 172.21.15.86:0/2354608236 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58fc042020 con 0x7f5904155d30 2024-04-20T21:23:00.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.798+0000 7f58fb400640 1 -- 172.21.15.86:0/2354608236 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58fc0338c0 con 0x7f5904155d30 2024-04-20T21:23:00.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.798+0000 7f5909400640 1 -- 172.21.15.86:0/2354608236 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f590414db70 con 0x7f5904155d30 2024-04-20T21:23:00.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.798+0000 7f58fb400640 1 -- 172.21.15.86:0/2354608236 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58fc038040 con 0x7f5904155d30 2024-04-20T21:23:00.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.798+0000 7f5909400640 1 -- 172.21.15.86:0/2354608236 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f590414e410 con 0x7f5904155d30 2024-04-20T21:23:00.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.799+0000 7f5909400640 1 -- 172.21.15.86:0/2354608236 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59040689e0 con 0x7f5904155d30 2024-04-20T21:23:00.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.800+0000 7f58fb400640 1 -- 172.21.15.86:0/2354608236 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f58fc045020 con 0x7f5904155d30 2024-04-20T21:23:00.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.801+0000 7f58fb400640 1 --2- 172.21.15.86:0/2354608236 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f58e006c8f0 0x7f58e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:00.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.802+0000 7f58fb400640 1 -- 172.21.15.86:0/2354608236 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f58fc0ad600 con 0x7f5904155d30 2024-04-20T21:23:00.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.802+0000 7f5902800640 1 --2- 172.21.15.86:0/2354608236 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f58e006c8f0 0x7f58e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:00.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.802+0000 7f5902800640 1 --2- 172.21.15.86:0/2354608236 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f58e006c8f0 0x7f58e006edb0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f58f4009f50 tx=0x7f58f400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:00.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:00.806+0000 7f58fb400640 1 -- 172.21.15.86:0/2354608236 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f58fc04c030 con 0x7f5904155d30 2024-04-20T21:23:01.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.013+0000 7f5909400640 1 -- 172.21.15.86:0/2354608236 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f590414e5e0 con 0x7f5904155d30 2024-04-20T21:23:01.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.013+0000 7f58fb400640 1 -- 172.21.15.86:0/2354608236 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) ==== 93+0+10 (secure 0 0 0) 0x7f58fc07b990 con 0x7f5904155d30 2024-04-20T21:23:01.012 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:23:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.016+0000 7f5909400640 1 -- 172.21.15.86:0/2354608236 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f58e006c8f0 msgr2=0x7f58e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.016+0000 7f5909400640 1 --2- 172.21.15.86:0/2354608236 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f58e006c8f0 0x7f58e006edb0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f58f4009f50 tx=0x7f58f400b040 comp rx=0 tx=0).stop 2024-04-20T21:23:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.016+0000 7f5909400640 1 -- 172.21.15.86:0/2354608236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5904155d30 msgr2=0x7f59041e87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.016+0000 7f5909400640 1 --2- 172.21.15.86:0/2354608236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5904155d30 0x7f59041e87f0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f58fc009890 tx=0x7f58fc0046c0 comp rx=0 tx=0).stop 2024-04-20T21:23:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.016+0000 7f5909400640 1 -- 172.21.15.86:0/2354608236 shutdown_connections 2024-04-20T21:23:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.016+0000 7f5909400640 1 --2- 172.21.15.86:0/2354608236 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f58e006c8f0 0x7f58e006edb0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.016+0000 7f5909400640 1 --2- 172.21.15.86:0/2354608236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5904155d30 0x7f59041e87f0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:01.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.016+0000 7f5909400640 1 --2- 172.21.15.86:0/2354608236 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59040b52b0 0x7f59041e82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:01.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.016+0000 7f5909400640 1 --2- 172.21.15.86:0/2354608236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f590406b980 0x7f5904146e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:01.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.016+0000 7f5909400640 1 -- 172.21.15.86:0/2354608236 >> 172.21.15.86:0/2354608236 conn(0x7f59040b2110 msgr2=0x7f59040d0760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:01.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.016+0000 7f5909400640 1 -- 172.21.15.86:0/2354608236 shutdown_connections 2024-04-20T21:23:01.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.016+0000 7f5909400640 1 -- 172.21.15.86:0/2354608236 wait complete. 2024-04-20T21:23:01.034 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:23:01.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.232+0000 7fbbfe200640 1 Processor -- start 2024-04-20T21:23:01.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.232+0000 7fbbfe200640 1 -- start start 2024-04-20T21:23:01.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.233+0000 7fbbfe200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbbf806f480 0x7fbbf806e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:01.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.233+0000 7fbbfe200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbbf80725d0 0x7fbbf80729e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:01.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.233+0000 7fbbfe200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbf8072f20 con 0x7fbbf815add0 2024-04-20T21:23:01.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.233+0000 7fbbfe200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fbbf806fc60 con 0x7fbbf80725d0 2024-04-20T21:23:01.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.233+0000 7fbbfe200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fbbf80cea10 con 0x7fbbf806f480 2024-04-20T21:23:01.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.234+0000 7fbbfd800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbbf80725d0 0x7fbbf80729e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:01.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.234+0000 7fbbfd800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbbf80725d0 0x7fbbf80729e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53170/0 (socket says 172.21.15.86:53170) 2024-04-20T21:23:01.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.234+0000 7fbbfd800640 1 -- 172.21.15.86:0/294722536 learned_addr learned my addr 172.21.15.86:0/294722536 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:01.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.234+0000 7fbbf7e00640 1 --2- 172.21.15.86:0/294722536 >> v2:172.21.15.144:3300/0 conn(0x7fbbf806f480 0x7fbbf806e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:01.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.234+0000 7fbbfd800640 1 -- 172.21.15.86:0/294722536 >> v2:172.21.15.144:3300/0 conn(0x7fbbf806f480 msgr2=0x7fbbf806e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:01.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.234+0000 7fbbfd800640 1 --2- 172.21.15.86:0/294722536 >> v2:172.21.15.144:3300/0 conn(0x7fbbf806f480 0x7fbbf806e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:01.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.234+0000 7fbbfd800640 1 -- 172.21.15.86:0/294722536 >> v1:172.21.15.144:6789/0 conn(0x7fbbf815add0 legacy=0x7fbbf80cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:01.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.234+0000 7fbbfd800640 1 -- 172.21.15.86:0/294722536 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbf806f980 con 0x7fbbf80725d0 2024-04-20T21:23:01.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.235+0000 7fbbfd800640 1 --2- 172.21.15.86:0/294722536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbf80725d0 0x7fbbf80729e0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fbbf806f410 tx=0x7fbbf002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36f4ceaca88700ae server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:01.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.236+0000 7fbbf7400640 1 -- 172.21.15.86:0/294722536 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf002ca80 con 0x7fbbf80725d0 2024-04-20T21:23:01.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.236+0000 7fbbf7400640 1 -- 172.21.15.86:0/294722536 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbf002cc20 con 0x7fbbf80725d0 2024-04-20T21:23:01.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.236+0000 7fbbf7400640 1 -- 172.21.15.86:0/294722536 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf00343f0 con 0x7fbbf80725d0 2024-04-20T21:23:01.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.236+0000 7fbbfe200640 1 -- 172.21.15.86:0/294722536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbf80725d0 msgr2=0x7fbbf80729e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:01.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.236+0000 7fbbfe200640 1 --2- 172.21.15.86:0/294722536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbf80725d0 0x7fbbf80729e0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fbbf806f410 tx=0x7fbbf002bee0 comp rx=0 tx=0).stop 2024-04-20T21:23:01.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.236+0000 7fbbfe200640 1 -- 172.21.15.86:0/294722536 shutdown_connections 2024-04-20T21:23:01.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.236+0000 7fbbfe200640 1 --2- 172.21.15.86:0/294722536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbf80725d0 0x7fbbf80729e0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:01.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.236+0000 7fbbfe200640 1 --2- 172.21.15.86:0/294722536 >> v2:172.21.15.144:3300/0 conn(0x7fbbf806f480 0x7fbbf806e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:01.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.236+0000 7fbbfe200640 1 -- 172.21.15.86:0/294722536 >> 172.21.15.86:0/294722536 conn(0x7fbbf8067cf0 msgr2=0x7fbbf806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:01.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.236+0000 7fbbfe200640 1 -- 172.21.15.86:0/294722536 shutdown_connections 2024-04-20T21:23:01.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.236+0000 7fbbfe200640 1 -- 172.21.15.86:0/294722536 wait complete. 2024-04-20T21:23:01.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.238+0000 7fbbfe200640 1 Processor -- start 2024-04-20T21:23:01.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.238+0000 7fbbfe200640 1 -- start start 2024-04-20T21:23:01.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.239+0000 7fbbfe200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbf806f480 0x7fbbf8141b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:01.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.239+0000 7fbbfe200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbf80725d0 0x7fbbf8140660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:01.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.239+0000 7fbbfe200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbbf80b4b30 0x7fbbf8140ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:01.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.239+0000 7fbbfe200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf80cf270 con 0x7fbbf80725d0 2024-04-20T21:23:01.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.239+0000 7fbbfe200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf80cef90 con 0x7fbbf806f480 2024-04-20T21:23:01.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.239+0000 7fbbfe200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbbf806ecc0 con 0x7fbbf80b4b30 2024-04-20T21:23:01.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.239+0000 7fbbf7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbf80725d0 0x7fbbf8140660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:01.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.239+0000 7fbbf7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbf80725d0 0x7fbbf8140660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53174/0 (socket says 172.21.15.86:53174) 2024-04-20T21:23:01.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.239+0000 7fbbf7e00640 1 -- 172.21.15.86:0/2087288350 learned_addr learned my addr 172.21.15.86:0/2087288350 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:01.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.240+0000 7fbbfce00640 1 --2- 172.21.15.86:0/2087288350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbf806f480 0x7fbbf8141b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:01.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.240+0000 7fbbfd800640 1 --2- 172.21.15.86:0/2087288350 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbbf80b4b30 0x7fbbf8140ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:01.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.240+0000 7fbbf7e00640 1 -- 172.21.15.86:0/2087288350 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbbf80b4b30 msgr2=0x7fbbf8140ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:01.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.240+0000 7fbbf7e00640 1 --2- 172.21.15.86:0/2087288350 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbbf80b4b30 0x7fbbf8140ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:01.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.240+0000 7fbbf7e00640 1 -- 172.21.15.86:0/2087288350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbf806f480 msgr2=0x7fbbf8141b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:01.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.240+0000 7fbbf7e00640 1 --2- 172.21.15.86:0/2087288350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbf806f480 0x7fbbf8141b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:01.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.240+0000 7fbbf7e00640 1 -- 172.21.15.86:0/2087288350 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbf8151d30 con 0x7fbbf80725d0 2024-04-20T21:23:01.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.240+0000 7fbbf7e00640 1 --2- 172.21.15.86:0/2087288350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbf80725d0 0x7fbbf8140660 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fbbe400c820 tx=0x7fbbe400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:01.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.240+0000 7fbbf4c00640 1 -- 172.21.15.86:0/2087288350 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe4015070 con 0x7fbbf80725d0 2024-04-20T21:23:01.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.240+0000 7fbbf4c00640 1 -- 172.21.15.86:0/2087288350 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbe4004220 con 0x7fbbf80725d0 2024-04-20T21:23:01.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.240+0000 7fbbf4c00640 1 -- 172.21.15.86:0/2087288350 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe400ce10 con 0x7fbbf80725d0 2024-04-20T21:23:01.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.240+0000 7fbbfe200640 1 -- 172.21.15.86:0/2087288350 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbf8152010 con 0x7fbbf80725d0 2024-04-20T21:23:01.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.241+0000 7fbbfe200640 1 -- 172.21.15.86:0/2087288350 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbf81525d0 con 0x7fbbf80725d0 2024-04-20T21:23:01.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.242+0000 7fbbfe200640 1 -- 172.21.15.86:0/2087288350 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbf8152900 con 0x7fbbf80725d0 2024-04-20T21:23:01.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.243+0000 7fbbf4c00640 1 -- 172.21.15.86:0/2087288350 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbbe4007830 con 0x7fbbf80725d0 2024-04-20T21:23:01.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.244+0000 7fbbf4c00640 1 --2- 172.21.15.86:0/2087288350 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbbd806c8f0 0x7fbbd806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:01.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.244+0000 7fbbf4c00640 1 -- 172.21.15.86:0/2087288350 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7fbbe408be20 con 0x7fbbf80725d0 2024-04-20T21:23:01.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.244+0000 7fbbfce00640 1 --2- 172.21.15.86:0/2087288350 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbbd806c8f0 0x7fbbd806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:01.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.245+0000 7fbbfce00640 1 --2- 172.21.15.86:0/2087288350 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbbd806c8f0 0x7fbbd806edb0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fbbec006fd0 tx=0x7fbbec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:01.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.248+0000 7fbbf4c00640 1 -- 172.21.15.86:0/2087288350 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbbe405a1b0 con 0x7fbbf80725d0 2024-04-20T21:23:01.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.448+0000 7fbbfe200640 1 -- 172.21.15.86:0/2087288350 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbbf80afb90 con 0x7fbbf80725d0 2024-04-20T21:23:01.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.449+0000 7fbbf4c00640 1 -- 172.21.15.86:0/2087288350 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v71) ==== 104+0+11 (secure 0 0 0) 0x7fbbf80afb90 con 0x7fbbf80725d0 2024-04-20T21:23:01.448 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-20T21:23:01.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.452+0000 7fbbfe200640 1 -- 172.21.15.86:0/2087288350 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbbd806c8f0 msgr2=0x7fbbd806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:01.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.452+0000 7fbbfe200640 1 --2- 172.21.15.86:0/2087288350 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbbd806c8f0 0x7fbbd806edb0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fbbec006fd0 tx=0x7fbbec008040 comp rx=0 tx=0).stop 2024-04-20T21:23:01.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.452+0000 7fbbfe200640 1 -- 172.21.15.86:0/2087288350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbf80725d0 msgr2=0x7fbbf8140660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:01.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.452+0000 7fbbfe200640 1 --2- 172.21.15.86:0/2087288350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbf80725d0 0x7fbbf8140660 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fbbe400c820 tx=0x7fbbe400cce0 comp rx=0 tx=0).stop 2024-04-20T21:23:01.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.452+0000 7fbbfe200640 1 -- 172.21.15.86:0/2087288350 shutdown_connections 2024-04-20T21:23:01.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.452+0000 7fbbfe200640 1 --2- 172.21.15.86:0/2087288350 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbbd806c8f0 0x7fbbd806edb0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:01.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.452+0000 7fbbfe200640 1 --2- 172.21.15.86:0/2087288350 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbbf80b4b30 0x7fbbf8140ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:01.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.452+0000 7fbbfe200640 1 --2- 172.21.15.86:0/2087288350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbf80725d0 0x7fbbf8140660 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:01.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.452+0000 7fbbfe200640 1 --2- 172.21.15.86:0/2087288350 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbbf806f480 0x7fbbf8141b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:01.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.452+0000 7fbbfe200640 1 -- 172.21.15.86:0/2087288350 >> 172.21.15.86:0/2087288350 conn(0x7fbbf8067cf0 msgr2=0x7fbbf806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:01.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.453+0000 7fbbfe200640 1 -- 172.21.15.86:0/2087288350 shutdown_connections 2024-04-20T21:23:01.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.453+0000 7fbbfe200640 1 -- 172.21.15.86:0/2087288350 wait complete. 2024-04-20T21:23:01.471 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:23:01.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.677+0000 7f70ec400640 1 Processor -- start 2024-04-20T21:23:01.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.677+0000 7f70ec400640 1 -- start start 2024-04-20T21:23:01.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.678+0000 7f70ec400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f70e415efb0 0x7f70e40af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:01.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.678+0000 7f70ec400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70e40701e0 0x7f70e4168560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:01.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.678+0000 7f70ec400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70e40b0330 con 0x7f70e40afe20 2024-04-20T21:23:01.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.678+0000 7f70ec400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f70e406e6f0 con 0x7f70e415efb0 2024-04-20T21:23:01.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.678+0000 7f70ec400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f70e406e9d0 con 0x7f70e40701e0 2024-04-20T21:23:01.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.679+0000 7f70e9000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f70e40afe20 0x7f70e415f390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54888/0 (socket says 172.21.15.86:54888) 2024-04-20T21:23:01.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.679+0000 7f70e9000640 1 -- 172.21.15.86:0/453597665 learned_addr learned my addr 172.21.15.86:0/453597665 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:23:01.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.679+0000 7f70e3e00640 1 -- 172.21.15.86:0/453597665 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 682331938 0 0) 0x7f70e40b0330 con 0x7f70e40afe20 2024-04-20T21:23:01.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.679+0000 7f70e3e00640 1 -- 172.21.15.86:0/453597665 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f70c8003520 con 0x7f70e40afe20 2024-04-20T21:23:01.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.679+0000 7f70ea400640 1 --2- 172.21.15.86:0/453597665 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70e40701e0 0x7f70e4168560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:01.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.679+0000 7f70e9a00640 1 --2- 172.21.15.86:0/453597665 >> v2:172.21.15.144:3300/0 conn(0x7f70e415efb0 0x7f70e40af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:01.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.680+0000 7f70e3e00640 1 -- 172.21.15.86:0/453597665 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3504297648 0 0) 0x7f70c8003520 con 0x7f70e40afe20 2024-04-20T21:23:01.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.680+0000 7f70e3e00640 1 -- 172.21.15.86:0/453597665 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70e40701e0 msgr2=0x7f70e4168560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:01.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.680+0000 7f70e3e00640 1 --2- 172.21.15.86:0/453597665 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70e40701e0 0x7f70e4168560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:01.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.680+0000 7f70e3e00640 1 -- 172.21.15.86:0/453597665 >> v2:172.21.15.144:3300/0 conn(0x7f70e415efb0 msgr2=0x7f70e40af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:01.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.680+0000 7f70e3e00640 1 --2- 172.21.15.86:0/453597665 >> v2:172.21.15.144:3300/0 conn(0x7f70e415efb0 0x7f70e40af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:01.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.680+0000 7f70e3e00640 1 -- 172.21.15.86:0/453597665 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70e406ecb0 con 0x7f70e40afe20 2024-04-20T21:23:01.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.680+0000 7f70e3e00640 1 -- 172.21.15.86:0/453597665 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f70d0001670 con 0x7f70e40afe20 2024-04-20T21:23:01.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.680+0000 7f70e3e00640 1 -- 172.21.15.86:0/453597665 >> v1:172.21.15.86:6789/0 conn(0x7f70e40afe20 legacy=0x7f70e415f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:01.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.680+0000 7f70e3e00640 1 --2- 172.21.15.86:0/453597665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70c80039b0 0x7f70c8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:01.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.680+0000 7f70e3e00640 1 --2- 172.21.15.86:0/453597665 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70e40701e0 0x7f70c8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:01.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.680+0000 7f70e3e00640 1 --2- 172.21.15.86:0/453597665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70c8004d40 0x7f70c80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:01.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.680+0000 7f70e3e00640 1 -- 172.21.15.86:0/453597665 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f70e406e6f0 con 0x7f70c8004d40 2024-04-20T21:23:01.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.680+0000 7f70e3e00640 1 -- 172.21.15.86:0/453597665 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f70e406e9d0 con 0x7f70c80039b0 2024-04-20T21:23:01.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.680+0000 7f70e3e00640 1 -- 172.21.15.86:0/453597665 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f70c800ba70 con 0x7f70e40701e0 2024-04-20T21:23:01.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.680+0000 7f70ea400640 1 --2- 172.21.15.86:0/453597665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70c8004d40 0x7f70c80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:01.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.680+0000 7f70ea400640 1 -- 172.21.15.86:0/453597665 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70e40701e0 msgr2=0x7f70c8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:01.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.680+0000 7f70ea400640 1 --2- 172.21.15.86:0/453597665 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70e40701e0 0x7f70c8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:01.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.680+0000 7f70ea400640 1 -- 172.21.15.86:0/453597665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70c80039b0 msgr2=0x7f70c8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:01.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.680+0000 7f70ea400640 1 --2- 172.21.15.86:0/453597665 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70c80039b0 0x7f70c8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:01.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.680+0000 7f70ea400640 1 -- 172.21.15.86:0/453597665 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c8003520 con 0x7f70c8004d40 2024-04-20T21:23:01.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.681+0000 7f70ea400640 1 --2- 172.21.15.86:0/453597665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70c8004d40 0x7f70c80051c0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f70dc007330 tx=0x7f70c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d34f44c3251d129b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:01.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.682+0000 7f70e3e00640 1 -- 172.21.15.86:0/453597665 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70dc039070 con 0x7f70c8004d40 2024-04-20T21:23:01.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.682+0000 7f70e3e00640 1 -- 172.21.15.86:0/453597665 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70dc02cc80 con 0x7f70c8004d40 2024-04-20T21:23:01.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.682+0000 7f70e3e00640 1 -- 172.21.15.86:0/453597665 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70dc009be0 con 0x7f70c8004d40 2024-04-20T21:23:01.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.682+0000 7f70ec400640 1 -- 172.21.15.86:0/453597665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70c8004d40 msgr2=0x7f70c80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:01.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.682+0000 7f70ec400640 1 --2- 172.21.15.86:0/453597665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70c8004d40 0x7f70c80051c0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f70dc007330 tx=0x7f70c8005830 comp rx=0 tx=0).stop 2024-04-20T21:23:01.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.682+0000 7f70ea400640 1 -- 172.21.15.86:0/453597665 reap_dead start 2024-04-20T21:23:01.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.682+0000 7f70ec400640 1 -- 172.21.15.86:0/453597665 shutdown_connections 2024-04-20T21:23:01.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.682+0000 7f70ec400640 1 -- 172.21.15.86:0/453597665 >> 172.21.15.86:0/453597665 conn(0x7f70e4073560 msgr2=0x7f70e406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:01.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.683+0000 7f70ec400640 1 -- 172.21.15.86:0/453597665 shutdown_connections 2024-04-20T21:23:01.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.683+0000 7f70ec400640 1 -- 172.21.15.86:0/453597665 wait complete. 2024-04-20T21:23:01.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.684+0000 7f70ec400640 1 Processor -- start 2024-04-20T21:23:01.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.684+0000 7f70ec400640 1 -- start start 2024-04-20T21:23:01.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.685+0000 7f70ec400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70c8004d40 0x7f70e41507d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:01.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.685+0000 7f70ec400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70e41f51c0 0x7f70e40b1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:01.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.685+0000 7f70ec400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70e41f5640 0x7f70e40b1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:01.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.685+0000 7f70ec400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f70e4067a60 con 0x7f70e41f5640 2024-04-20T21:23:01.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.685+0000 7f70ec400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f70e40678e0 con 0x7f70e41f51c0 2024-04-20T21:23:01.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.685+0000 7f70ec400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f70e415faa0 con 0x7f70c8004d40 2024-04-20T21:23:01.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.686+0000 7f70ea400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70e41f5640 0x7f70e40b1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:01.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.686+0000 7f70ea400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70e41f5640 0x7f70e40b1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53194/0 (socket says 172.21.15.86:53194) 2024-04-20T21:23:01.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.686+0000 7f70ea400640 1 -- 172.21.15.86:0/543713922 learned_addr learned my addr 172.21.15.86:0/543713922 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:01.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.686+0000 7f70ea400640 1 -- 172.21.15.86:0/543713922 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70c8004d40 msgr2=0x7f70e41507d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:01.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.686+0000 7f70e9a00640 1 --2- 172.21.15.86:0/543713922 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70c8004d40 0x7f70e41507d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:01.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.686+0000 7f70ea400640 1 --2- 172.21.15.86:0/543713922 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70c8004d40 0x7f70e41507d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:01.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.686+0000 7f70ea400640 1 -- 172.21.15.86:0/543713922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70e41f51c0 msgr2=0x7f70e40b1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:01.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.686+0000 7f70ea400640 1 --2- 172.21.15.86:0/543713922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70e41f51c0 0x7f70e40b1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:01.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.686+0000 7f70ea400640 1 -- 172.21.15.86:0/543713922 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70e415e200 con 0x7f70e41f5640 2024-04-20T21:23:01.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.686+0000 7f70ea400640 1 --2- 172.21.15.86:0/543713922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70e41f5640 0x7f70e40b1eb0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f70dc007450 tx=0x7f70dc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:01.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.687+0000 7f70e1600640 1 -- 172.21.15.86:0/543713922 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70dc039070 con 0x7f70e41f5640 2024-04-20T21:23:01.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.687+0000 7f70e1600640 1 -- 172.21.15.86:0/543713922 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70dc044070 con 0x7f70e41f5640 2024-04-20T21:23:01.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.687+0000 7f70e1600640 1 -- 172.21.15.86:0/543713922 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70dc009be0 con 0x7f70e41f5640 2024-04-20T21:23:01.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.687+0000 7f70ec400640 1 -- 172.21.15.86:0/543713922 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70e415dc40 con 0x7f70e41f5640 2024-04-20T21:23:01.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.687+0000 7f70ec400640 1 -- 172.21.15.86:0/543713922 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70e40b39a0 con 0x7f70e41f5640 2024-04-20T21:23:01.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.688+0000 7f70ec400640 1 -- 172.21.15.86:0/543713922 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70e41f4390 con 0x7f70e41f5640 2024-04-20T21:23:01.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.689+0000 7f70e1600640 1 -- 172.21.15.86:0/543713922 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f70dc03e810 con 0x7f70e41f5640 2024-04-20T21:23:01.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.690+0000 7f70e1600640 1 --2- 172.21.15.86:0/543713922 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f70c406c8f0 0x7f70c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:01.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.690+0000 7f70e1600640 1 -- 172.21.15.86:0/543713922 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f70dc0ada50 con 0x7f70e41f5640 2024-04-20T21:23:01.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.691+0000 7f70e9a00640 1 --2- 172.21.15.86:0/543713922 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f70c406c8f0 0x7f70c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:01.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.691+0000 7f70e9a00640 1 --2- 172.21.15.86:0/543713922 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f70c406c8f0 0x7f70c406edb0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f70d4006a90 tx=0x7f70d4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:01.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.695+0000 7f70e1600640 1 -- 172.21.15.86:0/543713922 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f70dc07bdd0 con 0x7f70e41f5640 2024-04-20T21:23:01.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.893+0000 7f70ec400640 1 -- 172.21.15.86:0/543713922 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f70e40690f0 con 0x7f70e41f5640 2024-04-20T21:23:01.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.894+0000 7f70e1600640 1 -- 172.21.15.86:0/543713922 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v71) ==== 100+0+11 (secure 0 0 0) 0x7f70dc07b6d0 con 0x7f70e41f5640 2024-04-20T21:23:01.893 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-20T21:23:01.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.896+0000 7f70ec400640 1 -- 172.21.15.86:0/543713922 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f70c406c8f0 msgr2=0x7f70c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:01.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.896+0000 7f70ec400640 1 --2- 172.21.15.86:0/543713922 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f70c406c8f0 0x7f70c406edb0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f70d4006a90 tx=0x7f70d4008040 comp rx=0 tx=0).stop 2024-04-20T21:23:01.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.896+0000 7f70ec400640 1 -- 172.21.15.86:0/543713922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70e41f5640 msgr2=0x7f70e40b1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:01.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.896+0000 7f70ec400640 1 --2- 172.21.15.86:0/543713922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70e41f5640 0x7f70e40b1eb0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f70dc007450 tx=0x7f70dc004ab0 comp rx=0 tx=0).stop 2024-04-20T21:23:01.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.896+0000 7f70ec400640 1 -- 172.21.15.86:0/543713922 shutdown_connections 2024-04-20T21:23:01.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.896+0000 7f70ec400640 1 --2- 172.21.15.86:0/543713922 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f70c406c8f0 0x7f70c406edb0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:01.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.896+0000 7f70ec400640 1 --2- 172.21.15.86:0/543713922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70e41f5640 0x7f70e40b1eb0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:01.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.896+0000 7f70ec400640 1 --2- 172.21.15.86:0/543713922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70e41f51c0 0x7f70e40b1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:01.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.896+0000 7f70ec400640 1 --2- 172.21.15.86:0/543713922 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70c8004d40 0x7f70e41507d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:01.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.896+0000 7f70ec400640 1 -- 172.21.15.86:0/543713922 >> 172.21.15.86:0/543713922 conn(0x7f70e4073560 msgr2=0x7f70e415aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:01.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.896+0000 7f70ec400640 1 -- 172.21.15.86:0/543713922 shutdown_connections 2024-04-20T21:23:01.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:01.896+0000 7f70ec400640 1 -- 172.21.15.86:0/543713922 wait complete. 2024-04-20T21:23:01.914 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T21:23:02.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.113+0000 7fc00a800640 1 Processor -- start 2024-04-20T21:23:02.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.113+0000 7fc00a800640 1 -- start start 2024-04-20T21:23:02.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.115+0000 7fc00a800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc00415b5f0 0x7fc0040b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:02.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.115+0000 7fc00a800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc00406f070 0x7fc0040b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:02.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.115+0000 7fc00a800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc00415b9d0 con 0x7fc0040cff80 2024-04-20T21:23:02.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.115+0000 7fc00a800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc0040b4650 con 0x7fc00415b5f0 2024-04-20T21:23:02.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.115+0000 7fc00a800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc0040af850 con 0x7fc00406f070 2024-04-20T21:23:02.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.115+0000 7fc003400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc00415b5f0 0x7fc0040b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:02.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.115+0000 7fc003e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fc0040cff80 0x7fc0040ce130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54900/0 (socket says 172.21.15.86:54900) 2024-04-20T21:23:02.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.115+0000 7fc003e00640 1 -- 172.21.15.86:0/2919192025 learned_addr learned my addr 172.21.15.86:0/2919192025 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:23:02.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.115+0000 7fbffbe00640 1 --2- 172.21.15.86:0/2919192025 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc00406f070 0x7fc0040b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:02.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.115+0000 7fc003400640 1 -- 172.21.15.86:0/2919192025 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc00406f070 msgr2=0x7fc0040b50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:02.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.116+0000 7fc003400640 1 --2- 172.21.15.86:0/2919192025 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc00406f070 0x7fc0040b50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:02.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.116+0000 7fc003400640 1 -- 172.21.15.86:0/2919192025 >> v1:172.21.15.86:6789/0 conn(0x7fc0040cff80 legacy=0x7fc0040ce130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:02.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.116+0000 7fc003400640 1 -- 172.21.15.86:0/2919192025 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0040afb30 con 0x7fc00415b5f0 2024-04-20T21:23:02.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.116+0000 7fc002a00640 1 -- 172.21.15.86:0/2919192025 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3799498638 0 0) 0x7fc00415b9d0 con 0x7fc0040cff80 2024-04-20T21:23:02.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.117+0000 7fc003400640 1 --2- 172.21.15.86:0/2919192025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc00415b5f0 0x7fc0040b3d40 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fbff40097a0 tx=0x7fbff402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f21d2d12ea83496 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:02.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.117+0000 7fc002a00640 1 -- 172.21.15.86:0/2919192025 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff4035020 con 0x7fc00415b5f0 2024-04-20T21:23:02.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.117+0000 7fc002a00640 1 -- 172.21.15.86:0/2919192025 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbff4031830 con 0x7fc00415b5f0 2024-04-20T21:23:02.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.117+0000 7fc002a00640 1 -- 172.21.15.86:0/2919192025 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff4031c00 con 0x7fc00415b5f0 2024-04-20T21:23:02.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.117+0000 7fc00a800640 1 -- 172.21.15.86:0/2919192025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc00415b5f0 msgr2=0x7fc0040b3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:02.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.117+0000 7fc00a800640 1 --2- 172.21.15.86:0/2919192025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc00415b5f0 0x7fc0040b3d40 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fbff40097a0 tx=0x7fbff402b8d0 comp rx=0 tx=0).stop 2024-04-20T21:23:02.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.118+0000 7fc00a800640 1 -- 172.21.15.86:0/2919192025 shutdown_connections 2024-04-20T21:23:02.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.118+0000 7fc00a800640 1 --2- 172.21.15.86:0/2919192025 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc00406f070 0x7fc0040b50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:02.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.118+0000 7fc00a800640 1 --2- 172.21.15.86:0/2919192025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc00415b5f0 0x7fc0040b3d40 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:02.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.118+0000 7fc00a800640 1 -- 172.21.15.86:0/2919192025 >> 172.21.15.86:0/2919192025 conn(0x7fc00406b980 msgr2=0x7fc00406bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:02.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.118+0000 7fc00a800640 1 -- 172.21.15.86:0/2919192025 shutdown_connections 2024-04-20T21:23:02.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.118+0000 7fc00a800640 1 -- 172.21.15.86:0/2919192025 wait complete. 2024-04-20T21:23:02.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.119+0000 7fc00a800640 1 Processor -- start 2024-04-20T21:23:02.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.119+0000 7fc00a800640 1 -- start start 2024-04-20T21:23:02.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.120+0000 7fc00a800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc00406f070 0x7fc004152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:02.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.120+0000 7fc00a800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0040cff80 0x7fc004153310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:02.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.120+0000 7fc00a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc00415b5f0 0x7fc00414f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:02.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.120+0000 7fc00a800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc0040b55e0 con 0x7fc00415b5f0 2024-04-20T21:23:02.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.120+0000 7fc00a800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc0040b4b80 con 0x7fc0040cff80 2024-04-20T21:23:02.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.120+0000 7fc00a800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc0040d0410 con 0x7fc00406f070 2024-04-20T21:23:02.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.121+0000 7fc003e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc00415b5f0 0x7fc00414f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:02.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.121+0000 7fc003e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc00415b5f0 0x7fc00414f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53214/0 (socket says 172.21.15.86:53214) 2024-04-20T21:23:02.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.121+0000 7fc003e00640 1 -- 172.21.15.86:0/3290015012 learned_addr learned my addr 172.21.15.86:0/3290015012 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:02.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.121+0000 7fc003400640 1 --2- 172.21.15.86:0/3290015012 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc00406f070 0x7fc004152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:02.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.121+0000 7fc003e00640 1 -- 172.21.15.86:0/3290015012 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc00406f070 msgr2=0x7fc004152dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:02.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.121+0000 7fc003e00640 1 --2- 172.21.15.86:0/3290015012 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc00406f070 0x7fc004152dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:02.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.121+0000 7fc003e00640 1 -- 172.21.15.86:0/3290015012 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0040cff80 msgr2=0x7fc004153310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:02.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.121+0000 7fc003e00640 1 --2- 172.21.15.86:0/3290015012 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0040cff80 0x7fc004153310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:02.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.121+0000 7fc003e00640 1 -- 172.21.15.86:0/3290015012 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc00414dd10 con 0x7fc00415b5f0 2024-04-20T21:23:02.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.121+0000 7fc003e00640 1 --2- 172.21.15.86:0/3290015012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc00415b5f0 0x7fc00414f8e0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fbffc00c7a0 tx=0x7fbffc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:02.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.122+0000 7fbffb400640 1 -- 172.21.15.86:0/3290015012 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc018070 con 0x7fc00415b5f0 2024-04-20T21:23:02.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.122+0000 7fbffb400640 1 -- 172.21.15.86:0/3290015012 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbffc00a640 con 0x7fc00415b5f0 2024-04-20T21:23:02.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.122+0000 7fbffb400640 1 -- 172.21.15.86:0/3290015012 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc00aab0 con 0x7fc00415b5f0 2024-04-20T21:23:02.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.122+0000 7fc00a800640 1 -- 172.21.15.86:0/3290015012 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc00414dff0 con 0x7fc00415b5f0 2024-04-20T21:23:02.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.122+0000 7fc00a800640 1 -- 172.21.15.86:0/3290015012 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc00414f0e0 con 0x7fc00415b5f0 2024-04-20T21:23:02.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.123+0000 7fc00a800640 1 -- 172.21.15.86:0/3290015012 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0040689e0 con 0x7fc00415b5f0 2024-04-20T21:23:02.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.124+0000 7fbffb400640 1 -- 172.21.15.86:0/3290015012 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbffc01b070 con 0x7fc00415b5f0 2024-04-20T21:23:02.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.125+0000 7fbffb400640 1 --2- 172.21.15.86:0/3290015012 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbfe006c8f0 0x7fbfe006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:02.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.125+0000 7fbffb400640 1 -- 172.21.15.86:0/3290015012 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7fbffc08b220 con 0x7fc00415b5f0 2024-04-20T21:23:02.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.125+0000 7fc003400640 1 --2- 172.21.15.86:0/3290015012 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbfe006c8f0 0x7fbfe006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:02.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.126+0000 7fc003400640 1 --2- 172.21.15.86:0/3290015012 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbfe006c8f0 0x7fbfe006edb0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fbff4036000 tx=0x7fbff403c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:02.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.129+0000 7fbffb400640 1 -- 172.21.15.86:0/3290015012 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbffc0595b0 con 0x7fc00415b5f0 2024-04-20T21:23:02.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.341+0000 7fc00a800640 1 -- 172.21.15.86:0/3290015012 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc00415ad40 con 0x7fc00415b5f0 2024-04-20T21:23:02.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.342+0000 7fbffb400640 1 -- 172.21.15.86:0/3290015012 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v71) ==== 80+0+0 (secure 0 0 0) 0x7fbffc058eb0 con 0x7fc00415b5f0 2024-04-20T21:23:02.340 INFO:teuthology.orchestra.run.smithi086.stderr:listed 0 entries 2024-04-20T21:23:02.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.344+0000 7fc00a800640 1 -- 172.21.15.86:0/3290015012 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbfe006c8f0 msgr2=0x7fbfe006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:02.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.344+0000 7fc00a800640 1 --2- 172.21.15.86:0/3290015012 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbfe006c8f0 0x7fbfe006edb0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fbff4036000 tx=0x7fbff403c040 comp rx=0 tx=0).stop 2024-04-20T21:23:02.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.344+0000 7fc00a800640 1 -- 172.21.15.86:0/3290015012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc00415b5f0 msgr2=0x7fc00414f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:02.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.344+0000 7fc00a800640 1 --2- 172.21.15.86:0/3290015012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc00415b5f0 0x7fc00414f8e0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fbffc00c7a0 tx=0x7fbffc00cc60 comp rx=0 tx=0).stop 2024-04-20T21:23:02.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.344+0000 7fc00a800640 1 -- 172.21.15.86:0/3290015012 shutdown_connections 2024-04-20T21:23:02.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.344+0000 7fc00a800640 1 --2- 172.21.15.86:0/3290015012 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbfe006c8f0 0x7fbfe006edb0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:02.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.344+0000 7fc00a800640 1 --2- 172.21.15.86:0/3290015012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc00415b5f0 0x7fc00414f8e0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:02.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.344+0000 7fc00a800640 1 --2- 172.21.15.86:0/3290015012 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0040cff80 0x7fc004153310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:02.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.344+0000 7fc00a800640 1 --2- 172.21.15.86:0/3290015012 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc00406f070 0x7fc004152dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:02.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.344+0000 7fc00a800640 1 -- 172.21.15.86:0/3290015012 >> 172.21.15.86:0/3290015012 conn(0x7fc00406b980 msgr2=0x7fc0040b00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:02.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.344+0000 7fc00a800640 1 -- 172.21.15.86:0/3290015012 shutdown_connections 2024-04-20T21:23:02.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.344+0000 7fc00a800640 1 -- 172.21.15.86:0/3290015012 wait complete. 2024-04-20T21:23:02.360 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:23:02.360 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:23:02.409 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:23:02.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.652+0000 7fa7cfa00640 1 Processor -- start 2024-04-20T21:23:02.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.652+0000 7fa7cfa00640 1 -- start start 2024-04-20T21:23:02.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.653+0000 7fa7cfa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa7c8144e90 0x7fa7c815ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:02.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.654+0000 7fa7cfa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7c8071d00 0x7fa7c8072190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:02.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.654+0000 7fa7cfa00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7c806dc30 con 0x7fa7c806f140 2024-04-20T21:23:02.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.654+0000 7fa7cfa00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa7c8145270 con 0x7fa7c8144e90 2024-04-20T21:23:02.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.654+0000 7fa7cfa00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa7c815f720 con 0x7fa7c8071d00 2024-04-20T21:23:02.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.654+0000 7fa7cd000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa7c8144e90 0x7fa7c815ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:02.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.654+0000 7fa7cd000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa7c8144e90 0x7fa7c815ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53218/0 (socket says 172.21.15.86:53218) 2024-04-20T21:23:02.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.654+0000 7fa7bfe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fa7c806f140 0x7fa7c80aeb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54914/0 (socket says 172.21.15.86:54914) 2024-04-20T21:23:02.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.654+0000 7fa7cd000640 1 -- 172.21.15.86:0/2089540956 learned_addr learned my addr 172.21.15.86:0/2089540956 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:02.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.654+0000 7fa7cd000640 1 -- 172.21.15.86:0/2089540956 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7c8071d00 msgr2=0x7fa7c8072190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:02.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.654+0000 7fa7cd000640 1 --2- 172.21.15.86:0/2089540956 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7c8071d00 0x7fa7c8072190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:02.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.654+0000 7fa7cd000640 1 -- 172.21.15.86:0/2089540956 >> v1:172.21.15.86:6789/0 conn(0x7fa7c806f140 legacy=0x7fa7c80aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:02.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.654+0000 7fa7cd000640 1 -- 172.21.15.86:0/2089540956 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7c815fa00 con 0x7fa7c8144e90 2024-04-20T21:23:02.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.655+0000 7fa7cd000640 1 --2- 172.21.15.86:0/2089540956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7c8144e90 0x7fa7c815ee10 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fa7c81453f0 tx=0x7fa7b002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=191d9452657372c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:02.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.656+0000 7fa7c7e00640 1 -- 172.21.15.86:0/2089540956 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0034070 con 0x7fa7c8144e90 2024-04-20T21:23:02.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.656+0000 7fa7c7e00640 1 -- 172.21.15.86:0/2089540956 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7b0030840 con 0x7fa7c8144e90 2024-04-20T21:23:02.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.656+0000 7fa7c7e00640 1 -- 172.21.15.86:0/2089540956 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b0030c10 con 0x7fa7c8144e90 2024-04-20T21:23:02.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.656+0000 7fa7cfa00640 1 -- 172.21.15.86:0/2089540956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7c8144e90 msgr2=0x7fa7c815ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:02.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.656+0000 7fa7cfa00640 1 --2- 172.21.15.86:0/2089540956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7c8144e90 0x7fa7c815ee10 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fa7c81453f0 tx=0x7fa7b002b980 comp rx=0 tx=0).stop 2024-04-20T21:23:02.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.656+0000 7fa7cfa00640 1 -- 172.21.15.86:0/2089540956 shutdown_connections 2024-04-20T21:23:02.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.656+0000 7fa7cfa00640 1 --2- 172.21.15.86:0/2089540956 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7c8071d00 0x7fa7c8072190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:02.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.656+0000 7fa7cfa00640 1 --2- 172.21.15.86:0/2089540956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7c8144e90 0x7fa7c815ee10 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:02.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.657+0000 7fa7cfa00640 1 -- 172.21.15.86:0/2089540956 >> 172.21.15.86:0/2089540956 conn(0x7fa7c80b2360 msgr2=0x7fa7c80b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:02.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.657+0000 7fa7cfa00640 1 -- 172.21.15.86:0/2089540956 shutdown_connections 2024-04-20T21:23:02.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.657+0000 7fa7cfa00640 1 -- 172.21.15.86:0/2089540956 wait complete. 2024-04-20T21:23:02.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.659+0000 7fa7cfa00640 1 Processor -- start 2024-04-20T21:23:02.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.659+0000 7fa7cfa00640 1 -- start start 2024-04-20T21:23:02.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.660+0000 7fa7cfa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7c806f140 0x7fa7c8140d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:02.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.660+0000 7fa7cfa00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7c8071d00 0x7fa7c80ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:02.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.660+0000 7fa7cfa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7c8144e90 0x7fa7c80aec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:02.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.660+0000 7fa7cfa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c815f350 con 0x7fa7c8144e90 2024-04-20T21:23:02.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.660+0000 7fa7cfa00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c806ded0 con 0x7fa7c8071d00 2024-04-20T21:23:02.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.660+0000 7fa7cfa00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa7c815f4d0 con 0x7fa7c806f140 2024-04-20T21:23:02.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.660+0000 7fa7cda00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7c8144e90 0x7fa7c80aec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:02.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.660+0000 7fa7cda00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7c8144e90 0x7fa7c80aec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53224/0 (socket says 172.21.15.86:53224) 2024-04-20T21:23:02.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.660+0000 7fa7cda00640 1 -- 172.21.15.86:0/4217696362 learned_addr learned my addr 172.21.15.86:0/4217696362 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:02.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.661+0000 7fa7cd000640 1 --2- 172.21.15.86:0/4217696362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7c806f140 0x7fa7c8140d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:02.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.661+0000 7fa7bfe00640 1 --2- 172.21.15.86:0/4217696362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7c8071d00 0x7fa7c80ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:02.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.661+0000 7fa7cda00640 1 -- 172.21.15.86:0/4217696362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7c806f140 msgr2=0x7fa7c8140d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:02.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.661+0000 7fa7cda00640 1 --2- 172.21.15.86:0/4217696362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7c806f140 0x7fa7c8140d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:02.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.661+0000 7fa7cda00640 1 -- 172.21.15.86:0/4217696362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7c8071d00 msgr2=0x7fa7c80ceb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:02.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.661+0000 7fa7cda00640 1 --2- 172.21.15.86:0/4217696362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7c8071d00 0x7fa7c80ceb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:02.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.661+0000 7fa7cda00640 1 -- 172.21.15.86:0/4217696362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7c8132410 con 0x7fa7c8144e90 2024-04-20T21:23:02.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.661+0000 7fa7cda00640 1 --2- 172.21.15.86:0/4217696362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7c8144e90 0x7fa7c80aec50 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fa7c000b970 tx=0x7fa7c000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:02.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.661+0000 7fa7c5600640 1 -- 172.21.15.86:0/4217696362 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0019070 con 0x7fa7c8144e90 2024-04-20T21:23:02.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.661+0000 7fa7c5600640 1 -- 172.21.15.86:0/4217696362 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7c000cb60 con 0x7fa7c8144e90 2024-04-20T21:23:02.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.661+0000 7fa7c5600640 1 -- 172.21.15.86:0/4217696362 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0014440 con 0x7fa7c8144e90 2024-04-20T21:23:02.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.662+0000 7fa7cfa00640 1 -- 172.21.15.86:0/4217696362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7c81326f0 con 0x7fa7c8144e90 2024-04-20T21:23:02.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.662+0000 7fa7cfa00640 1 -- 172.21.15.86:0/4217696362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7c8132cb0 con 0x7fa7c8144e90 2024-04-20T21:23:02.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.663+0000 7fa7cfa00640 1 -- 172.21.15.86:0/4217696362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7c80689e0 con 0x7fa7c8144e90 2024-04-20T21:23:02.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.664+0000 7fa7c5600640 1 -- 172.21.15.86:0/4217696362 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa7c001c070 con 0x7fa7c8144e90 2024-04-20T21:23:02.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.665+0000 7fa7c5600640 1 --2- 172.21.15.86:0/4217696362 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa7a406c8f0 0x7fa7a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:02.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.665+0000 7fa7c5600640 1 -- 172.21.15.86:0/4217696362 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7fa7c008b920 con 0x7fa7c8144e90 2024-04-20T21:23:02.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.665+0000 7fa7cd000640 1 --2- 172.21.15.86:0/4217696362 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa7a406c8f0 0x7fa7a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:02.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.666+0000 7fa7cd000640 1 --2- 172.21.15.86:0/4217696362 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa7a406c8f0 0x7fa7a406edb0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0036000 tx=0x7fa7b003b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:02.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.670+0000 7fa7c5600640 1 -- 172.21.15.86:0/4217696362 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa7c0059cb0 con 0x7fa7c8144e90 2024-04-20T21:23:02.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.871+0000 7fa7cfa00640 1 -- 172.21.15.86:0/4217696362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa7c81412c0 con 0x7fa7c8144e90 2024-04-20T21:23:02.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.873+0000 7fa7c5600640 1 -- 172.21.15.86:0/4217696362 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+16668 (secure 0 0 0) 0x7fa7c00595b0 con 0x7fa7c8144e90 2024-04-20T21:23:02.872 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:23:02.873 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":71,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:22:46.983093+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:22:43.125767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-20T21:22:44.410549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:23:02.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.876+0000 7fa7cfa00640 1 -- 172.21.15.86:0/4217696362 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa7a406c8f0 msgr2=0x7fa7a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:02.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.876+0000 7fa7cfa00640 1 --2- 172.21.15.86:0/4217696362 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa7a406c8f0 0x7fa7a406edb0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0036000 tx=0x7fa7b003b040 comp rx=0 tx=0).stop 2024-04-20T21:23:02.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.876+0000 7fa7cfa00640 1 -- 172.21.15.86:0/4217696362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7c8144e90 msgr2=0x7fa7c80aec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:02.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.876+0000 7fa7cfa00640 1 --2- 172.21.15.86:0/4217696362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7c8144e90 0x7fa7c80aec50 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fa7c000b970 tx=0x7fa7c000be30 comp rx=0 tx=0).stop 2024-04-20T21:23:02.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.876+0000 7fa7cfa00640 1 -- 172.21.15.86:0/4217696362 shutdown_connections 2024-04-20T21:23:02.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.876+0000 7fa7cfa00640 1 --2- 172.21.15.86:0/4217696362 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa7a406c8f0 0x7fa7a406edb0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:02.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.876+0000 7fa7cfa00640 1 --2- 172.21.15.86:0/4217696362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7c8144e90 0x7fa7c80aec50 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:02.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.876+0000 7fa7cfa00640 1 --2- 172.21.15.86:0/4217696362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7c8071d00 0x7fa7c80ceb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:02.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.876+0000 7fa7cfa00640 1 --2- 172.21.15.86:0/4217696362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7c806f140 0x7fa7c8140d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:02.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.876+0000 7fa7cfa00640 1 -- 172.21.15.86:0/4217696362 >> 172.21.15.86:0/4217696362 conn(0x7fa7c80b2360 msgr2=0x7fa7c81608d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:02.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.876+0000 7fa7cfa00640 1 -- 172.21.15.86:0/4217696362 shutdown_connections 2024-04-20T21:23:02.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:02.876+0000 7fa7cfa00640 1 -- 172.21.15.86:0/4217696362 wait complete. 2024-04-20T21:23:02.893 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:22:43.125767+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:22:44.410549+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:23:02.894 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:23:03.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.099+0000 7f4ed9400640 1 Processor -- start 2024-04-20T21:23:03.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.100+0000 7f4ed9400640 1 -- start start 2024-04-20T21:23:03.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.101+0000 7f4ed9400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4ed415e8f0 0x7f4ed4162640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:03.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.101+0000 7f4ed9400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4ed415ecd0 0x7f4ed4162b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:03.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.101+0000 7f4ed9400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ed40cf680 0x7f4ed40cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:03.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.101+0000 7f4ed9400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f4ed40d2fd0 con 0x7f4ed415ecd0 2024-04-20T21:23:03.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.101+0000 7f4ed9400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f4ed40cffe0 con 0x7f4ed415e8f0 2024-04-20T21:23:03.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.101+0000 7f4ed9400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f4ed40d29a0 con 0x7f4ed40cf680 2024-04-20T21:23:03.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.101+0000 7f4ed1e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4ed415ecd0 0x7f4ed4162b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:03.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.101+0000 7f4ed1e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4ed415ecd0 0x7f4ed4162b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53236/0 (socket says 172.21.15.86:53236) 2024-04-20T21:23:03.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.101+0000 7f4ed1e00640 1 -- 172.21.15.86:0/569503189 learned_addr learned my addr 172.21.15.86:0/569503189 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:03.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.102+0000 7f4ed3200640 1 --2- 172.21.15.86:0/569503189 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ed40cf680 0x7f4ed40cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:03.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.102+0000 7f4ed2800640 1 --2- 172.21.15.86:0/569503189 >> v2:172.21.15.144:3300/0 conn(0x7f4ed415e8f0 0x7f4ed4162640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:03.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.102+0000 7f4ed1e00640 1 -- 172.21.15.86:0/569503189 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ed40cf680 msgr2=0x7f4ed40cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:03.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.102+0000 7f4ed1e00640 1 --2- 172.21.15.86:0/569503189 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ed40cf680 0x7f4ed40cfaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:03.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.102+0000 7f4ed1e00640 1 -- 172.21.15.86:0/569503189 >> v2:172.21.15.144:3300/0 conn(0x7f4ed415e8f0 msgr2=0x7f4ed4162640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:03.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.102+0000 7f4ed1e00640 1 --2- 172.21.15.86:0/569503189 >> v2:172.21.15.144:3300/0 conn(0x7f4ed415e8f0 0x7f4ed4162640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:03.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.102+0000 7f4ed1e00640 1 -- 172.21.15.86:0/569503189 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ed41631b0 con 0x7f4ed415ecd0 2024-04-20T21:23:03.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.102+0000 7f4ed2800640 1 --2- 172.21.15.86:0/569503189 >> v2:172.21.15.144:3300/0 conn(0x7f4ed415e8f0 0x7f4ed4162640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:23:03.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.103+0000 7f4ed1e00640 1 --2- 172.21.15.86:0/569503189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ed415ecd0 0x7f4ed4162b80 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f4ed408d180 tx=0x7f4ebc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48c0680e7cded0ea server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:03.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.103+0000 7f4ed1400640 1 -- 172.21.15.86:0/569503189 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc035020 con 0x7f4ed415ecd0 2024-04-20T21:23:03.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.103+0000 7f4ed1400640 1 -- 172.21.15.86:0/569503189 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ebc031870 con 0x7f4ed415ecd0 2024-04-20T21:23:03.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.103+0000 7f4ed1400640 1 -- 172.21.15.86:0/569503189 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc031c40 con 0x7f4ed415ecd0 2024-04-20T21:23:03.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.104+0000 7f4ed9400640 1 -- 172.21.15.86:0/569503189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ed415ecd0 msgr2=0x7f4ed4162b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:03.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.104+0000 7f4ed9400640 1 --2- 172.21.15.86:0/569503189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ed415ecd0 0x7f4ed4162b80 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f4ed408d180 tx=0x7f4ebc02b910 comp rx=0 tx=0).stop 2024-04-20T21:23:03.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.104+0000 7f4ed9400640 1 -- 172.21.15.86:0/569503189 shutdown_connections 2024-04-20T21:23:03.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.104+0000 7f4ed9400640 1 --2- 172.21.15.86:0/569503189 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ed40cf680 0x7f4ed40cfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:03.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.104+0000 7f4ed9400640 1 --2- 172.21.15.86:0/569503189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ed415ecd0 0x7f4ed4162b80 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:03.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.104+0000 7f4ed9400640 1 --2- 172.21.15.86:0/569503189 >> v2:172.21.15.144:3300/0 conn(0x7f4ed415e8f0 0x7f4ed4162640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:03.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.104+0000 7f4ed9400640 1 -- 172.21.15.86:0/569503189 >> 172.21.15.86:0/569503189 conn(0x7f4ed406cc70 msgr2=0x7f4ed406d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:03.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.104+0000 7f4ed9400640 1 -- 172.21.15.86:0/569503189 shutdown_connections 2024-04-20T21:23:03.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.104+0000 7f4ed9400640 1 -- 172.21.15.86:0/569503189 wait complete. 2024-04-20T21:23:03.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.106+0000 7f4ed9400640 1 Processor -- start 2024-04-20T21:23:03.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.106+0000 7f4ed9400640 1 -- start start 2024-04-20T21:23:03.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.107+0000 7f4ed9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ed40cf680 0x7f4ed40cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:03.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.107+0000 7f4ed9400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ed415e8f0 0x7f4ed40ca300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:03.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.107+0000 7f4ed9400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ed415ecd0 0x7f4ed4159390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:03.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.107+0000 7f4ed9400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed40b7a50 con 0x7f4ed40cf680 2024-04-20T21:23:03.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.107+0000 7f4ed9400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed40d2c60 con 0x7f4ed415e8f0 2024-04-20T21:23:03.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.107+0000 7f4ed9400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f4ed40ba7f0 con 0x7f4ed415ecd0 2024-04-20T21:23:03.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.107+0000 7f4ed2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ed40cf680 0x7f4ed40cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:03.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.107+0000 7f4ed2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ed40cf680 0x7f4ed40cd8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53252/0 (socket says 172.21.15.86:53252) 2024-04-20T21:23:03.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.107+0000 7f4ed2800640 1 -- 172.21.15.86:0/79391170 learned_addr learned my addr 172.21.15.86:0/79391170 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:03.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.108+0000 7f4ed2800640 1 -- 172.21.15.86:0/79391170 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ed415ecd0 msgr2=0x7f4ed4159390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:03.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.108+0000 7f4ed2800640 1 --2- 172.21.15.86:0/79391170 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ed415ecd0 0x7f4ed4159390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:03.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.108+0000 7f4ed2800640 1 -- 172.21.15.86:0/79391170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ed415e8f0 msgr2=0x7f4ed40ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:03.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.108+0000 7f4ed2800640 1 --2- 172.21.15.86:0/79391170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ed415e8f0 0x7f4ed40ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:03.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.108+0000 7f4ed2800640 1 -- 172.21.15.86:0/79391170 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ed40cbd60 con 0x7f4ed40cf680 2024-04-20T21:23:03.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.108+0000 7f4ed2800640 1 --2- 172.21.15.86:0/79391170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ed40cf680 0x7f4ed40cd8d0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f4ec400c7f0 tx=0x7f4ec400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:03.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.108+0000 7f4ecaa00640 1 -- 172.21.15.86:0/79391170 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4015070 con 0x7f4ed40cf680 2024-04-20T21:23:03.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.108+0000 7f4ecaa00640 1 -- 172.21.15.86:0/79391170 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ec4007a30 con 0x7f4ed40cf680 2024-04-20T21:23:03.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.108+0000 7f4ed9400640 1 -- 172.21.15.86:0/79391170 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ed40cc040 con 0x7f4ed40cf680 2024-04-20T21:23:03.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.108+0000 7f4ecaa00640 1 -- 172.21.15.86:0/79391170 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4007e00 con 0x7f4ed40cf680 2024-04-20T21:23:03.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.108+0000 7f4ed9400640 1 -- 172.21.15.86:0/79391170 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ed40cc600 con 0x7f4ed40cf680 2024-04-20T21:23:03.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.109+0000 7f4ed9400640 1 -- 172.21.15.86:0/79391170 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ed40689e0 con 0x7f4ed40cf680 2024-04-20T21:23:03.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.110+0000 7f4ecaa00640 1 -- 172.21.15.86:0/79391170 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4ec4018070 con 0x7f4ed40cf680 2024-04-20T21:23:03.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.112+0000 7f4ecaa00640 1 --2- 172.21.15.86:0/79391170 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4eb006c8f0 0x7f4eb006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:03.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.112+0000 7f4ecaa00640 1 -- 172.21.15.86:0/79391170 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f4ec408bc60 con 0x7f4ed40cf680 2024-04-20T21:23:03.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.112+0000 7f4ed1e00640 1 --2- 172.21.15.86:0/79391170 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4eb006c8f0 0x7f4eb006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:03.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.113+0000 7f4ed1e00640 1 --2- 172.21.15.86:0/79391170 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4eb006c8f0 0x7f4eb006edb0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc036000 tx=0x7f4ebc030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:03.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.117+0000 7f4ecaa00640 1 -- 172.21.15.86:0/79391170 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4ec4059f60 con 0x7f4ed40cf680 2024-04-20T21:23:03.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.320+0000 7f4ed9400640 1 -- 172.21.15.86:0/79391170 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4ed40d2550 con 0x7f4ed40cf680 2024-04-20T21:23:03.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.321+0000 7f4ecaa00640 1 -- 172.21.15.86:0/79391170 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) ==== 93+0+10 (secure 0 0 0) 0x7f4ec4059860 con 0x7f4ed40cf680 2024-04-20T21:23:03.320 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:23:03.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.323+0000 7f4ed9400640 1 -- 172.21.15.86:0/79391170 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4eb006c8f0 msgr2=0x7f4eb006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:03.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.323+0000 7f4ed9400640 1 --2- 172.21.15.86:0/79391170 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4eb006c8f0 0x7f4eb006edb0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc036000 tx=0x7f4ebc030000 comp rx=0 tx=0).stop 2024-04-20T21:23:03.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.323+0000 7f4ed9400640 1 -- 172.21.15.86:0/79391170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ed40cf680 msgr2=0x7f4ed40cd8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:03.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.323+0000 7f4ed9400640 1 --2- 172.21.15.86:0/79391170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ed40cf680 0x7f4ed40cd8d0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f4ec400c7f0 tx=0x7f4ec400cbf0 comp rx=0 tx=0).stop 2024-04-20T21:23:03.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.324+0000 7f4ed9400640 1 -- 172.21.15.86:0/79391170 shutdown_connections 2024-04-20T21:23:03.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.324+0000 7f4ed9400640 1 --2- 172.21.15.86:0/79391170 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4eb006c8f0 0x7f4eb006edb0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:03.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.324+0000 7f4ed9400640 1 --2- 172.21.15.86:0/79391170 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ed415ecd0 0x7f4ed4159390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:03.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.324+0000 7f4ed9400640 1 --2- 172.21.15.86:0/79391170 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ed415e8f0 0x7f4ed40ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:03.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.324+0000 7f4ed9400640 1 --2- 172.21.15.86:0/79391170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ed40cf680 0x7f4ed40cd8d0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:03.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.324+0000 7f4ed9400640 1 -- 172.21.15.86:0/79391170 >> 172.21.15.86:0/79391170 conn(0x7f4ed406cc70 msgr2=0x7f4ed415f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:03.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.324+0000 7f4ed9400640 1 -- 172.21.15.86:0/79391170 shutdown_connections 2024-04-20T21:23:03.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.324+0000 7f4ed9400640 1 -- 172.21.15.86:0/79391170 wait complete. 2024-04-20T21:23:03.342 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:23:03.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.544+0000 7f1030400640 1 Processor -- start 2024-04-20T21:23:03.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.544+0000 7f1030400640 1 -- start start 2024-04-20T21:23:03.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.545+0000 7f1030400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1028146210 0x7f102806e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:03.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.546+0000 7f1030400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f102815cd40 con 0x7f102806bbf0 2024-04-20T21:23:03.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.546+0000 7f1030400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f102815d000 con 0x7f102806bfd0 2024-04-20T21:23:03.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.546+0000 7f1030400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f102815d2e0 con 0x7f1028146210 2024-04-20T21:23:03.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.546+0000 7f102da00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f102806bbf0 0x7f1028145b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54920/0 (socket says 172.21.15.86:54920) 2024-04-20T21:23:03.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.546+0000 7f102da00640 1 -- 172.21.15.86:0/1892656053 learned_addr learned my addr 172.21.15.86:0/1892656053 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:23:03.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.546+0000 7f102d000640 1 --2- 172.21.15.86:0/1892656053 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1028146210 0x7f102806e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:03.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.546+0000 7f1027e00640 1 -- 172.21.15.86:0/1892656053 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2811333758 0 0) 0x7f102815cd40 con 0x7f102806bbf0 2024-04-20T21:23:03.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.546+0000 7f1027e00640 1 -- 172.21.15.86:0/1892656053 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1010003520 con 0x7f102806bbf0 2024-04-20T21:23:03.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.547+0000 7f1027e00640 1 -- 172.21.15.86:0/1892656053 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1816746412 0 0) 0x7f1010003520 con 0x7f102806bbf0 2024-04-20T21:23:03.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.547+0000 7f1027e00640 1 -- 172.21.15.86:0/1892656053 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1028146210 msgr2=0x7f102806e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:03.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.547+0000 7f1027e00640 1 --2- 172.21.15.86:0/1892656053 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1028146210 0x7f102806e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:03.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.547+0000 7f1027e00640 1 -- 172.21.15.86:0/1892656053 >> v1:172.21.15.144:6789/0 conn(0x7f102806bfd0 legacy=0x7f10280cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:03.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.547+0000 7f1027e00640 1 -- 172.21.15.86:0/1892656053 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f102815d5c0 con 0x7f102806bbf0 2024-04-20T21:23:03.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.547+0000 7f1027e00640 1 -- 172.21.15.86:0/1892656053 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f101c001670 con 0x7f102806bbf0 2024-04-20T21:23:03.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.547+0000 7f1027e00640 1 -- 172.21.15.86:0/1892656053 >> v1:172.21.15.86:6789/0 conn(0x7f102806bbf0 legacy=0x7f1028145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:03.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.547+0000 7f1027e00640 1 --2- 172.21.15.86:0/1892656053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10100039e0 0x7f1010003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:03.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.547+0000 7f1027e00640 1 --2- 172.21.15.86:0/1892656053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1010004800 0x7f1010004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:03.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.547+0000 7f1027e00640 1 --2- 172.21.15.86:0/1892656053 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1028146210 0x7f10100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:03.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.547+0000 7f1027e00640 1 -- 172.21.15.86:0/1892656053 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f102815d2e0 con 0x7f1010004800 2024-04-20T21:23:03.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.547+0000 7f1027e00640 1 -- 172.21.15.86:0/1892656053 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f101000ba70 con 0x7f10100039e0 2024-04-20T21:23:03.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.547+0000 7f1027e00640 1 -- 172.21.15.86:0/1892656053 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f101000bc70 con 0x7f1028146210 2024-04-20T21:23:03.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.547+0000 7f102d000640 1 --2- 172.21.15.86:0/1892656053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1010004800 0x7f1010004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:03.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.548+0000 7f102da00640 1 --2- 172.21.15.86:0/1892656053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10100039e0 0x7f1010003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:03.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.548+0000 7f102d000640 1 -- 172.21.15.86:0/1892656053 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1028146210 msgr2=0x7f10100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:03.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.548+0000 7f102d000640 1 --2- 172.21.15.86:0/1892656053 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1028146210 0x7f10100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:03.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.548+0000 7f102d000640 1 -- 172.21.15.86:0/1892656053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10100039e0 msgr2=0x7f1010003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:03.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.548+0000 7f102d000640 1 --2- 172.21.15.86:0/1892656053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10100039e0 0x7f1010003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:03.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.548+0000 7f102d000640 1 -- 172.21.15.86:0/1892656053 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f102815d5c0 con 0x7f1010004800 2024-04-20T21:23:03.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.548+0000 7f102da00640 1 --2- 172.21.15.86:0/1892656053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f10100039e0 0x7f1010003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:23:03.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.549+0000 7f102d000640 1 --2- 172.21.15.86:0/1892656053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1010004800 0x7f1010004c80 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f1014009a90 tx=0x7f1010005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca3468e744d7e7ac server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:03.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.549+0000 7f1027e00640 1 -- 172.21.15.86:0/1892656053 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1014039070 con 0x7f1010004800 2024-04-20T21:23:03.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.549+0000 7f1027e00640 1 -- 172.21.15.86:0/1892656053 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f101402c6b0 con 0x7f1010004800 2024-04-20T21:23:03.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.549+0000 7f1027e00640 1 -- 172.21.15.86:0/1892656053 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1014009690 con 0x7f1010004800 2024-04-20T21:23:03.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.550+0000 7f1030400640 1 -- 172.21.15.86:0/1892656053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1010004800 msgr2=0x7f1010004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:03.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.550+0000 7f1030400640 1 --2- 172.21.15.86:0/1892656053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1010004800 0x7f1010004c80 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f1014009a90 tx=0x7f1010005830 comp rx=0 tx=0).stop 2024-04-20T21:23:03.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.550+0000 7f102e400640 1 -- 172.21.15.86:0/1892656053 reap_dead start 2024-04-20T21:23:03.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.550+0000 7f1030400640 1 -- 172.21.15.86:0/1892656053 shutdown_connections 2024-04-20T21:23:03.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.550+0000 7f1030400640 1 -- 172.21.15.86:0/1892656053 >> 172.21.15.86:0/1892656053 conn(0x7f10280b2110 msgr2=0x7f10280b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:03.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.550+0000 7f1030400640 1 -- 172.21.15.86:0/1892656053 shutdown_connections 2024-04-20T21:23:03.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.550+0000 7f1030400640 1 -- 172.21.15.86:0/1892656053 wait complete. 2024-04-20T21:23:03.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.552+0000 7f1030400640 1 Processor -- start 2024-04-20T21:23:03.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.552+0000 7f1030400640 1 -- start start 2024-04-20T21:23:03.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.553+0000 7f1030400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1010004800 0x7f10281f3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:03.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.553+0000 7f1030400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f102813f930 0x7f10280b0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:03.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.553+0000 7f1030400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f102813fdb0 0x7f10280b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:03.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.553+0000 7f1030400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f10280cddd0 con 0x7f1010004800 2024-04-20T21:23:03.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.553+0000 7f1030400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f10280cdc50 con 0x7f102813fdb0 2024-04-20T21:23:03.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.553+0000 7f1030400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f10280cdf50 con 0x7f102813f930 2024-04-20T21:23:03.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.553+0000 7f102da00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1010004800 0x7f10281f3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:03.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.553+0000 7f102da00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1010004800 0x7f10281f3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53268/0 (socket says 172.21.15.86:53268) 2024-04-20T21:23:03.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.553+0000 7f102da00640 1 -- 172.21.15.86:0/901484303 learned_addr learned my addr 172.21.15.86:0/901484303 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:03.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.554+0000 7f102d000640 1 --2- 172.21.15.86:0/901484303 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f102813f930 0x7f10280b0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:03.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.554+0000 7f102e400640 1 --2- 172.21.15.86:0/901484303 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f102813fdb0 0x7f10280b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:03.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.554+0000 7f102da00640 1 -- 172.21.15.86:0/901484303 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f102813f930 msgr2=0x7f10280b0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:03.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.554+0000 7f102da00640 1 --2- 172.21.15.86:0/901484303 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f102813f930 0x7f10280b0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:03.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.554+0000 7f102da00640 1 -- 172.21.15.86:0/901484303 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f102813fdb0 msgr2=0x7f10280b11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:03.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.554+0000 7f102da00640 1 --2- 172.21.15.86:0/901484303 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f102813fdb0 0x7f10280b11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:03.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.554+0000 7f102da00640 1 -- 172.21.15.86:0/901484303 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1028140310 con 0x7f1010004800 2024-04-20T21:23:03.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.554+0000 7f102e400640 1 --2- 172.21.15.86:0/901484303 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f102813fdb0 0x7f10280b11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:23:03.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.554+0000 7f102da00640 1 --2- 172.21.15.86:0/901484303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1010004800 0x7f10281f3f90 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f101c00d930 tx=0x7f101c00ddf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:03.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.554+0000 7f1025600640 1 -- 172.21.15.86:0/901484303 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101c019070 con 0x7f1010004800 2024-04-20T21:23:03.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.554+0000 7f1025600640 1 -- 172.21.15.86:0/901484303 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f101c007500 con 0x7f1010004800 2024-04-20T21:23:03.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.554+0000 7f1025600640 1 -- 172.21.15.86:0/901484303 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101c004a20 con 0x7f1010004800 2024-04-20T21:23:03.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.554+0000 7f1030400640 1 -- 172.21.15.86:0/901484303 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10281405f0 con 0x7f1010004800 2024-04-20T21:23:03.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.555+0000 7f1030400640 1 -- 172.21.15.86:0/901484303 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1028140bb0 con 0x7f1010004800 2024-04-20T21:23:03.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.556+0000 7f1030400640 1 -- 172.21.15.86:0/901484303 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10280689e0 con 0x7f1010004800 2024-04-20T21:23:03.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.557+0000 7f1025600640 1 -- 172.21.15.86:0/901484303 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f101c004bc0 con 0x7f1010004800 2024-04-20T21:23:03.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.558+0000 7f1025600640 1 --2- 172.21.15.86:0/901484303 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f100806c920 0x7f100806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:03.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.558+0000 7f1025600640 1 -- 172.21.15.86:0/901484303 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f101c08b670 con 0x7f1010004800 2024-04-20T21:23:03.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.558+0000 7f102d000640 1 --2- 172.21.15.86:0/901484303 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f100806c920 0x7f100806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:03.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.559+0000 7f102d000640 1 --2- 172.21.15.86:0/901484303 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f100806c920 0x7f100806ede0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1014036000 tx=0x7f1014030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:03.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.562+0000 7f1025600640 1 -- 172.21.15.86:0/901484303 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f101c059aa0 con 0x7f1010004800 2024-04-20T21:23:03.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.762+0000 7f1030400640 1 -- 172.21.15.86:0/901484303 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f102815dff0 con 0x7f1010004800 2024-04-20T21:23:03.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.763+0000 7f1025600640 1 -- 172.21.15.86:0/901484303 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v71) ==== 104+0+11 (secure 0 0 0) 0x7f101c0593a0 con 0x7f1010004800 2024-04-20T21:23:03.762 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-20T21:23:03.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.765+0000 7f1030400640 1 -- 172.21.15.86:0/901484303 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f100806c920 msgr2=0x7f100806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:03.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.765+0000 7f1030400640 1 --2- 172.21.15.86:0/901484303 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f100806c920 0x7f100806ede0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1014036000 tx=0x7f1014030000 comp rx=0 tx=0).stop 2024-04-20T21:23:03.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.765+0000 7f1030400640 1 -- 172.21.15.86:0/901484303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1010004800 msgr2=0x7f10281f3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:03.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.765+0000 7f1030400640 1 --2- 172.21.15.86:0/901484303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1010004800 0x7f10281f3f90 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f101c00d930 tx=0x7f101c00ddf0 comp rx=0 tx=0).stop 2024-04-20T21:23:03.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.765+0000 7f1030400640 1 -- 172.21.15.86:0/901484303 shutdown_connections 2024-04-20T21:23:03.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.765+0000 7f1030400640 1 --2- 172.21.15.86:0/901484303 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f100806c920 0x7f100806ede0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:03.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.765+0000 7f1030400640 1 --2- 172.21.15.86:0/901484303 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f102813fdb0 0x7f10280b11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:03.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.765+0000 7f1030400640 1 --2- 172.21.15.86:0/901484303 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f102813f930 0x7f10280b0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:03.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.765+0000 7f1030400640 1 --2- 172.21.15.86:0/901484303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1010004800 0x7f10281f3f90 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:03.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.766+0000 7f1030400640 1 -- 172.21.15.86:0/901484303 >> 172.21.15.86:0/901484303 conn(0x7f10280b2110 msgr2=0x7f1028071f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:03.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.766+0000 7f1030400640 1 -- 172.21.15.86:0/901484303 shutdown_connections 2024-04-20T21:23:03.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.766+0000 7f1030400640 1 -- 172.21.15.86:0/901484303 wait complete. 2024-04-20T21:23:03.783 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:23:03.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.985+0000 7fd1d2a00640 1 Processor -- start 2024-04-20T21:23:03.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.985+0000 7fd1d2a00640 1 -- start start 2024-04-20T21:23:03.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.987+0000 7fd1d2a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1cc06ef20 0x7fd1cc15de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:03.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.987+0000 7fd1d2a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1cc06f3d0 con 0x7fd1cc0af1f0 2024-04-20T21:23:03.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.987+0000 7fd1d2a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1cc06de80 con 0x7fd1cc06d9c0 2024-04-20T21:23:03.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.987+0000 7fd1d2a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd1cc06e160 con 0x7fd1cc06ef20 2024-04-20T21:23:03.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.987+0000 7fd1d1600640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd1cc0af1f0 0x7fd1cc15d520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54926/0 (socket says 172.21.15.86:54926) 2024-04-20T21:23:03.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.987+0000 7fd1d1600640 1 -- 172.21.15.86:0/1096765364 learned_addr learned my addr 172.21.15.86:0/1096765364 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:23:03.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.987+0000 7fd1d2000640 1 --2- 172.21.15.86:0/1096765364 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1cc06ef20 0x7fd1cc15de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:03.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.987+0000 7fd1cbe00640 1 -- 172.21.15.86:0/1096765364 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 609702988 0 0) 0x7fd1cc06f3d0 con 0x7fd1cc0af1f0 2024-04-20T21:23:03.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.987+0000 7fd1cbe00640 1 -- 172.21.15.86:0/1096765364 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd1b4003520 con 0x7fd1cc0af1f0 2024-04-20T21:23:03.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.988+0000 7fd1cbe00640 1 -- 172.21.15.86:0/1096765364 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2218039457 0 0) 0x7fd1b4003520 con 0x7fd1cc0af1f0 2024-04-20T21:23:03.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.988+0000 7fd1cbe00640 1 -- 172.21.15.86:0/1096765364 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1cc06ef20 msgr2=0x7fd1cc15de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:03.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.988+0000 7fd1cbe00640 1 --2- 172.21.15.86:0/1096765364 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1cc06ef20 0x7fd1cc15de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:03.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.988+0000 7fd1cbe00640 1 -- 172.21.15.86:0/1096765364 >> v1:172.21.15.144:6789/0 conn(0x7fd1cc06d9c0 legacy=0x7fd1cc0af5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:03.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.988+0000 7fd1cbe00640 1 -- 172.21.15.86:0/1096765364 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1cc06e440 con 0x7fd1cc0af1f0 2024-04-20T21:23:03.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.988+0000 7fd1cbe00640 1 -- 172.21.15.86:0/1096765364 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fd1c0001670 con 0x7fd1cc0af1f0 2024-04-20T21:23:03.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.988+0000 7fd1cbe00640 1 -- 172.21.15.86:0/1096765364 >> v1:172.21.15.86:6789/0 conn(0x7fd1cc0af1f0 legacy=0x7fd1cc15d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:03.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.988+0000 7fd1cbe00640 1 --2- 172.21.15.86:0/1096765364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1b40039e0 0x7fd1b4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:03.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.988+0000 7fd1cbe00640 1 --2- 172.21.15.86:0/1096765364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1b4004800 0x7fd1b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:03.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.988+0000 7fd1cbe00640 1 --2- 172.21.15.86:0/1096765364 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1cc06ef20 0x7fd1b4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:03.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.988+0000 7fd1cbe00640 1 -- 172.21.15.86:0/1096765364 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc06e160 con 0x7fd1b4004800 2024-04-20T21:23:03.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.988+0000 7fd1cbe00640 1 -- 172.21.15.86:0/1096765364 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b400baf0 con 0x7fd1b40039e0 2024-04-20T21:23:03.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.988+0000 7fd1cbe00640 1 -- 172.21.15.86:0/1096765364 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd1b400bcf0 con 0x7fd1cc06ef20 2024-04-20T21:23:03.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.988+0000 7fd1d0c00640 1 --2- 172.21.15.86:0/1096765364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1b4004800 0x7fd1b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:03.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.989+0000 7fd1d1600640 1 --2- 172.21.15.86:0/1096765364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1b40039e0 0x7fd1b4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:03.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.989+0000 7fd1d0c00640 1 -- 172.21.15.86:0/1096765364 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1cc06ef20 msgr2=0x7fd1b4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:03.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.989+0000 7fd1d2000640 1 --2- 172.21.15.86:0/1096765364 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1cc06ef20 0x7fd1b4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:03.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.989+0000 7fd1d0c00640 1 --2- 172.21.15.86:0/1096765364 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1cc06ef20 0x7fd1b4005200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:03.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.989+0000 7fd1d0c00640 1 -- 172.21.15.86:0/1096765364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1b40039e0 msgr2=0x7fd1b4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:03.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.989+0000 7fd1d0c00640 1 --2- 172.21.15.86:0/1096765364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1b40039e0 0x7fd1b4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:03.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.989+0000 7fd1d0c00640 1 -- 172.21.15.86:0/1096765364 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1b4003520 con 0x7fd1b4004800 2024-04-20T21:23:03.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.989+0000 7fd1d1600640 1 --2- 172.21.15.86:0/1096765364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1b40039e0 0x7fd1b4003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:23:03.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.990+0000 7fd1d0c00640 1 --2- 172.21.15.86:0/1096765364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1b4004800 0x7fd1b4004c80 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8001750 tx=0x7fd1b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=218c4d42fe184414 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:03.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.990+0000 7fd1cbe00640 1 -- 172.21.15.86:0/1096765364 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b8001370 con 0x7fd1b4004800 2024-04-20T21:23:03.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.990+0000 7fd1cbe00640 1 -- 172.21.15.86:0/1096765364 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1b80052d0 con 0x7fd1b4004800 2024-04-20T21:23:03.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.990+0000 7fd1cbe00640 1 -- 172.21.15.86:0/1096765364 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b8003830 con 0x7fd1b4004800 2024-04-20T21:23:03.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.991+0000 7fd1d2a00640 1 -- 172.21.15.86:0/1096765364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1b4004800 msgr2=0x7fd1b4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:03.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.991+0000 7fd1d2a00640 1 --2- 172.21.15.86:0/1096765364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1b4004800 0x7fd1b4004c80 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8001750 tx=0x7fd1b4005830 comp rx=0 tx=0).stop 2024-04-20T21:23:03.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.991+0000 7fd1d2000640 1 -- 172.21.15.86:0/1096765364 reap_dead start 2024-04-20T21:23:03.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.991+0000 7fd1d2a00640 1 -- 172.21.15.86:0/1096765364 shutdown_connections 2024-04-20T21:23:03.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.991+0000 7fd1d2a00640 1 -- 172.21.15.86:0/1096765364 >> 172.21.15.86:0/1096765364 conn(0x7fd1cc0b1f00 msgr2=0x7fd1cc0b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:03.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.991+0000 7fd1d2a00640 1 -- 172.21.15.86:0/1096765364 shutdown_connections 2024-04-20T21:23:03.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.991+0000 7fd1d2a00640 1 -- 172.21.15.86:0/1096765364 wait complete. 2024-04-20T21:23:03.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.992+0000 7fd1d2a00640 1 Processor -- start 2024-04-20T21:23:03.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.993+0000 7fd1d2a00640 1 -- start start 2024-04-20T21:23:04.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.994+0000 7fd1d2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1b4004800 0x7fd1cc1f4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:04.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.994+0000 7fd1d2a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1cc15cae0 0x7fd1cc0b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:04.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.994+0000 7fd1d2a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1cc15d790 0x7fd1cc15dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:04.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.994+0000 7fd1d2a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc15a3c0 con 0x7fd1b4004800 2024-04-20T21:23:04.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.994+0000 7fd1d2a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc06ea00 con 0x7fd1cc15d790 2024-04-20T21:23:04.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.994+0000 7fd1d2a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd1cc15a540 con 0x7fd1cc15cae0 2024-04-20T21:23:04.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.994+0000 7fd1d1600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1b4004800 0x7fd1cc1f4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:04.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.994+0000 7fd1d1600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1b4004800 0x7fd1cc1f4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53282/0 (socket says 172.21.15.86:53282) 2024-04-20T21:23:04.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.994+0000 7fd1d1600640 1 -- 172.21.15.86:0/1940236590 learned_addr learned my addr 172.21.15.86:0/1940236590 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:04.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.994+0000 7fd1d1600640 1 -- 172.21.15.86:0/1940236590 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1cc15cae0 msgr2=0x7fd1cc0b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:04.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.994+0000 7fd1d1600640 1 --2- 172.21.15.86:0/1940236590 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1cc15cae0 0x7fd1cc0b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:04.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.994+0000 7fd1d1600640 1 -- 172.21.15.86:0/1940236590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1cc15d790 msgr2=0x7fd1cc15dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:04.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.994+0000 7fd1d1600640 1 --2- 172.21.15.86:0/1940236590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1cc15d790 0x7fd1cc15dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:04.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.994+0000 7fd1d1600640 1 -- 172.21.15.86:0/1940236590 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1cc1f48b0 con 0x7fd1b4004800 2024-04-20T21:23:04.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.995+0000 7fd1d1600640 1 --2- 172.21.15.86:0/1940236590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1b4004800 0x7fd1cc1f4370 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fd1c000d940 tx=0x7fd1c000de00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:04.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.995+0000 7fd1c9600640 1 -- 172.21.15.86:0/1940236590 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c0019070 con 0x7fd1b4004800 2024-04-20T21:23:04.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.995+0000 7fd1c9600640 1 -- 172.21.15.86:0/1940236590 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1c00074e0 con 0x7fd1b4004800 2024-04-20T21:23:04.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.995+0000 7fd1c9600640 1 -- 172.21.15.86:0/1940236590 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c00049e0 con 0x7fd1b4004800 2024-04-20T21:23:04.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.995+0000 7fd1d2a00640 1 -- 172.21.15.86:0/1940236590 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1cc0b4780 con 0x7fd1b4004800 2024-04-20T21:23:04.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.995+0000 7fd1d2a00640 1 -- 172.21.15.86:0/1940236590 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1cc0b4d40 con 0x7fd1b4004800 2024-04-20T21:23:04.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.996+0000 7fd1d2a00640 1 -- 172.21.15.86:0/1940236590 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1cc04fd90 con 0x7fd1b4004800 2024-04-20T21:23:04.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.997+0000 7fd1c9600640 1 -- 172.21.15.86:0/1940236590 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd1c001c070 con 0x7fd1b4004800 2024-04-20T21:23:04.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.998+0000 7fd1c9600640 1 --2- 172.21.15.86:0/1940236590 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd1ac06c8f0 0x7fd1ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:04.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.998+0000 7fd1c9600640 1 -- 172.21.15.86:0/1940236590 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7fd1c008b6f0 con 0x7fd1b4004800 2024-04-20T21:23:04.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.998+0000 7fd1d0c00640 1 --2- 172.21.15.86:0/1940236590 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd1ac06c8f0 0x7fd1ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:04.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:03.999+0000 7fd1d0c00640 1 --2- 172.21.15.86:0/1940236590 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd1ac06c8f0 0x7fd1ac06edb0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8033000 tx=0x7fd1b802f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:04.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.003+0000 7fd1c9600640 1 -- 172.21.15.86:0/1940236590 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd1c0059bb0 con 0x7fd1b4004800 2024-04-20T21:23:04.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.198+0000 7fd1d2a00640 1 -- 172.21.15.86:0/1940236590 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd1cc0ad1e0 con 0x7fd1b4004800 2024-04-20T21:23:04.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.199+0000 7fd1c9600640 1 -- 172.21.15.86:0/1940236590 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v71) ==== 100+0+11 (secure 0 0 0) 0x7fd1c00594b0 con 0x7fd1b4004800 2024-04-20T21:23:04.198 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-20T21:23:04.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.201+0000 7fd1d2a00640 1 -- 172.21.15.86:0/1940236590 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd1ac06c8f0 msgr2=0x7fd1ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:04.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.201+0000 7fd1d2a00640 1 --2- 172.21.15.86:0/1940236590 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd1ac06c8f0 0x7fd1ac06edb0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8033000 tx=0x7fd1b802f000 comp rx=0 tx=0).stop 2024-04-20T21:23:04.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.201+0000 7fd1d2a00640 1 -- 172.21.15.86:0/1940236590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1b4004800 msgr2=0x7fd1cc1f4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:04.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.201+0000 7fd1d2a00640 1 --2- 172.21.15.86:0/1940236590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1b4004800 0x7fd1cc1f4370 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fd1c000d940 tx=0x7fd1c000de00 comp rx=0 tx=0).stop 2024-04-20T21:23:04.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.202+0000 7fd1d2a00640 1 -- 172.21.15.86:0/1940236590 shutdown_connections 2024-04-20T21:23:04.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.202+0000 7fd1d2a00640 1 --2- 172.21.15.86:0/1940236590 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd1ac06c8f0 0x7fd1ac06edb0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:04.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.202+0000 7fd1d2a00640 1 --2- 172.21.15.86:0/1940236590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1cc15d790 0x7fd1cc15dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:04.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.202+0000 7fd1d2a00640 1 --2- 172.21.15.86:0/1940236590 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1cc15cae0 0x7fd1cc0b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:04.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.202+0000 7fd1d2a00640 1 --2- 172.21.15.86:0/1940236590 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1b4004800 0x7fd1cc1f4370 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:04.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.202+0000 7fd1d2a00640 1 -- 172.21.15.86:0/1940236590 >> 172.21.15.86:0/1940236590 conn(0x7fd1cc0b1f00 msgr2=0x7fd1cc0b06b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:04.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.202+0000 7fd1d2a00640 1 -- 172.21.15.86:0/1940236590 shutdown_connections 2024-04-20T21:23:04.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.202+0000 7fd1d2a00640 1 -- 172.21.15.86:0/1940236590 wait complete. 2024-04-20T21:23:04.218 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T21:23:04.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.421+0000 7f45be200640 1 Processor -- start 2024-04-20T21:23:04.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.421+0000 7f45be200640 1 -- start start 2024-04-20T21:23:04.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.423+0000 7f45be200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45b8145fe0 0x7f45b80d02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:04.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.423+0000 7f45be200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f45b80d0810 0x7f45b80cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:04.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.423+0000 7f45be200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45b815dcc0 con 0x7f45b80cd610 2024-04-20T21:23:04.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.423+0000 7f45be200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f45b80d00d0 con 0x7f45b80d0810 2024-04-20T21:23:04.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.423+0000 7f45be200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f45b80d0c70 con 0x7f45b8145fe0 2024-04-20T21:23:04.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.424+0000 7f45b7400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45b8145fe0 0x7f45b80d02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:04.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.424+0000 7f45b7400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45b8145fe0 0x7f45b80d02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:57258/0 (socket says 172.21.15.86:57258) 2024-04-20T21:23:04.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.424+0000 7f45b7400640 1 -- 172.21.15.86:0/1184750739 learned_addr learned my addr 172.21.15.86:0/1184750739 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:04.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.424+0000 7f45b6a00640 1 --2- 172.21.15.86:0/1184750739 >> v2:172.21.15.144:3300/0 conn(0x7f45b80d0810 0x7f45b80cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:04.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.425+0000 7f45b7400640 1 -- 172.21.15.86:0/1184750739 >> v2:172.21.15.144:3300/0 conn(0x7f45b80d0810 msgr2=0x7f45b80cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:04.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.425+0000 7f45b7400640 1 --2- 172.21.15.86:0/1184750739 >> v2:172.21.15.144:3300/0 conn(0x7f45b80d0810 0x7f45b80cd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:04.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.425+0000 7f45b7400640 1 -- 172.21.15.86:0/1184750739 >> v1:172.21.15.144:6789/0 conn(0x7f45b80cd610 legacy=0x7f45b8160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:04.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.425+0000 7f45b7400640 1 -- 172.21.15.86:0/1184750739 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45b815cf00 con 0x7f45b8145fe0 2024-04-20T21:23:04.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.426+0000 7f45b7400640 1 --2- 172.21.15.86:0/1184750739 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45b8145fe0 0x7f45b80d02d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f45a8009870 tx=0x7f45a802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f20fd55612c5ccc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:04.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.427+0000 7f45b6000640 1 -- 172.21.15.86:0/1184750739 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a8034070 con 0x7f45b8145fe0 2024-04-20T21:23:04.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.427+0000 7f45b6000640 1 -- 172.21.15.86:0/1184750739 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45a8030840 con 0x7f45b8145fe0 2024-04-20T21:23:04.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.427+0000 7f45b6000640 1 -- 172.21.15.86:0/1184750739 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a8030c10 con 0x7f45b8145fe0 2024-04-20T21:23:04.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.427+0000 7f45be200640 1 -- 172.21.15.86:0/1184750739 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45b8145fe0 msgr2=0x7f45b80d02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:04.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.427+0000 7f45be200640 1 --2- 172.21.15.86:0/1184750739 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45b8145fe0 0x7f45b80d02d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f45a8009870 tx=0x7f45a802b980 comp rx=0 tx=0).stop 2024-04-20T21:23:04.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.428+0000 7f45be200640 1 -- 172.21.15.86:0/1184750739 shutdown_connections 2024-04-20T21:23:04.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.428+0000 7f45be200640 1 --2- 172.21.15.86:0/1184750739 >> v2:172.21.15.144:3300/0 conn(0x7f45b80d0810 0x7f45b80cd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:04.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.428+0000 7f45be200640 1 --2- 172.21.15.86:0/1184750739 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45b8145fe0 0x7f45b80d02d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:04.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.428+0000 7f45be200640 1 -- 172.21.15.86:0/1184750739 >> 172.21.15.86:0/1184750739 conn(0x7f45b806b980 msgr2=0x7f45b806bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:04.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.428+0000 7f45be200640 1 -- 172.21.15.86:0/1184750739 shutdown_connections 2024-04-20T21:23:04.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.428+0000 7f45be200640 1 -- 172.21.15.86:0/1184750739 wait complete. 2024-04-20T21:23:04.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.430+0000 7f45be200640 1 Processor -- start 2024-04-20T21:23:04.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.430+0000 7f45be200640 1 -- start start 2024-04-20T21:23:04.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.431+0000 7f45be200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45b80cd610 0x7f45b80aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:04.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.431+0000 7f45be200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45b80d0810 0x7f45b80ab790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:04.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.431+0000 7f45be200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45b8145fe0 0x7f45b80b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:04.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.431+0000 7f45be200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f45b80cdaa0 con 0x7f45b80d0810 2024-04-20T21:23:04.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.431+0000 7f45be200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f45b815d4c0 con 0x7f45b80cd610 2024-04-20T21:23:04.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.431+0000 7f45be200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f45b815d7a0 con 0x7f45b8145fe0 2024-04-20T21:23:04.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.431+0000 7f45b6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45b80d0810 0x7f45b80ab790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:04.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.432+0000 7f45b6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45b80d0810 0x7f45b80ab790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53288/0 (socket says 172.21.15.86:53288) 2024-04-20T21:23:04.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.432+0000 7f45b6a00640 1 -- 172.21.15.86:0/2281595772 learned_addr learned my addr 172.21.15.86:0/2281595772 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:04.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.432+0000 7f45b7400640 1 --2- 172.21.15.86:0/2281595772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45b80cd610 0x7f45b80aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:04.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.432+0000 7f45b7e00640 1 --2- 172.21.15.86:0/2281595772 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45b8145fe0 0x7f45b80b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:04.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.432+0000 7f45b6a00640 1 -- 172.21.15.86:0/2281595772 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45b8145fe0 msgr2=0x7f45b80b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:04.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.432+0000 7f45b6a00640 1 --2- 172.21.15.86:0/2281595772 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45b8145fe0 0x7f45b80b43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:04.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.432+0000 7f45b6a00640 1 -- 172.21.15.86:0/2281595772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45b80cd610 msgr2=0x7f45b80aebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:04.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.432+0000 7f45b6a00640 1 --2- 172.21.15.86:0/2281595772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45b80cd610 0x7f45b80aebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:04.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.432+0000 7f45b6a00640 1 -- 172.21.15.86:0/2281595772 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45b80aa9d0 con 0x7f45b80d0810 2024-04-20T21:23:04.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.432+0000 7f45b6a00640 1 --2- 172.21.15.86:0/2281595772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45b80d0810 0x7f45b80ab790 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f45a400e9e0 tx=0x7f45a400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:04.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.433+0000 7f45af400640 1 -- 172.21.15.86:0/2281595772 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a4017070 con 0x7f45b80d0810 2024-04-20T21:23:04.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.433+0000 7f45af400640 1 -- 172.21.15.86:0/2281595772 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45a400c9d0 con 0x7f45b80d0810 2024-04-20T21:23:04.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.433+0000 7f45be200640 1 -- 172.21.15.86:0/2281595772 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45b80aacb0 con 0x7f45b80d0810 2024-04-20T21:23:04.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.433+0000 7f45af400640 1 -- 172.21.15.86:0/2281595772 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a400cdc0 con 0x7f45b80d0810 2024-04-20T21:23:04.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.433+0000 7f45be200640 1 -- 172.21.15.86:0/2281595772 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45b8143150 con 0x7f45b80d0810 2024-04-20T21:23:04.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.434+0000 7f45be200640 1 -- 172.21.15.86:0/2281595772 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45b80689e0 con 0x7f45b80d0810 2024-04-20T21:23:04.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.436+0000 7f45af400640 1 -- 172.21.15.86:0/2281595772 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f45a401a070 con 0x7f45b80d0810 2024-04-20T21:23:04.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.436+0000 7f45af400640 1 --2- 172.21.15.86:0/2281595772 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f459806c8f0 0x7f459806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:04.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.437+0000 7f45af400640 1 -- 172.21.15.86:0/2281595772 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f45a408be60 con 0x7f45b80d0810 2024-04-20T21:23:04.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.437+0000 7f45b7400640 1 --2- 172.21.15.86:0/2281595772 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f459806c8f0 0x7f459806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:04.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.437+0000 7f45b7400640 1 --2- 172.21.15.86:0/2281595772 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f459806c8f0 0x7f459806edb0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f45a8036000 tx=0x7f45a803b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:04.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.441+0000 7f45af400640 1 -- 172.21.15.86:0/2281595772 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f45a405a320 con 0x7f45b80d0810 2024-04-20T21:23:04.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.642+0000 7f45be200640 1 -- 172.21.15.86:0/2281595772 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f45b80d1020 con 0x7f45b80d0810 2024-04-20T21:23:04.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.643+0000 7f45af400640 1 -- 172.21.15.86:0/2281595772 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v71) ==== 80+0+0 (secure 0 0 0) 0x7f45a4059c20 con 0x7f45b80d0810 2024-04-20T21:23:04.641 INFO:teuthology.orchestra.run.smithi086.stderr:listed 0 entries 2024-04-20T21:23:04.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.645+0000 7f45be200640 1 -- 172.21.15.86:0/2281595772 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f459806c8f0 msgr2=0x7f459806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:04.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.645+0000 7f45be200640 1 --2- 172.21.15.86:0/2281595772 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f459806c8f0 0x7f459806edb0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f45a8036000 tx=0x7f45a803b040 comp rx=0 tx=0).stop 2024-04-20T21:23:04.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.645+0000 7f45be200640 1 -- 172.21.15.86:0/2281595772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45b80d0810 msgr2=0x7f45b80ab790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:04.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.645+0000 7f45be200640 1 --2- 172.21.15.86:0/2281595772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45b80d0810 0x7f45b80ab790 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f45a400e9e0 tx=0x7f45a400eea0 comp rx=0 tx=0).stop 2024-04-20T21:23:04.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.646+0000 7f45be200640 1 -- 172.21.15.86:0/2281595772 shutdown_connections 2024-04-20T21:23:04.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.646+0000 7f45be200640 1 --2- 172.21.15.86:0/2281595772 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f459806c8f0 0x7f459806edb0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:04.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.646+0000 7f45be200640 1 --2- 172.21.15.86:0/2281595772 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f45b8145fe0 0x7f45b80b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:04.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.646+0000 7f45be200640 1 --2- 172.21.15.86:0/2281595772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45b80d0810 0x7f45b80ab790 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:04.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.646+0000 7f45be200640 1 --2- 172.21.15.86:0/2281595772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45b80cd610 0x7f45b80aebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:04.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.646+0000 7f45be200640 1 -- 172.21.15.86:0/2281595772 >> 172.21.15.86:0/2281595772 conn(0x7f45b806b980 msgr2=0x7f45b8144c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:04.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.646+0000 7f45be200640 1 -- 172.21.15.86:0/2281595772 shutdown_connections 2024-04-20T21:23:04.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:04.646+0000 7f45be200640 1 -- 172.21.15.86:0/2281595772 wait complete. 2024-04-20T21:23:04.662 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi086.front.sepia.ceph.com... 2024-04-20T21:23:04.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:23:04.663 DEBUG:teuthology.orchestra.run.smithi086:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-20T21:23:04.735 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[47282]: fuse finished with error 0 and tester_r 0 2024-04-20T21:23:04.739 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:23:10.840 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-20T21:23:10.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:23:10.840 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-20T21:23:10.899 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:23:11.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.104+0000 7fd0f0400640 1 Processor -- start 2024-04-20T21:23:11.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.104+0000 7fd0f0400640 1 -- start start 2024-04-20T21:23:11.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.106+0000 7fd0f0400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0e806e6d0 0x7fd0e8071d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:11.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.106+0000 7fd0f0400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd0e8072240 0x7fd0e815cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:11.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.106+0000 7fd0f0400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0e806eb30 con 0x7fd0e8146130 2024-04-20T21:23:11.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.106+0000 7fd0f0400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd0e806dc30 con 0x7fd0e8072240 2024-04-20T21:23:11.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.106+0000 7fd0f0400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd0e80cd8e0 con 0x7fd0e806e6d0 2024-04-20T21:23:11.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.106+0000 7fd0eda00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd0e8146130 0x7fd0e80ccdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:32936/0 (socket says 172.21.15.86:32936) 2024-04-20T21:23:11.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.106+0000 7fd0eda00640 1 -- 172.21.15.86:0/3747214938 learned_addr learned my addr 172.21.15.86:0/3747214938 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:23:11.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.106+0000 7fd0ee400640 1 --2- 172.21.15.86:0/3747214938 >> v2:172.21.15.86:3300/0 conn(0x7fd0e8072240 0x7fd0e815cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:11.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.107+0000 7fd0ed000640 1 --2- 172.21.15.86:0/3747214938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0e806e6d0 0x7fd0e8071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:11.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.107+0000 7fd0e7e00640 1 -- 172.21.15.86:0/3747214938 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1868489431 0 0) 0x7fd0dc001110 con 0x7fd0e8146130 2024-04-20T21:23:11.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.107+0000 7fd0e7e00640 1 -- 172.21.15.86:0/3747214938 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd0d0003520 con 0x7fd0e8146130 2024-04-20T21:23:11.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.107+0000 7fd0ee400640 1 -- 172.21.15.86:0/3747214938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0e806e6d0 msgr2=0x7fd0e8071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:11.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.107+0000 7fd0ee400640 1 --2- 172.21.15.86:0/3747214938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0e806e6d0 0x7fd0e8071d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:11.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.107+0000 7fd0ee400640 1 -- 172.21.15.86:0/3747214938 >> v1:172.21.15.86:6789/0 conn(0x7fd0e8146130 legacy=0x7fd0e80ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:11.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.107+0000 7fd0ee400640 1 -- 172.21.15.86:0/3747214938 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0e80cdbc0 con 0x7fd0e8072240 2024-04-20T21:23:11.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.107+0000 7fd0ed000640 1 --2- 172.21.15.86:0/3747214938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0e806e6d0 0x7fd0e8071d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:23:11.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.108+0000 7fd0ee400640 1 --2- 172.21.15.86:0/3747214938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0e8072240 0x7fd0e815cc20 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fd0e0009d90 tx=0x7fd0e002bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61a0fb29991082e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:11.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.108+0000 7fd0e7e00640 1 -- 172.21.15.86:0/3747214938 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e0039070 con 0x7fd0e8072240 2024-04-20T21:23:11.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.108+0000 7fd0e7e00640 1 -- 172.21.15.86:0/3747214938 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0e002cce0 con 0x7fd0e8072240 2024-04-20T21:23:11.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.108+0000 7fd0e7e00640 1 -- 172.21.15.86:0/3747214938 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e0034660 con 0x7fd0e8072240 2024-04-20T21:23:11.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.109+0000 7fd0f0400640 1 -- 172.21.15.86:0/3747214938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0e8072240 msgr2=0x7fd0e815cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:11.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.109+0000 7fd0f0400640 1 --2- 172.21.15.86:0/3747214938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0e8072240 0x7fd0e815cc20 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fd0e0009d90 tx=0x7fd0e002bf30 comp rx=0 tx=0).stop 2024-04-20T21:23:11.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.109+0000 7fd0f0400640 1 -- 172.21.15.86:0/3747214938 shutdown_connections 2024-04-20T21:23:11.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.109+0000 7fd0f0400640 1 --2- 172.21.15.86:0/3747214938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0e8072240 0x7fd0e815cc20 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:11.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.109+0000 7fd0f0400640 1 --2- 172.21.15.86:0/3747214938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0e806e6d0 0x7fd0e8071d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:11.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.109+0000 7fd0f0400640 1 -- 172.21.15.86:0/3747214938 >> 172.21.15.86:0/3747214938 conn(0x7fd0e80b2360 msgr2=0x7fd0e80b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:11.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.109+0000 7fd0f0400640 1 -- 172.21.15.86:0/3747214938 shutdown_connections 2024-04-20T21:23:11.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.109+0000 7fd0f0400640 1 -- 172.21.15.86:0/3747214938 wait complete. 2024-04-20T21:23:11.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.111+0000 7fd0f0400640 1 Processor -- start 2024-04-20T21:23:11.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.111+0000 7fd0f0400640 1 -- start start 2024-04-20T21:23:11.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.112+0000 7fd0f0400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0e806e6d0 0x7fd0e813b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:11.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.112+0000 7fd0f0400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0e8072240 0x7fd0e813a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:11.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.112+0000 7fd0f0400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0e8146130 0x7fd0e815bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:11.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.112+0000 7fd0f0400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e806f150 con 0x7fd0e8072240 2024-04-20T21:23:11.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.112+0000 7fd0f0400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e806e1b0 con 0x7fd0e806e6d0 2024-04-20T21:23:11.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.112+0000 7fd0f0400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd0e80cd4e0 con 0x7fd0e8146130 2024-04-20T21:23:11.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.112+0000 7fd0ed000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0e8072240 0x7fd0e813a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:11.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.112+0000 7fd0ed000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0e8072240 0x7fd0e813a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43870/0 (socket says 172.21.15.86:43870) 2024-04-20T21:23:11.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.112+0000 7fd0ed000640 1 -- 172.21.15.86:0/2635932296 learned_addr learned my addr 172.21.15.86:0/2635932296 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:11.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.112+0000 7fd0ee400640 1 --2- 172.21.15.86:0/2635932296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0e8146130 0x7fd0e815bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:11.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.112+0000 7fd0eda00640 1 --2- 172.21.15.86:0/2635932296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0e806e6d0 0x7fd0e813b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:11.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.112+0000 7fd0ed000640 1 -- 172.21.15.86:0/2635932296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0e8146130 msgr2=0x7fd0e815bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:11.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.112+0000 7fd0ed000640 1 --2- 172.21.15.86:0/2635932296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0e8146130 0x7fd0e815bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:11.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.112+0000 7fd0ed000640 1 -- 172.21.15.86:0/2635932296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0e806e6d0 msgr2=0x7fd0e813b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:11.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.112+0000 7fd0ed000640 1 --2- 172.21.15.86:0/2635932296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0e806e6d0 0x7fd0e813b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:11.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.112+0000 7fd0ed000640 1 -- 172.21.15.86:0/2635932296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0e813c380 con 0x7fd0e8072240 2024-04-20T21:23:11.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.112+0000 7fd0ee400640 1 --2- 172.21.15.86:0/2635932296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0e8146130 0x7fd0e815bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:23:11.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.112+0000 7fd0eda00640 1 --2- 172.21.15.86:0/2635932296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0e806e6d0 0x7fd0e813b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:23:11.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.113+0000 7fd0ed000640 1 --2- 172.21.15.86:0/2635932296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0e8072240 0x7fd0e813a6b0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fd0d400c820 tx=0x7fd0d400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:11.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.113+0000 7fd0e5600640 1 -- 172.21.15.86:0/2635932296 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d4015070 con 0x7fd0e8072240 2024-04-20T21:23:11.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.113+0000 7fd0e5600640 1 -- 172.21.15.86:0/2635932296 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0d4007a30 con 0x7fd0e8072240 2024-04-20T21:23:11.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.113+0000 7fd0e5600640 1 -- 172.21.15.86:0/2635932296 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d4007de0 con 0x7fd0e8072240 2024-04-20T21:23:11.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.113+0000 7fd0f0400640 1 -- 172.21.15.86:0/2635932296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0e813c660 con 0x7fd0e8072240 2024-04-20T21:23:11.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.113+0000 7fd0f0400640 1 -- 172.21.15.86:0/2635932296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0e813cc20 con 0x7fd0e8072240 2024-04-20T21:23:11.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.114+0000 7fd0f0400640 1 -- 172.21.15.86:0/2635932296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0e80689e0 con 0x7fd0e8072240 2024-04-20T21:23:11.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.115+0000 7fd0e5600640 1 -- 172.21.15.86:0/2635932296 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd0d4018070 con 0x7fd0e8072240 2024-04-20T21:23:11.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.117+0000 7fd0e5600640 1 --2- 172.21.15.86:0/2635932296 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd0c806c8f0 0x7fd0c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:11.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.117+0000 7fd0e5600640 1 -- 172.21.15.86:0/2635932296 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7fd0d408ba90 con 0x7fd0e8072240 2024-04-20T21:23:11.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.117+0000 7fd0eda00640 1 --2- 172.21.15.86:0/2635932296 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd0c806c8f0 0x7fd0c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:11.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.118+0000 7fd0eda00640 1 --2- 172.21.15.86:0/2635932296 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd0c806c8f0 0x7fd0c806edb0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc0012b0 tx=0x7fd0dc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:11.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.122+0000 7fd0e5600640 1 -- 172.21.15.86:0/2635932296 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd0d4059e20 con 0x7fd0e8072240 2024-04-20T21:23:11.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.364+0000 7fd0f0400640 1 -- 172.21.15.86:0/2635932296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd0e815ff50 con 0x7fd0e8072240 2024-04-20T21:23:11.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.365+0000 7fd0e5600640 1 -- 172.21.15.86:0/2635932296 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+5201 (secure 0 0 0) 0x7fd0d4059720 con 0x7fd0e8072240 2024-04-20T21:23:11.364 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:23:11.364 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":42,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2740872549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2740872549},{"type":"v1","addr":"172.21.15.86:6836","nonce":2740872549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5048,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/301899138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":301899138},{"type":"v1","addr":"172.21.15.144:6833","nonce":301899138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5051,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2816343379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2816343379},{"type":"v1","addr":"172.21.15.144:6835","nonce":2816343379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:22:45.431100+0000","modified":"2024-04-20T21:22:48.492788+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5407":{"gid":5407,"name":"a","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/202143875","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":202143875},{"type":"v1","addr":"172.21.15.86:6837","nonce":202143875}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5407,"qdb_cluster":[5407]},"id":5}]} 2024-04-20T21:23:11.365 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 42 2024-04-20T21:23:11.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.368+0000 7fd0f0400640 1 -- 172.21.15.86:0/2635932296 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd0c806c8f0 msgr2=0x7fd0c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:11.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.368+0000 7fd0f0400640 1 --2- 172.21.15.86:0/2635932296 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd0c806c8f0 0x7fd0c806edb0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc0012b0 tx=0x7fd0dc009040 comp rx=0 tx=0).stop 2024-04-20T21:23:11.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.368+0000 7fd0f0400640 1 -- 172.21.15.86:0/2635932296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0e8072240 msgr2=0x7fd0e813a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:11.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.368+0000 7fd0f0400640 1 --2- 172.21.15.86:0/2635932296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0e8072240 0x7fd0e813a6b0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fd0d400c820 tx=0x7fd0d400cce0 comp rx=0 tx=0).stop 2024-04-20T21:23:11.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.368+0000 7fd0f0400640 1 -- 172.21.15.86:0/2635932296 shutdown_connections 2024-04-20T21:23:11.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.368+0000 7fd0f0400640 1 --2- 172.21.15.86:0/2635932296 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd0c806c8f0 0x7fd0c806edb0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:11.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.368+0000 7fd0f0400640 1 --2- 172.21.15.86:0/2635932296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0e8146130 0x7fd0e815bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:11.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.369+0000 7fd0f0400640 1 --2- 172.21.15.86:0/2635932296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd0e8072240 0x7fd0e813a6b0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:11.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.369+0000 7fd0f0400640 1 --2- 172.21.15.86:0/2635932296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0e806e6d0 0x7fd0e813b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:11.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.369+0000 7fd0f0400640 1 -- 172.21.15.86:0/2635932296 >> 172.21.15.86:0/2635932296 conn(0x7fd0e80b2360 msgr2=0x7fd0e80b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:11.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.369+0000 7fd0f0400640 1 -- 172.21.15.86:0/2635932296 shutdown_connections 2024-04-20T21:23:11.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.369+0000 7fd0f0400640 1 -- 172.21.15.86:0/2635932296 wait complete. 2024-04-20T21:23:11.387 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-20T21:23:11.618 INFO:teuthology.orchestra.run.smithi086.stdout:{"message":"","return_code":0} 2024-04-20T21:23:11.618 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-20T21:23:11.618 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-20T21:23:11.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.820+0000 7f96e9400640 1 Processor -- start 2024-04-20T21:23:11.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.821+0000 7f96e9400640 1 -- start start 2024-04-20T21:23:11.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.822+0000 7f96e9400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f96e4070030 0x7f96e406c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:11.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.822+0000 7f96e9400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96e406ced0 0x7f96e40add20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:11.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.822+0000 7f96e9400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96e415eea0 con 0x7f96e40ce940 2024-04-20T21:23:11.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.822+0000 7f96e9400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f96e40704c0 con 0x7f96e4070030 2024-04-20T21:23:11.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.822+0000 7f96e9400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f96e406f020 con 0x7f96e406ced0 2024-04-20T21:23:11.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.822+0000 7f96e1e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f96e4070030 0x7f96e406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:11.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.822+0000 7f96e1e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f96e4070030 0x7f96e406c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43876/0 (socket says 172.21.15.86:43876) 2024-04-20T21:23:11.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.822+0000 7f96e1e00640 1 -- 172.21.15.86:0/2185712275 learned_addr learned my addr 172.21.15.86:0/2185712275 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:11.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.823+0000 7f96e3200640 1 --2- 172.21.15.86:0/2185712275 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96e406ced0 0x7f96e40add20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:11.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.823+0000 7f96e1e00640 1 -- 172.21.15.86:0/2185712275 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96e406ced0 msgr2=0x7f96e40add20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:11.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.823+0000 7f96e1e00640 1 --2- 172.21.15.86:0/2185712275 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96e406ced0 0x7f96e40add20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:11.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.823+0000 7f96e1e00640 1 -- 172.21.15.86:0/2185712275 >> v1:172.21.15.144:6789/0 conn(0x7f96e40ce940 legacy=0x7f96e40ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:11.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.823+0000 7f96e1e00640 1 -- 172.21.15.86:0/2185712275 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96e406f270 con 0x7f96e4070030 2024-04-20T21:23:11.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.824+0000 7f96e1e00640 1 --2- 172.21.15.86:0/2185712275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96e4070030 0x7f96e406c990 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f96cc009e50 tx=0x7f96cc02bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80ead8c4f0d18614 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:11.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.824+0000 7f96e1400640 1 -- 172.21.15.86:0/2185712275 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96cc02c950 con 0x7f96e4070030 2024-04-20T21:23:11.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.824+0000 7f96e1400640 1 -- 172.21.15.86:0/2185712275 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96cc02caf0 con 0x7f96e4070030 2024-04-20T21:23:11.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.824+0000 7f96e1400640 1 -- 172.21.15.86:0/2185712275 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96cc0343f0 con 0x7f96e4070030 2024-04-20T21:23:11.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.825+0000 7f96e9400640 1 -- 172.21.15.86:0/2185712275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96e4070030 msgr2=0x7f96e406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:11.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.825+0000 7f96e9400640 1 --2- 172.21.15.86:0/2185712275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96e4070030 0x7f96e406c990 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f96cc009e50 tx=0x7f96cc02bdb0 comp rx=0 tx=0).stop 2024-04-20T21:23:11.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.825+0000 7f96e9400640 1 -- 172.21.15.86:0/2185712275 shutdown_connections 2024-04-20T21:23:11.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.825+0000 7f96e9400640 1 --2- 172.21.15.86:0/2185712275 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96e406ced0 0x7f96e40add20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:11.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.825+0000 7f96e9400640 1 --2- 172.21.15.86:0/2185712275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96e4070030 0x7f96e406c990 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:11.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.825+0000 7f96e9400640 1 -- 172.21.15.86:0/2185712275 >> 172.21.15.86:0/2185712275 conn(0x7f96e40cd7f0 msgr2=0x7f96e40cdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:11.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.825+0000 7f96e9400640 1 -- 172.21.15.86:0/2185712275 shutdown_connections 2024-04-20T21:23:11.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.825+0000 7f96e9400640 1 -- 172.21.15.86:0/2185712275 wait complete. 2024-04-20T21:23:11.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.827+0000 7f96e9400640 1 Processor -- start 2024-04-20T21:23:11.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.827+0000 7f96e9400640 1 -- start start 2024-04-20T21:23:11.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.828+0000 7f96e9400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96e406ced0 0x7f96e40c6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:11.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.828+0000 7f96e9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96e4070030 0x7f96e41ea3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:11.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.828+0000 7f96e9400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96e40ce940 0x7f96e40ca010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:11.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.828+0000 7f96e9400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f96e406f830 con 0x7f96e4070030 2024-04-20T21:23:11.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.828+0000 7f96e9400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f96e406fba0 con 0x7f96e40ce940 2024-04-20T21:23:11.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.828+0000 7f96e9400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f96e4067c40 con 0x7f96e406ced0 2024-04-20T21:23:11.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.828+0000 7f96e1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96e4070030 0x7f96e41ea3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:11.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.828+0000 7f96e1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96e4070030 0x7f96e41ea3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43888/0 (socket says 172.21.15.86:43888) 2024-04-20T21:23:11.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.828+0000 7f96e1e00640 1 -- 172.21.15.86:0/2299689090 learned_addr learned my addr 172.21.15.86:0/2299689090 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:11.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.829+0000 7f96e2800640 1 --2- 172.21.15.86:0/2299689090 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96e406ced0 0x7f96e40c6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:11.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.829+0000 7f96e3200640 1 --2- 172.21.15.86:0/2299689090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96e40ce940 0x7f96e40ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:11.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.829+0000 7f96e1e00640 1 -- 172.21.15.86:0/2299689090 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96e406ced0 msgr2=0x7f96e40c6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:11.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.829+0000 7f96e1e00640 1 --2- 172.21.15.86:0/2299689090 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96e406ced0 0x7f96e40c6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:11.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.829+0000 7f96e1e00640 1 -- 172.21.15.86:0/2299689090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96e40ce940 msgr2=0x7f96e40ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:11.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.829+0000 7f96e1e00640 1 --2- 172.21.15.86:0/2299689090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96e40ce940 0x7f96e40ca010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:11.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.829+0000 7f96e1e00640 1 -- 172.21.15.86:0/2299689090 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96e4148b20 con 0x7f96e4070030 2024-04-20T21:23:11.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.829+0000 7f96e1e00640 1 --2- 172.21.15.86:0/2299689090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96e4070030 0x7f96e41ea3e0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f96cc036040 tx=0x7f96cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:11.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.830+0000 7f96daa00640 1 -- 172.21.15.86:0/2299689090 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96cc043020 con 0x7f96e4070030 2024-04-20T21:23:11.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.830+0000 7f96daa00640 1 -- 172.21.15.86:0/2299689090 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96cc03e450 con 0x7f96e4070030 2024-04-20T21:23:11.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.830+0000 7f96e9400640 1 -- 172.21.15.86:0/2299689090 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96e4148e00 con 0x7f96e4070030 2024-04-20T21:23:11.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.830+0000 7f96daa00640 1 -- 172.21.15.86:0/2299689090 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96cc03e840 con 0x7f96e4070030 2024-04-20T21:23:11.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.830+0000 7f96e9400640 1 -- 172.21.15.86:0/2299689090 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96e41ea920 con 0x7f96e4070030 2024-04-20T21:23:11.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.831+0000 7f96e9400640 1 -- 172.21.15.86:0/2299689090 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96e40689e0 con 0x7f96e4070030 2024-04-20T21:23:11.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.832+0000 7f96daa00640 1 -- 172.21.15.86:0/2299689090 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f96cc046020 con 0x7f96e4070030 2024-04-20T21:23:11.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.833+0000 7f96daa00640 1 --2- 172.21.15.86:0/2299689090 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f96c006c8f0 0x7f96c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:11.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.833+0000 7f96daa00640 1 -- 172.21.15.86:0/2299689090 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5709+0+0 (secure 0 0 0) 0x7f96cc0ae450 con 0x7f96e4070030 2024-04-20T21:23:11.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.833+0000 7f96e2800640 1 --2- 172.21.15.86:0/2299689090 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f96c006c8f0 0x7f96c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:11.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.834+0000 7f96e2800640 1 --2- 172.21.15.86:0/2299689090 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f96c006c8f0 0x7f96c006edb0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f96d4006fd0 tx=0x7f96d4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:11.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:11.838+0000 7f96daa00640 1 -- 172.21.15.86:0/2299689090 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f96cc04d030 con 0x7f96e4070030 2024-04-20T21:23:12.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.078+0000 7f96e9400640 1 -- 172.21.15.86:0/2299689090 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f96e40b1510 con 0x7f96e4070030 2024-04-20T21:23:12.077 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-04-20T21:23:12.079+0000 7f5bb9c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T21:23:12.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.081+0000 7f96daa00640 1 -- 172.21.15.86:0/2299689090 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v43) ==== 161+0+0 (secure 0 0 0) 0x7f96cc07c870 con 0x7f96e4070030 2024-04-20T21:23:12.081 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-20T21:23:12.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.085+0000 7f96e9400640 1 -- 172.21.15.86:0/2299689090 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f96c006c8f0 msgr2=0x7f96c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:12.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.085+0000 7f96e9400640 1 --2- 172.21.15.86:0/2299689090 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f96c006c8f0 0x7f96c006edb0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f96d4006fd0 tx=0x7f96d4008040 comp rx=0 tx=0).stop 2024-04-20T21:23:12.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.085+0000 7f96e9400640 1 -- 172.21.15.86:0/2299689090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96e4070030 msgr2=0x7f96e41ea3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:12.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.085+0000 7f96e9400640 1 --2- 172.21.15.86:0/2299689090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96e4070030 0x7f96e41ea3e0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f96cc036040 tx=0x7f96cc004a40 comp rx=0 tx=0).stop 2024-04-20T21:23:12.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.085+0000 7f96e9400640 1 -- 172.21.15.86:0/2299689090 shutdown_connections 2024-04-20T21:23:12.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.085+0000 7f96e9400640 1 --2- 172.21.15.86:0/2299689090 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f96c006c8f0 0x7f96c006edb0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:12.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.085+0000 7f96e9400640 1 --2- 172.21.15.86:0/2299689090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96e40ce940 0x7f96e40ca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:12.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.085+0000 7f96e9400640 1 --2- 172.21.15.86:0/2299689090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96e4070030 0x7f96e41ea3e0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:12.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.085+0000 7f96e9400640 1 --2- 172.21.15.86:0/2299689090 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96e406ced0 0x7f96e40c6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:12.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.085+0000 7f96e9400640 1 -- 172.21.15.86:0/2299689090 >> 172.21.15.86:0/2299689090 conn(0x7f96e40cd7f0 msgr2=0x7f96e40ae100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:12.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.085+0000 7f96e9400640 1 -- 172.21.15.86:0/2299689090 shutdown_connections 2024-04-20T21:23:12.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.086+0000 7f96e9400640 1 -- 172.21.15.86:0/2299689090 wait complete. 2024-04-20T21:23:12.103 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-20T21:23:12.108 INFO:tasks.ceph.mds.a.smithi086.stderr: -3785> 2024-04-20T21:22:50.345+0000 7fae43a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T21:23:12.172 INFO:tasks.ceph.mds.a.smithi086.stdout:starting mds.a at 2024-04-20T21:23:12.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.190+0000 7f8cd2138f40 1 Processor -- start 2024-04-20T21:23:12.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.190+0000 7f8cd2138f40 1 -- start start 2024-04-20T21:23:12.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.191+0000 7f8cd2138f40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557bfee7c8e0 0x557bfef44b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:12.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.191+0000 7f8cd2138f40 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557bfef4dda0 0x557bfefe2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:12.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.191+0000 7f8cd2138f40 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557bfef53e20 0x557bfeeaaf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:12.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.191+0000 7f8cd2138f40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557bfef54880 con 0x557bfee7c8e0 2024-04-20T21:23:12.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.191+0000 7f8cd2138f40 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557bfee23cc0 con 0x557bfef53e20 2024-04-20T21:23:12.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.191+0000 7f8cd2138f40 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x557bfef54b20 con 0x557bfef4dda0 2024-04-20T21:23:12.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.191+0000 7f8ccf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557bfee7c8e0 0x557bfef44b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:12.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.191+0000 7f8ccf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557bfee7c8e0 0x557bfef44b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43928/0 (socket says 172.21.15.86:43928) 2024-04-20T21:23:12.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.191+0000 7f8ccf400640 1 -- 172.21.15.86:0/452009820 learned_addr learned my addr 172.21.15.86:0/452009820 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:12.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.191+0000 7f8ccea00640 1 --2- 172.21.15.86:0/452009820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557bfef4dda0 0x557bfefe2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:12.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.191+0000 7f8ccf400640 1 -- 172.21.15.86:0/452009820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557bfef4dda0 msgr2=0x557bfefe2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:12.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.191+0000 7f8ccf400640 1 --2- 172.21.15.86:0/452009820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557bfef4dda0 0x557bfefe2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:12.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.191+0000 7f8ccf400640 1 -- 172.21.15.86:0/452009820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557bfef53e20 msgr2=0x557bfeeaaf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:12.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.191+0000 7f8ccfe00640 1 --2- 172.21.15.86:0/452009820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557bfef53e20 0x557bfeeaaf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:12.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.192+0000 7f8ccf400640 1 --2- 172.21.15.86:0/452009820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557bfef53e20 0x557bfeeaaf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:12.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.192+0000 7f8ccf400640 1 -- 172.21.15.86:0/452009820 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bfeeaa960 con 0x557bfee7c8e0 2024-04-20T21:23:12.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.192+0000 7f8ccf400640 1 --2- 172.21.15.86:0/452009820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557bfee7c8e0 0x557bfef44b10 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f8cc000a4a0 tx=0x7f8cc000a960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58b85c097c64d8d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:12.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.192+0000 7f8ccd600640 1 -- 172.21.15.86:0/452009820 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc000b8d0 con 0x557bfee7c8e0 2024-04-20T21:23:12.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.192+0000 7f8ccd600640 1 -- 172.21.15.86:0/452009820 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cc000f070 con 0x557bfee7c8e0 2024-04-20T21:23:12.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.192+0000 7f8ccd600640 1 -- 172.21.15.86:0/452009820 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc000bc90 con 0x557bfee7c8e0 2024-04-20T21:23:12.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.193+0000 7f8cd2138f40 1 -- 172.21.15.86:0/452009820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557bfee7c8e0 msgr2=0x557bfef44b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:12.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.193+0000 7f8cd2138f40 1 --2- 172.21.15.86:0/452009820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557bfee7c8e0 0x557bfef44b10 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f8cc000a4a0 tx=0x7f8cc000a960 comp rx=0 tx=0).stop 2024-04-20T21:23:12.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.193+0000 7f8cd2138f40 1 -- 172.21.15.86:0/452009820 shutdown_connections 2024-04-20T21:23:12.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.193+0000 7f8cd2138f40 1 --2- 172.21.15.86:0/452009820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557bfef53e20 0x557bfeeaaf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:12.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.193+0000 7f8cd2138f40 1 --2- 172.21.15.86:0/452009820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557bfef4dda0 0x557bfefe2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:12.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.193+0000 7f8cd2138f40 1 --2- 172.21.15.86:0/452009820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557bfee7c8e0 0x557bfef44b10 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:12.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.193+0000 7f8cd2138f40 1 -- 172.21.15.86:0/452009820 >> 172.21.15.86:0/452009820 conn(0x557bfeeca250 msgr2=0x557bfeec6e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:12.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.193+0000 7f8cd2138f40 1 -- 172.21.15.86:0/452009820 shutdown_connections 2024-04-20T21:23:12.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.193+0000 7f8cd2138f40 1 -- 172.21.15.86:0/452009820 wait complete. 2024-04-20T21:23:12.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.194+0000 7f8cd2138f40 1 Processor -- start 2024-04-20T21:23:12.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.194+0000 7f8cd2138f40 1 -- start start 2024-04-20T21:23:12.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.195+0000 7f8cd2138f40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557bfee7c8e0 0x557bfef509a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:12.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.195+0000 7f8cd2138f40 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557bfef4dda0 0x557bfef4abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:12.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.195+0000 7f8cd2138f40 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557bfef53e20 0x557bfef4b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:12.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.195+0000 7f8cd2138f40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557bfee7d280 con 0x557bfee7c8e0 2024-04-20T21:23:12.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.195+0000 7f8cd2138f40 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557bfeeaa590 con 0x557bfef4dda0 2024-04-20T21:23:12.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.195+0000 7f8cd2138f40 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x557bfeeab4f0 con 0x557bfef53e20 2024-04-20T21:23:12.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.195+0000 7f8ccf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557bfee7c8e0 0x557bfef509a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:12.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.195+0000 7f8ccf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557bfee7c8e0 0x557bfef509a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43940/0 (socket says 172.21.15.86:43940) 2024-04-20T21:23:12.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.195+0000 7f8ccf400640 1 -- 172.21.15.86:0/306632836 learned_addr learned my addr 172.21.15.86:0/306632836 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:12.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.195+0000 7f8ccf400640 1 -- 172.21.15.86:0/306632836 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557bfef53e20 msgr2=0x557bfef4b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:12.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.195+0000 7f8ccf400640 1 --2- 172.21.15.86:0/306632836 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557bfef53e20 0x557bfef4b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:12.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.195+0000 7f8ccf400640 1 -- 172.21.15.86:0/306632836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557bfef4dda0 msgr2=0x557bfef4abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:12.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.195+0000 7f8ccf400640 1 --2- 172.21.15.86:0/306632836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557bfef4dda0 0x557bfef4abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:12.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.195+0000 7f8ccf400640 1 -- 172.21.15.86:0/306632836 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bfeeaf530 con 0x557bfee7c8e0 2024-04-20T21:23:12.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.196+0000 7f8ccf400640 1 --2- 172.21.15.86:0/306632836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557bfee7c8e0 0x557bfef509a0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0010000 tx=0x7f8cc0002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:12.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.196+0000 7f8cc7e00640 1 -- 172.21.15.86:0/306632836 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0015910 con 0x557bfee7c8e0 2024-04-20T21:23:12.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.196+0000 7f8cc7e00640 1 -- 172.21.15.86:0/306632836 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cc0024070 con 0x557bfee7c8e0 2024-04-20T21:23:12.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.196+0000 7f8cd2138f40 1 -- 172.21.15.86:0/306632836 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557bfeeab740 con 0x557bfee7c8e0 2024-04-20T21:23:12.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.196+0000 7f8cc7e00640 1 -- 172.21.15.86:0/306632836 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0015d10 con 0x557bfee7c8e0 2024-04-20T21:23:12.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.196+0000 7f8cd2138f40 1 -- 172.21.15.86:0/306632836 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bfef501a0 con 0x557bfee7c8e0 2024-04-20T21:23:12.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.198+0000 7f8cc7e00640 1 -- 172.21.15.86:0/306632836 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8cc0029460 con 0x557bfee7c8e0 2024-04-20T21:23:12.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.199+0000 7f8cc7e00640 1 --2- 172.21.15.86:0/306632836 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8cb006b2d0 0x7f8cb006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:12.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.199+0000 7f8cc7e00640 1 -- 172.21.15.86:0/306632836 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5795+0+0 (secure 0 0 0) 0x7f8cc008c790 con 0x557bfee7c8e0 2024-04-20T21:23:12.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.199+0000 7f8ccea00640 1 --2- 172.21.15.86:0/306632836 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8cb006b2d0 0x7f8cb006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:12.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.199+0000 7f8cd2138f40 1 --2- 172.21.15.86:0/306632836 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x557bfeead760 0x557bfef42950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:12.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.199+0000 7f8cd2138f40 1 -- 172.21.15.86:0/306632836 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:1 10.10 10:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72) -- 0x557bfef42e90 con 0x557bfeead760 2024-04-20T21:23:12.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.200+0000 7f8ccea00640 1 --2- 172.21.15.86:0/306632836 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8cb006b2d0 0x7f8cb006d790 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc057000 tx=0x7f8cbc04f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:12.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.200+0000 7f8ccfe00640 1 --2- 172.21.15.86:0/306632836 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x557bfeead760 0x557bfef42950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:12.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.201+0000 7f8ccfe00640 1 --2- 172.21.15.86:0/306632836 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x557bfeead760 0x557bfef42950 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:12.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.205+0000 7f8ccfe00640 1 -- 172.21.15.86:0/306632836 <== osd.4 v2:172.21.15.144:6800/4087309475 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v72'11 uv11 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f8cc800b470 con 0x557bfeead760 2024-04-20T21:23:12.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.205+0000 7f8cd2138f40 1 -- 172.21.15.86:0/306632836 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x557bfeead760 msgr2=0x557bfef42950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:12.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.205+0000 7f8cd2138f40 1 --2- 172.21.15.86:0/306632836 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x557bfeead760 0x557bfef42950 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:12.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.205+0000 7f8cd2138f40 1 -- 172.21.15.86:0/306632836 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8cb006b2d0 msgr2=0x7f8cb006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:12.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.205+0000 7f8cd2138f40 1 --2- 172.21.15.86:0/306632836 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8cb006b2d0 0x7f8cb006d790 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc057000 tx=0x7f8cbc04f000 comp rx=0 tx=0).stop 2024-04-20T21:23:12.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.205+0000 7f8cd2138f40 1 -- 172.21.15.86:0/306632836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557bfee7c8e0 msgr2=0x557bfef509a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:12.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.205+0000 7f8cd2138f40 1 --2- 172.21.15.86:0/306632836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557bfee7c8e0 0x557bfef509a0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0010000 tx=0x7f8cc0002fb0 comp rx=0 tx=0).stop 2024-04-20T21:23:12.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.205+0000 7f8ccfe00640 1 -- 172.21.15.86:0/306632836 reap_dead start 2024-04-20T21:23:12.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.205+0000 7f8cd2138f40 1 -- 172.21.15.86:0/306632836 shutdown_connections 2024-04-20T21:23:12.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.205+0000 7f8cd2138f40 1 -- 172.21.15.86:0/306632836 >> 172.21.15.86:0/306632836 conn(0x557bfeeca250 msgr2=0x557bfef44180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:12.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.205+0000 7f8cd2138f40 1 -- 172.21.15.86:0/306632836 shutdown_connections 2024-04-20T21:23:12.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.205+0000 7f8cd2138f40 1 -- 172.21.15.86:0/306632836 wait complete. 2024-04-20T21:23:12.208 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-20T21:23:12.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.390+0000 7fc010200640 1 Processor -- start 2024-04-20T21:23:12.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.390+0000 7fc010200640 1 -- start start 2024-04-20T21:23:12.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.392+0000 7fc010200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0080d08e0 0x7fc00815f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:12.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.392+0000 7fc010200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0080b2920 con 0x7fc008160430 2024-04-20T21:23:12.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.392+0000 7fc010200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0080b2c00 con 0x7fc00806c3d0 2024-04-20T21:23:12.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.392+0000 7fc010200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc00806b980 con 0x7fc0080d08e0 2024-04-20T21:23:12.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.392+0000 7fc00e200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fc008160430 0x7fc0081608c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:32952/0 (socket says 172.21.15.86:32952) 2024-04-20T21:23:12.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.392+0000 7fc00e200640 1 -- 172.21.15.86:0/2559354106 learned_addr learned my addr 172.21.15.86:0/2559354106 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:23:12.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.392+0000 7fc00d800640 1 --2- 172.21.15.86:0/2559354106 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0080d08e0 0x7fc00815f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:12.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.393+0000 7fc007e00640 1 -- 172.21.15.86:0/2559354106 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 868390210 0 0) 0x7fc0080b2920 con 0x7fc008160430 2024-04-20T21:23:12.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.393+0000 7fc007e00640 1 -- 172.21.15.86:0/2559354106 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbff0003520 con 0x7fc008160430 2024-04-20T21:23:12.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.393+0000 7fc007e00640 1 -- 172.21.15.86:0/2559354106 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 241180785 0 0) 0x7fbff0003520 con 0x7fc008160430 2024-04-20T21:23:12.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.393+0000 7fc007e00640 1 -- 172.21.15.86:0/2559354106 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0080d08e0 msgr2=0x7fc00815f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:12.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.393+0000 7fc007e00640 1 --2- 172.21.15.86:0/2559354106 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0080d08e0 0x7fc00815f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:12.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.393+0000 7fc007e00640 1 -- 172.21.15.86:0/2559354106 >> v1:172.21.15.144:6789/0 conn(0x7fc00806c3d0 legacy=0x7fc0080d0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:12.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.393+0000 7fc007e00640 1 -- 172.21.15.86:0/2559354106 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc00806bbd0 con 0x7fc008160430 2024-04-20T21:23:12.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.393+0000 7fc007e00640 1 -- 172.21.15.86:0/2559354106 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fc000001670 con 0x7fc008160430 2024-04-20T21:23:12.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.393+0000 7fc007e00640 1 -- 172.21.15.86:0/2559354106 >> v1:172.21.15.86:6789/0 conn(0x7fc008160430 legacy=0x7fc0081608c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:12.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.393+0000 7fc007e00640 1 --2- 172.21.15.86:0/2559354106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbff00039e0 0x7fbff0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:12.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.393+0000 7fc007e00640 1 --2- 172.21.15.86:0/2559354106 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0080d08e0 0x7fbff0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:12.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.393+0000 7fc007e00640 1 --2- 172.21.15.86:0/2559354106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbff0004d40 0x7fbff00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:12.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.393+0000 7fc007e00640 1 -- 172.21.15.86:0/2559354106 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc00806b980 con 0x7fbff0004d40 2024-04-20T21:23:12.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.393+0000 7fc007e00640 1 -- 172.21.15.86:0/2559354106 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbff000baa0 con 0x7fbff00039e0 2024-04-20T21:23:12.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.393+0000 7fc007e00640 1 -- 172.21.15.86:0/2559354106 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbff000bca0 con 0x7fc0080d08e0 2024-04-20T21:23:12.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.393+0000 7fc00e200640 1 --2- 172.21.15.86:0/2559354106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbff0004d40 0x7fbff00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:12.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.394+0000 7fc00e200640 1 -- 172.21.15.86:0/2559354106 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0080d08e0 msgr2=0x7fbff0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:12.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.394+0000 7fc00e200640 1 --2- 172.21.15.86:0/2559354106 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0080d08e0 0x7fbff0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:12.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.394+0000 7fc00e200640 1 -- 172.21.15.86:0/2559354106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbff00039e0 msgr2=0x7fbff0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:12.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.394+0000 7fc00e200640 1 --2- 172.21.15.86:0/2559354106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbff00039e0 0x7fbff0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:12.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.394+0000 7fc00e200640 1 -- 172.21.15.86:0/2559354106 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbff0003520 con 0x7fbff0004d40 2024-04-20T21:23:12.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.395+0000 7fc00e200640 1 --2- 172.21.15.86:0/2559354106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbff0004d40 0x7fbff00051c0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fc00000c0e0 tx=0x7fbff0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8b6f3f643bd178a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:12.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.395+0000 7fc007e00640 1 -- 172.21.15.86:0/2559354106 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc000007c40 con 0x7fbff0004d40 2024-04-20T21:23:12.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.395+0000 7fc007e00640 1 -- 172.21.15.86:0/2559354106 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc000030070 con 0x7fbff0004d40 2024-04-20T21:23:12.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.395+0000 7fc007e00640 1 -- 172.21.15.86:0/2559354106 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc000005d30 con 0x7fbff0004d40 2024-04-20T21:23:12.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.396+0000 7fc010200640 1 -- 172.21.15.86:0/2559354106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbff0004d40 msgr2=0x7fbff00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:12.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.396+0000 7fc010200640 1 --2- 172.21.15.86:0/2559354106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbff0004d40 0x7fbff00051c0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fc00000c0e0 tx=0x7fbff0005830 comp rx=0 tx=0).stop 2024-04-20T21:23:12.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.396+0000 7fc00e200640 1 -- 172.21.15.86:0/2559354106 reap_dead start 2024-04-20T21:23:12.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.396+0000 7fc010200640 1 -- 172.21.15.86:0/2559354106 shutdown_connections 2024-04-20T21:23:12.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.396+0000 7fc010200640 1 -- 172.21.15.86:0/2559354106 >> 172.21.15.86:0/2559354106 conn(0x7fc00809a9e0 msgr2=0x7fc0080b2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:12.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.396+0000 7fc010200640 1 -- 172.21.15.86:0/2559354106 shutdown_connections 2024-04-20T21:23:12.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.396+0000 7fc010200640 1 -- 172.21.15.86:0/2559354106 wait complete. 2024-04-20T21:23:12.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.398+0000 7fc010200640 1 Processor -- start 2024-04-20T21:23:12.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.398+0000 7fc010200640 1 -- start start 2024-04-20T21:23:12.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.399+0000 7fc010200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbff0004d40 0x7fc0080b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:12.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.399+0000 7fc010200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0080cd780 0x7fc0080cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:12.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.399+0000 7fc010200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0081f8190 0x7fc0081f8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:12.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.399+0000 7fc010200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc008160fd0 con 0x7fbff0004d40 2024-04-20T21:23:12.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.399+0000 7fc010200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc00806bf40 con 0x7fc0081f8190 2024-04-20T21:23:12.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.399+0000 7fc010200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc0080b3d30 con 0x7fc0080cd780 2024-04-20T21:23:12.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.399+0000 7fc00d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbff0004d40 0x7fc0080b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:12.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.399+0000 7fc00d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbff0004d40 0x7fc0080b6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43956/0 (socket says 172.21.15.86:43956) 2024-04-20T21:23:12.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.399+0000 7fc00d800640 1 -- 172.21.15.86:0/685601337 learned_addr learned my addr 172.21.15.86:0/685601337 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:12.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.400+0000 7fc00e200640 1 --2- 172.21.15.86:0/685601337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0081f8190 0x7fc0081f8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:12.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.400+0000 7fc00d800640 1 -- 172.21.15.86:0/685601337 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0080cd780 msgr2=0x7fc0080cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:12.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.400+0000 7fc00d800640 1 --2- 172.21.15.86:0/685601337 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0080cd780 0x7fc0080cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:12.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.400+0000 7fc00d800640 1 -- 172.21.15.86:0/685601337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0081f8190 msgr2=0x7fc0081f8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:12.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.400+0000 7fc00d800640 1 --2- 172.21.15.86:0/685601337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0081f8190 0x7fc0081f8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:12.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.400+0000 7fc00d800640 1 -- 172.21.15.86:0/685601337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0081f5d60 con 0x7fbff0004d40 2024-04-20T21:23:12.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.400+0000 7fc00d800640 1 --2- 172.21.15.86:0/685601337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbff0004d40 0x7fc0080b6d00 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fbffc00ef10 tx=0x7fbffc00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:12.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.400+0000 7fc005600640 1 -- 172.21.15.86:0/685601337 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc00bda0 con 0x7fbff0004d40 2024-04-20T21:23:12.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.400+0000 7fc005600640 1 -- 172.21.15.86:0/685601337 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbffc019070 con 0x7fbff0004d40 2024-04-20T21:23:12.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.400+0000 7fc005600640 1 -- 172.21.15.86:0/685601337 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc003bd0 con 0x7fbff0004d40 2024-04-20T21:23:12.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.400+0000 7fc010200640 1 -- 172.21.15.86:0/685601337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0080ce340 con 0x7fbff0004d40 2024-04-20T21:23:12.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.400+0000 7fc010200640 1 -- 172.21.15.86:0/685601337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0080c8d30 con 0x7fbff0004d40 2024-04-20T21:23:12.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.402+0000 7fc010200640 1 -- 172.21.15.86:0/685601337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0080690f0 con 0x7fbff0004d40 2024-04-20T21:23:12.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.403+0000 7fc005600640 1 -- 172.21.15.86:0/685601337 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbffc005d10 con 0x7fbff0004d40 2024-04-20T21:23:12.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.403+0000 7fc005600640 1 --2- 172.21.15.86:0/685601337 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbfe806c8f0 0x7fbfe806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:12.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.404+0000 7fc005600640 1 -- 172.21.15.86:0/685601337 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5795+0+0 (secure 0 0 0) 0x7fbffc08bee0 con 0x7fbff0004d40 2024-04-20T21:23:12.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.404+0000 7fc00ce00640 1 --2- 172.21.15.86:0/685601337 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbfe806c8f0 0x7fbfe806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:12.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.405+0000 7fc00ce00640 1 --2- 172.21.15.86:0/685601337 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbfe806c8f0 0x7fbfe806edb0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc0081f2790 tx=0x7fbff4007990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:12.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.408+0000 7fc005600640 1 -- 172.21.15.86:0/685601337 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbffc05a2a0 con 0x7fbff0004d40 2024-04-20T21:23:12.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:12.645+0000 7fc010200640 1 -- 172.21.15.86:0/685601337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fc0080d0f70 con 0x7fbff0004d40 2024-04-20T21:23:13.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.086+0000 7fc005600640 1 -- 172.21.15.86:0/685601337 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v44) ==== 162+0+0 (secure 0 0 0) 0x7fbffc059ba0 con 0x7fbff0004d40 2024-04-20T21:23:13.085 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-20T21:23:13.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.089+0000 7fc010200640 1 -- 172.21.15.86:0/685601337 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbfe806c8f0 msgr2=0x7fbfe806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:13.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.089+0000 7fc010200640 1 --2- 172.21.15.86:0/685601337 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbfe806c8f0 0x7fbfe806edb0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc0081f2790 tx=0x7fbff4007990 comp rx=0 tx=0).stop 2024-04-20T21:23:13.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.089+0000 7fc010200640 1 -- 172.21.15.86:0/685601337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbff0004d40 msgr2=0x7fc0080b6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:13.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.089+0000 7fc010200640 1 --2- 172.21.15.86:0/685601337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbff0004d40 0x7fc0080b6d00 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fbffc00ef10 tx=0x7fbffc00b480 comp rx=0 tx=0).stop 2024-04-20T21:23:13.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.089+0000 7fc010200640 1 -- 172.21.15.86:0/685601337 shutdown_connections 2024-04-20T21:23:13.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.089+0000 7fc010200640 1 --2- 172.21.15.86:0/685601337 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbfe806c8f0 0x7fbfe806edb0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:13.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.089+0000 7fc010200640 1 --2- 172.21.15.86:0/685601337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0081f8190 0x7fc0081f8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:13.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.089+0000 7fc010200640 1 --2- 172.21.15.86:0/685601337 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0080cd780 0x7fc0080cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:13.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.089+0000 7fc010200640 1 --2- 172.21.15.86:0/685601337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbff0004d40 0x7fc0080b6d00 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:13.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.089+0000 7fc010200640 1 -- 172.21.15.86:0/685601337 >> 172.21.15.86:0/685601337 conn(0x7fc00809a9e0 msgr2=0x7fc00815e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:13.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.090+0000 7fc010200640 1 -- 172.21.15.86:0/685601337 shutdown_connections 2024-04-20T21:23:13.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.090+0000 7fc010200640 1 -- 172.21.15.86:0/685601337 wait complete. 2024-04-20T21:23:13.117 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:23:13.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.299+0000 7f498e000640 1 Processor -- start 2024-04-20T21:23:13.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.299+0000 7f498e000640 1 -- start start 2024-04-20T21:23:13.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.300+0000 7f498e000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f498815c010 0x7f498815c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:13.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.300+0000 7f498e000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f49880af7b0 0x7f49880adda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:13.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.300+0000 7f498e000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49880ae710 con 0x7f498806f4c0 2024-04-20T21:23:13.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.300+0000 7f498e000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f49880af320 con 0x7f49880af7b0 2024-04-20T21:23:13.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.300+0000 7f498e000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f498815c930 con 0x7f498815c010 2024-04-20T21:23:13.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.301+0000 7f498cc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f498815c010 0x7f498815c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:13.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.301+0000 7f498cc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f498815c010 0x7f498815c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:44780/0 (socket says 172.21.15.86:44780) 2024-04-20T21:23:13.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.301+0000 7f498cc00640 1 -- 172.21.15.86:0/1754089192 learned_addr learned my addr 172.21.15.86:0/1754089192 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:13.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.301+0000 7f497fe00640 1 --2- 172.21.15.86:0/1754089192 >> v2:172.21.15.144:3300/0 conn(0x7f49880af7b0 0x7f49880adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:13.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.302+0000 7f497fe00640 1 -- 172.21.15.86:0/1754089192 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f498815c010 msgr2=0x7f498815c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:13.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.302+0000 7f497fe00640 1 --2- 172.21.15.86:0/1754089192 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f498815c010 0x7f498815c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:13.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.302+0000 7f497fe00640 1 -- 172.21.15.86:0/1754089192 >> v1:172.21.15.144:6789/0 conn(0x7f498806f4c0 legacy=0x7f498815ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:13.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.303+0000 7f497fe00640 1 -- 172.21.15.86:0/1754089192 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49880ae9f0 con 0x7f49880af7b0 2024-04-20T21:23:13.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.302+0000 7f4987e00640 1 -- 172.21.15.86:0/1754089192 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 595578 0 0) 0x7f49880ae710 con 0x7f498806f4c0 2024-04-20T21:23:13.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.303+0000 7f498cc00640 1 --2- 172.21.15.86:0/1754089192 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f498815c010 0x7f498815c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:23:13.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.304+0000 7f497fe00640 1 --2- 172.21.15.86:0/1754089192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49880af7b0 0x7f49880adda0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f497000bd90 tx=0x7f497002dae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2f885c98850213a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:13.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.305+0000 7f4987e00640 1 -- 172.21.15.86:0/1754089192 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497002f8a0 con 0x7f49880af7b0 2024-04-20T21:23:13.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.305+0000 7f4987e00640 1 -- 172.21.15.86:0/1754089192 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f497002fa40 con 0x7f49880af7b0 2024-04-20T21:23:13.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.305+0000 7f4987e00640 1 -- 172.21.15.86:0/1754089192 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497002fe10 con 0x7f49880af7b0 2024-04-20T21:23:13.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.305+0000 7f498e000640 1 -- 172.21.15.86:0/1754089192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49880af7b0 msgr2=0x7f49880adda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:13.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.305+0000 7f498e000640 1 --2- 172.21.15.86:0/1754089192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49880af7b0 0x7f49880adda0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f497000bd90 tx=0x7f497002dae0 comp rx=0 tx=0).stop 2024-04-20T21:23:13.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.305+0000 7f498e000640 1 -- 172.21.15.86:0/1754089192 shutdown_connections 2024-04-20T21:23:13.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.305+0000 7f498e000640 1 --2- 172.21.15.86:0/1754089192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49880af7b0 0x7f49880adda0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:13.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.305+0000 7f498e000640 1 --2- 172.21.15.86:0/1754089192 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f498815c010 0x7f498815c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:13.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.305+0000 7f498e000640 1 -- 172.21.15.86:0/1754089192 >> 172.21.15.86:0/1754089192 conn(0x7f49880d0170 msgr2=0x7f49880d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:13.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.306+0000 7f498e000640 1 -- 172.21.15.86:0/1754089192 shutdown_connections 2024-04-20T21:23:13.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.306+0000 7f498e000640 1 -- 172.21.15.86:0/1754089192 wait complete. 2024-04-20T21:23:13.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.307+0000 7f498e000640 1 Processor -- start 2024-04-20T21:23:13.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.307+0000 7f498e000640 1 -- start start 2024-04-20T21:23:13.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.308+0000 7f498e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f498806f4c0 0x7f4988154660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:13.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.308+0000 7f498e000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f49880af7b0 0x7f49881f3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:13.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.308+0000 7f498e000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f498815c010 0x7f498809ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:13.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.308+0000 7f498e000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f49880ae2e0 con 0x7f498806f4c0 2024-04-20T21:23:13.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.308+0000 7f498e000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f49880aefb0 con 0x7f498815c010 2024-04-20T21:23:13.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.308+0000 7f498e000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f49880ae460 con 0x7f49880af7b0 2024-04-20T21:23:13.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.309+0000 7f498cc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f498806f4c0 0x7f4988154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:13.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.309+0000 7f498cc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f498806f4c0 0x7f4988154660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43970/0 (socket says 172.21.15.86:43970) 2024-04-20T21:23:13.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.309+0000 7f498cc00640 1 -- 172.21.15.86:0/1746268289 learned_addr learned my addr 172.21.15.86:0/1746268289 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:13.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.309+0000 7f497fe00640 1 --2- 172.21.15.86:0/1746268289 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f49880af7b0 0x7f49881f3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:13.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.309+0000 7f498cc00640 1 -- 172.21.15.86:0/1746268289 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f49880af7b0 msgr2=0x7f49881f3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:13.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.309+0000 7f498cc00640 1 --2- 172.21.15.86:0/1746268289 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f49880af7b0 0x7f49881f3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:13.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.309+0000 7f498cc00640 1 -- 172.21.15.86:0/1746268289 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f498815c010 msgr2=0x7f498809ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:13.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.309+0000 7f498cc00640 1 --2- 172.21.15.86:0/1746268289 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f498815c010 0x7f498809ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:13.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.309+0000 7f498cc00640 1 -- 172.21.15.86:0/1746268289 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4988159320 con 0x7f498806f4c0 2024-04-20T21:23:13.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.309+0000 7f498cc00640 1 --2- 172.21.15.86:0/1746268289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f498806f4c0 0x7f4988154660 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f497800e9b0 tx=0x7f497800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:13.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.310+0000 7f4985600640 1 -- 172.21.15.86:0/1746268289 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4978017070 con 0x7f498806f4c0 2024-04-20T21:23:13.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.310+0000 7f4985600640 1 -- 172.21.15.86:0/1746268289 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f497800c9a0 con 0x7f498806f4c0 2024-04-20T21:23:13.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.310+0000 7f498e000640 1 -- 172.21.15.86:0/1746268289 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4988159600 con 0x7f498806f4c0 2024-04-20T21:23:13.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.310+0000 7f4985600640 1 -- 172.21.15.86:0/1746268289 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497800cd50 con 0x7f498806f4c0 2024-04-20T21:23:13.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.310+0000 7f498e000640 1 -- 172.21.15.86:0/1746268289 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f498809cd20 con 0x7f498806f4c0 2024-04-20T21:23:13.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.311+0000 7f498e000640 1 -- 172.21.15.86:0/1746268289 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f498804fcf0 con 0x7f498806f4c0 2024-04-20T21:23:13.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.312+0000 7f4985600640 1 -- 172.21.15.86:0/1746268289 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f497801a070 con 0x7f498806f4c0 2024-04-20T21:23:13.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.313+0000 7f4985600640 1 --2- 172.21.15.86:0/1746268289 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f496406c920 0x7f496406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:13.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.313+0000 7f4985600640 1 -- 172.21.15.86:0/1746268289 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5795+0+0 (secure 0 0 0) 0x7f497808bdd0 con 0x7f498806f4c0 2024-04-20T21:23:13.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.313+0000 7f497fe00640 1 --2- 172.21.15.86:0/1746268289 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f496406c920 0x7f496406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:13.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.314+0000 7f497fe00640 1 --2- 172.21.15.86:0/1746268289 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f496406c920 0x7f496406ede0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4970038000 tx=0x7f4970007040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:13.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.318+0000 7f4985600640 1 -- 172.21.15.86:0/1746268289 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f497805a190 con 0x7f498806f4c0 2024-04-20T21:23:13.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.557+0000 7f498e000640 1 -- 172.21.15.86:0/1746268289 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f498815dc60 con 0x7f498806f4c0 2024-04-20T21:23:13.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.558+0000 7f4985600640 1 -- 172.21.15.86:0/1746268289 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+5198 (secure 0 0 0) 0x7f4978059a90 con 0x7f498806f4c0 2024-04-20T21:23:13.557 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:23:13.558 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":45,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2740872549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2740872549},{"type":"v1","addr":"172.21.15.86:6836","nonce":2740872549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5048,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/301899138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":301899138},{"type":"v1","addr":"172.21.15.144:6833","nonce":301899138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5051,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2816343379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2816343379},{"type":"v1","addr":"172.21.15.144:6835","nonce":2816343379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:22:45.431100+0000","modified":"2024-04-20T21:23:13.090137+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6987},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6987":{"gid":6987,"name":"a","rank":0,"incarnation":45,"state":"up:replay","state_seq":1,"addr":"172.21.15.86:6837/2254117576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2254117576},{"type":"v1","addr":"172.21.15.86:6837","nonce":2254117576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5}]} 2024-04-20T21:23:13.558 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 45 2024-04-20T21:23:13.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.561+0000 7f498e000640 1 -- 172.21.15.86:0/1746268289 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f496406c920 msgr2=0x7f496406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:13.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.561+0000 7f498e000640 1 --2- 172.21.15.86:0/1746268289 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f496406c920 0x7f496406ede0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4970038000 tx=0x7f4970007040 comp rx=0 tx=0).stop 2024-04-20T21:23:13.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.561+0000 7f498e000640 1 -- 172.21.15.86:0/1746268289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f498806f4c0 msgr2=0x7f4988154660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:13.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.561+0000 7f498e000640 1 --2- 172.21.15.86:0/1746268289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f498806f4c0 0x7f4988154660 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f497800e9b0 tx=0x7f497800ee70 comp rx=0 tx=0).stop 2024-04-20T21:23:13.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.561+0000 7f498e000640 1 -- 172.21.15.86:0/1746268289 shutdown_connections 2024-04-20T21:23:13.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.561+0000 7f498e000640 1 --2- 172.21.15.86:0/1746268289 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f496406c920 0x7f496406ede0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:13.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.561+0000 7f498e000640 1 --2- 172.21.15.86:0/1746268289 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f498815c010 0x7f498809ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:13.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.561+0000 7f498e000640 1 --2- 172.21.15.86:0/1746268289 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f49880af7b0 0x7f49881f3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:13.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.561+0000 7f498e000640 1 --2- 172.21.15.86:0/1746268289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f498806f4c0 0x7f4988154660 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:13.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.561+0000 7f498e000640 1 -- 172.21.15.86:0/1746268289 >> 172.21.15.86:0/1746268289 conn(0x7f49880d0170 msgr2=0x7f4988071b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:13.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.561+0000 7f498e000640 1 -- 172.21.15.86:0/1746268289 shutdown_connections 2024-04-20T21:23:13.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:13.561+0000 7f498e000640 1 -- 172.21.15.86:0/1746268289 wait complete. 2024-04-20T21:23:13.981 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 45, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T21:22:45.431100+0000', 'modified': '2024-04-20T21:23:13.090137+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 72, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6987}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6987': {'gid': 6987, 'name': 'a', 'rank': 0, 'incarnation': 45, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.86:6837/2254117576', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6835', 'nonce': 2254117576}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 2254117576}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-20T21:23:13.981 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6987:up:replay 2024-04-20T21:23:14.981 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:23:15.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.192+0000 7fdf49200640 1 Processor -- start 2024-04-20T21:23:15.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.192+0000 7fdf49200640 1 -- start start 2024-04-20T21:23:15.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.194+0000 7fdf49200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdf44144e90 0x7fdf4415ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:15.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.194+0000 7fdf49200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf4406f140 0x7fdf440aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:15.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.194+0000 7fdf49200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf4406dc30 con 0x7fdf440af0b0 2024-04-20T21:23:15.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.194+0000 7fdf49200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fdf44145270 con 0x7fdf44144e90 2024-04-20T21:23:15.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.194+0000 7fdf49200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdf4415f720 con 0x7fdf4406f140 2024-04-20T21:23:15.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.194+0000 7fdf42800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fdf440af0b0 0x7fdf44071d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:32968/0 (socket says 172.21.15.86:32968) 2024-04-20T21:23:15.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.194+0000 7fdf42800640 1 -- 172.21.15.86:0/383795560 learned_addr learned my addr 172.21.15.86:0/383795560 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:23:15.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.194+0000 7fdf41e00640 1 --2- 172.21.15.86:0/383795560 >> v2:172.21.15.144:3300/0 conn(0x7fdf44144e90 0x7fdf4415ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:15.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.194+0000 7fdf41400640 1 --2- 172.21.15.86:0/383795560 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf4406f140 0x7fdf440aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:15.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.195+0000 7fdf40a00640 1 -- 172.21.15.86:0/383795560 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1578782354 0 0) 0x7fdf4406dc30 con 0x7fdf440af0b0 2024-04-20T21:23:15.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.195+0000 7fdf40a00640 1 -- 172.21.15.86:0/383795560 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf28003520 con 0x7fdf440af0b0 2024-04-20T21:23:15.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.195+0000 7fdf40a00640 1 -- 172.21.15.86:0/383795560 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2822706859 0 0) 0x7fdf28003520 con 0x7fdf440af0b0 2024-04-20T21:23:15.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.195+0000 7fdf40a00640 1 -- 172.21.15.86:0/383795560 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf4406f140 msgr2=0x7fdf440aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:15.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.195+0000 7fdf40a00640 1 --2- 172.21.15.86:0/383795560 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf4406f140 0x7fdf440aeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:15.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.195+0000 7fdf40a00640 1 -- 172.21.15.86:0/383795560 >> v2:172.21.15.144:3300/0 conn(0x7fdf44144e90 msgr2=0x7fdf4415ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:15.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.195+0000 7fdf40a00640 1 --2- 172.21.15.86:0/383795560 >> v2:172.21.15.144:3300/0 conn(0x7fdf44144e90 0x7fdf4415ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:15.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.195+0000 7fdf40a00640 1 -- 172.21.15.86:0/383795560 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf4415fa00 con 0x7fdf440af0b0 2024-04-20T21:23:15.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.195+0000 7fdf40a00640 1 -- 172.21.15.86:0/383795560 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fdf3c0019f0 con 0x7fdf440af0b0 2024-04-20T21:23:15.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.195+0000 7fdf40a00640 1 -- 172.21.15.86:0/383795560 >> v1:172.21.15.86:6789/0 conn(0x7fdf440af0b0 legacy=0x7fdf44071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:15.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.195+0000 7fdf40a00640 1 --2- 172.21.15.86:0/383795560 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf4406f140 0x7fdf280039d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:15.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.195+0000 7fdf40a00640 1 --2- 172.21.15.86:0/383795560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf28003fa0 0x7fdf28004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:15.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.195+0000 7fdf40a00640 1 --2- 172.21.15.86:0/383795560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf28004d40 0x7fdf280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:15.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.195+0000 7fdf40a00640 1 -- 172.21.15.86:0/383795560 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdf44145270 con 0x7fdf28003fa0 2024-04-20T21:23:15.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.195+0000 7fdf40a00640 1 -- 172.21.15.86:0/383795560 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4415f720 con 0x7fdf28004d40 2024-04-20T21:23:15.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.195+0000 7fdf40a00640 1 -- 172.21.15.86:0/383795560 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdf2800b9f0 con 0x7fdf4406f140 2024-04-20T21:23:15.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.195+0000 7fdf41400640 1 --2- 172.21.15.86:0/383795560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf28003fa0 0x7fdf28004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:15.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.196+0000 7fdf42800640 1 --2- 172.21.15.86:0/383795560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf28004d40 0x7fdf280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:15.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.196+0000 7fdf41e00640 1 --2- 172.21.15.86:0/383795560 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf4406f140 0x7fdf280039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:15.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.196+0000 7fdf41400640 1 -- 172.21.15.86:0/383795560 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf4406f140 msgr2=0x7fdf280039d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:15.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.196+0000 7fdf41400640 1 --2- 172.21.15.86:0/383795560 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf4406f140 0x7fdf280039d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:15.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.196+0000 7fdf41400640 1 -- 172.21.15.86:0/383795560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf28004d40 msgr2=0x7fdf280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:15.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.196+0000 7fdf41400640 1 --2- 172.21.15.86:0/383795560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf28004d40 0x7fdf280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:15.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.196+0000 7fdf41400640 1 -- 172.21.15.86:0/383795560 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf28003520 con 0x7fdf28003fa0 2024-04-20T21:23:15.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.196+0000 7fdf42800640 1 --2- 172.21.15.86:0/383795560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf28004d40 0x7fdf280051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:23:15.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.197+0000 7fdf41400640 1 --2- 172.21.15.86:0/383795560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf28003fa0 0x7fdf28004800 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fdf30009ac0 tx=0x7fdf28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9577f6ce97fb680d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:15.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.197+0000 7fdf40a00640 1 -- 172.21.15.86:0/383795560 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf30039070 con 0x7fdf28003fa0 2024-04-20T21:23:15.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.197+0000 7fdf40a00640 1 -- 172.21.15.86:0/383795560 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf3002c6b0 con 0x7fdf28003fa0 2024-04-20T21:23:15.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.197+0000 7fdf40a00640 1 -- 172.21.15.86:0/383795560 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf30009690 con 0x7fdf28003fa0 2024-04-20T21:23:15.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.198+0000 7fdf49200640 1 -- 172.21.15.86:0/383795560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf28003fa0 msgr2=0x7fdf28004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:15.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.198+0000 7fdf49200640 1 --2- 172.21.15.86:0/383795560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf28003fa0 0x7fdf28004800 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fdf30009ac0 tx=0x7fdf28005830 comp rx=0 tx=0).stop 2024-04-20T21:23:15.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.198+0000 7fdf42800640 1 -- 172.21.15.86:0/383795560 reap_dead start 2024-04-20T21:23:15.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.198+0000 7fdf49200640 1 -- 172.21.15.86:0/383795560 shutdown_connections 2024-04-20T21:23:15.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.198+0000 7fdf49200640 1 -- 172.21.15.86:0/383795560 >> 172.21.15.86:0/383795560 conn(0x7fdf440b2360 msgr2=0x7fdf440b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:15.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.198+0000 7fdf49200640 1 -- 172.21.15.86:0/383795560 shutdown_connections 2024-04-20T21:23:15.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.198+0000 7fdf49200640 1 -- 172.21.15.86:0/383795560 wait complete. 2024-04-20T21:23:15.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.200+0000 7fdf49200640 1 Processor -- start 2024-04-20T21:23:15.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.200+0000 7fdf49200640 1 -- start start 2024-04-20T21:23:15.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.201+0000 7fdf49200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf28003fa0 0x7fdf4415e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:15.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.201+0000 7fdf49200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf441f7ae0 0x7fdf440cd460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:15.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.201+0000 7fdf49200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf440b0ae0 0x7fdf440b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:15.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.201+0000 7fdf49200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4415f350 con 0x7fdf28003fa0 2024-04-20T21:23:15.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.201+0000 7fdf49200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4406ded0 con 0x7fdf441f7ae0 2024-04-20T21:23:15.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.201+0000 7fdf49200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdf4415f4d0 con 0x7fdf440b0ae0 2024-04-20T21:23:15.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.201+0000 7fdf41e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf28003fa0 0x7fdf4415e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:15.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.201+0000 7fdf41e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf28003fa0 0x7fdf4415e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43988/0 (socket says 172.21.15.86:43988) 2024-04-20T21:23:15.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.201+0000 7fdf41e00640 1 -- 172.21.15.86:0/1942579488 learned_addr learned my addr 172.21.15.86:0/1942579488 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:15.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.201+0000 7fdf41400640 1 --2- 172.21.15.86:0/1942579488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf441f7ae0 0x7fdf440cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:15.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.201+0000 7fdf41e00640 1 -- 172.21.15.86:0/1942579488 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf440b0ae0 msgr2=0x7fdf440b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:15.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.201+0000 7fdf41e00640 1 --2- 172.21.15.86:0/1942579488 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf440b0ae0 0x7fdf440b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:15.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.201+0000 7fdf41e00640 1 -- 172.21.15.86:0/1942579488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf441f7ae0 msgr2=0x7fdf440cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:15.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.201+0000 7fdf41e00640 1 --2- 172.21.15.86:0/1942579488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf441f7ae0 0x7fdf440cd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:15.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.201+0000 7fdf41e00640 1 -- 172.21.15.86:0/1942579488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf4415e650 con 0x7fdf28003fa0 2024-04-20T21:23:15.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.201+0000 7fdf41400640 1 --2- 172.21.15.86:0/1942579488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf441f7ae0 0x7fdf440cd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:23:15.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.202+0000 7fdf41e00640 1 --2- 172.21.15.86:0/1942579488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf28003fa0 0x7fdf4415e110 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fdf340108e0 tx=0x7fdf34010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:15.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.202+0000 7fdf3a000640 1 -- 172.21.15.86:0/1942579488 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3400f040 con 0x7fdf28003fa0 2024-04-20T21:23:15.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.202+0000 7fdf3a000640 1 -- 172.21.15.86:0/1942579488 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf3400eb60 con 0x7fdf28003fa0 2024-04-20T21:23:15.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.202+0000 7fdf3a000640 1 -- 172.21.15.86:0/1942579488 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3401c400 con 0x7fdf28003fa0 2024-04-20T21:23:15.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.202+0000 7fdf49200640 1 -- 172.21.15.86:0/1942579488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf440cd9a0 con 0x7fdf28003fa0 2024-04-20T21:23:15.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.202+0000 7fdf49200640 1 -- 172.21.15.86:0/1942579488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf441f8380 con 0x7fdf28003fa0 2024-04-20T21:23:15.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.203+0000 7fdf49200640 1 -- 172.21.15.86:0/1942579488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf440689e0 con 0x7fdf28003fa0 2024-04-20T21:23:15.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.204+0000 7fdf3a000640 1 -- 172.21.15.86:0/1942579488 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdf3401c5a0 con 0x7fdf28003fa0 2024-04-20T21:23:15.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.205+0000 7fdf3a000640 1 --2- 172.21.15.86:0/1942579488 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdf2406c920 0x7fdf2406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:15.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.205+0000 7fdf3a000640 1 -- 172.21.15.86:0/1942579488 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5795+0+0 (secure 0 0 0) 0x7fdf3408c310 con 0x7fdf28003fa0 2024-04-20T21:23:15.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.206+0000 7fdf41400640 1 --2- 172.21.15.86:0/1942579488 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdf2406c920 0x7fdf2406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:15.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.206+0000 7fdf41400640 1 --2- 172.21.15.86:0/1942579488 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdf2406c920 0x7fdf2406ede0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fdf30036000 tx=0x7fdf30030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:15.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.210+0000 7fdf3a000640 1 -- 172.21.15.86:0/1942579488 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fdf34018030 con 0x7fdf28003fa0 2024-04-20T21:23:15.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.455+0000 7fdf49200640 1 -- 172.21.15.86:0/1942579488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf440af710 con 0x7fdf28003fa0 2024-04-20T21:23:15.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.456+0000 7fdf3a000640 1 -- 172.21.15.86:0/1942579488 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5198 (secure 0 0 0) 0x7fdf3405a080 con 0x7fdf28003fa0 2024-04-20T21:23:15.456 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:23:15.456 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2740872549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2740872549},{"type":"v1","addr":"172.21.15.86:6836","nonce":2740872549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5048,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/301899138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":301899138},{"type":"v1","addr":"172.21.15.144:6833","nonce":301899138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5051,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2816343379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2816343379},{"type":"v1","addr":"172.21.15.144:6835","nonce":2816343379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:22:45.431100+0000","modified":"2024-04-20T21:23:14.102519+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6987},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6987":{"gid":6987,"name":"a","rank":0,"incarnation":45,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.86:6837/2254117576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2254117576},{"type":"v1","addr":"172.21.15.86:6837","nonce":2254117576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5}]} 2024-04-20T21:23:15.456 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 47 2024-04-20T21:23:15.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.459+0000 7fdf49200640 1 -- 172.21.15.86:0/1942579488 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdf2406c920 msgr2=0x7fdf2406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:15.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.459+0000 7fdf49200640 1 --2- 172.21.15.86:0/1942579488 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdf2406c920 0x7fdf2406ede0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fdf30036000 tx=0x7fdf30030000 comp rx=0 tx=0).stop 2024-04-20T21:23:15.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.459+0000 7fdf49200640 1 -- 172.21.15.86:0/1942579488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf28003fa0 msgr2=0x7fdf4415e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:15.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.459+0000 7fdf49200640 1 --2- 172.21.15.86:0/1942579488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf28003fa0 0x7fdf4415e110 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fdf340108e0 tx=0x7fdf34010da0 comp rx=0 tx=0).stop 2024-04-20T21:23:15.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.459+0000 7fdf49200640 1 -- 172.21.15.86:0/1942579488 shutdown_connections 2024-04-20T21:23:15.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.459+0000 7fdf49200640 1 --2- 172.21.15.86:0/1942579488 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdf2406c920 0x7fdf2406ede0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:15.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.459+0000 7fdf49200640 1 --2- 172.21.15.86:0/1942579488 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdf440b0ae0 0x7fdf440b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:15.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.459+0000 7fdf49200640 1 --2- 172.21.15.86:0/1942579488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf441f7ae0 0x7fdf440cd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:15.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.459+0000 7fdf49200640 1 --2- 172.21.15.86:0/1942579488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf28003fa0 0x7fdf4415e110 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:15.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.459+0000 7fdf49200640 1 -- 172.21.15.86:0/1942579488 >> 172.21.15.86:0/1942579488 conn(0x7fdf440b2360 msgr2=0x7fdf440b4b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:15.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.459+0000 7fdf49200640 1 -- 172.21.15.86:0/1942579488 shutdown_connections 2024-04-20T21:23:15.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:15.459+0000 7fdf49200640 1 -- 172.21.15.86:0/1942579488 wait complete. 2024-04-20T21:23:15.879 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 47, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T21:22:45.431100+0000', 'modified': '2024-04-20T21:23:14.102519+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 72, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6987}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6987': {'gid': 6987, 'name': 'a', 'rank': 0, 'incarnation': 45, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.86:6837/2254117576', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6835', 'nonce': 2254117576}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 2254117576}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-20T21:23:15.879 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6987:up:rejoin 2024-04-20T21:23:16.634 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:23:16.642 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:23:16.880 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:23:17.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.093+0000 7f784a000640 1 Processor -- start 2024-04-20T21:23:17.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.093+0000 7f784a000640 1 -- start start 2024-04-20T21:23:17.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.095+0000 7f784a000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7844072410 0x7f784415cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:17.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.095+0000 7f784a000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f784406eb30 con 0x7f784406e6d0 2024-04-20T21:23:17.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.095+0000 7f784a000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f784406dc30 con 0x7f7844146130 2024-04-20T21:23:17.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.095+0000 7f784a000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f78440cd8e0 con 0x7f7844072410 2024-04-20T21:23:17.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.095+0000 7f7842a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f784406e6d0 0x7f7844071d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52028/0 (socket says 172.21.15.86:52028) 2024-04-20T21:23:17.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.095+0000 7f7842a00640 1 -- 172.21.15.86:0/3426600227 learned_addr learned my addr 172.21.15.86:0/3426600227 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:23:17.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.096+0000 7f7842000640 1 -- 172.21.15.86:0/3426600227 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1577281450 0 0) 0x7f7830001110 con 0x7f784406e6d0 2024-04-20T21:23:17.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.096+0000 7f7843e00640 1 --2- 172.21.15.86:0/3426600227 >> v2:172.21.15.144:3300/0 conn(0x7f7844072410 0x7f784415cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:17.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.096+0000 7f7842000640 1 -- 172.21.15.86:0/3426600227 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7828003520 con 0x7f784406e6d0 2024-04-20T21:23:17.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.096+0000 7f7842000640 1 -- 172.21.15.86:0/3426600227 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3861129950 0 0) 0x7f7828003520 con 0x7f784406e6d0 2024-04-20T21:23:17.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.096+0000 7f7842000640 1 -- 172.21.15.86:0/3426600227 >> v2:172.21.15.144:3300/0 conn(0x7f7844072410 msgr2=0x7f784415cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:17.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.096+0000 7f7842000640 1 --2- 172.21.15.86:0/3426600227 >> v2:172.21.15.144:3300/0 conn(0x7f7844072410 0x7f784415cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:17.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.096+0000 7f7842000640 1 -- 172.21.15.86:0/3426600227 >> v1:172.21.15.144:6789/0 conn(0x7f7844146130 legacy=0x7f78440ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:17.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.096+0000 7f7842000640 1 -- 172.21.15.86:0/3426600227 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78440cdbc0 con 0x7f784406e6d0 2024-04-20T21:23:17.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.096+0000 7f7842000640 1 -- 172.21.15.86:0/3426600227 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f7830001840 con 0x7f784406e6d0 2024-04-20T21:23:17.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.096+0000 7f7842000640 1 -- 172.21.15.86:0/3426600227 >> v1:172.21.15.86:6789/0 conn(0x7f784406e6d0 legacy=0x7f7844071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:17.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.096+0000 7f7842000640 1 --2- 172.21.15.86:0/3426600227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78280039b0 0x7f7828003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:17.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.096+0000 7f7842000640 1 --2- 172.21.15.86:0/3426600227 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7828004800 0x7f7828004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:17.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.096+0000 7f7842000640 1 --2- 172.21.15.86:0/3426600227 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78280051c0 0x7f782800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:17.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.096+0000 7f7842000640 1 -- 172.21.15.86:0/3426600227 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f78440cd8e0 con 0x7f78280039b0 2024-04-20T21:23:17.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.096+0000 7f7842000640 1 -- 172.21.15.86:0/3426600227 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f782800c070 con 0x7f78280051c0 2024-04-20T21:23:17.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.096+0000 7f7842000640 1 -- 172.21.15.86:0/3426600227 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f782800c270 con 0x7f7828004800 2024-04-20T21:23:17.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.097+0000 7f7843400640 1 --2- 172.21.15.86:0/3426600227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78280039b0 0x7f7828003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:17.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.097+0000 7f7842a00640 1 --2- 172.21.15.86:0/3426600227 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7828004800 0x7f7828004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:17.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.097+0000 7f7843e00640 1 --2- 172.21.15.86:0/3426600227 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78280051c0 0x7f782800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:17.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.097+0000 7f7843400640 1 -- 172.21.15.86:0/3426600227 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7828004800 msgr2=0x7f7828004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:17.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.097+0000 7f7843400640 1 --2- 172.21.15.86:0/3426600227 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7828004800 0x7f7828004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:17.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.097+0000 7f7843400640 1 -- 172.21.15.86:0/3426600227 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78280051c0 msgr2=0x7f782800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:17.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.097+0000 7f7843400640 1 --2- 172.21.15.86:0/3426600227 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78280051c0 0x7f782800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:17.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.097+0000 7f7843e00640 1 -- 172.21.15.86:0/3426600227 reap_dead start 2024-04-20T21:23:17.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.097+0000 7f7843400640 1 -- 172.21.15.86:0/3426600227 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7828003520 con 0x7f78280039b0 2024-04-20T21:23:17.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.099+0000 7f7843400640 1 --2- 172.21.15.86:0/3426600227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78280039b0 0x7f7828003dd0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f7834001750 tx=0x7f7828005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e137e368b39b5f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:17.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.099+0000 7f7842000640 1 -- 172.21.15.86:0/3426600227 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7834001370 con 0x7f78280039b0 2024-04-20T21:23:17.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.099+0000 7f7842000640 1 -- 172.21.15.86:0/3426600227 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78340052d0 con 0x7f78280039b0 2024-04-20T21:23:17.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.099+0000 7f7842000640 1 -- 172.21.15.86:0/3426600227 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7834003830 con 0x7f78280039b0 2024-04-20T21:23:17.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.100+0000 7f784a000640 1 -- 172.21.15.86:0/3426600227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78280039b0 msgr2=0x7f7828003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:17.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.100+0000 7f784a000640 1 --2- 172.21.15.86:0/3426600227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78280039b0 0x7f7828003dd0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f7834001750 tx=0x7f7828005830 comp rx=0 tx=0).stop 2024-04-20T21:23:17.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.100+0000 7f784a000640 1 -- 172.21.15.86:0/3426600227 shutdown_connections 2024-04-20T21:23:17.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.100+0000 7f784a000640 1 --2- 172.21.15.86:0/3426600227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78280039b0 0x7f7828003dd0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:17.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.100+0000 7f784a000640 1 -- 172.21.15.86:0/3426600227 >> 172.21.15.86:0/3426600227 conn(0x7f78440b2360 msgr2=0x7f78440b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:17.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.100+0000 7f784a000640 1 -- 172.21.15.86:0/3426600227 shutdown_connections 2024-04-20T21:23:17.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.100+0000 7f784a000640 1 -- 172.21.15.86:0/3426600227 wait complete. 2024-04-20T21:23:17.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.102+0000 7f784a000640 1 Processor -- start 2024-04-20T21:23:17.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.102+0000 7f784a000640 1 -- start start 2024-04-20T21:23:17.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.103+0000 7f784a000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78280039b0 0x7f784413a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:17.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.103+0000 7f784a000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7844152970 0x7f78441f7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:17.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.103+0000 7f784a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7844152df0 0x7f78441f83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:17.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.103+0000 7f784a000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f784406f150 con 0x7f7844152df0 2024-04-20T21:23:17.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.103+0000 7f784a000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f784406e1b0 con 0x7f7844152970 2024-04-20T21:23:17.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.103+0000 7f784a000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7844071d00 con 0x7f78280039b0 2024-04-20T21:23:17.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.103+0000 7f7843e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7844152df0 0x7f78441f83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:17.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.103+0000 7f7843e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7844152df0 0x7f78441f83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52206/0 (socket says 172.21.15.86:52206) 2024-04-20T21:23:17.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.103+0000 7f7843e00640 1 -- 172.21.15.86:0/4294657576 learned_addr learned my addr 172.21.15.86:0/4294657576 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:17.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.103+0000 7f7843400640 1 --2- 172.21.15.86:0/4294657576 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78280039b0 0x7f784413a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:17.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.103+0000 7f7842a00640 1 --2- 172.21.15.86:0/4294657576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7844152970 0x7f78441f7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:17.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.104+0000 7f7843e00640 1 -- 172.21.15.86:0/4294657576 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78280039b0 msgr2=0x7f784413a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:17.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.104+0000 7f7843e00640 1 --2- 172.21.15.86:0/4294657576 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78280039b0 0x7f784413a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:17.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.104+0000 7f7843e00640 1 -- 172.21.15.86:0/4294657576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7844152970 msgr2=0x7f78441f7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:17.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.104+0000 7f7843e00640 1 --2- 172.21.15.86:0/4294657576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7844152970 0x7f78441f7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:17.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.104+0000 7f7843e00640 1 -- 172.21.15.86:0/4294657576 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7844143e80 con 0x7f7844152df0 2024-04-20T21:23:17.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.104+0000 7f7843400640 1 --2- 172.21.15.86:0/4294657576 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78280039b0 0x7f784413a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:23:17.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.104+0000 7f7843e00640 1 --2- 172.21.15.86:0/4294657576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7844152df0 0x7f78441f83c0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f78440b00c0 tx=0x7f783c00f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:17.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.104+0000 7f783b400640 1 -- 172.21.15.86:0/4294657576 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783c016420 con 0x7f7844152df0 2024-04-20T21:23:17.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.104+0000 7f783b400640 1 -- 172.21.15.86:0/4294657576 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f783c01b070 con 0x7f7844152df0 2024-04-20T21:23:17.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.104+0000 7f784a000640 1 -- 172.21.15.86:0/4294657576 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7844144120 con 0x7f7844152df0 2024-04-20T21:23:17.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.104+0000 7f783b400640 1 -- 172.21.15.86:0/4294657576 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783c016820 con 0x7f7844152df0 2024-04-20T21:23:17.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.105+0000 7f784a000640 1 -- 172.21.15.86:0/4294657576 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78441446e0 con 0x7f7844152df0 2024-04-20T21:23:17.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.106+0000 7f784a000640 1 -- 172.21.15.86:0/4294657576 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78440689e0 con 0x7f7844152df0 2024-04-20T21:23:17.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.107+0000 7f783b400640 1 -- 172.21.15.86:0/4294657576 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f783c0169c0 con 0x7f7844152df0 2024-04-20T21:23:17.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.108+0000 7f783b400640 1 --2- 172.21.15.86:0/4294657576 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f782406c920 0x7f782406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:17.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.108+0000 7f783b400640 1 -- 172.21.15.86:0/4294657576 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5795+0+0 (secure 0 0 0) 0x7f783c08cb00 con 0x7f7844152df0 2024-04-20T21:23:17.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.108+0000 7f7843400640 1 --2- 172.21.15.86:0/4294657576 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f782406c920 0x7f782406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:17.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.109+0000 7f7843400640 1 --2- 172.21.15.86:0/4294657576 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f782406c920 0x7f782406ede0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7834033000 tx=0x7f7834009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:17.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.112+0000 7f783b400640 1 -- 172.21.15.86:0/4294657576 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f783c025020 con 0x7f7844152df0 2024-04-20T21:23:17.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.353+0000 7f784a000640 1 -- 172.21.15.86:0/4294657576 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f784415ff50 con 0x7f7844152df0 2024-04-20T21:23:17.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.354+0000 7f783b400640 1 -- 172.21.15.86:0/4294657576 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+5205 (secure 0 0 0) 0x7f783c013330 con 0x7f7844152df0 2024-04-20T21:23:17.353 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:23:17.353 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":48,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2740872549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2740872549},{"type":"v1","addr":"172.21.15.86:6836","nonce":2740872549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5048,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/301899138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":301899138},{"type":"v1","addr":"172.21.15.144:6833","nonce":301899138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5051,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2816343379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2816343379},{"type":"v1","addr":"172.21.15.144:6835","nonce":2816343379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:22:45.431100+0000","modified":"2024-04-20T21:23:16.111215+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6987},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6987":{"gid":6987,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6837/2254117576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2254117576},{"type":"v1","addr":"172.21.15.86:6837","nonce":2254117576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6987,"qdb_cluster":[6987]},"id":5}]} 2024-04-20T21:23:17.353 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 48 2024-04-20T21:23:17.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.356+0000 7f784a000640 1 -- 172.21.15.86:0/4294657576 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f782406c920 msgr2=0x7f782406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:17.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.356+0000 7f784a000640 1 --2- 172.21.15.86:0/4294657576 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f782406c920 0x7f782406ede0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7834033000 tx=0x7f7834009030 comp rx=0 tx=0).stop 2024-04-20T21:23:17.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.356+0000 7f784a000640 1 -- 172.21.15.86:0/4294657576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7844152df0 msgr2=0x7f78441f83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:17.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.356+0000 7f784a000640 1 --2- 172.21.15.86:0/4294657576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7844152df0 0x7f78441f83c0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f78440b00c0 tx=0x7f783c00f520 comp rx=0 tx=0).stop 2024-04-20T21:23:17.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.357+0000 7f784a000640 1 -- 172.21.15.86:0/4294657576 shutdown_connections 2024-04-20T21:23:17.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.357+0000 7f784a000640 1 --2- 172.21.15.86:0/4294657576 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f782406c920 0x7f782406ede0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:17.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.357+0000 7f784a000640 1 --2- 172.21.15.86:0/4294657576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7844152df0 0x7f78441f83c0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:17.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.357+0000 7f784a000640 1 --2- 172.21.15.86:0/4294657576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7844152970 0x7f78441f7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:17.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.357+0000 7f784a000640 1 --2- 172.21.15.86:0/4294657576 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78280039b0 0x7f784413a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:17.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.357+0000 7f784a000640 1 -- 172.21.15.86:0/4294657576 >> 172.21.15.86:0/4294657576 conn(0x7f78440b2360 msgr2=0x7f784415f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:17.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.357+0000 7f784a000640 1 -- 172.21.15.86:0/4294657576 shutdown_connections 2024-04-20T21:23:17.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.357+0000 7f784a000640 1 -- 172.21.15.86:0/4294657576 wait complete. 2024-04-20T21:23:17.777 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 48, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T21:22:45.431100+0000', 'modified': '2024-04-20T21:23:16.111215+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 72, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6987}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6987': {'gid': 6987, 'name': 'a', 'rank': 0, 'incarnation': 45, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.86:6837/2254117576', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6835', 'nonce': 2254117576}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 2254117576}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6987, 'qdb_cluster': [6987]} 2024-04-20T21:23:17.778 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-20T21:23:17.778 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-20T21:23:17.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.980+0000 7f1744800640 1 Processor -- start 2024-04-20T21:23:17.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.981+0000 7f1744800640 1 -- start start 2024-04-20T21:23:17.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.982+0000 7f1744800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f173c1623d0 0x7f173c162810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:17.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.982+0000 7f1744800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f173c0adb30 0x7f173c0adfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:17.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.982+0000 7f1744800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f173c163250 con 0x7f173c15e740 2024-04-20T21:23:17.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.982+0000 7f1744800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f173c162d50 con 0x7f173c0adb30 2024-04-20T21:23:17.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.982+0000 7f1744800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f173c0b34b0 con 0x7f173c1623d0 2024-04-20T21:23:17.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.982+0000 7f1742800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f173c0adb30 0x7f173c0adfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:17.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.983+0000 7f1742800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f173c0adb30 0x7f173c0adfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52218/0 (socket says 172.21.15.86:52218) 2024-04-20T21:23:17.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.983+0000 7f1742800640 1 -- 172.21.15.86:0/718675461 learned_addr learned my addr 172.21.15.86:0/718675461 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:17.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.983+0000 7f1742800640 1 -- 172.21.15.86:0/718675461 >> v2:172.21.15.144:3300/0 conn(0x7f173c1623d0 msgr2=0x7f173c162810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:17.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.983+0000 7f1742800640 1 --2- 172.21.15.86:0/718675461 >> v2:172.21.15.144:3300/0 conn(0x7f173c1623d0 0x7f173c162810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:17.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.983+0000 7f1742800640 1 -- 172.21.15.86:0/718675461 >> v1:172.21.15.144:6789/0 conn(0x7f173c15e740 legacy=0x7f173c0d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:17.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.983+0000 7f1742800640 1 -- 172.21.15.86:0/718675461 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f173c0b3790 con 0x7f173c0adb30 2024-04-20T21:23:17.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.984+0000 7f1742800640 1 --2- 172.21.15.86:0/718675461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f173c0adb30 0x7f173c0adfa0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f173c06dbe0 tx=0x7f173802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1f915d608302951 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:17.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.984+0000 7f1740a00640 1 -- 172.21.15.86:0/718675461 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173802ca80 con 0x7f173c0adb30 2024-04-20T21:23:17.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.985+0000 7f1740a00640 1 -- 172.21.15.86:0/718675461 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f173802cc20 con 0x7f173c0adb30 2024-04-20T21:23:17.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.985+0000 7f1740a00640 1 -- 172.21.15.86:0/718675461 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17380343f0 con 0x7f173c0adb30 2024-04-20T21:23:17.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.985+0000 7f1744800640 1 -- 172.21.15.86:0/718675461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f173c0adb30 msgr2=0x7f173c0adfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:17.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.985+0000 7f1744800640 1 --2- 172.21.15.86:0/718675461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f173c0adb30 0x7f173c0adfa0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f173c06dbe0 tx=0x7f173802bee0 comp rx=0 tx=0).stop 2024-04-20T21:23:17.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.985+0000 7f1744800640 1 -- 172.21.15.86:0/718675461 shutdown_connections 2024-04-20T21:23:17.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.985+0000 7f1744800640 1 --2- 172.21.15.86:0/718675461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f173c0adb30 0x7f173c0adfa0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:17.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.985+0000 7f1744800640 1 --2- 172.21.15.86:0/718675461 >> v2:172.21.15.144:3300/0 conn(0x7f173c1623d0 0x7f173c162810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:17.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.985+0000 7f1744800640 1 -- 172.21.15.86:0/718675461 >> 172.21.15.86:0/718675461 conn(0x7f173c06b980 msgr2=0x7f173c06bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:17.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.985+0000 7f1744800640 1 -- 172.21.15.86:0/718675461 shutdown_connections 2024-04-20T21:23:17.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.985+0000 7f1744800640 1 -- 172.21.15.86:0/718675461 wait complete. 2024-04-20T21:23:17.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.987+0000 7f1744800640 1 Processor -- start 2024-04-20T21:23:17.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.987+0000 7f1744800640 1 -- start start 2024-04-20T21:23:17.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.988+0000 7f1744800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f173c0adb30 0x7f173c14cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:17.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.988+0000 7f1744800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f173c15e740 0x7f173c150550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:17.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.988+0000 7f1744800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f173c1623d0 0x7f173c0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:17.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.988+0000 7f1744800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f173c0b4030 con 0x7f173c0adb30 2024-04-20T21:23:17.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.988+0000 7f1744800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f173c0b3d50 con 0x7f173c15e740 2024-04-20T21:23:17.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.988+0000 7f1744800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f173c0d11b0 con 0x7f173c1623d0 2024-04-20T21:23:17.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.988+0000 7f1741e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f173c0adb30 0x7f173c14cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:17.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.989+0000 7f1741e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f173c0adb30 0x7f173c14cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52228/0 (socket says 172.21.15.86:52228) 2024-04-20T21:23:17.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.989+0000 7f1741e00640 1 -- 172.21.15.86:0/1469081820 learned_addr learned my addr 172.21.15.86:0/1469081820 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:17.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.989+0000 7f1741400640 1 --2- 172.21.15.86:0/1469081820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f173c15e740 0x7f173c150550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:17.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.989+0000 7f1742800640 1 --2- 172.21.15.86:0/1469081820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f173c1623d0 0x7f173c0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:17.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.989+0000 7f1741e00640 1 -- 172.21.15.86:0/1469081820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f173c1623d0 msgr2=0x7f173c0b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:17.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.989+0000 7f1741e00640 1 --2- 172.21.15.86:0/1469081820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f173c1623d0 0x7f173c0b2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:17.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.989+0000 7f1741e00640 1 -- 172.21.15.86:0/1469081820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f173c15e740 msgr2=0x7f173c150550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:17.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.989+0000 7f1741e00640 1 --2- 172.21.15.86:0/1469081820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f173c15e740 0x7f173c150550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:17.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.989+0000 7f1741e00640 1 -- 172.21.15.86:0/1469081820 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f173c1592c0 con 0x7f173c0adb30 2024-04-20T21:23:17.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.989+0000 7f1741e00640 1 --2- 172.21.15.86:0/1469081820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f173c0adb30 0x7f173c14cec0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f173000c430 tx=0x7f173000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:17.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.990+0000 7f1736000640 1 -- 172.21.15.86:0/1469081820 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1730011790 con 0x7f173c0adb30 2024-04-20T21:23:17.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.990+0000 7f1736000640 1 -- 172.21.15.86:0/1469081820 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1730017070 con 0x7f173c0adb30 2024-04-20T21:23:17.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.990+0000 7f1736000640 1 -- 172.21.15.86:0/1469081820 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1730011bb0 con 0x7f173c0adb30 2024-04-20T21:23:17.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.990+0000 7f1744800640 1 -- 172.21.15.86:0/1469081820 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f173c1595a0 con 0x7f173c0adb30 2024-04-20T21:23:17.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.990+0000 7f1744800640 1 -- 172.21.15.86:0/1469081820 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f173c14dc40 con 0x7f173c0adb30 2024-04-20T21:23:17.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.993+0000 7f1736000640 1 -- 172.21.15.86:0/1469081820 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f173001c460 con 0x7f173c0adb30 2024-04-20T21:23:17.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.994+0000 7f1736000640 1 --2- 172.21.15.86:0/1469081820 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f172006c920 0x7f172006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:17.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.994+0000 7f1736000640 1 -- 172.21.15.86:0/1469081820 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5795+0+0 (secure 0 0 0) 0x7f173008ba30 con 0x7f173c0adb30 2024-04-20T21:23:17.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.994+0000 7f1741400640 1 --2- 172.21.15.86:0/1469081820 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f172006c920 0x7f172006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:17.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.994+0000 7f174761a740 1 Processor -- start 2024-04-20T21:23:17.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.995+0000 7f1741400640 1 --2- 172.21.15.86:0/1469081820 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f172006c920 0x7f172006ede0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f173c14ff30 tx=0x7f172c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:17.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.995+0000 7f174761a740 1 -- start start 2024-04-20T21:23:17.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.996+0000 7f174761a740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fffb171460 0x55fffb128a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:17.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.996+0000 7f174761a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fffb1274e0 0x55fffb128fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:17.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.996+0000 7f174761a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fffb123930 0x55fffb153770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:17.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.996+0000 7f174761a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fffb08f9d0 con 0x55fffb123930 2024-04-20T21:23:17.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.996+0000 7f174761a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fffb036270 con 0x55fffb1274e0 2024-04-20T21:23:17.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.996+0000 7f174761a740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55fffb08b990 con 0x55fffb171460 2024-04-20T21:23:17.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.996+0000 7f1742800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fffb123930 0x55fffb153770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:17.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.996+0000 7f1742800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fffb123930 0x55fffb153770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52238/0 (socket says 172.21.15.86:52238) 2024-04-20T21:23:17.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.996+0000 7f1742800640 1 -- 172.21.15.86:0/1688651693 learned_addr learned my addr 172.21.15.86:0/1688651693 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:17.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.996+0000 7f1742800640 1 -- 172.21.15.86:0/1688651693 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fffb171460 msgr2=0x55fffb128a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:17.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.996+0000 7f1742800640 1 --2- 172.21.15.86:0/1688651693 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fffb171460 0x55fffb128a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:17.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.996+0000 7f1741400640 1 --2- 172.21.15.86:0/1688651693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fffb1274e0 0x55fffb128fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:17.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.996+0000 7f1742800640 1 -- 172.21.15.86:0/1688651693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fffb1274e0 msgr2=0x55fffb128fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:17.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.996+0000 7f1742800640 1 --2- 172.21.15.86:0/1688651693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fffb1274e0 0x55fffb128fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:17.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.997+0000 7f1742800640 1 -- 172.21.15.86:0/1688651693 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fffada22d0 con 0x55fffb123930 2024-04-20T21:23:17.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.997+0000 7f1742800640 1 --2- 172.21.15.86:0/1688651693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fffb123930 0x55fffb153770 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f1738036040 tx=0x7f173802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48d07116362043a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:17.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.997+0000 7f172a600640 1 -- 172.21.15.86:0/1688651693 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173802cb00 con 0x55fffb123930 2024-04-20T21:23:17.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.997+0000 7f172a600640 1 -- 172.21.15.86:0/1688651693 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f173803e430 con 0x55fffb123930 2024-04-20T21:23:17.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.997+0000 7f172a600640 1 -- 172.21.15.86:0/1688651693 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173803e820 con 0x55fffb123930 2024-04-20T21:23:17.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.998+0000 7f174761a740 1 -- 172.21.15.86:0/1688651693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fffb123930 msgr2=0x55fffb153770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:17.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.998+0000 7f174761a740 1 --2- 172.21.15.86:0/1688651693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fffb123930 0x55fffb153770 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f1738036040 tx=0x7f173802e000 comp rx=0 tx=0).stop 2024-04-20T21:23:17.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.998+0000 7f174761a740 1 -- 172.21.15.86:0/1688651693 shutdown_connections 2024-04-20T21:23:17.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.998+0000 7f174761a740 1 --2- 172.21.15.86:0/1688651693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fffb123930 0x55fffb153770 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:17.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.998+0000 7f174761a740 1 --2- 172.21.15.86:0/1688651693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fffb1274e0 0x55fffb128fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:17.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.998+0000 7f174761a740 1 --2- 172.21.15.86:0/1688651693 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fffb171460 0x55fffb128a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:17.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.998+0000 7f174761a740 1 -- 172.21.15.86:0/1688651693 >> 172.21.15.86:0/1688651693 conn(0x55fffb125bf0 msgr2=0x55fffb16e770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:17.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.998+0000 7f174761a740 1 -- 172.21.15.86:0/1688651693 shutdown_connections 2024-04-20T21:23:17.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.998+0000 7f174761a740 1 -- 172.21.15.86:0/1688651693 wait complete. 2024-04-20T21:23:17.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.999+0000 7f174761a740 1 -- start start 2024-04-20T21:23:18.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:17.999+0000 7f174761a740 1 Processor -- start 2024-04-20T21:23:18.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.000+0000 7f174761a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fffb1274e0 0x55fffb151e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:18.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.001+0000 7f174761a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fffb171460 0x55fffb172e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:18.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.001+0000 7f174761a740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fffb16fd50 0x55fffb1701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:18.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.001+0000 7f174761a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fffb1279f0 con 0x55fffb171460 2024-04-20T21:23:18.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.001+0000 7f174761a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fffb0495e0 con 0x55fffb1274e0 2024-04-20T21:23:18.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.001+0000 7f174761a740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55fffb127cd0 con 0x55fffb16fd50 2024-04-20T21:23:18.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.001+0000 7f174761a740 10 client.0 _ll_register_callbacks cb 0x55fffada1ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:23:18.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.001+0000 7f1741400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fffb171460 0x55fffb172e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:18.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.001+0000 7f1741400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fffb171460 0x55fffb172e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52252/0 (socket says 172.21.15.86:52252) 2024-04-20T21:23:18.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.001+0000 7f1741400640 1 -- 172.21.15.86:0/4264857057 learned_addr learned my addr 172.21.15.86:0/4264857057 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:18.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.001+0000 7f1741e00640 1 --2- 172.21.15.86:0/4264857057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fffb1274e0 0x55fffb151e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:18.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.001+0000 7f1742800640 1 --2- 172.21.15.86:0/4264857057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fffb16fd50 0x55fffb1701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:18.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.001+0000 7f1741400640 1 -- 172.21.15.86:0/4264857057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fffb16fd50 msgr2=0x55fffb1701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:18.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.001+0000 7f1741400640 1 --2- 172.21.15.86:0/4264857057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fffb16fd50 0x55fffb1701d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:18.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.001+0000 7f1741400640 1 -- 172.21.15.86:0/4264857057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fffb1274e0 msgr2=0x55fffb151e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:18.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.001+0000 7f1741400640 1 --2- 172.21.15.86:0/4264857057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fffb1274e0 0x55fffb151e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:18.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.001+0000 7f1741400640 1 -- 172.21.15.86:0/4264857057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fffae6c6b0 con 0x55fffb171460 2024-04-20T21:23:18.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.002+0000 7f1741400640 1 --2- 172.21.15.86:0/4264857057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fffb171460 0x55fffb172e40 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f172c00ddc0 tx=0x7f172c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d12029a9e8a7147 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:18.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.002+0000 7f171ea00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:23:18.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.002+0000 7f171ea00640 1 -- 172.21.15.86:0/4264857057 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c01f410 con 0x55fffb171460 2024-04-20T21:23:18.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.002+0000 7f171ea00640 1 -- 172.21.15.86:0/4264857057 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(72..72 src has 1..72) ==== 5795+0+0 (secure 0 0 0) 0x7f172c01f5b0 con 0x55fffb171460 2024-04-20T21:23:18.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.002+0000 7f174761a740 1 -- 172.21.15.86:0/4264857057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f173803e430 con 0x55fffb171460 2024-04-20T21:23:18.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.002+0000 7f171ea00640 1 -- 172.21.15.86:0/4264857057 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f172c005920 con 0x55fffb171460 2024-04-20T21:23:18.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.002+0000 7f174761a740 10 client.7029 fetch_fsmap learned FSMap version 48 2024-04-20T21:23:18.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.002+0000 7f174761a740 1 -- 172.21.15.86:0/4264857057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) -- 0x55fffb173d30 con 0x55fffb171460 2024-04-20T21:23:18.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.003+0000 7f171ea00640 1 -- 172.21.15.86:0/4264857057 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f173803e430 con 0x55fffb171460 2024-04-20T21:23:18.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.003+0000 7f171ea00640 1 -- 172.21.15.86:0/4264857057 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 48) ==== 3495+0+0 (secure 0 0 0) 0x7f172c013070 con 0x55fffb171460 2024-04-20T21:23:18.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.003+0000 7f174761a740 10 client.7029 fetch_fsmap finished waiting for FSMap version 48 2024-04-20T21:23:18.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.003+0000 7f174761a740 10 client.7029 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-20T21:23:18.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.003+0000 7f174761a740 20 client.7029 populate_metadata read hostname 'smithi086' 2024-04-20T21:23:18.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.003+0000 7f174761a740 1 --2- 172.21.15.86:0/4264857057 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x55fffb1390d0 0x55fffb137bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:18.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.003+0000 7f174761a740 4 client.7029 mds_command: new command op to 6987 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:23:18.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.003+0000 7f174761a740 1 -- 172.21.15.86:0/4264857057 --> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fffb09d3f0 con 0x55fffb1390d0 2024-04-20T21:23:18.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.003+0000 7f1741e00640 1 --2- 172.21.15.86:0/4264857057 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x55fffb1390d0 0x55fffb137bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:18.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.004+0000 7f1741e00640 1 --2- 172.21.15.86:0/4264857057 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x55fffb1390d0 0x55fffb137bf0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98032adb0fdadbc6 server_cookie=51667d71f9d2f30c in_seq=0 out_seq=0 2024-04-20T21:23:18.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.004+0000 7f171ea00640 10 client.7029 ms_handle_connect on v2:172.21.15.86:6835/2254117576 2024-04-20T21:23:18.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.005+0000 7f171ea00640 1 -- 172.21.15.86:0/4264857057 <== mds.0 v2:172.21.15.86:6835/2254117576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fffb09d3f0 con 0x55fffb1390d0 2024-04-20T21:23:18.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.005+0000 7f171ea00640 10 client.7029 handle_command_reply: tid=0 2024-04-20T21:23:18.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.005+0000 7f174761a740 1 client.7029 shutdown 2024-04-20T21:23:18.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.005+0000 7f174761a740 1 -- 172.21.15.86:0/4264857057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fffb171460 msgr2=0x55fffb172e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:18.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.005+0000 7f174761a740 1 --2- 172.21.15.86:0/4264857057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fffb171460 0x55fffb172e40 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f172c00ddc0 tx=0x7f172c0046c0 comp rx=0 tx=0).stop 2024-04-20T21:23:18.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.006+0000 7f174761a740 1 -- 172.21.15.86:0/4264857057 shutdown_connections 2024-04-20T21:23:18.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.006+0000 7f174761a740 1 --2- 172.21.15.86:0/4264857057 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x55fffb1390d0 0x55fffb137bf0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:18.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.006+0000 7f174761a740 1 --2- 172.21.15.86:0/4264857057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fffb16fd50 0x55fffb1701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:18.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.006+0000 7f174761a740 1 --2- 172.21.15.86:0/4264857057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fffb171460 0x55fffb172e40 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:18.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.006+0000 7f174761a740 1 --2- 172.21.15.86:0/4264857057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fffb1274e0 0x55fffb151e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:18.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.006+0000 7f171ea00640 0 client.7029 ms_handle_reset on v2:172.21.15.86:6835/2254117576 2024-04-20T21:23:18.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.006+0000 7f174761a740 1 -- 172.21.15.86:0/4264857057 >> 172.21.15.86:0/4264857057 conn(0x55fffb123930 msgr2=0x55fffb124710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:18.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.006+0000 7f174761a740 1 -- 172.21.15.86:0/4264857057 shutdown_connections 2024-04-20T21:23:18.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.006+0000 7f174761a740 1 -- 172.21.15.86:0/4264857057 wait complete. 2024-04-20T21:23:18.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.006+0000 7f174761a740 20 client.7029 trim_cache size 0 max 16384 2024-04-20T21:23:18.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.027+0000 7f174761a740 1 Processor -- start 2024-04-20T21:23:18.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.027+0000 7f174761a740 1 -- start start 2024-04-20T21:23:18.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.028+0000 7f174761a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fffb1274e0 0x55fffb176020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:18.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.028+0000 7f174761a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fffb16fd50 0x55fffb1289a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:18.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.028+0000 7f174761a740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fffb171460 0x55fffb122850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:18.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.028+0000 7f174761a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f173c200e50 con 0x55fffb16fd50 2024-04-20T21:23:18.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.028+0000 7f174761a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f173c1ffe80 con 0x55fffb1274e0 2024-04-20T21:23:18.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.028+0000 7f174761a740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f173c1f90c0 con 0x55fffb171460 2024-04-20T21:23:18.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.028+0000 7f1741400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fffb16fd50 0x55fffb1289a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:18.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.028+0000 7f1741400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fffb16fd50 0x55fffb1289a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52260/0 (socket says 172.21.15.86:52260) 2024-04-20T21:23:18.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.028+0000 7f1741400640 1 -- 172.21.15.86:0/1343044038 learned_addr learned my addr 172.21.15.86:0/1343044038 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:18.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.028+0000 7f1741400640 1 -- 172.21.15.86:0/1343044038 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fffb171460 msgr2=0x55fffb122850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:18.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.028+0000 7f1741e00640 1 --2- 172.21.15.86:0/1343044038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fffb1274e0 0x55fffb176020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:18.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.028+0000 7f1741400640 1 --2- 172.21.15.86:0/1343044038 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fffb171460 0x55fffb122850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:18.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.028+0000 7f1741400640 1 -- 172.21.15.86:0/1343044038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fffb1274e0 msgr2=0x55fffb176020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:18.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.029+0000 7f1741400640 1 --2- 172.21.15.86:0/1343044038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fffb1274e0 0x55fffb176020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:18.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.029+0000 7f1741400640 1 -- 172.21.15.86:0/1343044038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fffb14b3d0 con 0x55fffb16fd50 2024-04-20T21:23:18.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.029+0000 7f1741400640 1 --2- 172.21.15.86:0/1343044038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fffb16fd50 0x55fffb1289a0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f172c000c00 tx=0x7f172c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=244cf3955ee4509a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:18.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.029+0000 7f1729c00640 1 -- 172.21.15.86:0/1343044038 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c005bb0 con 0x55fffb16fd50 2024-04-20T21:23:18.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.029+0000 7f1729c00640 1 -- 172.21.15.86:0/1343044038 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f172c024070 con 0x55fffb16fd50 2024-04-20T21:23:18.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.029+0000 7f1729c00640 1 -- 172.21.15.86:0/1343044038 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c006240 con 0x55fffb16fd50 2024-04-20T21:23:18.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.030+0000 7f174761a740 1 -- 172.21.15.86:0/1343044038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fffb16fd50 msgr2=0x55fffb1289a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:18.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.030+0000 7f174761a740 1 --2- 172.21.15.86:0/1343044038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fffb16fd50 0x55fffb1289a0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f172c000c00 tx=0x7f172c01d000 comp rx=0 tx=0).stop 2024-04-20T21:23:18.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.030+0000 7f174761a740 1 -- 172.21.15.86:0/1343044038 shutdown_connections 2024-04-20T21:23:18.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.030+0000 7f174761a740 1 --2- 172.21.15.86:0/1343044038 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fffb171460 0x55fffb122850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:18.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.030+0000 7f174761a740 1 --2- 172.21.15.86:0/1343044038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fffb16fd50 0x55fffb1289a0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:18.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.030+0000 7f174761a740 1 --2- 172.21.15.86:0/1343044038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fffb1274e0 0x55fffb176020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:18.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.030+0000 7f174761a740 1 -- 172.21.15.86:0/1343044038 >> 172.21.15.86:0/1343044038 conn(0x55fffb123930 msgr2=0x55fffb1295a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:18.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.030+0000 7f174761a740 1 -- 172.21.15.86:0/1343044038 shutdown_connections 2024-04-20T21:23:18.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.030+0000 7f174761a740 1 -- 172.21.15.86:0/1343044038 wait complete. 2024-04-20T21:23:18.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.031+0000 7f174761a740 1 -- start start 2024-04-20T21:23:18.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.031+0000 7f174761a740 1 Processor -- start 2024-04-20T21:23:18.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.032+0000 7f174761a740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fffb1274e0 0x55fffb14bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:18.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.032+0000 7f174761a740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fffb16fd50 0x55fffb1f5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:18.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.032+0000 7f174761a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fffb171460 0x55fffb1814a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:18.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.032+0000 7f174761a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fffb138f50 con 0x55fffb171460 2024-04-20T21:23:18.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.032+0000 7f174761a740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f173c1ff830 con 0x55fffb1274e0 2024-04-20T21:23:18.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.032+0000 7f174761a740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55fffb139570 con 0x55fffb16fd50 2024-04-20T21:23:18.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.032+0000 7f174761a740 10 client.0 _ll_register_callbacks cb 0x7f172c00b4c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:23:18.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.032+0000 7f1742800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fffb171460 0x55fffb1814a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:18.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.032+0000 7f1742800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fffb171460 0x55fffb1814a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52274/0 (socket says 172.21.15.86:52274) 2024-04-20T21:23:18.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.032+0000 7f1742800640 1 -- 172.21.15.86:0/2694363027 learned_addr learned my addr 172.21.15.86:0/2694363027 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:18.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.032+0000 7f1741e00640 1 --2- 172.21.15.86:0/2694363027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fffb1274e0 0x55fffb14bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:18.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.032+0000 7f1741400640 1 --2- 172.21.15.86:0/2694363027 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fffb16fd50 0x55fffb1f5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:18.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.033+0000 7f1742800640 1 -- 172.21.15.86:0/2694363027 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fffb16fd50 msgr2=0x55fffb1f5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:18.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.033+0000 7f1742800640 1 --2- 172.21.15.86:0/2694363027 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fffb16fd50 0x55fffb1f5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:18.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.033+0000 7f1742800640 1 -- 172.21.15.86:0/2694363027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fffb1274e0 msgr2=0x55fffb14bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:18.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.033+0000 7f1742800640 1 --2- 172.21.15.86:0/2694363027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fffb1274e0 0x55fffb14bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:18.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.033+0000 7f1742800640 1 -- 172.21.15.86:0/2694363027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fffb123300 con 0x55fffb171460 2024-04-20T21:23:18.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.033+0000 7f1742800640 1 --2- 172.21.15.86:0/2694363027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fffb171460 0x55fffb1814a0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f1738000f80 tx=0x7f1738004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9a849afc85e2f19 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:18.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.033+0000 7f171ea00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:23:18.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.033+0000 7f171ea00640 1 -- 172.21.15.86:0/2694363027 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173804e020 con 0x55fffb171460 2024-04-20T21:23:18.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.033+0000 7f174761a740 1 -- 172.21.15.86:0/2694363027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f172c024070 con 0x55fffb171460 2024-04-20T21:23:18.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.033+0000 7f171ea00640 1 -- 172.21.15.86:0/2694363027 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(72..72 src has 1..72) ==== 5795+0+0 (secure 0 0 0) 0x7f1738004240 con 0x55fffb171460 2024-04-20T21:23:18.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.034+0000 7f171ea00640 1 -- 172.21.15.86:0/2694363027 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f1738058050 con 0x55fffb171460 2024-04-20T21:23:18.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.034+0000 7f174761a740 10 client.7035 fetch_fsmap learned FSMap version 48 2024-04-20T21:23:18.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.034+0000 7f174761a740 1 -- 172.21.15.86:0/2694363027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) -- 0x55fffb1f5ab0 con 0x55fffb171460 2024-04-20T21:23:18.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.034+0000 7f171ea00640 1 -- 172.21.15.86:0/2694363027 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f172c024070 con 0x55fffb171460 2024-04-20T21:23:18.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.034+0000 7f171ea00640 1 -- 172.21.15.86:0/2694363027 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 48) ==== 3495+0+0 (secure 0 0 0) 0x7f173805b020 con 0x55fffb171460 2024-04-20T21:23:18.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.034+0000 7f174761a740 10 client.7035 fetch_fsmap finished waiting for FSMap version 48 2024-04-20T21:23:18.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.034+0000 7f174761a740 10 client.7035 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-20T21:23:18.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.034+0000 7f174761a740 20 client.7035 populate_metadata read hostname 'smithi086' 2024-04-20T21:23:18.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.034+0000 7f174761a740 1 --2- 172.21.15.86:0/2694363027 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x55fffb181a50 0x55fffb123e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:18.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.034+0000 7f174761a740 4 client.7035 mds_command: new command op to 6987 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T21:23:18.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.034+0000 7f174761a740 1 -- 172.21.15.86:0/2694363027 --> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] -- command(tid 0: {"prefix": "status"}) -- 0x55fffb097260 con 0x55fffb181a50 2024-04-20T21:23:18.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.034+0000 7f1741e00640 1 --2- 172.21.15.86:0/2694363027 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x55fffb181a50 0x55fffb123e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:18.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.035+0000 7f1741e00640 1 --2- 172.21.15.86:0/2694363027 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x55fffb181a50 0x55fffb123e10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8366d68b1b857782 server_cookie=797f3ddf3b7619d in_seq=0 out_seq=0 2024-04-20T21:23:18.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.035+0000 7f171ea00640 10 client.7035 ms_handle_connect on v2:172.21.15.86:6835/2254117576 2024-04-20T21:23:18.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.035+0000 7f171ea00640 1 -- 172.21.15.86:0/2694363027 <== mds.0 v2:172.21.15.86:6835/2254117576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55fffb097260 con 0x55fffb181a50 2024-04-20T21:23:18.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.035+0000 7f171ea00640 10 client.7035 handle_command_reply: tid=0 2024-04-20T21:23:18.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.035+0000 7f174761a740 1 client.7035 shutdown 2024-04-20T21:23:18.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.035+0000 7f174761a740 1 -- 172.21.15.86:0/2694363027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fffb171460 msgr2=0x55fffb1814a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:18.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.035+0000 7f174761a740 1 --2- 172.21.15.86:0/2694363027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fffb171460 0x55fffb1814a0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f1738000f80 tx=0x7f1738004a40 comp rx=0 tx=0).stop 2024-04-20T21:23:18.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.035+0000 7f174761a740 1 -- 172.21.15.86:0/2694363027 shutdown_connections 2024-04-20T21:23:18.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.035+0000 7f174761a740 1 --2- 172.21.15.86:0/2694363027 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x55fffb181a50 0x55fffb123e10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:18.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.035+0000 7f174761a740 1 --2- 172.21.15.86:0/2694363027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fffb171460 0x55fffb1814a0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:18.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.035+0000 7f174761a740 1 --2- 172.21.15.86:0/2694363027 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fffb16fd50 0x55fffb1f5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:18.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.035+0000 7f174761a740 1 --2- 172.21.15.86:0/2694363027 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fffb1274e0 0x55fffb14bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:18.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.035+0000 7f171ea00640 0 client.7035 ms_handle_reset on v2:172.21.15.86:6835/2254117576 2024-04-20T21:23:18.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.035+0000 7f174761a740 1 -- 172.21.15.86:0/2694363027 >> 172.21.15.86:0/2694363027 conn(0x55fffb123930 msgr2=0x55fffb151d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:18.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.036+0000 7f174761a740 1 -- 172.21.15.86:0/2694363027 shutdown_connections 2024-04-20T21:23:18.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.036+0000 7f174761a740 1 -- 172.21.15.86:0/2694363027 wait complete. 2024-04-20T21:23:18.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.036+0000 7f174761a740 20 client.7035 trim_cache size 0 max 16384 2024-04-20T21:23:18.038 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:23:18.039 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "9c2dec1d-d7ae-4a83-a812-79922808e278", 2024-04-20T21:23:18.039 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-04-20T21:23:18.039 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 6987, 2024-04-20T21:23:18.039 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-20T21:23:18.039 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-20T21:23:18.039 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-20T21:23:18.039 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 4.9361218039999999, 2024-04-20T21:23:18.039 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 48, 2024-04-20T21:23:18.039 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 72, 2024-04-20T21:23:18.039 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 72, 2024-04-20T21:23:18.039 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 5.8586615350000004 2024-04-20T21:23:18.039 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:23:18.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.036+0000 7f1744800640 1 -- 172.21.15.86:0/1469081820 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f172006c920 msgr2=0x7f172006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:18.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.036+0000 7f1744800640 1 --2- 172.21.15.86:0/1469081820 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f172006c920 0x7f172006ede0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f173c14ff30 tx=0x7f172c008040 comp rx=0 tx=0).stop 2024-04-20T21:23:18.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.036+0000 7f1744800640 1 -- 172.21.15.86:0/1469081820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f173c0adb30 msgr2=0x7f173c14cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:18.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.036+0000 7f1744800640 1 --2- 172.21.15.86:0/1469081820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f173c0adb30 0x7f173c14cec0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f173000c430 tx=0x7f173000c8f0 comp rx=0 tx=0).stop 2024-04-20T21:23:18.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.036+0000 7f1744800640 1 -- 172.21.15.86:0/1469081820 shutdown_connections 2024-04-20T21:23:18.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.036+0000 7f1744800640 1 --2- 172.21.15.86:0/1469081820 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f172006c920 0x7f172006ede0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:18.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.036+0000 7f1744800640 1 --2- 172.21.15.86:0/1469081820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f173c1623d0 0x7f173c0b2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:18.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.036+0000 7f1744800640 1 --2- 172.21.15.86:0/1469081820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f173c15e740 0x7f173c150550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:18.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.036+0000 7f1744800640 1 --2- 172.21.15.86:0/1469081820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f173c0adb30 0x7f173c14cec0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:18.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.036+0000 7f1744800640 1 -- 172.21.15.86:0/1469081820 >> 172.21.15.86:0/1469081820 conn(0x7f173c06b980 msgr2=0x7f173c0cf6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:18.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.036+0000 7f1744800640 1 -- 172.21.15.86:0/1469081820 shutdown_connections 2024-04-20T21:23:18.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.037+0000 7f1744800640 1 -- 172.21.15.86:0/1469081820 wait complete. 2024-04-20T21:23:18.053 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T21:23:18.101 INFO:tasks.ceph.mds.a.smithi086.stderr:2024-04-20T21:23:18.102+0000 7fe375e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T21:23:18.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.249+0000 7ffb56400640 1 Processor -- start 2024-04-20T21:23:18.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.249+0000 7ffb56400640 1 -- start start 2024-04-20T21:23:18.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.250+0000 7ffb56400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ffb50146130 0x7ffb5006e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:18.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.250+0000 7ffb56400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb500ccdd0 0x7ffb500cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:18.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.250+0000 7ffb56400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ffb50071d00 0x7ffb50072190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:18.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.250+0000 7ffb56400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ffb5006dc30 con 0x7ffb50071d00 2024-04-20T21:23:18.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.250+0000 7ffb56400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7ffb500cd8e0 con 0x7ffb50146130 2024-04-20T21:23:18.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.250+0000 7ffb56400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ffb500cdbc0 con 0x7ffb500ccdd0 2024-04-20T21:23:18.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.251+0000 7ffb4fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ffb50071d00 0x7ffb50072190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:18.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.251+0000 7ffb4fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ffb50071d00 0x7ffb50072190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52284/0 (socket says 172.21.15.86:52284) 2024-04-20T21:23:18.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.251+0000 7ffb4fe00640 1 -- 172.21.15.86:0/283186634 learned_addr learned my addr 172.21.15.86:0/283186634 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:18.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.251+0000 7ffb4f400640 1 --2- 172.21.15.86:0/283186634 >> v2:172.21.15.144:3300/0 conn(0x7ffb50146130 0x7ffb5006e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:18.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.251+0000 7ffb46a00640 1 --2- 172.21.15.86:0/283186634 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb500ccdd0 0x7ffb500cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:18.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.251+0000 7ffb4fe00640 1 -- 172.21.15.86:0/283186634 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb500ccdd0 msgr2=0x7ffb500cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:18.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.251+0000 7ffb4fe00640 1 --2- 172.21.15.86:0/283186634 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb500ccdd0 0x7ffb500cd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:18.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.251+0000 7ffb4fe00640 1 -- 172.21.15.86:0/283186634 >> v2:172.21.15.144:3300/0 conn(0x7ffb50146130 msgr2=0x7ffb5006e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:18.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.251+0000 7ffb4fe00640 1 --2- 172.21.15.86:0/283186634 >> v2:172.21.15.144:3300/0 conn(0x7ffb50146130 0x7ffb5006e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:18.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.251+0000 7ffb4fe00640 1 -- 172.21.15.86:0/283186634 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb5006ded0 con 0x7ffb50071d00 2024-04-20T21:23:18.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.251+0000 7ffb4f400640 1 --2- 172.21.15.86:0/283186634 >> v2:172.21.15.144:3300/0 conn(0x7ffb50146130 0x7ffb5006e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:23:18.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.252+0000 7ffb4fe00640 1 --2- 172.21.15.86:0/283186634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb50071d00 0x7ffb50072190 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7ffb500b5950 tx=0x7ffb4802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65a632e4ee1795b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:18.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.253+0000 7ffb4ea00640 1 -- 172.21.15.86:0/283186634 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb48034070 con 0x7ffb50071d00 2024-04-20T21:23:18.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.253+0000 7ffb4ea00640 1 -- 172.21.15.86:0/283186634 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb48030860 con 0x7ffb50071d00 2024-04-20T21:23:18.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.253+0000 7ffb4ea00640 1 -- 172.21.15.86:0/283186634 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb48030c30 con 0x7ffb50071d00 2024-04-20T21:23:18.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.253+0000 7ffb56400640 1 -- 172.21.15.86:0/283186634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb50071d00 msgr2=0x7ffb50072190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:18.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.253+0000 7ffb56400640 1 --2- 172.21.15.86:0/283186634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb50071d00 0x7ffb50072190 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7ffb500b5950 tx=0x7ffb4802b9a0 comp rx=0 tx=0).stop 2024-04-20T21:23:18.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.253+0000 7ffb56400640 1 -- 172.21.15.86:0/283186634 shutdown_connections 2024-04-20T21:23:18.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.253+0000 7ffb56400640 1 --2- 172.21.15.86:0/283186634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb50071d00 0x7ffb50072190 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:18.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.253+0000 7ffb56400640 1 --2- 172.21.15.86:0/283186634 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb500ccdd0 0x7ffb500cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:18.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.253+0000 7ffb56400640 1 --2- 172.21.15.86:0/283186634 >> v2:172.21.15.144:3300/0 conn(0x7ffb50146130 0x7ffb5006e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:18.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.253+0000 7ffb56400640 1 -- 172.21.15.86:0/283186634 >> 172.21.15.86:0/283186634 conn(0x7ffb500b2360 msgr2=0x7ffb500b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:18.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.253+0000 7ffb56400640 1 -- 172.21.15.86:0/283186634 shutdown_connections 2024-04-20T21:23:18.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.253+0000 7ffb56400640 1 -- 172.21.15.86:0/283186634 wait complete. 2024-04-20T21:23:18.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.255+0000 7ffb56400640 1 Processor -- start 2024-04-20T21:23:18.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.255+0000 7ffb56400640 1 -- start start 2024-04-20T21:23:18.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.256+0000 7ffb56400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb50071d00 0x7ffb500cbec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:18.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.256+0000 7ffb56400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb500ccdd0 0x7ffb500af000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:18.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.256+0000 7ffb56400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb50146130 0x7ffb500af540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:18.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.256+0000 7ffb56400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffb5015cc20 con 0x7ffb50071d00 2024-04-20T21:23:18.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.256+0000 7ffb56400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ffb5006f150 con 0x7ffb500ccdd0 2024-04-20T21:23:18.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.256+0000 7ffb56400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ffb5015cda0 con 0x7ffb50146130 2024-04-20T21:23:18.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.257+0000 7ffb4f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb50071d00 0x7ffb500cbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:18.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.257+0000 7ffb4f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb50071d00 0x7ffb500cbec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52290/0 (socket says 172.21.15.86:52290) 2024-04-20T21:23:18.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.257+0000 7ffb4f400640 1 -- 172.21.15.86:0/3741468389 learned_addr learned my addr 172.21.15.86:0/3741468389 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:18.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.257+0000 7ffb4fe00640 1 --2- 172.21.15.86:0/3741468389 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb50146130 0x7ffb500af540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:18.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.257+0000 7ffb46a00640 1 --2- 172.21.15.86:0/3741468389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb500ccdd0 0x7ffb500af000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:18.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.257+0000 7ffb4f400640 1 -- 172.21.15.86:0/3741468389 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb50146130 msgr2=0x7ffb500af540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:18.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.257+0000 7ffb4f400640 1 --2- 172.21.15.86:0/3741468389 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb50146130 0x7ffb500af540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:18.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.257+0000 7ffb4f400640 1 -- 172.21.15.86:0/3741468389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb500ccdd0 msgr2=0x7ffb500af000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:18.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.257+0000 7ffb4f400640 1 --2- 172.21.15.86:0/3741468389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb500ccdd0 0x7ffb500af000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:18.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.257+0000 7ffb4f400640 1 -- 172.21.15.86:0/3741468389 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb50131d70 con 0x7ffb50071d00 2024-04-20T21:23:18.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.257+0000 7ffb4f400640 1 --2- 172.21.15.86:0/3741468389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb50071d00 0x7ffb500cbec0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7ffb4000c7f0 tx=0x7ffb4000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:18.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.258+0000 7ffb47e00640 1 -- 172.21.15.86:0/3741468389 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb40015070 con 0x7ffb50071d00 2024-04-20T21:23:18.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.258+0000 7ffb47e00640 1 -- 172.21.15.86:0/3741468389 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb400077c0 con 0x7ffb50071d00 2024-04-20T21:23:18.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.258+0000 7ffb56400640 1 -- 172.21.15.86:0/3741468389 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb50132030 con 0x7ffb50071d00 2024-04-20T21:23:18.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.258+0000 7ffb47e00640 1 -- 172.21.15.86:0/3741468389 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb40007b70 con 0x7ffb50071d00 2024-04-20T21:23:18.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.258+0000 7ffb56400640 1 -- 172.21.15.86:0/3741468389 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb501325f0 con 0x7ffb50071d00 2024-04-20T21:23:18.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.259+0000 7ffb56400640 1 -- 172.21.15.86:0/3741468389 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb500689e0 con 0x7ffb50071d00 2024-04-20T21:23:18.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.260+0000 7ffb47e00640 1 -- 172.21.15.86:0/3741468389 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffb40018070 con 0x7ffb50071d00 2024-04-20T21:23:18.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.261+0000 7ffb47e00640 1 --2- 172.21.15.86:0/3741468389 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ffb2c06c8f0 0x7ffb2c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:18.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.261+0000 7ffb47e00640 1 -- 172.21.15.86:0/3741468389 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5795+0+0 (secure 0 0 0) 0x7ffb4008bc80 con 0x7ffb50071d00 2024-04-20T21:23:18.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.262+0000 7ffb46a00640 1 --2- 172.21.15.86:0/3741468389 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ffb2c06c8f0 0x7ffb2c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:18.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.262+0000 7ffb46a00640 1 --2- 172.21.15.86:0/3741468389 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ffb2c06c8f0 0x7ffb2c06edb0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ffb5015fcf0 tx=0x7ffb380039f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:18.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.266+0000 7ffb47e00640 1 -- 172.21.15.86:0/3741468389 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ffb4005a050 con 0x7ffb50071d00 2024-04-20T21:23:18.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.506+0000 7ffb56400640 1 -- 172.21.15.86:0/3741468389 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ffb5015ff50 con 0x7ffb50071d00 2024-04-20T21:23:18.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.506+0000 7ffb47e00640 1 -- 172.21.15.86:0/3741468389 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v48) ==== 53+0+73 (secure 0 0 0) 0x7ffb40059950 con 0x7ffb50071d00 2024-04-20T21:23:18.505 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T21:23:18.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.509+0000 7ffb56400640 1 -- 172.21.15.86:0/3741468389 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ffb2c06c8f0 msgr2=0x7ffb2c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:18.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.509+0000 7ffb56400640 1 --2- 172.21.15.86:0/3741468389 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ffb2c06c8f0 0x7ffb2c06edb0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ffb5015fcf0 tx=0x7ffb380039f0 comp rx=0 tx=0).stop 2024-04-20T21:23:18.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.509+0000 7ffb56400640 1 -- 172.21.15.86:0/3741468389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb50071d00 msgr2=0x7ffb500cbec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:18.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.509+0000 7ffb56400640 1 --2- 172.21.15.86:0/3741468389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb50071d00 0x7ffb500cbec0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7ffb4000c7f0 tx=0x7ffb4000ccb0 comp rx=0 tx=0).stop 2024-04-20T21:23:18.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.509+0000 7ffb56400640 1 -- 172.21.15.86:0/3741468389 shutdown_connections 2024-04-20T21:23:18.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.509+0000 7ffb56400640 1 --2- 172.21.15.86:0/3741468389 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ffb2c06c8f0 0x7ffb2c06edb0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:18.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.509+0000 7ffb56400640 1 --2- 172.21.15.86:0/3741468389 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb50146130 0x7ffb500af540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:18.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.509+0000 7ffb56400640 1 --2- 172.21.15.86:0/3741468389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb500ccdd0 0x7ffb500af000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:18.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.509+0000 7ffb56400640 1 --2- 172.21.15.86:0/3741468389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb50071d00 0x7ffb500cbec0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:18.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.509+0000 7ffb56400640 1 -- 172.21.15.86:0/3741468389 >> 172.21.15.86:0/3741468389 conn(0x7ffb500b2360 msgr2=0x7ffb50072510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:18.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.509+0000 7ffb56400640 1 -- 172.21.15.86:0/3741468389 shutdown_connections 2024-04-20T21:23:18.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18.509+0000 7ffb56400640 1 -- 172.21.15.86:0/3741468389 wait complete. 2024-04-20T21:23:18.527 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T21:23:18.527 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T21:23:18.527 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-04-20T21:23:18.527 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-20T21:23:18.527 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T21:23:18.527 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T21:23:18.527 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T21:23:18.527 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T21:23:18.527 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:23:18.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:23:18.528 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-04-20T21:23:18.582 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T21:23:18.582 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T21:23:18.582 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T21:23:18.582 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:23:18.582 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-20T21:23:18.582 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:23:18.582 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:23:18.583 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:23:18.583 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-04-20T21:23:18.583 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:23:18.583 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:23:18.583 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-04-20T21:23:18.583 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:23:18.583 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:23:18.583 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T21:23:18.583 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:23:18.583 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T21:23:18.583 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42357sec preferred_lft 42357sec 2024-04-20T21:23:18.584 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-04-20T21:23:18.584 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:23:18.584 INFO:teuthology.orchestra.run.smithi086.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T21:23:18.584 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 86:25:97:2c:9c:bb brd ff:ff:ff:ff:ff:ff 2024-04-20T21:23:18.584 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-20T21:23:18.584 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:23:18.584 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::a4fb:30ff:fe90:2d6a/64 scope link 2024-04-20T21:23:18.584 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:23:18.584 INFO:teuthology.orchestra.run.smithi086.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T21:23:18.584 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 86:25:97:2c:9c:bb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:23:18.584 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::8425:97ff:fe2c:9cbb/64 scope link 2024-04-20T21:23:18.584 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:23:18.584 INFO:teuthology.orchestra.run.smithi086.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T21:23:18.584 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c2:9c:8b:55:ca:84 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:23:18.585 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c09c:8bff:fe55:ca84/64 scope link 2024-04-20T21:23:18.585 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:23:18.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:23:18.585 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T21:23:18.659 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-20T21:23:18.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:23:18.663 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-04-20T21:23:18.713 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-04-20T21:23:18.714 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T21:23:18.714 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T21:23:18.714 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T21:23:18.714 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T21:23:18.714 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:23:18.714 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-20T21:23:18.714 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T21:23:18.714 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T21:23:18.714 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T21:23:18.714 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-20T21:23:18.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:23:18.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:23:18.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:23:18.919 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-20T21:23:18.966 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-20T21:23:18.966 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-20T21:23:18.967 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:23:18.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:23:18.967 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-04-20T21:23:19.097 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-20T21:23:19.097 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T21:23:19.097 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-20T21:23:19.152 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:23:19.152 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T21:23:19.153 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-20T21:23:19.202 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-04-20T21:23:19.282 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T21:23:19.335 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-04-20T21:23:19.335 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-04-20T21:23:19.335 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-04-20T21:23:19.335 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-04-20T21:23:19.335 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-04-20T21:23:19.335 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-04-20T21:23:19.335 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-04-20T21:23:19.335 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-04-20T21:23:19.335 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-04-20T21:23:19.335 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-04-20T21:23:19.336 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-04-20T21:23:19.336 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/selinux 2024-04-20T21:23:19.336 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T21:23:19.336 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-04-20T21:23:19.336 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-04-20T21:23:19.336 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-04-20T21:23:19.336 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-04-20T21:23:19.336 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-04-20T21:23:19.336 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-04-20T21:23:19.336 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T21:23:19.336 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T21:23:19.336 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T21:23:19.337 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T21:23:19.337 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-04-20T21:23:19.337 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-04-20T21:23:19.337 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T21:23:19.337 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T21:23:19.337 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T21:23:19.337 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T21:23:19.337 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-04-20T21:23:19.337 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:23:19.337 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:23:19.337 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:23:19.337 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:23:19.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:23:19.339 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-04-20T21:23:19.393 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T21:23:19.393 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-20T21:23:19.401 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-04-20T21:23:19.470 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T21:23:19.511 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-04-20T21:23:19.511 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-04-20T21:23:19.511 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-04-20T21:23:19.511 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-04-20T21:23:19.511 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-04-20T21:23:19.511 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-04-20T21:23:19.512 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-04-20T21:23:19.512 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-04-20T21:23:19.512 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-04-20T21:23:19.512 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-04-20T21:23:19.512 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-04-20T21:23:19.512 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/selinux 2024-04-20T21:23:19.512 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T21:23:19.512 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-04-20T21:23:19.512 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-04-20T21:23:19.512 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-04-20T21:23:19.512 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-04-20T21:23:19.512 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-04-20T21:23:19.513 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-04-20T21:23:19.513 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T21:23:19.513 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T21:23:19.513 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T21:23:19.513 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T21:23:19.513 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-04-20T21:23:19.513 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-04-20T21:23:19.513 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T21:23:19.513 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T21:23:19.513 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T21:23:19.513 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T21:23:19.513 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-04-20T21:23:19.513 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:23:19.513 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:23:19.513 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:23:19.514 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:23:19.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:23:19.515 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-04-20T21:23:19.565 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:2024-04-20T21:23:19.565+0000 7f2544644480 -1 init, newargv = 0x55b4c9902490 newargc=15 2024-04-20T21:23:19.565 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[48683]: starting ceph client 2024-04-20T21:23:19.573 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[48683]: starting fuse 2024-04-20T21:23:19.595 INFO:teuthology.orchestra.run.smithi086.stdout:41 2024-04-20T21:23:19.595 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-04-20T21:23:19.595 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer -- python3 -c ' 2024-04-20T21:23:19.595 DEBUG:teuthology.orchestra.run.smithi086:> import glob 2024-04-20T21:23:19.596 DEBUG:teuthology.orchestra.run.smithi086:> import re 2024-04-20T21:23:19.596 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:23:19.596 DEBUG:teuthology.orchestra.run.smithi086:> import subprocess 2024-04-20T21:23:19.596 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:23:19.596 DEBUG:teuthology.orchestra.run.smithi086:> def _find_admin_socket(client_name): 2024-04-20T21:23:19.596 DEBUG:teuthology.orchestra.run.smithi086:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-20T21:23:19.596 DEBUG:teuthology.orchestra.run.smithi086:> files = glob.glob(asok_path) 2024-04-20T21:23:19.596 DEBUG:teuthology.orchestra.run.smithi086:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-20T21:23:19.596 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:23:19.596 DEBUG:teuthology.orchestra.run.smithi086:> # Given a non-glob path, it better be there 2024-04-20T21:23:19.596 DEBUG:teuthology.orchestra.run.smithi086:> if "*" not in asok_path: 2024-04-20T21:23:19.596 DEBUG:teuthology.orchestra.run.smithi086:> assert(len(files) == 1) 2024-04-20T21:23:19.596 DEBUG:teuthology.orchestra.run.smithi086:> return files[0] 2024-04-20T21:23:19.596 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:23:19.596 DEBUG:teuthology.orchestra.run.smithi086:> for f in files: 2024-04-20T21:23:19.596 DEBUG:teuthology.orchestra.run.smithi086:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T21:23:19.597 DEBUG:teuthology.orchestra.run.smithi086:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T21:23:19.597 DEBUG:teuthology.orchestra.run.smithi086:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T21:23:19.597 DEBUG:teuthology.orchestra.run.smithi086:> contents = proc_f.read() 2024-04-20T21:23:19.597 DEBUG:teuthology.orchestra.run.smithi086:> if mountpoint in contents: 2024-04-20T21:23:19.597 DEBUG:teuthology.orchestra.run.smithi086:> return f 2024-04-20T21:23:19.597 DEBUG:teuthology.orchestra.run.smithi086:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T21:23:19.597 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:23:19.597 DEBUG:teuthology.orchestra.run.smithi086:> print(_find_admin_socket("client.0")) 2024-04-20T21:23:19.597 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:23:19.785 INFO:teuthology.orchestra.run.smithi086.stdout:/var/run/ceph/ceph-client.0.48683.asok 2024-04-20T21:23:19.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:23:19.802 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.48683.asok 2024-04-20T21:23:19.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:23:19.802 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.48683.asok status 2024-04-20T21:23:20.027 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:23:20.027 INFO:teuthology.orchestra.run.smithi086.stdout: "metadata": { 2024-04-20T21:23:20.027 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T21:23:20.027 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T21:23:20.028 INFO:teuthology.orchestra.run.smithi086.stdout: "entity_id": "0", 2024-04-20T21:23:20.028 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": "smithi086", 2024-04-20T21:23:20.028 INFO:teuthology.orchestra.run.smithi086.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-20T21:23:20.028 INFO:teuthology.orchestra.run.smithi086.stdout: "pid": "48683", 2024-04-20T21:23:20.028 INFO:teuthology.orchestra.run.smithi086.stdout: "root": "/" 2024-04-20T21:23:20.028 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:23:20.028 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_count": 0, 2024-04-20T21:23:20.028 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_pinned_count": 0, 2024-04-20T21:23:20.028 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 7047, 2024-04-20T21:23:20.028 INFO:teuthology.orchestra.run.smithi086.stdout: "inst": { 2024-04-20T21:23:20.028 INFO:teuthology.orchestra.run.smithi086.stdout: "name": { 2024-04-20T21:23:20.028 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "client", 2024-04-20T21:23:20.028 INFO:teuthology.orchestra.run.smithi086.stdout: "num": 7047 2024-04-20T21:23:20.028 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:23:20.028 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2024-04-20T21:23:20.029 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-04-20T21:23:20.029 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2024-04-20T21:23:20.029 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 854016995 2024-04-20T21:23:20.029 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:23:20.029 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:23:20.029 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2024-04-20T21:23:20.029 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-04-20T21:23:20.029 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2024-04-20T21:23:20.029 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 854016995 2024-04-20T21:23:20.029 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:23:20.029 INFO:teuthology.orchestra.run.smithi086.stdout: "inst_str": "client.7047 192.168.0.1:0/854016995", 2024-04-20T21:23:20.029 INFO:teuthology.orchestra.run.smithi086.stdout: "addr_str": "192.168.0.1:0/854016995", 2024-04-20T21:23:20.029 INFO:teuthology.orchestra.run.smithi086.stdout: "inode_count": 1, 2024-04-20T21:23:20.029 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_epoch": 48, 2024-04-20T21:23:20.029 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch": 72, 2024-04-20T21:23:20.029 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch_barrier": 0, 2024-04-20T21:23:20.029 INFO:teuthology.orchestra.run.smithi086.stdout: "blocklisted": false, 2024-04-20T21:23:20.030 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs" 2024-04-20T21:23:20.030 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:23:20.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:23:20.043 DEBUG:teuthology.orchestra.run.smithi086:> stat --file-system '--printf=%T 2024-04-20T21:23:20.043 DEBUG:teuthology.orchestra.run.smithi086:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-20T21:23:20.094 INFO:teuthology.orchestra.run.smithi086.stdout:fuseblk 2024-04-20T21:23:20.095 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-20T21:23:20.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:23:20.095 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-20T21:23:20.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:23:20.175 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls subdir/) 2024-04-20T21:23:20.236 INFO:teuthology.orchestra.run.smithi086.stdout:file_undamaged 2024-04-20T21:23:20.238 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-04-20T21:23:20.238 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:23:20.238 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-04-20T21:23:20.238 DEBUG:teuthology.orchestra.run.smithi086:> import json 2024-04-20T21:23:20.238 DEBUG:teuthology.orchestra.run.smithi086:> import sys 2024-04-20T21:23:20.238 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:23:20.239 DEBUG:teuthology.orchestra.run.smithi086:> try: 2024-04-20T21:23:20.239 DEBUG:teuthology.orchestra.run.smithi086:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged'"'"') 2024-04-20T21:23:20.239 DEBUG:teuthology.orchestra.run.smithi086:> except OSError as e: 2024-04-20T21:23:20.239 DEBUG:teuthology.orchestra.run.smithi086:> sys.exit(e.errno) 2024-04-20T21:23:20.239 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:23:20.239 DEBUG:teuthology.orchestra.run.smithi086:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-20T21:23:20.239 DEBUG:teuthology.orchestra.run.smithi086:> print(json.dumps( 2024-04-20T21:23:20.239 DEBUG:teuthology.orchestra.run.smithi086:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-20T21:23:20.239 DEBUG:teuthology.orchestra.run.smithi086:> indent=2)) 2024-04-20T21:23:20.239 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:23:20.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20 stdin-killer INFO: command exited with status 2: exiting normally with same code! 2024-04-20T21:23:20.398 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-20T21:23:20.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:23:20.398 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-04-20T21:23:20.499 INFO:teuthology.orchestra.run.smithi086.stdout:2 2024-04-20T21:23:20.499 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:23:20.499 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:23:20.553 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:23:20.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.755+0000 7f1057e00640 1 Processor -- start 2024-04-20T21:23:20.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.755+0000 7f1057e00640 1 -- start start 2024-04-20T21:23:20.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.756+0000 7f1057e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f105815d6a0 0x7f10580ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:20.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.757+0000 7f1057e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1058160890 con 0x7f1058156ae0 2024-04-20T21:23:20.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.757+0000 7f1057e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1058160b70 con 0x7f10580ad620 2024-04-20T21:23:20.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.757+0000 7f1057e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f105806db20 con 0x7f105815d6a0 2024-04-20T21:23:20.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.757+0000 7f1056a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f1058156ae0 0x7f1058160080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52042/0 (socket says 172.21.15.86:52042) 2024-04-20T21:23:20.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.757+0000 7f1056a00640 1 -- 172.21.15.86:0/3942722788 learned_addr learned my addr 172.21.15.86:0/3942722788 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:23:20.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.757+0000 7f1056000640 1 -- 172.21.15.86:0/3942722788 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 616020776 0 0) 0x7f1058160890 con 0x7f1058156ae0 2024-04-20T21:23:20.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.757+0000 7f1056000640 1 -- 172.21.15.86:0/3942722788 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f103c003520 con 0x7f1058156ae0 2024-04-20T21:23:20.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.757+0000 7f104e000640 1 --2- 172.21.15.86:0/3942722788 >> v2:172.21.15.144:3300/0 conn(0x7f105815d6a0 0x7f10580ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:20.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.758+0000 7f1056000640 1 -- 172.21.15.86:0/3942722788 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1877845534 0 0) 0x7f103c003520 con 0x7f1058156ae0 2024-04-20T21:23:20.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.758+0000 7f1056000640 1 -- 172.21.15.86:0/3942722788 >> v2:172.21.15.144:3300/0 conn(0x7f105815d6a0 msgr2=0x7f10580ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:20.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.758+0000 7f1056000640 1 --2- 172.21.15.86:0/3942722788 >> v2:172.21.15.144:3300/0 conn(0x7f105815d6a0 0x7f10580ad0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:20.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.758+0000 7f1056000640 1 -- 172.21.15.86:0/3942722788 >> v1:172.21.15.144:6789/0 conn(0x7f10580ad620 legacy=0x7f105806e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:20.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.758+0000 7f1056000640 1 -- 172.21.15.86:0/3942722788 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1058160e50 con 0x7f1058156ae0 2024-04-20T21:23:20.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.758+0000 7f1056000640 1 -- 172.21.15.86:0/3942722788 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f1048001670 con 0x7f1058156ae0 2024-04-20T21:23:20.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.758+0000 7f1056000640 1 -- 172.21.15.86:0/3942722788 >> v1:172.21.15.86:6789/0 conn(0x7f1058156ae0 legacy=0x7f1058160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:20.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.758+0000 7f1056000640 1 --2- 172.21.15.86:0/3942722788 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f103c003a00 0x7f103c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:20.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.758+0000 7f1056000640 1 --2- 172.21.15.86:0/3942722788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f103c004800 0x7f103c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:20.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.758+0000 7f1056000640 1 --2- 172.21.15.86:0/3942722788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f103c005210 0x7f103c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:20.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.758+0000 7f1056000640 1 -- 172.21.15.86:0/3942722788 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f105806db20 con 0x7f103c004800 2024-04-20T21:23:20.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.758+0000 7f1056000640 1 -- 172.21.15.86:0/3942722788 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f103c00c140 con 0x7f103c005210 2024-04-20T21:23:20.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.758+0000 7f1056000640 1 -- 172.21.15.86:0/3942722788 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f103c00c340 con 0x7f103c003a00 2024-04-20T21:23:20.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.758+0000 7f104e000640 1 --2- 172.21.15.86:0/3942722788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f103c004800 0x7f103c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:20.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.758+0000 7f104e000640 1 -- 172.21.15.86:0/3942722788 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f103c003a00 msgr2=0x7f103c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:20.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.759+0000 7f104e000640 1 --2- 172.21.15.86:0/3942722788 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f103c003a00 0x7f103c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:20.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.759+0000 7f104e000640 1 -- 172.21.15.86:0/3942722788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f103c005210 msgr2=0x7f103c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:20.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.759+0000 7f104e000640 1 --2- 172.21.15.86:0/3942722788 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f103c005210 0x7f103c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:20.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.759+0000 7f104e000640 1 -- 172.21.15.86:0/3942722788 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f103c003520 con 0x7f103c004800 2024-04-20T21:23:20.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.759+0000 7f1057400640 1 -- 172.21.15.86:0/3942722788 reap_dead start 2024-04-20T21:23:20.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.760+0000 7f104e000640 1 --2- 172.21.15.86:0/3942722788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f103c004800 0x7f103c004c80 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f104000ba00 tx=0x7f103c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e972ec7bca3f4fcb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:20.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.760+0000 7f1056000640 1 -- 172.21.15.86:0/3942722788 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1040036020 con 0x7f103c004800 2024-04-20T21:23:20.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.760+0000 7f1056000640 1 -- 172.21.15.86:0/3942722788 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1040032640 con 0x7f103c004800 2024-04-20T21:23:20.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.760+0000 7f1056000640 1 -- 172.21.15.86:0/3942722788 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f104000b600 con 0x7f103c004800 2024-04-20T21:23:20.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.760+0000 7f1057e00640 1 -- 172.21.15.86:0/3942722788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f103c004800 msgr2=0x7f103c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:20.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.760+0000 7f1057e00640 1 --2- 172.21.15.86:0/3942722788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f103c004800 0x7f103c004c80 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f104000ba00 tx=0x7f103c005830 comp rx=0 tx=0).stop 2024-04-20T21:23:20.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.761+0000 7f1057e00640 1 -- 172.21.15.86:0/3942722788 shutdown_connections 2024-04-20T21:23:20.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.761+0000 7f1057e00640 1 --2- 172.21.15.86:0/3942722788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f103c004800 0x7f103c004c80 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:20.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.761+0000 7f1057e00640 1 -- 172.21.15.86:0/3942722788 >> 172.21.15.86:0/3942722788 conn(0x7f10580d0170 msgr2=0x7f10580d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:20.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.761+0000 7f1057e00640 1 -- 172.21.15.86:0/3942722788 shutdown_connections 2024-04-20T21:23:20.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.761+0000 7f1057e00640 1 -- 172.21.15.86:0/3942722788 wait complete. 2024-04-20T21:23:20.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.762+0000 7f1057e00640 1 Processor -- start 2024-04-20T21:23:20.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.762+0000 7f1057e00640 1 -- start start 2024-04-20T21:23:20.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.763+0000 7f1057e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f103c004800 0x7f10581f43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:20.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.763+0000 7f1057e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10580b4610 0x7f1058157040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:20.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.763+0000 7f1057e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f105815c0e0 0x7f1058159310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:20.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.763+0000 7f1057e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1058156ec0 con 0x7f10580b4610 2024-04-20T21:23:20.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.763+0000 7f1057e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f10580ada50 con 0x7f103c004800 2024-04-20T21:23:20.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.763+0000 7f1057e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f10580ad0e0 con 0x7f105815c0e0 2024-04-20T21:23:20.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.764+0000 7f104e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10580b4610 0x7f1058157040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:20.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.764+0000 7f104e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10580b4610 0x7f1058157040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52306/0 (socket says 172.21.15.86:52306) 2024-04-20T21:23:20.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.764+0000 7f104e000640 1 -- 172.21.15.86:0/1041907144 learned_addr learned my addr 172.21.15.86:0/1041907144 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:20.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.764+0000 7f1056a00640 1 --2- 172.21.15.86:0/1041907144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f103c004800 0x7f10581f43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:20.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.764+0000 7f1057400640 1 --2- 172.21.15.86:0/1041907144 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f105815c0e0 0x7f1058159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:20.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.764+0000 7f104e000640 1 -- 172.21.15.86:0/1041907144 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f105815c0e0 msgr2=0x7f1058159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:20.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.764+0000 7f104e000640 1 --2- 172.21.15.86:0/1041907144 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f105815c0e0 0x7f1058159310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:20.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.764+0000 7f104e000640 1 -- 172.21.15.86:0/1041907144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f103c004800 msgr2=0x7f10581f43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:20.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.764+0000 7f104e000640 1 --2- 172.21.15.86:0/1041907144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f103c004800 0x7f10581f43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:20.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.764+0000 7f104e000640 1 -- 172.21.15.86:0/1041907144 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1058143f90 con 0x7f10580b4610 2024-04-20T21:23:20.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.764+0000 7f104e000640 1 --2- 172.21.15.86:0/1041907144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10580b4610 0x7f1058157040 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f1040035040 tx=0x7f1040004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:20.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.765+0000 7f104fe00640 1 -- 172.21.15.86:0/1041907144 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1040041070 con 0x7f10580b4610 2024-04-20T21:23:20.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.765+0000 7f104fe00640 1 -- 172.21.15.86:0/1041907144 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1040030a50 con 0x7f10580b4610 2024-04-20T21:23:20.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.765+0000 7f1057e00640 1 -- 172.21.15.86:0/1041907144 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1058144210 con 0x7f10580b4610 2024-04-20T21:23:20.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.765+0000 7f104fe00640 1 -- 172.21.15.86:0/1041907144 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1040036040 con 0x7f10580b4610 2024-04-20T21:23:20.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.765+0000 7f1057e00640 1 -- 172.21.15.86:0/1041907144 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1058145da0 con 0x7f10580b4610 2024-04-20T21:23:20.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.766+0000 7f1057e00640 1 -- 172.21.15.86:0/1041907144 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f105804fcf0 con 0x7f10580b4610 2024-04-20T21:23:20.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.767+0000 7f104fe00640 1 -- 172.21.15.86:0/1041907144 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f104004c060 con 0x7f10580b4610 2024-04-20T21:23:20.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.768+0000 7f104fe00640 1 --2- 172.21.15.86:0/1041907144 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f103406c920 0x7f103406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:20.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.768+0000 7f104fe00640 1 -- 172.21.15.86:0/1041907144 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5795+0+0 (secure 0 0 0) 0x7f10400add30 con 0x7f10580b4610 2024-04-20T21:23:20.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.768+0000 7f1056a00640 1 --2- 172.21.15.86:0/1041907144 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f103406c920 0x7f103406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:20.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.769+0000 7f1056a00640 1 --2- 172.21.15.86:0/1041907144 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f103406c920 0x7f103406ede0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f1058155c70 tx=0x7f1048009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:20.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.773+0000 7f104fe00640 1 -- 172.21.15.86:0/1041907144 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1058143f90 con 0x7f10580b4610 2024-04-20T21:23:20.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.974+0000 7f1057e00640 1 -- 172.21.15.86:0/1041907144 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1058145fe0 con 0x7f10580b4610 2024-04-20T21:23:20.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.976+0000 7f104fe00640 1 -- 172.21.15.86:0/1041907144 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+16795 (secure 0 0 0) 0x7f104007c0f0 con 0x7f10580b4610 2024-04-20T21:23:20.977 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:23:20.977 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":72,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:23:12.079595+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:22:43.125767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-20T21:22:44.410549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6837/202143875":"2024-04-21T21:23:12.079572+0000","172.21.15.86:6835/202143875":"2024-04-21T21:23:12.079572+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:23:20.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.979+0000 7f1057e00640 1 -- 172.21.15.86:0/1041907144 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f103406c920 msgr2=0x7f103406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:20.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.979+0000 7f1057e00640 1 --2- 172.21.15.86:0/1041907144 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f103406c920 0x7f103406ede0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f1058155c70 tx=0x7f1048009040 comp rx=0 tx=0).stop 2024-04-20T21:23:20.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.979+0000 7f1057e00640 1 -- 172.21.15.86:0/1041907144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10580b4610 msgr2=0x7f1058157040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:20.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.979+0000 7f1057e00640 1 --2- 172.21.15.86:0/1041907144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10580b4610 0x7f1058157040 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f1040035040 tx=0x7f1040004730 comp rx=0 tx=0).stop 2024-04-20T21:23:20.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.979+0000 7f1057e00640 1 -- 172.21.15.86:0/1041907144 shutdown_connections 2024-04-20T21:23:20.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.979+0000 7f1057e00640 1 --2- 172.21.15.86:0/1041907144 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f103406c920 0x7f103406ede0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:20.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.979+0000 7f1057e00640 1 --2- 172.21.15.86:0/1041907144 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f105815c0e0 0x7f1058159310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:20.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.979+0000 7f1057e00640 1 --2- 172.21.15.86:0/1041907144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10580b4610 0x7f1058157040 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:20.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.980+0000 7f1057e00640 1 --2- 172.21.15.86:0/1041907144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f103c004800 0x7f10581f43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:20.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.980+0000 7f1057e00640 1 -- 172.21.15.86:0/1041907144 >> 172.21.15.86:0/1041907144 conn(0x7f10580d0170 msgr2=0x7f10580729d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:20.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.980+0000 7f1057e00640 1 -- 172.21.15.86:0/1041907144 shutdown_connections 2024-04-20T21:23:20.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:20.980+0000 7f1057e00640 1 -- 172.21.15.86:0/1041907144 wait complete. 2024-04-20T21:23:20.999 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:22:43.125767+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:22:44.410549+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:23:20.999 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:23:21.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.206+0000 7fdfd2400640 1 Processor -- start 2024-04-20T21:23:21.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.206+0000 7fdfd2400640 1 -- start start 2024-04-20T21:23:21.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.208+0000 7fdfd2400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdfcc0aee50 0x7fdfcc06b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:21.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.208+0000 7fdfd2400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfcc0adb10 con 0x7fdfcc15eb30 2024-04-20T21:23:21.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.208+0000 7fdfd2400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfcc0addb0 con 0x7fdfcc0b4530 2024-04-20T21:23:21.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.208+0000 7fdfd2400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fdfcc0af7d0 con 0x7fdfcc0aee50 2024-04-20T21:23:21.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.208+0000 7fdfcb400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fdfcc15eb30 0x7fdfcc0b3e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52058/0 (socket says 172.21.15.86:52058) 2024-04-20T21:23:21.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.208+0000 7fdfcb400640 1 -- 172.21.15.86:0/1359363100 learned_addr learned my addr 172.21.15.86:0/1359363100 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:23:21.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.208+0000 7fdfcaa00640 1 --2- 172.21.15.86:0/1359363100 >> v2:172.21.15.86:3300/0 conn(0x7fdfcc0aee50 0x7fdfcc06b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:21.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.209+0000 7fdfcaa00640 1 -- 172.21.15.86:0/1359363100 >> v1:172.21.15.144:6789/0 conn(0x7fdfcc0b4530 legacy=0x7fdfcc06e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:21.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.209+0000 7fdfcaa00640 1 -- 172.21.15.86:0/1359363100 >> v1:172.21.15.86:6789/0 conn(0x7fdfcc15eb30 legacy=0x7fdfcc0b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:21.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.209+0000 7fdfcaa00640 1 -- 172.21.15.86:0/1359363100 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfcc0ae090 con 0x7fdfcc0aee50 2024-04-20T21:23:21.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.209+0000 7fdfca000640 1 -- 172.21.15.86:0/1359363100 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1386999924 0 0) 0x7fdfcc0adb10 con 0x7fdfcc15eb30 2024-04-20T21:23:21.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.210+0000 7fdfcaa00640 1 --2- 172.21.15.86:0/1359363100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfcc0aee50 0x7fdfcc06b980 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fdfb80097a0 tx=0x7fdfb802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ef58fa69b8069df server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:21.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.210+0000 7fdfca000640 1 -- 172.21.15.86:0/1359363100 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb8035020 con 0x7fdfcc0aee50 2024-04-20T21:23:21.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.210+0000 7fdfca000640 1 -- 172.21.15.86:0/1359363100 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfb8031830 con 0x7fdfcc0aee50 2024-04-20T21:23:21.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.210+0000 7fdfca000640 1 -- 172.21.15.86:0/1359363100 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb8031c00 con 0x7fdfcc0aee50 2024-04-20T21:23:21.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.211+0000 7fdfd2400640 1 -- 172.21.15.86:0/1359363100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfcc0aee50 msgr2=0x7fdfcc06b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:21.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.211+0000 7fdfd2400640 1 --2- 172.21.15.86:0/1359363100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfcc0aee50 0x7fdfcc06b980 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fdfb80097a0 tx=0x7fdfb802b8d0 comp rx=0 tx=0).stop 2024-04-20T21:23:21.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.211+0000 7fdfd2400640 1 -- 172.21.15.86:0/1359363100 shutdown_connections 2024-04-20T21:23:21.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.211+0000 7fdfd2400640 1 --2- 172.21.15.86:0/1359363100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfcc0aee50 0x7fdfcc06b980 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:21.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.211+0000 7fdfd2400640 1 -- 172.21.15.86:0/1359363100 >> 172.21.15.86:0/1359363100 conn(0x7fdfcc0b2110 msgr2=0x7fdfcc0b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:21.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.211+0000 7fdfd2400640 1 -- 172.21.15.86:0/1359363100 shutdown_connections 2024-04-20T21:23:21.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.211+0000 7fdfd2400640 1 -- 172.21.15.86:0/1359363100 wait complete. 2024-04-20T21:23:21.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.213+0000 7fdfd2400640 1 Processor -- start 2024-04-20T21:23:21.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.213+0000 7fdfd2400640 1 -- start start 2024-04-20T21:23:21.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.214+0000 7fdfd2400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfcc0aee50 0x7fdfcc147e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:21.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.214+0000 7fdfd2400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfcc0b4530 0x7fdfcc15b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:21.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.214+0000 7fdfd2400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfcc15eb30 0x7fdfcc15bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:21.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.214+0000 7fdfd2400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc0ae930 con 0x7fdfcc15eb30 2024-04-20T21:23:21.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.214+0000 7fdfd2400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc0ae650 con 0x7fdfcc0aee50 2024-04-20T21:23:21.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.214+0000 7fdfd2400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdfcc06bec0 con 0x7fdfcc0b4530 2024-04-20T21:23:21.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.214+0000 7fdfcbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfcc15eb30 0x7fdfcc15bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:21.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.214+0000 7fdfcbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfcc15eb30 0x7fdfcc15bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52326/0 (socket says 172.21.15.86:52326) 2024-04-20T21:23:21.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.214+0000 7fdfcbe00640 1 -- 172.21.15.86:0/1239236930 learned_addr learned my addr 172.21.15.86:0/1239236930 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:21.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.214+0000 7fdfcb400640 1 --2- 172.21.15.86:0/1239236930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfcc0aee50 0x7fdfcc147e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:21.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.214+0000 7fdfcaa00640 1 --2- 172.21.15.86:0/1239236930 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfcc0b4530 0x7fdfcc15b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:21.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.215+0000 7fdfcbe00640 1 -- 172.21.15.86:0/1239236930 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfcc0b4530 msgr2=0x7fdfcc15b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:21.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.215+0000 7fdfcbe00640 1 --2- 172.21.15.86:0/1239236930 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfcc0b4530 0x7fdfcc15b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:21.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.215+0000 7fdfcbe00640 1 -- 172.21.15.86:0/1239236930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfcc0aee50 msgr2=0x7fdfcc147e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:21.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.215+0000 7fdfcbe00640 1 --2- 172.21.15.86:0/1239236930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfcc0aee50 0x7fdfcc147e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:21.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.215+0000 7fdfcbe00640 1 -- 172.21.15.86:0/1239236930 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfcc153710 con 0x7fdfcc15eb30 2024-04-20T21:23:21.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.215+0000 7fdfcb400640 1 --2- 172.21.15.86:0/1239236930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfcc0aee50 0x7fdfcc147e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:23:21.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.215+0000 7fdfcbe00640 1 --2- 172.21.15.86:0/1239236930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfcc15eb30 0x7fdfcc15bf00 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fdfc400c730 tx=0x7fdfc400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:21.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.215+0000 7fdfc3400640 1 -- 172.21.15.86:0/1239236930 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc4016020 con 0x7fdfcc15eb30 2024-04-20T21:23:21.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.215+0000 7fdfc3400640 1 -- 172.21.15.86:0/1239236930 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfc400aa50 con 0x7fdfcc15eb30 2024-04-20T21:23:21.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.215+0000 7fdfc3400640 1 -- 172.21.15.86:0/1239236930 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc400ce40 con 0x7fdfcc15eb30 2024-04-20T21:23:21.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.215+0000 7fdfd2400640 1 -- 172.21.15.86:0/1239236930 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfcc1539f0 con 0x7fdfcc15eb30 2024-04-20T21:23:21.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.216+0000 7fdfd2400640 1 -- 172.21.15.86:0/1239236930 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfcc153fb0 con 0x7fdfcc15eb30 2024-04-20T21:23:21.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.217+0000 7fdfd2400640 1 -- 172.21.15.86:0/1239236930 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfcc0689e0 con 0x7fdfcc15eb30 2024-04-20T21:23:21.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.218+0000 7fdfc3400640 1 -- 172.21.15.86:0/1239236930 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdfc4019020 con 0x7fdfcc15eb30 2024-04-20T21:23:21.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.218+0000 7fdfc3400640 1 --2- 172.21.15.86:0/1239236930 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdfac06c920 0x7fdfac06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:21.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.219+0000 7fdfc3400640 1 -- 172.21.15.86:0/1239236930 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5795+0+0 (secure 0 0 0) 0x7fdfc408bc30 con 0x7fdfcc15eb30 2024-04-20T21:23:21.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.219+0000 7fdfcb400640 1 --2- 172.21.15.86:0/1239236930 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdfac06c920 0x7fdfac06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:21.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.219+0000 7fdfcb400640 1 --2- 172.21.15.86:0/1239236930 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdfac06c920 0x7fdfac06ede0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc000f60 tx=0x7fdfbc007ca0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:21.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.223+0000 7fdfc3400640 1 -- 172.21.15.86:0/1239236930 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fdfc4020030 con 0x7fdfcc15eb30 2024-04-20T21:23:21.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.422+0000 7fdfd2400640 1 -- 172.21.15.86:0/1239236930 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdfcc0b4140 con 0x7fdfcc15eb30 2024-04-20T21:23:21.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.423+0000 7fdfc3400640 1 -- 172.21.15.86:0/1239236930 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) ==== 93+0+10 (secure 0 0 0) 0x7fdfc40182c0 con 0x7fdfcc15eb30 2024-04-20T21:23:21.423 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:23:21.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.426+0000 7fdfd2400640 1 -- 172.21.15.86:0/1239236930 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdfac06c920 msgr2=0x7fdfac06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:21.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.426+0000 7fdfd2400640 1 --2- 172.21.15.86:0/1239236930 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdfac06c920 0x7fdfac06ede0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc000f60 tx=0x7fdfbc007ca0 comp rx=0 tx=0).stop 2024-04-20T21:23:21.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.426+0000 7fdfd2400640 1 -- 172.21.15.86:0/1239236930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfcc15eb30 msgr2=0x7fdfcc15bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:21.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.426+0000 7fdfd2400640 1 --2- 172.21.15.86:0/1239236930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfcc15eb30 0x7fdfcc15bf00 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fdfc400c730 tx=0x7fdfc400cbf0 comp rx=0 tx=0).stop 2024-04-20T21:23:21.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.426+0000 7fdfd2400640 1 -- 172.21.15.86:0/1239236930 shutdown_connections 2024-04-20T21:23:21.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.426+0000 7fdfd2400640 1 --2- 172.21.15.86:0/1239236930 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdfac06c920 0x7fdfac06ede0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:21.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.426+0000 7fdfd2400640 1 --2- 172.21.15.86:0/1239236930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfcc15eb30 0x7fdfcc15bf00 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:21.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.426+0000 7fdfd2400640 1 --2- 172.21.15.86:0/1239236930 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfcc0b4530 0x7fdfcc15b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:21.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.426+0000 7fdfd2400640 1 --2- 172.21.15.86:0/1239236930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfcc0aee50 0x7fdfcc147e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:21.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.426+0000 7fdfd2400640 1 -- 172.21.15.86:0/1239236930 >> 172.21.15.86:0/1239236930 conn(0x7fdfcc0b2110 msgr2=0x7fdfcc0b3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:21.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.426+0000 7fdfd2400640 1 -- 172.21.15.86:0/1239236930 shutdown_connections 2024-04-20T21:23:21.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.426+0000 7fdfd2400640 1 -- 172.21.15.86:0/1239236930 wait complete. 2024-04-20T21:23:21.446 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:23:21.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.655+0000 7f1f20200640 1 Processor -- start 2024-04-20T21:23:21.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.656+0000 7f1f20200640 1 -- start start 2024-04-20T21:23:21.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.657+0000 7f1f20200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1f18146210 0x7f1f1806e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:21.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.657+0000 7f1f20200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f1815cd40 con 0x7f1f1806bbf0 2024-04-20T21:23:21.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.657+0000 7f1f20200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f1815d000 con 0x7f1f1806bfd0 2024-04-20T21:23:21.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.657+0000 7f1f20200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f1f1815d2e0 con 0x7f1f18146210 2024-04-20T21:23:21.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.657+0000 7f1f1d800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f1f1806bbf0 0x7f1f18145b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52070/0 (socket says 172.21.15.86:52070) 2024-04-20T21:23:21.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.657+0000 7f1f1d800640 1 -- 172.21.15.86:0/4015333691 learned_addr learned my addr 172.21.15.86:0/4015333691 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:23:21.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.658+0000 7f1f17e00640 1 -- 172.21.15.86:0/4015333691 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3913913107 0 0) 0x7f1f1815cd40 con 0x7f1f1806bbf0 2024-04-20T21:23:21.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.658+0000 7f1f17e00640 1 -- 172.21.15.86:0/4015333691 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f00003520 con 0x7f1f1806bbf0 2024-04-20T21:23:21.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.658+0000 7f1f1ce00640 1 --2- 172.21.15.86:0/4015333691 >> v2:172.21.15.144:3300/0 conn(0x7f1f18146210 0x7f1f1806e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:21.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.658+0000 7f1f17e00640 1 -- 172.21.15.86:0/4015333691 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3856386555 0 0) 0x7f1f00003520 con 0x7f1f1806bbf0 2024-04-20T21:23:21.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.658+0000 7f1f17e00640 1 -- 172.21.15.86:0/4015333691 >> v2:172.21.15.144:3300/0 conn(0x7f1f18146210 msgr2=0x7f1f1806e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:21.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.658+0000 7f1f17e00640 1 --2- 172.21.15.86:0/4015333691 >> v2:172.21.15.144:3300/0 conn(0x7f1f18146210 0x7f1f1806e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:21.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.658+0000 7f1f17e00640 1 -- 172.21.15.86:0/4015333691 >> v1:172.21.15.144:6789/0 conn(0x7f1f1806bfd0 legacy=0x7f1f180cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:21.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.658+0000 7f1f17e00640 1 -- 172.21.15.86:0/4015333691 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f1815d5c0 con 0x7f1f1806bbf0 2024-04-20T21:23:21.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.658+0000 7f1f17e00640 1 -- 172.21.15.86:0/4015333691 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f1f0c001670 con 0x7f1f1806bbf0 2024-04-20T21:23:21.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.658+0000 7f1f17e00640 1 -- 172.21.15.86:0/4015333691 >> v1:172.21.15.86:6789/0 conn(0x7f1f1806bbf0 legacy=0x7f1f18145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:21.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.658+0000 7f1f17e00640 1 --2- 172.21.15.86:0/4015333691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f000039b0 0x7f1f00003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:21.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.658+0000 7f1f17e00640 1 --2- 172.21.15.86:0/4015333691 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f00004800 0x7f1f00004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:21.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.658+0000 7f1f17e00640 1 --2- 172.21.15.86:0/4015333691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f000051c0 0x7f1f0000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:21.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.658+0000 7f1f17e00640 1 -- 172.21.15.86:0/4015333691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1815d2e0 con 0x7f1f000051c0 2024-04-20T21:23:21.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.658+0000 7f1f17e00640 1 -- 172.21.15.86:0/4015333691 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1f0000c110 con 0x7f1f000039b0 2024-04-20T21:23:21.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.658+0000 7f1f17e00640 1 -- 172.21.15.86:0/4015333691 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1f0000c310 con 0x7f1f00004800 2024-04-20T21:23:21.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.658+0000 7f1f1e200640 1 --2- 172.21.15.86:0/4015333691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f000051c0 0x7f1f0000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:21.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.659+0000 7f1f1d800640 1 --2- 172.21.15.86:0/4015333691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f000039b0 0x7f1f00003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:21.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.659+0000 7f1f1ce00640 1 --2- 172.21.15.86:0/4015333691 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f00004800 0x7f1f00004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:21.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.659+0000 7f1f1e200640 1 -- 172.21.15.86:0/4015333691 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f00004800 msgr2=0x7f1f00004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:21.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.659+0000 7f1f1e200640 1 --2- 172.21.15.86:0/4015333691 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f00004800 0x7f1f00004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:21.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.659+0000 7f1f1e200640 1 -- 172.21.15.86:0/4015333691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f000039b0 msgr2=0x7f1f00003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:21.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.659+0000 7f1f1e200640 1 --2- 172.21.15.86:0/4015333691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f000039b0 0x7f1f00003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:21.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.659+0000 7f1f1e200640 1 -- 172.21.15.86:0/4015333691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f00003520 con 0x7f1f000051c0 2024-04-20T21:23:21.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.660+0000 7f1f1e200640 1 -- 172.21.15.86:0/4015333691 reap_dead start 2024-04-20T21:23:21.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.660+0000 7f1f1e200640 1 --2- 172.21.15.86:0/4015333691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f000051c0 0x7f1f0000b9c0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f1f10001e50 tx=0x7f1f00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af233bf8a7e2402b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:21.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.660+0000 7f1f17e00640 1 -- 172.21.15.86:0/4015333691 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f10001a70 con 0x7f1f000051c0 2024-04-20T21:23:21.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.660+0000 7f1f17e00640 1 -- 172.21.15.86:0/4015333691 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f10003bb0 con 0x7f1f000051c0 2024-04-20T21:23:21.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.660+0000 7f1f17e00640 1 -- 172.21.15.86:0/4015333691 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f100050d0 con 0x7f1f000051c0 2024-04-20T21:23:21.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.661+0000 7f1f20200640 1 -- 172.21.15.86:0/4015333691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f000051c0 msgr2=0x7f1f0000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:21.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.661+0000 7f1f20200640 1 --2- 172.21.15.86:0/4015333691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f000051c0 0x7f1f0000b9c0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f1f10001e50 tx=0x7f1f00005830 comp rx=0 tx=0).stop 2024-04-20T21:23:21.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.661+0000 7f1f20200640 1 -- 172.21.15.86:0/4015333691 shutdown_connections 2024-04-20T21:23:21.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.661+0000 7f1f20200640 1 --2- 172.21.15.86:0/4015333691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f000051c0 0x7f1f0000b9c0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:21.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.661+0000 7f1f20200640 1 -- 172.21.15.86:0/4015333691 >> 172.21.15.86:0/4015333691 conn(0x7f1f180b2110 msgr2=0x7f1f180b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:21.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.661+0000 7f1f20200640 1 -- 172.21.15.86:0/4015333691 shutdown_connections 2024-04-20T21:23:21.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.661+0000 7f1f20200640 1 -- 172.21.15.86:0/4015333691 wait complete. 2024-04-20T21:23:21.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.663+0000 7f1f20200640 1 Processor -- start 2024-04-20T21:23:21.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.663+0000 7f1f20200640 1 -- start start 2024-04-20T21:23:21.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.664+0000 7f1f20200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f000051c0 0x7f1f181e78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:21.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.664+0000 7f1f20200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f18136620 0x7f1f180b0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:21.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.664+0000 7f1f20200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f180b0ba0 0x7f1f180b1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:21.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.664+0000 7f1f20200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1806e180 con 0x7f1f000051c0 2024-04-20T21:23:21.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.664+0000 7f1f20200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1806e000 con 0x7f1f180b0ba0 2024-04-20T21:23:21.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.664+0000 7f1f20200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1f1806e300 con 0x7f1f18136620 2024-04-20T21:23:21.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.664+0000 7f1f1d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f000051c0 0x7f1f181e78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:21.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.664+0000 7f1f1d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f000051c0 0x7f1f181e78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52344/0 (socket says 172.21.15.86:52344) 2024-04-20T21:23:21.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.664+0000 7f1f1d800640 1 -- 172.21.15.86:0/2092824085 learned_addr learned my addr 172.21.15.86:0/2092824085 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:21.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.665+0000 7f1f1ce00640 1 --2- 172.21.15.86:0/2092824085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f18136620 0x7f1f180b0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:21.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.665+0000 7f1f1d800640 1 -- 172.21.15.86:0/2092824085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f18136620 msgr2=0x7f1f180b0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:21.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.665+0000 7f1f1d800640 1 --2- 172.21.15.86:0/2092824085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f18136620 0x7f1f180b0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:21.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.665+0000 7f1f1d800640 1 -- 172.21.15.86:0/2092824085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f180b0ba0 msgr2=0x7f1f180b1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:21.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.665+0000 7f1f1e200640 1 --2- 172.21.15.86:0/2092824085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f180b0ba0 0x7f1f180b1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:21.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.665+0000 7f1f1d800640 1 --2- 172.21.15.86:0/2092824085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f180b0ba0 0x7f1f180b1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:21.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.665+0000 7f1f1d800640 1 -- 172.21.15.86:0/2092824085 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f181e7ee0 con 0x7f1f000051c0 2024-04-20T21:23:21.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.665+0000 7f1f1ce00640 1 --2- 172.21.15.86:0/2092824085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f18136620 0x7f1f180b0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:23:21.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.665+0000 7f1f1d800640 1 --2- 172.21.15.86:0/2092824085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f000051c0 0x7f1f181e78f0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c00e760 tx=0x7f1f0c00ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:21.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.665+0000 7f1f15600640 1 -- 172.21.15.86:0/2092824085 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c00ee70 con 0x7f1f000051c0 2024-04-20T21:23:21.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.665+0000 7f1f15600640 1 -- 172.21.15.86:0/2092824085 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f0c018070 con 0x7f1f000051c0 2024-04-20T21:23:21.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.665+0000 7f1f15600640 1 -- 172.21.15.86:0/2092824085 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c00cab0 con 0x7f1f000051c0 2024-04-20T21:23:21.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.665+0000 7f1f20200640 1 -- 172.21.15.86:0/2092824085 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f181e81c0 con 0x7f1f000051c0 2024-04-20T21:23:21.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.665+0000 7f1f20200640 1 -- 172.21.15.86:0/2092824085 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f181e8780 con 0x7f1f000051c0 2024-04-20T21:23:21.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.667+0000 7f1f20200640 1 -- 172.21.15.86:0/2092824085 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f180689e0 con 0x7f1f000051c0 2024-04-20T21:23:21.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.668+0000 7f1f15600640 1 -- 172.21.15.86:0/2092824085 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1f0c00cc50 con 0x7f1f000051c0 2024-04-20T21:23:21.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.668+0000 7f1f15600640 1 --2- 172.21.15.86:0/2092824085 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1ef806c8f0 0x7f1ef806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:21.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.669+0000 7f1f15600640 1 -- 172.21.15.86:0/2092824085 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5795+0+0 (secure 0 0 0) 0x7f1f0c08bac0 con 0x7f1f000051c0 2024-04-20T21:23:21.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.669+0000 7f1f1ce00640 1 --2- 172.21.15.86:0/2092824085 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1ef806c8f0 0x7f1ef806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:21.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.669+0000 7f1f1ce00640 1 --2- 172.21.15.86:0/2092824085 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1ef806c8f0 0x7f1ef806edb0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1f18130b40 tx=0x7f1f04008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:21.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.673+0000 7f1f15600640 1 -- 172.21.15.86:0/2092824085 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1f0c059df0 con 0x7f1f000051c0 2024-04-20T21:23:21.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.870+0000 7f1f20200640 1 -- 172.21.15.86:0/2092824085 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1f180b1ed0 con 0x7f1f000051c0 2024-04-20T21:23:21.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.871+0000 7f1f15600640 1 -- 172.21.15.86:0/2092824085 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v72) ==== 104+0+11 (secure 0 0 0) 0x7f1f0c0596f0 con 0x7f1f000051c0 2024-04-20T21:23:21.871 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-20T21:23:21.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.874+0000 7f1f20200640 1 -- 172.21.15.86:0/2092824085 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1ef806c8f0 msgr2=0x7f1ef806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:21.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.874+0000 7f1f20200640 1 --2- 172.21.15.86:0/2092824085 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1ef806c8f0 0x7f1ef806edb0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1f18130b40 tx=0x7f1f04008040 comp rx=0 tx=0).stop 2024-04-20T21:23:21.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.874+0000 7f1f20200640 1 -- 172.21.15.86:0/2092824085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f000051c0 msgr2=0x7f1f181e78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:21.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.874+0000 7f1f20200640 1 --2- 172.21.15.86:0/2092824085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f000051c0 0x7f1f181e78f0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c00e760 tx=0x7f1f0c00ec20 comp rx=0 tx=0).stop 2024-04-20T21:23:21.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.874+0000 7f1f20200640 1 -- 172.21.15.86:0/2092824085 shutdown_connections 2024-04-20T21:23:21.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.874+0000 7f1f20200640 1 --2- 172.21.15.86:0/2092824085 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1ef806c8f0 0x7f1ef806edb0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:21.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.874+0000 7f1f20200640 1 --2- 172.21.15.86:0/2092824085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f180b0ba0 0x7f1f180b1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:21.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.874+0000 7f1f20200640 1 --2- 172.21.15.86:0/2092824085 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f18136620 0x7f1f180b0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:21.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.874+0000 7f1f20200640 1 --2- 172.21.15.86:0/2092824085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f000051c0 0x7f1f181e78f0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:21.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.874+0000 7f1f20200640 1 -- 172.21.15.86:0/2092824085 >> 172.21.15.86:0/2092824085 conn(0x7f1f180b2110 msgr2=0x7f1f1815f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:21.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.874+0000 7f1f20200640 1 -- 172.21.15.86:0/2092824085 shutdown_connections 2024-04-20T21:23:21.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:21.874+0000 7f1f20200640 1 -- 172.21.15.86:0/2092824085 wait complete. 2024-04-20T21:23:21.891 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:23:22.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.089+0000 7f6dce600640 1 Processor -- start 2024-04-20T21:23:22.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.089+0000 7f6dce600640 1 -- start start 2024-04-20T21:23:22.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.091+0000 7f6dce600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6dc806c090 0x7f6dc80cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:22.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.091+0000 7f6dce600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6dc80b4ba0 0x7f6dc80b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:22.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.091+0000 7f6dce600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6dc8156110 con 0x7f6dc8155d30 2024-04-20T21:23:22.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.091+0000 7f6dce600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f6dc80ce460 con 0x7f6dc80b4ba0 2024-04-20T21:23:22.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.091+0000 7f6dce600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6dc80ce740 con 0x7f6dc806c090 2024-04-20T21:23:22.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.091+0000 7f6dc6a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6dc806c090 0x7f6dc80cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:22.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.091+0000 7f6dc6a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6dc806c090 0x7f6dc80cf500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:56726/0 (socket says 172.21.15.86:56726) 2024-04-20T21:23:22.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.091+0000 7f6dc6a00640 1 -- 172.21.15.86:0/104836043 learned_addr learned my addr 172.21.15.86:0/104836043 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:22.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.092+0000 7f6dc7e00640 1 --2- 172.21.15.86:0/104836043 >> v2:172.21.15.144:3300/0 conn(0x7f6dc80b4ba0 0x7f6dc80b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:22.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.092+0000 7f6dc6a00640 1 -- 172.21.15.86:0/104836043 >> v2:172.21.15.144:3300/0 conn(0x7f6dc80b4ba0 msgr2=0x7f6dc80b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:22.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.092+0000 7f6dc6a00640 1 --2- 172.21.15.86:0/104836043 >> v2:172.21.15.144:3300/0 conn(0x7f6dc80b4ba0 0x7f6dc80b4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:22.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.092+0000 7f6dc6a00640 1 -- 172.21.15.86:0/104836043 >> v1:172.21.15.144:6789/0 conn(0x7f6dc8155d30 legacy=0x7f6dc806b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:22.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.092+0000 7f6dc6a00640 1 -- 172.21.15.86:0/104836043 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dc80cea20 con 0x7f6dc806c090 2024-04-20T21:23:22.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.093+0000 7f6dc6a00640 1 --2- 172.21.15.86:0/104836043 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6dc806c090 0x7f6dc80cf500 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6db4009870 tx=0x7f6db402b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51708c1b223baaf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:22.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.094+0000 7f6dc6000640 1 -- 172.21.15.86:0/104836043 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db40317c0 con 0x7f6dc806c090 2024-04-20T21:23:22.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.094+0000 7f6dc6000640 1 -- 172.21.15.86:0/104836043 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6db4037070 con 0x7f6dc806c090 2024-04-20T21:23:22.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.094+0000 7f6dc6000640 1 -- 172.21.15.86:0/104836043 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db4031b80 con 0x7f6dc806c090 2024-04-20T21:23:22.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.095+0000 7f6dce600640 1 -- 172.21.15.86:0/104836043 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6dc806c090 msgr2=0x7f6dc80cf500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:22.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.095+0000 7f6dce600640 1 --2- 172.21.15.86:0/104836043 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6dc806c090 0x7f6dc80cf500 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6db4009870 tx=0x7f6db402b860 comp rx=0 tx=0).stop 2024-04-20T21:23:22.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.095+0000 7f6dce600640 1 -- 172.21.15.86:0/104836043 shutdown_connections 2024-04-20T21:23:22.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.095+0000 7f6dce600640 1 --2- 172.21.15.86:0/104836043 >> v2:172.21.15.144:3300/0 conn(0x7f6dc80b4ba0 0x7f6dc80b4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:22.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.095+0000 7f6dce600640 1 --2- 172.21.15.86:0/104836043 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6dc806c090 0x7f6dc80cf500 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:22.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.095+0000 7f6dce600640 1 -- 172.21.15.86:0/104836043 >> 172.21.15.86:0/104836043 conn(0x7f6dc80b2110 msgr2=0x7f6dc80b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:22.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.095+0000 7f6dce600640 1 -- 172.21.15.86:0/104836043 shutdown_connections 2024-04-20T21:23:22.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.095+0000 7f6dce600640 1 -- 172.21.15.86:0/104836043 wait complete. 2024-04-20T21:23:22.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.096+0000 7f6dce600640 1 Processor -- start 2024-04-20T21:23:22.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.097+0000 7f6dce600640 1 -- start start 2024-04-20T21:23:22.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.098+0000 7f6dce600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6dc806c090 0x7f6dc814d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:22.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.098+0000 7f6dce600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6dc80b4ba0 0x7f6dc81365c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:22.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.098+0000 7f6dce600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6dc8155d30 0x7f6dc814c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:22.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.098+0000 7f6dce600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc80b5520 con 0x7f6dc8155d30 2024-04-20T21:23:22.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.098+0000 7f6dce600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc80cefe0 con 0x7f6dc806c090 2024-04-20T21:23:22.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.098+0000 7f6dce600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6dc80b56a0 con 0x7f6dc80b4ba0 2024-04-20T21:23:22.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.098+0000 7f6dc7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6dc8155d30 0x7f6dc814c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:22.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.098+0000 7f6dc7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6dc8155d30 0x7f6dc814c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52350/0 (socket says 172.21.15.86:52350) 2024-04-20T21:23:22.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.098+0000 7f6dc7e00640 1 -- 172.21.15.86:0/744097063 learned_addr learned my addr 172.21.15.86:0/744097063 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:22.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.098+0000 7f6dc6a00640 1 --2- 172.21.15.86:0/744097063 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6dc80b4ba0 0x7f6dc81365c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:22.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.098+0000 7f6dc7e00640 1 -- 172.21.15.86:0/744097063 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6dc80b4ba0 msgr2=0x7f6dc81365c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:22.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.098+0000 7f6dc7400640 1 --2- 172.21.15.86:0/744097063 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6dc806c090 0x7f6dc814d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:22.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.098+0000 7f6dc7e00640 1 --2- 172.21.15.86:0/744097063 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6dc80b4ba0 0x7f6dc81365c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:22.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.098+0000 7f6dc7e00640 1 -- 172.21.15.86:0/744097063 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6dc806c090 msgr2=0x7f6dc814d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:22.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.098+0000 7f6dc7e00640 1 --2- 172.21.15.86:0/744097063 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6dc806c090 0x7f6dc814d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:22.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.099+0000 7f6dc7e00640 1 -- 172.21.15.86:0/744097063 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dc815d440 con 0x7f6dc8155d30 2024-04-20T21:23:22.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.099+0000 7f6dc7e00640 1 --2- 172.21.15.86:0/744097063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6dc8155d30 0x7f6dc814c0a0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f6dc000efc0 tx=0x7f6dc000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:22.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.099+0000 7f6dbf400640 1 -- 172.21.15.86:0/744097063 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0018020 con 0x7f6dc8155d30 2024-04-20T21:23:22.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.099+0000 7f6dbf400640 1 -- 172.21.15.86:0/744097063 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dc0014480 con 0x7f6dc8155d30 2024-04-20T21:23:22.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.099+0000 7f6dbf400640 1 -- 172.21.15.86:0/744097063 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0014870 con 0x7f6dc8155d30 2024-04-20T21:23:22.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.099+0000 7f6dce600640 1 -- 172.21.15.86:0/744097063 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dc815d720 con 0x7f6dc8155d30 2024-04-20T21:23:22.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.099+0000 7f6dce600640 1 -- 172.21.15.86:0/744097063 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dc815dce0 con 0x7f6dc8155d30 2024-04-20T21:23:22.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.101+0000 7f6dce600640 1 -- 172.21.15.86:0/744097063 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dc80689e0 con 0x7f6dc8155d30 2024-04-20T21:23:22.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.102+0000 7f6dbf400640 1 -- 172.21.15.86:0/744097063 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6dc0005d10 con 0x7f6dc8155d30 2024-04-20T21:23:22.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.102+0000 7f6dbf400640 1 --2- 172.21.15.86:0/744097063 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6da806c8f0 0x7f6da806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:22.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.103+0000 7f6dbf400640 1 -- 172.21.15.86:0/744097063 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5795+0+0 (secure 0 0 0) 0x7f6dc008c900 con 0x7f6dc8155d30 2024-04-20T21:23:22.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.103+0000 7f6dc7400640 1 --2- 172.21.15.86:0/744097063 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6da806c8f0 0x7f6da806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:22.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.103+0000 7f6dc7400640 1 --2- 172.21.15.86:0/744097063 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6da806c8f0 0x7f6da806edb0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f6db8001500 tx=0x7f6db8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:22.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.107+0000 7f6dbf400640 1 -- 172.21.15.86:0/744097063 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6dc0022030 con 0x7f6dc8155d30 2024-04-20T21:23:22.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.310+0000 7f6dce600640 1 -- 172.21.15.86:0/744097063 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6dc80cc820 con 0x7f6dc8155d30 2024-04-20T21:23:22.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.311+0000 7f6dbf400640 1 -- 172.21.15.86:0/744097063 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v72) ==== 100+0+11 (secure 0 0 0) 0x7f6dc0010040 con 0x7f6dc8155d30 2024-04-20T21:23:22.311 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-20T21:23:22.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.313+0000 7f6dce600640 1 -- 172.21.15.86:0/744097063 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6da806c8f0 msgr2=0x7f6da806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:22.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.313+0000 7f6dce600640 1 --2- 172.21.15.86:0/744097063 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6da806c8f0 0x7f6da806edb0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f6db8001500 tx=0x7f6db8009040 comp rx=0 tx=0).stop 2024-04-20T21:23:22.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.313+0000 7f6dce600640 1 -- 172.21.15.86:0/744097063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6dc8155d30 msgr2=0x7f6dc814c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:22.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.313+0000 7f6dce600640 1 --2- 172.21.15.86:0/744097063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6dc8155d30 0x7f6dc814c0a0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f6dc000efc0 tx=0x7f6dc000c490 comp rx=0 tx=0).stop 2024-04-20T21:23:22.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.313+0000 7f6dce600640 1 -- 172.21.15.86:0/744097063 shutdown_connections 2024-04-20T21:23:22.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.313+0000 7f6dce600640 1 --2- 172.21.15.86:0/744097063 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6da806c8f0 0x7f6da806edb0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:22.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.313+0000 7f6dce600640 1 --2- 172.21.15.86:0/744097063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6dc8155d30 0x7f6dc814c0a0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:22.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.313+0000 7f6dce600640 1 --2- 172.21.15.86:0/744097063 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6dc80b4ba0 0x7f6dc81365c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:22.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.313+0000 7f6dce600640 1 --2- 172.21.15.86:0/744097063 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6dc806c090 0x7f6dc814d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:22.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.313+0000 7f6dce600640 1 -- 172.21.15.86:0/744097063 >> 172.21.15.86:0/744097063 conn(0x7f6dc80b2110 msgr2=0x7f6dc80d0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:22.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.313+0000 7f6dce600640 1 -- 172.21.15.86:0/744097063 shutdown_connections 2024-04-20T21:23:22.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.313+0000 7f6dce600640 1 -- 172.21.15.86:0/744097063 wait complete. 2024-04-20T21:23:22.333 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T21:23:22.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.532+0000 7efe29c00640 1 Processor -- start 2024-04-20T21:23:22.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.533+0000 7efe29c00640 1 -- start start 2024-04-20T21:23:22.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.534+0000 7efe29c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7efe2415e690 0x7efe2406ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:22.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.534+0000 7efe29c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe240d2cb0 0x7efe240d1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:22.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.534+0000 7efe29c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe240d3100 con 0x7efe240d0ee0 2024-04-20T21:23:22.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.534+0000 7efe29c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7efe240d18b0 con 0x7efe2415e690 2024-04-20T21:23:22.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.534+0000 7efe29c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7efe240d33b0 con 0x7efe240d2cb0 2024-04-20T21:23:22.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.534+0000 7efe23400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7efe240d0ee0 0x7efe240d25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52078/0 (socket says 172.21.15.86:52078) 2024-04-20T21:23:22.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.534+0000 7efe23400640 1 -- 172.21.15.86:0/3113039926 learned_addr learned my addr 172.21.15.86:0/3113039926 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:23:22.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.535+0000 7efe29200640 1 --2- 172.21.15.86:0/3113039926 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe240d2cb0 0x7efe240d1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:22.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.535+0000 7efe22a00640 1 -- 172.21.15.86:0/3113039926 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3897354516 0 0) 0x7efe240d3100 con 0x7efe240d0ee0 2024-04-20T21:23:22.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.535+0000 7efe22a00640 1 -- 172.21.15.86:0/3113039926 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe08003520 con 0x7efe240d0ee0 2024-04-20T21:23:22.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.535+0000 7efe23e00640 1 --2- 172.21.15.86:0/3113039926 >> v2:172.21.15.144:3300/0 conn(0x7efe2415e690 0x7efe2406ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:22.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.535+0000 7efe22a00640 1 -- 172.21.15.86:0/3113039926 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 684176671 0 0) 0x7efe08003520 con 0x7efe240d0ee0 2024-04-20T21:23:22.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.535+0000 7efe22a00640 1 -- 172.21.15.86:0/3113039926 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe240d2cb0 msgr2=0x7efe240d1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:22.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.535+0000 7efe22a00640 1 --2- 172.21.15.86:0/3113039926 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe240d2cb0 0x7efe240d1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:22.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.535+0000 7efe22a00640 1 -- 172.21.15.86:0/3113039926 >> v2:172.21.15.144:3300/0 conn(0x7efe2415e690 msgr2=0x7efe2406ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:22.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.535+0000 7efe22a00640 1 --2- 172.21.15.86:0/3113039926 >> v2:172.21.15.144:3300/0 conn(0x7efe2415e690 0x7efe2406ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:22.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.535+0000 7efe22a00640 1 -- 172.21.15.86:0/3113039926 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe240d3690 con 0x7efe240d0ee0 2024-04-20T21:23:22.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.535+0000 7efe22a00640 1 -- 172.21.15.86:0/3113039926 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7efe0c001810 con 0x7efe240d0ee0 2024-04-20T21:23:22.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.535+0000 7efe22a00640 1 -- 172.21.15.86:0/3113039926 >> v1:172.21.15.86:6789/0 conn(0x7efe240d0ee0 legacy=0x7efe240d25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:22.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.535+0000 7efe22a00640 1 --2- 172.21.15.86:0/3113039926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe080039b0 0x7efe08003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:22.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.535+0000 7efe22a00640 1 --2- 172.21.15.86:0/3113039926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe08004800 0x7efe08004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:22.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.535+0000 7efe22a00640 1 --2- 172.21.15.86:0/3113039926 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe240d2cb0 0x7efe080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:22.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.535+0000 7efe22a00640 1 -- 172.21.15.86:0/3113039926 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efe240d18b0 con 0x7efe08004800 2024-04-20T21:23:22.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.535+0000 7efe22a00640 1 -- 172.21.15.86:0/3113039926 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efe240d33b0 con 0x7efe080039b0 2024-04-20T21:23:22.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.535+0000 7efe22a00640 1 -- 172.21.15.86:0/3113039926 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7efe0800baf0 con 0x7efe240d2cb0 2024-04-20T21:23:22.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.536+0000 7efe23400640 1 --2- 172.21.15.86:0/3113039926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe08004800 0x7efe08004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:22.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.536+0000 7efe23e00640 1 --2- 172.21.15.86:0/3113039926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe080039b0 0x7efe08003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:22.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.536+0000 7efe29200640 1 --2- 172.21.15.86:0/3113039926 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe240d2cb0 0x7efe080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:22.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.536+0000 7efe23400640 1 -- 172.21.15.86:0/3113039926 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe240d2cb0 msgr2=0x7efe080051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:22.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.536+0000 7efe23400640 1 --2- 172.21.15.86:0/3113039926 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe240d2cb0 0x7efe080051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:22.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.536+0000 7efe23400640 1 -- 172.21.15.86:0/3113039926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe080039b0 msgr2=0x7efe08003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:22.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.536+0000 7efe23400640 1 --2- 172.21.15.86:0/3113039926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe080039b0 0x7efe08003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:22.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.536+0000 7efe23400640 1 -- 172.21.15.86:0/3113039926 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe08003520 con 0x7efe08004800 2024-04-20T21:23:22.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.537+0000 7efe23400640 1 --2- 172.21.15.86:0/3113039926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe08004800 0x7efe08004c80 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7efe240c8030 tx=0x7efe08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e349cff878c6029c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:22.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.537+0000 7efe22a00640 1 -- 172.21.15.86:0/3113039926 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0c007c80 con 0x7efe08004800 2024-04-20T21:23:22.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.537+0000 7efe22a00640 1 -- 172.21.15.86:0/3113039926 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe0c030070 con 0x7efe08004800 2024-04-20T21:23:22.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.537+0000 7efe22a00640 1 -- 172.21.15.86:0/3113039926 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0c0074b0 con 0x7efe08004800 2024-04-20T21:23:22.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.538+0000 7efe29c00640 1 -- 172.21.15.86:0/3113039926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe08004800 msgr2=0x7efe08004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:22.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.538+0000 7efe29c00640 1 --2- 172.21.15.86:0/3113039926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe08004800 0x7efe08004c80 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7efe240c8030 tx=0x7efe08005830 comp rx=0 tx=0).stop 2024-04-20T21:23:22.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.538+0000 7efe29200640 1 -- 172.21.15.86:0/3113039926 reap_dead start 2024-04-20T21:23:22.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.538+0000 7efe29c00640 1 -- 172.21.15.86:0/3113039926 shutdown_connections 2024-04-20T21:23:22.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.538+0000 7efe29c00640 1 -- 172.21.15.86:0/3113039926 >> 172.21.15.86:0/3113039926 conn(0x7efe2409d8a0 msgr2=0x7efe2408bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:22.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.538+0000 7efe29c00640 1 -- 172.21.15.86:0/3113039926 shutdown_connections 2024-04-20T21:23:22.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.538+0000 7efe29c00640 1 -- 172.21.15.86:0/3113039926 wait complete. 2024-04-20T21:23:22.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.540+0000 7efe29c00640 1 Processor -- start 2024-04-20T21:23:22.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.540+0000 7efe29c00640 1 -- start start 2024-04-20T21:23:22.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.541+0000 7efe29c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe08004800 0x7efe241f8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:22.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.541+0000 7efe29c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe24160840 0x7efe24156120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:22.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.541+0000 7efe29c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe240b5fa0 0x7efe240b6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:22.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.541+0000 7efe29c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efe240cef00 con 0x7efe24160840 2024-04-20T21:23:22.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.541+0000 7efe29c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efe240cec20 con 0x7efe08004800 2024-04-20T21:23:22.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.541+0000 7efe29c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7efe2406d360 con 0x7efe240b5fa0 2024-04-20T21:23:22.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.541+0000 7efe23400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe24160840 0x7efe24156120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:22.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.541+0000 7efe23400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe24160840 0x7efe24156120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52380/0 (socket says 172.21.15.86:52380) 2024-04-20T21:23:22.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.541+0000 7efe23400640 1 -- 172.21.15.86:0/2676601754 learned_addr learned my addr 172.21.15.86:0/2676601754 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:22.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.542+0000 7efe23400640 1 -- 172.21.15.86:0/2676601754 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe240b5fa0 msgr2=0x7efe240b6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:22.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.542+0000 7efe23400640 1 --2- 172.21.15.86:0/2676601754 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe240b5fa0 0x7efe240b6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:22.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.542+0000 7efe23400640 1 -- 172.21.15.86:0/2676601754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe08004800 msgr2=0x7efe241f8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:22.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.542+0000 7efe23400640 1 --2- 172.21.15.86:0/2676601754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe08004800 0x7efe241f8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:22.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.542+0000 7efe23400640 1 -- 172.21.15.86:0/2676601754 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe24160fe0 con 0x7efe24160840 2024-04-20T21:23:22.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.542+0000 7efe23400640 1 --2- 172.21.15.86:0/2676601754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe24160840 0x7efe24156120 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7efe0c000c00 tx=0x7efe240d3860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:22.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.542+0000 7efe1be00640 1 -- 172.21.15.86:0/2676601754 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0c001cf0 con 0x7efe24160840 2024-04-20T21:23:22.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.542+0000 7efe1be00640 1 -- 172.21.15.86:0/2676601754 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe0c02cbe0 con 0x7efe24160840 2024-04-20T21:23:22.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.542+0000 7efe29c00640 1 -- 172.21.15.86:0/2676601754 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe241f4e60 con 0x7efe24160840 2024-04-20T21:23:22.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.543+0000 7efe1be00640 1 -- 172.21.15.86:0/2676601754 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0c0057a0 con 0x7efe24160840 2024-04-20T21:23:22.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.543+0000 7efe29c00640 1 -- 172.21.15.86:0/2676601754 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe241f5420 con 0x7efe24160840 2024-04-20T21:23:22.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.544+0000 7efe29c00640 1 -- 172.21.15.86:0/2676601754 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe2404fcf0 con 0x7efe24160840 2024-04-20T21:23:22.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.545+0000 7efe1be00640 1 -- 172.21.15.86:0/2676601754 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efe0c03f070 con 0x7efe24160840 2024-04-20T21:23:22.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.546+0000 7efe1be00640 1 --2- 172.21.15.86:0/2676601754 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efe0006c920 0x7efe0006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:22.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.546+0000 7efe1be00640 1 -- 172.21.15.86:0/2676601754 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5795+0+0 (secure 0 0 0) 0x7efe0c0acfb0 con 0x7efe24160840 2024-04-20T21:23:22.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.546+0000 7efe23e00640 1 --2- 172.21.15.86:0/2676601754 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efe0006c920 0x7efe0006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:22.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.547+0000 7efe23e00640 1 --2- 172.21.15.86:0/2676601754 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efe0006c920 0x7efe0006ede0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7efe14004350 tx=0x7efe140039b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:22.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.550+0000 7efe1be00640 1 -- 172.21.15.86:0/2676601754 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7efe0c03f360 con 0x7efe24160840 2024-04-20T21:23:22.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.762+0000 7efe29c00640 1 -- 172.21.15.86:0/2676601754 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7efe2406cc00 con 0x7efe24160840 2024-04-20T21:23:22.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.763+0000 7efe1be00640 1 -- 172.21.15.86:0/2676601754 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v72) ==== 80+0+120 (secure 0 0 0) 0x7efe0c07b2e0 con 0x7efe24160840 2024-04-20T21:23:22.763 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6837/202143875 2024-04-21T21:23:12.079572+0000 2024-04-20T21:23:22.763 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6835/202143875 2024-04-21T21:23:12.079572+0000 2024-04-20T21:23:22.764 INFO:teuthology.orchestra.run.smithi086.stderr:listed 2 entries 2024-04-20T21:23:22.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.766+0000 7efe29c00640 1 -- 172.21.15.86:0/2676601754 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efe0006c920 msgr2=0x7efe0006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:22.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.766+0000 7efe29c00640 1 --2- 172.21.15.86:0/2676601754 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efe0006c920 0x7efe0006ede0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7efe14004350 tx=0x7efe140039b0 comp rx=0 tx=0).stop 2024-04-20T21:23:22.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.766+0000 7efe29c00640 1 -- 172.21.15.86:0/2676601754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe24160840 msgr2=0x7efe24156120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:22.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.766+0000 7efe29c00640 1 --2- 172.21.15.86:0/2676601754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe24160840 0x7efe24156120 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7efe0c000c00 tx=0x7efe240d3860 comp rx=0 tx=0).stop 2024-04-20T21:23:22.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.766+0000 7efe29c00640 1 -- 172.21.15.86:0/2676601754 shutdown_connections 2024-04-20T21:23:22.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.766+0000 7efe29c00640 1 --2- 172.21.15.86:0/2676601754 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efe0006c920 0x7efe0006ede0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:22.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.766+0000 7efe29c00640 1 --2- 172.21.15.86:0/2676601754 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe240b5fa0 0x7efe240b6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:22.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.766+0000 7efe29c00640 1 --2- 172.21.15.86:0/2676601754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe24160840 0x7efe24156120 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:22.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.766+0000 7efe29c00640 1 --2- 172.21.15.86:0/2676601754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe08004800 0x7efe241f8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:22.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.766+0000 7efe29c00640 1 -- 172.21.15.86:0/2676601754 >> 172.21.15.86:0/2676601754 conn(0x7efe2409d8a0 msgr2=0x7efe240b1f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:22.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.766+0000 7efe29c00640 1 -- 172.21.15.86:0/2676601754 shutdown_connections 2024-04-20T21:23:22.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:22.766+0000 7efe29c00640 1 -- 172.21.15.86:0/2676601754 wait complete. 2024-04-20T21:23:22.783 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:23:22.783 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:23:22.832 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:23:23.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.072+0000 7f7cf7200640 1 Processor -- start 2024-04-20T21:23:23.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.072+0000 7f7cf7200640 1 -- start start 2024-04-20T21:23:23.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.074+0000 7f7cf7200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7cf8156ae0 0x7f7cf815d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:23.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.074+0000 7f7cf7200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7cf8160080 0x7f7cf80ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:23.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.074+0000 7f7cf7200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cf8160890 con 0x7f7cf80ad620 2024-04-20T21:23:23.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.074+0000 7f7cf7200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f7cf806db20 con 0x7f7cf8160080 2024-04-20T21:23:23.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.074+0000 7f7cf7200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f7cf80ada50 con 0x7f7cf8156ae0 2024-04-20T21:23:23.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.074+0000 7f7cf6800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f7cf80ad620 0x7f7cf8141cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52084/0 (socket says 172.21.15.86:52084) 2024-04-20T21:23:23.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.074+0000 7f7cf6800640 1 -- 172.21.15.86:0/3212981573 learned_addr learned my addr 172.21.15.86:0/3212981573 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:23:23.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.074+0000 7f7ced400640 1 --2- 172.21.15.86:0/3212981573 >> v2:172.21.15.86:3300/0 conn(0x7f7cf8160080 0x7f7cf80ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:23.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.074+0000 7f7cf5e00640 1 --2- 172.21.15.86:0/3212981573 >> v2:172.21.15.144:3300/0 conn(0x7f7cf8156ae0 0x7f7cf815d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:23.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.075+0000 7f7cf5400640 1 -- 172.21.15.86:0/3212981573 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1422411439 0 0) 0x7f7cf8160890 con 0x7f7cf80ad620 2024-04-20T21:23:23.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.075+0000 7f7cf5400640 1 -- 172.21.15.86:0/3212981573 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7cdc003520 con 0x7f7cf80ad620 2024-04-20T21:23:23.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.075+0000 7f7ced400640 1 -- 172.21.15.86:0/3212981573 >> v2:172.21.15.144:3300/0 conn(0x7f7cf8156ae0 msgr2=0x7f7cf815d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:23.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.075+0000 7f7ced400640 1 --2- 172.21.15.86:0/3212981573 >> v2:172.21.15.144:3300/0 conn(0x7f7cf8156ae0 0x7f7cf815d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:23.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.075+0000 7f7ced400640 1 -- 172.21.15.86:0/3212981573 >> v1:172.21.15.86:6789/0 conn(0x7f7cf80ad620 legacy=0x7f7cf8141cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:23.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.075+0000 7f7ced400640 1 -- 172.21.15.86:0/3212981573 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cf8160b70 con 0x7f7cf8160080 2024-04-20T21:23:23.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.075+0000 7f7cf5e00640 1 --2- 172.21.15.86:0/3212981573 >> v2:172.21.15.144:3300/0 conn(0x7f7cf8156ae0 0x7f7cf815d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:23:23.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.076+0000 7f7ced400640 1 --2- 172.21.15.86:0/3212981573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cf8160080 0x7f7cf80ad0e0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f7cf806d460 tx=0x7f7ce002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ff24dac19d1fe8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:23.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.076+0000 7f7cf5400640 1 -- 172.21.15.86:0/3212981573 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce0035020 con 0x7f7cf8160080 2024-04-20T21:23:23.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.076+0000 7f7cf5400640 1 -- 172.21.15.86:0/3212981573 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ce0031830 con 0x7f7cf8160080 2024-04-20T21:23:23.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.076+0000 7f7cf5400640 1 -- 172.21.15.86:0/3212981573 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce0031c00 con 0x7f7cf8160080 2024-04-20T21:23:23.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.077+0000 7f7cf7200640 1 -- 172.21.15.86:0/3212981573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cf8160080 msgr2=0x7f7cf80ad0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:23.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.077+0000 7f7cf7200640 1 --2- 172.21.15.86:0/3212981573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cf8160080 0x7f7cf80ad0e0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f7cf806d460 tx=0x7f7ce002b8d0 comp rx=0 tx=0).stop 2024-04-20T21:23:23.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.077+0000 7f7cf7200640 1 -- 172.21.15.86:0/3212981573 shutdown_connections 2024-04-20T21:23:23.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.077+0000 7f7cf7200640 1 --2- 172.21.15.86:0/3212981573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cf8160080 0x7f7cf80ad0e0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:23.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.077+0000 7f7cf7200640 1 --2- 172.21.15.86:0/3212981573 >> v2:172.21.15.144:3300/0 conn(0x7f7cf8156ae0 0x7f7cf815d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:23.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.077+0000 7f7cf7200640 1 -- 172.21.15.86:0/3212981573 >> 172.21.15.86:0/3212981573 conn(0x7f7cf80d0170 msgr2=0x7f7cf80d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:23.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.077+0000 7f7cf7200640 1 -- 172.21.15.86:0/3212981573 shutdown_connections 2024-04-20T21:23:23.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.077+0000 7f7cf7200640 1 -- 172.21.15.86:0/3212981573 wait complete. 2024-04-20T21:23:23.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.079+0000 7f7cf7200640 1 Processor -- start 2024-04-20T21:23:23.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.079+0000 7f7cf7200640 1 -- start start 2024-04-20T21:23:23.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.080+0000 7f7cf7200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cf80ad620 0x7f7cf81f44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:23.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.080+0000 7f7cf7200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cf8156ae0 0x7f7cf81583f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:23.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.080+0000 7f7cf7200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7cf8160080 0x7f7cf815c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:23.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.080+0000 7f7cf7200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf8156ec0 con 0x7f7cf8156ae0 2024-04-20T21:23:23.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.080+0000 7f7cf7200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf806dd70 con 0x7f7cf80ad620 2024-04-20T21:23:23.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.080+0000 7f7cf7200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7cf8160500 con 0x7f7cf8160080 2024-04-20T21:23:23.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.080+0000 7f7ced400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cf8156ae0 0x7f7cf81583f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:23.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.080+0000 7f7ced400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cf8156ae0 0x7f7cf81583f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52394/0 (socket says 172.21.15.86:52394) 2024-04-20T21:23:23.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.080+0000 7f7ced400640 1 -- 172.21.15.86:0/1349751658 learned_addr learned my addr 172.21.15.86:0/1349751658 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:23.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.080+0000 7f7ced400640 1 -- 172.21.15.86:0/1349751658 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7cf8160080 msgr2=0x7f7cf815c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:23.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.080+0000 7f7ced400640 1 --2- 172.21.15.86:0/1349751658 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7cf8160080 0x7f7cf815c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:23.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.080+0000 7f7ced400640 1 -- 172.21.15.86:0/1349751658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cf80ad620 msgr2=0x7f7cf81f44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:23.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.080+0000 7f7ced400640 1 --2- 172.21.15.86:0/1349751658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cf80ad620 0x7f7cf81f44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:23.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.080+0000 7f7ced400640 1 -- 172.21.15.86:0/1349751658 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cf8153be0 con 0x7f7cf8156ae0 2024-04-20T21:23:23.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.081+0000 7f7ced400640 1 --2- 172.21.15.86:0/1349751658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cf8156ae0 0x7f7cf81583f0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0000c00 tx=0x7f7ce00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:23.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.081+0000 7f7ceea00640 1 -- 172.21.15.86:0/1349751658 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce0040070 con 0x7f7cf8156ae0 2024-04-20T21:23:23.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.081+0000 7f7ceea00640 1 -- 172.21.15.86:0/1349751658 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ce003a910 con 0x7f7cf8156ae0 2024-04-20T21:23:23.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.081+0000 7f7ceea00640 1 -- 172.21.15.86:0/1349751658 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce0035040 con 0x7f7cf8156ae0 2024-04-20T21:23:23.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.081+0000 7f7cf7200640 1 -- 172.21.15.86:0/1349751658 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cf81596a0 con 0x7f7cf8156ae0 2024-04-20T21:23:23.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.081+0000 7f7cf7200640 1 -- 172.21.15.86:0/1349751658 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cf8157bf0 con 0x7f7cf8156ae0 2024-04-20T21:23:23.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.082+0000 7f7cf7200640 1 -- 172.21.15.86:0/1349751658 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cf815fd50 con 0x7f7cf8156ae0 2024-04-20T21:23:23.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.083+0000 7f7ceea00640 1 -- 172.21.15.86:0/1349751658 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7ce0038020 con 0x7f7cf8156ae0 2024-04-20T21:23:23.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.084+0000 7f7ceea00640 1 --2- 172.21.15.86:0/1349751658 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7cd406c8f0 0x7f7cd406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:23.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.084+0000 7f7ceea00640 1 -- 172.21.15.86:0/1349751658 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5795+0+0 (secure 0 0 0) 0x7f7ce00ad4f0 con 0x7f7cf8156ae0 2024-04-20T21:23:23.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.084+0000 7f7cf5e00640 1 --2- 172.21.15.86:0/1349751658 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7cd406c8f0 0x7f7cd406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:23.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.085+0000 7f7cf5e00640 1 --2- 172.21.15.86:0/1349751658 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7cd406c8f0 0x7f7cd406edb0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7ce8006e80 tx=0x7f7ce8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:23.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.088+0000 7f7ceea00640 1 -- 172.21.15.86:0/1349751658 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7cf815fd50 con 0x7f7cf8156ae0 2024-04-20T21:23:23.150 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-04-20T21:23:23.150+0000 7f5bbce00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-20T21:23:23.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.290+0000 7f7cf7200640 1 -- 172.21.15.86:0/1349751658 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7cf8159bc0 con 0x7f7cf8156ae0 2024-04-20T21:23:23.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.292+0000 7f7ceea00640 1 -- 172.21.15.86:0/1349751658 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+16795 (secure 0 0 0) 0x7f7ce007b820 con 0x7f7cf8156ae0 2024-04-20T21:23:23.292 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:23:23.292 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":72,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:23:12.079595+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:22:43.125767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-20T21:22:44.410549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6837/202143875":"2024-04-21T21:23:12.079572+0000","172.21.15.86:6835/202143875":"2024-04-21T21:23:12.079572+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:23:23.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.294+0000 7f7cf7200640 1 -- 172.21.15.86:0/1349751658 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7cd406c8f0 msgr2=0x7f7cd406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:23.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.294+0000 7f7cf7200640 1 --2- 172.21.15.86:0/1349751658 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7cd406c8f0 0x7f7cd406edb0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7ce8006e80 tx=0x7f7ce8008040 comp rx=0 tx=0).stop 2024-04-20T21:23:23.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.294+0000 7f7cf7200640 1 -- 172.21.15.86:0/1349751658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cf8156ae0 msgr2=0x7f7cf81583f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:23.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.294+0000 7f7cf7200640 1 --2- 172.21.15.86:0/1349751658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cf8156ae0 0x7f7cf81583f0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0000c00 tx=0x7f7ce00046c0 comp rx=0 tx=0).stop 2024-04-20T21:23:23.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.295+0000 7f7cf7200640 1 -- 172.21.15.86:0/1349751658 shutdown_connections 2024-04-20T21:23:23.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.295+0000 7f7cf7200640 1 --2- 172.21.15.86:0/1349751658 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7cd406c8f0 0x7f7cd406edb0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:23.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.295+0000 7f7cf7200640 1 --2- 172.21.15.86:0/1349751658 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7cf8160080 0x7f7cf815c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:23.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.295+0000 7f7cf7200640 1 --2- 172.21.15.86:0/1349751658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cf8156ae0 0x7f7cf81583f0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:23.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.295+0000 7f7cf7200640 1 --2- 172.21.15.86:0/1349751658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cf80ad620 0x7f7cf81f44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:23.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.295+0000 7f7cf7200640 1 -- 172.21.15.86:0/1349751658 >> 172.21.15.86:0/1349751658 conn(0x7f7cf80d0170 msgr2=0x7f7cf8073600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:23.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.295+0000 7f7cf7200640 1 -- 172.21.15.86:0/1349751658 shutdown_connections 2024-04-20T21:23:23.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.295+0000 7f7cf7200640 1 -- 172.21.15.86:0/1349751658 wait complete. 2024-04-20T21:23:23.314 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:22:43.125767+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:22:44.410549+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:23:23.314 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:23:23.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.518+0000 7fdaaee00640 1 Processor -- start 2024-04-20T21:23:23.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.519+0000 7fdaaee00640 1 -- start start 2024-04-20T21:23:23.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.520+0000 7fdaaee00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdaa8155d30 0x7fdaa80cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:23.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.520+0000 7fdaaee00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdaa80b52b0 0x7fdaa806be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:23.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.520+0000 7fdaaee00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdaa8156110 con 0x7fdaa806b980 2024-04-20T21:23:23.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.520+0000 7fdaaee00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fdaa80b56e0 con 0x7fdaa8155d30 2024-04-20T21:23:23.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.520+0000 7fdaaee00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdaa80ce460 con 0x7fdaa80b52b0 2024-04-20T21:23:23.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.521+0000 7fdaa7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdaa8155d30 0x7fdaa80cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:23.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.521+0000 7fda9f400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fdaa806b980 0x7fdaa80b4ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52096/0 (socket says 172.21.15.86:52096) 2024-04-20T21:23:23.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.521+0000 7fda9f400640 1 -- 172.21.15.86:0/4237041825 learned_addr learned my addr 172.21.15.86:0/4237041825 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:23:23.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.521+0000 7fdaace00640 1 --2- 172.21.15.86:0/4237041825 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdaa80b52b0 0x7fdaa806be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:23.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.521+0000 7fdaa7e00640 1 -- 172.21.15.86:0/4237041825 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdaa80b52b0 msgr2=0x7fdaa806be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:23.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.521+0000 7fdaa7e00640 1 --2- 172.21.15.86:0/4237041825 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdaa80b52b0 0x7fdaa806be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:23.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.521+0000 7fdaa7e00640 1 -- 172.21.15.86:0/4237041825 >> v1:172.21.15.86:6789/0 conn(0x7fdaa806b980 legacy=0x7fdaa80b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:23.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.521+0000 7fdaa7e00640 1 -- 172.21.15.86:0/4237041825 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaa80ce740 con 0x7fdaa8155d30 2024-04-20T21:23:23.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.521+0000 7fdaace00640 1 --2- 172.21.15.86:0/4237041825 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdaa80b52b0 0x7fdaa806be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:23:23.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.522+0000 7fdaa7e00640 1 --2- 172.21.15.86:0/4237041825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaa8155d30 0x7fdaa80cf500 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fda980097a0 tx=0x7fda9802b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4975ed1a946c6b1c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:23.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.523+0000 7fdaa7400640 1 -- 172.21.15.86:0/4237041825 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda98039070 con 0x7fdaa8155d30 2024-04-20T21:23:23.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.523+0000 7fdaa7400640 1 -- 172.21.15.86:0/4237041825 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda980316f0 con 0x7fdaa8155d30 2024-04-20T21:23:23.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.523+0000 7fdaa7400640 1 -- 172.21.15.86:0/4237041825 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda98031ac0 con 0x7fdaa8155d30 2024-04-20T21:23:23.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.523+0000 7fdaaee00640 1 -- 172.21.15.86:0/4237041825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaa8155d30 msgr2=0x7fdaa80cf500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:23.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.523+0000 7fdaaee00640 1 --2- 172.21.15.86:0/4237041825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaa8155d30 0x7fdaa80cf500 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fda980097a0 tx=0x7fda9802b790 comp rx=0 tx=0).stop 2024-04-20T21:23:23.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.523+0000 7fdaaee00640 1 -- 172.21.15.86:0/4237041825 shutdown_connections 2024-04-20T21:23:23.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.523+0000 7fdaaee00640 1 --2- 172.21.15.86:0/4237041825 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdaa80b52b0 0x7fdaa806be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:23.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.523+0000 7fdaaee00640 1 --2- 172.21.15.86:0/4237041825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaa8155d30 0x7fdaa80cf500 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:23.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.523+0000 7fdaaee00640 1 -- 172.21.15.86:0/4237041825 >> 172.21.15.86:0/4237041825 conn(0x7fdaa80b2110 msgr2=0x7fdaa80b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:23.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.524+0000 7fdaaee00640 1 -- 172.21.15.86:0/4237041825 shutdown_connections 2024-04-20T21:23:23.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.524+0000 7fdaaee00640 1 -- 172.21.15.86:0/4237041825 wait complete. 2024-04-20T21:23:23.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.525+0000 7fdaaee00640 1 Processor -- start 2024-04-20T21:23:23.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.526+0000 7fdaaee00640 1 -- start start 2024-04-20T21:23:23.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.527+0000 7fdaaee00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaa806b980 0x7fdaa814d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:23.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.527+0000 7fdaaee00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdaa80b52b0 0x7fdaa81f43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:23.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.527+0000 7fdaaee00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdaa8155d30 0x7fdaa814fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:23.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.527+0000 7fdaaee00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa80cefe0 con 0x7fdaa806b980 2024-04-20T21:23:23.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.527+0000 7fdaaee00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa80ced00 con 0x7fdaa8155d30 2024-04-20T21:23:23.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.527+0000 7fdaaee00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdaa806ddf0 con 0x7fdaa80b52b0 2024-04-20T21:23:23.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.527+0000 7fdaa7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaa806b980 0x7fdaa814d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:23.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.527+0000 7fdaa7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaa806b980 0x7fdaa814d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52398/0 (socket says 172.21.15.86:52398) 2024-04-20T21:23:23.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.527+0000 7fdaa7e00640 1 -- 172.21.15.86:0/2766203794 learned_addr learned my addr 172.21.15.86:0/2766203794 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:23.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.527+0000 7fdaa7e00640 1 -- 172.21.15.86:0/2766203794 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdaa80b52b0 msgr2=0x7fdaa81f43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:23.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.527+0000 7fdaa7e00640 1 --2- 172.21.15.86:0/2766203794 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdaa80b52b0 0x7fdaa81f43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:23.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.527+0000 7fdaa7e00640 1 -- 172.21.15.86:0/2766203794 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdaa8155d30 msgr2=0x7fdaa814fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:23.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.527+0000 7fdaa7e00640 1 --2- 172.21.15.86:0/2766203794 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdaa8155d30 0x7fdaa814fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:23.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.528+0000 7fdaa7e00640 1 -- 172.21.15.86:0/2766203794 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaa814a490 con 0x7fdaa806b980 2024-04-20T21:23:23.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.528+0000 7fdaa7e00640 1 --2- 172.21.15.86:0/2766203794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaa806b980 0x7fdaa814d830 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fda98000c00 tx=0x7fda980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:23.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.528+0000 7fdaa4c00640 1 -- 172.21.15.86:0/2766203794 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda98043070 con 0x7fdaa806b980 2024-04-20T21:23:23.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.528+0000 7fdaa4c00640 1 -- 172.21.15.86:0/2766203794 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda98034680 con 0x7fdaa806b980 2024-04-20T21:23:23.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.528+0000 7fdaaee00640 1 -- 172.21.15.86:0/2766203794 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaa81f4910 con 0x7fdaa806b980 2024-04-20T21:23:23.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.528+0000 7fdaa4c00640 1 -- 172.21.15.86:0/2766203794 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda98039040 con 0x7fdaa806b980 2024-04-20T21:23:23.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.528+0000 7fdaaee00640 1 -- 172.21.15.86:0/2766203794 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaa814e3d0 con 0x7fdaa806b980 2024-04-20T21:23:23.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.530+0000 7fdaaee00640 1 -- 172.21.15.86:0/2766203794 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaa80689e0 con 0x7fdaa806b980 2024-04-20T21:23:23.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.530+0000 7fdaa4c00640 1 -- 172.21.15.86:0/2766203794 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fda98046070 con 0x7fdaa806b980 2024-04-20T21:23:23.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.531+0000 7fdaa4c00640 1 --2- 172.21.15.86:0/2766203794 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fda8406c8f0 0x7fda8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:23.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.531+0000 7fdaa4c00640 1 -- 172.21.15.86:0/2766203794 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5795+0+0 (secure 0 0 0) 0x7fda980ad7a0 con 0x7fdaa806b980 2024-04-20T21:23:23.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.532+0000 7fda9f400640 1 --2- 172.21.15.86:0/2766203794 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fda8406c8f0 0x7fda8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:23.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.532+0000 7fda9f400640 1 --2- 172.21.15.86:0/2766203794 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fda8406c8f0 0x7fda8406edb0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fda90005880 tx=0x7fda90007a20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:23.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.536+0000 7fdaa4c00640 1 -- 172.21.15.86:0/2766203794 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fdaa814a490 con 0x7fdaa806b980 2024-04-20T21:23:23.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.737+0000 7fdaaee00640 1 -- 172.21.15.86:0/2766203794 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdaa814a6e0 con 0x7fdaa806b980 2024-04-20T21:23:23.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.738+0000 7fdaa4c00640 1 -- 172.21.15.86:0/2766203794 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) ==== 93+0+10 (secure 0 0 0) 0x7fda9807bb70 con 0x7fdaa806b980 2024-04-20T21:23:23.738 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:23:23.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.740+0000 7fdaaee00640 1 -- 172.21.15.86:0/2766203794 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fda8406c8f0 msgr2=0x7fda8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:23.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.740+0000 7fdaaee00640 1 --2- 172.21.15.86:0/2766203794 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fda8406c8f0 0x7fda8406edb0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fda90005880 tx=0x7fda90007a20 comp rx=0 tx=0).stop 2024-04-20T21:23:23.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.740+0000 7fdaaee00640 1 -- 172.21.15.86:0/2766203794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaa806b980 msgr2=0x7fdaa814d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:23.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.740+0000 7fdaaee00640 1 --2- 172.21.15.86:0/2766203794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaa806b980 0x7fdaa814d830 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fda98000c00 tx=0x7fda980046c0 comp rx=0 tx=0).stop 2024-04-20T21:23:23.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.740+0000 7fdaaee00640 1 -- 172.21.15.86:0/2766203794 shutdown_connections 2024-04-20T21:23:23.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.740+0000 7fdaaee00640 1 --2- 172.21.15.86:0/2766203794 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fda8406c8f0 0x7fda8406edb0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:23.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.740+0000 7fdaaee00640 1 --2- 172.21.15.86:0/2766203794 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdaa8155d30 0x7fdaa814fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:23.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.740+0000 7fdaaee00640 1 --2- 172.21.15.86:0/2766203794 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdaa80b52b0 0x7fdaa81f43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:23.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.740+0000 7fdaaee00640 1 --2- 172.21.15.86:0/2766203794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaa806b980 0x7fdaa814d830 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:23.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.741+0000 7fdaaee00640 1 -- 172.21.15.86:0/2766203794 >> 172.21.15.86:0/2766203794 conn(0x7fdaa80b2110 msgr2=0x7fdaa80b24f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:23.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.741+0000 7fdaaee00640 1 -- 172.21.15.86:0/2766203794 shutdown_connections 2024-04-20T21:23:23.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.741+0000 7fdaaee00640 1 -- 172.21.15.86:0/2766203794 wait complete. 2024-04-20T21:23:23.759 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:23:23.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.957+0000 7fb28f200640 1 Processor -- start 2024-04-20T21:23:23.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.958+0000 7fb28f200640 1 -- start start 2024-04-20T21:23:23.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.959+0000 7fb28f200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb2900d45a0 0x7fb2900d3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:23.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.959+0000 7fb28f200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2900cf120 con 0x7fb29006d570 2024-04-20T21:23:23.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.959+0000 7fb28f200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb29006de70 con 0x7fb29015ea70 2024-04-20T21:23:23.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.959+0000 7fb28f200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fb29006e150 con 0x7fb2900d45a0 2024-04-20T21:23:23.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.959+0000 7fb28e800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fb29006d570 0x7fb2900cf8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52112/0 (socket says 172.21.15.86:52112) 2024-04-20T21:23:23.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.959+0000 7fb28e800640 1 -- 172.21.15.86:0/4012332784 learned_addr learned my addr 172.21.15.86:0/4012332784 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:23:23.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.960+0000 7fb28de00640 1 --2- 172.21.15.86:0/4012332784 >> v2:172.21.15.144:3300/0 conn(0x7fb2900d45a0 0x7fb2900d3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:23.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.960+0000 7fb28ca00640 1 -- 172.21.15.86:0/4012332784 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 404738032 0 0) 0x7fb2900cf120 con 0x7fb29006d570 2024-04-20T21:23:23.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.960+0000 7fb28ca00640 1 -- 172.21.15.86:0/4012332784 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb274003520 con 0x7fb29006d570 2024-04-20T21:23:23.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.960+0000 7fb28ca00640 1 -- 172.21.15.86:0/4012332784 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 61459858 0 0) 0x7fb274003520 con 0x7fb29006d570 2024-04-20T21:23:23.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.960+0000 7fb28ca00640 1 -- 172.21.15.86:0/4012332784 >> v2:172.21.15.144:3300/0 conn(0x7fb2900d45a0 msgr2=0x7fb2900d3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:23.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.960+0000 7fb28ca00640 1 --2- 172.21.15.86:0/4012332784 >> v2:172.21.15.144:3300/0 conn(0x7fb2900d45a0 0x7fb2900d3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:23.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.960+0000 7fb28ca00640 1 -- 172.21.15.86:0/4012332784 >> v1:172.21.15.144:6789/0 conn(0x7fb29015ea70 legacy=0x7fb2900d4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:23.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.960+0000 7fb28ca00640 1 -- 172.21.15.86:0/4012332784 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb29006e430 con 0x7fb29006d570 2024-04-20T21:23:23.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.960+0000 7fb28ca00640 1 -- 172.21.15.86:0/4012332784 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fb288001690 con 0x7fb29006d570 2024-04-20T21:23:23.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.960+0000 7fb28ca00640 1 -- 172.21.15.86:0/4012332784 >> v1:172.21.15.86:6789/0 conn(0x7fb29006d570 legacy=0x7fb2900cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:23.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.960+0000 7fb28ca00640 1 --2- 172.21.15.86:0/4012332784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb274003a20 0x7fb274003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:23.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.960+0000 7fb28ca00640 1 --2- 172.21.15.86:0/4012332784 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb274004800 0x7fb274004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:23.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.960+0000 7fb28ca00640 1 --2- 172.21.15.86:0/4012332784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb274005230 0x7fb27400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:23.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.960+0000 7fb28ca00640 1 -- 172.21.15.86:0/4012332784 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb29006e150 con 0x7fb274003a20 2024-04-20T21:23:23.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.960+0000 7fb28ca00640 1 -- 172.21.15.86:0/4012332784 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb27400c190 con 0x7fb274005230 2024-04-20T21:23:23.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.960+0000 7fb28ca00640 1 -- 172.21.15.86:0/4012332784 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb27400c390 con 0x7fb274004800 2024-04-20T21:23:23.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.961+0000 7fb28de00640 1 --2- 172.21.15.86:0/4012332784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb274003a20 0x7fb274003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:23.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.961+0000 7fb28d400640 1 --2- 172.21.15.86:0/4012332784 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb274004800 0x7fb274004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:23.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.961+0000 7fb28de00640 1 -- 172.21.15.86:0/4012332784 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb274004800 msgr2=0x7fb274004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:23.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.961+0000 7fb28e800640 1 --2- 172.21.15.86:0/4012332784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb274005230 0x7fb27400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:23.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.961+0000 7fb28de00640 1 --2- 172.21.15.86:0/4012332784 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb274004800 0x7fb274004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:23.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.961+0000 7fb28de00640 1 -- 172.21.15.86:0/4012332784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb274005230 msgr2=0x7fb27400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:23.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.961+0000 7fb28de00640 1 --2- 172.21.15.86:0/4012332784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb274005230 0x7fb27400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:23.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.961+0000 7fb28de00640 1 -- 172.21.15.86:0/4012332784 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb274003520 con 0x7fb274003a20 2024-04-20T21:23:23.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.961+0000 7fb28e800640 1 -- 172.21.15.86:0/4012332784 reap_dead start 2024-04-20T21:23:23.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.962+0000 7fb28de00640 1 --2- 172.21.15.86:0/4012332784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb274003a20 0x7fb274003e40 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fb280009fc0 tx=0x7fb274005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aad2fdfd590da5c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:23.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.962+0000 7fb28ca00640 1 -- 172.21.15.86:0/4012332784 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb280039070 con 0x7fb274003a20 2024-04-20T21:23:23.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.962+0000 7fb28ca00640 1 -- 172.21.15.86:0/4012332784 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb28002cb40 con 0x7fb274003a20 2024-04-20T21:23:23.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.963+0000 7fb28ca00640 1 -- 172.21.15.86:0/4012332784 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb280009c10 con 0x7fb274003a20 2024-04-20T21:23:23.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.963+0000 7fb28f200640 1 -- 172.21.15.86:0/4012332784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb274003a20 msgr2=0x7fb274003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:23.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.963+0000 7fb28f200640 1 --2- 172.21.15.86:0/4012332784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb274003a20 0x7fb274003e40 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fb280009fc0 tx=0x7fb274005830 comp rx=0 tx=0).stop 2024-04-20T21:23:23.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.963+0000 7fb28f200640 1 -- 172.21.15.86:0/4012332784 shutdown_connections 2024-04-20T21:23:23.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.963+0000 7fb28f200640 1 --2- 172.21.15.86:0/4012332784 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb274003a20 0x7fb274003e40 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:23.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.963+0000 7fb28f200640 1 -- 172.21.15.86:0/4012332784 >> 172.21.15.86:0/4012332784 conn(0x7fb29009d9d0 msgr2=0x7fb29008bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:23.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.963+0000 7fb28f200640 1 -- 172.21.15.86:0/4012332784 shutdown_connections 2024-04-20T21:23:23.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.963+0000 7fb28f200640 1 -- 172.21.15.86:0/4012332784 wait complete. 2024-04-20T21:23:23.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.965+0000 7fb28f200640 1 Processor -- start 2024-04-20T21:23:23.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.965+0000 7fb28f200640 1 -- start start 2024-04-20T21:23:23.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.966+0000 7fb28f200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb274003a20 0x7fb290156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:23.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.966+0000 7fb28f200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2901585d0 0x7fb2900cde70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:23.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.966+0000 7fb28f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb290158a30 0x7fb2900d0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:23.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.966+0000 7fb28f200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb29006db20 con 0x7fb290158a30 2024-04-20T21:23:23.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.966+0000 7fb28f200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb29006d9a0 con 0x7fb274003a20 2024-04-20T21:23:23.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.966+0000 7fb28f200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb2900d3260 con 0x7fb2901585d0 2024-04-20T21:23:23.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.967+0000 7fb28e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb290158a30 0x7fb2900d0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:23.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.967+0000 7fb28e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb290158a30 0x7fb2900d0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52416/0 (socket says 172.21.15.86:52416) 2024-04-20T21:23:23.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.967+0000 7fb28e800640 1 -- 172.21.15.86:0/1543562727 learned_addr learned my addr 172.21.15.86:0/1543562727 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:23.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.967+0000 7fb28de00640 1 --2- 172.21.15.86:0/1543562727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb274003a20 0x7fb290156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:23.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.967+0000 7fb28d400640 1 --2- 172.21.15.86:0/1543562727 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2901585d0 0x7fb2900cde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:23.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.967+0000 7fb28e800640 1 -- 172.21.15.86:0/1543562727 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2901585d0 msgr2=0x7fb2900cde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:23.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.967+0000 7fb28e800640 1 --2- 172.21.15.86:0/1543562727 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2901585d0 0x7fb2900cde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:23.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.967+0000 7fb28e800640 1 -- 172.21.15.86:0/1543562727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb274003a20 msgr2=0x7fb290156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:23.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.967+0000 7fb28e800640 1 --2- 172.21.15.86:0/1543562727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb274003a20 0x7fb290156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:23.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.967+0000 7fb28e800640 1 -- 172.21.15.86:0/1543562727 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb29015fc30 con 0x7fb290158a30 2024-04-20T21:23:23.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.967+0000 7fb28de00640 1 --2- 172.21.15.86:0/1543562727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb274003a20 0x7fb290156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:23:23.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.967+0000 7fb28e800640 1 --2- 172.21.15.86:0/1543562727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb290158a30 0x7fb2900d0140 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fb2880011e0 tx=0x7fb28800d950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:23.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.968+0000 7fb286000640 1 -- 172.21.15.86:0/1543562727 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb288007440 con 0x7fb290158a30 2024-04-20T21:23:23.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.968+0000 7fb286000640 1 -- 172.21.15.86:0/1543562727 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2880075e0 con 0x7fb290158a30 2024-04-20T21:23:23.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.968+0000 7fb286000640 1 -- 172.21.15.86:0/1543562727 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb288004a20 con 0x7fb290158a30 2024-04-20T21:23:23.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.968+0000 7fb28f200640 1 -- 172.21.15.86:0/1543562727 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2900cc5f0 con 0x7fb290158a30 2024-04-20T21:23:23.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.968+0000 7fb28f200640 1 -- 172.21.15.86:0/1543562727 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2900cc8d0 con 0x7fb290158a30 2024-04-20T21:23:23.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.969+0000 7fb28f200640 1 -- 172.21.15.86:0/1543562727 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb29004fd90 con 0x7fb290158a30 2024-04-20T21:23:23.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.970+0000 7fb286000640 1 -- 172.21.15.86:0/1543562727 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb288004bc0 con 0x7fb290158a30 2024-04-20T21:23:23.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.971+0000 7fb286000640 1 --2- 172.21.15.86:0/1543562727 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb26c06c8f0 0x7fb26c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:23.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.971+0000 7fb286000640 1 -- 172.21.15.86:0/1543562727 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5795+0+0 (secure 0 0 0) 0x7fb28808b470 con 0x7fb290158a30 2024-04-20T21:23:23.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.971+0000 7fb28de00640 1 --2- 172.21.15.86:0/1543562727 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb26c06c8f0 0x7fb26c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:23.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.972+0000 7fb28de00640 1 --2- 172.21.15.86:0/1543562727 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb26c06c8f0 0x7fb26c06edb0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb280036000 tx=0x7fb28002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:23.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:23.975+0000 7fb286000640 1 -- 172.21.15.86:0/1543562727 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb288059710 con 0x7fb290158a30 2024-04-20T21:23:24.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.176+0000 7fb28f200640 1 -- 172.21.15.86:0/1543562727 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb2900b4070 con 0x7fb290158a30 2024-04-20T21:23:24.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.177+0000 7fb286000640 1 -- 172.21.15.86:0/1543562727 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v72) ==== 104+0+11 (secure 0 0 0) 0x7fb288059010 con 0x7fb290158a30 2024-04-20T21:23:24.181 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-20T21:23:24.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.180+0000 7fb28f200640 1 -- 172.21.15.86:0/1543562727 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb26c06c8f0 msgr2=0x7fb26c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:24.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.180+0000 7fb28f200640 1 --2- 172.21.15.86:0/1543562727 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb26c06c8f0 0x7fb26c06edb0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb280036000 tx=0x7fb28002e000 comp rx=0 tx=0).stop 2024-04-20T21:23:24.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.180+0000 7fb28f200640 1 -- 172.21.15.86:0/1543562727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb290158a30 msgr2=0x7fb2900d0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:24.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.180+0000 7fb28f200640 1 --2- 172.21.15.86:0/1543562727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb290158a30 0x7fb2900d0140 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fb2880011e0 tx=0x7fb28800d950 comp rx=0 tx=0).stop 2024-04-20T21:23:24.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.180+0000 7fb28f200640 1 -- 172.21.15.86:0/1543562727 shutdown_connections 2024-04-20T21:23:24.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.180+0000 7fb28f200640 1 --2- 172.21.15.86:0/1543562727 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb26c06c8f0 0x7fb26c06edb0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:24.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.180+0000 7fb28f200640 1 --2- 172.21.15.86:0/1543562727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb290158a30 0x7fb2900d0140 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:24.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.180+0000 7fb28f200640 1 --2- 172.21.15.86:0/1543562727 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2901585d0 0x7fb2900cde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:24.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.180+0000 7fb28f200640 1 --2- 172.21.15.86:0/1543562727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb274003a20 0x7fb290156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:24.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.180+0000 7fb28f200640 1 -- 172.21.15.86:0/1543562727 >> 172.21.15.86:0/1543562727 conn(0x7fb29009d9d0 msgr2=0x7fb2900b5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:24.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.180+0000 7fb28f200640 1 -- 172.21.15.86:0/1543562727 shutdown_connections 2024-04-20T21:23:24.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.180+0000 7fb28f200640 1 -- 172.21.15.86:0/1543562727 wait complete. 2024-04-20T21:23:24.201 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:23:24.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.423+0000 7fc306a00640 1 Processor -- start 2024-04-20T21:23:24.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.423+0000 7fc306a00640 1 -- start start 2024-04-20T21:23:24.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.425+0000 7fc306a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc30006c090 0x7fc3000cd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:24.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.425+0000 7fc306a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc3000b3410 0x7fc3000cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:24.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.425+0000 7fc306a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3000b3910 con 0x7fc300162ed0 2024-04-20T21:23:24.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.425+0000 7fc306a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc3000b3bf0 con 0x7fc30006c090 2024-04-20T21:23:24.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.425+0000 7fc306a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc3000b3ed0 con 0x7fc3000b3410 2024-04-20T21:23:24.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.425+0000 7fc2f7400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc30006c090 0x7fc3000cd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:24.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.426+0000 7fc2ffe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fc300162ed0 0x7fc30006b980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52124/0 (socket says 172.21.15.86:52124) 2024-04-20T21:23:24.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.426+0000 7fc2ffe00640 1 -- 172.21.15.86:0/2989515240 learned_addr learned my addr 172.21.15.86:0/2989515240 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:23:24.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.426+0000 7fc304a00640 1 --2- 172.21.15.86:0/2989515240 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc3000b3410 0x7fc3000cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:24.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.426+0000 7fc2f7400640 1 -- 172.21.15.86:0/2989515240 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc3000b3410 msgr2=0x7fc3000cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:24.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.426+0000 7fc2f7400640 1 --2- 172.21.15.86:0/2989515240 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc3000b3410 0x7fc3000cd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:24.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.426+0000 7fc2f7400640 1 -- 172.21.15.86:0/2989515240 >> v1:172.21.15.86:6789/0 conn(0x7fc300162ed0 legacy=0x7fc30006b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:24.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.426+0000 7fc2f7400640 1 -- 172.21.15.86:0/2989515240 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3000b41b0 con 0x7fc30006c090 2024-04-20T21:23:24.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.427+0000 7fc2f7400640 1 --2- 172.21.15.86:0/2989515240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc30006c090 0x7fc3000cd360 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fc2e80097a0 tx=0x7fc2e802b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c90d3aaf62de23f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:24.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.428+0000 7fc2ff400640 1 -- 172.21.15.86:0/2989515240 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e8037070 con 0x7fc30006c090 2024-04-20T21:23:24.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.428+0000 7fc2ff400640 1 -- 172.21.15.86:0/2989515240 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2e8030710 con 0x7fc30006c090 2024-04-20T21:23:24.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.428+0000 7fc2ff400640 1 -- 172.21.15.86:0/2989515240 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e8030ae0 con 0x7fc30006c090 2024-04-20T21:23:24.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.428+0000 7fc306a00640 1 -- 172.21.15.86:0/2989515240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc30006c090 msgr2=0x7fc3000cd360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:24.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.428+0000 7fc306a00640 1 --2- 172.21.15.86:0/2989515240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc30006c090 0x7fc3000cd360 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fc2e80097a0 tx=0x7fc2e802b710 comp rx=0 tx=0).stop 2024-04-20T21:23:24.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.429+0000 7fc306a00640 1 -- 172.21.15.86:0/2989515240 shutdown_connections 2024-04-20T21:23:24.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.429+0000 7fc306a00640 1 --2- 172.21.15.86:0/2989515240 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc3000b3410 0x7fc3000cd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:24.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.429+0000 7fc306a00640 1 --2- 172.21.15.86:0/2989515240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc30006c090 0x7fc3000cd360 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:24.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.429+0000 7fc306a00640 1 -- 172.21.15.86:0/2989515240 >> 172.21.15.86:0/2989515240 conn(0x7fc3000b2110 msgr2=0x7fc3000b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:24.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.429+0000 7fc306a00640 1 -- 172.21.15.86:0/2989515240 shutdown_connections 2024-04-20T21:23:24.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.429+0000 7fc306a00640 1 -- 172.21.15.86:0/2989515240 wait complete. 2024-04-20T21:23:24.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.431+0000 7fc306a00640 1 Processor -- start 2024-04-20T21:23:24.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.431+0000 7fc306a00640 1 -- start start 2024-04-20T21:23:24.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.432+0000 7fc306a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc30006c090 0x7fc300158080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:24.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.432+0000 7fc306a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3000b3410 0x7fc3001657b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:24.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.432+0000 7fc306a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc300162ed0 0x7fc300158a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:24.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.432+0000 7fc306a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc30006fca0 con 0x7fc3000b3410 2024-04-20T21:23:24.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.432+0000 7fc306a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc30006fb20 con 0x7fc30006c090 2024-04-20T21:23:24.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.432+0000 7fc306a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc30006e160 con 0x7fc300162ed0 2024-04-20T21:23:24.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.432+0000 7fc2f7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3000b3410 0x7fc3001657b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:24.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.432+0000 7fc2f7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3000b3410 0x7fc3001657b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52428/0 (socket says 172.21.15.86:52428) 2024-04-20T21:23:24.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.432+0000 7fc2f7400640 1 -- 172.21.15.86:0/1274596305 learned_addr learned my addr 172.21.15.86:0/1274596305 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:24.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.432+0000 7fc2ffe00640 1 --2- 172.21.15.86:0/1274596305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc30006c090 0x7fc300158080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:24.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.433+0000 7fc304a00640 1 --2- 172.21.15.86:0/1274596305 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc300162ed0 0x7fc300158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:24.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.433+0000 7fc2f7400640 1 -- 172.21.15.86:0/1274596305 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc300162ed0 msgr2=0x7fc300158a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:24.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.433+0000 7fc2f7400640 1 --2- 172.21.15.86:0/1274596305 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc300162ed0 0x7fc300158a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:24.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.433+0000 7fc2f7400640 1 -- 172.21.15.86:0/1274596305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc30006c090 msgr2=0x7fc300158080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:24.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.433+0000 7fc2f7400640 1 --2- 172.21.15.86:0/1274596305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc30006c090 0x7fc300158080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:24.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.433+0000 7fc2f7400640 1 -- 172.21.15.86:0/1274596305 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc30015f750 con 0x7fc3000b3410 2024-04-20T21:23:24.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.433+0000 7fc304a00640 1 --2- 172.21.15.86:0/1274596305 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc300162ed0 0x7fc300158a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:23:24.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.433+0000 7fc2f7400640 1 --2- 172.21.15.86:0/1274596305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3000b3410 0x7fc3001657b0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8035040 tx=0x7fc2e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:24.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.433+0000 7fc2fcc00640 1 -- 172.21.15.86:0/1274596305 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e8041030 con 0x7fc3000b3410 2024-04-20T21:23:24.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.433+0000 7fc2fcc00640 1 -- 172.21.15.86:0/1274596305 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2e8030510 con 0x7fc3000b3410 2024-04-20T21:23:24.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.433+0000 7fc2fcc00640 1 -- 172.21.15.86:0/1274596305 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e8037040 con 0x7fc3000b3410 2024-04-20T21:23:24.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.433+0000 7fc306a00640 1 -- 172.21.15.86:0/1274596305 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc30015fa30 con 0x7fc3000b3410 2024-04-20T21:23:24.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.434+0000 7fc306a00640 1 -- 172.21.15.86:0/1274596305 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc30015fff0 con 0x7fc3000b3410 2024-04-20T21:23:24.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.435+0000 7fc306a00640 1 -- 172.21.15.86:0/1274596305 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3000b0da0 con 0x7fc3000b3410 2024-04-20T21:23:24.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.436+0000 7fc2fcc00640 1 -- 172.21.15.86:0/1274596305 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc2e8044070 con 0x7fc3000b3410 2024-04-20T21:23:24.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.437+0000 7fc2fcc00640 1 --2- 172.21.15.86:0/1274596305 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc2dc06c8f0 0x7fc2dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:24.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.437+0000 7fc2fcc00640 1 -- 172.21.15.86:0/1274596305 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5795+0+0 (secure 0 0 0) 0x7fc2e80ad580 con 0x7fc3000b3410 2024-04-20T21:23:24.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.437+0000 7fc2ffe00640 1 --2- 172.21.15.86:0/1274596305 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc2dc06c8f0 0x7fc2dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:24.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.438+0000 7fc2ffe00640 1 --2- 172.21.15.86:0/1274596305 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc2dc06c8f0 0x7fc2dc06edb0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc2f00012e0 tx=0x7fc2f0005b70 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:24.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.441+0000 7fc2fcc00640 1 -- 172.21.15.86:0/1274596305 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc3000b0da0 con 0x7fc3000b3410 2024-04-20T21:23:24.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.639+0000 7fc306a00640 1 -- 172.21.15.86:0/1274596305 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc3000689e0 con 0x7fc3000b3410 2024-04-20T21:23:24.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.640+0000 7fc2fcc00640 1 -- 172.21.15.86:0/1274596305 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v72) ==== 100+0+11 (secure 0 0 0) 0x7fc2e807b9e0 con 0x7fc3000b3410 2024-04-20T21:23:24.642 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-20T21:23:24.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.643+0000 7fc306a00640 1 -- 172.21.15.86:0/1274596305 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc2dc06c8f0 msgr2=0x7fc2dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:24.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.643+0000 7fc306a00640 1 --2- 172.21.15.86:0/1274596305 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc2dc06c8f0 0x7fc2dc06edb0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc2f00012e0 tx=0x7fc2f0005b70 comp rx=0 tx=0).stop 2024-04-20T21:23:24.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.643+0000 7fc306a00640 1 -- 172.21.15.86:0/1274596305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3000b3410 msgr2=0x7fc3001657b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:24.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.643+0000 7fc306a00640 1 --2- 172.21.15.86:0/1274596305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3000b3410 0x7fc3001657b0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8035040 tx=0x7fc2e80046c0 comp rx=0 tx=0).stop 2024-04-20T21:23:24.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.643+0000 7fc306a00640 1 -- 172.21.15.86:0/1274596305 shutdown_connections 2024-04-20T21:23:24.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.643+0000 7fc306a00640 1 --2- 172.21.15.86:0/1274596305 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc2dc06c8f0 0x7fc2dc06edb0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:24.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.643+0000 7fc306a00640 1 --2- 172.21.15.86:0/1274596305 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc300162ed0 0x7fc300158a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:24.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.643+0000 7fc306a00640 1 --2- 172.21.15.86:0/1274596305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3000b3410 0x7fc3001657b0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:24.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.643+0000 7fc306a00640 1 --2- 172.21.15.86:0/1274596305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc30006c090 0x7fc300158080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:24.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.643+0000 7fc306a00640 1 -- 172.21.15.86:0/1274596305 >> 172.21.15.86:0/1274596305 conn(0x7fc3000b2110 msgr2=0x7fc3000b0080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:24.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.643+0000 7fc306a00640 1 -- 172.21.15.86:0/1274596305 shutdown_connections 2024-04-20T21:23:24.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.643+0000 7fc306a00640 1 -- 172.21.15.86:0/1274596305 wait complete. 2024-04-20T21:23:24.662 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T21:23:24.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.858+0000 7fd788600640 1 Processor -- start 2024-04-20T21:23:24.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.858+0000 7fd788600640 1 -- start start 2024-04-20T21:23:24.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.860+0000 7fd788600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd7800b3330 0x7fd7800d0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:24.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.860+0000 7fd788600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7800d1220 con 0x7fd780162bd0 2024-04-20T21:23:24.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.860+0000 7fd788600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7800b3830 con 0x7fd78006e1e0 2024-04-20T21:23:24.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.860+0000 7fd788600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd7800b3fc0 con 0x7fd7800b3330 2024-04-20T21:23:24.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.861+0000 7fd785c00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd780162bd0 0x7fd7800cd0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52134/0 (socket says 172.21.15.86:52134) 2024-04-20T21:23:24.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.861+0000 7fd785c00640 1 -- 172.21.15.86:0/829834169 learned_addr learned my addr 172.21.15.86:0/829834169 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:23:24.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.861+0000 7fd785200640 1 --2- 172.21.15.86:0/829834169 >> v2:172.21.15.86:3300/0 conn(0x7fd7800b3330 0x7fd7800d0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:24.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.861+0000 7fd77fe00640 1 -- 172.21.15.86:0/829834169 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1298571987 0 0) 0x7fd7800d1220 con 0x7fd780162bd0 2024-04-20T21:23:24.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.861+0000 7fd77fe00640 1 -- 172.21.15.86:0/829834169 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd768003520 con 0x7fd780162bd0 2024-04-20T21:23:24.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.861+0000 7fd785200640 1 -- 172.21.15.86:0/829834169 >> v1:172.21.15.144:6789/0 conn(0x7fd78006e1e0 legacy=0x7fd78006e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:24.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.861+0000 7fd785200640 1 -- 172.21.15.86:0/829834169 >> v1:172.21.15.86:6789/0 conn(0x7fd780162bd0 legacy=0x7fd7800cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:24.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.861+0000 7fd785200640 1 -- 172.21.15.86:0/829834169 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7800b42a0 con 0x7fd7800b3330 2024-04-20T21:23:24.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.862+0000 7fd785200640 1 --2- 172.21.15.86:0/829834169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7800b3330 0x7fd7800d0ce0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fd76c0097a0 tx=0x7fd76c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e43614ffa4648c94 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:24.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.863+0000 7fd77fe00640 1 -- 172.21.15.86:0/829834169 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76c0315a0 con 0x7fd7800b3330 2024-04-20T21:23:24.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.863+0000 7fd77fe00640 1 -- 172.21.15.86:0/829834169 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd76c037070 con 0x7fd7800b3330 2024-04-20T21:23:24.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.863+0000 7fd77fe00640 1 -- 172.21.15.86:0/829834169 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76c031960 con 0x7fd7800b3330 2024-04-20T21:23:24.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.863+0000 7fd788600640 1 -- 172.21.15.86:0/829834169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7800b3330 msgr2=0x7fd7800d0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:24.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.863+0000 7fd788600640 1 --2- 172.21.15.86:0/829834169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7800b3330 0x7fd7800d0ce0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fd76c0097a0 tx=0x7fd76c02b840 comp rx=0 tx=0).stop 2024-04-20T21:23:24.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.863+0000 7fd788600640 1 -- 172.21.15.86:0/829834169 shutdown_connections 2024-04-20T21:23:24.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.863+0000 7fd788600640 1 --2- 172.21.15.86:0/829834169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7800b3330 0x7fd7800d0ce0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:24.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.863+0000 7fd788600640 1 -- 172.21.15.86:0/829834169 >> 172.21.15.86:0/829834169 conn(0x7fd78006b980 msgr2=0x7fd78006bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:24.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.863+0000 7fd788600640 1 -- 172.21.15.86:0/829834169 shutdown_connections 2024-04-20T21:23:24.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.863+0000 7fd788600640 1 -- 172.21.15.86:0/829834169 wait complete. 2024-04-20T21:23:24.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.865+0000 7fd788600640 1 Processor -- start 2024-04-20T21:23:24.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.865+0000 7fd788600640 1 -- start start 2024-04-20T21:23:24.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.866+0000 7fd788600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd78006e1e0 0x7fd7800b0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:24.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.866+0000 7fd788600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7800b3330 0x7fd780164300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:24.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.866+0000 7fd788600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd780162bd0 0x7fd780165bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:24.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.866+0000 7fd788600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd780163130 con 0x7fd7800b3330 2024-04-20T21:23:24.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.866+0000 7fd788600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd780162fb0 con 0x7fd78006e1e0 2024-04-20T21:23:24.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.866+0000 7fd788600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd780072040 con 0x7fd780162bd0 2024-04-20T21:23:24.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.866+0000 7fd785200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7800b3330 0x7fd780164300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:24.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.867+0000 7fd785200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7800b3330 0x7fd780164300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52444/0 (socket says 172.21.15.86:52444) 2024-04-20T21:23:24.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.867+0000 7fd785200640 1 -- 172.21.15.86:0/2366657379 learned_addr learned my addr 172.21.15.86:0/2366657379 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:24.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.867+0000 7fd785c00640 1 --2- 172.21.15.86:0/2366657379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd78006e1e0 0x7fd7800b0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:24.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.867+0000 7fd786600640 1 --2- 172.21.15.86:0/2366657379 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd780162bd0 0x7fd780165bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:24.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.867+0000 7fd785200640 1 -- 172.21.15.86:0/2366657379 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd780162bd0 msgr2=0x7fd780165bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:24.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.867+0000 7fd785200640 1 --2- 172.21.15.86:0/2366657379 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd780162bd0 0x7fd780165bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:24.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.867+0000 7fd785200640 1 -- 172.21.15.86:0/2366657379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd78006e1e0 msgr2=0x7fd7800b0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:24.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.867+0000 7fd785200640 1 --2- 172.21.15.86:0/2366657379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd78006e1e0 0x7fd7800b0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:24.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.867+0000 7fd785200640 1 -- 172.21.15.86:0/2366657379 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd780164b30 con 0x7fd7800b3330 2024-04-20T21:23:24.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.867+0000 7fd786600640 1 --2- 172.21.15.86:0/2366657379 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd780162bd0 0x7fd780165bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:23:24.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.867+0000 7fd785200640 1 --2- 172.21.15.86:0/2366657379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7800b3330 0x7fd780164300 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fd76c000c00 tx=0x7fd76c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:24.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.868+0000 7fd77d600640 1 -- 172.21.15.86:0/2366657379 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76c032630 con 0x7fd7800b3330 2024-04-20T21:23:24.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.868+0000 7fd77d600640 1 -- 172.21.15.86:0/2366657379 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd76c0327d0 con 0x7fd7800b3330 2024-04-20T21:23:24.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.868+0000 7fd788600640 1 -- 172.21.15.86:0/2366657379 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7801650f0 con 0x7fd7800b3330 2024-04-20T21:23:24.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.868+0000 7fd77d600640 1 -- 172.21.15.86:0/2366657379 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76c032bc0 con 0x7fd7800b3330 2024-04-20T21:23:24.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.868+0000 7fd788600640 1 -- 172.21.15.86:0/2366657379 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7801656b0 con 0x7fd7800b3330 2024-04-20T21:23:24.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.869+0000 7fd788600640 1 -- 172.21.15.86:0/2366657379 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7801648a0 con 0x7fd7800b3330 2024-04-20T21:23:24.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.870+0000 7fd77d600640 1 -- 172.21.15.86:0/2366657379 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd76c03b3f0 con 0x7fd7800b3330 2024-04-20T21:23:24.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.871+0000 7fd77d600640 1 --2- 172.21.15.86:0/2366657379 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd76006c8f0 0x7fd76006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:24.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.871+0000 7fd77d600640 1 -- 172.21.15.86:0/2366657379 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5795+0+0 (secure 0 0 0) 0x7fd76c02f030 con 0x7fd7800b3330 2024-04-20T21:23:24.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.871+0000 7fd785c00640 1 --2- 172.21.15.86:0/2366657379 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd76006c8f0 0x7fd76006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:24.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.872+0000 7fd785c00640 1 --2- 172.21.15.86:0/2366657379 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd76006c8f0 0x7fd76006edb0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd774005c00 tx=0x7fd774009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:24.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:24.875+0000 7fd77d600640 1 -- 172.21.15.86:0/2366657379 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd76c040070 con 0x7fd7800b3330 2024-04-20T21:23:25.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:25.089+0000 7fd788600640 1 -- 172.21.15.86:0/2366657379 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd7800689e0 con 0x7fd7800b3330 2024-04-20T21:23:25.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:25.089+0000 7fd77d600640 1 -- 172.21.15.86:0/2366657379 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v72) ==== 80+0+120 (secure 0 0 0) 0x7fd76c07b710 con 0x7fd7800b3330 2024-04-20T21:23:25.090 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6837/202143875 2024-04-21T21:23:12.079572+0000 2024-04-20T21:23:25.090 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6835/202143875 2024-04-21T21:23:12.079572+0000 2024-04-20T21:23:25.090 INFO:teuthology.orchestra.run.smithi086.stderr:listed 2 entries 2024-04-20T21:23:25.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:25.092+0000 7fd788600640 1 -- 172.21.15.86:0/2366657379 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd76006c8f0 msgr2=0x7fd76006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:25.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:25.092+0000 7fd788600640 1 --2- 172.21.15.86:0/2366657379 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd76006c8f0 0x7fd76006edb0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd774005c00 tx=0x7fd774009040 comp rx=0 tx=0).stop 2024-04-20T21:23:25.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:25.092+0000 7fd788600640 1 -- 172.21.15.86:0/2366657379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7800b3330 msgr2=0x7fd780164300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:25.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:25.092+0000 7fd788600640 1 --2- 172.21.15.86:0/2366657379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7800b3330 0x7fd780164300 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fd76c000c00 tx=0x7fd76c0046c0 comp rx=0 tx=0).stop 2024-04-20T21:23:25.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:25.092+0000 7fd788600640 1 -- 172.21.15.86:0/2366657379 shutdown_connections 2024-04-20T21:23:25.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:25.092+0000 7fd788600640 1 --2- 172.21.15.86:0/2366657379 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd76006c8f0 0x7fd76006edb0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:25.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:25.092+0000 7fd788600640 1 --2- 172.21.15.86:0/2366657379 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd780162bd0 0x7fd780165bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:25.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:25.092+0000 7fd788600640 1 --2- 172.21.15.86:0/2366657379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7800b3330 0x7fd780164300 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:25.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:25.092+0000 7fd788600640 1 --2- 172.21.15.86:0/2366657379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd78006e1e0 0x7fd7800b0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:25.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:25.092+0000 7fd788600640 1 -- 172.21.15.86:0/2366657379 >> 172.21.15.86:0/2366657379 conn(0x7fd78006b980 msgr2=0x7fd780161120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:25.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:25.093+0000 7fd788600640 1 -- 172.21.15.86:0/2366657379 shutdown_connections 2024-04-20T21:23:25.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:25.093+0000 7fd788600640 1 -- 172.21.15.86:0/2366657379 wait complete. 2024-04-20T21:23:25.111 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi086.front.sepia.ceph.com... 2024-04-20T21:23:25.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:23:25.111 DEBUG:teuthology.orchestra.run.smithi086:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-20T21:23:25.187 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[48683]: fuse finished with error 0 and tester_r 0 2024-04-20T21:23:25.191 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:23:31.293 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-20T21:23:31.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:23:31.293 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-20T21:23:31.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:23:31.353 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir recursive 2024-04-20T21:23:31.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.563+0000 7fcc70a00640 1 Processor -- start 2024-04-20T21:23:31.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.564+0000 7fcc70a00640 1 -- start start 2024-04-20T21:23:31.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.565+0000 7fcc70a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc6c0720b0 0x7fcc6c162cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:31.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.565+0000 7fcc70a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcc6c072490 0x7fcc6c15ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:31.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.565+0000 7fcc70a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc6c15f7c0 con 0x7fcc6c15f2d0 2024-04-20T21:23:31.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.565+0000 7fcc70a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fcc6c15fd80 con 0x7fcc6c072490 2024-04-20T21:23:31.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.565+0000 7fcc70a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fcc6c15faa0 con 0x7fcc6c0720b0 2024-04-20T21:23:31.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.566+0000 7fcc63e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcc6c072490 0x7fcc6c15ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:31.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.566+0000 7fcc63e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcc6c072490 0x7fcc6c15ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40638/0 (socket says 172.21.15.86:40638) 2024-04-20T21:23:31.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.566+0000 7fcc63e00640 1 -- 172.21.15.86:0/2130950254 learned_addr learned my addr 172.21.15.86:0/2130950254 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:31.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.566+0000 7fcc63e00640 1 -- 172.21.15.86:0/2130950254 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc6c0720b0 msgr2=0x7fcc6c162cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:31.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.566+0000 7fcc6b400640 1 --2- 172.21.15.86:0/2130950254 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc6c0720b0 0x7fcc6c162cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:31.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.566+0000 7fcc63e00640 1 --2- 172.21.15.86:0/2130950254 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc6c0720b0 0x7fcc6c162cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.566+0000 7fcc63e00640 1 -- 172.21.15.86:0/2130950254 >> v1:172.21.15.144:6789/0 conn(0x7fcc6c15f2d0 legacy=0x7fcc6c0d0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:31.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.566+0000 7fcc63e00640 1 -- 172.21.15.86:0/2130950254 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc6c1624b0 con 0x7fcc6c072490 2024-04-20T21:23:31.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.567+0000 7fcc63e00640 1 --2- 172.21.15.86:0/2130950254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc6c072490 0x7fcc6c15ed90 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c15ff00 tx=0x7fcc5402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f9c7940356ce0f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:31.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.568+0000 7fcc6aa00640 1 -- 172.21.15.86:0/2130950254 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc540317c0 con 0x7fcc6c072490 2024-04-20T21:23:31.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.568+0000 7fcc6aa00640 1 -- 172.21.15.86:0/2130950254 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc54037070 con 0x7fcc6c072490 2024-04-20T21:23:31.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.568+0000 7fcc6aa00640 1 -- 172.21.15.86:0/2130950254 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc54031b80 con 0x7fcc6c072490 2024-04-20T21:23:31.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.568+0000 7fcc70a00640 1 -- 172.21.15.86:0/2130950254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc6c072490 msgr2=0x7fcc6c15ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:31.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.568+0000 7fcc70a00640 1 --2- 172.21.15.86:0/2130950254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc6c072490 0x7fcc6c15ed90 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c15ff00 tx=0x7fcc5402b860 comp rx=0 tx=0).stop 2024-04-20T21:23:31.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.568+0000 7fcc70a00640 1 -- 172.21.15.86:0/2130950254 shutdown_connections 2024-04-20T21:23:31.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.568+0000 7fcc70a00640 1 --2- 172.21.15.86:0/2130950254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc6c072490 0x7fcc6c15ed90 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.568+0000 7fcc70a00640 1 --2- 172.21.15.86:0/2130950254 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc6c0720b0 0x7fcc6c162cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.568+0000 7fcc70a00640 1 -- 172.21.15.86:0/2130950254 >> 172.21.15.86:0/2130950254 conn(0x7fcc6c0b2180 msgr2=0x7fcc6c0b2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:31.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.568+0000 7fcc70a00640 1 -- 172.21.15.86:0/2130950254 shutdown_connections 2024-04-20T21:23:31.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.569+0000 7fcc70a00640 1 -- 172.21.15.86:0/2130950254 wait complete. 2024-04-20T21:23:31.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.570+0000 7fcc70a00640 1 Processor -- start 2024-04-20T21:23:31.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.570+0000 7fcc70a00640 1 -- start start 2024-04-20T21:23:31.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.571+0000 7fcc70a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc6c0720b0 0x7fcc6c09db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:31.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.571+0000 7fcc70a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc6c072490 0x7fcc6c144110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:31.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.571+0000 7fcc70a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc6c15f2d0 0x7fcc6c159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:31.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.571+0000 7fcc70a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6c0b5090 con 0x7fcc6c15f2d0 2024-04-20T21:23:31.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.571+0000 7fcc70a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6c1631f0 con 0x7fcc6c072490 2024-04-20T21:23:31.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.571+0000 7fcc70a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fcc6c0b5210 con 0x7fcc6c0720b0 2024-04-20T21:23:31.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.572+0000 7fcc6be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc6c15f2d0 0x7fcc6c159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:31.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.572+0000 7fcc6be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc6c15f2d0 0x7fcc6c159620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40644/0 (socket says 172.21.15.86:40644) 2024-04-20T21:23:31.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.572+0000 7fcc6be00640 1 -- 172.21.15.86:0/3649014246 learned_addr learned my addr 172.21.15.86:0/3649014246 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:31.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.572+0000 7fcc63e00640 1 --2- 172.21.15.86:0/3649014246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc6c072490 0x7fcc6c144110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:31.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.572+0000 7fcc6b400640 1 --2- 172.21.15.86:0/3649014246 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc6c0720b0 0x7fcc6c09db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:31.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.572+0000 7fcc6be00640 1 -- 172.21.15.86:0/3649014246 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc6c0720b0 msgr2=0x7fcc6c09db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:31.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.572+0000 7fcc6be00640 1 --2- 172.21.15.86:0/3649014246 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc6c0720b0 0x7fcc6c09db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.572+0000 7fcc6be00640 1 -- 172.21.15.86:0/3649014246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc6c072490 msgr2=0x7fcc6c144110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:31.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.572+0000 7fcc6be00640 1 --2- 172.21.15.86:0/3649014246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc6c072490 0x7fcc6c144110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.572+0000 7fcc6be00640 1 -- 172.21.15.86:0/3649014246 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc6c144650 con 0x7fcc6c15f2d0 2024-04-20T21:23:31.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.572+0000 7fcc63e00640 1 --2- 172.21.15.86:0/3649014246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc6c072490 0x7fcc6c144110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:23:31.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.572+0000 7fcc6be00640 1 --2- 172.21.15.86:0/3649014246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc6c15f2d0 0x7fcc6c159620 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fcc6400c780 tx=0x7fcc6400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:31.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.572+0000 7fcc63400640 1 -- 172.21.15.86:0/3649014246 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc64004b60 con 0x7fcc6c15f2d0 2024-04-20T21:23:31.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.573+0000 7fcc63400640 1 -- 172.21.15.86:0/3649014246 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc64004d00 con 0x7fcc6c15f2d0 2024-04-20T21:23:31.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.573+0000 7fcc63400640 1 -- 172.21.15.86:0/3649014246 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6400aa90 con 0x7fcc6c15f2d0 2024-04-20T21:23:31.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.573+0000 7fcc70a00640 1 -- 172.21.15.86:0/3649014246 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc6c1470e0 con 0x7fcc6c15f2d0 2024-04-20T21:23:31.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.573+0000 7fcc70a00640 1 -- 172.21.15.86:0/3649014246 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc6c146840 con 0x7fcc6c15f2d0 2024-04-20T21:23:31.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.575+0000 7fcc63400640 1 -- 172.21.15.86:0/3649014246 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcc6400ac30 con 0x7fcc6c15f2d0 2024-04-20T21:23:31.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.576+0000 7fcc63400640 1 --2- 172.21.15.86:0/3649014246 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcc4806c8f0 0x7fcc4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:31.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.576+0000 7fcc63400640 1 -- 172.21.15.86:0/3649014246 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5795+0+0 (secure 0 0 0) 0x7fcc6408be60 con 0x7fcc6c15f2d0 2024-04-20T21:23:31.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.576+0000 7fcc6b400640 1 --2- 172.21.15.86:0/3649014246 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcc4806c8f0 0x7fcc4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:31.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.576+0000 7fcc74d6b740 1 Processor -- start 2024-04-20T21:23:31.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.577+0000 7fcc74d6b740 1 -- start start 2024-04-20T21:23:31.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.577+0000 7fcc6b400640 1 --2- 172.21.15.86:0/3649014246 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcc4806c8f0 0x7fcc4806edb0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c00b7c0 tx=0x7fcc5c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:31.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.578+0000 7fcc74d6b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591453391e0 0x55914530c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:31.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.578+0000 7fcc74d6b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591453395c0 0x55914533bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:31.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.578+0000 7fcc74d6b740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55914530c6c0 0x55914533c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:31.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.578+0000 7fcc74d6b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559145276290 con 0x5591453395c0 2024-04-20T21:23:31.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.578+0000 7fcc74d6b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55914521c130 con 0x5591453391e0 2024-04-20T21:23:31.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.578+0000 7fcc74d6b740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x559145272250 con 0x55914530c6c0 2024-04-20T21:23:31.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.578+0000 7fcc63e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591453395c0 0x55914533bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:31.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.578+0000 7fcc63e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591453395c0 0x55914533bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40654/0 (socket says 172.21.15.86:40654) 2024-04-20T21:23:31.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.578+0000 7fcc63e00640 1 -- 172.21.15.86:0/2585051007 learned_addr learned my addr 172.21.15.86:0/2585051007 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:31.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.578+0000 7fcc6be00640 1 --2- 172.21.15.86:0/2585051007 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55914530c6c0 0x55914533c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:31.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.578+0000 7fcc63e00640 1 -- 172.21.15.86:0/2585051007 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55914530c6c0 msgr2=0x55914533c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:31.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.578+0000 7fcc63e00640 1 --2- 172.21.15.86:0/2585051007 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55914530c6c0 0x55914533c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.578+0000 7fcc6b400640 1 --2- 172.21.15.86:0/2585051007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591453391e0 0x55914530c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:31.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.578+0000 7fcc63e00640 1 -- 172.21.15.86:0/2585051007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591453391e0 msgr2=0x55914530c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:31.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.578+0000 7fcc63e00640 1 --2- 172.21.15.86:0/2585051007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591453391e0 0x55914530c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.578+0000 7fcc63e00640 1 -- 172.21.15.86:0/2585051007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559145054a20 con 0x5591453395c0 2024-04-20T21:23:31.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.579+0000 7fcc63e00640 1 --2- 172.21.15.86:0/2585051007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591453395c0 0x55914533bf70 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fcc5402b8a0 tx=0x7fcc54030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de9e6635681a60ce server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:31.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.579+0000 7fcc5b400640 1 -- 172.21.15.86:0/2585051007 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc54031c90 con 0x5591453395c0 2024-04-20T21:23:31.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.579+0000 7fcc5b400640 1 -- 172.21.15.86:0/2585051007 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc54037040 con 0x5591453395c0 2024-04-20T21:23:31.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.579+0000 7fcc5b400640 1 -- 172.21.15.86:0/2585051007 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc54032450 con 0x5591453395c0 2024-04-20T21:23:31.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.580+0000 7fcc74d6b740 1 -- 172.21.15.86:0/2585051007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591453395c0 msgr2=0x55914533bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:31.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.580+0000 7fcc74d6b740 1 --2- 172.21.15.86:0/2585051007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591453395c0 0x55914533bf70 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fcc5402b8a0 tx=0x7fcc54030000 comp rx=0 tx=0).stop 2024-04-20T21:23:31.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.580+0000 7fcc74d6b740 1 -- 172.21.15.86:0/2585051007 shutdown_connections 2024-04-20T21:23:31.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.580+0000 7fcc74d6b740 1 --2- 172.21.15.86:0/2585051007 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55914530c6c0 0x55914533c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.580+0000 7fcc74d6b740 1 --2- 172.21.15.86:0/2585051007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591453395c0 0x55914533bf70 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.580+0000 7fcc74d6b740 1 --2- 172.21.15.86:0/2585051007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591453391e0 0x55914530c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.580+0000 7fcc74d6b740 1 -- 172.21.15.86:0/2585051007 >> 172.21.15.86:0/2585051007 conn(0x55914530e3f0 msgr2=0x559145356f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:31.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.580+0000 7fcc74d6b740 1 -- 172.21.15.86:0/2585051007 shutdown_connections 2024-04-20T21:23:31.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.580+0000 7fcc74d6b740 1 -- 172.21.15.86:0/2585051007 wait complete. 2024-04-20T21:23:31.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.581+0000 7fcc74d6b740 1 -- start start 2024-04-20T21:23:31.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.581+0000 7fcc74d6b740 1 Processor -- start 2024-04-20T21:23:31.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.582+0000 7fcc74d6b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591453391e0 0x55914533fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:31.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.582+0000 7fcc74d6b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591453395c0 0x55914533da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:31.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.582+0000 7fcc74d6b740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55914533dfd0 0x55914536aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:31.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.582+0000 7fcc74d6b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559145310a90 con 0x5591453395c0 2024-04-20T21:23:31.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.582+0000 7fcc74d6b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55914522f6d0 con 0x5591453391e0 2024-04-20T21:23:31.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.582+0000 7fcc74d6b740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5591453132d0 con 0x55914533dfd0 2024-04-20T21:23:31.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.582+0000 7fcc74d6b740 10 client.0 _ll_register_callbacks cb 0x55914526ca70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:23:31.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.582+0000 7fcc63e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591453395c0 0x55914533da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:31.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.582+0000 7fcc63e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591453395c0 0x55914533da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40668/0 (socket says 172.21.15.86:40668) 2024-04-20T21:23:31.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.582+0000 7fcc63e00640 1 -- 172.21.15.86:0/3700482733 learned_addr learned my addr 172.21.15.86:0/3700482733 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:31.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.582+0000 7fcc6b400640 1 --2- 172.21.15.86:0/3700482733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591453391e0 0x55914533fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:31.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.583+0000 7fcc6be00640 1 --2- 172.21.15.86:0/3700482733 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55914533dfd0 0x55914536aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:31.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.583+0000 7fcc63e00640 1 -- 172.21.15.86:0/3700482733 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55914533dfd0 msgr2=0x55914536aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:31.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.583+0000 7fcc63e00640 1 --2- 172.21.15.86:0/3700482733 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55914533dfd0 0x55914536aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.583+0000 7fcc63e00640 1 -- 172.21.15.86:0/3700482733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591453391e0 msgr2=0x55914533fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:31.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.583+0000 7fcc63e00640 1 --2- 172.21.15.86:0/3700482733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591453391e0 0x55914533fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.583+0000 7fcc63e00640 1 -- 172.21.15.86:0/3700482733 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559144f48310 con 0x5591453395c0 2024-04-20T21:23:31.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.583+0000 7fcc6b400640 1 --2- 172.21.15.86:0/3700482733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591453391e0 0x55914533fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:23:31.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.583+0000 7fcc63e00640 1 --2- 172.21.15.86:0/3700482733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591453395c0 0x55914533da90 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fcc5402be70 tx=0x7fcc54004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aeb40fd0a439394 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:31.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.583+0000 7fcc4fe00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:23:31.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.583+0000 7fcc4fe00640 1 -- 172.21.15.86:0/3700482733 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc54054020 con 0x5591453395c0 2024-04-20T21:23:31.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.583+0000 7fcc4fe00640 1 -- 172.21.15.86:0/3700482733 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(72..72 src has 1..72) ==== 5795+0+0 (secure 0 0 0) 0x7fcc54003810 con 0x5591453395c0 2024-04-20T21:23:31.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.583+0000 7fcc74d6b740 1 -- 172.21.15.86:0/3700482733 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc54037040 con 0x5591453395c0 2024-04-20T21:23:31.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.583+0000 7fcc4fe00640 1 -- 172.21.15.86:0/3700482733 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fcc5405e050 con 0x5591453395c0 2024-04-20T21:23:31.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.584+0000 7fcc74d6b740 10 client.7131 fetch_fsmap learned FSMap version 49 2024-04-20T21:23:31.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.584+0000 7fcc74d6b740 1 -- 172.21.15.86:0/3700482733 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) -- 0x5591450b51b0 con 0x5591453395c0 2024-04-20T21:23:31.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.584+0000 7fcc4fe00640 1 -- 172.21.15.86:0/3700482733 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc54037040 con 0x5591453395c0 2024-04-20T21:23:31.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.584+0000 7fcc4fe00640 1 -- 172.21.15.86:0/3700482733 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 49) ==== 3495+0+0 (secure 0 0 0) 0x7fcc54061020 con 0x5591453395c0 2024-04-20T21:23:31.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.584+0000 7fcc74d6b740 10 client.7131 fetch_fsmap finished waiting for FSMap version 49 2024-04-20T21:23:31.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.584+0000 7fcc74d6b740 10 client.7131 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-20T21:23:31.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.584+0000 7fcc74d6b740 20 client.7131 populate_metadata read hostname 'smithi086' 2024-04-20T21:23:31.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.584+0000 7fcc74d6b740 1 --2- 172.21.15.86:0/3700482733 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x559145331300 0x559145331760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:31.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.584+0000 7fcc74d6b740 4 client.7131 mds_command: new command op to 6987 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:23:31.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.584+0000 7fcc74d6b740 1 -- 172.21.15.86:0/3700482733 --> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559145283cb0 con 0x559145331300 2024-04-20T21:23:31.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.584+0000 7fcc6b400640 1 --2- 172.21.15.86:0/3700482733 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x559145331300 0x559145331760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:31.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.585+0000 7fcc6b400640 1 --2- 172.21.15.86:0/3700482733 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x559145331300 0x559145331760 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6682a4047bea490 server_cookie=2342a445cb53b40e in_seq=0 out_seq=0 2024-04-20T21:23:31.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.585+0000 7fcc4fe00640 10 client.7131 ms_handle_connect on v2:172.21.15.86:6835/2254117576 2024-04-20T21:23:31.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.585+0000 7fcc4fe00640 1 -- 172.21.15.86:0/3700482733 <== mds.0 v2:172.21.15.86:6835/2254117576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559145283cb0 con 0x559145331300 2024-04-20T21:23:31.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.585+0000 7fcc4fe00640 10 client.7131 handle_command_reply: tid=0 2024-04-20T21:23:31.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.586+0000 7fcc74d6b740 1 client.7131 shutdown 2024-04-20T21:23:31.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.586+0000 7fcc74d6b740 1 -- 172.21.15.86:0/3700482733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591453395c0 msgr2=0x55914533da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:31.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.586+0000 7fcc74d6b740 1 --2- 172.21.15.86:0/3700482733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591453395c0 0x55914533da90 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fcc5402be70 tx=0x7fcc54004730 comp rx=0 tx=0).stop 2024-04-20T21:23:31.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.586+0000 7fcc74d6b740 1 -- 172.21.15.86:0/3700482733 shutdown_connections 2024-04-20T21:23:31.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.586+0000 7fcc74d6b740 1 --2- 172.21.15.86:0/3700482733 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x559145331300 0x559145331760 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.586+0000 7fcc74d6b740 1 --2- 172.21.15.86:0/3700482733 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55914533dfd0 0x55914536aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.586+0000 7fcc74d6b740 1 --2- 172.21.15.86:0/3700482733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591453395c0 0x55914533da90 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.586+0000 7fcc74d6b740 1 --2- 172.21.15.86:0/3700482733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591453391e0 0x55914533fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.586+0000 7fcc4fe00640 0 client.7131 ms_handle_reset on v2:172.21.15.86:6835/2254117576 2024-04-20T21:23:31.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.586+0000 7fcc74d6b740 1 -- 172.21.15.86:0/3700482733 >> 172.21.15.86:0/3700482733 conn(0x55914530c6c0 msgr2=0x55914535a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:31.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.586+0000 7fcc74d6b740 1 -- 172.21.15.86:0/3700482733 shutdown_connections 2024-04-20T21:23:31.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.586+0000 7fcc74d6b740 1 -- 172.21.15.86:0/3700482733 wait complete. 2024-04-20T21:23:31.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.586+0000 7fcc74d6b740 20 client.7131 trim_cache size 0 max 16384 2024-04-20T21:23:31.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.605+0000 7fcc74d6b740 1 Processor -- start 2024-04-20T21:23:31.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.605+0000 7fcc74d6b740 1 -- start start 2024-04-20T21:23:31.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.606+0000 7fcc74d6b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591453391e0 0x55914533cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:31.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.606+0000 7fcc74d6b740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5591453395c0 0x55914536a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:31.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.606+0000 7fcc74d6b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55914533dfd0 0x55914535a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:31.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.606+0000 7fcc74d6b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6c200d90 con 0x55914533dfd0 2024-04-20T21:23:31.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.606+0000 7fcc74d6b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6c161550 con 0x5591453391e0 2024-04-20T21:23:31.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.606+0000 7fcc74d6b740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fcc6c15dfc0 con 0x5591453395c0 2024-04-20T21:23:31.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.607+0000 7fcc6be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55914533dfd0 0x55914535a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:31.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.607+0000 7fcc6be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55914533dfd0 0x55914535a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40672/0 (socket says 172.21.15.86:40672) 2024-04-20T21:23:31.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.607+0000 7fcc6be00640 1 -- 172.21.15.86:0/3652396257 learned_addr learned my addr 172.21.15.86:0/3652396257 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:31.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.607+0000 7fcc63e00640 1 --2- 172.21.15.86:0/3652396257 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5591453395c0 0x55914536a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:31.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.607+0000 7fcc6be00640 1 -- 172.21.15.86:0/3652396257 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5591453395c0 msgr2=0x55914536a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:31.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.607+0000 7fcc6be00640 1 --2- 172.21.15.86:0/3652396257 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5591453395c0 0x55914536a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.607+0000 7fcc6be00640 1 -- 172.21.15.86:0/3652396257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591453391e0 msgr2=0x55914533cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:31.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.607+0000 7fcc6be00640 1 --2- 172.21.15.86:0/3652396257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591453391e0 0x55914533cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.607+0000 7fcc6be00640 1 -- 172.21.15.86:0/3652396257 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55914536b680 con 0x55914533dfd0 2024-04-20T21:23:31.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.607+0000 7fcc6be00640 1 --2- 172.21.15.86:0/3652396257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55914533dfd0 0x55914535a140 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fcc6408a290 tx=0x7fcc64059590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c0151b0c5319bbb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:31.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.608+0000 7fcc5aa00640 1 -- 172.21.15.86:0/3652396257 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6405ded0 con 0x55914533dfd0 2024-04-20T21:23:31.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.608+0000 7fcc5aa00640 1 -- 172.21.15.86:0/3652396257 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc64088070 con 0x55914533dfd0 2024-04-20T21:23:31.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.608+0000 7fcc5aa00640 1 -- 172.21.15.86:0/3652396257 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc64062700 con 0x55914533dfd0 2024-04-20T21:23:31.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.608+0000 7fcc74d6b740 1 -- 172.21.15.86:0/3652396257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55914533dfd0 msgr2=0x55914535a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:31.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.608+0000 7fcc74d6b740 1 --2- 172.21.15.86:0/3652396257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55914533dfd0 0x55914535a140 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fcc6408a290 tx=0x7fcc64059590 comp rx=0 tx=0).stop 2024-04-20T21:23:31.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.608+0000 7fcc74d6b740 1 -- 172.21.15.86:0/3652396257 shutdown_connections 2024-04-20T21:23:31.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.608+0000 7fcc74d6b740 1 --2- 172.21.15.86:0/3652396257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55914533dfd0 0x55914535a140 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.608+0000 7fcc74d6b740 1 --2- 172.21.15.86:0/3652396257 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5591453395c0 0x55914536a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.608+0000 7fcc74d6b740 1 --2- 172.21.15.86:0/3652396257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5591453391e0 0x55914533cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.608+0000 7fcc74d6b740 1 -- 172.21.15.86:0/3652396257 >> 172.21.15.86:0/3652396257 conn(0x55914530c6c0 msgr2=0x55914533ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:31.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.608+0000 7fcc74d6b740 1 -- 172.21.15.86:0/3652396257 shutdown_connections 2024-04-20T21:23:31.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.608+0000 7fcc74d6b740 1 -- 172.21.15.86:0/3652396257 wait complete. 2024-04-20T21:23:31.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.609+0000 7fcc74d6b740 1 -- start start 2024-04-20T21:23:31.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.609+0000 7fcc74d6b740 1 Processor -- start 2024-04-20T21:23:31.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.610+0000 7fcc74d6b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591453391e0 0x5591453d1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:31.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.610+0000 7fcc74d6b740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5591453395c0 0x5591453ca990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:31.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.610+0000 7fcc74d6b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55914533dfd0 0x559145356cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:31.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.610+0000 7fcc74d6b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55914530db50 con 0x5591453391e0 2024-04-20T21:23:31.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.610+0000 7fcc74d6b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6c160f50 con 0x55914533dfd0 2024-04-20T21:23:31.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.610+0000 7fcc74d6b740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x559145334920 con 0x5591453395c0 2024-04-20T21:23:31.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.610+0000 7fcc74d6b740 10 client.0 _ll_register_callbacks cb 0x7fcc540317e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:23:31.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.610+0000 7fcc6b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591453391e0 0x5591453d1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:31.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.610+0000 7fcc6b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591453391e0 0x5591453d1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40674/0 (socket says 172.21.15.86:40674) 2024-04-20T21:23:31.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.610+0000 7fcc6b400640 1 -- 172.21.15.86:0/1046156373 learned_addr learned my addr 172.21.15.86:0/1046156373 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:31.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.611+0000 7fcc6b400640 1 -- 172.21.15.86:0/1046156373 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5591453395c0 msgr2=0x5591453ca990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:31.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.611+0000 7fcc6b400640 1 --2- 172.21.15.86:0/1046156373 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5591453395c0 0x5591453ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.611+0000 7fcc6b400640 1 -- 172.21.15.86:0/1046156373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55914533dfd0 msgr2=0x559145356cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:31.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.611+0000 7fcc6b400640 1 --2- 172.21.15.86:0/1046156373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55914533dfd0 0x559145356cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.611+0000 7fcc6b400640 1 -- 172.21.15.86:0/1046156373 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559144f21490 con 0x5591453391e0 2024-04-20T21:23:31.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.611+0000 7fcc6b400640 1 --2- 172.21.15.86:0/1046156373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591453391e0 0x5591453d1490 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c006fd0 tx=0x7fcc5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df8f85ee11093840 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:31.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.611+0000 7fcc4fe00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:23:31.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.611+0000 7fcc4fe00640 1 -- 172.21.15.86:0/1046156373 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c02b060 con 0x5591453391e0 2024-04-20T21:23:31.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.611+0000 7fcc4fe00640 1 -- 172.21.15.86:0/1046156373 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(72..72 src has 1..72) ==== 5795+0+0 (secure 0 0 0) 0x7fcc5c012d20 con 0x5591453391e0 2024-04-20T21:23:31.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.611+0000 7fcc74d6b740 1 -- 172.21.15.86:0/1046156373 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc64088070 con 0x5591453391e0 2024-04-20T21:23:31.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.612+0000 7fcc4fe00640 1 -- 172.21.15.86:0/1046156373 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fcc5c031050 con 0x5591453391e0 2024-04-20T21:23:31.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.612+0000 7fcc74d6b740 10 client.7137 fetch_fsmap learned FSMap version 49 2024-04-20T21:23:31.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.612+0000 7fcc74d6b740 1 -- 172.21.15.86:0/1046156373 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) -- 0x55914533ad30 con 0x5591453391e0 2024-04-20T21:23:31.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.612+0000 7fcc4fe00640 1 -- 172.21.15.86:0/1046156373 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc64088070 con 0x5591453391e0 2024-04-20T21:23:31.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.612+0000 7fcc4fe00640 1 -- 172.21.15.86:0/1046156373 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 49) ==== 3495+0+0 (secure 0 0 0) 0x7fcc5c034020 con 0x5591453391e0 2024-04-20T21:23:31.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.612+0000 7fcc74d6b740 10 client.7137 fetch_fsmap finished waiting for FSMap version 49 2024-04-20T21:23:31.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.612+0000 7fcc74d6b740 10 client.7137 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-20T21:23:31.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.612+0000 7fcc74d6b740 20 client.7137 populate_metadata read hostname 'smithi086' 2024-04-20T21:23:31.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.612+0000 7fcc74d6b740 1 --2- 172.21.15.86:0/1046156373 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x559145357200 0x5591453377f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:31.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.612+0000 7fcc74d6b740 4 client.7137 mds_command: new command op to 6987 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}] 2024-04-20T21:23:31.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.612+0000 7fcc74d6b740 1 -- 172.21.15.86:0/1046156373 --> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}) -- 0x55914527db20 con 0x559145357200 2024-04-20T21:23:31.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.612+0000 7fcc63e00640 1 --2- 172.21.15.86:0/1046156373 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x559145357200 0x5591453377f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:31.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.613+0000 7fcc63e00640 1 --2- 172.21.15.86:0/1046156373 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x559145357200 0x5591453377f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ffa5e63dcaee48c server_cookie=22e6b1daa3b2dfc2 in_seq=0 out_seq=0 2024-04-20T21:23:31.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.613+0000 7fcc4fe00640 10 client.7137 ms_handle_connect on v2:172.21.15.86:6835/2254117576 2024-04-20T21:23:31.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.614+0000 7fcc4fe00640 1 -- 172.21.15.86:0/1046156373 <== mds.0 v2:172.21.15.86:6835/2254117576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55914527db20 con 0x559145357200 2024-04-20T21:23:31.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.614+0000 7fcc4fe00640 10 client.7137 handle_command_reply: tid=0 2024-04-20T21:23:31.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.614+0000 7fcc74d6b740 1 client.7137 shutdown 2024-04-20T21:23:31.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.614+0000 7fcc74d6b740 1 -- 172.21.15.86:0/1046156373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591453391e0 msgr2=0x5591453d1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:31.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.614+0000 7fcc74d6b740 1 --2- 172.21.15.86:0/1046156373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591453391e0 0x5591453d1490 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c006fd0 tx=0x7fcc5c0046c0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.614+0000 7fcc74d6b740 1 -- 172.21.15.86:0/1046156373 shutdown_connections 2024-04-20T21:23:31.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.614+0000 7fcc74d6b740 1 --2- 172.21.15.86:0/1046156373 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x559145357200 0x5591453377f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.614+0000 7fcc74d6b740 1 --2- 172.21.15.86:0/1046156373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55914533dfd0 0x559145356cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.614+0000 7fcc74d6b740 1 --2- 172.21.15.86:0/1046156373 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5591453395c0 0x5591453ca990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.614+0000 7fcc74d6b740 1 --2- 172.21.15.86:0/1046156373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591453391e0 0x5591453d1490 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.614+0000 7fcc4fe00640 0 client.7137 ms_handle_reset on v2:172.21.15.86:6835/2254117576 2024-04-20T21:23:31.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.614+0000 7fcc74d6b740 1 -- 172.21.15.86:0/1046156373 >> 172.21.15.86:0/1046156373 conn(0x55914530c6c0 msgr2=0x55914535a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:31.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.614+0000 7fcc74d6b740 1 -- 172.21.15.86:0/1046156373 shutdown_connections 2024-04-20T21:23:31.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.614+0000 7fcc74d6b740 1 -- 172.21.15.86:0/1046156373 wait complete. 2024-04-20T21:23:31.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.614+0000 7fcc74d6b740 20 client.7137 trim_cache size 0 max 16384 2024-04-20T21:23:31.617 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:23:31.617 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-20T21:23:31.617 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "cabe0e28-928f-48a7-9044-d8d1d78e6c7d", 2024-04-20T21:23:31.618 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-20T21:23:31.618 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:23:31.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.615+0000 7fcc70a00640 1 -- 172.21.15.86:0/3649014246 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcc4806c8f0 msgr2=0x7fcc4806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:31.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.615+0000 7fcc70a00640 1 --2- 172.21.15.86:0/3649014246 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcc4806c8f0 0x7fcc4806edb0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c00b7c0 tx=0x7fcc5c008040 comp rx=0 tx=0).stop 2024-04-20T21:23:31.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.615+0000 7fcc70a00640 1 -- 172.21.15.86:0/3649014246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc6c15f2d0 msgr2=0x7fcc6c159620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:31.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.615+0000 7fcc70a00640 1 --2- 172.21.15.86:0/3649014246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc6c15f2d0 0x7fcc6c159620 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fcc6400c780 tx=0x7fcc6400cc40 comp rx=0 tx=0).stop 2024-04-20T21:23:31.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.615+0000 7fcc70a00640 1 -- 172.21.15.86:0/3649014246 shutdown_connections 2024-04-20T21:23:31.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.615+0000 7fcc70a00640 1 --2- 172.21.15.86:0/3649014246 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcc4806c8f0 0x7fcc4806edb0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.615+0000 7fcc70a00640 1 --2- 172.21.15.86:0/3649014246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc6c15f2d0 0x7fcc6c159620 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.615+0000 7fcc70a00640 1 --2- 172.21.15.86:0/3649014246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc6c072490 0x7fcc6c144110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.615+0000 7fcc70a00640 1 --2- 172.21.15.86:0/3649014246 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc6c0720b0 0x7fcc6c09db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.615+0000 7fcc70a00640 1 -- 172.21.15.86:0/3649014246 >> 172.21.15.86:0/3649014246 conn(0x7fcc6c0b2180 msgr2=0x7fcc6c1547f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:31.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.615+0000 7fcc70a00640 1 -- 172.21.15.86:0/3649014246 shutdown_connections 2024-04-20T21:23:31.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.615+0000 7fcc70a00640 1 -- 172.21.15.86:0/3649014246 wait complete. 2024-04-20T21:23:31.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:23:31.632 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-20T21:23:31.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.830+0000 7fb78ac00640 1 Processor -- start 2024-04-20T21:23:31.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.830+0000 7fb78ac00640 1 -- start start 2024-04-20T21:23:31.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.832+0000 7fb78ac00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb78415eb30 0x7fb7840b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:31.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.832+0000 7fb78ac00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7840adb10 con 0x7fb78406c090 2024-04-20T21:23:31.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.832+0000 7fb78ac00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7840addb0 con 0x7fb7840aee50 2024-04-20T21:23:31.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.832+0000 7fb78ac00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb7840af7d0 con 0x7fb78415eb30 2024-04-20T21:23:31.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.833+0000 7fb788c00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fb78406c090 0x7fb78406e160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34576/0 (socket says 172.21.15.86:34576) 2024-04-20T21:23:31.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.833+0000 7fb788c00640 1 -- 172.21.15.86:0/361827577 learned_addr learned my addr 172.21.15.86:0/361827577 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:23:31.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.833+0000 7fb783400640 1 -- 172.21.15.86:0/361827577 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3969105014 0 0) 0x7fb7840adb10 con 0x7fb78406c090 2024-04-20T21:23:31.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.833+0000 7fb783400640 1 -- 172.21.15.86:0/361827577 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb768003520 con 0x7fb78406c090 2024-04-20T21:23:31.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.833+0000 7fb783e00640 1 --2- 172.21.15.86:0/361827577 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb78415eb30 0x7fb7840b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:31.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.834+0000 7fb783400640 1 -- 172.21.15.86:0/361827577 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3185917723 0 0) 0x7fb768003520 con 0x7fb78406c090 2024-04-20T21:23:31.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.834+0000 7fb783400640 1 -- 172.21.15.86:0/361827577 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb78415eb30 msgr2=0x7fb7840b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:31.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.834+0000 7fb783400640 1 --2- 172.21.15.86:0/361827577 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb78415eb30 0x7fb7840b3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.834+0000 7fb783400640 1 -- 172.21.15.86:0/361827577 >> v1:172.21.15.144:6789/0 conn(0x7fb7840aee50 legacy=0x7fb78406b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:31.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.834+0000 7fb783400640 1 -- 172.21.15.86:0/361827577 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7840ae090 con 0x7fb78406c090 2024-04-20T21:23:31.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.834+0000 7fb783400640 1 -- 172.21.15.86:0/361827577 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fb77c001670 con 0x7fb78406c090 2024-04-20T21:23:31.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.834+0000 7fb783400640 1 -- 172.21.15.86:0/361827577 >> v1:172.21.15.86:6789/0 conn(0x7fb78406c090 legacy=0x7fb78406e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:31.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.834+0000 7fb783400640 1 --2- 172.21.15.86:0/361827577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7680039e0 0x7fb768003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:31.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.834+0000 7fb783400640 1 --2- 172.21.15.86:0/361827577 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb78415eb30 0x7fb768004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:31.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.834+0000 7fb783400640 1 --2- 172.21.15.86:0/361827577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb768004d40 0x7fb7680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:31.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.834+0000 7fb783400640 1 -- 172.21.15.86:0/361827577 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb7840af7d0 con 0x7fb768004d40 2024-04-20T21:23:31.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.834+0000 7fb783400640 1 -- 172.21.15.86:0/361827577 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb76800bba0 con 0x7fb7680039e0 2024-04-20T21:23:31.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.834+0000 7fb783400640 1 -- 172.21.15.86:0/361827577 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb76800bda0 con 0x7fb78415eb30 2024-04-20T21:23:31.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.834+0000 7fb788c00640 1 --2- 172.21.15.86:0/361827577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb768004d40 0x7fb7680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:31.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.835+0000 7fb783e00640 1 --2- 172.21.15.86:0/361827577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7680039e0 0x7fb768003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:31.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.835+0000 7fb77b400640 1 --2- 172.21.15.86:0/361827577 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb78415eb30 0x7fb768004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:31.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.835+0000 7fb788c00640 1 -- 172.21.15.86:0/361827577 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb78415eb30 msgr2=0x7fb768004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:31.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.835+0000 7fb788c00640 1 --2- 172.21.15.86:0/361827577 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb78415eb30 0x7fb768004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.835+0000 7fb788c00640 1 -- 172.21.15.86:0/361827577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7680039e0 msgr2=0x7fb768003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:31.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.835+0000 7fb788c00640 1 --2- 172.21.15.86:0/361827577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7680039e0 0x7fb768003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.835+0000 7fb788c00640 1 -- 172.21.15.86:0/361827577 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb768003520 con 0x7fb768004d40 2024-04-20T21:23:31.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.836+0000 7fb788c00640 1 --2- 172.21.15.86:0/361827577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb768004d40 0x7fb7680051c0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fb7840b20c0 tx=0x7fb768005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9398d6dfe080ad server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:31.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.836+0000 7fb783400640 1 -- 172.21.15.86:0/361827577 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c007c60 con 0x7fb768004d40 2024-04-20T21:23:31.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.836+0000 7fb783400640 1 -- 172.21.15.86:0/361827577 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb77c030070 con 0x7fb768004d40 2024-04-20T21:23:31.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.836+0000 7fb783400640 1 -- 172.21.15.86:0/361827577 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c005d70 con 0x7fb768004d40 2024-04-20T21:23:31.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.837+0000 7fb78ac00640 1 -- 172.21.15.86:0/361827577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb768004d40 msgr2=0x7fb7680051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:31.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.837+0000 7fb78ac00640 1 --2- 172.21.15.86:0/361827577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb768004d40 0x7fb7680051c0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fb7840b20c0 tx=0x7fb768005830 comp rx=0 tx=0).stop 2024-04-20T21:23:31.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.837+0000 7fb788c00640 1 -- 172.21.15.86:0/361827577 reap_dead start 2024-04-20T21:23:31.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.837+0000 7fb78ac00640 1 -- 172.21.15.86:0/361827577 shutdown_connections 2024-04-20T21:23:31.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.837+0000 7fb78ac00640 1 -- 172.21.15.86:0/361827577 >> 172.21.15.86:0/361827577 conn(0x7fb7840b2110 msgr2=0x7fb7840b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:31.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.837+0000 7fb78ac00640 1 -- 172.21.15.86:0/361827577 shutdown_connections 2024-04-20T21:23:31.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.837+0000 7fb78ac00640 1 -- 172.21.15.86:0/361827577 wait complete. 2024-04-20T21:23:31.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.839+0000 7fb78ac00640 1 Processor -- start 2024-04-20T21:23:31.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.839+0000 7fb78ac00640 1 -- start start 2024-04-20T21:23:31.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.840+0000 7fb78ac00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb78414f0c0 0x7fb784157fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:31.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.840+0000 7fb78ac00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb7841f5450 0x7fb78406ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:31.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.840+0000 7fb78ac00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb78406f180 0x7fb7840cda70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:31.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.840+0000 7fb78ac00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb7840ae930 con 0x7fb78406f180 2024-04-20T21:23:31.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.840+0000 7fb78ac00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb7840ae650 con 0x7fb78414f0c0 2024-04-20T21:23:31.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.840+0000 7fb78ac00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb78406e870 con 0x7fb7841f5450 2024-04-20T21:23:31.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.840+0000 7fb788c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb78406f180 0x7fb7840cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:31.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.840+0000 7fb788c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb78406f180 0x7fb7840cda70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40700/0 (socket says 172.21.15.86:40700) 2024-04-20T21:23:31.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.840+0000 7fb788c00640 1 -- 172.21.15.86:0/3633230642 learned_addr learned my addr 172.21.15.86:0/3633230642 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:31.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.840+0000 7fb788c00640 1 -- 172.21.15.86:0/3633230642 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb7841f5450 msgr2=0x7fb78406ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:31.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.840+0000 7fb788c00640 1 --2- 172.21.15.86:0/3633230642 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb7841f5450 0x7fb78406ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.840+0000 7fb788c00640 1 -- 172.21.15.86:0/3633230642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb78414f0c0 msgr2=0x7fb784157fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:31.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.840+0000 7fb788c00640 1 --2- 172.21.15.86:0/3633230642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb78414f0c0 0x7fb784157fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.840+0000 7fb788c00640 1 -- 172.21.15.86:0/3633230642 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7841f5b40 con 0x7fb78406f180 2024-04-20T21:23:31.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.841+0000 7fb788c00640 1 --2- 172.21.15.86:0/3633230642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb78406f180 0x7fb7840cda70 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fb77c02c450 tx=0x7fb77c001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:31.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.841+0000 7fb780c00640 1 -- 172.21.15.86:0/3633230642 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c007c60 con 0x7fb78406f180 2024-04-20T21:23:31.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.841+0000 7fb780c00640 1 -- 172.21.15.86:0/3633230642 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb77c000e40 con 0x7fb78406f180 2024-04-20T21:23:31.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.841+0000 7fb780c00640 1 -- 172.21.15.86:0/3633230642 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c02cbf0 con 0x7fb78406f180 2024-04-20T21:23:31.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.841+0000 7fb78ac00640 1 -- 172.21.15.86:0/3633230642 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7841f5e20 con 0x7fb78406f180 2024-04-20T21:23:31.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.841+0000 7fb78ac00640 1 -- 172.21.15.86:0/3633230642 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7841f63e0 con 0x7fb78406f180 2024-04-20T21:23:31.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.843+0000 7fb780c00640 1 -- 172.21.15.86:0/3633230642 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb77c007070 con 0x7fb78406f180 2024-04-20T21:23:31.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.844+0000 7fb780c00640 1 --2- 172.21.15.86:0/3633230642 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb76006c920 0x7fb76006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:31.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.844+0000 7fb780c00640 1 -- 172.21.15.86:0/3633230642 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5795+0+0 (secure 0 0 0) 0x7fb77c0ac730 con 0x7fb78406f180 2024-04-20T21:23:31.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.845+0000 7fb783e00640 1 --2- 172.21.15.86:0/3633230642 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb76006c920 0x7fb76006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:31.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.845+0000 7fb78d9ea740 1 Processor -- start 2024-04-20T21:23:31.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.845+0000 7fb78d9ea740 1 -- start start 2024-04-20T21:23:31.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.845+0000 7fb783e00640 1 --2- 172.21.15.86:0/3633230642 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb76006c920 0x7fb76006ede0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb7740042d0 tx=0x7fb774003990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:31.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.846+0000 7fb78d9ea740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558bd94961f0 0x558bd94965d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:31.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.846+0000 7fb78d9ea740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bd9466210 0x558bd9493460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:31.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.846+0000 7fb78d9ea740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558bd9469d70 0x558bd94939a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:31.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.846+0000 7fb78d9ea740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558bd93d2150 con 0x558bd94961f0 2024-04-20T21:23:31.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.846+0000 7fb78d9ea740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558bd9378370 con 0x558bd9466210 2024-04-20T21:23:31.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.846+0000 7fb78d9ea740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x558bd93ce110 con 0x558bd9469d70 2024-04-20T21:23:31.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.846+0000 7fb783e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558bd94961f0 0x558bd94965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:31.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.846+0000 7fb783e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558bd94961f0 0x558bd94965d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40714/0 (socket says 172.21.15.86:40714) 2024-04-20T21:23:31.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.846+0000 7fb783e00640 1 -- 172.21.15.86:0/1006881841 learned_addr learned my addr 172.21.15.86:0/1006881841 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:31.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.846+0000 7fb783e00640 1 -- 172.21.15.86:0/1006881841 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558bd9469d70 msgr2=0x558bd94939a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:31.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.846+0000 7fb783e00640 1 --2- 172.21.15.86:0/1006881841 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558bd9469d70 0x558bd94939a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.846+0000 7fb783e00640 1 -- 172.21.15.86:0/1006881841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bd9466210 msgr2=0x558bd9493460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:31.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.846+0000 7fb783e00640 1 --2- 172.21.15.86:0/1006881841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bd9466210 0x558bd9493460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.847+0000 7fb783e00640 1 -- 172.21.15.86:0/1006881841 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bd91ae7d0 con 0x558bd94961f0 2024-04-20T21:23:31.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.847+0000 7fb783e00640 1 --2- 172.21.15.86:0/1006881841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558bd94961f0 0x558bd94965d0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fb774009e70 tx=0x7fb774012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bdf63381e5a5f24 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:31.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.847+0000 7fb773e00640 1 -- 172.21.15.86:0/1006881841 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb774026070 con 0x558bd94961f0 2024-04-20T21:23:31.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.847+0000 7fb773e00640 1 -- 172.21.15.86:0/1006881841 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7740147e0 con 0x558bd94961f0 2024-04-20T21:23:31.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.847+0000 7fb773e00640 1 -- 172.21.15.86:0/1006881841 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb774014bd0 con 0x558bd94961f0 2024-04-20T21:23:31.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.848+0000 7fb78d9ea740 1 -- 172.21.15.86:0/1006881841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558bd94961f0 msgr2=0x558bd94965d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:31.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.848+0000 7fb78d9ea740 1 --2- 172.21.15.86:0/1006881841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558bd94961f0 0x558bd94965d0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fb774009e70 tx=0x7fb774012000 comp rx=0 tx=0).stop 2024-04-20T21:23:31.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.848+0000 7fb78d9ea740 1 -- 172.21.15.86:0/1006881841 shutdown_connections 2024-04-20T21:23:31.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.848+0000 7fb78d9ea740 1 --2- 172.21.15.86:0/1006881841 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558bd9469d70 0x558bd94939a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.848+0000 7fb78d9ea740 1 --2- 172.21.15.86:0/1006881841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bd9466210 0x558bd9493460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.848+0000 7fb78d9ea740 1 --2- 172.21.15.86:0/1006881841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558bd94961f0 0x558bd94965d0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.848+0000 7fb78d9ea740 1 -- 172.21.15.86:0/1006881841 >> 172.21.15.86:0/1006881841 conn(0x558bd9468480 msgr2=0x558bd94b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:31.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.848+0000 7fb78d9ea740 1 -- 172.21.15.86:0/1006881841 shutdown_connections 2024-04-20T21:23:31.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.848+0000 7fb78d9ea740 1 -- 172.21.15.86:0/1006881841 wait complete. 2024-04-20T21:23:31.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.849+0000 7fb78d9ea740 1 -- start start 2024-04-20T21:23:31.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.849+0000 7fb78d9ea740 1 Processor -- start 2024-04-20T21:23:31.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.850+0000 7fb78d9ea740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558bd9469d70 0x558bd9499ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:31.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.850+0000 7fb78d9ea740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bd94961f0 0x558bd948eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:31.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.850+0000 7fb78d9ea740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558bd94b1610 0x558bd948f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:31.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.850+0000 7fb78d9ea740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558bd946af20 con 0x558bd9469d70 2024-04-20T21:23:31.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.850+0000 7fb78d9ea740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558bd938b740 con 0x558bd94961f0 2024-04-20T21:23:31.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.850+0000 7fb78d9ea740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x558bd946a310 con 0x558bd94b1610 2024-04-20T21:23:31.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.850+0000 7fb78d9ea740 10 client.0 _ll_register_callbacks cb 0x558bd9386b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:23:31.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.850+0000 7fb783e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558bd9469d70 0x558bd9499ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:31.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.850+0000 7fb783e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558bd9469d70 0x558bd9499ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40728/0 (socket says 172.21.15.86:40728) 2024-04-20T21:23:31.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.850+0000 7fb783e00640 1 -- 172.21.15.86:0/3122157610 learned_addr learned my addr 172.21.15.86:0/3122157610 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:31.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.851+0000 7fb783e00640 1 -- 172.21.15.86:0/3122157610 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558bd94b1610 msgr2=0x558bd948f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:31.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.851+0000 7fb783e00640 1 --2- 172.21.15.86:0/3122157610 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558bd94b1610 0x558bd948f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.851+0000 7fb783e00640 1 -- 172.21.15.86:0/3122157610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bd94961f0 msgr2=0x558bd948eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:31.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.851+0000 7fb783e00640 1 --2- 172.21.15.86:0/3122157610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bd94961f0 0x558bd948eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.851+0000 7fb783e00640 1 -- 172.21.15.86:0/3122157610 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bd90e43f0 con 0x558bd9469d70 2024-04-20T21:23:31.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.851+0000 7fb783e00640 1 --2- 172.21.15.86:0/3122157610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558bd9469d70 0x558bd9499ce0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fb774016c80 tx=0x7fb774016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=238743fe7ce58acf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:31.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.851+0000 7fb770c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:23:31.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.851+0000 7fb770c00640 1 -- 172.21.15.86:0/3122157610 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb774030020 con 0x558bd9469d70 2024-04-20T21:23:31.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.851+0000 7fb770c00640 1 -- 172.21.15.86:0/3122157610 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(72..72 src has 1..72) ==== 5795+0+0 (secure 0 0 0) 0x7fb774014df0 con 0x558bd9469d70 2024-04-20T21:23:31.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.851+0000 7fb78d9ea740 1 -- 172.21.15.86:0/3122157610 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7740147e0 con 0x558bd9469d70 2024-04-20T21:23:31.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.851+0000 7fb770c00640 1 -- 172.21.15.86:0/3122157610 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fb77403a050 con 0x558bd9469d70 2024-04-20T21:23:31.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.851+0000 7fb78d9ea740 10 client.7149 fetch_fsmap learned FSMap version 49 2024-04-20T21:23:31.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.851+0000 7fb78d9ea740 1 -- 172.21.15.86:0/3122157610 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) -- 0x558bd90a40f0 con 0x558bd9469d70 2024-04-20T21:23:31.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.852+0000 7fb770c00640 1 -- 172.21.15.86:0/3122157610 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7740147e0 con 0x558bd9469d70 2024-04-20T21:23:31.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.852+0000 7fb770c00640 1 -- 172.21.15.86:0/3122157610 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 49) ==== 3495+0+0 (secure 0 0 0) 0x7fb77403d020 con 0x558bd9469d70 2024-04-20T21:23:31.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.852+0000 7fb78d9ea740 10 client.7149 fetch_fsmap finished waiting for FSMap version 49 2024-04-20T21:23:31.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.852+0000 7fb78d9ea740 10 client.7149 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-20T21:23:31.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.852+0000 7fb78d9ea740 20 client.7149 populate_metadata read hostname 'smithi086' 2024-04-20T21:23:31.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.852+0000 7fb78d9ea740 1 --2- 172.21.15.86:0/3122157610 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x558bd9490650 0x558bd9490ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:31.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.852+0000 7fb78d9ea740 4 client.7149 mds_command: new command op to 6987 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:23:31.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.852+0000 7fb78d9ea740 1 -- 172.21.15.86:0/3122157610 --> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558bd93dfb70 con 0x558bd9490650 2024-04-20T21:23:31.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.852+0000 7fb77b400640 1 --2- 172.21.15.86:0/3122157610 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x558bd9490650 0x558bd9490ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:31.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.853+0000 7fb77b400640 1 --2- 172.21.15.86:0/3122157610 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x558bd9490650 0x558bd9490ae0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1976dc0a2ef5564f server_cookie=2353d7669d765eaa in_seq=0 out_seq=0 2024-04-20T21:23:31.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.853+0000 7fb770c00640 10 client.7149 ms_handle_connect on v2:172.21.15.86:6835/2254117576 2024-04-20T21:23:31.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.853+0000 7fb770c00640 1 -- 172.21.15.86:0/3122157610 <== mds.0 v2:172.21.15.86:6835/2254117576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558bd93dfb70 con 0x558bd9490650 2024-04-20T21:23:31.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.854+0000 7fb770c00640 10 client.7149 handle_command_reply: tid=0 2024-04-20T21:23:31.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.854+0000 7fb78d9ea740 1 client.7149 shutdown 2024-04-20T21:23:31.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.854+0000 7fb78d9ea740 1 -- 172.21.15.86:0/3122157610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558bd9469d70 msgr2=0x558bd9499ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:31.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.854+0000 7fb78d9ea740 1 --2- 172.21.15.86:0/3122157610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558bd9469d70 0x558bd9499ce0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fb774016c80 tx=0x7fb774016000 comp rx=0 tx=0).stop 2024-04-20T21:23:31.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.854+0000 7fb78d9ea740 1 -- 172.21.15.86:0/3122157610 shutdown_connections 2024-04-20T21:23:31.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.854+0000 7fb78d9ea740 1 --2- 172.21.15.86:0/3122157610 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x558bd9490650 0x558bd9490ae0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.854+0000 7fb78d9ea740 1 --2- 172.21.15.86:0/3122157610 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558bd94b1610 0x558bd948f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.854+0000 7fb78d9ea740 1 --2- 172.21.15.86:0/3122157610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bd94961f0 0x558bd948eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.854+0000 7fb78d9ea740 1 --2- 172.21.15.86:0/3122157610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558bd9469d70 0x558bd9499ce0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.854+0000 7fb770c00640 0 client.7149 ms_handle_reset on v2:172.21.15.86:6835/2254117576 2024-04-20T21:23:31.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.854+0000 7fb78d9ea740 1 -- 172.21.15.86:0/3122157610 >> 172.21.15.86:0/3122157610 conn(0x558bd9466210 msgr2=0x558bd9491ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:31.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.854+0000 7fb78d9ea740 1 -- 172.21.15.86:0/3122157610 shutdown_connections 2024-04-20T21:23:31.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.854+0000 7fb78d9ea740 1 -- 172.21.15.86:0/3122157610 wait complete. 2024-04-20T21:23:31.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.854+0000 7fb78d9ea740 20 client.7149 trim_cache size 0 max 16384 2024-04-20T21:23:31.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.873+0000 7fb78d9ea740 1 Processor -- start 2024-04-20T21:23:31.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.873+0000 7fb78d9ea740 1 -- start start 2024-04-20T21:23:31.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.874+0000 7fb78d9ea740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bd9469d70 0x558bd948c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:31.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.874+0000 7fb78d9ea740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558bd94961f0 0x558bd94af040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:31.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.874+0000 7fb78d9ea740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558bd94b1610 0x558bd94af580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:31.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.874+0000 7fb78d9ea740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb784201bc0 con 0x558bd94961f0 2024-04-20T21:23:31.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.874+0000 7fb78d9ea740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb784200b90 con 0x558bd9469d70 2024-04-20T21:23:31.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.874+0000 7fb78d9ea740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb7841fff70 con 0x558bd94b1610 2024-04-20T21:23:31.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.874+0000 7fb77b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558bd94961f0 0x558bd94af040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:31.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.875+0000 7fb77b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558bd94961f0 0x558bd94af040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40742/0 (socket says 172.21.15.86:40742) 2024-04-20T21:23:31.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.875+0000 7fb77b400640 1 -- 172.21.15.86:0/114119631 learned_addr learned my addr 172.21.15.86:0/114119631 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:31.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.875+0000 7fb788c00640 1 --2- 172.21.15.86:0/114119631 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558bd94b1610 0x558bd94af580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:31.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.875+0000 7fb77b400640 1 -- 172.21.15.86:0/114119631 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558bd94b1610 msgr2=0x558bd94af580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:31.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.875+0000 7fb77b400640 1 --2- 172.21.15.86:0/114119631 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558bd94b1610 0x558bd94af580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.875+0000 7fb77b400640 1 -- 172.21.15.86:0/114119631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bd9469d70 msgr2=0x558bd948c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:31.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.875+0000 7fb77b400640 1 --2- 172.21.15.86:0/114119631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bd9469d70 0x558bd948c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.875+0000 7fb77b400640 1 -- 172.21.15.86:0/114119631 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bd9493a80 con 0x558bd94961f0 2024-04-20T21:23:31.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.875+0000 7fb77b400640 1 --2- 172.21.15.86:0/114119631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558bd94961f0 0x558bd94af040 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fb76c007d90 tx=0x7fb76c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0097d013db8670c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:31.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.876+0000 7fb773400640 1 -- 172.21.15.86:0/114119631 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c001b90 con 0x558bd94961f0 2024-04-20T21:23:31.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.876+0000 7fb773400640 1 -- 172.21.15.86:0/114119631 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb76c01b020 con 0x558bd94961f0 2024-04-20T21:23:31.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.876+0000 7fb773400640 1 -- 172.21.15.86:0/114119631 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c0163c0 con 0x558bd94961f0 2024-04-20T21:23:31.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.876+0000 7fb78d9ea740 1 -- 172.21.15.86:0/114119631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558bd94961f0 msgr2=0x558bd94af040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:31.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.876+0000 7fb78d9ea740 1 --2- 172.21.15.86:0/114119631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558bd94961f0 0x558bd94af040 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fb76c007d90 tx=0x7fb76c009040 comp rx=0 tx=0).stop 2024-04-20T21:23:31.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.876+0000 7fb78d9ea740 1 -- 172.21.15.86:0/114119631 shutdown_connections 2024-04-20T21:23:31.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.876+0000 7fb78d9ea740 1 --2- 172.21.15.86:0/114119631 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558bd94b1610 0x558bd94af580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.876+0000 7fb78d9ea740 1 --2- 172.21.15.86:0/114119631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558bd94961f0 0x558bd94af040 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.876+0000 7fb78d9ea740 1 --2- 172.21.15.86:0/114119631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bd9469d70 0x558bd948c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.876+0000 7fb78d9ea740 1 -- 172.21.15.86:0/114119631 >> 172.21.15.86:0/114119631 conn(0x558bd9466210 msgr2=0x558bd946c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:31.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.876+0000 7fb78d9ea740 1 -- 172.21.15.86:0/114119631 shutdown_connections 2024-04-20T21:23:31.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.876+0000 7fb78d9ea740 1 -- 172.21.15.86:0/114119631 wait complete. 2024-04-20T21:23:31.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.877+0000 7fb78d9ea740 1 -- start start 2024-04-20T21:23:31.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.877+0000 7fb78d9ea740 1 Processor -- start 2024-04-20T21:23:31.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.878+0000 7fb78d9ea740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bd9469d70 0x558bd946b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:31.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.878+0000 7fb78d9ea740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558bd94961f0 0x558bd94c1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:31.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.878+0000 7fb78d9ea740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558bd94b1610 0x558bd94c1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:31.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.878+0000 7fb78d9ea740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb784200540 con 0x558bd94961f0 2024-04-20T21:23:31.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.878+0000 7fb78d9ea740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x558bd94d5760 con 0x558bd9469d70 2024-04-20T21:23:31.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.878+0000 7fb78d9ea740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x558bd94d59b0 con 0x558bd94b1610 2024-04-20T21:23:31.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.878+0000 7fb78d9ea740 10 client.0 _ll_register_callbacks cb 0x7fb774024070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:23:31.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.879+0000 7fb77b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558bd94961f0 0x558bd94c1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:31.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.879+0000 7fb77b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558bd94961f0 0x558bd94c1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40748/0 (socket says 172.21.15.86:40748) 2024-04-20T21:23:31.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.879+0000 7fb77b400640 1 -- 172.21.15.86:0/3710130439 learned_addr learned my addr 172.21.15.86:0/3710130439 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:23:31.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.879+0000 7fb783e00640 1 --2- 172.21.15.86:0/3710130439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bd9469d70 0x558bd946b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:31.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.879+0000 7fb788c00640 1 --2- 172.21.15.86:0/3710130439 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558bd94b1610 0x558bd94c1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:31.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.879+0000 7fb77b400640 1 -- 172.21.15.86:0/3710130439 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558bd94b1610 msgr2=0x558bd94c1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:31.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.879+0000 7fb77b400640 1 --2- 172.21.15.86:0/3710130439 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558bd94b1610 0x558bd94c1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.879+0000 7fb77b400640 1 -- 172.21.15.86:0/3710130439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bd9469d70 msgr2=0x558bd946b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:23:31.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.879+0000 7fb77b400640 1 --2- 172.21.15.86:0/3710130439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bd9469d70 0x558bd946b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.879+0000 7fb77b400640 1 -- 172.21.15.86:0/3710130439 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bd9496b80 con 0x558bd94961f0 2024-04-20T21:23:31.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.879+0000 7fb77b400640 1 --2- 172.21.15.86:0/3710130439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558bd94961f0 0x558bd94c1a30 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fb76c007aa0 tx=0x7fb76c005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c896e92d350f77b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:23:31.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.879+0000 7fb770c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:23:31.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.880+0000 7fb770c00640 1 -- 172.21.15.86:0/3710130439 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c024070 con 0x558bd94961f0 2024-04-20T21:23:31.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.880+0000 7fb770c00640 1 -- 172.21.15.86:0/3710130439 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(72..72 src has 1..72) ==== 5795+0+0 (secure 0 0 0) 0x7fb76c0037f0 con 0x558bd94961f0 2024-04-20T21:23:31.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.880+0000 7fb78d9ea740 1 -- 172.21.15.86:0/3710130439 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb76c01b020 con 0x558bd94961f0 2024-04-20T21:23:31.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.880+0000 7fb770c00640 1 -- 172.21.15.86:0/3710130439 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fb76c01a040 con 0x558bd94961f0 2024-04-20T21:23:31.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.880+0000 7fb78d9ea740 10 client.7155 fetch_fsmap learned FSMap version 49 2024-04-20T21:23:31.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.880+0000 7fb78d9ea740 1 -- 172.21.15.86:0/3710130439 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) -- 0x558bd9499a60 con 0x558bd94961f0 2024-04-20T21:23:31.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.880+0000 7fb770c00640 1 -- 172.21.15.86:0/3710130439 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb76c01b020 con 0x558bd94961f0 2024-04-20T21:23:31.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.880+0000 7fb770c00640 1 -- 172.21.15.86:0/3710130439 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 49) ==== 3495+0+0 (secure 0 0 0) 0x7fb76c003a10 con 0x558bd94961f0 2024-04-20T21:23:31.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.880+0000 7fb78d9ea740 10 client.7155 fetch_fsmap finished waiting for FSMap version 49 2024-04-20T21:23:31.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.880+0000 7fb78d9ea740 10 client.7155 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-20T21:23:31.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.880+0000 7fb78d9ea740 20 client.7155 populate_metadata read hostname 'smithi086' 2024-04-20T21:23:31.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.881+0000 7fb78d9ea740 1 --2- 172.21.15.86:0/3710130439 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x558bd9533860 0x558bd9537570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:23:31.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.881+0000 7fb78d9ea740 4 client.7155 mds_command: new command op to 6987 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:23:31.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.881+0000 7fb78d9ea740 1 -- 172.21.15.86:0/3710130439 --> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558bd93d99e0 con 0x558bd9533860 2024-04-20T21:23:31.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.881+0000 7fb783e00640 1 --2- 172.21.15.86:0/3710130439 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x558bd9533860 0x558bd9537570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:23:31.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.881+0000 7fb770c00640 10 client.7155 ms_handle_connect on v2:172.21.15.86:6835/2254117576 2024-04-20T21:23:31.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.881+0000 7fb783e00640 1 --2- 172.21.15.86:0/3710130439 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x558bd9533860 0x558bd9537570 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5d53bea4d4f6b0e server_cookie=6c7933d860927d98 in_seq=0 out_seq=0 2024-04-20T21:23:31.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.882+0000 7fb770c00640 1 -- 172.21.15.86:0/3710130439 <== mds.0 v2:172.21.15.86:6835/2254117576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558bd93d99e0 con 0x558bd9533860 2024-04-20T21:23:31.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.882+0000 7fb770c00640 10 client.7155 handle_command_reply: tid=0 2024-04-20T21:23:31.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.882+0000 7fb78d9ea740 1 client.7155 shutdown 2024-04-20T21:23:31.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.882+0000 7fb78d9ea740 1 -- 172.21.15.86:0/3710130439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558bd94961f0 msgr2=0x558bd94c1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:31.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.882+0000 7fb78d9ea740 1 --2- 172.21.15.86:0/3710130439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558bd94961f0 0x558bd94c1a30 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fb76c007aa0 tx=0x7fb76c005040 comp rx=0 tx=0).stop 2024-04-20T21:23:31.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.882+0000 7fb78d9ea740 1 -- 172.21.15.86:0/3710130439 shutdown_connections 2024-04-20T21:23:31.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.882+0000 7fb78d9ea740 1 --2- 172.21.15.86:0/3710130439 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x558bd9533860 0x558bd9537570 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.882+0000 7fb78d9ea740 1 --2- 172.21.15.86:0/3710130439 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x558bd94b1610 0x558bd94c1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.882+0000 7fb78d9ea740 1 --2- 172.21.15.86:0/3710130439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558bd94961f0 0x558bd94c1a30 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.882+0000 7fb770c00640 0 client.7155 ms_handle_reset on v2:172.21.15.86:6835/2254117576 2024-04-20T21:23:31.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.882+0000 7fb78d9ea740 1 --2- 172.21.15.86:0/3710130439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x558bd9469d70 0x558bd946b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.882+0000 7fb78d9ea740 1 -- 172.21.15.86:0/3710130439 >> 172.21.15.86:0/3710130439 conn(0x558bd9466210 msgr2=0x558bd94aef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:31.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.882+0000 7fb78d9ea740 1 -- 172.21.15.86:0/3710130439 shutdown_connections 2024-04-20T21:23:31.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.882+0000 7fb78d9ea740 1 -- 172.21.15.86:0/3710130439 wait complete. 2024-04-20T21:23:31.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.882+0000 7fb78d9ea740 20 client.7155 trim_cache size 0 max 16384 2024-04-20T21:23:31.891 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:23:31.891 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T21:23:31.891 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-20T21:23:31.891 INFO:teuthology.orchestra.run.smithi086.stdout: "cabe0e28-928f-48a7-9044-d8d1d78e6c7d": { 2024-04-20T21:23:31.891 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/subdir", 2024-04-20T21:23:31.891 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "cabe0e28-928f-48a7-9044-d8d1d78e6c7d", 2024-04-20T21:23:31.891 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive" 2024-04-20T21:23:31.891 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:23:31.891 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:23:31.891 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:23:31.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.883+0000 7fb78ac00640 1 -- 172.21.15.86:0/3633230642 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb76006c920 msgr2=0x7fb76006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:31.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.883+0000 7fb78ac00640 1 --2- 172.21.15.86:0/3633230642 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb76006c920 0x7fb76006ede0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb7740042d0 tx=0x7fb774003990 comp rx=0 tx=0).stop 2024-04-20T21:23:31.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.883+0000 7fb78ac00640 1 -- 172.21.15.86:0/3633230642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb78406f180 msgr2=0x7fb7840cda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:23:31.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.883+0000 7fb78ac00640 1 --2- 172.21.15.86:0/3633230642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb78406f180 0x7fb7840cda70 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fb77c02c450 tx=0x7fb77c001970 comp rx=0 tx=0).stop 2024-04-20T21:23:31.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.883+0000 7fb78ac00640 1 -- 172.21.15.86:0/3633230642 shutdown_connections 2024-04-20T21:23:31.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.883+0000 7fb78ac00640 1 --2- 172.21.15.86:0/3633230642 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb76006c920 0x7fb76006ede0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.883+0000 7fb78ac00640 1 --2- 172.21.15.86:0/3633230642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb78406f180 0x7fb7840cda70 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.883+0000 7fb78ac00640 1 --2- 172.21.15.86:0/3633230642 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb7841f5450 0x7fb78406ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.883+0000 7fb78ac00640 1 --2- 172.21.15.86:0/3633230642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb78414f0c0 0x7fb784157fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:23:31.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.883+0000 7fb78ac00640 1 -- 172.21.15.86:0/3633230642 >> 172.21.15.86:0/3633230642 conn(0x7fb7840b2110 msgr2=0x7fb7840b0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:23:31.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.883+0000 7fb78ac00640 1 -- 172.21.15.86:0/3633230642 shutdown_connections 2024-04-20T21:23:31.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:23:31.883+0000 7fb78ac00640 1 -- 172.21.15.86:0/3633230642 wait complete. 2024-04-20T21:23:31.902 INFO:tasks.cephfs.filesystem:scrub status for tag:cabe0e28-928f-48a7-9044-d8d1d78e6c7d - {'path': '/subdir', 'tag': 'cabe0e28-928f-48a7-9044-d8d1d78e6c7d', 'options': 'recursive'} 2024-04-20T21:23:46.761 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:23:46.770 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:24:01.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:24:01.903 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-20T21:24:02.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.124+0000 7fe2b2a00640 1 Processor -- start 2024-04-20T21:24:02.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.124+0000 7fe2b2a00640 1 -- start start 2024-04-20T21:24:02.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.126+0000 7fe2b2a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2ac0cfc70 0x7fe2ac0cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:02.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.126+0000 7fe2b2a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe2ac0cdbe0 0x7fe2ac06eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:02.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.126+0000 7fe2b2a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2ac15bcd0 con 0x7fe2ac0b3ba0 2024-04-20T21:24:02.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.126+0000 7fe2b2a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fe2ac15c8e0 con 0x7fe2ac0cdbe0 2024-04-20T21:24:02.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.126+0000 7fe2b2a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe2ac15bfb0 con 0x7fe2ac0cfc70 2024-04-20T21:24:02.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.126+0000 7fe2b2000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe2ac0b3ba0 0x7fe2ac071ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:48228/0 (socket says 172.21.15.86:48228) 2024-04-20T21:24:02.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.126+0000 7fe2b2000640 1 -- 172.21.15.86:0/211907324 learned_addr learned my addr 172.21.15.86:0/211907324 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:24:02.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.126+0000 7fe2b1600640 1 --2- 172.21.15.86:0/211907324 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2ac0cfc70 0x7fe2ac0cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:02.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.126+0000 7fe2b0c00640 1 --2- 172.21.15.86:0/211907324 >> v2:172.21.15.144:3300/0 conn(0x7fe2ac0cdbe0 0x7fe2ac06eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:02.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.127+0000 7fe2abe00640 1 -- 172.21.15.86:0/211907324 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 418833239 0 0) 0x7fe2ac15bcd0 con 0x7fe2ac0b3ba0 2024-04-20T21:24:02.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.127+0000 7fe2abe00640 1 -- 172.21.15.86:0/211907324 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe294003520 con 0x7fe2ac0b3ba0 2024-04-20T21:24:02.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.127+0000 7fe2abe00640 1 -- 172.21.15.86:0/211907324 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1594374169 0 0) 0x7fe294003520 con 0x7fe2ac0b3ba0 2024-04-20T21:24:02.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.127+0000 7fe2abe00640 1 -- 172.21.15.86:0/211907324 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2ac0cfc70 msgr2=0x7fe2ac0cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:02.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.127+0000 7fe2abe00640 1 --2- 172.21.15.86:0/211907324 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2ac0cfc70 0x7fe2ac0cd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.127+0000 7fe2abe00640 1 -- 172.21.15.86:0/211907324 >> v2:172.21.15.144:3300/0 conn(0x7fe2ac0cdbe0 msgr2=0x7fe2ac06eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:02.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.127+0000 7fe2abe00640 1 --2- 172.21.15.86:0/211907324 >> v2:172.21.15.144:3300/0 conn(0x7fe2ac0cdbe0 0x7fe2ac06eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.127+0000 7fe2abe00640 1 -- 172.21.15.86:0/211907324 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2ac15c290 con 0x7fe2ac0b3ba0 2024-04-20T21:24:02.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.127+0000 7fe2abe00640 1 -- 172.21.15.86:0/211907324 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fe2a40019f0 con 0x7fe2ac0b3ba0 2024-04-20T21:24:02.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.127+0000 7fe2abe00640 1 -- 172.21.15.86:0/211907324 >> v1:172.21.15.86:6789/0 conn(0x7fe2ac0b3ba0 legacy=0x7fe2ac071ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:02.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.127+0000 7fe2abe00640 1 --2- 172.21.15.86:0/211907324 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe294003a20 0x7fe294003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:02.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.127+0000 7fe2abe00640 1 --2- 172.21.15.86:0/211907324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe294004800 0x7fe294004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:02.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.127+0000 7fe2abe00640 1 --2- 172.21.15.86:0/211907324 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2ac0cfc70 0x7fe2940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:02.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.127+0000 7fe2abe00640 1 -- 172.21.15.86:0/211907324 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ac15c8e0 con 0x7fe294004800 2024-04-20T21:24:02.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.127+0000 7fe2abe00640 1 -- 172.21.15.86:0/211907324 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ac15bfb0 con 0x7fe294003a20 2024-04-20T21:24:02.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.127+0000 7fe2abe00640 1 -- 172.21.15.86:0/211907324 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe29400b9f0 con 0x7fe2ac0cfc70 2024-04-20T21:24:02.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.128+0000 7fe2b0c00640 1 --2- 172.21.15.86:0/211907324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe294004800 0x7fe294004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:02.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.128+0000 7fe2b1600640 1 --2- 172.21.15.86:0/211907324 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe294003a20 0x7fe294003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:02.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.128+0000 7fe2b2000640 1 --2- 172.21.15.86:0/211907324 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2ac0cfc70 0x7fe2940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:02.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.128+0000 7fe2b0c00640 1 -- 172.21.15.86:0/211907324 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2ac0cfc70 msgr2=0x7fe2940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:02.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.128+0000 7fe2b0c00640 1 --2- 172.21.15.86:0/211907324 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2ac0cfc70 0x7fe2940051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.128+0000 7fe2b0c00640 1 -- 172.21.15.86:0/211907324 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe294003a20 msgr2=0x7fe294003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:02.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.128+0000 7fe2b0c00640 1 --2- 172.21.15.86:0/211907324 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe294003a20 0x7fe294003e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.128+0000 7fe2b0c00640 1 -- 172.21.15.86:0/211907324 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe294003520 con 0x7fe294004800 2024-04-20T21:24:02.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.128+0000 7fe2b2000640 1 --2- 172.21.15.86:0/211907324 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2ac0cfc70 0x7fe2940051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:24:02.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.129+0000 7fe2b0c00640 1 --2- 172.21.15.86:0/211907324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe294004800 0x7fe294004c80 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fe298009ac0 tx=0x7fe294005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82c0c8f0f28f78ae server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:02.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.129+0000 7fe2abe00640 1 -- 172.21.15.86:0/211907324 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe298039070 con 0x7fe294004800 2024-04-20T21:24:02.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.129+0000 7fe2abe00640 1 -- 172.21.15.86:0/211907324 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe29802c710 con 0x7fe294004800 2024-04-20T21:24:02.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.130+0000 7fe2abe00640 1 -- 172.21.15.86:0/211907324 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe298009690 con 0x7fe294004800 2024-04-20T21:24:02.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.130+0000 7fe2b2a00640 1 -- 172.21.15.86:0/211907324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe294004800 msgr2=0x7fe294004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:02.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.130+0000 7fe2b2a00640 1 --2- 172.21.15.86:0/211907324 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe294004800 0x7fe294004c80 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fe298009ac0 tx=0x7fe294005830 comp rx=0 tx=0).stop 2024-04-20T21:24:02.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.130+0000 7fe2b2000640 1 -- 172.21.15.86:0/211907324 reap_dead start 2024-04-20T21:24:02.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.130+0000 7fe2b2a00640 1 -- 172.21.15.86:0/211907324 shutdown_connections 2024-04-20T21:24:02.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.130+0000 7fe2b2a00640 1 -- 172.21.15.86:0/211907324 >> 172.21.15.86:0/211907324 conn(0x7fe2ac067cf0 msgr2=0x7fe2ac06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:02.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.130+0000 7fe2b2a00640 1 -- 172.21.15.86:0/211907324 shutdown_connections 2024-04-20T21:24:02.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.130+0000 7fe2b2a00640 1 -- 172.21.15.86:0/211907324 wait complete. 2024-04-20T21:24:02.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.132+0000 7fe2b2a00640 1 Processor -- start 2024-04-20T21:24:02.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.133+0000 7fe2b2a00640 1 -- start start 2024-04-20T21:24:02.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.133+0000 7fe2b2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe294004800 0x7fe2ac0aa6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:02.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.133+0000 7fe2b2a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2ac0c6550 0x7fe2ac0c53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:02.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.134+0000 7fe2b2a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2ac0c5930 0x7fe2ac146910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:02.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.134+0000 7fe2b2a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ac0d0050 con 0x7fe294004800 2024-04-20T21:24:02.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.134+0000 7fe2b2a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ac0b3710 con 0x7fe2ac0c5930 2024-04-20T21:24:02.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.134+0000 7fe2b2a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe2ac0d01d0 con 0x7fe2ac0c6550 2024-04-20T21:24:02.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.134+0000 7fe2b1600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe294004800 0x7fe2ac0aa6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:02.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.134+0000 7fe2b1600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe294004800 0x7fe2ac0aa6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37084/0 (socket says 172.21.15.86:37084) 2024-04-20T21:24:02.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.134+0000 7fe2b1600640 1 -- 172.21.15.86:0/133397887 learned_addr learned my addr 172.21.15.86:0/133397887 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:02.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.134+0000 7fe2b1600640 1 -- 172.21.15.86:0/133397887 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2ac0c6550 msgr2=0x7fe2ac0c53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:02.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.134+0000 7fe2b1600640 1 --2- 172.21.15.86:0/133397887 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2ac0c6550 0x7fe2ac0c53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.134+0000 7fe2b1600640 1 -- 172.21.15.86:0/133397887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2ac0c5930 msgr2=0x7fe2ac146910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:02.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.134+0000 7fe2b1600640 1 --2- 172.21.15.86:0/133397887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2ac0c5930 0x7fe2ac146910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.134+0000 7fe2b1600640 1 -- 172.21.15.86:0/133397887 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2ac0a9620 con 0x7fe294004800 2024-04-20T21:24:02.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.135+0000 7fe2b1600640 1 --2- 172.21.15.86:0/133397887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe294004800 0x7fe2ac0aa6c0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0010860 tx=0x7fe2a0010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:02.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.135+0000 7fe2a9600640 1 -- 172.21.15.86:0/133397887 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a000f040 con 0x7fe294004800 2024-04-20T21:24:02.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.135+0000 7fe2a9600640 1 -- 172.21.15.86:0/133397887 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2a000eb60 con 0x7fe294004800 2024-04-20T21:24:02.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.135+0000 7fe2b2a00640 1 -- 172.21.15.86:0/133397887 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2ac0a9900 con 0x7fe294004800 2024-04-20T21:24:02.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.135+0000 7fe2a9600640 1 -- 172.21.15.86:0/133397887 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a001c400 con 0x7fe294004800 2024-04-20T21:24:02.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.135+0000 7fe2b2a00640 1 -- 172.21.15.86:0/133397887 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2ac0a9ec0 con 0x7fe294004800 2024-04-20T21:24:02.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.137+0000 7fe2a9600640 1 -- 172.21.15.86:0/133397887 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe2a001c5a0 con 0x7fe294004800 2024-04-20T21:24:02.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.138+0000 7fe2a9600640 1 --2- 172.21.15.86:0/133397887 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe28c06c8f0 0x7fe28c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:02.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.138+0000 7fe2a9600640 1 -- 172.21.15.86:0/133397887 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5882+0+0 (secure 0 0 0) 0x7fe2a008c1b0 con 0x7fe294004800 2024-04-20T21:24:02.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.139+0000 7fe2b0c00640 1 --2- 172.21.15.86:0/133397887 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe28c06c8f0 0x7fe28c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:02.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.139+0000 7fe2b6dc1740 1 Processor -- start 2024-04-20T21:24:02.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.139+0000 7fe2b6dc1740 1 -- start start 2024-04-20T21:24:02.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.139+0000 7fe2b0c00640 1 --2- 172.21.15.86:0/133397887 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe28c06c8f0 0x7fe28c06edb0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe298036000 tx=0x7fe29802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:02.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.140+0000 7fe2b6dc1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614592c13d0 0x5614592c17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:02.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.140+0000 7fe2b6dc1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614592913f0 0x5614592be640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:02.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.140+0000 7fe2b6dc1740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561459294f50 0x5614592beb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:02.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.140+0000 7fe2b6dc1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5614591fd150 con 0x5614592913f0 2024-04-20T21:24:02.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.140+0000 7fe2b6dc1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5614591a3370 con 0x5614592c13d0 2024-04-20T21:24:02.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.140+0000 7fe2b6dc1740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5614591f9110 con 0x561459294f50 2024-04-20T21:24:02.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.140+0000 7fe2b0c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614592913f0 0x5614592be640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:02.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.140+0000 7fe2b0c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614592913f0 0x5614592be640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37094/0 (socket says 172.21.15.86:37094) 2024-04-20T21:24:02.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.140+0000 7fe2b0c00640 1 -- 172.21.15.86:0/3422039863 learned_addr learned my addr 172.21.15.86:0/3422039863 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:02.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.141+0000 7fe2b2000640 1 --2- 172.21.15.86:0/3422039863 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561459294f50 0x5614592beb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:02.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.141+0000 7fe2b1600640 1 --2- 172.21.15.86:0/3422039863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614592c13d0 0x5614592c17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:02.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.141+0000 7fe2b0c00640 1 -- 172.21.15.86:0/3422039863 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561459294f50 msgr2=0x5614592beb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:02.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.141+0000 7fe2b0c00640 1 --2- 172.21.15.86:0/3422039863 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561459294f50 0x5614592beb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.141+0000 7fe2b0c00640 1 -- 172.21.15.86:0/3422039863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614592c13d0 msgr2=0x5614592c17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:02.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.141+0000 7fe2b0c00640 1 --2- 172.21.15.86:0/3422039863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614592c13d0 0x5614592c17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.141+0000 7fe2b0c00640 1 -- 172.21.15.86:0/3422039863 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561458fd97d0 con 0x5614592913f0 2024-04-20T21:24:02.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.141+0000 7fe2b1600640 1 --2- 172.21.15.86:0/3422039863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614592c13d0 0x5614592c17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:24:02.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.141+0000 7fe2b0c00640 1 --2- 172.21.15.86:0/3422039863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614592913f0 0x5614592be640 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fe298009ac0 tx=0x7fe298008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1415f31b09bf6d0f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:02.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.141+0000 7fe29dc00640 1 -- 172.21.15.86:0/3422039863 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe298050070 con 0x5614592913f0 2024-04-20T21:24:02.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.141+0000 7fe29dc00640 1 -- 172.21.15.86:0/3422039863 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe29803a040 con 0x5614592913f0 2024-04-20T21:24:02.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.141+0000 7fe29dc00640 1 -- 172.21.15.86:0/3422039863 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29802cc90 con 0x5614592913f0 2024-04-20T21:24:02.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.142+0000 7fe2b6dc1740 1 -- 172.21.15.86:0/3422039863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614592913f0 msgr2=0x5614592be640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:02.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.142+0000 7fe2b6dc1740 1 --2- 172.21.15.86:0/3422039863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614592913f0 0x5614592be640 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fe298009ac0 tx=0x7fe298008030 comp rx=0 tx=0).stop 2024-04-20T21:24:02.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.142+0000 7fe2b6dc1740 1 -- 172.21.15.86:0/3422039863 shutdown_connections 2024-04-20T21:24:02.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.142+0000 7fe2b6dc1740 1 --2- 172.21.15.86:0/3422039863 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561459294f50 0x5614592beb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.142+0000 7fe2b6dc1740 1 --2- 172.21.15.86:0/3422039863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614592913f0 0x5614592be640 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.142+0000 7fe2b6dc1740 1 --2- 172.21.15.86:0/3422039863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614592c13d0 0x5614592c17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.142+0000 7fe2b6dc1740 1 -- 172.21.15.86:0/3422039863 >> 172.21.15.86:0/3422039863 conn(0x561459293660 msgr2=0x5614592dc1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:02.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.142+0000 7fe2b6dc1740 1 -- 172.21.15.86:0/3422039863 shutdown_connections 2024-04-20T21:24:02.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.142+0000 7fe2b6dc1740 1 -- 172.21.15.86:0/3422039863 wait complete. 2024-04-20T21:24:02.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.143+0000 7fe2b6dc1740 1 -- start start 2024-04-20T21:24:02.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.143+0000 7fe2b6dc1740 1 Processor -- start 2024-04-20T21:24:02.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.144+0000 7fe2b6dc1740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561459294f50 0x5614592ef460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:02.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.144+0000 7fe2b6dc1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614592c13d0 0x5614592dc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:02.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.144+0000 7fe2b6dc1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614592a5f00 0x5614592dc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:02.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.144+0000 7fe2b6dc1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5614591fd150 con 0x5614592a5f00 2024-04-20T21:24:02.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.144+0000 7fe2b6dc1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5614591b6740 con 0x5614592c13d0 2024-04-20T21:24:02.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.144+0000 7fe2b6dc1740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x561459296100 con 0x561459294f50 2024-04-20T21:24:02.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.144+0000 7fe2b6dc1740 10 client.0 _ll_register_callbacks cb 0x5614591b1b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:24:02.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.144+0000 7fe2b2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614592a5f00 0x5614592dc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:02.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.144+0000 7fe2b2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614592a5f00 0x5614592dc6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37104/0 (socket says 172.21.15.86:37104) 2024-04-20T21:24:02.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.144+0000 7fe2b2000640 1 -- 172.21.15.86:0/4106595472 learned_addr learned my addr 172.21.15.86:0/4106595472 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:02.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.145+0000 7fe2b1600640 1 --2- 172.21.15.86:0/4106595472 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561459294f50 0x5614592ef460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:02.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.145+0000 7fe2b2000640 1 -- 172.21.15.86:0/4106595472 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561459294f50 msgr2=0x5614592ef460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:02.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.145+0000 7fe2b2000640 1 --2- 172.21.15.86:0/4106595472 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561459294f50 0x5614592ef460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.145+0000 7fe2b2000640 1 -- 172.21.15.86:0/4106595472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614592c13d0 msgr2=0x5614592dc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:02.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.145+0000 7fe2b2000640 1 --2- 172.21.15.86:0/4106595472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614592c13d0 0x5614592dc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.145+0000 7fe2b2000640 1 -- 172.21.15.86:0/4106595472 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561458f0f3f0 con 0x5614592a5f00 2024-04-20T21:24:02.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.145+0000 7fe2b2000640 1 --2- 172.21.15.86:0/4106595472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614592a5f00 0x5614592dc6b0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac070250 tx=0x7fe2a400eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2200eba6ccdaf7ce server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:02.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.145+0000 7fe292000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:24:02.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.145+0000 7fe292000640 1 -- 172.21.15.86:0/4106595472 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4019060 con 0x5614592a5f00 2024-04-20T21:24:02.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.145+0000 7fe292000640 1 -- 172.21.15.86:0/4106595472 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(74..74 src has 1..74) ==== 5882+0+0 (secure 0 0 0) 0x7fe2a4004d60 con 0x5614592a5f00 2024-04-20T21:24:02.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.146+0000 7fe2b6dc1740 1 -- 172.21.15.86:0/4106595472 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe29803a040 con 0x5614592a5f00 2024-04-20T21:24:02.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.146+0000 7fe292000640 1 -- 172.21.15.86:0/4106595472 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fe2a400c610 con 0x5614592a5f00 2024-04-20T21:24:02.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.146+0000 7fe2b6dc1740 10 client.7167 fetch_fsmap learned FSMap version 49 2024-04-20T21:24:02.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.146+0000 7fe2b6dc1740 1 -- 172.21.15.86:0/4106595472 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) -- 0x561458ecf0f0 con 0x5614592a5f00 2024-04-20T21:24:02.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.146+0000 7fe292000640 1 -- 172.21.15.86:0/4106595472 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe29803a040 con 0x5614592a5f00 2024-04-20T21:24:02.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.146+0000 7fe292000640 1 -- 172.21.15.86:0/4106595472 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 49) ==== 3495+0+0 (secure 0 0 0) 0x7fe2a400c7b0 con 0x5614592a5f00 2024-04-20T21:24:02.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.146+0000 7fe2b6dc1740 10 client.7167 fetch_fsmap finished waiting for FSMap version 49 2024-04-20T21:24:02.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.146+0000 7fe2b6dc1740 10 client.7167 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-20T21:24:02.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.146+0000 7fe2b6dc1740 20 client.7167 populate_metadata read hostname 'smithi086' 2024-04-20T21:24:02.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.146+0000 7fe2b6dc1740 1 --2- 172.21.15.86:0/4106595472 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x5614592da420 0x5614592efb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:02.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.146+0000 7fe2b6dc1740 4 client.7167 mds_command: new command op to 6987 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:24:02.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.146+0000 7fe2b6dc1740 1 -- 172.21.15.86:0/4106595472 --> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56145920ab70 con 0x5614592da420 2024-04-20T21:24:02.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.147+0000 7fe2b1600640 1 --2- 172.21.15.86:0/4106595472 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x5614592da420 0x5614592efb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:02.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.147+0000 7fe2b1600640 1 --2- 172.21.15.86:0/4106595472 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x5614592da420 0x5614592efb80 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dbe51957cc47c78 server_cookie=e517a743d371a123 in_seq=0 out_seq=0 2024-04-20T21:24:02.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.147+0000 7fe292000640 10 client.7167 ms_handle_connect on v2:172.21.15.86:6835/2254117576 2024-04-20T21:24:02.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.148+0000 7fe292000640 1 -- 172.21.15.86:0/4106595472 <== mds.0 v2:172.21.15.86:6835/2254117576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56145920ab70 con 0x5614592da420 2024-04-20T21:24:02.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.148+0000 7fe292000640 10 client.7167 handle_command_reply: tid=0 2024-04-20T21:24:02.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.148+0000 7fe2b6dc1740 1 client.7167 shutdown 2024-04-20T21:24:02.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.148+0000 7fe2b6dc1740 1 -- 172.21.15.86:0/4106595472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614592a5f00 msgr2=0x5614592dc6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:02.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.148+0000 7fe2b6dc1740 1 --2- 172.21.15.86:0/4106595472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614592a5f00 0x5614592dc6b0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac070250 tx=0x7fe2a400eb30 comp rx=0 tx=0).stop 2024-04-20T21:24:02.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.148+0000 7fe2b6dc1740 1 -- 172.21.15.86:0/4106595472 shutdown_connections 2024-04-20T21:24:02.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.148+0000 7fe2b6dc1740 1 --2- 172.21.15.86:0/4106595472 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x5614592da420 0x5614592efb80 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.148+0000 7fe2b6dc1740 1 --2- 172.21.15.86:0/4106595472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614592a5f00 0x5614592dc6b0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.148+0000 7fe2b6dc1740 1 --2- 172.21.15.86:0/4106595472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614592c13d0 0x5614592dc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.148+0000 7fe2b6dc1740 1 --2- 172.21.15.86:0/4106595472 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561459294f50 0x5614592ef460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.148+0000 7fe292000640 0 client.7167 ms_handle_reset on v2:172.21.15.86:6835/2254117576 2024-04-20T21:24:02.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.148+0000 7fe2b6dc1740 1 -- 172.21.15.86:0/4106595472 >> 172.21.15.86:0/4106595472 conn(0x5614592913f0 msgr2=0x5614592c4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:02.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.148+0000 7fe2b6dc1740 1 -- 172.21.15.86:0/4106595472 shutdown_connections 2024-04-20T21:24:02.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.148+0000 7fe2b6dc1740 1 -- 172.21.15.86:0/4106595472 wait complete. 2024-04-20T21:24:02.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.149+0000 7fe2b6dc1740 20 client.7167 trim_cache size 0 max 16384 2024-04-20T21:24:02.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.167+0000 7fe2b6dc1740 1 Processor -- start 2024-04-20T21:24:02.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.168+0000 7fe2b6dc1740 1 -- start start 2024-04-20T21:24:02.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.168+0000 7fe2b6dc1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561459294f50 0x5614592df6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:02.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.168+0000 7fe2b6dc1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614592a5f00 0x5614592decf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:02.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.168+0000 7fe2b6dc1740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5614592c13d0 0x5614592be110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:02.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.168+0000 7fe2b6dc1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ac201660 con 0x5614592a5f00 2024-04-20T21:24:02.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.168+0000 7fe2b6dc1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ac200650 con 0x561459294f50 2024-04-20T21:24:02.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.168+0000 7fe2b6dc1740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe2ac150090 con 0x5614592c13d0 2024-04-20T21:24:02.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.169+0000 7fe2b0c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614592a5f00 0x5614592decf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:02.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.169+0000 7fe2b0c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614592a5f00 0x5614592decf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37116/0 (socket says 172.21.15.86:37116) 2024-04-20T21:24:02.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.169+0000 7fe2b0c00640 1 -- 172.21.15.86:0/1504058845 learned_addr learned my addr 172.21.15.86:0/1504058845 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:02.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.169+0000 7fe2b2000640 1 --2- 172.21.15.86:0/1504058845 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5614592c13d0 0x5614592be110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:02.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.169+0000 7fe2b1600640 1 --2- 172.21.15.86:0/1504058845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561459294f50 0x5614592df6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:02.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.169+0000 7fe2b0c00640 1 -- 172.21.15.86:0/1504058845 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5614592c13d0 msgr2=0x5614592be110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:02.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.169+0000 7fe2b0c00640 1 --2- 172.21.15.86:0/1504058845 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5614592c13d0 0x5614592be110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.169+0000 7fe2b0c00640 1 -- 172.21.15.86:0/1504058845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561459294f50 msgr2=0x5614592df6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:02.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.170+0000 7fe2b0c00640 1 --2- 172.21.15.86:0/1504058845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561459294f50 0x5614592df6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.170+0000 7fe2b0c00640 1 -- 172.21.15.86:0/1504058845 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614592bc420 con 0x5614592a5f00 2024-04-20T21:24:02.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.170+0000 7fe2b0c00640 1 --2- 172.21.15.86:0/1504058845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614592a5f00 0x5614592decf0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fe298037040 tx=0x7fe29802e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b3639051cdaa9a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:02.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.170+0000 7fe29d200640 1 -- 172.21.15.86:0/1504058845 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29805b070 con 0x5614592a5f00 2024-04-20T21:24:02.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.170+0000 7fe29d200640 1 -- 172.21.15.86:0/1504058845 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe29803a040 con 0x5614592a5f00 2024-04-20T21:24:02.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.170+0000 7fe29d200640 1 -- 172.21.15.86:0/1504058845 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe298050040 con 0x5614592a5f00 2024-04-20T21:24:02.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.171+0000 7fe2b6dc1740 1 -- 172.21.15.86:0/1504058845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614592a5f00 msgr2=0x5614592decf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:02.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.171+0000 7fe2b6dc1740 1 --2- 172.21.15.86:0/1504058845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614592a5f00 0x5614592decf0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fe298037040 tx=0x7fe29802e070 comp rx=0 tx=0).stop 2024-04-20T21:24:02.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.171+0000 7fe2b6dc1740 1 -- 172.21.15.86:0/1504058845 shutdown_connections 2024-04-20T21:24:02.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.171+0000 7fe2b6dc1740 1 --2- 172.21.15.86:0/1504058845 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5614592c13d0 0x5614592be110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.171+0000 7fe2b6dc1740 1 --2- 172.21.15.86:0/1504058845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614592a5f00 0x5614592decf0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.171+0000 7fe2b6dc1740 1 --2- 172.21.15.86:0/1504058845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561459294f50 0x5614592df6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.171+0000 7fe2b6dc1740 1 -- 172.21.15.86:0/1504058845 >> 172.21.15.86:0/1504058845 conn(0x5614592913f0 msgr2=0x5614592a88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:02.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.171+0000 7fe2b6dc1740 1 -- 172.21.15.86:0/1504058845 shutdown_connections 2024-04-20T21:24:02.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.171+0000 7fe2b6dc1740 1 -- 172.21.15.86:0/1504058845 wait complete. 2024-04-20T21:24:02.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.172+0000 7fe2b6dc1740 1 -- start start 2024-04-20T21:24:02.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.172+0000 7fe2b6dc1740 1 Processor -- start 2024-04-20T21:24:02.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.173+0000 7fe2b6dc1740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561459294f50 0x5614592ddfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:02.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.173+0000 7fe2b6dc1740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614592a5f00 0x5614592b5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:02.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.173+0000 7fe2b6dc1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614592c13d0 0x5614592be010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:02.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.173+0000 7fe2b6dc1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ac200000 con 0x5614592c13d0 2024-04-20T21:24:02.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.173+0000 7fe2b6dc1740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56145920ab70 con 0x5614592a5f00 2024-04-20T21:24:02.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.173+0000 7fe2b6dc1740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5614592b5d30 con 0x561459294f50 2024-04-20T21:24:02.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.173+0000 7fe2b6dc1740 10 client.0 _ll_register_callbacks cb 0x7fe2a4001730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:24:02.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.173+0000 7fe2b2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614592c13d0 0x5614592be010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:02.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.173+0000 7fe2b2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614592c13d0 0x5614592be010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37124/0 (socket says 172.21.15.86:37124) 2024-04-20T21:24:02.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.173+0000 7fe2b2000640 1 -- 172.21.15.86:0/807596963 learned_addr learned my addr 172.21.15.86:0/807596963 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:02.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.174+0000 7fe2b1600640 1 --2- 172.21.15.86:0/807596963 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561459294f50 0x5614592ddfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:02.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.174+0000 7fe2b2000640 1 -- 172.21.15.86:0/807596963 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561459294f50 msgr2=0x5614592ddfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:02.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.174+0000 7fe2b2000640 1 --2- 172.21.15.86:0/807596963 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561459294f50 0x5614592ddfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.174+0000 7fe2b0c00640 1 --2- 172.21.15.86:0/807596963 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614592a5f00 0x5614592b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:02.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.174+0000 7fe2b2000640 1 -- 172.21.15.86:0/807596963 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614592a5f00 msgr2=0x5614592b5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:02.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.174+0000 7fe2b2000640 1 --2- 172.21.15.86:0/807596963 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614592a5f00 0x5614592b5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.174+0000 7fe2b2000640 1 -- 172.21.15.86:0/807596963 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614592fffa0 con 0x5614592c13d0 2024-04-20T21:24:02.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.174+0000 7fe2b1600640 1 --2- 172.21.15.86:0/807596963 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561459294f50 0x5614592ddfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:24:02.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.174+0000 7fe2b2000640 1 --2- 172.21.15.86:0/807596963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614592c13d0 0x5614592be010 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x5614592a9070 tx=0x7fe2ac15c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c3c38f15aa7874 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:02.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.174+0000 7fe292000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:24:02.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.174+0000 7fe292000640 1 -- 172.21.15.86:0/807596963 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a402d060 con 0x5614592c13d0 2024-04-20T21:24:02.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.175+0000 7fe292000640 1 -- 172.21.15.86:0/807596963 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(74..74 src has 1..74) ==== 5882+0+0 (secure 0 0 0) 0x7fe2a4004d70 con 0x5614592c13d0 2024-04-20T21:24:02.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.175+0000 7fe2b6dc1740 1 -- 172.21.15.86:0/807596963 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe29803a040 con 0x5614592c13d0 2024-04-20T21:24:02.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.175+0000 7fe292000640 1 -- 172.21.15.86:0/807596963 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fe2a4013440 con 0x5614592c13d0 2024-04-20T21:24:02.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.175+0000 7fe2b6dc1740 10 client.7173 fetch_fsmap learned FSMap version 49 2024-04-20T21:24:02.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.175+0000 7fe2b6dc1740 1 -- 172.21.15.86:0/807596963 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) -- 0x5614592bbab0 con 0x5614592c13d0 2024-04-20T21:24:02.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.175+0000 7fe292000640 1 -- 172.21.15.86:0/807596963 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe29803a040 con 0x5614592c13d0 2024-04-20T21:24:02.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.175+0000 7fe292000640 1 -- 172.21.15.86:0/807596963 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 49) ==== 3495+0+0 (secure 0 0 0) 0x7fe2a40135e0 con 0x5614592c13d0 2024-04-20T21:24:02.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.175+0000 7fe2b6dc1740 10 client.7173 fetch_fsmap finished waiting for FSMap version 49 2024-04-20T21:24:02.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.175+0000 7fe2b6dc1740 10 client.7173 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-20T21:24:02.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.175+0000 7fe2b6dc1740 20 client.7173 populate_metadata read hostname 'smithi086' 2024-04-20T21:24:02.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.175+0000 7fe2b6dc1740 1 --2- 172.21.15.86:0/807596963 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x5614592b5660 0x5614592be550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:02.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.175+0000 7fe2b6dc1740 4 client.7173 mds_command: new command op to 6987 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:24:02.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.175+0000 7fe2b6dc1740 1 -- 172.21.15.86:0/807596963 --> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561459292790 con 0x5614592b5660 2024-04-20T21:24:02.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.176+0000 7fe2b1600640 1 --2- 172.21.15.86:0/807596963 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x5614592b5660 0x5614592be550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:02.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.176+0000 7fe292000640 10 client.7173 ms_handle_connect on v2:172.21.15.86:6835/2254117576 2024-04-20T21:24:02.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.176+0000 7fe2b1600640 1 --2- 172.21.15.86:0/807596963 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x5614592b5660 0x5614592be550 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f46b5497936ce8b server_cookie=37bb5384f7e3efe9 in_seq=0 out_seq=0 2024-04-20T21:24:02.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.176+0000 7fe292000640 1 -- 172.21.15.86:0/807596963 <== mds.0 v2:172.21.15.86:6835/2254117576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561459292790 con 0x5614592b5660 2024-04-20T21:24:02.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.176+0000 7fe292000640 10 client.7173 handle_command_reply: tid=0 2024-04-20T21:24:02.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.176+0000 7fe2b6dc1740 1 client.7173 shutdown 2024-04-20T21:24:02.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.177+0000 7fe2b6dc1740 1 -- 172.21.15.86:0/807596963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614592c13d0 msgr2=0x5614592be010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:02.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.177+0000 7fe2b6dc1740 1 --2- 172.21.15.86:0/807596963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614592c13d0 0x5614592be010 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x5614592a9070 tx=0x7fe2ac15c460 comp rx=0 tx=0).stop 2024-04-20T21:24:02.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.177+0000 7fe2b6dc1740 1 -- 172.21.15.86:0/807596963 shutdown_connections 2024-04-20T21:24:02.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.177+0000 7fe2b6dc1740 1 --2- 172.21.15.86:0/807596963 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x5614592b5660 0x5614592be550 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.177+0000 7fe2b6dc1740 1 --2- 172.21.15.86:0/807596963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614592c13d0 0x5614592be010 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.177+0000 7fe292000640 0 client.7173 ms_handle_reset on v2:172.21.15.86:6835/2254117576 2024-04-20T21:24:02.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.177+0000 7fe2b6dc1740 1 --2- 172.21.15.86:0/807596963 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5614592a5f00 0x5614592b5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.177+0000 7fe2b6dc1740 1 --2- 172.21.15.86:0/807596963 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561459294f50 0x5614592ddfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.177+0000 7fe2b6dc1740 1 -- 172.21.15.86:0/807596963 >> 172.21.15.86:0/807596963 conn(0x5614592913f0 msgr2=0x561459290280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:02.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.177+0000 7fe2b6dc1740 1 -- 172.21.15.86:0/807596963 shutdown_connections 2024-04-20T21:24:02.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.177+0000 7fe2b6dc1740 1 -- 172.21.15.86:0/807596963 wait complete. 2024-04-20T21:24:02.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.177+0000 7fe2b6dc1740 20 client.7173 trim_cache size 0 max 16384 2024-04-20T21:24:02.180 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:24:02.180 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-20T21:24:02.180 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-20T21:24:02.180 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:24:02.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.178+0000 7fe2b2a00640 1 -- 172.21.15.86:0/133397887 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe28c06c8f0 msgr2=0x7fe28c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:02.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.178+0000 7fe2b2a00640 1 --2- 172.21.15.86:0/133397887 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe28c06c8f0 0x7fe28c06edb0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe298036000 tx=0x7fe29802e000 comp rx=0 tx=0).stop 2024-04-20T21:24:02.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.178+0000 7fe2b2a00640 1 -- 172.21.15.86:0/133397887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe294004800 msgr2=0x7fe2ac0aa6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:02.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.178+0000 7fe2b2a00640 1 --2- 172.21.15.86:0/133397887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe294004800 0x7fe2ac0aa6c0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0010860 tx=0x7fe2a0010d20 comp rx=0 tx=0).stop 2024-04-20T21:24:02.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.178+0000 7fe2b2a00640 1 -- 172.21.15.86:0/133397887 shutdown_connections 2024-04-20T21:24:02.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.178+0000 7fe2b2a00640 1 --2- 172.21.15.86:0/133397887 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe28c06c8f0 0x7fe28c06edb0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.178+0000 7fe2b2a00640 1 --2- 172.21.15.86:0/133397887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2ac0c5930 0x7fe2ac146910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.178+0000 7fe2b2a00640 1 --2- 172.21.15.86:0/133397887 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe2ac0c6550 0x7fe2ac0c53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.178+0000 7fe2b2a00640 1 --2- 172.21.15.86:0/133397887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe294004800 0x7fe2ac0aa6c0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.178+0000 7fe2b2a00640 1 -- 172.21.15.86:0/133397887 >> 172.21.15.86:0/133397887 conn(0x7fe2ac067cf0 msgr2=0x7fe2ac15ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:02.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.178+0000 7fe2b2a00640 1 -- 172.21.15.86:0/133397887 shutdown_connections 2024-04-20T21:24:02.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.178+0000 7fe2b2a00640 1 -- 172.21.15.86:0/133397887 wait complete. 2024-04-20T21:24:02.196 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T21:24:02.196 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:24:02.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.396+0000 7f86eb800640 1 Processor -- start 2024-04-20T21:24:02.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.396+0000 7f86eb800640 1 -- start start 2024-04-20T21:24:02.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.398+0000 7f86eb800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f86e406e6d0 0x7f86e4071d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:02.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.398+0000 7f86eb800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86e406eb30 con 0x7f86e4146130 2024-04-20T21:24:02.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.398+0000 7f86eb800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86e406dc30 con 0x7f86e4072240 2024-04-20T21:24:02.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.398+0000 7f86eb800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f86e4072690 con 0x7f86e406e6d0 2024-04-20T21:24:02.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.398+0000 7f86e3e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f86e406e6d0 0x7f86e4071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:02.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.398+0000 7f86e3e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f86e406e6d0 0x7f86e4071d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37130/0 (socket says 172.21.15.86:37130) 2024-04-20T21:24:02.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.398+0000 7f86e3e00640 1 -- 172.21.15.86:0/25088124 learned_addr learned my addr 172.21.15.86:0/25088124 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:02.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.398+0000 7f86e8e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f86e4146130 0x7f86e40ccdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:48232/0 (socket says 172.21.15.86:48232) 2024-04-20T21:24:02.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.398+0000 7f86e3e00640 1 -- 172.21.15.86:0/25088124 >> v1:172.21.15.144:6789/0 conn(0x7f86e4072240 legacy=0x7f86e415cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:02.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.398+0000 7f86e3e00640 1 -- 172.21.15.86:0/25088124 >> v1:172.21.15.86:6789/0 conn(0x7f86e4146130 legacy=0x7f86e40ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:02.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.399+0000 7f86e3e00640 1 -- 172.21.15.86:0/25088124 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86e40cd8e0 con 0x7f86e406e6d0 2024-04-20T21:24:02.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.400+0000 7f86e3e00640 1 --2- 172.21.15.86:0/25088124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86e406e6d0 0x7f86e4071d00 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f86e4073120 tx=0x7f86d402ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33fbe9186f19a334 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:02.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.400+0000 7f86e3400640 1 -- 172.21.15.86:0/25088124 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d402f830 con 0x7f86e406e6d0 2024-04-20T21:24:02.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.400+0000 7f86e3400640 1 -- 172.21.15.86:0/25088124 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86d402f9d0 con 0x7f86e406e6d0 2024-04-20T21:24:02.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.400+0000 7f86e3400640 1 -- 172.21.15.86:0/25088124 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d402fda0 con 0x7f86e406e6d0 2024-04-20T21:24:02.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.400+0000 7f86eb800640 1 -- 172.21.15.86:0/25088124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86e406e6d0 msgr2=0x7f86e4071d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:02.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.400+0000 7f86eb800640 1 --2- 172.21.15.86:0/25088124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86e406e6d0 0x7f86e4071d00 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f86e4073120 tx=0x7f86d402ba10 comp rx=0 tx=0).stop 2024-04-20T21:24:02.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.401+0000 7f86eb800640 1 -- 172.21.15.86:0/25088124 shutdown_connections 2024-04-20T21:24:02.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.401+0000 7f86eb800640 1 --2- 172.21.15.86:0/25088124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86e406e6d0 0x7f86e4071d00 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.401+0000 7f86eb800640 1 -- 172.21.15.86:0/25088124 >> 172.21.15.86:0/25088124 conn(0x7f86e40b2360 msgr2=0x7f86e40b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:02.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.401+0000 7f86eb800640 1 -- 172.21.15.86:0/25088124 shutdown_connections 2024-04-20T21:24:02.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.401+0000 7f86eb800640 1 -- 172.21.15.86:0/25088124 wait complete. 2024-04-20T21:24:02.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.402+0000 7f86eb800640 1 Processor -- start 2024-04-20T21:24:02.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.402+0000 7f86eb800640 1 -- start start 2024-04-20T21:24:02.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.403+0000 7f86eb800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f86e406e6d0 0x7f86e415c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:02.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.403+0000 7f86eb800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86e4072240 0x7f86e4144860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:02.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.403+0000 7f86eb800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86e4146130 0x7f86e4144da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:02.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.403+0000 7f86eb800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f86e406e1b0 con 0x7f86e4072240 2024-04-20T21:24:02.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.403+0000 7f86eb800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f86e406ded0 con 0x7f86e4146130 2024-04-20T21:24:02.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.403+0000 7f86eb800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f86e406f150 con 0x7f86e406e6d0 2024-04-20T21:24:02.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.404+0000 7f86e3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86e4072240 0x7f86e4144860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:02.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.404+0000 7f86e3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86e4072240 0x7f86e4144860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37142/0 (socket says 172.21.15.86:37142) 2024-04-20T21:24:02.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.404+0000 7f86e3e00640 1 -- 172.21.15.86:0/3091977965 learned_addr learned my addr 172.21.15.86:0/3091977965 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:02.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.404+0000 7f86e8e00640 1 --2- 172.21.15.86:0/3091977965 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f86e406e6d0 0x7f86e415c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:02.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.404+0000 7f86e3e00640 1 -- 172.21.15.86:0/3091977965 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f86e406e6d0 msgr2=0x7f86e415c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:02.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.404+0000 7f86e3e00640 1 --2- 172.21.15.86:0/3091977965 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f86e406e6d0 0x7f86e415c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.404+0000 7f86e3e00640 1 -- 172.21.15.86:0/3091977965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86e4146130 msgr2=0x7f86e4144da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:02.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.404+0000 7f86e9800640 1 --2- 172.21.15.86:0/3091977965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86e4146130 0x7f86e4144da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:02.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.404+0000 7f86e3e00640 1 --2- 172.21.15.86:0/3091977965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86e4146130 0x7f86e4144da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.404+0000 7f86e3e00640 1 -- 172.21.15.86:0/3091977965 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86e415cd30 con 0x7f86e4072240 2024-04-20T21:24:02.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.404+0000 7f86e9800640 1 --2- 172.21.15.86:0/3091977965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86e4146130 0x7f86e4144da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:24:02.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.405+0000 7f86e3e00640 1 --2- 172.21.15.86:0/3091977965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86e4072240 0x7f86e4144860 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f86d4000c00 tx=0x7f86d4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:02.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.405+0000 7f86e0c00640 1 -- 172.21.15.86:0/3091977965 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d4037d30 con 0x7f86e4072240 2024-04-20T21:24:02.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.405+0000 7f86e0c00640 1 -- 172.21.15.86:0/3091977965 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86d4046070 con 0x7f86e4072240 2024-04-20T21:24:02.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.405+0000 7f86e0c00640 1 -- 172.21.15.86:0/3091977965 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d4041620 con 0x7f86e4072240 2024-04-20T21:24:02.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.405+0000 7f86eb800640 1 -- 172.21.15.86:0/3091977965 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86e415d010 con 0x7f86e4072240 2024-04-20T21:24:02.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.405+0000 7f86eb800640 1 -- 172.21.15.86:0/3091977965 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86e415d5d0 con 0x7f86e4072240 2024-04-20T21:24:02.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.406+0000 7f86eb800640 1 -- 172.21.15.86:0/3091977965 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86e40689e0 con 0x7f86e4072240 2024-04-20T21:24:02.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.407+0000 7f86e0c00640 1 -- 172.21.15.86:0/3091977965 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f86d4040070 con 0x7f86e4072240 2024-04-20T21:24:02.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.408+0000 7f86e0c00640 1 --2- 172.21.15.86:0/3091977965 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f86c806c8f0 0x7f86c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:02.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.408+0000 7f86e0c00640 1 -- 172.21.15.86:0/3091977965 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5882+0+0 (secure 0 0 0) 0x7f86d40ae500 con 0x7f86e4072240 2024-04-20T21:24:02.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.409+0000 7f86e8e00640 1 --2- 172.21.15.86:0/3091977965 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f86c806c8f0 0x7f86c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:02.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.409+0000 7f86e8e00640 1 --2- 172.21.15.86:0/3091977965 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f86c806c8f0 0x7f86c806edb0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f86d8000f60 tx=0x7f86d8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:02.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.413+0000 7f86e0c00640 1 -- 172.21.15.86:0/3091977965 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f86d4033030 con 0x7f86e4072240 2024-04-20T21:24:02.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.654+0000 7f86eb800640 1 -- 172.21.15.86:0/3091977965 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f86e415d7a0 con 0x7f86e4072240 2024-04-20T21:24:02.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.655+0000 7f86e0c00640 1 -- 172.21.15.86:0/3091977965 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+5205 (secure 0 0 0) 0x7f86d407c9a0 con 0x7f86e4072240 2024-04-20T21:24:02.654 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:24:02.654 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":49,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2740872549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2740872549},{"type":"v1","addr":"172.21.15.86:6836","nonce":2740872549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5048,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/301899138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":301899138},{"type":"v1","addr":"172.21.15.144:6833","nonce":301899138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5051,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2816343379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2816343379},{"type":"v1","addr":"172.21.15.144:6835","nonce":2816343379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:22:45.431100+0000","modified":"2024-04-20T21:23:23.111255+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6987},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6987":{"gid":6987,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":6,"addr":"172.21.15.86:6837/2254117576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2254117576},{"type":"v1","addr":"172.21.15.86:6837","nonce":2254117576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6987,"qdb_cluster":[6987]},"id":5}]} 2024-04-20T21:24:02.655 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 49 2024-04-20T21:24:02.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.658+0000 7f86eb800640 1 -- 172.21.15.86:0/3091977965 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f86c806c8f0 msgr2=0x7f86c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:02.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.658+0000 7f86eb800640 1 --2- 172.21.15.86:0/3091977965 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f86c806c8f0 0x7f86c806edb0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f86d8000f60 tx=0x7f86d8009040 comp rx=0 tx=0).stop 2024-04-20T21:24:02.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.658+0000 7f86eb800640 1 -- 172.21.15.86:0/3091977965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86e4072240 msgr2=0x7f86e4144860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:02.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.658+0000 7f86eb800640 1 --2- 172.21.15.86:0/3091977965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86e4072240 0x7f86e4144860 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f86d4000c00 tx=0x7f86d4004730 comp rx=0 tx=0).stop 2024-04-20T21:24:02.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.658+0000 7f86eb800640 1 -- 172.21.15.86:0/3091977965 shutdown_connections 2024-04-20T21:24:02.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.658+0000 7f86eb800640 1 --2- 172.21.15.86:0/3091977965 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f86c806c8f0 0x7f86c806edb0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.658+0000 7f86eb800640 1 --2- 172.21.15.86:0/3091977965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86e4146130 0x7f86e4144da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.658+0000 7f86eb800640 1 --2- 172.21.15.86:0/3091977965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86e4072240 0x7f86e4144860 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.658+0000 7f86eb800640 1 --2- 172.21.15.86:0/3091977965 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f86e406e6d0 0x7f86e415c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.658+0000 7f86eb800640 1 -- 172.21.15.86:0/3091977965 >> 172.21.15.86:0/3091977965 conn(0x7f86e40b2360 msgr2=0x7f86e4143720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:02.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.658+0000 7f86eb800640 1 -- 172.21.15.86:0/3091977965 shutdown_connections 2024-04-20T21:24:02.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.658+0000 7f86eb800640 1 -- 172.21.15.86:0/3091977965 wait complete. 2024-04-20T21:24:02.675 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-04-20T21:24:02.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.875+0000 7f43b1200640 1 Processor -- start 2024-04-20T21:24:02.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.875+0000 7f43b1200640 1 -- start start 2024-04-20T21:24:02.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.877+0000 7f43b1200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43ac15e8f0 0x7f43ac162640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:02.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.877+0000 7f43b1200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f43ac15ecd0 0x7f43ac162b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:02.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.877+0000 7f43b1200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f43ac0cf680 0x7f43ac0cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:02.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.877+0000 7f43b1200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f43ac0d0010 con 0x7f43ac0cf680 2024-04-20T21:24:02.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.877+0000 7f43b1200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f43ac0d2fd0 con 0x7f43ac15ecd0 2024-04-20T21:24:02.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.877+0000 7f43b1200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f43ac0d29a0 con 0x7f43ac15e8f0 2024-04-20T21:24:02.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.877+0000 7f43aa800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f43ac0cf680 0x7f43ac0cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:02.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.877+0000 7f43aa800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f43ac0cf680 0x7f43ac0cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37156/0 (socket says 172.21.15.86:37156) 2024-04-20T21:24:02.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.877+0000 7f43aa800640 1 -- 172.21.15.86:0/821016364 learned_addr learned my addr 172.21.15.86:0/821016364 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:02.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.877+0000 7f43a9e00640 1 --2- 172.21.15.86:0/821016364 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43ac15e8f0 0x7f43ac162640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:02.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.877+0000 7f43aa800640 1 -- 172.21.15.86:0/821016364 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43ac15e8f0 msgr2=0x7f43ac162640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:02.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.877+0000 7f43a9400640 1 --2- 172.21.15.86:0/821016364 >> v2:172.21.15.144:3300/0 conn(0x7f43ac15ecd0 0x7f43ac162b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:02.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.877+0000 7f43aa800640 1 --2- 172.21.15.86:0/821016364 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43ac15e8f0 0x7f43ac162640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.878+0000 7f43aa800640 1 -- 172.21.15.86:0/821016364 >> v2:172.21.15.144:3300/0 conn(0x7f43ac15ecd0 msgr2=0x7f43ac162b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:02.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.878+0000 7f43aa800640 1 --2- 172.21.15.86:0/821016364 >> v2:172.21.15.144:3300/0 conn(0x7f43ac15ecd0 0x7f43ac162b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.878+0000 7f43aa800640 1 -- 172.21.15.86:0/821016364 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43ac1631b0 con 0x7f43ac0cf680 2024-04-20T21:24:02.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.878+0000 7f43a9e00640 1 --2- 172.21.15.86:0/821016364 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43ac15e8f0 0x7f43ac162640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:24:02.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.879+0000 7f43aa800640 1 --2- 172.21.15.86:0/821016364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43ac0cf680 0x7f43ac0cfad0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f43ac0d5940 tx=0x7f43a402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e8155b62076be1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:02.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.879+0000 7f43a8a00640 1 -- 172.21.15.86:0/821016364 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a402ca70 con 0x7f43ac0cf680 2024-04-20T21:24:02.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.879+0000 7f43a8a00640 1 -- 172.21.15.86:0/821016364 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43a402cc10 con 0x7f43ac0cf680 2024-04-20T21:24:02.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.879+0000 7f43a8a00640 1 -- 172.21.15.86:0/821016364 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a40343f0 con 0x7f43ac0cf680 2024-04-20T21:24:02.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.880+0000 7f43b1200640 1 -- 172.21.15.86:0/821016364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43ac0cf680 msgr2=0x7f43ac0cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:02.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.880+0000 7f43b1200640 1 --2- 172.21.15.86:0/821016364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43ac0cf680 0x7f43ac0cfad0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f43ac0d5940 tx=0x7f43a402bed0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.880+0000 7f43b1200640 1 -- 172.21.15.86:0/821016364 shutdown_connections 2024-04-20T21:24:02.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.880+0000 7f43b1200640 1 --2- 172.21.15.86:0/821016364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43ac0cf680 0x7f43ac0cfad0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.880+0000 7f43b1200640 1 --2- 172.21.15.86:0/821016364 >> v2:172.21.15.144:3300/0 conn(0x7f43ac15ecd0 0x7f43ac162b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.880+0000 7f43b1200640 1 --2- 172.21.15.86:0/821016364 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43ac15e8f0 0x7f43ac162640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.880+0000 7f43b1200640 1 -- 172.21.15.86:0/821016364 >> 172.21.15.86:0/821016364 conn(0x7f43ac06cc70 msgr2=0x7f43ac06d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:02.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.880+0000 7f43b1200640 1 -- 172.21.15.86:0/821016364 shutdown_connections 2024-04-20T21:24:02.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.880+0000 7f43b1200640 1 -- 172.21.15.86:0/821016364 wait complete. 2024-04-20T21:24:02.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.882+0000 7f43b1200640 1 Processor -- start 2024-04-20T21:24:02.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.882+0000 7f43b1200640 1 -- start start 2024-04-20T21:24:02.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.883+0000 7f43b1200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43ac0cf680 0x7f43ac14ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:02.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.883+0000 7f43b1200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43ac15e8f0 0x7f43ac0d4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:02.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.883+0000 7f43b1200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43ac15ecd0 0x7f43ac0d4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:02.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.883+0000 7f43b1200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f43ac0b7a50 con 0x7f43ac15e8f0 2024-04-20T21:24:02.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.883+0000 7f43b1200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f43ac0d2c60 con 0x7f43ac15ecd0 2024-04-20T21:24:02.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.883+0000 7f43b1200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f43ac0ba7f0 con 0x7f43ac0cf680 2024-04-20T21:24:02.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.883+0000 7f43a9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43ac15e8f0 0x7f43ac0d4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:02.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.883+0000 7f43a9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43ac15e8f0 0x7f43ac0d4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37170/0 (socket says 172.21.15.86:37170) 2024-04-20T21:24:02.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.884+0000 7f43a9400640 1 -- 172.21.15.86:0/1436967580 learned_addr learned my addr 172.21.15.86:0/1436967580 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:02.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.884+0000 7f43a9e00640 1 --2- 172.21.15.86:0/1436967580 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43ac0cf680 0x7f43ac14ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:02.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.884+0000 7f43a9400640 1 -- 172.21.15.86:0/1436967580 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43ac0cf680 msgr2=0x7f43ac14ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:02.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.884+0000 7f43aa800640 1 --2- 172.21.15.86:0/1436967580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43ac15ecd0 0x7f43ac0d4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:02.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.884+0000 7f43a9400640 1 --2- 172.21.15.86:0/1436967580 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43ac0cf680 0x7f43ac14ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.884+0000 7f43a9400640 1 -- 172.21.15.86:0/1436967580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43ac15ecd0 msgr2=0x7f43ac0d4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:02.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.884+0000 7f43a9400640 1 --2- 172.21.15.86:0/1436967580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43ac15ecd0 0x7f43ac0d4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.884+0000 7f43a9400640 1 -- 172.21.15.86:0/1436967580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43ac0cb980 con 0x7f43ac15e8f0 2024-04-20T21:24:02.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.884+0000 7f43a9400640 1 --2- 172.21.15.86:0/1436967580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43ac15e8f0 0x7f43ac0d4890 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f439800c820 tx=0x7f439800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:02.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.884+0000 7f43a2000640 1 -- 172.21.15.86:0/1436967580 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4398015070 con 0x7f43ac15e8f0 2024-04-20T21:24:02.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.885+0000 7f43a2000640 1 -- 172.21.15.86:0/1436967580 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4398004220 con 0x7f43ac15e8f0 2024-04-20T21:24:02.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.885+0000 7f43b1200640 1 -- 172.21.15.86:0/1436967580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43ac0cbc20 con 0x7f43ac15e8f0 2024-04-20T21:24:02.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.885+0000 7f43a2000640 1 -- 172.21.15.86:0/1436967580 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439800ce10 con 0x7f43ac15e8f0 2024-04-20T21:24:02.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.885+0000 7f43b1200640 1 -- 172.21.15.86:0/1436967580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43ac0cc1e0 con 0x7f43ac15e8f0 2024-04-20T21:24:02.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.887+0000 7f43a2000640 1 -- 172.21.15.86:0/1436967580 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4398007830 con 0x7f43ac15e8f0 2024-04-20T21:24:02.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.889+0000 7f43a2000640 1 --2- 172.21.15.86:0/1436967580 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f438c06c920 0x7f438c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:02.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.889+0000 7f43a2000640 1 -- 172.21.15.86:0/1436967580 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5882+0+0 (secure 0 0 0) 0x7f439808b9f0 con 0x7f43ac15e8f0 2024-04-20T21:24:02.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.889+0000 7f43a9e00640 1 --2- 172.21.15.86:0/1436967580 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f438c06c920 0x7f438c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:02.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.889+0000 7f43b3fcf740 1 Processor -- start 2024-04-20T21:24:02.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.889+0000 7f43b3fcf740 1 -- start start 2024-04-20T21:24:02.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.890+0000 7f43a9e00640 1 --2- 172.21.15.86:0/1436967580 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f438c06c920 0x7f438c06ede0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f43ac158240 tx=0x7f439c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:02.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.890+0000 7f43b3fcf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b47bf5eb0 0x562b47bc8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:02.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.890+0000 7f43b3fcf740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562b47bf6290 0x562b47bf8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:02.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.890+0000 7f43b3fcf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b47bc9390 0x562b47bf9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:02.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.890+0000 7f43b3fcf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562b47b34060 con 0x562b47bc9390 2024-04-20T21:24:02.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.890+0000 7f43b3fcf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562b47ad9f00 con 0x562b47bf5eb0 2024-04-20T21:24:02.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.890+0000 7f43b3fcf740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x562b47b30020 con 0x562b47bf6290 2024-04-20T21:24:02.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.891+0000 7f43aa800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b47bc9390 0x562b47bf9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:02.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.891+0000 7f43aa800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b47bc9390 0x562b47bf9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37176/0 (socket says 172.21.15.86:37176) 2024-04-20T21:24:02.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.891+0000 7f43aa800640 1 -- 172.21.15.86:0/2586147259 learned_addr learned my addr 172.21.15.86:0/2586147259 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:02.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.891+0000 7f43a9400640 1 --2- 172.21.15.86:0/2586147259 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562b47bf6290 0x562b47bf8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:02.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.891+0000 7f43aa800640 1 -- 172.21.15.86:0/2586147259 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562b47bf6290 msgr2=0x562b47bf8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:02.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.891+0000 7f43aa800640 1 --2- 172.21.15.86:0/2586147259 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562b47bf6290 0x562b47bf8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.891+0000 7f43a9e00640 1 --2- 172.21.15.86:0/2586147259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b47bf5eb0 0x562b47bc8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:02.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.891+0000 7f43aa800640 1 -- 172.21.15.86:0/2586147259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b47bf5eb0 msgr2=0x562b47bc8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:02.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.891+0000 7f43aa800640 1 --2- 172.21.15.86:0/2586147259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b47bf5eb0 0x562b47bc8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.891+0000 7f43aa800640 1 -- 172.21.15.86:0/2586147259 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b47ac2a30 con 0x562b47bc9390 2024-04-20T21:24:02.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.891+0000 7f43aa800640 1 --2- 172.21.15.86:0/2586147259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b47bc9390 0x562b47bf9180 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f43a402c4e0 tx=0x7f43a402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f3c9498478dc1bf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:02.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.892+0000 7f4396a00640 1 -- 172.21.15.86:0/2586147259 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a4045070 con 0x562b47bc9390 2024-04-20T21:24:02.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.892+0000 7f4396a00640 1 -- 172.21.15.86:0/2586147259 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43a402ca90 con 0x562b47bc9390 2024-04-20T21:24:02.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.892+0000 7f4396a00640 1 -- 172.21.15.86:0/2586147259 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a40406b0 con 0x562b47bc9390 2024-04-20T21:24:02.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.892+0000 7f43b3fcf740 1 -- 172.21.15.86:0/2586147259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b47bc9390 msgr2=0x562b47bf9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:02.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.892+0000 7f43b3fcf740 1 --2- 172.21.15.86:0/2586147259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b47bc9390 0x562b47bf9180 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f43a402c4e0 tx=0x7f43a402e000 comp rx=0 tx=0).stop 2024-04-20T21:24:02.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.892+0000 7f43b3fcf740 1 -- 172.21.15.86:0/2586147259 shutdown_connections 2024-04-20T21:24:02.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.892+0000 7f43b3fcf740 1 --2- 172.21.15.86:0/2586147259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b47bc9390 0x562b47bf9180 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.892+0000 7f43b3fcf740 1 --2- 172.21.15.86:0/2586147259 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562b47bf6290 0x562b47bf8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.892+0000 7f43b3fcf740 1 --2- 172.21.15.86:0/2586147259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b47bf5eb0 0x562b47bc8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.892+0000 7f43b3fcf740 1 -- 172.21.15.86:0/2586147259 >> 172.21.15.86:0/2586147259 conn(0x562b47bcb0c0 msgr2=0x562b47c13c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:02.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.893+0000 7f43b3fcf740 1 -- 172.21.15.86:0/2586147259 shutdown_connections 2024-04-20T21:24:02.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.893+0000 7f43b3fcf740 1 -- 172.21.15.86:0/2586147259 wait complete. 2024-04-20T21:24:02.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.893+0000 7f43b3fcf740 1 -- start start 2024-04-20T21:24:02.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.893+0000 7f43b3fcf740 1 Processor -- start 2024-04-20T21:24:02.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.894+0000 7f43b3fcf740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562b47bf5eb0 0x562b47bf3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:02.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.894+0000 7f43b3fcf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b47bf6290 0x562b47c23760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:02.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.894+0000 7f43b3fcf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b47c0a840 0x562b47bfc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:02.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.894+0000 7f43b3fcf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562b47bcd660 con 0x562b47bf6290 2024-04-20T21:24:02.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.894+0000 7f43b3fcf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562b47aed3d0 con 0x562b47c0a840 2024-04-20T21:24:02.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.894+0000 7f43b3fcf740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x562b47bcca50 con 0x562b47bf5eb0 2024-04-20T21:24:02.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.894+0000 7f43b3fcf740 10 client.0 _ll_register_callbacks cb 0x562b47abb960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:24:02.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.895+0000 7f43a9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b47bf6290 0x562b47c23760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:02.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.895+0000 7f43a9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b47bf6290 0x562b47c23760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37184/0 (socket says 172.21.15.86:37184) 2024-04-20T21:24:02.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.895+0000 7f43a9400640 1 -- 172.21.15.86:0/2844221464 learned_addr learned my addr 172.21.15.86:0/2844221464 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:02.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.895+0000 7f43a9e00640 1 --2- 172.21.15.86:0/2844221464 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562b47bf5eb0 0x562b47bf3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:02.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.895+0000 7f43a9400640 1 -- 172.21.15.86:0/2844221464 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562b47bf5eb0 msgr2=0x562b47bf3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:02.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.895+0000 7f43a9400640 1 --2- 172.21.15.86:0/2844221464 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562b47bf5eb0 0x562b47bf3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.895+0000 7f43a9400640 1 -- 172.21.15.86:0/2844221464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b47c0a840 msgr2=0x562b47bfc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:02.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.895+0000 7f43a9400640 1 --2- 172.21.15.86:0/2844221464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b47c0a840 0x562b47bfc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.895+0000 7f43a9400640 1 -- 172.21.15.86:0/2844221464 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b479127c0 con 0x562b47bf6290 2024-04-20T21:24:02.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.895+0000 7f43a9400640 1 --2- 172.21.15.86:0/2844221464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b47bf6290 0x562b47c23760 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f4398007410 tx=0x7f4398059660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf015d1b890d6146 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:02.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.895+0000 7f438b400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:24:02.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.896+0000 7f438b400640 1 -- 172.21.15.86:0/2844221464 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4398071070 con 0x562b47bf6290 2024-04-20T21:24:02.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.896+0000 7f438b400640 1 -- 172.21.15.86:0/2844221464 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(74..74 src has 1..74) ==== 5882+0+0 (secure 0 0 0) 0x7f439805eda0 con 0x562b47bf6290 2024-04-20T21:24:02.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.896+0000 7f43b3fcf740 1 -- 172.21.15.86:0/2844221464 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43a402ca90 con 0x562b47bf6290 2024-04-20T21:24:02.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.896+0000 7f438b400640 1 -- 172.21.15.86:0/2844221464 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f439801e040 con 0x562b47bf6290 2024-04-20T21:24:02.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.896+0000 7f43b3fcf740 10 client.7194 fetch_fsmap learned FSMap version 49 2024-04-20T21:24:02.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.896+0000 7f43b3fcf740 1 -- 172.21.15.86:0/2844221464 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) -- 0x562b47972f50 con 0x562b47bf6290 2024-04-20T21:24:02.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.896+0000 7f438b400640 1 -- 172.21.15.86:0/2844221464 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43a402ca90 con 0x562b47bf6290 2024-04-20T21:24:02.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.896+0000 7f438b400640 1 -- 172.21.15.86:0/2844221464 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 49) ==== 3495+0+0 (secure 0 0 0) 0x7f4398058ab0 con 0x562b47bf6290 2024-04-20T21:24:02.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.896+0000 7f43b3fcf740 10 client.7194 fetch_fsmap finished waiting for FSMap version 49 2024-04-20T21:24:02.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.896+0000 7f43b3fcf740 10 client.7194 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-20T21:24:02.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.896+0000 7f43b3fcf740 20 client.7194 populate_metadata read hostname 'smithi086' 2024-04-20T21:24:02.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.896+0000 7f43b3fcf740 1 --2- 172.21.15.86:0/2844221464 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x562b47bfa290 0x562b47bfa670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:02.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.896+0000 7f43b3fcf740 4 client.7194 mds_command: new command op to 6987 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:24:02.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.896+0000 7f43b3fcf740 1 -- 172.21.15.86:0/2844221464 --> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b47b41a80 con 0x562b47bfa290 2024-04-20T21:24:02.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.897+0000 7f43a9e00640 1 --2- 172.21.15.86:0/2844221464 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x562b47bfa290 0x562b47bfa670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:02.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.897+0000 7f438b400640 10 client.7194 ms_handle_connect on v2:172.21.15.86:6835/2254117576 2024-04-20T21:24:02.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.897+0000 7f43a9e00640 1 --2- 172.21.15.86:0/2844221464 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x562b47bfa290 0x562b47bfa670 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cac9dbf9e9f3891 server_cookie=b8ce4c48fcae32db in_seq=0 out_seq=0 2024-04-20T21:24:02.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.898+0000 7f438b400640 1 -- 172.21.15.86:0/2844221464 <== mds.0 v2:172.21.15.86:6835/2254117576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562b47b41a80 con 0x562b47bfa290 2024-04-20T21:24:02.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.898+0000 7f438b400640 10 client.7194 handle_command_reply: tid=0 2024-04-20T21:24:02.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.898+0000 7f43b3fcf740 1 client.7194 shutdown 2024-04-20T21:24:02.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.898+0000 7f43b3fcf740 1 -- 172.21.15.86:0/2844221464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b47bf6290 msgr2=0x562b47c23760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:02.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.898+0000 7f43b3fcf740 1 --2- 172.21.15.86:0/2844221464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b47bf6290 0x562b47c23760 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f4398007410 tx=0x7f4398059660 comp rx=0 tx=0).stop 2024-04-20T21:24:02.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.898+0000 7f43b3fcf740 1 -- 172.21.15.86:0/2844221464 shutdown_connections 2024-04-20T21:24:02.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.898+0000 7f43b3fcf740 1 --2- 172.21.15.86:0/2844221464 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x562b47bfa290 0x562b47bfa670 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.898+0000 7f43b3fcf740 1 --2- 172.21.15.86:0/2844221464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b47c0a840 0x562b47bfc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.898+0000 7f43b3fcf740 1 --2- 172.21.15.86:0/2844221464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b47bf6290 0x562b47c23760 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.898+0000 7f438b400640 0 client.7194 ms_handle_reset on v2:172.21.15.86:6835/2254117576 2024-04-20T21:24:02.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.898+0000 7f43b3fcf740 1 --2- 172.21.15.86:0/2844221464 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562b47bf5eb0 0x562b47bf3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.898+0000 7f43b3fcf740 1 -- 172.21.15.86:0/2844221464 >> 172.21.15.86:0/2844221464 conn(0x562b47bc9390 msgr2=0x562b47bf8c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:02.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.899+0000 7f43b3fcf740 1 -- 172.21.15.86:0/2844221464 shutdown_connections 2024-04-20T21:24:02.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.899+0000 7f43b3fcf740 1 -- 172.21.15.86:0/2844221464 wait complete. 2024-04-20T21:24:02.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.899+0000 7f43b3fcf740 20 client.7194 trim_cache size 0 max 16384 2024-04-20T21:24:02.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.913+0000 7f43b3fcf740 1 Processor -- start 2024-04-20T21:24:02.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.913+0000 7f43b3fcf740 1 -- start start 2024-04-20T21:24:02.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.914+0000 7f43b3fcf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b47bf5eb0 0x562b47bf4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:02.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.914+0000 7f43b3fcf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b47bf6290 0x562b47bf89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:02.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.914+0000 7f43b3fcf740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562b47c0a840 0x562b47bfb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:02.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.914+0000 7f43b3fcf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f43ac0cdb30 con 0x562b47bf6290 2024-04-20T21:24:02.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.914+0000 7f43b3fcf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f43ac1509c0 con 0x562b47bf5eb0 2024-04-20T21:24:02.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.914+0000 7f43b3fcf740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f43ac0cd4c0 con 0x562b47c0a840 2024-04-20T21:24:02.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.914+0000 7f43a9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b47bf6290 0x562b47bf89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:02.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.914+0000 7f43a9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b47bf6290 0x562b47bf89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37192/0 (socket says 172.21.15.86:37192) 2024-04-20T21:24:02.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.914+0000 7f43a9400640 1 -- 172.21.15.86:0/785268783 learned_addr learned my addr 172.21.15.86:0/785268783 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:02.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.915+0000 7f43aa800640 1 --2- 172.21.15.86:0/785268783 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562b47c0a840 0x562b47bfb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:02.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.915+0000 7f43a9e00640 1 --2- 172.21.15.86:0/785268783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b47bf5eb0 0x562b47bf4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:02.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.915+0000 7f43a9400640 1 -- 172.21.15.86:0/785268783 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562b47c0a840 msgr2=0x562b47bfb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:02.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.915+0000 7f43a9400640 1 --2- 172.21.15.86:0/785268783 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562b47c0a840 0x562b47bfb220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.915+0000 7f43a9400640 1 -- 172.21.15.86:0/785268783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b47bf5eb0 msgr2=0x562b47bf4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:02.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.915+0000 7f43a9400640 1 --2- 172.21.15.86:0/785268783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b47bf5eb0 0x562b47bf4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.915+0000 7f43a9400640 1 -- 172.21.15.86:0/785268783 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b478060b0 con 0x562b47bf6290 2024-04-20T21:24:02.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.915+0000 7f43a9400640 1 --2- 172.21.15.86:0/785268783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b47bf6290 0x562b47bf89b0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f43980595f0 tx=0x7f439801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=353a5de9c0ee9d8a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:02.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.915+0000 7f4396000640 1 -- 172.21.15.86:0/785268783 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4398071040 con 0x562b47bf6290 2024-04-20T21:24:02.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.915+0000 7f4396000640 1 -- 172.21.15.86:0/785268783 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f439805eba0 con 0x562b47bf6290 2024-04-20T21:24:02.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.915+0000 7f4396000640 1 -- 172.21.15.86:0/785268783 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4398007b70 con 0x562b47bf6290 2024-04-20T21:24:02.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.916+0000 7f43b3fcf740 1 -- 172.21.15.86:0/785268783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b47bf6290 msgr2=0x562b47bf89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:02.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.916+0000 7f43b3fcf740 1 --2- 172.21.15.86:0/785268783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b47bf6290 0x562b47bf89b0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f43980595f0 tx=0x7f439801b000 comp rx=0 tx=0).stop 2024-04-20T21:24:02.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.916+0000 7f43b3fcf740 1 -- 172.21.15.86:0/785268783 shutdown_connections 2024-04-20T21:24:02.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.916+0000 7f43b3fcf740 1 --2- 172.21.15.86:0/785268783 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562b47c0a840 0x562b47bfb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.916+0000 7f43b3fcf740 1 --2- 172.21.15.86:0/785268783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b47bf6290 0x562b47bf89b0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.916+0000 7f43b3fcf740 1 --2- 172.21.15.86:0/785268783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b47bf5eb0 0x562b47bf4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.916+0000 7f43b3fcf740 1 -- 172.21.15.86:0/785268783 >> 172.21.15.86:0/785268783 conn(0x562b47bc9390 msgr2=0x562b47c0d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:02.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.916+0000 7f43b3fcf740 1 -- 172.21.15.86:0/785268783 shutdown_connections 2024-04-20T21:24:02.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.916+0000 7f43b3fcf740 1 -- 172.21.15.86:0/785268783 wait complete. 2024-04-20T21:24:02.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.917+0000 7f43b3fcf740 1 -- start start 2024-04-20T21:24:02.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.917+0000 7f43b3fcf740 1 Processor -- start 2024-04-20T21:24:02.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.918+0000 7f43b3fcf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b47bf5eb0 0x562b47c09fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:02.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.918+0000 7f43b3fcf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b47bf6290 0x562b47bfc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:02.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.918+0000 7f43b3fcf740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562b47c0a840 0x562b47bfcb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:02.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.918+0000 7f43b3fcf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562b47c0b920 con 0x562b47bf6290 2024-04-20T21:24:02.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.918+0000 7f43b3fcf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562b47c0b310 con 0x562b47bf5eb0 2024-04-20T21:24:02.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.918+0000 7f43b3fcf740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x562b47c0cd80 con 0x562b47c0a840 2024-04-20T21:24:02.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.918+0000 7f43b3fcf740 10 client.0 _ll_register_callbacks cb 0x7f43980590b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:24:02.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.918+0000 7f43a9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b47bf6290 0x562b47bfc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:02.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.918+0000 7f43a9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b47bf6290 0x562b47bfc620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37196/0 (socket says 172.21.15.86:37196) 2024-04-20T21:24:02.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.918+0000 7f43a9400640 1 -- 172.21.15.86:0/2598330932 learned_addr learned my addr 172.21.15.86:0/2598330932 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:02.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.919+0000 7f43a9e00640 1 --2- 172.21.15.86:0/2598330932 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b47bf5eb0 0x562b47c09fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:02.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.919+0000 7f43a9400640 1 -- 172.21.15.86:0/2598330932 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562b47c0a840 msgr2=0x562b47bfcb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:02.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.919+0000 7f43aa800640 1 --2- 172.21.15.86:0/2598330932 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562b47c0a840 0x562b47bfcb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:02.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.919+0000 7f43a9400640 1 --2- 172.21.15.86:0/2598330932 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562b47c0a840 0x562b47bfcb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.919+0000 7f43a9400640 1 -- 172.21.15.86:0/2598330932 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b47bf5eb0 msgr2=0x562b47c09fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:02.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.919+0000 7f43a9400640 1 --2- 172.21.15.86:0/2598330932 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b47bf5eb0 0x562b47c09fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.919+0000 7f43a9400640 1 -- 172.21.15.86:0/2598330932 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b47c17130 con 0x562b47bf6290 2024-04-20T21:24:02.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.919+0000 7f43a9e00640 1 --2- 172.21.15.86:0/2598330932 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b47bf5eb0 0x562b47c09fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:24:02.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.919+0000 7f43a9400640 1 --2- 172.21.15.86:0/2598330932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b47bf6290 0x562b47bfc620 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f4398007410 tx=0x7f4398089d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cbd1bd8d3ead954 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:02.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.919+0000 7f438b400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:24:02.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.919+0000 7f438b400640 1 -- 172.21.15.86:0/2598330932 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439807e060 con 0x562b47bf6290 2024-04-20T21:24:02.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.919+0000 7f438b400640 1 -- 172.21.15.86:0/2598330932 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(74..74 src has 1..74) ==== 5882+0+0 (secure 0 0 0) 0x7f4398007d70 con 0x562b47bf6290 2024-04-20T21:24:02.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.919+0000 7f43b3fcf740 1 -- 172.21.15.86:0/2598330932 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f439805eba0 con 0x562b47bf6290 2024-04-20T21:24:02.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.919+0000 7f438b400640 1 -- 172.21.15.86:0/2598330932 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f4398071040 con 0x562b47bf6290 2024-04-20T21:24:02.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.919+0000 7f43b3fcf740 10 client.7200 fetch_fsmap learned FSMap version 49 2024-04-20T21:24:02.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.919+0000 7f43b3fcf740 1 -- 172.21.15.86:0/2598330932 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) -- 0x562b47bf40f0 con 0x562b47bf6290 2024-04-20T21:24:02.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.920+0000 7f438b400640 1 -- 172.21.15.86:0/2598330932 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f439805eba0 con 0x562b47bf6290 2024-04-20T21:24:02.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.920+0000 7f438b400640 1 -- 172.21.15.86:0/2598330932 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 49) ==== 3495+0+0 (secure 0 0 0) 0x7f43980553b0 con 0x562b47bf6290 2024-04-20T21:24:02.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.920+0000 7f43b3fcf740 10 client.7200 fetch_fsmap finished waiting for FSMap version 49 2024-04-20T21:24:02.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.920+0000 7f43b3fcf740 10 client.7200 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-20T21:24:02.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.920+0000 7f43b3fcf740 20 client.7200 populate_metadata read hostname 'smithi086' 2024-04-20T21:24:02.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.920+0000 7f43b3fcf740 1 --2- 172.21.15.86:0/2598330932 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x562b47bf2e70 0x562b47bf3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:02.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.920+0000 7f43b3fcf740 4 client.7200 mds_command: new command op to 6987 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-20T21:24:02.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.920+0000 7f43b3fcf740 1 -- 172.21.15.86:0/2598330932 --> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x562b47b3b8f0 con 0x562b47bf2e70 2024-04-20T21:24:02.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.920+0000 7f43a9e00640 1 --2- 172.21.15.86:0/2598330932 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x562b47bf2e70 0x562b47bf3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:02.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.921+0000 7f438b400640 10 client.7200 ms_handle_connect on v2:172.21.15.86:6835/2254117576 2024-04-20T21:24:02.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.921+0000 7f43a9e00640 1 --2- 172.21.15.86:0/2598330932 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x562b47bf2e70 0x562b47bf3250 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=129529feb39bb3c7 server_cookie=f8d152f809fab50c in_seq=0 out_seq=0 2024-04-20T21:24:02.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.921+0000 7f438b400640 1 -- 172.21.15.86:0/2598330932 <== mds.0 v2:172.21.15.86:6835/2254117576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+237 (crc 0 0 0) 0x562b47b3b8f0 con 0x562b47bf2e70 2024-04-20T21:24:02.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.921+0000 7f438b400640 10 client.7200 handle_command_reply: tid=0 2024-04-20T21:24:02.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.921+0000 7f43b3fcf740 1 client.7200 shutdown 2024-04-20T21:24:02.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.921+0000 7f43b3fcf740 1 -- 172.21.15.86:0/2598330932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b47bf6290 msgr2=0x562b47bfc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:02.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.921+0000 7f43b3fcf740 1 --2- 172.21.15.86:0/2598330932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b47bf6290 0x562b47bfc620 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f4398007410 tx=0x7f4398089d30 comp rx=0 tx=0).stop 2024-04-20T21:24:02.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.921+0000 7f43b3fcf740 1 -- 172.21.15.86:0/2598330932 shutdown_connections 2024-04-20T21:24:02.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.921+0000 7f43b3fcf740 1 --2- 172.21.15.86:0/2598330932 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x562b47bf2e70 0x562b47bf3250 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.921+0000 7f43b3fcf740 1 --2- 172.21.15.86:0/2598330932 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562b47c0a840 0x562b47bfcb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.921+0000 7f43b3fcf740 1 --2- 172.21.15.86:0/2598330932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b47bf6290 0x562b47bfc620 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.921+0000 7f438b400640 0 client.7200 ms_handle_reset on v2:172.21.15.86:6835/2254117576 2024-04-20T21:24:02.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.921+0000 7f43b3fcf740 1 --2- 172.21.15.86:0/2598330932 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562b47bf5eb0 0x562b47c09fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.921+0000 7f43b3fcf740 1 -- 172.21.15.86:0/2598330932 >> 172.21.15.86:0/2598330932 conn(0x562b47bc9390 msgr2=0x562b47bc7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:02.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.922+0000 7f43b3fcf740 1 -- 172.21.15.86:0/2598330932 shutdown_connections 2024-04-20T21:24:02.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.922+0000 7f43b3fcf740 1 -- 172.21.15.86:0/2598330932 wait complete. 2024-04-20T21:24:02.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.922+0000 7f43b3fcf740 20 client.7200 trim_cache size 0 max 16384 2024-04-20T21:24:02.928 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:24:02.928 INFO:teuthology.orchestra.run.smithi086.stdout:[ 2024-04-20T21:24:02.928 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:24:02.928 INFO:teuthology.orchestra.run.smithi086.stdout: "damage_type": "dentry", 2024-04-20T21:24:02.928 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 4154396678, 2024-04-20T21:24:02.928 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 1099511627776, 2024-04-20T21:24:02.929 INFO:teuthology.orchestra.run.smithi086.stdout: "frag": "*", 2024-04-20T21:24:02.929 INFO:teuthology.orchestra.run.smithi086.stdout: "dname": "file_to_be_damaged", 2024-04-20T21:24:02.929 INFO:teuthology.orchestra.run.smithi086.stdout: "snap_id": "head", 2024-04-20T21:24:02.929 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/subdir/file_to_be_damaged" 2024-04-20T21:24:02.929 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:24:02.929 INFO:teuthology.orchestra.run.smithi086.stdout:] 2024-04-20T21:24:02.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.922+0000 7f43b1200640 1 -- 172.21.15.86:0/1436967580 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f438c06c920 msgr2=0x7f438c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:02.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.922+0000 7f43b1200640 1 --2- 172.21.15.86:0/1436967580 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f438c06c920 0x7f438c06ede0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f43ac158240 tx=0x7f439c008040 comp rx=0 tx=0).stop 2024-04-20T21:24:02.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.922+0000 7f43b1200640 1 -- 172.21.15.86:0/1436967580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43ac15e8f0 msgr2=0x7f43ac0d4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:02.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.922+0000 7f43b1200640 1 --2- 172.21.15.86:0/1436967580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43ac15e8f0 0x7f43ac0d4890 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f439800c820 tx=0x7f439800cce0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.922+0000 7f43b1200640 1 -- 172.21.15.86:0/1436967580 shutdown_connections 2024-04-20T21:24:02.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.922+0000 7f43b1200640 1 --2- 172.21.15.86:0/1436967580 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f438c06c920 0x7f438c06ede0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.922+0000 7f43b1200640 1 --2- 172.21.15.86:0/1436967580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43ac15ecd0 0x7f43ac0d4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.922+0000 7f43b1200640 1 --2- 172.21.15.86:0/1436967580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43ac15e8f0 0x7f43ac0d4890 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.922+0000 7f43b1200640 1 --2- 172.21.15.86:0/1436967580 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f43ac0cf680 0x7f43ac14ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:02.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.922+0000 7f43b1200640 1 -- 172.21.15.86:0/1436967580 >> 172.21.15.86:0/1436967580 conn(0x7f43ac06cc70 msgr2=0x7f43ac06d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:02.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.923+0000 7f43b1200640 1 -- 172.21.15.86:0/1436967580 shutdown_connections 2024-04-20T21:24:02.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:02.923+0000 7f43b1200640 1 -- 172.21.15.86:0/1436967580 wait complete. 2024-04-20T21:24:02.938 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-20T21:24:03.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.131+0000 7fbde6800640 1 Processor -- start 2024-04-20T21:24:03.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.131+0000 7fbde6800640 1 -- start start 2024-04-20T21:24:03.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.132+0000 7fbde6800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbde00af7b0 0x7fbde015c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.132+0000 7fbde6800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbde006f4c0 0x7fbde006f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.132+0000 7fbde6800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbde00ae710 con 0x7fbde015c010 2024-04-20T21:24:03.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.132+0000 7fbde6800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fbde00af320 con 0x7fbde006f4c0 2024-04-20T21:24:03.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.132+0000 7fbde6800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbde00ae4b0 con 0x7fbde00af7b0 2024-04-20T21:24:03.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.132+0000 7fbde5e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbde006f4c0 0x7fbde006f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:03.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.132+0000 7fbde5e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbde006f4c0 0x7fbde006f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37204/0 (socket says 172.21.15.86:37204) 2024-04-20T21:24:03.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.132+0000 7fbde5e00640 1 -- 172.21.15.86:0/3647067131 learned_addr learned my addr 172.21.15.86:0/3647067131 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:03.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.133+0000 7fbde4a00640 1 --2- 172.21.15.86:0/3647067131 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbde00af7b0 0x7fbde015c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:03.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.133+0000 7fbde5e00640 1 -- 172.21.15.86:0/3647067131 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbde00af7b0 msgr2=0x7fbde015c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:03.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.133+0000 7fbde5e00640 1 --2- 172.21.15.86:0/3647067131 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbde00af7b0 0x7fbde015c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.133+0000 7fbde5e00640 1 -- 172.21.15.86:0/3647067131 >> v1:172.21.15.144:6789/0 conn(0x7fbde015c010 legacy=0x7fbde00adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:03.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.133+0000 7fbde5e00640 1 -- 172.21.15.86:0/3647067131 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbde00ae9f0 con 0x7fbde006f4c0 2024-04-20T21:24:03.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.134+0000 7fbde5e00640 1 --2- 172.21.15.86:0/3647067131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbde006f4c0 0x7fbde006f970 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fbdd800bb10 tx=0x7fbdd802e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ff49ec68a322f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:03.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.134+0000 7fbddfe00640 1 -- 172.21.15.86:0/3647067131 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd803b070 con 0x7fbde006f4c0 2024-04-20T21:24:03.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.135+0000 7fbddfe00640 1 -- 172.21.15.86:0/3647067131 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdd8036440 con 0x7fbde006f4c0 2024-04-20T21:24:03.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.135+0000 7fbddfe00640 1 -- 172.21.15.86:0/3647067131 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd8036810 con 0x7fbde006f4c0 2024-04-20T21:24:03.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.135+0000 7fbde6800640 1 -- 172.21.15.86:0/3647067131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbde006f4c0 msgr2=0x7fbde006f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:03.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.135+0000 7fbde6800640 1 --2- 172.21.15.86:0/3647067131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbde006f4c0 0x7fbde006f970 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fbdd800bb10 tx=0x7fbdd802e0c0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.135+0000 7fbde6800640 1 -- 172.21.15.86:0/3647067131 shutdown_connections 2024-04-20T21:24:03.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.135+0000 7fbde6800640 1 --2- 172.21.15.86:0/3647067131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbde006f4c0 0x7fbde006f970 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.135+0000 7fbde6800640 1 --2- 172.21.15.86:0/3647067131 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbde00af7b0 0x7fbde015c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.135+0000 7fbde6800640 1 -- 172.21.15.86:0/3647067131 >> 172.21.15.86:0/3647067131 conn(0x7fbde00d0170 msgr2=0x7fbde00d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:03.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.135+0000 7fbde6800640 1 -- 172.21.15.86:0/3647067131 shutdown_connections 2024-04-20T21:24:03.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.135+0000 7fbde6800640 1 -- 172.21.15.86:0/3647067131 wait complete. 2024-04-20T21:24:03.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.137+0000 7fbde6800640 1 Processor -- start 2024-04-20T21:24:03.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.137+0000 7fbde6800640 1 -- start start 2024-04-20T21:24:03.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.138+0000 7fbde6800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbde006f4c0 0x7fbde0159880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.138+0000 7fbde6800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbde00af7b0 0x7fbde0159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.138+0000 7fbde6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbde015c010 0x7fbde01f8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.138+0000 7fbde6800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbde00aefb0 con 0x7fbde015c010 2024-04-20T21:24:03.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.138+0000 7fbde6800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbde015c930 con 0x7fbde006f4c0 2024-04-20T21:24:03.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.138+0000 7fbde6800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbde0071b70 con 0x7fbde00af7b0 2024-04-20T21:24:03.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.138+0000 7fbde5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbde015c010 0x7fbde01f8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:03.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.138+0000 7fbde5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbde015c010 0x7fbde01f8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37206/0 (socket says 172.21.15.86:37206) 2024-04-20T21:24:03.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.138+0000 7fbde5e00640 1 -- 172.21.15.86:0/1066194076 learned_addr learned my addr 172.21.15.86:0/1066194076 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:03.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.139+0000 7fbde5400640 1 --2- 172.21.15.86:0/1066194076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbde006f4c0 0x7fbde0159880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:03.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.139+0000 7fbde4a00640 1 --2- 172.21.15.86:0/1066194076 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbde00af7b0 0x7fbde0159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:03.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.139+0000 7fbde5e00640 1 -- 172.21.15.86:0/1066194076 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbde00af7b0 msgr2=0x7fbde0159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:03.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.139+0000 7fbde5e00640 1 --2- 172.21.15.86:0/1066194076 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbde00af7b0 0x7fbde0159dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.139+0000 7fbde5e00640 1 -- 172.21.15.86:0/1066194076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbde006f4c0 msgr2=0x7fbde0159880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:03.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.139+0000 7fbde5e00640 1 --2- 172.21.15.86:0/1066194076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbde006f4c0 0x7fbde0159880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.139+0000 7fbde5e00640 1 -- 172.21.15.86:0/1066194076 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbde0149460 con 0x7fbde015c010 2024-04-20T21:24:03.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.139+0000 7fbde5e00640 1 --2- 172.21.15.86:0/1066194076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbde015c010 0x7fbde01f8040 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fbdd802e100 tx=0x7fbdd80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:03.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.139+0000 7fbddd600640 1 -- 172.21.15.86:0/1066194076 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd8046070 con 0x7fbde015c010 2024-04-20T21:24:03.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.140+0000 7fbddd600640 1 -- 172.21.15.86:0/1066194076 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdd803d050 con 0x7fbde015c010 2024-04-20T21:24:03.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.140+0000 7fbde6800640 1 -- 172.21.15.86:0/1066194076 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbde0149700 con 0x7fbde015c010 2024-04-20T21:24:03.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.140+0000 7fbddd600640 1 -- 172.21.15.86:0/1066194076 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd803b030 con 0x7fbde015c010 2024-04-20T21:24:03.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.140+0000 7fbde6800640 1 -- 172.21.15.86:0/1066194076 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbde0149cc0 con 0x7fbde015c010 2024-04-20T21:24:03.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.141+0000 7fbde6800640 1 -- 172.21.15.86:0/1066194076 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbde004fcf0 con 0x7fbde015c010 2024-04-20T21:24:03.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.142+0000 7fbddd600640 1 -- 172.21.15.86:0/1066194076 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbdd8049070 con 0x7fbde015c010 2024-04-20T21:24:03.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.143+0000 7fbddd600640 1 --2- 172.21.15.86:0/1066194076 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbdc006c8f0 0x7fbdc006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.143+0000 7fbddd600640 1 -- 172.21.15.86:0/1066194076 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5882+0+0 (secure 0 0 0) 0x7fbdd80ad880 con 0x7fbde015c010 2024-04-20T21:24:03.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.143+0000 7fbde5400640 1 --2- 172.21.15.86:0/1066194076 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbdc006c8f0 0x7fbdc006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:03.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.144+0000 7fbde5400640 1 --2- 172.21.15.86:0/1066194076 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbdc006c8f0 0x7fbdc006edb0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0006fd0 tx=0x7fbdd0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:03.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.147+0000 7fbddd600640 1 -- 172.21.15.86:0/1066194076 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbdd807bb60 con 0x7fbde015c010 2024-04-20T21:24:03.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.389+0000 7fbde6800640 1 -- 172.21.15.86:0/1066194076 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fbde015db80 con 0x7fbde015c010 2024-04-20T21:24:03.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.390+0000 7fbddd600640 1 -- 172.21.15.86:0/1066194076 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7fbdd807b460 con 0x7fbde015c010 2024-04-20T21:24:03.389 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:24:03.389 INFO:teuthology.orchestra.run.smithi086.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-04-20T21:24:03.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.393+0000 7fbde6800640 1 -- 172.21.15.86:0/1066194076 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbdc006c8f0 msgr2=0x7fbdc006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:03.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.393+0000 7fbde6800640 1 --2- 172.21.15.86:0/1066194076 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbdc006c8f0 0x7fbdc006edb0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0006fd0 tx=0x7fbdd0008040 comp rx=0 tx=0).stop 2024-04-20T21:24:03.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.393+0000 7fbde6800640 1 -- 172.21.15.86:0/1066194076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbde015c010 msgr2=0x7fbde01f8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:03.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.393+0000 7fbde6800640 1 --2- 172.21.15.86:0/1066194076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbde015c010 0x7fbde01f8040 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fbdd802e100 tx=0x7fbdd80046c0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.393+0000 7fbde6800640 1 -- 172.21.15.86:0/1066194076 shutdown_connections 2024-04-20T21:24:03.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.393+0000 7fbde6800640 1 --2- 172.21.15.86:0/1066194076 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbdc006c8f0 0x7fbdc006edb0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.393+0000 7fbde6800640 1 --2- 172.21.15.86:0/1066194076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbde015c010 0x7fbde01f8040 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.393+0000 7fbde6800640 1 --2- 172.21.15.86:0/1066194076 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbde00af7b0 0x7fbde0159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.393+0000 7fbde6800640 1 --2- 172.21.15.86:0/1066194076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbde006f4c0 0x7fbde0159880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.393+0000 7fbde6800640 1 -- 172.21.15.86:0/1066194076 >> 172.21.15.86:0/1066194076 conn(0x7fbde00d0170 msgr2=0x7fbde00afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:03.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.393+0000 7fbde6800640 1 -- 172.21.15.86:0/1066194076 shutdown_connections 2024-04-20T21:24:03.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.393+0000 7fbde6800640 1 -- 172.21.15.86:0/1066194076 wait complete. 2024-04-20T21:24:03.410 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-20T21:24:03.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:24:03.411 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir repair,recursive 2024-04-20T21:24:03.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.608+0000 7ffbcb200640 1 Processor -- start 2024-04-20T21:24:03.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.609+0000 7ffbcb200640 1 -- start start 2024-04-20T21:24:03.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.610+0000 7ffbcb200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbcc1666f0 0x7ffbcc06e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.610+0000 7ffbcb200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbcc166b70 con 0x7ffbcc0b49f0 2024-04-20T21:24:03.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.610+0000 7ffbcb200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbcc072800 con 0x7ffbcc06e730 2024-04-20T21:24:03.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.610+0000 7ffbcb200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ffbcc1676a0 con 0x7ffbcc1666f0 2024-04-20T21:24:03.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.611+0000 7ffbc9e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7ffbcc0b49f0 0x7ffbcc0b4dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:48246/0 (socket says 172.21.15.86:48246) 2024-04-20T21:24:03.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.611+0000 7ffbc9e00640 1 -- 172.21.15.86:0/3818948700 learned_addr learned my addr 172.21.15.86:0/3818948700 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:24:03.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.611+0000 7ffbc3e00640 1 --2- 172.21.15.86:0/3818948700 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbcc1666f0 0x7ffbcc06e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:03.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.611+0000 7ffbc9400640 1 -- 172.21.15.86:0/3818948700 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1778946408 0 0) 0x7ffbcc166b70 con 0x7ffbcc0b49f0 2024-04-20T21:24:03.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.611+0000 7ffbc9400640 1 -- 172.21.15.86:0/3818948700 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffbb0003520 con 0x7ffbcc0b49f0 2024-04-20T21:24:03.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.611+0000 7ffbc9400640 1 -- 172.21.15.86:0/3818948700 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3617481067 0 0) 0x7ffbb0003520 con 0x7ffbcc0b49f0 2024-04-20T21:24:03.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.612+0000 7ffbc9400640 1 -- 172.21.15.86:0/3818948700 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbcc1666f0 msgr2=0x7ffbcc06e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:03.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.612+0000 7ffbc9400640 1 --2- 172.21.15.86:0/3818948700 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbcc1666f0 0x7ffbcc06e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.612+0000 7ffbc9400640 1 -- 172.21.15.86:0/3818948700 >> v1:172.21.15.144:6789/0 conn(0x7ffbcc06e730 legacy=0x7ffbcc0b3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:03.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.612+0000 7ffbc9400640 1 -- 172.21.15.86:0/3818948700 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbcc166e00 con 0x7ffbcc0b49f0 2024-04-20T21:24:03.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.612+0000 7ffbc9400640 1 -- 172.21.15.86:0/3818948700 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7ffbbc001670 con 0x7ffbcc0b49f0 2024-04-20T21:24:03.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.612+0000 7ffbc9400640 1 -- 172.21.15.86:0/3818948700 >> v1:172.21.15.86:6789/0 conn(0x7ffbcc0b49f0 legacy=0x7ffbcc0b4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:03.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.612+0000 7ffbc9400640 1 --2- 172.21.15.86:0/3818948700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbb00039e0 0x7ffbb0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.612+0000 7ffbc9400640 1 --2- 172.21.15.86:0/3818948700 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbcc1666f0 0x7ffbb0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.612+0000 7ffbc9400640 1 --2- 172.21.15.86:0/3818948700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbb0004d40 0x7ffbb00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.612+0000 7ffbc9400640 1 -- 172.21.15.86:0/3818948700 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffbcc1676a0 con 0x7ffbb00039e0 2024-04-20T21:24:03.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.612+0000 7ffbc9400640 1 -- 172.21.15.86:0/3818948700 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb000bb50 con 0x7ffbb0004d40 2024-04-20T21:24:03.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.612+0000 7ffbc9400640 1 -- 172.21.15.86:0/3818948700 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ffbb000bd50 con 0x7ffbcc1666f0 2024-04-20T21:24:03.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.612+0000 7ffbc9e00640 1 --2- 172.21.15.86:0/3818948700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbb00039e0 0x7ffbb0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:03.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.612+0000 7ffbc9e00640 1 -- 172.21.15.86:0/3818948700 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbcc1666f0 msgr2=0x7ffbb0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:03.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.612+0000 7ffbc9e00640 1 --2- 172.21.15.86:0/3818948700 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbcc1666f0 0x7ffbb0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.612+0000 7ffbc9e00640 1 -- 172.21.15.86:0/3818948700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbb0004d40 msgr2=0x7ffbb00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:03.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.612+0000 7ffbc9e00640 1 --2- 172.21.15.86:0/3818948700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbb0004d40 0x7ffbb00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.612+0000 7ffbc9e00640 1 -- 172.21.15.86:0/3818948700 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb0003520 con 0x7ffbb00039e0 2024-04-20T21:24:03.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.613+0000 7ffbc9e00640 1 --2- 172.21.15.86:0/3818948700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbb00039e0 0x7ffbb0003e00 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc00c110 tx=0x7ffbb0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=943b059a8cf011c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:03.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.614+0000 7ffbc9400640 1 -- 172.21.15.86:0/3818948700 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc007c40 con 0x7ffbb00039e0 2024-04-20T21:24:03.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.614+0000 7ffbc9400640 1 -- 172.21.15.86:0/3818948700 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbbc030070 con 0x7ffbb00039e0 2024-04-20T21:24:03.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.614+0000 7ffbc9400640 1 -- 172.21.15.86:0/3818948700 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc005d30 con 0x7ffbb00039e0 2024-04-20T21:24:03.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.614+0000 7ffbcb200640 1 -- 172.21.15.86:0/3818948700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbb00039e0 msgr2=0x7ffbb0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:03.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.614+0000 7ffbcb200640 1 --2- 172.21.15.86:0/3818948700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbb00039e0 0x7ffbb0003e00 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc00c110 tx=0x7ffbb0005830 comp rx=0 tx=0).stop 2024-04-20T21:24:03.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.614+0000 7ffbca800640 1 -- 172.21.15.86:0/3818948700 reap_dead start 2024-04-20T21:24:03.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.614+0000 7ffbcb200640 1 -- 172.21.15.86:0/3818948700 shutdown_connections 2024-04-20T21:24:03.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.614+0000 7ffbcb200640 1 -- 172.21.15.86:0/3818948700 >> 172.21.15.86:0/3818948700 conn(0x7ffbcc0b2180 msgr2=0x7ffbcc0b2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:03.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.614+0000 7ffbcb200640 1 -- 172.21.15.86:0/3818948700 shutdown_connections 2024-04-20T21:24:03.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.614+0000 7ffbcb200640 1 -- 172.21.15.86:0/3818948700 wait complete. 2024-04-20T21:24:03.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.616+0000 7ffbcb200640 1 Processor -- start 2024-04-20T21:24:03.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.616+0000 7ffbcb200640 1 -- start start 2024-04-20T21:24:03.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.617+0000 7ffbcb200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbb00039e0 0x7ffbcc154740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.617+0000 7ffbcb200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbcc0aecc0 0x7ffbcc164830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.618+0000 7ffbcb200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbcc164d70 0x7ffbcc0ad040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.618+0000 7ffbcb200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffbcc06dcd0 con 0x7ffbb00039e0 2024-04-20T21:24:03.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.618+0000 7ffbcb200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ffbcc1673c0 con 0x7ffbcc164d70 2024-04-20T21:24:03.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.618+0000 7ffbcb200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ffbcc0b3d80 con 0x7ffbcc0aecc0 2024-04-20T21:24:03.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.618+0000 7ffbc9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbb00039e0 0x7ffbcc154740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:03.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.618+0000 7ffbc9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbb00039e0 0x7ffbcc154740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37232/0 (socket says 172.21.15.86:37232) 2024-04-20T21:24:03.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.618+0000 7ffbc9e00640 1 -- 172.21.15.86:0/3383840367 learned_addr learned my addr 172.21.15.86:0/3383840367 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:03.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.618+0000 7ffbc9e00640 1 -- 172.21.15.86:0/3383840367 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbcc0aecc0 msgr2=0x7ffbcc164830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:03.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.618+0000 7ffbc9e00640 1 --2- 172.21.15.86:0/3383840367 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbcc0aecc0 0x7ffbcc164830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.618+0000 7ffbc9e00640 1 -- 172.21.15.86:0/3383840367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbcc164d70 msgr2=0x7ffbcc0ad040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:03.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.618+0000 7ffbc9e00640 1 --2- 172.21.15.86:0/3383840367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbcc164d70 0x7ffbcc0ad040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.618+0000 7ffbc9e00640 1 -- 172.21.15.86:0/3383840367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbcc157880 con 0x7ffbb00039e0 2024-04-20T21:24:03.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.619+0000 7ffbc9e00640 1 --2- 172.21.15.86:0/3383840367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbb00039e0 0x7ffbcc154740 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc00c110 tx=0x7ffbcc073540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:03.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.619+0000 7ffbc2000640 1 -- 172.21.15.86:0/3383840367 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc007c40 con 0x7ffbb00039e0 2024-04-20T21:24:03.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.619+0000 7ffbc2000640 1 -- 172.21.15.86:0/3383840367 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbbc004bf0 con 0x7ffbb00039e0 2024-04-20T21:24:03.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.619+0000 7ffbc2000640 1 -- 172.21.15.86:0/3383840367 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc007070 con 0x7ffbb00039e0 2024-04-20T21:24:03.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.619+0000 7ffbcb200640 1 -- 172.21.15.86:0/3383840367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbcc0adc20 con 0x7ffbb00039e0 2024-04-20T21:24:03.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.619+0000 7ffbcb200640 1 -- 172.21.15.86:0/3383840367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbcc0ae1e0 con 0x7ffbb00039e0 2024-04-20T21:24:03.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.621+0000 7ffbc2000640 1 -- 172.21.15.86:0/3383840367 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffbbc047020 con 0x7ffbb00039e0 2024-04-20T21:24:03.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.623+0000 7ffbc2000640 1 --2- 172.21.15.86:0/3383840367 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ffba806c920 0x7ffba806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.623+0000 7ffbc2000640 1 -- 172.21.15.86:0/3383840367 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5882+0+0 (secure 0 0 0) 0x7ffbbc0acd40 con 0x7ffbb00039e0 2024-04-20T21:24:03.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.623+0000 7ffbc3e00640 1 --2- 172.21.15.86:0/3383840367 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ffba806c920 0x7ffba806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:03.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.624+0000 7ffbd35f8740 1 Processor -- start 2024-04-20T21:24:03.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.624+0000 7ffbd35f8740 1 -- start start 2024-04-20T21:24:03.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.624+0000 7ffbc3e00640 1 --2- 172.21.15.86:0/3383840367 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ffba806c920 0x7ffba806ede0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4006e80 tx=0x7ffbb4006dd0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:03.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.624+0000 7ffbd35f8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55974b07a890 0x55974b04d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.624+0000 7ffbd35f8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55974b07ac70 0x55974b07d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.625+0000 7ffbd35f8740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55974b04dd70 0x55974b07db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.625+0000 7ffbd35f8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55974afb2cf0 con 0x55974b07ac70 2024-04-20T21:24:03.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.625+0000 7ffbd35f8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55974afb7f40 con 0x55974b07a890 2024-04-20T21:24:03.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.625+0000 7ffbd35f8740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55974af5e500 con 0x55974b04dd70 2024-04-20T21:24:03.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.625+0000 7ffbc3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55974b07ac70 0x55974b07d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:03.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.625+0000 7ffbc3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55974b07ac70 0x55974b07d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37238/0 (socket says 172.21.15.86:37238) 2024-04-20T21:24:03.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.625+0000 7ffbc3e00640 1 -- 172.21.15.86:0/2931277279 learned_addr learned my addr 172.21.15.86:0/2931277279 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:03.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.625+0000 7ffbc3e00640 1 -- 172.21.15.86:0/2931277279 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55974b04dd70 msgr2=0x55974b07db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:03.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.625+0000 7ffbc3e00640 1 --2- 172.21.15.86:0/2931277279 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55974b04dd70 0x55974b07db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.625+0000 7ffbc3e00640 1 -- 172.21.15.86:0/2931277279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55974b07a890 msgr2=0x55974b04d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:03.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.625+0000 7ffbc3e00640 1 --2- 172.21.15.86:0/2931277279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55974b07a890 0x55974b04d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.625+0000 7ffbc3e00640 1 -- 172.21.15.86:0/2931277279 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55974ad92a50 con 0x55974b07ac70 2024-04-20T21:24:03.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.626+0000 7ffbc3e00640 1 --2- 172.21.15.86:0/2931277279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55974b07ac70 0x55974b07d620 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7ffbb400ea70 tx=0x7ffbb4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f60d4c33edda718e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:03.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.626+0000 7ffbba600640 1 -- 172.21.15.86:0/2931277279 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4023020 con 0x55974b07ac70 2024-04-20T21:24:03.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.626+0000 7ffbba600640 1 -- 172.21.15.86:0/2931277279 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbb4015a00 con 0x55974b07ac70 2024-04-20T21:24:03.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.626+0000 7ffbba600640 1 -- 172.21.15.86:0/2931277279 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb401b040 con 0x55974b07ac70 2024-04-20T21:24:03.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.626+0000 7ffbd35f8740 1 -- 172.21.15.86:0/2931277279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55974b07ac70 msgr2=0x55974b07d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:03.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.626+0000 7ffbd35f8740 1 --2- 172.21.15.86:0/2931277279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55974b07ac70 0x55974b07d620 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7ffbb400ea70 tx=0x7ffbb4011000 comp rx=0 tx=0).stop 2024-04-20T21:24:03.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.627+0000 7ffbd35f8740 1 -- 172.21.15.86:0/2931277279 shutdown_connections 2024-04-20T21:24:03.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.627+0000 7ffbd35f8740 1 --2- 172.21.15.86:0/2931277279 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55974b04dd70 0x55974b07db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.627+0000 7ffbd35f8740 1 --2- 172.21.15.86:0/2931277279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55974b07ac70 0x55974b07d620 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.627+0000 7ffbd35f8740 1 --2- 172.21.15.86:0/2931277279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55974b07a890 0x55974b04d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.627+0000 7ffbd35f8740 1 -- 172.21.15.86:0/2931277279 >> 172.21.15.86:0/2931277279 conn(0x55974b04faa0 msgr2=0x55974b098620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:03.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.627+0000 7ffbd35f8740 1 -- 172.21.15.86:0/2931277279 shutdown_connections 2024-04-20T21:24:03.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.627+0000 7ffbd35f8740 1 -- 172.21.15.86:0/2931277279 wait complete. 2024-04-20T21:24:03.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.627+0000 7ffbd35f8740 1 -- start start 2024-04-20T21:24:03.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.628+0000 7ffbd35f8740 1 Processor -- start 2024-04-20T21:24:03.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.628+0000 7ffbd35f8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55974b07a890 0x55974b079ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.628+0000 7ffbd35f8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55974b07ac70 0x55974b08fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.628+0000 7ffbd35f8740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55974b098b20 0x55974b08ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.628+0000 7ffbd35f8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55974af718c0 con 0x55974b07a890 2024-04-20T21:24:03.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.628+0000 7ffbd35f8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55974afb3f00 con 0x55974b07ac70 2024-04-20T21:24:03.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.628+0000 7ffbd35f8740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55974b052140 con 0x55974b098b20 2024-04-20T21:24:03.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.629+0000 7ffbd35f8740 10 client.0 _ll_register_callbacks cb 0x55974af73ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:24:03.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.629+0000 7ffbc9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55974b07a890 0x55974b079ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:03.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.629+0000 7ffbc9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55974b07a890 0x55974b079ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37254/0 (socket says 172.21.15.86:37254) 2024-04-20T21:24:03.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.629+0000 7ffbc9e00640 1 -- 172.21.15.86:0/565434165 learned_addr learned my addr 172.21.15.86:0/565434165 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:03.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.629+0000 7ffbc9e00640 1 -- 172.21.15.86:0/565434165 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55974b098b20 msgr2=0x55974b08ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:03.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.629+0000 7ffbc9e00640 1 --2- 172.21.15.86:0/565434165 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55974b098b20 0x55974b08ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.629+0000 7ffbc9e00640 1 -- 172.21.15.86:0/565434165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55974b07ac70 msgr2=0x55974b08fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:03.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.629+0000 7ffbc9e00640 1 --2- 172.21.15.86:0/565434165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55974b07ac70 0x55974b08fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.629+0000 7ffbc9e00640 1 -- 172.21.15.86:0/565434165 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55974ac8a3e0 con 0x55974b07a890 2024-04-20T21:24:03.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.629+0000 7ffbc9e00640 1 --2- 172.21.15.86:0/565434165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55974b07a890 0x55974b079ca0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc076eb0 tx=0x7ffbbc07a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af25c0e372a0ed1c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:03.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.629+0000 7ffbaea00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:24:03.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.629+0000 7ffbaea00640 1 -- 172.21.15.86:0/565434165 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc07ee00 con 0x55974b07a890 2024-04-20T21:24:03.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.630+0000 7ffbaea00640 1 -- 172.21.15.86:0/565434165 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(74..74 src has 1..74) ==== 5882+0+0 (secure 0 0 0) 0x7ffbbc07acb0 con 0x55974b07a890 2024-04-20T21:24:03.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.630+0000 7ffbd35f8740 1 -- 172.21.15.86:0/565434165 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbb4015a00 con 0x55974b07a890 2024-04-20T21:24:03.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.630+0000 7ffbaea00640 1 -- 172.21.15.86:0/565434165 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7ffbbc0a9050 con 0x55974b07a890 2024-04-20T21:24:03.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.630+0000 7ffbd35f8740 10 client.7218 fetch_fsmap learned FSMap version 49 2024-04-20T21:24:03.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.630+0000 7ffbd35f8740 1 -- 172.21.15.86:0/565434165 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) -- 0x55974adf31e0 con 0x55974b07a890 2024-04-20T21:24:03.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.630+0000 7ffbaea00640 1 -- 172.21.15.86:0/565434165 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbb4015a00 con 0x55974b07a890 2024-04-20T21:24:03.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.630+0000 7ffbaea00640 1 -- 172.21.15.86:0/565434165 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 49) ==== 3495+0+0 (secure 0 0 0) 0x7ffbbc0831a0 con 0x55974b07a890 2024-04-20T21:24:03.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.630+0000 7ffbd35f8740 10 client.7218 fetch_fsmap finished waiting for FSMap version 49 2024-04-20T21:24:03.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.630+0000 7ffbd35f8740 10 client.7218 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-20T21:24:03.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.630+0000 7ffbd35f8740 20 client.7218 populate_metadata read hostname 'smithi086' 2024-04-20T21:24:03.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.630+0000 7ffbd35f8740 1 --2- 172.21.15.86:0/565434165 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x55974b07fca0 0x55974b0a87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.630+0000 7ffbd35f8740 4 client.7218 mds_command: new command op to 6987 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:24:03.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.630+0000 7ffbd35f8740 1 -- 172.21.15.86:0/565434165 --> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55974afc5960 con 0x55974b07fca0 2024-04-20T21:24:03.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.631+0000 7ffbc3e00640 1 --2- 172.21.15.86:0/565434165 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x55974b07fca0 0x55974b0a87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:03.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.631+0000 7ffbaea00640 10 client.7218 ms_handle_connect on v2:172.21.15.86:6835/2254117576 2024-04-20T21:24:03.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.631+0000 7ffbc3e00640 1 --2- 172.21.15.86:0/565434165 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x55974b07fca0 0x55974b0a87b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4781cb2d967e5399 server_cookie=2c6a3463af82c01e in_seq=0 out_seq=0 2024-04-20T21:24:03.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.632+0000 7ffbaea00640 1 -- 172.21.15.86:0/565434165 <== mds.0 v2:172.21.15.86:6835/2254117576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55974afc5960 con 0x55974b07fca0 2024-04-20T21:24:03.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.632+0000 7ffbaea00640 10 client.7218 handle_command_reply: tid=0 2024-04-20T21:24:03.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.632+0000 7ffbd35f8740 1 client.7218 shutdown 2024-04-20T21:24:03.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.632+0000 7ffbd35f8740 1 -- 172.21.15.86:0/565434165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55974b07a890 msgr2=0x55974b079ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:03.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.632+0000 7ffbd35f8740 1 --2- 172.21.15.86:0/565434165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55974b07a890 0x55974b079ca0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc076eb0 tx=0x7ffbbc07a610 comp rx=0 tx=0).stop 2024-04-20T21:24:03.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.632+0000 7ffbd35f8740 1 -- 172.21.15.86:0/565434165 shutdown_connections 2024-04-20T21:24:03.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.632+0000 7ffbd35f8740 1 --2- 172.21.15.86:0/565434165 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x55974b07fca0 0x55974b0a87b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.632+0000 7ffbd35f8740 1 --2- 172.21.15.86:0/565434165 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55974b098b20 0x55974b08ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.632+0000 7ffbd35f8740 1 --2- 172.21.15.86:0/565434165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55974b07ac70 0x55974b08fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.632+0000 7ffbd35f8740 1 --2- 172.21.15.86:0/565434165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55974b07a890 0x55974b079ca0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.632+0000 7ffbd35f8740 1 -- 172.21.15.86:0/565434165 >> 172.21.15.86:0/565434165 conn(0x55974b04dd70 msgr2=0x55974b051840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:03.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.632+0000 7ffbaea00640 0 client.7218 ms_handle_reset on v2:172.21.15.86:6835/2254117576 2024-04-20T21:24:03.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.632+0000 7ffbd35f8740 1 -- 172.21.15.86:0/565434165 shutdown_connections 2024-04-20T21:24:03.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.632+0000 7ffbd35f8740 1 -- 172.21.15.86:0/565434165 wait complete. 2024-04-20T21:24:03.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.632+0000 7ffbd35f8740 20 client.7218 trim_cache size 0 max 16384 2024-04-20T21:24:03.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.651+0000 7ffbd35f8740 1 Processor -- start 2024-04-20T21:24:03.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.651+0000 7ffbd35f8740 1 -- start start 2024-04-20T21:24:03.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.652+0000 7ffbd35f8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55974b07a890 0x55974b0ac300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.652+0000 7ffbd35f8740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55974b07ac70 0x55974b09bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.652+0000 7ffbd35f8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55974b098b20 0x55974b09c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.652+0000 7ffbd35f8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffbcc205a90 con 0x55974b098b20 2024-04-20T21:24:03.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.652+0000 7ffbd35f8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ffbcc204b00 con 0x55974b07a890 2024-04-20T21:24:03.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.652+0000 7ffbd35f8740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ffbcc203e90 con 0x55974b07ac70 2024-04-20T21:24:03.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.652+0000 7ffbca800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55974b098b20 0x55974b09c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:03.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.652+0000 7ffbca800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55974b098b20 0x55974b09c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37268/0 (socket says 172.21.15.86:37268) 2024-04-20T21:24:03.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.652+0000 7ffbca800640 1 -- 172.21.15.86:0/3965178390 learned_addr learned my addr 172.21.15.86:0/3965178390 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:03.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.653+0000 7ffbca800640 1 -- 172.21.15.86:0/3965178390 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55974b07ac70 msgr2=0x55974b09bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:03.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.653+0000 7ffbc3e00640 1 --2- 172.21.15.86:0/3965178390 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55974b07ac70 0x55974b09bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:03.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.653+0000 7ffbca800640 1 --2- 172.21.15.86:0/3965178390 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55974b07ac70 0x55974b09bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.653+0000 7ffbca800640 1 -- 172.21.15.86:0/3965178390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55974b07a890 msgr2=0x55974b0ac300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:03.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.653+0000 7ffbca800640 1 --2- 172.21.15.86:0/3965178390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55974b07a890 0x55974b0ac300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.653+0000 7ffbca800640 1 -- 172.21.15.86:0/3965178390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55974ac634a0 con 0x55974b098b20 2024-04-20T21:24:03.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.653+0000 7ffbca800640 1 --2- 172.21.15.86:0/3965178390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55974b098b20 0x55974b09c110 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc075bd0 tx=0x7ffbc4001f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2a279e04757e0d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:03.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.653+0000 7ffbb9c00640 1 -- 172.21.15.86:0/3965178390 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc4017070 con 0x55974b098b20 2024-04-20T21:24:03.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.653+0000 7ffbb9c00640 1 -- 172.21.15.86:0/3965178390 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbc40070d0 con 0x55974b098b20 2024-04-20T21:24:03.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.653+0000 7ffbb9c00640 1 -- 172.21.15.86:0/3965178390 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc40074c0 con 0x55974b098b20 2024-04-20T21:24:03.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.654+0000 7ffbd35f8740 1 -- 172.21.15.86:0/3965178390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55974b098b20 msgr2=0x55974b09c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:03.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.654+0000 7ffbd35f8740 1 --2- 172.21.15.86:0/3965178390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55974b098b20 0x55974b09c110 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc075bd0 tx=0x7ffbc4001f90 comp rx=0 tx=0).stop 2024-04-20T21:24:03.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.654+0000 7ffbd35f8740 1 -- 172.21.15.86:0/3965178390 shutdown_connections 2024-04-20T21:24:03.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.654+0000 7ffbd35f8740 1 --2- 172.21.15.86:0/3965178390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55974b098b20 0x55974b09c110 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.654+0000 7ffbd35f8740 1 --2- 172.21.15.86:0/3965178390 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55974b07ac70 0x55974b09bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.654+0000 7ffbd35f8740 1 --2- 172.21.15.86:0/3965178390 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55974b07a890 0x55974b0ac300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.654+0000 7ffbd35f8740 1 -- 172.21.15.86:0/3965178390 >> 172.21.15.86:0/3965178390 conn(0x55974b04dd70 msgr2=0x55974b08fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:03.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.654+0000 7ffbd35f8740 1 -- 172.21.15.86:0/3965178390 shutdown_connections 2024-04-20T21:24:03.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.654+0000 7ffbd35f8740 1 -- 172.21.15.86:0/3965178390 wait complete. 2024-04-20T21:24:03.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.655+0000 7ffbd35f8740 1 -- start start 2024-04-20T21:24:03.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.655+0000 7ffbd35f8740 1 Processor -- start 2024-04-20T21:24:03.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.656+0000 7ffbd35f8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55974b07a890 0x55974b109730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.656+0000 7ffbd35f8740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55974b07ac70 0x55974b0532a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.656+0000 7ffbd35f8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55974b098b20 0x55974b0b2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.656+0000 7ffbd35f8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55974b097460 con 0x55974b07a890 2024-04-20T21:24:03.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.656+0000 7ffbd35f8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ffbcc2044b0 con 0x55974b098b20 2024-04-20T21:24:03.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.656+0000 7ffbd35f8740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55974b09add0 con 0x55974b07ac70 2024-04-20T21:24:03.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.656+0000 7ffbd35f8740 10 client.0 _ll_register_callbacks cb 0x7ffbbc007de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:24:03.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.656+0000 7ffbc9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55974b07a890 0x55974b109730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:03.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.656+0000 7ffbc9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55974b07a890 0x55974b109730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37278/0 (socket says 172.21.15.86:37278) 2024-04-20T21:24:03.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.656+0000 7ffbc9e00640 1 -- 172.21.15.86:0/3489821083 learned_addr learned my addr 172.21.15.86:0/3489821083 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:03.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.656+0000 7ffbc9e00640 1 -- 172.21.15.86:0/3489821083 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55974b07ac70 msgr2=0x55974b0532a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:03.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.656+0000 7ffbc9e00640 1 --2- 172.21.15.86:0/3489821083 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55974b07ac70 0x55974b0532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.656+0000 7ffbc9e00640 1 -- 172.21.15.86:0/3489821083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55974b098b20 msgr2=0x55974b0b2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:03.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.656+0000 7ffbc9e00640 1 --2- 172.21.15.86:0/3489821083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55974b098b20 0x55974b0b2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.656+0000 7ffbc9e00640 1 -- 172.21.15.86:0/3489821083 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55974b0b2890 con 0x55974b07a890 2024-04-20T21:24:03.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.657+0000 7ffbc9e00640 1 --2- 172.21.15.86:0/3489821083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55974b07a890 0x55974b109730 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc04c030 tx=0x7ffbbc03ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e639cdfda1220211 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:03.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.657+0000 7ffbaea00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:24:03.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.657+0000 7ffbaea00640 1 -- 172.21.15.86:0/3489821083 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc076cf0 con 0x55974b07a890 2024-04-20T21:24:03.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.657+0000 7ffbaea00640 1 -- 172.21.15.86:0/3489821083 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(74..74 src has 1..74) ==== 5882+0+0 (secure 0 0 0) 0x7ffbbc0812d0 con 0x55974b07a890 2024-04-20T21:24:03.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.657+0000 7ffbd35f8740 1 -- 172.21.15.86:0/3489821083 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbc40070d0 con 0x55974b07a890 2024-04-20T21:24:03.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.657+0000 7ffbaea00640 1 -- 172.21.15.86:0/3489821083 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7ffbbc07a590 con 0x55974b07a890 2024-04-20T21:24:03.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.657+0000 7ffbd35f8740 10 client.7224 fetch_fsmap learned FSMap version 49 2024-04-20T21:24:03.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.657+0000 7ffbd35f8740 1 -- 172.21.15.86:0/3489821083 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) -- 0x55974b04d2d0 con 0x55974b07a890 2024-04-20T21:24:03.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.657+0000 7ffbaea00640 1 -- 172.21.15.86:0/3489821083 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbc40070d0 con 0x55974b07a890 2024-04-20T21:24:03.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.657+0000 7ffbaea00640 1 -- 172.21.15.86:0/3489821083 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 49) ==== 3495+0+0 (secure 0 0 0) 0x7ffbbc0a4bb0 con 0x55974b07a890 2024-04-20T21:24:03.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.657+0000 7ffbd35f8740 10 client.7224 fetch_fsmap finished waiting for FSMap version 49 2024-04-20T21:24:03.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.657+0000 7ffbd35f8740 10 client.7224 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-20T21:24:03.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.657+0000 7ffbd35f8740 20 client.7224 populate_metadata read hostname 'smithi086' 2024-04-20T21:24:03.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.657+0000 7ffbd35f8740 1 --2- 172.21.15.86:0/3489821083 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x55974b04c870 0x55974b04ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.657+0000 7ffbd35f8740 4 client.7224 mds_command: new command op to 6987 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}] 2024-04-20T21:24:03.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.657+0000 7ffbd35f8740 1 -- 172.21.15.86:0/3489821083 --> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}) -- 0x55974afbf7d0 con 0x55974b04c870 2024-04-20T21:24:03.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.658+0000 7ffbc3e00640 1 --2- 172.21.15.86:0/3489821083 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x55974b04c870 0x55974b04ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:03.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.658+0000 7ffbc3e00640 1 --2- 172.21.15.86:0/3489821083 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x55974b04c870 0x55974b04ccb0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=568599c4e04fa14b server_cookie=ca489ead2233c4fd in_seq=0 out_seq=0 2024-04-20T21:24:03.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.658+0000 7ffbaea00640 10 client.7224 ms_handle_connect on v2:172.21.15.86:6835/2254117576 2024-04-20T21:24:03.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.659+0000 7ffbaea00640 1 -- 172.21.15.86:0/3489821083 <== mds.0 v2:172.21.15.86:6835/2254117576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55974afbf7d0 con 0x55974b04c870 2024-04-20T21:24:03.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.659+0000 7ffbaea00640 10 client.7224 handle_command_reply: tid=0 2024-04-20T21:24:03.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.659+0000 7ffbd35f8740 1 client.7224 shutdown 2024-04-20T21:24:03.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.659+0000 7ffbd35f8740 1 -- 172.21.15.86:0/3489821083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55974b07a890 msgr2=0x55974b109730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:03.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.659+0000 7ffbd35f8740 1 --2- 172.21.15.86:0/3489821083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55974b07a890 0x55974b109730 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc04c030 tx=0x7ffbbc03ef90 comp rx=0 tx=0).stop 2024-04-20T21:24:03.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.659+0000 7ffbd35f8740 1 -- 172.21.15.86:0/3489821083 shutdown_connections 2024-04-20T21:24:03.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.659+0000 7ffbd35f8740 1 --2- 172.21.15.86:0/3489821083 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x55974b04c870 0x55974b04ccb0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.659+0000 7ffbd35f8740 1 --2- 172.21.15.86:0/3489821083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55974b098b20 0x55974b0b2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.659+0000 7ffbd35f8740 1 --2- 172.21.15.86:0/3489821083 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55974b07ac70 0x55974b0532a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.659+0000 7ffbd35f8740 1 --2- 172.21.15.86:0/3489821083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55974b07a890 0x55974b109730 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.659+0000 7ffbaea00640 0 client.7224 ms_handle_reset on v2:172.21.15.86:6835/2254117576 2024-04-20T21:24:03.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.659+0000 7ffbd35f8740 1 -- 172.21.15.86:0/3489821083 >> 172.21.15.86:0/3489821083 conn(0x55974b04dd70 msgr2=0x55974b0aadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:03.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.659+0000 7ffbd35f8740 1 -- 172.21.15.86:0/3489821083 shutdown_connections 2024-04-20T21:24:03.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.659+0000 7ffbd35f8740 1 -- 172.21.15.86:0/3489821083 wait complete. 2024-04-20T21:24:03.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.659+0000 7ffbd35f8740 20 client.7224 trim_cache size 0 max 16384 2024-04-20T21:24:03.662 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:24:03.662 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-20T21:24:03.662 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "855a57fc-ebe5-457a-ab9f-b8574f87c5d6", 2024-04-20T21:24:03.663 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-20T21:24:03.663 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:24:03.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.660+0000 7ffbcb200640 1 -- 172.21.15.86:0/3383840367 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ffba806c920 msgr2=0x7ffba806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:03.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.660+0000 7ffbcb200640 1 --2- 172.21.15.86:0/3383840367 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ffba806c920 0x7ffba806ede0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4006e80 tx=0x7ffbb4006dd0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.660+0000 7ffbcb200640 1 -- 172.21.15.86:0/3383840367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbb00039e0 msgr2=0x7ffbcc154740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:03.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.660+0000 7ffbcb200640 1 --2- 172.21.15.86:0/3383840367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbb00039e0 0x7ffbcc154740 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc00c110 tx=0x7ffbcc073540 comp rx=0 tx=0).stop 2024-04-20T21:24:03.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.660+0000 7ffbcb200640 1 -- 172.21.15.86:0/3383840367 shutdown_connections 2024-04-20T21:24:03.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.660+0000 7ffbcb200640 1 --2- 172.21.15.86:0/3383840367 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ffba806c920 0x7ffba806ede0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.660+0000 7ffbcb200640 1 --2- 172.21.15.86:0/3383840367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffbcc164d70 0x7ffbcc0ad040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.660+0000 7ffbcb200640 1 --2- 172.21.15.86:0/3383840367 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffbcc0aecc0 0x7ffbcc164830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.660+0000 7ffbcb200640 1 --2- 172.21.15.86:0/3383840367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbb00039e0 0x7ffbcc154740 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.660+0000 7ffbcb200640 1 -- 172.21.15.86:0/3383840367 >> 172.21.15.86:0/3383840367 conn(0x7ffbcc0b2180 msgr2=0x7ffbcc0d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:03.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.660+0000 7ffbcb200640 1 -- 172.21.15.86:0/3383840367 shutdown_connections 2024-04-20T21:24:03.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.660+0000 7ffbcb200640 1 -- 172.21.15.86:0/3383840367 wait complete. 2024-04-20T21:24:03.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:24:03.676 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-20T21:24:03.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.876+0000 7f1e14200640 1 Processor -- start 2024-04-20T21:24:03.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.876+0000 7f1e14200640 1 -- start start 2024-04-20T21:24:03.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.878+0000 7f1e14200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1e0c15eb30 0x7f1e0c0b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.878+0000 7f1e14200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1e0c0aee50 0x7f1e0c0b4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.878+0000 7f1e14200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e0c06b980 0x7f1e0c06be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.878+0000 7f1e14200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f1e0c0af7d0 con 0x7f1e0c0aee50 2024-04-20T21:24:03.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.878+0000 7f1e14200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f1e0c06c370 con 0x7f1e0c15eb30 2024-04-20T21:24:03.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.878+0000 7f1e14200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1e0c0adb10 con 0x7f1e0c06b980 2024-04-20T21:24:03.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.878+0000 7f1e10e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1e0c0aee50 0x7f1e0c0b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:03.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.878+0000 7f1e10e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1e0c0aee50 0x7f1e0c0b4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37284/0 (socket says 172.21.15.86:37284) 2024-04-20T21:24:03.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.878+0000 7f1e10e00640 1 -- 172.21.15.86:0/3695681698 learned_addr learned my addr 172.21.15.86:0/3695681698 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:03.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.879+0000 7f1e12200640 1 --2- 172.21.15.86:0/3695681698 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e0c06b980 0x7f1e0c06be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:03.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.879+0000 7f1e10e00640 1 -- 172.21.15.86:0/3695681698 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e0c06b980 msgr2=0x7f1e0c06be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:03.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.879+0000 7f1e10e00640 1 --2- 172.21.15.86:0/3695681698 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e0c06b980 0x7f1e0c06be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.879+0000 7f1e11800640 1 --2- 172.21.15.86:0/3695681698 >> v2:172.21.15.144:3300/0 conn(0x7f1e0c15eb30 0x7f1e0c0b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:03.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.879+0000 7f1e10e00640 1 -- 172.21.15.86:0/3695681698 >> v2:172.21.15.144:3300/0 conn(0x7f1e0c15eb30 msgr2=0x7f1e0c0b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:03.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.879+0000 7f1e10e00640 1 --2- 172.21.15.86:0/3695681698 >> v2:172.21.15.144:3300/0 conn(0x7f1e0c15eb30 0x7f1e0c0b3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.879+0000 7f1e10e00640 1 -- 172.21.15.86:0/3695681698 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e0c0addb0 con 0x7f1e0c0aee50 2024-04-20T21:24:03.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.879+0000 7f1e12200640 1 --2- 172.21.15.86:0/3695681698 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e0c06b980 0x7f1e0c06be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:24:03.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.880+0000 7f1e10e00640 1 --2- 172.21.15.86:0/3695681698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e0c0aee50 0x7f1e0c0b4360 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c070130 tx=0x7f1df802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43435656e6a6428 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:03.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.880+0000 7f1e0be00640 1 -- 172.21.15.86:0/3695681698 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df8035020 con 0x7f1e0c0aee50 2024-04-20T21:24:03.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.880+0000 7f1e0be00640 1 -- 172.21.15.86:0/3695681698 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1df8031870 con 0x7f1e0c0aee50 2024-04-20T21:24:03.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.880+0000 7f1e0be00640 1 -- 172.21.15.86:0/3695681698 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df8031c40 con 0x7f1e0c0aee50 2024-04-20T21:24:03.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.881+0000 7f1e14200640 1 -- 172.21.15.86:0/3695681698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e0c0aee50 msgr2=0x7f1e0c0b4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:03.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.881+0000 7f1e14200640 1 --2- 172.21.15.86:0/3695681698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e0c0aee50 0x7f1e0c0b4360 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c070130 tx=0x7f1df802b910 comp rx=0 tx=0).stop 2024-04-20T21:24:03.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.881+0000 7f1e14200640 1 -- 172.21.15.86:0/3695681698 shutdown_connections 2024-04-20T21:24:03.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.881+0000 7f1e14200640 1 --2- 172.21.15.86:0/3695681698 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e0c06b980 0x7f1e0c06be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.881+0000 7f1e14200640 1 --2- 172.21.15.86:0/3695681698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e0c0aee50 0x7f1e0c0b4360 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.881+0000 7f1e14200640 1 --2- 172.21.15.86:0/3695681698 >> v2:172.21.15.144:3300/0 conn(0x7f1e0c15eb30 0x7f1e0c0b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.881+0000 7f1e14200640 1 -- 172.21.15.86:0/3695681698 >> 172.21.15.86:0/3695681698 conn(0x7f1e0c0b2110 msgr2=0x7f1e0c0b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:03.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.881+0000 7f1e14200640 1 -- 172.21.15.86:0/3695681698 shutdown_connections 2024-04-20T21:24:03.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.881+0000 7f1e14200640 1 -- 172.21.15.86:0/3695681698 wait complete. 2024-04-20T21:24:03.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.883+0000 7f1e14200640 1 Processor -- start 2024-04-20T21:24:03.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.883+0000 7f1e14200640 1 -- start start 2024-04-20T21:24:03.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.884+0000 7f1e14200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e0c06b980 0x7f1e0c149db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.884+0000 7f1e14200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e0c0aee50 0x7f1e0c0cf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.884+0000 7f1e14200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e0c15eb30 0x7f1e0c159c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.884+0000 7f1e14200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0c0ae650 con 0x7f1e0c06b980 2024-04-20T21:24:03.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.884+0000 7f1e14200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0c0ae370 con 0x7f1e0c15eb30 2024-04-20T21:24:03.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.884+0000 7f1e14200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1e0c0ae930 con 0x7f1e0c0aee50 2024-04-20T21:24:03.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.884+0000 7f1e11800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e0c06b980 0x7f1e0c149db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:03.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.884+0000 7f1e11800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e0c06b980 0x7f1e0c149db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37286/0 (socket says 172.21.15.86:37286) 2024-04-20T21:24:03.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.884+0000 7f1e11800640 1 -- 172.21.15.86:0/1795122257 learned_addr learned my addr 172.21.15.86:0/1795122257 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:03.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.884+0000 7f1e11800640 1 -- 172.21.15.86:0/1795122257 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e0c0aee50 msgr2=0x7f1e0c0cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:03.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.884+0000 7f1e11800640 1 --2- 172.21.15.86:0/1795122257 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e0c0aee50 0x7f1e0c0cf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.884+0000 7f1e11800640 1 -- 172.21.15.86:0/1795122257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e0c15eb30 msgr2=0x7f1e0c159c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:03.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.884+0000 7f1e11800640 1 --2- 172.21.15.86:0/1795122257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e0c15eb30 0x7f1e0c159c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.884+0000 7f1e11800640 1 -- 172.21.15.86:0/1795122257 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e0c0cfb80 con 0x7f1e0c06b980 2024-04-20T21:24:03.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.885+0000 7f1e11800640 1 --2- 172.21.15.86:0/1795122257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e0c06b980 0x7f1e0c149db0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f1e0000c820 tx=0x7f1e0000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:03.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.885+0000 7f1e09600640 1 -- 172.21.15.86:0/1795122257 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e00015070 con 0x7f1e0c06b980 2024-04-20T21:24:03.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.885+0000 7f1e09600640 1 -- 172.21.15.86:0/1795122257 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e00004240 con 0x7f1e0c06b980 2024-04-20T21:24:03.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.885+0000 7f1e14200640 1 -- 172.21.15.86:0/1795122257 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e0c14b7b0 con 0x7f1e0c06b980 2024-04-20T21:24:03.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.885+0000 7f1e09600640 1 -- 172.21.15.86:0/1795122257 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0000ce10 con 0x7f1e0c06b980 2024-04-20T21:24:03.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.885+0000 7f1e14200640 1 -- 172.21.15.86:0/1795122257 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e0c14bd70 con 0x7f1e0c06b980 2024-04-20T21:24:03.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.887+0000 7f1e09600640 1 -- 172.21.15.86:0/1795122257 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1e00007830 con 0x7f1e0c06b980 2024-04-20T21:24:03.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.889+0000 7f1e09600640 1 --2- 172.21.15.86:0/1795122257 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1dec06c8f0 0x7f1dec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.889+0000 7f1e09600640 1 -- 172.21.15.86:0/1795122257 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5882+0+0 (secure 0 0 0) 0x7f1e0008bc50 con 0x7f1e0c06b980 2024-04-20T21:24:03.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.889+0000 7f1e10e00640 1 --2- 172.21.15.86:0/1795122257 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1dec06c8f0 0x7f1dec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:03.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.890+0000 7f1e16e84740 1 Processor -- start 2024-04-20T21:24:03.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.890+0000 7f1e16e84740 1 -- start start 2024-04-20T21:24:03.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.890+0000 7f1e10e00640 1 --2- 172.21.15.86:0/1795122257 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1dec06c8f0 0x7f1dec06edb0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1df8036000 tx=0x7f1df8030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:03.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.890+0000 7f1e16e84740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fef19961f0 0x55fef19965d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.891+0000 7f1e16e84740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef1966210 0x55fef1993460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.891+0000 7f1e16e84740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fef1969d70 0x55fef19939a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.891+0000 7f1e16e84740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fef18d2150 con 0x55fef1966210 2024-04-20T21:24:03.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.891+0000 7f1e16e84740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fef1878370 con 0x55fef1969d70 2024-04-20T21:24:03.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.891+0000 7f1e16e84740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55fef18ce110 con 0x55fef19961f0 2024-04-20T21:24:03.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.891+0000 7f1e10e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef1966210 0x55fef1993460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:03.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.891+0000 7f1e10e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef1966210 0x55fef1993460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37292/0 (socket says 172.21.15.86:37292) 2024-04-20T21:24:03.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.891+0000 7f1e10e00640 1 -- 172.21.15.86:0/1794053105 learned_addr learned my addr 172.21.15.86:0/1794053105 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:03.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.891+0000 7f1e11800640 1 --2- 172.21.15.86:0/1794053105 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fef19961f0 0x55fef19965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:03.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.891+0000 7f1e10e00640 1 -- 172.21.15.86:0/1794053105 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fef19961f0 msgr2=0x55fef19965d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:03.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.891+0000 7f1e10e00640 1 --2- 172.21.15.86:0/1794053105 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fef19961f0 0x55fef19965d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.891+0000 7f1e10e00640 1 -- 172.21.15.86:0/1794053105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fef1969d70 msgr2=0x55fef19939a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:03.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.891+0000 7f1e10e00640 1 --2- 172.21.15.86:0/1794053105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fef1969d70 0x55fef19939a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.891+0000 7f1e10e00640 1 -- 172.21.15.86:0/1794053105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fef16ae7d0 con 0x55fef1966210 2024-04-20T21:24:03.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.891+0000 7f1e11800640 1 --2- 172.21.15.86:0/1794053105 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fef19961f0 0x55fef19965d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:24:03.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.892+0000 7f1e10e00640 1 --2- 172.21.15.86:0/1794053105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef1966210 0x55fef1993460 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f1df8009930 tx=0x7f1df8031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=998ecb977577c959 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:03.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.892+0000 7f1dfe000640 1 -- 172.21.15.86:0/1794053105 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df804a070 con 0x55fef1966210 2024-04-20T21:24:03.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.892+0000 7f1dfe000640 1 -- 172.21.15.86:0/1794053105 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1df803ad20 con 0x55fef1966210 2024-04-20T21:24:03.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.892+0000 7f1dfe000640 1 -- 172.21.15.86:0/1794053105 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df8040070 con 0x55fef1966210 2024-04-20T21:24:03.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.892+0000 7f1e16e84740 1 -- 172.21.15.86:0/1794053105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef1966210 msgr2=0x55fef1993460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:03.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.892+0000 7f1e16e84740 1 --2- 172.21.15.86:0/1794053105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef1966210 0x55fef1993460 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f1df8009930 tx=0x7f1df8031000 comp rx=0 tx=0).stop 2024-04-20T21:24:03.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.893+0000 7f1e16e84740 1 -- 172.21.15.86:0/1794053105 shutdown_connections 2024-04-20T21:24:03.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.893+0000 7f1e16e84740 1 --2- 172.21.15.86:0/1794053105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fef1969d70 0x55fef19939a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.893+0000 7f1e16e84740 1 --2- 172.21.15.86:0/1794053105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef1966210 0x55fef1993460 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.893+0000 7f1e16e84740 1 --2- 172.21.15.86:0/1794053105 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fef19961f0 0x55fef19965d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.893+0000 7f1e16e84740 1 -- 172.21.15.86:0/1794053105 >> 172.21.15.86:0/1794053105 conn(0x55fef1968480 msgr2=0x55fef19b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:03.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.893+0000 7f1e16e84740 1 -- 172.21.15.86:0/1794053105 shutdown_connections 2024-04-20T21:24:03.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.893+0000 7f1e16e84740 1 -- 172.21.15.86:0/1794053105 wait complete. 2024-04-20T21:24:03.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.893+0000 7f1e16e84740 1 -- start start 2024-04-20T21:24:03.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.894+0000 7f1e16e84740 1 Processor -- start 2024-04-20T21:24:03.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.895+0000 7f1e16e84740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fef1969d70 0x55fef19c10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.895+0000 7f1e16e84740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef19961f0 0x55fef198e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.895+0000 7f1e16e84740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fef198ca60 0x55fef198edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.895+0000 7f1e16e84740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fef196af20 con 0x55fef19961f0 2024-04-20T21:24:03.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.895+0000 7f1e16e84740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fef188b740 con 0x55fef1969d70 2024-04-20T21:24:03.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.895+0000 7f1e16e84740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55fef196a310 con 0x55fef198ca60 2024-04-20T21:24:03.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.895+0000 7f1e16e84740 10 client.0 _ll_register_callbacks cb 0x55fef1886b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:24:03.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.895+0000 7f1e10e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef19961f0 0x55fef198e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:03.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.895+0000 7f1e10e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef19961f0 0x55fef198e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37308/0 (socket says 172.21.15.86:37308) 2024-04-20T21:24:03.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.895+0000 7f1e10e00640 1 -- 172.21.15.86:0/2468076166 learned_addr learned my addr 172.21.15.86:0/2468076166 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:03.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.895+0000 7f1e10e00640 1 -- 172.21.15.86:0/2468076166 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fef198ca60 msgr2=0x55fef198edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:03.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.895+0000 7f1e10e00640 1 --2- 172.21.15.86:0/2468076166 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fef198ca60 0x55fef198edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.895+0000 7f1e10e00640 1 -- 172.21.15.86:0/2468076166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fef1969d70 msgr2=0x55fef19c10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:03.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.895+0000 7f1e11800640 1 --2- 172.21.15.86:0/2468076166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fef1969d70 0x55fef19c10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:03.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.895+0000 7f1e10e00640 1 --2- 172.21.15.86:0/2468076166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fef1969d70 0x55fef19c10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.895+0000 7f1e10e00640 1 -- 172.21.15.86:0/2468076166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fef15e43f0 con 0x55fef19961f0 2024-04-20T21:24:03.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.896+0000 7f1e10e00640 1 --2- 172.21.15.86:0/2468076166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef19961f0 0x55fef198e880 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f1df80090b0 tx=0x7f1df8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c3284c9a08e2f61 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:03.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.896+0000 7f1df2a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:24:03.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.896+0000 7f1df2a00640 1 -- 172.21.15.86:0/2468076166 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df804a070 con 0x55fef19961f0 2024-04-20T21:24:03.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.896+0000 7f1df2a00640 1 -- 172.21.15.86:0/2468076166 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(74..74 src has 1..74) ==== 5882+0+0 (secure 0 0 0) 0x7f1df8003a20 con 0x55fef19961f0 2024-04-20T21:24:03.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.896+0000 7f1e16e84740 1 -- 172.21.15.86:0/2468076166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1df803ad20 con 0x55fef19961f0 2024-04-20T21:24:03.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.896+0000 7f1df2a00640 1 -- 172.21.15.86:0/2468076166 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f1df805f050 con 0x55fef19961f0 2024-04-20T21:24:03.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.896+0000 7f1e16e84740 10 client.7236 fetch_fsmap learned FSMap version 49 2024-04-20T21:24:03.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.896+0000 7f1e16e84740 1 -- 172.21.15.86:0/2468076166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) -- 0x55fef15a40f0 con 0x55fef19961f0 2024-04-20T21:24:03.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.896+0000 7f1df2a00640 1 -- 172.21.15.86:0/2468076166 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1df803ad20 con 0x55fef19961f0 2024-04-20T21:24:03.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.897+0000 7f1df2a00640 1 -- 172.21.15.86:0/2468076166 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 49) ==== 3495+0+0 (secure 0 0 0) 0x7f1df8062020 con 0x55fef19961f0 2024-04-20T21:24:03.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.897+0000 7f1e16e84740 10 client.7236 fetch_fsmap finished waiting for FSMap version 49 2024-04-20T21:24:03.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.897+0000 7f1e16e84740 10 client.7236 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-20T21:24:03.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.897+0000 7f1e16e84740 20 client.7236 populate_metadata read hostname 'smithi086' 2024-04-20T21:24:03.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.897+0000 7f1e16e84740 1 --2- 172.21.15.86:0/2468076166 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x55fef1997d70 0x55fef19c4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.897+0000 7f1e16e84740 4 client.7236 mds_command: new command op to 6987 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:24:03.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.897+0000 7f1e16e84740 1 -- 172.21.15.86:0/2468076166 --> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fef18dfb70 con 0x55fef1997d70 2024-04-20T21:24:03.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.897+0000 7f1e11800640 1 --2- 172.21.15.86:0/2468076166 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x55fef1997d70 0x55fef19c4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:03.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.897+0000 7f1e11800640 1 --2- 172.21.15.86:0/2468076166 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x55fef1997d70 0x55fef19c4be0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d279495a679bec0b server_cookie=c44d8e3c6e36a7f in_seq=0 out_seq=0 2024-04-20T21:24:03.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.897+0000 7f1df2a00640 10 client.7236 ms_handle_connect on v2:172.21.15.86:6835/2254117576 2024-04-20T21:24:03.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.898+0000 7f1df2a00640 1 -- 172.21.15.86:0/2468076166 <== mds.0 v2:172.21.15.86:6835/2254117576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fef18dfb70 con 0x55fef1997d70 2024-04-20T21:24:03.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.898+0000 7f1df2a00640 10 client.7236 handle_command_reply: tid=0 2024-04-20T21:24:03.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.898+0000 7f1e16e84740 1 client.7236 shutdown 2024-04-20T21:24:03.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.898+0000 7f1e16e84740 1 -- 172.21.15.86:0/2468076166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef19961f0 msgr2=0x55fef198e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:03.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.898+0000 7f1e16e84740 1 --2- 172.21.15.86:0/2468076166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef19961f0 0x55fef198e880 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f1df80090b0 tx=0x7f1df8004730 comp rx=0 tx=0).stop 2024-04-20T21:24:03.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.898+0000 7f1e16e84740 1 -- 172.21.15.86:0/2468076166 shutdown_connections 2024-04-20T21:24:03.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.898+0000 7f1e16e84740 1 --2- 172.21.15.86:0/2468076166 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x55fef1997d70 0x55fef19c4be0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.898+0000 7f1e16e84740 1 --2- 172.21.15.86:0/2468076166 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fef198ca60 0x55fef198edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.898+0000 7f1e16e84740 1 --2- 172.21.15.86:0/2468076166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef19961f0 0x55fef198e880 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.898+0000 7f1e16e84740 1 --2- 172.21.15.86:0/2468076166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fef1969d70 0x55fef19c10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.898+0000 7f1df2a00640 0 client.7236 ms_handle_reset on v2:172.21.15.86:6835/2254117576 2024-04-20T21:24:03.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.899+0000 7f1e16e84740 1 -- 172.21.15.86:0/2468076166 >> 172.21.15.86:0/2468076166 conn(0x55fef1966210 msgr2=0x55fef19b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:03.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.899+0000 7f1e16e84740 1 -- 172.21.15.86:0/2468076166 shutdown_connections 2024-04-20T21:24:03.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.899+0000 7f1e16e84740 1 -- 172.21.15.86:0/2468076166 wait complete. 2024-04-20T21:24:03.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.899+0000 7f1e16e84740 20 client.7236 trim_cache size 0 max 16384 2024-04-20T21:24:03.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.917+0000 7f1e16e84740 1 Processor -- start 2024-04-20T21:24:03.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.918+0000 7f1e16e84740 1 -- start start 2024-04-20T21:24:03.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.918+0000 7f1e16e84740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fef1969d70 0x55fef198b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.918+0000 7f1e16e84740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fef198ca60 0x55fef196cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.918+0000 7f1e16e84740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef19961f0 0x55fef1996d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.918+0000 7f1e16e84740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0c200fe0 con 0x55fef19961f0 2024-04-20T21:24:03.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.918+0000 7f1e16e84740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0c1fffb0 con 0x55fef198ca60 2024-04-20T21:24:03.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.918+0000 7f1e16e84740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1e0c14ced0 con 0x55fef1969d70 2024-04-20T21:24:03.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.919+0000 7f1e12200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef19961f0 0x55fef1996d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:03.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.919+0000 7f1e12200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef19961f0 0x55fef1996d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37310/0 (socket says 172.21.15.86:37310) 2024-04-20T21:24:03.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.919+0000 7f1e12200640 1 -- 172.21.15.86:0/1218631979 learned_addr learned my addr 172.21.15.86:0/1218631979 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:03.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.919+0000 7f1e12200640 1 -- 172.21.15.86:0/1218631979 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fef1969d70 msgr2=0x55fef198b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:03.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.919+0000 7f1e12200640 1 --2- 172.21.15.86:0/1218631979 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fef1969d70 0x55fef198b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.919+0000 7f1e12200640 1 -- 172.21.15.86:0/1218631979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fef198ca60 msgr2=0x55fef196cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:03.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.919+0000 7f1e12200640 1 --2- 172.21.15.86:0/1218631979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fef198ca60 0x55fef196cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.919+0000 7f1e12200640 1 -- 172.21.15.86:0/1218631979 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fef1853ac0 con 0x55fef19961f0 2024-04-20T21:24:03.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.919+0000 7f1e12200640 1 --2- 172.21.15.86:0/1218631979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef19961f0 0x55fef1996d20 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f1e04007c90 tx=0x7f1e0400c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea9924e7ae2ae431 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:03.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.920+0000 7f1dfd600640 1 -- 172.21.15.86:0/1218631979 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e04014070 con 0x55fef19961f0 2024-04-20T21:24:03.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.920+0000 7f1dfd600640 1 -- 172.21.15.86:0/1218631979 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e0400f680 con 0x55fef19961f0 2024-04-20T21:24:03.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.920+0000 7f1dfd600640 1 -- 172.21.15.86:0/1218631979 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0400fa70 con 0x55fef19961f0 2024-04-20T21:24:03.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.920+0000 7f1e16e84740 1 -- 172.21.15.86:0/1218631979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef19961f0 msgr2=0x55fef1996d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:03.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.920+0000 7f1e16e84740 1 --2- 172.21.15.86:0/1218631979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef19961f0 0x55fef1996d20 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f1e04007c90 tx=0x7f1e0400c6d0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.920+0000 7f1e16e84740 1 -- 172.21.15.86:0/1218631979 shutdown_connections 2024-04-20T21:24:03.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.920+0000 7f1e16e84740 1 --2- 172.21.15.86:0/1218631979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef19961f0 0x55fef1996d20 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.920+0000 7f1e16e84740 1 --2- 172.21.15.86:0/1218631979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fef198ca60 0x55fef196cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.920+0000 7f1e16e84740 1 --2- 172.21.15.86:0/1218631979 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fef1969d70 0x55fef198b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.920+0000 7f1e16e84740 1 -- 172.21.15.86:0/1218631979 >> 172.21.15.86:0/1218631979 conn(0x55fef1966210 msgr2=0x55fef196c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:03.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.920+0000 7f1e16e84740 1 -- 172.21.15.86:0/1218631979 shutdown_connections 2024-04-20T21:24:03.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.921+0000 7f1e16e84740 1 -- 172.21.15.86:0/1218631979 wait complete. 2024-04-20T21:24:03.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.921+0000 7f1e16e84740 1 -- start start 2024-04-20T21:24:03.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.921+0000 7f1e16e84740 1 Processor -- start 2024-04-20T21:24:03.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.922+0000 7f1e16e84740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fef1969d70 0x55fef1998a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.922+0000 7f1e16e84740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fef198ca60 0x55fef1998f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.922+0000 7f1e16e84740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef19961f0 0x55fef1a2dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.922+0000 7f1e16e84740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0c14d4f0 con 0x55fef19961f0 2024-04-20T21:24:03.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.922+0000 7f1e16e84740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55fef1990cc0 con 0x55fef1969d70 2024-04-20T21:24:03.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.922+0000 7f1e16e84740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55fef1a2ac60 con 0x55fef198ca60 2024-04-20T21:24:03.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.922+0000 7f1e16e84740 10 client.0 _ll_register_callbacks cb 0x7f1df8031f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:24:03.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.923+0000 7f1e12200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef19961f0 0x55fef1a2dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:03.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.923+0000 7f1e12200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef19961f0 0x55fef1a2dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37316/0 (socket says 172.21.15.86:37316) 2024-04-20T21:24:03.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.923+0000 7f1e12200640 1 -- 172.21.15.86:0/754710276 learned_addr learned my addr 172.21.15.86:0/754710276 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:03.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.923+0000 7f1e11800640 1 --2- 172.21.15.86:0/754710276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fef1969d70 0x55fef1998a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:03.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.923+0000 7f1e10e00640 1 --2- 172.21.15.86:0/754710276 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fef198ca60 0x55fef1998f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:03.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.923+0000 7f1e12200640 1 -- 172.21.15.86:0/754710276 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fef198ca60 msgr2=0x55fef1998f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:03.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.923+0000 7f1e12200640 1 --2- 172.21.15.86:0/754710276 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fef198ca60 0x55fef1998f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.923+0000 7f1e12200640 1 -- 172.21.15.86:0/754710276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fef1969d70 msgr2=0x55fef1998a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:03.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.923+0000 7f1e12200640 1 --2- 172.21.15.86:0/754710276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fef1969d70 0x55fef1998a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.923+0000 7f1e12200640 1 -- 172.21.15.86:0/754710276 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fef19b3330 con 0x55fef19961f0 2024-04-20T21:24:03.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.923+0000 7f1e11800640 1 --2- 172.21.15.86:0/754710276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fef1969d70 0x55fef1998a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:24:03.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.923+0000 7f1e12200640 1 --2- 172.21.15.86:0/754710276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef19961f0 0x55fef1a2dfd0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f1e04013000 tx=0x7f1e040070f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae02bb6032af096e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:03.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.923+0000 7f1df2a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:24:03.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.924+0000 7f1df2a00640 1 -- 172.21.15.86:0/754710276 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e04014070 con 0x55fef19961f0 2024-04-20T21:24:03.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.924+0000 7f1df2a00640 1 -- 172.21.15.86:0/754710276 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(74..74 src has 1..74) ==== 5882+0+0 (secure 0 0 0) 0x7f1e0400f480 con 0x55fef19961f0 2024-04-20T21:24:03.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.924+0000 7f1e16e84740 1 -- 172.21.15.86:0/754710276 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e0400f680 con 0x55fef19961f0 2024-04-20T21:24:03.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.924+0000 7f1df2a00640 1 -- 172.21.15.86:0/754710276 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f1e04007ea0 con 0x55fef19961f0 2024-04-20T21:24:03.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.924+0000 7f1e16e84740 10 client.7242 fetch_fsmap learned FSMap version 49 2024-04-20T21:24:03.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.924+0000 7f1e16e84740 1 -- 172.21.15.86:0/754710276 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) -- 0x55fef1a2a9f0 con 0x55fef19961f0 2024-04-20T21:24:03.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.924+0000 7f1df2a00640 1 -- 172.21.15.86:0/754710276 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e0400f680 con 0x55fef19961f0 2024-04-20T21:24:03.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.924+0000 7f1df2a00640 1 -- 172.21.15.86:0/754710276 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 49) ==== 3495+0+0 (secure 0 0 0) 0x7f1e04004040 con 0x55fef19961f0 2024-04-20T21:24:03.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.924+0000 7f1e16e84740 10 client.7242 fetch_fsmap finished waiting for FSMap version 49 2024-04-20T21:24:03.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.924+0000 7f1e16e84740 10 client.7242 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-20T21:24:03.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.924+0000 7f1e16e84740 20 client.7242 populate_metadata read hostname 'smithi086' 2024-04-20T21:24:03.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.924+0000 7f1e16e84740 1 --2- 172.21.15.86:0/754710276 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x55fef19b4c10 0x55fef1a2e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:03.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.924+0000 7f1e16e84740 4 client.7242 mds_command: new command op to 6987 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:24:03.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.924+0000 7f1e16e84740 1 -- 172.21.15.86:0/754710276 --> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fef19984c0 con 0x55fef19b4c10 2024-04-20T21:24:03.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.925+0000 7f1e11800640 1 --2- 172.21.15.86:0/754710276 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x55fef19b4c10 0x55fef1a2e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:03.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.925+0000 7f1e11800640 1 --2- 172.21.15.86:0/754710276 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x55fef19b4c10 0x55fef1a2e510 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=868d89a9525f670 server_cookie=c2e76f2cc4cc7be in_seq=0 out_seq=0 2024-04-20T21:24:03.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.925+0000 7f1df2a00640 10 client.7242 ms_handle_connect on v2:172.21.15.86:6835/2254117576 2024-04-20T21:24:03.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.925+0000 7f1df2a00640 1 -- 172.21.15.86:0/754710276 <== mds.0 v2:172.21.15.86:6835/2254117576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x55fef19984c0 con 0x55fef19b4c10 2024-04-20T21:24:03.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.925+0000 7f1df2a00640 10 client.7242 handle_command_reply: tid=0 2024-04-20T21:24:03.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.925+0000 7f1e16e84740 1 client.7242 shutdown 2024-04-20T21:24:03.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.926+0000 7f1e16e84740 1 -- 172.21.15.86:0/754710276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef19961f0 msgr2=0x55fef1a2dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:03.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.926+0000 7f1e16e84740 1 --2- 172.21.15.86:0/754710276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef19961f0 0x55fef1a2dfd0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f1e04013000 tx=0x7f1e040070f0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.926+0000 7f1e16e84740 1 -- 172.21.15.86:0/754710276 shutdown_connections 2024-04-20T21:24:03.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.926+0000 7f1e16e84740 1 --2- 172.21.15.86:0/754710276 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x55fef19b4c10 0x55fef1a2e510 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.926+0000 7f1e16e84740 1 --2- 172.21.15.86:0/754710276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fef19961f0 0x55fef1a2dfd0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.926+0000 7f1e16e84740 1 --2- 172.21.15.86:0/754710276 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55fef198ca60 0x55fef1998f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.926+0000 7f1e16e84740 1 --2- 172.21.15.86:0/754710276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55fef1969d70 0x55fef1998a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.926+0000 7f1df2a00640 0 client.7242 ms_handle_reset on v2:172.21.15.86:6835/2254117576 2024-04-20T21:24:03.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.926+0000 7f1e16e84740 1 -- 172.21.15.86:0/754710276 >> 172.21.15.86:0/754710276 conn(0x55fef1966210 msgr2=0x55fef1996d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:03.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.926+0000 7f1e16e84740 1 -- 172.21.15.86:0/754710276 shutdown_connections 2024-04-20T21:24:03.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.926+0000 7f1e16e84740 1 -- 172.21.15.86:0/754710276 wait complete. 2024-04-20T21:24:03.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.926+0000 7f1e16e84740 20 client.7242 trim_cache size 0 max 16384 2024-04-20T21:24:03.929 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:24:03.929 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T21:24:03.929 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-20T21:24:03.929 INFO:teuthology.orchestra.run.smithi086.stdout: "855a57fc-ebe5-457a-ab9f-b8574f87c5d6": { 2024-04-20T21:24:03.929 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/subdir", 2024-04-20T21:24:03.929 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "855a57fc-ebe5-457a-ab9f-b8574f87c5d6", 2024-04-20T21:24:03.929 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,repair" 2024-04-20T21:24:03.929 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:24:03.930 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:24:03.930 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:24:03.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.927+0000 7f1e14200640 1 -- 172.21.15.86:0/1795122257 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1dec06c8f0 msgr2=0x7f1dec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:03.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.927+0000 7f1e14200640 1 --2- 172.21.15.86:0/1795122257 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1dec06c8f0 0x7f1dec06edb0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1df8036000 tx=0x7f1df8030000 comp rx=0 tx=0).stop 2024-04-20T21:24:03.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.927+0000 7f1e14200640 1 -- 172.21.15.86:0/1795122257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e0c06b980 msgr2=0x7f1e0c149db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:03.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.927+0000 7f1e14200640 1 --2- 172.21.15.86:0/1795122257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e0c06b980 0x7f1e0c149db0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f1e0000c820 tx=0x7f1e0000cce0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.927+0000 7f1e14200640 1 -- 172.21.15.86:0/1795122257 shutdown_connections 2024-04-20T21:24:03.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.927+0000 7f1e14200640 1 --2- 172.21.15.86:0/1795122257 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1dec06c8f0 0x7f1dec06edb0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.927+0000 7f1e14200640 1 --2- 172.21.15.86:0/1795122257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e0c15eb30 0x7f1e0c159c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.927+0000 7f1e14200640 1 --2- 172.21.15.86:0/1795122257 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1e0c0aee50 0x7f1e0c0cf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.927+0000 7f1e14200640 1 --2- 172.21.15.86:0/1795122257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e0c06b980 0x7f1e0c149db0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:03.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.927+0000 7f1e14200640 1 -- 172.21.15.86:0/1795122257 >> 172.21.15.86:0/1795122257 conn(0x7f1e0c0b2110 msgr2=0x7f1e0c15d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:03.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.927+0000 7f1e14200640 1 -- 172.21.15.86:0/1795122257 shutdown_connections 2024-04-20T21:24:03.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:03.927+0000 7f1e14200640 1 -- 172.21.15.86:0/1795122257 wait complete. 2024-04-20T21:24:03.942 INFO:tasks.cephfs.filesystem:scrub status for tag:855a57fc-ebe5-457a-ab9f-b8574f87c5d6 - {'path': '/subdir', 'tag': '855a57fc-ebe5-457a-ab9f-b8574f87c5d6', 'options': 'recursive,repair'} 2024-04-20T21:24:16.886 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:24:16.898 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:24:33.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:24:33.943 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-20T21:24:34.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.157+0000 7efcb9600640 1 Processor -- start 2024-04-20T21:24:34.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.157+0000 7efcb9600640 1 -- start start 2024-04-20T21:24:34.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.158+0000 7efcb9600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efcb415eb30 0x7efcb40b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:34.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.159+0000 7efcb9600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcb40adb10 con 0x7efcb40aee50 2024-04-20T21:24:34.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.159+0000 7efcb9600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcb40addb0 con 0x7efcb406c090 2024-04-20T21:24:34.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.159+0000 7efcb9600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7efcb40af7d0 con 0x7efcb415eb30 2024-04-20T21:24:34.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.159+0000 7efcabe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7efcb40aee50 0x7efcb406b980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:48544/0 (socket says 172.21.15.86:48544) 2024-04-20T21:24:34.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.159+0000 7efcabe00640 1 -- 172.21.15.86:0/497893573 learned_addr learned my addr 172.21.15.86:0/497893573 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:24:34.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.159+0000 7efcb2a00640 1 --2- 172.21.15.86:0/497893573 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efcb415eb30 0x7efcb40b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:34.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.159+0000 7efcb2000640 1 -- 172.21.15.86:0/497893573 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2992758465 0 0) 0x7efcb40adb10 con 0x7efcb40aee50 2024-04-20T21:24:34.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.159+0000 7efcb2000640 1 -- 172.21.15.86:0/497893573 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efc98003520 con 0x7efcb40aee50 2024-04-20T21:24:34.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.160+0000 7efcb2000640 1 -- 172.21.15.86:0/497893573 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2509659084 0 0) 0x7efc98003520 con 0x7efcb40aee50 2024-04-20T21:24:34.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.160+0000 7efcb2000640 1 -- 172.21.15.86:0/497893573 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efcb415eb30 msgr2=0x7efcb40b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:34.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.160+0000 7efcb2000640 1 --2- 172.21.15.86:0/497893573 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efcb415eb30 0x7efcb40b3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.160+0000 7efcb2000640 1 -- 172.21.15.86:0/497893573 >> v1:172.21.15.144:6789/0 conn(0x7efcb406c090 legacy=0x7efcb406e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:34.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.160+0000 7efcb2000640 1 -- 172.21.15.86:0/497893573 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcb40ae090 con 0x7efcb40aee50 2024-04-20T21:24:34.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.160+0000 7efcb2000640 1 -- 172.21.15.86:0/497893573 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7efc9c001670 con 0x7efcb40aee50 2024-04-20T21:24:34.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.160+0000 7efcb2000640 1 -- 172.21.15.86:0/497893573 >> v1:172.21.15.86:6789/0 conn(0x7efcb40aee50 legacy=0x7efcb406b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:34.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.160+0000 7efcb2000640 1 --2- 172.21.15.86:0/497893573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc980039e0 0x7efc98003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:34.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.160+0000 7efcb2000640 1 --2- 172.21.15.86:0/497893573 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efcb415eb30 0x7efc98004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:34.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.160+0000 7efcb2000640 1 --2- 172.21.15.86:0/497893573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc98004d40 0x7efc980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:34.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.160+0000 7efcb2000640 1 -- 172.21.15.86:0/497893573 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efcb40af7d0 con 0x7efc980039e0 2024-04-20T21:24:34.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.160+0000 7efcb2000640 1 -- 172.21.15.86:0/497893573 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efc9800ba70 con 0x7efc98004d40 2024-04-20T21:24:34.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.160+0000 7efcb2000640 1 -- 172.21.15.86:0/497893573 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7efc9800bc70 con 0x7efcb415eb30 2024-04-20T21:24:34.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.160+0000 7efcb2a00640 1 --2- 172.21.15.86:0/497893573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc980039e0 0x7efc98003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:34.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.161+0000 7efcabe00640 1 --2- 172.21.15.86:0/497893573 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efcb415eb30 0x7efc98004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:34.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.161+0000 7efcb3400640 1 --2- 172.21.15.86:0/497893573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc98004d40 0x7efc980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:34.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.161+0000 7efcb2a00640 1 -- 172.21.15.86:0/497893573 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efcb415eb30 msgr2=0x7efc98004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:34.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.161+0000 7efcb2a00640 1 --2- 172.21.15.86:0/497893573 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efcb415eb30 0x7efc98004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.161+0000 7efcb2a00640 1 -- 172.21.15.86:0/497893573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc98004d40 msgr2=0x7efc980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:34.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.161+0000 7efcb2a00640 1 --2- 172.21.15.86:0/497893573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc98004d40 0x7efc980051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.161+0000 7efcb2a00640 1 -- 172.21.15.86:0/497893573 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc98003520 con 0x7efc980039e0 2024-04-20T21:24:34.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.162+0000 7efcb2a00640 1 --2- 172.21.15.86:0/497893573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc980039e0 0x7efc98003e00 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7efca4009a90 tx=0x7efc98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1462b3ca66c3e9c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:34.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.162+0000 7efcb2000640 1 -- 172.21.15.86:0/497893573 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca4039070 con 0x7efc980039e0 2024-04-20T21:24:34.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.162+0000 7efcb2000640 1 -- 172.21.15.86:0/497893573 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efca402c680 con 0x7efc980039e0 2024-04-20T21:24:34.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.162+0000 7efcb2000640 1 -- 172.21.15.86:0/497893573 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca4009690 con 0x7efc980039e0 2024-04-20T21:24:34.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.163+0000 7efcb9600640 1 -- 172.21.15.86:0/497893573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc980039e0 msgr2=0x7efc98003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:34.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.163+0000 7efcb9600640 1 --2- 172.21.15.86:0/497893573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc980039e0 0x7efc98003e00 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7efca4009a90 tx=0x7efc98005830 comp rx=0 tx=0).stop 2024-04-20T21:24:34.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.163+0000 7efcb3400640 1 -- 172.21.15.86:0/497893573 reap_dead start 2024-04-20T21:24:34.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.163+0000 7efcb9600640 1 -- 172.21.15.86:0/497893573 shutdown_connections 2024-04-20T21:24:34.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.163+0000 7efcb9600640 1 -- 172.21.15.86:0/497893573 >> 172.21.15.86:0/497893573 conn(0x7efcb40b2110 msgr2=0x7efcb40b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:34.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.163+0000 7efcb9600640 1 -- 172.21.15.86:0/497893573 shutdown_connections 2024-04-20T21:24:34.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.163+0000 7efcb9600640 1 -- 172.21.15.86:0/497893573 wait complete. 2024-04-20T21:24:34.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.165+0000 7efcb9600640 1 Processor -- start 2024-04-20T21:24:34.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.165+0000 7efcb9600640 1 -- start start 2024-04-20T21:24:34.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.166+0000 7efcb9600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc980039e0 0x7efcb4158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:34.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.166+0000 7efcb9600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efcb4150100 0x7efcb41f42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:34.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.166+0000 7efcb9600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcb415d190 0x7efcb4149c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:34.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.166+0000 7efcb9600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efcb40ae930 con 0x7efcb415d190 2024-04-20T21:24:34.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.166+0000 7efcb9600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efcb40ae650 con 0x7efc980039e0 2024-04-20T21:24:34.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.166+0000 7efcb9600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7efcb406e870 con 0x7efcb4150100 2024-04-20T21:24:34.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.166+0000 7efcb3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcb415d190 0x7efcb4149c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:34.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.166+0000 7efcb3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcb415d190 0x7efcb4149c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45818/0 (socket says 172.21.15.86:45818) 2024-04-20T21:24:34.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.166+0000 7efcb3400640 1 -- 172.21.15.86:0/4058520715 learned_addr learned my addr 172.21.15.86:0/4058520715 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:34.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.166+0000 7efcb2a00640 1 --2- 172.21.15.86:0/4058520715 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc980039e0 0x7efcb4158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:34.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.166+0000 7efcabe00640 1 --2- 172.21.15.86:0/4058520715 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efcb4150100 0x7efcb41f42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:34.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.167+0000 7efcb3400640 1 -- 172.21.15.86:0/4058520715 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efcb4150100 msgr2=0x7efcb41f42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:34.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.167+0000 7efcb3400640 1 --2- 172.21.15.86:0/4058520715 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efcb4150100 0x7efcb41f42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.167+0000 7efcb3400640 1 -- 172.21.15.86:0/4058520715 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc980039e0 msgr2=0x7efcb4158050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:34.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.167+0000 7efcb3400640 1 --2- 172.21.15.86:0/4058520715 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc980039e0 0x7efcb4158050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.167+0000 7efcb3400640 1 -- 172.21.15.86:0/4058520715 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcb4158750 con 0x7efcb415d190 2024-04-20T21:24:34.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.167+0000 7efcb2a00640 1 --2- 172.21.15.86:0/4058520715 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc980039e0 0x7efcb4158050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:24:34.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.167+0000 7efcb3400640 1 --2- 172.21.15.86:0/4058520715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcb415d190 0x7efcb4149c70 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7efcb40b0010 tx=0x7efcac00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:34.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.167+0000 7efcaaa00640 1 -- 172.21.15.86:0/4058520715 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcac018070 con 0x7efcb415d190 2024-04-20T21:24:34.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.167+0000 7efcaaa00640 1 -- 172.21.15.86:0/4058520715 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcac003b70 con 0x7efcb415d190 2024-04-20T21:24:34.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.167+0000 7efcb9600640 1 -- 172.21.15.86:0/4058520715 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcb40cdea0 con 0x7efcb415d190 2024-04-20T21:24:34.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.167+0000 7efcaaa00640 1 -- 172.21.15.86:0/4058520715 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcac005070 con 0x7efcb415d190 2024-04-20T21:24:34.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.167+0000 7efcb9600640 1 -- 172.21.15.86:0/4058520715 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcb40ce460 con 0x7efcb415d190 2024-04-20T21:24:34.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.170+0000 7efcaaa00640 1 -- 172.21.15.86:0/4058520715 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efcac01b070 con 0x7efcb415d190 2024-04-20T21:24:34.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.171+0000 7efcaaa00640 1 --2- 172.21.15.86:0/4058520715 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efc9006c8f0 0x7efc9006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:34.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.171+0000 7efcaaa00640 1 -- 172.21.15.86:0/4058520715 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5882+0+0 (secure 0 0 0) 0x7efcac08b4a0 con 0x7efcb415d190 2024-04-20T21:24:34.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.171+0000 7efcb2a00640 1 --2- 172.21.15.86:0/4058520715 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efc9006c8f0 0x7efc9006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:34.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.171+0000 7efcbc3e2740 1 Processor -- start 2024-04-20T21:24:34.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.171+0000 7efcbc3e2740 1 -- start start 2024-04-20T21:24:34.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.172+0000 7efcb2a00640 1 --2- 172.21.15.86:0/4058520715 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efc9006c8f0 0x7efc9006edb0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7efca4036000 tx=0x7efca4030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:34.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.172+0000 7efcbc3e2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a98461f0 0x5595a98465d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:34.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.172+0000 7efcbc3e2740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5595a9816210 0x5595a9843460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:34.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.172+0000 7efcbc3e2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595a9819d70 0x5595a98439a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:34.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.172+0000 7efcbc3e2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5595a9782150 con 0x5595a98461f0 2024-04-20T21:24:34.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.172+0000 7efcbc3e2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5595a9728370 con 0x5595a9819d70 2024-04-20T21:24:34.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.172+0000 7efcbc3e2740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5595a977e110 con 0x5595a9816210 2024-04-20T21:24:34.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.172+0000 7efcb2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a98461f0 0x5595a98465d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:34.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.172+0000 7efcb2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a98461f0 0x5595a98465d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45822/0 (socket says 172.21.15.86:45822) 2024-04-20T21:24:34.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.172+0000 7efcb2a00640 1 -- 172.21.15.86:0/3866260705 learned_addr learned my addr 172.21.15.86:0/3866260705 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:34.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.173+0000 7efcb3400640 1 --2- 172.21.15.86:0/3866260705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595a9819d70 0x5595a98439a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:34.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.173+0000 7efcabe00640 1 --2- 172.21.15.86:0/3866260705 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5595a9816210 0x5595a9843460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:34.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.173+0000 7efcb2a00640 1 -- 172.21.15.86:0/3866260705 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5595a9816210 msgr2=0x5595a9843460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:34.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.173+0000 7efcb2a00640 1 --2- 172.21.15.86:0/3866260705 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5595a9816210 0x5595a9843460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.173+0000 7efcb2a00640 1 -- 172.21.15.86:0/3866260705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595a9819d70 msgr2=0x5595a98439a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:34.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.173+0000 7efcb2a00640 1 --2- 172.21.15.86:0/3866260705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595a9819d70 0x5595a98439a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.173+0000 7efcb2a00640 1 -- 172.21.15.86:0/3866260705 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595a955e7d0 con 0x5595a98461f0 2024-04-20T21:24:34.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.173+0000 7efcb3400640 1 --2- 172.21.15.86:0/3866260705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595a9819d70 0x5595a98439a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:24:34.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.173+0000 7efcb2a00640 1 --2- 172.21.15.86:0/3866260705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a98461f0 0x5595a98465d0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7efca4009b80 tx=0x7efca402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcf91ce5d9cf7fa3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:34.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.173+0000 7efca3000640 1 -- 172.21.15.86:0/3866260705 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca4051070 con 0x5595a98461f0 2024-04-20T21:24:34.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.174+0000 7efca3000640 1 -- 172.21.15.86:0/3866260705 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efca403a040 con 0x5595a98461f0 2024-04-20T21:24:34.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.174+0000 7efca3000640 1 -- 172.21.15.86:0/3866260705 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca4039050 con 0x5595a98461f0 2024-04-20T21:24:34.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.174+0000 7efcbc3e2740 1 -- 172.21.15.86:0/3866260705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a98461f0 msgr2=0x5595a98465d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:34.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.174+0000 7efcbc3e2740 1 --2- 172.21.15.86:0/3866260705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a98461f0 0x5595a98465d0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7efca4009b80 tx=0x7efca402e000 comp rx=0 tx=0).stop 2024-04-20T21:24:34.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.174+0000 7efcbc3e2740 1 -- 172.21.15.86:0/3866260705 shutdown_connections 2024-04-20T21:24:34.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.174+0000 7efcbc3e2740 1 --2- 172.21.15.86:0/3866260705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595a9819d70 0x5595a98439a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.174+0000 7efcbc3e2740 1 --2- 172.21.15.86:0/3866260705 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5595a9816210 0x5595a9843460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.174+0000 7efcbc3e2740 1 --2- 172.21.15.86:0/3866260705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a98461f0 0x5595a98465d0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.174+0000 7efcbc3e2740 1 -- 172.21.15.86:0/3866260705 >> 172.21.15.86:0/3866260705 conn(0x5595a9818480 msgr2=0x5595a9861000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:34.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.174+0000 7efcbc3e2740 1 -- 172.21.15.86:0/3866260705 shutdown_connections 2024-04-20T21:24:34.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.174+0000 7efcbc3e2740 1 -- 172.21.15.86:0/3866260705 wait complete. 2024-04-20T21:24:34.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.175+0000 7efcbc3e2740 1 -- start start 2024-04-20T21:24:34.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.175+0000 7efcbc3e2740 1 Processor -- start 2024-04-20T21:24:34.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.176+0000 7efcbc3e2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595a9819d70 0x5595a98490e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:34.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.176+0000 7efcbc3e2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a98461f0 0x5595a984bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:34.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.176+0000 7efcbc3e2740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5595a984c200 0x5595a98424a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:34.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.176+0000 7efcbc3e2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5595a981af20 con 0x5595a98461f0 2024-04-20T21:24:34.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.176+0000 7efcbc3e2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5595a973b740 con 0x5595a9819d70 2024-04-20T21:24:34.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.176+0000 7efcbc3e2740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5595a981a310 con 0x5595a984c200 2024-04-20T21:24:34.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.176+0000 7efcbc3e2740 10 client.0 _ll_register_callbacks cb 0x5595a9736b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:24:34.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.177+0000 7efcabe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a98461f0 0x5595a984bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:34.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.177+0000 7efcabe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a98461f0 0x5595a984bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45826/0 (socket says 172.21.15.86:45826) 2024-04-20T21:24:34.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.177+0000 7efcabe00640 1 -- 172.21.15.86:0/3011382095 learned_addr learned my addr 172.21.15.86:0/3011382095 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:34.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.177+0000 7efcb2a00640 1 --2- 172.21.15.86:0/3011382095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595a9819d70 0x5595a98490e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:34.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.177+0000 7efcabe00640 1 -- 172.21.15.86:0/3011382095 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5595a984c200 msgr2=0x5595a98424a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:34.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.177+0000 7efcabe00640 1 --2- 172.21.15.86:0/3011382095 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5595a984c200 0x5595a98424a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.177+0000 7efcabe00640 1 -- 172.21.15.86:0/3011382095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595a9819d70 msgr2=0x5595a98490e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:34.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.177+0000 7efcabe00640 1 --2- 172.21.15.86:0/3011382095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595a9819d70 0x5595a98490e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.177+0000 7efcabe00640 1 -- 172.21.15.86:0/3011382095 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595a94943f0 con 0x5595a98461f0 2024-04-20T21:24:34.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.177+0000 7efcabe00640 1 --2- 172.21.15.86:0/3011382095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a98461f0 0x5595a984bcc0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7efc9c0114d0 tx=0x7efc9c011990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bda7f92bc722c40 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:34.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.177+0000 7efc97400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:24:34.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.178+0000 7efc97400640 1 -- 172.21.15.86:0/3011382095 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9c010040 con 0x5595a98461f0 2024-04-20T21:24:34.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.178+0000 7efc97400640 1 -- 172.21.15.86:0/3011382095 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(74..74 src has 1..74) ==== 5882+0+0 (secure 0 0 0) 0x7efc9c0137a0 con 0x5595a98461f0 2024-04-20T21:24:34.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.178+0000 7efcbc3e2740 1 -- 172.21.15.86:0/3011382095 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efca403a040 con 0x5595a98461f0 2024-04-20T21:24:34.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.178+0000 7efc97400640 1 -- 172.21.15.86:0/3011382095 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7efc9c0139a0 con 0x5595a98461f0 2024-04-20T21:24:34.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.178+0000 7efcbc3e2740 10 client.7254 fetch_fsmap learned FSMap version 50 2024-04-20T21:24:34.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.178+0000 7efcbc3e2740 1 -- 172.21.15.86:0/3011382095 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x5595a94540f0 con 0x5595a98461f0 2024-04-20T21:24:34.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.178+0000 7efc97400640 1 -- 172.21.15.86:0/3011382095 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efca403a040 con 0x5595a98461f0 2024-04-20T21:24:34.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.178+0000 7efc97400640 1 -- 172.21.15.86:0/3011382095 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 50) ==== 3495+0+0 (secure 0 0 0) 0x7efc9c025080 con 0x5595a98461f0 2024-04-20T21:24:34.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.178+0000 7efcbc3e2740 10 client.7254 fetch_fsmap finished waiting for FSMap version 50 2024-04-20T21:24:34.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.178+0000 7efcbc3e2740 10 client.7254 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-20T21:24:34.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.178+0000 7efcbc3e2740 20 client.7254 populate_metadata read hostname 'smithi086' 2024-04-20T21:24:34.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.178+0000 7efcbc3e2740 1 --2- 172.21.15.86:0/3011382095 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x5595a984ae00 0x5595a984b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:34.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.178+0000 7efcbc3e2740 4 client.7254 mds_command: new command op to 6987 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:24:34.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.178+0000 7efcbc3e2740 1 -- 172.21.15.86:0/3011382095 --> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595a978fb70 con 0x5595a984ae00 2024-04-20T21:24:34.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.179+0000 7efcb2a00640 1 --2- 172.21.15.86:0/3011382095 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x5595a984ae00 0x5595a984b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:34.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.179+0000 7efcb2a00640 1 --2- 172.21.15.86:0/3011382095 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x5595a984ae00 0x5595a984b270 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e18c536a01431422 server_cookie=bc376c0cc452b103 in_seq=0 out_seq=0 2024-04-20T21:24:34.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.179+0000 7efc97400640 10 client.7254 ms_handle_connect on v2:172.21.15.86:6835/2254117576 2024-04-20T21:24:34.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.180+0000 7efc97400640 1 -- 172.21.15.86:0/3011382095 <== mds.0 v2:172.21.15.86:6835/2254117576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5595a978fb70 con 0x5595a984ae00 2024-04-20T21:24:34.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.180+0000 7efc97400640 10 client.7254 handle_command_reply: tid=0 2024-04-20T21:24:34.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.180+0000 7efcbc3e2740 1 client.7254 shutdown 2024-04-20T21:24:34.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.180+0000 7efcbc3e2740 1 -- 172.21.15.86:0/3011382095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a98461f0 msgr2=0x5595a984bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:34.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.180+0000 7efcbc3e2740 1 --2- 172.21.15.86:0/3011382095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a98461f0 0x5595a984bcc0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7efc9c0114d0 tx=0x7efc9c011990 comp rx=0 tx=0).stop 2024-04-20T21:24:34.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.180+0000 7efcbc3e2740 1 -- 172.21.15.86:0/3011382095 shutdown_connections 2024-04-20T21:24:34.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.180+0000 7efcbc3e2740 1 --2- 172.21.15.86:0/3011382095 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x5595a984ae00 0x5595a984b270 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.180+0000 7efcbc3e2740 1 --2- 172.21.15.86:0/3011382095 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5595a984c200 0x5595a98424a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.180+0000 7efcbc3e2740 1 --2- 172.21.15.86:0/3011382095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a98461f0 0x5595a984bcc0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.180+0000 7efcbc3e2740 1 --2- 172.21.15.86:0/3011382095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595a9819d70 0x5595a98490e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.180+0000 7efc97400640 0 client.7254 ms_handle_reset on v2:172.21.15.86:6835/2254117576 2024-04-20T21:24:34.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.180+0000 7efcbc3e2740 1 -- 172.21.15.86:0/3011382095 >> 172.21.15.86:0/3011382095 conn(0x5595a9816210 msgr2=0x5595a981b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:34.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.180+0000 7efcbc3e2740 1 -- 172.21.15.86:0/3011382095 shutdown_connections 2024-04-20T21:24:34.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.180+0000 7efcbc3e2740 1 -- 172.21.15.86:0/3011382095 wait complete. 2024-04-20T21:24:34.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.181+0000 7efcbc3e2740 20 client.7254 trim_cache size 0 max 16384 2024-04-20T21:24:34.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.199+0000 7efcbc3e2740 1 Processor -- start 2024-04-20T21:24:34.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.200+0000 7efcbc3e2740 1 -- start start 2024-04-20T21:24:34.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.200+0000 7efcbc3e2740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5595a9819d70 0x5595a987b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:34.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.200+0000 7efcbc3e2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595a98461f0 0x5595a984b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:34.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.200+0000 7efcbc3e2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a984c200 0x5595a9815390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:34.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.200+0000 7efcbc3e2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efcb4201860 con 0x5595a984c200 2024-04-20T21:24:34.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.200+0000 7efcbc3e2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efcb4200830 con 0x5595a98461f0 2024-04-20T21:24:34.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.200+0000 7efcbc3e2740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7efcb41ffb50 con 0x5595a9819d70 2024-04-20T21:24:34.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.201+0000 7efcb3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a984c200 0x5595a9815390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:34.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.201+0000 7efcb3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a984c200 0x5595a9815390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45830/0 (socket says 172.21.15.86:45830) 2024-04-20T21:24:34.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.201+0000 7efcb3400640 1 -- 172.21.15.86:0/1216237355 learned_addr learned my addr 172.21.15.86:0/1216237355 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:34.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.201+0000 7efcb3400640 1 -- 172.21.15.86:0/1216237355 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5595a9819d70 msgr2=0x5595a987b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:34.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.201+0000 7efcb2a00640 1 --2- 172.21.15.86:0/1216237355 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5595a9819d70 0x5595a987b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:34.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.201+0000 7efcb3400640 1 --2- 172.21.15.86:0/1216237355 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5595a9819d70 0x5595a987b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.201+0000 7efcb3400640 1 -- 172.21.15.86:0/1216237355 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595a98461f0 msgr2=0x5595a984b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:34.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.201+0000 7efcabe00640 1 --2- 172.21.15.86:0/1216237355 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595a98461f0 0x5595a984b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:34.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.201+0000 7efcb3400640 1 --2- 172.21.15.86:0/1216237355 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595a98461f0 0x5595a984b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.201+0000 7efcb3400640 1 -- 172.21.15.86:0/1216237355 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595a984bd70 con 0x5595a984c200 2024-04-20T21:24:34.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.201+0000 7efcb3400640 1 --2- 172.21.15.86:0/1216237355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a984c200 0x5595a9815390 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7efcac005040 tx=0x7efcac0591d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a2881bdca1c2498 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:34.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.202+0000 7efca2600640 1 -- 172.21.15.86:0/1216237355 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcac057410 con 0x5595a984c200 2024-04-20T21:24:34.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.202+0000 7efca2600640 1 -- 172.21.15.86:0/1216237355 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcac055780 con 0x5595a984c200 2024-04-20T21:24:34.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.202+0000 7efca2600640 1 -- 172.21.15.86:0/1216237355 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcac061860 con 0x5595a984c200 2024-04-20T21:24:34.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.202+0000 7efcbc3e2740 1 -- 172.21.15.86:0/1216237355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a984c200 msgr2=0x5595a9815390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:34.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.202+0000 7efcbc3e2740 1 --2- 172.21.15.86:0/1216237355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a984c200 0x5595a9815390 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7efcac005040 tx=0x7efcac0591d0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.202+0000 7efcbc3e2740 1 -- 172.21.15.86:0/1216237355 shutdown_connections 2024-04-20T21:24:34.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.202+0000 7efcbc3e2740 1 --2- 172.21.15.86:0/1216237355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a984c200 0x5595a9815390 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.202+0000 7efcbc3e2740 1 --2- 172.21.15.86:0/1216237355 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595a98461f0 0x5595a984b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.202+0000 7efcbc3e2740 1 --2- 172.21.15.86:0/1216237355 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5595a9819d70 0x5595a987b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.202+0000 7efcbc3e2740 1 -- 172.21.15.86:0/1216237355 >> 172.21.15.86:0/1216237355 conn(0x5595a9816210 msgr2=0x5595a9842270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:34.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.202+0000 7efcbc3e2740 1 -- 172.21.15.86:0/1216237355 shutdown_connections 2024-04-20T21:24:34.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.203+0000 7efcbc3e2740 1 -- 172.21.15.86:0/1216237355 wait complete. 2024-04-20T21:24:34.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.203+0000 7efcbc3e2740 1 -- start start 2024-04-20T21:24:34.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.203+0000 7efcbc3e2740 1 Processor -- start 2024-04-20T21:24:34.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.204+0000 7efcbc3e2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a9819d70 0x5595a98c1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:34.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.204+0000 7efcbc3e2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595a98461f0 0x5595a984d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:34.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.204+0000 7efcbc3e2740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5595a984c200 0x5595a984dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:34.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.204+0000 7efcbc3e2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5595a98166a0 con 0x5595a9819d70 2024-04-20T21:24:34.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.204+0000 7efcbc3e2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efcb42001a0 con 0x5595a98461f0 2024-04-20T21:24:34.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.204+0000 7efcbc3e2740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5595a9811c00 con 0x5595a984c200 2024-04-20T21:24:34.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.204+0000 7efcbc3e2740 10 client.0 _ll_register_callbacks cb 0x7efc9c001430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:24:34.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.205+0000 7efcb2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a9819d70 0x5595a98c1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:34.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.205+0000 7efcb2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a9819d70 0x5595a98c1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45840/0 (socket says 172.21.15.86:45840) 2024-04-20T21:24:34.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.205+0000 7efcb2a00640 1 -- 172.21.15.86:0/3866316959 learned_addr learned my addr 172.21.15.86:0/3866316959 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:34.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.205+0000 7efcb2a00640 1 -- 172.21.15.86:0/3866316959 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5595a984c200 msgr2=0x5595a984dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:34.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.205+0000 7efcb2a00640 1 --2- 172.21.15.86:0/3866316959 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5595a984c200 0x5595a984dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.205+0000 7efcb2a00640 1 -- 172.21.15.86:0/3866316959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595a98461f0 msgr2=0x5595a984d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:34.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.205+0000 7efcb2a00640 1 --2- 172.21.15.86:0/3866316959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595a98461f0 0x5595a984d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.205+0000 7efcb2a00640 1 -- 172.21.15.86:0/3866316959 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595a9862430 con 0x5595a9819d70 2024-04-20T21:24:34.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.205+0000 7efcb2a00640 1 --2- 172.21.15.86:0/3866316959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a9819d70 0x5595a98c1190 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7efca4037040 tx=0x7efca402e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23424f7a9fd75060 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:34.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.205+0000 7efc97400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:24:34.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.206+0000 7efc97400640 1 -- 172.21.15.86:0/3866316959 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca4051040 con 0x5595a9819d70 2024-04-20T21:24:34.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.206+0000 7efc97400640 1 -- 172.21.15.86:0/3866316959 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(74..74 src has 1..74) ==== 5882+0+0 (secure 0 0 0) 0x7efca4005b30 con 0x5595a9819d70 2024-04-20T21:24:34.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.206+0000 7efcbc3e2740 1 -- 172.21.15.86:0/3866316959 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcac055780 con 0x5595a9819d70 2024-04-20T21:24:34.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.206+0000 7efc97400640 1 -- 172.21.15.86:0/3866316959 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7efca4063050 con 0x5595a9819d70 2024-04-20T21:24:34.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.206+0000 7efcbc3e2740 10 client.7260 fetch_fsmap learned FSMap version 50 2024-04-20T21:24:34.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.206+0000 7efcbc3e2740 1 -- 172.21.15.86:0/3866316959 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x5595a9883f80 con 0x5595a9819d70 2024-04-20T21:24:34.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.206+0000 7efc97400640 1 -- 172.21.15.86:0/3866316959 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcac055780 con 0x5595a9819d70 2024-04-20T21:24:34.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.206+0000 7efc97400640 1 -- 172.21.15.86:0/3866316959 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 50) ==== 3495+0+0 (secure 0 0 0) 0x7efca4066020 con 0x5595a9819d70 2024-04-20T21:24:34.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.206+0000 7efcbc3e2740 10 client.7260 fetch_fsmap finished waiting for FSMap version 50 2024-04-20T21:24:34.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.206+0000 7efcbc3e2740 10 client.7260 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-20T21:24:34.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.206+0000 7efcbc3e2740 20 client.7260 populate_metadata read hostname 'smithi086' 2024-04-20T21:24:34.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.206+0000 7efcbc3e2740 1 --2- 172.21.15.86:0/3866316959 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x5595a9843480 0x5595a9872a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:34.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.206+0000 7efcbc3e2740 4 client.7260 mds_command: new command op to 6987 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:24:34.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.206+0000 7efcbc3e2740 1 -- 172.21.15.86:0/3866316959 --> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5595a97899e0 con 0x5595a9843480 2024-04-20T21:24:34.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.207+0000 7efcabe00640 1 --2- 172.21.15.86:0/3866316959 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x5595a9843480 0x5595a9872a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:34.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.207+0000 7efcabe00640 1 --2- 172.21.15.86:0/3866316959 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x5595a9843480 0x5595a9872a70 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b666235b7da0d33c server_cookie=49ec9416ee2be613 in_seq=0 out_seq=0 2024-04-20T21:24:34.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.207+0000 7efc97400640 10 client.7260 ms_handle_connect on v2:172.21.15.86:6835/2254117576 2024-04-20T21:24:34.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.207+0000 7efc97400640 1 -- 172.21.15.86:0/3866316959 <== mds.0 v2:172.21.15.86:6835/2254117576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5595a97899e0 con 0x5595a9843480 2024-04-20T21:24:34.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.207+0000 7efc97400640 10 client.7260 handle_command_reply: tid=0 2024-04-20T21:24:34.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.207+0000 7efcbc3e2740 1 client.7260 shutdown 2024-04-20T21:24:34.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.208+0000 7efcbc3e2740 1 -- 172.21.15.86:0/3866316959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a9819d70 msgr2=0x5595a98c1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:34.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.208+0000 7efcbc3e2740 1 --2- 172.21.15.86:0/3866316959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a9819d70 0x5595a98c1190 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7efca4037040 tx=0x7efca402e070 comp rx=0 tx=0).stop 2024-04-20T21:24:34.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.208+0000 7efcbc3e2740 1 -- 172.21.15.86:0/3866316959 shutdown_connections 2024-04-20T21:24:34.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.208+0000 7efcbc3e2740 1 --2- 172.21.15.86:0/3866316959 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x5595a9843480 0x5595a9872a70 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.208+0000 7efcbc3e2740 1 --2- 172.21.15.86:0/3866316959 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5595a984c200 0x5595a984dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.208+0000 7efcbc3e2740 1 --2- 172.21.15.86:0/3866316959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5595a98461f0 0x5595a984d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.208+0000 7efc97400640 0 client.7260 ms_handle_reset on v2:172.21.15.86:6835/2254117576 2024-04-20T21:24:34.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.208+0000 7efcbc3e2740 1 --2- 172.21.15.86:0/3866316959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595a9819d70 0x5595a98c1190 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.208+0000 7efcbc3e2740 1 -- 172.21.15.86:0/3866316959 >> 172.21.15.86:0/3866316959 conn(0x5595a9816210 msgr2=0x5595a9842d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:34.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.208+0000 7efcbc3e2740 1 -- 172.21.15.86:0/3866316959 shutdown_connections 2024-04-20T21:24:34.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.208+0000 7efcbc3e2740 1 -- 172.21.15.86:0/3866316959 wait complete. 2024-04-20T21:24:34.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.208+0000 7efcbc3e2740 20 client.7260 trim_cache size 0 max 16384 2024-04-20T21:24:34.211 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:24:34.211 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-20T21:24:34.212 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-20T21:24:34.212 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:24:34.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.209+0000 7efcb9600640 1 -- 172.21.15.86:0/4058520715 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efc9006c8f0 msgr2=0x7efc9006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:34.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.209+0000 7efcb9600640 1 --2- 172.21.15.86:0/4058520715 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efc9006c8f0 0x7efc9006edb0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7efca4036000 tx=0x7efca4030000 comp rx=0 tx=0).stop 2024-04-20T21:24:34.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.209+0000 7efcb9600640 1 -- 172.21.15.86:0/4058520715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcb415d190 msgr2=0x7efcb4149c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:34.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.209+0000 7efcb9600640 1 --2- 172.21.15.86:0/4058520715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcb415d190 0x7efcb4149c70 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7efcb40b0010 tx=0x7efcac00dc00 comp rx=0 tx=0).stop 2024-04-20T21:24:34.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.209+0000 7efcb9600640 1 -- 172.21.15.86:0/4058520715 shutdown_connections 2024-04-20T21:24:34.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.209+0000 7efcb9600640 1 --2- 172.21.15.86:0/4058520715 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efc9006c8f0 0x7efc9006edb0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.209+0000 7efcb9600640 1 --2- 172.21.15.86:0/4058520715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efcb415d190 0x7efcb4149c70 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.209+0000 7efcb9600640 1 --2- 172.21.15.86:0/4058520715 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efcb4150100 0x7efcb41f42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.209+0000 7efcb9600640 1 --2- 172.21.15.86:0/4058520715 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc980039e0 0x7efcb4158050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.209+0000 7efcb9600640 1 -- 172.21.15.86:0/4058520715 >> 172.21.15.86:0/4058520715 conn(0x7efcb40b2110 msgr2=0x7efcb40d0420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:34.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.209+0000 7efcb9600640 1 -- 172.21.15.86:0/4058520715 shutdown_connections 2024-04-20T21:24:34.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.209+0000 7efcb9600640 1 -- 172.21.15.86:0/4058520715 wait complete. 2024-04-20T21:24:34.224 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T21:24:34.225 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:24:34.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.424+0000 7f108fe00640 1 Processor -- start 2024-04-20T21:24:34.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.425+0000 7f108fe00640 1 -- start start 2024-04-20T21:24:34.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.427+0000 7f108fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f108806e6d0 0x7f1088071d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:34.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.427+0000 7f108fe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1088072240 0x7f108815cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:34.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.427+0000 7f108fe00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f108806eb30 con 0x7f1088146130 2024-04-20T21:24:34.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.427+0000 7f108fe00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f108806dc30 con 0x7f108806e6d0 2024-04-20T21:24:34.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.427+0000 7f108fe00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f10880cd8e0 con 0x7f1088072240 2024-04-20T21:24:34.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.428+0000 7f108ca00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f108806e6d0 0x7f1088071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:34.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.428+0000 7f108ca00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f108806e6d0 0x7f1088071d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45854/0 (socket says 172.21.15.86:45854) 2024-04-20T21:24:34.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.428+0000 7f108ca00640 1 -- 172.21.15.86:0/3902606449 learned_addr learned my addr 172.21.15.86:0/3902606449 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:34.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.428+0000 7f108de00640 1 --2- 172.21.15.86:0/3902606449 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1088072240 0x7f108815cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:34.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.428+0000 7f108ca00640 1 -- 172.21.15.86:0/3902606449 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1088072240 msgr2=0x7f108815cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:34.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.428+0000 7f108ca00640 1 --2- 172.21.15.86:0/3902606449 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1088072240 0x7f108815cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.428+0000 7f108ca00640 1 -- 172.21.15.86:0/3902606449 >> v1:172.21.15.144:6789/0 conn(0x7f1088146130 legacy=0x7f10880ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:34.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.428+0000 7f108ca00640 1 -- 172.21.15.86:0/3902606449 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10880cdbc0 con 0x7f108806e6d0 2024-04-20T21:24:34.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.429+0000 7f108ca00640 1 --2- 172.21.15.86:0/3902606449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f108806e6d0 0x7f1088071d00 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f1074009870 tx=0x7f107402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4663b55edea3179d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:34.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.430+0000 7f1087e00640 1 -- 172.21.15.86:0/3902606449 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1074034070 con 0x7f108806e6d0 2024-04-20T21:24:34.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.430+0000 7f1087e00640 1 -- 172.21.15.86:0/3902606449 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1074030840 con 0x7f108806e6d0 2024-04-20T21:24:34.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.430+0000 7f1087e00640 1 -- 172.21.15.86:0/3902606449 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1074030c10 con 0x7f108806e6d0 2024-04-20T21:24:34.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.430+0000 7f108fe00640 1 -- 172.21.15.86:0/3902606449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f108806e6d0 msgr2=0x7f1088071d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:34.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.430+0000 7f108fe00640 1 --2- 172.21.15.86:0/3902606449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f108806e6d0 0x7f1088071d00 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f1074009870 tx=0x7f107402b980 comp rx=0 tx=0).stop 2024-04-20T21:24:34.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.430+0000 7f108fe00640 1 -- 172.21.15.86:0/3902606449 shutdown_connections 2024-04-20T21:24:34.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.430+0000 7f108fe00640 1 --2- 172.21.15.86:0/3902606449 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1088072240 0x7f108815cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.430+0000 7f108fe00640 1 --2- 172.21.15.86:0/3902606449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f108806e6d0 0x7f1088071d00 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.430+0000 7f108fe00640 1 -- 172.21.15.86:0/3902606449 >> 172.21.15.86:0/3902606449 conn(0x7f10880b2360 msgr2=0x7f10880b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:34.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.430+0000 7f108fe00640 1 -- 172.21.15.86:0/3902606449 shutdown_connections 2024-04-20T21:24:34.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.430+0000 7f108fe00640 1 -- 172.21.15.86:0/3902606449 wait complete. 2024-04-20T21:24:34.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.432+0000 7f108fe00640 1 Processor -- start 2024-04-20T21:24:34.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.432+0000 7f108fe00640 1 -- start start 2024-04-20T21:24:34.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.433+0000 7f108fe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f108806e6d0 0x7f108813d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:34.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.433+0000 7f108fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1088072240 0x7f108813ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:34.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.433+0000 7f108fe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1088146130 0x7f1088139090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:34.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.433+0000 7f108fe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f108806f150 con 0x7f1088072240 2024-04-20T21:24:34.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.433+0000 7f108fe00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f108806e1b0 con 0x7f108806e6d0 2024-04-20T21:24:34.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.433+0000 7f108fe00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f10880cd4e0 con 0x7f1088146130 2024-04-20T21:24:34.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.433+0000 7f108ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1088072240 0x7f108813ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:34.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.434+0000 7f108ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1088072240 0x7f108813ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45860/0 (socket says 172.21.15.86:45860) 2024-04-20T21:24:34.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.434+0000 7f108ca00640 1 -- 172.21.15.86:0/2640966042 learned_addr learned my addr 172.21.15.86:0/2640966042 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:34.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.434+0000 7f108de00640 1 --2- 172.21.15.86:0/2640966042 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1088146130 0x7f1088139090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:34.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.434+0000 7f108d400640 1 --2- 172.21.15.86:0/2640966042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f108806e6d0 0x7f108813d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:34.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.434+0000 7f108ca00640 1 -- 172.21.15.86:0/2640966042 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1088146130 msgr2=0x7f1088139090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:34.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.434+0000 7f108ca00640 1 --2- 172.21.15.86:0/2640966042 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1088146130 0x7f1088139090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.434+0000 7f108ca00640 1 -- 172.21.15.86:0/2640966042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f108806e6d0 msgr2=0x7f108813d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:34.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.434+0000 7f108ca00640 1 --2- 172.21.15.86:0/2640966042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f108806e6d0 0x7f108813d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.434+0000 7f108ca00640 1 -- 172.21.15.86:0/2640966042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10881395d0 con 0x7f1088072240 2024-04-20T21:24:34.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.434+0000 7f108d400640 1 --2- 172.21.15.86:0/2640966042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f108806e6d0 0x7f108813d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:24:34.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.434+0000 7f108de00640 1 --2- 172.21.15.86:0/2640966042 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1088146130 0x7f1088139090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:24:34.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.434+0000 7f108ca00640 1 --2- 172.21.15.86:0/2640966042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1088072240 0x7f108813ec10 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f1074009870 tx=0x7f1074004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:34.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.435+0000 7f1085600640 1 -- 172.21.15.86:0/2640966042 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1074048070 con 0x7f1088072240 2024-04-20T21:24:34.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.435+0000 7f1085600640 1 -- 172.21.15.86:0/2640966042 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1074030640 con 0x7f1088072240 2024-04-20T21:24:34.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.435+0000 7f1085600640 1 -- 172.21.15.86:0/2640966042 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1074034040 con 0x7f1088072240 2024-04-20T21:24:34.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.435+0000 7f108fe00640 1 -- 172.21.15.86:0/2640966042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f108813db70 con 0x7f1088072240 2024-04-20T21:24:34.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.435+0000 7f108fe00640 1 -- 172.21.15.86:0/2640966042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f108813e130 con 0x7f1088072240 2024-04-20T21:24:34.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.436+0000 7f108fe00640 1 -- 172.21.15.86:0/2640966042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10880689e0 con 0x7f1088072240 2024-04-20T21:24:34.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.437+0000 7f1085600640 1 -- 172.21.15.86:0/2640966042 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f107404b070 con 0x7f1088072240 2024-04-20T21:24:34.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.438+0000 7f1085600640 1 --2- 172.21.15.86:0/2640966042 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f106806c8f0 0x7f106806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:34.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.438+0000 7f1085600640 1 -- 172.21.15.86:0/2640966042 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5882+0+0 (secure 0 0 0) 0x7f10740adb00 con 0x7f1088072240 2024-04-20T21:24:34.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.438+0000 7f108d400640 1 --2- 172.21.15.86:0/2640966042 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f106806c8f0 0x7f106806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:34.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.439+0000 7f108d400640 1 --2- 172.21.15.86:0/2640966042 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f106806c8f0 0x7f106806edb0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f107c00a830 tx=0x7f107c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:34.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.443+0000 7f1085600640 1 -- 172.21.15.86:0/2640966042 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f108806eb30 con 0x7f1088072240 2024-04-20T21:24:34.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.685+0000 7f108fe00640 1 -- 172.21.15.86:0/2640966042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f108815ff50 con 0x7f1088072240 2024-04-20T21:24:34.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.686+0000 7f1085600640 1 -- 172.21.15.86:0/2640966042 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+5206 (secure 0 0 0) 0x7f107407be70 con 0x7f1088072240 2024-04-20T21:24:34.686 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:24:34.686 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":50,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2740872549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2740872549},{"type":"v1","addr":"172.21.15.86:6836","nonce":2740872549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5048,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/301899138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":301899138},{"type":"v1","addr":"172.21.15.144:6833","nonce":301899138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5051,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2816343379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2816343379},{"type":"v1","addr":"172.21.15.144:6835","nonce":2816343379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:22:45.431100+0000","modified":"2024-04-20T21:24:11.116928+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6987},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6987":{"gid":6987,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":18,"addr":"172.21.15.86:6837/2254117576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2254117576},{"type":"v1","addr":"172.21.15.86:6837","nonce":2254117576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6987,"qdb_cluster":[6987]},"id":5}]} 2024-04-20T21:24:34.686 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 50 2024-04-20T21:24:34.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.689+0000 7f108fe00640 1 -- 172.21.15.86:0/2640966042 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f106806c8f0 msgr2=0x7f106806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:34.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.689+0000 7f108fe00640 1 --2- 172.21.15.86:0/2640966042 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f106806c8f0 0x7f106806edb0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f107c00a830 tx=0x7f107c008040 comp rx=0 tx=0).stop 2024-04-20T21:24:34.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.689+0000 7f108fe00640 1 -- 172.21.15.86:0/2640966042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1088072240 msgr2=0x7f108813ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:34.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.689+0000 7f108fe00640 1 --2- 172.21.15.86:0/2640966042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1088072240 0x7f108813ec10 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f1074009870 tx=0x7f1074004730 comp rx=0 tx=0).stop 2024-04-20T21:24:34.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.689+0000 7f108fe00640 1 -- 172.21.15.86:0/2640966042 shutdown_connections 2024-04-20T21:24:34.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.689+0000 7f108fe00640 1 --2- 172.21.15.86:0/2640966042 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f106806c8f0 0x7f106806edb0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.689+0000 7f108fe00640 1 --2- 172.21.15.86:0/2640966042 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1088146130 0x7f1088139090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.689+0000 7f108fe00640 1 --2- 172.21.15.86:0/2640966042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1088072240 0x7f108813ec10 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.689+0000 7f108fe00640 1 --2- 172.21.15.86:0/2640966042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f108806e6d0 0x7f108813d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.689+0000 7f108fe00640 1 -- 172.21.15.86:0/2640966042 >> 172.21.15.86:0/2640966042 conn(0x7f10880b2360 msgr2=0x7f1088144370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:34.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.689+0000 7f108fe00640 1 -- 172.21.15.86:0/2640966042 shutdown_connections 2024-04-20T21:24:34.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.689+0000 7f108fe00640 1 -- 172.21.15.86:0/2640966042 wait complete. 2024-04-20T21:24:34.705 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-04-20T21:24:34.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.907+0000 7f2094c00640 1 Processor -- start 2024-04-20T21:24:34.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.907+0000 7f2094c00640 1 -- start start 2024-04-20T21:24:34.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.909+0000 7f2094c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2090146210 0x7f209006e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:34.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.909+0000 7f2094c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f209006bfd0 0x7f20900cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:34.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.909+0000 7f2094c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f209015cd40 con 0x7f209006bbf0 2024-04-20T21:24:34.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.909+0000 7f2094c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f209015d000 con 0x7f2090146210 2024-04-20T21:24:34.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.909+0000 7f2094c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f209015d2e0 con 0x7f209006bfd0 2024-04-20T21:24:34.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.909+0000 7f208d400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2090146210 0x7f209006e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:34.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.909+0000 7f208d400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2090146210 0x7f209006e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45872/0 (socket says 172.21.15.86:45872) 2024-04-20T21:24:34.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.909+0000 7f208d400640 1 -- 172.21.15.86:0/3269601199 learned_addr learned my addr 172.21.15.86:0/3269601199 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:34.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.909+0000 7f208d400640 1 -- 172.21.15.86:0/3269601199 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f209006bfd0 msgr2=0x7f20900cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:34.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.909+0000 7f208d400640 1 --2- 172.21.15.86:0/3269601199 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f209006bfd0 0x7f20900cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.909+0000 7f208d400640 1 -- 172.21.15.86:0/3269601199 >> v1:172.21.15.86:6789/0 conn(0x7f209006bbf0 legacy=0x7f2090145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:34.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.909+0000 7f208d400640 1 -- 172.21.15.86:0/3269601199 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f209015d5c0 con 0x7f2090146210 2024-04-20T21:24:34.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.911+0000 7f208d400640 1 --2- 172.21.15.86:0/3269601199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2090146210 0x7f209006e000 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f2078009e50 tx=0x7f207802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60928a85673fc4bf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:34.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.911+0000 7f208ca00640 1 -- 172.21.15.86:0/3269601199 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207802cb00 con 0x7f2090146210 2024-04-20T21:24:34.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.911+0000 7f208ca00640 1 -- 172.21.15.86:0/3269601199 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f207802cca0 con 0x7f2090146210 2024-04-20T21:24:34.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.911+0000 7f208ca00640 1 -- 172.21.15.86:0/3269601199 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2078034610 con 0x7f2090146210 2024-04-20T21:24:34.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.911+0000 7f2094c00640 1 -- 172.21.15.86:0/3269601199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2090146210 msgr2=0x7f209006e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:34.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.911+0000 7f2094c00640 1 --2- 172.21.15.86:0/3269601199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2090146210 0x7f209006e000 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f2078009e50 tx=0x7f207802bf60 comp rx=0 tx=0).stop 2024-04-20T21:24:34.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.912+0000 7f2094c00640 1 -- 172.21.15.86:0/3269601199 shutdown_connections 2024-04-20T21:24:34.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.912+0000 7f2094c00640 1 --2- 172.21.15.86:0/3269601199 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f209006bfd0 0x7f20900cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.912+0000 7f2094c00640 1 --2- 172.21.15.86:0/3269601199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2090146210 0x7f209006e000 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.912+0000 7f2094c00640 1 -- 172.21.15.86:0/3269601199 >> 172.21.15.86:0/3269601199 conn(0x7f20900b2110 msgr2=0x7f20900b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:34.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.912+0000 7f2094c00640 1 -- 172.21.15.86:0/3269601199 shutdown_connections 2024-04-20T21:24:34.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.912+0000 7f2094c00640 1 -- 172.21.15.86:0/3269601199 wait complete. 2024-04-20T21:24:34.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.914+0000 7f2094c00640 1 Processor -- start 2024-04-20T21:24:34.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.914+0000 7f2094c00640 1 -- start start 2024-04-20T21:24:34.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.915+0000 7f2094c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f209006bbf0 0x7f20900b4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:34.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.915+0000 7f2094c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f209006bfd0 0x7f20900b4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:34.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.915+0000 7f2094c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2090146210 0x7f20901e5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:34.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.915+0000 7f2094c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f20900cdc00 con 0x7f209006bfd0 2024-04-20T21:24:34.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.915+0000 7f2094c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f20900cda80 con 0x7f209006bbf0 2024-04-20T21:24:34.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.915+0000 7f2094c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f20900cdd80 con 0x7f2090146210 2024-04-20T21:24:34.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.915+0000 7f208d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f209006bfd0 0x7f20900b4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:34.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.915+0000 7f208d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f209006bfd0 0x7f20900b4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45878/0 (socket says 172.21.15.86:45878) 2024-04-20T21:24:34.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.915+0000 7f208d400640 1 -- 172.21.15.86:0/3000551485 learned_addr learned my addr 172.21.15.86:0/3000551485 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:34.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.915+0000 7f208d400640 1 -- 172.21.15.86:0/3000551485 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2090146210 msgr2=0x7f20901e5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:34.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.915+0000 7f208d400640 1 --2- 172.21.15.86:0/3000551485 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2090146210 0x7f20901e5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.915+0000 7f208d400640 1 -- 172.21.15.86:0/3000551485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f209006bbf0 msgr2=0x7f20900b4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:34.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.915+0000 7f208d400640 1 --2- 172.21.15.86:0/3000551485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f209006bbf0 0x7f20900b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.915+0000 7f208d400640 1 -- 172.21.15.86:0/3000551485 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20901e5ec0 con 0x7f209006bfd0 2024-04-20T21:24:34.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.916+0000 7f208d400640 1 --2- 172.21.15.86:0/3000551485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f209006bfd0 0x7f20900b4bd0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f2078009e50 tx=0x7f2078004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:34.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.916+0000 7f2086000640 1 -- 172.21.15.86:0/3000551485 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2078044020 con 0x7f209006bfd0 2024-04-20T21:24:34.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.916+0000 7f2086000640 1 -- 172.21.15.86:0/3000551485 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f207802ce50 con 0x7f209006bfd0 2024-04-20T21:24:34.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.916+0000 7f2086000640 1 -- 172.21.15.86:0/3000551485 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20780408f0 con 0x7f209006bfd0 2024-04-20T21:24:34.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.916+0000 7f2094c00640 1 -- 172.21.15.86:0/3000551485 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20901320d0 con 0x7f209006bfd0 2024-04-20T21:24:34.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.916+0000 7f2094c00640 1 -- 172.21.15.86:0/3000551485 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2090132690 con 0x7f209006bfd0 2024-04-20T21:24:34.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.918+0000 7f2086000640 1 -- 172.21.15.86:0/3000551485 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2078047020 con 0x7f209006bfd0 2024-04-20T21:24:34.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.920+0000 7f2086000640 1 --2- 172.21.15.86:0/3000551485 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f206c06c8f0 0x7f206c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:34.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.920+0000 7f2086000640 1 -- 172.21.15.86:0/3000551485 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5882+0+0 (secure 0 0 0) 0x7f20780ae6b0 con 0x7f209006bfd0 2024-04-20T21:24:34.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.920+0000 7f208de00640 1 --2- 172.21.15.86:0/3000551485 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f206c06c8f0 0x7f206c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:34.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.920+0000 7f20979da740 1 Processor -- start 2024-04-20T21:24:34.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.921+0000 7f208de00640 1 --2- 172.21.15.86:0/3000551485 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f206c06c8f0 0x7f206c06edb0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f209006f6c0 tx=0x7f2088007970 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:34.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.921+0000 7f20979da740 1 -- start start 2024-04-20T21:24:34.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.921+0000 7f20979da740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651bc71ceb0 0x5651bc6efe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:34.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.921+0000 7f20979da740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651bc71d290 0x5651bc71fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:34.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.921+0000 7f20979da740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5651bc6f0390 0x5651bc720180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:34.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.921+0000 7f20979da740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5651bc65b060 con 0x5651bc71ceb0 2024-04-20T21:24:34.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.921+0000 7f20979da740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5651bc600f00 con 0x5651bc71d290 2024-04-20T21:24:34.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.921+0000 7f20979da740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5651bc657020 con 0x5651bc6f0390 2024-04-20T21:24:34.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.922+0000 7f208de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651bc71ceb0 0x5651bc6efe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:34.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.922+0000 7f208de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651bc71ceb0 0x5651bc6efe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45886/0 (socket says 172.21.15.86:45886) 2024-04-20T21:24:34.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.922+0000 7f208de00640 1 -- 172.21.15.86:0/1867519295 learned_addr learned my addr 172.21.15.86:0/1867519295 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:34.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.922+0000 7f208de00640 1 -- 172.21.15.86:0/1867519295 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5651bc6f0390 msgr2=0x5651bc720180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:34.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.922+0000 7f208de00640 1 --2- 172.21.15.86:0/1867519295 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5651bc6f0390 0x5651bc720180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.922+0000 7f208de00640 1 -- 172.21.15.86:0/1867519295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651bc71d290 msgr2=0x5651bc71fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:34.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.922+0000 7f208de00640 1 --2- 172.21.15.86:0/1867519295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651bc71d290 0x5651bc71fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.922+0000 7f208de00640 1 -- 172.21.15.86:0/1867519295 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651bc5e9a30 con 0x5651bc71ceb0 2024-04-20T21:24:34.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.922+0000 7f208de00640 1 --2- 172.21.15.86:0/1867519295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651bc71ceb0 0x5651bc6efe50 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f2088005560 tx=0x7f208800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beacc56b06f9e45a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:34.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.922+0000 7f207e600640 1 -- 172.21.15.86:0/1867519295 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2088018900 con 0x5651bc71ceb0 2024-04-20T21:24:34.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.923+0000 7f207e600640 1 -- 172.21.15.86:0/1867519295 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2088018aa0 con 0x5651bc71ceb0 2024-04-20T21:24:34.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.923+0000 7f207e600640 1 -- 172.21.15.86:0/1867519295 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2088018e50 con 0x5651bc71ceb0 2024-04-20T21:24:34.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.923+0000 7f20979da740 1 -- 172.21.15.86:0/1867519295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651bc71ceb0 msgr2=0x5651bc6efe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:34.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.923+0000 7f20979da740 1 --2- 172.21.15.86:0/1867519295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651bc71ceb0 0x5651bc6efe50 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f2088005560 tx=0x7f208800f000 comp rx=0 tx=0).stop 2024-04-20T21:24:34.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.923+0000 7f20979da740 1 -- 172.21.15.86:0/1867519295 shutdown_connections 2024-04-20T21:24:34.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.923+0000 7f20979da740 1 --2- 172.21.15.86:0/1867519295 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5651bc6f0390 0x5651bc720180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.923+0000 7f20979da740 1 --2- 172.21.15.86:0/1867519295 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651bc71d290 0x5651bc71fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.923+0000 7f20979da740 1 --2- 172.21.15.86:0/1867519295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651bc71ceb0 0x5651bc6efe50 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.923+0000 7f20979da740 1 -- 172.21.15.86:0/1867519295 >> 172.21.15.86:0/1867519295 conn(0x5651bc6f20c0 msgr2=0x5651bc73ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:34.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.923+0000 7f20979da740 1 -- 172.21.15.86:0/1867519295 shutdown_connections 2024-04-20T21:24:34.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.923+0000 7f20979da740 1 -- 172.21.15.86:0/1867519295 wait complete. 2024-04-20T21:24:34.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.924+0000 7f20979da740 1 -- start start 2024-04-20T21:24:34.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.924+0000 7f20979da740 1 Processor -- start 2024-04-20T21:24:34.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.925+0000 7f20979da740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5651bc71ceb0 0x5651bc70cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:34.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.925+0000 7f20979da740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651bc71d290 0x5651bc70c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:34.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.925+0000 7f20979da740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651bc7232a0 0x5651bc723720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:34.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.925+0000 7f20979da740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5651bc6f4660 con 0x5651bc7232a0 2024-04-20T21:24:34.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.925+0000 7f20979da740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5651bc6143d0 con 0x5651bc71d290 2024-04-20T21:24:34.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.925+0000 7f20979da740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5651bc6f3a50 con 0x5651bc71ceb0 2024-04-20T21:24:34.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.925+0000 7f20979da740 10 client.0 _ll_register_callbacks cb 0x5651bc5e2960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:24:34.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.925+0000 7f208e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651bc7232a0 0x5651bc723720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:34.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.925+0000 7f208e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651bc7232a0 0x5651bc723720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45888/0 (socket says 172.21.15.86:45888) 2024-04-20T21:24:34.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.925+0000 7f208e800640 1 -- 172.21.15.86:0/150934549 learned_addr learned my addr 172.21.15.86:0/150934549 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:34.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.926+0000 7f208d400640 1 --2- 172.21.15.86:0/150934549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651bc71d290 0x5651bc70c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:34.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.926+0000 7f208de00640 1 --2- 172.21.15.86:0/150934549 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5651bc71ceb0 0x5651bc70cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:34.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.926+0000 7f208e800640 1 -- 172.21.15.86:0/150934549 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5651bc71ceb0 msgr2=0x5651bc70cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:34.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.926+0000 7f208e800640 1 --2- 172.21.15.86:0/150934549 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5651bc71ceb0 0x5651bc70cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.926+0000 7f208e800640 1 -- 172.21.15.86:0/150934549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651bc71d290 msgr2=0x5651bc70c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:34.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.926+0000 7f208e800640 1 --2- 172.21.15.86:0/150934549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651bc71d290 0x5651bc70c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.926+0000 7f208e800640 1 -- 172.21.15.86:0/150934549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651bc4397c0 con 0x5651bc7232a0 2024-04-20T21:24:34.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.926+0000 7f208e800640 1 --2- 172.21.15.86:0/150934549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651bc7232a0 0x5651bc723720 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f2080009e00 tx=0x7f208000b4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce2b01b41a50fdff server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:34.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.926+0000 7f2072a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:24:34.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.926+0000 7f2072a00640 1 -- 172.21.15.86:0/150934549 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2080015020 con 0x5651bc7232a0 2024-04-20T21:24:34.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.926+0000 7f2072a00640 1 -- 172.21.15.86:0/150934549 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(74..74 src has 1..74) ==== 5882+0+0 (secure 0 0 0) 0x7f2080011420 con 0x5651bc7232a0 2024-04-20T21:24:34.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.926+0000 7f20979da740 1 -- 172.21.15.86:0/150934549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2088018aa0 con 0x5651bc7232a0 2024-04-20T21:24:34.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.927+0000 7f2072a00640 1 -- 172.21.15.86:0/150934549 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f208001f050 con 0x5651bc7232a0 2024-04-20T21:24:34.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.927+0000 7f20979da740 10 client.7281 fetch_fsmap learned FSMap version 50 2024-04-20T21:24:34.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.927+0000 7f20979da740 1 -- 172.21.15.86:0/150934549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x5651bc499f50 con 0x5651bc7232a0 2024-04-20T21:24:34.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.927+0000 7f2072a00640 1 -- 172.21.15.86:0/150934549 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2088018aa0 con 0x5651bc7232a0 2024-04-20T21:24:34.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.927+0000 7f2072a00640 1 -- 172.21.15.86:0/150934549 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 50) ==== 3495+0+0 (secure 0 0 0) 0x7f2080022020 con 0x5651bc7232a0 2024-04-20T21:24:34.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.927+0000 7f20979da740 10 client.7281 fetch_fsmap finished waiting for FSMap version 50 2024-04-20T21:24:34.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.927+0000 7f20979da740 10 client.7281 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-20T21:24:34.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.927+0000 7f20979da740 20 client.7281 populate_metadata read hostname 'smithi086' 2024-04-20T21:24:34.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.927+0000 7f20979da740 1 --2- 172.21.15.86:0/150934549 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x5651bc721350 0x5651bc7594d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:34.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.927+0000 7f20979da740 4 client.7281 mds_command: new command op to 6987 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:24:34.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.927+0000 7f20979da740 1 -- 172.21.15.86:0/150934549 --> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651bc668a80 con 0x5651bc721350 2024-04-20T21:24:34.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.927+0000 7f208de00640 1 --2- 172.21.15.86:0/150934549 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x5651bc721350 0x5651bc7594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:34.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.928+0000 7f208de00640 1 --2- 172.21.15.86:0/150934549 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x5651bc721350 0x5651bc7594d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37e1f614483e5c0c server_cookie=52c4d850a7991444 in_seq=0 out_seq=0 2024-04-20T21:24:34.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.928+0000 7f2072a00640 10 client.7281 ms_handle_connect on v2:172.21.15.86:6835/2254117576 2024-04-20T21:24:34.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.929+0000 7f2072a00640 1 -- 172.21.15.86:0/150934549 <== mds.0 v2:172.21.15.86:6835/2254117576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5651bc668a80 con 0x5651bc721350 2024-04-20T21:24:34.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.929+0000 7f2072a00640 10 client.7281 handle_command_reply: tid=0 2024-04-20T21:24:34.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.929+0000 7f20979da740 1 client.7281 shutdown 2024-04-20T21:24:34.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.929+0000 7f20979da740 1 -- 172.21.15.86:0/150934549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651bc7232a0 msgr2=0x5651bc723720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:34.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.929+0000 7f20979da740 1 --2- 172.21.15.86:0/150934549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651bc7232a0 0x5651bc723720 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f2080009e00 tx=0x7f208000b4b0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.929+0000 7f20979da740 1 -- 172.21.15.86:0/150934549 shutdown_connections 2024-04-20T21:24:34.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.929+0000 7f20979da740 1 --2- 172.21.15.86:0/150934549 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x5651bc721350 0x5651bc7594d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.929+0000 7f20979da740 1 --2- 172.21.15.86:0/150934549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651bc7232a0 0x5651bc723720 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.929+0000 7f20979da740 1 --2- 172.21.15.86:0/150934549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651bc71d290 0x5651bc70c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.929+0000 7f20979da740 1 --2- 172.21.15.86:0/150934549 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5651bc71ceb0 0x5651bc70cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.929+0000 7f2072a00640 0 client.7281 ms_handle_reset on v2:172.21.15.86:6835/2254117576 2024-04-20T21:24:34.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.929+0000 7f20979da740 1 -- 172.21.15.86:0/150934549 >> 172.21.15.86:0/150934549 conn(0x5651bc6f0390 msgr2=0x5651bc734aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:34.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.929+0000 7f20979da740 1 -- 172.21.15.86:0/150934549 shutdown_connections 2024-04-20T21:24:34.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.929+0000 7f20979da740 1 -- 172.21.15.86:0/150934549 wait complete. 2024-04-20T21:24:34.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.929+0000 7f20979da740 20 client.7281 trim_cache size 0 max 16384 2024-04-20T21:24:34.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.943+0000 7f20979da740 1 Processor -- start 2024-04-20T21:24:34.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.944+0000 7f20979da740 1 -- start start 2024-04-20T21:24:34.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.944+0000 7f20979da740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651bc71ceb0 0x5651bc73ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:34.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.944+0000 7f20979da740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651bc71d290 0x5651bc6eefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:34.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.944+0000 7f20979da740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5651bc7232a0 0x5651bc6ef4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:34.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.944+0000 7f20979da740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2090132e90 con 0x5651bc71ceb0 2024-04-20T21:24:34.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.944+0000 7f20979da740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2090134330 con 0x5651bc71d290 2024-04-20T21:24:34.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.944+0000 7f20979da740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2090134980 con 0x5651bc7232a0 2024-04-20T21:24:34.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.945+0000 7f208de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651bc71ceb0 0x5651bc73ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:34.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.945+0000 7f208de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651bc71ceb0 0x5651bc73ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45900/0 (socket says 172.21.15.86:45900) 2024-04-20T21:24:34.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.945+0000 7f208de00640 1 -- 172.21.15.86:0/2086181182 learned_addr learned my addr 172.21.15.86:0/2086181182 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:34.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.945+0000 7f208e800640 1 --2- 172.21.15.86:0/2086181182 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5651bc7232a0 0x5651bc6ef4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:34.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.945+0000 7f208de00640 1 -- 172.21.15.86:0/2086181182 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5651bc7232a0 msgr2=0x5651bc6ef4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:34.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.945+0000 7f208de00640 1 --2- 172.21.15.86:0/2086181182 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5651bc7232a0 0x5651bc6ef4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.945+0000 7f208de00640 1 -- 172.21.15.86:0/2086181182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651bc71d290 msgr2=0x5651bc6eefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:34.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.945+0000 7f208de00640 1 --2- 172.21.15.86:0/2086181182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651bc71d290 0x5651bc6eefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.945+0000 7f208de00640 1 -- 172.21.15.86:0/2086181182 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651bc32d0b0 con 0x5651bc71ceb0 2024-04-20T21:24:34.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.946+0000 7f208de00640 1 --2- 172.21.15.86:0/2086181182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651bc71ceb0 0x5651bc73ad30 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f2088000f60 tx=0x7f2088005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ed3fbb8a31838d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:34.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.946+0000 7f207dc00640 1 -- 172.21.15.86:0/2086181182 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2088034070 con 0x5651bc71ceb0 2024-04-20T21:24:34.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.946+0000 7f207dc00640 1 -- 172.21.15.86:0/2086181182 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2088018a00 con 0x5651bc71ceb0 2024-04-20T21:24:34.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.946+0000 7f207dc00640 1 -- 172.21.15.86:0/2086181182 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20880069f0 con 0x5651bc71ceb0 2024-04-20T21:24:34.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.946+0000 7f20979da740 1 -- 172.21.15.86:0/2086181182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651bc71ceb0 msgr2=0x5651bc73ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:34.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.946+0000 7f20979da740 1 --2- 172.21.15.86:0/2086181182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651bc71ceb0 0x5651bc73ad30 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f2088000f60 tx=0x7f2088005000 comp rx=0 tx=0).stop 2024-04-20T21:24:34.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.946+0000 7f20979da740 1 -- 172.21.15.86:0/2086181182 shutdown_connections 2024-04-20T21:24:34.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.946+0000 7f20979da740 1 --2- 172.21.15.86:0/2086181182 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5651bc7232a0 0x5651bc6ef4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.946+0000 7f20979da740 1 --2- 172.21.15.86:0/2086181182 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651bc71d290 0x5651bc6eefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.947+0000 7f20979da740 1 --2- 172.21.15.86:0/2086181182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651bc71ceb0 0x5651bc73ad30 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.947+0000 7f20979da740 1 -- 172.21.15.86:0/2086181182 >> 172.21.15.86:0/2086181182 conn(0x5651bc6f0390 msgr2=0x5651bc6eb6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:34.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.947+0000 7f20979da740 1 -- 172.21.15.86:0/2086181182 shutdown_connections 2024-04-20T21:24:34.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.947+0000 7f20979da740 1 -- 172.21.15.86:0/2086181182 wait complete. 2024-04-20T21:24:34.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.947+0000 7f20979da740 1 -- start start 2024-04-20T21:24:34.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.947+0000 7f20979da740 1 Processor -- start 2024-04-20T21:24:34.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.948+0000 7f20979da740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651bc71ceb0 0x5651bc6f3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:34.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.948+0000 7f20979da740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5651bc71d290 0x5651bc7b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:34.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.948+0000 7f20979da740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651bc7232a0 0x5651bc73d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:34.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.948+0000 7f20979da740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5651bc722d80 con 0x5651bc71ceb0 2024-04-20T21:24:34.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.948+0000 7f20979da740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5651bc7227c0 con 0x5651bc7232a0 2024-04-20T21:24:34.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.948+0000 7f20979da740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5651bc732db0 con 0x5651bc71d290 2024-04-20T21:24:34.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.948+0000 7f20979da740 10 client.0 _ll_register_callbacks cb 0x7f2080009960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:24:34.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.949+0000 7f208de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651bc71ceb0 0x5651bc6f3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:34.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.949+0000 7f208de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651bc71ceb0 0x5651bc6f3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45904/0 (socket says 172.21.15.86:45904) 2024-04-20T21:24:34.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.949+0000 7f208de00640 1 -- 172.21.15.86:0/2327625841 learned_addr learned my addr 172.21.15.86:0/2327625841 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:34.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.949+0000 7f208de00640 1 -- 172.21.15.86:0/2327625841 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5651bc71d290 msgr2=0x5651bc7b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:34.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.949+0000 7f208de00640 1 --2- 172.21.15.86:0/2327625841 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5651bc71d290 0x5651bc7b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.949+0000 7f208de00640 1 -- 172.21.15.86:0/2327625841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651bc7232a0 msgr2=0x5651bc73d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:34.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.949+0000 7f208de00640 1 --2- 172.21.15.86:0/2327625841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651bc7232a0 0x5651bc73d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.949+0000 7f208de00640 1 -- 172.21.15.86:0/2327625841 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651bc73dea0 con 0x5651bc71ceb0 2024-04-20T21:24:34.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.949+0000 7f208de00640 1 --2- 172.21.15.86:0/2327625841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651bc71ceb0 0x5651bc6f3ac0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f2088000f60 tx=0x7f2088010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de247f3d63acb4a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:34.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.949+0000 7f2072a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:24:34.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.949+0000 7f2072a00640 1 -- 172.21.15.86:0/2327625841 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2088034040 con 0x5651bc71ceb0 2024-04-20T21:24:34.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.949+0000 7f2072a00640 1 -- 172.21.15.86:0/2327625841 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(74..74 src has 1..74) ==== 5882+0+0 (secure 0 0 0) 0x7f2088018800 con 0x5651bc71ceb0 2024-04-20T21:24:34.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.949+0000 7f20979da740 1 -- 172.21.15.86:0/2327625841 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2088018a00 con 0x5651bc71ceb0 2024-04-20T21:24:34.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.950+0000 7f2072a00640 1 -- 172.21.15.86:0/2327625841 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f20880069e0 con 0x5651bc71ceb0 2024-04-20T21:24:34.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.950+0000 7f20979da740 10 client.7287 fetch_fsmap learned FSMap version 50 2024-04-20T21:24:34.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.950+0000 7f20979da740 1 -- 172.21.15.86:0/2327625841 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) -- 0x5651bc731f00 con 0x5651bc71ceb0 2024-04-20T21:24:34.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.950+0000 7f2072a00640 1 -- 172.21.15.86:0/2327625841 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2088018a00 con 0x5651bc71ceb0 2024-04-20T21:24:34.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.950+0000 7f2072a00640 1 -- 172.21.15.86:0/2327625841 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 50) ==== 3495+0+0 (secure 0 0 0) 0x7f2088006b80 con 0x5651bc71ceb0 2024-04-20T21:24:34.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.950+0000 7f20979da740 10 client.7287 fetch_fsmap finished waiting for FSMap version 50 2024-04-20T21:24:34.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.950+0000 7f20979da740 10 client.7287 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-20T21:24:34.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.950+0000 7f20979da740 20 client.7287 populate_metadata read hostname 'smithi086' 2024-04-20T21:24:34.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.950+0000 7f20979da740 1 --2- 172.21.15.86:0/2327625841 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x5651bc6eefa0 0x5651bc6ef380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:34.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.950+0000 7f20979da740 4 client.7287 mds_command: new command op to 6987 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-20T21:24:34.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.950+0000 7f20979da740 1 -- 172.21.15.86:0/2327625841 --> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x5651bc6628f0 con 0x5651bc6eefa0 2024-04-20T21:24:34.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.950+0000 7f208d400640 1 --2- 172.21.15.86:0/2327625841 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x5651bc6eefa0 0x5651bc6ef380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:34.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.951+0000 7f208d400640 1 --2- 172.21.15.86:0/2327625841 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x5651bc6eefa0 0x5651bc6ef380 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22c5d717ca41da42 server_cookie=3bb6b35d40997214 in_seq=0 out_seq=0 2024-04-20T21:24:34.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.951+0000 7f2072a00640 10 client.7287 ms_handle_connect on v2:172.21.15.86:6835/2254117576 2024-04-20T21:24:34.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.951+0000 7f2072a00640 1 -- 172.21.15.86:0/2327625841 <== mds.0 v2:172.21.15.86:6835/2254117576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5651bc6628f0 con 0x5651bc6eefa0 2024-04-20T21:24:34.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.951+0000 7f2072a00640 10 client.7287 handle_command_reply: tid=0 2024-04-20T21:24:34.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.951+0000 7f20979da740 1 client.7287 shutdown 2024-04-20T21:24:34.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.951+0000 7f20979da740 1 -- 172.21.15.86:0/2327625841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651bc71ceb0 msgr2=0x5651bc6f3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:34.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.951+0000 7f20979da740 1 --2- 172.21.15.86:0/2327625841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651bc71ceb0 0x5651bc6f3ac0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f2088000f60 tx=0x7f2088010000 comp rx=0 tx=0).stop 2024-04-20T21:24:34.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.952+0000 7f20979da740 1 -- 172.21.15.86:0/2327625841 shutdown_connections 2024-04-20T21:24:34.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.952+0000 7f20979da740 1 --2- 172.21.15.86:0/2327625841 >> [v2:172.21.15.86:6835/2254117576,v1:172.21.15.86:6837/2254117576] conn(0x5651bc6eefa0 0x5651bc6ef380 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.952+0000 7f20979da740 1 --2- 172.21.15.86:0/2327625841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651bc7232a0 0x5651bc73d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.952+0000 7f20979da740 1 --2- 172.21.15.86:0/2327625841 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5651bc71d290 0x5651bc7b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.952+0000 7f20979da740 1 --2- 172.21.15.86:0/2327625841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651bc71ceb0 0x5651bc6f3ac0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.952+0000 7f2072a00640 0 client.7287 ms_handle_reset on v2:172.21.15.86:6835/2254117576 2024-04-20T21:24:34.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.952+0000 7f20979da740 1 -- 172.21.15.86:0/2327625841 >> 172.21.15.86:0/2327625841 conn(0x5651bc6f0390 msgr2=0x5651bc70f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:34.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.952+0000 7f20979da740 1 -- 172.21.15.86:0/2327625841 shutdown_connections 2024-04-20T21:24:34.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.952+0000 7f20979da740 1 -- 172.21.15.86:0/2327625841 wait complete. 2024-04-20T21:24:34.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.952+0000 7f20979da740 20 client.7287 trim_cache size 0 max 16384 2024-04-20T21:24:34.961 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:24:34.961 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-20T21:24:34.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.952+0000 7f2094c00640 1 -- 172.21.15.86:0/3000551485 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f206c06c8f0 msgr2=0x7f206c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:34.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.952+0000 7f2094c00640 1 --2- 172.21.15.86:0/3000551485 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f206c06c8f0 0x7f206c06edb0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f209006f6c0 tx=0x7f2088007970 comp rx=0 tx=0).stop 2024-04-20T21:24:34.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.952+0000 7f2094c00640 1 -- 172.21.15.86:0/3000551485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f209006bfd0 msgr2=0x7f20900b4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:34.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.952+0000 7f2094c00640 1 --2- 172.21.15.86:0/3000551485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f209006bfd0 0x7f20900b4bd0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f2078009e50 tx=0x7f2078004a40 comp rx=0 tx=0).stop 2024-04-20T21:24:34.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.953+0000 7f2094c00640 1 -- 172.21.15.86:0/3000551485 shutdown_connections 2024-04-20T21:24:34.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.953+0000 7f2094c00640 1 --2- 172.21.15.86:0/3000551485 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f206c06c8f0 0x7f206c06edb0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.953+0000 7f2094c00640 1 --2- 172.21.15.86:0/3000551485 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2090146210 0x7f20901e5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.953+0000 7f2094c00640 1 --2- 172.21.15.86:0/3000551485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f209006bfd0 0x7f20900b4bd0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.953+0000 7f2094c00640 1 --2- 172.21.15.86:0/3000551485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f209006bbf0 0x7f20900b4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:34.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.953+0000 7f2094c00640 1 -- 172.21.15.86:0/3000551485 >> 172.21.15.86:0/3000551485 conn(0x7f20900b2110 msgr2=0x7f209015f130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:34.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.953+0000 7f2094c00640 1 -- 172.21.15.86:0/3000551485 shutdown_connections 2024-04-20T21:24:34.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:34.953+0000 7f2094c00640 1 -- 172.21.15.86:0/3000551485 wait complete. 2024-04-20T21:24:34.967 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-20T21:24:35.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.142+0000 7f6a1e200640 1 Processor -- start 2024-04-20T21:24:35.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.143+0000 7f6a1e200640 1 -- start start 2024-04-20T21:24:35.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.144+0000 7f6a1e200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6a1815ac10 0x7f6a180b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:35.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.144+0000 7f6a1e200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a1806dcd0 con 0x7f6a1815f210 2024-04-20T21:24:35.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.144+0000 7f6a1e200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a1806dfb0 con 0x7f6a18072ca0 2024-04-20T21:24:35.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.144+0000 7f6a1e200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f6a1806e8e0 con 0x7f6a1815ac10 2024-04-20T21:24:35.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.144+0000 7f6a17400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6a1815ac10 0x7f6a180b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:35.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.144+0000 7f6a0ea00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f6a1815f210 0x7f6a18072590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:48568/0 (socket says 172.21.15.86:48568) 2024-04-20T21:24:35.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.144+0000 7f6a0ea00640 1 -- 172.21.15.86:0/2673785557 learned_addr learned my addr 172.21.15.86:0/2673785557 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:24:35.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.145+0000 7f6a17400640 1 -- 172.21.15.86:0/2673785557 >> v1:172.21.15.144:6789/0 conn(0x7f6a18072ca0 legacy=0x7f6a180cea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:35.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.145+0000 7f6a17400640 1 -- 172.21.15.86:0/2673785557 >> v1:172.21.15.86:6789/0 conn(0x7f6a1815f210 legacy=0x7f6a18072590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:35.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.145+0000 7f6a17400640 1 -- 172.21.15.86:0/2673785557 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a1806e290 con 0x7f6a1815ac10 2024-04-20T21:24:35.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.145+0000 7f6a16a00640 1 -- 172.21.15.86:0/2673785557 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3746839355 0 0) 0x7f6a1806dcd0 con 0x7f6a1815f210 2024-04-20T21:24:35.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.146+0000 7f6a17400640 1 --2- 172.21.15.86:0/2673785557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a1815ac10 0x7f6a180b53b0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f6a000097a0 tx=0x7f6a0002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb84e4b13573bd59 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:35.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.146+0000 7f6a16a00640 1 -- 172.21.15.86:0/2673785557 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a000315a0 con 0x7f6a1815ac10 2024-04-20T21:24:35.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.146+0000 7f6a16a00640 1 -- 172.21.15.86:0/2673785557 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a00037070 con 0x7f6a1815ac10 2024-04-20T21:24:35.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.146+0000 7f6a16a00640 1 -- 172.21.15.86:0/2673785557 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a00031960 con 0x7f6a1815ac10 2024-04-20T21:24:35.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.147+0000 7f6a1e200640 1 -- 172.21.15.86:0/2673785557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a1815ac10 msgr2=0x7f6a180b53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:35.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.147+0000 7f6a1e200640 1 --2- 172.21.15.86:0/2673785557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a1815ac10 0x7f6a180b53b0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f6a000097a0 tx=0x7f6a0002b840 comp rx=0 tx=0).stop 2024-04-20T21:24:35.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.147+0000 7f6a1e200640 1 -- 172.21.15.86:0/2673785557 shutdown_connections 2024-04-20T21:24:35.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.147+0000 7f6a1e200640 1 --2- 172.21.15.86:0/2673785557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a1815ac10 0x7f6a180b53b0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:35.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.147+0000 7f6a1e200640 1 -- 172.21.15.86:0/2673785557 >> 172.21.15.86:0/2673785557 conn(0x7f6a180b2360 msgr2=0x7f6a180b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:35.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.147+0000 7f6a1e200640 1 -- 172.21.15.86:0/2673785557 shutdown_connections 2024-04-20T21:24:35.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.147+0000 7f6a1e200640 1 -- 172.21.15.86:0/2673785557 wait complete. 2024-04-20T21:24:35.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.149+0000 7f6a1e200640 1 Processor -- start 2024-04-20T21:24:35.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.149+0000 7f6a1e200640 1 -- start start 2024-04-20T21:24:35.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.150+0000 7f6a1e200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a18072ca0 0x7f6a181f3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:35.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.150+0000 7f6a1e200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a1815ac10 0x7f6a181f3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:35.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.150+0000 7f6a1e200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a1815f210 0x7f6a18143400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:35.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.150+0000 7f6a1e200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6a1815f650 con 0x7f6a18072ca0 2024-04-20T21:24:35.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.150+0000 7f6a1e200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6a1815aff0 con 0x7f6a1815f210 2024-04-20T21:24:35.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.150+0000 7f6a1e200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6a1815f7d0 con 0x7f6a1815ac10 2024-04-20T21:24:35.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.150+0000 7f6a17400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a18072ca0 0x7f6a181f3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:35.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.150+0000 7f6a17400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a18072ca0 0x7f6a181f3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45928/0 (socket says 172.21.15.86:45928) 2024-04-20T21:24:35.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.150+0000 7f6a17400640 1 -- 172.21.15.86:0/3571763051 learned_addr learned my addr 172.21.15.86:0/3571763051 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:35.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.150+0000 7f6a17e00640 1 --2- 172.21.15.86:0/3571763051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a1815f210 0x7f6a18143400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:35.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.150+0000 7f6a0ea00640 1 --2- 172.21.15.86:0/3571763051 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a1815ac10 0x7f6a181f3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:35.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.150+0000 7f6a17400640 1 -- 172.21.15.86:0/3571763051 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a1815ac10 msgr2=0x7f6a181f3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:35.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.151+0000 7f6a17400640 1 --2- 172.21.15.86:0/3571763051 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a1815ac10 0x7f6a181f3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:35.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.151+0000 7f6a17400640 1 -- 172.21.15.86:0/3571763051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a1815f210 msgr2=0x7f6a18143400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:35.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.151+0000 7f6a17400640 1 --2- 172.21.15.86:0/3571763051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a1815f210 0x7f6a18143400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:35.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.151+0000 7f6a17400640 1 -- 172.21.15.86:0/3571763051 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a18157450 con 0x7f6a18072ca0 2024-04-20T21:24:35.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.151+0000 7f6a17e00640 1 --2- 172.21.15.86:0/3571763051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a1815f210 0x7f6a18143400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:24:35.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.151+0000 7f6a17400640 1 --2- 172.21.15.86:0/3571763051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a18072ca0 0x7f6a181f3750 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f6a000097a0 tx=0x7f6a000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:35.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.151+0000 7f6a0fe00640 1 -- 172.21.15.86:0/3571763051 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a00032840 con 0x7f6a18072ca0 2024-04-20T21:24:35.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.151+0000 7f6a0fe00640 1 -- 172.21.15.86:0/3571763051 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a000329e0 con 0x7f6a18072ca0 2024-04-20T21:24:35.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.151+0000 7f6a0fe00640 1 -- 172.21.15.86:0/3571763051 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a00032dd0 con 0x7f6a18072ca0 2024-04-20T21:24:35.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.151+0000 7f6a1e200640 1 -- 172.21.15.86:0/3571763051 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a18159840 con 0x7f6a18072ca0 2024-04-20T21:24:35.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.151+0000 7f6a1e200640 1 -- 172.21.15.86:0/3571763051 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a18157730 con 0x7f6a18072ca0 2024-04-20T21:24:35.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.152+0000 7f6a1e200640 1 -- 172.21.15.86:0/3571763051 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a18159f80 con 0x7f6a18072ca0 2024-04-20T21:24:35.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.153+0000 7f6a0fe00640 1 -- 172.21.15.86:0/3571763051 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6a0003b3f0 con 0x7f6a18072ca0 2024-04-20T21:24:35.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.154+0000 7f6a0fe00640 1 --2- 172.21.15.86:0/3571763051 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f69f406c8f0 0x7f69f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:35.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.154+0000 7f6a0fe00640 1 -- 172.21.15.86:0/3571763051 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5882+0+0 (secure 0 0 0) 0x7f6a0002f030 con 0x7f6a18072ca0 2024-04-20T21:24:35.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.155+0000 7f6a0ea00640 1 --2- 172.21.15.86:0/3571763051 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f69f406c8f0 0x7f69f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:35.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.155+0000 7f6a0ea00640 1 --2- 172.21.15.86:0/3571763051 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f69f406c8f0 0x7f69f406edb0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6a181515b0 tx=0x7f6a08009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:35.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.159+0000 7f6a0fe00640 1 -- 172.21.15.86:0/3571763051 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6a00040070 con 0x7f6a18072ca0 2024-04-20T21:24:35.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.397+0000 7f6a1e200640 1 -- 172.21.15.86:0/3571763051 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f6a180689e0 con 0x7f6a18072ca0 2024-04-20T21:24:35.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.398+0000 7f6a0fe00640 1 -- 172.21.15.86:0/3571763051 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f6a0007bf30 con 0x7f6a18072ca0 2024-04-20T21:24:35.397 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:24:35.398 INFO:teuthology.orchestra.run.smithi086.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-20T21:24:35.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.401+0000 7f6a1e200640 1 -- 172.21.15.86:0/3571763051 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f69f406c8f0 msgr2=0x7f69f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:35.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.401+0000 7f6a1e200640 1 --2- 172.21.15.86:0/3571763051 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f69f406c8f0 0x7f69f406edb0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6a181515b0 tx=0x7f6a08009040 comp rx=0 tx=0).stop 2024-04-20T21:24:35.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.401+0000 7f6a1e200640 1 -- 172.21.15.86:0/3571763051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a18072ca0 msgr2=0x7f6a181f3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:35.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.401+0000 7f6a1e200640 1 --2- 172.21.15.86:0/3571763051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a18072ca0 0x7f6a181f3750 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f6a000097a0 tx=0x7f6a000046c0 comp rx=0 tx=0).stop 2024-04-20T21:24:35.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.401+0000 7f6a1e200640 1 -- 172.21.15.86:0/3571763051 shutdown_connections 2024-04-20T21:24:35.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.401+0000 7f6a1e200640 1 --2- 172.21.15.86:0/3571763051 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f69f406c8f0 0x7f69f406edb0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:35.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.401+0000 7f6a1e200640 1 --2- 172.21.15.86:0/3571763051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a1815f210 0x7f6a18143400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:35.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.401+0000 7f6a1e200640 1 --2- 172.21.15.86:0/3571763051 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a1815ac10 0x7f6a181f3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:35.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.401+0000 7f6a1e200640 1 --2- 172.21.15.86:0/3571763051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a18072ca0 0x7f6a181f3750 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:35.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.401+0000 7f6a1e200640 1 -- 172.21.15.86:0/3571763051 >> 172.21.15.86:0/3571763051 conn(0x7f6a180b2360 msgr2=0x7f6a181540c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:35.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.401+0000 7f6a1e200640 1 -- 172.21.15.86:0/3571763051 shutdown_connections 2024-04-20T21:24:35.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.401+0000 7f6a1e200640 1 -- 172.21.15.86:0/3571763051 wait complete. 2024-04-20T21:24:35.418 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-20T21:24:35.418 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T21:24:35.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.617+0000 7fddc6000640 1 Processor -- start 2024-04-20T21:24:35.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.618+0000 7fddc6000640 1 -- start start 2024-04-20T21:24:35.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.619+0000 7fddc6000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fddc006f2a0 0x7fddc0167410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:35.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.619+0000 7fddc6000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddc0160890 con 0x7fddc0156ae0 2024-04-20T21:24:35.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.619+0000 7fddc6000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddc0160b70 con 0x7fddc015d6a0 2024-04-20T21:24:35.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.619+0000 7fddc6000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fddc006db20 con 0x7fddc006f2a0 2024-04-20T21:24:35.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.619+0000 7fddc4c00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fddc0156ae0 0x7fddc0160080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:48572/0 (socket says 172.21.15.86:48572) 2024-04-20T21:24:35.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.619+0000 7fddc4c00640 1 -- 172.21.15.86:0/3390682574 learned_addr learned my addr 172.21.15.86:0/3390682574 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:24:35.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.620+0000 7fddc5600640 1 --2- 172.21.15.86:0/3390682574 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fddc006f2a0 0x7fddc0167410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:35.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.620+0000 7fddbfe00640 1 -- 172.21.15.86:0/3390682574 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1063701096 0 0) 0x7fddc0160890 con 0x7fddc0156ae0 2024-04-20T21:24:35.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.620+0000 7fddbfe00640 1 -- 172.21.15.86:0/3390682574 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdda4003520 con 0x7fddc0156ae0 2024-04-20T21:24:35.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.620+0000 7fddbfe00640 1 -- 172.21.15.86:0/3390682574 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3512224688 0 0) 0x7fdda4003520 con 0x7fddc0156ae0 2024-04-20T21:24:35.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.620+0000 7fddbfe00640 1 -- 172.21.15.86:0/3390682574 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fddc006f2a0 msgr2=0x7fddc0167410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:35.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.620+0000 7fddbfe00640 1 --2- 172.21.15.86:0/3390682574 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fddc006f2a0 0x7fddc0167410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:35.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.620+0000 7fddbfe00640 1 -- 172.21.15.86:0/3390682574 >> v1:172.21.15.144:6789/0 conn(0x7fddc015d6a0 legacy=0x7fddc00ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:35.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.620+0000 7fddbfe00640 1 -- 172.21.15.86:0/3390682574 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddc0160e50 con 0x7fddc0156ae0 2024-04-20T21:24:35.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.620+0000 7fddbfe00640 1 -- 172.21.15.86:0/3390682574 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fddb0001690 con 0x7fddc0156ae0 2024-04-20T21:24:35.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.620+0000 7fddbfe00640 1 -- 172.21.15.86:0/3390682574 >> v1:172.21.15.86:6789/0 conn(0x7fddc0156ae0 legacy=0x7fddc0160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:35.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.620+0000 7fddbfe00640 1 --2- 172.21.15.86:0/3390682574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdda40039e0 0x7fdda4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:35.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.620+0000 7fddbfe00640 1 --2- 172.21.15.86:0/3390682574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdda4004800 0x7fdda4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:35.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.621+0000 7fddbfe00640 1 --2- 172.21.15.86:0/3390682574 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fddc006f2a0 0x7fdda40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:35.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.621+0000 7fddbfe00640 1 -- 172.21.15.86:0/3390682574 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fddc006db20 con 0x7fdda40039e0 2024-04-20T21:24:35.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.621+0000 7fddbfe00640 1 -- 172.21.15.86:0/3390682574 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdda400baa0 con 0x7fdda4004800 2024-04-20T21:24:35.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.621+0000 7fddbfe00640 1 -- 172.21.15.86:0/3390682574 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdda400bca0 con 0x7fddc006f2a0 2024-04-20T21:24:35.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.621+0000 7fddc4c00640 1 --2- 172.21.15.86:0/3390682574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdda40039e0 0x7fdda4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:35.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.621+0000 7fddc4c00640 1 -- 172.21.15.86:0/3390682574 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fddc006f2a0 msgr2=0x7fdda40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:35.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.621+0000 7fddc4c00640 1 --2- 172.21.15.86:0/3390682574 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fddc006f2a0 0x7fdda40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:35.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.621+0000 7fddc4c00640 1 -- 172.21.15.86:0/3390682574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdda4004800 msgr2=0x7fdda4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:35.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.621+0000 7fddc4c00640 1 --2- 172.21.15.86:0/3390682574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdda4004800 0x7fdda4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:35.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.621+0000 7fddc4c00640 1 -- 172.21.15.86:0/3390682574 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdda4003520 con 0x7fdda40039e0 2024-04-20T21:24:35.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.622+0000 7fddc4c00640 1 --2- 172.21.15.86:0/3390682574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdda40039e0 0x7fdda4003e00 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fddb0001eb0 tx=0x7fdda4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea6a3b711f23c440 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:35.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.622+0000 7fddbfe00640 1 -- 172.21.15.86:0/3390682574 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb0007c80 con 0x7fdda40039e0 2024-04-20T21:24:35.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.623+0000 7fddbfe00640 1 -- 172.21.15.86:0/3390682574 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddb0030070 con 0x7fdda40039e0 2024-04-20T21:24:35.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.623+0000 7fddbfe00640 1 -- 172.21.15.86:0/3390682574 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb0005d30 con 0x7fdda40039e0 2024-04-20T21:24:35.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.623+0000 7fddc6000640 1 -- 172.21.15.86:0/3390682574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdda40039e0 msgr2=0x7fdda4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:35.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.623+0000 7fddc6000640 1 --2- 172.21.15.86:0/3390682574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdda40039e0 0x7fdda4003e00 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fddb0001eb0 tx=0x7fdda4005830 comp rx=0 tx=0).stop 2024-04-20T21:24:35.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.623+0000 7fddc5600640 1 -- 172.21.15.86:0/3390682574 reap_dead start 2024-04-20T21:24:35.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.623+0000 7fddc6000640 1 -- 172.21.15.86:0/3390682574 shutdown_connections 2024-04-20T21:24:35.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.623+0000 7fddc6000640 1 -- 172.21.15.86:0/3390682574 >> 172.21.15.86:0/3390682574 conn(0x7fddc00d0170 msgr2=0x7fddc00d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:35.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.623+0000 7fddc6000640 1 -- 172.21.15.86:0/3390682574 shutdown_connections 2024-04-20T21:24:35.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.623+0000 7fddc6000640 1 -- 172.21.15.86:0/3390682574 wait complete. 2024-04-20T21:24:35.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.625+0000 7fddc6000640 1 Processor -- start 2024-04-20T21:24:35.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.625+0000 7fddc6000640 1 -- start start 2024-04-20T21:24:35.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.626+0000 7fddc6000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdda40039e0 0x7fddc0157040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:35.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.626+0000 7fddc6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddc015ce10 0x7fddc0157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:35.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.626+0000 7fddc6000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddc0157ac0 0x7fddc015b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:35.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.626+0000 7fddc6000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fddc00ad7f0 con 0x7fddc015ce10 2024-04-20T21:24:35.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.626+0000 7fddc6000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fddc0156ec0 con 0x7fddc0157ac0 2024-04-20T21:24:35.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.626+0000 7fddc6000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fddc00ad970 con 0x7fdda40039e0 2024-04-20T21:24:35.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.626+0000 7fddb7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddc015ce10 0x7fddc0157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:35.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.626+0000 7fddb7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddc015ce10 0x7fddc0157580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45946/0 (socket says 172.21.15.86:45946) 2024-04-20T21:24:35.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.626+0000 7fddb7e00640 1 -- 172.21.15.86:0/66837038 learned_addr learned my addr 172.21.15.86:0/66837038 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:35.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.627+0000 7fddc5600640 1 --2- 172.21.15.86:0/66837038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddc0157ac0 0x7fddc015b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:35.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.627+0000 7fddc4c00640 1 --2- 172.21.15.86:0/66837038 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdda40039e0 0x7fddc0157040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:35.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.627+0000 7fddb7e00640 1 -- 172.21.15.86:0/66837038 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdda40039e0 msgr2=0x7fddc0157040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:35.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.627+0000 7fddb7e00640 1 --2- 172.21.15.86:0/66837038 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdda40039e0 0x7fddc0157040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:35.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.627+0000 7fddb7e00640 1 -- 172.21.15.86:0/66837038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddc0157ac0 msgr2=0x7fddc015b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:35.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.627+0000 7fddb7e00640 1 --2- 172.21.15.86:0/66837038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddc0157ac0 0x7fddc015b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:35.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.627+0000 7fddb7e00640 1 -- 172.21.15.86:0/66837038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddc0159160 con 0x7fddc015ce10 2024-04-20T21:24:35.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.627+0000 7fddb7e00640 1 --2- 172.21.15.86:0/66837038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddc015ce10 0x7fddc0157580 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fdda800c530 tx=0x7fdda800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:35.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.627+0000 7fddbd600640 1 -- 172.21.15.86:0/66837038 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda80052d0 con 0x7fddc015ce10 2024-04-20T21:24:35.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.627+0000 7fddbd600640 1 -- 172.21.15.86:0/66837038 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdda8005470 con 0x7fddc015ce10 2024-04-20T21:24:35.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.628+0000 7fddc6000640 1 -- 172.21.15.86:0/66837038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddc0159440 con 0x7fddc015ce10 2024-04-20T21:24:35.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.628+0000 7fddbd600640 1 -- 172.21.15.86:0/66837038 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda8012870 con 0x7fddc015ce10 2024-04-20T21:24:35.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.628+0000 7fddc6000640 1 -- 172.21.15.86:0/66837038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddc015b390 con 0x7fddc015ce10 2024-04-20T21:24:35.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.629+0000 7fddc6000640 1 -- 172.21.15.86:0/66837038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddc004fcf0 con 0x7fddc015ce10 2024-04-20T21:24:35.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.630+0000 7fddbd600640 1 -- 172.21.15.86:0/66837038 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdda8012a10 con 0x7fddc015ce10 2024-04-20T21:24:35.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.631+0000 7fddbd600640 1 --2- 172.21.15.86:0/66837038 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdd9c06c920 0x7fdd9c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:35.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.631+0000 7fddbd600640 1 -- 172.21.15.86:0/66837038 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5882+0+0 (secure 0 0 0) 0x7fdda808b870 con 0x7fddc015ce10 2024-04-20T21:24:35.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.632+0000 7fddc4c00640 1 --2- 172.21.15.86:0/66837038 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdd9c06c920 0x7fdd9c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:35.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.632+0000 7fddc4c00640 1 --2- 172.21.15.86:0/66837038 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdd9c06c920 0x7fdd9c06ede0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fddb0034000 tx=0x7fddb0009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:35.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.636+0000 7fddbd600640 1 -- 172.21.15.86:0/66837038 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fdda8059b50 con 0x7fddc015ce10 2024-04-20T21:24:35.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.876+0000 7fddc6000640 1 -- 172.21.15.86:0/66837038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fddc006fb40 con 0x7fddc015ce10 2024-04-20T21:24:35.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.877+0000 7fddbd600640 1 -- 172.21.15.86:0/66837038 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v50) ==== 53+0+73 (secure 0 0 0) 0x7fdda8059450 con 0x7fddc015ce10 2024-04-20T21:24:35.876 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T21:24:35.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.879+0000 7fddc6000640 1 -- 172.21.15.86:0/66837038 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdd9c06c920 msgr2=0x7fdd9c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:35.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.879+0000 7fddc6000640 1 --2- 172.21.15.86:0/66837038 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdd9c06c920 0x7fdd9c06ede0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fddb0034000 tx=0x7fddb0009030 comp rx=0 tx=0).stop 2024-04-20T21:24:35.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.879+0000 7fddc6000640 1 -- 172.21.15.86:0/66837038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddc015ce10 msgr2=0x7fddc0157580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:35.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.879+0000 7fddc6000640 1 --2- 172.21.15.86:0/66837038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddc015ce10 0x7fddc0157580 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fdda800c530 tx=0x7fdda800c9f0 comp rx=0 tx=0).stop 2024-04-20T21:24:35.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.879+0000 7fddc6000640 1 -- 172.21.15.86:0/66837038 shutdown_connections 2024-04-20T21:24:35.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.879+0000 7fddc6000640 1 --2- 172.21.15.86:0/66837038 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdd9c06c920 0x7fdd9c06ede0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:35.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.879+0000 7fddc6000640 1 --2- 172.21.15.86:0/66837038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddc0157ac0 0x7fddc015b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:35.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.879+0000 7fddc6000640 1 --2- 172.21.15.86:0/66837038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fddc015ce10 0x7fddc0157580 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:35.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.879+0000 7fddc6000640 1 --2- 172.21.15.86:0/66837038 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdda40039e0 0x7fddc0157040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:35.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.879+0000 7fddc6000640 1 -- 172.21.15.86:0/66837038 >> 172.21.15.86:0/66837038 conn(0x7fddc00d0170 msgr2=0x7fddc00b4930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:35.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.879+0000 7fddc6000640 1 -- 172.21.15.86:0/66837038 shutdown_connections 2024-04-20T21:24:35.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:35.880+0000 7fddc6000640 1 -- 172.21.15.86:0/66837038 wait complete. 2024-04-20T21:24:35.897 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T21:24:35.898 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T21:24:35.898 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-04-20T21:24:35.898 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-20T21:24:35.898 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T21:24:35.898 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T21:24:35.898 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T21:24:35.898 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T21:24:35.898 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:24:35.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:24:35.899 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-04-20T21:24:35.947 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T21:24:35.947 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T21:24:35.947 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T21:24:35.947 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:24:35.948 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-20T21:24:35.948 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:24:35.948 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:24:35.948 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:24:35.948 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-04-20T21:24:35.948 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:24:35.948 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:24:35.948 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-04-20T21:24:35.948 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:24:35.948 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:24:35.948 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T21:24:35.948 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:24:35.948 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T21:24:35.948 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42280sec preferred_lft 42280sec 2024-04-20T21:24:35.949 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-04-20T21:24:35.949 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:24:35.949 INFO:teuthology.orchestra.run.smithi086.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T21:24:35.949 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 86:25:97:2c:9c:bb brd ff:ff:ff:ff:ff:ff 2024-04-20T21:24:35.949 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-20T21:24:35.949 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:24:35.949 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::a4fb:30ff:fe90:2d6a/64 scope link 2024-04-20T21:24:35.949 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:24:35.949 INFO:teuthology.orchestra.run.smithi086.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T21:24:35.949 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 86:25:97:2c:9c:bb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:24:35.949 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::8425:97ff:fe2c:9cbb/64 scope link 2024-04-20T21:24:35.949 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:24:35.949 INFO:teuthology.orchestra.run.smithi086.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T21:24:35.949 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c2:9c:8b:55:ca:84 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:24:35.950 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c09c:8bff:fe55:ca84/64 scope link 2024-04-20T21:24:35.950 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:24:35.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:24:35.951 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T21:24:36.026 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-20T21:24:36.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:24:36.030 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-04-20T21:24:36.083 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-04-20T21:24:36.083 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T21:24:36.083 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T21:24:36.083 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T21:24:36.083 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T21:24:36.084 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:24:36.084 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-20T21:24:36.084 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T21:24:36.084 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T21:24:36.084 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T21:24:36.084 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-20T21:24:36.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:24:36.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:24:36.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:24:36.290 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-20T21:24:36.341 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-20T21:24:36.341 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-20T21:24:36.342 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:24:36.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:24:36.343 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-04-20T21:24:36.421 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-20T21:24:36.421 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T21:24:36.421 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-20T21:24:36.473 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:24:36.474 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T21:24:36.474 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-20T21:24:36.523 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-04-20T21:24:36.599 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T21:24:36.650 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-04-20T21:24:36.650 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-04-20T21:24:36.650 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-04-20T21:24:36.650 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-04-20T21:24:36.651 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-04-20T21:24:36.651 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-04-20T21:24:36.651 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-04-20T21:24:36.651 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-04-20T21:24:36.651 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-04-20T21:24:36.651 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-04-20T21:24:36.651 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-04-20T21:24:36.651 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/selinux 2024-04-20T21:24:36.651 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T21:24:36.651 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-04-20T21:24:36.651 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-04-20T21:24:36.651 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-04-20T21:24:36.652 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-04-20T21:24:36.652 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-04-20T21:24:36.652 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-04-20T21:24:36.652 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T21:24:36.652 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T21:24:36.652 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T21:24:36.652 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T21:24:36.652 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-04-20T21:24:36.652 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-04-20T21:24:36.652 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T21:24:36.652 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T21:24:36.652 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T21:24:36.652 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T21:24:36.653 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-04-20T21:24:36.653 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:24:36.653 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:24:36.653 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:24:36.653 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:24:36.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:24:36.654 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-04-20T21:24:36.705 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T21:24:36.706 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-20T21:24:36.713 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-04-20T21:24:36.782 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T21:24:36.829 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-04-20T21:24:36.829 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-04-20T21:24:36.829 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-04-20T21:24:36.829 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-04-20T21:24:36.829 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-04-20T21:24:36.829 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-04-20T21:24:36.829 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-04-20T21:24:36.829 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-04-20T21:24:36.829 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-04-20T21:24:36.830 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-04-20T21:24:36.830 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-04-20T21:24:36.830 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/selinux 2024-04-20T21:24:36.830 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T21:24:36.830 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-04-20T21:24:36.830 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-04-20T21:24:36.830 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-04-20T21:24:36.830 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-04-20T21:24:36.830 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-04-20T21:24:36.830 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-04-20T21:24:36.830 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T21:24:36.830 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T21:24:36.830 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T21:24:36.831 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T21:24:36.831 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-04-20T21:24:36.831 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-04-20T21:24:36.831 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T21:24:36.831 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T21:24:36.831 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T21:24:36.831 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T21:24:36.831 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-04-20T21:24:36.831 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:24:36.831 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:24:36.831 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:24:36.831 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:24:36.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:24:36.832 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-04-20T21:24:36.881 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:2024-04-20T21:24:36.882+0000 7fe2e7e42480 -1 init, newargv = 0x55ae49e0f360 newargc=15 2024-04-20T21:24:36.881 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[50375]: starting ceph client 2024-04-20T21:24:36.889 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[50375]: starting fuse 2024-04-20T21:24:36.915 INFO:teuthology.orchestra.run.smithi086.stdout:41 2024-04-20T21:24:36.915 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-04-20T21:24:36.915 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer -- python3 -c ' 2024-04-20T21:24:36.915 DEBUG:teuthology.orchestra.run.smithi086:> import glob 2024-04-20T21:24:36.915 DEBUG:teuthology.orchestra.run.smithi086:> import re 2024-04-20T21:24:36.915 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:24:36.915 DEBUG:teuthology.orchestra.run.smithi086:> import subprocess 2024-04-20T21:24:36.915 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:24:36.916 DEBUG:teuthology.orchestra.run.smithi086:> def _find_admin_socket(client_name): 2024-04-20T21:24:36.916 DEBUG:teuthology.orchestra.run.smithi086:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-20T21:24:36.916 DEBUG:teuthology.orchestra.run.smithi086:> files = glob.glob(asok_path) 2024-04-20T21:24:36.916 DEBUG:teuthology.orchestra.run.smithi086:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-20T21:24:36.916 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:24:36.916 DEBUG:teuthology.orchestra.run.smithi086:> # Given a non-glob path, it better be there 2024-04-20T21:24:36.916 DEBUG:teuthology.orchestra.run.smithi086:> if "*" not in asok_path: 2024-04-20T21:24:36.916 DEBUG:teuthology.orchestra.run.smithi086:> assert(len(files) == 1) 2024-04-20T21:24:36.916 DEBUG:teuthology.orchestra.run.smithi086:> return files[0] 2024-04-20T21:24:36.916 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:24:36.916 DEBUG:teuthology.orchestra.run.smithi086:> for f in files: 2024-04-20T21:24:36.916 DEBUG:teuthology.orchestra.run.smithi086:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T21:24:36.917 DEBUG:teuthology.orchestra.run.smithi086:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T21:24:36.917 DEBUG:teuthology.orchestra.run.smithi086:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T21:24:36.917 DEBUG:teuthology.orchestra.run.smithi086:> contents = proc_f.read() 2024-04-20T21:24:36.917 DEBUG:teuthology.orchestra.run.smithi086:> if mountpoint in contents: 2024-04-20T21:24:36.917 DEBUG:teuthology.orchestra.run.smithi086:> return f 2024-04-20T21:24:36.917 DEBUG:teuthology.orchestra.run.smithi086:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T21:24:36.917 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:24:36.917 DEBUG:teuthology.orchestra.run.smithi086:> print(_find_admin_socket("client.0")) 2024-04-20T21:24:36.917 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:24:37.101 INFO:teuthology.orchestra.run.smithi086.stdout:/var/run/ceph/ceph-client.0.50375.asok 2024-04-20T21:24:37.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:24:37.119 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.50375.asok 2024-04-20T21:24:37.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:24:37.119 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.50375.asok status 2024-04-20T21:24:37.346 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:24:37.346 INFO:teuthology.orchestra.run.smithi086.stdout: "metadata": { 2024-04-20T21:24:37.346 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T21:24:37.346 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T21:24:37.346 INFO:teuthology.orchestra.run.smithi086.stdout: "entity_id": "0", 2024-04-20T21:24:37.346 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": "smithi086", 2024-04-20T21:24:37.346 INFO:teuthology.orchestra.run.smithi086.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-20T21:24:37.346 INFO:teuthology.orchestra.run.smithi086.stdout: "pid": "50375", 2024-04-20T21:24:37.347 INFO:teuthology.orchestra.run.smithi086.stdout: "root": "/" 2024-04-20T21:24:37.347 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:24:37.347 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_count": 0, 2024-04-20T21:24:37.347 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_pinned_count": 0, 2024-04-20T21:24:37.347 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 7308, 2024-04-20T21:24:37.347 INFO:teuthology.orchestra.run.smithi086.stdout: "inst": { 2024-04-20T21:24:37.347 INFO:teuthology.orchestra.run.smithi086.stdout: "name": { 2024-04-20T21:24:37.347 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "client", 2024-04-20T21:24:37.347 INFO:teuthology.orchestra.run.smithi086.stdout: "num": 7308 2024-04-20T21:24:37.347 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:24:37.347 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2024-04-20T21:24:37.347 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-04-20T21:24:37.347 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2024-04-20T21:24:37.347 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 3012304196 2024-04-20T21:24:37.347 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:24:37.347 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:24:37.348 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2024-04-20T21:24:37.348 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-04-20T21:24:37.348 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2024-04-20T21:24:37.348 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 3012304196 2024-04-20T21:24:37.348 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:24:37.348 INFO:teuthology.orchestra.run.smithi086.stdout: "inst_str": "client.7308 192.168.0.1:0/3012304196", 2024-04-20T21:24:37.348 INFO:teuthology.orchestra.run.smithi086.stdout: "addr_str": "192.168.0.1:0/3012304196", 2024-04-20T21:24:37.348 INFO:teuthology.orchestra.run.smithi086.stdout: "inode_count": 1, 2024-04-20T21:24:37.348 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_epoch": 50, 2024-04-20T21:24:37.348 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch": 74, 2024-04-20T21:24:37.348 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch_barrier": 0, 2024-04-20T21:24:37.348 INFO:teuthology.orchestra.run.smithi086.stdout: "blocklisted": false, 2024-04-20T21:24:37.348 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs" 2024-04-20T21:24:37.349 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:24:37.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:24:37.361 DEBUG:teuthology.orchestra.run.smithi086:> stat --file-system '--printf=%T 2024-04-20T21:24:37.361 DEBUG:teuthology.orchestra.run.smithi086:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-20T21:24:37.413 INFO:teuthology.orchestra.run.smithi086.stdout:fuseblk 2024-04-20T21:24:37.413 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-20T21:24:37.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:24:37.413 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-20T21:24:37.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:24:37.497 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-04-20T21:24:37.557 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-20T21:24:37.557 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-20T21:24:37.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.667+0000 7f814a836f40 1 Processor -- start 2024-04-20T21:24:37.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.667+0000 7f814a836f40 1 -- start start 2024-04-20T21:24:37.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.668+0000 7f814a836f40 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5634a88cfe30 0x5634a895ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:37.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.668+0000 7f814a836f40 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5634a89427e0 0x5634a895f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:37.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.668+0000 7f814a836f40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634a8945ed0 0x5634a8944360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:37.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.668+0000 7f814a836f40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5634a88d1a50 con 0x5634a8945ed0 2024-04-20T21:24:37.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.668+0000 7f814a836f40 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5634a882ccc0 con 0x5634a88cfe30 2024-04-20T21:24:37.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.668+0000 7f814a836f40 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5634a88d1cf0 con 0x5634a89427e0 2024-04-20T21:24:37.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.668+0000 7f8147c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634a8945ed0 0x5634a8944360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:37.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.668+0000 7f8147c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634a8945ed0 0x5634a8944360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51456/0 (socket says 172.21.15.86:51456) 2024-04-20T21:24:37.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.668+0000 7f8147c00640 1 -- 172.21.15.86:0/2288077089 learned_addr learned my addr 172.21.15.86:0/2288077089 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:37.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.669+0000 7f8146800640 1 --2- 172.21.15.86:0/2288077089 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5634a89427e0 0x5634a895f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:37.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.669+0000 7f8147200640 1 --2- 172.21.15.86:0/2288077089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5634a88cfe30 0x5634a895ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:37.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.669+0000 7f8147c00640 1 -- 172.21.15.86:0/2288077089 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5634a89427e0 msgr2=0x5634a895f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:37.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.669+0000 7f8147c00640 1 --2- 172.21.15.86:0/2288077089 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5634a89427e0 0x5634a895f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:37.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.669+0000 7f8147c00640 1 -- 172.21.15.86:0/2288077089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5634a88cfe30 msgr2=0x5634a895ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:37.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.669+0000 7f8147c00640 1 --2- 172.21.15.86:0/2288077089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5634a88cfe30 0x5634a895ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:37.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.669+0000 7f8147c00640 1 -- 172.21.15.86:0/2288077089 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634a89672b0 con 0x5634a8945ed0 2024-04-20T21:24:37.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.669+0000 7f8147c00640 1 --2- 172.21.15.86:0/2288077089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634a8945ed0 0x5634a8944360 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f8140044680 tx=0x7f814004c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f71fab6aa3e31d8e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:37.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.669+0000 7f8145400640 1 -- 172.21.15.86:0/2288077089 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8140063070 con 0x5634a8945ed0 2024-04-20T21:24:37.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.669+0000 7f8145400640 1 -- 172.21.15.86:0/2288077089 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81400525e0 con 0x5634a8945ed0 2024-04-20T21:24:37.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.670+0000 7f8145400640 1 -- 172.21.15.86:0/2288077089 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8140052990 con 0x5634a8945ed0 2024-04-20T21:24:37.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.670+0000 7f814a836f40 1 -- 172.21.15.86:0/2288077089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634a8945ed0 msgr2=0x5634a8944360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:37.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.670+0000 7f814a836f40 1 --2- 172.21.15.86:0/2288077089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634a8945ed0 0x5634a8944360 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f8140044680 tx=0x7f814004c000 comp rx=0 tx=0).stop 2024-04-20T21:24:37.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.670+0000 7f814a836f40 1 -- 172.21.15.86:0/2288077089 shutdown_connections 2024-04-20T21:24:37.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.670+0000 7f814a836f40 1 --2- 172.21.15.86:0/2288077089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634a8945ed0 0x5634a8944360 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:37.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.670+0000 7f814a836f40 1 --2- 172.21.15.86:0/2288077089 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5634a89427e0 0x5634a895f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:37.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.670+0000 7f814a836f40 1 --2- 172.21.15.86:0/2288077089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5634a88cfe30 0x5634a895ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:37.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.670+0000 7f814a836f40 1 -- 172.21.15.86:0/2288077089 >> 172.21.15.86:0/2288077089 conn(0x5634a888a950 msgr2=0x5634a89666d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:37.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.670+0000 7f814a836f40 1 -- 172.21.15.86:0/2288077089 shutdown_connections 2024-04-20T21:24:37.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.670+0000 7f814a836f40 1 -- 172.21.15.86:0/2288077089 wait complete. 2024-04-20T21:24:37.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.671+0000 7f814a836f40 1 Processor -- start 2024-04-20T21:24:37.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.671+0000 7f814a836f40 1 -- start start 2024-04-20T21:24:37.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.672+0000 7f814a836f40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634a88cfe30 0x5634a88b7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:37.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.672+0000 7f814a836f40 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5634a89427e0 0x5634a893f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:37.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.672+0000 7f814a836f40 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5634a8945ed0 0x5634a893ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:37.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.672+0000 7f814a836f40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5634a88b4670 con 0x5634a88cfe30 2024-04-20T21:24:37.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.672+0000 7f814a836f40 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5634a88b43d0 con 0x5634a89427e0 2024-04-20T21:24:37.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.672+0000 7f814a836f40 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5634a89414e0 con 0x5634a8945ed0 2024-04-20T21:24:37.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.672+0000 7f8147200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634a88cfe30 0x5634a88b7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:37.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.672+0000 7f8147200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634a88cfe30 0x5634a88b7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51466/0 (socket says 172.21.15.86:51466) 2024-04-20T21:24:37.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.672+0000 7f8147200640 1 -- 172.21.15.86:0/3761707304 learned_addr learned my addr 172.21.15.86:0/3761707304 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:37.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.673+0000 7f8147c00640 1 --2- 172.21.15.86:0/3761707304 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5634a8945ed0 0x5634a893ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:37.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.673+0000 7f8146800640 1 --2- 172.21.15.86:0/3761707304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5634a89427e0 0x5634a893f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:37.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.673+0000 7f8147200640 1 -- 172.21.15.86:0/3761707304 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5634a8945ed0 msgr2=0x5634a893ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:37.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.673+0000 7f8147200640 1 --2- 172.21.15.86:0/3761707304 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5634a8945ed0 0x5634a893ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:37.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.673+0000 7f8147200640 1 -- 172.21.15.86:0/3761707304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5634a89427e0 msgr2=0x5634a893f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:37.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.673+0000 7f8147200640 1 --2- 172.21.15.86:0/3761707304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5634a89427e0 0x5634a893f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:37.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.673+0000 7f8147200640 1 -- 172.21.15.86:0/3761707304 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634a8944b40 con 0x5634a88cfe30 2024-04-20T21:24:37.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.673+0000 7f8147200640 1 --2- 172.21.15.86:0/3761707304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634a88cfe30 0x5634a88b7e40 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f8138009ec0 tx=0x7f8138008400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:37.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.673+0000 7f813fe00640 1 -- 172.21.15.86:0/3761707304 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8138008e40 con 0x5634a88cfe30 2024-04-20T21:24:37.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.674+0000 7f813fe00640 1 -- 172.21.15.86:0/3761707304 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81380143f0 con 0x5634a88cfe30 2024-04-20T21:24:37.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.674+0000 7f814a836f40 1 -- 172.21.15.86:0/3761707304 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5634a8944e20 con 0x5634a88cfe30 2024-04-20T21:24:37.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.674+0000 7f813fe00640 1 -- 172.21.15.86:0/3761707304 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81380147a0 con 0x5634a88cfe30 2024-04-20T21:24:37.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.674+0000 7f814a836f40 1 -- 172.21.15.86:0/3761707304 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634a88b7920 con 0x5634a88cfe30 2024-04-20T21:24:37.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.676+0000 7f813fe00640 1 -- 172.21.15.86:0/3761707304 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f813801b020 con 0x5634a88cfe30 2024-04-20T21:24:37.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.677+0000 7f813fe00640 1 --2- 172.21.15.86:0/3761707304 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f812806b2d0 0x7f812806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:37.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.677+0000 7f813fe00640 1 -- 172.21.15.86:0/3761707304 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5882+0+0 (secure 0 0 0) 0x7f813808a120 con 0x5634a88cfe30 2024-04-20T21:24:37.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.677+0000 7f8146800640 1 --2- 172.21.15.86:0/3761707304 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f812806b2d0 0x7f812806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:37.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.677+0000 7f814a836f40 1 --2- 172.21.15.86:0/3761707304 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x5634a893faf0 0x5634a89f8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:37.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.677+0000 7f814a836f40 1 -- 172.21.15.86:0/3761707304 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:1 10.10 10:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74) -- 0x5634a893d700 con 0x5634a893faf0 2024-04-20T21:24:37.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.678+0000 7f8146800640 1 --2- 172.21.15.86:0/3761707304 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f812806b2d0 0x7f812806d790 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x5634a8945260 tx=0x7f8134000bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:37.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.678+0000 7f8147c00640 1 --2- 172.21.15.86:0/3761707304 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x5634a893faf0 0x5634a89f8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:37.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.679+0000 7f8147c00640 1 --2- 172.21.15.86:0/3761707304 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x5634a893faf0 0x5634a89f8870 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:37.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.683+0000 7f8147c00640 1 -- 172.21.15.86:0/3761707304 <== osd.4 v2:172.21.15.144:6800/4087309475 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v74'12 uv12 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f8140052bf0 con 0x5634a893faf0 2024-04-20T21:24:37.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.683+0000 7f814a836f40 1 -- 172.21.15.86:0/3761707304 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x5634a893faf0 msgr2=0x5634a89f8870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:37.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.683+0000 7f814a836f40 1 --2- 172.21.15.86:0/3761707304 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x5634a893faf0 0x5634a89f8870 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:37.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.683+0000 7f814a836f40 1 -- 172.21.15.86:0/3761707304 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f812806b2d0 msgr2=0x7f812806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:37.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.683+0000 7f814a836f40 1 --2- 172.21.15.86:0/3761707304 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f812806b2d0 0x7f812806d790 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x5634a8945260 tx=0x7f8134000bc0 comp rx=0 tx=0).stop 2024-04-20T21:24:37.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.683+0000 7f814a836f40 1 -- 172.21.15.86:0/3761707304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634a88cfe30 msgr2=0x5634a88b7e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:37.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.683+0000 7f814a836f40 1 --2- 172.21.15.86:0/3761707304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5634a88cfe30 0x5634a88b7e40 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f8138009ec0 tx=0x7f8138008400 comp rx=0 tx=0).stop 2024-04-20T21:24:37.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.683+0000 7f8147c00640 1 -- 172.21.15.86:0/3761707304 reap_dead start 2024-04-20T21:24:37.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.683+0000 7f814a836f40 1 -- 172.21.15.86:0/3761707304 shutdown_connections 2024-04-20T21:24:37.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.683+0000 7f814a836f40 1 -- 172.21.15.86:0/3761707304 >> 172.21.15.86:0/3761707304 conn(0x5634a888a950 msgr2=0x5634a8966990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:37.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.684+0000 7f814a836f40 1 -- 172.21.15.86:0/3761707304 shutdown_connections 2024-04-20T21:24:37.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:37.684+0000 7f814a836f40 1 -- 172.21.15.86:0/3761707304 wait complete. 2024-04-20T21:24:37.687 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-20T21:24:37.687 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T21:24:37.773 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T21:24:37.888 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-20T21:24:37.889 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T21:24:37.889 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:24:37.889 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:24:37.947 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:24:38.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.155+0000 7f7f4f000640 1 Processor -- start 2024-04-20T21:24:38.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.155+0000 7f7f4f000640 1 -- start start 2024-04-20T21:24:38.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.157+0000 7f7f4f000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7f480af8d0 0x7f7f48141f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:38.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.157+0000 7f7f4f000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f48071c10 con 0x7f7f48162c40 2024-04-20T21:24:38.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.157+0000 7f7f4f000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f4815eaf0 con 0x7f7f480b3810 2024-04-20T21:24:38.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.157+0000 7f7f4f000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f7f48142490 con 0x7f7f480af8d0 2024-04-20T21:24:38.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.157+0000 7f7f4d200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f7f48162c40 0x7f7f480b3100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57980/0 (socket says 172.21.15.86:57980) 2024-04-20T21:24:38.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.157+0000 7f7f4d200640 1 -- 172.21.15.86:0/1182058366 learned_addr learned my addr 172.21.15.86:0/1182058366 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:24:38.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.157+0000 7f7f4dc00640 1 --2- 172.21.15.86:0/1182058366 >> v2:172.21.15.144:3300/0 conn(0x7f7f480af8d0 0x7f7f48141f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:38.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.157+0000 7f7f47e00640 1 -- 172.21.15.86:0/1182058366 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1491958383 0 0) 0x7f7f48071c10 con 0x7f7f48162c40 2024-04-20T21:24:38.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.158+0000 7f7f47e00640 1 -- 172.21.15.86:0/1182058366 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f30003520 con 0x7f7f48162c40 2024-04-20T21:24:38.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.158+0000 7f7f47e00640 1 -- 172.21.15.86:0/1182058366 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3070845348 0 0) 0x7f7f30003520 con 0x7f7f48162c40 2024-04-20T21:24:38.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.158+0000 7f7f47e00640 1 -- 172.21.15.86:0/1182058366 >> v2:172.21.15.144:3300/0 conn(0x7f7f480af8d0 msgr2=0x7f7f48141f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:38.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.158+0000 7f7f47e00640 1 --2- 172.21.15.86:0/1182058366 >> v2:172.21.15.144:3300/0 conn(0x7f7f480af8d0 0x7f7f48141f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:38.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.158+0000 7f7f47e00640 1 -- 172.21.15.86:0/1182058366 >> v1:172.21.15.144:6789/0 conn(0x7f7f480b3810 legacy=0x7f7f48072990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:38.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.158+0000 7f7f47e00640 1 -- 172.21.15.86:0/1182058366 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f48071eb0 con 0x7f7f48162c40 2024-04-20T21:24:38.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.158+0000 7f7f47e00640 1 -- 172.21.15.86:0/1182058366 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f7f34001690 con 0x7f7f48162c40 2024-04-20T21:24:38.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.158+0000 7f7f47e00640 1 -- 172.21.15.86:0/1182058366 >> v1:172.21.15.86:6789/0 conn(0x7f7f48162c40 legacy=0x7f7f480b3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:38.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.158+0000 7f7f47e00640 1 --2- 172.21.15.86:0/1182058366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f300039b0 0x7f7f30003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:38.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.158+0000 7f7f47e00640 1 --2- 172.21.15.86:0/1182058366 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7f30004800 0x7f7f30004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:38.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.158+0000 7f7f47e00640 1 --2- 172.21.15.86:0/1182058366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f300052e0 0x7f7f3000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:38.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.158+0000 7f7f47e00640 1 -- 172.21.15.86:0/1182058366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3000c180 con 0x7f7f300052e0 2024-04-20T21:24:38.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.158+0000 7f7f47e00640 1 -- 172.21.15.86:0/1182058366 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3000c330 con 0x7f7f300039b0 2024-04-20T21:24:38.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.158+0000 7f7f47e00640 1 -- 172.21.15.86:0/1182058366 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7f3000c530 con 0x7f7f30004800 2024-04-20T21:24:38.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.158+0000 7f7f4e600640 1 --2- 172.21.15.86:0/1182058366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f300052e0 0x7f7f3000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:38.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.159+0000 7f7f4dc00640 1 --2- 172.21.15.86:0/1182058366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f300039b0 0x7f7f30003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:38.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.159+0000 7f7f4d200640 1 --2- 172.21.15.86:0/1182058366 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7f30004800 0x7f7f30004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:38.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.159+0000 7f7f4e600640 1 -- 172.21.15.86:0/1182058366 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7f30004800 msgr2=0x7f7f30004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:38.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.159+0000 7f7f4e600640 1 --2- 172.21.15.86:0/1182058366 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7f30004800 0x7f7f30004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:38.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.159+0000 7f7f4e600640 1 -- 172.21.15.86:0/1182058366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f300039b0 msgr2=0x7f7f30003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:38.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.159+0000 7f7f4e600640 1 --2- 172.21.15.86:0/1182058366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f300039b0 0x7f7f30003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:38.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.159+0000 7f7f4e600640 1 -- 172.21.15.86:0/1182058366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f30003520 con 0x7f7f300052e0 2024-04-20T21:24:38.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.159+0000 7f7f4dc00640 1 --2- 172.21.15.86:0/1182058366 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f300039b0 0x7f7f30003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:24:38.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.160+0000 7f7f4e600640 1 -- 172.21.15.86:0/1182058366 reap_dead start 2024-04-20T21:24:38.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.160+0000 7f7f4e600640 1 --2- 172.21.15.86:0/1182058366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f300052e0 0x7f7f3000b9c0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f7f40001e20 tx=0x7f7f30005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fff57064957cd71 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:38.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.161+0000 7f7f47e00640 1 -- 172.21.15.86:0/1182058366 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f40001a70 con 0x7f7f300052e0 2024-04-20T21:24:38.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.161+0000 7f7f47e00640 1 -- 172.21.15.86:0/1182058366 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f40003b70 con 0x7f7f300052e0 2024-04-20T21:24:38.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.161+0000 7f7f47e00640 1 -- 172.21.15.86:0/1182058366 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f400050d0 con 0x7f7f300052e0 2024-04-20T21:24:38.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.161+0000 7f7f4f000640 1 -- 172.21.15.86:0/1182058366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f300052e0 msgr2=0x7f7f3000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:38.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.161+0000 7f7f4f000640 1 --2- 172.21.15.86:0/1182058366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f300052e0 0x7f7f3000b9c0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f7f40001e20 tx=0x7f7f30005830 comp rx=0 tx=0).stop 2024-04-20T21:24:38.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.161+0000 7f7f4f000640 1 -- 172.21.15.86:0/1182058366 shutdown_connections 2024-04-20T21:24:38.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.161+0000 7f7f4f000640 1 --2- 172.21.15.86:0/1182058366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f300052e0 0x7f7f3000b9c0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:38.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.161+0000 7f7f4f000640 1 -- 172.21.15.86:0/1182058366 >> 172.21.15.86:0/1182058366 conn(0x7f7f480d0170 msgr2=0x7f7f480d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:38.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.161+0000 7f7f4f000640 1 -- 172.21.15.86:0/1182058366 shutdown_connections 2024-04-20T21:24:38.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.161+0000 7f7f4f000640 1 -- 172.21.15.86:0/1182058366 wait complete. 2024-04-20T21:24:38.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.163+0000 7f7f4f000640 1 Processor -- start 2024-04-20T21:24:38.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.164+0000 7f7f4f000640 1 -- start start 2024-04-20T21:24:38.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.164+0000 7f7f4f000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7f300052e0 0x7f7f48147080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:38.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.165+0000 7f7f4f000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f48160f90 0x7f7f48161410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:38.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.165+0000 7f7f4f000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f4815f6f0 0x7f7f4815fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:38.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.165+0000 7f7f4f000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7f48163100 con 0x7f7f4815f6f0 2024-04-20T21:24:38.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.165+0000 7f7f4f000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7f48072470 con 0x7f7f48160f90 2024-04-20T21:24:38.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.165+0000 7f7f4f000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7f48148050 con 0x7f7f300052e0 2024-04-20T21:24:38.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.165+0000 7f7f4e600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f4815f6f0 0x7f7f4815fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:38.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.165+0000 7f7f4e600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f4815f6f0 0x7f7f4815fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51494/0 (socket says 172.21.15.86:51494) 2024-04-20T21:24:38.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.165+0000 7f7f4e600640 1 -- 172.21.15.86:0/825475883 learned_addr learned my addr 172.21.15.86:0/825475883 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:38.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.165+0000 7f7f4e600640 1 -- 172.21.15.86:0/825475883 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7f300052e0 msgr2=0x7f7f48147080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:38.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.165+0000 7f7f4e600640 1 --2- 172.21.15.86:0/825475883 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7f300052e0 0x7f7f48147080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:38.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.165+0000 7f7f4e600640 1 -- 172.21.15.86:0/825475883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f48160f90 msgr2=0x7f7f48161410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:38.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.165+0000 7f7f4d200640 1 --2- 172.21.15.86:0/825475883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f48160f90 0x7f7f48161410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:38.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.165+0000 7f7f4e600640 1 --2- 172.21.15.86:0/825475883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f48160f90 0x7f7f48161410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:38.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.165+0000 7f7f4e600640 1 -- 172.21.15.86:0/825475883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f481f40c0 con 0x7f7f4815f6f0 2024-04-20T21:24:38.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.166+0000 7f7f4e600640 1 --2- 172.21.15.86:0/825475883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f4815f6f0 0x7f7f4815fb70 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f7f4002c040 tx=0x7f7f400014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:38.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.166+0000 7f7f45600640 1 -- 172.21.15.86:0/825475883 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f40037460 con 0x7f7f4815f6f0 2024-04-20T21:24:38.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.166+0000 7f7f45600640 1 -- 172.21.15.86:0/825475883 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f40037600 con 0x7f7f4815f6f0 2024-04-20T21:24:38.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.166+0000 7f7f4f000640 1 -- 172.21.15.86:0/825475883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f48160470 con 0x7f7f4815f6f0 2024-04-20T21:24:38.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.166+0000 7f7f45600640 1 -- 172.21.15.86:0/825475883 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f400379b0 con 0x7f7f4815f6f0 2024-04-20T21:24:38.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.166+0000 7f7f4f000640 1 -- 172.21.15.86:0/825475883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f48160a30 con 0x7f7f4815f6f0 2024-04-20T21:24:38.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.167+0000 7f7f4f000640 1 -- 172.21.15.86:0/825475883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f4804fcf0 con 0x7f7f4815f6f0 2024-04-20T21:24:38.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.168+0000 7f7f45600640 1 -- 172.21.15.86:0/825475883 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7f40037b50 con 0x7f7f4815f6f0 2024-04-20T21:24:38.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.170+0000 7f7f45600640 1 --2- 172.21.15.86:0/825475883 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7f2806c8f0 0x7f7f2806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:38.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.170+0000 7f7f45600640 1 -- 172.21.15.86:0/825475883 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5882+0+0 (secure 0 0 0) 0x7f7f400ad740 con 0x7f7f4815f6f0 2024-04-20T21:24:38.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.170+0000 7f7f4dc00640 1 --2- 172.21.15.86:0/825475883 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7f2806c8f0 0x7f7f2806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:38.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.171+0000 7f7f4dc00640 1 --2- 172.21.15.86:0/825475883 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7f2806c8f0 0x7f7f2806edb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c006f00 tx=0x7f7f3c006e90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:38.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.175+0000 7f7f45600640 1 -- 172.21.15.86:0/825475883 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7f481f40c0 con 0x7f7f4815f6f0 2024-04-20T21:24:38.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.377+0000 7f7f4f000640 1 -- 172.21.15.86:0/825475883 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7f480b08c0 con 0x7f7f4815f6f0 2024-04-20T21:24:38.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.379+0000 7f7f45600640 1 -- 172.21.15.86:0/825475883 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+16941 (secure 0 0 0) 0x7f7f40033030 con 0x7f7f4815f6f0 2024-04-20T21:24:38.379 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:24:38.379 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":74,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:23:35.220547+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:22:43.125767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-20T21:22:44.410549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.2d","mappings":[{"from":6,"to":0}]},{"pgid":"11.2e","mappings":[{"from":6,"to":0}]},{"pgid":"11.33","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6837/202143875":"2024-04-21T21:23:12.079572+0000","172.21.15.86:6835/202143875":"2024-04-21T21:23:12.079572+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:24:38.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.382+0000 7f7f4f000640 1 -- 172.21.15.86:0/825475883 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7f2806c8f0 msgr2=0x7f7f2806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:38.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.382+0000 7f7f4f000640 1 --2- 172.21.15.86:0/825475883 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7f2806c8f0 0x7f7f2806edb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c006f00 tx=0x7f7f3c006e90 comp rx=0 tx=0).stop 2024-04-20T21:24:38.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.382+0000 7f7f4f000640 1 -- 172.21.15.86:0/825475883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f4815f6f0 msgr2=0x7f7f4815fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:38.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.382+0000 7f7f4f000640 1 --2- 172.21.15.86:0/825475883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f4815f6f0 0x7f7f4815fb70 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f7f4002c040 tx=0x7f7f400014a0 comp rx=0 tx=0).stop 2024-04-20T21:24:38.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.382+0000 7f7f4f000640 1 -- 172.21.15.86:0/825475883 shutdown_connections 2024-04-20T21:24:38.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.382+0000 7f7f4f000640 1 --2- 172.21.15.86:0/825475883 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7f2806c8f0 0x7f7f2806edb0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:38.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.382+0000 7f7f4f000640 1 --2- 172.21.15.86:0/825475883 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f4815f6f0 0x7f7f4815fb70 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:38.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.382+0000 7f7f4f000640 1 --2- 172.21.15.86:0/825475883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f48160f90 0x7f7f48161410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:38.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.382+0000 7f7f4f000640 1 --2- 172.21.15.86:0/825475883 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7f300052e0 0x7f7f48147080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:38.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.382+0000 7f7f4f000640 1 -- 172.21.15.86:0/825475883 >> 172.21.15.86:0/825475883 conn(0x7f7f480d0170 msgr2=0x7f7f4806e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:38.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.382+0000 7f7f4f000640 1 -- 172.21.15.86:0/825475883 shutdown_connections 2024-04-20T21:24:38.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.382+0000 7f7f4f000640 1 -- 172.21.15.86:0/825475883 wait complete. 2024-04-20T21:24:38.399 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:22:43.125767+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:22:44.410549+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:24:38.400 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:24:38.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.604+0000 7fef1d400640 1 Processor -- start 2024-04-20T21:24:38.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.604+0000 7fef1d400640 1 -- start start 2024-04-20T21:24:38.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.606+0000 7fef1d400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fef180b3200 0x7fef180b3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:38.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.606+0000 7fef1d400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fef181420a0 0x7fef18142510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:38.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.606+0000 7fef1d400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef18148180 con 0x7fef180af980 2024-04-20T21:24:38.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.606+0000 7fef1d400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fef180b3ba0 con 0x7fef180b3200 2024-04-20T21:24:38.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.606+0000 7fef1d400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fef1806f450 con 0x7fef181420a0 2024-04-20T21:24:38.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.606+0000 7fef0f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fef180b3200 0x7fef180b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:38.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.606+0000 7fef0f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fef180b3200 0x7fef180b3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51502/0 (socket says 172.21.15.86:51502) 2024-04-20T21:24:38.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.606+0000 7fef0f400640 1 -- 172.21.15.86:0/3578556902 learned_addr learned my addr 172.21.15.86:0/3578556902 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:38.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.607+0000 7fef1ca00640 1 --2- 172.21.15.86:0/3578556902 >> v2:172.21.15.144:3300/0 conn(0x7fef181420a0 0x7fef18142510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:38.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.607+0000 7fef0f400640 1 -- 172.21.15.86:0/3578556902 >> v2:172.21.15.144:3300/0 conn(0x7fef181420a0 msgr2=0x7fef18142510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:38.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.607+0000 7fef0f400640 1 --2- 172.21.15.86:0/3578556902 >> v2:172.21.15.144:3300/0 conn(0x7fef181420a0 0x7fef18142510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:38.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.607+0000 7fef0f400640 1 -- 172.21.15.86:0/3578556902 >> v1:172.21.15.144:6789/0 conn(0x7fef180af980 legacy=0x7fef18147a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:38.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.607+0000 7fef0f400640 1 -- 172.21.15.86:0/3578556902 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef1806e840 con 0x7fef180b3200 2024-04-20T21:24:38.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.608+0000 7fef0f400640 1 --2- 172.21.15.86:0/3578556902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef180b3200 0x7fef180b3660 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fef1806fdf0 tx=0x7fef0002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb33521e0001db2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:38.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.608+0000 7fef17400640 1 -- 172.21.15.86:0/3578556902 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef00035020 con 0x7fef180b3200 2024-04-20T21:24:38.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.608+0000 7fef17400640 1 -- 172.21.15.86:0/3578556902 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef00031640 con 0x7fef180b3200 2024-04-20T21:24:38.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.608+0000 7fef17400640 1 -- 172.21.15.86:0/3578556902 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0002d040 con 0x7fef180b3200 2024-04-20T21:24:38.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.609+0000 7fef1d400640 1 -- 172.21.15.86:0/3578556902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef180b3200 msgr2=0x7fef180b3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:38.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.609+0000 7fef1d400640 1 --2- 172.21.15.86:0/3578556902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef180b3200 0x7fef180b3660 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fef1806fdf0 tx=0x7fef0002b8f0 comp rx=0 tx=0).stop 2024-04-20T21:24:38.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.609+0000 7fef1d400640 1 -- 172.21.15.86:0/3578556902 shutdown_connections 2024-04-20T21:24:38.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.609+0000 7fef1d400640 1 --2- 172.21.15.86:0/3578556902 >> v2:172.21.15.144:3300/0 conn(0x7fef181420a0 0x7fef18142510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:38.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.609+0000 7fef1d400640 1 --2- 172.21.15.86:0/3578556902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef180b3200 0x7fef180b3660 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:38.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.609+0000 7fef1d400640 1 -- 172.21.15.86:0/3578556902 >> 172.21.15.86:0/3578556902 conn(0x7fef180b1f00 msgr2=0x7fef180b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:38.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.609+0000 7fef1d400640 1 -- 172.21.15.86:0/3578556902 shutdown_connections 2024-04-20T21:24:38.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.609+0000 7fef1d400640 1 -- 172.21.15.86:0/3578556902 wait complete. 2024-04-20T21:24:38.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.611+0000 7fef1d400640 1 Processor -- start 2024-04-20T21:24:38.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.611+0000 7fef1d400640 1 -- start start 2024-04-20T21:24:38.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.612+0000 7fef1d400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef180af980 0x7fef18099700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:38.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.612+0000 7fef1d400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef180b3200 0x7fef1809c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:38.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.612+0000 7fef1d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef181420a0 0x7fef18142610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:38.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.612+0000 7fef1d400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fef1806f0e0 con 0x7fef181420a0 2024-04-20T21:24:38.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.612+0000 7fef1d400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fef1806ee00 con 0x7fef180af980 2024-04-20T21:24:38.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.612+0000 7fef1d400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fef1806e3d0 con 0x7fef180b3200 2024-04-20T21:24:38.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.613+0000 7fef1ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef181420a0 0x7fef18142610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:38.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.613+0000 7fef1ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef181420a0 0x7fef18142610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51518/0 (socket says 172.21.15.86:51518) 2024-04-20T21:24:38.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.613+0000 7fef1ca00640 1 -- 172.21.15.86:0/1277516803 learned_addr learned my addr 172.21.15.86:0/1277516803 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:38.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.613+0000 7fef17e00640 1 --2- 172.21.15.86:0/1277516803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef180af980 0x7fef18099700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:38.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.613+0000 7fef1ca00640 1 -- 172.21.15.86:0/1277516803 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef180b3200 msgr2=0x7fef1809c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:38.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.613+0000 7fef0f400640 1 --2- 172.21.15.86:0/1277516803 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef180b3200 0x7fef1809c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:38.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.613+0000 7fef1ca00640 1 --2- 172.21.15.86:0/1277516803 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef180b3200 0x7fef1809c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:38.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.613+0000 7fef1ca00640 1 -- 172.21.15.86:0/1277516803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef180af980 msgr2=0x7fef18099700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:38.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.613+0000 7fef1ca00640 1 --2- 172.21.15.86:0/1277516803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef180af980 0x7fef18099700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:38.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.613+0000 7fef1ca00640 1 -- 172.21.15.86:0/1277516803 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef1809cb60 con 0x7fef181420a0 2024-04-20T21:24:38.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.613+0000 7fef17e00640 1 --2- 172.21.15.86:0/1277516803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef180af980 0x7fef18099700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:24:38.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.613+0000 7fef1ca00640 1 --2- 172.21.15.86:0/1277516803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef181420a0 0x7fef18142610 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fef1000d830 tx=0x7fef1000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:38.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.614+0000 7fef14c00640 1 -- 172.21.15.86:0/1277516803 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1000b660 con 0x7fef181420a0 2024-04-20T21:24:38.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.614+0000 7fef1d400640 1 -- 172.21.15.86:0/1277516803 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef1809ce40 con 0x7fef181420a0 2024-04-20T21:24:38.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.614+0000 7fef14c00640 1 -- 172.21.15.86:0/1277516803 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef10019070 con 0x7fef181420a0 2024-04-20T21:24:38.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.614+0000 7fef14c00640 1 -- 172.21.15.86:0/1277516803 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1000ba60 con 0x7fef181420a0 2024-04-20T21:24:38.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.614+0000 7fef1d400640 1 -- 172.21.15.86:0/1277516803 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef1809d400 con 0x7fef181420a0 2024-04-20T21:24:38.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.615+0000 7fef1d400640 1 -- 172.21.15.86:0/1277516803 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef1804fd90 con 0x7fef181420a0 2024-04-20T21:24:38.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.616+0000 7fef14c00640 1 -- 172.21.15.86:0/1277516803 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fef1001e460 con 0x7fef181420a0 2024-04-20T21:24:38.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.617+0000 7fef14c00640 1 --2- 172.21.15.86:0/1277516803 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7feef406c8f0 0x7feef406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:38.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.617+0000 7fef14c00640 1 -- 172.21.15.86:0/1277516803 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5882+0+0 (secure 0 0 0) 0x7fef1008c350 con 0x7fef181420a0 2024-04-20T21:24:38.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.617+0000 7fef17e00640 1 --2- 172.21.15.86:0/1277516803 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7feef406c8f0 0x7feef406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:38.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.618+0000 7fef17e00640 1 --2- 172.21.15.86:0/1277516803 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7feef406c8f0 0x7feef406edb0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fef08007e90 tx=0x7fef080073a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:38.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.621+0000 7fef14c00640 1 -- 172.21.15.86:0/1277516803 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fef1005a5b0 con 0x7fef181420a0 2024-04-20T21:24:38.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.825+0000 7fef1d400640 1 -- 172.21.15.86:0/1277516803 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fef18073be0 con 0x7fef181420a0 2024-04-20T21:24:38.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.825+0000 7fef14c00640 1 -- 172.21.15.86:0/1277516803 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) ==== 93+0+10 (secure 0 0 0) 0x7fef10059eb0 con 0x7fef181420a0 2024-04-20T21:24:38.825 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:24:38.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.828+0000 7fef1d400640 1 -- 172.21.15.86:0/1277516803 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7feef406c8f0 msgr2=0x7feef406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:38.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.828+0000 7fef1d400640 1 --2- 172.21.15.86:0/1277516803 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7feef406c8f0 0x7feef406edb0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fef08007e90 tx=0x7fef080073a0 comp rx=0 tx=0).stop 2024-04-20T21:24:38.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.828+0000 7fef1d400640 1 -- 172.21.15.86:0/1277516803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef181420a0 msgr2=0x7fef18142610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:38.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.828+0000 7fef1d400640 1 --2- 172.21.15.86:0/1277516803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef181420a0 0x7fef18142610 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fef1000d830 tx=0x7fef1000dcf0 comp rx=0 tx=0).stop 2024-04-20T21:24:38.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.828+0000 7fef1d400640 1 -- 172.21.15.86:0/1277516803 shutdown_connections 2024-04-20T21:24:38.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.828+0000 7fef1d400640 1 --2- 172.21.15.86:0/1277516803 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7feef406c8f0 0x7feef406edb0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:38.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.828+0000 7fef1d400640 1 --2- 172.21.15.86:0/1277516803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef181420a0 0x7fef18142610 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:38.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.828+0000 7fef1d400640 1 --2- 172.21.15.86:0/1277516803 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef180b3200 0x7fef1809c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:38.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.828+0000 7fef1d400640 1 --2- 172.21.15.86:0/1277516803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef180af980 0x7fef18099700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:38.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.828+0000 7fef1d400640 1 -- 172.21.15.86:0/1277516803 >> 172.21.15.86:0/1277516803 conn(0x7fef180b1f00 msgr2=0x7fef18072f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:38.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.829+0000 7fef1d400640 1 -- 172.21.15.86:0/1277516803 shutdown_connections 2024-04-20T21:24:38.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:38.829+0000 7fef1d400640 1 -- 172.21.15.86:0/1277516803 wait complete. 2024-04-20T21:24:38.845 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:24:39.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.036+0000 7fe80c400640 1 Processor -- start 2024-04-20T21:24:39.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.036+0000 7fe80c400640 1 -- start start 2024-04-20T21:24:39.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.037+0000 7fe80c400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe80406bbf0 0x7fe80406e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:39.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.037+0000 7fe80c400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe80406bfd0 0x7fe804145b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:39.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.037+0000 7fe80c400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe80415cd40 con 0x7fe804146040 2024-04-20T21:24:39.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.037+0000 7fe80c400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fe80415d000 con 0x7fe80406bbf0 2024-04-20T21:24:39.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.037+0000 7fe80c400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe80415d2e0 con 0x7fe80406bfd0 2024-04-20T21:24:39.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.038+0000 7fe809000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe80406bfd0 0x7fe804145b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:39.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.038+0000 7fe809000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe80406bfd0 0x7fe804145b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:57570/0 (socket says 172.21.15.86:57570) 2024-04-20T21:24:39.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.038+0000 7fe809000640 1 -- 172.21.15.86:0/3479282616 learned_addr learned my addr 172.21.15.86:0/3479282616 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:39.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.038+0000 7fe809a00640 1 --2- 172.21.15.86:0/3479282616 >> v2:172.21.15.144:3300/0 conn(0x7fe80406bbf0 0x7fe80406e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:39.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.039+0000 7fe809000640 1 -- 172.21.15.86:0/3479282616 >> v2:172.21.15.144:3300/0 conn(0x7fe80406bbf0 msgr2=0x7fe80406e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:39.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.039+0000 7fe809000640 1 --2- 172.21.15.86:0/3479282616 >> v2:172.21.15.144:3300/0 conn(0x7fe80406bbf0 0x7fe80406e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:39.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.039+0000 7fe809000640 1 -- 172.21.15.86:0/3479282616 >> v1:172.21.15.144:6789/0 conn(0x7fe804146040 legacy=0x7fe8040cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:39.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.039+0000 7fe809000640 1 -- 172.21.15.86:0/3479282616 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe80415d5c0 con 0x7fe80406bfd0 2024-04-20T21:24:39.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.039+0000 7fe809a00640 1 --2- 172.21.15.86:0/3479282616 >> v2:172.21.15.144:3300/0 conn(0x7fe80406bbf0 0x7fe80406e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:24:39.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.041+0000 7fe809000640 1 --2- 172.21.15.86:0/3479282616 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe80406bfd0 0x7fe804145b00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0009870 tx=0x7fe7f002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c38dccc216e9258 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:39.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.041+0000 7fe803e00640 1 -- 172.21.15.86:0/3479282616 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f0034070 con 0x7fe80406bfd0 2024-04-20T21:24:39.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.041+0000 7fe803e00640 1 -- 172.21.15.86:0/3479282616 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7f0030840 con 0x7fe80406bfd0 2024-04-20T21:24:39.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.041+0000 7fe803e00640 1 -- 172.21.15.86:0/3479282616 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f0030c10 con 0x7fe80406bfd0 2024-04-20T21:24:39.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.042+0000 7fe80c400640 1 -- 172.21.15.86:0/3479282616 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe80406bfd0 msgr2=0x7fe804145b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:39.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.042+0000 7fe80c400640 1 --2- 172.21.15.86:0/3479282616 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe80406bfd0 0x7fe804145b00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0009870 tx=0x7fe7f002b980 comp rx=0 tx=0).stop 2024-04-20T21:24:39.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.042+0000 7fe80c400640 1 -- 172.21.15.86:0/3479282616 shutdown_connections 2024-04-20T21:24:39.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.042+0000 7fe80c400640 1 --2- 172.21.15.86:0/3479282616 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe80406bfd0 0x7fe804145b00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:39.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.042+0000 7fe80c400640 1 --2- 172.21.15.86:0/3479282616 >> v2:172.21.15.144:3300/0 conn(0x7fe80406bbf0 0x7fe80406e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:39.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.042+0000 7fe80c400640 1 -- 172.21.15.86:0/3479282616 >> 172.21.15.86:0/3479282616 conn(0x7fe8040b2110 msgr2=0x7fe8040b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:39.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.042+0000 7fe80c400640 1 -- 172.21.15.86:0/3479282616 shutdown_connections 2024-04-20T21:24:39.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.043+0000 7fe80c400640 1 -- 172.21.15.86:0/3479282616 wait complete. 2024-04-20T21:24:39.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.044+0000 7fe80c400640 1 Processor -- start 2024-04-20T21:24:39.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.044+0000 7fe80c400640 1 -- start start 2024-04-20T21:24:39.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.045+0000 7fe80c400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe80406bbf0 0x7fe8040b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:39.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.045+0000 7fe80c400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe80406bfd0 0x7fe8040b1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:39.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.045+0000 7fe80c400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe804146040 0x7fe8041e6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:39.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.045+0000 7fe80c400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe8040cddd0 con 0x7fe80406bfd0 2024-04-20T21:24:39.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.045+0000 7fe80c400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe8040cdc50 con 0x7fe804146040 2024-04-20T21:24:39.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.045+0000 7fe80c400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe8040cdf50 con 0x7fe80406bbf0 2024-04-20T21:24:39.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.046+0000 7fe809000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe80406bfd0 0x7fe8040b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:39.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.046+0000 7fe809000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe80406bfd0 0x7fe8040b1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51532/0 (socket says 172.21.15.86:51532) 2024-04-20T21:24:39.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.046+0000 7fe809000640 1 -- 172.21.15.86:0/1725378791 learned_addr learned my addr 172.21.15.86:0/1725378791 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:39.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.046+0000 7fe809a00640 1 --2- 172.21.15.86:0/1725378791 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe80406bbf0 0x7fe8040b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:39.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.046+0000 7fe80a400640 1 --2- 172.21.15.86:0/1725378791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe804146040 0x7fe8041e6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:39.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.046+0000 7fe809000640 1 -- 172.21.15.86:0/1725378791 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe80406bbf0 msgr2=0x7fe8040b0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:39.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.046+0000 7fe809000640 1 --2- 172.21.15.86:0/1725378791 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe80406bbf0 0x7fe8040b0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:39.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.046+0000 7fe809000640 1 -- 172.21.15.86:0/1725378791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe804146040 msgr2=0x7fe8041e6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:39.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.046+0000 7fe809000640 1 --2- 172.21.15.86:0/1725378791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe804146040 0x7fe8041e6e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:39.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.046+0000 7fe809000640 1 -- 172.21.15.86:0/1725378791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8041f8460 con 0x7fe80406bfd0 2024-04-20T21:24:39.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.046+0000 7fe809000640 1 --2- 172.21.15.86:0/1725378791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe80406bfd0 0x7fe8040b1410 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0000c00 tx=0x7fe7f0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:39.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.047+0000 7fe801600640 1 -- 172.21.15.86:0/1725378791 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f0048070 con 0x7fe80406bfd0 2024-04-20T21:24:39.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.047+0000 7fe801600640 1 -- 172.21.15.86:0/1725378791 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7f0030640 con 0x7fe80406bfd0 2024-04-20T21:24:39.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.047+0000 7fe80c400640 1 -- 172.21.15.86:0/1725378791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8041f8740 con 0x7fe80406bfd0 2024-04-20T21:24:39.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.047+0000 7fe801600640 1 -- 172.21.15.86:0/1725378791 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f0034040 con 0x7fe80406bfd0 2024-04-20T21:24:39.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.047+0000 7fe80c400640 1 -- 172.21.15.86:0/1725378791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8041f8d00 con 0x7fe80406bfd0 2024-04-20T21:24:39.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.048+0000 7fe80c400640 1 -- 172.21.15.86:0/1725378791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8040689e0 con 0x7fe80406bfd0 2024-04-20T21:24:39.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.049+0000 7fe801600640 1 -- 172.21.15.86:0/1725378791 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe7f004b070 con 0x7fe80406bfd0 2024-04-20T21:24:39.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.050+0000 7fe801600640 1 --2- 172.21.15.86:0/1725378791 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe7e406c8f0 0x7fe7e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:39.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.050+0000 7fe801600640 1 -- 172.21.15.86:0/1725378791 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5882+0+0 (secure 0 0 0) 0x7fe7f00ada70 con 0x7fe80406bfd0 2024-04-20T21:24:39.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.050+0000 7fe809a00640 1 --2- 172.21.15.86:0/1725378791 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe7e406c8f0 0x7fe7e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:39.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.051+0000 7fe809a00640 1 --2- 172.21.15.86:0/1725378791 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe7e406c8f0 0x7fe7e406edb0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8009560 tx=0x7fe7f8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:39.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.054+0000 7fe801600640 1 -- 172.21.15.86:0/1725378791 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe7f007bde0 con 0x7fe80406bfd0 2024-04-20T21:24:39.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.251+0000 7fe80c400640 1 -- 172.21.15.86:0/1725378791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe8040b5500 con 0x7fe80406bfd0 2024-04-20T21:24:39.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.252+0000 7fe801600640 1 -- 172.21.15.86:0/1725378791 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v74) ==== 104+0+11 (secure 0 0 0) 0x7fe7f007b6e0 con 0x7fe80406bfd0 2024-04-20T21:24:39.251 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-20T21:24:39.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.255+0000 7fe80c400640 1 -- 172.21.15.86:0/1725378791 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe7e406c8f0 msgr2=0x7fe7e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:39.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.255+0000 7fe80c400640 1 --2- 172.21.15.86:0/1725378791 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe7e406c8f0 0x7fe7e406edb0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8009560 tx=0x7fe7f8008040 comp rx=0 tx=0).stop 2024-04-20T21:24:39.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.255+0000 7fe80c400640 1 -- 172.21.15.86:0/1725378791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe80406bfd0 msgr2=0x7fe8040b1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:39.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.255+0000 7fe80c400640 1 --2- 172.21.15.86:0/1725378791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe80406bfd0 0x7fe8040b1410 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0000c00 tx=0x7fe7f0004730 comp rx=0 tx=0).stop 2024-04-20T21:24:39.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.255+0000 7fe80c400640 1 -- 172.21.15.86:0/1725378791 shutdown_connections 2024-04-20T21:24:39.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.255+0000 7fe80c400640 1 --2- 172.21.15.86:0/1725378791 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe7e406c8f0 0x7fe7e406edb0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:39.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.255+0000 7fe80c400640 1 --2- 172.21.15.86:0/1725378791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe804146040 0x7fe8041e6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:39.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.255+0000 7fe80c400640 1 --2- 172.21.15.86:0/1725378791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe80406bfd0 0x7fe8040b1410 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:39.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.255+0000 7fe80c400640 1 --2- 172.21.15.86:0/1725378791 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe80406bbf0 0x7fe8040b0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:39.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.255+0000 7fe80c400640 1 -- 172.21.15.86:0/1725378791 >> 172.21.15.86:0/1725378791 conn(0x7fe8040b2110 msgr2=0x7fe80415e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:39.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.255+0000 7fe80c400640 1 -- 172.21.15.86:0/1725378791 shutdown_connections 2024-04-20T21:24:39.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.255+0000 7fe80c400640 1 -- 172.21.15.86:0/1725378791 wait complete. 2024-04-20T21:24:39.273 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:24:39.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.467+0000 7f61e7e00640 1 Processor -- start 2024-04-20T21:24:39.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.468+0000 7f61e7e00640 1 -- start start 2024-04-20T21:24:39.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.469+0000 7f61e7e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61e815e870 0x7f61e806ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:39.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.469+0000 7f61e7e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f61e80d4310 0x7f61e80d4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:39.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.469+0000 7f61e7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f61e80cfb80 0x7f61e80d0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:39.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.469+0000 7f61e7e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f61e80cf150 con 0x7f61e80cfb80 2024-04-20T21:24:39.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.469+0000 7f61e7e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f61e80d35b0 con 0x7f61e80d4310 2024-04-20T21:24:39.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.469+0000 7f61e7e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f61e80d4cb0 con 0x7f61e815e870 2024-04-20T21:24:39.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.470+0000 7f61e7400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f61e80cfb80 0x7f61e80d0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:39.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.470+0000 7f61e7400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f61e80cfb80 0x7f61e80d0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51542/0 (socket says 172.21.15.86:51542) 2024-04-20T21:24:39.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.470+0000 7f61e7400640 1 -- 172.21.15.86:0/2159636812 learned_addr learned my addr 172.21.15.86:0/2159636812 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:39.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.470+0000 7f61e6a00640 1 --2- 172.21.15.86:0/2159636812 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61e815e870 0x7f61e806ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:39.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.470+0000 7f61dfe00640 1 --2- 172.21.15.86:0/2159636812 >> v2:172.21.15.144:3300/0 conn(0x7f61e80d4310 0x7f61e80d4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:39.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.470+0000 7f61e7400640 1 -- 172.21.15.86:0/2159636812 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61e815e870 msgr2=0x7f61e806ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:39.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.470+0000 7f61e7400640 1 --2- 172.21.15.86:0/2159636812 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61e815e870 0x7f61e806ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:39.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.470+0000 7f61e7400640 1 -- 172.21.15.86:0/2159636812 >> v2:172.21.15.144:3300/0 conn(0x7f61e80d4310 msgr2=0x7f61e80d4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:39.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.470+0000 7f61e7400640 1 --2- 172.21.15.86:0/2159636812 >> v2:172.21.15.144:3300/0 conn(0x7f61e80d4310 0x7f61e80d4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:39.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.470+0000 7f61e7400640 1 -- 172.21.15.86:0/2159636812 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e80d2ff0 con 0x7f61e80cfb80 2024-04-20T21:24:39.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.470+0000 7f61e6a00640 1 --2- 172.21.15.86:0/2159636812 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61e815e870 0x7f61e806ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:24:39.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.471+0000 7f61e7400640 1 --2- 172.21.15.86:0/2159636812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61e80cfb80 0x7f61e80d0010 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f61e80d2200 tx=0x7f61e002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37344c3a3820b456 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:39.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.472+0000 7f61e6000640 1 -- 172.21.15.86:0/2159636812 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e002ca80 con 0x7f61e80cfb80 2024-04-20T21:24:39.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.472+0000 7f61e6000640 1 -- 172.21.15.86:0/2159636812 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61e002cc20 con 0x7f61e80cfb80 2024-04-20T21:24:39.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.472+0000 7f61e6000640 1 -- 172.21.15.86:0/2159636812 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e00343f0 con 0x7f61e80cfb80 2024-04-20T21:24:39.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.472+0000 7f61e7e00640 1 -- 172.21.15.86:0/2159636812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61e80cfb80 msgr2=0x7f61e80d0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:39.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.472+0000 7f61e7e00640 1 --2- 172.21.15.86:0/2159636812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61e80cfb80 0x7f61e80d0010 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f61e80d2200 tx=0x7f61e002bee0 comp rx=0 tx=0).stop 2024-04-20T21:24:39.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.472+0000 7f61e7e00640 1 -- 172.21.15.86:0/2159636812 shutdown_connections 2024-04-20T21:24:39.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.472+0000 7f61e7e00640 1 --2- 172.21.15.86:0/2159636812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61e80cfb80 0x7f61e80d0010 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:39.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.472+0000 7f61e7e00640 1 --2- 172.21.15.86:0/2159636812 >> v2:172.21.15.144:3300/0 conn(0x7f61e80d4310 0x7f61e80d4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:39.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.472+0000 7f61e7e00640 1 --2- 172.21.15.86:0/2159636812 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61e815e870 0x7f61e806ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:39.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.472+0000 7f61e7e00640 1 -- 172.21.15.86:0/2159636812 >> 172.21.15.86:0/2159636812 conn(0x7f61e809d9c0 msgr2=0x7f61e808bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:39.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.472+0000 7f61e7e00640 1 -- 172.21.15.86:0/2159636812 shutdown_connections 2024-04-20T21:24:39.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.472+0000 7f61e7e00640 1 -- 172.21.15.86:0/2159636812 wait complete. 2024-04-20T21:24:39.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.474+0000 7f61e7e00640 1 Processor -- start 2024-04-20T21:24:39.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.474+0000 7f61e7e00640 1 -- start start 2024-04-20T21:24:39.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.475+0000 7f61e7e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61e80cfb80 0x7f61e8161470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:39.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.475+0000 7f61e7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61e80d4310 0x7f61e815bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:39.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.475+0000 7f61e7e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61e815e870 0x7f61e80cb750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:39.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.475+0000 7f61e7e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f61e80cf660 con 0x7f61e80d4310 2024-04-20T21:24:39.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.475+0000 7f61e7e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f61e80cf380 con 0x7f61e815e870 2024-04-20T21:24:39.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.475+0000 7f61e7e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f61e80d0550 con 0x7f61e80cfb80 2024-04-20T21:24:39.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.475+0000 7f61dfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61e80d4310 0x7f61e815bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:39.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.476+0000 7f61dfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61e80d4310 0x7f61e815bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51544/0 (socket says 172.21.15.86:51544) 2024-04-20T21:24:39.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.476+0000 7f61dfe00640 1 -- 172.21.15.86:0/3616626485 learned_addr learned my addr 172.21.15.86:0/3616626485 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:39.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.476+0000 7f61e6a00640 1 --2- 172.21.15.86:0/3616626485 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61e80cfb80 0x7f61e8161470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:39.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.476+0000 7f61e7400640 1 --2- 172.21.15.86:0/3616626485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61e815e870 0x7f61e80cb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:39.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.476+0000 7f61dfe00640 1 -- 172.21.15.86:0/3616626485 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61e80cfb80 msgr2=0x7f61e8161470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:39.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.476+0000 7f61dfe00640 1 --2- 172.21.15.86:0/3616626485 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61e80cfb80 0x7f61e8161470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:39.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.476+0000 7f61dfe00640 1 -- 172.21.15.86:0/3616626485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61e815e870 msgr2=0x7f61e80cb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:39.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.476+0000 7f61dfe00640 1 --2- 172.21.15.86:0/3616626485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61e815e870 0x7f61e80cb750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:39.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.476+0000 7f61dfe00640 1 -- 172.21.15.86:0/3616626485 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e80b5590 con 0x7f61e80d4310 2024-04-20T21:24:39.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.476+0000 7f61dfe00640 1 --2- 172.21.15.86:0/3616626485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61e80d4310 0x7f61e815bb90 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f61d000c7f0 tx=0x7f61d000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:39.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.477+0000 7f61dea00640 1 -- 172.21.15.86:0/3616626485 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d0015070 con 0x7f61e80d4310 2024-04-20T21:24:39.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.477+0000 7f61dea00640 1 -- 172.21.15.86:0/3616626485 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61d0004240 con 0x7f61e80d4310 2024-04-20T21:24:39.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.477+0000 7f61dea00640 1 -- 172.21.15.86:0/3616626485 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d0007620 con 0x7f61e80d4310 2024-04-20T21:24:39.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.477+0000 7f61e7e00640 1 -- 172.21.15.86:0/3616626485 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61e80b5810 con 0x7f61e80d4310 2024-04-20T21:24:39.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.477+0000 7f61e7e00640 1 -- 172.21.15.86:0/3616626485 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61e80b5d90 con 0x7f61e80d4310 2024-04-20T21:24:39.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.478+0000 7f61e7e00640 1 -- 172.21.15.86:0/3616626485 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61e804fd90 con 0x7f61e80d4310 2024-04-20T21:24:39.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.479+0000 7f61dea00640 1 -- 172.21.15.86:0/3616626485 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f61d00079d0 con 0x7f61e80d4310 2024-04-20T21:24:39.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.480+0000 7f61dea00640 1 --2- 172.21.15.86:0/3616626485 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f61c406c8f0 0x7f61c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:39.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.480+0000 7f61dea00640 1 -- 172.21.15.86:0/3616626485 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5882+0+0 (secure 0 0 0) 0x7f61d008bfe0 con 0x7f61e80d4310 2024-04-20T21:24:39.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.480+0000 7f61e6a00640 1 --2- 172.21.15.86:0/3616626485 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f61c406c8f0 0x7f61c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:39.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.481+0000 7f61e6a00640 1 --2- 172.21.15.86:0/3616626485 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f61c406c8f0 0x7f61c406edb0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f61d800a910 tx=0x7f61d8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:39.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.485+0000 7f61dea00640 1 -- 172.21.15.86:0/3616626485 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f61e80b5590 con 0x7f61e80d4310 2024-04-20T21:24:39.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.683+0000 7f61e7e00640 1 -- 172.21.15.86:0/3616626485 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f61e806dde0 con 0x7f61e80d4310 2024-04-20T21:24:39.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.685+0000 7f61dea00640 1 -- 172.21.15.86:0/3616626485 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v74) ==== 100+0+11 (secure 0 0 0) 0x7f61d005a390 con 0x7f61e80d4310 2024-04-20T21:24:39.684 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-20T21:24:39.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.687+0000 7f61e7e00640 1 -- 172.21.15.86:0/3616626485 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f61c406c8f0 msgr2=0x7f61c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:39.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.687+0000 7f61e7e00640 1 --2- 172.21.15.86:0/3616626485 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f61c406c8f0 0x7f61c406edb0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f61d800a910 tx=0x7f61d8008040 comp rx=0 tx=0).stop 2024-04-20T21:24:39.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.687+0000 7f61e7e00640 1 -- 172.21.15.86:0/3616626485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61e80d4310 msgr2=0x7f61e815bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:39.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.687+0000 7f61e7e00640 1 --2- 172.21.15.86:0/3616626485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61e80d4310 0x7f61e815bb90 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f61d000c7f0 tx=0x7f61d000ccb0 comp rx=0 tx=0).stop 2024-04-20T21:24:39.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.687+0000 7f61e7e00640 1 -- 172.21.15.86:0/3616626485 shutdown_connections 2024-04-20T21:24:39.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.687+0000 7f61e7e00640 1 --2- 172.21.15.86:0/3616626485 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f61c406c8f0 0x7f61c406edb0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:39.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.687+0000 7f61e7e00640 1 --2- 172.21.15.86:0/3616626485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61e815e870 0x7f61e80cb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:39.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.687+0000 7f61e7e00640 1 --2- 172.21.15.86:0/3616626485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61e80d4310 0x7f61e815bb90 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:39.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.687+0000 7f61e7e00640 1 --2- 172.21.15.86:0/3616626485 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61e80cfb80 0x7f61e8161470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:39.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.687+0000 7f61e7e00640 1 -- 172.21.15.86:0/3616626485 >> 172.21.15.86:0/3616626485 conn(0x7f61e809d9c0 msgr2=0x7f61e80ca1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:39.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.688+0000 7f61e7e00640 1 -- 172.21.15.86:0/3616626485 shutdown_connections 2024-04-20T21:24:39.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.688+0000 7f61e7e00640 1 -- 172.21.15.86:0/3616626485 wait complete. 2024-04-20T21:24:39.704 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T21:24:39.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.908+0000 7ffba2e00640 1 Processor -- start 2024-04-20T21:24:39.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.908+0000 7ffba2e00640 1 -- start start 2024-04-20T21:24:39.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.910+0000 7ffba2e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb9c145fe0 0x7ffb9c0d02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:39.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.910+0000 7ffba2e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ffb9c0cd7e0 0x7ffb9c071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:39.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.910+0000 7ffba2e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb9c15dcc0 con 0x7ffb9c0d0810 2024-04-20T21:24:39.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.910+0000 7ffba2e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7ffb9c0d00d0 con 0x7ffb9c0cd7e0 2024-04-20T21:24:39.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.910+0000 7ffba2e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ffb9c0d0c70 con 0x7ffb9c145fe0 2024-04-20T21:24:39.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.910+0000 7ffb9b400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7ffb9c0d0810 0x7ffb9c0cd0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57992/0 (socket says 172.21.15.86:57992) 2024-04-20T21:24:39.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.910+0000 7ffb9b400640 1 -- 172.21.15.86:0/2161492304 learned_addr learned my addr 172.21.15.86:0/2161492304 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:24:39.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.911+0000 7ffb9be00640 1 --2- 172.21.15.86:0/2161492304 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb9c145fe0 0x7ffb9c0d02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:39.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.911+0000 7ffba0e00640 1 --2- 172.21.15.86:0/2161492304 >> v2:172.21.15.144:3300/0 conn(0x7ffb9c0cd7e0 0x7ffb9c071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:39.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.911+0000 7ffb9aa00640 1 -- 172.21.15.86:0/2161492304 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1085385350 0 0) 0x7ffb9c15dcc0 con 0x7ffb9c0d0810 2024-04-20T21:24:39.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.911+0000 7ffb9aa00640 1 -- 172.21.15.86:0/2161492304 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb80003520 con 0x7ffb9c0d0810 2024-04-20T21:24:39.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.911+0000 7ffb9aa00640 1 -- 172.21.15.86:0/2161492304 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 304319926 0 0) 0x7ffb80003520 con 0x7ffb9c0d0810 2024-04-20T21:24:39.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.911+0000 7ffb9aa00640 1 -- 172.21.15.86:0/2161492304 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb9c145fe0 msgr2=0x7ffb9c0d02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:39.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.911+0000 7ffb9aa00640 1 --2- 172.21.15.86:0/2161492304 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb9c145fe0 0x7ffb9c0d02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:39.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.911+0000 7ffb9aa00640 1 -- 172.21.15.86:0/2161492304 >> v2:172.21.15.144:3300/0 conn(0x7ffb9c0cd7e0 msgr2=0x7ffb9c071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:39.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.911+0000 7ffb9aa00640 1 --2- 172.21.15.86:0/2161492304 >> v2:172.21.15.144:3300/0 conn(0x7ffb9c0cd7e0 0x7ffb9c071cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:39.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.911+0000 7ffb9aa00640 1 -- 172.21.15.86:0/2161492304 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb9c15cf00 con 0x7ffb9c0d0810 2024-04-20T21:24:39.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.911+0000 7ffb9aa00640 1 -- 172.21.15.86:0/2161492304 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7ffb84001670 con 0x7ffb9c0d0810 2024-04-20T21:24:39.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.911+0000 7ffb9aa00640 1 -- 172.21.15.86:0/2161492304 >> v1:172.21.15.86:6789/0 conn(0x7ffb9c0d0810 legacy=0x7ffb9c0cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:39.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.911+0000 7ffb9aa00640 1 --2- 172.21.15.86:0/2161492304 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb9c145fe0 0x7ffb800039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:39.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.911+0000 7ffb9aa00640 1 --2- 172.21.15.86:0/2161492304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb80003f30 0x7ffb80004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:39.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.911+0000 7ffb9aa00640 1 --2- 172.21.15.86:0/2161492304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb80004d40 0x7ffb800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:39.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.911+0000 7ffb9aa00640 1 -- 172.21.15.86:0/2161492304 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9c0d00d0 con 0x7ffb80003f30 2024-04-20T21:24:39.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.911+0000 7ffb9aa00640 1 -- 172.21.15.86:0/2161492304 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8000b9c0 con 0x7ffb80004d40 2024-04-20T21:24:39.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.911+0000 7ffb9aa00640 1 -- 172.21.15.86:0/2161492304 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ffb8000bb40 con 0x7ffb9c145fe0 2024-04-20T21:24:39.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.912+0000 7ffb9b400640 1 --2- 172.21.15.86:0/2161492304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb80003f30 0x7ffb80004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:39.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.912+0000 7ffb9be00640 1 --2- 172.21.15.86:0/2161492304 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb9c145fe0 0x7ffb800039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:39.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.912+0000 7ffb9b400640 1 -- 172.21.15.86:0/2161492304 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb9c145fe0 msgr2=0x7ffb800039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:39.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.912+0000 7ffb9b400640 1 --2- 172.21.15.86:0/2161492304 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb9c145fe0 0x7ffb800039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:39.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.912+0000 7ffba0e00640 1 --2- 172.21.15.86:0/2161492304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb80004d40 0x7ffb800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:39.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.912+0000 7ffb9b400640 1 -- 172.21.15.86:0/2161492304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb80004d40 msgr2=0x7ffb800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:39.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.912+0000 7ffb9b400640 1 --2- 172.21.15.86:0/2161492304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb80004d40 0x7ffb800051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:39.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.912+0000 7ffb9b400640 1 -- 172.21.15.86:0/2161492304 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb80003520 con 0x7ffb80003f30 2024-04-20T21:24:39.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.913+0000 7ffb9b400640 1 --2- 172.21.15.86:0/2161492304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb80003f30 0x7ffb80004800 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7ffb8400c240 tx=0x7ffb80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1620dd23b6a9b0c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:39.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.913+0000 7ffb9aa00640 1 -- 172.21.15.86:0/2161492304 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb84007d30 con 0x7ffb80003f30 2024-04-20T21:24:39.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.913+0000 7ffb9aa00640 1 -- 172.21.15.86:0/2161492304 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb84030070 con 0x7ffb80003f30 2024-04-20T21:24:39.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.913+0000 7ffb9aa00640 1 -- 172.21.15.86:0/2161492304 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8402cda0 con 0x7ffb80003f30 2024-04-20T21:24:39.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.914+0000 7ffba2e00640 1 -- 172.21.15.86:0/2161492304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb80003f30 msgr2=0x7ffb80004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:39.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.914+0000 7ffba2e00640 1 --2- 172.21.15.86:0/2161492304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb80003f30 0x7ffb80004800 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7ffb8400c240 tx=0x7ffb80005830 comp rx=0 tx=0).stop 2024-04-20T21:24:39.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.914+0000 7ffba0e00640 1 -- 172.21.15.86:0/2161492304 reap_dead start 2024-04-20T21:24:39.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.914+0000 7ffba2e00640 1 -- 172.21.15.86:0/2161492304 shutdown_connections 2024-04-20T21:24:39.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.914+0000 7ffba2e00640 1 -- 172.21.15.86:0/2161492304 >> 172.21.15.86:0/2161492304 conn(0x7ffb9c06b980 msgr2=0x7ffb9c06bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:39.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.914+0000 7ffba2e00640 1 -- 172.21.15.86:0/2161492304 shutdown_connections 2024-04-20T21:24:39.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.914+0000 7ffba2e00640 1 -- 172.21.15.86:0/2161492304 wait complete. 2024-04-20T21:24:39.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.916+0000 7ffba2e00640 1 Processor -- start 2024-04-20T21:24:39.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.916+0000 7ffba2e00640 1 -- start start 2024-04-20T21:24:39.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.917+0000 7ffba2e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb80003f30 0x7ffb9c0b2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:39.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.917+0000 7ffba2e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb9c133b70 0x7ffb9c1327c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:39.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.917+0000 7ffba2e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb9c130ed0 0x7ffb9c132f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:39.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.917+0000 7ffba2e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9c15d7a0 con 0x7ffb9c130ed0 2024-04-20T21:24:39.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.917+0000 7ffba2e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9c15d4c0 con 0x7ffb80003f30 2024-04-20T21:24:39.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.917+0000 7ffba2e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ffb9c1463c0 con 0x7ffb9c133b70 2024-04-20T21:24:39.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.917+0000 7ffba0e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb9c130ed0 0x7ffb9c132f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:39.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.917+0000 7ffba0e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb9c130ed0 0x7ffb9c132f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51558/0 (socket says 172.21.15.86:51558) 2024-04-20T21:24:39.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.917+0000 7ffba0e00640 1 -- 172.21.15.86:0/445667709 learned_addr learned my addr 172.21.15.86:0/445667709 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:39.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.918+0000 7ffb9be00640 1 --2- 172.21.15.86:0/445667709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb80003f30 0x7ffb9c0b2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:39.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.918+0000 7ffba0e00640 1 -- 172.21.15.86:0/445667709 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb9c133b70 msgr2=0x7ffb9c1327c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:39.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.918+0000 7ffba0e00640 1 --2- 172.21.15.86:0/445667709 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb9c133b70 0x7ffb9c1327c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:39.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.918+0000 7ffba0e00640 1 -- 172.21.15.86:0/445667709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb80003f30 msgr2=0x7ffb9c0b2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:39.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.918+0000 7ffba0e00640 1 --2- 172.21.15.86:0/445667709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb80003f30 0x7ffb9c0b2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:39.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.918+0000 7ffba0e00640 1 -- 172.21.15.86:0/445667709 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb9c132d00 con 0x7ffb9c130ed0 2024-04-20T21:24:39.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.918+0000 7ffb9be00640 1 --2- 172.21.15.86:0/445667709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb80003f30 0x7ffb9c0b2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:24:39.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.918+0000 7ffba0e00640 1 --2- 172.21.15.86:0/445667709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb9c130ed0 0x7ffb9c132f60 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c0b5660 tx=0x7ffb94010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:39.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.918+0000 7ffb93e00640 1 -- 172.21.15.86:0/445667709 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9400f040 con 0x7ffb9c130ed0 2024-04-20T21:24:39.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.918+0000 7ffb93e00640 1 -- 172.21.15.86:0/445667709 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb9401c070 con 0x7ffb9c130ed0 2024-04-20T21:24:39.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.918+0000 7ffb93e00640 1 -- 172.21.15.86:0/445667709 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9400eb00 con 0x7ffb9c130ed0 2024-04-20T21:24:39.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.918+0000 7ffba2e00640 1 -- 172.21.15.86:0/445667709 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb9c131720 con 0x7ffb9c130ed0 2024-04-20T21:24:39.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.918+0000 7ffba2e00640 1 -- 172.21.15.86:0/445667709 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb9c131ce0 con 0x7ffb9c130ed0 2024-04-20T21:24:39.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.919+0000 7ffba2e00640 1 -- 172.21.15.86:0/445667709 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb9c070580 con 0x7ffb9c130ed0 2024-04-20T21:24:39.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.920+0000 7ffb93e00640 1 -- 172.21.15.86:0/445667709 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffb9400eca0 con 0x7ffb9c130ed0 2024-04-20T21:24:39.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.921+0000 7ffb93e00640 1 --2- 172.21.15.86:0/445667709 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ffb7806c8f0 0x7ffb7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:39.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.921+0000 7ffb93e00640 1 -- 172.21.15.86:0/445667709 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5882+0+0 (secure 0 0 0) 0x7ffb9408b9f0 con 0x7ffb9c130ed0 2024-04-20T21:24:39.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.921+0000 7ffb9be00640 1 --2- 172.21.15.86:0/445667709 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ffb7806c8f0 0x7ffb7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:39.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.922+0000 7ffb9be00640 1 --2- 172.21.15.86:0/445667709 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ffb7806c8f0 0x7ffb7806edb0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c012430 tx=0x7ffb8c00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:39.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:39.926+0000 7ffb93e00640 1 -- 172.21.15.86:0/445667709 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ffb9c070580 con 0x7ffb9c130ed0 2024-04-20T21:24:40.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.140+0000 7ffba2e00640 1 -- 172.21.15.86:0/445667709 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ffb9c0689e0 con 0x7ffb9c130ed0 2024-04-20T21:24:40.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.141+0000 7ffb93e00640 1 -- 172.21.15.86:0/445667709 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v74) ==== 80+0+120 (secure 0 0 0) 0x7ffb94026020 con 0x7ffb9c130ed0 2024-04-20T21:24:40.140 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6837/202143875 2024-04-21T21:23:12.079572+0000 2024-04-20T21:24:40.140 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6835/202143875 2024-04-21T21:23:12.079572+0000 2024-04-20T21:24:40.140 INFO:teuthology.orchestra.run.smithi086.stderr:listed 2 entries 2024-04-20T21:24:40.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.143+0000 7ffba2e00640 1 -- 172.21.15.86:0/445667709 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ffb7806c8f0 msgr2=0x7ffb7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:40.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.143+0000 7ffba2e00640 1 --2- 172.21.15.86:0/445667709 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ffb7806c8f0 0x7ffb7806edb0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c012430 tx=0x7ffb8c00b040 comp rx=0 tx=0).stop 2024-04-20T21:24:40.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.143+0000 7ffba2e00640 1 -- 172.21.15.86:0/445667709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb9c130ed0 msgr2=0x7ffb9c132f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:40.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.143+0000 7ffba2e00640 1 --2- 172.21.15.86:0/445667709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb9c130ed0 0x7ffb9c132f60 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c0b5660 tx=0x7ffb94010c20 comp rx=0 tx=0).stop 2024-04-20T21:24:40.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.144+0000 7ffba2e00640 1 -- 172.21.15.86:0/445667709 shutdown_connections 2024-04-20T21:24:40.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.144+0000 7ffba2e00640 1 --2- 172.21.15.86:0/445667709 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ffb7806c8f0 0x7ffb7806edb0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:40.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.144+0000 7ffba2e00640 1 --2- 172.21.15.86:0/445667709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb9c130ed0 0x7ffb9c132f60 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:40.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.144+0000 7ffba2e00640 1 --2- 172.21.15.86:0/445667709 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb9c133b70 0x7ffb9c1327c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:40.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.144+0000 7ffba2e00640 1 --2- 172.21.15.86:0/445667709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb80003f30 0x7ffb9c0b2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:40.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.144+0000 7ffba2e00640 1 -- 172.21.15.86:0/445667709 >> 172.21.15.86:0/445667709 conn(0x7ffb9c06b980 msgr2=0x7ffb9c1601b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:40.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.144+0000 7ffba2e00640 1 -- 172.21.15.86:0/445667709 shutdown_connections 2024-04-20T21:24:40.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.144+0000 7ffba2e00640 1 -- 172.21.15.86:0/445667709 wait complete. 2024-04-20T21:24:40.161 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi086.front.sepia.ceph.com... 2024-04-20T21:24:40.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:24:40.162 DEBUG:teuthology.orchestra.run.smithi086:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-20T21:24:40.238 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[50375]: fuse finished with error 0 and tester_r 0 2024-04-20T21:24:40.243 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-20T21:24:40.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:24:40.243 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-20T21:24:40.418 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T21:24:40.421 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:24:40.421 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:24:40.477 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T21:24:40.477 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2024-04-20T21:24:40.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:24:40.477 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-20T21:24:40.527 INFO:teuthology.orchestra.run.smithi144.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-20T21:24:40.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:24:40.528 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T21:24:40.528 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:24:40.528 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:24:40.585 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052731,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248996k,nr_inodes=812249,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T21:24:40.585 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-20T21:24:40.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:24:40.585 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-20T21:24:40.635 INFO:teuthology.orchestra.run.smithi086.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-20T21:24:40.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:24:40.636 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T21:24:40.637 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:24:40.637 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:24:40.692 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T21:24:40.692 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2024-04-20T21:24:40.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:24:40.692 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-20T21:24:40.742 INFO:teuthology.orchestra.run.smithi144.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-20T21:24:40.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:24:40.744 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:24:40.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.948+0000 7f96b9200640 1 Processor -- start 2024-04-20T21:24:40.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.949+0000 7f96b9200640 1 -- start start 2024-04-20T21:24:40.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.950+0000 7f96b9200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f96b4146130 0x7f96b406e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:40.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.950+0000 7f96b9200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f96b4072410 0x7f96b40cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:40.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.950+0000 7f96b9200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96b406dc30 con 0x7f96b40ccdd0 2024-04-20T21:24:40.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.950+0000 7f96b9200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f96b40cd8e0 con 0x7f96b4072410 2024-04-20T21:24:40.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.950+0000 7f96b9200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f96b40cdbc0 con 0x7f96b4146130 2024-04-20T21:24:40.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.950+0000 7f96b2800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f96b4072410 0x7f96b40cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:40.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.951+0000 7f96b2800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f96b4072410 0x7f96b40cd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51564/0 (socket says 172.21.15.86:51564) 2024-04-20T21:24:40.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.951+0000 7f96b2800640 1 -- 172.21.15.86:0/327921058 learned_addr learned my addr 172.21.15.86:0/327921058 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:40.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.951+0000 7f96b1e00640 1 --2- 172.21.15.86:0/327921058 >> v2:172.21.15.144:3300/0 conn(0x7f96b4146130 0x7f96b406e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:40.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.951+0000 7f96b2800640 1 -- 172.21.15.86:0/327921058 >> v2:172.21.15.144:3300/0 conn(0x7f96b4146130 msgr2=0x7f96b406e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:40.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.951+0000 7f96b2800640 1 --2- 172.21.15.86:0/327921058 >> v2:172.21.15.144:3300/0 conn(0x7f96b4146130 0x7f96b406e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:40.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.951+0000 7f96b2800640 1 -- 172.21.15.86:0/327921058 >> v1:172.21.15.86:6789/0 conn(0x7f96b40ccdd0 legacy=0x7f96b4071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:40.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.951+0000 7f96b2800640 1 -- 172.21.15.86:0/327921058 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b406ded0 con 0x7f96b4072410 2024-04-20T21:24:40.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.952+0000 7f96b2800640 1 --2- 172.21.15.86:0/327921058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b4072410 0x7f96b40cd230 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f96b40b5950 tx=0x7f96ac02ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=958f533b1307a502 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:40.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.952+0000 7f96b1400640 1 -- 172.21.15.86:0/327921058 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ac02f850 con 0x7f96b4072410 2024-04-20T21:24:40.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.952+0000 7f96b1400640 1 -- 172.21.15.86:0/327921058 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96ac02f9f0 con 0x7f96b4072410 2024-04-20T21:24:40.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.952+0000 7f96b1400640 1 -- 172.21.15.86:0/327921058 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ac02fdc0 con 0x7f96b4072410 2024-04-20T21:24:40.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.953+0000 7f96b9200640 1 -- 172.21.15.86:0/327921058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b4072410 msgr2=0x7f96b40cd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:40.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.953+0000 7f96b9200640 1 --2- 172.21.15.86:0/327921058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b4072410 0x7f96b40cd230 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f96b40b5950 tx=0x7f96ac02ba30 comp rx=0 tx=0).stop 2024-04-20T21:24:40.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.953+0000 7f96b9200640 1 -- 172.21.15.86:0/327921058 shutdown_connections 2024-04-20T21:24:40.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.953+0000 7f96b9200640 1 --2- 172.21.15.86:0/327921058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b4072410 0x7f96b40cd230 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:40.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.953+0000 7f96b9200640 1 --2- 172.21.15.86:0/327921058 >> v2:172.21.15.144:3300/0 conn(0x7f96b4146130 0x7f96b406e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:40.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.953+0000 7f96b9200640 1 -- 172.21.15.86:0/327921058 >> 172.21.15.86:0/327921058 conn(0x7f96b40b2360 msgr2=0x7f96b40b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:40.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.953+0000 7f96b9200640 1 -- 172.21.15.86:0/327921058 shutdown_connections 2024-04-20T21:24:40.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.953+0000 7f96b9200640 1 -- 172.21.15.86:0/327921058 wait complete. 2024-04-20T21:24:40.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.955+0000 7f96b9200640 1 Processor -- start 2024-04-20T21:24:40.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.955+0000 7f96b9200640 1 -- start start 2024-04-20T21:24:40.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.956+0000 7f96b9200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96b4072410 0x7f96b4140270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:40.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.956+0000 7f96b9200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b40ccdd0 0x7f96b413cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:40.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.956+0000 7f96b9200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96b4146130 0x7f96b413e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:40.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.956+0000 7f96b9200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f96b415cc20 con 0x7f96b40ccdd0 2024-04-20T21:24:40.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.956+0000 7f96b9200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f96b406f150 con 0x7f96b4072410 2024-04-20T21:24:40.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.956+0000 7f96b9200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f96b415cda0 con 0x7f96b4146130 2024-04-20T21:24:40.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.957+0000 7f96a9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b40ccdd0 0x7f96b413cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:40.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.957+0000 7f96a9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b40ccdd0 0x7f96b413cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51566/0 (socket says 172.21.15.86:51566) 2024-04-20T21:24:40.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.957+0000 7f96a9400640 1 -- 172.21.15.86:0/372407029 learned_addr learned my addr 172.21.15.86:0/372407029 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:40.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.957+0000 7f96b1e00640 1 --2- 172.21.15.86:0/372407029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96b4072410 0x7f96b4140270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:40.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.957+0000 7f96a9400640 1 -- 172.21.15.86:0/372407029 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96b4146130 msgr2=0x7f96b413e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:40.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.957+0000 7f96b2800640 1 --2- 172.21.15.86:0/372407029 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96b4146130 0x7f96b413e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:40.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.957+0000 7f96a9400640 1 --2- 172.21.15.86:0/372407029 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96b4146130 0x7f96b413e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:40.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.957+0000 7f96a9400640 1 -- 172.21.15.86:0/372407029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96b4072410 msgr2=0x7f96b4140270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:40.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.957+0000 7f96a9400640 1 --2- 172.21.15.86:0/372407029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96b4072410 0x7f96b4140270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:40.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.957+0000 7f96a9400640 1 -- 172.21.15.86:0/372407029 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b40ad470 con 0x7f96b40ccdd0 2024-04-20T21:24:40.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.957+0000 7f96a9400640 1 --2- 172.21.15.86:0/372407029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b40ccdd0 0x7f96b413cf10 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f969c00c430 tx=0x7f969c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:40.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.958+0000 7f96aaa00640 1 -- 172.21.15.86:0/372407029 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969c017070 con 0x7f96b40ccdd0 2024-04-20T21:24:40.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.958+0000 7f96aaa00640 1 -- 172.21.15.86:0/372407029 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f969c0052d0 con 0x7f96b40ccdd0 2024-04-20T21:24:40.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.958+0000 7f96aaa00640 1 -- 172.21.15.86:0/372407029 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969c003850 con 0x7f96b40ccdd0 2024-04-20T21:24:40.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.958+0000 7f96b9200640 1 -- 172.21.15.86:0/372407029 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96b413ec40 con 0x7f96b40ccdd0 2024-04-20T21:24:40.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.958+0000 7f96b9200640 1 -- 172.21.15.86:0/372407029 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96b413f200 con 0x7f96b40ccdd0 2024-04-20T21:24:40.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.959+0000 7f96b9200640 1 -- 172.21.15.86:0/372407029 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96b40681f0 con 0x7f96b40ccdd0 2024-04-20T21:24:40.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.960+0000 7f96aaa00640 1 -- 172.21.15.86:0/372407029 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f969c01a070 con 0x7f96b40ccdd0 2024-04-20T21:24:40.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.961+0000 7f96aaa00640 1 --2- 172.21.15.86:0/372407029 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f969006c920 0x7f969006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:40.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.961+0000 7f96aaa00640 1 -- 172.21.15.86:0/372407029 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5882+0+0 (secure 0 0 0) 0x7f969c08afa0 con 0x7f96b40ccdd0 2024-04-20T21:24:40.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.961+0000 7f96b1e00640 1 --2- 172.21.15.86:0/372407029 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f969006c920 0x7f969006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:40.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.962+0000 7f96b1e00640 1 --2- 172.21.15.86:0/372407029 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f969006c920 0x7f969006ede0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f96a4009e90 tx=0x7f96a400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:40.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:40.965+0000 7f96aaa00640 1 -- 172.21.15.86:0/372407029 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f96b40681f0 con 0x7f96b40ccdd0 2024-04-20T21:24:41.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.207+0000 7f96b9200640 1 -- 172.21.15.86:0/372407029 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f96b40b2740 con 0x7f96b40ccdd0 2024-04-20T21:24:41.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.208+0000 7f96aaa00640 1 -- 172.21.15.86:0/372407029 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+5206 (secure 0 0 0) 0x7f96b40b2740 con 0x7f96b40ccdd0 2024-04-20T21:24:41.207 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:24:41.207 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":50,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2740872549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2740872549},{"type":"v1","addr":"172.21.15.86:6836","nonce":2740872549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5048,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/301899138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":301899138},{"type":"v1","addr":"172.21.15.144:6833","nonce":301899138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5051,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2816343379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2816343379},{"type":"v1","addr":"172.21.15.144:6835","nonce":2816343379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:22:45.431100+0000","modified":"2024-04-20T21:24:11.116928+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6987},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6987":{"gid":6987,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":18,"addr":"172.21.15.86:6837/2254117576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2254117576},{"type":"v1","addr":"172.21.15.86:6837","nonce":2254117576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6987,"qdb_cluster":[6987]},"id":5}]} 2024-04-20T21:24:41.207 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 50 2024-04-20T21:24:41.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.210+0000 7f96b9200640 1 -- 172.21.15.86:0/372407029 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f969006c920 msgr2=0x7f969006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:41.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.210+0000 7f96b9200640 1 --2- 172.21.15.86:0/372407029 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f969006c920 0x7f969006ede0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f96a4009e90 tx=0x7f96a400b040 comp rx=0 tx=0).stop 2024-04-20T21:24:41.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.210+0000 7f96b9200640 1 -- 172.21.15.86:0/372407029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b40ccdd0 msgr2=0x7f96b413cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:41.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.210+0000 7f96b9200640 1 --2- 172.21.15.86:0/372407029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b40ccdd0 0x7f96b413cf10 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f969c00c430 tx=0x7f969c00c8f0 comp rx=0 tx=0).stop 2024-04-20T21:24:41.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.211+0000 7f96b9200640 1 -- 172.21.15.86:0/372407029 shutdown_connections 2024-04-20T21:24:41.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.211+0000 7f96b9200640 1 --2- 172.21.15.86:0/372407029 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f969006c920 0x7f969006ede0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:41.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.211+0000 7f96b9200640 1 --2- 172.21.15.86:0/372407029 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96b4146130 0x7f96b413e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:41.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.211+0000 7f96b9200640 1 --2- 172.21.15.86:0/372407029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96b40ccdd0 0x7f96b413cf10 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:41.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.211+0000 7f96b9200640 1 --2- 172.21.15.86:0/372407029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96b4072410 0x7f96b4140270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:41.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.211+0000 7f96b9200640 1 -- 172.21.15.86:0/372407029 >> 172.21.15.86:0/372407029 conn(0x7f96b40b2360 msgr2=0x7f96b415eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:41.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.211+0000 7f96b9200640 1 -- 172.21.15.86:0/372407029 shutdown_connections 2024-04-20T21:24:41.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.211+0000 7f96b9200640 1 -- 172.21.15.86:0/372407029 wait complete. 2024-04-20T21:24:41.230 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:24:41.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.420+0000 7f413ce00640 1 Processor -- start 2024-04-20T21:24:41.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.421+0000 7f413ce00640 1 -- start start 2024-04-20T21:24:41.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.422+0000 7f413ce00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f41380d50c0 0x7f41380d3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:41.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.422+0000 7f413ce00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f41380cf9c0 0x7f4138160000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:41.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.422+0000 7f413ce00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41380d3950 con 0x7f413815ece0 2024-04-20T21:24:41.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.422+0000 7f413ce00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f41380cec00 con 0x7f41380cf9c0 2024-04-20T21:24:41.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.422+0000 7f413ce00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f41380ceee0 con 0x7f41380d50c0 2024-04-20T21:24:41.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.422+0000 7f4137e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f41380cf9c0 0x7f4138160000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:41.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.422+0000 7f4137e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f41380cf9c0 0x7f4138160000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51568/0 (socket says 172.21.15.86:51568) 2024-04-20T21:24:41.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.422+0000 7f4137e00640 1 -- 172.21.15.86:0/1048049521 learned_addr learned my addr 172.21.15.86:0/1048049521 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:41.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.423+0000 7f412ea00640 1 --2- 172.21.15.86:0/1048049521 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f41380d50c0 0x7f41380d3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:41.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.423+0000 7f4137e00640 1 -- 172.21.15.86:0/1048049521 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f41380d50c0 msgr2=0x7f41380d3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:41.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.423+0000 7f4137e00640 1 --2- 172.21.15.86:0/1048049521 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f41380d50c0 0x7f41380d3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:41.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.423+0000 7f4137e00640 1 -- 172.21.15.86:0/1048049521 >> v1:172.21.15.144:6789/0 conn(0x7f413815ece0 legacy=0x7f413815f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:41.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.423+0000 7f4137e00640 1 -- 172.21.15.86:0/1048049521 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41380cf1c0 con 0x7f41380cf9c0 2024-04-20T21:24:41.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.424+0000 7f4137e00640 1 --2- 172.21.15.86:0/1048049521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41380cf9c0 0x7f4138160000 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f4130009e50 tx=0x7f413002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4d18f8361063b1e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:41.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.424+0000 7f4136a00640 1 -- 172.21.15.86:0/1048049521 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413002c9e0 con 0x7f41380cf9c0 2024-04-20T21:24:41.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.424+0000 7f4136a00640 1 -- 172.21.15.86:0/1048049521 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f413002cb80 con 0x7f41380cf9c0 2024-04-20T21:24:41.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.424+0000 7f4136a00640 1 -- 172.21.15.86:0/1048049521 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41300343f0 con 0x7f41380cf9c0 2024-04-20T21:24:41.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.425+0000 7f413ce00640 1 -- 172.21.15.86:0/1048049521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41380cf9c0 msgr2=0x7f4138160000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:41.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.425+0000 7f413ce00640 1 --2- 172.21.15.86:0/1048049521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41380cf9c0 0x7f4138160000 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f4130009e50 tx=0x7f413002be40 comp rx=0 tx=0).stop 2024-04-20T21:24:41.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.425+0000 7f413ce00640 1 -- 172.21.15.86:0/1048049521 shutdown_connections 2024-04-20T21:24:41.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.425+0000 7f413ce00640 1 --2- 172.21.15.86:0/1048049521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41380cf9c0 0x7f4138160000 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:41.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.425+0000 7f413ce00640 1 --2- 172.21.15.86:0/1048049521 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f41380d50c0 0x7f41380d3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:41.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.425+0000 7f413ce00640 1 -- 172.21.15.86:0/1048049521 >> 172.21.15.86:0/1048049521 conn(0x7f413808bf70 msgr2=0x7f41380d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:41.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.425+0000 7f413ce00640 1 -- 172.21.15.86:0/1048049521 shutdown_connections 2024-04-20T21:24:41.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.425+0000 7f413ce00640 1 -- 172.21.15.86:0/1048049521 wait complete. 2024-04-20T21:24:41.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.427+0000 7f413ce00640 1 Processor -- start 2024-04-20T21:24:41.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.428+0000 7f413ce00640 1 -- start start 2024-04-20T21:24:41.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.429+0000 7f413ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41380cf9c0 0x7f413814ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:41.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.429+0000 7f413ce00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41380d50c0 0x7f41380cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:41.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.429+0000 7f413ce00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f413815ece0 0x7f41380b9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:41.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.429+0000 7f413ce00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f413815ea50 con 0x7f41380cf9c0 2024-04-20T21:24:41.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.429+0000 7f413ce00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f41380d3640 con 0x7f41380d50c0 2024-04-20T21:24:41.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.429+0000 7f413ce00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f41380cfe40 con 0x7f413815ece0 2024-04-20T21:24:41.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.429+0000 7f4137400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41380cf9c0 0x7f413814ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:41.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.429+0000 7f4137400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41380cf9c0 0x7f413814ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51578/0 (socket says 172.21.15.86:51578) 2024-04-20T21:24:41.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.429+0000 7f4137400640 1 -- 172.21.15.86:0/410399898 learned_addr learned my addr 172.21.15.86:0/410399898 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:41.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.429+0000 7f412ea00640 1 --2- 172.21.15.86:0/410399898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41380d50c0 0x7f41380cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:41.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.429+0000 7f4137400640 1 -- 172.21.15.86:0/410399898 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f413815ece0 msgr2=0x7f41380b9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:41.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.429+0000 7f4137400640 1 --2- 172.21.15.86:0/410399898 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f413815ece0 0x7f41380b9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:41.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.429+0000 7f4137400640 1 -- 172.21.15.86:0/410399898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41380d50c0 msgr2=0x7f41380cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:41.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.429+0000 7f4137400640 1 --2- 172.21.15.86:0/410399898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41380d50c0 0x7f41380cbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:41.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.429+0000 7f4137400640 1 -- 172.21.15.86:0/410399898 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41381594f0 con 0x7f41380cf9c0 2024-04-20T21:24:41.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.429+0000 7f412ea00640 1 --2- 172.21.15.86:0/410399898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41380d50c0 0x7f41380cbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:24:41.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.430+0000 7f4137400640 1 --2- 172.21.15.86:0/410399898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41380cf9c0 0x7f413814ca10 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f412800b3f0 tx=0x7f412800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:41.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.430+0000 7f412fe00640 1 -- 172.21.15.86:0/410399898 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4128015020 con 0x7f41380cf9c0 2024-04-20T21:24:41.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.430+0000 7f412fe00640 1 -- 172.21.15.86:0/410399898 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4128011830 con 0x7f41380cf9c0 2024-04-20T21:24:41.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.430+0000 7f412fe00640 1 -- 172.21.15.86:0/410399898 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4128011c20 con 0x7f41380cf9c0 2024-04-20T21:24:41.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.430+0000 7f413ce00640 1 -- 172.21.15.86:0/410399898 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4138159790 con 0x7f41380cf9c0 2024-04-20T21:24:41.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.430+0000 7f413ce00640 1 -- 172.21.15.86:0/410399898 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4138159d10 con 0x7f41380cf9c0 2024-04-20T21:24:41.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.431+0000 7f413ce00640 1 -- 172.21.15.86:0/410399898 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f413804fcf0 con 0x7f41380cf9c0 2024-04-20T21:24:41.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.432+0000 7f412fe00640 1 -- 172.21.15.86:0/410399898 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4128018020 con 0x7f41380cf9c0 2024-04-20T21:24:41.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.433+0000 7f412fe00640 1 --2- 172.21.15.86:0/410399898 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f411406c8f0 0x7f411406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:41.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.433+0000 7f412fe00640 1 -- 172.21.15.86:0/410399898 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5882+0+0 (secure 0 0 0) 0x7f412808b940 con 0x7f41380cf9c0 2024-04-20T21:24:41.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.434+0000 7f412ea00640 1 --2- 172.21.15.86:0/410399898 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f411406c8f0 0x7f411406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:41.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.434+0000 7f412ea00640 1 --2- 172.21.15.86:0/410399898 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f411406c8f0 0x7f411406edb0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f41380cb550 tx=0x7f4120008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:41.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.438+0000 7f412fe00640 1 -- 172.21.15.86:0/410399898 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f412801f030 con 0x7f41380cf9c0 2024-04-20T21:24:41.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.641+0000 7f413ce00640 1 -- 172.21.15.86:0/410399898 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f413808e320 con 0x7f41380cf9c0 2024-04-20T21:24:41.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.643+0000 7f412fe00640 1 -- 172.21.15.86:0/410399898 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+16941 (secure 0 0 0) 0x7f41280172c0 con 0x7f41380cf9c0 2024-04-20T21:24:41.643 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:24:41.643 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":74,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:23:35.220547+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:22:43.125767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-20T21:22:44.410549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.2d","mappings":[{"from":6,"to":0}]},{"pgid":"11.2e","mappings":[{"from":6,"to":0}]},{"pgid":"11.33","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6837/202143875":"2024-04-21T21:23:12.079572+0000","172.21.15.86:6835/202143875":"2024-04-21T21:23:12.079572+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:24:41.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.646+0000 7f413ce00640 1 -- 172.21.15.86:0/410399898 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f411406c8f0 msgr2=0x7f411406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:41.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.646+0000 7f413ce00640 1 --2- 172.21.15.86:0/410399898 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f411406c8f0 0x7f411406edb0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f41380cb550 tx=0x7f4120008040 comp rx=0 tx=0).stop 2024-04-20T21:24:41.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.646+0000 7f413ce00640 1 -- 172.21.15.86:0/410399898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41380cf9c0 msgr2=0x7f413814ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:41.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.646+0000 7f413ce00640 1 --2- 172.21.15.86:0/410399898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41380cf9c0 0x7f413814ca10 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f412800b3f0 tx=0x7f412800b8b0 comp rx=0 tx=0).stop 2024-04-20T21:24:41.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.646+0000 7f413ce00640 1 -- 172.21.15.86:0/410399898 shutdown_connections 2024-04-20T21:24:41.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.646+0000 7f413ce00640 1 --2- 172.21.15.86:0/410399898 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f411406c8f0 0x7f411406edb0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:41.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.646+0000 7f413ce00640 1 --2- 172.21.15.86:0/410399898 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f413815ece0 0x7f41380b9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:41.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.646+0000 7f413ce00640 1 --2- 172.21.15.86:0/410399898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41380d50c0 0x7f41380cbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:41.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.646+0000 7f413ce00640 1 --2- 172.21.15.86:0/410399898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41380cf9c0 0x7f413814ca10 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:41.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.646+0000 7f413ce00640 1 -- 172.21.15.86:0/410399898 >> 172.21.15.86:0/410399898 conn(0x7f413808bf70 msgr2=0x7f413815f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:41.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.647+0000 7f413ce00640 1 -- 172.21.15.86:0/410399898 shutdown_connections 2024-04-20T21:24:41.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.647+0000 7f413ce00640 1 -- 172.21.15.86:0/410399898 wait complete. 2024-04-20T21:24:41.664 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:22:43.125767+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:22:44.410549+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:24:41.664 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:24:41.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.869+0000 7f09fdc00640 1 Processor -- start 2024-04-20T21:24:41.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.870+0000 7f09fdc00640 1 -- start start 2024-04-20T21:24:41.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.871+0000 7f09fdc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09f8161ca0 0x7f09f80d4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:41.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.871+0000 7f09fdc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f09f80d2be0 0x7f09f80d0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:41.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.871+0000 7f09fdc00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f09f80d13b0 0x7f09f80cf240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:41.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.871+0000 7f09fdc00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f09f80d1820 con 0x7f09f80d2be0 2024-04-20T21:24:41.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.871+0000 7f09fdc00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f09f80d3730 con 0x7f09f80d13b0 2024-04-20T21:24:41.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.871+0000 7f09fdc00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f09f8162080 con 0x7f09f8161ca0 2024-04-20T21:24:41.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.871+0000 7f09ef400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f09f80d2be0 0x7f09f80d0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:41.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.871+0000 7f09ef400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f09f80d2be0 0x7f09f80d0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51582/0 (socket says 172.21.15.86:51582) 2024-04-20T21:24:41.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.871+0000 7f09ef400640 1 -- 172.21.15.86:0/53251053 learned_addr learned my addr 172.21.15.86:0/53251053 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:41.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.872+0000 7f09fd200640 1 --2- 172.21.15.86:0/53251053 >> v2:172.21.15.144:3300/0 conn(0x7f09f80d13b0 0x7f09f80cf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:41.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.872+0000 7f09ef400640 1 -- 172.21.15.86:0/53251053 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09f8161ca0 msgr2=0x7f09f80d4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:41.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.872+0000 7f09f7e00640 1 --2- 172.21.15.86:0/53251053 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09f8161ca0 0x7f09f80d4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:41.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.872+0000 7f09ef400640 1 --2- 172.21.15.86:0/53251053 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09f8161ca0 0x7f09f80d4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:41.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.872+0000 7f09ef400640 1 -- 172.21.15.86:0/53251053 >> v2:172.21.15.144:3300/0 conn(0x7f09f80d13b0 msgr2=0x7f09f80cf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:41.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.872+0000 7f09ef400640 1 --2- 172.21.15.86:0/53251053 >> v2:172.21.15.144:3300/0 conn(0x7f09f80d13b0 0x7f09f80cf240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:41.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.872+0000 7f09ef400640 1 -- 172.21.15.86:0/53251053 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09f80d5500 con 0x7f09f80d2be0 2024-04-20T21:24:41.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.872+0000 7f09fd200640 1 --2- 172.21.15.86:0/53251053 >> v2:172.21.15.144:3300/0 conn(0x7f09f80d13b0 0x7f09f80cf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:24:41.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.872+0000 7f09f7e00640 1 --2- 172.21.15.86:0/53251053 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09f8161ca0 0x7f09f80d4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:24:41.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.873+0000 7f09ef400640 1 --2- 172.21.15.86:0/53251053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09f80d2be0 0x7f09f80d0e70 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f09f808c680 tx=0x7f09e002b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48454066d3fa61db server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:41.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.873+0000 7f09f7400640 1 -- 172.21.15.86:0/53251053 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e0035020 con 0x7f09f80d2be0 2024-04-20T21:24:41.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.873+0000 7f09f7400640 1 -- 172.21.15.86:0/53251053 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09e0031880 con 0x7f09f80d2be0 2024-04-20T21:24:41.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.873+0000 7f09f7400640 1 -- 172.21.15.86:0/53251053 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e0031c50 con 0x7f09f80d2be0 2024-04-20T21:24:41.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.874+0000 7f09fdc00640 1 -- 172.21.15.86:0/53251053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09f80d2be0 msgr2=0x7f09f80d0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:41.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.874+0000 7f09fdc00640 1 --2- 172.21.15.86:0/53251053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09f80d2be0 0x7f09f80d0e70 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f09f808c680 tx=0x7f09e002b920 comp rx=0 tx=0).stop 2024-04-20T21:24:41.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.874+0000 7f09fdc00640 1 -- 172.21.15.86:0/53251053 shutdown_connections 2024-04-20T21:24:41.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.874+0000 7f09fdc00640 1 --2- 172.21.15.86:0/53251053 >> v2:172.21.15.144:3300/0 conn(0x7f09f80d13b0 0x7f09f80cf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:41.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.874+0000 7f09fdc00640 1 --2- 172.21.15.86:0/53251053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09f80d2be0 0x7f09f80d0e70 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:41.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.874+0000 7f09fdc00640 1 --2- 172.21.15.86:0/53251053 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09f8161ca0 0x7f09f80d4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:41.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.874+0000 7f09fdc00640 1 -- 172.21.15.86:0/53251053 >> 172.21.15.86:0/53251053 conn(0x7f09f808bf70 msgr2=0x7f09f80d4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:41.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.874+0000 7f09fdc00640 1 -- 172.21.15.86:0/53251053 shutdown_connections 2024-04-20T21:24:41.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.874+0000 7f09fdc00640 1 -- 172.21.15.86:0/53251053 wait complete. 2024-04-20T21:24:41.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.876+0000 7f09fdc00640 1 Processor -- start 2024-04-20T21:24:41.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.876+0000 7f09fdc00640 1 -- start start 2024-04-20T21:24:41.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.877+0000 7f09fdc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09f80d13b0 0x7f09f815ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:41.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.877+0000 7f09fdc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09f80d2be0 0x7f09f80a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:41.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.877+0000 7f09fdc00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09f8161ca0 0x7f09f80a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:41.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.877+0000 7f09fdc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f09f80b7620 con 0x7f09f80d2be0 2024-04-20T21:24:41.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.877+0000 7f09fdc00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f09f80d3450 con 0x7f09f8161ca0 2024-04-20T21:24:41.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.877+0000 7f09fdc00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f09f80b7900 con 0x7f09f80d13b0 2024-04-20T21:24:41.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.877+0000 7f09ef400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09f80d2be0 0x7f09f80a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:41.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.877+0000 7f09ef400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09f80d2be0 0x7f09f80a1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51596/0 (socket says 172.21.15.86:51596) 2024-04-20T21:24:41.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.877+0000 7f09ef400640 1 -- 172.21.15.86:0/2799641080 learned_addr learned my addr 172.21.15.86:0/2799641080 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:41.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.878+0000 7f09f7e00640 1 --2- 172.21.15.86:0/2799641080 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09f80d13b0 0x7f09f815ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:41.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.878+0000 7f09ef400640 1 -- 172.21.15.86:0/2799641080 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09f80d13b0 msgr2=0x7f09f815ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:41.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.878+0000 7f09ef400640 1 --2- 172.21.15.86:0/2799641080 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09f80d13b0 0x7f09f815ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:41.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.878+0000 7f09ef400640 1 -- 172.21.15.86:0/2799641080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09f8161ca0 msgr2=0x7f09f80a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:41.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.878+0000 7f09ef400640 1 --2- 172.21.15.86:0/2799641080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09f8161ca0 0x7f09f80a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:41.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.878+0000 7f09ef400640 1 -- 172.21.15.86:0/2799641080 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09f80a2030 con 0x7f09f80d2be0 2024-04-20T21:24:41.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.878+0000 7f09f7e00640 1 --2- 172.21.15.86:0/2799641080 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09f80d13b0 0x7f09f815ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:24:41.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.878+0000 7f09ef400640 1 --2- 172.21.15.86:0/2799641080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09f80d2be0 0x7f09f80a1500 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f09e0000c00 tx=0x7f09e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:41.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.879+0000 7f09f4c00640 1 -- 172.21.15.86:0/2799641080 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e00443f0 con 0x7f09f80d2be0 2024-04-20T21:24:41.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.879+0000 7f09f4c00640 1 -- 172.21.15.86:0/2799641080 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09e0044590 con 0x7f09f80d2be0 2024-04-20T21:24:41.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.879+0000 7f09f4c00640 1 -- 172.21.15.86:0/2799641080 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e0044980 con 0x7f09f80d2be0 2024-04-20T21:24:41.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.879+0000 7f09fdc00640 1 -- 172.21.15.86:0/2799641080 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09f80c8370 con 0x7f09f80d2be0 2024-04-20T21:24:41.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.879+0000 7f09fdc00640 1 -- 172.21.15.86:0/2799641080 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09f809f180 con 0x7f09f80d2be0 2024-04-20T21:24:41.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.880+0000 7f09fdc00640 1 -- 172.21.15.86:0/2799641080 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09f80c8160 con 0x7f09f80d2be0 2024-04-20T21:24:41.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.881+0000 7f09f4c00640 1 -- 172.21.15.86:0/2799641080 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f09e004b060 con 0x7f09f80d2be0 2024-04-20T21:24:41.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.882+0000 7f09f4c00640 1 --2- 172.21.15.86:0/2799641080 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f09d406c8f0 0x7f09d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:41.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.882+0000 7f09f4c00640 1 -- 172.21.15.86:0/2799641080 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5882+0+0 (secure 0 0 0) 0x7f09e00adf80 con 0x7f09f80d2be0 2024-04-20T21:24:41.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.883+0000 7f09f7e00640 1 --2- 172.21.15.86:0/2799641080 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f09d406c8f0 0x7f09d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:41.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.883+0000 7f09f7e00640 1 --2- 172.21.15.86:0/2799641080 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f09d406c8f0 0x7f09d406edb0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f09f80b7f40 tx=0x7f09e8007430 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:41.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:41.887+0000 7f09f4c00640 1 -- 172.21.15.86:0/2799641080 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f09e003f070 con 0x7f09f80d2be0 2024-04-20T21:24:42.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.069+0000 7f09fdc00640 1 -- 172.21.15.86:0/2799641080 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f09f804fd60 con 0x7f09f80d2be0 2024-04-20T21:24:42.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.070+0000 7f09f4c00640 1 -- 172.21.15.86:0/2799641080 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) ==== 93+0+10 (secure 0 0 0) 0x7f09e007c2f0 con 0x7f09f80d2be0 2024-04-20T21:24:42.069 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:24:42.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.072+0000 7f09fdc00640 1 -- 172.21.15.86:0/2799641080 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f09d406c8f0 msgr2=0x7f09d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:42.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.072+0000 7f09fdc00640 1 --2- 172.21.15.86:0/2799641080 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f09d406c8f0 0x7f09d406edb0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f09f80b7f40 tx=0x7f09e8007430 comp rx=0 tx=0).stop 2024-04-20T21:24:42.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.072+0000 7f09fdc00640 1 -- 172.21.15.86:0/2799641080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09f80d2be0 msgr2=0x7f09f80a1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:42.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.072+0000 7f09fdc00640 1 --2- 172.21.15.86:0/2799641080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09f80d2be0 0x7f09f80a1500 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f09e0000c00 tx=0x7f09e00046c0 comp rx=0 tx=0).stop 2024-04-20T21:24:42.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.072+0000 7f09fdc00640 1 -- 172.21.15.86:0/2799641080 shutdown_connections 2024-04-20T21:24:42.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.072+0000 7f09fdc00640 1 --2- 172.21.15.86:0/2799641080 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f09d406c8f0 0x7f09d406edb0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:42.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.072+0000 7f09fdc00640 1 --2- 172.21.15.86:0/2799641080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09f8161ca0 0x7f09f80a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:42.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.072+0000 7f09fdc00640 1 --2- 172.21.15.86:0/2799641080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09f80d2be0 0x7f09f80a1500 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:42.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.072+0000 7f09fdc00640 1 --2- 172.21.15.86:0/2799641080 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09f80d13b0 0x7f09f815ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:42.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.072+0000 7f09fdc00640 1 -- 172.21.15.86:0/2799641080 >> 172.21.15.86:0/2799641080 conn(0x7f09f808bf70 msgr2=0x7f09f80b8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:42.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.073+0000 7f09fdc00640 1 -- 172.21.15.86:0/2799641080 shutdown_connections 2024-04-20T21:24:42.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.073+0000 7f09fdc00640 1 -- 172.21.15.86:0/2799641080 wait complete. 2024-04-20T21:24:42.089 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:24:42.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.292+0000 7f5937200640 1 Processor -- start 2024-04-20T21:24:42.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.292+0000 7f5937200640 1 -- start start 2024-04-20T21:24:42.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.294+0000 7f5937200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f593006c090 0x7f59300cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:42.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.294+0000 7f5937200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59300b4ba0 0x7f59300b5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:42.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.294+0000 7f5937200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5930156110 con 0x7f5930155d30 2024-04-20T21:24:42.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.294+0000 7f5937200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f59300ce460 con 0x7f593006c090 2024-04-20T21:24:42.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.294+0000 7f5937200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f59300ce740 con 0x7f59300b4ba0 2024-04-20T21:24:42.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.294+0000 7f592f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f593006c090 0x7f59300cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:42.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.294+0000 7f592fe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f5930155d30 0x7f593006b980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:58004/0 (socket says 172.21.15.86:58004) 2024-04-20T21:24:42.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.294+0000 7f592fe00640 1 -- 172.21.15.86:0/1189610060 learned_addr learned my addr 172.21.15.86:0/1189610060 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:24:42.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.294+0000 7f5935200640 1 --2- 172.21.15.86:0/1189610060 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59300b4ba0 0x7f59300b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:42.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.294+0000 7f592f400640 1 -- 172.21.15.86:0/1189610060 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59300b4ba0 msgr2=0x7f59300b5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:42.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.294+0000 7f592f400640 1 --2- 172.21.15.86:0/1189610060 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59300b4ba0 0x7f59300b5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:42.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.294+0000 7f592f400640 1 -- 172.21.15.86:0/1189610060 >> v1:172.21.15.86:6789/0 conn(0x7f5930155d30 legacy=0x7f593006b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:42.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.294+0000 7f592f400640 1 -- 172.21.15.86:0/1189610060 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59300cea20 con 0x7f593006c090 2024-04-20T21:24:42.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.294+0000 7f592ea00640 1 -- 172.21.15.86:0/1189610060 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 346993099 0 0) 0x7f5930156110 con 0x7f5930155d30 2024-04-20T21:24:42.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.294+0000 7f5935200640 1 --2- 172.21.15.86:0/1189610060 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59300b4ba0 0x7f59300b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:24:42.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.295+0000 7f592f400640 1 --2- 172.21.15.86:0/1189610060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f593006c090 0x7f59300cf500 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f59200097a0 tx=0x7f592002b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d1d4f3aca052c1b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:42.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.296+0000 7f592ea00640 1 -- 172.21.15.86:0/1189610060 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5920038070 con 0x7f593006c090 2024-04-20T21:24:42.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.296+0000 7f592ea00640 1 -- 172.21.15.86:0/1189610060 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5920031700 con 0x7f593006c090 2024-04-20T21:24:42.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.296+0000 7f592ea00640 1 -- 172.21.15.86:0/1189610060 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5920031ad0 con 0x7f593006c090 2024-04-20T21:24:42.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.296+0000 7f5937200640 1 -- 172.21.15.86:0/1189610060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f593006c090 msgr2=0x7f59300cf500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:42.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.296+0000 7f5937200640 1 --2- 172.21.15.86:0/1189610060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f593006c090 0x7f59300cf500 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f59200097a0 tx=0x7f592002b7a0 comp rx=0 tx=0).stop 2024-04-20T21:24:42.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.297+0000 7f5937200640 1 -- 172.21.15.86:0/1189610060 shutdown_connections 2024-04-20T21:24:42.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.297+0000 7f5937200640 1 --2- 172.21.15.86:0/1189610060 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59300b4ba0 0x7f59300b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:42.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.297+0000 7f5937200640 1 --2- 172.21.15.86:0/1189610060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f593006c090 0x7f59300cf500 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:42.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.297+0000 7f5937200640 1 -- 172.21.15.86:0/1189610060 >> 172.21.15.86:0/1189610060 conn(0x7f59300b2110 msgr2=0x7f59300b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:42.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.297+0000 7f5937200640 1 -- 172.21.15.86:0/1189610060 shutdown_connections 2024-04-20T21:24:42.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.297+0000 7f5937200640 1 -- 172.21.15.86:0/1189610060 wait complete. 2024-04-20T21:24:42.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.298+0000 7f5937200640 1 Processor -- start 2024-04-20T21:24:42.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.298+0000 7f5937200640 1 -- start start 2024-04-20T21:24:42.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.299+0000 7f5937200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f593006c090 0x7f5930150770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:42.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.299+0000 7f5937200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59300b4ba0 0x7f5930167c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:42.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.299+0000 7f5937200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5930155d30 0x7f5930142b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:42.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.299+0000 7f5937200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f59300b5570 con 0x7f59300b4ba0 2024-04-20T21:24:42.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.299+0000 7f5937200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f59300cefe0 con 0x7f593006c090 2024-04-20T21:24:42.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.299+0000 7f5937200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f59300b56f0 con 0x7f5930155d30 2024-04-20T21:24:42.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.300+0000 7f592f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59300b4ba0 0x7f5930167c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:42.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.300+0000 7f592f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59300b4ba0 0x7f5930167c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51616/0 (socket says 172.21.15.86:51616) 2024-04-20T21:24:42.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.300+0000 7f592f400640 1 -- 172.21.15.86:0/2398094106 learned_addr learned my addr 172.21.15.86:0/2398094106 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:42.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.300+0000 7f5935200640 1 --2- 172.21.15.86:0/2398094106 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5930155d30 0x7f5930142b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:42.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.300+0000 7f592f400640 1 -- 172.21.15.86:0/2398094106 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5930155d30 msgr2=0x7f5930142b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:42.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.300+0000 7f592f400640 1 --2- 172.21.15.86:0/2398094106 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5930155d30 0x7f5930142b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:42.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.300+0000 7f592f400640 1 -- 172.21.15.86:0/2398094106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f593006c090 msgr2=0x7f5930150770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:42.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.300+0000 7f592f400640 1 --2- 172.21.15.86:0/2398094106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f593006c090 0x7f5930150770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:42.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.300+0000 7f592f400640 1 -- 172.21.15.86:0/2398094106 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f593014eb50 con 0x7f59300b4ba0 2024-04-20T21:24:42.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.300+0000 7f592f400640 1 --2- 172.21.15.86:0/2398094106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59300b4ba0 0x7f5930167c70 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f5920036040 tx=0x7f59200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:42.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.301+0000 7f5927e00640 1 -- 172.21.15.86:0/2398094106 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5920042070 con 0x7f59300b4ba0 2024-04-20T21:24:42.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.301+0000 7f5927e00640 1 -- 172.21.15.86:0/2398094106 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5920033a30 con 0x7f59300b4ba0 2024-04-20T21:24:42.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.301+0000 7f5927e00640 1 -- 172.21.15.86:0/2398094106 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5920038040 con 0x7f59300b4ba0 2024-04-20T21:24:42.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.301+0000 7f5937200640 1 -- 172.21.15.86:0/2398094106 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f593014ee30 con 0x7f59300b4ba0 2024-04-20T21:24:42.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.301+0000 7f5937200640 1 -- 172.21.15.86:0/2398094106 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f593014f6d0 con 0x7f59300b4ba0 2024-04-20T21:24:42.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.302+0000 7f5937200640 1 -- 172.21.15.86:0/2398094106 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f593014f000 con 0x7f59300b4ba0 2024-04-20T21:24:42.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.303+0000 7f5927e00640 1 -- 172.21.15.86:0/2398094106 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f592003b070 con 0x7f59300b4ba0 2024-04-20T21:24:42.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.304+0000 7f5927e00640 1 --2- 172.21.15.86:0/2398094106 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f590c06c8f0 0x7f590c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:42.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.304+0000 7f5927e00640 1 -- 172.21.15.86:0/2398094106 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5882+0+0 (secure 0 0 0) 0x7f59200ad660 con 0x7f59300b4ba0 2024-04-20T21:24:42.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.304+0000 7f592fe00640 1 --2- 172.21.15.86:0/2398094106 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f590c06c8f0 0x7f590c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:42.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.305+0000 7f592fe00640 1 --2- 172.21.15.86:0/2398094106 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f590c06c8f0 0x7f590c06edb0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5918007fd0 tx=0x7f5918007920 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:42.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.308+0000 7f5927e00640 1 -- 172.21.15.86:0/2398094106 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5920030070 con 0x7f59300b4ba0 2024-04-20T21:24:42.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.509+0000 7f5937200640 1 -- 172.21.15.86:0/2398094106 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f593014f8d0 con 0x7f59300b4ba0 2024-04-20T21:24:42.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.510+0000 7f5927e00640 1 -- 172.21.15.86:0/2398094106 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v74) ==== 104+0+11 (secure 0 0 0) 0x7f593014f8d0 con 0x7f59300b4ba0 2024-04-20T21:24:42.509 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-20T21:24:42.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.513+0000 7f5937200640 1 -- 172.21.15.86:0/2398094106 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f590c06c8f0 msgr2=0x7f590c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:42.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.513+0000 7f5937200640 1 --2- 172.21.15.86:0/2398094106 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f590c06c8f0 0x7f590c06edb0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5918007fd0 tx=0x7f5918007920 comp rx=0 tx=0).stop 2024-04-20T21:24:42.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.513+0000 7f5937200640 1 -- 172.21.15.86:0/2398094106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59300b4ba0 msgr2=0x7f5930167c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:42.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.513+0000 7f5937200640 1 --2- 172.21.15.86:0/2398094106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59300b4ba0 0x7f5930167c70 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f5920036040 tx=0x7f59200046c0 comp rx=0 tx=0).stop 2024-04-20T21:24:42.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.513+0000 7f5937200640 1 -- 172.21.15.86:0/2398094106 shutdown_connections 2024-04-20T21:24:42.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.513+0000 7f5937200640 1 --2- 172.21.15.86:0/2398094106 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f590c06c8f0 0x7f590c06edb0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:42.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.513+0000 7f5937200640 1 --2- 172.21.15.86:0/2398094106 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5930155d30 0x7f5930142b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:42.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.513+0000 7f5937200640 1 --2- 172.21.15.86:0/2398094106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59300b4ba0 0x7f5930167c70 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:42.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.513+0000 7f5937200640 1 --2- 172.21.15.86:0/2398094106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f593006c090 0x7f5930150770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:42.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.513+0000 7f5937200640 1 -- 172.21.15.86:0/2398094106 >> 172.21.15.86:0/2398094106 conn(0x7f59300b2110 msgr2=0x7f59300b1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:42.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.513+0000 7f5937200640 1 -- 172.21.15.86:0/2398094106 shutdown_connections 2024-04-20T21:24:42.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.513+0000 7f5937200640 1 -- 172.21.15.86:0/2398094106 wait complete. 2024-04-20T21:24:42.531 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:24:42.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.733+0000 7f943a000640 1 Processor -- start 2024-04-20T21:24:42.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.733+0000 7f943a000640 1 -- start start 2024-04-20T21:24:42.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.734+0000 7f943a000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9434162180 0x7f94340b34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:42.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.734+0000 7f943a000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94340d2b90 con 0x7f94340d02c0 2024-04-20T21:24:42.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.734+0000 7f943a000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94340d2e70 con 0x7f94340d1070 2024-04-20T21:24:42.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.734+0000 7f943a000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f94340d0750 con 0x7f9434162180 2024-04-20T21:24:42.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.735+0000 7f9439600640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f94340d02c0 0x7f94340b47c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:58020/0 (socket says 172.21.15.86:58020) 2024-04-20T21:24:42.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.735+0000 7f9439600640 1 -- 172.21.15.86:0/2836261466 learned_addr learned my addr 172.21.15.86:0/2836261466 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:24:42.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.735+0000 7f9438c00640 1 --2- 172.21.15.86:0/2836261466 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9434162180 0x7f94340b34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:42.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.735+0000 7f9433400640 1 -- 172.21.15.86:0/2836261466 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2134952416 0 0) 0x7f94340d2b90 con 0x7f94340d02c0 2024-04-20T21:24:42.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.736+0000 7f9433400640 1 -- 172.21.15.86:0/2836261466 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9418003520 con 0x7f94340d02c0 2024-04-20T21:24:42.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.736+0000 7f9433400640 1 -- 172.21.15.86:0/2836261466 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3374889269 0 0) 0x7f9418003520 con 0x7f94340d02c0 2024-04-20T21:24:42.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.736+0000 7f9433400640 1 -- 172.21.15.86:0/2836261466 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9434162180 msgr2=0x7f94340b34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:42.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.736+0000 7f9433400640 1 --2- 172.21.15.86:0/2836261466 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9434162180 0x7f94340b34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:42.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.736+0000 7f9433400640 1 -- 172.21.15.86:0/2836261466 >> v1:172.21.15.144:6789/0 conn(0x7f94340d1070 legacy=0x7f94340b40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:42.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.736+0000 7f9433400640 1 -- 172.21.15.86:0/2836261466 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94340d3150 con 0x7f94340d02c0 2024-04-20T21:24:42.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.736+0000 7f9433400640 1 -- 172.21.15.86:0/2836261466 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f942c0019f0 con 0x7f94340d02c0 2024-04-20T21:24:42.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.736+0000 7f9433400640 1 -- 172.21.15.86:0/2836261466 >> v1:172.21.15.86:6789/0 conn(0x7f94340d02c0 legacy=0x7f94340b47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:42.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.736+0000 7f9433400640 1 --2- 172.21.15.86:0/2836261466 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9434162180 0x7f94180039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:42.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.736+0000 7f9433400640 1 --2- 172.21.15.86:0/2836261466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9418003f20 0x7f9418004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:42.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.736+0000 7f9433400640 1 --2- 172.21.15.86:0/2836261466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9418004d40 0x7f94180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:42.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.736+0000 7f9433400640 1 -- 172.21.15.86:0/2836261466 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f94340d0750 con 0x7f9418004d40 2024-04-20T21:24:42.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.736+0000 7f9433400640 1 -- 172.21.15.86:0/2836261466 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f941800ba20 con 0x7f9418003f20 2024-04-20T21:24:42.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.736+0000 7f9433400640 1 -- 172.21.15.86:0/2836261466 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f941800bc20 con 0x7f9434162180 2024-04-20T21:24:42.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.736+0000 7f9439600640 1 --2- 172.21.15.86:0/2836261466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9418004d40 0x7f94180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:42.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.737+0000 7f9438c00640 1 --2- 172.21.15.86:0/2836261466 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9434162180 0x7f94180039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:42.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.737+0000 7f9433e00640 1 --2- 172.21.15.86:0/2836261466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9418003f20 0x7f9418004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:42.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.737+0000 7f9439600640 1 -- 172.21.15.86:0/2836261466 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9434162180 msgr2=0x7f94180039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:42.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.737+0000 7f9439600640 1 --2- 172.21.15.86:0/2836261466 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9434162180 0x7f94180039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:42.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.737+0000 7f9439600640 1 -- 172.21.15.86:0/2836261466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9418003f20 msgr2=0x7f9418004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:42.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.737+0000 7f9439600640 1 --2- 172.21.15.86:0/2836261466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9418003f20 0x7f9418004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:42.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.737+0000 7f9439600640 1 -- 172.21.15.86:0/2836261466 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9418003520 con 0x7f9418004d40 2024-04-20T21:24:42.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.738+0000 7f9439600640 1 --2- 172.21.15.86:0/2836261466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9418004d40 0x7f94180051c0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f942c00a810 tx=0x7f9418005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b5abebfb80d496 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:42.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.738+0000 7f9433400640 1 -- 172.21.15.86:0/2836261466 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f942c00a430 con 0x7f9418004d40 2024-04-20T21:24:42.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.738+0000 7f9433400640 1 -- 172.21.15.86:0/2836261466 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f942c034630 con 0x7f9418004d40 2024-04-20T21:24:42.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.738+0000 7f9433400640 1 -- 172.21.15.86:0/2836261466 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f942c034a60 con 0x7f9418004d40 2024-04-20T21:24:42.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.739+0000 7f943a000640 1 -- 172.21.15.86:0/2836261466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9418004d40 msgr2=0x7f94180051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:42.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.739+0000 7f943a000640 1 --2- 172.21.15.86:0/2836261466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9418004d40 0x7f94180051c0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f942c00a810 tx=0x7f9418005830 comp rx=0 tx=0).stop 2024-04-20T21:24:42.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.739+0000 7f9439600640 1 -- 172.21.15.86:0/2836261466 reap_dead start 2024-04-20T21:24:42.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.739+0000 7f943a000640 1 -- 172.21.15.86:0/2836261466 shutdown_connections 2024-04-20T21:24:42.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.739+0000 7f943a000640 1 -- 172.21.15.86:0/2836261466 >> 172.21.15.86:0/2836261466 conn(0x7f943408bf70 msgr2=0x7f94340d46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:42.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.739+0000 7f943a000640 1 -- 172.21.15.86:0/2836261466 shutdown_connections 2024-04-20T21:24:42.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.739+0000 7f943a000640 1 -- 172.21.15.86:0/2836261466 wait complete. 2024-04-20T21:24:42.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.741+0000 7f943a000640 1 Processor -- start 2024-04-20T21:24:42.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.741+0000 7f943a000640 1 -- start start 2024-04-20T21:24:42.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.742+0000 7f943a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9418004d40 0x7f94340cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:42.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.742+0000 7f943a000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9434158be0 0x7f9434158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:42.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.742+0000 7f943a000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94340c8a20 0x7f94340b8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:42.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.742+0000 7f943a000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f94340b34a0 con 0x7f9418004d40 2024-04-20T21:24:42.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.742+0000 7f943a000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f94340b5e50 con 0x7f94340c8a20 2024-04-20T21:24:42.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.742+0000 7f943a000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f94340b3620 con 0x7f9434158be0 2024-04-20T21:24:42.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.742+0000 7f9438c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9418004d40 0x7f94340cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:42.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.742+0000 7f9438c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9418004d40 0x7f94340cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51636/0 (socket says 172.21.15.86:51636) 2024-04-20T21:24:42.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.742+0000 7f9438c00640 1 -- 172.21.15.86:0/3295590115 learned_addr learned my addr 172.21.15.86:0/3295590115 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:42.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.743+0000 7f9433e00640 1 --2- 172.21.15.86:0/3295590115 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9434158be0 0x7f9434158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:42.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.743+0000 7f9439600640 1 --2- 172.21.15.86:0/3295590115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94340c8a20 0x7f94340b8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:42.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.743+0000 7f9438c00640 1 -- 172.21.15.86:0/3295590115 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9434158be0 msgr2=0x7f9434158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:42.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.743+0000 7f9438c00640 1 --2- 172.21.15.86:0/3295590115 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9434158be0 0x7f9434158270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:42.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.743+0000 7f9438c00640 1 -- 172.21.15.86:0/3295590115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94340c8a20 msgr2=0x7f94340b8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:42.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.743+0000 7f9438c00640 1 --2- 172.21.15.86:0/3295590115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94340c8a20 0x7f94340b8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:42.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.743+0000 7f9438c00640 1 -- 172.21.15.86:0/3295590115 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94340b7f40 con 0x7f9418004d40 2024-04-20T21:24:42.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.743+0000 7f9433e00640 1 --2- 172.21.15.86:0/3295590115 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9434158be0 0x7f9434158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:24:42.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.743+0000 7f9438c00640 1 --2- 172.21.15.86:0/3295590115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9418004d40 0x7f94340cd7c0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f9424011630 tx=0x7f9424011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:42.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.743+0000 7f9430c00640 1 -- 172.21.15.86:0/3295590115 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f942400f920 con 0x7f9418004d40 2024-04-20T21:24:42.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.743+0000 7f9430c00640 1 -- 172.21.15.86:0/3295590115 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f942400fac0 con 0x7f9418004d40 2024-04-20T21:24:42.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.743+0000 7f9430c00640 1 -- 172.21.15.86:0/3295590115 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f942400fe70 con 0x7f9418004d40 2024-04-20T21:24:42.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.743+0000 7f943a000640 1 -- 172.21.15.86:0/3295590115 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94340b8220 con 0x7f9418004d40 2024-04-20T21:24:42.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.744+0000 7f943a000640 1 -- 172.21.15.86:0/3295590115 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94340cdd00 con 0x7f9418004d40 2024-04-20T21:24:42.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.745+0000 7f943a000640 1 -- 172.21.15.86:0/3295590115 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f943404fd90 con 0x7f9418004d40 2024-04-20T21:24:42.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.746+0000 7f9430c00640 1 -- 172.21.15.86:0/3295590115 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9424015460 con 0x7f9418004d40 2024-04-20T21:24:42.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.747+0000 7f9430c00640 1 --2- 172.21.15.86:0/3295590115 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f941006c8f0 0x7f941006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:42.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.747+0000 7f9430c00640 1 -- 172.21.15.86:0/3295590115 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5882+0+0 (secure 0 0 0) 0x7f942408c6a0 con 0x7f9418004d40 2024-04-20T21:24:42.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.747+0000 7f9433e00640 1 --2- 172.21.15.86:0/3295590115 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f941006c8f0 0x7f941006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:42.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.748+0000 7f9433e00640 1 --2- 172.21.15.86:0/3295590115 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f941006c8f0 0x7f941006edb0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f94340ce3b0 tx=0x7f941c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:42.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.752+0000 7f9430c00640 1 -- 172.21.15.86:0/3295590115 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9424019030 con 0x7f9418004d40 2024-04-20T21:24:42.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.950+0000 7f943a000640 1 -- 172.21.15.86:0/3295590115 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f94340b5110 con 0x7f9418004d40 2024-04-20T21:24:42.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.951+0000 7f9430c00640 1 -- 172.21.15.86:0/3295590115 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v74) ==== 100+0+11 (secure 0 0 0) 0x7f942405a990 con 0x7f9418004d40 2024-04-20T21:24:42.950 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-20T21:24:42.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.953+0000 7f943a000640 1 -- 172.21.15.86:0/3295590115 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f941006c8f0 msgr2=0x7f941006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:42.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.953+0000 7f943a000640 1 --2- 172.21.15.86:0/3295590115 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f941006c8f0 0x7f941006edb0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f94340ce3b0 tx=0x7f941c009040 comp rx=0 tx=0).stop 2024-04-20T21:24:42.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.953+0000 7f943a000640 1 -- 172.21.15.86:0/3295590115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9418004d40 msgr2=0x7f94340cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:42.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.953+0000 7f943a000640 1 --2- 172.21.15.86:0/3295590115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9418004d40 0x7f94340cd7c0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f9424011630 tx=0x7f9424011af0 comp rx=0 tx=0).stop 2024-04-20T21:24:42.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.953+0000 7f943a000640 1 -- 172.21.15.86:0/3295590115 shutdown_connections 2024-04-20T21:24:42.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.953+0000 7f943a000640 1 --2- 172.21.15.86:0/3295590115 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f941006c8f0 0x7f941006edb0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:42.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.953+0000 7f943a000640 1 --2- 172.21.15.86:0/3295590115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94340c8a20 0x7f94340b8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:42.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.953+0000 7f943a000640 1 --2- 172.21.15.86:0/3295590115 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9434158be0 0x7f9434158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:42.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.953+0000 7f943a000640 1 --2- 172.21.15.86:0/3295590115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9418004d40 0x7f94340cd7c0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:42.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.953+0000 7f943a000640 1 -- 172.21.15.86:0/3295590115 >> 172.21.15.86:0/3295590115 conn(0x7f943408bf70 msgr2=0x7f943406da90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:42.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.953+0000 7f943a000640 1 -- 172.21.15.86:0/3295590115 shutdown_connections 2024-04-20T21:24:42.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:42.953+0000 7f943a000640 1 -- 172.21.15.86:0/3295590115 wait complete. 2024-04-20T21:24:42.970 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:24:43.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.163+0000 7fc6b4400640 1 Processor -- start 2024-04-20T21:24:43.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.163+0000 7fc6b4400640 1 -- start start 2024-04-20T21:24:43.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.165+0000 7fc6b4400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc6ac146130 0x7fc6ac06e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:43.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.165+0000 7fc6b4400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6ac06dc30 con 0x7fc6ac0ccdd0 2024-04-20T21:24:43.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.165+0000 7fc6b4400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6ac0cd8e0 con 0x7fc6ac072410 2024-04-20T21:24:43.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.165+0000 7fc6b4400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc6ac0cdbc0 con 0x7fc6ac146130 2024-04-20T21:24:43.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.165+0000 7fc6b1a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc6ac146130 0x7fc6ac06e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:43.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.165+0000 7fc6b1a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc6ac146130 0x7fc6ac06e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51640/0 (socket says 172.21.15.86:51640) 2024-04-20T21:24:43.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.165+0000 7fc6b1a00640 1 -- 172.21.15.86:0/856871908 learned_addr learned my addr 172.21.15.86:0/856871908 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:43.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.165+0000 7fc6b1000640 1 --1- 172.21.15.86:0/856871908 >> v1:172.21.15.86:6789/0 conn(0x7fc6ac0ccdd0 0x7fc6ac071d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:58036/0 (socket says 172.21.15.86:58036) 2024-04-20T21:24:43.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.166+0000 7fc6b1a00640 1 -- 172.21.15.86:0/856871908 >> v1:172.21.15.144:6789/0 conn(0x7fc6ac072410 legacy=0x7fc6ac15cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:43.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.166+0000 7fc6b1a00640 1 -- 172.21.15.86:0/856871908 >> v1:172.21.15.86:6789/0 conn(0x7fc6ac0ccdd0 legacy=0x7fc6ac071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:43.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.166+0000 7fc6b1a00640 1 -- 172.21.15.86:0/856871908 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6ac06ded0 con 0x7fc6ac146130 2024-04-20T21:24:43.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.166+0000 7fc6abe00640 1 -- 172.21.15.86:0/856871908 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3771266217 0 0) 0x7fc6ac06dc30 con 0x7fc6ac0ccdd0 2024-04-20T21:24:43.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.167+0000 7fc6b1a00640 1 --2- 172.21.15.86:0/856871908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6ac146130 0x7fc6ac06e6d0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0009870 tx=0x7fc6a002ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ead3fe445c72924 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:43.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.168+0000 7fc6abe00640 1 -- 172.21.15.86:0/856871908 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a002f840 con 0x7fc6ac146130 2024-04-20T21:24:43.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.168+0000 7fc6abe00640 1 -- 172.21.15.86:0/856871908 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6a002f9e0 con 0x7fc6ac146130 2024-04-20T21:24:43.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.168+0000 7fc6abe00640 1 -- 172.21.15.86:0/856871908 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a002fdb0 con 0x7fc6ac146130 2024-04-20T21:24:43.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.168+0000 7fc6b4400640 1 -- 172.21.15.86:0/856871908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6ac146130 msgr2=0x7fc6ac06e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:43.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.168+0000 7fc6b4400640 1 --2- 172.21.15.86:0/856871908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6ac146130 0x7fc6ac06e6d0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0009870 tx=0x7fc6a002ba20 comp rx=0 tx=0).stop 2024-04-20T21:24:43.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.168+0000 7fc6b4400640 1 -- 172.21.15.86:0/856871908 shutdown_connections 2024-04-20T21:24:43.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.168+0000 7fc6b4400640 1 --2- 172.21.15.86:0/856871908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6ac146130 0x7fc6ac06e6d0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:43.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.168+0000 7fc6b4400640 1 -- 172.21.15.86:0/856871908 >> 172.21.15.86:0/856871908 conn(0x7fc6ac0b2360 msgr2=0x7fc6ac0b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:43.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.168+0000 7fc6b4400640 1 -- 172.21.15.86:0/856871908 shutdown_connections 2024-04-20T21:24:43.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.168+0000 7fc6b4400640 1 -- 172.21.15.86:0/856871908 wait complete. 2024-04-20T21:24:43.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.170+0000 7fc6b4400640 1 Processor -- start 2024-04-20T21:24:43.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.170+0000 7fc6b4400640 1 -- start start 2024-04-20T21:24:43.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.171+0000 7fc6b4400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6ac072410 0x7fc6ac15f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:43.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.171+0000 7fc6b4400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6ac0ccdd0 0x7fc6ac15bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:43.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.171+0000 7fc6b4400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc6ac146130 0x7fc6ac15c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:43.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.171+0000 7fc6b4400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ac0cd230 con 0x7fc6ac0ccdd0 2024-04-20T21:24:43.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.171+0000 7fc6b4400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ac06f150 con 0x7fc6ac072410 2024-04-20T21:24:43.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.171+0000 7fc6b4400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc6ac0cd3b0 con 0x7fc6ac146130 2024-04-20T21:24:43.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.171+0000 7fc6b1000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6ac0ccdd0 0x7fc6ac15bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:43.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.171+0000 7fc6b1000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6ac0ccdd0 0x7fc6ac15bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51642/0 (socket says 172.21.15.86:51642) 2024-04-20T21:24:43.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.171+0000 7fc6b1000640 1 -- 172.21.15.86:0/743429342 learned_addr learned my addr 172.21.15.86:0/743429342 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:43.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.172+0000 7fc6b1a00640 1 --2- 172.21.15.86:0/743429342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6ac072410 0x7fc6ac15f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:43.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.172+0000 7fc6b1000640 1 -- 172.21.15.86:0/743429342 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc6ac146130 msgr2=0x7fc6ac15c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:43.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.172+0000 7fc6b1000640 1 --2- 172.21.15.86:0/743429342 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc6ac146130 0x7fc6ac15c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:43.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.172+0000 7fc6b1000640 1 -- 172.21.15.86:0/743429342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6ac072410 msgr2=0x7fc6ac15f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:43.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.172+0000 7fc6b1000640 1 --2- 172.21.15.86:0/743429342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6ac072410 0x7fc6ac15f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:43.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.172+0000 7fc6b1000640 1 -- 172.21.15.86:0/743429342 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6ac0c6270 con 0x7fc6ac0ccdd0 2024-04-20T21:24:43.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.172+0000 7fc6b1000640 1 --2- 172.21.15.86:0/743429342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6ac0ccdd0 0x7fc6ac15bcc0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fc69c00c4b0 tx=0x7fc69c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:43.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.172+0000 7fc6a9600640 1 -- 172.21.15.86:0/743429342 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69c018070 con 0x7fc6ac0ccdd0 2024-04-20T21:24:43.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.172+0000 7fc6a9600640 1 -- 172.21.15.86:0/743429342 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc69c007480 con 0x7fc6ac0ccdd0 2024-04-20T21:24:43.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.172+0000 7fc6a9600640 1 -- 172.21.15.86:0/743429342 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69c004820 con 0x7fc6ac0ccdd0 2024-04-20T21:24:43.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.172+0000 7fc6b4400640 1 -- 172.21.15.86:0/743429342 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6ac0c6550 con 0x7fc6ac0ccdd0 2024-04-20T21:24:43.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.172+0000 7fc6b4400640 1 -- 172.21.15.86:0/743429342 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6ac0c6b10 con 0x7fc6ac0ccdd0 2024-04-20T21:24:43.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.174+0000 7fc6b4400640 1 -- 172.21.15.86:0/743429342 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6ac0689e0 con 0x7fc6ac0ccdd0 2024-04-20T21:24:43.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.174+0000 7fc6a9600640 1 -- 172.21.15.86:0/743429342 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc69c0049e0 con 0x7fc6ac0ccdd0 2024-04-20T21:24:43.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.175+0000 7fc6a9600640 1 --2- 172.21.15.86:0/743429342 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc69006c8f0 0x7fc69006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:43.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.175+0000 7fc6a9600640 1 -- 172.21.15.86:0/743429342 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5882+0+0 (secure 0 0 0) 0x7fc69c08af20 con 0x7fc6ac0ccdd0 2024-04-20T21:24:43.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.176+0000 7fc6b1a00640 1 --2- 172.21.15.86:0/743429342 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc69006c8f0 0x7fc69006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:43.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.177+0000 7fc6b1a00640 1 --2- 172.21.15.86:0/743429342 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc69006c8f0 0x7fc69006edb0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0036000 tx=0x7fc6a0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:43.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.180+0000 7fc6a9600640 1 -- 172.21.15.86:0/743429342 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc69c059360 con 0x7fc6ac0ccdd0 2024-04-20T21:24:43.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.400+0000 7fc6b4400640 1 -- 172.21.15.86:0/743429342 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc6ac15ff50 con 0x7fc6ac0ccdd0 2024-04-20T21:24:43.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.401+0000 7fc6a9600640 1 -- 172.21.15.86:0/743429342 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+5206 (secure 0 0 0) 0x7fc69c058c60 con 0x7fc6ac0ccdd0 2024-04-20T21:24:43.400 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:24:43.400 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":50,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2740872549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2740872549},{"type":"v1","addr":"172.21.15.86:6836","nonce":2740872549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5048,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/301899138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":301899138},{"type":"v1","addr":"172.21.15.144:6833","nonce":301899138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5051,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2816343379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2816343379},{"type":"v1","addr":"172.21.15.144:6835","nonce":2816343379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:22:45.431100+0000","modified":"2024-04-20T21:24:11.116928+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6987},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6987":{"gid":6987,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":18,"addr":"172.21.15.86:6837/2254117576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2254117576},{"type":"v1","addr":"172.21.15.86:6837","nonce":2254117576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6987,"qdb_cluster":[6987]},"id":5}]} 2024-04-20T21:24:43.400 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 50 2024-04-20T21:24:43.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.403+0000 7fc6b4400640 1 -- 172.21.15.86:0/743429342 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc69006c8f0 msgr2=0x7fc69006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:43.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.403+0000 7fc6b4400640 1 --2- 172.21.15.86:0/743429342 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc69006c8f0 0x7fc69006edb0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0036000 tx=0x7fc6a0008030 comp rx=0 tx=0).stop 2024-04-20T21:24:43.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.403+0000 7fc6b4400640 1 -- 172.21.15.86:0/743429342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6ac0ccdd0 msgr2=0x7fc6ac15bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:43.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.403+0000 7fc6b4400640 1 --2- 172.21.15.86:0/743429342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6ac0ccdd0 0x7fc6ac15bcc0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fc69c00c4b0 tx=0x7fc69c00c970 comp rx=0 tx=0).stop 2024-04-20T21:24:43.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.403+0000 7fc6b4400640 1 -- 172.21.15.86:0/743429342 shutdown_connections 2024-04-20T21:24:43.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.403+0000 7fc6b4400640 1 --2- 172.21.15.86:0/743429342 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc69006c8f0 0x7fc69006edb0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:43.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.403+0000 7fc6b4400640 1 --2- 172.21.15.86:0/743429342 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc6ac146130 0x7fc6ac15c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:43.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.403+0000 7fc6b4400640 1 --2- 172.21.15.86:0/743429342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc6ac0ccdd0 0x7fc6ac15bcc0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:43.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.403+0000 7fc6b4400640 1 --2- 172.21.15.86:0/743429342 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6ac072410 0x7fc6ac15f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:43.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.403+0000 7fc6b4400640 1 -- 172.21.15.86:0/743429342 >> 172.21.15.86:0/743429342 conn(0x7fc6ac0b2360 msgr2=0x7fc6ac0b48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:43.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.404+0000 7fc6b4400640 1 -- 172.21.15.86:0/743429342 shutdown_connections 2024-04-20T21:24:43.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.404+0000 7fc6b4400640 1 -- 172.21.15.86:0/743429342 wait complete. 2024-04-20T21:24:43.421 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:24:43.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.615+0000 7f391a000640 1 Processor -- start 2024-04-20T21:24:43.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.616+0000 7f391a000640 1 -- start start 2024-04-20T21:24:43.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.617+0000 7f391a000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f39141629e0 0x7f39140b3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:43.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.617+0000 7f391a000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3914162dc0 0x7f39140b3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:43.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.617+0000 7f391a000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f391406e640 con 0x7f391406f6e0 2024-04-20T21:24:43.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.617+0000 7f391a000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f391406e920 con 0x7f39141629e0 2024-04-20T21:24:43.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.617+0000 7f391a000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f391406ec00 con 0x7f3914162dc0 2024-04-20T21:24:43.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.618+0000 7f3918c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f39141629e0 0x7f39140b3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:43.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.618+0000 7f3918c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f39141629e0 0x7f39140b3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51656/0 (socket says 172.21.15.86:51656) 2024-04-20T21:24:43.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.618+0000 7f3918c00640 1 -- 172.21.15.86:0/926418834 learned_addr learned my addr 172.21.15.86:0/926418834 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:43.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.618+0000 7f3918c00640 1 -- 172.21.15.86:0/926418834 >> v2:172.21.15.144:3300/0 conn(0x7f3914162dc0 msgr2=0x7f39140b3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:43.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.618+0000 7f3918c00640 1 --2- 172.21.15.86:0/926418834 >> v2:172.21.15.144:3300/0 conn(0x7f3914162dc0 0x7f39140b3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:43.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.618+0000 7f3918c00640 1 -- 172.21.15.86:0/926418834 >> v1:172.21.15.144:6789/0 conn(0x7f391406f6e0 legacy=0x7f39140ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:43.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.618+0000 7f3918c00640 1 -- 172.21.15.86:0/926418834 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f391406eee0 con 0x7f39141629e0 2024-04-20T21:24:43.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.619+0000 7f3918c00640 1 --2- 172.21.15.86:0/926418834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39141629e0 0x7f39140b3100 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f391406d8c0 tx=0x7f38fc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbb10a85d9926a73 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:43.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.619+0000 7f3913e00640 1 -- 172.21.15.86:0/926418834 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38fc035020 con 0x7f39141629e0 2024-04-20T21:24:43.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.619+0000 7f3913e00640 1 -- 172.21.15.86:0/926418834 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38fc031870 con 0x7f39141629e0 2024-04-20T21:24:43.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.619+0000 7f3913e00640 1 -- 172.21.15.86:0/926418834 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38fc031c40 con 0x7f39141629e0 2024-04-20T21:24:43.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.620+0000 7f391a000640 1 -- 172.21.15.86:0/926418834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39141629e0 msgr2=0x7f39140b3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:43.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.620+0000 7f391a000640 1 --2- 172.21.15.86:0/926418834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39141629e0 0x7f39140b3100 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f391406d8c0 tx=0x7f38fc02b910 comp rx=0 tx=0).stop 2024-04-20T21:24:43.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.620+0000 7f391a000640 1 -- 172.21.15.86:0/926418834 shutdown_connections 2024-04-20T21:24:43.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.620+0000 7f391a000640 1 --2- 172.21.15.86:0/926418834 >> v2:172.21.15.144:3300/0 conn(0x7f3914162dc0 0x7f39140b3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:43.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.620+0000 7f391a000640 1 --2- 172.21.15.86:0/926418834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39141629e0 0x7f39140b3100 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:43.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.620+0000 7f391a000640 1 -- 172.21.15.86:0/926418834 >> 172.21.15.86:0/926418834 conn(0x7f39140d0170 msgr2=0x7f39140d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:43.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.620+0000 7f391a000640 1 -- 172.21.15.86:0/926418834 shutdown_connections 2024-04-20T21:24:43.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.620+0000 7f391a000640 1 -- 172.21.15.86:0/926418834 wait complete. 2024-04-20T21:24:43.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.622+0000 7f391a000640 1 Processor -- start 2024-04-20T21:24:43.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.622+0000 7f391a000640 1 -- start start 2024-04-20T21:24:43.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.623+0000 7f391a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f391406f6e0 0x7f3914153860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:43.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.624+0000 7f391a000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f39141629e0 0x7f3914158e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:43.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.624+0000 7f391a000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3914159360 0x7f391415deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:43.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.624+0000 7f391a000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f39140ad970 con 0x7f391406f6e0 2024-04-20T21:24:43.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.624+0000 7f391a000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f39140ad7f0 con 0x7f3914159360 2024-04-20T21:24:43.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.624+0000 7f391a000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f39140adaf0 con 0x7f39141629e0 2024-04-20T21:24:43.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.624+0000 7f3918c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f391406f6e0 0x7f3914153860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:43.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.624+0000 7f3918c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f391406f6e0 0x7f3914153860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51664/0 (socket says 172.21.15.86:51664) 2024-04-20T21:24:43.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.624+0000 7f3918c00640 1 -- 172.21.15.86:0/2886251858 learned_addr learned my addr 172.21.15.86:0/2886251858 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:43.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.624+0000 7f3918c00640 1 -- 172.21.15.86:0/2886251858 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f39141629e0 msgr2=0x7f3914158e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:43.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.624+0000 7f3918c00640 1 --2- 172.21.15.86:0/2886251858 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f39141629e0 0x7f3914158e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:43.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.624+0000 7f3918c00640 1 -- 172.21.15.86:0/2886251858 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3914159360 msgr2=0x7f391415deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:43.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.624+0000 7f3918c00640 1 --2- 172.21.15.86:0/2886251858 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3914159360 0x7f391415deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:43.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.624+0000 7f3918c00640 1 -- 172.21.15.86:0/2886251858 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39140ce2f0 con 0x7f391406f6e0 2024-04-20T21:24:43.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.625+0000 7f3918c00640 1 --2- 172.21.15.86:0/2886251858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f391406f6e0 0x7f3914153860 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f38fc000c00 tx=0x7f38fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:43.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.625+0000 7f3911600640 1 -- 172.21.15.86:0/2886251858 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38fc03ad60 con 0x7f391406f6e0 2024-04-20T21:24:43.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.625+0000 7f3911600640 1 -- 172.21.15.86:0/2886251858 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38fc0443f0 con 0x7f391406f6e0 2024-04-20T21:24:43.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.625+0000 7f3911600640 1 -- 172.21.15.86:0/2886251858 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38fc0447e0 con 0x7f391406f6e0 2024-04-20T21:24:43.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.625+0000 7f391a000640 1 -- 172.21.15.86:0/2886251858 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39140ce560 con 0x7f391406f6e0 2024-04-20T21:24:43.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.625+0000 7f391a000640 1 -- 172.21.15.86:0/2886251858 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39140ceb20 con 0x7f391406f6e0 2024-04-20T21:24:43.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.626+0000 7f391a000640 1 -- 172.21.15.86:0/2886251858 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f391404fcf0 con 0x7f391406f6e0 2024-04-20T21:24:43.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.627+0000 7f3911600640 1 -- 172.21.15.86:0/2886251858 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f38fc04b060 con 0x7f391406f6e0 2024-04-20T21:24:43.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.628+0000 7f3911600640 1 --2- 172.21.15.86:0/2886251858 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f38f006c8f0 0x7f38f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:43.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.628+0000 7f3911600640 1 -- 172.21.15.86:0/2886251858 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5882+0+0 (secure 0 0 0) 0x7f38fc0adff0 con 0x7f391406f6e0 2024-04-20T21:24:43.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.628+0000 7f390be00640 1 --2- 172.21.15.86:0/2886251858 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f38f006c8f0 0x7f38f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:43.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.629+0000 7f390be00640 1 --2- 172.21.15.86:0/2886251858 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f38f006c8f0 0x7f38f006edb0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f39140cf5e0 tx=0x7f3904006bf0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:43.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.632+0000 7f3911600640 1 -- 172.21.15.86:0/2886251858 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f38fc03f070 con 0x7f391406f6e0 2024-04-20T21:24:43.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.832+0000 7f391a000640 1 -- 172.21.15.86:0/2886251858 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f391415f920 con 0x7f391406f6e0 2024-04-20T21:24:43.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.835+0000 7f3911600640 1 -- 172.21.15.86:0/2886251858 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+16941 (secure 0 0 0) 0x7f38fc02f320 con 0x7f391406f6e0 2024-04-20T21:24:43.835 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:24:43.835 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":74,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:23:35.220547+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:22:43.125767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-20T21:22:44.410549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.2d","mappings":[{"from":6,"to":0}]},{"pgid":"11.2e","mappings":[{"from":6,"to":0}]},{"pgid":"11.33","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6837/202143875":"2024-04-21T21:23:12.079572+0000","172.21.15.86:6835/202143875":"2024-04-21T21:23:12.079572+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:24:43.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.838+0000 7f391a000640 1 -- 172.21.15.86:0/2886251858 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f38f006c8f0 msgr2=0x7f38f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:43.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.838+0000 7f391a000640 1 --2- 172.21.15.86:0/2886251858 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f38f006c8f0 0x7f38f006edb0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f39140cf5e0 tx=0x7f3904006bf0 comp rx=0 tx=0).stop 2024-04-20T21:24:43.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.838+0000 7f391a000640 1 -- 172.21.15.86:0/2886251858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f391406f6e0 msgr2=0x7f3914153860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:43.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.838+0000 7f391a000640 1 --2- 172.21.15.86:0/2886251858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f391406f6e0 0x7f3914153860 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f38fc000c00 tx=0x7f38fc0046c0 comp rx=0 tx=0).stop 2024-04-20T21:24:43.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.839+0000 7f391a000640 1 -- 172.21.15.86:0/2886251858 shutdown_connections 2024-04-20T21:24:43.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.839+0000 7f391a000640 1 --2- 172.21.15.86:0/2886251858 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f38f006c8f0 0x7f38f006edb0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:43.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.839+0000 7f391a000640 1 --2- 172.21.15.86:0/2886251858 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3914159360 0x7f391415deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:43.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.839+0000 7f391a000640 1 --2- 172.21.15.86:0/2886251858 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f39141629e0 0x7f3914158e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:43.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.839+0000 7f391a000640 1 --2- 172.21.15.86:0/2886251858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f391406f6e0 0x7f3914153860 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:43.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.839+0000 7f391a000640 1 -- 172.21.15.86:0/2886251858 >> 172.21.15.86:0/2886251858 conn(0x7f39140d0170 msgr2=0x7f39140b4790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:43.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.839+0000 7f391a000640 1 -- 172.21.15.86:0/2886251858 shutdown_connections 2024-04-20T21:24:43.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:43.839+0000 7f391a000640 1 -- 172.21.15.86:0/2886251858 wait complete. 2024-04-20T21:24:43.857 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-20T21:24:43.857 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:24:44.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.057+0000 7fe6ccc00640 1 Processor -- start 2024-04-20T21:24:44.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.057+0000 7fe6ccc00640 1 -- start start 2024-04-20T21:24:44.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.059+0000 7fe6ccc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6c8146130 0x7fe6c806e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:44.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.059+0000 7fe6ccc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe6c8072410 0x7fe6c80cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:44.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.059+0000 7fe6ccc00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6c806dc30 con 0x7fe6c80ccdd0 2024-04-20T21:24:44.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.059+0000 7fe6ccc00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe6c80cd8e0 con 0x7fe6c8072410 2024-04-20T21:24:44.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.059+0000 7fe6ccc00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe6c80cdbc0 con 0x7fe6c8146130 2024-04-20T21:24:44.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.059+0000 7fe6c6800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe6c8072410 0x7fe6c80cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:44.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.059+0000 7fe6c6800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe6c8072410 0x7fe6c80cd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51674/0 (socket says 172.21.15.86:51674) 2024-04-20T21:24:44.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.059+0000 7fe6c6800640 1 -- 172.21.15.86:0/1110049560 learned_addr learned my addr 172.21.15.86:0/1110049560 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:44.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.060+0000 7fe6c5e00640 1 --2- 172.21.15.86:0/1110049560 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6c8146130 0x7fe6c806e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:44.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.060+0000 7fe6c6800640 1 -- 172.21.15.86:0/1110049560 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6c8146130 msgr2=0x7fe6c806e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:44.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.060+0000 7fe6c6800640 1 --2- 172.21.15.86:0/1110049560 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6c8146130 0x7fe6c806e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:44.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.060+0000 7fe6c6800640 1 -- 172.21.15.86:0/1110049560 >> v1:172.21.15.144:6789/0 conn(0x7fe6c80ccdd0 legacy=0x7fe6c8071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:44.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.060+0000 7fe6c6800640 1 -- 172.21.15.86:0/1110049560 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6c806ded0 con 0x7fe6c8072410 2024-04-20T21:24:44.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.061+0000 7fe6c6800640 1 --2- 172.21.15.86:0/1110049560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6c8072410 0x7fe6c80cd230 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0009870 tx=0x7fe6c002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5deb40e0885fdca server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:44.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.061+0000 7fe6c4a00640 1 -- 172.21.15.86:0/1110049560 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c0034070 con 0x7fe6c8072410 2024-04-20T21:24:44.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.061+0000 7fe6c4a00640 1 -- 172.21.15.86:0/1110049560 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6c0030840 con 0x7fe6c8072410 2024-04-20T21:24:44.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.061+0000 7fe6c4a00640 1 -- 172.21.15.86:0/1110049560 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c0030c10 con 0x7fe6c8072410 2024-04-20T21:24:44.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.062+0000 7fe6ccc00640 1 -- 172.21.15.86:0/1110049560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6c8072410 msgr2=0x7fe6c80cd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:44.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.062+0000 7fe6ccc00640 1 --2- 172.21.15.86:0/1110049560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6c8072410 0x7fe6c80cd230 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0009870 tx=0x7fe6c002b980 comp rx=0 tx=0).stop 2024-04-20T21:24:44.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.062+0000 7fe6ccc00640 1 -- 172.21.15.86:0/1110049560 shutdown_connections 2024-04-20T21:24:44.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.062+0000 7fe6ccc00640 1 --2- 172.21.15.86:0/1110049560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6c8072410 0x7fe6c80cd230 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:44.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.062+0000 7fe6ccc00640 1 --2- 172.21.15.86:0/1110049560 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6c8146130 0x7fe6c806e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:44.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.062+0000 7fe6ccc00640 1 -- 172.21.15.86:0/1110049560 >> 172.21.15.86:0/1110049560 conn(0x7fe6c80b2360 msgr2=0x7fe6c80b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:44.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.062+0000 7fe6ccc00640 1 -- 172.21.15.86:0/1110049560 shutdown_connections 2024-04-20T21:24:44.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.062+0000 7fe6ccc00640 1 -- 172.21.15.86:0/1110049560 wait complete. 2024-04-20T21:24:44.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.064+0000 7fe6ccc00640 1 Processor -- start 2024-04-20T21:24:44.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.064+0000 7fe6ccc00640 1 -- start start 2024-04-20T21:24:44.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.065+0000 7fe6ccc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6c8072410 0x7fe6c813dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:44.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.065+0000 7fe6ccc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6c80ccdd0 0x7fe6c813bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:44.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.065+0000 7fe6ccc00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6c8146130 0x7fe6c8144bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:44.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.065+0000 7fe6ccc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c815cc20 con 0x7fe6c80ccdd0 2024-04-20T21:24:44.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.065+0000 7fe6ccc00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c806f150 con 0x7fe6c8146130 2024-04-20T21:24:44.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.065+0000 7fe6ccc00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe6c815cda0 con 0x7fe6c8072410 2024-04-20T21:24:44.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.065+0000 7fe6c5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6c80ccdd0 0x7fe6c813bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:44.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.065+0000 7fe6c5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6c80ccdd0 0x7fe6c813bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51678/0 (socket says 172.21.15.86:51678) 2024-04-20T21:24:44.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.065+0000 7fe6c5400640 1 -- 172.21.15.86:0/4037702299 learned_addr learned my addr 172.21.15.86:0/4037702299 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:44.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.065+0000 7fe6c5e00640 1 --2- 172.21.15.86:0/4037702299 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6c8072410 0x7fe6c813dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:44.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.065+0000 7fe6c5400640 1 -- 172.21.15.86:0/4037702299 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6c8072410 msgr2=0x7fe6c813dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:44.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.065+0000 7fe6c5400640 1 --2- 172.21.15.86:0/4037702299 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6c8072410 0x7fe6c813dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:44.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.065+0000 7fe6c5400640 1 -- 172.21.15.86:0/4037702299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6c8146130 msgr2=0x7fe6c8144bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:44.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.065+0000 7fe6c5400640 1 --2- 172.21.15.86:0/4037702299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6c8146130 0x7fe6c8144bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:44.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.065+0000 7fe6c5400640 1 -- 172.21.15.86:0/4037702299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6c813e0f0 con 0x7fe6c80ccdd0 2024-04-20T21:24:44.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.066+0000 7fe6c5400640 1 --2- 172.21.15.86:0/4037702299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6c80ccdd0 0x7fe6c813bd50 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fe6b800c7c0 tx=0x7fe6b800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:44.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.066+0000 7fe6be000640 1 -- 172.21.15.86:0/4037702299 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b8016020 con 0x7fe6c80ccdd0 2024-04-20T21:24:44.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.066+0000 7fe6be000640 1 -- 172.21.15.86:0/4037702299 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6b800aa50 con 0x7fe6c80ccdd0 2024-04-20T21:24:44.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.066+0000 7fe6ccc00640 1 -- 172.21.15.86:0/4037702299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6c80c9a50 con 0x7fe6c80ccdd0 2024-04-20T21:24:44.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.066+0000 7fe6be000640 1 -- 172.21.15.86:0/4037702299 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b8012420 con 0x7fe6c80ccdd0 2024-04-20T21:24:44.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.066+0000 7fe6ccc00640 1 -- 172.21.15.86:0/4037702299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6c80ca010 con 0x7fe6c80ccdd0 2024-04-20T21:24:44.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.067+0000 7fe6ccc00640 1 -- 172.21.15.86:0/4037702299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6c80689e0 con 0x7fe6c80ccdd0 2024-04-20T21:24:44.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.068+0000 7fe6be000640 1 -- 172.21.15.86:0/4037702299 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe6b8019020 con 0x7fe6c80ccdd0 2024-04-20T21:24:44.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.069+0000 7fe6be000640 1 --2- 172.21.15.86:0/4037702299 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe6a406c8f0 0x7fe6a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:44.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.069+0000 7fe6be000640 1 -- 172.21.15.86:0/4037702299 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5882+0+0 (secure 0 0 0) 0x7fe6b808bc50 con 0x7fe6c80ccdd0 2024-04-20T21:24:44.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.069+0000 7fe6c5e00640 1 --2- 172.21.15.86:0/4037702299 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe6a406c8f0 0x7fe6a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:44.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.070+0000 7fe6c5e00640 1 --2- 172.21.15.86:0/4037702299 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe6a406c8f0 0x7fe6a406edb0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0006fd0 tx=0x7fe6b0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:44.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.074+0000 7fe6be000640 1 -- 172.21.15.86:0/4037702299 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe6b8020030 con 0x7fe6c80ccdd0 2024-04-20T21:24:44.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.314+0000 7fe6ccc00640 1 -- 172.21.15.86:0/4037702299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6c80b1760 con 0x7fe6c80ccdd0 2024-04-20T21:24:44.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.315+0000 7fe6be000640 1 -- 172.21.15.86:0/4037702299 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+5206 (secure 0 0 0) 0x7fe6b8059fc0 con 0x7fe6c80ccdd0 2024-04-20T21:24:44.314 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:24:44.314 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":50,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2740872549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2740872549},{"type":"v1","addr":"172.21.15.86:6836","nonce":2740872549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5048,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/301899138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":301899138},{"type":"v1","addr":"172.21.15.144:6833","nonce":301899138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5051,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2816343379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2816343379},{"type":"v1","addr":"172.21.15.144:6835","nonce":2816343379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:22:45.431100+0000","modified":"2024-04-20T21:24:11.116928+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6987},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6987":{"gid":6987,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":18,"addr":"172.21.15.86:6837/2254117576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2254117576},{"type":"v1","addr":"172.21.15.86:6837","nonce":2254117576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6987,"qdb_cluster":[6987]},"id":5}]} 2024-04-20T21:24:44.314 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 50 2024-04-20T21:24:44.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.317+0000 7fe6ccc00640 1 -- 172.21.15.86:0/4037702299 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe6a406c8f0 msgr2=0x7fe6a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:44.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.317+0000 7fe6ccc00640 1 --2- 172.21.15.86:0/4037702299 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe6a406c8f0 0x7fe6a406edb0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0006fd0 tx=0x7fe6b0008040 comp rx=0 tx=0).stop 2024-04-20T21:24:44.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.317+0000 7fe6ccc00640 1 -- 172.21.15.86:0/4037702299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6c80ccdd0 msgr2=0x7fe6c813bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:44.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.317+0000 7fe6ccc00640 1 --2- 172.21.15.86:0/4037702299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6c80ccdd0 0x7fe6c813bd50 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fe6b800c7c0 tx=0x7fe6b800cc80 comp rx=0 tx=0).stop 2024-04-20T21:24:44.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.318+0000 7fe6ccc00640 1 -- 172.21.15.86:0/4037702299 shutdown_connections 2024-04-20T21:24:44.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.318+0000 7fe6ccc00640 1 --2- 172.21.15.86:0/4037702299 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe6a406c8f0 0x7fe6a406edb0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:44.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.318+0000 7fe6ccc00640 1 --2- 172.21.15.86:0/4037702299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6c8146130 0x7fe6c8144bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:44.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.318+0000 7fe6ccc00640 1 --2- 172.21.15.86:0/4037702299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6c80ccdd0 0x7fe6c813bd50 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:44.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.318+0000 7fe6ccc00640 1 --2- 172.21.15.86:0/4037702299 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6c8072410 0x7fe6c813dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:44.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.318+0000 7fe6ccc00640 1 -- 172.21.15.86:0/4037702299 >> 172.21.15.86:0/4037702299 conn(0x7fe6c80b2360 msgr2=0x7fe6c80b48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:44.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.318+0000 7fe6ccc00640 1 -- 172.21.15.86:0/4037702299 shutdown_connections 2024-04-20T21:24:44.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.318+0000 7fe6ccc00640 1 -- 172.21.15.86:0/4037702299 wait complete. 2024-04-20T21:24:44.335 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:24:44.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.527+0000 7fd4ea400640 1 Processor -- start 2024-04-20T21:24:44.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.527+0000 7fd4ea400640 1 -- start start 2024-04-20T21:24:44.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.528+0000 7fd4ea400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd4e4156ae0 0x7fd4e415d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:44.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.528+0000 7fd4ea400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4e40ad7f0 0x7fd4e406e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:44.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.528+0000 7fd4ea400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4e4160890 con 0x7fd4e4160080 2024-04-20T21:24:44.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.528+0000 7fd4ea400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fd4e406db20 con 0x7fd4e4156ae0 2024-04-20T21:24:44.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.528+0000 7fd4ea400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd4e4160b70 con 0x7fd4e40ad7f0 2024-04-20T21:24:44.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.529+0000 7fd4e9a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4e40ad7f0 0x7fd4e406e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:44.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.529+0000 7fd4e3e00640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fd4e4160080 0x7fd4e40ad0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.86:53924/0 (socket says 172.21.15.86:53924) 2024-04-20T21:24:44.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.529+0000 7fd4e3e00640 1 -- 172.21.15.86:0/557957175 learned_addr learned my addr 172.21.15.86:0/557957175 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:24:44.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.529+0000 7fd4e9000640 1 --2- 172.21.15.86:0/557957175 >> v2:172.21.15.144:3300/0 conn(0x7fd4e4156ae0 0x7fd4e415d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:44.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.530+0000 7fd4e9000640 1 -- 172.21.15.86:0/557957175 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4e40ad7f0 msgr2=0x7fd4e406e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:44.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.530+0000 7fd4e9000640 1 --2- 172.21.15.86:0/557957175 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4e40ad7f0 0x7fd4e406e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:44.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.530+0000 7fd4e9000640 1 -- 172.21.15.86:0/557957175 >> v1:172.21.15.144:6789/0 conn(0x7fd4e4160080 legacy=0x7fd4e40ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:44.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.530+0000 7fd4e9000640 1 -- 172.21.15.86:0/557957175 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4e4160e50 con 0x7fd4e4156ae0 2024-04-20T21:24:44.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.531+0000 7fd4e9000640 1 --2- 172.21.15.86:0/557957175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4e4156ae0 0x7fd4e415d6a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd4d80097a0 tx=0x7fd4d802b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68eaeab68a08874 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:44.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.532+0000 7fd4e3400640 1 -- 172.21.15.86:0/557957175 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d8035020 con 0x7fd4e4156ae0 2024-04-20T21:24:44.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.532+0000 7fd4e3400640 1 -- 172.21.15.86:0/557957175 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4d8031810 con 0x7fd4e4156ae0 2024-04-20T21:24:44.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.532+0000 7fd4e3400640 1 -- 172.21.15.86:0/557957175 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d8031be0 con 0x7fd4e4156ae0 2024-04-20T21:24:44.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.532+0000 7fd4ea400640 1 -- 172.21.15.86:0/557957175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4e4156ae0 msgr2=0x7fd4e415d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:44.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.532+0000 7fd4ea400640 1 --2- 172.21.15.86:0/557957175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4e4156ae0 0x7fd4e415d6a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd4d80097a0 tx=0x7fd4d802b8b0 comp rx=0 tx=0).stop 2024-04-20T21:24:44.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.533+0000 7fd4ea400640 1 -- 172.21.15.86:0/557957175 shutdown_connections 2024-04-20T21:24:44.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.533+0000 7fd4ea400640 1 --2- 172.21.15.86:0/557957175 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4e40ad7f0 0x7fd4e406e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:44.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.533+0000 7fd4ea400640 1 --2- 172.21.15.86:0/557957175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4e4156ae0 0x7fd4e415d6a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:44.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.533+0000 7fd4ea400640 1 -- 172.21.15.86:0/557957175 >> 172.21.15.86:0/557957175 conn(0x7fd4e40d0170 msgr2=0x7fd4e40d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:44.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.533+0000 7fd4ea400640 1 -- 172.21.15.86:0/557957175 shutdown_connections 2024-04-20T21:24:44.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.533+0000 7fd4ea400640 1 -- 172.21.15.86:0/557957175 wait complete. 2024-04-20T21:24:44.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.534+0000 7fd4ea400640 1 Processor -- start 2024-04-20T21:24:44.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.534+0000 7fd4ea400640 1 -- start start 2024-04-20T21:24:44.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.535+0000 7fd4ea400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4e40ad7f0 0x7fd4e409a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:44.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.535+0000 7fd4ea400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4e4156ae0 0x7fd4e415f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:44.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.535+0000 7fd4ea400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4e4160080 0x7fd4e415f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:44.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.535+0000 7fd4ea400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e4160500 con 0x7fd4e40ad7f0 2024-04-20T21:24:44.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.535+0000 7fd4ea400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e4156ec0 con 0x7fd4e4156ae0 2024-04-20T21:24:44.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.535+0000 7fd4ea400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd4e406e7d0 con 0x7fd4e4160080 2024-04-20T21:24:44.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.536+0000 7fd4e9000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4e40ad7f0 0x7fd4e409a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:44.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.536+0000 7fd4e9000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4e40ad7f0 0x7fd4e409a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51692/0 (socket says 172.21.15.86:51692) 2024-04-20T21:24:44.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.536+0000 7fd4e9000640 1 -- 172.21.15.86:0/3120881306 learned_addr learned my addr 172.21.15.86:0/3120881306 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:44.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.536+0000 7fd4e3e00640 1 --2- 172.21.15.86:0/3120881306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4e4156ae0 0x7fd4e415f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:44.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.536+0000 7fd4e9000640 1 -- 172.21.15.86:0/3120881306 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4e4160080 msgr2=0x7fd4e415f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:44.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.536+0000 7fd4e9000640 1 --2- 172.21.15.86:0/3120881306 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4e4160080 0x7fd4e415f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:44.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.536+0000 7fd4e9000640 1 -- 172.21.15.86:0/3120881306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4e4156ae0 msgr2=0x7fd4e415f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:44.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.536+0000 7fd4e9000640 1 --2- 172.21.15.86:0/3120881306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4e4156ae0 0x7fd4e415f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:44.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.536+0000 7fd4e9000640 1 -- 172.21.15.86:0/3120881306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4e4157f90 con 0x7fd4e40ad7f0 2024-04-20T21:24:44.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.536+0000 7fd4e9000640 1 --2- 172.21.15.86:0/3120881306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4e40ad7f0 0x7fd4e409a260 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fd4d802bec0 tx=0x7fd4d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:44.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.537+0000 7fd4e0c00640 1 -- 172.21.15.86:0/3120881306 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d8041070 con 0x7fd4e40ad7f0 2024-04-20T21:24:44.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.537+0000 7fd4e0c00640 1 -- 172.21.15.86:0/3120881306 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4d803ac00 con 0x7fd4e40ad7f0 2024-04-20T21:24:44.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.537+0000 7fd4e0c00640 1 -- 172.21.15.86:0/3120881306 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d8035040 con 0x7fd4e40ad7f0 2024-04-20T21:24:44.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.537+0000 7fd4ea400640 1 -- 172.21.15.86:0/3120881306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4e4143fe0 con 0x7fd4e40ad7f0 2024-04-20T21:24:44.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.537+0000 7fd4ea400640 1 -- 172.21.15.86:0/3120881306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4e41445a0 con 0x7fd4e40ad7f0 2024-04-20T21:24:44.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.538+0000 7fd4ea400640 1 -- 172.21.15.86:0/3120881306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4e404fcf0 con 0x7fd4e40ad7f0 2024-04-20T21:24:44.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.539+0000 7fd4e0c00640 1 -- 172.21.15.86:0/3120881306 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd4d804c060 con 0x7fd4e40ad7f0 2024-04-20T21:24:44.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.540+0000 7fd4e0c00640 1 --2- 172.21.15.86:0/3120881306 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd4c406c8f0 0x7fd4c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:44.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.540+0000 7fd4e0c00640 1 -- 172.21.15.86:0/3120881306 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5882+0+0 (secure 0 0 0) 0x7fd4d80ad680 con 0x7fd4e40ad7f0 2024-04-20T21:24:44.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.540+0000 7fd4e3e00640 1 --2- 172.21.15.86:0/3120881306 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd4c406c8f0 0x7fd4c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:44.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.540+0000 7fd4e3e00640 1 --2- 172.21.15.86:0/3120881306 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd4c406c8f0 0x7fd4c406edb0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fd4e40b0ff0 tx=0x7fd4d0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:44.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.544+0000 7fd4e0c00640 1 -- 172.21.15.86:0/3120881306 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd4d807ba90 con 0x7fd4e40ad7f0 2024-04-20T21:24:44.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.781+0000 7fd4ea400640 1 -- 172.21.15.86:0/3120881306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4e406fb40 con 0x7fd4e40ad7f0 2024-04-20T21:24:44.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.783+0000 7fd4e0c00640 1 -- 172.21.15.86:0/3120881306 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+5206 (secure 0 0 0) 0x7fd4d807b390 con 0x7fd4e40ad7f0 2024-04-20T21:24:44.782 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:24:44.782 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":50,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2740872549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2740872549},{"type":"v1","addr":"172.21.15.86:6836","nonce":2740872549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5048,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/301899138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":301899138},{"type":"v1","addr":"172.21.15.144:6833","nonce":301899138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5051,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2816343379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2816343379},{"type":"v1","addr":"172.21.15.144:6835","nonce":2816343379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:22:45.431100+0000","modified":"2024-04-20T21:24:11.116928+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6987},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6987":{"gid":6987,"name":"a","rank":0,"incarnation":45,"state":"up:active","state_seq":18,"addr":"172.21.15.86:6837/2254117576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2254117576},{"type":"v1","addr":"172.21.15.86:6837","nonce":2254117576}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6987,"qdb_cluster":[6987]},"id":5}]} 2024-04-20T21:24:44.782 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 50 2024-04-20T21:24:44.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.785+0000 7fd4ea400640 1 -- 172.21.15.86:0/3120881306 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd4c406c8f0 msgr2=0x7fd4c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:44.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.785+0000 7fd4ea400640 1 --2- 172.21.15.86:0/3120881306 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd4c406c8f0 0x7fd4c406edb0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fd4e40b0ff0 tx=0x7fd4d0009040 comp rx=0 tx=0).stop 2024-04-20T21:24:44.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.785+0000 7fd4ea400640 1 -- 172.21.15.86:0/3120881306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4e40ad7f0 msgr2=0x7fd4e409a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:44.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.785+0000 7fd4ea400640 1 --2- 172.21.15.86:0/3120881306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4e40ad7f0 0x7fd4e409a260 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fd4d802bec0 tx=0x7fd4d80046c0 comp rx=0 tx=0).stop 2024-04-20T21:24:44.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.785+0000 7fd4ea400640 1 -- 172.21.15.86:0/3120881306 shutdown_connections 2024-04-20T21:24:44.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.785+0000 7fd4ea400640 1 --2- 172.21.15.86:0/3120881306 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd4c406c8f0 0x7fd4c406edb0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:44.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.785+0000 7fd4ea400640 1 --2- 172.21.15.86:0/3120881306 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4e4160080 0x7fd4e415f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:44.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.785+0000 7fd4ea400640 1 --2- 172.21.15.86:0/3120881306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4e4156ae0 0x7fd4e415f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:44.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.785+0000 7fd4ea400640 1 --2- 172.21.15.86:0/3120881306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4e40ad7f0 0x7fd4e409a260 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:44.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.785+0000 7fd4ea400640 1 -- 172.21.15.86:0/3120881306 >> 172.21.15.86:0/3120881306 conn(0x7fd4e40d0170 msgr2=0x7fd4e40730d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:44.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.786+0000 7fd4ea400640 1 -- 172.21.15.86:0/3120881306 shutdown_connections 2024-04-20T21:24:44.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:44.786+0000 7fd4ea400640 1 -- 172.21.15.86:0/3120881306 wait complete. 2024-04-20T21:24:44.803 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:24:45.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.005+0000 7f64f6e00640 1 Processor -- start 2024-04-20T21:24:45.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.005+0000 7f64f6e00640 1 -- start start 2024-04-20T21:24:45.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.007+0000 7f64f6e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f64f00af7b0 0x7f64f015c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:45.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.007+0000 7f64f6e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f64f006f4c0 0x7f64f006f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:45.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.007+0000 7f64f6e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64f00ae710 con 0x7f64f015c010 2024-04-20T21:24:45.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.007+0000 7f64f6e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f64f00af320 con 0x7f64f00af7b0 2024-04-20T21:24:45.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.007+0000 7f64f6e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f64f00ae4b0 con 0x7f64f006f4c0 2024-04-20T21:24:45.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.007+0000 7f64f5000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f64f00af7b0 0x7f64f015c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:45.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.007+0000 7f64f5000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f64f00af7b0 0x7f64f015c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51702/0 (socket says 172.21.15.86:51702) 2024-04-20T21:24:45.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.007+0000 7f64f5000640 1 -- 172.21.15.86:0/1471758857 learned_addr learned my addr 172.21.15.86:0/1471758857 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:45.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.008+0000 7f64f6400640 1 --2- 172.21.15.86:0/1471758857 >> v2:172.21.15.144:3300/0 conn(0x7f64f006f4c0 0x7f64f006f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:45.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.008+0000 7f64f5000640 1 -- 172.21.15.86:0/1471758857 >> v2:172.21.15.144:3300/0 conn(0x7f64f006f4c0 msgr2=0x7f64f006f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:45.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.008+0000 7f64f5000640 1 --2- 172.21.15.86:0/1471758857 >> v2:172.21.15.144:3300/0 conn(0x7f64f006f4c0 0x7f64f006f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:45.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.008+0000 7f64f5000640 1 -- 172.21.15.86:0/1471758857 >> v1:172.21.15.144:6789/0 conn(0x7f64f015c010 legacy=0x7f64f00adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:45.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.008+0000 7f64f5000640 1 -- 172.21.15.86:0/1471758857 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64f00ae9f0 con 0x7f64f00af7b0 2024-04-20T21:24:45.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.008+0000 7f64f6400640 1 --2- 172.21.15.86:0/1471758857 >> v2:172.21.15.144:3300/0 conn(0x7f64f006f4c0 0x7f64f006f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:24:45.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.009+0000 7f64f5000640 1 --2- 172.21.15.86:0/1471758857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64f00af7b0 0x7f64f015c3f0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f64f00af4a0 tx=0x7f64dc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ba17c02ff3c8f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:45.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.009+0000 7f64efe00640 1 -- 172.21.15.86:0/1471758857 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64dc035020 con 0x7f64f00af7b0 2024-04-20T21:24:45.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.009+0000 7f64efe00640 1 -- 172.21.15.86:0/1471758857 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64dc031870 con 0x7f64f00af7b0 2024-04-20T21:24:45.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.009+0000 7f64efe00640 1 -- 172.21.15.86:0/1471758857 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64dc031c40 con 0x7f64f00af7b0 2024-04-20T21:24:45.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.010+0000 7f64f6e00640 1 -- 172.21.15.86:0/1471758857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64f00af7b0 msgr2=0x7f64f015c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:45.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.010+0000 7f64f6e00640 1 --2- 172.21.15.86:0/1471758857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64f00af7b0 0x7f64f015c3f0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f64f00af4a0 tx=0x7f64dc02b910 comp rx=0 tx=0).stop 2024-04-20T21:24:45.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.010+0000 7f64f6e00640 1 -- 172.21.15.86:0/1471758857 shutdown_connections 2024-04-20T21:24:45.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.010+0000 7f64f6e00640 1 --2- 172.21.15.86:0/1471758857 >> v2:172.21.15.144:3300/0 conn(0x7f64f006f4c0 0x7f64f006f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:45.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.010+0000 7f64f6e00640 1 --2- 172.21.15.86:0/1471758857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64f00af7b0 0x7f64f015c3f0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:45.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.010+0000 7f64f6e00640 1 -- 172.21.15.86:0/1471758857 >> 172.21.15.86:0/1471758857 conn(0x7f64f00d0170 msgr2=0x7f64f00d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:45.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.010+0000 7f64f6e00640 1 -- 172.21.15.86:0/1471758857 shutdown_connections 2024-04-20T21:24:45.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.010+0000 7f64f6e00640 1 -- 172.21.15.86:0/1471758857 wait complete. 2024-04-20T21:24:45.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.012+0000 7f64f6e00640 1 Processor -- start 2024-04-20T21:24:45.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.012+0000 7f64f6e00640 1 -- start start 2024-04-20T21:24:45.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.013+0000 7f64f6e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64f006f4c0 0x7f64f01f1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:45.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.013+0000 7f64f6e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64f00af7b0 0x7f64f0099800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:45.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.013+0000 7f64f6e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64f015c010 0x7f64f015d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:45.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.013+0000 7f64f6e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f64f006fe30 con 0x7f64f00af7b0 2024-04-20T21:24:45.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.013+0000 7f64f6e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f64f015c930 con 0x7f64f006f4c0 2024-04-20T21:24:45.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.013+0000 7f64f6e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f64f00aefb0 con 0x7f64f015c010 2024-04-20T21:24:45.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.013+0000 7f64f5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64f00af7b0 0x7f64f0099800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:45.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.013+0000 7f64f5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64f00af7b0 0x7f64f0099800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51706/0 (socket says 172.21.15.86:51706) 2024-04-20T21:24:45.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.013+0000 7f64f5000640 1 -- 172.21.15.86:0/3872443237 learned_addr learned my addr 172.21.15.86:0/3872443237 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:45.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.013+0000 7f64f5000640 1 -- 172.21.15.86:0/3872443237 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64f015c010 msgr2=0x7f64f015d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:45.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.013+0000 7f64f6400640 1 --2- 172.21.15.86:0/3872443237 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64f015c010 0x7f64f015d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:45.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.013+0000 7f64f5a00640 1 --2- 172.21.15.86:0/3872443237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64f006f4c0 0x7f64f01f1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:45.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.013+0000 7f64f5000640 1 --2- 172.21.15.86:0/3872443237 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64f015c010 0x7f64f015d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:45.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.013+0000 7f64f5000640 1 -- 172.21.15.86:0/3872443237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64f006f4c0 msgr2=0x7f64f01f1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:45.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.013+0000 7f64f5000640 1 --2- 172.21.15.86:0/3872443237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64f006f4c0 0x7f64f01f1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:45.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.013+0000 7f64f5000640 1 -- 172.21.15.86:0/3872443237 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64f01f1940 con 0x7f64f00af7b0 2024-04-20T21:24:45.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.014+0000 7f64f5000640 1 --2- 172.21.15.86:0/3872443237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64f00af7b0 0x7f64f0099800 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f64dc000c00 tx=0x7f64dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:45.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.014+0000 7f64ed600640 1 -- 172.21.15.86:0/3872443237 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64dc040070 con 0x7f64f00af7b0 2024-04-20T21:24:45.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.014+0000 7f64ed600640 1 -- 172.21.15.86:0/3872443237 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64dc03a420 con 0x7f64f00af7b0 2024-04-20T21:24:45.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.014+0000 7f64ed600640 1 -- 172.21.15.86:0/3872443237 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64dc035040 con 0x7f64f00af7b0 2024-04-20T21:24:45.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.014+0000 7f64f6e00640 1 -- 172.21.15.86:0/3872443237 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64f009a260 con 0x7f64f00af7b0 2024-04-20T21:24:45.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.014+0000 7f64f6e00640 1 -- 172.21.15.86:0/3872443237 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64f009a520 con 0x7f64f00af7b0 2024-04-20T21:24:45.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.015+0000 7f64f6e00640 1 -- 172.21.15.86:0/3872443237 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64f004fcf0 con 0x7f64f00af7b0 2024-04-20T21:24:45.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.016+0000 7f64ed600640 1 -- 172.21.15.86:0/3872443237 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f64dc04b060 con 0x7f64f00af7b0 2024-04-20T21:24:45.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.017+0000 7f64ed600640 1 --2- 172.21.15.86:0/3872443237 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f64d006c920 0x7f64d006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:45.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.017+0000 7f64ed600640 1 -- 172.21.15.86:0/3872443237 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5882+0+0 (secure 0 0 0) 0x7f64dc0adbe0 con 0x7f64f00af7b0 2024-04-20T21:24:45.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.017+0000 7f64f5a00640 1 --2- 172.21.15.86:0/3872443237 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f64d006c920 0x7f64d006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:45.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.018+0000 7f64f5a00640 1 --2- 172.21.15.86:0/3872443237 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f64d006c920 0x7f64d006ede0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f64e000a750 tx=0x7f64e0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:45.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.021+0000 7f64ed600640 1 -- 172.21.15.86:0/3872443237 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f64dc07bf50 con 0x7f64f00af7b0 2024-04-20T21:24:45.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.221+0000 7f64f6e00640 1 -- 172.21.15.86:0/3872443237 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f64f0099d40 con 0x7f64f00af7b0 2024-04-20T21:24:45.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.225+0000 7f64ed600640 1 -- 172.21.15.86:0/3872443237 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+16941 (secure 0 0 0) 0x7f64dc07b850 con 0x7f64f00af7b0 2024-04-20T21:24:45.224 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:24:45.224 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":74,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:23:35.220547+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:22:43.125767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-20T21:22:44.410549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.2d","mappings":[{"from":6,"to":0}]},{"pgid":"11.2e","mappings":[{"from":6,"to":0}]},{"pgid":"11.33","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6837/202143875":"2024-04-21T21:23:12.079572+0000","172.21.15.86:6835/202143875":"2024-04-21T21:23:12.079572+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:24:45.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.227+0000 7f64f6e00640 1 -- 172.21.15.86:0/3872443237 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f64d006c920 msgr2=0x7f64d006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:45.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.227+0000 7f64f6e00640 1 --2- 172.21.15.86:0/3872443237 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f64d006c920 0x7f64d006ede0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f64e000a750 tx=0x7f64e0008040 comp rx=0 tx=0).stop 2024-04-20T21:24:45.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.227+0000 7f64f6e00640 1 -- 172.21.15.86:0/3872443237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64f00af7b0 msgr2=0x7f64f0099800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:45.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.227+0000 7f64f6e00640 1 --2- 172.21.15.86:0/3872443237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64f00af7b0 0x7f64f0099800 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f64dc000c00 tx=0x7f64dc0046c0 comp rx=0 tx=0).stop 2024-04-20T21:24:45.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.228+0000 7f64f6e00640 1 -- 172.21.15.86:0/3872443237 shutdown_connections 2024-04-20T21:24:45.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.228+0000 7f64f6e00640 1 --2- 172.21.15.86:0/3872443237 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f64d006c920 0x7f64d006ede0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:45.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.228+0000 7f64f6e00640 1 --2- 172.21.15.86:0/3872443237 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64f015c010 0x7f64f015d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:45.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.228+0000 7f64f6e00640 1 --2- 172.21.15.86:0/3872443237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64f00af7b0 0x7f64f0099800 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:45.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.228+0000 7f64f6e00640 1 --2- 172.21.15.86:0/3872443237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64f006f4c0 0x7f64f01f1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:45.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.228+0000 7f64f6e00640 1 -- 172.21.15.86:0/3872443237 >> 172.21.15.86:0/3872443237 conn(0x7f64f00d0170 msgr2=0x7f64f00b06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:45.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.228+0000 7f64f6e00640 1 -- 172.21.15.86:0/3872443237 shutdown_connections 2024-04-20T21:24:45.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.228+0000 7f64f6e00640 1 -- 172.21.15.86:0/3872443237 wait complete. 2024-04-20T21:24:45.246 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-20T21:24:45.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.447+0000 7fdba3200640 1 Processor -- start 2024-04-20T21:24:45.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.447+0000 7fdba3200640 1 -- start start 2024-04-20T21:24:45.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.448+0000 7fdba3200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb9c0af870 0x7fdb9c071b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:45.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.448+0000 7fdba3200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdb9c0d07c0 0x7fdb9c0720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:45.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.448+0000 7fdba3200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb9c0ce1c0 con 0x7fdb9c073540 2024-04-20T21:24:45.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.448+0000 7fdba3200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fdb9c0ce4a0 con 0x7fdb9c0d07c0 2024-04-20T21:24:45.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.448+0000 7fdba3200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdb9c0ce780 con 0x7fdb9c0af870 2024-04-20T21:24:45.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.448+0000 7fdba1e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fdb9c073540 0x7fdb9c0d00b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:58052/0 (socket says 172.21.15.86:58052) 2024-04-20T21:24:45.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.449+0000 7fdba1e00640 1 -- 172.21.15.86:0/464126869 learned_addr learned my addr 172.21.15.86:0/464126869 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:24:45.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.449+0000 7fdba2800640 1 --2- 172.21.15.86:0/464126869 >> v2:172.21.15.144:3300/0 conn(0x7fdb9c0d07c0 0x7fdb9c0720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:45.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.449+0000 7fdba1400640 1 --2- 172.21.15.86:0/464126869 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb9c0af870 0x7fdb9c071b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:45.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.449+0000 7fdba0a00640 1 -- 172.21.15.86:0/464126869 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1034527793 0 0) 0x7fdb9c0ce1c0 con 0x7fdb9c073540 2024-04-20T21:24:45.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.449+0000 7fdba0a00640 1 -- 172.21.15.86:0/464126869 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb84003520 con 0x7fdb9c073540 2024-04-20T21:24:45.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.449+0000 7fdba0a00640 1 -- 172.21.15.86:0/464126869 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2491997128 0 0) 0x7fdb84003520 con 0x7fdb9c073540 2024-04-20T21:24:45.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.449+0000 7fdba0a00640 1 -- 172.21.15.86:0/464126869 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb9c0af870 msgr2=0x7fdb9c071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:45.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.449+0000 7fdba0a00640 1 --2- 172.21.15.86:0/464126869 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb9c0af870 0x7fdb9c071b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:45.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.449+0000 7fdba0a00640 1 -- 172.21.15.86:0/464126869 >> v2:172.21.15.144:3300/0 conn(0x7fdb9c0d07c0 msgr2=0x7fdb9c0720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:45.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.449+0000 7fdba0a00640 1 --2- 172.21.15.86:0/464126869 >> v2:172.21.15.144:3300/0 conn(0x7fdb9c0d07c0 0x7fdb9c0720b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:45.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.449+0000 7fdba0a00640 1 -- 172.21.15.86:0/464126869 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb9c0cea60 con 0x7fdb9c073540 2024-04-20T21:24:45.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.449+0000 7fdba0a00640 1 -- 172.21.15.86:0/464126869 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fdb90001670 con 0x7fdb9c073540 2024-04-20T21:24:45.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.449+0000 7fdba0a00640 1 -- 172.21.15.86:0/464126869 >> v1:172.21.15.86:6789/0 conn(0x7fdb9c073540 legacy=0x7fdb9c0d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:45.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.449+0000 7fdba0a00640 1 --2- 172.21.15.86:0/464126869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb840039b0 0x7fdb84003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:45.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.450+0000 7fdba0a00640 1 --2- 172.21.15.86:0/464126869 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb84004800 0x7fdb84004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:45.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.450+0000 7fdba0a00640 1 --2- 172.21.15.86:0/464126869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb84005180 0x7fdb8400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:45.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.450+0000 7fdba0a00640 1 -- 172.21.15.86:0/464126869 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c0ce4a0 con 0x7fdb84005180 2024-04-20T21:24:45.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.450+0000 7fdba0a00640 1 -- 172.21.15.86:0/464126869 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c0ce780 con 0x7fdb840039b0 2024-04-20T21:24:45.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.450+0000 7fdba0a00640 1 -- 172.21.15.86:0/464126869 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdb84004600 con 0x7fdb84004800 2024-04-20T21:24:45.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.450+0000 7fdba2800640 1 --2- 172.21.15.86:0/464126869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb84005180 0x7fdb8400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:45.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.450+0000 7fdba1400640 1 --2- 172.21.15.86:0/464126869 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb84004800 0x7fdb84004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:45.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.450+0000 7fdba2800640 1 -- 172.21.15.86:0/464126869 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb84004800 msgr2=0x7fdb84004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:45.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.450+0000 7fdba2800640 1 --2- 172.21.15.86:0/464126869 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb84004800 0x7fdb84004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:45.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.450+0000 7fdba2800640 1 -- 172.21.15.86:0/464126869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb840039b0 msgr2=0x7fdb84003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:45.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.450+0000 7fdba2800640 1 --2- 172.21.15.86:0/464126869 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb840039b0 0x7fdb84003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:45.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.450+0000 7fdba2800640 1 -- 172.21.15.86:0/464126869 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb84003520 con 0x7fdb84005180 2024-04-20T21:24:45.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.451+0000 7fdba2800640 1 --2- 172.21.15.86:0/464126869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb84005180 0x7fdb8400b9c0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fdb98009fc0 tx=0x7fdb84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67f8ed6f6f7663a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:45.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.452+0000 7fdba0a00640 1 -- 172.21.15.86:0/464126869 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb98039070 con 0x7fdb84005180 2024-04-20T21:24:45.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.452+0000 7fdba0a00640 1 -- 172.21.15.86:0/464126869 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb9802cc60 con 0x7fdb84005180 2024-04-20T21:24:45.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.452+0000 7fdba0a00640 1 -- 172.21.15.86:0/464126869 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb98009c10 con 0x7fdb84005180 2024-04-20T21:24:45.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.452+0000 7fdba3200640 1 -- 172.21.15.86:0/464126869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb84005180 msgr2=0x7fdb8400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:45.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.452+0000 7fdba3200640 1 --2- 172.21.15.86:0/464126869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb84005180 0x7fdb8400b9c0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fdb98009fc0 tx=0x7fdb84005830 comp rx=0 tx=0).stop 2024-04-20T21:24:45.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.452+0000 7fdba2800640 1 -- 172.21.15.86:0/464126869 reap_dead start 2024-04-20T21:24:45.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.452+0000 7fdba3200640 1 -- 172.21.15.86:0/464126869 shutdown_connections 2024-04-20T21:24:45.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.452+0000 7fdba3200640 1 -- 172.21.15.86:0/464126869 >> 172.21.15.86:0/464126869 conn(0x7fdb9c06bb60 msgr2=0x7fdb9c06bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:45.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.452+0000 7fdba3200640 1 -- 172.21.15.86:0/464126869 shutdown_connections 2024-04-20T21:24:45.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.452+0000 7fdba3200640 1 -- 172.21.15.86:0/464126869 wait complete. 2024-04-20T21:24:45.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.455+0000 7fdba3200640 1 Processor -- start 2024-04-20T21:24:45.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.455+0000 7fdba3200640 1 -- start start 2024-04-20T21:24:45.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.456+0000 7fdba3200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb9c09da70 0x7fdb9c0cc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:45.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.456+0000 7fdba3200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb9c09de50 0x7fdb9c0c71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:45.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.456+0000 7fdba3200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb9c06dea0 0x7fdb9c0c7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:45.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.456+0000 7fdba3200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c0cfa00 con 0x7fdb9c09da70 2024-04-20T21:24:45.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.456+0000 7fdba3200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c073920 con 0x7fdb9c09de50 2024-04-20T21:24:45.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.456+0000 7fdba3200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdb9c0cfb80 con 0x7fdb9c06dea0 2024-04-20T21:24:45.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.456+0000 7fdba1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb9c09da70 0x7fdb9c0cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:45.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.456+0000 7fdba1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb9c09da70 0x7fdb9c0cc230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51726/0 (socket says 172.21.15.86:51726) 2024-04-20T21:24:45.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.456+0000 7fdba1e00640 1 -- 172.21.15.86:0/1218155360 learned_addr learned my addr 172.21.15.86:0/1218155360 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:45.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.457+0000 7fdba1400640 1 --2- 172.21.15.86:0/1218155360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb9c09de50 0x7fdb9c0c71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:45.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.457+0000 7fdba2800640 1 --2- 172.21.15.86:0/1218155360 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb9c06dea0 0x7fdb9c0c7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:45.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.457+0000 7fdba1e00640 1 -- 172.21.15.86:0/1218155360 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb9c06dea0 msgr2=0x7fdb9c0c7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:45.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.457+0000 7fdba1e00640 1 --2- 172.21.15.86:0/1218155360 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb9c06dea0 0x7fdb9c0c7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:45.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.457+0000 7fdba1e00640 1 -- 172.21.15.86:0/1218155360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb9c09de50 msgr2=0x7fdb9c0c71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:45.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.457+0000 7fdba1e00640 1 --2- 172.21.15.86:0/1218155360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb9c09de50 0x7fdb9c0c71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:45.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.457+0000 7fdba1e00640 1 -- 172.21.15.86:0/1218155360 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb9c158a20 con 0x7fdb9c09da70 2024-04-20T21:24:45.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.457+0000 7fdba1400640 1 --2- 172.21.15.86:0/1218155360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb9c09de50 0x7fdb9c0c71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:24:45.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.457+0000 7fdba1e00640 1 --2- 172.21.15.86:0/1218155360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb9c09da70 0x7fdb9c0cc230 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fdb9000cad0 tx=0x7fdb9000ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:45.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.457+0000 7fdb96000640 1 -- 172.21.15.86:0/1218155360 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb90018070 con 0x7fdb9c09da70 2024-04-20T21:24:45.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.457+0000 7fdb96000640 1 -- 172.21.15.86:0/1218155360 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb90004cb0 con 0x7fdb9c09da70 2024-04-20T21:24:45.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.457+0000 7fdba3200640 1 -- 172.21.15.86:0/1218155360 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb9c0c7f70 con 0x7fdb9c09da70 2024-04-20T21:24:45.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.457+0000 7fdb96000640 1 -- 172.21.15.86:0/1218155360 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb90004e60 con 0x7fdb9c09da70 2024-04-20T21:24:45.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.458+0000 7fdba3200640 1 -- 172.21.15.86:0/1218155360 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb9c0c8250 con 0x7fdb9c09da70 2024-04-20T21:24:45.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.459+0000 7fdba3200640 1 -- 172.21.15.86:0/1218155360 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb9c0ccab0 con 0x7fdb9c09da70 2024-04-20T21:24:45.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.460+0000 7fdb96000640 1 -- 172.21.15.86:0/1218155360 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdb9001b070 con 0x7fdb9c09da70 2024-04-20T21:24:45.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.461+0000 7fdb96000640 1 --2- 172.21.15.86:0/1218155360 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdb8006c860 0x7fdb8006ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:45.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.461+0000 7fdb96000640 1 -- 172.21.15.86:0/1218155360 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5882+0+0 (secure 0 0 0) 0x7fdb9008b250 con 0x7fdb9c09da70 2024-04-20T21:24:45.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.461+0000 7fdba1400640 1 --2- 172.21.15.86:0/1218155360 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdb8006c860 0x7fdb8006ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:45.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.462+0000 7fdba1400640 1 --2- 172.21.15.86:0/1218155360 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdb8006c860 0x7fdb8006ed20 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c06c470 tx=0x7fdb8c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:45.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.465+0000 7fdb96000640 1 -- 172.21.15.86:0/1218155360 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fdb90059600 con 0x7fdb9c09da70 2024-04-20T21:24:45.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.705+0000 7fdba3200640 1 -- 172.21.15.86:0/1218155360 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fdb9c04fcf0 con 0x7fdb9c09da70 2024-04-20T21:24:45.705 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-04-20T21:24:45.706+0000 7f5bb9c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T21:24:45.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.709+0000 7fdb96000640 1 -- 172.21.15.86:0/1218155360 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v51) ==== 161+0+0 (secure 0 0 0) 0x7fdb90058f00 con 0x7fdb9c09da70 2024-04-20T21:24:45.708 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-20T21:24:45.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.713+0000 7fdba3200640 1 -- 172.21.15.86:0/1218155360 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdb8006c860 msgr2=0x7fdb8006ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:45.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.713+0000 7fdba3200640 1 --2- 172.21.15.86:0/1218155360 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdb8006c860 0x7fdb8006ed20 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c06c470 tx=0x7fdb8c008040 comp rx=0 tx=0).stop 2024-04-20T21:24:45.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.713+0000 7fdba3200640 1 -- 172.21.15.86:0/1218155360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb9c09da70 msgr2=0x7fdb9c0cc230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:45.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.713+0000 7fdba3200640 1 --2- 172.21.15.86:0/1218155360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb9c09da70 0x7fdb9c0cc230 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fdb9000cad0 tx=0x7fdb9000ced0 comp rx=0 tx=0).stop 2024-04-20T21:24:45.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.713+0000 7fdba3200640 1 -- 172.21.15.86:0/1218155360 shutdown_connections 2024-04-20T21:24:45.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.713+0000 7fdba3200640 1 --2- 172.21.15.86:0/1218155360 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdb8006c860 0x7fdb8006ed20 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:45.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.713+0000 7fdba3200640 1 --2- 172.21.15.86:0/1218155360 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb9c06dea0 0x7fdb9c0c7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:45.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.713+0000 7fdba3200640 1 --2- 172.21.15.86:0/1218155360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb9c09de50 0x7fdb9c0c71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:45.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.713+0000 7fdba3200640 1 --2- 172.21.15.86:0/1218155360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb9c09da70 0x7fdb9c0cc230 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:45.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.713+0000 7fdba3200640 1 -- 172.21.15.86:0/1218155360 >> 172.21.15.86:0/1218155360 conn(0x7fdb9c06bb60 msgr2=0x7fdb9c0caf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:45.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.713+0000 7fdba3200640 1 -- 172.21.15.86:0/1218155360 shutdown_connections 2024-04-20T21:24:45.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.713+0000 7fdba3200640 1 -- 172.21.15.86:0/1218155360 wait complete. 2024-04-20T21:24:45.727 INFO:tasks.ceph.mds.a.smithi086.stderr: -4729> 2024-04-20T21:23:18.102+0000 7fe375e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T21:24:45.739 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-20T21:24:45.782 INFO:tasks.ceph.mds.a.smithi086.stdout:starting mds.a at 2024-04-20T21:24:45.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.917+0000 7fece3200640 1 Processor -- start 2024-04-20T21:24:45.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.917+0000 7fece3200640 1 -- start start 2024-04-20T21:24:45.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.918+0000 7fece3200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fece40cfc70 0x7fece40cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:45.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.918+0000 7fece3200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fece406f5c0 0x7fece4071ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:45.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.918+0000 7fece3200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fece415bcd0 con 0x7fece40cdbe0 2024-04-20T21:24:45.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.919+0000 7fece3200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fece415c8e0 con 0x7fece40cfc70 2024-04-20T21:24:45.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.919+0000 7fece3200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fece415bfb0 con 0x7fece406f5c0 2024-04-20T21:24:45.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.919+0000 7fece2800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fece406f5c0 0x7fece4071ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:45.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.919+0000 7fece2800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fece406f5c0 0x7fece4071ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:58572/0 (socket says 172.21.15.86:58572) 2024-04-20T21:24:45.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.919+0000 7fece2800640 1 -- 172.21.15.86:0/1356293099 learned_addr learned my addr 172.21.15.86:0/1356293099 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:45.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.920+0000 7fece1e00640 1 --2- 172.21.15.86:0/1356293099 >> v2:172.21.15.144:3300/0 conn(0x7fece40cfc70 0x7fece40cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:45.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.920+0000 7fece2800640 1 -- 172.21.15.86:0/1356293099 >> v2:172.21.15.144:3300/0 conn(0x7fece40cfc70 msgr2=0x7fece40cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:45.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.920+0000 7fece2800640 1 --2- 172.21.15.86:0/1356293099 >> v2:172.21.15.144:3300/0 conn(0x7fece40cfc70 0x7fece40cd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:45.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.920+0000 7fece0a00640 1 -- 172.21.15.86:0/1356293099 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1791570705 0 0) 0x7fece415bcd0 con 0x7fece40cdbe0 2024-04-20T21:24:45.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.920+0000 7fece2800640 1 -- 172.21.15.86:0/1356293099 >> v1:172.21.15.144:6789/0 conn(0x7fece40cdbe0 legacy=0x7fece406eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:45.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.920+0000 7fece2800640 1 -- 172.21.15.86:0/1356293099 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fece415c290 con 0x7fece406f5c0 2024-04-20T21:24:45.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.922+0000 7fece2800640 1 --2- 172.21.15.86:0/1356293099 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fece406f5c0 0x7fece4071ef0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fecdc009e50 tx=0x7fecdc02bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6860ddfc3d99d227 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:45.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.922+0000 7fece0a00640 1 -- 172.21.15.86:0/1356293099 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecdc02cb00 con 0x7fece406f5c0 2024-04-20T21:24:45.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.922+0000 7fece0a00640 1 -- 172.21.15.86:0/1356293099 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecdc02cca0 con 0x7fece406f5c0 2024-04-20T21:24:45.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.922+0000 7fece0a00640 1 -- 172.21.15.86:0/1356293099 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecdc034610 con 0x7fece406f5c0 2024-04-20T21:24:45.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.923+0000 7fece3200640 1 -- 172.21.15.86:0/1356293099 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fece406f5c0 msgr2=0x7fece4071ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:45.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.923+0000 7fece3200640 1 --2- 172.21.15.86:0/1356293099 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fece406f5c0 0x7fece4071ef0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fecdc009e50 tx=0x7fecdc02bf60 comp rx=0 tx=0).stop 2024-04-20T21:24:45.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.923+0000 7fece3200640 1 -- 172.21.15.86:0/1356293099 shutdown_connections 2024-04-20T21:24:45.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.923+0000 7fece3200640 1 --2- 172.21.15.86:0/1356293099 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fece406f5c0 0x7fece4071ef0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:45.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.923+0000 7fece3200640 1 --2- 172.21.15.86:0/1356293099 >> v2:172.21.15.144:3300/0 conn(0x7fece40cfc70 0x7fece40cd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:45.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.923+0000 7fece3200640 1 -- 172.21.15.86:0/1356293099 >> 172.21.15.86:0/1356293099 conn(0x7fece4067cf0 msgr2=0x7fece406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:45.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.923+0000 7fece3200640 1 -- 172.21.15.86:0/1356293099 shutdown_connections 2024-04-20T21:24:45.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.923+0000 7fece3200640 1 -- 172.21.15.86:0/1356293099 wait complete. 2024-04-20T21:24:45.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.924+0000 7fece3200640 1 Processor -- start 2024-04-20T21:24:45.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.925+0000 7fece3200640 1 -- start start 2024-04-20T21:24:45.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.925+0000 7fece3200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fece406f5c0 0x7fece415a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:45.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.925+0000 7fece3200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fece40cdbe0 0x7fece4072130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:45.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.925+0000 7fece3200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fece40cfc70 0x7fece4072670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:45.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.925+0000 7fece3200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fece40d0050 con 0x7fece40cfc70 2024-04-20T21:24:45.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.925+0000 7fece3200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fece40b3710 con 0x7fece40cdbe0 2024-04-20T21:24:45.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.925+0000 7fece3200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fece40d01d0 con 0x7fece406f5c0 2024-04-20T21:24:45.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.926+0000 7fece2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fece40cfc70 0x7fece4072670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:45.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.926+0000 7fece2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fece40cfc70 0x7fece4072670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42782/0 (socket says 172.21.15.86:42782) 2024-04-20T21:24:45.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.926+0000 7fece2800640 1 -- 172.21.15.86:0/1039191333 learned_addr learned my addr 172.21.15.86:0/1039191333 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:45.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.926+0000 7fece1e00640 1 --2- 172.21.15.86:0/1039191333 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fece406f5c0 0x7fece415a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:45.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.926+0000 7fece2800640 1 -- 172.21.15.86:0/1039191333 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fece406f5c0 msgr2=0x7fece415a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:45.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.926+0000 7fece2800640 1 --2- 172.21.15.86:0/1039191333 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fece406f5c0 0x7fece415a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:45.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.926+0000 7fece2800640 1 -- 172.21.15.86:0/1039191333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fece40cdbe0 msgr2=0x7fece4072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:45.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.926+0000 7fece2800640 1 --2- 172.21.15.86:0/1039191333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fece40cdbe0 0x7fece4072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:45.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.926+0000 7fece2800640 1 -- 172.21.15.86:0/1039191333 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fece41f1fd0 con 0x7fece40cfc70 2024-04-20T21:24:45.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.926+0000 7fece1e00640 1 --2- 172.21.15.86:0/1039191333 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fece406f5c0 0x7fece415a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:24:45.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.927+0000 7fece2800640 1 --2- 172.21.15.86:0/1039191333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fece40cfc70 0x7fece4072670 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fecdc009e50 tx=0x7fecdc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:45.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.927+0000 7fecda000640 1 -- 172.21.15.86:0/1039191333 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecdc043020 con 0x7fece40cfc70 2024-04-20T21:24:45.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.927+0000 7fecda000640 1 -- 172.21.15.86:0/1039191333 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecdc02ce50 con 0x7fece40cfc70 2024-04-20T21:24:45.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.927+0000 7fece3200640 1 -- 172.21.15.86:0/1039191333 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fece41f22b0 con 0x7fece40cfc70 2024-04-20T21:24:45.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.927+0000 7fecda000640 1 -- 172.21.15.86:0/1039191333 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecdc009510 con 0x7fece40cfc70 2024-04-20T21:24:45.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.927+0000 7fece3200640 1 -- 172.21.15.86:0/1039191333 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fece41f2870 con 0x7fece40cfc70 2024-04-20T21:24:45.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.928+0000 7fece3200640 1 -- 172.21.15.86:0/1039191333 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fece404fd90 con 0x7fece40cfc70 2024-04-20T21:24:45.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.929+0000 7fecda000640 1 -- 172.21.15.86:0/1039191333 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fecdc046020 con 0x7fece40cfc70 2024-04-20T21:24:45.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.930+0000 7fecda000640 1 --2- 172.21.15.86:0/1039191333 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fecc406c8f0 0x7fecc406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:45.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.930+0000 7fecda000640 1 -- 172.21.15.86:0/1039191333 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5968+0+0 (secure 0 0 0) 0x7fecdc0ae1c0 con 0x7fece40cfc70 2024-04-20T21:24:45.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.930+0000 7fece1e00640 1 --2- 172.21.15.86:0/1039191333 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fecc406c8f0 0x7fecc406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:45.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.931+0000 7fece1e00640 1 --2- 172.21.15.86:0/1039191333 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fecc406c8f0 0x7fecc406edb0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fecd4009ac0 tx=0x7fecd4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:45.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:45.935+0000 7fecda000640 1 -- 172.21.15.86:0/1039191333 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fecdc04d030 con 0x7fece40cfc70 2024-04-20T21:24:46.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.160+0000 7fece3200640 1 -- 172.21.15.86:0/1039191333 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fece40afbc0 con 0x7fece40cfc70 2024-04-20T21:24:46.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.713+0000 7fecda000640 1 -- 172.21.15.86:0/1039191333 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v52) ==== 104+0+0 (secure 0 0 0) 0x7fecdc03c350 con 0x7fece40cfc70 2024-04-20T21:24:46.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.716+0000 7fece3200640 1 -- 172.21.15.86:0/1039191333 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fecc406c8f0 msgr2=0x7fecc406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:46.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.716+0000 7fece3200640 1 --2- 172.21.15.86:0/1039191333 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fecc406c8f0 0x7fecc406edb0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fecd4009ac0 tx=0x7fecd4008040 comp rx=0 tx=0).stop 2024-04-20T21:24:46.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.716+0000 7fece3200640 1 -- 172.21.15.86:0/1039191333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fece40cfc70 msgr2=0x7fece4072670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:46.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.716+0000 7fece3200640 1 --2- 172.21.15.86:0/1039191333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fece40cfc70 0x7fece4072670 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fecdc009e50 tx=0x7fecdc004a40 comp rx=0 tx=0).stop 2024-04-20T21:24:46.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.717+0000 7fece3200640 1 -- 172.21.15.86:0/1039191333 shutdown_connections 2024-04-20T21:24:46.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.717+0000 7fece3200640 1 --2- 172.21.15.86:0/1039191333 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fecc406c8f0 0x7fecc406edb0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:46.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.717+0000 7fece3200640 1 --2- 172.21.15.86:0/1039191333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fece40cfc70 0x7fece4072670 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:46.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.717+0000 7fece3200640 1 --2- 172.21.15.86:0/1039191333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fece40cdbe0 0x7fece4072130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:46.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.717+0000 7fece3200640 1 --2- 172.21.15.86:0/1039191333 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fece406f5c0 0x7fece415a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:46.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.717+0000 7fece3200640 1 -- 172.21.15.86:0/1039191333 >> 172.21.15.86:0/1039191333 conn(0x7fece4067cf0 msgr2=0x7fece40b4730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:46.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.717+0000 7fece3200640 1 -- 172.21.15.86:0/1039191333 shutdown_connections 2024-04-20T21:24:46.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.717+0000 7fece3200640 1 -- 172.21.15.86:0/1039191333 wait complete. 2024-04-20T21:24:46.737 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-20T21:24:46.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.989+0000 7f4e35a00640 1 Processor -- start 2024-04-20T21:24:46.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.989+0000 7f4e35a00640 1 -- start start 2024-04-20T21:24:46.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.991+0000 7f4e35a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e3015ec20 0x7f4e300c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:46.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.991+0000 7f4e35a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4e30163080 0x7f4e300d4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:46.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.991+0000 7f4e35a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e300d3510 con 0x7f4e3015e840 2024-04-20T21:24:46.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.991+0000 7f4e35a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f4e30163510 con 0x7f4e30163080 2024-04-20T21:24:46.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.991+0000 7f4e35a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f4e300cf5b0 con 0x7f4e3015ec20 2024-04-20T21:24:46.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.991+0000 7f4e2ea00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f4e3015e840 0x7f4e300d2d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:50484/0 (socket says 172.21.15.86:50484) 2024-04-20T21:24:46.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.991+0000 7f4e2ea00640 1 -- 172.21.15.86:0/3842144206 learned_addr learned my addr 172.21.15.86:0/3842144206 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:24:46.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.992+0000 7f4e2f400640 1 --2- 172.21.15.86:0/3842144206 >> v2:172.21.15.144:3300/0 conn(0x7f4e30163080 0x7f4e300d4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:46.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.992+0000 7f4e27e00640 1 --2- 172.21.15.86:0/3842144206 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e3015ec20 0x7f4e300c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:46.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.992+0000 7f4e2e000640 1 -- 172.21.15.86:0/3842144206 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 868378890 0 0) 0x7f4e300d3510 con 0x7f4e3015e840 2024-04-20T21:24:46.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.992+0000 7f4e2e000640 1 -- 172.21.15.86:0/3842144206 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e14003520 con 0x7f4e3015e840 2024-04-20T21:24:46.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.992+0000 7f4e2e000640 1 -- 172.21.15.86:0/3842144206 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1205885443 0 0) 0x7f4e14003520 con 0x7f4e3015e840 2024-04-20T21:24:46.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.992+0000 7f4e2e000640 1 -- 172.21.15.86:0/3842144206 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e3015ec20 msgr2=0x7f4e300c82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:46.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.992+0000 7f4e2e000640 1 --2- 172.21.15.86:0/3842144206 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e3015ec20 0x7f4e300c82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:46.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.992+0000 7f4e2e000640 1 -- 172.21.15.86:0/3842144206 >> v2:172.21.15.144:3300/0 conn(0x7f4e30163080 msgr2=0x7f4e300d4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:46.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.992+0000 7f4e2e000640 1 --2- 172.21.15.86:0/3842144206 >> v2:172.21.15.144:3300/0 conn(0x7f4e30163080 0x7f4e300d4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:46.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.992+0000 7f4e2e000640 1 -- 172.21.15.86:0/3842144206 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e300cf890 con 0x7f4e3015e840 2024-04-20T21:24:46.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.992+0000 7f4e2e000640 1 -- 172.21.15.86:0/3842144206 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f4e20001670 con 0x7f4e3015e840 2024-04-20T21:24:46.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.992+0000 7f4e2e000640 1 -- 172.21.15.86:0/3842144206 >> v1:172.21.15.86:6789/0 conn(0x7f4e3015e840 legacy=0x7f4e300d2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:46.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.992+0000 7f4e2e000640 1 --2- 172.21.15.86:0/3842144206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e140039b0 0x7f4e14003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:46.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.992+0000 7f4e2e000640 1 --2- 172.21.15.86:0/3842144206 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e3015ec20 0x7f4e14004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:46.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.992+0000 7f4e2e000640 1 --2- 172.21.15.86:0/3842144206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e14004d40 0x7f4e140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:46.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.992+0000 7f4e2e000640 1 -- 172.21.15.86:0/3842144206 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4e30163510 con 0x7f4e14004d40 2024-04-20T21:24:46.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.992+0000 7f4e2e000640 1 -- 172.21.15.86:0/3842144206 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4e300cf5b0 con 0x7f4e140039b0 2024-04-20T21:24:46.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.992+0000 7f4e2e000640 1 -- 172.21.15.86:0/3842144206 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f4e1400b9c0 con 0x7f4e3015ec20 2024-04-20T21:24:46.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.993+0000 7f4e2f400640 1 --2- 172.21.15.86:0/3842144206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e14004d40 0x7f4e140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:46.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.993+0000 7f4e2ea00640 1 --2- 172.21.15.86:0/3842144206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e140039b0 0x7f4e14003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:46.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.993+0000 7f4e27e00640 1 --2- 172.21.15.86:0/3842144206 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e3015ec20 0x7f4e14004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:46.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.993+0000 7f4e2f400640 1 -- 172.21.15.86:0/3842144206 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e3015ec20 msgr2=0x7f4e14004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:46.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.993+0000 7f4e2f400640 1 --2- 172.21.15.86:0/3842144206 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e3015ec20 0x7f4e14004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:46.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.993+0000 7f4e2f400640 1 -- 172.21.15.86:0/3842144206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e140039b0 msgr2=0x7f4e14003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:46.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.993+0000 7f4e2f400640 1 --2- 172.21.15.86:0/3842144206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e140039b0 0x7f4e14003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:46.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.993+0000 7f4e2f400640 1 -- 172.21.15.86:0/3842144206 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e14003520 con 0x7f4e14004d40 2024-04-20T21:24:46.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.994+0000 7f4e2f400640 1 --2- 172.21.15.86:0/3842144206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e14004d40 0x7f4e140051c0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f4e28009fc0 tx=0x7f4e14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a69e463f27bdfb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:46.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.994+0000 7f4e2e000640 1 -- 172.21.15.86:0/3842144206 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e28039070 con 0x7f4e14004d40 2024-04-20T21:24:46.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.994+0000 7f4e2e000640 1 -- 172.21.15.86:0/3842144206 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e2802cc60 con 0x7f4e14004d40 2024-04-20T21:24:46.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.994+0000 7f4e2e000640 1 -- 172.21.15.86:0/3842144206 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e28009c10 con 0x7f4e14004d40 2024-04-20T21:24:46.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.995+0000 7f4e35a00640 1 -- 172.21.15.86:0/3842144206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e14004d40 msgr2=0x7f4e140051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:46.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.995+0000 7f4e35a00640 1 --2- 172.21.15.86:0/3842144206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e14004d40 0x7f4e140051c0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f4e28009fc0 tx=0x7f4e14005830 comp rx=0 tx=0).stop 2024-04-20T21:24:46.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.995+0000 7f4e2f400640 1 -- 172.21.15.86:0/3842144206 reap_dead start 2024-04-20T21:24:46.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.995+0000 7f4e35a00640 1 -- 172.21.15.86:0/3842144206 shutdown_connections 2024-04-20T21:24:46.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.995+0000 7f4e35a00640 1 -- 172.21.15.86:0/3842144206 >> 172.21.15.86:0/3842144206 conn(0x7f4e3009da40 msgr2=0x7f4e3008bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:46.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.995+0000 7f4e35a00640 1 -- 172.21.15.86:0/3842144206 shutdown_connections 2024-04-20T21:24:46.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.995+0000 7f4e35a00640 1 -- 172.21.15.86:0/3842144206 wait complete. 2024-04-20T21:24:46.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.997+0000 7f4e35a00640 1 Processor -- start 2024-04-20T21:24:46.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.997+0000 7f4e35a00640 1 -- start start 2024-04-20T21:24:46.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.998+0000 7f4e35a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e14004d40 0x7f4e300b9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:46.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.998+0000 7f4e35a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e3015bb00 0x7f4e300cdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:46.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.998+0000 7f4e35a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e300ce500 0x7f4e30160670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:46.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.998+0000 7f4e35a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4e300c82a0 con 0x7f4e300ce500 2024-04-20T21:24:46.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.998+0000 7f4e35a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4e300cf260 con 0x7f4e3015bb00 2024-04-20T21:24:46.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.998+0000 7f4e35a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f4e300c8420 con 0x7f4e14004d40 2024-04-20T21:24:46.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.998+0000 7f4e2f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e300ce500 0x7f4e30160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:46.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.998+0000 7f4e2f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e300ce500 0x7f4e30160670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42788/0 (socket says 172.21.15.86:42788) 2024-04-20T21:24:46.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.998+0000 7f4e2f400640 1 -- 172.21.15.86:0/1815733091 learned_addr learned my addr 172.21.15.86:0/1815733091 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:46.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.999+0000 7f4e2f400640 1 -- 172.21.15.86:0/1815733091 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e14004d40 msgr2=0x7f4e300b9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:47.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.999+0000 7f4e27e00640 1 --2- 172.21.15.86:0/1815733091 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e3015bb00 0x7f4e300cdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:47.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.999+0000 7f4e2f400640 1 --2- 172.21.15.86:0/1815733091 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e14004d40 0x7f4e300b9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:47.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.999+0000 7f4e2f400640 1 -- 172.21.15.86:0/1815733091 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e3015bb00 msgr2=0x7f4e300cdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:47.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.999+0000 7f4e2f400640 1 --2- 172.21.15.86:0/1815733091 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e3015bb00 0x7f4e300cdfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:47.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.999+0000 7f4e2f400640 1 -- 172.21.15.86:0/1815733091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e301f4a60 con 0x7f4e300ce500 2024-04-20T21:24:47.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.999+0000 7f4e2f400640 1 --2- 172.21.15.86:0/1815733091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e300ce500 0x7f4e30160670 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f4e2800e230 tx=0x7f4e28009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:47.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.999+0000 7f4e26a00640 1 -- 172.21.15.86:0/1815733091 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e28039030 con 0x7f4e300ce500 2024-04-20T21:24:47.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.999+0000 7f4e26a00640 1 -- 172.21.15.86:0/1815733091 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e2803a040 con 0x7f4e300ce500 2024-04-20T21:24:47.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.999+0000 7f4e35a00640 1 -- 172.21.15.86:0/1815733091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e301f4d40 con 0x7f4e300ce500 2024-04-20T21:24:47.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:46.999+0000 7f4e26a00640 1 -- 172.21.15.86:0/1815733091 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e28003990 con 0x7f4e300ce500 2024-04-20T21:24:47.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.000+0000 7f4e35a00640 1 -- 172.21.15.86:0/1815733091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e301f5300 con 0x7f4e300ce500 2024-04-20T21:24:47.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.001+0000 7f4e35a00640 1 -- 172.21.15.86:0/1815733091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e300689e0 con 0x7f4e300ce500 2024-04-20T21:24:47.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.002+0000 7f4e26a00640 1 -- 172.21.15.86:0/1815733091 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4e28045030 con 0x7f4e300ce500 2024-04-20T21:24:47.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.003+0000 7f4e26a00640 1 --2- 172.21.15.86:0/1815733091 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4e0c06c8f0 0x7f4e0c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:47.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.003+0000 7f4e26a00640 1 -- 172.21.15.86:0/1815733091 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5968+0+0 (secure 0 0 0) 0x7f4e280adb80 con 0x7f4e300ce500 2024-04-20T21:24:47.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.003+0000 7f4e2ea00640 1 --2- 172.21.15.86:0/1815733091 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4e0c06c8f0 0x7f4e0c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:47.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.004+0000 7f4e2ea00640 1 --2- 172.21.15.86:0/1815733091 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4e0c06c8f0 0x7f4e0c06edb0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f4e20001eb0 tx=0x7f4e20009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:47.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.007+0000 7f4e26a00640 1 -- 172.21.15.86:0/1815733091 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4e2804c020 con 0x7f4e300ce500 2024-04-20T21:24:47.019 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:24:47.024 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:24:47.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.209+0000 7f4e35a00640 1 -- 172.21.15.86:0/1815733091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4e300cff90 con 0x7f4e300ce500 2024-04-20T21:24:47.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.716+0000 7f4e26a00640 1 -- 172.21.15.86:0/1815733091 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v76) ==== 181+0+0 (secure 0 0 0) 0x7f4e2807bec0 con 0x7f4e300ce500 2024-04-20T21:24:47.716 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' removed 2024-04-20T21:24:47.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.721+0000 7f4e35a00640 1 -- 172.21.15.86:0/1815733091 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4e0c06c8f0 msgr2=0x7f4e0c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:47.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.721+0000 7f4e35a00640 1 --2- 172.21.15.86:0/1815733091 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4e0c06c8f0 0x7f4e0c06edb0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f4e20001eb0 tx=0x7f4e20009040 comp rx=0 tx=0).stop 2024-04-20T21:24:47.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.721+0000 7f4e35a00640 1 -- 172.21.15.86:0/1815733091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e300ce500 msgr2=0x7f4e30160670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:47.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.721+0000 7f4e35a00640 1 --2- 172.21.15.86:0/1815733091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e300ce500 0x7f4e30160670 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f4e2800e230 tx=0x7f4e28009750 comp rx=0 tx=0).stop 2024-04-20T21:24:47.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.721+0000 7f4e35a00640 1 -- 172.21.15.86:0/1815733091 shutdown_connections 2024-04-20T21:24:47.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.721+0000 7f4e35a00640 1 --2- 172.21.15.86:0/1815733091 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4e0c06c8f0 0x7f4e0c06edb0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:47.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.721+0000 7f4e35a00640 1 --2- 172.21.15.86:0/1815733091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e300ce500 0x7f4e30160670 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:47.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.721+0000 7f4e35a00640 1 --2- 172.21.15.86:0/1815733091 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e3015bb00 0x7f4e300cdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:47.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.721+0000 7f4e35a00640 1 --2- 172.21.15.86:0/1815733091 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4e14004d40 0x7f4e300b9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:47.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.721+0000 7f4e35a00640 1 -- 172.21.15.86:0/1815733091 >> 172.21.15.86:0/1815733091 conn(0x7f4e3009da40 msgr2=0x7f4e300d41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:47.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.722+0000 7f4e35a00640 1 -- 172.21.15.86:0/1815733091 shutdown_connections 2024-04-20T21:24:47.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.723+0000 7f4e35a00640 1 -- 172.21.15.86:0/1815733091 wait complete. 2024-04-20T21:24:47.750 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-20T21:24:47.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.932+0000 7f0719800640 1 Processor -- start 2024-04-20T21:24:47.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.932+0000 7f0719800640 1 -- start start 2024-04-20T21:24:47.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.934+0000 7f0719800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f071415fc60 0x7f07140b3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:47.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.934+0000 7f0719800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0714073300 con 0x7f071415b4f0 2024-04-20T21:24:47.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.934+0000 7f0719800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07140735b0 con 0x7f07140b4220 2024-04-20T21:24:47.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.934+0000 7f0719800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f07140b3200 con 0x7f071415fc60 2024-04-20T21:24:47.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.934+0000 7f0713e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f071415b4f0 0x7f0714072bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:50492/0 (socket says 172.21.15.86:50492) 2024-04-20T21:24:47.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.934+0000 7f0713e00640 1 -- 172.21.15.86:0/2000905228 learned_addr learned my addr 172.21.15.86:0/2000905228 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:24:47.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.934+0000 7f0713400640 1 -- 172.21.15.86:0/2000905228 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3939707060 0 0) 0x7f0714073300 con 0x7f071415b4f0 2024-04-20T21:24:47.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.935+0000 7f0713400640 1 -- 172.21.15.86:0/2000905228 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f06f8003520 con 0x7f071415b4f0 2024-04-20T21:24:47.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.935+0000 7f0713400640 1 -- 172.21.15.86:0/2000905228 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1408122081 0 0) 0x7f06f8003520 con 0x7f071415b4f0 2024-04-20T21:24:47.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.935+0000 7f0713400640 1 -- 172.21.15.86:0/2000905228 >> v2:172.21.15.144:3300/0 conn(0x7f071415fc60 msgr2=0x7f07140b3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:47.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.935+0000 7f070be00640 1 --2- 172.21.15.86:0/2000905228 >> v2:172.21.15.144:3300/0 conn(0x7f071415fc60 0x7f07140b3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:47.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.935+0000 7f0713400640 1 --2- 172.21.15.86:0/2000905228 >> v2:172.21.15.144:3300/0 conn(0x7f071415fc60 0x7f07140b3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:47.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.935+0000 7f0713400640 1 -- 172.21.15.86:0/2000905228 >> v1:172.21.15.144:6789/0 conn(0x7f07140b4220 legacy=0x7f07140d0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:47.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.935+0000 7f0713400640 1 -- 172.21.15.86:0/2000905228 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07140b34e0 con 0x7f071415b4f0 2024-04-20T21:24:47.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.935+0000 7f0713400640 1 -- 172.21.15.86:0/2000905228 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f0704001670 con 0x7f071415b4f0 2024-04-20T21:24:47.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.935+0000 7f0713400640 1 -- 172.21.15.86:0/2000905228 >> v1:172.21.15.86:6789/0 conn(0x7f071415b4f0 legacy=0x7f0714072bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:47.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.935+0000 7f0713400640 1 --2- 172.21.15.86:0/2000905228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06f80039b0 0x7f06f8003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:47.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.935+0000 7f0713400640 1 --2- 172.21.15.86:0/2000905228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06f8004800 0x7f06f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:47.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.935+0000 7f0713400640 1 --2- 172.21.15.86:0/2000905228 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06f80052e0 0x7f06f800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:47.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.935+0000 7f0713400640 1 -- 172.21.15.86:0/2000905228 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f07140b3200 con 0x7f06f80039b0 2024-04-20T21:24:47.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.935+0000 7f0713400640 1 -- 172.21.15.86:0/2000905228 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f06f800c1e0 con 0x7f06f8004800 2024-04-20T21:24:47.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.935+0000 7f0713400640 1 -- 172.21.15.86:0/2000905228 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f06f800c3e0 con 0x7f06f80052e0 2024-04-20T21:24:47.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.935+0000 7f0713e00640 1 --2- 172.21.15.86:0/2000905228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06f80039b0 0x7f06f8003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:47.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.936+0000 7f0713e00640 1 -- 172.21.15.86:0/2000905228 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06f80052e0 msgr2=0x7f06f800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:47.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.936+0000 7f0713e00640 1 --2- 172.21.15.86:0/2000905228 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06f80052e0 0x7f06f800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:47.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.936+0000 7f0713e00640 1 -- 172.21.15.86:0/2000905228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06f8004800 msgr2=0x7f06f8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:47.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.936+0000 7f0713e00640 1 --2- 172.21.15.86:0/2000905228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06f8004800 0x7f06f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:47.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.936+0000 7f0713e00640 1 -- 172.21.15.86:0/2000905228 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06f8003520 con 0x7f06f80039b0 2024-04-20T21:24:47.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.936+0000 7f0718e00640 1 -- 172.21.15.86:0/2000905228 reap_dead start 2024-04-20T21:24:47.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.937+0000 7f0713e00640 1 --2- 172.21.15.86:0/2000905228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06f80039b0 0x7f06f8003db0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f070400c210 tx=0x7f06f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f8b500de83db6a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:47.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.937+0000 7f0713400640 1 -- 172.21.15.86:0/2000905228 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0704007d30 con 0x7f06f80039b0 2024-04-20T21:24:47.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.937+0000 7f0713400640 1 -- 172.21.15.86:0/2000905228 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0704030070 con 0x7f06f80039b0 2024-04-20T21:24:47.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.937+0000 7f0713400640 1 -- 172.21.15.86:0/2000905228 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0704005d70 con 0x7f06f80039b0 2024-04-20T21:24:47.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.938+0000 7f0719800640 1 -- 172.21.15.86:0/2000905228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06f80039b0 msgr2=0x7f06f8003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:47.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.938+0000 7f0719800640 1 --2- 172.21.15.86:0/2000905228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06f80039b0 0x7f06f8003db0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f070400c210 tx=0x7f06f8005830 comp rx=0 tx=0).stop 2024-04-20T21:24:47.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.938+0000 7f0719800640 1 -- 172.21.15.86:0/2000905228 shutdown_connections 2024-04-20T21:24:47.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.938+0000 7f0719800640 1 --2- 172.21.15.86:0/2000905228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06f80039b0 0x7f06f8003db0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:47.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.938+0000 7f0719800640 1 -- 172.21.15.86:0/2000905228 >> 172.21.15.86:0/2000905228 conn(0x7f07140b2180 msgr2=0x7f07140b2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:47.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.938+0000 7f0719800640 1 -- 172.21.15.86:0/2000905228 shutdown_connections 2024-04-20T21:24:47.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.938+0000 7f0719800640 1 -- 172.21.15.86:0/2000905228 wait complete. 2024-04-20T21:24:47.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.939+0000 7f0719800640 1 Processor -- start 2024-04-20T21:24:47.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.940+0000 7f0719800640 1 -- start start 2024-04-20T21:24:47.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.940+0000 7f0719800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06f80039b0 0x7f07140ce130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:47.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.940+0000 7f0719800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f071409d620 0x7f0714157b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:47.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.940+0000 7f0719800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f071414e290 0x7f071414e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:47.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.940+0000 7f0719800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f07140b3e60 con 0x7f071414e290 2024-04-20T21:24:47.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.940+0000 7f0719800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f07140b3ce0 con 0x7f071409d620 2024-04-20T21:24:47.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.940+0000 7f0719800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f07140b3fe0 con 0x7f06f80039b0 2024-04-20T21:24:47.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.941+0000 7f0718e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f071414e290 0x7f071414e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:47.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.941+0000 7f0718e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f071414e290 0x7f071414e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42804/0 (socket says 172.21.15.86:42804) 2024-04-20T21:24:47.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.941+0000 7f0718e00640 1 -- 172.21.15.86:0/4141665308 learned_addr learned my addr 172.21.15.86:0/4141665308 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:47.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.941+0000 7f0718e00640 1 -- 172.21.15.86:0/4141665308 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06f80039b0 msgr2=0x7f07140ce130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:47.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.941+0000 7f0718e00640 1 --2- 172.21.15.86:0/4141665308 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06f80039b0 0x7f07140ce130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:47.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.941+0000 7f0718e00640 1 -- 172.21.15.86:0/4141665308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f071409d620 msgr2=0x7f0714157b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:47.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.941+0000 7f0718e00640 1 --2- 172.21.15.86:0/4141665308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f071409d620 0x7f0714157b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:47.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.941+0000 7f0718e00640 1 -- 172.21.15.86:0/4141665308 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f071414ec50 con 0x7f071414e290 2024-04-20T21:24:47.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.941+0000 7f0718e00640 1 --2- 172.21.15.86:0/4141665308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f071414e290 0x7f071414e710 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f07140cca40 tx=0x7f070c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:47.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.942+0000 7f0710c00640 1 -- 172.21.15.86:0/4141665308 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f070c017070 con 0x7f071414e290 2024-04-20T21:24:47.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.942+0000 7f0710c00640 1 -- 172.21.15.86:0/4141665308 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f070c0070d0 con 0x7f071414e290 2024-04-20T21:24:47.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.942+0000 7f0710c00640 1 -- 172.21.15.86:0/4141665308 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f070c0074c0 con 0x7f071414e290 2024-04-20T21:24:47.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.942+0000 7f0719800640 1 -- 172.21.15.86:0/4141665308 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07141f8e40 con 0x7f071414e290 2024-04-20T21:24:47.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.942+0000 7f0719800640 1 -- 172.21.15.86:0/4141665308 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07141f85a0 con 0x7f071414e290 2024-04-20T21:24:47.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.943+0000 7f0719800640 1 -- 172.21.15.86:0/4141665308 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f071404fcf0 con 0x7f071414e290 2024-04-20T21:24:47.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.944+0000 7f0710c00640 1 -- 172.21.15.86:0/4141665308 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f070c01a070 con 0x7f071414e290 2024-04-20T21:24:47.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.945+0000 7f0710c00640 1 --2- 172.21.15.86:0/4141665308 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f06f006c8f0 0x7f06f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:47.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.945+0000 7f0710c00640 1 -- 172.21.15.86:0/4141665308 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5530+0+0 (secure 0 0 0) 0x7f070c08ac50 con 0x7f071414e290 2024-04-20T21:24:47.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.945+0000 7f0713e00640 1 --2- 172.21.15.86:0/4141665308 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f06f006c8f0 0x7f06f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:47.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.946+0000 7f0713e00640 1 --2- 172.21.15.86:0/4141665308 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f06f006c8f0 0x7f06f006edb0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0704034000 tx=0x7f0704007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:47.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:47.950+0000 7f0710c00640 1 -- 172.21.15.86:0/4141665308 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f070c090020 con 0x7f071414e290 2024-04-20T21:24:48.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.153+0000 7f0719800640 1 -- 172.21.15.86:0/4141665308 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f071414cb90 con 0x7f071414e290 2024-04-20T21:24:48.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.737+0000 7f0710c00640 1 -- 172.21.15.86:0/4141665308 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v77) ==== 169+0+0 (secure 0 0 0) 0x7f070c0591c0 con 0x7f071414e290 2024-04-20T21:24:48.736 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' removed 2024-04-20T21:24:48.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.740+0000 7f0719800640 1 -- 172.21.15.86:0/4141665308 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f06f006c8f0 msgr2=0x7f06f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:48.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.740+0000 7f0719800640 1 --2- 172.21.15.86:0/4141665308 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f06f006c8f0 0x7f06f006edb0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0704034000 tx=0x7f0704007000 comp rx=0 tx=0).stop 2024-04-20T21:24:48.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.740+0000 7f0719800640 1 -- 172.21.15.86:0/4141665308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f071414e290 msgr2=0x7f071414e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:48.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.740+0000 7f0719800640 1 --2- 172.21.15.86:0/4141665308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f071414e290 0x7f071414e710 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f07140cca40 tx=0x7f070c00c960 comp rx=0 tx=0).stop 2024-04-20T21:24:48.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.741+0000 7f0719800640 1 -- 172.21.15.86:0/4141665308 shutdown_connections 2024-04-20T21:24:48.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.741+0000 7f0719800640 1 --2- 172.21.15.86:0/4141665308 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f06f006c8f0 0x7f06f006edb0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:48.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.741+0000 7f0719800640 1 --2- 172.21.15.86:0/4141665308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f071414e290 0x7f071414e710 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:48.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.741+0000 7f0719800640 1 --2- 172.21.15.86:0/4141665308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f071409d620 0x7f0714157b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:48.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.741+0000 7f0719800640 1 --2- 172.21.15.86:0/4141665308 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06f80039b0 0x7f07140ce130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:48.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.741+0000 7f0719800640 1 -- 172.21.15.86:0/4141665308 >> 172.21.15.86:0/4141665308 conn(0x7f07140b2180 msgr2=0x7f071414f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:48.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.741+0000 7f0719800640 1 -- 172.21.15.86:0/4141665308 shutdown_connections 2024-04-20T21:24:48.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.741+0000 7f0719800640 1 -- 172.21.15.86:0/4141665308 wait complete. 2024-04-20T21:24:48.765 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-20T21:24:48.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.960+0000 7f265b200640 1 Processor -- start 2024-04-20T21:24:48.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.960+0000 7f265b200640 1 -- start start 2024-04-20T21:24:48.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.961+0000 7f265b200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f265c0cfc70 0x7f265c0cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:48.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.961+0000 7f265b200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f265c15bcd0 con 0x7f265c0cdbe0 2024-04-20T21:24:48.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.961+0000 7f265b200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f265c15bfb0 con 0x7f265c06f5c0 2024-04-20T21:24:48.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.961+0000 7f265b200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f265c15c8e0 con 0x7f265c0cfc70 2024-04-20T21:24:48.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.961+0000 7f2651400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f265c0cdbe0 0x7f265c06eeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:50496/0 (socket says 172.21.15.86:50496) 2024-04-20T21:24:48.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.961+0000 7f2651400640 1 -- 172.21.15.86:0/410596880 learned_addr learned my addr 172.21.15.86:0/410596880 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:24:48.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.962+0000 7f2659e00640 1 --2- 172.21.15.86:0/410596880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f265c0cfc70 0x7f265c0cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:48.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.962+0000 7f2659400640 1 -- 172.21.15.86:0/410596880 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2231770336 0 0) 0x7f265c15bcd0 con 0x7f265c0cdbe0 2024-04-20T21:24:48.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.962+0000 7f2659400640 1 -- 172.21.15.86:0/410596880 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2640005730 con 0x7f265c0cdbe0 2024-04-20T21:24:48.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.962+0000 7f2659400640 1 -- 172.21.15.86:0/410596880 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4141830291 0 0) 0x7f2640005730 con 0x7f265c0cdbe0 2024-04-20T21:24:48.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.962+0000 7f2659400640 1 -- 172.21.15.86:0/410596880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f265c0cfc70 msgr2=0x7f265c0cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:48.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.962+0000 7f2659400640 1 --2- 172.21.15.86:0/410596880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f265c0cfc70 0x7f265c0cd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:48.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.962+0000 7f2659400640 1 -- 172.21.15.86:0/410596880 >> v1:172.21.15.144:6789/0 conn(0x7f265c06f5c0 legacy=0x7f265c071ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:48.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.962+0000 7f2659400640 1 -- 172.21.15.86:0/410596880 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f265c15c290 con 0x7f265c0cdbe0 2024-04-20T21:24:48.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.962+0000 7f2659400640 1 -- 172.21.15.86:0/410596880 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f2644001670 con 0x7f265c0cdbe0 2024-04-20T21:24:48.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.962+0000 7f2659400640 1 -- 172.21.15.86:0/410596880 >> v1:172.21.15.86:6789/0 conn(0x7f265c0cdbe0 legacy=0x7f265c06eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:48.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.962+0000 7f2659400640 1 --2- 172.21.15.86:0/410596880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f265c0cfc70 0x7f2640005c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:48.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.962+0000 7f2659400640 1 --2- 172.21.15.86:0/410596880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26400061e0 0x7f2640006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:48.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.962+0000 7f2659400640 1 --2- 172.21.15.86:0/410596880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2640006f50 0x7f26400073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:48.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.962+0000 7f2659400640 1 -- 172.21.15.86:0/410596880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f265c15c8e0 con 0x7f2640006f50 2024-04-20T21:24:48.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.962+0000 7f2659400640 1 -- 172.21.15.86:0/410596880 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f264000dd30 con 0x7f26400061e0 2024-04-20T21:24:48.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.962+0000 7f2659400640 1 -- 172.21.15.86:0/410596880 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f264000df30 con 0x7f265c0cfc70 2024-04-20T21:24:48.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.963+0000 7f265a800640 1 --2- 172.21.15.86:0/410596880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2640006f50 0x7f26400073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:48.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.963+0000 7f2651400640 1 --2- 172.21.15.86:0/410596880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26400061e0 0x7f2640006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:48.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.963+0000 7f2659e00640 1 --2- 172.21.15.86:0/410596880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f265c0cfc70 0x7f2640005c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:48.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.963+0000 7f265a800640 1 -- 172.21.15.86:0/410596880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f265c0cfc70 msgr2=0x7f2640005c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:48.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.963+0000 7f265a800640 1 --2- 172.21.15.86:0/410596880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f265c0cfc70 0x7f2640005c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:48.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.963+0000 7f265a800640 1 -- 172.21.15.86:0/410596880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26400061e0 msgr2=0x7f2640006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:48.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.963+0000 7f265a800640 1 --2- 172.21.15.86:0/410596880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26400061e0 0x7f2640006a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:48.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.963+0000 7f265a800640 1 -- 172.21.15.86:0/410596880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2640005730 con 0x7f2640006f50 2024-04-20T21:24:48.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.964+0000 7f265a800640 1 --2- 172.21.15.86:0/410596880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2640006f50 0x7f26400073d0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f2654001e50 tx=0x7f2640007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59e4254d3b0ad3b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:48.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.964+0000 7f2659400640 1 -- 172.21.15.86:0/410596880 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2654001a70 con 0x7f2640006f50 2024-04-20T21:24:48.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.964+0000 7f2659400640 1 -- 172.21.15.86:0/410596880 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2654003b70 con 0x7f2640006f50 2024-04-20T21:24:48.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.964+0000 7f2659400640 1 -- 172.21.15.86:0/410596880 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26540050d0 con 0x7f2640006f50 2024-04-20T21:24:48.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.965+0000 7f265b200640 1 -- 172.21.15.86:0/410596880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2640006f50 msgr2=0x7f26400073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:48.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.965+0000 7f265b200640 1 --2- 172.21.15.86:0/410596880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2640006f50 0x7f26400073d0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f2654001e50 tx=0x7f2640007a40 comp rx=0 tx=0).stop 2024-04-20T21:24:48.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.965+0000 7f265a800640 1 -- 172.21.15.86:0/410596880 reap_dead start 2024-04-20T21:24:48.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.965+0000 7f265b200640 1 -- 172.21.15.86:0/410596880 shutdown_connections 2024-04-20T21:24:48.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.965+0000 7f265b200640 1 -- 172.21.15.86:0/410596880 >> 172.21.15.86:0/410596880 conn(0x7f265c067cf0 msgr2=0x7f265c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:48.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.965+0000 7f265b200640 1 -- 172.21.15.86:0/410596880 shutdown_connections 2024-04-20T21:24:48.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.965+0000 7f265b200640 1 -- 172.21.15.86:0/410596880 wait complete. 2024-04-20T21:24:48.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.967+0000 7f265b200640 1 Processor -- start 2024-04-20T21:24:48.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.967+0000 7f265b200640 1 -- start start 2024-04-20T21:24:48.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.968+0000 7f265b200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f265c144e00 0x7f265c1529e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:48.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.968+0000 7f265b200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f265c1f85a0 0x7f265c1f7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:48.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.968+0000 7f265b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f265c1f7e60 0x7f265c153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:48.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.968+0000 7f265b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f265c0d0050 con 0x7f265c1f7e60 2024-04-20T21:24:48.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.968+0000 7f265b200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f265c0b3710 con 0x7f265c144e00 2024-04-20T21:24:48.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.968+0000 7f265b200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f265c0d01d0 con 0x7f265c1f85a0 2024-04-20T21:24:48.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.968+0000 7f265a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f265c1f7e60 0x7f265c153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:48.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.968+0000 7f265a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f265c1f7e60 0x7f265c153350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42834/0 (socket says 172.21.15.86:42834) 2024-04-20T21:24:48.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.968+0000 7f265a800640 1 -- 172.21.15.86:0/105377187 learned_addr learned my addr 172.21.15.86:0/105377187 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:48.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.969+0000 7f2651400640 1 --2- 172.21.15.86:0/105377187 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f265c1f85a0 0x7f265c1f7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:48.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.969+0000 7f2659e00640 1 --2- 172.21.15.86:0/105377187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f265c144e00 0x7f265c1529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:48.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.969+0000 7f265a800640 1 -- 172.21.15.86:0/105377187 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f265c1f85a0 msgr2=0x7f265c1f7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:48.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.969+0000 7f265a800640 1 --2- 172.21.15.86:0/105377187 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f265c1f85a0 0x7f265c1f7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:48.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.969+0000 7f265a800640 1 -- 172.21.15.86:0/105377187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f265c144e00 msgr2=0x7f265c1529e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:48.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.969+0000 7f265a800640 1 --2- 172.21.15.86:0/105377187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f265c144e00 0x7f265c1529e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:48.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.969+0000 7f265a800640 1 -- 172.21.15.86:0/105377187 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f265c155aa0 con 0x7f265c1f7e60 2024-04-20T21:24:48.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.969+0000 7f265a800640 1 --2- 172.21.15.86:0/105377187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f265c1f7e60 0x7f265c153350 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f265402c060 tx=0x7f26540014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:48.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.969+0000 7f2652a00640 1 -- 172.21.15.86:0/105377187 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2654001a70 con 0x7f265c1f7e60 2024-04-20T21:24:48.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.969+0000 7f2652a00640 1 -- 172.21.15.86:0/105377187 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2654003970 con 0x7f265c1f7e60 2024-04-20T21:24:48.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.969+0000 7f265b200640 1 -- 172.21.15.86:0/105377187 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f265c1f4330 con 0x7f265c1f7e60 2024-04-20T21:24:48.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.969+0000 7f2652a00640 1 -- 172.21.15.86:0/105377187 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2654007db0 con 0x7f265c1f7e60 2024-04-20T21:24:48.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.970+0000 7f265b200640 1 -- 172.21.15.86:0/105377187 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f265c1557c0 con 0x7f265c1f7e60 2024-04-20T21:24:48.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.971+0000 7f265b200640 1 -- 172.21.15.86:0/105377187 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f265c04fd90 con 0x7f265c1f7e60 2024-04-20T21:24:48.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.972+0000 7f2652a00640 1 -- 172.21.15.86:0/105377187 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2654007030 con 0x7f265c1f7e60 2024-04-20T21:24:48.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.973+0000 7f2652a00640 1 --2- 172.21.15.86:0/105377187 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f263806c920 0x7f263806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:48.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.973+0000 7f2652a00640 1 -- 172.21.15.86:0/105377187 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5029+0+0 (secure 0 0 0) 0x7f26540acb30 con 0x7f265c1f7e60 2024-04-20T21:24:48.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.973+0000 7f2659e00640 1 --2- 172.21.15.86:0/105377187 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f263806c920 0x7f263806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:48.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.973+0000 7f2659e00640 1 --2- 172.21.15.86:0/105377187 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f263806c920 0x7f263806ede0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f264c009a10 tx=0x7f264c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:48.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:48.977+0000 7f2652a00640 1 -- 172.21.15.86:0/105377187 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2654033020 con 0x7f265c1f7e60 2024-04-20T21:24:49.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.177+0000 7f265b200640 1 -- 172.21.15.86:0/105377187 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f265c153130 con 0x7f265c1f7e60 2024-04-20T21:24:49.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.185+0000 7f2652a00640 1 -- 172.21.15.86:0/105377187 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) ==== 104+0+0 (secure 0 0 0) 0x7f265407b0d0 con 0x7f265c1f7e60 2024-04-20T21:24:49.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.187+0000 7f265b200640 1 -- 172.21.15.86:0/105377187 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f263806c920 msgr2=0x7f263806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:49.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.187+0000 7f265b200640 1 --2- 172.21.15.86:0/105377187 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f263806c920 0x7f263806ede0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f264c009a10 tx=0x7f264c008040 comp rx=0 tx=0).stop 2024-04-20T21:24:49.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.187+0000 7f265b200640 1 -- 172.21.15.86:0/105377187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f265c1f7e60 msgr2=0x7f265c153350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:49.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.187+0000 7f265b200640 1 --2- 172.21.15.86:0/105377187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f265c1f7e60 0x7f265c153350 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f265402c060 tx=0x7f26540014a0 comp rx=0 tx=0).stop 2024-04-20T21:24:49.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.188+0000 7f265b200640 1 -- 172.21.15.86:0/105377187 shutdown_connections 2024-04-20T21:24:49.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.188+0000 7f265b200640 1 --2- 172.21.15.86:0/105377187 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f263806c920 0x7f263806ede0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:49.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.188+0000 7f265b200640 1 --2- 172.21.15.86:0/105377187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f265c1f7e60 0x7f265c153350 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:49.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.188+0000 7f265b200640 1 --2- 172.21.15.86:0/105377187 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f265c1f85a0 0x7f265c1f7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:49.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.188+0000 7f265b200640 1 --2- 172.21.15.86:0/105377187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f265c144e00 0x7f265c1529e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:49.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.188+0000 7f265b200640 1 -- 172.21.15.86:0/105377187 >> 172.21.15.86:0/105377187 conn(0x7f265c067cf0 msgr2=0x7f265c15b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:49.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.188+0000 7f265b200640 1 -- 172.21.15.86:0/105377187 shutdown_connections 2024-04-20T21:24:49.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.188+0000 7f265b200640 1 -- 172.21.15.86:0/105377187 wait complete. 2024-04-20T21:24:49.204 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-04-20T21:24:49.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.405+0000 7f1295a00640 1 Processor -- start 2024-04-20T21:24:49.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.405+0000 7f1295a00640 1 -- start start 2024-04-20T21:24:49.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.406+0000 7f1295a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f129015ac10 0x7f12900b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:49.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.406+0000 7f1295a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f129006dcd0 con 0x7f1290072ca0 2024-04-20T21:24:49.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.406+0000 7f1295a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f129006dfb0 con 0x7f129015f210 2024-04-20T21:24:49.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.406+0000 7f1295a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f129006e8e0 con 0x7f129015ac10 2024-04-20T21:24:49.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.407+0000 7f128f400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f1290072ca0 0x7f12900cea10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:50500/0 (socket says 172.21.15.86:50500) 2024-04-20T21:24:49.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.407+0000 7f128f400640 1 -- 172.21.15.86:0/3690133801 learned_addr learned my addr 172.21.15.86:0/3690133801 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:24:49.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.407+0000 7f128ea00640 1 --2- 172.21.15.86:0/3690133801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f129015ac10 0x7f12900b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:49.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.407+0000 7f128e000640 1 -- 172.21.15.86:0/3690133801 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3769086265 0 0) 0x7f129006dcd0 con 0x7f1290072ca0 2024-04-20T21:24:49.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.407+0000 7f128e000640 1 -- 172.21.15.86:0/3690133801 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1274003440 con 0x7f1290072ca0 2024-04-20T21:24:49.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.408+0000 7f128e000640 1 -- 172.21.15.86:0/3690133801 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1337840961 0 0) 0x7f1274003440 con 0x7f1290072ca0 2024-04-20T21:24:49.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.408+0000 7f128e000640 1 -- 172.21.15.86:0/3690133801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f129015ac10 msgr2=0x7f12900b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:49.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.408+0000 7f128e000640 1 --2- 172.21.15.86:0/3690133801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f129015ac10 0x7f12900b53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:49.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.408+0000 7f128e000640 1 -- 172.21.15.86:0/3690133801 >> v1:172.21.15.144:6789/0 conn(0x7f129015f210 legacy=0x7f1290072590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:49.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.408+0000 7f128e000640 1 -- 172.21.15.86:0/3690133801 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f129006e290 con 0x7f1290072ca0 2024-04-20T21:24:49.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.408+0000 7f128e000640 1 -- 172.21.15.86:0/3690133801 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f1288001670 con 0x7f1290072ca0 2024-04-20T21:24:49.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.408+0000 7f128e000640 1 -- 172.21.15.86:0/3690133801 >> v1:172.21.15.86:6789/0 conn(0x7f1290072ca0 legacy=0x7f12900cea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:49.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.408+0000 7f128e000640 1 --2- 172.21.15.86:0/3690133801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f129015ac10 0x7f1274003ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:49.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.408+0000 7f128e000640 1 --2- 172.21.15.86:0/3690133801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1274004150 0x7f127400a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:49.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.408+0000 7f128e000640 1 --2- 172.21.15.86:0/3690133801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f127400b0e0 0x7f127400d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:49.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.408+0000 7f128e000640 1 -- 172.21.15.86:0/3690133801 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f127400de20 con 0x7f1274004150 2024-04-20T21:24:49.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.408+0000 7f128e000640 1 -- 172.21.15.86:0/3690133801 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f127400dfd0 con 0x7f127400b0e0 2024-04-20T21:24:49.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.408+0000 7f128e000640 1 -- 172.21.15.86:0/3690133801 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f127400e1d0 con 0x7f129015ac10 2024-04-20T21:24:49.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.408+0000 7f1286000640 1 --2- 172.21.15.86:0/3690133801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1274004150 0x7f127400a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:49.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.409+0000 7f128ea00640 1 --2- 172.21.15.86:0/3690133801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f129015ac10 0x7f1274003ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:49.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.409+0000 7f1286000640 1 -- 172.21.15.86:0/3690133801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f129015ac10 msgr2=0x7f1274003ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:49.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.409+0000 7f1286000640 1 --2- 172.21.15.86:0/3690133801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f129015ac10 0x7f1274003ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:49.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.409+0000 7f1286000640 1 -- 172.21.15.86:0/3690133801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f127400b0e0 msgr2=0x7f127400d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:49.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.409+0000 7f1286000640 1 --2- 172.21.15.86:0/3690133801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f127400b0e0 0x7f127400d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:49.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.409+0000 7f1286000640 1 -- 172.21.15.86:0/3690133801 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1274003440 con 0x7f1274004150 2024-04-20T21:24:49.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.410+0000 7f1286000640 1 --2- 172.21.15.86:0/3690133801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1274004150 0x7f127400a8e0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f1278001750 tx=0x7f1274003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e5ac1d2376d2c53 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:49.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.410+0000 7f128e000640 1 -- 172.21.15.86:0/3690133801 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1278001370 con 0x7f1274004150 2024-04-20T21:24:49.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.410+0000 7f128e000640 1 -- 172.21.15.86:0/3690133801 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12780052d0 con 0x7f1274004150 2024-04-20T21:24:49.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.410+0000 7f128e000640 1 -- 172.21.15.86:0/3690133801 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1278003830 con 0x7f1274004150 2024-04-20T21:24:49.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.410+0000 7f1295a00640 1 -- 172.21.15.86:0/3690133801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1274004150 msgr2=0x7f127400a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:49.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.410+0000 7f1295a00640 1 --2- 172.21.15.86:0/3690133801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1274004150 0x7f127400a8e0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f1278001750 tx=0x7f1274003730 comp rx=0 tx=0).stop 2024-04-20T21:24:49.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.411+0000 7f128f400640 1 -- 172.21.15.86:0/3690133801 reap_dead start 2024-04-20T21:24:49.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.411+0000 7f1295a00640 1 -- 172.21.15.86:0/3690133801 shutdown_connections 2024-04-20T21:24:49.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.411+0000 7f1295a00640 1 -- 172.21.15.86:0/3690133801 >> 172.21.15.86:0/3690133801 conn(0x7f12900b2360 msgr2=0x7f12900b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:49.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.411+0000 7f1295a00640 1 -- 172.21.15.86:0/3690133801 shutdown_connections 2024-04-20T21:24:49.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.411+0000 7f1295a00640 1 -- 172.21.15.86:0/3690133801 wait complete. 2024-04-20T21:24:49.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.412+0000 7f1295a00640 1 Processor -- start 2024-04-20T21:24:49.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.413+0000 7f1295a00640 1 -- start start 2024-04-20T21:24:49.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.413+0000 7f1295a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1274004150 0x7f129015b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:49.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.413+0000 7f1295a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1290142bb0 0x7f1290158f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:49.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.413+0000 7f1295a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1290159480 0x7f12901f0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:49.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.413+0000 7f1295a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f129015f650 con 0x7f1290159480 2024-04-20T21:24:49.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.413+0000 7f1295a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f129015aff0 con 0x7f1274004150 2024-04-20T21:24:49.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.413+0000 7f1295a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f129015f7d0 con 0x7f1290142bb0 2024-04-20T21:24:49.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.414+0000 7f128f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1290159480 0x7f12901f0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:49.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.414+0000 7f128f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1290159480 0x7f12901f0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42858/0 (socket says 172.21.15.86:42858) 2024-04-20T21:24:49.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.414+0000 7f128f400640 1 -- 172.21.15.86:0/3489588647 learned_addr learned my addr 172.21.15.86:0/3489588647 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:49.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.414+0000 7f1286000640 1 --2- 172.21.15.86:0/3489588647 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1290142bb0 0x7f1290158f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:49.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.414+0000 7f128ea00640 1 --2- 172.21.15.86:0/3489588647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1274004150 0x7f129015b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:49.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.414+0000 7f128f400640 1 -- 172.21.15.86:0/3489588647 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1290142bb0 msgr2=0x7f1290158f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:49.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.414+0000 7f128f400640 1 --2- 172.21.15.86:0/3489588647 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1290142bb0 0x7f1290158f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:49.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.414+0000 7f128f400640 1 -- 172.21.15.86:0/3489588647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1274004150 msgr2=0x7f129015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:49.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.414+0000 7f128f400640 1 --2- 172.21.15.86:0/3489588647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1274004150 0x7f129015b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:49.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.414+0000 7f128f400640 1 -- 172.21.15.86:0/3489588647 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1290157150 con 0x7f1290159480 2024-04-20T21:24:49.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.414+0000 7f1286000640 1 --2- 172.21.15.86:0/3489588647 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1290142bb0 0x7f1290158f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:24:49.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.415+0000 7f128f400640 1 --2- 172.21.15.86:0/3489588647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1290159480 0x7f12901f0190 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f129006f420 tx=0x7f128800cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:49.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.415+0000 7f1287400640 1 -- 172.21.15.86:0/3489588647 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1288007230 con 0x7f1290159480 2024-04-20T21:24:49.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.415+0000 7f1287400640 1 -- 172.21.15.86:0/3489588647 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12880073d0 con 0x7f1290159480 2024-04-20T21:24:49.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.415+0000 7f1287400640 1 -- 172.21.15.86:0/3489588647 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128800ce50 con 0x7f1290159480 2024-04-20T21:24:49.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.415+0000 7f1295a00640 1 -- 172.21.15.86:0/3489588647 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1290157410 con 0x7f1290159480 2024-04-20T21:24:49.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.415+0000 7f1295a00640 1 -- 172.21.15.86:0/3489588647 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12901579d0 con 0x7f1290159480 2024-04-20T21:24:49.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.416+0000 7f1295a00640 1 -- 172.21.15.86:0/3489588647 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12900689e0 con 0x7f1290159480 2024-04-20T21:24:49.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.417+0000 7f1287400640 1 -- 172.21.15.86:0/3489588647 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1288011070 con 0x7f1290159480 2024-04-20T21:24:49.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.418+0000 7f1287400640 1 --2- 172.21.15.86:0/3489588647 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f126c06c920 0x7f126c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:49.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.418+0000 7f1287400640 1 -- 172.21.15.86:0/3489588647 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5029+0+0 (secure 0 0 0) 0x7f128808ab00 con 0x7f1290159480 2024-04-20T21:24:49.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.419+0000 7f128ea00640 1 --2- 172.21.15.86:0/3489588647 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f126c06c920 0x7f126c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:49.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.419+0000 7f128ea00640 1 --2- 172.21.15.86:0/3489588647 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f126c06c920 0x7f126c06ede0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f129015d570 tx=0x7f128000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:49.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.423+0000 7f1287400640 1 -- 172.21.15.86:0/3489588647 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f12880591d0 con 0x7f1290159480 2024-04-20T21:24:49.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.714+0000 7f1295a00640 1 -- 172.21.15.86:0/3489588647 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) -- 0x7f1290157ba0 con 0x7f1290159480 2024-04-20T21:24:49.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.746+0000 7f1287400640 1 -- 172.21.15.86:0/3489588647 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v354) ==== 166+0+0 (secure 0 0 0) 0x7f1288058ad0 con 0x7f1290159480 2024-04-20T21:24:49.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.749+0000 7f1295a00640 1 -- 172.21.15.86:0/3489588647 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f126c06c920 msgr2=0x7f126c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:49.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.749+0000 7f1295a00640 1 --2- 172.21.15.86:0/3489588647 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f126c06c920 0x7f126c06ede0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f129015d570 tx=0x7f128000b040 comp rx=0 tx=0).stop 2024-04-20T21:24:49.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.749+0000 7f1295a00640 1 -- 172.21.15.86:0/3489588647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1290159480 msgr2=0x7f12901f0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:49.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.749+0000 7f1295a00640 1 --2- 172.21.15.86:0/3489588647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1290159480 0x7f12901f0190 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f129006f420 tx=0x7f128800cb10 comp rx=0 tx=0).stop 2024-04-20T21:24:49.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.749+0000 7f1295a00640 1 -- 172.21.15.86:0/3489588647 shutdown_connections 2024-04-20T21:24:49.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.749+0000 7f1295a00640 1 --2- 172.21.15.86:0/3489588647 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f126c06c920 0x7f126c06ede0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:49.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.749+0000 7f1295a00640 1 --2- 172.21.15.86:0/3489588647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1290159480 0x7f12901f0190 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:49.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.749+0000 7f1295a00640 1 --2- 172.21.15.86:0/3489588647 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1290142bb0 0x7f1290158f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:49.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.749+0000 7f1295a00640 1 --2- 172.21.15.86:0/3489588647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1274004150 0x7f129015b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:49.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.749+0000 7f1295a00640 1 -- 172.21.15.86:0/3489588647 >> 172.21.15.86:0/3489588647 conn(0x7f12900b2360 msgr2=0x7f1290151ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:49.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.749+0000 7f1295a00640 1 -- 172.21.15.86:0/3489588647 shutdown_connections 2024-04-20T21:24:49.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.749+0000 7f1295a00640 1 -- 172.21.15.86:0/3489588647 wait complete. 2024-04-20T21:24:49.767 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-04-20T21:24:49.767 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-04-20T21:24:49.768 INFO:tasks.cephfs_test_runner:test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-20T21:24:49.768 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:24:49.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.957+0000 7fe35fe00640 1 Processor -- start 2024-04-20T21:24:49.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.957+0000 7fe35fe00640 1 -- start start 2024-04-20T21:24:49.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.959+0000 7fe35fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe36015ece0 0x7fe36015f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:49.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.959+0000 7fe35fe00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe36015f600 con 0x7fe3600d50c0 2024-04-20T21:24:49.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.959+0000 7fe35fe00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3600d3950 con 0x7fe3600cf9c0 2024-04-20T21:24:49.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.959+0000 7fe35fe00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe3600cec00 con 0x7fe36015ece0 2024-04-20T21:24:49.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.959+0000 7fe35ea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe36015ece0 0x7fe36015f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:49.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.959+0000 7fe35ea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe36015ece0 0x7fe36015f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42870/0 (socket says 172.21.15.86:42870) 2024-04-20T21:24:49.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.959+0000 7fe35ea00640 1 -- 172.21.15.86:0/948338556 learned_addr learned my addr 172.21.15.86:0/948338556 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:49.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.960+0000 7fe35ea00640 1 -- 172.21.15.86:0/948338556 >> v1:172.21.15.144:6789/0 conn(0x7fe3600cf9c0 legacy=0x7fe360160000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:49.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.960+0000 7fe35ea00640 1 -- 172.21.15.86:0/948338556 >> v1:172.21.15.86:6789/0 conn(0x7fe3600d50c0 legacy=0x7fe3600d3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:49.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.960+0000 7fe35ea00640 1 -- 172.21.15.86:0/948338556 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3600ceee0 con 0x7fe36015ece0 2024-04-20T21:24:49.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.960+0000 7fe35d600640 1 -- 172.21.15.86:0/948338556 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3497661376 0 0) 0x7fe36015f600 con 0x7fe3600d50c0 2024-04-20T21:24:49.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.962+0000 7fe35ea00640 1 --2- 172.21.15.86:0/948338556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe36015ece0 0x7fe36015f0c0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fe350009870 tx=0x7fe35002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79fe8d0a05c2a845 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:49.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.962+0000 7fe35d600640 1 -- 172.21.15.86:0/948338556 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe350034070 con 0x7fe36015ece0 2024-04-20T21:24:49.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.962+0000 7fe35d600640 1 -- 172.21.15.86:0/948338556 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe350030860 con 0x7fe36015ece0 2024-04-20T21:24:49.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.962+0000 7fe35d600640 1 -- 172.21.15.86:0/948338556 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe350030c30 con 0x7fe36015ece0 2024-04-20T21:24:49.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.963+0000 7fe35fe00640 1 -- 172.21.15.86:0/948338556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe36015ece0 msgr2=0x7fe36015f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:49.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.963+0000 7fe35fe00640 1 --2- 172.21.15.86:0/948338556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe36015ece0 0x7fe36015f0c0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fe350009870 tx=0x7fe35002b9a0 comp rx=0 tx=0).stop 2024-04-20T21:24:49.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.963+0000 7fe35fe00640 1 -- 172.21.15.86:0/948338556 shutdown_connections 2024-04-20T21:24:49.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.963+0000 7fe35fe00640 1 --2- 172.21.15.86:0/948338556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe36015ece0 0x7fe36015f0c0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:49.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.963+0000 7fe35fe00640 1 -- 172.21.15.86:0/948338556 >> 172.21.15.86:0/948338556 conn(0x7fe36008bf70 msgr2=0x7fe3600d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:49.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.963+0000 7fe35fe00640 1 -- 172.21.15.86:0/948338556 shutdown_connections 2024-04-20T21:24:49.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.963+0000 7fe35fe00640 1 -- 172.21.15.86:0/948338556 wait complete. 2024-04-20T21:24:49.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.965+0000 7fe35fe00640 1 Processor -- start 2024-04-20T21:24:49.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.965+0000 7fe35fe00640 1 -- start start 2024-04-20T21:24:49.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.966+0000 7fe35fe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3600cf9c0 0x7fe3601f45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:49.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.966+0000 7fe35fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3600d50c0 0x7fe36015d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:49.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.966+0000 7fe35fe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe36015ece0 0x7fe3600cc630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:49.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.966+0000 7fe35fe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe36015ea50 con 0x7fe3600d50c0 2024-04-20T21:24:49.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.966+0000 7fe35fe00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe3600cf4a0 con 0x7fe3600cf9c0 2024-04-20T21:24:49.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.966+0000 7fe35fe00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe3600cfe70 con 0x7fe36015ece0 2024-04-20T21:24:49.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.966+0000 7fe35e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3600d50c0 0x7fe36015d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:49.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.966+0000 7fe35e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3600d50c0 0x7fe36015d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42880/0 (socket says 172.21.15.86:42880) 2024-04-20T21:24:49.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.966+0000 7fe35e000640 1 -- 172.21.15.86:0/328187939 learned_addr learned my addr 172.21.15.86:0/328187939 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:49.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.966+0000 7fe35ea00640 1 --2- 172.21.15.86:0/328187939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3600cf9c0 0x7fe3601f45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:49.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.966+0000 7fe35e000640 1 -- 172.21.15.86:0/328187939 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe36015ece0 msgr2=0x7fe3600cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:49.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.966+0000 7fe35e000640 1 --2- 172.21.15.86:0/328187939 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe36015ece0 0x7fe3600cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:49.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.967+0000 7fe35e000640 1 -- 172.21.15.86:0/328187939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3600cf9c0 msgr2=0x7fe3601f45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:49.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.967+0000 7fe35e000640 1 --2- 172.21.15.86:0/328187939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3600cf9c0 0x7fe3601f45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:49.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.967+0000 7fe35e000640 1 -- 172.21.15.86:0/328187939 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3601e8100 con 0x7fe3600d50c0 2024-04-20T21:24:49.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.967+0000 7fe35e000640 1 --2- 172.21.15.86:0/328187939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3600d50c0 0x7fe36015d150 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fe34800c710 tx=0x7fe34800cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:49.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.967+0000 7fe356a00640 1 -- 172.21.15.86:0/328187939 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe348003da0 con 0x7fe3600d50c0 2024-04-20T21:24:49.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.967+0000 7fe356a00640 1 -- 172.21.15.86:0/328187939 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe34800ce20 con 0x7fe3600d50c0 2024-04-20T21:24:49.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.967+0000 7fe35fe00640 1 -- 172.21.15.86:0/328187939 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3601e83a0 con 0x7fe3600d50c0 2024-04-20T21:24:49.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.967+0000 7fe356a00640 1 -- 172.21.15.86:0/328187939 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3480052b0 con 0x7fe3600d50c0 2024-04-20T21:24:49.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.967+0000 7fe35fe00640 1 -- 172.21.15.86:0/328187939 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3600caf10 con 0x7fe3600d50c0 2024-04-20T21:24:49.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.968+0000 7fe35fe00640 1 -- 172.21.15.86:0/328187939 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe36004fcf0 con 0x7fe3600d50c0 2024-04-20T21:24:49.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.969+0000 7fe356a00640 1 -- 172.21.15.86:0/328187939 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe348005450 con 0x7fe3600d50c0 2024-04-20T21:24:49.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.970+0000 7fe356a00640 1 --2- 172.21.15.86:0/328187939 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe33c06c8f0 0x7fe33c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:49.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.970+0000 7fe356a00640 1 -- 172.21.15.86:0/328187939 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5029+0+0 (secure 0 0 0) 0x7fe34808b7a0 con 0x7fe3600d50c0 2024-04-20T21:24:49.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.970+0000 7fe35ea00640 1 --2- 172.21.15.86:0/328187939 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe33c06c8f0 0x7fe33c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:49.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.971+0000 7fe35ea00640 1 --2- 172.21.15.86:0/328187939 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe33c06c8f0 0x7fe33c06edb0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe350036000 tx=0x7fe350008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:49.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:49.975+0000 7fe356a00640 1 -- 172.21.15.86:0/328187939 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe348059e70 con 0x7fe3600d50c0 2024-04-20T21:24:50.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.176+0000 7fe35fe00640 1 -- 172.21.15.86:0/328187939 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe36008e320 con 0x7fe3600d50c0 2024-04-20T21:24:50.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.178+0000 7fe356a00640 1 -- 172.21.15.86:0/328187939 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+13149 (secure 0 0 0) 0x7fe348059770 con 0x7fe3600d50c0 2024-04-20T21:24:50.178 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:24:50.178 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":77,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:24:48.734191+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6837/2254117576":"2024-04-21T21:24:45.707002+0000","172.21.15.86:6835/2254117576":"2024-04-21T21:24:45.707002+0000","172.21.15.86:6837/202143875":"2024-04-21T21:23:12.079572+0000","172.21.15.86:6835/202143875":"2024-04-21T21:23:12.079572+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:24:50.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.181+0000 7fe35fe00640 1 -- 172.21.15.86:0/328187939 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe33c06c8f0 msgr2=0x7fe33c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:50.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.181+0000 7fe35fe00640 1 --2- 172.21.15.86:0/328187939 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe33c06c8f0 0x7fe33c06edb0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe350036000 tx=0x7fe350008030 comp rx=0 tx=0).stop 2024-04-20T21:24:50.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.181+0000 7fe35fe00640 1 -- 172.21.15.86:0/328187939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3600d50c0 msgr2=0x7fe36015d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:50.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.181+0000 7fe35fe00640 1 --2- 172.21.15.86:0/328187939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3600d50c0 0x7fe36015d150 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fe34800c710 tx=0x7fe34800cbd0 comp rx=0 tx=0).stop 2024-04-20T21:24:50.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.181+0000 7fe35fe00640 1 -- 172.21.15.86:0/328187939 shutdown_connections 2024-04-20T21:24:50.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.181+0000 7fe35fe00640 1 --2- 172.21.15.86:0/328187939 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe33c06c8f0 0x7fe33c06edb0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:50.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.181+0000 7fe35fe00640 1 --2- 172.21.15.86:0/328187939 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe36015ece0 0x7fe3600cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:50.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.181+0000 7fe35fe00640 1 --2- 172.21.15.86:0/328187939 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3600d50c0 0x7fe36015d150 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:50.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.181+0000 7fe35fe00640 1 --2- 172.21.15.86:0/328187939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3600cf9c0 0x7fe3601f45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:50.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.181+0000 7fe35fe00640 1 -- 172.21.15.86:0/328187939 >> 172.21.15.86:0/328187939 conn(0x7fe36008bf70 msgr2=0x7fe36006dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:50.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.181+0000 7fe35fe00640 1 -- 172.21.15.86:0/328187939 shutdown_connections 2024-04-20T21:24:50.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.181+0000 7fe35fe00640 1 -- 172.21.15.86:0/328187939 wait complete. 2024-04-20T21:24:50.199 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:24:50.199 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:24:50.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.404+0000 7fb966600640 1 Processor -- start 2024-04-20T21:24:50.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.404+0000 7fb966600640 1 -- start start 2024-04-20T21:24:50.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.406+0000 7fb966600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb960155d30 0x7fb9600cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:50.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.406+0000 7fb966600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb96006b980 0x7fb96006be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:50.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.406+0000 7fb966600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb960156110 con 0x7fb9600b4ba0 2024-04-20T21:24:50.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.406+0000 7fb966600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb9600b56e0 con 0x7fb960155d30 2024-04-20T21:24:50.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.406+0000 7fb966600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb9600ce460 con 0x7fb96006b980 2024-04-20T21:24:50.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.406+0000 7fb95f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb960155d30 0x7fb9600cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:50.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.406+0000 7fb95f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb960155d30 0x7fb9600cf500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42892/0 (socket says 172.21.15.86:42892) 2024-04-20T21:24:50.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.406+0000 7fb95f400640 1 -- 172.21.15.86:0/3307956958 learned_addr learned my addr 172.21.15.86:0/3307956958 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:50.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.407+0000 7fb95ea00640 1 --2- 172.21.15.86:0/3307956958 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb96006b980 0x7fb96006be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:50.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.407+0000 7fb95f400640 1 -- 172.21.15.86:0/3307956958 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb96006b980 msgr2=0x7fb96006be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:50.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.407+0000 7fb95f400640 1 --2- 172.21.15.86:0/3307956958 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb96006b980 0x7fb96006be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:50.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.407+0000 7fb95f400640 1 -- 172.21.15.86:0/3307956958 >> v1:172.21.15.86:6789/0 conn(0x7fb9600b4ba0 legacy=0x7fb9600b4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:50.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.407+0000 7fb95f400640 1 -- 172.21.15.86:0/3307956958 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9600ce740 con 0x7fb960155d30 2024-04-20T21:24:50.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.407+0000 7fb95e000640 1 -- 172.21.15.86:0/3307956958 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 768545348 0 0) 0x7fb960156110 con 0x7fb9600b4ba0 2024-04-20T21:24:50.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.408+0000 7fb95f400640 1 --2- 172.21.15.86:0/3307956958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb960155d30 0x7fb9600cf500 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fb950009e50 tx=0x7fb95002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af7a29103c79b6a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:50.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.408+0000 7fb95e000640 1 -- 172.21.15.86:0/3307956958 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb95002c9e0 con 0x7fb960155d30 2024-04-20T21:24:50.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.408+0000 7fb95e000640 1 -- 172.21.15.86:0/3307956958 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb95002cb80 con 0x7fb960155d30 2024-04-20T21:24:50.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.408+0000 7fb95e000640 1 -- 172.21.15.86:0/3307956958 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9500343f0 con 0x7fb960155d30 2024-04-20T21:24:50.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.409+0000 7fb966600640 1 -- 172.21.15.86:0/3307956958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb960155d30 msgr2=0x7fb9600cf500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:50.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.409+0000 7fb966600640 1 --2- 172.21.15.86:0/3307956958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb960155d30 0x7fb9600cf500 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fb950009e50 tx=0x7fb95002be40 comp rx=0 tx=0).stop 2024-04-20T21:24:50.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.409+0000 7fb966600640 1 -- 172.21.15.86:0/3307956958 shutdown_connections 2024-04-20T21:24:50.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.409+0000 7fb966600640 1 --2- 172.21.15.86:0/3307956958 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb96006b980 0x7fb96006be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:50.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.409+0000 7fb966600640 1 --2- 172.21.15.86:0/3307956958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb960155d30 0x7fb9600cf500 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:50.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.409+0000 7fb966600640 1 -- 172.21.15.86:0/3307956958 >> 172.21.15.86:0/3307956958 conn(0x7fb9600b2110 msgr2=0x7fb9600b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:50.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.409+0000 7fb966600640 1 -- 172.21.15.86:0/3307956958 shutdown_connections 2024-04-20T21:24:50.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.409+0000 7fb966600640 1 -- 172.21.15.86:0/3307956958 wait complete. 2024-04-20T21:24:50.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.411+0000 7fb966600640 1 Processor -- start 2024-04-20T21:24:50.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.411+0000 7fb966600640 1 -- start start 2024-04-20T21:24:50.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.412+0000 7fb966600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb96006b980 0x7fb960142db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:50.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.412+0000 7fb966600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9600b4ba0 0x7fb96014c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:50.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.412+0000 7fb966600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb960155d30 0x7fb960140860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:50.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.412+0000 7fb966600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb9600cefe0 con 0x7fb9600b4ba0 2024-04-20T21:24:50.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.412+0000 7fb966600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb9600ced00 con 0x7fb96006b980 2024-04-20T21:24:50.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.412+0000 7fb966600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb96006ddf0 con 0x7fb960155d30 2024-04-20T21:24:50.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.412+0000 7fb95ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9600b4ba0 0x7fb96014c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:50.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.412+0000 7fb95ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9600b4ba0 0x7fb96014c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42896/0 (socket says 172.21.15.86:42896) 2024-04-20T21:24:50.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.412+0000 7fb95ea00640 1 -- 172.21.15.86:0/2221508943 learned_addr learned my addr 172.21.15.86:0/2221508943 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:50.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.412+0000 7fb95fe00640 1 --2- 172.21.15.86:0/2221508943 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb960155d30 0x7fb960140860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:50.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.412+0000 7fb95f400640 1 --2- 172.21.15.86:0/2221508943 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb96006b980 0x7fb960142db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:50.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.412+0000 7fb95ea00640 1 -- 172.21.15.86:0/2221508943 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb960155d30 msgr2=0x7fb960140860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:50.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.412+0000 7fb95ea00640 1 --2- 172.21.15.86:0/2221508943 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb960155d30 0x7fb960140860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:50.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.412+0000 7fb95ea00640 1 -- 172.21.15.86:0/2221508943 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb96006b980 msgr2=0x7fb960142db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:50.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.412+0000 7fb95ea00640 1 --2- 172.21.15.86:0/2221508943 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb96006b980 0x7fb960142db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:50.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.412+0000 7fb95ea00640 1 -- 172.21.15.86:0/2221508943 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb96015e300 con 0x7fb9600b4ba0 2024-04-20T21:24:50.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.413+0000 7fb95ea00640 1 --2- 172.21.15.86:0/2221508943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9600b4ba0 0x7fb96014c350 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fb94c00d800 tx=0x7fb94c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:50.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.413+0000 7fb957400640 1 -- 172.21.15.86:0/2221508943 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94c00b6c0 con 0x7fb9600b4ba0 2024-04-20T21:24:50.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.413+0000 7fb957400640 1 -- 172.21.15.86:0/2221508943 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb94c019070 con 0x7fb9600b4ba0 2024-04-20T21:24:50.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.413+0000 7fb957400640 1 -- 172.21.15.86:0/2221508943 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94c00ba80 con 0x7fb9600b4ba0 2024-04-20T21:24:50.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.413+0000 7fb966600640 1 -- 172.21.15.86:0/2221508943 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb96015e5a0 con 0x7fb9600b4ba0 2024-04-20T21:24:50.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.413+0000 7fb966600640 1 -- 172.21.15.86:0/2221508943 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb960168470 con 0x7fb9600b4ba0 2024-04-20T21:24:50.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.414+0000 7fb966600640 1 -- 172.21.15.86:0/2221508943 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9600689e0 con 0x7fb9600b4ba0 2024-04-20T21:24:50.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.415+0000 7fb957400640 1 -- 172.21.15.86:0/2221508943 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb94c00bc20 con 0x7fb9600b4ba0 2024-04-20T21:24:50.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.416+0000 7fb957400640 1 --2- 172.21.15.86:0/2221508943 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb94006c920 0x7fb94006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:50.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.416+0000 7fb957400640 1 -- 172.21.15.86:0/2221508943 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5029+0+0 (secure 0 0 0) 0x7fb94c08bd50 con 0x7fb9600b4ba0 2024-04-20T21:24:50.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.417+0000 7fb95f400640 1 --2- 172.21.15.86:0/2221508943 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb94006c920 0x7fb94006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:50.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.417+0000 7fb95f400640 1 --2- 172.21.15.86:0/2221508943 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb94006c920 0x7fb94006ede0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb950035000 tx=0x7fb950030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:50.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.421+0000 7fb957400640 1 -- 172.21.15.86:0/2221508943 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb94c01c360 con 0x7fb9600b4ba0 2024-04-20T21:24:50.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.625+0000 7fb966600640 1 -- 172.21.15.86:0/2221508943 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb96006e4e0 con 0x7fb9600b4ba0 2024-04-20T21:24:50.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.626+0000 7fb957400640 1 -- 172.21.15.86:0/2221508943 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) ==== 93+0+10 (secure 0 0 0) 0x7fb94c05a410 con 0x7fb9600b4ba0 2024-04-20T21:24:50.625 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:24:50.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.628+0000 7fb966600640 1 -- 172.21.15.86:0/2221508943 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb94006c920 msgr2=0x7fb94006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:50.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.628+0000 7fb966600640 1 --2- 172.21.15.86:0/2221508943 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb94006c920 0x7fb94006ede0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb950035000 tx=0x7fb950030000 comp rx=0 tx=0).stop 2024-04-20T21:24:50.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.628+0000 7fb966600640 1 -- 172.21.15.86:0/2221508943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9600b4ba0 msgr2=0x7fb96014c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:50.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.628+0000 7fb966600640 1 --2- 172.21.15.86:0/2221508943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9600b4ba0 0x7fb96014c350 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fb94c00d800 tx=0x7fb94c00dcc0 comp rx=0 tx=0).stop 2024-04-20T21:24:50.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.628+0000 7fb966600640 1 -- 172.21.15.86:0/2221508943 shutdown_connections 2024-04-20T21:24:50.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.628+0000 7fb966600640 1 --2- 172.21.15.86:0/2221508943 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb94006c920 0x7fb94006ede0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:50.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.628+0000 7fb966600640 1 --2- 172.21.15.86:0/2221508943 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb960155d30 0x7fb960140860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:50.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.628+0000 7fb966600640 1 --2- 172.21.15.86:0/2221508943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9600b4ba0 0x7fb96014c350 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:50.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.628+0000 7fb966600640 1 --2- 172.21.15.86:0/2221508943 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb96006b980 0x7fb960142db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:50.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.628+0000 7fb966600640 1 -- 172.21.15.86:0/2221508943 >> 172.21.15.86:0/2221508943 conn(0x7fb9600b2110 msgr2=0x7fb9601530e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:50.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.629+0000 7fb966600640 1 -- 172.21.15.86:0/2221508943 shutdown_connections 2024-04-20T21:24:50.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.629+0000 7fb966600640 1 -- 172.21.15.86:0/2221508943 wait complete. 2024-04-20T21:24:50.647 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-04-20T21:24:50.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.847+0000 7f7b33200640 1 Processor -- start 2024-04-20T21:24:50.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.847+0000 7f7b33200640 1 -- start start 2024-04-20T21:24:50.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.849+0000 7f7b33200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b2c06e6d0 0x7f7b2c071d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:50.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.849+0000 7f7b33200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b2c06eb30 con 0x7f7b2c146130 2024-04-20T21:24:50.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.849+0000 7f7b33200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b2c06dc30 con 0x7f7b2c072240 2024-04-20T21:24:50.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.849+0000 7f7b33200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7b2c0cd8e0 con 0x7f7b2c06e6d0 2024-04-20T21:24:50.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.849+0000 7f7b2be00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f7b2c146130 0x7f7b2c0ccdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:50522/0 (socket says 172.21.15.86:50522) 2024-04-20T21:24:50.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.849+0000 7f7b2be00640 1 -- 172.21.15.86:0/1417421221 learned_addr learned my addr 172.21.15.86:0/1417421221 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:24:50.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.850+0000 7f7b2b400640 1 --2- 172.21.15.86:0/1417421221 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b2c06e6d0 0x7f7b2c071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:50.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.850+0000 7f7b2aa00640 1 -- 172.21.15.86:0/1417421221 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3222869725 0 0) 0x7f7b1c001490 con 0x7f7b2c146130 2024-04-20T21:24:50.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.850+0000 7f7b2aa00640 1 -- 172.21.15.86:0/1417421221 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b10003520 con 0x7f7b2c146130 2024-04-20T21:24:50.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.850+0000 7f7b2aa00640 1 -- 172.21.15.86:0/1417421221 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 536349727 0 0) 0x7f7b10003520 con 0x7f7b2c146130 2024-04-20T21:24:50.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.850+0000 7f7b2aa00640 1 -- 172.21.15.86:0/1417421221 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b2c06e6d0 msgr2=0x7f7b2c071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:50.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.850+0000 7f7b2aa00640 1 --2- 172.21.15.86:0/1417421221 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b2c06e6d0 0x7f7b2c071d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:50.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.850+0000 7f7b2aa00640 1 -- 172.21.15.86:0/1417421221 >> v1:172.21.15.144:6789/0 conn(0x7f7b2c072240 legacy=0x7f7b2c15cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:50.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.850+0000 7f7b2aa00640 1 -- 172.21.15.86:0/1417421221 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b2c0cdbc0 con 0x7f7b2c146130 2024-04-20T21:24:50.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.850+0000 7f7b2aa00640 1 -- 172.21.15.86:0/1417421221 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f7b1c001bc0 con 0x7f7b2c146130 2024-04-20T21:24:50.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.850+0000 7f7b2aa00640 1 -- 172.21.15.86:0/1417421221 >> v1:172.21.15.86:6789/0 conn(0x7f7b2c146130 legacy=0x7f7b2c0ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:50.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.850+0000 7f7b2aa00640 1 --2- 172.21.15.86:0/1417421221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b100039e0 0x7f7b10003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:50.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.850+0000 7f7b2aa00640 1 --2- 172.21.15.86:0/1417421221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b10004800 0x7f7b10004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:50.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.850+0000 7f7b2aa00640 1 --2- 172.21.15.86:0/1417421221 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b2c06e6d0 0x7f7b10005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:50.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.850+0000 7f7b2aa00640 1 -- 172.21.15.86:0/1417421221 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2c0cd8e0 con 0x7f7b10004800 2024-04-20T21:24:50.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.850+0000 7f7b2aa00640 1 -- 172.21.15.86:0/1417421221 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1000bb80 con 0x7f7b100039e0 2024-04-20T21:24:50.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.850+0000 7f7b2aa00640 1 -- 172.21.15.86:0/1417421221 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7b1000bd80 con 0x7f7b2c06e6d0 2024-04-20T21:24:50.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.851+0000 7f7b2b400640 1 --2- 172.21.15.86:0/1417421221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b10004800 0x7f7b10004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:50.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.851+0000 7f7b2be00640 1 --2- 172.21.15.86:0/1417421221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b100039e0 0x7f7b10003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:50.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.851+0000 7f7b31200640 1 --2- 172.21.15.86:0/1417421221 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b2c06e6d0 0x7f7b10005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:50.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.851+0000 7f7b2b400640 1 -- 172.21.15.86:0/1417421221 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b2c06e6d0 msgr2=0x7f7b10005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:50.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.851+0000 7f7b2b400640 1 --2- 172.21.15.86:0/1417421221 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b2c06e6d0 0x7f7b10005200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:50.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.851+0000 7f7b2b400640 1 -- 172.21.15.86:0/1417421221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b100039e0 msgr2=0x7f7b10003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:50.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.851+0000 7f7b2b400640 1 --2- 172.21.15.86:0/1417421221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b100039e0 0x7f7b10003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:50.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.851+0000 7f7b2b400640 1 -- 172.21.15.86:0/1417421221 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b10003520 con 0x7f7b10004800 2024-04-20T21:24:50.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.853+0000 7f7b2b400640 1 --2- 172.21.15.86:0/1417421221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b10004800 0x7f7b10004c80 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f7b1400ba30 tx=0x7f7b10005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cce1ddabcdbc7a5e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:50.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.853+0000 7f7b2aa00640 1 -- 172.21.15.86:0/1417421221 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b14036020 con 0x7f7b10004800 2024-04-20T21:24:50.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.853+0000 7f7b2aa00640 1 -- 172.21.15.86:0/1417421221 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b14032640 con 0x7f7b10004800 2024-04-20T21:24:50.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.853+0000 7f7b2aa00640 1 -- 172.21.15.86:0/1417421221 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1400b600 con 0x7f7b10004800 2024-04-20T21:24:50.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.854+0000 7f7b33200640 1 -- 172.21.15.86:0/1417421221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b10004800 msgr2=0x7f7b10004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:50.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.854+0000 7f7b33200640 1 --2- 172.21.15.86:0/1417421221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b10004800 0x7f7b10004c80 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f7b1400ba30 tx=0x7f7b10005830 comp rx=0 tx=0).stop 2024-04-20T21:24:50.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.854+0000 7f7b31200640 1 -- 172.21.15.86:0/1417421221 reap_dead start 2024-04-20T21:24:50.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.854+0000 7f7b33200640 1 -- 172.21.15.86:0/1417421221 shutdown_connections 2024-04-20T21:24:50.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.854+0000 7f7b33200640 1 -- 172.21.15.86:0/1417421221 >> 172.21.15.86:0/1417421221 conn(0x7f7b2c0b2360 msgr2=0x7f7b2c0b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:50.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.854+0000 7f7b33200640 1 -- 172.21.15.86:0/1417421221 shutdown_connections 2024-04-20T21:24:50.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.854+0000 7f7b33200640 1 -- 172.21.15.86:0/1417421221 wait complete. 2024-04-20T21:24:50.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.855+0000 7f7b33200640 1 Processor -- start 2024-04-20T21:24:50.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.856+0000 7f7b33200640 1 -- start start 2024-04-20T21:24:50.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.856+0000 7f7b33200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b10004800 0x7f7b2c1e5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:50.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.856+0000 7f7b33200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b2c13db60 0x7f7b2c13fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:50.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.857+0000 7f7b33200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b2c140090 0x7f7b2c1f8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:50.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.857+0000 7f7b33200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2c06f150 con 0x7f7b10004800 2024-04-20T21:24:50.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.857+0000 7f7b33200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2c06e1b0 con 0x7f7b2c140090 2024-04-20T21:24:50.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.857+0000 7f7b33200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7b2c0cd4e0 con 0x7f7b2c13db60 2024-04-20T21:24:50.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.857+0000 7f7b2be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b10004800 0x7f7b2c1e5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:50.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.857+0000 7f7b2be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b10004800 0x7f7b2c1e5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42912/0 (socket says 172.21.15.86:42912) 2024-04-20T21:24:50.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.857+0000 7f7b2be00640 1 -- 172.21.15.86:0/3868279245 learned_addr learned my addr 172.21.15.86:0/3868279245 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:50.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.857+0000 7f7b31200640 1 --2- 172.21.15.86:0/3868279245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b2c140090 0x7f7b2c1f8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:50.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.857+0000 7f7b2be00640 1 -- 172.21.15.86:0/3868279245 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b2c13db60 msgr2=0x7f7b2c13fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:50.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.857+0000 7f7b2b400640 1 --2- 172.21.15.86:0/3868279245 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b2c13db60 0x7f7b2c13fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:50.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.857+0000 7f7b2be00640 1 --2- 172.21.15.86:0/3868279245 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b2c13db60 0x7f7b2c13fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:50.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.857+0000 7f7b2be00640 1 -- 172.21.15.86:0/3868279245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b2c140090 msgr2=0x7f7b2c1f8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:50.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.857+0000 7f7b2be00640 1 --2- 172.21.15.86:0/3868279245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b2c140090 0x7f7b2c1f8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:50.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.857+0000 7f7b2be00640 1 -- 172.21.15.86:0/3868279245 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b2c1f9570 con 0x7f7b10004800 2024-04-20T21:24:50.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.857+0000 7f7b31200640 1 --2- 172.21.15.86:0/3868279245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b2c140090 0x7f7b2c1f8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:24:50.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.858+0000 7f7b2be00640 1 --2- 172.21.15.86:0/3868279245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b10004800 0x7f7b2c1e5a00 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c007a40 tx=0x7f7b1c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:50.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.858+0000 7f7b23e00640 1 -- 172.21.15.86:0/3868279245 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c017020 con 0x7f7b10004800 2024-04-20T21:24:50.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.858+0000 7f7b23e00640 1 -- 172.21.15.86:0/3868279245 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b1c00ee50 con 0x7f7b10004800 2024-04-20T21:24:50.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.858+0000 7f7b23e00640 1 -- 172.21.15.86:0/3868279245 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c013660 con 0x7f7b10004800 2024-04-20T21:24:50.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.858+0000 7f7b33200640 1 -- 172.21.15.86:0/3868279245 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b2c1f9290 con 0x7f7b10004800 2024-04-20T21:24:50.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.858+0000 7f7b33200640 1 -- 172.21.15.86:0/3868279245 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b2c1e5200 con 0x7f7b10004800 2024-04-20T21:24:50.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.859+0000 7f7b33200640 1 -- 172.21.15.86:0/3868279245 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b2c0689e0 con 0x7f7b10004800 2024-04-20T21:24:50.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.860+0000 7f7b23e00640 1 -- 172.21.15.86:0/3868279245 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7b1c01a020 con 0x7f7b10004800 2024-04-20T21:24:50.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.861+0000 7f7b23e00640 1 --2- 172.21.15.86:0/3868279245 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7b0806c8f0 0x7f7b0806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:50.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.861+0000 7f7b23e00640 1 -- 172.21.15.86:0/3868279245 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5029+0+0 (secure 0 0 0) 0x7f7b1c08c070 con 0x7f7b10004800 2024-04-20T21:24:50.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.862+0000 7f7b2b400640 1 --2- 172.21.15.86:0/3868279245 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7b0806c8f0 0x7f7b0806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:50.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.862+0000 7f7b2b400640 1 --2- 172.21.15.86:0/3868279245 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7b0806c8f0 0x7f7b0806edb0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f7b1400ba00 tx=0x7f7b14030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:50.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:50.866+0000 7f7b23e00640 1 -- 172.21.15.86:0/3868279245 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7b1c021020 con 0x7f7b10004800 2024-04-20T21:24:51.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.140+0000 7f7b33200640 1 -- 172.21.15.86:0/3868279245 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) -- 0x7f7b2c140470 con 0x7f7b10004800 2024-04-20T21:24:51.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.760+0000 7f7b23e00640 1 -- 172.21.15.86:0/3868279245 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v356) ==== 170+0+0 (secure 0 0 0) 0x7f7b2c06eb30 con 0x7f7b10004800 2024-04-20T21:24:51.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.764+0000 7f7b33200640 1 -- 172.21.15.86:0/3868279245 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7b0806c8f0 msgr2=0x7f7b0806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:51.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.764+0000 7f7b33200640 1 --2- 172.21.15.86:0/3868279245 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7b0806c8f0 0x7f7b0806edb0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f7b1400ba00 tx=0x7f7b14030000 comp rx=0 tx=0).stop 2024-04-20T21:24:51.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.764+0000 7f7b33200640 1 -- 172.21.15.86:0/3868279245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b10004800 msgr2=0x7f7b2c1e5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:51.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.764+0000 7f7b33200640 1 --2- 172.21.15.86:0/3868279245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b10004800 0x7f7b2c1e5a00 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c007a40 tx=0x7f7b1c00e530 comp rx=0 tx=0).stop 2024-04-20T21:24:51.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.764+0000 7f7b33200640 1 -- 172.21.15.86:0/3868279245 shutdown_connections 2024-04-20T21:24:51.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.764+0000 7f7b33200640 1 --2- 172.21.15.86:0/3868279245 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7b0806c8f0 0x7f7b0806edb0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:51.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.764+0000 7f7b33200640 1 --2- 172.21.15.86:0/3868279245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b2c140090 0x7f7b2c1f8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:51.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.764+0000 7f7b33200640 1 --2- 172.21.15.86:0/3868279245 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7b2c13db60 0x7f7b2c13fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:51.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.764+0000 7f7b33200640 1 --2- 172.21.15.86:0/3868279245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b10004800 0x7f7b2c1e5a00 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:51.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.764+0000 7f7b33200640 1 -- 172.21.15.86:0/3868279245 >> 172.21.15.86:0/3868279245 conn(0x7f7b2c0b2360 msgr2=0x7f7b2c0b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:51.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.764+0000 7f7b33200640 1 -- 172.21.15.86:0/3868279245 shutdown_connections 2024-04-20T21:24:51.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.764+0000 7f7b33200640 1 -- 172.21.15.86:0/3868279245 wait complete. 2024-04-20T21:24:51.783 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-20T21:24:51.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.988+0000 7f81d6000640 1 Processor -- start 2024-04-20T21:24:51.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.989+0000 7f81d6000640 1 -- start start 2024-04-20T21:24:51.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.990+0000 7f81d6000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81d00d45a0 0x7f81d00d3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:51.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.990+0000 7f81d6000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f81d006d570 0x7f81d00cf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:51.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.990+0000 7f81d6000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81d00cf120 con 0x7f81d015ea70 2024-04-20T21:24:51.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.990+0000 7f81d6000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f81d006de70 con 0x7f81d006d570 2024-04-20T21:24:51.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.990+0000 7f81d6000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f81d006e150 con 0x7f81d00d45a0 2024-04-20T21:24:51.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.990+0000 7f81cfe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f81d015ea70 0x7f81d00d4980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:50532/0 (socket says 172.21.15.86:50532) 2024-04-20T21:24:51.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.990+0000 7f81cfe00640 1 -- 172.21.15.86:0/1744581199 learned_addr learned my addr 172.21.15.86:0/1744581199 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:24:51.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.990+0000 7f81d5600640 1 --2- 172.21.15.86:0/1744581199 >> v2:172.21.15.86:3300/0 conn(0x7f81d006d570 0x7f81d00cf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:51.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.991+0000 7f81d4c00640 1 --2- 172.21.15.86:0/1744581199 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81d00d45a0 0x7f81d00d3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:51.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.991+0000 7f81cf400640 1 -- 172.21.15.86:0/1744581199 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3930576048 0 0) 0x7f81d00cf120 con 0x7f81d015ea70 2024-04-20T21:24:51.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.991+0000 7f81cf400640 1 -- 172.21.15.86:0/1744581199 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f81b0003520 con 0x7f81d015ea70 2024-04-20T21:24:51.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.991+0000 7f81d5600640 1 -- 172.21.15.86:0/1744581199 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81d00d45a0 msgr2=0x7f81d00d3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:51.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.991+0000 7f81d5600640 1 --2- 172.21.15.86:0/1744581199 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81d00d45a0 0x7f81d00d3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:51.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.991+0000 7f81d5600640 1 -- 172.21.15.86:0/1744581199 >> v1:172.21.15.86:6789/0 conn(0x7f81d015ea70 legacy=0x7f81d00d4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:51.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.991+0000 7f81d5600640 1 -- 172.21.15.86:0/1744581199 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81d006e430 con 0x7f81d006d570 2024-04-20T21:24:51.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.992+0000 7f81d5600640 1 --2- 172.21.15.86:0/1744581199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81d006d570 0x7f81d00cf8e0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f81c80097a0 tx=0x7f81c802b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e7c416e256bd69 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:51.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.993+0000 7f81cf400640 1 -- 172.21.15.86:0/1744581199 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c8031780 con 0x7f81d006d570 2024-04-20T21:24:51.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.993+0000 7f81cf400640 1 -- 172.21.15.86:0/1744581199 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81c8037070 con 0x7f81d006d570 2024-04-20T21:24:51.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.993+0000 7f81cf400640 1 -- 172.21.15.86:0/1744581199 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c8031b40 con 0x7f81d006d570 2024-04-20T21:24:51.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.993+0000 7f81d6000640 1 -- 172.21.15.86:0/1744581199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81d006d570 msgr2=0x7f81d00cf8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:51.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.993+0000 7f81d6000640 1 --2- 172.21.15.86:0/1744581199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81d006d570 0x7f81d00cf8e0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f81c80097a0 tx=0x7f81c802b820 comp rx=0 tx=0).stop 2024-04-20T21:24:51.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.993+0000 7f81d6000640 1 -- 172.21.15.86:0/1744581199 shutdown_connections 2024-04-20T21:24:51.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.993+0000 7f81d6000640 1 --2- 172.21.15.86:0/1744581199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81d006d570 0x7f81d00cf8e0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:51.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.993+0000 7f81d6000640 1 --2- 172.21.15.86:0/1744581199 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81d00d45a0 0x7f81d00d3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:51.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.993+0000 7f81d6000640 1 -- 172.21.15.86:0/1744581199 >> 172.21.15.86:0/1744581199 conn(0x7f81d009d9d0 msgr2=0x7f81d008bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:51.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.994+0000 7f81d6000640 1 -- 172.21.15.86:0/1744581199 shutdown_connections 2024-04-20T21:24:51.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.994+0000 7f81d6000640 1 -- 172.21.15.86:0/1744581199 wait complete. 2024-04-20T21:24:51.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.995+0000 7f81d6000640 1 Processor -- start 2024-04-20T21:24:51.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.996+0000 7f81d6000640 1 -- start start 2024-04-20T21:24:51.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.996+0000 7f81d6000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81d006d570 0x7f81d015a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:51.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.997+0000 7f81d6000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81d00d45a0 0x7f81d00c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:51.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.997+0000 7f81d6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81d015ea70 0x7f81d01f4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:51.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.997+0000 7f81d6000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f81d006db80 con 0x7f81d015ea70 2024-04-20T21:24:51.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.997+0000 7f81d6000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f81d006da00 con 0x7f81d006d570 2024-04-20T21:24:51.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.997+0000 7f81d6000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f81d00cfe20 con 0x7f81d00d45a0 2024-04-20T21:24:51.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.997+0000 7f81d5600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81d015ea70 0x7f81d01f4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:51.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.997+0000 7f81d5600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81d015ea70 0x7f81d01f4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42930/0 (socket says 172.21.15.86:42930) 2024-04-20T21:24:51.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.997+0000 7f81d5600640 1 -- 172.21.15.86:0/3600374719 learned_addr learned my addr 172.21.15.86:0/3600374719 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:51.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.997+0000 7f81cfe00640 1 --2- 172.21.15.86:0/3600374719 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81d00d45a0 0x7f81d00c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:51.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.997+0000 7f81d5600640 1 -- 172.21.15.86:0/3600374719 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81d00d45a0 msgr2=0x7f81d00c9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:51.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.997+0000 7f81d5600640 1 --2- 172.21.15.86:0/3600374719 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81d00d45a0 0x7f81d00c9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:51.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.997+0000 7f81d5600640 1 -- 172.21.15.86:0/3600374719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81d006d570 msgr2=0x7f81d015a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:51.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.997+0000 7f81d5600640 1 --2- 172.21.15.86:0/3600374719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81d006d570 0x7f81d015a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:51.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.997+0000 7f81d5600640 1 -- 172.21.15.86:0/3600374719 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81d015ae30 con 0x7f81d015ea70 2024-04-20T21:24:51.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.998+0000 7f81d5600640 1 --2- 172.21.15.86:0/3600374719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81d015ea70 0x7f81d01f4c00 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f81c8034040 tx=0x7f81c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:51.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.998+0000 7f81ccc00640 1 -- 172.21.15.86:0/3600374719 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c8032610 con 0x7f81d015ea70 2024-04-20T21:24:51.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.998+0000 7f81ccc00640 1 -- 172.21.15.86:0/3600374719 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81c80327b0 con 0x7f81d015ea70 2024-04-20T21:24:51.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.998+0000 7f81ccc00640 1 -- 172.21.15.86:0/3600374719 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c8032ba0 con 0x7f81d015ea70 2024-04-20T21:24:51.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.998+0000 7f81d6000640 1 -- 172.21.15.86:0/3600374719 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81d00ca890 con 0x7f81d015ea70 2024-04-20T21:24:51.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:51.998+0000 7f81d6000640 1 -- 172.21.15.86:0/3600374719 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81d00cae50 con 0x7f81d015ea70 2024-04-20T21:24:51.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.000+0000 7f81d6000640 1 -- 172.21.15.86:0/3600374719 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81d00d4040 con 0x7f81d015ea70 2024-04-20T21:24:51.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.000+0000 7f81ccc00640 1 -- 172.21.15.86:0/3600374719 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f81c804c060 con 0x7f81d015ea70 2024-04-20T21:24:52.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.001+0000 7f81ccc00640 1 --2- 172.21.15.86:0/3600374719 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f81ac06c8f0 0x7f81ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:52.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.001+0000 7f81ccc00640 1 -- 172.21.15.86:0/3600374719 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5029+0+0 (secure 0 0 0) 0x7f81c80ad110 con 0x7f81d015ea70 2024-04-20T21:24:52.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.001+0000 7f81d4c00640 1 --2- 172.21.15.86:0/3600374719 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f81ac06c8f0 0x7f81ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:52.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.002+0000 7f81d4c00640 1 --2- 172.21.15.86:0/3600374719 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f81ac06c8f0 0x7f81ac06edb0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f81c00098f0 tx=0x7f81c0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:52.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.006+0000 7f81ccc00640 1 -- 172.21.15.86:0/3600374719 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f81d00d4040 con 0x7f81d015ea70 2024-04-20T21:24:52.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.205+0000 7f81d6000640 1 -- 172.21.15.86:0/3600374719 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f81d01f44d0 con 0x7f81d015ea70 2024-04-20T21:24:52.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.211+0000 7f81ccc00640 1 -- 172.21.15.86:0/3600374719 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7f81c8040070 con 0x7f81d015ea70 2024-04-20T21:24:52.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.214+0000 7f81d6000640 1 -- 172.21.15.86:0/3600374719 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f81ac06c8f0 msgr2=0x7f81ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:52.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.214+0000 7f81d6000640 1 --2- 172.21.15.86:0/3600374719 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f81ac06c8f0 0x7f81ac06edb0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f81c00098f0 tx=0x7f81c0008040 comp rx=0 tx=0).stop 2024-04-20T21:24:52.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.214+0000 7f81d6000640 1 -- 172.21.15.86:0/3600374719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81d015ea70 msgr2=0x7f81d01f4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:52.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.214+0000 7f81d6000640 1 --2- 172.21.15.86:0/3600374719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81d015ea70 0x7f81d01f4c00 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f81c8034040 tx=0x7f81c80046c0 comp rx=0 tx=0).stop 2024-04-20T21:24:52.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.214+0000 7f81d6000640 1 -- 172.21.15.86:0/3600374719 shutdown_connections 2024-04-20T21:24:52.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.214+0000 7f81d6000640 1 --2- 172.21.15.86:0/3600374719 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f81ac06c8f0 0x7f81ac06edb0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:52.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.214+0000 7f81d6000640 1 --2- 172.21.15.86:0/3600374719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81d015ea70 0x7f81d01f4c00 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:52.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.214+0000 7f81d6000640 1 --2- 172.21.15.86:0/3600374719 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f81d00d45a0 0x7f81d00c9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:52.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.214+0000 7f81d6000640 1 --2- 172.21.15.86:0/3600374719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81d006d570 0x7f81d015a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:52.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.214+0000 7f81d6000640 1 -- 172.21.15.86:0/3600374719 >> 172.21.15.86:0/3600374719 conn(0x7f81d009d9d0 msgr2=0x7f81d00b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:52.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.214+0000 7f81d6000640 1 -- 172.21.15.86:0/3600374719 shutdown_connections 2024-04-20T21:24:52.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.214+0000 7f81d6000640 1 -- 172.21.15.86:0/3600374719 wait complete. 2024-04-20T21:24:52.230 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-20T21:24:52.230 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T21:24:52.331 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T21:24:52.447 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-20T21:24:52.447 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:24:52.447 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:24:52.505 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052731,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248996k,nr_inodes=812249,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T21:24:52.505 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:24:52.505 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:24:52.563 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T21:24:52.563 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:24:52.563 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:24:52.620 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052731,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248996k,nr_inodes=812249,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T21:24:52.621 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:24:52.621 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:24:52.678 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T21:24:52.679 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-20T21:24:52.679 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:24:52.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.879+0000 7f1a78a00640 1 Processor -- start 2024-04-20T21:24:52.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.879+0000 7f1a78a00640 1 -- start start 2024-04-20T21:24:52.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.881+0000 7f1a78a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a740cd170 0x7f1a7406ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:52.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.881+0000 7f1a78a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1a740cd5d0 0x7f1a7406f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:52.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.881+0000 7f1a78a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a740cdb50 con 0x7f1a74144e90 2024-04-20T21:24:52.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.881+0000 7f1a78a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f1a74145270 con 0x7f1a740cd5d0 2024-04-20T21:24:52.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.881+0000 7f1a78a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1a740b3e50 con 0x7f1a740cd170 2024-04-20T21:24:52.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.881+0000 7f1a71e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f1a74144e90 0x7f1a740b4930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:50534/0 (socket says 172.21.15.86:50534) 2024-04-20T21:24:52.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.881+0000 7f1a71e00640 1 -- 172.21.15.86:0/937869267 learned_addr learned my addr 172.21.15.86:0/937869267 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:24:52.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.881+0000 7f1a72800640 1 --2- 172.21.15.86:0/937869267 >> v2:172.21.15.144:3300/0 conn(0x7f1a740cd5d0 0x7f1a7406f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:52.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.881+0000 7f1a71400640 1 --2- 172.21.15.86:0/937869267 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a740cd170 0x7f1a7406ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:52.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.882+0000 7f1a70a00640 1 -- 172.21.15.86:0/937869267 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1409670593 0 0) 0x7f1a740cdb50 con 0x7f1a74144e90 2024-04-20T21:24:52.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.882+0000 7f1a70a00640 1 -- 172.21.15.86:0/937869267 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a58003520 con 0x7f1a74144e90 2024-04-20T21:24:52.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.882+0000 7f1a70a00640 1 -- 172.21.15.86:0/937869267 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2624258901 0 0) 0x7f1a58003520 con 0x7f1a74144e90 2024-04-20T21:24:52.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.882+0000 7f1a70a00640 1 -- 172.21.15.86:0/937869267 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a740cd170 msgr2=0x7f1a7406ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:52.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.882+0000 7f1a70a00640 1 --2- 172.21.15.86:0/937869267 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a740cd170 0x7f1a7406ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:52.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.882+0000 7f1a70a00640 1 -- 172.21.15.86:0/937869267 >> v2:172.21.15.144:3300/0 conn(0x7f1a740cd5d0 msgr2=0x7f1a7406f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:52.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.882+0000 7f1a70a00640 1 --2- 172.21.15.86:0/937869267 >> v2:172.21.15.144:3300/0 conn(0x7f1a740cd5d0 0x7f1a7406f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:52.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.882+0000 7f1a70a00640 1 -- 172.21.15.86:0/937869267 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a740b4130 con 0x7f1a74144e90 2024-04-20T21:24:52.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.882+0000 7f1a70a00640 1 -- 172.21.15.86:0/937869267 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f1a64001670 con 0x7f1a74144e90 2024-04-20T21:24:52.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.882+0000 7f1a70a00640 1 -- 172.21.15.86:0/937869267 >> v1:172.21.15.86:6789/0 conn(0x7f1a74144e90 legacy=0x7f1a740b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:52.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.882+0000 7f1a70a00640 1 --2- 172.21.15.86:0/937869267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a580039b0 0x7f1a58003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:52.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.882+0000 7f1a70a00640 1 --2- 172.21.15.86:0/937869267 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a740cd170 0x7f1a58004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:52.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.882+0000 7f1a70a00640 1 --2- 172.21.15.86:0/937869267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a58004d40 0x7f1a580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:52.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.882+0000 7f1a70a00640 1 -- 172.21.15.86:0/937869267 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1a74145270 con 0x7f1a580039b0 2024-04-20T21:24:52.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.882+0000 7f1a70a00640 1 -- 172.21.15.86:0/937869267 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1a740b3e50 con 0x7f1a58004d40 2024-04-20T21:24:52.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.882+0000 7f1a70a00640 1 -- 172.21.15.86:0/937869267 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1a5800ba70 con 0x7f1a740cd170 2024-04-20T21:24:52.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.883+0000 7f1a71e00640 1 --2- 172.21.15.86:0/937869267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a580039b0 0x7f1a58003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:52.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.883+0000 7f1a72800640 1 --2- 172.21.15.86:0/937869267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a58004d40 0x7f1a580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:52.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.883+0000 7f1a71400640 1 --2- 172.21.15.86:0/937869267 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a740cd170 0x7f1a58004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:52.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.883+0000 7f1a71e00640 1 -- 172.21.15.86:0/937869267 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a740cd170 msgr2=0x7f1a58004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:52.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.883+0000 7f1a71e00640 1 --2- 172.21.15.86:0/937869267 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a740cd170 0x7f1a58004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:52.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.883+0000 7f1a71e00640 1 -- 172.21.15.86:0/937869267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a58004d40 msgr2=0x7f1a580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:52.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.883+0000 7f1a71e00640 1 --2- 172.21.15.86:0/937869267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a58004d40 0x7f1a580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:52.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.883+0000 7f1a71e00640 1 -- 172.21.15.86:0/937869267 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a58003520 con 0x7f1a580039b0 2024-04-20T21:24:52.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.884+0000 7f1a71e00640 1 --2- 172.21.15.86:0/937869267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a580039b0 0x7f1a58003dd0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f1a6400c240 tx=0x7f1a58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edac8cdc7ff9532b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:52.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.884+0000 7f1a70a00640 1 -- 172.21.15.86:0/937869267 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a64007d30 con 0x7f1a580039b0 2024-04-20T21:24:52.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.884+0000 7f1a70a00640 1 -- 172.21.15.86:0/937869267 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a64030070 con 0x7f1a580039b0 2024-04-20T21:24:52.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.884+0000 7f1a70a00640 1 -- 172.21.15.86:0/937869267 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a6402cd60 con 0x7f1a580039b0 2024-04-20T21:24:52.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.885+0000 7f1a78a00640 1 -- 172.21.15.86:0/937869267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a580039b0 msgr2=0x7f1a58003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:52.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.885+0000 7f1a78a00640 1 --2- 172.21.15.86:0/937869267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a580039b0 0x7f1a58003dd0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f1a6400c240 tx=0x7f1a58005830 comp rx=0 tx=0).stop 2024-04-20T21:24:52.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.885+0000 7f1a72800640 1 -- 172.21.15.86:0/937869267 reap_dead start 2024-04-20T21:24:52.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.885+0000 7f1a78a00640 1 -- 172.21.15.86:0/937869267 shutdown_connections 2024-04-20T21:24:52.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.885+0000 7f1a78a00640 1 -- 172.21.15.86:0/937869267 >> 172.21.15.86:0/937869267 conn(0x7f1a740b2360 msgr2=0x7f1a740b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:52.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.885+0000 7f1a78a00640 1 -- 172.21.15.86:0/937869267 shutdown_connections 2024-04-20T21:24:52.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.885+0000 7f1a78a00640 1 -- 172.21.15.86:0/937869267 wait complete. 2024-04-20T21:24:52.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.887+0000 7f1a78a00640 1 Processor -- start 2024-04-20T21:24:52.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.887+0000 7f1a78a00640 1 -- start start 2024-04-20T21:24:52.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.888+0000 7f1a78a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a580039b0 0x7f1a7413d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:52.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.888+0000 7f1a78a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a741e4c50 0x7f1a741e50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:52.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.888+0000 7f1a78a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a741f4790 0x7f1a741f4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:52.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.888+0000 7f1a78a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1a740d0190 con 0x7f1a580039b0 2024-04-20T21:24:52.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.888+0000 7f1a78a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1a740d0010 con 0x7f1a741e4c50 2024-04-20T21:24:52.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.888+0000 7f1a78a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1a7406ed40 con 0x7f1a741f4790 2024-04-20T21:24:52.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.888+0000 7f1a71e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a580039b0 0x7f1a7413d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:52.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.888+0000 7f1a71e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a580039b0 0x7f1a7413d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42944/0 (socket says 172.21.15.86:42944) 2024-04-20T21:24:52.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.888+0000 7f1a71e00640 1 -- 172.21.15.86:0/3096365552 learned_addr learned my addr 172.21.15.86:0/3096365552 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:52.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.889+0000 7f1a72800640 1 --2- 172.21.15.86:0/3096365552 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a741f4790 0x7f1a741f4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:52.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.889+0000 7f1a71400640 1 --2- 172.21.15.86:0/3096365552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a741e4c50 0x7f1a741e50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:52.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.889+0000 7f1a71e00640 1 -- 172.21.15.86:0/3096365552 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a741f4790 msgr2=0x7f1a741f4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:52.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.889+0000 7f1a71e00640 1 --2- 172.21.15.86:0/3096365552 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a741f4790 0x7f1a741f4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:52.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.889+0000 7f1a71e00640 1 -- 172.21.15.86:0/3096365552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a741e4c50 msgr2=0x7f1a741e50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:52.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.889+0000 7f1a71e00640 1 --2- 172.21.15.86:0/3096365552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a741e4c50 0x7f1a741e50d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:52.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.889+0000 7f1a71e00640 1 -- 172.21.15.86:0/3096365552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a741f54e0 con 0x7f1a580039b0 2024-04-20T21:24:52.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.889+0000 7f1a71e00640 1 --2- 172.21.15.86:0/3096365552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a580039b0 0x7f1a7413d3a0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f1a6400c360 tx=0x7f1a740b4300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:52.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.889+0000 7f1a6a000640 1 -- 172.21.15.86:0/3096365552 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a64007d30 con 0x7f1a580039b0 2024-04-20T21:24:52.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.889+0000 7f1a6a000640 1 -- 172.21.15.86:0/3096365552 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a64004bf0 con 0x7f1a580039b0 2024-04-20T21:24:52.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.889+0000 7f1a78a00640 1 -- 172.21.15.86:0/3096365552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a741f57c0 con 0x7f1a580039b0 2024-04-20T21:24:52.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.889+0000 7f1a6a000640 1 -- 172.21.15.86:0/3096365552 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a64001c70 con 0x7f1a580039b0 2024-04-20T21:24:52.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.889+0000 7f1a78a00640 1 -- 172.21.15.86:0/3096365552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a741f5d80 con 0x7f1a580039b0 2024-04-20T21:24:52.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.891+0000 7f1a78a00640 1 -- 172.21.15.86:0/3096365552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a740ce8b0 con 0x7f1a580039b0 2024-04-20T21:24:52.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.891+0000 7f1a6a000640 1 -- 172.21.15.86:0/3096365552 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1a64007070 con 0x7f1a580039b0 2024-04-20T21:24:52.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.892+0000 7f1a6a000640 1 --2- 172.21.15.86:0/3096365552 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1a5006c8f0 0x7f1a5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:52.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.892+0000 7f1a6a000640 1 -- 172.21.15.86:0/3096365552 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5029+0+0 (secure 0 0 0) 0x7f1a640ac970 con 0x7f1a580039b0 2024-04-20T21:24:52.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.893+0000 7f1a71400640 1 --2- 172.21.15.86:0/3096365552 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1a5006c8f0 0x7f1a5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:52.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.893+0000 7f1a71400640 1 --2- 172.21.15.86:0/3096365552 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1a5006c8f0 0x7f1a5006edb0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f1a741f7cc0 tx=0x7f1a5c00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:52.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:52.897+0000 7f1a6a000640 1 -- 172.21.15.86:0/3096365552 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1a740ce8b0 con 0x7f1a580039b0 2024-04-20T21:24:53.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.138+0000 7f1a78a00640 1 -- 172.21.15.86:0/3096365552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1a740689e0 con 0x7f1a580039b0 2024-04-20T21:24:53.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.139+0000 7f1a6a000640 1 -- 172.21.15.86:0/3096365552 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+3869 (secure 0 0 0) 0x7f1a6407afa0 con 0x7f1a580039b0 2024-04-20T21:24:53.138 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:24:53.138 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":52,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5039,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2740872549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2740872549},{"type":"v1","addr":"172.21.15.86:6836","nonce":2740872549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5048,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/301899138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":301899138},{"type":"v1","addr":"172.21.15.144:6833","nonce":301899138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5051,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2816343379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2816343379},{"type":"v1","addr":"172.21.15.144:6835","nonce":2816343379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":7425,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/627167154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":627167154},{"type":"v1","addr":"172.21.15.86:6837","nonce":627167154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52}],"filesystems":[]} 2024-04-20T21:24:53.138 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 52 2024-04-20T21:24:53.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.141+0000 7f1a78a00640 1 -- 172.21.15.86:0/3096365552 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1a5006c8f0 msgr2=0x7f1a5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:53.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.141+0000 7f1a78a00640 1 --2- 172.21.15.86:0/3096365552 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1a5006c8f0 0x7f1a5006edb0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f1a741f7cc0 tx=0x7f1a5c00b040 comp rx=0 tx=0).stop 2024-04-20T21:24:53.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.141+0000 7f1a78a00640 1 -- 172.21.15.86:0/3096365552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a580039b0 msgr2=0x7f1a7413d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:53.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.141+0000 7f1a78a00640 1 --2- 172.21.15.86:0/3096365552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a580039b0 0x7f1a7413d3a0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f1a6400c360 tx=0x7f1a740b4300 comp rx=0 tx=0).stop 2024-04-20T21:24:53.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.142+0000 7f1a78a00640 1 -- 172.21.15.86:0/3096365552 shutdown_connections 2024-04-20T21:24:53.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.142+0000 7f1a78a00640 1 --2- 172.21.15.86:0/3096365552 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1a5006c8f0 0x7f1a5006edb0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:53.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.142+0000 7f1a78a00640 1 --2- 172.21.15.86:0/3096365552 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1a741f4790 0x7f1a741f4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:53.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.142+0000 7f1a78a00640 1 --2- 172.21.15.86:0/3096365552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a741e4c50 0x7f1a741e50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:53.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.142+0000 7f1a78a00640 1 --2- 172.21.15.86:0/3096365552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a580039b0 0x7f1a7413d3a0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:53.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.142+0000 7f1a78a00640 1 -- 172.21.15.86:0/3096365552 >> 172.21.15.86:0/3096365552 conn(0x7f1a740b2360 msgr2=0x7f1a741441a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:53.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.142+0000 7f1a78a00640 1 -- 172.21.15.86:0/3096365552 shutdown_connections 2024-04-20T21:24:53.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.142+0000 7f1a78a00640 1 -- 172.21.15.86:0/3096365552 wait complete. 2024-04-20T21:24:53.159 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-20T21:24:53.159 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-20T21:24:53.160 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-20T21:24:53.160 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:24:53.160 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-20T21:24:53.160 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-20T21:24:53.160 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-20T21:24:53.160 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:24:53.160 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-20T21:24:53.160 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-20T21:24:53.161 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-20T21:24:53.161 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:24:53.161 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-20T21:24:53.161 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-20T21:24:53.161 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-20T21:24:53.161 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:24:53.179 INFO:tasks.ceph.mds.a:Stopped 2024-04-20T21:24:53.179 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-20T21:24:53.182 INFO:tasks.ceph.mds.c:Stopped 2024-04-20T21:24:53.182 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-20T21:24:53.183 INFO:tasks.ceph.mds.d:Stopped 2024-04-20T21:24:53.183 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-20T21:24:53.184 INFO:tasks.ceph.mds.b:Stopped 2024-04-20T21:24:53.185 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-20T21:24:53.227 INFO:tasks.ceph.mds.a:Started 2024-04-20T21:24:53.228 DEBUG:teuthology.parallel:result is None 2024-04-20T21:24:53.232 INFO:tasks.ceph.mds.c:Started 2024-04-20T21:24:53.232 DEBUG:teuthology.parallel:result is None 2024-04-20T21:24:53.232 INFO:tasks.ceph.mds.d:Started 2024-04-20T21:24:53.233 DEBUG:teuthology.parallel:result is None 2024-04-20T21:24:53.240 INFO:tasks.ceph.mds.b:Started 2024-04-20T21:24:53.240 DEBUG:teuthology.parallel:result is None 2024-04-20T21:24:53.240 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-20T21:24:53.373 INFO:tasks.ceph.mds.c.smithi086.stdout:starting mds.c at 2024-04-20T21:24:53.374 INFO:tasks.ceph.mds.a.smithi086.stdout:starting mds.a at 2024-04-20T21:24:53.387 INFO:tasks.ceph.mds.d.smithi144.stdout:starting mds.d at 2024-04-20T21:24:53.388 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2024-04-20T21:24:53.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.419+0000 7f0623e00640 1 Processor -- start 2024-04-20T21:24:53.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.419+0000 7f0623e00640 1 -- start start 2024-04-20T21:24:53.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.420+0000 7f0623e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06240d2cb0 0x7f06240d1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:53.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.420+0000 7f0623e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06240d3100 con 0x7f062415e690 2024-04-20T21:24:53.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.420+0000 7f0623e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06240d33b0 con 0x7f06240d0ee0 2024-04-20T21:24:53.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.420+0000 7f0623e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f06240d18b0 con 0x7f06240d2cb0 2024-04-20T21:24:53.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.421+0000 7f0622a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f062415e690 0x7f062406ce20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:50564/0 (socket says 172.21.15.86:50564) 2024-04-20T21:24:53.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.421+0000 7f0622a00640 1 -- 172.21.15.86:0/3763014098 learned_addr learned my addr 172.21.15.86:0/3763014098 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:24:53.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.421+0000 7f0623400640 1 --2- 172.21.15.86:0/3763014098 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06240d2cb0 0x7f06240d1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:53.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.421+0000 7f0622000640 1 -- 172.21.15.86:0/3763014098 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2923372523 0 0) 0x7f06240d3100 con 0x7f062415e690 2024-04-20T21:24:53.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.421+0000 7f0622000640 1 -- 172.21.15.86:0/3763014098 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0608003520 con 0x7f062415e690 2024-04-20T21:24:53.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.421+0000 7f0622000640 1 -- 172.21.15.86:0/3763014098 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2087474739 0 0) 0x7f06240d33b0 con 0x7f06240d0ee0 2024-04-20T21:24:53.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.421+0000 7f0622000640 1 -- 172.21.15.86:0/3763014098 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f06240d3100 con 0x7f06240d0ee0 2024-04-20T21:24:53.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.421+0000 7f0622000640 1 -- 172.21.15.86:0/3763014098 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1385335936 0 0) 0x7f0608003520 con 0x7f062415e690 2024-04-20T21:24:53.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.421+0000 7f0623400640 1 -- 172.21.15.86:0/3763014098 >> v1:172.21.15.144:6789/0 conn(0x7f06240d0ee0 legacy=0x7f06240d25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:53.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.421+0000 7f0623400640 1 -- 172.21.15.86:0/3763014098 >> v1:172.21.15.86:6789/0 conn(0x7f062415e690 legacy=0x7f062406ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:53.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.421+0000 7f0623400640 1 -- 172.21.15.86:0/3763014098 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06240d3690 con 0x7f06240d2cb0 2024-04-20T21:24:53.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.423+0000 7f0623400640 1 --2- 172.21.15.86:0/3763014098 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06240d2cb0 0x7f06240d1370 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f061c0097d0 tx=0x7f061c02b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8034954bf7d300c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:53.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.423+0000 7f0622000640 1 -- 172.21.15.86:0/3763014098 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c037070 con 0x7f06240d2cb0 2024-04-20T21:24:53.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.423+0000 7f0622000640 1 -- 172.21.15.86:0/3763014098 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f061c030510 con 0x7f06240d2cb0 2024-04-20T21:24:53.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.423+0000 7f0622000640 1 -- 172.21.15.86:0/3763014098 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0614001690 con 0x7f06240d2cb0 2024-04-20T21:24:53.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.423+0000 7f0623e00640 1 -- 172.21.15.86:0/3763014098 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06240d2cb0 msgr2=0x7f06240d1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:53.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.423+0000 7f0623e00640 1 --2- 172.21.15.86:0/3763014098 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06240d2cb0 0x7f06240d1370 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f061c0097d0 tx=0x7f061c02b800 comp rx=0 tx=0).stop 2024-04-20T21:24:53.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.424+0000 7f0623e00640 1 -- 172.21.15.86:0/3763014098 shutdown_connections 2024-04-20T21:24:53.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.424+0000 7f0623e00640 1 --2- 172.21.15.86:0/3763014098 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06240d2cb0 0x7f06240d1370 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:53.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.424+0000 7f0623e00640 1 -- 172.21.15.86:0/3763014098 >> 172.21.15.86:0/3763014098 conn(0x7f062409d8a0 msgr2=0x7f062408bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:53.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.424+0000 7f0623e00640 1 -- 172.21.15.86:0/3763014098 shutdown_connections 2024-04-20T21:24:53.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.424+0000 7f0623e00640 1 -- 172.21.15.86:0/3763014098 wait complete. 2024-04-20T21:24:53.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.425+0000 7f0623e00640 1 Processor -- start 2024-04-20T21:24:53.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.426+0000 7f0623e00640 1 -- start start 2024-04-20T21:24:53.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.426+0000 7f0623e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06240d0ee0 0x7f0624150310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:53.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.426+0000 7f0623e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06240d2cb0 0x7f06240b5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:53.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.427+0000 7f0623e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f062415e690 0x7f06240b3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:53.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.427+0000 7f0623e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f06240cef00 con 0x7f062415e690 2024-04-20T21:24:53.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.427+0000 7f0623e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f06240cec20 con 0x7f06240d0ee0 2024-04-20T21:24:53.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.427+0000 7f0623e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f062415ea70 con 0x7f06240d2cb0 2024-04-20T21:24:53.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.427+0000 7f0623400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f062415e690 0x7f06240b3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:53.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.427+0000 7f0623400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f062415e690 0x7f06240b3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42980/0 (socket says 172.21.15.86:42980) 2024-04-20T21:24:53.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.427+0000 7f0623400640 1 -- 172.21.15.86:0/3456723921 learned_addr learned my addr 172.21.15.86:0/3456723921 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:53.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.427+0000 7f0622a00640 1 --2- 172.21.15.86:0/3456723921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06240d0ee0 0x7f0624150310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:53.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.427+0000 7f061a000640 1 --2- 172.21.15.86:0/3456723921 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06240d2cb0 0x7f06240b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:53.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.427+0000 7f0623400640 1 -- 172.21.15.86:0/3456723921 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06240d2cb0 msgr2=0x7f06240b5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:53.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.427+0000 7f0623400640 1 --2- 172.21.15.86:0/3456723921 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06240d2cb0 0x7f06240b5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:53.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.427+0000 7f0623400640 1 -- 172.21.15.86:0/3456723921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06240d0ee0 msgr2=0x7f0624150310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:53.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.427+0000 7f0623400640 1 --2- 172.21.15.86:0/3456723921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06240d0ee0 0x7f0624150310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:53.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.427+0000 7f0623400640 1 -- 172.21.15.86:0/3456723921 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06240b5e80 con 0x7f062415e690 2024-04-20T21:24:53.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.428+0000 7f0623400640 1 --2- 172.21.15.86:0/3456723921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f062415e690 0x7f06240b3f90 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f061c0097d0 tx=0x7f061c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:53.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.428+0000 7f061b400640 1 -- 172.21.15.86:0/3456723921 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c042020 con 0x7f062415e690 2024-04-20T21:24:53.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.428+0000 7f061b400640 1 -- 172.21.15.86:0/3456723921 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f061c003610 con 0x7f062415e690 2024-04-20T21:24:53.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.428+0000 7f061b400640 1 -- 172.21.15.86:0/3456723921 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c037040 con 0x7f062415e690 2024-04-20T21:24:53.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.428+0000 7f0623e00640 1 -- 172.21.15.86:0/3456723921 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06240b48a0 con 0x7f062415e690 2024-04-20T21:24:53.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.428+0000 7f0623e00640 1 -- 172.21.15.86:0/3456723921 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06240b4e60 con 0x7f062415e690 2024-04-20T21:24:53.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.429+0000 7f0623e00640 1 -- 172.21.15.86:0/3456723921 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f062404fcf0 con 0x7f062415e690 2024-04-20T21:24:53.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.430+0000 7f061b400640 1 -- 172.21.15.86:0/3456723921 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f061c045020 con 0x7f062415e690 2024-04-20T21:24:53.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.431+0000 7f061b400640 1 --2- 172.21.15.86:0/3456723921 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f060006c8f0 0x7f060006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:53.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.431+0000 7f061b400640 1 -- 172.21.15.86:0/3456723921 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5029+0+0 (secure 0 0 0) 0x7f061c0ad630 con 0x7f062415e690 2024-04-20T21:24:53.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.431+0000 7f0622a00640 1 --2- 172.21.15.86:0/3456723921 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f060006c8f0 0x7f060006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:53.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.432+0000 7f0622a00640 1 --2- 172.21.15.86:0/3456723921 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f060006c8f0 0x7f060006edb0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f0614000c00 tx=0x7f0614001550 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:53.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.435+0000 7f061b400640 1 -- 172.21.15.86:0/3456723921 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f061c04c030 con 0x7f062415e690 2024-04-20T21:24:53.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.647+0000 7f0623e00640 1 -- 172.21.15.86:0/3456723921 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f062406cc00 con 0x7f062415e690 2024-04-20T21:24:53.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.782+0000 7f061b400640 1 -- 172.21.15.86:0/3456723921 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v79) ==== 97+0+0 (secure 0 0 0) 0x7f061c07bc60 con 0x7f062415e690 2024-04-20T21:24:53.782 INFO:teuthology.orchestra.run.smithi086.stderr: removed all blocklist entries 2024-04-20T21:24:53.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.785+0000 7f0623e00640 1 -- 172.21.15.86:0/3456723921 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f060006c8f0 msgr2=0x7f060006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:53.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.785+0000 7f0623e00640 1 --2- 172.21.15.86:0/3456723921 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f060006c8f0 0x7f060006edb0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f0614000c00 tx=0x7f0614001550 comp rx=0 tx=0).stop 2024-04-20T21:24:53.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.785+0000 7f0623e00640 1 -- 172.21.15.86:0/3456723921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f062415e690 msgr2=0x7f06240b3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:53.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.785+0000 7f0623e00640 1 --2- 172.21.15.86:0/3456723921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f062415e690 0x7f06240b3f90 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f061c0097d0 tx=0x7f061c0046c0 comp rx=0 tx=0).stop 2024-04-20T21:24:53.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.785+0000 7f0623e00640 1 -- 172.21.15.86:0/3456723921 shutdown_connections 2024-04-20T21:24:53.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.786+0000 7f0623e00640 1 --2- 172.21.15.86:0/3456723921 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f060006c8f0 0x7f060006edb0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:53.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.786+0000 7f0623e00640 1 --2- 172.21.15.86:0/3456723921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f062415e690 0x7f06240b3f90 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:53.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.786+0000 7f0623e00640 1 --2- 172.21.15.86:0/3456723921 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06240d2cb0 0x7f06240b5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:53.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.786+0000 7f0623e00640 1 --2- 172.21.15.86:0/3456723921 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06240d0ee0 0x7f0624150310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:53.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.786+0000 7f0623e00640 1 -- 172.21.15.86:0/3456723921 >> 172.21.15.86:0/3456723921 conn(0x7f062409d8a0 msgr2=0x7f062406df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:53.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.786+0000 7f0623e00640 1 -- 172.21.15.86:0/3456723921 shutdown_connections 2024-04-20T21:24:53.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.786+0000 7f0623e00640 1 -- 172.21.15.86:0/3456723921 wait complete. 2024-04-20T21:24:53.806 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-20T21:24:53.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.991+0000 7f0ebea00640 1 Processor -- start 2024-04-20T21:24:53.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.991+0000 7f0ebea00640 1 -- start start 2024-04-20T21:24:53.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.993+0000 7f0ebea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0eb8144d00 0x7f0eb806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:53.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.993+0000 7f0ebea00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0eb8071d40 0x7f0eb80721a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:53.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.993+0000 7f0ebea00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0eb806ef60 con 0x7f0eb815ee50 2024-04-20T21:24:53.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.993+0000 7f0ebea00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f0eb80726e0 con 0x7f0eb8144d00 2024-04-20T21:24:53.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.993+0000 7f0ebea00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f0eb806e590 con 0x7f0eb8071d40 2024-04-20T21:24:53.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.993+0000 7f0eb7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0eb8144d00 0x7f0eb806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:53.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.993+0000 7f0ebca00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f0eb815ee50 0x7f0eb815f2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:50576/0 (socket says 172.21.15.86:50576) 2024-04-20T21:24:53.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.993+0000 7f0ebca00640 1 -- 172.21.15.86:0/2069895502 learned_addr learned my addr 172.21.15.86:0/2069895502 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:24:53.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.993+0000 7f0eb7400640 1 --2- 172.21.15.86:0/2069895502 >> v2:172.21.15.144:3300/0 conn(0x7f0eb8071d40 0x7f0eb80721a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:53.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.994+0000 7f0eb7e00640 1 -- 172.21.15.86:0/2069895502 >> v2:172.21.15.144:3300/0 conn(0x7f0eb8071d40 msgr2=0x7f0eb80721a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:53.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.994+0000 7f0eb7e00640 1 --2- 172.21.15.86:0/2069895502 >> v2:172.21.15.144:3300/0 conn(0x7f0eb8071d40 0x7f0eb80721a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:53.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.994+0000 7f0eb7e00640 1 -- 172.21.15.86:0/2069895502 >> v1:172.21.15.86:6789/0 conn(0x7f0eb815ee50 legacy=0x7f0eb815f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:53.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.994+0000 7f0eb7e00640 1 -- 172.21.15.86:0/2069895502 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eb815fbb0 con 0x7f0eb8144d00 2024-04-20T21:24:53.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.994+0000 7f0eb6a00640 1 -- 172.21.15.86:0/2069895502 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1262040850 0 0) 0x7f0eb806ef60 con 0x7f0eb815ee50 2024-04-20T21:24:53.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.995+0000 7f0eb7e00640 1 --2- 172.21.15.86:0/2069895502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0eb8144d00 0x7f0eb806ea20 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8070140 tx=0x7f0ea802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ef0bd2e62674565 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:53.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.995+0000 7f0eb6a00640 1 -- 172.21.15.86:0/2069895502 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea8035020 con 0x7f0eb8144d00 2024-04-20T21:24:53.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.995+0000 7f0eb6a00640 1 -- 172.21.15.86:0/2069895502 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ea8031830 con 0x7f0eb8144d00 2024-04-20T21:24:53.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.995+0000 7f0eb6a00640 1 -- 172.21.15.86:0/2069895502 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea8031c00 con 0x7f0eb8144d00 2024-04-20T21:24:53.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.996+0000 7f0ebea00640 1 -- 172.21.15.86:0/2069895502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0eb8144d00 msgr2=0x7f0eb806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:53.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.996+0000 7f0ebea00640 1 --2- 172.21.15.86:0/2069895502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0eb8144d00 0x7f0eb806ea20 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8070140 tx=0x7f0ea802b8d0 comp rx=0 tx=0).stop 2024-04-20T21:24:53.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.996+0000 7f0ebea00640 1 -- 172.21.15.86:0/2069895502 shutdown_connections 2024-04-20T21:24:53.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.996+0000 7f0ebea00640 1 --2- 172.21.15.86:0/2069895502 >> v2:172.21.15.144:3300/0 conn(0x7f0eb8071d40 0x7f0eb80721a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:53.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.996+0000 7f0ebea00640 1 --2- 172.21.15.86:0/2069895502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0eb8144d00 0x7f0eb806ea20 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:53.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.996+0000 7f0ebea00640 1 -- 172.21.15.86:0/2069895502 >> 172.21.15.86:0/2069895502 conn(0x7f0eb806b980 msgr2=0x7f0eb806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:53.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.996+0000 7f0ebea00640 1 -- 172.21.15.86:0/2069895502 shutdown_connections 2024-04-20T21:24:53.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.996+0000 7f0ebea00640 1 -- 172.21.15.86:0/2069895502 wait complete. 2024-04-20T21:24:53.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.998+0000 7f0ebea00640 1 Processor -- start 2024-04-20T21:24:53.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.998+0000 7f0ebea00640 1 -- start start 2024-04-20T21:24:54.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.999+0000 7f0ebea00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0eb8071d40 0x7f0eb813fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:54.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.999+0000 7f0ebea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0eb8144d00 0x7f0eb813b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:54.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.999+0000 7f0ebea00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0eb815ee50 0x7f0eb814f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:54.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.999+0000 7f0ebea00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb81450e0 con 0x7f0eb8144d00 2024-04-20T21:24:54.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.999+0000 7f0ebea00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb806e220 con 0x7f0eb815ee50 2024-04-20T21:24:54.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.999+0000 7f0ebea00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0eb8145260 con 0x7f0eb8071d40 2024-04-20T21:24:54.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.999+0000 7f0eb7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0eb8144d00 0x7f0eb813b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:54.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.999+0000 7f0eb7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0eb8144d00 0x7f0eb813b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42994/0 (socket says 172.21.15.86:42994) 2024-04-20T21:24:54.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:53.999+0000 7f0eb7400640 1 -- 172.21.15.86:0/3026927153 learned_addr learned my addr 172.21.15.86:0/3026927153 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:54.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.000+0000 7f0eb7e00640 1 --2- 172.21.15.86:0/3026927153 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0eb8071d40 0x7f0eb813fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:54.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.000+0000 7f0ebca00640 1 --2- 172.21.15.86:0/3026927153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0eb815ee50 0x7f0eb814f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:54.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.000+0000 7f0eb7400640 1 -- 172.21.15.86:0/3026927153 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0eb8071d40 msgr2=0x7f0eb813fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:54.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.000+0000 7f0eb7400640 1 --2- 172.21.15.86:0/3026927153 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0eb8071d40 0x7f0eb813fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:54.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.000+0000 7f0eb7400640 1 -- 172.21.15.86:0/3026927153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0eb815ee50 msgr2=0x7f0eb814f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:54.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.000+0000 7f0eb7400640 1 --2- 172.21.15.86:0/3026927153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0eb815ee50 0x7f0eb814f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:54.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.000+0000 7f0eb7400640 1 -- 172.21.15.86:0/3026927153 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eb814e3c0 con 0x7f0eb8144d00 2024-04-20T21:24:54.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.000+0000 7f0eb7400640 1 --2- 172.21.15.86:0/3026927153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0eb8144d00 0x7f0eb813b780 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f0ea400d800 tx=0x7f0ea400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:54.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.000+0000 7f0eafe00640 1 -- 172.21.15.86:0/3026927153 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea400b6c0 con 0x7f0eb8144d00 2024-04-20T21:24:54.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.000+0000 7f0eafe00640 1 -- 172.21.15.86:0/3026927153 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ea4019070 con 0x7f0eb8144d00 2024-04-20T21:24:54.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.000+0000 7f0eafe00640 1 -- 172.21.15.86:0/3026927153 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea400ba80 con 0x7f0eb8144d00 2024-04-20T21:24:54.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.000+0000 7f0ebea00640 1 -- 172.21.15.86:0/3026927153 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0eb814e6a0 con 0x7f0eb8144d00 2024-04-20T21:24:54.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.001+0000 7f0ebea00640 1 -- 172.21.15.86:0/3026927153 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0eb814ec60 con 0x7f0eb8144d00 2024-04-20T21:24:54.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.002+0000 7f0ebea00640 1 -- 172.21.15.86:0/3026927153 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0eb80689e0 con 0x7f0eb8144d00 2024-04-20T21:24:54.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.003+0000 7f0eafe00640 1 -- 172.21.15.86:0/3026927153 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0ea400bc20 con 0x7f0eb8144d00 2024-04-20T21:24:54.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.004+0000 7f0eafe00640 1 --2- 172.21.15.86:0/3026927153 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0e9806c8f0 0x7f0e9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:54.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.004+0000 7f0eafe00640 1 -- 172.21.15.86:0/3026927153 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4857+0+0 (secure 0 0 0) 0x7f0ea408bd50 con 0x7f0eb8144d00 2024-04-20T21:24:54.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.004+0000 7f0eb7e00640 1 --2- 172.21.15.86:0/3026927153 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0e9806c8f0 0x7f0e9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:54.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.005+0000 7f0eb7e00640 1 --2- 172.21.15.86:0/3026927153 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0e9806c8f0 0x7f0e9806edb0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8036000 tx=0x7f0ea8008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:54.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.009+0000 7f0eafe00640 1 -- 172.21.15.86:0/3026927153 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0ea401c360 con 0x7f0eb8144d00 2024-04-20T21:24:54.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.263+0000 7f0ebea00640 1 -- 172.21.15.86:0/3026927153 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f0eb81455f0 con 0x7f0eb8144d00 2024-04-20T21:24:54.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.264+0000 7f0eafe00640 1 -- 172.21.15.86:0/3026927153 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) ==== 80+0+5304 (secure 0 0 0) 0x7f0ea405a470 con 0x7f0eb8144d00 2024-04-20T21:24:54.264 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:24:54.264 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:24:54.264 INFO:teuthology.orchestra.run.smithi086.stdout: "auth_dump": [ 2024-04-20T21:24:54.264 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:24:54.264 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.a", 2024-04-20T21:24:54.264 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDAMCRmOpS/LRAAPZUUQNf6TyeJ2tF7YPnIKw==", 2024-04-20T21:24:54.264 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:24:54.264 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:24:54.264 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-20T21:24:54.264 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-20T21:24:54.264 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:24:54.264 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:24:54.265 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:24:54.265 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:24:54.265 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.b", 2024-04-20T21:24:54.265 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDBMCRmCJLpDRAA/JiLRSAlQRNMpDC+qDSr0g==", 2024-04-20T21:24:54.265 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:24:54.265 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:24:54.265 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-20T21:24:54.265 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-20T21:24:54.265 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:24:54.265 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:24:54.265 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:24:54.265 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:24:54.265 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.c", 2024-04-20T21:24:54.265 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDAMCRmPHOFOxAA4tUsw/n6oSKrNmzPD+NKyw==", 2024-04-20T21:24:54.266 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:24:54.266 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:24:54.266 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-20T21:24:54.266 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-20T21:24:54.266 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:24:54.266 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:24:54.266 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:24:54.266 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:24:54.266 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.d", 2024-04-20T21:24:54.266 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDBMCRmZnPXHBAA0jXwBmGehaO7qens4boRIQ==", 2024-04-20T21:24:54.266 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:24:54.266 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:24:54.266 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-20T21:24:54.266 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-20T21:24:54.267 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:24:54.267 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:24:54.267 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:24:54.267 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:24:54.267 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.0", 2024-04-20T21:24:54.267 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDFMCRm4w9UHhAA56y4dJyDASI+IoZv2XrEoQ==", 2024-04-20T21:24:54.267 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:24:54.267 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:24:54.267 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:24:54.267 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:24:54.267 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:24:54.267 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:24:54.267 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:24:54.267 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.1", 2024-04-20T21:24:54.268 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDIMCRmFo7xKRAAou5B04+J1gZtTzW/Mo+nRQ==", 2024-04-20T21:24:54.268 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:24:54.268 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:24:54.268 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:24:54.268 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:24:54.268 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:24:54.268 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:24:54.268 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:24:54.268 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.2", 2024-04-20T21:24:54.268 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDLMCRm7/jFNRAA4KcRsapPFLaKf0NaH5MApg==", 2024-04-20T21:24:54.268 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:24:54.268 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:24:54.268 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:24:54.268 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:24:54.268 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:24:54.269 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:24:54.269 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:24:54.269 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.3", 2024-04-20T21:24:54.269 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDPMCRmUTPMExAA0/dubb4ehsWIwZPS/ao68w==", 2024-04-20T21:24:54.269 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:24:54.269 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:24:54.269 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:24:54.269 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:24:54.269 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:24:54.269 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:24:54.269 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:24:54.269 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.4", 2024-04-20T21:24:54.269 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDVMCRmB4F2KRAA8Xj0DfH5Ou8TUrO8X/BQyw==", 2024-04-20T21:24:54.269 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:24:54.270 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:24:54.270 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:24:54.270 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:24:54.270 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:24:54.270 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:24:54.270 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:24:54.270 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.5", 2024-04-20T21:24:54.270 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDYMCRmSOvYNRAAODYXmRHMGPrCAxcHwlS7Ww==", 2024-04-20T21:24:54.270 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:24:54.270 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:24:54.270 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:24:54.270 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:24:54.270 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:24:54.270 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:24:54.271 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:24:54.271 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.6", 2024-04-20T21:24:54.271 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDcMCRmENJ9BBAAi/KBUhbYGNXEyiSjSHavWQ==", 2024-04-20T21:24:54.271 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:24:54.271 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:24:54.271 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:24:54.271 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:24:54.271 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:24:54.271 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:24:54.271 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:24:54.271 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.7", 2024-04-20T21:24:54.271 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDfMCRmfTHTDhAALL9GZBSGjWYrhgFw/YI6IA==", 2024-04-20T21:24:54.271 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:24:54.271 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:24:54.271 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:24:54.272 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:24:54.272 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:24:54.272 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:24:54.272 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:24:54.272 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.0", 2024-04-20T21:24:54.272 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDBMCRmHWxyKRAANzhRi4eajp68h/jWkhFMnw==", 2024-04-20T21:24:54.272 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:24:54.272 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:24:54.272 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow r", 2024-04-20T21:24:54.272 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-20T21:24:54.272 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:24:54.272 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:24:54.272 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:24:54.272 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.1", 2024-04-20T21:24:54.273 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDBMCRmgBJAOxAADJLzO97tjEpKn6kUs8g2QA==", 2024-04-20T21:24:54.273 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:24:54.273 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:24:54.273 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow r", 2024-04-20T21:24:54.273 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow rw", 2024-04-20T21:24:54.273 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rwx" 2024-04-20T21:24:54.273 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:24:54.273 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:24:54.273 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:24:54.273 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.2", 2024-04-20T21:24:54.273 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDBMCRmQAdJMhAAiUmKhPEeuV+UPqQ3gynG9w==", 2024-04-20T21:24:54.273 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:24:54.273 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:24:54.273 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow r", 2024-04-20T21:24:54.273 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow rw", 2024-04-20T21:24:54.274 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rwx" 2024-04-20T21:24:54.274 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:24:54.274 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:24:54.274 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:24:54.274 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.3", 2024-04-20T21:24:54.274 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDCMCRm8yMSCBAAFgSVW0XYu7fa40YxHITljQ==", 2024-04-20T21:24:54.274 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:24:54.274 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:24:54.274 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow r", 2024-04-20T21:24:54.274 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow rw", 2024-04-20T21:24:54.274 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rwx" 2024-04-20T21:24:54.274 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:24:54.274 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:24:54.274 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:24:54.275 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.admin", 2024-04-20T21:24:54.275 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQC/MCRmyru0KBAApSqv1hZRcLvrRwPMnqylqQ==", 2024-04-20T21:24:54.275 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:24:54.275 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-04-20T21:24:54.275 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-20T21:24:54.275 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-20T21:24:54.275 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:24:54.275 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:24:54.275 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:24:54.275 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:24:54.275 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mgr.x", 2024-04-20T21:24:54.275 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDAMCRmvTgjIxAACeSV8gNkL5LV3xzPS6VPeg==", 2024-04-20T21:24:54.275 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:24:54.275 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-04-20T21:24:54.276 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile mgr", 2024-04-20T21:24:54.276 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:24:54.276 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:24:54.276 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:24:54.276 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:24:54.276 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mgr.y", 2024-04-20T21:24:54.276 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDAMCRmSn3VGBAA0eu3R53f3gFgscWFMRkDzg==", 2024-04-20T21:24:54.276 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:24:54.276 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-04-20T21:24:54.276 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile mgr", 2024-04-20T21:24:54.276 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:24:54.276 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:24:54.276 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:24:54.276 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-20T21:24:54.276 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:24:54.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.266+0000 7f0ebea00640 1 -- 172.21.15.86:0/3026927153 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0e9806c8f0 msgr2=0x7f0e9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:54.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.266+0000 7f0ebea00640 1 --2- 172.21.15.86:0/3026927153 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0e9806c8f0 0x7f0e9806edb0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8036000 tx=0x7f0ea8008030 comp rx=0 tx=0).stop 2024-04-20T21:24:54.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.266+0000 7f0ebea00640 1 -- 172.21.15.86:0/3026927153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0eb8144d00 msgr2=0x7f0eb813b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:54.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.266+0000 7f0ebea00640 1 --2- 172.21.15.86:0/3026927153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0eb8144d00 0x7f0eb813b780 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f0ea400d800 tx=0x7f0ea400dcc0 comp rx=0 tx=0).stop 2024-04-20T21:24:54.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.266+0000 7f0ebea00640 1 -- 172.21.15.86:0/3026927153 shutdown_connections 2024-04-20T21:24:54.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.266+0000 7f0ebea00640 1 --2- 172.21.15.86:0/3026927153 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0e9806c8f0 0x7f0e9806edb0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:54.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.266+0000 7f0ebea00640 1 --2- 172.21.15.86:0/3026927153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0eb815ee50 0x7f0eb814f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:54.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.266+0000 7f0ebea00640 1 --2- 172.21.15.86:0/3026927153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0eb8144d00 0x7f0eb813b780 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:54.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.266+0000 7f0ebea00640 1 --2- 172.21.15.86:0/3026927153 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0eb8071d40 0x7f0eb813fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:54.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.266+0000 7f0ebea00640 1 -- 172.21.15.86:0/3026927153 >> 172.21.15.86:0/3026927153 conn(0x7f0eb806b980 msgr2=0x7f0eb806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:54.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.267+0000 7f0ebea00640 1 -- 172.21.15.86:0/3026927153 shutdown_connections 2024-04-20T21:24:54.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.267+0000 7f0ebea00640 1 -- 172.21.15.86:0/3026927153 wait complete. 2024-04-20T21:24:54.284 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:24:54.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.461+0000 7f725e800640 1 Processor -- start 2024-04-20T21:24:54.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.462+0000 7f725e800640 1 -- start start 2024-04-20T21:24:54.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.463+0000 7f725e800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f725815c010 0x7f725815c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:54.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.463+0000 7f725e800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72580af7b0 0x7f72580adda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:54.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.463+0000 7f725e800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72580ae710 con 0x7f725806f4c0 2024-04-20T21:24:54.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.463+0000 7f725e800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f72580af320 con 0x7f725815c010 2024-04-20T21:24:54.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.463+0000 7f725e800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f725815c930 con 0x7f72580af7b0 2024-04-20T21:24:54.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.463+0000 7f725d400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f725815c010 0x7f725815c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:54.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.463+0000 7f725d400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f725815c010 0x7f725815c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42998/0 (socket says 172.21.15.86:42998) 2024-04-20T21:24:54.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.463+0000 7f725d400640 1 -- 172.21.15.86:0/3441792428 learned_addr learned my addr 172.21.15.86:0/3441792428 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:54.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.464+0000 7f725ca00640 1 --2- 172.21.15.86:0/3441792428 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72580af7b0 0x7f72580adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:54.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.464+0000 7f725d400640 1 -- 172.21.15.86:0/3441792428 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72580af7b0 msgr2=0x7f72580adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:54.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.464+0000 7f725d400640 1 --2- 172.21.15.86:0/3441792428 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72580af7b0 0x7f72580adda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:54.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.464+0000 7f725d400640 1 -- 172.21.15.86:0/3441792428 >> v1:172.21.15.86:6789/0 conn(0x7f725806f4c0 legacy=0x7f725815ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:54.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.464+0000 7f725d400640 1 -- 172.21.15.86:0/3441792428 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72580ae9f0 con 0x7f725815c010 2024-04-20T21:24:54.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.464+0000 7f725ca00640 1 --2- 172.21.15.86:0/3441792428 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72580af7b0 0x7f72580adda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:24:54.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.465+0000 7f725d400640 1 --2- 172.21.15.86:0/3441792428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f725815c010 0x7f725815c3f0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f724c00bb10 tx=0x7f724c02e160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb8020e1c3e0fb71 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:54.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.465+0000 7f7257e00640 1 -- 172.21.15.86:0/3441792428 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724c03b070 con 0x7f725815c010 2024-04-20T21:24:54.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.465+0000 7f7257e00640 1 -- 172.21.15.86:0/3441792428 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f724c036440 con 0x7f725815c010 2024-04-20T21:24:54.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.465+0000 7f7257e00640 1 -- 172.21.15.86:0/3441792428 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724c036810 con 0x7f725815c010 2024-04-20T21:24:54.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.466+0000 7f725e800640 1 -- 172.21.15.86:0/3441792428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f725815c010 msgr2=0x7f725815c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:54.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.466+0000 7f725e800640 1 --2- 172.21.15.86:0/3441792428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f725815c010 0x7f725815c3f0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f724c00bb10 tx=0x7f724c02e160 comp rx=0 tx=0).stop 2024-04-20T21:24:54.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.466+0000 7f725e800640 1 -- 172.21.15.86:0/3441792428 shutdown_connections 2024-04-20T21:24:54.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.466+0000 7f725e800640 1 --2- 172.21.15.86:0/3441792428 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72580af7b0 0x7f72580adda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:54.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.466+0000 7f725e800640 1 --2- 172.21.15.86:0/3441792428 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f725815c010 0x7f725815c3f0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:54.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.466+0000 7f725e800640 1 -- 172.21.15.86:0/3441792428 >> 172.21.15.86:0/3441792428 conn(0x7f72580d0170 msgr2=0x7f72580d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:54.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.466+0000 7f725e800640 1 -- 172.21.15.86:0/3441792428 shutdown_connections 2024-04-20T21:24:54.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.466+0000 7f725e800640 1 -- 172.21.15.86:0/3441792428 wait complete. 2024-04-20T21:24:54.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.467+0000 7f725e800640 1 Processor -- start 2024-04-20T21:24:54.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.467+0000 7f725e800640 1 -- start start 2024-04-20T21:24:54.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.468+0000 7f725e800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f725806f4c0 0x7f7258099ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:54.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.468+0000 7f725e800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72580af7b0 0x7f72581587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:54.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.468+0000 7f725e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f725815c010 0x7f7258152b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:54.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.468+0000 7f725e800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f72580ae2e0 con 0x7f725815c010 2024-04-20T21:24:54.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.468+0000 7f725e800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f72580aefb0 con 0x7f725806f4c0 2024-04-20T21:24:54.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.468+0000 7f725e800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f72580ae460 con 0x7f72580af7b0 2024-04-20T21:24:54.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.469+0000 7f725de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f725815c010 0x7f7258152b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:54.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.469+0000 7f725de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f725815c010 0x7f7258152b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43014/0 (socket says 172.21.15.86:43014) 2024-04-20T21:24:54.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.469+0000 7f725de00640 1 -- 172.21.15.86:0/377328246 learned_addr learned my addr 172.21.15.86:0/377328246 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:54.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.469+0000 7f725de00640 1 -- 172.21.15.86:0/377328246 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72580af7b0 msgr2=0x7f72581587a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:54.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.469+0000 7f725de00640 1 --2- 172.21.15.86:0/377328246 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72580af7b0 0x7f72581587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:54.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.469+0000 7f725de00640 1 -- 172.21.15.86:0/377328246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f725806f4c0 msgr2=0x7f7258099ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:54.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.469+0000 7f725de00640 1 --2- 172.21.15.86:0/377328246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f725806f4c0 0x7f7258099ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:54.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.469+0000 7f725de00640 1 -- 172.21.15.86:0/377328246 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7258159140 con 0x7f725815c010 2024-04-20T21:24:54.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.469+0000 7f725de00640 1 --2- 172.21.15.86:0/377328246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f725815c010 0x7f7258152b70 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f725000c7a0 tx=0x7f725000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:54.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.470+0000 7f7255600640 1 -- 172.21.15.86:0/377328246 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7250003b70 con 0x7f725815c010 2024-04-20T21:24:54.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.470+0000 7f7255600640 1 -- 172.21.15.86:0/377328246 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7250003d30 con 0x7f725815c010 2024-04-20T21:24:54.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.470+0000 7f7255600640 1 -- 172.21.15.86:0/377328246 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725000aa90 con 0x7f725815c010 2024-04-20T21:24:54.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.470+0000 7f725e800640 1 -- 172.21.15.86:0/377328246 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7258159420 con 0x7f725815c010 2024-04-20T21:24:54.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.470+0000 7f725e800640 1 -- 172.21.15.86:0/377328246 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72581599e0 con 0x7f725815c010 2024-04-20T21:24:54.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.471+0000 7f725e800640 1 -- 172.21.15.86:0/377328246 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f725804fcf0 con 0x7f725815c010 2024-04-20T21:24:54.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.472+0000 7f7255600640 1 -- 172.21.15.86:0/377328246 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f725000ac30 con 0x7f725815c010 2024-04-20T21:24:54.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.473+0000 7f7255600640 1 --2- 172.21.15.86:0/377328246 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f723c06c920 0x7f723c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:54.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.473+0000 7f7255600640 1 -- 172.21.15.86:0/377328246 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4857+0+0 (secure 0 0 0) 0x7f725008b830 con 0x7f725815c010 2024-04-20T21:24:54.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.473+0000 7f725d400640 1 --2- 172.21.15.86:0/377328246 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f723c06c920 0x7f723c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:54.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.474+0000 7f725d400640 1 --2- 172.21.15.86:0/377328246 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f723c06c920 0x7f723c06ede0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f724c038000 tx=0x7f724c030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:54.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.477+0000 7f7255600640 1 -- 172.21.15.86:0/377328246 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7250059e80 con 0x7f725815c010 2024-04-20T21:24:54.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.663+0000 7f725e800640 1 -- 172.21.15.86:0/377328246 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7258152950 con 0x7f725815c010 2024-04-20T21:24:54.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.665+0000 7f7255600640 1 -- 172.21.15.86:0/377328246 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+12892 (secure 0 0 0) 0x7f7250059780 con 0x7f725815c010 2024-04-20T21:24:54.665 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:24:54.665 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":81,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:24:53.800154+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:24:54.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.667+0000 7f725e800640 1 -- 172.21.15.86:0/377328246 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f723c06c920 msgr2=0x7f723c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:54.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.667+0000 7f725e800640 1 --2- 172.21.15.86:0/377328246 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f723c06c920 0x7f723c06ede0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f724c038000 tx=0x7f724c030000 comp rx=0 tx=0).stop 2024-04-20T21:24:54.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.667+0000 7f725e800640 1 -- 172.21.15.86:0/377328246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f725815c010 msgr2=0x7f7258152b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:54.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.667+0000 7f725e800640 1 --2- 172.21.15.86:0/377328246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f725815c010 0x7f7258152b70 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f725000c7a0 tx=0x7f725000cc60 comp rx=0 tx=0).stop 2024-04-20T21:24:54.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.668+0000 7f725e800640 1 -- 172.21.15.86:0/377328246 shutdown_connections 2024-04-20T21:24:54.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.668+0000 7f725e800640 1 --2- 172.21.15.86:0/377328246 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f723c06c920 0x7f723c06ede0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:54.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.668+0000 7f725e800640 1 --2- 172.21.15.86:0/377328246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f725815c010 0x7f7258152b70 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:54.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.668+0000 7f725e800640 1 --2- 172.21.15.86:0/377328246 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f72580af7b0 0x7f72581587a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:54.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.668+0000 7f725e800640 1 --2- 172.21.15.86:0/377328246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f725806f4c0 0x7f7258099ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:54.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.668+0000 7f725e800640 1 -- 172.21.15.86:0/377328246 >> 172.21.15.86:0/377328246 conn(0x7f72580d0170 msgr2=0x7f725815d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:54.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.668+0000 7f725e800640 1 -- 172.21.15.86:0/377328246 shutdown_connections 2024-04-20T21:24:54.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.668+0000 7f725e800640 1 -- 172.21.15.86:0/377328246 wait complete. 2024-04-20T21:24:54.686 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:24:54.686 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:24:54.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.880+0000 7fa7c7e00640 1 Processor -- start 2024-04-20T21:24:54.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.880+0000 7fa7c7e00640 1 -- start start 2024-04-20T21:24:54.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.882+0000 7fa7c7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa7c806ef20 0x7fa7c815de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:54.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.882+0000 7fa7c7e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7c806de80 con 0x7fa7c806d9c0 2024-04-20T21:24:54.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.882+0000 7fa7c7e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7c806e160 con 0x7fa7c80af1f0 2024-04-20T21:24:54.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.882+0000 7fa7c7e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa7c806e440 con 0x7fa7c806ef20 2024-04-20T21:24:54.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.882+0000 7fa7c7400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa7c806ef20 0x7fa7c815de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:54.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.882+0000 7fa7c7400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa7c806ef20 0x7fa7c815de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43020/0 (socket says 172.21.15.86:43020) 2024-04-20T21:24:54.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.882+0000 7fa7be000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fa7c806d9c0 0x7fa7c80af5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:50594/0 (socket says 172.21.15.86:50594) 2024-04-20T21:24:54.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.882+0000 7fa7c7400640 1 -- 172.21.15.86:0/2793169624 learned_addr learned my addr 172.21.15.86:0/2793169624 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:54.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.883+0000 7fa7c7400640 1 -- 172.21.15.86:0/2793169624 >> v1:172.21.15.144:6789/0 conn(0x7fa7c80af1f0 legacy=0x7fa7c815d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:54.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.883+0000 7fa7c7400640 1 -- 172.21.15.86:0/2793169624 >> v1:172.21.15.86:6789/0 conn(0x7fa7c806d9c0 legacy=0x7fa7c80af5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:54.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.883+0000 7fa7c7400640 1 -- 172.21.15.86:0/2793169624 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7c806e720 con 0x7fa7c806ef20 2024-04-20T21:24:54.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.883+0000 7fa7c6000640 1 -- 172.21.15.86:0/2793169624 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3793764235 0 0) 0x7fa7c806de80 con 0x7fa7c806d9c0 2024-04-20T21:24:54.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.884+0000 7fa7c7400640 1 --2- 172.21.15.86:0/2793169624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7c806ef20 0x7fa7c815de70 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fa7c0009870 tx=0x7fa7c002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edc92d91ae7c669a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:54.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.884+0000 7fa7c6000640 1 -- 172.21.15.86:0/2793169624 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0034070 con 0x7fa7c806ef20 2024-04-20T21:24:54.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.884+0000 7fa7c6000640 1 -- 172.21.15.86:0/2793169624 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7c0030860 con 0x7fa7c806ef20 2024-04-20T21:24:54.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.884+0000 7fa7c6000640 1 -- 172.21.15.86:0/2793169624 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0030c30 con 0x7fa7c806ef20 2024-04-20T21:24:54.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.885+0000 7fa7c7e00640 1 -- 172.21.15.86:0/2793169624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7c806ef20 msgr2=0x7fa7c815de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:54.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.885+0000 7fa7c7e00640 1 --2- 172.21.15.86:0/2793169624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7c806ef20 0x7fa7c815de70 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fa7c0009870 tx=0x7fa7c002b9a0 comp rx=0 tx=0).stop 2024-04-20T21:24:54.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.885+0000 7fa7c7e00640 1 -- 172.21.15.86:0/2793169624 shutdown_connections 2024-04-20T21:24:54.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.885+0000 7fa7c7e00640 1 --2- 172.21.15.86:0/2793169624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7c806ef20 0x7fa7c815de70 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:54.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.885+0000 7fa7c7e00640 1 -- 172.21.15.86:0/2793169624 >> 172.21.15.86:0/2793169624 conn(0x7fa7c80b1f00 msgr2=0x7fa7c80b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:54.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.885+0000 7fa7c7e00640 1 -- 172.21.15.86:0/2793169624 shutdown_connections 2024-04-20T21:24:54.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.885+0000 7fa7c7e00640 1 -- 172.21.15.86:0/2793169624 wait complete. 2024-04-20T21:24:54.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.886+0000 7fa7c7e00640 1 Processor -- start 2024-04-20T21:24:54.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.887+0000 7fa7c7e00640 1 -- start start 2024-04-20T21:24:54.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.887+0000 7fa7c7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7c806d9c0 0x7fa7c8159830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:54.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.888+0000 7fa7c7e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7c806ef20 0x7fa7c81534a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:54.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.888+0000 7fa7c7e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7c80af1f0 0x7fa7c8099470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:54.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.888+0000 7fa7c7e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c815a3c0 con 0x7fa7c806d9c0 2024-04-20T21:24:54.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.888+0000 7fa7c7e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c806f390 con 0x7fa7c806ef20 2024-04-20T21:24:54.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.888+0000 7fa7c7e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa7c815a540 con 0x7fa7c80af1f0 2024-04-20T21:24:54.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.888+0000 7fa7c6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7c806d9c0 0x7fa7c8159830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:54.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.888+0000 7fa7c6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7c806d9c0 0x7fa7c8159830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43030/0 (socket says 172.21.15.86:43030) 2024-04-20T21:24:54.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.888+0000 7fa7c6a00640 1 -- 172.21.15.86:0/1015108597 learned_addr learned my addr 172.21.15.86:0/1015108597 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:54.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.888+0000 7fa7c6a00640 1 -- 172.21.15.86:0/1015108597 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7c80af1f0 msgr2=0x7fa7c8099470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:54.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.888+0000 7fa7c6a00640 1 --2- 172.21.15.86:0/1015108597 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7c80af1f0 0x7fa7c8099470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:54.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.888+0000 7fa7c6a00640 1 -- 172.21.15.86:0/1015108597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7c806ef20 msgr2=0x7fa7c81534a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:54.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.888+0000 7fa7c6a00640 1 --2- 172.21.15.86:0/1015108597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7c806ef20 0x7fa7c81534a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:54.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.888+0000 7fa7c6a00640 1 -- 172.21.15.86:0/1015108597 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7c8158790 con 0x7fa7c806d9c0 2024-04-20T21:24:54.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.889+0000 7fa7c6a00640 1 --2- 172.21.15.86:0/1015108597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7c806d9c0 0x7fa7c8159830 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fa7b800b3f0 tx=0x7fa7b800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:54.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.889+0000 7fa7bfe00640 1 -- 172.21.15.86:0/1015108597 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b8015020 con 0x7fa7c806d9c0 2024-04-20T21:24:54.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.889+0000 7fa7bfe00640 1 -- 172.21.15.86:0/1015108597 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7b8011830 con 0x7fa7c806d9c0 2024-04-20T21:24:54.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.889+0000 7fa7c7e00640 1 -- 172.21.15.86:0/1015108597 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7c8158a70 con 0x7fa7c806d9c0 2024-04-20T21:24:54.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.889+0000 7fa7bfe00640 1 -- 172.21.15.86:0/1015108597 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b8011c20 con 0x7fa7c806d9c0 2024-04-20T21:24:54.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.889+0000 7fa7c7e00640 1 -- 172.21.15.86:0/1015108597 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7c8159030 con 0x7fa7c806d9c0 2024-04-20T21:24:54.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.890+0000 7fa7c7e00640 1 -- 172.21.15.86:0/1015108597 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7c804fd90 con 0x7fa7c806d9c0 2024-04-20T21:24:54.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.891+0000 7fa7bfe00640 1 -- 172.21.15.86:0/1015108597 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa7b8018020 con 0x7fa7c806d9c0 2024-04-20T21:24:54.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.892+0000 7fa7bfe00640 1 --2- 172.21.15.86:0/1015108597 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa7a406c8f0 0x7fa7a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:54.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.892+0000 7fa7bfe00640 1 -- 172.21.15.86:0/1015108597 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4857+0+0 (secure 0 0 0) 0x7fa7b808b470 con 0x7fa7c806d9c0 2024-04-20T21:24:54.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.892+0000 7fa7be000640 1 --2- 172.21.15.86:0/1015108597 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa7a406c8f0 0x7fa7a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:54.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.893+0000 7fa7be000640 1 --2- 172.21.15.86:0/1015108597 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa7a406c8f0 0x7fa7a406edb0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0001560 tx=0x7fa7b0007ca0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:54.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:54.896+0000 7fa7bfe00640 1 -- 172.21.15.86:0/1015108597 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa7b801f030 con 0x7fa7c806d9c0 2024-04-20T21:24:55.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.079+0000 7fa7c7e00640 1 -- 172.21.15.86:0/1015108597 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa7c80b2580 con 0x7fa7c806d9c0 2024-04-20T21:24:55.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.080+0000 7fa7bfe00640 1 -- 172.21.15.86:0/1015108597 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7fa7b80172c0 con 0x7fa7c806d9c0 2024-04-20T21:24:55.080 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:24:55.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.082+0000 7fa7c7e00640 1 -- 172.21.15.86:0/1015108597 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa7a406c8f0 msgr2=0x7fa7a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:55.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.082+0000 7fa7c7e00640 1 --2- 172.21.15.86:0/1015108597 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa7a406c8f0 0x7fa7a406edb0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0001560 tx=0x7fa7b0007ca0 comp rx=0 tx=0).stop 2024-04-20T21:24:55.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.082+0000 7fa7c7e00640 1 -- 172.21.15.86:0/1015108597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7c806d9c0 msgr2=0x7fa7c8159830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:55.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.082+0000 7fa7c7e00640 1 --2- 172.21.15.86:0/1015108597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7c806d9c0 0x7fa7c8159830 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fa7b800b3f0 tx=0x7fa7b800b8b0 comp rx=0 tx=0).stop 2024-04-20T21:24:55.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.082+0000 7fa7c7e00640 1 -- 172.21.15.86:0/1015108597 shutdown_connections 2024-04-20T21:24:55.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.082+0000 7fa7c7e00640 1 --2- 172.21.15.86:0/1015108597 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa7a406c8f0 0x7fa7a406edb0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:55.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.082+0000 7fa7c7e00640 1 --2- 172.21.15.86:0/1015108597 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7c80af1f0 0x7fa7c8099470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:55.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.082+0000 7fa7c7e00640 1 --2- 172.21.15.86:0/1015108597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7c806ef20 0x7fa7c81534a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:55.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.082+0000 7fa7c7e00640 1 --2- 172.21.15.86:0/1015108597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7c806d9c0 0x7fa7c8159830 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:55.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.082+0000 7fa7c7e00640 1 -- 172.21.15.86:0/1015108597 >> 172.21.15.86:0/1015108597 conn(0x7fa7c80b1f00 msgr2=0x7fa7c8154dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:55.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.083+0000 7fa7c7e00640 1 -- 172.21.15.86:0/1015108597 shutdown_connections 2024-04-20T21:24:55.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.083+0000 7fa7c7e00640 1 -- 172.21.15.86:0/1015108597 wait complete. 2024-04-20T21:24:55.100 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-20T21:24:55.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.290+0000 7f6169000640 1 Processor -- start 2024-04-20T21:24:55.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.290+0000 7f6169000640 1 -- start start 2024-04-20T21:24:55.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.292+0000 7f6169000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6164162c40 0x7f6164141f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:55.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.292+0000 7f6169000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6164071c10 con 0x7f61640af8d0 2024-04-20T21:24:55.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.292+0000 7f6169000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f616415eaf0 con 0x7f61640b3810 2024-04-20T21:24:55.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.292+0000 7f6169000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f6164142490 con 0x7f6164162c40 2024-04-20T21:24:55.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.292+0000 7f6163400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f61640af8d0 0x7f61640b3100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:50608/0 (socket says 172.21.15.86:50608) 2024-04-20T21:24:55.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.292+0000 7f6163400640 1 -- 172.21.15.86:0/1737720349 learned_addr learned my addr 172.21.15.86:0/1737720349 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:24:55.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.292+0000 7f6163e00640 1 --2- 172.21.15.86:0/1737720349 >> v2:172.21.15.86:3300/0 conn(0x7f6164162c40 0x7f6164141f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:55.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.293+0000 7f6162a00640 1 -- 172.21.15.86:0/1737720349 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3316998531 0 0) 0x7f6164071c10 con 0x7f61640af8d0 2024-04-20T21:24:55.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.293+0000 7f6162a00640 1 -- 172.21.15.86:0/1737720349 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6144003520 con 0x7f61640af8d0 2024-04-20T21:24:55.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.293+0000 7f6163e00640 1 -- 172.21.15.86:0/1737720349 >> v1:172.21.15.144:6789/0 conn(0x7f61640b3810 legacy=0x7f6164072990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:55.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.293+0000 7f6163e00640 1 -- 172.21.15.86:0/1737720349 >> v1:172.21.15.86:6789/0 conn(0x7f61640af8d0 legacy=0x7f61640b3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:55.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.293+0000 7f6163e00640 1 -- 172.21.15.86:0/1737720349 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6164071eb0 con 0x7f6164162c40 2024-04-20T21:24:55.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.294+0000 7f6163e00640 1 --2- 172.21.15.86:0/1737720349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6164162c40 0x7f6164141f50 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f615c009d90 tx=0x7f615c02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e60dee9aa123c646 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:55.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.294+0000 7f6162a00640 1 -- 172.21.15.86:0/1737720349 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f615c039070 con 0x7f6164162c40 2024-04-20T21:24:55.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.294+0000 7f6162a00640 1 -- 172.21.15.86:0/1737720349 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f615c02cbd0 con 0x7f6164162c40 2024-04-20T21:24:55.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.294+0000 7f6162a00640 1 -- 172.21.15.86:0/1737720349 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f615c034440 con 0x7f6164162c40 2024-04-20T21:24:55.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.295+0000 7f6169000640 1 -- 172.21.15.86:0/1737720349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6164162c40 msgr2=0x7f6164141f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:55.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.295+0000 7f6169000640 1 --2- 172.21.15.86:0/1737720349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6164162c40 0x7f6164141f50 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f615c009d90 tx=0x7f615c02be20 comp rx=0 tx=0).stop 2024-04-20T21:24:55.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.295+0000 7f6169000640 1 -- 172.21.15.86:0/1737720349 shutdown_connections 2024-04-20T21:24:55.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.295+0000 7f6169000640 1 --2- 172.21.15.86:0/1737720349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6164162c40 0x7f6164141f50 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:55.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.295+0000 7f6169000640 1 -- 172.21.15.86:0/1737720349 >> 172.21.15.86:0/1737720349 conn(0x7f61640d0170 msgr2=0x7f61640d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:55.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.295+0000 7f6169000640 1 -- 172.21.15.86:0/1737720349 shutdown_connections 2024-04-20T21:24:55.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.295+0000 7f6169000640 1 -- 172.21.15.86:0/1737720349 wait complete. 2024-04-20T21:24:55.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.297+0000 7f6169000640 1 Processor -- start 2024-04-20T21:24:55.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.297+0000 7f6169000640 1 -- start start 2024-04-20T21:24:55.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.298+0000 7f6169000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61640af8d0 0x7f6164157fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:55.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.298+0000 7f6169000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61640b3810 0x7f6164144ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:55.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.298+0000 7f6169000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6164162c40 0x7f6164145430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:55.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.298+0000 7f6169000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6164163100 con 0x7f6164162c40 2024-04-20T21:24:55.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.298+0000 7f6169000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6164072470 con 0x7f61640af8d0 2024-04-20T21:24:55.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.298+0000 7f6169000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6164148050 con 0x7f61640b3810 2024-04-20T21:24:55.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.298+0000 7f6163e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6164162c40 0x7f6164145430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:55.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.298+0000 7f6163e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6164162c40 0x7f6164145430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43052/0 (socket says 172.21.15.86:43052) 2024-04-20T21:24:55.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.298+0000 7f6163e00640 1 -- 172.21.15.86:0/3511044209 learned_addr learned my addr 172.21.15.86:0/3511044209 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:55.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.298+0000 7f6163400640 1 --2- 172.21.15.86:0/3511044209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61640af8d0 0x7f6164157fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:55.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.298+0000 7f6163e00640 1 -- 172.21.15.86:0/3511044209 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61640b3810 msgr2=0x7f6164144ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:55.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.298+0000 7f6163e00640 1 --2- 172.21.15.86:0/3511044209 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61640b3810 0x7f6164144ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:55.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.298+0000 7f6163e00640 1 -- 172.21.15.86:0/3511044209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61640af8d0 msgr2=0x7f6164157fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:55.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.298+0000 7f6163e00640 1 --2- 172.21.15.86:0/3511044209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61640af8d0 0x7f6164157fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:55.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.298+0000 7f6163e00640 1 -- 172.21.15.86:0/3511044209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61641469b0 con 0x7f6164162c40 2024-04-20T21:24:55.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.298+0000 7f6163400640 1 --2- 172.21.15.86:0/3511044209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61640af8d0 0x7f6164157fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:24:55.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.299+0000 7f6163e00640 1 --2- 172.21.15.86:0/3511044209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6164162c40 0x7f6164145430 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f615c009eb0 tx=0x7f615c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:55.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.299+0000 7f615b400640 1 -- 172.21.15.86:0/3511044209 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f615c039050 con 0x7f6164162c40 2024-04-20T21:24:55.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.299+0000 7f615b400640 1 -- 172.21.15.86:0/3511044209 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f615c045070 con 0x7f6164162c40 2024-04-20T21:24:55.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.299+0000 7f615b400640 1 -- 172.21.15.86:0/3511044209 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f615c009450 con 0x7f6164162c40 2024-04-20T21:24:55.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.299+0000 7f6169000640 1 -- 172.21.15.86:0/3511044209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f616415d8d0 con 0x7f6164162c40 2024-04-20T21:24:55.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.299+0000 7f6169000640 1 -- 172.21.15.86:0/3511044209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6164146c50 con 0x7f6164162c40 2024-04-20T21:24:55.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.300+0000 7f6169000640 1 -- 172.21.15.86:0/3511044209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f616404fcf0 con 0x7f6164162c40 2024-04-20T21:24:55.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.301+0000 7f615b400640 1 -- 172.21.15.86:0/3511044209 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f615c03e850 con 0x7f6164162c40 2024-04-20T21:24:55.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.302+0000 7f615b400640 1 --2- 172.21.15.86:0/3511044209 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f614006c8f0 0x7f614006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:55.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.302+0000 7f615b400640 1 -- 172.21.15.86:0/3511044209 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4857+0+0 (secure 0 0 0) 0x7f615c0ae4f0 con 0x7f6164162c40 2024-04-20T21:24:55.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.302+0000 7f6163400640 1 --2- 172.21.15.86:0/3511044209 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f614006c8f0 0x7f614006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:55.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.303+0000 7f6163400640 1 --2- 172.21.15.86:0/3511044209 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f614006c8f0 0x7f614006edb0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6154000f60 tx=0x7f6154009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:55.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.306+0000 7f615b400640 1 -- 172.21.15.86:0/3511044209 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f615c07ce40 con 0x7f6164162c40 2024-04-20T21:24:55.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.505+0000 7f6169000640 1 -- 172.21.15.86:0/3511044209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f6164072a30 con 0x7f6164162c40 2024-04-20T21:24:55.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.506+0000 7f615b400640 1 -- 172.21.15.86:0/3511044209 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) ==== 84+0+65 (secure 0 0 0) 0x7f615c07c740 con 0x7f6164162c40 2024-04-20T21:24:55.506 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:24:55.506 INFO:teuthology.orchestra.run.smithi086.stdout:[ 2024-04-20T21:24:55.506 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:24:55.506 INFO:teuthology.orchestra.run.smithi086.stdout: "poolnum": 1, 2024-04-20T21:24:55.506 INFO:teuthology.orchestra.run.smithi086.stdout: "poolname": ".mgr" 2024-04-20T21:24:55.506 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:24:55.507 INFO:teuthology.orchestra.run.smithi086.stdout:] 2024-04-20T21:24:55.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.508+0000 7f6169000640 1 -- 172.21.15.86:0/3511044209 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f614006c8f0 msgr2=0x7f614006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:55.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.508+0000 7f6169000640 1 --2- 172.21.15.86:0/3511044209 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f614006c8f0 0x7f614006edb0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6154000f60 tx=0x7f6154009040 comp rx=0 tx=0).stop 2024-04-20T21:24:55.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.508+0000 7f6169000640 1 -- 172.21.15.86:0/3511044209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6164162c40 msgr2=0x7f6164145430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:55.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.508+0000 7f6169000640 1 --2- 172.21.15.86:0/3511044209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6164162c40 0x7f6164145430 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f615c009eb0 tx=0x7f615c004a40 comp rx=0 tx=0).stop 2024-04-20T21:24:55.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.508+0000 7f6169000640 1 -- 172.21.15.86:0/3511044209 shutdown_connections 2024-04-20T21:24:55.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.508+0000 7f6169000640 1 --2- 172.21.15.86:0/3511044209 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f614006c8f0 0x7f614006edb0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:55.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.508+0000 7f6169000640 1 --2- 172.21.15.86:0/3511044209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6164162c40 0x7f6164145430 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:55.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.508+0000 7f6169000640 1 --2- 172.21.15.86:0/3511044209 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61640b3810 0x7f6164144ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:55.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.508+0000 7f6169000640 1 --2- 172.21.15.86:0/3511044209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61640af8d0 0x7f6164157fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:55.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.508+0000 7f6169000640 1 -- 172.21.15.86:0/3511044209 >> 172.21.15.86:0/3511044209 conn(0x7f61640d0170 msgr2=0x7f61640b14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:55.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.509+0000 7f6169000640 1 -- 172.21.15.86:0/3511044209 shutdown_connections 2024-04-20T21:24:55.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.509+0000 7f6169000640 1 -- 172.21.15.86:0/3511044209 wait complete. 2024-04-20T21:24:55.527 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-20T21:24:55.527 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-20T21:24:55.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.732+0000 7f23f6c00640 1 Processor -- start 2024-04-20T21:24:55.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.732+0000 7f23f6c00640 1 -- start start 2024-04-20T21:24:55.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.734+0000 7f23f6c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f23f0161ef0 0x7f23f00cee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:55.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.734+0000 7f23f6c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f23f00b64b0 0x7f23f0162c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:55.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.734+0000 7f23f6c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23f01622d0 con 0x7f23f00b6060 2024-04-20T21:24:55.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.734+0000 7f23f6c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f23f00d4f70 con 0x7f23f00b64b0 2024-04-20T21:24:55.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.734+0000 7f23f6c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f23f00b3e20 con 0x7f23f0161ef0 2024-04-20T21:24:55.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.734+0000 7f23f6200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f23f00b64b0 0x7f23f0162c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:55.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.734+0000 7f23f6200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f23f00b64b0 0x7f23f0162c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43066/0 (socket says 172.21.15.86:43066) 2024-04-20T21:24:55.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.734+0000 7f23f6200640 1 -- 172.21.15.86:0/69538826 learned_addr learned my addr 172.21.15.86:0/69538826 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:55.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.735+0000 7f23f5800640 1 --2- 172.21.15.86:0/69538826 >> v2:172.21.15.144:3300/0 conn(0x7f23f0161ef0 0x7f23f00cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:55.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.735+0000 7f23f6200640 1 -- 172.21.15.86:0/69538826 >> v2:172.21.15.144:3300/0 conn(0x7f23f0161ef0 msgr2=0x7f23f00cee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:55.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.735+0000 7f23f6200640 1 --2- 172.21.15.86:0/69538826 >> v2:172.21.15.144:3300/0 conn(0x7f23f0161ef0 0x7f23f00cee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:55.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.735+0000 7f23f6200640 1 -- 172.21.15.86:0/69538826 >> v1:172.21.15.86:6789/0 conn(0x7f23f00b6060 legacy=0x7f23f0162560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:55.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.735+0000 7f23f6200640 1 -- 172.21.15.86:0/69538826 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23f00b3860 con 0x7f23f00b64b0 2024-04-20T21:24:55.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.735+0000 7f23f5800640 1 --2- 172.21.15.86:0/69538826 >> v2:172.21.15.144:3300/0 conn(0x7f23f0161ef0 0x7f23f00cee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:24:55.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.736+0000 7f23f6200640 1 --2- 172.21.15.86:0/69538826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23f00b64b0 0x7f23f0162c70 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f23f008cee0 tx=0x7f23e802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6139db65b60007d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:55.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.736+0000 7f23efe00640 1 -- 172.21.15.86:0/69538826 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e8034070 con 0x7f23f00b64b0 2024-04-20T21:24:55.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.736+0000 7f23efe00640 1 -- 172.21.15.86:0/69538826 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23e8030860 con 0x7f23f00b64b0 2024-04-20T21:24:55.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.736+0000 7f23efe00640 1 -- 172.21.15.86:0/69538826 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e8030c30 con 0x7f23f00b64b0 2024-04-20T21:24:55.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.737+0000 7f23f6c00640 1 -- 172.21.15.86:0/69538826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23f00b64b0 msgr2=0x7f23f0162c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:55.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.737+0000 7f23f6c00640 1 --2- 172.21.15.86:0/69538826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23f00b64b0 0x7f23f0162c70 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f23f008cee0 tx=0x7f23e802b9a0 comp rx=0 tx=0).stop 2024-04-20T21:24:55.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.737+0000 7f23f6c00640 1 -- 172.21.15.86:0/69538826 shutdown_connections 2024-04-20T21:24:55.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.737+0000 7f23f6c00640 1 --2- 172.21.15.86:0/69538826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23f00b64b0 0x7f23f0162c70 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:55.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.737+0000 7f23f6c00640 1 --2- 172.21.15.86:0/69538826 >> v2:172.21.15.144:3300/0 conn(0x7f23f0161ef0 0x7f23f00cee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:55.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.737+0000 7f23f6c00640 1 -- 172.21.15.86:0/69538826 >> 172.21.15.86:0/69538826 conn(0x7f23f008bf70 msgr2=0x7f23f00d45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:55.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.737+0000 7f23f6c00640 1 -- 172.21.15.86:0/69538826 shutdown_connections 2024-04-20T21:24:55.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.737+0000 7f23f6c00640 1 -- 172.21.15.86:0/69538826 wait complete. 2024-04-20T21:24:55.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.739+0000 7f23f6c00640 1 Processor -- start 2024-04-20T21:24:55.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.739+0000 7f23f6c00640 1 -- start start 2024-04-20T21:24:55.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.739+0000 7f23f6c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23f00b6060 0x7f23f014fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:55.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.739+0000 7f23f6c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23f00b64b0 0x7f23f00b7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:55.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.739+0000 7f23f6c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23f0161ef0 0x7f23f01f4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:55.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.740+0000 7f23f6c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f23f006ce20 con 0x7f23f0161ef0 2024-04-20T21:24:55.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.740+0000 7f23f6c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f23f006cb40 con 0x7f23f00b64b0 2024-04-20T21:24:55.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.740+0000 7f23f6c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f23f00b68c0 con 0x7f23f00b6060 2024-04-20T21:24:55.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.740+0000 7f23f6200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23f0161ef0 0x7f23f01f4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:55.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.740+0000 7f23f6200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23f0161ef0 0x7f23f01f4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43070/0 (socket says 172.21.15.86:43070) 2024-04-20T21:24:55.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.740+0000 7f23f6200640 1 -- 172.21.15.86:0/3202246924 learned_addr learned my addr 172.21.15.86:0/3202246924 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:55.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.740+0000 7f23f6200640 1 -- 172.21.15.86:0/3202246924 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23f00b6060 msgr2=0x7f23f014fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:55.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.740+0000 7f23f6200640 1 --2- 172.21.15.86:0/3202246924 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23f00b6060 0x7f23f014fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:55.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.740+0000 7f23f6200640 1 -- 172.21.15.86:0/3202246924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23f00b64b0 msgr2=0x7f23f00b7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:55.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.740+0000 7f23f6200640 1 --2- 172.21.15.86:0/3202246924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23f00b64b0 0x7f23f00b7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:55.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.740+0000 7f23f6200640 1 -- 172.21.15.86:0/3202246924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23f0150180 con 0x7f23f0161ef0 2024-04-20T21:24:55.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.741+0000 7f23f6200640 1 --2- 172.21.15.86:0/3202246924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23f0161ef0 0x7f23f01f4340 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f23e8009960 tx=0x7f23e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:55.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.741+0000 7f23ed600640 1 -- 172.21.15.86:0/3202246924 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e8048070 con 0x7f23f0161ef0 2024-04-20T21:24:55.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.741+0000 7f23ed600640 1 -- 172.21.15.86:0/3202246924 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23e8030660 con 0x7f23f0161ef0 2024-04-20T21:24:55.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.741+0000 7f23ed600640 1 -- 172.21.15.86:0/3202246924 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e8034040 con 0x7f23f0161ef0 2024-04-20T21:24:55.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.741+0000 7f23f6c00640 1 -- 172.21.15.86:0/3202246924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23f01f4e50 con 0x7f23f0161ef0 2024-04-20T21:24:55.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.741+0000 7f23f6c00640 1 -- 172.21.15.86:0/3202246924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23f00b7640 con 0x7f23f0161ef0 2024-04-20T21:24:55.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.742+0000 7f23f6c00640 1 -- 172.21.15.86:0/3202246924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23f00b7430 con 0x7f23f0161ef0 2024-04-20T21:24:55.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.743+0000 7f23ed600640 1 -- 172.21.15.86:0/3202246924 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f23e804b070 con 0x7f23f0161ef0 2024-04-20T21:24:55.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.744+0000 7f23ed600640 1 --2- 172.21.15.86:0/3202246924 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f23d006c8f0 0x7f23d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:55.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.744+0000 7f23ed600640 1 -- 172.21.15.86:0/3202246924 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4857+0+0 (secure 0 0 0) 0x7f23e80ada30 con 0x7f23f0161ef0 2024-04-20T21:24:55.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.744+0000 7f23f5800640 1 --2- 172.21.15.86:0/3202246924 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f23d006c8f0 0x7f23d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:55.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.745+0000 7f23f5800640 1 --2- 172.21.15.86:0/3202246924 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f23d006c8f0 0x7f23d006edb0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f23e4008e00 tx=0x7f23e4008330 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:55.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.748+0000 7f23ed600640 1 -- 172.21.15.86:0/3202246924 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f23f0150180 con 0x7f23f0161ef0 2024-04-20T21:24:55.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:55.946+0000 7f23f6c00640 1 -- 172.21.15.86:0/3202246924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f23f004fcf0 con 0x7f23f0161ef0 2024-04-20T21:24:56.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:56.803+0000 7f23ed600640 1 -- 172.21.15.86:0/3202246924 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) ==== 138+0+0 (secure 0 0 0) 0x7f23e8030830 con 0x7f23f0161ef0 2024-04-20T21:24:56.804 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' created 2024-04-20T21:24:56.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:56.807+0000 7f23f6c00640 1 -- 172.21.15.86:0/3202246924 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f23d006c8f0 msgr2=0x7f23d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:56.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:56.807+0000 7f23f6c00640 1 --2- 172.21.15.86:0/3202246924 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f23d006c8f0 0x7f23d006edb0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f23e4008e00 tx=0x7f23e4008330 comp rx=0 tx=0).stop 2024-04-20T21:24:56.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:56.807+0000 7f23f6c00640 1 -- 172.21.15.86:0/3202246924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23f0161ef0 msgr2=0x7f23f01f4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:56.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:56.807+0000 7f23f6c00640 1 --2- 172.21.15.86:0/3202246924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23f0161ef0 0x7f23f01f4340 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f23e8009960 tx=0x7f23e8004730 comp rx=0 tx=0).stop 2024-04-20T21:24:56.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:56.807+0000 7f23f6c00640 1 -- 172.21.15.86:0/3202246924 shutdown_connections 2024-04-20T21:24:56.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:56.807+0000 7f23f6c00640 1 --2- 172.21.15.86:0/3202246924 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f23d006c8f0 0x7f23d006edb0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:56.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:56.807+0000 7f23f6c00640 1 --2- 172.21.15.86:0/3202246924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23f0161ef0 0x7f23f01f4340 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:56.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:56.807+0000 7f23f6c00640 1 --2- 172.21.15.86:0/3202246924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23f00b64b0 0x7f23f00b7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:56.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:56.807+0000 7f23f6c00640 1 --2- 172.21.15.86:0/3202246924 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23f00b6060 0x7f23f014fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:56.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:56.807+0000 7f23f6c00640 1 -- 172.21.15.86:0/3202246924 >> 172.21.15.86:0/3202246924 conn(0x7f23f008bf70 msgr2=0x7f23f00d4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:56.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:56.808+0000 7f23f6c00640 1 -- 172.21.15.86:0/3202246924 shutdown_connections 2024-04-20T21:24:56.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:56.808+0000 7f23f6c00640 1 -- 172.21.15.86:0/3202246924 wait complete. 2024-04-20T21:24:56.837 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-20T21:24:57.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.022+0000 7efd37200640 1 Processor -- start 2024-04-20T21:24:57.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.022+0000 7efd37200640 1 -- start start 2024-04-20T21:24:57.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.024+0000 7efd37200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd3015eb30 0x7efd300b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:57.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.024+0000 7efd37200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7efd3006c090 0x7efd300b4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:57.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.024+0000 7efd37200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd300adb10 con 0x7efd300aee50 2024-04-20T21:24:57.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.024+0000 7efd37200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7efd300af7d0 con 0x7efd3006c090 2024-04-20T21:24:57.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.024+0000 7efd37200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7efd300addb0 con 0x7efd3015eb30 2024-04-20T21:24:57.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.024+0000 7efd27400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7efd300aee50 0x7efd3006b980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:37918/0 (socket says 172.21.15.86:37918) 2024-04-20T21:24:57.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.024+0000 7efd27400640 1 -- 172.21.15.86:0/2678417897 learned_addr learned my addr 172.21.15.86:0/2678417897 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:24:57.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.025+0000 7efd2f400640 1 -- 172.21.15.86:0/2678417897 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 332622217 0 0) 0x7efd300adb10 con 0x7efd300aee50 2024-04-20T21:24:57.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.025+0000 7efd2f400640 1 -- 172.21.15.86:0/2678417897 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd14003520 con 0x7efd300aee50 2024-04-20T21:24:57.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.025+0000 7efd2fe00640 1 --2- 172.21.15.86:0/2678417897 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd3015eb30 0x7efd300b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:57.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.025+0000 7efd35200640 1 --2- 172.21.15.86:0/2678417897 >> v2:172.21.15.144:3300/0 conn(0x7efd3006c090 0x7efd300b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:57.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.025+0000 7efd2f400640 1 -- 172.21.15.86:0/2678417897 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 356164006 0 0) 0x7efd14003520 con 0x7efd300aee50 2024-04-20T21:24:57.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.025+0000 7efd2f400640 1 -- 172.21.15.86:0/2678417897 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd3015eb30 msgr2=0x7efd300b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:57.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.025+0000 7efd2f400640 1 --2- 172.21.15.86:0/2678417897 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd3015eb30 0x7efd300b3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:57.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.025+0000 7efd2f400640 1 -- 172.21.15.86:0/2678417897 >> v2:172.21.15.144:3300/0 conn(0x7efd3006c090 msgr2=0x7efd300b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:57.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.025+0000 7efd2f400640 1 --2- 172.21.15.86:0/2678417897 >> v2:172.21.15.144:3300/0 conn(0x7efd3006c090 0x7efd300b4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:57.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.025+0000 7efd2f400640 1 -- 172.21.15.86:0/2678417897 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd300ae090 con 0x7efd300aee50 2024-04-20T21:24:57.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.025+0000 7efd2f400640 1 -- 172.21.15.86:0/2678417897 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7efd18001670 con 0x7efd300aee50 2024-04-20T21:24:57.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.025+0000 7efd2f400640 1 -- 172.21.15.86:0/2678417897 >> v1:172.21.15.86:6789/0 conn(0x7efd300aee50 legacy=0x7efd3006b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:57.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.025+0000 7efd2f400640 1 --2- 172.21.15.86:0/2678417897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd140039b0 0x7efd14003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:57.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.025+0000 7efd2f400640 1 --2- 172.21.15.86:0/2678417897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd14004800 0x7efd14004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:57.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.025+0000 7efd2f400640 1 --2- 172.21.15.86:0/2678417897 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd3015eb30 0x7efd140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:57.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.025+0000 7efd2f400640 1 -- 172.21.15.86:0/2678417897 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efd300af7d0 con 0x7efd140039b0 2024-04-20T21:24:57.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.025+0000 7efd2f400640 1 -- 172.21.15.86:0/2678417897 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efd300addb0 con 0x7efd14004800 2024-04-20T21:24:57.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.025+0000 7efd2f400640 1 -- 172.21.15.86:0/2678417897 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7efd1400ba70 con 0x7efd3015eb30 2024-04-20T21:24:57.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.026+0000 7efd2fe00640 1 --2- 172.21.15.86:0/2678417897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd140039b0 0x7efd14003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:57.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.026+0000 7efd35200640 1 --2- 172.21.15.86:0/2678417897 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd3015eb30 0x7efd140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:57.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.026+0000 7efd2fe00640 1 -- 172.21.15.86:0/2678417897 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd3015eb30 msgr2=0x7efd140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:57.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.026+0000 7efd2fe00640 1 --2- 172.21.15.86:0/2678417897 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd3015eb30 0x7efd140051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:57.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.026+0000 7efd2fe00640 1 -- 172.21.15.86:0/2678417897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd14004800 msgr2=0x7efd14004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:57.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.026+0000 7efd2fe00640 1 --2- 172.21.15.86:0/2678417897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd14004800 0x7efd14004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:57.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.026+0000 7efd2fe00640 1 -- 172.21.15.86:0/2678417897 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd14003520 con 0x7efd140039b0 2024-04-20T21:24:57.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.027+0000 7efd2fe00640 1 --2- 172.21.15.86:0/2678417897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd140039b0 0x7efd14003dd0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7efd20007330 tx=0x7efd14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd07468cfd37ee8e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:57.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.027+0000 7efd2f400640 1 -- 172.21.15.86:0/2678417897 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd20039070 con 0x7efd140039b0 2024-04-20T21:24:57.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.027+0000 7efd2f400640 1 -- 172.21.15.86:0/2678417897 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd2002cc70 con 0x7efd140039b0 2024-04-20T21:24:57.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.027+0000 7efd2f400640 1 -- 172.21.15.86:0/2678417897 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd20009be0 con 0x7efd140039b0 2024-04-20T21:24:57.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.028+0000 7efd37200640 1 -- 172.21.15.86:0/2678417897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd140039b0 msgr2=0x7efd14003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:57.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.028+0000 7efd37200640 1 --2- 172.21.15.86:0/2678417897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd140039b0 0x7efd14003dd0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7efd20007330 tx=0x7efd14005830 comp rx=0 tx=0).stop 2024-04-20T21:24:57.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.028+0000 7efd35200640 1 -- 172.21.15.86:0/2678417897 reap_dead start 2024-04-20T21:24:57.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.028+0000 7efd37200640 1 -- 172.21.15.86:0/2678417897 shutdown_connections 2024-04-20T21:24:57.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.028+0000 7efd37200640 1 -- 172.21.15.86:0/2678417897 >> 172.21.15.86:0/2678417897 conn(0x7efd300b2110 msgr2=0x7efd300b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:57.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.028+0000 7efd37200640 1 -- 172.21.15.86:0/2678417897 shutdown_connections 2024-04-20T21:24:57.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.028+0000 7efd37200640 1 -- 172.21.15.86:0/2678417897 wait complete. 2024-04-20T21:24:57.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.030+0000 7efd37200640 1 Processor -- start 2024-04-20T21:24:57.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.030+0000 7efd37200640 1 -- start start 2024-04-20T21:24:57.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.031+0000 7efd37200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd140039b0 0x7efd3014bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:57.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.031+0000 7efd37200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd300cf6c0 0x7efd300cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:57.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.031+0000 7efd37200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd301577c0 0x7efd30157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:57.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.031+0000 7efd37200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efd300ae930 con 0x7efd300cf6c0 2024-04-20T21:24:57.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.031+0000 7efd37200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efd300ae650 con 0x7efd140039b0 2024-04-20T21:24:57.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.031+0000 7efd37200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7efd300b3e20 con 0x7efd301577c0 2024-04-20T21:24:57.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.031+0000 7efd27400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd300cf6c0 0x7efd300cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:57.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.031+0000 7efd27400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd300cf6c0 0x7efd300cfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43260/0 (socket says 172.21.15.86:43260) 2024-04-20T21:24:57.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.031+0000 7efd27400640 1 -- 172.21.15.86:0/1554713257 learned_addr learned my addr 172.21.15.86:0/1554713257 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:57.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.031+0000 7efd27400640 1 -- 172.21.15.86:0/1554713257 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd301577c0 msgr2=0x7efd30157c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:57.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.031+0000 7efd27400640 1 --2- 172.21.15.86:0/1554713257 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd301577c0 0x7efd30157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:57.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.031+0000 7efd27400640 1 -- 172.21.15.86:0/1554713257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd140039b0 msgr2=0x7efd3014bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:57.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.031+0000 7efd27400640 1 --2- 172.21.15.86:0/1554713257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd140039b0 0x7efd3014bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:57.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.031+0000 7efd27400640 1 -- 172.21.15.86:0/1554713257 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd300ccf00 con 0x7efd300cf6c0 2024-04-20T21:24:57.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.032+0000 7efd27400640 1 --2- 172.21.15.86:0/1554713257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd300cf6c0 0x7efd300cfb40 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7efd1800c760 tx=0x7efd1800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:57.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.032+0000 7efd2cc00640 1 -- 172.21.15.86:0/1554713257 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd18018070 con 0x7efd300cf6c0 2024-04-20T21:24:57.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.032+0000 7efd2cc00640 1 -- 172.21.15.86:0/1554713257 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd180074e0 con 0x7efd300cf6c0 2024-04-20T21:24:57.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.032+0000 7efd2cc00640 1 -- 172.21.15.86:0/1554713257 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd18004a20 con 0x7efd300cf6c0 2024-04-20T21:24:57.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.032+0000 7efd37200640 1 -- 172.21.15.86:0/1554713257 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd300cd170 con 0x7efd300cf6c0 2024-04-20T21:24:57.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.032+0000 7efd37200640 1 -- 172.21.15.86:0/1554713257 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd300cd730 con 0x7efd300cf6c0 2024-04-20T21:24:57.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.033+0000 7efd37200640 1 -- 172.21.15.86:0/1554713257 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd300689e0 con 0x7efd300cf6c0 2024-04-20T21:24:57.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.034+0000 7efd2cc00640 1 -- 172.21.15.86:0/1554713257 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efd1801b070 con 0x7efd300cf6c0 2024-04-20T21:24:57.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.035+0000 7efd2cc00640 1 --2- 172.21.15.86:0/1554713257 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efd0c06c8f0 0x7efd0c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:57.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.035+0000 7efd2cc00640 1 -- 172.21.15.86:0/1554713257 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5227+0+0 (secure 0 0 0) 0x7efd1808af60 con 0x7efd300cf6c0 2024-04-20T21:24:57.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.035+0000 7efd2fe00640 1 --2- 172.21.15.86:0/1554713257 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efd0c06c8f0 0x7efd0c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:57.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.036+0000 7efd2fe00640 1 --2- 172.21.15.86:0/1554713257 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efd0c06c8f0 0x7efd0c06edb0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7efd20036000 tx=0x7efd2002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:57.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.040+0000 7efd2cc00640 1 -- 172.21.15.86:0/1554713257 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7efd180594d0 con 0x7efd300cf6c0 2024-04-20T21:24:57.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.226+0000 7efd37200640 1 -- 172.21.15.86:0/1554713257 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7efd3006ec10 con 0x7efd300cf6c0 2024-04-20T21:24:57.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.809+0000 7efd2cc00640 1 -- 172.21.15.86:0/1554713257 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v83) ==== 170+0+0 (secure 0 0 0) 0x7efd18058dd0 con 0x7efd300cf6c0 2024-04-20T21:24:57.810 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' created 2024-04-20T21:24:57.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.814+0000 7efd37200640 1 -- 172.21.15.86:0/1554713257 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efd0c06c8f0 msgr2=0x7efd0c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:57.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.814+0000 7efd37200640 1 --2- 172.21.15.86:0/1554713257 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efd0c06c8f0 0x7efd0c06edb0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7efd20036000 tx=0x7efd2002e000 comp rx=0 tx=0).stop 2024-04-20T21:24:57.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.814+0000 7efd37200640 1 -- 172.21.15.86:0/1554713257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd300cf6c0 msgr2=0x7efd300cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:57.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.815+0000 7efd37200640 1 --2- 172.21.15.86:0/1554713257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd300cf6c0 0x7efd300cfb40 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7efd1800c760 tx=0x7efd1800cc20 comp rx=0 tx=0).stop 2024-04-20T21:24:57.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.815+0000 7efd37200640 1 -- 172.21.15.86:0/1554713257 shutdown_connections 2024-04-20T21:24:57.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.815+0000 7efd37200640 1 --2- 172.21.15.86:0/1554713257 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efd0c06c8f0 0x7efd0c06edb0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:57.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.815+0000 7efd37200640 1 --2- 172.21.15.86:0/1554713257 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd301577c0 0x7efd30157c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:57.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.815+0000 7efd37200640 1 --2- 172.21.15.86:0/1554713257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd300cf6c0 0x7efd300cfb40 secure :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7efd1800c760 tx=0x7efd1800cc20 comp rx=0 tx=0).stop 2024-04-20T21:24:57.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.815+0000 7efd37200640 1 --2- 172.21.15.86:0/1554713257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd140039b0 0x7efd3014bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:57.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.815+0000 7efd37200640 1 -- 172.21.15.86:0/1554713257 >> 172.21.15.86:0/1554713257 conn(0x7efd300b2110 msgr2=0x7efd300b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:57.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.817+0000 7efd37200640 1 -- 172.21.15.86:0/1554713257 shutdown_connections 2024-04-20T21:24:57.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:57.817+0000 7efd37200640 1 -- 172.21.15.86:0/1554713257 wait complete. 2024-04-20T21:24:57.854 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-20T21:24:58.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.030+0000 7fc4c5c00640 1 Processor -- start 2024-04-20T21:24:58.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.031+0000 7fc4c5c00640 1 -- start start 2024-04-20T21:24:58.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.032+0000 7fc4c5c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc4c0158950 0x7fc4c0071db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:58.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.032+0000 7fc4c5c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc4c00722f0 0x7fc4c00b3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:58.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.032+0000 7fc4c5c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4c00b4400 0x7fc4c015ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:58.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.032+0000 7fc4c5c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc4c00b4bf0 con 0x7fc4c0158950 2024-04-20T21:24:58.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.032+0000 7fc4c5c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fc4c00b4ed0 con 0x7fc4c00722f0 2024-04-20T21:24:58.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.032+0000 7fc4c5c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc4c006de90 con 0x7fc4c00b4400 2024-04-20T21:24:58.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.032+0000 7fc4bea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc4c0158950 0x7fc4c0071db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:58.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.032+0000 7fc4bea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc4c0158950 0x7fc4c0071db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43274/0 (socket says 172.21.15.86:43274) 2024-04-20T21:24:58.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.032+0000 7fc4bea00640 1 -- 172.21.15.86:0/3913234636 learned_addr learned my addr 172.21.15.86:0/3913234636 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:58.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.033+0000 7fc4bf400640 1 --2- 172.21.15.86:0/3913234636 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4c00b4400 0x7fc4c015ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:58.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.033+0000 7fc4b7e00640 1 --2- 172.21.15.86:0/3913234636 >> v2:172.21.15.144:3300/0 conn(0x7fc4c00722f0 0x7fc4c00b3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:58.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.033+0000 7fc4bea00640 1 -- 172.21.15.86:0/3913234636 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4c00b4400 msgr2=0x7fc4c015ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:58.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.033+0000 7fc4bea00640 1 --2- 172.21.15.86:0/3913234636 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4c00b4400 0x7fc4c015ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:58.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.033+0000 7fc4bea00640 1 -- 172.21.15.86:0/3913234636 >> v2:172.21.15.144:3300/0 conn(0x7fc4c00722f0 msgr2=0x7fc4c00b3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:58.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.033+0000 7fc4bea00640 1 --2- 172.21.15.86:0/3913234636 >> v2:172.21.15.144:3300/0 conn(0x7fc4c00722f0 0x7fc4c00b3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:58.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.033+0000 7fc4bea00640 1 -- 172.21.15.86:0/3913234636 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c006e170 con 0x7fc4c0158950 2024-04-20T21:24:58.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.033+0000 7fc4b7e00640 1 --2- 172.21.15.86:0/3913234636 >> v2:172.21.15.144:3300/0 conn(0x7fc4c00722f0 0x7fc4c00b3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:24:58.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.033+0000 7fc4bf400640 1 --2- 172.21.15.86:0/3913234636 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4c00b4400 0x7fc4c015ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:24:58.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.034+0000 7fc4bea00640 1 --2- 172.21.15.86:0/3913234636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4c0158950 0x7fc4c0071db0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fc4c006ca00 tx=0x7fc4b002deb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87d50149433edc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:58.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.034+0000 7fc4be000640 1 -- 172.21.15.86:0/3913234636 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b0034070 con 0x7fc4c0158950 2024-04-20T21:24:58.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.034+0000 7fc4be000640 1 -- 172.21.15.86:0/3913234636 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4b002ec60 con 0x7fc4c0158950 2024-04-20T21:24:58.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.034+0000 7fc4be000640 1 -- 172.21.15.86:0/3913234636 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b0038620 con 0x7fc4c0158950 2024-04-20T21:24:58.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.035+0000 7fc4c5c00640 1 -- 172.21.15.86:0/3913234636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4c0158950 msgr2=0x7fc4c0071db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:58.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.035+0000 7fc4c5c00640 1 --2- 172.21.15.86:0/3913234636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4c0158950 0x7fc4c0071db0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fc4c006ca00 tx=0x7fc4b002deb0 comp rx=0 tx=0).stop 2024-04-20T21:24:58.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.035+0000 7fc4c5c00640 1 -- 172.21.15.86:0/3913234636 shutdown_connections 2024-04-20T21:24:58.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.035+0000 7fc4c5c00640 1 --2- 172.21.15.86:0/3913234636 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4c00b4400 0x7fc4c015ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:58.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.035+0000 7fc4c5c00640 1 --2- 172.21.15.86:0/3913234636 >> v2:172.21.15.144:3300/0 conn(0x7fc4c00722f0 0x7fc4c00b3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:58.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.035+0000 7fc4c5c00640 1 --2- 172.21.15.86:0/3913234636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4c0158950 0x7fc4c0071db0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:58.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.035+0000 7fc4c5c00640 1 -- 172.21.15.86:0/3913234636 >> 172.21.15.86:0/3913234636 conn(0x7fc4c00b2110 msgr2=0x7fc4c00b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:58.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.035+0000 7fc4c5c00640 1 -- 172.21.15.86:0/3913234636 shutdown_connections 2024-04-20T21:24:58.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.036+0000 7fc4c5c00640 1 -- 172.21.15.86:0/3913234636 wait complete. 2024-04-20T21:24:58.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.037+0000 7fc4c5c00640 1 Processor -- start 2024-04-20T21:24:58.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.037+0000 7fc4c5c00640 1 -- start start 2024-04-20T21:24:58.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.038+0000 7fc4c5c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4c00722f0 0x7fc4c00cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:58.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.038+0000 7fc4c5c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4c00b4400 0x7fc4c01480e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:58.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.038+0000 7fc4c5c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4c0158950 0x7fc4c00cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:58.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.038+0000 7fc4c5c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c00b4840 con 0x7fc4c0158950 2024-04-20T21:24:58.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.038+0000 7fc4c5c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c0158d30 con 0x7fc4c00722f0 2024-04-20T21:24:58.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.038+0000 7fc4c5c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc4c00ce010 con 0x7fc4c00b4400 2024-04-20T21:24:58.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.039+0000 7fc4bf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4c0158950 0x7fc4c00cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:58.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.039+0000 7fc4bf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4c0158950 0x7fc4c00cf0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43288/0 (socket says 172.21.15.86:43288) 2024-04-20T21:24:58.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.039+0000 7fc4bf400640 1 -- 172.21.15.86:0/866488935 learned_addr learned my addr 172.21.15.86:0/866488935 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:58.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.039+0000 7fc4bea00640 1 --2- 172.21.15.86:0/866488935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4c00722f0 0x7fc4c00cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:58.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.039+0000 7fc4b7e00640 1 --2- 172.21.15.86:0/866488935 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4c00b4400 0x7fc4c01480e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:58.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.039+0000 7fc4bf400640 1 -- 172.21.15.86:0/866488935 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4c00b4400 msgr2=0x7fc4c01480e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:58.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.039+0000 7fc4bf400640 1 --2- 172.21.15.86:0/866488935 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4c00b4400 0x7fc4c01480e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:58.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.039+0000 7fc4bf400640 1 -- 172.21.15.86:0/866488935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4c00722f0 msgr2=0x7fc4c00cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:58.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.039+0000 7fc4bf400640 1 --2- 172.21.15.86:0/866488935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4c00722f0 0x7fc4c00cd9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:58.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.039+0000 7fc4bf400640 1 -- 172.21.15.86:0/866488935 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c00ce2f0 con 0x7fc4c0158950 2024-04-20T21:24:58.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.039+0000 7fc4b7e00640 1 --2- 172.21.15.86:0/866488935 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4c00b4400 0x7fc4c01480e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:24:58.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.039+0000 7fc4bf400640 1 --2- 172.21.15.86:0/866488935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4c0158950 0x7fc4c00cf0b0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fc4b800cef0 tx=0x7fc4b800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:58.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.040+0000 7fc4b6a00640 1 -- 172.21.15.86:0/866488935 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b801a070 con 0x7fc4c0158950 2024-04-20T21:24:58.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.040+0000 7fc4b6a00640 1 -- 172.21.15.86:0/866488935 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4b800d860 con 0x7fc4c0158950 2024-04-20T21:24:58.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.040+0000 7fc4b6a00640 1 -- 172.21.15.86:0/866488935 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b800dc50 con 0x7fc4c0158950 2024-04-20T21:24:58.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.040+0000 7fc4c5c00640 1 -- 172.21.15.86:0/866488935 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4c00ce5d0 con 0x7fc4c0158950 2024-04-20T21:24:58.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.040+0000 7fc4c5c00640 1 -- 172.21.15.86:0/866488935 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4c0148620 con 0x7fc4c0158950 2024-04-20T21:24:58.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.041+0000 7fc4c5c00640 1 -- 172.21.15.86:0/866488935 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4c0156d80 con 0x7fc4c0158950 2024-04-20T21:24:58.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.043+0000 7fc4b6a00640 1 -- 172.21.15.86:0/866488935 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc4b801d070 con 0x7fc4c0158950 2024-04-20T21:24:58.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.043+0000 7fc4b6a00640 1 --2- 172.21.15.86:0/866488935 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc49c06c920 0x7fc49c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:58.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.044+0000 7fc4b6a00640 1 -- 172.21.15.86:0/866488935 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5609+0+0 (secure 0 0 0) 0x7fc4b808c2d0 con 0x7fc4c0158950 2024-04-20T21:24:58.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.044+0000 7fc4bea00640 1 --2- 172.21.15.86:0/866488935 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc49c06c920 0x7fc49c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:58.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.045+0000 7fc4bea00640 1 --2- 172.21.15.86:0/866488935 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc49c06c920 0x7fc49c06ede0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0039000 tx=0x7fc4b0034000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:58.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.053+0000 7fc4b6a00640 1 -- 172.21.15.86:0/866488935 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc4c0156d80 con 0x7fc4c0158950 2024-04-20T21:24:58.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.296+0000 7fc4c5c00640 1 -- 172.21.15.86:0/866488935 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fc4c00689e0 con 0x7fc4c0158950 2024-04-20T21:24:58.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.830+0000 7fc4b6a00640 1 -- 172.21.15.86:0/866488935 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v55) ==== 174+0+0 (secure 0 0 0) 0x7fc4b805a8b0 con 0x7fc4c0158950 2024-04-20T21:24:58.830 INFO:teuthology.orchestra.run.smithi086.stderr:new fs with metadata pool 12 and data pool 13 2024-04-20T21:24:58.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.834+0000 7fc4c5c00640 1 -- 172.21.15.86:0/866488935 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc49c06c920 msgr2=0x7fc49c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:58.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.834+0000 7fc4c5c00640 1 --2- 172.21.15.86:0/866488935 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc49c06c920 0x7fc49c06ede0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0039000 tx=0x7fc4b0034000 comp rx=0 tx=0).stop 2024-04-20T21:24:58.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.834+0000 7fc4c5c00640 1 -- 172.21.15.86:0/866488935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4c0158950 msgr2=0x7fc4c00cf0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:58.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.834+0000 7fc4c5c00640 1 --2- 172.21.15.86:0/866488935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4c0158950 0x7fc4c00cf0b0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fc4b800cef0 tx=0x7fc4b800a710 comp rx=0 tx=0).stop 2024-04-20T21:24:58.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.834+0000 7fc4c5c00640 1 -- 172.21.15.86:0/866488935 shutdown_connections 2024-04-20T21:24:58.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.834+0000 7fc4c5c00640 1 --2- 172.21.15.86:0/866488935 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc49c06c920 0x7fc49c06ede0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:58.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.834+0000 7fc4c5c00640 1 --2- 172.21.15.86:0/866488935 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4c0158950 0x7fc4c00cf0b0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:58.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.834+0000 7fc4c5c00640 1 --2- 172.21.15.86:0/866488935 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4c00b4400 0x7fc4c01480e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:58.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.834+0000 7fc4c5c00640 1 --2- 172.21.15.86:0/866488935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4c00722f0 0x7fc4c00cd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:58.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.834+0000 7fc4c5c00640 1 -- 172.21.15.86:0/866488935 >> 172.21.15.86:0/866488935 conn(0x7fc4c00b2110 msgr2=0x7fc4c015d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:58.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.835+0000 7fc4c5c00640 1 -- 172.21.15.86:0/866488935 shutdown_connections 2024-04-20T21:24:58.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:58.835+0000 7fc4c5c00640 1 -- 172.21.15.86:0/866488935 wait complete. 2024-04-20T21:24:58.876 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:24:59.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.049+0000 7f2009c00640 1 Processor -- start 2024-04-20T21:24:59.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.049+0000 7f2009c00640 1 -- start start 2024-04-20T21:24:59.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.051+0000 7f2009c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2004144e90 0x7f200415ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:59.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.051+0000 7f2009c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f200406dc30 con 0x7f200406f140 2024-04-20T21:24:59.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.051+0000 7f2009c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f200415f720 con 0x7f2004071d00 2024-04-20T21:24:59.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.051+0000 7f2009c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f2004145270 con 0x7f2004144e90 2024-04-20T21:24:59.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.051+0000 7f2002a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2004144e90 0x7f200415ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:59.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.052+0000 7f1ffa000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f200406f140 0x7f20040aeb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:37926/0 (socket says 172.21.15.86:37926) 2024-04-20T21:24:59.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.052+0000 7f2002a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2004144e90 0x7f200415ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43292/0 (socket says 172.21.15.86:43292) 2024-04-20T21:24:59.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.052+0000 7f1ffa000640 1 -- 172.21.15.86:0/1456315837 learned_addr learned my addr 172.21.15.86:0/1456315837 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:24:59.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.052+0000 7f2002a00640 1 -- 172.21.15.86:0/1456315837 >> v1:172.21.15.144:6789/0 conn(0x7f2004071d00 legacy=0x7f200406e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:59.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.052+0000 7f2002a00640 1 -- 172.21.15.86:0/1456315837 >> v1:172.21.15.86:6789/0 conn(0x7f200406f140 legacy=0x7f20040aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:59.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.052+0000 7f2002a00640 1 -- 172.21.15.86:0/1456315837 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f200415fa00 con 0x7f2004144e90 2024-04-20T21:24:59.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.052+0000 7f2002000640 1 -- 172.21.15.86:0/1456315837 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4015807987 0 0) 0x7f200406dc30 con 0x7f200406f140 2024-04-20T21:24:59.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.053+0000 7f2002a00640 1 --2- 172.21.15.86:0/1456315837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2004144e90 0x7f200415ee10 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4009d90 tx=0x7f1ff402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=776fb4f82fe73c17 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:59.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.054+0000 7f2002000640 1 -- 172.21.15.86:0/1456315837 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4039070 con 0x7f2004144e90 2024-04-20T21:24:59.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.054+0000 7f2002000640 1 -- 172.21.15.86:0/1456315837 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ff402cc60 con 0x7f2004144e90 2024-04-20T21:24:59.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.054+0000 7f2002000640 1 -- 172.21.15.86:0/1456315837 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4034660 con 0x7f2004144e90 2024-04-20T21:24:59.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.054+0000 7f2009c00640 1 -- 172.21.15.86:0/1456315837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2004144e90 msgr2=0x7f200415ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:59.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.054+0000 7f2009c00640 1 --2- 172.21.15.86:0/1456315837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2004144e90 0x7f200415ee10 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4009d90 tx=0x7f1ff402beb0 comp rx=0 tx=0).stop 2024-04-20T21:24:59.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.054+0000 7f2009c00640 1 -- 172.21.15.86:0/1456315837 shutdown_connections 2024-04-20T21:24:59.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.055+0000 7f2009c00640 1 --2- 172.21.15.86:0/1456315837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2004144e90 0x7f200415ee10 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:59.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.055+0000 7f2009c00640 1 -- 172.21.15.86:0/1456315837 >> 172.21.15.86:0/1456315837 conn(0x7f20040b2360 msgr2=0x7f20040b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:59.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.055+0000 7f2009c00640 1 -- 172.21.15.86:0/1456315837 shutdown_connections 2024-04-20T21:24:59.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.055+0000 7f2009c00640 1 -- 172.21.15.86:0/1456315837 wait complete. 2024-04-20T21:24:59.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.056+0000 7f2009c00640 1 Processor -- start 2024-04-20T21:24:59.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.057+0000 7f2009c00640 1 -- start start 2024-04-20T21:24:59.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.058+0000 7f2009c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f200406f140 0x7f2004135290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:59.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.058+0000 7f2009c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2004071d00 0x7f20040b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:59.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.058+0000 7f2009c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2004144e90 0x7f20040b1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:59.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.058+0000 7f2009c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f200415f350 con 0x7f200406f140 2024-04-20T21:24:59.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.058+0000 7f2009c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f200406ded0 con 0x7f2004144e90 2024-04-20T21:24:59.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.058+0000 7f2009c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f200415f4d0 con 0x7f2004071d00 2024-04-20T21:24:59.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.058+0000 7f2002a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f200406f140 0x7f2004135290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:59.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.058+0000 7f2002a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f200406f140 0x7f2004135290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43306/0 (socket says 172.21.15.86:43306) 2024-04-20T21:24:59.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.058+0000 7f2002a00640 1 -- 172.21.15.86:0/2077257767 learned_addr learned my addr 172.21.15.86:0/2077257767 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:59.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.058+0000 7f2002a00640 1 -- 172.21.15.86:0/2077257767 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2004071d00 msgr2=0x7f20040b08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:59.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.058+0000 7f1ffa000640 1 --2- 172.21.15.86:0/2077257767 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2004071d00 0x7f20040b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:59.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.058+0000 7f2002a00640 1 --2- 172.21.15.86:0/2077257767 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2004071d00 0x7f20040b08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:59.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.058+0000 7f2002a00640 1 -- 172.21.15.86:0/2077257767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2004144e90 msgr2=0x7f20040b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:59.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.058+0000 7f2003400640 1 --2- 172.21.15.86:0/2077257767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2004144e90 0x7f20040b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:59.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.058+0000 7f2002a00640 1 --2- 172.21.15.86:0/2077257767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2004144e90 0x7f20040b1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:59.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.059+0000 7f2002a00640 1 -- 172.21.15.86:0/2077257767 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20041357d0 con 0x7f200406f140 2024-04-20T21:24:59.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.059+0000 7f2002a00640 1 --2- 172.21.15.86:0/2077257767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f200406f140 0x7f2004135290 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4009d90 tx=0x7f1ff4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:59.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.059+0000 7f1ffb400640 1 -- 172.21.15.86:0/2077257767 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4044030 con 0x7f200406f140 2024-04-20T21:24:59.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.059+0000 7f1ffb400640 1 -- 172.21.15.86:0/2077257767 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ff403a040 con 0x7f200406f140 2024-04-20T21:24:59.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.059+0000 7f1ffb400640 1 -- 172.21.15.86:0/2077257767 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4039070 con 0x7f200406f140 2024-04-20T21:24:59.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.059+0000 7f2009c00640 1 -- 172.21.15.86:0/2077257767 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20040af820 con 0x7f200406f140 2024-04-20T21:24:59.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.059+0000 7f2009c00640 1 -- 172.21.15.86:0/2077257767 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20040afde0 con 0x7f200406f140 2024-04-20T21:24:59.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.060+0000 7f2009c00640 1 -- 172.21.15.86:0/2077257767 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20040689e0 con 0x7f200406f140 2024-04-20T21:24:59.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.061+0000 7f1ffb400640 1 -- 172.21.15.86:0/2077257767 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1ff4047070 con 0x7f200406f140 2024-04-20T21:24:59.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.062+0000 7f1ffb400640 1 --2- 172.21.15.86:0/2077257767 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1fe006c8f0 0x7f1fe006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:59.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.062+0000 7f1ffb400640 1 -- 172.21.15.86:0/2077257767 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7f1ff40ae360 con 0x7f200406f140 2024-04-20T21:24:59.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.063+0000 7f1ffa000640 1 --2- 172.21.15.86:0/2077257767 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1fe006c8f0 0x7f1fe006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:59.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.063+0000 7f1ffa000640 1 --2- 172.21.15.86:0/2077257767 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1fe006c8f0 0x7f1fe006edb0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f2004132580 tx=0x7f1fec009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:59.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.067+0000 7f1ffb400640 1 -- 172.21.15.86:0/2077257767 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1ff404e070 con 0x7f200406f140 2024-04-20T21:24:59.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.267+0000 7f2009c00640 1 -- 172.21.15.86:0/2077257767 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f20040728c0 con 0x7f200406f140 2024-04-20T21:24:59.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.270+0000 7f1ffb400640 1 -- 172.21.15.86:0/2077257767 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+16673 (secure 0 0 0) 0x7f1ff407c6f0 con 0x7f200406f140 2024-04-20T21:24:59.270 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:24:59.270 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":84,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:24:58.299327+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:24:55.964540+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-20T21:24:57.241527+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:24:59.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.272+0000 7f2009c00640 1 -- 172.21.15.86:0/2077257767 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1fe006c8f0 msgr2=0x7f1fe006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:59.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.272+0000 7f2009c00640 1 --2- 172.21.15.86:0/2077257767 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1fe006c8f0 0x7f1fe006edb0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f2004132580 tx=0x7f1fec009040 comp rx=0 tx=0).stop 2024-04-20T21:24:59.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.272+0000 7f2009c00640 1 -- 172.21.15.86:0/2077257767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f200406f140 msgr2=0x7f2004135290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:59.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.272+0000 7f2009c00640 1 --2- 172.21.15.86:0/2077257767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f200406f140 0x7f2004135290 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4009d90 tx=0x7f1ff4004a40 comp rx=0 tx=0).stop 2024-04-20T21:24:59.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.272+0000 7f2009c00640 1 -- 172.21.15.86:0/2077257767 shutdown_connections 2024-04-20T21:24:59.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.272+0000 7f2009c00640 1 --2- 172.21.15.86:0/2077257767 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1fe006c8f0 0x7f1fe006edb0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:59.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.272+0000 7f2009c00640 1 --2- 172.21.15.86:0/2077257767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2004144e90 0x7f20040b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:59.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.272+0000 7f2009c00640 1 --2- 172.21.15.86:0/2077257767 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2004071d00 0x7f20040b08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:59.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.272+0000 7f2009c00640 1 --2- 172.21.15.86:0/2077257767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f200406f140 0x7f2004135290 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:59.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.272+0000 7f2009c00640 1 -- 172.21.15.86:0/2077257767 >> 172.21.15.86:0/2077257767 conn(0x7f20040b2360 msgr2=0x7f2004072150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:59.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.272+0000 7f2009c00640 1 -- 172.21.15.86:0/2077257767 shutdown_connections 2024-04-20T21:24:59.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.273+0000 7f2009c00640 1 -- 172.21.15.86:0/2077257767 wait complete. 2024-04-20T21:24:59.292 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:24:59.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.493+0000 7fa818c00640 1 Processor -- start 2024-04-20T21:24:59.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.494+0000 7fa818c00640 1 -- start start 2024-04-20T21:24:59.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.495+0000 7fa818c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8140af7b0 0x7fa81415c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:59.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.495+0000 7fa818c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa81406f4c0 0x7fa81406f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:59.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.495+0000 7fa818c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8140ae710 con 0x7fa81415c010 2024-04-20T21:24:59.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.495+0000 7fa818c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa8140af320 con 0x7fa81406f4c0 2024-04-20T21:24:59.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.495+0000 7fa818c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa8140ae4b0 con 0x7fa8140af7b0 2024-04-20T21:24:59.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.495+0000 7fa813400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fa81415c010 0x7fa8140adda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:37942/0 (socket says 172.21.15.86:37942) 2024-04-20T21:24:59.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.495+0000 7fa813400640 1 -- 172.21.15.86:0/3534779294 learned_addr learned my addr 172.21.15.86:0/3534779294 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:24:59.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.495+0000 7fa813e00640 1 --2- 172.21.15.86:0/3534779294 >> v2:172.21.15.86:3300/0 conn(0x7fa81406f4c0 0x7fa81406f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:59.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.496+0000 7fa80aa00640 1 --2- 172.21.15.86:0/3534779294 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8140af7b0 0x7fa81415c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:59.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.496+0000 7fa812a00640 1 -- 172.21.15.86:0/3534779294 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4165286552 0 0) 0x7fa8140ae710 con 0x7fa81415c010 2024-04-20T21:24:59.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.496+0000 7fa812a00640 1 -- 172.21.15.86:0/3534779294 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa7f8003520 con 0x7fa81415c010 2024-04-20T21:24:59.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.496+0000 7fa813e00640 1 -- 172.21.15.86:0/3534779294 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8140af7b0 msgr2=0x7fa81415c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:59.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.496+0000 7fa813e00640 1 --2- 172.21.15.86:0/3534779294 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8140af7b0 0x7fa81415c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:59.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.496+0000 7fa813e00640 1 -- 172.21.15.86:0/3534779294 >> v1:172.21.15.86:6789/0 conn(0x7fa81415c010 legacy=0x7fa8140adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:59.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.496+0000 7fa813e00640 1 -- 172.21.15.86:0/3534779294 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8140ae9f0 con 0x7fa81406f4c0 2024-04-20T21:24:59.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.497+0000 7fa813e00640 1 --2- 172.21.15.86:0/3534779294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa81406f4c0 0x7fa81406f970 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fa80c009d90 tx=0x7fa80c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2f409c97b8fa516 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:59.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.497+0000 7fa812a00640 1 -- 172.21.15.86:0/3534779294 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80c039070 con 0x7fa81406f4c0 2024-04-20T21:24:59.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.497+0000 7fa812a00640 1 -- 172.21.15.86:0/3534779294 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa80c02cc50 con 0x7fa81406f4c0 2024-04-20T21:24:59.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.498+0000 7fa812a00640 1 -- 172.21.15.86:0/3534779294 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80c034440 con 0x7fa81406f4c0 2024-04-20T21:24:59.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.498+0000 7fa818c00640 1 -- 172.21.15.86:0/3534779294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa81406f4c0 msgr2=0x7fa81406f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:59.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.498+0000 7fa818c00640 1 --2- 172.21.15.86:0/3534779294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa81406f4c0 0x7fa81406f970 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fa80c009d90 tx=0x7fa80c02bea0 comp rx=0 tx=0).stop 2024-04-20T21:24:59.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.498+0000 7fa818c00640 1 -- 172.21.15.86:0/3534779294 shutdown_connections 2024-04-20T21:24:59.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.498+0000 7fa818c00640 1 --2- 172.21.15.86:0/3534779294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa81406f4c0 0x7fa81406f970 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:59.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.498+0000 7fa818c00640 1 --2- 172.21.15.86:0/3534779294 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8140af7b0 0x7fa81415c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:59.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.498+0000 7fa818c00640 1 -- 172.21.15.86:0/3534779294 >> 172.21.15.86:0/3534779294 conn(0x7fa8140d0170 msgr2=0x7fa8140d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:59.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.498+0000 7fa818c00640 1 -- 172.21.15.86:0/3534779294 shutdown_connections 2024-04-20T21:24:59.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.498+0000 7fa818c00640 1 -- 172.21.15.86:0/3534779294 wait complete. 2024-04-20T21:24:59.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.500+0000 7fa818c00640 1 Processor -- start 2024-04-20T21:24:59.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.500+0000 7fa818c00640 1 -- start start 2024-04-20T21:24:59.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.501+0000 7fa818c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa81406f4c0 0x7fa8140b4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:59.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.501+0000 7fa818c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8140af7b0 0x7fa8141f3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:59.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.501+0000 7fa818c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa81415c010 0x7fa81409b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:59.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.501+0000 7fa818c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa8140aefb0 con 0x7fa8140af7b0 2024-04-20T21:24:59.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.501+0000 7fa818c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa81415c930 con 0x7fa81406f4c0 2024-04-20T21:24:59.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.501+0000 7fa818c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa814071b70 con 0x7fa81415c010 2024-04-20T21:24:59.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.501+0000 7fa80aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8140af7b0 0x7fa8141f3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:59.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.501+0000 7fa80aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8140af7b0 0x7fa8141f3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43334/0 (socket says 172.21.15.86:43334) 2024-04-20T21:24:59.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.501+0000 7fa80aa00640 1 -- 172.21.15.86:0/2866051744 learned_addr learned my addr 172.21.15.86:0/2866051744 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:59.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.502+0000 7fa813400640 1 --2- 172.21.15.86:0/2866051744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa81406f4c0 0x7fa8140b4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:59.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.502+0000 7fa80aa00640 1 -- 172.21.15.86:0/2866051744 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa81415c010 msgr2=0x7fa81409b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:59.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.502+0000 7fa80aa00640 1 --2- 172.21.15.86:0/2866051744 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa81415c010 0x7fa81409b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:59.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.502+0000 7fa80aa00640 1 -- 172.21.15.86:0/2866051744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa81406f4c0 msgr2=0x7fa8140b4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:59.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.502+0000 7fa80aa00640 1 --2- 172.21.15.86:0/2866051744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa81406f4c0 0x7fa8140b4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:59.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.502+0000 7fa80aa00640 1 -- 172.21.15.86:0/2866051744 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8141575d0 con 0x7fa8140af7b0 2024-04-20T21:24:59.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.502+0000 7fa813400640 1 --2- 172.21.15.86:0/2866051744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa81406f4c0 0x7fa8140b4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:24:59.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.502+0000 7fa80aa00640 1 --2- 172.21.15.86:0/2866051744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8140af7b0 0x7fa8141f3d90 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc00d800 tx=0x7fa7fc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:59.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.502+0000 7fa80be00640 1 -- 172.21.15.86:0/2866051744 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc00b6c0 con 0x7fa8140af7b0 2024-04-20T21:24:59.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.502+0000 7fa80be00640 1 -- 172.21.15.86:0/2866051744 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7fc019070 con 0x7fa8140af7b0 2024-04-20T21:24:59.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.502+0000 7fa818c00640 1 -- 172.21.15.86:0/2866051744 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa81409a400 con 0x7fa8140af7b0 2024-04-20T21:24:59.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.502+0000 7fa80be00640 1 -- 172.21.15.86:0/2866051744 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc00ba80 con 0x7fa8140af7b0 2024-04-20T21:24:59.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.503+0000 7fa818c00640 1 -- 172.21.15.86:0/2866051744 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa81409a9c0 con 0x7fa8140af7b0 2024-04-20T21:24:59.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.504+0000 7fa818c00640 1 -- 172.21.15.86:0/2866051744 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa81404fcf0 con 0x7fa8140af7b0 2024-04-20T21:24:59.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.505+0000 7fa80be00640 1 -- 172.21.15.86:0/2866051744 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa7fc00bc20 con 0x7fa8140af7b0 2024-04-20T21:24:59.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.506+0000 7fa80be00640 1 --2- 172.21.15.86:0/2866051744 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa7f006c8f0 0x7fa7f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:59.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.506+0000 7fa80be00640 1 -- 172.21.15.86:0/2866051744 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7fa7fc08c090 con 0x7fa8140af7b0 2024-04-20T21:24:59.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.506+0000 7fa813400640 1 --2- 172.21.15.86:0/2866051744 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa7f006c8f0 0x7fa7f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:59.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.507+0000 7fa813400640 1 --2- 172.21.15.86:0/2866051744 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa7f006c8f0 0x7fa7f006edb0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa804005880 tx=0x7fa804009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:59.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.511+0000 7fa80be00640 1 -- 172.21.15.86:0/2866051744 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa7fc01c360 con 0x7fa8140af7b0 2024-04-20T21:24:59.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.713+0000 7fa818c00640 1 -- 172.21.15.86:0/2866051744 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa81406ddd0 con 0x7fa8140af7b0 2024-04-20T21:24:59.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.716+0000 7fa80be00640 1 -- 172.21.15.86:0/2866051744 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+16673 (secure 0 0 0) 0x7fa7fc05a420 con 0x7fa8140af7b0 2024-04-20T21:24:59.716 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:24:59.717 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":84,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:24:58.299327+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:24:55.964540+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-20T21:24:57.241527+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:24:59.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.719+0000 7fa818c00640 1 -- 172.21.15.86:0/2866051744 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa7f006c8f0 msgr2=0x7fa7f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:59.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.719+0000 7fa818c00640 1 --2- 172.21.15.86:0/2866051744 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa7f006c8f0 0x7fa7f006edb0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa804005880 tx=0x7fa804009040 comp rx=0 tx=0).stop 2024-04-20T21:24:59.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.719+0000 7fa818c00640 1 -- 172.21.15.86:0/2866051744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8140af7b0 msgr2=0x7fa8141f3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:59.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.719+0000 7fa818c00640 1 --2- 172.21.15.86:0/2866051744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8140af7b0 0x7fa8141f3d90 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc00d800 tx=0x7fa7fc00dcc0 comp rx=0 tx=0).stop 2024-04-20T21:24:59.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.719+0000 7fa818c00640 1 -- 172.21.15.86:0/2866051744 shutdown_connections 2024-04-20T21:24:59.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.719+0000 7fa818c00640 1 --2- 172.21.15.86:0/2866051744 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa7f006c8f0 0x7fa7f006edb0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:59.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.719+0000 7fa818c00640 1 --2- 172.21.15.86:0/2866051744 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa81415c010 0x7fa81409b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:59.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.719+0000 7fa818c00640 1 --2- 172.21.15.86:0/2866051744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8140af7b0 0x7fa8141f3d90 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:59.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.719+0000 7fa818c00640 1 --2- 172.21.15.86:0/2866051744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa81406f4c0 0x7fa8140b4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:59.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.719+0000 7fa818c00640 1 -- 172.21.15.86:0/2866051744 >> 172.21.15.86:0/2866051744 conn(0x7fa8140d0170 msgr2=0x7fa814072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:59.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.719+0000 7fa818c00640 1 -- 172.21.15.86:0/2866051744 shutdown_connections 2024-04-20T21:24:59.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.719+0000 7fa818c00640 1 -- 172.21.15.86:0/2866051744 wait complete. 2024-04-20T21:24:59.737 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-20T21:24:59.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.942+0000 7fdb96400640 1 Processor -- start 2024-04-20T21:24:59.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.942+0000 7fdb96400640 1 -- start start 2024-04-20T21:24:59.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.944+0000 7fdb96400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdb90146210 0x7fdb9006e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:59.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.944+0000 7fdb96400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdb9006bfd0 0x7fdb900cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:59.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.944+0000 7fdb96400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb9015cd40 con 0x7fdb9006bbf0 2024-04-20T21:24:59.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.944+0000 7fdb96400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fdb9015d000 con 0x7fdb9006bfd0 2024-04-20T21:24:59.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.944+0000 7fdb96400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fdb9015d2e0 con 0x7fdb90146210 2024-04-20T21:24:59.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.944+0000 7fdb8fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdb9006bfd0 0x7fdb900cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:59.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.944+0000 7fdb8fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdb9006bfd0 0x7fdb900cd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43338/0 (socket says 172.21.15.86:43338) 2024-04-20T21:24:59.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.944+0000 7fdb8fe00640 1 -- 172.21.15.86:0/3578077045 learned_addr learned my addr 172.21.15.86:0/3578077045 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:59.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.945+0000 7fdb8ea00640 1 --2- 172.21.15.86:0/3578077045 >> v2:172.21.15.144:3300/0 conn(0x7fdb90146210 0x7fdb9006e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:59.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.945+0000 7fdb8fe00640 1 -- 172.21.15.86:0/3578077045 >> v2:172.21.15.144:3300/0 conn(0x7fdb90146210 msgr2=0x7fdb9006e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:59.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.945+0000 7fdb8fe00640 1 --2- 172.21.15.86:0/3578077045 >> v2:172.21.15.144:3300/0 conn(0x7fdb90146210 0x7fdb9006e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:59.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.945+0000 7fdb8fe00640 1 -- 172.21.15.86:0/3578077045 >> v1:172.21.15.144:6789/0 conn(0x7fdb9006bbf0 legacy=0x7fdb90145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:24:59.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.945+0000 7fdb8fe00640 1 -- 172.21.15.86:0/3578077045 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb9015d5c0 con 0x7fdb9006bfd0 2024-04-20T21:24:59.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.945+0000 7fdb8ea00640 1 --2- 172.21.15.86:0/3578077045 >> v2:172.21.15.144:3300/0 conn(0x7fdb90146210 0x7fdb9006e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:24:59.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.946+0000 7fdb8fe00640 1 --2- 172.21.15.86:0/3578077045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb9006bfd0 0x7fdb900cd540 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fdb9006dcc0 tx=0x7fdb8802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5159b6b26ca18fd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:59.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.946+0000 7fdb8e000640 1 -- 172.21.15.86:0/3578077045 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8802ca80 con 0x7fdb9006bfd0 2024-04-20T21:24:59.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.947+0000 7fdb8e000640 1 -- 172.21.15.86:0/3578077045 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb8802cc20 con 0x7fdb9006bfd0 2024-04-20T21:24:59.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.947+0000 7fdb8e000640 1 -- 172.21.15.86:0/3578077045 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb880343f0 con 0x7fdb9006bfd0 2024-04-20T21:24:59.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.947+0000 7fdb96400640 1 -- 172.21.15.86:0/3578077045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb9006bfd0 msgr2=0x7fdb900cd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:59.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.947+0000 7fdb96400640 1 --2- 172.21.15.86:0/3578077045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb9006bfd0 0x7fdb900cd540 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fdb9006dcc0 tx=0x7fdb8802bee0 comp rx=0 tx=0).stop 2024-04-20T21:24:59.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.947+0000 7fdb96400640 1 -- 172.21.15.86:0/3578077045 shutdown_connections 2024-04-20T21:24:59.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.947+0000 7fdb96400640 1 --2- 172.21.15.86:0/3578077045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb9006bfd0 0x7fdb900cd540 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:59.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.947+0000 7fdb96400640 1 --2- 172.21.15.86:0/3578077045 >> v2:172.21.15.144:3300/0 conn(0x7fdb90146210 0x7fdb9006e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:59.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.947+0000 7fdb96400640 1 -- 172.21.15.86:0/3578077045 >> 172.21.15.86:0/3578077045 conn(0x7fdb900b2110 msgr2=0x7fdb900b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:24:59.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.947+0000 7fdb96400640 1 -- 172.21.15.86:0/3578077045 shutdown_connections 2024-04-20T21:24:59.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.947+0000 7fdb96400640 1 -- 172.21.15.86:0/3578077045 wait complete. 2024-04-20T21:24:59.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.949+0000 7fdb96400640 1 Processor -- start 2024-04-20T21:24:59.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.949+0000 7fdb96400640 1 -- start start 2024-04-20T21:24:59.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.950+0000 7fdb96400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb9006bbf0 0x7fdb90143330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:59.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.950+0000 7fdb96400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb9006bfd0 0x7fdb900ae0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:59.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.950+0000 7fdb96400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb90146210 0x7fdb900ae600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:59.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.950+0000 7fdb96400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdb900cdc00 con 0x7fdb90146210 2024-04-20T21:24:59.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.950+0000 7fdb96400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdb900cda80 con 0x7fdb9006bfd0 2024-04-20T21:24:59.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.950+0000 7fdb96400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdb900cdd80 con 0x7fdb9006bbf0 2024-04-20T21:24:59.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.950+0000 7fdb8fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb90146210 0x7fdb900ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:59.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.951+0000 7fdb8fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb90146210 0x7fdb900ae600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43340/0 (socket says 172.21.15.86:43340) 2024-04-20T21:24:59.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.951+0000 7fdb8fe00640 1 -- 172.21.15.86:0/1681860301 learned_addr learned my addr 172.21.15.86:0/1681860301 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:24:59.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.951+0000 7fdb8f400640 1 --2- 172.21.15.86:0/1681860301 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb9006bbf0 0x7fdb90143330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:59.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.951+0000 7fdb8ea00640 1 --2- 172.21.15.86:0/1681860301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb9006bfd0 0x7fdb900ae0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:59.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.951+0000 7fdb8fe00640 1 -- 172.21.15.86:0/1681860301 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb9006bbf0 msgr2=0x7fdb90143330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:59.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.951+0000 7fdb8fe00640 1 --2- 172.21.15.86:0/1681860301 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb9006bbf0 0x7fdb90143330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:59.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.951+0000 7fdb8fe00640 1 -- 172.21.15.86:0/1681860301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb9006bfd0 msgr2=0x7fdb900ae0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:24:59.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.951+0000 7fdb8fe00640 1 --2- 172.21.15.86:0/1681860301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb9006bfd0 0x7fdb900ae0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:24:59.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.951+0000 7fdb8fe00640 1 -- 172.21.15.86:0/1681860301 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb90143870 con 0x7fdb90146210 2024-04-20T21:24:59.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.951+0000 7fdb8fe00640 1 --2- 172.21.15.86:0/1681860301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb90146210 0x7fdb900ae600 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fdb88009f40 tx=0x7fdb88004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:59.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.952+0000 7fdb87400640 1 -- 172.21.15.86:0/1681860301 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb88043020 con 0x7fdb90146210 2024-04-20T21:24:59.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.952+0000 7fdb87400640 1 -- 172.21.15.86:0/1681860301 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb8803e450 con 0x7fdb90146210 2024-04-20T21:24:59.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.952+0000 7fdb87400640 1 -- 172.21.15.86:0/1681860301 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb88009510 con 0x7fdb90146210 2024-04-20T21:24:59.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.952+0000 7fdb96400640 1 -- 172.21.15.86:0/1681860301 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb900aa700 con 0x7fdb90146210 2024-04-20T21:24:59.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.952+0000 7fdb96400640 1 -- 172.21.15.86:0/1681860301 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb900aacc0 con 0x7fdb90146210 2024-04-20T21:24:59.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.953+0000 7fdb96400640 1 -- 172.21.15.86:0/1681860301 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb900689e0 con 0x7fdb90146210 2024-04-20T21:24:59.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.954+0000 7fdb87400640 1 -- 172.21.15.86:0/1681860301 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdb88046020 con 0x7fdb90146210 2024-04-20T21:24:59.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.955+0000 7fdb87400640 1 --2- 172.21.15.86:0/1681860301 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdb7006c8f0 0x7fdb7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:24:59.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.955+0000 7fdb87400640 1 -- 172.21.15.86:0/1681860301 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7fdb880ae3b0 con 0x7fdb90146210 2024-04-20T21:24:59.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.955+0000 7fdb8f400640 1 --2- 172.21.15.86:0/1681860301 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdb7006c8f0 0x7fdb7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:24:59.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.956+0000 7fdb8f400640 1 --2- 172.21.15.86:0/1681860301 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdb7006c8f0 0x7fdb7006edb0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fdb80004ea0 tx=0x7fdb80009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:24:59.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:24:59.959+0000 7fdb87400640 1 -- 172.21.15.86:0/1681860301 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fdb8804d030 con 0x7fdb90146210 2024-04-20T21:25:00.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:00.210+0000 7fdb96400640 1 -- 172.21.15.86:0/1681860301 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fdb9006c780 con 0x7fdb90146210 2024-04-20T21:25:00.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:00.868+0000 7fdb87400640 1 -- 172.21.15.86:0/1681860301 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v58) ==== 118+0+0 (secure 0 0 0) 0x7fdb8807c870 con 0x7fdb90146210 2024-04-20T21:25:00.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:00.872+0000 7fdb96400640 1 -- 172.21.15.86:0/1681860301 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdb7006c8f0 msgr2=0x7fdb7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:00.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:00.872+0000 7fdb96400640 1 --2- 172.21.15.86:0/1681860301 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdb7006c8f0 0x7fdb7006edb0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fdb80004ea0 tx=0x7fdb80009040 comp rx=0 tx=0).stop 2024-04-20T21:25:00.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:00.873+0000 7fdb96400640 1 -- 172.21.15.86:0/1681860301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb90146210 msgr2=0x7fdb900ae600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:00.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:00.873+0000 7fdb96400640 1 --2- 172.21.15.86:0/1681860301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb90146210 0x7fdb900ae600 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fdb88009f40 tx=0x7fdb88004a40 comp rx=0 tx=0).stop 2024-04-20T21:25:00.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:00.873+0000 7fdb96400640 1 -- 172.21.15.86:0/1681860301 shutdown_connections 2024-04-20T21:25:00.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:00.873+0000 7fdb96400640 1 --2- 172.21.15.86:0/1681860301 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdb7006c8f0 0x7fdb7006edb0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:00.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:00.873+0000 7fdb96400640 1 --2- 172.21.15.86:0/1681860301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb90146210 0x7fdb900ae600 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:00.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:00.873+0000 7fdb96400640 1 --2- 172.21.15.86:0/1681860301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb9006bfd0 0x7fdb900ae0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:00.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:00.873+0000 7fdb96400640 1 --2- 172.21.15.86:0/1681860301 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdb9006bbf0 0x7fdb90143330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:00.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:00.873+0000 7fdb96400640 1 -- 172.21.15.86:0/1681860301 >> 172.21.15.86:0/1681860301 conn(0x7fdb900b2110 msgr2=0x7fdb900727f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:00.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:00.873+0000 7fdb96400640 1 -- 172.21.15.86:0/1681860301 shutdown_connections 2024-04-20T21:25:00.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:00.873+0000 7fdb96400640 1 -- 172.21.15.86:0/1681860301 wait complete. 2024-04-20T21:25:00.893 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-20T21:25:00.894 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-20T21:25:01.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.092+0000 7f0c9be00640 1 Processor -- start 2024-04-20T21:25:01.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.092+0000 7f0c9be00640 1 -- start start 2024-04-20T21:25:01.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.093+0000 7f0c9be00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c9c0b3ba0 0x7f0c9c06eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:01.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.093+0000 7f0c9be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0c9c06f3f0 0x7f0c9c071ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:01.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.093+0000 7f0c9be00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c9c0cddb0 con 0x7f0c9c0cfc70 2024-04-20T21:25:01.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.093+0000 7f0c9be00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f0c9c15c8e0 con 0x7f0c9c06f3f0 2024-04-20T21:25:01.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.093+0000 7f0c9be00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0c9c06f820 con 0x7f0c9c0b3ba0 2024-04-20T21:25:01.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.094+0000 7f0c9aa00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f0c9c0cfc70 0x7f0c9c0cd6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:37958/0 (socket says 172.21.15.86:37958) 2024-04-20T21:25:01.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.094+0000 7f0c9aa00640 1 -- 172.21.15.86:0/950465223 learned_addr learned my addr 172.21.15.86:0/950465223 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:25:01.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.094+0000 7f0c9b400640 1 --2- 172.21.15.86:0/950465223 >> v2:172.21.15.86:3300/0 conn(0x7f0c9c06f3f0 0x7f0c9c071ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:01.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.094+0000 7f0c93e00640 1 --2- 172.21.15.86:0/950465223 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c9c0b3ba0 0x7f0c9c06eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:01.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.094+0000 7f0c9b400640 1 -- 172.21.15.86:0/950465223 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c9c0b3ba0 msgr2=0x7f0c9c06eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:01.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.094+0000 7f0c9a000640 1 -- 172.21.15.86:0/950465223 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3859715533 0 0) 0x7f0c9c0cddb0 con 0x7f0c9c0cfc70 2024-04-20T21:25:01.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.094+0000 7f0c9b400640 1 --2- 172.21.15.86:0/950465223 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c9c0b3ba0 0x7f0c9c06eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:01.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.094+0000 7f0c9b400640 1 -- 172.21.15.86:0/950465223 >> v1:172.21.15.86:6789/0 conn(0x7f0c9c0cfc70 legacy=0x7f0c9c0cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:01.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.094+0000 7f0c9b400640 1 -- 172.21.15.86:0/950465223 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c9c15bcd0 con 0x7f0c9c06f3f0 2024-04-20T21:25:01.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.095+0000 7f0c9b400640 1 --2- 172.21.15.86:0/950465223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c9c06f3f0 0x7f0c9c071ef0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f0c94009d90 tx=0x7f0c9402e0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2665becb2a0bb25d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:01.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.095+0000 7f0c9a000640 1 -- 172.21.15.86:0/950465223 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9403b070 con 0x7f0c9c06f3f0 2024-04-20T21:25:01.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.095+0000 7f0c9a000640 1 -- 172.21.15.86:0/950465223 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c94036440 con 0x7f0c9c06f3f0 2024-04-20T21:25:01.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.095+0000 7f0c9a000640 1 -- 172.21.15.86:0/950465223 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9402ee60 con 0x7f0c9c06f3f0 2024-04-20T21:25:01.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.096+0000 7f0c9be00640 1 -- 172.21.15.86:0/950465223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c9c06f3f0 msgr2=0x7f0c9c071ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:01.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.096+0000 7f0c9be00640 1 --2- 172.21.15.86:0/950465223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c9c06f3f0 0x7f0c9c071ef0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f0c94009d90 tx=0x7f0c9402e0b0 comp rx=0 tx=0).stop 2024-04-20T21:25:01.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.096+0000 7f0c9be00640 1 -- 172.21.15.86:0/950465223 shutdown_connections 2024-04-20T21:25:01.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.096+0000 7f0c9be00640 1 --2- 172.21.15.86:0/950465223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c9c06f3f0 0x7f0c9c071ef0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:01.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.096+0000 7f0c9be00640 1 --2- 172.21.15.86:0/950465223 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c9c0b3ba0 0x7f0c9c06eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:01.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.096+0000 7f0c9be00640 1 -- 172.21.15.86:0/950465223 >> 172.21.15.86:0/950465223 conn(0x7f0c9c067cf0 msgr2=0x7f0c9c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:01.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.096+0000 7f0c9be00640 1 -- 172.21.15.86:0/950465223 shutdown_connections 2024-04-20T21:25:01.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.096+0000 7f0c9be00640 1 -- 172.21.15.86:0/950465223 wait complete. 2024-04-20T21:25:01.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.098+0000 7f0c9be00640 1 Processor -- start 2024-04-20T21:25:01.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.098+0000 7f0c9be00640 1 -- start start 2024-04-20T21:25:01.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.099+0000 7f0c9be00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c9c06f3f0 0x7f0c9c15a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:01.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.099+0000 7f0c9be00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c9c0b3ba0 0x7f0c9c06dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:01.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.099+0000 7f0c9be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c9c0cfc70 0x7f0c9c06e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:01.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.099+0000 7f0c9be00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0c9c15c570 con 0x7f0c9c0cfc70 2024-04-20T21:25:01.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.099+0000 7f0c9be00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0c9c15c290 con 0x7f0c9c06f3f0 2024-04-20T21:25:01.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.099+0000 7f0c9be00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0c9c0b3710 con 0x7f0c9c0b3ba0 2024-04-20T21:25:01.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.099+0000 7f0c9b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c9c0cfc70 0x7f0c9c06e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:01.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.099+0000 7f0c9b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c9c0cfc70 0x7f0c9c06e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43362/0 (socket says 172.21.15.86:43362) 2024-04-20T21:25:01.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.099+0000 7f0c9b400640 1 -- 172.21.15.86:0/1034259469 learned_addr learned my addr 172.21.15.86:0/1034259469 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:01.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.100+0000 7f0c93e00640 1 --2- 172.21.15.86:0/1034259469 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c9c0b3ba0 0x7f0c9c06dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:01.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.100+0000 7f0c9b400640 1 -- 172.21.15.86:0/1034259469 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c9c0b3ba0 msgr2=0x7f0c9c06dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:01.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.100+0000 7f0c9aa00640 1 --2- 172.21.15.86:0/1034259469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c9c06f3f0 0x7f0c9c15a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:01.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.100+0000 7f0c9b400640 1 --2- 172.21.15.86:0/1034259469 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c9c0b3ba0 0x7f0c9c06dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:01.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.100+0000 7f0c9b400640 1 -- 172.21.15.86:0/1034259469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c9c06f3f0 msgr2=0x7f0c9c15a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:01.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.100+0000 7f0c9b400640 1 --2- 172.21.15.86:0/1034259469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c9c06f3f0 0x7f0c9c15a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:01.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.100+0000 7f0c9b400640 1 -- 172.21.15.86:0/1034259469 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c9c13f550 con 0x7f0c9c0cfc70 2024-04-20T21:25:01.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.100+0000 7f0c9b400640 1 --2- 172.21.15.86:0/1034259469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c9c0cfc70 0x7f0c9c06e2e0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f0c94009d90 tx=0x7f0c94004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:01.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.100+0000 7f0c92a00640 1 -- 172.21.15.86:0/1034259469 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9403b050 con 0x7f0c9c0cfc70 2024-04-20T21:25:01.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.100+0000 7f0c92a00640 1 -- 172.21.15.86:0/1034259469 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c94047070 con 0x7f0c9c0cfc70 2024-04-20T21:25:01.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.100+0000 7f0c92a00640 1 -- 172.21.15.86:0/1034259469 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c94034d40 con 0x7f0c9c0cfc70 2024-04-20T21:25:01.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.100+0000 7f0c9be00640 1 -- 172.21.15.86:0/1034259469 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c9c153eb0 con 0x7f0c9c0cfc70 2024-04-20T21:25:01.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.101+0000 7f0c9be00640 1 -- 172.21.15.86:0/1034259469 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c9c13fae0 con 0x7f0c9c0cfc70 2024-04-20T21:25:01.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.102+0000 7f0c9be00640 1 -- 172.21.15.86:0/1034259469 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c9c04fd90 con 0x7f0c9c0cfc70 2024-04-20T21:25:01.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.103+0000 7f0c92a00640 1 -- 172.21.15.86:0/1034259469 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0c9404c460 con 0x7f0c9c0cfc70 2024-04-20T21:25:01.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.104+0000 7f0c92a00640 1 --2- 172.21.15.86:0/1034259469 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0c7806c8f0 0x7f0c7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:01.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.104+0000 7f0c92a00640 1 -- 172.21.15.86:0/1034259469 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7f0c940b0cf0 con 0x7f0c9c0cfc70 2024-04-20T21:25:01.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.104+0000 7f0c9aa00640 1 --2- 172.21.15.86:0/1034259469 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0c7806c8f0 0x7f0c7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:01.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.105+0000 7f0c9aa00640 1 --2- 172.21.15.86:0/1034259469 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0c7806c8f0 0x7f0c7806edb0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c007fd0 tx=0x7f0c8c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:01.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.108+0000 7f0c92a00640 1 -- 172.21.15.86:0/1034259469 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0c94036e20 con 0x7f0c9c0cfc70 2024-04-20T21:25:01.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.353+0000 7f0c9be00640 1 -- 172.21.15.86:0/1034259469 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f0c9c068e70 con 0x7f0c9c0cfc70 2024-04-20T21:25:01.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.872+0000 7f0c92a00640 1 -- 172.21.15.86:0/1034259469 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v59) ==== 122+0+0 (secure 0 0 0) 0x7f0c9407f1d0 con 0x7f0c9c0cfc70 2024-04-20T21:25:01.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.876+0000 7f0c9be00640 1 -- 172.21.15.86:0/1034259469 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0c7806c8f0 msgr2=0x7f0c7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:01.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.876+0000 7f0c9be00640 1 --2- 172.21.15.86:0/1034259469 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0c7806c8f0 0x7f0c7806edb0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c007fd0 tx=0x7f0c8c009040 comp rx=0 tx=0).stop 2024-04-20T21:25:01.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.876+0000 7f0c9be00640 1 -- 172.21.15.86:0/1034259469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c9c0cfc70 msgr2=0x7f0c9c06e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:01.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.876+0000 7f0c9be00640 1 --2- 172.21.15.86:0/1034259469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c9c0cfc70 0x7f0c9c06e2e0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f0c94009d90 tx=0x7f0c94004a40 comp rx=0 tx=0).stop 2024-04-20T21:25:01.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.876+0000 7f0c9be00640 1 -- 172.21.15.86:0/1034259469 shutdown_connections 2024-04-20T21:25:01.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.876+0000 7f0c9be00640 1 --2- 172.21.15.86:0/1034259469 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0c7806c8f0 0x7f0c7806edb0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:01.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.876+0000 7f0c9be00640 1 --2- 172.21.15.86:0/1034259469 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c9c0cfc70 0x7f0c9c06e2e0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:01.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.876+0000 7f0c9be00640 1 --2- 172.21.15.86:0/1034259469 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c9c0b3ba0 0x7f0c9c06dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:01.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.876+0000 7f0c9be00640 1 --2- 172.21.15.86:0/1034259469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c9c06f3f0 0x7f0c9c15a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:01.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.876+0000 7f0c9be00640 1 -- 172.21.15.86:0/1034259469 >> 172.21.15.86:0/1034259469 conn(0x7f0c9c067cf0 msgr2=0x7f0c9c14f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:01.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.876+0000 7f0c9be00640 1 -- 172.21.15.86:0/1034259469 shutdown_connections 2024-04-20T21:25:01.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:01.876+0000 7f0c9be00640 1 -- 172.21.15.86:0/1034259469 wait complete. 2024-04-20T21:25:01.897 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-20T21:25:01.897 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:25:02.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.093+0000 7fa539000640 1 Processor -- start 2024-04-20T21:25:02.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.093+0000 7fa539000640 1 -- start start 2024-04-20T21:25:02.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.095+0000 7fa539000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa534144e90 0x7fa5340b4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:02.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.095+0000 7fa539000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa5340cd170 0x7fa53406ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:02.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.095+0000 7fa539000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5340cdb50 con 0x7fa5340cd5d0 2024-04-20T21:25:02.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.095+0000 7fa539000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fa534145270 con 0x7fa5340cd170 2024-04-20T21:25:02.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.095+0000 7fa539000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa5340b3e50 con 0x7fa534144e90 2024-04-20T21:25:02.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.096+0000 7fa531e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa534144e90 0x7fa5340b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:02.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.096+0000 7fa532800640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fa5340cd5d0 0x7fa53415bb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.86:55582/0 (socket says 172.21.15.86:55582) 2024-04-20T21:25:02.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.096+0000 7fa532800640 1 -- 172.21.15.86:0/771142646 learned_addr learned my addr 172.21.15.86:0/771142646 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:25:02.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.096+0000 7fa531400640 1 --2- 172.21.15.86:0/771142646 >> v2:172.21.15.144:3300/0 conn(0x7fa5340cd170 0x7fa53406ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:02.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.097+0000 7fa530a00640 1 -- 172.21.15.86:0/771142646 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 957750994 0 0) 0x7fa5340cdb50 con 0x7fa5340cd5d0 2024-04-20T21:25:02.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.097+0000 7fa530a00640 1 -- 172.21.15.86:0/771142646 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa518005730 con 0x7fa5340cd5d0 2024-04-20T21:25:02.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.097+0000 7fa531e00640 1 -- 172.21.15.86:0/771142646 >> v2:172.21.15.144:3300/0 conn(0x7fa5340cd170 msgr2=0x7fa53406ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:02.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.097+0000 7fa531e00640 1 --2- 172.21.15.86:0/771142646 >> v2:172.21.15.144:3300/0 conn(0x7fa5340cd170 0x7fa53406ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:02.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.097+0000 7fa531e00640 1 -- 172.21.15.86:0/771142646 >> v1:172.21.15.144:6789/0 conn(0x7fa5340cd5d0 legacy=0x7fa53415bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:02.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.097+0000 7fa531e00640 1 -- 172.21.15.86:0/771142646 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5340b4130 con 0x7fa534144e90 2024-04-20T21:25:02.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.098+0000 7fa531e00640 1 --2- 172.21.15.86:0/771142646 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa534144e90 0x7fa5340b4930 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa5240097a0 tx=0x7fa52402b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39fe642e70cd7fba server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:02.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.099+0000 7fa530a00640 1 -- 172.21.15.86:0/771142646 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5240315a0 con 0x7fa534144e90 2024-04-20T21:25:02.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.099+0000 7fa530a00640 1 -- 172.21.15.86:0/771142646 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa524037070 con 0x7fa534144e90 2024-04-20T21:25:02.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.099+0000 7fa530a00640 1 -- 172.21.15.86:0/771142646 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa524031940 con 0x7fa534144e90 2024-04-20T21:25:02.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.099+0000 7fa539000640 1 -- 172.21.15.86:0/771142646 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa534144e90 msgr2=0x7fa5340b4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:02.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.099+0000 7fa539000640 1 --2- 172.21.15.86:0/771142646 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa534144e90 0x7fa5340b4930 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa5240097a0 tx=0x7fa52402b840 comp rx=0 tx=0).stop 2024-04-20T21:25:02.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.099+0000 7fa539000640 1 -- 172.21.15.86:0/771142646 shutdown_connections 2024-04-20T21:25:02.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.099+0000 7fa539000640 1 --2- 172.21.15.86:0/771142646 >> v2:172.21.15.144:3300/0 conn(0x7fa5340cd170 0x7fa53406ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:02.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.099+0000 7fa539000640 1 --2- 172.21.15.86:0/771142646 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa534144e90 0x7fa5340b4930 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:02.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.099+0000 7fa539000640 1 -- 172.21.15.86:0/771142646 >> 172.21.15.86:0/771142646 conn(0x7fa5340b2360 msgr2=0x7fa5340b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:02.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.099+0000 7fa539000640 1 -- 172.21.15.86:0/771142646 shutdown_connections 2024-04-20T21:25:02.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.099+0000 7fa539000640 1 -- 172.21.15.86:0/771142646 wait complete. 2024-04-20T21:25:02.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.101+0000 7fa539000640 1 Processor -- start 2024-04-20T21:25:02.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.101+0000 7fa539000640 1 -- start start 2024-04-20T21:25:02.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.102+0000 7fa539000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5340cd170 0x7fa534138fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:02.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.102+0000 7fa539000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5340cd5d0 0x7fa5341f3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:02.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.102+0000 7fa539000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa534144e90 0x7fa5340ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:02.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.102+0000 7fa539000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa5340d0010 con 0x7fa534144e90 2024-04-20T21:25:02.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.102+0000 7fa539000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa5340b4e70 con 0x7fa5340cd5d0 2024-04-20T21:25:02.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.102+0000 7fa539000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa5340d0190 con 0x7fa5340cd170 2024-04-20T21:25:02.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.102+0000 7fa532800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa534144e90 0x7fa5340ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:02.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.103+0000 7fa532800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa534144e90 0x7fa5340ae8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43376/0 (socket says 172.21.15.86:43376) 2024-04-20T21:25:02.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.103+0000 7fa532800640 1 -- 172.21.15.86:0/3653705146 learned_addr learned my addr 172.21.15.86:0/3653705146 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:02.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.103+0000 7fa531e00640 1 --2- 172.21.15.86:0/3653705146 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5340cd170 0x7fa534138fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:02.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.103+0000 7fa531400640 1 --2- 172.21.15.86:0/3653705146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5340cd5d0 0x7fa5341f3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:02.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.103+0000 7fa532800640 1 -- 172.21.15.86:0/3653705146 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5340cd170 msgr2=0x7fa534138fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:02.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.103+0000 7fa532800640 1 --2- 172.21.15.86:0/3653705146 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5340cd170 0x7fa534138fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:02.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.103+0000 7fa532800640 1 -- 172.21.15.86:0/3653705146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5340cd5d0 msgr2=0x7fa5341f3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:02.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.103+0000 7fa532800640 1 --2- 172.21.15.86:0/3653705146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5340cd5d0 0x7fa5341f3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:02.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.103+0000 7fa532800640 1 -- 172.21.15.86:0/3653705146 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5341f4840 con 0x7fa534144e90 2024-04-20T21:25:02.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.103+0000 7fa531400640 1 --2- 172.21.15.86:0/3653705146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5340cd5d0 0x7fa5341f3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:25:02.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.103+0000 7fa532800640 1 --2- 172.21.15.86:0/3653705146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa534144e90 0x7fa5340ae8a0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fa52c00c430 tx=0x7fa52c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:02.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.103+0000 7fa52a000640 1 -- 172.21.15.86:0/3653705146 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52c017070 con 0x7fa534144e90 2024-04-20T21:25:02.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.104+0000 7fa52a000640 1 -- 172.21.15.86:0/3653705146 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa52c005ce0 con 0x7fa534144e90 2024-04-20T21:25:02.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.104+0000 7fa52a000640 1 -- 172.21.15.86:0/3653705146 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52c007310 con 0x7fa534144e90 2024-04-20T21:25:02.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.104+0000 7fa539000640 1 -- 172.21.15.86:0/3653705146 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5341f4b20 con 0x7fa534144e90 2024-04-20T21:25:02.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.104+0000 7fa539000640 1 -- 172.21.15.86:0/3653705146 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa534138a80 con 0x7fa534144e90 2024-04-20T21:25:02.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.105+0000 7fa539000640 1 -- 172.21.15.86:0/3653705146 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5340689e0 con 0x7fa534144e90 2024-04-20T21:25:02.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.106+0000 7fa52a000640 1 -- 172.21.15.86:0/3653705146 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa52c01a070 con 0x7fa534144e90 2024-04-20T21:25:02.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.107+0000 7fa52a000640 1 --2- 172.21.15.86:0/3653705146 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa51006c8f0 0x7fa51006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:02.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.107+0000 7fa52a000640 1 -- 172.21.15.86:0/3653705146 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7fa52c08b0c0 con 0x7fa534144e90 2024-04-20T21:25:02.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.107+0000 7fa531e00640 1 --2- 172.21.15.86:0/3653705146 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa51006c8f0 0x7fa51006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:02.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.108+0000 7fa531e00640 1 --2- 172.21.15.86:0/3653705146 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa51006c8f0 0x7fa51006edb0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa524036000 tx=0x7fa524008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:02.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.111+0000 7fa52a000640 1 -- 172.21.15.86:0/3653705146 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa52c059450 con 0x7fa534144e90 2024-04-20T21:25:02.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.353+0000 7fa539000640 1 -- 172.21.15.86:0/3653705146 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa5340c96d0 con 0x7fa534144e90 2024-04-20T21:25:02.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.354+0000 7fa52a000640 1 -- 172.21.15.86:0/3653705146 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5207 (secure 0 0 0) 0x7fa52c058d50 con 0x7fa534144e90 2024-04-20T21:25:02.354 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:25:02.354 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":59,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7506,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/4013863257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4013863257},{"type":"v1","addr":"172.21.15.86:6836","nonce":4013863257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":7509,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2044883073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2044883073},{"type":"v1","addr":"172.21.15.86:6837","nonce":2044883073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7518,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/2351381466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2351381466},{"type":"v1","addr":"172.21.15.144:6834","nonce":2351381466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:24:58.299257+0000","modified":"2024-04-20T21:25:01.355818+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7521":{"gid":7521,"name":"b","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6835/1740492227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":1740492227},{"type":"v1","addr":"172.21.15.144:6835","nonce":1740492227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7521,"qdb_cluster":[7521]},"id":6}]} 2024-04-20T21:25:02.355 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 59 2024-04-20T21:25:02.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.356+0000 7fa539000640 1 -- 172.21.15.86:0/3653705146 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa51006c8f0 msgr2=0x7fa51006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:02.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.356+0000 7fa539000640 1 --2- 172.21.15.86:0/3653705146 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa51006c8f0 0x7fa51006edb0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa524036000 tx=0x7fa524008030 comp rx=0 tx=0).stop 2024-04-20T21:25:02.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.356+0000 7fa539000640 1 -- 172.21.15.86:0/3653705146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa534144e90 msgr2=0x7fa5340ae8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:02.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.356+0000 7fa539000640 1 --2- 172.21.15.86:0/3653705146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa534144e90 0x7fa5340ae8a0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fa52c00c430 tx=0x7fa52c00c8f0 comp rx=0 tx=0).stop 2024-04-20T21:25:02.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.356+0000 7fa539000640 1 -- 172.21.15.86:0/3653705146 shutdown_connections 2024-04-20T21:25:02.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.356+0000 7fa539000640 1 --2- 172.21.15.86:0/3653705146 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa51006c8f0 0x7fa51006edb0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:02.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.356+0000 7fa539000640 1 --2- 172.21.15.86:0/3653705146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa534144e90 0x7fa5340ae8a0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:02.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.356+0000 7fa539000640 1 --2- 172.21.15.86:0/3653705146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5340cd5d0 0x7fa5341f3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:02.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.356+0000 7fa539000640 1 --2- 172.21.15.86:0/3653705146 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5340cd170 0x7fa534138fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:02.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.356+0000 7fa539000640 1 -- 172.21.15.86:0/3653705146 >> 172.21.15.86:0/3653705146 conn(0x7fa5340b2360 msgr2=0x7fa534142590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:02.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.357+0000 7fa539000640 1 -- 172.21.15.86:0/3653705146 shutdown_connections 2024-04-20T21:25:02.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.357+0000 7fa539000640 1 -- 172.21.15.86:0/3653705146 wait complete. 2024-04-20T21:25:02.376 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:25:02.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.577+0000 7fd07fe00640 1 Processor -- start 2024-04-20T21:25:02.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.577+0000 7fd07fe00640 1 -- start start 2024-04-20T21:25:02.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.579+0000 7fd07fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd08015d6a0 0x7fd0800ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:02.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.579+0000 7fd07fe00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd0800ad620 0x7fd08006e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:02.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.579+0000 7fd07fe00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd080160890 con 0x7fd080156ae0 2024-04-20T21:25:02.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.579+0000 7fd07fe00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd08006db20 con 0x7fd08015d6a0 2024-04-20T21:25:02.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.579+0000 7fd07fe00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fd0800ada50 con 0x7fd0800ad620 2024-04-20T21:25:02.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.579+0000 7fd076000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd08015d6a0 0x7fd0800ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:02.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.579+0000 7fd076000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd08015d6a0 0x7fd0800ad0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43388/0 (socket says 172.21.15.86:43388) 2024-04-20T21:25:02.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.579+0000 7fd076000640 1 -- 172.21.15.86:0/4100236202 learned_addr learned my addr 172.21.15.86:0/4100236202 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:02.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.579+0000 7fd07ea00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd080156ae0 0x7fd080160080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:37970/0 (socket says 172.21.15.86:37970) 2024-04-20T21:25:02.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.580+0000 7fd07f400640 1 --2- 172.21.15.86:0/4100236202 >> v2:172.21.15.144:3300/0 conn(0x7fd0800ad620 0x7fd08006e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:02.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.580+0000 7fd076000640 1 -- 172.21.15.86:0/4100236202 >> v2:172.21.15.144:3300/0 conn(0x7fd0800ad620 msgr2=0x7fd08006e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:02.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.580+0000 7fd076000640 1 --2- 172.21.15.86:0/4100236202 >> v2:172.21.15.144:3300/0 conn(0x7fd0800ad620 0x7fd08006e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:02.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.580+0000 7fd076000640 1 -- 172.21.15.86:0/4100236202 >> v1:172.21.15.86:6789/0 conn(0x7fd080156ae0 legacy=0x7fd080160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:02.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.580+0000 7fd076000640 1 -- 172.21.15.86:0/4100236202 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd080160b70 con 0x7fd08015d6a0 2024-04-20T21:25:02.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.580+0000 7fd07f400640 1 --2- 172.21.15.86:0/4100236202 >> v2:172.21.15.144:3300/0 conn(0x7fd0800ad620 0x7fd08006e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:25:02.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.581+0000 7fd076000640 1 --2- 172.21.15.86:0/4100236202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd08015d6a0 0x7fd0800ad0e0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fd0800d0e10 tx=0x7fd06802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba4ed86a338bda95 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:02.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.581+0000 7fd07e000640 1 -- 172.21.15.86:0/4100236202 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd068034070 con 0x7fd08015d6a0 2024-04-20T21:25:02.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.581+0000 7fd07e000640 1 -- 172.21.15.86:0/4100236202 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd068030860 con 0x7fd08015d6a0 2024-04-20T21:25:02.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.581+0000 7fd07e000640 1 -- 172.21.15.86:0/4100236202 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd068030c30 con 0x7fd08015d6a0 2024-04-20T21:25:02.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.582+0000 7fd07fe00640 1 -- 172.21.15.86:0/4100236202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd08015d6a0 msgr2=0x7fd0800ad0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:02.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.582+0000 7fd07fe00640 1 --2- 172.21.15.86:0/4100236202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd08015d6a0 0x7fd0800ad0e0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fd0800d0e10 tx=0x7fd06802b9a0 comp rx=0 tx=0).stop 2024-04-20T21:25:02.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.582+0000 7fd07fe00640 1 -- 172.21.15.86:0/4100236202 shutdown_connections 2024-04-20T21:25:02.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.582+0000 7fd07fe00640 1 --2- 172.21.15.86:0/4100236202 >> v2:172.21.15.144:3300/0 conn(0x7fd0800ad620 0x7fd08006e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:02.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.582+0000 7fd07fe00640 1 --2- 172.21.15.86:0/4100236202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd08015d6a0 0x7fd0800ad0e0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:02.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.582+0000 7fd07fe00640 1 -- 172.21.15.86:0/4100236202 >> 172.21.15.86:0/4100236202 conn(0x7fd0800d0170 msgr2=0x7fd0800d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:02.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.582+0000 7fd07fe00640 1 -- 172.21.15.86:0/4100236202 shutdown_connections 2024-04-20T21:25:02.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.582+0000 7fd07fe00640 1 -- 172.21.15.86:0/4100236202 wait complete. 2024-04-20T21:25:02.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.584+0000 7fd07fe00640 1 Processor -- start 2024-04-20T21:25:02.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.585+0000 7fd07fe00640 1 -- start start 2024-04-20T21:25:02.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.586+0000 7fd07fe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0800ad620 0x7fd080142da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:02.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.586+0000 7fd07fe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd080156ae0 0x7fd0801592b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:02.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.586+0000 7fd07fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd08015d6a0 0x7fd08015ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:02.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.586+0000 7fd07fe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd080156ec0 con 0x7fd08015d6a0 2024-04-20T21:25:02.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.586+0000 7fd07fe00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd08006dd70 con 0x7fd080156ae0 2024-04-20T21:25:02.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.586+0000 7fd07fe00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd08006e7d0 con 0x7fd0800ad620 2024-04-20T21:25:02.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.586+0000 7fd07f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd08015d6a0 0x7fd08015ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:02.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.586+0000 7fd07f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd08015d6a0 0x7fd08015ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43390/0 (socket says 172.21.15.86:43390) 2024-04-20T21:25:02.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.586+0000 7fd07f400640 1 -- 172.21.15.86:0/3002749491 learned_addr learned my addr 172.21.15.86:0/3002749491 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:02.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.586+0000 7fd07ea00640 1 --2- 172.21.15.86:0/3002749491 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0800ad620 0x7fd080142da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:02.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.586+0000 7fd07f400640 1 -- 172.21.15.86:0/3002749491 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0800ad620 msgr2=0x7fd080142da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:02.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.586+0000 7fd076000640 1 --2- 172.21.15.86:0/3002749491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd080156ae0 0x7fd0801592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:02.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.587+0000 7fd07f400640 1 --2- 172.21.15.86:0/3002749491 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0800ad620 0x7fd080142da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:02.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.587+0000 7fd07f400640 1 -- 172.21.15.86:0/3002749491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd080156ae0 msgr2=0x7fd0801592b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:02.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.587+0000 7fd07f400640 1 --2- 172.21.15.86:0/3002749491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd080156ae0 0x7fd0801592b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:02.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.587+0000 7fd07f400640 1 -- 172.21.15.86:0/3002749491 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd080155fe0 con 0x7fd08015d6a0 2024-04-20T21:25:02.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.587+0000 7fd076000640 1 --2- 172.21.15.86:0/3002749491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd080156ae0 0x7fd0801592b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:25:02.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.587+0000 7fd07f400640 1 --2- 172.21.15.86:0/3002749491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd08015d6a0 0x7fd08015ad80 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fd07800cef0 tx=0x7fd07800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:02.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.587+0000 7fd077e00640 1 -- 172.21.15.86:0/3002749491 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07801a070 con 0x7fd08015d6a0 2024-04-20T21:25:02.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.587+0000 7fd077e00640 1 -- 172.21.15.86:0/3002749491 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd07800d860 con 0x7fd08015d6a0 2024-04-20T21:25:02.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.587+0000 7fd077e00640 1 -- 172.21.15.86:0/3002749491 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07800dc50 con 0x7fd08015d6a0 2024-04-20T21:25:02.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.587+0000 7fd07fe00640 1 -- 172.21.15.86:0/3002749491 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd080146040 con 0x7fd08015d6a0 2024-04-20T21:25:02.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.587+0000 7fd07fe00640 1 -- 172.21.15.86:0/3002749491 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd080145a80 con 0x7fd08015d6a0 2024-04-20T21:25:02.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.589+0000 7fd07fe00640 1 -- 172.21.15.86:0/3002749491 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd08004fcf0 con 0x7fd08015d6a0 2024-04-20T21:25:02.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.590+0000 7fd077e00640 1 -- 172.21.15.86:0/3002749491 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd07801d070 con 0x7fd08015d6a0 2024-04-20T21:25:02.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.590+0000 7fd077e00640 1 --2- 172.21.15.86:0/3002749491 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd05c06c920 0x7fd05c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:02.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.591+0000 7fd077e00640 1 -- 172.21.15.86:0/3002749491 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7fd07808c510 con 0x7fd08015d6a0 2024-04-20T21:25:02.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.591+0000 7fd07ea00640 1 --2- 172.21.15.86:0/3002749491 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd05c06c920 0x7fd05c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:02.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.591+0000 7fd07ea00640 1 --2- 172.21.15.86:0/3002749491 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd05c06c920 0x7fd05c06ede0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd070000f60 tx=0x7fd070009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:02.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.595+0000 7fd077e00640 1 -- 172.21.15.86:0/3002749491 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd07805a8d0 con 0x7fd08015d6a0 2024-04-20T21:25:02.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.796+0000 7fd07fe00640 1 -- 172.21.15.86:0/3002749491 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd08006f1f0 con 0x7fd08015d6a0 2024-04-20T21:25:02.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.798+0000 7fd077e00640 1 -- 172.21.15.86:0/3002749491 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+16660 (secure 0 0 0) 0x7fd07805a1d0 con 0x7fd08015d6a0 2024-04-20T21:25:02.799 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:25:02.799 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":85,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:24:59.846179+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:24:55.964540+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-20T21:24:57.241527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:25:02.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.801+0000 7fd07fe00640 1 -- 172.21.15.86:0/3002749491 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd05c06c920 msgr2=0x7fd05c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:02.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.801+0000 7fd07fe00640 1 --2- 172.21.15.86:0/3002749491 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd05c06c920 0x7fd05c06ede0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd070000f60 tx=0x7fd070009040 comp rx=0 tx=0).stop 2024-04-20T21:25:02.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.801+0000 7fd07fe00640 1 -- 172.21.15.86:0/3002749491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd08015d6a0 msgr2=0x7fd08015ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:02.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.801+0000 7fd07fe00640 1 --2- 172.21.15.86:0/3002749491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd08015d6a0 0x7fd08015ad80 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fd07800cef0 tx=0x7fd07800a710 comp rx=0 tx=0).stop 2024-04-20T21:25:02.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.801+0000 7fd07fe00640 1 -- 172.21.15.86:0/3002749491 shutdown_connections 2024-04-20T21:25:02.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.801+0000 7fd07fe00640 1 --2- 172.21.15.86:0/3002749491 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd05c06c920 0x7fd05c06ede0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:02.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.801+0000 7fd07fe00640 1 --2- 172.21.15.86:0/3002749491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd08015d6a0 0x7fd08015ad80 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:02.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.801+0000 7fd07fe00640 1 --2- 172.21.15.86:0/3002749491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd080156ae0 0x7fd0801592b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:02.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.801+0000 7fd07fe00640 1 --2- 172.21.15.86:0/3002749491 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd0800ad620 0x7fd080142da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:02.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.801+0000 7fd07fe00640 1 -- 172.21.15.86:0/3002749491 >> 172.21.15.86:0/3002749491 conn(0x7fd0800d0170 msgr2=0x7fd0800ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:02.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.801+0000 7fd07fe00640 1 -- 172.21.15.86:0/3002749491 shutdown_connections 2024-04-20T21:25:02.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:02.801+0000 7fd07fe00640 1 -- 172.21.15.86:0/3002749491 wait complete. 2024-04-20T21:25:02.821 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-20T21:25:02.821 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T21:25:03.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.011+0000 7ff5c1800640 1 Processor -- start 2024-04-20T21:25:03.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.011+0000 7ff5c1800640 1 -- start start 2024-04-20T21:25:03.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.012+0000 7ff5c1800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff5bc15c010 0x7ff5bc15c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:03.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.012+0000 7ff5c1800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff5bc0af7b0 0x7ff5bc0adda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:03.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.012+0000 7ff5c1800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5bc06f4c0 0x7ff5bc06f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:03.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.012+0000 7ff5c1800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff5bc06fe70 con 0x7ff5bc15c010 2024-04-20T21:25:03.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.012+0000 7ff5c1800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7ff5bc0af320 con 0x7ff5bc0af7b0 2024-04-20T21:25:03.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.012+0000 7ff5c1800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff5bc15c930 con 0x7ff5bc06f4c0 2024-04-20T21:25:03.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.013+0000 7ff5bbe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff5bc15c010 0x7ff5bc15c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:03.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.013+0000 7ff5bbe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff5bc15c010 0x7ff5bc15c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43400/0 (socket says 172.21.15.86:43400) 2024-04-20T21:25:03.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.013+0000 7ff5bbe00640 1 -- 172.21.15.86:0/1056818943 learned_addr learned my addr 172.21.15.86:0/1056818943 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:03.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.013+0000 7ff5bbe00640 1 -- 172.21.15.86:0/1056818943 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5bc06f4c0 msgr2=0x7ff5bc06f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:03.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.013+0000 7ff5c0e00640 1 --2- 172.21.15.86:0/1056818943 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5bc06f4c0 0x7ff5bc06f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:03.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.013+0000 7ff5bbe00640 1 --2- 172.21.15.86:0/1056818943 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5bc06f4c0 0x7ff5bc06f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:03.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.013+0000 7ff5bbe00640 1 -- 172.21.15.86:0/1056818943 >> v2:172.21.15.144:3300/0 conn(0x7ff5bc0af7b0 msgr2=0x7ff5bc0adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:03.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.013+0000 7ff5bbe00640 1 --2- 172.21.15.86:0/1056818943 >> v2:172.21.15.144:3300/0 conn(0x7ff5bc0af7b0 0x7ff5bc0adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:03.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.013+0000 7ff5bbe00640 1 -- 172.21.15.86:0/1056818943 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5bc0ae710 con 0x7ff5bc15c010 2024-04-20T21:25:03.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.014+0000 7ff5bbe00640 1 --2- 172.21.15.86:0/1056818943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5bc15c010 0x7ff5bc15c3f0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc145230 tx=0x7ff5ac02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7c9e79751e2f68a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:03.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.015+0000 7ff5baa00640 1 -- 172.21.15.86:0/1056818943 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac035020 con 0x7ff5bc15c010 2024-04-20T21:25:03.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.015+0000 7ff5baa00640 1 -- 172.21.15.86:0/1056818943 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5ac031640 con 0x7ff5bc15c010 2024-04-20T21:25:03.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.015+0000 7ff5baa00640 1 -- 172.21.15.86:0/1056818943 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac02d040 con 0x7ff5bc15c010 2024-04-20T21:25:03.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.015+0000 7ff5c1800640 1 -- 172.21.15.86:0/1056818943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5bc15c010 msgr2=0x7ff5bc15c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:03.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.015+0000 7ff5c1800640 1 --2- 172.21.15.86:0/1056818943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5bc15c010 0x7ff5bc15c3f0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc145230 tx=0x7ff5ac02b8f0 comp rx=0 tx=0).stop 2024-04-20T21:25:03.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.015+0000 7ff5c1800640 1 -- 172.21.15.86:0/1056818943 shutdown_connections 2024-04-20T21:25:03.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.015+0000 7ff5c1800640 1 --2- 172.21.15.86:0/1056818943 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5bc06f4c0 0x7ff5bc06f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:03.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.015+0000 7ff5c1800640 1 --2- 172.21.15.86:0/1056818943 >> v2:172.21.15.144:3300/0 conn(0x7ff5bc0af7b0 0x7ff5bc0adda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:03.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.015+0000 7ff5c1800640 1 --2- 172.21.15.86:0/1056818943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5bc15c010 0x7ff5bc15c3f0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:03.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.015+0000 7ff5c1800640 1 -- 172.21.15.86:0/1056818943 >> 172.21.15.86:0/1056818943 conn(0x7ff5bc0d0170 msgr2=0x7ff5bc0d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:03.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.015+0000 7ff5c1800640 1 -- 172.21.15.86:0/1056818943 shutdown_connections 2024-04-20T21:25:03.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.015+0000 7ff5c1800640 1 -- 172.21.15.86:0/1056818943 wait complete. 2024-04-20T21:25:03.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.017+0000 7ff5c1800640 1 Processor -- start 2024-04-20T21:25:03.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.017+0000 7ff5c1800640 1 -- start start 2024-04-20T21:25:03.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.018+0000 7ff5c1800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5bc06f4c0 0x7ff5bc0b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:03.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.018+0000 7ff5c1800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5bc0af7b0 0x7ff5bc143640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:03.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.018+0000 7ff5c1800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5bc15c010 0x7ff5bc0b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:03.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.018+0000 7ff5c1800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff5bc0aefb0 con 0x7ff5bc06f4c0 2024-04-20T21:25:03.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.018+0000 7ff5c1800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff5bc0aecd0 con 0x7ff5bc0af7b0 2024-04-20T21:25:03.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.018+0000 7ff5c1800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff5bc0ae2e0 con 0x7ff5bc15c010 2024-04-20T21:25:03.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.018+0000 7ff5bbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5bc06f4c0 0x7ff5bc0b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:03.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.018+0000 7ff5bbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5bc06f4c0 0x7ff5bc0b4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43404/0 (socket says 172.21.15.86:43404) 2024-04-20T21:25:03.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.018+0000 7ff5bbe00640 1 -- 172.21.15.86:0/2896643951 learned_addr learned my addr 172.21.15.86:0/2896643951 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:03.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.019+0000 7ff5c0e00640 1 --2- 172.21.15.86:0/2896643951 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5bc15c010 0x7ff5bc0b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:03.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.019+0000 7ff5bb400640 1 --2- 172.21.15.86:0/2896643951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5bc0af7b0 0x7ff5bc143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:03.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.019+0000 7ff5bbe00640 1 -- 172.21.15.86:0/2896643951 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5bc15c010 msgr2=0x7ff5bc0b0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:03.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.019+0000 7ff5bbe00640 1 --2- 172.21.15.86:0/2896643951 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5bc15c010 0x7ff5bc0b0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:03.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.019+0000 7ff5bbe00640 1 -- 172.21.15.86:0/2896643951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5bc0af7b0 msgr2=0x7ff5bc143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:03.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.019+0000 7ff5bbe00640 1 --2- 172.21.15.86:0/2896643951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5bc0af7b0 0x7ff5bc143640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:03.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.019+0000 7ff5bbe00640 1 -- 172.21.15.86:0/2896643951 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5bc1589b0 con 0x7ff5bc06f4c0 2024-04-20T21:25:03.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.019+0000 7ff5c0e00640 1 --2- 172.21.15.86:0/2896643951 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5bc15c010 0x7ff5bc0b0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:25:03.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.019+0000 7ff5bbe00640 1 --2- 172.21.15.86:0/2896643951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5bc06f4c0 0x7ff5bc0b4e30 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac02b930 tx=0x7ff5ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:03.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.019+0000 7ff5b3e00640 1 -- 172.21.15.86:0/2896643951 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac03ac60 con 0x7ff5bc06f4c0 2024-04-20T21:25:03.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.019+0000 7ff5b3e00640 1 -- 172.21.15.86:0/2896643951 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5ac03ae20 con 0x7ff5bc06f4c0 2024-04-20T21:25:03.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.019+0000 7ff5b3e00640 1 -- 172.21.15.86:0/2896643951 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac044630 con 0x7ff5bc06f4c0 2024-04-20T21:25:03.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.019+0000 7ff5c1800640 1 -- 172.21.15.86:0/2896643951 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5bc158c90 con 0x7ff5bc06f4c0 2024-04-20T21:25:03.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.020+0000 7ff5c1800640 1 -- 172.21.15.86:0/2896643951 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5bc0b3e30 con 0x7ff5bc06f4c0 2024-04-20T21:25:03.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.021+0000 7ff5c1800640 1 -- 172.21.15.86:0/2896643951 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5bc04fcf0 con 0x7ff5bc06f4c0 2024-04-20T21:25:03.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.021+0000 7ff5b3e00640 1 -- 172.21.15.86:0/2896643951 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff5ac038020 con 0x7ff5bc06f4c0 2024-04-20T21:25:03.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.022+0000 7ff5b3e00640 1 --2- 172.21.15.86:0/2896643951 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff59806c8f0 0x7ff59806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:03.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.022+0000 7ff5b3e00640 1 -- 172.21.15.86:0/2896643951 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7ff5ac0ad960 con 0x7ff5bc06f4c0 2024-04-20T21:25:03.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.023+0000 7ff5bb400640 1 --2- 172.21.15.86:0/2896643951 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff59806c8f0 0x7ff59806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:03.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.023+0000 7ff5bb400640 1 --2- 172.21.15.86:0/2896643951 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff59806c8f0 0x7ff59806edb0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc072f20 tx=0x7ff5a4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:03.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.027+0000 7ff5b3e00640 1 -- 172.21.15.86:0/2896643951 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff5ac03f070 con 0x7ff5bc06f4c0 2024-04-20T21:25:03.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.226+0000 7ff5c1800640 1 -- 172.21.15.86:0/2896643951 --> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff5bc158e60 con 0x7ff59806c8f0 2024-04-20T21:25:03.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.235+0000 7ff5b3e00640 1 -- 172.21.15.86:0/2896643951 <== mgr.4107 v2:172.21.15.86:6832/302045240 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252082 (secure 0 0 0) 0x7ff5bc158e60 con 0x7ff59806c8f0 2024-04-20T21:25:03.238 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:25:03.241 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-20T21:25:03.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.241+0000 7ff5c1800640 1 -- 172.21.15.86:0/2896643951 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff59806c8f0 msgr2=0x7ff59806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:03.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.241+0000 7ff5c1800640 1 --2- 172.21.15.86:0/2896643951 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff59806c8f0 0x7ff59806edb0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc072f20 tx=0x7ff5a4008040 comp rx=0 tx=0).stop 2024-04-20T21:25:03.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.241+0000 7ff5c1800640 1 -- 172.21.15.86:0/2896643951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5bc06f4c0 msgr2=0x7ff5bc0b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:03.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.241+0000 7ff5c1800640 1 --2- 172.21.15.86:0/2896643951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5bc06f4c0 0x7ff5bc0b4e30 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac02b930 tx=0x7ff5ac0046c0 comp rx=0 tx=0).stop 2024-04-20T21:25:03.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.242+0000 7ff5c1800640 1 -- 172.21.15.86:0/2896643951 shutdown_connections 2024-04-20T21:25:03.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.242+0000 7ff5c1800640 1 --2- 172.21.15.86:0/2896643951 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff59806c8f0 0x7ff59806edb0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:03.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.242+0000 7ff5c1800640 1 --2- 172.21.15.86:0/2896643951 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5bc15c010 0x7ff5bc0b0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:03.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.242+0000 7ff5c1800640 1 --2- 172.21.15.86:0/2896643951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5bc0af7b0 0x7ff5bc143640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:03.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.242+0000 7ff5c1800640 1 --2- 172.21.15.86:0/2896643951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5bc06f4c0 0x7ff5bc0b4e30 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:03.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.242+0000 7ff5c1800640 1 -- 172.21.15.86:0/2896643951 >> 172.21.15.86:0/2896643951 conn(0x7ff5bc0d0170 msgr2=0x7ff5bc0afd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:03.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.242+0000 7ff5c1800640 1 -- 172.21.15.86:0/2896643951 shutdown_connections 2024-04-20T21:25:03.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.242+0000 7ff5c1800640 1 -- 172.21.15.86:0/2896643951 wait complete. 2024-04-20T21:25:03.260 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":299,"stamp":"2024-04-20T21:25:02.781606+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":132,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258836,"kb_used_data":9372,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715884,"statfs":{"total":773094113280,"available":772829065216,"internally_reserved":0,"allocated":9596928,"data_stored":4751112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.975035"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.863801+0000","last_change":"2024-04-20T21:24:58.860219+0000","last_active":"2024-04-20T21:24:59.863801+0000","last_peered":"2024-04-20T21:24:59.863801+0000","last_clean":"2024-04-20T21:24:59.863801+0000","last_became_active":"2024-04-20T21:24:58.859934+0000","last_became_peered":"2024-04-20T21:24:58.859934+0000","last_unstale":"2024-04-20T21:24:59.863801+0000","last_undegraded":"2024-04-20T21:24:59.863801+0000","last_fullsized":"2024-04-20T21:24:59.863801+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:44:55.916343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.862100+0000","last_change":"2024-04-20T21:24:58.868066+0000","last_active":"2024-04-20T21:24:59.862100+0000","last_peered":"2024-04-20T21:24:59.862100+0000","last_clean":"2024-04-20T21:24:59.862100+0000","last_became_active":"2024-04-20T21:24:58.867725+0000","last_became_peered":"2024-04-20T21:24:58.867725+0000","last_unstale":"2024-04-20T21:24:59.862100+0000","last_undegraded":"2024-04-20T21:24:59.862100+0000","last_fullsized":"2024-04-20T21:24:59.862100+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:21:18.524282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.005010+0000","last_change":"2024-04-20T21:24:58.862953+0000","last_active":"2024-04-20T21:25:00.005010+0000","last_peered":"2024-04-20T21:25:00.005010+0000","last_clean":"2024-04-20T21:25:00.005010+0000","last_became_active":"2024-04-20T21:24:58.862527+0000","last_became_peered":"2024-04-20T21:24:58.862527+0000","last_unstale":"2024-04-20T21:25:00.005010+0000","last_undegraded":"2024-04-20T21:25:00.005010+0000","last_fullsized":"2024-04-20T21:25:00.005010+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:31:50.250214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.007567+0000","last_change":"2024-04-20T21:24:58.873888+0000","last_active":"2024-04-20T21:25:00.007567+0000","last_peered":"2024-04-20T21:25:00.007567+0000","last_clean":"2024-04-20T21:25:00.007567+0000","last_became_active":"2024-04-20T21:24:58.871153+0000","last_became_peered":"2024-04-20T21:24:58.871153+0000","last_unstale":"2024-04-20T21:25:00.007567+0000","last_undegraded":"2024-04-20T21:25:00.007567+0000","last_fullsized":"2024-04-20T21:25:00.007567+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:33:44.999424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.861730+0000","last_change":"2024-04-20T21:24:58.879526+0000","last_active":"2024-04-20T21:24:59.861730+0000","last_peered":"2024-04-20T21:24:59.861730+0000","last_clean":"2024-04-20T21:24:59.861730+0000","last_became_active":"2024-04-20T21:24:58.879157+0000","last_became_peered":"2024-04-20T21:24:58.879157+0000","last_unstale":"2024-04-20T21:24:59.861730+0000","last_undegraded":"2024-04-20T21:24:59.861730+0000","last_fullsized":"2024-04-20T21:24:59.861730+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:25:56.141401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.004598+0000","last_change":"2024-04-20T21:24:58.864812+0000","last_active":"2024-04-20T21:25:00.004598+0000","last_peered":"2024-04-20T21:25:00.004598+0000","last_clean":"2024-04-20T21:25:00.004598+0000","last_became_active":"2024-04-20T21:24:58.864251+0000","last_became_peered":"2024-04-20T21:24:58.864251+0000","last_unstale":"2024-04-20T21:25:00.004598+0000","last_undegraded":"2024-04-20T21:25:00.004598+0000","last_fullsized":"2024-04-20T21:25:00.004598+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:07:53.702653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.005019+0000","last_change":"2024-04-20T21:24:58.873317+0000","last_active":"2024-04-20T21:25:00.005019+0000","last_peered":"2024-04-20T21:25:00.005019+0000","last_clean":"2024-04-20T21:25:00.005019+0000","last_became_active":"2024-04-20T21:24:58.867489+0000","last_became_peered":"2024-04-20T21:24:58.867489+0000","last_unstale":"2024-04-20T21:25:00.005019+0000","last_undegraded":"2024-04-20T21:25:00.005019+0000","last_fullsized":"2024-04-20T21:25:00.005019+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:19:34.543192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.004584+0000","last_change":"2024-04-20T21:24:58.878924+0000","last_active":"2024-04-20T21:25:00.004584+0000","last_peered":"2024-04-20T21:25:00.004584+0000","last_clean":"2024-04-20T21:25:00.004584+0000","last_became_active":"2024-04-20T21:24:58.878410+0000","last_became_peered":"2024-04-20T21:24:58.878410+0000","last_unstale":"2024-04-20T21:25:00.004584+0000","last_undegraded":"2024-04-20T21:25:00.004584+0000","last_fullsized":"2024-04-20T21:25:00.004584+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:54:17.428125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.005565+0000","last_change":"2024-04-20T21:24:58.866708+0000","last_active":"2024-04-20T21:25:00.005565+0000","last_peered":"2024-04-20T21:25:00.005565+0000","last_clean":"2024-04-20T21:25:00.005565+0000","last_became_active":"2024-04-20T21:24:58.865109+0000","last_became_peered":"2024-04-20T21:24:58.865109+0000","last_unstale":"2024-04-20T21:25:00.005565+0000","last_undegraded":"2024-04-20T21:25:00.005565+0000","last_fullsized":"2024-04-20T21:25:00.005565+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:19:46.273364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.864554+0000","last_change":"2024-04-20T21:24:58.859111+0000","last_active":"2024-04-20T21:24:59.864554+0000","last_peered":"2024-04-20T21:24:59.864554+0000","last_clean":"2024-04-20T21:24:59.864554+0000","last_became_active":"2024-04-20T21:24:58.858791+0000","last_became_peered":"2024-04-20T21:24:58.858791+0000","last_unstale":"2024-04-20T21:24:59.864554+0000","last_undegraded":"2024-04-20T21:24:59.864554+0000","last_fullsized":"2024-04-20T21:24:59.864554+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:15:18.594034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.862450+0000","last_change":"2024-04-20T21:24:58.873582+0000","last_active":"2024-04-20T21:24:59.862450+0000","last_peered":"2024-04-20T21:24:59.862450+0000","last_clean":"2024-04-20T21:24:59.862450+0000","last_became_active":"2024-04-20T21:24:58.873181+0000","last_became_peered":"2024-04-20T21:24:58.873181+0000","last_unstale":"2024-04-20T21:24:59.862450+0000","last_undegraded":"2024-04-20T21:24:59.862450+0000","last_fullsized":"2024-04-20T21:24:59.862450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:03:19.069181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.863687+0000","last_change":"2024-04-20T21:24:58.860151+0000","last_active":"2024-04-20T21:24:59.863687+0000","last_peered":"2024-04-20T21:24:59.863687+0000","last_clean":"2024-04-20T21:24:59.863687+0000","last_became_active":"2024-04-20T21:24:58.858389+0000","last_became_peered":"2024-04-20T21:24:58.858389+0000","last_unstale":"2024-04-20T21:24:59.863687+0000","last_undegraded":"2024-04-20T21:24:59.863687+0000","last_fullsized":"2024-04-20T21:24:59.863687+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:57:14.691985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.005452+0000","last_change":"2024-04-20T21:24:58.865960+0000","last_active":"2024-04-20T21:25:00.005452+0000","last_peered":"2024-04-20T21:25:00.005452+0000","last_clean":"2024-04-20T21:25:00.005452+0000","last_became_active":"2024-04-20T21:24:58.865636+0000","last_became_peered":"2024-04-20T21:24:58.865636+0000","last_unstale":"2024-04-20T21:25:00.005452+0000","last_undegraded":"2024-04-20T21:25:00.005452+0000","last_fullsized":"2024-04-20T21:25:00.005452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:03:04.971095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.007530+0000","last_change":"2024-04-20T21:24:58.872800+0000","last_active":"2024-04-20T21:25:00.007530+0000","last_peered":"2024-04-20T21:25:00.007530+0000","last_clean":"2024-04-20T21:25:00.007530+0000","last_became_active":"2024-04-20T21:24:58.872456+0000","last_became_peered":"2024-04-20T21:24:58.872456+0000","last_unstale":"2024-04-20T21:25:00.007530+0000","last_undegraded":"2024-04-20T21:25:00.007530+0000","last_fullsized":"2024-04-20T21:25:00.007530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:16:20.955774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.005466+0000","last_change":"2024-04-20T21:24:58.862822+0000","last_active":"2024-04-20T21:25:00.005466+0000","last_peered":"2024-04-20T21:25:00.005466+0000","last_clean":"2024-04-20T21:25:00.005466+0000","last_became_active":"2024-04-20T21:24:58.862304+0000","last_became_peered":"2024-04-20T21:24:58.862304+0000","last_unstale":"2024-04-20T21:25:00.005466+0000","last_undegraded":"2024-04-20T21:25:00.005466+0000","last_fullsized":"2024-04-20T21:25:00.005466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:55:37.672502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.861478+0000","last_change":"2024-04-20T21:24:58.852755+0000","last_active":"2024-04-20T21:24:59.861478+0000","last_peered":"2024-04-20T21:24:59.861478+0000","last_clean":"2024-04-20T21:24:59.861478+0000","last_became_active":"2024-04-20T21:24:58.851389+0000","last_became_peered":"2024-04-20T21:24:58.851389+0000","last_unstale":"2024-04-20T21:24:59.861478+0000","last_undegraded":"2024-04-20T21:24:59.861478+0000","last_fullsized":"2024-04-20T21:24:59.861478+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:53:39.858430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.863944+0000","last_change":"2024-04-20T21:24:58.855876+0000","last_active":"2024-04-20T21:24:59.863944+0000","last_peered":"2024-04-20T21:24:59.863944+0000","last_clean":"2024-04-20T21:24:59.863944+0000","last_became_active":"2024-04-20T21:24:58.855594+0000","last_became_peered":"2024-04-20T21:24:58.855594+0000","last_unstale":"2024-04-20T21:24:59.863944+0000","last_undegraded":"2024-04-20T21:24:59.863944+0000","last_fullsized":"2024-04-20T21:24:59.863944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:10:48.245778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.007206+0000","last_change":"2024-04-20T21:24:58.866512+0000","last_active":"2024-04-20T21:25:00.007206+0000","last_peered":"2024-04-20T21:25:00.007206+0000","last_clean":"2024-04-20T21:25:00.007206+0000","last_became_active":"2024-04-20T21:24:58.862188+0000","last_became_peered":"2024-04-20T21:24:58.862188+0000","last_unstale":"2024-04-20T21:25:00.007206+0000","last_undegraded":"2024-04-20T21:25:00.007206+0000","last_fullsized":"2024-04-20T21:25:00.007206+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:49:13.697624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.008675+0000","last_change":"2024-04-20T21:24:58.858015+0000","last_active":"2024-04-20T21:25:00.008675+0000","last_peered":"2024-04-20T21:25:00.008675+0000","last_clean":"2024-04-20T21:25:00.008675+0000","last_became_active":"2024-04-20T21:24:58.857654+0000","last_became_peered":"2024-04-20T21:24:58.857654+0000","last_unstale":"2024-04-20T21:25:00.008675+0000","last_undegraded":"2024-04-20T21:25:00.008675+0000","last_fullsized":"2024-04-20T21:25:00.008675+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:43:31.020198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.007900+0000","last_change":"2024-04-20T21:24:58.864303+0000","last_active":"2024-04-20T21:25:00.007900+0000","last_peered":"2024-04-20T21:25:00.007900+0000","last_clean":"2024-04-20T21:25:00.007900+0000","last_became_active":"2024-04-20T21:24:58.864035+0000","last_became_peered":"2024-04-20T21:24:58.864035+0000","last_unstale":"2024-04-20T21:25:00.007900+0000","last_undegraded":"2024-04-20T21:25:00.007900+0000","last_fullsized":"2024-04-20T21:25:00.007900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:46:41.694861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.007373+0000","last_change":"2024-04-20T21:24:58.863754+0000","last_active":"2024-04-20T21:25:00.007373+0000","last_peered":"2024-04-20T21:25:00.007373+0000","last_clean":"2024-04-20T21:25:00.007373+0000","last_became_active":"2024-04-20T21:24:58.863162+0000","last_became_peered":"2024-04-20T21:24:58.863162+0000","last_unstale":"2024-04-20T21:25:00.007373+0000","last_undegraded":"2024-04-20T21:25:00.007373+0000","last_fullsized":"2024-04-20T21:25:00.007373+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:29:51.537408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.861860+0000","last_change":"2024-04-20T21:24:58.878056+0000","last_active":"2024-04-20T21:24:59.861860+0000","last_peered":"2024-04-20T21:24:59.861860+0000","last_clean":"2024-04-20T21:24:59.861860+0000","last_became_active":"2024-04-20T21:24:58.877793+0000","last_became_peered":"2024-04-20T21:24:58.877793+0000","last_unstale":"2024-04-20T21:24:59.861860+0000","last_undegraded":"2024-04-20T21:24:59.861860+0000","last_fullsized":"2024-04-20T21:24:59.861860+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:48:22.499537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.864060+0000","last_change":"2024-04-20T21:24:58.861713+0000","last_active":"2024-04-20T21:24:59.864060+0000","last_peered":"2024-04-20T21:24:59.864060+0000","last_clean":"2024-04-20T21:24:59.864060+0000","last_became_active":"2024-04-20T21:24:58.861381+0000","last_became_peered":"2024-04-20T21:24:58.861381+0000","last_unstale":"2024-04-20T21:24:59.864060+0000","last_undegraded":"2024-04-20T21:24:59.864060+0000","last_fullsized":"2024-04-20T21:24:59.864060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:44:29.007861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.005194+0000","last_change":"2024-04-20T21:24:58.861005+0000","last_active":"2024-04-20T21:25:00.005194+0000","last_peered":"2024-04-20T21:25:00.005194+0000","last_clean":"2024-04-20T21:25:00.005194+0000","last_became_active":"2024-04-20T21:24:58.860715+0000","last_became_peered":"2024-04-20T21:24:58.860715+0000","last_unstale":"2024-04-20T21:25:00.005194+0000","last_undegraded":"2024-04-20T21:25:00.005194+0000","last_fullsized":"2024-04-20T21:25:00.005194+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:30:58.449132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.862380+0000","last_change":"2024-04-20T21:24:58.874737+0000","last_active":"2024-04-20T21:24:59.862380+0000","last_peered":"2024-04-20T21:24:59.862380+0000","last_clean":"2024-04-20T21:24:59.862380+0000","last_became_active":"2024-04-20T21:24:58.874452+0000","last_became_peered":"2024-04-20T21:24:58.874452+0000","last_unstale":"2024-04-20T21:24:59.862380+0000","last_undegraded":"2024-04-20T21:24:59.862380+0000","last_fullsized":"2024-04-20T21:24:59.862380+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:36:50.375496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.006022+0000","last_change":"2024-04-20T21:24:58.871689+0000","last_active":"2024-04-20T21:25:00.006022+0000","last_peered":"2024-04-20T21:25:00.006022+0000","last_clean":"2024-04-20T21:25:00.006022+0000","last_became_active":"2024-04-20T21:24:58.871117+0000","last_became_peered":"2024-04-20T21:24:58.871117+0000","last_unstale":"2024-04-20T21:25:00.006022+0000","last_undegraded":"2024-04-20T21:25:00.006022+0000","last_fullsized":"2024-04-20T21:25:00.006022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:16:18.582014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.862144+0000","last_change":"2024-04-20T21:24:58.856734+0000","last_active":"2024-04-20T21:24:59.862144+0000","last_peered":"2024-04-20T21:24:59.862144+0000","last_clean":"2024-04-20T21:24:59.862144+0000","last_became_active":"2024-04-20T21:24:58.856019+0000","last_became_peered":"2024-04-20T21:24:58.856019+0000","last_unstale":"2024-04-20T21:24:59.862144+0000","last_undegraded":"2024-04-20T21:24:59.862144+0000","last_fullsized":"2024-04-20T21:24:59.862144+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:31:50.250214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.863853+0000","last_change":"2024-04-20T21:24:58.860915+0000","last_active":"2024-04-20T21:24:59.863853+0000","last_peered":"2024-04-20T21:24:59.863853+0000","last_clean":"2024-04-20T21:24:59.863853+0000","last_became_active":"2024-04-20T21:24:58.860555+0000","last_became_peered":"2024-04-20T21:24:58.860555+0000","last_unstale":"2024-04-20T21:24:59.863853+0000","last_undegraded":"2024-04-20T21:24:59.863853+0000","last_fullsized":"2024-04-20T21:24:59.863853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:22:35.562141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.861667+0000","last_change":"2024-04-20T21:24:58.872404+0000","last_active":"2024-04-20T21:24:59.861667+0000","last_peered":"2024-04-20T21:24:59.861667+0000","last_clean":"2024-04-20T21:24:59.861667+0000","last_became_active":"2024-04-20T21:24:58.872131+0000","last_became_peered":"2024-04-20T21:24:58.872131+0000","last_unstale":"2024-04-20T21:24:59.861667+0000","last_undegraded":"2024-04-20T21:24:59.861667+0000","last_fullsized":"2024-04-20T21:24:59.861667+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:16:05.557935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.862525+0000","last_change":"2024-04-20T21:24:58.870631+0000","last_active":"2024-04-20T21:24:59.862525+0000","last_peered":"2024-04-20T21:24:59.862525+0000","last_clean":"2024-04-20T21:24:59.862525+0000","last_became_active":"2024-04-20T21:24:58.870126+0000","last_became_peered":"2024-04-20T21:24:58.870126+0000","last_unstale":"2024-04-20T21:24:59.862525+0000","last_undegraded":"2024-04-20T21:24:59.862525+0000","last_fullsized":"2024-04-20T21:24:59.862525+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:00:15.798523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.861985+0000","last_change":"2024-04-20T21:24:58.857710+0000","last_active":"2024-04-20T21:24:59.861985+0000","last_peered":"2024-04-20T21:24:59.861985+0000","last_clean":"2024-04-20T21:24:59.861985+0000","last_became_active":"2024-04-20T21:24:58.857392+0000","last_became_peered":"2024-04-20T21:24:58.857392+0000","last_unstale":"2024-04-20T21:24:59.861985+0000","last_undegraded":"2024-04-20T21:24:59.861985+0000","last_fullsized":"2024-04-20T21:24:59.861985+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:15:35.117140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.863447+0000","last_change":"2024-04-20T21:24:58.877663+0000","last_active":"2024-04-20T21:24:59.863447+0000","last_peered":"2024-04-20T21:24:59.863447+0000","last_clean":"2024-04-20T21:24:59.863447+0000","last_became_active":"2024-04-20T21:24:58.877232+0000","last_became_peered":"2024-04-20T21:24:58.877232+0000","last_unstale":"2024-04-20T21:24:59.863447+0000","last_undegraded":"2024-04-20T21:24:59.863447+0000","last_fullsized":"2024-04-20T21:24:59.863447+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:31:15.019045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.862179+0000","last_change":"2024-04-20T21:24:58.861604+0000","last_active":"2024-04-20T21:24:59.862179+0000","last_peered":"2024-04-20T21:24:59.862179+0000","last_clean":"2024-04-20T21:24:59.862179+0000","last_became_active":"2024-04-20T21:24:58.861309+0000","last_became_peered":"2024-04-20T21:24:58.861309+0000","last_unstale":"2024-04-20T21:24:59.862179+0000","last_undegraded":"2024-04-20T21:24:59.862179+0000","last_fullsized":"2024-04-20T21:24:59.862179+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:03:04.971095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"84'4","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.864438+0000","last_change":"2024-04-20T21:24:57.850348+0000","last_active":"2024-04-20T21:24:59.864438+0000","last_peered":"2024-04-20T21:24:59.864438+0000","last_clean":"2024-04-20T21:24:59.864438+0000","last_became_active":"2024-04-20T21:24:57.849983+0000","last_became_peered":"2024-04-20T21:24:57.849983+0000","last_unstale":"2024-04-20T21:24:59.864438+0000","last_undegraded":"2024-04-20T21:24:59.864438+0000","last_fullsized":"2024-04-20T21:24:59.864438+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:19:17.769565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.863975+0000","last_change":"2024-04-20T21:24:58.860907+0000","last_active":"2024-04-20T21:24:59.863975+0000","last_peered":"2024-04-20T21:24:59.863975+0000","last_clean":"2024-04-20T21:24:59.863975+0000","last_became_active":"2024-04-20T21:24:58.860609+0000","last_became_peered":"2024-04-20T21:24:58.860609+0000","last_unstale":"2024-04-20T21:24:59.863975+0000","last_undegraded":"2024-04-20T21:24:59.863975+0000","last_fullsized":"2024-04-20T21:24:59.863975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:59:29.884590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"84'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.863380+0000","last_change":"2024-04-20T21:24:57.841870+0000","last_active":"2024-04-20T21:24:59.863380+0000","last_peered":"2024-04-20T21:24:59.863380+0000","last_clean":"2024-04-20T21:24:59.863380+0000","last_became_active":"2024-04-20T21:24:57.841518+0000","last_became_peered":"2024-04-20T21:24:57.841518+0000","last_unstale":"2024-04-20T21:24:59.863380+0000","last_undegraded":"2024-04-20T21:24:59.863380+0000","last_fullsized":"2024-04-20T21:24:59.863380+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:57:13.677134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.864013+0000","last_change":"2024-04-20T21:24:58.867722+0000","last_active":"2024-04-20T21:24:59.864013+0000","last_peered":"2024-04-20T21:24:59.864013+0000","last_clean":"2024-04-20T21:24:59.864013+0000","last_became_active":"2024-04-20T21:24:58.867429+0000","last_became_peered":"2024-04-20T21:24:58.867429+0000","last_unstale":"2024-04-20T21:24:59.864013+0000","last_undegraded":"2024-04-20T21:24:59.864013+0000","last_fullsized":"2024-04-20T21:24:59.864013+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:03:09.674975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"84'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.008780+0000","last_change":"2024-04-20T21:24:57.841558+0000","last_active":"2024-04-20T21:25:00.008780+0000","last_peered":"2024-04-20T21:25:00.008780+0000","last_clean":"2024-04-20T21:25:00.008780+0000","last_became_active":"2024-04-20T21:24:57.841165+0000","last_became_peered":"2024-04-20T21:24:57.841165+0000","last_unstale":"2024-04-20T21:25:00.008780+0000","last_undegraded":"2024-04-20T21:25:00.008780+0000","last_fullsized":"2024-04-20T21:25:00.008780+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:15:17.579183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.006076+0000","last_change":"2024-04-20T21:24:58.868769+0000","last_active":"2024-04-20T21:25:00.006076+0000","last_peered":"2024-04-20T21:25:00.006076+0000","last_clean":"2024-04-20T21:25:00.006076+0000","last_became_active":"2024-04-20T21:24:58.868460+0000","last_became_peered":"2024-04-20T21:24:58.868460+0000","last_unstale":"2024-04-20T21:25:00.006076+0000","last_undegraded":"2024-04-20T21:25:00.006076+0000","last_fullsized":"2024-04-20T21:25:00.006076+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:16:20.655880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"84'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.008125+0000","last_change":"2024-04-20T21:24:57.846244+0000","last_active":"2024-04-20T21:25:00.008125+0000","last_peered":"2024-04-20T21:25:00.008125+0000","last_clean":"2024-04-20T21:25:00.008125+0000","last_became_active":"2024-04-20T21:24:57.845970+0000","last_became_peered":"2024-04-20T21:24:57.845970+0000","last_unstale":"2024-04-20T21:25:00.008125+0000","last_undegraded":"2024-04-20T21:25:00.008125+0000","last_fullsized":"2024-04-20T21:25:00.008125+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:21:08.607476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.862614+0000","last_change":"2024-04-20T21:24:58.868267+0000","last_active":"2024-04-20T21:24:59.862614+0000","last_peered":"2024-04-20T21:24:59.862614+0000","last_clean":"2024-04-20T21:24:59.862614+0000","last_became_active":"2024-04-20T21:24:58.867985+0000","last_became_peered":"2024-04-20T21:24:58.867985+0000","last_unstale":"2024-04-20T21:24:59.862614+0000","last_undegraded":"2024-04-20T21:24:59.862614+0000","last_fullsized":"2024-04-20T21:24:59.862614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:46:53.063475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"84'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.861952+0000","last_change":"2024-04-20T21:24:57.842637+0000","last_active":"2024-04-20T21:24:59.861952+0000","last_peered":"2024-04-20T21:24:59.861952+0000","last_clean":"2024-04-20T21:24:59.861952+0000","last_became_active":"2024-04-20T21:24:57.842309+0000","last_became_peered":"2024-04-20T21:24:57.842309+0000","last_unstale":"2024-04-20T21:24:59.861952+0000","last_undegraded":"2024-04-20T21:24:59.861952+0000","last_fullsized":"2024-04-20T21:24:59.861952+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:31:14.004194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.861835+0000","last_change":"2024-04-20T21:24:58.862889+0000","last_active":"2024-04-20T21:24:59.861835+0000","last_peered":"2024-04-20T21:24:59.861835+0000","last_clean":"2024-04-20T21:24:59.861835+0000","last_became_active":"2024-04-20T21:24:58.862433+0000","last_became_peered":"2024-04-20T21:24:58.862433+0000","last_unstale":"2024-04-20T21:24:59.861835+0000","last_undegraded":"2024-04-20T21:24:59.861835+0000","last_fullsized":"2024-04-20T21:24:59.861835+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:14:07.253016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"84'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.861807+0000","last_change":"2024-04-20T21:24:57.841922+0000","last_active":"2024-04-20T21:24:59.861807+0000","last_peered":"2024-04-20T21:24:59.861807+0000","last_clean":"2024-04-20T21:24:59.861807+0000","last_became_active":"2024-04-20T21:24:57.841637+0000","last_became_peered":"2024-04-20T21:24:57.841637+0000","last_unstale":"2024-04-20T21:24:59.861807+0000","last_undegraded":"2024-04-20T21:24:59.861807+0000","last_fullsized":"2024-04-20T21:24:59.861807+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:46:05.086336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.006234+0000","last_change":"2024-04-20T21:24:58.866862+0000","last_active":"2024-04-20T21:25:00.006234+0000","last_peered":"2024-04-20T21:25:00.006234+0000","last_clean":"2024-04-20T21:25:00.006234+0000","last_became_active":"2024-04-20T21:24:58.866504+0000","last_became_peered":"2024-04-20T21:24:58.866504+0000","last_unstale":"2024-04-20T21:25:00.006234+0000","last_undegraded":"2024-04-20T21:25:00.006234+0000","last_fullsized":"2024-04-20T21:25:00.006234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:38:19.387593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.005975+0000","last_change":"2024-04-20T21:24:57.849101+0000","last_active":"2024-04-20T21:25:00.005975+0000","last_peered":"2024-04-20T21:25:00.005975+0000","last_clean":"2024-04-20T21:25:00.005975+0000","last_became_active":"2024-04-20T21:24:57.848676+0000","last_became_peered":"2024-04-20T21:24:57.848676+0000","last_unstale":"2024-04-20T21:25:00.005975+0000","last_undegraded":"2024-04-20T21:25:00.005975+0000","last_fullsized":"2024-04-20T21:25:00.005975+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:03:08.660125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.005866+0000","last_change":"2024-04-20T21:24:58.860404+0000","last_active":"2024-04-20T21:25:00.005866+0000","last_peered":"2024-04-20T21:25:00.005866+0000","last_clean":"2024-04-20T21:25:00.005866+0000","last_became_active":"2024-04-20T21:24:58.860075+0000","last_became_peered":"2024-04-20T21:24:58.860075+0000","last_unstale":"2024-04-20T21:25:00.005866+0000","last_undegraded":"2024-04-20T21:25:00.005866+0000","last_fullsized":"2024-04-20T21:25:00.005866+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:16:18.582014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.861486+0000","last_change":"2024-04-20T21:24:57.848064+0000","last_active":"2024-04-20T21:24:59.861486+0000","last_peered":"2024-04-20T21:24:59.861486+0000","last_clean":"2024-04-20T21:24:59.861486+0000","last_became_active":"2024-04-20T21:24:57.847202+0000","last_became_peered":"2024-04-20T21:24:57.847202+0000","last_unstale":"2024-04-20T21:24:59.861486+0000","last_undegraded":"2024-04-20T21:24:59.861486+0000","last_fullsized":"2024-04-20T21:24:59.861486+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:36:30.447062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.006664+0000","last_change":"2024-04-20T21:24:58.875155+0000","last_active":"2024-04-20T21:25:00.006664+0000","last_peered":"2024-04-20T21:25:00.006664+0000","last_clean":"2024-04-20T21:25:00.006664+0000","last_became_active":"2024-04-20T21:24:58.872257+0000","last_became_peered":"2024-04-20T21:24:58.872257+0000","last_unstale":"2024-04-20T21:25:00.006664+0000","last_undegraded":"2024-04-20T21:25:00.006664+0000","last_fullsized":"2024-04-20T21:25:00.006664+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:16:20.655880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.864639+0000","last_change":"2024-04-20T21:24:57.851985+0000","last_active":"2024-04-20T21:24:59.864639+0000","last_peered":"2024-04-20T21:24:59.864639+0000","last_clean":"2024-04-20T21:24:59.864639+0000","last_became_active":"2024-04-20T21:24:57.851650+0000","last_became_peered":"2024-04-20T21:24:57.851650+0000","last_unstale":"2024-04-20T21:24:59.864639+0000","last_undegraded":"2024-04-20T21:24:59.864639+0000","last_fullsized":"2024-04-20T21:24:59.864639+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:46:52.048624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.005971+0000","last_change":"2024-04-20T21:24:58.867812+0000","last_active":"2024-04-20T21:25:00.005971+0000","last_peered":"2024-04-20T21:25:00.005971+0000","last_clean":"2024-04-20T21:25:00.005971+0000","last_became_active":"2024-04-20T21:24:58.865494+0000","last_became_peered":"2024-04-20T21:24:58.865494+0000","last_unstale":"2024-04-20T21:25:00.005971+0000","last_undegraded":"2024-04-20T21:25:00.005971+0000","last_fullsized":"2024-04-20T21:25:00.005971+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:25:56.141401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.864495+0000","last_change":"2024-04-20T21:24:57.848638+0000","last_active":"2024-04-20T21:24:59.864495+0000","last_peered":"2024-04-20T21:24:59.864495+0000","last_clean":"2024-04-20T21:24:59.864495+0000","last_became_active":"2024-04-20T21:24:57.848224+0000","last_became_peered":"2024-04-20T21:24:57.848224+0000","last_unstale":"2024-04-20T21:24:59.864495+0000","last_undegraded":"2024-04-20T21:24:59.864495+0000","last_fullsized":"2024-04-20T21:24:59.864495+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:30:50.898747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.005746+0000","last_change":"2024-04-20T21:24:58.867490+0000","last_active":"2024-04-20T21:25:00.005746+0000","last_peered":"2024-04-20T21:25:00.005746+0000","last_clean":"2024-04-20T21:25:00.005746+0000","last_became_active":"2024-04-20T21:24:58.867221+0000","last_became_peered":"2024-04-20T21:24:58.867221+0000","last_unstale":"2024-04-20T21:25:00.005746+0000","last_undegraded":"2024-04-20T21:25:00.005746+0000","last_fullsized":"2024-04-20T21:25:00.005746+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:31:50.250214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"84'3","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.863552+0000","last_change":"2024-04-20T21:24:57.851827+0000","last_active":"2024-04-20T21:24:59.863552+0000","last_peered":"2024-04-20T21:24:59.863552+0000","last_clean":"2024-04-20T21:24:59.863552+0000","last_became_active":"2024-04-20T21:24:57.851559+0000","last_became_peered":"2024-04-20T21:24:57.851559+0000","last_unstale":"2024-04-20T21:24:59.863552+0000","last_undegraded":"2024-04-20T21:24:59.863552+0000","last_fullsized":"2024-04-20T21:24:59.863552+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:36:49.360645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.862532+0000","last_change":"2024-04-20T21:24:58.856834+0000","last_active":"2024-04-20T21:24:59.862532+0000","last_peered":"2024-04-20T21:24:59.862532+0000","last_clean":"2024-04-20T21:24:59.862532+0000","last_became_active":"2024-04-20T21:24:58.856495+0000","last_became_peered":"2024-04-20T21:24:58.856495+0000","last_unstale":"2024-04-20T21:24:59.862532+0000","last_undegraded":"2024-04-20T21:24:59.862532+0000","last_fullsized":"2024-04-20T21:24:59.862532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:16:18.582014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"84'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.005878+0000","last_change":"2024-04-20T21:24:57.845006+0000","last_active":"2024-04-20T21:25:00.005878+0000","last_peered":"2024-04-20T21:25:00.005878+0000","last_clean":"2024-04-20T21:25:00.005878+0000","last_became_active":"2024-04-20T21:24:57.844696+0000","last_became_peered":"2024-04-20T21:24:57.844696+0000","last_unstale":"2024-04-20T21:25:00.005878+0000","last_undegraded":"2024-04-20T21:25:00.005878+0000","last_fullsized":"2024-04-20T21:25:00.005878+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:02:23.398563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.005506+0000","last_change":"2024-04-20T21:24:58.861457+0000","last_active":"2024-04-20T21:25:00.005506+0000","last_peered":"2024-04-20T21:25:00.005506+0000","last_clean":"2024-04-20T21:25:00.005506+0000","last_became_active":"2024-04-20T21:24:58.861146+0000","last_became_peered":"2024-04-20T21:24:58.861146+0000","last_unstale":"2024-04-20T21:25:00.005506+0000","last_undegraded":"2024-04-20T21:25:00.005506+0000","last_fullsized":"2024-04-20T21:25:00.005506+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:15:35.117140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"84'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.862863+0000","last_change":"2024-04-20T21:24:57.840595+0000","last_active":"2024-04-20T21:24:59.862863+0000","last_peered":"2024-04-20T21:24:59.862863+0000","last_clean":"2024-04-20T21:24:59.862863+0000","last_became_active":"2024-04-20T21:24:57.840331+0000","last_became_peered":"2024-04-20T21:24:57.840331+0000","last_unstale":"2024-04-20T21:24:59.862863+0000","last_undegraded":"2024-04-20T21:24:59.862863+0000","last_fullsized":"2024-04-20T21:24:59.862863+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:16:19.940923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.005532+0000","last_change":"2024-04-20T21:24:58.881303+0000","last_active":"2024-04-20T21:25:00.005532+0000","last_peered":"2024-04-20T21:25:00.005532+0000","last_clean":"2024-04-20T21:25:00.005532+0000","last_became_active":"2024-04-20T21:24:58.880969+0000","last_became_peered":"2024-04-20T21:24:58.880969+0000","last_unstale":"2024-04-20T21:25:00.005532+0000","last_undegraded":"2024-04-20T21:25:00.005532+0000","last_fullsized":"2024-04-20T21:25:00.005532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:43:31.020198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.006688+0000","last_change":"2024-04-20T21:24:57.831315+0000","last_active":"2024-04-20T21:25:00.006688+0000","last_peered":"2024-04-20T21:25:00.006688+0000","last_clean":"2024-04-20T21:25:00.006688+0000","last_became_active":"2024-04-20T21:24:57.830496+0000","last_became_peered":"2024-04-20T21:24:57.830496+0000","last_unstale":"2024-04-20T21:25:00.006688+0000","last_undegraded":"2024-04-20T21:25:00.006688+0000","last_fullsized":"2024-04-20T21:25:00.006688+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:15:34.102290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.861950+0000","last_change":"2024-04-20T21:24:58.876953+0000","last_active":"2024-04-20T21:24:59.861950+0000","last_peered":"2024-04-20T21:24:59.861950+0000","last_clean":"2024-04-20T21:24:59.861950+0000","last_became_active":"2024-04-20T21:24:58.872812+0000","last_became_peered":"2024-04-20T21:24:58.872812+0000","last_unstale":"2024-04-20T21:24:59.861950+0000","last_undegraded":"2024-04-20T21:24:59.861950+0000","last_fullsized":"2024-04-20T21:24:59.861950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:19:06.776492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.004964+0000","last_change":"2024-04-20T21:24:57.846425+0000","last_active":"2024-04-20T21:25:00.004964+0000","last_peered":"2024-04-20T21:25:00.004964+0000","last_clean":"2024-04-20T21:25:00.004964+0000","last_became_active":"2024-04-20T21:24:57.846119+0000","last_became_peered":"2024-04-20T21:24:57.846119+0000","last_unstale":"2024-04-20T21:25:00.004964+0000","last_undegraded":"2024-04-20T21:25:00.004964+0000","last_fullsized":"2024-04-20T21:25:00.004964+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:38:19.894629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.861427+0000","last_change":"2024-04-20T21:24:58.868039+0000","last_active":"2024-04-20T21:24:59.861427+0000","last_peered":"2024-04-20T21:24:59.861427+0000","last_clean":"2024-04-20T21:24:59.861427+0000","last_became_active":"2024-04-20T21:24:58.864180+0000","last_became_peered":"2024-04-20T21:24:58.864180+0000","last_unstale":"2024-04-20T21:24:59.861427+0000","last_undegraded":"2024-04-20T21:24:59.861427+0000","last_fullsized":"2024-04-20T21:24:59.861427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:46:37.542502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"84'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.006195+0000","last_change":"2024-04-20T21:24:57.845058+0000","last_active":"2024-04-20T21:25:00.006195+0000","last_peered":"2024-04-20T21:25:00.006195+0000","last_clean":"2024-04-20T21:25:00.006195+0000","last_became_active":"2024-04-20T21:24:57.844088+0000","last_became_peered":"2024-04-20T21:24:57.844088+0000","last_unstale":"2024-04-20T21:25:00.006195+0000","last_undegraded":"2024-04-20T21:25:00.006195+0000","last_fullsized":"2024-04-20T21:25:00.006195+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:59:28.869739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.006228+0000","last_change":"2024-04-20T21:24:58.869796+0000","last_active":"2024-04-20T21:25:00.006228+0000","last_peered":"2024-04-20T21:25:00.006228+0000","last_clean":"2024-04-20T21:25:00.006228+0000","last_became_active":"2024-04-20T21:24:58.869046+0000","last_became_peered":"2024-04-20T21:24:58.869046+0000","last_unstale":"2024-04-20T21:25:00.006228+0000","last_undegraded":"2024-04-20T21:25:00.006228+0000","last_fullsized":"2024-04-20T21:25:00.006228+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:10:48.245778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"84'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.864681+0000","last_change":"2024-04-20T21:24:57.848118+0000","last_active":"2024-04-20T21:24:59.864681+0000","last_peered":"2024-04-20T21:24:59.864681+0000","last_clean":"2024-04-20T21:24:59.864681+0000","last_became_active":"2024-04-20T21:24:57.847626+0000","last_became_peered":"2024-04-20T21:24:57.847626+0000","last_unstale":"2024-04-20T21:24:59.864681+0000","last_undegraded":"2024-04-20T21:24:59.864681+0000","last_fullsized":"2024-04-20T21:24:59.864681+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:13:54.508013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.864142+0000","last_change":"2024-04-20T21:24:58.868908+0000","last_active":"2024-04-20T21:24:59.864142+0000","last_peered":"2024-04-20T21:24:59.864142+0000","last_clean":"2024-04-20T21:24:59.864142+0000","last_became_active":"2024-04-20T21:24:58.868533+0000","last_became_peered":"2024-04-20T21:24:58.868533+0000","last_unstale":"2024-04-20T21:24:59.864142+0000","last_undegraded":"2024-04-20T21:24:59.864142+0000","last_fullsized":"2024-04-20T21:24:59.864142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:21:09.622327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"84'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.009304+0000","last_change":"2024-04-20T21:24:57.849023+0000","last_active":"2024-04-20T21:25:00.009304+0000","last_peered":"2024-04-20T21:25:00.009304+0000","last_clean":"2024-04-20T21:25:00.009304+0000","last_became_active":"2024-04-20T21:24:57.848617+0000","last_became_peered":"2024-04-20T21:24:57.848617+0000","last_unstale":"2024-04-20T21:25:00.009304+0000","last_undegraded":"2024-04-20T21:25:00.009304+0000","last_fullsized":"2024-04-20T21:25:00.009304+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:54:16.413274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.862307+0000","last_change":"2024-04-20T21:24:58.869280+0000","last_active":"2024-04-20T21:24:59.862307+0000","last_peered":"2024-04-20T21:24:59.862307+0000","last_clean":"2024-04-20T21:24:59.862307+0000","last_became_active":"2024-04-20T21:24:58.868854+0000","last_became_peered":"2024-04-20T21:24:58.868854+0000","last_unstale":"2024-04-20T21:24:59.862307+0000","last_undegraded":"2024-04-20T21:24:59.862307+0000","last_fullsized":"2024-04-20T21:24:59.862307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:43:31.020198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.008264+0000","last_change":"2024-04-20T21:24:57.848333+0000","last_active":"2024-04-20T21:25:00.008264+0000","last_peered":"2024-04-20T21:25:00.008264+0000","last_clean":"2024-04-20T21:25:00.008264+0000","last_became_active":"2024-04-20T21:24:57.845753+0000","last_became_peered":"2024-04-20T21:24:57.845753+0000","last_unstale":"2024-04-20T21:25:00.008264+0000","last_undegraded":"2024-04-20T21:25:00.008264+0000","last_fullsized":"2024-04-20T21:25:00.008264+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:07:52.687802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.007734+0000","last_change":"2024-04-20T21:24:58.872593+0000","last_active":"2024-04-20T21:25:00.007734+0000","last_peered":"2024-04-20T21:25:00.007734+0000","last_clean":"2024-04-20T21:25:00.007734+0000","last_became_active":"2024-04-20T21:24:58.872039+0000","last_became_peered":"2024-04-20T21:24:58.872039+0000","last_unstale":"2024-04-20T21:25:00.007734+0000","last_undegraded":"2024-04-20T21:25:00.007734+0000","last_fullsized":"2024-04-20T21:25:00.007734+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:54:59.083007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"84'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.007086+0000","last_change":"2024-04-20T21:24:57.834647+0000","last_active":"2024-04-20T21:25:00.007086+0000","last_peered":"2024-04-20T21:25:00.007086+0000","last_clean":"2024-04-20T21:25:00.007086+0000","last_became_active":"2024-04-20T21:24:57.834334+0000","last_became_peered":"2024-04-20T21:24:57.834334+0000","last_unstale":"2024-04-20T21:25:00.007086+0000","last_undegraded":"2024-04-20T21:25:00.007086+0000","last_fullsized":"2024-04-20T21:25:00.007086+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:14:06.238165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.005587+0000","last_change":"2024-04-20T21:24:58.866476+0000","last_active":"2024-04-20T21:25:00.005587+0000","last_peered":"2024-04-20T21:25:00.005587+0000","last_clean":"2024-04-20T21:25:00.005587+0000","last_became_active":"2024-04-20T21:24:58.866185+0000","last_became_peered":"2024-04-20T21:24:58.866185+0000","last_unstale":"2024-04-20T21:25:00.005587+0000","last_undegraded":"2024-04-20T21:25:00.005587+0000","last_fullsized":"2024-04-20T21:25:00.005587+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:53:39.858430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.007814+0000","last_change":"2024-04-20T21:24:57.843053+0000","last_active":"2024-04-20T21:25:00.007814+0000","last_peered":"2024-04-20T21:25:00.007814+0000","last_clean":"2024-04-20T21:25:00.007814+0000","last_became_active":"2024-04-20T21:24:57.842067+0000","last_became_peered":"2024-04-20T21:24:57.842067+0000","last_unstale":"2024-04-20T21:25:00.007814+0000","last_undegraded":"2024-04-20T21:25:00.007814+0000","last_fullsized":"2024-04-20T21:25:00.007814+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:57:50.784411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.007147+0000","last_change":"2024-04-20T21:24:58.865864+0000","last_active":"2024-04-20T21:25:00.007147+0000","last_peered":"2024-04-20T21:25:00.007147+0000","last_clean":"2024-04-20T21:25:00.007147+0000","last_became_active":"2024-04-20T21:24:58.865594+0000","last_became_peered":"2024-04-20T21:24:58.865594+0000","last_unstale":"2024-04-20T21:25:00.007147+0000","last_undegraded":"2024-04-20T21:25:00.007147+0000","last_fullsized":"2024-04-20T21:25:00.007147+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:42:51.118239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.005024+0000","last_change":"2024-04-20T21:24:57.848088+0000","last_active":"2024-04-20T21:25:00.005024+0000","last_peered":"2024-04-20T21:25:00.005024+0000","last_clean":"2024-04-20T21:25:00.005024+0000","last_became_active":"2024-04-20T21:24:57.847784+0000","last_became_peered":"2024-04-20T21:24:57.847784+0000","last_unstale":"2024-04-20T21:25:00.005024+0000","last_undegraded":"2024-04-20T21:25:00.005024+0000","last_fullsized":"2024-04-20T21:25:00.005024+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:44:21.335046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.004882+0000","last_change":"2024-04-20T21:24:58.874494+0000","last_active":"2024-04-20T21:25:00.004882+0000","last_peered":"2024-04-20T21:25:00.004882+0000","last_clean":"2024-04-20T21:25:00.004882+0000","last_became_active":"2024-04-20T21:24:58.874218+0000","last_became_peered":"2024-04-20T21:24:58.874218+0000","last_unstale":"2024-04-20T21:25:00.004882+0000","last_undegraded":"2024-04-20T21:25:00.004882+0000","last_fullsized":"2024-04-20T21:25:00.004882+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:15:18.594034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.004308+0000","last_change":"2024-04-20T21:24:57.836748+0000","last_active":"2024-04-20T21:25:00.004308+0000","last_peered":"2024-04-20T21:25:00.004308+0000","last_clean":"2024-04-20T21:25:00.004308+0000","last_became_active":"2024-04-20T21:24:57.836442+0000","last_became_peered":"2024-04-20T21:24:57.836442+0000","last_unstale":"2024-04-20T21:25:00.004308+0000","last_undegraded":"2024-04-20T21:25:00.004308+0000","last_fullsized":"2024-04-20T21:25:00.004308+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:25:55.126550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.862598+0000","last_change":"2024-04-20T21:24:58.868235+0000","last_active":"2024-04-20T21:24:59.862598+0000","last_peered":"2024-04-20T21:24:59.862598+0000","last_clean":"2024-04-20T21:24:59.862598+0000","last_became_active":"2024-04-20T21:24:58.867797+0000","last_became_peered":"2024-04-20T21:24:58.867797+0000","last_unstale":"2024-04-20T21:24:59.862598+0000","last_undegraded":"2024-04-20T21:24:59.862598+0000","last_fullsized":"2024-04-20T21:24:59.862598+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:16:20.655880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"84'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.862495+0000","last_change":"2024-04-20T21:24:57.839629+0000","last_active":"2024-04-20T21:24:59.862495+0000","last_peered":"2024-04-20T21:24:59.862495+0000","last_clean":"2024-04-20T21:24:59.862495+0000","last_became_active":"2024-04-20T21:24:57.839237+0000","last_became_peered":"2024-04-20T21:24:57.839237+0000","last_unstale":"2024-04-20T21:24:59.862495+0000","last_undegraded":"2024-04-20T21:24:59.862495+0000","last_fullsized":"2024-04-20T21:24:59.862495+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:01:25.431255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.005553+0000","last_change":"2024-04-20T21:24:58.873694+0000","last_active":"2024-04-20T21:25:00.005553+0000","last_peered":"2024-04-20T21:25:00.005553+0000","last_clean":"2024-04-20T21:25:00.005553+0000","last_became_active":"2024-04-20T21:24:58.873385+0000","last_became_peered":"2024-04-20T21:24:58.873385+0000","last_unstale":"2024-04-20T21:25:00.005553+0000","last_undegraded":"2024-04-20T21:25:00.005553+0000","last_fullsized":"2024-04-20T21:25:00.005553+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:55:37.672502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.862205+0000","last_change":"2024-04-20T21:24:57.846774+0000","last_active":"2024-04-20T21:24:59.862205+0000","last_peered":"2024-04-20T21:24:59.862205+0000","last_clean":"2024-04-20T21:24:59.862205+0000","last_became_active":"2024-04-20T21:24:57.846445+0000","last_became_peered":"2024-04-20T21:24:57.846445+0000","last_unstale":"2024-04-20T21:24:59.862205+0000","last_undegraded":"2024-04-20T21:24:59.862205+0000","last_fullsized":"2024-04-20T21:24:59.862205+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:33:43.984573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.864169+0000","last_change":"2024-04-20T21:24:58.863528+0000","last_active":"2024-04-20T21:24:59.864169+0000","last_peered":"2024-04-20T21:24:59.864169+0000","last_clean":"2024-04-20T21:24:59.864169+0000","last_became_active":"2024-04-20T21:24:58.863162+0000","last_became_peered":"2024-04-20T21:24:58.863162+0000","last_unstale":"2024-04-20T21:24:59.864169+0000","last_undegraded":"2024-04-20T21:24:59.864169+0000","last_fullsized":"2024-04-20T21:24:59.864169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:54:17.428125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.863816+0000","last_change":"2024-04-20T21:24:57.848202+0000","last_active":"2024-04-20T21:24:59.863816+0000","last_peered":"2024-04-20T21:24:59.863816+0000","last_clean":"2024-04-20T21:24:59.863816+0000","last_became_active":"2024-04-20T21:24:57.847601+0000","last_became_peered":"2024-04-20T21:24:57.847601+0000","last_unstale":"2024-04-20T21:24:59.863816+0000","last_undegraded":"2024-04-20T21:24:59.863816+0000","last_fullsized":"2024-04-20T21:24:59.863816+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:00:14.783672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.862427+0000","last_change":"2024-04-20T21:24:58.862205+0000","last_active":"2024-04-20T21:24:59.862427+0000","last_peered":"2024-04-20T21:24:59.862427+0000","last_clean":"2024-04-20T21:24:59.862427+0000","last_became_active":"2024-04-20T21:24:58.861501+0000","last_became_peered":"2024-04-20T21:24:58.861501+0000","last_unstale":"2024-04-20T21:24:59.862427+0000","last_undegraded":"2024-04-20T21:24:59.862427+0000","last_fullsized":"2024-04-20T21:24:59.862427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:55:37.672502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.864357+0000","last_change":"2024-04-20T21:24:57.847038+0000","last_active":"2024-04-20T21:24:59.864357+0000","last_peered":"2024-04-20T21:24:59.864357+0000","last_clean":"2024-04-20T21:24:59.864357+0000","last_became_active":"2024-04-20T21:24:57.846453+0000","last_became_peered":"2024-04-20T21:24:57.846453+0000","last_unstale":"2024-04-20T21:24:59.864357+0000","last_undegraded":"2024-04-20T21:24:59.864357+0000","last_fullsized":"2024-04-20T21:24:59.864357+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:03:18.054330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.004982+0000","last_change":"2024-04-20T21:24:58.864569+0000","last_active":"2024-04-20T21:25:00.004982+0000","last_peered":"2024-04-20T21:25:00.004982+0000","last_clean":"2024-04-20T21:25:00.004982+0000","last_became_active":"2024-04-20T21:24:58.864224+0000","last_became_peered":"2024-04-20T21:24:58.864224+0000","last_unstale":"2024-04-20T21:25:00.004982+0000","last_undegraded":"2024-04-20T21:25:00.004982+0000","last_fullsized":"2024-04-20T21:25:00.004982+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:50:38.147663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.006057+0000","last_change":"2024-04-20T21:24:57.843606+0000","last_active":"2024-04-20T21:25:00.006057+0000","last_peered":"2024-04-20T21:25:00.006057+0000","last_clean":"2024-04-20T21:25:00.006057+0000","last_became_active":"2024-04-20T21:24:57.842392+0000","last_became_peered":"2024-04-20T21:24:57.842392+0000","last_unstale":"2024-04-20T21:25:00.006057+0000","last_undegraded":"2024-04-20T21:25:00.006057+0000","last_fullsized":"2024-04-20T21:25:00.006057+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:44:27.993010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.005844+0000","last_change":"2024-04-20T21:24:58.866969+0000","last_active":"2024-04-20T21:25:00.005844+0000","last_peered":"2024-04-20T21:25:00.005844+0000","last_clean":"2024-04-20T21:25:00.005844+0000","last_became_active":"2024-04-20T21:24:58.864164+0000","last_became_peered":"2024-04-20T21:24:58.864164+0000","last_unstale":"2024-04-20T21:25:00.005844+0000","last_undegraded":"2024-04-20T21:25:00.005844+0000","last_fullsized":"2024-04-20T21:25:00.005844+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:03:04.971095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.861933+0000","last_change":"2024-04-20T21:24:57.841023+0000","last_active":"2024-04-20T21:24:59.861933+0000","last_peered":"2024-04-20T21:24:59.861933+0000","last_clean":"2024-04-20T21:24:59.861933+0000","last_became_active":"2024-04-20T21:24:57.840669+0000","last_became_peered":"2024-04-20T21:24:57.840669+0000","last_unstale":"2024-04-20T21:24:59.861933+0000","last_undegraded":"2024-04-20T21:24:59.861933+0000","last_fullsized":"2024-04-20T21:24:59.861933+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:02:23.398563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.006509+0000","last_change":"2024-04-20T21:24:58.878246+0000","last_active":"2024-04-20T21:25:00.006509+0000","last_peered":"2024-04-20T21:25:00.006509+0000","last_clean":"2024-04-20T21:25:00.006509+0000","last_became_active":"2024-04-20T21:24:58.877900+0000","last_became_peered":"2024-04-20T21:24:58.877900+0000","last_unstale":"2024-04-20T21:25:00.006509+0000","last_undegraded":"2024-04-20T21:25:00.006509+0000","last_fullsized":"2024-04-20T21:25:00.006509+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:19:11.569824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.864426+0000","last_change":"2024-04-20T21:24:57.850383+0000","last_active":"2024-04-20T21:24:59.864426+0000","last_peered":"2024-04-20T21:24:59.864426+0000","last_clean":"2024-04-20T21:24:59.864426+0000","last_became_active":"2024-04-20T21:24:57.850069+0000","last_became_peered":"2024-04-20T21:24:57.850069+0000","last_unstale":"2024-04-20T21:24:59.864426+0000","last_undegraded":"2024-04-20T21:24:59.864426+0000","last_fullsized":"2024-04-20T21:24:59.864426+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:21:17.509431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.864304+0000","last_change":"2024-04-20T21:24:58.862654+0000","last_active":"2024-04-20T21:24:59.864304+0000","last_peered":"2024-04-20T21:24:59.864304+0000","last_clean":"2024-04-20T21:24:59.864304+0000","last_became_active":"2024-04-20T21:24:58.862360+0000","last_became_peered":"2024-04-20T21:24:58.862360+0000","last_unstale":"2024-04-20T21:24:59.864304+0000","last_undegraded":"2024-04-20T21:24:59.864304+0000","last_fullsized":"2024-04-20T21:24:59.864304+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:07:53.702653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.861459+0000","last_change":"2024-04-20T21:24:57.850740+0000","last_active":"2024-04-20T21:24:59.861459+0000","last_peered":"2024-04-20T21:24:59.861459+0000","last_clean":"2024-04-20T21:24:59.861459+0000","last_became_active":"2024-04-20T21:24:57.850376+0000","last_became_peered":"2024-04-20T21:24:57.850376+0000","last_unstale":"2024-04-20T21:24:59.861459+0000","last_undegraded":"2024-04-20T21:24:59.861459+0000","last_fullsized":"2024-04-20T21:24:59.861459+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:38:19.894629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.862534+0000","last_change":"2024-04-20T21:24:58.867774+0000","last_active":"2024-04-20T21:24:59.862534+0000","last_peered":"2024-04-20T21:24:59.862534+0000","last_clean":"2024-04-20T21:24:59.862534+0000","last_became_active":"2024-04-20T21:24:58.867089+0000","last_became_peered":"2024-04-20T21:24:58.867089+0000","last_unstale":"2024-04-20T21:24:59.862534+0000","last_undegraded":"2024-04-20T21:24:59.862534+0000","last_fullsized":"2024-04-20T21:24:59.862534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:19:18.784416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"84'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.005771+0000","last_change":"2024-04-20T21:24:57.840537+0000","last_active":"2024-04-20T21:25:00.005771+0000","last_peered":"2024-04-20T21:25:00.005771+0000","last_clean":"2024-04-20T21:25:00.005771+0000","last_became_active":"2024-04-20T21:24:57.840146+0000","last_became_peered":"2024-04-20T21:24:57.840146+0000","last_unstale":"2024-04-20T21:25:00.005771+0000","last_undegraded":"2024-04-20T21:25:00.005771+0000","last_fullsized":"2024-04-20T21:25:00.005771+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:21:08.607476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":287,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.006352+0000","last_change":"2024-04-20T21:17:43.059637+0000","last_active":"2024-04-20T21:25:00.006352+0000","last_peered":"2024-04-20T21:25:00.006352+0000","last_clean":"2024-04-20T21:25:00.006352+0000","last_became_active":"2024-04-20T21:17:43.056207+0000","last_became_peered":"2024-04-20T21:17:43.056207+0000","last_unstale":"2024-04-20T21:25:00.006352+0000","last_undegraded":"2024-04-20T21:25:00.006352+0000","last_fullsized":"2024-04-20T21:25:00.006352+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:41.036290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:41.036290+0000","last_clean_scrub_stamp":"2024-04-20T21:17:41.036290+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:58:51.091337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574938,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32784,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339056,"statfs":{"total":96636764160,"available":96603193344,"internally_reserved":0,"allocated":1642496,"data_stored":1036920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574939,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32212,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339628,"statfs":{"total":96636764160,"available":96603779072,"internally_reserved":0,"allocated":1056768,"data_stored":447176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574941,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32224,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339616,"statfs":{"total":96636764160,"available":96603766784,"internally_reserved":0,"allocated":1060864,"data_stored":448116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574940,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32192,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339648,"statfs":{"total":96636764160,"available":96603799552,"internally_reserved":0,"allocated":1032192,"data_stored":440182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574940,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32212,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339628,"statfs":{"total":96636764160,"available":96603779072,"internally_reserved":0,"allocated":1056768,"data_stored":448094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1048576,"data_stored":447064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32220,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339620,"statfs":{"total":96636764160,"available":96603770880,"internally_reserved":0,"allocated":1056768,"data_stored":446676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574940,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32788,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339052,"statfs":{"total":96636764160,"available":96603189248,"internally_reserved":0,"allocated":1642496,"data_stored":1036884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T21:25:03.265 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T21:25:03.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.503+0000 7f25bde00640 1 Processor -- start 2024-04-20T21:25:03.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.503+0000 7f25bde00640 1 -- start start 2024-04-20T21:25:03.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.505+0000 7f25bde00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f25b8071d00 0x7f25b8072180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:03.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.505+0000 7f25bde00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25b806fb70 con 0x7f25b8163200 2024-04-20T21:25:03.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.505+0000 7f25bde00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25b815d6a0 con 0x7f25b815d2c0 2024-04-20T21:25:03.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.505+0000 7f25bde00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f25b8166900 con 0x7f25b8071d00 2024-04-20T21:25:03.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.505+0000 7f25b6000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f25b8071d00 0x7f25b8072180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:03.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.505+0000 7f25b6000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f25b8071d00 0x7f25b8072180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43410/0 (socket says 172.21.15.86:43410) 2024-04-20T21:25:03.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.505+0000 7f25b6000640 1 -- 172.21.15.86:0/4266178555 learned_addr learned my addr 172.21.15.86:0/4266178555 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:03.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.506+0000 7f25b6000640 1 -- 172.21.15.86:0/4266178555 >> v1:172.21.15.144:6789/0 conn(0x7f25b815d2c0 legacy=0x7f25b806f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:03.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.506+0000 7f25b6000640 1 -- 172.21.15.86:0/4266178555 >> v1:172.21.15.86:6789/0 conn(0x7f25b8163200 legacy=0x7f25b8163650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:03.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.506+0000 7f25b6000640 1 -- 172.21.15.86:0/4266178555 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25b8166be0 con 0x7f25b8071d00 2024-04-20T21:25:03.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.507+0000 7f25b6000640 1 --2- 172.21.15.86:0/4266178555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25b8071d00 0x7f25b8072180 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f25a4009870 tx=0x7f25a402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a151b96d73aa83e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:03.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.507+0000 7f25b5600640 1 -- 172.21.15.86:0/4266178555 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a4034070 con 0x7f25b8071d00 2024-04-20T21:25:03.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.507+0000 7f25b5600640 1 -- 172.21.15.86:0/4266178555 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25a4030860 con 0x7f25b8071d00 2024-04-20T21:25:03.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.507+0000 7f25b5600640 1 -- 172.21.15.86:0/4266178555 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a4030c30 con 0x7f25b8071d00 2024-04-20T21:25:03.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.508+0000 7f25bde00640 1 -- 172.21.15.86:0/4266178555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25b8071d00 msgr2=0x7f25b8072180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:03.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.508+0000 7f25bde00640 1 --2- 172.21.15.86:0/4266178555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25b8071d00 0x7f25b8072180 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f25a4009870 tx=0x7f25a402b9a0 comp rx=0 tx=0).stop 2024-04-20T21:25:03.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.508+0000 7f25bde00640 1 -- 172.21.15.86:0/4266178555 shutdown_connections 2024-04-20T21:25:03.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.508+0000 7f25bde00640 1 --2- 172.21.15.86:0/4266178555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25b8071d00 0x7f25b8072180 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:03.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.508+0000 7f25bde00640 1 -- 172.21.15.86:0/4266178555 >> 172.21.15.86:0/4266178555 conn(0x7f25b80b2360 msgr2=0x7f25b80b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:03.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.508+0000 7f25bde00640 1 -- 172.21.15.86:0/4266178555 shutdown_connections 2024-04-20T21:25:03.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.508+0000 7f25bde00640 1 -- 172.21.15.86:0/4266178555 wait complete. 2024-04-20T21:25:03.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.510+0000 7f25bde00640 1 Processor -- start 2024-04-20T21:25:03.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.510+0000 7f25bde00640 1 -- start start 2024-04-20T21:25:03.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.511+0000 7f25bde00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25b8071d00 0x7f25b81423f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:03.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.511+0000 7f25bde00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25b815d2c0 0x7f25b815c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:03.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.511+0000 7f25bde00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25b8163200 0x7f25b806dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:03.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.511+0000 7f25bde00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f25b81676c0 con 0x7f25b8071d00 2024-04-20T21:25:03.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.511+0000 7f25bde00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f25b81671a0 con 0x7f25b815d2c0 2024-04-20T21:25:03.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.511+0000 7f25bde00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f25b80b42c0 con 0x7f25b8163200 2024-04-20T21:25:03.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.511+0000 7f25b6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25b8071d00 0x7f25b81423f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:03.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.511+0000 7f25b6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25b8071d00 0x7f25b81423f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43422/0 (socket says 172.21.15.86:43422) 2024-04-20T21:25:03.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.511+0000 7f25b6a00640 1 -- 172.21.15.86:0/2702042208 learned_addr learned my addr 172.21.15.86:0/2702042208 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:03.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.511+0000 7f25b6000640 1 --2- 172.21.15.86:0/2702042208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25b815d2c0 0x7f25b815c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:03.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.511+0000 7f25b6a00640 1 -- 172.21.15.86:0/2702042208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25b8163200 msgr2=0x7f25b806dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:03.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.511+0000 7f25b7400640 1 --2- 172.21.15.86:0/2702042208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25b8163200 0x7f25b806dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:03.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.512+0000 7f25b6a00640 1 --2- 172.21.15.86:0/2702042208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25b8163200 0x7f25b806dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:03.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.512+0000 7f25b6a00640 1 -- 172.21.15.86:0/2702042208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25b815d2c0 msgr2=0x7f25b815c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:03.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.512+0000 7f25b6a00640 1 --2- 172.21.15.86:0/2702042208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25b815d2c0 0x7f25b815c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:03.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.512+0000 7f25b6a00640 1 -- 172.21.15.86:0/2702042208 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25b8164f20 con 0x7f25b8071d00 2024-04-20T21:25:03.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.512+0000 7f25b7400640 1 --2- 172.21.15.86:0/2702042208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25b8163200 0x7f25b806dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:25:03.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.512+0000 7f25b6a00640 1 --2- 172.21.15.86:0/2702042208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25b8071d00 0x7f25b81423f0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f25a800c430 tx=0x7f25a800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:03.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.512+0000 7f25af400640 1 -- 172.21.15.86:0/2702042208 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a8011790 con 0x7f25b8071d00 2024-04-20T21:25:03.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.512+0000 7f25af400640 1 -- 172.21.15.86:0/2702042208 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25a8017070 con 0x7f25b8071d00 2024-04-20T21:25:03.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.512+0000 7f25bde00640 1 -- 172.21.15.86:0/2702042208 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25b8165200 con 0x7f25b8071d00 2024-04-20T21:25:03.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.512+0000 7f25af400640 1 -- 172.21.15.86:0/2702042208 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a8011bb0 con 0x7f25b8071d00 2024-04-20T21:25:03.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.512+0000 7f25bde00640 1 -- 172.21.15.86:0/2702042208 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25b81657c0 con 0x7f25b8071d00 2024-04-20T21:25:03.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.513+0000 7f25bde00640 1 -- 172.21.15.86:0/2702042208 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25b80689e0 con 0x7f25b8071d00 2024-04-20T21:25:03.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.514+0000 7f25af400640 1 -- 172.21.15.86:0/2702042208 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f25a801c460 con 0x7f25b8071d00 2024-04-20T21:25:03.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.515+0000 7f25af400640 1 --2- 172.21.15.86:0/2702042208 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f259806c8f0 0x7f259806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:03.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.515+0000 7f25af400640 1 -- 172.21.15.86:0/2702042208 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7f25a808b980 con 0x7f25b8071d00 2024-04-20T21:25:03.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.516+0000 7f25b6000640 1 --2- 172.21.15.86:0/2702042208 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f259806c8f0 0x7f259806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:03.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.516+0000 7f25b6000640 1 --2- 172.21.15.86:0/2702042208 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f259806c8f0 0x7f259806edb0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f25a4036000 tx=0x7f25a403b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:03.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.520+0000 7f25af400640 1 -- 172.21.15.86:0/2702042208 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f25a8059e20 con 0x7f25b8071d00 2024-04-20T21:25:03.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.715+0000 7f25bde00640 1 -- 172.21.15.86:0/2702042208 --> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f25b806ec60 con 0x7f259806c8f0 2024-04-20T21:25:03.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.733+0000 7f25af400640 1 -- 172.21.15.86:0/2702042208 <== mgr.4107 v2:172.21.15.86:6832/302045240 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252082 (secure 0 0 0) 0x7f25b806ec60 con 0x7f259806c8f0 2024-04-20T21:25:03.735 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:25:03.738 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-20T21:25:03.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.740+0000 7f25bde00640 1 -- 172.21.15.86:0/2702042208 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f259806c8f0 msgr2=0x7f259806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:03.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.740+0000 7f25bde00640 1 --2- 172.21.15.86:0/2702042208 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f259806c8f0 0x7f259806edb0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f25a4036000 tx=0x7f25a403b040 comp rx=0 tx=0).stop 2024-04-20T21:25:03.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.740+0000 7f25bde00640 1 -- 172.21.15.86:0/2702042208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25b8071d00 msgr2=0x7f25b81423f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:03.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.740+0000 7f25bde00640 1 --2- 172.21.15.86:0/2702042208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25b8071d00 0x7f25b81423f0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f25a800c430 tx=0x7f25a800c8f0 comp rx=0 tx=0).stop 2024-04-20T21:25:03.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.740+0000 7f25bde00640 1 -- 172.21.15.86:0/2702042208 shutdown_connections 2024-04-20T21:25:03.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.740+0000 7f25bde00640 1 --2- 172.21.15.86:0/2702042208 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f259806c8f0 0x7f259806edb0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:03.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.740+0000 7f25bde00640 1 --2- 172.21.15.86:0/2702042208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f25b8163200 0x7f25b806dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:03.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.740+0000 7f25bde00640 1 --2- 172.21.15.86:0/2702042208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25b815d2c0 0x7f25b815c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:03.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.740+0000 7f25bde00640 1 --2- 172.21.15.86:0/2702042208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25b8071d00 0x7f25b81423f0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:03.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.740+0000 7f25bde00640 1 -- 172.21.15.86:0/2702042208 >> 172.21.15.86:0/2702042208 conn(0x7f25b80b2360 msgr2=0x7f25b80b51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:03.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.740+0000 7f25bde00640 1 -- 172.21.15.86:0/2702042208 shutdown_connections 2024-04-20T21:25:03.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:03.740+0000 7f25bde00640 1 -- 172.21.15.86:0/2702042208 wait complete. 2024-04-20T21:25:03.759 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":299,"stamp":"2024-04-20T21:25:02.781606+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":132,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258836,"kb_used_data":9372,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715884,"statfs":{"total":773094113280,"available":772829065216,"internally_reserved":0,"allocated":9596928,"data_stored":4751112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.975035"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.863801+0000","last_change":"2024-04-20T21:24:58.860219+0000","last_active":"2024-04-20T21:24:59.863801+0000","last_peered":"2024-04-20T21:24:59.863801+0000","last_clean":"2024-04-20T21:24:59.863801+0000","last_became_active":"2024-04-20T21:24:58.859934+0000","last_became_peered":"2024-04-20T21:24:58.859934+0000","last_unstale":"2024-04-20T21:24:59.863801+0000","last_undegraded":"2024-04-20T21:24:59.863801+0000","last_fullsized":"2024-04-20T21:24:59.863801+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:44:55.916343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.862100+0000","last_change":"2024-04-20T21:24:58.868066+0000","last_active":"2024-04-20T21:24:59.862100+0000","last_peered":"2024-04-20T21:24:59.862100+0000","last_clean":"2024-04-20T21:24:59.862100+0000","last_became_active":"2024-04-20T21:24:58.867725+0000","last_became_peered":"2024-04-20T21:24:58.867725+0000","last_unstale":"2024-04-20T21:24:59.862100+0000","last_undegraded":"2024-04-20T21:24:59.862100+0000","last_fullsized":"2024-04-20T21:24:59.862100+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:21:18.524282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.005010+0000","last_change":"2024-04-20T21:24:58.862953+0000","last_active":"2024-04-20T21:25:00.005010+0000","last_peered":"2024-04-20T21:25:00.005010+0000","last_clean":"2024-04-20T21:25:00.005010+0000","last_became_active":"2024-04-20T21:24:58.862527+0000","last_became_peered":"2024-04-20T21:24:58.862527+0000","last_unstale":"2024-04-20T21:25:00.005010+0000","last_undegraded":"2024-04-20T21:25:00.005010+0000","last_fullsized":"2024-04-20T21:25:00.005010+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:31:50.250214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.007567+0000","last_change":"2024-04-20T21:24:58.873888+0000","last_active":"2024-04-20T21:25:00.007567+0000","last_peered":"2024-04-20T21:25:00.007567+0000","last_clean":"2024-04-20T21:25:00.007567+0000","last_became_active":"2024-04-20T21:24:58.871153+0000","last_became_peered":"2024-04-20T21:24:58.871153+0000","last_unstale":"2024-04-20T21:25:00.007567+0000","last_undegraded":"2024-04-20T21:25:00.007567+0000","last_fullsized":"2024-04-20T21:25:00.007567+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:33:44.999424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.861730+0000","last_change":"2024-04-20T21:24:58.879526+0000","last_active":"2024-04-20T21:24:59.861730+0000","last_peered":"2024-04-20T21:24:59.861730+0000","last_clean":"2024-04-20T21:24:59.861730+0000","last_became_active":"2024-04-20T21:24:58.879157+0000","last_became_peered":"2024-04-20T21:24:58.879157+0000","last_unstale":"2024-04-20T21:24:59.861730+0000","last_undegraded":"2024-04-20T21:24:59.861730+0000","last_fullsized":"2024-04-20T21:24:59.861730+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:25:56.141401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.004598+0000","last_change":"2024-04-20T21:24:58.864812+0000","last_active":"2024-04-20T21:25:00.004598+0000","last_peered":"2024-04-20T21:25:00.004598+0000","last_clean":"2024-04-20T21:25:00.004598+0000","last_became_active":"2024-04-20T21:24:58.864251+0000","last_became_peered":"2024-04-20T21:24:58.864251+0000","last_unstale":"2024-04-20T21:25:00.004598+0000","last_undegraded":"2024-04-20T21:25:00.004598+0000","last_fullsized":"2024-04-20T21:25:00.004598+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:07:53.702653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.005019+0000","last_change":"2024-04-20T21:24:58.873317+0000","last_active":"2024-04-20T21:25:00.005019+0000","last_peered":"2024-04-20T21:25:00.005019+0000","last_clean":"2024-04-20T21:25:00.005019+0000","last_became_active":"2024-04-20T21:24:58.867489+0000","last_became_peered":"2024-04-20T21:24:58.867489+0000","last_unstale":"2024-04-20T21:25:00.005019+0000","last_undegraded":"2024-04-20T21:25:00.005019+0000","last_fullsized":"2024-04-20T21:25:00.005019+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:19:34.543192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.004584+0000","last_change":"2024-04-20T21:24:58.878924+0000","last_active":"2024-04-20T21:25:00.004584+0000","last_peered":"2024-04-20T21:25:00.004584+0000","last_clean":"2024-04-20T21:25:00.004584+0000","last_became_active":"2024-04-20T21:24:58.878410+0000","last_became_peered":"2024-04-20T21:24:58.878410+0000","last_unstale":"2024-04-20T21:25:00.004584+0000","last_undegraded":"2024-04-20T21:25:00.004584+0000","last_fullsized":"2024-04-20T21:25:00.004584+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:54:17.428125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.005565+0000","last_change":"2024-04-20T21:24:58.866708+0000","last_active":"2024-04-20T21:25:00.005565+0000","last_peered":"2024-04-20T21:25:00.005565+0000","last_clean":"2024-04-20T21:25:00.005565+0000","last_became_active":"2024-04-20T21:24:58.865109+0000","last_became_peered":"2024-04-20T21:24:58.865109+0000","last_unstale":"2024-04-20T21:25:00.005565+0000","last_undegraded":"2024-04-20T21:25:00.005565+0000","last_fullsized":"2024-04-20T21:25:00.005565+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:19:46.273364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.864554+0000","last_change":"2024-04-20T21:24:58.859111+0000","last_active":"2024-04-20T21:24:59.864554+0000","last_peered":"2024-04-20T21:24:59.864554+0000","last_clean":"2024-04-20T21:24:59.864554+0000","last_became_active":"2024-04-20T21:24:58.858791+0000","last_became_peered":"2024-04-20T21:24:58.858791+0000","last_unstale":"2024-04-20T21:24:59.864554+0000","last_undegraded":"2024-04-20T21:24:59.864554+0000","last_fullsized":"2024-04-20T21:24:59.864554+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:15:18.594034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.862450+0000","last_change":"2024-04-20T21:24:58.873582+0000","last_active":"2024-04-20T21:24:59.862450+0000","last_peered":"2024-04-20T21:24:59.862450+0000","last_clean":"2024-04-20T21:24:59.862450+0000","last_became_active":"2024-04-20T21:24:58.873181+0000","last_became_peered":"2024-04-20T21:24:58.873181+0000","last_unstale":"2024-04-20T21:24:59.862450+0000","last_undegraded":"2024-04-20T21:24:59.862450+0000","last_fullsized":"2024-04-20T21:24:59.862450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:03:19.069181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.863687+0000","last_change":"2024-04-20T21:24:58.860151+0000","last_active":"2024-04-20T21:24:59.863687+0000","last_peered":"2024-04-20T21:24:59.863687+0000","last_clean":"2024-04-20T21:24:59.863687+0000","last_became_active":"2024-04-20T21:24:58.858389+0000","last_became_peered":"2024-04-20T21:24:58.858389+0000","last_unstale":"2024-04-20T21:24:59.863687+0000","last_undegraded":"2024-04-20T21:24:59.863687+0000","last_fullsized":"2024-04-20T21:24:59.863687+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:57:14.691985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.005452+0000","last_change":"2024-04-20T21:24:58.865960+0000","last_active":"2024-04-20T21:25:00.005452+0000","last_peered":"2024-04-20T21:25:00.005452+0000","last_clean":"2024-04-20T21:25:00.005452+0000","last_became_active":"2024-04-20T21:24:58.865636+0000","last_became_peered":"2024-04-20T21:24:58.865636+0000","last_unstale":"2024-04-20T21:25:00.005452+0000","last_undegraded":"2024-04-20T21:25:00.005452+0000","last_fullsized":"2024-04-20T21:25:00.005452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:03:04.971095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.007530+0000","last_change":"2024-04-20T21:24:58.872800+0000","last_active":"2024-04-20T21:25:00.007530+0000","last_peered":"2024-04-20T21:25:00.007530+0000","last_clean":"2024-04-20T21:25:00.007530+0000","last_became_active":"2024-04-20T21:24:58.872456+0000","last_became_peered":"2024-04-20T21:24:58.872456+0000","last_unstale":"2024-04-20T21:25:00.007530+0000","last_undegraded":"2024-04-20T21:25:00.007530+0000","last_fullsized":"2024-04-20T21:25:00.007530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:16:20.955774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.005466+0000","last_change":"2024-04-20T21:24:58.862822+0000","last_active":"2024-04-20T21:25:00.005466+0000","last_peered":"2024-04-20T21:25:00.005466+0000","last_clean":"2024-04-20T21:25:00.005466+0000","last_became_active":"2024-04-20T21:24:58.862304+0000","last_became_peered":"2024-04-20T21:24:58.862304+0000","last_unstale":"2024-04-20T21:25:00.005466+0000","last_undegraded":"2024-04-20T21:25:00.005466+0000","last_fullsized":"2024-04-20T21:25:00.005466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:55:37.672502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.861478+0000","last_change":"2024-04-20T21:24:58.852755+0000","last_active":"2024-04-20T21:24:59.861478+0000","last_peered":"2024-04-20T21:24:59.861478+0000","last_clean":"2024-04-20T21:24:59.861478+0000","last_became_active":"2024-04-20T21:24:58.851389+0000","last_became_peered":"2024-04-20T21:24:58.851389+0000","last_unstale":"2024-04-20T21:24:59.861478+0000","last_undegraded":"2024-04-20T21:24:59.861478+0000","last_fullsized":"2024-04-20T21:24:59.861478+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:53:39.858430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.863944+0000","last_change":"2024-04-20T21:24:58.855876+0000","last_active":"2024-04-20T21:24:59.863944+0000","last_peered":"2024-04-20T21:24:59.863944+0000","last_clean":"2024-04-20T21:24:59.863944+0000","last_became_active":"2024-04-20T21:24:58.855594+0000","last_became_peered":"2024-04-20T21:24:58.855594+0000","last_unstale":"2024-04-20T21:24:59.863944+0000","last_undegraded":"2024-04-20T21:24:59.863944+0000","last_fullsized":"2024-04-20T21:24:59.863944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:10:48.245778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.007206+0000","last_change":"2024-04-20T21:24:58.866512+0000","last_active":"2024-04-20T21:25:00.007206+0000","last_peered":"2024-04-20T21:25:00.007206+0000","last_clean":"2024-04-20T21:25:00.007206+0000","last_became_active":"2024-04-20T21:24:58.862188+0000","last_became_peered":"2024-04-20T21:24:58.862188+0000","last_unstale":"2024-04-20T21:25:00.007206+0000","last_undegraded":"2024-04-20T21:25:00.007206+0000","last_fullsized":"2024-04-20T21:25:00.007206+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:49:13.697624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.008675+0000","last_change":"2024-04-20T21:24:58.858015+0000","last_active":"2024-04-20T21:25:00.008675+0000","last_peered":"2024-04-20T21:25:00.008675+0000","last_clean":"2024-04-20T21:25:00.008675+0000","last_became_active":"2024-04-20T21:24:58.857654+0000","last_became_peered":"2024-04-20T21:24:58.857654+0000","last_unstale":"2024-04-20T21:25:00.008675+0000","last_undegraded":"2024-04-20T21:25:00.008675+0000","last_fullsized":"2024-04-20T21:25:00.008675+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:43:31.020198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.007900+0000","last_change":"2024-04-20T21:24:58.864303+0000","last_active":"2024-04-20T21:25:00.007900+0000","last_peered":"2024-04-20T21:25:00.007900+0000","last_clean":"2024-04-20T21:25:00.007900+0000","last_became_active":"2024-04-20T21:24:58.864035+0000","last_became_peered":"2024-04-20T21:24:58.864035+0000","last_unstale":"2024-04-20T21:25:00.007900+0000","last_undegraded":"2024-04-20T21:25:00.007900+0000","last_fullsized":"2024-04-20T21:25:00.007900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:46:41.694861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.007373+0000","last_change":"2024-04-20T21:24:58.863754+0000","last_active":"2024-04-20T21:25:00.007373+0000","last_peered":"2024-04-20T21:25:00.007373+0000","last_clean":"2024-04-20T21:25:00.007373+0000","last_became_active":"2024-04-20T21:24:58.863162+0000","last_became_peered":"2024-04-20T21:24:58.863162+0000","last_unstale":"2024-04-20T21:25:00.007373+0000","last_undegraded":"2024-04-20T21:25:00.007373+0000","last_fullsized":"2024-04-20T21:25:00.007373+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:29:51.537408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.861860+0000","last_change":"2024-04-20T21:24:58.878056+0000","last_active":"2024-04-20T21:24:59.861860+0000","last_peered":"2024-04-20T21:24:59.861860+0000","last_clean":"2024-04-20T21:24:59.861860+0000","last_became_active":"2024-04-20T21:24:58.877793+0000","last_became_peered":"2024-04-20T21:24:58.877793+0000","last_unstale":"2024-04-20T21:24:59.861860+0000","last_undegraded":"2024-04-20T21:24:59.861860+0000","last_fullsized":"2024-04-20T21:24:59.861860+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:48:22.499537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.864060+0000","last_change":"2024-04-20T21:24:58.861713+0000","last_active":"2024-04-20T21:24:59.864060+0000","last_peered":"2024-04-20T21:24:59.864060+0000","last_clean":"2024-04-20T21:24:59.864060+0000","last_became_active":"2024-04-20T21:24:58.861381+0000","last_became_peered":"2024-04-20T21:24:58.861381+0000","last_unstale":"2024-04-20T21:24:59.864060+0000","last_undegraded":"2024-04-20T21:24:59.864060+0000","last_fullsized":"2024-04-20T21:24:59.864060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:44:29.007861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.005194+0000","last_change":"2024-04-20T21:24:58.861005+0000","last_active":"2024-04-20T21:25:00.005194+0000","last_peered":"2024-04-20T21:25:00.005194+0000","last_clean":"2024-04-20T21:25:00.005194+0000","last_became_active":"2024-04-20T21:24:58.860715+0000","last_became_peered":"2024-04-20T21:24:58.860715+0000","last_unstale":"2024-04-20T21:25:00.005194+0000","last_undegraded":"2024-04-20T21:25:00.005194+0000","last_fullsized":"2024-04-20T21:25:00.005194+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:30:58.449132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.862380+0000","last_change":"2024-04-20T21:24:58.874737+0000","last_active":"2024-04-20T21:24:59.862380+0000","last_peered":"2024-04-20T21:24:59.862380+0000","last_clean":"2024-04-20T21:24:59.862380+0000","last_became_active":"2024-04-20T21:24:58.874452+0000","last_became_peered":"2024-04-20T21:24:58.874452+0000","last_unstale":"2024-04-20T21:24:59.862380+0000","last_undegraded":"2024-04-20T21:24:59.862380+0000","last_fullsized":"2024-04-20T21:24:59.862380+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:36:50.375496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.006022+0000","last_change":"2024-04-20T21:24:58.871689+0000","last_active":"2024-04-20T21:25:00.006022+0000","last_peered":"2024-04-20T21:25:00.006022+0000","last_clean":"2024-04-20T21:25:00.006022+0000","last_became_active":"2024-04-20T21:24:58.871117+0000","last_became_peered":"2024-04-20T21:24:58.871117+0000","last_unstale":"2024-04-20T21:25:00.006022+0000","last_undegraded":"2024-04-20T21:25:00.006022+0000","last_fullsized":"2024-04-20T21:25:00.006022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:16:18.582014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.862144+0000","last_change":"2024-04-20T21:24:58.856734+0000","last_active":"2024-04-20T21:24:59.862144+0000","last_peered":"2024-04-20T21:24:59.862144+0000","last_clean":"2024-04-20T21:24:59.862144+0000","last_became_active":"2024-04-20T21:24:58.856019+0000","last_became_peered":"2024-04-20T21:24:58.856019+0000","last_unstale":"2024-04-20T21:24:59.862144+0000","last_undegraded":"2024-04-20T21:24:59.862144+0000","last_fullsized":"2024-04-20T21:24:59.862144+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:31:50.250214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.863853+0000","last_change":"2024-04-20T21:24:58.860915+0000","last_active":"2024-04-20T21:24:59.863853+0000","last_peered":"2024-04-20T21:24:59.863853+0000","last_clean":"2024-04-20T21:24:59.863853+0000","last_became_active":"2024-04-20T21:24:58.860555+0000","last_became_peered":"2024-04-20T21:24:58.860555+0000","last_unstale":"2024-04-20T21:24:59.863853+0000","last_undegraded":"2024-04-20T21:24:59.863853+0000","last_fullsized":"2024-04-20T21:24:59.863853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:22:35.562141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.861667+0000","last_change":"2024-04-20T21:24:58.872404+0000","last_active":"2024-04-20T21:24:59.861667+0000","last_peered":"2024-04-20T21:24:59.861667+0000","last_clean":"2024-04-20T21:24:59.861667+0000","last_became_active":"2024-04-20T21:24:58.872131+0000","last_became_peered":"2024-04-20T21:24:58.872131+0000","last_unstale":"2024-04-20T21:24:59.861667+0000","last_undegraded":"2024-04-20T21:24:59.861667+0000","last_fullsized":"2024-04-20T21:24:59.861667+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:16:05.557935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.862525+0000","last_change":"2024-04-20T21:24:58.870631+0000","last_active":"2024-04-20T21:24:59.862525+0000","last_peered":"2024-04-20T21:24:59.862525+0000","last_clean":"2024-04-20T21:24:59.862525+0000","last_became_active":"2024-04-20T21:24:58.870126+0000","last_became_peered":"2024-04-20T21:24:58.870126+0000","last_unstale":"2024-04-20T21:24:59.862525+0000","last_undegraded":"2024-04-20T21:24:59.862525+0000","last_fullsized":"2024-04-20T21:24:59.862525+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:00:15.798523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.861985+0000","last_change":"2024-04-20T21:24:58.857710+0000","last_active":"2024-04-20T21:24:59.861985+0000","last_peered":"2024-04-20T21:24:59.861985+0000","last_clean":"2024-04-20T21:24:59.861985+0000","last_became_active":"2024-04-20T21:24:58.857392+0000","last_became_peered":"2024-04-20T21:24:58.857392+0000","last_unstale":"2024-04-20T21:24:59.861985+0000","last_undegraded":"2024-04-20T21:24:59.861985+0000","last_fullsized":"2024-04-20T21:24:59.861985+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:15:35.117140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.863447+0000","last_change":"2024-04-20T21:24:58.877663+0000","last_active":"2024-04-20T21:24:59.863447+0000","last_peered":"2024-04-20T21:24:59.863447+0000","last_clean":"2024-04-20T21:24:59.863447+0000","last_became_active":"2024-04-20T21:24:58.877232+0000","last_became_peered":"2024-04-20T21:24:58.877232+0000","last_unstale":"2024-04-20T21:24:59.863447+0000","last_undegraded":"2024-04-20T21:24:59.863447+0000","last_fullsized":"2024-04-20T21:24:59.863447+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:31:15.019045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.862179+0000","last_change":"2024-04-20T21:24:58.861604+0000","last_active":"2024-04-20T21:24:59.862179+0000","last_peered":"2024-04-20T21:24:59.862179+0000","last_clean":"2024-04-20T21:24:59.862179+0000","last_became_active":"2024-04-20T21:24:58.861309+0000","last_became_peered":"2024-04-20T21:24:58.861309+0000","last_unstale":"2024-04-20T21:24:59.862179+0000","last_undegraded":"2024-04-20T21:24:59.862179+0000","last_fullsized":"2024-04-20T21:24:59.862179+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:03:04.971095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"84'4","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.864438+0000","last_change":"2024-04-20T21:24:57.850348+0000","last_active":"2024-04-20T21:24:59.864438+0000","last_peered":"2024-04-20T21:24:59.864438+0000","last_clean":"2024-04-20T21:24:59.864438+0000","last_became_active":"2024-04-20T21:24:57.849983+0000","last_became_peered":"2024-04-20T21:24:57.849983+0000","last_unstale":"2024-04-20T21:24:59.864438+0000","last_undegraded":"2024-04-20T21:24:59.864438+0000","last_fullsized":"2024-04-20T21:24:59.864438+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:19:17.769565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.863975+0000","last_change":"2024-04-20T21:24:58.860907+0000","last_active":"2024-04-20T21:24:59.863975+0000","last_peered":"2024-04-20T21:24:59.863975+0000","last_clean":"2024-04-20T21:24:59.863975+0000","last_became_active":"2024-04-20T21:24:58.860609+0000","last_became_peered":"2024-04-20T21:24:58.860609+0000","last_unstale":"2024-04-20T21:24:59.863975+0000","last_undegraded":"2024-04-20T21:24:59.863975+0000","last_fullsized":"2024-04-20T21:24:59.863975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:59:29.884590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"84'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.863380+0000","last_change":"2024-04-20T21:24:57.841870+0000","last_active":"2024-04-20T21:24:59.863380+0000","last_peered":"2024-04-20T21:24:59.863380+0000","last_clean":"2024-04-20T21:24:59.863380+0000","last_became_active":"2024-04-20T21:24:57.841518+0000","last_became_peered":"2024-04-20T21:24:57.841518+0000","last_unstale":"2024-04-20T21:24:59.863380+0000","last_undegraded":"2024-04-20T21:24:59.863380+0000","last_fullsized":"2024-04-20T21:24:59.863380+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:57:13.677134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.864013+0000","last_change":"2024-04-20T21:24:58.867722+0000","last_active":"2024-04-20T21:24:59.864013+0000","last_peered":"2024-04-20T21:24:59.864013+0000","last_clean":"2024-04-20T21:24:59.864013+0000","last_became_active":"2024-04-20T21:24:58.867429+0000","last_became_peered":"2024-04-20T21:24:58.867429+0000","last_unstale":"2024-04-20T21:24:59.864013+0000","last_undegraded":"2024-04-20T21:24:59.864013+0000","last_fullsized":"2024-04-20T21:24:59.864013+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:03:09.674975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"84'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.008780+0000","last_change":"2024-04-20T21:24:57.841558+0000","last_active":"2024-04-20T21:25:00.008780+0000","last_peered":"2024-04-20T21:25:00.008780+0000","last_clean":"2024-04-20T21:25:00.008780+0000","last_became_active":"2024-04-20T21:24:57.841165+0000","last_became_peered":"2024-04-20T21:24:57.841165+0000","last_unstale":"2024-04-20T21:25:00.008780+0000","last_undegraded":"2024-04-20T21:25:00.008780+0000","last_fullsized":"2024-04-20T21:25:00.008780+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:15:17.579183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.006076+0000","last_change":"2024-04-20T21:24:58.868769+0000","last_active":"2024-04-20T21:25:00.006076+0000","last_peered":"2024-04-20T21:25:00.006076+0000","last_clean":"2024-04-20T21:25:00.006076+0000","last_became_active":"2024-04-20T21:24:58.868460+0000","last_became_peered":"2024-04-20T21:24:58.868460+0000","last_unstale":"2024-04-20T21:25:00.006076+0000","last_undegraded":"2024-04-20T21:25:00.006076+0000","last_fullsized":"2024-04-20T21:25:00.006076+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:16:20.655880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"84'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.008125+0000","last_change":"2024-04-20T21:24:57.846244+0000","last_active":"2024-04-20T21:25:00.008125+0000","last_peered":"2024-04-20T21:25:00.008125+0000","last_clean":"2024-04-20T21:25:00.008125+0000","last_became_active":"2024-04-20T21:24:57.845970+0000","last_became_peered":"2024-04-20T21:24:57.845970+0000","last_unstale":"2024-04-20T21:25:00.008125+0000","last_undegraded":"2024-04-20T21:25:00.008125+0000","last_fullsized":"2024-04-20T21:25:00.008125+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:21:08.607476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.862614+0000","last_change":"2024-04-20T21:24:58.868267+0000","last_active":"2024-04-20T21:24:59.862614+0000","last_peered":"2024-04-20T21:24:59.862614+0000","last_clean":"2024-04-20T21:24:59.862614+0000","last_became_active":"2024-04-20T21:24:58.867985+0000","last_became_peered":"2024-04-20T21:24:58.867985+0000","last_unstale":"2024-04-20T21:24:59.862614+0000","last_undegraded":"2024-04-20T21:24:59.862614+0000","last_fullsized":"2024-04-20T21:24:59.862614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:46:53.063475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"84'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.861952+0000","last_change":"2024-04-20T21:24:57.842637+0000","last_active":"2024-04-20T21:24:59.861952+0000","last_peered":"2024-04-20T21:24:59.861952+0000","last_clean":"2024-04-20T21:24:59.861952+0000","last_became_active":"2024-04-20T21:24:57.842309+0000","last_became_peered":"2024-04-20T21:24:57.842309+0000","last_unstale":"2024-04-20T21:24:59.861952+0000","last_undegraded":"2024-04-20T21:24:59.861952+0000","last_fullsized":"2024-04-20T21:24:59.861952+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:31:14.004194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.861835+0000","last_change":"2024-04-20T21:24:58.862889+0000","last_active":"2024-04-20T21:24:59.861835+0000","last_peered":"2024-04-20T21:24:59.861835+0000","last_clean":"2024-04-20T21:24:59.861835+0000","last_became_active":"2024-04-20T21:24:58.862433+0000","last_became_peered":"2024-04-20T21:24:58.862433+0000","last_unstale":"2024-04-20T21:24:59.861835+0000","last_undegraded":"2024-04-20T21:24:59.861835+0000","last_fullsized":"2024-04-20T21:24:59.861835+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:14:07.253016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"84'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.861807+0000","last_change":"2024-04-20T21:24:57.841922+0000","last_active":"2024-04-20T21:24:59.861807+0000","last_peered":"2024-04-20T21:24:59.861807+0000","last_clean":"2024-04-20T21:24:59.861807+0000","last_became_active":"2024-04-20T21:24:57.841637+0000","last_became_peered":"2024-04-20T21:24:57.841637+0000","last_unstale":"2024-04-20T21:24:59.861807+0000","last_undegraded":"2024-04-20T21:24:59.861807+0000","last_fullsized":"2024-04-20T21:24:59.861807+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:46:05.086336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.006234+0000","last_change":"2024-04-20T21:24:58.866862+0000","last_active":"2024-04-20T21:25:00.006234+0000","last_peered":"2024-04-20T21:25:00.006234+0000","last_clean":"2024-04-20T21:25:00.006234+0000","last_became_active":"2024-04-20T21:24:58.866504+0000","last_became_peered":"2024-04-20T21:24:58.866504+0000","last_unstale":"2024-04-20T21:25:00.006234+0000","last_undegraded":"2024-04-20T21:25:00.006234+0000","last_fullsized":"2024-04-20T21:25:00.006234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:38:19.387593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.005975+0000","last_change":"2024-04-20T21:24:57.849101+0000","last_active":"2024-04-20T21:25:00.005975+0000","last_peered":"2024-04-20T21:25:00.005975+0000","last_clean":"2024-04-20T21:25:00.005975+0000","last_became_active":"2024-04-20T21:24:57.848676+0000","last_became_peered":"2024-04-20T21:24:57.848676+0000","last_unstale":"2024-04-20T21:25:00.005975+0000","last_undegraded":"2024-04-20T21:25:00.005975+0000","last_fullsized":"2024-04-20T21:25:00.005975+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:03:08.660125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.005866+0000","last_change":"2024-04-20T21:24:58.860404+0000","last_active":"2024-04-20T21:25:00.005866+0000","last_peered":"2024-04-20T21:25:00.005866+0000","last_clean":"2024-04-20T21:25:00.005866+0000","last_became_active":"2024-04-20T21:24:58.860075+0000","last_became_peered":"2024-04-20T21:24:58.860075+0000","last_unstale":"2024-04-20T21:25:00.005866+0000","last_undegraded":"2024-04-20T21:25:00.005866+0000","last_fullsized":"2024-04-20T21:25:00.005866+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:16:18.582014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.861486+0000","last_change":"2024-04-20T21:24:57.848064+0000","last_active":"2024-04-20T21:24:59.861486+0000","last_peered":"2024-04-20T21:24:59.861486+0000","last_clean":"2024-04-20T21:24:59.861486+0000","last_became_active":"2024-04-20T21:24:57.847202+0000","last_became_peered":"2024-04-20T21:24:57.847202+0000","last_unstale":"2024-04-20T21:24:59.861486+0000","last_undegraded":"2024-04-20T21:24:59.861486+0000","last_fullsized":"2024-04-20T21:24:59.861486+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:36:30.447062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.006664+0000","last_change":"2024-04-20T21:24:58.875155+0000","last_active":"2024-04-20T21:25:00.006664+0000","last_peered":"2024-04-20T21:25:00.006664+0000","last_clean":"2024-04-20T21:25:00.006664+0000","last_became_active":"2024-04-20T21:24:58.872257+0000","last_became_peered":"2024-04-20T21:24:58.872257+0000","last_unstale":"2024-04-20T21:25:00.006664+0000","last_undegraded":"2024-04-20T21:25:00.006664+0000","last_fullsized":"2024-04-20T21:25:00.006664+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:16:20.655880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.864639+0000","last_change":"2024-04-20T21:24:57.851985+0000","last_active":"2024-04-20T21:24:59.864639+0000","last_peered":"2024-04-20T21:24:59.864639+0000","last_clean":"2024-04-20T21:24:59.864639+0000","last_became_active":"2024-04-20T21:24:57.851650+0000","last_became_peered":"2024-04-20T21:24:57.851650+0000","last_unstale":"2024-04-20T21:24:59.864639+0000","last_undegraded":"2024-04-20T21:24:59.864639+0000","last_fullsized":"2024-04-20T21:24:59.864639+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:46:52.048624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.005971+0000","last_change":"2024-04-20T21:24:58.867812+0000","last_active":"2024-04-20T21:25:00.005971+0000","last_peered":"2024-04-20T21:25:00.005971+0000","last_clean":"2024-04-20T21:25:00.005971+0000","last_became_active":"2024-04-20T21:24:58.865494+0000","last_became_peered":"2024-04-20T21:24:58.865494+0000","last_unstale":"2024-04-20T21:25:00.005971+0000","last_undegraded":"2024-04-20T21:25:00.005971+0000","last_fullsized":"2024-04-20T21:25:00.005971+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:25:56.141401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.864495+0000","last_change":"2024-04-20T21:24:57.848638+0000","last_active":"2024-04-20T21:24:59.864495+0000","last_peered":"2024-04-20T21:24:59.864495+0000","last_clean":"2024-04-20T21:24:59.864495+0000","last_became_active":"2024-04-20T21:24:57.848224+0000","last_became_peered":"2024-04-20T21:24:57.848224+0000","last_unstale":"2024-04-20T21:24:59.864495+0000","last_undegraded":"2024-04-20T21:24:59.864495+0000","last_fullsized":"2024-04-20T21:24:59.864495+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:30:50.898747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.005746+0000","last_change":"2024-04-20T21:24:58.867490+0000","last_active":"2024-04-20T21:25:00.005746+0000","last_peered":"2024-04-20T21:25:00.005746+0000","last_clean":"2024-04-20T21:25:00.005746+0000","last_became_active":"2024-04-20T21:24:58.867221+0000","last_became_peered":"2024-04-20T21:24:58.867221+0000","last_unstale":"2024-04-20T21:25:00.005746+0000","last_undegraded":"2024-04-20T21:25:00.005746+0000","last_fullsized":"2024-04-20T21:25:00.005746+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:31:50.250214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"84'3","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.863552+0000","last_change":"2024-04-20T21:24:57.851827+0000","last_active":"2024-04-20T21:24:59.863552+0000","last_peered":"2024-04-20T21:24:59.863552+0000","last_clean":"2024-04-20T21:24:59.863552+0000","last_became_active":"2024-04-20T21:24:57.851559+0000","last_became_peered":"2024-04-20T21:24:57.851559+0000","last_unstale":"2024-04-20T21:24:59.863552+0000","last_undegraded":"2024-04-20T21:24:59.863552+0000","last_fullsized":"2024-04-20T21:24:59.863552+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:36:49.360645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.862532+0000","last_change":"2024-04-20T21:24:58.856834+0000","last_active":"2024-04-20T21:24:59.862532+0000","last_peered":"2024-04-20T21:24:59.862532+0000","last_clean":"2024-04-20T21:24:59.862532+0000","last_became_active":"2024-04-20T21:24:58.856495+0000","last_became_peered":"2024-04-20T21:24:58.856495+0000","last_unstale":"2024-04-20T21:24:59.862532+0000","last_undegraded":"2024-04-20T21:24:59.862532+0000","last_fullsized":"2024-04-20T21:24:59.862532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:16:18.582014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"84'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.005878+0000","last_change":"2024-04-20T21:24:57.845006+0000","last_active":"2024-04-20T21:25:00.005878+0000","last_peered":"2024-04-20T21:25:00.005878+0000","last_clean":"2024-04-20T21:25:00.005878+0000","last_became_active":"2024-04-20T21:24:57.844696+0000","last_became_peered":"2024-04-20T21:24:57.844696+0000","last_unstale":"2024-04-20T21:25:00.005878+0000","last_undegraded":"2024-04-20T21:25:00.005878+0000","last_fullsized":"2024-04-20T21:25:00.005878+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:02:23.398563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.005506+0000","last_change":"2024-04-20T21:24:58.861457+0000","last_active":"2024-04-20T21:25:00.005506+0000","last_peered":"2024-04-20T21:25:00.005506+0000","last_clean":"2024-04-20T21:25:00.005506+0000","last_became_active":"2024-04-20T21:24:58.861146+0000","last_became_peered":"2024-04-20T21:24:58.861146+0000","last_unstale":"2024-04-20T21:25:00.005506+0000","last_undegraded":"2024-04-20T21:25:00.005506+0000","last_fullsized":"2024-04-20T21:25:00.005506+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:15:35.117140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"84'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.862863+0000","last_change":"2024-04-20T21:24:57.840595+0000","last_active":"2024-04-20T21:24:59.862863+0000","last_peered":"2024-04-20T21:24:59.862863+0000","last_clean":"2024-04-20T21:24:59.862863+0000","last_became_active":"2024-04-20T21:24:57.840331+0000","last_became_peered":"2024-04-20T21:24:57.840331+0000","last_unstale":"2024-04-20T21:24:59.862863+0000","last_undegraded":"2024-04-20T21:24:59.862863+0000","last_fullsized":"2024-04-20T21:24:59.862863+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:16:19.940923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.005532+0000","last_change":"2024-04-20T21:24:58.881303+0000","last_active":"2024-04-20T21:25:00.005532+0000","last_peered":"2024-04-20T21:25:00.005532+0000","last_clean":"2024-04-20T21:25:00.005532+0000","last_became_active":"2024-04-20T21:24:58.880969+0000","last_became_peered":"2024-04-20T21:24:58.880969+0000","last_unstale":"2024-04-20T21:25:00.005532+0000","last_undegraded":"2024-04-20T21:25:00.005532+0000","last_fullsized":"2024-04-20T21:25:00.005532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:43:31.020198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.006688+0000","last_change":"2024-04-20T21:24:57.831315+0000","last_active":"2024-04-20T21:25:00.006688+0000","last_peered":"2024-04-20T21:25:00.006688+0000","last_clean":"2024-04-20T21:25:00.006688+0000","last_became_active":"2024-04-20T21:24:57.830496+0000","last_became_peered":"2024-04-20T21:24:57.830496+0000","last_unstale":"2024-04-20T21:25:00.006688+0000","last_undegraded":"2024-04-20T21:25:00.006688+0000","last_fullsized":"2024-04-20T21:25:00.006688+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:15:34.102290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.861950+0000","last_change":"2024-04-20T21:24:58.876953+0000","last_active":"2024-04-20T21:24:59.861950+0000","last_peered":"2024-04-20T21:24:59.861950+0000","last_clean":"2024-04-20T21:24:59.861950+0000","last_became_active":"2024-04-20T21:24:58.872812+0000","last_became_peered":"2024-04-20T21:24:58.872812+0000","last_unstale":"2024-04-20T21:24:59.861950+0000","last_undegraded":"2024-04-20T21:24:59.861950+0000","last_fullsized":"2024-04-20T21:24:59.861950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:19:06.776492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.004964+0000","last_change":"2024-04-20T21:24:57.846425+0000","last_active":"2024-04-20T21:25:00.004964+0000","last_peered":"2024-04-20T21:25:00.004964+0000","last_clean":"2024-04-20T21:25:00.004964+0000","last_became_active":"2024-04-20T21:24:57.846119+0000","last_became_peered":"2024-04-20T21:24:57.846119+0000","last_unstale":"2024-04-20T21:25:00.004964+0000","last_undegraded":"2024-04-20T21:25:00.004964+0000","last_fullsized":"2024-04-20T21:25:00.004964+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:38:19.894629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.861427+0000","last_change":"2024-04-20T21:24:58.868039+0000","last_active":"2024-04-20T21:24:59.861427+0000","last_peered":"2024-04-20T21:24:59.861427+0000","last_clean":"2024-04-20T21:24:59.861427+0000","last_became_active":"2024-04-20T21:24:58.864180+0000","last_became_peered":"2024-04-20T21:24:58.864180+0000","last_unstale":"2024-04-20T21:24:59.861427+0000","last_undegraded":"2024-04-20T21:24:59.861427+0000","last_fullsized":"2024-04-20T21:24:59.861427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:46:37.542502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"84'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.006195+0000","last_change":"2024-04-20T21:24:57.845058+0000","last_active":"2024-04-20T21:25:00.006195+0000","last_peered":"2024-04-20T21:25:00.006195+0000","last_clean":"2024-04-20T21:25:00.006195+0000","last_became_active":"2024-04-20T21:24:57.844088+0000","last_became_peered":"2024-04-20T21:24:57.844088+0000","last_unstale":"2024-04-20T21:25:00.006195+0000","last_undegraded":"2024-04-20T21:25:00.006195+0000","last_fullsized":"2024-04-20T21:25:00.006195+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:59:28.869739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.006228+0000","last_change":"2024-04-20T21:24:58.869796+0000","last_active":"2024-04-20T21:25:00.006228+0000","last_peered":"2024-04-20T21:25:00.006228+0000","last_clean":"2024-04-20T21:25:00.006228+0000","last_became_active":"2024-04-20T21:24:58.869046+0000","last_became_peered":"2024-04-20T21:24:58.869046+0000","last_unstale":"2024-04-20T21:25:00.006228+0000","last_undegraded":"2024-04-20T21:25:00.006228+0000","last_fullsized":"2024-04-20T21:25:00.006228+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:10:48.245778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"84'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.864681+0000","last_change":"2024-04-20T21:24:57.848118+0000","last_active":"2024-04-20T21:24:59.864681+0000","last_peered":"2024-04-20T21:24:59.864681+0000","last_clean":"2024-04-20T21:24:59.864681+0000","last_became_active":"2024-04-20T21:24:57.847626+0000","last_became_peered":"2024-04-20T21:24:57.847626+0000","last_unstale":"2024-04-20T21:24:59.864681+0000","last_undegraded":"2024-04-20T21:24:59.864681+0000","last_fullsized":"2024-04-20T21:24:59.864681+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:13:54.508013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.864142+0000","last_change":"2024-04-20T21:24:58.868908+0000","last_active":"2024-04-20T21:24:59.864142+0000","last_peered":"2024-04-20T21:24:59.864142+0000","last_clean":"2024-04-20T21:24:59.864142+0000","last_became_active":"2024-04-20T21:24:58.868533+0000","last_became_peered":"2024-04-20T21:24:58.868533+0000","last_unstale":"2024-04-20T21:24:59.864142+0000","last_undegraded":"2024-04-20T21:24:59.864142+0000","last_fullsized":"2024-04-20T21:24:59.864142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:21:09.622327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"84'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.009304+0000","last_change":"2024-04-20T21:24:57.849023+0000","last_active":"2024-04-20T21:25:00.009304+0000","last_peered":"2024-04-20T21:25:00.009304+0000","last_clean":"2024-04-20T21:25:00.009304+0000","last_became_active":"2024-04-20T21:24:57.848617+0000","last_became_peered":"2024-04-20T21:24:57.848617+0000","last_unstale":"2024-04-20T21:25:00.009304+0000","last_undegraded":"2024-04-20T21:25:00.009304+0000","last_fullsized":"2024-04-20T21:25:00.009304+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:54:16.413274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.862307+0000","last_change":"2024-04-20T21:24:58.869280+0000","last_active":"2024-04-20T21:24:59.862307+0000","last_peered":"2024-04-20T21:24:59.862307+0000","last_clean":"2024-04-20T21:24:59.862307+0000","last_became_active":"2024-04-20T21:24:58.868854+0000","last_became_peered":"2024-04-20T21:24:58.868854+0000","last_unstale":"2024-04-20T21:24:59.862307+0000","last_undegraded":"2024-04-20T21:24:59.862307+0000","last_fullsized":"2024-04-20T21:24:59.862307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:43:31.020198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.008264+0000","last_change":"2024-04-20T21:24:57.848333+0000","last_active":"2024-04-20T21:25:00.008264+0000","last_peered":"2024-04-20T21:25:00.008264+0000","last_clean":"2024-04-20T21:25:00.008264+0000","last_became_active":"2024-04-20T21:24:57.845753+0000","last_became_peered":"2024-04-20T21:24:57.845753+0000","last_unstale":"2024-04-20T21:25:00.008264+0000","last_undegraded":"2024-04-20T21:25:00.008264+0000","last_fullsized":"2024-04-20T21:25:00.008264+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:07:52.687802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.007734+0000","last_change":"2024-04-20T21:24:58.872593+0000","last_active":"2024-04-20T21:25:00.007734+0000","last_peered":"2024-04-20T21:25:00.007734+0000","last_clean":"2024-04-20T21:25:00.007734+0000","last_became_active":"2024-04-20T21:24:58.872039+0000","last_became_peered":"2024-04-20T21:24:58.872039+0000","last_unstale":"2024-04-20T21:25:00.007734+0000","last_undegraded":"2024-04-20T21:25:00.007734+0000","last_fullsized":"2024-04-20T21:25:00.007734+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:54:59.083007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"84'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.007086+0000","last_change":"2024-04-20T21:24:57.834647+0000","last_active":"2024-04-20T21:25:00.007086+0000","last_peered":"2024-04-20T21:25:00.007086+0000","last_clean":"2024-04-20T21:25:00.007086+0000","last_became_active":"2024-04-20T21:24:57.834334+0000","last_became_peered":"2024-04-20T21:24:57.834334+0000","last_unstale":"2024-04-20T21:25:00.007086+0000","last_undegraded":"2024-04-20T21:25:00.007086+0000","last_fullsized":"2024-04-20T21:25:00.007086+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:14:06.238165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.005587+0000","last_change":"2024-04-20T21:24:58.866476+0000","last_active":"2024-04-20T21:25:00.005587+0000","last_peered":"2024-04-20T21:25:00.005587+0000","last_clean":"2024-04-20T21:25:00.005587+0000","last_became_active":"2024-04-20T21:24:58.866185+0000","last_became_peered":"2024-04-20T21:24:58.866185+0000","last_unstale":"2024-04-20T21:25:00.005587+0000","last_undegraded":"2024-04-20T21:25:00.005587+0000","last_fullsized":"2024-04-20T21:25:00.005587+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:53:39.858430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.007814+0000","last_change":"2024-04-20T21:24:57.843053+0000","last_active":"2024-04-20T21:25:00.007814+0000","last_peered":"2024-04-20T21:25:00.007814+0000","last_clean":"2024-04-20T21:25:00.007814+0000","last_became_active":"2024-04-20T21:24:57.842067+0000","last_became_peered":"2024-04-20T21:24:57.842067+0000","last_unstale":"2024-04-20T21:25:00.007814+0000","last_undegraded":"2024-04-20T21:25:00.007814+0000","last_fullsized":"2024-04-20T21:25:00.007814+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:57:50.784411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.007147+0000","last_change":"2024-04-20T21:24:58.865864+0000","last_active":"2024-04-20T21:25:00.007147+0000","last_peered":"2024-04-20T21:25:00.007147+0000","last_clean":"2024-04-20T21:25:00.007147+0000","last_became_active":"2024-04-20T21:24:58.865594+0000","last_became_peered":"2024-04-20T21:24:58.865594+0000","last_unstale":"2024-04-20T21:25:00.007147+0000","last_undegraded":"2024-04-20T21:25:00.007147+0000","last_fullsized":"2024-04-20T21:25:00.007147+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:42:51.118239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.005024+0000","last_change":"2024-04-20T21:24:57.848088+0000","last_active":"2024-04-20T21:25:00.005024+0000","last_peered":"2024-04-20T21:25:00.005024+0000","last_clean":"2024-04-20T21:25:00.005024+0000","last_became_active":"2024-04-20T21:24:57.847784+0000","last_became_peered":"2024-04-20T21:24:57.847784+0000","last_unstale":"2024-04-20T21:25:00.005024+0000","last_undegraded":"2024-04-20T21:25:00.005024+0000","last_fullsized":"2024-04-20T21:25:00.005024+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:44:21.335046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.004882+0000","last_change":"2024-04-20T21:24:58.874494+0000","last_active":"2024-04-20T21:25:00.004882+0000","last_peered":"2024-04-20T21:25:00.004882+0000","last_clean":"2024-04-20T21:25:00.004882+0000","last_became_active":"2024-04-20T21:24:58.874218+0000","last_became_peered":"2024-04-20T21:24:58.874218+0000","last_unstale":"2024-04-20T21:25:00.004882+0000","last_undegraded":"2024-04-20T21:25:00.004882+0000","last_fullsized":"2024-04-20T21:25:00.004882+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:15:18.594034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.004308+0000","last_change":"2024-04-20T21:24:57.836748+0000","last_active":"2024-04-20T21:25:00.004308+0000","last_peered":"2024-04-20T21:25:00.004308+0000","last_clean":"2024-04-20T21:25:00.004308+0000","last_became_active":"2024-04-20T21:24:57.836442+0000","last_became_peered":"2024-04-20T21:24:57.836442+0000","last_unstale":"2024-04-20T21:25:00.004308+0000","last_undegraded":"2024-04-20T21:25:00.004308+0000","last_fullsized":"2024-04-20T21:25:00.004308+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:25:55.126550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.862598+0000","last_change":"2024-04-20T21:24:58.868235+0000","last_active":"2024-04-20T21:24:59.862598+0000","last_peered":"2024-04-20T21:24:59.862598+0000","last_clean":"2024-04-20T21:24:59.862598+0000","last_became_active":"2024-04-20T21:24:58.867797+0000","last_became_peered":"2024-04-20T21:24:58.867797+0000","last_unstale":"2024-04-20T21:24:59.862598+0000","last_undegraded":"2024-04-20T21:24:59.862598+0000","last_fullsized":"2024-04-20T21:24:59.862598+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:16:20.655880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"84'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.862495+0000","last_change":"2024-04-20T21:24:57.839629+0000","last_active":"2024-04-20T21:24:59.862495+0000","last_peered":"2024-04-20T21:24:59.862495+0000","last_clean":"2024-04-20T21:24:59.862495+0000","last_became_active":"2024-04-20T21:24:57.839237+0000","last_became_peered":"2024-04-20T21:24:57.839237+0000","last_unstale":"2024-04-20T21:24:59.862495+0000","last_undegraded":"2024-04-20T21:24:59.862495+0000","last_fullsized":"2024-04-20T21:24:59.862495+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:01:25.431255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.005553+0000","last_change":"2024-04-20T21:24:58.873694+0000","last_active":"2024-04-20T21:25:00.005553+0000","last_peered":"2024-04-20T21:25:00.005553+0000","last_clean":"2024-04-20T21:25:00.005553+0000","last_became_active":"2024-04-20T21:24:58.873385+0000","last_became_peered":"2024-04-20T21:24:58.873385+0000","last_unstale":"2024-04-20T21:25:00.005553+0000","last_undegraded":"2024-04-20T21:25:00.005553+0000","last_fullsized":"2024-04-20T21:25:00.005553+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:55:37.672502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.862205+0000","last_change":"2024-04-20T21:24:57.846774+0000","last_active":"2024-04-20T21:24:59.862205+0000","last_peered":"2024-04-20T21:24:59.862205+0000","last_clean":"2024-04-20T21:24:59.862205+0000","last_became_active":"2024-04-20T21:24:57.846445+0000","last_became_peered":"2024-04-20T21:24:57.846445+0000","last_unstale":"2024-04-20T21:24:59.862205+0000","last_undegraded":"2024-04-20T21:24:59.862205+0000","last_fullsized":"2024-04-20T21:24:59.862205+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:33:43.984573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.864169+0000","last_change":"2024-04-20T21:24:58.863528+0000","last_active":"2024-04-20T21:24:59.864169+0000","last_peered":"2024-04-20T21:24:59.864169+0000","last_clean":"2024-04-20T21:24:59.864169+0000","last_became_active":"2024-04-20T21:24:58.863162+0000","last_became_peered":"2024-04-20T21:24:58.863162+0000","last_unstale":"2024-04-20T21:24:59.864169+0000","last_undegraded":"2024-04-20T21:24:59.864169+0000","last_fullsized":"2024-04-20T21:24:59.864169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:54:17.428125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.863816+0000","last_change":"2024-04-20T21:24:57.848202+0000","last_active":"2024-04-20T21:24:59.863816+0000","last_peered":"2024-04-20T21:24:59.863816+0000","last_clean":"2024-04-20T21:24:59.863816+0000","last_became_active":"2024-04-20T21:24:57.847601+0000","last_became_peered":"2024-04-20T21:24:57.847601+0000","last_unstale":"2024-04-20T21:24:59.863816+0000","last_undegraded":"2024-04-20T21:24:59.863816+0000","last_fullsized":"2024-04-20T21:24:59.863816+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:00:14.783672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.862427+0000","last_change":"2024-04-20T21:24:58.862205+0000","last_active":"2024-04-20T21:24:59.862427+0000","last_peered":"2024-04-20T21:24:59.862427+0000","last_clean":"2024-04-20T21:24:59.862427+0000","last_became_active":"2024-04-20T21:24:58.861501+0000","last_became_peered":"2024-04-20T21:24:58.861501+0000","last_unstale":"2024-04-20T21:24:59.862427+0000","last_undegraded":"2024-04-20T21:24:59.862427+0000","last_fullsized":"2024-04-20T21:24:59.862427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:55:37.672502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.864357+0000","last_change":"2024-04-20T21:24:57.847038+0000","last_active":"2024-04-20T21:24:59.864357+0000","last_peered":"2024-04-20T21:24:59.864357+0000","last_clean":"2024-04-20T21:24:59.864357+0000","last_became_active":"2024-04-20T21:24:57.846453+0000","last_became_peered":"2024-04-20T21:24:57.846453+0000","last_unstale":"2024-04-20T21:24:59.864357+0000","last_undegraded":"2024-04-20T21:24:59.864357+0000","last_fullsized":"2024-04-20T21:24:59.864357+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:03:18.054330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.004982+0000","last_change":"2024-04-20T21:24:58.864569+0000","last_active":"2024-04-20T21:25:00.004982+0000","last_peered":"2024-04-20T21:25:00.004982+0000","last_clean":"2024-04-20T21:25:00.004982+0000","last_became_active":"2024-04-20T21:24:58.864224+0000","last_became_peered":"2024-04-20T21:24:58.864224+0000","last_unstale":"2024-04-20T21:25:00.004982+0000","last_undegraded":"2024-04-20T21:25:00.004982+0000","last_fullsized":"2024-04-20T21:25:00.004982+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:50:38.147663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.006057+0000","last_change":"2024-04-20T21:24:57.843606+0000","last_active":"2024-04-20T21:25:00.006057+0000","last_peered":"2024-04-20T21:25:00.006057+0000","last_clean":"2024-04-20T21:25:00.006057+0000","last_became_active":"2024-04-20T21:24:57.842392+0000","last_became_peered":"2024-04-20T21:24:57.842392+0000","last_unstale":"2024-04-20T21:25:00.006057+0000","last_undegraded":"2024-04-20T21:25:00.006057+0000","last_fullsized":"2024-04-20T21:25:00.006057+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:44:27.993010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.005844+0000","last_change":"2024-04-20T21:24:58.866969+0000","last_active":"2024-04-20T21:25:00.005844+0000","last_peered":"2024-04-20T21:25:00.005844+0000","last_clean":"2024-04-20T21:25:00.005844+0000","last_became_active":"2024-04-20T21:24:58.864164+0000","last_became_peered":"2024-04-20T21:24:58.864164+0000","last_unstale":"2024-04-20T21:25:00.005844+0000","last_undegraded":"2024-04-20T21:25:00.005844+0000","last_fullsized":"2024-04-20T21:25:00.005844+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:03:04.971095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.861933+0000","last_change":"2024-04-20T21:24:57.841023+0000","last_active":"2024-04-20T21:24:59.861933+0000","last_peered":"2024-04-20T21:24:59.861933+0000","last_clean":"2024-04-20T21:24:59.861933+0000","last_became_active":"2024-04-20T21:24:57.840669+0000","last_became_peered":"2024-04-20T21:24:57.840669+0000","last_unstale":"2024-04-20T21:24:59.861933+0000","last_undegraded":"2024-04-20T21:24:59.861933+0000","last_fullsized":"2024-04-20T21:24:59.861933+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:02:23.398563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.006509+0000","last_change":"2024-04-20T21:24:58.878246+0000","last_active":"2024-04-20T21:25:00.006509+0000","last_peered":"2024-04-20T21:25:00.006509+0000","last_clean":"2024-04-20T21:25:00.006509+0000","last_became_active":"2024-04-20T21:24:58.877900+0000","last_became_peered":"2024-04-20T21:24:58.877900+0000","last_unstale":"2024-04-20T21:25:00.006509+0000","last_undegraded":"2024-04-20T21:25:00.006509+0000","last_fullsized":"2024-04-20T21:25:00.006509+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:19:11.569824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.864426+0000","last_change":"2024-04-20T21:24:57.850383+0000","last_active":"2024-04-20T21:24:59.864426+0000","last_peered":"2024-04-20T21:24:59.864426+0000","last_clean":"2024-04-20T21:24:59.864426+0000","last_became_active":"2024-04-20T21:24:57.850069+0000","last_became_peered":"2024-04-20T21:24:57.850069+0000","last_unstale":"2024-04-20T21:24:59.864426+0000","last_undegraded":"2024-04-20T21:24:59.864426+0000","last_fullsized":"2024-04-20T21:24:59.864426+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:21:17.509431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.864304+0000","last_change":"2024-04-20T21:24:58.862654+0000","last_active":"2024-04-20T21:24:59.864304+0000","last_peered":"2024-04-20T21:24:59.864304+0000","last_clean":"2024-04-20T21:24:59.864304+0000","last_became_active":"2024-04-20T21:24:58.862360+0000","last_became_peered":"2024-04-20T21:24:58.862360+0000","last_unstale":"2024-04-20T21:24:59.864304+0000","last_undegraded":"2024-04-20T21:24:59.864304+0000","last_fullsized":"2024-04-20T21:24:59.864304+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:07:53.702653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.861459+0000","last_change":"2024-04-20T21:24:57.850740+0000","last_active":"2024-04-20T21:24:59.861459+0000","last_peered":"2024-04-20T21:24:59.861459+0000","last_clean":"2024-04-20T21:24:59.861459+0000","last_became_active":"2024-04-20T21:24:57.850376+0000","last_became_peered":"2024-04-20T21:24:57.850376+0000","last_unstale":"2024-04-20T21:24:59.861459+0000","last_undegraded":"2024-04-20T21:24:59.861459+0000","last_fullsized":"2024-04-20T21:24:59.861459+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:38:19.894629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:24:59.862534+0000","last_change":"2024-04-20T21:24:58.867774+0000","last_active":"2024-04-20T21:24:59.862534+0000","last_peered":"2024-04-20T21:24:59.862534+0000","last_clean":"2024-04-20T21:24:59.862534+0000","last_became_active":"2024-04-20T21:24:58.867089+0000","last_became_peered":"2024-04-20T21:24:58.867089+0000","last_unstale":"2024-04-20T21:24:59.862534+0000","last_undegraded":"2024-04-20T21:24:59.862534+0000","last_fullsized":"2024-04-20T21:24:59.862534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:57.804765+0000","last_clean_scrub_stamp":"2024-04-20T21:24:57.804765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:19:18.784416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"84'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.005771+0000","last_change":"2024-04-20T21:24:57.840537+0000","last_active":"2024-04-20T21:25:00.005771+0000","last_peered":"2024-04-20T21:25:00.005771+0000","last_clean":"2024-04-20T21:25:00.005771+0000","last_became_active":"2024-04-20T21:24:57.840146+0000","last_became_peered":"2024-04-20T21:24:57.840146+0000","last_unstale":"2024-04-20T21:25:00.005771+0000","last_undegraded":"2024-04-20T21:25:00.005771+0000","last_fullsized":"2024-04-20T21:25:00.005771+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:24:56.789914+0000","last_clean_scrub_stamp":"2024-04-20T21:24:56.789914+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:21:08.607476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":287,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-20T21:25:00.006352+0000","last_change":"2024-04-20T21:17:43.059637+0000","last_active":"2024-04-20T21:25:00.006352+0000","last_peered":"2024-04-20T21:25:00.006352+0000","last_clean":"2024-04-20T21:25:00.006352+0000","last_became_active":"2024-04-20T21:17:43.056207+0000","last_became_peered":"2024-04-20T21:17:43.056207+0000","last_unstale":"2024-04-20T21:25:00.006352+0000","last_undegraded":"2024-04-20T21:25:00.006352+0000","last_fullsized":"2024-04-20T21:25:00.006352+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:41.036290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:41.036290+0000","last_clean_scrub_stamp":"2024-04-20T21:17:41.036290+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:58:51.091337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574938,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32784,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339056,"statfs":{"total":96636764160,"available":96603193344,"internally_reserved":0,"allocated":1642496,"data_stored":1036920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574939,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32212,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339628,"statfs":{"total":96636764160,"available":96603779072,"internally_reserved":0,"allocated":1056768,"data_stored":447176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574941,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32224,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339616,"statfs":{"total":96636764160,"available":96603766784,"internally_reserved":0,"allocated":1060864,"data_stored":448116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574940,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32192,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339648,"statfs":{"total":96636764160,"available":96603799552,"internally_reserved":0,"allocated":1032192,"data_stored":440182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574940,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32212,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339628,"statfs":{"total":96636764160,"available":96603779072,"internally_reserved":0,"allocated":1056768,"data_stored":448094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1048576,"data_stored":447064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32220,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339620,"statfs":{"total":96636764160,"available":96603770880,"internally_reserved":0,"allocated":1056768,"data_stored":446676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574940,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32788,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339052,"statfs":{"total":96636764160,"available":96603189248,"internally_reserved":0,"allocated":1642496,"data_stored":1036884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T21:25:03.763 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-20T21:25:03.764 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-20T21:25:04.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.009+0000 7fe4f6e00640 1 Processor -- start 2024-04-20T21:25:04.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.009+0000 7fe4f6e00640 1 -- start start 2024-04-20T21:25:04.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.011+0000 7fe4f6e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4f009b6d0 0x7fe4f015d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:04.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.011+0000 7fe4f6e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe4f00d0150 0x7fe4f00d05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:04.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.011+0000 7fe4f6e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4f00cd2d0 con 0x7fe4f009a9e0 2024-04-20T21:25:04.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.011+0000 7fe4f6e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fe4f00cd070 con 0x7fe4f00d0150 2024-04-20T21:25:04.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.011+0000 7fe4f6e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe4f00cd5b0 con 0x7fe4f009b6d0 2024-04-20T21:25:04.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.012+0000 7fe4efe00640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fe4f009a9e0 0x7fe4f009adc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.86:55600/0 (socket says 172.21.15.86:55600) 2024-04-20T21:25:04.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.012+0000 7fe4efe00640 1 -- 172.21.15.86:0/3026151642 learned_addr learned my addr 172.21.15.86:0/3026151642 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:25:04.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.012+0000 7fe4f4e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe4f00d0150 0x7fe4f00d05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:04.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.012+0000 7fe4e7e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4f009b6d0 0x7fe4f015d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:04.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.013+0000 7fe4ef400640 1 -- 172.21.15.86:0/3026151642 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 383581315 0 0) 0x7fe4f00cd2d0 con 0x7fe4f009a9e0 2024-04-20T21:25:04.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.013+0000 7fe4ef400640 1 -- 172.21.15.86:0/3026151642 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe4d4003520 con 0x7fe4f009a9e0 2024-04-20T21:25:04.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.013+0000 7fe4f4e00640 1 -- 172.21.15.86:0/3026151642 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4f009b6d0 msgr2=0x7fe4f015d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:04.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.013+0000 7fe4f4e00640 1 --2- 172.21.15.86:0/3026151642 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4f009b6d0 0x7fe4f015d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:04.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.013+0000 7fe4f4e00640 1 -- 172.21.15.86:0/3026151642 >> v1:172.21.15.144:6789/0 conn(0x7fe4f009a9e0 legacy=0x7fe4f009adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:04.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.013+0000 7fe4f4e00640 1 -- 172.21.15.86:0/3026151642 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4f00cd890 con 0x7fe4f00d0150 2024-04-20T21:25:04.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.014+0000 7fe4f4e00640 1 --2- 172.21.15.86:0/3026151642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4f00d0150 0x7fe4f00d05e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8009d90 tx=0x7fe4e802bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35caf8066998019 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:04.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.015+0000 7fe4ef400640 1 -- 172.21.15.86:0/3026151642 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8039070 con 0x7fe4f00d0150 2024-04-20T21:25:04.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.015+0000 7fe4ef400640 1 -- 172.21.15.86:0/3026151642 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4e802cc50 con 0x7fe4f00d0150 2024-04-20T21:25:04.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.015+0000 7fe4ef400640 1 -- 172.21.15.86:0/3026151642 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8034440 con 0x7fe4f00d0150 2024-04-20T21:25:04.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.015+0000 7fe4f6e00640 1 -- 172.21.15.86:0/3026151642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4f00d0150 msgr2=0x7fe4f00d05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:04.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.015+0000 7fe4f6e00640 1 --2- 172.21.15.86:0/3026151642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4f00d0150 0x7fe4f00d05e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8009d90 tx=0x7fe4e802bea0 comp rx=0 tx=0).stop 2024-04-20T21:25:04.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.015+0000 7fe4f6e00640 1 -- 172.21.15.86:0/3026151642 shutdown_connections 2024-04-20T21:25:04.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.015+0000 7fe4f6e00640 1 --2- 172.21.15.86:0/3026151642 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4f00d0150 0x7fe4f00d05e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:04.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.015+0000 7fe4f6e00640 1 --2- 172.21.15.86:0/3026151642 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4f009b6d0 0x7fe4f015d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:04.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.015+0000 7fe4f6e00640 1 -- 172.21.15.86:0/3026151642 >> 172.21.15.86:0/3026151642 conn(0x7fe4f00b4070 msgr2=0x7fe4f00b4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:04.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.016+0000 7fe4f6e00640 1 -- 172.21.15.86:0/3026151642 shutdown_connections 2024-04-20T21:25:04.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.016+0000 7fe4f6e00640 1 -- 172.21.15.86:0/3026151642 wait complete. 2024-04-20T21:25:04.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.017+0000 7fe4f6e00640 1 Processor -- start 2024-04-20T21:25:04.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.017+0000 7fe4f6e00640 1 -- start start 2024-04-20T21:25:04.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.018+0000 7fe4f6e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4f009a9e0 0x7fe4f0154a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:04.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.018+0000 7fe4f6e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4f009b6d0 0x7fe4f00c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:04.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.018+0000 7fe4f6e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4f00d0150 0x7fe4f006e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:04.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.018+0000 7fe4f6e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f00ce370 con 0x7fe4f009b6d0 2024-04-20T21:25:04.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.018+0000 7fe4f6e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f00cde50 con 0x7fe4f009a9e0 2024-04-20T21:25:04.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.018+0000 7fe4f6e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe4f015c890 con 0x7fe4f00d0150 2024-04-20T21:25:04.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.019+0000 7fe4e7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4f009b6d0 0x7fe4f00c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:04.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.019+0000 7fe4e7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4f009b6d0 0x7fe4f00c6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43434/0 (socket says 172.21.15.86:43434) 2024-04-20T21:25:04.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.019+0000 7fe4e7e00640 1 -- 172.21.15.86:0/279943851 learned_addr learned my addr 172.21.15.86:0/279943851 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:04.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.019+0000 7fe4e7e00640 1 -- 172.21.15.86:0/279943851 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4f00d0150 msgr2=0x7fe4f006e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:04.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.019+0000 7fe4e7e00640 1 --2- 172.21.15.86:0/279943851 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4f00d0150 0x7fe4f006e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:04.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.019+0000 7fe4e7e00640 1 -- 172.21.15.86:0/279943851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4f009a9e0 msgr2=0x7fe4f0154a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:04.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.019+0000 7fe4efe00640 1 --2- 172.21.15.86:0/279943851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4f009a9e0 0x7fe4f0154a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:04.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.019+0000 7fe4e7e00640 1 --2- 172.21.15.86:0/279943851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4f009a9e0 0x7fe4f0154a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:04.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.019+0000 7fe4e7e00640 1 -- 172.21.15.86:0/279943851 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4f00b6820 con 0x7fe4f009b6d0 2024-04-20T21:25:04.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.019+0000 7fe4efe00640 1 --2- 172.21.15.86:0/279943851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4f009a9e0 0x7fe4f0154a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:25:04.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.019+0000 7fe4e7e00640 1 --2- 172.21.15.86:0/279943851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4f009b6d0 0x7fe4f00c6b10 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fe4d800e9b0 tx=0x7fe4d800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:04.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.020+0000 7fe4ecc00640 1 -- 172.21.15.86:0/279943851 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d8017070 con 0x7fe4f009b6d0 2024-04-20T21:25:04.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.020+0000 7fe4ecc00640 1 -- 172.21.15.86:0/279943851 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4d800c9a0 con 0x7fe4f009b6d0 2024-04-20T21:25:04.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.020+0000 7fe4ecc00640 1 -- 172.21.15.86:0/279943851 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d800cd90 con 0x7fe4f009b6d0 2024-04-20T21:25:04.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.020+0000 7fe4f6e00640 1 -- 172.21.15.86:0/279943851 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4f00c7460 con 0x7fe4f009b6d0 2024-04-20T21:25:04.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.020+0000 7fe4f6e00640 1 -- 172.21.15.86:0/279943851 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4f0154f80 con 0x7fe4f009b6d0 2024-04-20T21:25:04.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.021+0000 7fe4f6e00640 1 -- 172.21.15.86:0/279943851 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4f00689e0 con 0x7fe4f009b6d0 2024-04-20T21:25:04.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.022+0000 7fe4ecc00640 1 -- 172.21.15.86:0/279943851 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe4d801a070 con 0x7fe4f009b6d0 2024-04-20T21:25:04.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.023+0000 7fe4ecc00640 1 --2- 172.21.15.86:0/279943851 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe4cc06c920 0x7fe4cc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:04.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.024+0000 7fe4ecc00640 1 -- 172.21.15.86:0/279943851 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7fe4d808bb50 con 0x7fe4f009b6d0 2024-04-20T21:25:04.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.024+0000 7fe4efe00640 1 --2- 172.21.15.86:0/279943851 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe4cc06c920 0x7fe4cc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:04.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.025+0000 7fe4efe00640 1 --2- 172.21.15.86:0/279943851 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe4cc06c920 0x7fe4cc06ede0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0005880 tx=0x7fe4e0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:04.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.028+0000 7fe4ecc00640 1 -- 172.21.15.86:0/279943851 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe4d8059f80 con 0x7fe4f009b6d0 2024-04-20T21:25:04.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.302+0000 7fe4f6e00640 1 -- 172.21.15.86:0/279943851 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fe4f00ccc70 con 0x7fe4f009b6d0 2024-04-20T21:25:04.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.306+0000 7fe4ecc00640 1 -- 172.21.15.86:0/279943851 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) ==== 190+0+147 (secure 0 0 0) 0x7fe4d8059880 con 0x7fe4f009b6d0 2024-04-20T21:25:04.306 INFO:teuthology.orchestra.run.smithi086.stdout:[client.0] 2024-04-20T21:25:04.306 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQDBMCRmHWxyKRAANzhRi4eajp68h/jWkhFMnw== 2024-04-20T21:25:04.306 INFO:teuthology.orchestra.run.smithi086.stdout: caps mds = "allow" 2024-04-20T21:25:04.306 INFO:teuthology.orchestra.run.smithi086.stdout: caps mon = "allow r" 2024-04-20T21:25:04.306 INFO:teuthology.orchestra.run.smithi086.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-20T21:25:04.306 INFO:teuthology.orchestra.run.smithi086.stderr:updated caps for client.0 2024-04-20T21:25:04.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.308+0000 7fe4f6e00640 1 -- 172.21.15.86:0/279943851 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe4cc06c920 msgr2=0x7fe4cc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:04.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.308+0000 7fe4f6e00640 1 --2- 172.21.15.86:0/279943851 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe4cc06c920 0x7fe4cc06ede0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0005880 tx=0x7fe4e0009040 comp rx=0 tx=0).stop 2024-04-20T21:25:04.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.308+0000 7fe4f6e00640 1 -- 172.21.15.86:0/279943851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4f009b6d0 msgr2=0x7fe4f00c6b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:04.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.308+0000 7fe4f6e00640 1 --2- 172.21.15.86:0/279943851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4f009b6d0 0x7fe4f00c6b10 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fe4d800e9b0 tx=0x7fe4d800ee70 comp rx=0 tx=0).stop 2024-04-20T21:25:04.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.308+0000 7fe4f6e00640 1 -- 172.21.15.86:0/279943851 shutdown_connections 2024-04-20T21:25:04.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.308+0000 7fe4f6e00640 1 --2- 172.21.15.86:0/279943851 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe4cc06c920 0x7fe4cc06ede0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:04.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.308+0000 7fe4f6e00640 1 --2- 172.21.15.86:0/279943851 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4f00d0150 0x7fe4f006e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:04.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.308+0000 7fe4f6e00640 1 --2- 172.21.15.86:0/279943851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4f009b6d0 0x7fe4f00c6b10 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:04.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.308+0000 7fe4f6e00640 1 --2- 172.21.15.86:0/279943851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4f009a9e0 0x7fe4f0154a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:04.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.308+0000 7fe4f6e00640 1 -- 172.21.15.86:0/279943851 >> 172.21.15.86:0/279943851 conn(0x7fe4f00b4070 msgr2=0x7fe4f00714e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:04.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.309+0000 7fe4f6e00640 1 -- 172.21.15.86:0/279943851 shutdown_connections 2024-04-20T21:25:04.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.309+0000 7fe4f6e00640 1 -- 172.21.15.86:0/279943851 wait complete. 2024-04-20T21:25:04.328 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:25:04.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.518+0000 7ff033e00640 1 Processor -- start 2024-04-20T21:25:04.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.518+0000 7ff033e00640 1 -- start start 2024-04-20T21:25:04.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.519+0000 7ff033e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff034156ae0 0x7ff03415d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:04.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.519+0000 7ff033e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff034160080 0x7ff0340ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:04.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.519+0000 7ff033e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff034160890 con 0x7ff0340ad620 2024-04-20T21:25:04.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.519+0000 7ff033e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7ff03406db20 con 0x7ff034156ae0 2024-04-20T21:25:04.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.519+0000 7ff033e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff034160b70 con 0x7ff034160080 2024-04-20T21:25:04.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.520+0000 7ff033400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7ff0340ad620 0x7ff034141cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:37994/0 (socket says 172.21.15.86:37994) 2024-04-20T21:25:04.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.520+0000 7ff033400640 1 -- 172.21.15.86:0/4176653922 learned_addr learned my addr 172.21.15.86:0/4176653922 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:25:04.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.520+0000 7ff032a00640 1 --2- 172.21.15.86:0/4176653922 >> v2:172.21.15.144:3300/0 conn(0x7ff034156ae0 0x7ff03415d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:04.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.520+0000 7ff032000640 1 --2- 172.21.15.86:0/4176653922 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff034160080 0x7ff0340ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:04.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.520+0000 7ff031600640 1 -- 172.21.15.86:0/4176653922 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4086553310 0 0) 0x7ff034160890 con 0x7ff0340ad620 2024-04-20T21:25:04.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.520+0000 7ff031600640 1 -- 172.21.15.86:0/4176653922 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff018003520 con 0x7ff0340ad620 2024-04-20T21:25:04.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.520+0000 7ff031600640 1 -- 172.21.15.86:0/4176653922 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4157722371 0 0) 0x7ff018003520 con 0x7ff0340ad620 2024-04-20T21:25:04.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.520+0000 7ff031600640 1 -- 172.21.15.86:0/4176653922 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff034160080 msgr2=0x7ff0340ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:04.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.520+0000 7ff031600640 1 --2- 172.21.15.86:0/4176653922 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff034160080 0x7ff0340ad0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:04.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.520+0000 7ff031600640 1 -- 172.21.15.86:0/4176653922 >> v2:172.21.15.144:3300/0 conn(0x7ff034156ae0 msgr2=0x7ff03415d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:04.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.520+0000 7ff031600640 1 --2- 172.21.15.86:0/4176653922 >> v2:172.21.15.144:3300/0 conn(0x7ff034156ae0 0x7ff03415d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:04.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.521+0000 7ff031600640 1 -- 172.21.15.86:0/4176653922 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff034160e50 con 0x7ff0340ad620 2024-04-20T21:25:04.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.521+0000 7ff031600640 1 -- 172.21.15.86:0/4176653922 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7ff02c0019f0 con 0x7ff0340ad620 2024-04-20T21:25:04.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.521+0000 7ff031600640 1 -- 172.21.15.86:0/4176653922 >> v1:172.21.15.86:6789/0 conn(0x7ff0340ad620 legacy=0x7ff034141cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:04.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.521+0000 7ff031600640 1 --2- 172.21.15.86:0/4176653922 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff034160080 0x7ff0180039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:04.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.521+0000 7ff031600640 1 --2- 172.21.15.86:0/4176653922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff018003f30 0x7ff018004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:04.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.521+0000 7ff031600640 1 --2- 172.21.15.86:0/4176653922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff018004d40 0x7ff0180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:04.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.521+0000 7ff031600640 1 -- 172.21.15.86:0/4176653922 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff03406db20 con 0x7ff018003f30 2024-04-20T21:25:04.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.521+0000 7ff031600640 1 -- 172.21.15.86:0/4176653922 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff034160b70 con 0x7ff018004d40 2024-04-20T21:25:04.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.521+0000 7ff031600640 1 -- 172.21.15.86:0/4176653922 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff01800ba10 con 0x7ff034160080 2024-04-20T21:25:04.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.521+0000 7ff032000640 1 --2- 172.21.15.86:0/4176653922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff018003f30 0x7ff018004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:04.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.521+0000 7ff032a00640 1 --2- 172.21.15.86:0/4176653922 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff034160080 0x7ff0180039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:04.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.521+0000 7ff033400640 1 --2- 172.21.15.86:0/4176653922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff018004d40 0x7ff0180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:04.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.521+0000 7ff032000640 1 -- 172.21.15.86:0/4176653922 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff034160080 msgr2=0x7ff0180039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:04.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.521+0000 7ff032000640 1 --2- 172.21.15.86:0/4176653922 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff034160080 0x7ff0180039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:04.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.521+0000 7ff032000640 1 -- 172.21.15.86:0/4176653922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff018004d40 msgr2=0x7ff0180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:04.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.521+0000 7ff032000640 1 --2- 172.21.15.86:0/4176653922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff018004d40 0x7ff0180051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:04.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.521+0000 7ff032000640 1 -- 172.21.15.86:0/4176653922 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff018003520 con 0x7ff018003f30 2024-04-20T21:25:04.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.523+0000 7ff032000640 1 --2- 172.21.15.86:0/4176653922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff018003f30 0x7ff018004800 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7ff02000ba30 tx=0x7ff018005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bafc5a01af03d792 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:04.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.523+0000 7ff031600640 1 -- 172.21.15.86:0/4176653922 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff020036020 con 0x7ff018003f30 2024-04-20T21:25:04.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.523+0000 7ff031600640 1 -- 172.21.15.86:0/4176653922 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff020032640 con 0x7ff018003f30 2024-04-20T21:25:04.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.523+0000 7ff031600640 1 -- 172.21.15.86:0/4176653922 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02000b600 con 0x7ff018003f30 2024-04-20T21:25:04.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.524+0000 7ff033e00640 1 -- 172.21.15.86:0/4176653922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff018003f30 msgr2=0x7ff018004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:04.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.524+0000 7ff033e00640 1 --2- 172.21.15.86:0/4176653922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff018003f30 0x7ff018004800 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7ff02000ba30 tx=0x7ff018005830 comp rx=0 tx=0).stop 2024-04-20T21:25:04.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.524+0000 7ff033400640 1 -- 172.21.15.86:0/4176653922 reap_dead start 2024-04-20T21:25:04.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.524+0000 7ff033e00640 1 -- 172.21.15.86:0/4176653922 shutdown_connections 2024-04-20T21:25:04.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.524+0000 7ff033e00640 1 -- 172.21.15.86:0/4176653922 >> 172.21.15.86:0/4176653922 conn(0x7ff0340d0170 msgr2=0x7ff0340d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:04.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.524+0000 7ff033e00640 1 -- 172.21.15.86:0/4176653922 shutdown_connections 2024-04-20T21:25:04.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.524+0000 7ff033e00640 1 -- 172.21.15.86:0/4176653922 wait complete. 2024-04-20T21:25:04.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.525+0000 7ff033e00640 1 Processor -- start 2024-04-20T21:25:04.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.525+0000 7ff033e00640 1 -- start start 2024-04-20T21:25:04.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.526+0000 7ff033e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff018003f30 0x7ff0341454d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:04.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.526+0000 7ff033e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff03409a260 0x7ff03409a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:04.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.526+0000 7ff033e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff03409c450 0x7ff03409c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:04.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.526+0000 7ff033e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff034160500 con 0x7ff018003f30 2024-04-20T21:25:04.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.526+0000 7ff033e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff034156ec0 con 0x7ff03409c450 2024-04-20T21:25:04.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.526+0000 7ff033e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff03406f2a0 con 0x7ff03409a260 2024-04-20T21:25:04.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.527+0000 7ff032a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff018003f30 0x7ff0341454d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:04.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.527+0000 7ff032a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff018003f30 0x7ff0341454d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43444/0 (socket says 172.21.15.86:43444) 2024-04-20T21:25:04.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.527+0000 7ff032a00640 1 -- 172.21.15.86:0/2418852373 learned_addr learned my addr 172.21.15.86:0/2418852373 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:04.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.527+0000 7ff032a00640 1 -- 172.21.15.86:0/2418852373 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff03409a260 msgr2=0x7ff03409a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:04.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.527+0000 7ff032a00640 1 --2- 172.21.15.86:0/2418852373 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff03409a260 0x7ff03409a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:04.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.527+0000 7ff032a00640 1 -- 172.21.15.86:0/2418852373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff03409c450 msgr2=0x7ff03409c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:04.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.527+0000 7ff032a00640 1 --2- 172.21.15.86:0/2418852373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff03409c450 0x7ff03409c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:04.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.527+0000 7ff032a00640 1 -- 172.21.15.86:0/2418852373 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff03409c180 con 0x7ff018003f30 2024-04-20T21:25:04.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.527+0000 7ff032a00640 1 --2- 172.21.15.86:0/2418852373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff018003f30 0x7ff0341454d0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7ff0240108e0 tx=0x7ff024010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:04.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.527+0000 7ff02aa00640 1 -- 172.21.15.86:0/2418852373 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02400f040 con 0x7ff018003f30 2024-04-20T21:25:04.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.527+0000 7ff02aa00640 1 -- 172.21.15.86:0/2418852373 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff02400eb60 con 0x7ff018003f30 2024-04-20T21:25:04.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.527+0000 7ff02aa00640 1 -- 172.21.15.86:0/2418852373 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02401c400 con 0x7ff018003f30 2024-04-20T21:25:04.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.528+0000 7ff033e00640 1 -- 172.21.15.86:0/2418852373 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff03409d210 con 0x7ff018003f30 2024-04-20T21:25:04.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.528+0000 7ff033e00640 1 -- 172.21.15.86:0/2418852373 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff03409d7d0 con 0x7ff018003f30 2024-04-20T21:25:04.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.529+0000 7ff033e00640 1 -- 172.21.15.86:0/2418852373 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff03404fcf0 con 0x7ff018003f30 2024-04-20T21:25:04.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.530+0000 7ff02aa00640 1 -- 172.21.15.86:0/2418852373 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff02401c5a0 con 0x7ff018003f30 2024-04-20T21:25:04.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.531+0000 7ff02aa00640 1 --2- 172.21.15.86:0/2418852373 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff01406c8f0 0x7ff01406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:04.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.531+0000 7ff02aa00640 1 -- 172.21.15.86:0/2418852373 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7ff02408c450 con 0x7ff018003f30 2024-04-20T21:25:04.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.531+0000 7ff032000640 1 --2- 172.21.15.86:0/2418852373 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff01406c8f0 0x7ff01406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:04.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.532+0000 7ff032000640 1 --2- 172.21.15.86:0/2418852373 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff01406c8f0 0x7ff01406edb0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff02000ba00 tx=0x7ff020032000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:04.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.536+0000 7ff02aa00640 1 -- 172.21.15.86:0/2418852373 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff024018030 con 0x7ff018003f30 2024-04-20T21:25:04.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.766+0000 7ff033e00640 1 -- 172.21.15.86:0/2418852373 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff034145db0 con 0x7ff018003f30 2024-04-20T21:25:04.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.767+0000 7ff02aa00640 1 -- 172.21.15.86:0/2418852373 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5207 (secure 0 0 0) 0x7ff02405a100 con 0x7ff018003f30 2024-04-20T21:25:04.767 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:25:04.767 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":59,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7506,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/4013863257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4013863257},{"type":"v1","addr":"172.21.15.86:6836","nonce":4013863257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":7509,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2044883073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2044883073},{"type":"v1","addr":"172.21.15.86:6837","nonce":2044883073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7518,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/2351381466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2351381466},{"type":"v1","addr":"172.21.15.144:6834","nonce":2351381466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:24:58.299257+0000","modified":"2024-04-20T21:25:01.355818+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7521":{"gid":7521,"name":"b","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6835/1740492227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":1740492227},{"type":"v1","addr":"172.21.15.144:6835","nonce":1740492227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7521,"qdb_cluster":[7521]},"id":6}]} 2024-04-20T21:25:04.767 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 59 2024-04-20T21:25:04.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.769+0000 7ff033e00640 1 -- 172.21.15.86:0/2418852373 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff01406c8f0 msgr2=0x7ff01406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:04.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.769+0000 7ff033e00640 1 --2- 172.21.15.86:0/2418852373 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff01406c8f0 0x7ff01406edb0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff02000ba00 tx=0x7ff020032000 comp rx=0 tx=0).stop 2024-04-20T21:25:04.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.769+0000 7ff033e00640 1 -- 172.21.15.86:0/2418852373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff018003f30 msgr2=0x7ff0341454d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:04.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.769+0000 7ff033e00640 1 --2- 172.21.15.86:0/2418852373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff018003f30 0x7ff0341454d0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7ff0240108e0 tx=0x7ff024010da0 comp rx=0 tx=0).stop 2024-04-20T21:25:04.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.769+0000 7ff033e00640 1 -- 172.21.15.86:0/2418852373 shutdown_connections 2024-04-20T21:25:04.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.769+0000 7ff033e00640 1 --2- 172.21.15.86:0/2418852373 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff01406c8f0 0x7ff01406edb0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:04.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.769+0000 7ff033e00640 1 --2- 172.21.15.86:0/2418852373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff03409c450 0x7ff03409c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:04.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.769+0000 7ff033e00640 1 --2- 172.21.15.86:0/2418852373 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff03409a260 0x7ff03409a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:04.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.769+0000 7ff033e00640 1 --2- 172.21.15.86:0/2418852373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff018003f30 0x7ff0341454d0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:04.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.769+0000 7ff033e00640 1 -- 172.21.15.86:0/2418852373 >> 172.21.15.86:0/2418852373 conn(0x7ff0340d0170 msgr2=0x7ff0340b3d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:04.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.769+0000 7ff033e00640 1 -- 172.21.15.86:0/2418852373 shutdown_connections 2024-04-20T21:25:04.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:04.769+0000 7ff033e00640 1 -- 172.21.15.86:0/2418852373 wait complete. 2024-04-20T21:25:04.797 INFO:tasks.ceph.mds.b.smithi144.stderr:2024-04-20T21:25:04.798+0000 7f9ad0c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T21:25:05.188 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 59, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T21:24:58.299257+0000', 'modified': '2024-04-20T21:25:01.355818+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7521}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7521': {'gid': 7521, 'name': 'b', 'rank': 0, 'incarnation': 55, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.144:6835/1740492227', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6833', 'nonce': 1740492227}, {'type': 'v1', 'addr': '172.21.15.144:6835', 'nonce': 1740492227}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7521, 'qdb_cluster': [7521]} 2024-04-20T21:25:05.188 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-20T21:25:05.188 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-20T21:25:05.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.391+0000 7f5c53e00640 1 Processor -- start 2024-04-20T21:25:05.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.391+0000 7f5c53e00640 1 -- start start 2024-04-20T21:25:05.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.393+0000 7f5c53e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5c5415e690 0x7f5c5406ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:05.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.393+0000 7f5c53e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c540d3100 con 0x7f5c540d2cb0 2024-04-20T21:25:05.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.393+0000 7f5c53e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c540d33b0 con 0x7f5c540d0ee0 2024-04-20T21:25:05.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.393+0000 7f5c53e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f5c540d3690 con 0x7f5c5415e690 2024-04-20T21:25:05.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.393+0000 7f5c53400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f5c540d2cb0 0x7f5c540cf420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:38008/0 (socket says 172.21.15.86:38008) 2024-04-20T21:25:05.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.393+0000 7f5c53400640 1 -- 172.21.15.86:0/1281070281 learned_addr learned my addr 172.21.15.86:0/1281070281 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:25:05.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.394+0000 7f5c52a00640 1 --2- 172.21.15.86:0/1281070281 >> v2:172.21.15.144:3300/0 conn(0x7f5c5415e690 0x7f5c5406ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:05.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.394+0000 7f5c52000640 1 -- 172.21.15.86:0/1281070281 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2541796981 0 0) 0x7f5c540d3100 con 0x7f5c540d2cb0 2024-04-20T21:25:05.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.394+0000 7f5c52000640 1 -- 172.21.15.86:0/1281070281 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5c38003520 con 0x7f5c540d2cb0 2024-04-20T21:25:05.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.394+0000 7f5c52000640 1 -- 172.21.15.86:0/1281070281 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 334040190 0 0) 0x7f5c38003520 con 0x7f5c540d2cb0 2024-04-20T21:25:05.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.394+0000 7f5c52000640 1 -- 172.21.15.86:0/1281070281 >> v2:172.21.15.144:3300/0 conn(0x7f5c5415e690 msgr2=0x7f5c5406ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:05.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.394+0000 7f5c52000640 1 --2- 172.21.15.86:0/1281070281 >> v2:172.21.15.144:3300/0 conn(0x7f5c5415e690 0x7f5c5406ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:05.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.394+0000 7f5c52000640 1 -- 172.21.15.86:0/1281070281 >> v1:172.21.15.144:6789/0 conn(0x7f5c540d0ee0 legacy=0x7f5c540d25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:05.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.394+0000 7f5c52000640 1 -- 172.21.15.86:0/1281070281 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c540d3970 con 0x7f5c540d2cb0 2024-04-20T21:25:05.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.394+0000 7f5c52000640 1 -- 172.21.15.86:0/1281070281 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f5c4c0019f0 con 0x7f5c540d2cb0 2024-04-20T21:25:05.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.394+0000 7f5c52000640 1 -- 172.21.15.86:0/1281070281 >> v1:172.21.15.86:6789/0 conn(0x7f5c540d2cb0 legacy=0x7f5c540cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:05.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.394+0000 7f5c52000640 1 --2- 172.21.15.86:0/1281070281 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c380039b0 0x7f5c38003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:05.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.394+0000 7f5c52000640 1 --2- 172.21.15.86:0/1281070281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c38004800 0x7f5c38004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:05.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.394+0000 7f5c52000640 1 --2- 172.21.15.86:0/1281070281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c380051c0 0x7f5c3800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:05.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.394+0000 7f5c52000640 1 -- 172.21.15.86:0/1281070281 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5c540d3690 con 0x7f5c380051c0 2024-04-20T21:25:05.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.394+0000 7f5c52000640 1 -- 172.21.15.86:0/1281070281 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5c3800c060 con 0x7f5c38004800 2024-04-20T21:25:05.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.394+0000 7f5c52000640 1 -- 172.21.15.86:0/1281070281 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5c3800c260 con 0x7f5c380039b0 2024-04-20T21:25:05.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.395+0000 7f5c53400640 1 --2- 172.21.15.86:0/1281070281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c380051c0 0x7f5c3800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:05.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.395+0000 7f5c52a00640 1 --2- 172.21.15.86:0/1281070281 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c380039b0 0x7f5c38003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:05.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.395+0000 7f5c53400640 1 -- 172.21.15.86:0/1281070281 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c380039b0 msgr2=0x7f5c38003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:05.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.395+0000 7f5c53400640 1 --2- 172.21.15.86:0/1281070281 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c380039b0 0x7f5c38003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:05.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.395+0000 7f5c53400640 1 -- 172.21.15.86:0/1281070281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c38004800 msgr2=0x7f5c38004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:05.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.395+0000 7f5c4be00640 1 --2- 172.21.15.86:0/1281070281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c38004800 0x7f5c38004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:05.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.395+0000 7f5c53400640 1 --2- 172.21.15.86:0/1281070281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c38004800 0x7f5c38004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:05.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.395+0000 7f5c53400640 1 -- 172.21.15.86:0/1281070281 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c38003520 con 0x7f5c380051c0 2024-04-20T21:25:05.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.396+0000 7f5c53400640 1 -- 172.21.15.86:0/1281070281 reap_dead start 2024-04-20T21:25:05.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.396+0000 7f5c53400640 1 --2- 172.21.15.86:0/1281070281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c380051c0 0x7f5c3800b9c0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c00a860 tx=0x7f5c38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f99f576eb43712 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:05.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.396+0000 7f5c52000640 1 -- 172.21.15.86:0/1281070281 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c00a480 con 0x7f5c380051c0 2024-04-20T21:25:05.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.397+0000 7f5c52000640 1 -- 172.21.15.86:0/1281070281 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c4c034630 con 0x7f5c380051c0 2024-04-20T21:25:05.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.397+0000 7f5c52000640 1 -- 172.21.15.86:0/1281070281 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c034a20 con 0x7f5c380051c0 2024-04-20T21:25:05.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.397+0000 7f5c53e00640 1 -- 172.21.15.86:0/1281070281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c380051c0 msgr2=0x7f5c3800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:05.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.397+0000 7f5c53e00640 1 --2- 172.21.15.86:0/1281070281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c380051c0 0x7f5c3800b9c0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c00a860 tx=0x7f5c38005830 comp rx=0 tx=0).stop 2024-04-20T21:25:05.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.397+0000 7f5c53e00640 1 -- 172.21.15.86:0/1281070281 shutdown_connections 2024-04-20T21:25:05.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.397+0000 7f5c53e00640 1 --2- 172.21.15.86:0/1281070281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c380051c0 0x7f5c3800b9c0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:05.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.397+0000 7f5c53e00640 1 -- 172.21.15.86:0/1281070281 >> 172.21.15.86:0/1281070281 conn(0x7f5c5409d8a0 msgr2=0x7f5c5408bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:05.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.397+0000 7f5c53e00640 1 -- 172.21.15.86:0/1281070281 shutdown_connections 2024-04-20T21:25:05.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.397+0000 7f5c53e00640 1 -- 172.21.15.86:0/1281070281 wait complete. 2024-04-20T21:25:05.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.399+0000 7f5c53e00640 1 Processor -- start 2024-04-20T21:25:05.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.399+0000 7f5c53e00640 1 -- start start 2024-04-20T21:25:05.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.400+0000 7f5c53e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c380051c0 0x7f5c540b4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:05.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.401+0000 7f5c53e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c540c92e0 0x7f5c5415ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:05.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.401+0000 7f5c53e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c540c9760 0x7f5c5415f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:05.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.401+0000 7f5c53e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5415ea70 con 0x7f5c540c92e0 2024-04-20T21:25:05.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.401+0000 7f5c53e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5c540cef00 con 0x7f5c380051c0 2024-04-20T21:25:05.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.401+0000 7f5c53e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5c5415ebf0 con 0x7f5c540c9760 2024-04-20T21:25:05.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.401+0000 7f5c4be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c540c92e0 0x7f5c5415ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:05.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.401+0000 7f5c4be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c540c92e0 0x7f5c5415ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43462/0 (socket says 172.21.15.86:43462) 2024-04-20T21:25:05.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.401+0000 7f5c4be00640 1 -- 172.21.15.86:0/4168050214 learned_addr learned my addr 172.21.15.86:0/4168050214 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:05.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.401+0000 7f5c4be00640 1 -- 172.21.15.86:0/4168050214 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c540c9760 msgr2=0x7f5c5415f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:05.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.401+0000 7f5c4be00640 1 --2- 172.21.15.86:0/4168050214 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c540c9760 0x7f5c5415f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:05.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.401+0000 7f5c52a00640 1 --2- 172.21.15.86:0/4168050214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c380051c0 0x7f5c540b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:05.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.401+0000 7f5c4be00640 1 -- 172.21.15.86:0/4168050214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c380051c0 msgr2=0x7f5c540b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:05.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.401+0000 7f5c4be00640 1 --2- 172.21.15.86:0/4168050214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c380051c0 0x7f5c540b4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:05.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.401+0000 7f5c4be00640 1 -- 172.21.15.86:0/4168050214 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c540b4a30 con 0x7f5c540c92e0 2024-04-20T21:25:05.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.402+0000 7f5c4be00640 1 --2- 172.21.15.86:0/4168050214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c540c92e0 0x7f5c5415ed70 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c00d4d0 tx=0x7f5c3c00d990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:05.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.402+0000 7f5c4b400640 1 -- 172.21.15.86:0/4168050214 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c005480 con 0x7f5c540c92e0 2024-04-20T21:25:05.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.402+0000 7f5c4b400640 1 -- 172.21.15.86:0/4168050214 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c3c017070 con 0x7f5c540c92e0 2024-04-20T21:25:05.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.402+0000 7f5c53e00640 1 -- 172.21.15.86:0/4168050214 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c540b4d10 con 0x7f5c540c92e0 2024-04-20T21:25:05.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.402+0000 7f5c4b400640 1 -- 172.21.15.86:0/4168050214 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c003880 con 0x7f5c540c92e0 2024-04-20T21:25:05.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.402+0000 7f5c53e00640 1 -- 172.21.15.86:0/4168050214 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c540b52d0 con 0x7f5c540c92e0 2024-04-20T21:25:05.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.404+0000 7f5c4b400640 1 -- 172.21.15.86:0/4168050214 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5c3c003a20 con 0x7f5c540c92e0 2024-04-20T21:25:05.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.405+0000 7f5c4b400640 1 --2- 172.21.15.86:0/4168050214 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5c3006c8f0 0x7f5c3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:05.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.405+0000 7f5c4b400640 1 -- 172.21.15.86:0/4168050214 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7f5c3c08b9e0 con 0x7f5c540c92e0 2024-04-20T21:25:05.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.405+0000 7f5c52a00640 1 --2- 172.21.15.86:0/4168050214 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5c3006c8f0 0x7f5c3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:05.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.406+0000 7f5c52a00640 1 --2- 172.21.15.86:0/4168050214 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5c3006c8f0 0x7f5c3006edb0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f5c44009910 tx=0x7f5c44008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:05.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.406+0000 7f5c5ca59740 1 Processor -- start 2024-04-20T21:25:05.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.406+0000 7f5c5ca59740 1 -- start start 2024-04-20T21:25:05.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.407+0000 7f5c5ca59740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8235d8670 0x55b82358fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:05.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.407+0000 7f5c5ca59740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b82358e6f0 0x55b8235901e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:05.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.407+0000 7f5c5ca59740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b82358ab90 0x55b8235ba980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:05.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.407+0000 7f5c5ca59740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b8234f69d0 con 0x55b82358ab90 2024-04-20T21:25:05.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.407+0000 7f5c5ca59740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b82349d270 con 0x55b8235d8670 2024-04-20T21:25:05.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.407+0000 7f5c5ca59740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55b8234f2990 con 0x55b82358e6f0 2024-04-20T21:25:05.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.407+0000 7f5c53400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b82358ab90 0x55b8235ba980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:05.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.407+0000 7f5c53400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b82358ab90 0x55b8235ba980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43478/0 (socket says 172.21.15.86:43478) 2024-04-20T21:25:05.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.407+0000 7f5c53400640 1 -- 172.21.15.86:0/1739985818 learned_addr learned my addr 172.21.15.86:0/1739985818 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:05.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.408+0000 7f5c52a00640 1 --2- 172.21.15.86:0/1739985818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8235d8670 0x55b82358fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:05.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.408+0000 7f5c4be00640 1 --2- 172.21.15.86:0/1739985818 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b82358e6f0 0x55b8235901e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:05.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.408+0000 7f5c53400640 1 -- 172.21.15.86:0/1739985818 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b82358e6f0 msgr2=0x55b8235901e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:05.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.408+0000 7f5c53400640 1 --2- 172.21.15.86:0/1739985818 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b82358e6f0 0x55b8235901e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:05.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.408+0000 7f5c53400640 1 -- 172.21.15.86:0/1739985818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8235d8670 msgr2=0x55b82358fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:05.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.408+0000 7f5c53400640 1 --2- 172.21.15.86:0/1739985818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8235d8670 0x55b82358fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:05.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.408+0000 7f5c53400640 1 -- 172.21.15.86:0/1739985818 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8232092d0 con 0x55b82358ab90 2024-04-20T21:25:05.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.408+0000 7f5c53400640 1 --2- 172.21.15.86:0/1739985818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b82358ab90 0x55b8235ba980 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c02cc70 tx=0x7f5c4c007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddc63a510b280409 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:05.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.408+0000 7f5c43400640 1 -- 172.21.15.86:0/1739985818 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c00a480 con 0x55b82358ab90 2024-04-20T21:25:05.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.408+0000 7f5c43400640 1 -- 172.21.15.86:0/1739985818 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c4c001730 con 0x55b82358ab90 2024-04-20T21:25:05.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.408+0000 7f5c43400640 1 -- 172.21.15.86:0/1739985818 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c0018e0 con 0x55b82358ab90 2024-04-20T21:25:05.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.409+0000 7f5c5ca59740 1 -- 172.21.15.86:0/1739985818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b82358ab90 msgr2=0x55b8235ba980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:05.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.409+0000 7f5c5ca59740 1 --2- 172.21.15.86:0/1739985818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b82358ab90 0x55b8235ba980 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c02cc70 tx=0x7f5c4c007000 comp rx=0 tx=0).stop 2024-04-20T21:25:05.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.409+0000 7f5c5ca59740 1 -- 172.21.15.86:0/1739985818 shutdown_connections 2024-04-20T21:25:05.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.409+0000 7f5c5ca59740 1 --2- 172.21.15.86:0/1739985818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b82358ab90 0x55b8235ba980 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:05.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.409+0000 7f5c5ca59740 1 --2- 172.21.15.86:0/1739985818 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b82358e6f0 0x55b8235901e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:05.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.409+0000 7f5c5ca59740 1 --2- 172.21.15.86:0/1739985818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8235d8670 0x55b82358fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:05.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.409+0000 7f5c5ca59740 1 -- 172.21.15.86:0/1739985818 >> 172.21.15.86:0/1739985818 conn(0x55b82358ce00 msgr2=0x55b8235d5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:05.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.409+0000 7f5c5ca59740 1 -- 172.21.15.86:0/1739985818 shutdown_connections 2024-04-20T21:25:05.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.409+0000 7f5c5ca59740 1 -- 172.21.15.86:0/1739985818 wait complete. 2024-04-20T21:25:05.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.410+0000 7f5c5ca59740 1 -- start start 2024-04-20T21:25:05.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.410+0000 7f5c5ca59740 1 Processor -- start 2024-04-20T21:25:05.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.411+0000 7f5c5ca59740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b82358e6f0 0x55b82359e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:05.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.411+0000 7f5c5ca59740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b8235d8670 0x55b8235b1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:05.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.411+0000 7f5c5ca59740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8235a0c10 0x55b8235d6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:05.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.411+0000 7f5c5ca59740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b82358ec00 con 0x55b82358e6f0 2024-04-20T21:25:05.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.411+0000 7f5c5ca59740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b8234b05e0 con 0x55b8235a0c10 2024-04-20T21:25:05.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.411+0000 7f5c5ca59740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55b82358eee0 con 0x55b8235d8670 2024-04-20T21:25:05.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.411+0000 7f5c5ca59740 10 client.0 _ll_register_callbacks cb 0x55b823208ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:25:05.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.411+0000 7f5c52a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b82358e6f0 0x55b82359e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:05.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.411+0000 7f5c52a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b82358e6f0 0x55b82359e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43480/0 (socket says 172.21.15.86:43480) 2024-04-20T21:25:05.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.411+0000 7f5c52a00640 1 -- 172.21.15.86:0/1721872277 learned_addr learned my addr 172.21.15.86:0/1721872277 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:05.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.412+0000 7f5c4be00640 1 --2- 172.21.15.86:0/1721872277 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b8235d8670 0x55b8235b1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:05.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.412+0000 7f5c53400640 1 --2- 172.21.15.86:0/1721872277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8235a0c10 0x55b8235d6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:05.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.412+0000 7f5c52a00640 1 -- 172.21.15.86:0/1721872277 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b8235d8670 msgr2=0x55b8235b1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:05.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.412+0000 7f5c52a00640 1 --2- 172.21.15.86:0/1721872277 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b8235d8670 0x55b8235b1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:05.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.412+0000 7f5c52a00640 1 -- 172.21.15.86:0/1721872277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8235a0c10 msgr2=0x55b8235d6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:05.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.412+0000 7f5c52a00640 1 --2- 172.21.15.86:0/1721872277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8235a0c10 0x55b8235d6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:05.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.412+0000 7f5c52a00640 1 -- 172.21.15.86:0/1721872277 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8232d36b0 con 0x55b82358e6f0 2024-04-20T21:25:05.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.412+0000 7f5c4be00640 1 --2- 172.21.15.86:0/1721872277 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b8235d8670 0x55b8235b1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:25:05.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.412+0000 7f5c52a00640 1 --2- 172.21.15.86:0/1721872277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b82358e6f0 0x55b82359e1c0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f5c440156f0 tx=0x7f5c44004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7a8ad6fe7853bf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:05.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.412+0000 7f5c37e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:25:05.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.412+0000 7f5c37e00640 1 -- 172.21.15.86:0/1721872277 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c44026070 con 0x55b82358e6f0 2024-04-20T21:25:05.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.412+0000 7f5c5ca59740 1 -- 172.21.15.86:0/1721872277 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c4c001730 con 0x55b82358e6f0 2024-04-20T21:25:05.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.412+0000 7f5c37e00640 1 -- 172.21.15.86:0/1721872277 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7f5c44014730 con 0x55b82358e6f0 2024-04-20T21:25:05.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.413+0000 7f5c37e00640 1 -- 172.21.15.86:0/1721872277 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f5c44031050 con 0x55b82358e6f0 2024-04-20T21:25:05.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.413+0000 7f5c5ca59740 10 client.7665 fetch_fsmap learned FSMap version 59 2024-04-20T21:25:05.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.413+0000 7f5c5ca59740 1 -- 172.21.15.86:0/1721872277 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) -- 0x55b82359efd0 con 0x55b82358e6f0 2024-04-20T21:25:05.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.413+0000 7f5c37e00640 1 -- 172.21.15.86:0/1721872277 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c4c001730 con 0x55b82358e6f0 2024-04-20T21:25:05.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.413+0000 7f5c37e00640 1 -- 172.21.15.86:0/1721872277 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 59) ==== 3495+0+0 (secure 0 0 0) 0x7f5c44034020 con 0x55b82358e6f0 2024-04-20T21:25:05.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.413+0000 7f5c5ca59740 10 client.7665 fetch_fsmap finished waiting for FSMap version 59 2024-04-20T21:25:05.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.413+0000 7f5c5ca59740 10 client.7665 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-20T21:25:05.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.413+0000 7f5c5ca59740 20 client.7665 populate_metadata read hostname 'smithi086' 2024-04-20T21:25:05.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.413+0000 7f5c5ca59740 1 --2- 172.21.15.86:0/1721872277 >> [v2:172.21.15.144:6833/1740492227,v1:172.21.15.144:6835/1740492227] conn(0x55b82359ffa0 0x55b8235ea100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:05.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.413+0000 7f5c5ca59740 4 client.7665 mds_command: new command op to 7521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:25:05.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.413+0000 7f5c5ca59740 1 -- 172.21.15.86:0/1721872277 --> [v2:172.21.15.144:6833/1740492227,v1:172.21.15.144:6835/1740492227] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8235043f0 con 0x55b82359ffa0 2024-04-20T21:25:05.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.414+0000 7f5c4be00640 1 --2- 172.21.15.86:0/1721872277 >> [v2:172.21.15.144:6833/1740492227,v1:172.21.15.144:6835/1740492227] conn(0x55b82359ffa0 0x55b8235ea100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:05.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.415+0000 7f5c4be00640 1 --2- 172.21.15.86:0/1721872277 >> [v2:172.21.15.144:6833/1740492227,v1:172.21.15.144:6835/1740492227] conn(0x55b82359ffa0 0x55b8235ea100 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f46b208378b3849 server_cookie=e4dcaa941316c7ac in_seq=0 out_seq=0 2024-04-20T21:25:05.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.415+0000 7f5c37e00640 10 client.7665 ms_handle_connect on v2:172.21.15.144:6833/1740492227 2024-04-20T21:25:05.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.416+0000 7f5c37e00640 1 -- 172.21.15.86:0/1721872277 <== mds.0 v2:172.21.15.144:6833/1740492227 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b8235043f0 con 0x55b82359ffa0 2024-04-20T21:25:05.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.416+0000 7f5c37e00640 10 client.7665 handle_command_reply: tid=0 2024-04-20T21:25:05.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.416+0000 7f5c5ca59740 1 client.7665 shutdown 2024-04-20T21:25:05.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.416+0000 7f5c5ca59740 1 -- 172.21.15.86:0/1721872277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b82358e6f0 msgr2=0x55b82359e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:05.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.416+0000 7f5c5ca59740 1 --2- 172.21.15.86:0/1721872277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b82358e6f0 0x55b82359e1c0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f5c440156f0 tx=0x7f5c44004730 comp rx=0 tx=0).stop 2024-04-20T21:25:05.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.417+0000 7f5c5ca59740 1 -- 172.21.15.86:0/1721872277 shutdown_connections 2024-04-20T21:25:05.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.417+0000 7f5c5ca59740 1 --2- 172.21.15.86:0/1721872277 >> [v2:172.21.15.144:6833/1740492227,v1:172.21.15.144:6835/1740492227] conn(0x55b82359ffa0 0x55b8235ea100 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:05.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.417+0000 7f5c5ca59740 1 --2- 172.21.15.86:0/1721872277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8235a0c10 0x55b8235d6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:05.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.417+0000 7f5c5ca59740 1 --2- 172.21.15.86:0/1721872277 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b8235d8670 0x55b8235b1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:05.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.417+0000 7f5c37e00640 0 client.7665 ms_handle_reset on v2:172.21.15.144:6833/1740492227 2024-04-20T21:25:05.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.417+0000 7f5c5ca59740 1 --2- 172.21.15.86:0/1721872277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b82358e6f0 0x55b82359e1c0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:05.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.417+0000 7f5c5ca59740 1 -- 172.21.15.86:0/1721872277 >> 172.21.15.86:0/1721872277 conn(0x55b82358ab90 msgr2=0x55b82358fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:05.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.417+0000 7f5c5ca59740 1 -- 172.21.15.86:0/1721872277 shutdown_connections 2024-04-20T21:25:05.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.417+0000 7f5c5ca59740 1 -- 172.21.15.86:0/1721872277 wait complete. 2024-04-20T21:25:05.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.417+0000 7f5c5ca59740 20 client.7665 trim_cache size 0 max 16384 2024-04-20T21:25:05.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.436+0000 7f5c5ca59740 1 Processor -- start 2024-04-20T21:25:05.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.436+0000 7f5c5ca59740 1 -- start start 2024-04-20T21:25:05.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.437+0000 7f5c5ca59740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b82358e6f0 0x55b8235b0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:05.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.437+0000 7f5c5ca59740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8235a0c10 0x55b82358b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:05.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.437+0000 7f5c5ca59740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b8235d8670 0x55b82358b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:05.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.437+0000 7f5c5ca59740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5c542003c0 con 0x55b82358e6f0 2024-04-20T21:25:05.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.437+0000 7f5c5ca59740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5c541ff750 con 0x55b8235a0c10 2024-04-20T21:25:05.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.437+0000 7f5c5ca59740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5c541ffda0 con 0x55b8235d8670 2024-04-20T21:25:05.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.437+0000 7f5c52a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b82358e6f0 0x55b8235b0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:05.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.437+0000 7f5c52a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b82358e6f0 0x55b8235b0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43494/0 (socket says 172.21.15.86:43494) 2024-04-20T21:25:05.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.437+0000 7f5c52a00640 1 -- 172.21.15.86:0/4233511006 learned_addr learned my addr 172.21.15.86:0/4233511006 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:05.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.438+0000 7f5c52a00640 1 -- 172.21.15.86:0/4233511006 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b8235d8670 msgr2=0x55b82358b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:05.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.438+0000 7f5c52a00640 1 --2- 172.21.15.86:0/4233511006 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b8235d8670 0x55b82358b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:05.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.438+0000 7f5c52a00640 1 -- 172.21.15.86:0/4233511006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8235a0c10 msgr2=0x55b82358b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:05.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.438+0000 7f5c52a00640 1 --2- 172.21.15.86:0/4233511006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8235a0c10 0x55b82358b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:05.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.438+0000 7f5c52a00640 1 -- 172.21.15.86:0/4233511006 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8235d6f60 con 0x55b82358e6f0 2024-04-20T21:25:05.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.438+0000 7f5c52a00640 1 --2- 172.21.15.86:0/4233511006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b82358e6f0 0x55b8235b0d70 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f5c440147b0 tx=0x7f5c44022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4e723b9321b58f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:05.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.438+0000 7f5c42a00640 1 -- 172.21.15.86:0/4233511006 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c44026070 con 0x55b82358e6f0 2024-04-20T21:25:05.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.438+0000 7f5c42a00640 1 -- 172.21.15.86:0/4233511006 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c4403a070 con 0x55b82358e6f0 2024-04-20T21:25:05.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.438+0000 7f5c42a00640 1 -- 172.21.15.86:0/4233511006 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c44014a20 con 0x55b82358e6f0 2024-04-20T21:25:05.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.439+0000 7f5c5ca59740 1 -- 172.21.15.86:0/4233511006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b82358e6f0 msgr2=0x55b8235b0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:05.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.439+0000 7f5c5ca59740 1 --2- 172.21.15.86:0/4233511006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b82358e6f0 0x55b8235b0d70 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f5c440147b0 tx=0x7f5c44022000 comp rx=0 tx=0).stop 2024-04-20T21:25:05.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.439+0000 7f5c5ca59740 1 -- 172.21.15.86:0/4233511006 shutdown_connections 2024-04-20T21:25:05.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.439+0000 7f5c5ca59740 1 --2- 172.21.15.86:0/4233511006 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b8235d8670 0x55b82358b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:05.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.439+0000 7f5c5ca59740 1 --2- 172.21.15.86:0/4233511006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8235a0c10 0x55b82358b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:05.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.439+0000 7f5c5ca59740 1 --2- 172.21.15.86:0/4233511006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b82358e6f0 0x55b8235b0d70 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:05.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.439+0000 7f5c5ca59740 1 -- 172.21.15.86:0/4233511006 >> 172.21.15.86:0/4233511006 conn(0x55b82358ab90 msgr2=0x55b823586280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:05.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.439+0000 7f5c5ca59740 1 -- 172.21.15.86:0/4233511006 shutdown_connections 2024-04-20T21:25:05.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.439+0000 7f5c5ca59740 1 -- 172.21.15.86:0/4233511006 wait complete. 2024-04-20T21:25:05.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.440+0000 7f5c5ca59740 1 -- start start 2024-04-20T21:25:05.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.440+0000 7f5c5ca59740 1 Processor -- start 2024-04-20T21:25:05.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.441+0000 7f5c5ca59740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b82358e6f0 0x55b8235cb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:05.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.441+0000 7f5c5ca59740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b8235a0c10 0x55b8235d5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:05.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.441+0000 7f5c5ca59740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8235d8670 0x55b8235e82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:05.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.441+0000 7f5c5ca59740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b82359fe20 con 0x55b82358e6f0 2024-04-20T21:25:05.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.441+0000 7f5c5ca59740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b82359f370 con 0x55b8235d8670 2024-04-20T21:25:05.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.441+0000 7f5c5ca59740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55b8235a0440 con 0x55b8235a0c10 2024-04-20T21:25:05.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.441+0000 7f5c5ca59740 10 client.0 _ll_register_callbacks cb 0x7f5c440095e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:25:05.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.441+0000 7f5c52a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b82358e6f0 0x55b8235cb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:05.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.441+0000 7f5c52a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b82358e6f0 0x55b8235cb550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43500/0 (socket says 172.21.15.86:43500) 2024-04-20T21:25:05.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.441+0000 7f5c52a00640 1 -- 172.21.15.86:0/3612986541 learned_addr learned my addr 172.21.15.86:0/3612986541 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:05.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.441+0000 7f5c52a00640 1 -- 172.21.15.86:0/3612986541 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b8235a0c10 msgr2=0x55b8235d5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:05.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.441+0000 7f5c52a00640 1 --2- 172.21.15.86:0/3612986541 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b8235a0c10 0x55b8235d5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:05.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.441+0000 7f5c52a00640 1 -- 172.21.15.86:0/3612986541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8235d8670 msgr2=0x55b8235e82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:05.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.441+0000 7f5c52a00640 1 --2- 172.21.15.86:0/3612986541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8235d8670 0x55b8235e82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:05.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.441+0000 7f5c52a00640 1 -- 172.21.15.86:0/3612986541 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8235a36b0 con 0x55b82358e6f0 2024-04-20T21:25:05.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.442+0000 7f5c52a00640 1 --2- 172.21.15.86:0/3612986541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b82358e6f0 0x55b8235cb550 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f5c44011630 tx=0x7f5c44016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f294d39d47528efe server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:05.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.442+0000 7f5c37e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:25:05.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.442+0000 7f5c37e00640 1 -- 172.21.15.86:0/3612986541 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c44044020 con 0x55b82358e6f0 2024-04-20T21:25:05.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.442+0000 7f5c37e00640 1 -- 172.21.15.86:0/3612986541 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7f5c44011e40 con 0x55b82358e6f0 2024-04-20T21:25:05.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.442+0000 7f5c5ca59740 1 -- 172.21.15.86:0/3612986541 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c4403a070 con 0x55b82358e6f0 2024-04-20T21:25:05.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.442+0000 7f5c37e00640 1 -- 172.21.15.86:0/3612986541 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f5c44026050 con 0x55b82358e6f0 2024-04-20T21:25:05.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.442+0000 7f5c5ca59740 10 client.7671 fetch_fsmap learned FSMap version 59 2024-04-20T21:25:05.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.442+0000 7f5c5ca59740 1 -- 172.21.15.86:0/3612986541 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) -- 0x55b8235d9990 con 0x55b82358e6f0 2024-04-20T21:25:05.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.443+0000 7f5c37e00640 1 -- 172.21.15.86:0/3612986541 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c4403a070 con 0x55b82358e6f0 2024-04-20T21:25:05.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.443+0000 7f5c37e00640 1 -- 172.21.15.86:0/3612986541 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 59) ==== 3495+0+0 (secure 0 0 0) 0x7f5c44051020 con 0x55b82358e6f0 2024-04-20T21:25:05.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.443+0000 7f5c5ca59740 10 client.7671 fetch_fsmap finished waiting for FSMap version 59 2024-04-20T21:25:05.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.443+0000 7f5c5ca59740 10 client.7671 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-20T21:25:05.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.443+0000 7f5c5ca59740 20 client.7671 populate_metadata read hostname 'smithi086' 2024-04-20T21:25:05.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.443+0000 7f5c5ca59740 1 --2- 172.21.15.86:0/3612986541 >> [v2:172.21.15.144:6833/1740492227,v1:172.21.15.144:6835/1740492227] conn(0x55b8235b5c80 0x55b8235b6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:05.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.443+0000 7f5c5ca59740 4 client.7671 mds_command: new command op to 7521 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T21:25:05.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.443+0000 7f5c5ca59740 1 -- 172.21.15.86:0/3612986541 --> [v2:172.21.15.144:6833/1740492227,v1:172.21.15.144:6835/1740492227] -- command(tid 0: {"prefix": "status"}) -- 0x55b8234fe260 con 0x55b8235b5c80 2024-04-20T21:25:05.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.443+0000 7f5c4be00640 1 --2- 172.21.15.86:0/3612986541 >> [v2:172.21.15.144:6833/1740492227,v1:172.21.15.144:6835/1740492227] conn(0x55b8235b5c80 0x55b8235b6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:05.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.444+0000 7f5c4be00640 1 --2- 172.21.15.86:0/3612986541 >> [v2:172.21.15.144:6833/1740492227,v1:172.21.15.144:6835/1740492227] conn(0x55b8235b5c80 0x55b8235b6060 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbf0fb0d24234860 server_cookie=325426991c142a26 in_seq=0 out_seq=0 2024-04-20T21:25:05.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.444+0000 7f5c37e00640 10 client.7671 ms_handle_connect on v2:172.21.15.144:6833/1740492227 2024-04-20T21:25:05.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.445+0000 7f5c37e00640 1 -- 172.21.15.86:0/3612986541 <== mds.0 v2:172.21.15.144:6833/1740492227 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55b8234fe260 con 0x55b8235b5c80 2024-04-20T21:25:05.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.445+0000 7f5c37e00640 10 client.7671 handle_command_reply: tid=0 2024-04-20T21:25:05.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.445+0000 7f5c5ca59740 1 client.7671 shutdown 2024-04-20T21:25:05.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.445+0000 7f5c5ca59740 1 -- 172.21.15.86:0/3612986541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b82358e6f0 msgr2=0x55b8235cb550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:05.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.445+0000 7f5c5ca59740 1 --2- 172.21.15.86:0/3612986541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b82358e6f0 0x55b8235cb550 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f5c44011630 tx=0x7f5c44016000 comp rx=0 tx=0).stop 2024-04-20T21:25:05.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.445+0000 7f5c5ca59740 1 -- 172.21.15.86:0/3612986541 shutdown_connections 2024-04-20T21:25:05.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.445+0000 7f5c5ca59740 1 --2- 172.21.15.86:0/3612986541 >> [v2:172.21.15.144:6833/1740492227,v1:172.21.15.144:6835/1740492227] conn(0x55b8235b5c80 0x55b8235b6060 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:05.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.445+0000 7f5c5ca59740 1 --2- 172.21.15.86:0/3612986541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b8235d8670 0x55b8235e82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:05.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.445+0000 7f5c5ca59740 1 --2- 172.21.15.86:0/3612986541 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b8235a0c10 0x55b8235d5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:05.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.445+0000 7f5c5ca59740 1 --2- 172.21.15.86:0/3612986541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b82358e6f0 0x55b8235cb550 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:05.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.445+0000 7f5c37e00640 0 client.7671 ms_handle_reset on v2:172.21.15.144:6833/1740492227 2024-04-20T21:25:05.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.445+0000 7f5c5ca59740 1 -- 172.21.15.86:0/3612986541 >> 172.21.15.86:0/3612986541 conn(0x55b82358ab90 msgr2=0x55b823590840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:05.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.445+0000 7f5c5ca59740 1 -- 172.21.15.86:0/3612986541 shutdown_connections 2024-04-20T21:25:05.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.445+0000 7f5c5ca59740 1 -- 172.21.15.86:0/3612986541 wait complete. 2024-04-20T21:25:05.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.445+0000 7f5c5ca59740 20 client.7671 trim_cache size 0 max 16384 2024-04-20T21:25:05.449 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:25:05.449 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "9c2dec1d-d7ae-4a83-a812-79922808e278", 2024-04-20T21:25:05.449 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-04-20T21:25:05.449 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 7521, 2024-04-20T21:25:05.449 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-20T21:25:05.449 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-20T21:25:05.449 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-20T21:25:05.449 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 6.6125004790000004, 2024-04-20T21:25:05.449 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 59, 2024-04-20T21:25:05.449 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 85, 2024-04-20T21:25:05.449 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 85, 2024-04-20T21:25:05.449 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 12.053065645 2024-04-20T21:25:05.450 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:25:05.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.446+0000 7f5c53e00640 1 -- 172.21.15.86:0/4168050214 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5c3006c8f0 msgr2=0x7f5c3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:05.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.446+0000 7f5c53e00640 1 --2- 172.21.15.86:0/4168050214 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5c3006c8f0 0x7f5c3006edb0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f5c44009910 tx=0x7f5c44008040 comp rx=0 tx=0).stop 2024-04-20T21:25:05.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.446+0000 7f5c53e00640 1 -- 172.21.15.86:0/4168050214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c540c92e0 msgr2=0x7f5c5415ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:05.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.446+0000 7f5c53e00640 1 --2- 172.21.15.86:0/4168050214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c540c92e0 0x7f5c5415ed70 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c00d4d0 tx=0x7f5c3c00d990 comp rx=0 tx=0).stop 2024-04-20T21:25:05.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.446+0000 7f5c53e00640 1 -- 172.21.15.86:0/4168050214 shutdown_connections 2024-04-20T21:25:05.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.446+0000 7f5c53e00640 1 --2- 172.21.15.86:0/4168050214 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5c3006c8f0 0x7f5c3006edb0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:05.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.446+0000 7f5c53e00640 1 --2- 172.21.15.86:0/4168050214 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5c540c9760 0x7f5c5415f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:05.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.446+0000 7f5c53e00640 1 --2- 172.21.15.86:0/4168050214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c540c92e0 0x7f5c5415ed70 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:05.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.446+0000 7f5c53e00640 1 --2- 172.21.15.86:0/4168050214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c380051c0 0x7f5c540b4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:05.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.446+0000 7f5c53e00640 1 -- 172.21.15.86:0/4168050214 >> 172.21.15.86:0/4168050214 conn(0x7f5c5409d8a0 msgr2=0x7f5c540b1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:05.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.446+0000 7f5c53e00640 1 -- 172.21.15.86:0/4168050214 shutdown_connections 2024-04-20T21:25:05.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.446+0000 7f5c53e00640 1 -- 172.21.15.86:0/4168050214 wait complete. 2024-04-20T21:25:05.463 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T21:25:05.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.649+0000 7efd77e00640 1 Processor -- start 2024-04-20T21:25:05.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.649+0000 7efd77e00640 1 -- start start 2024-04-20T21:25:05.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.651+0000 7efd77e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd7815d6a0 0x7efd780ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:05.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.651+0000 7efd77e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7efd780ad620 0x7efd7806e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:05.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.651+0000 7efd77e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd78160890 con 0x7efd78156ae0 2024-04-20T21:25:05.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.651+0000 7efd77e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7efd7806db20 con 0x7efd780ad620 2024-04-20T21:25:05.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.651+0000 7efd77e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7efd78160b70 con 0x7efd7815d6a0 2024-04-20T21:25:05.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.651+0000 7efd77400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7efd780ad620 0x7efd7806e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:05.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.651+0000 7efd77400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7efd780ad620 0x7efd7806e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43512/0 (socket says 172.21.15.86:43512) 2024-04-20T21:25:05.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.651+0000 7efd77400640 1 -- 172.21.15.86:0/3439331564 learned_addr learned my addr 172.21.15.86:0/3439331564 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:05.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.651+0000 7efd6e000640 1 --2- 172.21.15.86:0/3439331564 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd7815d6a0 0x7efd780ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:05.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.652+0000 7efd77400640 1 -- 172.21.15.86:0/3439331564 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd7815d6a0 msgr2=0x7efd780ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:05.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.652+0000 7efd77400640 1 --2- 172.21.15.86:0/3439331564 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd7815d6a0 0x7efd780ad0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:05.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.652+0000 7efd77400640 1 -- 172.21.15.86:0/3439331564 >> v1:172.21.15.144:6789/0 conn(0x7efd78156ae0 legacy=0x7efd78160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:05.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.652+0000 7efd77400640 1 -- 172.21.15.86:0/3439331564 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd78160e50 con 0x7efd780ad620 2024-04-20T21:25:05.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.652+0000 7efd6e000640 1 --2- 172.21.15.86:0/3439331564 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd7815d6a0 0x7efd780ad0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:25:05.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.653+0000 7efd77400640 1 --2- 172.21.15.86:0/3439331564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd780ad620 0x7efd7806e290 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7efd70009e50 tx=0x7efd7002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac1a6ac6c4492e9a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:05.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.653+0000 7efd76000640 1 -- 172.21.15.86:0/3439331564 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7002ca70 con 0x7efd780ad620 2024-04-20T21:25:05.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.653+0000 7efd76000640 1 -- 172.21.15.86:0/3439331564 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd7002cc10 con 0x7efd780ad620 2024-04-20T21:25:05.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.653+0000 7efd76000640 1 -- 172.21.15.86:0/3439331564 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd700343f0 con 0x7efd780ad620 2024-04-20T21:25:05.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.654+0000 7efd77e00640 1 -- 172.21.15.86:0/3439331564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd780ad620 msgr2=0x7efd7806e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:05.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.654+0000 7efd77e00640 1 --2- 172.21.15.86:0/3439331564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd780ad620 0x7efd7806e290 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7efd70009e50 tx=0x7efd7002bed0 comp rx=0 tx=0).stop 2024-04-20T21:25:05.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.654+0000 7efd77e00640 1 -- 172.21.15.86:0/3439331564 shutdown_connections 2024-04-20T21:25:05.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.654+0000 7efd77e00640 1 --2- 172.21.15.86:0/3439331564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd780ad620 0x7efd7806e290 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:05.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.654+0000 7efd77e00640 1 --2- 172.21.15.86:0/3439331564 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd7815d6a0 0x7efd780ad0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:05.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.654+0000 7efd77e00640 1 -- 172.21.15.86:0/3439331564 >> 172.21.15.86:0/3439331564 conn(0x7efd780d0170 msgr2=0x7efd780d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:05.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.654+0000 7efd77e00640 1 -- 172.21.15.86:0/3439331564 shutdown_connections 2024-04-20T21:25:05.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.654+0000 7efd77e00640 1 -- 172.21.15.86:0/3439331564 wait complete. 2024-04-20T21:25:05.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.655+0000 7efd77e00640 1 Processor -- start 2024-04-20T21:25:05.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.656+0000 7efd77e00640 1 -- start start 2024-04-20T21:25:05.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.656+0000 7efd77e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd780ad620 0x7efd78145510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:05.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.656+0000 7efd77e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd78156ae0 0x7efd781f1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:05.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.656+0000 7efd77e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd7815d6a0 0x7efd781542d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:05.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.656+0000 7efd77e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efd7806e7d0 con 0x7efd780ad620 2024-04-20T21:25:05.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.656+0000 7efd77e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efd78156ec0 con 0x7efd78156ae0 2024-04-20T21:25:05.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.656+0000 7efd77e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7efd7806e950 con 0x7efd7815d6a0 2024-04-20T21:25:05.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.657+0000 7efd76a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd780ad620 0x7efd78145510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:05.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.657+0000 7efd76a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd780ad620 0x7efd78145510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43518/0 (socket says 172.21.15.86:43518) 2024-04-20T21:25:05.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.657+0000 7efd76a00640 1 -- 172.21.15.86:0/515630156 learned_addr learned my addr 172.21.15.86:0/515630156 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:05.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.657+0000 7efd76a00640 1 -- 172.21.15.86:0/515630156 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd7815d6a0 msgr2=0x7efd781542d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:05.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.657+0000 7efd76a00640 1 --2- 172.21.15.86:0/515630156 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd7815d6a0 0x7efd781542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:05.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.657+0000 7efd76a00640 1 -- 172.21.15.86:0/515630156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd78156ae0 msgr2=0x7efd781f1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:05.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.657+0000 7efd76a00640 1 --2- 172.21.15.86:0/515630156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd78156ae0 0x7efd781f1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:05.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.657+0000 7efd76a00640 1 -- 172.21.15.86:0/515630156 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd7815c860 con 0x7efd780ad620 2024-04-20T21:25:05.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.657+0000 7efd76a00640 1 --2- 172.21.15.86:0/515630156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd780ad620 0x7efd78145510 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7efd6800c430 tx=0x7efd6800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:05.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.658+0000 7efd6f400640 1 -- 172.21.15.86:0/515630156 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd68011790 con 0x7efd780ad620 2024-04-20T21:25:05.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.658+0000 7efd6f400640 1 -- 172.21.15.86:0/515630156 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd68017070 con 0x7efd780ad620 2024-04-20T21:25:05.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.658+0000 7efd77e00640 1 -- 172.21.15.86:0/515630156 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd7815c580 con 0x7efd780ad620 2024-04-20T21:25:05.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.658+0000 7efd6f400640 1 -- 172.21.15.86:0/515630156 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd68011b70 con 0x7efd780ad620 2024-04-20T21:25:05.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.658+0000 7efd77e00640 1 -- 172.21.15.86:0/515630156 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd78153ad0 con 0x7efd780ad620 2024-04-20T21:25:05.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.659+0000 7efd77e00640 1 -- 172.21.15.86:0/515630156 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd7804fcf0 con 0x7efd780ad620 2024-04-20T21:25:05.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.660+0000 7efd6f400640 1 -- 172.21.15.86:0/515630156 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efd68011d10 con 0x7efd780ad620 2024-04-20T21:25:05.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.661+0000 7efd6f400640 1 --2- 172.21.15.86:0/515630156 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efd5406c8f0 0x7efd5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:05.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.661+0000 7efd6f400640 1 -- 172.21.15.86:0/515630156 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7efd6808b9a0 con 0x7efd780ad620 2024-04-20T21:25:05.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.661+0000 7efd6e000640 1 --2- 172.21.15.86:0/515630156 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efd5406c8f0 0x7efd5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:05.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.662+0000 7efd6e000640 1 --2- 172.21.15.86:0/515630156 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efd5406c8f0 0x7efd5406edb0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7efd60006fd0 tx=0x7efd60008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:05.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.665+0000 7efd6f400640 1 -- 172.21.15.86:0/515630156 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7efd68059dd0 con 0x7efd780ad620 2024-04-20T21:25:05.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.910+0000 7efd77e00640 1 -- 172.21.15.86:0/515630156 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7efd780ad210 con 0x7efd780ad620 2024-04-20T21:25:05.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.911+0000 7efd6f400640 1 -- 172.21.15.86:0/515630156 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v59) ==== 53+0+73 (secure 0 0 0) 0x7efd680596d0 con 0x7efd780ad620 2024-04-20T21:25:05.911 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T21:25:05.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.913+0000 7efd77e00640 1 -- 172.21.15.86:0/515630156 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efd5406c8f0 msgr2=0x7efd5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:05.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.914+0000 7efd77e00640 1 --2- 172.21.15.86:0/515630156 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efd5406c8f0 0x7efd5406edb0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7efd60006fd0 tx=0x7efd60008040 comp rx=0 tx=0).stop 2024-04-20T21:25:05.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.914+0000 7efd77e00640 1 -- 172.21.15.86:0/515630156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd780ad620 msgr2=0x7efd78145510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:05.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.914+0000 7efd77e00640 1 --2- 172.21.15.86:0/515630156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd780ad620 0x7efd78145510 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7efd6800c430 tx=0x7efd6800c8f0 comp rx=0 tx=0).stop 2024-04-20T21:25:05.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.914+0000 7efd77e00640 1 -- 172.21.15.86:0/515630156 shutdown_connections 2024-04-20T21:25:05.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.914+0000 7efd77e00640 1 --2- 172.21.15.86:0/515630156 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efd5406c8f0 0x7efd5406edb0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:05.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.914+0000 7efd77e00640 1 --2- 172.21.15.86:0/515630156 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efd7815d6a0 0x7efd781542d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:05.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.914+0000 7efd77e00640 1 --2- 172.21.15.86:0/515630156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd78156ae0 0x7efd781f1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:05.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.914+0000 7efd77e00640 1 --2- 172.21.15.86:0/515630156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd780ad620 0x7efd78145510 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:05.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.914+0000 7efd77e00640 1 -- 172.21.15.86:0/515630156 >> 172.21.15.86:0/515630156 conn(0x7efd780d0170 msgr2=0x7efd780b2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:05.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.914+0000 7efd77e00640 1 -- 172.21.15.86:0/515630156 shutdown_connections 2024-04-20T21:25:05.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:05.914+0000 7efd77e00640 1 -- 172.21.15.86:0/515630156 wait complete. 2024-04-20T21:25:05.933 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T21:25:05.933 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T21:25:05.933 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-04-20T21:25:05.933 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-20T21:25:05.933 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T21:25:05.933 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T21:25:05.933 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T21:25:05.933 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T21:25:05.933 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:25:05.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:25:05.934 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-04-20T21:25:05.981 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T21:25:05.981 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T21:25:05.981 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T21:25:05.981 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:25:05.981 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-20T21:25:05.981 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:25:05.981 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:25:05.982 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:25:05.982 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-04-20T21:25:05.982 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:25:05.982 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:25:05.982 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-04-20T21:25:05.982 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:25:05.982 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:25:05.982 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T21:25:05.982 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:25:05.982 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T21:25:05.982 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42250sec preferred_lft 42250sec 2024-04-20T21:25:05.982 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-04-20T21:25:05.982 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:25:05.982 INFO:teuthology.orchestra.run.smithi086.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T21:25:05.982 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 86:25:97:2c:9c:bb brd ff:ff:ff:ff:ff:ff 2024-04-20T21:25:05.983 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-20T21:25:05.983 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:25:05.983 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::a4fb:30ff:fe90:2d6a/64 scope link 2024-04-20T21:25:05.983 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:25:05.983 INFO:teuthology.orchestra.run.smithi086.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T21:25:05.983 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 86:25:97:2c:9c:bb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:25:05.983 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::8425:97ff:fe2c:9cbb/64 scope link 2024-04-20T21:25:05.983 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:25:05.983 INFO:teuthology.orchestra.run.smithi086.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T21:25:05.983 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c2:9c:8b:55:ca:84 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:25:05.983 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c09c:8bff:fe55:ca84/64 scope link 2024-04-20T21:25:05.983 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:25:05.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:25:05.984 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T21:25:06.056 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-20T21:25:06.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:25:06.060 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-04-20T21:25:06.108 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-04-20T21:25:06.109 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T21:25:06.109 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T21:25:06.109 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T21:25:06.109 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T21:25:06.109 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:25:06.109 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-20T21:25:06.109 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T21:25:06.109 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T21:25:06.109 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T21:25:06.110 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-20T21:25:06.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:25:06.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:25:06.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:25:06.308 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-20T21:25:06.355 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-20T21:25:06.356 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-20T21:25:06.357 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:25:06.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:25:06.357 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-04-20T21:25:06.485 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-20T21:25:06.485 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T21:25:06.485 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-20T21:25:06.539 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:25:06.540 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T21:25:06.540 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-20T21:25:06.590 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-04-20T21:25:06.677 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T21:25:06.731 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-04-20T21:25:06.731 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-04-20T21:25:06.731 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-04-20T21:25:06.731 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-04-20T21:25:06.731 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-04-20T21:25:06.731 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-04-20T21:25:06.731 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-04-20T21:25:06.731 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-04-20T21:25:06.731 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-04-20T21:25:06.731 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-04-20T21:25:06.731 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-04-20T21:25:06.732 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/selinux 2024-04-20T21:25:06.732 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T21:25:06.732 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-04-20T21:25:06.732 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-04-20T21:25:06.732 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-04-20T21:25:06.732 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-04-20T21:25:06.732 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-04-20T21:25:06.732 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-04-20T21:25:06.732 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T21:25:06.732 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T21:25:06.732 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T21:25:06.732 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T21:25:06.732 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-04-20T21:25:06.732 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-04-20T21:25:06.733 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T21:25:06.733 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T21:25:06.733 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T21:25:06.733 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T21:25:06.733 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-04-20T21:25:06.733 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:25:06.733 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:25:06.733 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:25:06.733 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:25:06.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:25:06.734 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-04-20T21:25:06.787 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T21:25:06.787 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-20T21:25:06.792 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-04-20T21:25:06.859 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T21:25:06.903 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-04-20T21:25:06.903 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-04-20T21:25:06.903 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-04-20T21:25:06.903 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-04-20T21:25:06.903 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-04-20T21:25:06.903 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-04-20T21:25:06.903 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-04-20T21:25:06.903 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-04-20T21:25:06.903 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-04-20T21:25:06.903 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-04-20T21:25:06.904 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-04-20T21:25:06.904 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/selinux 2024-04-20T21:25:06.904 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T21:25:06.904 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-04-20T21:25:06.904 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-04-20T21:25:06.904 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-04-20T21:25:06.904 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-04-20T21:25:06.904 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-04-20T21:25:06.904 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-04-20T21:25:06.904 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T21:25:06.904 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T21:25:06.904 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T21:25:06.904 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T21:25:06.904 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-04-20T21:25:06.905 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-04-20T21:25:06.905 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T21:25:06.905 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T21:25:06.905 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T21:25:06.905 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T21:25:06.905 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-04-20T21:25:06.905 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:25:06.905 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:25:06.905 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:25:06.905 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:25:06.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:25:06.906 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-04-20T21:25:06.959 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:2024-04-20T21:25:06.960+0000 7f1f0a648480 -1 init, newargv = 0x55990002c280 newargc=15 2024-04-20T21:25:06.960 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[53050]: starting ceph client 2024-04-20T21:25:06.970 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[53050]: starting fuse 2024-04-20T21:25:06.988 INFO:teuthology.orchestra.run.smithi086.stdout:41 2024-04-20T21:25:06.988 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-04-20T21:25:06.989 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer -- python3 -c ' 2024-04-20T21:25:06.989 DEBUG:teuthology.orchestra.run.smithi086:> import glob 2024-04-20T21:25:06.989 DEBUG:teuthology.orchestra.run.smithi086:> import re 2024-04-20T21:25:06.989 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:25:06.989 DEBUG:teuthology.orchestra.run.smithi086:> import subprocess 2024-04-20T21:25:06.989 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:25:06.989 DEBUG:teuthology.orchestra.run.smithi086:> def _find_admin_socket(client_name): 2024-04-20T21:25:06.989 DEBUG:teuthology.orchestra.run.smithi086:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-20T21:25:06.989 DEBUG:teuthology.orchestra.run.smithi086:> files = glob.glob(asok_path) 2024-04-20T21:25:06.989 DEBUG:teuthology.orchestra.run.smithi086:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-20T21:25:06.989 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:25:06.989 DEBUG:teuthology.orchestra.run.smithi086:> # Given a non-glob path, it better be there 2024-04-20T21:25:06.989 DEBUG:teuthology.orchestra.run.smithi086:> if "*" not in asok_path: 2024-04-20T21:25:06.989 DEBUG:teuthology.orchestra.run.smithi086:> assert(len(files) == 1) 2024-04-20T21:25:06.989 DEBUG:teuthology.orchestra.run.smithi086:> return files[0] 2024-04-20T21:25:06.989 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:25:06.990 DEBUG:teuthology.orchestra.run.smithi086:> for f in files: 2024-04-20T21:25:06.990 DEBUG:teuthology.orchestra.run.smithi086:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T21:25:06.990 DEBUG:teuthology.orchestra.run.smithi086:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T21:25:06.990 DEBUG:teuthology.orchestra.run.smithi086:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T21:25:06.990 DEBUG:teuthology.orchestra.run.smithi086:> contents = proc_f.read() 2024-04-20T21:25:06.990 DEBUG:teuthology.orchestra.run.smithi086:> if mountpoint in contents: 2024-04-20T21:25:06.990 DEBUG:teuthology.orchestra.run.smithi086:> return f 2024-04-20T21:25:06.990 DEBUG:teuthology.orchestra.run.smithi086:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T21:25:06.990 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:25:06.990 DEBUG:teuthology.orchestra.run.smithi086:> print(_find_admin_socket("client.0")) 2024-04-20T21:25:06.990 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:25:07.178 INFO:teuthology.orchestra.run.smithi086.stdout:/var/run/ceph/ceph-client.0.53050.asok 2024-04-20T21:25:07.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:25:07.196 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.53050.asok 2024-04-20T21:25:07.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:25:07.196 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.53050.asok status 2024-04-20T21:25:07.423 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:25:07.424 INFO:teuthology.orchestra.run.smithi086.stdout: "metadata": { 2024-04-20T21:25:07.424 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T21:25:07.424 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T21:25:07.424 INFO:teuthology.orchestra.run.smithi086.stdout: "entity_id": "0", 2024-04-20T21:25:07.424 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": "smithi086", 2024-04-20T21:25:07.424 INFO:teuthology.orchestra.run.smithi086.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-20T21:25:07.424 INFO:teuthology.orchestra.run.smithi086.stdout: "pid": "53050", 2024-04-20T21:25:07.424 INFO:teuthology.orchestra.run.smithi086.stdout: "root": "/" 2024-04-20T21:25:07.424 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:25:07.424 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_count": 0, 2024-04-20T21:25:07.424 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_pinned_count": 0, 2024-04-20T21:25:07.424 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 7683, 2024-04-20T21:25:07.424 INFO:teuthology.orchestra.run.smithi086.stdout: "inst": { 2024-04-20T21:25:07.425 INFO:teuthology.orchestra.run.smithi086.stdout: "name": { 2024-04-20T21:25:07.425 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "client", 2024-04-20T21:25:07.425 INFO:teuthology.orchestra.run.smithi086.stdout: "num": 7683 2024-04-20T21:25:07.425 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:25:07.425 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2024-04-20T21:25:07.425 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-04-20T21:25:07.425 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2024-04-20T21:25:07.425 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 2639132107 2024-04-20T21:25:07.425 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:25:07.425 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:25:07.425 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2024-04-20T21:25:07.425 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-04-20T21:25:07.425 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2024-04-20T21:25:07.425 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 2639132107 2024-04-20T21:25:07.425 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:25:07.426 INFO:teuthology.orchestra.run.smithi086.stdout: "inst_str": "client.7683 192.168.0.1:0/2639132107", 2024-04-20T21:25:07.426 INFO:teuthology.orchestra.run.smithi086.stdout: "addr_str": "192.168.0.1:0/2639132107", 2024-04-20T21:25:07.426 INFO:teuthology.orchestra.run.smithi086.stdout: "inode_count": 1, 2024-04-20T21:25:07.426 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_epoch": 59, 2024-04-20T21:25:07.426 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch": 85, 2024-04-20T21:25:07.426 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch_barrier": 0, 2024-04-20T21:25:07.426 INFO:teuthology.orchestra.run.smithi086.stdout: "blocklisted": false, 2024-04-20T21:25:07.426 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs" 2024-04-20T21:25:07.426 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:25:07.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:25:07.439 DEBUG:teuthology.orchestra.run.smithi086:> stat --file-system '--printf=%T 2024-04-20T21:25:07.439 DEBUG:teuthology.orchestra.run.smithi086:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-20T21:25:07.489 INFO:teuthology.orchestra.run.smithi086.stdout:fuseblk 2024-04-20T21:25:07.489 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-20T21:25:07.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:25:07.489 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-20T21:25:07.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:25:07.572 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir) 2024-04-20T21:25:07.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:25:07.647 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir/file) 2024-04-20T21:25:07.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:25:07.718 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir testdir) 2024-04-20T21:25:07.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:25:07.781 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln dir/file testdir/hardlink) 2024-04-20T21:25:07.856 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-04-20T21:25:07.856 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:25:07.857 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-04-20T21:25:07.857 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:25:07.857 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir").st_ino) 2024-04-20T21:25:07.857 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:25:07.996 INFO:teuthology.orchestra.run.smithi086.stdout:1099511627776 2024-04-20T21:25:08.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:25:08.009 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:25:08.009 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:25:08.106 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:25:08.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.303+0000 7fe63ac00640 1 Processor -- start 2024-04-20T21:25:08.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.303+0000 7fe63ac00640 1 -- start start 2024-04-20T21:25:08.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.304+0000 7fe63ac00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe63415c010 0x7fe63415c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:08.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.304+0000 7fe63ac00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe63406f4c0 0x7fe63406f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:08.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.304+0000 7fe63ac00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6340ae710 con 0x7fe6340af7b0 2024-04-20T21:25:08.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.304+0000 7fe63ac00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fe6340af320 con 0x7fe63406f4c0 2024-04-20T21:25:08.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.304+0000 7fe63ac00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe63415c930 con 0x7fe63415c010 2024-04-20T21:25:08.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.305+0000 7fe638e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe6340af7b0 0x7fe6340adda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57460/0 (socket says 172.21.15.86:57460) 2024-04-20T21:25:08.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.305+0000 7fe638e00640 1 -- 172.21.15.86:0/2281780825 learned_addr learned my addr 172.21.15.86:0/2281780825 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:25:08.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.305+0000 7fe639800640 1 --2- 172.21.15.86:0/2281780825 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe63415c010 0x7fe63415c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:08.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.305+0000 7fe63a200640 1 --2- 172.21.15.86:0/2281780825 >> v2:172.21.15.144:3300/0 conn(0x7fe63406f4c0 0x7fe63406f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:08.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.305+0000 7fe633e00640 1 -- 172.21.15.86:0/2281780825 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 144935266 0 0) 0x7fe6340ae710 con 0x7fe6340af7b0 2024-04-20T21:25:08.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.305+0000 7fe633e00640 1 -- 172.21.15.86:0/2281780825 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe618003520 con 0x7fe6340af7b0 2024-04-20T21:25:08.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.305+0000 7fe633e00640 1 -- 172.21.15.86:0/2281780825 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1624460824 0 0) 0x7fe618003520 con 0x7fe6340af7b0 2024-04-20T21:25:08.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.305+0000 7fe633e00640 1 -- 172.21.15.86:0/2281780825 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe63415c010 msgr2=0x7fe63415c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:08.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.305+0000 7fe633e00640 1 --2- 172.21.15.86:0/2281780825 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe63415c010 0x7fe63415c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:08.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.305+0000 7fe633e00640 1 -- 172.21.15.86:0/2281780825 >> v2:172.21.15.144:3300/0 conn(0x7fe63406f4c0 msgr2=0x7fe63406f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:08.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.305+0000 7fe633e00640 1 --2- 172.21.15.86:0/2281780825 >> v2:172.21.15.144:3300/0 conn(0x7fe63406f4c0 0x7fe63406f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:08.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.306+0000 7fe633e00640 1 -- 172.21.15.86:0/2281780825 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6340ae9f0 con 0x7fe6340af7b0 2024-04-20T21:25:08.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.306+0000 7fe633e00640 1 -- 172.21.15.86:0/2281780825 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fe6200019f0 con 0x7fe6340af7b0 2024-04-20T21:25:08.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.306+0000 7fe639800640 1 --2- 172.21.15.86:0/2281780825 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe63415c010 0x7fe63415c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:25:08.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.306+0000 7fe63a200640 1 --2- 172.21.15.86:0/2281780825 >> v2:172.21.15.144:3300/0 conn(0x7fe63406f4c0 0x7fe63406f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:25:08.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.306+0000 7fe633e00640 1 -- 172.21.15.86:0/2281780825 >> v1:172.21.15.86:6789/0 conn(0x7fe6340af7b0 legacy=0x7fe6340adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:08.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.306+0000 7fe633e00640 1 --2- 172.21.15.86:0/2281780825 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe63415c010 0x7fe6180039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:08.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.306+0000 7fe633e00640 1 --2- 172.21.15.86:0/2281780825 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe618003fa0 0x7fe618004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:08.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.306+0000 7fe633e00640 1 --2- 172.21.15.86:0/2281780825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe618004d40 0x7fe6180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:08.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.306+0000 7fe633e00640 1 -- 172.21.15.86:0/2281780825 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe6340af320 con 0x7fe618004d40 2024-04-20T21:25:08.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.306+0000 7fe633e00640 1 -- 172.21.15.86:0/2281780825 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe63415c930 con 0x7fe618003fa0 2024-04-20T21:25:08.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.306+0000 7fe633e00640 1 -- 172.21.15.86:0/2281780825 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe61800dcb0 con 0x7fe63415c010 2024-04-20T21:25:08.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.306+0000 7fe63a200640 1 --2- 172.21.15.86:0/2281780825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe618004d40 0x7fe6180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:08.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.306+0000 7fe639800640 1 --2- 172.21.15.86:0/2281780825 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe63415c010 0x7fe6180039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:08.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.306+0000 7fe63a200640 1 -- 172.21.15.86:0/2281780825 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe63415c010 msgr2=0x7fe6180039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:08.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.306+0000 7fe63a200640 1 --2- 172.21.15.86:0/2281780825 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe63415c010 0x7fe6180039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:08.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.306+0000 7fe63a200640 1 -- 172.21.15.86:0/2281780825 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe618003fa0 msgr2=0x7fe618004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:08.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.306+0000 7fe638e00640 1 --2- 172.21.15.86:0/2281780825 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe618003fa0 0x7fe618004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:08.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.306+0000 7fe63a200640 1 --2- 172.21.15.86:0/2281780825 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe618003fa0 0x7fe618004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:08.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.307+0000 7fe63a200640 1 -- 172.21.15.86:0/2281780825 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe618003520 con 0x7fe618004d40 2024-04-20T21:25:08.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.307+0000 7fe639800640 1 --2- 172.21.15.86:0/2281780825 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe63415c010 0x7fe6180039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:25:08.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.308+0000 7fe63a200640 1 --2- 172.21.15.86:0/2281780825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe618004d40 0x7fe6180051c0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fe6240098a0 tx=0x7fe618005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50431569875cd4d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:08.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.308+0000 7fe633e00640 1 -- 172.21.15.86:0/2281780825 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62400f040 con 0x7fe618004d40 2024-04-20T21:25:08.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.308+0000 7fe633e00640 1 -- 172.21.15.86:0/2281780825 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe624004230 con 0x7fe618004d40 2024-04-20T21:25:08.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.308+0000 7fe633e00640 1 -- 172.21.15.86:0/2281780825 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe624003610 con 0x7fe618004d40 2024-04-20T21:25:08.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.308+0000 7fe63ac00640 1 -- 172.21.15.86:0/2281780825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe618004d40 msgr2=0x7fe6180051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:08.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.308+0000 7fe63ac00640 1 --2- 172.21.15.86:0/2281780825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe618004d40 0x7fe6180051c0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fe6240098a0 tx=0x7fe618005830 comp rx=0 tx=0).stop 2024-04-20T21:25:08.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.308+0000 7fe63a200640 1 -- 172.21.15.86:0/2281780825 reap_dead start 2024-04-20T21:25:08.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.308+0000 7fe63ac00640 1 -- 172.21.15.86:0/2281780825 shutdown_connections 2024-04-20T21:25:08.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.308+0000 7fe63ac00640 1 -- 172.21.15.86:0/2281780825 >> 172.21.15.86:0/2281780825 conn(0x7fe6340d0170 msgr2=0x7fe6340d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:08.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.308+0000 7fe63ac00640 1 -- 172.21.15.86:0/2281780825 shutdown_connections 2024-04-20T21:25:08.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.309+0000 7fe63ac00640 1 -- 172.21.15.86:0/2281780825 wait complete. 2024-04-20T21:25:08.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.310+0000 7fe63ac00640 1 Processor -- start 2024-04-20T21:25:08.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.310+0000 7fe63ac00640 1 -- start start 2024-04-20T21:25:08.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.311+0000 7fe63ac00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6340998c0 0x7fe6341585b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:08.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.311+0000 7fe63ac00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6341465c0 0x7fe6341555a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:08.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.311+0000 7fe63ac00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe634155ae0 0x7fe634145280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:08.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.311+0000 7fe63ac00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe6340aefb0 con 0x7fe634155ae0 2024-04-20T21:25:08.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.311+0000 7fe63ac00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe6340ae4b0 con 0x7fe6340998c0 2024-04-20T21:25:08.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.311+0000 7fe63ac00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe63415c3f0 con 0x7fe6341465c0 2024-04-20T21:25:08.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.312+0000 7fe63a200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe634155ae0 0x7fe634145280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:08.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.312+0000 7fe63a200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe634155ae0 0x7fe634145280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60048/0 (socket says 172.21.15.86:60048) 2024-04-20T21:25:08.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.312+0000 7fe63a200640 1 -- 172.21.15.86:0/2401947360 learned_addr learned my addr 172.21.15.86:0/2401947360 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:08.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.312+0000 7fe639800640 1 --2- 172.21.15.86:0/2401947360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6340998c0 0x7fe6341585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:08.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.312+0000 7fe638e00640 1 --2- 172.21.15.86:0/2401947360 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6341465c0 0x7fe6341555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:08.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.312+0000 7fe63a200640 1 -- 172.21.15.86:0/2401947360 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6341465c0 msgr2=0x7fe6341555a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:08.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.312+0000 7fe63a200640 1 --2- 172.21.15.86:0/2401947360 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6341465c0 0x7fe6341555a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:08.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.312+0000 7fe63a200640 1 -- 172.21.15.86:0/2401947360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6340998c0 msgr2=0x7fe6341585b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:08.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.312+0000 7fe63a200640 1 --2- 172.21.15.86:0/2401947360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6340998c0 0x7fe6341585b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:08.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.312+0000 7fe63a200640 1 -- 172.21.15.86:0/2401947360 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe634146a40 con 0x7fe634155ae0 2024-04-20T21:25:08.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.312+0000 7fe638e00640 1 --2- 172.21.15.86:0/2401947360 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6341465c0 0x7fe6341555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:25:08.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.313+0000 7fe63a200640 1 --2- 172.21.15.86:0/2401947360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe634155ae0 0x7fe634145280 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fe624009990 tx=0x7fe624004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:08.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.313+0000 7fe631600640 1 -- 172.21.15.86:0/2401947360 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62400f000 con 0x7fe634155ae0 2024-04-20T21:25:08.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.313+0000 7fe631600640 1 -- 172.21.15.86:0/2401947360 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe624043410 con 0x7fe634155ae0 2024-04-20T21:25:08.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.313+0000 7fe631600640 1 -- 172.21.15.86:0/2401947360 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6240437e0 con 0x7fe634155ae0 2024-04-20T21:25:08.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.313+0000 7fe63ac00640 1 -- 172.21.15.86:0/2401947360 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe634146cf0 con 0x7fe634155ae0 2024-04-20T21:25:08.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.313+0000 7fe63ac00640 1 -- 172.21.15.86:0/2401947360 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6341472b0 con 0x7fe634155ae0 2024-04-20T21:25:08.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.314+0000 7fe63ac00640 1 -- 172.21.15.86:0/2401947360 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe63404fcf0 con 0x7fe634155ae0 2024-04-20T21:25:08.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.315+0000 7fe631600640 1 -- 172.21.15.86:0/2401947360 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe624041070 con 0x7fe634155ae0 2024-04-20T21:25:08.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.316+0000 7fe631600640 1 --2- 172.21.15.86:0/2401947360 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe61406c920 0x7fe61406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:08.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.316+0000 7fe631600640 1 -- 172.21.15.86:0/2401947360 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7fe6240acab0 con 0x7fe634155ae0 2024-04-20T21:25:08.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.317+0000 7fe639800640 1 --2- 172.21.15.86:0/2401947360 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe61406c920 0x7fe61406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:08.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.317+0000 7fe639800640 1 --2- 172.21.15.86:0/2401947360 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe61406c920 0x7fe61406ede0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe62c006fd0 tx=0x7fe62c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:08.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.322+0000 7fe631600640 1 -- 172.21.15.86:0/2401947360 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe624033070 con 0x7fe634155ae0 2024-04-20T21:25:08.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.524+0000 7fe63ac00640 1 -- 172.21.15.86:0/2401947360 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe6340cf560 con 0x7fe634155ae0 2024-04-20T21:25:08.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.527+0000 7fe631600640 1 -- 172.21.15.86:0/2401947360 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+16660 (secure 0 0 0) 0x7fe62407ae40 con 0x7fe634155ae0 2024-04-20T21:25:08.526 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:25:08.526 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":85,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:24:59.846179+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:24:55.964540+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-20T21:24:57.241527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:25:08.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.529+0000 7fe63ac00640 1 -- 172.21.15.86:0/2401947360 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe61406c920 msgr2=0x7fe61406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:08.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.529+0000 7fe63ac00640 1 --2- 172.21.15.86:0/2401947360 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe61406c920 0x7fe61406ede0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe62c006fd0 tx=0x7fe62c008040 comp rx=0 tx=0).stop 2024-04-20T21:25:08.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.529+0000 7fe63ac00640 1 -- 172.21.15.86:0/2401947360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe634155ae0 msgr2=0x7fe634145280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:08.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.529+0000 7fe63ac00640 1 --2- 172.21.15.86:0/2401947360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe634155ae0 0x7fe634145280 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fe624009990 tx=0x7fe624004730 comp rx=0 tx=0).stop 2024-04-20T21:25:08.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.530+0000 7fe63ac00640 1 -- 172.21.15.86:0/2401947360 shutdown_connections 2024-04-20T21:25:08.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.530+0000 7fe63ac00640 1 --2- 172.21.15.86:0/2401947360 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe61406c920 0x7fe61406ede0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:08.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.530+0000 7fe63ac00640 1 --2- 172.21.15.86:0/2401947360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe634155ae0 0x7fe634145280 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:08.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.530+0000 7fe63ac00640 1 --2- 172.21.15.86:0/2401947360 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6341465c0 0x7fe6341555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:08.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.530+0000 7fe63ac00640 1 --2- 172.21.15.86:0/2401947360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6340998c0 0x7fe6341585b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:08.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.530+0000 7fe63ac00640 1 -- 172.21.15.86:0/2401947360 >> 172.21.15.86:0/2401947360 conn(0x7fe6340d0170 msgr2=0x7fe63406e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:08.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.530+0000 7fe63ac00640 1 -- 172.21.15.86:0/2401947360 shutdown_connections 2024-04-20T21:25:08.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.530+0000 7fe63ac00640 1 -- 172.21.15.86:0/2401947360 wait complete. 2024-04-20T21:25:08.548 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:24:55.964540+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:24:57.241527+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:25:08.548 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:25:08.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.753+0000 7f3de3e00640 1 Processor -- start 2024-04-20T21:25:08.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.753+0000 7f3de3e00640 1 -- start start 2024-04-20T21:25:08.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.754+0000 7f3de3e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3de40af980 0x7f3de4147a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:08.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.754+0000 7f3de3e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3de40b3200 0x7f3de40b3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:08.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.754+0000 7f3de3e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3de406e840 con 0x7f3de4143f10 2024-04-20T21:25:08.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.755+0000 7f3de3e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f3de40b3ba0 con 0x7f3de40b3200 2024-04-20T21:25:08.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.755+0000 7f3de3e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3de406f450 con 0x7f3de40af980 2024-04-20T21:25:08.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.755+0000 7f3de2a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3de40af980 0x7f3de4147a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:08.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.755+0000 7f3de2a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3de40af980 0x7f3de4147a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:53918/0 (socket says 172.21.15.86:53918) 2024-04-20T21:25:08.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.755+0000 7f3de2a00640 1 -- 172.21.15.86:0/3383964400 learned_addr learned my addr 172.21.15.86:0/3383964400 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:08.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.756+0000 7f3de2000640 1 --2- 172.21.15.86:0/3383964400 >> v2:172.21.15.144:3300/0 conn(0x7f3de40b3200 0x7f3de40b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:08.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.756+0000 7f3de2a00640 1 -- 172.21.15.86:0/3383964400 >> v2:172.21.15.144:3300/0 conn(0x7f3de40b3200 msgr2=0x7f3de40b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:08.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.756+0000 7f3de2a00640 1 --2- 172.21.15.86:0/3383964400 >> v2:172.21.15.144:3300/0 conn(0x7f3de40b3200 0x7f3de40b3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:08.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.756+0000 7f3de2a00640 1 -- 172.21.15.86:0/3383964400 >> v1:172.21.15.144:6789/0 conn(0x7f3de4143f10 legacy=0x7f3de41443a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:08.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.756+0000 7f3de2a00640 1 -- 172.21.15.86:0/3383964400 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3de406eb20 con 0x7f3de40af980 2024-04-20T21:25:08.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.758+0000 7f3de2a00640 1 --2- 172.21.15.86:0/3383964400 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3de40af980 0x7f3de4147a70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4009870 tx=0x7f3dd402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9e3cc8ead89cebd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:08.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.758+0000 7f3de1600640 1 -- 172.21.15.86:0/3383964400 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4034070 con 0x7f3de40af980 2024-04-20T21:25:08.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.758+0000 7f3de1600640 1 -- 172.21.15.86:0/3383964400 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dd4030840 con 0x7f3de40af980 2024-04-20T21:25:08.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.758+0000 7f3de1600640 1 -- 172.21.15.86:0/3383964400 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4030c10 con 0x7f3de40af980 2024-04-20T21:25:08.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.759+0000 7f3de3e00640 1 -- 172.21.15.86:0/3383964400 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3de40af980 msgr2=0x7f3de4147a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:08.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.759+0000 7f3de3e00640 1 --2- 172.21.15.86:0/3383964400 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3de40af980 0x7f3de4147a70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4009870 tx=0x7f3dd402b980 comp rx=0 tx=0).stop 2024-04-20T21:25:08.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.759+0000 7f3de3e00640 1 -- 172.21.15.86:0/3383964400 shutdown_connections 2024-04-20T21:25:08.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.759+0000 7f3de3e00640 1 --2- 172.21.15.86:0/3383964400 >> v2:172.21.15.144:3300/0 conn(0x7f3de40b3200 0x7f3de40b3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:08.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.759+0000 7f3de3e00640 1 --2- 172.21.15.86:0/3383964400 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3de40af980 0x7f3de4147a70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:08.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.759+0000 7f3de3e00640 1 -- 172.21.15.86:0/3383964400 >> 172.21.15.86:0/3383964400 conn(0x7f3de40b1f00 msgr2=0x7f3de40b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:08.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.759+0000 7f3de3e00640 1 -- 172.21.15.86:0/3383964400 shutdown_connections 2024-04-20T21:25:08.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.759+0000 7f3de3e00640 1 -- 172.21.15.86:0/3383964400 wait complete. 2024-04-20T21:25:08.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.761+0000 7f3de3e00640 1 Processor -- start 2024-04-20T21:25:08.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.761+0000 7f3de3e00640 1 -- start start 2024-04-20T21:25:08.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.762+0000 7f3de3e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3de40af980 0x7f3de414be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:08.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.762+0000 7f3de3e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3de40b3200 0x7f3de4159020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:08.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.762+0000 7f3de3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3de4143f10 0x7f3de4159560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:08.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.762+0000 7f3de3e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3de406e3d0 con 0x7f3de4143f10 2024-04-20T21:25:08.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.762+0000 7f3de3e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3de406f0e0 con 0x7f3de40b3200 2024-04-20T21:25:08.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.762+0000 7f3de3e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3de406e550 con 0x7f3de40af980 2024-04-20T21:25:08.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.762+0000 7f3de3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3de4143f10 0x7f3de4159560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:08.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.762+0000 7f3de3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3de4143f10 0x7f3de4159560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60052/0 (socket says 172.21.15.86:60052) 2024-04-20T21:25:08.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.762+0000 7f3de3400640 1 -- 172.21.15.86:0/434799040 learned_addr learned my addr 172.21.15.86:0/434799040 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:08.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.763+0000 7f3de2a00640 1 --2- 172.21.15.86:0/434799040 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3de40af980 0x7f3de414be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:08.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.763+0000 7f3de2000640 1 --2- 172.21.15.86:0/434799040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3de40b3200 0x7f3de4159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:08.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.763+0000 7f3de3400640 1 -- 172.21.15.86:0/434799040 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3de40af980 msgr2=0x7f3de414be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:08.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.763+0000 7f3de3400640 1 --2- 172.21.15.86:0/434799040 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3de40af980 0x7f3de414be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:08.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.763+0000 7f3de3400640 1 -- 172.21.15.86:0/434799040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3de40b3200 msgr2=0x7f3de4159020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:08.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.763+0000 7f3de3400640 1 --2- 172.21.15.86:0/434799040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3de40b3200 0x7f3de4159020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:08.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.763+0000 7f3de3400640 1 -- 172.21.15.86:0/434799040 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3de4155870 con 0x7f3de4143f10 2024-04-20T21:25:08.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.763+0000 7f3de3400640 1 --2- 172.21.15.86:0/434799040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3de4143f10 0x7f3de4159560 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc00c7a0 tx=0x7f3ddc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:08.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.763+0000 7f3ddaa00640 1 -- 172.21.15.86:0/434799040 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc003b70 con 0x7f3de4143f10 2024-04-20T21:25:08.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.763+0000 7f3ddaa00640 1 -- 172.21.15.86:0/434799040 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ddc003d30 con 0x7f3de4143f10 2024-04-20T21:25:08.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.763+0000 7f3de3e00640 1 -- 172.21.15.86:0/434799040 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3de41f5100 con 0x7f3de4143f10 2024-04-20T21:25:08.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.764+0000 7f3ddaa00640 1 -- 172.21.15.86:0/434799040 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc00aa90 con 0x7f3de4143f10 2024-04-20T21:25:08.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.764+0000 7f3de3e00640 1 -- 172.21.15.86:0/434799040 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3de41f56c0 con 0x7f3de4143f10 2024-04-20T21:25:08.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.765+0000 7f3de3e00640 1 -- 172.21.15.86:0/434799040 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3de404fd90 con 0x7f3de4143f10 2024-04-20T21:25:08.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.766+0000 7f3ddaa00640 1 -- 172.21.15.86:0/434799040 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3ddc00ac30 con 0x7f3de4143f10 2024-04-20T21:25:08.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.766+0000 7f3ddaa00640 1 --2- 172.21.15.86:0/434799040 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3dc006c8f0 0x7f3dc006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:08.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.767+0000 7f3ddaa00640 1 -- 172.21.15.86:0/434799040 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7f3ddc08ba70 con 0x7f3de4143f10 2024-04-20T21:25:08.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.767+0000 7f3de2a00640 1 --2- 172.21.15.86:0/434799040 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3dc006c8f0 0x7f3dc006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:08.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.768+0000 7f3de2a00640 1 --2- 172.21.15.86:0/434799040 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3dc006c8f0 0x7f3dc006edb0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4036000 tx=0x7f3dd403b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:08.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.771+0000 7f3ddaa00640 1 -- 172.21.15.86:0/434799040 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3ddc059e90 con 0x7f3de4143f10 2024-04-20T21:25:08.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.970+0000 7f3de3e00640 1 -- 172.21.15.86:0/434799040 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3de40b1a80 con 0x7f3de4143f10 2024-04-20T21:25:08.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.971+0000 7f3ddaa00640 1 -- 172.21.15.86:0/434799040 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) ==== 93+0+10 (secure 0 0 0) 0x7f3ddc059790 con 0x7f3de4143f10 2024-04-20T21:25:08.970 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:25:08.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.973+0000 7f3de3e00640 1 -- 172.21.15.86:0/434799040 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3dc006c8f0 msgr2=0x7f3dc006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:08.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.973+0000 7f3de3e00640 1 --2- 172.21.15.86:0/434799040 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3dc006c8f0 0x7f3dc006edb0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4036000 tx=0x7f3dd403b040 comp rx=0 tx=0).stop 2024-04-20T21:25:08.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.973+0000 7f3de3e00640 1 -- 172.21.15.86:0/434799040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3de4143f10 msgr2=0x7f3de4159560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:08.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.973+0000 7f3de3e00640 1 --2- 172.21.15.86:0/434799040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3de4143f10 0x7f3de4159560 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc00c7a0 tx=0x7f3ddc00cc60 comp rx=0 tx=0).stop 2024-04-20T21:25:08.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.974+0000 7f3de3e00640 1 -- 172.21.15.86:0/434799040 shutdown_connections 2024-04-20T21:25:08.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.974+0000 7f3de3e00640 1 --2- 172.21.15.86:0/434799040 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3dc006c8f0 0x7f3dc006edb0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:08.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.974+0000 7f3de3e00640 1 --2- 172.21.15.86:0/434799040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3de4143f10 0x7f3de4159560 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:08.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.974+0000 7f3de3e00640 1 --2- 172.21.15.86:0/434799040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3de40b3200 0x7f3de4159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:08.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.974+0000 7f3de3e00640 1 --2- 172.21.15.86:0/434799040 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3de40af980 0x7f3de414be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:08.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.974+0000 7f3de3e00640 1 -- 172.21.15.86:0/434799040 >> 172.21.15.86:0/434799040 conn(0x7f3de40b1f00 msgr2=0x7f3de40b0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:08.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.974+0000 7f3de3e00640 1 -- 172.21.15.86:0/434799040 shutdown_connections 2024-04-20T21:25:08.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:08.974+0000 7f3de3e00640 1 -- 172.21.15.86:0/434799040 wait complete. 2024-04-20T21:25:08.991 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:25:09.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.192+0000 7f3ccd800640 1 Processor -- start 2024-04-20T21:25:09.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.192+0000 7f3ccd800640 1 -- start start 2024-04-20T21:25:09.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.193+0000 7f3ccd800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cc8155d30 0x7f3cc80cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:09.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.193+0000 7f3ccd800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3cc806b980 0x7f3cc806be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:09.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.193+0000 7f3ccd800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cc8156110 con 0x7f3cc80b4ba0 2024-04-20T21:25:09.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.193+0000 7f3ccd800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f3cc80b56e0 con 0x7f3cc806b980 2024-04-20T21:25:09.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.193+0000 7f3ccd800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3cc80ce460 con 0x7f3cc8155d30 2024-04-20T21:25:09.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.194+0000 7f3cc6a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cc8155d30 0x7f3cc80cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:09.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.194+0000 7f3cc7400640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f3cc80b4ba0 0x7f3cc80b4fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.86:40932/0 (socket says 172.21.15.86:40932) 2024-04-20T21:25:09.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.194+0000 7f3cc6a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cc8155d30 0x7f3cc80cf500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:53934/0 (socket says 172.21.15.86:53934) 2024-04-20T21:25:09.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.194+0000 7f3cc7400640 1 -- 172.21.15.86:0/3035024075 learned_addr learned my addr 172.21.15.86:0/3035024075 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:25:09.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.194+0000 7f3cbfe00640 1 --2- 172.21.15.86:0/3035024075 >> v2:172.21.15.144:3300/0 conn(0x7f3cc806b980 0x7f3cc806be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:09.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.195+0000 7f3cc6a00640 1 -- 172.21.15.86:0/3035024075 >> v2:172.21.15.144:3300/0 conn(0x7f3cc806b980 msgr2=0x7f3cc806be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:09.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.195+0000 7f3cc6a00640 1 --2- 172.21.15.86:0/3035024075 >> v2:172.21.15.144:3300/0 conn(0x7f3cc806b980 0x7f3cc806be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:09.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.195+0000 7f3cc6a00640 1 -- 172.21.15.86:0/3035024075 >> v1:172.21.15.144:6789/0 conn(0x7f3cc80b4ba0 legacy=0x7f3cc80b4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:09.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.195+0000 7f3cc6a00640 1 -- 172.21.15.86:0/3035024075 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cc80ce740 con 0x7f3cc8155d30 2024-04-20T21:25:09.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.196+0000 7f3cc6a00640 1 --2- 172.21.15.86:0/3035024075 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cc8155d30 0x7f3cc80cf500 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3cb80097a0 tx=0x7f3cb802b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=488f3b52c4c3ae5f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:09.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.197+0000 7f3cc6000640 1 -- 172.21.15.86:0/3035024075 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb8039070 con 0x7f3cc8155d30 2024-04-20T21:25:09.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.197+0000 7f3cc6000640 1 -- 172.21.15.86:0/3035024075 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cb80316f0 con 0x7f3cc8155d30 2024-04-20T21:25:09.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.197+0000 7f3cc6000640 1 -- 172.21.15.86:0/3035024075 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb8031ac0 con 0x7f3cc8155d30 2024-04-20T21:25:09.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.197+0000 7f3ccd800640 1 -- 172.21.15.86:0/3035024075 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cc8155d30 msgr2=0x7f3cc80cf500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:09.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.197+0000 7f3ccd800640 1 --2- 172.21.15.86:0/3035024075 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cc8155d30 0x7f3cc80cf500 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3cb80097a0 tx=0x7f3cb802b790 comp rx=0 tx=0).stop 2024-04-20T21:25:09.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.197+0000 7f3ccd800640 1 -- 172.21.15.86:0/3035024075 shutdown_connections 2024-04-20T21:25:09.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.197+0000 7f3ccd800640 1 --2- 172.21.15.86:0/3035024075 >> v2:172.21.15.144:3300/0 conn(0x7f3cc806b980 0x7f3cc806be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:09.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.197+0000 7f3ccd800640 1 --2- 172.21.15.86:0/3035024075 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cc8155d30 0x7f3cc80cf500 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:09.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.197+0000 7f3ccd800640 1 -- 172.21.15.86:0/3035024075 >> 172.21.15.86:0/3035024075 conn(0x7f3cc80b2110 msgr2=0x7f3cc80b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:09.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.197+0000 7f3ccd800640 1 -- 172.21.15.86:0/3035024075 shutdown_connections 2024-04-20T21:25:09.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.198+0000 7f3ccd800640 1 -- 172.21.15.86:0/3035024075 wait complete. 2024-04-20T21:25:09.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.199+0000 7f3ccd800640 1 Processor -- start 2024-04-20T21:25:09.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.199+0000 7f3ccd800640 1 -- start start 2024-04-20T21:25:09.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.200+0000 7f3ccd800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cc806b980 0x7f3cc814acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:09.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.200+0000 7f3ccd800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cc80b4ba0 0x7f3cc81f42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:09.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.200+0000 7f3ccd800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cc8155d30 0x7f3cc814ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:09.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.200+0000 7f3ccd800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc80cefe0 con 0x7f3cc80b4ba0 2024-04-20T21:25:09.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.200+0000 7f3ccd800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc80ced00 con 0x7f3cc806b980 2024-04-20T21:25:09.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.200+0000 7f3ccd800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3cc806ddf0 con 0x7f3cc8155d30 2024-04-20T21:25:09.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.201+0000 7f3cbfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cc80b4ba0 0x7f3cc81f42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:09.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.201+0000 7f3cbfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cc80b4ba0 0x7f3cc81f42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60058/0 (socket says 172.21.15.86:60058) 2024-04-20T21:25:09.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.201+0000 7f3cbfe00640 1 -- 172.21.15.86:0/2132008511 learned_addr learned my addr 172.21.15.86:0/2132008511 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:09.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.201+0000 7f3cc6a00640 1 --2- 172.21.15.86:0/2132008511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cc806b980 0x7f3cc814acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:09.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.201+0000 7f3cc7400640 1 --2- 172.21.15.86:0/2132008511 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cc8155d30 0x7f3cc814ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:09.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.201+0000 7f3cbfe00640 1 -- 172.21.15.86:0/2132008511 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cc8155d30 msgr2=0x7f3cc814ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:09.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.201+0000 7f3cbfe00640 1 --2- 172.21.15.86:0/2132008511 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cc8155d30 0x7f3cc814ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:09.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.201+0000 7f3cbfe00640 1 -- 172.21.15.86:0/2132008511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cc806b980 msgr2=0x7f3cc814acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:09.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.201+0000 7f3cbfe00640 1 --2- 172.21.15.86:0/2132008511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cc806b980 0x7f3cc814acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:09.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.201+0000 7f3cbfe00640 1 -- 172.21.15.86:0/2132008511 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cc81f75e0 con 0x7f3cc80b4ba0 2024-04-20T21:25:09.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.202+0000 7f3cbfe00640 1 --2- 172.21.15.86:0/2132008511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cc80b4ba0 0x7f3cc81f42b0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f3cb000d800 tx=0x7f3cb000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:09.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.202+0000 7f3cbea00640 1 -- 172.21.15.86:0/2132008511 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb000b6c0 con 0x7f3cc80b4ba0 2024-04-20T21:25:09.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.202+0000 7f3cbea00640 1 -- 172.21.15.86:0/2132008511 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cb0019070 con 0x7f3cc80b4ba0 2024-04-20T21:25:09.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.202+0000 7f3ccd800640 1 -- 172.21.15.86:0/2132008511 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cc814eb10 con 0x7f3cc80b4ba0 2024-04-20T21:25:09.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.202+0000 7f3cbea00640 1 -- 172.21.15.86:0/2132008511 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb000ba80 con 0x7f3cc80b4ba0 2024-04-20T21:25:09.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.202+0000 7f3ccd800640 1 -- 172.21.15.86:0/2132008511 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cc814f0d0 con 0x7f3cc80b4ba0 2024-04-20T21:25:09.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.203+0000 7f3ccd800640 1 -- 172.21.15.86:0/2132008511 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cc80689e0 con 0x7f3cc80b4ba0 2024-04-20T21:25:09.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.204+0000 7f3cbea00640 1 -- 172.21.15.86:0/2132008511 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3cb000bc20 con 0x7f3cc80b4ba0 2024-04-20T21:25:09.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.205+0000 7f3cbea00640 1 --2- 172.21.15.86:0/2132008511 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3ca406c8f0 0x7f3ca406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:09.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.205+0000 7f3cbea00640 1 -- 172.21.15.86:0/2132008511 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7f3cb008c0e0 con 0x7f3cc80b4ba0 2024-04-20T21:25:09.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.205+0000 7f3cc6a00640 1 --2- 172.21.15.86:0/2132008511 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3ca406c8f0 0x7f3ca406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:09.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.206+0000 7f3cc6a00640 1 --2- 172.21.15.86:0/2132008511 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3ca406c8f0 0x7f3ca406edb0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8036000 tx=0x7f3cb8008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:09.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.210+0000 7f3cbea00640 1 -- 172.21.15.86:0/2132008511 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3cb001c360 con 0x7f3cc80b4ba0 2024-04-20T21:25:09.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.410+0000 7f3ccd800640 1 -- 172.21.15.86:0/2132008511 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3cc814b1f0 con 0x7f3cc80b4ba0 2024-04-20T21:25:09.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.411+0000 7f3cbea00640 1 -- 172.21.15.86:0/2132008511 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) ==== 104+0+11 (secure 0 0 0) 0x7f3cb005a470 con 0x7f3cc80b4ba0 2024-04-20T21:25:09.410 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-20T21:25:09.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.413+0000 7f3ccd800640 1 -- 172.21.15.86:0/2132008511 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3ca406c8f0 msgr2=0x7f3ca406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:09.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.413+0000 7f3ccd800640 1 --2- 172.21.15.86:0/2132008511 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3ca406c8f0 0x7f3ca406edb0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8036000 tx=0x7f3cb8008030 comp rx=0 tx=0).stop 2024-04-20T21:25:09.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.413+0000 7f3ccd800640 1 -- 172.21.15.86:0/2132008511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cc80b4ba0 msgr2=0x7f3cc81f42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:09.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.413+0000 7f3ccd800640 1 --2- 172.21.15.86:0/2132008511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cc80b4ba0 0x7f3cc81f42b0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f3cb000d800 tx=0x7f3cb000dcc0 comp rx=0 tx=0).stop 2024-04-20T21:25:09.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.414+0000 7f3ccd800640 1 -- 172.21.15.86:0/2132008511 shutdown_connections 2024-04-20T21:25:09.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.414+0000 7f3ccd800640 1 --2- 172.21.15.86:0/2132008511 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3ca406c8f0 0x7f3ca406edb0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:09.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.414+0000 7f3ccd800640 1 --2- 172.21.15.86:0/2132008511 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3cc8155d30 0x7f3cc814ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:09.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.414+0000 7f3ccd800640 1 --2- 172.21.15.86:0/2132008511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3cc80b4ba0 0x7f3cc81f42b0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:09.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.414+0000 7f3ccd800640 1 --2- 172.21.15.86:0/2132008511 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3cc806b980 0x7f3cc814acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:09.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.414+0000 7f3ccd800640 1 -- 172.21.15.86:0/2132008511 >> 172.21.15.86:0/2132008511 conn(0x7f3cc80b2110 msgr2=0x7f3cc806be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:09.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.414+0000 7f3ccd800640 1 -- 172.21.15.86:0/2132008511 shutdown_connections 2024-04-20T21:25:09.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.414+0000 7f3ccd800640 1 -- 172.21.15.86:0/2132008511 wait complete. 2024-04-20T21:25:09.430 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:25:09.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.632+0000 7fe947200640 1 Processor -- start 2024-04-20T21:25:09.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.633+0000 7fe947200640 1 -- start start 2024-04-20T21:25:09.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.634+0000 7fe947200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe94815e870 0x7fe94806ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:09.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.634+0000 7fe947200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9480d4310 0x7fe9480d4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:09.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.634+0000 7fe947200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9480d4cb0 con 0x7fe9480cfb80 2024-04-20T21:25:09.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.634+0000 7fe947200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe9480cf150 con 0x7fe94815e870 2024-04-20T21:25:09.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.634+0000 7fe947200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe9480d35b0 con 0x7fe9480d4310 2024-04-20T21:25:09.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.635+0000 7fe945e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe94815e870 0x7fe94806ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:09.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.635+0000 7fe946800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe9480cfb80 0x7fe9480d0010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57466/0 (socket says 172.21.15.86:57466) 2024-04-20T21:25:09.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.635+0000 7fe946800640 1 -- 172.21.15.86:0/938758004 learned_addr learned my addr 172.21.15.86:0/938758004 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:25:09.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.635+0000 7fe945400640 1 --2- 172.21.15.86:0/938758004 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9480d4310 0x7fe9480d4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:09.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.635+0000 7fe945e00640 1 -- 172.21.15.86:0/938758004 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9480d4310 msgr2=0x7fe9480d4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:09.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.635+0000 7fe945e00640 1 --2- 172.21.15.86:0/938758004 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9480d4310 0x7fe9480d4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:09.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.635+0000 7fe945e00640 1 -- 172.21.15.86:0/938758004 >> v1:172.21.15.86:6789/0 conn(0x7fe9480cfb80 legacy=0x7fe9480d0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:09.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.635+0000 7fe945e00640 1 -- 172.21.15.86:0/938758004 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9480d2ff0 con 0x7fe94815e870 2024-04-20T21:25:09.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.635+0000 7fe944a00640 1 -- 172.21.15.86:0/938758004 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 875731070 0 0) 0x7fe9480d4cb0 con 0x7fe9480cfb80 2024-04-20T21:25:09.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.637+0000 7fe945e00640 1 --2- 172.21.15.86:0/938758004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe94815e870 0x7fe94806ce80 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fe9380097a0 tx=0x7fe93802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57702857ab15259 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:09.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.637+0000 7fe944a00640 1 -- 172.21.15.86:0/938758004 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe938035020 con 0x7fe94815e870 2024-04-20T21:25:09.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.637+0000 7fe944a00640 1 -- 172.21.15.86:0/938758004 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe938031810 con 0x7fe94815e870 2024-04-20T21:25:09.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.637+0000 7fe944a00640 1 -- 172.21.15.86:0/938758004 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe938031be0 con 0x7fe94815e870 2024-04-20T21:25:09.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.637+0000 7fe947200640 1 -- 172.21.15.86:0/938758004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe94815e870 msgr2=0x7fe94806ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:09.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.637+0000 7fe947200640 1 --2- 172.21.15.86:0/938758004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe94815e870 0x7fe94806ce80 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fe9380097a0 tx=0x7fe93802b8b0 comp rx=0 tx=0).stop 2024-04-20T21:25:09.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.638+0000 7fe947200640 1 -- 172.21.15.86:0/938758004 shutdown_connections 2024-04-20T21:25:09.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.638+0000 7fe947200640 1 --2- 172.21.15.86:0/938758004 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9480d4310 0x7fe9480d4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:09.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.638+0000 7fe947200640 1 --2- 172.21.15.86:0/938758004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe94815e870 0x7fe94806ce80 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:09.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.638+0000 7fe947200640 1 -- 172.21.15.86:0/938758004 >> 172.21.15.86:0/938758004 conn(0x7fe94809d9c0 msgr2=0x7fe94808bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:09.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.638+0000 7fe947200640 1 -- 172.21.15.86:0/938758004 shutdown_connections 2024-04-20T21:25:09.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.638+0000 7fe947200640 1 -- 172.21.15.86:0/938758004 wait complete. 2024-04-20T21:25:09.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.640+0000 7fe947200640 1 Processor -- start 2024-04-20T21:25:09.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.640+0000 7fe947200640 1 -- start start 2024-04-20T21:25:09.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.641+0000 7fe947200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9480cfb80 0x7fe9480cb750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:09.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.641+0000 7fe947200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9480d4310 0x7fe9481f84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:09.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.641+0000 7fe947200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe94815e870 0x7fe948154e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:09.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.641+0000 7fe947200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe9480cf660 con 0x7fe9480cfb80 2024-04-20T21:25:09.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.641+0000 7fe947200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe9480cf380 con 0x7fe9480d4310 2024-04-20T21:25:09.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.641+0000 7fe947200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe94806d3c0 con 0x7fe94815e870 2024-04-20T21:25:09.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.641+0000 7fe945e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9480cfb80 0x7fe9480cb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:09.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.642+0000 7fe945e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9480cfb80 0x7fe9480cb750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60076/0 (socket says 172.21.15.86:60076) 2024-04-20T21:25:09.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.642+0000 7fe945e00640 1 -- 172.21.15.86:0/3836155354 learned_addr learned my addr 172.21.15.86:0/3836155354 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:09.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.642+0000 7fe945400640 1 --2- 172.21.15.86:0/3836155354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9480d4310 0x7fe9481f84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:09.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.642+0000 7fe946800640 1 --2- 172.21.15.86:0/3836155354 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe94815e870 0x7fe948154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:09.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.642+0000 7fe945e00640 1 -- 172.21.15.86:0/3836155354 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe94815e870 msgr2=0x7fe948154e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:09.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.642+0000 7fe945e00640 1 --2- 172.21.15.86:0/3836155354 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe94815e870 0x7fe948154e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:09.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.642+0000 7fe945e00640 1 -- 172.21.15.86:0/3836155354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9480d4310 msgr2=0x7fe9481f84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:09.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.642+0000 7fe945e00640 1 --2- 172.21.15.86:0/3836155354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9480d4310 0x7fe9481f84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:09.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.642+0000 7fe945e00640 1 -- 172.21.15.86:0/3836155354 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9480cd2b0 con 0x7fe9480cfb80 2024-04-20T21:25:09.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.642+0000 7fe945400640 1 --2- 172.21.15.86:0/3836155354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9480d4310 0x7fe9481f84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:25:09.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.642+0000 7fe945e00640 1 --2- 172.21.15.86:0/3836155354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9480cfb80 0x7fe9480cb750 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fe938034040 tx=0x7fe9380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:09.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.643+0000 7fe93e000640 1 -- 172.21.15.86:0/3836155354 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe938040070 con 0x7fe9480cfb80 2024-04-20T21:25:09.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.643+0000 7fe93e000640 1 -- 172.21.15.86:0/3836155354 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe938039cf0 con 0x7fe9480cfb80 2024-04-20T21:25:09.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.643+0000 7fe93e000640 1 -- 172.21.15.86:0/3836155354 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe938035040 con 0x7fe9480cfb80 2024-04-20T21:25:09.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.643+0000 7fe947200640 1 -- 172.21.15.86:0/3836155354 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9480cd530 con 0x7fe9480cfb80 2024-04-20T21:25:09.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.643+0000 7fe947200640 1 -- 172.21.15.86:0/3836155354 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9480cdaf0 con 0x7fe9480cfb80 2024-04-20T21:25:09.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.644+0000 7fe947200640 1 -- 172.21.15.86:0/3836155354 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe94804fd90 con 0x7fe9480cfb80 2024-04-20T21:25:09.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.645+0000 7fe93e000640 1 -- 172.21.15.86:0/3836155354 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe938038020 con 0x7fe9480cfb80 2024-04-20T21:25:09.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.646+0000 7fe93e000640 1 --2- 172.21.15.86:0/3836155354 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe92406c8f0 0x7fe92406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:09.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.647+0000 7fe93e000640 1 -- 172.21.15.86:0/3836155354 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7fe9380ad7e0 con 0x7fe9480cfb80 2024-04-20T21:25:09.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.647+0000 7fe945400640 1 --2- 172.21.15.86:0/3836155354 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe92406c8f0 0x7fe92406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:09.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.647+0000 7fe945400640 1 --2- 172.21.15.86:0/3836155354 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe92406c8f0 0x7fe92406edb0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fe948161c50 tx=0x7fe93000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:09.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.651+0000 7fe93e000640 1 -- 172.21.15.86:0/3836155354 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe9480cd2b0 con 0x7fe9480cfb80 2024-04-20T21:25:09.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.848+0000 7fe947200640 1 -- 172.21.15.86:0/3836155354 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe948068e70 con 0x7fe9480cfb80 2024-04-20T21:25:09.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.849+0000 7fe93e000640 1 -- 172.21.15.86:0/3836155354 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) ==== 100+0+11 (secure 0 0 0) 0x7fe93807bc10 con 0x7fe9480cfb80 2024-04-20T21:25:09.848 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-20T21:25:09.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.852+0000 7fe947200640 1 -- 172.21.15.86:0/3836155354 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe92406c8f0 msgr2=0x7fe92406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:09.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.852+0000 7fe947200640 1 --2- 172.21.15.86:0/3836155354 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe92406c8f0 0x7fe92406edb0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fe948161c50 tx=0x7fe93000b040 comp rx=0 tx=0).stop 2024-04-20T21:25:09.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.852+0000 7fe947200640 1 -- 172.21.15.86:0/3836155354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9480cfb80 msgr2=0x7fe9480cb750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:09.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.852+0000 7fe947200640 1 --2- 172.21.15.86:0/3836155354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9480cfb80 0x7fe9480cb750 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fe938034040 tx=0x7fe9380046c0 comp rx=0 tx=0).stop 2024-04-20T21:25:09.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.852+0000 7fe947200640 1 -- 172.21.15.86:0/3836155354 shutdown_connections 2024-04-20T21:25:09.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.852+0000 7fe947200640 1 --2- 172.21.15.86:0/3836155354 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe92406c8f0 0x7fe92406edb0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:09.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.852+0000 7fe947200640 1 --2- 172.21.15.86:0/3836155354 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe94815e870 0x7fe948154e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:09.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.852+0000 7fe947200640 1 --2- 172.21.15.86:0/3836155354 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9480d4310 0x7fe9481f84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:09.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.852+0000 7fe947200640 1 --2- 172.21.15.86:0/3836155354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9480cfb80 0x7fe9480cb750 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:09.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.852+0000 7fe947200640 1 -- 172.21.15.86:0/3836155354 >> 172.21.15.86:0/3836155354 conn(0x7fe94809d9c0 msgr2=0x7fe9480b60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:09.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.852+0000 7fe947200640 1 -- 172.21.15.86:0/3836155354 shutdown_connections 2024-04-20T21:25:09.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:09.852+0000 7fe947200640 1 -- 172.21.15.86:0/3836155354 wait complete. 2024-04-20T21:25:09.869 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T21:25:10.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.066+0000 7f3534e00640 1 Processor -- start 2024-04-20T21:25:10.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.066+0000 7f3534e00640 1 -- start start 2024-04-20T21:25:10.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.067+0000 7f3534e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f353015e9b0 0x7f353015f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:10.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.067+0000 7f3534e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f353006e1b0 0x7f3530071b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:10.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.067+0000 7f3534e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35300b49a0 con 0x7f35300720b0 2024-04-20T21:25:10.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.067+0000 7f3534e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f353006dd20 con 0x7f353015e9b0 2024-04-20T21:25:10.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.067+0000 7f3534e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f35300b4c40 con 0x7f353006e1b0 2024-04-20T21:25:10.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.067+0000 7f352f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f353015e9b0 0x7f353015f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:10.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.067+0000 7f352f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f353015e9b0 0x7f353015f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60090/0 (socket says 172.21.15.86:60090) 2024-04-20T21:25:10.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.067+0000 7f352f400640 1 -- 172.21.15.86:0/3164779371 learned_addr learned my addr 172.21.15.86:0/3164779371 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:10.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.068+0000 7f3527e00640 1 --2- 172.21.15.86:0/3164779371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f353006e1b0 0x7f3530071b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:10.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.068+0000 7f352f400640 1 -- 172.21.15.86:0/3164779371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f353006e1b0 msgr2=0x7f3530071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:10.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.068+0000 7f352f400640 1 --2- 172.21.15.86:0/3164779371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f353006e1b0 0x7f3530071b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:10.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.068+0000 7f352f400640 1 -- 172.21.15.86:0/3164779371 >> v1:172.21.15.144:6789/0 conn(0x7f35300720b0 legacy=0x7f353006e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:10.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.068+0000 7f352f400640 1 -- 172.21.15.86:0/3164779371 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35300b4f20 con 0x7f353015e9b0 2024-04-20T21:25:10.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.069+0000 7f352f400640 1 --2- 172.21.15.86:0/3164779371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f353015e9b0 0x7f353015f920 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f3520009870 tx=0x7f352002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b31d10ef2613f700 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:10.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.070+0000 7f352ea00640 1 -- 172.21.15.86:0/3164779371 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3520035020 con 0x7f353015e9b0 2024-04-20T21:25:10.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.070+0000 7f352ea00640 1 -- 172.21.15.86:0/3164779371 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3520031640 con 0x7f353015e9b0 2024-04-20T21:25:10.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.070+0000 7f352ea00640 1 -- 172.21.15.86:0/3164779371 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352002d040 con 0x7f353015e9b0 2024-04-20T21:25:10.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.070+0000 7f3534e00640 1 -- 172.21.15.86:0/3164779371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f353015e9b0 msgr2=0x7f353015f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:10.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.070+0000 7f3534e00640 1 --2- 172.21.15.86:0/3164779371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f353015e9b0 0x7f353015f920 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f3520009870 tx=0x7f352002b8f0 comp rx=0 tx=0).stop 2024-04-20T21:25:10.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.071+0000 7f3534e00640 1 -- 172.21.15.86:0/3164779371 shutdown_connections 2024-04-20T21:25:10.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.071+0000 7f3534e00640 1 --2- 172.21.15.86:0/3164779371 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f353006e1b0 0x7f3530071b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:10.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.071+0000 7f3534e00640 1 --2- 172.21.15.86:0/3164779371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f353015e9b0 0x7f353015f920 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:10.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.071+0000 7f3534e00640 1 -- 172.21.15.86:0/3164779371 >> 172.21.15.86:0/3164779371 conn(0x7f353006bb60 msgr2=0x7f353006bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:10.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.071+0000 7f3534e00640 1 -- 172.21.15.86:0/3164779371 shutdown_connections 2024-04-20T21:25:10.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.071+0000 7f3534e00640 1 -- 172.21.15.86:0/3164779371 wait complete. 2024-04-20T21:25:10.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.073+0000 7f3534e00640 1 Processor -- start 2024-04-20T21:25:10.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.073+0000 7f3534e00640 1 -- start start 2024-04-20T21:25:10.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.074+0000 7f3534e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f353006e1b0 0x7f35301e4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:10.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.074+0000 7f3534e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35300720b0 0x7f35300b0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:10.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.074+0000 7f3534e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f353015e9b0 0x7f35300b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:10.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.074+0000 7f3534e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3530072540 con 0x7f353015e9b0 2024-04-20T21:25:10.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.074+0000 7f3534e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f353015f400 con 0x7f353006e1b0 2024-04-20T21:25:10.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.074+0000 7f3534e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f353015ed90 con 0x7f35300720b0 2024-04-20T21:25:10.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.074+0000 7f352fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f353015e9b0 0x7f35300b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:10.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.074+0000 7f352fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f353015e9b0 0x7f35300b0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60098/0 (socket says 172.21.15.86:60098) 2024-04-20T21:25:10.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.074+0000 7f352fe00640 1 -- 172.21.15.86:0/3842790896 learned_addr learned my addr 172.21.15.86:0/3842790896 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:10.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.075+0000 7f3527e00640 1 --2- 172.21.15.86:0/3842790896 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35300720b0 0x7f35300b0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:10.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.075+0000 7f352f400640 1 --2- 172.21.15.86:0/3842790896 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f353006e1b0 0x7f35301e4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:10.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.075+0000 7f352fe00640 1 -- 172.21.15.86:0/3842790896 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35300720b0 msgr2=0x7f35300b0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:10.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.075+0000 7f352fe00640 1 --2- 172.21.15.86:0/3842790896 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35300720b0 0x7f35300b0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:10.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.075+0000 7f352fe00640 1 -- 172.21.15.86:0/3842790896 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f353006e1b0 msgr2=0x7f35301e4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:10.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.075+0000 7f352fe00640 1 --2- 172.21.15.86:0/3842790896 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f353006e1b0 0x7f35301e4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:10.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.075+0000 7f352fe00640 1 -- 172.21.15.86:0/3842790896 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35301e5800 con 0x7f353015e9b0 2024-04-20T21:25:10.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.075+0000 7f352f400640 1 --2- 172.21.15.86:0/3842790896 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f353006e1b0 0x7f35301e4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:25:10.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.075+0000 7f3527e00640 1 --2- 172.21.15.86:0/3842790896 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35300720b0 0x7f35300b0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:25:10.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.075+0000 7f352fe00640 1 --2- 172.21.15.86:0/3842790896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f353015e9b0 0x7f35300b0940 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f352800c780 tx=0x7f352800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:10.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.075+0000 7f3527400640 1 -- 172.21.15.86:0/3842790896 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3528004b60 con 0x7f353015e9b0 2024-04-20T21:25:10.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.075+0000 7f3527400640 1 -- 172.21.15.86:0/3842790896 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3528004d00 con 0x7f353015e9b0 2024-04-20T21:25:10.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.075+0000 7f3527400640 1 -- 172.21.15.86:0/3842790896 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352800aa90 con 0x7f353015e9b0 2024-04-20T21:25:10.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.076+0000 7f3534e00640 1 -- 172.21.15.86:0/3842790896 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35301e5ae0 con 0x7f353015e9b0 2024-04-20T21:25:10.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.076+0000 7f3534e00640 1 -- 172.21.15.86:0/3842790896 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35301f6040 con 0x7f353015e9b0 2024-04-20T21:25:10.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.077+0000 7f3534e00640 1 -- 172.21.15.86:0/3842790896 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35301e6000 con 0x7f353015e9b0 2024-04-20T21:25:10.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.078+0000 7f3527400640 1 -- 172.21.15.86:0/3842790896 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f352800ac30 con 0x7f353015e9b0 2024-04-20T21:25:10.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.079+0000 7f3527400640 1 --2- 172.21.15.86:0/3842790896 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f350c06c8f0 0x7f350c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:10.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.080+0000 7f3527400640 1 -- 172.21.15.86:0/3842790896 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7f352808be60 con 0x7f353015e9b0 2024-04-20T21:25:10.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.080+0000 7f352f400640 1 --2- 172.21.15.86:0/3842790896 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f350c06c8f0 0x7f350c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:10.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.080+0000 7f352f400640 1 --2- 172.21.15.86:0/3842790896 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f350c06c8f0 0x7f350c06edb0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f3520036000 tx=0x7f352003c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:10.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.085+0000 7f3527400640 1 -- 172.21.15.86:0/3842790896 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f35301e6000 con 0x7f353015e9b0 2024-04-20T21:25:10.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.300+0000 7f3534e00640 1 -- 172.21.15.86:0/3842790896 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f35301f6240 con 0x7f353015e9b0 2024-04-20T21:25:10.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.301+0000 7f3527400640 1 -- 172.21.15.86:0/3842790896 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v85) ==== 80+0+0 (secure 0 0 0) 0x7f35301f6240 con 0x7f353015e9b0 2024-04-20T21:25:10.300 INFO:teuthology.orchestra.run.smithi086.stderr:listed 0 entries 2024-04-20T21:25:10.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.304+0000 7f3534e00640 1 -- 172.21.15.86:0/3842790896 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f350c06c8f0 msgr2=0x7f350c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:10.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.304+0000 7f3534e00640 1 --2- 172.21.15.86:0/3842790896 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f350c06c8f0 0x7f350c06edb0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f3520036000 tx=0x7f352003c040 comp rx=0 tx=0).stop 2024-04-20T21:25:10.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.304+0000 7f3534e00640 1 -- 172.21.15.86:0/3842790896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f353015e9b0 msgr2=0x7f35300b0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:10.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.304+0000 7f3534e00640 1 --2- 172.21.15.86:0/3842790896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f353015e9b0 0x7f35300b0940 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f352800c780 tx=0x7f352800cc40 comp rx=0 tx=0).stop 2024-04-20T21:25:10.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.304+0000 7f3534e00640 1 -- 172.21.15.86:0/3842790896 shutdown_connections 2024-04-20T21:25:10.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.304+0000 7f3534e00640 1 --2- 172.21.15.86:0/3842790896 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f350c06c8f0 0x7f350c06edb0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:10.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.304+0000 7f3534e00640 1 --2- 172.21.15.86:0/3842790896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f353015e9b0 0x7f35300b0940 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:10.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.304+0000 7f3534e00640 1 --2- 172.21.15.86:0/3842790896 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f35300720b0 0x7f35300b0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:10.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.304+0000 7f3534e00640 1 --2- 172.21.15.86:0/3842790896 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f353006e1b0 0x7f35301e4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:10.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.304+0000 7f3534e00640 1 -- 172.21.15.86:0/3842790896 >> 172.21.15.86:0/3842790896 conn(0x7f353006bb60 msgr2=0x7f35300acfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:10.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.304+0000 7f3534e00640 1 -- 172.21.15.86:0/3842790896 shutdown_connections 2024-04-20T21:25:10.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.304+0000 7f3534e00640 1 -- 172.21.15.86:0/3842790896 wait complete. 2024-04-20T21:25:10.321 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:25:10.321 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:25:10.369 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:25:10.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.599+0000 7fb07e400640 1 Processor -- start 2024-04-20T21:25:10.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.600+0000 7fb07e400640 1 -- start start 2024-04-20T21:25:10.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.601+0000 7fb07e400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb07809ae40 0x7fb0780b3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:10.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.601+0000 7fb07e400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb07815efd0 con 0x7fb07809a9e0 2024-04-20T21:25:10.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.602+0000 7fb07e400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0780b1dc0 con 0x7fb07815ebf0 2024-04-20T21:25:10.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.602+0000 7fb07e400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fb0780b20a0 con 0x7fb07809ae40 2024-04-20T21:25:10.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.602+0000 7fb076a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fb07809a9e0 0x7fb0780b2e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57478/0 (socket says 172.21.15.86:57478) 2024-04-20T21:25:10.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.602+0000 7fb076a00640 1 -- 172.21.15.86:0/2722940061 learned_addr learned my addr 172.21.15.86:0/2722940061 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:25:10.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.602+0000 7fb077e00640 1 --2- 172.21.15.86:0/2722940061 >> v2:172.21.15.144:3300/0 conn(0x7fb07809ae40 0x7fb0780b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:10.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.602+0000 7fb076000640 1 -- 172.21.15.86:0/2722940061 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 136805148 0 0) 0x7fb07815efd0 con 0x7fb07809a9e0 2024-04-20T21:25:10.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.602+0000 7fb076000640 1 -- 172.21.15.86:0/2722940061 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb05c003520 con 0x7fb07809a9e0 2024-04-20T21:25:10.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.603+0000 7fb076000640 1 -- 172.21.15.86:0/2722940061 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1580844383 0 0) 0x7fb05c003520 con 0x7fb07809a9e0 2024-04-20T21:25:10.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.603+0000 7fb076000640 1 -- 172.21.15.86:0/2722940061 >> v2:172.21.15.144:3300/0 conn(0x7fb07809ae40 msgr2=0x7fb0780b3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:10.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.603+0000 7fb076000640 1 --2- 172.21.15.86:0/2722940061 >> v2:172.21.15.144:3300/0 conn(0x7fb07809ae40 0x7fb0780b3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:10.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.603+0000 7fb076000640 1 -- 172.21.15.86:0/2722940061 >> v1:172.21.15.144:6789/0 conn(0x7fb07815ebf0 legacy=0x7fb0780d12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:10.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.603+0000 7fb076000640 1 -- 172.21.15.86:0/2722940061 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0780b2380 con 0x7fb07809a9e0 2024-04-20T21:25:10.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.603+0000 7fb076000640 1 -- 172.21.15.86:0/2722940061 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fb0680019f0 con 0x7fb07809a9e0 2024-04-20T21:25:10.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.603+0000 7fb076000640 1 -- 172.21.15.86:0/2722940061 >> v1:172.21.15.86:6789/0 conn(0x7fb07809a9e0 legacy=0x7fb0780b2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:10.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.603+0000 7fb076000640 1 --2- 172.21.15.86:0/2722940061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb05c003a00 0x7fb05c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:10.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.603+0000 7fb076000640 1 --2- 172.21.15.86:0/2722940061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb05c004800 0x7fb05c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:10.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.603+0000 7fb076000640 1 --2- 172.21.15.86:0/2722940061 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb05c0051c0 0x7fb05c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:10.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.603+0000 7fb076000640 1 -- 172.21.15.86:0/2722940061 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb0780b20a0 con 0x7fb05c003a00 2024-04-20T21:25:10.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.603+0000 7fb076000640 1 -- 172.21.15.86:0/2722940061 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb05c00c0c0 con 0x7fb05c004800 2024-04-20T21:25:10.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.603+0000 7fb076000640 1 -- 172.21.15.86:0/2722940061 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb05c00c2c0 con 0x7fb05c0051c0 2024-04-20T21:25:10.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.603+0000 7fb077400640 1 --2- 172.21.15.86:0/2722940061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb05c003a00 0x7fb05c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:10.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.603+0000 7fb076a00640 1 --2- 172.21.15.86:0/2722940061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb05c004800 0x7fb05c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:10.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.603+0000 7fb077e00640 1 --2- 172.21.15.86:0/2722940061 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb05c0051c0 0x7fb05c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:10.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.604+0000 7fb077400640 1 -- 172.21.15.86:0/2722940061 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb05c0051c0 msgr2=0x7fb05c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:10.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.604+0000 7fb077400640 1 --2- 172.21.15.86:0/2722940061 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb05c0051c0 0x7fb05c00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:10.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.604+0000 7fb077400640 1 -- 172.21.15.86:0/2722940061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb05c004800 msgr2=0x7fb05c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:10.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.604+0000 7fb077400640 1 --2- 172.21.15.86:0/2722940061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb05c004800 0x7fb05c004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:10.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.604+0000 7fb077400640 1 -- 172.21.15.86:0/2722940061 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb05c003520 con 0x7fb05c003a00 2024-04-20T21:25:10.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.604+0000 7fb077e00640 1 -- 172.21.15.86:0/2722940061 reap_dead start 2024-04-20T21:25:10.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.605+0000 7fb077400640 1 --2- 172.21.15.86:0/2722940061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb05c003a00 0x7fb05c003e20 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fb060001750 tx=0x7fb05c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=399e65a135cbc396 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:10.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.605+0000 7fb076000640 1 -- 172.21.15.86:0/2722940061 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb060001370 con 0x7fb05c003a00 2024-04-20T21:25:10.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.605+0000 7fb076000640 1 -- 172.21.15.86:0/2722940061 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0600052d0 con 0x7fb05c003a00 2024-04-20T21:25:10.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.605+0000 7fb076000640 1 -- 172.21.15.86:0/2722940061 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb060003830 con 0x7fb05c003a00 2024-04-20T21:25:10.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.605+0000 7fb07e400640 1 -- 172.21.15.86:0/2722940061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb05c003a00 msgr2=0x7fb05c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:10.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.605+0000 7fb07e400640 1 --2- 172.21.15.86:0/2722940061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb05c003a00 0x7fb05c003e20 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fb060001750 tx=0x7fb05c005830 comp rx=0 tx=0).stop 2024-04-20T21:25:10.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.606+0000 7fb07e400640 1 -- 172.21.15.86:0/2722940061 shutdown_connections 2024-04-20T21:25:10.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.606+0000 7fb07e400640 1 --2- 172.21.15.86:0/2722940061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb05c003a00 0x7fb05c003e20 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:10.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.606+0000 7fb07e400640 1 -- 172.21.15.86:0/2722940061 >> 172.21.15.86:0/2722940061 conn(0x7fb0780b3ff0 msgr2=0x7fb0780b4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:10.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.606+0000 7fb07e400640 1 -- 172.21.15.86:0/2722940061 shutdown_connections 2024-04-20T21:25:10.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.606+0000 7fb07e400640 1 -- 172.21.15.86:0/2722940061 wait complete. 2024-04-20T21:25:10.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.608+0000 7fb07e400640 1 Processor -- start 2024-04-20T21:25:10.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.608+0000 7fb07e400640 1 -- start start 2024-04-20T21:25:10.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.609+0000 7fb07e400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb05c003a00 0x7fb0780b7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:10.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.609+0000 7fb07e400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb078160600 0x7fb078153880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:10.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.609+0000 7fb07e400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0780c8000 0x7fb0780c8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:10.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.609+0000 7fb07e400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb07809b2d0 con 0x7fb0780c8000 2024-04-20T21:25:10.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.609+0000 7fb07e400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb0780b2940 con 0x7fb05c003a00 2024-04-20T21:25:10.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.609+0000 7fb07e400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb0780ccb50 con 0x7fb078160600 2024-04-20T21:25:10.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.609+0000 7fb077e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0780c8000 0x7fb0780c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:10.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.609+0000 7fb077e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0780c8000 0x7fb0780c8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60108/0 (socket says 172.21.15.86:60108) 2024-04-20T21:25:10.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.609+0000 7fb077e00640 1 -- 172.21.15.86:0/3957495532 learned_addr learned my addr 172.21.15.86:0/3957495532 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:10.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.609+0000 7fb076a00640 1 --2- 172.21.15.86:0/3957495532 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb078160600 0x7fb078153880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:10.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.609+0000 7fb077400640 1 --2- 172.21.15.86:0/3957495532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb05c003a00 0x7fb0780b7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:10.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.609+0000 7fb077e00640 1 -- 172.21.15.86:0/3957495532 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb078160600 msgr2=0x7fb078153880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:10.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.609+0000 7fb077e00640 1 --2- 172.21.15.86:0/3957495532 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb078160600 0x7fb078153880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:10.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.609+0000 7fb077e00640 1 -- 172.21.15.86:0/3957495532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb05c003a00 msgr2=0x7fb0780b7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:10.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.609+0000 7fb077e00640 1 --2- 172.21.15.86:0/3957495532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb05c003a00 0x7fb0780b7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:10.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.609+0000 7fb077e00640 1 -- 172.21.15.86:0/3957495532 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb078153dc0 con 0x7fb0780c8000 2024-04-20T21:25:10.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.609+0000 7fb076a00640 1 --2- 172.21.15.86:0/3957495532 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb078160600 0x7fb078153880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:25:10.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.609+0000 7fb077400640 1 --2- 172.21.15.86:0/3957495532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb05c003a00 0x7fb0780b7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:25:10.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.610+0000 7fb077e00640 1 --2- 172.21.15.86:0/3957495532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0780c8000 0x7fb0780c8480 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fb0780b1bb0 tx=0x7fb070010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:10.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.610+0000 7fb06f400640 1 -- 172.21.15.86:0/3957495532 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07000f040 con 0x7fb0780c8000 2024-04-20T21:25:10.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.610+0000 7fb06f400640 1 -- 172.21.15.86:0/3957495532 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb07000e890 con 0x7fb0780c8000 2024-04-20T21:25:10.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.610+0000 7fb07e400640 1 -- 172.21.15.86:0/3957495532 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb07815f560 con 0x7fb0780c8000 2024-04-20T21:25:10.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.610+0000 7fb06f400640 1 -- 172.21.15.86:0/3957495532 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07000ec80 con 0x7fb0780c8000 2024-04-20T21:25:10.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.610+0000 7fb07e400640 1 -- 172.21.15.86:0/3957495532 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb07815fb20 con 0x7fb0780c8000 2024-04-20T21:25:10.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.611+0000 7fb07e400640 1 -- 172.21.15.86:0/3957495532 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0780c89c0 con 0x7fb0780c8000 2024-04-20T21:25:10.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.612+0000 7fb06f400640 1 -- 172.21.15.86:0/3957495532 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb07001c400 con 0x7fb0780c8000 2024-04-20T21:25:10.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.614+0000 7fb06f400640 1 --2- 172.21.15.86:0/3957495532 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb05406c920 0x7fb05406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:10.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.614+0000 7fb06f400640 1 -- 172.21.15.86:0/3957495532 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7fb07008bd90 con 0x7fb0780c8000 2024-04-20T21:25:10.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.614+0000 7fb077400640 1 --2- 172.21.15.86:0/3957495532 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb05406c920 0x7fb05406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:10.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.615+0000 7fb077400640 1 --2- 172.21.15.86:0/3957495532 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb05406c920 0x7fb05406ede0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fb060033000 tx=0x7fb060009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:10.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.618+0000 7fb06f400640 1 -- 172.21.15.86:0/3957495532 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb070018030 con 0x7fb0780c8000 2024-04-20T21:25:10.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.822+0000 7fb07e400640 1 -- 172.21.15.86:0/3957495532 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb0780689e0 con 0x7fb0780c8000 2024-04-20T21:25:10.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.824+0000 7fb06f400640 1 -- 172.21.15.86:0/3957495532 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+16660 (secure 0 0 0) 0x7fb070059c00 con 0x7fb0780c8000 2024-04-20T21:25:10.824 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:25:10.824 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":85,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:24:59.846179+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:24:55.964540+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-20T21:24:57.241527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:25:10.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.827+0000 7fb07e400640 1 -- 172.21.15.86:0/3957495532 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb05406c920 msgr2=0x7fb05406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:10.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.827+0000 7fb07e400640 1 --2- 172.21.15.86:0/3957495532 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb05406c920 0x7fb05406ede0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fb060033000 tx=0x7fb060009030 comp rx=0 tx=0).stop 2024-04-20T21:25:10.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.827+0000 7fb07e400640 1 -- 172.21.15.86:0/3957495532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0780c8000 msgr2=0x7fb0780c8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:10.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.827+0000 7fb07e400640 1 --2- 172.21.15.86:0/3957495532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0780c8000 0x7fb0780c8480 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fb0780b1bb0 tx=0x7fb070010c20 comp rx=0 tx=0).stop 2024-04-20T21:25:10.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.827+0000 7fb07e400640 1 -- 172.21.15.86:0/3957495532 shutdown_connections 2024-04-20T21:25:10.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.827+0000 7fb07e400640 1 --2- 172.21.15.86:0/3957495532 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb05406c920 0x7fb05406ede0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:10.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.827+0000 7fb07e400640 1 --2- 172.21.15.86:0/3957495532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0780c8000 0x7fb0780c8480 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:10.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.827+0000 7fb07e400640 1 --2- 172.21.15.86:0/3957495532 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb078160600 0x7fb078153880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:10.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.827+0000 7fb07e400640 1 --2- 172.21.15.86:0/3957495532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb05c003a00 0x7fb0780b7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:10.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.827+0000 7fb07e400640 1 -- 172.21.15.86:0/3957495532 >> 172.21.15.86:0/3957495532 conn(0x7fb0780b3ff0 msgr2=0x7fb0780af6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:10.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.827+0000 7fb07e400640 1 -- 172.21.15.86:0/3957495532 shutdown_connections 2024-04-20T21:25:10.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:10.827+0000 7fb07e400640 1 -- 172.21.15.86:0/3957495532 wait complete. 2024-04-20T21:25:10.846 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:24:55.964540+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:24:57.241527+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:25:10.846 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:25:11.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.051+0000 7f1ca5000640 1 Processor -- start 2024-04-20T21:25:11.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.051+0000 7f1ca5000640 1 -- start start 2024-04-20T21:25:11.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.053+0000 7f1ca5000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ca006b980 0x7f1ca006bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:11.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.053+0000 7f1ca5000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1ca015f010 0x7f1ca015f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:11.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.053+0000 7f1ca5000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ca006c2a0 con 0x7f1ca009aac0 2024-04-20T21:25:11.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.053+0000 7f1ca5000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f1ca00b2e70 con 0x7f1ca015f010 2024-04-20T21:25:11.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.053+0000 7f1ca5000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1ca00b25d0 con 0x7f1ca006b980 2024-04-20T21:25:11.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.053+0000 7f1c9fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1ca015f010 0x7f1ca015f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:11.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.053+0000 7f1c96a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f1ca009aac0 0x7f1ca00b6060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57480/0 (socket says 172.21.15.86:57480) 2024-04-20T21:25:11.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.053+0000 7f1c96a00640 1 -- 172.21.15.86:0/3731131150 learned_addr learned my addr 172.21.15.86:0/3731131150 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:25:11.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.054+0000 7f1c9f400640 1 --2- 172.21.15.86:0/3731131150 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ca006b980 0x7f1ca006bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:11.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.054+0000 7f1c9fe00640 1 -- 172.21.15.86:0/3731131150 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ca006b980 msgr2=0x7f1ca006bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:11.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.054+0000 7f1c9fe00640 1 --2- 172.21.15.86:0/3731131150 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ca006b980 0x7f1ca006bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:11.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.054+0000 7f1c9fe00640 1 -- 172.21.15.86:0/3731131150 >> v1:172.21.15.86:6789/0 conn(0x7f1ca009aac0 legacy=0x7f1ca00b6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:11.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.054+0000 7f1c9ea00640 1 -- 172.21.15.86:0/3731131150 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3710502844 0 0) 0x7f1ca006c2a0 con 0x7f1ca009aac0 2024-04-20T21:25:11.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.054+0000 7f1c9fe00640 1 -- 172.21.15.86:0/3731131150 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca00b28b0 con 0x7f1ca015f010 2024-04-20T21:25:11.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.055+0000 7f1c9fe00640 1 --2- 172.21.15.86:0/3731131150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ca015f010 0x7f1ca015f4a0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f1c980097a0 tx=0x7f1c9802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2579f0063b131d79 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:11.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.056+0000 7f1c9ea00640 1 -- 172.21.15.86:0/3731131150 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c980315a0 con 0x7f1ca015f010 2024-04-20T21:25:11.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.056+0000 7f1c9ea00640 1 -- 172.21.15.86:0/3731131150 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c98037070 con 0x7f1ca015f010 2024-04-20T21:25:11.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.056+0000 7f1c9ea00640 1 -- 172.21.15.86:0/3731131150 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c98031960 con 0x7f1ca015f010 2024-04-20T21:25:11.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.056+0000 7f1ca5000640 1 -- 172.21.15.86:0/3731131150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ca015f010 msgr2=0x7f1ca015f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:11.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.056+0000 7f1ca5000640 1 --2- 172.21.15.86:0/3731131150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ca015f010 0x7f1ca015f4a0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f1c980097a0 tx=0x7f1c9802b840 comp rx=0 tx=0).stop 2024-04-20T21:25:11.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.056+0000 7f1ca5000640 1 -- 172.21.15.86:0/3731131150 shutdown_connections 2024-04-20T21:25:11.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.056+0000 7f1ca5000640 1 --2- 172.21.15.86:0/3731131150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ca015f010 0x7f1ca015f4a0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:11.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.056+0000 7f1ca5000640 1 --2- 172.21.15.86:0/3731131150 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ca006b980 0x7f1ca006bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:11.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.056+0000 7f1ca5000640 1 -- 172.21.15.86:0/3731131150 >> 172.21.15.86:0/3731131150 conn(0x7f1ca0073220 msgr2=0x7f1ca00b2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:11.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.056+0000 7f1ca5000640 1 -- 172.21.15.86:0/3731131150 shutdown_connections 2024-04-20T21:25:11.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.056+0000 7f1ca5000640 1 -- 172.21.15.86:0/3731131150 wait complete. 2024-04-20T21:25:11.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.058+0000 7f1ca5000640 1 Processor -- start 2024-04-20T21:25:11.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.058+0000 7f1ca5000640 1 -- start start 2024-04-20T21:25:11.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.059+0000 7f1ca5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ca006b980 0x7f1ca00cc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:11.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.059+0000 7f1ca5000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ca009aac0 0x7f1ca0157be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:11.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.059+0000 7f1ca5000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ca015f010 0x7f1ca0148e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:11.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.059+0000 7f1ca5000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca015f9e0 con 0x7f1ca006b980 2024-04-20T21:25:11.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.059+0000 7f1ca5000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca00b6770 con 0x7f1ca009aac0 2024-04-20T21:25:11.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.059+0000 7f1ca5000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1ca015fb60 con 0x7f1ca015f010 2024-04-20T21:25:11.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.059+0000 7f1c9f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ca006b980 0x7f1ca00cc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:11.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.059+0000 7f1c9f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ca006b980 0x7f1ca00cc0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60120/0 (socket says 172.21.15.86:60120) 2024-04-20T21:25:11.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.059+0000 7f1c9f400640 1 -- 172.21.15.86:0/627054543 learned_addr learned my addr 172.21.15.86:0/627054543 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:11.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.060+0000 7f1c9fe00640 1 --2- 172.21.15.86:0/627054543 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ca015f010 0x7f1ca0148e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:11.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.060+0000 7f1c96a00640 1 --2- 172.21.15.86:0/627054543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ca009aac0 0x7f1ca0157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:11.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.060+0000 7f1c9f400640 1 -- 172.21.15.86:0/627054543 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ca015f010 msgr2=0x7f1ca0148e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:11.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.060+0000 7f1c9f400640 1 --2- 172.21.15.86:0/627054543 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ca015f010 0x7f1ca0148e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:11.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.060+0000 7f1c9f400640 1 -- 172.21.15.86:0/627054543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ca009aac0 msgr2=0x7f1ca0157be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:11.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.060+0000 7f1c9f400640 1 --2- 172.21.15.86:0/627054543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ca009aac0 0x7f1ca0157be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:11.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.060+0000 7f1c9f400640 1 -- 172.21.15.86:0/627054543 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca0158120 con 0x7f1ca006b980 2024-04-20T21:25:11.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.060+0000 7f1c9f400640 1 --2- 172.21.15.86:0/627054543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ca006b980 0x7f1ca00cc0d0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f1c90007e40 tx=0x7f1c9000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:11.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.061+0000 7f1c97e00640 1 -- 172.21.15.86:0/627054543 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c90014620 con 0x7f1ca006b980 2024-04-20T21:25:11.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.061+0000 7f1c97e00640 1 -- 172.21.15.86:0/627054543 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c90019070 con 0x7f1ca006b980 2024-04-20T21:25:11.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.061+0000 7f1ca5000640 1 -- 172.21.15.86:0/627054543 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ca00cc720 con 0x7f1ca006b980 2024-04-20T21:25:11.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.061+0000 7f1c97e00640 1 -- 172.21.15.86:0/627054543 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c90014a00 con 0x7f1ca006b980 2024-04-20T21:25:11.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.061+0000 7f1ca5000640 1 -- 172.21.15.86:0/627054543 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ca00ccce0 con 0x7f1ca006b980 2024-04-20T21:25:11.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.062+0000 7f1ca5000640 1 -- 172.21.15.86:0/627054543 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ca00b64d0 con 0x7f1ca006b980 2024-04-20T21:25:11.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.063+0000 7f1c97e00640 1 -- 172.21.15.86:0/627054543 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1c90014ba0 con 0x7f1ca006b980 2024-04-20T21:25:11.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.064+0000 7f1c97e00640 1 --2- 172.21.15.86:0/627054543 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1c7c06c920 0x7f1c7c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:11.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.064+0000 7f1c97e00640 1 -- 172.21.15.86:0/627054543 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7f1c9008c3e0 con 0x7f1ca006b980 2024-04-20T21:25:11.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.065+0000 7f1c96a00640 1 --2- 172.21.15.86:0/627054543 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1c7c06c920 0x7f1c7c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:11.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.065+0000 7f1c96a00640 1 --2- 172.21.15.86:0/627054543 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1c7c06c920 0x7f1c7c06ede0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1c88007fd0 tx=0x7f1c88009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:11.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.069+0000 7f1c97e00640 1 -- 172.21.15.86:0/627054543 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1ca00b64d0 con 0x7f1ca006b980 2024-04-20T21:25:11.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.270+0000 7f1ca5000640 1 -- 172.21.15.86:0/627054543 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1ca006ee20 con 0x7f1ca006b980 2024-04-20T21:25:11.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.271+0000 7f1c97e00640 1 -- 172.21.15.86:0/627054543 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) ==== 93+0+10 (secure 0 0 0) 0x7f1c9005a800 con 0x7f1ca006b980 2024-04-20T21:25:11.270 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:25:11.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.274+0000 7f1ca5000640 1 -- 172.21.15.86:0/627054543 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1c7c06c920 msgr2=0x7f1c7c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:11.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.274+0000 7f1ca5000640 1 --2- 172.21.15.86:0/627054543 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1c7c06c920 0x7f1c7c06ede0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1c88007fd0 tx=0x7f1c88009040 comp rx=0 tx=0).stop 2024-04-20T21:25:11.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.274+0000 7f1ca5000640 1 -- 172.21.15.86:0/627054543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ca006b980 msgr2=0x7f1ca00cc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:11.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.274+0000 7f1ca5000640 1 --2- 172.21.15.86:0/627054543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ca006b980 0x7f1ca00cc0d0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f1c90007e40 tx=0x7f1c9000e4b0 comp rx=0 tx=0).stop 2024-04-20T21:25:11.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.274+0000 7f1ca5000640 1 -- 172.21.15.86:0/627054543 shutdown_connections 2024-04-20T21:25:11.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.274+0000 7f1ca5000640 1 --2- 172.21.15.86:0/627054543 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1c7c06c920 0x7f1c7c06ede0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:11.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.274+0000 7f1ca5000640 1 --2- 172.21.15.86:0/627054543 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ca015f010 0x7f1ca0148e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:11.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.274+0000 7f1ca5000640 1 --2- 172.21.15.86:0/627054543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ca009aac0 0x7f1ca0157be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:11.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.274+0000 7f1ca5000640 1 --2- 172.21.15.86:0/627054543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ca006b980 0x7f1ca00cc0d0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:11.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.274+0000 7f1ca5000640 1 -- 172.21.15.86:0/627054543 >> 172.21.15.86:0/627054543 conn(0x7f1ca0073220 msgr2=0x7f1ca015de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:11.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.274+0000 7f1ca5000640 1 -- 172.21.15.86:0/627054543 shutdown_connections 2024-04-20T21:25:11.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.274+0000 7f1ca5000640 1 -- 172.21.15.86:0/627054543 wait complete. 2024-04-20T21:25:11.292 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:25:11.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.497+0000 7f44f5e00640 1 Processor -- start 2024-04-20T21:25:11.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.497+0000 7f44f5e00640 1 -- start start 2024-04-20T21:25:11.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.499+0000 7f44f5e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44f015ea70 0x7f44f00d3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:11.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.499+0000 7f44f5e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f44f006d570 0x7f44f00cf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:11.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.499+0000 7f44f5e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44f00cf120 con 0x7f44f00d45a0 2024-04-20T21:25:11.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.499+0000 7f44f5e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f44f006de70 con 0x7f44f006d570 2024-04-20T21:25:11.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.499+0000 7f44f5e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f44f006e150 con 0x7f44f015ea70 2024-04-20T21:25:11.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.499+0000 7f44f4a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f44f00d45a0 0x7f44f00d4980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57488/0 (socket says 172.21.15.86:57488) 2024-04-20T21:25:11.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.499+0000 7f44f4a00640 1 -- 172.21.15.86:0/1900214157 learned_addr learned my addr 172.21.15.86:0/1900214157 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:25:11.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.499+0000 7f44f5400640 1 --2- 172.21.15.86:0/1900214157 >> v2:172.21.15.86:3300/0 conn(0x7f44f006d570 0x7f44f00cf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:11.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.500+0000 7f44e7e00640 1 --2- 172.21.15.86:0/1900214157 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44f015ea70 0x7f44f00d3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:11.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.500+0000 7f44efe00640 1 -- 172.21.15.86:0/1900214157 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 838987169 0 0) 0x7f44f00cf120 con 0x7f44f00d45a0 2024-04-20T21:25:11.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.500+0000 7f44efe00640 1 -- 172.21.15.86:0/1900214157 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f44d4003520 con 0x7f44f00d45a0 2024-04-20T21:25:11.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.500+0000 7f44f5400640 1 -- 172.21.15.86:0/1900214157 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44f015ea70 msgr2=0x7f44f00d3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:11.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.500+0000 7f44f5400640 1 --2- 172.21.15.86:0/1900214157 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44f015ea70 0x7f44f00d3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:11.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.500+0000 7f44f5400640 1 -- 172.21.15.86:0/1900214157 >> v1:172.21.15.86:6789/0 conn(0x7f44f00d45a0 legacy=0x7f44f00d4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:11.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.500+0000 7f44f5400640 1 -- 172.21.15.86:0/1900214157 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44f006e430 con 0x7f44f006d570 2024-04-20T21:25:11.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.501+0000 7f44f5400640 1 --2- 172.21.15.86:0/1900214157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44f006d570 0x7f44f00cf8e0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f44e8009d90 tx=0x7f44e802be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=869fe9b9bed5d1c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:11.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.501+0000 7f44efe00640 1 -- 172.21.15.86:0/1900214157 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e8039070 con 0x7f44f006d570 2024-04-20T21:25:11.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.502+0000 7f44efe00640 1 -- 172.21.15.86:0/1900214157 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44e802cbc0 con 0x7f44f006d570 2024-04-20T21:25:11.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.502+0000 7f44efe00640 1 -- 172.21.15.86:0/1900214157 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e8034440 con 0x7f44f006d570 2024-04-20T21:25:11.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.502+0000 7f44f5e00640 1 -- 172.21.15.86:0/1900214157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44f006d570 msgr2=0x7f44f00cf8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:11.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.502+0000 7f44f5e00640 1 --2- 172.21.15.86:0/1900214157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44f006d570 0x7f44f00cf8e0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f44e8009d90 tx=0x7f44e802be10 comp rx=0 tx=0).stop 2024-04-20T21:25:11.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.502+0000 7f44f5e00640 1 -- 172.21.15.86:0/1900214157 shutdown_connections 2024-04-20T21:25:11.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.502+0000 7f44f5e00640 1 --2- 172.21.15.86:0/1900214157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44f006d570 0x7f44f00cf8e0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:11.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.502+0000 7f44f5e00640 1 --2- 172.21.15.86:0/1900214157 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44f015ea70 0x7f44f00d3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:11.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.502+0000 7f44f5e00640 1 -- 172.21.15.86:0/1900214157 >> 172.21.15.86:0/1900214157 conn(0x7f44f009d9d0 msgr2=0x7f44f008bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:11.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.502+0000 7f44f5e00640 1 -- 172.21.15.86:0/1900214157 shutdown_connections 2024-04-20T21:25:11.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.502+0000 7f44f5e00640 1 -- 172.21.15.86:0/1900214157 wait complete. 2024-04-20T21:25:11.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.504+0000 7f44f5e00640 1 Processor -- start 2024-04-20T21:25:11.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.504+0000 7f44f5e00640 1 -- start start 2024-04-20T21:25:11.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.505+0000 7f44f5e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44f006d570 0x7f44f015aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:11.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.505+0000 7f44f5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44f00d45a0 0x7f44f00cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:11.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.505+0000 7f44f5e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44f015ea70 0x7f44f00cdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:11.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.505+0000 7f44f5e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f44f006db80 con 0x7f44f00d45a0 2024-04-20T21:25:11.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.505+0000 7f44f5e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f44f006da00 con 0x7f44f015ea70 2024-04-20T21:25:11.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.505+0000 7f44f5e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f44f00cfe20 con 0x7f44f006d570 2024-04-20T21:25:11.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.505+0000 7f44e7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44f00d45a0 0x7f44f00cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:11.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.505+0000 7f44e7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44f00d45a0 0x7f44f00cc1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60132/0 (socket says 172.21.15.86:60132) 2024-04-20T21:25:11.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.505+0000 7f44e7e00640 1 -- 172.21.15.86:0/3305193285 learned_addr learned my addr 172.21.15.86:0/3305193285 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:11.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.506+0000 7f44f4a00640 1 --2- 172.21.15.86:0/3305193285 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44f006d570 0x7f44f015aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:11.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.506+0000 7f44f5400640 1 --2- 172.21.15.86:0/3305193285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44f015ea70 0x7f44f00cdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:11.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.506+0000 7f44e7e00640 1 -- 172.21.15.86:0/3305193285 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44f006d570 msgr2=0x7f44f015aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:11.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.506+0000 7f44e7e00640 1 --2- 172.21.15.86:0/3305193285 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44f006d570 0x7f44f015aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:11.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.506+0000 7f44e7e00640 1 -- 172.21.15.86:0/3305193285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44f015ea70 msgr2=0x7f44f00cdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:11.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.506+0000 7f44e7e00640 1 --2- 172.21.15.86:0/3305193285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44f015ea70 0x7f44f00cdeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:11.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.506+0000 7f44e7e00640 1 -- 172.21.15.86:0/3305193285 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44f01609e0 con 0x7f44f00d45a0 2024-04-20T21:25:11.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.506+0000 7f44f5400640 1 --2- 172.21.15.86:0/3305193285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44f015ea70 0x7f44f00cdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:25:11.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.506+0000 7f44e7e00640 1 --2- 172.21.15.86:0/3305193285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44f00d45a0 0x7f44f00cc1d0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f44d800d620 tx=0x7f44d800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:11.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.506+0000 7f44ed600640 1 -- 172.21.15.86:0/3305193285 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d80125f0 con 0x7f44f00d45a0 2024-04-20T21:25:11.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.506+0000 7f44ed600640 1 -- 172.21.15.86:0/3305193285 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44d8019070 con 0x7f44f00d45a0 2024-04-20T21:25:11.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.507+0000 7f44f5e00640 1 -- 172.21.15.86:0/3305193285 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44f0160cc0 con 0x7f44f00d45a0 2024-04-20T21:25:11.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.507+0000 7f44ed600640 1 -- 172.21.15.86:0/3305193285 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d80129d0 con 0x7f44f00d45a0 2024-04-20T21:25:11.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.507+0000 7f44f5e00640 1 -- 172.21.15.86:0/3305193285 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44f0161280 con 0x7f44f00d45a0 2024-04-20T21:25:11.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.508+0000 7f44f5e00640 1 -- 172.21.15.86:0/3305193285 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44f004fd90 con 0x7f44f00d45a0 2024-04-20T21:25:11.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.509+0000 7f44ed600640 1 -- 172.21.15.86:0/3305193285 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f44d8012b70 con 0x7f44f00d45a0 2024-04-20T21:25:11.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.510+0000 7f44ed600640 1 --2- 172.21.15.86:0/3305193285 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f44cc06c8f0 0x7f44cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:11.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.510+0000 7f44ed600640 1 -- 172.21.15.86:0/3305193285 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7f44d808bf40 con 0x7f44f00d45a0 2024-04-20T21:25:11.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.510+0000 7f44f4a00640 1 --2- 172.21.15.86:0/3305193285 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f44cc06c8f0 0x7f44cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:11.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.511+0000 7f44f4a00640 1 --2- 172.21.15.86:0/3305193285 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f44cc06c8f0 0x7f44cc06edb0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f44e0005880 tx=0x7f44e0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:11.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.514+0000 7f44ed600640 1 -- 172.21.15.86:0/3305193285 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f44d805a360 con 0x7f44f00d45a0 2024-04-20T21:25:11.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.707+0000 7f44f5e00640 1 -- 172.21.15.86:0/3305193285 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f44f015e090 con 0x7f44f00d45a0 2024-04-20T21:25:11.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.708+0000 7f44ed600640 1 -- 172.21.15.86:0/3305193285 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) ==== 104+0+11 (secure 0 0 0) 0x7f44d8059c60 con 0x7f44f00d45a0 2024-04-20T21:25:11.706 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-20T21:25:11.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.710+0000 7f44f5e00640 1 -- 172.21.15.86:0/3305193285 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f44cc06c8f0 msgr2=0x7f44cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:11.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.710+0000 7f44f5e00640 1 --2- 172.21.15.86:0/3305193285 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f44cc06c8f0 0x7f44cc06edb0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f44e0005880 tx=0x7f44e0009040 comp rx=0 tx=0).stop 2024-04-20T21:25:11.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.710+0000 7f44f5e00640 1 -- 172.21.15.86:0/3305193285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44f00d45a0 msgr2=0x7f44f00cc1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:11.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.710+0000 7f44f5e00640 1 --2- 172.21.15.86:0/3305193285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44f00d45a0 0x7f44f00cc1d0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f44d800d620 tx=0x7f44d800dae0 comp rx=0 tx=0).stop 2024-04-20T21:25:11.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.710+0000 7f44f5e00640 1 -- 172.21.15.86:0/3305193285 shutdown_connections 2024-04-20T21:25:11.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.710+0000 7f44f5e00640 1 --2- 172.21.15.86:0/3305193285 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f44cc06c8f0 0x7f44cc06edb0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:11.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.710+0000 7f44f5e00640 1 --2- 172.21.15.86:0/3305193285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44f015ea70 0x7f44f00cdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:11.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.710+0000 7f44f5e00640 1 --2- 172.21.15.86:0/3305193285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44f00d45a0 0x7f44f00cc1d0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:11.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.710+0000 7f44f5e00640 1 --2- 172.21.15.86:0/3305193285 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44f006d570 0x7f44f015aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:11.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.710+0000 7f44f5e00640 1 -- 172.21.15.86:0/3305193285 >> 172.21.15.86:0/3305193285 conn(0x7f44f009d9d0 msgr2=0x7f44f00d3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:11.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.710+0000 7f44f5e00640 1 -- 172.21.15.86:0/3305193285 shutdown_connections 2024-04-20T21:25:11.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.710+0000 7f44f5e00640 1 -- 172.21.15.86:0/3305193285 wait complete. 2024-04-20T21:25:11.727 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:25:11.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.924+0000 7f9daaa00640 1 Processor -- start 2024-04-20T21:25:11.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.925+0000 7f9daaa00640 1 -- start start 2024-04-20T21:25:11.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.926+0000 7f9daaa00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9da40b3200 0x7f9da40b3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:11.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.926+0000 7f9daaa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9da41420a0 0x7f9da4142510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:11.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.926+0000 7f9daaa00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9da4148180 con 0x7f9da40af980 2024-04-20T21:25:11.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.926+0000 7f9daaa00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9da40b3ba0 con 0x7f9da41420a0 2024-04-20T21:25:11.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.926+0000 7f9daaa00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f9da406f450 con 0x7f9da40b3200 2024-04-20T21:25:11.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.926+0000 7f9daa000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9da41420a0 0x7f9da4142510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:11.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.927+0000 7f9daa000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9da41420a0 0x7f9da4142510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60148/0 (socket says 172.21.15.86:60148) 2024-04-20T21:25:11.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.927+0000 7f9daa000640 1 -- 172.21.15.86:0/1046730584 learned_addr learned my addr 172.21.15.86:0/1046730584 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:11.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.927+0000 7f9daa000640 1 -- 172.21.15.86:0/1046730584 >> v2:172.21.15.144:3300/0 conn(0x7f9da40b3200 msgr2=0x7f9da40b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:11.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.927+0000 7f9daa000640 1 --2- 172.21.15.86:0/1046730584 >> v2:172.21.15.144:3300/0 conn(0x7f9da40b3200 0x7f9da40b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:11.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.927+0000 7f9daa000640 1 -- 172.21.15.86:0/1046730584 >> v1:172.21.15.144:6789/0 conn(0x7f9da40af980 legacy=0x7f9da4147a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:11.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.927+0000 7f9daa000640 1 -- 172.21.15.86:0/1046730584 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da406e840 con 0x7f9da41420a0 2024-04-20T21:25:11.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.928+0000 7f9daa000640 1 --2- 172.21.15.86:0/1046730584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9da41420a0 0x7f9da4142510 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f9da4071a00 tx=0x7f9d9c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fbca9fc978775e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:11.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.928+0000 7f9da3e00640 1 -- 172.21.15.86:0/1046730584 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c02ca70 con 0x7f9da41420a0 2024-04-20T21:25:11.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.928+0000 7f9da3e00640 1 -- 172.21.15.86:0/1046730584 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d9c02cc10 con 0x7f9da41420a0 2024-04-20T21:25:11.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.928+0000 7f9da3e00640 1 -- 172.21.15.86:0/1046730584 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c0343f0 con 0x7f9da41420a0 2024-04-20T21:25:11.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.929+0000 7f9daaa00640 1 -- 172.21.15.86:0/1046730584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9da41420a0 msgr2=0x7f9da4142510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:11.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.929+0000 7f9daaa00640 1 --2- 172.21.15.86:0/1046730584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9da41420a0 0x7f9da4142510 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f9da4071a00 tx=0x7f9d9c02bed0 comp rx=0 tx=0).stop 2024-04-20T21:25:11.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.929+0000 7f9daaa00640 1 -- 172.21.15.86:0/1046730584 shutdown_connections 2024-04-20T21:25:11.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.929+0000 7f9daaa00640 1 --2- 172.21.15.86:0/1046730584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9da41420a0 0x7f9da4142510 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:11.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.929+0000 7f9daaa00640 1 --2- 172.21.15.86:0/1046730584 >> v2:172.21.15.144:3300/0 conn(0x7f9da40b3200 0x7f9da40b3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:11.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.929+0000 7f9daaa00640 1 -- 172.21.15.86:0/1046730584 >> 172.21.15.86:0/1046730584 conn(0x7f9da40b1f00 msgr2=0x7f9da40b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:11.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.929+0000 7f9daaa00640 1 -- 172.21.15.86:0/1046730584 shutdown_connections 2024-04-20T21:25:11.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.929+0000 7f9daaa00640 1 -- 172.21.15.86:0/1046730584 wait complete. 2024-04-20T21:25:11.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.931+0000 7f9daaa00640 1 Processor -- start 2024-04-20T21:25:11.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.931+0000 7f9daaa00640 1 -- start start 2024-04-20T21:25:11.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.932+0000 7f9daaa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9da40af980 0x7f9da40998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:11.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.932+0000 7f9daaa00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9da40b3200 0x7f9da41e89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:11.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.932+0000 7f9daaa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9da41420a0 0x7f9da415fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:11.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.932+0000 7f9daaa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9da406f0e0 con 0x7f9da40af980 2024-04-20T21:25:11.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.932+0000 7f9daaa00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9da406ee00 con 0x7f9da40b3200 2024-04-20T21:25:11.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.932+0000 7f9daaa00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9da406e3d0 con 0x7f9da41420a0 2024-04-20T21:25:11.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.932+0000 7f9da9600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9da40af980 0x7f9da40998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:11.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.932+0000 7f9da9600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9da40af980 0x7f9da40998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60156/0 (socket says 172.21.15.86:60156) 2024-04-20T21:25:11.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.932+0000 7f9da9600640 1 -- 172.21.15.86:0/764296120 learned_addr learned my addr 172.21.15.86:0/764296120 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:11.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.933+0000 7f9da8c00640 1 --2- 172.21.15.86:0/764296120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9da40b3200 0x7f9da41e89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:11.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.933+0000 7f9daa000640 1 --2- 172.21.15.86:0/764296120 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9da41420a0 0x7f9da415fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:11.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.933+0000 7f9da9600640 1 -- 172.21.15.86:0/764296120 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9da41420a0 msgr2=0x7f9da415fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:11.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.933+0000 7f9da9600640 1 --2- 172.21.15.86:0/764296120 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9da41420a0 0x7f9da415fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:11.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.933+0000 7f9da9600640 1 -- 172.21.15.86:0/764296120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9da40b3200 msgr2=0x7f9da41e89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:11.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.933+0000 7f9da9600640 1 --2- 172.21.15.86:0/764296120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9da40b3200 0x7f9da41e89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:11.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.933+0000 7f9da9600640 1 -- 172.21.15.86:0/764296120 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da41e9250 con 0x7f9da40af980 2024-04-20T21:25:11.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.933+0000 7f9da8c00640 1 --2- 172.21.15.86:0/764296120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9da40b3200 0x7f9da41e89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:25:11.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.933+0000 7f9da9600640 1 --2- 172.21.15.86:0/764296120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9da40af980 0x7f9da40998c0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f9d9800b3e0 tx=0x7f9d9800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:11.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.933+0000 7f9da1600640 1 -- 172.21.15.86:0/764296120 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d98015020 con 0x7f9da40af980 2024-04-20T21:25:11.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.933+0000 7f9da1600640 1 -- 172.21.15.86:0/764296120 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d98011850 con 0x7f9da40af980 2024-04-20T21:25:11.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.933+0000 7f9da1600640 1 -- 172.21.15.86:0/764296120 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d98011c40 con 0x7f9da40af980 2024-04-20T21:25:11.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.933+0000 7f9daaa00640 1 -- 172.21.15.86:0/764296120 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9da41e9530 con 0x7f9da40af980 2024-04-20T21:25:11.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.934+0000 7f9daaa00640 1 -- 172.21.15.86:0/764296120 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9da41e9af0 con 0x7f9da40af980 2024-04-20T21:25:11.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.935+0000 7f9daaa00640 1 -- 172.21.15.86:0/764296120 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9da404fd90 con 0x7f9da40af980 2024-04-20T21:25:11.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.936+0000 7f9da1600640 1 -- 172.21.15.86:0/764296120 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9d98018020 con 0x7f9da40af980 2024-04-20T21:25:11.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.936+0000 7f9da1600640 1 --2- 172.21.15.86:0/764296120 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9d8406c8f0 0x7f9d8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:11.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.937+0000 7f9da1600640 1 -- 172.21.15.86:0/764296120 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7f9d9808bc40 con 0x7f9da40af980 2024-04-20T21:25:11.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.937+0000 7f9da8c00640 1 --2- 172.21.15.86:0/764296120 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9d8406c8f0 0x7f9d8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:11.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.938+0000 7f9da8c00640 1 --2- 172.21.15.86:0/764296120 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9d8406c8f0 0x7f9d8406edb0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f9da409e530 tx=0x7f9d90008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:11.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:11.941+0000 7f9da1600640 1 -- 172.21.15.86:0/764296120 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9d9801f020 con 0x7f9da40af980 2024-04-20T21:25:12.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.140+0000 7f9daaa00640 1 -- 172.21.15.86:0/764296120 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9da40b0930 con 0x7f9da40af980 2024-04-20T21:25:12.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.141+0000 7f9da1600640 1 -- 172.21.15.86:0/764296120 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) ==== 100+0+11 (secure 0 0 0) 0x7f9d98059f40 con 0x7f9da40af980 2024-04-20T21:25:12.140 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-20T21:25:12.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.143+0000 7f9daaa00640 1 -- 172.21.15.86:0/764296120 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9d8406c8f0 msgr2=0x7f9d8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:12.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.143+0000 7f9daaa00640 1 --2- 172.21.15.86:0/764296120 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9d8406c8f0 0x7f9d8406edb0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f9da409e530 tx=0x7f9d90008040 comp rx=0 tx=0).stop 2024-04-20T21:25:12.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.143+0000 7f9daaa00640 1 -- 172.21.15.86:0/764296120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9da40af980 msgr2=0x7f9da40998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:12.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.143+0000 7f9daaa00640 1 --2- 172.21.15.86:0/764296120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9da40af980 0x7f9da40998c0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f9d9800b3e0 tx=0x7f9d9800b8a0 comp rx=0 tx=0).stop 2024-04-20T21:25:12.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.144+0000 7f9daaa00640 1 -- 172.21.15.86:0/764296120 shutdown_connections 2024-04-20T21:25:12.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.144+0000 7f9daaa00640 1 --2- 172.21.15.86:0/764296120 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9d8406c8f0 0x7f9d8406edb0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:12.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.144+0000 7f9daaa00640 1 --2- 172.21.15.86:0/764296120 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9da41420a0 0x7f9da415fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:12.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.144+0000 7f9daaa00640 1 --2- 172.21.15.86:0/764296120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9da40b3200 0x7f9da41e89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:12.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.144+0000 7f9daaa00640 1 --2- 172.21.15.86:0/764296120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9da40af980 0x7f9da40998c0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:12.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.144+0000 7f9daaa00640 1 -- 172.21.15.86:0/764296120 >> 172.21.15.86:0/764296120 conn(0x7f9da40b1f00 msgr2=0x7f9da41479e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:12.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.144+0000 7f9daaa00640 1 -- 172.21.15.86:0/764296120 shutdown_connections 2024-04-20T21:25:12.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.144+0000 7f9daaa00640 1 -- 172.21.15.86:0/764296120 wait complete. 2024-04-20T21:25:12.161 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T21:25:12.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.363+0000 7f66f8000640 1 Processor -- start 2024-04-20T21:25:12.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.363+0000 7f66f8000640 1 -- start start 2024-04-20T21:25:12.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.365+0000 7f66f8000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f66f0145fe0 0x7f66f00d02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:12.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.365+0000 7f66f8000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66f00d0810 0x7f66f00cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:12.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.365+0000 7f66f8000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f66f00cd610 0x7f66f0160350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:12.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.365+0000 7f66f8000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f66f00d00d0 con 0x7f66f00cd610 2024-04-20T21:25:12.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.365+0000 7f66f8000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f66f00d0c70 con 0x7f66f0145fe0 2024-04-20T21:25:12.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.365+0000 7f66f8000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f66f00cdaa0 con 0x7f66f00d0810 2024-04-20T21:25:12.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.365+0000 7f66f6000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f66f00cd610 0x7f66f0160350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:12.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.365+0000 7f66f6000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f66f00cd610 0x7f66f0160350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60160/0 (socket says 172.21.15.86:60160) 2024-04-20T21:25:12.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.365+0000 7f66f6000640 1 -- 172.21.15.86:0/3840828948 learned_addr learned my addr 172.21.15.86:0/3840828948 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:12.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.366+0000 7f66f4c00640 1 --2- 172.21.15.86:0/3840828948 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66f00d0810 0x7f66f00cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:12.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.366+0000 7f66f6000640 1 -- 172.21.15.86:0/3840828948 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66f00d0810 msgr2=0x7f66f00cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:12.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.366+0000 7f66f5600640 1 --2- 172.21.15.86:0/3840828948 >> v2:172.21.15.144:3300/0 conn(0x7f66f0145fe0 0x7f66f00d02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:12.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.366+0000 7f66f6000640 1 --2- 172.21.15.86:0/3840828948 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66f00d0810 0x7f66f00cd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:12.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.366+0000 7f66f6000640 1 -- 172.21.15.86:0/3840828948 >> v2:172.21.15.144:3300/0 conn(0x7f66f0145fe0 msgr2=0x7f66f00d02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:12.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.366+0000 7f66f6000640 1 --2- 172.21.15.86:0/3840828948 >> v2:172.21.15.144:3300/0 conn(0x7f66f0145fe0 0x7f66f00d02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:12.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.366+0000 7f66f6000640 1 -- 172.21.15.86:0/3840828948 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66f015dcc0 con 0x7f66f00cd610 2024-04-20T21:25:12.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.366+0000 7f66f4c00640 1 --2- 172.21.15.86:0/3840828948 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66f00d0810 0x7f66f00cd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:25:12.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.367+0000 7f66f6000640 1 --2- 172.21.15.86:0/3840828948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66f00cd610 0x7f66f0160350 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f66f006d5c0 tx=0x7f66e802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c024086ffe9439 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:12.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.367+0000 7f66efe00640 1 -- 172.21.15.86:0/3840828948 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e802ca70 con 0x7f66f00cd610 2024-04-20T21:25:12.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.367+0000 7f66efe00640 1 -- 172.21.15.86:0/3840828948 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66e802cc10 con 0x7f66f00cd610 2024-04-20T21:25:12.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.367+0000 7f66efe00640 1 -- 172.21.15.86:0/3840828948 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e80343f0 con 0x7f66f00cd610 2024-04-20T21:25:12.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.368+0000 7f66f8000640 1 -- 172.21.15.86:0/3840828948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66f00cd610 msgr2=0x7f66f0160350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:12.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.368+0000 7f66f8000640 1 --2- 172.21.15.86:0/3840828948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66f00cd610 0x7f66f0160350 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f66f006d5c0 tx=0x7f66e802bed0 comp rx=0 tx=0).stop 2024-04-20T21:25:12.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.368+0000 7f66f8000640 1 -- 172.21.15.86:0/3840828948 shutdown_connections 2024-04-20T21:25:12.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.368+0000 7f66f8000640 1 --2- 172.21.15.86:0/3840828948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66f00cd610 0x7f66f0160350 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:12.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.368+0000 7f66f8000640 1 --2- 172.21.15.86:0/3840828948 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66f00d0810 0x7f66f00cd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:12.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.368+0000 7f66f8000640 1 --2- 172.21.15.86:0/3840828948 >> v2:172.21.15.144:3300/0 conn(0x7f66f0145fe0 0x7f66f00d02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:12.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.368+0000 7f66f8000640 1 -- 172.21.15.86:0/3840828948 >> 172.21.15.86:0/3840828948 conn(0x7f66f006b980 msgr2=0x7f66f006bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:12.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.368+0000 7f66f8000640 1 -- 172.21.15.86:0/3840828948 shutdown_connections 2024-04-20T21:25:12.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.368+0000 7f66f8000640 1 -- 172.21.15.86:0/3840828948 wait complete. 2024-04-20T21:25:12.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.370+0000 7f66f8000640 1 Processor -- start 2024-04-20T21:25:12.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.370+0000 7f66f8000640 1 -- start start 2024-04-20T21:25:12.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.371+0000 7f66f8000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66f00cd610 0x7f66f0135bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:12.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.371+0000 7f66f8000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66f00d0810 0x7f66f0140580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:12.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.371+0000 7f66f8000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66f0145fe0 0x7f66f0141f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:12.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.371+0000 7f66f8000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f66f015d4c0 con 0x7f66f00d0810 2024-04-20T21:25:12.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.371+0000 7f66f8000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f66f015d1e0 con 0x7f66f0145fe0 2024-04-20T21:25:12.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.371+0000 7f66f8000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f66f015d7a0 con 0x7f66f00cd610 2024-04-20T21:25:12.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.371+0000 7f66f4c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66f00d0810 0x7f66f0140580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:12.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.371+0000 7f66f4c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66f00d0810 0x7f66f0140580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60172/0 (socket says 172.21.15.86:60172) 2024-04-20T21:25:12.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.371+0000 7f66f4c00640 1 -- 172.21.15.86:0/3660832434 learned_addr learned my addr 172.21.15.86:0/3660832434 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:12.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.371+0000 7f66f4c00640 1 -- 172.21.15.86:0/3660832434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66f00cd610 msgr2=0x7f66f0135bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:12.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.371+0000 7f66f4c00640 1 --2- 172.21.15.86:0/3660832434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66f00cd610 0x7f66f0135bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:12.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.371+0000 7f66f4c00640 1 -- 172.21.15.86:0/3660832434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66f0145fe0 msgr2=0x7f66f0141f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:12.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.371+0000 7f66f4c00640 1 --2- 172.21.15.86:0/3660832434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66f0145fe0 0x7f66f0141f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:12.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.371+0000 7f66f4c00640 1 -- 172.21.15.86:0/3660832434 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66f0136100 con 0x7f66f00d0810 2024-04-20T21:25:12.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.372+0000 7f66f4c00640 1 --2- 172.21.15.86:0/3660832434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66f00d0810 0x7f66f0140580 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f66dc00c7c0 tx=0x7f66dc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:12.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.372+0000 7f66ed600640 1 -- 172.21.15.86:0/3660832434 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66dc015070 con 0x7f66f00d0810 2024-04-20T21:25:12.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.372+0000 7f66ed600640 1 -- 172.21.15.86:0/3660832434 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66dc007a30 con 0x7f66f00d0810 2024-04-20T21:25:12.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.372+0000 7f66ed600640 1 -- 172.21.15.86:0/3660832434 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66dc007e00 con 0x7f66f00d0810 2024-04-20T21:25:12.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.372+0000 7f66f8000640 1 -- 172.21.15.86:0/3660832434 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66f0141080 con 0x7f66f00d0810 2024-04-20T21:25:12.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.372+0000 7f66f8000640 1 -- 172.21.15.86:0/3660832434 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66f0141640 con 0x7f66f00d0810 2024-04-20T21:25:12.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.373+0000 7f66f8000640 1 -- 172.21.15.86:0/3660832434 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66f00689e0 con 0x7f66f00d0810 2024-04-20T21:25:12.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.374+0000 7f66ed600640 1 -- 172.21.15.86:0/3660832434 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f66dc018070 con 0x7f66f00d0810 2024-04-20T21:25:12.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.375+0000 7f66ed600640 1 --2- 172.21.15.86:0/3660832434 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f66d006c8f0 0x7f66d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:12.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.375+0000 7f66ed600640 1 -- 172.21.15.86:0/3660832434 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7f66dc08bad0 con 0x7f66f00d0810 2024-04-20T21:25:12.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.375+0000 7f66f5600640 1 --2- 172.21.15.86:0/3660832434 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f66d006c8f0 0x7f66d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:12.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.376+0000 7f66f5600640 1 --2- 172.21.15.86:0/3660832434 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f66d006c8f0 0x7f66d006edb0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f66e400a810 tx=0x7f66e4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:12.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.379+0000 7f66ed600640 1 -- 172.21.15.86:0/3660832434 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f66dc059e60 con 0x7f66f00d0810 2024-04-20T21:25:12.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.589+0000 7f66f8000640 1 -- 172.21.15.86:0/3660832434 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f66f00d1020 con 0x7f66f00d0810 2024-04-20T21:25:12.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.590+0000 7f66ed600640 1 -- 172.21.15.86:0/3660832434 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v85) ==== 80+0+0 (secure 0 0 0) 0x7f66dc059760 con 0x7f66f00d0810 2024-04-20T21:25:12.589 INFO:teuthology.orchestra.run.smithi086.stderr:listed 0 entries 2024-04-20T21:25:12.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.592+0000 7f66f8000640 1 -- 172.21.15.86:0/3660832434 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f66d006c8f0 msgr2=0x7f66d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:12.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.592+0000 7f66f8000640 1 --2- 172.21.15.86:0/3660832434 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f66d006c8f0 0x7f66d006edb0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f66e400a810 tx=0x7f66e4008040 comp rx=0 tx=0).stop 2024-04-20T21:25:12.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.592+0000 7f66f8000640 1 -- 172.21.15.86:0/3660832434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66f00d0810 msgr2=0x7f66f0140580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:12.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.592+0000 7f66f8000640 1 --2- 172.21.15.86:0/3660832434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66f00d0810 0x7f66f0140580 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f66dc00c7c0 tx=0x7f66dc00cc80 comp rx=0 tx=0).stop 2024-04-20T21:25:12.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.592+0000 7f66f8000640 1 -- 172.21.15.86:0/3660832434 shutdown_connections 2024-04-20T21:25:12.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.592+0000 7f66f8000640 1 --2- 172.21.15.86:0/3660832434 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f66d006c8f0 0x7f66d006edb0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:12.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.592+0000 7f66f8000640 1 --2- 172.21.15.86:0/3660832434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f66f0145fe0 0x7f66f0141f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:12.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.592+0000 7f66f8000640 1 --2- 172.21.15.86:0/3660832434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66f00d0810 0x7f66f0140580 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:12.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.592+0000 7f66f8000640 1 --2- 172.21.15.86:0/3660832434 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f66f00cd610 0x7f66f0135bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:12.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.592+0000 7f66f8000640 1 -- 172.21.15.86:0/3660832434 >> 172.21.15.86:0/3660832434 conn(0x7f66f006b980 msgr2=0x7f66f0143b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:12.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.592+0000 7f66f8000640 1 -- 172.21.15.86:0/3660832434 shutdown_connections 2024-04-20T21:25:12.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:12.592+0000 7f66f8000640 1 -- 172.21.15.86:0/3660832434 wait complete. 2024-04-20T21:25:12.610 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi086.front.sepia.ceph.com... 2024-04-20T21:25:12.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:25:12.610 DEBUG:teuthology.orchestra.run.smithi086:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-20T21:25:12.689 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[53050]: fuse finished with error 0 and tester_r 0 2024-04-20T21:25:12.695 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:25:17.149 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:25:17.157 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:25:18.796 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-20T21:25:18.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:25:18.796 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-20T21:25:18.854 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:25:19.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.056+0000 7efe20a00640 1 Processor -- start 2024-04-20T21:25:19.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.057+0000 7efe20a00640 1 -- start start 2024-04-20T21:25:19.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.058+0000 7efe20a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7efe1c0d0d20 0x7efe1c0d1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:19.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.058+0000 7efe20a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7efe1c0d5150 0x7efe1c1629f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:19.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.058+0000 7efe20a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe1c0d1640 con 0x7efe1c0d4cf0 2024-04-20T21:25:19.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.058+0000 7efe20a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7efe1c0cfea0 con 0x7efe1c0d0d20 2024-04-20T21:25:19.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.058+0000 7efe20a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7efe1c0d31c0 con 0x7efe1c0d5150 2024-04-20T21:25:19.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.058+0000 7efe1b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7efe1c0d0d20 0x7efe1c0d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:19.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.058+0000 7efe1aa00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7efe1c0d4cf0 0x7efe1c1622e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:53630/0 (socket says 172.21.15.86:53630) 2024-04-20T21:25:19.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.058+0000 7efe1aa00640 1 -- 172.21.15.86:0/3246716349 learned_addr learned my addr 172.21.15.86:0/3246716349 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:25:19.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.059+0000 7efe1b400640 1 -- 172.21.15.86:0/3246716349 >> v2:172.21.15.144:3300/0 conn(0x7efe1c0d5150 msgr2=0x7efe1c1629f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:19.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.059+0000 7efe1b400640 1 --2- 172.21.15.86:0/3246716349 >> v2:172.21.15.144:3300/0 conn(0x7efe1c0d5150 0x7efe1c1629f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:19.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.059+0000 7efe1b400640 1 -- 172.21.15.86:0/3246716349 >> v1:172.21.15.86:6789/0 conn(0x7efe1c0d4cf0 legacy=0x7efe1c1622e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:19.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.059+0000 7efe1a000640 1 -- 172.21.15.86:0/3246716349 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 758482236 0 0) 0x7efe1c0d1640 con 0x7efe1c0d4cf0 2024-04-20T21:25:19.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.059+0000 7efe1b400640 1 -- 172.21.15.86:0/3246716349 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe1c0cf290 con 0x7efe1c0d0d20 2024-04-20T21:25:19.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.060+0000 7efe1b400640 1 --2- 172.21.15.86:0/3246716349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe1c0d0d20 0x7efe1c0d1100 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7efe1c073d80 tx=0x7efe0402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5fe827cf85b0ab7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:19.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.060+0000 7efe1a000640 1 -- 172.21.15.86:0/3246716349 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe04035020 con 0x7efe1c0d0d20 2024-04-20T21:25:19.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.060+0000 7efe1a000640 1 -- 172.21.15.86:0/3246716349 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe04031830 con 0x7efe1c0d0d20 2024-04-20T21:25:19.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.060+0000 7efe1a000640 1 -- 172.21.15.86:0/3246716349 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe04031c00 con 0x7efe1c0d0d20 2024-04-20T21:25:19.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.061+0000 7efe20a00640 1 -- 172.21.15.86:0/3246716349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe1c0d0d20 msgr2=0x7efe1c0d1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:19.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.061+0000 7efe20a00640 1 --2- 172.21.15.86:0/3246716349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe1c0d0d20 0x7efe1c0d1100 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7efe1c073d80 tx=0x7efe0402b8d0 comp rx=0 tx=0).stop 2024-04-20T21:25:19.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.061+0000 7efe20a00640 1 -- 172.21.15.86:0/3246716349 shutdown_connections 2024-04-20T21:25:19.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.061+0000 7efe20a00640 1 --2- 172.21.15.86:0/3246716349 >> v2:172.21.15.144:3300/0 conn(0x7efe1c0d5150 0x7efe1c1629f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:19.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.061+0000 7efe20a00640 1 --2- 172.21.15.86:0/3246716349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe1c0d0d20 0x7efe1c0d1100 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:19.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.061+0000 7efe20a00640 1 -- 172.21.15.86:0/3246716349 >> 172.21.15.86:0/3246716349 conn(0x7efe1c08bf70 msgr2=0x7efe1c0d45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:19.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.061+0000 7efe20a00640 1 -- 172.21.15.86:0/3246716349 shutdown_connections 2024-04-20T21:25:19.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.061+0000 7efe20a00640 1 -- 172.21.15.86:0/3246716349 wait complete. 2024-04-20T21:25:19.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.063+0000 7efe20a00640 1 Processor -- start 2024-04-20T21:25:19.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.064+0000 7efe20a00640 1 -- start start 2024-04-20T21:25:19.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.065+0000 7efe20a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe1c0d0d20 0x7efe1c15a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:19.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.065+0000 7efe20a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe1c0d4cf0 0x7efe1c1f7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:19.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.065+0000 7efe20a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe1c0d5150 0x7efe1c1591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:19.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.065+0000 7efe20a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efe1c0cfb30 con 0x7efe1c0d0d20 2024-04-20T21:25:19.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.065+0000 7efe20a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efe1c0cf850 con 0x7efe1c0d4cf0 2024-04-20T21:25:19.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.065+0000 7efe20a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7efe1c15fc30 con 0x7efe1c0d5150 2024-04-20T21:25:19.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.065+0000 7efe1b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe1c0d0d20 0x7efe1c15a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:19.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.065+0000 7efe1b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe1c0d0d20 0x7efe1c15a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53820/0 (socket says 172.21.15.86:53820) 2024-04-20T21:25:19.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.065+0000 7efe1b400640 1 -- 172.21.15.86:0/4059459969 learned_addr learned my addr 172.21.15.86:0/4059459969 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:19.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.066+0000 7efe1aa00640 1 --2- 172.21.15.86:0/4059459969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe1c0d4cf0 0x7efe1c1f7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:19.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.066+0000 7efe1b400640 1 -- 172.21.15.86:0/4059459969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe1c0d5150 msgr2=0x7efe1c1591e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:19.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.066+0000 7efe1b400640 1 --2- 172.21.15.86:0/4059459969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe1c0d5150 0x7efe1c1591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:19.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.066+0000 7efe1b400640 1 -- 172.21.15.86:0/4059459969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe1c0d4cf0 msgr2=0x7efe1c1f7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:19.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.066+0000 7efe1b400640 1 --2- 172.21.15.86:0/4059459969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe1c0d4cf0 0x7efe1c1f7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:19.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.066+0000 7efe1b400640 1 -- 172.21.15.86:0/4059459969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe1c1f8750 con 0x7efe1c0d0d20 2024-04-20T21:25:19.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.066+0000 7efe1aa00640 1 --2- 172.21.15.86:0/4059459969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe1c0d4cf0 0x7efe1c1f7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:25:19.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.066+0000 7efe1b400640 1 --2- 172.21.15.86:0/4059459969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe1c0d0d20 0x7efe1c15a080 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7efe04000c00 tx=0x7efe040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:19.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.066+0000 7efe13400640 1 -- 172.21.15.86:0/4059459969 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0403a9b0 con 0x7efe1c0d0d20 2024-04-20T21:25:19.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.067+0000 7efe13400640 1 -- 172.21.15.86:0/4059459969 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe0403ab50 con 0x7efe1c0d0d20 2024-04-20T21:25:19.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.067+0000 7efe13400640 1 -- 172.21.15.86:0/4059459969 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe040443f0 con 0x7efe1c0d0d20 2024-04-20T21:25:19.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.067+0000 7efe20a00640 1 -- 172.21.15.86:0/4059459969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe1c1f8a30 con 0x7efe1c0d0d20 2024-04-20T21:25:19.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.067+0000 7efe20a00640 1 -- 172.21.15.86:0/4059459969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe1c0b9570 con 0x7efe1c0d0d20 2024-04-20T21:25:19.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.068+0000 7efe20a00640 1 -- 172.21.15.86:0/4059459969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe1c04fcf0 con 0x7efe1c0d0d20 2024-04-20T21:25:19.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.069+0000 7efe13400640 1 -- 172.21.15.86:0/4059459969 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efe0404b060 con 0x7efe1c0d0d20 2024-04-20T21:25:19.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.070+0000 7efe13400640 1 --2- 172.21.15.86:0/4059459969 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efdf806c8f0 0x7efdf806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:19.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.070+0000 7efe13400640 1 -- 172.21.15.86:0/4059459969 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7efe040ad8f0 con 0x7efe1c0d0d20 2024-04-20T21:25:19.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.070+0000 7efe1aa00640 1 --2- 172.21.15.86:0/4059459969 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efdf806c8f0 0x7efdf806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:19.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.071+0000 7efe1aa00640 1 --2- 172.21.15.86:0/4059459969 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efdf806c8f0 0x7efdf806edb0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7efe0c007fd0 tx=0x7efe0c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:19.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.075+0000 7efe13400640 1 -- 172.21.15.86:0/4059459969 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7efe0403f070 con 0x7efe1c0d0d20 2024-04-20T21:25:19.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.308+0000 7efe20a00640 1 -- 172.21.15.86:0/4059459969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe1c0d1c20 con 0x7efe1c0d0d20 2024-04-20T21:25:19.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.309+0000 7efe13400640 1 -- 172.21.15.86:0/4059459969 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5207 (secure 0 0 0) 0x7efe0402f2e0 con 0x7efe1c0d0d20 2024-04-20T21:25:19.309 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:25:19.309 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":59,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7506,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/4013863257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4013863257},{"type":"v1","addr":"172.21.15.86:6836","nonce":4013863257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":7509,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2044883073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2044883073},{"type":"v1","addr":"172.21.15.86:6837","nonce":2044883073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7518,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/2351381466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2351381466},{"type":"v1","addr":"172.21.15.144:6834","nonce":2351381466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:24:58.299257+0000","modified":"2024-04-20T21:25:01.355818+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7521":{"gid":7521,"name":"b","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.144:6835/1740492227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":1740492227},{"type":"v1","addr":"172.21.15.144:6835","nonce":1740492227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7521,"qdb_cluster":[7521]},"id":6}]} 2024-04-20T21:25:19.309 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 59 2024-04-20T21:25:19.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.312+0000 7efe20a00640 1 -- 172.21.15.86:0/4059459969 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efdf806c8f0 msgr2=0x7efdf806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:19.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.312+0000 7efe20a00640 1 --2- 172.21.15.86:0/4059459969 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efdf806c8f0 0x7efdf806edb0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7efe0c007fd0 tx=0x7efe0c009040 comp rx=0 tx=0).stop 2024-04-20T21:25:19.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.312+0000 7efe20a00640 1 -- 172.21.15.86:0/4059459969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe1c0d0d20 msgr2=0x7efe1c15a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:19.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.312+0000 7efe20a00640 1 --2- 172.21.15.86:0/4059459969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe1c0d0d20 0x7efe1c15a080 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7efe04000c00 tx=0x7efe040046c0 comp rx=0 tx=0).stop 2024-04-20T21:25:19.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.312+0000 7efe20a00640 1 -- 172.21.15.86:0/4059459969 shutdown_connections 2024-04-20T21:25:19.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.312+0000 7efe20a00640 1 --2- 172.21.15.86:0/4059459969 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efdf806c8f0 0x7efdf806edb0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:19.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.312+0000 7efe20a00640 1 --2- 172.21.15.86:0/4059459969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe1c0d5150 0x7efe1c1591e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:19.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.312+0000 7efe20a00640 1 --2- 172.21.15.86:0/4059459969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe1c0d4cf0 0x7efe1c1f7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:19.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.312+0000 7efe20a00640 1 --2- 172.21.15.86:0/4059459969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe1c0d0d20 0x7efe1c15a080 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:19.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.312+0000 7efe20a00640 1 -- 172.21.15.86:0/4059459969 >> 172.21.15.86:0/4059459969 conn(0x7efe1c08bf70 msgr2=0x7efe1c0b63f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:19.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.312+0000 7efe20a00640 1 -- 172.21.15.86:0/4059459969 shutdown_connections 2024-04-20T21:25:19.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.312+0000 7efe20a00640 1 -- 172.21.15.86:0/4059459969 wait complete. 2024-04-20T21:25:19.329 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-04-20T21:25:19.579 INFO:teuthology.orchestra.run.smithi144.stdout:{"message":"","return_code":0} 2024-04-20T21:25:19.579 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-20T21:25:19.579 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-20T21:25:19.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.783+0000 7f379e800640 1 Processor -- start 2024-04-20T21:25:19.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.784+0000 7f379e800640 1 -- start start 2024-04-20T21:25:19.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.785+0000 7f379e800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f37980ce790 0x7f37980d13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:19.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.785+0000 7f379e800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f37980d1910 0x7f379806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:19.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.785+0000 7f379e800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f379806ddf0 con 0x7f379806d9e0 2024-04-20T21:25:19.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.785+0000 7f379e800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f379809ae10 con 0x7f37980d1910 2024-04-20T21:25:19.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.785+0000 7f379e800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f379809b0d0 con 0x7f37980ce790 2024-04-20T21:25:19.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.785+0000 7f3796a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f37980d1910 0x7f379806d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:19.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.785+0000 7f3796a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f37980d1910 0x7f379806d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53836/0 (socket says 172.21.15.86:53836) 2024-04-20T21:25:19.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.785+0000 7f3796a00640 1 -- 172.21.15.86:0/3291001461 learned_addr learned my addr 172.21.15.86:0/3291001461 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:19.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.786+0000 7f3796a00640 1 -- 172.21.15.86:0/3291001461 >> v2:172.21.15.144:3300/0 conn(0x7f37980ce790 msgr2=0x7f37980d13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:19.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.786+0000 7f3796a00640 1 --2- 172.21.15.86:0/3291001461 >> v2:172.21.15.144:3300/0 conn(0x7f37980ce790 0x7f37980d13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:19.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.786+0000 7f3796a00640 1 -- 172.21.15.86:0/3291001461 >> v1:172.21.15.144:6789/0 conn(0x7f379806d9e0 legacy=0x7f379815e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:19.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.786+0000 7f3796a00640 1 -- 172.21.15.86:0/3291001461 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f379809b3b0 con 0x7f37980d1910 2024-04-20T21:25:19.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.787+0000 7f3796a00640 1 --2- 172.21.15.86:0/3291001461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37980d1910 0x7f379806d4a0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f3798073500 tx=0x7f378002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=267299aa41c313e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:19.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.787+0000 7f3796000640 1 -- 172.21.15.86:0/3291001461 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378002ca80 con 0x7f37980d1910 2024-04-20T21:25:19.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.787+0000 7f3796000640 1 -- 172.21.15.86:0/3291001461 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f378002cc20 con 0x7f37980d1910 2024-04-20T21:25:19.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.787+0000 7f3796000640 1 -- 172.21.15.86:0/3291001461 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37800343f0 con 0x7f37980d1910 2024-04-20T21:25:19.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.788+0000 7f379e800640 1 -- 172.21.15.86:0/3291001461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37980d1910 msgr2=0x7f379806d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:19.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.788+0000 7f379e800640 1 --2- 172.21.15.86:0/3291001461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37980d1910 0x7f379806d4a0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f3798073500 tx=0x7f378002bee0 comp rx=0 tx=0).stop 2024-04-20T21:25:19.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.788+0000 7f379e800640 1 -- 172.21.15.86:0/3291001461 shutdown_connections 2024-04-20T21:25:19.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.788+0000 7f379e800640 1 --2- 172.21.15.86:0/3291001461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37980d1910 0x7f379806d4a0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:19.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.788+0000 7f379e800640 1 --2- 172.21.15.86:0/3291001461 >> v2:172.21.15.144:3300/0 conn(0x7f37980ce790 0x7f37980d13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:19.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.788+0000 7f379e800640 1 -- 172.21.15.86:0/3291001461 >> 172.21.15.86:0/3291001461 conn(0x7f37980b3fe0 msgr2=0x7f37980b43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:19.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.788+0000 7f379e800640 1 -- 172.21.15.86:0/3291001461 shutdown_connections 2024-04-20T21:25:19.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.788+0000 7f379e800640 1 -- 172.21.15.86:0/3291001461 wait complete. 2024-04-20T21:25:19.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.790+0000 7f379e800640 1 Processor -- start 2024-04-20T21:25:19.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.790+0000 7f379e800640 1 -- start start 2024-04-20T21:25:19.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.791+0000 7f379e800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f379806d9e0 0x7f3798147c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:19.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.791+0000 7f379e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37980ce790 0x7f3798156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:19.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.791+0000 7f379e800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37980ca610 0x7f37980caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:19.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.791+0000 7f379e800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f37980ceb70 con 0x7f37980ce790 2024-04-20T21:25:19.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.791+0000 7f379e800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f379809b970 con 0x7f379806d9e0 2024-04-20T21:25:19.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.791+0000 7f379e800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f37980cecf0 con 0x7f37980ca610 2024-04-20T21:25:19.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.791+0000 7f3796a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37980ce790 0x7f3798156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:19.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.791+0000 7f3796a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37980ce790 0x7f3798156ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53842/0 (socket says 172.21.15.86:53842) 2024-04-20T21:25:19.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.791+0000 7f3796a00640 1 -- 172.21.15.86:0/4282066989 learned_addr learned my addr 172.21.15.86:0/4282066989 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:19.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.792+0000 7f3797400640 1 --2- 172.21.15.86:0/4282066989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f379806d9e0 0x7f3798147c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:19.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.792+0000 7f3797e00640 1 --2- 172.21.15.86:0/4282066989 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37980ca610 0x7f37980caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:19.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.792+0000 7f3796a00640 1 -- 172.21.15.86:0/4282066989 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37980ca610 msgr2=0x7f37980caaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:19.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.792+0000 7f3796a00640 1 --2- 172.21.15.86:0/4282066989 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37980ca610 0x7f37980caaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:19.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.792+0000 7f3796a00640 1 -- 172.21.15.86:0/4282066989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f379806d9e0 msgr2=0x7f3798147c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:19.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.792+0000 7f3796a00640 1 --2- 172.21.15.86:0/4282066989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f379806d9e0 0x7f3798147c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:19.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.792+0000 7f3796a00640 1 -- 172.21.15.86:0/4282066989 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3798148be0 con 0x7f37980ce790 2024-04-20T21:25:19.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.792+0000 7f3796a00640 1 --2- 172.21.15.86:0/4282066989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37980ce790 0x7f3798156ab0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f37800093e0 tx=0x7f3780004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:19.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.793+0000 7f378f400640 1 -- 172.21.15.86:0/4282066989 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3780044020 con 0x7f37980ce790 2024-04-20T21:25:19.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.793+0000 7f378f400640 1 -- 172.21.15.86:0/4282066989 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3780040670 con 0x7f37980ce790 2024-04-20T21:25:19.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.793+0000 7f378f400640 1 -- 172.21.15.86:0/4282066989 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3780040a60 con 0x7f37980ce790 2024-04-20T21:25:19.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.793+0000 7f379e800640 1 -- 172.21.15.86:0/4282066989 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37980cb0d0 con 0x7f37980ce790 2024-04-20T21:25:19.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.793+0000 7f379e800640 1 -- 172.21.15.86:0/4282066989 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37980cb690 con 0x7f37980ce790 2024-04-20T21:25:19.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.794+0000 7f379e800640 1 -- 172.21.15.86:0/4282066989 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37980689e0 con 0x7f37980ce790 2024-04-20T21:25:19.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.795+0000 7f378f400640 1 -- 172.21.15.86:0/4282066989 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3780047020 con 0x7f37980ce790 2024-04-20T21:25:19.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.796+0000 7f378f400640 1 --2- 172.21.15.86:0/4282066989 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f377406c8f0 0x7f377406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:19.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.796+0000 7f378f400640 1 -- 172.21.15.86:0/4282066989 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7f37800ae890 con 0x7f37980ce790 2024-04-20T21:25:19.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.796+0000 7f3797400640 1 --2- 172.21.15.86:0/4282066989 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f377406c8f0 0x7f377406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:19.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.797+0000 7f3797400640 1 --2- 172.21.15.86:0/4282066989 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f377406c8f0 0x7f377406edb0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f378800b810 tx=0x7f3788008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:19.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:19.800+0000 7f378f400640 1 -- 172.21.15.86:0/4282066989 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f378004e030 con 0x7f37980ce790 2024-04-20T21:25:20.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.039+0000 7f379e800640 1 -- 172.21.15.86:0/4282066989 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f3798156ff0 con 0x7f37980ce790 2024-04-20T21:25:20.039 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-04-20T21:25:20.040+0000 7f5bb9c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T21:25:20.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.043+0000 7f378f400640 1 -- 172.21.15.86:0/4282066989 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v60) ==== 161+0+0 (secure 0 0 0) 0x7f379806ddf0 con 0x7f37980ce790 2024-04-20T21:25:20.042 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-20T21:25:20.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.047+0000 7f379e800640 1 -- 172.21.15.86:0/4282066989 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f377406c8f0 msgr2=0x7f377406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:20.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.047+0000 7f379e800640 1 --2- 172.21.15.86:0/4282066989 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f377406c8f0 0x7f377406edb0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f378800b810 tx=0x7f3788008040 comp rx=0 tx=0).stop 2024-04-20T21:25:20.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.047+0000 7f379e800640 1 -- 172.21.15.86:0/4282066989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37980ce790 msgr2=0x7f3798156ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:20.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.047+0000 7f379e800640 1 --2- 172.21.15.86:0/4282066989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37980ce790 0x7f3798156ab0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f37800093e0 tx=0x7f3780004a40 comp rx=0 tx=0).stop 2024-04-20T21:25:20.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.047+0000 7f379e800640 1 -- 172.21.15.86:0/4282066989 shutdown_connections 2024-04-20T21:25:20.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.047+0000 7f379e800640 1 --2- 172.21.15.86:0/4282066989 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f377406c8f0 0x7f377406edb0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:20.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.047+0000 7f379e800640 1 --2- 172.21.15.86:0/4282066989 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f37980ca610 0x7f37980caaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:20.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.047+0000 7f379e800640 1 --2- 172.21.15.86:0/4282066989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f37980ce790 0x7f3798156ab0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:20.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.047+0000 7f379e800640 1 --2- 172.21.15.86:0/4282066989 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f379806d9e0 0x7f3798147c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:20.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.047+0000 7f379e800640 1 -- 172.21.15.86:0/4282066989 >> 172.21.15.86:0/4282066989 conn(0x7f37980b3fe0 msgr2=0x7f3798160ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:20.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.047+0000 7f379e800640 1 -- 172.21.15.86:0/4282066989 shutdown_connections 2024-04-20T21:25:20.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.047+0000 7f379e800640 1 -- 172.21.15.86:0/4282066989 wait complete. 2024-04-20T21:25:20.065 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 10000000000.00000000 2024-04-20T21:25:20.074 INFO:tasks.ceph.mds.b.smithi144.stderr: -4070> 2024-04-20T21:25:04.798+0000 7f9ad0c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T21:25:20.161 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2024-04-20T21:25:20.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.170+0000 7fcc62a4df40 1 Processor -- start 2024-04-20T21:25:20.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.170+0000 7fcc62a4df40 1 -- start start 2024-04-20T21:25:20.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.171+0000 7fcc62a4df40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ae1472cf0 0x562ae151f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:20.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.171+0000 7fcc62a4df40 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ae151e9d0 0x562ae1510190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:20.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.171+0000 7fcc62a4df40 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562ae151edb0 0x562ae15106d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:20.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.171+0000 7fcc62a4df40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562ae13eacc0 con 0x562ae1472cf0 2024-04-20T21:25:20.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.171+0000 7fcc62a4df40 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562ae13eed00 con 0x562ae151e9d0 2024-04-20T21:25:20.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.171+0000 7fcc62a4df40 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x562ae1442700 con 0x562ae151edb0 2024-04-20T21:25:20.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.171+0000 7fcc5f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ae1472cf0 0x562ae151f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:20.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.171+0000 7fcc5f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ae1472cf0 0x562ae151f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53872/0 (socket says 172.21.15.86:53872) 2024-04-20T21:25:20.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.171+0000 7fcc5f400640 1 -- 172.21.15.86:0/663991460 learned_addr learned my addr 172.21.15.86:0/663991460 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:20.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.172+0000 7fcc5ea00640 1 --2- 172.21.15.86:0/663991460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ae151e9d0 0x562ae1510190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:20.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.172+0000 7fcc5fe00640 1 --2- 172.21.15.86:0/663991460 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562ae151edb0 0x562ae15106d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:20.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.172+0000 7fcc5f400640 1 -- 172.21.15.86:0/663991460 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562ae151edb0 msgr2=0x562ae15106d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:20.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.172+0000 7fcc5f400640 1 --2- 172.21.15.86:0/663991460 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562ae151edb0 0x562ae15106d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:20.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.172+0000 7fcc5f400640 1 -- 172.21.15.86:0/663991460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ae151e9d0 msgr2=0x562ae1510190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:20.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.172+0000 7fcc5f400640 1 --2- 172.21.15.86:0/663991460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ae151e9d0 0x562ae1510190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:20.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.172+0000 7fcc5f400640 1 -- 172.21.15.86:0/663991460 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ae151a5c0 con 0x562ae1472cf0 2024-04-20T21:25:20.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.172+0000 7fcc5ea00640 1 --2- 172.21.15.86:0/663991460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ae151e9d0 0x562ae1510190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:25:20.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.172+0000 7fcc5fe00640 1 --2- 172.21.15.86:0/663991460 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562ae151edb0 0x562ae15106d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:25:20.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.172+0000 7fcc5f400640 1 --2- 172.21.15.86:0/663991460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ae1472cf0 0x562ae151f410 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fcc5000b2c0 tx=0x7fcc5000b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b383280801686fa3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:20.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.172+0000 7fcc5d600640 1 -- 172.21.15.86:0/663991460 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc50008070 con 0x562ae1472cf0 2024-04-20T21:25:20.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.172+0000 7fcc5d600640 1 -- 172.21.15.86:0/663991460 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc5000d500 con 0x562ae1472cf0 2024-04-20T21:25:20.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.172+0000 7fcc5d600640 1 -- 172.21.15.86:0/663991460 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc5000d930 con 0x562ae1472cf0 2024-04-20T21:25:20.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.173+0000 7fcc62a4df40 1 -- 172.21.15.86:0/663991460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ae1472cf0 msgr2=0x562ae151f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:20.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.173+0000 7fcc62a4df40 1 --2- 172.21.15.86:0/663991460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ae1472cf0 0x562ae151f410 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fcc5000b2c0 tx=0x7fcc5000b780 comp rx=0 tx=0).stop 2024-04-20T21:25:20.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.173+0000 7fcc62a4df40 1 -- 172.21.15.86:0/663991460 shutdown_connections 2024-04-20T21:25:20.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.173+0000 7fcc62a4df40 1 --2- 172.21.15.86:0/663991460 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562ae151edb0 0x562ae15106d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:20.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.173+0000 7fcc62a4df40 1 --2- 172.21.15.86:0/663991460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ae151e9d0 0x562ae1510190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:20.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.173+0000 7fcc62a4df40 1 --2- 172.21.15.86:0/663991460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ae1472cf0 0x562ae151f410 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:20.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.173+0000 7fcc62a4df40 1 -- 172.21.15.86:0/663991460 >> 172.21.15.86:0/663991460 conn(0x562ae1441220 msgr2=0x562ae1446750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:20.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.173+0000 7fcc62a4df40 1 -- 172.21.15.86:0/663991460 shutdown_connections 2024-04-20T21:25:20.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.173+0000 7fcc62a4df40 1 -- 172.21.15.86:0/663991460 wait complete. 2024-04-20T21:25:20.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.174+0000 7fcc62a4df40 1 Processor -- start 2024-04-20T21:25:20.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.174+0000 7fcc62a4df40 1 -- start start 2024-04-20T21:25:20.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.175+0000 7fcc62a4df40 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ae1472cf0 0x562ae146f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:20.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.175+0000 7fcc62a4df40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ae151e9d0 0x562ae150bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:20.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.175+0000 7fcc62a4df40 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562ae151edb0 0x562ae1446630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:20.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.175+0000 7fcc62a4df40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562ae151a320 con 0x562ae151e9d0 2024-04-20T21:25:20.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.175+0000 7fcc62a4df40 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562ae1510c10 con 0x562ae1472cf0 2024-04-20T21:25:20.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.175+0000 7fcc62a4df40 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x562ae148bf70 con 0x562ae151edb0 2024-04-20T21:25:20.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.175+0000 7fcc5ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ae151e9d0 0x562ae150bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:20.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.175+0000 7fcc5ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ae151e9d0 0x562ae150bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53878/0 (socket says 172.21.15.86:53878) 2024-04-20T21:25:20.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.175+0000 7fcc5ea00640 1 -- 172.21.15.86:0/2674006547 learned_addr learned my addr 172.21.15.86:0/2674006547 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:20.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.175+0000 7fcc5fe00640 1 --2- 172.21.15.86:0/2674006547 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562ae151edb0 0x562ae1446630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:20.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.175+0000 7fcc5f400640 1 --2- 172.21.15.86:0/2674006547 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ae1472cf0 0x562ae146f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:20.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.175+0000 7fcc5ea00640 1 -- 172.21.15.86:0/2674006547 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562ae151edb0 msgr2=0x562ae1446630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:20.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.175+0000 7fcc5ea00640 1 --2- 172.21.15.86:0/2674006547 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562ae151edb0 0x562ae1446630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:20.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.175+0000 7fcc5ea00640 1 -- 172.21.15.86:0/2674006547 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ae1472cf0 msgr2=0x562ae146f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:20.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.175+0000 7fcc5ea00640 1 --2- 172.21.15.86:0/2674006547 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ae1472cf0 0x562ae146f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:20.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.175+0000 7fcc5ea00640 1 -- 172.21.15.86:0/2674006547 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ae151fb70 con 0x562ae151e9d0 2024-04-20T21:25:20.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.176+0000 7fcc5ea00640 1 --2- 172.21.15.86:0/2674006547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ae151e9d0 0x562ae150bc40 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c058040 tx=0x7fcc4c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:20.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.176+0000 7fcc57e00640 1 -- 172.21.15.86:0/2674006547 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c05a040 con 0x562ae151e9d0 2024-04-20T21:25:20.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.176+0000 7fcc57e00640 1 -- 172.21.15.86:0/2674006547 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc4c067070 con 0x562ae151e9d0 2024-04-20T21:25:20.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.176+0000 7fcc62a4df40 1 -- 172.21.15.86:0/2674006547 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562ae146fdf0 con 0x562ae151e9d0 2024-04-20T21:25:20.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.176+0000 7fcc57e00640 1 -- 172.21.15.86:0/2674006547 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c060470 con 0x562ae151e9d0 2024-04-20T21:25:20.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.176+0000 7fcc62a4df40 1 -- 172.21.15.86:0/2674006547 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ae15197b0 con 0x562ae151e9d0 2024-04-20T21:25:20.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.178+0000 7fcc57e00640 1 -- 172.21.15.86:0/2674006547 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcc4c060610 con 0x562ae151e9d0 2024-04-20T21:25:20.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.179+0000 7fcc57e00640 1 --2- 172.21.15.86:0/2674006547 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcc4006b300 0x7fcc4006d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:20.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.179+0000 7fcc57e00640 1 -- 172.21.15.86:0/2674006547 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5795+0+0 (secure 0 0 0) 0x7fcc4c0cedb0 con 0x562ae151e9d0 2024-04-20T21:25:20.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.179+0000 7fcc5f400640 1 --2- 172.21.15.86:0/2674006547 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcc4006b300 0x7fcc4006d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:20.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.180+0000 7fcc62a4df40 1 --2- 172.21.15.86:0/2674006547 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x562ae15b51e0 0x562ae150fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:20.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.180+0000 7fcc62a4df40 1 -- 172.21.15.86:0/2674006547 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:1 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e86) -- 0x562ae151bb00 con 0x562ae15b51e0 2024-04-20T21:25:20.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.180+0000 7fcc5fe00640 1 --2- 172.21.15.86:0/2674006547 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x562ae15b51e0 0x562ae150fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:20.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.180+0000 7fcc5f400640 1 --2- 172.21.15.86:0/2674006547 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcc4006b300 0x7fcc4006d7c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fcc5000b2c0 tx=0x7fcc50009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:20.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.181+0000 7fcc5fe00640 1 --2- 172.21.15.86:0/2674006547 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x562ae15b51e0 0x562ae150fd60 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:20.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.185+0000 7fcc5fe00640 1 -- 172.21.15.86:0/2674006547 <== osd.0 v2:172.21.15.86:6800/2471076035 1 ==== osd_op_reply(1 10000000000.00000000 [delete] v86'11 uv11 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fcc58001510 con 0x562ae15b51e0 2024-04-20T21:25:20.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.185+0000 7fcc62a4df40 1 -- 172.21.15.86:0/2674006547 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x562ae15b51e0 msgr2=0x562ae150fd60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:20.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.185+0000 7fcc62a4df40 1 --2- 172.21.15.86:0/2674006547 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x562ae15b51e0 0x562ae150fd60 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:20.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.185+0000 7fcc62a4df40 1 -- 172.21.15.86:0/2674006547 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcc4006b300 msgr2=0x7fcc4006d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:20.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.185+0000 7fcc62a4df40 1 --2- 172.21.15.86:0/2674006547 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcc4006b300 0x7fcc4006d7c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fcc5000b2c0 tx=0x7fcc50009000 comp rx=0 tx=0).stop 2024-04-20T21:25:20.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.185+0000 7fcc62a4df40 1 -- 172.21.15.86:0/2674006547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ae151e9d0 msgr2=0x562ae150bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:20.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.185+0000 7fcc62a4df40 1 --2- 172.21.15.86:0/2674006547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ae151e9d0 0x562ae150bc40 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c058040 tx=0x7fcc4c002fb0 comp rx=0 tx=0).stop 2024-04-20T21:25:20.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.185+0000 7fcc5fe00640 1 -- 172.21.15.86:0/2674006547 reap_dead start 2024-04-20T21:25:20.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.185+0000 7fcc62a4df40 1 -- 172.21.15.86:0/2674006547 shutdown_connections 2024-04-20T21:25:20.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.185+0000 7fcc62a4df40 1 -- 172.21.15.86:0/2674006547 >> 172.21.15.86:0/2674006547 conn(0x562ae1441220 msgr2=0x562ae1441600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:20.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.185+0000 7fcc62a4df40 1 -- 172.21.15.86:0/2674006547 shutdown_connections 2024-04-20T21:25:20.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.185+0000 7fcc62a4df40 1 -- 172.21.15.86:0/2674006547 wait complete. 2024-04-20T21:25:20.189 DEBUG:teuthology.orchestra.run.smithi086:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset 2024-04-20T21:25:20.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.273+0000 7f77f6446280 1 -- start start 2024-04-20T21:25:20.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.273+0000 7f77f6446280 1 Processor -- start 2024-04-20T21:25:20.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.274+0000 7f77f6446280 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560cdd9d4c00 0x560cde73c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:20.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.274+0000 7f77f6446280 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560cdd9d5800 0x560cde73c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:20.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.274+0000 7f77f6446280 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560cdd9d5c00 0x560cde73cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:20.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.274+0000 7f77f6446280 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560cde894180 con 0x560cdd9d4c00 2024-04-20T21:25:20.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.274+0000 7f77f6446280 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560cde894300 con 0x560cdd9d5800 2024-04-20T21:25:20.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.274+0000 7f77f6446280 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x560cde894480 con 0x560cdd9d5c00 2024-04-20T21:25:20.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.274+0000 7f77f3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560cdd9d4c00 0x560cde73c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:20.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.275+0000 7f77f3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560cdd9d4c00 0x560cde73c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53896/0 (socket says 172.21.15.86:53896) 2024-04-20T21:25:20.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.275+0000 7f77f3e00640 1 -- 172.21.15.86:0/498112300 learned_addr learned my addr 172.21.15.86:0/498112300 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:20.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.275+0000 7f77f3e00640 1 -- 172.21.15.86:0/498112300 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560cdd9d5c00 msgr2=0x560cde73cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:20.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.275+0000 7f77f3e00640 1 --2- 172.21.15.86:0/498112300 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560cdd9d5c00 0x560cde73cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:20.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.275+0000 7f77f3400640 1 --2- 172.21.15.86:0/498112300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560cdd9d5800 0x560cde73c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:20.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.275+0000 7f77f3e00640 1 -- 172.21.15.86:0/498112300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560cdd9d5800 msgr2=0x560cde73c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:20.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.275+0000 7f77f3e00640 1 --2- 172.21.15.86:0/498112300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560cdd9d5800 0x560cde73c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:20.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.275+0000 7f77f3e00640 1 -- 172.21.15.86:0/498112300 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cdda08780 con 0x560cdd9d4c00 2024-04-20T21:25:20.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.275+0000 7f77f3e00640 1 --2- 172.21.15.86:0/498112300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560cdd9d4c00 0x560cde73c000 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x560cde7fd4a0 tx=0x560cde770f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f38b6445ba9e9a30 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:20.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.275+0000 7f77f0c00640 1 -- 172.21.15.86:0/498112300 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x560cdd9fa680 con 0x560cdd9d4c00 2024-04-20T21:25:20.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.275+0000 7f77f0c00640 1 -- 172.21.15.86:0/498112300 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x560cdda00700 con 0x560cdd9d4c00 2024-04-20T21:25:20.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.275+0000 7f77f0c00640 1 -- 172.21.15.86:0/498112300 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x560cdd9fab60 con 0x560cdd9d4c00 2024-04-20T21:25:20.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.276+0000 7f77f6446280 1 -- 172.21.15.86:0/498112300 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cdda094a0 con 0x560cdd9d4c00 2024-04-20T21:25:20.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.276+0000 7f77f6446280 1 -- 172.21.15.86:0/498112300 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=0}) -- 0x560cdda08000 con 0x560cdd9d4c00 2024-04-20T21:25:20.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.276+0000 7f77f6446280 4 waiting for MDS map... 2024-04-20T21:25:20.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.276+0000 7f77f0c00640 1 -- 172.21.15.86:0/498112300 <== mon.0 v2:172.21.15.86:3300/0 4 ==== osd_map(86..86 src has 1..86) ==== 5795+0+0 (secure 0 0 0) 0x560cde8a0600 con 0x560cdd9d4c00 2024-04-20T21:25:20.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.276+0000 7f77f0c00640 1 -- 172.21.15.86:0/498112300 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 60) ==== 2886+0+0 (secure 0 0 0) 0x560cdda2f180 con 0x560cdd9d4c00 2024-04-20T21:25:20.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.276+0000 7f77f6446280 4 Got MDS map 60 2024-04-20T21:25:20.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.276+0000 7f77f6446280 10 main: JournalTool::main 2024-04-20T21:25:20.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.276+0000 7f77f6446280 4 main: JournalTool: connecting to RADOS... 2024-04-20T21:25:20.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.277+0000 7f77f6446280 1 Processor -- start 2024-04-20T21:25:20.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.277+0000 7f77f6446280 1 -- start start 2024-04-20T21:25:20.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.278+0000 7f77f6446280 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560cde8a3400 0x560cde73d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:20.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.278+0000 7f77f6446280 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560cde8a3800 0x560cde73d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:20.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.278+0000 7f77f6446280 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560cde8a3c00 0x560cde73db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:20.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.278+0000 7f77f6446280 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560cde894600 con 0x560cde8a3400 2024-04-20T21:25:20.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.278+0000 7f77f6446280 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560cde894780 con 0x560cde8a3800 2024-04-20T21:25:20.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.278+0000 7f77f6446280 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x560cde894900 con 0x560cde8a3c00 2024-04-20T21:25:20.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.278+0000 7f77f4800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560cde8a3400 0x560cde73d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:20.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.278+0000 7f77f4800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560cde8a3400 0x560cde73d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53910/0 (socket says 172.21.15.86:53910) 2024-04-20T21:25:20.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.278+0000 7f77f4800640 1 -- 172.21.15.86:0/3048564362 learned_addr learned my addr 172.21.15.86:0/3048564362 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:20.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.278+0000 7f77f4800640 1 -- 172.21.15.86:0/3048564362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560cde8a3c00 msgr2=0x560cde73db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:20.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.278+0000 7f77f4800640 1 --2- 172.21.15.86:0/3048564362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560cde8a3c00 0x560cde73db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:20.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.278+0000 7f77f4800640 1 -- 172.21.15.86:0/3048564362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560cde8a3800 msgr2=0x560cde73d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:20.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.278+0000 7f77f4800640 1 --2- 172.21.15.86:0/3048564362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560cde8a3800 0x560cde73d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:20.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.278+0000 7f77f4800640 1 -- 172.21.15.86:0/3048564362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cdda083c0 con 0x560cde8a3400 2024-04-20T21:25:20.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.279+0000 7f77f4800640 1 --2- 172.21.15.86:0/3048564362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560cde8a3400 0x560cde73d080 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x560cde8b82d0 tx=0x560cde8c0d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19e1723e58a15dc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:20.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.279+0000 7f77ee400640 1 -- 172.21.15.86:0/3048564362 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x560cdd9fa680 con 0x560cde8a3400 2024-04-20T21:25:20.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.279+0000 7f77ee400640 1 -- 172.21.15.86:0/3048564362 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x560cdda00fc0 con 0x560cde8a3400 2024-04-20T21:25:20.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.279+0000 7f77ee400640 1 -- 172.21.15.86:0/3048564362 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x560cdd9fab60 con 0x560cde8a3400 2024-04-20T21:25:20.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.279+0000 7f77f6446280 1 -- 172.21.15.86:0/3048564362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560cde8a3400 msgr2=0x560cde73d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:20.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.279+0000 7f77f6446280 1 --2- 172.21.15.86:0/3048564362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560cde8a3400 0x560cde73d080 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x560cde8b82d0 tx=0x560cde8c0d20 comp rx=0 tx=0).stop 2024-04-20T21:25:20.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.280+0000 7f77f6446280 1 -- 172.21.15.86:0/3048564362 shutdown_connections 2024-04-20T21:25:20.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.280+0000 7f77f6446280 1 --2- 172.21.15.86:0/3048564362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560cde8a3c00 0x560cde73db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:20.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.280+0000 7f77f6446280 1 --2- 172.21.15.86:0/3048564362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560cde8a3800 0x560cde73d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:20.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.280+0000 7f77f6446280 1 --2- 172.21.15.86:0/3048564362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560cde8a3400 0x560cde73d080 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:20.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.280+0000 7f77f6446280 1 -- 172.21.15.86:0/3048564362 >> 172.21.15.86:0/3048564362 conn(0x560cde8a3000 msgr2=0x560cddab9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:20.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.280+0000 7f77f6446280 1 -- 172.21.15.86:0/3048564362 shutdown_connections 2024-04-20T21:25:20.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.280+0000 7f77f6446280 1 -- 172.21.15.86:0/3048564362 wait complete. 2024-04-20T21:25:20.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.280+0000 7f77f6446280 1 Processor -- start 2024-04-20T21:25:20.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.280+0000 7f77f6446280 1 -- start start 2024-04-20T21:25:20.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.281+0000 7f77f6446280 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560cde8a3400 0x560cde73d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:20.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.281+0000 7f77f6446280 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560cde8a3800 0x560cde73d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:20.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.281+0000 7f77f6446280 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560cde8a3c00 0x560cde73db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:20.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.281+0000 7f77f6446280 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560cde895c80 con 0x560cde8a3800 2024-04-20T21:25:20.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.281+0000 7f77f6446280 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560cde895b00 con 0x560cde8a3400 2024-04-20T21:25:20.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.281+0000 7f77f6446280 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x560cde895980 con 0x560cde8a3c00 2024-04-20T21:25:20.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.281+0000 7f77f3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560cde8a3800 0x560cde73d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:20.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.281+0000 7f77f3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560cde8a3800 0x560cde73d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53916/0 (socket says 172.21.15.86:53916) 2024-04-20T21:25:20.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.281+0000 7f77f3e00640 1 -- 172.21.15.86:0/1009521896 learned_addr learned my addr 172.21.15.86:0/1009521896 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:20.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.282+0000 7f77f3e00640 1 -- 172.21.15.86:0/1009521896 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560cde8a3c00 msgr2=0x560cde73db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:20.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.282+0000 7f77f3e00640 1 --2- 172.21.15.86:0/1009521896 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560cde8a3c00 0x560cde73db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:20.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.282+0000 7f77f3e00640 1 -- 172.21.15.86:0/1009521896 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560cde8a3400 msgr2=0x560cde73d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:20.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.282+0000 7f77f4800640 1 --2- 172.21.15.86:0/1009521896 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560cde8a3400 0x560cde73d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:20.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.282+0000 7f77f3e00640 1 --2- 172.21.15.86:0/1009521896 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560cde8a3400 0x560cde73d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:20.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.282+0000 7f77f3e00640 1 -- 172.21.15.86:0/1009521896 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cdda08780 con 0x560cde8a3800 2024-04-20T21:25:20.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.282+0000 7f77f3e00640 1 --2- 172.21.15.86:0/1009521896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560cde8a3800 0x560cde73d600 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x560cde8b95c0 tx=0x560cde7700c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:20.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.282+0000 7f77ec600640 1 -- 172.21.15.86:0/1009521896 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x560cdd9fa820 con 0x560cde8a3800 2024-04-20T21:25:20.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.282+0000 7f77ec600640 1 -- 172.21.15.86:0/1009521896 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x560cdda00e00 con 0x560cde8a3800 2024-04-20T21:25:20.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.282+0000 7f77ec600640 1 -- 172.21.15.86:0/1009521896 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x560cdd9fa680 con 0x560cde8a3800 2024-04-20T21:25:20.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.282+0000 7f77f6446280 1 -- 172.21.15.86:0/1009521896 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560cdda094a0 con 0x560cde8a3800 2024-04-20T21:25:20.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.283+0000 7f77f6446280 1 -- 172.21.15.86:0/1009521896 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cdda08960 con 0x560cde8a3800 2024-04-20T21:25:20.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.283+0000 7f77f6446280 4 main: JournalTool: resolving pool 12 2024-04-20T21:25:20.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.284+0000 7f77ec600640 1 -- 172.21.15.86:0/1009521896 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x560cde8d4c00 con 0x560cde8a3800 2024-04-20T21:25:20.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.285+0000 7f77ec600640 1 --2- 172.21.15.86:0/1009521896 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x560cde91fc00 0x560cde73e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:20.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.285+0000 7f77ec600640 1 -- 172.21.15.86:0/1009521896 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5795+0+0 (secure 0 0 0) 0x560cde924c00 con 0x560cde8a3800 2024-04-20T21:25:20.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.285+0000 7f77f6446280 4 main: JournalTool: creating IoCtx.. 2024-04-20T21:25:20.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.285+0000 7f77f6446280 4 main: Executing for rank 0 2024-04-20T21:25:20.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.285+0000 7f77f4800640 1 --2- 172.21.15.86:0/1009521896 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x560cde91fc00 0x560cde73e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:20.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.286+0000 7f77f4800640 1 --2- 172.21.15.86:0/1009521896 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x560cde91fc00 0x560cde73e100 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x560cde8b82d0 tx=0x560cde8c0c00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:20.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.286+0000 7f77f6446280 1 -- start start 2024-04-20T21:25:20.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.286+0000 7f77f6446280 1 Processor -- start 2024-04-20T21:25:20.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.287+0000 7f77f6446280 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560cddda4400 0x560cde73e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:20.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.287+0000 7f77f6446280 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560cddda4800 0x560cde73ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:20.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.287+0000 7f77f6446280 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560cddda4c00 0x560cde73f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:20.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.287+0000 7f77f6446280 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560cde895680 con 0x560cddda4800 2024-04-20T21:25:20.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.287+0000 7f77f6446280 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560cde894600 con 0x560cddda4400 2024-04-20T21:25:20.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.287+0000 7f77f6446280 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x560cde894780 con 0x560cddda4c00 2024-04-20T21:25:20.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.287+0000 7f77f3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560cddda4800 0x560cde73ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:20.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.287+0000 7f77f3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560cddda4800 0x560cde73ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53922/0 (socket says 172.21.15.86:53922) 2024-04-20T21:25:20.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.287+0000 7f77f3e00640 1 -- 172.21.15.86:0/3724628938 learned_addr learned my addr 172.21.15.86:0/3724628938 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:20.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.287+0000 7f77f3e00640 1 -- 172.21.15.86:0/3724628938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560cddda4c00 msgr2=0x560cde73f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:20.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.287+0000 7f77f3e00640 1 --2- 172.21.15.86:0/3724628938 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560cddda4c00 0x560cde73f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:20.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.287+0000 7f77f3e00640 1 -- 172.21.15.86:0/3724628938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560cddda4400 msgr2=0x560cde73e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:20.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.287+0000 7f77f3e00640 1 --2- 172.21.15.86:0/3724628938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560cddda4400 0x560cde73e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:20.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.287+0000 7f77f3e00640 1 -- 172.21.15.86:0/3724628938 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cdda09680 con 0x560cddda4800 2024-04-20T21:25:20.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.288+0000 7f77f3e00640 1 --2- 172.21.15.86:0/3724628938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560cddda4800 0x560cde73ec00 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x560cde95b620 tx=0x560cddcd78c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afff8332a0024ad server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:20.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.288+0000 7f77e8a00640 1 -- 172.21.15.86:0/3724628938 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x560cddcef6c0 con 0x560cddda4800 2024-04-20T21:25:20.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.288+0000 7f77e8a00640 1 -- 172.21.15.86:0/3724628938 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x560cdda00c40 con 0x560cddda4800 2024-04-20T21:25:20.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.288+0000 7f77e8a00640 1 -- 172.21.15.86:0/3724628938 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x560cddcef860 con 0x560cddda4800 2024-04-20T21:25:20.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.288+0000 7f77f6446280 1 -- 172.21.15.86:0/3724628938 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cdda08000 con 0x560cddda4800 2024-04-20T21:25:20.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.288+0000 7f77f6446280 1 -- 172.21.15.86:0/3724628938 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=0}) -- 0x560cdda08780 con 0x560cddda4800 2024-04-20T21:25:20.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.288+0000 7f77f6446280 4 waiting for MDS map... 2024-04-20T21:25:20.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.288+0000 7f77e8a00640 1 -- 172.21.15.86:0/3724628938 <== mon.0 v2:172.21.15.86:3300/0 4 ==== osd_map(86..86 src has 1..86) ==== 5795+0+0 (secure 0 0 0) 0x560cdddb4600 con 0x560cddda4800 2024-04-20T21:25:20.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.288+0000 7f77e8a00640 1 -- 172.21.15.86:0/3724628938 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 60) ==== 2886+0+0 (secure 0 0 0) 0x560cdda2f500 con 0x560cddda4800 2024-04-20T21:25:20.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.288+0000 7f77f6446280 4 Got MDS map 60 2024-04-20T21:25:20.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.288+0000 7f77f6446280 1 --2- 172.21.15.86:0/3724628938 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x560cddda5000 0x560cde73f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:20.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.289+0000 7f77f6446280 1 -- 172.21.15.86:0/3724628938 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:1 12.f 12:f1f69726:::400.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x560cdddbe800 con 0x560cddda5000 2024-04-20T21:25:20.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.289+0000 7f77f4800640 1 --2- 172.21.15.86:0/3724628938 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x560cddda5000 0x560cde73f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:20.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.290+0000 7f77f4800640 1 --2- 172.21.15.86:0/3724628938 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x560cddda5000 0x560cde73f700 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=8773645633b55957 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:20.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.291+0000 7f77f4800640 1 -- 172.21.15.86:0/3724628938 <== osd.6 v2:172.21.15.144:6824/1893803079 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x560cddace000 con 0x560cddda5000 2024-04-20T21:25:20.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.292+0000 7f77f6446280 1 --2- 172.21.15.86:0/3724628938 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x560cdddbec00 0x560cddddc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:20.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.292+0000 7f77f6446280 1 -- 172.21.15.86:0/3724628938 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:2 12.14 12:292cf221:::200.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x560cdddbf000 con 0x560cdddbec00 2024-04-20T21:25:20.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.292+0000 7f77f3400640 1 --2- 172.21.15.86:0/3724628938 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x560cdddbec00 0x560cddddc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:20.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.294+0000 7f77f3400640 1 --2- 172.21.15.86:0/3724628938 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x560cdddbec00 0x560cddddc000 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=cb7d509f727e4dd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:20.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.299+0000 7f77f3400640 1 -- 172.21.15.86:0/3724628938 <== osd.5 v2:172.21.15.144:6808/1635974704 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv20 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x560cddace500 con 0x560cdddbec00 2024-04-20T21:25:20.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.299+0000 7f77e6c00640 1 -- 172.21.15.86:0/3724628938 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:3 12.14 12:2e2fa760:::200.00000001:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e86) -- 0x560cde7c5400 con 0x560cdddbec00 2024-04-20T21:25:20.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.300+0000 7f77e6c00640 1 --2- 172.21.15.86:0/3724628938 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x560cde7c5000 0x560cdddddb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:20.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.300+0000 7f77e6c00640 1 -- 172.21.15.86:0/3724628938 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:4 12.1b 12:ddb4e4d7:::200.00000002:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e86) -- 0x560cde7c4800 con 0x560cde7c5000 2024-04-20T21:25:20.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.300+0000 7f77f4800640 1 --2- 172.21.15.86:0/3724628938 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x560cde7c5000 0x560cdddddb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:20.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.300+0000 7f77f3400640 1 -- 172.21.15.86:0/3724628938 <== osd.5 v2:172.21.15.144:6808/1635974704 2 ==== osd_op_reply(3 200.00000001 [stat out=16b] v0'0 uv15 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x560cddace500 con 0x560cdddbec00 2024-04-20T21:25:20.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.301+0000 7f77f4800640 1 --2- 172.21.15.86:0/3724628938 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x560cde7c5000 0x560cdddddb80 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=88a0599b8c36d123 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:20.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.302+0000 7f77f4800640 1 -- 172.21.15.86:0/3724628938 <== osd.4 v2:172.21.15.144:6800/4087309475 1 ==== osd_op_reply(4 200.00000002 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x560cddace000 con 0x560cde7c5000 2024-04-20T21:25:20.301 INFO:teuthology.orchestra.run.smithi086.stdout:old journal was 4209834~892 2024-04-20T21:25:20.301 INFO:teuthology.orchestra.run.smithi086.stdout:new journal start will be 8388608 (4177882 bytes past old end) 2024-04-20T21:25:20.301 INFO:teuthology.orchestra.run.smithi086.stdout:writing journal head 2024-04-20T21:25:20.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.302+0000 7f77f6446280 1 -- 172.21.15.86:0/3724628938 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:5 12.14 12:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) -- 0x560cdddbf400 con 0x560cdddbec00 2024-04-20T21:25:20.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.305+0000 7f77f3400640 1 -- 172.21.15.86:0/3724628938 <== osd.5 v2:172.21.15.144:6808/1635974704 3 ==== osd_op_reply(5 200.00000000 [writefull 0~90] v86'21 uv21 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x560cddace500 con 0x560cdddbec00 2024-04-20T21:25:20.304 INFO:teuthology.orchestra.run.smithi086.stdout:writing EResetJournal entry 2024-04-20T21:25:20.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.305+0000 7f77f6446280 1 -- 172.21.15.86:0/3724628938 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:6 12.1b 12:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) -- 0x560cdddbf800 con 0x560cde7c5000 2024-04-20T21:25:20.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.305+0000 7f77f6446280 1 --2- 172.21.15.86:0/3724628938 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x560cdddbfc00 0x560cddddc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:20.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.305+0000 7f77f6446280 1 -- 172.21.15.86:0/3724628938 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:7 12.b 12:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) -- 0x560cdddbf000 con 0x560cdddbfc00 2024-04-20T21:25:20.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.305+0000 7f77f6446280 1 -- 172.21.15.86:0/3724628938 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:8 12.1f 12:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) -- 0x560cde7c4c00 con 0x560cddda5000 2024-04-20T21:25:20.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.305+0000 7f77f6446280 1 -- 172.21.15.86:0/3724628938 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:9 12.1a 12:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) -- 0x560cdddbe800 con 0x560cde7c5000 2024-04-20T21:25:20.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.305+0000 7f77f6446280 1 -- 172.21.15.86:0/3724628938 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:10 12.1e 12:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) -- 0x560cdddbf800 con 0x560cde7c5000 2024-04-20T21:25:20.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.305+0000 7f77f6446280 1 --2- 172.21.15.86:0/3724628938 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x560cde7c4800 0x560cddddcb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:20.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.305+0000 7f77f6446280 1 -- 172.21.15.86:0/3724628938 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:11 12.1d 12:bec1251c:::200.00000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) -- 0x560cde91e800 con 0x560cde7c4800 2024-04-20T21:25:20.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.306+0000 7f77f3e00640 1 --2- 172.21.15.86:0/3724628938 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x560cdddbfc00 0x560cddddc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:20.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.306+0000 7f77f3400640 1 --2- 172.21.15.86:0/3724628938 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x560cde7c4800 0x560cddddcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:20.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.306+0000 7f77f3e00640 1 --2- 172.21.15.86:0/3724628938 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x560cdddbfc00 0x560cddddc580 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=52dc409b28bb5965 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:20.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.306+0000 7f77f3400640 1 --2- 172.21.15.86:0/3724628938 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x560cde7c4800 0x560cddddcb00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=1ec15dded6eea0a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:20.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.308+0000 7f77f3e00640 1 -- 172.21.15.86:0/3724628938 <== osd.3 v2:172.21.15.86:6808/794135519 1 ==== osd_op_reply(7 200.00000003 [delete] v86'4 uv3 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x560cddace780 con 0x560cdddbfc00 2024-04-20T21:25:20.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.308+0000 7f77f3400640 1 -- 172.21.15.86:0/3724628938 <== osd.0 v2:172.21.15.86:6800/2471076035 1 ==== osd_op_reply(11 200.00000007 [delete] v86'3 uv2 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x560cddace500 con 0x560cde7c4800 2024-04-20T21:25:20.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.309+0000 7f77f4800640 1 -- 172.21.15.86:0/3724628938 <== osd.4 v2:172.21.15.144:6800/4087309475 2 ==== osd_op_reply(6 200.00000002 [delete] v86'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x560cddace000 con 0x560cde7c5000 2024-04-20T21:25:20.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.309+0000 7f77e6c00640 1 -- 172.21.15.86:0/3724628938 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:12 12.1b 12:ddb4e4d7:::200.00000002:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) -- 0x560cde7de000 con 0x560cde7c5000 2024-04-20T21:25:20.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.309+0000 7f77f4800640 1 -- 172.21.15.86:0/3724628938 <== osd.6 v2:172.21.15.144:6824/1893803079 2 ==== osd_op_reply(8 200.00000004 [delete] v86'7 uv6 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x560cddace000 con 0x560cddda5000 2024-04-20T21:25:20.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.309+0000 7f77f4800640 1 -- 172.21.15.86:0/3724628938 <== osd.4 v2:172.21.15.144:6800/4087309475 3 ==== osd_op_reply(9 200.00000005 [delete] v86'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x560cddace000 con 0x560cde7c5000 2024-04-20T21:25:20.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.309+0000 7f77f4800640 1 -- 172.21.15.86:0/3724628938 <== osd.4 v2:172.21.15.144:6800/4087309475 4 ==== osd_op_reply(10 200.00000006 [delete] v86'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x560cddace000 con 0x560cde7c5000 2024-04-20T21:25:20.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.312+0000 7f77f4800640 1 -- 172.21.15.86:0/3724628938 <== osd.4 v2:172.21.15.144:6800/4087309475 5 ==== osd_op_reply(12 200.00000002 [write 0~48] v86'3 uv3 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x560cddace000 con 0x560cde7c5000 2024-04-20T21:25:20.311 INFO:teuthology.orchestra.run.smithi086.stdout:done 2024-04-20T21:25:20.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.312+0000 7f77f6446280 1 -- 172.21.15.86:0/3724628938 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x560cde7c4800 msgr2=0x560cddddcb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:20.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.312+0000 7f77f6446280 1 --2- 172.21.15.86:0/3724628938 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x560cde7c4800 0x560cddddcb00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:20.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.312+0000 7f77f6446280 1 -- 172.21.15.86:0/3724628938 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x560cdddbfc00 msgr2=0x560cddddc580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:20.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.312+0000 7f77f6446280 1 --2- 172.21.15.86:0/3724628938 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x560cdddbfc00 0x560cddddc580 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:20.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.312+0000 7f77f6446280 1 -- 172.21.15.86:0/3724628938 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x560cde7c5000 msgr2=0x560cdddddb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:20.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.312+0000 7f77f6446280 1 --2- 172.21.15.86:0/3724628938 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x560cde7c5000 0x560cdddddb80 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:20.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.312+0000 7f77f6446280 1 -- 172.21.15.86:0/3724628938 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x560cdddbec00 msgr2=0x560cddddc000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:20.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.312+0000 7f77f6446280 1 --2- 172.21.15.86:0/3724628938 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x560cdddbec00 0x560cddddc000 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:20.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.312+0000 7f77f6446280 1 -- 172.21.15.86:0/3724628938 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x560cddda5000 msgr2=0x560cde73f700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:20.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.312+0000 7f77f6446280 1 --2- 172.21.15.86:0/3724628938 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x560cddda5000 0x560cde73f700 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:20.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.313+0000 7f77f3400640 1 -- 172.21.15.86:0/3724628938 reap_dead start 2024-04-20T21:25:20.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.313+0000 7f77f3400640 1 -- 172.21.15.86:0/3724628938 reap_dead start 2024-04-20T21:25:20.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.313+0000 7f77f3400640 1 -- 172.21.15.86:0/3724628938 reap_dead start 2024-04-20T21:25:20.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.313+0000 7f77f6446280 1 -- 172.21.15.86:0/3724628938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560cddda4800 msgr2=0x560cde73ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:20.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.313+0000 7f77f6446280 1 --2- 172.21.15.86:0/3724628938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560cddda4800 0x560cde73ec00 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x560cde95b620 tx=0x560cddcd78c0 comp rx=0 tx=0).stop 2024-04-20T21:25:20.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.313+0000 7f77f6446280 1 -- 172.21.15.86:0/3724628938 shutdown_connections 2024-04-20T21:25:20.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.313+0000 7f77f6446280 1 --2- 172.21.15.86:0/3724628938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560cddda4800 0x560cde73ec00 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:20.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.313+0000 7f77f6446280 1 -- 172.21.15.86:0/3724628938 >> 172.21.15.86:0/3724628938 conn(0x560cddda4000 msgr2=0x560cddabb800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:20.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.313+0000 7f77f6446280 1 -- 172.21.15.86:0/3724628938 shutdown_connections 2024-04-20T21:25:20.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.313+0000 7f77f6446280 1 -- 172.21.15.86:0/3724628938 wait complete. 2024-04-20T21:25:20.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.313+0000 7f77f6446280 1 -- 172.21.15.86:0/1009521896 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x560cde91fc00 msgr2=0x560cde73e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:20.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.313+0000 7f77f6446280 1 --2- 172.21.15.86:0/1009521896 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x560cde91fc00 0x560cde73e100 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x560cde8b82d0 tx=0x560cde8c0c00 comp rx=0 tx=0).stop 2024-04-20T21:25:20.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.313+0000 7f77f6446280 1 -- 172.21.15.86:0/1009521896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560cde8a3800 msgr2=0x560cde73d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:20.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.313+0000 7f77f6446280 1 --2- 172.21.15.86:0/1009521896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560cde8a3800 0x560cde73d600 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x560cde8b95c0 tx=0x560cde7700c0 comp rx=0 tx=0).stop 2024-04-20T21:25:20.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.313+0000 7f77f6446280 1 -- 172.21.15.86:0/1009521896 shutdown_connections 2024-04-20T21:25:20.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.313+0000 7f77f6446280 1 --2- 172.21.15.86:0/1009521896 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x560cde91fc00 0x560cde73e100 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:20.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.313+0000 7f77f6446280 1 --2- 172.21.15.86:0/1009521896 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560cde8a3c00 0x560cde73db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:20.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.313+0000 7f77f6446280 1 --2- 172.21.15.86:0/1009521896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560cde8a3800 0x560cde73d600 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:20.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.313+0000 7f77f6446280 1 --2- 172.21.15.86:0/1009521896 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560cde8a3400 0x560cde73d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:20.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.313+0000 7f77f6446280 1 -- 172.21.15.86:0/1009521896 >> 172.21.15.86:0/1009521896 conn(0x560cde8a3000 msgr2=0x560cddab9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:20.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.314+0000 7f77f6446280 1 -- 172.21.15.86:0/1009521896 shutdown_connections 2024-04-20T21:25:20.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.314+0000 7f77f6446280 1 -- 172.21.15.86:0/1009521896 wait complete. 2024-04-20T21:25:20.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.314+0000 7f77f6446280 1 -- 172.21.15.86:0/498112300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560cdd9d4c00 msgr2=0x560cde73c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:20.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.314+0000 7f77f6446280 1 --2- 172.21.15.86:0/498112300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560cdd9d4c00 0x560cde73c000 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x560cde7fd4a0 tx=0x560cde770f00 comp rx=0 tx=0).stop 2024-04-20T21:25:20.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.314+0000 7f77f6446280 1 -- 172.21.15.86:0/498112300 shutdown_connections 2024-04-20T21:25:20.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.314+0000 7f77f6446280 1 --2- 172.21.15.86:0/498112300 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560cdd9d5c00 0x560cde73cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:20.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.314+0000 7f77f6446280 1 --2- 172.21.15.86:0/498112300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560cdd9d5800 0x560cde73c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:20.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.314+0000 7f77f6446280 1 --2- 172.21.15.86:0/498112300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560cdd9d4c00 0x560cde73c000 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:20.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.314+0000 7f77f6446280 1 -- 172.21.15.86:0/498112300 >> 172.21.15.86:0/498112300 conn(0x560cdd9d5000 msgr2=0x560cddab9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:20.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.314+0000 7f77f6446280 1 -- 172.21.15.86:0/498112300 shutdown_connections 2024-04-20T21:25:20.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.314+0000 7f77f6446280 1 -- 172.21.15.86:0/498112300 wait complete. 2024-04-20T21:25:20.319 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset'] in time 0:00:00.129580, result: old journal was 4209834~892 new journal start will be 8388608 (4177882 bytes past old end) writing journal head writing EResetJournal entry done 2024-04-20T21:25:20.319 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-20T21:25:20.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.501+0000 7f262e800640 1 Processor -- start 2024-04-20T21:25:20.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.501+0000 7f262e800640 1 -- start start 2024-04-20T21:25:20.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.503+0000 7f262e800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f262815f990 0x7f262806dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:20.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.503+0000 7f262e800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26280aecd0 con 0x7f2628071d40 2024-04-20T21:25:20.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.503+0000 7f262e800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f262806e2e0 con 0x7f2628161c60 2024-04-20T21:25:20.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.503+0000 7f262e800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f262806e580 con 0x7f262815f990 2024-04-20T21:25:20.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.503+0000 7f262de00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f2628071d40 0x7f26280af1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:53646/0 (socket says 172.21.15.86:53646) 2024-04-20T21:25:20.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.503+0000 7f262de00640 1 -- 172.21.15.86:0/3639897957 learned_addr learned my addr 172.21.15.86:0/3639897957 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:25:20.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.503+0000 7f2627e00640 1 -- 172.21.15.86:0/3639897957 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3742033701 0 0) 0x7f26280aecd0 con 0x7f2628071d40 2024-04-20T21:25:20.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.503+0000 7f262ca00640 1 --2- 172.21.15.86:0/3639897957 >> v2:172.21.15.144:3300/0 conn(0x7f262815f990 0x7f262806dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:20.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.504+0000 7f2627e00640 1 -- 172.21.15.86:0/3639897957 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2610003520 con 0x7f2628071d40 2024-04-20T21:25:20.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.504+0000 7f2627e00640 1 -- 172.21.15.86:0/3639897957 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1848284607 0 0) 0x7f2610003520 con 0x7f2628071d40 2024-04-20T21:25:20.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.504+0000 7f2627e00640 1 -- 172.21.15.86:0/3639897957 >> v2:172.21.15.144:3300/0 conn(0x7f262815f990 msgr2=0x7f262806dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:20.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.504+0000 7f2627e00640 1 --2- 172.21.15.86:0/3639897957 >> v2:172.21.15.144:3300/0 conn(0x7f262815f990 0x7f262806dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:20.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.504+0000 7f2627e00640 1 -- 172.21.15.86:0/3639897957 >> v1:172.21.15.144:6789/0 conn(0x7f2628161c60 legacy=0x7f262815ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:20.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.504+0000 7f2627e00640 1 -- 172.21.15.86:0/3639897957 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f262806e860 con 0x7f2628071d40 2024-04-20T21:25:20.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.504+0000 7f2627e00640 1 -- 172.21.15.86:0/3639897957 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f2620001a10 con 0x7f2628071d40 2024-04-20T21:25:20.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.504+0000 7f2627e00640 1 -- 172.21.15.86:0/3639897957 >> v1:172.21.15.86:6789/0 conn(0x7f2628071d40 legacy=0x7f26280af1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:20.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.504+0000 7f2627e00640 1 --2- 172.21.15.86:0/3639897957 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2610003a20 0x7f2610003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:20.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.504+0000 7f2627e00640 1 --2- 172.21.15.86:0/3639897957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2610004800 0x7f2610004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:20.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.504+0000 7f2627e00640 1 --2- 172.21.15.86:0/3639897957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2610005230 0x7f261000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:20.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.504+0000 7f2627e00640 1 -- 172.21.15.86:0/3639897957 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f262806e580 con 0x7f2610004800 2024-04-20T21:25:20.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.504+0000 7f2627e00640 1 -- 172.21.15.86:0/3639897957 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f261000c190 con 0x7f2610005230 2024-04-20T21:25:20.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.504+0000 7f2627e00640 1 -- 172.21.15.86:0/3639897957 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f261000c390 con 0x7f2610003a20 2024-04-20T21:25:20.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.504+0000 7f262ca00640 1 --2- 172.21.15.86:0/3639897957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2610004800 0x7f2610004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:20.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.505+0000 7f262de00640 1 --2- 172.21.15.86:0/3639897957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2610005230 0x7f261000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:20.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.505+0000 7f262d400640 1 --2- 172.21.15.86:0/3639897957 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2610003a20 0x7f2610003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:20.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.505+0000 7f262ca00640 1 -- 172.21.15.86:0/3639897957 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2610003a20 msgr2=0x7f2610003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:20.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.505+0000 7f262ca00640 1 --2- 172.21.15.86:0/3639897957 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2610003a20 0x7f2610003e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:20.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.505+0000 7f262ca00640 1 -- 172.21.15.86:0/3639897957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2610005230 msgr2=0x7f261000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:20.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.505+0000 7f262ca00640 1 --2- 172.21.15.86:0/3639897957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2610005230 0x7f261000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:20.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.505+0000 7f262ca00640 1 -- 172.21.15.86:0/3639897957 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2610003520 con 0x7f2610004800 2024-04-20T21:25:20.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.505+0000 7f262de00640 1 --2- 172.21.15.86:0/3639897957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2610005230 0x7f261000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:25:20.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.505+0000 7f262de00640 1 -- 172.21.15.86:0/3639897957 reap_dead start 2024-04-20T21:25:20.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.506+0000 7f262ca00640 1 --2- 172.21.15.86:0/3639897957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2610004800 0x7f2610004c80 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f261400ba30 tx=0x7f2610005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69a8a2c156098369 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:20.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.506+0000 7f2627e00640 1 -- 172.21.15.86:0/3639897957 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2614036020 con 0x7f2610004800 2024-04-20T21:25:20.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.506+0000 7f2627e00640 1 -- 172.21.15.86:0/3639897957 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2614032640 con 0x7f2610004800 2024-04-20T21:25:20.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.506+0000 7f2627e00640 1 -- 172.21.15.86:0/3639897957 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261400b600 con 0x7f2610004800 2024-04-20T21:25:20.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.507+0000 7f262e800640 1 -- 172.21.15.86:0/3639897957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2610004800 msgr2=0x7f2610004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:20.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.507+0000 7f262e800640 1 --2- 172.21.15.86:0/3639897957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2610004800 0x7f2610004c80 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f261400ba30 tx=0x7f2610005830 comp rx=0 tx=0).stop 2024-04-20T21:25:20.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.507+0000 7f262e800640 1 -- 172.21.15.86:0/3639897957 shutdown_connections 2024-04-20T21:25:20.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.507+0000 7f262e800640 1 --2- 172.21.15.86:0/3639897957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2610004800 0x7f2610004c80 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:20.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.507+0000 7f262e800640 1 -- 172.21.15.86:0/3639897957 >> 172.21.15.86:0/3639897957 conn(0x7f2628067cf0 msgr2=0x7f262806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:20.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.507+0000 7f262e800640 1 -- 172.21.15.86:0/3639897957 shutdown_connections 2024-04-20T21:25:20.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.507+0000 7f262e800640 1 -- 172.21.15.86:0/3639897957 wait complete. 2024-04-20T21:25:20.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.508+0000 7f262e800640 1 Processor -- start 2024-04-20T21:25:20.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.509+0000 7f262e800640 1 -- start start 2024-04-20T21:25:20.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.509+0000 7f262e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2610004800 0x7f2628138420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:20.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.509+0000 7f262e800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26280ce9f0 0x7f26281587f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:20.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.510+0000 7f262e800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f262815c350 0x7f262815c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:20.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.510+0000 7f262e800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2628162040 con 0x7f2610004800 2024-04-20T21:25:20.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.510+0000 7f262e800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f262806ee20 con 0x7f262815c350 2024-04-20T21:25:20.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.510+0000 7f262e800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f262806dda0 con 0x7f26280ce9f0 2024-04-20T21:25:20.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.510+0000 7f262d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2610004800 0x7f2628138420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:20.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.510+0000 7f262d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2610004800 0x7f2628138420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53928/0 (socket says 172.21.15.86:53928) 2024-04-20T21:25:20.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.510+0000 7f262d400640 1 -- 172.21.15.86:0/3245246421 learned_addr learned my addr 172.21.15.86:0/3245246421 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:20.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.510+0000 7f262d400640 1 -- 172.21.15.86:0/3245246421 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26280ce9f0 msgr2=0x7f26281587f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:20.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.510+0000 7f262d400640 1 --2- 172.21.15.86:0/3245246421 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26280ce9f0 0x7f26281587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:20.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.510+0000 7f262d400640 1 -- 172.21.15.86:0/3245246421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f262815c350 msgr2=0x7f262815c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:20.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.510+0000 7f262d400640 1 --2- 172.21.15.86:0/3245246421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f262815c350 0x7f262815c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:20.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.510+0000 7f262d400640 1 -- 172.21.15.86:0/3245246421 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f262815cd10 con 0x7f2610004800 2024-04-20T21:25:20.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.510+0000 7f262d400640 1 --2- 172.21.15.86:0/3245246421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2610004800 0x7f2628138420 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f261c00d470 tx=0x7f261c00d930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:20.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.511+0000 7f2625600640 1 -- 172.21.15.86:0/3245246421 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c005480 con 0x7f2610004800 2024-04-20T21:25:20.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.511+0000 7f2625600640 1 -- 172.21.15.86:0/3245246421 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f261c017070 con 0x7f2610004800 2024-04-20T21:25:20.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.511+0000 7f2625600640 1 -- 172.21.15.86:0/3245246421 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c003810 con 0x7f2610004800 2024-04-20T21:25:20.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.511+0000 7f262e800640 1 -- 172.21.15.86:0/3245246421 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26280cf160 con 0x7f2610004800 2024-04-20T21:25:20.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.511+0000 7f262e800640 1 -- 172.21.15.86:0/3245246421 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26280cf720 con 0x7f2610004800 2024-04-20T21:25:20.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.512+0000 7f262e800640 1 -- 172.21.15.86:0/3245246421 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f262804fd90 con 0x7f2610004800 2024-04-20T21:25:20.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.513+0000 7f2625600640 1 -- 172.21.15.86:0/3245246421 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f261c0039b0 con 0x7f2610004800 2024-04-20T21:25:20.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.514+0000 7f2625600640 1 --2- 172.21.15.86:0/3245246421 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f260806c920 0x7f260806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:20.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.514+0000 7f2625600640 1 -- 172.21.15.86:0/3245246421 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5795+0+0 (secure 0 0 0) 0x7f261c08b960 con 0x7f2610004800 2024-04-20T21:25:20.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.514+0000 7f262ca00640 1 --2- 172.21.15.86:0/3245246421 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f260806c920 0x7f260806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:20.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.515+0000 7f262ca00640 1 --2- 172.21.15.86:0/3245246421 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f260806c920 0x7f260806ede0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f261400ba00 tx=0x7f2614032000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:20.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.518+0000 7f2625600640 1 -- 172.21.15.86:0/3245246421 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f261c059c90 con 0x7f2610004800 2024-04-20T21:25:20.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:20.743+0000 7f262e800640 1 -- 172.21.15.86:0/3245246421 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f26280cfe90 con 0x7f2610004800 2024-04-20T21:25:21.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.047+0000 7f2625600640 1 -- 172.21.15.86:0/3245246421 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v61) ==== 162+0+0 (secure 0 0 0) 0x7f261c08b030 con 0x7f2610004800 2024-04-20T21:25:21.046 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-20T21:25:21.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.050+0000 7f262e800640 1 -- 172.21.15.86:0/3245246421 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f260806c920 msgr2=0x7f260806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:21.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.050+0000 7f262e800640 1 --2- 172.21.15.86:0/3245246421 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f260806c920 0x7f260806ede0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f261400ba00 tx=0x7f2614032000 comp rx=0 tx=0).stop 2024-04-20T21:25:21.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.050+0000 7f262e800640 1 -- 172.21.15.86:0/3245246421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2610004800 msgr2=0x7f2628138420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:21.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.050+0000 7f262e800640 1 --2- 172.21.15.86:0/3245246421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2610004800 0x7f2628138420 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f261c00d470 tx=0x7f261c00d930 comp rx=0 tx=0).stop 2024-04-20T21:25:21.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.050+0000 7f262e800640 1 -- 172.21.15.86:0/3245246421 shutdown_connections 2024-04-20T21:25:21.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.050+0000 7f262e800640 1 --2- 172.21.15.86:0/3245246421 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f260806c920 0x7f260806ede0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:21.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.050+0000 7f262e800640 1 --2- 172.21.15.86:0/3245246421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f262815c350 0x7f262815c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:21.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.050+0000 7f262e800640 1 --2- 172.21.15.86:0/3245246421 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26280ce9f0 0x7f26281587f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:21.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.050+0000 7f262e800640 1 --2- 172.21.15.86:0/3245246421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2610004800 0x7f2628138420 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:21.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.050+0000 7f262e800640 1 -- 172.21.15.86:0/3245246421 >> 172.21.15.86:0/3245246421 conn(0x7f2628067cf0 msgr2=0x7f26280b19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:21.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.050+0000 7f262e800640 1 -- 172.21.15.86:0/3245246421 shutdown_connections 2024-04-20T21:25:21.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.050+0000 7f262e800640 1 -- 172.21.15.86:0/3245246421 wait complete. 2024-04-20T21:25:21.074 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:25:21.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.277+0000 7f493da00640 1 Processor -- start 2024-04-20T21:25:21.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.277+0000 7f493da00640 1 -- start start 2024-04-20T21:25:21.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.279+0000 7f493da00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f49380b3810 0x7f4938141f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:21.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.279+0000 7f493da00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4938162c40 0x7f4938072990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:21.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.279+0000 7f493da00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4938071c10 con 0x7f49380af8d0 2024-04-20T21:25:21.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.279+0000 7f493da00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f4938142490 con 0x7f49380b3810 2024-04-20T21:25:21.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.279+0000 7f493da00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f493815eaf0 con 0x7f4938162c40 2024-04-20T21:25:21.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.280+0000 7f4937400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f49380b3810 0x7f4938141f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:21.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.280+0000 7f4937e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f49380af8d0 0x7f49380b3100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:53650/0 (socket says 172.21.15.86:53650) 2024-04-20T21:25:21.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.280+0000 7f4937e00640 1 -- 172.21.15.86:0/4096808266 learned_addr learned my addr 172.21.15.86:0/4096808266 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:25:21.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.280+0000 7f493d000640 1 --2- 172.21.15.86:0/4096808266 >> v2:172.21.15.144:3300/0 conn(0x7f4938162c40 0x7f4938072990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:21.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.280+0000 7f4937400640 1 -- 172.21.15.86:0/4096808266 >> v2:172.21.15.144:3300/0 conn(0x7f4938162c40 msgr2=0x7f4938072990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:21.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.280+0000 7f4937400640 1 --2- 172.21.15.86:0/4096808266 >> v2:172.21.15.144:3300/0 conn(0x7f4938162c40 0x7f4938072990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:21.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.280+0000 7f4937400640 1 -- 172.21.15.86:0/4096808266 >> v1:172.21.15.86:6789/0 conn(0x7f49380af8d0 legacy=0x7f49380b3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:21.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.280+0000 7f4937400640 1 -- 172.21.15.86:0/4096808266 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4938071eb0 con 0x7f49380b3810 2024-04-20T21:25:21.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.280+0000 7f493d000640 1 --2- 172.21.15.86:0/4096808266 >> v2:172.21.15.144:3300/0 conn(0x7f4938162c40 0x7f4938072990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:25:21.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.280+0000 7f4936a00640 1 -- 172.21.15.86:0/4096808266 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 900208304 0 0) 0x7f4938071c10 con 0x7f49380af8d0 2024-04-20T21:25:21.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.282+0000 7f4937400640 1 --2- 172.21.15.86:0/4096808266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49380b3810 0x7f4938141f50 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f4938141e20 tx=0x7f492002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc6379a7fc0584b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:21.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.282+0000 7f4936a00640 1 -- 172.21.15.86:0/4096808266 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49200315a0 con 0x7f49380b3810 2024-04-20T21:25:21.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.282+0000 7f4936a00640 1 -- 172.21.15.86:0/4096808266 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4920037070 con 0x7f49380b3810 2024-04-20T21:25:21.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.282+0000 7f4936a00640 1 -- 172.21.15.86:0/4096808266 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4920031960 con 0x7f49380b3810 2024-04-20T21:25:21.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.283+0000 7f493da00640 1 -- 172.21.15.86:0/4096808266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49380b3810 msgr2=0x7f4938141f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:21.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.283+0000 7f493da00640 1 --2- 172.21.15.86:0/4096808266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49380b3810 0x7f4938141f50 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f4938141e20 tx=0x7f492002b840 comp rx=0 tx=0).stop 2024-04-20T21:25:21.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.283+0000 7f493da00640 1 -- 172.21.15.86:0/4096808266 shutdown_connections 2024-04-20T21:25:21.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.283+0000 7f493da00640 1 --2- 172.21.15.86:0/4096808266 >> v2:172.21.15.144:3300/0 conn(0x7f4938162c40 0x7f4938072990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:21.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.283+0000 7f493da00640 1 --2- 172.21.15.86:0/4096808266 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49380b3810 0x7f4938141f50 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:21.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.283+0000 7f493da00640 1 -- 172.21.15.86:0/4096808266 >> 172.21.15.86:0/4096808266 conn(0x7f49380d0170 msgr2=0x7f49380d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:21.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.283+0000 7f493da00640 1 -- 172.21.15.86:0/4096808266 shutdown_connections 2024-04-20T21:25:21.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.283+0000 7f493da00640 1 -- 172.21.15.86:0/4096808266 wait complete. 2024-04-20T21:25:21.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.285+0000 7f493da00640 1 Processor -- start 2024-04-20T21:25:21.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.285+0000 7f493da00640 1 -- start start 2024-04-20T21:25:21.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.286+0000 7f493da00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f49380af8d0 0x7f49381f3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:21.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.286+0000 7f493da00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49380b3810 0x7f49381f4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:21.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.286+0000 7f493da00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4938162c40 0x7f49381f5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:21.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.286+0000 7f493da00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4938163100 con 0x7f49380b3810 2024-04-20T21:25:21.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.286+0000 7f493da00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4938072470 con 0x7f4938162c40 2024-04-20T21:25:21.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.286+0000 7f493da00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f4938148050 con 0x7f49380af8d0 2024-04-20T21:25:21.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.286+0000 7f4937400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49380b3810 0x7f49381f4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:21.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.286+0000 7f4937400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49380b3810 0x7f49381f4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53944/0 (socket says 172.21.15.86:53944) 2024-04-20T21:25:21.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.286+0000 7f4937400640 1 -- 172.21.15.86:0/3773907904 learned_addr learned my addr 172.21.15.86:0/3773907904 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:21.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.286+0000 7f4937400640 1 -- 172.21.15.86:0/3773907904 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f49380af8d0 msgr2=0x7f49381f3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:21.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.286+0000 7f4937400640 1 --2- 172.21.15.86:0/3773907904 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f49380af8d0 0x7f49381f3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:21.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.286+0000 7f4937400640 1 -- 172.21.15.86:0/3773907904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4938162c40 msgr2=0x7f49381f5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:21.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.286+0000 7f4937400640 1 --2- 172.21.15.86:0/3773907904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4938162c40 0x7f49381f5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:21.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.286+0000 7f4937400640 1 -- 172.21.15.86:0/3773907904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49381448b0 con 0x7f49380b3810 2024-04-20T21:25:21.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.287+0000 7f4937400640 1 --2- 172.21.15.86:0/3773907904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49380b3810 0x7f49381f4680 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f4920000c00 tx=0x7f49200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:21.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.287+0000 7f492fe00640 1 -- 172.21.15.86:0/3773907904 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4920032840 con 0x7f49380b3810 2024-04-20T21:25:21.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.287+0000 7f492fe00640 1 -- 172.21.15.86:0/3773907904 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4920032a00 con 0x7f49380b3810 2024-04-20T21:25:21.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.287+0000 7f492fe00640 1 -- 172.21.15.86:0/3773907904 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4920032df0 con 0x7f49380b3810 2024-04-20T21:25:21.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.287+0000 7f493da00640 1 -- 172.21.15.86:0/3773907904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4938154aa0 con 0x7f49380b3810 2024-04-20T21:25:21.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.287+0000 7f493da00640 1 -- 172.21.15.86:0/3773907904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4938161a40 con 0x7f49380b3810 2024-04-20T21:25:21.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.289+0000 7f493da00640 1 -- 172.21.15.86:0/3773907904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f493804fcf0 con 0x7f49380b3810 2024-04-20T21:25:21.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.290+0000 7f492fe00640 1 -- 172.21.15.86:0/3773907904 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f492004d3f0 con 0x7f49380b3810 2024-04-20T21:25:21.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.291+0000 7f492fe00640 1 --2- 172.21.15.86:0/3773907904 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f491406c8f0 0x7f491406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:21.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.291+0000 7f492fe00640 1 -- 172.21.15.86:0/3773907904 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5795+0+0 (secure 0 0 0) 0x7f492002f030 con 0x7f49380b3810 2024-04-20T21:25:21.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.292+0000 7f4937e00640 1 --2- 172.21.15.86:0/3773907904 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f491406c8f0 0x7f491406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:21.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.292+0000 7f4937e00640 1 --2- 172.21.15.86:0/3773907904 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f491406c8f0 0x7f491406edb0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f4928005880 tx=0x7f4928007950 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:21.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.296+0000 7f492fe00640 1 -- 172.21.15.86:0/3773907904 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4920040070 con 0x7f49380b3810 2024-04-20T21:25:21.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.543+0000 7f493da00640 1 -- 172.21.15.86:0/3773907904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4938144690 con 0x7f49380b3810 2024-04-20T21:25:21.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.545+0000 7f492fe00640 1 -- 172.21.15.86:0/3773907904 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5201 (secure 0 0 0) 0x7f492007bb70 con 0x7f49380b3810 2024-04-20T21:25:21.546 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:25:21.546 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7506,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/4013863257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4013863257},{"type":"v1","addr":"172.21.15.86:6836","nonce":4013863257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":7509,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2044883073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2044883073},{"type":"v1","addr":"172.21.15.86:6837","nonce":2044883073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7518,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/2351381466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2351381466},{"type":"v1","addr":"172.21.15.144:6834","nonce":2351381466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:24:58.299257+0000","modified":"2024-04-20T21:25:21.052158+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7773},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7773":{"gid":7773,"name":"b","rank":0,"incarnation":62,"state":"up:replay","state_seq":1,"addr":"172.21.15.144:6835/3981403733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3981403733},{"type":"v1","addr":"172.21.15.144:6835","nonce":3981403733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-20T21:25:21.548 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 62 2024-04-20T21:25:21.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.548+0000 7f493da00640 1 -- 172.21.15.86:0/3773907904 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f491406c8f0 msgr2=0x7f491406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:21.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.548+0000 7f493da00640 1 --2- 172.21.15.86:0/3773907904 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f491406c8f0 0x7f491406edb0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f4928005880 tx=0x7f4928007950 comp rx=0 tx=0).stop 2024-04-20T21:25:21.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.548+0000 7f493da00640 1 -- 172.21.15.86:0/3773907904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49380b3810 msgr2=0x7f49381f4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:21.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.548+0000 7f493da00640 1 --2- 172.21.15.86:0/3773907904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49380b3810 0x7f49381f4680 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f4920000c00 tx=0x7f49200046c0 comp rx=0 tx=0).stop 2024-04-20T21:25:21.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.548+0000 7f493da00640 1 -- 172.21.15.86:0/3773907904 shutdown_connections 2024-04-20T21:25:21.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.548+0000 7f493da00640 1 --2- 172.21.15.86:0/3773907904 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f491406c8f0 0x7f491406edb0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:21.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.548+0000 7f493da00640 1 --2- 172.21.15.86:0/3773907904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4938162c40 0x7f49381f5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:21.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.548+0000 7f493da00640 1 --2- 172.21.15.86:0/3773907904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49380b3810 0x7f49381f4680 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:21.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.548+0000 7f493da00640 1 --2- 172.21.15.86:0/3773907904 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f49380af8d0 0x7f49381f3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:21.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.548+0000 7f493da00640 1 -- 172.21.15.86:0/3773907904 >> 172.21.15.86:0/3773907904 conn(0x7f49380d0170 msgr2=0x7f49381470f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:21.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.548+0000 7f493da00640 1 -- 172.21.15.86:0/3773907904 shutdown_connections 2024-04-20T21:25:21.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:21.548+0000 7f493da00640 1 -- 172.21.15.86:0/3773907904 wait complete. 2024-04-20T21:25:21.970 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 62, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T21:24:58.299257+0000', 'modified': '2024-04-20T21:25:21.052158+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 86, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7773}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7773': {'gid': 7773, 'name': 'b', 'rank': 0, 'incarnation': 62, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.144:6835/3981403733', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6833', 'nonce': 3981403733}, {'type': 'v1', 'addr': '172.21.15.144:6835', 'nonce': 3981403733}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-20T21:25:21.970 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7773:up:replay 2024-04-20T21:25:22.971 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:25:23.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.179+0000 7f2982600640 1 Processor -- start 2024-04-20T21:25:23.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.179+0000 7f2982600640 1 -- start start 2024-04-20T21:25:23.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.181+0000 7f2982600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f297c146130 0x7f297c06e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:23.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.181+0000 7f2982600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f297c0ccdd0 0x7f297c0cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:23.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.181+0000 7f2982600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f297c06dc30 con 0x7f297c071d00 2024-04-20T21:25:23.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.181+0000 7f2982600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f297c0cd8e0 con 0x7f297c146130 2024-04-20T21:25:23.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.181+0000 7f2982600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f297c0cdbc0 con 0x7f297c0ccdd0 2024-04-20T21:25:23.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.181+0000 7f297be00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f297c071d00 0x7f297c15cc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:53652/0 (socket says 172.21.15.86:53652) 2024-04-20T21:25:23.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.181+0000 7f297be00640 1 -- 172.21.15.86:0/844406291 learned_addr learned my addr 172.21.15.86:0/844406291 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:25:23.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.181+0000 7f297b400640 1 --2- 172.21.15.86:0/844406291 >> v2:172.21.15.144:3300/0 conn(0x7f297c146130 0x7f297c06e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:23.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.181+0000 7f297aa00640 1 --2- 172.21.15.86:0/844406291 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f297c0ccdd0 0x7f297c0cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:23.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.181+0000 7f297a000640 1 -- 172.21.15.86:0/844406291 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2004455034 0 0) 0x7f297c06dc30 con 0x7f297c071d00 2024-04-20T21:25:23.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.182+0000 7f297a000640 1 -- 172.21.15.86:0/844406291 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2960003520 con 0x7f297c071d00 2024-04-20T21:25:23.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.182+0000 7f297a000640 1 -- 172.21.15.86:0/844406291 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3682183470 0 0) 0x7f2960003520 con 0x7f297c071d00 2024-04-20T21:25:23.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.182+0000 7f297a000640 1 -- 172.21.15.86:0/844406291 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f297c0ccdd0 msgr2=0x7f297c0cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:23.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.182+0000 7f297a000640 1 --2- 172.21.15.86:0/844406291 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f297c0ccdd0 0x7f297c0cd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:23.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.182+0000 7f297a000640 1 -- 172.21.15.86:0/844406291 >> v2:172.21.15.144:3300/0 conn(0x7f297c146130 msgr2=0x7f297c06e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:23.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.182+0000 7f297a000640 1 --2- 172.21.15.86:0/844406291 >> v2:172.21.15.144:3300/0 conn(0x7f297c146130 0x7f297c06e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:23.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.182+0000 7f297a000640 1 -- 172.21.15.86:0/844406291 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f297c06ded0 con 0x7f297c071d00 2024-04-20T21:25:23.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.182+0000 7f297a000640 1 -- 172.21.15.86:0/844406291 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f29740019f0 con 0x7f297c071d00 2024-04-20T21:25:23.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.182+0000 7f297a000640 1 -- 172.21.15.86:0/844406291 >> v1:172.21.15.86:6789/0 conn(0x7f297c071d00 legacy=0x7f297c15cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:23.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.182+0000 7f297a000640 1 --2- 172.21.15.86:0/844406291 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f297c0ccdd0 0x7f29600039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:23.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.182+0000 7f297a000640 1 --2- 172.21.15.86:0/844406291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2960003f80 0x7f2960004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:23.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.182+0000 7f297a000640 1 --2- 172.21.15.86:0/844406291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2960004d40 0x7f29600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:23.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.182+0000 7f297a000640 1 -- 172.21.15.86:0/844406291 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f297c0cd8e0 con 0x7f2960003f80 2024-04-20T21:25:23.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.182+0000 7f297a000640 1 -- 172.21.15.86:0/844406291 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f297c0cdbc0 con 0x7f2960004d40 2024-04-20T21:25:23.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.182+0000 7f297a000640 1 -- 172.21.15.86:0/844406291 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f296000ba40 con 0x7f297c0ccdd0 2024-04-20T21:25:23.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.182+0000 7f297aa00640 1 --2- 172.21.15.86:0/844406291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2960003f80 0x7f2960004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:23.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.183+0000 7f297b400640 1 --2- 172.21.15.86:0/844406291 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f297c0ccdd0 0x7f29600039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:23.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.183+0000 7f297aa00640 1 -- 172.21.15.86:0/844406291 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f297c0ccdd0 msgr2=0x7f29600039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:23.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.183+0000 7f297aa00640 1 --2- 172.21.15.86:0/844406291 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f297c0ccdd0 0x7f29600039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:23.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.183+0000 7f297aa00640 1 -- 172.21.15.86:0/844406291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2960004d40 msgr2=0x7f29600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:23.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.183+0000 7f297aa00640 1 --2- 172.21.15.86:0/844406291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2960004d40 0x7f29600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:23.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.183+0000 7f297aa00640 1 -- 172.21.15.86:0/844406291 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2960003520 con 0x7f2960003f80 2024-04-20T21:25:23.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.184+0000 7f297aa00640 1 --2- 172.21.15.86:0/844406291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2960003f80 0x7f2960004800 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f2968009ac0 tx=0x7f2960005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b7e739ca7b7203d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:23.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.184+0000 7f297a000640 1 -- 172.21.15.86:0/844406291 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2968039070 con 0x7f2960003f80 2024-04-20T21:25:23.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.184+0000 7f297a000640 1 -- 172.21.15.86:0/844406291 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f296802c6b0 con 0x7f2960003f80 2024-04-20T21:25:23.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.184+0000 7f297a000640 1 -- 172.21.15.86:0/844406291 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2968009690 con 0x7f2960003f80 2024-04-20T21:25:23.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.185+0000 7f2982600640 1 -- 172.21.15.86:0/844406291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2960003f80 msgr2=0x7f2960004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:23.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.185+0000 7f2982600640 1 --2- 172.21.15.86:0/844406291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2960003f80 0x7f2960004800 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f2968009ac0 tx=0x7f2960005830 comp rx=0 tx=0).stop 2024-04-20T21:25:23.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.185+0000 7f297be00640 1 -- 172.21.15.86:0/844406291 reap_dead start 2024-04-20T21:25:23.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.185+0000 7f2982600640 1 -- 172.21.15.86:0/844406291 shutdown_connections 2024-04-20T21:25:23.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.185+0000 7f2982600640 1 -- 172.21.15.86:0/844406291 >> 172.21.15.86:0/844406291 conn(0x7f297c0b2360 msgr2=0x7f297c0b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:23.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.185+0000 7f2982600640 1 -- 172.21.15.86:0/844406291 shutdown_connections 2024-04-20T21:25:23.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.185+0000 7f2982600640 1 -- 172.21.15.86:0/844406291 wait complete. 2024-04-20T21:25:23.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.187+0000 7f2982600640 1 Processor -- start 2024-04-20T21:25:23.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.187+0000 7f2982600640 1 -- start start 2024-04-20T21:25:23.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.188+0000 7f2982600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2960003f80 0x7f297c143f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:23.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.188+0000 7f2982600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f297c0af2d0 0x7f297c15bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:23.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.188+0000 7f2982600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f297c0b1910 0x7f297c13e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:23.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.188+0000 7f2982600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f297c0cd230 con 0x7f2960003f80 2024-04-20T21:25:23.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.188+0000 7f2982600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f297c06f150 con 0x7f297c0af2d0 2024-04-20T21:25:23.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.188+0000 7f2982600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f297c0cd3b0 con 0x7f297c0b1910 2024-04-20T21:25:23.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.188+0000 7f297b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2960003f80 0x7f297c143f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:23.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.188+0000 7f297b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2960003f80 0x7f297c143f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53952/0 (socket says 172.21.15.86:53952) 2024-04-20T21:25:23.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.188+0000 7f297b400640 1 -- 172.21.15.86:0/2808376079 learned_addr learned my addr 172.21.15.86:0/2808376079 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:23.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.188+0000 7f297b400640 1 -- 172.21.15.86:0/2808376079 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f297c0b1910 msgr2=0x7f297c13e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:23.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.188+0000 7f297b400640 1 --2- 172.21.15.86:0/2808376079 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f297c0b1910 0x7f297c13e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:23.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.188+0000 7f297b400640 1 -- 172.21.15.86:0/2808376079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f297c0af2d0 msgr2=0x7f297c15bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:23.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.188+0000 7f297b400640 1 --2- 172.21.15.86:0/2808376079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f297c0af2d0 0x7f297c15bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:23.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.188+0000 7f297b400640 1 -- 172.21.15.86:0/2808376079 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f297c143160 con 0x7f2960003f80 2024-04-20T21:25:23.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.189+0000 7f297b400640 1 --2- 172.21.15.86:0/2808376079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2960003f80 0x7f297c143f20 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f296c0108e0 tx=0x7f296c010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:23.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.189+0000 7f2973400640 1 -- 172.21.15.86:0/2808376079 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c00f040 con 0x7f2960003f80 2024-04-20T21:25:23.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.189+0000 7f2973400640 1 -- 172.21.15.86:0/2808376079 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f296c00eb60 con 0x7f2960003f80 2024-04-20T21:25:23.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.189+0000 7f2973400640 1 -- 172.21.15.86:0/2808376079 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c01c400 con 0x7f2960003f80 2024-04-20T21:25:23.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.189+0000 7f2982600640 1 -- 172.21.15.86:0/2808376079 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f297c143440 con 0x7f2960003f80 2024-04-20T21:25:23.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.189+0000 7f2982600640 1 -- 172.21.15.86:0/2808376079 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f297c143a00 con 0x7f2960003f80 2024-04-20T21:25:23.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.190+0000 7f2982600640 1 -- 172.21.15.86:0/2808376079 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f297c0689e0 con 0x7f2960003f80 2024-04-20T21:25:23.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.191+0000 7f2973400640 1 -- 172.21.15.86:0/2808376079 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f296c01c5a0 con 0x7f2960003f80 2024-04-20T21:25:23.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.192+0000 7f2973400640 1 --2- 172.21.15.86:0/2808376079 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f295c06c8f0 0x7f295c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:23.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.192+0000 7f2973400640 1 -- 172.21.15.86:0/2808376079 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5795+0+0 (secure 0 0 0) 0x7f296c08c440 con 0x7f2960003f80 2024-04-20T21:25:23.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.192+0000 7f297aa00640 1 --2- 172.21.15.86:0/2808376079 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f295c06c8f0 0x7f295c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:23.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.193+0000 7f297aa00640 1 --2- 172.21.15.86:0/2808376079 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f295c06c8f0 0x7f295c06edb0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f2968036000 tx=0x7f296802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:23.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.197+0000 7f2973400640 1 -- 172.21.15.86:0/2808376079 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f296c018030 con 0x7f2960003f80 2024-04-20T21:25:23.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.445+0000 7f2982600640 1 -- 172.21.15.86:0/2808376079 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f297c0b3360 con 0x7f2960003f80 2024-04-20T21:25:23.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.447+0000 7f2973400640 1 -- 172.21.15.86:0/2808376079 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+5201 (secure 0 0 0) 0x7f296c05a770 con 0x7f2960003f80 2024-04-20T21:25:23.446 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:25:23.446 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":64,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7506,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/4013863257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4013863257},{"type":"v1","addr":"172.21.15.86:6836","nonce":4013863257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":7509,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2044883073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2044883073},{"type":"v1","addr":"172.21.15.86:6837","nonce":2044883073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7518,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/2351381466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2351381466},{"type":"v1","addr":"172.21.15.144:6834","nonce":2351381466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:24:58.299257+0000","modified":"2024-04-20T21:25:22.062758+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7773},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7773":{"gid":7773,"name":"b","rank":0,"incarnation":62,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.144:6835/3981403733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3981403733},{"type":"v1","addr":"172.21.15.144:6835","nonce":3981403733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-20T21:25:23.446 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 64 2024-04-20T21:25:23.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.449+0000 7f2982600640 1 -- 172.21.15.86:0/2808376079 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f295c06c8f0 msgr2=0x7f295c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:23.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.449+0000 7f2982600640 1 --2- 172.21.15.86:0/2808376079 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f295c06c8f0 0x7f295c06edb0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f2968036000 tx=0x7f296802e000 comp rx=0 tx=0).stop 2024-04-20T21:25:23.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.449+0000 7f2982600640 1 -- 172.21.15.86:0/2808376079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2960003f80 msgr2=0x7f297c143f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:23.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.449+0000 7f2982600640 1 --2- 172.21.15.86:0/2808376079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2960003f80 0x7f297c143f20 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f296c0108e0 tx=0x7f296c010da0 comp rx=0 tx=0).stop 2024-04-20T21:25:23.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.450+0000 7f2982600640 1 -- 172.21.15.86:0/2808376079 shutdown_connections 2024-04-20T21:25:23.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.450+0000 7f2982600640 1 --2- 172.21.15.86:0/2808376079 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f295c06c8f0 0x7f295c06edb0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:23.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.450+0000 7f2982600640 1 --2- 172.21.15.86:0/2808376079 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f297c0b1910 0x7f297c13e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:23.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.450+0000 7f2982600640 1 --2- 172.21.15.86:0/2808376079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f297c0af2d0 0x7f297c15bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:23.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.450+0000 7f2982600640 1 --2- 172.21.15.86:0/2808376079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2960003f80 0x7f297c143f20 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:23.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.450+0000 7f2982600640 1 -- 172.21.15.86:0/2808376079 >> 172.21.15.86:0/2808376079 conn(0x7f297c0b2360 msgr2=0x7f297c15cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:23.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.450+0000 7f2982600640 1 -- 172.21.15.86:0/2808376079 shutdown_connections 2024-04-20T21:25:23.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:23.450+0000 7f2982600640 1 -- 172.21.15.86:0/2808376079 wait complete. 2024-04-20T21:25:23.868 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 64, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T21:24:58.299257+0000', 'modified': '2024-04-20T21:25:22.062758+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 86, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7773}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7773': {'gid': 7773, 'name': 'b', 'rank': 0, 'incarnation': 62, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.144:6835/3981403733', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6833', 'nonce': 3981403733}, {'type': 'v1', 'addr': '172.21.15.144:6835', 'nonce': 3981403733}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-20T21:25:23.869 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7773:up:rejoin 2024-04-20T21:25:24.869 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:25:25.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.080+0000 7f205ac00640 1 Processor -- start 2024-04-20T21:25:25.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.080+0000 7f205ac00640 1 -- start start 2024-04-20T21:25:25.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.081+0000 7f205ac00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f205406f450 0x7f205415bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:25.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.081+0000 7f205ac00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20540cdb50 con 0x7f20540cd170 2024-04-20T21:25:25.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.082+0000 7f205ac00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20540b3e50 con 0x7f2054144e90 2024-04-20T21:25:25.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.082+0000 7f205ac00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2054145270 con 0x7f205406f450 2024-04-20T21:25:25.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.082+0000 7f2053400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f20540cd170 0x7f205406ed40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:53656/0 (socket says 172.21.15.86:53656) 2024-04-20T21:25:25.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.082+0000 7f2053400640 1 -- 172.21.15.86:0/3211521459 learned_addr learned my addr 172.21.15.86:0/3211521459 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:25:25.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.082+0000 7f2058c00640 1 --2- 172.21.15.86:0/3211521459 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f205406f450 0x7f205415bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:25.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.082+0000 7f2052a00640 1 -- 172.21.15.86:0/3211521459 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1786106477 0 0) 0x7f20540cdb50 con 0x7f20540cd170 2024-04-20T21:25:25.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.082+0000 7f2052a00640 1 -- 172.21.15.86:0/3211521459 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2038005730 con 0x7f20540cd170 2024-04-20T21:25:25.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.083+0000 7f2052a00640 1 -- 172.21.15.86:0/3211521459 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3732417169 0 0) 0x7f2038005730 con 0x7f20540cd170 2024-04-20T21:25:25.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.083+0000 7f2052a00640 1 -- 172.21.15.86:0/3211521459 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f205406f450 msgr2=0x7f205415bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:25.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.083+0000 7f2052a00640 1 --2- 172.21.15.86:0/3211521459 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f205406f450 0x7f205415bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:25.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.083+0000 7f2052a00640 1 -- 172.21.15.86:0/3211521459 >> v1:172.21.15.144:6789/0 conn(0x7f2054144e90 legacy=0x7f20540b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:25.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.083+0000 7f2052a00640 1 -- 172.21.15.86:0/3211521459 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20540b4130 con 0x7f20540cd170 2024-04-20T21:25:25.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.083+0000 7f2052a00640 1 -- 172.21.15.86:0/3211521459 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f2040001670 con 0x7f20540cd170 2024-04-20T21:25:25.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.083+0000 7f2052a00640 1 -- 172.21.15.86:0/3211521459 >> v1:172.21.15.86:6789/0 conn(0x7f20540cd170 legacy=0x7f205406ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:25.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.083+0000 7f2052a00640 1 --2- 172.21.15.86:0/3211521459 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2038005bf0 0x7f2038006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:25.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.083+0000 7f2052a00640 1 --2- 172.21.15.86:0/3211521459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2038006a10 0x7f2038006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:25.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.083+0000 7f2052a00640 1 --2- 172.21.15.86:0/3211521459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2038007420 0x7f203800dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:25.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.083+0000 7f2052a00640 1 -- 172.21.15.86:0/3211521459 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2054145270 con 0x7f2038007420 2024-04-20T21:25:25.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.083+0000 7f2052a00640 1 -- 172.21.15.86:0/3211521459 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f203800e320 con 0x7f2038006a10 2024-04-20T21:25:25.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.083+0000 7f2052a00640 1 -- 172.21.15.86:0/3211521459 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f203800e520 con 0x7f2038005bf0 2024-04-20T21:25:25.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.083+0000 7f2058c00640 1 --2- 172.21.15.86:0/3211521459 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f205406f450 0x7f205415bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:25:25.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.083+0000 7f2058c00640 1 --2- 172.21.15.86:0/3211521459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2038007420 0x7f203800dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:25.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.083+0000 7f2053e00640 1 --2- 172.21.15.86:0/3211521459 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2038005bf0 0x7f2038006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:25.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.084+0000 7f2058c00640 1 -- 172.21.15.86:0/3211521459 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2038005bf0 msgr2=0x7f2038006010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:25.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.084+0000 7f2053400640 1 --2- 172.21.15.86:0/3211521459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2038006a10 0x7f2038006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:25.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.084+0000 7f2058c00640 1 --2- 172.21.15.86:0/3211521459 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2038005bf0 0x7f2038006010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:25.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.084+0000 7f2058c00640 1 -- 172.21.15.86:0/3211521459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2038006a10 msgr2=0x7f2038006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:25.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.084+0000 7f2058c00640 1 --2- 172.21.15.86:0/3211521459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2038006a10 0x7f2038006e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:25.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.084+0000 7f2058c00640 1 -- 172.21.15.86:0/3211521459 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2038005730 con 0x7f2038007420 2024-04-20T21:25:25.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.084+0000 7f2053e00640 1 --2- 172.21.15.86:0/3211521459 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2038005bf0 0x7f2038006010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:25:25.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.085+0000 7f2058c00640 1 --2- 172.21.15.86:0/3211521459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2038007420 0x7f203800dbd0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f20540b35c0 tx=0x7f2038007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5ed9ef9af0db0aa server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:25.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.085+0000 7f2052a00640 1 -- 172.21.15.86:0/3211521459 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204c009b60 con 0x7f2038007420 2024-04-20T21:25:25.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.085+0000 7f2052a00640 1 -- 172.21.15.86:0/3211521459 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f204c02cc40 con 0x7f2038007420 2024-04-20T21:25:25.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.085+0000 7f2052a00640 1 -- 172.21.15.86:0/3211521459 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204c003990 con 0x7f2038007420 2024-04-20T21:25:25.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.086+0000 7f205ac00640 1 -- 172.21.15.86:0/3211521459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2038007420 msgr2=0x7f203800dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:25.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.086+0000 7f205ac00640 1 --2- 172.21.15.86:0/3211521459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2038007420 0x7f203800dbd0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f20540b35c0 tx=0x7f2038007a40 comp rx=0 tx=0).stop 2024-04-20T21:25:25.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.086+0000 7f2058c00640 1 -- 172.21.15.86:0/3211521459 reap_dead start 2024-04-20T21:25:25.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.086+0000 7f205ac00640 1 -- 172.21.15.86:0/3211521459 shutdown_connections 2024-04-20T21:25:25.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.086+0000 7f205ac00640 1 -- 172.21.15.86:0/3211521459 >> 172.21.15.86:0/3211521459 conn(0x7f20540b2360 msgr2=0x7f20540b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:25.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.086+0000 7f205ac00640 1 -- 172.21.15.86:0/3211521459 shutdown_connections 2024-04-20T21:25:25.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.086+0000 7f205ac00640 1 -- 172.21.15.86:0/3211521459 wait complete. 2024-04-20T21:25:25.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.088+0000 7f205ac00640 1 Processor -- start 2024-04-20T21:25:25.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.088+0000 7f205ac00640 1 -- start start 2024-04-20T21:25:25.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.089+0000 7f205ac00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2038007420 0x7f205413ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:25.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.089+0000 7f205ac00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20540c7560 0x7f20540c79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:25.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.089+0000 7f205ac00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f205413df60 0x7f205413e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:25.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.089+0000 7f205ac00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f20540d0190 con 0x7f2038007420 2024-04-20T21:25:25.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.089+0000 7f205ac00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f20540d0010 con 0x7f205413df60 2024-04-20T21:25:25.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.089+0000 7f205ac00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f20540cd600 con 0x7f20540c7560 2024-04-20T21:25:25.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.089+0000 7f2053e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2038007420 0x7f205413ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:25.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.089+0000 7f2053e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2038007420 0x7f205413ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53972/0 (socket says 172.21.15.86:53972) 2024-04-20T21:25:25.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.089+0000 7f2053e00640 1 -- 172.21.15.86:0/3118789757 learned_addr learned my addr 172.21.15.86:0/3118789757 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:25.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.090+0000 7f2053e00640 1 -- 172.21.15.86:0/3118789757 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20540c7560 msgr2=0x7f20540c79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:25.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.090+0000 7f2053e00640 1 --2- 172.21.15.86:0/3118789757 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20540c7560 0x7f20540c79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:25.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.090+0000 7f2053e00640 1 -- 172.21.15.86:0/3118789757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f205413df60 msgr2=0x7f205413e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:25.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.090+0000 7f2053e00640 1 --2- 172.21.15.86:0/3118789757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f205413df60 0x7f205413e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:25.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.090+0000 7f2053e00640 1 -- 172.21.15.86:0/3118789757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f205413d230 con 0x7f2038007420 2024-04-20T21:25:25.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.090+0000 7f2053e00640 1 --2- 172.21.15.86:0/3118789757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2038007420 0x7f205413ccf0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f204400d7a0 tx=0x7f204400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:25.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.090+0000 7f204be00640 1 -- 172.21.15.86:0/3118789757 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2044005070 con 0x7f2038007420 2024-04-20T21:25:25.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.090+0000 7f204be00640 1 -- 172.21.15.86:0/3118789757 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2044005210 con 0x7f2038007420 2024-04-20T21:25:25.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.090+0000 7f205ac00640 1 -- 172.21.15.86:0/3118789757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f205413ecc0 con 0x7f2038007420 2024-04-20T21:25:25.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.090+0000 7f204be00640 1 -- 172.21.15.86:0/3118789757 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2044007030 con 0x7f2038007420 2024-04-20T21:25:25.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.090+0000 7f205ac00640 1 -- 172.21.15.86:0/3118789757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20540c58e0 con 0x7f2038007420 2024-04-20T21:25:25.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.092+0000 7f205ac00640 1 -- 172.21.15.86:0/3118789757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20540c53f0 con 0x7f2038007420 2024-04-20T21:25:25.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.092+0000 7f204be00640 1 -- 172.21.15.86:0/3118789757 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f204401a020 con 0x7f2038007420 2024-04-20T21:25:25.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.093+0000 7f204be00640 1 --2- 172.21.15.86:0/3118789757 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f203406c920 0x7f203406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:25.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.093+0000 7f204be00640 1 -- 172.21.15.86:0/3118789757 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5795+0+0 (secure 0 0 0) 0x7f204408b340 con 0x7f2038007420 2024-04-20T21:25:25.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.094+0000 7f2053400640 1 --2- 172.21.15.86:0/3118789757 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f203406c920 0x7f203406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:25.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.095+0000 7f2053400640 1 --2- 172.21.15.86:0/3118789757 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f203406c920 0x7f203406ede0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f2040001200 tx=0x7f2040009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:25.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.098+0000 7f204be00640 1 -- 172.21.15.86:0/3118789757 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f205413d230 con 0x7f2038007420 2024-04-20T21:25:25.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.337+0000 7f205ac00640 1 -- 172.21.15.86:0/3118789757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20540689e0 con 0x7f2038007420 2024-04-20T21:25:25.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.339+0000 7f204be00640 1 -- 172.21.15.86:0/3118789757 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+5208 (secure 0 0 0) 0x7f2044059700 con 0x7f2038007420 2024-04-20T21:25:25.338 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:25:25.338 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":65,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7506,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/4013863257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4013863257},{"type":"v1","addr":"172.21.15.86:6836","nonce":4013863257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":7509,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2044883073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2044883073},{"type":"v1","addr":"172.21.15.86:6837","nonce":2044883073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7518,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/2351381466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2351381466},{"type":"v1","addr":"172.21.15.144:6834","nonce":2351381466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:24:58.299257+0000","modified":"2024-04-20T21:25:24.062448+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7773},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7773":{"gid":7773,"name":"b","rank":0,"incarnation":62,"state":"up:active","state_seq":4,"addr":"172.21.15.144:6835/3981403733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3981403733},{"type":"v1","addr":"172.21.15.144:6835","nonce":3981403733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7773,"qdb_cluster":[7773]},"id":6}]} 2024-04-20T21:25:25.338 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 65 2024-04-20T21:25:25.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.341+0000 7f205ac00640 1 -- 172.21.15.86:0/3118789757 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f203406c920 msgr2=0x7f203406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:25.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.341+0000 7f205ac00640 1 --2- 172.21.15.86:0/3118789757 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f203406c920 0x7f203406ede0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f2040001200 tx=0x7f2040009040 comp rx=0 tx=0).stop 2024-04-20T21:25:25.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.341+0000 7f205ac00640 1 -- 172.21.15.86:0/3118789757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2038007420 msgr2=0x7f205413ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:25.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.341+0000 7f205ac00640 1 --2- 172.21.15.86:0/3118789757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2038007420 0x7f205413ccf0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f204400d7a0 tx=0x7f204400dc60 comp rx=0 tx=0).stop 2024-04-20T21:25:25.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.341+0000 7f205ac00640 1 -- 172.21.15.86:0/3118789757 shutdown_connections 2024-04-20T21:25:25.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.341+0000 7f205ac00640 1 --2- 172.21.15.86:0/3118789757 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f203406c920 0x7f203406ede0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:25.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.341+0000 7f205ac00640 1 --2- 172.21.15.86:0/3118789757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f205413df60 0x7f205413e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:25.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.341+0000 7f205ac00640 1 --2- 172.21.15.86:0/3118789757 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f20540c7560 0x7f20540c79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:25.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.341+0000 7f205ac00640 1 --2- 172.21.15.86:0/3118789757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2038007420 0x7f205413ccf0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:25.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.341+0000 7f205ac00640 1 -- 172.21.15.86:0/3118789757 >> 172.21.15.86:0/3118789757 conn(0x7f20540b2360 msgr2=0x7f2054071d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:25.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.342+0000 7f205ac00640 1 -- 172.21.15.86:0/3118789757 shutdown_connections 2024-04-20T21:25:25.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.342+0000 7f205ac00640 1 -- 172.21.15.86:0/3118789757 wait complete. 2024-04-20T21:25:25.763 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 65, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T21:24:58.299257+0000', 'modified': '2024-04-20T21:25:24.062448+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 86, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7773}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7773': {'gid': 7773, 'name': 'b', 'rank': 0, 'incarnation': 62, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.144:6835/3981403733', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6833', 'nonce': 3981403733}, {'type': 'v1', 'addr': '172.21.15.144:6835', 'nonce': 3981403733}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7773, 'qdb_cluster': [7773]} 2024-04-20T21:25:25.763 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-20T21:25:25.763 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-20T21:25:25.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.974+0000 7f61b7200640 1 Processor -- start 2024-04-20T21:25:25.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.975+0000 7f61b7200640 1 -- start start 2024-04-20T21:25:25.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.976+0000 7f61b7200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f61b815e790 0x7f61b80d3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:25.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.976+0000 7f61b7200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61b80d4f70 0x7f61b80b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:25.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.976+0000 7f61b7200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61b80cffd0 con 0x7f61b815eb70 2024-04-20T21:25:25.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.976+0000 7f61b7200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f61b80d2860 con 0x7f61b815e790 2024-04-20T21:25:25.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.976+0000 7f61b7200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f61b80d2b40 con 0x7f61b80d4f70 2024-04-20T21:25:25.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.977+0000 7f61b5400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f61b815eb70 0x7f61b80cf780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54248/0 (socket says 172.21.15.86:54248) 2024-04-20T21:25:25.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.977+0000 7f61b5400640 1 -- 172.21.15.86:0/1549645254 learned_addr learned my addr 172.21.15.86:0/1549645254 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:25:25.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.977+0000 7f61b5e00640 1 --2- 172.21.15.86:0/1549645254 >> v2:172.21.15.144:3300/0 conn(0x7f61b815e790 0x7f61b80d3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:25.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.977+0000 7f61b6800640 1 --2- 172.21.15.86:0/1549645254 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61b80d4f70 0x7f61b80b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:25.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.977+0000 7f61b4a00640 1 -- 172.21.15.86:0/1549645254 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4230822652 0 0) 0x7f61b80cffd0 con 0x7f61b815eb70 2024-04-20T21:25:25.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.977+0000 7f61b4a00640 1 -- 172.21.15.86:0/1549645254 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f619c003520 con 0x7f61b815eb70 2024-04-20T21:25:25.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.977+0000 7f61b4a00640 1 -- 172.21.15.86:0/1549645254 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 743061 0 0) 0x7f619c003520 con 0x7f61b815eb70 2024-04-20T21:25:25.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.977+0000 7f61b4a00640 1 -- 172.21.15.86:0/1549645254 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61b80d4f70 msgr2=0x7f61b80b5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:25.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.977+0000 7f61b4a00640 1 --2- 172.21.15.86:0/1549645254 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61b80d4f70 0x7f61b80b5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:25.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.978+0000 7f61b4a00640 1 -- 172.21.15.86:0/1549645254 >> v2:172.21.15.144:3300/0 conn(0x7f61b815e790 msgr2=0x7f61b80d3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:25.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.978+0000 7f61b4a00640 1 --2- 172.21.15.86:0/1549645254 >> v2:172.21.15.144:3300/0 conn(0x7f61b815e790 0x7f61b80d3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:25.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.978+0000 7f61b4a00640 1 -- 172.21.15.86:0/1549645254 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61b80d2e20 con 0x7f61b815eb70 2024-04-20T21:25:25.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.978+0000 7f61b4a00640 1 -- 172.21.15.86:0/1549645254 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f61a8001a10 con 0x7f61b815eb70 2024-04-20T21:25:25.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.978+0000 7f61b4a00640 1 -- 172.21.15.86:0/1549645254 >> v1:172.21.15.86:6789/0 conn(0x7f61b815eb70 legacy=0x7f61b80cf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:25.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.978+0000 7f61b4a00640 1 --2- 172.21.15.86:0/1549645254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f619c0039b0 0x7f619c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:25.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.978+0000 7f61b4a00640 1 --2- 172.21.15.86:0/1549645254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f619c004800 0x7f619c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:25.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.978+0000 7f61b4a00640 1 --2- 172.21.15.86:0/1549645254 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61b80d4f70 0x7f619c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:25.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.978+0000 7f61b4a00640 1 -- 172.21.15.86:0/1549645254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f61b80d2860 con 0x7f619c0039b0 2024-04-20T21:25:25.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.978+0000 7f61b4a00640 1 -- 172.21.15.86:0/1549645254 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f61b80d2b40 con 0x7f619c004800 2024-04-20T21:25:25.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.978+0000 7f61b4a00640 1 -- 172.21.15.86:0/1549645254 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f619c00ba40 con 0x7f61b80d4f70 2024-04-20T21:25:25.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.978+0000 7f61b5e00640 1 --2- 172.21.15.86:0/1549645254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f619c0039b0 0x7f619c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:25.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.978+0000 7f61b5e00640 1 -- 172.21.15.86:0/1549645254 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61b80d4f70 msgr2=0x7f619c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:25.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.978+0000 7f61b5e00640 1 --2- 172.21.15.86:0/1549645254 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f61b80d4f70 0x7f619c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:25.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.978+0000 7f61b5e00640 1 -- 172.21.15.86:0/1549645254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f619c004800 msgr2=0x7f619c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:25.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.978+0000 7f61b5e00640 1 --2- 172.21.15.86:0/1549645254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f619c004800 0x7f619c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:25.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.978+0000 7f61b5e00640 1 -- 172.21.15.86:0/1549645254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f619c003520 con 0x7f619c0039b0 2024-04-20T21:25:25.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.980+0000 7f61b5e00640 1 --2- 172.21.15.86:0/1549645254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f619c0039b0 0x7f619c003dd0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f61a0009ac0 tx=0x7f619c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43ed8d5f3203c6bc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:25.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.980+0000 7f61b4a00640 1 -- 172.21.15.86:0/1549645254 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a0039070 con 0x7f619c0039b0 2024-04-20T21:25:25.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.980+0000 7f61b4a00640 1 -- 172.21.15.86:0/1549645254 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61a002c710 con 0x7f619c0039b0 2024-04-20T21:25:25.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.980+0000 7f61b4a00640 1 -- 172.21.15.86:0/1549645254 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a0009690 con 0x7f619c0039b0 2024-04-20T21:25:25.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.980+0000 7f61b7200640 1 -- 172.21.15.86:0/1549645254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f619c0039b0 msgr2=0x7f619c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:25.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.980+0000 7f61b7200640 1 --2- 172.21.15.86:0/1549645254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f619c0039b0 0x7f619c003dd0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f61a0009ac0 tx=0x7f619c005830 comp rx=0 tx=0).stop 2024-04-20T21:25:25.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.980+0000 7f61b6800640 1 -- 172.21.15.86:0/1549645254 reap_dead start 2024-04-20T21:25:25.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.981+0000 7f61b7200640 1 -- 172.21.15.86:0/1549645254 shutdown_connections 2024-04-20T21:25:25.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.981+0000 7f61b7200640 1 -- 172.21.15.86:0/1549645254 >> 172.21.15.86:0/1549645254 conn(0x7f61b808bf70 msgr2=0x7f61b80d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:25.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.981+0000 7f61b7200640 1 -- 172.21.15.86:0/1549645254 shutdown_connections 2024-04-20T21:25:25.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.981+0000 7f61b7200640 1 -- 172.21.15.86:0/1549645254 wait complete. 2024-04-20T21:25:25.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.982+0000 7f61b7200640 1 Processor -- start 2024-04-20T21:25:25.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.983+0000 7f61b7200640 1 -- start start 2024-04-20T21:25:25.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.984+0000 7f61b7200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f619c0039b0 0x7f61b81f72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:25.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.984+0000 7f61b7200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61b80a0100 0x7f61b80a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:25.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.984+0000 7f61b7200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61b81f4680 0x7f61b81f4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:25.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.984+0000 7f61b7200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f61b80b6110 con 0x7f61b80a0100 2024-04-20T21:25:25.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.984+0000 7f61b7200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f61b80b5f90 con 0x7f61b81f4680 2024-04-20T21:25:25.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.984+0000 7f61b7200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f61b80b6290 con 0x7f619c0039b0 2024-04-20T21:25:25.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.984+0000 7f61b5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61b80a0100 0x7f61b80a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:25.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.984+0000 7f61b5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61b80a0100 0x7f61b80a0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36396/0 (socket says 172.21.15.86:36396) 2024-04-20T21:25:25.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.984+0000 7f61b5400640 1 -- 172.21.15.86:0/1318136018 learned_addr learned my addr 172.21.15.86:0/1318136018 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:25.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.984+0000 7f61b5e00640 1 --2- 172.21.15.86:0/1318136018 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f619c0039b0 0x7f61b81f72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:25.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.984+0000 7f61b5400640 1 -- 172.21.15.86:0/1318136018 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f619c0039b0 msgr2=0x7f61b81f72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:25.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.984+0000 7f61b5400640 1 --2- 172.21.15.86:0/1318136018 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f619c0039b0 0x7f61b81f72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:25.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.984+0000 7f61b5400640 1 -- 172.21.15.86:0/1318136018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61b81f4680 msgr2=0x7f61b81f4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:25.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.984+0000 7f61b5400640 1 --2- 172.21.15.86:0/1318136018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61b81f4680 0x7f61b81f4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:25.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.984+0000 7f61b5400640 1 -- 172.21.15.86:0/1318136018 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61b80a0eb0 con 0x7f61b80a0100 2024-04-20T21:25:25.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.985+0000 7f61b5400640 1 --2- 172.21.15.86:0/1318136018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61b80a0100 0x7f61b80a0580 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f61a800ca80 tx=0x7f61a800cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:25.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.985+0000 7f61ae000640 1 -- 172.21.15.86:0/1318136018 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a8018070 con 0x7f61b80a0100 2024-04-20T21:25:25.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.985+0000 7f61ae000640 1 -- 172.21.15.86:0/1318136018 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61a800a7a0 con 0x7f61b80a0100 2024-04-20T21:25:25.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.985+0000 7f61ae000640 1 -- 172.21.15.86:0/1318136018 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a800ab90 con 0x7f61b80a0100 2024-04-20T21:25:25.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.985+0000 7f61b7200640 1 -- 172.21.15.86:0/1318136018 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61b80a1190 con 0x7f61b80a0100 2024-04-20T21:25:25.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.985+0000 7f61b7200640 1 -- 172.21.15.86:0/1318136018 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61b80a1750 con 0x7f61b80a0100 2024-04-20T21:25:25.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.988+0000 7f61ae000640 1 -- 172.21.15.86:0/1318136018 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f61a80070d0 con 0x7f61b80a0100 2024-04-20T21:25:25.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.989+0000 7f61ae000640 1 --2- 172.21.15.86:0/1318136018 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f619406c8f0 0x7f619406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:25.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.989+0000 7f61ae000640 1 -- 172.21.15.86:0/1318136018 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5795+0+0 (secure 0 0 0) 0x7f61a808b360 con 0x7f61b80a0100 2024-04-20T21:25:25.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.989+0000 7f61b5e00640 1 --2- 172.21.15.86:0/1318136018 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f619406c8f0 0x7f619406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:25.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.989+0000 7f61bfc09740 1 Processor -- start 2024-04-20T21:25:25.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.990+0000 7f61bfc09740 1 -- start start 2024-04-20T21:25:25.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.990+0000 7f61b5e00640 1 --2- 172.21.15.86:0/1318136018 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f619406c8f0 0x7f619406edb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f61a0036000 tx=0x7f61a0030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:25.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.990+0000 7f61bfc09740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0a68cc670 0x55b0a6883ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:25.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.990+0000 7f61bfc09740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a68826f0 0x55b0a68841e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:25.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.990+0000 7f61bfc09740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b0a687eb90 0x55b0a68ae980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:25.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.991+0000 7f61bfc09740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b0a67ea9d0 con 0x55b0a68826f0 2024-04-20T21:25:25.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.991+0000 7f61bfc09740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b0a6791270 con 0x55b0a68cc670 2024-04-20T21:25:25.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.991+0000 7f61bfc09740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55b0a67e6990 con 0x55b0a687eb90 2024-04-20T21:25:25.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.991+0000 7f61b5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a68826f0 0x55b0a68841e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:25.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.991+0000 7f61b5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a68826f0 0x55b0a68841e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36410/0 (socket says 172.21.15.86:36410) 2024-04-20T21:25:25.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.991+0000 7f61b5400640 1 -- 172.21.15.86:0/1401033570 learned_addr learned my addr 172.21.15.86:0/1401033570 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:25.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.991+0000 7f61b5400640 1 -- 172.21.15.86:0/1401033570 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b0a687eb90 msgr2=0x55b0a68ae980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:25.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.991+0000 7f61b5400640 1 --2- 172.21.15.86:0/1401033570 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b0a687eb90 0x55b0a68ae980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:25.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.991+0000 7f61b5400640 1 -- 172.21.15.86:0/1401033570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0a68cc670 msgr2=0x55b0a6883ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:25.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.991+0000 7f61b5400640 1 --2- 172.21.15.86:0/1401033570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0a68cc670 0x55b0a6883ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:25.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.991+0000 7f61b5400640 1 -- 172.21.15.86:0/1401033570 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0a64fd2d0 con 0x55b0a68826f0 2024-04-20T21:25:25.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.991+0000 7f61b5400640 1 --2- 172.21.15.86:0/1401033570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a68826f0 0x55b0a68841e0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f61a8058fb0 tx=0x7f61a8001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b4f650d45aae53 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:25.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.992+0000 7f61a6a00640 1 -- 172.21.15.86:0/1401033570 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a8057320 con 0x55b0a68826f0 2024-04-20T21:25:25.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.992+0000 7f61a6a00640 1 -- 172.21.15.86:0/1401033570 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61a808b0c0 con 0x55b0a68826f0 2024-04-20T21:25:25.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.992+0000 7f61a6a00640 1 -- 172.21.15.86:0/1401033570 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a8055690 con 0x55b0a68826f0 2024-04-20T21:25:25.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.992+0000 7f61bfc09740 1 -- 172.21.15.86:0/1401033570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a68826f0 msgr2=0x55b0a68841e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:25.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.992+0000 7f61bfc09740 1 --2- 172.21.15.86:0/1401033570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a68826f0 0x55b0a68841e0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f61a8058fb0 tx=0x7f61a8001250 comp rx=0 tx=0).stop 2024-04-20T21:25:25.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.992+0000 7f61bfc09740 1 -- 172.21.15.86:0/1401033570 shutdown_connections 2024-04-20T21:25:25.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.992+0000 7f61bfc09740 1 --2- 172.21.15.86:0/1401033570 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b0a687eb90 0x55b0a68ae980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:25.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.992+0000 7f61bfc09740 1 --2- 172.21.15.86:0/1401033570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a68826f0 0x55b0a68841e0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:25.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.992+0000 7f61bfc09740 1 --2- 172.21.15.86:0/1401033570 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0a68cc670 0x55b0a6883ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:25.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.992+0000 7f61bfc09740 1 -- 172.21.15.86:0/1401033570 >> 172.21.15.86:0/1401033570 conn(0x55b0a6880e00 msgr2=0x55b0a68c9980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:25.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.992+0000 7f61bfc09740 1 -- 172.21.15.86:0/1401033570 shutdown_connections 2024-04-20T21:25:25.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.993+0000 7f61bfc09740 1 -- 172.21.15.86:0/1401033570 wait complete. 2024-04-20T21:25:25.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.993+0000 7f61bfc09740 1 -- start start 2024-04-20T21:25:25.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.993+0000 7f61bfc09740 1 Processor -- start 2024-04-20T21:25:25.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.994+0000 7f61bfc09740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0a68826f0 0x55b0a68a4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:25.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.994+0000 7f61bfc09740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b0a68cc670 0x55b0a68ce050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:25.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.994+0000 7f61bfc09740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a6894530 0x55b0a68949b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:25.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.994+0000 7f61bfc09740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b0a6882c00 con 0x55b0a6894530 2024-04-20T21:25:25.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.994+0000 7f61bfc09740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b0a67a45e0 con 0x55b0a68826f0 2024-04-20T21:25:25.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.994+0000 7f61bfc09740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55b0a6882ee0 con 0x55b0a68cc670 2024-04-20T21:25:25.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.995+0000 7f61bfc09740 10 client.0 _ll_register_callbacks cb 0x55b0a64fcac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:25:25.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.995+0000 7f61b6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a6894530 0x55b0a68949b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:25.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.995+0000 7f61b6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a6894530 0x55b0a68949b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36422/0 (socket says 172.21.15.86:36422) 2024-04-20T21:25:25.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.995+0000 7f61b6800640 1 -- 172.21.15.86:0/1094904032 learned_addr learned my addr 172.21.15.86:0/1094904032 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:25.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.995+0000 7f61b6800640 1 -- 172.21.15.86:0/1094904032 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b0a68cc670 msgr2=0x55b0a68ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:25.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.995+0000 7f61b6800640 1 --2- 172.21.15.86:0/1094904032 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b0a68cc670 0x55b0a68ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:25.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.995+0000 7f61b6800640 1 -- 172.21.15.86:0/1094904032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0a68826f0 msgr2=0x55b0a68a4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:25.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.995+0000 7f61b6800640 1 --2- 172.21.15.86:0/1094904032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0a68826f0 0x55b0a68a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:25.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.995+0000 7f61b6800640 1 -- 172.21.15.86:0/1094904032 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0a65c76b0 con 0x55b0a6894530 2024-04-20T21:25:25.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.995+0000 7f61b6800640 1 --2- 172.21.15.86:0/1094904032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a6894530 0x55b0a68949b0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f61b815ef50 tx=0x7f61b000e890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5616bb5484d1fb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:25.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.995+0000 7f619b400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:25:25.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.996+0000 7f619b400640 1 -- 172.21.15.86:0/1094904032 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b0017070 con 0x55b0a6894530 2024-04-20T21:25:25.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.996+0000 7f619b400640 1 -- 172.21.15.86:0/1094904032 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5795+0+0 (secure 0 0 0) 0x7f61b0013680 con 0x55b0a6894530 2024-04-20T21:25:25.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.996+0000 7f61bfc09740 1 -- 172.21.15.86:0/1094904032 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61a808b0c0 con 0x55b0a6894530 2024-04-20T21:25:25.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.996+0000 7f619b400640 1 -- 172.21.15.86:0/1094904032 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f61b0022050 con 0x55b0a6894530 2024-04-20T21:25:25.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.996+0000 7f61bfc09740 10 client.7833 fetch_fsmap learned FSMap version 65 2024-04-20T21:25:25.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.996+0000 7f61bfc09740 1 -- 172.21.15.86:0/1094904032 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) -- 0x55b0a6895540 con 0x55b0a6894530 2024-04-20T21:25:25.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.996+0000 7f619b400640 1 -- 172.21.15.86:0/1094904032 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61a808b0c0 con 0x55b0a6894530 2024-04-20T21:25:25.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.996+0000 7f619b400640 1 -- 172.21.15.86:0/1094904032 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 65) ==== 3495+0+0 (secure 0 0 0) 0x7f61b0025020 con 0x55b0a6894530 2024-04-20T21:25:25.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.997+0000 7f61bfc09740 10 client.7833 fetch_fsmap finished waiting for FSMap version 65 2024-04-20T21:25:25.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.997+0000 7f61bfc09740 10 client.7833 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-20T21:25:25.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.997+0000 7f61bfc09740 20 client.7833 populate_metadata read hostname 'smithi086' 2024-04-20T21:25:25.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.997+0000 7f61bfc09740 1 --2- 172.21.15.86:0/1094904032 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x55b0a68a7060 0x55b0a68e8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:25.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.997+0000 7f61bfc09740 4 client.7833 mds_command: new command op to 7773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:25:25.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.997+0000 7f61bfc09740 1 -- 172.21.15.86:0/1094904032 --> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0a67f83f0 con 0x55b0a68a7060 2024-04-20T21:25:25.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.997+0000 7f61b5e00640 1 --2- 172.21.15.86:0/1094904032 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x55b0a68a7060 0x55b0a68e8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:25.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.998+0000 7f61b5e00640 1 --2- 172.21.15.86:0/1094904032 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x55b0a68a7060 0x55b0a68e8b00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1304a69be09a5241 server_cookie=ee4b2f3c5aeb638e in_seq=0 out_seq=0 2024-04-20T21:25:25.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:25.998+0000 7f619b400640 10 client.7833 ms_handle_connect on v2:172.21.15.144:6833/3981403733 2024-04-20T21:25:26.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.001+0000 7f619b400640 1 -- 172.21.15.86:0/1094904032 <== mds.0 v2:172.21.15.144:6833/3981403733 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b0a67f83f0 con 0x55b0a68a7060 2024-04-20T21:25:26.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.001+0000 7f619b400640 10 client.7833 handle_command_reply: tid=0 2024-04-20T21:25:26.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.001+0000 7f61bfc09740 1 client.7833 shutdown 2024-04-20T21:25:26.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.001+0000 7f61bfc09740 1 -- 172.21.15.86:0/1094904032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a6894530 msgr2=0x55b0a68949b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:26.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.001+0000 7f61bfc09740 1 --2- 172.21.15.86:0/1094904032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a6894530 0x55b0a68949b0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f61b815ef50 tx=0x7f61b000e890 comp rx=0 tx=0).stop 2024-04-20T21:25:26.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.001+0000 7f61bfc09740 1 -- 172.21.15.86:0/1094904032 shutdown_connections 2024-04-20T21:25:26.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.001+0000 7f61bfc09740 1 --2- 172.21.15.86:0/1094904032 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x55b0a68a7060 0x55b0a68e8b00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:26.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.001+0000 7f61bfc09740 1 --2- 172.21.15.86:0/1094904032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a6894530 0x55b0a68949b0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:26.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.001+0000 7f61bfc09740 1 --2- 172.21.15.86:0/1094904032 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b0a68cc670 0x55b0a68ce050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:26.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.001+0000 7f61bfc09740 1 --2- 172.21.15.86:0/1094904032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0a68826f0 0x55b0a68a4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:26.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.001+0000 7f619b400640 0 client.7833 ms_handle_reset on v2:172.21.15.144:6833/3981403733 2024-04-20T21:25:26.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.001+0000 7f61bfc09740 1 -- 172.21.15.86:0/1094904032 >> 172.21.15.86:0/1094904032 conn(0x55b0a687eb90 msgr2=0x55b0a68847b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:26.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.001+0000 7f61bfc09740 1 -- 172.21.15.86:0/1094904032 shutdown_connections 2024-04-20T21:25:26.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.002+0000 7f61bfc09740 1 -- 172.21.15.86:0/1094904032 wait complete. 2024-04-20T21:25:26.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.002+0000 7f61bfc09740 20 client.7833 trim_cache size 0 max 16384 2024-04-20T21:25:26.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.021+0000 7f61bfc09740 1 Processor -- start 2024-04-20T21:25:26.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.021+0000 7f61bfc09740 1 -- start start 2024-04-20T21:25:26.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.022+0000 7f61bfc09740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0a68826f0 0x55b0a6893650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:26.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.022+0000 7f61bfc09740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a6894530 0x55b0a687e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:26.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.022+0000 7f61bfc09740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b0a68cc670 0x55b0a68847b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:26.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.022+0000 7f61bfc09740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f61b8200400 con 0x55b0a6894530 2024-04-20T21:25:26.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.022+0000 7f61bfc09740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f61b80b9420 con 0x55b0a68826f0 2024-04-20T21:25:26.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.022+0000 7f61bfc09740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f61b81ffdb0 con 0x55b0a68cc670 2024-04-20T21:25:26.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.022+0000 7f61b5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a6894530 0x55b0a687e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:26.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.023+0000 7f61b5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a6894530 0x55b0a687e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36430/0 (socket says 172.21.15.86:36430) 2024-04-20T21:25:26.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.023+0000 7f61b5400640 1 -- 172.21.15.86:0/2825976274 learned_addr learned my addr 172.21.15.86:0/2825976274 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:26.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.023+0000 7f61b5400640 1 -- 172.21.15.86:0/2825976274 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b0a68cc670 msgr2=0x55b0a68847b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:26.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.023+0000 7f61b5400640 1 --2- 172.21.15.86:0/2825976274 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b0a68cc670 0x55b0a68847b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:26.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.023+0000 7f61b5400640 1 -- 172.21.15.86:0/2825976274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0a68826f0 msgr2=0x55b0a6893650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:26.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.023+0000 7f61b5400640 1 --2- 172.21.15.86:0/2825976274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0a68826f0 0x55b0a6893650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:26.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.023+0000 7f61b5400640 1 -- 172.21.15.86:0/2825976274 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0a68ac860 con 0x55b0a6894530 2024-04-20T21:25:26.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.023+0000 7f61b5400640 1 --2- 172.21.15.86:0/2825976274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a6894530 0x55b0a687e400 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f61a80616d0 tx=0x7f61a808b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53e59d343fd2ab55 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:26.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.023+0000 7f61a6000640 1 -- 172.21.15.86:0/2825976274 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a8057320 con 0x55b0a6894530 2024-04-20T21:25:26.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.023+0000 7f61a6000640 1 -- 172.21.15.86:0/2825976274 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61a8076070 con 0x55b0a6894530 2024-04-20T21:25:26.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.023+0000 7f61a6000640 1 -- 172.21.15.86:0/2825976274 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a800ad30 con 0x55b0a6894530 2024-04-20T21:25:26.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.024+0000 7f61bfc09740 1 -- 172.21.15.86:0/2825976274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a6894530 msgr2=0x55b0a687e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:26.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.024+0000 7f61bfc09740 1 --2- 172.21.15.86:0/2825976274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a6894530 0x55b0a687e400 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f61a80616d0 tx=0x7f61a808b050 comp rx=0 tx=0).stop 2024-04-20T21:25:26.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.024+0000 7f61bfc09740 1 -- 172.21.15.86:0/2825976274 shutdown_connections 2024-04-20T21:25:26.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.024+0000 7f61bfc09740 1 --2- 172.21.15.86:0/2825976274 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b0a68cc670 0x55b0a68847b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:26.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.024+0000 7f61bfc09740 1 --2- 172.21.15.86:0/2825976274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a6894530 0x55b0a687e400 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:26.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.024+0000 7f61bfc09740 1 --2- 172.21.15.86:0/2825976274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0a68826f0 0x55b0a6893650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:26.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.024+0000 7f61bfc09740 1 -- 172.21.15.86:0/2825976274 >> 172.21.15.86:0/2825976274 conn(0x55b0a687eb90 msgr2=0x55b0a68a4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:26.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.024+0000 7f61bfc09740 1 -- 172.21.15.86:0/2825976274 shutdown_connections 2024-04-20T21:25:26.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.024+0000 7f61bfc09740 1 -- 172.21.15.86:0/2825976274 wait complete. 2024-04-20T21:25:26.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.025+0000 7f61bfc09740 1 -- start start 2024-04-20T21:25:26.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.025+0000 7f61bfc09740 1 Processor -- start 2024-04-20T21:25:26.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.026+0000 7f61bfc09740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b0a68826f0 0x55b0a68847f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:26.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.026+0000 7f61bfc09740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0a6894530 0x55b0a68cab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:26.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.026+0000 7f61bfc09740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a68cc670 0x55b0a68cb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:26.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.026+0000 7f61bfc09740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b0a68ce690 con 0x55b0a68cc670 2024-04-20T21:25:26.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.026+0000 7f61bfc09740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b0a6895880 con 0x55b0a6894530 2024-04-20T21:25:26.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.026+0000 7f61bfc09740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55b0a68a5c80 con 0x55b0a68826f0 2024-04-20T21:25:26.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.026+0000 7f61bfc09740 10 client.0 _ll_register_callbacks cb 0x7f61b0009300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:25:26.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.026+0000 7f61b6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a68cc670 0x55b0a68cb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:26.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.026+0000 7f61b6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a68cc670 0x55b0a68cb060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36432/0 (socket says 172.21.15.86:36432) 2024-04-20T21:25:26.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.026+0000 7f61b6800640 1 -- 172.21.15.86:0/1917732247 learned_addr learned my addr 172.21.15.86:0/1917732247 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:26.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.027+0000 7f61b5e00640 1 --2- 172.21.15.86:0/1917732247 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b0a68826f0 0x55b0a68847f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:26.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.027+0000 7f61b5400640 1 --2- 172.21.15.86:0/1917732247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0a6894530 0x55b0a68cab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:26.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.027+0000 7f61b6800640 1 -- 172.21.15.86:0/1917732247 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b0a68826f0 msgr2=0x55b0a68847f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:26.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.027+0000 7f61b6800640 1 --2- 172.21.15.86:0/1917732247 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b0a68826f0 0x55b0a68847f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:26.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.027+0000 7f61b6800640 1 -- 172.21.15.86:0/1917732247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0a6894530 msgr2=0x55b0a68cab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:26.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.027+0000 7f61b6800640 1 --2- 172.21.15.86:0/1917732247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0a6894530 0x55b0a68cab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:26.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.027+0000 7f61b6800640 1 -- 172.21.15.86:0/1917732247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0a68eccc0 con 0x55b0a68cc670 2024-04-20T21:25:26.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.027+0000 7f61b6800640 1 --2- 172.21.15.86:0/1917732247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a68cc670 0x55b0a68cb060 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f61b0010010 tx=0x7f61b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85814b3e40e2ce68 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:26.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.027+0000 7f619b400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:25:26.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.027+0000 7f619b400640 1 -- 172.21.15.86:0/1917732247 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b002b060 con 0x55b0a68cc670 2024-04-20T21:25:26.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.027+0000 7f619b400640 1 -- 172.21.15.86:0/1917732247 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5795+0+0 (secure 0 0 0) 0x7f61b0005da0 con 0x55b0a68cc670 2024-04-20T21:25:26.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.027+0000 7f61bfc09740 1 -- 172.21.15.86:0/1917732247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61a8076070 con 0x55b0a68cc670 2024-04-20T21:25:26.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.027+0000 7f619b400640 1 -- 172.21.15.86:0/1917732247 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f61b0031050 con 0x55b0a68cc670 2024-04-20T21:25:26.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.027+0000 7f61bfc09740 10 client.7839 fetch_fsmap learned FSMap version 65 2024-04-20T21:25:26.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.028+0000 7f61bfc09740 1 -- 172.21.15.86:0/1917732247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) -- 0x55b0a694d800 con 0x55b0a68cc670 2024-04-20T21:25:26.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.028+0000 7f619b400640 1 -- 172.21.15.86:0/1917732247 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61a8076070 con 0x55b0a68cc670 2024-04-20T21:25:26.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.028+0000 7f619b400640 1 -- 172.21.15.86:0/1917732247 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 65) ==== 3495+0+0 (secure 0 0 0) 0x7f61b0034020 con 0x55b0a68cc670 2024-04-20T21:25:26.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.028+0000 7f61bfc09740 10 client.7839 fetch_fsmap finished waiting for FSMap version 65 2024-04-20T21:25:26.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.028+0000 7f61bfc09740 10 client.7839 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-20T21:25:26.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.028+0000 7f61bfc09740 20 client.7839 populate_metadata read hostname 'smithi086' 2024-04-20T21:25:26.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.028+0000 7f61bfc09740 1 --2- 172.21.15.86:0/1917732247 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x55b0a68cca50 0x55b0a68cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:26.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.028+0000 7f61bfc09740 4 client.7839 mds_command: new command op to 7773 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T21:25:26.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.028+0000 7f61bfc09740 1 -- 172.21.15.86:0/1917732247 --> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] -- command(tid 0: {"prefix": "status"}) -- 0x55b0a68a77b0 con 0x55b0a68cca50 2024-04-20T21:25:26.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.029+0000 7f61b5e00640 1 --2- 172.21.15.86:0/1917732247 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x55b0a68cca50 0x55b0a68cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:26.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.030+0000 7f61b5e00640 1 --2- 172.21.15.86:0/1917732247 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x55b0a68cca50 0x55b0a68cce30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1720a7f1e561bb54 server_cookie=bdb7bba5dc24ffc7 in_seq=0 out_seq=0 2024-04-20T21:25:26.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.030+0000 7f619b400640 10 client.7839 ms_handle_connect on v2:172.21.15.144:6833/3981403733 2024-04-20T21:25:26.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.030+0000 7f619b400640 1 -- 172.21.15.86:0/1917732247 <== mds.0 v2:172.21.15.144:6833/3981403733 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55b0a68a77b0 con 0x55b0a68cca50 2024-04-20T21:25:26.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.030+0000 7f619b400640 10 client.7839 handle_command_reply: tid=0 2024-04-20T21:25:26.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.030+0000 7f61bfc09740 1 client.7839 shutdown 2024-04-20T21:25:26.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.030+0000 7f61bfc09740 1 -- 172.21.15.86:0/1917732247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a68cc670 msgr2=0x55b0a68cb060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:26.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.030+0000 7f61bfc09740 1 --2- 172.21.15.86:0/1917732247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a68cc670 0x55b0a68cb060 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f61b0010010 tx=0x7f61b00046c0 comp rx=0 tx=0).stop 2024-04-20T21:25:26.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.030+0000 7f61bfc09740 1 -- 172.21.15.86:0/1917732247 shutdown_connections 2024-04-20T21:25:26.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.030+0000 7f61bfc09740 1 --2- 172.21.15.86:0/1917732247 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x55b0a68cca50 0x55b0a68cce30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:26.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.030+0000 7f61bfc09740 1 --2- 172.21.15.86:0/1917732247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0a68cc670 0x55b0a68cb060 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:26.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.030+0000 7f61bfc09740 1 --2- 172.21.15.86:0/1917732247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0a6894530 0x55b0a68cab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:26.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.030+0000 7f61bfc09740 1 --2- 172.21.15.86:0/1917732247 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b0a68826f0 0x55b0a68847f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:26.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.030+0000 7f619b400640 0 client.7839 ms_handle_reset on v2:172.21.15.144:6833/3981403733 2024-04-20T21:25:26.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.030+0000 7f61bfc09740 1 -- 172.21.15.86:0/1917732247 >> 172.21.15.86:0/1917732247 conn(0x55b0a687eb90 msgr2=0x55b0a68a4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:26.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.030+0000 7f61bfc09740 1 -- 172.21.15.86:0/1917732247 shutdown_connections 2024-04-20T21:25:26.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.031+0000 7f61bfc09740 1 -- 172.21.15.86:0/1917732247 wait complete. 2024-04-20T21:25:26.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.031+0000 7f61bfc09740 20 client.7839 trim_cache size 0 max 16384 2024-04-20T21:25:26.032 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:25:26.032 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "9c2dec1d-d7ae-4a83-a812-79922808e278", 2024-04-20T21:25:26.032 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-04-20T21:25:26.032 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 7773, 2024-04-20T21:25:26.032 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-20T21:25:26.032 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-20T21:25:26.032 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-20T21:25:26.032 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 4.9669130800000003, 2024-04-20T21:25:26.033 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 65, 2024-04-20T21:25:26.033 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 86, 2024-04-20T21:25:26.033 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 86, 2024-04-20T21:25:26.033 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 5.8637506220000004 2024-04-20T21:25:26.033 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:25:26.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.031+0000 7f61b7200640 1 -- 172.21.15.86:0/1318136018 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f619406c8f0 msgr2=0x7f619406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:26.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.031+0000 7f61b7200640 1 --2- 172.21.15.86:0/1318136018 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f619406c8f0 0x7f619406edb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f61a0036000 tx=0x7f61a0030000 comp rx=0 tx=0).stop 2024-04-20T21:25:26.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.031+0000 7f61b7200640 1 -- 172.21.15.86:0/1318136018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61b80a0100 msgr2=0x7f61b80a0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:26.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.031+0000 7f61b7200640 1 --2- 172.21.15.86:0/1318136018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61b80a0100 0x7f61b80a0580 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f61a800ca80 tx=0x7f61a800cf40 comp rx=0 tx=0).stop 2024-04-20T21:25:26.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.031+0000 7f61b7200640 1 -- 172.21.15.86:0/1318136018 shutdown_connections 2024-04-20T21:25:26.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.031+0000 7f61b7200640 1 --2- 172.21.15.86:0/1318136018 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f619406c8f0 0x7f619406edb0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:26.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.031+0000 7f61b7200640 1 --2- 172.21.15.86:0/1318136018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61b81f4680 0x7f61b81f4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:26.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.031+0000 7f61b7200640 1 --2- 172.21.15.86:0/1318136018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f61b80a0100 0x7f61b80a0580 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:26.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.031+0000 7f61b7200640 1 --2- 172.21.15.86:0/1318136018 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f619c0039b0 0x7f61b81f72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:26.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.031+0000 7f61b7200640 1 -- 172.21.15.86:0/1318136018 >> 172.21.15.86:0/1318136018 conn(0x7f61b808bf70 msgr2=0x7f61b80d3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:26.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.031+0000 7f61b7200640 1 -- 172.21.15.86:0/1318136018 shutdown_connections 2024-04-20T21:25:26.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.032+0000 7f61b7200640 1 -- 172.21.15.86:0/1318136018 wait complete. 2024-04-20T21:25:26.048 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T21:25:26.066 INFO:tasks.ceph.mds.b.smithi144.stderr:2024-04-20T21:25:26.067+0000 7f741b200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T21:25:26.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.244+0000 7fbce4e00640 1 Processor -- start 2024-04-20T21:25:26.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.244+0000 7fbce4e00640 1 -- start start 2024-04-20T21:25:26.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.245+0000 7fbce4e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbce0146130 0x7fbce006e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:26.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.245+0000 7fbce4e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbce00ccdd0 0x7fbce00cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:26.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.245+0000 7fbce4e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbce0071d00 0x7fbce0072190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:26.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.245+0000 7fbce4e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fbce006dc30 con 0x7fbce00ccdd0 2024-04-20T21:25:26.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.245+0000 7fbce4e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fbce00cd8e0 con 0x7fbce0146130 2024-04-20T21:25:26.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.245+0000 7fbce4e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbce00cdbc0 con 0x7fbce0071d00 2024-04-20T21:25:26.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.246+0000 7fbcd5400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbce00ccdd0 0x7fbce00cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:26.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.246+0000 7fbcd5400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbce00ccdd0 0x7fbce00cd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36446/0 (socket says 172.21.15.86:36446) 2024-04-20T21:25:26.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.246+0000 7fbcd5400640 1 -- 172.21.15.86:0/3332705546 learned_addr learned my addr 172.21.15.86:0/3332705546 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:26.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.246+0000 7fbcd5400640 1 -- 172.21.15.86:0/3332705546 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbce0071d00 msgr2=0x7fbce0072190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:26.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.246+0000 7fbcd5400640 1 --2- 172.21.15.86:0/3332705546 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbce0071d00 0x7fbce0072190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:26.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.246+0000 7fbcd5400640 1 -- 172.21.15.86:0/3332705546 >> v2:172.21.15.144:3300/0 conn(0x7fbce0146130 msgr2=0x7fbce006e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:26.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.246+0000 7fbcd5400640 1 --2- 172.21.15.86:0/3332705546 >> v2:172.21.15.144:3300/0 conn(0x7fbce0146130 0x7fbce006e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:26.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.246+0000 7fbcd5400640 1 -- 172.21.15.86:0/3332705546 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbce006ded0 con 0x7fbce00ccdd0 2024-04-20T21:25:26.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.247+0000 7fbcd5400640 1 --2- 172.21.15.86:0/3332705546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbce00ccdd0 0x7fbce00cd230 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fbce00b5950 tx=0x7fbcc802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e078dd64c00883 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:26.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.248+0000 7fbcdd400640 1 -- 172.21.15.86:0/3332705546 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc802cb00 con 0x7fbce00ccdd0 2024-04-20T21:25:26.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.248+0000 7fbcdd400640 1 -- 172.21.15.86:0/3332705546 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcc802cca0 con 0x7fbce00ccdd0 2024-04-20T21:25:26.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.248+0000 7fbcdd400640 1 -- 172.21.15.86:0/3332705546 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc8034610 con 0x7fbce00ccdd0 2024-04-20T21:25:26.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.248+0000 7fbce4e00640 1 -- 172.21.15.86:0/3332705546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbce00ccdd0 msgr2=0x7fbce00cd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:26.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.248+0000 7fbce4e00640 1 --2- 172.21.15.86:0/3332705546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbce00ccdd0 0x7fbce00cd230 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fbce00b5950 tx=0x7fbcc802bf60 comp rx=0 tx=0).stop 2024-04-20T21:25:26.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.248+0000 7fbce4e00640 1 -- 172.21.15.86:0/3332705546 shutdown_connections 2024-04-20T21:25:26.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.248+0000 7fbce4e00640 1 --2- 172.21.15.86:0/3332705546 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbce0071d00 0x7fbce0072190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:26.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.248+0000 7fbce4e00640 1 --2- 172.21.15.86:0/3332705546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbce00ccdd0 0x7fbce00cd230 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:26.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.248+0000 7fbce4e00640 1 --2- 172.21.15.86:0/3332705546 >> v2:172.21.15.144:3300/0 conn(0x7fbce0146130 0x7fbce006e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:26.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.248+0000 7fbce4e00640 1 -- 172.21.15.86:0/3332705546 >> 172.21.15.86:0/3332705546 conn(0x7fbce00b2360 msgr2=0x7fbce00b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:26.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.248+0000 7fbce4e00640 1 -- 172.21.15.86:0/3332705546 shutdown_connections 2024-04-20T21:25:26.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.249+0000 7fbce4e00640 1 -- 172.21.15.86:0/3332705546 wait complete. 2024-04-20T21:25:26.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.250+0000 7fbce4e00640 1 Processor -- start 2024-04-20T21:25:26.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.250+0000 7fbce4e00640 1 -- start start 2024-04-20T21:25:26.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.251+0000 7fbce4e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbce0071d00 0x7fbce01e4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:26.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.251+0000 7fbce4e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbce00ccdd0 0x7fbce01e6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:26.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.251+0000 7fbce4e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbce0146130 0x7fbce00b0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:26.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.251+0000 7fbce4e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbce015cc20 con 0x7fbce00ccdd0 2024-04-20T21:25:26.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.251+0000 7fbce4e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbce006f150 con 0x7fbce0071d00 2024-04-20T21:25:26.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.251+0000 7fbce4e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbce015cda0 con 0x7fbce0146130 2024-04-20T21:25:26.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.251+0000 7fbcd5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbce00ccdd0 0x7fbce01e6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:26.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.252+0000 7fbcd5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbce00ccdd0 0x7fbce01e6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36450/0 (socket says 172.21.15.86:36450) 2024-04-20T21:25:26.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.252+0000 7fbcd5400640 1 -- 172.21.15.86:0/3290856121 learned_addr learned my addr 172.21.15.86:0/3290856121 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:26.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.252+0000 7fbcd5400640 1 -- 172.21.15.86:0/3290856121 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbce0146130 msgr2=0x7fbce00b0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:26.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.252+0000 7fbcde800640 1 --2- 172.21.15.86:0/3290856121 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbce0146130 0x7fbce00b0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:26.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.252+0000 7fbcdde00640 1 --2- 172.21.15.86:0/3290856121 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbce0071d00 0x7fbce01e4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:26.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.252+0000 7fbcd5400640 1 --2- 172.21.15.86:0/3290856121 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbce0146130 0x7fbce00b0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:26.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.252+0000 7fbcd5400640 1 -- 172.21.15.86:0/3290856121 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbce0071d00 msgr2=0x7fbce01e4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:26.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.252+0000 7fbcd5400640 1 --2- 172.21.15.86:0/3290856121 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbce0071d00 0x7fbce01e4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:26.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.252+0000 7fbcd5400640 1 -- 172.21.15.86:0/3290856121 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbce015c3f0 con 0x7fbce00ccdd0 2024-04-20T21:25:26.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.252+0000 7fbcdde00640 1 --2- 172.21.15.86:0/3290856121 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbce0071d00 0x7fbce01e4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:25:26.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.252+0000 7fbcd5400640 1 --2- 172.21.15.86:0/3290856121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbce00ccdd0 0x7fbce01e6620 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8000f80 tx=0x7fbcc8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:26.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.253+0000 7fbcd6a00640 1 -- 172.21.15.86:0/3290856121 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc8043020 con 0x7fbce00ccdd0 2024-04-20T21:25:26.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.253+0000 7fbcd6a00640 1 -- 172.21.15.86:0/3290856121 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcc802ce50 con 0x7fbce00ccdd0 2024-04-20T21:25:26.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.253+0000 7fbcd6a00640 1 -- 172.21.15.86:0/3290856121 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc803e690 con 0x7fbce00ccdd0 2024-04-20T21:25:26.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.253+0000 7fbce4e00640 1 -- 172.21.15.86:0/3290856121 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbce01e5580 con 0x7fbce00ccdd0 2024-04-20T21:25:26.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.253+0000 7fbce4e00640 1 -- 172.21.15.86:0/3290856121 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbce01e5b40 con 0x7fbce00ccdd0 2024-04-20T21:25:26.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.254+0000 7fbce4e00640 1 -- 172.21.15.86:0/3290856121 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbce00689e0 con 0x7fbce00ccdd0 2024-04-20T21:25:26.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.255+0000 7fbcd6a00640 1 -- 172.21.15.86:0/3290856121 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbcc8046020 con 0x7fbce00ccdd0 2024-04-20T21:25:26.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.256+0000 7fbcd6a00640 1 --2- 172.21.15.86:0/3290856121 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbcbc06c8f0 0x7fbcbc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:26.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.256+0000 7fbcd6a00640 1 -- 172.21.15.86:0/3290856121 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5795+0+0 (secure 0 0 0) 0x7fbcc80ae450 con 0x7fbce00ccdd0 2024-04-20T21:25:26.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.256+0000 7fbcdde00640 1 --2- 172.21.15.86:0/3290856121 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbcbc06c8f0 0x7fbcbc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:26.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.257+0000 7fbcdde00640 1 --2- 172.21.15.86:0/3290856121 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbcbc06c8f0 0x7fbcbc06edb0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fbcd000a870 tx=0x7fbcd0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:26.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.260+0000 7fbcd6a00640 1 -- 172.21.15.86:0/3290856121 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbcc804d030 con 0x7fbce00ccdd0 2024-04-20T21:25:26.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.499+0000 7fbce4e00640 1 -- 172.21.15.86:0/3290856121 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fbce015ff50 con 0x7fbce00ccdd0 2024-04-20T21:25:26.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.500+0000 7fbcd6a00640 1 -- 172.21.15.86:0/3290856121 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v65) ==== 53+0+73 (secure 0 0 0) 0x7fbce015c3f0 con 0x7fbce00ccdd0 2024-04-20T21:25:26.499 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T21:25:26.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.503+0000 7fbce4e00640 1 -- 172.21.15.86:0/3290856121 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbcbc06c8f0 msgr2=0x7fbcbc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:26.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.503+0000 7fbce4e00640 1 --2- 172.21.15.86:0/3290856121 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbcbc06c8f0 0x7fbcbc06edb0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fbcd000a870 tx=0x7fbcd0008040 comp rx=0 tx=0).stop 2024-04-20T21:25:26.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.503+0000 7fbce4e00640 1 -- 172.21.15.86:0/3290856121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbce00ccdd0 msgr2=0x7fbce01e6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:26.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.503+0000 7fbce4e00640 1 --2- 172.21.15.86:0/3290856121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbce00ccdd0 0x7fbce01e6620 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8000f80 tx=0x7fbcc8004a40 comp rx=0 tx=0).stop 2024-04-20T21:25:26.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.503+0000 7fbce4e00640 1 -- 172.21.15.86:0/3290856121 shutdown_connections 2024-04-20T21:25:26.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.503+0000 7fbce4e00640 1 --2- 172.21.15.86:0/3290856121 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbcbc06c8f0 0x7fbcbc06edb0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:26.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.503+0000 7fbce4e00640 1 --2- 172.21.15.86:0/3290856121 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbce0146130 0x7fbce00b0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:26.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.503+0000 7fbce4e00640 1 --2- 172.21.15.86:0/3290856121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbce00ccdd0 0x7fbce01e6620 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:26.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.503+0000 7fbce4e00640 1 --2- 172.21.15.86:0/3290856121 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbce0071d00 0x7fbce01e4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:26.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.503+0000 7fbce4e00640 1 -- 172.21.15.86:0/3290856121 >> 172.21.15.86:0/3290856121 conn(0x7fbce00b2360 msgr2=0x7fbce00b0410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:26.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.503+0000 7fbce4e00640 1 -- 172.21.15.86:0/3290856121 shutdown_connections 2024-04-20T21:25:26.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26.503+0000 7fbce4e00640 1 -- 172.21.15.86:0/3290856121 wait complete. 2024-04-20T21:25:26.520 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T21:25:26.520 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T21:25:26.520 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-04-20T21:25:26.520 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-20T21:25:26.520 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T21:25:26.520 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T21:25:26.521 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T21:25:26.521 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T21:25:26.521 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:25:26.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:25:26.521 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-04-20T21:25:26.573 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T21:25:26.573 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T21:25:26.573 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T21:25:26.573 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:25:26.573 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-20T21:25:26.573 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:25:26.574 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:25:26.574 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:25:26.574 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-04-20T21:25:26.574 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:25:26.574 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:25:26.574 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-04-20T21:25:26.574 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:25:26.574 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:25:26.574 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T21:25:26.574 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:25:26.574 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T21:25:26.574 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42229sec preferred_lft 42229sec 2024-04-20T21:25:26.574 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-04-20T21:25:26.575 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:25:26.575 INFO:teuthology.orchestra.run.smithi086.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T21:25:26.575 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 86:25:97:2c:9c:bb brd ff:ff:ff:ff:ff:ff 2024-04-20T21:25:26.575 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-20T21:25:26.575 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:25:26.575 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::a4fb:30ff:fe90:2d6a/64 scope link 2024-04-20T21:25:26.575 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:25:26.575 INFO:teuthology.orchestra.run.smithi086.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T21:25:26.575 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 86:25:97:2c:9c:bb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:25:26.575 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::8425:97ff:fe2c:9cbb/64 scope link 2024-04-20T21:25:26.575 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:25:26.575 INFO:teuthology.orchestra.run.smithi086.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T21:25:26.575 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c2:9c:8b:55:ca:84 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:25:26.575 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c09c:8bff:fe55:ca84/64 scope link 2024-04-20T21:25:26.576 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:25:26.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:25:26.576 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T21:25:26.652 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-20T21:25:26.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:25:26.656 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-04-20T21:25:26.710 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-04-20T21:25:26.710 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T21:25:26.710 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T21:25:26.710 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T21:25:26.710 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T21:25:26.710 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:25:26.711 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-20T21:25:26.711 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T21:25:26.711 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T21:25:26.711 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T21:25:26.711 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-20T21:25:26.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:25:26.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:25:26.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:25:26.920 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-20T21:25:26.966 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-20T21:25:26.966 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-20T21:25:26.967 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:25:26.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:25:26.967 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-04-20T21:25:27.096 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-20T21:25:27.096 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T21:25:27.096 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-20T21:25:27.147 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:25:27.147 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T21:25:27.147 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-20T21:25:27.197 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-04-20T21:25:27.277 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T21:25:27.334 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-04-20T21:25:27.334 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-04-20T21:25:27.335 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-04-20T21:25:27.335 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-04-20T21:25:27.335 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-04-20T21:25:27.335 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-04-20T21:25:27.335 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-04-20T21:25:27.335 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-04-20T21:25:27.335 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-04-20T21:25:27.335 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-04-20T21:25:27.335 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-04-20T21:25:27.335 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/selinux 2024-04-20T21:25:27.335 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T21:25:27.335 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-04-20T21:25:27.335 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-04-20T21:25:27.336 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-04-20T21:25:27.336 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-04-20T21:25:27.336 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-04-20T21:25:27.336 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-04-20T21:25:27.336 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T21:25:27.336 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T21:25:27.336 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T21:25:27.336 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T21:25:27.336 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-04-20T21:25:27.336 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-04-20T21:25:27.336 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T21:25:27.336 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T21:25:27.336 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T21:25:27.336 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T21:25:27.337 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-04-20T21:25:27.337 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:25:27.337 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:25:27.337 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:25:27.337 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:25:27.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:25:27.338 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-04-20T21:25:27.388 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T21:25:27.388 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-20T21:25:27.393 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-04-20T21:25:27.459 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T21:25:27.498 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-04-20T21:25:27.498 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-04-20T21:25:27.498 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-04-20T21:25:27.498 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-04-20T21:25:27.498 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-04-20T21:25:27.498 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-04-20T21:25:27.498 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-04-20T21:25:27.498 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-04-20T21:25:27.499 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-04-20T21:25:27.499 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-04-20T21:25:27.499 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-04-20T21:25:27.499 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/selinux 2024-04-20T21:25:27.499 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T21:25:27.499 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-04-20T21:25:27.499 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-04-20T21:25:27.499 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-04-20T21:25:27.499 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-04-20T21:25:27.499 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-04-20T21:25:27.499 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-04-20T21:25:27.499 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T21:25:27.499 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T21:25:27.499 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T21:25:27.499 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T21:25:27.500 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-04-20T21:25:27.500 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-04-20T21:25:27.500 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T21:25:27.500 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T21:25:27.500 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T21:25:27.500 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T21:25:27.500 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-04-20T21:25:27.500 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:25:27.500 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:25:27.500 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:25:27.500 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:25:27.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:25:27.501 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-04-20T21:25:27.554 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:2024-04-20T21:25:27.556+0000 7fec876cf480 -1 init, newargv = 0x55fbb2096130 newargc=15 2024-04-20T21:25:27.555 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[54464]: starting ceph client 2024-04-20T21:25:27.566 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[54464]: starting fuse 2024-04-20T21:25:27.584 INFO:teuthology.orchestra.run.smithi086.stdout:41 2024-04-20T21:25:27.584 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-04-20T21:25:27.585 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer -- python3 -c ' 2024-04-20T21:25:27.585 DEBUG:teuthology.orchestra.run.smithi086:> import glob 2024-04-20T21:25:27.585 DEBUG:teuthology.orchestra.run.smithi086:> import re 2024-04-20T21:25:27.585 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:25:27.585 DEBUG:teuthology.orchestra.run.smithi086:> import subprocess 2024-04-20T21:25:27.585 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:25:27.585 DEBUG:teuthology.orchestra.run.smithi086:> def _find_admin_socket(client_name): 2024-04-20T21:25:27.585 DEBUG:teuthology.orchestra.run.smithi086:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-20T21:25:27.585 DEBUG:teuthology.orchestra.run.smithi086:> files = glob.glob(asok_path) 2024-04-20T21:25:27.585 DEBUG:teuthology.orchestra.run.smithi086:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-20T21:25:27.585 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:25:27.585 DEBUG:teuthology.orchestra.run.smithi086:> # Given a non-glob path, it better be there 2024-04-20T21:25:27.585 DEBUG:teuthology.orchestra.run.smithi086:> if "*" not in asok_path: 2024-04-20T21:25:27.585 DEBUG:teuthology.orchestra.run.smithi086:> assert(len(files) == 1) 2024-04-20T21:25:27.586 DEBUG:teuthology.orchestra.run.smithi086:> return files[0] 2024-04-20T21:25:27.586 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:25:27.586 DEBUG:teuthology.orchestra.run.smithi086:> for f in files: 2024-04-20T21:25:27.586 DEBUG:teuthology.orchestra.run.smithi086:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T21:25:27.586 DEBUG:teuthology.orchestra.run.smithi086:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T21:25:27.586 DEBUG:teuthology.orchestra.run.smithi086:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T21:25:27.586 DEBUG:teuthology.orchestra.run.smithi086:> contents = proc_f.read() 2024-04-20T21:25:27.586 DEBUG:teuthology.orchestra.run.smithi086:> if mountpoint in contents: 2024-04-20T21:25:27.586 DEBUG:teuthology.orchestra.run.smithi086:> return f 2024-04-20T21:25:27.586 DEBUG:teuthology.orchestra.run.smithi086:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T21:25:27.586 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:25:27.586 DEBUG:teuthology.orchestra.run.smithi086:> print(_find_admin_socket("client.0")) 2024-04-20T21:25:27.586 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:25:27.777 INFO:teuthology.orchestra.run.smithi086.stdout:/var/run/ceph/ceph-client.0.54464.asok 2024-04-20T21:25:27.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:25:27.795 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.54464.asok 2024-04-20T21:25:27.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:25:27.795 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.54464.asok status 2024-04-20T21:25:28.017 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:25:28.017 INFO:teuthology.orchestra.run.smithi086.stdout: "metadata": { 2024-04-20T21:25:28.017 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T21:25:28.017 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T21:25:28.018 INFO:teuthology.orchestra.run.smithi086.stdout: "entity_id": "0", 2024-04-20T21:25:28.018 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": "smithi086", 2024-04-20T21:25:28.018 INFO:teuthology.orchestra.run.smithi086.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-20T21:25:28.018 INFO:teuthology.orchestra.run.smithi086.stdout: "pid": "54464", 2024-04-20T21:25:28.018 INFO:teuthology.orchestra.run.smithi086.stdout: "root": "/" 2024-04-20T21:25:28.018 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:25:28.018 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_count": 0, 2024-04-20T21:25:28.018 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_pinned_count": 0, 2024-04-20T21:25:28.018 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 7854, 2024-04-20T21:25:28.018 INFO:teuthology.orchestra.run.smithi086.stdout: "inst": { 2024-04-20T21:25:28.018 INFO:teuthology.orchestra.run.smithi086.stdout: "name": { 2024-04-20T21:25:28.018 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "client", 2024-04-20T21:25:28.018 INFO:teuthology.orchestra.run.smithi086.stdout: "num": 7854 2024-04-20T21:25:28.018 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:25:28.019 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2024-04-20T21:25:28.019 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-04-20T21:25:28.019 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2024-04-20T21:25:28.019 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 3504728314 2024-04-20T21:25:28.019 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:25:28.019 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:25:28.019 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2024-04-20T21:25:28.019 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-04-20T21:25:28.019 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2024-04-20T21:25:28.019 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 3504728314 2024-04-20T21:25:28.019 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:25:28.019 INFO:teuthology.orchestra.run.smithi086.stdout: "inst_str": "client.7854 192.168.0.1:0/3504728314", 2024-04-20T21:25:28.019 INFO:teuthology.orchestra.run.smithi086.stdout: "addr_str": "192.168.0.1:0/3504728314", 2024-04-20T21:25:28.019 INFO:teuthology.orchestra.run.smithi086.stdout: "inode_count": 1, 2024-04-20T21:25:28.020 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_epoch": 65, 2024-04-20T21:25:28.020 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch": 86, 2024-04-20T21:25:28.020 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch_barrier": 0, 2024-04-20T21:25:28.020 INFO:teuthology.orchestra.run.smithi086.stdout: "blocklisted": false, 2024-04-20T21:25:28.020 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs" 2024-04-20T21:25:28.020 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:25:28.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:25:28.032 DEBUG:teuthology.orchestra.run.smithi086:> stat --file-system '--printf=%T 2024-04-20T21:25:28.032 DEBUG:teuthology.orchestra.run.smithi086:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-20T21:25:28.085 INFO:teuthology.orchestra.run.smithi086.stdout:fuseblk 2024-04-20T21:25:28.085 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-20T21:25:28.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:25:28.086 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-20T21:25:28.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:25:28.159 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat testdir/hardlink) 2024-04-20T21:25:28.218 INFO:tasks.ceph.mds.b.smithi144.stderr:2024-04-20T21:25:28.219+0000 7f7416200640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-04-20T21:25:28.220 INFO:teuthology.orchestra.run.smithi086.stderr:stat: cannot statx 'testdir/hardlink': Input/output error 2024-04-20T21:25:28.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:25:28.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:25:28.222 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive 2024-04-20T21:25:28.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.428+0000 7f23d5600640 1 Processor -- start 2024-04-20T21:25:28.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.428+0000 7f23d5600640 1 -- start start 2024-04-20T21:25:28.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.430+0000 7f23d5600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23d006f450 0x7f23d00cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:28.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.430+0000 7f23d5600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f23d00cd700 0x7f23d00b39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:28.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.430+0000 7f23d5600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23d015ca50 con 0x7f23d00afb60 2024-04-20T21:25:28.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.430+0000 7f23d5600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f23d00cdef0 con 0x7f23d00cd700 2024-04-20T21:25:28.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.430+0000 7f23d5600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f23d015bc90 con 0x7f23d006f450 2024-04-20T21:25:28.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.430+0000 7f23cea00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f23d00afb60 0x7f23d006ed40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54252/0 (socket says 172.21.15.86:54252) 2024-04-20T21:25:28.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.430+0000 7f23cea00640 1 -- 172.21.15.86:0/2253086081 learned_addr learned my addr 172.21.15.86:0/2253086081 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:25:28.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.431+0000 7f23ce000640 1 --2- 172.21.15.86:0/2253086081 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23d006f450 0x7f23d00cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:28.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.431+0000 7f23cf400640 1 --2- 172.21.15.86:0/2253086081 >> v2:172.21.15.144:3300/0 conn(0x7f23d00cd700 0x7f23d00b39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:28.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.431+0000 7f23cd600640 1 -- 172.21.15.86:0/2253086081 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 718312583 0 0) 0x7f23d015ca50 con 0x7f23d00afb60 2024-04-20T21:25:28.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.431+0000 7f23cd600640 1 -- 172.21.15.86:0/2253086081 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f23b4003520 con 0x7f23d00afb60 2024-04-20T21:25:28.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.431+0000 7f23cd600640 1 -- 172.21.15.86:0/2253086081 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1330079721 0 0) 0x7f23b4003520 con 0x7f23d00afb60 2024-04-20T21:25:28.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.431+0000 7f23cd600640 1 -- 172.21.15.86:0/2253086081 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23d006f450 msgr2=0x7f23d00cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:28.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.431+0000 7f23cd600640 1 --2- 172.21.15.86:0/2253086081 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23d006f450 0x7f23d00cd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.431+0000 7f23cd600640 1 -- 172.21.15.86:0/2253086081 >> v2:172.21.15.144:3300/0 conn(0x7f23d00cd700 msgr2=0x7f23d00b39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:28.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.431+0000 7f23cd600640 1 --2- 172.21.15.86:0/2253086081 >> v2:172.21.15.144:3300/0 conn(0x7f23d00cd700 0x7f23d00b39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.431+0000 7f23cd600640 1 -- 172.21.15.86:0/2253086081 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23d015bf70 con 0x7f23d00afb60 2024-04-20T21:25:28.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.431+0000 7f23cd600640 1 -- 172.21.15.86:0/2253086081 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f23c0001670 con 0x7f23d00afb60 2024-04-20T21:25:28.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.431+0000 7f23ce000640 1 --2- 172.21.15.86:0/2253086081 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23d006f450 0x7f23d00cd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:25:28.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.431+0000 7f23cd600640 1 -- 172.21.15.86:0/2253086081 >> v1:172.21.15.86:6789/0 conn(0x7f23d00afb60 legacy=0x7f23d006ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:28.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.431+0000 7f23cd600640 1 --2- 172.21.15.86:0/2253086081 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23b40039b0 0x7f23b4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:28.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.431+0000 7f23cd600640 1 --2- 172.21.15.86:0/2253086081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23b4004800 0x7f23b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:28.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.431+0000 7f23cd600640 1 --2- 172.21.15.86:0/2253086081 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23d006f450 0x7f23b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:28.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.431+0000 7f23cd600640 1 -- 172.21.15.86:0/2253086081 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f23d00cdef0 con 0x7f23b4004800 2024-04-20T21:25:28.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.431+0000 7f23cd600640 1 -- 172.21.15.86:0/2253086081 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f23d015bc90 con 0x7f23b40039b0 2024-04-20T21:25:28.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.431+0000 7f23cd600640 1 -- 172.21.15.86:0/2253086081 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f23b400ba40 con 0x7f23d006f450 2024-04-20T21:25:28.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.431+0000 7f23ce000640 1 --2- 172.21.15.86:0/2253086081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23b4004800 0x7f23b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:28.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.432+0000 7f23cf400640 1 --2- 172.21.15.86:0/2253086081 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23d006f450 0x7f23b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:28.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.432+0000 7f23cea00640 1 --2- 172.21.15.86:0/2253086081 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23b40039b0 0x7f23b4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:28.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.432+0000 7f23ce000640 1 -- 172.21.15.86:0/2253086081 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23d006f450 msgr2=0x7f23b40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:28.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.432+0000 7f23ce000640 1 --2- 172.21.15.86:0/2253086081 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23d006f450 0x7f23b40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.432+0000 7f23ce000640 1 -- 172.21.15.86:0/2253086081 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23b40039b0 msgr2=0x7f23b4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:28.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.432+0000 7f23ce000640 1 --2- 172.21.15.86:0/2253086081 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23b40039b0 0x7f23b4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.432+0000 7f23ce000640 1 -- 172.21.15.86:0/2253086081 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23b4003520 con 0x7f23b4004800 2024-04-20T21:25:28.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.432+0000 7f23cea00640 1 --2- 172.21.15.86:0/2253086081 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23b40039b0 0x7f23b4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:25:28.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.433+0000 7f23ce000640 1 --2- 172.21.15.86:0/2253086081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23b4004800 0x7f23b4004c80 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f23bc0098b0 tx=0x7f23b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeab8f4147e9b463 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:28.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.433+0000 7f23cd600640 1 -- 172.21.15.86:0/2253086081 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23bc034070 con 0x7f23b4004800 2024-04-20T21:25:28.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.433+0000 7f23cd600640 1 -- 172.21.15.86:0/2253086081 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23bc030630 con 0x7f23b4004800 2024-04-20T21:25:28.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.433+0000 7f23cd600640 1 -- 172.21.15.86:0/2253086081 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23bc00f040 con 0x7f23b4004800 2024-04-20T21:25:28.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.434+0000 7f23d5600640 1 -- 172.21.15.86:0/2253086081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23b4004800 msgr2=0x7f23b4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:28.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.434+0000 7f23d5600640 1 --2- 172.21.15.86:0/2253086081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23b4004800 0x7f23b4004c80 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f23bc0098b0 tx=0x7f23b4005830 comp rx=0 tx=0).stop 2024-04-20T21:25:28.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.434+0000 7f23cf400640 1 -- 172.21.15.86:0/2253086081 reap_dead start 2024-04-20T21:25:28.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.434+0000 7f23d5600640 1 -- 172.21.15.86:0/2253086081 shutdown_connections 2024-04-20T21:25:28.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.434+0000 7f23d5600640 1 -- 172.21.15.86:0/2253086081 >> 172.21.15.86:0/2253086081 conn(0x7f23d00b2180 msgr2=0x7f23d00b2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:28.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.434+0000 7f23d5600640 1 -- 172.21.15.86:0/2253086081 shutdown_connections 2024-04-20T21:25:28.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.434+0000 7f23d5600640 1 -- 172.21.15.86:0/2253086081 wait complete. 2024-04-20T21:25:28.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.436+0000 7f23d5600640 1 Processor -- start 2024-04-20T21:25:28.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.436+0000 7f23d5600640 1 -- start start 2024-04-20T21:25:28.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.437+0000 7f23d5600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23b4004800 0x7f23d0142180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:28.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.437+0000 7f23d5600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23d0154aa0 0x7f23d0154f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:28.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.437+0000 7f23d5600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23d01562c0 0x7f23d0156770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:28.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.437+0000 7f23d5600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f23d00cdb90 con 0x7f23d01562c0 2024-04-20T21:25:28.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.437+0000 7f23d5600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f23d015c530 con 0x7f23d0154aa0 2024-04-20T21:25:28.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.437+0000 7f23d5600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f23d00cd1c0 con 0x7f23b4004800 2024-04-20T21:25:28.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.437+0000 7f23cf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23d01562c0 0x7f23d0156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:28.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.437+0000 7f23cf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23d01562c0 0x7f23d0156770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36460/0 (socket says 172.21.15.86:36460) 2024-04-20T21:25:28.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.437+0000 7f23cf400640 1 -- 172.21.15.86:0/3507952774 learned_addr learned my addr 172.21.15.86:0/3507952774 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:28.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.438+0000 7f23cea00640 1 --2- 172.21.15.86:0/3507952774 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23b4004800 0x7f23d0142180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:28.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.438+0000 7f23ce000640 1 --2- 172.21.15.86:0/3507952774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23d0154aa0 0x7f23d0154f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:28.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.438+0000 7f23cf400640 1 -- 172.21.15.86:0/3507952774 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23b4004800 msgr2=0x7f23d0142180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:28.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.438+0000 7f23cf400640 1 --2- 172.21.15.86:0/3507952774 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23b4004800 0x7f23d0142180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.438+0000 7f23cf400640 1 -- 172.21.15.86:0/3507952774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23d0154aa0 msgr2=0x7f23d0154f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:28.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.438+0000 7f23cf400640 1 --2- 172.21.15.86:0/3507952774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23d0154aa0 0x7f23d0154f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.438+0000 7f23cf400640 1 -- 172.21.15.86:0/3507952774 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23d0156ff0 con 0x7f23d01562c0 2024-04-20T21:25:28.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.438+0000 7f23cf400640 1 --2- 172.21.15.86:0/3507952774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23d01562c0 0x7f23d0156770 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f23d01560d0 tx=0x7f23c8007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:28.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.438+0000 7f23c6a00640 1 -- 172.21.15.86:0/3507952774 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c8014410 con 0x7f23d01562c0 2024-04-20T21:25:28.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.438+0000 7f23c6a00640 1 -- 172.21.15.86:0/3507952774 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23c8019070 con 0x7f23d01562c0 2024-04-20T21:25:28.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.438+0000 7f23c6a00640 1 -- 172.21.15.86:0/3507952774 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c8014810 con 0x7f23d01562c0 2024-04-20T21:25:28.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.438+0000 7f23d5600640 1 -- 172.21.15.86:0/3507952774 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23d01572d0 con 0x7f23d01562c0 2024-04-20T21:25:28.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.439+0000 7f23d5600640 1 -- 172.21.15.86:0/3507952774 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23d0157890 con 0x7f23d01562c0 2024-04-20T21:25:28.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.441+0000 7f23c6a00640 1 -- 172.21.15.86:0/3507952774 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f23c80149b0 con 0x7f23d01562c0 2024-04-20T21:25:28.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.442+0000 7f23c6a00640 1 --2- 172.21.15.86:0/3507952774 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f23b006c920 0x7f23b006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:28.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.442+0000 7f23c6a00640 1 -- 172.21.15.86:0/3507952774 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5795+0+0 (secure 0 0 0) 0x7f23c808c1f0 con 0x7f23d01562c0 2024-04-20T21:25:28.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.442+0000 7f23cea00640 1 --2- 172.21.15.86:0/3507952774 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f23b006c920 0x7f23b006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:28.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.442+0000 7f23d83bf740 1 Processor -- start 2024-04-20T21:25:28.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.442+0000 7f23d83bf740 1 -- start start 2024-04-20T21:25:28.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.443+0000 7f23cea00640 1 --2- 172.21.15.86:0/3507952774 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f23b006c920 0x7f23b006ede0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f23c0001860 tx=0x7f23c0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:28.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.443+0000 7f23d83bf740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4a92ee250 0x55d4a92c11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:28.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.443+0000 7f23d83bf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4a92ee630 0x55d4a92f0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:28.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.443+0000 7f23d83bf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4a92c1730 0x55d4a92f1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:28.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.443+0000 7f23d83bf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d4a922bd50 con 0x55d4a92c1730 2024-04-20T21:25:28.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.443+0000 7f23d83bf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d4a91d22b0 con 0x55d4a92ee630 2024-04-20T21:25:28.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.443+0000 7f23d83bf740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55d4a9227d10 con 0x55d4a92ee250 2024-04-20T21:25:28.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.444+0000 7f23cf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4a92c1730 0x55d4a92f1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:28.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.444+0000 7f23cf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4a92c1730 0x55d4a92f1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36468/0 (socket says 172.21.15.86:36468) 2024-04-20T21:25:28.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.444+0000 7f23cf400640 1 -- 172.21.15.86:0/2833188859 learned_addr learned my addr 172.21.15.86:0/2833188859 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:28.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.444+0000 7f23ce000640 1 --2- 172.21.15.86:0/2833188859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4a92ee630 0x55d4a92f0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:28.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.444+0000 7f23cea00640 1 --2- 172.21.15.86:0/2833188859 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4a92ee250 0x55d4a92c11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:28.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.444+0000 7f23cf400640 1 -- 172.21.15.86:0/2833188859 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4a92ee250 msgr2=0x55d4a92c11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:28.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.444+0000 7f23cf400640 1 --2- 172.21.15.86:0/2833188859 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4a92ee250 0x55d4a92c11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.444+0000 7f23cf400640 1 -- 172.21.15.86:0/2833188859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4a92ee630 msgr2=0x55d4a92f0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:28.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.444+0000 7f23cf400640 1 --2- 172.21.15.86:0/2833188859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4a92ee630 0x55d4a92f0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.444+0000 7f23cf400640 1 -- 172.21.15.86:0/2833188859 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4a91e9230 con 0x55d4a92c1730 2024-04-20T21:25:28.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.444+0000 7f23cf400640 1 --2- 172.21.15.86:0/2833188859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4a92c1730 0x55d4a92f1520 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f23c8002230 tx=0x7f23c8059bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90c79b44fe2990ff server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:28.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.445+0000 7f23bb000640 1 -- 172.21.15.86:0/2833188859 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c805a520 con 0x55d4a92c1730 2024-04-20T21:25:28.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.445+0000 7f23bb000640 1 -- 172.21.15.86:0/2833188859 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23c8059e20 con 0x55d4a92c1730 2024-04-20T21:25:28.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.445+0000 7f23bb000640 1 -- 172.21.15.86:0/2833188859 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c805e1d0 con 0x55d4a92c1730 2024-04-20T21:25:28.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.445+0000 7f23d83bf740 1 -- 172.21.15.86:0/2833188859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4a92c1730 msgr2=0x55d4a92f1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:28.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.445+0000 7f23d83bf740 1 --2- 172.21.15.86:0/2833188859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4a92c1730 0x55d4a92f1520 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f23c8002230 tx=0x7f23c8059bf0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.445+0000 7f23d83bf740 1 -- 172.21.15.86:0/2833188859 shutdown_connections 2024-04-20T21:25:28.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.445+0000 7f23d83bf740 1 --2- 172.21.15.86:0/2833188859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4a92c1730 0x55d4a92f1520 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.445+0000 7f23d83bf740 1 --2- 172.21.15.86:0/2833188859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4a92ee630 0x55d4a92f0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.445+0000 7f23d83bf740 1 --2- 172.21.15.86:0/2833188859 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4a92ee250 0x55d4a92c11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.445+0000 7f23d83bf740 1 -- 172.21.15.86:0/2833188859 >> 172.21.15.86:0/2833188859 conn(0x55d4a92c3460 msgr2=0x55d4a930bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:28.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.446+0000 7f23d83bf740 1 -- 172.21.15.86:0/2833188859 shutdown_connections 2024-04-20T21:25:28.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.446+0000 7f23d83bf740 1 -- 172.21.15.86:0/2833188859 wait complete. 2024-04-20T21:25:28.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.446+0000 7f23d83bf740 1 -- start start 2024-04-20T21:25:28.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.446+0000 7f23d83bf740 1 Processor -- start 2024-04-20T21:25:28.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.447+0000 7f23d83bf740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4a92ee250 0x55d4a92eb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:28.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.447+0000 7f23d83bf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4a92ee630 0x55d4a9304c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:28.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.447+0000 7f23d83bf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4a9303090 0x55d4a93051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:28.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.447+0000 7f23d83bf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d4a922bd50 con 0x55d4a9303090 2024-04-20T21:25:28.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.447+0000 7f23d83bf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55d4a91e55e0 con 0x55d4a92ee630 2024-04-20T21:25:28.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.447+0000 7f23d83bf740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55d4a92c5a00 con 0x55d4a92ee250 2024-04-20T21:25:28.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.448+0000 7f23d83bf740 10 client.0 _ll_register_callbacks cb 0x55d4a91b3fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:25:28.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.448+0000 7f23cf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4a9303090 0x55d4a93051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:28.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.448+0000 7f23cf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4a9303090 0x55d4a93051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36472/0 (socket says 172.21.15.86:36472) 2024-04-20T21:25:28.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.448+0000 7f23cf400640 1 -- 172.21.15.86:0/2336505909 learned_addr learned my addr 172.21.15.86:0/2336505909 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:28.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.448+0000 7f23cea00640 1 --2- 172.21.15.86:0/2336505909 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4a92ee250 0x55d4a92eb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:28.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.448+0000 7f23cf400640 1 -- 172.21.15.86:0/2336505909 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4a92ee250 msgr2=0x55d4a92eb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:28.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.448+0000 7f23cf400640 1 --2- 172.21.15.86:0/2336505909 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4a92ee250 0x55d4a92eb7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.448+0000 7f23cf400640 1 -- 172.21.15.86:0/2336505909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4a92ee630 msgr2=0x55d4a9304c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:28.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.448+0000 7f23ce000640 1 --2- 172.21.15.86:0/2336505909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4a92ee630 0x55d4a9304c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:28.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.448+0000 7f23cf400640 1 --2- 172.21.15.86:0/2336505909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4a92ee630 0x55d4a9304c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.448+0000 7f23cf400640 1 -- 172.21.15.86:0/2336505909 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4a900a7b0 con 0x55d4a9303090 2024-04-20T21:25:28.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.448+0000 7f23cf400640 1 --2- 172.21.15.86:0/2336505909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4a9303090 0x55d4a93051c0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f23c80624d0 tx=0x7f23c801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fa54b291b4013bd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:28.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.448+0000 7f23af400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:25:28.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.449+0000 7f23af400640 1 -- 172.21.15.86:0/2336505909 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c805e1d0 con 0x55d4a9303090 2024-04-20T21:25:28.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.449+0000 7f23af400640 1 -- 172.21.15.86:0/2336505909 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5795+0+0 (secure 0 0 0) 0x7f23c8073d10 con 0x55d4a9303090 2024-04-20T21:25:28.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.449+0000 7f23d83bf740 1 -- 172.21.15.86:0/2336505909 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23c8059e20 con 0x55d4a9303090 2024-04-20T21:25:28.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.449+0000 7f23af400640 1 -- 172.21.15.86:0/2336505909 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f23c8079300 con 0x55d4a9303090 2024-04-20T21:25:28.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.449+0000 7f23d83bf740 10 client.7866 fetch_fsmap learned FSMap version 65 2024-04-20T21:25:28.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.449+0000 7f23d83bf740 1 -- 172.21.15.86:0/2336505909 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) -- 0x55d4a906af40 con 0x55d4a9303090 2024-04-20T21:25:28.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.449+0000 7f23af400640 1 -- 172.21.15.86:0/2336505909 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23c8059e20 con 0x55d4a9303090 2024-04-20T21:25:28.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.449+0000 7f23af400640 1 -- 172.21.15.86:0/2336505909 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 65) ==== 3495+0+0 (secure 0 0 0) 0x7f23c8062570 con 0x55d4a9303090 2024-04-20T21:25:28.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.449+0000 7f23d83bf740 10 client.7866 fetch_fsmap finished waiting for FSMap version 65 2024-04-20T21:25:28.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.449+0000 7f23d83bf740 10 client.7866 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-04-20T21:25:28.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.449+0000 7f23d83bf740 20 client.7866 populate_metadata read hostname 'smithi086' 2024-04-20T21:25:28.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.449+0000 7f23d83bf740 1 --2- 172.21.15.86:0/2336505909 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x55d4a92f2210 0x55d4a92f25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:28.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.449+0000 7f23d83bf740 4 client.7866 mds_command: new command op to 7773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:25:28.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.449+0000 7f23d83bf740 1 -- 172.21.15.86:0/2336505909 --> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4a9239770 con 0x55d4a92f2210 2024-04-20T21:25:28.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.450+0000 7f23cea00640 1 --2- 172.21.15.86:0/2336505909 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x55d4a92f2210 0x55d4a92f25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:28.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.451+0000 7f23cea00640 1 --2- 172.21.15.86:0/2336505909 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x55d4a92f2210 0x55d4a92f25f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9c9ddb7e0d1fa8a server_cookie=f7cff6278d172e3f in_seq=0 out_seq=0 2024-04-20T21:25:28.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.451+0000 7f23af400640 10 client.7866 ms_handle_connect on v2:172.21.15.144:6833/3981403733 2024-04-20T21:25:28.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.453+0000 7f23af400640 1 -- 172.21.15.86:0/2336505909 <== mds.0 v2:172.21.15.144:6833/3981403733 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d4a9239770 con 0x55d4a92f2210 2024-04-20T21:25:28.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.453+0000 7f23af400640 10 client.7866 handle_command_reply: tid=0 2024-04-20T21:25:28.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.453+0000 7f23d83bf740 1 client.7866 shutdown 2024-04-20T21:25:28.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.453+0000 7f23d83bf740 1 -- 172.21.15.86:0/2336505909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4a9303090 msgr2=0x55d4a93051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:28.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.453+0000 7f23d83bf740 1 --2- 172.21.15.86:0/2336505909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4a9303090 0x55d4a93051c0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f23c80624d0 tx=0x7f23c801e000 comp rx=0 tx=0).stop 2024-04-20T21:25:28.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.453+0000 7f23d83bf740 1 -- 172.21.15.86:0/2336505909 shutdown_connections 2024-04-20T21:25:28.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.453+0000 7f23d83bf740 1 --2- 172.21.15.86:0/2336505909 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x55d4a92f2210 0x55d4a92f25f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.453+0000 7f23d83bf740 1 --2- 172.21.15.86:0/2336505909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4a9303090 0x55d4a93051c0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.453+0000 7f23d83bf740 1 --2- 172.21.15.86:0/2336505909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4a92ee630 0x55d4a9304c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.453+0000 7f23d83bf740 1 --2- 172.21.15.86:0/2336505909 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4a92ee250 0x55d4a92eb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.453+0000 7f23af400640 0 client.7866 ms_handle_reset on v2:172.21.15.144:6833/3981403733 2024-04-20T21:25:28.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.453+0000 7f23d83bf740 1 -- 172.21.15.86:0/2336505909 >> 172.21.15.86:0/2336505909 conn(0x55d4a92c1730 msgr2=0x55d4a92f0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:28.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.454+0000 7f23d83bf740 1 -- 172.21.15.86:0/2336505909 shutdown_connections 2024-04-20T21:25:28.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.454+0000 7f23d83bf740 1 -- 172.21.15.86:0/2336505909 wait complete. 2024-04-20T21:25:28.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.454+0000 7f23d83bf740 20 client.7866 trim_cache size 0 max 16384 2024-04-20T21:25:28.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.473+0000 7f23d83bf740 1 Processor -- start 2024-04-20T21:25:28.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.473+0000 7f23d83bf740 1 -- start start 2024-04-20T21:25:28.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.474+0000 7f23d83bf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4a92ee250 0x55d4a92c0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:28.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.474+0000 7f23d83bf740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4a92ee630 0x55d4a92f27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:28.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.474+0000 7f23d83bf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4a9303090 0x55d4a931ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:28.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.474+0000 7f23d83bf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d4a92f5580 con 0x55d4a9303090 2024-04-20T21:25:28.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.474+0000 7f23d83bf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f23d0201700 con 0x55d4a92ee250 2024-04-20T21:25:28.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.474+0000 7f23d83bf740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f23d0200750 con 0x55d4a92ee630 2024-04-20T21:25:28.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.474+0000 7f23cf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4a9303090 0x55d4a931ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:28.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.474+0000 7f23cf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4a9303090 0x55d4a931ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36484/0 (socket says 172.21.15.86:36484) 2024-04-20T21:25:28.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.474+0000 7f23cf400640 1 -- 172.21.15.86:0/195117862 learned_addr learned my addr 172.21.15.86:0/195117862 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:28.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.475+0000 7f23cf400640 1 -- 172.21.15.86:0/195117862 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4a92ee630 msgr2=0x55d4a92f27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:28.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.475+0000 7f23ce000640 1 --2- 172.21.15.86:0/195117862 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4a92ee630 0x55d4a92f27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:28.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.475+0000 7f23cf400640 1 --2- 172.21.15.86:0/195117862 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4a92ee630 0x55d4a92f27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.475+0000 7f23cf400640 1 -- 172.21.15.86:0/195117862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4a92ee250 msgr2=0x55d4a92c0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:28.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.475+0000 7f23cea00640 1 --2- 172.21.15.86:0/195117862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4a92ee250 0x55d4a92c0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:28.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.475+0000 7f23cf400640 1 --2- 172.21.15.86:0/195117862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4a92ee250 0x55d4a92c0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.475+0000 7f23cf400640 1 -- 172.21.15.86:0/195117862 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4a8efe0a0 con 0x55d4a9303090 2024-04-20T21:25:28.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.475+0000 7f23cf400640 1 --2- 172.21.15.86:0/195117862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4a9303090 0x55d4a931ddb0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f23c80567a0 tx=0x7f23c8085070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bba57975eaff4bc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:28.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.475+0000 7f23ba600640 1 -- 172.21.15.86:0/195117862 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c805e1d0 con 0x55d4a9303090 2024-04-20T21:25:28.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.475+0000 7f23ba600640 1 -- 172.21.15.86:0/195117862 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23c8087070 con 0x55d4a9303090 2024-04-20T21:25:28.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.475+0000 7f23ba600640 1 -- 172.21.15.86:0/195117862 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c8059e20 con 0x55d4a9303090 2024-04-20T21:25:28.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.476+0000 7f23d83bf740 1 -- 172.21.15.86:0/195117862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4a9303090 msgr2=0x55d4a931ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:28.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.476+0000 7f23d83bf740 1 --2- 172.21.15.86:0/195117862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4a9303090 0x55d4a931ddb0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f23c80567a0 tx=0x7f23c8085070 comp rx=0 tx=0).stop 2024-04-20T21:25:28.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.476+0000 7f23d83bf740 1 -- 172.21.15.86:0/195117862 shutdown_connections 2024-04-20T21:25:28.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.476+0000 7f23d83bf740 1 --2- 172.21.15.86:0/195117862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4a9303090 0x55d4a931ddb0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.476+0000 7f23d83bf740 1 --2- 172.21.15.86:0/195117862 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4a92ee630 0x55d4a92f27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.476+0000 7f23d83bf740 1 --2- 172.21.15.86:0/195117862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4a92ee250 0x55d4a92c0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.476+0000 7f23d83bf740 1 -- 172.21.15.86:0/195117862 >> 172.21.15.86:0/195117862 conn(0x55d4a92c1730 msgr2=0x55d4a92c70b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:28.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.476+0000 7f23d83bf740 1 -- 172.21.15.86:0/195117862 shutdown_connections 2024-04-20T21:25:28.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.476+0000 7f23d83bf740 1 -- 172.21.15.86:0/195117862 wait complete. 2024-04-20T21:25:28.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.477+0000 7f23d83bf740 1 -- start start 2024-04-20T21:25:28.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.477+0000 7f23d83bf740 1 Processor -- start 2024-04-20T21:25:28.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.478+0000 7f23d83bf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4a92ee250 0x55d4a9388090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:28.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.478+0000 7f23d83bf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4a92ee630 0x55d4a93078e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:28.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.478+0000 7f23d83bf740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4a9303090 0x55d4a9302170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:28.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.478+0000 7f23d83bf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f23d0200100 con 0x55d4a92ee250 2024-04-20T21:25:28.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.478+0000 7f23d83bf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f23d0140500 con 0x55d4a92ee630 2024-04-20T21:25:28.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.478+0000 7f23d83bf740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55d4a9307e20 con 0x55d4a9303090 2024-04-20T21:25:28.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.478+0000 7f23d83bf740 10 client.0 _ll_register_callbacks cb 0x7f23c805a520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:25:28.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.478+0000 7f23cea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4a92ee250 0x55d4a9388090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:28.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.478+0000 7f23cea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4a92ee250 0x55d4a9388090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36490/0 (socket says 172.21.15.86:36490) 2024-04-20T21:25:28.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.478+0000 7f23cea00640 1 -- 172.21.15.86:0/390988343 learned_addr learned my addr 172.21.15.86:0/390988343 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:28.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.479+0000 7f23ce000640 1 --2- 172.21.15.86:0/390988343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4a92ee630 0x55d4a93078e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:28.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.479+0000 7f23cea00640 1 -- 172.21.15.86:0/390988343 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4a9303090 msgr2=0x55d4a9302170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:28.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.479+0000 7f23cea00640 1 --2- 172.21.15.86:0/390988343 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4a9303090 0x55d4a9302170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.479+0000 7f23cea00640 1 -- 172.21.15.86:0/390988343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4a92ee630 msgr2=0x55d4a93078e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:28.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.479+0000 7f23cea00640 1 --2- 172.21.15.86:0/390988343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4a92ee630 0x55d4a93078e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.479+0000 7f23cea00640 1 -- 172.21.15.86:0/390988343 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4a92f3c40 con 0x55d4a92ee250 2024-04-20T21:25:28.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.479+0000 7f23cea00640 1 --2- 172.21.15.86:0/390988343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4a92ee250 0x55d4a9388090 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x55d4a92f3290 tx=0x7f23d015c140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=341c930baf821c93 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:28.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.479+0000 7f23af400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:25:28.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.479+0000 7f23af400640 1 -- 172.21.15.86:0/390988343 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c0019040 con 0x55d4a92ee250 2024-04-20T21:25:28.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.479+0000 7f23af400640 1 -- 172.21.15.86:0/390988343 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5795+0+0 (secure 0 0 0) 0x7f23c0010040 con 0x55d4a92ee250 2024-04-20T21:25:28.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.479+0000 7f23d83bf740 1 -- 172.21.15.86:0/390988343 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23c8087070 con 0x55d4a92ee250 2024-04-20T21:25:28.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.480+0000 7f23af400640 1 -- 172.21.15.86:0/390988343 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f23c0033050 con 0x55d4a92ee250 2024-04-20T21:25:28.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.480+0000 7f23d83bf740 10 client.7872 fetch_fsmap learned FSMap version 65 2024-04-20T21:25:28.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.480+0000 7f23d83bf740 1 -- 172.21.15.86:0/390988343 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) -- 0x55d4a92ecb30 con 0x55d4a92ee250 2024-04-20T21:25:28.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.480+0000 7f23af400640 1 -- 172.21.15.86:0/390988343 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23c8087070 con 0x55d4a92ee250 2024-04-20T21:25:28.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.480+0000 7f23af400640 1 -- 172.21.15.86:0/390988343 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 65) ==== 3495+0+0 (secure 0 0 0) 0x7f23c0036020 con 0x55d4a92ee250 2024-04-20T21:25:28.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.480+0000 7f23d83bf740 10 client.7872 fetch_fsmap finished waiting for FSMap version 65 2024-04-20T21:25:28.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.480+0000 7f23d83bf740 10 client.7872 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-04-20T21:25:28.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.480+0000 7f23d83bf740 20 client.7872 populate_metadata read hostname 'smithi086' 2024-04-20T21:25:28.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.480+0000 7f23d83bf740 1 --2- 172.21.15.86:0/390988343 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x55d4a93026b0 0x55d4a92ebf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:28.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.480+0000 7f23d83bf740 4 client.7872 mds_command: new command op to 7773 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}] 2024-04-20T21:25:28.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.480+0000 7f23d83bf740 1 -- 172.21.15.86:0/390988343 --> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}) -- 0x55d4a92335e0 con 0x55d4a93026b0 2024-04-20T21:25:28.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.481+0000 7f23ce000640 1 --2- 172.21.15.86:0/390988343 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x55d4a93026b0 0x55d4a92ebf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:28.483 INFO:tasks.ceph.mds.b.smithi144.stderr:2024-04-20T21:25:28.483+0000 7f7416200640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 ap=1 f(v0 m2024-04-20T21:25:07.713157+0000 1=1+0) n(v0 rc2024-04-20T21:25:07.851790+0000 2=1+1) | dirfrag=0 authpin=1 scrubqueue=0 0x561f42e0a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-20T21:25:07.713157+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-20T21:25:07.851790+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-04-20T21:25:28.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.482+0000 7f23ce000640 1 --2- 172.21.15.86:0/390988343 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x55d4a93026b0 0x55d4a92ebf70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93c84e32b7ed16e0 server_cookie=9e187ab8b9e29230 in_seq=0 out_seq=0 2024-04-20T21:25:28.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.482+0000 7f23af400640 10 client.7872 ms_handle_connect on v2:172.21.15.144:6833/3981403733 2024-04-20T21:25:28.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.483+0000 7f23af400640 1 -- 172.21.15.86:0/390988343 <== mds.0 v2:172.21.15.144:6833/3981403733 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d4a92335e0 con 0x55d4a93026b0 2024-04-20T21:25:28.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.483+0000 7f23af400640 10 client.7872 handle_command_reply: tid=0 2024-04-20T21:25:28.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.483+0000 7f23d83bf740 1 client.7872 shutdown 2024-04-20T21:25:28.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.483+0000 7f23d83bf740 1 -- 172.21.15.86:0/390988343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4a92ee250 msgr2=0x55d4a9388090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:28.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.483+0000 7f23d83bf740 1 --2- 172.21.15.86:0/390988343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4a92ee250 0x55d4a9388090 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x55d4a92f3290 tx=0x7f23d015c140 comp rx=0 tx=0).stop 2024-04-20T21:25:28.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.483+0000 7f23d83bf740 1 -- 172.21.15.86:0/390988343 shutdown_connections 2024-04-20T21:25:28.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.483+0000 7f23d83bf740 1 --2- 172.21.15.86:0/390988343 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x55d4a93026b0 0x55d4a92ebf70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.483+0000 7f23d83bf740 1 --2- 172.21.15.86:0/390988343 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55d4a9303090 0x55d4a9302170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.483+0000 7f23d83bf740 1 --2- 172.21.15.86:0/390988343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55d4a92ee630 0x55d4a93078e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.483+0000 7f23d83bf740 1 --2- 172.21.15.86:0/390988343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4a92ee250 0x55d4a9388090 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.483+0000 7f23af400640 0 client.7872 ms_handle_reset on v2:172.21.15.144:6833/3981403733 2024-04-20T21:25:28.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.483+0000 7f23d83bf740 1 -- 172.21.15.86:0/390988343 >> 172.21.15.86:0/390988343 conn(0x55d4a92c1730 msgr2=0x55d4a92f2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:28.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.483+0000 7f23d83bf740 1 -- 172.21.15.86:0/390988343 shutdown_connections 2024-04-20T21:25:28.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.483+0000 7f23d83bf740 1 -- 172.21.15.86:0/390988343 wait complete. 2024-04-20T21:25:28.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.483+0000 7f23d83bf740 20 client.7872 trim_cache size 0 max 16384 2024-04-20T21:25:28.486 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:25:28.486 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-20T21:25:28.486 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "5f0633da-6271-47ec-a18c-7e6a48250ab0", 2024-04-20T21:25:28.486 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-20T21:25:28.486 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:25:28.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.484+0000 7f23d5600640 1 -- 172.21.15.86:0/3507952774 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f23b006c920 msgr2=0x7f23b006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:28.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.484+0000 7f23d5600640 1 --2- 172.21.15.86:0/3507952774 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f23b006c920 0x7f23b006ede0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f23c0001860 tx=0x7f23c0009040 comp rx=0 tx=0).stop 2024-04-20T21:25:28.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.484+0000 7f23d5600640 1 -- 172.21.15.86:0/3507952774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23d01562c0 msgr2=0x7f23d0156770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:28.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.484+0000 7f23d5600640 1 --2- 172.21.15.86:0/3507952774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23d01562c0 0x7f23d0156770 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f23d01560d0 tx=0x7f23c8007700 comp rx=0 tx=0).stop 2024-04-20T21:25:28.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.484+0000 7f23d5600640 1 -- 172.21.15.86:0/3507952774 shutdown_connections 2024-04-20T21:25:28.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.484+0000 7f23d5600640 1 --2- 172.21.15.86:0/3507952774 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f23b006c920 0x7f23b006ede0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.484+0000 7f23d5600640 1 --2- 172.21.15.86:0/3507952774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23d01562c0 0x7f23d0156770 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.484+0000 7f23d5600640 1 --2- 172.21.15.86:0/3507952774 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23d0154aa0 0x7f23d0154f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.484+0000 7f23d5600640 1 --2- 172.21.15.86:0/3507952774 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f23b4004800 0x7f23d0142180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.484+0000 7f23d5600640 1 -- 172.21.15.86:0/3507952774 >> 172.21.15.86:0/3507952774 conn(0x7f23d00b2180 msgr2=0x7f23d0071d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:28.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.484+0000 7f23d5600640 1 -- 172.21.15.86:0/3507952774 shutdown_connections 2024-04-20T21:25:28.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.484+0000 7f23d5600640 1 -- 172.21.15.86:0/3507952774 wait complete. 2024-04-20T21:25:28.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:25:28.502 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-20T21:25:28.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.691+0000 7f442e400640 1 Processor -- start 2024-04-20T21:25:28.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.691+0000 7f442e400640 1 -- start start 2024-04-20T21:25:28.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.692+0000 7f442e400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f442806bbf0 0x7f442806e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:28.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.693+0000 7f442e400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f442806bfd0 0x7f4428145b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:28.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.693+0000 7f442e400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4428146040 0x7f44280cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:28.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.693+0000 7f442e400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f442815cd40 con 0x7f442806bfd0 2024-04-20T21:25:28.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.693+0000 7f442e400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f442815d000 con 0x7f4428146040 2024-04-20T21:25:28.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.693+0000 7f442e400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f442815d2e0 con 0x7f442806bbf0 2024-04-20T21:25:28.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.693+0000 7f4426a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f442806bfd0 0x7f4428145b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:28.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.693+0000 7f4426a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f442806bfd0 0x7f4428145b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36492/0 (socket says 172.21.15.86:36492) 2024-04-20T21:25:28.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.693+0000 7f4426a00640 1 -- 172.21.15.86:0/1422627345 learned_addr learned my addr 172.21.15.86:0/1422627345 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:28.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.693+0000 7f4427400640 1 --2- 172.21.15.86:0/1422627345 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f442806bbf0 0x7f442806e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:28.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.693+0000 7f4426a00640 1 -- 172.21.15.86:0/1422627345 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f442806bbf0 msgr2=0x7f442806e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:28.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.693+0000 7f4427e00640 1 --2- 172.21.15.86:0/1422627345 >> v2:172.21.15.144:3300/0 conn(0x7f4428146040 0x7f44280cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:28.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.694+0000 7f4426a00640 1 --2- 172.21.15.86:0/1422627345 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f442806bbf0 0x7f442806e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.694+0000 7f4426a00640 1 -- 172.21.15.86:0/1422627345 >> v2:172.21.15.144:3300/0 conn(0x7f4428146040 msgr2=0x7f44280cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:28.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.694+0000 7f4426a00640 1 --2- 172.21.15.86:0/1422627345 >> v2:172.21.15.144:3300/0 conn(0x7f4428146040 0x7f44280cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.694+0000 7f4426a00640 1 -- 172.21.15.86:0/1422627345 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f442815d5c0 con 0x7f442806bfd0 2024-04-20T21:25:28.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.694+0000 7f4427e00640 1 --2- 172.21.15.86:0/1422627345 >> v2:172.21.15.144:3300/0 conn(0x7f4428146040 0x7f44280cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:25:28.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.695+0000 7f4426a00640 1 --2- 172.21.15.86:0/1422627345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f442806bfd0 0x7f4428145b00 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f44280cf110 tx=0x7f441402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ca0be03c7bb6c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:28.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.695+0000 7f4426000640 1 -- 172.21.15.86:0/1422627345 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4414035020 con 0x7f442806bfd0 2024-04-20T21:25:28.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.695+0000 7f4426000640 1 -- 172.21.15.86:0/1422627345 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4414031870 con 0x7f442806bfd0 2024-04-20T21:25:28.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.695+0000 7f4426000640 1 -- 172.21.15.86:0/1422627345 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4414031c40 con 0x7f442806bfd0 2024-04-20T21:25:28.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.696+0000 7f442e400640 1 -- 172.21.15.86:0/1422627345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f442806bfd0 msgr2=0x7f4428145b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:28.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.696+0000 7f442e400640 1 --2- 172.21.15.86:0/1422627345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f442806bfd0 0x7f4428145b00 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f44280cf110 tx=0x7f441402b910 comp rx=0 tx=0).stop 2024-04-20T21:25:28.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.696+0000 7f442e400640 1 -- 172.21.15.86:0/1422627345 shutdown_connections 2024-04-20T21:25:28.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.696+0000 7f442e400640 1 --2- 172.21.15.86:0/1422627345 >> v2:172.21.15.144:3300/0 conn(0x7f4428146040 0x7f44280cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.696+0000 7f442e400640 1 --2- 172.21.15.86:0/1422627345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f442806bfd0 0x7f4428145b00 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.696+0000 7f442e400640 1 --2- 172.21.15.86:0/1422627345 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f442806bbf0 0x7f442806e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.696+0000 7f442e400640 1 -- 172.21.15.86:0/1422627345 >> 172.21.15.86:0/1422627345 conn(0x7f44280b2110 msgr2=0x7f44280b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:28.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.696+0000 7f442e400640 1 -- 172.21.15.86:0/1422627345 shutdown_connections 2024-04-20T21:25:28.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.696+0000 7f442e400640 1 -- 172.21.15.86:0/1422627345 wait complete. 2024-04-20T21:25:28.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.698+0000 7f442e400640 1 Processor -- start 2024-04-20T21:25:28.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.699+0000 7f442e400640 1 -- start start 2024-04-20T21:25:28.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.700+0000 7f442e400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f442806bbf0 0x7f44281347d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:28.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.700+0000 7f442e400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f442806bfd0 0x7f44280b4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:28.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.700+0000 7f442e400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4428146040 0x7f44281e6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:28.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.700+0000 7f442e400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f44280cdc00 con 0x7f442806bfd0 2024-04-20T21:25:28.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.700+0000 7f442e400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f44280cda80 con 0x7f4428146040 2024-04-20T21:25:28.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.700+0000 7f442e400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f44280cdd80 con 0x7f442806bbf0 2024-04-20T21:25:28.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.700+0000 7f4426a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f442806bfd0 0x7f44280b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:28.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.700+0000 7f4426a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f442806bfd0 0x7f44280b4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36506/0 (socket says 172.21.15.86:36506) 2024-04-20T21:25:28.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.700+0000 7f4426a00640 1 -- 172.21.15.86:0/2155896740 learned_addr learned my addr 172.21.15.86:0/2155896740 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:28.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.701+0000 7f4427400640 1 --2- 172.21.15.86:0/2155896740 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f442806bbf0 0x7f44281347d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:28.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.701+0000 7f4426a00640 1 -- 172.21.15.86:0/2155896740 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f442806bbf0 msgr2=0x7f44281347d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:28.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.701+0000 7f4426a00640 1 --2- 172.21.15.86:0/2155896740 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f442806bbf0 0x7f44281347d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.701+0000 7f4427e00640 1 --2- 172.21.15.86:0/2155896740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4428146040 0x7f44281e6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:28.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.701+0000 7f4426a00640 1 -- 172.21.15.86:0/2155896740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4428146040 msgr2=0x7f44281e6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:28.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.701+0000 7f4426a00640 1 --2- 172.21.15.86:0/2155896740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4428146040 0x7f44281e6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.701+0000 7f4426a00640 1 -- 172.21.15.86:0/2155896740 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4428134d10 con 0x7f442806bfd0 2024-04-20T21:25:28.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.701+0000 7f4426a00640 1 --2- 172.21.15.86:0/2155896740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f442806bfd0 0x7f44280b4440 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f441402b950 tx=0x7f44140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:28.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.701+0000 7f441f400640 1 -- 172.21.15.86:0/2155896740 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4414040070 con 0x7f442806bfd0 2024-04-20T21:25:28.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.701+0000 7f441f400640 1 -- 172.21.15.86:0/2155896740 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4414030d00 con 0x7f442806bfd0 2024-04-20T21:25:28.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.701+0000 7f441f400640 1 -- 172.21.15.86:0/2155896740 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4414035040 con 0x7f442806bfd0 2024-04-20T21:25:28.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.701+0000 7f442e400640 1 -- 172.21.15.86:0/2155896740 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44281e6c70 con 0x7f442806bfd0 2024-04-20T21:25:28.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.702+0000 7f442e400640 1 -- 172.21.15.86:0/2155896740 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44281e7230 con 0x7f442806bfd0 2024-04-20T21:25:28.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.704+0000 7f441f400640 1 -- 172.21.15.86:0/2155896740 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f441404b060 con 0x7f442806bfd0 2024-04-20T21:25:28.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.705+0000 7f441f400640 1 --2- 172.21.15.86:0/2155896740 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f440806c8f0 0x7f440806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:28.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.705+0000 7f441f400640 1 -- 172.21.15.86:0/2155896740 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5795+0+0 (secure 0 0 0) 0x7f44140ad9a0 con 0x7f442806bfd0 2024-04-20T21:25:28.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.705+0000 7f4427400640 1 --2- 172.21.15.86:0/2155896740 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f440806c8f0 0x7f440806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:28.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.705+0000 7f44311e7740 1 Processor -- start 2024-04-20T21:25:28.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.706+0000 7f44311e7740 1 -- start start 2024-04-20T21:25:28.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.706+0000 7f4427400640 1 --2- 172.21.15.86:0/2155896740 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f440806c8f0 0x7f440806edb0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f4418006fd0 tx=0x7f4418008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:28.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.706+0000 7f44311e7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55608bae31f0 0x55608bae35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:28.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.706+0000 7f44311e7740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55608bab3210 0x55608bae0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:28.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.706+0000 7f44311e7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55608bab6d70 0x55608bae09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:28.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.706+0000 7f44311e7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55608ba1f150 con 0x55608bab6d70 2024-04-20T21:25:28.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.706+0000 7f44311e7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55608b9c5370 con 0x55608bae31f0 2024-04-20T21:25:28.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.706+0000 7f44311e7740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55608ba1b110 con 0x55608bab3210 2024-04-20T21:25:28.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.707+0000 7f4427e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55608bab6d70 0x55608bae09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:28.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.707+0000 7f4427e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55608bab6d70 0x55608bae09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36516/0 (socket says 172.21.15.86:36516) 2024-04-20T21:25:28.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.707+0000 7f4427e00640 1 -- 172.21.15.86:0/4268235389 learned_addr learned my addr 172.21.15.86:0/4268235389 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:28.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.707+0000 7f4427e00640 1 -- 172.21.15.86:0/4268235389 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55608bab3210 msgr2=0x55608bae0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:28.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.707+0000 7f4427e00640 1 --2- 172.21.15.86:0/4268235389 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55608bab3210 0x55608bae0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.707+0000 7f4427e00640 1 -- 172.21.15.86:0/4268235389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55608bae31f0 msgr2=0x55608bae35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:28.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.707+0000 7f4427400640 1 --2- 172.21.15.86:0/4268235389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55608bae31f0 0x55608bae35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:28.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.707+0000 7f4427e00640 1 --2- 172.21.15.86:0/4268235389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55608bae31f0 0x55608bae35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.707+0000 7f4427e00640 1 -- 172.21.15.86:0/4268235389 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55608b7fb7d0 con 0x55608bab6d70 2024-04-20T21:25:28.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.708+0000 7f4427e00640 1 --2- 172.21.15.86:0/4268235389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55608bab6d70 0x55608bae09a0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f4420008f60 tx=0x7f4420008f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=408cee02c68b28a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:28.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.708+0000 7f4413e00640 1 -- 172.21.15.86:0/4268235389 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442000f860 con 0x55608bab6d70 2024-04-20T21:25:28.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.708+0000 7f4413e00640 1 -- 172.21.15.86:0/4268235389 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f442000fa00 con 0x55608bab6d70 2024-04-20T21:25:28.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.708+0000 7f4413e00640 1 -- 172.21.15.86:0/4268235389 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442000fdd0 con 0x55608bab6d70 2024-04-20T21:25:28.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.709+0000 7f44311e7740 1 -- 172.21.15.86:0/4268235389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55608bab6d70 msgr2=0x55608bae09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:28.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.709+0000 7f44311e7740 1 --2- 172.21.15.86:0/4268235389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55608bab6d70 0x55608bae09a0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f4420008f60 tx=0x7f4420008f90 comp rx=0 tx=0).stop 2024-04-20T21:25:28.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.709+0000 7f44311e7740 1 -- 172.21.15.86:0/4268235389 shutdown_connections 2024-04-20T21:25:28.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.709+0000 7f44311e7740 1 --2- 172.21.15.86:0/4268235389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55608bab6d70 0x55608bae09a0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.709+0000 7f44311e7740 1 --2- 172.21.15.86:0/4268235389 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55608bab3210 0x55608bae0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.709+0000 7f44311e7740 1 --2- 172.21.15.86:0/4268235389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55608bae31f0 0x55608bae35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.709+0000 7f44311e7740 1 -- 172.21.15.86:0/4268235389 >> 172.21.15.86:0/4268235389 conn(0x55608bab5480 msgr2=0x55608bafe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:28.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.709+0000 7f44311e7740 1 -- 172.21.15.86:0/4268235389 shutdown_connections 2024-04-20T21:25:28.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.709+0000 7f44311e7740 1 -- 172.21.15.86:0/4268235389 wait complete. 2024-04-20T21:25:28.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.710+0000 7f44311e7740 1 -- start start 2024-04-20T21:25:28.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.710+0000 7f44311e7740 1 Processor -- start 2024-04-20T21:25:28.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.711+0000 7f44311e7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55608bab6d70 0x55608baf4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:28.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.711+0000 7f44311e7740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55608bae31f0 0x55608baf6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:28.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.711+0000 7f44311e7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55608badfa00 0x55608bb0ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:28.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.711+0000 7f44311e7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55608bab7f20 con 0x55608bab6d70 2024-04-20T21:25:28.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.711+0000 7f44311e7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55608b9d8740 con 0x55608badfa00 2024-04-20T21:25:28.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.711+0000 7f44311e7740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55608bab7310 con 0x55608bae31f0 2024-04-20T21:25:28.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.711+0000 7f44311e7740 10 client.0 _ll_register_callbacks cb 0x55608b9d3b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:25:28.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.711+0000 7f4427400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55608bab6d70 0x55608baf4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:28.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.711+0000 7f4427400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55608bab6d70 0x55608baf4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36522/0 (socket says 172.21.15.86:36522) 2024-04-20T21:25:28.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.711+0000 7f4427400640 1 -- 172.21.15.86:0/520387082 learned_addr learned my addr 172.21.15.86:0/520387082 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:28.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.711+0000 7f4427400640 1 -- 172.21.15.86:0/520387082 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55608bae31f0 msgr2=0x55608baf6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:28.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.711+0000 7f4427400640 1 --2- 172.21.15.86:0/520387082 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55608bae31f0 0x55608baf6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.711+0000 7f4427400640 1 -- 172.21.15.86:0/520387082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55608badfa00 msgr2=0x55608bb0ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:28.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.711+0000 7f4427400640 1 --2- 172.21.15.86:0/520387082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55608badfa00 0x55608bb0ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.711+0000 7f4427400640 1 -- 172.21.15.86:0/520387082 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55608b7313f0 con 0x55608bab6d70 2024-04-20T21:25:28.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.712+0000 7f4427400640 1 --2- 172.21.15.86:0/520387082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55608bab6d70 0x55608baf4a90 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f4418015cd0 tx=0x7f4418004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c6862c540b91363 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:28.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.712+0000 7f4410c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:25:28.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.712+0000 7f4410c00640 1 -- 172.21.15.86:0/520387082 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441800f040 con 0x55608bab6d70 2024-04-20T21:25:28.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.712+0000 7f4410c00640 1 -- 172.21.15.86:0/520387082 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5795+0+0 (secure 0 0 0) 0x7f441800ee00 con 0x55608bab6d70 2024-04-20T21:25:28.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.712+0000 7f44311e7740 1 -- 172.21.15.86:0/520387082 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f442000fa00 con 0x55608bab6d70 2024-04-20T21:25:28.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.712+0000 7f4410c00640 1 -- 172.21.15.86:0/520387082 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f441802d050 con 0x55608bab6d70 2024-04-20T21:25:28.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.712+0000 7f44311e7740 10 client.7884 fetch_fsmap learned FSMap version 65 2024-04-20T21:25:28.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.712+0000 7f44311e7740 1 -- 172.21.15.86:0/520387082 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) -- 0x55608b6f10f0 con 0x55608bab6d70 2024-04-20T21:25:28.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.712+0000 7f4410c00640 1 -- 172.21.15.86:0/520387082 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f442000fa00 con 0x55608bab6d70 2024-04-20T21:25:28.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.713+0000 7f4410c00640 1 -- 172.21.15.86:0/520387082 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 65) ==== 3495+0+0 (secure 0 0 0) 0x7f4418030020 con 0x55608bab6d70 2024-04-20T21:25:28.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.713+0000 7f44311e7740 10 client.7884 fetch_fsmap finished waiting for FSMap version 65 2024-04-20T21:25:28.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.713+0000 7f44311e7740 10 client.7884 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-04-20T21:25:28.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.713+0000 7f44311e7740 20 client.7884 populate_metadata read hostname 'smithi086' 2024-04-20T21:25:28.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.713+0000 7f44311e7740 1 --2- 172.21.15.86:0/520387082 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x55608baf7210 0x55608baf7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:28.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.713+0000 7f44311e7740 4 client.7884 mds_command: new command op to 7773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:25:28.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.713+0000 7f44311e7740 1 -- 172.21.15.86:0/520387082 --> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55608ba2cb70 con 0x55608baf7210 2024-04-20T21:25:28.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.713+0000 7f4426a00640 1 --2- 172.21.15.86:0/520387082 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x55608baf7210 0x55608baf7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:28.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.714+0000 7f4426a00640 1 --2- 172.21.15.86:0/520387082 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x55608baf7210 0x55608baf7670 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b737c7ca816615b server_cookie=596b68504f6ebb17 in_seq=0 out_seq=0 2024-04-20T21:25:28.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.714+0000 7f4410c00640 10 client.7884 ms_handle_connect on v2:172.21.15.144:6833/3981403733 2024-04-20T21:25:28.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.715+0000 7f4410c00640 1 -- 172.21.15.86:0/520387082 <== mds.0 v2:172.21.15.144:6833/3981403733 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f4414093e50 con 0x55608baf7210 2024-04-20T21:25:28.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.715+0000 7f4410c00640 10 client.7884 handle_command_reply: tid=0 2024-04-20T21:25:28.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.716+0000 7f44311e7740 1 client.7884 shutdown 2024-04-20T21:25:28.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.716+0000 7f44311e7740 1 -- 172.21.15.86:0/520387082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55608bab6d70 msgr2=0x55608baf4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:28.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.716+0000 7f44311e7740 1 --2- 172.21.15.86:0/520387082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55608bab6d70 0x55608baf4a90 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f4418015cd0 tx=0x7f4418004730 comp rx=0 tx=0).stop 2024-04-20T21:25:28.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.716+0000 7f44311e7740 1 -- 172.21.15.86:0/520387082 shutdown_connections 2024-04-20T21:25:28.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.716+0000 7f44311e7740 1 --2- 172.21.15.86:0/520387082 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x55608baf7210 0x55608baf7670 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.716+0000 7f44311e7740 1 --2- 172.21.15.86:0/520387082 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55608badfa00 0x55608bb0ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.716+0000 7f44311e7740 1 --2- 172.21.15.86:0/520387082 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55608bae31f0 0x55608baf6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.716+0000 7f44311e7740 1 --2- 172.21.15.86:0/520387082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55608bab6d70 0x55608baf4a90 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.716+0000 7f44311e7740 1 -- 172.21.15.86:0/520387082 >> 172.21.15.86:0/520387082 conn(0x55608bab3210 msgr2=0x55608bab9240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:28.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.716+0000 7f4410c00640 0 client.7884 ms_handle_reset on v2:172.21.15.144:6833/3981403733 2024-04-20T21:25:28.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.716+0000 7f44311e7740 1 -- 172.21.15.86:0/520387082 shutdown_connections 2024-04-20T21:25:28.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.716+0000 7f44311e7740 1 -- 172.21.15.86:0/520387082 wait complete. 2024-04-20T21:25:28.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.716+0000 7f44311e7740 20 client.7884 trim_cache size 0 max 16384 2024-04-20T21:25:28.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.735+0000 7f44311e7740 1 Processor -- start 2024-04-20T21:25:28.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.736+0000 7f44311e7740 1 -- start start 2024-04-20T21:25:28.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.736+0000 7f44311e7740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55608bab6d70 0x55608badef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:28.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.736+0000 7f44311e7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55608badfa00 0x55608bae0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:28.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.736+0000 7f44311e7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55608bae31f0 0x55608bae0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:28.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.736+0000 7f44311e7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4428200fa0 con 0x55608bae31f0 2024-04-20T21:25:28.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.736+0000 7f44311e7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f44281e5ba0 con 0x55608badfa00 2024-04-20T21:25:28.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.736+0000 7f44311e7740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f44281e4f00 con 0x55608bab6d70 2024-04-20T21:25:28.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.737+0000 7f4427e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55608bae31f0 0x55608bae0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:28.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.737+0000 7f4427e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55608bae31f0 0x55608bae0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36526/0 (socket says 172.21.15.86:36526) 2024-04-20T21:25:28.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.737+0000 7f4427e00640 1 -- 172.21.15.86:0/2587051987 learned_addr learned my addr 172.21.15.86:0/2587051987 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:28.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.737+0000 7f4426a00640 1 --2- 172.21.15.86:0/2587051987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55608badfa00 0x55608bae0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:28.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.737+0000 7f4427400640 1 --2- 172.21.15.86:0/2587051987 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55608bab6d70 0x55608badef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:28.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.737+0000 7f4427e00640 1 -- 172.21.15.86:0/2587051987 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55608bab6d70 msgr2=0x55608badef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:28.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.737+0000 7f4427e00640 1 --2- 172.21.15.86:0/2587051987 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55608bab6d70 0x55608badef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.737+0000 7f4427e00640 1 -- 172.21.15.86:0/2587051987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55608badfa00 msgr2=0x55608bae0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:28.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.737+0000 7f4427e00640 1 --2- 172.21.15.86:0/2587051987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55608badfa00 0x55608bae0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.737+0000 7f4427e00640 1 -- 172.21.15.86:0/2587051987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55608bae6bc0 con 0x55608bae31f0 2024-04-20T21:25:28.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.737+0000 7f4426a00640 1 --2- 172.21.15.86:0/2587051987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55608badfa00 0x55608bae0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:25:28.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.738+0000 7f4427e00640 1 --2- 172.21.15.86:0/2587051987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55608bae31f0 0x55608bae0200 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f442000dc90 tx=0x7f4420010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f31f0f172bc9224 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:28.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.738+0000 7f4413400640 1 -- 172.21.15.86:0/2587051987 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442000f860 con 0x55608bae31f0 2024-04-20T21:25:28.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.738+0000 7f4413400640 1 -- 172.21.15.86:0/2587051987 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4420027070 con 0x55608bae31f0 2024-04-20T21:25:28.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.738+0000 7f4413400640 1 -- 172.21.15.86:0/2587051987 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442000fc60 con 0x55608bae31f0 2024-04-20T21:25:28.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.738+0000 7f44311e7740 1 -- 172.21.15.86:0/2587051987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55608bae31f0 msgr2=0x55608bae0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:28.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.738+0000 7f44311e7740 1 --2- 172.21.15.86:0/2587051987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55608bae31f0 0x55608bae0200 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f442000dc90 tx=0x7f4420010000 comp rx=0 tx=0).stop 2024-04-20T21:25:28.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.739+0000 7f44311e7740 1 -- 172.21.15.86:0/2587051987 shutdown_connections 2024-04-20T21:25:28.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.739+0000 7f44311e7740 1 --2- 172.21.15.86:0/2587051987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55608bae31f0 0x55608bae0200 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.739+0000 7f44311e7740 1 --2- 172.21.15.86:0/2587051987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55608badfa00 0x55608bae0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.739+0000 7f44311e7740 1 --2- 172.21.15.86:0/2587051987 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55608bab6d70 0x55608badef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.739+0000 7f44311e7740 1 -- 172.21.15.86:0/2587051987 >> 172.21.15.86:0/2587051987 conn(0x55608bab3210 msgr2=0x55608baae750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:28.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.739+0000 7f44311e7740 1 -- 172.21.15.86:0/2587051987 shutdown_connections 2024-04-20T21:25:28.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.739+0000 7f44311e7740 1 -- 172.21.15.86:0/2587051987 wait complete. 2024-04-20T21:25:28.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.739+0000 7f44311e7740 1 -- start start 2024-04-20T21:25:28.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.740+0000 7f44311e7740 1 Processor -- start 2024-04-20T21:25:28.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.741+0000 7f44311e7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55608bab6d70 0x55608bae0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:28.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.741+0000 7f44311e7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55608badfa00 0x55608bae14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:28.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.741+0000 7f44311e7740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55608bae31f0 0x55608bade4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:28.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.741+0000 7f44311e7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55608bab4c50 con 0x55608bab6d70 2024-04-20T21:25:28.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.741+0000 7f44311e7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f44281e5550 con 0x55608badfa00 2024-04-20T21:25:28.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.741+0000 7f44311e7740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55608b9a0ac0 con 0x55608bae31f0 2024-04-20T21:25:28.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.741+0000 7f44311e7740 10 client.0 _ll_register_callbacks cb 0x7f4418009400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:25:28.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.741+0000 7f4427400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55608bab6d70 0x55608bae0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:28.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.741+0000 7f4427400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55608bab6d70 0x55608bae0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36536/0 (socket says 172.21.15.86:36536) 2024-04-20T21:25:28.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.741+0000 7f4427400640 1 -- 172.21.15.86:0/633966188 learned_addr learned my addr 172.21.15.86:0/633966188 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:28.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.741+0000 7f4427400640 1 -- 172.21.15.86:0/633966188 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55608bae31f0 msgr2=0x55608bade4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:28.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.741+0000 7f4427400640 1 --2- 172.21.15.86:0/633966188 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55608bae31f0 0x55608bade4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.741+0000 7f4427400640 1 -- 172.21.15.86:0/633966188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55608badfa00 msgr2=0x55608bae14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:28.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.741+0000 7f4427400640 1 --2- 172.21.15.86:0/633966188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55608badfa00 0x55608bae14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.741+0000 7f4427400640 1 -- 172.21.15.86:0/633966188 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55608bb81260 con 0x55608bab6d70 2024-04-20T21:25:28.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.741+0000 7f4427400640 1 --2- 172.21.15.86:0/633966188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55608bab6d70 0x55608bae0f70 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f4418015cd0 tx=0x7f4418013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1014b2a73d676b6b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:28.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.742+0000 7f4410c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:25:28.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.742+0000 7f4410c00640 1 -- 172.21.15.86:0/633966188 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441803c680 con 0x55608bab6d70 2024-04-20T21:25:28.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.742+0000 7f4410c00640 1 -- 172.21.15.86:0/633966188 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5795+0+0 (secure 0 0 0) 0x7f441803c820 con 0x55608bab6d70 2024-04-20T21:25:28.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.742+0000 7f44311e7740 1 -- 172.21.15.86:0/633966188 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4420027070 con 0x55608bab6d70 2024-04-20T21:25:28.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.742+0000 7f4410c00640 1 -- 172.21.15.86:0/633966188 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f4418044050 con 0x55608bab6d70 2024-04-20T21:25:28.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.742+0000 7f44311e7740 10 client.7890 fetch_fsmap learned FSMap version 65 2024-04-20T21:25:28.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.742+0000 7f44311e7740 1 -- 172.21.15.86:0/633966188 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) -- 0x55608bb82130 con 0x55608bab6d70 2024-04-20T21:25:28.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.742+0000 7f4410c00640 1 -- 172.21.15.86:0/633966188 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4420027070 con 0x55608bab6d70 2024-04-20T21:25:28.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.742+0000 7f4410c00640 1 -- 172.21.15.86:0/633966188 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 65) ==== 3495+0+0 (secure 0 0 0) 0x7f4418030040 con 0x55608bab6d70 2024-04-20T21:25:28.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.742+0000 7f44311e7740 10 client.7890 fetch_fsmap finished waiting for FSMap version 65 2024-04-20T21:25:28.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.742+0000 7f44311e7740 10 client.7890 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-04-20T21:25:28.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.742+0000 7f44311e7740 20 client.7890 populate_metadata read hostname 'smithi086' 2024-04-20T21:25:28.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.742+0000 7f44311e7740 1 --2- 172.21.15.86:0/633966188 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x55608bad2bf0 0x55608badea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:28.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.742+0000 7f44311e7740 4 client.7890 mds_command: new command op to 7773 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:25:28.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.742+0000 7f44311e7740 1 -- 172.21.15.86:0/633966188 --> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55608bab3a10 con 0x55608bad2bf0 2024-04-20T21:25:28.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.743+0000 7f4426a00640 1 --2- 172.21.15.86:0/633966188 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x55608bad2bf0 0x55608badea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:28.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.744+0000 7f4410c00640 10 client.7890 ms_handle_connect on v2:172.21.15.144:6833/3981403733 2024-04-20T21:25:28.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.744+0000 7f4426a00640 1 --2- 172.21.15.86:0/633966188 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x55608bad2bf0 0x55608badea10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b144f87b891f2be server_cookie=fcad3a742f00587f in_seq=0 out_seq=0 2024-04-20T21:25:28.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.745+0000 7f4410c00640 1 -- 172.21.15.86:0/633966188 <== mds.0 v2:172.21.15.144:6833/3981403733 1 ==== command_reply(tid 0: 0 ) ==== 8+0+256 (crc 0 0 0) 0x55608bab3a10 con 0x55608bad2bf0 2024-04-20T21:25:28.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.745+0000 7f4410c00640 10 client.7890 handle_command_reply: tid=0 2024-04-20T21:25:28.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.745+0000 7f44311e7740 1 client.7890 shutdown 2024-04-20T21:25:28.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.745+0000 7f44311e7740 1 -- 172.21.15.86:0/633966188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55608bab6d70 msgr2=0x55608bae0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:28.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.745+0000 7f44311e7740 1 --2- 172.21.15.86:0/633966188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55608bab6d70 0x55608bae0f70 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f4418015cd0 tx=0x7f4418013000 comp rx=0 tx=0).stop 2024-04-20T21:25:28.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.745+0000 7f44311e7740 1 -- 172.21.15.86:0/633966188 shutdown_connections 2024-04-20T21:25:28.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.745+0000 7f44311e7740 1 --2- 172.21.15.86:0/633966188 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x55608bad2bf0 0x55608badea10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.745+0000 7f44311e7740 1 --2- 172.21.15.86:0/633966188 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55608bae31f0 0x55608bade4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.745+0000 7f44311e7740 1 --2- 172.21.15.86:0/633966188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55608badfa00 0x55608bae14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.745+0000 7f4410c00640 0 client.7890 ms_handle_reset on v2:172.21.15.144:6833/3981403733 2024-04-20T21:25:28.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.745+0000 7f44311e7740 1 --2- 172.21.15.86:0/633966188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55608bab6d70 0x55608bae0f70 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.745+0000 7f44311e7740 1 -- 172.21.15.86:0/633966188 >> 172.21.15.86:0/633966188 conn(0x55608bab3210 msgr2=0x55608bae0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:28.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.745+0000 7f44311e7740 1 -- 172.21.15.86:0/633966188 shutdown_connections 2024-04-20T21:25:28.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.745+0000 7f44311e7740 1 -- 172.21.15.86:0/633966188 wait complete. 2024-04-20T21:25:28.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.745+0000 7f44311e7740 20 client.7890 trim_cache size 0 max 16384 2024-04-20T21:25:28.747 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:25:28.747 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T21:25:28.747 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-20T21:25:28.747 INFO:teuthology.orchestra.run.smithi086.stdout: "5f0633da-6271-47ec-a18c-7e6a48250ab0": { 2024-04-20T21:25:28.748 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/dir", 2024-04-20T21:25:28.748 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "5f0633da-6271-47ec-a18c-7e6a48250ab0", 2024-04-20T21:25:28.748 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive" 2024-04-20T21:25:28.748 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:25:28.748 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:25:28.748 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:25:28.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.746+0000 7f442e400640 1 -- 172.21.15.86:0/2155896740 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f440806c8f0 msgr2=0x7f440806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:28.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.746+0000 7f442e400640 1 --2- 172.21.15.86:0/2155896740 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f440806c8f0 0x7f440806edb0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f4418006fd0 tx=0x7f4418008040 comp rx=0 tx=0).stop 2024-04-20T21:25:28.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.746+0000 7f442e400640 1 -- 172.21.15.86:0/2155896740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f442806bfd0 msgr2=0x7f44280b4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:28.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.746+0000 7f442e400640 1 --2- 172.21.15.86:0/2155896740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f442806bfd0 0x7f44280b4440 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f441402b950 tx=0x7f44140046c0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.746+0000 7f442e400640 1 -- 172.21.15.86:0/2155896740 shutdown_connections 2024-04-20T21:25:28.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.746+0000 7f442e400640 1 --2- 172.21.15.86:0/2155896740 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f440806c8f0 0x7f440806edb0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.746+0000 7f442e400640 1 --2- 172.21.15.86:0/2155896740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4428146040 0x7f44281e6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.746+0000 7f442e400640 1 --2- 172.21.15.86:0/2155896740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f442806bfd0 0x7f44280b4440 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.746+0000 7f442e400640 1 --2- 172.21.15.86:0/2155896740 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f442806bbf0 0x7f44281347d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:28.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.746+0000 7f442e400640 1 -- 172.21.15.86:0/2155896740 >> 172.21.15.86:0/2155896740 conn(0x7f44280b2110 msgr2=0x7f44280cf310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:28.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.746+0000 7f442e400640 1 -- 172.21.15.86:0/2155896740 shutdown_connections 2024-04-20T21:25:28.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:28.746+0000 7f442e400640 1 -- 172.21.15.86:0/2155896740 wait complete. 2024-04-20T21:25:28.762 INFO:tasks.cephfs.filesystem:scrub status for tag:5f0633da-6271-47ec-a18c-7e6a48250ab0 - {'path': '/dir', 'tag': '5f0633da-6271-47ec-a18c-7e6a48250ab0', 'options': 'recursive'} 2024-04-20T21:25:31.608 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-04-20T21:25:31.609+0000 7f5bbce00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-20T21:25:47.305 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:25:47.314 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:25:58.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:25:58.763 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-20T21:25:58.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.978+0000 7fec82a00640 1 Processor -- start 2024-04-20T21:25:58.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.978+0000 7fec82a00640 1 -- start start 2024-04-20T21:25:58.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.979+0000 7fec82a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fec7c158950 0x7fec7c071db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:58.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.979+0000 7fec82a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec7c0722f0 0x7fec7c0b3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:58.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.979+0000 7fec82a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec7c0b4bf0 con 0x7fec7c0b4400 2024-04-20T21:25:58.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.979+0000 7fec82a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fec7c072750 con 0x7fec7c158950 2024-04-20T21:25:58.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.979+0000 7fec82a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fec7c0b4ed0 con 0x7fec7c0722f0 2024-04-20T21:25:58.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.980+0000 7fec80a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fec7c0b4400 0x7fec7c15ea60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:50934/0 (socket says 172.21.15.86:50934) 2024-04-20T21:25:58.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.980+0000 7fec80a00640 1 -- 172.21.15.86:0/3415709218 learned_addr learned my addr 172.21.15.86:0/3415709218 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:25:58.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.980+0000 7fec73400640 1 --2- 172.21.15.86:0/3415709218 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec7c0722f0 0x7fec7c0b3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:58.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.980+0000 7fec7be00640 1 --2- 172.21.15.86:0/3415709218 >> v2:172.21.15.144:3300/0 conn(0x7fec7c158950 0x7fec7c071db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:58.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.980+0000 7fec7b400640 1 -- 172.21.15.86:0/3415709218 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2448519818 0 0) 0x7fec7c0b4bf0 con 0x7fec7c0b4400 2024-04-20T21:25:58.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.980+0000 7fec7b400640 1 -- 172.21.15.86:0/3415709218 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec60003440 con 0x7fec7c0b4400 2024-04-20T21:25:58.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.981+0000 7fec7b400640 1 -- 172.21.15.86:0/3415709218 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2252739148 0 0) 0x7fec60003440 con 0x7fec7c0b4400 2024-04-20T21:25:58.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.981+0000 7fec7b400640 1 -- 172.21.15.86:0/3415709218 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec7c0722f0 msgr2=0x7fec7c0b3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:58.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.981+0000 7fec7b400640 1 --2- 172.21.15.86:0/3415709218 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec7c0722f0 0x7fec7c0b3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:58.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.981+0000 7fec7b400640 1 -- 172.21.15.86:0/3415709218 >> v2:172.21.15.144:3300/0 conn(0x7fec7c158950 msgr2=0x7fec7c071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:58.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.981+0000 7fec7b400640 1 --2- 172.21.15.86:0/3415709218 >> v2:172.21.15.144:3300/0 conn(0x7fec7c158950 0x7fec7c071db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:58.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.981+0000 7fec7b400640 1 -- 172.21.15.86:0/3415709218 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec7c06de90 con 0x7fec7c0b4400 2024-04-20T21:25:58.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.981+0000 7fec7b400640 1 -- 172.21.15.86:0/3415709218 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fec740019f0 con 0x7fec7c0b4400 2024-04-20T21:25:58.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.981+0000 7fec7b400640 1 -- 172.21.15.86:0/3415709218 >> v1:172.21.15.86:6789/0 conn(0x7fec7c0b4400 legacy=0x7fec7c15ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:58.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.981+0000 7fec7b400640 1 --2- 172.21.15.86:0/3415709218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec60003aa0 0x7fec60003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:58.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.981+0000 7fec7b400640 1 --2- 172.21.15.86:0/3415709218 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec7c0722f0 0x7fec6000a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:58.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.981+0000 7fec7b400640 1 --2- 172.21.15.86:0/3415709218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec6000ae30 0x7fec6000d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:58.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.981+0000 7fec7b400640 1 -- 172.21.15.86:0/3415709218 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fec7c0b4ed0 con 0x7fec6000ae30 2024-04-20T21:25:58.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.981+0000 7fec7b400640 1 -- 172.21.15.86:0/3415709218 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fec6000dad0 con 0x7fec60003aa0 2024-04-20T21:25:58.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.981+0000 7fec7b400640 1 -- 172.21.15.86:0/3415709218 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fec6000dcd0 con 0x7fec7c0722f0 2024-04-20T21:25:58.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.981+0000 7fec80a00640 1 --2- 172.21.15.86:0/3415709218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec6000ae30 0x7fec6000d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:58.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.982+0000 7fec80a00640 1 -- 172.21.15.86:0/3415709218 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec7c0722f0 msgr2=0x7fec6000a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:58.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.982+0000 7fec80a00640 1 --2- 172.21.15.86:0/3415709218 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec7c0722f0 0x7fec6000a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:58.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.982+0000 7fec80a00640 1 -- 172.21.15.86:0/3415709218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec60003aa0 msgr2=0x7fec60003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:58.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.982+0000 7fec80a00640 1 --2- 172.21.15.86:0/3415709218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec60003aa0 0x7fec60003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:58.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.982+0000 7fec80a00640 1 -- 172.21.15.86:0/3415709218 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec60003440 con 0x7fec6000ae30 2024-04-20T21:25:58.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.983+0000 7fec80a00640 1 --2- 172.21.15.86:0/3415709218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec6000ae30 0x7fec6000d2c0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fec7400a880 tx=0x7fec60003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca332e8ca3d6529 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:58.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.983+0000 7fec7b400640 1 -- 172.21.15.86:0/3415709218 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7400a4a0 con 0x7fec6000ae30 2024-04-20T21:25:58.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.983+0000 7fec7b400640 1 -- 172.21.15.86:0/3415709218 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec74034650 con 0x7fec6000ae30 2024-04-20T21:25:58.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.983+0000 7fec7b400640 1 -- 172.21.15.86:0/3415709218 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec74034a40 con 0x7fec6000ae30 2024-04-20T21:25:58.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.984+0000 7fec82a00640 1 -- 172.21.15.86:0/3415709218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec6000ae30 msgr2=0x7fec6000d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:58.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.984+0000 7fec82a00640 1 --2- 172.21.15.86:0/3415709218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec6000ae30 0x7fec6000d2c0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fec7400a880 tx=0x7fec60003730 comp rx=0 tx=0).stop 2024-04-20T21:25:58.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.984+0000 7fec80a00640 1 -- 172.21.15.86:0/3415709218 reap_dead start 2024-04-20T21:25:58.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.984+0000 7fec82a00640 1 -- 172.21.15.86:0/3415709218 shutdown_connections 2024-04-20T21:25:58.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.984+0000 7fec82a00640 1 -- 172.21.15.86:0/3415709218 >> 172.21.15.86:0/3415709218 conn(0x7fec7c0b2110 msgr2=0x7fec7c0b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:58.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.984+0000 7fec82a00640 1 -- 172.21.15.86:0/3415709218 shutdown_connections 2024-04-20T21:25:58.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.984+0000 7fec82a00640 1 -- 172.21.15.86:0/3415709218 wait complete. 2024-04-20T21:25:58.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.986+0000 7fec82a00640 1 Processor -- start 2024-04-20T21:25:58.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.986+0000 7fec82a00640 1 -- start start 2024-04-20T21:25:58.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.987+0000 7fec82a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec6000ae30 0x7fec7c0d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:58.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.987+0000 7fec82a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec7c0ad850 0x7fec7c151f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:58.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.987+0000 7fec82a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec7c0aee60 0x7fec7c0af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:58.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.987+0000 7fec82a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fec7c0b4870 con 0x7fec7c0aee60 2024-04-20T21:25:58.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.987+0000 7fec82a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fec7c06e450 con 0x7fec6000ae30 2024-04-20T21:25:58.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.987+0000 7fec82a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fec7c158d30 con 0x7fec7c0ad850 2024-04-20T21:25:58.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.987+0000 7fec80a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec7c0aee60 0x7fec7c0af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:58.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.987+0000 7fec80a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec7c0aee60 0x7fec7c0af2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42876/0 (socket says 172.21.15.86:42876) 2024-04-20T21:25:58.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.987+0000 7fec80a00640 1 -- 172.21.15.86:0/1364471954 learned_addr learned my addr 172.21.15.86:0/1364471954 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:58.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.987+0000 7fec7be00640 1 --2- 172.21.15.86:0/1364471954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec6000ae30 0x7fec7c0d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:58.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.987+0000 7fec73400640 1 --2- 172.21.15.86:0/1364471954 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec7c0ad850 0x7fec7c151f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:58.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.987+0000 7fec80a00640 1 -- 172.21.15.86:0/1364471954 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec7c0ad850 msgr2=0x7fec7c151f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:58.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.987+0000 7fec80a00640 1 --2- 172.21.15.86:0/1364471954 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec7c0ad850 0x7fec7c151f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:58.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.987+0000 7fec80a00640 1 -- 172.21.15.86:0/1364471954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec6000ae30 msgr2=0x7fec7c0d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:58.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.987+0000 7fec80a00640 1 --2- 172.21.15.86:0/1364471954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec6000ae30 0x7fec7c0d02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:58.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.987+0000 7fec80a00640 1 -- 172.21.15.86:0/1364471954 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec7c0addc0 con 0x7fec7c0aee60 2024-04-20T21:25:58.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.987+0000 7fec7be00640 1 --2- 172.21.15.86:0/1364471954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec6000ae30 0x7fec7c0d02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:25:58.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.988+0000 7fec80a00640 1 --2- 172.21.15.86:0/1364471954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec7c0aee60 0x7fec7c0af2e0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fec7400a9a0 tx=0x7fec7c0b4dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:58.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.988+0000 7fec78c00640 1 -- 172.21.15.86:0/1364471954 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec74045070 con 0x7fec7c0aee60 2024-04-20T21:25:58.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.988+0000 7fec78c00640 1 -- 172.21.15.86:0/1364471954 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec7403a040 con 0x7fec7c0aee60 2024-04-20T21:25:58.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.988+0000 7fec78c00640 1 -- 172.21.15.86:0/1364471954 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec74001ff0 con 0x7fec7c0aee60 2024-04-20T21:25:58.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.988+0000 7fec82a00640 1 -- 172.21.15.86:0/1364471954 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec7c0ae0a0 con 0x7fec7c0aee60 2024-04-20T21:25:58.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.988+0000 7fec82a00640 1 -- 172.21.15.86:0/1364471954 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec7c0ae660 con 0x7fec7c0aee60 2024-04-20T21:25:58.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.991+0000 7fec78c00640 1 -- 172.21.15.86:0/1364471954 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fec74048070 con 0x7fec7c0aee60 2024-04-20T21:25:58.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.992+0000 7fec78c00640 1 --2- 172.21.15.86:0/1364471954 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fec5806c920 0x7fec5806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:58.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.992+0000 7fec78c00640 1 -- 172.21.15.86:0/1364471954 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7fec740ad7e0 con 0x7fec7c0aee60 2024-04-20T21:25:58.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.992+0000 7fec7be00640 1 --2- 172.21.15.86:0/1364471954 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fec5806c920 0x7fec5806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:58.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.993+0000 7fec85815740 1 Processor -- start 2024-04-20T21:25:58.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.993+0000 7fec85815740 1 -- start start 2024-04-20T21:25:58.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.993+0000 7fec7be00640 1 --2- 172.21.15.86:0/1364471954 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fec5806c920 0x7fec5806ede0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fec7c1f7d60 tx=0x7fec6c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:58.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.994+0000 7fec85815740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636d7e881f0 0x5636d7e885d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:58.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.994+0000 7fec85815740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5636d7e58210 0x5636d7e85460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:58.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.994+0000 7fec85815740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636d7e5bd70 0x5636d7e859a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:58.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.994+0000 7fec85815740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5636d7dc4150 con 0x5636d7e5bd70 2024-04-20T21:25:58.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.994+0000 7fec85815740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5636d7d6a370 con 0x5636d7e881f0 2024-04-20T21:25:58.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.994+0000 7fec85815740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5636d7dc0110 con 0x5636d7e58210 2024-04-20T21:25:58.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.994+0000 7fec80a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636d7e5bd70 0x5636d7e859a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:58.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.994+0000 7fec80a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636d7e5bd70 0x5636d7e859a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42890/0 (socket says 172.21.15.86:42890) 2024-04-20T21:25:58.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.994+0000 7fec80a00640 1 -- 172.21.15.86:0/3811137363 learned_addr learned my addr 172.21.15.86:0/3811137363 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:58.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.994+0000 7fec80a00640 1 -- 172.21.15.86:0/3811137363 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5636d7e58210 msgr2=0x5636d7e85460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:58.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.994+0000 7fec80a00640 1 --2- 172.21.15.86:0/3811137363 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5636d7e58210 0x5636d7e85460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:58.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.994+0000 7fec80a00640 1 -- 172.21.15.86:0/3811137363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636d7e881f0 msgr2=0x5636d7e885d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:58.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.994+0000 7fec80a00640 1 --2- 172.21.15.86:0/3811137363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636d7e881f0 0x5636d7e885d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:58.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.994+0000 7fec80a00640 1 -- 172.21.15.86:0/3811137363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636d7ba07d0 con 0x5636d7e5bd70 2024-04-20T21:25:58.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.995+0000 7fec80a00640 1 --2- 172.21.15.86:0/3811137363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636d7e5bd70 0x5636d7e859a0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fec740abb90 tx=0x7fec7407aeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f7638ac9754873 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:58.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.995+0000 7fec6be00640 1 -- 172.21.15.86:0/3811137363 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec74083f80 con 0x5636d7e5bd70 2024-04-20T21:25:58.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.995+0000 7fec6be00640 1 -- 172.21.15.86:0/3811137363 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec74079750 con 0x5636d7e5bd70 2024-04-20T21:25:58.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.995+0000 7fec6be00640 1 -- 172.21.15.86:0/3811137363 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7407a8a0 con 0x5636d7e5bd70 2024-04-20T21:25:58.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.996+0000 7fec85815740 1 -- 172.21.15.86:0/3811137363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636d7e5bd70 msgr2=0x5636d7e859a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:58.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.996+0000 7fec85815740 1 --2- 172.21.15.86:0/3811137363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636d7e5bd70 0x5636d7e859a0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fec740abb90 tx=0x7fec7407aeb0 comp rx=0 tx=0).stop 2024-04-20T21:25:58.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.996+0000 7fec85815740 1 -- 172.21.15.86:0/3811137363 shutdown_connections 2024-04-20T21:25:58.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.996+0000 7fec85815740 1 --2- 172.21.15.86:0/3811137363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636d7e5bd70 0x5636d7e859a0 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:58.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.996+0000 7fec85815740 1 --2- 172.21.15.86:0/3811137363 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5636d7e58210 0x5636d7e85460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:58.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.996+0000 7fec85815740 1 --2- 172.21.15.86:0/3811137363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636d7e881f0 0x5636d7e885d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:58.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.996+0000 7fec85815740 1 -- 172.21.15.86:0/3811137363 >> 172.21.15.86:0/3811137363 conn(0x5636d7e5a480 msgr2=0x5636d7ea3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:58.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.996+0000 7fec85815740 1 -- 172.21.15.86:0/3811137363 shutdown_connections 2024-04-20T21:25:58.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.996+0000 7fec85815740 1 -- 172.21.15.86:0/3811137363 wait complete. 2024-04-20T21:25:58.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.997+0000 7fec85815740 1 -- start start 2024-04-20T21:25:59.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.997+0000 7fec85815740 1 Processor -- start 2024-04-20T21:25:59.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.998+0000 7fec85815740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5636d7e5bd70 0x5636d7e80840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:59.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.998+0000 7fec85815740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636d7e881f0 0x5636d7e8b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:59.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.998+0000 7fec85815740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636d7ea06a0 0x5636d7eb6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:59.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.998+0000 7fec85815740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5636d7d6a370 con 0x5636d7ea06a0 2024-04-20T21:25:59.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.998+0000 7fec85815740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5636d7d7d740 con 0x5636d7e881f0 2024-04-20T21:25:59.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.998+0000 7fec85815740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5636d7dc4150 con 0x5636d7e5bd70 2024-04-20T21:25:59.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.998+0000 7fec85815740 10 client.0 _ll_register_callbacks cb 0x5636d7d78b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:25:59.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.998+0000 7fec80a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636d7ea06a0 0x5636d7eb6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:59.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.998+0000 7fec80a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636d7ea06a0 0x5636d7eb6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42904/0 (socket says 172.21.15.86:42904) 2024-04-20T21:25:59.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.998+0000 7fec80a00640 1 -- 172.21.15.86:0/164761433 learned_addr learned my addr 172.21.15.86:0/164761433 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:59.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.998+0000 7fec7be00640 1 --2- 172.21.15.86:0/164761433 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5636d7e5bd70 0x5636d7e80840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:59.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.999+0000 7fec80a00640 1 -- 172.21.15.86:0/164761433 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5636d7e5bd70 msgr2=0x5636d7e80840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:59.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.999+0000 7fec80a00640 1 --2- 172.21.15.86:0/164761433 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5636d7e5bd70 0x5636d7e80840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.999+0000 7fec73400640 1 --2- 172.21.15.86:0/164761433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636d7e881f0 0x5636d7e8b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:59.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.999+0000 7fec80a00640 1 -- 172.21.15.86:0/164761433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636d7e881f0 msgr2=0x5636d7e8b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:59.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.999+0000 7fec80a00640 1 --2- 172.21.15.86:0/164761433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636d7e881f0 0x5636d7e8b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.999+0000 7fec80a00640 1 -- 172.21.15.86:0/164761433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636d7ad63f0 con 0x5636d7ea06a0 2024-04-20T21:25:59.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.999+0000 7fec80a00640 1 --2- 172.21.15.86:0/164761433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636d7ea06a0 0x5636d7eb6940 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fec7400a880 tx=0x7fec7400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff5a0f29e5ceb834 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:59.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.999+0000 7fec68c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:25:59.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.999+0000 7fec68c00640 1 -- 172.21.15.86:0/164761433 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec74099320 con 0x5636d7ea06a0 2024-04-20T21:25:59.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.999+0000 7fec68c00640 1 -- 172.21.15.86:0/164761433 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7fec740a6ca0 con 0x5636d7ea06a0 2024-04-20T21:25:59.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.999+0000 7fec85815740 1 -- 172.21.15.86:0/164761433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec74079750 con 0x5636d7ea06a0 2024-04-20T21:25:59.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.999+0000 7fec68c00640 1 -- 172.21.15.86:0/164761433 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fec740a6ea0 con 0x5636d7ea06a0 2024-04-20T21:25:59.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.999+0000 7fec85815740 10 client.7902 fetch_fsmap learned FSMap version 66 2024-04-20T21:25:59.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:58.999+0000 7fec85815740 1 -- 172.21.15.86:0/164761433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x5636d7a960f0 con 0x5636d7ea06a0 2024-04-20T21:25:59.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.000+0000 7fec68c00640 1 -- 172.21.15.86:0/164761433 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec74079750 con 0x5636d7ea06a0 2024-04-20T21:25:59.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.000+0000 7fec68c00640 1 -- 172.21.15.86:0/164761433 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 66) ==== 3495+0+0 (secure 0 0 0) 0x7fec74093b80 con 0x5636d7ea06a0 2024-04-20T21:25:59.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.000+0000 7fec85815740 10 client.7902 fetch_fsmap finished waiting for FSMap version 66 2024-04-20T21:25:59.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.000+0000 7fec85815740 10 client.7902 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-04-20T21:25:59.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.000+0000 7fec85815740 20 client.7902 populate_metadata read hostname 'smithi086' 2024-04-20T21:25:59.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.000+0000 7fec85815740 1 --2- 172.21.15.86:0/164761433 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x5636d7e85b60 0x5636d7e85f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:59.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.000+0000 7fec85815740 4 client.7902 mds_command: new command op to 7773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:25:59.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.000+0000 7fec85815740 1 -- 172.21.15.86:0/164761433 --> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636d7dd1b70 con 0x5636d7e85b60 2024-04-20T21:25:59.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.000+0000 7fec7be00640 1 --2- 172.21.15.86:0/164761433 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x5636d7e85b60 0x5636d7e85f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:59.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.001+0000 7fec7be00640 1 --2- 172.21.15.86:0/164761433 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x5636d7e85b60 0x5636d7e85f40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d906b03ba8211473 server_cookie=1e430a248d923e64 in_seq=0 out_seq=0 2024-04-20T21:25:59.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.001+0000 7fec68c00640 10 client.7902 ms_handle_connect on v2:172.21.15.144:6833/3981403733 2024-04-20T21:25:59.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.003+0000 7fec68c00640 1 -- 172.21.15.86:0/164761433 <== mds.0 v2:172.21.15.144:6833/3981403733 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5636d7dd1b70 con 0x5636d7e85b60 2024-04-20T21:25:59.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.003+0000 7fec68c00640 10 client.7902 handle_command_reply: tid=0 2024-04-20T21:25:59.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.004+0000 7fec85815740 1 client.7902 shutdown 2024-04-20T21:25:59.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.004+0000 7fec85815740 1 -- 172.21.15.86:0/164761433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636d7ea06a0 msgr2=0x5636d7eb6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:59.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.004+0000 7fec85815740 1 --2- 172.21.15.86:0/164761433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636d7ea06a0 0x5636d7eb6940 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fec7400a880 tx=0x7fec7400a060 comp rx=0 tx=0).stop 2024-04-20T21:25:59.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.004+0000 7fec85815740 1 -- 172.21.15.86:0/164761433 shutdown_connections 2024-04-20T21:25:59.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.004+0000 7fec85815740 1 --2- 172.21.15.86:0/164761433 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x5636d7e85b60 0x5636d7e85f40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.004+0000 7fec85815740 1 --2- 172.21.15.86:0/164761433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636d7ea06a0 0x5636d7eb6940 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.004+0000 7fec85815740 1 --2- 172.21.15.86:0/164761433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636d7e881f0 0x5636d7e8b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.004+0000 7fec85815740 1 --2- 172.21.15.86:0/164761433 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5636d7e5bd70 0x5636d7e80840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.004+0000 7fec85815740 1 -- 172.21.15.86:0/164761433 >> 172.21.15.86:0/164761433 conn(0x5636d7e58210 msgr2=0x5636d7ea6510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:59.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.004+0000 7fec68c00640 0 client.7902 ms_handle_reset on v2:172.21.15.144:6833/3981403733 2024-04-20T21:25:59.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.004+0000 7fec85815740 1 -- 172.21.15.86:0/164761433 shutdown_connections 2024-04-20T21:25:59.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.004+0000 7fec85815740 1 -- 172.21.15.86:0/164761433 wait complete. 2024-04-20T21:25:59.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.004+0000 7fec85815740 20 client.7902 trim_cache size 0 max 16384 2024-04-20T21:25:59.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.023+0000 7fec85815740 1 Processor -- start 2024-04-20T21:25:59.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.023+0000 7fec85815740 1 -- start start 2024-04-20T21:25:59.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.024+0000 7fec85815740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636d7e5bd70 0x5636d7e7fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:59.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.024+0000 7fec85815740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636d7e881f0 0x5636d7ea6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:59.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.024+0000 7fec85815740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5636d7ea06a0 0x5636d7ea6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:59.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.024+0000 7fec85815740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fec7c201460 con 0x5636d7e881f0 2024-04-20T21:25:59.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.024+0000 7fec85815740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fec7c200470 con 0x5636d7e5bd70 2024-04-20T21:25:59.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.024+0000 7fec85815740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fec7c0ce3e0 con 0x5636d7ea06a0 2024-04-20T21:25:59.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.024+0000 7fec73400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636d7e881f0 0x5636d7ea6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:59.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.024+0000 7fec73400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636d7e881f0 0x5636d7ea6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42906/0 (socket says 172.21.15.86:42906) 2024-04-20T21:25:59.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.024+0000 7fec73400640 1 -- 172.21.15.86:0/2386342162 learned_addr learned my addr 172.21.15.86:0/2386342162 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:59.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.025+0000 7fec73400640 1 -- 172.21.15.86:0/2386342162 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5636d7ea06a0 msgr2=0x5636d7ea6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:59.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.025+0000 7fec73400640 1 --2- 172.21.15.86:0/2386342162 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5636d7ea06a0 0x5636d7ea6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.025+0000 7fec73400640 1 -- 172.21.15.86:0/2386342162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636d7e5bd70 msgr2=0x5636d7e7fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:59.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.025+0000 7fec73400640 1 --2- 172.21.15.86:0/2386342162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636d7e5bd70 0x5636d7e7fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.025+0000 7fec73400640 1 -- 172.21.15.86:0/2386342162 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636d7e54190 con 0x5636d7e881f0 2024-04-20T21:25:59.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.025+0000 7fec73400640 1 --2- 172.21.15.86:0/2386342162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636d7e881f0 0x5636d7ea6510 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fec640097f0 tx=0x7fec64009cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc41e78624898ead server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:59.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.025+0000 7fec6b400640 1 -- 172.21.15.86:0/2386342162 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec64019020 con 0x5636d7e881f0 2024-04-20T21:25:59.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.025+0000 7fec6b400640 1 -- 172.21.15.86:0/2386342162 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec64004230 con 0x5636d7e881f0 2024-04-20T21:25:59.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.025+0000 7fec6b400640 1 -- 172.21.15.86:0/2386342162 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec64011740 con 0x5636d7e881f0 2024-04-20T21:25:59.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.026+0000 7fec85815740 1 -- 172.21.15.86:0/2386342162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636d7e881f0 msgr2=0x5636d7ea6510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:59.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.026+0000 7fec85815740 1 --2- 172.21.15.86:0/2386342162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636d7e881f0 0x5636d7ea6510 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fec640097f0 tx=0x7fec64009cb0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.026+0000 7fec85815740 1 -- 172.21.15.86:0/2386342162 shutdown_connections 2024-04-20T21:25:59.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.026+0000 7fec85815740 1 --2- 172.21.15.86:0/2386342162 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5636d7ea06a0 0x5636d7ea6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.026+0000 7fec85815740 1 --2- 172.21.15.86:0/2386342162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636d7e881f0 0x5636d7ea6510 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.026+0000 7fec85815740 1 --2- 172.21.15.86:0/2386342162 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636d7e5bd70 0x5636d7e7fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.026+0000 7fec85815740 1 -- 172.21.15.86:0/2386342162 >> 172.21.15.86:0/2386342162 conn(0x5636d7e58210 msgr2=0x5636d7ea0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:59.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.026+0000 7fec85815740 1 -- 172.21.15.86:0/2386342162 shutdown_connections 2024-04-20T21:25:59.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.026+0000 7fec85815740 1 -- 172.21.15.86:0/2386342162 wait complete. 2024-04-20T21:25:59.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.027+0000 7fec85815740 1 -- start start 2024-04-20T21:25:59.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.027+0000 7fec85815740 1 Processor -- start 2024-04-20T21:25:59.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.028+0000 7fec85815740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5636d7e5bd70 0x5636d7ea6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:59.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.028+0000 7fec85815740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636d7e881f0 0x5636d7ea6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:59.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.028+0000 7fec85815740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636d7ea06a0 0x5636d7f25840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:59.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.028+0000 7fec85815740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fec7c0b0d40 con 0x5636d7e881f0 2024-04-20T21:25:59.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.028+0000 7fec85815740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5636d7e58fd0 con 0x5636d7ea06a0 2024-04-20T21:25:59.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.028+0000 7fec85815740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5636d7f26760 con 0x5636d7e5bd70 2024-04-20T21:25:59.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.028+0000 7fec85815740 10 client.0 _ll_register_callbacks cb 0x7fec7407b020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:25:59.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.028+0000 7fec73400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636d7e881f0 0x5636d7ea6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:59.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.028+0000 7fec73400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636d7e881f0 0x5636d7ea6a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42910/0 (socket says 172.21.15.86:42910) 2024-04-20T21:25:59.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.028+0000 7fec73400640 1 -- 172.21.15.86:0/254129596 learned_addr learned my addr 172.21.15.86:0/254129596 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:59.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.028+0000 7fec7be00640 1 --2- 172.21.15.86:0/254129596 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5636d7e5bd70 0x5636d7ea6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:59.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.028+0000 7fec73400640 1 -- 172.21.15.86:0/254129596 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5636d7e5bd70 msgr2=0x5636d7ea6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:59.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.028+0000 7fec73400640 1 --2- 172.21.15.86:0/254129596 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5636d7e5bd70 0x5636d7ea6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.028+0000 7fec80a00640 1 --2- 172.21.15.86:0/254129596 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636d7ea06a0 0x5636d7f25840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:59.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.028+0000 7fec73400640 1 -- 172.21.15.86:0/254129596 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636d7ea06a0 msgr2=0x5636d7f25840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:59.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.028+0000 7fec73400640 1 --2- 172.21.15.86:0/254129596 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636d7ea06a0 0x5636d7f25840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.028+0000 7fec73400640 1 -- 172.21.15.86:0/254129596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636d7e842d0 con 0x5636d7e881f0 2024-04-20T21:25:59.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.029+0000 7fec73400640 1 --2- 172.21.15.86:0/254129596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636d7e881f0 0x5636d7ea6a50 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fec64018040 tx=0x7fec640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cda14ea9fbe51e8e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:59.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.029+0000 7fec68c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:25:59.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.029+0000 7fec68c00640 1 -- 172.21.15.86:0/254129596 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec64004030 con 0x5636d7e881f0 2024-04-20T21:25:59.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.029+0000 7fec68c00640 1 -- 172.21.15.86:0/254129596 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7fec64006b40 con 0x5636d7e881f0 2024-04-20T21:25:59.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.029+0000 7fec85815740 1 -- 172.21.15.86:0/254129596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec64004230 con 0x5636d7e881f0 2024-04-20T21:25:59.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.029+0000 7fec68c00640 1 -- 172.21.15.86:0/254129596 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fec640043e0 con 0x5636d7e881f0 2024-04-20T21:25:59.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.029+0000 7fec85815740 10 client.7908 fetch_fsmap learned FSMap version 66 2024-04-20T21:25:59.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.029+0000 7fec85815740 1 -- 172.21.15.86:0/254129596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x5636d7e9fe90 con 0x5636d7e881f0 2024-04-20T21:25:59.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.030+0000 7fec68c00640 1 -- 172.21.15.86:0/254129596 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec64004230 con 0x5636d7e881f0 2024-04-20T21:25:59.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.030+0000 7fec68c00640 1 -- 172.21.15.86:0/254129596 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 66) ==== 3495+0+0 (secure 0 0 0) 0x7fec64032050 con 0x5636d7e881f0 2024-04-20T21:25:59.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.030+0000 7fec85815740 10 client.7908 fetch_fsmap finished waiting for FSMap version 66 2024-04-20T21:25:59.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.030+0000 7fec85815740 10 client.7908 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-04-20T21:25:59.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.030+0000 7fec85815740 20 client.7908 populate_metadata read hostname 'smithi086' 2024-04-20T21:25:59.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.030+0000 7fec85815740 1 --2- 172.21.15.86:0/254129596 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x5636d7f23080 0x5636d7f25d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:59.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.030+0000 7fec85815740 4 client.7908 mds_command: new command op to 7773 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:25:59.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.030+0000 7fec85815740 1 -- 172.21.15.86:0/254129596 --> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5636d7dcb9e0 con 0x5636d7f23080 2024-04-20T21:25:59.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.030+0000 7fec7be00640 1 --2- 172.21.15.86:0/254129596 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x5636d7f23080 0x5636d7f25d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:59.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.032+0000 7fec7be00640 1 --2- 172.21.15.86:0/254129596 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x5636d7f23080 0x5636d7f25d80 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=778a83e86f8bdd92 server_cookie=1951001ac478346d in_seq=0 out_seq=0 2024-04-20T21:25:59.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.032+0000 7fec68c00640 10 client.7908 ms_handle_connect on v2:172.21.15.144:6833/3981403733 2024-04-20T21:25:59.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.032+0000 7fec68c00640 1 -- 172.21.15.86:0/254129596 <== mds.0 v2:172.21.15.144:6833/3981403733 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5636d7dcb9e0 con 0x5636d7f23080 2024-04-20T21:25:59.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.032+0000 7fec68c00640 10 client.7908 handle_command_reply: tid=0 2024-04-20T21:25:59.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.032+0000 7fec85815740 1 client.7908 shutdown 2024-04-20T21:25:59.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.033+0000 7fec85815740 1 -- 172.21.15.86:0/254129596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636d7e881f0 msgr2=0x5636d7ea6a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:59.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.033+0000 7fec85815740 1 --2- 172.21.15.86:0/254129596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636d7e881f0 0x5636d7ea6a50 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fec64018040 tx=0x7fec640046c0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.033+0000 7fec85815740 1 -- 172.21.15.86:0/254129596 shutdown_connections 2024-04-20T21:25:59.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.033+0000 7fec85815740 1 --2- 172.21.15.86:0/254129596 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x5636d7f23080 0x5636d7f25d80 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.033+0000 7fec85815740 1 --2- 172.21.15.86:0/254129596 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5636d7ea06a0 0x5636d7f25840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.033+0000 7fec85815740 1 --2- 172.21.15.86:0/254129596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636d7e881f0 0x5636d7ea6a50 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.033+0000 7fec85815740 1 --2- 172.21.15.86:0/254129596 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5636d7e5bd70 0x5636d7ea6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.033+0000 7fec85815740 1 -- 172.21.15.86:0/254129596 >> 172.21.15.86:0/254129596 conn(0x5636d7e58210 msgr2=0x5636d7e57030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:59.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.033+0000 7fec68c00640 0 client.7908 ms_handle_reset on v2:172.21.15.144:6833/3981403733 2024-04-20T21:25:59.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.033+0000 7fec85815740 1 -- 172.21.15.86:0/254129596 shutdown_connections 2024-04-20T21:25:59.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.033+0000 7fec85815740 1 -- 172.21.15.86:0/254129596 wait complete. 2024-04-20T21:25:59.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.033+0000 7fec85815740 20 client.7908 trim_cache size 0 max 16384 2024-04-20T21:25:59.036 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:25:59.036 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-20T21:25:59.036 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-20T21:25:59.036 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:25:59.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.034+0000 7fec82a00640 1 -- 172.21.15.86:0/1364471954 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fec5806c920 msgr2=0x7fec5806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:59.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.034+0000 7fec82a00640 1 --2- 172.21.15.86:0/1364471954 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fec5806c920 0x7fec5806ede0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fec7c1f7d60 tx=0x7fec6c008040 comp rx=0 tx=0).stop 2024-04-20T21:25:59.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.034+0000 7fec82a00640 1 -- 172.21.15.86:0/1364471954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec7c0aee60 msgr2=0x7fec7c0af2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:59.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.034+0000 7fec82a00640 1 --2- 172.21.15.86:0/1364471954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec7c0aee60 0x7fec7c0af2e0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fec7400a9a0 tx=0x7fec7c0b4dc0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.034+0000 7fec82a00640 1 -- 172.21.15.86:0/1364471954 shutdown_connections 2024-04-20T21:25:59.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.034+0000 7fec82a00640 1 --2- 172.21.15.86:0/1364471954 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fec5806c920 0x7fec5806ede0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.034+0000 7fec82a00640 1 --2- 172.21.15.86:0/1364471954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec7c0aee60 0x7fec7c0af2e0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.034+0000 7fec82a00640 1 --2- 172.21.15.86:0/1364471954 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fec7c0ad850 0x7fec7c151f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.034+0000 7fec82a00640 1 --2- 172.21.15.86:0/1364471954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec6000ae30 0x7fec7c0d02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.034+0000 7fec82a00640 1 -- 172.21.15.86:0/1364471954 >> 172.21.15.86:0/1364471954 conn(0x7fec7c0b2110 msgr2=0x7fec7c15c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:59.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.034+0000 7fec82a00640 1 -- 172.21.15.86:0/1364471954 shutdown_connections 2024-04-20T21:25:59.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.034+0000 7fec82a00640 1 -- 172.21.15.86:0/1364471954 wait complete. 2024-04-20T21:25:59.051 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T21:25:59.051 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:25:59.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.254+0000 7f2b8b200640 1 Processor -- start 2024-04-20T21:25:59.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.254+0000 7f2b8b200640 1 -- start start 2024-04-20T21:25:59.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.255+0000 7f2b8b200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2b8c156ae0 0x7f2b8c15d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:59.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.255+0000 7f2b8b200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2b8c0ad7f0 0x7f2b8c06e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:59.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.255+0000 7f2b8b200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b8c160890 con 0x7f2b8c160080 2024-04-20T21:25:59.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.255+0000 7f2b8b200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f2b8c06db20 con 0x7f2b8c156ae0 2024-04-20T21:25:59.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.255+0000 7f2b8b200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2b8c160b70 con 0x7f2b8c0ad7f0 2024-04-20T21:25:59.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.256+0000 7f2b89e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2b8c156ae0 0x7f2b8c15d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:59.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.256+0000 7f2b89e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2b8c156ae0 0x7f2b8c15d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42916/0 (socket says 172.21.15.86:42916) 2024-04-20T21:25:59.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.256+0000 7f2b89e00640 1 -- 172.21.15.86:0/2311979048 learned_addr learned my addr 172.21.15.86:0/2311979048 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:59.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.256+0000 7f2b89400640 1 --1- 172.21.15.86:0/2311979048 >> v1:172.21.15.144:6789/0 conn(0x7f2b8c160080 0x7f2b8c0ad0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.86:34884/0 (socket says 172.21.15.86:34884) 2024-04-20T21:25:59.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.256+0000 7f2b8a800640 1 --2- 172.21.15.86:0/2311979048 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2b8c0ad7f0 0x7f2b8c06e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:59.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.256+0000 7f2b89e00640 1 -- 172.21.15.86:0/2311979048 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2b8c0ad7f0 msgr2=0x7f2b8c06e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:59.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.256+0000 7f2b89e00640 1 --2- 172.21.15.86:0/2311979048 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2b8c0ad7f0 0x7f2b8c06e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.256+0000 7f2b89e00640 1 -- 172.21.15.86:0/2311979048 >> v1:172.21.15.144:6789/0 conn(0x7f2b8c160080 legacy=0x7f2b8c0ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:59.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.256+0000 7f2b89e00640 1 -- 172.21.15.86:0/2311979048 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b8c160e50 con 0x7f2b8c156ae0 2024-04-20T21:25:59.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.258+0000 7f2b89e00640 1 --2- 172.21.15.86:0/2311979048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b8c156ae0 0x7f2b8c15d6a0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c009870 tx=0x7f2b7c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f4a1302b325d2ae server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:59.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.258+0000 7f2b88a00640 1 -- 172.21.15.86:0/2311979048 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c034070 con 0x7f2b8c156ae0 2024-04-20T21:25:59.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.258+0000 7f2b88a00640 1 -- 172.21.15.86:0/2311979048 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b7c030840 con 0x7f2b8c156ae0 2024-04-20T21:25:59.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.258+0000 7f2b88a00640 1 -- 172.21.15.86:0/2311979048 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c030c10 con 0x7f2b8c156ae0 2024-04-20T21:25:59.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.259+0000 7f2b8b200640 1 -- 172.21.15.86:0/2311979048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b8c156ae0 msgr2=0x7f2b8c15d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:59.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.259+0000 7f2b8b200640 1 --2- 172.21.15.86:0/2311979048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b8c156ae0 0x7f2b8c15d6a0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c009870 tx=0x7f2b7c02b980 comp rx=0 tx=0).stop 2024-04-20T21:25:59.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.259+0000 7f2b8b200640 1 -- 172.21.15.86:0/2311979048 shutdown_connections 2024-04-20T21:25:59.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.259+0000 7f2b8b200640 1 --2- 172.21.15.86:0/2311979048 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2b8c0ad7f0 0x7f2b8c06e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.259+0000 7f2b8b200640 1 --2- 172.21.15.86:0/2311979048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b8c156ae0 0x7f2b8c15d6a0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.259+0000 7f2b8b200640 1 -- 172.21.15.86:0/2311979048 >> 172.21.15.86:0/2311979048 conn(0x7f2b8c0d0170 msgr2=0x7f2b8c0d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:59.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.259+0000 7f2b8b200640 1 -- 172.21.15.86:0/2311979048 shutdown_connections 2024-04-20T21:25:59.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.259+0000 7f2b8b200640 1 -- 172.21.15.86:0/2311979048 wait complete. 2024-04-20T21:25:59.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.261+0000 7f2b8b200640 1 Processor -- start 2024-04-20T21:25:59.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.261+0000 7f2b8b200640 1 -- start start 2024-04-20T21:25:59.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.261+0000 7f2b8b200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b8c0ad7f0 0x7f2b8c1f4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:59.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.262+0000 7f2b8b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b8c156ae0 0x7f2b8c15ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:59.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.262+0000 7f2b8b200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2b8c160080 0x7f2b8c15f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:59.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.262+0000 7f2b8b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8c160500 con 0x7f2b8c156ae0 2024-04-20T21:25:59.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.262+0000 7f2b8b200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8c156ec0 con 0x7f2b8c0ad7f0 2024-04-20T21:25:59.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.262+0000 7f2b8b200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2b8c06e7d0 con 0x7f2b8c160080 2024-04-20T21:25:59.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.262+0000 7f2b89400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b8c156ae0 0x7f2b8c15ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:59.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.262+0000 7f2b89400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b8c156ae0 0x7f2b8c15ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42922/0 (socket says 172.21.15.86:42922) 2024-04-20T21:25:59.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.262+0000 7f2b89400640 1 -- 172.21.15.86:0/2492257105 learned_addr learned my addr 172.21.15.86:0/2492257105 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:59.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.262+0000 7f2b89400640 1 -- 172.21.15.86:0/2492257105 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2b8c160080 msgr2=0x7f2b8c15f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:59.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.262+0000 7f2b89400640 1 --2- 172.21.15.86:0/2492257105 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2b8c160080 0x7f2b8c15f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.262+0000 7f2b89400640 1 -- 172.21.15.86:0/2492257105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b8c0ad7f0 msgr2=0x7f2b8c1f4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:59.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.262+0000 7f2b89400640 1 --2- 172.21.15.86:0/2492257105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b8c0ad7f0 0x7f2b8c1f4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.262+0000 7f2b89400640 1 -- 172.21.15.86:0/2492257105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b8c0af6b0 con 0x7f2b8c156ae0 2024-04-20T21:25:59.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.263+0000 7f2b89400640 1 --2- 172.21.15.86:0/2492257105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b8c156ae0 0x7f2b8c15ec80 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f2b7400c3c0 tx=0x7f2b7400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:59.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.263+0000 7f2b82000640 1 -- 172.21.15.86:0/2492257105 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b740074e0 con 0x7f2b8c156ae0 2024-04-20T21:25:59.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.263+0000 7f2b82000640 1 -- 172.21.15.86:0/2492257105 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b74007680 con 0x7f2b8c156ae0 2024-04-20T21:25:59.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.263+0000 7f2b82000640 1 -- 172.21.15.86:0/2492257105 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b74004a00 con 0x7f2b8c156ae0 2024-04-20T21:25:59.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.263+0000 7f2b8b200640 1 -- 172.21.15.86:0/2492257105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b8c145c80 con 0x7f2b8c156ae0 2024-04-20T21:25:59.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.263+0000 7f2b8b200640 1 -- 172.21.15.86:0/2492257105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b8c157570 con 0x7f2b8c156ae0 2024-04-20T21:25:59.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.264+0000 7f2b8b200640 1 -- 172.21.15.86:0/2492257105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b8c04fcf0 con 0x7f2b8c156ae0 2024-04-20T21:25:59.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.265+0000 7f2b82000640 1 -- 172.21.15.86:0/2492257105 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2b7401a020 con 0x7f2b8c156ae0 2024-04-20T21:25:59.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.266+0000 7f2b82000640 1 --2- 172.21.15.86:0/2492257105 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2b6806c8f0 0x7f2b6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:59.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.266+0000 7f2b82000640 1 -- 172.21.15.86:0/2492257105 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7f2b7408b370 con 0x7f2b8c156ae0 2024-04-20T21:25:59.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.266+0000 7f2b89e00640 1 --2- 172.21.15.86:0/2492257105 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2b6806c8f0 0x7f2b6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:59.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.267+0000 7f2b89e00640 1 --2- 172.21.15.86:0/2492257105 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2b6806c8f0 0x7f2b6806edb0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c036000 tx=0x7f2b7c03b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:59.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.271+0000 7f2b82000640 1 -- 172.21.15.86:0/2492257105 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2b8c0af6b0 con 0x7f2b8c156ae0 2024-04-20T21:25:59.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.513+0000 7f2b8b200640 1 -- 172.21.15.86:0/2492257105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b8c071c10 con 0x7f2b8c156ae0 2024-04-20T21:25:59.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.514+0000 7f2b82000640 1 -- 172.21.15.86:0/2492257105 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5208 (secure 0 0 0) 0x7f2b74059740 con 0x7f2b8c156ae0 2024-04-20T21:25:59.513 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:25:59.513 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":66,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7506,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/4013863257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4013863257},{"type":"v1","addr":"172.21.15.86:6836","nonce":4013863257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":7509,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2044883073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2044883073},{"type":"v1","addr":"172.21.15.86:6837","nonce":2044883073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7518,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/2351381466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2351381466},{"type":"v1","addr":"172.21.15.144:6834","nonce":2351381466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:24:58.299257+0000","modified":"2024-04-20T21:25:31.064401+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7773},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7773":{"gid":7773,"name":"b","rank":0,"incarnation":62,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6835/3981403733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3981403733},{"type":"v1","addr":"172.21.15.144:6835","nonce":3981403733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7773,"qdb_cluster":[7773]},"id":6}]} 2024-04-20T21:25:59.513 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 66 2024-04-20T21:25:59.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.516+0000 7f2b8b200640 1 -- 172.21.15.86:0/2492257105 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2b6806c8f0 msgr2=0x7f2b6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:59.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.516+0000 7f2b8b200640 1 --2- 172.21.15.86:0/2492257105 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2b6806c8f0 0x7f2b6806edb0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c036000 tx=0x7f2b7c03b040 comp rx=0 tx=0).stop 2024-04-20T21:25:59.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.516+0000 7f2b8b200640 1 -- 172.21.15.86:0/2492257105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b8c156ae0 msgr2=0x7f2b8c15ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:59.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.516+0000 7f2b8b200640 1 --2- 172.21.15.86:0/2492257105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b8c156ae0 0x7f2b8c15ec80 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f2b7400c3c0 tx=0x7f2b7400c880 comp rx=0 tx=0).stop 2024-04-20T21:25:59.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.517+0000 7f2b8b200640 1 -- 172.21.15.86:0/2492257105 shutdown_connections 2024-04-20T21:25:59.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.517+0000 7f2b8b200640 1 --2- 172.21.15.86:0/2492257105 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2b6806c8f0 0x7f2b6806edb0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.517+0000 7f2b8b200640 1 --2- 172.21.15.86:0/2492257105 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2b8c160080 0x7f2b8c15f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.517+0000 7f2b8b200640 1 --2- 172.21.15.86:0/2492257105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2b8c156ae0 0x7f2b8c15ec80 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.517+0000 7f2b8b200640 1 --2- 172.21.15.86:0/2492257105 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b8c0ad7f0 0x7f2b8c1f4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.517+0000 7f2b8b200640 1 -- 172.21.15.86:0/2492257105 >> 172.21.15.86:0/2492257105 conn(0x7f2b8c0d0170 msgr2=0x7f2b8c1551f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:59.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.517+0000 7f2b8b200640 1 -- 172.21.15.86:0/2492257105 shutdown_connections 2024-04-20T21:25:59.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.517+0000 7f2b8b200640 1 -- 172.21.15.86:0/2492257105 wait complete. 2024-04-20T21:25:59.535 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b damage ls --format=json-pretty 2024-04-20T21:25:59.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.732+0000 7fd12ae00640 1 Processor -- start 2024-04-20T21:25:59.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.732+0000 7fd12ae00640 1 -- start start 2024-04-20T21:25:59.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.734+0000 7fd12ae00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1240b27b0 0x7fd12406b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:59.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.734+0000 7fd12ae00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd1240b0b90 0x7fd1240b1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:59.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.734+0000 7fd12ae00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd12406bfd0 con 0x7fd1240cd540 2024-04-20T21:25:59.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.734+0000 7fd12ae00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd12406c590 con 0x7fd1240b0b90 2024-04-20T21:25:59.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.734+0000 7fd12ae00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd1240ce0e0 con 0x7fd1240b27b0 2024-04-20T21:25:59.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.734+0000 7fd129000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd1240cd540 0x7fd1240ce570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:50936/0 (socket says 172.21.15.86:50936) 2024-04-20T21:25:59.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.734+0000 7fd129000640 1 -- 172.21.15.86:0/841362225 learned_addr learned my addr 172.21.15.86:0/841362225 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:25:59.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.735+0000 7fd129a00640 1 --2- 172.21.15.86:0/841362225 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1240b27b0 0x7fd12406b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:59.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.735+0000 7fd12a400640 1 --2- 172.21.15.86:0/841362225 >> v2:172.21.15.86:3300/0 conn(0x7fd1240b0b90 0x7fd1240b1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:59.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.735+0000 7fd12a400640 1 -- 172.21.15.86:0/841362225 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1240b27b0 msgr2=0x7fd12406b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:59.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.735+0000 7fd12a400640 1 --2- 172.21.15.86:0/841362225 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1240b27b0 0x7fd12406b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.735+0000 7fd12a400640 1 -- 172.21.15.86:0/841362225 >> v1:172.21.15.86:6789/0 conn(0x7fd1240cd540 legacy=0x7fd1240ce570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:59.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.736+0000 7fd12a400640 1 -- 172.21.15.86:0/841362225 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd12406c2b0 con 0x7fd1240b0b90 2024-04-20T21:25:59.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.736+0000 7fd129a00640 1 --2- 172.21.15.86:0/841362225 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1240b27b0 0x7fd12406b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:25:59.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.738+0000 7fd12a400640 1 --2- 172.21.15.86:0/841362225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1240b0b90 0x7fd1240b1020 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fd11c009d90 tx=0x7fd11c02be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0a61faed23f1c19 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:59.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.738+0000 7fd123e00640 1 -- 172.21.15.86:0/841362225 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11c039070 con 0x7fd1240b0b90 2024-04-20T21:25:59.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.738+0000 7fd123e00640 1 -- 172.21.15.86:0/841362225 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd11c02cbc0 con 0x7fd1240b0b90 2024-04-20T21:25:59.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.738+0000 7fd123e00640 1 -- 172.21.15.86:0/841362225 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11c034440 con 0x7fd1240b0b90 2024-04-20T21:25:59.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.739+0000 7fd12ae00640 1 -- 172.21.15.86:0/841362225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1240b0b90 msgr2=0x7fd1240b1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:59.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.739+0000 7fd12ae00640 1 --2- 172.21.15.86:0/841362225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1240b0b90 0x7fd1240b1020 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fd11c009d90 tx=0x7fd11c02be10 comp rx=0 tx=0).stop 2024-04-20T21:25:59.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.739+0000 7fd12ae00640 1 -- 172.21.15.86:0/841362225 shutdown_connections 2024-04-20T21:25:59.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.739+0000 7fd12ae00640 1 --2- 172.21.15.86:0/841362225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1240b0b90 0x7fd1240b1020 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.739+0000 7fd12ae00640 1 --2- 172.21.15.86:0/841362225 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1240b27b0 0x7fd12406b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.739+0000 7fd12ae00640 1 -- 172.21.15.86:0/841362225 >> 172.21.15.86:0/841362225 conn(0x7fd1240b2f60 msgr2=0x7fd1240b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:59.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.739+0000 7fd12ae00640 1 -- 172.21.15.86:0/841362225 shutdown_connections 2024-04-20T21:25:59.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.739+0000 7fd12ae00640 1 -- 172.21.15.86:0/841362225 wait complete. 2024-04-20T21:25:59.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.742+0000 7fd12ae00640 1 Processor -- start 2024-04-20T21:25:59.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.742+0000 7fd12ae00640 1 -- start start 2024-04-20T21:25:59.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.743+0000 7fd12ae00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1240b0b90 0x7fd1241f5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:59.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.743+0000 7fd12ae00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1240b27b0 0x7fd1240ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:59.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.743+0000 7fd12ae00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1240cd540 0x7fd1241f7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:59.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.743+0000 7fd12ae00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd1240b1560 con 0x7fd1240cd540 2024-04-20T21:25:59.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.743+0000 7fd12ae00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd1240cec80 con 0x7fd1240b27b0 2024-04-20T21:25:59.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.743+0000 7fd12ae00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd1240cd9a0 con 0x7fd1240b0b90 2024-04-20T21:25:59.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.743+0000 7fd12a400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1240cd540 0x7fd1241f7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:59.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.744+0000 7fd12a400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1240cd540 0x7fd1241f7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42940/0 (socket says 172.21.15.86:42940) 2024-04-20T21:25:59.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.744+0000 7fd12a400640 1 -- 172.21.15.86:0/2036960330 learned_addr learned my addr 172.21.15.86:0/2036960330 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:59.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.744+0000 7fd129a00640 1 --2- 172.21.15.86:0/2036960330 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1240b0b90 0x7fd1241f5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:59.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.744+0000 7fd129000640 1 --2- 172.21.15.86:0/2036960330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1240b27b0 0x7fd1240ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:59.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.744+0000 7fd12a400640 1 -- 172.21.15.86:0/2036960330 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1240b0b90 msgr2=0x7fd1241f5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:59.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.744+0000 7fd12a400640 1 --2- 172.21.15.86:0/2036960330 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1240b0b90 0x7fd1241f5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.744+0000 7fd12a400640 1 -- 172.21.15.86:0/2036960330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1240b27b0 msgr2=0x7fd1240ae4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:59.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.744+0000 7fd12a400640 1 --2- 172.21.15.86:0/2036960330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1240b27b0 0x7fd1240ae4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.744+0000 7fd12a400640 1 -- 172.21.15.86:0/2036960330 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1241f45c0 con 0x7fd1240cd540 2024-04-20T21:25:59.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.744+0000 7fd129000640 1 --2- 172.21.15.86:0/2036960330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1240b27b0 0x7fd1240ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:25:59.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.744+0000 7fd12a400640 1 --2- 172.21.15.86:0/2036960330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1240cd540 0x7fd1241f7bb0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fd11c009d90 tx=0x7fd11c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:59.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.745+0000 7fd121600640 1 -- 172.21.15.86:0/2036960330 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11c039050 con 0x7fd1240cd540 2024-04-20T21:25:59.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.745+0000 7fd121600640 1 -- 172.21.15.86:0/2036960330 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd11c045070 con 0x7fd1240cd540 2024-04-20T21:25:59.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.745+0000 7fd121600640 1 -- 172.21.15.86:0/2036960330 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11c009450 con 0x7fd1240cd540 2024-04-20T21:25:59.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.745+0000 7fd12ae00640 1 -- 172.21.15.86:0/2036960330 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1241f5580 con 0x7fd1240cd540 2024-04-20T21:25:59.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.745+0000 7fd12ae00640 1 -- 172.21.15.86:0/2036960330 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1241f8930 con 0x7fd1240cd540 2024-04-20T21:25:59.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.748+0000 7fd121600640 1 -- 172.21.15.86:0/2036960330 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd11c03e850 con 0x7fd1240cd540 2024-04-20T21:25:59.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.749+0000 7fd12f1b7740 1 Processor -- start 2024-04-20T21:25:59.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.749+0000 7fd121600640 1 --2- 172.21.15.86:0/2036960330 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd10406c8f0 0x7fd10406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:59.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.749+0000 7fd12f1b7740 1 -- start start 2024-04-20T21:25:59.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.749+0000 7fd121600640 1 -- 172.21.15.86:0/2036960330 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7fd11c0ae8a0 con 0x7fd1240cd540 2024-04-20T21:25:59.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.749+0000 7fd129000640 1 --2- 172.21.15.86:0/2036960330 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd10406c8f0 0x7fd10406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:59.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.750+0000 7fd129000640 1 --2- 172.21.15.86:0/2036960330 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd10406c8f0 0x7fd10406edb0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd1240cb9b0 tx=0x7fd110009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:59.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.750+0000 7fd12f1b7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a72021a0b0 0x55a7201ed050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:59.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.750+0000 7fd12f1b7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a72021a490 0x55a72021ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:59.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.750+0000 7fd12f1b7740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a7201ed590 0x55a72021d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:59.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.750+0000 7fd12f1b7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a720158060 con 0x55a72021a490 2024-04-20T21:25:59.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.750+0000 7fd12f1b7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a7200fdf00 con 0x55a72021a0b0 2024-04-20T21:25:59.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.750+0000 7fd12f1b7740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55a720154020 con 0x55a7201ed590 2024-04-20T21:25:59.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.750+0000 7fd129000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a72021a490 0x55a72021ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:59.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.750+0000 7fd129000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a72021a490 0x55a72021ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42956/0 (socket says 172.21.15.86:42956) 2024-04-20T21:25:59.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.750+0000 7fd129000640 1 -- 172.21.15.86:0/1310452417 learned_addr learned my addr 172.21.15.86:0/1310452417 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:59.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.751+0000 7fd129a00640 1 --2- 172.21.15.86:0/1310452417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a72021a0b0 0x55a7201ed050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:59.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.751+0000 7fd12a400640 1 --2- 172.21.15.86:0/1310452417 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a7201ed590 0x55a72021d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:59.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.751+0000 7fd129000640 1 -- 172.21.15.86:0/1310452417 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a7201ed590 msgr2=0x55a72021d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:59.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.751+0000 7fd129000640 1 --2- 172.21.15.86:0/1310452417 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a7201ed590 0x55a72021d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.751+0000 7fd129000640 1 -- 172.21.15.86:0/1310452417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a72021a0b0 msgr2=0x55a7201ed050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:59.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.751+0000 7fd129000640 1 --2- 172.21.15.86:0/1310452417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a72021a0b0 0x55a7201ed050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.751+0000 7fd129000640 1 -- 172.21.15.86:0/1310452417 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7200e6a30 con 0x55a72021a490 2024-04-20T21:25:59.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.751+0000 7fd129000640 1 --2- 172.21.15.86:0/1310452417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a72021a490 0x55a72021ce40 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fd110005890 tx=0x7fd110011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6ff1b5df0c2d2aa server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:59.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.751+0000 7fd116000640 1 -- 172.21.15.86:0/1310452417 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd110007710 con 0x55a72021a490 2024-04-20T21:25:59.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.752+0000 7fd116000640 1 -- 172.21.15.86:0/1310452417 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd11000e7b0 con 0x55a72021a490 2024-04-20T21:25:59.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.752+0000 7fd116000640 1 -- 172.21.15.86:0/1310452417 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11000eb60 con 0x55a72021a490 2024-04-20T21:25:59.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.752+0000 7fd12f1b7740 1 -- 172.21.15.86:0/1310452417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a72021a490 msgr2=0x55a72021ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:59.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.752+0000 7fd12f1b7740 1 --2- 172.21.15.86:0/1310452417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a72021a490 0x55a72021ce40 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fd110005890 tx=0x7fd110011000 comp rx=0 tx=0).stop 2024-04-20T21:25:59.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.752+0000 7fd12f1b7740 1 -- 172.21.15.86:0/1310452417 shutdown_connections 2024-04-20T21:25:59.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.752+0000 7fd12f1b7740 1 --2- 172.21.15.86:0/1310452417 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a7201ed590 0x55a72021d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.752+0000 7fd12f1b7740 1 --2- 172.21.15.86:0/1310452417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a72021a490 0x55a72021ce40 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.752+0000 7fd12f1b7740 1 --2- 172.21.15.86:0/1310452417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a72021a0b0 0x55a7201ed050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.752+0000 7fd12f1b7740 1 -- 172.21.15.86:0/1310452417 >> 172.21.15.86:0/1310452417 conn(0x55a7201ef2c0 msgr2=0x55a720237e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:59.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.753+0000 7fd12f1b7740 1 -- 172.21.15.86:0/1310452417 shutdown_connections 2024-04-20T21:25:59.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.753+0000 7fd12f1b7740 1 -- 172.21.15.86:0/1310452417 wait complete. 2024-04-20T21:25:59.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.753+0000 7fd12f1b7740 1 -- start start 2024-04-20T21:25:59.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.753+0000 7fd12f1b7740 1 Processor -- start 2024-04-20T21:25:59.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.754+0000 7fd12f1b7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a72021a0b0 0x55a720232470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:59.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.754+0000 7fd12f1b7740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a72021a490 0x55a7202329b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:59.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.754+0000 7fd12f1b7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a72021f540 0x55a72021f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:59.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.754+0000 7fd12f1b7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a7201f1860 con 0x55a72021f540 2024-04-20T21:25:59.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.755+0000 7fd12f1b7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a7201113d0 con 0x55a72021a0b0 2024-04-20T21:25:59.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.755+0000 7fd12f1b7740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55a7201f0c50 con 0x55a72021a490 2024-04-20T21:25:59.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.755+0000 7fd12f1b7740 10 client.0 _ll_register_callbacks cb 0x55a7200df960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:25:59.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.755+0000 7fd12a400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a72021f540 0x55a72021f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:59.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.755+0000 7fd12a400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a72021f540 0x55a72021f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42960/0 (socket says 172.21.15.86:42960) 2024-04-20T21:25:59.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.755+0000 7fd12a400640 1 -- 172.21.15.86:0/2372433358 learned_addr learned my addr 172.21.15.86:0/2372433358 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:59.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.755+0000 7fd129a00640 1 --2- 172.21.15.86:0/2372433358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a72021a0b0 0x55a720232470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:59.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.755+0000 7fd12a400640 1 -- 172.21.15.86:0/2372433358 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a72021a490 msgr2=0x55a7202329b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:59.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.755+0000 7fd129000640 1 --2- 172.21.15.86:0/2372433358 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a72021a490 0x55a7202329b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:59.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.755+0000 7fd12a400640 1 --2- 172.21.15.86:0/2372433358 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a72021a490 0x55a7202329b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.755+0000 7fd12a400640 1 -- 172.21.15.86:0/2372433358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a72021a0b0 msgr2=0x55a720232470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:59.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.755+0000 7fd12a400640 1 --2- 172.21.15.86:0/2372433358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a72021a0b0 0x55a720232470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.755+0000 7fd12a400640 1 -- 172.21.15.86:0/2372433358 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a71ff367c0 con 0x55a72021f540 2024-04-20T21:25:59.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.755+0000 7fd129a00640 1 --2- 172.21.15.86:0/2372433358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a72021a0b0 0x55a720232470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:25:59.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.756+0000 7fd12a400640 1 --2- 172.21.15.86:0/2372433358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a72021f540 0x55a72021f9c0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fd11c0acc80 tx=0x7fd11c02cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42f2204bc0854397 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:59.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.756+0000 7fd10aa00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:25:59.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.756+0000 7fd10aa00640 1 -- 172.21.15.86:0/2372433358 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11c0ab060 con 0x55a72021f540 2024-04-20T21:25:59.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.756+0000 7fd10aa00640 1 -- 172.21.15.86:0/2372433358 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7fd11c084f50 con 0x55a72021f540 2024-04-20T21:25:59.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.756+0000 7fd12f1b7740 1 -- 172.21.15.86:0/2372433358 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd11000e7b0 con 0x55a72021f540 2024-04-20T21:25:59.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.756+0000 7fd10aa00640 1 -- 172.21.15.86:0/2372433358 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fd11c08a050 con 0x55a72021f540 2024-04-20T21:25:59.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.756+0000 7fd12f1b7740 10 client.7929 fetch_fsmap learned FSMap version 66 2024-04-20T21:25:59.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.756+0000 7fd12f1b7740 1 -- 172.21.15.86:0/2372433358 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x55a71ff96f50 con 0x55a72021f540 2024-04-20T21:25:59.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.756+0000 7fd10aa00640 1 -- 172.21.15.86:0/2372433358 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd11000e7b0 con 0x55a72021f540 2024-04-20T21:25:59.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.756+0000 7fd10aa00640 1 -- 172.21.15.86:0/2372433358 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 66) ==== 3495+0+0 (secure 0 0 0) 0x7fd11c08d020 con 0x55a72021f540 2024-04-20T21:25:59.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.757+0000 7fd12f1b7740 10 client.7929 fetch_fsmap finished waiting for FSMap version 66 2024-04-20T21:25:59.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.757+0000 7fd12f1b7740 10 client.7929 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-20T21:25:59.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.757+0000 7fd12f1b7740 20 client.7929 populate_metadata read hostname 'smithi086' 2024-04-20T21:25:59.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.757+0000 7fd12f1b7740 1 --2- 172.21.15.86:0/2372433358 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x55a72020cfa0 0x55a72020d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:59.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.757+0000 7fd12f1b7740 4 client.7929 mds_command: new command op to 7773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:25:59.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.757+0000 7fd12f1b7740 1 -- 172.21.15.86:0/2372433358 --> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a720165a80 con 0x55a72020cfa0 2024-04-20T21:25:59.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.757+0000 7fd129a00640 1 --2- 172.21.15.86:0/2372433358 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x55a72020cfa0 0x55a72020d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:59.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.758+0000 7fd129a00640 1 --2- 172.21.15.86:0/2372433358 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x55a72020cfa0 0x55a72020d380 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10a9231fddddd9fe server_cookie=18852561389e0d7d in_seq=0 out_seq=0 2024-04-20T21:25:59.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.758+0000 7fd10aa00640 10 client.7929 ms_handle_connect on v2:172.21.15.144:6833/3981403733 2024-04-20T21:25:59.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.760+0000 7fd10aa00640 1 -- 172.21.15.86:0/2372433358 <== mds.0 v2:172.21.15.144:6833/3981403733 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a720165a80 con 0x55a72020cfa0 2024-04-20T21:25:59.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.760+0000 7fd10aa00640 10 client.7929 handle_command_reply: tid=0 2024-04-20T21:25:59.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.761+0000 7fd12f1b7740 1 client.7929 shutdown 2024-04-20T21:25:59.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.761+0000 7fd12f1b7740 1 -- 172.21.15.86:0/2372433358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a72021f540 msgr2=0x55a72021f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:59.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.761+0000 7fd12f1b7740 1 --2- 172.21.15.86:0/2372433358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a72021f540 0x55a72021f9c0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fd11c0acc80 tx=0x7fd11c02cc80 comp rx=0 tx=0).stop 2024-04-20T21:25:59.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.761+0000 7fd12f1b7740 1 -- 172.21.15.86:0/2372433358 shutdown_connections 2024-04-20T21:25:59.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.761+0000 7fd12f1b7740 1 --2- 172.21.15.86:0/2372433358 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x55a72020cfa0 0x55a72020d380 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.761+0000 7fd12f1b7740 1 --2- 172.21.15.86:0/2372433358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a72021f540 0x55a72021f9c0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.761+0000 7fd12f1b7740 1 --2- 172.21.15.86:0/2372433358 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a72021a490 0x55a7202329b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.761+0000 7fd12f1b7740 1 --2- 172.21.15.86:0/2372433358 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a72021a0b0 0x55a720232470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.761+0000 7fd10aa00640 0 client.7929 ms_handle_reset on v2:172.21.15.144:6833/3981403733 2024-04-20T21:25:59.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.761+0000 7fd12f1b7740 1 -- 172.21.15.86:0/2372433358 >> 172.21.15.86:0/2372433358 conn(0x55a7201ed590 msgr2=0x55a72021ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:59.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.761+0000 7fd12f1b7740 1 -- 172.21.15.86:0/2372433358 shutdown_connections 2024-04-20T21:25:59.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.761+0000 7fd12f1b7740 1 -- 172.21.15.86:0/2372433358 wait complete. 2024-04-20T21:25:59.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.761+0000 7fd12f1b7740 20 client.7929 trim_cache size 0 max 16384 2024-04-20T21:25:59.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.776+0000 7fd12f1b7740 1 Processor -- start 2024-04-20T21:25:59.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.776+0000 7fd12f1b7740 1 -- start start 2024-04-20T21:25:59.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.777+0000 7fd12f1b7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a72021a0b0 0x55a720238a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:59.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.777+0000 7fd12f1b7740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a72021a490 0x55a72023b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:59.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.777+0000 7fd12f1b7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a72021f540 0x55a72023b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:59.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.777+0000 7fd12f1b7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd1240c9230 con 0x55a72021a0b0 2024-04-20T21:25:59.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.777+0000 7fd12f1b7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd1240c8590 con 0x55a72021f540 2024-04-20T21:25:59.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.777+0000 7fd12f1b7740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd1240c8be0 con 0x55a72021a490 2024-04-20T21:25:59.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.777+0000 7fd129a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a72021a0b0 0x55a720238a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:59.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.777+0000 7fd129a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a72021a0b0 0x55a720238a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42974/0 (socket says 172.21.15.86:42974) 2024-04-20T21:25:59.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.777+0000 7fd129a00640 1 -- 172.21.15.86:0/1221053536 learned_addr learned my addr 172.21.15.86:0/1221053536 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:59.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.778+0000 7fd129a00640 1 -- 172.21.15.86:0/1221053536 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a72021a490 msgr2=0x55a72023b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:59.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.778+0000 7fd129a00640 1 --2- 172.21.15.86:0/1221053536 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a72021a490 0x55a72023b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.778+0000 7fd129a00640 1 -- 172.21.15.86:0/1221053536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a72021f540 msgr2=0x55a72023b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:59.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.778+0000 7fd129a00640 1 --2- 172.21.15.86:0/1221053536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a72021f540 0x55a72023b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.778+0000 7fd129a00640 1 -- 172.21.15.86:0/1221053536 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a71fe2a0b0 con 0x55a72021a0b0 2024-04-20T21:25:59.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.778+0000 7fd129a00640 1 --2- 172.21.15.86:0/1221053536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a72021a0b0 0x55a720238a10 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x55a72024c4f0 tx=0x7fd118008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30760279656198a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:59.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.778+0000 7fd115600640 1 -- 172.21.15.86:0/1221053536 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd118025070 con 0x55a72021a0b0 2024-04-20T21:25:59.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.778+0000 7fd115600640 1 -- 172.21.15.86:0/1221053536 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd118017bb0 con 0x55a72021a0b0 2024-04-20T21:25:59.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.778+0000 7fd115600640 1 -- 172.21.15.86:0/1221053536 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd118004030 con 0x55a72021a0b0 2024-04-20T21:25:59.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.779+0000 7fd12f1b7740 1 -- 172.21.15.86:0/1221053536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a72021a0b0 msgr2=0x55a720238a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:59.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.779+0000 7fd12f1b7740 1 --2- 172.21.15.86:0/1221053536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a72021a0b0 0x55a720238a10 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x55a72024c4f0 tx=0x7fd118008040 comp rx=0 tx=0).stop 2024-04-20T21:25:59.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.779+0000 7fd12f1b7740 1 -- 172.21.15.86:0/1221053536 shutdown_connections 2024-04-20T21:25:59.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.779+0000 7fd12f1b7740 1 --2- 172.21.15.86:0/1221053536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a72021f540 0x55a72023b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.779+0000 7fd12f1b7740 1 --2- 172.21.15.86:0/1221053536 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a72021a490 0x55a72023b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.779+0000 7fd12f1b7740 1 --2- 172.21.15.86:0/1221053536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a72021a0b0 0x55a720238a10 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.779+0000 7fd12f1b7740 1 -- 172.21.15.86:0/1221053536 >> 172.21.15.86:0/1221053536 conn(0x55a7201ed590 msgr2=0x55a72022eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:59.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.779+0000 7fd12f1b7740 1 -- 172.21.15.86:0/1221053536 shutdown_connections 2024-04-20T21:25:59.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.779+0000 7fd12f1b7740 1 -- 172.21.15.86:0/1221053536 wait complete. 2024-04-20T21:25:59.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.780+0000 7fd12f1b7740 1 -- start start 2024-04-20T21:25:59.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.780+0000 7fd12f1b7740 1 Processor -- start 2024-04-20T21:25:59.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.781+0000 7fd12f1b7740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a72021a0b0 0x55a72020dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:59.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.781+0000 7fd12f1b7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a72021a490 0x55a72021dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:59.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.781+0000 7fd12f1b7740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a72021f540 0x55a72023b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:59.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.781+0000 7fd12f1b7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a72020c4a0 con 0x55a72021a490 2024-04-20T21:25:59.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.781+0000 7fd12f1b7740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a72020c120 con 0x55a72021f540 2024-04-20T21:25:59.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.781+0000 7fd12f1b7740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55a72024bd20 con 0x55a72021a0b0 2024-04-20T21:25:59.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.781+0000 7fd12f1b7740 10 client.0 _ll_register_callbacks cb 0x7fd11c07c1d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:25:59.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.781+0000 7fd129000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a72021a490 0x55a72021dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:59.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.781+0000 7fd129000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a72021a490 0x55a72021dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42980/0 (socket says 172.21.15.86:42980) 2024-04-20T21:25:59.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.781+0000 7fd129000640 1 -- 172.21.15.86:0/327673302 learned_addr learned my addr 172.21.15.86:0/327673302 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:25:59.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.782+0000 7fd12a400640 1 --2- 172.21.15.86:0/327673302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a72021f540 0x55a72023b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:59.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.782+0000 7fd129a00640 1 --2- 172.21.15.86:0/327673302 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a72021a0b0 0x55a72020dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:59.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.782+0000 7fd129000640 1 -- 172.21.15.86:0/327673302 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a72021a0b0 msgr2=0x55a72020dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:59.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.782+0000 7fd129000640 1 --2- 172.21.15.86:0/327673302 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a72021a0b0 0x55a72020dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.782+0000 7fd129000640 1 -- 172.21.15.86:0/327673302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a72021f540 msgr2=0x55a72023b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:25:59.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.782+0000 7fd129000640 1 --2- 172.21.15.86:0/327673302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a72021f540 0x55a72023b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.782+0000 7fd129000640 1 -- 172.21.15.86:0/327673302 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a72029ab10 con 0x55a72021a490 2024-04-20T21:25:59.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.782+0000 7fd129000640 1 --2- 172.21.15.86:0/327673302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a72021a490 0x55a72021dbf0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fd110000f60 tx=0x7fd110000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dc28a4d270309ad server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:25:59.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.782+0000 7fd10aa00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:25:59.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.782+0000 7fd10aa00640 1 -- 172.21.15.86:0/327673302 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11002f060 con 0x55a72021a490 2024-04-20T21:25:59.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.782+0000 7fd10aa00640 1 -- 172.21.15.86:0/327673302 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7fd11000ebf0 con 0x55a72021a490 2024-04-20T21:25:59.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.782+0000 7fd12f1b7740 1 -- 172.21.15.86:0/327673302 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd118017bb0 con 0x55a72021a490 2024-04-20T21:25:59.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.783+0000 7fd10aa00640 1 -- 172.21.15.86:0/327673302 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fd110035050 con 0x55a72021a490 2024-04-20T21:25:59.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.783+0000 7fd12f1b7740 10 client.7935 fetch_fsmap learned FSMap version 66 2024-04-20T21:25:59.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.783+0000 7fd12f1b7740 1 -- 172.21.15.86:0/327673302 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x55a72029a610 con 0x55a72021a490 2024-04-20T21:25:59.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.783+0000 7fd10aa00640 1 -- 172.21.15.86:0/327673302 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd118017bb0 con 0x55a72021a490 2024-04-20T21:25:59.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.783+0000 7fd10aa00640 1 -- 172.21.15.86:0/327673302 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 66) ==== 3495+0+0 (secure 0 0 0) 0x7fd110038020 con 0x55a72021a490 2024-04-20T21:25:59.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.783+0000 7fd12f1b7740 10 client.7935 fetch_fsmap finished waiting for FSMap version 66 2024-04-20T21:25:59.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.783+0000 7fd12f1b7740 10 client.7935 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-20T21:25:59.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.783+0000 7fd12f1b7740 20 client.7935 populate_metadata read hostname 'smithi086' 2024-04-20T21:25:59.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.783+0000 7fd12f1b7740 1 --2- 172.21.15.86:0/327673302 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x55a72023ba20 0x55a720246510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:25:59.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.783+0000 7fd12f1b7740 4 client.7935 mds_command: new command op to 7773 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-20T21:25:59.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.783+0000 7fd12f1b7740 1 -- 172.21.15.86:0/327673302 --> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55a72015f8f0 con 0x55a72023ba20 2024-04-20T21:25:59.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.784+0000 7fd129a00640 1 --2- 172.21.15.86:0/327673302 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x55a72023ba20 0x55a720246510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:25:59.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.785+0000 7fd129a00640 1 --2- 172.21.15.86:0/327673302 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x55a72023ba20 0x55a720246510 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7e1f3d547198606 server_cookie=898f938f3c052cfb in_seq=0 out_seq=0 2024-04-20T21:25:59.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.785+0000 7fd10aa00640 10 client.7935 ms_handle_connect on v2:172.21.15.144:6833/3981403733 2024-04-20T21:25:59.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.785+0000 7fd10aa00640 1 -- 172.21.15.86:0/327673302 <== mds.0 v2:172.21.15.144:6833/3981403733 1 ==== command_reply(tid 0: 0 ) ==== 8+0+420 (crc 0 0 0) 0x55a72015f8f0 con 0x55a72023ba20 2024-04-20T21:25:59.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.785+0000 7fd10aa00640 10 client.7935 handle_command_reply: tid=0 2024-04-20T21:25:59.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.785+0000 7fd12f1b7740 1 client.7935 shutdown 2024-04-20T21:25:59.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.786+0000 7fd12f1b7740 1 -- 172.21.15.86:0/327673302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a72021a490 msgr2=0x55a72021dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:59.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.786+0000 7fd12f1b7740 1 --2- 172.21.15.86:0/327673302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a72021a490 0x55a72021dbf0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fd110000f60 tx=0x7fd110000ed0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.786+0000 7fd12f1b7740 1 -- 172.21.15.86:0/327673302 shutdown_connections 2024-04-20T21:25:59.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.786+0000 7fd12f1b7740 1 --2- 172.21.15.86:0/327673302 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x55a72023ba20 0x55a720246510 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.786+0000 7fd12f1b7740 1 --2- 172.21.15.86:0/327673302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a72021f540 0x55a72023b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.786+0000 7fd12f1b7740 1 --2- 172.21.15.86:0/327673302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a72021a490 0x55a72021dbf0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.786+0000 7fd12f1b7740 1 --2- 172.21.15.86:0/327673302 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a72021a0b0 0x55a72020dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.786+0000 7fd10aa00640 0 client.7935 ms_handle_reset on v2:172.21.15.144:6833/3981403733 2024-04-20T21:25:59.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.786+0000 7fd12f1b7740 1 -- 172.21.15.86:0/327673302 >> 172.21.15.86:0/327673302 conn(0x55a7201ed590 msgr2=0x55a720218e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:59.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.786+0000 7fd12f1b7740 1 -- 172.21.15.86:0/327673302 shutdown_connections 2024-04-20T21:25:59.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.786+0000 7fd12f1b7740 1 -- 172.21.15.86:0/327673302 wait complete. 2024-04-20T21:25:59.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.786+0000 7fd12f1b7740 20 client.7935 trim_cache size 0 max 16384 2024-04-20T21:25:59.789 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:25:59.789 INFO:teuthology.orchestra.run.smithi086.stdout:[ 2024-04-20T21:25:59.789 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:25:59.790 INFO:teuthology.orchestra.run.smithi086.stdout: "damage_type": "dir_frag", 2024-04-20T21:25:59.790 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 1317037189, 2024-04-20T21:25:59.790 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 1099511627776, 2024-04-20T21:25:59.790 INFO:teuthology.orchestra.run.smithi086.stdout: "frag": "*", 2024-04-20T21:25:59.790 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/dir" 2024-04-20T21:25:59.790 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:25:59.790 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:25:59.790 INFO:teuthology.orchestra.run.smithi086.stdout: "damage_type": "backtrace", 2024-04-20T21:25:59.790 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 2022046859, 2024-04-20T21:25:59.790 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 1099511627776, 2024-04-20T21:25:59.790 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/dir" 2024-04-20T21:25:59.790 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:25:59.790 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:25:59.790 INFO:teuthology.orchestra.run.smithi086.stdout: "damage_type": "backtrace", 2024-04-20T21:25:59.791 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 4026297729, 2024-04-20T21:25:59.791 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 1099511627777, 2024-04-20T21:25:59.791 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/testdir/hardlink" 2024-04-20T21:25:59.791 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:25:59.791 INFO:teuthology.orchestra.run.smithi086.stdout:] 2024-04-20T21:25:59.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.787+0000 7fd12ae00640 1 -- 172.21.15.86:0/2036960330 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd10406c8f0 msgr2=0x7fd10406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:59.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.787+0000 7fd12ae00640 1 --2- 172.21.15.86:0/2036960330 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd10406c8f0 0x7fd10406edb0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd1240cb9b0 tx=0x7fd110009040 comp rx=0 tx=0).stop 2024-04-20T21:25:59.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.787+0000 7fd12ae00640 1 -- 172.21.15.86:0/2036960330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1240cd540 msgr2=0x7fd1241f7bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:25:59.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.787+0000 7fd12ae00640 1 --2- 172.21.15.86:0/2036960330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1240cd540 0x7fd1241f7bb0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fd11c009d90 tx=0x7fd11c004a40 comp rx=0 tx=0).stop 2024-04-20T21:25:59.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.787+0000 7fd12ae00640 1 -- 172.21.15.86:0/2036960330 shutdown_connections 2024-04-20T21:25:59.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.787+0000 7fd12ae00640 1 --2- 172.21.15.86:0/2036960330 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd10406c8f0 0x7fd10406edb0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.787+0000 7fd12ae00640 1 --2- 172.21.15.86:0/2036960330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1240cd540 0x7fd1241f7bb0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.787+0000 7fd12ae00640 1 --2- 172.21.15.86:0/2036960330 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1240b27b0 0x7fd1240ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.787+0000 7fd12ae00640 1 --2- 172.21.15.86:0/2036960330 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd1240b0b90 0x7fd1241f5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:25:59.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.787+0000 7fd12ae00640 1 -- 172.21.15.86:0/2036960330 >> 172.21.15.86:0/2036960330 conn(0x7fd1240b2f60 msgr2=0x7fd1240b01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:25:59.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.787+0000 7fd12ae00640 1 -- 172.21.15.86:0/2036960330 shutdown_connections 2024-04-20T21:25:59.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:25:59.787+0000 7fd12ae00640 1 -- 172.21.15.86:0/2036960330 wait complete. 2024-04-20T21:25:59.804 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-20T21:26:00.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.003+0000 7fc101400640 1 Processor -- start 2024-04-20T21:26:00.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.003+0000 7fc101400640 1 -- start start 2024-04-20T21:26:00.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.005+0000 7fc101400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0fc0b53b0 0x7fc0fc072590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:00.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.005+0000 7fc101400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc0fc072ad0 0x7fc0fc0cea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:00.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.005+0000 7fc101400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0fc06dcd0 con 0x7fc0fc15ac10 2024-04-20T21:26:00.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.005+0000 7fc101400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc0fc06e8e0 con 0x7fc0fc072ad0 2024-04-20T21:26:00.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.005+0000 7fc101400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc0fc0b57f0 con 0x7fc0fc0b53b0 2024-04-20T21:26:00.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.005+0000 7fc0fa800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fc0fc15ac10 0x7fc0fc15f210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:50940/0 (socket says 172.21.15.86:50940) 2024-04-20T21:26:00.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.005+0000 7fc0fa800640 1 -- 172.21.15.86:0/1946641807 learned_addr learned my addr 172.21.15.86:0/1946641807 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:26:00.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.005+0000 7fc0fb200640 1 --2- 172.21.15.86:0/1946641807 >> v2:172.21.15.86:3300/0 conn(0x7fc0fc072ad0 0x7fc0fc0cea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:00.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.005+0000 7fc0f9400640 1 -- 172.21.15.86:0/1946641807 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2543841831 0 0) 0x7fc0fc06dcd0 con 0x7fc0fc15ac10 2024-04-20T21:26:00.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.005+0000 7fc0f9e00640 1 --2- 172.21.15.86:0/1946641807 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0fc0b53b0 0x7fc0fc072590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:00.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.005+0000 7fc0f9400640 1 -- 172.21.15.86:0/1946641807 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc0dc003520 con 0x7fc0fc15ac10 2024-04-20T21:26:00.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.006+0000 7fc0fb200640 1 -- 172.21.15.86:0/1946641807 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0fc0b53b0 msgr2=0x7fc0fc072590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:00.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.006+0000 7fc0fb200640 1 --2- 172.21.15.86:0/1946641807 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0fc0b53b0 0x7fc0fc072590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.006+0000 7fc0fb200640 1 -- 172.21.15.86:0/1946641807 >> v1:172.21.15.86:6789/0 conn(0x7fc0fc15ac10 legacy=0x7fc0fc15f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:00.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.006+0000 7fc0fb200640 1 -- 172.21.15.86:0/1946641807 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0fc06dfb0 con 0x7fc0fc072ad0 2024-04-20T21:26:00.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.007+0000 7fc0fb200640 1 --2- 172.21.15.86:0/1946641807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0fc072ad0 0x7fc0fc0cea10 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fc0f40097a0 tx=0x7fc0f402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77dda282103b42c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:00.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.007+0000 7fc0f9400640 1 -- 172.21.15.86:0/1946641807 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4035020 con 0x7fc0fc072ad0 2024-04-20T21:26:00.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.007+0000 7fc0f9400640 1 -- 172.21.15.86:0/1946641807 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0f4031830 con 0x7fc0fc072ad0 2024-04-20T21:26:00.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.007+0000 7fc0f9400640 1 -- 172.21.15.86:0/1946641807 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4031c00 con 0x7fc0fc072ad0 2024-04-20T21:26:00.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.008+0000 7fc101400640 1 -- 172.21.15.86:0/1946641807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0fc072ad0 msgr2=0x7fc0fc0cea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:00.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.008+0000 7fc101400640 1 --2- 172.21.15.86:0/1946641807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0fc072ad0 0x7fc0fc0cea10 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fc0f40097a0 tx=0x7fc0f402b8d0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.008+0000 7fc101400640 1 -- 172.21.15.86:0/1946641807 shutdown_connections 2024-04-20T21:26:00.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.008+0000 7fc101400640 1 --2- 172.21.15.86:0/1946641807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0fc072ad0 0x7fc0fc0cea10 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.008+0000 7fc101400640 1 --2- 172.21.15.86:0/1946641807 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0fc0b53b0 0x7fc0fc072590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.008+0000 7fc101400640 1 -- 172.21.15.86:0/1946641807 >> 172.21.15.86:0/1946641807 conn(0x7fc0fc0b2360 msgr2=0x7fc0fc0b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:00.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.008+0000 7fc101400640 1 -- 172.21.15.86:0/1946641807 shutdown_connections 2024-04-20T21:26:00.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.008+0000 7fc101400640 1 -- 172.21.15.86:0/1946641807 wait complete. 2024-04-20T21:26:00.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.009+0000 7fc101400640 1 Processor -- start 2024-04-20T21:26:00.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.010+0000 7fc101400640 1 -- start start 2024-04-20T21:26:00.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.010+0000 7fc101400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0fc072ad0 0x7fc0fc147820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:00.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.011+0000 7fc101400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0fc0b53b0 0x7fc0fc0b34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:00.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.011+0000 7fc101400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0fc15ac10 0x7fc0fc144480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:00.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.011+0000 7fc101400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc0fc06e570 con 0x7fc0fc0b53b0 2024-04-20T21:26:00.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.011+0000 7fc101400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc0fc15aff0 con 0x7fc0fc072ad0 2024-04-20T21:26:00.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.011+0000 7fc101400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc0fc071d00 con 0x7fc0fc15ac10 2024-04-20T21:26:00.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.011+0000 7fc0f9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0fc0b53b0 0x7fc0fc0b34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:00.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.011+0000 7fc0f9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0fc0b53b0 0x7fc0fc0b34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43008/0 (socket says 172.21.15.86:43008) 2024-04-20T21:26:00.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.011+0000 7fc0f9e00640 1 -- 172.21.15.86:0/950211820 learned_addr learned my addr 172.21.15.86:0/950211820 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:00.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.011+0000 7fc0f9e00640 1 -- 172.21.15.86:0/950211820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0fc15ac10 msgr2=0x7fc0fc144480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:00.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.011+0000 7fc0f9e00640 1 --2- 172.21.15.86:0/950211820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0fc15ac10 0x7fc0fc144480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.011+0000 7fc0f9e00640 1 -- 172.21.15.86:0/950211820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0fc072ad0 msgr2=0x7fc0fc147820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:00.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.011+0000 7fc0f9e00640 1 --2- 172.21.15.86:0/950211820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0fc072ad0 0x7fc0fc147820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.011+0000 7fc0f9e00640 1 -- 172.21.15.86:0/950211820 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0fc146780 con 0x7fc0fc0b53b0 2024-04-20T21:26:00.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.012+0000 7fc0f9e00640 1 --2- 172.21.15.86:0/950211820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0fc0b53b0 0x7fc0fc0b34b0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4007ef0 tx=0x7fc0e400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:00.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.012+0000 7fc0f2a00640 1 -- 172.21.15.86:0/950211820 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4014620 con 0x7fc0fc0b53b0 2024-04-20T21:26:00.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.012+0000 7fc0f2a00640 1 -- 172.21.15.86:0/950211820 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0e4019070 con 0x7fc0fc0b53b0 2024-04-20T21:26:00.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.012+0000 7fc0f2a00640 1 -- 172.21.15.86:0/950211820 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4014a00 con 0x7fc0fc0b53b0 2024-04-20T21:26:00.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.012+0000 7fc101400640 1 -- 172.21.15.86:0/950211820 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0fc146a60 con 0x7fc0fc0b53b0 2024-04-20T21:26:00.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.012+0000 7fc101400640 1 -- 172.21.15.86:0/950211820 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0fc147020 con 0x7fc0fc0b53b0 2024-04-20T21:26:00.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.013+0000 7fc101400640 1 -- 172.21.15.86:0/950211820 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0fc0689e0 con 0x7fc0fc0b53b0 2024-04-20T21:26:00.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.014+0000 7fc0f2a00640 1 -- 172.21.15.86:0/950211820 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc0e4014ba0 con 0x7fc0fc0b53b0 2024-04-20T21:26:00.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.016+0000 7fc0f2a00640 1 --2- 172.21.15.86:0/950211820 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc0d806c8f0 0x7fc0d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:00.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.016+0000 7fc0f2a00640 1 -- 172.21.15.86:0/950211820 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7fc0e408c5e0 con 0x7fc0fc0b53b0 2024-04-20T21:26:00.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.016+0000 7fc0fa800640 1 --2- 172.21.15.86:0/950211820 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc0d806c8f0 0x7fc0d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:00.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.017+0000 7fc0fa800640 1 --2- 172.21.15.86:0/950211820 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc0d806c8f0 0x7fc0d806edb0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec000f60 tx=0x7fc0ec0078f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:00.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.020+0000 7fc0f2a00640 1 -- 172.21.15.86:0/950211820 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc0e405a890 con 0x7fc0fc0b53b0 2024-04-20T21:26:00.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.266+0000 7fc101400640 1 -- 172.21.15.86:0/950211820 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc0fc06ee90 con 0x7fc0fc0b53b0 2024-04-20T21:26:00.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.266+0000 7fc0f2a00640 1 -- 172.21.15.86:0/950211820 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7fc0e405a190 con 0x7fc0fc0b53b0 2024-04-20T21:26:00.265 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:26:00.266 INFO:teuthology.orchestra.run.smithi086.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-04-20T21:26:00.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.269+0000 7fc101400640 1 -- 172.21.15.86:0/950211820 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc0d806c8f0 msgr2=0x7fc0d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:00.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.269+0000 7fc101400640 1 --2- 172.21.15.86:0/950211820 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc0d806c8f0 0x7fc0d806edb0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec000f60 tx=0x7fc0ec0078f0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.269+0000 7fc101400640 1 -- 172.21.15.86:0/950211820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0fc0b53b0 msgr2=0x7fc0fc0b34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:00.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.269+0000 7fc101400640 1 --2- 172.21.15.86:0/950211820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0fc0b53b0 0x7fc0fc0b34b0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4007ef0 tx=0x7fc0e400e4b0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.269+0000 7fc101400640 1 -- 172.21.15.86:0/950211820 shutdown_connections 2024-04-20T21:26:00.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.269+0000 7fc101400640 1 --2- 172.21.15.86:0/950211820 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc0d806c8f0 0x7fc0d806edb0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.269+0000 7fc101400640 1 --2- 172.21.15.86:0/950211820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0fc15ac10 0x7fc0fc144480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.269+0000 7fc101400640 1 --2- 172.21.15.86:0/950211820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0fc0b53b0 0x7fc0fc0b34b0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.269+0000 7fc101400640 1 --2- 172.21.15.86:0/950211820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0fc072ad0 0x7fc0fc147820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.269+0000 7fc101400640 1 -- 172.21.15.86:0/950211820 >> 172.21.15.86:0/950211820 conn(0x7fc0fc0b2360 msgr2=0x7fc0fc0b1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:00.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.270+0000 7fc101400640 1 -- 172.21.15.86:0/950211820 shutdown_connections 2024-04-20T21:26:00.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.270+0000 7fc101400640 1 -- 172.21.15.86:0/950211820 wait complete. 2024-04-20T21:26:00.287 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-20T21:26:00.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:26:00.287 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-04-20T21:26:00.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.488+0000 7f3395e00640 1 Processor -- start 2024-04-20T21:26:00.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.489+0000 7f3395e00640 1 -- start start 2024-04-20T21:26:00.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.490+0000 7f3395e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f339006f0b0 0x7f3390145a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:00.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.490+0000 7f3395e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f339006e010 con 0x7f3390145fd0 2024-04-20T21:26:00.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.490+0000 7f3395e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f339006e2f0 con 0x7f3390071d00 2024-04-20T21:26:00.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.490+0000 7f3395e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f339006e5d0 con 0x7f339006f0b0 2024-04-20T21:26:00.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.491+0000 7f338f400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f3390145fd0 0x7f339015cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:50946/0 (socket says 172.21.15.86:50946) 2024-04-20T21:26:00.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.491+0000 7f338f400640 1 -- 172.21.15.86:0/815303344 learned_addr learned my addr 172.21.15.86:0/815303344 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:26:00.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.491+0000 7f338e000640 1 -- 172.21.15.86:0/815303344 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3740710171 0 0) 0x7f339006e010 con 0x7f3390145fd0 2024-04-20T21:26:00.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.491+0000 7f338e000640 1 -- 172.21.15.86:0/815303344 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3374003520 con 0x7f3390145fd0 2024-04-20T21:26:00.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.491+0000 7f3387e00640 1 --2- 172.21.15.86:0/815303344 >> v2:172.21.15.144:3300/0 conn(0x7f339006f0b0 0x7f3390145a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:00.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.491+0000 7f338e000640 1 -- 172.21.15.86:0/815303344 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1687645879 0 0) 0x7f3374003520 con 0x7f3390145fd0 2024-04-20T21:26:00.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.491+0000 7f338e000640 1 -- 172.21.15.86:0/815303344 >> v2:172.21.15.144:3300/0 conn(0x7f339006f0b0 msgr2=0x7f3390145a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:00.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.491+0000 7f338e000640 1 --2- 172.21.15.86:0/815303344 >> v2:172.21.15.144:3300/0 conn(0x7f339006f0b0 0x7f3390145a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.491+0000 7f338e000640 1 -- 172.21.15.86:0/815303344 >> v1:172.21.15.144:6789/0 conn(0x7f3390071d00 legacy=0x7f33900720e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:00.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.491+0000 7f338e000640 1 -- 172.21.15.86:0/815303344 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f339006e8b0 con 0x7f3390145fd0 2024-04-20T21:26:00.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.491+0000 7f338e000640 1 -- 172.21.15.86:0/815303344 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f33880019f0 con 0x7f3390145fd0 2024-04-20T21:26:00.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.491+0000 7f338e000640 1 -- 172.21.15.86:0/815303344 >> v1:172.21.15.86:6789/0 conn(0x7f3390145fd0 legacy=0x7f339015cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:00.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.492+0000 7f338e000640 1 --2- 172.21.15.86:0/815303344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33740039b0 0x7f3374003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:00.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.492+0000 7f338e000640 1 --2- 172.21.15.86:0/815303344 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3374004800 0x7f3374004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:00.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.492+0000 7f338e000640 1 --2- 172.21.15.86:0/815303344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33740051c0 0x7f337400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:00.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.492+0000 7f338e000640 1 -- 172.21.15.86:0/815303344 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f339006e5d0 con 0x7f33740051c0 2024-04-20T21:26:00.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.492+0000 7f338e000640 1 -- 172.21.15.86:0/815303344 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f337400c110 con 0x7f33740039b0 2024-04-20T21:26:00.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.492+0000 7f338e000640 1 -- 172.21.15.86:0/815303344 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f337400c310 con 0x7f3374004800 2024-04-20T21:26:00.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.492+0000 7f338f400640 1 --2- 172.21.15.86:0/815303344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33740051c0 0x7f337400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:00.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.492+0000 7f3387e00640 1 --2- 172.21.15.86:0/815303344 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3374004800 0x7f3374004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:00.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.492+0000 7f338ea00640 1 --2- 172.21.15.86:0/815303344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33740039b0 0x7f3374003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:00.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.492+0000 7f338f400640 1 -- 172.21.15.86:0/815303344 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3374004800 msgr2=0x7f3374004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:00.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.492+0000 7f338f400640 1 --2- 172.21.15.86:0/815303344 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3374004800 0x7f3374004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.492+0000 7f338f400640 1 -- 172.21.15.86:0/815303344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33740039b0 msgr2=0x7f3374003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:00.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.492+0000 7f338f400640 1 --2- 172.21.15.86:0/815303344 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33740039b0 0x7f3374003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.492+0000 7f338f400640 1 -- 172.21.15.86:0/815303344 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3374003520 con 0x7f33740051c0 2024-04-20T21:26:00.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.493+0000 7f338f400640 1 -- 172.21.15.86:0/815303344 reap_dead start 2024-04-20T21:26:00.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.493+0000 7f338f400640 1 --2- 172.21.15.86:0/815303344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33740051c0 0x7f337400b9c0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f3390146400 tx=0x7f3374005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f65e34236c127c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:00.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.494+0000 7f338e000640 1 -- 172.21.15.86:0/815303344 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338800a4a0 con 0x7f33740051c0 2024-04-20T21:26:00.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.494+0000 7f338e000640 1 -- 172.21.15.86:0/815303344 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3388034650 con 0x7f33740051c0 2024-04-20T21:26:00.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.494+0000 7f338e000640 1 -- 172.21.15.86:0/815303344 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3388034a40 con 0x7f33740051c0 2024-04-20T21:26:00.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.494+0000 7f3395e00640 1 -- 172.21.15.86:0/815303344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33740051c0 msgr2=0x7f337400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:00.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.494+0000 7f3395e00640 1 --2- 172.21.15.86:0/815303344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33740051c0 0x7f337400b9c0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f3390146400 tx=0x7f3374005830 comp rx=0 tx=0).stop 2024-04-20T21:26:00.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.495+0000 7f3395e00640 1 -- 172.21.15.86:0/815303344 shutdown_connections 2024-04-20T21:26:00.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.495+0000 7f3395e00640 1 --2- 172.21.15.86:0/815303344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33740051c0 0x7f337400b9c0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.495+0000 7f3395e00640 1 -- 172.21.15.86:0/815303344 >> 172.21.15.86:0/815303344 conn(0x7f33900b2180 msgr2=0x7f33900b2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:00.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.495+0000 7f3395e00640 1 -- 172.21.15.86:0/815303344 shutdown_connections 2024-04-20T21:26:00.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.495+0000 7f3395e00640 1 -- 172.21.15.86:0/815303344 wait complete. 2024-04-20T21:26:00.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.496+0000 7f3395e00640 1 Processor -- start 2024-04-20T21:26:00.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.497+0000 7f3395e00640 1 -- start start 2024-04-20T21:26:00.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.497+0000 7f3395e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f33740051c0 0x7f33901f6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:00.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.498+0000 7f3395e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33901f7730 0x7f33901f7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:00.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.498+0000 7f3395e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33901435b0 0x7f3390143a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:00.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.498+0000 7f3395e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f339006dc30 con 0x7f33901435b0 2024-04-20T21:26:00.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.498+0000 7f3395e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f339006f510 con 0x7f33901f7730 2024-04-20T21:26:00.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.498+0000 7f3395e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f339006ddb0 con 0x7f33740051c0 2024-04-20T21:26:00.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.498+0000 7f338f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33901435b0 0x7f3390143a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:00.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.498+0000 7f338f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33901435b0 0x7f3390143a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43024/0 (socket says 172.21.15.86:43024) 2024-04-20T21:26:00.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.498+0000 7f338f400640 1 -- 172.21.15.86:0/802632306 learned_addr learned my addr 172.21.15.86:0/802632306 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:00.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.498+0000 7f338ea00640 1 --2- 172.21.15.86:0/802632306 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f33740051c0 0x7f33901f6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:00.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.498+0000 7f3387e00640 1 --2- 172.21.15.86:0/802632306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33901f7730 0x7f33901f7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:00.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.498+0000 7f338f400640 1 -- 172.21.15.86:0/802632306 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f33740051c0 msgr2=0x7f33901f6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:00.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.498+0000 7f338f400640 1 --2- 172.21.15.86:0/802632306 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f33740051c0 0x7f33901f6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.498+0000 7f338f400640 1 -- 172.21.15.86:0/802632306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33901f7730 msgr2=0x7f33901f7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:00.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.498+0000 7f338f400640 1 --2- 172.21.15.86:0/802632306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33901f7730 0x7f33901f7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.498+0000 7f338f400640 1 -- 172.21.15.86:0/802632306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3390144330 con 0x7f33901435b0 2024-04-20T21:26:00.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.498+0000 7f338ea00640 1 --2- 172.21.15.86:0/802632306 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f33740051c0 0x7f33901f6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:26:00.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.498+0000 7f3387e00640 1 --2- 172.21.15.86:0/802632306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33901f7730 0x7f33901f7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:26:00.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.499+0000 7f338f400640 1 --2- 172.21.15.86:0/802632306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33901435b0 0x7f3390143a30 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f3388002230 tx=0x7f3388001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:00.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.499+0000 7f3387400640 1 -- 172.21.15.86:0/802632306 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3388039040 con 0x7f33901435b0 2024-04-20T21:26:00.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.499+0000 7f3387400640 1 -- 172.21.15.86:0/802632306 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3388043070 con 0x7f33901435b0 2024-04-20T21:26:00.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.499+0000 7f3395e00640 1 -- 172.21.15.86:0/802632306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3390144610 con 0x7f33901435b0 2024-04-20T21:26:00.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.499+0000 7f3387400640 1 -- 172.21.15.86:0/802632306 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338800a480 con 0x7f33901435b0 2024-04-20T21:26:00.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.499+0000 7f3395e00640 1 -- 172.21.15.86:0/802632306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f339012de50 con 0x7f33901435b0 2024-04-20T21:26:00.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.501+0000 7f3387400640 1 -- 172.21.15.86:0/802632306 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3388004b60 con 0x7f33901435b0 2024-04-20T21:26:00.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.502+0000 7f3387400640 1 --2- 172.21.15.86:0/802632306 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f336c06c8f0 0x7f336c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:00.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.502+0000 7f3387400640 1 -- 172.21.15.86:0/802632306 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7f33880ad9b0 con 0x7f33901435b0 2024-04-20T21:26:00.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.503+0000 7f338ea00640 1 --2- 172.21.15.86:0/802632306 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f336c06c8f0 0x7f336c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:00.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.503+0000 7f3398b70740 1 Processor -- start 2024-04-20T21:26:00.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.503+0000 7f3398b70740 1 -- start start 2024-04-20T21:26:00.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.503+0000 7f338ea00640 1 --2- 172.21.15.86:0/802632306 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f336c06c8f0 0x7f336c06edb0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f3380000c00 tx=0x7f3380009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:00.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.504+0000 7f3398b70740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa7b721fe0 0x55aa7b6f4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:00.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.504+0000 7f3398b70740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa7b7223c0 0x55aa7b724d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:00.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.504+0000 7f3398b70740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aa7b6f54c0 0x55aa7b7252b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:00.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.504+0000 7f3398b70740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55aa7b65fd20 con 0x55aa7b721fe0 2024-04-20T21:26:00.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.504+0000 7f3398b70740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55aa7b6063e0 con 0x55aa7b7223c0 2024-04-20T21:26:00.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.504+0000 7f3398b70740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55aa7b65bce0 con 0x55aa7b6f54c0 2024-04-20T21:26:00.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.504+0000 7f338ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa7b721fe0 0x55aa7b6f4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:00.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.504+0000 7f338ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa7b721fe0 0x55aa7b6f4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43036/0 (socket says 172.21.15.86:43036) 2024-04-20T21:26:00.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.504+0000 7f338ea00640 1 -- 172.21.15.86:0/3964050257 learned_addr learned my addr 172.21.15.86:0/3964050257 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:00.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.504+0000 7f3387e00640 1 --2- 172.21.15.86:0/3964050257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa7b7223c0 0x55aa7b724d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:00.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.505+0000 7f338ea00640 1 -- 172.21.15.86:0/3964050257 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aa7b6f54c0 msgr2=0x55aa7b7252b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:00.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.505+0000 7f338ea00640 1 --2- 172.21.15.86:0/3964050257 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aa7b6f54c0 0x55aa7b7252b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.505+0000 7f338ea00640 1 -- 172.21.15.86:0/3964050257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa7b7223c0 msgr2=0x55aa7b724d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:00.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.505+0000 7f338ea00640 1 --2- 172.21.15.86:0/3964050257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa7b7223c0 0x55aa7b724d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.505+0000 7f338ea00640 1 -- 172.21.15.86:0/3964050257 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa7b625980 con 0x55aa7b721fe0 2024-04-20T21:26:00.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.505+0000 7f3387e00640 1 --2- 172.21.15.86:0/3964050257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa7b7223c0 0x55aa7b724d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:26:00.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.505+0000 7f338ea00640 1 --2- 172.21.15.86:0/3964050257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa7b721fe0 0x55aa7b6f4f80 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f338000e960 tx=0x7f3380011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8b0cb9b3b4de607 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:00.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.505+0000 7f337f400640 1 -- 172.21.15.86:0/3964050257 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338001a050 con 0x55aa7b721fe0 2024-04-20T21:26:00.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.505+0000 7f337f400640 1 -- 172.21.15.86:0/3964050257 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3380024070 con 0x55aa7b721fe0 2024-04-20T21:26:00.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.505+0000 7f337f400640 1 -- 172.21.15.86:0/3964050257 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3380013b70 con 0x55aa7b721fe0 2024-04-20T21:26:00.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.506+0000 7f3398b70740 1 -- 172.21.15.86:0/3964050257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa7b721fe0 msgr2=0x55aa7b6f4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:00.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.506+0000 7f3398b70740 1 --2- 172.21.15.86:0/3964050257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa7b721fe0 0x55aa7b6f4f80 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f338000e960 tx=0x7f3380011000 comp rx=0 tx=0).stop 2024-04-20T21:26:00.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.506+0000 7f3398b70740 1 -- 172.21.15.86:0/3964050257 shutdown_connections 2024-04-20T21:26:00.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.506+0000 7f3398b70740 1 --2- 172.21.15.86:0/3964050257 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aa7b6f54c0 0x55aa7b7252b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.506+0000 7f3398b70740 1 --2- 172.21.15.86:0/3964050257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa7b7223c0 0x55aa7b724d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.506+0000 7f3398b70740 1 --2- 172.21.15.86:0/3964050257 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa7b721fe0 0x55aa7b6f4f80 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.506+0000 7f3398b70740 1 -- 172.21.15.86:0/3964050257 >> 172.21.15.86:0/3964050257 conn(0x55aa7b6f71f0 msgr2=0x55aa7b73fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:00.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.506+0000 7f3398b70740 1 -- 172.21.15.86:0/3964050257 shutdown_connections 2024-04-20T21:26:00.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.506+0000 7f3398b70740 1 -- 172.21.15.86:0/3964050257 wait complete. 2024-04-20T21:26:00.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.507+0000 7f3398b70740 1 -- start start 2024-04-20T21:26:00.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.507+0000 7f3398b70740 1 Processor -- start 2024-04-20T21:26:00.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.508+0000 7f3398b70740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aa7b721fe0 0x55aa7b714d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:00.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.508+0000 7f3398b70740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa7b7223c0 0x55aa7b753860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:00.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.508+0000 7f3398b70740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa7b718940 0x55aa7b726a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:00.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.508+0000 7f3398b70740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55aa7b6f9890 con 0x55aa7b718940 2024-04-20T21:26:00.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.508+0000 7f3398b70740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55aa7b6197a0 con 0x55aa7b7223c0 2024-04-20T21:26:00.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.508+0000 7f3398b70740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55aa7b6fc0d0 con 0x55aa7b721fe0 2024-04-20T21:26:00.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.508+0000 7f3398b70740 10 client.0 _ll_register_callbacks cb 0x55aa7b61bb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:26:00.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.508+0000 7f338f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa7b718940 0x55aa7b726a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:00.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.508+0000 7f338f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa7b718940 0x55aa7b726a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43042/0 (socket says 172.21.15.86:43042) 2024-04-20T21:26:00.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.508+0000 7f338f400640 1 -- 172.21.15.86:0/3324528279 learned_addr learned my addr 172.21.15.86:0/3324528279 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:00.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.509+0000 7f3387e00640 1 --2- 172.21.15.86:0/3324528279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa7b7223c0 0x55aa7b753860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:00.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.509+0000 7f338ea00640 1 --2- 172.21.15.86:0/3324528279 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aa7b721fe0 0x55aa7b714d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:00.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.509+0000 7f338f400640 1 -- 172.21.15.86:0/3324528279 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aa7b721fe0 msgr2=0x55aa7b714d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:00.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.509+0000 7f338f400640 1 --2- 172.21.15.86:0/3324528279 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aa7b721fe0 0x55aa7b714d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.509+0000 7f338f400640 1 -- 172.21.15.86:0/3324528279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa7b7223c0 msgr2=0x55aa7b753860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:00.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.509+0000 7f338f400640 1 --2- 172.21.15.86:0/3324528279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa7b7223c0 0x55aa7b753860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.509+0000 7f338f400640 1 -- 172.21.15.86:0/3324528279 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa7b43a7e0 con 0x55aa7b718940 2024-04-20T21:26:00.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.509+0000 7f338ea00640 1 --2- 172.21.15.86:0/3324528279 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aa7b721fe0 0x55aa7b714d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:26:00.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.509+0000 7f338f400640 1 --2- 172.21.15.86:0/3324528279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa7b718940 0x55aa7b726a90 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f3388083d30 tx=0x7f3388034710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e95a619cb41a221 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:00.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.509+0000 7f3373e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:26:00.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.509+0000 7f3373e00640 1 -- 172.21.15.86:0/3324528279 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338807fa30 con 0x55aa7b718940 2024-04-20T21:26:00.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.509+0000 7f3373e00640 1 -- 172.21.15.86:0/3324528279 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7f3388077cf0 con 0x55aa7b718940 2024-04-20T21:26:00.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.509+0000 7f3398b70740 1 -- 172.21.15.86:0/3324528279 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3380024070 con 0x55aa7b718940 2024-04-20T21:26:00.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.510+0000 7f3373e00640 1 -- 172.21.15.86:0/3324528279 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f3388093e60 con 0x55aa7b718940 2024-04-20T21:26:00.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.510+0000 7f3398b70740 10 client.7956 fetch_fsmap learned FSMap version 66 2024-04-20T21:26:00.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.510+0000 7f3398b70740 1 -- 172.21.15.86:0/3324528279 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x55aa7b49af70 con 0x55aa7b718940 2024-04-20T21:26:00.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.510+0000 7f3373e00640 1 -- 172.21.15.86:0/3324528279 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3380024070 con 0x55aa7b718940 2024-04-20T21:26:00.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.510+0000 7f3373e00640 1 -- 172.21.15.86:0/3324528279 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 66) ==== 3495+0+0 (secure 0 0 0) 0x7f3388083dd0 con 0x55aa7b718940 2024-04-20T21:26:00.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.510+0000 7f3398b70740 10 client.7956 fetch_fsmap finished waiting for FSMap version 66 2024-04-20T21:26:00.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.510+0000 7f3398b70740 10 client.7956 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-04-20T21:26:00.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.510+0000 7f3398b70740 20 client.7956 populate_metadata read hostname 'smithi086' 2024-04-20T21:26:00.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.510+0000 7f3398b70740 1 --2- 172.21.15.86:0/3324528279 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x55aa7b71f9e0 0x55aa7b71fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:00.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.510+0000 7f3398b70740 4 client.7956 mds_command: new command op to 7773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:26:00.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.510+0000 7f3398b70740 1 -- 172.21.15.86:0/3324528279 --> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa7b66d740 con 0x55aa7b71f9e0 2024-04-20T21:26:00.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.511+0000 7f338ea00640 1 --2- 172.21.15.86:0/3324528279 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x55aa7b71f9e0 0x55aa7b71fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:00.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.512+0000 7f338ea00640 1 --2- 172.21.15.86:0/3324528279 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x55aa7b71f9e0 0x55aa7b71fdf0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4cb17082f80c899 server_cookie=a78518df0836067c in_seq=0 out_seq=0 2024-04-20T21:26:00.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.512+0000 7f3373e00640 10 client.7956 ms_handle_connect on v2:172.21.15.144:6833/3981403733 2024-04-20T21:26:00.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.513+0000 7f3373e00640 1 -- 172.21.15.86:0/3324528279 <== mds.0 v2:172.21.15.144:6833/3981403733 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55aa7b66d740 con 0x55aa7b71f9e0 2024-04-20T21:26:00.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.513+0000 7f3373e00640 10 client.7956 handle_command_reply: tid=0 2024-04-20T21:26:00.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.513+0000 7f3398b70740 1 client.7956 shutdown 2024-04-20T21:26:00.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.514+0000 7f3398b70740 1 -- 172.21.15.86:0/3324528279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa7b718940 msgr2=0x55aa7b726a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:00.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.514+0000 7f3398b70740 1 --2- 172.21.15.86:0/3324528279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa7b718940 0x55aa7b726a90 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f3388083d30 tx=0x7f3388034710 comp rx=0 tx=0).stop 2024-04-20T21:26:00.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.514+0000 7f3398b70740 1 -- 172.21.15.86:0/3324528279 shutdown_connections 2024-04-20T21:26:00.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.514+0000 7f3398b70740 1 --2- 172.21.15.86:0/3324528279 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x55aa7b71f9e0 0x55aa7b71fdf0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.514+0000 7f3398b70740 1 --2- 172.21.15.86:0/3324528279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa7b718940 0x55aa7b726a90 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.514+0000 7f3398b70740 1 --2- 172.21.15.86:0/3324528279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa7b7223c0 0x55aa7b753860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.514+0000 7f3398b70740 1 --2- 172.21.15.86:0/3324528279 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aa7b721fe0 0x55aa7b714d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.514+0000 7f3373e00640 0 client.7956 ms_handle_reset on v2:172.21.15.144:6833/3981403733 2024-04-20T21:26:00.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.514+0000 7f3398b70740 1 -- 172.21.15.86:0/3324528279 >> 172.21.15.86:0/3324528279 conn(0x55aa7b6f54c0 msgr2=0x55aa7b743280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:00.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.514+0000 7f3398b70740 1 -- 172.21.15.86:0/3324528279 shutdown_connections 2024-04-20T21:26:00.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.514+0000 7f3398b70740 1 -- 172.21.15.86:0/3324528279 wait complete. 2024-04-20T21:26:00.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.514+0000 7f3398b70740 20 client.7956 trim_cache size 0 max 16384 2024-04-20T21:26:00.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.533+0000 7f3398b70740 1 Processor -- start 2024-04-20T21:26:00.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.533+0000 7f3398b70740 1 -- start start 2024-04-20T21:26:00.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.534+0000 7f3398b70740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa7b718940 0x55aa7b71fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:00.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.534+0000 7f3398b70740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa7b721fe0 0x55aa7b727a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:00.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.534+0000 7f3398b70740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aa7b7223c0 0x55aa7b743280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:00.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.534+0000 7f3398b70740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f33902015d0 con 0x55aa7b721fe0 2024-04-20T21:26:00.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.534+0000 7f3398b70740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f33902005a0 con 0x55aa7b718940 2024-04-20T21:26:00.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.534+0000 7f3398b70740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f339012f930 con 0x55aa7b7223c0 2024-04-20T21:26:00.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.534+0000 7f3387e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa7b721fe0 0x55aa7b727a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:00.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.535+0000 7f3387e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa7b721fe0 0x55aa7b727a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43058/0 (socket says 172.21.15.86:43058) 2024-04-20T21:26:00.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.535+0000 7f3387e00640 1 -- 172.21.15.86:0/715177453 learned_addr learned my addr 172.21.15.86:0/715177453 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:00.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.535+0000 7f338ea00640 1 --2- 172.21.15.86:0/715177453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa7b718940 0x55aa7b71fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:00.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.535+0000 7f3387e00640 1 -- 172.21.15.86:0/715177453 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aa7b7223c0 msgr2=0x55aa7b743280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:00.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.535+0000 7f3387e00640 1 --2- 172.21.15.86:0/715177453 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aa7b7223c0 0x55aa7b743280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.535+0000 7f3387e00640 1 -- 172.21.15.86:0/715177453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa7b718940 msgr2=0x55aa7b71fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:00.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.535+0000 7f3387e00640 1 --2- 172.21.15.86:0/715177453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa7b718940 0x55aa7b71fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.535+0000 7f3387e00640 1 -- 172.21.15.86:0/715177453 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa7b332170 con 0x55aa7b721fe0 2024-04-20T21:26:00.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.535+0000 7f3387e00640 1 --2- 172.21.15.86:0/715177453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa7b721fe0 0x55aa7b727a50 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f3378010aa0 tx=0x7f3378010f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=505875507d9b73c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:00.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.535+0000 7f337ea00640 1 -- 172.21.15.86:0/715177453 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3378019030 con 0x55aa7b721fe0 2024-04-20T21:26:00.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.535+0000 7f337ea00640 1 -- 172.21.15.86:0/715177453 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3378012c70 con 0x55aa7b721fe0 2024-04-20T21:26:00.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.535+0000 7f337ea00640 1 -- 172.21.15.86:0/715177453 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3378012e40 con 0x55aa7b721fe0 2024-04-20T21:26:00.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.536+0000 7f3398b70740 1 -- 172.21.15.86:0/715177453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa7b721fe0 msgr2=0x55aa7b727a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:00.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.536+0000 7f3398b70740 1 --2- 172.21.15.86:0/715177453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa7b721fe0 0x55aa7b727a50 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f3378010aa0 tx=0x7f3378010f60 comp rx=0 tx=0).stop 2024-04-20T21:26:00.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.536+0000 7f3398b70740 1 -- 172.21.15.86:0/715177453 shutdown_connections 2024-04-20T21:26:00.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.536+0000 7f3398b70740 1 --2- 172.21.15.86:0/715177453 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aa7b7223c0 0x55aa7b743280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.536+0000 7f3398b70740 1 --2- 172.21.15.86:0/715177453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa7b721fe0 0x55aa7b727a50 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.536+0000 7f3398b70740 1 --2- 172.21.15.86:0/715177453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa7b718940 0x55aa7b71fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.536+0000 7f3398b70740 1 -- 172.21.15.86:0/715177453 >> 172.21.15.86:0/715177453 conn(0x55aa7b6f54c0 msgr2=0x55aa7b726850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:00.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.536+0000 7f3398b70740 1 -- 172.21.15.86:0/715177453 shutdown_connections 2024-04-20T21:26:00.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.536+0000 7f3398b70740 1 -- 172.21.15.86:0/715177453 wait complete. 2024-04-20T21:26:00.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.537+0000 7f3398b70740 1 -- start start 2024-04-20T21:26:00.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.537+0000 7f3398b70740 1 Processor -- start 2024-04-20T21:26:00.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.538+0000 7f3398b70740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa7b718940 0x55aa7b6f9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:00.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.538+0000 7f3398b70740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa7b721fe0 0x55aa7b752ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:00.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.538+0000 7f3398b70740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aa7b7223c0 0x55aa7b719140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:00.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.538+0000 7f3398b70740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55aa7b7247e0 con 0x55aa7b718940 2024-04-20T21:26:00.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.538+0000 7f3398b70740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f33901fff50 con 0x55aa7b721fe0 2024-04-20T21:26:00.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.538+0000 7f3398b70740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55aa7b6261d0 con 0x55aa7b7223c0 2024-04-20T21:26:00.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.538+0000 7f3398b70740 10 client.0 _ll_register_callbacks cb 0x7f3388082050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:26:00.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.538+0000 7f338ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa7b718940 0x55aa7b6f9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:00.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.538+0000 7f338ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa7b718940 0x55aa7b6f9c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43060/0 (socket says 172.21.15.86:43060) 2024-04-20T21:26:00.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.538+0000 7f338ea00640 1 -- 172.21.15.86:0/573499038 learned_addr learned my addr 172.21.15.86:0/573499038 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:00.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.539+0000 7f338f400640 1 --2- 172.21.15.86:0/573499038 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aa7b7223c0 0x55aa7b719140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:00.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.539+0000 7f3387e00640 1 --2- 172.21.15.86:0/573499038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa7b721fe0 0x55aa7b752ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:00.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.539+0000 7f338ea00640 1 -- 172.21.15.86:0/573499038 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aa7b7223c0 msgr2=0x55aa7b719140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:00.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.539+0000 7f338ea00640 1 --2- 172.21.15.86:0/573499038 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aa7b7223c0 0x55aa7b719140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.539+0000 7f338ea00640 1 -- 172.21.15.86:0/573499038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa7b721fe0 msgr2=0x55aa7b752ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:00.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.539+0000 7f338ea00640 1 --2- 172.21.15.86:0/573499038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa7b721fe0 0x55aa7b752ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.539+0000 7f338ea00640 1 -- 172.21.15.86:0/573499038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa7b7b4450 con 0x55aa7b718940 2024-04-20T21:26:00.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.539+0000 7f338f400640 1 --2- 172.21.15.86:0/573499038 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aa7b7223c0 0x55aa7b719140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:26:00.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.539+0000 7f338ea00640 1 --2- 172.21.15.86:0/573499038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa7b718940 0x55aa7b6f9c20 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f3380009010 tx=0x7f3380015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=402bc25671197747 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:00.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.539+0000 7f3373e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:26:00.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.539+0000 7f3373e00640 1 -- 172.21.15.86:0/573499038 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338001a040 con 0x55aa7b718940 2024-04-20T21:26:00.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.539+0000 7f3373e00640 1 -- 172.21.15.86:0/573499038 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7f3380013980 con 0x55aa7b718940 2024-04-20T21:26:00.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.539+0000 7f3398b70740 1 -- 172.21.15.86:0/573499038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3378012c70 con 0x55aa7b718940 2024-04-20T21:26:00.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.540+0000 7f3373e00640 1 -- 172.21.15.86:0/573499038 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f3380013b80 con 0x55aa7b718940 2024-04-20T21:26:00.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.540+0000 7f3398b70740 10 client.7962 fetch_fsmap learned FSMap version 66 2024-04-20T21:26:00.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.540+0000 7f3398b70740 1 -- 172.21.15.86:0/573499038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x55aa7b6fa750 con 0x55aa7b718940 2024-04-20T21:26:00.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.540+0000 7f3373e00640 1 -- 172.21.15.86:0/573499038 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3378012c70 con 0x55aa7b718940 2024-04-20T21:26:00.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.540+0000 7f3373e00640 1 -- 172.21.15.86:0/573499038 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 66) ==== 3495+0+0 (secure 0 0 0) 0x7f3380043020 con 0x55aa7b718940 2024-04-20T21:26:00.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.540+0000 7f3398b70740 10 client.7962 fetch_fsmap finished waiting for FSMap version 66 2024-04-20T21:26:00.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.540+0000 7f3398b70740 10 client.7962 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-04-20T21:26:00.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.540+0000 7f3398b70740 20 client.7962 populate_metadata read hostname 'smithi086' 2024-04-20T21:26:00.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.540+0000 7f3398b70740 1 --2- 172.21.15.86:0/573499038 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x55aa7b743280 0x55aa7b743690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:00.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.540+0000 7f3398b70740 4 client.7962 mds_command: new command op to 7773 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-04-20T21:26:00.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.540+0000 7f3398b70740 1 -- 172.21.15.86:0/573499038 --> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) -- 0x55aa7b6675b0 con 0x55aa7b743280 2024-04-20T21:26:00.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.541+0000 7f3387e00640 1 --2- 172.21.15.86:0/573499038 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x55aa7b743280 0x55aa7b743690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:00.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.542+0000 7f3387e00640 1 --2- 172.21.15.86:0/573499038 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x55aa7b743280 0x55aa7b743690 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=926240c1978c1e21 server_cookie=d563d18fc31aaf00 in_seq=0 out_seq=0 2024-04-20T21:26:00.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.542+0000 7f3373e00640 10 client.7962 ms_handle_connect on v2:172.21.15.144:6833/3981403733 2024-04-20T21:26:00.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.543+0000 7f3373e00640 1 -- 172.21.15.86:0/573499038 <== mds.0 v2:172.21.15.144:6833/3981403733 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55aa7b6675b0 con 0x55aa7b743280 2024-04-20T21:26:00.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.543+0000 7f3373e00640 10 client.7962 handle_command_reply: tid=0 2024-04-20T21:26:00.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.543+0000 7f3398b70740 1 client.7962 shutdown 2024-04-20T21:26:00.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.543+0000 7f3398b70740 1 -- 172.21.15.86:0/573499038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa7b718940 msgr2=0x55aa7b6f9c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:00.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.543+0000 7f3398b70740 1 --2- 172.21.15.86:0/573499038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa7b718940 0x55aa7b6f9c20 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f3380009010 tx=0x7f3380015000 comp rx=0 tx=0).stop 2024-04-20T21:26:00.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.543+0000 7f3398b70740 1 -- 172.21.15.86:0/573499038 shutdown_connections 2024-04-20T21:26:00.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.543+0000 7f3398b70740 1 --2- 172.21.15.86:0/573499038 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x55aa7b743280 0x55aa7b743690 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.543+0000 7f3398b70740 1 --2- 172.21.15.86:0/573499038 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aa7b7223c0 0x55aa7b719140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.543+0000 7f3398b70740 1 --2- 172.21.15.86:0/573499038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aa7b721fe0 0x55aa7b752ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.543+0000 7f3373e00640 0 client.7962 ms_handle_reset on v2:172.21.15.144:6833/3981403733 2024-04-20T21:26:00.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.543+0000 7f3398b70740 1 --2- 172.21.15.86:0/573499038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aa7b718940 0x55aa7b6f9c20 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.543+0000 7f3398b70740 1 -- 172.21.15.86:0/573499038 >> 172.21.15.86:0/573499038 conn(0x55aa7b6f54c0 msgr2=0x55aa7b7426c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:00.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.544+0000 7f3398b70740 1 -- 172.21.15.86:0/573499038 shutdown_connections 2024-04-20T21:26:00.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.544+0000 7f3398b70740 1 -- 172.21.15.86:0/573499038 wait complete. 2024-04-20T21:26:00.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.544+0000 7f3398b70740 20 client.7962 trim_cache size 0 max 16384 2024-04-20T21:26:00.545 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:26:00.545 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-20T21:26:00.545 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "e2a76b86-8e01-4180-8000-777ebcbc1af9", 2024-04-20T21:26:00.545 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-20T21:26:00.545 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:26:00.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.544+0000 7f3395e00640 1 -- 172.21.15.86:0/802632306 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f336c06c8f0 msgr2=0x7f336c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:00.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.544+0000 7f3395e00640 1 --2- 172.21.15.86:0/802632306 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f336c06c8f0 0x7f336c06edb0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f3380000c00 tx=0x7f3380009040 comp rx=0 tx=0).stop 2024-04-20T21:26:00.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.544+0000 7f3395e00640 1 -- 172.21.15.86:0/802632306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33901435b0 msgr2=0x7f3390143a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:00.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.544+0000 7f3395e00640 1 --2- 172.21.15.86:0/802632306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33901435b0 0x7f3390143a30 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f3388002230 tx=0x7f3388001dc0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.544+0000 7f3395e00640 1 -- 172.21.15.86:0/802632306 shutdown_connections 2024-04-20T21:26:00.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.544+0000 7f3395e00640 1 --2- 172.21.15.86:0/802632306 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f336c06c8f0 0x7f336c06edb0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.544+0000 7f3395e00640 1 --2- 172.21.15.86:0/802632306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33901435b0 0x7f3390143a30 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.544+0000 7f3395e00640 1 --2- 172.21.15.86:0/802632306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33901f7730 0x7f33901f7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.544+0000 7f3395e00640 1 --2- 172.21.15.86:0/802632306 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f33740051c0 0x7f33901f6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.544+0000 7f3395e00640 1 -- 172.21.15.86:0/802632306 >> 172.21.15.86:0/802632306 conn(0x7f33900b2180 msgr2=0x7f33900ada30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:00.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.545+0000 7f3395e00640 1 -- 172.21.15.86:0/802632306 shutdown_connections 2024-04-20T21:26:00.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.545+0000 7f3395e00640 1 -- 172.21.15.86:0/802632306 wait complete. 2024-04-20T21:26:00.547 INFO:tasks.ceph.mds.b.smithi144.stderr:2024-04-20T21:26:00.544+0000 7f7416200640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 pv17 ap=3 DIRTYPARENT f(v0 m2024-04-20T21:25:07.713157+0000 1=1+0)->f(v1 m2024-04-20T21:25:07.713157+0000) n(v0 rc2024-04-20T21:25:07.851790+0000 2=1+1)->n(v0 rc2024-04-20T21:25:07.851790+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) | dirtyscattered=2 lock=1 dirfrag=1 dirtyrstat=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x561f42e0a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-20T21:25:07.713157+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-20T21:25:07.851790+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-04-20T21:26:00.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:26:00.560 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-20T21:26:00.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.754+0000 7f749ca00640 1 Processor -- start 2024-04-20T21:26:00.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.754+0000 7f749ca00640 1 -- start start 2024-04-20T21:26:00.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.756+0000 7f749ca00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f749815ecc0 0x7f74980d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:00.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.756+0000 7f749ca00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f74980d4ba0 0x7f74980cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:00.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.756+0000 7f749ca00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74980d1f60 con 0x7f749815e8e0 2024-04-20T21:26:00.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.756+0000 7f749ca00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f74980d4ff0 con 0x7f749815ecc0 2024-04-20T21:26:00.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.756+0000 7f749ca00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f74980cf7f0 con 0x7f74980d4ba0 2024-04-20T21:26:00.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.756+0000 7f748ea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f749815ecc0 0x7f74980d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:00.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.756+0000 7f748ea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f749815ecc0 0x7f74980d4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43062/0 (socket says 172.21.15.86:43062) 2024-04-20T21:26:00.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.756+0000 7f748ea00640 1 -- 172.21.15.86:0/2224611292 learned_addr learned my addr 172.21.15.86:0/2224611292 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:00.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.757+0000 7f748ea00640 1 -- 172.21.15.86:0/2224611292 >> v2:172.21.15.144:3300/0 conn(0x7f74980d4ba0 msgr2=0x7f74980cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:00.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.757+0000 7f748ea00640 1 --2- 172.21.15.86:0/2224611292 >> v2:172.21.15.144:3300/0 conn(0x7f74980d4ba0 0x7f74980cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.757+0000 7f748ea00640 1 -- 172.21.15.86:0/2224611292 >> v1:172.21.15.144:6789/0 conn(0x7f749815e8e0 legacy=0x7f749806d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:00.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.757+0000 7f748ea00640 1 -- 172.21.15.86:0/2224611292 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74980d19a0 con 0x7f749815ecc0 2024-04-20T21:26:00.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.758+0000 7f748ea00640 1 --2- 172.21.15.86:0/2224611292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f749815ecc0 0x7f74980d4660 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f74980cb750 tx=0x7f748002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20328af82bb9b803 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:00.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.758+0000 7f7496a00640 1 -- 172.21.15.86:0/2224611292 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7480035020 con 0x7f749815ecc0 2024-04-20T21:26:00.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.758+0000 7f7496a00640 1 -- 172.21.15.86:0/2224611292 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7480031870 con 0x7f749815ecc0 2024-04-20T21:26:00.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.758+0000 7f7496a00640 1 -- 172.21.15.86:0/2224611292 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7480031c40 con 0x7f749815ecc0 2024-04-20T21:26:00.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.759+0000 7f749ca00640 1 -- 172.21.15.86:0/2224611292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f749815ecc0 msgr2=0x7f74980d4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:00.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.759+0000 7f749ca00640 1 --2- 172.21.15.86:0/2224611292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f749815ecc0 0x7f74980d4660 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f74980cb750 tx=0x7f748002b910 comp rx=0 tx=0).stop 2024-04-20T21:26:00.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.759+0000 7f749ca00640 1 -- 172.21.15.86:0/2224611292 shutdown_connections 2024-04-20T21:26:00.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.759+0000 7f749ca00640 1 --2- 172.21.15.86:0/2224611292 >> v2:172.21.15.144:3300/0 conn(0x7f74980d4ba0 0x7f74980cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.759+0000 7f749ca00640 1 --2- 172.21.15.86:0/2224611292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f749815ecc0 0x7f74980d4660 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.759+0000 7f749ca00640 1 -- 172.21.15.86:0/2224611292 >> 172.21.15.86:0/2224611292 conn(0x7f749809d9d0 msgr2=0x7f749808bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:00.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.759+0000 7f749ca00640 1 -- 172.21.15.86:0/2224611292 shutdown_connections 2024-04-20T21:26:00.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.759+0000 7f749ca00640 1 -- 172.21.15.86:0/2224611292 wait complete. 2024-04-20T21:26:00.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.760+0000 7f749ca00640 1 Processor -- start 2024-04-20T21:26:00.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.761+0000 7f749ca00640 1 -- start start 2024-04-20T21:26:00.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.761+0000 7f749ca00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74980d4ba0 0x7f7498151d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:00.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.761+0000 7f749ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f749815e8e0 0x7f749815afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:00.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.761+0000 7f749ca00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74980cab90 0x7f749815b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:00.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.762+0000 7f749ca00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f749806d910 con 0x7f749815e8e0 2024-04-20T21:26:00.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.762+0000 7f749ca00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f74980cf480 con 0x7f74980cab90 2024-04-20T21:26:00.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.762+0000 7f749ca00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f74980d01c0 con 0x7f74980d4ba0 2024-04-20T21:26:00.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.762+0000 7f748ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f749815e8e0 0x7f749815afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:00.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.762+0000 7f748ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f749815e8e0 0x7f749815afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43070/0 (socket says 172.21.15.86:43070) 2024-04-20T21:26:00.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.762+0000 7f748ea00640 1 -- 172.21.15.86:0/3685703243 learned_addr learned my addr 172.21.15.86:0/3685703243 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:00.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.762+0000 7f748ea00640 1 -- 172.21.15.86:0/3685703243 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74980d4ba0 msgr2=0x7f7498151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:00.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.762+0000 7f748ea00640 1 --2- 172.21.15.86:0/3685703243 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74980d4ba0 0x7f7498151d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.762+0000 7f748ea00640 1 -- 172.21.15.86:0/3685703243 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74980cab90 msgr2=0x7f749815b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:00.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.762+0000 7f748ea00640 1 --2- 172.21.15.86:0/3685703243 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74980cab90 0x7f749815b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.762+0000 7f748ea00640 1 -- 172.21.15.86:0/3685703243 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74980c93b0 con 0x7f749815e8e0 2024-04-20T21:26:00.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.762+0000 7f748ea00640 1 --2- 172.21.15.86:0/3685703243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f749815e8e0 0x7f749815afe0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f7480000c00 tx=0x7f74800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:00.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.763+0000 7f748fe00640 1 -- 172.21.15.86:0/3685703243 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748003ad60 con 0x7f749815e8e0 2024-04-20T21:26:00.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.763+0000 7f748fe00640 1 -- 172.21.15.86:0/3685703243 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74800443f0 con 0x7f749815e8e0 2024-04-20T21:26:00.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.763+0000 7f748fe00640 1 -- 172.21.15.86:0/3685703243 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74800447e0 con 0x7f749815e8e0 2024-04-20T21:26:00.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.763+0000 7f749ca00640 1 -- 172.21.15.86:0/3685703243 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f749815fbf0 con 0x7f749815e8e0 2024-04-20T21:26:00.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.763+0000 7f749ca00640 1 -- 172.21.15.86:0/3685703243 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74981522a0 con 0x7f749815e8e0 2024-04-20T21:26:00.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.765+0000 7f748fe00640 1 -- 172.21.15.86:0/3685703243 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f748004b060 con 0x7f749815e8e0 2024-04-20T21:26:00.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.766+0000 7f748fe00640 1 --2- 172.21.15.86:0/3685703243 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f747406c8f0 0x7f747406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:00.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.766+0000 7f748fe00640 1 -- 172.21.15.86:0/3685703243 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7f74800add00 con 0x7f749815e8e0 2024-04-20T21:26:00.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.766+0000 7f7497400640 1 --2- 172.21.15.86:0/3685703243 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f747406c8f0 0x7f747406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:00.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.767+0000 7f7497400640 1 --2- 172.21.15.86:0/3685703243 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f747406c8f0 0x7f747406edb0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f7488001560 tx=0x7f7488007990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:00.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.767+0000 7f74a0e1d740 1 Processor -- start 2024-04-20T21:26:00.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.767+0000 7f74a0e1d740 1 -- start start 2024-04-20T21:26:00.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.768+0000 7f74a0e1d740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562dcf3713d0 0x562dcf3717b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:00.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.768+0000 7f74a0e1d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562dcf3413f0 0x562dcf36e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:00.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.768+0000 7f74a0e1d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562dcf344f50 0x562dcf36eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:00.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.768+0000 7f74a0e1d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562dcf2ad150 con 0x562dcf344f50 2024-04-20T21:26:00.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.768+0000 7f74a0e1d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562dcf253370 con 0x562dcf3413f0 2024-04-20T21:26:00.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.768+0000 7f74a0e1d740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x562dcf2a9110 con 0x562dcf3713d0 2024-04-20T21:26:00.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.768+0000 7f7497e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562dcf344f50 0x562dcf36eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:00.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.768+0000 7f7497e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562dcf344f50 0x562dcf36eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43074/0 (socket says 172.21.15.86:43074) 2024-04-20T21:26:00.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.768+0000 7f7497e00640 1 -- 172.21.15.86:0/362220717 learned_addr learned my addr 172.21.15.86:0/362220717 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:00.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.769+0000 7f748ea00640 1 --2- 172.21.15.86:0/362220717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562dcf3413f0 0x562dcf36e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:00.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.769+0000 7f7497e00640 1 -- 172.21.15.86:0/362220717 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562dcf3713d0 msgr2=0x562dcf3717b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:00.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.769+0000 7f7497400640 1 --2- 172.21.15.86:0/362220717 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562dcf3713d0 0x562dcf3717b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:00.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.769+0000 7f7497e00640 1 --2- 172.21.15.86:0/362220717 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562dcf3713d0 0x562dcf3717b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.769+0000 7f7497e00640 1 -- 172.21.15.86:0/362220717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562dcf3413f0 msgr2=0x562dcf36e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:00.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.769+0000 7f7497e00640 1 --2- 172.21.15.86:0/362220717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562dcf3413f0 0x562dcf36e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.769+0000 7f7497e00640 1 -- 172.21.15.86:0/362220717 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562dcf0897d0 con 0x562dcf344f50 2024-04-20T21:26:00.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.769+0000 7f748ea00640 1 --2- 172.21.15.86:0/362220717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562dcf3413f0 0x562dcf36e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:26:00.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.769+0000 7f7497e00640 1 --2- 172.21.15.86:0/362220717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562dcf344f50 0x562dcf36eb80 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f7490009d00 tx=0x7f749000b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c55d78f661ee340b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:00.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.769+0000 7f7487400640 1 -- 172.21.15.86:0/362220717 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749000bd40 con 0x562dcf344f50 2024-04-20T21:26:00.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.769+0000 7f7487400640 1 -- 172.21.15.86:0/362220717 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7490017070 con 0x562dcf344f50 2024-04-20T21:26:00.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.769+0000 7f7487400640 1 -- 172.21.15.86:0/362220717 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7490012670 con 0x562dcf344f50 2024-04-20T21:26:00.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.770+0000 7f74a0e1d740 1 -- 172.21.15.86:0/362220717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562dcf344f50 msgr2=0x562dcf36eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:00.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.770+0000 7f74a0e1d740 1 --2- 172.21.15.86:0/362220717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562dcf344f50 0x562dcf36eb80 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f7490009d00 tx=0x7f749000b420 comp rx=0 tx=0).stop 2024-04-20T21:26:00.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.770+0000 7f74a0e1d740 1 -- 172.21.15.86:0/362220717 shutdown_connections 2024-04-20T21:26:00.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.770+0000 7f74a0e1d740 1 --2- 172.21.15.86:0/362220717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562dcf344f50 0x562dcf36eb80 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.770+0000 7f74a0e1d740 1 --2- 172.21.15.86:0/362220717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562dcf3413f0 0x562dcf36e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.770+0000 7f74a0e1d740 1 --2- 172.21.15.86:0/362220717 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562dcf3713d0 0x562dcf3717b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.770+0000 7f74a0e1d740 1 -- 172.21.15.86:0/362220717 >> 172.21.15.86:0/362220717 conn(0x562dcf343660 msgr2=0x562dcf38c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:00.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.770+0000 7f74a0e1d740 1 -- 172.21.15.86:0/362220717 shutdown_connections 2024-04-20T21:26:00.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.770+0000 7f74a0e1d740 1 -- 172.21.15.86:0/362220717 wait complete. 2024-04-20T21:26:00.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.771+0000 7f74a0e1d740 1 -- start start 2024-04-20T21:26:00.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.771+0000 7f74a0e1d740 1 Processor -- start 2024-04-20T21:26:00.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.772+0000 7f74a0e1d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562dcf344f50 0x562dcf3777a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:00.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.772+0000 7f74a0e1d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562dcf3713d0 0x562dcf3729d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:00.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.772+0000 7f74a0e1d740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562dcf378330 0x562dcf3787b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:00.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.772+0000 7f74a0e1d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562dcf346100 con 0x562dcf3713d0 2024-04-20T21:26:00.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.772+0000 7f74a0e1d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562dcf266740 con 0x562dcf344f50 2024-04-20T21:26:00.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.772+0000 7f74a0e1d740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x562dcf3454f0 con 0x562dcf378330 2024-04-20T21:26:00.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.772+0000 7f74a0e1d740 10 client.0 _ll_register_callbacks cb 0x562dcf261b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:26:00.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.772+0000 7f748ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562dcf3713d0 0x562dcf3729d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:00.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.772+0000 7f748ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562dcf3713d0 0x562dcf3729d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43078/0 (socket says 172.21.15.86:43078) 2024-04-20T21:26:00.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.772+0000 7f748ea00640 1 -- 172.21.15.86:0/2283398546 learned_addr learned my addr 172.21.15.86:0/2283398546 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:00.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.772+0000 7f748ea00640 1 -- 172.21.15.86:0/2283398546 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562dcf378330 msgr2=0x562dcf3787b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:00.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.772+0000 7f748ea00640 1 --2- 172.21.15.86:0/2283398546 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562dcf378330 0x562dcf3787b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.772+0000 7f748ea00640 1 -- 172.21.15.86:0/2283398546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562dcf344f50 msgr2=0x562dcf3777a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:00.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.773+0000 7f7497400640 1 --2- 172.21.15.86:0/2283398546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562dcf344f50 0x562dcf3777a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:00.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.773+0000 7f748ea00640 1 --2- 172.21.15.86:0/2283398546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562dcf344f50 0x562dcf3777a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.773+0000 7f748ea00640 1 -- 172.21.15.86:0/2283398546 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562dcefbf3f0 con 0x562dcf3713d0 2024-04-20T21:26:00.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.773+0000 7f748ea00640 1 --2- 172.21.15.86:0/2283398546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562dcf3713d0 0x562dcf3729d0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f748002b950 tx=0x7f748007b3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1af83760d017025b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:00.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.773+0000 7f747be00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:26:00.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.773+0000 7f747be00640 1 -- 172.21.15.86:0/2283398546 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74800aa060 con 0x562dcf3713d0 2024-04-20T21:26:00.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.773+0000 7f747be00640 1 -- 172.21.15.86:0/2283398546 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7f748007bc30 con 0x562dcf3713d0 2024-04-20T21:26:00.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.773+0000 7f74a0e1d740 1 -- 172.21.15.86:0/2283398546 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7490017070 con 0x562dcf3713d0 2024-04-20T21:26:00.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.773+0000 7f747be00640 1 -- 172.21.15.86:0/2283398546 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f748007fd80 con 0x562dcf3713d0 2024-04-20T21:26:00.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.773+0000 7f74a0e1d740 10 client.7974 fetch_fsmap learned FSMap version 66 2024-04-20T21:26:00.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.773+0000 7f74a0e1d740 1 -- 172.21.15.86:0/2283398546 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x562dcef7f0f0 con 0x562dcf3713d0 2024-04-20T21:26:00.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.773+0000 7f747be00640 1 -- 172.21.15.86:0/2283398546 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7490017070 con 0x562dcf3713d0 2024-04-20T21:26:00.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.774+0000 7f747be00640 1 -- 172.21.15.86:0/2283398546 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 66) ==== 3495+0+0 (secure 0 0 0) 0x7f748009c070 con 0x562dcf3713d0 2024-04-20T21:26:00.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.774+0000 7f74a0e1d740 10 client.7974 fetch_fsmap finished waiting for FSMap version 66 2024-04-20T21:26:00.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.774+0000 7f74a0e1d740 10 client.7974 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-04-20T21:26:00.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.774+0000 7f74a0e1d740 20 client.7974 populate_metadata read hostname 'smithi086' 2024-04-20T21:26:00.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.774+0000 7f74a0e1d740 1 --2- 172.21.15.86:0/2283398546 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x562dcf3a0070 0x562dcf3a0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:00.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.774+0000 7f74a0e1d740 4 client.7974 mds_command: new command op to 7773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:26:00.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.774+0000 7f74a0e1d740 1 -- 172.21.15.86:0/2283398546 --> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562dcf2bab70 con 0x562dcf3a0070 2024-04-20T21:26:00.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.774+0000 7f7497400640 1 --2- 172.21.15.86:0/2283398546 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x562dcf3a0070 0x562dcf3a0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:00.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.775+0000 7f7497400640 1 --2- 172.21.15.86:0/2283398546 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x562dcf3a0070 0x562dcf3a0520 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfd70723fcd24e60 server_cookie=852cd556f023844c in_seq=0 out_seq=0 2024-04-20T21:26:00.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.775+0000 7f747be00640 10 client.7974 ms_handle_connect on v2:172.21.15.144:6833/3981403733 2024-04-20T21:26:00.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.777+0000 7f747be00640 1 -- 172.21.15.86:0/2283398546 <== mds.0 v2:172.21.15.144:6833/3981403733 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562dcf2bab70 con 0x562dcf3a0070 2024-04-20T21:26:00.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.777+0000 7f747be00640 10 client.7974 handle_command_reply: tid=0 2024-04-20T21:26:00.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.777+0000 7f74a0e1d740 1 client.7974 shutdown 2024-04-20T21:26:00.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.777+0000 7f74a0e1d740 1 -- 172.21.15.86:0/2283398546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562dcf3713d0 msgr2=0x562dcf3729d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:00.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.777+0000 7f74a0e1d740 1 --2- 172.21.15.86:0/2283398546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562dcf3713d0 0x562dcf3729d0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f748002b950 tx=0x7f748007b3f0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.777+0000 7f74a0e1d740 1 -- 172.21.15.86:0/2283398546 shutdown_connections 2024-04-20T21:26:00.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.777+0000 7f74a0e1d740 1 --2- 172.21.15.86:0/2283398546 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x562dcf3a0070 0x562dcf3a0520 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.777+0000 7f74a0e1d740 1 --2- 172.21.15.86:0/2283398546 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562dcf378330 0x562dcf3787b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.777+0000 7f74a0e1d740 1 --2- 172.21.15.86:0/2283398546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562dcf3713d0 0x562dcf3729d0 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.777+0000 7f74a0e1d740 1 --2- 172.21.15.86:0/2283398546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562dcf344f50 0x562dcf3777a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.777+0000 7f747be00640 0 client.7974 ms_handle_reset on v2:172.21.15.144:6833/3981403733 2024-04-20T21:26:00.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.777+0000 7f74a0e1d740 1 -- 172.21.15.86:0/2283398546 >> 172.21.15.86:0/2283398546 conn(0x562dcf3413f0 msgr2=0x562dcf347530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:00.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.778+0000 7f74a0e1d740 1 -- 172.21.15.86:0/2283398546 shutdown_connections 2024-04-20T21:26:00.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.778+0000 7f74a0e1d740 1 -- 172.21.15.86:0/2283398546 wait complete. 2024-04-20T21:26:00.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.778+0000 7f74a0e1d740 20 client.7974 trim_cache size 0 max 16384 2024-04-20T21:26:00.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.796+0000 7f74a0e1d740 1 Processor -- start 2024-04-20T21:26:00.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.796+0000 7f74a0e1d740 1 -- start start 2024-04-20T21:26:00.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.797+0000 7f74a0e1d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562dcf344f50 0x562dcf38cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:00.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.797+0000 7f74a0e1d740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562dcf3713d0 0x562dcf38ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:00.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.797+0000 7f74a0e1d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562dcf378330 0x562dcf38f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:00.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.797+0000 7f74a0e1d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7498200e80 con 0x562dcf344f50 2024-04-20T21:26:00.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.797+0000 7f74a0e1d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7498161700 con 0x562dcf378330 2024-04-20T21:26:00.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.797+0000 7f74a0e1d740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f74980c9050 con 0x562dcf3713d0 2024-04-20T21:26:00.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.797+0000 7f7497400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562dcf344f50 0x562dcf38cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:00.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.797+0000 7f7497400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562dcf344f50 0x562dcf38cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43094/0 (socket says 172.21.15.86:43094) 2024-04-20T21:26:00.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.797+0000 7f7497400640 1 -- 172.21.15.86:0/2431628071 learned_addr learned my addr 172.21.15.86:0/2431628071 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:00.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.797+0000 7f7497400640 1 -- 172.21.15.86:0/2431628071 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562dcf3713d0 msgr2=0x562dcf38ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:00.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.797+0000 7f7497400640 1 --2- 172.21.15.86:0/2431628071 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562dcf3713d0 0x562dcf38ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.797+0000 7f7497400640 1 -- 172.21.15.86:0/2431628071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562dcf378330 msgr2=0x562dcf38f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:00.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.797+0000 7f7497400640 1 --2- 172.21.15.86:0/2431628071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562dcf378330 0x562dcf38f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.797+0000 7f7497400640 1 -- 172.21.15.86:0/2431628071 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562dcf371b20 con 0x562dcf344f50 2024-04-20T21:26:00.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.797+0000 7f7497400640 1 --2- 172.21.15.86:0/2431628071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562dcf344f50 0x562dcf38cc40 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f7488003620 tx=0x7f7488010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3106f6c9bc4d69fb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:00.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.798+0000 7f7486a00640 1 -- 172.21.15.86:0/2431628071 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7488029070 con 0x562dcf344f50 2024-04-20T21:26:00.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.798+0000 7f7486a00640 1 -- 172.21.15.86:0/2431628071 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7488003e50 con 0x562dcf344f50 2024-04-20T21:26:00.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.798+0000 7f7486a00640 1 -- 172.21.15.86:0/2431628071 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7488007290 con 0x562dcf344f50 2024-04-20T21:26:00.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.798+0000 7f74a0e1d740 1 -- 172.21.15.86:0/2431628071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562dcf344f50 msgr2=0x562dcf38cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:00.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.798+0000 7f74a0e1d740 1 --2- 172.21.15.86:0/2431628071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562dcf344f50 0x562dcf38cc40 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f7488003620 tx=0x7f7488010000 comp rx=0 tx=0).stop 2024-04-20T21:26:00.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.798+0000 7f74a0e1d740 1 -- 172.21.15.86:0/2431628071 shutdown_connections 2024-04-20T21:26:00.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.798+0000 7f74a0e1d740 1 --2- 172.21.15.86:0/2431628071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562dcf378330 0x562dcf38f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.798+0000 7f74a0e1d740 1 --2- 172.21.15.86:0/2431628071 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562dcf3713d0 0x562dcf38ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.798+0000 7f74a0e1d740 1 --2- 172.21.15.86:0/2431628071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562dcf344f50 0x562dcf38cc40 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.798+0000 7f74a0e1d740 1 -- 172.21.15.86:0/2431628071 >> 172.21.15.86:0/2431628071 conn(0x562dcf3413f0 msgr2=0x562dcf39ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:00.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.798+0000 7f74a0e1d740 1 -- 172.21.15.86:0/2431628071 shutdown_connections 2024-04-20T21:26:00.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.798+0000 7f74a0e1d740 1 -- 172.21.15.86:0/2431628071 wait complete. 2024-04-20T21:26:00.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.799+0000 7f74a0e1d740 1 -- start start 2024-04-20T21:26:00.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.799+0000 7f74a0e1d740 1 Processor -- start 2024-04-20T21:26:00.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.800+0000 7f74a0e1d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562dcf344f50 0x562dcf3af030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:00.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.800+0000 7f74a0e1d740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562dcf3713d0 0x562dcf3af570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:00.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.800+0000 7f74a0e1d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562dcf378330 0x562dcf39c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:00.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.800+0000 7f74a0e1d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562dcf379740 con 0x562dcf378330 2024-04-20T21:26:00.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.800+0000 7f74a0e1d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f74981610b0 con 0x562dcf344f50 2024-04-20T21:26:00.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.800+0000 7f74a0e1d740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x562dcf38e9b0 con 0x562dcf3713d0 2024-04-20T21:26:00.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.800+0000 7f74a0e1d740 10 client.0 _ll_register_callbacks cb 0x7f7480006cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:26:00.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.800+0000 7f7497e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562dcf378330 0x562dcf39c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:00.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.800+0000 7f7497e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562dcf378330 0x562dcf39c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43096/0 (socket says 172.21.15.86:43096) 2024-04-20T21:26:00.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.800+0000 7f7497e00640 1 -- 172.21.15.86:0/3677270050 learned_addr learned my addr 172.21.15.86:0/3677270050 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:00.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.801+0000 7f7497e00640 1 -- 172.21.15.86:0/3677270050 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562dcf3713d0 msgr2=0x562dcf3af570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:00.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.801+0000 7f7497400640 1 --2- 172.21.15.86:0/3677270050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562dcf344f50 0x562dcf3af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:00.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.801+0000 7f7497e00640 1 --2- 172.21.15.86:0/3677270050 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562dcf3713d0 0x562dcf3af570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.801+0000 7f7497e00640 1 -- 172.21.15.86:0/3677270050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562dcf344f50 msgr2=0x562dcf3af030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:00.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.801+0000 7f7497e00640 1 --2- 172.21.15.86:0/3677270050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562dcf344f50 0x562dcf3af030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.801+0000 7f7497e00640 1 -- 172.21.15.86:0/3677270050 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562dcf36e840 con 0x562dcf378330 2024-04-20T21:26:00.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.801+0000 7f7497e00640 1 --2- 172.21.15.86:0/3677270050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562dcf378330 0x562dcf39c750 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f7490000f80 tx=0x7f7490012730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f254646d3bd6998 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:00.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.801+0000 7f747be00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:26:00.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.801+0000 7f747be00640 1 -- 172.21.15.86:0/3677270050 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7490022020 con 0x562dcf378330 2024-04-20T21:26:00.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.801+0000 7f747be00640 1 -- 172.21.15.86:0/3677270050 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7f7490012c30 con 0x562dcf378330 2024-04-20T21:26:00.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.801+0000 7f74a0e1d740 1 -- 172.21.15.86:0/3677270050 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7488003e50 con 0x562dcf378330 2024-04-20T21:26:00.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.802+0000 7f747be00640 1 -- 172.21.15.86:0/3677270050 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f7490012e30 con 0x562dcf378330 2024-04-20T21:26:00.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.802+0000 7f74a0e1d740 10 client.7980 fetch_fsmap learned FSMap version 66 2024-04-20T21:26:00.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.802+0000 7f74a0e1d740 1 -- 172.21.15.86:0/3677270050 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x562dcf3b06e0 con 0x562dcf378330 2024-04-20T21:26:00.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.802+0000 7f747be00640 1 -- 172.21.15.86:0/3677270050 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7488003e50 con 0x562dcf378330 2024-04-20T21:26:00.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.802+0000 7f747be00640 1 -- 172.21.15.86:0/3677270050 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 66) ==== 3495+0+0 (secure 0 0 0) 0x7f7490011070 con 0x562dcf378330 2024-04-20T21:26:00.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.802+0000 7f74a0e1d740 10 client.7980 fetch_fsmap finished waiting for FSMap version 66 2024-04-20T21:26:00.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.802+0000 7f74a0e1d740 10 client.7980 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-04-20T21:26:00.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.802+0000 7f74a0e1d740 20 client.7980 populate_metadata read hostname 'smithi086' 2024-04-20T21:26:00.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.802+0000 7f74a0e1d740 1 --2- 172.21.15.86:0/3677270050 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x562dcf375390 0x562dcf39cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:00.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.802+0000 7f74a0e1d740 4 client.7980 mds_command: new command op to 7773 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:26:00.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.802+0000 7f74a0e1d740 1 -- 172.21.15.86:0/3677270050 --> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562dcf2b49e0 con 0x562dcf375390 2024-04-20T21:26:00.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.802+0000 7f7497400640 1 --2- 172.21.15.86:0/3677270050 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x562dcf375390 0x562dcf39cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:00.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.803+0000 7f7497400640 1 --2- 172.21.15.86:0/3677270050 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x562dcf375390 0x562dcf39cc90 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2ee2e1efa76ecdf server_cookie=9aabe1a52a29548a in_seq=0 out_seq=0 2024-04-20T21:26:00.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.804+0000 7f747be00640 10 client.7980 ms_handle_connect on v2:172.21.15.144:6833/3981403733 2024-04-20T21:26:00.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.804+0000 7f747be00640 1 -- 172.21.15.86:0/3677270050 <== mds.0 v2:172.21.15.144:6833/3981403733 1 ==== command_reply(tid 0: 0 ) ==== 8+0+263 (crc 0 0 0) 0x562dcf2b49e0 con 0x562dcf375390 2024-04-20T21:26:00.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.804+0000 7f747be00640 10 client.7980 handle_command_reply: tid=0 2024-04-20T21:26:00.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.804+0000 7f74a0e1d740 1 client.7980 shutdown 2024-04-20T21:26:00.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.804+0000 7f74a0e1d740 1 -- 172.21.15.86:0/3677270050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562dcf378330 msgr2=0x562dcf39c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:00.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.804+0000 7f74a0e1d740 1 --2- 172.21.15.86:0/3677270050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562dcf378330 0x562dcf39c750 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f7490000f80 tx=0x7f7490012730 comp rx=0 tx=0).stop 2024-04-20T21:26:00.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.804+0000 7f74a0e1d740 1 -- 172.21.15.86:0/3677270050 shutdown_connections 2024-04-20T21:26:00.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.804+0000 7f74a0e1d740 1 --2- 172.21.15.86:0/3677270050 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x562dcf375390 0x562dcf39cc90 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.804+0000 7f74a0e1d740 1 --2- 172.21.15.86:0/3677270050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562dcf378330 0x562dcf39c750 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.804+0000 7f74a0e1d740 1 --2- 172.21.15.86:0/3677270050 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562dcf3713d0 0x562dcf3af570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.804+0000 7f74a0e1d740 1 --2- 172.21.15.86:0/3677270050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562dcf344f50 0x562dcf3af030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.804+0000 7f747be00640 0 client.7980 ms_handle_reset on v2:172.21.15.144:6833/3981403733 2024-04-20T21:26:00.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.804+0000 7f74a0e1d740 1 -- 172.21.15.86:0/3677270050 >> 172.21.15.86:0/3677270050 conn(0x562dcf3413f0 msgr2=0x562dcf38cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:00.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.804+0000 7f74a0e1d740 1 -- 172.21.15.86:0/3677270050 shutdown_connections 2024-04-20T21:26:00.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.804+0000 7f74a0e1d740 1 -- 172.21.15.86:0/3677270050 wait complete. 2024-04-20T21:26:00.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.805+0000 7f74a0e1d740 20 client.7980 trim_cache size 0 max 16384 2024-04-20T21:26:00.808 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:26:00.808 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T21:26:00.808 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-20T21:26:00.808 INFO:teuthology.orchestra.run.smithi086.stdout: "e2a76b86-8e01-4180-8000-777ebcbc1af9": { 2024-04-20T21:26:00.808 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/dir", 2024-04-20T21:26:00.808 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "e2a76b86-8e01-4180-8000-777ebcbc1af9", 2024-04-20T21:26:00.808 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,repair" 2024-04-20T21:26:00.808 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:26:00.808 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:26:00.808 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:26:00.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.805+0000 7f749ca00640 1 -- 172.21.15.86:0/3685703243 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f747406c8f0 msgr2=0x7f747406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:00.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.805+0000 7f749ca00640 1 --2- 172.21.15.86:0/3685703243 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f747406c8f0 0x7f747406edb0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f7488001560 tx=0x7f7488007990 comp rx=0 tx=0).stop 2024-04-20T21:26:00.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.805+0000 7f749ca00640 1 -- 172.21.15.86:0/3685703243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f749815e8e0 msgr2=0x7f749815afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:00.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.805+0000 7f749ca00640 1 --2- 172.21.15.86:0/3685703243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f749815e8e0 0x7f749815afe0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f7480000c00 tx=0x7f74800046c0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.805+0000 7f749ca00640 1 -- 172.21.15.86:0/3685703243 shutdown_connections 2024-04-20T21:26:00.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.805+0000 7f749ca00640 1 --2- 172.21.15.86:0/3685703243 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f747406c8f0 0x7f747406edb0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.805+0000 7f749ca00640 1 --2- 172.21.15.86:0/3685703243 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74980cab90 0x7f749815b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.805+0000 7f749ca00640 1 --2- 172.21.15.86:0/3685703243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f749815e8e0 0x7f749815afe0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.805+0000 7f749ca00640 1 --2- 172.21.15.86:0/3685703243 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74980d4ba0 0x7f7498151d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:00.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.805+0000 7f749ca00640 1 -- 172.21.15.86:0/3685703243 >> 172.21.15.86:0/3685703243 conn(0x7f749809d9d0 msgr2=0x7f749808bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:00.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.805+0000 7f749ca00640 1 -- 172.21.15.86:0/3685703243 shutdown_connections 2024-04-20T21:26:00.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:00.805+0000 7f749ca00640 1 -- 172.21.15.86:0/3685703243 wait complete. 2024-04-20T21:26:00.819 INFO:tasks.cephfs.filesystem:scrub status for tag:e2a76b86-8e01-4180-8000-777ebcbc1af9 - {'path': '/dir', 'tag': 'e2a76b86-8e01-4180-8000-777ebcbc1af9', 'options': 'recursive,repair'} 2024-04-20T21:26:17.438 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:26:17.447 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:26:30.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:26:30.821 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-20T21:26:31.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.033+0000 7fe3d0c00640 1 Processor -- start 2024-04-20T21:26:31.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.033+0000 7fe3d0c00640 1 -- start start 2024-04-20T21:26:31.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.035+0000 7fe3d0c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe3cc162ed0 0x7fe3cc06b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:31.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.035+0000 7fe3d0c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe3cc0ce370 0x7fe3cc06bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:31.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.035+0000 7fe3d0c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3cc0b3910 con 0x7fe3cc0b3410 2024-04-20T21:26:31.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.035+0000 7fe3d0c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe3cc0b3bf0 con 0x7fe3cc0ce370 2024-04-20T21:26:31.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.035+0000 7fe3d0c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fe3cc0b3ed0 con 0x7fe3cc162ed0 2024-04-20T21:26:31.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.035+0000 7fe3ca800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe3cc0ce370 0x7fe3cc06bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:31.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.035+0000 7fe3ca800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe3cc0ce370 0x7fe3cc06bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43672/0 (socket says 172.21.15.86:43672) 2024-04-20T21:26:31.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.035+0000 7fe3ca800640 1 -- 172.21.15.86:0/2046941281 learned_addr learned my addr 172.21.15.86:0/2046941281 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:31.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.036+0000 7fe3c9e00640 1 --2- 172.21.15.86:0/2046941281 >> v2:172.21.15.144:3300/0 conn(0x7fe3cc162ed0 0x7fe3cc06b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:31.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.036+0000 7fe3ca800640 1 -- 172.21.15.86:0/2046941281 >> v2:172.21.15.144:3300/0 conn(0x7fe3cc162ed0 msgr2=0x7fe3cc06b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:31.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.036+0000 7fe3ca800640 1 --2- 172.21.15.86:0/2046941281 >> v2:172.21.15.144:3300/0 conn(0x7fe3cc162ed0 0x7fe3cc06b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.036+0000 7fe3ca800640 1 -- 172.21.15.86:0/2046941281 >> v1:172.21.15.144:6789/0 conn(0x7fe3cc0b3410 legacy=0x7fe3cc071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:31.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.036+0000 7fe3ca800640 1 -- 172.21.15.86:0/2046941281 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3cc0b41b0 con 0x7fe3cc0ce370 2024-04-20T21:26:31.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.037+0000 7fe3ca800640 1 --2- 172.21.15.86:0/2046941281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3cc0ce370 0x7fe3cc06bec0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc0b58e0 tx=0x7fe3c402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e26d97ab084e12d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:31.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.037+0000 7fe3c8a00640 1 -- 172.21.15.86:0/2046941281 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c402cb20 con 0x7fe3cc0ce370 2024-04-20T21:26:31.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.037+0000 7fe3c8a00640 1 -- 172.21.15.86:0/2046941281 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3c402ccc0 con 0x7fe3cc0ce370 2024-04-20T21:26:31.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.037+0000 7fe3c8a00640 1 -- 172.21.15.86:0/2046941281 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c4034610 con 0x7fe3cc0ce370 2024-04-20T21:26:31.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.038+0000 7fe3d0c00640 1 -- 172.21.15.86:0/2046941281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3cc0ce370 msgr2=0x7fe3cc06bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:31.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.038+0000 7fe3d0c00640 1 --2- 172.21.15.86:0/2046941281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3cc0ce370 0x7fe3cc06bec0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc0b58e0 tx=0x7fe3c402bf80 comp rx=0 tx=0).stop 2024-04-20T21:26:31.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.038+0000 7fe3d0c00640 1 -- 172.21.15.86:0/2046941281 shutdown_connections 2024-04-20T21:26:31.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.038+0000 7fe3d0c00640 1 --2- 172.21.15.86:0/2046941281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3cc0ce370 0x7fe3cc06bec0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.038+0000 7fe3d0c00640 1 --2- 172.21.15.86:0/2046941281 >> v2:172.21.15.144:3300/0 conn(0x7fe3cc162ed0 0x7fe3cc06b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.038+0000 7fe3d0c00640 1 -- 172.21.15.86:0/2046941281 >> 172.21.15.86:0/2046941281 conn(0x7fe3cc0b2110 msgr2=0x7fe3cc0b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:31.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.038+0000 7fe3d0c00640 1 -- 172.21.15.86:0/2046941281 shutdown_connections 2024-04-20T21:26:31.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.038+0000 7fe3d0c00640 1 -- 172.21.15.86:0/2046941281 wait complete. 2024-04-20T21:26:31.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.040+0000 7fe3d0c00640 1 Processor -- start 2024-04-20T21:26:31.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.040+0000 7fe3d0c00640 1 -- start start 2024-04-20T21:26:31.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.041+0000 7fe3d0c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3cc0b3410 0x7fe3cc163f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:31.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.041+0000 7fe3d0c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3cc0ce370 0x7fe3cc15bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:31.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.041+0000 7fe3d0c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe3cc162ed0 0x7fe3cc164d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:31.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.041+0000 7fe3d0c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe3cc06fca0 con 0x7fe3cc0b3410 2024-04-20T21:26:31.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.041+0000 7fe3d0c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe3cc06fb20 con 0x7fe3cc0ce370 2024-04-20T21:26:31.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.041+0000 7fe3d0c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe3cc06e160 con 0x7fe3cc162ed0 2024-04-20T21:26:31.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.041+0000 7fe3c9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3cc0b3410 0x7fe3cc163f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:31.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.041+0000 7fe3c9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3cc0b3410 0x7fe3cc163f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43674/0 (socket says 172.21.15.86:43674) 2024-04-20T21:26:31.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.041+0000 7fe3c9e00640 1 -- 172.21.15.86:0/4218850747 learned_addr learned my addr 172.21.15.86:0/4218850747 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:31.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.042+0000 7fe3c9400640 1 --2- 172.21.15.86:0/4218850747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3cc0ce370 0x7fe3cc15bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:31.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.042+0000 7fe3ca800640 1 --2- 172.21.15.86:0/4218850747 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe3cc162ed0 0x7fe3cc164d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:31.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.042+0000 7fe3c9e00640 1 -- 172.21.15.86:0/4218850747 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe3cc162ed0 msgr2=0x7fe3cc164d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:31.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.042+0000 7fe3c9e00640 1 --2- 172.21.15.86:0/4218850747 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe3cc162ed0 0x7fe3cc164d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.042+0000 7fe3c9e00640 1 -- 172.21.15.86:0/4218850747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3cc0ce370 msgr2=0x7fe3cc15bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:31.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.042+0000 7fe3c9e00640 1 --2- 172.21.15.86:0/4218850747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3cc0ce370 0x7fe3cc15bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.042+0000 7fe3c9e00640 1 -- 172.21.15.86:0/4218850747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3cc165840 con 0x7fe3cc0b3410 2024-04-20T21:26:31.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.042+0000 7fe3c9e00640 1 --2- 172.21.15.86:0/4218850747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3cc0b3410 0x7fe3cc163f50 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc00d5f0 tx=0x7fe3bc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:31.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.042+0000 7fe3c2000640 1 -- 172.21.15.86:0/4218850747 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc0125f0 con 0x7fe3cc0b3410 2024-04-20T21:26:31.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.043+0000 7fe3c2000640 1 -- 172.21.15.86:0/4218850747 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3bc019070 con 0x7fe3cc0b3410 2024-04-20T21:26:31.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.043+0000 7fe3d0c00640 1 -- 172.21.15.86:0/4218850747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3cc163470 con 0x7fe3cc0b3410 2024-04-20T21:26:31.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.043+0000 7fe3c2000640 1 -- 172.21.15.86:0/4218850747 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc0129d0 con 0x7fe3cc0b3410 2024-04-20T21:26:31.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.043+0000 7fe3d0c00640 1 -- 172.21.15.86:0/4218850747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3cc163a30 con 0x7fe3cc0b3410 2024-04-20T21:26:31.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.045+0000 7fe3c2000640 1 -- 172.21.15.86:0/4218850747 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe3bc012b70 con 0x7fe3cc0b3410 2024-04-20T21:26:31.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.046+0000 7fe3c2000640 1 --2- 172.21.15.86:0/4218850747 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe3ac06c8f0 0x7fe3ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:31.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.046+0000 7fe3c2000640 1 -- 172.21.15.86:0/4218850747 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7fe3bc08bfe0 con 0x7fe3cc0b3410 2024-04-20T21:26:31.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.046+0000 7fe3c9400640 1 --2- 172.21.15.86:0/4218850747 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe3ac06c8f0 0x7fe3ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:31.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.047+0000 7fe3d39ac740 1 Processor -- start 2024-04-20T21:26:31.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.047+0000 7fe3d39ac740 1 -- start start 2024-04-20T21:26:31.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.047+0000 7fe3c9400640 1 --2- 172.21.15.86:0/4218850747 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe3ac06c8f0 0x7fe3ac06edb0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc15c9d0 tx=0x7fe3b8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:31.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.048+0000 7fe3d39ac740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ef11ca1f0 0x555ef11ca5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:31.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.048+0000 7fe3d39ac740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555ef119a210 0x555ef11c7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:31.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.048+0000 7fe3d39ac740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ef119dd70 0x555ef11c79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:31.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.048+0000 7fe3d39ac740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555ef1106150 con 0x555ef119dd70 2024-04-20T21:26:31.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.048+0000 7fe3d39ac740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555ef10ac370 con 0x555ef11ca1f0 2024-04-20T21:26:31.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.048+0000 7fe3d39ac740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x555ef1102110 con 0x555ef119a210 2024-04-20T21:26:31.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.048+0000 7fe3ca800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ef119dd70 0x555ef11c79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:31.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.048+0000 7fe3ca800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ef119dd70 0x555ef11c79a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43678/0 (socket says 172.21.15.86:43678) 2024-04-20T21:26:31.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.048+0000 7fe3ca800640 1 -- 172.21.15.86:0/2641514021 learned_addr learned my addr 172.21.15.86:0/2641514021 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:31.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.048+0000 7fe3ca800640 1 -- 172.21.15.86:0/2641514021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555ef119a210 msgr2=0x555ef11c7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:31.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.048+0000 7fe3c9e00640 1 --2- 172.21.15.86:0/2641514021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ef11ca1f0 0x555ef11ca5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:31.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.048+0000 7fe3ca800640 1 --2- 172.21.15.86:0/2641514021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555ef119a210 0x555ef11c7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.048+0000 7fe3ca800640 1 -- 172.21.15.86:0/2641514021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ef11ca1f0 msgr2=0x555ef11ca5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:31.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.049+0000 7fe3ca800640 1 --2- 172.21.15.86:0/2641514021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ef11ca1f0 0x555ef11ca5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.049+0000 7fe3ca800640 1 -- 172.21.15.86:0/2641514021 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ef0ee27d0 con 0x555ef119dd70 2024-04-20T21:26:31.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.049+0000 7fe3c9e00640 1 --2- 172.21.15.86:0/2641514021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ef11ca1f0 0x555ef11ca5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:26:31.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.049+0000 7fe3ca800640 1 --2- 172.21.15.86:0/2641514021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ef119dd70 0x555ef11c79a0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4009f40 tx=0x7fe3c402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d8b69236119eeb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:31.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.049+0000 7fe3b6600640 1 -- 172.21.15.86:0/2641514021 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c4045070 con 0x555ef119dd70 2024-04-20T21:26:31.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.049+0000 7fe3b6600640 1 -- 172.21.15.86:0/2641514021 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3c402cb20 con 0x555ef119dd70 2024-04-20T21:26:31.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.049+0000 7fe3b6600640 1 -- 172.21.15.86:0/2641514021 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c402ccd0 con 0x555ef119dd70 2024-04-20T21:26:31.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.050+0000 7fe3d39ac740 1 -- 172.21.15.86:0/2641514021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ef119dd70 msgr2=0x555ef11c79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:31.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.050+0000 7fe3d39ac740 1 --2- 172.21.15.86:0/2641514021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ef119dd70 0x555ef11c79a0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4009f40 tx=0x7fe3c402e000 comp rx=0 tx=0).stop 2024-04-20T21:26:31.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.050+0000 7fe3d39ac740 1 -- 172.21.15.86:0/2641514021 shutdown_connections 2024-04-20T21:26:31.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.050+0000 7fe3d39ac740 1 --2- 172.21.15.86:0/2641514021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ef119dd70 0x555ef11c79a0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.050+0000 7fe3d39ac740 1 --2- 172.21.15.86:0/2641514021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555ef119a210 0x555ef11c7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.050+0000 7fe3d39ac740 1 --2- 172.21.15.86:0/2641514021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ef11ca1f0 0x555ef11ca5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.050+0000 7fe3d39ac740 1 -- 172.21.15.86:0/2641514021 >> 172.21.15.86:0/2641514021 conn(0x555ef119c480 msgr2=0x555ef11e5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:31.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.050+0000 7fe3d39ac740 1 -- 172.21.15.86:0/2641514021 shutdown_connections 2024-04-20T21:26:31.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.050+0000 7fe3d39ac740 1 -- 172.21.15.86:0/2641514021 wait complete. 2024-04-20T21:26:31.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.051+0000 7fe3d39ac740 1 -- start start 2024-04-20T21:26:31.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.051+0000 7fe3d39ac740 1 Processor -- start 2024-04-20T21:26:31.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.052+0000 7fe3d39ac740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ef119dd70 0x555ef11cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:31.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.052+0000 7fe3d39ac740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ef11ca1f0 0x555ef11f8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:31.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.052+0000 7fe3d39ac740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555ef11cc590 0x555ef11c8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:31.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.052+0000 7fe3d39ac740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555ef119ef20 con 0x555ef11ca1f0 2024-04-20T21:26:31.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.052+0000 7fe3d39ac740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555ef10bf740 con 0x555ef119dd70 2024-04-20T21:26:31.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.052+0000 7fe3d39ac740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x555ef119e310 con 0x555ef11cc590 2024-04-20T21:26:31.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.052+0000 7fe3d39ac740 10 client.0 _ll_register_callbacks cb 0x555ef10bab20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:26:31.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.052+0000 7fe3c9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ef11ca1f0 0x555ef11f8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:31.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.052+0000 7fe3c9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ef11ca1f0 0x555ef11f8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43692/0 (socket says 172.21.15.86:43692) 2024-04-20T21:26:31.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.052+0000 7fe3c9400640 1 -- 172.21.15.86:0/2881768988 learned_addr learned my addr 172.21.15.86:0/2881768988 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:31.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.052+0000 7fe3c9e00640 1 --2- 172.21.15.86:0/2881768988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ef119dd70 0x555ef11cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:31.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.052+0000 7fe3ca800640 1 --2- 172.21.15.86:0/2881768988 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555ef11cc590 0x555ef11c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:31.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.053+0000 7fe3c9400640 1 -- 172.21.15.86:0/2881768988 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555ef11cc590 msgr2=0x555ef11c8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:31.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.053+0000 7fe3c9400640 1 --2- 172.21.15.86:0/2881768988 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555ef11cc590 0x555ef11c8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.053+0000 7fe3c9400640 1 -- 172.21.15.86:0/2881768988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ef119dd70 msgr2=0x555ef11cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:31.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.053+0000 7fe3c9400640 1 --2- 172.21.15.86:0/2881768988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ef119dd70 0x555ef11cb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.053+0000 7fe3c9400640 1 -- 172.21.15.86:0/2881768988 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ef0e183f0 con 0x555ef11ca1f0 2024-04-20T21:26:31.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.053+0000 7fe3c9e00640 1 --2- 172.21.15.86:0/2881768988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ef119dd70 0x555ef11cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:26:31.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.053+0000 7fe3c9400640 1 --2- 172.21.15.86:0/2881768988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ef11ca1f0 0x555ef11f8e00 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc15a990 tx=0x7fe3b8003c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62fd98fc6298daa0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:31.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.053+0000 7fe3aaa00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:26:31.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.053+0000 7fe3aaa00640 1 -- 172.21.15.86:0/2881768988 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b800e9d0 con 0x555ef11ca1f0 2024-04-20T21:26:31.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.053+0000 7fe3aaa00640 1 -- 172.21.15.86:0/2881768988 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7fe3b8023b40 con 0x555ef11ca1f0 2024-04-20T21:26:31.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.053+0000 7fe3d39ac740 1 -- 172.21.15.86:0/2881768988 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3c402cb20 con 0x555ef11ca1f0 2024-04-20T21:26:31.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.053+0000 7fe3aaa00640 1 -- 172.21.15.86:0/2881768988 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fe3b8023d40 con 0x555ef11ca1f0 2024-04-20T21:26:31.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.053+0000 7fe3d39ac740 10 client.7992 fetch_fsmap learned FSMap version 66 2024-04-20T21:26:31.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.053+0000 7fe3d39ac740 1 -- 172.21.15.86:0/2881768988 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x555ef0dd80f0 con 0x555ef11ca1f0 2024-04-20T21:26:31.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.054+0000 7fe3aaa00640 1 -- 172.21.15.86:0/2881768988 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3c402cb20 con 0x555ef11ca1f0 2024-04-20T21:26:31.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.054+0000 7fe3aaa00640 1 -- 172.21.15.86:0/2881768988 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 66) ==== 3495+0+0 (secure 0 0 0) 0x7fe3b800eb70 con 0x555ef11ca1f0 2024-04-20T21:26:31.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.054+0000 7fe3d39ac740 10 client.7992 fetch_fsmap finished waiting for FSMap version 66 2024-04-20T21:26:31.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.054+0000 7fe3d39ac740 10 client.7992 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-04-20T21:26:31.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.054+0000 7fe3d39ac740 20 client.7992 populate_metadata read hostname 'smithi086' 2024-04-20T21:26:31.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.054+0000 7fe3d39ac740 1 --2- 172.21.15.86:0/2881768988 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x555ef11c8c50 0x555ef11c3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:31.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.054+0000 7fe3d39ac740 4 client.7992 mds_command: new command op to 7773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:26:31.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.054+0000 7fe3d39ac740 1 -- 172.21.15.86:0/2881768988 --> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ef1113b70 con 0x555ef11c8c50 2024-04-20T21:26:31.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.055+0000 7fe3c9e00640 1 --2- 172.21.15.86:0/2881768988 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x555ef11c8c50 0x555ef11c3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:31.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.055+0000 7fe3c9e00640 1 --2- 172.21.15.86:0/2881768988 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x555ef11c8c50 0x555ef11c3700 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=867238ebd0274c06 server_cookie=7bfdb016a8727314 in_seq=0 out_seq=0 2024-04-20T21:26:31.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.055+0000 7fe3aaa00640 10 client.7992 ms_handle_connect on v2:172.21.15.144:6833/3981403733 2024-04-20T21:26:31.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.057+0000 7fe3aaa00640 1 -- 172.21.15.86:0/2881768988 <== mds.0 v2:172.21.15.144:6833/3981403733 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555ef1113b70 con 0x555ef11c8c50 2024-04-20T21:26:31.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.057+0000 7fe3aaa00640 10 client.7992 handle_command_reply: tid=0 2024-04-20T21:26:31.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.057+0000 7fe3d39ac740 1 client.7992 shutdown 2024-04-20T21:26:31.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.058+0000 7fe3d39ac740 1 -- 172.21.15.86:0/2881768988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ef11ca1f0 msgr2=0x555ef11f8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:31.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.058+0000 7fe3d39ac740 1 --2- 172.21.15.86:0/2881768988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ef11ca1f0 0x555ef11f8e00 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc15a990 tx=0x7fe3b8003c40 comp rx=0 tx=0).stop 2024-04-20T21:26:31.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.058+0000 7fe3d39ac740 1 -- 172.21.15.86:0/2881768988 shutdown_connections 2024-04-20T21:26:31.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.058+0000 7fe3d39ac740 1 --2- 172.21.15.86:0/2881768988 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x555ef11c8c50 0x555ef11c3700 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.058+0000 7fe3d39ac740 1 --2- 172.21.15.86:0/2881768988 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555ef11cc590 0x555ef11c8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.058+0000 7fe3d39ac740 1 --2- 172.21.15.86:0/2881768988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ef11ca1f0 0x555ef11f8e00 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.058+0000 7fe3d39ac740 1 --2- 172.21.15.86:0/2881768988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ef119dd70 0x555ef11cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.058+0000 7fe3d39ac740 1 -- 172.21.15.86:0/2881768988 >> 172.21.15.86:0/2881768988 conn(0x555ef119a210 msgr2=0x555ef11e65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:31.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.058+0000 7fe3aaa00640 0 client.7992 ms_handle_reset on v2:172.21.15.144:6833/3981403733 2024-04-20T21:26:31.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.058+0000 7fe3d39ac740 1 -- 172.21.15.86:0/2881768988 shutdown_connections 2024-04-20T21:26:31.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.058+0000 7fe3d39ac740 1 -- 172.21.15.86:0/2881768988 wait complete. 2024-04-20T21:26:31.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.058+0000 7fe3d39ac740 20 client.7992 trim_cache size 0 max 16384 2024-04-20T21:26:31.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.077+0000 7fe3d39ac740 1 Processor -- start 2024-04-20T21:26:31.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.077+0000 7fe3d39ac740 1 -- start start 2024-04-20T21:26:31.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.078+0000 7fe3d39ac740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ef119dd70 0x555ef11e6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:31.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.078+0000 7fe3d39ac740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555ef11ca1f0 0x555ef11e67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:31.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.078+0000 7fe3d39ac740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ef11cc590 0x555ef11c6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:31.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.078+0000 7fe3d39ac740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe3cc205350 con 0x555ef11cc590 2024-04-20T21:26:31.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.078+0000 7fe3d39ac740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe3cc204390 con 0x555ef119dd70 2024-04-20T21:26:31.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.078+0000 7fe3d39ac740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe3cc15e400 con 0x555ef11ca1f0 2024-04-20T21:26:31.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.078+0000 7fe3ca800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ef11cc590 0x555ef11c6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:31.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.078+0000 7fe3ca800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ef11cc590 0x555ef11c6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43708/0 (socket says 172.21.15.86:43708) 2024-04-20T21:26:31.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.078+0000 7fe3ca800640 1 -- 172.21.15.86:0/1389365127 learned_addr learned my addr 172.21.15.86:0/1389365127 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:31.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.079+0000 7fe3c9e00640 1 --2- 172.21.15.86:0/1389365127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ef119dd70 0x555ef11e6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:31.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.079+0000 7fe3c9400640 1 --2- 172.21.15.86:0/1389365127 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555ef11ca1f0 0x555ef11e67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:31.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.079+0000 7fe3ca800640 1 -- 172.21.15.86:0/1389365127 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555ef11ca1f0 msgr2=0x555ef11e67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:31.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.079+0000 7fe3ca800640 1 --2- 172.21.15.86:0/1389365127 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555ef11ca1f0 0x555ef11e67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.079+0000 7fe3ca800640 1 -- 172.21.15.86:0/1389365127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ef119dd70 msgr2=0x555ef11e6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:31.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.079+0000 7fe3ca800640 1 --2- 172.21.15.86:0/1389365127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ef119dd70 0x555ef11e6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.079+0000 7fe3ca800640 1 -- 172.21.15.86:0/1389365127 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ef11ca940 con 0x555ef11cc590 2024-04-20T21:26:31.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.079+0000 7fe3ca800640 1 --2- 172.21.15.86:0/1389365127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ef11cc590 0x555ef11c6af0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fe3c402c590 tx=0x7fe3c4009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9c2e020eb097ede server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:31.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.079+0000 7fe3b5c00640 1 -- 172.21.15.86:0/1389365127 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c4045040 con 0x555ef11cc590 2024-04-20T21:26:31.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.079+0000 7fe3b5c00640 1 -- 172.21.15.86:0/1389365127 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3c4003bb0 con 0x555ef11cc590 2024-04-20T21:26:31.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.079+0000 7fe3b5c00640 1 -- 172.21.15.86:0/1389365127 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c402c920 con 0x555ef11cc590 2024-04-20T21:26:31.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.080+0000 7fe3d39ac740 1 -- 172.21.15.86:0/1389365127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ef11cc590 msgr2=0x555ef11c6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:31.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.080+0000 7fe3d39ac740 1 --2- 172.21.15.86:0/1389365127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ef11cc590 0x555ef11c6af0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fe3c402c590 tx=0x7fe3c4009000 comp rx=0 tx=0).stop 2024-04-20T21:26:31.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.080+0000 7fe3d39ac740 1 -- 172.21.15.86:0/1389365127 shutdown_connections 2024-04-20T21:26:31.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.080+0000 7fe3d39ac740 1 --2- 172.21.15.86:0/1389365127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ef11cc590 0x555ef11c6af0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.080+0000 7fe3d39ac740 1 --2- 172.21.15.86:0/1389365127 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555ef11ca1f0 0x555ef11e67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.080+0000 7fe3d39ac740 1 --2- 172.21.15.86:0/1389365127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ef119dd70 0x555ef11e6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.080+0000 7fe3d39ac740 1 -- 172.21.15.86:0/1389365127 >> 172.21.15.86:0/1389365127 conn(0x555ef119a210 msgr2=0x555ef11cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:31.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.080+0000 7fe3d39ac740 1 -- 172.21.15.86:0/1389365127 shutdown_connections 2024-04-20T21:26:31.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.080+0000 7fe3d39ac740 1 -- 172.21.15.86:0/1389365127 wait complete. 2024-04-20T21:26:31.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.081+0000 7fe3d39ac740 1 -- start start 2024-04-20T21:26:31.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.081+0000 7fe3d39ac740 1 Processor -- start 2024-04-20T21:26:31.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.082+0000 7fe3d39ac740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ef119dd70 0x555ef11e7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:31.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.082+0000 7fe3d39ac740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555ef11ca1f0 0x555ef12600c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:31.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.082+0000 7fe3d39ac740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ef11cc590 0x555ef1260600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:31.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.082+0000 7fe3d39ac740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe3cc15ea50 con 0x555ef11cc590 2024-04-20T21:26:31.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.082+0000 7fe3d39ac740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x555ef1087ac0 con 0x555ef119dd70 2024-04-20T21:26:31.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.082+0000 7fe3d39ac740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x555ef1260e50 con 0x555ef11ca1f0 2024-04-20T21:26:31.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.082+0000 7fe3d39ac740 10 client.0 _ll_register_callbacks cb 0x7fe3b80012c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:26:31.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.082+0000 7fe3ca800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ef11cc590 0x555ef1260600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:31.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.082+0000 7fe3ca800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ef11cc590 0x555ef1260600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43710/0 (socket says 172.21.15.86:43710) 2024-04-20T21:26:31.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.082+0000 7fe3ca800640 1 -- 172.21.15.86:0/1980339876 learned_addr learned my addr 172.21.15.86:0/1980339876 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:31.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.083+0000 7fe3c9e00640 1 --2- 172.21.15.86:0/1980339876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ef119dd70 0x555ef11e7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:31.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.083+0000 7fe3c9400640 1 --2- 172.21.15.86:0/1980339876 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555ef11ca1f0 0x555ef12600c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:31.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.083+0000 7fe3ca800640 1 -- 172.21.15.86:0/1980339876 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555ef11ca1f0 msgr2=0x555ef12600c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:31.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.083+0000 7fe3ca800640 1 --2- 172.21.15.86:0/1980339876 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555ef11ca1f0 0x555ef12600c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.083+0000 7fe3ca800640 1 -- 172.21.15.86:0/1980339876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ef119dd70 msgr2=0x555ef11e7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:31.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.083+0000 7fe3ca800640 1 --2- 172.21.15.86:0/1980339876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ef119dd70 0x555ef11e7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.083+0000 7fe3ca800640 1 -- 172.21.15.86:0/1980339876 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ef11be3d0 con 0x555ef11cc590 2024-04-20T21:26:31.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.083+0000 7fe3c9e00640 1 --2- 172.21.15.86:0/1980339876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ef119dd70 0x555ef11e7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:26:31.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.083+0000 7fe3ca800640 1 --2- 172.21.15.86:0/1980339876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ef11cc590 0x555ef1260600 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fe3c404b040 tx=0x7fe3c4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6e8254daa4c14 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:31.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.083+0000 7fe3aaa00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:26:31.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.083+0000 7fe3aaa00640 1 -- 172.21.15.86:0/1980339876 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c4045040 con 0x555ef11cc590 2024-04-20T21:26:31.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.083+0000 7fe3aaa00640 1 -- 172.21.15.86:0/1980339876 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7fe3c4054680 con 0x555ef11cc590 2024-04-20T21:26:31.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.083+0000 7fe3d39ac740 1 -- 172.21.15.86:0/1980339876 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3c4003bb0 con 0x555ef11cc590 2024-04-20T21:26:31.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.084+0000 7fe3aaa00640 1 -- 172.21.15.86:0/1980339876 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fe3c405e050 con 0x555ef11cc590 2024-04-20T21:26:31.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.084+0000 7fe3d39ac740 10 client.7998 fetch_fsmap learned FSMap version 66 2024-04-20T21:26:31.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.084+0000 7fe3d39ac740 1 -- 172.21.15.86:0/1980339876 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x555ef11c5d20 con 0x555ef11cc590 2024-04-20T21:26:31.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.084+0000 7fe3aaa00640 1 -- 172.21.15.86:0/1980339876 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3c4003bb0 con 0x555ef11cc590 2024-04-20T21:26:31.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.084+0000 7fe3aaa00640 1 -- 172.21.15.86:0/1980339876 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 66) ==== 3495+0+0 (secure 0 0 0) 0x7fe3c4061020 con 0x555ef11cc590 2024-04-20T21:26:31.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.084+0000 7fe3d39ac740 10 client.7998 fetch_fsmap finished waiting for FSMap version 66 2024-04-20T21:26:31.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.084+0000 7fe3d39ac740 10 client.7998 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-04-20T21:26:31.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.084+0000 7fe3d39ac740 20 client.7998 populate_metadata read hostname 'smithi086' 2024-04-20T21:26:31.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.084+0000 7fe3d39ac740 1 --2- 172.21.15.86:0/1980339876 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x555ef11e6290 0x555ef11e66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:31.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.084+0000 7fe3d39ac740 4 client.7998 mds_command: new command op to 7773 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:26:31.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.084+0000 7fe3d39ac740 1 -- 172.21.15.86:0/1980339876 --> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555ef110d9e0 con 0x555ef11e6290 2024-04-20T21:26:31.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.085+0000 7fe3c9e00640 1 --2- 172.21.15.86:0/1980339876 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x555ef11e6290 0x555ef11e66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:31.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.086+0000 7fe3c9e00640 1 --2- 172.21.15.86:0/1980339876 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x555ef11e6290 0x555ef11e66a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d826e2466113608 server_cookie=5c4617cf8a7375ba in_seq=0 out_seq=0 2024-04-20T21:26:31.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.086+0000 7fe3aaa00640 10 client.7998 ms_handle_connect on v2:172.21.15.144:6833/3981403733 2024-04-20T21:26:31.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.086+0000 7fe3aaa00640 1 -- 172.21.15.86:0/1980339876 <== mds.0 v2:172.21.15.144:6833/3981403733 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555ef110d9e0 con 0x555ef11e6290 2024-04-20T21:26:31.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.086+0000 7fe3aaa00640 10 client.7998 handle_command_reply: tid=0 2024-04-20T21:26:31.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.086+0000 7fe3d39ac740 1 client.7998 shutdown 2024-04-20T21:26:31.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.086+0000 7fe3d39ac740 1 -- 172.21.15.86:0/1980339876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ef11cc590 msgr2=0x555ef1260600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:31.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.086+0000 7fe3d39ac740 1 --2- 172.21.15.86:0/1980339876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ef11cc590 0x555ef1260600 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fe3c404b040 tx=0x7fe3c4004ab0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.087+0000 7fe3d39ac740 1 -- 172.21.15.86:0/1980339876 shutdown_connections 2024-04-20T21:26:31.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.087+0000 7fe3d39ac740 1 --2- 172.21.15.86:0/1980339876 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x555ef11e6290 0x555ef11e66a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.087+0000 7fe3d39ac740 1 --2- 172.21.15.86:0/1980339876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555ef11cc590 0x555ef1260600 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.087+0000 7fe3d39ac740 1 --2- 172.21.15.86:0/1980339876 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x555ef11ca1f0 0x555ef12600c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.087+0000 7fe3d39ac740 1 --2- 172.21.15.86:0/1980339876 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x555ef119dd70 0x555ef11e7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.087+0000 7fe3d39ac740 1 -- 172.21.15.86:0/1980339876 >> 172.21.15.86:0/1980339876 conn(0x555ef119a210 msgr2=0x555ef11c00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:31.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.087+0000 7fe3aaa00640 0 client.7998 ms_handle_reset on v2:172.21.15.144:6833/3981403733 2024-04-20T21:26:31.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.087+0000 7fe3d39ac740 1 -- 172.21.15.86:0/1980339876 shutdown_connections 2024-04-20T21:26:31.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.087+0000 7fe3d39ac740 1 -- 172.21.15.86:0/1980339876 wait complete. 2024-04-20T21:26:31.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.087+0000 7fe3d39ac740 20 client.7998 trim_cache size 0 max 16384 2024-04-20T21:26:31.090 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:26:31.091 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-20T21:26:31.091 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-20T21:26:31.091 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:26:31.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.087+0000 7fe3d0c00640 1 -- 172.21.15.86:0/4218850747 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe3ac06c8f0 msgr2=0x7fe3ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:31.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.087+0000 7fe3d0c00640 1 --2- 172.21.15.86:0/4218850747 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe3ac06c8f0 0x7fe3ac06edb0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc15c9d0 tx=0x7fe3b8009040 comp rx=0 tx=0).stop 2024-04-20T21:26:31.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.087+0000 7fe3d0c00640 1 -- 172.21.15.86:0/4218850747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3cc0b3410 msgr2=0x7fe3cc163f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:31.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.087+0000 7fe3d0c00640 1 --2- 172.21.15.86:0/4218850747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3cc0b3410 0x7fe3cc163f50 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc00d5f0 tx=0x7fe3bc00dab0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.088+0000 7fe3d0c00640 1 -- 172.21.15.86:0/4218850747 shutdown_connections 2024-04-20T21:26:31.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.088+0000 7fe3d0c00640 1 --2- 172.21.15.86:0/4218850747 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe3ac06c8f0 0x7fe3ac06edb0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.088+0000 7fe3d0c00640 1 --2- 172.21.15.86:0/4218850747 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe3cc162ed0 0x7fe3cc164d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.088+0000 7fe3d0c00640 1 --2- 172.21.15.86:0/4218850747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3cc0ce370 0x7fe3cc15bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.088+0000 7fe3d0c00640 1 --2- 172.21.15.86:0/4218850747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3cc0b3410 0x7fe3cc163f50 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.088+0000 7fe3d0c00640 1 -- 172.21.15.86:0/4218850747 >> 172.21.15.86:0/4218850747 conn(0x7fe3cc0b2110 msgr2=0x7fe3cc15d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:31.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.088+0000 7fe3d0c00640 1 -- 172.21.15.86:0/4218850747 shutdown_connections 2024-04-20T21:26:31.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.088+0000 7fe3d0c00640 1 -- 172.21.15.86:0/4218850747 wait complete. 2024-04-20T21:26:31.104 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T21:26:31.104 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:26:31.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.300+0000 7f274b200640 1 Processor -- start 2024-04-20T21:26:31.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.300+0000 7f274b200640 1 -- start start 2024-04-20T21:26:31.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.302+0000 7f274b200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f274c15d6a0 0x7f274c0ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:31.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.302+0000 7f274b200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f274c0ad620 0x7f274c06e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:31.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.302+0000 7f274b200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f274c160890 con 0x7f274c156ae0 2024-04-20T21:26:31.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.302+0000 7f274b200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f274c0ada90 con 0x7f274c15d6a0 2024-04-20T21:26:31.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.302+0000 7f274b200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f274c06db20 con 0x7f274c0ad620 2024-04-20T21:26:31.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.302+0000 7f274a800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f274c0ad620 0x7f274c06e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:31.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.302+0000 7f2741400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f274c15d6a0 0x7f274c0ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:31.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.302+0000 7f274a800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f274c0ad620 0x7f274c06e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:51400/0 (socket says 172.21.15.86:51400) 2024-04-20T21:26:31.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.302+0000 7f274a800640 1 -- 172.21.15.86:0/2570033665 learned_addr learned my addr 172.21.15.86:0/2570033665 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:31.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.302+0000 7f2749e00640 1 --1- 172.21.15.86:0/2570033665 >> v1:172.21.15.144:6789/0 conn(0x7f274c156ae0 0x7f274c160080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.86:50322/0 (socket says 172.21.15.86:50322) 2024-04-20T21:26:31.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.303+0000 7f2749400640 1 -- 172.21.15.86:0/2570033665 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1838979747 0 0) 0x7f274c160890 con 0x7f274c156ae0 2024-04-20T21:26:31.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.303+0000 7f274a800640 1 -- 172.21.15.86:0/2570033665 >> v2:172.21.15.144:3300/0 conn(0x7f274c15d6a0 msgr2=0x7f274c0ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:31.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.303+0000 7f274a800640 1 --2- 172.21.15.86:0/2570033665 >> v2:172.21.15.144:3300/0 conn(0x7f274c15d6a0 0x7f274c0ad0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.303+0000 7f274a800640 1 -- 172.21.15.86:0/2570033665 >> v1:172.21.15.144:6789/0 conn(0x7f274c156ae0 legacy=0x7f274c160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:31.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.303+0000 7f274a800640 1 -- 172.21.15.86:0/2570033665 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f274c160b70 con 0x7f274c0ad620 2024-04-20T21:26:31.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.303+0000 7f2741400640 1 --2- 172.21.15.86:0/2570033665 >> v2:172.21.15.144:3300/0 conn(0x7f274c15d6a0 0x7f274c0ad0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:26:31.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.305+0000 7f274a800640 1 --2- 172.21.15.86:0/2570033665 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f274c0ad620 0x7f274c06e290 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2744009e50 tx=0x7f274402bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0e2720f01d567f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:31.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.305+0000 7f2749400640 1 -- 172.21.15.86:0/2570033665 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274402cb00 con 0x7f274c0ad620 2024-04-20T21:26:31.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.305+0000 7f2749400640 1 -- 172.21.15.86:0/2570033665 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f274402cca0 con 0x7f274c0ad620 2024-04-20T21:26:31.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.305+0000 7f2749400640 1 -- 172.21.15.86:0/2570033665 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2744034610 con 0x7f274c0ad620 2024-04-20T21:26:31.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.306+0000 7f274b200640 1 -- 172.21.15.86:0/2570033665 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f274c0ad620 msgr2=0x7f274c06e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:31.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.306+0000 7f274b200640 1 --2- 172.21.15.86:0/2570033665 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f274c0ad620 0x7f274c06e290 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2744009e50 tx=0x7f274402bf60 comp rx=0 tx=0).stop 2024-04-20T21:26:31.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.306+0000 7f274b200640 1 -- 172.21.15.86:0/2570033665 shutdown_connections 2024-04-20T21:26:31.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.306+0000 7f274b200640 1 --2- 172.21.15.86:0/2570033665 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f274c0ad620 0x7f274c06e290 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.306+0000 7f274b200640 1 --2- 172.21.15.86:0/2570033665 >> v2:172.21.15.144:3300/0 conn(0x7f274c15d6a0 0x7f274c0ad0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.306+0000 7f274b200640 1 -- 172.21.15.86:0/2570033665 >> 172.21.15.86:0/2570033665 conn(0x7f274c0d0170 msgr2=0x7f274c0d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:31.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.306+0000 7f274b200640 1 -- 172.21.15.86:0/2570033665 shutdown_connections 2024-04-20T21:26:31.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.306+0000 7f274b200640 1 -- 172.21.15.86:0/2570033665 wait complete. 2024-04-20T21:26:31.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.308+0000 7f274b200640 1 Processor -- start 2024-04-20T21:26:31.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.308+0000 7f274b200640 1 -- start start 2024-04-20T21:26:31.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.309+0000 7f274b200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f274c0ad620 0x7f274c1f7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:31.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.309+0000 7f274b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f274c156ae0 0x7f274c157990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:31.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.309+0000 7f274b200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f274c15d6a0 0x7f274c1446a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:31.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.309+0000 7f274b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f274c156ec0 con 0x7f274c156ae0 2024-04-20T21:26:31.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.309+0000 7f274b200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f274c06dd70 con 0x7f274c15d6a0 2024-04-20T21:26:31.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.309+0000 7f274b200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f274c06e7d0 con 0x7f274c0ad620 2024-04-20T21:26:31.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.310+0000 7f2741400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f274c156ae0 0x7f274c157990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:31.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.310+0000 7f2741400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f274c156ae0 0x7f274c157990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43726/0 (socket says 172.21.15.86:43726) 2024-04-20T21:26:31.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.310+0000 7f2741400640 1 -- 172.21.15.86:0/2015099589 learned_addr learned my addr 172.21.15.86:0/2015099589 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:31.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.310+0000 7f2749e00640 1 --2- 172.21.15.86:0/2015099589 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f274c0ad620 0x7f274c1f7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:31.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.310+0000 7f274a800640 1 --2- 172.21.15.86:0/2015099589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f274c15d6a0 0x7f274c1446a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:31.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.310+0000 7f2741400640 1 -- 172.21.15.86:0/2015099589 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f274c0ad620 msgr2=0x7f274c1f7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:31.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.310+0000 7f2741400640 1 --2- 172.21.15.86:0/2015099589 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f274c0ad620 0x7f274c1f7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.310+0000 7f2741400640 1 -- 172.21.15.86:0/2015099589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f274c15d6a0 msgr2=0x7f274c1446a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:31.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.310+0000 7f2741400640 1 --2- 172.21.15.86:0/2015099589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f274c15d6a0 0x7f274c1446a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.310+0000 7f2741400640 1 -- 172.21.15.86:0/2015099589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f274c1f7cc0 con 0x7f274c156ae0 2024-04-20T21:26:31.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.310+0000 7f274a800640 1 --2- 172.21.15.86:0/2015099589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f274c15d6a0 0x7f274c1446a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:26:31.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.310+0000 7f2741400640 1 --2- 172.21.15.86:0/2015099589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f274c156ae0 0x7f274c157990 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f273400e960 tx=0x7f273400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:31.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.311+0000 7f2742a00640 1 -- 172.21.15.86:0/2015099589 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2734017070 con 0x7f274c156ae0 2024-04-20T21:26:31.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.311+0000 7f2742a00640 1 -- 172.21.15.86:0/2015099589 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f273400ca00 con 0x7f274c156ae0 2024-04-20T21:26:31.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.311+0000 7f2742a00640 1 -- 172.21.15.86:0/2015099589 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273400cdd0 con 0x7f274c156ae0 2024-04-20T21:26:31.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.311+0000 7f274b200640 1 -- 172.21.15.86:0/2015099589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f274c0cee20 con 0x7f274c156ae0 2024-04-20T21:26:31.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.311+0000 7f274b200640 1 -- 172.21.15.86:0/2015099589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f274c072720 con 0x7f274c156ae0 2024-04-20T21:26:31.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.312+0000 7f274b200640 1 -- 172.21.15.86:0/2015099589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f274c072520 con 0x7f274c156ae0 2024-04-20T21:26:31.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.313+0000 7f2742a00640 1 -- 172.21.15.86:0/2015099589 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f273401a070 con 0x7f274c156ae0 2024-04-20T21:26:31.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.314+0000 7f2742a00640 1 --2- 172.21.15.86:0/2015099589 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f272806c8f0 0x7f272806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:31.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.314+0000 7f2742a00640 1 -- 172.21.15.86:0/2015099589 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7f273408ba70 con 0x7f274c156ae0 2024-04-20T21:26:31.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.314+0000 7f2749e00640 1 --2- 172.21.15.86:0/2015099589 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f272806c8f0 0x7f272806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:31.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.315+0000 7f2749e00640 1 --2- 172.21.15.86:0/2015099589 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f272806c8f0 0x7f272806edb0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f273c000f60 tx=0x7f273c007b90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:31.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.319+0000 7f2742a00640 1 -- 172.21.15.86:0/2015099589 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f274c1f7cc0 con 0x7f274c156ae0 2024-04-20T21:26:31.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.559+0000 7f274b200640 1 -- 172.21.15.86:0/2015099589 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f274c04fcf0 con 0x7f274c156ae0 2024-04-20T21:26:31.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.561+0000 7f2742a00640 1 -- 172.21.15.86:0/2015099589 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5208 (secure 0 0 0) 0x7f2734059e40 con 0x7f274c156ae0 2024-04-20T21:26:31.561 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:26:31.561 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":66,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7506,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/4013863257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4013863257},{"type":"v1","addr":"172.21.15.86:6836","nonce":4013863257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":7509,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2044883073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2044883073},{"type":"v1","addr":"172.21.15.86:6837","nonce":2044883073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7518,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/2351381466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2351381466},{"type":"v1","addr":"172.21.15.144:6834","nonce":2351381466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:24:58.299257+0000","modified":"2024-04-20T21:25:31.064401+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7773},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7773":{"gid":7773,"name":"b","rank":0,"incarnation":62,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6835/3981403733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3981403733},{"type":"v1","addr":"172.21.15.144:6835","nonce":3981403733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7773,"qdb_cluster":[7773]},"id":6}]} 2024-04-20T21:26:31.561 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 66 2024-04-20T21:26:31.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.563+0000 7f274b200640 1 -- 172.21.15.86:0/2015099589 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f272806c8f0 msgr2=0x7f272806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:31.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.563+0000 7f274b200640 1 --2- 172.21.15.86:0/2015099589 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f272806c8f0 0x7f272806edb0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f273c000f60 tx=0x7f273c007b90 comp rx=0 tx=0).stop 2024-04-20T21:26:31.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.563+0000 7f274b200640 1 -- 172.21.15.86:0/2015099589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f274c156ae0 msgr2=0x7f274c157990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:31.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.563+0000 7f274b200640 1 --2- 172.21.15.86:0/2015099589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f274c156ae0 0x7f274c157990 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f273400e960 tx=0x7f273400ee20 comp rx=0 tx=0).stop 2024-04-20T21:26:31.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.564+0000 7f274b200640 1 -- 172.21.15.86:0/2015099589 shutdown_connections 2024-04-20T21:26:31.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.564+0000 7f274b200640 1 --2- 172.21.15.86:0/2015099589 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f272806c8f0 0x7f272806edb0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.564+0000 7f274b200640 1 --2- 172.21.15.86:0/2015099589 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f274c15d6a0 0x7f274c1446a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.564+0000 7f274b200640 1 --2- 172.21.15.86:0/2015099589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f274c156ae0 0x7f274c157990 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.564+0000 7f274b200640 1 --2- 172.21.15.86:0/2015099589 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f274c0ad620 0x7f274c1f7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.564+0000 7f274b200640 1 -- 172.21.15.86:0/2015099589 >> 172.21.15.86:0/2015099589 conn(0x7f274c0d0170 msgr2=0x7f274c06eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:31.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.564+0000 7f274b200640 1 -- 172.21.15.86:0/2015099589 shutdown_connections 2024-04-20T21:26:31.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.564+0000 7f274b200640 1 -- 172.21.15.86:0/2015099589 wait complete. 2024-04-20T21:26:31.584 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b damage ls --format=json-pretty 2024-04-20T21:26:31.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.786+0000 7fbb2c000640 1 Processor -- start 2024-04-20T21:26:31.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.786+0000 7fbb2c000640 1 -- start start 2024-04-20T21:26:31.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.788+0000 7fbb2c000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb2406bbf0 0x7fbb2406e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:31.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.788+0000 7fbb2c000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb2415cd40 con 0x7fbb24146210 2024-04-20T21:26:31.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.788+0000 7fbb2c000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb2415d000 con 0x7fbb2406bfd0 2024-04-20T21:26:31.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.788+0000 7fbb2c000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbb2415d2e0 con 0x7fbb2406bbf0 2024-04-20T21:26:31.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.788+0000 7fbb2a000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fbb24146210 0x7fbb240cd540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34720/0 (socket says 172.21.15.86:34720) 2024-04-20T21:26:31.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.788+0000 7fbb2a000640 1 -- 172.21.15.86:0/4101322531 learned_addr learned my addr 172.21.15.86:0/4101322531 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:26:31.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.789+0000 7fbb29600640 1 --2- 172.21.15.86:0/4101322531 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb2406bbf0 0x7fbb2406e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:31.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.789+0000 7fbb23e00640 1 -- 172.21.15.86:0/4101322531 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2399891216 0 0) 0x7fbb2415cd40 con 0x7fbb24146210 2024-04-20T21:26:31.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.789+0000 7fbb23e00640 1 -- 172.21.15.86:0/4101322531 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbb0c003520 con 0x7fbb24146210 2024-04-20T21:26:31.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.789+0000 7fbb23e00640 1 -- 172.21.15.86:0/4101322531 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 666645707 0 0) 0x7fbb0c003520 con 0x7fbb24146210 2024-04-20T21:26:31.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.789+0000 7fbb23e00640 1 -- 172.21.15.86:0/4101322531 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb2406bbf0 msgr2=0x7fbb2406e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:31.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.789+0000 7fbb23e00640 1 --2- 172.21.15.86:0/4101322531 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb2406bbf0 0x7fbb2406e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.789+0000 7fbb23e00640 1 -- 172.21.15.86:0/4101322531 >> v1:172.21.15.144:6789/0 conn(0x7fbb2406bfd0 legacy=0x7fbb24145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:31.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.789+0000 7fbb23e00640 1 -- 172.21.15.86:0/4101322531 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb2415d5c0 con 0x7fbb24146210 2024-04-20T21:26:31.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.789+0000 7fbb23e00640 1 -- 172.21.15.86:0/4101322531 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fbb1c0019f0 con 0x7fbb24146210 2024-04-20T21:26:31.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.789+0000 7fbb23e00640 1 -- 172.21.15.86:0/4101322531 >> v1:172.21.15.86:6789/0 conn(0x7fbb24146210 legacy=0x7fbb240cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:31.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.789+0000 7fbb23e00640 1 --2- 172.21.15.86:0/4101322531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb0c0039e0 0x7fbb0c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:31.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.789+0000 7fbb23e00640 1 --2- 172.21.15.86:0/4101322531 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb2406bbf0 0x7fbb0c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:31.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.789+0000 7fbb23e00640 1 --2- 172.21.15.86:0/4101322531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb0c004d40 0x7fbb0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:31.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.789+0000 7fbb23e00640 1 -- 172.21.15.86:0/4101322531 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2415d2e0 con 0x7fbb0c004d40 2024-04-20T21:26:31.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.789+0000 7fbb23e00640 1 -- 172.21.15.86:0/4101322531 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c00baa0 con 0x7fbb0c0039e0 2024-04-20T21:26:31.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.789+0000 7fbb23e00640 1 -- 172.21.15.86:0/4101322531 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbb0c00bca0 con 0x7fbb2406bbf0 2024-04-20T21:26:31.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.790+0000 7fbb2a000640 1 --2- 172.21.15.86:0/4101322531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb0c004d40 0x7fbb0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:31.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.790+0000 7fbb29600640 1 --2- 172.21.15.86:0/4101322531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb0c0039e0 0x7fbb0c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:31.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.790+0000 7fbb2a000640 1 -- 172.21.15.86:0/4101322531 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb2406bbf0 msgr2=0x7fbb0c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:31.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.790+0000 7fbb28c00640 1 --2- 172.21.15.86:0/4101322531 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb2406bbf0 0x7fbb0c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:31.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.790+0000 7fbb2a000640 1 --2- 172.21.15.86:0/4101322531 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb2406bbf0 0x7fbb0c004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.790+0000 7fbb2a000640 1 -- 172.21.15.86:0/4101322531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb0c0039e0 msgr2=0x7fbb0c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:31.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.790+0000 7fbb2a000640 1 --2- 172.21.15.86:0/4101322531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb0c0039e0 0x7fbb0c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.790+0000 7fbb2a000640 1 -- 172.21.15.86:0/4101322531 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb0c003520 con 0x7fbb0c004d40 2024-04-20T21:26:31.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.790+0000 7fbb29600640 1 --2- 172.21.15.86:0/4101322531 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb0c0039e0 0x7fbb0c003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:26:31.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.791+0000 7fbb2a000640 1 --2- 172.21.15.86:0/4101322531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb0c004d40 0x7fbb0c0051c0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fbb240cce70 tx=0x7fbb0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27f595acb89535a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:31.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.792+0000 7fbb23e00640 1 -- 172.21.15.86:0/4101322531 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c00a390 con 0x7fbb0c004d40 2024-04-20T21:26:31.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.792+0000 7fbb23e00640 1 -- 172.21.15.86:0/4101322531 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb1c034650 con 0x7fbb0c004d40 2024-04-20T21:26:31.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.792+0000 7fbb23e00640 1 -- 172.21.15.86:0/4101322531 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c034a40 con 0x7fbb0c004d40 2024-04-20T21:26:31.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.792+0000 7fbb2c000640 1 -- 172.21.15.86:0/4101322531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb0c004d40 msgr2=0x7fbb0c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:31.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.792+0000 7fbb2c000640 1 --2- 172.21.15.86:0/4101322531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb0c004d40 0x7fbb0c0051c0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fbb240cce70 tx=0x7fbb0c005830 comp rx=0 tx=0).stop 2024-04-20T21:26:31.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.792+0000 7fbb2a000640 1 -- 172.21.15.86:0/4101322531 reap_dead start 2024-04-20T21:26:31.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.792+0000 7fbb2c000640 1 -- 172.21.15.86:0/4101322531 shutdown_connections 2024-04-20T21:26:31.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.792+0000 7fbb2c000640 1 -- 172.21.15.86:0/4101322531 >> 172.21.15.86:0/4101322531 conn(0x7fbb240b2110 msgr2=0x7fbb240b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:31.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.792+0000 7fbb2c000640 1 -- 172.21.15.86:0/4101322531 shutdown_connections 2024-04-20T21:26:31.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.792+0000 7fbb2c000640 1 -- 172.21.15.86:0/4101322531 wait complete. 2024-04-20T21:26:31.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.794+0000 7fbb2c000640 1 Processor -- start 2024-04-20T21:26:31.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.794+0000 7fbb2c000640 1 -- start start 2024-04-20T21:26:31.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.795+0000 7fbb2c000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb0c004d40 0x7fbb240c9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:31.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.795+0000 7fbb2c000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb2415c320 0x7fbb240c6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:31.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.795+0000 7fbb2c000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb241e7f10 0x7fbb241e83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:31.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.795+0000 7fbb2c000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbb240cddd0 con 0x7fbb241e7f10 2024-04-20T21:26:31.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.795+0000 7fbb2c000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbb240cdc50 con 0x7fbb2415c320 2024-04-20T21:26:31.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.795+0000 7fbb2c000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbb240cdf50 con 0x7fbb0c004d40 2024-04-20T21:26:31.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.795+0000 7fbb2a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb241e7f10 0x7fbb241e83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:31.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.795+0000 7fbb2a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb241e7f10 0x7fbb241e83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43752/0 (socket says 172.21.15.86:43752) 2024-04-20T21:26:31.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.795+0000 7fbb2a000640 1 -- 172.21.15.86:0/322864733 learned_addr learned my addr 172.21.15.86:0/322864733 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:31.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.796+0000 7fbb29600640 1 --2- 172.21.15.86:0/322864733 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb0c004d40 0x7fbb240c9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:31.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.796+0000 7fbb28c00640 1 --2- 172.21.15.86:0/322864733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb2415c320 0x7fbb240c6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:31.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.796+0000 7fbb2a000640 1 -- 172.21.15.86:0/322864733 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb0c004d40 msgr2=0x7fbb240c9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:31.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.796+0000 7fbb2a000640 1 --2- 172.21.15.86:0/322864733 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb0c004d40 0x7fbb240c9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.796+0000 7fbb2a000640 1 -- 172.21.15.86:0/322864733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb2415c320 msgr2=0x7fbb240c6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:31.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.796+0000 7fbb2a000640 1 --2- 172.21.15.86:0/322864733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb2415c320 0x7fbb240c6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.796+0000 7fbb2a000640 1 -- 172.21.15.86:0/322864733 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb241f2630 con 0x7fbb241e7f10 2024-04-20T21:26:31.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.796+0000 7fbb2a000640 1 --2- 172.21.15.86:0/322864733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb241e7f10 0x7fbb241e83a0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c00a830 tx=0x7fbb2415d790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:31.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.796+0000 7fbb21600640 1 -- 172.21.15.86:0/322864733 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c001250 con 0x7fbb241e7f10 2024-04-20T21:26:31.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.797+0000 7fbb21600640 1 -- 172.21.15.86:0/322864733 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb1c044070 con 0x7fbb241e7f10 2024-04-20T21:26:31.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.797+0000 7fbb21600640 1 -- 172.21.15.86:0/322864733 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c034a00 con 0x7fbb241e7f10 2024-04-20T21:26:31.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.797+0000 7fbb2c000640 1 -- 172.21.15.86:0/322864733 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb240c6f90 con 0x7fbb241e7f10 2024-04-20T21:26:31.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.797+0000 7fbb2c000640 1 -- 172.21.15.86:0/322864733 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb240c7550 con 0x7fbb241e7f10 2024-04-20T21:26:31.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.799+0000 7fbb21600640 1 -- 172.21.15.86:0/322864733 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbb1c034ba0 con 0x7fbb241e7f10 2024-04-20T21:26:31.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.800+0000 7fbb21600640 1 --2- 172.21.15.86:0/322864733 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbb0406cac0 0x7fbb0406ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:31.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.800+0000 7fbb21600640 1 -- 172.21.15.86:0/322864733 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7fbb1c0adf00 con 0x7fbb241e7f10 2024-04-20T21:26:31.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.800+0000 7fbb29600640 1 --2- 172.21.15.86:0/322864733 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbb0406cac0 0x7fbb0406ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:31.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.801+0000 7fbb2edbf740 1 Processor -- start 2024-04-20T21:26:31.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.801+0000 7fbb2edbf740 1 -- start start 2024-04-20T21:26:31.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.801+0000 7fbb29600640 1 --2- 172.21.15.86:0/322864733 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbb0406cac0 0x7fbb0406ef80 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fbb18009aa0 tx=0x7fbb18008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:31.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.802+0000 7fbb2edbf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55849eae1a30 0x55849eadf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:31.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.802+0000 7fbb2edbf740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55849eac1270 0x55849ea98f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:31.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.802+0000 7fbb2edbf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55849ea994a0 0x55849eac3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:31.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.802+0000 7fbb2edbf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55849e9ff060 con 0x55849eae1a30 2024-04-20T21:26:31.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.802+0000 7fbb2edbf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55849e9a4f00 con 0x55849ea994a0 2024-04-20T21:26:31.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.802+0000 7fbb2edbf740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55849e9fb020 con 0x55849eac1270 2024-04-20T21:26:31.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.802+0000 7fbb29600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55849eae1a30 0x55849eadf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:31.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.802+0000 7fbb29600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55849eae1a30 0x55849eadf5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43762/0 (socket says 172.21.15.86:43762) 2024-04-20T21:26:31.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.802+0000 7fbb29600640 1 -- 172.21.15.86:0/2845787015 learned_addr learned my addr 172.21.15.86:0/2845787015 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:31.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.802+0000 7fbb28c00640 1 --2- 172.21.15.86:0/2845787015 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55849eac1270 0x55849ea98f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:31.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.802+0000 7fbb2a000640 1 --2- 172.21.15.86:0/2845787015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55849ea994a0 0x55849eac3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:31.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.802+0000 7fbb29600640 1 -- 172.21.15.86:0/2845787015 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55849eac1270 msgr2=0x55849ea98f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:31.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.802+0000 7fbb29600640 1 --2- 172.21.15.86:0/2845787015 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55849eac1270 0x55849ea98f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.802+0000 7fbb29600640 1 -- 172.21.15.86:0/2845787015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55849ea994a0 msgr2=0x55849eac3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:31.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.802+0000 7fbb29600640 1 --2- 172.21.15.86:0/2845787015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55849ea994a0 0x55849eac3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.802+0000 7fbb29600640 1 -- 172.21.15.86:0/2845787015 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55849e98da30 con 0x55849eae1a30 2024-04-20T21:26:31.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.802+0000 7fbb2a000640 1 --2- 172.21.15.86:0/2845787015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55849ea994a0 0x55849eac3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:26:31.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.803+0000 7fbb29600640 1 --2- 172.21.15.86:0/2845787015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55849eae1a30 0x55849eadf5f0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fbb18012d90 tx=0x7fbb18015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaaad2563f0691b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:31.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.803+0000 7fbb15c00640 1 -- 172.21.15.86:0/2845787015 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1800f040 con 0x55849eae1a30 2024-04-20T21:26:31.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.803+0000 7fbb15c00640 1 -- 172.21.15.86:0/2845787015 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb1801b030 con 0x55849eae1a30 2024-04-20T21:26:31.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.803+0000 7fbb15c00640 1 -- 172.21.15.86:0/2845787015 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb18004020 con 0x55849eae1a30 2024-04-20T21:26:31.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.804+0000 7fbb2edbf740 1 -- 172.21.15.86:0/2845787015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55849eae1a30 msgr2=0x55849eadf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:31.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.804+0000 7fbb2edbf740 1 --2- 172.21.15.86:0/2845787015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55849eae1a30 0x55849eadf5f0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fbb18012d90 tx=0x7fbb18015000 comp rx=0 tx=0).stop 2024-04-20T21:26:31.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.804+0000 7fbb2edbf740 1 -- 172.21.15.86:0/2845787015 shutdown_connections 2024-04-20T21:26:31.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.804+0000 7fbb2edbf740 1 --2- 172.21.15.86:0/2845787015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55849ea994a0 0x55849eac3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.804+0000 7fbb2edbf740 1 --2- 172.21.15.86:0/2845787015 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55849eac1270 0x55849ea98f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.804+0000 7fbb2edbf740 1 --2- 172.21.15.86:0/2845787015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55849eae1a30 0x55849eadf5f0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.804+0000 7fbb2edbf740 1 -- 172.21.15.86:0/2845787015 >> 172.21.15.86:0/2845787015 conn(0x55849ea960c0 msgr2=0x55849eadec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:31.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.804+0000 7fbb2edbf740 1 -- 172.21.15.86:0/2845787015 shutdown_connections 2024-04-20T21:26:31.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.804+0000 7fbb2edbf740 1 -- 172.21.15.86:0/2845787015 wait complete. 2024-04-20T21:26:31.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.805+0000 7fbb2edbf740 1 -- start start 2024-04-20T21:26:31.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.805+0000 7fbb2edbf740 1 Processor -- start 2024-04-20T21:26:31.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.806+0000 7fbb2edbf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55849eac1270 0x55849eada4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:31.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.806+0000 7fbb2edbf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55849eae1a30 0x55849eadaa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:31.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.806+0000 7fbb2edbf740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55849eac0ac0 0x55849eac2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:31.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.806+0000 7fbb2edbf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55849ea983c0 con 0x55849eae1a30 2024-04-20T21:26:31.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.806+0000 7fbb2edbf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55849e9b83d0 con 0x55849eac1270 2024-04-20T21:26:31.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.806+0000 7fbb2edbf740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55849ea946f0 con 0x55849eac0ac0 2024-04-20T21:26:31.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.806+0000 7fbb2edbf740 10 client.0 _ll_register_callbacks cb 0x55849e986960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:26:31.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.806+0000 7fbb28c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55849eae1a30 0x55849eadaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:31.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.806+0000 7fbb28c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55849eae1a30 0x55849eadaa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43776/0 (socket says 172.21.15.86:43776) 2024-04-20T21:26:31.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.806+0000 7fbb28c00640 1 -- 172.21.15.86:0/3890088533 learned_addr learned my addr 172.21.15.86:0/3890088533 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:31.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.807+0000 7fbb2a000640 1 --2- 172.21.15.86:0/3890088533 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55849eac0ac0 0x55849eac2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:31.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.807+0000 7fbb29600640 1 --2- 172.21.15.86:0/3890088533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55849eac1270 0x55849eada4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:31.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.807+0000 7fbb28c00640 1 -- 172.21.15.86:0/3890088533 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55849eac0ac0 msgr2=0x55849eac2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:31.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.807+0000 7fbb28c00640 1 --2- 172.21.15.86:0/3890088533 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55849eac0ac0 0x55849eac2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.807+0000 7fbb28c00640 1 -- 172.21.15.86:0/3890088533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55849eac1270 msgr2=0x55849eada4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:31.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.807+0000 7fbb28c00640 1 --2- 172.21.15.86:0/3890088533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55849eac1270 0x55849eada4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.807+0000 7fbb28c00640 1 -- 172.21.15.86:0/3890088533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55849e7dd7c0 con 0x55849eae1a30 2024-04-20T21:26:31.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.807+0000 7fbb2a000640 1 --2- 172.21.15.86:0/3890088533 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55849eac0ac0 0x55849eac2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:26:31.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.807+0000 7fbb28c00640 1 --2- 172.21.15.86:0/3890088533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55849eae1a30 0x55849eadaa20 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fbb10001e00 tx=0x7fbb1000f6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=753843f0ad09b726 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:31.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.807+0000 7fbb0a000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:26:31.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.807+0000 7fbb0a000640 1 -- 172.21.15.86:0/3890088533 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb100052d0 con 0x55849eae1a30 2024-04-20T21:26:31.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.807+0000 7fbb0a000640 1 -- 172.21.15.86:0/3890088533 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7fbb100035f0 con 0x55849eae1a30 2024-04-20T21:26:31.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.807+0000 7fbb2edbf740 1 -- 172.21.15.86:0/3890088533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb1801b030 con 0x55849eae1a30 2024-04-20T21:26:31.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.807+0000 7fbb0a000640 1 -- 172.21.15.86:0/3890088533 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fbb10020050 con 0x55849eae1a30 2024-04-20T21:26:31.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.808+0000 7fbb2edbf740 10 client.8013 fetch_fsmap learned FSMap version 66 2024-04-20T21:26:31.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.808+0000 7fbb2edbf740 1 -- 172.21.15.86:0/3890088533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x55849e83df50 con 0x55849eae1a30 2024-04-20T21:26:31.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.808+0000 7fbb0a000640 1 -- 172.21.15.86:0/3890088533 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb1801b030 con 0x55849eae1a30 2024-04-20T21:26:31.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.808+0000 7fbb0a000640 1 -- 172.21.15.86:0/3890088533 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 66) ==== 3495+0+0 (secure 0 0 0) 0x7fbb10023020 con 0x55849eae1a30 2024-04-20T21:26:31.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.808+0000 7fbb2edbf740 10 client.8013 fetch_fsmap finished waiting for FSMap version 66 2024-04-20T21:26:31.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.808+0000 7fbb2edbf740 10 client.8013 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-20T21:26:31.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.808+0000 7fbb2edbf740 20 client.8013 populate_metadata read hostname 'smithi086' 2024-04-20T21:26:31.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.808+0000 7fbb2edbf740 1 --2- 172.21.15.86:0/3890088533 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x55849eaddb60 0x55849eaf3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:31.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.808+0000 7fbb2edbf740 4 client.8013 mds_command: new command op to 7773 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:26:31.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.808+0000 7fbb2edbf740 1 -- 172.21.15.86:0/3890088533 --> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55849ea0ca80 con 0x55849eaddb60 2024-04-20T21:26:31.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.809+0000 7fbb29600640 1 --2- 172.21.15.86:0/3890088533 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x55849eaddb60 0x55849eaf3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:31.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.810+0000 7fbb29600640 1 --2- 172.21.15.86:0/3890088533 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x55849eaddb60 0x55849eaf3640 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50db246f02feae9f server_cookie=2be66d429a854d0d in_seq=0 out_seq=0 2024-04-20T21:26:31.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.810+0000 7fbb0a000640 10 client.8013 ms_handle_connect on v2:172.21.15.144:6833/3981403733 2024-04-20T21:26:31.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.811+0000 7fbb0a000640 1 -- 172.21.15.86:0/3890088533 <== mds.0 v2:172.21.15.144:6833/3981403733 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55849ea0ca80 con 0x55849eaddb60 2024-04-20T21:26:31.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.812+0000 7fbb0a000640 10 client.8013 handle_command_reply: tid=0 2024-04-20T21:26:31.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.812+0000 7fbb2edbf740 1 client.8013 shutdown 2024-04-20T21:26:31.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.812+0000 7fbb2edbf740 1 -- 172.21.15.86:0/3890088533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55849eae1a30 msgr2=0x55849eadaa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:31.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.812+0000 7fbb2edbf740 1 --2- 172.21.15.86:0/3890088533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55849eae1a30 0x55849eadaa20 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fbb10001e00 tx=0x7fbb1000f6c0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.812+0000 7fbb2edbf740 1 -- 172.21.15.86:0/3890088533 shutdown_connections 2024-04-20T21:26:31.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.812+0000 7fbb2edbf740 1 --2- 172.21.15.86:0/3890088533 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x55849eaddb60 0x55849eaf3640 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.812+0000 7fbb2edbf740 1 --2- 172.21.15.86:0/3890088533 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55849eac0ac0 0x55849eac2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.812+0000 7fbb2edbf740 1 --2- 172.21.15.86:0/3890088533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55849eae1a30 0x55849eadaa20 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.812+0000 7fbb0a000640 0 client.8013 ms_handle_reset on v2:172.21.15.144:6833/3981403733 2024-04-20T21:26:31.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.812+0000 7fbb2edbf740 1 --2- 172.21.15.86:0/3890088533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55849eac1270 0x55849eada4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.812+0000 7fbb2edbf740 1 -- 172.21.15.86:0/3890088533 >> 172.21.15.86:0/3890088533 conn(0x55849ea994a0 msgr2=0x55849ea92fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:31.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.812+0000 7fbb2edbf740 1 -- 172.21.15.86:0/3890088533 shutdown_connections 2024-04-20T21:26:31.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.812+0000 7fbb2edbf740 1 -- 172.21.15.86:0/3890088533 wait complete. 2024-04-20T21:26:31.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.812+0000 7fbb2edbf740 20 client.8013 trim_cache size 0 max 16384 2024-04-20T21:26:31.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.827+0000 7fbb2edbf740 1 Processor -- start 2024-04-20T21:26:31.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.827+0000 7fbb2edbf740 1 -- start start 2024-04-20T21:26:31.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.828+0000 7fbb2edbf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55849eac0ac0 0x55849eac0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:31.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.828+0000 7fbb2edbf740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55849eac1270 0x55849eafcd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:31.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.828+0000 7fbb2edbf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55849eae1a30 0x55849eac1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:31.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.828+0000 7fbb2edbf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbb240b4d70 con 0x55849eae1a30 2024-04-20T21:26:31.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.828+0000 7fbb2edbf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbb240c89d0 con 0x55849eac0ac0 2024-04-20T21:26:31.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.828+0000 7fbb2edbf740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbb240b4720 con 0x55849eac1270 2024-04-20T21:26:31.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.828+0000 7fbb2a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55849eae1a30 0x55849eac1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:31.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.828+0000 7fbb2a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55849eae1a30 0x55849eac1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43792/0 (socket says 172.21.15.86:43792) 2024-04-20T21:26:31.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.828+0000 7fbb2a000640 1 -- 172.21.15.86:0/3042697370 learned_addr learned my addr 172.21.15.86:0/3042697370 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:31.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.829+0000 7fbb28c00640 1 --2- 172.21.15.86:0/3042697370 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55849eac1270 0x55849eafcd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:31.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.829+0000 7fbb29600640 1 --2- 172.21.15.86:0/3042697370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55849eac0ac0 0x55849eac0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:31.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.829+0000 7fbb2a000640 1 -- 172.21.15.86:0/3042697370 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55849eac1270 msgr2=0x55849eafcd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:31.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.829+0000 7fbb2a000640 1 --2- 172.21.15.86:0/3042697370 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55849eac1270 0x55849eafcd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.829+0000 7fbb2a000640 1 -- 172.21.15.86:0/3042697370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55849eac0ac0 msgr2=0x55849eac0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:31.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.829+0000 7fbb2a000640 1 --2- 172.21.15.86:0/3042697370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55849eac0ac0 0x55849eac0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.829+0000 7fbb2a000640 1 -- 172.21.15.86:0/3042697370 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55849e6d10b0 con 0x55849eae1a30 2024-04-20T21:26:31.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.829+0000 7fbb2a000640 1 --2- 172.21.15.86:0/3042697370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55849eae1a30 0x55849eac1a20 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c001e90 tx=0x7fbb1c07b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c6bf4281525bc6d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:31.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.829+0000 7fbb15200640 1 -- 172.21.15.86:0/3042697370 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c07c2d0 con 0x55849eae1a30 2024-04-20T21:26:31.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.829+0000 7fbb15200640 1 -- 172.21.15.86:0/3042697370 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb1c0aa070 con 0x55849eae1a30 2024-04-20T21:26:31.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.829+0000 7fbb15200640 1 -- 172.21.15.86:0/3042697370 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c00a430 con 0x55849eae1a30 2024-04-20T21:26:31.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.830+0000 7fbb2edbf740 1 -- 172.21.15.86:0/3042697370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55849eae1a30 msgr2=0x55849eac1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:31.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.830+0000 7fbb2edbf740 1 --2- 172.21.15.86:0/3042697370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55849eae1a30 0x55849eac1a20 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c001e90 tx=0x7fbb1c07b830 comp rx=0 tx=0).stop 2024-04-20T21:26:31.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.830+0000 7fbb2edbf740 1 -- 172.21.15.86:0/3042697370 shutdown_connections 2024-04-20T21:26:31.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.830+0000 7fbb2edbf740 1 --2- 172.21.15.86:0/3042697370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55849eae1a30 0x55849eac1a20 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.830+0000 7fbb2edbf740 1 --2- 172.21.15.86:0/3042697370 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55849eac1270 0x55849eafcd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.830+0000 7fbb2edbf740 1 --2- 172.21.15.86:0/3042697370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55849eac0ac0 0x55849eac0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.830+0000 7fbb2edbf740 1 -- 172.21.15.86:0/3042697370 >> 172.21.15.86:0/3042697370 conn(0x55849ea994a0 msgr2=0x55849ea8f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:31.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.830+0000 7fbb2edbf740 1 -- 172.21.15.86:0/3042697370 shutdown_connections 2024-04-20T21:26:31.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.830+0000 7fbb2edbf740 1 -- 172.21.15.86:0/3042697370 wait complete. 2024-04-20T21:26:31.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.831+0000 7fbb2edbf740 1 -- start start 2024-04-20T21:26:31.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.831+0000 7fbb2edbf740 1 Processor -- start 2024-04-20T21:26:31.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.832+0000 7fbb2edbf740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55849eac0ac0 0x55849eabf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:31.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.832+0000 7fbb2edbf740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55849eac1270 0x55849eb5c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:31.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.832+0000 7fbb2edbf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55849eae1a30 0x55849eabef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:31.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.832+0000 7fbb2edbf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55849eadd9e0 con 0x55849eae1a30 2024-04-20T21:26:31.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.832+0000 7fbb2edbf740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55849eadcd80 con 0x55849eac0ac0 2024-04-20T21:26:31.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.832+0000 7fbb2edbf740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55849eaddf90 con 0x55849eac1270 2024-04-20T21:26:31.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.832+0000 7fbb2edbf740 10 client.0 _ll_register_callbacks cb 0x7fbb10001980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:26:31.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.832+0000 7fbb2a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55849eae1a30 0x55849eabef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:31.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.832+0000 7fbb2a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55849eae1a30 0x55849eabef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43800/0 (socket says 172.21.15.86:43800) 2024-04-20T21:26:31.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.832+0000 7fbb2a000640 1 -- 172.21.15.86:0/1097675693 learned_addr learned my addr 172.21.15.86:0/1097675693 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:31.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.833+0000 7fbb29600640 1 --2- 172.21.15.86:0/1097675693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55849eac0ac0 0x55849eabf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:31.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.833+0000 7fbb2a000640 1 -- 172.21.15.86:0/1097675693 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55849eac1270 msgr2=0x55849eb5c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:31.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.833+0000 7fbb28c00640 1 --2- 172.21.15.86:0/1097675693 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55849eac1270 0x55849eb5c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:31.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.833+0000 7fbb2a000640 1 --2- 172.21.15.86:0/1097675693 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55849eac1270 0x55849eb5c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.833+0000 7fbb2a000640 1 -- 172.21.15.86:0/1097675693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55849eac0ac0 msgr2=0x55849eabf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:31.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.833+0000 7fbb2a000640 1 --2- 172.21.15.86:0/1097675693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55849eac0ac0 0x55849eabf810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.833+0000 7fbb2a000640 1 -- 172.21.15.86:0/1097675693 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55849ea98920 con 0x55849eae1a30 2024-04-20T21:26:31.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.833+0000 7fbb2a000640 1 --2- 172.21.15.86:0/1097675693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55849eae1a30 0x55849eabef30 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c07b8d0 tx=0x7fbb1c001e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b86b651043ddcf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:31.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.833+0000 7fbb0a000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:26:31.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.833+0000 7fbb0a000640 1 -- 172.21.15.86:0/1097675693 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c0a0060 con 0x55849eae1a30 2024-04-20T21:26:31.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.833+0000 7fbb0a000640 1 -- 172.21.15.86:0/1097675693 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7fbb1c078240 con 0x55849eae1a30 2024-04-20T21:26:31.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.833+0000 7fbb2edbf740 1 -- 172.21.15.86:0/1097675693 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb1c0aa070 con 0x55849eae1a30 2024-04-20T21:26:31.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.834+0000 7fbb0a000640 1 -- 172.21.15.86:0/1097675693 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7fbb1c09b220 con 0x55849eae1a30 2024-04-20T21:26:31.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.834+0000 7fbb2edbf740 10 client.8019 fetch_fsmap learned FSMap version 66 2024-04-20T21:26:31.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.834+0000 7fbb2edbf740 1 -- 172.21.15.86:0/1097675693 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) -- 0x55849eada630 con 0x55849eae1a30 2024-04-20T21:26:31.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.834+0000 7fbb0a000640 1 -- 172.21.15.86:0/1097675693 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb1c0aa070 con 0x55849eae1a30 2024-04-20T21:26:31.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.834+0000 7fbb0a000640 1 -- 172.21.15.86:0/1097675693 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 66) ==== 3495+0+0 (secure 0 0 0) 0x7fbb1c07b020 con 0x55849eae1a30 2024-04-20T21:26:31.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.834+0000 7fbb2edbf740 10 client.8019 fetch_fsmap finished waiting for FSMap version 66 2024-04-20T21:26:31.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.834+0000 7fbb2edbf740 10 client.8019 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-20T21:26:31.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.834+0000 7fbb2edbf740 20 client.8019 populate_metadata read hostname 'smithi086' 2024-04-20T21:26:31.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.834+0000 7fbb2edbf740 1 --2- 172.21.15.86:0/1097675693 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x55849eaea060 0x55849eaea440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:31.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.834+0000 7fbb2edbf740 4 client.8019 mds_command: new command op to 7773 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-20T21:26:31.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.834+0000 7fbb2edbf740 1 -- 172.21.15.86:0/1097675693 --> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55849ea94ed0 con 0x55849eaea060 2024-04-20T21:26:31.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.835+0000 7fbb29600640 1 --2- 172.21.15.86:0/1097675693 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x55849eaea060 0x55849eaea440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:31.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.836+0000 7fbb29600640 1 --2- 172.21.15.86:0/1097675693 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x55849eaea060 0x55849eaea440 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=701d319858d31ad4 server_cookie=942848ddf0f10db0 in_seq=0 out_seq=0 2024-04-20T21:26:31.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.836+0000 7fbb0a000640 10 client.8019 ms_handle_connect on v2:172.21.15.144:6833/3981403733 2024-04-20T21:26:31.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.836+0000 7fbb0a000640 1 -- 172.21.15.86:0/1097675693 <== mds.0 v2:172.21.15.144:6833/3981403733 1 ==== command_reply(tid 0: 0 ) ==== 8+0+144 (crc 0 0 0) 0x55849ea94ed0 con 0x55849eaea060 2024-04-20T21:26:31.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.836+0000 7fbb0a000640 10 client.8019 handle_command_reply: tid=0 2024-04-20T21:26:31.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.836+0000 7fbb2edbf740 1 client.8019 shutdown 2024-04-20T21:26:31.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.837+0000 7fbb2edbf740 1 -- 172.21.15.86:0/1097675693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55849eae1a30 msgr2=0x55849eabef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:31.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.837+0000 7fbb2edbf740 1 --2- 172.21.15.86:0/1097675693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55849eae1a30 0x55849eabef30 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c07b8d0 tx=0x7fbb1c001e20 comp rx=0 tx=0).stop 2024-04-20T21:26:31.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.837+0000 7fbb2edbf740 1 -- 172.21.15.86:0/1097675693 shutdown_connections 2024-04-20T21:26:31.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.837+0000 7fbb2edbf740 1 --2- 172.21.15.86:0/1097675693 >> [v2:172.21.15.144:6833/3981403733,v1:172.21.15.144:6835/3981403733] conn(0x55849eaea060 0x55849eaea440 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.837+0000 7fbb2edbf740 1 --2- 172.21.15.86:0/1097675693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55849eae1a30 0x55849eabef30 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.837+0000 7fbb2edbf740 1 --2- 172.21.15.86:0/1097675693 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55849eac1270 0x55849eb5c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.837+0000 7fbb0a000640 0 client.8019 ms_handle_reset on v2:172.21.15.144:6833/3981403733 2024-04-20T21:26:31.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.837+0000 7fbb2edbf740 1 --2- 172.21.15.86:0/1097675693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55849eac0ac0 0x55849eabf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.837+0000 7fbb2edbf740 1 -- 172.21.15.86:0/1097675693 >> 172.21.15.86:0/1097675693 conn(0x55849ea994a0 msgr2=0x55849ea8f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:31.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.837+0000 7fbb2edbf740 1 -- 172.21.15.86:0/1097675693 shutdown_connections 2024-04-20T21:26:31.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.837+0000 7fbb2edbf740 1 -- 172.21.15.86:0/1097675693 wait complete. 2024-04-20T21:26:31.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.837+0000 7fbb2edbf740 20 client.8019 trim_cache size 0 max 16384 2024-04-20T21:26:31.840 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:26:31.840 INFO:teuthology.orchestra.run.smithi086.stdout:[ 2024-04-20T21:26:31.840 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:26:31.840 INFO:teuthology.orchestra.run.smithi086.stdout: "damage_type": "backtrace", 2024-04-20T21:26:31.840 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 4026297729, 2024-04-20T21:26:31.840 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 1099511627777, 2024-04-20T21:26:31.840 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/testdir/hardlink" 2024-04-20T21:26:31.841 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:26:31.841 INFO:teuthology.orchestra.run.smithi086.stdout:] 2024-04-20T21:26:31.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.838+0000 7fbb2c000640 1 -- 172.21.15.86:0/322864733 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbb0406cac0 msgr2=0x7fbb0406ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:31.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.838+0000 7fbb2c000640 1 --2- 172.21.15.86:0/322864733 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbb0406cac0 0x7fbb0406ef80 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fbb18009aa0 tx=0x7fbb18008040 comp rx=0 tx=0).stop 2024-04-20T21:26:31.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.838+0000 7fbb2c000640 1 -- 172.21.15.86:0/322864733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb241e7f10 msgr2=0x7fbb241e83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:31.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.838+0000 7fbb2c000640 1 --2- 172.21.15.86:0/322864733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb241e7f10 0x7fbb241e83a0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c00a830 tx=0x7fbb2415d790 comp rx=0 tx=0).stop 2024-04-20T21:26:31.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.838+0000 7fbb2c000640 1 -- 172.21.15.86:0/322864733 shutdown_connections 2024-04-20T21:26:31.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.838+0000 7fbb2c000640 1 --2- 172.21.15.86:0/322864733 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbb0406cac0 0x7fbb0406ef80 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.838+0000 7fbb2c000640 1 --2- 172.21.15.86:0/322864733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb241e7f10 0x7fbb241e83a0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.838+0000 7fbb2c000640 1 --2- 172.21.15.86:0/322864733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbb2415c320 0x7fbb240c6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.838+0000 7fbb2c000640 1 --2- 172.21.15.86:0/322864733 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbb0c004d40 0x7fbb240c9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:31.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.838+0000 7fbb2c000640 1 -- 172.21.15.86:0/322864733 >> 172.21.15.86:0/322864733 conn(0x7fbb240b2110 msgr2=0x7fbb2415f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:31.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.838+0000 7fbb2c000640 1 -- 172.21.15.86:0/322864733 shutdown_connections 2024-04-20T21:26:31.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:31.838+0000 7fbb2c000640 1 -- 172.21.15.86:0/322864733 wait complete. 2024-04-20T21:26:31.855 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:26:31.855 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:26:31.903 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:26:32.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.153+0000 7f9e20c00640 1 Processor -- start 2024-04-20T21:26:32.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.153+0000 7f9e20c00640 1 -- start start 2024-04-20T21:26:32.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.155+0000 7f9e20c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e1c15c010 0x7f9e1c15c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:32.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.155+0000 7f9e20c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e1c0ae710 con 0x7f9e1c06f4c0 2024-04-20T21:26:32.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.155+0000 7f9e20c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e1c0ae9f0 con 0x7f9e1c0af7b0 2024-04-20T21:26:32.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.155+0000 7f9e20c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9e1c0af320 con 0x7f9e1c15c010 2024-04-20T21:26:32.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.155+0000 7f9e1be00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f9e1c06f4c0 0x7f9e1c071b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34736/0 (socket says 172.21.15.86:34736) 2024-04-20T21:26:32.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.155+0000 7f9e1be00640 1 -- 172.21.15.86:0/2022444418 learned_addr learned my addr 172.21.15.86:0/2022444418 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:26:32.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.155+0000 7f9e1a000640 1 -- 172.21.15.86:0/2022444418 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2447271191 0 0) 0x7f9e1c0ae710 con 0x7f9e1c06f4c0 2024-04-20T21:26:32.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.155+0000 7f9e1a000640 1 -- 172.21.15.86:0/2022444418 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9e00003520 con 0x7f9e1c06f4c0 2024-04-20T21:26:32.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.156+0000 7f9e1b400640 1 --2- 172.21.15.86:0/2022444418 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e1c15c010 0x7f9e1c15c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:32.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.156+0000 7f9e1a000640 1 -- 172.21.15.86:0/2022444418 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 443432792 0 0) 0x7f9e00003520 con 0x7f9e1c06f4c0 2024-04-20T21:26:32.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.156+0000 7f9e1a000640 1 -- 172.21.15.86:0/2022444418 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e1c15c010 msgr2=0x7f9e1c15c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:32.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.156+0000 7f9e1a000640 1 --2- 172.21.15.86:0/2022444418 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e1c15c010 0x7f9e1c15c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:32.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.156+0000 7f9e1a000640 1 -- 172.21.15.86:0/2022444418 >> v1:172.21.15.144:6789/0 conn(0x7f9e1c0af7b0 legacy=0x7f9e1c0adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:32.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.156+0000 7f9e1a000640 1 -- 172.21.15.86:0/2022444418 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e1c0aecd0 con 0x7f9e1c06f4c0 2024-04-20T21:26:32.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.156+0000 7f9e1a000640 1 -- 172.21.15.86:0/2022444418 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f9e140019f0 con 0x7f9e1c06f4c0 2024-04-20T21:26:32.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.156+0000 7f9e1a000640 1 -- 172.21.15.86:0/2022444418 >> v1:172.21.15.86:6789/0 conn(0x7f9e1c06f4c0 legacy=0x7f9e1c071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:32.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.156+0000 7f9e1a000640 1 --2- 172.21.15.86:0/2022444418 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e1c15c010 0x7f9e000039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:32.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.156+0000 7f9e1a000640 1 --2- 172.21.15.86:0/2022444418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e00004010 0x7f9e00004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:32.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.156+0000 7f9e1a000640 1 --2- 172.21.15.86:0/2022444418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e00004d40 0x7f9e000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:32.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.156+0000 7f9e1a000640 1 -- 172.21.15.86:0/2022444418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c0af320 con 0x7f9e00004010 2024-04-20T21:26:32.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.156+0000 7f9e1a000640 1 -- 172.21.15.86:0/2022444418 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0000bb80 con 0x7f9e00004d40 2024-04-20T21:26:32.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.156+0000 7f9e1a000640 1 -- 172.21.15.86:0/2022444418 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9e0000df90 con 0x7f9e1c15c010 2024-04-20T21:26:32.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.156+0000 7f9e1aa00640 1 --2- 172.21.15.86:0/2022444418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e00004010 0x7f9e00004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:32.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.157+0000 7f9e1b400640 1 --2- 172.21.15.86:0/2022444418 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e1c15c010 0x7f9e000039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:32.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.157+0000 7f9e1aa00640 1 -- 172.21.15.86:0/2022444418 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e1c15c010 msgr2=0x7f9e000039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:32.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.157+0000 7f9e1aa00640 1 --2- 172.21.15.86:0/2022444418 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e1c15c010 0x7f9e000039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:32.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.157+0000 7f9e1be00640 1 --2- 172.21.15.86:0/2022444418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e00004d40 0x7f9e000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:32.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.157+0000 7f9e1aa00640 1 -- 172.21.15.86:0/2022444418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e00004d40 msgr2=0x7f9e000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:32.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.157+0000 7f9e1aa00640 1 --2- 172.21.15.86:0/2022444418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e00004d40 0x7f9e000051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:32.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.157+0000 7f9e1aa00640 1 -- 172.21.15.86:0/2022444418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e00003520 con 0x7f9e00004010 2024-04-20T21:26:32.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.158+0000 7f9e1aa00640 1 --2- 172.21.15.86:0/2022444418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e00004010 0x7f9e00004800 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f9e08001750 tx=0x7f9e00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=748357be63df9853 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:32.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.158+0000 7f9e1a000640 1 -- 172.21.15.86:0/2022444418 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e08001370 con 0x7f9e00004010 2024-04-20T21:26:32.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.158+0000 7f9e1a000640 1 -- 172.21.15.86:0/2022444418 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e080052d0 con 0x7f9e00004010 2024-04-20T21:26:32.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.158+0000 7f9e1a000640 1 -- 172.21.15.86:0/2022444418 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e08003830 con 0x7f9e00004010 2024-04-20T21:26:32.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.159+0000 7f9e20c00640 1 -- 172.21.15.86:0/2022444418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e00004010 msgr2=0x7f9e00004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:32.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.159+0000 7f9e20c00640 1 --2- 172.21.15.86:0/2022444418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e00004010 0x7f9e00004800 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f9e08001750 tx=0x7f9e00005830 comp rx=0 tx=0).stop 2024-04-20T21:26:32.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.159+0000 7f9e1be00640 1 -- 172.21.15.86:0/2022444418 reap_dead start 2024-04-20T21:26:32.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.159+0000 7f9e20c00640 1 -- 172.21.15.86:0/2022444418 shutdown_connections 2024-04-20T21:26:32.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.159+0000 7f9e20c00640 1 -- 172.21.15.86:0/2022444418 >> 172.21.15.86:0/2022444418 conn(0x7f9e1c0d0170 msgr2=0x7f9e1c0d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:32.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.159+0000 7f9e20c00640 1 -- 172.21.15.86:0/2022444418 shutdown_connections 2024-04-20T21:26:32.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.159+0000 7f9e20c00640 1 -- 172.21.15.86:0/2022444418 wait complete. 2024-04-20T21:26:32.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.161+0000 7f9e20c00640 1 Processor -- start 2024-04-20T21:26:32.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.161+0000 7f9e20c00640 1 -- start start 2024-04-20T21:26:32.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.162+0000 7f9e20c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e00004010 0x7f9e1c15e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:32.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.162+0000 7f9e20c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e1c0cecc0 0x7f9e1c1f3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:32.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.162+0000 7f9e20c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e1c1f4210 0x7f9e1c1562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:32.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.162+0000 7f9e20c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c06f970 con 0x7f9e00004010 2024-04-20T21:26:32.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.162+0000 7f9e20c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c0ae4b0 con 0x7f9e1c1f4210 2024-04-20T21:26:32.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.162+0000 7f9e20c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9e1c06faf0 con 0x7f9e1c0cecc0 2024-04-20T21:26:32.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.162+0000 7f9e1b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e00004010 0x7f9e1c15e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:32.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.162+0000 7f9e1b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e00004010 0x7f9e1c15e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43812/0 (socket says 172.21.15.86:43812) 2024-04-20T21:26:32.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.162+0000 7f9e1b400640 1 -- 172.21.15.86:0/3234028660 learned_addr learned my addr 172.21.15.86:0/3234028660 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:32.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.162+0000 7f9e1b400640 1 -- 172.21.15.86:0/3234028660 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e1c0cecc0 msgr2=0x7f9e1c1f3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:32.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.162+0000 7f9e1b400640 1 --2- 172.21.15.86:0/3234028660 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e1c0cecc0 0x7f9e1c1f3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:32.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.163+0000 7f9e1b400640 1 -- 172.21.15.86:0/3234028660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e1c1f4210 msgr2=0x7f9e1c1562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:32.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.163+0000 7f9e1be00640 1 --2- 172.21.15.86:0/3234028660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e1c1f4210 0x7f9e1c1562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:32.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.163+0000 7f9e1b400640 1 --2- 172.21.15.86:0/3234028660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e1c1f4210 0x7f9e1c1562c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:32.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.163+0000 7f9e1b400640 1 -- 172.21.15.86:0/3234028660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e1c154730 con 0x7f9e00004010 2024-04-20T21:26:32.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.163+0000 7f9e1b400640 1 --2- 172.21.15.86:0/3234028660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e00004010 0x7f9e1c15e4d0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c00f5b0 tx=0x7f9e0c00fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:32.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.163+0000 7f9e13400640 1 -- 172.21.15.86:0/3234028660 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c019020 con 0x7f9e00004010 2024-04-20T21:26:32.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.163+0000 7f9e13400640 1 -- 172.21.15.86:0/3234028660 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e0c014870 con 0x7f9e00004010 2024-04-20T21:26:32.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.163+0000 7f9e13400640 1 -- 172.21.15.86:0/3234028660 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c014c60 con 0x7f9e00004010 2024-04-20T21:26:32.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.163+0000 7f9e20c00640 1 -- 172.21.15.86:0/3234028660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e1c0b39a0 con 0x7f9e00004010 2024-04-20T21:26:32.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.163+0000 7f9e20c00640 1 -- 172.21.15.86:0/3234028660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e1c154a10 con 0x7f9e00004010 2024-04-20T21:26:32.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.164+0000 7f9e20c00640 1 -- 172.21.15.86:0/3234028660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e1c04fcf0 con 0x7f9e00004010 2024-04-20T21:26:32.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.165+0000 7f9e13400640 1 -- 172.21.15.86:0/3234028660 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9e0c01c020 con 0x7f9e00004010 2024-04-20T21:26:32.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.167+0000 7f9e13400640 1 --2- 172.21.15.86:0/3234028660 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9dfc06c8f0 0x7f9dfc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:32.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.167+0000 7f9e13400640 1 -- 172.21.15.86:0/3234028660 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7f9e0c08c3e0 con 0x7f9e00004010 2024-04-20T21:26:32.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.167+0000 7f9e1aa00640 1 --2- 172.21.15.86:0/3234028660 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9dfc06c8f0 0x7f9dfc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:32.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.168+0000 7f9e1aa00640 1 --2- 172.21.15.86:0/3234028660 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9dfc06c8f0 0x7f9dfc06edb0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f9e08033000 tx=0x7f9e0802f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:32.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.171+0000 7f9e13400640 1 -- 172.21.15.86:0/3234028660 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9e0c023020 con 0x7f9e00004010 2024-04-20T21:26:32.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.373+0000 7f9e20c00640 1 -- 172.21.15.86:0/3234028660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9e1c15a8b0 con 0x7f9e00004010 2024-04-20T21:26:32.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.376+0000 7f9e13400640 1 -- 172.21.15.86:0/3234028660 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+16841 (secure 0 0 0) 0x7f9e0c026350 con 0x7f9e00004010 2024-04-20T21:26:32.377 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:26:32.377 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":88,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:25:35.641040+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:24:55.964540+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-20T21:24:57.241527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.3e","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.144:6835/1740492227":"2024-04-21T21:25:20.040733+0000","172.21.15.144:6833/1740492227":"2024-04-21T21:25:20.040733+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:26:32.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.379+0000 7f9e20c00640 1 -- 172.21.15.86:0/3234028660 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9dfc06c8f0 msgr2=0x7f9dfc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:32.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.379+0000 7f9e20c00640 1 --2- 172.21.15.86:0/3234028660 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9dfc06c8f0 0x7f9dfc06edb0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f9e08033000 tx=0x7f9e0802f000 comp rx=0 tx=0).stop 2024-04-20T21:26:32.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.379+0000 7f9e20c00640 1 -- 172.21.15.86:0/3234028660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e00004010 msgr2=0x7f9e1c15e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:32.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.379+0000 7f9e20c00640 1 --2- 172.21.15.86:0/3234028660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e00004010 0x7f9e1c15e4d0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c00f5b0 tx=0x7f9e0c00fa70 comp rx=0 tx=0).stop 2024-04-20T21:26:32.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.379+0000 7f9e20c00640 1 -- 172.21.15.86:0/3234028660 shutdown_connections 2024-04-20T21:26:32.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.379+0000 7f9e20c00640 1 --2- 172.21.15.86:0/3234028660 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9dfc06c8f0 0x7f9dfc06edb0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:32.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.379+0000 7f9e20c00640 1 --2- 172.21.15.86:0/3234028660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e1c1f4210 0x7f9e1c1562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:32.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.379+0000 7f9e20c00640 1 --2- 172.21.15.86:0/3234028660 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9e1c0cecc0 0x7f9e1c1f3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:32.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.379+0000 7f9e20c00640 1 --2- 172.21.15.86:0/3234028660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e00004010 0x7f9e1c15e4d0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:32.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.379+0000 7f9e20c00640 1 -- 172.21.15.86:0/3234028660 >> 172.21.15.86:0/3234028660 conn(0x7f9e1c0d0170 msgr2=0x7f9e1c159c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:32.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.379+0000 7f9e20c00640 1 -- 172.21.15.86:0/3234028660 shutdown_connections 2024-04-20T21:26:32.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.379+0000 7f9e20c00640 1 -- 172.21.15.86:0/3234028660 wait complete. 2024-04-20T21:26:32.398 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:24:55.964540+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:24:57.241527+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:26:32.399 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:26:32.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.605+0000 7fcd6e000640 1 Processor -- start 2024-04-20T21:26:32.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.605+0000 7fcd6e000640 1 -- start start 2024-04-20T21:26:32.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.607+0000 7fcd6e000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fcd6806e0d0 0x7fcd6806e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:32.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.607+0000 7fcd6e000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd681555a0 con 0x7fcd680b3650 2024-04-20T21:26:32.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.607+0000 7fcd6e000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd6815b920 con 0x7fcd68155120 2024-04-20T21:26:32.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.607+0000 7fcd6e000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fcd680b3a30 con 0x7fcd6806e0d0 2024-04-20T21:26:32.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.607+0000 7fcd6cc00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fcd680b3650 0x7fcd6815b0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34738/0 (socket says 172.21.15.86:34738) 2024-04-20T21:26:32.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.607+0000 7fcd6cc00640 1 -- 172.21.15.86:0/1437915127 learned_addr learned my addr 172.21.15.86:0/1437915127 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:26:32.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.607+0000 7fcd67400640 1 -- 172.21.15.86:0/1437915127 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3426881591 0 0) 0x7fcd681555a0 con 0x7fcd680b3650 2024-04-20T21:26:32.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.607+0000 7fcd67400640 1 -- 172.21.15.86:0/1437915127 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcd50005730 con 0x7fcd680b3650 2024-04-20T21:26:32.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.607+0000 7fcd6d600640 1 --2- 172.21.15.86:0/1437915127 >> v2:172.21.15.144:3300/0 conn(0x7fcd6806e0d0 0x7fcd6806e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:32.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.608+0000 7fcd67400640 1 -- 172.21.15.86:0/1437915127 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4025469886 0 0) 0x7fcd50005730 con 0x7fcd680b3650 2024-04-20T21:26:32.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.608+0000 7fcd67400640 1 -- 172.21.15.86:0/1437915127 >> v2:172.21.15.144:3300/0 conn(0x7fcd6806e0d0 msgr2=0x7fcd6806e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:32.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.608+0000 7fcd67400640 1 --2- 172.21.15.86:0/1437915127 >> v2:172.21.15.144:3300/0 conn(0x7fcd6806e0d0 0x7fcd6806e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:32.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.608+0000 7fcd67400640 1 -- 172.21.15.86:0/1437915127 >> v1:172.21.15.144:6789/0 conn(0x7fcd68155120 legacy=0x7fcd6806d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:32.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.608+0000 7fcd67400640 1 -- 172.21.15.86:0/1437915127 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd6815bc00 con 0x7fcd680b3650 2024-04-20T21:26:32.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.608+0000 7fcd67400640 1 -- 172.21.15.86:0/1437915127 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fcd5c001670 con 0x7fcd680b3650 2024-04-20T21:26:32.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.608+0000 7fcd67400640 1 -- 172.21.15.86:0/1437915127 >> v1:172.21.15.86:6789/0 conn(0x7fcd680b3650 legacy=0x7fcd6815b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:32.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.608+0000 7fcd67400640 1 --2- 172.21.15.86:0/1437915127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd50005bc0 0x7fcd50005fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:32.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.608+0000 7fcd67400640 1 --2- 172.21.15.86:0/1437915127 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd50006a10 0x7fcd50006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:32.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.608+0000 7fcd67400640 1 --2- 172.21.15.86:0/1437915127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd500073d0 0x7fcd5000dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:32.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.608+0000 7fcd67400640 1 -- 172.21.15.86:0/1437915127 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcd680b3a30 con 0x7fcd50005bc0 2024-04-20T21:26:32.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.608+0000 7fcd67400640 1 -- 172.21.15.86:0/1437915127 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5000e300 con 0x7fcd500073d0 2024-04-20T21:26:32.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.608+0000 7fcd67400640 1 -- 172.21.15.86:0/1437915127 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fcd5000e500 con 0x7fcd50006a10 2024-04-20T21:26:32.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.608+0000 7fcd6cc00640 1 --2- 172.21.15.86:0/1437915127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd50005bc0 0x7fcd50005fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:32.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.609+0000 7fcd6cc00640 1 -- 172.21.15.86:0/1437915127 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd50006a10 msgr2=0x7fcd50006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:32.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.609+0000 7fcd67e00640 1 --2- 172.21.15.86:0/1437915127 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd50006a10 0x7fcd50006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:32.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.609+0000 7fcd6cc00640 1 --2- 172.21.15.86:0/1437915127 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd50006a10 0x7fcd50006e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:32.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.609+0000 7fcd6cc00640 1 -- 172.21.15.86:0/1437915127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd500073d0 msgr2=0x7fcd5000dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:32.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.609+0000 7fcd6d600640 1 --2- 172.21.15.86:0/1437915127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd500073d0 0x7fcd5000dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:32.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.609+0000 7fcd6cc00640 1 --2- 172.21.15.86:0/1437915127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd500073d0 0x7fcd5000dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:32.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.609+0000 7fcd6cc00640 1 -- 172.21.15.86:0/1437915127 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd50005730 con 0x7fcd50005bc0 2024-04-20T21:26:32.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.609+0000 7fcd6d600640 1 -- 172.21.15.86:0/1437915127 reap_dead start 2024-04-20T21:26:32.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.610+0000 7fcd6cc00640 1 --2- 172.21.15.86:0/1437915127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd50005bc0 0x7fcd50005fc0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c007fd0 tx=0x7fcd50007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=508eb88e51b39dec server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:32.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.610+0000 7fcd67400640 1 -- 172.21.15.86:0/1437915127 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c007d50 con 0x7fcd50005bc0 2024-04-20T21:26:32.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.610+0000 7fcd67400640 1 -- 172.21.15.86:0/1437915127 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd5c030070 con 0x7fcd50005bc0 2024-04-20T21:26:32.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.610+0000 7fcd67400640 1 -- 172.21.15.86:0/1437915127 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c02cd80 con 0x7fcd50005bc0 2024-04-20T21:26:32.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.611+0000 7fcd6e000640 1 -- 172.21.15.86:0/1437915127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd50005bc0 msgr2=0x7fcd50005fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:32.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.611+0000 7fcd6e000640 1 --2- 172.21.15.86:0/1437915127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd50005bc0 0x7fcd50005fc0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c007fd0 tx=0x7fcd50007a40 comp rx=0 tx=0).stop 2024-04-20T21:26:32.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.611+0000 7fcd6e000640 1 -- 172.21.15.86:0/1437915127 shutdown_connections 2024-04-20T21:26:32.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.611+0000 7fcd6e000640 1 --2- 172.21.15.86:0/1437915127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd50005bc0 0x7fcd50005fc0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:32.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.611+0000 7fcd6e000640 1 -- 172.21.15.86:0/1437915127 >> 172.21.15.86:0/1437915127 conn(0x7fcd680b1f00 msgr2=0x7fcd680b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:32.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.611+0000 7fcd6e000640 1 -- 172.21.15.86:0/1437915127 shutdown_connections 2024-04-20T21:26:32.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.611+0000 7fcd6e000640 1 -- 172.21.15.86:0/1437915127 wait complete. 2024-04-20T21:26:32.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.613+0000 7fcd6e000640 1 Processor -- start 2024-04-20T21:26:32.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.613+0000 7fcd6e000640 1 -- start start 2024-04-20T21:26:32.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.614+0000 7fcd6e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd50005bc0 0x7fcd6814d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:32.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.614+0000 7fcd6e000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd681f4bd0 0x7fcd68158130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:32.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.614+0000 7fcd6e000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd68158670 0x7fcd681569f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:32.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.614+0000 7fcd6e000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6806d9c0 con 0x7fcd50005bc0 2024-04-20T21:26:32.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.614+0000 7fcd6e000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6815c530 con 0x7fcd681f4bd0 2024-04-20T21:26:32.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.614+0000 7fcd6e000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fcd6806db40 con 0x7fcd68158670 2024-04-20T21:26:32.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.614+0000 7fcd6cc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd50005bc0 0x7fcd6814d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:32.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.614+0000 7fcd6cc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd50005bc0 0x7fcd6814d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43832/0 (socket says 172.21.15.86:43832) 2024-04-20T21:26:32.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.614+0000 7fcd6cc00640 1 -- 172.21.15.86:0/2182812727 learned_addr learned my addr 172.21.15.86:0/2182812727 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:32.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.615+0000 7fcd6cc00640 1 -- 172.21.15.86:0/2182812727 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd68158670 msgr2=0x7fcd681569f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:32.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.615+0000 7fcd6d600640 1 --2- 172.21.15.86:0/2182812727 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd68158670 0x7fcd681569f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:32.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.615+0000 7fcd6cc00640 1 --2- 172.21.15.86:0/2182812727 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd68158670 0x7fcd681569f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:32.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.615+0000 7fcd67e00640 1 --2- 172.21.15.86:0/2182812727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd681f4bd0 0x7fcd68158130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:32.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.615+0000 7fcd6cc00640 1 -- 172.21.15.86:0/2182812727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd681f4bd0 msgr2=0x7fcd68158130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:32.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.615+0000 7fcd6cc00640 1 --2- 172.21.15.86:0/2182812727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd681f4bd0 0x7fcd68158130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:32.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.615+0000 7fcd6cc00640 1 -- 172.21.15.86:0/2182812727 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd68155950 con 0x7fcd50005bc0 2024-04-20T21:26:32.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.615+0000 7fcd67e00640 1 --2- 172.21.15.86:0/2182812727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd681f4bd0 0x7fcd68158130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:26:32.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.615+0000 7fcd6cc00640 1 --2- 172.21.15.86:0/2182812727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd50005bc0 0x7fcd6814d820 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c00c380 tx=0x7fcd5c02e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:32.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.615+0000 7fcd64c00640 1 -- 172.21.15.86:0/2182812727 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c007d50 con 0x7fcd50005bc0 2024-04-20T21:26:32.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.615+0000 7fcd64c00640 1 -- 172.21.15.86:0/2182812727 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd5c004bf0 con 0x7fcd50005bc0 2024-04-20T21:26:32.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.615+0000 7fcd64c00640 1 -- 172.21.15.86:0/2182812727 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c007070 con 0x7fcd50005bc0 2024-04-20T21:26:32.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.615+0000 7fcd6e000640 1 -- 172.21.15.86:0/2182812727 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd68155c30 con 0x7fcd50005bc0 2024-04-20T21:26:32.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.615+0000 7fcd6e000640 1 -- 172.21.15.86:0/2182812727 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd681561f0 con 0x7fcd50005bc0 2024-04-20T21:26:32.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.617+0000 7fcd6e000640 1 -- 172.21.15.86:0/2182812727 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd6804fd90 con 0x7fcd50005bc0 2024-04-20T21:26:32.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.618+0000 7fcd64c00640 1 -- 172.21.15.86:0/2182812727 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcd5c048020 con 0x7fcd50005bc0 2024-04-20T21:26:32.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.619+0000 7fcd64c00640 1 --2- 172.21.15.86:0/2182812727 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcd4806c8f0 0x7fcd4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:32.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.619+0000 7fcd64c00640 1 -- 172.21.15.86:0/2182812727 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7fcd5c0acf60 con 0x7fcd50005bc0 2024-04-20T21:26:32.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.619+0000 7fcd67e00640 1 --2- 172.21.15.86:0/2182812727 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcd4806c8f0 0x7fcd4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:32.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.620+0000 7fcd67e00640 1 --2- 172.21.15.86:0/2182812727 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcd4806c8f0 0x7fcd4806edb0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fcd6809e400 tx=0x7fcd54009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:32.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.624+0000 7fcd64c00640 1 -- 172.21.15.86:0/2182812727 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fcd5c07b3b0 con 0x7fcd50005bc0 2024-04-20T21:26:32.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.825+0000 7fcd6e000640 1 -- 172.21.15.86:0/2182812727 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcd68158a50 con 0x7fcd50005bc0 2024-04-20T21:26:32.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.826+0000 7fcd64c00640 1 -- 172.21.15.86:0/2182812727 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7fcd5c07acb0 con 0x7fcd50005bc0 2024-04-20T21:26:32.826 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:26:32.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.828+0000 7fcd6e000640 1 -- 172.21.15.86:0/2182812727 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcd4806c8f0 msgr2=0x7fcd4806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:32.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.828+0000 7fcd6e000640 1 --2- 172.21.15.86:0/2182812727 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcd4806c8f0 0x7fcd4806edb0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fcd6809e400 tx=0x7fcd54009040 comp rx=0 tx=0).stop 2024-04-20T21:26:32.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.828+0000 7fcd6e000640 1 -- 172.21.15.86:0/2182812727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd50005bc0 msgr2=0x7fcd6814d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:32.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.828+0000 7fcd6e000640 1 --2- 172.21.15.86:0/2182812727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd50005bc0 0x7fcd6814d820 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c00c380 tx=0x7fcd5c02e070 comp rx=0 tx=0).stop 2024-04-20T21:26:32.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.829+0000 7fcd6e000640 1 -- 172.21.15.86:0/2182812727 shutdown_connections 2024-04-20T21:26:32.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.829+0000 7fcd6e000640 1 --2- 172.21.15.86:0/2182812727 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcd4806c8f0 0x7fcd4806edb0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:32.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.829+0000 7fcd6e000640 1 --2- 172.21.15.86:0/2182812727 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcd68158670 0x7fcd681569f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:32.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.829+0000 7fcd6e000640 1 --2- 172.21.15.86:0/2182812727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd681f4bd0 0x7fcd68158130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:32.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.829+0000 7fcd6e000640 1 --2- 172.21.15.86:0/2182812727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd50005bc0 0x7fcd6814d820 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:32.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.829+0000 7fcd6e000640 1 -- 172.21.15.86:0/2182812727 >> 172.21.15.86:0/2182812727 conn(0x7fcd680b1f00 msgr2=0x7fcd68073940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:32.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.829+0000 7fcd6e000640 1 -- 172.21.15.86:0/2182812727 shutdown_connections 2024-04-20T21:26:32.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:32.829+0000 7fcd6e000640 1 -- 172.21.15.86:0/2182812727 wait complete. 2024-04-20T21:26:32.848 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:26:33.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.043+0000 7f3547000640 1 Processor -- start 2024-04-20T21:26:33.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.044+0000 7f3547000640 1 -- start start 2024-04-20T21:26:33.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.045+0000 7f3547000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f354006fed0 0x7f35400cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:33.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.045+0000 7f3547000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f354009a9e0 0x7f354009ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:33.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.045+0000 7f3547000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3540070390 con 0x7f3540160310 2024-04-20T21:26:33.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.045+0000 7f3547000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f3540070fa0 con 0x7f354006fed0 2024-04-20T21:26:33.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.045+0000 7f3547000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3540070670 con 0x7f354009a9e0 2024-04-20T21:26:33.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.046+0000 7f3545c00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f3540160310 0x7f35401606f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34744/0 (socket says 172.21.15.86:34744) 2024-04-20T21:26:33.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.046+0000 7f3545c00640 1 -- 172.21.15.86:0/1936624870 learned_addr learned my addr 172.21.15.86:0/1936624870 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:26:33.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.046+0000 7f3545200640 1 --2- 172.21.15.86:0/1936624870 >> v2:172.21.15.86:3300/0 conn(0x7f354006fed0 0x7f35400cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:33.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.046+0000 7f3546600640 1 --2- 172.21.15.86:0/1936624870 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f354009a9e0 0x7f354009ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:33.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.046+0000 7f353fe00640 1 -- 172.21.15.86:0/1936624870 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2000582679 0 0) 0x7f3540070390 con 0x7f3540160310 2024-04-20T21:26:33.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.046+0000 7f353fe00640 1 -- 172.21.15.86:0/1936624870 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3524003520 con 0x7f3540160310 2024-04-20T21:26:33.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.046+0000 7f3545200640 1 -- 172.21.15.86:0/1936624870 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f354009a9e0 msgr2=0x7f354009ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:33.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.046+0000 7f3545200640 1 --2- 172.21.15.86:0/1936624870 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f354009a9e0 0x7f354009ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:33.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.046+0000 7f3545200640 1 -- 172.21.15.86:0/1936624870 >> v1:172.21.15.86:6789/0 conn(0x7f3540160310 legacy=0x7f35401606f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:33.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.046+0000 7f3545200640 1 -- 172.21.15.86:0/1936624870 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3540070950 con 0x7f354006fed0 2024-04-20T21:26:33.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.046+0000 7f3546600640 1 --2- 172.21.15.86:0/1936624870 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f354009a9e0 0x7f354009ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:26:33.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.047+0000 7f3545200640 1 --2- 172.21.15.86:0/1936624870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f354006fed0 0x7f35400cddf0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f352c0097a0 tx=0x7f352c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef3c2957708f8153 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:33.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.048+0000 7f353fe00640 1 -- 172.21.15.86:0/1936624870 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352c035020 con 0x7f354006fed0 2024-04-20T21:26:33.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.048+0000 7f353fe00640 1 -- 172.21.15.86:0/1936624870 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f352c031810 con 0x7f354006fed0 2024-04-20T21:26:33.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.048+0000 7f353fe00640 1 -- 172.21.15.86:0/1936624870 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352c031be0 con 0x7f354006fed0 2024-04-20T21:26:33.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.048+0000 7f3547000640 1 -- 172.21.15.86:0/1936624870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f354006fed0 msgr2=0x7f35400cddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:33.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.048+0000 7f3547000640 1 --2- 172.21.15.86:0/1936624870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f354006fed0 0x7f35400cddf0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f352c0097a0 tx=0x7f352c02b8b0 comp rx=0 tx=0).stop 2024-04-20T21:26:33.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.048+0000 7f3547000640 1 -- 172.21.15.86:0/1936624870 shutdown_connections 2024-04-20T21:26:33.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.048+0000 7f3547000640 1 --2- 172.21.15.86:0/1936624870 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f354009a9e0 0x7f354009ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:33.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.048+0000 7f3547000640 1 --2- 172.21.15.86:0/1936624870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f354006fed0 0x7f35400cddf0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:33.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.048+0000 7f3547000640 1 -- 172.21.15.86:0/1936624870 >> 172.21.15.86:0/1936624870 conn(0x7f35400b3f90 msgr2=0x7f35400b43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:33.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.049+0000 7f3547000640 1 -- 172.21.15.86:0/1936624870 shutdown_connections 2024-04-20T21:26:33.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.049+0000 7f3547000640 1 -- 172.21.15.86:0/1936624870 wait complete. 2024-04-20T21:26:33.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.050+0000 7f3547000640 1 Processor -- start 2024-04-20T21:26:33.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.050+0000 7f3547000640 1 -- start start 2024-04-20T21:26:33.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.051+0000 7f3547000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f354006fed0 0x7f354014d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:33.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.051+0000 7f3547000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f354009a9e0 0x7f354014d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:33.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.051+0000 7f3547000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3540160310 0x7f354014f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:33.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.051+0000 7f3547000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f35400ce330 con 0x7f354009a9e0 2024-04-20T21:26:33.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.051+0000 7f3547000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f354015db80 con 0x7f3540160310 2024-04-20T21:26:33.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.051+0000 7f3547000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f35400ce4b0 con 0x7f354006fed0 2024-04-20T21:26:33.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.052+0000 7f3545200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f354009a9e0 0x7f354014d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:33.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.052+0000 7f3545200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f354009a9e0 0x7f354014d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43838/0 (socket says 172.21.15.86:43838) 2024-04-20T21:26:33.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.052+0000 7f3545200640 1 -- 172.21.15.86:0/2781634801 learned_addr learned my addr 172.21.15.86:0/2781634801 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:33.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.052+0000 7f3546600640 1 --2- 172.21.15.86:0/2781634801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3540160310 0x7f354014f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:33.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.052+0000 7f3545c00640 1 --2- 172.21.15.86:0/2781634801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f354006fed0 0x7f354014d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:33.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.052+0000 7f3545200640 1 -- 172.21.15.86:0/2781634801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f354006fed0 msgr2=0x7f354014d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:33.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.052+0000 7f3545200640 1 --2- 172.21.15.86:0/2781634801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f354006fed0 0x7f354014d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:33.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.052+0000 7f3545200640 1 -- 172.21.15.86:0/2781634801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3540160310 msgr2=0x7f354014f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:33.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.052+0000 7f3545200640 1 --2- 172.21.15.86:0/2781634801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3540160310 0x7f354014f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:33.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.052+0000 7f3545200640 1 -- 172.21.15.86:0/2781634801 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35400af290 con 0x7f354009a9e0 2024-04-20T21:26:33.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.052+0000 7f3546600640 1 --2- 172.21.15.86:0/2781634801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3540160310 0x7f354014f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:26:33.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.052+0000 7f3545200640 1 --2- 172.21.15.86:0/2781634801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f354009a9e0 0x7f354014d9a0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f352c0098c0 tx=0x7f352c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:33.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.053+0000 7f353d600640 1 -- 172.21.15.86:0/2781634801 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352c041070 con 0x7f354009a9e0 2024-04-20T21:26:33.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.053+0000 7f353d600640 1 -- 172.21.15.86:0/2781634801 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f352c03ad00 con 0x7f354009a9e0 2024-04-20T21:26:33.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.053+0000 7f3547000640 1 -- 172.21.15.86:0/2781634801 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35400af570 con 0x7f354009a9e0 2024-04-20T21:26:33.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.053+0000 7f353d600640 1 -- 172.21.15.86:0/2781634801 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352c035040 con 0x7f354009a9e0 2024-04-20T21:26:33.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.053+0000 7f3547000640 1 -- 172.21.15.86:0/2781634801 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35400cc470 con 0x7f354009a9e0 2024-04-20T21:26:33.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.054+0000 7f3547000640 1 -- 172.21.15.86:0/2781634801 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f354004fd90 con 0x7f354009a9e0 2024-04-20T21:26:33.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.055+0000 7f353d600640 1 -- 172.21.15.86:0/2781634801 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f352c04c060 con 0x7f354009a9e0 2024-04-20T21:26:33.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.056+0000 7f353d600640 1 --2- 172.21.15.86:0/2781634801 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f352006c920 0x7f352006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:33.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.056+0000 7f353d600640 1 -- 172.21.15.86:0/2781634801 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7f352c0ad790 con 0x7f354009a9e0 2024-04-20T21:26:33.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.056+0000 7f3545c00640 1 --2- 172.21.15.86:0/2781634801 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f352006c920 0x7f352006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:33.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.057+0000 7f3545c00640 1 --2- 172.21.15.86:0/2781634801 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f352006c920 0x7f352006ede0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f3530007fd0 tx=0x7f3530009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:33.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.061+0000 7f353d600640 1 -- 172.21.15.86:0/2781634801 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f352c04b2c0 con 0x7f354009a9e0 2024-04-20T21:26:33.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.257+0000 7f3547000640 1 -- 172.21.15.86:0/2781634801 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3540069300 con 0x7f354009a9e0 2024-04-20T21:26:33.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.258+0000 7f353d600640 1 -- 172.21.15.86:0/2781634801 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f352c07bad0 con 0x7f354009a9e0 2024-04-20T21:26:33.258 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-20T21:26:33.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.260+0000 7f3547000640 1 -- 172.21.15.86:0/2781634801 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f352006c920 msgr2=0x7f352006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:33.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.260+0000 7f3547000640 1 --2- 172.21.15.86:0/2781634801 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f352006c920 0x7f352006ede0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f3530007fd0 tx=0x7f3530009040 comp rx=0 tx=0).stop 2024-04-20T21:26:33.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.260+0000 7f3547000640 1 -- 172.21.15.86:0/2781634801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f354009a9e0 msgr2=0x7f354014d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:33.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.260+0000 7f3547000640 1 --2- 172.21.15.86:0/2781634801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f354009a9e0 0x7f354014d9a0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f352c0098c0 tx=0x7f352c0046c0 comp rx=0 tx=0).stop 2024-04-20T21:26:33.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.260+0000 7f3547000640 1 -- 172.21.15.86:0/2781634801 shutdown_connections 2024-04-20T21:26:33.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.260+0000 7f3547000640 1 --2- 172.21.15.86:0/2781634801 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f352006c920 0x7f352006ede0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:33.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.260+0000 7f3547000640 1 --2- 172.21.15.86:0/2781634801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3540160310 0x7f354014f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:33.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.260+0000 7f3547000640 1 --2- 172.21.15.86:0/2781634801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f354009a9e0 0x7f354014d9a0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:33.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.260+0000 7f3547000640 1 --2- 172.21.15.86:0/2781634801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f354006fed0 0x7f354014d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:33.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.260+0000 7f3547000640 1 -- 172.21.15.86:0/2781634801 >> 172.21.15.86:0/2781634801 conn(0x7f35400b3f90 msgr2=0x7f35400b1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:33.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.261+0000 7f3547000640 1 -- 172.21.15.86:0/2781634801 shutdown_connections 2024-04-20T21:26:33.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.261+0000 7f3547000640 1 -- 172.21.15.86:0/2781634801 wait complete. 2024-04-20T21:26:33.280 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:26:33.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.477+0000 7efe5b000640 1 Processor -- start 2024-04-20T21:26:33.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.477+0000 7efe5b000640 1 -- start start 2024-04-20T21:26:33.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.479+0000 7efe5b000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe54155d30 0x7efe540cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:33.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.479+0000 7efe5b000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7efe5406b980 0x7efe5406be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:33.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.479+0000 7efe5b000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe54156110 con 0x7efe540b4ba0 2024-04-20T21:26:33.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.479+0000 7efe5b000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7efe540b56e0 con 0x7efe5406b980 2024-04-20T21:26:33.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.479+0000 7efe5b000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7efe540ce460 con 0x7efe54155d30 2024-04-20T21:26:33.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.479+0000 7efe59000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7efe540b4ba0 0x7efe540b4fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34758/0 (socket says 172.21.15.86:34758) 2024-04-20T21:26:33.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.479+0000 7efe59000640 1 -- 172.21.15.86:0/3396646062 learned_addr learned my addr 172.21.15.86:0/3396646062 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:26:33.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.480+0000 7efe53e00640 1 --2- 172.21.15.86:0/3396646062 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe54155d30 0x7efe540cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:33.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.480+0000 7efe4b400640 1 --2- 172.21.15.86:0/3396646062 >> v2:172.21.15.144:3300/0 conn(0x7efe5406b980 0x7efe5406be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:33.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.480+0000 7efe53400640 1 -- 172.21.15.86:0/3396646062 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1952312675 0 0) 0x7efe54156110 con 0x7efe540b4ba0 2024-04-20T21:26:33.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.480+0000 7efe53400640 1 -- 172.21.15.86:0/3396646062 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe38003520 con 0x7efe540b4ba0 2024-04-20T21:26:33.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.480+0000 7efe53400640 1 -- 172.21.15.86:0/3396646062 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2642666475 0 0) 0x7efe38003520 con 0x7efe540b4ba0 2024-04-20T21:26:33.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.480+0000 7efe53400640 1 -- 172.21.15.86:0/3396646062 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe54155d30 msgr2=0x7efe540cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:33.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.480+0000 7efe53400640 1 --2- 172.21.15.86:0/3396646062 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe54155d30 0x7efe540cf500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:33.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.480+0000 7efe53400640 1 -- 172.21.15.86:0/3396646062 >> v2:172.21.15.144:3300/0 conn(0x7efe5406b980 msgr2=0x7efe5406be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:33.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.480+0000 7efe53400640 1 --2- 172.21.15.86:0/3396646062 >> v2:172.21.15.144:3300/0 conn(0x7efe5406b980 0x7efe5406be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:33.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.480+0000 7efe53400640 1 -- 172.21.15.86:0/3396646062 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe540ce740 con 0x7efe540b4ba0 2024-04-20T21:26:33.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.480+0000 7efe53400640 1 -- 172.21.15.86:0/3396646062 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7efe4c001670 con 0x7efe540b4ba0 2024-04-20T21:26:33.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.480+0000 7efe53400640 1 -- 172.21.15.86:0/3396646062 >> v1:172.21.15.86:6789/0 conn(0x7efe540b4ba0 legacy=0x7efe540b4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:33.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.480+0000 7efe53400640 1 --2- 172.21.15.86:0/3396646062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe380039b0 0x7efe38003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:33.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.480+0000 7efe53400640 1 --2- 172.21.15.86:0/3396646062 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe54155d30 0x7efe38004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:33.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.480+0000 7efe53400640 1 --2- 172.21.15.86:0/3396646062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe38004d40 0x7efe380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:33.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.480+0000 7efe53400640 1 -- 172.21.15.86:0/3396646062 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efe540b56e0 con 0x7efe380039b0 2024-04-20T21:26:33.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.480+0000 7efe53400640 1 -- 172.21.15.86:0/3396646062 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efe540ce460 con 0x7efe38004d40 2024-04-20T21:26:33.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.480+0000 7efe53400640 1 -- 172.21.15.86:0/3396646062 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7efe3800baf0 con 0x7efe54155d30 2024-04-20T21:26:33.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.481+0000 7efe53e00640 1 --2- 172.21.15.86:0/3396646062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe380039b0 0x7efe38003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:33.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.481+0000 7efe4b400640 1 --2- 172.21.15.86:0/3396646062 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe54155d30 0x7efe38004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:33.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.481+0000 7efe59000640 1 --2- 172.21.15.86:0/3396646062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe38004d40 0x7efe380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:33.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.481+0000 7efe53e00640 1 -- 172.21.15.86:0/3396646062 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe54155d30 msgr2=0x7efe38004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:33.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.481+0000 7efe53e00640 1 --2- 172.21.15.86:0/3396646062 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe54155d30 0x7efe38004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:33.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.481+0000 7efe53e00640 1 -- 172.21.15.86:0/3396646062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe38004d40 msgr2=0x7efe380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:33.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.481+0000 7efe53e00640 1 --2- 172.21.15.86:0/3396646062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe38004d40 0x7efe380051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:33.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.481+0000 7efe53e00640 1 -- 172.21.15.86:0/3396646062 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe38003520 con 0x7efe380039b0 2024-04-20T21:26:33.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.482+0000 7efe53e00640 1 --2- 172.21.15.86:0/3396646062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe380039b0 0x7efe38003dd0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7efe44007360 tx=0x7efe38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed85d4b424c1fcf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:33.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.482+0000 7efe53400640 1 -- 172.21.15.86:0/3396646062 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe44039070 con 0x7efe380039b0 2024-04-20T21:26:33.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.482+0000 7efe53400640 1 -- 172.21.15.86:0/3396646062 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe4402ccd0 con 0x7efe380039b0 2024-04-20T21:26:33.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.482+0000 7efe53400640 1 -- 172.21.15.86:0/3396646062 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe44009be0 con 0x7efe380039b0 2024-04-20T21:26:33.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.483+0000 7efe5b000640 1 -- 172.21.15.86:0/3396646062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe380039b0 msgr2=0x7efe38003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:33.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.483+0000 7efe5b000640 1 --2- 172.21.15.86:0/3396646062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe380039b0 0x7efe38003dd0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7efe44007360 tx=0x7efe38005830 comp rx=0 tx=0).stop 2024-04-20T21:26:33.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.483+0000 7efe59000640 1 -- 172.21.15.86:0/3396646062 reap_dead start 2024-04-20T21:26:33.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.483+0000 7efe5b000640 1 -- 172.21.15.86:0/3396646062 shutdown_connections 2024-04-20T21:26:33.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.483+0000 7efe5b000640 1 -- 172.21.15.86:0/3396646062 >> 172.21.15.86:0/3396646062 conn(0x7efe540b2110 msgr2=0x7efe540b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:33.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.483+0000 7efe5b000640 1 -- 172.21.15.86:0/3396646062 shutdown_connections 2024-04-20T21:26:33.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.483+0000 7efe5b000640 1 -- 172.21.15.86:0/3396646062 wait complete. 2024-04-20T21:26:33.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.485+0000 7efe5b000640 1 Processor -- start 2024-04-20T21:26:33.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.485+0000 7efe5b000640 1 -- start start 2024-04-20T21:26:33.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.486+0000 7efe5b000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe380039b0 0x7efe54142ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:33.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.486+0000 7efe5b000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe540cbd10 0x7efe5415a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:33.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.486+0000 7efe5b000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe541f5820 0x7efe541f5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:33.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.486+0000 7efe5b000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efe540cefe0 con 0x7efe541f5820 2024-04-20T21:26:33.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.486+0000 7efe5b000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efe540ced00 con 0x7efe540cbd10 2024-04-20T21:26:33.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.486+0000 7efe5b000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7efe540cf500 con 0x7efe380039b0 2024-04-20T21:26:33.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.486+0000 7efe59000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe541f5820 0x7efe541f5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:33.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.486+0000 7efe59000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe541f5820 0x7efe541f5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43856/0 (socket says 172.21.15.86:43856) 2024-04-20T21:26:33.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.486+0000 7efe59000640 1 -- 172.21.15.86:0/2131886357 learned_addr learned my addr 172.21.15.86:0/2131886357 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:33.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.487+0000 7efe4b400640 1 --2- 172.21.15.86:0/2131886357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe540cbd10 0x7efe5415a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:33.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.487+0000 7efe53e00640 1 --2- 172.21.15.86:0/2131886357 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe380039b0 0x7efe54142ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:33.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.487+0000 7efe59000640 1 -- 172.21.15.86:0/2131886357 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe380039b0 msgr2=0x7efe54142ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:33.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.487+0000 7efe59000640 1 --2- 172.21.15.86:0/2131886357 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe380039b0 0x7efe54142ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:33.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.487+0000 7efe59000640 1 -- 172.21.15.86:0/2131886357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe540cbd10 msgr2=0x7efe5415a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:33.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.487+0000 7efe59000640 1 --2- 172.21.15.86:0/2131886357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe540cbd10 0x7efe5415a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:33.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.487+0000 7efe59000640 1 -- 172.21.15.86:0/2131886357 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe5415b120 con 0x7efe541f5820 2024-04-20T21:26:33.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.487+0000 7efe53e00640 1 --2- 172.21.15.86:0/2131886357 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe380039b0 0x7efe54142ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:26:33.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.487+0000 7efe59000640 1 --2- 172.21.15.86:0/2131886357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe541f5820 0x7efe541f5ca0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7efe4c001120 tx=0x7efe4c00d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:33.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.487+0000 7efe50c00640 1 -- 172.21.15.86:0/2131886357 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c007440 con 0x7efe541f5820 2024-04-20T21:26:33.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.487+0000 7efe50c00640 1 -- 172.21.15.86:0/2131886357 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe4c007600 con 0x7efe541f5820 2024-04-20T21:26:33.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.487+0000 7efe50c00640 1 -- 172.21.15.86:0/2131886357 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c004a20 con 0x7efe541f5820 2024-04-20T21:26:33.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.487+0000 7efe5b000640 1 -- 172.21.15.86:0/2131886357 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe541575c0 con 0x7efe541f5820 2024-04-20T21:26:33.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.488+0000 7efe5b000640 1 -- 172.21.15.86:0/2131886357 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe540cc190 con 0x7efe541f5820 2024-04-20T21:26:33.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.489+0000 7efe5b000640 1 -- 172.21.15.86:0/2131886357 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe540689e0 con 0x7efe541f5820 2024-04-20T21:26:33.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.490+0000 7efe50c00640 1 -- 172.21.15.86:0/2131886357 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efe4c01b060 con 0x7efe541f5820 2024-04-20T21:26:33.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.491+0000 7efe50c00640 1 --2- 172.21.15.86:0/2131886357 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efe3006c8f0 0x7efe3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:33.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.491+0000 7efe50c00640 1 -- 172.21.15.86:0/2131886357 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7efe4c08b4a0 con 0x7efe541f5820 2024-04-20T21:26:33.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.491+0000 7efe53e00640 1 --2- 172.21.15.86:0/2131886357 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efe3006c8f0 0x7efe3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:33.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.492+0000 7efe53e00640 1 --2- 172.21.15.86:0/2131886357 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efe3006c8f0 0x7efe3006edb0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7efe44036000 tx=0x7efe44030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:33.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.495+0000 7efe50c00640 1 -- 172.21.15.86:0/2131886357 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7efe4c0598a0 con 0x7efe541f5820 2024-04-20T21:26:33.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.694+0000 7efe5b000640 1 -- 172.21.15.86:0/2131886357 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7efe540b3730 con 0x7efe541f5820 2024-04-20T21:26:33.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.694+0000 7efe50c00640 1 -- 172.21.15.86:0/2131886357 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7efe4c0591a0 con 0x7efe541f5820 2024-04-20T21:26:33.694 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-20T21:26:33.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.697+0000 7efe5b000640 1 -- 172.21.15.86:0/2131886357 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efe3006c8f0 msgr2=0x7efe3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:33.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.697+0000 7efe5b000640 1 --2- 172.21.15.86:0/2131886357 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efe3006c8f0 0x7efe3006edb0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7efe44036000 tx=0x7efe44030000 comp rx=0 tx=0).stop 2024-04-20T21:26:33.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.697+0000 7efe5b000640 1 -- 172.21.15.86:0/2131886357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe541f5820 msgr2=0x7efe541f5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:33.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.697+0000 7efe5b000640 1 --2- 172.21.15.86:0/2131886357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe541f5820 0x7efe541f5ca0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7efe4c001120 tx=0x7efe4c00d9d0 comp rx=0 tx=0).stop 2024-04-20T21:26:33.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.697+0000 7efe5b000640 1 -- 172.21.15.86:0/2131886357 shutdown_connections 2024-04-20T21:26:33.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.697+0000 7efe5b000640 1 --2- 172.21.15.86:0/2131886357 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efe3006c8f0 0x7efe3006edb0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:33.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.697+0000 7efe5b000640 1 --2- 172.21.15.86:0/2131886357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe541f5820 0x7efe541f5ca0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:33.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.697+0000 7efe5b000640 1 --2- 172.21.15.86:0/2131886357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe540cbd10 0x7efe5415a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:33.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.697+0000 7efe5b000640 1 --2- 172.21.15.86:0/2131886357 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efe380039b0 0x7efe54142ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:33.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.697+0000 7efe5b000640 1 -- 172.21.15.86:0/2131886357 >> 172.21.15.86:0/2131886357 conn(0x7efe540b2110 msgr2=0x7efe540b3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:33.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.697+0000 7efe5b000640 1 -- 172.21.15.86:0/2131886357 shutdown_connections 2024-04-20T21:26:33.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.697+0000 7efe5b000640 1 -- 172.21.15.86:0/2131886357 wait complete. 2024-04-20T21:26:33.714 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T21:26:33.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.911+0000 7fc053000640 1 Processor -- start 2024-04-20T21:26:33.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.911+0000 7fc053000640 1 -- start start 2024-04-20T21:26:33.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.913+0000 7fc053000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc04c145fe0 0x7fc04c0d02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:33.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.913+0000 7fc053000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc04c0cd7e0 0x7fc04c071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:33.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.913+0000 7fc053000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc04c15dcc0 con 0x7fc04c0d0810 2024-04-20T21:26:33.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.913+0000 7fc053000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fc04c0d00d0 con 0x7fc04c0cd7e0 2024-04-20T21:26:33.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.913+0000 7fc053000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc04c0d0c70 con 0x7fc04c145fe0 2024-04-20T21:26:33.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.914+0000 7fc043400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fc04c0d0810 0x7fc04c0cd0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34766/0 (socket says 172.21.15.86:34766) 2024-04-20T21:26:33.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.914+0000 7fc043400640 1 -- 172.21.15.86:0/1323472504 learned_addr learned my addr 172.21.15.86:0/1323472504 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:26:33.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.914+0000 7fc04be00640 1 --2- 172.21.15.86:0/1323472504 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc04c145fe0 0x7fc04c0d02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:33.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.914+0000 7fc051000640 1 --2- 172.21.15.86:0/1323472504 >> v2:172.21.15.144:3300/0 conn(0x7fc04c0cd7e0 0x7fc04c071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:33.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.914+0000 7fc04b400640 1 -- 172.21.15.86:0/1323472504 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4101748607 0 0) 0x7fc04c15dcc0 con 0x7fc04c0d0810 2024-04-20T21:26:33.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.914+0000 7fc04b400640 1 -- 172.21.15.86:0/1323472504 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc030003520 con 0x7fc04c0d0810 2024-04-20T21:26:33.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.915+0000 7fc04b400640 1 -- 172.21.15.86:0/1323472504 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3675392294 0 0) 0x7fc030003520 con 0x7fc04c0d0810 2024-04-20T21:26:33.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.915+0000 7fc04b400640 1 -- 172.21.15.86:0/1323472504 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc04c145fe0 msgr2=0x7fc04c0d02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:33.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.915+0000 7fc04b400640 1 --2- 172.21.15.86:0/1323472504 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc04c145fe0 0x7fc04c0d02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:33.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.915+0000 7fc04b400640 1 -- 172.21.15.86:0/1323472504 >> v2:172.21.15.144:3300/0 conn(0x7fc04c0cd7e0 msgr2=0x7fc04c071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:33.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.915+0000 7fc04b400640 1 --2- 172.21.15.86:0/1323472504 >> v2:172.21.15.144:3300/0 conn(0x7fc04c0cd7e0 0x7fc04c071cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:33.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.915+0000 7fc04b400640 1 -- 172.21.15.86:0/1323472504 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc04c15cf00 con 0x7fc04c0d0810 2024-04-20T21:26:33.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.915+0000 7fc04b400640 1 -- 172.21.15.86:0/1323472504 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fc034001670 con 0x7fc04c0d0810 2024-04-20T21:26:33.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.915+0000 7fc04b400640 1 -- 172.21.15.86:0/1323472504 >> v1:172.21.15.86:6789/0 conn(0x7fc04c0d0810 legacy=0x7fc04c0cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:33.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.915+0000 7fc04b400640 1 --2- 172.21.15.86:0/1323472504 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc04c145fe0 0x7fc0300039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:33.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.915+0000 7fc04b400640 1 --2- 172.21.15.86:0/1323472504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc030003f30 0x7fc030004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:33.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.915+0000 7fc04b400640 1 --2- 172.21.15.86:0/1323472504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc030004d40 0x7fc0300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:33.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.915+0000 7fc04b400640 1 -- 172.21.15.86:0/1323472504 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc04c0d00d0 con 0x7fc030003f30 2024-04-20T21:26:33.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.915+0000 7fc04b400640 1 -- 172.21.15.86:0/1323472504 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc03000b9c0 con 0x7fc030004d40 2024-04-20T21:26:33.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.915+0000 7fc04b400640 1 -- 172.21.15.86:0/1323472504 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc03000bb40 con 0x7fc04c145fe0 2024-04-20T21:26:33.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.915+0000 7fc043400640 1 --2- 172.21.15.86:0/1323472504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc030003f30 0x7fc030004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:33.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.916+0000 7fc04be00640 1 --2- 172.21.15.86:0/1323472504 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc04c145fe0 0x7fc0300039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:33.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.916+0000 7fc043400640 1 -- 172.21.15.86:0/1323472504 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc04c145fe0 msgr2=0x7fc0300039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:33.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.916+0000 7fc043400640 1 --2- 172.21.15.86:0/1323472504 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc04c145fe0 0x7fc0300039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:33.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.916+0000 7fc043400640 1 -- 172.21.15.86:0/1323472504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc030004d40 msgr2=0x7fc0300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:33.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.916+0000 7fc043400640 1 --2- 172.21.15.86:0/1323472504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc030004d40 0x7fc0300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:33.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.916+0000 7fc043400640 1 -- 172.21.15.86:0/1323472504 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc030003520 con 0x7fc030003f30 2024-04-20T21:26:33.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.916+0000 7fc04be00640 1 --2- 172.21.15.86:0/1323472504 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc04c145fe0 0x7fc0300039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:26:33.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.917+0000 7fc043400640 1 --2- 172.21.15.86:0/1323472504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc030003f30 0x7fc030004800 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fc03400c240 tx=0x7fc030005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe95c26e39e63b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:33.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.917+0000 7fc04b400640 1 -- 172.21.15.86:0/1323472504 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc034007d30 con 0x7fc030003f30 2024-04-20T21:26:33.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.917+0000 7fc04b400640 1 -- 172.21.15.86:0/1323472504 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc034030070 con 0x7fc030003f30 2024-04-20T21:26:33.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.917+0000 7fc04b400640 1 -- 172.21.15.86:0/1323472504 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03402cd60 con 0x7fc030003f30 2024-04-20T21:26:33.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.917+0000 7fc053000640 1 -- 172.21.15.86:0/1323472504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc030003f30 msgr2=0x7fc030004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:33.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.917+0000 7fc053000640 1 --2- 172.21.15.86:0/1323472504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc030003f30 0x7fc030004800 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fc03400c240 tx=0x7fc030005830 comp rx=0 tx=0).stop 2024-04-20T21:26:33.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.918+0000 7fc051000640 1 -- 172.21.15.86:0/1323472504 reap_dead start 2024-04-20T21:26:33.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.918+0000 7fc053000640 1 -- 172.21.15.86:0/1323472504 shutdown_connections 2024-04-20T21:26:33.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.918+0000 7fc053000640 1 -- 172.21.15.86:0/1323472504 >> 172.21.15.86:0/1323472504 conn(0x7fc04c06b980 msgr2=0x7fc04c06bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:33.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.918+0000 7fc053000640 1 -- 172.21.15.86:0/1323472504 shutdown_connections 2024-04-20T21:26:33.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.918+0000 7fc053000640 1 -- 172.21.15.86:0/1323472504 wait complete. 2024-04-20T21:26:33.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.919+0000 7fc053000640 1 Processor -- start 2024-04-20T21:26:33.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.920+0000 7fc053000640 1 -- start start 2024-04-20T21:26:33.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.920+0000 7fc053000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc030003f30 0x7fc04c142ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:33.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.921+0000 7fc053000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc04c0c6b30 0x7fc04c0c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:33.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.921+0000 7fc053000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc04c0b21c0 0x7fc04c0b2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:33.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.921+0000 7fc053000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc04c15d7a0 con 0x7fc04c0c6b30 2024-04-20T21:26:33.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.921+0000 7fc053000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc04c15d4c0 con 0x7fc030003f30 2024-04-20T21:26:33.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.921+0000 7fc053000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc04c1463c0 con 0x7fc04c0b21c0 2024-04-20T21:26:33.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.921+0000 7fc043400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc04c0c6b30 0x7fc04c0c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:33.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.921+0000 7fc043400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc04c0c6b30 0x7fc04c0c6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43872/0 (socket says 172.21.15.86:43872) 2024-04-20T21:26:33.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.921+0000 7fc043400640 1 -- 172.21.15.86:0/478272301 learned_addr learned my addr 172.21.15.86:0/478272301 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:33.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.921+0000 7fc04be00640 1 --2- 172.21.15.86:0/478272301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc030003f30 0x7fc04c142ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:33.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.921+0000 7fc051000640 1 --2- 172.21.15.86:0/478272301 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc04c0b21c0 0x7fc04c0b2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:33.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.921+0000 7fc043400640 1 -- 172.21.15.86:0/478272301 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc04c0b21c0 msgr2=0x7fc04c0b2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:33.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.921+0000 7fc043400640 1 --2- 172.21.15.86:0/478272301 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc04c0b21c0 0x7fc04c0b2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:33.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.921+0000 7fc043400640 1 -- 172.21.15.86:0/478272301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc030003f30 msgr2=0x7fc04c142ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:33.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.921+0000 7fc043400640 1 --2- 172.21.15.86:0/478272301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc030003f30 0x7fc04c142ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:33.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.921+0000 7fc043400640 1 -- 172.21.15.86:0/478272301 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc04c0b2f10 con 0x7fc04c0c6b30 2024-04-20T21:26:33.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.921+0000 7fc051000640 1 --2- 172.21.15.86:0/478272301 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc04c0b21c0 0x7fc04c0b2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:26:33.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.921+0000 7fc04be00640 1 --2- 172.21.15.86:0/478272301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc030003f30 0x7fc04c142ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:26:33.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.922+0000 7fc043400640 1 --2- 172.21.15.86:0/478272301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc04c0c6b30 0x7fc04c0c6fb0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fc03400c360 tx=0x7fc04c15d0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:33.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.922+0000 7fc048c00640 1 -- 172.21.15.86:0/478272301 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc034001c70 con 0x7fc04c0c6b30 2024-04-20T21:26:33.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.922+0000 7fc048c00640 1 -- 172.21.15.86:0/478272301 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc034004bf0 con 0x7fc04c0c6b30 2024-04-20T21:26:33.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.922+0000 7fc048c00640 1 -- 172.21.15.86:0/478272301 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc034007070 con 0x7fc04c0c6b30 2024-04-20T21:26:33.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.922+0000 7fc053000640 1 -- 172.21.15.86:0/478272301 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc04c0b31f0 con 0x7fc04c0c6b30 2024-04-20T21:26:33.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.922+0000 7fc053000640 1 -- 172.21.15.86:0/478272301 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc04c0b37b0 con 0x7fc04c0c6b30 2024-04-20T21:26:33.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.923+0000 7fc053000640 1 -- 172.21.15.86:0/478272301 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc04c0689e0 con 0x7fc04c0c6b30 2024-04-20T21:26:33.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.924+0000 7fc048c00640 1 -- 172.21.15.86:0/478272301 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc034007210 con 0x7fc04c0c6b30 2024-04-20T21:26:33.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.925+0000 7fc048c00640 1 --2- 172.21.15.86:0/478272301 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc02806c8f0 0x7fc02806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:33.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.925+0000 7fc048c00640 1 -- 172.21.15.86:0/478272301 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7fc0340acfe0 con 0x7fc04c0c6b30 2024-04-20T21:26:33.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.925+0000 7fc04be00640 1 --2- 172.21.15.86:0/478272301 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc02806c8f0 0x7fc02806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:33.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.926+0000 7fc04be00640 1 --2- 172.21.15.86:0/478272301 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc02806c8f0 0x7fc02806edb0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fc03c007440 tx=0x7fc03c00d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:33.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:33.929+0000 7fc048c00640 1 -- 172.21.15.86:0/478272301 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc03407b410 con 0x7fc04c0c6b30 2024-04-20T21:26:34.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.129+0000 7fc053000640 1 -- 172.21.15.86:0/478272301 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc04c0d1020 con 0x7fc04c0c6b30 2024-04-20T21:26:34.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.130+0000 7fc048c00640 1 -- 172.21.15.86:0/478272301 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v88) ==== 80+0+124 (secure 0 0 0) 0x7fc03407ad10 con 0x7fc04c0c6b30 2024-04-20T21:26:34.130 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.144:6835/1740492227 2024-04-21T21:25:20.040733+0000 2024-04-20T21:26:34.130 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.144:6833/1740492227 2024-04-21T21:25:20.040733+0000 2024-04-20T21:26:34.130 INFO:teuthology.orchestra.run.smithi086.stderr:listed 2 entries 2024-04-20T21:26:34.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.132+0000 7fc053000640 1 -- 172.21.15.86:0/478272301 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc02806c8f0 msgr2=0x7fc02806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:34.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.132+0000 7fc053000640 1 --2- 172.21.15.86:0/478272301 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc02806c8f0 0x7fc02806edb0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fc03c007440 tx=0x7fc03c00d040 comp rx=0 tx=0).stop 2024-04-20T21:26:34.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.132+0000 7fc053000640 1 -- 172.21.15.86:0/478272301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc04c0c6b30 msgr2=0x7fc04c0c6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:34.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.132+0000 7fc053000640 1 --2- 172.21.15.86:0/478272301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc04c0c6b30 0x7fc04c0c6fb0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fc03400c360 tx=0x7fc04c15d0d0 comp rx=0 tx=0).stop 2024-04-20T21:26:34.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.132+0000 7fc053000640 1 -- 172.21.15.86:0/478272301 shutdown_connections 2024-04-20T21:26:34.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.132+0000 7fc053000640 1 --2- 172.21.15.86:0/478272301 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc02806c8f0 0x7fc02806edb0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:34.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.132+0000 7fc053000640 1 --2- 172.21.15.86:0/478272301 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc04c0b21c0 0x7fc04c0b2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:34.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.132+0000 7fc053000640 1 --2- 172.21.15.86:0/478272301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc04c0c6b30 0x7fc04c0c6fb0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:34.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.132+0000 7fc053000640 1 --2- 172.21.15.86:0/478272301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc030003f30 0x7fc04c142ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:34.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.132+0000 7fc053000640 1 -- 172.21.15.86:0/478272301 >> 172.21.15.86:0/478272301 conn(0x7fc04c06b980 msgr2=0x7fc04c15eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:34.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.132+0000 7fc053000640 1 -- 172.21.15.86:0/478272301 shutdown_connections 2024-04-20T21:26:34.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.132+0000 7fc053000640 1 -- 172.21.15.86:0/478272301 wait complete. 2024-04-20T21:26:34.151 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:26:34.151 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:26:34.200 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:26:34.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.441+0000 7f7490a00640 1 Processor -- start 2024-04-20T21:26:34.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.441+0000 7f7490a00640 1 -- start start 2024-04-20T21:26:34.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.443+0000 7f7490a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f748c0d5150 0x7f748c0d28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:34.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.443+0000 7f7490a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f748c0cf290 con 0x7f748c0d0d20 2024-04-20T21:26:34.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.443+0000 7f7490a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f748c0cf570 con 0x7f748c0d4cf0 2024-04-20T21:26:34.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.443+0000 7f7490a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f748c0cfea0 con 0x7f748c0d5150 2024-04-20T21:26:34.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.443+0000 7f748a800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f748c0d0d20 0x7f748c0d1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34782/0 (socket says 172.21.15.86:34782) 2024-04-20T21:26:34.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.443+0000 7f748a800640 1 -- 172.21.15.86:0/3475264808 learned_addr learned my addr 172.21.15.86:0/3475264808 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:26:34.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.443+0000 7f7489e00640 1 -- 172.21.15.86:0/3475264808 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2038858781 0 0) 0x7f748c0cf290 con 0x7f748c0d0d20 2024-04-20T21:26:34.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.444+0000 7f7489e00640 1 -- 172.21.15.86:0/3475264808 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7470003520 con 0x7f748c0d0d20 2024-04-20T21:26:34.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.443+0000 7f748b200640 1 --2- 172.21.15.86:0/3475264808 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f748c0d5150 0x7f748c0d28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:34.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.444+0000 7f7489e00640 1 -- 172.21.15.86:0/3475264808 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2117632237 0 0) 0x7f7470003520 con 0x7f748c0d0d20 2024-04-20T21:26:34.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.444+0000 7f7489e00640 1 -- 172.21.15.86:0/3475264808 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f748c0d5150 msgr2=0x7f748c0d28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:34.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.444+0000 7f7489e00640 1 --2- 172.21.15.86:0/3475264808 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f748c0d5150 0x7f748c0d28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:34.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.444+0000 7f7489e00640 1 -- 172.21.15.86:0/3475264808 >> v1:172.21.15.144:6789/0 conn(0x7f748c0d4cf0 legacy=0x7f748c0d21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:34.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.444+0000 7f7489e00640 1 -- 172.21.15.86:0/3475264808 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f748c0cf850 con 0x7f748c0d0d20 2024-04-20T21:26:34.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.444+0000 7f7489e00640 1 -- 172.21.15.86:0/3475264808 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f747c001670 con 0x7f748c0d0d20 2024-04-20T21:26:34.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.444+0000 7f7489e00640 1 -- 172.21.15.86:0/3475264808 >> v1:172.21.15.86:6789/0 conn(0x7f748c0d0d20 legacy=0x7f748c0d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:34.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.444+0000 7f7489e00640 1 --2- 172.21.15.86:0/3475264808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74700039e0 0x7f7470003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:34.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.444+0000 7f7489e00640 1 --2- 172.21.15.86:0/3475264808 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f748c0d5150 0x7f7470004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:34.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.444+0000 7f7489e00640 1 --2- 172.21.15.86:0/3475264808 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7470004d40 0x7f74700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:34.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.444+0000 7f7489e00640 1 -- 172.21.15.86:0/3475264808 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f748c0cfea0 con 0x7f74700039e0 2024-04-20T21:26:34.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.444+0000 7f7489e00640 1 -- 172.21.15.86:0/3475264808 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f747000bbd0 con 0x7f7470004d40 2024-04-20T21:26:34.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.444+0000 7f7489e00640 1 -- 172.21.15.86:0/3475264808 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f747000bdd0 con 0x7f748c0d5150 2024-04-20T21:26:34.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.444+0000 7f748a800640 1 --2- 172.21.15.86:0/3475264808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74700039e0 0x7f7470003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:34.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.445+0000 7f7481e00640 1 --2- 172.21.15.86:0/3475264808 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f748c0d5150 0x7f7470004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:34.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.445+0000 7f748a800640 1 -- 172.21.15.86:0/3475264808 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f748c0d5150 msgr2=0x7f7470004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:34.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.445+0000 7f748b200640 1 --2- 172.21.15.86:0/3475264808 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7470004d40 0x7f74700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:34.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.445+0000 7f748a800640 1 --2- 172.21.15.86:0/3475264808 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f748c0d5150 0x7f7470004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:34.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.445+0000 7f748a800640 1 -- 172.21.15.86:0/3475264808 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7470004d40 msgr2=0x7f74700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:34.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.445+0000 7f748a800640 1 --2- 172.21.15.86:0/3475264808 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7470004d40 0x7f74700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:34.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.445+0000 7f748a800640 1 -- 172.21.15.86:0/3475264808 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7470003520 con 0x7f74700039e0 2024-04-20T21:26:34.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.445+0000 7f748b200640 1 --2- 172.21.15.86:0/3475264808 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7470004d40 0x7f74700051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:26:34.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.446+0000 7f748a800640 1 --2- 172.21.15.86:0/3475264808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74700039e0 0x7f7470003e00 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f747c00c190 tx=0x7f7470005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bf5b58cf0ee5226 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:34.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.446+0000 7f7489e00640 1 -- 172.21.15.86:0/3475264808 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747c007c60 con 0x7f74700039e0 2024-04-20T21:26:34.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.446+0000 7f7489e00640 1 -- 172.21.15.86:0/3475264808 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f747c030070 con 0x7f74700039e0 2024-04-20T21:26:34.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.446+0000 7f7489e00640 1 -- 172.21.15.86:0/3475264808 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747c005d70 con 0x7f74700039e0 2024-04-20T21:26:34.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.447+0000 7f7490a00640 1 -- 172.21.15.86:0/3475264808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74700039e0 msgr2=0x7f7470003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:34.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.447+0000 7f7490a00640 1 --2- 172.21.15.86:0/3475264808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74700039e0 0x7f7470003e00 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f747c00c190 tx=0x7f7470005830 comp rx=0 tx=0).stop 2024-04-20T21:26:34.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.447+0000 7f748b200640 1 -- 172.21.15.86:0/3475264808 reap_dead start 2024-04-20T21:26:34.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.447+0000 7f7490a00640 1 -- 172.21.15.86:0/3475264808 shutdown_connections 2024-04-20T21:26:34.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.447+0000 7f7490a00640 1 -- 172.21.15.86:0/3475264808 >> 172.21.15.86:0/3475264808 conn(0x7f748c08bf70 msgr2=0x7f748c0d45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:34.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.447+0000 7f7490a00640 1 -- 172.21.15.86:0/3475264808 shutdown_connections 2024-04-20T21:26:34.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.447+0000 7f7490a00640 1 -- 172.21.15.86:0/3475264808 wait complete. 2024-04-20T21:26:34.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.449+0000 7f7490a00640 1 Processor -- start 2024-04-20T21:26:34.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.449+0000 7f7490a00640 1 -- start start 2024-04-20T21:26:34.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.450+0000 7f7490a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74700039e0 0x7f748c154ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:34.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.450+0000 7f7490a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f748c155ef0 0x7f748c0b6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:34.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.450+0000 7f7490a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f748c1553c0 0x7f748c0b6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:34.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.450+0000 7f7490a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f748c0d0330 con 0x7f74700039e0 2024-04-20T21:26:34.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.450+0000 7f7490a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f748c15fc30 con 0x7f748c1553c0 2024-04-20T21:26:34.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.450+0000 7f7490a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f748c1551e0 con 0x7f748c155ef0 2024-04-20T21:26:34.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.450+0000 7f748a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74700039e0 0x7f748c154ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:34.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.450+0000 7f748a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74700039e0 0x7f748c154ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43892/0 (socket says 172.21.15.86:43892) 2024-04-20T21:26:34.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.450+0000 7f748a800640 1 -- 172.21.15.86:0/1348234172 learned_addr learned my addr 172.21.15.86:0/1348234172 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:34.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.450+0000 7f748a800640 1 -- 172.21.15.86:0/1348234172 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f748c155ef0 msgr2=0x7f748c0b6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:34.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.450+0000 7f748a800640 1 --2- 172.21.15.86:0/1348234172 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f748c155ef0 0x7f748c0b6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:34.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.450+0000 7f748a800640 1 -- 172.21.15.86:0/1348234172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f748c1553c0 msgr2=0x7f748c0b6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:34.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.450+0000 7f748a800640 1 --2- 172.21.15.86:0/1348234172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f748c1553c0 0x7f748c0b6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:34.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.450+0000 7f748a800640 1 -- 172.21.15.86:0/1348234172 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f748c15d320 con 0x7f74700039e0 2024-04-20T21:26:34.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.451+0000 7f748a800640 1 --2- 172.21.15.86:0/1348234172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74700039e0 0x7f748c154ca0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f747c02c4c0 tx=0x7f747c001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:34.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.451+0000 7f7483400640 1 -- 172.21.15.86:0/1348234172 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747c004bf0 con 0x7f74700039e0 2024-04-20T21:26:34.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.451+0000 7f7483400640 1 -- 172.21.15.86:0/1348234172 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f747c000ed0 con 0x7f74700039e0 2024-04-20T21:26:34.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.451+0000 7f7490a00640 1 -- 172.21.15.86:0/1348234172 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f748c0b7d40 con 0x7f74700039e0 2024-04-20T21:26:34.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.451+0000 7f7483400640 1 -- 172.21.15.86:0/1348234172 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747c007070 con 0x7f74700039e0 2024-04-20T21:26:34.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.451+0000 7f7490a00640 1 -- 172.21.15.86:0/1348234172 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f748c0b8300 con 0x7f74700039e0 2024-04-20T21:26:34.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.452+0000 7f7490a00640 1 -- 172.21.15.86:0/1348234172 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f748c04fcf0 con 0x7f74700039e0 2024-04-20T21:26:34.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.453+0000 7f7483400640 1 -- 172.21.15.86:0/1348234172 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f747c048020 con 0x7f74700039e0 2024-04-20T21:26:34.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.454+0000 7f7483400640 1 --2- 172.21.15.86:0/1348234172 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f746806c920 0x7f746806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:34.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.454+0000 7f7483400640 1 -- 172.21.15.86:0/1348234172 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7f747c0acf70 con 0x7f74700039e0 2024-04-20T21:26:34.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.454+0000 7f7481e00640 1 --2- 172.21.15.86:0/1348234172 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f746806c920 0x7f746806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:34.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.455+0000 7f7481e00640 1 --2- 172.21.15.86:0/1348234172 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f746806c920 0x7f746806ede0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f7474007fd0 tx=0x7f7474009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:34.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.459+0000 7f7483400640 1 -- 172.21.15.86:0/1348234172 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f747c007d00 con 0x7f74700039e0 2024-04-20T21:26:34.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.642+0000 7f7490a00640 1 -- 172.21.15.86:0/1348234172 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f748c0d04b0 con 0x7f74700039e0 2024-04-20T21:26:34.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.645+0000 7f7483400640 1 -- 172.21.15.86:0/1348234172 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+16841 (secure 0 0 0) 0x7f747c07b340 con 0x7f74700039e0 2024-04-20T21:26:34.645 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:26:34.645 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":88,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:25:35.641040+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:24:55.964540+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-20T21:24:57.241527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.3e","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.144:6835/1740492227":"2024-04-21T21:25:20.040733+0000","172.21.15.144:6833/1740492227":"2024-04-21T21:25:20.040733+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:26:34.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.647+0000 7f7490a00640 1 -- 172.21.15.86:0/1348234172 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f746806c920 msgr2=0x7f746806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:34.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.647+0000 7f7490a00640 1 --2- 172.21.15.86:0/1348234172 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f746806c920 0x7f746806ede0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f7474007fd0 tx=0x7f7474009040 comp rx=0 tx=0).stop 2024-04-20T21:26:34.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.647+0000 7f7490a00640 1 -- 172.21.15.86:0/1348234172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74700039e0 msgr2=0x7f748c154ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:34.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.647+0000 7f7490a00640 1 --2- 172.21.15.86:0/1348234172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74700039e0 0x7f748c154ca0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f747c02c4c0 tx=0x7f747c001970 comp rx=0 tx=0).stop 2024-04-20T21:26:34.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.647+0000 7f7490a00640 1 -- 172.21.15.86:0/1348234172 shutdown_connections 2024-04-20T21:26:34.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.647+0000 7f7490a00640 1 --2- 172.21.15.86:0/1348234172 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f746806c920 0x7f746806ede0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:34.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.647+0000 7f7490a00640 1 --2- 172.21.15.86:0/1348234172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f748c1553c0 0x7f748c0b6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:34.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.647+0000 7f7490a00640 1 --2- 172.21.15.86:0/1348234172 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f748c155ef0 0x7f748c0b6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:34.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.647+0000 7f7490a00640 1 --2- 172.21.15.86:0/1348234172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74700039e0 0x7f748c154ca0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:34.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.647+0000 7f7490a00640 1 -- 172.21.15.86:0/1348234172 >> 172.21.15.86:0/1348234172 conn(0x7f748c08bf70 msgr2=0x7f748c0b1590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:34.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.648+0000 7f7490a00640 1 -- 172.21.15.86:0/1348234172 shutdown_connections 2024-04-20T21:26:34.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.648+0000 7f7490a00640 1 -- 172.21.15.86:0/1348234172 wait complete. 2024-04-20T21:26:34.665 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:24:55.964540+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:24:57.241527+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:26:34.666 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:26:34.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.869+0000 7fd281200640 1 Processor -- start 2024-04-20T21:26:34.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.869+0000 7fd281200640 1 -- start start 2024-04-20T21:26:34.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.870+0000 7fd281200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd27c0c6850 0x7fd27c1678a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:34.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.870+0000 7fd281200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd27c0d11b0 con 0x7fd27c15fd80 2024-04-20T21:26:34.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.871+0000 7fd281200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd27c0cd330 con 0x7fd27c0d0bd0 2024-04-20T21:26:34.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.871+0000 7fd281200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fd27c0cd100 con 0x7fd27c0c6850 2024-04-20T21:26:34.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.871+0000 7fd279e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd27c15fd80 0x7fd27c0cde10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:34792/0 (socket says 172.21.15.86:34792) 2024-04-20T21:26:34.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.871+0000 7fd279e00640 1 -- 172.21.15.86:0/2482175115 learned_addr learned my addr 172.21.15.86:0/2482175115 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:26:34.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.871+0000 7fd278a00640 1 -- 172.21.15.86:0/2482175115 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 92307295 0 0) 0x7fd27c0d11b0 con 0x7fd27c15fd80 2024-04-20T21:26:34.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.871+0000 7fd278a00640 1 -- 172.21.15.86:0/2482175115 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd260003520 con 0x7fd27c15fd80 2024-04-20T21:26:34.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.872+0000 7fd27a800640 1 --2- 172.21.15.86:0/2482175115 >> v2:172.21.15.144:3300/0 conn(0x7fd27c0c6850 0x7fd27c1678a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:34.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.872+0000 7fd278a00640 1 -- 172.21.15.86:0/2482175115 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1123234378 0 0) 0x7fd260003520 con 0x7fd27c15fd80 2024-04-20T21:26:34.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.872+0000 7fd278a00640 1 -- 172.21.15.86:0/2482175115 >> v2:172.21.15.144:3300/0 conn(0x7fd27c0c6850 msgr2=0x7fd27c1678a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:34.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.872+0000 7fd278a00640 1 --2- 172.21.15.86:0/2482175115 >> v2:172.21.15.144:3300/0 conn(0x7fd27c0c6850 0x7fd27c1678a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:34.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.872+0000 7fd278a00640 1 -- 172.21.15.86:0/2482175115 >> v1:172.21.15.144:6789/0 conn(0x7fd27c0d0bd0 legacy=0x7fd27c0c6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:34.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.872+0000 7fd278a00640 1 -- 172.21.15.86:0/2482175115 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd27c0cd610 con 0x7fd27c15fd80 2024-04-20T21:26:34.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.872+0000 7fd278a00640 1 -- 172.21.15.86:0/2482175115 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fd26c001670 con 0x7fd27c15fd80 2024-04-20T21:26:34.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.872+0000 7fd278a00640 1 -- 172.21.15.86:0/2482175115 >> v1:172.21.15.86:6789/0 conn(0x7fd27c15fd80 legacy=0x7fd27c0cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:34.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.872+0000 7fd278a00640 1 --2- 172.21.15.86:0/2482175115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2600039b0 0x7fd260003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:34.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.872+0000 7fd278a00640 1 --2- 172.21.15.86:0/2482175115 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd260004800 0x7fd260004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:34.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.872+0000 7fd278a00640 1 --2- 172.21.15.86:0/2482175115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2600052e0 0x7fd26000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:34.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.872+0000 7fd278a00640 1 -- 172.21.15.86:0/2482175115 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd26000c160 con 0x7fd2600039b0 2024-04-20T21:26:34.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.872+0000 7fd278a00640 1 -- 172.21.15.86:0/2482175115 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd26000c310 con 0x7fd2600052e0 2024-04-20T21:26:34.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.872+0000 7fd278a00640 1 -- 172.21.15.86:0/2482175115 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd26000c510 con 0x7fd260004800 2024-04-20T21:26:34.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.872+0000 7fd279e00640 1 --2- 172.21.15.86:0/2482175115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2600039b0 0x7fd260003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:34.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.873+0000 7fd279400640 1 --2- 172.21.15.86:0/2482175115 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd260004800 0x7fd260004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:34.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.873+0000 7fd27a800640 1 --2- 172.21.15.86:0/2482175115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2600052e0 0x7fd26000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:34.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.873+0000 7fd279e00640 1 -- 172.21.15.86:0/2482175115 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd260004800 msgr2=0x7fd260004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:34.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.873+0000 7fd279e00640 1 --2- 172.21.15.86:0/2482175115 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd260004800 0x7fd260004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:34.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.873+0000 7fd279e00640 1 -- 172.21.15.86:0/2482175115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2600052e0 msgr2=0x7fd26000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:34.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.873+0000 7fd279e00640 1 --2- 172.21.15.86:0/2482175115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2600052e0 0x7fd26000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:34.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.873+0000 7fd279e00640 1 -- 172.21.15.86:0/2482175115 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd260003520 con 0x7fd2600039b0 2024-04-20T21:26:34.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.873+0000 7fd27a800640 1 --2- 172.21.15.86:0/2482175115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2600052e0 0x7fd26000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:26:34.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.873+0000 7fd27a800640 1 -- 172.21.15.86:0/2482175115 reap_dead start 2024-04-20T21:26:34.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.874+0000 7fd279e00640 1 --2- 172.21.15.86:0/2482175115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2600039b0 0x7fd260003db0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fd26c007fd0 tx=0x7fd260005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f513a746664b3b67 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:34.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.875+0000 7fd278a00640 1 -- 172.21.15.86:0/2482175115 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd26c007d50 con 0x7fd2600039b0 2024-04-20T21:26:34.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.875+0000 7fd278a00640 1 -- 172.21.15.86:0/2482175115 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd26c030070 con 0x7fd2600039b0 2024-04-20T21:26:34.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.875+0000 7fd278a00640 1 -- 172.21.15.86:0/2482175115 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd26c02cd40 con 0x7fd2600039b0 2024-04-20T21:26:34.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.875+0000 7fd281200640 1 -- 172.21.15.86:0/2482175115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2600039b0 msgr2=0x7fd260003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:34.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.875+0000 7fd281200640 1 --2- 172.21.15.86:0/2482175115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2600039b0 0x7fd260003db0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fd26c007fd0 tx=0x7fd260005830 comp rx=0 tx=0).stop 2024-04-20T21:26:34.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.876+0000 7fd281200640 1 -- 172.21.15.86:0/2482175115 shutdown_connections 2024-04-20T21:26:34.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.876+0000 7fd281200640 1 --2- 172.21.15.86:0/2482175115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2600039b0 0x7fd260003db0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:34.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.876+0000 7fd281200640 1 -- 172.21.15.86:0/2482175115 >> 172.21.15.86:0/2482175115 conn(0x7fd27c0b40e0 msgr2=0x7fd27c0b44f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:34.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.876+0000 7fd281200640 1 -- 172.21.15.86:0/2482175115 shutdown_connections 2024-04-20T21:26:34.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.876+0000 7fd281200640 1 -- 172.21.15.86:0/2482175115 wait complete. 2024-04-20T21:26:34.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.877+0000 7fd281200640 1 Processor -- start 2024-04-20T21:26:34.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.878+0000 7fd281200640 1 -- start start 2024-04-20T21:26:34.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.878+0000 7fd281200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2600039b0 0x7fd27c0c96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:34.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.878+0000 7fd281200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd27c155590 0x7fd27c154cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:34.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.878+0000 7fd281200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd27c15b640 0x7fd27c06dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:34.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.878+0000 7fd281200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd27c160160 con 0x7fd27c15b640 2024-04-20T21:26:34.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.878+0000 7fd281200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd27c09bb80 con 0x7fd2600039b0 2024-04-20T21:26:34.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.878+0000 7fd281200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd27c1602e0 con 0x7fd27c155590 2024-04-20T21:26:34.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.879+0000 7fd27a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd27c15b640 0x7fd27c06dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:34.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.879+0000 7fd27a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd27c15b640 0x7fd27c06dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43918/0 (socket says 172.21.15.86:43918) 2024-04-20T21:26:34.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.879+0000 7fd27a800640 1 -- 172.21.15.86:0/2009173127 learned_addr learned my addr 172.21.15.86:0/2009173127 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:34.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.879+0000 7fd279400640 1 --2- 172.21.15.86:0/2009173127 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd27c155590 0x7fd27c154cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:34.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.879+0000 7fd279e00640 1 --2- 172.21.15.86:0/2009173127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2600039b0 0x7fd27c0c96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:34.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.879+0000 7fd27a800640 1 -- 172.21.15.86:0/2009173127 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd27c155590 msgr2=0x7fd27c154cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:34.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.879+0000 7fd27a800640 1 --2- 172.21.15.86:0/2009173127 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd27c155590 0x7fd27c154cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:34.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.879+0000 7fd27a800640 1 -- 172.21.15.86:0/2009173127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2600039b0 msgr2=0x7fd27c0c96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:34.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.879+0000 7fd27a800640 1 --2- 172.21.15.86:0/2009173127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2600039b0 0x7fd27c0c96c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:34.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.879+0000 7fd27a800640 1 -- 172.21.15.86:0/2009173127 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd27c0b64f0 con 0x7fd27c15b640 2024-04-20T21:26:34.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.880+0000 7fd27a800640 1 --2- 172.21.15.86:0/2009173127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd27c15b640 0x7fd27c06dd10 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fd27c075920 tx=0x7fd274009d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:34.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.880+0000 7fd272000640 1 -- 172.21.15.86:0/2009173127 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd274019070 con 0x7fd27c15b640 2024-04-20T21:26:34.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.880+0000 7fd272000640 1 -- 172.21.15.86:0/2009173127 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd274007d20 con 0x7fd27c15b640 2024-04-20T21:26:34.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.880+0000 7fd272000640 1 -- 172.21.15.86:0/2009173127 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd274014640 con 0x7fd27c15b640 2024-04-20T21:26:34.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.880+0000 7fd281200640 1 -- 172.21.15.86:0/2009173127 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd27c15bc00 con 0x7fd27c15b640 2024-04-20T21:26:34.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.880+0000 7fd281200640 1 -- 172.21.15.86:0/2009173127 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd27c0b8160 con 0x7fd27c15b640 2024-04-20T21:26:34.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.882+0000 7fd281200640 1 -- 172.21.15.86:0/2009173127 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd27c0690f0 con 0x7fd27c15b640 2024-04-20T21:26:34.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.883+0000 7fd272000640 1 -- 172.21.15.86:0/2009173127 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd27401c070 con 0x7fd27c15b640 2024-04-20T21:26:34.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.884+0000 7fd272000640 1 --2- 172.21.15.86:0/2009173127 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd25c06c8f0 0x7fd25c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:34.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.884+0000 7fd272000640 1 -- 172.21.15.86:0/2009173127 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7fd27408b8b0 con 0x7fd27c15b640 2024-04-20T21:26:34.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.884+0000 7fd279e00640 1 --2- 172.21.15.86:0/2009173127 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd25c06c8f0 0x7fd25c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:34.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.885+0000 7fd279e00640 1 --2- 172.21.15.86:0/2009173127 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd25c06c8f0 0x7fd25c06edb0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd26c034000 tx=0x7fd26c039040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:34.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:34.888+0000 7fd272000640 1 -- 172.21.15.86:0/2009173127 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd274059b60 con 0x7fd27c15b640 2024-04-20T21:26:35.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.091+0000 7fd281200640 1 -- 172.21.15.86:0/2009173127 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd27c0b66c0 con 0x7fd27c15b640 2024-04-20T21:26:35.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.091+0000 7fd272000640 1 -- 172.21.15.86:0/2009173127 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7fd274059460 con 0x7fd27c15b640 2024-04-20T21:26:35.091 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:26:35.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.094+0000 7fd281200640 1 -- 172.21.15.86:0/2009173127 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd25c06c8f0 msgr2=0x7fd25c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:35.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.094+0000 7fd281200640 1 --2- 172.21.15.86:0/2009173127 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd25c06c8f0 0x7fd25c06edb0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd26c034000 tx=0x7fd26c039040 comp rx=0 tx=0).stop 2024-04-20T21:26:35.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.094+0000 7fd281200640 1 -- 172.21.15.86:0/2009173127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd27c15b640 msgr2=0x7fd27c06dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:35.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.094+0000 7fd281200640 1 --2- 172.21.15.86:0/2009173127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd27c15b640 0x7fd27c06dd10 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fd27c075920 tx=0x7fd274009d30 comp rx=0 tx=0).stop 2024-04-20T21:26:35.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.094+0000 7fd281200640 1 -- 172.21.15.86:0/2009173127 shutdown_connections 2024-04-20T21:26:35.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.094+0000 7fd281200640 1 --2- 172.21.15.86:0/2009173127 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd25c06c8f0 0x7fd25c06edb0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:35.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.094+0000 7fd281200640 1 --2- 172.21.15.86:0/2009173127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd27c15b640 0x7fd27c06dd10 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:35.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.094+0000 7fd281200640 1 --2- 172.21.15.86:0/2009173127 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd27c155590 0x7fd27c154cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:35.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.094+0000 7fd281200640 1 --2- 172.21.15.86:0/2009173127 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2600039b0 0x7fd27c0c96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:35.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.094+0000 7fd281200640 1 -- 172.21.15.86:0/2009173127 >> 172.21.15.86:0/2009173127 conn(0x7fd27c0b40e0 msgr2=0x7fd27c15d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:35.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.094+0000 7fd281200640 1 -- 172.21.15.86:0/2009173127 shutdown_connections 2024-04-20T21:26:35.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.094+0000 7fd281200640 1 -- 172.21.15.86:0/2009173127 wait complete. 2024-04-20T21:26:35.115 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:26:35.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.318+0000 7fe9a4c00640 1 Processor -- start 2024-04-20T21:26:35.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.318+0000 7fe9a4c00640 1 -- start start 2024-04-20T21:26:35.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.320+0000 7fe9a4c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe9a0162ed0 0x7fe9a006b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:35.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.320+0000 7fe9a4c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9a00b3410 0x7fe9a006bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:35.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.320+0000 7fe9a4c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe9a006e160 0x7fe9a0071db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:35.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.320+0000 7fe9a4c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe9a00b3910 con 0x7fe9a0162ed0 2024-04-20T21:26:35.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.320+0000 7fe9a4c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fe9a00b3bf0 con 0x7fe9a006e160 2024-04-20T21:26:35.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.320+0000 7fe9a4c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe9a00b3ed0 con 0x7fe9a00b3410 2024-04-20T21:26:35.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.320+0000 7fe99de00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe9a0162ed0 0x7fe9a006b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:35.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.320+0000 7fe99de00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe9a0162ed0 0x7fe9a006b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43932/0 (socket says 172.21.15.86:43932) 2024-04-20T21:26:35.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.320+0000 7fe99de00640 1 -- 172.21.15.86:0/2443866281 learned_addr learned my addr 172.21.15.86:0/2443866281 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:35.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.320+0000 7fe99d400640 1 --2- 172.21.15.86:0/2443866281 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9a00b3410 0x7fe9a006bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:35.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.320+0000 7fe99e800640 1 --2- 172.21.15.86:0/2443866281 >> v2:172.21.15.144:3300/0 conn(0x7fe9a006e160 0x7fe9a0071db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:35.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.321+0000 7fe99de00640 1 -- 172.21.15.86:0/2443866281 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9a00b3410 msgr2=0x7fe9a006bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:35.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.321+0000 7fe99de00640 1 --2- 172.21.15.86:0/2443866281 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9a00b3410 0x7fe9a006bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:35.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.321+0000 7fe99de00640 1 -- 172.21.15.86:0/2443866281 >> v2:172.21.15.144:3300/0 conn(0x7fe9a006e160 msgr2=0x7fe9a0071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:35.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.321+0000 7fe99de00640 1 --2- 172.21.15.86:0/2443866281 >> v2:172.21.15.144:3300/0 conn(0x7fe9a006e160 0x7fe9a0071db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:35.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.321+0000 7fe99de00640 1 -- 172.21.15.86:0/2443866281 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9a00b41b0 con 0x7fe9a0162ed0 2024-04-20T21:26:35.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.322+0000 7fe99de00640 1 --2- 172.21.15.86:0/2443866281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9a0162ed0 0x7fe9a006b980 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fe9a006de80 tx=0x7fe99002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80b77d801a451693 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:35.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.323+0000 7fe99ca00640 1 -- 172.21.15.86:0/2443866281 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe990035020 con 0x7fe9a0162ed0 2024-04-20T21:26:35.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.323+0000 7fe99ca00640 1 -- 172.21.15.86:0/2443866281 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe990031870 con 0x7fe9a0162ed0 2024-04-20T21:26:35.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.323+0000 7fe99ca00640 1 -- 172.21.15.86:0/2443866281 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe990031c40 con 0x7fe9a0162ed0 2024-04-20T21:26:35.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.323+0000 7fe9a4c00640 1 -- 172.21.15.86:0/2443866281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9a0162ed0 msgr2=0x7fe9a006b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:35.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.323+0000 7fe9a4c00640 1 --2- 172.21.15.86:0/2443866281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9a0162ed0 0x7fe9a006b980 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fe9a006de80 tx=0x7fe99002b910 comp rx=0 tx=0).stop 2024-04-20T21:26:35.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.324+0000 7fe9a4c00640 1 -- 172.21.15.86:0/2443866281 shutdown_connections 2024-04-20T21:26:35.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.324+0000 7fe9a4c00640 1 --2- 172.21.15.86:0/2443866281 >> v2:172.21.15.144:3300/0 conn(0x7fe9a006e160 0x7fe9a0071db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:35.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.324+0000 7fe9a4c00640 1 --2- 172.21.15.86:0/2443866281 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9a00b3410 0x7fe9a006bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:35.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.324+0000 7fe9a4c00640 1 --2- 172.21.15.86:0/2443866281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9a0162ed0 0x7fe9a006b980 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:35.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.324+0000 7fe9a4c00640 1 -- 172.21.15.86:0/2443866281 >> 172.21.15.86:0/2443866281 conn(0x7fe9a00b2110 msgr2=0x7fe9a00b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:35.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.324+0000 7fe9a4c00640 1 -- 172.21.15.86:0/2443866281 shutdown_connections 2024-04-20T21:26:35.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.324+0000 7fe9a4c00640 1 -- 172.21.15.86:0/2443866281 wait complete. 2024-04-20T21:26:35.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.326+0000 7fe9a4c00640 1 Processor -- start 2024-04-20T21:26:35.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.326+0000 7fe9a4c00640 1 -- start start 2024-04-20T21:26:35.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.327+0000 7fe9a4c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9a006e160 0x7fe9a0158c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:35.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.327+0000 7fe9a4c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9a00b3410 0x7fe9a014a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:35.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.327+0000 7fe9a4c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9a0162ed0 0x7fe9a015b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:35.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.327+0000 7fe9a4c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a006fca0 con 0x7fe9a006e160 2024-04-20T21:26:35.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.327+0000 7fe9a4c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a006fb20 con 0x7fe9a00b3410 2024-04-20T21:26:35.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.327+0000 7fe9a4c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe9a00722f0 con 0x7fe9a0162ed0 2024-04-20T21:26:35.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.327+0000 7fe99de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9a006e160 0x7fe9a0158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:35.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.327+0000 7fe99de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9a006e160 0x7fe9a0158c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43934/0 (socket says 172.21.15.86:43934) 2024-04-20T21:26:35.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.327+0000 7fe99de00640 1 -- 172.21.15.86:0/312183927 learned_addr learned my addr 172.21.15.86:0/312183927 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:35.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.328+0000 7fe99de00640 1 -- 172.21.15.86:0/312183927 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9a0162ed0 msgr2=0x7fe9a015b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:35.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.328+0000 7fe99de00640 1 --2- 172.21.15.86:0/312183927 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9a0162ed0 0x7fe9a015b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:35.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.328+0000 7fe99de00640 1 -- 172.21.15.86:0/312183927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9a00b3410 msgr2=0x7fe9a014a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:35.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.328+0000 7fe99de00640 1 --2- 172.21.15.86:0/312183927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9a00b3410 0x7fe9a014a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:35.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.328+0000 7fe99de00640 1 -- 172.21.15.86:0/312183927 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9a0157e90 con 0x7fe9a006e160 2024-04-20T21:26:35.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.328+0000 7fe99de00640 1 --2- 172.21.15.86:0/312183927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9a006e160 0x7fe9a0158c90 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fe99002b950 tx=0x7fe9900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:35.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.328+0000 7fe996000640 1 -- 172.21.15.86:0/312183927 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe990040070 con 0x7fe9a006e160 2024-04-20T21:26:35.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.328+0000 7fe996000640 1 -- 172.21.15.86:0/312183927 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe990030d00 con 0x7fe9a006e160 2024-04-20T21:26:35.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.328+0000 7fe996000640 1 -- 172.21.15.86:0/312183927 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe990035040 con 0x7fe9a006e160 2024-04-20T21:26:35.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.328+0000 7fe9a4c00640 1 -- 172.21.15.86:0/312183927 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9a0159fc0 con 0x7fe9a006e160 2024-04-20T21:26:35.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.329+0000 7fe9a4c00640 1 -- 172.21.15.86:0/312183927 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9a015a580 con 0x7fe9a006e160 2024-04-20T21:26:35.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.330+0000 7fe9a4c00640 1 -- 172.21.15.86:0/312183927 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9a00689e0 con 0x7fe9a006e160 2024-04-20T21:26:35.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.331+0000 7fe996000640 1 -- 172.21.15.86:0/312183927 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe99004b060 con 0x7fe9a006e160 2024-04-20T21:26:35.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.331+0000 7fe996000640 1 --2- 172.21.15.86:0/312183927 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe98006c8f0 0x7fe98006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:35.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.332+0000 7fe996000640 1 -- 172.21.15.86:0/312183927 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7fe9900ada00 con 0x7fe9a006e160 2024-04-20T21:26:35.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.332+0000 7fe99d400640 1 --2- 172.21.15.86:0/312183927 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe98006c8f0 0x7fe98006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:35.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.333+0000 7fe99d400640 1 --2- 172.21.15.86:0/312183927 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe98006c8f0 0x7fe98006edb0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fe9a01fbde0 tx=0x7fe98c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:35.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.336+0000 7fe996000640 1 -- 172.21.15.86:0/312183927 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe99007bd40 con 0x7fe9a006e160 2024-04-20T21:26:35.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.534+0000 7fe9a4c00640 1 -- 172.21.15.86:0/312183927 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe9a01612a0 con 0x7fe9a006e160 2024-04-20T21:26:35.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.535+0000 7fe996000640 1 -- 172.21.15.86:0/312183927 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7fe99007b640 con 0x7fe9a006e160 2024-04-20T21:26:35.535 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-20T21:26:35.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.538+0000 7fe9a4c00640 1 -- 172.21.15.86:0/312183927 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe98006c8f0 msgr2=0x7fe98006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:35.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.538+0000 7fe9a4c00640 1 --2- 172.21.15.86:0/312183927 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe98006c8f0 0x7fe98006edb0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fe9a01fbde0 tx=0x7fe98c008040 comp rx=0 tx=0).stop 2024-04-20T21:26:35.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.538+0000 7fe9a4c00640 1 -- 172.21.15.86:0/312183927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9a006e160 msgr2=0x7fe9a0158c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:35.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.538+0000 7fe9a4c00640 1 --2- 172.21.15.86:0/312183927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9a006e160 0x7fe9a0158c90 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fe99002b950 tx=0x7fe9900046c0 comp rx=0 tx=0).stop 2024-04-20T21:26:35.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.538+0000 7fe9a4c00640 1 -- 172.21.15.86:0/312183927 shutdown_connections 2024-04-20T21:26:35.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.538+0000 7fe9a4c00640 1 --2- 172.21.15.86:0/312183927 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe98006c8f0 0x7fe98006edb0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:35.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.538+0000 7fe9a4c00640 1 --2- 172.21.15.86:0/312183927 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9a0162ed0 0x7fe9a015b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:35.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.538+0000 7fe9a4c00640 1 --2- 172.21.15.86:0/312183927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9a00b3410 0x7fe9a014a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:35.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.538+0000 7fe9a4c00640 1 --2- 172.21.15.86:0/312183927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9a006e160 0x7fe9a0158c90 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:35.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.538+0000 7fe9a4c00640 1 -- 172.21.15.86:0/312183927 >> 172.21.15.86:0/312183927 conn(0x7fe9a00b2110 msgr2=0x7fe9a00b24f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:35.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.538+0000 7fe9a4c00640 1 -- 172.21.15.86:0/312183927 shutdown_connections 2024-04-20T21:26:35.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.539+0000 7fe9a4c00640 1 -- 172.21.15.86:0/312183927 wait complete. 2024-04-20T21:26:35.557 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:26:35.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.758+0000 7f713fe00640 1 Processor -- start 2024-04-20T21:26:35.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.758+0000 7f713fe00640 1 -- start start 2024-04-20T21:26:35.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.759+0000 7f713fe00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f71400b3650 0x7f7140155120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:35.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.759+0000 7f713fe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f714015b0d0 0x7f714015e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:35.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.760+0000 7f713fe00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f714015b920 con 0x7f714015ec00 2024-04-20T21:26:35.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.760+0000 7f713fe00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f71400b3a30 con 0x7f71400b3650 2024-04-20T21:26:35.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.760+0000 7f713fe00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f714015c530 con 0x7f714015b0d0 2024-04-20T21:26:35.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.760+0000 7f7136000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f714015b0d0 0x7f714015e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:35.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.760+0000 7f713ea00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f71400b3650 0x7f7140155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:35.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.760+0000 7f713f400640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f714015ec00 0x7f714006d9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.86:50370/0 (socket says 172.21.15.86:50370) 2024-04-20T21:26:35.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.760+0000 7f713f400640 1 -- 172.21.15.86:0/858713987 learned_addr learned my addr 172.21.15.86:0/858713987 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:26:35.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.761+0000 7f7136000640 1 -- 172.21.15.86:0/858713987 >> v2:172.21.15.144:3300/0 conn(0x7f71400b3650 msgr2=0x7f7140155120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:35.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.761+0000 7f7136000640 1 --2- 172.21.15.86:0/858713987 >> v2:172.21.15.144:3300/0 conn(0x7f71400b3650 0x7f7140155120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:35.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.761+0000 7f7136000640 1 -- 172.21.15.86:0/858713987 >> v1:172.21.15.144:6789/0 conn(0x7f714015ec00 legacy=0x7f714006d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:35.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.761+0000 7f7136000640 1 -- 172.21.15.86:0/858713987 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f714015bc00 con 0x7f714015b0d0 2024-04-20T21:26:35.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.761+0000 7f713ea00640 1 --2- 172.21.15.86:0/858713987 >> v2:172.21.15.144:3300/0 conn(0x7f71400b3650 0x7f7140155120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:26:35.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.763+0000 7f7136000640 1 --2- 172.21.15.86:0/858713987 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f714015b0d0 0x7f714015e6c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f71280097a0 tx=0x7f712802dae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=319d9b9afe3001ff server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:35.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.763+0000 7f713e000640 1 -- 172.21.15.86:0/858713987 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712802f8a0 con 0x7f714015b0d0 2024-04-20T21:26:35.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.763+0000 7f713e000640 1 -- 172.21.15.86:0/858713987 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f712802fa40 con 0x7f714015b0d0 2024-04-20T21:26:35.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.763+0000 7f713e000640 1 -- 172.21.15.86:0/858713987 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712802fe10 con 0x7f714015b0d0 2024-04-20T21:26:35.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.764+0000 7f713fe00640 1 -- 172.21.15.86:0/858713987 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f714015b0d0 msgr2=0x7f714015e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:35.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.764+0000 7f713fe00640 1 --2- 172.21.15.86:0/858713987 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f714015b0d0 0x7f714015e6c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f71280097a0 tx=0x7f712802dae0 comp rx=0 tx=0).stop 2024-04-20T21:26:35.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.764+0000 7f713fe00640 1 -- 172.21.15.86:0/858713987 shutdown_connections 2024-04-20T21:26:35.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.764+0000 7f713fe00640 1 --2- 172.21.15.86:0/858713987 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f714015b0d0 0x7f714015e6c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:35.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.764+0000 7f713fe00640 1 --2- 172.21.15.86:0/858713987 >> v2:172.21.15.144:3300/0 conn(0x7f71400b3650 0x7f7140155120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:35.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.764+0000 7f713fe00640 1 -- 172.21.15.86:0/858713987 >> 172.21.15.86:0/858713987 conn(0x7f71400b1f00 msgr2=0x7f71400b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:35.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.764+0000 7f713fe00640 1 -- 172.21.15.86:0/858713987 shutdown_connections 2024-04-20T21:26:35.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.764+0000 7f713fe00640 1 -- 172.21.15.86:0/858713987 wait complete. 2024-04-20T21:26:35.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.766+0000 7f713fe00640 1 Processor -- start 2024-04-20T21:26:35.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.766+0000 7f713fe00640 1 -- start start 2024-04-20T21:26:35.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.767+0000 7f713fe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71400b3650 0x7f714014d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:35.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.767+0000 7f713fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f714015b0d0 0x7f714009b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:35.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.767+0000 7f713fe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f714015ec00 0x7f714009afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:35.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.767+0000 7f713fe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f714015b6d0 con 0x7f714015b0d0 2024-04-20T21:26:35.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.767+0000 7f713fe00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f714015b550 con 0x7f714015ec00 2024-04-20T21:26:35.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.767+0000 7f713fe00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f714015f070 con 0x7f71400b3650 2024-04-20T21:26:35.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.767+0000 7f7136000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f714015b0d0 0x7f714009b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:35.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.767+0000 7f7136000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f714015b0d0 0x7f714009b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43942/0 (socket says 172.21.15.86:43942) 2024-04-20T21:26:35.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.767+0000 7f7136000640 1 -- 172.21.15.86:0/1092764312 learned_addr learned my addr 172.21.15.86:0/1092764312 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:35.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.768+0000 7f713f400640 1 --2- 172.21.15.86:0/1092764312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f714015ec00 0x7f714009afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:35.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.768+0000 7f713ea00640 1 --2- 172.21.15.86:0/1092764312 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71400b3650 0x7f714014d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:35.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.768+0000 7f7136000640 1 -- 172.21.15.86:0/1092764312 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71400b3650 msgr2=0x7f714014d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:35.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.768+0000 7f7136000640 1 --2- 172.21.15.86:0/1092764312 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71400b3650 0x7f714014d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:35.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.768+0000 7f7136000640 1 -- 172.21.15.86:0/1092764312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f714015ec00 msgr2=0x7f714009afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:35.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.768+0000 7f7136000640 1 --2- 172.21.15.86:0/1092764312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f714015ec00 0x7f714009afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:35.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.768+0000 7f7136000640 1 -- 172.21.15.86:0/1092764312 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f714009c1b0 con 0x7f714015b0d0 2024-04-20T21:26:35.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.768+0000 7f7136000640 1 --2- 172.21.15.86:0/1092764312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f714015b0d0 0x7f714009b960 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f71280098c0 tx=0x7f71280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:35.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.768+0000 7f7137400640 1 -- 172.21.15.86:0/1092764312 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7128047070 con 0x7f714015b0d0 2024-04-20T21:26:35.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.768+0000 7f7137400640 1 -- 172.21.15.86:0/1092764312 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7128030ab0 con 0x7f714015b0d0 2024-04-20T21:26:35.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.768+0000 7f7137400640 1 -- 172.21.15.86:0/1092764312 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7128042410 con 0x7f714015b0d0 2024-04-20T21:26:35.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.768+0000 7f713fe00640 1 -- 172.21.15.86:0/1092764312 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f714009c490 con 0x7f714015b0d0 2024-04-20T21:26:35.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.769+0000 7f713fe00640 1 -- 172.21.15.86:0/1092764312 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f714009a7e0 con 0x7f714015b0d0 2024-04-20T21:26:35.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.770+0000 7f713fe00640 1 -- 172.21.15.86:0/1092764312 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f714006e960 con 0x7f714015b0d0 2024-04-20T21:26:35.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.771+0000 7f7137400640 1 -- 172.21.15.86:0/1092764312 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7128039400 con 0x7f714015b0d0 2024-04-20T21:26:35.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.771+0000 7f7137400640 1 --2- 172.21.15.86:0/1092764312 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f711c06c940 0x7f711c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:35.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.772+0000 7f7137400640 1 -- 172.21.15.86:0/1092764312 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7f71280ad4c0 con 0x7f714015b0d0 2024-04-20T21:26:35.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.772+0000 7f713ea00640 1 --2- 172.21.15.86:0/1092764312 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f711c06c940 0x7f711c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:35.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.772+0000 7f713ea00640 1 --2- 172.21.15.86:0/1092764312 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f711c06c940 0x7f711c06ee00 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7130007440 tx=0x7f713000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:35.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.776+0000 7f7137400640 1 -- 172.21.15.86:0/1092764312 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f714006e960 con 0x7f714015b0d0 2024-04-20T21:26:35.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.979+0000 7f713fe00640 1 -- 172.21.15.86:0/1092764312 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f71400b10c0 con 0x7f714015b0d0 2024-04-20T21:26:35.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.980+0000 7f7137400640 1 -- 172.21.15.86:0/1092764312 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f7128035030 con 0x7f714015b0d0 2024-04-20T21:26:35.980 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-20T21:26:35.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.982+0000 7f713fe00640 1 -- 172.21.15.86:0/1092764312 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f711c06c940 msgr2=0x7f711c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:35.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.982+0000 7f713fe00640 1 --2- 172.21.15.86:0/1092764312 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f711c06c940 0x7f711c06ee00 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7130007440 tx=0x7f713000b040 comp rx=0 tx=0).stop 2024-04-20T21:26:35.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.983+0000 7f713fe00640 1 -- 172.21.15.86:0/1092764312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f714015b0d0 msgr2=0x7f714009b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:35.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.983+0000 7f713fe00640 1 --2- 172.21.15.86:0/1092764312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f714015b0d0 0x7f714009b960 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f71280098c0 tx=0x7f71280046c0 comp rx=0 tx=0).stop 2024-04-20T21:26:35.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.983+0000 7f713fe00640 1 -- 172.21.15.86:0/1092764312 shutdown_connections 2024-04-20T21:26:35.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.983+0000 7f713fe00640 1 --2- 172.21.15.86:0/1092764312 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f711c06c940 0x7f711c06ee00 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:35.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.983+0000 7f713fe00640 1 --2- 172.21.15.86:0/1092764312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f714015ec00 0x7f714009afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:35.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.983+0000 7f713fe00640 1 --2- 172.21.15.86:0/1092764312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f714015b0d0 0x7f714009b960 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:35.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.983+0000 7f713fe00640 1 --2- 172.21.15.86:0/1092764312 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f71400b3650 0x7f714014d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:35.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.983+0000 7f713fe00640 1 -- 172.21.15.86:0/1092764312 >> 172.21.15.86:0/1092764312 conn(0x7f71400b1f00 msgr2=0x7f714006d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:35.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.983+0000 7f713fe00640 1 -- 172.21.15.86:0/1092764312 shutdown_connections 2024-04-20T21:26:35.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:35.983+0000 7f713fe00640 1 -- 172.21.15.86:0/1092764312 wait complete. 2024-04-20T21:26:36.002 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T21:26:36.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.200+0000 7f2000200640 1 Processor -- start 2024-04-20T21:26:36.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.200+0000 7f2000200640 1 -- start start 2024-04-20T21:26:36.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.201+0000 7f2000200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ff806f070 0x7f1ff80b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:36.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.201+0000 7f2000200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ff815b9d0 con 0x7f1ff80cff80 2024-04-20T21:26:36.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.202+0000 7f2000200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ff80af850 con 0x7f1ff815b5f0 2024-04-20T21:26:36.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.202+0000 7f2000200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1ff80b4650 con 0x7f1ff806f070 2024-04-20T21:26:36.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.202+0000 7f1ffe200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f1ff80cff80 0x7f1ff80ce130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40032/0 (socket says 172.21.15.86:40032) 2024-04-20T21:26:36.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.202+0000 7f1ffe200640 1 -- 172.21.15.86:0/1818047407 learned_addr learned my addr 172.21.15.86:0/1818047407 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:26:36.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.202+0000 7f1ffce00640 1 --2- 172.21.15.86:0/1818047407 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ff806f070 0x7f1ff80b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:36.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.202+0000 7f1ff7e00640 1 -- 172.21.15.86:0/1818047407 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 829634198 0 0) 0x7f1ff815b9d0 con 0x7f1ff80cff80 2024-04-20T21:26:36.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.202+0000 7f1ff7e00640 1 -- 172.21.15.86:0/1818047407 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1fe0003520 con 0x7f1ff80cff80 2024-04-20T21:26:36.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.203+0000 7f1ff7e00640 1 -- 172.21.15.86:0/1818047407 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 205285953 0 0) 0x7f1fe0003520 con 0x7f1ff80cff80 2024-04-20T21:26:36.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.203+0000 7f1ff7e00640 1 -- 172.21.15.86:0/1818047407 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ff806f070 msgr2=0x7f1ff80b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:36.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.203+0000 7f1ff7e00640 1 --2- 172.21.15.86:0/1818047407 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ff806f070 0x7f1ff80b3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:36.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.203+0000 7f1ff7e00640 1 -- 172.21.15.86:0/1818047407 >> v1:172.21.15.144:6789/0 conn(0x7f1ff815b5f0 legacy=0x7f1ff80b50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:36.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.203+0000 7f1ff7e00640 1 -- 172.21.15.86:0/1818047407 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ff80afb30 con 0x7f1ff80cff80 2024-04-20T21:26:36.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.203+0000 7f1ff7e00640 1 -- 172.21.15.86:0/1818047407 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f1ff00019f0 con 0x7f1ff80cff80 2024-04-20T21:26:36.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.203+0000 7f1ff7e00640 1 -- 172.21.15.86:0/1818047407 >> v1:172.21.15.86:6789/0 conn(0x7f1ff80cff80 legacy=0x7f1ff80ce130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:36.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.203+0000 7f1ff7e00640 1 --2- 172.21.15.86:0/1818047407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fe00039e0 0x7f1fe0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:36.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.203+0000 7f1ff7e00640 1 --2- 172.21.15.86:0/1818047407 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ff806f070 0x7f1fe0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:36.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.203+0000 7f1ff7e00640 1 --2- 172.21.15.86:0/1818047407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fe0004d40 0x7f1fe00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:36.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.203+0000 7f1ff7e00640 1 -- 172.21.15.86:0/1818047407 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff80b4650 con 0x7f1fe00039e0 2024-04-20T21:26:36.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.203+0000 7f1ff7e00640 1 -- 172.21.15.86:0/1818047407 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe000ba70 con 0x7f1fe0004d40 2024-04-20T21:26:36.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.203+0000 7f1ff7e00640 1 -- 172.21.15.86:0/1818047407 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1fe000de80 con 0x7f1ff806f070 2024-04-20T21:26:36.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.203+0000 7f1ffd800640 1 --2- 172.21.15.86:0/1818047407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fe00039e0 0x7f1fe0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:36.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.204+0000 7f1ffe200640 1 --2- 172.21.15.86:0/1818047407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fe0004d40 0x7f1fe00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:36.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.204+0000 7f1ffce00640 1 --2- 172.21.15.86:0/1818047407 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ff806f070 0x7f1fe0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:36.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.204+0000 7f1ffd800640 1 -- 172.21.15.86:0/1818047407 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ff806f070 msgr2=0x7f1fe0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:36.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.204+0000 7f1ffd800640 1 --2- 172.21.15.86:0/1818047407 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ff806f070 0x7f1fe0004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:36.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.204+0000 7f1ffd800640 1 -- 172.21.15.86:0/1818047407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fe0004d40 msgr2=0x7f1fe00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:36.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.204+0000 7f1ffd800640 1 --2- 172.21.15.86:0/1818047407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fe0004d40 0x7f1fe00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:36.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.204+0000 7f1ffd800640 1 -- 172.21.15.86:0/1818047407 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe0003520 con 0x7f1fe00039e0 2024-04-20T21:26:36.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.204+0000 7f1ffe200640 1 --2- 172.21.15.86:0/1818047407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fe0004d40 0x7f1fe00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:26:36.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.205+0000 7f1ffd800640 1 --2- 172.21.15.86:0/1818047407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fe00039e0 0x7f1fe0003e00 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f1fec001720 tx=0x7f1fe0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0eac3a405fce031 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:36.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.205+0000 7f1ff7e00640 1 -- 172.21.15.86:0/1818047407 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fec001370 con 0x7f1fe00039e0 2024-04-20T21:26:36.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.205+0000 7f1ff7e00640 1 -- 172.21.15.86:0/1818047407 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fec0052d0 con 0x7f1fe00039e0 2024-04-20T21:26:36.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.205+0000 7f1ff7e00640 1 -- 172.21.15.86:0/1818047407 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fec003830 con 0x7f1fe00039e0 2024-04-20T21:26:36.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.206+0000 7f2000200640 1 -- 172.21.15.86:0/1818047407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fe00039e0 msgr2=0x7f1fe0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:36.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.206+0000 7f2000200640 1 --2- 172.21.15.86:0/1818047407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fe00039e0 0x7f1fe0003e00 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f1fec001720 tx=0x7f1fe0005830 comp rx=0 tx=0).stop 2024-04-20T21:26:36.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.206+0000 7f1ffe200640 1 -- 172.21.15.86:0/1818047407 reap_dead start 2024-04-20T21:26:36.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.206+0000 7f2000200640 1 -- 172.21.15.86:0/1818047407 shutdown_connections 2024-04-20T21:26:36.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.206+0000 7f2000200640 1 -- 172.21.15.86:0/1818047407 >> 172.21.15.86:0/1818047407 conn(0x7f1ff806b980 msgr2=0x7f1ff806bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:36.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.206+0000 7f2000200640 1 -- 172.21.15.86:0/1818047407 shutdown_connections 2024-04-20T21:26:36.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.206+0000 7f2000200640 1 -- 172.21.15.86:0/1818047407 wait complete. 2024-04-20T21:26:36.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.208+0000 7f2000200640 1 Processor -- start 2024-04-20T21:26:36.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.208+0000 7f2000200640 1 -- start start 2024-04-20T21:26:36.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.209+0000 7f2000200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fe00039e0 0x7f1ff815b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:36.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.209+0000 7f2000200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ff8144d00 0x7f1ff80b07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:36.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.209+0000 7f2000200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ff80b0cf0 0x7f1ff815a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:36.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.209+0000 7f2000200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff80d0410 con 0x7f1fe00039e0 2024-04-20T21:26:36.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.209+0000 7f2000200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff80b4b80 con 0x7f1ff8144d00 2024-04-20T21:26:36.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.209+0000 7f2000200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1ff80d0590 con 0x7f1ff80b0cf0 2024-04-20T21:26:36.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.209+0000 7f1ffd800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fe00039e0 0x7f1ff815b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:36.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.209+0000 7f1ffd800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fe00039e0 0x7f1ff815b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45578/0 (socket says 172.21.15.86:45578) 2024-04-20T21:26:36.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.209+0000 7f1ffd800640 1 -- 172.21.15.86:0/426163363 learned_addr learned my addr 172.21.15.86:0/426163363 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:36.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.209+0000 7f1ffd800640 1 -- 172.21.15.86:0/426163363 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ff80b0cf0 msgr2=0x7f1ff815a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:36.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.209+0000 7f1ffd800640 1 --2- 172.21.15.86:0/426163363 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ff80b0cf0 0x7f1ff815a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:36.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.209+0000 7f1ffd800640 1 -- 172.21.15.86:0/426163363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ff8144d00 msgr2=0x7f1ff80b07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:36.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.209+0000 7f1ffd800640 1 --2- 172.21.15.86:0/426163363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ff8144d00 0x7f1ff80b07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:36.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.209+0000 7f1ffd800640 1 -- 172.21.15.86:0/426163363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ff8145770 con 0x7f1fe00039e0 2024-04-20T21:26:36.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.210+0000 7f1ffd800640 1 --2- 172.21.15.86:0/426163363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fe00039e0 0x7f1ff815b9d0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f1fec034040 tx=0x7f1fec001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:36.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.210+0000 7f1ff5600640 1 -- 172.21.15.86:0/426163363 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fec042070 con 0x7f1fe00039e0 2024-04-20T21:26:36.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.210+0000 7f1ff5600640 1 -- 172.21.15.86:0/426163363 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fec005480 con 0x7f1fe00039e0 2024-04-20T21:26:36.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.210+0000 7f1ff5600640 1 -- 172.21.15.86:0/426163363 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fec007a30 con 0x7f1fe00039e0 2024-04-20T21:26:36.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.210+0000 7f2000200640 1 -- 172.21.15.86:0/426163363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ff8145a50 con 0x7f1fe00039e0 2024-04-20T21:26:36.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.210+0000 7f2000200640 1 -- 172.21.15.86:0/426163363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ff8146010 con 0x7f1fe00039e0 2024-04-20T21:26:36.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.211+0000 7f2000200640 1 -- 172.21.15.86:0/426163363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ff80b1110 con 0x7f1fe00039e0 2024-04-20T21:26:36.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.212+0000 7f1ff5600640 1 -- 172.21.15.86:0/426163363 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1fec045070 con 0x7f1fe00039e0 2024-04-20T21:26:36.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.213+0000 7f1ff5600640 1 --2- 172.21.15.86:0/426163363 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1fd806c920 0x7f1fd806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:36.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.213+0000 7f1ff5600640 1 -- 172.21.15.86:0/426163363 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7f1fec0ad840 con 0x7f1fe00039e0 2024-04-20T21:26:36.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.213+0000 7f1ffce00640 1 --2- 172.21.15.86:0/426163363 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1fd806c920 0x7f1fd806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:36.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.214+0000 7f1ffce00640 1 --2- 172.21.15.86:0/426163363 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1fd806c920 0x7f1fd806ede0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1ff80cf550 tx=0x7f1fe4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:36.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.218+0000 7f1ff5600640 1 -- 172.21.15.86:0/426163363 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1fec07bca0 con 0x7f1fe00039e0 2024-04-20T21:26:36.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.429+0000 7f2000200640 1 -- 172.21.15.86:0/426163363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1ff80689e0 con 0x7f1fe00039e0 2024-04-20T21:26:36.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.430+0000 7f1ff5600640 1 -- 172.21.15.86:0/426163363 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v88) ==== 80+0+124 (secure 0 0 0) 0x7f1fec07b5a0 con 0x7f1fe00039e0 2024-04-20T21:26:36.430 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.144:6835/1740492227 2024-04-21T21:25:20.040733+0000 2024-04-20T21:26:36.430 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.144:6833/1740492227 2024-04-21T21:25:20.040733+0000 2024-04-20T21:26:36.430 INFO:teuthology.orchestra.run.smithi086.stderr:listed 2 entries 2024-04-20T21:26:36.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.432+0000 7f2000200640 1 -- 172.21.15.86:0/426163363 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1fd806c920 msgr2=0x7f1fd806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:36.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.432+0000 7f2000200640 1 --2- 172.21.15.86:0/426163363 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1fd806c920 0x7f1fd806ede0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1ff80cf550 tx=0x7f1fe4008040 comp rx=0 tx=0).stop 2024-04-20T21:26:36.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.432+0000 7f2000200640 1 -- 172.21.15.86:0/426163363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fe00039e0 msgr2=0x7f1ff815b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:36.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.432+0000 7f2000200640 1 --2- 172.21.15.86:0/426163363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fe00039e0 0x7f1ff815b9d0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f1fec034040 tx=0x7f1fec001120 comp rx=0 tx=0).stop 2024-04-20T21:26:36.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.433+0000 7f2000200640 1 -- 172.21.15.86:0/426163363 shutdown_connections 2024-04-20T21:26:36.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.433+0000 7f2000200640 1 --2- 172.21.15.86:0/426163363 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1fd806c920 0x7f1fd806ede0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:36.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.433+0000 7f2000200640 1 --2- 172.21.15.86:0/426163363 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ff80b0cf0 0x7f1ff815a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:36.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.433+0000 7f2000200640 1 --2- 172.21.15.86:0/426163363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ff8144d00 0x7f1ff80b07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:36.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.433+0000 7f2000200640 1 --2- 172.21.15.86:0/426163363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fe00039e0 0x7f1ff815b9d0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:36.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.433+0000 7f2000200640 1 -- 172.21.15.86:0/426163363 >> 172.21.15.86:0/426163363 conn(0x7f1ff806b980 msgr2=0x7f1ff80afe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:36.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.433+0000 7f2000200640 1 -- 172.21.15.86:0/426163363 shutdown_connections 2024-04-20T21:26:36.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:36.433+0000 7f2000200640 1 -- 172.21.15.86:0/426163363 wait complete. 2024-04-20T21:26:36.452 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi086.front.sepia.ceph.com... 2024-04-20T21:26:36.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:26:36.452 DEBUG:teuthology.orchestra.run.smithi086:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-20T21:26:36.524 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[54464]: fuse finished with error 0 and tester_r 0 2024-04-20T21:26:36.529 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:26:42.631 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-20T21:26:42.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:26:42.631 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-20T21:26:42.691 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:26:42.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.902+0000 7fa917200640 1 Processor -- start 2024-04-20T21:26:42.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.902+0000 7fa917200640 1 -- start start 2024-04-20T21:26:42.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.903+0000 7fa917200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa910146130 0x7fa91006e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:42.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.903+0000 7fa917200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa91006dc30 con 0x7fa910072410 2024-04-20T21:26:42.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.904+0000 7fa917200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9100cd8e0 con 0x7fa9100ccdd0 2024-04-20T21:26:42.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.904+0000 7fa917200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa9100cdbc0 con 0x7fa910146130 2024-04-20T21:26:42.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.904+0000 7fa90fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa910146130 0x7fa91006e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:42.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.904+0000 7fa90fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa910146130 0x7fa91006e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45594/0 (socket says 172.21.15.86:45594) 2024-04-20T21:26:42.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.904+0000 7fa90fe00640 1 -- 172.21.15.86:0/2671850135 learned_addr learned my addr 172.21.15.86:0/2671850135 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:42.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.904+0000 7fa915200640 1 --1- 172.21.15.86:0/2671850135 >> v1:172.21.15.86:6789/0 conn(0x7fa910072410 0x7fa91015cc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40034/0 (socket says 172.21.15.86:40034) 2024-04-20T21:26:42.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.904+0000 7fa90fe00640 1 -- 172.21.15.86:0/2671850135 >> v1:172.21.15.144:6789/0 conn(0x7fa9100ccdd0 legacy=0x7fa910071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:42.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.904+0000 7fa90fe00640 1 -- 172.21.15.86:0/2671850135 >> v1:172.21.15.86:6789/0 conn(0x7fa910072410 legacy=0x7fa91015cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:42.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.904+0000 7fa90fe00640 1 -- 172.21.15.86:0/2671850135 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa91006ded0 con 0x7fa910146130 2024-04-20T21:26:42.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.905+0000 7fa90fe00640 1 --2- 172.21.15.86:0/2671850135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa910146130 0x7fa91006e6d0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fa900009870 tx=0x7fa90002ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c7c12a7d96a0d7e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:42.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.906+0000 7fa90ea00640 1 -- 172.21.15.86:0/2671850135 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90002f850 con 0x7fa910146130 2024-04-20T21:26:42.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.906+0000 7fa90ea00640 1 -- 172.21.15.86:0/2671850135 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa90002f9f0 con 0x7fa910146130 2024-04-20T21:26:42.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.906+0000 7fa90ea00640 1 -- 172.21.15.86:0/2671850135 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90002fdc0 con 0x7fa910146130 2024-04-20T21:26:42.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.906+0000 7fa917200640 1 -- 172.21.15.86:0/2671850135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa910146130 msgr2=0x7fa91006e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:42.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.906+0000 7fa917200640 1 --2- 172.21.15.86:0/2671850135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa910146130 0x7fa91006e6d0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fa900009870 tx=0x7fa90002ba30 comp rx=0 tx=0).stop 2024-04-20T21:26:42.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.907+0000 7fa917200640 1 -- 172.21.15.86:0/2671850135 shutdown_connections 2024-04-20T21:26:42.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.907+0000 7fa917200640 1 --2- 172.21.15.86:0/2671850135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa910146130 0x7fa91006e6d0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:42.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.907+0000 7fa917200640 1 -- 172.21.15.86:0/2671850135 >> 172.21.15.86:0/2671850135 conn(0x7fa9100b2360 msgr2=0x7fa9100b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:42.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.907+0000 7fa917200640 1 -- 172.21.15.86:0/2671850135 shutdown_connections 2024-04-20T21:26:42.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.907+0000 7fa917200640 1 -- 172.21.15.86:0/2671850135 wait complete. 2024-04-20T21:26:42.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.908+0000 7fa917200640 1 Processor -- start 2024-04-20T21:26:42.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.909+0000 7fa917200640 1 -- start start 2024-04-20T21:26:42.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.909+0000 7fa917200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa910072410 0x7fa9101e7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:42.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.909+0000 7fa917200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9100ccdd0 0x7fa9101e8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:42.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.909+0000 7fa917200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa910146130 0x7fa91012fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:42.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.909+0000 7fa917200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa9100cd230 con 0x7fa910072410 2024-04-20T21:26:42.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.909+0000 7fa917200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa91006f150 con 0x7fa9100ccdd0 2024-04-20T21:26:42.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.909+0000 7fa917200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa9100cd3b0 con 0x7fa910146130 2024-04-20T21:26:42.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.910+0000 7fa90fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa910072410 0x7fa9101e7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:42.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.910+0000 7fa90fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa910072410 0x7fa9101e7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45598/0 (socket says 172.21.15.86:45598) 2024-04-20T21:26:42.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.910+0000 7fa90fe00640 1 -- 172.21.15.86:0/3296480858 learned_addr learned my addr 172.21.15.86:0/3296480858 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:42.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.910+0000 7fa90fe00640 1 -- 172.21.15.86:0/3296480858 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa910146130 msgr2=0x7fa91012fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:42.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.910+0000 7fa90fe00640 1 --2- 172.21.15.86:0/3296480858 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa910146130 0x7fa91012fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:42.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.910+0000 7fa90fe00640 1 -- 172.21.15.86:0/3296480858 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9100ccdd0 msgr2=0x7fa9101e8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:42.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.910+0000 7fa90fe00640 1 --2- 172.21.15.86:0/3296480858 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9100ccdd0 0x7fa9101e8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:42.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.910+0000 7fa90fe00640 1 -- 172.21.15.86:0/3296480858 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa91012ed60 con 0x7fa910072410 2024-04-20T21:26:42.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.910+0000 7fa90fe00640 1 --2- 172.21.15.86:0/3296480858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa910072410 0x7fa9101e7d20 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fa900000c00 tx=0x7fa900004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:42.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.911+0000 7fa907e00640 1 -- 172.21.15.86:0/3296480858 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa900037d10 con 0x7fa910072410 2024-04-20T21:26:42.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.911+0000 7fa907e00640 1 -- 172.21.15.86:0/3296480858 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa900046070 con 0x7fa910072410 2024-04-20T21:26:42.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.911+0000 7fa907e00640 1 -- 172.21.15.86:0/3296480858 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa900041620 con 0x7fa910072410 2024-04-20T21:26:42.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.911+0000 7fa917200640 1 -- 172.21.15.86:0/3296480858 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa91012f040 con 0x7fa910072410 2024-04-20T21:26:42.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.911+0000 7fa917200640 1 -- 172.21.15.86:0/3296480858 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9101e5400 con 0x7fa910072410 2024-04-20T21:26:42.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.912+0000 7fa917200640 1 -- 172.21.15.86:0/3296480858 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9100689e0 con 0x7fa910072410 2024-04-20T21:26:42.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.913+0000 7fa907e00640 1 -- 172.21.15.86:0/3296480858 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa900040070 con 0x7fa910072410 2024-04-20T21:26:42.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.914+0000 7fa907e00640 1 --2- 172.21.15.86:0/3296480858 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa8f006c8f0 0x7fa8f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:42.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.914+0000 7fa907e00640 1 -- 172.21.15.86:0/3296480858 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7fa9000ae310 con 0x7fa910072410 2024-04-20T21:26:42.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.914+0000 7fa90f400640 1 --2- 172.21.15.86:0/3296480858 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa8f006c8f0 0x7fa8f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:42.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.915+0000 7fa90f400640 1 --2- 172.21.15.86:0/3296480858 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa8f006c8f0 0x7fa8f006edb0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fa9101f7ee0 tx=0x7fa8fc007990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:42.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:42.919+0000 7fa907e00640 1 -- 172.21.15.86:0/3296480858 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa900033030 con 0x7fa910072410 2024-04-20T21:26:43.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.151+0000 7fa917200640 1 -- 172.21.15.86:0/3296480858 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa91015ff50 con 0x7fa910072410 2024-04-20T21:26:43.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.153+0000 7fa907e00640 1 -- 172.21.15.86:0/3296480858 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5208 (secure 0 0 0) 0x7fa90007c760 con 0x7fa910072410 2024-04-20T21:26:43.152 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:26:43.152 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":66,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7506,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/4013863257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4013863257},{"type":"v1","addr":"172.21.15.86:6836","nonce":4013863257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":7509,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2044883073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2044883073},{"type":"v1","addr":"172.21.15.86:6837","nonce":2044883073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54},{"gid":7518,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/2351381466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2351381466},{"type":"v1","addr":"172.21.15.144:6834","nonce":2351381466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:24:58.299257+0000","modified":"2024-04-20T21:25:31.064401+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7773},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7773":{"gid":7773,"name":"b","rank":0,"incarnation":62,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6835/3981403733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":3981403733},{"type":"v1","addr":"172.21.15.144:6835","nonce":3981403733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7773,"qdb_cluster":[7773]},"id":6}]} 2024-04-20T21:26:43.152 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 66 2024-04-20T21:26:43.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.155+0000 7fa917200640 1 -- 172.21.15.86:0/3296480858 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa8f006c8f0 msgr2=0x7fa8f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:43.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.155+0000 7fa917200640 1 --2- 172.21.15.86:0/3296480858 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa8f006c8f0 0x7fa8f006edb0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fa9101f7ee0 tx=0x7fa8fc007990 comp rx=0 tx=0).stop 2024-04-20T21:26:43.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.155+0000 7fa917200640 1 -- 172.21.15.86:0/3296480858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa910072410 msgr2=0x7fa9101e7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:43.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.155+0000 7fa917200640 1 --2- 172.21.15.86:0/3296480858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa910072410 0x7fa9101e7d20 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fa900000c00 tx=0x7fa900004730 comp rx=0 tx=0).stop 2024-04-20T21:26:43.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.155+0000 7fa917200640 1 -- 172.21.15.86:0/3296480858 shutdown_connections 2024-04-20T21:26:43.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.155+0000 7fa917200640 1 --2- 172.21.15.86:0/3296480858 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa8f006c8f0 0x7fa8f006edb0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:43.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.155+0000 7fa917200640 1 --2- 172.21.15.86:0/3296480858 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa910146130 0x7fa91012fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:43.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.155+0000 7fa917200640 1 --2- 172.21.15.86:0/3296480858 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9100ccdd0 0x7fa9101e8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:43.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.155+0000 7fa917200640 1 --2- 172.21.15.86:0/3296480858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa910072410 0x7fa9101e7d20 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:43.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.155+0000 7fa917200640 1 -- 172.21.15.86:0/3296480858 >> 172.21.15.86:0/3296480858 conn(0x7fa9100b2360 msgr2=0x7fa91015cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:43.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.155+0000 7fa917200640 1 -- 172.21.15.86:0/3296480858 shutdown_connections 2024-04-20T21:26:43.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.155+0000 7fa917200640 1 -- 172.21.15.86:0/3296480858 wait complete. 2024-04-20T21:26:43.173 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-04-20T21:26:43.417 INFO:teuthology.orchestra.run.smithi144.stdout:{"message":"","return_code":0} 2024-04-20T21:26:43.418 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-20T21:26:43.418 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-20T21:26:43.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.614+0000 7fa580c00640 1 Processor -- start 2024-04-20T21:26:43.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.614+0000 7fa580c00640 1 -- start start 2024-04-20T21:26:43.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.616+0000 7fa580c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa57c0af7e0 0x7fa57c0b4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:43.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.616+0000 7fa580c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa57c0cd220 0x7fa57c0b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:43.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.616+0000 7fa580c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa57c160160 con 0x7fa57c15c590 2024-04-20T21:26:43.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.616+0000 7fa580c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa57c160440 con 0x7fa57c0cd220 2024-04-20T21:26:43.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.616+0000 7fa580c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa57c160720 con 0x7fa57c0af7e0 2024-04-20T21:26:43.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.616+0000 7fa57be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa57c0cd220 0x7fa57c0b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:43.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.616+0000 7fa57be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa57c0cd220 0x7fa57c0b46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45602/0 (socket says 172.21.15.86:45602) 2024-04-20T21:26:43.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.616+0000 7fa57be00640 1 -- 172.21.15.86:0/1728719229 learned_addr learned my addr 172.21.15.86:0/1728719229 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:43.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.616+0000 7fa57be00640 1 -- 172.21.15.86:0/1728719229 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa57c0af7e0 msgr2=0x7fa57c0b4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:43.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.616+0000 7fa57be00640 1 --2- 172.21.15.86:0/1728719229 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa57c0af7e0 0x7fa57c0b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:43.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.616+0000 7fa57be00640 1 -- 172.21.15.86:0/1728719229 >> v1:172.21.15.144:6789/0 conn(0x7fa57c15c590 legacy=0x7fa57c0ccb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:43.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.616+0000 7fa57be00640 1 -- 172.21.15.86:0/1728719229 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa57c160a00 con 0x7fa57c0cd220 2024-04-20T21:26:43.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.618+0000 7fa57be00640 1 --2- 172.21.15.86:0/1728719229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa57c0cd220 0x7fa57c0b46d0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fa574009870 tx=0x7fa57402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3348223dbc6057d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:43.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.618+0000 7fa57a000640 1 -- 172.21.15.86:0/1728719229 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa574035020 con 0x7fa57c0cd220 2024-04-20T21:26:43.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.618+0000 7fa57a000640 1 -- 172.21.15.86:0/1728719229 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa574031640 con 0x7fa57c0cd220 2024-04-20T21:26:43.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.618+0000 7fa57a000640 1 -- 172.21.15.86:0/1728719229 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57402d040 con 0x7fa57c0cd220 2024-04-20T21:26:43.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.618+0000 7fa580c00640 1 -- 172.21.15.86:0/1728719229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa57c0cd220 msgr2=0x7fa57c0b46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:43.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.619+0000 7fa580c00640 1 --2- 172.21.15.86:0/1728719229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa57c0cd220 0x7fa57c0b46d0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fa574009870 tx=0x7fa57402b8f0 comp rx=0 tx=0).stop 2024-04-20T21:26:43.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.619+0000 7fa580c00640 1 -- 172.21.15.86:0/1728719229 shutdown_connections 2024-04-20T21:26:43.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.619+0000 7fa580c00640 1 --2- 172.21.15.86:0/1728719229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa57c0cd220 0x7fa57c0b46d0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:43.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.619+0000 7fa580c00640 1 --2- 172.21.15.86:0/1728719229 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa57c0af7e0 0x7fa57c0b4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:43.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.619+0000 7fa580c00640 1 -- 172.21.15.86:0/1728719229 >> 172.21.15.86:0/1728719229 conn(0x7fa57c06bb60 msgr2=0x7fa57c06bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:43.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.619+0000 7fa580c00640 1 -- 172.21.15.86:0/1728719229 shutdown_connections 2024-04-20T21:26:43.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.619+0000 7fa580c00640 1 -- 172.21.15.86:0/1728719229 wait complete. 2024-04-20T21:26:43.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.620+0000 7fa580c00640 1 Processor -- start 2024-04-20T21:26:43.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.620+0000 7fa580c00640 1 -- start start 2024-04-20T21:26:43.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.621+0000 7fa580c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa57c0af7e0 0x7fa57c0c6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:43.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.621+0000 7fa580c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa57c0cd220 0x7fa57c158070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:43.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.621+0000 7fa580c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa57c15c590 0x7fa57c0c45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:43.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.621+0000 7fa580c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa57c15c970 con 0x7fa57c0af7e0 2024-04-20T21:26:43.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.621+0000 7fa580c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa57c06f3d0 con 0x7fa57c0cd220 2024-04-20T21:26:43.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.621+0000 7fa580c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa57c160fc0 con 0x7fa57c15c590 2024-04-20T21:26:43.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.621+0000 7fa57b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa57c0af7e0 0x7fa57c0c6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:43.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.622+0000 7fa57b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa57c0af7e0 0x7fa57c0c6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45616/0 (socket says 172.21.15.86:45616) 2024-04-20T21:26:43.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.622+0000 7fa57b400640 1 -- 172.21.15.86:0/1603050299 learned_addr learned my addr 172.21.15.86:0/1603050299 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:43.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.622+0000 7fa57b400640 1 -- 172.21.15.86:0/1603050299 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa57c15c590 msgr2=0x7fa57c0c45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:43.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.622+0000 7fa57b400640 1 --2- 172.21.15.86:0/1603050299 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa57c15c590 0x7fa57c0c45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:43.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.622+0000 7fa57b400640 1 -- 172.21.15.86:0/1603050299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa57c0cd220 msgr2=0x7fa57c158070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:43.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.622+0000 7fa57b400640 1 --2- 172.21.15.86:0/1603050299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa57c0cd220 0x7fa57c158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:43.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.622+0000 7fa57b400640 1 -- 172.21.15.86:0/1603050299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa57c0ad930 con 0x7fa57c0af7e0 2024-04-20T21:26:43.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.622+0000 7fa57b400640 1 --2- 172.21.15.86:0/1603050299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa57c0af7e0 0x7fa57c0c6630 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fa56c00ba50 tx=0x7fa56c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:43.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.622+0000 7fa573400640 1 -- 172.21.15.86:0/1603050299 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56c019070 con 0x7fa57c0af7e0 2024-04-20T21:26:43.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.622+0000 7fa573400640 1 -- 172.21.15.86:0/1603050299 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa56c00cc20 con 0x7fa57c0af7e0 2024-04-20T21:26:43.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.622+0000 7fa580c00640 1 -- 172.21.15.86:0/1603050299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa57c0adc10 con 0x7fa57c0af7e0 2024-04-20T21:26:43.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.622+0000 7fa573400640 1 -- 172.21.15.86:0/1603050299 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56c014440 con 0x7fa57c0af7e0 2024-04-20T21:26:43.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.623+0000 7fa580c00640 1 -- 172.21.15.86:0/1603050299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa57c0ae1d0 con 0x7fa57c0af7e0 2024-04-20T21:26:43.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.624+0000 7fa580c00640 1 -- 172.21.15.86:0/1603050299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa57c04fcf0 con 0x7fa57c0af7e0 2024-04-20T21:26:43.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.625+0000 7fa573400640 1 -- 172.21.15.86:0/1603050299 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa56c01c070 con 0x7fa57c0af7e0 2024-04-20T21:26:43.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.625+0000 7fa573400640 1 --2- 172.21.15.86:0/1603050299 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa55806c8f0 0x7fa55806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:43.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.626+0000 7fa573400640 1 -- 172.21.15.86:0/1603050299 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7fa56c08b8b0 con 0x7fa57c0af7e0 2024-04-20T21:26:43.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.626+0000 7fa57aa00640 1 --2- 172.21.15.86:0/1603050299 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa55806c8f0 0x7fa55806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:43.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.626+0000 7fa57aa00640 1 --2- 172.21.15.86:0/1603050299 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa55806c8f0 0x7fa55806edb0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fa57c0ca5b0 tx=0x7fa564006c90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:43.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.630+0000 7fa573400640 1 -- 172.21.15.86:0/1603050299 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa56c059630 con 0x7fa57c0af7e0 2024-04-20T21:26:43.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.868+0000 7fa580c00640 1 -- 172.21.15.86:0/1603050299 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa57c0cd6a0 con 0x7fa57c0af7e0 2024-04-20T21:26:43.867 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-04-20T21:26:43.869+0000 7f5bb9c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T21:26:43.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.872+0000 7fa573400640 1 -- 172.21.15.86:0/1603050299 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v67) ==== 161+0+0 (secure 0 0 0) 0x7fa56c05d930 con 0x7fa57c0af7e0 2024-04-20T21:26:43.872 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-20T21:26:43.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.875+0000 7fa580c00640 1 -- 172.21.15.86:0/1603050299 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa55806c8f0 msgr2=0x7fa55806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:43.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.875+0000 7fa580c00640 1 --2- 172.21.15.86:0/1603050299 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa55806c8f0 0x7fa55806edb0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fa57c0ca5b0 tx=0x7fa564006c90 comp rx=0 tx=0).stop 2024-04-20T21:26:43.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.875+0000 7fa580c00640 1 -- 172.21.15.86:0/1603050299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa57c0af7e0 msgr2=0x7fa57c0c6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:43.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.875+0000 7fa580c00640 1 --2- 172.21.15.86:0/1603050299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa57c0af7e0 0x7fa57c0c6630 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fa56c00ba50 tx=0x7fa56c00bf10 comp rx=0 tx=0).stop 2024-04-20T21:26:43.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.875+0000 7fa580c00640 1 -- 172.21.15.86:0/1603050299 shutdown_connections 2024-04-20T21:26:43.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.875+0000 7fa580c00640 1 --2- 172.21.15.86:0/1603050299 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa55806c8f0 0x7fa55806edb0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:43.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.875+0000 7fa580c00640 1 --2- 172.21.15.86:0/1603050299 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa57c15c590 0x7fa57c0c45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:43.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.875+0000 7fa580c00640 1 --2- 172.21.15.86:0/1603050299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa57c0cd220 0x7fa57c158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:43.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.875+0000 7fa580c00640 1 --2- 172.21.15.86:0/1603050299 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa57c0af7e0 0x7fa57c0c6630 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:43.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.875+0000 7fa580c00640 1 -- 172.21.15.86:0/1603050299 >> 172.21.15.86:0/1603050299 conn(0x7fa57c06bb60 msgr2=0x7fa57c06bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:43.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.875+0000 7fa580c00640 1 -- 172.21.15.86:0/1603050299 shutdown_connections 2024-04-20T21:26:43.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.875+0000 7fa580c00640 1 -- 172.21.15.86:0/1603050299 wait complete. 2024-04-20T21:26:43.895 DEBUG:teuthology.orchestra.run.smithi086:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-04-20T21:26:43.908 INFO:tasks.ceph.mds.b.smithi144.stderr: -4918> 2024-04-20T21:25:26.067+0000 7f741b200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T21:26:43.910 INFO:tasks.ceph.mds.b.smithi144.stderr: -4543> 2024-04-20T21:25:28.219+0000 7f7416200640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-04-20T21:26:43.911 INFO:tasks.ceph.mds.b.smithi144.stderr: -4377> 2024-04-20T21:25:28.483+0000 7f7416200640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 ap=1 f(v0 m2024-04-20T21:25:07.713157+0000 1=1+0) n(v0 rc2024-04-20T21:25:07.851790+0000 2=1+1) | dirfrag=0 authpin=1 scrubqueue=0 0x561f42e0a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-20T21:25:07.713157+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-20T21:25:07.851790+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-04-20T21:26:43.920 INFO:tasks.ceph.mds.b.smithi144.stderr: -2571> 2024-04-20T21:26:00.544+0000 7f7416200640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 pv17 ap=3 DIRTYPARENT f(v0 m2024-04-20T21:25:07.713157+0000 1=1+0)->f(v1 m2024-04-20T21:25:07.713157+0000) n(v0 rc2024-04-20T21:25:07.851790+0000 2=1+1)->n(v0 rc2024-04-20T21:25:07.851790+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) | dirtyscattered=2 lock=1 dirfrag=1 dirtyrstat=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x561f42e0a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-20T21:25:07.713157+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-20T21:25:07.851790+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-04-20T21:26:43.988 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2024-04-20T21:26:43.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.994+0000 7f0d17c1d740 1 -- start start 2024-04-20T21:26:43.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.994+0000 7f0d17c1d740 1 Processor -- start 2024-04-20T21:26:43.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.995+0000 7f0d17c1d740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ee2e5eac00 0x55ee2f356000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:43.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.995+0000 7f0d17c1d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee2e5eb800 0x55ee2f356580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:43.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.995+0000 7f0d17c1d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee2e5ebc00 0x55ee2f356b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:43.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.995+0000 7f0d17c1d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ee2f35a780 con 0x55ee2e5eb800 2024-04-20T21:26:43.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.995+0000 7f0d17c1d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ee2f35a600 con 0x55ee2e5ebc00 2024-04-20T21:26:43.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.995+0000 7f0d17c1d740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55ee2f35a480 con 0x55ee2e5eac00 2024-04-20T21:26:43.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.995+0000 7f0d14800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee2e5eb800 0x55ee2f356580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:43.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.995+0000 7f0d14800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee2e5eb800 0x55ee2f356580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45622/0 (socket says 172.21.15.86:45622) 2024-04-20T21:26:43.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.995+0000 7f0d14800640 1 -- 172.21.15.86:0/3822797471 learned_addr learned my addr 172.21.15.86:0/3822797471 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:43.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.995+0000 7f0d15c00640 1 --2- 172.21.15.86:0/3822797471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee2e5ebc00 0x55ee2f356b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:43.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.995+0000 7f0d15200640 1 --2- 172.21.15.86:0/3822797471 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ee2e5eac00 0x55ee2f356000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:43.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.996+0000 7f0d14800640 1 -- 172.21.15.86:0/3822797471 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ee2e5eac00 msgr2=0x55ee2f356000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:43.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.996+0000 7f0d14800640 1 --2- 172.21.15.86:0/3822797471 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ee2e5eac00 0x55ee2f356000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:43.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.996+0000 7f0d14800640 1 -- 172.21.15.86:0/3822797471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee2e5ebc00 msgr2=0x55ee2f356b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:43.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.996+0000 7f0d14800640 1 --2- 172.21.15.86:0/3822797471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee2e5ebc00 0x55ee2f356b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:43.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.996+0000 7f0d14800640 1 -- 172.21.15.86:0/3822797471 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee2e617680 con 0x55ee2e5eb800 2024-04-20T21:26:43.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.996+0000 7f0d14800640 1 --2- 172.21.15.86:0/3822797471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee2e5eb800 0x55ee2f356580 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x55ee2f417860 tx=0x55ee2f38f020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f85784ab1233ab4d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:43.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.996+0000 7f0d12000640 1 -- 172.21.15.86:0/3822797471 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ee2e610680 con 0x55ee2e5eb800 2024-04-20T21:26:43.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.996+0000 7f0d12000640 1 -- 172.21.15.86:0/3822797471 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55ee2e634e00 con 0x55ee2e5eb800 2024-04-20T21:26:43.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.996+0000 7f0d12000640 1 -- 172.21.15.86:0/3822797471 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ee2e610b60 con 0x55ee2e5eb800 2024-04-20T21:26:43.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.996+0000 7f0d17c1d740 1 -- 172.21.15.86:0/3822797471 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee2e6174a0 con 0x55ee2e5eb800 2024-04-20T21:26:43.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.996+0000 7f0d17c1d740 1 -- 172.21.15.86:0/3822797471 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55ee2e617a40 con 0x55ee2e5eb800 2024-04-20T21:26:43.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.996+0000 7f0d17c1d740 4 waiting for MDS map... 2024-04-20T21:26:43.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.997+0000 7f0d12000640 1 -- 172.21.15.86:0/3822797471 <== mon.0 v2:172.21.15.86:3300/0 4 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x55ee2f4cc600 con 0x55ee2e5eb800 2024-04-20T21:26:43.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.997+0000 7f0d12000640 1 -- 172.21.15.86:0/3822797471 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 67) ==== 2886+0+0 (secure 0 0 0) 0x55ee2e643180 con 0x55ee2e5eb800 2024-04-20T21:26:43.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.997+0000 7f0d17c1d740 4 Got MDS map 67 2024-04-20T21:26:43.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.997+0000 7f0d17c1d740 4 datascan.main: Using metadata pool output 2024-04-20T21:26:43.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.997+0000 7f0d17c1d740 4 datascan.main: connecting to RADOS... 2024-04-20T21:26:43.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.998+0000 7f0d17c1d740 1 Processor -- start 2024-04-20T21:26:43.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.998+0000 7f0d17c1d740 1 -- start start 2024-04-20T21:26:44.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.999+0000 7f0d17c1d740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ee2f4b7400 0x55ee2f357080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.999+0000 7f0d17c1d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee2f4b7800 0x55ee2f357600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.999+0000 7f0d17c1d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee2f4b7c00 0x55ee2f357b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.999+0000 7f0d17c1d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ee2f35a300 con 0x55ee2f4b7800 2024-04-20T21:26:44.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.999+0000 7f0d17c1d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ee2f35a000 con 0x55ee2f4b7c00 2024-04-20T21:26:44.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.999+0000 7f0d17c1d740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55ee2e64a900 con 0x55ee2f4b7400 2024-04-20T21:26:44.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.999+0000 7f0d15200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee2f4b7800 0x55ee2f357600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.999+0000 7f0d15200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee2f4b7800 0x55ee2f357600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45634/0 (socket says 172.21.15.86:45634) 2024-04-20T21:26:44.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.999+0000 7f0d15200640 1 -- 172.21.15.86:0/917476657 learned_addr learned my addr 172.21.15.86:0/917476657 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:44.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.999+0000 7f0d15c00640 1 --2- 172.21.15.86:0/917476657 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ee2f4b7400 0x55ee2f357080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.999+0000 7f0d14800640 1 --2- 172.21.15.86:0/917476657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee2f4b7c00 0x55ee2f357b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.999+0000 7f0d15200640 1 -- 172.21.15.86:0/917476657 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ee2f4b7400 msgr2=0x55ee2f357080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:44.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.999+0000 7f0d15200640 1 --2- 172.21.15.86:0/917476657 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ee2f4b7400 0x55ee2f357080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.999+0000 7f0d15200640 1 -- 172.21.15.86:0/917476657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee2f4b7c00 msgr2=0x55ee2f357b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:44.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.999+0000 7f0d15200640 1 --2- 172.21.15.86:0/917476657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee2f4b7c00 0x55ee2f357b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.999+0000 7f0d15200640 1 -- 172.21.15.86:0/917476657 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee2e617860 con 0x55ee2f4b7800 2024-04-20T21:26:44.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.999+0000 7f0d15c00640 1 --2- 172.21.15.86:0/917476657 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ee2f4b7400 0x55ee2f357080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:26:44.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:43.999+0000 7f0d14800640 1 --2- 172.21.15.86:0/917476657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee2f4b7c00 0x55ee2f357b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:26:44.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.000+0000 7f0d15200640 1 --2- 172.21.15.86:0/917476657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee2f4b7800 0x55ee2f357600 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x55ee2f3d8c90 tx=0x55ee2f3cec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40ff541071d3c5e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:44.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.000+0000 7f0d0f800640 1 -- 172.21.15.86:0/917476657 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ee2e610680 con 0x55ee2f4b7800 2024-04-20T21:26:44.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.000+0000 7f0d0f800640 1 -- 172.21.15.86:0/917476657 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55ee2e634540 con 0x55ee2f4b7800 2024-04-20T21:26:44.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.000+0000 7f0d0f800640 1 -- 172.21.15.86:0/917476657 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ee2e610b60 con 0x55ee2f4b7800 2024-04-20T21:26:44.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.000+0000 7f0d17c1d740 1 -- 172.21.15.86:0/917476657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee2f4b7800 msgr2=0x55ee2f357600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:44.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.000+0000 7f0d17c1d740 1 --2- 172.21.15.86:0/917476657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee2f4b7800 0x55ee2f357600 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x55ee2f3d8c90 tx=0x55ee2f3cec00 comp rx=0 tx=0).stop 2024-04-20T21:26:44.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.000+0000 7f0d17c1d740 1 -- 172.21.15.86:0/917476657 shutdown_connections 2024-04-20T21:26:44.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.000+0000 7f0d17c1d740 1 --2- 172.21.15.86:0/917476657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee2f4b7c00 0x55ee2f357b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.000+0000 7f0d17c1d740 1 --2- 172.21.15.86:0/917476657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee2f4b7800 0x55ee2f357600 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.000+0000 7f0d17c1d740 1 --2- 172.21.15.86:0/917476657 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ee2f4b7400 0x55ee2f357080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.000+0000 7f0d17c1d740 1 -- 172.21.15.86:0/917476657 >> 172.21.15.86:0/917476657 conn(0x55ee2f4b7000 msgr2=0x55ee2e6d5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:44.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.000+0000 7f0d17c1d740 1 -- 172.21.15.86:0/917476657 shutdown_connections 2024-04-20T21:26:44.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.000+0000 7f0d17c1d740 1 -- 172.21.15.86:0/917476657 wait complete. 2024-04-20T21:26:44.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.001+0000 7f0d17c1d740 1 Processor -- start 2024-04-20T21:26:44.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.001+0000 7f0d17c1d740 1 -- start start 2024-04-20T21:26:44.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.002+0000 7f0d17c1d740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ee2f4b7400 0x55ee2f357080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.002+0000 7f0d17c1d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee2f4b7800 0x55ee2f357600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.002+0000 7f0d17c1d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee2f4b7c00 0x55ee2f357b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.002+0000 7f0d17c1d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ee2f35a300 con 0x55ee2f4b7800 2024-04-20T21:26:44.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.002+0000 7f0d17c1d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ee2f35a000 con 0x55ee2f4b7c00 2024-04-20T21:26:44.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.002+0000 7f0d17c1d740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55ee2f35b500 con 0x55ee2f4b7400 2024-04-20T21:26:44.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.002+0000 7f0d15200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee2f4b7800 0x55ee2f357600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.002+0000 7f0d15c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ee2f4b7400 0x55ee2f357080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.002+0000 7f0d15200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee2f4b7800 0x55ee2f357600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45644/0 (socket says 172.21.15.86:45644) 2024-04-20T21:26:44.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.002+0000 7f0d15200640 1 -- 172.21.15.86:0/993970898 learned_addr learned my addr 172.21.15.86:0/993970898 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:44.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.002+0000 7f0d14800640 1 --2- 172.21.15.86:0/993970898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee2f4b7c00 0x55ee2f357b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.003+0000 7f0d15200640 1 -- 172.21.15.86:0/993970898 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ee2f4b7400 msgr2=0x55ee2f357080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:44.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.003+0000 7f0d15200640 1 --2- 172.21.15.86:0/993970898 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ee2f4b7400 0x55ee2f357080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.003+0000 7f0d15200640 1 -- 172.21.15.86:0/993970898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee2f4b7c00 msgr2=0x55ee2f357b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:44.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.003+0000 7f0d15200640 1 --2- 172.21.15.86:0/993970898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee2f4b7c00 0x55ee2f357b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.003+0000 7f0d15200640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee2e617680 con 0x55ee2f4b7800 2024-04-20T21:26:44.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.003+0000 7f0d15c00640 1 --2- 172.21.15.86:0/993970898 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ee2f4b7400 0x55ee2f357080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:26:44.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.003+0000 7f0d15200640 1 --2- 172.21.15.86:0/993970898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee2f4b7800 0x55ee2f357600 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x55ee2f3d9f50 tx=0x55ee2f38efc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:44.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.003+0000 7f0d0da00640 1 -- 172.21.15.86:0/993970898 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ee2e6104e0 con 0x55ee2f4b7800 2024-04-20T21:26:44.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.003+0000 7f0d0da00640 1 -- 172.21.15.86:0/993970898 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55ee2e634c40 con 0x55ee2f4b7800 2024-04-20T21:26:44.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.003+0000 7f0d0da00640 1 -- 172.21.15.86:0/993970898 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ee2e610680 con 0x55ee2f4b7800 2024-04-20T21:26:44.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.003+0000 7f0d17c1d740 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ee2e6174a0 con 0x55ee2f4b7800 2024-04-20T21:26:44.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.004+0000 7f0d17c1d740 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee2e6172c0 con 0x55ee2f4b7800 2024-04-20T21:26:44.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.004+0000 7f0d17c1d740 4 datascan.init: resolving metadata pool 12 2024-04-20T21:26:44.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.005+0000 7f0d0da00640 1 -- 172.21.15.86:0/993970898 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x55ee2f4ecc00 con 0x55ee2f4b7800 2024-04-20T21:26:44.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.006+0000 7f0d0da00640 1 --2- 172.21.15.86:0/993970898 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x55ee2f538c00 0x55ee2f358100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.006+0000 7f0d0da00640 1 -- 172.21.15.86:0/993970898 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x55ee2f53aa00 con 0x55ee2f4b7800 2024-04-20T21:26:44.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.006+0000 7f0d17c1d740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-20T21:26:44.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.006+0000 7f0d17c1d740 4 datascan.main: data pool 'cephfs_data' has ID 13 2024-04-20T21:26:44.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.006+0000 7f0d17c1d740 4 datascan.main: opening data pool 'cephfs_data' 2024-04-20T21:26:44.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.006+0000 7f0d17c1d740 1 --2- 172.21.15.86:0/993970898 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x55ee2f539000 0x55ee2f358680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.006+0000 7f0d17c1d740 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:1 13.0 13:00000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2f539400 con 0x55ee2f539000 2024-04-20T21:26:44.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.006+0000 7f0d15c00640 1 --2- 172.21.15.86:0/993970898 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x55ee2f538c00 0x55ee2f358100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.006+0000 7f0d14800640 1 --2- 172.21.15.86:0/993970898 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x55ee2f539000 0x55ee2f358680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.007+0000 7f0d15c00640 1 --2- 172.21.15.86:0/993970898 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x55ee2f538c00 0x55ee2f358100 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x55ee2e6995c0 tx=0x55ee2f38ed80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:44.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.007+0000 7f0d14800640 1 --2- 172.21.15.86:0/993970898 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x55ee2f539000 0x55ee2f358680 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:44.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.010+0000 7f0d14800640 1 -- 172.21.15.86:0/993970898 <== osd.7 v2:172.21.15.144:6816/2100967057 1 ==== osd_op_reply(1 [pgnls start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378000 con 0x55ee2f539000 2024-04-20T21:26:44.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.010+0000 7f0d0ee00640 1 --2- 172.21.15.86:0/993970898 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x55ee2f3fa000 0x55ee2e982000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.010+0000 7f0d0ee00640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:2 13.20 13:04000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2f3fa400 con 0x55ee2f3fa000 2024-04-20T21:26:44.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.010+0000 7f0d15c00640 1 --2- 172.21.15.86:0/993970898 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x55ee2f3fa000 0x55ee2e982000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.011+0000 7f0d15c00640 1 --2- 172.21.15.86:0/993970898 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x55ee2f3fa000 0x55ee2e982000 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:44.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.011+0000 7f0d15c00640 1 -- 172.21.15.86:0/993970898 <== osd.6 v2:172.21.15.144:6824/1893803079 1 ==== osd_op_reply(2 [pgnls start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378500 con 0x55ee2f3fa000 2024-04-20T21:26:44.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.011+0000 7f0d0f800640 1 --2- 172.21.15.86:0/993970898 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x55ee2f3fac00 0x55ee2e982580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.011+0000 7f0d0f800640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:3 13.10 13:08000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2f3fb000 con 0x55ee2f3fac00 2024-04-20T21:26:44.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.012+0000 7f0d15200640 1 --2- 172.21.15.86:0/993970898 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x55ee2f3fac00 0x55ee2e982580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.012+0000 7f0d15200640 1 --2- 172.21.15.86:0/993970898 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x55ee2f3fac00 0x55ee2e982580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:44.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.013+0000 7f0d15200640 1 -- 172.21.15.86:0/993970898 <== osd.1 v2:172.21.15.86:6817/3149564961 1 ==== osd_op_reply(3 [pgnls start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378780 con 0x55ee2f3fac00 2024-04-20T21:26:44.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.013+0000 7f0d0ee00640 1 --2- 172.21.15.86:0/993970898 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x55ee2f3fa800 0x55ee2e982b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.013+0000 7f0d0ee00640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:4 13.30 13:0c000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2f3fb800 con 0x55ee2f3fa800 2024-04-20T21:26:44.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.013+0000 7f0d14800640 1 --2- 172.21.15.86:0/993970898 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x55ee2f3fa800 0x55ee2e982b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.014+0000 7f0d14800640 1 --2- 172.21.15.86:0/993970898 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x55ee2f3fa800 0x55ee2e982b00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:44.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.014+0000 7f0d14800640 1 -- 172.21.15.86:0/993970898 <== osd.4 v2:172.21.15.144:6800/4087309475 1 ==== osd_op_reply(4 [pgnls start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378000 con 0x55ee2f3fa800 2024-04-20T21:26:44.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.014+0000 7f0d0f800640 1 --2- 172.21.15.86:0/993970898 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x55ee2f3fb400 0x55ee2e983600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.014+0000 7f0d0f800640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:5 13.8 13:10000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2e996000 con 0x55ee2f3fb400 2024-04-20T21:26:44.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.014+0000 7f0d15c00640 1 --2- 172.21.15.86:0/993970898 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x55ee2f3fb400 0x55ee2e983600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.015+0000 7f0d15c00640 1 --2- 172.21.15.86:0/993970898 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x55ee2f3fb400 0x55ee2e983600 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:44.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.018+0000 7f0d15c00640 1 -- 172.21.15.86:0/993970898 <== osd.2 v2:172.21.15.86:6816/3437611468 1 ==== osd_op_reply(5 [pgnls start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378500 con 0x55ee2f3fb400 2024-04-20T21:26:44.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.018+0000 7f0d0ee00640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:6 13.28 13:14000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2f3fbc00 con 0x55ee2f3fb400 2024-04-20T21:26:44.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.018+0000 7f0d15c00640 1 -- 172.21.15.86:0/993970898 <== osd.2 v2:172.21.15.86:6816/3437611468 2 ==== osd_op_reply(6 [pgnls start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378500 con 0x55ee2f3fb400 2024-04-20T21:26:44.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.018+0000 7f0d0f800640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:7 13.18 13:18000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2e996400 con 0x55ee2f539000 2024-04-20T21:26:44.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.019+0000 7f0d14800640 1 -- 172.21.15.86:0/993970898 <== osd.7 v2:172.21.15.144:6816/2100967057 2 ==== osd_op_reply(7 [pgnls start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378000 con 0x55ee2f539000 2024-04-20T21:26:44.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.019+0000 7f0d0ee00640 1 --2- 172.21.15.86:0/993970898 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x55ee2e996c00 0x55ee2e983080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.019+0000 7f0d0ee00640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:8 13.38 13:1c000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2e997000 con 0x55ee2e996c00 2024-04-20T21:26:44.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.019+0000 7f0d15200640 1 --2- 172.21.15.86:0/993970898 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x55ee2e996c00 0x55ee2e983080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.020+0000 7f0d15200640 1 --2- 172.21.15.86:0/993970898 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x55ee2e996c00 0x55ee2e983080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:44.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.022+0000 7f0d15200640 1 -- 172.21.15.86:0/993970898 <== osd.5 v2:172.21.15.144:6808/1635974704 1 ==== osd_op_reply(8 [pgnls start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378780 con 0x55ee2e996c00 2024-04-20T21:26:44.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.022+0000 7f0d0f800640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:9 13.4 13:20000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2e996800 con 0x55ee2f3fa000 2024-04-20T21:26:44.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.023+0000 7f0d15c00640 1 -- 172.21.15.86:0/993970898 <== osd.6 v2:172.21.15.144:6824/1893803079 2 ==== osd_op_reply(9 [pgnls start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378500 con 0x55ee2f3fa000 2024-04-20T21:26:44.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.023+0000 7f0d0ee00640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:10 13.24 13:24000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2e997400 con 0x55ee2f3fa000 2024-04-20T21:26:44.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.023+0000 7f0d15c00640 1 -- 172.21.15.86:0/993970898 <== osd.6 v2:172.21.15.144:6824/1893803079 3 ==== osd_op_reply(10 [pgnls start_epoch 89 out=49b] v85'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378500 con 0x55ee2f3fa000 2024-04-20T21:26:44.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.023+0000 7f0d0f800640 1 --2- 172.21.15.86:0/993970898 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x55ee2e996000 0x55ee2e983b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.023+0000 7f0d0f800640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:11 13.14 13:28000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2f3fa400 con 0x55ee2e996000 2024-04-20T21:26:44.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.024+0000 7f0d14800640 1 --2- 172.21.15.86:0/993970898 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x55ee2e996000 0x55ee2e983b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.024+0000 7f0d14800640 1 --2- 172.21.15.86:0/993970898 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x55ee2e996000 0x55ee2e983b80 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:44.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.027+0000 7f0d14800640 1 -- 172.21.15.86:0/993970898 <== osd.0 v2:172.21.15.86:6800/2471076035 1 ==== osd_op_reply(11 [pgnls start_epoch 89 out=49b] v85'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378000 con 0x55ee2e996000 2024-04-20T21:26:44.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.027+0000 7f0d0ee00640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:12 13.34 13:2c000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2e997800 con 0x55ee2f3fa000 2024-04-20T21:26:44.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.028+0000 7f0d15c00640 1 -- 172.21.15.86:0/993970898 <== osd.6 v2:172.21.15.144:6824/1893803079 4 ==== osd_op_reply(12 [pgnls start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378500 con 0x55ee2f3fa000 2024-04-20T21:26:44.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.028+0000 7f0d0f800640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:13 13.c 13:30000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2f3fbc00 con 0x55ee2f3fac00 2024-04-20T21:26:44.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.028+0000 7f0d15200640 1 -- 172.21.15.86:0/993970898 <== osd.1 v2:172.21.15.86:6817/3149564961 2 ==== osd_op_reply(13 [pgnls start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378780 con 0x55ee2f3fac00 2024-04-20T21:26:44.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.028+0000 7f0d0ee00640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:14 13.2c 13:34000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2e996400 con 0x55ee2f3fb400 2024-04-20T21:26:44.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.029+0000 7f0d15c00640 1 -- 172.21.15.86:0/993970898 <== osd.2 v2:172.21.15.86:6816/3437611468 3 ==== osd_op_reply(14 [pgnls start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378500 con 0x55ee2f3fb400 2024-04-20T21:26:44.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.029+0000 7f0d0f800640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:15 13.1c 13:38000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2f389400 con 0x55ee2f3fa000 2024-04-20T21:26:44.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.029+0000 7f0d15c00640 1 -- 172.21.15.86:0/993970898 <== osd.6 v2:172.21.15.144:6824/1893803079 5 ==== osd_op_reply(15 [pgnls start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378500 con 0x55ee2f3fa000 2024-04-20T21:26:44.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.029+0000 7f0d0ee00640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:16 13.3c 13:3c000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2f3fb800 con 0x55ee2f3fb400 2024-04-20T21:26:44.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.030+0000 7f0d15c00640 1 -- 172.21.15.86:0/993970898 <== osd.2 v2:172.21.15.86:6816/3437611468 4 ==== osd_op_reply(16 [pgnls start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378500 con 0x55ee2f3fb400 2024-04-20T21:26:44.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.030+0000 7f0d0f800640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:17 13.2 13:40000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2e996800 con 0x55ee2e996c00 2024-04-20T21:26:44.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.030+0000 7f0d15200640 1 -- 172.21.15.86:0/993970898 <== osd.5 v2:172.21.15.144:6808/1635974704 2 ==== osd_op_reply(17 [pgnls start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378780 con 0x55ee2e996c00 2024-04-20T21:26:44.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.030+0000 7f0d0ee00640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:18 13.22 13:44000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2f3fa400 con 0x55ee2f3fac00 2024-04-20T21:26:44.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.031+0000 7f0d15200640 1 -- 172.21.15.86:0/993970898 <== osd.1 v2:172.21.15.86:6817/3149564961 3 ==== osd_op_reply(18 [pgnls start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378780 con 0x55ee2f3fac00 2024-04-20T21:26:44.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.031+0000 7f0d0f800640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:19 13.12 13:48000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2e997c00 con 0x55ee2e996000 2024-04-20T21:26:44.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.031+0000 7f0d14800640 1 -- 172.21.15.86:0/993970898 <== osd.0 v2:172.21.15.86:6800/2471076035 2 ==== osd_op_reply(19 [pgnls start_epoch 89 out=49b] v85'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378000 con 0x55ee2e996000 2024-04-20T21:26:44.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.031+0000 7f0d0ee00640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:20 13.32 13:4c000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2f539400 con 0x55ee2f3fb400 2024-04-20T21:26:44.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.031+0000 7f0d15c00640 1 -- 172.21.15.86:0/993970898 <== osd.2 v2:172.21.15.86:6816/3437611468 5 ==== osd_op_reply(20 [pgnls start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378500 con 0x55ee2f3fb400 2024-04-20T21:26:44.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.031+0000 7f0d0f800640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:21 13.a 13:50000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2e997400 con 0x55ee2f3fb400 2024-04-20T21:26:44.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.032+0000 7f0d15c00640 1 -- 172.21.15.86:0/993970898 <== osd.2 v2:172.21.15.86:6816/3437611468 6 ==== osd_op_reply(21 [pgnls start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378500 con 0x55ee2f3fb400 2024-04-20T21:26:44.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.032+0000 7f0d0ee00640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:22 13.2a 13:54000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2f4b6800 con 0x55ee2f3fac00 2024-04-20T21:26:44.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.032+0000 7f0d15200640 1 -- 172.21.15.86:0/993970898 <== osd.1 v2:172.21.15.86:6817/3149564961 4 ==== osd_op_reply(22 [pgnls start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378780 con 0x55ee2f3fac00 2024-04-20T21:26:44.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.032+0000 7f0d0f800640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:23 13.1a 13:58000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2f3fb800 con 0x55ee2f3fac00 2024-04-20T21:26:44.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.032+0000 7f0d15200640 1 -- 172.21.15.86:0/993970898 <== osd.1 v2:172.21.15.86:6817/3149564961 5 ==== osd_op_reply(23 [pgnls start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378780 con 0x55ee2f3fac00 2024-04-20T21:26:44.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.033+0000 7f0d0ee00640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:24 13.3a 13:5c000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2e997800 con 0x55ee2e996c00 2024-04-20T21:26:44.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.033+0000 7f0d15200640 1 -- 172.21.15.86:0/993970898 <== osd.5 v2:172.21.15.144:6808/1635974704 3 ==== osd_op_reply(24 [pgnls start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378780 con 0x55ee2e996c00 2024-04-20T21:26:44.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.033+0000 7f0d0f800640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:25 13.6 13:60000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2f3fb000 con 0x55ee2f3fa800 2024-04-20T21:26:44.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.034+0000 7f0d14800640 1 -- 172.21.15.86:0/993970898 <== osd.4 v2:172.21.15.144:6800/4087309475 2 ==== osd_op_reply(25 [pgnls start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378000 con 0x55ee2f3fa800 2024-04-20T21:26:44.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.034+0000 7f0d0ee00640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:26 13.26 13:64000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2e996400 con 0x55ee2f539000 2024-04-20T21:26:44.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.035+0000 7f0d14800640 1 -- 172.21.15.86:0/993970898 <== osd.7 v2:172.21.15.144:6816/2100967057 3 ==== osd_op_reply(26 [pgnls start_epoch 89 out=49b] v85'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378000 con 0x55ee2f539000 2024-04-20T21:26:44.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.035+0000 7f0d0f800640 1 --2- 172.21.15.86:0/993970898 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x55ee2f3fbc00 0x55ee2e984100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.035+0000 7f0d0f800640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:27 13.16 13:68000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2f539400 con 0x55ee2f3fbc00 2024-04-20T21:26:44.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.035+0000 7f0d15c00640 1 --2- 172.21.15.86:0/993970898 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x55ee2f3fbc00 0x55ee2e984100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.036+0000 7f0d15c00640 1 --2- 172.21.15.86:0/993970898 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x55ee2f3fbc00 0x55ee2e984100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:44.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.039+0000 7f0d15c00640 1 -- 172.21.15.86:0/993970898 <== osd.3 v2:172.21.15.86:6808/794135519 1 ==== osd_op_reply(27 [pgnls start_epoch 89 out=49b] v85'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378500 con 0x55ee2f3fbc00 2024-04-20T21:26:44.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.039+0000 7f0d0ee00640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:28 13.36 13:6c000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2e997000 con 0x55ee2f3fa000 2024-04-20T21:26:44.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.040+0000 7f0d15c00640 1 -- 172.21.15.86:0/993970898 <== osd.6 v2:172.21.15.144:6824/1893803079 6 ==== osd_op_reply(28 [pgnls start_epoch 89 out=49b] v85'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378500 con 0x55ee2f3fa000 2024-04-20T21:26:44.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.040+0000 7f0d0f800640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:29 13.e 13:70000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2f389400 con 0x55ee2f539000 2024-04-20T21:26:44.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.040+0000 7f0d14800640 1 -- 172.21.15.86:0/993970898 <== osd.7 v2:172.21.15.144:6816/2100967057 4 ==== osd_op_reply(29 [pgnls start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378000 con 0x55ee2f539000 2024-04-20T21:26:44.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.040+0000 7f0d0ee00640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:30 13.2e 13:74000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2e996800 con 0x55ee2f3fb400 2024-04-20T21:26:44.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.041+0000 7f0d15c00640 1 -- 172.21.15.86:0/993970898 <== osd.2 v2:172.21.15.86:6816/3437611468 7 ==== osd_op_reply(30 [pgnls start_epoch 89 out=49b] v85'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378500 con 0x55ee2f3fb400 2024-04-20T21:26:44.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.041+0000 7f0d0f800640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:31 13.1e 13:78000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2e9ac000 con 0x55ee2f3fa800 2024-04-20T21:26:44.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.042+0000 7f0d14800640 1 -- 172.21.15.86:0/993970898 <== osd.4 v2:172.21.15.144:6800/4087309475 3 ==== osd_op_reply(31 [pgnls start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378000 con 0x55ee2f3fa800 2024-04-20T21:26:44.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.042+0000 7f0d0ee00640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:32 13.3e 13:7c000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2e9adc00 con 0x55ee2f3fbc00 2024-04-20T21:26:44.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.042+0000 7f0d15c00640 1 -- 172.21.15.86:0/993970898 <== osd.3 v2:172.21.15.86:6808/794135519 2 ==== osd_op_reply(32 [pgnls start_epoch 89 out=49b] v85'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378500 con 0x55ee2f3fbc00 2024-04-20T21:26:44.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.042+0000 7f0d0f800640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:33 13.1 13:80000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2e9ac400 con 0x55ee2f3fa000 2024-04-20T21:26:44.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.043+0000 7f0d15c00640 1 -- 172.21.15.86:0/993970898 <== osd.6 v2:172.21.15.144:6824/1893803079 7 ==== osd_op_reply(33 [pgnls start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378500 con 0x55ee2f3fa000 2024-04-20T21:26:44.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.043+0000 7f0d0ee00640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:34 13.21 13:84000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2e9ae000 con 0x55ee2f3fa800 2024-04-20T21:26:44.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.045+0000 7f0d14800640 1 -- 172.21.15.86:0/993970898 <== osd.4 v2:172.21.15.144:6800/4087309475 4 ==== osd_op_reply(34 [pgnls start_epoch 89 out=49b] v85'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378000 con 0x55ee2f3fa800 2024-04-20T21:26:44.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.045+0000 7f0d0f800640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:35 13.11 13:88000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2e9ac800 con 0x55ee2f3fac00 2024-04-20T21:26:44.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.045+0000 7f0d15200640 1 -- 172.21.15.86:0/993970898 <== osd.1 v2:172.21.15.86:6817/3149564961 6 ==== osd_op_reply(35 [pgnls start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378780 con 0x55ee2f3fac00 2024-04-20T21:26:44.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.045+0000 7f0d0ee00640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:36 13.31 13:8c000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2e9ae400 con 0x55ee2f3fbc00 2024-04-20T21:26:44.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.046+0000 7f0d15c00640 1 -- 172.21.15.86:0/993970898 <== osd.3 v2:172.21.15.86:6808/794135519 3 ==== osd_op_reply(36 [pgnls start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378500 con 0x55ee2f3fbc00 2024-04-20T21:26:44.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.046+0000 7f0d0f800640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:37 13.9 13:90000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2e9acc00 con 0x55ee2e996c00 2024-04-20T21:26:44.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.047+0000 7f0d15200640 1 -- 172.21.15.86:0/993970898 <== osd.5 v2:172.21.15.144:6808/1635974704 4 ==== osd_op_reply(37 [pgnls start_epoch 89 out=49b] v85'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378780 con 0x55ee2e996c00 2024-04-20T21:26:44.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.047+0000 7f0d0ee00640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:38 13.29 13:94000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2e9ae800 con 0x55ee2f3fa000 2024-04-20T21:26:44.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.048+0000 7f0d15c00640 1 -- 172.21.15.86:0/993970898 <== osd.6 v2:172.21.15.144:6824/1893803079 8 ==== osd_op_reply(38 [pgnls start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378500 con 0x55ee2f3fa000 2024-04-20T21:26:44.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.048+0000 7f0d0f800640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:39 13.19 13:98000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2e9ad000 con 0x55ee2f3fbc00 2024-04-20T21:26:44.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.048+0000 7f0d15c00640 1 -- 172.21.15.86:0/993970898 <== osd.3 v2:172.21.15.86:6808/794135519 4 ==== osd_op_reply(39 [pgnls start_epoch 89 out=49b] v85'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378500 con 0x55ee2f3fbc00 2024-04-20T21:26:44.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.048+0000 7f0d0ee00640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:40 13.39 13:9c000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2e9aec00 con 0x55ee2f3fb400 2024-04-20T21:26:44.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.049+0000 7f0d15c00640 1 -- 172.21.15.86:0/993970898 <== osd.2 v2:172.21.15.86:6816/3437611468 8 ==== osd_op_reply(40 [pgnls start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378500 con 0x55ee2f3fb400 2024-04-20T21:26:44.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.049+0000 7f0d0f800640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:41 13.5 13:a0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2e9ad400 con 0x55ee2f3fa800 2024-04-20T21:26:44.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.050+0000 7f0d14800640 1 -- 172.21.15.86:0/993970898 <== osd.4 v2:172.21.15.144:6800/4087309475 5 ==== osd_op_reply(41 [pgnls start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378000 con 0x55ee2f3fa800 2024-04-20T21:26:44.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.050+0000 7f0d0ee00640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:42 13.25 13:a4000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2e9af000 con 0x55ee2f3fa800 2024-04-20T21:26:44.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.051+0000 7f0d14800640 1 -- 172.21.15.86:0/993970898 <== osd.4 v2:172.21.15.144:6800/4087309475 6 ==== osd_op_reply(42 [pgnls start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378000 con 0x55ee2f3fa800 2024-04-20T21:26:44.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.051+0000 7f0d0f800640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:43 13.15 13:a8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2e9ad800 con 0x55ee2f3fa800 2024-04-20T21:26:44.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.052+0000 7f0d14800640 1 -- 172.21.15.86:0/993970898 <== osd.4 v2:172.21.15.144:6800/4087309475 7 ==== osd_op_reply(43 [pgnls start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378000 con 0x55ee2f3fa800 2024-04-20T21:26:44.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.052+0000 7f0d0ee00640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:44 13.35 13:ac000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2e9af400 con 0x55ee2f3fac00 2024-04-20T21:26:44.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.053+0000 7f0d15200640 1 -- 172.21.15.86:0/993970898 <== osd.1 v2:172.21.15.86:6817/3149564961 7 ==== osd_op_reply(44 [pgnls start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378780 con 0x55ee2f3fac00 2024-04-20T21:26:44.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.053+0000 7f0d0f800640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:45 13.d 13:b0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2e997400 con 0x55ee2f3fa800 2024-04-20T21:26:44.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.054+0000 7f0d14800640 1 -- 172.21.15.86:0/993970898 <== osd.4 v2:172.21.15.144:6800/4087309475 8 ==== osd_op_reply(45 [pgnls start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378000 con 0x55ee2f3fa800 2024-04-20T21:26:44.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.054+0000 7f0d0ee00640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:46 13.2d 13:b4000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2e9af800 con 0x55ee2e996000 2024-04-20T21:26:44.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.054+0000 7f0d14800640 1 -- 172.21.15.86:0/993970898 <== osd.0 v2:172.21.15.86:6800/2471076035 3 ==== osd_op_reply(46 [pgnls start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378000 con 0x55ee2e996000 2024-04-20T21:26:44.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.054+0000 7f0d0f800640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:47 13.1d 13:b8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2e997000 con 0x55ee2f539000 2024-04-20T21:26:44.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.055+0000 7f0d14800640 1 -- 172.21.15.86:0/993970898 <== osd.7 v2:172.21.15.144:6816/2100967057 5 ==== osd_op_reply(47 [pgnls start_epoch 89 out=49b] v85'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378000 con 0x55ee2f539000 2024-04-20T21:26:44.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.056+0000 7f0d0ee00640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:48 13.3d 13:bc000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2e9afc00 con 0x55ee2f539000 2024-04-20T21:26:44.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.057+0000 7f0d14800640 1 -- 172.21.15.86:0/993970898 <== osd.7 v2:172.21.15.144:6816/2100967057 6 ==== osd_op_reply(48 [pgnls start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378000 con 0x55ee2f539000 2024-04-20T21:26:44.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.057+0000 7f0d0f800640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:49 13.3 13:c0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2e996800 con 0x55ee2e996000 2024-04-20T21:26:44.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.057+0000 7f0d14800640 1 -- 172.21.15.86:0/993970898 <== osd.0 v2:172.21.15.86:6800/2471076035 4 ==== osd_op_reply(49 [pgnls start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378000 con 0x55ee2e996000 2024-04-20T21:26:44.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.057+0000 7f0d0ee00640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:50 13.23 13:c4000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2e9ad400 con 0x55ee2f3fac00 2024-04-20T21:26:44.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.058+0000 7f0d15200640 1 -- 172.21.15.86:0/993970898 <== osd.1 v2:172.21.15.86:6817/3149564961 8 ==== osd_op_reply(50 [pgnls start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378780 con 0x55ee2f3fac00 2024-04-20T21:26:44.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.058+0000 7f0d0f800640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:51 13.13 13:c8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2f3fa400 con 0x55ee2f539000 2024-04-20T21:26:44.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.059+0000 7f0d14800640 1 -- 172.21.15.86:0/993970898 <== osd.7 v2:172.21.15.144:6816/2100967057 7 ==== osd_op_reply(51 [pgnls start_epoch 89 out=49b] v85'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378000 con 0x55ee2f539000 2024-04-20T21:26:44.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.059+0000 7f0d0ee00640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:52 13.33 13:cc000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2e9ad800 con 0x55ee2f539000 2024-04-20T21:26:44.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.060+0000 7f0d14800640 1 -- 172.21.15.86:0/993970898 <== osd.7 v2:172.21.15.144:6816/2100967057 8 ==== osd_op_reply(52 [pgnls start_epoch 89 out=49b] v85'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378000 con 0x55ee2f539000 2024-04-20T21:26:44.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.060+0000 7f0d0f800640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:53 13.b 13:d0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2f3fb800 con 0x55ee2e996c00 2024-04-20T21:26:44.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.062+0000 7f0d15200640 1 -- 172.21.15.86:0/993970898 <== osd.5 v2:172.21.15.144:6808/1635974704 5 ==== osd_op_reply(53 [pgnls start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378780 con 0x55ee2e996c00 2024-04-20T21:26:44.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.062+0000 7f0d0ee00640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:54 13.2b 13:d4000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2e9adc00 con 0x55ee2f3fb400 2024-04-20T21:26:44.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.062+0000 7f0d15c00640 1 -- 172.21.15.86:0/993970898 <== osd.2 v2:172.21.15.86:6816/3437611468 9 ==== osd_op_reply(54 [pgnls start_epoch 89 out=49b] v85'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378500 con 0x55ee2f3fb400 2024-04-20T21:26:44.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.063+0000 7f0d0f800640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:55 13.1b 13:d8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2f3fb000 con 0x55ee2f3fa800 2024-04-20T21:26:44.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.064+0000 7f0d14800640 1 -- 172.21.15.86:0/993970898 <== osd.4 v2:172.21.15.144:6800/4087309475 9 ==== osd_op_reply(55 [pgnls start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378000 con 0x55ee2f3fa800 2024-04-20T21:26:44.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.064+0000 7f0d0ee00640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:56 13.3b 13:dc000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2f389400 con 0x55ee2f3fa800 2024-04-20T21:26:44.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.065+0000 7f0d14800640 1 -- 172.21.15.86:0/993970898 <== osd.4 v2:172.21.15.144:6800/4087309475 10 ==== osd_op_reply(56 [pgnls start_epoch 89 out=49b] v85'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378000 con 0x55ee2f3fa800 2024-04-20T21:26:44.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.065+0000 7f0d0f800640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:57 13.7 13:e0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2f4b6800 con 0x55ee2f539000 2024-04-20T21:26:44.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.066+0000 7f0d14800640 1 -- 172.21.15.86:0/993970898 <== osd.7 v2:172.21.15.144:6816/2100967057 9 ==== osd_op_reply(57 [pgnls start_epoch 89 out=81b] v85'6 uv6 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55ee2f378000 con 0x55ee2f539000 2024-04-20T21:26:44.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.066+0000 7f0d17c1d740 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:58 13.7 13:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x55ee2f539800 con 0x55ee2f539000 2024-04-20T21:26:44.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.067+0000 7f0d14800640 1 -- 172.21.15.86:0/993970898 <== osd.7 v2:172.21.15.144:6816/2100967057 10 ==== osd_op_reply(58 10000000001.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x55ee2f378000 con 0x55ee2f539000 2024-04-20T21:26:44.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.067+0000 7f0d17c1d740 10 datascan.operator(): handling object 1099511627777.0 2024-04-20T21:26:44.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.067+0000 7f0d17c1d740 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:59 13.7 13:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89) -- 0x55ee2f539c00 con 0x55ee2f539000 2024-04-20T21:26:44.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.072+0000 7f0d14800640 1 -- 172.21.15.86:0/993970898 <== osd.7 v2:172.21.15.144:6816/2100967057 11 ==== osd_op_reply(59 10000000001.00000000 [call] v89'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55ee2f378000 con 0x55ee2f539000 2024-04-20T21:26:44.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.072+0000 7f0d17c1d740 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:60 13.27 13:e4000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2f539800 con 0x55ee2f3fac00 2024-04-20T21:26:44.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.073+0000 7f0d15200640 1 -- 172.21.15.86:0/993970898 <== osd.1 v2:172.21.15.86:6817/3149564961 9 ==== osd_op_reply(60 [pgnls start_epoch 89 out=49b] v85'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378780 con 0x55ee2f3fac00 2024-04-20T21:26:44.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.073+0000 7f0d0f800640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:61 13.17 13:e8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2f539400 con 0x55ee2e996c00 2024-04-20T21:26:44.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.074+0000 7f0d15200640 1 -- 172.21.15.86:0/993970898 <== osd.5 v2:172.21.15.144:6808/1635974704 6 ==== osd_op_reply(61 [pgnls start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378780 con 0x55ee2e996c00 2024-04-20T21:26:44.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.074+0000 7f0d0ee00640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:62 13.37 13:ec000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2e997400 con 0x55ee2e996c00 2024-04-20T21:26:44.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.075+0000 7f0d15200640 1 -- 172.21.15.86:0/993970898 <== osd.5 v2:172.21.15.144:6808/1635974704 7 ==== osd_op_reply(62 [pgnls start_epoch 89 out=49b] v85'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378780 con 0x55ee2e996c00 2024-04-20T21:26:44.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.075+0000 7f0d0f800640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:63 13.f 13:f0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2e9acc00 con 0x55ee2f3fa000 2024-04-20T21:26:44.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.076+0000 7f0d15c00640 1 -- 172.21.15.86:0/993970898 <== osd.6 v2:172.21.15.144:6824/1893803079 9 ==== osd_op_reply(63 [pgnls start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378500 con 0x55ee2f3fa000 2024-04-20T21:26:44.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.076+0000 7f0d0ee00640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:64 13.2f 13:f4000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2e997000 con 0x55ee2f3fa800 2024-04-20T21:26:44.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.077+0000 7f0d14800640 1 -- 172.21.15.86:0/993970898 <== osd.4 v2:172.21.15.144:6800/4087309475 11 ==== osd_op_reply(64 [pgnls start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378000 con 0x55ee2f3fa800 2024-04-20T21:26:44.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.077+0000 7f0d0f800640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:65 13.1f 13:f8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2e9ad400 con 0x55ee2f3fa000 2024-04-20T21:26:44.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.078+0000 7f0d15c00640 1 -- 172.21.15.86:0/993970898 <== osd.6 v2:172.21.15.144:6824/1893803079 10 ==== osd_op_reply(65 [pgnls start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378500 con 0x55ee2f3fa000 2024-04-20T21:26:44.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.078+0000 7f0d0ee00640 1 -- 172.21.15.86:0/993970898 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:66 13.3f 13:fc000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55ee2e997800 con 0x55ee2f3fa000 2024-04-20T21:26:44.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.079+0000 7f0d15c00640 1 -- 172.21.15.86:0/993970898 <== osd.6 v2:172.21.15.144:6824/1893803079 11 ==== osd_op_reply(66 [pgnls start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ee2f378500 con 0x55ee2f3fa000 2024-04-20T21:26:44.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.079+0000 7f0d17c1d740 1 -- 172.21.15.86:0/993970898 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x55ee2e996000 msgr2=0x55ee2e983b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:44.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.079+0000 7f0d17c1d740 1 --2- 172.21.15.86:0/993970898 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x55ee2e996000 0x55ee2e983b80 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.079+0000 7f0d17c1d740 1 -- 172.21.15.86:0/993970898 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x55ee2f3fac00 msgr2=0x55ee2e982580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:44.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.079+0000 7f0d17c1d740 1 --2- 172.21.15.86:0/993970898 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x55ee2f3fac00 0x55ee2e982580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.079+0000 7f0d17c1d740 1 -- 172.21.15.86:0/993970898 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x55ee2f3fb400 msgr2=0x55ee2e983600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:44.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.079+0000 7f0d17c1d740 1 --2- 172.21.15.86:0/993970898 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x55ee2f3fb400 0x55ee2e983600 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.079+0000 7f0d17c1d740 1 -- 172.21.15.86:0/993970898 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x55ee2f3fbc00 msgr2=0x55ee2e984100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:44.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.079+0000 7f0d17c1d740 1 --2- 172.21.15.86:0/993970898 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x55ee2f3fbc00 0x55ee2e984100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.079+0000 7f0d17c1d740 1 -- 172.21.15.86:0/993970898 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x55ee2f3fa800 msgr2=0x55ee2e982b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:44.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.079+0000 7f0d17c1d740 1 --2- 172.21.15.86:0/993970898 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x55ee2f3fa800 0x55ee2e982b00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.079+0000 7f0d17c1d740 1 -- 172.21.15.86:0/993970898 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x55ee2e996c00 msgr2=0x55ee2e983080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:44.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.079+0000 7f0d17c1d740 1 --2- 172.21.15.86:0/993970898 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x55ee2e996c00 0x55ee2e983080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.079+0000 7f0d17c1d740 1 -- 172.21.15.86:0/993970898 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x55ee2f3fa000 msgr2=0x55ee2e982000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:44.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.079+0000 7f0d17c1d740 1 --2- 172.21.15.86:0/993970898 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x55ee2f3fa000 0x55ee2e982000 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.079+0000 7f0d17c1d740 1 -- 172.21.15.86:0/993970898 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x55ee2f539000 msgr2=0x55ee2f358680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:44.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.079+0000 7f0d17c1d740 1 --2- 172.21.15.86:0/993970898 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x55ee2f539000 0x55ee2f358680 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.079+0000 7f0d15200640 1 -- 172.21.15.86:0/993970898 reap_dead start 2024-04-20T21:26:44.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.079+0000 7f0d15200640 1 -- 172.21.15.86:0/993970898 reap_dead start 2024-04-20T21:26:44.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.079+0000 7f0d15200640 1 -- 172.21.15.86:0/993970898 reap_dead start 2024-04-20T21:26:44.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.080+0000 7f0d15200640 1 -- 172.21.15.86:0/993970898 reap_dead start 2024-04-20T21:26:44.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.080+0000 7f0d17c1d740 1 -- 172.21.15.86:0/993970898 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x55ee2f538c00 msgr2=0x55ee2f358100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:44.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.080+0000 7f0d17c1d740 1 --2- 172.21.15.86:0/993970898 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x55ee2f538c00 0x55ee2f358100 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x55ee2e6995c0 tx=0x55ee2f38ed80 comp rx=0 tx=0).stop 2024-04-20T21:26:44.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.080+0000 7f0d17c1d740 1 -- 172.21.15.86:0/993970898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee2f4b7800 msgr2=0x55ee2f357600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:44.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.080+0000 7f0d17c1d740 1 --2- 172.21.15.86:0/993970898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee2f4b7800 0x55ee2f357600 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x55ee2f3d9f50 tx=0x55ee2f38efc0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.080+0000 7f0d17c1d740 1 -- 172.21.15.86:0/993970898 shutdown_connections 2024-04-20T21:26:44.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.080+0000 7f0d17c1d740 1 --2- 172.21.15.86:0/993970898 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x55ee2f538c00 0x55ee2f358100 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.080+0000 7f0d17c1d740 1 --2- 172.21.15.86:0/993970898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee2f4b7800 0x55ee2f357600 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.080+0000 7f0d17c1d740 1 -- 172.21.15.86:0/993970898 >> 172.21.15.86:0/993970898 conn(0x55ee2f4b7000 msgr2=0x55ee2e6d5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:44.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.080+0000 7f0d17c1d740 1 -- 172.21.15.86:0/993970898 shutdown_connections 2024-04-20T21:26:44.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.081+0000 7f0d17c1d740 1 -- 172.21.15.86:0/993970898 wait complete. 2024-04-20T21:26:44.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.081+0000 7f0d17c1d740 1 -- 172.21.15.86:0/3822797471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee2e5eb800 msgr2=0x55ee2f356580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:44.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.081+0000 7f0d17c1d740 1 --2- 172.21.15.86:0/3822797471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee2e5eb800 0x55ee2f356580 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x55ee2f417860 tx=0x55ee2f38f020 comp rx=0 tx=0).stop 2024-04-20T21:26:44.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.082+0000 7f0d17c1d740 1 -- 172.21.15.86:0/3822797471 shutdown_connections 2024-04-20T21:26:44.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.082+0000 7f0d17c1d740 1 --2- 172.21.15.86:0/3822797471 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ee2e5ebc00 0x55ee2f356b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.082+0000 7f0d17c1d740 1 --2- 172.21.15.86:0/3822797471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ee2e5eb800 0x55ee2f356580 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.082+0000 7f0d17c1d740 1 --2- 172.21.15.86:0/3822797471 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ee2e5eac00 0x55ee2f356000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.082+0000 7f0d17c1d740 1 -- 172.21.15.86:0/3822797471 >> 172.21.15.86:0/3822797471 conn(0x55ee2e5eb000 msgr2=0x55ee2e6d5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:44.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.082+0000 7f0d17c1d740 1 -- 172.21.15.86:0/3822797471 shutdown_connections 2024-04-20T21:26:44.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.082+0000 7f0d17c1d740 1 -- 172.21.15.86:0/3822797471 wait complete. 2024-04-20T21:26:44.088 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.193260, result: 2024-04-20T21:26:44.089 DEBUG:teuthology.orchestra.run.smithi086:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes cephfs_data 2024-04-20T21:26:44.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.189+0000 7f8e76d4c740 1 -- start start 2024-04-20T21:26:44.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.189+0000 7f8e76d4c740 1 Processor -- start 2024-04-20T21:26:44.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.190+0000 7f8e76d4c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b931830000 0x55b931866580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.190+0000 7f8e76d4c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b931831c00 0x55b931866000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.190+0000 7f8e76d4c740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b93195c000 0x55b931867080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.190+0000 7f8e76d4c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b931962000 con 0x55b931830000 2024-04-20T21:26:44.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.190+0000 7f8e76d4c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b931962180 con 0x55b931831c00 2024-04-20T21:26:44.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.190+0000 7f8e76d4c740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55b931962300 con 0x55b93195c000 2024-04-20T21:26:44.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.190+0000 7f8e74e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b931830000 0x55b931866580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.191+0000 7f8e74e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b931830000 0x55b931866580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45666/0 (socket says 172.21.15.86:45666) 2024-04-20T21:26:44.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.191+0000 7f8e74e00640 1 -- 172.21.15.86:0/3213281158 learned_addr learned my addr 172.21.15.86:0/3213281158 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:44.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.191+0000 7f8e74e00640 1 -- 172.21.15.86:0/3213281158 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b93195c000 msgr2=0x55b931867080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:44.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.191+0000 7f8e74e00640 1 --2- 172.21.15.86:0/3213281158 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b93195c000 0x55b931867080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.191+0000 7f8e74e00640 1 -- 172.21.15.86:0/3213281158 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b931831c00 msgr2=0x55b931866000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:44.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.191+0000 7f8e74e00640 1 --2- 172.21.15.86:0/3213281158 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b931831c00 0x55b931866000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.191+0000 7f8e74e00640 1 -- 172.21.15.86:0/3213281158 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b930abf680 con 0x55b931830000 2024-04-20T21:26:44.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.191+0000 7f8e74e00640 1 --2- 172.21.15.86:0/3213281158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b931830000 0x55b931866580 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x55b9318b8f60 tx=0x55b9318328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46f1cd653e015d90 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:44.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.191+0000 7f8e71c00640 1 -- 172.21.15.86:0/3213281158 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b930ab8b60 con 0x55b931830000 2024-04-20T21:26:44.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.192+0000 7f8e71c00640 1 -- 172.21.15.86:0/3213281158 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55b930adc540 con 0x55b931830000 2024-04-20T21:26:44.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.192+0000 7f8e71c00640 1 -- 172.21.15.86:0/3213281158 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b930ab8d00 con 0x55b931830000 2024-04-20T21:26:44.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.192+0000 7f8e76d4c740 1 -- 172.21.15.86:0/3213281158 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b930abf2c0 con 0x55b931830000 2024-04-20T21:26:44.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.192+0000 7f8e76d4c740 1 -- 172.21.15.86:0/3213281158 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55b930abeb40 con 0x55b931830000 2024-04-20T21:26:44.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.192+0000 7f8e76d4c740 4 waiting for MDS map... 2024-04-20T21:26:44.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.192+0000 7f8e71c00640 1 -- 172.21.15.86:0/3213281158 <== mon.0 v2:172.21.15.86:3300/0 4 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x55b93196a400 con 0x55b931830000 2024-04-20T21:26:44.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.192+0000 7f8e71c00640 1 -- 172.21.15.86:0/3213281158 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 67) ==== 2886+0+0 (secure 0 0 0) 0x55b930aeb180 con 0x55b931830000 2024-04-20T21:26:44.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.192+0000 7f8e76d4c740 4 Got MDS map 67 2024-04-20T21:26:44.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.192+0000 7f8e76d4c740 4 datascan.main: Using metadata pool output 2024-04-20T21:26:44.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.192+0000 7f8e76d4c740 4 datascan.main: connecting to RADOS... 2024-04-20T21:26:44.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.193+0000 7f8e76d4c740 1 Processor -- start 2024-04-20T21:26:44.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.193+0000 7f8e76d4c740 1 -- start start 2024-04-20T21:26:44.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.194+0000 7f8e76d4c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93195c800 0x55b931867600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.194+0000 7f8e76d4c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b93195cc00 0x55b931867b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.194+0000 7f8e76d4c740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b93195d000 0x55b931868100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.194+0000 7f8e76d4c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b931962480 con 0x55b93195c800 2024-04-20T21:26:44.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.194+0000 7f8e76d4c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b931962600 con 0x55b93195cc00 2024-04-20T21:26:44.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.194+0000 7f8e76d4c740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55b931962780 con 0x55b93195d000 2024-04-20T21:26:44.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.194+0000 7f8e75800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93195c800 0x55b931867600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.194+0000 7f8e75800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93195c800 0x55b931867600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45672/0 (socket says 172.21.15.86:45672) 2024-04-20T21:26:44.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.194+0000 7f8e75800640 1 -- 172.21.15.86:0/2920018645 learned_addr learned my addr 172.21.15.86:0/2920018645 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:44.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.195+0000 7f8e74e00640 1 --2- 172.21.15.86:0/2920018645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b93195cc00 0x55b931867b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.195+0000 7f8e75800640 1 -- 172.21.15.86:0/2920018645 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b93195d000 msgr2=0x55b931868100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:44.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.195+0000 7f8e74400640 1 --2- 172.21.15.86:0/2920018645 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b93195d000 0x55b931868100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.195+0000 7f8e75800640 1 --2- 172.21.15.86:0/2920018645 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b93195d000 0x55b931868100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.195+0000 7f8e75800640 1 -- 172.21.15.86:0/2920018645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b93195cc00 msgr2=0x55b931867b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:44.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.195+0000 7f8e75800640 1 --2- 172.21.15.86:0/2920018645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b93195cc00 0x55b931867b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.195+0000 7f8e75800640 1 -- 172.21.15.86:0/2920018645 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b930abe3c0 con 0x55b93195c800 2024-04-20T21:26:44.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.195+0000 7f8e74e00640 1 --2- 172.21.15.86:0/2920018645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b93195cc00 0x55b931867b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:26:44.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.195+0000 7f8e75800640 1 --2- 172.21.15.86:0/2920018645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93195c800 0x55b931867600 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x55b931976f00 tx=0x55b93197ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36c1952fe34d6b57 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:44.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.195+0000 7f8e6f400640 1 -- 172.21.15.86:0/2920018645 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b930ab8b60 con 0x55b93195c800 2024-04-20T21:26:44.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.195+0000 7f8e6f400640 1 -- 172.21.15.86:0/2920018645 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55b930adcc40 con 0x55b93195c800 2024-04-20T21:26:44.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.195+0000 7f8e6f400640 1 -- 172.21.15.86:0/2920018645 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b930ab8d00 con 0x55b93195c800 2024-04-20T21:26:44.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.196+0000 7f8e76d4c740 1 -- 172.21.15.86:0/2920018645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93195c800 msgr2=0x55b931867600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:44.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.196+0000 7f8e76d4c740 1 --2- 172.21.15.86:0/2920018645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93195c800 0x55b931867600 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x55b931976f00 tx=0x55b93197ccc0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.196+0000 7f8e76d4c740 1 -- 172.21.15.86:0/2920018645 shutdown_connections 2024-04-20T21:26:44.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.196+0000 7f8e76d4c740 1 --2- 172.21.15.86:0/2920018645 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b93195d000 0x55b931868100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.196+0000 7f8e76d4c740 1 --2- 172.21.15.86:0/2920018645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b93195cc00 0x55b931867b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.196+0000 7f8e76d4c740 1 --2- 172.21.15.86:0/2920018645 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93195c800 0x55b931867600 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.196+0000 7f8e76d4c740 1 -- 172.21.15.86:0/2920018645 >> 172.21.15.86:0/2920018645 conn(0x55b93195c400 msgr2=0x55b930b61000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:44.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.196+0000 7f8e76d4c740 1 -- 172.21.15.86:0/2920018645 shutdown_connections 2024-04-20T21:26:44.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.196+0000 7f8e76d4c740 1 -- 172.21.15.86:0/2920018645 wait complete. 2024-04-20T21:26:44.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.197+0000 7f8e76d4c740 1 Processor -- start 2024-04-20T21:26:44.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.197+0000 7f8e76d4c740 1 -- start start 2024-04-20T21:26:44.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.197+0000 7f8e76d4c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b93195c800 0x55b931867600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.197+0000 7f8e76d4c740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b93195cc00 0x55b931867b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.197+0000 7f8e76d4c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93195d000 0x55b931868100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.198+0000 7f8e76d4c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b931963e00 con 0x55b93195d000 2024-04-20T21:26:44.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.198+0000 7f8e76d4c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b931963c80 con 0x55b93195c800 2024-04-20T21:26:44.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.198+0000 7f8e76d4c740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55b931963b00 con 0x55b93195cc00 2024-04-20T21:26:44.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.198+0000 7f8e74400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93195d000 0x55b931868100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.198+0000 7f8e74400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93195d000 0x55b931868100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45676/0 (socket says 172.21.15.86:45676) 2024-04-20T21:26:44.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.198+0000 7f8e74400640 1 -- 172.21.15.86:0/3252742221 learned_addr learned my addr 172.21.15.86:0/3252742221 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:44.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.198+0000 7f8e75800640 1 --2- 172.21.15.86:0/3252742221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b93195c800 0x55b931867600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.198+0000 7f8e74e00640 1 --2- 172.21.15.86:0/3252742221 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b93195cc00 0x55b931867b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.198+0000 7f8e74400640 1 -- 172.21.15.86:0/3252742221 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b93195cc00 msgr2=0x55b931867b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:44.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.198+0000 7f8e74400640 1 --2- 172.21.15.86:0/3252742221 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b93195cc00 0x55b931867b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.198+0000 7f8e74400640 1 -- 172.21.15.86:0/3252742221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b93195c800 msgr2=0x55b931867600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:44.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.198+0000 7f8e74400640 1 --2- 172.21.15.86:0/3252742221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b93195c800 0x55b931867600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.198+0000 7f8e74400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b930abfa40 con 0x55b93195d000 2024-04-20T21:26:44.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.198+0000 7f8e74e00640 1 --2- 172.21.15.86:0/3252742221 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b93195cc00 0x55b931867b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:26:44.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.198+0000 7f8e75800640 1 --2- 172.21.15.86:0/3252742221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b93195c800 0x55b931867600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:26:44.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.199+0000 7f8e74400640 1 --2- 172.21.15.86:0/3252742221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93195d000 0x55b931868100 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x55b931977c50 tx=0x55b93197cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:44.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.199+0000 7f8e6d600640 1 -- 172.21.15.86:0/3252742221 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b930ab8b60 con 0x55b93195d000 2024-04-20T21:26:44.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.199+0000 7f8e6d600640 1 -- 172.21.15.86:0/3252742221 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55b930add340 con 0x55b93195d000 2024-04-20T21:26:44.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.199+0000 7f8e6d600640 1 -- 172.21.15.86:0/3252742221 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b930ab8d00 con 0x55b93195d000 2024-04-20T21:26:44.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.199+0000 7f8e76d4c740 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b930abf4a0 con 0x55b93195d000 2024-04-20T21:26:44.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.199+0000 7f8e76d4c740 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b930abfc20 con 0x55b93195d000 2024-04-20T21:26:44.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.199+0000 7f8e76d4c740 4 datascan.init: resolving metadata pool 12 2024-04-20T21:26:44.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.201+0000 7f8e6d600640 1 -- 172.21.15.86:0/3252742221 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x55b9319d2c00 con 0x55b93195d000 2024-04-20T21:26:44.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.201+0000 7f8e6d600640 1 --2- 172.21.15.86:0/3252742221 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x55b930e08000 0x55b931868680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.201+0000 7f8e6d600640 1 -- 172.21.15.86:0/3252742221 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x55b9319d0c00 con 0x55b93195d000 2024-04-20T21:26:44.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.202+0000 7f8e76d4c740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-20T21:26:44.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.202+0000 7f8e76d4c740 4 datascan.main: data pool 'cephfs_data' has ID 13 2024-04-20T21:26:44.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.202+0000 7f8e76d4c740 4 datascan.main: opening data pool 'cephfs_data' 2024-04-20T21:26:44.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.202+0000 7f8e76d4c740 1 --2- 172.21.15.86:0/3252742221 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x55b930e08400 0x55b931868c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.202+0000 7f8e76d4c740 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:1 12.14 12:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x55b930e08800 con 0x55b930e08400 2024-04-20T21:26:44.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.202+0000 7f8e75800640 1 --2- 172.21.15.86:0/3252742221 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x55b930e08000 0x55b931868680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.202+0000 7f8e74e00640 1 --2- 172.21.15.86:0/3252742221 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x55b930e08400 0x55b931868c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.202+0000 7f8e75800640 1 --2- 172.21.15.86:0/3252742221 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x55b930e08000 0x55b931868680 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x55b931976f00 tx=0x55b93197cc00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:44.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.203+0000 7f8e74e00640 1 --2- 172.21.15.86:0/3252742221 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x55b930e08400 0x55b931868c00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:44.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.204+0000 7f8e74e00640 1 -- 172.21.15.86:0/3252742221 <== osd.5 v2:172.21.15.144:6808/1635974704 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv25 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x55b931820000 con 0x55b930e08400 2024-04-20T21:26:44.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.204+0000 7f8e76d4c740 1 --2- 172.21.15.86:0/3252742221 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x55b930e08c00 0x55b931869180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.204+0000 7f8e76d4c740 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:2 12.1f 12:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x55b930e09000 con 0x55b930e08c00 2024-04-20T21:26:44.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.205+0000 7f8e75800640 1 --2- 172.21.15.86:0/3252742221 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x55b930e08c00 0x55b931869180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.206+0000 7f8e75800640 1 --2- 172.21.15.86:0/3252742221 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x55b930e08c00 0x55b931869180 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:44.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.206+0000 7f8e75800640 1 -- 172.21.15.86:0/3252742221 <== osd.6 v2:172.21.15.144:6824/1893803079 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) ==== 162+0+16 (crc 0 0 0) 0x55b931820500 con 0x55b930e08c00 2024-04-20T21:26:44.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.206+0000 7f8e76d4c740 1 --2- 172.21.15.86:0/3252742221 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x55b930e09400 0x55b931869700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.206+0000 7f8e76d4c740 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:3 13.0 13:00000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e09800 con 0x55b930e09400 2024-04-20T21:26:44.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.207+0000 7f8e74e00640 1 --2- 172.21.15.86:0/3252742221 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x55b930e09400 0x55b931869700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.208+0000 7f8e74e00640 1 --2- 172.21.15.86:0/3252742221 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x55b930e09400 0x55b931869700 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:44.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.209+0000 7f8e74e00640 1 -- 172.21.15.86:0/3252742221 <== osd.7 v2:172.21.15.144:6816/2100967057 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820000 con 0x55b930e09400 2024-04-20T21:26:44.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.209+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:4 13.20 13:04000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e09c00 con 0x55b930e08c00 2024-04-20T21:26:44.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.210+0000 7f8e75800640 1 -- 172.21.15.86:0/3252742221 <== osd.6 v2:172.21.15.144:6824/1893803079 2 ==== osd_op_reply(4 [pgnls-filter start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820500 con 0x55b930e08c00 2024-04-20T21:26:44.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.210+0000 7f8e6f400640 1 --2- 172.21.15.86:0/3252742221 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x55b930e2e000 0x55b930e34000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.210+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:5 13.10 13:08000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e2e400 con 0x55b930e2e000 2024-04-20T21:26:44.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.210+0000 7f8e74400640 1 --2- 172.21.15.86:0/3252742221 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x55b930e2e000 0x55b930e34000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.211+0000 7f8e74400640 1 --2- 172.21.15.86:0/3252742221 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x55b930e2e000 0x55b930e34000 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:44.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.211+0000 7f8e74400640 1 -- 172.21.15.86:0/3252742221 <== osd.1 v2:172.21.15.86:6817/3149564961 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820780 con 0x55b930e2e000 2024-04-20T21:26:44.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.211+0000 7f8e6ea00640 1 --2- 172.21.15.86:0/3252742221 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x55b930e2ec00 0x55b930e34580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.211+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:6 13.30 13:0c000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e2f000 con 0x55b930e2ec00 2024-04-20T21:26:44.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.212+0000 7f8e75800640 1 --2- 172.21.15.86:0/3252742221 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x55b930e2ec00 0x55b930e34580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.213+0000 7f8e75800640 1 --2- 172.21.15.86:0/3252742221 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x55b930e2ec00 0x55b930e34580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:44.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.214+0000 7f8e75800640 1 -- 172.21.15.86:0/3252742221 <== osd.4 v2:172.21.15.144:6800/4087309475 1 ==== osd_op_reply(6 [pgnls-filter start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820500 con 0x55b930e2ec00 2024-04-20T21:26:44.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.214+0000 7f8e6f400640 1 --2- 172.21.15.86:0/3252742221 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x55b930e2e800 0x55b930e34b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.214+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:7 13.8 13:10000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e2f000 con 0x55b930e2e800 2024-04-20T21:26:44.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.214+0000 7f8e74e00640 1 --2- 172.21.15.86:0/3252742221 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x55b930e2e800 0x55b930e34b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.215+0000 7f8e74e00640 1 --2- 172.21.15.86:0/3252742221 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x55b930e2e800 0x55b930e34b00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:44.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.215+0000 7f8e74e00640 1 -- 172.21.15.86:0/3252742221 <== osd.2 v2:172.21.15.86:6816/3437611468 1 ==== osd_op_reply(7 [pgnls-filter start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820000 con 0x55b930e2e800 2024-04-20T21:26:44.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.215+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:8 13.28 13:14000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e2f400 con 0x55b930e2e800 2024-04-20T21:26:44.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.215+0000 7f8e74e00640 1 -- 172.21.15.86:0/3252742221 <== osd.2 v2:172.21.15.86:6816/3437611468 2 ==== osd_op_reply(8 [pgnls-filter start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820000 con 0x55b930e2e800 2024-04-20T21:26:44.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.215+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:9 13.18 13:18000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e09000 con 0x55b930e09400 2024-04-20T21:26:44.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.217+0000 7f8e74e00640 1 -- 172.21.15.86:0/3252742221 <== osd.7 v2:172.21.15.144:6816/2100967057 2 ==== osd_op_reply(9 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820000 con 0x55b930e09400 2024-04-20T21:26:44.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.217+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:10 13.38 13:1c000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e2f800 con 0x55b930e08400 2024-04-20T21:26:44.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.218+0000 7f8e74e00640 1 -- 172.21.15.86:0/3252742221 <== osd.5 v2:172.21.15.144:6808/1635974704 2 ==== osd_op_reply(10 [pgnls-filter start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820000 con 0x55b930e08400 2024-04-20T21:26:44.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.218+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:11 13.4 13:20000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e09c00 con 0x55b930e08c00 2024-04-20T21:26:44.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.218+0000 7f8e75800640 1 -- 172.21.15.86:0/3252742221 <== osd.6 v2:172.21.15.144:6824/1893803079 3 ==== osd_op_reply(11 [pgnls-filter start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820500 con 0x55b930e08c00 2024-04-20T21:26:44.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.219+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:12 13.24 13:24000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e2fc00 con 0x55b930e08c00 2024-04-20T21:26:44.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.220+0000 7f8e75800640 1 -- 172.21.15.86:0/3252742221 <== osd.6 v2:172.21.15.144:6824/1893803079 4 ==== osd_op_reply(12 [pgnls-filter start_epoch 89 out=49b] v85'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820500 con 0x55b930e08c00 2024-04-20T21:26:44.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.220+0000 7f8e6f400640 1 --2- 172.21.15.86:0/3252742221 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x55b930e08800 0x55b930e35080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.220+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:13 13.14 13:28000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e09800 con 0x55b930e08800 2024-04-20T21:26:44.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.220+0000 7f8e74400640 1 --2- 172.21.15.86:0/3252742221 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x55b930e08800 0x55b930e35080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.220+0000 7f8e74400640 1 --2- 172.21.15.86:0/3252742221 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x55b930e08800 0x55b930e35080 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:44.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.221+0000 7f8e74400640 1 -- 172.21.15.86:0/3252742221 <== osd.0 v2:172.21.15.86:6800/2471076035 1 ==== osd_op_reply(13 [pgnls-filter start_epoch 89 out=49b] v85'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820780 con 0x55b930e08800 2024-04-20T21:26:44.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.221+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:14 13.34 13:2c000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e09000 con 0x55b930e08c00 2024-04-20T21:26:44.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.222+0000 7f8e75800640 1 -- 172.21.15.86:0/3252742221 <== osd.6 v2:172.21.15.144:6824/1893803079 5 ==== osd_op_reply(14 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820500 con 0x55b930e08c00 2024-04-20T21:26:44.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.222+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:15 13.c 13:30000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e2f000 con 0x55b930e2e000 2024-04-20T21:26:44.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.222+0000 7f8e74400640 1 -- 172.21.15.86:0/3252742221 <== osd.1 v2:172.21.15.86:6817/3149564961 2 ==== osd_op_reply(15 [pgnls-filter start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820780 con 0x55b930e2e000 2024-04-20T21:26:44.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.222+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:16 13.2c 13:34000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e4a000 con 0x55b930e2e800 2024-04-20T21:26:44.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.222+0000 7f8e74e00640 1 -- 172.21.15.86:0/3252742221 <== osd.2 v2:172.21.15.86:6816/3437611468 3 ==== osd_op_reply(16 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820000 con 0x55b930e2e800 2024-04-20T21:26:44.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.222+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:17 13.1c 13:38000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e2f400 con 0x55b930e08c00 2024-04-20T21:26:44.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.223+0000 7f8e75800640 1 -- 172.21.15.86:0/3252742221 <== osd.6 v2:172.21.15.144:6824/1893803079 6 ==== osd_op_reply(17 [pgnls-filter start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820500 con 0x55b930e08c00 2024-04-20T21:26:44.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.224+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:18 13.3c 13:3c000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e4a400 con 0x55b930e2e800 2024-04-20T21:26:44.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.224+0000 7f8e74e00640 1 -- 172.21.15.86:0/3252742221 <== osd.2 v2:172.21.15.86:6816/3437611468 4 ==== osd_op_reply(18 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820000 con 0x55b930e2e800 2024-04-20T21:26:44.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.224+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:19 13.2 13:40000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e4b400 con 0x55b930e08400 2024-04-20T21:26:44.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.225+0000 7f8e74e00640 1 -- 172.21.15.86:0/3252742221 <== osd.5 v2:172.21.15.144:6808/1635974704 3 ==== osd_op_reply(19 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820000 con 0x55b930e08400 2024-04-20T21:26:44.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.225+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:20 13.22 13:44000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e4a800 con 0x55b930e2e000 2024-04-20T21:26:44.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.225+0000 7f8e74400640 1 -- 172.21.15.86:0/3252742221 <== osd.1 v2:172.21.15.86:6817/3149564961 3 ==== osd_op_reply(20 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820780 con 0x55b930e2e000 2024-04-20T21:26:44.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.225+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:21 13.12 13:48000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e2fc00 con 0x55b930e08800 2024-04-20T21:26:44.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.226+0000 7f8e74400640 1 -- 172.21.15.86:0/3252742221 <== osd.0 v2:172.21.15.86:6800/2471076035 2 ==== osd_op_reply(21 [pgnls-filter start_epoch 89 out=49b] v85'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820780 con 0x55b930e08800 2024-04-20T21:26:44.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.226+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:22 13.32 13:4c000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e4ac00 con 0x55b930e2e800 2024-04-20T21:26:44.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.226+0000 7f8e74e00640 1 -- 172.21.15.86:0/3252742221 <== osd.2 v2:172.21.15.86:6816/3437611468 5 ==== osd_op_reply(22 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820000 con 0x55b930e2e800 2024-04-20T21:26:44.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.226+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:23 13.a 13:50000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e2f400 con 0x55b930e2e800 2024-04-20T21:26:44.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.226+0000 7f8e74e00640 1 -- 172.21.15.86:0/3252742221 <== osd.2 v2:172.21.15.86:6816/3437611468 6 ==== osd_op_reply(23 [pgnls-filter start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820000 con 0x55b930e2e800 2024-04-20T21:26:44.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.226+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:24 13.2a 13:54000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e4b000 con 0x55b930e2e000 2024-04-20T21:26:44.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.227+0000 7f8e74400640 1 -- 172.21.15.86:0/3252742221 <== osd.1 v2:172.21.15.86:6817/3149564961 4 ==== osd_op_reply(24 [pgnls-filter start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820780 con 0x55b930e2e000 2024-04-20T21:26:44.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.227+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:25 13.1a 13:58000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e09c00 con 0x55b930e2e000 2024-04-20T21:26:44.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.227+0000 7f8e74400640 1 -- 172.21.15.86:0/3252742221 <== osd.1 v2:172.21.15.86:6817/3149564961 5 ==== osd_op_reply(25 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820780 con 0x55b930e2e000 2024-04-20T21:26:44.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.227+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:26 13.3a 13:5c000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e4a000 con 0x55b930e08400 2024-04-20T21:26:44.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.228+0000 7f8e74e00640 1 -- 172.21.15.86:0/3252742221 <== osd.5 v2:172.21.15.144:6808/1635974704 4 ==== osd_op_reply(26 [pgnls-filter start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820000 con 0x55b930e08400 2024-04-20T21:26:44.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.228+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:27 13.6 13:60000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e09000 con 0x55b930e2ec00 2024-04-20T21:26:44.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.229+0000 7f8e75800640 1 -- 172.21.15.86:0/3252742221 <== osd.4 v2:172.21.15.144:6800/4087309475 2 ==== osd_op_reply(27 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820500 con 0x55b930e2ec00 2024-04-20T21:26:44.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.229+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:28 13.26 13:64000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e2f800 con 0x55b930e09400 2024-04-20T21:26:44.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.230+0000 7f8e74e00640 1 -- 172.21.15.86:0/3252742221 <== osd.7 v2:172.21.15.144:6816/2100967057 3 ==== osd_op_reply(28 [pgnls-filter start_epoch 89 out=49b] v85'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820000 con 0x55b930e09400 2024-04-20T21:26:44.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.230+0000 7f8e6f400640 1 --2- 172.21.15.86:0/3252742221 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x55b930e4a400 0x55b930e35600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.230+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:29 13.16 13:68000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e4b400 con 0x55b930e4a400 2024-04-20T21:26:44.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.231+0000 7f8e75800640 1 --2- 172.21.15.86:0/3252742221 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x55b930e4a400 0x55b930e35600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.232+0000 7f8e75800640 1 --2- 172.21.15.86:0/3252742221 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x55b930e4a400 0x55b930e35600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:44.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.232+0000 7f8e75800640 1 -- 172.21.15.86:0/3252742221 <== osd.3 v2:172.21.15.86:6808/794135519 1 ==== osd_op_reply(29 [pgnls-filter start_epoch 89 out=49b] v85'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820500 con 0x55b930e4a400 2024-04-20T21:26:44.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.232+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:30 13.36 13:6c000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e2f400 con 0x55b930e08c00 2024-04-20T21:26:44.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.233+0000 7f8e75800640 1 -- 172.21.15.86:0/3252742221 <== osd.6 v2:172.21.15.144:6824/1893803079 7 ==== osd_op_reply(30 [pgnls-filter start_epoch 89 out=49b] v85'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820500 con 0x55b930e08c00 2024-04-20T21:26:44.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.233+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:31 13.e 13:70000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e4ac00 con 0x55b930e09400 2024-04-20T21:26:44.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.234+0000 7f8e74e00640 1 -- 172.21.15.86:0/3252742221 <== osd.7 v2:172.21.15.144:6816/2100967057 4 ==== osd_op_reply(31 [pgnls-filter start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820000 con 0x55b930e09400 2024-04-20T21:26:44.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.234+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:32 13.2e 13:74000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e2e400 con 0x55b930e2e800 2024-04-20T21:26:44.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.235+0000 7f8e74e00640 1 -- 172.21.15.86:0/3252742221 <== osd.2 v2:172.21.15.86:6816/3437611468 7 ==== osd_op_reply(32 [pgnls-filter start_epoch 89 out=49b] v85'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820000 con 0x55b930e2e800 2024-04-20T21:26:44.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.235+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:33 13.1e 13:78000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e4a800 con 0x55b930e2ec00 2024-04-20T21:26:44.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.236+0000 7f8e75800640 1 -- 172.21.15.86:0/3252742221 <== osd.4 v2:172.21.15.144:6800/4087309475 3 ==== osd_op_reply(33 [pgnls-filter start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820500 con 0x55b930e2ec00 2024-04-20T21:26:44.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.236+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:34 13.3e 13:7c000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e2f000 con 0x55b930e4a400 2024-04-20T21:26:44.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.237+0000 7f8e75800640 1 -- 172.21.15.86:0/3252742221 <== osd.3 v2:172.21.15.86:6808/794135519 2 ==== osd_op_reply(34 [pgnls-filter start_epoch 89 out=49b] v85'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820500 con 0x55b930e4a400 2024-04-20T21:26:44.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.237+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:35 13.1 13:80000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e4b800 con 0x55b930e08c00 2024-04-20T21:26:44.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.238+0000 7f8e75800640 1 -- 172.21.15.86:0/3252742221 <== osd.6 v2:172.21.15.144:6824/1893803079 8 ==== osd_op_reply(35 [pgnls-filter start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820500 con 0x55b930e08c00 2024-04-20T21:26:44.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.238+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:36 13.21 13:84000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e09800 con 0x55b930e2ec00 2024-04-20T21:26:44.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.239+0000 7f8e75800640 1 -- 172.21.15.86:0/3252742221 <== osd.4 v2:172.21.15.144:6800/4087309475 4 ==== osd_op_reply(36 [pgnls-filter start_epoch 89 out=49b] v85'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820500 con 0x55b930e2ec00 2024-04-20T21:26:44.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.239+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:37 13.11 13:88000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e4bc00 con 0x55b930e2e000 2024-04-20T21:26:44.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.239+0000 7f8e74400640 1 -- 172.21.15.86:0/3252742221 <== osd.1 v2:172.21.15.86:6817/3149564961 6 ==== osd_op_reply(37 [pgnls-filter start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820780 con 0x55b930e2e000 2024-04-20T21:26:44.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.239+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:38 13.31 13:8c000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e2f400 con 0x55b930e4a400 2024-04-20T21:26:44.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.240+0000 7f8e75800640 1 -- 172.21.15.86:0/3252742221 <== osd.3 v2:172.21.15.86:6808/794135519 3 ==== osd_op_reply(38 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820500 con 0x55b930e4a400 2024-04-20T21:26:44.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.240+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:39 13.9 13:90000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e54000 con 0x55b930e08400 2024-04-20T21:26:44.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.241+0000 7f8e74e00640 1 -- 172.21.15.86:0/3252742221 <== osd.5 v2:172.21.15.144:6808/1635974704 5 ==== osd_op_reply(39 [pgnls-filter start_epoch 89 out=49b] v85'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820000 con 0x55b930e08400 2024-04-20T21:26:44.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.241+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:40 13.29 13:94000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e2f000 con 0x55b930e08c00 2024-04-20T21:26:44.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.242+0000 7f8e75800640 1 -- 172.21.15.86:0/3252742221 <== osd.6 v2:172.21.15.144:6824/1893803079 9 ==== osd_op_reply(40 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820500 con 0x55b930e08c00 2024-04-20T21:26:44.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.242+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:41 13.19 13:98000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e54400 con 0x55b930e4a400 2024-04-20T21:26:44.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.243+0000 7f8e75800640 1 -- 172.21.15.86:0/3252742221 <== osd.3 v2:172.21.15.86:6808/794135519 4 ==== osd_op_reply(41 [pgnls-filter start_epoch 89 out=49b] v85'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820500 con 0x55b930e4a400 2024-04-20T21:26:44.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.243+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:42 13.39 13:9c000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e4b400 con 0x55b930e2e800 2024-04-20T21:26:44.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.243+0000 7f8e74e00640 1 -- 172.21.15.86:0/3252742221 <== osd.2 v2:172.21.15.86:6816/3437611468 8 ==== osd_op_reply(42 [pgnls-filter start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820000 con 0x55b930e2e800 2024-04-20T21:26:44.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.243+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:43 13.5 13:a0000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e54800 con 0x55b930e2ec00 2024-04-20T21:26:44.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.244+0000 7f8e75800640 1 -- 172.21.15.86:0/3252742221 <== osd.4 v2:172.21.15.144:6800/4087309475 5 ==== osd_op_reply(43 [pgnls-filter start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820500 con 0x55b930e2ec00 2024-04-20T21:26:44.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.244+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:44 13.25 13:a4000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e4a800 con 0x55b930e2ec00 2024-04-20T21:26:44.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.246+0000 7f8e75800640 1 -- 172.21.15.86:0/3252742221 <== osd.4 v2:172.21.15.144:6800/4087309475 6 ==== osd_op_reply(44 [pgnls-filter start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820500 con 0x55b930e2ec00 2024-04-20T21:26:44.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.246+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:45 13.15 13:a8000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e54c00 con 0x55b930e2ec00 2024-04-20T21:26:44.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.247+0000 7f8e75800640 1 -- 172.21.15.86:0/3252742221 <== osd.4 v2:172.21.15.144:6800/4087309475 7 ==== osd_op_reply(45 [pgnls-filter start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820500 con 0x55b930e2ec00 2024-04-20T21:26:44.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.247+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:46 13.35 13:ac000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e09000 con 0x55b930e2e000 2024-04-20T21:26:44.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.247+0000 7f8e74400640 1 -- 172.21.15.86:0/3252742221 <== osd.1 v2:172.21.15.86:6817/3149564961 7 ==== osd_op_reply(46 [pgnls-filter start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820780 con 0x55b930e2e000 2024-04-20T21:26:44.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.247+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:47 13.d 13:b0000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e55000 con 0x55b930e2ec00 2024-04-20T21:26:44.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.248+0000 7f8e75800640 1 -- 172.21.15.86:0/3252742221 <== osd.4 v2:172.21.15.144:6800/4087309475 8 ==== osd_op_reply(47 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820500 con 0x55b930e2ec00 2024-04-20T21:26:44.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.248+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:48 13.2d 13:b4000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e2fc00 con 0x55b930e08800 2024-04-20T21:26:44.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.249+0000 7f8e74400640 1 -- 172.21.15.86:0/3252742221 <== osd.0 v2:172.21.15.86:6800/2471076035 3 ==== osd_op_reply(48 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820780 con 0x55b930e08800 2024-04-20T21:26:44.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.249+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:49 13.1d 13:b8000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e55400 con 0x55b930e09400 2024-04-20T21:26:44.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.250+0000 7f8e74e00640 1 -- 172.21.15.86:0/3252742221 <== osd.7 v2:172.21.15.144:6816/2100967057 5 ==== osd_op_reply(49 [pgnls-filter start_epoch 89 out=49b] v85'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820000 con 0x55b930e09400 2024-04-20T21:26:44.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.250+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:50 13.3d 13:bc000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e2f400 con 0x55b930e09400 2024-04-20T21:26:44.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.251+0000 7f8e74e00640 1 -- 172.21.15.86:0/3252742221 <== osd.7 v2:172.21.15.144:6816/2100967057 6 ==== osd_op_reply(50 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820000 con 0x55b930e09400 2024-04-20T21:26:44.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.251+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:51 13.3 13:c0000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e55800 con 0x55b930e08800 2024-04-20T21:26:44.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.252+0000 7f8e74400640 1 -- 172.21.15.86:0/3252742221 <== osd.0 v2:172.21.15.86:6800/2471076035 4 ==== osd_op_reply(51 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820780 con 0x55b930e08800 2024-04-20T21:26:44.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.252+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:52 13.23 13:c4000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e09c00 con 0x55b930e2e000 2024-04-20T21:26:44.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.252+0000 7f8e74400640 1 -- 172.21.15.86:0/3252742221 <== osd.1 v2:172.21.15.86:6817/3149564961 8 ==== osd_op_reply(52 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820780 con 0x55b930e2e000 2024-04-20T21:26:44.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.252+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:53 13.13 13:c8000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e54000 con 0x55b930e09400 2024-04-20T21:26:44.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.253+0000 7f8e74e00640 1 -- 172.21.15.86:0/3252742221 <== osd.7 v2:172.21.15.144:6816/2100967057 7 ==== osd_op_reply(53 [pgnls-filter start_epoch 89 out=49b] v85'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820000 con 0x55b930e09400 2024-04-20T21:26:44.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.254+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:54 13.33 13:cc000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e09000 con 0x55b930e09400 2024-04-20T21:26:44.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.255+0000 7f8e74e00640 1 -- 172.21.15.86:0/3252742221 <== osd.7 v2:172.21.15.144:6816/2100967057 8 ==== osd_op_reply(54 [pgnls-filter start_epoch 89 out=49b] v85'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820000 con 0x55b930e09400 2024-04-20T21:26:44.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.255+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:55 13.b 13:d0000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e55400 con 0x55b930e08400 2024-04-20T21:26:44.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.256+0000 7f8e74e00640 1 -- 172.21.15.86:0/3252742221 <== osd.5 v2:172.21.15.144:6808/1635974704 6 ==== osd_op_reply(55 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820000 con 0x55b930e08400 2024-04-20T21:26:44.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.256+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:56 13.2b 13:d4000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e09800 con 0x55b930e2e800 2024-04-20T21:26:44.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.256+0000 7f8e74e00640 1 -- 172.21.15.86:0/3252742221 <== osd.2 v2:172.21.15.86:6816/3437611468 9 ==== osd_op_reply(56 [pgnls-filter start_epoch 89 out=49b] v85'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820000 con 0x55b930e2e800 2024-04-20T21:26:44.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.256+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:57 13.1b 13:d8000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e2f800 con 0x55b930e2ec00 2024-04-20T21:26:44.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.258+0000 7f8e75800640 1 -- 172.21.15.86:0/3252742221 <== osd.4 v2:172.21.15.144:6800/4087309475 9 ==== osd_op_reply(57 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820500 con 0x55b930e2ec00 2024-04-20T21:26:44.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.258+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:58 13.3b 13:dc000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e4b000 con 0x55b930e2ec00 2024-04-20T21:26:44.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.259+0000 7f8e75800640 1 -- 172.21.15.86:0/3252742221 <== osd.4 v2:172.21.15.144:6800/4087309475 10 ==== osd_op_reply(58 [pgnls-filter start_epoch 89 out=49b] v85'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820500 con 0x55b930e2ec00 2024-04-20T21:26:44.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.259+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:59 13.7 13:e0000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e2e400 con 0x55b930e09400 2024-04-20T21:26:44.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.260+0000 7f8e74e00640 1 -- 172.21.15.86:0/3252742221 <== osd.7 v2:172.21.15.144:6816/2100967057 9 ==== osd_op_reply(59 [pgnls-filter start_epoch 89 out=81b] v89'7 uv7 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55b931820000 con 0x55b930e09400 2024-04-20T21:26:44.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.260+0000 7f8e76d4c740 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:60 13.0 13:00000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e54400 con 0x55b930e09400 2024-04-20T21:26:44.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.261+0000 7f8e74e00640 1 -- 172.21.15.86:0/3252742221 <== osd.7 v2:172.21.15.144:6816/2100967057 10 ==== osd_op_reply(60 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820000 con 0x55b930e09400 2024-04-20T21:26:44.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.261+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:61 13.20 13:04000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e2f000 con 0x55b930e08c00 2024-04-20T21:26:44.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.263+0000 7f8e75800640 1 -- 172.21.15.86:0/3252742221 <== osd.6 v2:172.21.15.144:6824/1893803079 10 ==== osd_op_reply(61 [pgnls-filter start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820500 con 0x55b930e08c00 2024-04-20T21:26:44.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.263+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:62 13.10 13:08000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e4bc00 con 0x55b930e2e000 2024-04-20T21:26:44.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.263+0000 7f8e74400640 1 -- 172.21.15.86:0/3252742221 <== osd.1 v2:172.21.15.86:6817/3149564961 9 ==== osd_op_reply(62 [pgnls-filter start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820780 con 0x55b930e2e000 2024-04-20T21:26:44.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.263+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:63 13.30 13:0c000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e4a000 con 0x55b930e2ec00 2024-04-20T21:26:44.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.265+0000 7f8e75800640 1 -- 172.21.15.86:0/3252742221 <== osd.4 v2:172.21.15.144:6800/4087309475 11 ==== osd_op_reply(63 [pgnls-filter start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820500 con 0x55b930e2ec00 2024-04-20T21:26:44.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.265+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:64 13.8 13:10000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e4b800 con 0x55b930e2e800 2024-04-20T21:26:44.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.266+0000 7f8e74e00640 1 -- 172.21.15.86:0/3252742221 <== osd.2 v2:172.21.15.86:6816/3437611468 10 ==== osd_op_reply(64 [pgnls-filter start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820000 con 0x55b930e2e800 2024-04-20T21:26:44.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.266+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:65 13.28 13:14000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e4ac00 con 0x55b930e2e800 2024-04-20T21:26:44.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.266+0000 7f8e74e00640 1 -- 172.21.15.86:0/3252742221 <== osd.2 v2:172.21.15.86:6816/3437611468 11 ==== osd_op_reply(65 [pgnls-filter start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820000 con 0x55b930e2e800 2024-04-20T21:26:44.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.266+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:66 13.18 13:18000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e2f400 con 0x55b930e09400 2024-04-20T21:26:44.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.267+0000 7f8e74e00640 1 -- 172.21.15.86:0/3252742221 <== osd.7 v2:172.21.15.144:6816/2100967057 11 ==== osd_op_reply(66 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820000 con 0x55b930e09400 2024-04-20T21:26:44.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.268+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:67 13.38 13:1c000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e4b400 con 0x55b930e08400 2024-04-20T21:26:44.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.269+0000 7f8e74e00640 1 -- 172.21.15.86:0/3252742221 <== osd.5 v2:172.21.15.144:6808/1635974704 7 ==== osd_op_reply(67 [pgnls-filter start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820000 con 0x55b930e08400 2024-04-20T21:26:44.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.269+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:68 13.4 13:20000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e2e400 con 0x55b930e08c00 2024-04-20T21:26:44.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.270+0000 7f8e75800640 1 -- 172.21.15.86:0/3252742221 <== osd.6 v2:172.21.15.144:6824/1893803079 11 ==== osd_op_reply(68 [pgnls-filter start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820500 con 0x55b930e08c00 2024-04-20T21:26:44.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.270+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:69 13.24 13:24000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e4b800 con 0x55b930e08c00 2024-04-20T21:26:44.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.271+0000 7f8e75800640 1 -- 172.21.15.86:0/3252742221 <== osd.6 v2:172.21.15.144:6824/1893803079 12 ==== osd_op_reply(69 [pgnls-filter start_epoch 89 out=49b] v85'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820500 con 0x55b930e08c00 2024-04-20T21:26:44.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.271+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:70 13.14 13:28000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e54000 con 0x55b930e08800 2024-04-20T21:26:44.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.272+0000 7f8e74400640 1 -- 172.21.15.86:0/3252742221 <== osd.0 v2:172.21.15.86:6800/2471076035 5 ==== osd_op_reply(70 [pgnls-filter start_epoch 89 out=49b] v85'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820780 con 0x55b930e08800 2024-04-20T21:26:44.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.272+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:71 13.34 13:2c000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e4a800 con 0x55b930e08c00 2024-04-20T21:26:44.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.273+0000 7f8e75800640 1 -- 172.21.15.86:0/3252742221 <== osd.6 v2:172.21.15.144:6824/1893803079 13 ==== osd_op_reply(71 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820500 con 0x55b930e08c00 2024-04-20T21:26:44.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.273+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:72 13.c 13:30000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e55400 con 0x55b930e2e000 2024-04-20T21:26:44.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.274+0000 7f8e74400640 1 -- 172.21.15.86:0/3252742221 <== osd.1 v2:172.21.15.86:6817/3149564961 10 ==== osd_op_reply(72 [pgnls-filter start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820780 con 0x55b930e2e000 2024-04-20T21:26:44.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.274+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:73 13.2c 13:34000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e4b000 con 0x55b930e2e800 2024-04-20T21:26:44.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.275+0000 7f8e74e00640 1 -- 172.21.15.86:0/3252742221 <== osd.2 v2:172.21.15.86:6816/3437611468 12 ==== osd_op_reply(73 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820000 con 0x55b930e2e800 2024-04-20T21:26:44.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.275+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:74 13.1c 13:38000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e54400 con 0x55b930e08c00 2024-04-20T21:26:44.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.276+0000 7f8e75800640 1 -- 172.21.15.86:0/3252742221 <== osd.6 v2:172.21.15.144:6824/1893803079 14 ==== osd_op_reply(74 [pgnls-filter start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820500 con 0x55b930e08c00 2024-04-20T21:26:44.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.276+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:75 13.3c 13:3c000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e4a000 con 0x55b930e2e800 2024-04-20T21:26:44.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.277+0000 7f8e74e00640 1 -- 172.21.15.86:0/3252742221 <== osd.2 v2:172.21.15.86:6816/3437611468 13 ==== osd_op_reply(75 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820000 con 0x55b930e2e800 2024-04-20T21:26:44.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.277+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:76 13.2 13:40000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e54c00 con 0x55b930e08400 2024-04-20T21:26:44.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.278+0000 7f8e74e00640 1 -- 172.21.15.86:0/3252742221 <== osd.5 v2:172.21.15.144:6808/1635974704 8 ==== osd_op_reply(76 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820000 con 0x55b930e08400 2024-04-20T21:26:44.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.278+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:77 13.22 13:44000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e2f800 con 0x55b930e2e000 2024-04-20T21:26:44.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.279+0000 7f8e74400640 1 -- 172.21.15.86:0/3252742221 <== osd.1 v2:172.21.15.86:6817/3149564961 11 ==== osd_op_reply(77 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820780 con 0x55b930e2e000 2024-04-20T21:26:44.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.279+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:78 13.12 13:48000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e55000 con 0x55b930e08800 2024-04-20T21:26:44.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.280+0000 7f8e74400640 1 -- 172.21.15.86:0/3252742221 <== osd.0 v2:172.21.15.86:6800/2471076035 6 ==== osd_op_reply(78 [pgnls-filter start_epoch 89 out=49b] v85'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820780 con 0x55b930e08800 2024-04-20T21:26:44.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.280+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:79 13.32 13:4c000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e2f000 con 0x55b930e2e800 2024-04-20T21:26:44.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.281+0000 7f8e74e00640 1 -- 172.21.15.86:0/3252742221 <== osd.2 v2:172.21.15.86:6816/3437611468 14 ==== osd_op_reply(79 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820000 con 0x55b930e2e800 2024-04-20T21:26:44.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.281+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:80 13.a 13:50000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e09000 con 0x55b930e2e800 2024-04-20T21:26:44.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.281+0000 7f8e74e00640 1 -- 172.21.15.86:0/3252742221 <== osd.2 v2:172.21.15.86:6816/3437611468 15 ==== osd_op_reply(80 [pgnls-filter start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820000 con 0x55b930e2e800 2024-04-20T21:26:44.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.282+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:81 13.2a 13:54000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e55800 con 0x55b930e2e000 2024-04-20T21:26:44.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.282+0000 7f8e74400640 1 -- 172.21.15.86:0/3252742221 <== osd.1 v2:172.21.15.86:6817/3149564961 12 ==== osd_op_reply(81 [pgnls-filter start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820780 con 0x55b930e2e000 2024-04-20T21:26:44.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.282+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:82 13.1a 13:58000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e09800 con 0x55b930e2e000 2024-04-20T21:26:44.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.283+0000 7f8e74400640 1 -- 172.21.15.86:0/3252742221 <== osd.1 v2:172.21.15.86:6817/3149564961 13 ==== osd_op_reply(82 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820780 con 0x55b930e2e000 2024-04-20T21:26:44.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.283+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:83 13.3a 13:5c000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e54000 con 0x55b930e08400 2024-04-20T21:26:44.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.285+0000 7f8e74e00640 1 -- 172.21.15.86:0/3252742221 <== osd.5 v2:172.21.15.144:6808/1635974704 9 ==== osd_op_reply(83 [pgnls-filter start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820000 con 0x55b930e08400 2024-04-20T21:26:44.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.285+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:84 13.6 13:60000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e54c00 con 0x55b930e2ec00 2024-04-20T21:26:44.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.286+0000 7f8e75800640 1 -- 172.21.15.86:0/3252742221 <== osd.4 v2:172.21.15.144:6800/4087309475 12 ==== osd_op_reply(84 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820500 con 0x55b930e2ec00 2024-04-20T21:26:44.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.286+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:85 13.26 13:64000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e55400 con 0x55b930e09400 2024-04-20T21:26:44.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.287+0000 7f8e74e00640 1 -- 172.21.15.86:0/3252742221 <== osd.7 v2:172.21.15.144:6816/2100967057 12 ==== osd_op_reply(85 [pgnls-filter start_epoch 89 out=49b] v85'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820000 con 0x55b930e09400 2024-04-20T21:26:44.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.287+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:86 13.16 13:68000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e54000 con 0x55b930e4a400 2024-04-20T21:26:44.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.288+0000 7f8e75800640 1 -- 172.21.15.86:0/3252742221 <== osd.3 v2:172.21.15.86:6808/794135519 5 ==== osd_op_reply(86 [pgnls-filter start_epoch 89 out=49b] v85'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820500 con 0x55b930e4a400 2024-04-20T21:26:44.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.288+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:87 13.36 13:6c000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e4bc00 con 0x55b930e08c00 2024-04-20T21:26:44.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.289+0000 7f8e75800640 1 -- 172.21.15.86:0/3252742221 <== osd.6 v2:172.21.15.144:6824/1893803079 15 ==== osd_op_reply(87 [pgnls-filter start_epoch 89 out=49b] v85'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820500 con 0x55b930e08c00 2024-04-20T21:26:44.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.289+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:88 13.e 13:70000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e09000 con 0x55b930e09400 2024-04-20T21:26:44.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.290+0000 7f8e74e00640 1 -- 172.21.15.86:0/3252742221 <== osd.7 v2:172.21.15.144:6816/2100967057 13 ==== osd_op_reply(88 [pgnls-filter start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820000 con 0x55b930e09400 2024-04-20T21:26:44.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.290+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:89 13.2e 13:74000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e09c00 con 0x55b930e2e800 2024-04-20T21:26:44.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.290+0000 7f8e74e00640 1 -- 172.21.15.86:0/3252742221 <== osd.2 v2:172.21.15.86:6816/3437611468 16 ==== osd_op_reply(89 [pgnls-filter start_epoch 89 out=49b] v85'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820000 con 0x55b930e2e800 2024-04-20T21:26:44.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.290+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:90 13.1e 13:78000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e2f400 con 0x55b930e2ec00 2024-04-20T21:26:44.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.292+0000 7f8e75800640 1 -- 172.21.15.86:0/3252742221 <== osd.4 v2:172.21.15.144:6800/4087309475 13 ==== osd_op_reply(90 [pgnls-filter start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820500 con 0x55b930e2ec00 2024-04-20T21:26:44.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.292+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:91 13.3e 13:7c000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e2fc00 con 0x55b930e4a400 2024-04-20T21:26:44.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.293+0000 7f8e75800640 1 -- 172.21.15.86:0/3252742221 <== osd.3 v2:172.21.15.86:6808/794135519 6 ==== osd_op_reply(91 [pgnls-filter start_epoch 89 out=49b] v85'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820500 con 0x55b930e4a400 2024-04-20T21:26:44.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.293+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:92 13.1 13:80000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e2f000 con 0x55b930e08c00 2024-04-20T21:26:44.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.294+0000 7f8e75800640 1 -- 172.21.15.86:0/3252742221 <== osd.6 v2:172.21.15.144:6824/1893803079 16 ==== osd_op_reply(92 [pgnls-filter start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820500 con 0x55b930e08c00 2024-04-20T21:26:44.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.294+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:93 13.21 13:84000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e2e400 con 0x55b930e2ec00 2024-04-20T21:26:44.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.295+0000 7f8e75800640 1 -- 172.21.15.86:0/3252742221 <== osd.4 v2:172.21.15.144:6800/4087309475 14 ==== osd_op_reply(93 [pgnls-filter start_epoch 89 out=49b] v85'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820500 con 0x55b930e2ec00 2024-04-20T21:26:44.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.295+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:94 13.11 13:88000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e4ac00 con 0x55b930e2e000 2024-04-20T21:26:44.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.296+0000 7f8e74400640 1 -- 172.21.15.86:0/3252742221 <== osd.1 v2:172.21.15.86:6817/3149564961 14 ==== osd_op_reply(94 [pgnls-filter start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820780 con 0x55b930e2e000 2024-04-20T21:26:44.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.296+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:95 13.31 13:8c000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e2f400 con 0x55b930e4a400 2024-04-20T21:26:44.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.297+0000 7f8e75800640 1 -- 172.21.15.86:0/3252742221 <== osd.3 v2:172.21.15.86:6808/794135519 7 ==== osd_op_reply(95 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820500 con 0x55b930e4a400 2024-04-20T21:26:44.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.297+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:96 13.9 13:90000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e4b400 con 0x55b930e08400 2024-04-20T21:26:44.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.298+0000 7f8e74e00640 1 -- 172.21.15.86:0/3252742221 <== osd.5 v2:172.21.15.144:6808/1635974704 10 ==== osd_op_reply(96 [pgnls-filter start_epoch 89 out=49b] v85'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820000 con 0x55b930e08400 2024-04-20T21:26:44.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.299+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:97 13.29 13:94000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e4b800 con 0x55b930e08c00 2024-04-20T21:26:44.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.300+0000 7f8e75800640 1 -- 172.21.15.86:0/3252742221 <== osd.6 v2:172.21.15.144:6824/1893803079 17 ==== osd_op_reply(97 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820500 con 0x55b930e08c00 2024-04-20T21:26:44.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.300+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:98 13.19 13:98000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e4b000 con 0x55b930e4a400 2024-04-20T21:26:44.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.301+0000 7f8e75800640 1 -- 172.21.15.86:0/3252742221 <== osd.3 v2:172.21.15.86:6808/794135519 8 ==== osd_op_reply(98 [pgnls-filter start_epoch 89 out=49b] v85'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820500 con 0x55b930e4a400 2024-04-20T21:26:44.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.301+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:99 13.39 13:9c000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e4a800 con 0x55b930e2e800 2024-04-20T21:26:44.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.302+0000 7f8e74e00640 1 -- 172.21.15.86:0/3252742221 <== osd.2 v2:172.21.15.86:6816/3437611468 17 ==== osd_op_reply(99 [pgnls-filter start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820000 con 0x55b930e2e800 2024-04-20T21:26:44.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.302+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:100 13.5 13:a0000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e4a000 con 0x55b930e2ec00 2024-04-20T21:26:44.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.303+0000 7f8e75800640 1 -- 172.21.15.86:0/3252742221 <== osd.4 v2:172.21.15.144:6800/4087309475 15 ==== osd_op_reply(100 [pgnls-filter start_epoch 89 out=49b] v85'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820500 con 0x55b930e2ec00 2024-04-20T21:26:44.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.303+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:101 13.25 13:a4000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e4bc00 con 0x55b930e2ec00 2024-04-20T21:26:44.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.304+0000 7f8e75800640 1 -- 172.21.15.86:0/3252742221 <== osd.4 v2:172.21.15.144:6800/4087309475 16 ==== osd_op_reply(101 [pgnls-filter start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820500 con 0x55b930e2ec00 2024-04-20T21:26:44.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.305+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:102 13.15 13:a8000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e62000 con 0x55b930e2ec00 2024-04-20T21:26:44.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.306+0000 7f8e75800640 1 -- 172.21.15.86:0/3252742221 <== osd.4 v2:172.21.15.144:6800/4087309475 17 ==== osd_op_reply(102 [pgnls-filter start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820500 con 0x55b930e2ec00 2024-04-20T21:26:44.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.306+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:103 13.35 13:ac000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e54400 con 0x55b930e2e000 2024-04-20T21:26:44.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.307+0000 7f8e74400640 1 -- 172.21.15.86:0/3252742221 <== osd.1 v2:172.21.15.86:6817/3149564961 15 ==== osd_op_reply(103 [pgnls-filter start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820780 con 0x55b930e2e000 2024-04-20T21:26:44.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.307+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:104 13.d 13:b0000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e62400 con 0x55b930e2ec00 2024-04-20T21:26:44.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.308+0000 7f8e75800640 1 -- 172.21.15.86:0/3252742221 <== osd.4 v2:172.21.15.144:6800/4087309475 18 ==== osd_op_reply(104 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820500 con 0x55b930e2ec00 2024-04-20T21:26:44.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.308+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:105 13.2d 13:b4000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e54c00 con 0x55b930e08800 2024-04-20T21:26:44.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.309+0000 7f8e74400640 1 -- 172.21.15.86:0/3252742221 <== osd.0 v2:172.21.15.86:6800/2471076035 7 ==== osd_op_reply(105 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820780 con 0x55b930e08800 2024-04-20T21:26:44.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.309+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:106 13.1d 13:b8000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e62800 con 0x55b930e09400 2024-04-20T21:26:44.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.310+0000 7f8e74e00640 1 -- 172.21.15.86:0/3252742221 <== osd.7 v2:172.21.15.144:6816/2100967057 14 ==== osd_op_reply(106 [pgnls-filter start_epoch 89 out=49b] v85'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820000 con 0x55b930e09400 2024-04-20T21:26:44.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.310+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:107 13.3d 13:bc000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e54000 con 0x55b930e09400 2024-04-20T21:26:44.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.311+0000 7f8e74e00640 1 -- 172.21.15.86:0/3252742221 <== osd.7 v2:172.21.15.144:6816/2100967057 15 ==== osd_op_reply(107 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820000 con 0x55b930e09400 2024-04-20T21:26:44.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.311+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:108 13.3 13:c0000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e62c00 con 0x55b930e08800 2024-04-20T21:26:44.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.312+0000 7f8e74400640 1 -- 172.21.15.86:0/3252742221 <== osd.0 v2:172.21.15.86:6800/2471076035 8 ==== osd_op_reply(108 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820780 con 0x55b930e08800 2024-04-20T21:26:44.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.312+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:109 13.23 13:c4000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e4ac00 con 0x55b930e2e000 2024-04-20T21:26:44.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.313+0000 7f8e74400640 1 -- 172.21.15.86:0/3252742221 <== osd.1 v2:172.21.15.86:6817/3149564961 16 ==== osd_op_reply(109 [pgnls-filter start_epoch 89 out=49b] v85'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820780 con 0x55b930e2e000 2024-04-20T21:26:44.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.313+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:110 13.13 13:c8000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e63000 con 0x55b930e09400 2024-04-20T21:26:44.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.315+0000 7f8e74e00640 1 -- 172.21.15.86:0/3252742221 <== osd.7 v2:172.21.15.144:6816/2100967057 16 ==== osd_op_reply(110 [pgnls-filter start_epoch 89 out=49b] v85'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820000 con 0x55b930e09400 2024-04-20T21:26:44.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.315+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:111 13.33 13:cc000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e4b800 con 0x55b930e09400 2024-04-20T21:26:44.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.316+0000 7f8e74e00640 1 -- 172.21.15.86:0/3252742221 <== osd.7 v2:172.21.15.144:6816/2100967057 17 ==== osd_op_reply(111 [pgnls-filter start_epoch 89 out=49b] v85'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820000 con 0x55b930e09400 2024-04-20T21:26:44.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.316+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:112 13.b 13:d0000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e63400 con 0x55b930e08400 2024-04-20T21:26:44.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.317+0000 7f8e74e00640 1 -- 172.21.15.86:0/3252742221 <== osd.5 v2:172.21.15.144:6808/1635974704 11 ==== osd_op_reply(112 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820000 con 0x55b930e08400 2024-04-20T21:26:44.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.317+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:113 13.2b 13:d4000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e2f800 con 0x55b930e2e800 2024-04-20T21:26:44.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.318+0000 7f8e74e00640 1 -- 172.21.15.86:0/3252742221 <== osd.2 v2:172.21.15.86:6816/3437611468 18 ==== osd_op_reply(113 [pgnls-filter start_epoch 89 out=49b] v85'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820000 con 0x55b930e2e800 2024-04-20T21:26:44.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.319+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:114 13.1b 13:d8000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e63800 con 0x55b930e2ec00 2024-04-20T21:26:44.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.320+0000 7f8e75800640 1 -- 172.21.15.86:0/3252742221 <== osd.4 v2:172.21.15.144:6800/4087309475 19 ==== osd_op_reply(114 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820500 con 0x55b930e2ec00 2024-04-20T21:26:44.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.320+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:115 13.3b 13:dc000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e2fc00 con 0x55b930e2ec00 2024-04-20T21:26:44.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.321+0000 7f8e75800640 1 -- 172.21.15.86:0/3252742221 <== osd.4 v2:172.21.15.144:6800/4087309475 20 ==== osd_op_reply(115 [pgnls-filter start_epoch 89 out=49b] v85'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820500 con 0x55b930e2ec00 2024-04-20T21:26:44.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.321+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:116 13.7 13:e0000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e63c00 con 0x55b930e09400 2024-04-20T21:26:44.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.322+0000 7f8e74e00640 1 -- 172.21.15.86:0/3252742221 <== osd.7 v2:172.21.15.144:6816/2100967057 18 ==== osd_op_reply(116 [pgnls-filter start_epoch 89 out=81b] v89'7 uv7 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55b931820000 con 0x55b930e09400 2024-04-20T21:26:44.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.322+0000 7f8e76d4c740 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2024-04-20T21:26:44.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.322+0000 7f8e76d4c740 10 datascan.operator(): handling object 10000000001.0 2024-04-20T21:26:44.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.322+0000 7f8e76d4c740 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:117 13.7 13:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x55b930e55c00 con 0x55b930e09400 2024-04-20T21:26:44.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.324+0000 7f8e74e00640 1 -- 172.21.15.86:0/3252742221 <== osd.7 v2:172.21.15.144:6816/2100967057 19 ==== osd_op_reply(117 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv7 ondisk = 0) ==== 416+0+157 (crc 0 0 0) 0x55b931820000 con 0x55b930e09400 2024-04-20T21:26:44.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.324+0000 7f8e76d4c740 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2024-04-20T21:26:44.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.324+0000 7f8e76d4c740 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/file 2024-04-20T21:26:44.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.324+0000 7f8e76d4c740 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=file 2024-04-20T21:26:44.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.324+0000 7f8e76d4c740 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:118 12.10 12:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x55b930e4b400 con 0x55b930e08800 2024-04-20T21:26:44.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.325+0000 7f8e74400640 1 -- 172.21.15.86:0/3252742221 <== osd.0 v2:172.21.15.86:6800/2471076035 9 ==== osd_op_reply(118 10000000000.00000000 [getxattr (59) out=59b] v0'0 uv17 ondisk = 0) ==== 164+0+59 (crc 0 0 0) 0x55b931820780 con 0x55b930e08800 2024-04-20T21:26:44.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.325+0000 7f8e76d4c740 20 datascan.get_frag_of: got backtrace parent 0x1/dir 2024-04-20T21:26:44.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.325+0000 7f8e76d4c740 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:119 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x55b930e4a800 con 0x55b930e08c00 2024-04-20T21:26:44.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.326+0000 7f8e75800640 1 -- 172.21.15.86:0/3252742221 <== osd.6 v2:172.21.15.144:6824/1893803079 18 ==== osd_op_reply(119 1.00000000 [omap-get-vals-by-keys out=520b] v0'0 uv9 ondisk = 0) ==== 154+0+520 (crc 0 0 0) 0x55b931820500 con 0x55b930e08c00 2024-04-20T21:26:44.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.326+0000 7f8e76d4c740 20 datascan.read_dentry: oid=1.00000000 dname=dir frag=*, r=0 2024-04-20T21:26:44.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.326+0000 7f8e76d4c740 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*) 2024-04-20T21:26:44.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.326+0000 7f8e76d4c740 20 datascan.get_frag_of: frag is * 2024-04-20T21:26:44.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.326+0000 7f8e76d4c740 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:120 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x55b930e4b800 con 0x55b930e08800 2024-04-20T21:26:44.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.327+0000 7f8e74400640 1 -- 172.21.15.86:0/3252742221 <== osd.0 v2:172.21.15.86:6800/2471076035 10 ==== osd_op_reply(120 10000000000.00000000 [omap-get-header out=274b] v0'0 uv17 ondisk = 0) ==== 164+0+274 (crc 0 0 0) 0x55b931820780 con 0x55b930e08800 2024-04-20T21:26:44.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.328+0000 7f8e76d4c740 10 datascan.find_or_create_dirfrag: read_version = 17 2024-04-20T21:26:44.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.328+0000 7f8e76d4c740 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2024-04-20T21:26:44.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.328+0000 7f8e76d4c740 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:121 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x55b930e4b000 con 0x55b930e08800 2024-04-20T21:26:44.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.329+0000 7f8e74400640 1 -- 172.21.15.86:0/3252742221 <== osd.0 v2:172.21.15.86:6800/2471076035 11 ==== osd_op_reply(121 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv17 ondisk = 0) ==== 164+0+4 (crc 0 0 0) 0x55b931820780 con 0x55b930e08800 2024-04-20T21:26:44.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.329+0000 7f8e76d4c740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-04-20T21:26:44.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.329+0000 7f8e76d4c740 20 datascan.read_dentry: file_head not found in result 2024-04-20T21:26:44.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.329+0000 7f8e76d4c740 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/file with size=0 bytes 2024-04-20T21:26:44.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.329+0000 7f8e76d4c740 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:122 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89) -- 0x55b930e55400 con 0x55b930e08800 2024-04-20T21:26:44.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.332+0000 7f8e74400640 1 -- 172.21.15.86:0/3252742221 <== osd.0 v2:172.21.15.86:6800/2471076035 12 ==== osd_op_reply(122 10000000000.00000000 [omap-set-vals] v89'18 uv18 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55b931820780 con 0x55b930e08800 2024-04-20T21:26:44.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.333+0000 7f8e76d4c740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-04-20T21:26:44.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.333+0000 7f8e76d4c740 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:123 13.27 13:e4000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e54000 con 0x55b930e2e000 2024-04-20T21:26:44.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.334+0000 7f8e74400640 1 -- 172.21.15.86:0/3252742221 <== osd.1 v2:172.21.15.86:6817/3149564961 17 ==== osd_op_reply(123 [pgnls-filter start_epoch 89 out=49b] v85'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820780 con 0x55b930e2e000 2024-04-20T21:26:44.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.334+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:124 13.17 13:e8000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e62800 con 0x55b930e08400 2024-04-20T21:26:44.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.335+0000 7f8e74e00640 1 -- 172.21.15.86:0/3252742221 <== osd.5 v2:172.21.15.144:6808/1635974704 12 ==== osd_op_reply(124 [pgnls-filter start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820000 con 0x55b930e08400 2024-04-20T21:26:44.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.335+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:125 13.37 13:ec000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e2f000 con 0x55b930e08400 2024-04-20T21:26:44.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.336+0000 7f8e74e00640 1 -- 172.21.15.86:0/3252742221 <== osd.5 v2:172.21.15.144:6808/1635974704 13 ==== osd_op_reply(125 [pgnls-filter start_epoch 89 out=49b] v85'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820000 con 0x55b930e08400 2024-04-20T21:26:44.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.336+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:126 13.f 13:f0000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e63000 con 0x55b930e08c00 2024-04-20T21:26:44.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.337+0000 7f8e75800640 1 -- 172.21.15.86:0/3252742221 <== osd.6 v2:172.21.15.144:6824/1893803079 19 ==== osd_op_reply(126 [pgnls-filter start_epoch 89 out=49b] v85'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820500 con 0x55b930e08c00 2024-04-20T21:26:44.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.338+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:127 13.2f 13:f4000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e2e400 con 0x55b930e2ec00 2024-04-20T21:26:44.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.339+0000 7f8e75800640 1 -- 172.21.15.86:0/3252742221 <== osd.4 v2:172.21.15.144:6800/4087309475 21 ==== osd_op_reply(127 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820500 con 0x55b930e2ec00 2024-04-20T21:26:44.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.339+0000 7f8e6ea00640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:128 13.1f 13:f8000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e63400 con 0x55b930e08c00 2024-04-20T21:26:44.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.340+0000 7f8e75800640 1 -- 172.21.15.86:0/3252742221 <== osd.6 v2:172.21.15.144:6824/1893803079 20 ==== osd_op_reply(128 [pgnls-filter start_epoch 89 out=49b] v85'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820500 con 0x55b930e08c00 2024-04-20T21:26:44.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.340+0000 7f8e6f400640 1 -- 172.21.15.86:0/3252742221 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:129 13.3f 13:fc000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x55b930e2f400 con 0x55b930e08c00 2024-04-20T21:26:44.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.341+0000 7f8e75800640 1 -- 172.21.15.86:0/3252742221 <== osd.6 v2:172.21.15.144:6824/1893803079 21 ==== osd_op_reply(129 [pgnls-filter start_epoch 89 out=49b] v85'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b931820500 con 0x55b930e08c00 2024-04-20T21:26:44.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.341+0000 7f8e76d4c740 1 -- 172.21.15.86:0/3252742221 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x55b930e08800 msgr2=0x55b930e35080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:44.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.341+0000 7f8e76d4c740 1 --2- 172.21.15.86:0/3252742221 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x55b930e08800 0x55b930e35080 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.341+0000 7f8e76d4c740 1 -- 172.21.15.86:0/3252742221 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x55b930e2e000 msgr2=0x55b930e34000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:44.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.341+0000 7f8e76d4c740 1 --2- 172.21.15.86:0/3252742221 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x55b930e2e000 0x55b930e34000 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.341+0000 7f8e76d4c740 1 -- 172.21.15.86:0/3252742221 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x55b930e2e800 msgr2=0x55b930e34b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:44.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.341+0000 7f8e76d4c740 1 --2- 172.21.15.86:0/3252742221 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x55b930e2e800 0x55b930e34b00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.341+0000 7f8e76d4c740 1 -- 172.21.15.86:0/3252742221 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x55b930e4a400 msgr2=0x55b930e35600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:44.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.341+0000 7f8e76d4c740 1 --2- 172.21.15.86:0/3252742221 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x55b930e4a400 0x55b930e35600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.341+0000 7f8e76d4c740 1 -- 172.21.15.86:0/3252742221 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x55b930e2ec00 msgr2=0x55b930e34580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:44.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.341+0000 7f8e76d4c740 1 --2- 172.21.15.86:0/3252742221 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x55b930e2ec00 0x55b930e34580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.341+0000 7f8e76d4c740 1 -- 172.21.15.86:0/3252742221 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x55b930e08400 msgr2=0x55b931868c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:44.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.341+0000 7f8e76d4c740 1 --2- 172.21.15.86:0/3252742221 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x55b930e08400 0x55b931868c00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.341+0000 7f8e76d4c740 1 -- 172.21.15.86:0/3252742221 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x55b930e08c00 msgr2=0x55b931869180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:44.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.341+0000 7f8e76d4c740 1 --2- 172.21.15.86:0/3252742221 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x55b930e08c00 0x55b931869180 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.341+0000 7f8e76d4c740 1 -- 172.21.15.86:0/3252742221 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x55b930e09400 msgr2=0x55b931869700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:44.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.341+0000 7f8e76d4c740 1 --2- 172.21.15.86:0/3252742221 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x55b930e09400 0x55b931869700 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.342+0000 7f8e74400640 1 -- 172.21.15.86:0/3252742221 reap_dead start 2024-04-20T21:26:44.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.342+0000 7f8e74400640 1 -- 172.21.15.86:0/3252742221 reap_dead start 2024-04-20T21:26:44.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.342+0000 7f8e76d4c740 1 -- 172.21.15.86:0/3252742221 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x55b930e08000 msgr2=0x55b931868680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:44.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.342+0000 7f8e76d4c740 1 --2- 172.21.15.86:0/3252742221 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x55b930e08000 0x55b931868680 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x55b931976f00 tx=0x55b93197cc00 comp rx=0 tx=0).stop 2024-04-20T21:26:44.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.342+0000 7f8e76d4c740 1 -- 172.21.15.86:0/3252742221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93195d000 msgr2=0x55b931868100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:44.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.342+0000 7f8e76d4c740 1 --2- 172.21.15.86:0/3252742221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93195d000 0x55b931868100 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x55b931977c50 tx=0x55b93197cde0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.343+0000 7f8e76d4c740 1 -- 172.21.15.86:0/3252742221 shutdown_connections 2024-04-20T21:26:44.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.343+0000 7f8e76d4c740 1 --2- 172.21.15.86:0/3252742221 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x55b930e08000 0x55b931868680 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.343+0000 7f8e76d4c740 1 --2- 172.21.15.86:0/3252742221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b93195d000 0x55b931868100 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.343+0000 7f8e76d4c740 1 -- 172.21.15.86:0/3252742221 >> 172.21.15.86:0/3252742221 conn(0x55b93195c400 msgr2=0x55b930b61000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:44.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.343+0000 7f8e76d4c740 1 -- 172.21.15.86:0/3252742221 shutdown_connections 2024-04-20T21:26:44.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.343+0000 7f8e76d4c740 1 -- 172.21.15.86:0/3252742221 wait complete. 2024-04-20T21:26:44.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.344+0000 7f8e76d4c740 1 -- 172.21.15.86:0/3213281158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b931830000 msgr2=0x55b931866580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:44.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.344+0000 7f8e76d4c740 1 --2- 172.21.15.86:0/3213281158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b931830000 0x55b931866580 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x55b9318b8f60 tx=0x55b9318328a0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.344+0000 7f8e76d4c740 1 -- 172.21.15.86:0/3213281158 shutdown_connections 2024-04-20T21:26:44.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.344+0000 7f8e76d4c740 1 --2- 172.21.15.86:0/3213281158 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b93195c000 0x55b931867080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.344+0000 7f8e76d4c740 1 --2- 172.21.15.86:0/3213281158 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b931831c00 0x55b931866000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.345+0000 7f8e76d4c740 1 --2- 172.21.15.86:0/3213281158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b931830000 0x55b931866580 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.345+0000 7f8e76d4c740 1 -- 172.21.15.86:0/3213281158 >> 172.21.15.86:0/3213281158 conn(0x55b930a93000 msgr2=0x55b930b60000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:44.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.345+0000 7f8e76d4c740 1 -- 172.21.15.86:0/3213281158 shutdown_connections 2024-04-20T21:26:44.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.345+0000 7f8e76d4c740 1 -- 172.21.15.86:0/3213281158 wait complete. 2024-04-20T21:26:44.357 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', 'cephfs_data'] in time 0:00:00.268419, result: 2024-04-20T21:26:44.357 DEBUG:teuthology.orchestra.run.smithi086:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-04-20T21:26:44.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.464+0000 7fc0f280c740 1 -- start start 2024-04-20T21:26:44.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.464+0000 7fc0f280c740 1 Processor -- start 2024-04-20T21:26:44.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.465+0000 7fc0f280c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c03d18c00 0x556c04a8e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.465+0000 7fc0f280c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c03d19800 0x556c04a8e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.465+0000 7fc0f280c740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556c03d19c00 0x556c04a8eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.465+0000 7fc0f280c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556c04b02180 con 0x556c03d18c00 2024-04-20T21:26:44.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.465+0000 7fc0f280c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556c04b02300 con 0x556c03d19800 2024-04-20T21:26:44.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.465+0000 7fc0f280c740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x556c04b02480 con 0x556c03d19c00 2024-04-20T21:26:44.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.465+0000 7fc0f0000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c03d18c00 0x556c04a8e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.465+0000 7fc0f0000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c03d18c00 0x556c04a8e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45698/0 (socket says 172.21.15.86:45698) 2024-04-20T21:26:44.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.465+0000 7fc0f0000640 1 -- 172.21.15.86:0/4008134708 learned_addr learned my addr 172.21.15.86:0/4008134708 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:44.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.466+0000 7fc0ef600640 1 --2- 172.21.15.86:0/4008134708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c03d19800 0x556c04a8e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.466+0000 7fc0f0a00640 1 --2- 172.21.15.86:0/4008134708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556c03d19c00 0x556c04a8eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.466+0000 7fc0f0000640 1 -- 172.21.15.86:0/4008134708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556c03d19c00 msgr2=0x556c04a8eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:44.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.466+0000 7fc0f0000640 1 --2- 172.21.15.86:0/4008134708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556c03d19c00 0x556c04a8eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.466+0000 7fc0f0000640 1 -- 172.21.15.86:0/4008134708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c03d19800 msgr2=0x556c04a8e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:44.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.466+0000 7fc0f0000640 1 --2- 172.21.15.86:0/4008134708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c03d19800 0x556c04a8e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.466+0000 7fc0f0000640 1 -- 172.21.15.86:0/4008134708 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c03d45680 con 0x556c03d18c00 2024-04-20T21:26:44.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.466+0000 7fc0f0000640 1 --2- 172.21.15.86:0/4008134708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c03d18c00 0x556c04a8e000 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x556c04b3b170 tx=0x556c04ab8a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dae661f62abce5c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:44.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.466+0000 7fc0ece00640 1 -- 172.21.15.86:0/4008134708 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556c03d3e9c0 con 0x556c03d18c00 2024-04-20T21:26:44.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.467+0000 7fc0ece00640 1 -- 172.21.15.86:0/4008134708 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x556c03d621c0 con 0x556c03d18c00 2024-04-20T21:26:44.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.467+0000 7fc0ece00640 1 -- 172.21.15.86:0/4008134708 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556c03d3e680 con 0x556c03d18c00 2024-04-20T21:26:44.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.467+0000 7fc0f280c740 1 -- 172.21.15.86:0/4008134708 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c03d45860 con 0x556c03d18c00 2024-04-20T21:26:44.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.467+0000 7fc0f280c740 1 -- 172.21.15.86:0/4008134708 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=0}) -- 0x556c03d452c0 con 0x556c03d18c00 2024-04-20T21:26:44.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.467+0000 7fc0f280c740 4 waiting for MDS map... 2024-04-20T21:26:44.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.467+0000 7fc0ece00640 1 -- 172.21.15.86:0/4008134708 <== mon.0 v2:172.21.15.86:3300/0 4 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x556c04bf8200 con 0x556c03d18c00 2024-04-20T21:26:44.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.467+0000 7fc0ece00640 1 -- 172.21.15.86:0/4008134708 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 67) ==== 2886+0+0 (secure 0 0 0) 0x556c03d71180 con 0x556c03d18c00 2024-04-20T21:26:44.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.467+0000 7fc0f280c740 4 Got MDS map 67 2024-04-20T21:26:44.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.467+0000 7fc0f280c740 4 datascan.main: Using metadata pool output 2024-04-20T21:26:44.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.467+0000 7fc0f280c740 4 datascan.main: connecting to RADOS... 2024-04-20T21:26:44.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.468+0000 7fc0f280c740 1 Processor -- start 2024-04-20T21:26:44.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.468+0000 7fc0f280c740 1 -- start start 2024-04-20T21:26:44.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.469+0000 7fc0f280c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c04c40000 0x556c04a8f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.469+0000 7fc0f280c740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556c04c40400 0x556c04a8f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.469+0000 7fc0f280c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c04c40800 0x556c04a8fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.469+0000 7fc0f280c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556c04b02600 con 0x556c04c40800 2024-04-20T21:26:44.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.469+0000 7fc0f280c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556c04b02780 con 0x556c04c40000 2024-04-20T21:26:44.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.469+0000 7fc0f280c740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x556c04b02900 con 0x556c04c40400 2024-04-20T21:26:44.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.469+0000 7fc0ef600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c04c40800 0x556c04a8fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.469+0000 7fc0ef600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c04c40800 0x556c04a8fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45704/0 (socket says 172.21.15.86:45704) 2024-04-20T21:26:44.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.469+0000 7fc0ef600640 1 -- 172.21.15.86:0/2373733064 learned_addr learned my addr 172.21.15.86:0/2373733064 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:44.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.469+0000 7fc0f0000640 1 --2- 172.21.15.86:0/2373733064 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556c04c40400 0x556c04a8f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.470+0000 7fc0f0a00640 1 --2- 172.21.15.86:0/2373733064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c04c40000 0x556c04a8f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.470+0000 7fc0ef600640 1 -- 172.21.15.86:0/2373733064 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556c04c40400 msgr2=0x556c04a8f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:44.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.470+0000 7fc0ef600640 1 --2- 172.21.15.86:0/2373733064 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556c04c40400 0x556c04a8f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.470+0000 7fc0ef600640 1 -- 172.21.15.86:0/2373733064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c04c40000 msgr2=0x556c04a8f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:44.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.470+0000 7fc0ef600640 1 --2- 172.21.15.86:0/2373733064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c04c40000 0x556c04a8f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.470+0000 7fc0ef600640 1 -- 172.21.15.86:0/2373733064 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c03d454a0 con 0x556c04c40800 2024-04-20T21:26:44.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.470+0000 7fc0f0a00640 1 --2- 172.21.15.86:0/2373733064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c04c40000 0x556c04a8f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:26:44.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.470+0000 7fc0ef600640 1 --2- 172.21.15.86:0/2373733064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c04c40800 0x556c04a8fb80 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x556c04bf2f30 tx=0x556c04afec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1536b6672127c62f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:44.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.470+0000 7fc0ea600640 1 -- 172.21.15.86:0/2373733064 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556c03d3e9c0 con 0x556c04c40800 2024-04-20T21:26:44.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.470+0000 7fc0ea600640 1 -- 172.21.15.86:0/2373733064 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x556c03d62e00 con 0x556c04c40800 2024-04-20T21:26:44.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.470+0000 7fc0ea600640 1 -- 172.21.15.86:0/2373733064 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556c03d3e680 con 0x556c04c40800 2024-04-20T21:26:44.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.471+0000 7fc0f280c740 1 -- 172.21.15.86:0/2373733064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c04c40800 msgr2=0x556c04a8fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:44.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.471+0000 7fc0f280c740 1 --2- 172.21.15.86:0/2373733064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c04c40800 0x556c04a8fb80 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x556c04bf2f30 tx=0x556c04afec00 comp rx=0 tx=0).stop 2024-04-20T21:26:44.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.471+0000 7fc0f280c740 1 -- 172.21.15.86:0/2373733064 shutdown_connections 2024-04-20T21:26:44.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.471+0000 7fc0f280c740 1 --2- 172.21.15.86:0/2373733064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c04c40800 0x556c04a8fb80 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.471+0000 7fc0f280c740 1 --2- 172.21.15.86:0/2373733064 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556c04c40400 0x556c04a8f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.471+0000 7fc0f280c740 1 --2- 172.21.15.86:0/2373733064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c04c40000 0x556c04a8f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.471+0000 7fc0f280c740 1 -- 172.21.15.86:0/2373733064 >> 172.21.15.86:0/2373733064 conn(0x556c04ab7c00 msgr2=0x556c03e05000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:44.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.471+0000 7fc0f280c740 1 -- 172.21.15.86:0/2373733064 shutdown_connections 2024-04-20T21:26:44.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.471+0000 7fc0f280c740 1 -- 172.21.15.86:0/2373733064 wait complete. 2024-04-20T21:26:44.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.472+0000 7fc0f280c740 1 Processor -- start 2024-04-20T21:26:44.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.472+0000 7fc0f280c740 1 -- start start 2024-04-20T21:26:44.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.473+0000 7fc0f280c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c04c40000 0x556c04a8f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.473+0000 7fc0f280c740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556c04c40400 0x556c04a8f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.473+0000 7fc0f280c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c04c40800 0x556c04a8fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.473+0000 7fc0f280c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556c04b03c80 con 0x556c04c40000 2024-04-20T21:26:44.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.473+0000 7fc0f280c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x556c04b03b00 con 0x556c04c40800 2024-04-20T21:26:44.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.473+0000 7fc0f280c740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x556c04b03980 con 0x556c04c40400 2024-04-20T21:26:44.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.473+0000 7fc0f0a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c04c40000 0x556c04a8f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.473+0000 7fc0f0a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c04c40000 0x556c04a8f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45708/0 (socket says 172.21.15.86:45708) 2024-04-20T21:26:44.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.473+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 learned_addr learned my addr 172.21.15.86:0/2303541732 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:44.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.473+0000 7fc0ef600640 1 --2- 172.21.15.86:0/2303541732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c04c40800 0x556c04a8fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.473+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556c04c40400 msgr2=0x556c04a8f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:44.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.473+0000 7fc0f0000640 1 --2- 172.21.15.86:0/2303541732 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556c04c40400 0x556c04a8f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.473+0000 7fc0f0a00640 1 --2- 172.21.15.86:0/2303541732 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556c04c40400 0x556c04a8f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.474+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c04c40800 msgr2=0x556c04a8fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:44.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.474+0000 7fc0f0a00640 1 --2- 172.21.15.86:0/2303541732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c04c40800 0x556c04a8fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.474+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c03d450e0 con 0x556c04c40000 2024-04-20T21:26:44.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.474+0000 7fc0f0a00640 1 --2- 172.21.15.86:0/2303541732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c04c40000 0x556c04a8f080 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x556c04bf3c20 tx=0x556c04afed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:44.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.474+0000 7fc0e8800640 1 -- 172.21.15.86:0/2303541732 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556c03d3e9c0 con 0x556c04c40000 2024-04-20T21:26:44.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.474+0000 7fc0e8800640 1 -- 172.21.15.86:0/2303541732 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x556c03d62c40 con 0x556c04c40000 2024-04-20T21:26:44.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.474+0000 7fc0e8800640 1 -- 172.21.15.86:0/2303541732 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556c03d3e680 con 0x556c04c40000 2024-04-20T21:26:44.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.474+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556c03d45a40 con 0x556c04c40000 2024-04-20T21:26:44.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.474+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c03d443c0 con 0x556c04c40000 2024-04-20T21:26:44.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.474+0000 7fc0f280c740 4 datascan.init: resolving metadata pool 12 2024-04-20T21:26:44.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.476+0000 7fc0e8800640 1 -- 172.21.15.86:0/2303541732 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x556c04c5cc00 con 0x556c04c40000 2024-04-20T21:26:44.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.477+0000 7fc0e8800640 1 --2- 172.21.15.86:0/2303541732 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x556c04c6cc00 0x556c04a90100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.477+0000 7fc0e8800640 1 -- 172.21.15.86:0/2303541732 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x556c04c47a00 con 0x556c04c40000 2024-04-20T21:26:44.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.477+0000 7fc0f280c740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-20T21:26:44.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.477+0000 7fc0f280c740 4 datascan.main: resolving metadata pool 12 2024-04-20T21:26:44.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.477+0000 7fc0f280c740 1 --2- 172.21.15.86:0/2303541732 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x556c04c40c00 0x556c04a90680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.477+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:1 12.0 12:00000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c04c41000 con 0x556c04c40c00 2024-04-20T21:26:44.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.477+0000 7fc0f0000640 1 --2- 172.21.15.86:0/2303541732 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x556c04c6cc00 0x556c04a90100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.478+0000 7fc0ef600640 1 --2- 172.21.15.86:0/2303541732 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x556c04c40c00 0x556c04a90680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.478+0000 7fc0f0000640 1 --2- 172.21.15.86:0/2303541732 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x556c04c6cc00 0x556c04a90100 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x556c04b3bd10 tx=0x556c03dccd20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:44.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.479+0000 7fc0ef600640 1 --2- 172.21.15.86:0/2303541732 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x556c04c40c00 0x556c04a90680 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:44.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.480+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 <== osd.4 v2:172.21.15.144:6800/4087309475 1 ==== osd_op_reply(1 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556c04aa6000 con 0x556c04c40c00 2024-04-20T21:26:44.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.480+0000 7fc0ef600640 1 --2- 172.21.15.86:0/2303541732 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x556c04c41c00 0x556c040a8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.480+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:2 12.10 12:08000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c04c41400 con 0x556c04c41c00 2024-04-20T21:26:44.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.480+0000 7fc0f0a00640 1 --2- 172.21.15.86:0/2303541732 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x556c04c41c00 0x556c040a8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.481+0000 7fc0f0a00640 1 --2- 172.21.15.86:0/2303541732 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x556c04c41c00 0x556c040a8000 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:44.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.481+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.0 v2:172.21.15.86:6800/2471076035 1 ==== osd_op_reply(2 [pgnls start_epoch 89 out=177b] v89'18 uv18 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c41c00 2024-04-20T21:26:44.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.481+0000 7fc0f0a00640 1 --2- 172.21.15.86:0/2303541732 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x556c04c41400 0x556c040a8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.481+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:3 12.8 12:10000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c0402a000 con 0x556c04c41400 2024-04-20T21:26:44.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.481+0000 7fc0f0000640 1 --2- 172.21.15.86:0/2303541732 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x556c04c41400 0x556c040a8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.482+0000 7fc0f0000640 1 --2- 172.21.15.86:0/2303541732 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x556c04c41400 0x556c040a8580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:44.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.482+0000 7fc0f0000640 1 -- 172.21.15.86:0/2303541732 <== osd.3 v2:172.21.15.86:6808/794135519 1 ==== osd_op_reply(3 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556c04aa6780 con 0x556c04c41400 2024-04-20T21:26:44.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.482+0000 7fc0f0000640 1 --2- 172.21.15.86:0/2303541732 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x556c0402a000 0x556c040a8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.482+0000 7fc0f0000640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:4 12.18 12:18000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c04c6d800 con 0x556c0402a000 2024-04-20T21:26:44.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.483+0000 7fc0ef600640 1 --2- 172.21.15.86:0/2303541732 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x556c0402a000 0x556c040a8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.483+0000 7fc0ef600640 1 --2- 172.21.15.86:0/2303541732 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x556c0402a000 0x556c040a8b00 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:44.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.484+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 <== osd.1 v2:172.21.15.86:6817/3149564961 1 ==== osd_op_reply(4 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556c04aa6000 con 0x556c0402a000 2024-04-20T21:26:44.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.484+0000 7fc0ef600640 1 --2- 172.21.15.86:0/2303541732 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x556c04c6d800 0x556c040a9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.484+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:5 12.4 12:20000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c04c41800 con 0x556c04c6d800 2024-04-20T21:26:44.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.484+0000 7fc0f0a00640 1 --2- 172.21.15.86:0/2303541732 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x556c04c6d800 0x556c040a9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.485+0000 7fc0f0a00640 1 --2- 172.21.15.86:0/2303541732 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x556c04c6d800 0x556c040a9080 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:44.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.486+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.6 v2:172.21.15.144:6824/1893803079 1 ==== osd_op_reply(5 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c6d800 2024-04-20T21:26:44.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.486+0000 7fc0f0a00640 1 --2- 172.21.15.86:0/2303541732 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x556c04c41800 0x556c040a9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.486+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:6 12.14 12:28000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c0402a400 con 0x556c04c41800 2024-04-20T21:26:44.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.487+0000 7fc0f0000640 1 --2- 172.21.15.86:0/2303541732 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x556c04c41800 0x556c040a9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.488+0000 7fc0f0000640 1 --2- 172.21.15.86:0/2303541732 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x556c04c41800 0x556c040a9b80 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:44.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.489+0000 7fc0f0000640 1 -- 172.21.15.86:0/2303541732 <== osd.5 v2:172.21.15.144:6808/1635974704 1 ==== osd_op_reply(6 [pgnls start_epoch 89 out=149b] v88'28 uv28 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x556c04aa6780 con 0x556c04c41800 2024-04-20T21:26:44.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.489+0000 7fc0f0000640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:7 12.c 12:30000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c0402a400 con 0x556c04c41c00 2024-04-20T21:26:44.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.489+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.0 v2:172.21.15.86:6800/2471076035 2 ==== osd_op_reply(7 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c41c00 2024-04-20T21:26:44.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.489+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:8 12.1c 12:38000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c0402a400 con 0x556c04c41c00 2024-04-20T21:26:44.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.490+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.0 v2:172.21.15.86:6800/2471076035 3 ==== osd_op_reply(8 [pgnls start_epoch 89 out=73b] v88'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c41c00 2024-04-20T21:26:44.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.490+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:9 12.2 12:40000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c0402a400 con 0x556c04c40c00 2024-04-20T21:26:44.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.490+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 <== osd.4 v2:172.21.15.144:6800/4087309475 2 ==== osd_op_reply(9 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556c04aa6000 con 0x556c04c40c00 2024-04-20T21:26:44.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.490+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:10 12.12 12:48000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c0402a400 con 0x556c04c41400 2024-04-20T21:26:44.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.491+0000 7fc0f0000640 1 -- 172.21.15.86:0/2303541732 <== osd.3 v2:172.21.15.86:6808/794135519 2 ==== osd_op_reply(10 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556c04aa6780 con 0x556c04c41400 2024-04-20T21:26:44.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.491+0000 7fc0f0000640 1 --2- 172.21.15.86:0/2303541732 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x556c0402a400 0x556c040aa680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.491+0000 7fc0f0000640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:11 12.a 12:50000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c04c6dc00 con 0x556c0402a400 2024-04-20T21:26:44.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.491+0000 7fc0ef600640 1 --2- 172.21.15.86:0/2303541732 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x556c0402a400 0x556c040aa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.492+0000 7fc0ef600640 1 --2- 172.21.15.86:0/2303541732 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x556c0402a400 0x556c040aa680 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:44.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.492+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 <== osd.2 v2:172.21.15.86:6816/3437611468 1 ==== osd_op_reply(11 [pgnls start_epoch 89 out=81b] v88'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x556c04aa6000 con 0x556c0402a400 2024-04-20T21:26:44.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.492+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:12 12.1a 12:58000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c04c6dc00 con 0x556c04c40c00 2024-04-20T21:26:44.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.493+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 <== osd.4 v2:172.21.15.144:6800/4087309475 3 ==== osd_op_reply(12 [pgnls start_epoch 89 out=49b] v86'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556c04aa6000 con 0x556c04c40c00 2024-04-20T21:26:44.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.493+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:13 12.6 12:60000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c04c6dc00 con 0x556c0402a000 2024-04-20T21:26:44.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.493+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 <== osd.1 v2:172.21.15.86:6817/3149564961 2 ==== osd_op_reply(13 [pgnls start_epoch 89 out=77b] v88'9 uv9 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x556c04aa6000 con 0x556c0402a000 2024-04-20T21:26:44.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.493+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:14 12.16 12:68000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c04c6dc00 con 0x556c04c6d800 2024-04-20T21:26:44.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.494+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.6 v2:172.21.15.144:6824/1893803079 2 ==== osd_op_reply(14 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c6d800 2024-04-20T21:26:44.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.494+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:15 12.e 12:70000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c04c6dc00 con 0x556c04c41c00 2024-04-20T21:26:44.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.495+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.0 v2:172.21.15.86:6800/2471076035 4 ==== osd_op_reply(15 [pgnls start_epoch 89 out=73b] v88'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c41c00 2024-04-20T21:26:44.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.495+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:16 12.1e 12:78000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c04c6dc00 con 0x556c04c40c00 2024-04-20T21:26:44.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.496+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 <== osd.4 v2:172.21.15.144:6800/4087309475 4 ==== osd_op_reply(16 [pgnls start_epoch 89 out=49b] v86'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556c04aa6000 con 0x556c04c40c00 2024-04-20T21:26:44.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.496+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:17 12.1 12:80000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c04c6dc00 con 0x556c04c6d800 2024-04-20T21:26:44.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.497+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.6 v2:172.21.15.144:6824/1893803079 3 ==== osd_op_reply(17 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c6d800 2024-04-20T21:26:44.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.497+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:18 12.11 12:88000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c04c6dc00 con 0x556c04c41800 2024-04-20T21:26:44.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.499+0000 7fc0f0000640 1 -- 172.21.15.86:0/2303541732 <== osd.5 v2:172.21.15.144:6808/1635974704 2 ==== osd_op_reply(18 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556c04aa6780 con 0x556c04c41800 2024-04-20T21:26:44.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.499+0000 7fc0f0000640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:19 12.9 12:90000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c04c6dc00 con 0x556c0402a400 2024-04-20T21:26:44.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.499+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 <== osd.2 v2:172.21.15.86:6816/3437611468 2 ==== osd_op_reply(19 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556c04aa6000 con 0x556c0402a400 2024-04-20T21:26:44.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.499+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:20 12.19 12:98000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c04c6dc00 con 0x556c04c41c00 2024-04-20T21:26:44.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.500+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.0 v2:172.21.15.86:6800/2471076035 5 ==== osd_op_reply(20 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c41c00 2024-04-20T21:26:44.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.500+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:21 12.5 12:a0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c04c6dc00 con 0x556c04c6d800 2024-04-20T21:26:44.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.501+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.6 v2:172.21.15.144:6824/1893803079 4 ==== osd_op_reply(21 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c6d800 2024-04-20T21:26:44.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.501+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:22 12.15 12:a8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c04c6dc00 con 0x556c04c6d800 2024-04-20T21:26:44.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.502+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.6 v2:172.21.15.144:6824/1893803079 5 ==== osd_op_reply(22 [pgnls start_epoch 89 out=97b] v88'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c6d800 2024-04-20T21:26:44.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.502+0000 7fc0f0a00640 1 --2- 172.21.15.86:0/2303541732 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x556c04c6dc00 0x556c040aa100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.502+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:23 12.d 12:b0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c0402a800 con 0x556c04c6dc00 2024-04-20T21:26:44.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.502+0000 7fc0f0a00640 1 --2- 172.21.15.86:0/2303541732 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x556c04c6dc00 0x556c040aa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.504+0000 7fc0f0a00640 1 --2- 172.21.15.86:0/2303541732 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x556c04c6dc00 0x556c040aa100 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:44.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.504+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.7 v2:172.21.15.144:6816/2100967057 1 ==== osd_op_reply(23 [pgnls start_epoch 89 out=74b] v84'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c6dc00 2024-04-20T21:26:44.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.505+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:24 12.1d 12:b8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c0402a800 con 0x556c04c41c00 2024-04-20T21:26:44.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.505+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.0 v2:172.21.15.86:6800/2471076035 6 ==== osd_op_reply(24 [pgnls start_epoch 89 out=73b] v88'5 uv5 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c41c00 2024-04-20T21:26:44.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.505+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:25 12.3 12:c0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c0402a800 con 0x556c04c41c00 2024-04-20T21:26:44.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.506+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.0 v2:172.21.15.86:6800/2471076035 7 ==== osd_op_reply(25 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c41c00 2024-04-20T21:26:44.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.506+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:26 12.13 12:c8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c0402a800 con 0x556c0402a000 2024-04-20T21:26:44.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.506+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 <== osd.1 v2:172.21.15.86:6817/3149564961 3 ==== osd_op_reply(26 [pgnls start_epoch 89 out=98b] v88'4 uv4 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x556c04aa6000 con 0x556c0402a000 2024-04-20T21:26:44.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.506+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:27 12.b 12:d0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c0402a800 con 0x556c04c41400 2024-04-20T21:26:44.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.507+0000 7fc0f0000640 1 -- 172.21.15.86:0/2303541732 <== osd.3 v2:172.21.15.86:6808/794135519 3 ==== osd_op_reply(27 [pgnls start_epoch 89 out=76b] v88'6 uv6 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x556c04aa6780 con 0x556c04c41400 2024-04-20T21:26:44.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.507+0000 7fc0f0000640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:28 12.1b 12:d8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c0402a800 con 0x556c04c40c00 2024-04-20T21:26:44.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.508+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 <== osd.4 v2:172.21.15.144:6800/4087309475 5 ==== osd_op_reply(28 [pgnls start_epoch 89 out=73b] v88'10 uv10 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x556c04aa6000 con 0x556c04c40c00 2024-04-20T21:26:44.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.508+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:29 12.7 12:e0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c0402a800 con 0x556c0402a000 2024-04-20T21:26:44.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.508+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 <== osd.1 v2:172.21.15.86:6817/3149564961 4 ==== osd_op_reply(29 [pgnls start_epoch 89 out=73b] v88'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x556c04aa6000 con 0x556c0402a000 2024-04-20T21:26:44.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.508+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:30 12.17 12:e8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c0402a800 con 0x556c04c6d800 2024-04-20T21:26:44.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.510+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.6 v2:172.21.15.144:6824/1893803079 6 ==== osd_op_reply(30 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c6d800 2024-04-20T21:26:44.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.510+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:31 12.f 12:f0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c0402a800 con 0x556c04c6d800 2024-04-20T21:26:44.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.511+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.6 v2:172.21.15.144:6824/1893803079 7 ==== osd_op_reply(31 [pgnls start_epoch 89 out=73b] v84'1 uv1 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c6d800 2024-04-20T21:26:44.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.511+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:32 12.1f 12:f8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c0402a800 con 0x556c04c6d800 2024-04-20T21:26:44.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.512+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.6 v2:172.21.15.144:6824/1893803079 8 ==== osd_op_reply(32 [pgnls start_epoch 89 out=101b] v88'9 uv9 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c6d800 2024-04-20T21:26:44.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.512+0000 7fc0f280c740 10 datascan.scan_links: step 1: handling object 601.00000000 2024-04-20T21:26:44.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.512+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:33 12.10 12:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556c0402ac00 con 0x556c04c41c00 2024-04-20T21:26:44.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.513+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.0 v2:172.21.15.86:6800/2471076035 8 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c41c00 2024-04-20T21:26:44.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.513+0000 7fc0f280c740 10 datascan.scan_links: step 1: handling object 602.00000000 2024-04-20T21:26:44.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.513+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:34 12.10 12:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556c0402b000 con 0x556c04c41c00 2024-04-20T21:26:44.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.513+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.0 v2:172.21.15.86:6800/2471076035 9 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c41c00 2024-04-20T21:26:44.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.513+0000 7fc0f280c740 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-04-20T21:26:44.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.513+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:35 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556c0402b400 con 0x556c04c41c00 2024-04-20T21:26:44.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.514+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.0 v2:172.21.15.86:6800/2471076035 10 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c41c00 2024-04-20T21:26:44.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.514+0000 7fc0f280c740 20 datascan.scan_links: injected ino detected 2024-04-20T21:26:44.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.514+0000 7fc0f280c740 10 datascan.scan_links: step 1: handling object 600.00000000 2024-04-20T21:26:44.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.514+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:36 12.10 12:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556c0402b800 con 0x556c04c41c00 2024-04-20T21:26:44.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.515+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.0 v2:172.21.15.86:6800/2471076035 11 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c41c00 2024-04-20T21:26:44.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.515+0000 7fc0f280c740 10 datascan.scan_links: step 1: handling object 603.00000000 2024-04-20T21:26:44.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.515+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:37 12.10 12:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556c0402bc00 con 0x556c04c41c00 2024-04-20T21:26:44.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.515+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.0 v2:172.21.15.86:6800/2471076035 12 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c41c00 2024-04-20T21:26:44.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.515+0000 7fc0f280c740 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-04-20T21:26:44.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.515+0000 7fc0f280c740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-20T21:26:44.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.515+0000 7fc0f280c740 10 datascan.scan_links: step 1: handling object 200.00000000 2024-04-20T21:26:44.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.515+0000 7fc0f280c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-20T21:26:44.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.515+0000 7fc0f280c740 10 datascan.scan_links: step 1: handling object 200.00000001 2024-04-20T21:26:44.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.515+0000 7fc0f280c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-20T21:26:44.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.515+0000 7fc0f280c740 10 datascan.scan_links: step 1: handling object 606.00000000 2024-04-20T21:26:44.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.515+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:38 12.14 12:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556c0402a800 con 0x556c04c41800 2024-04-20T21:26:44.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.516+0000 7fc0f0000640 1 -- 172.21.15.86:0/2303541732 <== osd.5 v2:172.21.15.144:6808/1635974704 3 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv26 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556c04aa6780 con 0x556c04c41800 2024-04-20T21:26:44.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.516+0000 7fc0f280c740 10 datascan.scan_links: step 1: handling object 607.00000000 2024-04-20T21:26:44.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.516+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:39 12.1c 12:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556c0402ac00 con 0x556c04c41c00 2024-04-20T21:26:44.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.517+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.0 v2:172.21.15.86:6800/2471076035 13 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c41c00 2024-04-20T21:26:44.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.517+0000 7fc0f280c740 10 datascan.scan_links: step 1: handling object 100000001f6.00000000 2024-04-20T21:26:44.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.517+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:40 12.a 12:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556c0402b000 con 0x556c0402a400 2024-04-20T21:26:44.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.518+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 <== osd.2 v2:172.21.15.86:6816/3437611468 3 ==== osd_op_reply(40 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) ==== 164+0+54 (crc 0 0 0) 0x556c04aa6000 con 0x556c0402a400 2024-04-20T21:26:44.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.518+0000 7fc0f280c740 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-04-20T21:26:44.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.518+0000 7fc0f280c740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-20T21:26:44.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.518+0000 7fc0f280c740 10 datascan.scan_links: step 1: handling object 608.00000000 2024-04-20T21:26:44.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.518+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:41 12.e 12:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556c0402b400 con 0x556c04c41c00 2024-04-20T21:26:44.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.519+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.0 v2:172.21.15.86:6800/2471076035 14 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c41c00 2024-04-20T21:26:44.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.519+0000 7fc0f280c740 10 datascan.scan_links: step 1: handling object 604.00000000 2024-04-20T21:26:44.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.519+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:42 12.15 12:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556c0402b800 con 0x556c04c6d800 2024-04-20T21:26:44.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.520+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.6 v2:172.21.15.144:6824/1893803079 9 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c6d800 2024-04-20T21:26:44.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.520+0000 7fc0f280c740 10 datascan.scan_links: step 1: handling object 500.00000000 2024-04-20T21:26:44.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.520+0000 7fc0f280c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-20T21:26:44.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.520+0000 7fc0f280c740 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-04-20T21:26:44.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.520+0000 7fc0f280c740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-20T21:26:44.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.520+0000 7fc0f280c740 10 datascan.scan_links: step 1: handling object 605.00000000 2024-04-20T21:26:44.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.520+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:43 12.1d 12:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556c040da000 con 0x556c04c41c00 2024-04-20T21:26:44.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.521+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.0 v2:172.21.15.86:6800/2471076035 15 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c41c00 2024-04-20T21:26:44.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.521+0000 7fc0f280c740 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-04-20T21:26:44.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.521+0000 7fc0f280c740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-20T21:26:44.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.521+0000 7fc0f280c740 10 datascan.scan_links: step 1: handling object 100.00000000 2024-04-20T21:26:44.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.521+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:44 12.13 12:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556c040da400 con 0x556c0402a000 2024-04-20T21:26:44.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.522+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 <== osd.1 v2:172.21.15.86:6817/3149564961 5 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x556c04aa6000 con 0x556c0402a000 2024-04-20T21:26:44.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.522+0000 7fc0f280c740 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-04-20T21:26:44.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.522+0000 7fc0f280c740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-20T21:26:44.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.522+0000 7fc0f280c740 10 datascan.scan_links: step 1: handling object 200.00000002 2024-04-20T21:26:44.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.522+0000 7fc0f280c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-04-20T21:26:44.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.522+0000 7fc0f280c740 10 datascan.scan_links: step 1: handling object 609.00000000 2024-04-20T21:26:44.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.522+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:45 12.7 12:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556c040da800 con 0x556c0402a000 2024-04-20T21:26:44.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.522+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 <== osd.1 v2:172.21.15.86:6817/3149564961 6 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556c04aa6000 con 0x556c0402a000 2024-04-20T21:26:44.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.522+0000 7fc0f280c740 10 datascan.scan_links: step 1: handling object 400.00000000 2024-04-20T21:26:44.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.522+0000 7fc0f280c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-20T21:26:44.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.522+0000 7fc0f280c740 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-04-20T21:26:44.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.522+0000 7fc0f280c740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-20T21:26:44.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.522+0000 7fc0f280c740 10 datascan.scan_links: step 1: handling object 1.00000000 2024-04-20T21:26:44.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.522+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:46 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556c040dac00 con 0x556c04c6d800 2024-04-20T21:26:44.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.524+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.6 v2:172.21.15.144:6824/1893803079 10 ==== osd_op_reply(46 1.00000000 [omap-get-vals out=1041b] v0'0 uv9 ondisk = 0) ==== 154+0+1041 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c6d800 2024-04-20T21:26:44.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.524+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:47 12.0 12:00000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c040db000 con 0x556c04c40c00 2024-04-20T21:26:44.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.525+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 <== osd.4 v2:172.21.15.144:6800/4087309475 6 ==== osd_op_reply(47 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556c04aa6000 con 0x556c04c40c00 2024-04-20T21:26:44.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.525+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:48 12.10 12:08000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c040db000 con 0x556c04c41c00 2024-04-20T21:26:44.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.526+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.0 v2:172.21.15.86:6800/2471076035 16 ==== osd_op_reply(48 [pgnls start_epoch 89 out=177b] v89'18 uv18 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c41c00 2024-04-20T21:26:44.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.526+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:49 12.8 12:10000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c040db000 con 0x556c04c41400 2024-04-20T21:26:44.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.526+0000 7fc0f0000640 1 -- 172.21.15.86:0/2303541732 <== osd.3 v2:172.21.15.86:6808/794135519 4 ==== osd_op_reply(49 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556c04aa6780 con 0x556c04c41400 2024-04-20T21:26:44.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.527+0000 7fc0f0000640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:50 12.18 12:18000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c040db000 con 0x556c0402a000 2024-04-20T21:26:44.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.527+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 <== osd.1 v2:172.21.15.86:6817/3149564961 7 ==== osd_op_reply(50 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556c04aa6000 con 0x556c0402a000 2024-04-20T21:26:44.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.527+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:51 12.4 12:20000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c040db000 con 0x556c04c6d800 2024-04-20T21:26:44.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.528+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.6 v2:172.21.15.144:6824/1893803079 11 ==== osd_op_reply(51 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c6d800 2024-04-20T21:26:44.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.528+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:52 12.14 12:28000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c040db000 con 0x556c04c41800 2024-04-20T21:26:44.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.530+0000 7fc0f0000640 1 -- 172.21.15.86:0/2303541732 <== osd.5 v2:172.21.15.144:6808/1635974704 4 ==== osd_op_reply(52 [pgnls start_epoch 89 out=149b] v88'28 uv28 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x556c04aa6780 con 0x556c04c41800 2024-04-20T21:26:44.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.530+0000 7fc0f0000640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:53 12.c 12:30000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c040db000 con 0x556c04c41c00 2024-04-20T21:26:44.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.530+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.0 v2:172.21.15.86:6800/2471076035 17 ==== osd_op_reply(53 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c41c00 2024-04-20T21:26:44.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.530+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:54 12.1c 12:38000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c040db000 con 0x556c04c41c00 2024-04-20T21:26:44.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.531+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.0 v2:172.21.15.86:6800/2471076035 18 ==== osd_op_reply(54 [pgnls start_epoch 89 out=73b] v88'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c41c00 2024-04-20T21:26:44.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.531+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:55 12.2 12:40000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c040db000 con 0x556c04c40c00 2024-04-20T21:26:44.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.532+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 <== osd.4 v2:172.21.15.144:6800/4087309475 7 ==== osd_op_reply(55 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556c04aa6000 con 0x556c04c40c00 2024-04-20T21:26:44.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.532+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:56 12.12 12:48000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c040db000 con 0x556c04c41400 2024-04-20T21:26:44.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.533+0000 7fc0f0000640 1 -- 172.21.15.86:0/2303541732 <== osd.3 v2:172.21.15.86:6808/794135519 5 ==== osd_op_reply(56 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556c04aa6780 con 0x556c04c41400 2024-04-20T21:26:44.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.533+0000 7fc0f0000640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:57 12.a 12:50000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c040db000 con 0x556c0402a400 2024-04-20T21:26:44.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.534+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 <== osd.2 v2:172.21.15.86:6816/3437611468 4 ==== osd_op_reply(57 [pgnls start_epoch 89 out=81b] v88'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x556c04aa6000 con 0x556c0402a400 2024-04-20T21:26:44.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.534+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:58 12.1a 12:58000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c040db000 con 0x556c04c40c00 2024-04-20T21:26:44.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.535+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 <== osd.4 v2:172.21.15.144:6800/4087309475 8 ==== osd_op_reply(58 [pgnls start_epoch 89 out=49b] v86'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556c04aa6000 con 0x556c04c40c00 2024-04-20T21:26:44.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.535+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:59 12.6 12:60000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c040db000 con 0x556c0402a000 2024-04-20T21:26:44.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.536+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 <== osd.1 v2:172.21.15.86:6817/3149564961 8 ==== osd_op_reply(59 [pgnls start_epoch 89 out=77b] v88'9 uv9 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x556c04aa6000 con 0x556c0402a000 2024-04-20T21:26:44.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.536+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:60 12.16 12:68000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c040db000 con 0x556c04c6d800 2024-04-20T21:26:44.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.537+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.6 v2:172.21.15.144:6824/1893803079 12 ==== osd_op_reply(60 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c6d800 2024-04-20T21:26:44.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.537+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:61 12.e 12:70000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c040db000 con 0x556c04c41c00 2024-04-20T21:26:44.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.538+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.0 v2:172.21.15.86:6800/2471076035 19 ==== osd_op_reply(61 [pgnls start_epoch 89 out=73b] v88'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c41c00 2024-04-20T21:26:44.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.538+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:62 12.1e 12:78000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c040db000 con 0x556c04c40c00 2024-04-20T21:26:44.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.539+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 <== osd.4 v2:172.21.15.144:6800/4087309475 9 ==== osd_op_reply(62 [pgnls start_epoch 89 out=49b] v86'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556c04aa6000 con 0x556c04c40c00 2024-04-20T21:26:44.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.540+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:63 12.1 12:80000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c040db000 con 0x556c04c6d800 2024-04-20T21:26:44.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.541+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.6 v2:172.21.15.144:6824/1893803079 13 ==== osd_op_reply(63 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c6d800 2024-04-20T21:26:44.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.541+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:64 12.11 12:88000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c040db000 con 0x556c04c41800 2024-04-20T21:26:44.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.542+0000 7fc0f0000640 1 -- 172.21.15.86:0/2303541732 <== osd.5 v2:172.21.15.144:6808/1635974704 5 ==== osd_op_reply(64 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556c04aa6780 con 0x556c04c41800 2024-04-20T21:26:44.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.542+0000 7fc0f0000640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:65 12.9 12:90000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c040db000 con 0x556c0402a400 2024-04-20T21:26:44.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.543+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 <== osd.2 v2:172.21.15.86:6816/3437611468 5 ==== osd_op_reply(65 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556c04aa6000 con 0x556c0402a400 2024-04-20T21:26:44.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.543+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:66 12.19 12:98000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c040db000 con 0x556c04c41c00 2024-04-20T21:26:44.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.544+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.0 v2:172.21.15.86:6800/2471076035 20 ==== osd_op_reply(66 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c41c00 2024-04-20T21:26:44.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.544+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:67 12.5 12:a0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c040db000 con 0x556c04c6d800 2024-04-20T21:26:44.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.545+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.6 v2:172.21.15.144:6824/1893803079 14 ==== osd_op_reply(67 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c6d800 2024-04-20T21:26:44.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.545+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:68 12.15 12:a8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c040db000 con 0x556c04c6d800 2024-04-20T21:26:44.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.546+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.6 v2:172.21.15.144:6824/1893803079 15 ==== osd_op_reply(68 [pgnls start_epoch 89 out=97b] v88'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c6d800 2024-04-20T21:26:44.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.546+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:69 12.d 12:b0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c040db000 con 0x556c04c6dc00 2024-04-20T21:26:44.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.547+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.7 v2:172.21.15.144:6816/2100967057 2 ==== osd_op_reply(69 [pgnls start_epoch 89 out=74b] v84'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c6dc00 2024-04-20T21:26:44.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.547+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:70 12.1d 12:b8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c040db000 con 0x556c04c41c00 2024-04-20T21:26:44.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.548+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.0 v2:172.21.15.86:6800/2471076035 21 ==== osd_op_reply(70 [pgnls start_epoch 89 out=73b] v88'5 uv5 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c41c00 2024-04-20T21:26:44.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.548+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:71 12.3 12:c0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c040db000 con 0x556c04c41c00 2024-04-20T21:26:44.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.549+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.0 v2:172.21.15.86:6800/2471076035 22 ==== osd_op_reply(71 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c41c00 2024-04-20T21:26:44.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.549+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:72 12.13 12:c8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c040db000 con 0x556c0402a000 2024-04-20T21:26:44.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.550+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 <== osd.1 v2:172.21.15.86:6817/3149564961 9 ==== osd_op_reply(72 [pgnls start_epoch 89 out=98b] v88'4 uv4 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x556c04aa6000 con 0x556c0402a000 2024-04-20T21:26:44.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.550+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:73 12.b 12:d0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c040db000 con 0x556c04c41400 2024-04-20T21:26:44.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.551+0000 7fc0f0000640 1 -- 172.21.15.86:0/2303541732 <== osd.3 v2:172.21.15.86:6808/794135519 6 ==== osd_op_reply(73 [pgnls start_epoch 89 out=76b] v88'6 uv6 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x556c04aa6780 con 0x556c04c41400 2024-04-20T21:26:44.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.551+0000 7fc0f0000640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:74 12.1b 12:d8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c040db000 con 0x556c04c40c00 2024-04-20T21:26:44.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.552+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 <== osd.4 v2:172.21.15.144:6800/4087309475 10 ==== osd_op_reply(74 [pgnls start_epoch 89 out=73b] v88'10 uv10 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x556c04aa6000 con 0x556c04c40c00 2024-04-20T21:26:44.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.552+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:75 12.7 12:e0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c040db000 con 0x556c0402a000 2024-04-20T21:26:44.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.552+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 <== osd.1 v2:172.21.15.86:6817/3149564961 10 ==== osd_op_reply(75 [pgnls start_epoch 89 out=73b] v88'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x556c04aa6000 con 0x556c0402a000 2024-04-20T21:26:44.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.553+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:76 12.17 12:e8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c040db000 con 0x556c04c6d800 2024-04-20T21:26:44.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.554+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.6 v2:172.21.15.144:6824/1893803079 16 ==== osd_op_reply(76 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c6d800 2024-04-20T21:26:44.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.554+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:77 12.f 12:f0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c040db000 con 0x556c04c6d800 2024-04-20T21:26:44.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.555+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.6 v2:172.21.15.144:6824/1893803079 17 ==== osd_op_reply(77 [pgnls start_epoch 89 out=73b] v84'1 uv1 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c6d800 2024-04-20T21:26:44.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.555+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:78 12.1f 12:f8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x556c040db000 con 0x556c04c6d800 2024-04-20T21:26:44.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.556+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.6 v2:172.21.15.144:6824/1893803079 18 ==== osd_op_reply(78 [pgnls start_epoch 89 out=101b] v88'9 uv9 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c6d800 2024-04-20T21:26:44.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.556+0000 7fc0f280c740 10 datascan.scan_links: step 2: handling object 601.00000000 2024-04-20T21:26:44.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.556+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:79 12.10 12:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556c040db400 con 0x556c04c41c00 2024-04-20T21:26:44.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.557+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.0 v2:172.21.15.86:6800/2471076035 23 ==== osd_op_reply(79 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c41c00 2024-04-20T21:26:44.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.557+0000 7fc0f280c740 10 datascan.scan_links: step 2: handling object 602.00000000 2024-04-20T21:26:44.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.557+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:80 12.10 12:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556c040db800 con 0x556c04c41c00 2024-04-20T21:26:44.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.558+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.0 v2:172.21.15.86:6800/2471076035 24 ==== osd_op_reply(80 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c41c00 2024-04-20T21:26:44.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.558+0000 7fc0f280c740 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-04-20T21:26:44.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.558+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:81 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556c040dbc00 con 0x556c04c41c00 2024-04-20T21:26:44.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.558+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.0 v2:172.21.15.86:6800/2471076035 25 ==== osd_op_reply(81 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c41c00 2024-04-20T21:26:44.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.559+0000 7fc0f280c740 20 datascan.scan_links: injected ino detected 2024-04-20T21:26:44.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.559+0000 7fc0f280c740 -1 datascan.scan_links: Bad nlink on 0x10000000001 expected 2 has 1 2024-04-20T21:26:44.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.559+0000 7fc0f280c740 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2024-04-20T21:26:44.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.559+0000 7fc0f280c740 10 datascan.scan_links: step 2: handling object 600.00000000 2024-04-20T21:26:44.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.559+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:82 12.10 12:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556c040da000 con 0x556c04c41c00 2024-04-20T21:26:44.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.559+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.0 v2:172.21.15.86:6800/2471076035 26 ==== osd_op_reply(82 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c41c00 2024-04-20T21:26:44.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.560+0000 7fc0f280c740 10 datascan.scan_links: step 2: handling object 603.00000000 2024-04-20T21:26:44.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.560+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:83 12.10 12:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556c040dac00 con 0x556c04c41c00 2024-04-20T21:26:44.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.560+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.0 v2:172.21.15.86:6800/2471076035 27 ==== osd_op_reply(83 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c41c00 2024-04-20T21:26:44.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.560+0000 7fc0f280c740 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-04-20T21:26:44.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.560+0000 7fc0f280c740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-20T21:26:44.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.560+0000 7fc0f280c740 10 datascan.scan_links: step 2: handling object 200.00000000 2024-04-20T21:26:44.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.560+0000 7fc0f280c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-20T21:26:44.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.560+0000 7fc0f280c740 10 datascan.scan_links: step 2: handling object 200.00000001 2024-04-20T21:26:44.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.560+0000 7fc0f280c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-20T21:26:44.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.560+0000 7fc0f280c740 10 datascan.scan_links: step 2: handling object 606.00000000 2024-04-20T21:26:44.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.560+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:84 12.14 12:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556c0402bc00 con 0x556c04c41800 2024-04-20T21:26:44.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.562+0000 7fc0f0000640 1 -- 172.21.15.86:0/2303541732 <== osd.5 v2:172.21.15.144:6808/1635974704 6 ==== osd_op_reply(84 606.00000000 [omap-get-vals out=5b] v0'0 uv26 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556c04aa6780 con 0x556c04c41800 2024-04-20T21:26:44.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.562+0000 7fc0f280c740 10 datascan.scan_links: step 2: handling object 607.00000000 2024-04-20T21:26:44.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.562+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:85 12.1c 12:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556c0402ac00 con 0x556c04c41c00 2024-04-20T21:26:44.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.563+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.0 v2:172.21.15.86:6800/2471076035 28 ==== osd_op_reply(85 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c41c00 2024-04-20T21:26:44.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.563+0000 7fc0f280c740 10 datascan.scan_links: step 2: handling object 100000001f6.00000000 2024-04-20T21:26:44.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.563+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:86 12.a 12:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556c0402b400 con 0x556c0402a400 2024-04-20T21:26:44.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.564+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 <== osd.2 v2:172.21.15.86:6816/3437611468 6 ==== osd_op_reply(86 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) ==== 164+0+54 (crc 0 0 0) 0x556c04aa6000 con 0x556c0402a400 2024-04-20T21:26:44.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.564+0000 7fc0f280c740 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-04-20T21:26:44.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.564+0000 7fc0f280c740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-20T21:26:44.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.564+0000 7fc0f280c740 10 datascan.scan_links: step 2: handling object 608.00000000 2024-04-20T21:26:44.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.564+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:87 12.e 12:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556c0402b800 con 0x556c04c41c00 2024-04-20T21:26:44.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.565+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.0 v2:172.21.15.86:6800/2471076035 29 ==== osd_op_reply(87 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c41c00 2024-04-20T21:26:44.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.565+0000 7fc0f280c740 10 datascan.scan_links: step 2: handling object 604.00000000 2024-04-20T21:26:44.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.565+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:88 12.15 12:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556c0402ac00 con 0x556c04c6d800 2024-04-20T21:26:44.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.566+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.6 v2:172.21.15.144:6824/1893803079 19 ==== osd_op_reply(88 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c6d800 2024-04-20T21:26:44.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.567+0000 7fc0f280c740 10 datascan.scan_links: step 2: handling object 500.00000000 2024-04-20T21:26:44.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.567+0000 7fc0f280c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-20T21:26:44.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.567+0000 7fc0f280c740 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-04-20T21:26:44.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.567+0000 7fc0f280c740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-20T21:26:44.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.567+0000 7fc0f280c740 10 datascan.scan_links: step 2: handling object 605.00000000 2024-04-20T21:26:44.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.567+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556c040db000 con 0x556c04c41c00 2024-04-20T21:26:44.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.568+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.0 v2:172.21.15.86:6800/2471076035 30 ==== osd_op_reply(89 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c41c00 2024-04-20T21:26:44.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.568+0000 7fc0f280c740 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-04-20T21:26:44.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.568+0000 7fc0f280c740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-20T21:26:44.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.568+0000 7fc0f280c740 10 datascan.scan_links: step 2: handling object 100.00000000 2024-04-20T21:26:44.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.568+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:90 12.13 12:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556c040db400 con 0x556c0402a000 2024-04-20T21:26:44.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.569+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 <== osd.1 v2:172.21.15.86:6817/3149564961 11 ==== osd_op_reply(90 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x556c04aa6000 con 0x556c0402a000 2024-04-20T21:26:44.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.569+0000 7fc0f280c740 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-04-20T21:26:44.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.569+0000 7fc0f280c740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-20T21:26:44.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.569+0000 7fc0f280c740 10 datascan.scan_links: step 2: handling object 200.00000002 2024-04-20T21:26:44.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.569+0000 7fc0f280c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-04-20T21:26:44.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.569+0000 7fc0f280c740 10 datascan.scan_links: step 2: handling object 609.00000000 2024-04-20T21:26:44.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.569+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:91 12.7 12:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556c040db800 con 0x556c0402a000 2024-04-20T21:26:44.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.570+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 <== osd.1 v2:172.21.15.86:6817/3149564961 12 ==== osd_op_reply(91 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x556c04aa6000 con 0x556c0402a000 2024-04-20T21:26:44.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.570+0000 7fc0f280c740 10 datascan.scan_links: step 2: handling object 400.00000000 2024-04-20T21:26:44.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.570+0000 7fc0f280c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-20T21:26:44.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.570+0000 7fc0f280c740 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-04-20T21:26:44.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.570+0000 7fc0f280c740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-20T21:26:44.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.570+0000 7fc0f280c740 10 datascan.scan_links: step 2: handling object 1.00000000 2024-04-20T21:26:44.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.570+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:92 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556c040dbc00 con 0x556c04c6d800 2024-04-20T21:26:44.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.571+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.6 v2:172.21.15.144:6824/1893803079 20 ==== osd_op_reply(92 1.00000000 [omap-get-vals out=1041b] v0'0 uv9 ondisk = 0) ==== 154+0+1041 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c6d800 2024-04-20T21:26:44.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.571+0000 7fc0f280c740 10 datascan.scan_links: processing 0 dup_primaries, 1 remote_links 2024-04-20T21:26:44.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.571+0000 7fc0f280c740 10 datascan.scan_links: removing dup dentries from 0 objects 2024-04-20T21:26:44.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.571+0000 7fc0f280c740 10 datascan.scan_links: processing 1 bad_nlink_inos 2024-04-20T21:26:44.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.571+0000 7fc0f280c740 10 datascan.scan_links: handling bad_nlink_ino 0x10000000001 2024-04-20T21:26:44.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.571+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:93 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556c040da000 con 0x556c04c41c00 2024-04-20T21:26:44.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.572+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.0 v2:172.21.15.86:6800/2471076035 31 ==== osd_op_reply(93 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv18 ondisk = 0) ==== 164+0+505 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c41c00 2024-04-20T21:26:44.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.572+0000 7fc0f280c740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-04-20T21:26:44.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.572+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:94 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89) -- 0x556c040dac00 con 0x556c04c41c00 2024-04-20T21:26:44.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.576+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.0 v2:172.21.15.86:6800/2471076035 32 ==== osd_op_reply(94 10000000000.00000000 [omap-set-vals] v89'19 uv19 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c41c00 2024-04-20T21:26:44.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.577+0000 7fc0f280c740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-04-20T21:26:44.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.577+0000 7fc0f280c740 10 datascan.scan_links: processing 1 injected_inos 2024-04-20T21:26:44.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.577+0000 7fc0f280c740 10 datascan.scan_links: handling injected_ino 0x10000000001 2024-04-20T21:26:44.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.577+0000 7fc0f280c740 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:file 2024-04-20T21:26:44.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.577+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:95 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556c040da400 con 0x556c04c41c00 2024-04-20T21:26:44.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.577+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.0 v2:172.21.15.86:6800/2471076035 33 ==== osd_op_reply(95 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv19 ondisk = 0) ==== 164+0+505 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c41c00 2024-04-20T21:26:44.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.577+0000 7fc0f280c740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-04-20T21:26:44.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.577+0000 7fc0f280c740 20 datascan.scan_links: first is now 2 2024-04-20T21:26:44.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.578+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:96 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89) -- 0x556c040da800 con 0x556c04c41c00 2024-04-20T21:26:44.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.581+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.0 v2:172.21.15.86:6800/2471076035 34 ==== osd_op_reply(96 10000000000.00000000 [omap-set-vals] v89'20 uv20 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x556c04aa6500 con 0x556c04c41c00 2024-04-20T21:26:44.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.581+0000 7fc0f280c740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-04-20T21:26:44.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.581+0000 7fc0f280c740 10 datascan.scan_links: updating inotable 2024-04-20T21:26:44.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.581+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:97 12.13 12:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556c040db400 con 0x556c0402a000 2024-04-20T21:26:44.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.582+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 <== osd.1 v2:172.21.15.86:6817/3149564961 13 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv3 ondisk = 0) ==== 157+0+50 (crc 0 0 0) 0x556c04aa6000 con 0x556c0402a000 2024-04-20T21:26:44.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.582+0000 7fc0f280c740 10 mds.0.inotable: skip_inos was [0x10000000002~0x1f4,0x100000001f7~0xfffffffe09] 2024-04-20T21:26:44.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.582+0000 7fc0f280c740 10 mds.0.inotable: skip_inos now [0x100000001f7~0xfffffffe09] 2024-04-20T21:26:44.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.582+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:98 12.13 12:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89) -- 0x556c0402b000 con 0x556c0402a000 2024-04-20T21:26:44.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.586+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 <== osd.1 v2:172.21.15.86:6817/3149564961 14 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v89'5 uv5 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x556c04aa6000 con 0x556c0402a000 2024-04-20T21:26:44.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.586+0000 7fc0f280c740 10 datascan.scan_links: updating snaptable 2024-04-20T21:26:44.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.586+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:99 12.d 12:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x556c0402b400 con 0x556c04c6dc00 2024-04-20T21:26:44.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.588+0000 7fc0f0a00640 1 -- 172.21.15.86:0/2303541732 <== osd.7 v2:172.21.15.144:6816/2100967057 3 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) ==== 157+0+70 (crc 0 0 0) 0x556c04aa6280 con 0x556c04c6dc00 2024-04-20T21:26:44.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.588+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x556c04c41c00 msgr2=0x556c040a8000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:44.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.588+0000 7fc0f280c740 1 --2- 172.21.15.86:0/2303541732 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x556c04c41c00 0x556c040a8000 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.588+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x556c0402a000 msgr2=0x556c040a8b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:44.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.588+0000 7fc0f280c740 1 --2- 172.21.15.86:0/2303541732 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x556c0402a000 0x556c040a8b00 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.588+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x556c0402a400 msgr2=0x556c040aa680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:44.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.588+0000 7fc0f280c740 1 --2- 172.21.15.86:0/2303541732 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x556c0402a400 0x556c040aa680 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.588+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x556c04c41400 msgr2=0x556c040a8580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:44.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.588+0000 7fc0f280c740 1 --2- 172.21.15.86:0/2303541732 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x556c04c41400 0x556c040a8580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.588+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x556c04c40c00 msgr2=0x556c04a90680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:44.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.588+0000 7fc0f280c740 1 --2- 172.21.15.86:0/2303541732 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x556c04c40c00 0x556c04a90680 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.588+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x556c04c41800 msgr2=0x556c040a9b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:44.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.588+0000 7fc0f280c740 1 --2- 172.21.15.86:0/2303541732 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x556c04c41800 0x556c040a9b80 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.588+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x556c04c6d800 msgr2=0x556c040a9080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:44.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.588+0000 7fc0f280c740 1 --2- 172.21.15.86:0/2303541732 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x556c04c6d800 0x556c040a9080 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.588+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x556c04c6dc00 msgr2=0x556c040aa100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:44.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.588+0000 7fc0f280c740 1 --2- 172.21.15.86:0/2303541732 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x556c04c6dc00 0x556c040aa100 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.588+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 reap_dead start 2024-04-20T21:26:44.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.588+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 reap_dead start 2024-04-20T21:26:44.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.588+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x556c04c6cc00 msgr2=0x556c04a90100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:44.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.588+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 reap_dead start 2024-04-20T21:26:44.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.588+0000 7fc0f280c740 1 --2- 172.21.15.86:0/2303541732 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x556c04c6cc00 0x556c04a90100 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x556c04b3bd10 tx=0x556c03dccd20 comp rx=0 tx=0).stop 2024-04-20T21:26:44.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.588+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c04c40000 msgr2=0x556c04a8f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:44.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.588+0000 7fc0f280c740 1 --2- 172.21.15.86:0/2303541732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c04c40000 0x556c04a8f080 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x556c04bf3c20 tx=0x556c04afed20 comp rx=0 tx=0).stop 2024-04-20T21:26:44.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.588+0000 7fc0ef600640 1 -- 172.21.15.86:0/2303541732 reap_dead start 2024-04-20T21:26:44.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.589+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 shutdown_connections 2024-04-20T21:26:44.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.589+0000 7fc0f280c740 1 --2- 172.21.15.86:0/2303541732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c04c40000 0x556c04a8f080 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.589+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 >> 172.21.15.86:0/2303541732 conn(0x556c04ab7c00 msgr2=0x556c03e05000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:44.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.589+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 shutdown_connections 2024-04-20T21:26:44.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.589+0000 7fc0f280c740 1 -- 172.21.15.86:0/2303541732 wait complete. 2024-04-20T21:26:44.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.590+0000 7fc0f280c740 1 -- 172.21.15.86:0/4008134708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c03d18c00 msgr2=0x556c04a8e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:44.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.590+0000 7fc0f280c740 1 --2- 172.21.15.86:0/4008134708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c03d18c00 0x556c04a8e000 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x556c04b3b170 tx=0x556c04ab8a80 comp rx=0 tx=0).stop 2024-04-20T21:26:44.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.591+0000 7fc0f280c740 1 -- 172.21.15.86:0/4008134708 shutdown_connections 2024-04-20T21:26:44.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.591+0000 7fc0f280c740 1 --2- 172.21.15.86:0/4008134708 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x556c03d19c00 0x556c04a8eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.591+0000 7fc0f280c740 1 --2- 172.21.15.86:0/4008134708 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x556c03d19800 0x556c04a8e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.591+0000 7fc0f280c740 1 --2- 172.21.15.86:0/4008134708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c03d18c00 0x556c04a8e000 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.591+0000 7fc0f280c740 1 -- 172.21.15.86:0/4008134708 >> 172.21.15.86:0/4008134708 conn(0x556c03d19000 msgr2=0x556c03e04800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:44.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.591+0000 7fc0f280c740 1 -- 172.21.15.86:0/4008134708 shutdown_connections 2024-04-20T21:26:44.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.591+0000 7fc0f280c740 1 -- 172.21.15.86:0/4008134708 wait complete. 2024-04-20T21:26:44.603 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.245740, result: 2024-04-20T21:26:44.603 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-20T21:26:44.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.813+0000 7f1ba4e00640 1 Processor -- start 2024-04-20T21:26:44.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.813+0000 7f1ba4e00640 1 -- start start 2024-04-20T21:26:44.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.814+0000 7f1ba4e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1ba006bbf0 0x7f1ba006e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.814+0000 7f1ba4e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1ba006bfd0 0x7f1ba0145b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.814+0000 7f1ba4e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ba015cd40 con 0x7f1ba0146040 2024-04-20T21:26:44.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.814+0000 7f1ba4e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f1ba01464b0 con 0x7f1ba006bbf0 2024-04-20T21:26:44.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.814+0000 7f1ba4e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f1ba015d000 con 0x7f1ba006bfd0 2024-04-20T21:26:44.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.815+0000 7f1b9de00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1ba006bbf0 0x7f1ba006e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.815+0000 7f1b9de00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1ba006bbf0 0x7f1ba006e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45712/0 (socket says 172.21.15.86:45712) 2024-04-20T21:26:44.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.815+0000 7f1b9de00640 1 -- 172.21.15.86:0/307793973 learned_addr learned my addr 172.21.15.86:0/307793973 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:44.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.815+0000 7f1b9de00640 1 -- 172.21.15.86:0/307793973 >> v2:172.21.15.144:3300/0 conn(0x7f1ba006bfd0 msgr2=0x7f1ba0145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:44.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.815+0000 7f1b9d400640 1 --2- 172.21.15.86:0/307793973 >> v2:172.21.15.144:3300/0 conn(0x7f1ba006bfd0 0x7f1ba0145b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.815+0000 7f1b9de00640 1 --2- 172.21.15.86:0/307793973 >> v2:172.21.15.144:3300/0 conn(0x7f1ba006bfd0 0x7f1ba0145b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.815+0000 7f1b9de00640 1 -- 172.21.15.86:0/307793973 >> v1:172.21.15.144:6789/0 conn(0x7f1ba0146040 legacy=0x7f1ba00cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:44.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.815+0000 7f1b9de00640 1 -- 172.21.15.86:0/307793973 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba015d2e0 con 0x7f1ba006bbf0 2024-04-20T21:26:44.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.816+0000 7f1b9de00640 1 --2- 172.21.15.86:0/307793973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ba006bbf0 0x7f1ba006e000 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f1ba006dcc0 tx=0x7f1b9002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e669cedec2eba8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:44.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.817+0000 7f1b9ca00640 1 -- 172.21.15.86:0/307793973 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b90035020 con 0x7f1ba006bbf0 2024-04-20T21:26:44.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.817+0000 7f1b9ca00640 1 -- 172.21.15.86:0/307793973 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b90031870 con 0x7f1ba006bbf0 2024-04-20T21:26:44.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.817+0000 7f1b9ca00640 1 -- 172.21.15.86:0/307793973 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b90031c40 con 0x7f1ba006bbf0 2024-04-20T21:26:44.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.817+0000 7f1ba4e00640 1 -- 172.21.15.86:0/307793973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ba006bbf0 msgr2=0x7f1ba006e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:44.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.817+0000 7f1ba4e00640 1 --2- 172.21.15.86:0/307793973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ba006bbf0 0x7f1ba006e000 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f1ba006dcc0 tx=0x7f1b9002b910 comp rx=0 tx=0).stop 2024-04-20T21:26:44.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.817+0000 7f1ba4e00640 1 -- 172.21.15.86:0/307793973 shutdown_connections 2024-04-20T21:26:44.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.817+0000 7f1ba4e00640 1 --2- 172.21.15.86:0/307793973 >> v2:172.21.15.144:3300/0 conn(0x7f1ba006bfd0 0x7f1ba0145b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.817+0000 7f1ba4e00640 1 --2- 172.21.15.86:0/307793973 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ba006bbf0 0x7f1ba006e000 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.817+0000 7f1ba4e00640 1 -- 172.21.15.86:0/307793973 >> 172.21.15.86:0/307793973 conn(0x7f1ba00b2110 msgr2=0x7f1ba00b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:44.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.817+0000 7f1ba4e00640 1 -- 172.21.15.86:0/307793973 shutdown_connections 2024-04-20T21:26:44.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.817+0000 7f1ba4e00640 1 -- 172.21.15.86:0/307793973 wait complete. 2024-04-20T21:26:44.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.819+0000 7f1ba4e00640 1 Processor -- start 2024-04-20T21:26:44.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.819+0000 7f1ba4e00640 1 -- start start 2024-04-20T21:26:44.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.820+0000 7f1ba4e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ba006bbf0 0x7f1ba00b0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.820+0000 7f1ba4e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ba006bfd0 0x7f1ba00c78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.820+0000 7f1ba4e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ba0146040 0x7f1ba00a9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.820+0000 7f1ba4e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba00cdc50 con 0x7f1ba006bfd0 2024-04-20T21:26:44.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.820+0000 7f1ba4e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba015d8a0 con 0x7f1ba0146040 2024-04-20T21:26:44.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.820+0000 7f1ba4e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1ba00cddd0 con 0x7f1ba006bbf0 2024-04-20T21:26:44.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.820+0000 7f1b9d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ba006bfd0 0x7f1ba00c78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.821+0000 7f1b9d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ba006bfd0 0x7f1ba00c78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45726/0 (socket says 172.21.15.86:45726) 2024-04-20T21:26:44.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.821+0000 7f1b9d400640 1 -- 172.21.15.86:0/2969430953 learned_addr learned my addr 172.21.15.86:0/2969430953 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:44.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.821+0000 7f1b9e800640 1 --2- 172.21.15.86:0/2969430953 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ba0146040 0x7f1ba00a9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.821+0000 7f1b9d400640 1 -- 172.21.15.86:0/2969430953 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ba006bbf0 msgr2=0x7f1ba00b0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:44.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.821+0000 7f1b9de00640 1 --2- 172.21.15.86:0/2969430953 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ba006bbf0 0x7f1ba00b0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.821+0000 7f1b9d400640 1 --2- 172.21.15.86:0/2969430953 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ba006bbf0 0x7f1ba00b0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.821+0000 7f1b9d400640 1 -- 172.21.15.86:0/2969430953 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ba0146040 msgr2=0x7f1ba00a9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:44.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.821+0000 7f1b9d400640 1 --2- 172.21.15.86:0/2969430953 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ba0146040 0x7f1ba00a9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:44.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.821+0000 7f1b9d400640 1 -- 172.21.15.86:0/2969430953 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba00b12b0 con 0x7f1ba006bfd0 2024-04-20T21:26:44.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.821+0000 7f1b9e800640 1 --2- 172.21.15.86:0/2969430953 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ba0146040 0x7f1ba00a9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:26:44.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.821+0000 7f1b9de00640 1 --2- 172.21.15.86:0/2969430953 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ba006bbf0 0x7f1ba00b0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:26:44.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.821+0000 7f1b9d400640 1 --2- 172.21.15.86:0/2969430953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ba006bfd0 0x7f1ba00c78c0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c00c820 tx=0x7f1b8c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:44.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.822+0000 7f1b96000640 1 -- 172.21.15.86:0/2969430953 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c015070 con 0x7f1ba006bfd0 2024-04-20T21:26:44.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.822+0000 7f1b96000640 1 -- 172.21.15.86:0/2969430953 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b8c004240 con 0x7f1ba006bfd0 2024-04-20T21:26:44.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.822+0000 7f1b96000640 1 -- 172.21.15.86:0/2969430953 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c00ce10 con 0x7f1ba006bfd0 2024-04-20T21:26:44.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.822+0000 7f1ba4e00640 1 -- 172.21.15.86:0/2969430953 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ba00c80f0 con 0x7f1ba006bfd0 2024-04-20T21:26:44.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.822+0000 7f1ba4e00640 1 -- 172.21.15.86:0/2969430953 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ba00c86b0 con 0x7f1ba006bfd0 2024-04-20T21:26:44.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.823+0000 7f1ba4e00640 1 -- 172.21.15.86:0/2969430953 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ba00689e0 con 0x7f1ba006bfd0 2024-04-20T21:26:44.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.824+0000 7f1b96000640 1 -- 172.21.15.86:0/2969430953 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1b8c0077d0 con 0x7f1ba006bfd0 2024-04-20T21:26:44.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.825+0000 7f1b96000640 1 --2- 172.21.15.86:0/2969430953 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1b8006c8f0 0x7f1b8006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:44.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.825+0000 7f1b96000640 1 -- 172.21.15.86:0/2969430953 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7f1b8c08bb30 con 0x7f1ba006bfd0 2024-04-20T21:26:44.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.825+0000 7f1b9de00640 1 --2- 172.21.15.86:0/2969430953 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1b8006c8f0 0x7f1b8006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:44.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.826+0000 7f1b9de00640 1 --2- 172.21.15.86:0/2969430953 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1b8006c8f0 0x7f1b8006edb0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f1b90036000 tx=0x7f1b90008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:44.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:44.829+0000 7f1b96000640 1 -- 172.21.15.86:0/2969430953 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1b8c059df0 con 0x7f1ba006bfd0 2024-04-20T21:26:45.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:45.078+0000 7f1ba4e00640 1 -- 172.21.15.86:0/2969430953 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f1ba006ed90 con 0x7f1ba006bfd0 2024-04-20T21:26:45.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:45.879+0000 7f1b96000640 1 -- 172.21.15.86:0/2969430953 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v69) ==== 162+0+0 (secure 0 0 0) 0x7f1b8c0596f0 con 0x7f1ba006bfd0 2024-04-20T21:26:45.878 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-20T21:26:45.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:45.882+0000 7f1ba4e00640 1 -- 172.21.15.86:0/2969430953 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1b8006c8f0 msgr2=0x7f1b8006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:45.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:45.882+0000 7f1ba4e00640 1 --2- 172.21.15.86:0/2969430953 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1b8006c8f0 0x7f1b8006edb0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f1b90036000 tx=0x7f1b90008030 comp rx=0 tx=0).stop 2024-04-20T21:26:45.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:45.882+0000 7f1ba4e00640 1 -- 172.21.15.86:0/2969430953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ba006bfd0 msgr2=0x7f1ba00c78c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:45.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:45.882+0000 7f1ba4e00640 1 --2- 172.21.15.86:0/2969430953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ba006bfd0 0x7f1ba00c78c0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c00c820 tx=0x7f1b8c00cce0 comp rx=0 tx=0).stop 2024-04-20T21:26:45.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:45.882+0000 7f1ba4e00640 1 -- 172.21.15.86:0/2969430953 shutdown_connections 2024-04-20T21:26:45.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:45.882+0000 7f1ba4e00640 1 --2- 172.21.15.86:0/2969430953 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1b8006c8f0 0x7f1b8006edb0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:45.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:45.882+0000 7f1ba4e00640 1 --2- 172.21.15.86:0/2969430953 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ba0146040 0x7f1ba00a9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:45.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:45.882+0000 7f1ba4e00640 1 --2- 172.21.15.86:0/2969430953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ba006bfd0 0x7f1ba00c78c0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:45.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:45.882+0000 7f1ba4e00640 1 --2- 172.21.15.86:0/2969430953 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ba006bbf0 0x7f1ba00b0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:45.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:45.882+0000 7f1ba4e00640 1 -- 172.21.15.86:0/2969430953 >> 172.21.15.86:0/2969430953 conn(0x7f1ba00b2110 msgr2=0x7f1ba006e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:45.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:45.882+0000 7f1ba4e00640 1 -- 172.21.15.86:0/2969430953 shutdown_connections 2024-04-20T21:26:45.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:45.883+0000 7f1ba4e00640 1 -- 172.21.15.86:0/2969430953 wait complete. 2024-04-20T21:26:45.902 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:26:46.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.084+0000 7f5571400640 1 Processor -- start 2024-04-20T21:26:46.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.084+0000 7f5571400640 1 -- start start 2024-04-20T21:26:46.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.086+0000 7f5571400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f556c146130 0x7f556c06e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:46.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.086+0000 7f5571400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f556c072410 0x7f556c0cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:46.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.086+0000 7f5571400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f556c06dc30 con 0x7f556c0ccdd0 2024-04-20T21:26:46.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.086+0000 7f5571400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f556c0cd8e0 con 0x7f556c072410 2024-04-20T21:26:46.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.086+0000 7f5571400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f556c0cdbc0 con 0x7f556c146130 2024-04-20T21:26:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.086+0000 7f5569e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f556c0ccdd0 0x7f556c071d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:36778/0 (socket says 172.21.15.86:36778) 2024-04-20T21:26:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.086+0000 7f5569e00640 1 -- 172.21.15.86:0/1495101795 learned_addr learned my addr 172.21.15.86:0/1495101795 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:26:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.086+0000 7f556a800640 1 --2- 172.21.15.86:0/1495101795 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f556c146130 0x7f556c06e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.086+0000 7f556b200640 1 --2- 172.21.15.86:0/1495101795 >> v2:172.21.15.144:3300/0 conn(0x7f556c072410 0x7f556c0cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.087+0000 7f5569400640 1 -- 172.21.15.86:0/1495101795 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3882758910 0 0) 0x7f556c06dc30 con 0x7f556c0ccdd0 2024-04-20T21:26:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.087+0000 7f5569400640 1 -- 172.21.15.86:0/1495101795 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5550003520 con 0x7f556c0ccdd0 2024-04-20T21:26:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.087+0000 7f5569400640 1 -- 172.21.15.86:0/1495101795 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 560088328 0 0) 0x7f5550003520 con 0x7f556c0ccdd0 2024-04-20T21:26:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.087+0000 7f5569400640 1 -- 172.21.15.86:0/1495101795 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f556c146130 msgr2=0x7f556c06e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.087+0000 7f5569400640 1 --2- 172.21.15.86:0/1495101795 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f556c146130 0x7f556c06e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.087+0000 7f5569400640 1 -- 172.21.15.86:0/1495101795 >> v2:172.21.15.144:3300/0 conn(0x7f556c072410 msgr2=0x7f556c0cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.087+0000 7f5569400640 1 --2- 172.21.15.86:0/1495101795 >> v2:172.21.15.144:3300/0 conn(0x7f556c072410 0x7f556c0cd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:46.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.087+0000 7f5569400640 1 -- 172.21.15.86:0/1495101795 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f556c06ded0 con 0x7f556c0ccdd0 2024-04-20T21:26:46.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.087+0000 7f5569400640 1 -- 172.21.15.86:0/1495101795 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f5558001670 con 0x7f556c0ccdd0 2024-04-20T21:26:46.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.087+0000 7f5569400640 1 -- 172.21.15.86:0/1495101795 >> v1:172.21.15.86:6789/0 conn(0x7f556c0ccdd0 legacy=0x7f556c071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:46.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.087+0000 7f5569400640 1 --2- 172.21.15.86:0/1495101795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55500039b0 0x7f5550003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:46.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.087+0000 7f5569400640 1 --2- 172.21.15.86:0/1495101795 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f556c146130 0x7f5550004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:46.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.087+0000 7f5569400640 1 --2- 172.21.15.86:0/1495101795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5550004d40 0x7f55500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:46.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.087+0000 7f5569400640 1 -- 172.21.15.86:0/1495101795 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f556c0cd8e0 con 0x7f5550004d40 2024-04-20T21:26:46.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.087+0000 7f5569400640 1 -- 172.21.15.86:0/1495101795 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f556c0cdbc0 con 0x7f55500039b0 2024-04-20T21:26:46.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.087+0000 7f5569400640 1 -- 172.21.15.86:0/1495101795 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f555000b9c0 con 0x7f556c146130 2024-04-20T21:26:46.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.087+0000 7f556b200640 1 --2- 172.21.15.86:0/1495101795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5550004d40 0x7f55500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:46.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.088+0000 7f5569e00640 1 --2- 172.21.15.86:0/1495101795 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f556c146130 0x7f5550004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:46.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.088+0000 7f556b200640 1 -- 172.21.15.86:0/1495101795 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f556c146130 msgr2=0x7f5550004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:46.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.088+0000 7f556a800640 1 --2- 172.21.15.86:0/1495101795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55500039b0 0x7f5550003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:46.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.088+0000 7f556b200640 1 --2- 172.21.15.86:0/1495101795 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f556c146130 0x7f5550004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:46.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.088+0000 7f556b200640 1 -- 172.21.15.86:0/1495101795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55500039b0 msgr2=0x7f5550003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:46.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.088+0000 7f556b200640 1 --2- 172.21.15.86:0/1495101795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f55500039b0 0x7f5550003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:46.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.088+0000 7f556b200640 1 -- 172.21.15.86:0/1495101795 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5550003520 con 0x7f5550004d40 2024-04-20T21:26:46.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.088+0000 7f5569e00640 1 --2- 172.21.15.86:0/1495101795 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f556c146130 0x7f5550004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:26:46.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.089+0000 7f556b200640 1 --2- 172.21.15.86:0/1495101795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5550004d40 0x7f55500051c0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f5564009f90 tx=0x7f5550005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a4269257301e81c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:46.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.089+0000 7f5569400640 1 -- 172.21.15.86:0/1495101795 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5564039070 con 0x7f5550004d40 2024-04-20T21:26:46.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.089+0000 7f5569400640 1 -- 172.21.15.86:0/1495101795 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f556402cc20 con 0x7f5550004d40 2024-04-20T21:26:46.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.089+0000 7f5569400640 1 -- 172.21.15.86:0/1495101795 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5564009c10 con 0x7f5550004d40 2024-04-20T21:26:46.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.090+0000 7f5571400640 1 -- 172.21.15.86:0/1495101795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5550004d40 msgr2=0x7f55500051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:46.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.090+0000 7f5571400640 1 --2- 172.21.15.86:0/1495101795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5550004d40 0x7f55500051c0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f5564009f90 tx=0x7f5550005830 comp rx=0 tx=0).stop 2024-04-20T21:26:46.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.090+0000 7f556b200640 1 -- 172.21.15.86:0/1495101795 reap_dead start 2024-04-20T21:26:46.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.090+0000 7f5571400640 1 -- 172.21.15.86:0/1495101795 shutdown_connections 2024-04-20T21:26:46.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.090+0000 7f5571400640 1 -- 172.21.15.86:0/1495101795 >> 172.21.15.86:0/1495101795 conn(0x7f556c0b2360 msgr2=0x7f556c0b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:46.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.090+0000 7f5571400640 1 -- 172.21.15.86:0/1495101795 shutdown_connections 2024-04-20T21:26:46.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.090+0000 7f5571400640 1 -- 172.21.15.86:0/1495101795 wait complete. 2024-04-20T21:26:46.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.092+0000 7f5571400640 1 Processor -- start 2024-04-20T21:26:46.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.092+0000 7f5571400640 1 -- start start 2024-04-20T21:26:46.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.093+0000 7f5571400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5550004d40 0x7f556c15c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:46.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.093+0000 7f5571400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f556c13ed60 0x7f556c1f7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:46.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.093+0000 7f5571400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f556c1f7f60 0x7f556c140660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:46.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.093+0000 7f5571400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f556c06e6d0 con 0x7f556c13ed60 2024-04-20T21:26:46.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.093+0000 7f5571400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f556c06f150 con 0x7f5550004d40 2024-04-20T21:26:46.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.093+0000 7f5571400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f556c06e850 con 0x7f556c1f7f60 2024-04-20T21:26:46.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.093+0000 7f5569e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f556c13ed60 0x7f556c1f7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:46.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.093+0000 7f5569e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f556c13ed60 0x7f556c1f7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55800/0 (socket says 172.21.15.86:55800) 2024-04-20T21:26:46.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.093+0000 7f5569e00640 1 -- 172.21.15.86:0/2702675930 learned_addr learned my addr 172.21.15.86:0/2702675930 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:46.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.093+0000 7f5569e00640 1 -- 172.21.15.86:0/2702675930 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f556c1f7f60 msgr2=0x7f556c140660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:46.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.093+0000 7f556a800640 1 --2- 172.21.15.86:0/2702675930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5550004d40 0x7f556c15c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:46.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.093+0000 7f5569e00640 1 --2- 172.21.15.86:0/2702675930 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f556c1f7f60 0x7f556c140660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:46.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.093+0000 7f5569e00640 1 -- 172.21.15.86:0/2702675930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5550004d40 msgr2=0x7f556c15c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.093+0000 7f5569e00640 1 --2- 172.21.15.86:0/2702675930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5550004d40 0x7f556c15c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.093+0000 7f5569e00640 1 -- 172.21.15.86:0/2702675930 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f556c1f8400 con 0x7f556c13ed60 2024-04-20T21:26:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.094+0000 7f5569e00640 1 --2- 172.21.15.86:0/2702675930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f556c13ed60 0x7f556c1f7a20 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f555800d840 tx=0x7f555800dd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.094+0000 7f5562a00640 1 -- 172.21.15.86:0/2702675930 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5558019070 con 0x7f556c13ed60 2024-04-20T21:26:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.094+0000 7f5562a00640 1 -- 172.21.15.86:0/2702675930 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55580074e0 con 0x7f556c13ed60 2024-04-20T21:26:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.094+0000 7f5571400640 1 -- 172.21.15.86:0/2702675930 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f556c1f86e0 con 0x7f556c13ed60 2024-04-20T21:26:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.094+0000 7f5562a00640 1 -- 172.21.15.86:0/2702675930 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5558004a00 con 0x7f556c13ed60 2024-04-20T21:26:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.094+0000 7f5571400640 1 -- 172.21.15.86:0/2702675930 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f556c0b3bb0 con 0x7f556c13ed60 2024-04-20T21:26:46.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.095+0000 7f5571400640 1 -- 172.21.15.86:0/2702675930 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f556c0689e0 con 0x7f556c13ed60 2024-04-20T21:26:46.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.096+0000 7f5562a00640 1 -- 172.21.15.86:0/2702675930 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f555801c070 con 0x7f556c13ed60 2024-04-20T21:26:46.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.097+0000 7f5562a00640 1 --2- 172.21.15.86:0/2702675930 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f554c06c920 0x7f554c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:46.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.097+0000 7f5562a00640 1 -- 172.21.15.86:0/2702675930 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7f555808b810 con 0x7f556c13ed60 2024-04-20T21:26:46.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.097+0000 7f556a800640 1 --2- 172.21.15.86:0/2702675930 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f554c06c920 0x7f554c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:46.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.098+0000 7f556a800640 1 --2- 172.21.15.86:0/2702675930 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f554c06c920 0x7f554c06ede0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f556c142dd0 tx=0x7f555c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:46.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.101+0000 7f5562a00640 1 -- 172.21.15.86:0/2702675930 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5558059ad0 con 0x7f556c13ed60 2024-04-20T21:26:46.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.346+0000 7f5571400640 1 -- 172.21.15.86:0/2702675930 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f556c15ff50 con 0x7f556c13ed60 2024-04-20T21:26:46.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.348+0000 7f5562a00640 1 -- 172.21.15.86:0/2702675930 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) ==== 94+0+5201 (secure 0 0 0) 0x7f55580593d0 con 0x7f556c13ed60 2024-04-20T21:26:46.347 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:26:46.347 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":70,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5887,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2433619638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2433619638},{"type":"v1","addr":"172.21.15.144:6835","nonce":2433619638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68},{"gid":7506,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/4013863257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4013863257},{"type":"v1","addr":"172.21.15.86:6836","nonce":4013863257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":7509,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2044883073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2044883073},{"type":"v1","addr":"172.21.15.86:6837","nonce":2044883073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:24:58.299257+0000","modified":"2024-04-20T21:26:45.883615+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7518},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7518":{"gid":7518,"name":"d","rank":0,"incarnation":70,"state":"up:replay","state_seq":1,"addr":"172.21.15.144:6834/2351381466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2351381466},{"type":"v1","addr":"172.21.15.144:6834","nonce":2351381466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-20T21:26:46.347 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 70 2024-04-20T21:26:46.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.351+0000 7f5571400640 1 -- 172.21.15.86:0/2702675930 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f554c06c920 msgr2=0x7f554c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:46.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.351+0000 7f5571400640 1 --2- 172.21.15.86:0/2702675930 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f554c06c920 0x7f554c06ede0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f556c142dd0 tx=0x7f555c008040 comp rx=0 tx=0).stop 2024-04-20T21:26:46.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.351+0000 7f5571400640 1 -- 172.21.15.86:0/2702675930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f556c13ed60 msgr2=0x7f556c1f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:46.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.351+0000 7f5571400640 1 --2- 172.21.15.86:0/2702675930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f556c13ed60 0x7f556c1f7a20 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f555800d840 tx=0x7f555800dd00 comp rx=0 tx=0).stop 2024-04-20T21:26:46.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.351+0000 7f5571400640 1 -- 172.21.15.86:0/2702675930 shutdown_connections 2024-04-20T21:26:46.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.351+0000 7f5571400640 1 --2- 172.21.15.86:0/2702675930 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f554c06c920 0x7f554c06ede0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:46.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.351+0000 7f5571400640 1 --2- 172.21.15.86:0/2702675930 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f556c1f7f60 0x7f556c140660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:46.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.351+0000 7f5571400640 1 --2- 172.21.15.86:0/2702675930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f556c13ed60 0x7f556c1f7a20 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:46.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.351+0000 7f5571400640 1 --2- 172.21.15.86:0/2702675930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5550004d40 0x7f556c15c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:46.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.351+0000 7f5571400640 1 -- 172.21.15.86:0/2702675930 >> 172.21.15.86:0/2702675930 conn(0x7f556c0b2360 msgr2=0x7f556c0b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:46.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.351+0000 7f5571400640 1 -- 172.21.15.86:0/2702675930 shutdown_connections 2024-04-20T21:26:46.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:46.351+0000 7f5571400640 1 -- 172.21.15.86:0/2702675930 wait complete. 2024-04-20T21:26:46.771 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 70, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T21:24:58.299257+0000', 'modified': '2024-04-20T21:26:45.883615+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 89, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7518}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7518': {'gid': 7518, 'name': 'd', 'rank': 0, 'incarnation': 70, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.144:6834/2351381466', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6832', 'nonce': 2351381466}, {'type': 'v1', 'addr': '172.21.15.144:6834', 'nonce': 2351381466}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-20T21:26:46.772 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7518:up:replay 2024-04-20T21:26:47.571 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:26:47.580 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:26:47.772 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:26:47.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.984+0000 7f607c600640 1 Processor -- start 2024-04-20T21:26:47.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.984+0000 7f607c600640 1 -- start start 2024-04-20T21:26:47.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.986+0000 7f607c600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6074144e90 0x7f60740b4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:47.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.986+0000 7f607c600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f607406f450 0x7f607415bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:47.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.986+0000 7f607c600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60740cdb50 con 0x7f60740cd170 2024-04-20T21:26:47.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.986+0000 7f607c600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f6074145270 con 0x7f6074144e90 2024-04-20T21:26:47.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.986+0000 7f607c600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f60740b3e50 con 0x7f607406f450 2024-04-20T21:26:47.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.986+0000 7f6079c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6074144e90 0x7f60740b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:47.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.986+0000 7f6079c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6074144e90 0x7f60740b4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55816/0 (socket says 172.21.15.86:55816) 2024-04-20T21:26:47.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.986+0000 7f6079c00640 1 -- 172.21.15.86:0/1494559932 learned_addr learned my addr 172.21.15.86:0/1494559932 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:47.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.986+0000 7f607a600640 1 --2- 172.21.15.86:0/1494559932 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f607406f450 0x7f607415bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:47.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.986+0000 7f6079c00640 1 -- 172.21.15.86:0/1494559932 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f607406f450 msgr2=0x7f607415bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:47.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.986+0000 7f6079c00640 1 --2- 172.21.15.86:0/1494559932 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f607406f450 0x7f607415bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:47.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.986+0000 7f6079c00640 1 -- 172.21.15.86:0/1494559932 >> v1:172.21.15.86:6789/0 conn(0x7f60740cd170 legacy=0x7f607406ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:47.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.987+0000 7f6079c00640 1 -- 172.21.15.86:0/1494559932 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60740b4130 con 0x7f6074144e90 2024-04-20T21:26:47.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.987+0000 7f6073e00640 1 -- 172.21.15.86:0/1494559932 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2245329886 0 0) 0x7f60740cdb50 con 0x7f60740cd170 2024-04-20T21:26:47.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.987+0000 7f607a600640 1 --2- 172.21.15.86:0/1494559932 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f607406f450 0x7f607415bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:26:47.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.988+0000 7f6079c00640 1 --2- 172.21.15.86:0/1494559932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6074144e90 0x7f60740b4930 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f6064009870 tx=0x7f606402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60d8aee9e025b794 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:47.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.988+0000 7f6073e00640 1 -- 172.21.15.86:0/1494559932 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6064034070 con 0x7f6074144e90 2024-04-20T21:26:47.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.988+0000 7f6073e00640 1 -- 172.21.15.86:0/1494559932 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6064030840 con 0x7f6074144e90 2024-04-20T21:26:47.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.988+0000 7f6073e00640 1 -- 172.21.15.86:0/1494559932 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6064030c10 con 0x7f6074144e90 2024-04-20T21:26:47.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.988+0000 7f607c600640 1 -- 172.21.15.86:0/1494559932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6074144e90 msgr2=0x7f60740b4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:47.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.988+0000 7f607c600640 1 --2- 172.21.15.86:0/1494559932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6074144e90 0x7f60740b4930 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f6064009870 tx=0x7f606402b980 comp rx=0 tx=0).stop 2024-04-20T21:26:47.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.989+0000 7f607c600640 1 -- 172.21.15.86:0/1494559932 shutdown_connections 2024-04-20T21:26:47.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.989+0000 7f607c600640 1 --2- 172.21.15.86:0/1494559932 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f607406f450 0x7f607415bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:47.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.989+0000 7f607c600640 1 --2- 172.21.15.86:0/1494559932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6074144e90 0x7f60740b4930 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:47.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.989+0000 7f607c600640 1 -- 172.21.15.86:0/1494559932 >> 172.21.15.86:0/1494559932 conn(0x7f60740b2360 msgr2=0x7f60740b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:47.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.989+0000 7f607c600640 1 -- 172.21.15.86:0/1494559932 shutdown_connections 2024-04-20T21:26:47.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.989+0000 7f607c600640 1 -- 172.21.15.86:0/1494559932 wait complete. 2024-04-20T21:26:47.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.990+0000 7f607c600640 1 Processor -- start 2024-04-20T21:26:47.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.990+0000 7f607c600640 1 -- start start 2024-04-20T21:26:47.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.991+0000 7f607c600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f607406f450 0x7f607415b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:47.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.991+0000 7f607c600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60740cd170 0x7f6074143410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:47.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.991+0000 7f607c600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6074144e90 0x7f6074143950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:47.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.991+0000 7f607c600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f60740d0010 con 0x7f60740cd170 2024-04-20T21:26:47.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.991+0000 7f607c600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f60740b4e70 con 0x7f6074144e90 2024-04-20T21:26:47.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.991+0000 7f607c600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f60740d0190 con 0x7f607406f450 2024-04-20T21:26:47.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.992+0000 7f6079200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60740cd170 0x7f6074143410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:47.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.992+0000 7f6079200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60740cd170 0x7f6074143410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55822/0 (socket says 172.21.15.86:55822) 2024-04-20T21:26:47.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.992+0000 7f6079200640 1 -- 172.21.15.86:0/1437546380 learned_addr learned my addr 172.21.15.86:0/1437546380 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:47.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.992+0000 7f6079c00640 1 --2- 172.21.15.86:0/1437546380 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f607406f450 0x7f607415b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:47.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.992+0000 7f6079200640 1 -- 172.21.15.86:0/1437546380 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f607406f450 msgr2=0x7f607415b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:47.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.992+0000 7f6079200640 1 --2- 172.21.15.86:0/1437546380 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f607406f450 0x7f607415b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:47.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.992+0000 7f6079200640 1 -- 172.21.15.86:0/1437546380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6074144e90 msgr2=0x7f6074143950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:47.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.992+0000 7f6079200640 1 --2- 172.21.15.86:0/1437546380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6074144e90 0x7f6074143950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:47.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.992+0000 7f6079200640 1 -- 172.21.15.86:0/1437546380 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6074168ed0 con 0x7f60740cd170 2024-04-20T21:26:47.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.992+0000 7f6079200640 1 --2- 172.21.15.86:0/1437546380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60740cd170 0x7f6074143410 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f606000c4e0 tx=0x7f606000c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:47.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.992+0000 7f6071600640 1 -- 172.21.15.86:0/1437546380 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6060017070 con 0x7f60740cd170 2024-04-20T21:26:47.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.993+0000 7f6071600640 1 -- 172.21.15.86:0/1437546380 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60600035f0 con 0x7f60740cd170 2024-04-20T21:26:47.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.993+0000 7f6071600640 1 -- 172.21.15.86:0/1437546380 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60600039e0 con 0x7f60740cd170 2024-04-20T21:26:47.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.993+0000 7f607c600640 1 -- 172.21.15.86:0/1437546380 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60741682c0 con 0x7f60740cd170 2024-04-20T21:26:47.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.993+0000 7f607c600640 1 -- 172.21.15.86:0/1437546380 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6074168880 con 0x7f60740cd170 2024-04-20T21:26:47.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.994+0000 7f607c600640 1 -- 172.21.15.86:0/1437546380 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60740689e0 con 0x7f60740cd170 2024-04-20T21:26:47.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.995+0000 7f6071600640 1 -- 172.21.15.86:0/1437546380 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f606001a070 con 0x7f60740cd170 2024-04-20T21:26:47.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.996+0000 7f6071600640 1 --2- 172.21.15.86:0/1437546380 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f605406c8f0 0x7f605406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:47.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.996+0000 7f6071600640 1 -- 172.21.15.86:0/1437546380 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7f606008ae90 con 0x7f60740cd170 2024-04-20T21:26:47.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.996+0000 7f6079c00640 1 --2- 172.21.15.86:0/1437546380 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f605406c8f0 0x7f605406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:47.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:47.997+0000 7f6079c00640 1 --2- 172.21.15.86:0/1437546380 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f605406c8f0 0x7f605406edb0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f6064036000 tx=0x7f6064008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:47.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:48.001+0000 7f6071600640 1 -- 172.21.15.86:0/1437546380 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6060058b90 con 0x7f60740cd170 2024-04-20T21:26:48.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:48.238+0000 7f607c600640 1 -- 172.21.15.86:0/1437546380 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f607406cce0 con 0x7f60740cd170 2024-04-20T21:26:48.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:48.239+0000 7f6071600640 1 -- 172.21.15.86:0/1437546380 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5202 (secure 0 0 0) 0x7f606008c070 con 0x7f60740cd170 2024-04-20T21:26:48.238 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:26:48.238 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":72,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5887,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2433619638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2433619638},{"type":"v1","addr":"172.21.15.144:6835","nonce":2433619638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68},{"gid":7506,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/4013863257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4013863257},{"type":"v1","addr":"172.21.15.86:6836","nonce":4013863257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":7509,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2044883073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2044883073},{"type":"v1","addr":"172.21.15.86:6837","nonce":2044883073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:24:58.299257+0000","modified":"2024-04-20T21:26:46.903173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7518},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7518":{"gid":7518,"name":"d","rank":0,"incarnation":70,"state":"up:rejoin","state_seq":31,"addr":"172.21.15.144:6834/2351381466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2351381466},{"type":"v1","addr":"172.21.15.144:6834","nonce":2351381466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-04-20T21:26:48.239 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 72 2024-04-20T21:26:48.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:48.242+0000 7f607c600640 1 -- 172.21.15.86:0/1437546380 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f605406c8f0 msgr2=0x7f605406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:48.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:48.242+0000 7f607c600640 1 --2- 172.21.15.86:0/1437546380 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f605406c8f0 0x7f605406edb0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f6064036000 tx=0x7f6064008030 comp rx=0 tx=0).stop 2024-04-20T21:26:48.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:48.242+0000 7f607c600640 1 -- 172.21.15.86:0/1437546380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60740cd170 msgr2=0x7f6074143410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:48.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:48.242+0000 7f607c600640 1 --2- 172.21.15.86:0/1437546380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60740cd170 0x7f6074143410 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f606000c4e0 tx=0x7f606000c9a0 comp rx=0 tx=0).stop 2024-04-20T21:26:48.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:48.242+0000 7f607c600640 1 -- 172.21.15.86:0/1437546380 shutdown_connections 2024-04-20T21:26:48.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:48.242+0000 7f607c600640 1 --2- 172.21.15.86:0/1437546380 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f605406c8f0 0x7f605406edb0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:48.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:48.242+0000 7f607c600640 1 --2- 172.21.15.86:0/1437546380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6074144e90 0x7f6074143950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:48.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:48.242+0000 7f607c600640 1 --2- 172.21.15.86:0/1437546380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60740cd170 0x7f6074143410 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:48.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:48.242+0000 7f607c600640 1 --2- 172.21.15.86:0/1437546380 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f607406f450 0x7f607415b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:48.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:48.242+0000 7f607c600640 1 -- 172.21.15.86:0/1437546380 >> 172.21.15.86:0/1437546380 conn(0x7f60740b2360 msgr2=0x7f607413e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:48.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:48.242+0000 7f607c600640 1 -- 172.21.15.86:0/1437546380 shutdown_connections 2024-04-20T21:26:48.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:48.242+0000 7f607c600640 1 -- 172.21.15.86:0/1437546380 wait complete. 2024-04-20T21:26:48.663 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 72, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T21:24:58.299257+0000', 'modified': '2024-04-20T21:26:46.903173+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 89, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7518}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7518': {'gid': 7518, 'name': 'd', 'rank': 0, 'incarnation': 70, 'state': 'up:rejoin', 'state_seq': 31, 'addr': '172.21.15.144:6834/2351381466', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6832', 'nonce': 2351381466}, {'type': 'v1', 'addr': '172.21.15.144:6834', 'nonce': 2351381466}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-20T21:26:48.663 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7518:up:rejoin 2024-04-20T21:26:49.664 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:26:49.804 INFO:tasks.ceph.mds.d.smithi144.stderr:2024-04-20T21:26:49.805+0000 7fec9ae00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T21:26:49.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.878+0000 7fbd58400640 1 Processor -- start 2024-04-20T21:26:49.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.878+0000 7fbd58400640 1 -- start start 2024-04-20T21:26:49.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.880+0000 7fbd58400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbd50144e90 0x7fbd5015ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:49.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.880+0000 7fbd58400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd5006dc30 con 0x7fbd5006f140 2024-04-20T21:26:49.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.880+0000 7fbd58400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd5015f720 con 0x7fbd50071d00 2024-04-20T21:26:49.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.880+0000 7fbd58400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fbd50145270 con 0x7fbd50144e90 2024-04-20T21:26:49.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.880+0000 7fbd55a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbd50144e90 0x7fbd5015ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:49.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.880+0000 7fbd55a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbd50144e90 0x7fbd5015ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55830/0 (socket says 172.21.15.86:55830) 2024-04-20T21:26:49.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.880+0000 7fbd55000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fbd5006f140 0x7fbd500aeb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:36800/0 (socket says 172.21.15.86:36800) 2024-04-20T21:26:49.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.880+0000 7fbd55a00640 1 -- 172.21.15.86:0/70297174 learned_addr learned my addr 172.21.15.86:0/70297174 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:49.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.881+0000 7fbd55a00640 1 -- 172.21.15.86:0/70297174 >> v1:172.21.15.144:6789/0 conn(0x7fbd50071d00 legacy=0x7fbd5006e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:49.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.881+0000 7fbd55a00640 1 -- 172.21.15.86:0/70297174 >> v1:172.21.15.86:6789/0 conn(0x7fbd5006f140 legacy=0x7fbd500aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:49.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.881+0000 7fbd55a00640 1 -- 172.21.15.86:0/70297174 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd5015fa00 con 0x7fbd50144e90 2024-04-20T21:26:49.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.882+0000 7fbd55a00640 1 --2- 172.21.15.86:0/70297174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd50144e90 0x7fbd5015ee10 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fbd48009870 tx=0x7fbd4802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4362e66f16864bf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:49.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.882+0000 7fbd4fe00640 1 -- 172.21.15.86:0/70297174 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd48034070 con 0x7fbd50144e90 2024-04-20T21:26:49.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.882+0000 7fbd4fe00640 1 -- 172.21.15.86:0/70297174 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd48030860 con 0x7fbd50144e90 2024-04-20T21:26:49.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.882+0000 7fbd4fe00640 1 -- 172.21.15.86:0/70297174 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd48030c30 con 0x7fbd50144e90 2024-04-20T21:26:49.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.883+0000 7fbd58400640 1 -- 172.21.15.86:0/70297174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd50144e90 msgr2=0x7fbd5015ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:49.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.883+0000 7fbd58400640 1 --2- 172.21.15.86:0/70297174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd50144e90 0x7fbd5015ee10 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fbd48009870 tx=0x7fbd4802b9a0 comp rx=0 tx=0).stop 2024-04-20T21:26:49.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.883+0000 7fbd58400640 1 -- 172.21.15.86:0/70297174 shutdown_connections 2024-04-20T21:26:49.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.883+0000 7fbd58400640 1 --2- 172.21.15.86:0/70297174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd50144e90 0x7fbd5015ee10 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:49.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.883+0000 7fbd58400640 1 -- 172.21.15.86:0/70297174 >> 172.21.15.86:0/70297174 conn(0x7fbd500b2360 msgr2=0x7fbd500b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:49.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.883+0000 7fbd58400640 1 -- 172.21.15.86:0/70297174 shutdown_connections 2024-04-20T21:26:49.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.883+0000 7fbd58400640 1 -- 172.21.15.86:0/70297174 wait complete. 2024-04-20T21:26:49.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.885+0000 7fbd58400640 1 Processor -- start 2024-04-20T21:26:49.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.885+0000 7fbd58400640 1 -- start start 2024-04-20T21:26:49.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.886+0000 7fbd58400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd5006f140 0x7fbd5015e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:49.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.886+0000 7fbd58400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd50071d00 0x7fbd5013fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:49.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.886+0000 7fbd58400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd50144e90 0x7fbd50141b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:49.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.886+0000 7fbd58400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5015f350 con 0x7fbd50144e90 2024-04-20T21:26:49.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.886+0000 7fbd58400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5006ded0 con 0x7fbd5006f140 2024-04-20T21:26:49.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.886+0000 7fbd58400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbd5015f4d0 con 0x7fbd50071d00 2024-04-20T21:26:49.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.886+0000 7fbd56400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd50144e90 0x7fbd50141b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:49.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.886+0000 7fbd56400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd50144e90 0x7fbd50141b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55834/0 (socket says 172.21.15.86:55834) 2024-04-20T21:26:49.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.886+0000 7fbd56400640 1 -- 172.21.15.86:0/640937259 learned_addr learned my addr 172.21.15.86:0/640937259 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:49.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.886+0000 7fbd55a00640 1 --2- 172.21.15.86:0/640937259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd5006f140 0x7fbd5015e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:49.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.886+0000 7fbd55000640 1 --2- 172.21.15.86:0/640937259 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd50071d00 0x7fbd5013fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:49.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.886+0000 7fbd56400640 1 -- 172.21.15.86:0/640937259 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd50071d00 msgr2=0x7fbd5013fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:49.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.886+0000 7fbd56400640 1 --2- 172.21.15.86:0/640937259 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd50071d00 0x7fbd5013fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:49.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.886+0000 7fbd56400640 1 -- 172.21.15.86:0/640937259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd5006f140 msgr2=0x7fbd5015e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:49.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.886+0000 7fbd56400640 1 --2- 172.21.15.86:0/640937259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd5006f140 0x7fbd5015e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:49.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.887+0000 7fbd56400640 1 -- 172.21.15.86:0/640937259 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd50140500 con 0x7fbd50144e90 2024-04-20T21:26:49.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.887+0000 7fbd56400640 1 --2- 172.21.15.86:0/640937259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd50144e90 0x7fbd50141b50 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fbd4000b410 tx=0x7fbd4000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:49.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.887+0000 7fbd4d600640 1 -- 172.21.15.86:0/640937259 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd40015030 con 0x7fbd50144e90 2024-04-20T21:26:49.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.887+0000 7fbd4d600640 1 -- 172.21.15.86:0/640937259 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd40004020 con 0x7fbd50144e90 2024-04-20T21:26:49.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.887+0000 7fbd4d600640 1 -- 172.21.15.86:0/640937259 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd40004410 con 0x7fbd50144e90 2024-04-20T21:26:49.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.887+0000 7fbd58400640 1 -- 172.21.15.86:0/640937259 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd500afc60 con 0x7fbd50144e90 2024-04-20T21:26:49.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.887+0000 7fbd58400640 1 -- 172.21.15.86:0/640937259 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd500b0220 con 0x7fbd50144e90 2024-04-20T21:26:49.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.888+0000 7fbd58400640 1 -- 172.21.15.86:0/640937259 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd500689e0 con 0x7fbd50144e90 2024-04-20T21:26:49.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.889+0000 7fbd4d600640 1 -- 172.21.15.86:0/640937259 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbd40018070 con 0x7fbd50144e90 2024-04-20T21:26:49.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.890+0000 7fbd4d600640 1 --2- 172.21.15.86:0/640937259 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbd3006c8f0 0x7fbd3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:49.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.890+0000 7fbd4d600640 1 -- 172.21.15.86:0/640937259 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7fbd4008b780 con 0x7fbd50144e90 2024-04-20T21:26:49.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.890+0000 7fbd55a00640 1 --2- 172.21.15.86:0/640937259 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbd3006c8f0 0x7fbd3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:49.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.891+0000 7fbd55a00640 1 --2- 172.21.15.86:0/640937259 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbd3006c8f0 0x7fbd3006edb0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fbd48036000 tx=0x7fbd4803b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:49.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:49.895+0000 7fbd4d600640 1 -- 172.21.15.86:0/640937259 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbd40059a40 con 0x7fbd50144e90 2024-04-20T21:26:50.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.117+0000 7fbd58400640 1 -- 172.21.15.86:0/640937259 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd50072530 con 0x7fbd50144e90 2024-04-20T21:26:50.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.118+0000 7fbd4d600640 1 -- 172.21.15.86:0/640937259 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5209 (secure 0 0 0) 0x7fbd40059340 con 0x7fbd50144e90 2024-04-20T21:26:50.117 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:26:50.117 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":73,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5887,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2433619638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2433619638},{"type":"v1","addr":"172.21.15.144:6835","nonce":2433619638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68},{"gid":7506,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/4013863257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4013863257},{"type":"v1","addr":"172.21.15.86:6836","nonce":4013863257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":7509,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2044883073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2044883073},{"type":"v1","addr":"172.21.15.86:6837","nonce":2044883073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:24:58.299257+0000","modified":"2024-04-20T21:26:48.911892+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7518},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7518":{"gid":7518,"name":"d","rank":0,"incarnation":70,"state":"up:active","state_seq":32,"addr":"172.21.15.144:6834/2351381466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2351381466},{"type":"v1","addr":"172.21.15.144:6834","nonce":2351381466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7518,"qdb_cluster":[7518]},"id":6}]} 2024-04-20T21:26:50.118 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 73 2024-04-20T21:26:50.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.120+0000 7fbd58400640 1 -- 172.21.15.86:0/640937259 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbd3006c8f0 msgr2=0x7fbd3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:50.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.120+0000 7fbd58400640 1 --2- 172.21.15.86:0/640937259 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbd3006c8f0 0x7fbd3006edb0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fbd48036000 tx=0x7fbd4803b040 comp rx=0 tx=0).stop 2024-04-20T21:26:50.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.120+0000 7fbd58400640 1 -- 172.21.15.86:0/640937259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd50144e90 msgr2=0x7fbd50141b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:50.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.120+0000 7fbd58400640 1 --2- 172.21.15.86:0/640937259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd50144e90 0x7fbd50141b50 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fbd4000b410 tx=0x7fbd4000b8d0 comp rx=0 tx=0).stop 2024-04-20T21:26:50.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.121+0000 7fbd58400640 1 -- 172.21.15.86:0/640937259 shutdown_connections 2024-04-20T21:26:50.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.121+0000 7fbd58400640 1 --2- 172.21.15.86:0/640937259 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbd3006c8f0 0x7fbd3006edb0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:50.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.121+0000 7fbd58400640 1 --2- 172.21.15.86:0/640937259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd50144e90 0x7fbd50141b50 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:50.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.121+0000 7fbd58400640 1 --2- 172.21.15.86:0/640937259 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd50071d00 0x7fbd5013fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:50.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.121+0000 7fbd58400640 1 --2- 172.21.15.86:0/640937259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd5006f140 0x7fbd5015e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:50.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.121+0000 7fbd58400640 1 -- 172.21.15.86:0/640937259 >> 172.21.15.86:0/640937259 conn(0x7fbd500b2360 msgr2=0x7fbd50161210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:50.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.121+0000 7fbd58400640 1 -- 172.21.15.86:0/640937259 shutdown_connections 2024-04-20T21:26:50.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.121+0000 7fbd58400640 1 -- 172.21.15.86:0/640937259 wait complete. 2024-04-20T21:26:50.540 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 73, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T21:24:58.299257+0000', 'modified': '2024-04-20T21:26:48.911892+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 89, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7518}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7518': {'gid': 7518, 'name': 'd', 'rank': 0, 'incarnation': 70, 'state': 'up:active', 'state_seq': 32, 'addr': '172.21.15.144:6834/2351381466', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.144:6832', 'nonce': 2351381466}, {'type': 'v1', 'addr': '172.21.15.144:6834', 'nonce': 2351381466}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7518, 'qdb_cluster': [7518]} 2024-04-20T21:26:50.540 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-20T21:26:50.540 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-20T21:26:50.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.747+0000 7feea4000640 1 Processor -- start 2024-04-20T21:26:50.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.748+0000 7feea4000640 1 -- start start 2024-04-20T21:26:50.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.749+0000 7feea4000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee9c15ca70 0x7fee9c0cd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:50.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.749+0000 7feea4000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fee9c15ce50 0x7fee9c09a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:50.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.749+0000 7feea4000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fee9c09b580 0x7fee9c09af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:50.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.749+0000 7feea4000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fee9c0ce280 con 0x7fee9c15ce50 2024-04-20T21:26:50.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.749+0000 7feea4000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fee9c0ce7b0 con 0x7fee9c09b580 2024-04-20T21:26:50.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.749+0000 7feea4000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fee9c0c6800 con 0x7fee9c15ca70 2024-04-20T21:26:50.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.750+0000 7feea0c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fee9c15ce50 0x7fee9c09a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:50.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.750+0000 7feea0c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fee9c15ce50 0x7fee9c09a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55842/0 (socket says 172.21.15.86:55842) 2024-04-20T21:26:50.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.750+0000 7feea0c00640 1 -- 172.21.15.86:0/1629056842 learned_addr learned my addr 172.21.15.86:0/1629056842 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:50.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.750+0000 7feea1600640 1 --2- 172.21.15.86:0/1629056842 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee9c15ca70 0x7fee9c0cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:50.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.750+0000 7feea2000640 1 --2- 172.21.15.86:0/1629056842 >> v2:172.21.15.144:3300/0 conn(0x7fee9c09b580 0x7fee9c09af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:50.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.750+0000 7feea0c00640 1 -- 172.21.15.86:0/1629056842 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee9c15ca70 msgr2=0x7fee9c0cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:50.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.750+0000 7feea0c00640 1 --2- 172.21.15.86:0/1629056842 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee9c15ca70 0x7fee9c0cd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:50.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.750+0000 7feea0c00640 1 -- 172.21.15.86:0/1629056842 >> v2:172.21.15.144:3300/0 conn(0x7fee9c09b580 msgr2=0x7fee9c09af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:50.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.750+0000 7feea0c00640 1 --2- 172.21.15.86:0/1629056842 >> v2:172.21.15.144:3300/0 conn(0x7fee9c09b580 0x7fee9c09af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:50.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.750+0000 7feea0c00640 1 -- 172.21.15.86:0/1629056842 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee9c0c9480 con 0x7fee9c15ce50 2024-04-20T21:26:50.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.750+0000 7feea1600640 1 --2- 172.21.15.86:0/1629056842 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee9c15ca70 0x7fee9c0cd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:26:50.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.751+0000 7feea0c00640 1 --2- 172.21.15.86:0/1629056842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee9c15ce50 0x7fee9c09a9e0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fee9c0c91e0 tx=0x7fee8802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1daef1df1a942e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:50.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.752+0000 7fee9be00640 1 -- 172.21.15.86:0/1629056842 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee88035020 con 0x7fee9c15ce50 2024-04-20T21:26:50.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.752+0000 7fee9be00640 1 -- 172.21.15.86:0/1629056842 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee88031870 con 0x7fee9c15ce50 2024-04-20T21:26:50.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.752+0000 7fee9be00640 1 -- 172.21.15.86:0/1629056842 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee88031c40 con 0x7fee9c15ce50 2024-04-20T21:26:50.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.752+0000 7feea4000640 1 -- 172.21.15.86:0/1629056842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee9c15ce50 msgr2=0x7fee9c09a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:50.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.752+0000 7feea4000640 1 --2- 172.21.15.86:0/1629056842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee9c15ce50 0x7fee9c09a9e0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fee9c0c91e0 tx=0x7fee8802b910 comp rx=0 tx=0).stop 2024-04-20T21:26:50.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.753+0000 7feea4000640 1 -- 172.21.15.86:0/1629056842 shutdown_connections 2024-04-20T21:26:50.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.753+0000 7feea4000640 1 --2- 172.21.15.86:0/1629056842 >> v2:172.21.15.144:3300/0 conn(0x7fee9c09b580 0x7fee9c09af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:50.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.753+0000 7feea4000640 1 --2- 172.21.15.86:0/1629056842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee9c15ce50 0x7fee9c09a9e0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:50.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.753+0000 7feea4000640 1 --2- 172.21.15.86:0/1629056842 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee9c15ca70 0x7fee9c0cd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:50.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.753+0000 7feea4000640 1 -- 172.21.15.86:0/1629056842 >> 172.21.15.86:0/1629056842 conn(0x7fee9c0b40b0 msgr2=0x7fee9c0b44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:50.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.753+0000 7feea4000640 1 -- 172.21.15.86:0/1629056842 shutdown_connections 2024-04-20T21:26:50.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.753+0000 7feea4000640 1 -- 172.21.15.86:0/1629056842 wait complete. 2024-04-20T21:26:50.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.754+0000 7feea4000640 1 Processor -- start 2024-04-20T21:26:50.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.755+0000 7feea4000640 1 -- start start 2024-04-20T21:26:50.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.755+0000 7feea4000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee9c09b580 0x7fee9c0b5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:50.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.755+0000 7feea4000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee9c15ca70 0x7fee9c14fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:50.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.755+0000 7feea4000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee9c15ce50 0x7fee9c0b4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:50.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.755+0000 7feea4000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fee9c0c6490 con 0x7fee9c09b580 2024-04-20T21:26:50.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.755+0000 7feea4000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fee9c0c61b0 con 0x7fee9c15ce50 2024-04-20T21:26:50.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.755+0000 7feea4000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fee9c0d04c0 con 0x7fee9c15ca70 2024-04-20T21:26:50.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.756+0000 7feea1600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee9c09b580 0x7fee9c0b5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:50.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.756+0000 7feea1600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee9c09b580 0x7fee9c0b5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55846/0 (socket says 172.21.15.86:55846) 2024-04-20T21:26:50.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.756+0000 7feea1600640 1 -- 172.21.15.86:0/661906177 learned_addr learned my addr 172.21.15.86:0/661906177 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:50.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.756+0000 7feea1600640 1 -- 172.21.15.86:0/661906177 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee9c15ca70 msgr2=0x7fee9c14fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:50.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.756+0000 7feea1600640 1 --2- 172.21.15.86:0/661906177 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee9c15ca70 0x7fee9c14fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:50.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.756+0000 7feea1600640 1 -- 172.21.15.86:0/661906177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee9c15ce50 msgr2=0x7fee9c0b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:50.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.756+0000 7feea1600640 1 --2- 172.21.15.86:0/661906177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee9c15ce50 0x7fee9c0b4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:50.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.756+0000 7feea1600640 1 -- 172.21.15.86:0/661906177 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee9c14ff70 con 0x7fee9c09b580 2024-04-20T21:26:50.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.756+0000 7feea1600640 1 --2- 172.21.15.86:0/661906177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee9c09b580 0x7fee9c0b5e20 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fee8c00e7e0 tx=0x7fee8c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:50.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.757+0000 7fee99600640 1 -- 172.21.15.86:0/661906177 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c00f040 con 0x7fee9c09b580 2024-04-20T21:26:50.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.757+0000 7fee99600640 1 -- 172.21.15.86:0/661906177 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee8c004030 con 0x7fee9c09b580 2024-04-20T21:26:50.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.757+0000 7fee99600640 1 -- 172.21.15.86:0/661906177 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c004400 con 0x7fee9c09b580 2024-04-20T21:26:50.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.757+0000 7feea4000640 1 -- 172.21.15.86:0/661906177 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee9c0cb9e0 con 0x7fee9c09b580 2024-04-20T21:26:50.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.757+0000 7feea4000640 1 -- 172.21.15.86:0/661906177 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee9c0cbfa0 con 0x7fee9c09b580 2024-04-20T21:26:50.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.759+0000 7fee99600640 1 -- 172.21.15.86:0/661906177 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fee8c00c6d0 con 0x7fee9c09b580 2024-04-20T21:26:50.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.760+0000 7fee99600640 1 --2- 172.21.15.86:0/661906177 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fee7c06c8f0 0x7fee7c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:50.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.761+0000 7fee99600640 1 -- 172.21.15.86:0/661906177 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7fee8c08ba40 con 0x7fee9c09b580 2024-04-20T21:26:50.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.761+0000 7feea0c00640 1 --2- 172.21.15.86:0/661906177 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fee7c06c8f0 0x7fee7c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:50.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.761+0000 7feea6e09740 1 Processor -- start 2024-04-20T21:26:50.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.761+0000 7feea6e09740 1 -- start start 2024-04-20T21:26:50.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.761+0000 7feea0c00640 1 --2- 172.21.15.86:0/661906177 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fee7c06c8f0 0x7fee7c06edb0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fee88036000 tx=0x7fee88030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:50.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.762+0000 7feea6e09740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddeb6cc460 0x55ddeb683a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:50.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.762+0000 7feea6e09740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ddeb6824e0 0x55ddeb683fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:50.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.762+0000 7feea6e09740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ddeb67e930 0x55ddeb6ae770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:50.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.762+0000 7feea6e09740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ddeb5ea9d0 con 0x55ddeb6cc460 2024-04-20T21:26:50.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.762+0000 7feea6e09740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ddeb591270 con 0x55ddeb67e930 2024-04-20T21:26:50.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.762+0000 7feea6e09740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55ddeb5e6990 con 0x55ddeb6824e0 2024-04-20T21:26:50.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.762+0000 7feea1600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddeb6cc460 0x55ddeb683a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:50.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.762+0000 7feea1600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddeb6cc460 0x55ddeb683a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55858/0 (socket says 172.21.15.86:55858) 2024-04-20T21:26:50.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.762+0000 7feea1600640 1 -- 172.21.15.86:0/748143807 learned_addr learned my addr 172.21.15.86:0/748143807 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:50.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.763+0000 7feea0c00640 1 --2- 172.21.15.86:0/748143807 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ddeb6824e0 0x55ddeb683fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:50.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.763+0000 7feea1600640 1 -- 172.21.15.86:0/748143807 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ddeb6824e0 msgr2=0x55ddeb683fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:50.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.763+0000 7feea1600640 1 --2- 172.21.15.86:0/748143807 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ddeb6824e0 0x55ddeb683fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:50.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.763+0000 7feea1600640 1 -- 172.21.15.86:0/748143807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ddeb67e930 msgr2=0x55ddeb6ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:50.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.763+0000 7feea2000640 1 --2- 172.21.15.86:0/748143807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ddeb67e930 0x55ddeb6ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:50.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.763+0000 7feea1600640 1 --2- 172.21.15.86:0/748143807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ddeb67e930 0x55ddeb6ae770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:50.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.763+0000 7feea1600640 1 -- 172.21.15.86:0/748143807 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddeb2fd2d0 con 0x55ddeb6cc460 2024-04-20T21:26:50.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.763+0000 7feea1600640 1 --2- 172.21.15.86:0/748143807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddeb6cc460 0x55ddeb683a90 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fee8c006dc0 tx=0x7fee8c059750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=574c0bfba61ecafd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:50.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.763+0000 7fee91c00640 1 -- 172.21.15.86:0/748143807 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c088070 con 0x55ddeb6cc460 2024-04-20T21:26:50.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.763+0000 7fee91c00640 1 -- 172.21.15.86:0/748143807 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee8c055c80 con 0x55ddeb6cc460 2024-04-20T21:26:50.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.763+0000 7fee91c00640 1 -- 172.21.15.86:0/748143807 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c020050 con 0x55ddeb6cc460 2024-04-20T21:26:50.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.764+0000 7feea6e09740 1 -- 172.21.15.86:0/748143807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddeb6cc460 msgr2=0x55ddeb683a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:50.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.764+0000 7feea6e09740 1 --2- 172.21.15.86:0/748143807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddeb6cc460 0x55ddeb683a90 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fee8c006dc0 tx=0x7fee8c059750 comp rx=0 tx=0).stop 2024-04-20T21:26:50.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.764+0000 7feea6e09740 1 -- 172.21.15.86:0/748143807 shutdown_connections 2024-04-20T21:26:50.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.764+0000 7feea6e09740 1 --2- 172.21.15.86:0/748143807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ddeb67e930 0x55ddeb6ae770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:50.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.764+0000 7feea6e09740 1 --2- 172.21.15.86:0/748143807 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ddeb6824e0 0x55ddeb683fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:50.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.764+0000 7feea6e09740 1 --2- 172.21.15.86:0/748143807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddeb6cc460 0x55ddeb683a90 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:50.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.764+0000 7feea6e09740 1 -- 172.21.15.86:0/748143807 >> 172.21.15.86:0/748143807 conn(0x55ddeb680bf0 msgr2=0x55ddeb6c9770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:50.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.764+0000 7feea6e09740 1 -- 172.21.15.86:0/748143807 shutdown_connections 2024-04-20T21:26:50.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.764+0000 7feea6e09740 1 -- 172.21.15.86:0/748143807 wait complete. 2024-04-20T21:26:50.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.765+0000 7feea6e09740 1 -- start start 2024-04-20T21:26:50.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.765+0000 7feea6e09740 1 Processor -- start 2024-04-20T21:26:50.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.766+0000 7feea6e09740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ddeb6824e0 0x55ddeb6926a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:50.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.766+0000 7feea6e09740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ddeb6cc460 0x55ddeb690c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:50.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.766+0000 7feea6e09740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddeb693370 0x55ddeb6937f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:50.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.766+0000 7feea6e09740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ddeb6829f0 con 0x55ddeb693370 2024-04-20T21:26:50.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.766+0000 7feea6e09740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ddeb5a45e0 con 0x55ddeb6cc460 2024-04-20T21:26:50.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.766+0000 7feea6e09740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55ddeb682cd0 con 0x55ddeb6824e0 2024-04-20T21:26:50.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.766+0000 7feea6e09740 10 client.0 _ll_register_callbacks cb 0x55ddeb2fcac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:26:50.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.767+0000 7feea2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddeb693370 0x55ddeb6937f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:50.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.767+0000 7feea2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddeb693370 0x55ddeb6937f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55874/0 (socket says 172.21.15.86:55874) 2024-04-20T21:26:50.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.767+0000 7feea2000640 1 -- 172.21.15.86:0/3394846164 learned_addr learned my addr 172.21.15.86:0/3394846164 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:50.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.767+0000 7feea1600640 1 --2- 172.21.15.86:0/3394846164 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ddeb6824e0 0x55ddeb6926a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:50.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.767+0000 7feea2000640 1 -- 172.21.15.86:0/3394846164 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ddeb6824e0 msgr2=0x55ddeb6926a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:50.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.767+0000 7feea2000640 1 --2- 172.21.15.86:0/3394846164 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ddeb6824e0 0x55ddeb6926a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:50.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.767+0000 7feea2000640 1 -- 172.21.15.86:0/3394846164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ddeb6cc460 msgr2=0x55ddeb690c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:50.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.767+0000 7feea2000640 1 --2- 172.21.15.86:0/3394846164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ddeb6cc460 0x55ddeb690c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:50.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.767+0000 7feea2000640 1 -- 172.21.15.86:0/3394846164 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddeb3c76b0 con 0x55ddeb693370 2024-04-20T21:26:50.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.767+0000 7feea2000640 1 --2- 172.21.15.86:0/3394846164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddeb693370 0x55ddeb6937f0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fee940079d0 tx=0x7fee94007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e362b1f8bb9d56b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:50.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.767+0000 7fee86000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:26:50.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.768+0000 7fee86000640 1 -- 172.21.15.86:0/3394846164 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee94018020 con 0x55ddeb693370 2024-04-20T21:26:50.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.768+0000 7fee86000640 1 -- 172.21.15.86:0/3394846164 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7fee9400fba0 con 0x55ddeb693370 2024-04-20T21:26:50.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.768+0000 7feea6e09740 1 -- 172.21.15.86:0/3394846164 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee8c055c80 con 0x55ddeb693370 2024-04-20T21:26:50.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.768+0000 7fee86000640 1 -- 172.21.15.86:0/3394846164 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7fee94022050 con 0x55ddeb693370 2024-04-20T21:26:50.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.768+0000 7feea6e09740 10 client.8175 fetch_fsmap learned FSMap version 73 2024-04-20T21:26:50.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.768+0000 7feea6e09740 1 -- 172.21.15.86:0/3394846164 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) -- 0x55ddeb6918e0 con 0x55ddeb693370 2024-04-20T21:26:50.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.768+0000 7fee86000640 1 -- 172.21.15.86:0/3394846164 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee8c055c80 con 0x55ddeb693370 2024-04-20T21:26:50.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.768+0000 7fee86000640 1 -- 172.21.15.86:0/3394846164 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 73) ==== 3495+0+0 (secure 0 0 0) 0x7fee94025020 con 0x55ddeb693370 2024-04-20T21:26:50.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.768+0000 7feea6e09740 10 client.8175 fetch_fsmap finished waiting for FSMap version 73 2024-04-20T21:26:50.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.768+0000 7feea6e09740 10 client.8175 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-20T21:26:50.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.768+0000 7feea6e09740 20 client.8175 populate_metadata read hostname 'smithi086' 2024-04-20T21:26:50.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.768+0000 7feea6e09740 1 --2- 172.21.15.86:0/3394846164 >> [v2:172.21.15.144:6832/2351381466,v1:172.21.15.144:6834/2351381466] conn(0x55ddeb6aa340 0x55ddeb6e9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:50.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.768+0000 7feea6e09740 4 client.8175 mds_command: new command op to 7518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:26:50.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.768+0000 7feea6e09740 1 -- 172.21.15.86:0/3394846164 --> [v2:172.21.15.144:6832/2351381466,v1:172.21.15.144:6834/2351381466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ddeb5f83f0 con 0x55ddeb6aa340 2024-04-20T21:26:50.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.769+0000 7feea1600640 1 --2- 172.21.15.86:0/3394846164 >> [v2:172.21.15.144:6832/2351381466,v1:172.21.15.144:6834/2351381466] conn(0x55ddeb6aa340 0x55ddeb6e9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:50.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.770+0000 7feea1600640 1 --2- 172.21.15.86:0/3394846164 >> [v2:172.21.15.144:6832/2351381466,v1:172.21.15.144:6834/2351381466] conn(0x55ddeb6aa340 0x55ddeb6e9010 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac0d1fe2f31f1e4a server_cookie=9d709b9e2234983 in_seq=0 out_seq=0 2024-04-20T21:26:50.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.770+0000 7fee86000640 10 client.8175 ms_handle_connect on v2:172.21.15.144:6832/2351381466 2024-04-20T21:26:50.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.772+0000 7fee86000640 1 -- 172.21.15.86:0/3394846164 <== mds.0 v2:172.21.15.144:6832/2351381466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ddeb5f83f0 con 0x55ddeb6aa340 2024-04-20T21:26:50.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.772+0000 7fee86000640 10 client.8175 handle_command_reply: tid=0 2024-04-20T21:26:50.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.772+0000 7feea6e09740 1 client.8175 shutdown 2024-04-20T21:26:50.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.772+0000 7feea6e09740 1 -- 172.21.15.86:0/3394846164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddeb693370 msgr2=0x55ddeb6937f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:50.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.772+0000 7feea6e09740 1 --2- 172.21.15.86:0/3394846164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddeb693370 0x55ddeb6937f0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fee940079d0 tx=0x7fee94007dd0 comp rx=0 tx=0).stop 2024-04-20T21:26:50.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.773+0000 7feea6e09740 1 -- 172.21.15.86:0/3394846164 shutdown_connections 2024-04-20T21:26:50.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.773+0000 7feea6e09740 1 --2- 172.21.15.86:0/3394846164 >> [v2:172.21.15.144:6832/2351381466,v1:172.21.15.144:6834/2351381466] conn(0x55ddeb6aa340 0x55ddeb6e9010 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:50.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.773+0000 7feea6e09740 1 --2- 172.21.15.86:0/3394846164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddeb693370 0x55ddeb6937f0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:50.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.773+0000 7feea6e09740 1 --2- 172.21.15.86:0/3394846164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ddeb6cc460 0x55ddeb690c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:50.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.773+0000 7feea6e09740 1 --2- 172.21.15.86:0/3394846164 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ddeb6824e0 0x55ddeb6926a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:50.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.773+0000 7fee86000640 0 client.8175 ms_handle_reset on v2:172.21.15.144:6832/2351381466 2024-04-20T21:26:50.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.773+0000 7feea6e09740 1 -- 172.21.15.86:0/3394846164 >> 172.21.15.86:0/3394846164 conn(0x55ddeb67e930 msgr2=0x55ddeb6c9570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:50.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.773+0000 7feea6e09740 1 -- 172.21.15.86:0/3394846164 shutdown_connections 2024-04-20T21:26:50.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.773+0000 7feea6e09740 1 -- 172.21.15.86:0/3394846164 wait complete. 2024-04-20T21:26:50.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.773+0000 7feea6e09740 20 client.8175 trim_cache size 0 max 16384 2024-04-20T21:26:50.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.793+0000 7feea6e09740 1 Processor -- start 2024-04-20T21:26:50.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.793+0000 7feea6e09740 1 -- start start 2024-04-20T21:26:50.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.793+0000 7feea6e09740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ddeb6824e0 0x55ddeb6845a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:50.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.793+0000 7feea6e09740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ddeb693370 0x55ddeb684ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:50.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.793+0000 7feea6e09740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddeb6cc460 0x55ddeb6d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:50.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.793+0000 7feea6e09740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fee9c201120 con 0x55ddeb6cc460 2024-04-20T21:26:50.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.793+0000 7feea6e09740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fee9c200120 con 0x55ddeb6824e0 2024-04-20T21:26:50.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.793+0000 7feea6e09740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fee9c159040 con 0x55ddeb693370 2024-04-20T21:26:50.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.794+0000 7feea2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddeb6cc460 0x55ddeb6d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:50.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.794+0000 7feea2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddeb6cc460 0x55ddeb6d0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55886/0 (socket says 172.21.15.86:55886) 2024-04-20T21:26:50.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.794+0000 7feea2000640 1 -- 172.21.15.86:0/38999761 learned_addr learned my addr 172.21.15.86:0/38999761 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:50.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.794+0000 7feea1600640 1 --2- 172.21.15.86:0/38999761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ddeb6824e0 0x55ddeb6845a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:50.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.794+0000 7feea2000640 1 -- 172.21.15.86:0/38999761 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ddeb693370 msgr2=0x55ddeb684ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:50.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.794+0000 7feea2000640 1 --2- 172.21.15.86:0/38999761 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ddeb693370 0x55ddeb684ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:50.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.794+0000 7feea2000640 1 -- 172.21.15.86:0/38999761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ddeb6824e0 msgr2=0x55ddeb6845a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:50.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.794+0000 7feea2000640 1 --2- 172.21.15.86:0/38999761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ddeb6824e0 0x55ddeb6845a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:50.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.794+0000 7feea2000640 1 -- 172.21.15.86:0/38999761 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddeb6ad380 con 0x55ddeb6cc460 2024-04-20T21:26:50.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.794+0000 7feea1600640 1 --2- 172.21.15.86:0/38999761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ddeb6824e0 0x55ddeb6845a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:26:50.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.795+0000 7feea2000640 1 --2- 172.21.15.86:0/38999761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddeb6cc460 0x55ddeb6d0e10 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fee94007130 tx=0x7fee94014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cb8a0f8c7be5217 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:50.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.795+0000 7fee91200640 1 -- 172.21.15.86:0/38999761 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee94018040 con 0x55ddeb6cc460 2024-04-20T21:26:50.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.795+0000 7fee91200640 1 -- 172.21.15.86:0/38999761 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee9400fb20 con 0x55ddeb6cc460 2024-04-20T21:26:50.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.795+0000 7fee91200640 1 -- 172.21.15.86:0/38999761 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee94003bd0 con 0x55ddeb6cc460 2024-04-20T21:26:50.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.796+0000 7feea6e09740 1 -- 172.21.15.86:0/38999761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddeb6cc460 msgr2=0x55ddeb6d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:50.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.796+0000 7feea6e09740 1 --2- 172.21.15.86:0/38999761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddeb6cc460 0x55ddeb6d0e10 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fee94007130 tx=0x7fee94014000 comp rx=0 tx=0).stop 2024-04-20T21:26:50.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.796+0000 7feea6e09740 1 -- 172.21.15.86:0/38999761 shutdown_connections 2024-04-20T21:26:50.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.796+0000 7feea6e09740 1 --2- 172.21.15.86:0/38999761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddeb6cc460 0x55ddeb6d0e10 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:50.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.796+0000 7feea6e09740 1 --2- 172.21.15.86:0/38999761 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ddeb693370 0x55ddeb684ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:50.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.796+0000 7feea6e09740 1 --2- 172.21.15.86:0/38999761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ddeb6824e0 0x55ddeb6845a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:50.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.796+0000 7feea6e09740 1 -- 172.21.15.86:0/38999761 >> 172.21.15.86:0/38999761 conn(0x55ddeb67e930 msgr2=0x55ddeb692c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:50.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.796+0000 7feea6e09740 1 -- 172.21.15.86:0/38999761 shutdown_connections 2024-04-20T21:26:50.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.796+0000 7feea6e09740 1 -- 172.21.15.86:0/38999761 wait complete. 2024-04-20T21:26:50.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.796+0000 7feea6e09740 1 -- start start 2024-04-20T21:26:50.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.797+0000 7feea6e09740 1 Processor -- start 2024-04-20T21:26:50.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.798+0000 7feea6e09740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ddeb6824e0 0x55ddeb746f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:50.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.798+0000 7feea6e09740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddeb693370 0x55ddeb6b2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:50.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.798+0000 7feea6e09740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ddeb6cc460 0x55ddeb750560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:50.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.798+0000 7feea6e09740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ddeb694c50 con 0x55ddeb693370 2024-04-20T21:26:50.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.798+0000 7feea6e09740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fee9c159690 con 0x55ddeb6824e0 2024-04-20T21:26:50.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.798+0000 7feea6e09740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55ddeb6a6d70 con 0x55ddeb6cc460 2024-04-20T21:26:50.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.798+0000 7feea6e09740 10 client.0 _ll_register_callbacks cb 0x7fee94009710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:26:50.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.798+0000 7feea0c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddeb693370 0x55ddeb6b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:50.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.798+0000 7feea0c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddeb693370 0x55ddeb6b2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55896/0 (socket says 172.21.15.86:55896) 2024-04-20T21:26:50.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.798+0000 7feea0c00640 1 -- 172.21.15.86:0/3145470086 learned_addr learned my addr 172.21.15.86:0/3145470086 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:50.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.798+0000 7feea1600640 1 --2- 172.21.15.86:0/3145470086 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ddeb6824e0 0x55ddeb746f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:50.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.798+0000 7feea2000640 1 --2- 172.21.15.86:0/3145470086 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ddeb6cc460 0x55ddeb750560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:50.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.799+0000 7feea0c00640 1 -- 172.21.15.86:0/3145470086 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ddeb6cc460 msgr2=0x55ddeb750560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:50.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.799+0000 7feea0c00640 1 --2- 172.21.15.86:0/3145470086 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ddeb6cc460 0x55ddeb750560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:50.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.799+0000 7feea0c00640 1 -- 172.21.15.86:0/3145470086 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ddeb6824e0 msgr2=0x55ddeb746f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:50.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.799+0000 7feea0c00640 1 --2- 172.21.15.86:0/3145470086 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ddeb6824e0 0x55ddeb746f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:50.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.799+0000 7feea0c00640 1 -- 172.21.15.86:0/3145470086 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddeb6b19d0 con 0x55ddeb693370 2024-04-20T21:26:50.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.799+0000 7feea0c00640 1 --2- 172.21.15.86:0/3145470086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddeb693370 0x55ddeb6b2170 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fee88009960 tx=0x7fee880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1478e6b148ffda76 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:50.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.799+0000 7fee86000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:26:50.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.799+0000 7fee86000640 1 -- 172.21.15.86:0/3145470086 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee88040070 con 0x55ddeb693370 2024-04-20T21:26:50.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.799+0000 7fee86000640 1 -- 172.21.15.86:0/3145470086 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7fee88050d90 con 0x55ddeb693370 2024-04-20T21:26:50.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.799+0000 7feea6e09740 1 -- 172.21.15.86:0/3145470086 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee9400fb20 con 0x55ddeb693370 2024-04-20T21:26:50.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.800+0000 7fee86000640 1 -- 172.21.15.86:0/3145470086 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7fee88035040 con 0x55ddeb693370 2024-04-20T21:26:50.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.800+0000 7feea6e09740 10 client.8181 fetch_fsmap learned FSMap version 73 2024-04-20T21:26:50.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.800+0000 7feea6e09740 1 -- 172.21.15.86:0/3145470086 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) -- 0x55ddeb6a3cd0 con 0x55ddeb693370 2024-04-20T21:26:50.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.800+0000 7fee86000640 1 -- 172.21.15.86:0/3145470086 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee9400fb20 con 0x55ddeb693370 2024-04-20T21:26:50.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.800+0000 7fee86000640 1 -- 172.21.15.86:0/3145470086 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 73) ==== 3495+0+0 (secure 0 0 0) 0x7fee8804f610 con 0x55ddeb693370 2024-04-20T21:26:50.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.800+0000 7feea6e09740 10 client.8181 fetch_fsmap finished waiting for FSMap version 73 2024-04-20T21:26:50.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.800+0000 7feea6e09740 10 client.8181 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-20T21:26:50.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.800+0000 7feea6e09740 20 client.8181 populate_metadata read hostname 'smithi086' 2024-04-20T21:26:50.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.800+0000 7feea6e09740 1 --2- 172.21.15.86:0/3145470086 >> [v2:172.21.15.144:6832/2351381466,v1:172.21.15.144:6834/2351381466] conn(0x55ddeb684a30 0x55ddeb67d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:50.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.800+0000 7feea6e09740 4 client.8181 mds_command: new command op to 7518 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T21:26:50.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.800+0000 7feea6e09740 1 -- 172.21.15.86:0/3145470086 --> [v2:172.21.15.144:6832/2351381466,v1:172.21.15.144:6834/2351381466] -- command(tid 0: {"prefix": "status"}) -- 0x55ddeb6cc2b0 con 0x55ddeb684a30 2024-04-20T21:26:50.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.801+0000 7feea1600640 1 --2- 172.21.15.86:0/3145470086 >> [v2:172.21.15.144:6832/2351381466,v1:172.21.15.144:6834/2351381466] conn(0x55ddeb684a30 0x55ddeb67d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:50.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.802+0000 7feea1600640 1 --2- 172.21.15.86:0/3145470086 >> [v2:172.21.15.144:6832/2351381466,v1:172.21.15.144:6834/2351381466] conn(0x55ddeb684a30 0x55ddeb67d850 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98b0f9d13300d33a server_cookie=3d91731a39f2979d in_seq=0 out_seq=0 2024-04-20T21:26:50.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.802+0000 7fee86000640 10 client.8181 ms_handle_connect on v2:172.21.15.144:6832/2351381466 2024-04-20T21:26:50.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.802+0000 7fee86000640 1 -- 172.21.15.86:0/3145470086 <== mds.0 v2:172.21.15.144:6832/2351381466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55ddeb6cc2b0 con 0x55ddeb684a30 2024-04-20T21:26:50.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.802+0000 7fee86000640 10 client.8181 handle_command_reply: tid=0 2024-04-20T21:26:50.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.802+0000 7feea6e09740 1 client.8181 shutdown 2024-04-20T21:26:50.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.803+0000 7feea6e09740 1 -- 172.21.15.86:0/3145470086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddeb693370 msgr2=0x55ddeb6b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:50.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.803+0000 7feea6e09740 1 --2- 172.21.15.86:0/3145470086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddeb693370 0x55ddeb6b2170 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fee88009960 tx=0x7fee880046c0 comp rx=0 tx=0).stop 2024-04-20T21:26:50.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.803+0000 7feea6e09740 1 -- 172.21.15.86:0/3145470086 shutdown_connections 2024-04-20T21:26:50.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.803+0000 7feea6e09740 1 --2- 172.21.15.86:0/3145470086 >> [v2:172.21.15.144:6832/2351381466,v1:172.21.15.144:6834/2351381466] conn(0x55ddeb684a30 0x55ddeb67d850 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:50.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.803+0000 7feea6e09740 1 --2- 172.21.15.86:0/3145470086 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ddeb6cc460 0x55ddeb750560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:50.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.803+0000 7feea6e09740 1 --2- 172.21.15.86:0/3145470086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ddeb693370 0x55ddeb6b2170 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:50.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.803+0000 7feea6e09740 1 --2- 172.21.15.86:0/3145470086 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ddeb6824e0 0x55ddeb746f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:50.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.803+0000 7feea6e09740 1 -- 172.21.15.86:0/3145470086 >> 172.21.15.86:0/3145470086 conn(0x55ddeb67e930 msgr2=0x55ddeb6aa720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:50.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.803+0000 7fee86000640 0 client.8181 ms_handle_reset on v2:172.21.15.144:6832/2351381466 2024-04-20T21:26:50.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.803+0000 7feea6e09740 1 -- 172.21.15.86:0/3145470086 shutdown_connections 2024-04-20T21:26:50.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.803+0000 7feea6e09740 1 -- 172.21.15.86:0/3145470086 wait complete. 2024-04-20T21:26:50.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.803+0000 7feea6e09740 20 client.8181 trim_cache size 0 max 16384 2024-04-20T21:26:50.806 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:26:50.806 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "9c2dec1d-d7ae-4a83-a812-79922808e278", 2024-04-20T21:26:50.806 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-04-20T21:26:50.806 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 7518, 2024-04-20T21:26:50.806 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-20T21:26:50.806 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-20T21:26:50.807 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-20T21:26:50.807 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 4.9154114829999997, 2024-04-20T21:26:50.807 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 73, 2024-04-20T21:26:50.807 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 89, 2024-04-20T21:26:50.807 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 89, 2024-04-20T21:26:50.807 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 117.409525855 2024-04-20T21:26:50.807 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:26:50.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.804+0000 7feea4000640 1 -- 172.21.15.86:0/661906177 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fee7c06c8f0 msgr2=0x7fee7c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:50.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.804+0000 7feea4000640 1 --2- 172.21.15.86:0/661906177 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fee7c06c8f0 0x7fee7c06edb0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fee88036000 tx=0x7fee88030000 comp rx=0 tx=0).stop 2024-04-20T21:26:50.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.804+0000 7feea4000640 1 -- 172.21.15.86:0/661906177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee9c09b580 msgr2=0x7fee9c0b5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:50.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.804+0000 7feea4000640 1 --2- 172.21.15.86:0/661906177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee9c09b580 0x7fee9c0b5e20 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fee8c00e7e0 tx=0x7fee8c00eca0 comp rx=0 tx=0).stop 2024-04-20T21:26:50.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.804+0000 7feea4000640 1 -- 172.21.15.86:0/661906177 shutdown_connections 2024-04-20T21:26:50.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.804+0000 7feea4000640 1 --2- 172.21.15.86:0/661906177 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fee7c06c8f0 0x7fee7c06edb0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:50.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.804+0000 7feea4000640 1 --2- 172.21.15.86:0/661906177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee9c15ce50 0x7fee9c0b4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:50.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.804+0000 7feea4000640 1 --2- 172.21.15.86:0/661906177 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee9c15ca70 0x7fee9c14fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:50.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.804+0000 7feea4000640 1 --2- 172.21.15.86:0/661906177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee9c09b580 0x7fee9c0b5e20 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:50.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.804+0000 7feea4000640 1 -- 172.21.15.86:0/661906177 >> 172.21.15.86:0/661906177 conn(0x7fee9c0b40b0 msgr2=0x7fee9c0cd280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:50.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.804+0000 7feea4000640 1 -- 172.21.15.86:0/661906177 shutdown_connections 2024-04-20T21:26:50.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:50.804+0000 7feea4000640 1 -- 172.21.15.86:0/661906177 wait complete. 2024-04-20T21:26:50.819 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T21:26:51.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.022+0000 7f3be2200640 1 Processor -- start 2024-04-20T21:26:51.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.022+0000 7f3be2200640 1 -- start start 2024-04-20T21:26:51.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.024+0000 7f3be2200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3bdc06fbd0 0x7f3bdc15c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:51.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.024+0000 7f3be2200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bdc0ae710 con 0x7f3bdc0af7b0 2024-04-20T21:26:51.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.024+0000 7f3be2200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bdc0ae9f0 con 0x7f3bdc15c010 2024-04-20T21:26:51.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.024+0000 7f3be2200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f3bdc0af320 con 0x7f3bdc06fbd0 2024-04-20T21:26:51.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.024+0000 7f3be1800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3bdc06fbd0 0x7f3bdc15c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:51.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.024+0000 7f3bd3e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f3bdc0af7b0 0x7f3bdc06f4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:36810/0 (socket says 172.21.15.86:36810) 2024-04-20T21:26:51.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.024+0000 7f3bd3e00640 1 -- 172.21.15.86:0/2458144295 learned_addr learned my addr 172.21.15.86:0/2458144295 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:26:51.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.025+0000 7f3be1800640 1 -- 172.21.15.86:0/2458144295 >> v1:172.21.15.144:6789/0 conn(0x7f3bdc15c010 legacy=0x7f3bdc0adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:51.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.025+0000 7f3be1800640 1 -- 172.21.15.86:0/2458144295 >> v1:172.21.15.86:6789/0 conn(0x7f3bdc0af7b0 legacy=0x7f3bdc06f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:51.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.025+0000 7f3be1800640 1 -- 172.21.15.86:0/2458144295 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bdc0aecd0 con 0x7f3bdc06fbd0 2024-04-20T21:26:51.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.026+0000 7f3be1800640 1 --2- 172.21.15.86:0/2458144295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bdc06fbd0 0x7f3bdc15c3f0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4009d90 tx=0x7f3bd402e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=370fb6c0751010ec server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:51.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.026+0000 7f3bdbe00640 1 -- 172.21.15.86:0/2458144295 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd403b070 con 0x7f3bdc06fbd0 2024-04-20T21:26:51.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.026+0000 7f3bdbe00640 1 -- 172.21.15.86:0/2458144295 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bd4036440 con 0x7f3bdc06fbd0 2024-04-20T21:26:51.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.026+0000 7f3bdbe00640 1 -- 172.21.15.86:0/2458144295 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd4036810 con 0x7f3bdc06fbd0 2024-04-20T21:26:51.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.026+0000 7f3be2200640 1 -- 172.21.15.86:0/2458144295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bdc06fbd0 msgr2=0x7f3bdc15c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:51.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.026+0000 7f3be2200640 1 --2- 172.21.15.86:0/2458144295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bdc06fbd0 0x7f3bdc15c3f0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4009d90 tx=0x7f3bd402e0c0 comp rx=0 tx=0).stop 2024-04-20T21:26:51.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.027+0000 7f3be2200640 1 -- 172.21.15.86:0/2458144295 shutdown_connections 2024-04-20T21:26:51.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.027+0000 7f3be2200640 1 --2- 172.21.15.86:0/2458144295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bdc06fbd0 0x7f3bdc15c3f0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:51.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.027+0000 7f3be2200640 1 -- 172.21.15.86:0/2458144295 >> 172.21.15.86:0/2458144295 conn(0x7f3bdc0d0170 msgr2=0x7f3bdc0d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:51.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.027+0000 7f3be2200640 1 -- 172.21.15.86:0/2458144295 shutdown_connections 2024-04-20T21:26:51.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.027+0000 7f3be2200640 1 -- 172.21.15.86:0/2458144295 wait complete. 2024-04-20T21:26:51.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.028+0000 7f3be2200640 1 Processor -- start 2024-04-20T21:26:51.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.029+0000 7f3be2200640 1 -- start start 2024-04-20T21:26:51.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.029+0000 7f3be2200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3bdc06fbd0 0x7f3bdc09db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:51.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.030+0000 7f3be2200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bdc0af7b0 0x7f3bdc1e5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:51.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.030+0000 7f3be2200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3bdc15c010 0x7f3bdc1452b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:51.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.030+0000 7f3be2200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3bdc15c930 con 0x7f3bdc0af7b0 2024-04-20T21:26:51.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.030+0000 7f3be2200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3bdc0ae4b0 con 0x7f3bdc15c010 2024-04-20T21:26:51.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.030+0000 7f3be2200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3bdc06e3b0 con 0x7f3bdc06fbd0 2024-04-20T21:26:51.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.030+0000 7f3bd3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bdc0af7b0 0x7f3bdc1e5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:51.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.030+0000 7f3bd3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bdc0af7b0 0x7f3bdc1e5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55920/0 (socket says 172.21.15.86:55920) 2024-04-20T21:26:51.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.030+0000 7f3bd3e00640 1 -- 172.21.15.86:0/3728290500 learned_addr learned my addr 172.21.15.86:0/3728290500 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:51.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.030+0000 7f3be0e00640 1 --2- 172.21.15.86:0/3728290500 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3bdc06fbd0 0x7f3bdc09db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:51.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.030+0000 7f3bd3e00640 1 -- 172.21.15.86:0/3728290500 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3bdc06fbd0 msgr2=0x7f3bdc09db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:51.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.030+0000 7f3be1800640 1 --2- 172.21.15.86:0/3728290500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3bdc15c010 0x7f3bdc1452b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:51.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.030+0000 7f3bd3e00640 1 --2- 172.21.15.86:0/3728290500 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3bdc06fbd0 0x7f3bdc09db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:51.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.030+0000 7f3bd3e00640 1 -- 172.21.15.86:0/3728290500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3bdc15c010 msgr2=0x7f3bdc1452b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:51.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.030+0000 7f3bd3e00640 1 --2- 172.21.15.86:0/3728290500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3bdc15c010 0x7f3bdc1452b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:51.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.030+0000 7f3bd3e00640 1 -- 172.21.15.86:0/3728290500 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bdc148340 con 0x7f3bdc0af7b0 2024-04-20T21:26:51.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.030+0000 7f3be0e00640 1 --2- 172.21.15.86:0/3728290500 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3bdc06fbd0 0x7f3bdc09db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:26:51.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.031+0000 7f3bd3e00640 1 --2- 172.21.15.86:0/3728290500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bdc0af7b0 0x7f3bdc1e5fe0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f3bc400c400 tx=0x7f3bc400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:51.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.031+0000 7f3bd9600640 1 -- 172.21.15.86:0/3728290500 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc4017070 con 0x7f3bdc0af7b0 2024-04-20T21:26:51.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.031+0000 7f3bd9600640 1 -- 172.21.15.86:0/3728290500 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bc4005ce0 con 0x7f3bdc0af7b0 2024-04-20T21:26:51.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.031+0000 7f3be2200640 1 -- 172.21.15.86:0/3728290500 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bdc148620 con 0x7f3bdc0af7b0 2024-04-20T21:26:51.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.031+0000 7f3bd9600640 1 -- 172.21.15.86:0/3728290500 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc40072f0 con 0x7f3bdc0af7b0 2024-04-20T21:26:51.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.031+0000 7f3be2200640 1 -- 172.21.15.86:0/3728290500 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bdc148be0 con 0x7f3bdc0af7b0 2024-04-20T21:26:51.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.032+0000 7f3be2200640 1 -- 172.21.15.86:0/3728290500 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bdc1490d0 con 0x7f3bdc0af7b0 2024-04-20T21:26:51.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.033+0000 7f3bd9600640 1 -- 172.21.15.86:0/3728290500 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3bc401a070 con 0x7f3bdc0af7b0 2024-04-20T21:26:51.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.034+0000 7f3bd9600640 1 --2- 172.21.15.86:0/3728290500 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3bb806c8f0 0x7f3bb806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:51.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.034+0000 7f3bd9600640 1 -- 172.21.15.86:0/3728290500 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7f3bc408b0d0 con 0x7f3bdc0af7b0 2024-04-20T21:26:51.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.034+0000 7f3be0e00640 1 --2- 172.21.15.86:0/3728290500 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3bb806c8f0 0x7f3bb806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:51.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.035+0000 7f3be0e00640 1 --2- 172.21.15.86:0/3728290500 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3bb806c8f0 0x7f3bb806edb0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc00a830 tx=0x7f3bcc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:51.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.038+0000 7f3bd9600640 1 -- 172.21.15.86:0/3728290500 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3bc4059390 con 0x7f3bdc0af7b0 2024-04-20T21:26:51.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.283+0000 7f3be2200640 1 -- 172.21.15.86:0/3728290500 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3bdc15b260 con 0x7f3bdc0af7b0 2024-04-20T21:26:51.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.284+0000 7f3bd9600640 1 -- 172.21.15.86:0/3728290500 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v73) ==== 53+0+73 (secure 0 0 0) 0x7f3bc4058c90 con 0x7f3bdc0af7b0 2024-04-20T21:26:51.283 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T21:26:51.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.287+0000 7f3be2200640 1 -- 172.21.15.86:0/3728290500 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3bb806c8f0 msgr2=0x7f3bb806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:51.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.287+0000 7f3be2200640 1 --2- 172.21.15.86:0/3728290500 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3bb806c8f0 0x7f3bb806edb0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc00a830 tx=0x7f3bcc008040 comp rx=0 tx=0).stop 2024-04-20T21:26:51.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.287+0000 7f3be2200640 1 -- 172.21.15.86:0/3728290500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bdc0af7b0 msgr2=0x7f3bdc1e5fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:51.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.287+0000 7f3be2200640 1 --2- 172.21.15.86:0/3728290500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bdc0af7b0 0x7f3bdc1e5fe0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f3bc400c400 tx=0x7f3bc400c8c0 comp rx=0 tx=0).stop 2024-04-20T21:26:51.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.287+0000 7f3be2200640 1 -- 172.21.15.86:0/3728290500 shutdown_connections 2024-04-20T21:26:51.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.287+0000 7f3be2200640 1 --2- 172.21.15.86:0/3728290500 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3bb806c8f0 0x7f3bb806edb0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:51.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.287+0000 7f3be2200640 1 --2- 172.21.15.86:0/3728290500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3bdc15c010 0x7f3bdc1452b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:51.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.287+0000 7f3be2200640 1 --2- 172.21.15.86:0/3728290500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bdc0af7b0 0x7f3bdc1e5fe0 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:51.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.287+0000 7f3be2200640 1 --2- 172.21.15.86:0/3728290500 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3bdc06fbd0 0x7f3bdc09db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:51.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.287+0000 7f3be2200640 1 -- 172.21.15.86:0/3728290500 >> 172.21.15.86:0/3728290500 conn(0x7f3bdc0d0170 msgr2=0x7f3bdc15e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:51.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.288+0000 7f3be2200640 1 -- 172.21.15.86:0/3728290500 shutdown_connections 2024-04-20T21:26:51.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51.288+0000 7f3be2200640 1 -- 172.21.15.86:0/3728290500 wait complete. 2024-04-20T21:26:51.305 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T21:26:51.305 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T21:26:51.305 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-04-20T21:26:51.305 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-20T21:26:51.305 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T21:26:51.305 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T21:26:51.305 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T21:26:51.305 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T21:26:51.305 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:26:51.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:26:51.306 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-04-20T21:26:51.359 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T21:26:51.359 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T21:26:51.359 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T21:26:51.359 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:26:51.359 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-20T21:26:51.359 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:26:51.359 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:26:51.359 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:26:51.359 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-04-20T21:26:51.359 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:26:51.360 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:26:51.360 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-04-20T21:26:51.360 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:26:51.360 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:26:51.360 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T21:26:51.360 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:26:51.360 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T21:26:51.360 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42145sec preferred_lft 42145sec 2024-04-20T21:26:51.360 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-04-20T21:26:51.360 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:26:51.360 INFO:teuthology.orchestra.run.smithi086.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T21:26:51.360 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 86:25:97:2c:9c:bb brd ff:ff:ff:ff:ff:ff 2024-04-20T21:26:51.360 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-20T21:26:51.360 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:26:51.360 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::a4fb:30ff:fe90:2d6a/64 scope link 2024-04-20T21:26:51.361 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:26:51.361 INFO:teuthology.orchestra.run.smithi086.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T21:26:51.361 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 86:25:97:2c:9c:bb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:26:51.361 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::8425:97ff:fe2c:9cbb/64 scope link 2024-04-20T21:26:51.361 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:26:51.361 INFO:teuthology.orchestra.run.smithi086.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T21:26:51.361 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c2:9c:8b:55:ca:84 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:26:51.361 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c09c:8bff:fe55:ca84/64 scope link 2024-04-20T21:26:51.361 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:26:51.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:26:51.362 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T21:26:51.441 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-20T21:26:51.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:26:51.445 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-04-20T21:26:51.499 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-04-20T21:26:51.499 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T21:26:51.499 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T21:26:51.499 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T21:26:51.499 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T21:26:51.500 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:26:51.500 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-20T21:26:51.500 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T21:26:51.500 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T21:26:51.500 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T21:26:51.500 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-20T21:26:51.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:26:51.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:26:51.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:26:51.712 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-20T21:26:51.765 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-20T21:26:51.765 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-20T21:26:51.766 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:26:51.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:26:51.766 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-04-20T21:26:51.849 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-20T21:26:51.849 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T21:26:51.849 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-20T21:26:51.903 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:26:51.904 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T21:26:51.904 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-20T21:26:51.955 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-04-20T21:26:52.032 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T21:26:52.083 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-04-20T21:26:52.083 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-04-20T21:26:52.083 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-04-20T21:26:52.083 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-04-20T21:26:52.083 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-04-20T21:26:52.083 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-04-20T21:26:52.083 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-04-20T21:26:52.083 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-04-20T21:26:52.083 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-04-20T21:26:52.083 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-04-20T21:26:52.083 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-04-20T21:26:52.084 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/selinux 2024-04-20T21:26:52.084 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T21:26:52.084 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-04-20T21:26:52.084 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-04-20T21:26:52.084 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-04-20T21:26:52.084 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-04-20T21:26:52.084 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-04-20T21:26:52.084 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-04-20T21:26:52.084 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T21:26:52.084 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T21:26:52.084 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T21:26:52.084 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T21:26:52.084 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-04-20T21:26:52.084 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-04-20T21:26:52.084 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T21:26:52.084 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T21:26:52.085 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T21:26:52.085 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T21:26:52.085 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-04-20T21:26:52.085 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:26:52.085 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:26:52.085 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:26:52.085 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:26:52.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:26:52.086 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-04-20T21:26:52.137 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T21:26:52.137 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-20T21:26:52.142 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-04-20T21:26:52.209 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T21:26:52.249 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-04-20T21:26:52.249 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-04-20T21:26:52.249 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-04-20T21:26:52.249 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-04-20T21:26:52.249 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-04-20T21:26:52.249 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-04-20T21:26:52.249 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-04-20T21:26:52.249 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-04-20T21:26:52.250 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-04-20T21:26:52.250 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-04-20T21:26:52.250 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-04-20T21:26:52.250 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/selinux 2024-04-20T21:26:52.250 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T21:26:52.250 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-04-20T21:26:52.250 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-04-20T21:26:52.250 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-04-20T21:26:52.250 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-04-20T21:26:52.250 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-04-20T21:26:52.250 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-04-20T21:26:52.250 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T21:26:52.250 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T21:26:52.250 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T21:26:52.251 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T21:26:52.251 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-04-20T21:26:52.251 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-04-20T21:26:52.251 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T21:26:52.251 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T21:26:52.251 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T21:26:52.251 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T21:26:52.251 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-04-20T21:26:52.251 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:26:52.251 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:26:52.251 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:26:52.251 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:26:52.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:26:52.252 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-04-20T21:26:52.309 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:2024-04-20T21:26:52.311+0000 7f21e0b87480 -1 init, newargv = 0x55a4132f34a0 newargc=15 2024-04-20T21:26:52.310 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[56538]: starting ceph client 2024-04-20T21:26:52.321 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[56538]: starting fuse 2024-04-20T21:26:52.334 INFO:teuthology.orchestra.run.smithi086.stdout:41 2024-04-20T21:26:52.334 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-04-20T21:26:52.334 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer -- python3 -c ' 2024-04-20T21:26:52.334 DEBUG:teuthology.orchestra.run.smithi086:> import glob 2024-04-20T21:26:52.334 DEBUG:teuthology.orchestra.run.smithi086:> import re 2024-04-20T21:26:52.334 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:26:52.334 DEBUG:teuthology.orchestra.run.smithi086:> import subprocess 2024-04-20T21:26:52.334 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:26:52.334 DEBUG:teuthology.orchestra.run.smithi086:> def _find_admin_socket(client_name): 2024-04-20T21:26:52.334 DEBUG:teuthology.orchestra.run.smithi086:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-20T21:26:52.335 DEBUG:teuthology.orchestra.run.smithi086:> files = glob.glob(asok_path) 2024-04-20T21:26:52.335 DEBUG:teuthology.orchestra.run.smithi086:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-20T21:26:52.335 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:26:52.335 DEBUG:teuthology.orchestra.run.smithi086:> # Given a non-glob path, it better be there 2024-04-20T21:26:52.335 DEBUG:teuthology.orchestra.run.smithi086:> if "*" not in asok_path: 2024-04-20T21:26:52.335 DEBUG:teuthology.orchestra.run.smithi086:> assert(len(files) == 1) 2024-04-20T21:26:52.335 DEBUG:teuthology.orchestra.run.smithi086:> return files[0] 2024-04-20T21:26:52.335 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:26:52.335 DEBUG:teuthology.orchestra.run.smithi086:> for f in files: 2024-04-20T21:26:52.335 DEBUG:teuthology.orchestra.run.smithi086:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T21:26:52.335 DEBUG:teuthology.orchestra.run.smithi086:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T21:26:52.335 DEBUG:teuthology.orchestra.run.smithi086:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T21:26:52.335 DEBUG:teuthology.orchestra.run.smithi086:> contents = proc_f.read() 2024-04-20T21:26:52.335 DEBUG:teuthology.orchestra.run.smithi086:> if mountpoint in contents: 2024-04-20T21:26:52.335 DEBUG:teuthology.orchestra.run.smithi086:> return f 2024-04-20T21:26:52.335 DEBUG:teuthology.orchestra.run.smithi086:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T21:26:52.335 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:26:52.335 DEBUG:teuthology.orchestra.run.smithi086:> print(_find_admin_socket("client.0")) 2024-04-20T21:26:52.335 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:26:52.524 INFO:teuthology.orchestra.run.smithi086.stdout:/var/run/ceph/ceph-client.0.56538.asok 2024-04-20T21:26:52.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:26:52.540 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.56538.asok 2024-04-20T21:26:52.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:26:52.540 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.56538.asok status 2024-04-20T21:26:52.764 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:26:52.764 INFO:teuthology.orchestra.run.smithi086.stdout: "metadata": { 2024-04-20T21:26:52.764 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T21:26:52.764 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T21:26:52.764 INFO:teuthology.orchestra.run.smithi086.stdout: "entity_id": "0", 2024-04-20T21:26:52.765 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": "smithi086", 2024-04-20T21:26:52.765 INFO:teuthology.orchestra.run.smithi086.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-20T21:26:52.765 INFO:teuthology.orchestra.run.smithi086.stdout: "pid": "56538", 2024-04-20T21:26:52.765 INFO:teuthology.orchestra.run.smithi086.stdout: "root": "/" 2024-04-20T21:26:52.765 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:26:52.765 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_count": 0, 2024-04-20T21:26:52.765 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_pinned_count": 0, 2024-04-20T21:26:52.765 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 8196, 2024-04-20T21:26:52.765 INFO:teuthology.orchestra.run.smithi086.stdout: "inst": { 2024-04-20T21:26:52.765 INFO:teuthology.orchestra.run.smithi086.stdout: "name": { 2024-04-20T21:26:52.765 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "client", 2024-04-20T21:26:52.765 INFO:teuthology.orchestra.run.smithi086.stdout: "num": 8196 2024-04-20T21:26:52.765 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:26:52.766 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2024-04-20T21:26:52.766 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-04-20T21:26:52.766 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2024-04-20T21:26:52.766 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 3786554125 2024-04-20T21:26:52.766 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:26:52.766 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:26:52.766 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2024-04-20T21:26:52.766 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-04-20T21:26:52.766 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2024-04-20T21:26:52.766 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 3786554125 2024-04-20T21:26:52.766 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:26:52.766 INFO:teuthology.orchestra.run.smithi086.stdout: "inst_str": "client.8196 192.168.0.1:0/3786554125", 2024-04-20T21:26:52.766 INFO:teuthology.orchestra.run.smithi086.stdout: "addr_str": "192.168.0.1:0/3786554125", 2024-04-20T21:26:52.766 INFO:teuthology.orchestra.run.smithi086.stdout: "inode_count": 1, 2024-04-20T21:26:52.767 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_epoch": 73, 2024-04-20T21:26:52.767 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch": 89, 2024-04-20T21:26:52.767 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch_barrier": 0, 2024-04-20T21:26:52.767 INFO:teuthology.orchestra.run.smithi086.stdout: "blocklisted": false, 2024-04-20T21:26:52.767 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs" 2024-04-20T21:26:52.767 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:26:52.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:26:52.779 DEBUG:teuthology.orchestra.run.smithi086:> stat --file-system '--printf=%T 2024-04-20T21:26:52.779 DEBUG:teuthology.orchestra.run.smithi086:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-20T21:26:52.831 INFO:teuthology.orchestra.run.smithi086.stdout:fuseblk 2024-04-20T21:26:52.831 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-20T21:26:52.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:26:52.832 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-20T21:26:52.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:26:52.912 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-04-20T21:26:53.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.118+0000 7f0756000640 1 Processor -- start 2024-04-20T21:26:53.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.118+0000 7f0756000640 1 -- start start 2024-04-20T21:26:53.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.120+0000 7f0756000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0750071d00 0x7f075006f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:53.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.120+0000 7f0756000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07500720e0 0x7f0750145a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:53.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.120+0000 7f0756000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f075006e010 con 0x7f0750145fd0 2024-04-20T21:26:53.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.120+0000 7f0756000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f075006f5f0 con 0x7f0750071d00 2024-04-20T21:26:53.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.120+0000 7f0756000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f075006e2f0 con 0x7f07500720e0 2024-04-20T21:26:53.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.120+0000 7f074f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0750071d00 0x7f075006f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:53.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.120+0000 7f074fe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f0750145fd0 0x7f075015cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:36826/0 (socket says 172.21.15.86:36826) 2024-04-20T21:26:53.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.120+0000 7f074fe00640 1 -- 172.21.15.86:0/2136062090 learned_addr learned my addr 172.21.15.86:0/2136062090 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:26:53.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.120+0000 7f0747e00640 1 --2- 172.21.15.86:0/2136062090 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07500720e0 0x7f0750145a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:53.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.121+0000 7f074f400640 1 -- 172.21.15.86:0/2136062090 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07500720e0 msgr2=0x7f0750145a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:53.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.121+0000 7f074f400640 1 --2- 172.21.15.86:0/2136062090 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07500720e0 0x7f0750145a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.121+0000 7f074f400640 1 -- 172.21.15.86:0/2136062090 >> v1:172.21.15.86:6789/0 conn(0x7f0750145fd0 legacy=0x7f075015cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:53.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.121+0000 7f074f400640 1 -- 172.21.15.86:0/2136062090 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f075006e5d0 con 0x7f0750071d00 2024-04-20T21:26:53.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.122+0000 7f074f400640 1 --2- 172.21.15.86:0/2136062090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0750071d00 0x7f075006f0b0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f07400097a0 tx=0x7f074002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb33f06a51349b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:53.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.123+0000 7f074ea00640 1 -- 172.21.15.86:0/2136062090 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0740035020 con 0x7f0750071d00 2024-04-20T21:26:53.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.123+0000 7f074ea00640 1 -- 172.21.15.86:0/2136062090 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0740031830 con 0x7f0750071d00 2024-04-20T21:26:53.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.123+0000 7f074ea00640 1 -- 172.21.15.86:0/2136062090 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0740031c00 con 0x7f0750071d00 2024-04-20T21:26:53.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.123+0000 7f0756000640 1 -- 172.21.15.86:0/2136062090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0750071d00 msgr2=0x7f075006f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:53.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.123+0000 7f0756000640 1 --2- 172.21.15.86:0/2136062090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0750071d00 0x7f075006f0b0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f07400097a0 tx=0x7f074002b8d0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.123+0000 7f0756000640 1 -- 172.21.15.86:0/2136062090 shutdown_connections 2024-04-20T21:26:53.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.123+0000 7f0756000640 1 --2- 172.21.15.86:0/2136062090 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f07500720e0 0x7f0750145a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.123+0000 7f0756000640 1 --2- 172.21.15.86:0/2136062090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0750071d00 0x7f075006f0b0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.123+0000 7f0756000640 1 -- 172.21.15.86:0/2136062090 >> 172.21.15.86:0/2136062090 conn(0x7f07500b2180 msgr2=0x7f07500b2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:53.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.124+0000 7f0756000640 1 -- 172.21.15.86:0/2136062090 shutdown_connections 2024-04-20T21:26:53.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.124+0000 7f0756000640 1 -- 172.21.15.86:0/2136062090 wait complete. 2024-04-20T21:26:53.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.126+0000 7f0756000640 1 Processor -- start 2024-04-20T21:26:53.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.126+0000 7f0756000640 1 -- start start 2024-04-20T21:26:53.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.127+0000 7f0756000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0750071d00 0x7f0750142020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:53.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.127+0000 7f0756000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07500720e0 0x7f075013acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:53.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.127+0000 7f0756000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0750145fd0 0x7f075013fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:53.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.127+0000 7f0756000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0750072540 con 0x7f07500720e0 2024-04-20T21:26:53.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.127+0000 7f0756000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f075006eb90 con 0x7f0750071d00 2024-04-20T21:26:53.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.127+0000 7f0756000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f075006dc30 con 0x7f0750145fd0 2024-04-20T21:26:53.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.128+0000 7f0747e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07500720e0 0x7f075013acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:53.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.128+0000 7f0747e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07500720e0 0x7f075013acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55936/0 (socket says 172.21.15.86:55936) 2024-04-20T21:26:53.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.128+0000 7f0747e00640 1 -- 172.21.15.86:0/3073881304 learned_addr learned my addr 172.21.15.86:0/3073881304 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:53.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.128+0000 7f074fe00640 1 --2- 172.21.15.86:0/3073881304 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0750145fd0 0x7f075013fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:53.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.128+0000 7f074f400640 1 --2- 172.21.15.86:0/3073881304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0750071d00 0x7f0750142020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:53.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.128+0000 7f0747e00640 1 -- 172.21.15.86:0/3073881304 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0750145fd0 msgr2=0x7f075013fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:53.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.128+0000 7f0747e00640 1 --2- 172.21.15.86:0/3073881304 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0750145fd0 0x7f075013fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.128+0000 7f0747e00640 1 -- 172.21.15.86:0/3073881304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0750071d00 msgr2=0x7f0750142020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:53.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.128+0000 7f0747e00640 1 --2- 172.21.15.86:0/3073881304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0750071d00 0x7f0750142020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.128+0000 7f0747e00640 1 -- 172.21.15.86:0/3073881304 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0750144e50 con 0x7f07500720e0 2024-04-20T21:26:53.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.129+0000 7f0747e00640 1 --2- 172.21.15.86:0/3073881304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07500720e0 0x7f075013acd0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f073800d7d0 tx=0x7f073800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:53.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.129+0000 7f0747400640 1 -- 172.21.15.86:0/3073881304 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0738018020 con 0x7f07500720e0 2024-04-20T21:26:53.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.129+0000 7f0747400640 1 -- 172.21.15.86:0/3073881304 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07380096c0 con 0x7f07500720e0 2024-04-20T21:26:53.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.129+0000 7f0747400640 1 -- 172.21.15.86:0/3073881304 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0738009ab0 con 0x7f07500720e0 2024-04-20T21:26:53.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.129+0000 7f0756000640 1 -- 172.21.15.86:0/3073881304 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0750142560 con 0x7f07500720e0 2024-04-20T21:26:53.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.129+0000 7f0756000640 1 -- 172.21.15.86:0/3073881304 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07500b3690 con 0x7f07500720e0 2024-04-20T21:26:53.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.131+0000 7f0747400640 1 -- 172.21.15.86:0/3073881304 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f073801b020 con 0x7f07500720e0 2024-04-20T21:26:53.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.132+0000 7f0747400640 1 --2- 172.21.15.86:0/3073881304 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f072c06c8f0 0x7f072c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:53.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.132+0000 7f0747400640 1 -- 172.21.15.86:0/3073881304 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7f073808c2c0 con 0x7f07500720e0 2024-04-20T21:26:53.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.133+0000 7f074f400640 1 --2- 172.21.15.86:0/3073881304 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f072c06c8f0 0x7f072c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:53.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.133+0000 7f0758df2740 1 Processor -- start 2024-04-20T21:26:53.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.134+0000 7f074f400640 1 --2- 172.21.15.86:0/3073881304 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f072c06c8f0 0x7f072c06edb0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f0740036000 tx=0x7f0740008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:53.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.134+0000 7f0758df2740 1 -- start start 2024-04-20T21:26:53.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.134+0000 7f0758df2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ced3f09fe0 0x55ced3edcf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:53.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.134+0000 7f0758df2740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ced3f0a3c0 0x55ced3f0cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:53.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.134+0000 7f0758df2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ced3edd4c0 0x55ced3f0d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:53.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.134+0000 7f0758df2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ced3e47d20 con 0x55ced3f09fe0 2024-04-20T21:26:53.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.134+0000 7f0758df2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ced3dee3e0 con 0x55ced3edd4c0 2024-04-20T21:26:53.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.134+0000 7f0758df2740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55ced3e43ce0 con 0x55ced3f0a3c0 2024-04-20T21:26:53.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.135+0000 7f074f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ced3f09fe0 0x55ced3edcf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:53.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.135+0000 7f074f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ced3f09fe0 0x55ced3edcf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55952/0 (socket says 172.21.15.86:55952) 2024-04-20T21:26:53.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.135+0000 7f074f400640 1 -- 172.21.15.86:0/2687159327 learned_addr learned my addr 172.21.15.86:0/2687159327 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:53.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.135+0000 7f074f400640 1 -- 172.21.15.86:0/2687159327 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ced3f0a3c0 msgr2=0x55ced3f0cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:53.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.135+0000 7f074f400640 1 --2- 172.21.15.86:0/2687159327 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ced3f0a3c0 0x55ced3f0cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.135+0000 7f074f400640 1 -- 172.21.15.86:0/2687159327 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ced3edd4c0 msgr2=0x55ced3f0d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:53.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.135+0000 7f074fe00640 1 --2- 172.21.15.86:0/2687159327 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ced3edd4c0 0x55ced3f0d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:53.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.135+0000 7f074f400640 1 --2- 172.21.15.86:0/2687159327 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ced3edd4c0 0x55ced3f0d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.135+0000 7f074f400640 1 -- 172.21.15.86:0/2687159327 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ced3e0d980 con 0x55ced3f09fe0 2024-04-20T21:26:53.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.136+0000 7f074f400640 1 --2- 172.21.15.86:0/2687159327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ced3f09fe0 0x55ced3edcf80 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f0740034040 tx=0x7f0740030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15c7a693f5f43ac6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:53.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.136+0000 7f073f400640 1 -- 172.21.15.86:0/2687159327 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074004a070 con 0x55ced3f09fe0 2024-04-20T21:26:53.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.136+0000 7f073f400640 1 -- 172.21.15.86:0/2687159327 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f074003ab70 con 0x55ced3f09fe0 2024-04-20T21:26:53.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.136+0000 7f073f400640 1 -- 172.21.15.86:0/2687159327 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0740035040 con 0x55ced3f09fe0 2024-04-20T21:26:53.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.136+0000 7f0758df2740 1 -- 172.21.15.86:0/2687159327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ced3f09fe0 msgr2=0x55ced3edcf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:53.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.136+0000 7f0758df2740 1 --2- 172.21.15.86:0/2687159327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ced3f09fe0 0x55ced3edcf80 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f0740034040 tx=0x7f0740030000 comp rx=0 tx=0).stop 2024-04-20T21:26:53.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.137+0000 7f0758df2740 1 -- 172.21.15.86:0/2687159327 shutdown_connections 2024-04-20T21:26:53.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.137+0000 7f0758df2740 1 --2- 172.21.15.86:0/2687159327 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ced3edd4c0 0x55ced3f0d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.137+0000 7f0758df2740 1 --2- 172.21.15.86:0/2687159327 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ced3f0a3c0 0x55ced3f0cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.137+0000 7f0758df2740 1 --2- 172.21.15.86:0/2687159327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ced3f09fe0 0x55ced3edcf80 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.137+0000 7f0758df2740 1 -- 172.21.15.86:0/2687159327 >> 172.21.15.86:0/2687159327 conn(0x55ced3edf1f0 msgr2=0x55ced3f27d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:53.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.137+0000 7f0758df2740 1 -- 172.21.15.86:0/2687159327 shutdown_connections 2024-04-20T21:26:53.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.137+0000 7f0758df2740 1 -- 172.21.15.86:0/2687159327 wait complete. 2024-04-20T21:26:53.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.137+0000 7f0758df2740 1 -- start start 2024-04-20T21:26:53.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.138+0000 7f0758df2740 1 Processor -- start 2024-04-20T21:26:53.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.139+0000 7f0758df2740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ced3f09fe0 0x55ced3f05bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:53.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.139+0000 7f0758df2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ced3f0a3c0 0x55ced3f0ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:53.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.139+0000 7f0758df2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ced3f03770 0x55ced3f0bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:53.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.139+0000 7f0758df2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ced3ee1890 con 0x55ced3f03770 2024-04-20T21:26:53.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.139+0000 7f0758df2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ced3e017a0 con 0x55ced3f0a3c0 2024-04-20T21:26:53.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.139+0000 7f0758df2740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55ced3ee40d0 con 0x55ced3f09fe0 2024-04-20T21:26:53.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.139+0000 7f0758df2740 10 client.0 _ll_register_callbacks cb 0x55ced3e03b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:26:53.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.139+0000 7f074fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ced3f03770 0x55ced3f0bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:53.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.139+0000 7f074fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ced3f03770 0x55ced3f0bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55956/0 (socket says 172.21.15.86:55956) 2024-04-20T21:26:53.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.139+0000 7f074fe00640 1 -- 172.21.15.86:0/3293363146 learned_addr learned my addr 172.21.15.86:0/3293363146 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:53.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.139+0000 7f074fe00640 1 -- 172.21.15.86:0/3293363146 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ced3f09fe0 msgr2=0x55ced3f05bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:53.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.139+0000 7f074fe00640 1 --2- 172.21.15.86:0/3293363146 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ced3f09fe0 0x55ced3f05bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.139+0000 7f074fe00640 1 -- 172.21.15.86:0/3293363146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ced3f0a3c0 msgr2=0x55ced3f0ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:53.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.139+0000 7f0747e00640 1 --2- 172.21.15.86:0/3293363146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ced3f0a3c0 0x55ced3f0ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:53.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.139+0000 7f074fe00640 1 --2- 172.21.15.86:0/3293363146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ced3f0a3c0 0x55ced3f0ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.139+0000 7f074fe00640 1 -- 172.21.15.86:0/3293363146 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ced3c227e0 con 0x55ced3f03770 2024-04-20T21:26:53.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.140+0000 7f074fe00640 1 --2- 172.21.15.86:0/3293363146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ced3f03770 0x55ced3f0bf60 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f074800f820 tx=0x7f074800fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dce884a2d7193334 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:53.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.140+0000 7f0733e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:26:53.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.140+0000 7f0733e00640 1 -- 172.21.15.86:0/3293363146 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0748017020 con 0x55ced3f03770 2024-04-20T21:26:53.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.140+0000 7f0733e00640 1 -- 172.21.15.86:0/3293363146 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7f074800d8e0 con 0x55ced3f03770 2024-04-20T21:26:53.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.140+0000 7f0758df2740 1 -- 172.21.15.86:0/3293363146 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f074003ab70 con 0x55ced3f03770 2024-04-20T21:26:53.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.140+0000 7f0733e00640 1 -- 172.21.15.86:0/3293363146 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f0748021050 con 0x55ced3f03770 2024-04-20T21:26:53.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.140+0000 7f0758df2740 10 client.8211 fetch_fsmap learned FSMap version 73 2024-04-20T21:26:53.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.140+0000 7f0758df2740 1 -- 172.21.15.86:0/3293363146 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) -- 0x55ced3c82f70 con 0x55ced3f03770 2024-04-20T21:26:53.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.141+0000 7f0733e00640 1 -- 172.21.15.86:0/3293363146 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f074003ab70 con 0x55ced3f03770 2024-04-20T21:26:53.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.141+0000 7f0733e00640 1 -- 172.21.15.86:0/3293363146 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 73) ==== 3495+0+0 (secure 0 0 0) 0x7f0748024020 con 0x55ced3f03770 2024-04-20T21:26:53.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.141+0000 7f0758df2740 10 client.8211 fetch_fsmap finished waiting for FSMap version 73 2024-04-20T21:26:53.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.141+0000 7f0758df2740 10 client.8211 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-04-20T21:26:53.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.141+0000 7f0758df2740 20 client.8211 populate_metadata read hostname 'smithi086' 2024-04-20T21:26:53.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.141+0000 7f0758df2740 1 --2- 172.21.15.86:0/3293363146 >> [v2:172.21.15.144:6832/2351381466,v1:172.21.15.144:6834/2351381466] conn(0x55ced3f0ff60 0x55ced3f10370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:53.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.141+0000 7f0758df2740 4 client.8211 mds_command: new command op to 7518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:26:53.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.141+0000 7f0758df2740 1 -- 172.21.15.86:0/3293363146 --> [v2:172.21.15.144:6832/2351381466,v1:172.21.15.144:6834/2351381466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ced3e55740 con 0x55ced3f0ff60 2024-04-20T21:26:53.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.142+0000 7f074f400640 1 --2- 172.21.15.86:0/3293363146 >> [v2:172.21.15.144:6832/2351381466,v1:172.21.15.144:6834/2351381466] conn(0x55ced3f0ff60 0x55ced3f10370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:53.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.143+0000 7f074f400640 1 --2- 172.21.15.86:0/3293363146 >> [v2:172.21.15.144:6832/2351381466,v1:172.21.15.144:6834/2351381466] conn(0x55ced3f0ff60 0x55ced3f10370 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ec477a80e0641d5 server_cookie=abb84bd37215e2f8 in_seq=0 out_seq=0 2024-04-20T21:26:53.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.143+0000 7f0733e00640 10 client.8211 ms_handle_connect on v2:172.21.15.144:6832/2351381466 2024-04-20T21:26:53.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.145+0000 7f0733e00640 1 -- 172.21.15.86:0/3293363146 <== mds.0 v2:172.21.15.144:6832/2351381466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ced3e55740 con 0x55ced3f0ff60 2024-04-20T21:26:53.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.145+0000 7f0733e00640 10 client.8211 handle_command_reply: tid=0 2024-04-20T21:26:53.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.145+0000 7f0758df2740 1 client.8211 shutdown 2024-04-20T21:26:53.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.145+0000 7f0758df2740 1 -- 172.21.15.86:0/3293363146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ced3f03770 msgr2=0x55ced3f0bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:53.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.145+0000 7f0758df2740 1 --2- 172.21.15.86:0/3293363146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ced3f03770 0x55ced3f0bf60 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f074800f820 tx=0x7f074800fce0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.145+0000 7f0758df2740 1 -- 172.21.15.86:0/3293363146 shutdown_connections 2024-04-20T21:26:53.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.145+0000 7f0758df2740 1 --2- 172.21.15.86:0/3293363146 >> [v2:172.21.15.144:6832/2351381466,v1:172.21.15.144:6834/2351381466] conn(0x55ced3f0ff60 0x55ced3f10370 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.145+0000 7f0758df2740 1 --2- 172.21.15.86:0/3293363146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ced3f03770 0x55ced3f0bf60 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.145+0000 7f0758df2740 1 --2- 172.21.15.86:0/3293363146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ced3f0a3c0 0x55ced3f0ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.145+0000 7f0758df2740 1 --2- 172.21.15.86:0/3293363146 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ced3f09fe0 0x55ced3f05bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.145+0000 7f0733e00640 0 client.8211 ms_handle_reset on v2:172.21.15.144:6832/2351381466 2024-04-20T21:26:53.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.145+0000 7f0758df2740 1 -- 172.21.15.86:0/3293363146 >> 172.21.15.86:0/3293363146 conn(0x55ced3edd4c0 msgr2=0x55ced3f2b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:53.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.145+0000 7f0758df2740 1 -- 172.21.15.86:0/3293363146 shutdown_connections 2024-04-20T21:26:53.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.145+0000 7f0758df2740 1 -- 172.21.15.86:0/3293363146 wait complete. 2024-04-20T21:26:53.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.145+0000 7f0758df2740 20 client.8211 trim_cache size 0 max 16384 2024-04-20T21:26:53.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.165+0000 7f0758df2740 1 Processor -- start 2024-04-20T21:26:53.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.165+0000 7f0758df2740 1 -- start start 2024-04-20T21:26:53.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.165+0000 7f0758df2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ced3f03770 0x55ced3f05940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:53.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.166+0000 7f0758df2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ced3f09fe0 0x55ced3f2b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:53.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.166+0000 7f0758df2740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ced3f0a3c0 0x55ced3f0a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:53.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.166+0000 7f0758df2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0750200f50 con 0x55ced3f09fe0 2024-04-20T21:26:53.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.166+0000 7f0758df2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0750200050 con 0x55ced3f03770 2024-04-20T21:26:53.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.166+0000 7f0758df2740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f07500b4e90 con 0x55ced3f0a3c0 2024-04-20T21:26:53.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.166+0000 7f0747e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ced3f09fe0 0x55ced3f2b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:53.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.166+0000 7f0747e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ced3f09fe0 0x55ced3f2b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55964/0 (socket says 172.21.15.86:55964) 2024-04-20T21:26:53.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.166+0000 7f0747e00640 1 -- 172.21.15.86:0/988065272 learned_addr learned my addr 172.21.15.86:0/988065272 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:53.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.166+0000 7f074f400640 1 --2- 172.21.15.86:0/988065272 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ced3f03770 0x55ced3f05940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:53.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.166+0000 7f0747e00640 1 -- 172.21.15.86:0/988065272 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ced3f0a3c0 msgr2=0x55ced3f0a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:53.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.166+0000 7f0747e00640 1 --2- 172.21.15.86:0/988065272 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ced3f0a3c0 0x55ced3f0a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.166+0000 7f0747e00640 1 -- 172.21.15.86:0/988065272 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ced3f03770 msgr2=0x55ced3f05940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:53.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.166+0000 7f0747e00640 1 --2- 172.21.15.86:0/988065272 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ced3f03770 0x55ced3f05940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.166+0000 7f0747e00640 1 -- 172.21.15.86:0/988065272 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ced3b1a170 con 0x55ced3f09fe0 2024-04-20T21:26:53.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.167+0000 7f0747e00640 1 --2- 172.21.15.86:0/988065272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ced3f09fe0 0x55ced3f2b470 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f0738059ea0 tx=0x7f073808c0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6d928615a730df7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:53.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.167+0000 7f073ea00640 1 -- 172.21.15.86:0/988065272 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0738072070 con 0x55ced3f09fe0 2024-04-20T21:26:53.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.167+0000 7f073ea00640 1 -- 172.21.15.86:0/988065272 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0738059f10 con 0x55ced3f09fe0 2024-04-20T21:26:53.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.167+0000 7f073ea00640 1 -- 172.21.15.86:0/988065272 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073805e2c0 con 0x55ced3f09fe0 2024-04-20T21:26:53.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.167+0000 7f0758df2740 1 -- 172.21.15.86:0/988065272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ced3f09fe0 msgr2=0x55ced3f2b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:53.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.167+0000 7f0758df2740 1 --2- 172.21.15.86:0/988065272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ced3f09fe0 0x55ced3f2b470 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f0738059ea0 tx=0x7f073808c0b0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.167+0000 7f0758df2740 1 -- 172.21.15.86:0/988065272 shutdown_connections 2024-04-20T21:26:53.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.167+0000 7f0758df2740 1 --2- 172.21.15.86:0/988065272 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ced3f0a3c0 0x55ced3f0a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.167+0000 7f0758df2740 1 --2- 172.21.15.86:0/988065272 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ced3f09fe0 0x55ced3f2b470 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.167+0000 7f0758df2740 1 --2- 172.21.15.86:0/988065272 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ced3f03770 0x55ced3f05940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.167+0000 7f0758df2740 1 -- 172.21.15.86:0/988065272 >> 172.21.15.86:0/988065272 conn(0x55ced3edd4c0 msgr2=0x55ced3f27750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:53.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.168+0000 7f0758df2740 1 -- 172.21.15.86:0/988065272 shutdown_connections 2024-04-20T21:26:53.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.168+0000 7f0758df2740 1 -- 172.21.15.86:0/988065272 wait complete. 2024-04-20T21:26:53.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.168+0000 7f0758df2740 1 -- start start 2024-04-20T21:26:53.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.168+0000 7f0758df2740 1 Processor -- start 2024-04-20T21:26:53.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.169+0000 7f0758df2740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ced3f03770 0x55ced3f28ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:53.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.169+0000 7f0758df2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ced3f09fe0 0x55ced3f293e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:53.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.169+0000 7f0758df2740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ced3f0a3c0 0x55ced3f29920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:53.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.169+0000 7f0758df2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f07500ae430 con 0x55ced3f09fe0 2024-04-20T21:26:53.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.169+0000 7f0758df2740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ced3f99390 con 0x55ced3f0a3c0 2024-04-20T21:26:53.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.169+0000 7f0758df2740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55ced3ee0950 con 0x55ced3f03770 2024-04-20T21:26:53.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.170+0000 7f0758df2740 10 client.0 _ll_register_callbacks cb 0x7f0748001970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:26:53.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.170+0000 7f0747e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ced3f09fe0 0x55ced3f293e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:53.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.170+0000 7f0747e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ced3f09fe0 0x55ced3f293e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55974/0 (socket says 172.21.15.86:55974) 2024-04-20T21:26:53.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.170+0000 7f0747e00640 1 -- 172.21.15.86:0/1429475100 learned_addr learned my addr 172.21.15.86:0/1429475100 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:53.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.170+0000 7f0747e00640 1 -- 172.21.15.86:0/1429475100 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ced3f03770 msgr2=0x55ced3f28ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:53.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.170+0000 7f0747e00640 1 --2- 172.21.15.86:0/1429475100 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ced3f03770 0x55ced3f28ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.170+0000 7f0747e00640 1 -- 172.21.15.86:0/1429475100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ced3f0a3c0 msgr2=0x55ced3f29920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:53.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.170+0000 7f0747e00640 1 --2- 172.21.15.86:0/1429475100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ced3f0a3c0 0x55ced3f29920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.170+0000 7f0747e00640 1 -- 172.21.15.86:0/1429475100 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ced3f094e0 con 0x55ced3f09fe0 2024-04-20T21:26:53.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.170+0000 7f0747e00640 1 --2- 172.21.15.86:0/1429475100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ced3f09fe0 0x55ced3f293e0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f07380094b0 tx=0x7f07380069d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e101b6a46af87057 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:53.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.170+0000 7f0733e00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:26:53.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.170+0000 7f0733e00640 1 -- 172.21.15.86:0/1429475100 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0738072040 con 0x55ced3f09fe0 2024-04-20T21:26:53.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.171+0000 7f0733e00640 1 -- 172.21.15.86:0/1429475100 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7f073808a320 con 0x55ced3f09fe0 2024-04-20T21:26:53.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.171+0000 7f0758df2740 1 -- 172.21.15.86:0/1429475100 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0738059f10 con 0x55ced3f09fe0 2024-04-20T21:26:53.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.171+0000 7f0733e00640 1 -- 172.21.15.86:0/1429475100 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f0738056580 con 0x55ced3f09fe0 2024-04-20T21:26:53.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.171+0000 7f0758df2740 10 client.8217 fetch_fsmap learned FSMap version 73 2024-04-20T21:26:53.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.171+0000 7f0758df2740 1 -- 172.21.15.86:0/1429475100 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) -- 0x55ced3ee0620 con 0x55ced3f09fe0 2024-04-20T21:26:53.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.171+0000 7f0733e00640 1 -- 172.21.15.86:0/1429475100 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0738059f10 con 0x55ced3f09fe0 2024-04-20T21:26:53.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.171+0000 7f0733e00640 1 -- 172.21.15.86:0/1429475100 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 73) ==== 3495+0+0 (secure 0 0 0) 0x7f0738013050 con 0x55ced3f09fe0 2024-04-20T21:26:53.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.171+0000 7f0758df2740 10 client.8217 fetch_fsmap finished waiting for FSMap version 73 2024-04-20T21:26:53.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.171+0000 7f0758df2740 10 client.8217 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-04-20T21:26:53.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.171+0000 7f0758df2740 20 client.8217 populate_metadata read hostname 'smithi086' 2024-04-20T21:26:53.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.171+0000 7f0758df2740 1 --2- 172.21.15.86:0/1429475100 >> [v2:172.21.15.144:6832/2351381466,v1:172.21.15.144:6834/2351381466] conn(0x55ced3f93500 0x55ced3edbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:53.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.171+0000 7f0758df2740 4 client.8217 mds_command: new command op to 7518 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-04-20T21:26:53.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.171+0000 7f0758df2740 1 -- 172.21.15.86:0/1429475100 --> [v2:172.21.15.144:6832/2351381466,v1:172.21.15.144:6834/2351381466] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) -- 0x55ced3f054b0 con 0x55ced3f93500 2024-04-20T21:26:53.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.172+0000 7f074f400640 1 --2- 172.21.15.86:0/1429475100 >> [v2:172.21.15.144:6832/2351381466,v1:172.21.15.144:6834/2351381466] conn(0x55ced3f93500 0x55ced3edbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:53.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.173+0000 7f074f400640 1 --2- 172.21.15.86:0/1429475100 >> [v2:172.21.15.144:6832/2351381466,v1:172.21.15.144:6834/2351381466] conn(0x55ced3f93500 0x55ced3edbdc0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2147002ef3db451 server_cookie=aafa35e7f5ec09a2 in_seq=0 out_seq=0 2024-04-20T21:26:53.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.173+0000 7f0733e00640 10 client.8217 ms_handle_connect on v2:172.21.15.144:6832/2351381466 2024-04-20T21:26:53.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.174+0000 7f0733e00640 1 -- 172.21.15.86:0/1429475100 <== mds.0 v2:172.21.15.144:6832/2351381466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ced3f054b0 con 0x55ced3f93500 2024-04-20T21:26:53.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.174+0000 7f0733e00640 10 client.8217 handle_command_reply: tid=0 2024-04-20T21:26:53.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.174+0000 7f0758df2740 1 client.8217 shutdown 2024-04-20T21:26:53.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.174+0000 7f0758df2740 1 -- 172.21.15.86:0/1429475100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ced3f09fe0 msgr2=0x55ced3f293e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:53.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.174+0000 7f0758df2740 1 --2- 172.21.15.86:0/1429475100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ced3f09fe0 0x55ced3f293e0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f07380094b0 tx=0x7f07380069d0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.174+0000 7f0758df2740 1 -- 172.21.15.86:0/1429475100 shutdown_connections 2024-04-20T21:26:53.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.174+0000 7f0758df2740 1 --2- 172.21.15.86:0/1429475100 >> [v2:172.21.15.144:6832/2351381466,v1:172.21.15.144:6834/2351381466] conn(0x55ced3f93500 0x55ced3edbdc0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.174+0000 7f0758df2740 1 --2- 172.21.15.86:0/1429475100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ced3f0a3c0 0x55ced3f29920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.174+0000 7f0758df2740 1 --2- 172.21.15.86:0/1429475100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ced3f09fe0 0x55ced3f293e0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.174+0000 7f0758df2740 1 --2- 172.21.15.86:0/1429475100 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ced3f03770 0x55ced3f28ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.174+0000 7f0733e00640 0 client.8217 ms_handle_reset on v2:172.21.15.144:6832/2351381466 2024-04-20T21:26:53.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.174+0000 7f0758df2740 1 -- 172.21.15.86:0/1429475100 >> 172.21.15.86:0/1429475100 conn(0x55ced3edd4c0 msgr2=0x55ced3f05940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:53.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.174+0000 7f0758df2740 1 -- 172.21.15.86:0/1429475100 shutdown_connections 2024-04-20T21:26:53.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.174+0000 7f0758df2740 1 -- 172.21.15.86:0/1429475100 wait complete. 2024-04-20T21:26:53.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.175+0000 7f0758df2740 20 client.8217 trim_cache size 0 max 16384 2024-04-20T21:26:53.176 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:26:53.176 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-20T21:26:53.176 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "3cb25e17-ba12-4f61-b489-162e5b5df35a", 2024-04-20T21:26:53.176 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-20T21:26:53.176 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:26:53.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.175+0000 7f0756000640 1 -- 172.21.15.86:0/3073881304 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f072c06c8f0 msgr2=0x7f072c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:53.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.175+0000 7f0756000640 1 --2- 172.21.15.86:0/3073881304 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f072c06c8f0 0x7f072c06edb0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f0740036000 tx=0x7f0740008030 comp rx=0 tx=0).stop 2024-04-20T21:26:53.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.175+0000 7f0756000640 1 -- 172.21.15.86:0/3073881304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07500720e0 msgr2=0x7f075013acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:53.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.175+0000 7f0756000640 1 --2- 172.21.15.86:0/3073881304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07500720e0 0x7f075013acd0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f073800d7d0 tx=0x7f073800dc90 comp rx=0 tx=0).stop 2024-04-20T21:26:53.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.175+0000 7f0756000640 1 -- 172.21.15.86:0/3073881304 shutdown_connections 2024-04-20T21:26:53.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.175+0000 7f0756000640 1 --2- 172.21.15.86:0/3073881304 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f072c06c8f0 0x7f072c06edb0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.175+0000 7f0756000640 1 --2- 172.21.15.86:0/3073881304 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0750145fd0 0x7f075013fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.175+0000 7f0756000640 1 --2- 172.21.15.86:0/3073881304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07500720e0 0x7f075013acd0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.175+0000 7f0756000640 1 --2- 172.21.15.86:0/3073881304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0750071d00 0x7f0750142020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.175+0000 7f0756000640 1 -- 172.21.15.86:0/3073881304 >> 172.21.15.86:0/3073881304 conn(0x7f07500b2180 msgr2=0x7f075015d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:53.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.176+0000 7f0756000640 1 -- 172.21.15.86:0/3073881304 shutdown_connections 2024-04-20T21:26:53.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.176+0000 7f0756000640 1 -- 172.21.15.86:0/3073881304 wait complete. 2024-04-20T21:26:53.178 INFO:tasks.ceph.mds.d.smithi144.stderr:2024-04-20T21:26:53.177+0000 7fec95e00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() | dirtyparent=1 authpin=1 scrubqueue=0 0x564e4b572680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(13)0x10000000001:[<0x10000000000/file v6>,<0x1/dir v13>]//[]","memoryvalue":"(13)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-20T21:26:53.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:26:53.192 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-20T21:26:53.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.383+0000 7f3dd7000640 1 Processor -- start 2024-04-20T21:26:53.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.383+0000 7f3dd7000640 1 -- start start 2024-04-20T21:26:53.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.385+0000 7f3dd7000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3dd006bbf0 0x7f3dd006e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:53.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.385+0000 7f3dd7000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3dd006bfd0 0x7f3dd0145b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:53.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.385+0000 7f3dd7000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3dd0146040 0x7f3dd00cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:53.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.385+0000 7f3dd7000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f3dd01464b0 con 0x7f3dd006bbf0 2024-04-20T21:26:53.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.385+0000 7f3dd7000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f3dd015cd40 con 0x7f3dd006bfd0 2024-04-20T21:26:53.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.385+0000 7f3dd7000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3dd015d000 con 0x7f3dd0146040 2024-04-20T21:26:53.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.385+0000 7f3dcfe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3dd006bbf0 0x7f3dd006e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:53.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.385+0000 7f3dcfe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3dd006bbf0 0x7f3dd006e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55976/0 (socket says 172.21.15.86:55976) 2024-04-20T21:26:53.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.385+0000 7f3dcfe00640 1 -- 172.21.15.86:0/3073776376 learned_addr learned my addr 172.21.15.86:0/3073776376 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:53.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.386+0000 7f3dd5000640 1 --2- 172.21.15.86:0/3073776376 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3dd0146040 0x7f3dd00cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:53.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.386+0000 7f3dcfe00640 1 -- 172.21.15.86:0/3073776376 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3dd0146040 msgr2=0x7f3dd00cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:53.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.386+0000 7f3dc7400640 1 --2- 172.21.15.86:0/3073776376 >> v2:172.21.15.144:3300/0 conn(0x7f3dd006bfd0 0x7f3dd0145b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:53.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.386+0000 7f3dcfe00640 1 --2- 172.21.15.86:0/3073776376 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3dd0146040 0x7f3dd00cd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.386+0000 7f3dcfe00640 1 -- 172.21.15.86:0/3073776376 >> v2:172.21.15.144:3300/0 conn(0x7f3dd006bfd0 msgr2=0x7f3dd0145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:53.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.386+0000 7f3dcfe00640 1 --2- 172.21.15.86:0/3073776376 >> v2:172.21.15.144:3300/0 conn(0x7f3dd006bfd0 0x7f3dd0145b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.386+0000 7f3dcfe00640 1 -- 172.21.15.86:0/3073776376 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dd015d2e0 con 0x7f3dd006bbf0 2024-04-20T21:26:53.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.387+0000 7f3dcfe00640 1 --2- 172.21.15.86:0/3073776376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3dd006bbf0 0x7f3dd006e000 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f3dd006dcc0 tx=0x7f3dc002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540ab81e4752311f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:53.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.387+0000 7f3dcf400640 1 -- 172.21.15.86:0/3073776376 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0035020 con 0x7f3dd006bbf0 2024-04-20T21:26:53.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.387+0000 7f3dcf400640 1 -- 172.21.15.86:0/3073776376 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dc0031870 con 0x7f3dd006bbf0 2024-04-20T21:26:53.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.387+0000 7f3dcf400640 1 -- 172.21.15.86:0/3073776376 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0031c40 con 0x7f3dd006bbf0 2024-04-20T21:26:53.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.388+0000 7f3dd7000640 1 -- 172.21.15.86:0/3073776376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3dd006bbf0 msgr2=0x7f3dd006e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:53.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.388+0000 7f3dd7000640 1 --2- 172.21.15.86:0/3073776376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3dd006bbf0 0x7f3dd006e000 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f3dd006dcc0 tx=0x7f3dc002b910 comp rx=0 tx=0).stop 2024-04-20T21:26:53.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.388+0000 7f3dd7000640 1 -- 172.21.15.86:0/3073776376 shutdown_connections 2024-04-20T21:26:53.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.388+0000 7f3dd7000640 1 --2- 172.21.15.86:0/3073776376 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3dd0146040 0x7f3dd00cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.388+0000 7f3dd7000640 1 --2- 172.21.15.86:0/3073776376 >> v2:172.21.15.144:3300/0 conn(0x7f3dd006bfd0 0x7f3dd0145b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.388+0000 7f3dd7000640 1 --2- 172.21.15.86:0/3073776376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3dd006bbf0 0x7f3dd006e000 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.388+0000 7f3dd7000640 1 -- 172.21.15.86:0/3073776376 >> 172.21.15.86:0/3073776376 conn(0x7f3dd00b2110 msgr2=0x7f3dd00b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:53.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.388+0000 7f3dd7000640 1 -- 172.21.15.86:0/3073776376 shutdown_connections 2024-04-20T21:26:53.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.388+0000 7f3dd7000640 1 -- 172.21.15.86:0/3073776376 wait complete. 2024-04-20T21:26:53.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.390+0000 7f3dd7000640 1 Processor -- start 2024-04-20T21:26:53.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.390+0000 7f3dd7000640 1 -- start start 2024-04-20T21:26:53.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.391+0000 7f3dd7000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3dd006bbf0 0x7f3dd01f32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:53.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.391+0000 7f3dd7000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3dd006bfd0 0x7f3dd00ca650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:53.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.391+0000 7f3dd7000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3dd0146040 0x7f3dd00cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:53.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.391+0000 7f3dd7000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd00cda80 con 0x7f3dd006bbf0 2024-04-20T21:26:53.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.391+0000 7f3dd7000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd015d8a0 con 0x7f3dd0146040 2024-04-20T21:26:53.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.391+0000 7f3dd7000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3dd00cdc00 con 0x7f3dd006bfd0 2024-04-20T21:26:53.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.391+0000 7f3dcfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3dd006bbf0 0x7f3dd01f32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:53.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.391+0000 7f3dcfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3dd006bbf0 0x7f3dd01f32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55986/0 (socket says 172.21.15.86:55986) 2024-04-20T21:26:53.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.391+0000 7f3dcfe00640 1 -- 172.21.15.86:0/1928695924 learned_addr learned my addr 172.21.15.86:0/1928695924 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:53.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.391+0000 7f3dd5000640 1 --2- 172.21.15.86:0/1928695924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3dd0146040 0x7f3dd00cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:53.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.391+0000 7f3dc7400640 1 --2- 172.21.15.86:0/1928695924 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3dd006bfd0 0x7f3dd00ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:53.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.391+0000 7f3dcfe00640 1 -- 172.21.15.86:0/1928695924 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3dd006bfd0 msgr2=0x7f3dd00ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:53.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.391+0000 7f3dcfe00640 1 --2- 172.21.15.86:0/1928695924 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3dd006bfd0 0x7f3dd00ca650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.391+0000 7f3dcfe00640 1 -- 172.21.15.86:0/1928695924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3dd0146040 msgr2=0x7f3dd00cb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:53.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.391+0000 7f3dcfe00640 1 --2- 172.21.15.86:0/1928695924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3dd0146040 0x7f3dd00cb1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.391+0000 7f3dcfe00640 1 -- 172.21.15.86:0/1928695924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dd01f3820 con 0x7f3dd006bbf0 2024-04-20T21:26:53.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.392+0000 7f3dcfe00640 1 --2- 172.21.15.86:0/1928695924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3dd006bbf0 0x7f3dd01f32e0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f3dc002b950 tx=0x7f3dc00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:53.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.392+0000 7f3dccc00640 1 -- 172.21.15.86:0/1928695924 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0040070 con 0x7f3dd006bbf0 2024-04-20T21:26:53.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.392+0000 7f3dccc00640 1 -- 172.21.15.86:0/1928695924 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dc003a420 con 0x7f3dd006bbf0 2024-04-20T21:26:53.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.392+0000 7f3dccc00640 1 -- 172.21.15.86:0/1928695924 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0035040 con 0x7f3dd006bbf0 2024-04-20T21:26:53.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.392+0000 7f3dd7000640 1 -- 172.21.15.86:0/1928695924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dd00c95b0 con 0x7f3dd006bbf0 2024-04-20T21:26:53.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.392+0000 7f3dd7000640 1 -- 172.21.15.86:0/1928695924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dd00c9b70 con 0x7f3dd006bbf0 2024-04-20T21:26:53.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.395+0000 7f3dccc00640 1 -- 172.21.15.86:0/1928695924 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3dc004b060 con 0x7f3dd006bbf0 2024-04-20T21:26:53.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.395+0000 7f3dccc00640 1 --2- 172.21.15.86:0/1928695924 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3dac06c8f0 0x7f3dac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:53.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.396+0000 7f3dccc00640 1 -- 172.21.15.86:0/1928695924 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7f3dc00adc50 con 0x7f3dd006bbf0 2024-04-20T21:26:53.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.396+0000 7f3dc7400640 1 --2- 172.21.15.86:0/1928695924 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3dac06c8f0 0x7f3dac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:53.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.396+0000 7f3dd9d74740 1 Processor -- start 2024-04-20T21:26:53.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.396+0000 7f3dd9d74740 1 -- start start 2024-04-20T21:26:53.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.396+0000 7f3dc7400640 1 --2- 172.21.15.86:0/1928695924 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3dac06c8f0 0x7f3dac06edb0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3dd00c7880 tx=0x7f3db80083c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:53.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.397+0000 7f3dd9d74740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55616b1121f0 0x55616b1125d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:53.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.397+0000 7f3dd9d74740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55616b0e2210 0x55616b10f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:53.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.397+0000 7f3dd9d74740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55616b0e5d70 0x55616b10f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:53.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.397+0000 7f3dd9d74740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55616b04e150 con 0x55616b0e2210 2024-04-20T21:26:53.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.397+0000 7f3dd9d74740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55616aff4370 con 0x55616b0e5d70 2024-04-20T21:26:53.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.397+0000 7f3dd9d74740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55616b04a110 con 0x55616b1121f0 2024-04-20T21:26:53.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.397+0000 7f3dc7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55616b0e2210 0x55616b10f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:53.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.397+0000 7f3dc7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55616b0e2210 0x55616b10f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55994/0 (socket says 172.21.15.86:55994) 2024-04-20T21:26:53.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.397+0000 7f3dc7400640 1 -- 172.21.15.86:0/2756366407 learned_addr learned my addr 172.21.15.86:0/2756366407 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:53.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.398+0000 7f3dcfe00640 1 --2- 172.21.15.86:0/2756366407 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55616b1121f0 0x55616b1125d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:53.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.398+0000 7f3dd5000640 1 --2- 172.21.15.86:0/2756366407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55616b0e5d70 0x55616b10f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:53.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.398+0000 7f3dc7400640 1 -- 172.21.15.86:0/2756366407 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55616b1121f0 msgr2=0x55616b1125d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:53.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.398+0000 7f3dc7400640 1 --2- 172.21.15.86:0/2756366407 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55616b1121f0 0x55616b1125d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.398+0000 7f3dc7400640 1 -- 172.21.15.86:0/2756366407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55616b0e5d70 msgr2=0x55616b10f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:53.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.398+0000 7f3dc7400640 1 --2- 172.21.15.86:0/2756366407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55616b0e5d70 0x55616b10f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.398+0000 7f3dc7400640 1 -- 172.21.15.86:0/2756366407 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55616ae2a7d0 con 0x55616b0e2210 2024-04-20T21:26:53.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.398+0000 7f3dd5000640 1 --2- 172.21.15.86:0/2756366407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55616b0e5d70 0x55616b10f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:26:53.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.398+0000 7f3dc7400640 1 --2- 172.21.15.86:0/2756366407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55616b0e2210 0x55616b10f460 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f3db800e740 tx=0x7f3db8010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63171c97f851b9d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:53.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.398+0000 7f3dc4c00640 1 -- 172.21.15.86:0/2756366407 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db801b8f0 con 0x55616b0e2210 2024-04-20T21:26:53.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.399+0000 7f3dc4c00640 1 -- 172.21.15.86:0/2756366407 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3db801ba90 con 0x55616b0e2210 2024-04-20T21:26:53.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.399+0000 7f3dc4c00640 1 -- 172.21.15.86:0/2756366407 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db801be60 con 0x55616b0e2210 2024-04-20T21:26:53.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.399+0000 7f3dd9d74740 1 -- 172.21.15.86:0/2756366407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55616b0e2210 msgr2=0x55616b10f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:53.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.399+0000 7f3dd9d74740 1 --2- 172.21.15.86:0/2756366407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55616b0e2210 0x55616b10f460 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f3db800e740 tx=0x7f3db8010000 comp rx=0 tx=0).stop 2024-04-20T21:26:53.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.399+0000 7f3dd9d74740 1 -- 172.21.15.86:0/2756366407 shutdown_connections 2024-04-20T21:26:53.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.399+0000 7f3dd9d74740 1 --2- 172.21.15.86:0/2756366407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55616b0e5d70 0x55616b10f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.399+0000 7f3dd9d74740 1 --2- 172.21.15.86:0/2756366407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55616b0e2210 0x55616b10f460 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.399+0000 7f3dd9d74740 1 --2- 172.21.15.86:0/2756366407 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55616b1121f0 0x55616b1125d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.399+0000 7f3dd9d74740 1 -- 172.21.15.86:0/2756366407 >> 172.21.15.86:0/2756366407 conn(0x55616b0e4480 msgr2=0x55616b12d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:53.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.399+0000 7f3dd9d74740 1 -- 172.21.15.86:0/2756366407 shutdown_connections 2024-04-20T21:26:53.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.399+0000 7f3dd9d74740 1 -- 172.21.15.86:0/2756366407 wait complete. 2024-04-20T21:26:53.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.400+0000 7f3dd9d74740 1 -- start start 2024-04-20T21:26:53.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.400+0000 7f3dd9d74740 1 Processor -- start 2024-04-20T21:26:53.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.401+0000 7f3dd9d74740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55616b0e5d70 0x55616b104590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:53.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.401+0000 7f3dd9d74740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55616b1121f0 0x55616b112b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:53.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.401+0000 7f3dd9d74740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55616b102150 0x55616b113070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:53.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.401+0000 7f3dd9d74740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55616b0e6f20 con 0x55616b1121f0 2024-04-20T21:26:53.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.401+0000 7f3dd9d74740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55616b007740 con 0x55616b102150 2024-04-20T21:26:53.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.401+0000 7f3dd9d74740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55616b0e6310 con 0x55616b0e5d70 2024-04-20T21:26:53.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.401+0000 7f3dd9d74740 10 client.0 _ll_register_callbacks cb 0x55616b002b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:26:53.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.402+0000 7f3dc7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55616b1121f0 0x55616b112b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:53.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.402+0000 7f3dc7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55616b1121f0 0x55616b112b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55998/0 (socket says 172.21.15.86:55998) 2024-04-20T21:26:53.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.402+0000 7f3dc7400640 1 -- 172.21.15.86:0/1343088850 learned_addr learned my addr 172.21.15.86:0/1343088850 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:53.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.402+0000 7f3dd5000640 1 --2- 172.21.15.86:0/1343088850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55616b102150 0x55616b113070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:53.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.402+0000 7f3dcfe00640 1 --2- 172.21.15.86:0/1343088850 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55616b0e5d70 0x55616b104590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:53.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.402+0000 7f3dc7400640 1 -- 172.21.15.86:0/1343088850 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55616b0e5d70 msgr2=0x55616b104590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:53.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.402+0000 7f3dc7400640 1 --2- 172.21.15.86:0/1343088850 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55616b0e5d70 0x55616b104590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.402+0000 7f3dc7400640 1 -- 172.21.15.86:0/1343088850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55616b102150 msgr2=0x55616b113070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:53.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.402+0000 7f3dc7400640 1 --2- 172.21.15.86:0/1343088850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55616b102150 0x55616b113070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.402+0000 7f3dc7400640 1 -- 172.21.15.86:0/1343088850 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55616ad603f0 con 0x55616b1121f0 2024-04-20T21:26:53.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.402+0000 7f3dd5000640 1 --2- 172.21.15.86:0/1343088850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55616b102150 0x55616b113070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:26:53.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.402+0000 7f3dc7400640 1 --2- 172.21.15.86:0/1343088850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55616b1121f0 0x55616b112b30 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f3db800e740 tx=0x7f3db8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fbb4665c12a56c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:53.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.402+0000 7f3dbd600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:26:53.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.403+0000 7f3dbd600640 1 -- 172.21.15.86:0/1343088850 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db8003da0 con 0x55616b1121f0 2024-04-20T21:26:53.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.403+0000 7f3dbd600640 1 -- 172.21.15.86:0/1343088850 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7f3db8030c00 con 0x55616b1121f0 2024-04-20T21:26:53.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.403+0000 7f3dd9d74740 1 -- 172.21.15.86:0/1343088850 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3db801ba90 con 0x55616b1121f0 2024-04-20T21:26:53.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.403+0000 7f3dbd600640 1 -- 172.21.15.86:0/1343088850 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f3db8030e00 con 0x55616b1121f0 2024-04-20T21:26:53.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.403+0000 7f3dd9d74740 10 client.8229 fetch_fsmap learned FSMap version 73 2024-04-20T21:26:53.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.403+0000 7f3dd9d74740 1 -- 172.21.15.86:0/1343088850 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) -- 0x55616ad200f0 con 0x55616b1121f0 2024-04-20T21:26:53.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.403+0000 7f3dbd600640 1 -- 172.21.15.86:0/1343088850 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3db801ba90 con 0x55616b1121f0 2024-04-20T21:26:53.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.403+0000 7f3dbd600640 1 -- 172.21.15.86:0/1343088850 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 73) ==== 3495+0+0 (secure 0 0 0) 0x7f3db800b050 con 0x55616b1121f0 2024-04-20T21:26:53.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.404+0000 7f3dd9d74740 10 client.8229 fetch_fsmap finished waiting for FSMap version 73 2024-04-20T21:26:53.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.404+0000 7f3dd9d74740 10 client.8229 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-04-20T21:26:53.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.404+0000 7f3dd9d74740 20 client.8229 populate_metadata read hostname 'smithi086' 2024-04-20T21:26:53.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.404+0000 7f3dd9d74740 1 --2- 172.21.15.86:0/1343088850 >> [v2:172.21.15.144:6832/2351381466,v1:172.21.15.144:6834/2351381466] conn(0x55616b115210 0x55616b115620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:53.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.404+0000 7f3dd9d74740 4 client.8229 mds_command: new command op to 7518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:26:53.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.404+0000 7f3dd9d74740 1 -- 172.21.15.86:0/1343088850 --> [v2:172.21.15.144:6832/2351381466,v1:172.21.15.144:6834/2351381466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55616b05bb70 con 0x55616b115210 2024-04-20T21:26:53.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.404+0000 7f3dcfe00640 1 --2- 172.21.15.86:0/1343088850 >> [v2:172.21.15.144:6832/2351381466,v1:172.21.15.144:6834/2351381466] conn(0x55616b115210 0x55616b115620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:53.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.405+0000 7f3dcfe00640 1 --2- 172.21.15.86:0/1343088850 >> [v2:172.21.15.144:6832/2351381466,v1:172.21.15.144:6834/2351381466] conn(0x55616b115210 0x55616b115620 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba555ecc0c32974a server_cookie=573746167ed92610 in_seq=0 out_seq=0 2024-04-20T21:26:53.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.405+0000 7f3dbd600640 10 client.8229 ms_handle_connect on v2:172.21.15.144:6832/2351381466 2024-04-20T21:26:53.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.407+0000 7f3dbd600640 1 -- 172.21.15.86:0/1343088850 <== mds.0 v2:172.21.15.144:6832/2351381466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55616b05bb70 con 0x55616b115210 2024-04-20T21:26:53.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.407+0000 7f3dbd600640 10 client.8229 handle_command_reply: tid=0 2024-04-20T21:26:53.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.407+0000 7f3dd9d74740 1 client.8229 shutdown 2024-04-20T21:26:53.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.407+0000 7f3dd9d74740 1 -- 172.21.15.86:0/1343088850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55616b1121f0 msgr2=0x55616b112b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:53.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.407+0000 7f3dd9d74740 1 --2- 172.21.15.86:0/1343088850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55616b1121f0 0x55616b112b30 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f3db800e740 tx=0x7f3db8004ab0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.407+0000 7f3dd9d74740 1 -- 172.21.15.86:0/1343088850 shutdown_connections 2024-04-20T21:26:53.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.407+0000 7f3dd9d74740 1 --2- 172.21.15.86:0/1343088850 >> [v2:172.21.15.144:6832/2351381466,v1:172.21.15.144:6834/2351381466] conn(0x55616b115210 0x55616b115620 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.407+0000 7f3dd9d74740 1 --2- 172.21.15.86:0/1343088850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55616b102150 0x55616b113070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.407+0000 7f3dd9d74740 1 --2- 172.21.15.86:0/1343088850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55616b1121f0 0x55616b112b30 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.407+0000 7f3dd9d74740 1 --2- 172.21.15.86:0/1343088850 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55616b0e5d70 0x55616b104590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.407+0000 7f3dbd600640 0 client.8229 ms_handle_reset on v2:172.21.15.144:6832/2351381466 2024-04-20T21:26:53.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.407+0000 7f3dd9d74740 1 -- 172.21.15.86:0/1343088850 >> 172.21.15.86:0/1343088850 conn(0x55616b0e2210 msgr2=0x55616b0e70a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:53.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.408+0000 7f3dd9d74740 1 -- 172.21.15.86:0/1343088850 shutdown_connections 2024-04-20T21:26:53.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.408+0000 7f3dd9d74740 1 -- 172.21.15.86:0/1343088850 wait complete. 2024-04-20T21:26:53.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.408+0000 7f3dd9d74740 20 client.8229 trim_cache size 0 max 16384 2024-04-20T21:26:53.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.427+0000 7f3dd9d74740 1 Processor -- start 2024-04-20T21:26:53.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.427+0000 7f3dd9d74740 1 -- start start 2024-04-20T21:26:53.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.428+0000 7f3dd9d74740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55616b0e5d70 0x55616b0ddc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:53.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.428+0000 7f3dd9d74740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55616b102150 0x55616b12fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:53.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.428+0000 7f3dd9d74740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55616b1121f0 0x55616b1155f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:53.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.428+0000 7f3dd9d74740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd01f6ab0 con 0x55616b0e5d70 2024-04-20T21:26:53.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.428+0000 7f3dd9d74740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd00b5390 con 0x55616b1121f0 2024-04-20T21:26:53.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.428+0000 7f3dd9d74740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3dd00b4720 con 0x55616b102150 2024-04-20T21:26:53.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.428+0000 7f3dcfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55616b0e5d70 0x55616b0ddc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:53.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.428+0000 7f3dcfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55616b0e5d70 0x55616b0ddc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56004/0 (socket says 172.21.15.86:56004) 2024-04-20T21:26:53.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.428+0000 7f3dcfe00640 1 -- 172.21.15.86:0/1470496805 learned_addr learned my addr 172.21.15.86:0/1470496805 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:53.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.428+0000 7f3dcfe00640 1 -- 172.21.15.86:0/1470496805 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55616b102150 msgr2=0x55616b12fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:53.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.428+0000 7f3dcfe00640 1 --2- 172.21.15.86:0/1470496805 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55616b102150 0x55616b12fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.428+0000 7f3dcfe00640 1 -- 172.21.15.86:0/1470496805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55616b1121f0 msgr2=0x55616b1155f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:53.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.428+0000 7f3dcfe00640 1 --2- 172.21.15.86:0/1470496805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55616b1121f0 0x55616b1155f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.428+0000 7f3dcfe00640 1 -- 172.21.15.86:0/1470496805 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55616b13c800 con 0x55616b0e5d70 2024-04-20T21:26:53.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.429+0000 7f3dcfe00640 1 --2- 172.21.15.86:0/1470496805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55616b0e5d70 0x55616b0ddc00 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f3dc002bf20 tx=0x7f3dc0046040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e71bd9b38e778795 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:53.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.429+0000 7f3dbfe00640 1 -- 172.21.15.86:0/1470496805 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc00843f0 con 0x55616b0e5d70 2024-04-20T21:26:53.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.429+0000 7f3dbfe00640 1 -- 172.21.15.86:0/1470496805 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dc00798b0 con 0x55616b0e5d70 2024-04-20T21:26:53.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.429+0000 7f3dbfe00640 1 -- 172.21.15.86:0/1470496805 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0079b10 con 0x55616b0e5d70 2024-04-20T21:26:53.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.430+0000 7f3dd9d74740 1 -- 172.21.15.86:0/1470496805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55616b0e5d70 msgr2=0x55616b0ddc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:53.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.430+0000 7f3dd9d74740 1 --2- 172.21.15.86:0/1470496805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55616b0e5d70 0x55616b0ddc00 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f3dc002bf20 tx=0x7f3dc0046040 comp rx=0 tx=0).stop 2024-04-20T21:26:53.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.430+0000 7f3dd9d74740 1 -- 172.21.15.86:0/1470496805 shutdown_connections 2024-04-20T21:26:53.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.430+0000 7f3dd9d74740 1 --2- 172.21.15.86:0/1470496805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55616b1121f0 0x55616b1155f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.430+0000 7f3dd9d74740 1 --2- 172.21.15.86:0/1470496805 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55616b102150 0x55616b12fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.430+0000 7f3dd9d74740 1 --2- 172.21.15.86:0/1470496805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55616b0e5d70 0x55616b0ddc00 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.430+0000 7f3dd9d74740 1 -- 172.21.15.86:0/1470496805 >> 172.21.15.86:0/1470496805 conn(0x55616b0e2210 msgr2=0x55616b130510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:53.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.430+0000 7f3dd9d74740 1 -- 172.21.15.86:0/1470496805 shutdown_connections 2024-04-20T21:26:53.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.430+0000 7f3dd9d74740 1 -- 172.21.15.86:0/1470496805 wait complete. 2024-04-20T21:26:53.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.430+0000 7f3dd9d74740 1 -- start start 2024-04-20T21:26:53.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.431+0000 7f3dd9d74740 1 Processor -- start 2024-04-20T21:26:53.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.431+0000 7f3dd9d74740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55616b0e5d70 0x55616b14b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:53.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.431+0000 7f3dd9d74740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55616b102150 0x55616b14b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:53.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.432+0000 7f3dd9d74740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55616b1121f0 0x55616b102d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:53.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.432+0000 7f3dd9d74740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55616b101b00 con 0x55616b0e5d70 2024-04-20T21:26:53.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.432+0000 7f3dd9d74740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd00b4d40 con 0x55616b102150 2024-04-20T21:26:53.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.432+0000 7f3dd9d74740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55616b107da0 con 0x55616b1121f0 2024-04-20T21:26:53.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.432+0000 7f3dd9d74740 10 client.0 _ll_register_callbacks cb 0x7f3db800e610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:26:53.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.432+0000 7f3dcfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55616b0e5d70 0x55616b14b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:53.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.432+0000 7f3dcfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55616b0e5d70 0x55616b14b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56012/0 (socket says 172.21.15.86:56012) 2024-04-20T21:26:53.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.432+0000 7f3dcfe00640 1 -- 172.21.15.86:0/1229947058 learned_addr learned my addr 172.21.15.86:0/1229947058 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:26:53.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.432+0000 7f3dcfe00640 1 -- 172.21.15.86:0/1229947058 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55616b1121f0 msgr2=0x55616b102d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:53.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.432+0000 7f3dcfe00640 1 --2- 172.21.15.86:0/1229947058 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55616b1121f0 0x55616b102d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.432+0000 7f3dcfe00640 1 -- 172.21.15.86:0/1229947058 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55616b102150 msgr2=0x55616b14b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:26:53.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.432+0000 7f3dcfe00640 1 --2- 172.21.15.86:0/1229947058 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55616b102150 0x55616b14b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.432+0000 7f3dcfe00640 1 -- 172.21.15.86:0/1229947058 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55616b1abf10 con 0x55616b0e5d70 2024-04-20T21:26:53.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.432+0000 7f3dcfe00640 1 --2- 172.21.15.86:0/1229947058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55616b0e5d70 0x55616b14b2c0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0042000 tx=0x7f3dc003c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e1a1a78ef18e8b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:26:53.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.432+0000 7f3dbd600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:26:53.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.433+0000 7f3dbd600640 1 -- 172.21.15.86:0/1229947058 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc00843f0 con 0x55616b0e5d70 2024-04-20T21:26:53.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.433+0000 7f3dbd600640 1 -- 172.21.15.86:0/1229947058 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7f3dc0079b10 con 0x55616b0e5d70 2024-04-20T21:26:53.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.433+0000 7f3dd9d74740 1 -- 172.21.15.86:0/1229947058 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3dc00798b0 con 0x55616b0e5d70 2024-04-20T21:26:53.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.433+0000 7f3dbd600640 1 -- 172.21.15.86:0/1229947058 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f3dc0031730 con 0x55616b0e5d70 2024-04-20T21:26:53.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.433+0000 7f3dd9d74740 10 client.8235 fetch_fsmap learned FSMap version 73 2024-04-20T21:26:53.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.433+0000 7f3dd9d74740 1 -- 172.21.15.86:0/1229947058 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) -- 0x55616b115e30 con 0x55616b0e5d70 2024-04-20T21:26:53.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.433+0000 7f3dbd600640 1 -- 172.21.15.86:0/1229947058 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dc00798b0 con 0x55616b0e5d70 2024-04-20T21:26:53.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.433+0000 7f3dbd600640 1 -- 172.21.15.86:0/1229947058 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 73) ==== 3495+0+0 (secure 0 0 0) 0x7f3dc00ac090 con 0x55616b0e5d70 2024-04-20T21:26:53.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.433+0000 7f3dd9d74740 10 client.8235 fetch_fsmap finished waiting for FSMap version 73 2024-04-20T21:26:53.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.433+0000 7f3dd9d74740 10 client.8235 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-04-20T21:26:53.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.433+0000 7f3dd9d74740 20 client.8235 populate_metadata read hostname 'smithi086' 2024-04-20T21:26:53.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.433+0000 7f3dd9d74740 1 --2- 172.21.15.86:0/1229947058 >> [v2:172.21.15.144:6832/2351381466,v1:172.21.15.144:6834/2351381466] conn(0x55616b0e2610 0x55616b103280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:26:53.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.433+0000 7f3dd9d74740 4 client.8235 mds_command: new command op to 7518 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:26:53.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.433+0000 7f3dd9d74740 1 -- 172.21.15.86:0/1229947058 --> [v2:172.21.15.144:6832/2351381466,v1:172.21.15.144:6834/2351381466] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55616b0e3b70 con 0x55616b0e2610 2024-04-20T21:26:53.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.434+0000 7f3dc7400640 1 --2- 172.21.15.86:0/1229947058 >> [v2:172.21.15.144:6832/2351381466,v1:172.21.15.144:6834/2351381466] conn(0x55616b0e2610 0x55616b103280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:26:53.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.435+0000 7f3dc7400640 1 --2- 172.21.15.86:0/1229947058 >> [v2:172.21.15.144:6832/2351381466,v1:172.21.15.144:6834/2351381466] conn(0x55616b0e2610 0x55616b103280 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be442398ddec6595 server_cookie=5e5cfc20dbe72bee in_seq=0 out_seq=0 2024-04-20T21:26:53.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.435+0000 7f3dbd600640 10 client.8235 ms_handle_connect on v2:172.21.15.144:6832/2351381466 2024-04-20T21:26:53.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.435+0000 7f3dbd600640 1 -- 172.21.15.86:0/1229947058 <== mds.0 v2:172.21.15.144:6832/2351381466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+263 (crc 0 0 0) 0x55616b0e3b70 con 0x55616b0e2610 2024-04-20T21:26:53.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.435+0000 7f3dbd600640 10 client.8235 handle_command_reply: tid=0 2024-04-20T21:26:53.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.435+0000 7f3dd9d74740 1 client.8235 shutdown 2024-04-20T21:26:53.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.436+0000 7f3dd9d74740 1 -- 172.21.15.86:0/1229947058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55616b0e5d70 msgr2=0x55616b14b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:53.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.436+0000 7f3dd9d74740 1 --2- 172.21.15.86:0/1229947058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55616b0e5d70 0x55616b14b2c0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0042000 tx=0x7f3dc003c040 comp rx=0 tx=0).stop 2024-04-20T21:26:53.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.436+0000 7f3dd9d74740 1 -- 172.21.15.86:0/1229947058 shutdown_connections 2024-04-20T21:26:53.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.436+0000 7f3dd9d74740 1 --2- 172.21.15.86:0/1229947058 >> [v2:172.21.15.144:6832/2351381466,v1:172.21.15.144:6834/2351381466] conn(0x55616b0e2610 0x55616b103280 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.436+0000 7f3dd9d74740 1 --2- 172.21.15.86:0/1229947058 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55616b1121f0 0x55616b102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.436+0000 7f3dd9d74740 1 --2- 172.21.15.86:0/1229947058 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55616b102150 0x55616b14b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.436+0000 7f3dd9d74740 1 --2- 172.21.15.86:0/1229947058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55616b0e5d70 0x55616b14b2c0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.436+0000 7f3dbd600640 0 client.8235 ms_handle_reset on v2:172.21.15.144:6832/2351381466 2024-04-20T21:26:53.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.436+0000 7f3dd9d74740 1 -- 172.21.15.86:0/1229947058 >> 172.21.15.86:0/1229947058 conn(0x55616b0e2210 msgr2=0x55616b12fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:53.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.436+0000 7f3dd9d74740 1 -- 172.21.15.86:0/1229947058 shutdown_connections 2024-04-20T21:26:53.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.436+0000 7f3dd9d74740 1 -- 172.21.15.86:0/1229947058 wait complete. 2024-04-20T21:26:53.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.436+0000 7f3dd9d74740 20 client.8235 trim_cache size 0 max 16384 2024-04-20T21:26:53.438 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:26:53.438 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T21:26:53.439 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-20T21:26:53.439 INFO:teuthology.orchestra.run.smithi086.stdout: "3cb25e17-ba12-4f61-b489-162e5b5df35a": { 2024-04-20T21:26:53.439 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/dir", 2024-04-20T21:26:53.439 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "3cb25e17-ba12-4f61-b489-162e5b5df35a", 2024-04-20T21:26:53.439 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,repair" 2024-04-20T21:26:53.439 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:26:53.440 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:26:53.440 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:26:53.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.437+0000 7f3dd7000640 1 -- 172.21.15.86:0/1928695924 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3dac06c8f0 msgr2=0x7f3dac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:53.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.437+0000 7f3dd7000640 1 --2- 172.21.15.86:0/1928695924 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3dac06c8f0 0x7f3dac06edb0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3dd00c7880 tx=0x7f3db80083c0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.437+0000 7f3dd7000640 1 -- 172.21.15.86:0/1928695924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3dd006bbf0 msgr2=0x7f3dd01f32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:26:53.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.437+0000 7f3dd7000640 1 --2- 172.21.15.86:0/1928695924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3dd006bbf0 0x7f3dd01f32e0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f3dc002b950 tx=0x7f3dc00046c0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.437+0000 7f3dd7000640 1 -- 172.21.15.86:0/1928695924 shutdown_connections 2024-04-20T21:26:53.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.437+0000 7f3dd7000640 1 --2- 172.21.15.86:0/1928695924 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3dac06c8f0 0x7f3dac06edb0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.437+0000 7f3dd7000640 1 --2- 172.21.15.86:0/1928695924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3dd0146040 0x7f3dd00cb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.437+0000 7f3dd7000640 1 --2- 172.21.15.86:0/1928695924 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3dd006bfd0 0x7f3dd00ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.437+0000 7f3dd7000640 1 --2- 172.21.15.86:0/1928695924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3dd006bbf0 0x7f3dd01f32e0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:26:53.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.437+0000 7f3dd7000640 1 -- 172.21.15.86:0/1928695924 >> 172.21.15.86:0/1928695924 conn(0x7f3dd00b2110 msgr2=0x7f3dd006e330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:26:53.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.437+0000 7f3dd7000640 1 -- 172.21.15.86:0/1928695924 shutdown_connections 2024-04-20T21:26:53.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:26:53.437+0000 7f3dd7000640 1 -- 172.21.15.86:0/1928695924 wait complete. 2024-04-20T21:26:53.453 INFO:tasks.cephfs.filesystem:scrub status for tag:3cb25e17-ba12-4f61-b489-162e5b5df35a - {'path': '/dir', 'tag': '3cb25e17-ba12-4f61-b489-162e5b5df35a', 'options': 'recursive,repair'} 2024-04-20T21:27:17.702 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:27:17.710 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:27:23.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:27:23.454 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-20T21:27:23.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.664+0000 7f67d6200640 1 Processor -- start 2024-04-20T21:27:23.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.665+0000 7f67d6200640 1 -- start start 2024-04-20T21:27:23.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.666+0000 7f67d6200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f67d015e8e0 0x7f67d006d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:23.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.666+0000 7f67d6200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f67d00d4d70 0x7f67d00cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:23.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.666+0000 7f67d6200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67d00d1f60 con 0x7f67d015ecc0 2024-04-20T21:27:23.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.666+0000 7f67d6200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f67d00cf7f0 con 0x7f67d015e8e0 2024-04-20T21:27:23.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.666+0000 7f67d6200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f67d00d19a0 con 0x7f67d00d4d70 2024-04-20T21:27:23.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.666+0000 7f67d4e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f67d015e8e0 0x7f67d006d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:23.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.666+0000 7f67d4e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f67d015e8e0 0x7f67d006d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37500/0 (socket says 172.21.15.86:37500) 2024-04-20T21:27:23.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.666+0000 7f67d4e00640 1 -- 172.21.15.86:0/4139086116 learned_addr learned my addr 172.21.15.86:0/4139086116 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:23.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.667+0000 7f67d5800640 1 --2- 172.21.15.86:0/4139086116 >> v2:172.21.15.144:3300/0 conn(0x7f67d00d4d70 0x7f67d00cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:23.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.667+0000 7f67d4e00640 1 -- 172.21.15.86:0/4139086116 >> v2:172.21.15.144:3300/0 conn(0x7f67d00d4d70 msgr2=0x7f67d00cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:23.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.667+0000 7f67d4e00640 1 --2- 172.21.15.86:0/4139086116 >> v2:172.21.15.144:3300/0 conn(0x7f67d00d4d70 0x7f67d00cfc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:23.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.667+0000 7f67d4e00640 1 -- 172.21.15.86:0/4139086116 >> v1:172.21.15.144:6789/0 conn(0x7f67d015ecc0 legacy=0x7f67d00d4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:23.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.667+0000 7f67d4e00640 1 -- 172.21.15.86:0/4139086116 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67d00d1c80 con 0x7f67d015e8e0 2024-04-20T21:27:23.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.667+0000 7f67d5800640 1 --2- 172.21.15.86:0/4139086116 >> v2:172.21.15.144:3300/0 conn(0x7f67d00d4d70 0x7f67d00cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:27:23.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.668+0000 7f67d4e00640 1 --2- 172.21.15.86:0/4139086116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67d015e8e0 0x7f67d006d200 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f67d00d57d0 tx=0x7f67c002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96dd9be902d9b2a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:23.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.668+0000 7f67cfe00640 1 -- 172.21.15.86:0/4139086116 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c0035020 con 0x7f67d015e8e0 2024-04-20T21:27:23.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.668+0000 7f67cfe00640 1 -- 172.21.15.86:0/4139086116 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67c0031870 con 0x7f67d015e8e0 2024-04-20T21:27:23.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.668+0000 7f67cfe00640 1 -- 172.21.15.86:0/4139086116 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c0031c40 con 0x7f67d015e8e0 2024-04-20T21:27:23.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.669+0000 7f67d6200640 1 -- 172.21.15.86:0/4139086116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67d015e8e0 msgr2=0x7f67d006d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:23.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.669+0000 7f67d6200640 1 --2- 172.21.15.86:0/4139086116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67d015e8e0 0x7f67d006d200 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f67d00d57d0 tx=0x7f67c002b910 comp rx=0 tx=0).stop 2024-04-20T21:27:23.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.669+0000 7f67d6200640 1 -- 172.21.15.86:0/4139086116 shutdown_connections 2024-04-20T21:27:23.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.669+0000 7f67d6200640 1 --2- 172.21.15.86:0/4139086116 >> v2:172.21.15.144:3300/0 conn(0x7f67d00d4d70 0x7f67d00cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:23.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.669+0000 7f67d6200640 1 --2- 172.21.15.86:0/4139086116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67d015e8e0 0x7f67d006d200 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:23.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.669+0000 7f67d6200640 1 -- 172.21.15.86:0/4139086116 >> 172.21.15.86:0/4139086116 conn(0x7f67d009d9d0 msgr2=0x7f67d008bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:23.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.669+0000 7f67d6200640 1 -- 172.21.15.86:0/4139086116 shutdown_connections 2024-04-20T21:27:23.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.669+0000 7f67d6200640 1 -- 172.21.15.86:0/4139086116 wait complete. 2024-04-20T21:27:23.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.671+0000 7f67d6200640 1 Processor -- start 2024-04-20T21:27:23.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.671+0000 7f67d6200640 1 -- start start 2024-04-20T21:27:23.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.672+0000 7f67d6200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67d00d4d70 0x7f67d00cb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:23.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.672+0000 7f67d6200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67d015e8e0 0x7f67d0162a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:23.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.672+0000 7f67d6200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67d00ca600 0x7f67d00caa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:23.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.672+0000 7f67d6200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f67d006d8c0 con 0x7f67d015e8e0 2024-04-20T21:27:23.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.672+0000 7f67d6200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f67d006d740 con 0x7f67d00d4d70 2024-04-20T21:27:23.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.672+0000 7f67d6200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f67d00d01c0 con 0x7f67d00ca600 2024-04-20T21:27:23.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.672+0000 7f67c7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67d015e8e0 0x7f67d0162a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:23.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.672+0000 7f67c7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67d015e8e0 0x7f67d0162a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37506/0 (socket says 172.21.15.86:37506) 2024-04-20T21:27:23.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.672+0000 7f67c7e00640 1 -- 172.21.15.86:0/2685959560 learned_addr learned my addr 172.21.15.86:0/2685959560 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:23.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.673+0000 7f67d4e00640 1 --2- 172.21.15.86:0/2685959560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67d00d4d70 0x7f67d00cb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:23.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.673+0000 7f67d5800640 1 --2- 172.21.15.86:0/2685959560 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67d00ca600 0x7f67d00caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:23.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.673+0000 7f67c7e00640 1 -- 172.21.15.86:0/2685959560 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67d00ca600 msgr2=0x7f67d00caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:23.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.673+0000 7f67c7e00640 1 --2- 172.21.15.86:0/2685959560 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67d00ca600 0x7f67d00caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:23.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.673+0000 7f67c7e00640 1 -- 172.21.15.86:0/2685959560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67d00d4d70 msgr2=0x7f67d00cb7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:23.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.673+0000 7f67c7e00640 1 --2- 172.21.15.86:0/2685959560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67d00d4d70 0x7f67d00cb7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:23.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.673+0000 7f67c7e00640 1 -- 172.21.15.86:0/2685959560 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67d00b7490 con 0x7f67d015e8e0 2024-04-20T21:27:23.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.673+0000 7f67c7e00640 1 --2- 172.21.15.86:0/2685959560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67d015e8e0 0x7f67d0162a40 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f67b800b3e0 tx=0x7f67b800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:23.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.673+0000 7f67cd600640 1 -- 172.21.15.86:0/2685959560 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b8015020 con 0x7f67d015e8e0 2024-04-20T21:27:23.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.673+0000 7f67cd600640 1 -- 172.21.15.86:0/2685959560 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67b8011850 con 0x7f67d015e8e0 2024-04-20T21:27:23.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.673+0000 7f67d6200640 1 -- 172.21.15.86:0/2685959560 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67d00b7770 con 0x7f67d015e8e0 2024-04-20T21:27:23.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.673+0000 7f67cd600640 1 -- 172.21.15.86:0/2685959560 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b8011c40 con 0x7f67d015e8e0 2024-04-20T21:27:23.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.673+0000 7f67d6200640 1 -- 172.21.15.86:0/2685959560 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67d00b7d30 con 0x7f67d015e8e0 2024-04-20T21:27:23.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.676+0000 7f67cd600640 1 -- 172.21.15.86:0/2685959560 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f67b8018020 con 0x7f67d015e8e0 2024-04-20T21:27:23.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.677+0000 7f67cd600640 1 --2- 172.21.15.86:0/2685959560 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f67ac06c920 0x7f67ac06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:23.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.677+0000 7f67cd600640 1 -- 172.21.15.86:0/2685959560 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7f67b808bcd0 con 0x7f67d015e8e0 2024-04-20T21:27:23.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.677+0000 7f67d4e00640 1 --2- 172.21.15.86:0/2685959560 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f67ac06c920 0x7f67ac06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:23.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.677+0000 7f67da403740 1 Processor -- start 2024-04-20T21:27:23.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.677+0000 7f67da403740 1 -- start start 2024-04-20T21:27:23.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.678+0000 7f67d4e00640 1 --2- 172.21.15.86:0/2685959560 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f67ac06c920 0x7f67ac06ede0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f67c0036000 tx=0x7f67c003c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:23.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.678+0000 7f67da403740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ef5ecb3d0 0x563ef5ecb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:23.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.678+0000 7f67da403740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563ef5e9b3f0 0x563ef5ec8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:23.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.678+0000 7f67da403740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ef5e9ef50 0x563ef5ec8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:23.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.678+0000 7f67da403740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563ef5e07150 con 0x563ef5e9ef50 2024-04-20T21:27:23.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.678+0000 7f67da403740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563ef5dad370 con 0x563ef5ecb3d0 2024-04-20T21:27:23.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.678+0000 7f67da403740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x563ef5e03110 con 0x563ef5e9b3f0 2024-04-20T21:27:23.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.679+0000 7f67d5800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ef5e9ef50 0x563ef5ec8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:23.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.679+0000 7f67d5800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ef5e9ef50 0x563ef5ec8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37520/0 (socket says 172.21.15.86:37520) 2024-04-20T21:27:23.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.679+0000 7f67d5800640 1 -- 172.21.15.86:0/3362775739 learned_addr learned my addr 172.21.15.86:0/3362775739 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:23.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.679+0000 7f67c7e00640 1 --2- 172.21.15.86:0/3362775739 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563ef5e9b3f0 0x563ef5ec8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:23.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.679+0000 7f67d4e00640 1 --2- 172.21.15.86:0/3362775739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ef5ecb3d0 0x563ef5ecb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:23.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.679+0000 7f67d5800640 1 -- 172.21.15.86:0/3362775739 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563ef5e9b3f0 msgr2=0x563ef5ec8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:23.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.679+0000 7f67d5800640 1 --2- 172.21.15.86:0/3362775739 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563ef5e9b3f0 0x563ef5ec8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:23.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.679+0000 7f67d5800640 1 -- 172.21.15.86:0/3362775739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ef5ecb3d0 msgr2=0x563ef5ecb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:23.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.679+0000 7f67d5800640 1 --2- 172.21.15.86:0/3362775739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ef5ecb3d0 0x563ef5ecb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:23.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.679+0000 7f67d5800640 1 -- 172.21.15.86:0/3362775739 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ef5be37d0 con 0x563ef5e9ef50 2024-04-20T21:27:23.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.679+0000 7f67d4e00640 1 --2- 172.21.15.86:0/3362775739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ef5ecb3d0 0x563ef5ecb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:27:23.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.679+0000 7f67c7e00640 1 --2- 172.21.15.86:0/3362775739 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563ef5e9b3f0 0x563ef5ec8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:27:23.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.679+0000 7f67d5800640 1 --2- 172.21.15.86:0/3362775739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ef5e9ef50 0x563ef5ec8b80 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f67c800d430 tx=0x7f67c800d8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aca760ad8ee1d50 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:23.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.680+0000 7f67c5600640 1 -- 172.21.15.86:0/3362775739 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c801a070 con 0x563ef5e9ef50 2024-04-20T21:27:23.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.680+0000 7f67c5600640 1 -- 172.21.15.86:0/3362775739 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67c800b6e0 con 0x563ef5e9ef50 2024-04-20T21:27:23.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.680+0000 7f67c5600640 1 -- 172.21.15.86:0/3362775739 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c800bab0 con 0x563ef5e9ef50 2024-04-20T21:27:23.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.680+0000 7f67da403740 1 -- 172.21.15.86:0/3362775739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ef5e9ef50 msgr2=0x563ef5ec8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:23.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.680+0000 7f67da403740 1 --2- 172.21.15.86:0/3362775739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ef5e9ef50 0x563ef5ec8b80 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f67c800d430 tx=0x7f67c800d8f0 comp rx=0 tx=0).stop 2024-04-20T21:27:23.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.680+0000 7f67da403740 1 -- 172.21.15.86:0/3362775739 shutdown_connections 2024-04-20T21:27:23.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.680+0000 7f67da403740 1 --2- 172.21.15.86:0/3362775739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ef5e9ef50 0x563ef5ec8b80 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:23.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.680+0000 7f67da403740 1 --2- 172.21.15.86:0/3362775739 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563ef5e9b3f0 0x563ef5ec8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:23.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.680+0000 7f67da403740 1 --2- 172.21.15.86:0/3362775739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ef5ecb3d0 0x563ef5ecb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:23.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.680+0000 7f67da403740 1 -- 172.21.15.86:0/3362775739 >> 172.21.15.86:0/3362775739 conn(0x563ef5e9d660 msgr2=0x563ef5ee61e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:23.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.681+0000 7f67da403740 1 -- 172.21.15.86:0/3362775739 shutdown_connections 2024-04-20T21:27:23.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.681+0000 7f67da403740 1 -- 172.21.15.86:0/3362775739 wait complete. 2024-04-20T21:27:23.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.681+0000 7f67da403740 1 -- start start 2024-04-20T21:27:23.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.681+0000 7f67da403740 1 Processor -- start 2024-04-20T21:27:23.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.682+0000 7f67da403740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563ef5e9ef50 0x563ef5ecbd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:23.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.682+0000 7f67da403740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ef5ecb3d0 0x563ef5eb6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:23.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.683+0000 7f67da403740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ef5ec11c0 0x563ef5ecec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:23.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.683+0000 7f67da403740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563ef5ea0100 con 0x563ef5ec11c0 2024-04-20T21:27:23.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.683+0000 7f67da403740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x563ef5dc0740 con 0x563ef5ecb3d0 2024-04-20T21:27:23.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.683+0000 7f67da403740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x563ef5e9f4f0 con 0x563ef5e9ef50 2024-04-20T21:27:23.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.683+0000 7f67da403740 10 client.0 _ll_register_callbacks cb 0x563ef5dbbb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:27:23.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.683+0000 7f67d5800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ef5ec11c0 0x563ef5ecec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:23.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.683+0000 7f67d5800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ef5ec11c0 0x563ef5ecec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37530/0 (socket says 172.21.15.86:37530) 2024-04-20T21:27:23.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.683+0000 7f67d5800640 1 -- 172.21.15.86:0/3012181112 learned_addr learned my addr 172.21.15.86:0/3012181112 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:23.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.683+0000 7f67d4e00640 1 --2- 172.21.15.86:0/3012181112 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563ef5e9ef50 0x563ef5ecbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:23.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.683+0000 7f67d5800640 1 -- 172.21.15.86:0/3012181112 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563ef5e9ef50 msgr2=0x563ef5ecbd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:23.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.683+0000 7f67d5800640 1 --2- 172.21.15.86:0/3012181112 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563ef5e9ef50 0x563ef5ecbd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:23.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.683+0000 7f67d5800640 1 -- 172.21.15.86:0/3012181112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ef5ecb3d0 msgr2=0x563ef5eb6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:23.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.683+0000 7f67d5800640 1 --2- 172.21.15.86:0/3012181112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ef5ecb3d0 0x563ef5eb6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:23.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.683+0000 7f67d5800640 1 -- 172.21.15.86:0/3012181112 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ef5b193f0 con 0x563ef5ec11c0 2024-04-20T21:27:23.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.684+0000 7f67d5800640 1 --2- 172.21.15.86:0/3012181112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ef5ec11c0 0x563ef5ecec90 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f67c8018040 tx=0x7f67c8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22d4bd15ee414fbd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:23.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.684+0000 7f67be000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:27:23.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.684+0000 7f67be000640 1 -- 172.21.15.86:0/3012181112 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c8024070 con 0x563ef5ec11c0 2024-04-20T21:27:23.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.684+0000 7f67be000640 1 -- 172.21.15.86:0/3012181112 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7f67c801f660 con 0x563ef5ec11c0 2024-04-20T21:27:23.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.684+0000 7f67da403740 1 -- 172.21.15.86:0/3012181112 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67c800b6e0 con 0x563ef5ec11c0 2024-04-20T21:27:23.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.684+0000 7f67be000640 1 -- 172.21.15.86:0/3012181112 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f67c802f050 con 0x563ef5ec11c0 2024-04-20T21:27:23.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.684+0000 7f67da403740 10 client.8247 fetch_fsmap learned FSMap version 73 2024-04-20T21:27:23.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.684+0000 7f67da403740 1 -- 172.21.15.86:0/3012181112 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) -- 0x563ef5ad90f0 con 0x563ef5ec11c0 2024-04-20T21:27:23.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.684+0000 7f67be000640 1 -- 172.21.15.86:0/3012181112 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67c800b6e0 con 0x563ef5ec11c0 2024-04-20T21:27:23.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.685+0000 7f67be000640 1 -- 172.21.15.86:0/3012181112 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 73) ==== 3495+0+0 (secure 0 0 0) 0x7f67c8032020 con 0x563ef5ec11c0 2024-04-20T21:27:23.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.685+0000 7f67da403740 10 client.8247 fetch_fsmap finished waiting for FSMap version 73 2024-04-20T21:27:23.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.685+0000 7f67da403740 10 client.8247 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-04-20T21:27:23.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.685+0000 7f67da403740 20 client.8247 populate_metadata read hostname 'smithi086' 2024-04-20T21:27:23.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.685+0000 7f67da403740 1 --2- 172.21.15.86:0/3012181112 >> [v2:172.21.15.144:6832/2351381466,v1:172.21.15.144:6834/2351381466] conn(0x563ef5eccd80 0x563ef5ecd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:23.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.685+0000 7f67da403740 4 client.8247 mds_command: new command op to 7518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:27:23.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.685+0000 7f67da403740 1 -- 172.21.15.86:0/3012181112 --> [v2:172.21.15.144:6832/2351381466,v1:172.21.15.144:6834/2351381466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ef5e14b70 con 0x563ef5eccd80 2024-04-20T21:27:23.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.685+0000 7f67d4e00640 1 --2- 172.21.15.86:0/3012181112 >> [v2:172.21.15.144:6832/2351381466,v1:172.21.15.144:6834/2351381466] conn(0x563ef5eccd80 0x563ef5ecd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:23.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.687+0000 7f67d4e00640 1 --2- 172.21.15.86:0/3012181112 >> [v2:172.21.15.144:6832/2351381466,v1:172.21.15.144:6834/2351381466] conn(0x563ef5eccd80 0x563ef5ecd1d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=deaa42f27d7ce67c server_cookie=da8b631c8005ac66 in_seq=0 out_seq=0 2024-04-20T21:27:23.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.687+0000 7f67be000640 10 client.8247 ms_handle_connect on v2:172.21.15.144:6832/2351381466 2024-04-20T21:27:23.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.688+0000 7f67be000640 1 -- 172.21.15.86:0/3012181112 <== mds.0 v2:172.21.15.144:6832/2351381466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563ef5e14b70 con 0x563ef5eccd80 2024-04-20T21:27:23.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.688+0000 7f67be000640 10 client.8247 handle_command_reply: tid=0 2024-04-20T21:27:23.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.689+0000 7f67da403740 1 client.8247 shutdown 2024-04-20T21:27:23.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.689+0000 7f67da403740 1 -- 172.21.15.86:0/3012181112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ef5ec11c0 msgr2=0x563ef5ecec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:23.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.689+0000 7f67da403740 1 --2- 172.21.15.86:0/3012181112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ef5ec11c0 0x563ef5ecec90 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f67c8018040 tx=0x7f67c8004ab0 comp rx=0 tx=0).stop 2024-04-20T21:27:23.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.689+0000 7f67da403740 1 -- 172.21.15.86:0/3012181112 shutdown_connections 2024-04-20T21:27:23.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.689+0000 7f67da403740 1 --2- 172.21.15.86:0/3012181112 >> [v2:172.21.15.144:6832/2351381466,v1:172.21.15.144:6834/2351381466] conn(0x563ef5eccd80 0x563ef5ecd1d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:23.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.689+0000 7f67da403740 1 --2- 172.21.15.86:0/3012181112 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ef5ec11c0 0x563ef5ecec90 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:23.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.689+0000 7f67da403740 1 --2- 172.21.15.86:0/3012181112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ef5ecb3d0 0x563ef5eb6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:23.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.689+0000 7f67da403740 1 --2- 172.21.15.86:0/3012181112 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563ef5e9ef50 0x563ef5ecbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:23.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.689+0000 7f67da403740 1 -- 172.21.15.86:0/3012181112 >> 172.21.15.86:0/3012181112 conn(0x563ef5e9b3f0 msgr2=0x563ef5ee7780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:23.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.689+0000 7f67be000640 0 client.8247 ms_handle_reset on v2:172.21.15.144:6832/2351381466 2024-04-20T21:27:23.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.689+0000 7f67da403740 1 -- 172.21.15.86:0/3012181112 shutdown_connections 2024-04-20T21:27:23.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.689+0000 7f67da403740 1 -- 172.21.15.86:0/3012181112 wait complete. 2024-04-20T21:27:23.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.689+0000 7f67da403740 20 client.8247 trim_cache size 0 max 16384 2024-04-20T21:27:23.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.709+0000 7f67da403740 1 Processor -- start 2024-04-20T21:27:23.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.709+0000 7f67da403740 1 -- start start 2024-04-20T21:27:23.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.710+0000 7f67da403740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ef5e9ef50 0x563ef5e9b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:23.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.710+0000 7f67da403740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563ef5ec11c0 0x563ef5ece130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:23.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.710+0000 7f67da403740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ef5ecb3d0 0x563ef5ee9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:23.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.710+0000 7f67da403740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f67d0200eb0 con 0x563ef5ecb3d0 2024-04-20T21:27:23.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.710+0000 7f67da403740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f67d0161010 con 0x563ef5e9ef50 2024-04-20T21:27:23.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.710+0000 7f67da403740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f67d00b8dc0 con 0x563ef5ec11c0 2024-04-20T21:27:23.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.710+0000 7f67d5800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ef5ecb3d0 0x563ef5ee9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:23.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.710+0000 7f67d5800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ef5ecb3d0 0x563ef5ee9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37542/0 (socket says 172.21.15.86:37542) 2024-04-20T21:27:23.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.710+0000 7f67d5800640 1 -- 172.21.15.86:0/2858075094 learned_addr learned my addr 172.21.15.86:0/2858075094 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:23.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.710+0000 7f67c7e00640 1 --2- 172.21.15.86:0/2858075094 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563ef5ec11c0 0x563ef5ece130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:23.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.710+0000 7f67d4e00640 1 --2- 172.21.15.86:0/2858075094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ef5e9ef50 0x563ef5e9b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:23.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.711+0000 7f67d5800640 1 -- 172.21.15.86:0/2858075094 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563ef5ec11c0 msgr2=0x563ef5ece130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:23.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.711+0000 7f67d5800640 1 --2- 172.21.15.86:0/2858075094 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563ef5ec11c0 0x563ef5ece130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:23.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.711+0000 7f67d5800640 1 -- 172.21.15.86:0/2858075094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ef5e9ef50 msgr2=0x563ef5e9b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:23.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.711+0000 7f67d5800640 1 --2- 172.21.15.86:0/2858075094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ef5e9ef50 0x563ef5e9b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:23.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.711+0000 7f67d5800640 1 -- 172.21.15.86:0/2858075094 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ef5e9cf00 con 0x563ef5ecb3d0 2024-04-20T21:27:23.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.711+0000 7f67d5800640 1 --2- 172.21.15.86:0/2858075094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ef5ecb3d0 0x563ef5ee9ca0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f67c8000f80 tx=0x7f67c8021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=957f292bc8d5cf9c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:23.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.711+0000 7f67c4c00640 1 -- 172.21.15.86:0/2858075094 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c8024040 con 0x563ef5ecb3d0 2024-04-20T21:27:23.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.711+0000 7f67c4c00640 1 -- 172.21.15.86:0/2858075094 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67c801f660 con 0x563ef5ecb3d0 2024-04-20T21:27:23.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.711+0000 7f67c4c00640 1 -- 172.21.15.86:0/2858075094 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c801a040 con 0x563ef5ecb3d0 2024-04-20T21:27:23.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.712+0000 7f67da403740 1 -- 172.21.15.86:0/2858075094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ef5ecb3d0 msgr2=0x563ef5ee9ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:23.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.712+0000 7f67da403740 1 --2- 172.21.15.86:0/2858075094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ef5ecb3d0 0x563ef5ee9ca0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f67c8000f80 tx=0x7f67c8021000 comp rx=0 tx=0).stop 2024-04-20T21:27:23.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.712+0000 7f67da403740 1 -- 172.21.15.86:0/2858075094 shutdown_connections 2024-04-20T21:27:23.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.712+0000 7f67da403740 1 --2- 172.21.15.86:0/2858075094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ef5ecb3d0 0x563ef5ee9ca0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:23.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.712+0000 7f67da403740 1 --2- 172.21.15.86:0/2858075094 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563ef5ec11c0 0x563ef5ece130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:23.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.712+0000 7f67da403740 1 --2- 172.21.15.86:0/2858075094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ef5e9ef50 0x563ef5e9b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:23.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.712+0000 7f67da403740 1 -- 172.21.15.86:0/2858075094 >> 172.21.15.86:0/2858075094 conn(0x563ef5e9b3f0 msgr2=0x563ef5e96aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:23.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.712+0000 7f67da403740 1 -- 172.21.15.86:0/2858075094 shutdown_connections 2024-04-20T21:27:23.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.712+0000 7f67da403740 1 -- 172.21.15.86:0/2858075094 wait complete. 2024-04-20T21:27:23.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.713+0000 7f67da403740 1 -- start start 2024-04-20T21:27:23.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.713+0000 7f67da403740 1 Processor -- start 2024-04-20T21:27:23.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.714+0000 7f67da403740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ef5e9ef50 0x563ef5e96aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:23.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.714+0000 7f67da403740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ef5ec11c0 0x563ef5ec21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:23.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.714+0000 7f67da403740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563ef5ecb3d0 0x563ef5ec26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:23.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.714+0000 7f67da403740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563ef5d88ac0 con 0x563ef5ec11c0 2024-04-20T21:27:23.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.714+0000 7f67da403740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f67d01609c0 con 0x563ef5e9ef50 2024-04-20T21:27:23.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.714+0000 7f67da403740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x563ef5ec0510 con 0x563ef5ecb3d0 2024-04-20T21:27:23.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.714+0000 7f67da403740 10 client.0 _ll_register_callbacks cb 0x7f67c8018070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:27:23.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.714+0000 7f67c7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ef5ec11c0 0x563ef5ec21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:23.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.714+0000 7f67c7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ef5ec11c0 0x563ef5ec21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37548/0 (socket says 172.21.15.86:37548) 2024-04-20T21:27:23.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.714+0000 7f67c7e00640 1 -- 172.21.15.86:0/2989126695 learned_addr learned my addr 172.21.15.86:0/2989126695 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:23.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.715+0000 7f67c7e00640 1 -- 172.21.15.86:0/2989126695 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563ef5ecb3d0 msgr2=0x563ef5ec26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:23.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.715+0000 7f67d5800640 1 --2- 172.21.15.86:0/2989126695 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563ef5ecb3d0 0x563ef5ec26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:23.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.715+0000 7f67c7e00640 1 --2- 172.21.15.86:0/2989126695 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563ef5ecb3d0 0x563ef5ec26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:23.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.715+0000 7f67d4e00640 1 --2- 172.21.15.86:0/2989126695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ef5e9ef50 0x563ef5e96aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:23.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.715+0000 7f67c7e00640 1 -- 172.21.15.86:0/2989126695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ef5e9ef50 msgr2=0x563ef5e96aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:23.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.715+0000 7f67c7e00640 1 --2- 172.21.15.86:0/2989126695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ef5e9ef50 0x563ef5e96aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:23.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.715+0000 7f67c7e00640 1 -- 172.21.15.86:0/2989126695 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ef5eb3b20 con 0x563ef5ec11c0 2024-04-20T21:27:23.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.715+0000 7f67c7e00640 1 --2- 172.21.15.86:0/2989126695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ef5ec11c0 0x563ef5ec21b0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f67b8059790 tx=0x7f67b805a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20fd35b29bac823c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:23.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.715+0000 7f67be000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:27:23.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.715+0000 7f67be000640 1 -- 172.21.15.86:0/2989126695 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b8087020 con 0x563ef5ec11c0 2024-04-20T21:27:23.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.715+0000 7f67be000640 1 -- 172.21.15.86:0/2989126695 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7f67b8055e70 con 0x563ef5ec11c0 2024-04-20T21:27:23.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.715+0000 7f67da403740 1 -- 172.21.15.86:0/2989126695 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67c801f660 con 0x563ef5ec11c0 2024-04-20T21:27:23.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.716+0000 7f67be000640 1 -- 172.21.15.86:0/2989126695 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f67b805ab30 con 0x563ef5ec11c0 2024-04-20T21:27:23.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.716+0000 7f67da403740 10 client.8253 fetch_fsmap learned FSMap version 73 2024-04-20T21:27:23.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.716+0000 7f67da403740 1 -- 172.21.15.86:0/2989126695 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) -- 0x563ef5eb43c0 con 0x563ef5ec11c0 2024-04-20T21:27:23.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.716+0000 7f67be000640 1 -- 172.21.15.86:0/2989126695 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67c801f660 con 0x563ef5ec11c0 2024-04-20T21:27:23.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.716+0000 7f67be000640 1 -- 172.21.15.86:0/2989126695 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 73) ==== 3495+0+0 (secure 0 0 0) 0x7f67b805acd0 con 0x563ef5ec11c0 2024-04-20T21:27:23.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.716+0000 7f67da403740 10 client.8253 fetch_fsmap finished waiting for FSMap version 73 2024-04-20T21:27:23.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.716+0000 7f67da403740 10 client.8253 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-04-20T21:27:23.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.716+0000 7f67da403740 20 client.8253 populate_metadata read hostname 'smithi086' 2024-04-20T21:27:23.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.716+0000 7f67da403740 1 --2- 172.21.15.86:0/2989126695 >> [v2:172.21.15.144:6832/2351381466,v1:172.21.15.144:6834/2351381466] conn(0x563ef5eb4c00 0x563ef5ead1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:23.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.716+0000 7f67da403740 4 client.8253 mds_command: new command op to 7518 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:27:23.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.716+0000 7f67da403740 1 -- 172.21.15.86:0/2989126695 --> [v2:172.21.15.144:6832/2351381466,v1:172.21.15.144:6834/2351381466] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563ef5e0e9e0 con 0x563ef5eb4c00 2024-04-20T21:27:23.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.717+0000 7f67d4e00640 1 --2- 172.21.15.86:0/2989126695 >> [v2:172.21.15.144:6832/2351381466,v1:172.21.15.144:6834/2351381466] conn(0x563ef5eb4c00 0x563ef5ead1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:23.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.717+0000 7f67d4e00640 1 --2- 172.21.15.86:0/2989126695 >> [v2:172.21.15.144:6832/2351381466,v1:172.21.15.144:6834/2351381466] conn(0x563ef5eb4c00 0x563ef5ead1a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a7cc41ec06d4d41 server_cookie=b8d62e2007868009 in_seq=0 out_seq=0 2024-04-20T21:27:23.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.718+0000 7f67be000640 10 client.8253 ms_handle_connect on v2:172.21.15.144:6832/2351381466 2024-04-20T21:27:23.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.718+0000 7f67be000640 1 -- 172.21.15.86:0/2989126695 <== mds.0 v2:172.21.15.144:6832/2351381466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563ef5e0e9e0 con 0x563ef5eb4c00 2024-04-20T21:27:23.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.718+0000 7f67be000640 10 client.8253 handle_command_reply: tid=0 2024-04-20T21:27:23.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.718+0000 7f67da403740 1 client.8253 shutdown 2024-04-20T21:27:23.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.718+0000 7f67da403740 1 -- 172.21.15.86:0/2989126695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ef5ec11c0 msgr2=0x563ef5ec21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:23.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.718+0000 7f67da403740 1 --2- 172.21.15.86:0/2989126695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ef5ec11c0 0x563ef5ec21b0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f67b8059790 tx=0x7f67b805a040 comp rx=0 tx=0).stop 2024-04-20T21:27:23.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.718+0000 7f67da403740 1 -- 172.21.15.86:0/2989126695 shutdown_connections 2024-04-20T21:27:23.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.718+0000 7f67da403740 1 --2- 172.21.15.86:0/2989126695 >> [v2:172.21.15.144:6832/2351381466,v1:172.21.15.144:6834/2351381466] conn(0x563ef5eb4c00 0x563ef5ead1a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:23.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.718+0000 7f67da403740 1 --2- 172.21.15.86:0/2989126695 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x563ef5ecb3d0 0x563ef5ec26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:23.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.718+0000 7f67da403740 1 --2- 172.21.15.86:0/2989126695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563ef5ec11c0 0x563ef5ec21b0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:23.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.718+0000 7f67da403740 1 --2- 172.21.15.86:0/2989126695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x563ef5e9ef50 0x563ef5e96aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:23.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.718+0000 7f67be000640 0 client.8253 ms_handle_reset on v2:172.21.15.144:6832/2351381466 2024-04-20T21:27:23.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.718+0000 7f67da403740 1 -- 172.21.15.86:0/2989126695 >> 172.21.15.86:0/2989126695 conn(0x563ef5e9b3f0 msgr2=0x563ef5ec68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:23.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.718+0000 7f67da403740 1 -- 172.21.15.86:0/2989126695 shutdown_connections 2024-04-20T21:27:23.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.718+0000 7f67da403740 1 -- 172.21.15.86:0/2989126695 wait complete. 2024-04-20T21:27:23.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.719+0000 7f67da403740 20 client.8253 trim_cache size 0 max 16384 2024-04-20T21:27:23.721 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:27:23.721 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-20T21:27:23.721 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-20T21:27:23.721 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:27:23.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.719+0000 7f67d6200640 1 -- 172.21.15.86:0/2685959560 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f67ac06c920 msgr2=0x7f67ac06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:23.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.719+0000 7f67d6200640 1 --2- 172.21.15.86:0/2685959560 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f67ac06c920 0x7f67ac06ede0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f67c0036000 tx=0x7f67c003c040 comp rx=0 tx=0).stop 2024-04-20T21:27:23.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.719+0000 7f67d6200640 1 -- 172.21.15.86:0/2685959560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67d015e8e0 msgr2=0x7f67d0162a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:23.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.719+0000 7f67d6200640 1 --2- 172.21.15.86:0/2685959560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67d015e8e0 0x7f67d0162a40 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f67b800b3e0 tx=0x7f67b800b8a0 comp rx=0 tx=0).stop 2024-04-20T21:27:23.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.719+0000 7f67d6200640 1 -- 172.21.15.86:0/2685959560 shutdown_connections 2024-04-20T21:27:23.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.719+0000 7f67d6200640 1 --2- 172.21.15.86:0/2685959560 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f67ac06c920 0x7f67ac06ede0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:23.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.719+0000 7f67d6200640 1 --2- 172.21.15.86:0/2685959560 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67d00ca600 0x7f67d00caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:23.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.719+0000 7f67d6200640 1 --2- 172.21.15.86:0/2685959560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67d015e8e0 0x7f67d0162a40 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:23.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.719+0000 7f67d6200640 1 --2- 172.21.15.86:0/2685959560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67d00d4d70 0x7f67d00cb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:23.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.719+0000 7f67d6200640 1 -- 172.21.15.86:0/2685959560 >> 172.21.15.86:0/2685959560 conn(0x7f67d009d9d0 msgr2=0x7f67d00d4660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:23.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.719+0000 7f67d6200640 1 -- 172.21.15.86:0/2685959560 shutdown_connections 2024-04-20T21:27:23.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.719+0000 7f67d6200640 1 -- 172.21.15.86:0/2685959560 wait complete. 2024-04-20T21:27:23.737 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T21:27:23.738 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:27:23.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.930+0000 7f3139200640 1 Processor -- start 2024-04-20T21:27:23.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.930+0000 7f3139200640 1 -- start start 2024-04-20T21:27:23.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.932+0000 7f3139200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f313415c010 0x7f313415c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:23.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.932+0000 7f3139200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f31340af7b0 0x7f31340adda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:23.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.932+0000 7f3139200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31340ae710 con 0x7f313406f4c0 2024-04-20T21:27:23.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.932+0000 7f3139200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f31340af320 con 0x7f31340af7b0 2024-04-20T21:27:23.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.932+0000 7f3139200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f313415c930 con 0x7f313415c010 2024-04-20T21:27:23.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.932+0000 7f3132a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f31340af7b0 0x7f31340adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:23.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.932+0000 7f3132a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f31340af7b0 0x7f31340adda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37560/0 (socket says 172.21.15.86:37560) 2024-04-20T21:27:23.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.932+0000 7f3132a00640 1 -- 172.21.15.86:0/2479188822 learned_addr learned my addr 172.21.15.86:0/2479188822 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:23.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.932+0000 7f3132a00640 1 -- 172.21.15.86:0/2479188822 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f313415c010 msgr2=0x7f313415c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:23.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.932+0000 7f3132a00640 1 --2- 172.21.15.86:0/2479188822 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f313415c010 0x7f313415c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:23.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.932+0000 7f3132a00640 1 -- 172.21.15.86:0/2479188822 >> v1:172.21.15.144:6789/0 conn(0x7f313406f4c0 legacy=0x7f313415ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:23.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.932+0000 7f3132a00640 1 -- 172.21.15.86:0/2479188822 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31340ae9f0 con 0x7f31340af7b0 2024-04-20T21:27:23.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.933+0000 7f3132a00640 1 --2- 172.21.15.86:0/2479188822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31340af7b0 0x7f31340adda0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f311c009870 tx=0x7f311c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85e83ca78c2d93d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:23.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.934+0000 7f3132000640 1 -- 172.21.15.86:0/2479188822 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c035020 con 0x7f31340af7b0 2024-04-20T21:27:23.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.934+0000 7f3132000640 1 -- 172.21.15.86:0/2479188822 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f311c031640 con 0x7f31340af7b0 2024-04-20T21:27:23.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.934+0000 7f3132000640 1 -- 172.21.15.86:0/2479188822 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c02d040 con 0x7f31340af7b0 2024-04-20T21:27:23.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.934+0000 7f3139200640 1 -- 172.21.15.86:0/2479188822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31340af7b0 msgr2=0x7f31340adda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:23.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.934+0000 7f3139200640 1 --2- 172.21.15.86:0/2479188822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31340af7b0 0x7f31340adda0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f311c009870 tx=0x7f311c02b8f0 comp rx=0 tx=0).stop 2024-04-20T21:27:23.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.934+0000 7f3139200640 1 -- 172.21.15.86:0/2479188822 shutdown_connections 2024-04-20T21:27:23.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.934+0000 7f3139200640 1 --2- 172.21.15.86:0/2479188822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31340af7b0 0x7f31340adda0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:23.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.935+0000 7f3139200640 1 --2- 172.21.15.86:0/2479188822 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f313415c010 0x7f313415c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:23.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.935+0000 7f3139200640 1 -- 172.21.15.86:0/2479188822 >> 172.21.15.86:0/2479188822 conn(0x7f31340d0170 msgr2=0x7f31340d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:23.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.935+0000 7f3139200640 1 -- 172.21.15.86:0/2479188822 shutdown_connections 2024-04-20T21:27:23.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.935+0000 7f3139200640 1 -- 172.21.15.86:0/2479188822 wait complete. 2024-04-20T21:27:23.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.936+0000 7f3139200640 1 Processor -- start 2024-04-20T21:27:23.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.936+0000 7f3139200640 1 -- start start 2024-04-20T21:27:23.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.937+0000 7f3139200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f313406f4c0 0x7f313415b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:23.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.937+0000 7f3139200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31340af7b0 0x7f313409a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:23.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.937+0000 7f3139200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f313415c010 0x7f3134154eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:23.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.937+0000 7f3139200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f31340ae2e0 con 0x7f313406f4c0 2024-04-20T21:27:23.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.937+0000 7f3139200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f31340aefb0 con 0x7f313415c010 2024-04-20T21:27:23.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.937+0000 7f3139200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f31340ae460 con 0x7f31340af7b0 2024-04-20T21:27:23.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.938+0000 7f3133400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f313406f4c0 0x7f313415b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:23.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.938+0000 7f3133400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f313406f4c0 0x7f313415b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37562/0 (socket says 172.21.15.86:37562) 2024-04-20T21:27:23.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.938+0000 7f3133400640 1 -- 172.21.15.86:0/2353648417 learned_addr learned my addr 172.21.15.86:0/2353648417 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:23.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.938+0000 7f3132a00640 1 --2- 172.21.15.86:0/2353648417 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31340af7b0 0x7f313409a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:23.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.938+0000 7f3133e00640 1 --2- 172.21.15.86:0/2353648417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f313415c010 0x7f3134154eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:23.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.938+0000 7f3133400640 1 -- 172.21.15.86:0/2353648417 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31340af7b0 msgr2=0x7f313409a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:23.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.938+0000 7f3133400640 1 --2- 172.21.15.86:0/2353648417 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31340af7b0 0x7f313409a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:23.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.938+0000 7f3133400640 1 -- 172.21.15.86:0/2353648417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f313415c010 msgr2=0x7f3134154eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:23.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.938+0000 7f3133400640 1 --2- 172.21.15.86:0/2353648417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f313415c010 0x7f3134154eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:23.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.938+0000 7f3133400640 1 -- 172.21.15.86:0/2353648417 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f313415b0b0 con 0x7f313406f4c0 2024-04-20T21:27:23.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.939+0000 7f3133400640 1 --2- 172.21.15.86:0/2353648417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f313406f4c0 0x7f313415b4f0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f312400b440 tx=0x7f312400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:23.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.939+0000 7f312b400640 1 -- 172.21.15.86:0/2353648417 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3124015020 con 0x7f313406f4c0 2024-04-20T21:27:23.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.939+0000 7f312b400640 1 -- 172.21.15.86:0/2353648417 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3124011890 con 0x7f313406f4c0 2024-04-20T21:27:23.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.939+0000 7f312b400640 1 -- 172.21.15.86:0/2353648417 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3124011c80 con 0x7f313406f4c0 2024-04-20T21:27:23.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.939+0000 7f3139200640 1 -- 172.21.15.86:0/2353648417 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f313409e1b0 con 0x7f313406f4c0 2024-04-20T21:27:23.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.939+0000 7f3139200640 1 -- 172.21.15.86:0/2353648417 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f313409ded0 con 0x7f313406f4c0 2024-04-20T21:27:23.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.940+0000 7f3139200640 1 -- 172.21.15.86:0/2353648417 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f313404fcf0 con 0x7f313406f4c0 2024-04-20T21:27:23.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.941+0000 7f312b400640 1 -- 172.21.15.86:0/2353648417 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3124018020 con 0x7f313406f4c0 2024-04-20T21:27:23.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.942+0000 7f312b400640 1 --2- 172.21.15.86:0/2353648417 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f311006c920 0x7f311006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:23.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.942+0000 7f312b400640 1 -- 172.21.15.86:0/2353648417 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7f312408bd10 con 0x7f313406f4c0 2024-04-20T21:27:23.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.943+0000 7f3132a00640 1 --2- 172.21.15.86:0/2353648417 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f311006c920 0x7f311006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:23.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.943+0000 7f3132a00640 1 --2- 172.21.15.86:0/2353648417 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f311006c920 0x7f311006ede0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f311c036000 tx=0x7f311c03c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:23.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:23.947+0000 7f312b400640 1 -- 172.21.15.86:0/2353648417 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3124091020 con 0x7f313406f4c0 2024-04-20T21:27:24.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.187+0000 7f3139200640 1 -- 172.21.15.86:0/2353648417 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f31340737d0 con 0x7f313406f4c0 2024-04-20T21:27:24.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.188+0000 7f312b400640 1 -- 172.21.15.86:0/2353648417 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5209 (secure 0 0 0) 0x7f312401f020 con 0x7f313406f4c0 2024-04-20T21:27:24.188 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:27:24.189 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":73,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5887,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2433619638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2433619638},{"type":"v1","addr":"172.21.15.144:6835","nonce":2433619638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68},{"gid":7506,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/4013863257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4013863257},{"type":"v1","addr":"172.21.15.86:6836","nonce":4013863257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":7509,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2044883073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2044883073},{"type":"v1","addr":"172.21.15.86:6837","nonce":2044883073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:24:58.299257+0000","modified":"2024-04-20T21:26:48.911892+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7518},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7518":{"gid":7518,"name":"d","rank":0,"incarnation":70,"state":"up:active","state_seq":32,"addr":"172.21.15.144:6834/2351381466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2351381466},{"type":"v1","addr":"172.21.15.144:6834","nonce":2351381466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7518,"qdb_cluster":[7518]},"id":6}]} 2024-04-20T21:27:24.189 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 73 2024-04-20T21:27:24.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.191+0000 7f3139200640 1 -- 172.21.15.86:0/2353648417 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f311006c920 msgr2=0x7f311006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:24.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.191+0000 7f3139200640 1 --2- 172.21.15.86:0/2353648417 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f311006c920 0x7f311006ede0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f311c036000 tx=0x7f311c03c040 comp rx=0 tx=0).stop 2024-04-20T21:27:24.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.191+0000 7f3139200640 1 -- 172.21.15.86:0/2353648417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f313406f4c0 msgr2=0x7f313415b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:24.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.191+0000 7f3139200640 1 --2- 172.21.15.86:0/2353648417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f313406f4c0 0x7f313415b4f0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f312400b440 tx=0x7f312400b900 comp rx=0 tx=0).stop 2024-04-20T21:27:24.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.191+0000 7f3139200640 1 -- 172.21.15.86:0/2353648417 shutdown_connections 2024-04-20T21:27:24.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.191+0000 7f3139200640 1 --2- 172.21.15.86:0/2353648417 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f311006c920 0x7f311006ede0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:24.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.191+0000 7f3139200640 1 --2- 172.21.15.86:0/2353648417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f313415c010 0x7f3134154eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:24.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.191+0000 7f3139200640 1 --2- 172.21.15.86:0/2353648417 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f31340af7b0 0x7f313409a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:24.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.191+0000 7f3139200640 1 --2- 172.21.15.86:0/2353648417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f313406f4c0 0x7f313415b4f0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:24.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.191+0000 7f3139200640 1 -- 172.21.15.86:0/2353648417 >> 172.21.15.86:0/2353648417 conn(0x7f31340d0170 msgr2=0x7f3134071b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:24.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.191+0000 7f3139200640 1 -- 172.21.15.86:0/2353648417 shutdown_connections 2024-04-20T21:27:24.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.191+0000 7f3139200640 1 -- 172.21.15.86:0/2353648417 wait complete. 2024-04-20T21:27:24.210 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d damage ls --format=json-pretty 2024-04-20T21:27:24.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.410+0000 7f2c8ec00640 1 Processor -- start 2024-04-20T21:27:24.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.410+0000 7f2c8ec00640 1 -- start start 2024-04-20T21:27:24.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.411+0000 7f2c8ec00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f2c880cf500 0x7f2c880b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:24.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.411+0000 7f2c8ec00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c88156110 con 0x7f2c88155d30 2024-04-20T21:27:24.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.411+0000 7f2c8ec00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c880ce460 con 0x7f2c8806c090 2024-04-20T21:27:24.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.411+0000 7f2c8ec00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f2c880ce740 con 0x7f2c880cf500 2024-04-20T21:27:24.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.412+0000 7f2c87e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f2c88155d30 0x7f2c8806b980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:59402/0 (socket says 172.21.15.86:59402) 2024-04-20T21:27:24.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.412+0000 7f2c87e00640 1 -- 172.21.15.86:0/2204283166 learned_addr learned my addr 172.21.15.86:0/2204283166 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:27:24.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.412+0000 7f2c8cc00640 1 --2- 172.21.15.86:0/2204283166 >> v2:172.21.15.144:3300/0 conn(0x7f2c880cf500 0x7f2c880b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:24.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.412+0000 7f2c87400640 1 -- 172.21.15.86:0/2204283166 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1092500250 0 0) 0x7f2c88156110 con 0x7f2c88155d30 2024-04-20T21:27:24.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.412+0000 7f2c87400640 1 -- 172.21.15.86:0/2204283166 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2c6c003520 con 0x7f2c88155d30 2024-04-20T21:27:24.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.412+0000 7f2c87400640 1 -- 172.21.15.86:0/2204283166 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1899428453 0 0) 0x7f2c6c003520 con 0x7f2c88155d30 2024-04-20T21:27:24.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.412+0000 7f2c87400640 1 -- 172.21.15.86:0/2204283166 >> v2:172.21.15.144:3300/0 conn(0x7f2c880cf500 msgr2=0x7f2c880b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:24.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.412+0000 7f2c87400640 1 --2- 172.21.15.86:0/2204283166 >> v2:172.21.15.144:3300/0 conn(0x7f2c880cf500 0x7f2c880b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:24.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.412+0000 7f2c87400640 1 -- 172.21.15.86:0/2204283166 >> v1:172.21.15.144:6789/0 conn(0x7f2c8806c090 legacy=0x7f2c880b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:24.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.412+0000 7f2c87400640 1 -- 172.21.15.86:0/2204283166 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c880cea20 con 0x7f2c88155d30 2024-04-20T21:27:24.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.412+0000 7f2c87400640 1 -- 172.21.15.86:0/2204283166 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f2c78001670 con 0x7f2c88155d30 2024-04-20T21:27:24.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.412+0000 7f2c8cc00640 1 --2- 172.21.15.86:0/2204283166 >> v2:172.21.15.144:3300/0 conn(0x7f2c880cf500 0x7f2c880b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:27:24.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.412+0000 7f2c87400640 1 -- 172.21.15.86:0/2204283166 >> v1:172.21.15.86:6789/0 conn(0x7f2c88155d30 legacy=0x7f2c8806b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:24.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.412+0000 7f2c87400640 1 --2- 172.21.15.86:0/2204283166 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c6c003a00 0x7f2c6c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:24.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.412+0000 7f2c87400640 1 --2- 172.21.15.86:0/2204283166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c6c004800 0x7f2c6c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:24.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.413+0000 7f2c87400640 1 --2- 172.21.15.86:0/2204283166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c6c0053c0 0x7f2c6c00dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:24.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.413+0000 7f2c87400640 1 -- 172.21.15.86:0/2204283166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2c880ce740 con 0x7f2c6c0053c0 2024-04-20T21:27:24.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.413+0000 7f2c87400640 1 -- 172.21.15.86:0/2204283166 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2c6c00e490 con 0x7f2c6c004800 2024-04-20T21:27:24.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.413+0000 7f2c87400640 1 -- 172.21.15.86:0/2204283166 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2c6c00e690 con 0x7f2c6c003a00 2024-04-20T21:27:24.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.413+0000 7f2c8cc00640 1 --2- 172.21.15.86:0/2204283166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c6c0053c0 0x7f2c6c00dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:24.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.413+0000 7f2c7f400640 1 --2- 172.21.15.86:0/2204283166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c6c004800 0x7f2c6c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:24.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.413+0000 7f2c87e00640 1 --2- 172.21.15.86:0/2204283166 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c6c003a00 0x7f2c6c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:24.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.413+0000 7f2c8cc00640 1 -- 172.21.15.86:0/2204283166 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c6c003a00 msgr2=0x7f2c6c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:24.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.413+0000 7f2c8cc00640 1 --2- 172.21.15.86:0/2204283166 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c6c003a00 0x7f2c6c003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:24.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.413+0000 7f2c8cc00640 1 -- 172.21.15.86:0/2204283166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c6c004800 msgr2=0x7f2c6c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:24.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.413+0000 7f2c8cc00640 1 --2- 172.21.15.86:0/2204283166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c6c004800 0x7f2c6c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:24.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.413+0000 7f2c8cc00640 1 -- 172.21.15.86:0/2204283166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c6c003520 con 0x7f2c6c0053c0 2024-04-20T21:27:24.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.413+0000 7f2c7f400640 1 --2- 172.21.15.86:0/2204283166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c6c004800 0x7f2c6c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:27:24.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.414+0000 7f2c8cc00640 1 -- 172.21.15.86:0/2204283166 reap_dead start 2024-04-20T21:27:24.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.414+0000 7f2c8cc00640 1 --2- 172.21.15.86:0/2204283166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c6c0053c0 0x7f2c6c00dbd0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f2c800073e0 tx=0x7f2c6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1920c34c0768b365 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:24.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.415+0000 7f2c87400640 1 -- 172.21.15.86:0/2204283166 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80039070 con 0x7f2c6c0053c0 2024-04-20T21:27:24.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.415+0000 7f2c87400640 1 -- 172.21.15.86:0/2204283166 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c8002ce00 con 0x7f2c6c0053c0 2024-04-20T21:27:24.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.415+0000 7f2c87400640 1 -- 172.21.15.86:0/2204283166 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80009c80 con 0x7f2c6c0053c0 2024-04-20T21:27:24.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.415+0000 7f2c8ec00640 1 -- 172.21.15.86:0/2204283166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c6c0053c0 msgr2=0x7f2c6c00dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:24.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.415+0000 7f2c8ec00640 1 --2- 172.21.15.86:0/2204283166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c6c0053c0 0x7f2c6c00dbd0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f2c800073e0 tx=0x7f2c6c005830 comp rx=0 tx=0).stop 2024-04-20T21:27:24.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.415+0000 7f2c8ec00640 1 -- 172.21.15.86:0/2204283166 shutdown_connections 2024-04-20T21:27:24.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.415+0000 7f2c8ec00640 1 --2- 172.21.15.86:0/2204283166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c6c0053c0 0x7f2c6c00dbd0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:24.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.415+0000 7f2c8ec00640 1 -- 172.21.15.86:0/2204283166 >> 172.21.15.86:0/2204283166 conn(0x7f2c880b2110 msgr2=0x7f2c880b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:24.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.415+0000 7f2c8ec00640 1 -- 172.21.15.86:0/2204283166 shutdown_connections 2024-04-20T21:27:24.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.415+0000 7f2c8ec00640 1 -- 172.21.15.86:0/2204283166 wait complete. 2024-04-20T21:27:24.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.417+0000 7f2c8ec00640 1 Processor -- start 2024-04-20T21:27:24.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.417+0000 7f2c8ec00640 1 -- start start 2024-04-20T21:27:24.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.418+0000 7f2c8ec00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c6c0053c0 0x7f2c88143f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:24.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.418+0000 7f2c8ec00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c8815e910 0x7f2c88156340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:24.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.418+0000 7f2c8ec00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c88151c40 0x7f2c881520c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:24.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.418+0000 7f2c8ec00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8806b980 con 0x7f2c88151c40 2024-04-20T21:27:24.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.418+0000 7f2c8ec00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2c880cefe0 con 0x7f2c8815e910 2024-04-20T21:27:24.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.418+0000 7f2c8ec00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2c8806bb00 con 0x7f2c6c0053c0 2024-04-20T21:27:24.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.418+0000 7f2c8cc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c88151c40 0x7f2c881520c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:24.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.419+0000 7f2c8cc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c88151c40 0x7f2c881520c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37584/0 (socket says 172.21.15.86:37584) 2024-04-20T21:27:24.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.419+0000 7f2c8cc00640 1 -- 172.21.15.86:0/3409563967 learned_addr learned my addr 172.21.15.86:0/3409563967 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:24.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.419+0000 7f2c87e00640 1 --2- 172.21.15.86:0/3409563967 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c6c0053c0 0x7f2c88143f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:24.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.419+0000 7f2c8cc00640 1 -- 172.21.15.86:0/3409563967 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c6c0053c0 msgr2=0x7f2c88143f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:24.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.419+0000 7f2c8cc00640 1 --2- 172.21.15.86:0/3409563967 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c6c0053c0 0x7f2c88143f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:24.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.419+0000 7f2c8cc00640 1 -- 172.21.15.86:0/3409563967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c8815e910 msgr2=0x7f2c88156340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:24.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.419+0000 7f2c7f400640 1 --2- 172.21.15.86:0/3409563967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c8815e910 0x7f2c88156340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:24.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.419+0000 7f2c8cc00640 1 --2- 172.21.15.86:0/3409563967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c8815e910 0x7f2c88156340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:24.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.419+0000 7f2c8cc00640 1 -- 172.21.15.86:0/3409563967 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c88152600 con 0x7f2c88151c40 2024-04-20T21:27:24.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.419+0000 7f2c8cc00640 1 --2- 172.21.15.86:0/3409563967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c88151c40 0x7f2c881520c0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f2c800074d0 tx=0x7f2c80004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:24.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.420+0000 7f2c84c00640 1 -- 172.21.15.86:0/3409563967 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80039070 con 0x7f2c88151c40 2024-04-20T21:27:24.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.420+0000 7f2c84c00640 1 -- 172.21.15.86:0/3409563967 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c8003a040 con 0x7f2c88151c40 2024-04-20T21:27:24.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.420+0000 7f2c84c00640 1 -- 172.21.15.86:0/3409563967 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80009c80 con 0x7f2c88151c40 2024-04-20T21:27:24.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.420+0000 7f2c8ec00640 1 -- 172.21.15.86:0/3409563967 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c881444a0 con 0x7f2c88151c40 2024-04-20T21:27:24.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.420+0000 7f2c8ec00640 1 -- 172.21.15.86:0/3409563967 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c88156e70 con 0x7f2c88151c40 2024-04-20T21:27:24.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.422+0000 7f2c84c00640 1 -- 172.21.15.86:0/3409563967 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2c80046020 con 0x7f2c88151c40 2024-04-20T21:27:24.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.423+0000 7f2c84c00640 1 --2- 172.21.15.86:0/3409563967 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2c6406c8f0 0x7f2c6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:24.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.423+0000 7f2c84c00640 1 -- 172.21.15.86:0/3409563967 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7f2c800ad240 con 0x7f2c88151c40 2024-04-20T21:27:24.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.423+0000 7f2c87e00640 1 --2- 172.21.15.86:0/3409563967 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2c6406c8f0 0x7f2c6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:24.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.424+0000 7f2c91967740 1 Processor -- start 2024-04-20T21:27:24.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.424+0000 7f2c87e00640 1 --2- 172.21.15.86:0/3409563967 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2c6406c8f0 0x7f2c6406edb0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f2c780017e0 tx=0x7f2c78009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:24.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.424+0000 7f2c91967740 1 -- start start 2024-04-20T21:27:24.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.425+0000 7f2c91967740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc553b7eb0 0x55cc5538ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:24.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.425+0000 7f2c91967740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55cc553b8290 0x55cc553bac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:24.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.425+0000 7f2c91967740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc5538b390 0x55cc553bb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:24.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.425+0000 7f2c91967740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cc552f6060 con 0x55cc553b7eb0 2024-04-20T21:27:24.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.425+0000 7f2c91967740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55cc5529bf00 con 0x55cc5538b390 2024-04-20T21:27:24.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.425+0000 7f2c91967740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55cc552f2020 con 0x55cc553b8290 2024-04-20T21:27:24.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.425+0000 7f2c87e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc553b7eb0 0x55cc5538ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:24.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.425+0000 7f2c87e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc553b7eb0 0x55cc5538ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37596/0 (socket says 172.21.15.86:37596) 2024-04-20T21:27:24.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.425+0000 7f2c87e00640 1 -- 172.21.15.86:0/1475218763 learned_addr learned my addr 172.21.15.86:0/1475218763 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:24.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.426+0000 7f2c87e00640 1 -- 172.21.15.86:0/1475218763 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55cc553b8290 msgr2=0x55cc553bac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:24.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.426+0000 7f2c87e00640 1 --2- 172.21.15.86:0/1475218763 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55cc553b8290 0x55cc553bac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:24.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.426+0000 7f2c87e00640 1 -- 172.21.15.86:0/1475218763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc5538b390 msgr2=0x55cc553bb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:24.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.426+0000 7f2c87e00640 1 --2- 172.21.15.86:0/1475218763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc5538b390 0x55cc553bb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:24.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.426+0000 7f2c87e00640 1 -- 172.21.15.86:0/1475218763 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc55284a30 con 0x55cc553b7eb0 2024-04-20T21:27:24.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.426+0000 7f2c87e00640 1 --2- 172.21.15.86:0/1475218763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc553b7eb0 0x55cc5538ae50 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f2c78007150 tx=0x7f2c78011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf13daf584cc505d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:24.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.426+0000 7f2c77e00640 1 -- 172.21.15.86:0/1475218763 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7800ea10 con 0x55cc553b7eb0 2024-04-20T21:27:24.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.426+0000 7f2c77e00640 1 -- 172.21.15.86:0/1475218763 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c7800ebb0 con 0x55cc553b7eb0 2024-04-20T21:27:24.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.426+0000 7f2c77e00640 1 -- 172.21.15.86:0/1475218763 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c78013480 con 0x55cc553b7eb0 2024-04-20T21:27:24.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.427+0000 7f2c91967740 1 -- 172.21.15.86:0/1475218763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc553b7eb0 msgr2=0x55cc5538ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:24.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.427+0000 7f2c91967740 1 --2- 172.21.15.86:0/1475218763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc553b7eb0 0x55cc5538ae50 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f2c78007150 tx=0x7f2c78011000 comp rx=0 tx=0).stop 2024-04-20T21:27:24.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.427+0000 7f2c91967740 1 -- 172.21.15.86:0/1475218763 shutdown_connections 2024-04-20T21:27:24.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.427+0000 7f2c91967740 1 --2- 172.21.15.86:0/1475218763 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc5538b390 0x55cc553bb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:24.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.427+0000 7f2c91967740 1 --2- 172.21.15.86:0/1475218763 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55cc553b8290 0x55cc553bac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:24.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.427+0000 7f2c91967740 1 --2- 172.21.15.86:0/1475218763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc553b7eb0 0x55cc5538ae50 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:24.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.427+0000 7f2c91967740 1 -- 172.21.15.86:0/1475218763 >> 172.21.15.86:0/1475218763 conn(0x55cc5538d0c0 msgr2=0x55cc553d5c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:24.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.427+0000 7f2c91967740 1 -- 172.21.15.86:0/1475218763 shutdown_connections 2024-04-20T21:27:24.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.427+0000 7f2c91967740 1 -- 172.21.15.86:0/1475218763 wait complete. 2024-04-20T21:27:24.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.428+0000 7f2c91967740 1 -- start start 2024-04-20T21:27:24.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.428+0000 7f2c91967740 1 Processor -- start 2024-04-20T21:27:24.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.429+0000 7f2c91967740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc553b7eb0 0x55cc553bdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:24.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.429+0000 7f2c91967740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55cc553b8290 0x55cc553ce5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:24.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.429+0000 7f2c91967740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc553ceb30 0x55cc553d2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:24.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.429+0000 7f2c91967740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cc5538f660 con 0x55cc553b7eb0 2024-04-20T21:27:24.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.429+0000 7f2c91967740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55cc552af3d0 con 0x55cc553ceb30 2024-04-20T21:27:24.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.429+0000 7f2c91967740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55cc5538ea50 con 0x55cc553b8290 2024-04-20T21:27:24.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.429+0000 7f2c91967740 10 client.0 _ll_register_callbacks cb 0x55cc5527d960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:27:24.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.429+0000 7f2c87e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc553b7eb0 0x55cc553bdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:24.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.429+0000 7f2c87e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc553b7eb0 0x55cc553bdfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37598/0 (socket says 172.21.15.86:37598) 2024-04-20T21:27:24.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.429+0000 7f2c87e00640 1 -- 172.21.15.86:0/2041395898 learned_addr learned my addr 172.21.15.86:0/2041395898 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:24.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.430+0000 7f2c8cc00640 1 --2- 172.21.15.86:0/2041395898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc553ceb30 0x55cc553d2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:24.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.430+0000 7f2c7f400640 1 --2- 172.21.15.86:0/2041395898 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55cc553b8290 0x55cc553ce5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:24.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.430+0000 7f2c87e00640 1 -- 172.21.15.86:0/2041395898 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55cc553b8290 msgr2=0x55cc553ce5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:24.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.430+0000 7f2c87e00640 1 --2- 172.21.15.86:0/2041395898 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55cc553b8290 0x55cc553ce5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:24.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.430+0000 7f2c87e00640 1 -- 172.21.15.86:0/2041395898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc553ceb30 msgr2=0x55cc553d2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:24.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.430+0000 7f2c87e00640 1 --2- 172.21.15.86:0/2041395898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc553ceb30 0x55cc553d2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:24.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.430+0000 7f2c87e00640 1 -- 172.21.15.86:0/2041395898 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc550d47c0 con 0x55cc553b7eb0 2024-04-20T21:27:24.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.430+0000 7f2c8cc00640 1 --2- 172.21.15.86:0/2041395898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc553ceb30 0x55cc553d2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:27:24.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.430+0000 7f2c87e00640 1 --2- 172.21.15.86:0/2041395898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc553b7eb0 0x55cc553bdfe0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f2c78005bc0 tx=0x7f2c880ce630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe0e73d4d577514e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:24.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.430+0000 7f2c74c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:27:24.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.430+0000 7f2c74c00640 1 -- 172.21.15.86:0/2041395898 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c78013690 con 0x55cc553b7eb0 2024-04-20T21:27:24.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.430+0000 7f2c74c00640 1 -- 172.21.15.86:0/2041395898 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7f2c78013830 con 0x55cc553b7eb0 2024-04-20T21:27:24.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.430+0000 7f2c91967740 1 -- 172.21.15.86:0/2041395898 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c7800ebb0 con 0x55cc553b7eb0 2024-04-20T21:27:24.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.431+0000 7f2c74c00640 1 -- 172.21.15.86:0/2041395898 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f2c78036050 con 0x55cc553b7eb0 2024-04-20T21:27:24.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.431+0000 7f2c91967740 10 client.8271 fetch_fsmap learned FSMap version 73 2024-04-20T21:27:24.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.431+0000 7f2c91967740 1 -- 172.21.15.86:0/2041395898 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) -- 0x55cc55134f50 con 0x55cc553b7eb0 2024-04-20T21:27:24.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.431+0000 7f2c74c00640 1 -- 172.21.15.86:0/2041395898 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c7800ebb0 con 0x55cc553b7eb0 2024-04-20T21:27:24.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.431+0000 7f2c74c00640 1 -- 172.21.15.86:0/2041395898 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 73) ==== 3495+0+0 (secure 0 0 0) 0x7f2c78039020 con 0x55cc553b7eb0 2024-04-20T21:27:24.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.431+0000 7f2c91967740 10 client.8271 fetch_fsmap finished waiting for FSMap version 73 2024-04-20T21:27:24.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.431+0000 7f2c91967740 10 client.8271 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-20T21:27:24.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.431+0000 7f2c91967740 20 client.8271 populate_metadata read hostname 'smithi086' 2024-04-20T21:27:24.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.431+0000 7f2c91967740 1 --2- 172.21.15.86:0/2041395898 >> [v2:172.21.15.144:6832/2351381466,v1:172.21.15.144:6834/2351381466] conn(0x55cc553f1df0 0x55cc553f41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:24.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.431+0000 7f2c91967740 4 client.8271 mds_command: new command op to 7518 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:27:24.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.431+0000 7f2c91967740 1 -- 172.21.15.86:0/2041395898 --> [v2:172.21.15.144:6832/2351381466,v1:172.21.15.144:6834/2351381466] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc55303a80 con 0x55cc553f1df0 2024-04-20T21:27:24.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.432+0000 7f2c7f400640 1 --2- 172.21.15.86:0/2041395898 >> [v2:172.21.15.144:6832/2351381466,v1:172.21.15.144:6834/2351381466] conn(0x55cc553f1df0 0x55cc553f41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:24.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.433+0000 7f2c7f400640 1 --2- 172.21.15.86:0/2041395898 >> [v2:172.21.15.144:6832/2351381466,v1:172.21.15.144:6834/2351381466] conn(0x55cc553f1df0 0x55cc553f41e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0cbc74d01e75982 server_cookie=dbf32ae88342d2cf in_seq=0 out_seq=0 2024-04-20T21:27:24.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.433+0000 7f2c74c00640 10 client.8271 ms_handle_connect on v2:172.21.15.144:6832/2351381466 2024-04-20T21:27:24.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.434+0000 7f2c74c00640 1 -- 172.21.15.86:0/2041395898 <== mds.0 v2:172.21.15.144:6832/2351381466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cc55303a80 con 0x55cc553f1df0 2024-04-20T21:27:24.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.434+0000 7f2c74c00640 10 client.8271 handle_command_reply: tid=0 2024-04-20T21:27:24.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.434+0000 7f2c91967740 1 client.8271 shutdown 2024-04-20T21:27:24.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.435+0000 7f2c91967740 1 -- 172.21.15.86:0/2041395898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc553b7eb0 msgr2=0x55cc553bdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:24.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.435+0000 7f2c91967740 1 --2- 172.21.15.86:0/2041395898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc553b7eb0 0x55cc553bdfe0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f2c78005bc0 tx=0x7f2c880ce630 comp rx=0 tx=0).stop 2024-04-20T21:27:24.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.435+0000 7f2c91967740 1 -- 172.21.15.86:0/2041395898 shutdown_connections 2024-04-20T21:27:24.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.435+0000 7f2c91967740 1 --2- 172.21.15.86:0/2041395898 >> [v2:172.21.15.144:6832/2351381466,v1:172.21.15.144:6834/2351381466] conn(0x55cc553f1df0 0x55cc553f41e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:24.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.435+0000 7f2c91967740 1 --2- 172.21.15.86:0/2041395898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc553ceb30 0x55cc553d2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:24.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.435+0000 7f2c91967740 1 --2- 172.21.15.86:0/2041395898 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55cc553b8290 0x55cc553ce5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:24.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.435+0000 7f2c74c00640 0 client.8271 ms_handle_reset on v2:172.21.15.144:6832/2351381466 2024-04-20T21:27:24.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.435+0000 7f2c91967740 1 --2- 172.21.15.86:0/2041395898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc553b7eb0 0x55cc553bdfe0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:24.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.435+0000 7f2c91967740 1 -- 172.21.15.86:0/2041395898 >> 172.21.15.86:0/2041395898 conn(0x55cc5538b390 msgr2=0x55cc553bac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:24.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.435+0000 7f2c91967740 1 -- 172.21.15.86:0/2041395898 shutdown_connections 2024-04-20T21:27:24.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.435+0000 7f2c91967740 1 -- 172.21.15.86:0/2041395898 wait complete. 2024-04-20T21:27:24.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.435+0000 7f2c91967740 20 client.8271 trim_cache size 0 max 16384 2024-04-20T21:27:24.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.450+0000 7f2c91967740 1 Processor -- start 2024-04-20T21:27:24.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.450+0000 7f2c91967740 1 -- start start 2024-04-20T21:27:24.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.450+0000 7f2c91967740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc553b7eb0 0x55cc553b72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:24.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.450+0000 7f2c91967740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc553b8290 0x55cc5538f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:24.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.450+0000 7f2c91967740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55cc553ceb30 0x55cc553d09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:24.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.450+0000 7f2c91967740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2c88200550 con 0x55cc553b7eb0 2024-04-20T21:27:24.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.450+0000 7f2c91967740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8815b6a0 con 0x55cc553b8290 2024-04-20T21:27:24.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.450+0000 7f2c91967740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2c881fff30 con 0x55cc553ceb30 2024-04-20T21:27:24.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.451+0000 7f2c87e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc553b7eb0 0x55cc553b72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:24.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.451+0000 7f2c87e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc553b7eb0 0x55cc553b72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37600/0 (socket says 172.21.15.86:37600) 2024-04-20T21:27:24.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.451+0000 7f2c87e00640 1 -- 172.21.15.86:0/3607463224 learned_addr learned my addr 172.21.15.86:0/3607463224 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:24.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.451+0000 7f2c87e00640 1 -- 172.21.15.86:0/3607463224 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55cc553ceb30 msgr2=0x55cc553d09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:24.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.451+0000 7f2c87e00640 1 --2- 172.21.15.86:0/3607463224 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55cc553ceb30 0x55cc553d09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:24.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.451+0000 7f2c87e00640 1 -- 172.21.15.86:0/3607463224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc553b8290 msgr2=0x55cc5538f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:24.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.451+0000 7f2c87e00640 1 --2- 172.21.15.86:0/3607463224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc553b8290 0x55cc5538f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:24.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.451+0000 7f2c87e00640 1 -- 172.21.15.86:0/3607463224 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc54fc80b0 con 0x55cc553b7eb0 2024-04-20T21:27:24.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.451+0000 7f2c87e00640 1 --2- 172.21.15.86:0/3607463224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc553b7eb0 0x55cc553b72d0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f2c78001480 tx=0x7f2c78025050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eb257f42290f1e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:24.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.452+0000 7f2c77400640 1 -- 172.21.15.86:0/3607463224 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c78013b90 con 0x55cc553b7eb0 2024-04-20T21:27:24.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.452+0000 7f2c77400640 1 -- 172.21.15.86:0/3607463224 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c78013690 con 0x55cc553b7eb0 2024-04-20T21:27:24.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.452+0000 7f2c77400640 1 -- 172.21.15.86:0/3607463224 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c78021450 con 0x55cc553b7eb0 2024-04-20T21:27:24.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.452+0000 7f2c91967740 1 -- 172.21.15.86:0/3607463224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc553b7eb0 msgr2=0x55cc553b72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:24.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.452+0000 7f2c91967740 1 --2- 172.21.15.86:0/3607463224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc553b7eb0 0x55cc553b72d0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f2c78001480 tx=0x7f2c78025050 comp rx=0 tx=0).stop 2024-04-20T21:27:24.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.452+0000 7f2c91967740 1 -- 172.21.15.86:0/3607463224 shutdown_connections 2024-04-20T21:27:24.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.452+0000 7f2c91967740 1 --2- 172.21.15.86:0/3607463224 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55cc553ceb30 0x55cc553d09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:24.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.452+0000 7f2c91967740 1 --2- 172.21.15.86:0/3607463224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc553b8290 0x55cc5538f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:24.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.452+0000 7f2c91967740 1 --2- 172.21.15.86:0/3607463224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc553b7eb0 0x55cc553b72d0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:24.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.452+0000 7f2c91967740 1 -- 172.21.15.86:0/3607463224 >> 172.21.15.86:0/3607463224 conn(0x55cc5538b390 msgr2=0x55cc55391e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:24.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.452+0000 7f2c91967740 1 -- 172.21.15.86:0/3607463224 shutdown_connections 2024-04-20T21:27:24.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.453+0000 7f2c91967740 1 -- 172.21.15.86:0/3607463224 wait complete. 2024-04-20T21:27:24.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.453+0000 7f2c91967740 1 -- start start 2024-04-20T21:27:24.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.453+0000 7f2c91967740 1 Processor -- start 2024-04-20T21:27:24.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.454+0000 7f2c91967740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc553b7eb0 0x55cc553b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:24.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.454+0000 7f2c91967740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55cc553b8290 0x55cc553b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:24.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.454+0000 7f2c91967740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc553ceb30 0x55cc5544fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:24.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.454+0000 7f2c91967740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cc553be5c0 con 0x55cc553ceb30 2024-04-20T21:27:24.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.454+0000 7f2c91967740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55cc553b79d0 con 0x55cc553b7eb0 2024-04-20T21:27:24.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.454+0000 7f2c91967740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55cc553d0260 con 0x55cc553b8290 2024-04-20T21:27:24.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.454+0000 7f2c91967740 10 client.0 _ll_register_callbacks cb 0x7f2c78001bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:27:24.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.454+0000 7f2c8cc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc553ceb30 0x55cc5544fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:24.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.455+0000 7f2c8cc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc553ceb30 0x55cc5544fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37610/0 (socket says 172.21.15.86:37610) 2024-04-20T21:27:24.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.455+0000 7f2c8cc00640 1 -- 172.21.15.86:0/3457426041 learned_addr learned my addr 172.21.15.86:0/3457426041 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:24.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.455+0000 7f2c8cc00640 1 -- 172.21.15.86:0/3457426041 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55cc553b8290 msgr2=0x55cc553b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:24.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.455+0000 7f2c8cc00640 1 --2- 172.21.15.86:0/3457426041 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55cc553b8290 0x55cc553b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:24.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.455+0000 7f2c8cc00640 1 -- 172.21.15.86:0/3457426041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc553b7eb0 msgr2=0x55cc553b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:24.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.455+0000 7f2c87e00640 1 --2- 172.21.15.86:0/3457426041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc553b7eb0 0x55cc553b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:24.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.455+0000 7f2c8cc00640 1 --2- 172.21.15.86:0/3457426041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc553b7eb0 0x55cc553b1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:24.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.455+0000 7f2c8cc00640 1 -- 172.21.15.86:0/3457426041 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc553b4770 con 0x55cc553ceb30 2024-04-20T21:27:24.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.455+0000 7f2c8cc00640 1 --2- 172.21.15.86:0/3457426041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc553ceb30 0x55cc5544fe40 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f2c80083540 tx=0x7f2c8007ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d1d693337578bf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:24.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.455+0000 7f2c74c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:27:24.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.455+0000 7f2c74c00640 1 -- 172.21.15.86:0/3457426041 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8007ae90 con 0x55cc553ceb30 2024-04-20T21:27:24.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.455+0000 7f2c74c00640 1 -- 172.21.15.86:0/3457426041 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7f2c800835e0 con 0x55cc553ceb30 2024-04-20T21:27:24.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.455+0000 7f2c91967740 1 -- 172.21.15.86:0/3457426041 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c78013690 con 0x55cc553ceb30 2024-04-20T21:27:24.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.456+0000 7f2c74c00640 1 -- 172.21.15.86:0/3457426041 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f2c8007f240 con 0x55cc553ceb30 2024-04-20T21:27:24.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.456+0000 7f2c91967740 10 client.8277 fetch_fsmap learned FSMap version 73 2024-04-20T21:27:24.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.456+0000 7f2c91967740 1 -- 172.21.15.86:0/3457426041 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) -- 0x55cc553b4200 con 0x55cc553ceb30 2024-04-20T21:27:24.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.456+0000 7f2c74c00640 1 -- 172.21.15.86:0/3457426041 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c78013690 con 0x55cc553ceb30 2024-04-20T21:27:24.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.456+0000 7f2c74c00640 1 -- 172.21.15.86:0/3457426041 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 73) ==== 3495+0+0 (secure 0 0 0) 0x7f2c80093180 con 0x55cc553ceb30 2024-04-20T21:27:24.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.456+0000 7f2c91967740 10 client.8277 fetch_fsmap finished waiting for FSMap version 73 2024-04-20T21:27:24.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.456+0000 7f2c91967740 10 client.8277 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-20T21:27:24.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.456+0000 7f2c91967740 20 client.8277 populate_metadata read hostname 'smithi086' 2024-04-20T21:27:24.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.456+0000 7f2c91967740 1 --2- 172.21.15.86:0/3457426041 >> [v2:172.21.15.144:6832/2351381466,v1:172.21.15.144:6834/2351381466] conn(0x55cc5538ed80 0x55cc553cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:24.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.456+0000 7f2c91967740 4 client.8277 mds_command: new command op to 7518 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-20T21:27:24.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.456+0000 7f2c91967740 1 -- 172.21.15.86:0/3457426041 --> [v2:172.21.15.144:6832/2351381466,v1:172.21.15.144:6834/2351381466] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55cc552fd8f0 con 0x55cc5538ed80 2024-04-20T21:27:24.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.457+0000 7f2c87e00640 1 --2- 172.21.15.86:0/3457426041 >> [v2:172.21.15.144:6832/2351381466,v1:172.21.15.144:6834/2351381466] conn(0x55cc5538ed80 0x55cc553cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:24.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.458+0000 7f2c87e00640 1 --2- 172.21.15.86:0/3457426041 >> [v2:172.21.15.144:6832/2351381466,v1:172.21.15.144:6834/2351381466] conn(0x55cc5538ed80 0x55cc553cd8a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=310952c4720d2fe6 server_cookie=7cfb0550944c13c in_seq=0 out_seq=0 2024-04-20T21:27:24.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.458+0000 7f2c74c00640 10 client.8277 ms_handle_connect on v2:172.21.15.144:6832/2351381466 2024-04-20T21:27:24.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.458+0000 7f2c74c00640 1 -- 172.21.15.86:0/3457426041 <== mds.0 v2:172.21.15.144:6832/2351381466 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cc552fd8f0 con 0x55cc5538ed80 2024-04-20T21:27:24.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.458+0000 7f2c74c00640 10 client.8277 handle_command_reply: tid=0 2024-04-20T21:27:24.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.458+0000 7f2c91967740 1 client.8277 shutdown 2024-04-20T21:27:24.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.459+0000 7f2c91967740 1 -- 172.21.15.86:0/3457426041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc553ceb30 msgr2=0x55cc5544fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:24.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.459+0000 7f2c91967740 1 --2- 172.21.15.86:0/3457426041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc553ceb30 0x55cc5544fe40 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f2c80083540 tx=0x7f2c8007ad20 comp rx=0 tx=0).stop 2024-04-20T21:27:24.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.459+0000 7f2c91967740 1 -- 172.21.15.86:0/3457426041 shutdown_connections 2024-04-20T21:27:24.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.459+0000 7f2c91967740 1 --2- 172.21.15.86:0/3457426041 >> [v2:172.21.15.144:6832/2351381466,v1:172.21.15.144:6834/2351381466] conn(0x55cc5538ed80 0x55cc553cd8a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:24.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.459+0000 7f2c91967740 1 --2- 172.21.15.86:0/3457426041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc553ceb30 0x55cc5544fe40 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:24.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.459+0000 7f2c91967740 1 --2- 172.21.15.86:0/3457426041 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55cc553b8290 0x55cc553b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:24.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.459+0000 7f2c74c00640 0 client.8277 ms_handle_reset on v2:172.21.15.144:6832/2351381466 2024-04-20T21:27:24.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.459+0000 7f2c91967740 1 --2- 172.21.15.86:0/3457426041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cc553b7eb0 0x55cc553b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:24.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.459+0000 7f2c91967740 1 -- 172.21.15.86:0/3457426041 >> 172.21.15.86:0/3457426041 conn(0x55cc5538b390 msgr2=0x55cc553b7240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:24.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.459+0000 7f2c91967740 1 -- 172.21.15.86:0/3457426041 shutdown_connections 2024-04-20T21:27:24.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.459+0000 7f2c91967740 1 -- 172.21.15.86:0/3457426041 wait complete. 2024-04-20T21:27:24.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.459+0000 7f2c91967740 20 client.8277 trim_cache size 0 max 16384 2024-04-20T21:27:24.462 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:27:24.462 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-20T21:27:24.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.460+0000 7f2c8ec00640 1 -- 172.21.15.86:0/3409563967 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2c6406c8f0 msgr2=0x7f2c6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:24.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.460+0000 7f2c8ec00640 1 --2- 172.21.15.86:0/3409563967 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2c6406c8f0 0x7f2c6406edb0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f2c780017e0 tx=0x7f2c78009040 comp rx=0 tx=0).stop 2024-04-20T21:27:24.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.460+0000 7f2c8ec00640 1 -- 172.21.15.86:0/3409563967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c88151c40 msgr2=0x7f2c881520c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:24.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.460+0000 7f2c8ec00640 1 --2- 172.21.15.86:0/3409563967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c88151c40 0x7f2c881520c0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f2c800074d0 tx=0x7f2c80004a40 comp rx=0 tx=0).stop 2024-04-20T21:27:24.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.460+0000 7f2c8ec00640 1 -- 172.21.15.86:0/3409563967 shutdown_connections 2024-04-20T21:27:24.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.460+0000 7f2c8ec00640 1 --2- 172.21.15.86:0/3409563967 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2c6406c8f0 0x7f2c6406edb0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:24.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.460+0000 7f2c8ec00640 1 --2- 172.21.15.86:0/3409563967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c88151c40 0x7f2c881520c0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:24.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.460+0000 7f2c8ec00640 1 --2- 172.21.15.86:0/3409563967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c8815e910 0x7f2c88156340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:24.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.460+0000 7f2c8ec00640 1 --2- 172.21.15.86:0/3409563967 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c6c0053c0 0x7f2c88143f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:24.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.460+0000 7f2c8ec00640 1 -- 172.21.15.86:0/3409563967 >> 172.21.15.86:0/3409563967 conn(0x7f2c880b2110 msgr2=0x7f2c880b3ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:24.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.460+0000 7f2c8ec00640 1 -- 172.21.15.86:0/3409563967 shutdown_connections 2024-04-20T21:27:24.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.460+0000 7f2c8ec00640 1 -- 172.21.15.86:0/3409563967 wait complete. 2024-04-20T21:27:24.477 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-20T21:27:24.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.672+0000 7fef3d600640 1 Processor -- start 2024-04-20T21:27:24.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.673+0000 7fef3d600640 1 -- start start 2024-04-20T21:27:24.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.674+0000 7fef3d600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fef38144e90 0x7fef380b4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:24.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.674+0000 7fef3d600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef380cd170 0x7fef3806ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:24.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.674+0000 7fef3d600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef380cdb50 con 0x7fef380cd5d0 2024-04-20T21:27:24.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.674+0000 7fef3d600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fef38145270 con 0x7fef38144e90 2024-04-20T21:27:24.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.674+0000 7fef3d600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fef380b3e50 con 0x7fef380cd170 2024-04-20T21:27:24.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.675+0000 7fef37400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fef380cd5d0 0x7fef3815bb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:59416/0 (socket says 172.21.15.86:59416) 2024-04-20T21:27:24.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.675+0000 7fef37400640 1 -- 172.21.15.86:0/332385548 learned_addr learned my addr 172.21.15.86:0/332385548 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:27:24.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.675+0000 7fef36a00640 1 --2- 172.21.15.86:0/332385548 >> v2:172.21.15.144:3300/0 conn(0x7fef38144e90 0x7fef380b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:24.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.675+0000 7fef2e000640 1 --2- 172.21.15.86:0/332385548 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef380cd170 0x7fef3806ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:24.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.675+0000 7fef36000640 1 -- 172.21.15.86:0/332385548 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3786388578 0 0) 0x7fef380cdb50 con 0x7fef380cd5d0 2024-04-20T21:27:24.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.675+0000 7fef36000640 1 -- 172.21.15.86:0/332385548 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef1c005730 con 0x7fef380cd5d0 2024-04-20T21:27:24.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.675+0000 7fef36000640 1 -- 172.21.15.86:0/332385548 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1969869353 0 0) 0x7fef1c005730 con 0x7fef380cd5d0 2024-04-20T21:27:24.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.675+0000 7fef36000640 1 -- 172.21.15.86:0/332385548 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef380cd170 msgr2=0x7fef3806ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:24.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.675+0000 7fef36000640 1 --2- 172.21.15.86:0/332385548 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef380cd170 0x7fef3806ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:24.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.675+0000 7fef36000640 1 -- 172.21.15.86:0/332385548 >> v2:172.21.15.144:3300/0 conn(0x7fef38144e90 msgr2=0x7fef380b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:24.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.675+0000 7fef36000640 1 --2- 172.21.15.86:0/332385548 >> v2:172.21.15.144:3300/0 conn(0x7fef38144e90 0x7fef380b4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:24.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.675+0000 7fef36000640 1 -- 172.21.15.86:0/332385548 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef380b4130 con 0x7fef380cd5d0 2024-04-20T21:27:24.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.675+0000 7fef36000640 1 -- 172.21.15.86:0/332385548 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fef300019f0 con 0x7fef380cd5d0 2024-04-20T21:27:24.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.675+0000 7fef36000640 1 -- 172.21.15.86:0/332385548 >> v1:172.21.15.86:6789/0 conn(0x7fef380cd5d0 legacy=0x7fef3815bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:24.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.675+0000 7fef36000640 1 --2- 172.21.15.86:0/332385548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef1c005bc0 0x7fef1c005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:24.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.676+0000 7fef36000640 1 --2- 172.21.15.86:0/332385548 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef380cd170 0x7fef1c006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:24.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.676+0000 7fef36000640 1 --2- 172.21.15.86:0/332385548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef1c006f50 0x7fef1c0073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:24.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.676+0000 7fef36000640 1 -- 172.21.15.86:0/332385548 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fef38145270 con 0x7fef1c006f50 2024-04-20T21:27:24.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.676+0000 7fef36000640 1 -- 172.21.15.86:0/332385548 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fef380b3e50 con 0x7fef1c005bc0 2024-04-20T21:27:24.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.676+0000 7fef36000640 1 -- 172.21.15.86:0/332385548 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fef1c00dcb0 con 0x7fef380cd170 2024-04-20T21:27:24.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.676+0000 7fef37400640 1 --2- 172.21.15.86:0/332385548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef1c006f50 0x7fef1c0073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:24.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.676+0000 7fef2e000640 1 --2- 172.21.15.86:0/332385548 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef380cd170 0x7fef1c006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:24.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.676+0000 7fef36a00640 1 --2- 172.21.15.86:0/332385548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef1c005bc0 0x7fef1c005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:24.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.676+0000 7fef37400640 1 -- 172.21.15.86:0/332385548 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef380cd170 msgr2=0x7fef1c006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:24.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.676+0000 7fef37400640 1 --2- 172.21.15.86:0/332385548 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef380cd170 0x7fef1c006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:24.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.676+0000 7fef37400640 1 -- 172.21.15.86:0/332385548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef1c005bc0 msgr2=0x7fef1c005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:24.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.676+0000 7fef37400640 1 --2- 172.21.15.86:0/332385548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef1c005bc0 0x7fef1c005fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:24.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.676+0000 7fef37400640 1 -- 172.21.15.86:0/332385548 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef1c005730 con 0x7fef1c006f50 2024-04-20T21:27:24.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.676+0000 7fef2e000640 1 --2- 172.21.15.86:0/332385548 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef380cd170 0x7fef1c006a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:27:24.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.677+0000 7fef37400640 1 --2- 172.21.15.86:0/332385548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef1c006f50 0x7fef1c0073d0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fef3000a880 tx=0x7fef1c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66700108ecebe7d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:24.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.678+0000 7fef36000640 1 -- 172.21.15.86:0/332385548 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3000a4a0 con 0x7fef1c006f50 2024-04-20T21:27:24.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.678+0000 7fef36000640 1 -- 172.21.15.86:0/332385548 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef30034650 con 0x7fef1c006f50 2024-04-20T21:27:24.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.678+0000 7fef36000640 1 -- 172.21.15.86:0/332385548 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef30034a40 con 0x7fef1c006f50 2024-04-20T21:27:24.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.678+0000 7fef3d600640 1 -- 172.21.15.86:0/332385548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef1c006f50 msgr2=0x7fef1c0073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:24.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.678+0000 7fef3d600640 1 --2- 172.21.15.86:0/332385548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef1c006f50 0x7fef1c0073d0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fef3000a880 tx=0x7fef1c007a40 comp rx=0 tx=0).stop 2024-04-20T21:27:24.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.678+0000 7fef37400640 1 -- 172.21.15.86:0/332385548 reap_dead start 2024-04-20T21:27:24.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.678+0000 7fef3d600640 1 -- 172.21.15.86:0/332385548 shutdown_connections 2024-04-20T21:27:24.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.678+0000 7fef3d600640 1 -- 172.21.15.86:0/332385548 >> 172.21.15.86:0/332385548 conn(0x7fef380b2360 msgr2=0x7fef380b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:24.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.678+0000 7fef3d600640 1 -- 172.21.15.86:0/332385548 shutdown_connections 2024-04-20T21:27:24.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.678+0000 7fef3d600640 1 -- 172.21.15.86:0/332385548 wait complete. 2024-04-20T21:27:24.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.680+0000 7fef3d600640 1 Processor -- start 2024-04-20T21:27:24.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.681+0000 7fef3d600640 1 -- start start 2024-04-20T21:27:24.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.681+0000 7fef3d600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef1c006f50 0x7fef3813fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:24.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.681+0000 7fef3d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef3813cd10 0x7fef381f4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:24.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.681+0000 7fef3d600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef3813d190 0x7fef3813e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:24.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.682+0000 7fef3d600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fef380d0010 con 0x7fef3813cd10 2024-04-20T21:27:24.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.682+0000 7fef3d600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fef380b4e70 con 0x7fef3813d190 2024-04-20T21:27:24.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.682+0000 7fef3d600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fef380d0190 con 0x7fef1c006f50 2024-04-20T21:27:24.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.682+0000 7fef2e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef3813cd10 0x7fef381f4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:24.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.682+0000 7fef2e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef3813cd10 0x7fef381f4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37628/0 (socket says 172.21.15.86:37628) 2024-04-20T21:27:24.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.682+0000 7fef2e000640 1 -- 172.21.15.86:0/151302023 learned_addr learned my addr 172.21.15.86:0/151302023 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:24.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.682+0000 7fef2e000640 1 -- 172.21.15.86:0/151302023 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef1c006f50 msgr2=0x7fef3813fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:24.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.682+0000 7fef2e000640 1 --2- 172.21.15.86:0/151302023 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef1c006f50 0x7fef3813fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:24.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.682+0000 7fef2e000640 1 -- 172.21.15.86:0/151302023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef3813d190 msgr2=0x7fef3813e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:24.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.682+0000 7fef2e000640 1 --2- 172.21.15.86:0/151302023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef3813d190 0x7fef3813e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:24.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.682+0000 7fef2e000640 1 -- 172.21.15.86:0/151302023 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef3813d6f0 con 0x7fef3813cd10 2024-04-20T21:27:24.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.682+0000 7fef2e000640 1 --2- 172.21.15.86:0/151302023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef3813cd10 0x7fef381f4a60 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fef2000e7e0 tx=0x7fef2000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:24.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.683+0000 7fef2f400640 1 -- 172.21.15.86:0/151302023 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef20017070 con 0x7fef3813cd10 2024-04-20T21:27:24.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.683+0000 7fef2f400640 1 -- 172.21.15.86:0/151302023 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef2000c650 con 0x7fef3813cd10 2024-04-20T21:27:24.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.683+0000 7fef2f400640 1 -- 172.21.15.86:0/151302023 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2000f040 con 0x7fef3813cd10 2024-04-20T21:27:24.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.683+0000 7fef3d600640 1 -- 172.21.15.86:0/151302023 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef3813d9d0 con 0x7fef3813cd10 2024-04-20T21:27:24.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.683+0000 7fef3d600640 1 -- 172.21.15.86:0/151302023 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef381f7800 con 0x7fef3813cd10 2024-04-20T21:27:24.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.684+0000 7fef3d600640 1 -- 172.21.15.86:0/151302023 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef380689e0 con 0x7fef3813cd10 2024-04-20T21:27:24.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.685+0000 7fef2f400640 1 -- 172.21.15.86:0/151302023 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fef2001a070 con 0x7fef3813cd10 2024-04-20T21:27:24.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.686+0000 7fef2f400640 1 --2- 172.21.15.86:0/151302023 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fef1406c8f0 0x7fef1406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:24.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.686+0000 7fef2f400640 1 -- 172.21.15.86:0/151302023 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7fef2008b860 con 0x7fef3813cd10 2024-04-20T21:27:24.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.686+0000 7fef36a00640 1 --2- 172.21.15.86:0/151302023 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fef1406c8f0 0x7fef1406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:24.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.687+0000 7fef36a00640 1 --2- 172.21.15.86:0/151302023 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fef1406c8f0 0x7fef1406edb0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fef280099d0 tx=0x7fef28008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:24.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.691+0000 7fef2f400640 1 -- 172.21.15.86:0/151302023 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fef20059b20 con 0x7fef3813cd10 2024-04-20T21:27:24.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.934+0000 7fef3d600640 1 -- 172.21.15.86:0/151302023 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fef381f4040 con 0x7fef3813cd10 2024-04-20T21:27:24.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.935+0000 7fef2f400640 1 -- 172.21.15.86:0/151302023 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fef20059420 con 0x7fef3813cd10 2024-04-20T21:27:24.935 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:27:24.935 INFO:teuthology.orchestra.run.smithi086.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-20T21:27:24.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.937+0000 7fef3d600640 1 -- 172.21.15.86:0/151302023 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fef1406c8f0 msgr2=0x7fef1406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:24.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.937+0000 7fef3d600640 1 --2- 172.21.15.86:0/151302023 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fef1406c8f0 0x7fef1406edb0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fef280099d0 tx=0x7fef28008040 comp rx=0 tx=0).stop 2024-04-20T21:27:24.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.937+0000 7fef3d600640 1 -- 172.21.15.86:0/151302023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef3813cd10 msgr2=0x7fef381f4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:24.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.937+0000 7fef3d600640 1 --2- 172.21.15.86:0/151302023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef3813cd10 0x7fef381f4a60 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fef2000e7e0 tx=0x7fef2000eca0 comp rx=0 tx=0).stop 2024-04-20T21:27:24.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.937+0000 7fef3d600640 1 -- 172.21.15.86:0/151302023 shutdown_connections 2024-04-20T21:27:24.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.937+0000 7fef3d600640 1 --2- 172.21.15.86:0/151302023 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fef1406c8f0 0x7fef1406edb0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:24.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.937+0000 7fef3d600640 1 --2- 172.21.15.86:0/151302023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef3813d190 0x7fef3813e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:24.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.937+0000 7fef3d600640 1 --2- 172.21.15.86:0/151302023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef3813cd10 0x7fef381f4a60 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:24.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.938+0000 7fef3d600640 1 --2- 172.21.15.86:0/151302023 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fef1c006f50 0x7fef3813fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:24.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.938+0000 7fef3d600640 1 -- 172.21.15.86:0/151302023 >> 172.21.15.86:0/151302023 conn(0x7fef380b2360 msgr2=0x7fef380b09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:24.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.938+0000 7fef3d600640 1 -- 172.21.15.86:0/151302023 shutdown_connections 2024-04-20T21:27:24.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:24.938+0000 7fef3d600640 1 -- 172.21.15.86:0/151302023 wait complete. 2024-04-20T21:27:24.956 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-20T21:27:24.956 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-20T21:27:24.957 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T21:27:25.062 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T21:27:25.184 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-20T21:27:25.184 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T21:27:25.184 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:27:25.184 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:27:25.243 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:27:25.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.447+0000 7f942d800640 1 Processor -- start 2024-04-20T21:27:25.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.448+0000 7f942d800640 1 -- start start 2024-04-20T21:27:25.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.449+0000 7f942d800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f94280b3810 0x7f94280ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:25.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.449+0000 7f942d800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f942806e640 con 0x7f94281629e0 2024-04-20T21:27:25.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.449+0000 7f942d800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f942806e920 con 0x7f9428162dc0 2024-04-20T21:27:25.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.450+0000 7f942d800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f942806ec00 con 0x7f94280b3810 2024-04-20T21:27:25.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.450+0000 7f942ce00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f94280b3810 0x7f94280ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:25.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.450+0000 7f9427e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f94281629e0 0x7f94280b3100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:59424/0 (socket says 172.21.15.86:59424) 2024-04-20T21:27:25.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.450+0000 7f9427e00640 1 -- 172.21.15.86:0/4134046124 learned_addr learned my addr 172.21.15.86:0/4134046124 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:27:25.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.450+0000 7f9426a00640 1 -- 172.21.15.86:0/4134046124 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3559888858 0 0) 0x7f942806e640 con 0x7f94281629e0 2024-04-20T21:27:25.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.450+0000 7f9426a00640 1 -- 172.21.15.86:0/4134046124 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9408003520 con 0x7f94281629e0 2024-04-20T21:27:25.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.450+0000 7f942ce00640 1 -- 172.21.15.86:0/4134046124 >> v1:172.21.15.144:6789/0 conn(0x7f9428162dc0 legacy=0x7f9428072b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:25.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.450+0000 7f942ce00640 1 -- 172.21.15.86:0/4134046124 >> v1:172.21.15.86:6789/0 conn(0x7f94281629e0 legacy=0x7f94280b3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:25.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.450+0000 7f942ce00640 1 -- 172.21.15.86:0/4134046124 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f942806eee0 con 0x7f94280b3810 2024-04-20T21:27:25.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.452+0000 7f942ce00640 1 --2- 172.21.15.86:0/4134046124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94280b3810 0x7f94280ad0e0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f94200097a0 tx=0x7f942002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7025bfd9bc51f48 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:25.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.452+0000 7f9426a00640 1 -- 172.21.15.86:0/4134046124 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94200315a0 con 0x7f94280b3810 2024-04-20T21:27:25.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.452+0000 7f9426a00640 1 -- 172.21.15.86:0/4134046124 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9420037070 con 0x7f94280b3810 2024-04-20T21:27:25.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.452+0000 7f9426a00640 1 -- 172.21.15.86:0/4134046124 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9420031960 con 0x7f94280b3810 2024-04-20T21:27:25.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.452+0000 7f942d800640 1 -- 172.21.15.86:0/4134046124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94280b3810 msgr2=0x7f94280ad0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:25.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.452+0000 7f942d800640 1 --2- 172.21.15.86:0/4134046124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94280b3810 0x7f94280ad0e0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f94200097a0 tx=0x7f942002b840 comp rx=0 tx=0).stop 2024-04-20T21:27:25.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.453+0000 7f942d800640 1 -- 172.21.15.86:0/4134046124 shutdown_connections 2024-04-20T21:27:25.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.453+0000 7f942d800640 1 --2- 172.21.15.86:0/4134046124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94280b3810 0x7f94280ad0e0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:25.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.453+0000 7f942d800640 1 -- 172.21.15.86:0/4134046124 >> 172.21.15.86:0/4134046124 conn(0x7f94280d0170 msgr2=0x7f94280d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:25.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.453+0000 7f942d800640 1 -- 172.21.15.86:0/4134046124 shutdown_connections 2024-04-20T21:27:25.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.453+0000 7f942d800640 1 -- 172.21.15.86:0/4134046124 wait complete. 2024-04-20T21:27:25.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.454+0000 7f942d800640 1 Processor -- start 2024-04-20T21:27:25.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.454+0000 7f942d800640 1 -- start start 2024-04-20T21:27:25.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.455+0000 7f942d800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94280b3810 0x7f9428164be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:25.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.455+0000 7f942d800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94281629e0 0x7f942815c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:25.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.455+0000 7f942d800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94281f9100 0x7f94280ce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:25.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.455+0000 7f942d800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9428073410 con 0x7f94281f9100 2024-04-20T21:27:25.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.455+0000 7f942d800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9428073290 con 0x7f94280b3810 2024-04-20T21:27:25.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.455+0000 7f942d800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9428073590 con 0x7f94281629e0 2024-04-20T21:27:25.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.456+0000 7f942ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94281f9100 0x7f94280ce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:25.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.456+0000 7f942ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94281f9100 0x7f94280ce4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37648/0 (socket says 172.21.15.86:37648) 2024-04-20T21:27:25.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.456+0000 7f942ce00640 1 -- 172.21.15.86:0/488192449 learned_addr learned my addr 172.21.15.86:0/488192449 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:25.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.456+0000 7f9427400640 1 --2- 172.21.15.86:0/488192449 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94281629e0 0x7f942815c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:25.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.456+0000 7f942ce00640 1 -- 172.21.15.86:0/488192449 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94281629e0 msgr2=0x7f942815c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:25.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.456+0000 7f942ce00640 1 --2- 172.21.15.86:0/488192449 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94281629e0 0x7f942815c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:25.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.456+0000 7f942ce00640 1 -- 172.21.15.86:0/488192449 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94280b3810 msgr2=0x7f9428164be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:25.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.456+0000 7f942ce00640 1 --2- 172.21.15.86:0/488192449 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94280b3810 0x7f9428164be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:25.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.456+0000 7f942ce00640 1 -- 172.21.15.86:0/488192449 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94281f9610 con 0x7f94281f9100 2024-04-20T21:27:25.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.456+0000 7f942ce00640 1 --2- 172.21.15.86:0/488192449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94281f9100 0x7f94280ce4b0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f94200097a0 tx=0x7f94200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:25.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.457+0000 7f941fe00640 1 -- 172.21.15.86:0/488192449 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9420032630 con 0x7f94281f9100 2024-04-20T21:27:25.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.457+0000 7f941fe00640 1 -- 172.21.15.86:0/488192449 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94200327f0 con 0x7f94281f9100 2024-04-20T21:27:25.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.457+0000 7f941fe00640 1 -- 172.21.15.86:0/488192449 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9420032be0 con 0x7f94281f9100 2024-04-20T21:27:25.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.457+0000 7f942d800640 1 -- 172.21.15.86:0/488192449 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f942815ba20 con 0x7f94281f9100 2024-04-20T21:27:25.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.457+0000 7f942d800640 1 -- 172.21.15.86:0/488192449 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f942815bfe0 con 0x7f94281f9100 2024-04-20T21:27:25.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.458+0000 7f942d800640 1 -- 172.21.15.86:0/488192449 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f942804fcf0 con 0x7f94281f9100 2024-04-20T21:27:25.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.459+0000 7f941fe00640 1 -- 172.21.15.86:0/488192449 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f942004d3f0 con 0x7f94281f9100 2024-04-20T21:27:25.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.460+0000 7f941fe00640 1 --2- 172.21.15.86:0/488192449 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f940406c8f0 0x7f940406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:25.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.460+0000 7f941fe00640 1 -- 172.21.15.86:0/488192449 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7f942002f030 con 0x7f94281f9100 2024-04-20T21:27:25.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.460+0000 7f9427e00640 1 --2- 172.21.15.86:0/488192449 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f940406c8f0 0x7f940406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:25.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.461+0000 7f9427e00640 1 --2- 172.21.15.86:0/488192449 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f940406c8f0 0x7f940406edb0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9418000f60 tx=0x7f94180078f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:25.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.466+0000 7f941fe00640 1 -- 172.21.15.86:0/488192449 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9420040070 con 0x7f94281f9100 2024-04-20T21:27:25.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.669+0000 7f942d800640 1 -- 172.21.15.86:0/488192449 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f94280720f0 con 0x7f94281f9100 2024-04-20T21:27:25.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.672+0000 7f941fe00640 1 -- 172.21.15.86:0/488192449 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+16973 (secure 0 0 0) 0x7f942007b450 con 0x7f94281f9100 2024-04-20T21:27:25.673 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:27:25.673 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":89,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:26:43.869364+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:24:55.964540+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-20T21:24:57.241527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.3e","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.144:6835/3981403733":"2024-04-21T21:26:43.869334+0000","172.21.15.144:6833/3981403733":"2024-04-21T21:26:43.869334+0000","172.21.15.144:6835/1740492227":"2024-04-21T21:25:20.040733+0000","172.21.15.144:6833/1740492227":"2024-04-21T21:25:20.040733+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:27:25.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.674+0000 7f942d800640 1 -- 172.21.15.86:0/488192449 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f940406c8f0 msgr2=0x7f940406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:25.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.674+0000 7f942d800640 1 --2- 172.21.15.86:0/488192449 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f940406c8f0 0x7f940406edb0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9418000f60 tx=0x7f94180078f0 comp rx=0 tx=0).stop 2024-04-20T21:27:25.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.674+0000 7f942d800640 1 -- 172.21.15.86:0/488192449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94281f9100 msgr2=0x7f94280ce4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:25.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.674+0000 7f942d800640 1 --2- 172.21.15.86:0/488192449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94281f9100 0x7f94280ce4b0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f94200097a0 tx=0x7f94200046c0 comp rx=0 tx=0).stop 2024-04-20T21:27:25.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.674+0000 7f942d800640 1 -- 172.21.15.86:0/488192449 shutdown_connections 2024-04-20T21:27:25.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.674+0000 7f942d800640 1 --2- 172.21.15.86:0/488192449 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f940406c8f0 0x7f940406edb0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:25.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.674+0000 7f942d800640 1 --2- 172.21.15.86:0/488192449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94281f9100 0x7f94280ce4b0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:25.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.674+0000 7f942d800640 1 --2- 172.21.15.86:0/488192449 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f94281629e0 0x7f942815c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:25.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.674+0000 7f942d800640 1 --2- 172.21.15.86:0/488192449 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94280b3810 0x7f9428164be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:25.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.674+0000 7f942d800640 1 -- 172.21.15.86:0/488192449 >> 172.21.15.86:0/488192449 conn(0x7f94280d0170 msgr2=0x7f94280ada10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:25.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.674+0000 7f942d800640 1 -- 172.21.15.86:0/488192449 shutdown_connections 2024-04-20T21:27:25.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.675+0000 7f942d800640 1 -- 172.21.15.86:0/488192449 wait complete. 2024-04-20T21:27:25.693 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:24:55.964540+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:24:57.241527+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:27:25.693 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:27:25.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.894+0000 7fe0e4800640 1 Processor -- start 2024-04-20T21:27:25.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.894+0000 7fe0e4800640 1 -- start start 2024-04-20T21:27:25.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.896+0000 7fe0e4800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe0dc06bbf0 0x7fe0dc06e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:25.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.896+0000 7fe0e4800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe0dc146210 0x7fe0dc0cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:25.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.896+0000 7fe0e4800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0dc15cd40 con 0x7fe0dc06bfd0 2024-04-20T21:27:25.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.896+0000 7fe0e4800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe0dc15d000 con 0x7fe0dc06bbf0 2024-04-20T21:27:25.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.896+0000 7fe0e4800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fe0dc15d2e0 con 0x7fe0dc146210 2024-04-20T21:27:25.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.896+0000 7fe0e1e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe0dc06bbf0 0x7fe0dc06e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:25.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.896+0000 7fe0e1400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe0dc06bfd0 0x7fe0dc145b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:35800/0 (socket says 172.21.15.86:35800) 2024-04-20T21:27:25.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.896+0000 7fe0e1400640 1 -- 172.21.15.86:0/233287997 learned_addr learned my addr 172.21.15.86:0/233287997 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:27:25.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.897+0000 7fe0e2800640 1 --2- 172.21.15.86:0/233287997 >> v2:172.21.15.144:3300/0 conn(0x7fe0dc146210 0x7fe0dc0cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:25.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.897+0000 7fe0e1e00640 1 -- 172.21.15.86:0/233287997 >> v2:172.21.15.144:3300/0 conn(0x7fe0dc146210 msgr2=0x7fe0dc0cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:25.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.897+0000 7fe0e1e00640 1 --2- 172.21.15.86:0/233287997 >> v2:172.21.15.144:3300/0 conn(0x7fe0dc146210 0x7fe0dc0cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:25.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.897+0000 7fe0e1e00640 1 -- 172.21.15.86:0/233287997 >> v1:172.21.15.86:6789/0 conn(0x7fe0dc06bfd0 legacy=0x7fe0dc145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:25.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.897+0000 7fe0e0a00640 1 -- 172.21.15.86:0/233287997 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3924826381 0 0) 0x7fe0dc15cd40 con 0x7fe0dc06bfd0 2024-04-20T21:27:25.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.897+0000 7fe0e1e00640 1 -- 172.21.15.86:0/233287997 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0dc15d5c0 con 0x7fe0dc06bbf0 2024-04-20T21:27:25.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.898+0000 7fe0e1e00640 1 --2- 172.21.15.86:0/233287997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0dc06bbf0 0x7fe0dc06e000 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc06f6c0 tx=0x7fe0d002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45fd07c9b621ab22 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:25.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.899+0000 7fe0e0a00640 1 -- 172.21.15.86:0/233287997 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d0035020 con 0x7fe0dc06bbf0 2024-04-20T21:27:25.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.899+0000 7fe0e0a00640 1 -- 172.21.15.86:0/233287997 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0d0031830 con 0x7fe0dc06bbf0 2024-04-20T21:27:25.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.899+0000 7fe0e0a00640 1 -- 172.21.15.86:0/233287997 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d0031c00 con 0x7fe0dc06bbf0 2024-04-20T21:27:25.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.899+0000 7fe0e4800640 1 -- 172.21.15.86:0/233287997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0dc06bbf0 msgr2=0x7fe0dc06e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:25.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.899+0000 7fe0e4800640 1 --2- 172.21.15.86:0/233287997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0dc06bbf0 0x7fe0dc06e000 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc06f6c0 tx=0x7fe0d002b8d0 comp rx=0 tx=0).stop 2024-04-20T21:27:25.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.899+0000 7fe0e4800640 1 -- 172.21.15.86:0/233287997 shutdown_connections 2024-04-20T21:27:25.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.899+0000 7fe0e4800640 1 --2- 172.21.15.86:0/233287997 >> v2:172.21.15.144:3300/0 conn(0x7fe0dc146210 0x7fe0dc0cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:25.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.899+0000 7fe0e4800640 1 --2- 172.21.15.86:0/233287997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0dc06bbf0 0x7fe0dc06e000 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:25.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.899+0000 7fe0e4800640 1 -- 172.21.15.86:0/233287997 >> 172.21.15.86:0/233287997 conn(0x7fe0dc0b2110 msgr2=0x7fe0dc0b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:25.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.899+0000 7fe0e4800640 1 -- 172.21.15.86:0/233287997 shutdown_connections 2024-04-20T21:27:25.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.899+0000 7fe0e4800640 1 -- 172.21.15.86:0/233287997 wait complete. 2024-04-20T21:27:25.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.901+0000 7fe0e4800640 1 Processor -- start 2024-04-20T21:27:25.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.901+0000 7fe0e4800640 1 -- start start 2024-04-20T21:27:25.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.902+0000 7fe0e4800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0dc06bbf0 0x7fe0dc15c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:25.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.902+0000 7fe0e4800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0dc06bfd0 0x7fe0dc0c7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:25.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.902+0000 7fe0e4800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0dc146210 0x7fe0dc0c8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:25.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.902+0000 7fe0e4800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe0dc0cdc00 con 0x7fe0dc06bbf0 2024-04-20T21:27:25.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.902+0000 7fe0e4800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe0dc0cda80 con 0x7fe0dc06bfd0 2024-04-20T21:27:25.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.902+0000 7fe0e4800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe0dc0cdd80 con 0x7fe0dc146210 2024-04-20T21:27:25.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.902+0000 7fe0e1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0dc06bbf0 0x7fe0dc15c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:25.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.903+0000 7fe0e1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0dc06bbf0 0x7fe0dc15c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60114/0 (socket says 172.21.15.86:60114) 2024-04-20T21:27:25.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.903+0000 7fe0e1e00640 1 -- 172.21.15.86:0/1344772091 learned_addr learned my addr 172.21.15.86:0/1344772091 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:25.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.903+0000 7fe0e1400640 1 --2- 172.21.15.86:0/1344772091 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0dc06bfd0 0x7fe0dc0c7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:25.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.903+0000 7fe0e1e00640 1 -- 172.21.15.86:0/1344772091 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0dc146210 msgr2=0x7fe0dc0c8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:25.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.903+0000 7fe0e2800640 1 --2- 172.21.15.86:0/1344772091 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0dc146210 0x7fe0dc0c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:25.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.903+0000 7fe0e1e00640 1 --2- 172.21.15.86:0/1344772091 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0dc146210 0x7fe0dc0c8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:25.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.903+0000 7fe0e1e00640 1 -- 172.21.15.86:0/1344772091 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0dc06bfd0 msgr2=0x7fe0dc0c7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:25.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.903+0000 7fe0e1e00640 1 --2- 172.21.15.86:0/1344772091 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0dc06bfd0 0x7fe0dc0c7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:25.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.903+0000 7fe0e1e00640 1 -- 172.21.15.86:0/1344772091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0dc15c5e0 con 0x7fe0dc06bbf0 2024-04-20T21:27:25.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.903+0000 7fe0e1400640 1 --2- 172.21.15.86:0/1344772091 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0dc06bfd0 0x7fe0dc0c7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:27:25.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.903+0000 7fe0e1e00640 1 --2- 172.21.15.86:0/1344772091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0dc06bbf0 0x7fe0dc15c0a0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fe0d0000c00 tx=0x7fe0d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:25.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.904+0000 7fe0d6000640 1 -- 172.21.15.86:0/1344772091 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d0035040 con 0x7fe0dc06bbf0 2024-04-20T21:27:25.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.904+0000 7fe0d6000640 1 -- 172.21.15.86:0/1344772091 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0d0004230 con 0x7fe0dc06bbf0 2024-04-20T21:27:25.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.904+0000 7fe0d6000640 1 -- 172.21.15.86:0/1344772091 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d003a700 con 0x7fe0dc06bbf0 2024-04-20T21:27:25.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.904+0000 7fe0e4800640 1 -- 172.21.15.86:0/1344772091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0dc13f830 con 0x7fe0dc06bbf0 2024-04-20T21:27:25.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.904+0000 7fe0e4800640 1 -- 172.21.15.86:0/1344772091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0dc13fdf0 con 0x7fe0dc06bbf0 2024-04-20T21:27:25.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.905+0000 7fe0e4800640 1 -- 172.21.15.86:0/1344772091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0dc0689e0 con 0x7fe0dc06bbf0 2024-04-20T21:27:25.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.906+0000 7fe0d6000640 1 -- 172.21.15.86:0/1344772091 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe0d0038020 con 0x7fe0dc06bbf0 2024-04-20T21:27:25.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.907+0000 7fe0d6000640 1 --2- 172.21.15.86:0/1344772091 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe0c006c8f0 0x7fe0c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:25.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.907+0000 7fe0d6000640 1 -- 172.21.15.86:0/1344772091 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7fe0d00ada40 con 0x7fe0dc06bbf0 2024-04-20T21:27:25.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.907+0000 7fe0e1400640 1 --2- 172.21.15.86:0/1344772091 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe0c006c8f0 0x7fe0c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:25.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.908+0000 7fe0e1400640 1 --2- 172.21.15.86:0/1344772091 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe0c006c8f0 0x7fe0c006edb0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc146160 tx=0x7fe0cc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:25.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:25.912+0000 7fe0d6000640 1 -- 172.21.15.86:0/1344772091 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe0dc15c5e0 con 0x7fe0dc06bbf0 2024-04-20T21:27:26.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.109+0000 7fe0e4800640 1 -- 172.21.15.86:0/1344772091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe0dc06c780 con 0x7fe0dc06bbf0 2024-04-20T21:27:26.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.110+0000 7fe0d6000640 1 -- 172.21.15.86:0/1344772091 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7fe0d007bd00 con 0x7fe0dc06bbf0 2024-04-20T21:27:26.110 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:27:26.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.112+0000 7fe0e4800640 1 -- 172.21.15.86:0/1344772091 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe0c006c8f0 msgr2=0x7fe0c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:26.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.112+0000 7fe0e4800640 1 --2- 172.21.15.86:0/1344772091 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe0c006c8f0 0x7fe0c006edb0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc146160 tx=0x7fe0cc009040 comp rx=0 tx=0).stop 2024-04-20T21:27:26.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.112+0000 7fe0e4800640 1 -- 172.21.15.86:0/1344772091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0dc06bbf0 msgr2=0x7fe0dc15c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:26.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.112+0000 7fe0e4800640 1 --2- 172.21.15.86:0/1344772091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0dc06bbf0 0x7fe0dc15c0a0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fe0d0000c00 tx=0x7fe0d00046c0 comp rx=0 tx=0).stop 2024-04-20T21:27:26.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.113+0000 7fe0e4800640 1 -- 172.21.15.86:0/1344772091 shutdown_connections 2024-04-20T21:27:26.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.113+0000 7fe0e4800640 1 --2- 172.21.15.86:0/1344772091 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe0c006c8f0 0x7fe0c006edb0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:26.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.113+0000 7fe0e4800640 1 --2- 172.21.15.86:0/1344772091 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe0dc146210 0x7fe0dc0c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:26.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.113+0000 7fe0e4800640 1 --2- 172.21.15.86:0/1344772091 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0dc06bfd0 0x7fe0dc0c7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:26.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.113+0000 7fe0e4800640 1 --2- 172.21.15.86:0/1344772091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0dc06bbf0 0x7fe0dc15c0a0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:26.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.113+0000 7fe0e4800640 1 -- 172.21.15.86:0/1344772091 >> 172.21.15.86:0/1344772091 conn(0x7fe0dc0b2110 msgr2=0x7fe0dc06e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:26.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.113+0000 7fe0e4800640 1 -- 172.21.15.86:0/1344772091 shutdown_connections 2024-04-20T21:27:26.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.113+0000 7fe0e4800640 1 -- 172.21.15.86:0/1344772091 wait complete. 2024-04-20T21:27:26.132 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:27:26.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.330+0000 7feb21200640 1 Processor -- start 2024-04-20T21:27:26.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.331+0000 7feb21200640 1 -- start start 2024-04-20T21:27:26.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.332+0000 7feb21200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7feb1c15c5c0 0x7feb1c0cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:26.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.332+0000 7feb21200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7feb1c0b66b0 0x7feb1c06ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:26.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.332+0000 7feb21200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb1c15c9a0 con 0x7feb1c0afff0 2024-04-20T21:27:26.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.332+0000 7feb21200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7feb1c0d01e0 con 0x7feb1c15c5c0 2024-04-20T21:27:26.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.332+0000 7feb21200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7feb1c0d04c0 con 0x7feb1c0b66b0 2024-04-20T21:27:26.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.333+0000 7feb1b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7feb1c15c5c0 0x7feb1c0cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:26.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.333+0000 7feb1b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7feb1c15c5c0 0x7feb1c0cfb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60124/0 (socket says 172.21.15.86:60124) 2024-04-20T21:27:26.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.333+0000 7feb12a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7feb1c0afff0 0x7feb1c0b5fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:35802/0 (socket says 172.21.15.86:35802) 2024-04-20T21:27:26.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.333+0000 7feb1b400640 1 -- 172.21.15.86:0/4251993291 learned_addr learned my addr 172.21.15.86:0/4251993291 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:26.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.333+0000 7feb1b400640 1 -- 172.21.15.86:0/4251993291 >> v2:172.21.15.144:3300/0 conn(0x7feb1c0b66b0 msgr2=0x7feb1c06ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:26.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.333+0000 7feb1be00640 1 --2- 172.21.15.86:0/4251993291 >> v2:172.21.15.144:3300/0 conn(0x7feb1c0b66b0 0x7feb1c06ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:26.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.333+0000 7feb1b400640 1 --2- 172.21.15.86:0/4251993291 >> v2:172.21.15.144:3300/0 conn(0x7feb1c0b66b0 0x7feb1c06ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:26.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.333+0000 7feb1b400640 1 -- 172.21.15.86:0/4251993291 >> v1:172.21.15.86:6789/0 conn(0x7feb1c0afff0 legacy=0x7feb1c0b5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:26.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.333+0000 7feb1b400640 1 -- 172.21.15.86:0/4251993291 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb1c0d07a0 con 0x7feb1c15c5c0 2024-04-20T21:27:26.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.333+0000 7feb1aa00640 1 -- 172.21.15.86:0/4251993291 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3025699514 0 0) 0x7feb1c15c9a0 con 0x7feb1c0afff0 2024-04-20T21:27:26.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.334+0000 7feb1b400640 1 --2- 172.21.15.86:0/4251993291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb1c15c5c0 0x7feb1c0cfb20 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7feb1c0b3d40 tx=0x7feb0c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b23c79ca77f7510 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:26.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.335+0000 7feb1aa00640 1 -- 172.21.15.86:0/4251993291 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0c02ca80 con 0x7feb1c15c5c0 2024-04-20T21:27:26.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.335+0000 7feb1aa00640 1 -- 172.21.15.86:0/4251993291 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb0c02cc20 con 0x7feb1c15c5c0 2024-04-20T21:27:26.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.335+0000 7feb1aa00640 1 -- 172.21.15.86:0/4251993291 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0c0343f0 con 0x7feb1c15c5c0 2024-04-20T21:27:26.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.335+0000 7feb21200640 1 -- 172.21.15.86:0/4251993291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb1c15c5c0 msgr2=0x7feb1c0cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:26.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.335+0000 7feb21200640 1 --2- 172.21.15.86:0/4251993291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb1c15c5c0 0x7feb1c0cfb20 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7feb1c0b3d40 tx=0x7feb0c02bee0 comp rx=0 tx=0).stop 2024-04-20T21:27:26.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.336+0000 7feb21200640 1 -- 172.21.15.86:0/4251993291 shutdown_connections 2024-04-20T21:27:26.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.336+0000 7feb21200640 1 --2- 172.21.15.86:0/4251993291 >> v2:172.21.15.144:3300/0 conn(0x7feb1c0b66b0 0x7feb1c06ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:26.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.336+0000 7feb21200640 1 --2- 172.21.15.86:0/4251993291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb1c15c5c0 0x7feb1c0cfb20 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:26.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.336+0000 7feb21200640 1 -- 172.21.15.86:0/4251993291 >> 172.21.15.86:0/4251993291 conn(0x7feb1c0733d0 msgr2=0x7feb1c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:26.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.336+0000 7feb21200640 1 -- 172.21.15.86:0/4251993291 shutdown_connections 2024-04-20T21:27:26.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.336+0000 7feb21200640 1 -- 172.21.15.86:0/4251993291 wait complete. 2024-04-20T21:27:26.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.337+0000 7feb21200640 1 Processor -- start 2024-04-20T21:27:26.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.338+0000 7feb21200640 1 -- start start 2024-04-20T21:27:26.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.338+0000 7feb21200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb1c0afff0 0x7feb1c1f3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:26.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.339+0000 7feb21200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb1c0b66b0 0x7feb1c0b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:26.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.339+0000 7feb21200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb1c15c5c0 0x7feb1c0b5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:26.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.339+0000 7feb21200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c06f2c0 con 0x7feb1c0b66b0 2024-04-20T21:27:26.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.339+0000 7feb21200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c06f140 con 0x7feb1c15c5c0 2024-04-20T21:27:26.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.339+0000 7feb21200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7feb1c06f440 con 0x7feb1c0afff0 2024-04-20T21:27:26.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.339+0000 7feb12a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb1c0b66b0 0x7feb1c0b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:26.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.339+0000 7feb12a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb1c0b66b0 0x7feb1c0b4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60126/0 (socket says 172.21.15.86:60126) 2024-04-20T21:27:26.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.339+0000 7feb12a00640 1 -- 172.21.15.86:0/656090910 learned_addr learned my addr 172.21.15.86:0/656090910 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:26.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.339+0000 7feb1b400640 1 --2- 172.21.15.86:0/656090910 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb1c0afff0 0x7feb1c1f3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:26.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.339+0000 7feb1be00640 1 --2- 172.21.15.86:0/656090910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb1c15c5c0 0x7feb1c0b5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:26.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.339+0000 7feb12a00640 1 -- 172.21.15.86:0/656090910 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb1c0afff0 msgr2=0x7feb1c1f3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:26.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.339+0000 7feb12a00640 1 --2- 172.21.15.86:0/656090910 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb1c0afff0 0x7feb1c1f3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:26.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.339+0000 7feb12a00640 1 -- 172.21.15.86:0/656090910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb1c15c5c0 msgr2=0x7feb1c0b5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:26.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.339+0000 7feb12a00640 1 --2- 172.21.15.86:0/656090910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb1c15c5c0 0x7feb1c0b5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:26.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.339+0000 7feb12a00640 1 -- 172.21.15.86:0/656090910 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb1c1f8340 con 0x7feb1c0b66b0 2024-04-20T21:27:26.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.339+0000 7feb1b400640 1 --2- 172.21.15.86:0/656090910 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb1c0afff0 0x7feb1c1f3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:27:26.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.340+0000 7feb12a00640 1 --2- 172.21.15.86:0/656090910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb1c0b66b0 0x7feb1c0b4de0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7feb0400c4b0 tx=0x7feb0400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:26.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.340+0000 7feb13e00640 1 -- 172.21.15.86:0/656090910 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb04018070 con 0x7feb1c0b66b0 2024-04-20T21:27:26.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.340+0000 7feb13e00640 1 -- 172.21.15.86:0/656090910 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb04007480 con 0x7feb1c0b66b0 2024-04-20T21:27:26.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.340+0000 7feb13e00640 1 -- 172.21.15.86:0/656090910 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb04004820 con 0x7feb1c0b66b0 2024-04-20T21:27:26.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.340+0000 7feb21200640 1 -- 172.21.15.86:0/656090910 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb1c14a560 con 0x7feb1c0b66b0 2024-04-20T21:27:26.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.340+0000 7feb21200640 1 -- 172.21.15.86:0/656090910 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb1c14ab20 con 0x7feb1c0b66b0 2024-04-20T21:27:26.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.341+0000 7feb21200640 1 -- 172.21.15.86:0/656090910 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb1c14a350 con 0x7feb1c0b66b0 2024-04-20T21:27:26.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.342+0000 7feb13e00640 1 -- 172.21.15.86:0/656090910 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feb040049e0 con 0x7feb1c0b66b0 2024-04-20T21:27:26.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.343+0000 7feb13e00640 1 --2- 172.21.15.86:0/656090910 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7feaf806c8f0 0x7feaf806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:26.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.343+0000 7feb13e00640 1 -- 172.21.15.86:0/656090910 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7feb0408aff0 con 0x7feb1c0b66b0 2024-04-20T21:27:26.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.343+0000 7feb1b400640 1 --2- 172.21.15.86:0/656090910 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7feaf806c8f0 0x7feaf806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:26.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.344+0000 7feb1b400640 1 --2- 172.21.15.86:0/656090910 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7feaf806c8f0 0x7feaf806edb0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7feb0c035000 tx=0x7feb0c030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:26.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.348+0000 7feb13e00640 1 -- 172.21.15.86:0/656090910 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7feb04059390 con 0x7feb1c0b66b0 2024-04-20T21:27:26.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.549+0000 7feb21200640 1 -- 172.21.15.86:0/656090910 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7feb1c04fd90 con 0x7feb1c0b66b0 2024-04-20T21:27:26.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.550+0000 7feb13e00640 1 -- 172.21.15.86:0/656090910 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7feb04058c90 con 0x7feb1c0b66b0 2024-04-20T21:27:26.550 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-20T21:27:26.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.552+0000 7feb21200640 1 -- 172.21.15.86:0/656090910 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7feaf806c8f0 msgr2=0x7feaf806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:26.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.552+0000 7feb21200640 1 --2- 172.21.15.86:0/656090910 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7feaf806c8f0 0x7feaf806edb0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7feb0c035000 tx=0x7feb0c030000 comp rx=0 tx=0).stop 2024-04-20T21:27:26.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.552+0000 7feb21200640 1 -- 172.21.15.86:0/656090910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb1c0b66b0 msgr2=0x7feb1c0b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:26.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.552+0000 7feb21200640 1 --2- 172.21.15.86:0/656090910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb1c0b66b0 0x7feb1c0b4de0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7feb0400c4b0 tx=0x7feb0400c970 comp rx=0 tx=0).stop 2024-04-20T21:27:26.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.552+0000 7feb21200640 1 -- 172.21.15.86:0/656090910 shutdown_connections 2024-04-20T21:27:26.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.552+0000 7feb21200640 1 --2- 172.21.15.86:0/656090910 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7feaf806c8f0 0x7feaf806edb0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:26.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.552+0000 7feb21200640 1 --2- 172.21.15.86:0/656090910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb1c15c5c0 0x7feb1c0b5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:26.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.552+0000 7feb21200640 1 --2- 172.21.15.86:0/656090910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb1c0b66b0 0x7feb1c0b4de0 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:26.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.552+0000 7feb21200640 1 --2- 172.21.15.86:0/656090910 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feb1c0afff0 0x7feb1c1f3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:26.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.552+0000 7feb21200640 1 -- 172.21.15.86:0/656090910 >> 172.21.15.86:0/656090910 conn(0x7feb1c0733d0 msgr2=0x7feb1c0af280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:26.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.552+0000 7feb21200640 1 -- 172.21.15.86:0/656090910 shutdown_connections 2024-04-20T21:27:26.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.553+0000 7feb21200640 1 -- 172.21.15.86:0/656090910 wait complete. 2024-04-20T21:27:26.571 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:27:26.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.762+0000 7fc4e7a00640 1 Processor -- start 2024-04-20T21:27:26.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.762+0000 7fc4e7a00640 1 -- start start 2024-04-20T21:27:26.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.763+0000 7fc4e7a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc4e006c090 0x7fc4e00cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:26.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.763+0000 7fc4e7a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4e00b4ba0 0x7fc4e00b5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:26.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.763+0000 7fc4e7a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4e0156110 con 0x7fc4e0155d30 2024-04-20T21:27:26.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.763+0000 7fc4e7a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc4e00ce460 con 0x7fc4e006c090 2024-04-20T21:27:26.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.763+0000 7fc4e7a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc4e00ce740 con 0x7fc4e00b4ba0 2024-04-20T21:27:26.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.764+0000 7fc4dfe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc4e006c090 0x7fc4e00cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:26.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.764+0000 7fc4dfe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc4e006c090 0x7fc4e00cf500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60128/0 (socket says 172.21.15.86:60128) 2024-04-20T21:27:26.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.764+0000 7fc4dfe00640 1 -- 172.21.15.86:0/679949182 learned_addr learned my addr 172.21.15.86:0/679949182 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:26.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.764+0000 7fc4e5a00640 1 --2- 172.21.15.86:0/679949182 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4e00b4ba0 0x7fc4e00b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:26.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.764+0000 7fc4dfe00640 1 -- 172.21.15.86:0/679949182 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4e00b4ba0 msgr2=0x7fc4e00b5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:26.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.764+0000 7fc4dfe00640 1 --2- 172.21.15.86:0/679949182 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4e00b4ba0 0x7fc4e00b5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:26.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.764+0000 7fc4dfe00640 1 -- 172.21.15.86:0/679949182 >> v1:172.21.15.144:6789/0 conn(0x7fc4e0155d30 legacy=0x7fc4e006b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:26.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.764+0000 7fc4dfe00640 1 -- 172.21.15.86:0/679949182 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4e00cea20 con 0x7fc4e006c090 2024-04-20T21:27:26.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.764+0000 7fc4e5a00640 1 --2- 172.21.15.86:0/679949182 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4e00b4ba0 0x7fc4e00b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:27:26.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.766+0000 7fc4dfe00640 1 --2- 172.21.15.86:0/679949182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4e006c090 0x7fc4e00cf500 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc009870 tx=0x7fc4cc02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e6bc232af86ff12 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:26.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.766+0000 7fc4df400640 1 -- 172.21.15.86:0/679949182 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc0317c0 con 0x7fc4e006c090 2024-04-20T21:27:26.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.766+0000 7fc4df400640 1 -- 172.21.15.86:0/679949182 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4cc037070 con 0x7fc4e006c090 2024-04-20T21:27:26.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.766+0000 7fc4df400640 1 -- 172.21.15.86:0/679949182 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc031b80 con 0x7fc4e006c090 2024-04-20T21:27:26.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.767+0000 7fc4e7a00640 1 -- 172.21.15.86:0/679949182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4e006c090 msgr2=0x7fc4e00cf500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:26.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.767+0000 7fc4e7a00640 1 --2- 172.21.15.86:0/679949182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4e006c090 0x7fc4e00cf500 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc009870 tx=0x7fc4cc02b860 comp rx=0 tx=0).stop 2024-04-20T21:27:26.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.767+0000 7fc4e7a00640 1 -- 172.21.15.86:0/679949182 shutdown_connections 2024-04-20T21:27:26.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.767+0000 7fc4e7a00640 1 --2- 172.21.15.86:0/679949182 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4e00b4ba0 0x7fc4e00b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:26.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.767+0000 7fc4e7a00640 1 --2- 172.21.15.86:0/679949182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4e006c090 0x7fc4e00cf500 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:26.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.767+0000 7fc4e7a00640 1 -- 172.21.15.86:0/679949182 >> 172.21.15.86:0/679949182 conn(0x7fc4e00b2110 msgr2=0x7fc4e00b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:26.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.767+0000 7fc4e7a00640 1 -- 172.21.15.86:0/679949182 shutdown_connections 2024-04-20T21:27:26.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.767+0000 7fc4e7a00640 1 -- 172.21.15.86:0/679949182 wait complete. 2024-04-20T21:27:26.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.769+0000 7fc4e7a00640 1 Processor -- start 2024-04-20T21:27:26.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.769+0000 7fc4e7a00640 1 -- start start 2024-04-20T21:27:26.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.770+0000 7fc4e7a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4e006c090 0x7fc4e014ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:26.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.770+0000 7fc4e7a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4e00b4ba0 0x7fc4e014b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:26.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.770+0000 7fc4e7a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4e0155d30 0x7fc4e014bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:26.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.770+0000 7fc4e7a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e00b5570 con 0x7fc4e006c090 2024-04-20T21:27:26.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.770+0000 7fc4e7a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e00cefe0 con 0x7fc4e0155d30 2024-04-20T21:27:26.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.770+0000 7fc4e7a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc4e00b56f0 con 0x7fc4e00b4ba0 2024-04-20T21:27:26.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.770+0000 7fc4e5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4e006c090 0x7fc4e014ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:26.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.770+0000 7fc4e5000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4e006c090 0x7fc4e014ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60134/0 (socket says 172.21.15.86:60134) 2024-04-20T21:27:26.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.770+0000 7fc4e5000640 1 -- 172.21.15.86:0/2020591828 learned_addr learned my addr 172.21.15.86:0/2020591828 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:26.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.770+0000 7fc4dfe00640 1 --2- 172.21.15.86:0/2020591828 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4e00b4ba0 0x7fc4e014b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:26.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.770+0000 7fc4e5000640 1 -- 172.21.15.86:0/2020591828 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4e00b4ba0 msgr2=0x7fc4e014b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:26.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.770+0000 7fc4e5a00640 1 --2- 172.21.15.86:0/2020591828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4e0155d30 0x7fc4e014bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:26.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.770+0000 7fc4e5000640 1 --2- 172.21.15.86:0/2020591828 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4e00b4ba0 0x7fc4e014b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:26.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.770+0000 7fc4e5000640 1 -- 172.21.15.86:0/2020591828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4e0155d30 msgr2=0x7fc4e014bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:26.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.770+0000 7fc4e5000640 1 --2- 172.21.15.86:0/2020591828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4e0155d30 0x7fc4e014bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:26.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.770+0000 7fc4e5000640 1 -- 172.21.15.86:0/2020591828 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4e01504b0 con 0x7fc4e006c090 2024-04-20T21:27:26.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.770+0000 7fc4dfe00640 1 --2- 172.21.15.86:0/2020591828 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4e00b4ba0 0x7fc4e014b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:27:26.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.771+0000 7fc4e5000640 1 --2- 172.21.15.86:0/2020591828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4e006c090 0x7fc4e014ab20 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fc4d400c400 tx=0x7fc4d400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:26.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.771+0000 7fc4dcc00640 1 -- 172.21.15.86:0/2020591828 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d40052d0 con 0x7fc4e006c090 2024-04-20T21:27:26.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.771+0000 7fc4dcc00640 1 -- 172.21.15.86:0/2020591828 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4d4005490 con 0x7fc4e006c090 2024-04-20T21:27:26.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.771+0000 7fc4dcc00640 1 -- 172.21.15.86:0/2020591828 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4007260 con 0x7fc4e006c090 2024-04-20T21:27:26.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.771+0000 7fc4e7a00640 1 -- 172.21.15.86:0/2020591828 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4e014e7b0 con 0x7fc4e006c090 2024-04-20T21:27:26.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.771+0000 7fc4e7a00640 1 -- 172.21.15.86:0/2020591828 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4e0150750 con 0x7fc4e006c090 2024-04-20T21:27:26.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.772+0000 7fc4e7a00640 1 -- 172.21.15.86:0/2020591828 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4e00689e0 con 0x7fc4e006c090 2024-04-20T21:27:26.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.774+0000 7fc4dcc00640 1 -- 172.21.15.86:0/2020591828 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc4d4007400 con 0x7fc4e006c090 2024-04-20T21:27:26.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.775+0000 7fc4dcc00640 1 --2- 172.21.15.86:0/2020591828 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc4c006c920 0x7fc4c006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:26.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.775+0000 7fc4dcc00640 1 -- 172.21.15.86:0/2020591828 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7fc4d408b1d0 con 0x7fc4e006c090 2024-04-20T21:27:26.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.775+0000 7fc4dfe00640 1 --2- 172.21.15.86:0/2020591828 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc4c006c920 0x7fc4c006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:26.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.776+0000 7fc4dfe00640 1 --2- 172.21.15.86:0/2020591828 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc4c006c920 0x7fc4c006ede0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc036000 tx=0x7fc4cc008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:26.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.780+0000 7fc4dcc00640 1 -- 172.21.15.86:0/2020591828 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc4d4059490 con 0x7fc4e006c090 2024-04-20T21:27:26.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.981+0000 7fc4e7a00640 1 -- 172.21.15.86:0/2020591828 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc4e006cd90 con 0x7fc4e006c090 2024-04-20T21:27:26.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.981+0000 7fc4dcc00640 1 -- 172.21.15.86:0/2020591828 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7fc4d4058d90 con 0x7fc4e006c090 2024-04-20T21:27:26.981 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-20T21:27:26.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.984+0000 7fc4e7a00640 1 -- 172.21.15.86:0/2020591828 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc4c006c920 msgr2=0x7fc4c006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:26.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.984+0000 7fc4e7a00640 1 --2- 172.21.15.86:0/2020591828 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc4c006c920 0x7fc4c006ede0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc036000 tx=0x7fc4cc008030 comp rx=0 tx=0).stop 2024-04-20T21:27:26.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.984+0000 7fc4e7a00640 1 -- 172.21.15.86:0/2020591828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4e006c090 msgr2=0x7fc4e014ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:26.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.984+0000 7fc4e7a00640 1 --2- 172.21.15.86:0/2020591828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4e006c090 0x7fc4e014ab20 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fc4d400c400 tx=0x7fc4d400c8c0 comp rx=0 tx=0).stop 2024-04-20T21:27:26.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.984+0000 7fc4e7a00640 1 -- 172.21.15.86:0/2020591828 shutdown_connections 2024-04-20T21:27:26.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.984+0000 7fc4e7a00640 1 --2- 172.21.15.86:0/2020591828 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc4c006c920 0x7fc4c006ede0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:26.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.984+0000 7fc4e7a00640 1 --2- 172.21.15.86:0/2020591828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc4e0155d30 0x7fc4e014bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:26.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.984+0000 7fc4e7a00640 1 --2- 172.21.15.86:0/2020591828 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc4e00b4ba0 0x7fc4e014b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:26.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.984+0000 7fc4e7a00640 1 --2- 172.21.15.86:0/2020591828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4e006c090 0x7fc4e014ab20 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:26.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.984+0000 7fc4e7a00640 1 -- 172.21.15.86:0/2020591828 >> 172.21.15.86:0/2020591828 conn(0x7fc4e00b2110 msgr2=0x7fc4e015e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:26.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.985+0000 7fc4e7a00640 1 -- 172.21.15.86:0/2020591828 shutdown_connections 2024-04-20T21:27:26.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:26.985+0000 7fc4e7a00640 1 -- 172.21.15.86:0/2020591828 wait complete. 2024-04-20T21:27:27.003 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T21:27:27.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.203+0000 7f0ead400640 1 Processor -- start 2024-04-20T21:27:27.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.204+0000 7f0ead400640 1 -- start start 2024-04-20T21:27:27.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.205+0000 7f0ead400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ea815e690 0x7f0ea806ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:27.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.205+0000 7f0ead400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0ea80d0ee0 0x7f0ea80d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:27.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.205+0000 7f0ead400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ea80d33b0 con 0x7f0ea80d25a0 2024-04-20T21:27:27.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.205+0000 7f0ead400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f0ea80d1880 con 0x7f0ea80d0ee0 2024-04-20T21:27:27.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.205+0000 7f0ead400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0ea80d3690 con 0x7f0ea815e690 2024-04-20T21:27:27.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.205+0000 7f0ea7400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0ea80d0ee0 0x7f0ea80d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:27.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.205+0000 7f0ea7400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0ea80d0ee0 0x7f0ea80d1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60148/0 (socket says 172.21.15.86:60148) 2024-04-20T21:27:27.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.205+0000 7f0ea7400640 1 -- 172.21.15.86:0/1590160427 learned_addr learned my addr 172.21.15.86:0/1590160427 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:27.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.206+0000 7f0ea7e00640 1 --2- 172.21.15.86:0/1590160427 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ea815e690 0x7f0ea806ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:27.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.206+0000 7f0ea7400640 1 -- 172.21.15.86:0/1590160427 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ea815e690 msgr2=0x7f0ea806ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:27.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.206+0000 7f0ea7400640 1 --2- 172.21.15.86:0/1590160427 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ea815e690 0x7f0ea806ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:27.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.206+0000 7f0ea7400640 1 -- 172.21.15.86:0/1590160427 >> v1:172.21.15.144:6789/0 conn(0x7f0ea80d25a0 legacy=0x7f0ea80d29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:27.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.206+0000 7f0ea7400640 1 -- 172.21.15.86:0/1590160427 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ea80d3970 con 0x7f0ea80d0ee0 2024-04-20T21:27:27.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.207+0000 7f0ea7400640 1 --2- 172.21.15.86:0/1590160427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ea80d0ee0 0x7f0ea80d1340 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f0e90009ea0 tx=0x7f0e9002bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8109d477d492e0f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:27.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.208+0000 7f0ea6a00640 1 -- 172.21.15.86:0/1590160427 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9002cab0 con 0x7f0ea80d0ee0 2024-04-20T21:27:27.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.208+0000 7f0ea6a00640 1 -- 172.21.15.86:0/1590160427 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e9002cc50 con 0x7f0ea80d0ee0 2024-04-20T21:27:27.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.208+0000 7f0ea6a00640 1 -- 172.21.15.86:0/1590160427 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e900343f0 con 0x7f0ea80d0ee0 2024-04-20T21:27:27.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.208+0000 7f0ead400640 1 -- 172.21.15.86:0/1590160427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ea80d0ee0 msgr2=0x7f0ea80d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:27.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.208+0000 7f0ead400640 1 --2- 172.21.15.86:0/1590160427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ea80d0ee0 0x7f0ea80d1340 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f0e90009ea0 tx=0x7f0e9002bf10 comp rx=0 tx=0).stop 2024-04-20T21:27:27.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.208+0000 7f0ead400640 1 -- 172.21.15.86:0/1590160427 shutdown_connections 2024-04-20T21:27:27.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.208+0000 7f0ead400640 1 --2- 172.21.15.86:0/1590160427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ea80d0ee0 0x7f0ea80d1340 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:27.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.208+0000 7f0ead400640 1 --2- 172.21.15.86:0/1590160427 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ea815e690 0x7f0ea806ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:27.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.208+0000 7f0ead400640 1 -- 172.21.15.86:0/1590160427 >> 172.21.15.86:0/1590160427 conn(0x7f0ea809d8a0 msgr2=0x7f0ea808bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:27.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.209+0000 7f0ead400640 1 -- 172.21.15.86:0/1590160427 shutdown_connections 2024-04-20T21:27:27.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.209+0000 7f0ead400640 1 -- 172.21.15.86:0/1590160427 wait complete. 2024-04-20T21:27:27.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.211+0000 7f0ead400640 1 Processor -- start 2024-04-20T21:27:27.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.211+0000 7f0ead400640 1 -- start start 2024-04-20T21:27:27.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.212+0000 7f0ead400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ea80d0ee0 0x7f0ea8151ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:27.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.212+0000 7f0ead400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ea80d25a0 0x7f0ea8158700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:27.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.212+0000 7f0ead400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ea815e690 0x7f0ea8158c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:27.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.212+0000 7f0ead400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea806d360 con 0x7f0ea815e690 2024-04-20T21:27:27.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.212+0000 7f0ead400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea80cef00 con 0x7f0ea80d0ee0 2024-04-20T21:27:27.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.212+0000 7f0ead400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0ea80d30d0 con 0x7f0ea80d25a0 2024-04-20T21:27:27.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.212+0000 7f0eaca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ea815e690 0x7f0ea8158c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:27.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.213+0000 7f0eaca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ea815e690 0x7f0ea8158c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60152/0 (socket says 172.21.15.86:60152) 2024-04-20T21:27:27.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.213+0000 7f0eaca00640 1 -- 172.21.15.86:0/2300189489 learned_addr learned my addr 172.21.15.86:0/2300189489 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:27.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.213+0000 7f0ea7e00640 1 --2- 172.21.15.86:0/2300189489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ea80d0ee0 0x7f0ea8151ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:27.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.213+0000 7f0ea7400640 1 --2- 172.21.15.86:0/2300189489 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ea80d25a0 0x7f0ea8158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:27.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.213+0000 7f0eaca00640 1 -- 172.21.15.86:0/2300189489 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ea80d25a0 msgr2=0x7f0ea8158700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:27.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.213+0000 7f0eaca00640 1 --2- 172.21.15.86:0/2300189489 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ea80d25a0 0x7f0ea8158700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:27.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.213+0000 7f0eaca00640 1 -- 172.21.15.86:0/2300189489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ea80d0ee0 msgr2=0x7f0ea8151ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:27.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.213+0000 7f0eaca00640 1 --2- 172.21.15.86:0/2300189489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ea80d0ee0 0x7f0ea8151ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:27.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.213+0000 7f0eaca00640 1 -- 172.21.15.86:0/2300189489 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ea815b250 con 0x7f0ea815e690 2024-04-20T21:27:27.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.213+0000 7f0ea7400640 1 --2- 172.21.15.86:0/2300189489 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ea80d25a0 0x7f0ea8158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:27:27.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.213+0000 7f0eaca00640 1 --2- 172.21.15.86:0/2300189489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ea815e690 0x7f0ea8158c40 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f0ea000c410 tx=0x7f0ea000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:27.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.213+0000 7f0e9fe00640 1 -- 172.21.15.86:0/2300189489 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0011790 con 0x7f0ea815e690 2024-04-20T21:27:27.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.214+0000 7f0e9fe00640 1 -- 172.21.15.86:0/2300189489 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ea0017070 con 0x7f0ea815e690 2024-04-20T21:27:27.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.214+0000 7f0ead400640 1 -- 172.21.15.86:0/2300189489 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ea815b530 con 0x7f0ea815e690 2024-04-20T21:27:27.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.214+0000 7f0e9fe00640 1 -- 172.21.15.86:0/2300189489 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0011bb0 con 0x7f0ea815e690 2024-04-20T21:27:27.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.214+0000 7f0ead400640 1 -- 172.21.15.86:0/2300189489 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ea815fff0 con 0x7f0ea815e690 2024-04-20T21:27:27.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.215+0000 7f0ead400640 1 -- 172.21.15.86:0/2300189489 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ea804fcf0 con 0x7f0ea815e690 2024-04-20T21:27:27.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.216+0000 7f0e9fe00640 1 -- 172.21.15.86:0/2300189489 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0ea001c460 con 0x7f0ea815e690 2024-04-20T21:27:27.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.217+0000 7f0e9fe00640 1 --2- 172.21.15.86:0/2300189489 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0e8406c8f0 0x7f0e8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:27.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.217+0000 7f0e9fe00640 1 -- 172.21.15.86:0/2300189489 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7f0ea008b770 con 0x7f0ea815e690 2024-04-20T21:27:27.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.217+0000 7f0ea7e00640 1 --2- 172.21.15.86:0/2300189489 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0e8406c8f0 0x7f0e8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:27.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.218+0000 7f0ea7e00640 1 --2- 172.21.15.86:0/2300189489 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0e8406c8f0 0x7f0e8406edb0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f0e98006fd0 tx=0x7f0e98008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:27.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.221+0000 7f0e9fe00640 1 -- 172.21.15.86:0/2300189489 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0ea0021020 con 0x7f0ea815e690 2024-04-20T21:27:27.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.433+0000 7f0ead400640 1 -- 172.21.15.86:0/2300189489 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0ea806cc00 con 0x7f0ea815e690 2024-04-20T21:27:27.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.434+0000 7f0e9fe00640 1 -- 172.21.15.86:0/2300189489 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v89) ==== 80+0+248 (secure 0 0 0) 0x7f0ea0059b60 con 0x7f0ea815e690 2024-04-20T21:27:27.434 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.144:6835/3981403733 2024-04-21T21:26:43.869334+0000 2024-04-20T21:27:27.434 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.144:6833/3981403733 2024-04-21T21:26:43.869334+0000 2024-04-20T21:27:27.434 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.144:6835/1740492227 2024-04-21T21:25:20.040733+0000 2024-04-20T21:27:27.434 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.144:6833/1740492227 2024-04-21T21:25:20.040733+0000 2024-04-20T21:27:27.434 INFO:teuthology.orchestra.run.smithi086.stderr:listed 4 entries 2024-04-20T21:27:27.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.436+0000 7f0ead400640 1 -- 172.21.15.86:0/2300189489 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0e8406c8f0 msgr2=0x7f0e8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:27.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.436+0000 7f0ead400640 1 --2- 172.21.15.86:0/2300189489 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0e8406c8f0 0x7f0e8406edb0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f0e98006fd0 tx=0x7f0e98008040 comp rx=0 tx=0).stop 2024-04-20T21:27:27.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.436+0000 7f0ead400640 1 -- 172.21.15.86:0/2300189489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ea815e690 msgr2=0x7f0ea8158c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:27.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.436+0000 7f0ead400640 1 --2- 172.21.15.86:0/2300189489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ea815e690 0x7f0ea8158c40 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f0ea000c410 tx=0x7f0ea000c8d0 comp rx=0 tx=0).stop 2024-04-20T21:27:27.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.437+0000 7f0ead400640 1 -- 172.21.15.86:0/2300189489 shutdown_connections 2024-04-20T21:27:27.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.437+0000 7f0ead400640 1 --2- 172.21.15.86:0/2300189489 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0e8406c8f0 0x7f0e8406edb0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:27.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.437+0000 7f0ead400640 1 --2- 172.21.15.86:0/2300189489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ea815e690 0x7f0ea8158c40 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:27.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.437+0000 7f0ead400640 1 --2- 172.21.15.86:0/2300189489 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ea80d25a0 0x7f0ea8158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:27.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.437+0000 7f0ead400640 1 --2- 172.21.15.86:0/2300189489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ea80d0ee0 0x7f0ea8151ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:27.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.437+0000 7f0ead400640 1 -- 172.21.15.86:0/2300189489 >> 172.21.15.86:0/2300189489 conn(0x7f0ea809d8a0 msgr2=0x7f0ea806dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:27.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.437+0000 7f0ead400640 1 -- 172.21.15.86:0/2300189489 shutdown_connections 2024-04-20T21:27:27.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:27.437+0000 7f0ead400640 1 -- 172.21.15.86:0/2300189489 wait complete. 2024-04-20T21:27:27.455 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi086.front.sepia.ceph.com... 2024-04-20T21:27:27.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:27:27.455 DEBUG:teuthology.orchestra.run.smithi086:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-20T21:27:27.531 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[56538]: fuse finished with error 0 and tester_r 0 2024-04-20T21:27:27.536 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-20T21:27:27.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:27:27.536 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-20T21:27:27.715 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T21:27:27.716 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:27:27.716 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:27:27.772 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T21:27:27.772 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2024-04-20T21:27:27.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:27:27.773 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-20T21:27:27.825 INFO:teuthology.orchestra.run.smithi144.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-20T21:27:27.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:27:27.830 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T21:27:27.831 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:27:27.831 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:27:27.888 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052731,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248996k,nr_inodes=812249,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T21:27:27.888 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-20T21:27:27.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:27:27.888 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-20T21:27:27.939 INFO:teuthology.orchestra.run.smithi086.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-20T21:27:27.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:27:27.940 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T21:27:27.940 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:27:27.940 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:27:27.997 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T21:27:27.997 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2024-04-20T21:27:27.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:27:27.997 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-20T21:27:28.050 INFO:teuthology.orchestra.run.smithi144.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-20T21:27:28.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:27:28.052 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:27:28.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.263+0000 7fa477200640 1 Processor -- start 2024-04-20T21:27:28.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.263+0000 7fa477200640 1 -- start start 2024-04-20T21:27:28.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.264+0000 7fa477200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa47806f2a0 0x7fa478167410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:28.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.264+0000 7fa477200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa478160890 con 0x7fa478156ae0 2024-04-20T21:27:28.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.264+0000 7fa477200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa478160b70 con 0x7fa47815d6a0 2024-04-20T21:27:28.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.264+0000 7fa477200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa47806db20 con 0x7fa47806f2a0 2024-04-20T21:27:28.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.265+0000 7fa476800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa47806f2a0 0x7fa478167410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:28.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.265+0000 7fa476800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa47806f2a0 0x7fa478167410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60160/0 (socket says 172.21.15.86:60160) 2024-04-20T21:27:28.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.265+0000 7fa475e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fa478156ae0 0x7fa478160080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:35804/0 (socket says 172.21.15.86:35804) 2024-04-20T21:27:28.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.265+0000 7fa476800640 1 -- 172.21.15.86:0/2594483665 learned_addr learned my addr 172.21.15.86:0/2594483665 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:28.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.265+0000 7fa476800640 1 -- 172.21.15.86:0/2594483665 >> v1:172.21.15.144:6789/0 conn(0x7fa47815d6a0 legacy=0x7fa4780ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:28.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.265+0000 7fa476800640 1 -- 172.21.15.86:0/2594483665 >> v1:172.21.15.86:6789/0 conn(0x7fa478156ae0 legacy=0x7fa478160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:28.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.265+0000 7fa476800640 1 -- 172.21.15.86:0/2594483665 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa478160e50 con 0x7fa47806f2a0 2024-04-20T21:27:28.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.265+0000 7fa475400640 1 -- 172.21.15.86:0/2594483665 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4059226721 0 0) 0x7fa478160890 con 0x7fa478156ae0 2024-04-20T21:27:28.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.266+0000 7fa476800640 1 --2- 172.21.15.86:0/2594483665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa47806f2a0 0x7fa478167410 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fa470009870 tx=0x7fa47002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0a191855c2e7586 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:28.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.267+0000 7fa475400640 1 -- 172.21.15.86:0/2594483665 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa470034070 con 0x7fa47806f2a0 2024-04-20T21:27:28.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.267+0000 7fa475400640 1 -- 172.21.15.86:0/2594483665 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa470030860 con 0x7fa47806f2a0 2024-04-20T21:27:28.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.267+0000 7fa475400640 1 -- 172.21.15.86:0/2594483665 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa470030c30 con 0x7fa47806f2a0 2024-04-20T21:27:28.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.267+0000 7fa477200640 1 -- 172.21.15.86:0/2594483665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa47806f2a0 msgr2=0x7fa478167410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:28.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.267+0000 7fa477200640 1 --2- 172.21.15.86:0/2594483665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa47806f2a0 0x7fa478167410 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fa470009870 tx=0x7fa47002b9a0 comp rx=0 tx=0).stop 2024-04-20T21:27:28.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.267+0000 7fa477200640 1 -- 172.21.15.86:0/2594483665 shutdown_connections 2024-04-20T21:27:28.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.267+0000 7fa477200640 1 --2- 172.21.15.86:0/2594483665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa47806f2a0 0x7fa478167410 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:28.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.267+0000 7fa477200640 1 -- 172.21.15.86:0/2594483665 >> 172.21.15.86:0/2594483665 conn(0x7fa4780d0170 msgr2=0x7fa4780d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:28.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.267+0000 7fa477200640 1 -- 172.21.15.86:0/2594483665 shutdown_connections 2024-04-20T21:27:28.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.268+0000 7fa477200640 1 -- 172.21.15.86:0/2594483665 wait complete. 2024-04-20T21:27:28.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.269+0000 7fa477200640 1 Processor -- start 2024-04-20T21:27:28.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.269+0000 7fa477200640 1 -- start start 2024-04-20T21:27:28.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.270+0000 7fa477200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa47806f2a0 0x7fa4781f7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:28.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.270+0000 7fa477200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa478156ae0 0x7fa478093bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:28.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.270+0000 7fa477200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa47815d6a0 0x7fa478097820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:28.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.270+0000 7fa477200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa4780ad7f0 con 0x7fa47806f2a0 2024-04-20T21:27:28.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.270+0000 7fa477200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa478156ec0 con 0x7fa47815d6a0 2024-04-20T21:27:28.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.270+0000 7fa477200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa4780ad970 con 0x7fa478156ae0 2024-04-20T21:27:28.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.270+0000 7fa475e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa47806f2a0 0x7fa4781f7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:28.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.271+0000 7fa475e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa47806f2a0 0x7fa4781f7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60172/0 (socket says 172.21.15.86:60172) 2024-04-20T21:27:28.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.271+0000 7fa475e00640 1 -- 172.21.15.86:0/2910398020 learned_addr learned my addr 172.21.15.86:0/2910398020 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:28.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.271+0000 7fa46d400640 1 --2- 172.21.15.86:0/2910398020 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa478156ae0 0x7fa478093bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:28.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.271+0000 7fa475e00640 1 -- 172.21.15.86:0/2910398020 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa478156ae0 msgr2=0x7fa478093bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:28.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.271+0000 7fa476800640 1 --2- 172.21.15.86:0/2910398020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa47815d6a0 0x7fa478097820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:28.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.271+0000 7fa475e00640 1 --2- 172.21.15.86:0/2910398020 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa478156ae0 0x7fa478093bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:28.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.271+0000 7fa475e00640 1 -- 172.21.15.86:0/2910398020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa47815d6a0 msgr2=0x7fa478097820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:28.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.271+0000 7fa475e00640 1 --2- 172.21.15.86:0/2910398020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa47815d6a0 0x7fa478097820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:28.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.271+0000 7fa475e00640 1 -- 172.21.15.86:0/2910398020 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4781f8010 con 0x7fa47806f2a0 2024-04-20T21:27:28.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.271+0000 7fa46d400640 1 --2- 172.21.15.86:0/2910398020 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa478156ae0 0x7fa478093bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:27:28.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.271+0000 7fa475e00640 1 --2- 172.21.15.86:0/2910398020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa47806f2a0 0x7fa4781f7a30 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fa46800c4d0 tx=0x7fa46800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:28.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.272+0000 7fa46ea00640 1 -- 172.21.15.86:0/2910398020 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa468018070 con 0x7fa47806f2a0 2024-04-20T21:27:28.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.272+0000 7fa46ea00640 1 -- 172.21.15.86:0/2910398020 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa468007500 con 0x7fa47806f2a0 2024-04-20T21:27:28.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.272+0000 7fa46ea00640 1 -- 172.21.15.86:0/2910398020 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa468004a20 con 0x7fa47806f2a0 2024-04-20T21:27:28.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.272+0000 7fa477200640 1 -- 172.21.15.86:0/2910398020 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4781f82f0 con 0x7fa47806f2a0 2024-04-20T21:27:28.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.272+0000 7fa477200640 1 -- 172.21.15.86:0/2910398020 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4781f88b0 con 0x7fa47806f2a0 2024-04-20T21:27:28.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.273+0000 7fa477200640 1 -- 172.21.15.86:0/2910398020 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa47804fcf0 con 0x7fa47806f2a0 2024-04-20T21:27:28.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.274+0000 7fa46ea00640 1 -- 172.21.15.86:0/2910398020 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa468004bc0 con 0x7fa47806f2a0 2024-04-20T21:27:28.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.275+0000 7fa46ea00640 1 --2- 172.21.15.86:0/2910398020 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa45406c8f0 0x7fa45406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:28.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.275+0000 7fa46ea00640 1 -- 172.21.15.86:0/2910398020 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7fa46808aed0 con 0x7fa47806f2a0 2024-04-20T21:27:28.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.275+0000 7fa46d400640 1 --2- 172.21.15.86:0/2910398020 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa45406c8f0 0x7fa45406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:28.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.276+0000 7fa46d400640 1 --2- 172.21.15.86:0/2910398020 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa45406c8f0 0x7fa45406edb0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fa478099c40 tx=0x7fa4600073a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:28.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.280+0000 7fa46ea00640 1 -- 172.21.15.86:0/2910398020 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa468059220 con 0x7fa47806f2a0 2024-04-20T21:27:28.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.518+0000 7fa477200640 1 -- 172.21.15.86:0/2910398020 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa47806fb40 con 0x7fa47806f2a0 2024-04-20T21:27:28.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.519+0000 7fa46ea00640 1 -- 172.21.15.86:0/2910398020 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5209 (secure 0 0 0) 0x7fa468058b20 con 0x7fa47806f2a0 2024-04-20T21:27:28.519 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:27:28.519 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":73,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5887,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2433619638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2433619638},{"type":"v1","addr":"172.21.15.144:6835","nonce":2433619638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68},{"gid":7506,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/4013863257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4013863257},{"type":"v1","addr":"172.21.15.86:6836","nonce":4013863257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":7509,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2044883073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2044883073},{"type":"v1","addr":"172.21.15.86:6837","nonce":2044883073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:24:58.299257+0000","modified":"2024-04-20T21:26:48.911892+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7518},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7518":{"gid":7518,"name":"d","rank":0,"incarnation":70,"state":"up:active","state_seq":32,"addr":"172.21.15.144:6834/2351381466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2351381466},{"type":"v1","addr":"172.21.15.144:6834","nonce":2351381466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7518,"qdb_cluster":[7518]},"id":6}]} 2024-04-20T21:27:28.519 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 73 2024-04-20T21:27:28.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.521+0000 7fa477200640 1 -- 172.21.15.86:0/2910398020 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa45406c8f0 msgr2=0x7fa45406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:28.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.521+0000 7fa477200640 1 --2- 172.21.15.86:0/2910398020 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa45406c8f0 0x7fa45406edb0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fa478099c40 tx=0x7fa4600073a0 comp rx=0 tx=0).stop 2024-04-20T21:27:28.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.521+0000 7fa477200640 1 -- 172.21.15.86:0/2910398020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa47806f2a0 msgr2=0x7fa4781f7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:28.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.521+0000 7fa477200640 1 --2- 172.21.15.86:0/2910398020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa47806f2a0 0x7fa4781f7a30 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fa46800c4d0 tx=0x7fa46800c990 comp rx=0 tx=0).stop 2024-04-20T21:27:28.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.522+0000 7fa477200640 1 -- 172.21.15.86:0/2910398020 shutdown_connections 2024-04-20T21:27:28.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.522+0000 7fa477200640 1 --2- 172.21.15.86:0/2910398020 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa45406c8f0 0x7fa45406edb0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:28.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.522+0000 7fa477200640 1 --2- 172.21.15.86:0/2910398020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa47815d6a0 0x7fa478097820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:28.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.522+0000 7fa477200640 1 --2- 172.21.15.86:0/2910398020 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa478156ae0 0x7fa478093bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:28.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.522+0000 7fa477200640 1 --2- 172.21.15.86:0/2910398020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa47806f2a0 0x7fa4781f7a30 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:28.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.522+0000 7fa477200640 1 -- 172.21.15.86:0/2910398020 >> 172.21.15.86:0/2910398020 conn(0x7fa4780d0170 msgr2=0x7fa4780afd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:28.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.522+0000 7fa477200640 1 -- 172.21.15.86:0/2910398020 shutdown_connections 2024-04-20T21:27:28.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.522+0000 7fa477200640 1 -- 172.21.15.86:0/2910398020 wait complete. 2024-04-20T21:27:28.541 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:27:28.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.742+0000 7f7edb400640 1 Processor -- start 2024-04-20T21:27:28.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.742+0000 7f7edb400640 1 -- start start 2024-04-20T21:27:28.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.743+0000 7f7edb400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7ed4072410 0x7f7ed415cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:28.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.743+0000 7f7edb400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ed406eb30 con 0x7f7ed406e6d0 2024-04-20T21:27:28.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.743+0000 7f7edb400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ed406dc30 con 0x7f7ed4146130 2024-04-20T21:27:28.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.743+0000 7f7edb400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f7ed40cd8e0 con 0x7f7ed4072410 2024-04-20T21:27:28.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.743+0000 7f7ed9400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7ed4072410 0x7f7ed415cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:28.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.743+0000 7f7ecbe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f7ed406e6d0 0x7f7ed4071d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:35806/0 (socket says 172.21.15.86:35806) 2024-04-20T21:27:28.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.743+0000 7f7ecbe00640 1 -- 172.21.15.86:0/2518685046 learned_addr learned my addr 172.21.15.86:0/2518685046 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:27:28.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.743+0000 7f7ed9400640 1 -- 172.21.15.86:0/2518685046 >> v1:172.21.15.144:6789/0 conn(0x7f7ed4146130 legacy=0x7f7ed40ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:28.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.743+0000 7f7ed9400640 1 -- 172.21.15.86:0/2518685046 >> v1:172.21.15.86:6789/0 conn(0x7f7ed406e6d0 legacy=0x7f7ed4071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:28.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.743+0000 7f7ed9400640 1 -- 172.21.15.86:0/2518685046 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed40cdbc0 con 0x7f7ed4072410 2024-04-20T21:27:28.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.743+0000 7f7ed3e00640 1 -- 172.21.15.86:0/2518685046 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 607027607 0 0) 0x7f7ebc001110 con 0x7f7ed406e6d0 2024-04-20T21:27:28.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.745+0000 7f7ed9400640 1 --2- 172.21.15.86:0/2518685046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ed4072410 0x7f7ed415cc20 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc009d90 tx=0x7f7ecc02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dab3d2273ca1c9f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:28.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.745+0000 7f7ed3e00640 1 -- 172.21.15.86:0/2518685046 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc039070 con 0x7f7ed4072410 2024-04-20T21:27:28.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.745+0000 7f7ed3e00640 1 -- 172.21.15.86:0/2518685046 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ecc02cd00 con 0x7f7ed4072410 2024-04-20T21:27:28.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.745+0000 7f7ed3e00640 1 -- 172.21.15.86:0/2518685046 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc034660 con 0x7f7ed4072410 2024-04-20T21:27:28.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.745+0000 7f7edb400640 1 -- 172.21.15.86:0/2518685046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ed4072410 msgr2=0x7f7ed415cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:28.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.745+0000 7f7edb400640 1 --2- 172.21.15.86:0/2518685046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ed4072410 0x7f7ed415cc20 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc009d90 tx=0x7f7ecc02bf50 comp rx=0 tx=0).stop 2024-04-20T21:27:28.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.745+0000 7f7edb400640 1 -- 172.21.15.86:0/2518685046 shutdown_connections 2024-04-20T21:27:28.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.745+0000 7f7edb400640 1 --2- 172.21.15.86:0/2518685046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ed4072410 0x7f7ed415cc20 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:28.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.745+0000 7f7edb400640 1 -- 172.21.15.86:0/2518685046 >> 172.21.15.86:0/2518685046 conn(0x7f7ed40b2360 msgr2=0x7f7ed40b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:28.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.746+0000 7f7edb400640 1 -- 172.21.15.86:0/2518685046 shutdown_connections 2024-04-20T21:27:28.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.746+0000 7f7edb400640 1 -- 172.21.15.86:0/2518685046 wait complete. 2024-04-20T21:27:28.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.747+0000 7f7edb400640 1 Processor -- start 2024-04-20T21:27:28.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.747+0000 7f7edb400640 1 -- start start 2024-04-20T21:27:28.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.748+0000 7f7edb400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ed406e6d0 0x7f7ed415f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:28.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.748+0000 7f7edb400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ed4072410 0x7f7ed41307e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:28.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.748+0000 7f7edb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ed4146130 0x7f7ed415bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:28.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.748+0000 7f7edb400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed406f150 con 0x7f7ed4146130 2024-04-20T21:27:28.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.748+0000 7f7edb400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed406e1b0 con 0x7f7ed4072410 2024-04-20T21:27:28.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.748+0000 7f7edb400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7ed40cd4e0 con 0x7f7ed406e6d0 2024-04-20T21:27:28.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.749+0000 7f7ed9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ed4146130 0x7f7ed415bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:28.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.749+0000 7f7ed9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ed4146130 0x7f7ed415bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60200/0 (socket says 172.21.15.86:60200) 2024-04-20T21:27:28.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.749+0000 7f7ed9400640 1 -- 172.21.15.86:0/2532809747 learned_addr learned my addr 172.21.15.86:0/2532809747 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:28.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.749+0000 7f7ecbe00640 1 --2- 172.21.15.86:0/2532809747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ed4072410 0x7f7ed41307e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:28.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.749+0000 7f7ed9400640 1 -- 172.21.15.86:0/2532809747 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ed406e6d0 msgr2=0x7f7ed415f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:28.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.749+0000 7f7ed9400640 1 --2- 172.21.15.86:0/2532809747 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ed406e6d0 0x7f7ed415f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:28.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.749+0000 7f7ed9400640 1 -- 172.21.15.86:0/2532809747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ed4072410 msgr2=0x7f7ed41307e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:28.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.749+0000 7f7ed9400640 1 --2- 172.21.15.86:0/2532809747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ed4072410 0x7f7ed41307e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:28.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.749+0000 7f7ed9400640 1 -- 172.21.15.86:0/2532809747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed41319d0 con 0x7f7ed4146130 2024-04-20T21:27:28.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.749+0000 7f7ed9400640 1 --2- 172.21.15.86:0/2532809747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ed4146130 0x7f7ed415bb00 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc009d90 tx=0x7f7ecc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:28.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.750+0000 7f7ed1600640 1 -- 172.21.15.86:0/2532809747 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc044020 con 0x7f7ed4146130 2024-04-20T21:27:28.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.750+0000 7f7ed1600640 1 -- 172.21.15.86:0/2532809747 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ecc03f470 con 0x7f7ed4146130 2024-04-20T21:27:28.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.750+0000 7f7edb400640 1 -- 172.21.15.86:0/2532809747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ed415d170 con 0x7f7ed4146130 2024-04-20T21:27:28.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.750+0000 7f7ed1600640 1 -- 172.21.15.86:0/2532809747 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc039040 con 0x7f7ed4146130 2024-04-20T21:27:28.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.750+0000 7f7edb400640 1 -- 172.21.15.86:0/2532809747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ed415da10 con 0x7f7ed4146130 2024-04-20T21:27:28.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.751+0000 7f7edb400640 1 -- 172.21.15.86:0/2532809747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ed415d560 con 0x7f7ed4146130 2024-04-20T21:27:28.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.752+0000 7f7ed1600640 1 -- 172.21.15.86:0/2532809747 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7ecc047020 con 0x7f7ed4146130 2024-04-20T21:27:28.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.753+0000 7f7ed1600640 1 --2- 172.21.15.86:0/2532809747 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7eb006c8f0 0x7f7eb006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:28.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.754+0000 7f7ed1600640 1 -- 172.21.15.86:0/2532809747 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7f7ecc0ae940 con 0x7f7ed4146130 2024-04-20T21:27:28.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.754+0000 7f7ed8a00640 1 --2- 172.21.15.86:0/2532809747 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7eb006c8f0 0x7f7eb006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:28.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.754+0000 7f7ed8a00640 1 --2- 172.21.15.86:0/2532809747 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7eb006c8f0 0x7f7eb006edb0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4003cc0 tx=0x7f7ec40079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:28.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.759+0000 7f7ed1600640 1 -- 172.21.15.86:0/2532809747 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7ecc04e030 con 0x7f7ed4146130 2024-04-20T21:27:28.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.962+0000 7f7edb400640 1 -- 172.21.15.86:0/2532809747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7ed40689e0 con 0x7f7ed4146130 2024-04-20T21:27:28.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.964+0000 7f7ed1600640 1 -- 172.21.15.86:0/2532809747 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+16973 (secure 0 0 0) 0x7f7ecc07cc00 con 0x7f7ed4146130 2024-04-20T21:27:28.964 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:27:28.964 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":89,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:26:43.869364+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:24:55.964540+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-20T21:24:57.241527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.3e","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.144:6835/3981403733":"2024-04-21T21:26:43.869334+0000","172.21.15.144:6833/3981403733":"2024-04-21T21:26:43.869334+0000","172.21.15.144:6835/1740492227":"2024-04-21T21:25:20.040733+0000","172.21.15.144:6833/1740492227":"2024-04-21T21:25:20.040733+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:27:28.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.967+0000 7f7edb400640 1 -- 172.21.15.86:0/2532809747 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7eb006c8f0 msgr2=0x7f7eb006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:28.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.967+0000 7f7edb400640 1 --2- 172.21.15.86:0/2532809747 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7eb006c8f0 0x7f7eb006edb0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4003cc0 tx=0x7f7ec40079b0 comp rx=0 tx=0).stop 2024-04-20T21:27:28.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.967+0000 7f7edb400640 1 -- 172.21.15.86:0/2532809747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ed4146130 msgr2=0x7f7ed415bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:28.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.967+0000 7f7edb400640 1 --2- 172.21.15.86:0/2532809747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ed4146130 0x7f7ed415bb00 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc009d90 tx=0x7f7ecc004a40 comp rx=0 tx=0).stop 2024-04-20T21:27:28.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.967+0000 7f7edb400640 1 -- 172.21.15.86:0/2532809747 shutdown_connections 2024-04-20T21:27:28.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.967+0000 7f7edb400640 1 --2- 172.21.15.86:0/2532809747 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7eb006c8f0 0x7f7eb006edb0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:28.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.967+0000 7f7edb400640 1 --2- 172.21.15.86:0/2532809747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ed4146130 0x7f7ed415bb00 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:28.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.967+0000 7f7edb400640 1 --2- 172.21.15.86:0/2532809747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7ed4072410 0x7f7ed41307e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:28.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.967+0000 7f7edb400640 1 --2- 172.21.15.86:0/2532809747 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7ed406e6d0 0x7f7ed415f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:28.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.968+0000 7f7edb400640 1 -- 172.21.15.86:0/2532809747 >> 172.21.15.86:0/2532809747 conn(0x7f7ed40b2360 msgr2=0x7f7ed406c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:28.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.968+0000 7f7edb400640 1 -- 172.21.15.86:0/2532809747 shutdown_connections 2024-04-20T21:27:28.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:28.968+0000 7f7edb400640 1 -- 172.21.15.86:0/2532809747 wait complete. 2024-04-20T21:27:28.987 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:24:55.964540+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:24:57.241527+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:27:28.988 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:27:29.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.187+0000 7f2616400640 1 Processor -- start 2024-04-20T21:27:29.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.187+0000 7f2616400640 1 -- start start 2024-04-20T21:27:29.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.187+0000 7f2616400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f26100af1f0 0x7f261006ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:29.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.187+0000 7f2616400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f261006d9c0 0x7f261015d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:29.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.188+0000 7f2616400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f261015da60 con 0x7f26100af5d0 2024-04-20T21:27:29.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.188+0000 7f2616400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f261006f460 con 0x7f26100af1f0 2024-04-20T21:27:29.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.188+0000 7f2616400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f261006de80 con 0x7f261006d9c0 2024-04-20T21:27:29.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.188+0000 7f2615000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f26100af1f0 0x7f261006ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:29.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.188+0000 7f2615000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f26100af1f0 0x7f261006ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60216/0 (socket says 172.21.15.86:60216) 2024-04-20T21:27:29.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.188+0000 7f2615a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f26100af5d0 0x7f261015de70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:35808/0 (socket says 172.21.15.86:35808) 2024-04-20T21:27:29.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.188+0000 7f2615000640 1 -- 172.21.15.86:0/1110339311 learned_addr learned my addr 172.21.15.86:0/1110339311 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:29.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.188+0000 7f260fe00640 1 --2- 172.21.15.86:0/1110339311 >> v2:172.21.15.144:3300/0 conn(0x7f261006d9c0 0x7f261015d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:29.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.189+0000 7f2615000640 1 -- 172.21.15.86:0/1110339311 >> v2:172.21.15.144:3300/0 conn(0x7f261006d9c0 msgr2=0x7f261015d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:29.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.189+0000 7f2615000640 1 --2- 172.21.15.86:0/1110339311 >> v2:172.21.15.144:3300/0 conn(0x7f261006d9c0 0x7f261015d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:29.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.189+0000 7f2615000640 1 -- 172.21.15.86:0/1110339311 >> v1:172.21.15.86:6789/0 conn(0x7f26100af5d0 legacy=0x7f261015de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:29.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.189+0000 7f260f400640 1 -- 172.21.15.86:0/1110339311 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3833396562 0 0) 0x7f2608001490 con 0x7f26100af5d0 2024-04-20T21:27:29.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.189+0000 7f2615000640 1 -- 172.21.15.86:0/1110339311 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f261006e160 con 0x7f26100af1f0 2024-04-20T21:27:29.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.190+0000 7f2615000640 1 --2- 172.21.15.86:0/1110339311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26100af1f0 0x7f261006ef20 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f26100d1380 tx=0x7f260402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3525e3d9c08fb56 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:29.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.190+0000 7f260f400640 1 -- 172.21.15.86:0/1110339311 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2604034070 con 0x7f26100af1f0 2024-04-20T21:27:29.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.190+0000 7f260f400640 1 -- 172.21.15.86:0/1110339311 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2604030860 con 0x7f26100af1f0 2024-04-20T21:27:29.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.190+0000 7f260f400640 1 -- 172.21.15.86:0/1110339311 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2604030c30 con 0x7f26100af1f0 2024-04-20T21:27:29.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.191+0000 7f2616400640 1 -- 172.21.15.86:0/1110339311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26100af1f0 msgr2=0x7f261006ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:29.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.191+0000 7f2616400640 1 --2- 172.21.15.86:0/1110339311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26100af1f0 0x7f261006ef20 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f26100d1380 tx=0x7f260402b9a0 comp rx=0 tx=0).stop 2024-04-20T21:27:29.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.191+0000 7f2616400640 1 -- 172.21.15.86:0/1110339311 shutdown_connections 2024-04-20T21:27:29.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.191+0000 7f2616400640 1 --2- 172.21.15.86:0/1110339311 >> v2:172.21.15.144:3300/0 conn(0x7f261006d9c0 0x7f261015d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:29.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.191+0000 7f2616400640 1 --2- 172.21.15.86:0/1110339311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26100af1f0 0x7f261006ef20 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:29.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.191+0000 7f2616400640 1 -- 172.21.15.86:0/1110339311 >> 172.21.15.86:0/1110339311 conn(0x7f26100b1f00 msgr2=0x7f26100b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:29.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.191+0000 7f2616400640 1 -- 172.21.15.86:0/1110339311 shutdown_connections 2024-04-20T21:27:29.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.191+0000 7f2616400640 1 -- 172.21.15.86:0/1110339311 wait complete. 2024-04-20T21:27:29.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.193+0000 7f2616400640 1 Processor -- start 2024-04-20T21:27:29.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.193+0000 7f2616400640 1 -- start start 2024-04-20T21:27:29.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.194+0000 7f2616400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f261006d9c0 0x7f26101f4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:29.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.194+0000 7f2616400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26100af1f0 0x7f261008f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:29.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.194+0000 7f2616400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26100af5d0 0x7f261015b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:29.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.194+0000 7f2616400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f261006ea00 con 0x7f261006d9c0 2024-04-20T21:27:29.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.194+0000 7f2616400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f261006e720 con 0x7f26100af5d0 2024-04-20T21:27:29.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.194+0000 7f2616400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f26100afa40 con 0x7f26100af1f0 2024-04-20T21:27:29.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.195+0000 7f2615000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f261006d9c0 0x7f26101f4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:29.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.195+0000 7f2615000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f261006d9c0 0x7f26101f4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60224/0 (socket says 172.21.15.86:60224) 2024-04-20T21:27:29.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.195+0000 7f2615000640 1 -- 172.21.15.86:0/2792446583 learned_addr learned my addr 172.21.15.86:0/2792446583 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:29.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.195+0000 7f260fe00640 1 --2- 172.21.15.86:0/2792446583 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26100af1f0 0x7f261008f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:29.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.195+0000 7f2615a00640 1 --2- 172.21.15.86:0/2792446583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26100af5d0 0x7f261015b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:29.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.195+0000 7f2615000640 1 -- 172.21.15.86:0/2792446583 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26100af1f0 msgr2=0x7f261008f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:29.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.195+0000 7f2615000640 1 --2- 172.21.15.86:0/2792446583 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26100af1f0 0x7f261008f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:29.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.195+0000 7f2615000640 1 -- 172.21.15.86:0/2792446583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26100af5d0 msgr2=0x7f261015b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:29.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.195+0000 7f2615000640 1 --2- 172.21.15.86:0/2792446583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26100af5d0 0x7f261015b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:29.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.195+0000 7f2615000640 1 -- 172.21.15.86:0/2792446583 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26101445c0 con 0x7f261006d9c0 2024-04-20T21:27:29.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.196+0000 7f2615000640 1 --2- 172.21.15.86:0/2792446583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f261006d9c0 0x7f26101f4450 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f2604000c00 tx=0x7f26040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:29.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.196+0000 7f260cc00640 1 -- 172.21.15.86:0/2792446583 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2604034040 con 0x7f261006d9c0 2024-04-20T21:27:29.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.196+0000 7f260cc00640 1 -- 172.21.15.86:0/2792446583 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f260403ab90 con 0x7f261006d9c0 2024-04-20T21:27:29.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.196+0000 7f260cc00640 1 -- 172.21.15.86:0/2792446583 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2604004030 con 0x7f261006d9c0 2024-04-20T21:27:29.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.196+0000 7f2616400640 1 -- 172.21.15.86:0/2792446583 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26101448a0 con 0x7f261006d9c0 2024-04-20T21:27:29.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.196+0000 7f2616400640 1 -- 172.21.15.86:0/2792446583 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2610144e60 con 0x7f261006d9c0 2024-04-20T21:27:29.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.197+0000 7f2616400640 1 -- 172.21.15.86:0/2792446583 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2610073a60 con 0x7f261006d9c0 2024-04-20T21:27:29.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.198+0000 7f260cc00640 1 -- 172.21.15.86:0/2792446583 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f260404b060 con 0x7f261006d9c0 2024-04-20T21:27:29.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.199+0000 7f260cc00640 1 --2- 172.21.15.86:0/2792446583 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f25f006c8f0 0x7f25f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:29.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.199+0000 7f260cc00640 1 -- 172.21.15.86:0/2792446583 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7f26040adb80 con 0x7f261006d9c0 2024-04-20T21:27:29.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.200+0000 7f260fe00640 1 --2- 172.21.15.86:0/2792446583 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f25f006c8f0 0x7f25f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:29.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.200+0000 7f260fe00640 1 --2- 172.21.15.86:0/2792446583 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f25f006c8f0 0x7f25f006edb0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f25fc006fd0 tx=0x7f25fc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:29.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.205+0000 7f260cc00640 1 -- 172.21.15.86:0/2792446583 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2610073a60 con 0x7f261006d9c0 2024-04-20T21:27:29.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.400+0000 7f2616400640 1 -- 172.21.15.86:0/2792446583 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2610156430 con 0x7f261006d9c0 2024-04-20T21:27:29.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.401+0000 7f260cc00640 1 -- 172.21.15.86:0/2792446583 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f260403f070 con 0x7f261006d9c0 2024-04-20T21:27:29.400 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:27:29.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.404+0000 7f2616400640 1 -- 172.21.15.86:0/2792446583 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f25f006c8f0 msgr2=0x7f25f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:29.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.404+0000 7f2616400640 1 --2- 172.21.15.86:0/2792446583 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f25f006c8f0 0x7f25f006edb0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f25fc006fd0 tx=0x7f25fc008040 comp rx=0 tx=0).stop 2024-04-20T21:27:29.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.404+0000 7f2616400640 1 -- 172.21.15.86:0/2792446583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f261006d9c0 msgr2=0x7f26101f4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:29.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.404+0000 7f2616400640 1 --2- 172.21.15.86:0/2792446583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f261006d9c0 0x7f26101f4450 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f2604000c00 tx=0x7f26040046c0 comp rx=0 tx=0).stop 2024-04-20T21:27:29.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.404+0000 7f2616400640 1 -- 172.21.15.86:0/2792446583 shutdown_connections 2024-04-20T21:27:29.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.404+0000 7f2616400640 1 --2- 172.21.15.86:0/2792446583 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f25f006c8f0 0x7f25f006edb0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:29.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.404+0000 7f2616400640 1 --2- 172.21.15.86:0/2792446583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26100af5d0 0x7f261015b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:29.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.404+0000 7f2616400640 1 --2- 172.21.15.86:0/2792446583 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26100af1f0 0x7f261008f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:29.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.404+0000 7f2616400640 1 --2- 172.21.15.86:0/2792446583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f261006d9c0 0x7f26101f4450 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:29.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.404+0000 7f2616400640 1 -- 172.21.15.86:0/2792446583 >> 172.21.15.86:0/2792446583 conn(0x7f26100b1f00 msgr2=0x7f2610073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:29.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.404+0000 7f2616400640 1 -- 172.21.15.86:0/2792446583 shutdown_connections 2024-04-20T21:27:29.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.404+0000 7f2616400640 1 -- 172.21.15.86:0/2792446583 wait complete. 2024-04-20T21:27:29.420 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:27:29.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.611+0000 7f8731800640 1 Processor -- start 2024-04-20T21:27:29.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.611+0000 7f8731800640 1 -- start start 2024-04-20T21:27:29.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.613+0000 7f8731800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f872c161c60 0x7f872c15f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:29.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.613+0000 7f8731800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f872c15ff10 0x7f872c06dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:29.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.613+0000 7f8731800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f872c071d40 0x7f872c0721d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:29.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.613+0000 7f8731800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f872c0aecd0 con 0x7f872c161c60 2024-04-20T21:27:29.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.613+0000 7f8731800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f872c06e2e0 con 0x7f872c15ff10 2024-04-20T21:27:29.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.613+0000 7f8731800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f872c06e580 con 0x7f872c071d40 2024-04-20T21:27:29.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.613+0000 7f872be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f872c161c60 0x7f872c15f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:29.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.613+0000 7f872be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f872c161c60 0x7f872c15f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60236/0 (socket says 172.21.15.86:60236) 2024-04-20T21:27:29.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.613+0000 7f872be00640 1 -- 172.21.15.86:0/2904229579 learned_addr learned my addr 172.21.15.86:0/2904229579 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:29.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.613+0000 7f8730e00640 1 --2- 172.21.15.86:0/2904229579 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f872c071d40 0x7f872c0721d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:29.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.613+0000 7f872b400640 1 --2- 172.21.15.86:0/2904229579 >> v2:172.21.15.144:3300/0 conn(0x7f872c15ff10 0x7f872c06dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:29.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.614+0000 7f872be00640 1 -- 172.21.15.86:0/2904229579 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f872c071d40 msgr2=0x7f872c0721d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:29.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.614+0000 7f872be00640 1 --2- 172.21.15.86:0/2904229579 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f872c071d40 0x7f872c0721d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:29.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.614+0000 7f872be00640 1 -- 172.21.15.86:0/2904229579 >> v2:172.21.15.144:3300/0 conn(0x7f872c15ff10 msgr2=0x7f872c06dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:29.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.614+0000 7f872be00640 1 --2- 172.21.15.86:0/2904229579 >> v2:172.21.15.144:3300/0 conn(0x7f872c15ff10 0x7f872c06dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:29.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.614+0000 7f872be00640 1 -- 172.21.15.86:0/2904229579 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f872c06e860 con 0x7f872c161c60 2024-04-20T21:27:29.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.615+0000 7f872be00640 1 --2- 172.21.15.86:0/2904229579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f872c161c60 0x7f872c15f990 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f872c06fe30 tx=0x7f871c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b77a946ec3703d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:29.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.616+0000 7f872aa00640 1 -- 172.21.15.86:0/2904229579 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871c035020 con 0x7f872c161c60 2024-04-20T21:27:29.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.616+0000 7f872aa00640 1 -- 172.21.15.86:0/2904229579 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f871c031870 con 0x7f872c161c60 2024-04-20T21:27:29.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.616+0000 7f872aa00640 1 -- 172.21.15.86:0/2904229579 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871c031c40 con 0x7f872c161c60 2024-04-20T21:27:29.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.616+0000 7f8731800640 1 -- 172.21.15.86:0/2904229579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f872c161c60 msgr2=0x7f872c15f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:29.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.616+0000 7f8731800640 1 --2- 172.21.15.86:0/2904229579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f872c161c60 0x7f872c15f990 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f872c06fe30 tx=0x7f871c02b910 comp rx=0 tx=0).stop 2024-04-20T21:27:29.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.616+0000 7f8731800640 1 -- 172.21.15.86:0/2904229579 shutdown_connections 2024-04-20T21:27:29.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.616+0000 7f8731800640 1 --2- 172.21.15.86:0/2904229579 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f872c071d40 0x7f872c0721d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:29.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.616+0000 7f8731800640 1 --2- 172.21.15.86:0/2904229579 >> v2:172.21.15.144:3300/0 conn(0x7f872c15ff10 0x7f872c06dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:29.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.616+0000 7f8731800640 1 --2- 172.21.15.86:0/2904229579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f872c161c60 0x7f872c15f990 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:29.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.616+0000 7f8731800640 1 -- 172.21.15.86:0/2904229579 >> 172.21.15.86:0/2904229579 conn(0x7f872c067cf0 msgr2=0x7f872c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:29.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.616+0000 7f8731800640 1 -- 172.21.15.86:0/2904229579 shutdown_connections 2024-04-20T21:27:29.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.616+0000 7f8731800640 1 -- 172.21.15.86:0/2904229579 wait complete. 2024-04-20T21:27:29.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.618+0000 7f8731800640 1 Processor -- start 2024-04-20T21:27:29.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.618+0000 7f8731800640 1 -- start start 2024-04-20T21:27:29.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.619+0000 7f8731800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f872c071d40 0x7f872c1f8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:29.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.619+0000 7f8731800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f872c15ff10 0x7f872c1371b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:29.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.619+0000 7f8731800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f872c161c60 0x7f872c0b5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:29.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.619+0000 7f8731800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f872c162040 con 0x7f872c071d40 2024-04-20T21:27:29.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.619+0000 7f8731800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f872c06ee20 con 0x7f872c15ff10 2024-04-20T21:27:29.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.619+0000 7f8731800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f872c160370 con 0x7f872c161c60 2024-04-20T21:27:29.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.619+0000 7f872be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f872c071d40 0x7f872c1f8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:29.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.619+0000 7f872be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f872c071d40 0x7f872c1f8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60244/0 (socket says 172.21.15.86:60244) 2024-04-20T21:27:29.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.619+0000 7f872be00640 1 -- 172.21.15.86:0/408378799 learned_addr learned my addr 172.21.15.86:0/408378799 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:29.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.620+0000 7f872b400640 1 --2- 172.21.15.86:0/408378799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f872c15ff10 0x7f872c1371b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:29.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.620+0000 7f872be00640 1 -- 172.21.15.86:0/408378799 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f872c161c60 msgr2=0x7f872c0b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:29.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.620+0000 7f8730e00640 1 --2- 172.21.15.86:0/408378799 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f872c161c60 0x7f872c0b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:29.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.620+0000 7f872be00640 1 --2- 172.21.15.86:0/408378799 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f872c161c60 0x7f872c0b5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:29.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.620+0000 7f872be00640 1 -- 172.21.15.86:0/408378799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f872c15ff10 msgr2=0x7f872c1371b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:29.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.620+0000 7f872be00640 1 --2- 172.21.15.86:0/408378799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f872c15ff10 0x7f872c1371b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:29.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.620+0000 7f872be00640 1 -- 172.21.15.86:0/408378799 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f872c1f8690 con 0x7f872c071d40 2024-04-20T21:27:29.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.620+0000 7f872b400640 1 --2- 172.21.15.86:0/408378799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f872c15ff10 0x7f872c1371b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:27:29.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.620+0000 7f872be00640 1 --2- 172.21.15.86:0/408378799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f872c071d40 0x7f872c1f8150 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f871c02b950 tx=0x7f871c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:29.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.620+0000 7f8723e00640 1 -- 172.21.15.86:0/408378799 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871c040070 con 0x7f872c071d40 2024-04-20T21:27:29.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.620+0000 7f8723e00640 1 -- 172.21.15.86:0/408378799 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f871c030d00 con 0x7f872c071d40 2024-04-20T21:27:29.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.620+0000 7f8723e00640 1 -- 172.21.15.86:0/408378799 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871c035040 con 0x7f872c071d40 2024-04-20T21:27:29.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.620+0000 7f8731800640 1 -- 172.21.15.86:0/408378799 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f872c1396e0 con 0x7f872c071d40 2024-04-20T21:27:29.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.621+0000 7f8731800640 1 -- 172.21.15.86:0/408378799 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f872c139ca0 con 0x7f872c071d40 2024-04-20T21:27:29.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.622+0000 7f8731800640 1 -- 172.21.15.86:0/408378799 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f872c04fd90 con 0x7f872c071d40 2024-04-20T21:27:29.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.623+0000 7f8723e00640 1 -- 172.21.15.86:0/408378799 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f871c04b060 con 0x7f872c071d40 2024-04-20T21:27:29.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.623+0000 7f8723e00640 1 --2- 172.21.15.86:0/408378799 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f870c06c8f0 0x7f870c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:29.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.624+0000 7f8723e00640 1 -- 172.21.15.86:0/408378799 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7f871c0ada20 con 0x7f872c071d40 2024-04-20T21:27:29.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.624+0000 7f872b400640 1 --2- 172.21.15.86:0/408378799 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f870c06c8f0 0x7f870c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:29.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.625+0000 7f872b400640 1 --2- 172.21.15.86:0/408378799 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f870c06c8f0 0x7f870c06edb0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f8718006fd0 tx=0x7f8718008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:29.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.628+0000 7f8723e00640 1 -- 172.21.15.86:0/408378799 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f872c1f8690 con 0x7f872c071d40 2024-04-20T21:27:29.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.813+0000 7f8731800640 1 -- 172.21.15.86:0/408378799 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f872c0b35d0 con 0x7f872c071d40 2024-04-20T21:27:29.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.815+0000 7f8723e00640 1 -- 172.21.15.86:0/408378799 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f871c07bd70 con 0x7f872c071d40 2024-04-20T21:27:29.814 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-20T21:27:29.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.817+0000 7f8731800640 1 -- 172.21.15.86:0/408378799 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f870c06c8f0 msgr2=0x7f870c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:29.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.817+0000 7f8731800640 1 --2- 172.21.15.86:0/408378799 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f870c06c8f0 0x7f870c06edb0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f8718006fd0 tx=0x7f8718008040 comp rx=0 tx=0).stop 2024-04-20T21:27:29.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.817+0000 7f8731800640 1 -- 172.21.15.86:0/408378799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f872c071d40 msgr2=0x7f872c1f8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:29.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.817+0000 7f8731800640 1 --2- 172.21.15.86:0/408378799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f872c071d40 0x7f872c1f8150 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f871c02b950 tx=0x7f871c0046c0 comp rx=0 tx=0).stop 2024-04-20T21:27:29.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.817+0000 7f8731800640 1 -- 172.21.15.86:0/408378799 shutdown_connections 2024-04-20T21:27:29.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.817+0000 7f8731800640 1 --2- 172.21.15.86:0/408378799 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f870c06c8f0 0x7f870c06edb0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:29.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.817+0000 7f8731800640 1 --2- 172.21.15.86:0/408378799 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f872c161c60 0x7f872c0b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:29.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.817+0000 7f8731800640 1 --2- 172.21.15.86:0/408378799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f872c15ff10 0x7f872c1371b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:29.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.817+0000 7f8731800640 1 --2- 172.21.15.86:0/408378799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f872c071d40 0x7f872c1f8150 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:29.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.817+0000 7f8731800640 1 -- 172.21.15.86:0/408378799 >> 172.21.15.86:0/408378799 conn(0x7f872c067cf0 msgr2=0x7f872c0b1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:29.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.818+0000 7f8731800640 1 -- 172.21.15.86:0/408378799 shutdown_connections 2024-04-20T21:27:29.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:29.818+0000 7f8731800640 1 -- 172.21.15.86:0/408378799 wait complete. 2024-04-20T21:27:29.834 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:27:30.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.026+0000 7f96cba00640 1 Processor -- start 2024-04-20T21:27:30.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.027+0000 7f96cba00640 1 -- start start 2024-04-20T21:27:30.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.028+0000 7f96cba00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f96c406c090 0x7f96c40cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:30.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.028+0000 7f96cba00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f96c40b4ba0 0x7f96c40b5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:30.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.028+0000 7f96cba00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96c4156110 con 0x7f96c4155d30 2024-04-20T21:27:30.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.028+0000 7f96cba00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f96c40ce460 con 0x7f96c406c090 2024-04-20T21:27:30.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.028+0000 7f96cba00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f96c40ce740 con 0x7f96c40b4ba0 2024-04-20T21:27:30.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.028+0000 7f96c3e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f96c406c090 0x7f96c40cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:30.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.028+0000 7f96c9000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f96c4155d30 0x7f96c406b980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:35824/0 (socket says 172.21.15.86:35824) 2024-04-20T21:27:30.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.028+0000 7f96c9000640 1 -- 172.21.15.86:0/1400396207 learned_addr learned my addr 172.21.15.86:0/1400396207 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:27:30.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.029+0000 7f96c3e00640 1 -- 172.21.15.86:0/1400396207 >> v2:172.21.15.144:3300/0 conn(0x7f96c40b4ba0 msgr2=0x7f96c40b5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:30.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.029+0000 7f96c9a00640 1 --2- 172.21.15.86:0/1400396207 >> v2:172.21.15.144:3300/0 conn(0x7f96c40b4ba0 0x7f96c40b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:30.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.029+0000 7f96c3e00640 1 --2- 172.21.15.86:0/1400396207 >> v2:172.21.15.144:3300/0 conn(0x7f96c40b4ba0 0x7f96c40b5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:30.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.029+0000 7f96c3e00640 1 -- 172.21.15.86:0/1400396207 >> v1:172.21.15.86:6789/0 conn(0x7f96c4155d30 legacy=0x7f96c406b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:30.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.029+0000 7f96c3e00640 1 -- 172.21.15.86:0/1400396207 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96c40cea20 con 0x7f96c406c090 2024-04-20T21:27:30.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.029+0000 7f96c9a00640 1 --2- 172.21.15.86:0/1400396207 >> v2:172.21.15.144:3300/0 conn(0x7f96c40b4ba0 0x7f96c40b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:27:30.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.029+0000 7f96c3400640 1 -- 172.21.15.86:0/1400396207 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2287308682 0 0) 0x7f96c4156110 con 0x7f96c4155d30 2024-04-20T21:27:30.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.030+0000 7f96c3e00640 1 --2- 172.21.15.86:0/1400396207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96c406c090 0x7f96c40cf500 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f96c40b4960 tx=0x7f96ac02bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a233dbb3ec6d0ff4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:30.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.030+0000 7f96c3400640 1 -- 172.21.15.86:0/1400396207 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ac039070 con 0x7f96c406c090 2024-04-20T21:27:30.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.030+0000 7f96c3400640 1 -- 172.21.15.86:0/1400396207 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96ac02cb60 con 0x7f96c406c090 2024-04-20T21:27:30.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.030+0000 7f96c3400640 1 -- 172.21.15.86:0/1400396207 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ac034440 con 0x7f96c406c090 2024-04-20T21:27:30.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.031+0000 7f96cba00640 1 -- 172.21.15.86:0/1400396207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96c406c090 msgr2=0x7f96c40cf500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:30.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.031+0000 7f96cba00640 1 --2- 172.21.15.86:0/1400396207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96c406c090 0x7f96c40cf500 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f96c40b4960 tx=0x7f96ac02bdb0 comp rx=0 tx=0).stop 2024-04-20T21:27:30.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.031+0000 7f96cba00640 1 -- 172.21.15.86:0/1400396207 shutdown_connections 2024-04-20T21:27:30.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.031+0000 7f96cba00640 1 --2- 172.21.15.86:0/1400396207 >> v2:172.21.15.144:3300/0 conn(0x7f96c40b4ba0 0x7f96c40b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:30.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.031+0000 7f96cba00640 1 --2- 172.21.15.86:0/1400396207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96c406c090 0x7f96c40cf500 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:30.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.031+0000 7f96cba00640 1 -- 172.21.15.86:0/1400396207 >> 172.21.15.86:0/1400396207 conn(0x7f96c40b2110 msgr2=0x7f96c40b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:30.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.031+0000 7f96cba00640 1 -- 172.21.15.86:0/1400396207 shutdown_connections 2024-04-20T21:27:30.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.031+0000 7f96cba00640 1 -- 172.21.15.86:0/1400396207 wait complete. 2024-04-20T21:27:30.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.033+0000 7f96cba00640 1 Processor -- start 2024-04-20T21:27:30.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.033+0000 7f96cba00640 1 -- start start 2024-04-20T21:27:30.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.034+0000 7f96cba00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96c406c090 0x7f96c415cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:30.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.034+0000 7f96cba00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96c40b4ba0 0x7f96c4145e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:30.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.034+0000 7f96cba00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96c4155d30 0x7f96c41f4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:30.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.034+0000 7f96cba00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f96c40b5570 con 0x7f96c40b4ba0 2024-04-20T21:27:30.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.034+0000 7f96cba00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f96c40cefe0 con 0x7f96c4155d30 2024-04-20T21:27:30.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.034+0000 7f96cba00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f96c40b56f0 con 0x7f96c406c090 2024-04-20T21:27:30.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.034+0000 7f96c3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96c40b4ba0 0x7f96c4145e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:30.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.034+0000 7f96c3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96c40b4ba0 0x7f96c4145e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60264/0 (socket says 172.21.15.86:60264) 2024-04-20T21:27:30.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.034+0000 7f96c3e00640 1 -- 172.21.15.86:0/1467022367 learned_addr learned my addr 172.21.15.86:0/1467022367 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:30.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.034+0000 7f96c3e00640 1 -- 172.21.15.86:0/1467022367 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96c406c090 msgr2=0x7f96c415cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:30.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.034+0000 7f96c3e00640 1 --2- 172.21.15.86:0/1467022367 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96c406c090 0x7f96c415cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:30.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.034+0000 7f96c3e00640 1 -- 172.21.15.86:0/1467022367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96c4155d30 msgr2=0x7f96c41f4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:30.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.034+0000 7f96c3e00640 1 --2- 172.21.15.86:0/1467022367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96c4155d30 0x7f96c41f4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:30.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.034+0000 7f96c3e00640 1 -- 172.21.15.86:0/1467022367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96c41f4d70 con 0x7f96c40b4ba0 2024-04-20T21:27:30.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.035+0000 7f96c3e00640 1 --2- 172.21.15.86:0/1467022367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96c40b4ba0 0x7f96c4145e30 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f96ac037040 tx=0x7f96ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:30.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.035+0000 7f96c0c00640 1 -- 172.21.15.86:0/1467022367 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ac039050 con 0x7f96c40b4ba0 2024-04-20T21:27:30.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.035+0000 7f96c0c00640 1 -- 172.21.15.86:0/1467022367 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96ac03a040 con 0x7f96c40b4ba0 2024-04-20T21:27:30.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.035+0000 7f96c0c00640 1 -- 172.21.15.86:0/1467022367 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ac03e610 con 0x7f96c40b4ba0 2024-04-20T21:27:30.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.035+0000 7f96cba00640 1 -- 172.21.15.86:0/1467022367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96c41f5050 con 0x7f96c40b4ba0 2024-04-20T21:27:30.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.035+0000 7f96cba00640 1 -- 172.21.15.86:0/1467022367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96c414dd20 con 0x7f96c40b4ba0 2024-04-20T21:27:30.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.036+0000 7f96cba00640 1 -- 172.21.15.86:0/1467022367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96c414d830 con 0x7f96c40b4ba0 2024-04-20T21:27:30.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.037+0000 7f96c0c00640 1 -- 172.21.15.86:0/1467022367 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f96ac047020 con 0x7f96c40b4ba0 2024-04-20T21:27:30.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.038+0000 7f96c0c00640 1 --2- 172.21.15.86:0/1467022367 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f96a006c8f0 0x7f96a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:30.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.038+0000 7f96c0c00640 1 -- 172.21.15.86:0/1467022367 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7f96ac0ae1f0 con 0x7f96c40b4ba0 2024-04-20T21:27:30.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.038+0000 7f96c9000640 1 --2- 172.21.15.86:0/1467022367 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f96a006c8f0 0x7f96a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:30.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.039+0000 7f96c9000640 1 --2- 172.21.15.86:0/1467022367 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f96a006c8f0 0x7f96a006edb0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f96b4007fd0 tx=0x7f96b4007920 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:30.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.042+0000 7f96c0c00640 1 -- 172.21.15.86:0/1467022367 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f96ac04e020 con 0x7f96c40b4ba0 2024-04-20T21:27:30.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.238+0000 7f96cba00640 1 -- 172.21.15.86:0/1467022367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f96c40689e0 con 0x7f96c40b4ba0 2024-04-20T21:27:30.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.239+0000 7f96c0c00640 1 -- 172.21.15.86:0/1467022367 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f96ac07c4b0 con 0x7f96c40b4ba0 2024-04-20T21:27:30.238 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-20T21:27:30.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.242+0000 7f96cba00640 1 -- 172.21.15.86:0/1467022367 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f96a006c8f0 msgr2=0x7f96a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:30.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.242+0000 7f96cba00640 1 --2- 172.21.15.86:0/1467022367 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f96a006c8f0 0x7f96a006edb0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f96b4007fd0 tx=0x7f96b4007920 comp rx=0 tx=0).stop 2024-04-20T21:27:30.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.242+0000 7f96cba00640 1 -- 172.21.15.86:0/1467022367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96c40b4ba0 msgr2=0x7f96c4145e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:30.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.242+0000 7f96cba00640 1 --2- 172.21.15.86:0/1467022367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96c40b4ba0 0x7f96c4145e30 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f96ac037040 tx=0x7f96ac004a40 comp rx=0 tx=0).stop 2024-04-20T21:27:30.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.242+0000 7f96cba00640 1 -- 172.21.15.86:0/1467022367 shutdown_connections 2024-04-20T21:27:30.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.242+0000 7f96cba00640 1 --2- 172.21.15.86:0/1467022367 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f96a006c8f0 0x7f96a006edb0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:30.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.242+0000 7f96cba00640 1 --2- 172.21.15.86:0/1467022367 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96c4155d30 0x7f96c41f4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:30.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.242+0000 7f96cba00640 1 --2- 172.21.15.86:0/1467022367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96c40b4ba0 0x7f96c4145e30 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:30.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.242+0000 7f96cba00640 1 --2- 172.21.15.86:0/1467022367 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96c406c090 0x7f96c415cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:30.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.242+0000 7f96cba00640 1 -- 172.21.15.86:0/1467022367 >> 172.21.15.86:0/1467022367 conn(0x7f96c40b2110 msgr2=0x7f96c406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:30.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.242+0000 7f96cba00640 1 -- 172.21.15.86:0/1467022367 shutdown_connections 2024-04-20T21:27:30.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.242+0000 7f96cba00640 1 -- 172.21.15.86:0/1467022367 wait complete. 2024-04-20T21:27:30.259 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:27:30.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.453+0000 7fe9ad800640 1 Processor -- start 2024-04-20T21:27:30.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.454+0000 7fe9ad800640 1 -- start start 2024-04-20T21:27:30.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.455+0000 7fe9ad800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe9a815eaf0 0x7fe9a80ba1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:30.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.455+0000 7fe9ad800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe9a80b4030 0x7fe9a80d17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:30.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.455+0000 7fe9ad800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9a80d1d00 0x7fe9a80b6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:30.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.455+0000 7fe9ad800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe9a8163110 con 0x7fe9a80b4030 2024-04-20T21:27:30.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.455+0000 7fe9ad800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fe9a80b7a00 con 0x7fe9a815eaf0 2024-04-20T21:27:30.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.455+0000 7fe9ad800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe9a80d2150 con 0x7fe9a80d1d00 2024-04-20T21:27:30.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.456+0000 7fe9a6000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe9a80b4030 0x7fe9a80d17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:30.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.456+0000 7fe9a6000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe9a80b4030 0x7fe9a80d17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60270/0 (socket says 172.21.15.86:60270) 2024-04-20T21:27:30.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.456+0000 7fe9a6000640 1 -- 172.21.15.86:0/4077148531 learned_addr learned my addr 172.21.15.86:0/4077148531 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:30.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.456+0000 7fe9a7400640 1 --2- 172.21.15.86:0/4077148531 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9a80d1d00 0x7fe9a80b6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:30.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.456+0000 7fe9a6a00640 1 --2- 172.21.15.86:0/4077148531 >> v2:172.21.15.144:3300/0 conn(0x7fe9a815eaf0 0x7fe9a80ba1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:30.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.456+0000 7fe9a6000640 1 -- 172.21.15.86:0/4077148531 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9a80d1d00 msgr2=0x7fe9a80b6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:30.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.456+0000 7fe9a6000640 1 --2- 172.21.15.86:0/4077148531 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9a80d1d00 0x7fe9a80b6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:30.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.456+0000 7fe9a6000640 1 -- 172.21.15.86:0/4077148531 >> v2:172.21.15.144:3300/0 conn(0x7fe9a815eaf0 msgr2=0x7fe9a80ba1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:30.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.456+0000 7fe9a6000640 1 --2- 172.21.15.86:0/4077148531 >> v2:172.21.15.144:3300/0 conn(0x7fe9a815eaf0 0x7fe9a80ba1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:30.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.456+0000 7fe9a6000640 1 -- 172.21.15.86:0/4077148531 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9a815eed0 con 0x7fe9a80b4030 2024-04-20T21:27:30.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.457+0000 7fe9a6000640 1 --2- 172.21.15.86:0/4077148531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9a80b4030 0x7fe9a80d17c0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fe9a806d8e0 tx=0x7fe99002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9761d8fed47f6a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:30.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.457+0000 7fe9a5600640 1 -- 172.21.15.86:0/4077148531 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe990034070 con 0x7fe9a80b4030 2024-04-20T21:27:30.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.457+0000 7fe9a5600640 1 -- 172.21.15.86:0/4077148531 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe990030860 con 0x7fe9a80b4030 2024-04-20T21:27:30.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.457+0000 7fe9a5600640 1 -- 172.21.15.86:0/4077148531 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe990030c30 con 0x7fe9a80b4030 2024-04-20T21:27:30.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.458+0000 7fe9ad800640 1 -- 172.21.15.86:0/4077148531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9a80b4030 msgr2=0x7fe9a80d17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:30.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.458+0000 7fe9ad800640 1 --2- 172.21.15.86:0/4077148531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9a80b4030 0x7fe9a80d17c0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fe9a806d8e0 tx=0x7fe99002b9a0 comp rx=0 tx=0).stop 2024-04-20T21:27:30.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.458+0000 7fe9ad800640 1 -- 172.21.15.86:0/4077148531 shutdown_connections 2024-04-20T21:27:30.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.458+0000 7fe9ad800640 1 --2- 172.21.15.86:0/4077148531 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9a80d1d00 0x7fe9a80b6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:30.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.458+0000 7fe9ad800640 1 --2- 172.21.15.86:0/4077148531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9a80b4030 0x7fe9a80d17c0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:30.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.458+0000 7fe9ad800640 1 --2- 172.21.15.86:0/4077148531 >> v2:172.21.15.144:3300/0 conn(0x7fe9a815eaf0 0x7fe9a80ba1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:30.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.458+0000 7fe9ad800640 1 -- 172.21.15.86:0/4077148531 >> 172.21.15.86:0/4077148531 conn(0x7fe9a80678a0 msgr2=0x7fe9a8067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:30.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.458+0000 7fe9ad800640 1 -- 172.21.15.86:0/4077148531 shutdown_connections 2024-04-20T21:27:30.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.458+0000 7fe9ad800640 1 -- 172.21.15.86:0/4077148531 wait complete. 2024-04-20T21:27:30.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.460+0000 7fe9ad800640 1 Processor -- start 2024-04-20T21:27:30.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.460+0000 7fe9ad800640 1 -- start start 2024-04-20T21:27:30.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.461+0000 7fe9ad800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9a80b4030 0x7fe9a80b6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:30.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.461+0000 7fe9ad800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9a80d1d00 0x7fe9a81e8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:30.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.461+0000 7fe9ad800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9a815eaf0 0x7fe9a8150ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:30.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.461+0000 7fe9ad800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a8162da0 con 0x7fe9a80d1d00 2024-04-20T21:27:30.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.461+0000 7fe9ad800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a8162ac0 con 0x7fe9a80b4030 2024-04-20T21:27:30.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.461+0000 7fe9ad800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe9a80ba730 con 0x7fe9a815eaf0 2024-04-20T21:27:30.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.461+0000 7fe9a6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9a80d1d00 0x7fe9a81e8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:30.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.461+0000 7fe9a6000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9a80d1d00 0x7fe9a81e8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60278/0 (socket says 172.21.15.86:60278) 2024-04-20T21:27:30.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.461+0000 7fe9a6000640 1 -- 172.21.15.86:0/196715675 learned_addr learned my addr 172.21.15.86:0/196715675 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:30.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.462+0000 7fe9a7400640 1 --2- 172.21.15.86:0/196715675 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9a815eaf0 0x7fe9a8150ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:30.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.462+0000 7fe9a6a00640 1 --2- 172.21.15.86:0/196715675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9a80b4030 0x7fe9a80b6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:30.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.462+0000 7fe9a6000640 1 -- 172.21.15.86:0/196715675 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9a815eaf0 msgr2=0x7fe9a8150ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:30.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.462+0000 7fe9a6000640 1 --2- 172.21.15.86:0/196715675 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9a815eaf0 0x7fe9a8150ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:30.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.462+0000 7fe9a6000640 1 -- 172.21.15.86:0/196715675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9a80b4030 msgr2=0x7fe9a80b6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:30.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.462+0000 7fe9a6000640 1 --2- 172.21.15.86:0/196715675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9a80b4030 0x7fe9a80b6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:30.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.462+0000 7fe9a6000640 1 -- 172.21.15.86:0/196715675 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9a80b44d0 con 0x7fe9a80d1d00 2024-04-20T21:27:30.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.462+0000 7fe9a7400640 1 --2- 172.21.15.86:0/196715675 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9a815eaf0 0x7fe9a8150ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:27:30.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.462+0000 7fe9a6a00640 1 --2- 172.21.15.86:0/196715675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9a80b4030 0x7fe9a80b6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:27:30.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.462+0000 7fe9a6000640 1 --2- 172.21.15.86:0/196715675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9a80d1d00 0x7fe9a81e8220 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fe990000c00 tx=0x7fe990004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:30.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.462+0000 7fe99ea00640 1 -- 172.21.15.86:0/196715675 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe990048070 con 0x7fe9a80d1d00 2024-04-20T21:27:30.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.462+0000 7fe99ea00640 1 -- 172.21.15.86:0/196715675 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe990030660 con 0x7fe9a80d1d00 2024-04-20T21:27:30.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.462+0000 7fe99ea00640 1 -- 172.21.15.86:0/196715675 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe990034040 con 0x7fe9a80d1d00 2024-04-20T21:27:30.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.462+0000 7fe9ad800640 1 -- 172.21.15.86:0/196715675 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9a81e8760 con 0x7fe9a80d1d00 2024-04-20T21:27:30.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.463+0000 7fe9ad800640 1 -- 172.21.15.86:0/196715675 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9a80b4a50 con 0x7fe9a80d1d00 2024-04-20T21:27:30.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.464+0000 7fe9ad800640 1 -- 172.21.15.86:0/196715675 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9a80689e0 con 0x7fe9a80d1d00 2024-04-20T21:27:30.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.465+0000 7fe99ea00640 1 -- 172.21.15.86:0/196715675 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe990037070 con 0x7fe9a80d1d00 2024-04-20T21:27:30.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.466+0000 7fe99ea00640 1 --2- 172.21.15.86:0/196715675 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe98406c8f0 0x7fe98406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:30.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.466+0000 7fe99ea00640 1 -- 172.21.15.86:0/196715675 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7fe9900add60 con 0x7fe9a80d1d00 2024-04-20T21:27:30.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.466+0000 7fe9a6a00640 1 --2- 172.21.15.86:0/196715675 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe98406c8f0 0x7fe98406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:30.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.467+0000 7fe9a6a00640 1 --2- 172.21.15.86:0/196715675 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe98406c8f0 0x7fe98406edb0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fe998006fd0 tx=0x7fe998008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:30.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.470+0000 7fe99ea00640 1 -- 172.21.15.86:0/196715675 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe99007c140 con 0x7fe9a80d1d00 2024-04-20T21:27:30.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.687+0000 7fe9ad800640 1 -- 172.21.15.86:0/196715675 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe9a8075060 con 0x7fe9a80d1d00 2024-04-20T21:27:30.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.688+0000 7fe99ea00640 1 -- 172.21.15.86:0/196715675 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5209 (secure 0 0 0) 0x7fe990030830 con 0x7fe9a80d1d00 2024-04-20T21:27:30.688 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:27:30.688 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":73,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5887,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2433619638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2433619638},{"type":"v1","addr":"172.21.15.144:6835","nonce":2433619638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68},{"gid":7506,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/4013863257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4013863257},{"type":"v1","addr":"172.21.15.86:6836","nonce":4013863257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":7509,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2044883073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2044883073},{"type":"v1","addr":"172.21.15.86:6837","nonce":2044883073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:24:58.299257+0000","modified":"2024-04-20T21:26:48.911892+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7518},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7518":{"gid":7518,"name":"d","rank":0,"incarnation":70,"state":"up:active","state_seq":32,"addr":"172.21.15.144:6834/2351381466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2351381466},{"type":"v1","addr":"172.21.15.144:6834","nonce":2351381466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7518,"qdb_cluster":[7518]},"id":6}]} 2024-04-20T21:27:30.688 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 73 2024-04-20T21:27:30.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.691+0000 7fe9ad800640 1 -- 172.21.15.86:0/196715675 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe98406c8f0 msgr2=0x7fe98406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:30.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.691+0000 7fe9ad800640 1 --2- 172.21.15.86:0/196715675 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe98406c8f0 0x7fe98406edb0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fe998006fd0 tx=0x7fe998008040 comp rx=0 tx=0).stop 2024-04-20T21:27:30.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.691+0000 7fe9ad800640 1 -- 172.21.15.86:0/196715675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9a80d1d00 msgr2=0x7fe9a81e8220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:30.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.691+0000 7fe9ad800640 1 --2- 172.21.15.86:0/196715675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9a80d1d00 0x7fe9a81e8220 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fe990000c00 tx=0x7fe990004730 comp rx=0 tx=0).stop 2024-04-20T21:27:30.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.691+0000 7fe9ad800640 1 -- 172.21.15.86:0/196715675 shutdown_connections 2024-04-20T21:27:30.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.691+0000 7fe9ad800640 1 --2- 172.21.15.86:0/196715675 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe98406c8f0 0x7fe98406edb0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:30.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.691+0000 7fe9ad800640 1 --2- 172.21.15.86:0/196715675 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9a815eaf0 0x7fe9a8150ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:30.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.691+0000 7fe9ad800640 1 --2- 172.21.15.86:0/196715675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9a80d1d00 0x7fe9a81e8220 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:30.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.691+0000 7fe9ad800640 1 --2- 172.21.15.86:0/196715675 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9a80b4030 0x7fe9a80b6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:30.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.691+0000 7fe9ad800640 1 -- 172.21.15.86:0/196715675 >> 172.21.15.86:0/196715675 conn(0x7fe9a80678a0 msgr2=0x7fe9a80d3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:30.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.691+0000 7fe9ad800640 1 -- 172.21.15.86:0/196715675 shutdown_connections 2024-04-20T21:27:30.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.691+0000 7fe9ad800640 1 -- 172.21.15.86:0/196715675 wait complete. 2024-04-20T21:27:30.706 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:27:30.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.898+0000 7fd410800640 1 Processor -- start 2024-04-20T21:27:30.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.899+0000 7fd410800640 1 -- start start 2024-04-20T21:27:30.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.900+0000 7fd410800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd408146130 0x7fd40806e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:30.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.900+0000 7fd410800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd4080ccdd0 0x7fd4080cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:30.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.900+0000 7fd410800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd40806dc30 con 0x7fd408071d00 2024-04-20T21:27:30.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.900+0000 7fd410800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd4080cd8e0 con 0x7fd4080ccdd0 2024-04-20T21:27:30.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.900+0000 7fd410800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fd4080cdbc0 con 0x7fd408146130 2024-04-20T21:27:30.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.901+0000 7fd40e800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd408071d00 0x7fd40815cc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:35834/0 (socket says 172.21.15.86:35834) 2024-04-20T21:27:30.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.901+0000 7fd40e800640 1 -- 172.21.15.86:0/1543202234 learned_addr learned my addr 172.21.15.86:0/1543202234 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:27:30.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.901+0000 7fd40d400640 1 --2- 172.21.15.86:0/1543202234 >> v2:172.21.15.86:3300/0 conn(0x7fd4080ccdd0 0x7fd4080cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:30.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.901+0000 7fd40de00640 1 --2- 172.21.15.86:0/1543202234 >> v2:172.21.15.144:3300/0 conn(0x7fd408146130 0x7fd40806e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:30.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.901+0000 7fd40ca00640 1 -- 172.21.15.86:0/1543202234 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4132778038 0 0) 0x7fd40806dc30 con 0x7fd408071d00 2024-04-20T21:27:30.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.901+0000 7fd40ca00640 1 -- 172.21.15.86:0/1543202234 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd3f0003520 con 0x7fd408071d00 2024-04-20T21:27:30.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.901+0000 7fd40d400640 1 -- 172.21.15.86:0/1543202234 >> v2:172.21.15.144:3300/0 conn(0x7fd408146130 msgr2=0x7fd40806e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:30.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.901+0000 7fd40d400640 1 --2- 172.21.15.86:0/1543202234 >> v2:172.21.15.144:3300/0 conn(0x7fd408146130 0x7fd40806e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:30.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.901+0000 7fd40d400640 1 -- 172.21.15.86:0/1543202234 >> v1:172.21.15.86:6789/0 conn(0x7fd408071d00 legacy=0x7fd40815cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:30.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.901+0000 7fd40d400640 1 -- 172.21.15.86:0/1543202234 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd40806ded0 con 0x7fd4080ccdd0 2024-04-20T21:27:30.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.902+0000 7fd40d400640 1 --2- 172.21.15.86:0/1543202234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4080ccdd0 0x7fd4080cd230 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fd40806de00 tx=0x7fd3f802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a14750599cbdb8fe server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:30.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.902+0000 7fd40ca00640 1 -- 172.21.15.86:0/1543202234 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f8034070 con 0x7fd4080ccdd0 2024-04-20T21:27:30.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.902+0000 7fd40ca00640 1 -- 172.21.15.86:0/1543202234 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3f8030610 con 0x7fd4080ccdd0 2024-04-20T21:27:30.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.903+0000 7fd40ca00640 1 -- 172.21.15.86:0/1543202234 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f80309e0 con 0x7fd4080ccdd0 2024-04-20T21:27:30.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.903+0000 7fd410800640 1 -- 172.21.15.86:0/1543202234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4080ccdd0 msgr2=0x7fd4080cd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:30.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.903+0000 7fd410800640 1 --2- 172.21.15.86:0/1543202234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4080ccdd0 0x7fd4080cd230 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fd40806de00 tx=0x7fd3f802b960 comp rx=0 tx=0).stop 2024-04-20T21:27:30.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.903+0000 7fd410800640 1 -- 172.21.15.86:0/1543202234 shutdown_connections 2024-04-20T21:27:30.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.903+0000 7fd410800640 1 --2- 172.21.15.86:0/1543202234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4080ccdd0 0x7fd4080cd230 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:30.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.903+0000 7fd410800640 1 --2- 172.21.15.86:0/1543202234 >> v2:172.21.15.144:3300/0 conn(0x7fd408146130 0x7fd40806e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:30.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.903+0000 7fd410800640 1 -- 172.21.15.86:0/1543202234 >> 172.21.15.86:0/1543202234 conn(0x7fd4080b2360 msgr2=0x7fd4080b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:30.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.903+0000 7fd410800640 1 -- 172.21.15.86:0/1543202234 shutdown_connections 2024-04-20T21:27:30.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.903+0000 7fd410800640 1 -- 172.21.15.86:0/1543202234 wait complete. 2024-04-20T21:27:30.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.905+0000 7fd410800640 1 Processor -- start 2024-04-20T21:27:30.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.905+0000 7fd410800640 1 -- start start 2024-04-20T21:27:30.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.906+0000 7fd410800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd408071d00 0x7fd40815c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:30.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.906+0000 7fd410800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4080ccdd0 0x7fd4080ac3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:30.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.906+0000 7fd410800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd408146130 0x7fd4080aa0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:30.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.906+0000 7fd410800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd408072150 con 0x7fd408071d00 2024-04-20T21:27:30.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.906+0000 7fd410800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd40806f150 con 0x7fd4080ccdd0 2024-04-20T21:27:30.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.906+0000 7fd410800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd4080722d0 con 0x7fd408146130 2024-04-20T21:27:30.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.906+0000 7fd40de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd408071d00 0x7fd40815c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:30.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.906+0000 7fd40de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd408071d00 0x7fd40815c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60296/0 (socket says 172.21.15.86:60296) 2024-04-20T21:27:30.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.906+0000 7fd40de00640 1 -- 172.21.15.86:0/1931844567 learned_addr learned my addr 172.21.15.86:0/1931844567 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:30.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.906+0000 7fd40de00640 1 -- 172.21.15.86:0/1931844567 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd408146130 msgr2=0x7fd4080aa0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:30.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.906+0000 7fd40de00640 1 --2- 172.21.15.86:0/1931844567 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd408146130 0x7fd4080aa0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:30.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.906+0000 7fd40de00640 1 -- 172.21.15.86:0/1931844567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4080ccdd0 msgr2=0x7fd4080ac3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:30.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.906+0000 7fd40de00640 1 --2- 172.21.15.86:0/1931844567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4080ccdd0 0x7fd4080ac3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:30.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.906+0000 7fd40de00640 1 -- 172.21.15.86:0/1931844567 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4080aa820 con 0x7fd408071d00 2024-04-20T21:27:30.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.907+0000 7fd40de00640 1 --2- 172.21.15.86:0/1931844567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd408071d00 0x7fd40815c0e0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc00e870 tx=0x7fd3fc00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:30.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.907+0000 7fd402000640 1 -- 172.21.15.86:0/1931844567 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc017070 con 0x7fd408071d00 2024-04-20T21:27:30.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.907+0000 7fd402000640 1 -- 172.21.15.86:0/1931844567 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3fc00c940 con 0x7fd408071d00 2024-04-20T21:27:30.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.907+0000 7fd402000640 1 -- 172.21.15.86:0/1931844567 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc00cd30 con 0x7fd408071d00 2024-04-20T21:27:30.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.907+0000 7fd410800640 1 -- 172.21.15.86:0/1931844567 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4080aab00 con 0x7fd408071d00 2024-04-20T21:27:30.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.907+0000 7fd410800640 1 -- 172.21.15.86:0/1931844567 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4081317d0 con 0x7fd408071d00 2024-04-20T21:27:30.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.908+0000 7fd410800640 1 -- 172.21.15.86:0/1931844567 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4080689e0 con 0x7fd408071d00 2024-04-20T21:27:30.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.909+0000 7fd402000640 1 -- 172.21.15.86:0/1931844567 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd3fc01a070 con 0x7fd408071d00 2024-04-20T21:27:30.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.910+0000 7fd402000640 1 --2- 172.21.15.86:0/1931844567 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd3ec06c8f0 0x7fd3ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:30.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.910+0000 7fd402000640 1 -- 172.21.15.86:0/1931844567 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7fd3fc08bec0 con 0x7fd408071d00 2024-04-20T21:27:30.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.910+0000 7fd40d400640 1 --2- 172.21.15.86:0/1931844567 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd3ec06c8f0 0x7fd3ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:30.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.911+0000 7fd40d400640 1 --2- 172.21.15.86:0/1931844567 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd3ec06c8f0 0x7fd3ec06edb0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8036000 tx=0x7fd3f8030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:30.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:30.914+0000 7fd402000640 1 -- 172.21.15.86:0/1931844567 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd3fc05a180 con 0x7fd408071d00 2024-04-20T21:27:31.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.113+0000 7fd410800640 1 -- 172.21.15.86:0/1931844567 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd4080ac9d0 con 0x7fd408071d00 2024-04-20T21:27:31.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.115+0000 7fd402000640 1 -- 172.21.15.86:0/1931844567 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+16973 (secure 0 0 0) 0x7fd3fc059a80 con 0x7fd408071d00 2024-04-20T21:27:31.115 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:27:31.115 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":89,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:26:43.869364+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:24:55.964540+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-20T21:24:57.241527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.3e","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.144:6835/3981403733":"2024-04-21T21:26:43.869334+0000","172.21.15.144:6833/3981403733":"2024-04-21T21:26:43.869334+0000","172.21.15.144:6835/1740492227":"2024-04-21T21:25:20.040733+0000","172.21.15.144:6833/1740492227":"2024-04-21T21:25:20.040733+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:27:31.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.118+0000 7fd410800640 1 -- 172.21.15.86:0/1931844567 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd3ec06c8f0 msgr2=0x7fd3ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:31.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.118+0000 7fd410800640 1 --2- 172.21.15.86:0/1931844567 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd3ec06c8f0 0x7fd3ec06edb0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8036000 tx=0x7fd3f8030000 comp rx=0 tx=0).stop 2024-04-20T21:27:31.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.118+0000 7fd410800640 1 -- 172.21.15.86:0/1931844567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd408071d00 msgr2=0x7fd40815c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:31.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.118+0000 7fd410800640 1 --2- 172.21.15.86:0/1931844567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd408071d00 0x7fd40815c0e0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc00e870 tx=0x7fd3fc00ed30 comp rx=0 tx=0).stop 2024-04-20T21:27:31.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.118+0000 7fd410800640 1 -- 172.21.15.86:0/1931844567 shutdown_connections 2024-04-20T21:27:31.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.118+0000 7fd410800640 1 --2- 172.21.15.86:0/1931844567 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd3ec06c8f0 0x7fd3ec06edb0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:31.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.118+0000 7fd410800640 1 --2- 172.21.15.86:0/1931844567 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd408146130 0x7fd4080aa0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:31.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.118+0000 7fd410800640 1 --2- 172.21.15.86:0/1931844567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4080ccdd0 0x7fd4080ac3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:31.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.118+0000 7fd410800640 1 --2- 172.21.15.86:0/1931844567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd408071d00 0x7fd40815c0e0 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:31.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.118+0000 7fd410800640 1 -- 172.21.15.86:0/1931844567 >> 172.21.15.86:0/1931844567 conn(0x7fd4080b2360 msgr2=0x7fd40815ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:31.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.118+0000 7fd410800640 1 -- 172.21.15.86:0/1931844567 shutdown_connections 2024-04-20T21:27:31.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.118+0000 7fd410800640 1 -- 172.21.15.86:0/1931844567 wait complete. 2024-04-20T21:27:31.136 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-20T21:27:31.136 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:27:31.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.333+0000 7f9b56400640 1 Processor -- start 2024-04-20T21:27:31.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.334+0000 7f9b56400640 1 -- start start 2024-04-20T21:27:31.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.335+0000 7f9b56400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9b50144e90 0x7f9b500b4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:31.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.335+0000 7f9b56400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9b500cd170 0x7f9b5006ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:31.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.335+0000 7f9b56400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b500cd600 0x7f9b5006f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:31.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.335+0000 7f9b56400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9b50145270 con 0x7f9b50144e90 2024-04-20T21:27:31.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.335+0000 7f9b56400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f9b500cdb50 con 0x7f9b500cd170 2024-04-20T21:27:31.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.335+0000 7f9b56400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9b500b3e50 con 0x7f9b500cd600 2024-04-20T21:27:31.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.336+0000 7f9b4f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9b50144e90 0x7f9b500b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:31.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.336+0000 7f9b4f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9b50144e90 0x7f9b500b4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60300/0 (socket says 172.21.15.86:60300) 2024-04-20T21:27:31.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.336+0000 7f9b4f400640 1 -- 172.21.15.86:0/589828715 learned_addr learned my addr 172.21.15.86:0/589828715 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:31.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.336+0000 7f9b4f400640 1 -- 172.21.15.86:0/589828715 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b500cd600 msgr2=0x7f9b5006f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:31.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.336+0000 7f9b46a00640 1 --2- 172.21.15.86:0/589828715 >> v2:172.21.15.144:3300/0 conn(0x7f9b500cd170 0x7f9b5006ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:31.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.336+0000 7f9b4fe00640 1 --2- 172.21.15.86:0/589828715 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b500cd600 0x7f9b5006f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:31.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.336+0000 7f9b4f400640 1 --2- 172.21.15.86:0/589828715 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b500cd600 0x7f9b5006f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:31.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.336+0000 7f9b4f400640 1 -- 172.21.15.86:0/589828715 >> v2:172.21.15.144:3300/0 conn(0x7f9b500cd170 msgr2=0x7f9b5006ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:31.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.336+0000 7f9b4f400640 1 --2- 172.21.15.86:0/589828715 >> v2:172.21.15.144:3300/0 conn(0x7f9b500cd170 0x7f9b5006ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:31.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.336+0000 7f9b4f400640 1 -- 172.21.15.86:0/589828715 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b500b4130 con 0x7f9b50144e90 2024-04-20T21:27:31.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.337+0000 7f9b4f400640 1 --2- 172.21.15.86:0/589828715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b50144e90 0x7f9b500b4930 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f9b5006fa20 tx=0x7f9b4002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94cc5225911b5383 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:31.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.337+0000 7f9b4ea00640 1 -- 172.21.15.86:0/589828715 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b40035020 con 0x7f9b50144e90 2024-04-20T21:27:31.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.338+0000 7f9b4ea00640 1 -- 172.21.15.86:0/589828715 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b40031870 con 0x7f9b50144e90 2024-04-20T21:27:31.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.338+0000 7f9b4ea00640 1 -- 172.21.15.86:0/589828715 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b40031c40 con 0x7f9b50144e90 2024-04-20T21:27:31.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.338+0000 7f9b56400640 1 -- 172.21.15.86:0/589828715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b50144e90 msgr2=0x7f9b500b4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:31.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.338+0000 7f9b56400640 1 --2- 172.21.15.86:0/589828715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b50144e90 0x7f9b500b4930 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f9b5006fa20 tx=0x7f9b4002b910 comp rx=0 tx=0).stop 2024-04-20T21:27:31.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.338+0000 7f9b56400640 1 -- 172.21.15.86:0/589828715 shutdown_connections 2024-04-20T21:27:31.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.338+0000 7f9b56400640 1 --2- 172.21.15.86:0/589828715 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b500cd600 0x7f9b5006f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:31.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.338+0000 7f9b56400640 1 --2- 172.21.15.86:0/589828715 >> v2:172.21.15.144:3300/0 conn(0x7f9b500cd170 0x7f9b5006ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:31.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.338+0000 7f9b56400640 1 --2- 172.21.15.86:0/589828715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b50144e90 0x7f9b500b4930 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:31.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.338+0000 7f9b56400640 1 -- 172.21.15.86:0/589828715 >> 172.21.15.86:0/589828715 conn(0x7f9b500b2360 msgr2=0x7f9b500b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:31.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.338+0000 7f9b56400640 1 -- 172.21.15.86:0/589828715 shutdown_connections 2024-04-20T21:27:31.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.338+0000 7f9b56400640 1 -- 172.21.15.86:0/589828715 wait complete. 2024-04-20T21:27:31.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.340+0000 7f9b56400640 1 Processor -- start 2024-04-20T21:27:31.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.340+0000 7f9b56400640 1 -- start start 2024-04-20T21:27:31.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.341+0000 7f9b56400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b500cd170 0x7f9b501f7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:31.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.341+0000 7f9b56400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b500cd600 0x7f9b501e23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:31.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.341+0000 7f9b56400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b50144e90 0x7f9b501e2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:31.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.341+0000 7f9b56400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9b500d0010 con 0x7f9b50144e90 2024-04-20T21:27:31.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.341+0000 7f9b56400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9b500b4e70 con 0x7f9b500cd170 2024-04-20T21:27:31.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.341+0000 7f9b56400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9b500d0190 con 0x7f9b500cd600 2024-04-20T21:27:31.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.341+0000 7f9b4fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b50144e90 0x7f9b501e2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:31.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.342+0000 7f9b4fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b50144e90 0x7f9b501e2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60302/0 (socket says 172.21.15.86:60302) 2024-04-20T21:27:31.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.342+0000 7f9b4fe00640 1 -- 172.21.15.86:0/2385737546 learned_addr learned my addr 172.21.15.86:0/2385737546 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:31.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.342+0000 7f9b46a00640 1 --2- 172.21.15.86:0/2385737546 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b500cd600 0x7f9b501e23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:31.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.342+0000 7f9b4f400640 1 --2- 172.21.15.86:0/2385737546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b500cd170 0x7f9b501f7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:31.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.342+0000 7f9b4fe00640 1 -- 172.21.15.86:0/2385737546 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b500cd600 msgr2=0x7f9b501e23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:31.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.342+0000 7f9b4fe00640 1 --2- 172.21.15.86:0/2385737546 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b500cd600 0x7f9b501e23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:31.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.342+0000 7f9b4fe00640 1 -- 172.21.15.86:0/2385737546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b500cd170 msgr2=0x7f9b501f7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:31.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.342+0000 7f9b4fe00640 1 --2- 172.21.15.86:0/2385737546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b500cd170 0x7f9b501f7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:31.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.342+0000 7f9b4fe00640 1 -- 172.21.15.86:0/2385737546 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b501e1750 con 0x7f9b50144e90 2024-04-20T21:27:31.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.342+0000 7f9b46a00640 1 --2- 172.21.15.86:0/2385737546 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b500cd600 0x7f9b501e23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:27:31.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.342+0000 7f9b4fe00640 1 --2- 172.21.15.86:0/2385737546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b50144e90 0x7f9b501e2900 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f9b4800cd70 tx=0x7f9b4800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:31.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.343+0000 7f9b47e00640 1 -- 172.21.15.86:0/2385737546 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b48003b90 con 0x7f9b50144e90 2024-04-20T21:27:31.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.343+0000 7f9b47e00640 1 -- 172.21.15.86:0/2385737546 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b48003d30 con 0x7f9b50144e90 2024-04-20T21:27:31.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.343+0000 7f9b47e00640 1 -- 172.21.15.86:0/2385737546 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b48004240 con 0x7f9b50144e90 2024-04-20T21:27:31.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.343+0000 7f9b56400640 1 -- 172.21.15.86:0/2385737546 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b501e1a30 con 0x7f9b50144e90 2024-04-20T21:27:31.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.343+0000 7f9b56400640 1 -- 172.21.15.86:0/2385737546 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b50140a00 con 0x7f9b50144e90 2024-04-20T21:27:31.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.344+0000 7f9b56400640 1 -- 172.21.15.86:0/2385737546 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b500ce620 con 0x7f9b50144e90 2024-04-20T21:27:31.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.345+0000 7f9b47e00640 1 -- 172.21.15.86:0/2385737546 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9b480043e0 con 0x7f9b50144e90 2024-04-20T21:27:31.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.346+0000 7f9b47e00640 1 --2- 172.21.15.86:0/2385737546 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9b2c06c8f0 0x7f9b2c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:31.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.346+0000 7f9b47e00640 1 -- 172.21.15.86:0/2385737546 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7f9b4808c750 con 0x7f9b50144e90 2024-04-20T21:27:31.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.346+0000 7f9b4f400640 1 --2- 172.21.15.86:0/2385737546 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9b2c06c8f0 0x7f9b2c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:31.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.347+0000 7f9b4f400640 1 --2- 172.21.15.86:0/2385737546 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9b2c06c8f0 0x7f9b2c06edb0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f9b40036000 tx=0x7f9b40008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:31.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.350+0000 7f9b47e00640 1 -- 172.21.15.86:0/2385737546 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9b500ce620 con 0x7f9b50144e90 2024-04-20T21:27:31.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.592+0000 7f9b56400640 1 -- 172.21.15.86:0/2385737546 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9b500689e0 con 0x7f9b50144e90 2024-04-20T21:27:31.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.593+0000 7f9b47e00640 1 -- 172.21.15.86:0/2385737546 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5209 (secure 0 0 0) 0x7f9b4801d340 con 0x7f9b50144e90 2024-04-20T21:27:31.592 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:27:31.593 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":73,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5887,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2433619638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2433619638},{"type":"v1","addr":"172.21.15.144:6835","nonce":2433619638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68},{"gid":7506,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/4013863257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4013863257},{"type":"v1","addr":"172.21.15.86:6836","nonce":4013863257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":7509,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2044883073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2044883073},{"type":"v1","addr":"172.21.15.86:6837","nonce":2044883073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:24:58.299257+0000","modified":"2024-04-20T21:26:48.911892+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7518},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7518":{"gid":7518,"name":"d","rank":0,"incarnation":70,"state":"up:active","state_seq":32,"addr":"172.21.15.144:6834/2351381466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2351381466},{"type":"v1","addr":"172.21.15.144:6834","nonce":2351381466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7518,"qdb_cluster":[7518]},"id":6}]} 2024-04-20T21:27:31.593 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 73 2024-04-20T21:27:31.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.596+0000 7f9b56400640 1 -- 172.21.15.86:0/2385737546 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9b2c06c8f0 msgr2=0x7f9b2c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:31.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.596+0000 7f9b56400640 1 --2- 172.21.15.86:0/2385737546 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9b2c06c8f0 0x7f9b2c06edb0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f9b40036000 tx=0x7f9b40008030 comp rx=0 tx=0).stop 2024-04-20T21:27:31.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.596+0000 7f9b56400640 1 -- 172.21.15.86:0/2385737546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b50144e90 msgr2=0x7f9b501e2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:31.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.596+0000 7f9b56400640 1 --2- 172.21.15.86:0/2385737546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b50144e90 0x7f9b501e2900 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f9b4800cd70 tx=0x7f9b4800a660 comp rx=0 tx=0).stop 2024-04-20T21:27:31.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.596+0000 7f9b56400640 1 -- 172.21.15.86:0/2385737546 shutdown_connections 2024-04-20T21:27:31.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.596+0000 7f9b56400640 1 --2- 172.21.15.86:0/2385737546 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9b2c06c8f0 0x7f9b2c06edb0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:31.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.596+0000 7f9b56400640 1 --2- 172.21.15.86:0/2385737546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b50144e90 0x7f9b501e2900 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:31.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.596+0000 7f9b56400640 1 --2- 172.21.15.86:0/2385737546 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b500cd600 0x7f9b501e23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:31.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.596+0000 7f9b56400640 1 --2- 172.21.15.86:0/2385737546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b500cd170 0x7f9b501f7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:31.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.596+0000 7f9b56400640 1 -- 172.21.15.86:0/2385737546 >> 172.21.15.86:0/2385737546 conn(0x7f9b500b2360 msgr2=0x7f9b500af4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:31.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.596+0000 7f9b56400640 1 -- 172.21.15.86:0/2385737546 shutdown_connections 2024-04-20T21:27:31.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.596+0000 7f9b56400640 1 -- 172.21.15.86:0/2385737546 wait complete. 2024-04-20T21:27:31.613 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:27:31.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.810+0000 7f8486000640 1 Processor -- start 2024-04-20T21:27:31.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.810+0000 7f8486000640 1 -- start start 2024-04-20T21:27:31.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.812+0000 7f8486000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84800ad490 0x7f84800b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:31.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.812+0000 7f8486000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f84800b40a0 0x7f848006efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:31.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.812+0000 7f8486000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8480073bc0 con 0x7f848015aa40 2024-04-20T21:27:31.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.812+0000 7f8486000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f84800b4530 con 0x7f84800b40a0 2024-04-20T21:27:31.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.812+0000 7f8486000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8480073ea0 con 0x7f84800ad490 2024-04-20T21:27:31.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.812+0000 7f8484c00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f848015aa40 0x7f8480073100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:35848/0 (socket says 172.21.15.86:35848) 2024-04-20T21:27:31.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.812+0000 7f8484c00640 1 -- 172.21.15.86:0/1735113975 learned_addr learned my addr 172.21.15.86:0/1735113975 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:27:31.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.812+0000 7f8477e00640 1 --2- 172.21.15.86:0/1735113975 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84800ad490 0x7f84800b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:31.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.812+0000 7f847fe00640 1 -- 172.21.15.86:0/1735113975 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1205958250 0 0) 0x7f8480073bc0 con 0x7f848015aa40 2024-04-20T21:27:31.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.812+0000 7f8485600640 1 --2- 172.21.15.86:0/1735113975 >> v2:172.21.15.144:3300/0 conn(0x7f84800b40a0 0x7f848006efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:31.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.813+0000 7f847fe00640 1 -- 172.21.15.86:0/1735113975 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8464005730 con 0x7f848015aa40 2024-04-20T21:27:31.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.813+0000 7f847fe00640 1 -- 172.21.15.86:0/1735113975 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 5708406 0 0) 0x7f8464005730 con 0x7f848015aa40 2024-04-20T21:27:31.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.813+0000 7f847fe00640 1 -- 172.21.15.86:0/1735113975 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84800ad490 msgr2=0x7f84800b3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:31.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.813+0000 7f847fe00640 1 --2- 172.21.15.86:0/1735113975 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84800ad490 0x7f84800b3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:31.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.813+0000 7f847fe00640 1 -- 172.21.15.86:0/1735113975 >> v2:172.21.15.144:3300/0 conn(0x7f84800b40a0 msgr2=0x7f848006efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:31.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.813+0000 7f847fe00640 1 --2- 172.21.15.86:0/1735113975 >> v2:172.21.15.144:3300/0 conn(0x7f84800b40a0 0x7f848006efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:31.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.813+0000 7f847fe00640 1 -- 172.21.15.86:0/1735113975 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f848006e4e0 con 0x7f848015aa40 2024-04-20T21:27:31.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.813+0000 7f847fe00640 1 -- 172.21.15.86:0/1735113975 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f8470001670 con 0x7f848015aa40 2024-04-20T21:27:31.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.813+0000 7f847fe00640 1 -- 172.21.15.86:0/1735113975 >> v1:172.21.15.86:6789/0 conn(0x7f848015aa40 legacy=0x7f8480073100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:31.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.813+0000 7f847fe00640 1 --2- 172.21.15.86:0/1735113975 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84800ad490 0x7f8464005bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:31.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.813+0000 7f847fe00640 1 --2- 172.21.15.86:0/1735113975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8464006240 0x7f8464006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:31.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.813+0000 7f847fe00640 1 --2- 172.21.15.86:0/1735113975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8464006f50 0x7f84640073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:31.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.813+0000 7f847fe00640 1 -- 172.21.15.86:0/1735113975 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f84800b4530 con 0x7f8464006f50 2024-04-20T21:27:31.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.813+0000 7f847fe00640 1 -- 172.21.15.86:0/1735113975 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8480073ea0 con 0x7f8464006240 2024-04-20T21:27:31.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.813+0000 7f847fe00640 1 -- 172.21.15.86:0/1735113975 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f846400dde0 con 0x7f84800ad490 2024-04-20T21:27:31.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.813+0000 7f8485600640 1 --2- 172.21.15.86:0/1735113975 >> v2:172.21.15.144:3300/0 conn(0x7f84800b40a0 0x7f848006efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:27:31.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.813+0000 7f8485600640 1 --2- 172.21.15.86:0/1735113975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8464006f50 0x7f84640073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:31.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.814+0000 7f8484c00640 1 --2- 172.21.15.86:0/1735113975 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84800ad490 0x7f8464005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:31.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.814+0000 7f8477e00640 1 --2- 172.21.15.86:0/1735113975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8464006240 0x7f8464006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:31.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.814+0000 7f8485600640 1 -- 172.21.15.86:0/1735113975 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84800ad490 msgr2=0x7f8464005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:31.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.814+0000 7f8485600640 1 --2- 172.21.15.86:0/1735113975 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f84800ad490 0x7f8464005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:31.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.814+0000 7f8485600640 1 -- 172.21.15.86:0/1735113975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8464006240 msgr2=0x7f8464006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:31.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.814+0000 7f8485600640 1 --2- 172.21.15.86:0/1735113975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8464006240 0x7f8464006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:31.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.814+0000 7f8485600640 1 -- 172.21.15.86:0/1735113975 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8464005730 con 0x7f8464006f50 2024-04-20T21:27:31.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.815+0000 7f8485600640 1 --2- 172.21.15.86:0/1735113975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8464006f50 0x7f84640073d0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f84780073e0 tx=0x7f8464007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=885f2e7aa04e7656 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:31.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.815+0000 7f847fe00640 1 -- 172.21.15.86:0/1735113975 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8478039070 con 0x7f8464006f50 2024-04-20T21:27:31.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.815+0000 7f847fe00640 1 -- 172.21.15.86:0/1735113975 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f847802cdc0 con 0x7f8464006f50 2024-04-20T21:27:31.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.815+0000 7f847fe00640 1 -- 172.21.15.86:0/1735113975 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8478009c70 con 0x7f8464006f50 2024-04-20T21:27:31.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.816+0000 7f8486000640 1 -- 172.21.15.86:0/1735113975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8464006f50 msgr2=0x7f84640073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:31.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.816+0000 7f8486000640 1 --2- 172.21.15.86:0/1735113975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8464006f50 0x7f84640073d0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f84780073e0 tx=0x7f8464007a40 comp rx=0 tx=0).stop 2024-04-20T21:27:31.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.816+0000 7f8485600640 1 -- 172.21.15.86:0/1735113975 reap_dead start 2024-04-20T21:27:31.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.816+0000 7f8486000640 1 -- 172.21.15.86:0/1735113975 shutdown_connections 2024-04-20T21:27:31.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.816+0000 7f8486000640 1 -- 172.21.15.86:0/1735113975 >> 172.21.15.86:0/1735113975 conn(0x7f84800d0170 msgr2=0x7f84800d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:31.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.816+0000 7f8486000640 1 -- 172.21.15.86:0/1735113975 shutdown_connections 2024-04-20T21:27:31.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.816+0000 7f8486000640 1 -- 172.21.15.86:0/1735113975 wait complete. 2024-04-20T21:27:31.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.817+0000 7f8486000640 1 Processor -- start 2024-04-20T21:27:31.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.818+0000 7f8486000640 1 -- start start 2024-04-20T21:27:31.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.818+0000 7f8486000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8464006f50 0x7f84801f3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:31.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.818+0000 7f8486000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f848009c1b0 0x7f848009c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:31.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.818+0000 7f8486000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f848009cb70 0x7f84801f7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:31.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.819+0000 7f8486000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f84800ad920 con 0x7f848009cb70 2024-04-20T21:27:31.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.819+0000 7f8486000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f848006eaa0 con 0x7f848009c1b0 2024-04-20T21:27:31.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.819+0000 7f8486000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f84800b33a0 con 0x7f8464006f50 2024-04-20T21:27:31.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.819+0000 7f8485600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f848009cb70 0x7f84801f7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:31.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.819+0000 7f8485600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f848009cb70 0x7f84801f7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60318/0 (socket says 172.21.15.86:60318) 2024-04-20T21:27:31.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.819+0000 7f8485600640 1 -- 172.21.15.86:0/419957691 learned_addr learned my addr 172.21.15.86:0/419957691 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:31.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.819+0000 7f8484c00640 1 --2- 172.21.15.86:0/419957691 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8464006f50 0x7f84801f3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:31.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.819+0000 7f8477e00640 1 --2- 172.21.15.86:0/419957691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f848009c1b0 0x7f848009c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:31.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.819+0000 7f8485600640 1 -- 172.21.15.86:0/419957691 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8464006f50 msgr2=0x7f84801f3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:31.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.819+0000 7f8485600640 1 --2- 172.21.15.86:0/419957691 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8464006f50 0x7f84801f3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:31.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.819+0000 7f8485600640 1 -- 172.21.15.86:0/419957691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f848009c1b0 msgr2=0x7f848009c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:31.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.819+0000 7f8485600640 1 --2- 172.21.15.86:0/419957691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f848009c1b0 0x7f848009c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:31.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.819+0000 7f8485600640 1 -- 172.21.15.86:0/419957691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f848009cff0 con 0x7f848009cb70 2024-04-20T21:27:31.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.820+0000 7f8485600640 1 --2- 172.21.15.86:0/419957691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f848009cb70 0x7f84801f7ae0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f8478007500 tx=0x7f8478004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:31.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.820+0000 7f847d600640 1 -- 172.21.15.86:0/419957691 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8478042020 con 0x7f848009cb70 2024-04-20T21:27:31.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.820+0000 7f847d600640 1 -- 172.21.15.86:0/419957691 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f847803a040 con 0x7f848009cb70 2024-04-20T21:27:31.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.820+0000 7f847d600640 1 -- 172.21.15.86:0/419957691 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8478039070 con 0x7f848009cb70 2024-04-20T21:27:31.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.820+0000 7f8486000640 1 -- 172.21.15.86:0/419957691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f848009dd00 con 0x7f848009cb70 2024-04-20T21:27:31.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.820+0000 7f8486000640 1 -- 172.21.15.86:0/419957691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8480146580 con 0x7f848009cb70 2024-04-20T21:27:31.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.821+0000 7f8486000640 1 -- 172.21.15.86:0/419957691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8480146780 con 0x7f848009cb70 2024-04-20T21:27:31.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.822+0000 7f847d600640 1 -- 172.21.15.86:0/419957691 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8478045020 con 0x7f848009cb70 2024-04-20T21:27:31.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.823+0000 7f847d600640 1 --2- 172.21.15.86:0/419957691 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f845c06c920 0x7f845c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:31.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.823+0000 7f847d600640 1 -- 172.21.15.86:0/419957691 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7f84780ada40 con 0x7f848009cb70 2024-04-20T21:27:31.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.823+0000 7f8484c00640 1 --2- 172.21.15.86:0/419957691 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f845c06c920 0x7f845c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:31.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.824+0000 7f8484c00640 1 --2- 172.21.15.86:0/419957691 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f845c06c920 0x7f845c06ede0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f84700057f0 tx=0x7f8470000f90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:31.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:31.828+0000 7f847d600640 1 -- 172.21.15.86:0/419957691 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8480146780 con 0x7f848009cb70 2024-04-20T21:27:32.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.070+0000 7f8486000640 1 -- 172.21.15.86:0/419957691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f848004fcf0 con 0x7f848009cb70 2024-04-20T21:27:32.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.071+0000 7f847d600640 1 -- 172.21.15.86:0/419957691 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5209 (secure 0 0 0) 0x7f847804c030 con 0x7f848009cb70 2024-04-20T21:27:32.070 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:27:32.070 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":73,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5887,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2433619638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2433619638},{"type":"v1","addr":"172.21.15.144:6835","nonce":2433619638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68},{"gid":7506,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/4013863257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4013863257},{"type":"v1","addr":"172.21.15.86:6836","nonce":4013863257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":7509,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2044883073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2044883073},{"type":"v1","addr":"172.21.15.86:6837","nonce":2044883073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:24:58.299257+0000","modified":"2024-04-20T21:26:48.911892+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7518},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7518":{"gid":7518,"name":"d","rank":0,"incarnation":70,"state":"up:active","state_seq":32,"addr":"172.21.15.144:6834/2351381466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":2351381466},{"type":"v1","addr":"172.21.15.144:6834","nonce":2351381466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7518,"qdb_cluster":[7518]},"id":6}]} 2024-04-20T21:27:32.071 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 73 2024-04-20T21:27:32.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.074+0000 7f8486000640 1 -- 172.21.15.86:0/419957691 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f845c06c920 msgr2=0x7f845c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:32.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.074+0000 7f8486000640 1 --2- 172.21.15.86:0/419957691 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f845c06c920 0x7f845c06ede0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f84700057f0 tx=0x7f8470000f90 comp rx=0 tx=0).stop 2024-04-20T21:27:32.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.074+0000 7f8486000640 1 -- 172.21.15.86:0/419957691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f848009cb70 msgr2=0x7f84801f7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:32.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.074+0000 7f8486000640 1 --2- 172.21.15.86:0/419957691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f848009cb70 0x7f84801f7ae0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f8478007500 tx=0x7f8478004ab0 comp rx=0 tx=0).stop 2024-04-20T21:27:32.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.074+0000 7f8486000640 1 -- 172.21.15.86:0/419957691 shutdown_connections 2024-04-20T21:27:32.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.074+0000 7f8486000640 1 --2- 172.21.15.86:0/419957691 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f845c06c920 0x7f845c06ede0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:32.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.074+0000 7f8486000640 1 --2- 172.21.15.86:0/419957691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f848009cb70 0x7f84801f7ae0 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:32.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.074+0000 7f8486000640 1 --2- 172.21.15.86:0/419957691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f848009c1b0 0x7f848009c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:32.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.074+0000 7f8486000640 1 --2- 172.21.15.86:0/419957691 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8464006f50 0x7f84801f3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:32.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.074+0000 7f8486000640 1 -- 172.21.15.86:0/419957691 >> 172.21.15.86:0/419957691 conn(0x7f84800d0170 msgr2=0x7f84800733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:32.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.074+0000 7f8486000640 1 -- 172.21.15.86:0/419957691 shutdown_connections 2024-04-20T21:27:32.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.074+0000 7f8486000640 1 -- 172.21.15.86:0/419957691 wait complete. 2024-04-20T21:27:32.092 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:27:32.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.276+0000 7f2787800640 1 Processor -- start 2024-04-20T21:27:32.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.276+0000 7f2787800640 1 -- start start 2024-04-20T21:27:32.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.278+0000 7f2787800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f27800ce090 0x7f2780070090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:32.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.278+0000 7f2787800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27800b01b0 con 0x7f278006c990 2024-04-20T21:27:32.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.278+0000 7f2787800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27800b0490 con 0x7f278015ca50 2024-04-20T21:27:32.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.278+0000 7f2787800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f2780161250 con 0x7f27800ce090 2024-04-20T21:27:32.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.278+0000 7f2784e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f278006c990 0x7f278006cd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:35862/0 (socket says 172.21.15.86:35862) 2024-04-20T21:27:32.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.278+0000 7f2784e00640 1 -- 172.21.15.86:0/355362827 learned_addr learned my addr 172.21.15.86:0/355362827 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:27:32.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.279+0000 7f2785800640 1 --2- 172.21.15.86:0/355362827 >> v2:172.21.15.144:3300/0 conn(0x7f27800ce090 0x7f2780070090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:32.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.279+0000 7f277f400640 1 -- 172.21.15.86:0/355362827 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1351218377 0 0) 0x7f27800b01b0 con 0x7f278006c990 2024-04-20T21:27:32.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.279+0000 7f277f400640 1 -- 172.21.15.86:0/355362827 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2764003520 con 0x7f278006c990 2024-04-20T21:27:32.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.279+0000 7f277f400640 1 -- 172.21.15.86:0/355362827 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2159822327 0 0) 0x7f2764003520 con 0x7f278006c990 2024-04-20T21:27:32.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.279+0000 7f277f400640 1 -- 172.21.15.86:0/355362827 >> v2:172.21.15.144:3300/0 conn(0x7f27800ce090 msgr2=0x7f2780070090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:32.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.279+0000 7f277f400640 1 --2- 172.21.15.86:0/355362827 >> v2:172.21.15.144:3300/0 conn(0x7f27800ce090 0x7f2780070090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:32.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.279+0000 7f277f400640 1 -- 172.21.15.86:0/355362827 >> v1:172.21.15.144:6789/0 conn(0x7f278015ca50 legacy=0x7f2780160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:32.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.279+0000 7f277f400640 1 -- 172.21.15.86:0/355362827 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2780160340 con 0x7f278006c990 2024-04-20T21:27:32.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.279+0000 7f277f400640 1 -- 172.21.15.86:0/355362827 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f27700019f0 con 0x7f278006c990 2024-04-20T21:27:32.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.279+0000 7f277f400640 1 -- 172.21.15.86:0/355362827 >> v1:172.21.15.86:6789/0 conn(0x7f278006c990 legacy=0x7f278006cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:32.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.279+0000 7f277f400640 1 --2- 172.21.15.86:0/355362827 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f27640039b0 0x7f2764003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:32.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.279+0000 7f277f400640 1 --2- 172.21.15.86:0/355362827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2764004800 0x7f2764004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:32.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.279+0000 7f277f400640 1 --2- 172.21.15.86:0/355362827 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27640051c0 0x7f276400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:32.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.279+0000 7f277f400640 1 -- 172.21.15.86:0/355362827 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2780161250 con 0x7f2764004800 2024-04-20T21:27:32.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.279+0000 7f277f400640 1 -- 172.21.15.86:0/355362827 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f276400c110 con 0x7f27640051c0 2024-04-20T21:27:32.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.279+0000 7f277f400640 1 -- 172.21.15.86:0/355362827 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f276400c310 con 0x7f27640039b0 2024-04-20T21:27:32.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.279+0000 7f2785800640 1 --2- 172.21.15.86:0/355362827 >> v2:172.21.15.144:3300/0 conn(0x7f27800ce090 0x7f2780070090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:27:32.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.280+0000 7f277fe00640 1 --2- 172.21.15.86:0/355362827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2764004800 0x7f2764004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:32.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.280+0000 7f2784e00640 1 --2- 172.21.15.86:0/355362827 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f27640039b0 0x7f2764003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:32.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.280+0000 7f2785800640 1 --2- 172.21.15.86:0/355362827 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27640051c0 0x7f276400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:32.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.280+0000 7f277fe00640 1 -- 172.21.15.86:0/355362827 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f27640039b0 msgr2=0x7f2764003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:32.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.280+0000 7f277fe00640 1 --2- 172.21.15.86:0/355362827 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f27640039b0 0x7f2764003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:32.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.280+0000 7f277fe00640 1 -- 172.21.15.86:0/355362827 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27640051c0 msgr2=0x7f276400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:32.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.280+0000 7f277fe00640 1 --2- 172.21.15.86:0/355362827 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27640051c0 0x7f276400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:32.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.280+0000 7f277fe00640 1 -- 172.21.15.86:0/355362827 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2764003520 con 0x7f2764004800 2024-04-20T21:27:32.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.280+0000 7f2785800640 1 -- 172.21.15.86:0/355362827 reap_dead start 2024-04-20T21:27:32.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.281+0000 7f277fe00640 1 --2- 172.21.15.86:0/355362827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2764004800 0x7f2764004c80 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f276c001750 tx=0x7f2764005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24569f36d48391d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:32.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.281+0000 7f277f400640 1 -- 172.21.15.86:0/355362827 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276c001370 con 0x7f2764004800 2024-04-20T21:27:32.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.281+0000 7f277f400640 1 -- 172.21.15.86:0/355362827 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f276c0052d0 con 0x7f2764004800 2024-04-20T21:27:32.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.281+0000 7f277f400640 1 -- 172.21.15.86:0/355362827 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276c003830 con 0x7f2764004800 2024-04-20T21:27:32.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.282+0000 7f2787800640 1 -- 172.21.15.86:0/355362827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2764004800 msgr2=0x7f2764004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:32.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.282+0000 7f2787800640 1 --2- 172.21.15.86:0/355362827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2764004800 0x7f2764004c80 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f276c001750 tx=0x7f2764005830 comp rx=0 tx=0).stop 2024-04-20T21:27:32.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.282+0000 7f2787800640 1 -- 172.21.15.86:0/355362827 shutdown_connections 2024-04-20T21:27:32.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.282+0000 7f2787800640 1 --2- 172.21.15.86:0/355362827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2764004800 0x7f2764004c80 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:32.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.282+0000 7f2787800640 1 -- 172.21.15.86:0/355362827 >> 172.21.15.86:0/355362827 conn(0x7f2780072540 msgr2=0x7f2780072950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:32.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.282+0000 7f2787800640 1 -- 172.21.15.86:0/355362827 shutdown_connections 2024-04-20T21:27:32.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.282+0000 7f2787800640 1 -- 172.21.15.86:0/355362827 wait complete. 2024-04-20T21:27:32.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.284+0000 7f2787800640 1 Processor -- start 2024-04-20T21:27:32.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.284+0000 7f2787800640 1 -- start start 2024-04-20T21:27:32.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.285+0000 7f2787800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2764004800 0x7f278014f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:32.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.285+0000 7f2787800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f278013fda0 0x7f2780140220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:32.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.285+0000 7f2787800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f27800b5010 0x7f27800b5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:32.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.285+0000 7f2787800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2780070090 con 0x7f278013fda0 2024-04-20T21:27:32.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.285+0000 7f2787800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2780073750 con 0x7f2764004800 2024-04-20T21:27:32.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.285+0000 7f2787800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2780070210 con 0x7f27800b5010 2024-04-20T21:27:32.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.285+0000 7f277fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f278013fda0 0x7f2780140220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:32.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.285+0000 7f277fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f278013fda0 0x7f2780140220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60338/0 (socket says 172.21.15.86:60338) 2024-04-20T21:27:32.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.285+0000 7f277fe00640 1 -- 172.21.15.86:0/3416700072 learned_addr learned my addr 172.21.15.86:0/3416700072 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:32.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.286+0000 7f2784e00640 1 --2- 172.21.15.86:0/3416700072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2764004800 0x7f278014f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:32.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.286+0000 7f2785800640 1 --2- 172.21.15.86:0/3416700072 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f27800b5010 0x7f27800b5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:32.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.286+0000 7f277fe00640 1 -- 172.21.15.86:0/3416700072 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f27800b5010 msgr2=0x7f27800b5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:32.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.286+0000 7f277fe00640 1 --2- 172.21.15.86:0/3416700072 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f27800b5010 0x7f27800b5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:32.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.286+0000 7f277fe00640 1 -- 172.21.15.86:0/3416700072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2764004800 msgr2=0x7f278014f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:32.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.286+0000 7f277fe00640 1 --2- 172.21.15.86:0/3416700072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2764004800 0x7f278014f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:32.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.286+0000 7f277fe00640 1 -- 172.21.15.86:0/3416700072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27801501c0 con 0x7f278013fda0 2024-04-20T21:27:32.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.286+0000 7f2785800640 1 --2- 172.21.15.86:0/3416700072 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f27800b5010 0x7f27800b5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:27:32.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.286+0000 7f277fe00640 1 --2- 172.21.15.86:0/3416700072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f278013fda0 0x7f2780140220 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f276c000c00 tx=0x7f276c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:32.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.286+0000 7f277cc00640 1 -- 172.21.15.86:0/3416700072 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276c041020 con 0x7f278013fda0 2024-04-20T21:27:32.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.286+0000 7f277cc00640 1 -- 172.21.15.86:0/3416700072 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f276c005480 con 0x7f278013fda0 2024-04-20T21:27:32.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.286+0000 7f277cc00640 1 -- 172.21.15.86:0/3416700072 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276c007a30 con 0x7f278013fda0 2024-04-20T21:27:32.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.286+0000 7f2787800640 1 -- 172.21.15.86:0/3416700072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27801504a0 con 0x7f278013fda0 2024-04-20T21:27:32.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.287+0000 7f2787800640 1 -- 172.21.15.86:0/3416700072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2780150a60 con 0x7f278013fda0 2024-04-20T21:27:32.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.288+0000 7f2787800640 1 -- 172.21.15.86:0/3416700072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2780070af0 con 0x7f278013fda0 2024-04-20T21:27:32.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.289+0000 7f277cc00640 1 -- 172.21.15.86:0/3416700072 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f276c044020 con 0x7f278013fda0 2024-04-20T21:27:32.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.290+0000 7f277cc00640 1 --2- 172.21.15.86:0/3416700072 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f276006c920 0x7f276006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:32.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.290+0000 7f277cc00640 1 -- 172.21.15.86:0/3416700072 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7f276c0ad5d0 con 0x7f278013fda0 2024-04-20T21:27:32.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.290+0000 7f2784e00640 1 --2- 172.21.15.86:0/3416700072 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f276006c920 0x7f276006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:32.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.291+0000 7f2784e00640 1 --2- 172.21.15.86:0/3416700072 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f276006c920 0x7f276006ede0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f2780144420 tx=0x7f2770009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:32.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.294+0000 7f277cc00640 1 -- 172.21.15.86:0/3416700072 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2780070af0 con 0x7f278013fda0 2024-04-20T21:27:32.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.495+0000 7f2787800640 1 -- 172.21.15.86:0/3416700072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f27800689e0 con 0x7f278013fda0 2024-04-20T21:27:32.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.497+0000 7f277cc00640 1 -- 172.21.15.86:0/3416700072 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+16973 (secure 0 0 0) 0x7f276c04b030 con 0x7f278013fda0 2024-04-20T21:27:32.496 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:27:32.497 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":89,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:26:43.869364+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:24:55.964540+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-20T21:24:57.241527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.3e","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.144:6835/3981403733":"2024-04-21T21:26:43.869334+0000","172.21.15.144:6833/3981403733":"2024-04-21T21:26:43.869334+0000","172.21.15.144:6835/1740492227":"2024-04-21T21:25:20.040733+0000","172.21.15.144:6833/1740492227":"2024-04-21T21:25:20.040733+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:27:32.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.500+0000 7f2787800640 1 -- 172.21.15.86:0/3416700072 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f276006c920 msgr2=0x7f276006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:32.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.500+0000 7f2787800640 1 --2- 172.21.15.86:0/3416700072 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f276006c920 0x7f276006ede0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f2780144420 tx=0x7f2770009040 comp rx=0 tx=0).stop 2024-04-20T21:27:32.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.500+0000 7f2787800640 1 -- 172.21.15.86:0/3416700072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f278013fda0 msgr2=0x7f2780140220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:32.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.500+0000 7f2787800640 1 --2- 172.21.15.86:0/3416700072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f278013fda0 0x7f2780140220 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f276c000c00 tx=0x7f276c001120 comp rx=0 tx=0).stop 2024-04-20T21:27:32.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.500+0000 7f2787800640 1 -- 172.21.15.86:0/3416700072 shutdown_connections 2024-04-20T21:27:32.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.500+0000 7f2787800640 1 --2- 172.21.15.86:0/3416700072 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f276006c920 0x7f276006ede0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:32.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.500+0000 7f2787800640 1 --2- 172.21.15.86:0/3416700072 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f27800b5010 0x7f27800b5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:32.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.500+0000 7f2787800640 1 --2- 172.21.15.86:0/3416700072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f278013fda0 0x7f2780140220 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:32.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.500+0000 7f2787800640 1 --2- 172.21.15.86:0/3416700072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2764004800 0x7f278014f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:32.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.500+0000 7f2787800640 1 -- 172.21.15.86:0/3416700072 >> 172.21.15.86:0/3416700072 conn(0x7f2780072540 msgr2=0x7f27800b14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:32.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.500+0000 7f2787800640 1 -- 172.21.15.86:0/3416700072 shutdown_connections 2024-04-20T21:27:32.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.500+0000 7f2787800640 1 -- 172.21.15.86:0/3416700072 wait complete. 2024-04-20T21:27:32.516 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-20T21:27:32.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.703+0000 7f44af000640 1 Processor -- start 2024-04-20T21:27:32.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.703+0000 7f44af000640 1 -- start start 2024-04-20T21:27:32.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.704+0000 7f44af000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f44a80af870 0x7f44a8071b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:32.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.704+0000 7f44af000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44a80ce1c0 con 0x7f44a8073540 2024-04-20T21:27:32.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.704+0000 7f44af000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44a80ce4a0 con 0x7f44a80d07c0 2024-04-20T21:27:32.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.704+0000 7f44af000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f44a80ce780 con 0x7f44a80af870 2024-04-20T21:27:32.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.705+0000 7f44adc00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f44a8073540 0x7f44a80d00b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:35868/0 (socket says 172.21.15.86:35868) 2024-04-20T21:27:32.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.705+0000 7f44adc00640 1 -- 172.21.15.86:0/1441399042 learned_addr learned my addr 172.21.15.86:0/1441399042 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:27:32.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.705+0000 7f44a7e00640 1 -- 172.21.15.86:0/1441399042 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 431149959 0 0) 0x7f44a80ce1c0 con 0x7f44a8073540 2024-04-20T21:27:32.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.705+0000 7f44a7e00640 1 -- 172.21.15.86:0/1441399042 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4490005730 con 0x7f44a8073540 2024-04-20T21:27:32.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.705+0000 7f44ad200640 1 --2- 172.21.15.86:0/1441399042 >> v2:172.21.15.144:3300/0 conn(0x7f44a80af870 0x7f44a8071b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:32.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.706+0000 7f44a7e00640 1 -- 172.21.15.86:0/1441399042 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4289612592 0 0) 0x7f4490005730 con 0x7f44a8073540 2024-04-20T21:27:32.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.706+0000 7f44a7e00640 1 -- 172.21.15.86:0/1441399042 >> v2:172.21.15.144:3300/0 conn(0x7f44a80af870 msgr2=0x7f44a8071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:32.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.706+0000 7f44a7e00640 1 --2- 172.21.15.86:0/1441399042 >> v2:172.21.15.144:3300/0 conn(0x7f44a80af870 0x7f44a8071b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:32.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.706+0000 7f44a7e00640 1 -- 172.21.15.86:0/1441399042 >> v1:172.21.15.144:6789/0 conn(0x7f44a80d07c0 legacy=0x7f44a80720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:32.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.706+0000 7f44a7e00640 1 -- 172.21.15.86:0/1441399042 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a80cea60 con 0x7f44a8073540 2024-04-20T21:27:32.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.706+0000 7f44a7e00640 1 -- 172.21.15.86:0/1441399042 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f449c0019f0 con 0x7f44a8073540 2024-04-20T21:27:32.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.706+0000 7f44ad200640 1 --2- 172.21.15.86:0/1441399042 >> v2:172.21.15.144:3300/0 conn(0x7f44a80af870 0x7f44a8071b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:27:32.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.706+0000 7f44a7e00640 1 -- 172.21.15.86:0/1441399042 >> v1:172.21.15.86:6789/0 conn(0x7f44a8073540 legacy=0x7f44a80d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:32.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.706+0000 7f44a7e00640 1 --2- 172.21.15.86:0/1441399042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4490005c10 0x7f4490006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:32.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.706+0000 7f44a7e00640 1 --2- 172.21.15.86:0/1441399042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4490006a10 0x7f4490006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:32.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.706+0000 7f44a7e00640 1 --2- 172.21.15.86:0/1441399042 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4490007420 0x7f449000dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:32.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.706+0000 7f44a7e00640 1 -- 172.21.15.86:0/1441399042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f44a80ce780 con 0x7f4490005c10 2024-04-20T21:27:32.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.706+0000 7f44a7e00640 1 -- 172.21.15.86:0/1441399042 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f449000e350 con 0x7f4490006a10 2024-04-20T21:27:32.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.706+0000 7f44a7e00640 1 -- 172.21.15.86:0/1441399042 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f449000e550 con 0x7f4490007420 2024-04-20T21:27:32.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.706+0000 7f44adc00640 1 --2- 172.21.15.86:0/1441399042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4490005c10 0x7f4490006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:32.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.706+0000 7f44adc00640 1 -- 172.21.15.86:0/1441399042 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4490007420 msgr2=0x7f449000dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:32.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.706+0000 7f44adc00640 1 --2- 172.21.15.86:0/1441399042 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4490007420 0x7f449000dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:32.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.706+0000 7f44adc00640 1 -- 172.21.15.86:0/1441399042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4490006a10 msgr2=0x7f4490006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:32.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.706+0000 7f44adc00640 1 --2- 172.21.15.86:0/1441399042 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4490006a10 0x7f4490006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:32.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.706+0000 7f44adc00640 1 -- 172.21.15.86:0/1441399042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4490005730 con 0x7f4490005c10 2024-04-20T21:27:32.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.706+0000 7f44ae600640 1 -- 172.21.15.86:0/1441399042 reap_dead start 2024-04-20T21:27:32.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.707+0000 7f44adc00640 1 --2- 172.21.15.86:0/1441399042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4490005c10 0x7f4490006010 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f449c00a760 tx=0x7f4490007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=598cc97cedcf1af server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:32.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.708+0000 7f44a7e00640 1 -- 172.21.15.86:0/1441399042 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449c00a3b0 con 0x7f4490005c10 2024-04-20T21:27:32.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.708+0000 7f44a7e00640 1 -- 172.21.15.86:0/1441399042 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f449c034630 con 0x7f4490005c10 2024-04-20T21:27:32.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.708+0000 7f44a7e00640 1 -- 172.21.15.86:0/1441399042 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449c034a20 con 0x7f4490005c10 2024-04-20T21:27:32.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.708+0000 7f44af000640 1 -- 172.21.15.86:0/1441399042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4490005c10 msgr2=0x7f4490006010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:32.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.708+0000 7f44af000640 1 --2- 172.21.15.86:0/1441399042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4490005c10 0x7f4490006010 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f449c00a760 tx=0x7f4490007a40 comp rx=0 tx=0).stop 2024-04-20T21:27:32.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.708+0000 7f44af000640 1 -- 172.21.15.86:0/1441399042 shutdown_connections 2024-04-20T21:27:32.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.708+0000 7f44af000640 1 --2- 172.21.15.86:0/1441399042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4490005c10 0x7f4490006010 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:32.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.708+0000 7f44af000640 1 -- 172.21.15.86:0/1441399042 >> 172.21.15.86:0/1441399042 conn(0x7f44a806bb60 msgr2=0x7f44a806bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:32.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.708+0000 7f44af000640 1 -- 172.21.15.86:0/1441399042 shutdown_connections 2024-04-20T21:27:32.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.708+0000 7f44af000640 1 -- 172.21.15.86:0/1441399042 wait complete. 2024-04-20T21:27:32.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.710+0000 7f44af000640 1 Processor -- start 2024-04-20T21:27:32.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.710+0000 7f44af000640 1 -- start start 2024-04-20T21:27:32.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.711+0000 7f44af000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4490005c10 0x7f44a80aec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:32.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.711+0000 7f44af000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44a815d630 0x7f44a80c4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:32.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.711+0000 7f44af000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44a80c5290 0x7f44a80c5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:32.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.711+0000 7f44af000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f44a80cfa00 con 0x7f44a80c5290 2024-04-20T21:27:32.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.711+0000 7f44af000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f44a8073920 con 0x7f44a815d630 2024-04-20T21:27:32.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.711+0000 7f44af000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f44a80cfb80 con 0x7f4490005c10 2024-04-20T21:27:32.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.711+0000 7f44ae600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44a80c5290 0x7f44a80c5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:32.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.711+0000 7f44ae600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44a80c5290 0x7f44a80c5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60362/0 (socket says 172.21.15.86:60362) 2024-04-20T21:27:32.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.711+0000 7f44ae600640 1 -- 172.21.15.86:0/2069202987 learned_addr learned my addr 172.21.15.86:0/2069202987 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:32.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.712+0000 7f44ad200640 1 --2- 172.21.15.86:0/2069202987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44a815d630 0x7f44a80c4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:32.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.712+0000 7f44adc00640 1 --2- 172.21.15.86:0/2069202987 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4490005c10 0x7f44a80aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:32.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.712+0000 7f44ae600640 1 -- 172.21.15.86:0/2069202987 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4490005c10 msgr2=0x7f44a80aec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:32.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.712+0000 7f44ae600640 1 --2- 172.21.15.86:0/2069202987 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4490005c10 0x7f44a80aec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:32.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.712+0000 7f44ae600640 1 -- 172.21.15.86:0/2069202987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44a815d630 msgr2=0x7f44a80c4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:32.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.712+0000 7f44ae600640 1 --2- 172.21.15.86:0/2069202987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44a815d630 0x7f44a80c4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:32.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.712+0000 7f44ae600640 1 -- 172.21.15.86:0/2069202987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a80c5c80 con 0x7f44a80c5290 2024-04-20T21:27:32.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.712+0000 7f44adc00640 1 --2- 172.21.15.86:0/2069202987 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4490005c10 0x7f44a80aec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:27:32.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.712+0000 7f44ad200640 1 --2- 172.21.15.86:0/2069202987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44a815d630 0x7f44a80c4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:27:32.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.712+0000 7f44ae600640 1 --2- 172.21.15.86:0/2069202987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44a80c5290 0x7f44a80c5740 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f44a80c3c20 tx=0x7f44a000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:32.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.712+0000 7f44a5600640 1 -- 172.21.15.86:0/2069202987 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a0017070 con 0x7f44a80c5290 2024-04-20T21:27:32.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.712+0000 7f44a5600640 1 -- 172.21.15.86:0/2069202987 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44a0007500 con 0x7f44a80c5290 2024-04-20T21:27:32.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.712+0000 7f44af000640 1 -- 172.21.15.86:0/2069202987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44a815b210 con 0x7f44a80c5290 2024-04-20T21:27:32.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.713+0000 7f44a5600640 1 -- 172.21.15.86:0/2069202987 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a0005a00 con 0x7f44a80c5290 2024-04-20T21:27:32.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.713+0000 7f44af000640 1 -- 172.21.15.86:0/2069202987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44a815de10 con 0x7f44a80c5290 2024-04-20T21:27:32.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.714+0000 7f44af000640 1 -- 172.21.15.86:0/2069202987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44a804fcf0 con 0x7f44a80c5290 2024-04-20T21:27:32.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.715+0000 7f44a5600640 1 -- 172.21.15.86:0/2069202987 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f44a001a070 con 0x7f44a80c5290 2024-04-20T21:27:32.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.715+0000 7f44a5600640 1 --2- 172.21.15.86:0/2069202987 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f448806c8f0 0x7f448806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:32.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.716+0000 7f44a5600640 1 -- 172.21.15.86:0/2069202987 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7f44a008ac00 con 0x7f44a80c5290 2024-04-20T21:27:32.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.716+0000 7f44adc00640 1 --2- 172.21.15.86:0/2069202987 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f448806c8f0 0x7f448806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:32.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.716+0000 7f44adc00640 1 --2- 172.21.15.86:0/2069202987 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f448806c8f0 0x7f448806edb0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f449c034000 tx=0x7f449c036040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:32.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.720+0000 7f44a5600640 1 -- 172.21.15.86:0/2069202987 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f44a80c5c80 con 0x7f44a80c5290 2024-04-20T21:27:32.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.960+0000 7f44af000640 1 -- 172.21.15.86:0/2069202987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f44a8099e00 con 0x7f44a80c5290 2024-04-20T21:27:32.960 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-04-20T21:27:32.961+0000 7f5bb9c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T21:27:32.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.966+0000 7f44a5600640 1 -- 172.21.15.86:0/2069202987 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v74) ==== 161+0+0 (secure 0 0 0) 0x7f44a0058ec0 con 0x7f44a80c5290 2024-04-20T21:27:32.965 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-20T21:27:32.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.969+0000 7f44af000640 1 -- 172.21.15.86:0/2069202987 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f448806c8f0 msgr2=0x7f448806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:32.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.969+0000 7f44af000640 1 --2- 172.21.15.86:0/2069202987 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f448806c8f0 0x7f448806edb0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f449c034000 tx=0x7f449c036040 comp rx=0 tx=0).stop 2024-04-20T21:27:32.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.969+0000 7f44af000640 1 -- 172.21.15.86:0/2069202987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44a80c5290 msgr2=0x7f44a80c5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:32.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.969+0000 7f44af000640 1 --2- 172.21.15.86:0/2069202987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44a80c5290 0x7f44a80c5740 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f44a80c3c20 tx=0x7f44a000c820 comp rx=0 tx=0).stop 2024-04-20T21:27:32.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.969+0000 7f44af000640 1 -- 172.21.15.86:0/2069202987 shutdown_connections 2024-04-20T21:27:32.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.969+0000 7f44af000640 1 --2- 172.21.15.86:0/2069202987 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f448806c8f0 0x7f448806edb0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:32.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.969+0000 7f44af000640 1 --2- 172.21.15.86:0/2069202987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44a80c5290 0x7f44a80c5740 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:32.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.969+0000 7f44af000640 1 --2- 172.21.15.86:0/2069202987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44a815d630 0x7f44a80c4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:32.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.969+0000 7f44af000640 1 --2- 172.21.15.86:0/2069202987 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4490005c10 0x7f44a80aec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:32.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.969+0000 7f44af000640 1 -- 172.21.15.86:0/2069202987 >> 172.21.15.86:0/2069202987 conn(0x7f44a806bb60 msgr2=0x7f44a80992d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:32.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.970+0000 7f44af000640 1 -- 172.21.15.86:0/2069202987 shutdown_connections 2024-04-20T21:27:32.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:32.970+0000 7f44af000640 1 -- 172.21.15.86:0/2069202987 wait complete. 2024-04-20T21:27:32.988 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-20T21:27:32.998 INFO:tasks.ceph.mds.d.smithi144.stderr: -2516> 2024-04-20T21:26:49.805+0000 7fec9ae00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T21:27:33.001 INFO:tasks.ceph.mds.d.smithi144.stderr: -2037> 2024-04-20T21:26:53.177+0000 7fec95e00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() | dirtyparent=1 authpin=1 scrubqueue=0 0x564e4b572680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(13)0x10000000001:[<0x10000000000/file v6>,<0x1/dir v13>]//[]","memoryvalue":"(13)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-20T21:27:33.080 INFO:tasks.ceph.mds.d.smithi144.stdout:starting mds.d at 2024-04-20T21:27:33.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.185+0000 7fda15400640 1 Processor -- start 2024-04-20T21:27:33.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.185+0000 7fda15400640 1 -- start start 2024-04-20T21:27:33.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.186+0000 7fda15400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fda10146210 0x7fda1006e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:33.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.187+0000 7fda15400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fda1006bfd0 0x7fda100cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:33.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.187+0000 7fda15400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda1015cd40 con 0x7fda1006bbf0 2024-04-20T21:27:33.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.187+0000 7fda15400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fda1015d000 con 0x7fda10146210 2024-04-20T21:27:33.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.187+0000 7fda15400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fda1015d2e0 con 0x7fda1006bfd0 2024-04-20T21:27:33.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.187+0000 7fda07e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fda10146210 0x7fda1006e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:33.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.187+0000 7fda07e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fda10146210 0x7fda1006e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60368/0 (socket says 172.21.15.86:60368) 2024-04-20T21:27:33.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.187+0000 7fda07e00640 1 -- 172.21.15.86:0/2414433477 learned_addr learned my addr 172.21.15.86:0/2414433477 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:33.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.187+0000 7fda0f200640 1 --2- 172.21.15.86:0/2414433477 >> v2:172.21.15.144:3300/0 conn(0x7fda1006bfd0 0x7fda100cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:33.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.187+0000 7fda07e00640 1 -- 172.21.15.86:0/2414433477 >> v2:172.21.15.144:3300/0 conn(0x7fda1006bfd0 msgr2=0x7fda100cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:33.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.187+0000 7fda07e00640 1 --2- 172.21.15.86:0/2414433477 >> v2:172.21.15.144:3300/0 conn(0x7fda1006bfd0 0x7fda100cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:33.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.187+0000 7fda07e00640 1 -- 172.21.15.86:0/2414433477 >> v1:172.21.15.144:6789/0 conn(0x7fda1006bbf0 legacy=0x7fda10145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:33.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.187+0000 7fda07e00640 1 -- 172.21.15.86:0/2414433477 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda1015d5c0 con 0x7fda10146210 2024-04-20T21:27:33.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.187+0000 7fda0f200640 1 --2- 172.21.15.86:0/2414433477 >> v2:172.21.15.144:3300/0 conn(0x7fda1006bfd0 0x7fda100cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:27:33.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.188+0000 7fda07e00640 1 --2- 172.21.15.86:0/2414433477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda10146210 0x7fda1006e000 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fda1006f6c0 tx=0x7fd9f802b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f64586bec4107ef6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:33.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.189+0000 7fda0de00640 1 -- 172.21.15.86:0/2414433477 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8035020 con 0x7fda10146210 2024-04-20T21:27:33.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.189+0000 7fda0de00640 1 -- 172.21.15.86:0/2414433477 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9f8031890 con 0x7fda10146210 2024-04-20T21:27:33.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.189+0000 7fda0de00640 1 -- 172.21.15.86:0/2414433477 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8031c60 con 0x7fda10146210 2024-04-20T21:27:33.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.189+0000 7fda15400640 1 -- 172.21.15.86:0/2414433477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda10146210 msgr2=0x7fda1006e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:33.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.189+0000 7fda15400640 1 --2- 172.21.15.86:0/2414433477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda10146210 0x7fda1006e000 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fda1006f6c0 tx=0x7fd9f802b930 comp rx=0 tx=0).stop 2024-04-20T21:27:33.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.189+0000 7fda15400640 1 -- 172.21.15.86:0/2414433477 shutdown_connections 2024-04-20T21:27:33.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.189+0000 7fda15400640 1 --2- 172.21.15.86:0/2414433477 >> v2:172.21.15.144:3300/0 conn(0x7fda1006bfd0 0x7fda100cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:33.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.189+0000 7fda15400640 1 --2- 172.21.15.86:0/2414433477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda10146210 0x7fda1006e000 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:33.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.190+0000 7fda15400640 1 -- 172.21.15.86:0/2414433477 >> 172.21.15.86:0/2414433477 conn(0x7fda100b2110 msgr2=0x7fda100b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:33.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.190+0000 7fda15400640 1 -- 172.21.15.86:0/2414433477 shutdown_connections 2024-04-20T21:27:33.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.190+0000 7fda15400640 1 -- 172.21.15.86:0/2414433477 wait complete. 2024-04-20T21:27:33.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.191+0000 7fda15400640 1 Processor -- start 2024-04-20T21:27:33.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.191+0000 7fda15400640 1 -- start start 2024-04-20T21:27:33.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.192+0000 7fda15400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda1006bbf0 0x7fda101f75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:33.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.192+0000 7fda15400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda1006bfd0 0x7fda100c5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:33.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.192+0000 7fda15400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda10146210 0x7fda100c6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:33.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.192+0000 7fda15400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fda100cdc00 con 0x7fda1006bbf0 2024-04-20T21:27:33.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.192+0000 7fda15400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fda100cda80 con 0x7fda10146210 2024-04-20T21:27:33.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.192+0000 7fda15400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fda100cdd80 con 0x7fda1006bfd0 2024-04-20T21:27:33.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.193+0000 7fda0e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda1006bbf0 0x7fda101f75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:33.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.193+0000 7fda0e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda1006bbf0 0x7fda101f75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60370/0 (socket says 172.21.15.86:60370) 2024-04-20T21:27:33.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.193+0000 7fda0e800640 1 -- 172.21.15.86:0/2089457602 learned_addr learned my addr 172.21.15.86:0/2089457602 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:33.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.193+0000 7fda0e800640 1 -- 172.21.15.86:0/2089457602 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda1006bfd0 msgr2=0x7fda100c5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:33.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.193+0000 7fda0e800640 1 --2- 172.21.15.86:0/2089457602 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda1006bfd0 0x7fda100c5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:33.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.193+0000 7fda0e800640 1 -- 172.21.15.86:0/2089457602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda10146210 msgr2=0x7fda100c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:33.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.193+0000 7fda0e800640 1 --2- 172.21.15.86:0/2089457602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda10146210 0x7fda100c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:33.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.193+0000 7fda0e800640 1 -- 172.21.15.86:0/2089457602 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda100ae4b0 con 0x7fda1006bbf0 2024-04-20T21:27:33.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.193+0000 7fda0e800640 1 --2- 172.21.15.86:0/2089457602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda1006bbf0 0x7fda101f75c0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fda0000c410 tx=0x7fda0000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:33.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.194+0000 7fda06a00640 1 -- 172.21.15.86:0/2089457602 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda00017070 con 0x7fda1006bbf0 2024-04-20T21:27:33.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.194+0000 7fda06a00640 1 -- 172.21.15.86:0/2089457602 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda000035f0 con 0x7fda1006bbf0 2024-04-20T21:27:33.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.194+0000 7fda06a00640 1 -- 172.21.15.86:0/2089457602 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda00003a20 con 0x7fda1006bbf0 2024-04-20T21:27:33.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.194+0000 7fda15400640 1 -- 172.21.15.86:0/2089457602 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda100ae790 con 0x7fda1006bbf0 2024-04-20T21:27:33.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.194+0000 7fda15400640 1 -- 172.21.15.86:0/2089457602 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda100aed50 con 0x7fda1006bbf0 2024-04-20T21:27:33.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.195+0000 7fda15400640 1 -- 172.21.15.86:0/2089457602 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda100af130 con 0x7fda1006bbf0 2024-04-20T21:27:33.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.196+0000 7fda06a00640 1 -- 172.21.15.86:0/2089457602 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fda00003bc0 con 0x7fda1006bbf0 2024-04-20T21:27:33.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.197+0000 7fda06a00640 1 --2- 172.21.15.86:0/2089457602 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd9ec06c8f0 0x7fd9ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:33.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.197+0000 7fda06a00640 1 -- 172.21.15.86:0/2089457602 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5996+0+0 (secure 0 0 0) 0x7fda0008aff0 con 0x7fda1006bbf0 2024-04-20T21:27:33.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.197+0000 7fda07e00640 1 --2- 172.21.15.86:0/2089457602 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd9ec06c8f0 0x7fd9ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:33.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.198+0000 7fda07e00640 1 --2- 172.21.15.86:0/2089457602 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd9ec06c8f0 0x7fd9ec06edb0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8036000 tx=0x7fd9f8030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:33.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.201+0000 7fda06a00640 1 -- 172.21.15.86:0/2089457602 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fda100af130 con 0x7fda1006bbf0 2024-04-20T21:27:33.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.445+0000 7fda15400640 1 -- 172.21.15.86:0/2089457602 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fda100689e0 con 0x7fda1006bbf0 2024-04-20T21:27:33.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.970+0000 7fda06a00640 1 -- 172.21.15.86:0/2089457602 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v75) ==== 104+0+0 (secure 0 0 0) 0x7fda000592f0 con 0x7fda1006bbf0 2024-04-20T21:27:33.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.973+0000 7fda15400640 1 -- 172.21.15.86:0/2089457602 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd9ec06c8f0 msgr2=0x7fd9ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:33.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.973+0000 7fda15400640 1 --2- 172.21.15.86:0/2089457602 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd9ec06c8f0 0x7fd9ec06edb0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8036000 tx=0x7fd9f8030000 comp rx=0 tx=0).stop 2024-04-20T21:27:33.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.974+0000 7fda15400640 1 -- 172.21.15.86:0/2089457602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda1006bbf0 msgr2=0x7fda101f75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:33.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.974+0000 7fda15400640 1 --2- 172.21.15.86:0/2089457602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda1006bbf0 0x7fda101f75c0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fda0000c410 tx=0x7fda0000c8d0 comp rx=0 tx=0).stop 2024-04-20T21:27:33.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.974+0000 7fda15400640 1 -- 172.21.15.86:0/2089457602 shutdown_connections 2024-04-20T21:27:33.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.974+0000 7fda15400640 1 --2- 172.21.15.86:0/2089457602 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd9ec06c8f0 0x7fd9ec06edb0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:33.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.974+0000 7fda15400640 1 --2- 172.21.15.86:0/2089457602 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fda10146210 0x7fda100c6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:33.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.974+0000 7fda15400640 1 --2- 172.21.15.86:0/2089457602 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fda1006bfd0 0x7fda100c5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:33.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.974+0000 7fda15400640 1 --2- 172.21.15.86:0/2089457602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda1006bbf0 0x7fda101f75c0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:33.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.974+0000 7fda15400640 1 -- 172.21.15.86:0/2089457602 >> 172.21.15.86:0/2089457602 conn(0x7fda100b2110 msgr2=0x7fda10071f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:33.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.974+0000 7fda15400640 1 -- 172.21.15.86:0/2089457602 shutdown_connections 2024-04-20T21:27:33.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:33.974+0000 7fda15400640 1 -- 172.21.15.86:0/2089457602 wait complete. 2024-04-20T21:27:33.996 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-20T21:27:34.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.206+0000 7f9b65c00640 1 Processor -- start 2024-04-20T21:27:34.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.206+0000 7f9b65c00640 1 -- start start 2024-04-20T21:27:34.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.208+0000 7f9b65c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b60071d00 0x7f9b6006f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:34.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.208+0000 7f9b65c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9b6015cc70 0x7f9b6015d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:34.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.208+0000 7f9b65c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b6006e010 con 0x7f9b600720e0 2024-04-20T21:27:34.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.208+0000 7f9b65c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9b6006f5f0 con 0x7f9b6015cc70 2024-04-20T21:27:34.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.208+0000 7f9b65c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9b6006e2f0 con 0x7f9b60071d00 2024-04-20T21:27:34.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.208+0000 7f9b5f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9b6015cc70 0x7f9b6015d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:34.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.208+0000 7f9b5f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9b6015cc70 0x7f9b6015d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60380/0 (socket says 172.21.15.86:60380) 2024-04-20T21:27:34.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.208+0000 7f9b5f400640 1 -- 172.21.15.86:0/4193447405 learned_addr learned my addr 172.21.15.86:0/4193447405 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:34.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.209+0000 7f9b5ea00640 1 --2- 172.21.15.86:0/4193447405 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b60071d00 0x7f9b6006f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:34.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.209+0000 7f9b5f400640 1 -- 172.21.15.86:0/4193447405 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b60071d00 msgr2=0x7f9b6006f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:34.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.209+0000 7f9b5f400640 1 --2- 172.21.15.86:0/4193447405 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b60071d00 0x7f9b6006f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:34.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.209+0000 7f9b5f400640 1 -- 172.21.15.86:0/4193447405 >> v1:172.21.15.144:6789/0 conn(0x7f9b600720e0 legacy=0x7f9b60145a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:34.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.209+0000 7f9b5f400640 1 -- 172.21.15.86:0/4193447405 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b6006e5d0 con 0x7f9b6015cc70 2024-04-20T21:27:34.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.209+0000 7f9b5ea00640 1 --2- 172.21.15.86:0/4193447405 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b60071d00 0x7f9b6006f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:27:34.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.210+0000 7f9b5f400640 1 --2- 172.21.15.86:0/4193447405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b6015cc70 0x7f9b6015d100 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f9b58009e50 tx=0x7f9b5802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f32bfeb94e1459b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:34.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.210+0000 7f9b5e000640 1 -- 172.21.15.86:0/4193447405 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b5802ca80 con 0x7f9b6015cc70 2024-04-20T21:27:34.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.210+0000 7f9b5e000640 1 -- 172.21.15.86:0/4193447405 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b5802cc20 con 0x7f9b6015cc70 2024-04-20T21:27:34.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.210+0000 7f9b5e000640 1 -- 172.21.15.86:0/4193447405 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b580343f0 con 0x7f9b6015cc70 2024-04-20T21:27:34.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.211+0000 7f9b65c00640 1 -- 172.21.15.86:0/4193447405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b6015cc70 msgr2=0x7f9b6015d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:34.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.211+0000 7f9b65c00640 1 --2- 172.21.15.86:0/4193447405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b6015cc70 0x7f9b6015d100 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f9b58009e50 tx=0x7f9b5802bee0 comp rx=0 tx=0).stop 2024-04-20T21:27:34.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.211+0000 7f9b65c00640 1 -- 172.21.15.86:0/4193447405 shutdown_connections 2024-04-20T21:27:34.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.211+0000 7f9b65c00640 1 --2- 172.21.15.86:0/4193447405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b6015cc70 0x7f9b6015d100 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:34.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.211+0000 7f9b65c00640 1 --2- 172.21.15.86:0/4193447405 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b60071d00 0x7f9b6006f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:34.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.211+0000 7f9b65c00640 1 -- 172.21.15.86:0/4193447405 >> 172.21.15.86:0/4193447405 conn(0x7f9b600b2180 msgr2=0x7f9b600b2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:34.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.211+0000 7f9b65c00640 1 -- 172.21.15.86:0/4193447405 shutdown_connections 2024-04-20T21:27:34.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.211+0000 7f9b65c00640 1 -- 172.21.15.86:0/4193447405 wait complete. 2024-04-20T21:27:34.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.213+0000 7f9b65c00640 1 Processor -- start 2024-04-20T21:27:34.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.214+0000 7f9b65c00640 1 -- start start 2024-04-20T21:27:34.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.215+0000 7f9b65c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b60071d00 0x7f9b601f1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:34.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.215+0000 7f9b65c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b600720e0 0x7f9b601e5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:34.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.215+0000 7f9b65c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b6015cc70 0x7f9b601e5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:34.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.215+0000 7f9b65c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9b601461a0 con 0x7f9b60071d00 2024-04-20T21:27:34.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.215+0000 7f9b65c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6006eb90 con 0x7f9b6015cc70 2024-04-20T21:27:34.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.215+0000 7f9b65c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9b60072540 con 0x7f9b600720e0 2024-04-20T21:27:34.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.215+0000 7f9b5ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b60071d00 0x7f9b601f1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:34.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.215+0000 7f9b5ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b60071d00 0x7f9b601f1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60396/0 (socket says 172.21.15.86:60396) 2024-04-20T21:27:34.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.215+0000 7f9b5ea00640 1 -- 172.21.15.86:0/4150961421 learned_addr learned my addr 172.21.15.86:0/4150961421 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:34.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.216+0000 7f9b5ea00640 1 -- 172.21.15.86:0/4150961421 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b600720e0 msgr2=0x7f9b601e5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:34.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.216+0000 7f9b5ea00640 1 --2- 172.21.15.86:0/4150961421 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b600720e0 0x7f9b601e5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:34.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.216+0000 7f9b5ea00640 1 -- 172.21.15.86:0/4150961421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b6015cc70 msgr2=0x7f9b601e5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:34.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.216+0000 7f9b5ea00640 1 --2- 172.21.15.86:0/4150961421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b6015cc70 0x7f9b601e5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:34.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.216+0000 7f9b5ea00640 1 -- 172.21.15.86:0/4150961421 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b601f1fc0 con 0x7f9b60071d00 2024-04-20T21:27:34.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.216+0000 7f9b5ea00640 1 --2- 172.21.15.86:0/4150961421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b60071d00 0x7f9b601f1a80 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f9b5000c820 tx=0x7f9b5000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:34.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.216+0000 7f9b57400640 1 -- 172.21.15.86:0/4150961421 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b50015070 con 0x7f9b60071d00 2024-04-20T21:27:34.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.216+0000 7f9b57400640 1 -- 172.21.15.86:0/4150961421 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b50007a30 con 0x7f9b60071d00 2024-04-20T21:27:34.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.216+0000 7f9b57400640 1 -- 172.21.15.86:0/4150961421 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b50007de0 con 0x7f9b60071d00 2024-04-20T21:27:34.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.216+0000 7f9b65c00640 1 -- 172.21.15.86:0/4150961421 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b600b4240 con 0x7f9b60071d00 2024-04-20T21:27:34.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.216+0000 7f9b65c00640 1 -- 172.21.15.86:0/4150961421 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b600b4800 con 0x7f9b60071d00 2024-04-20T21:27:34.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.218+0000 7f9b65c00640 1 -- 172.21.15.86:0/4150961421 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b600689e0 con 0x7f9b60071d00 2024-04-20T21:27:34.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.218+0000 7f9b57400640 1 -- 172.21.15.86:0/4150961421 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9b50018070 con 0x7f9b60071d00 2024-04-20T21:27:34.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.219+0000 7f9b57400640 1 --2- 172.21.15.86:0/4150961421 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9b3c06c920 0x7f9b3c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:34.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.219+0000 7f9b57400640 1 -- 172.21.15.86:0/4150961421 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5996+0+0 (secure 0 0 0) 0x7f9b5008bb90 con 0x7f9b60071d00 2024-04-20T21:27:34.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.220+0000 7f9b56000640 1 --2- 172.21.15.86:0/4150961421 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9b3c06c920 0x7f9b3c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:34.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.220+0000 7f9b56000640 1 --2- 172.21.15.86:0/4150961421 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9b3c06c920 0x7f9b3c06ede0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f9b6013f400 tx=0x7f9b48006bb0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:34.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.224+0000 7f9b57400640 1 -- 172.21.15.86:0/4150961421 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9b50059e00 con 0x7f9b60071d00 2024-04-20T21:27:34.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:34.422+0000 7f9b65c00640 1 -- 172.21.15.86:0/4150961421 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9b6013f430 con 0x7f9b60071d00 2024-04-20T21:27:35.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.124+0000 7f9b57400640 1 -- 172.21.15.86:0/4150961421 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v91) ==== 181+0+0 (secure 0 0 0) 0x7f9b50059700 con 0x7f9b60071d00 2024-04-20T21:27:35.124 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' removed 2024-04-20T21:27:35.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.128+0000 7f9b65c00640 1 -- 172.21.15.86:0/4150961421 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9b3c06c920 msgr2=0x7f9b3c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:35.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.128+0000 7f9b65c00640 1 --2- 172.21.15.86:0/4150961421 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9b3c06c920 0x7f9b3c06ede0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f9b6013f400 tx=0x7f9b48006bb0 comp rx=0 tx=0).stop 2024-04-20T21:27:35.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.128+0000 7f9b65c00640 1 -- 172.21.15.86:0/4150961421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b60071d00 msgr2=0x7f9b601f1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:35.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.128+0000 7f9b65c00640 1 --2- 172.21.15.86:0/4150961421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b60071d00 0x7f9b601f1a80 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f9b5000c820 tx=0x7f9b5000cce0 comp rx=0 tx=0).stop 2024-04-20T21:27:35.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.128+0000 7f9b65c00640 1 -- 172.21.15.86:0/4150961421 shutdown_connections 2024-04-20T21:27:35.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.128+0000 7f9b65c00640 1 --2- 172.21.15.86:0/4150961421 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9b3c06c920 0x7f9b3c06ede0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:35.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.128+0000 7f9b65c00640 1 --2- 172.21.15.86:0/4150961421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b6015cc70 0x7f9b601e5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:35.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.128+0000 7f9b65c00640 1 --2- 172.21.15.86:0/4150961421 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9b600720e0 0x7f9b601e5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:35.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.128+0000 7f9b65c00640 1 --2- 172.21.15.86:0/4150961421 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b60071d00 0x7f9b601f1a80 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:35.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.128+0000 7f9b65c00640 1 -- 172.21.15.86:0/4150961421 >> 172.21.15.86:0/4150961421 conn(0x7f9b600b2180 msgr2=0x7f9b600b0080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:35.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.128+0000 7f9b65c00640 1 -- 172.21.15.86:0/4150961421 shutdown_connections 2024-04-20T21:27:35.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.128+0000 7f9b65c00640 1 -- 172.21.15.86:0/4150961421 wait complete. 2024-04-20T21:27:35.150 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-20T21:27:35.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.356+0000 7f0946c00640 1 Processor -- start 2024-04-20T21:27:35.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.357+0000 7f0946c00640 1 -- start start 2024-04-20T21:27:35.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.358+0000 7f0946c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f09400720b0 0x7f0940162cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:35.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.358+0000 7f0946c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0940072490 0x7f094015ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:35.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.358+0000 7f0946c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f094015f2d0 0x7f09400d0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:35.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.358+0000 7f0946c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f094015fd80 con 0x7f09400720b0 2024-04-20T21:27:35.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.358+0000 7f0946c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f094015f7c0 con 0x7f094015f2d0 2024-04-20T21:27:35.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.358+0000 7f0946c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f094015faa0 con 0x7f0940072490 2024-04-20T21:27:35.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.358+0000 7f0945800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f09400720b0 0x7f0940162cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:35.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.358+0000 7f0945800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f09400720b0 0x7f0940162cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60412/0 (socket says 172.21.15.86:60412) 2024-04-20T21:27:35.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.358+0000 7f0945800640 1 -- 172.21.15.86:0/3997817315 learned_addr learned my addr 172.21.15.86:0/3997817315 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:35.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.359+0000 7f0946200640 1 --2- 172.21.15.86:0/3997817315 >> v2:172.21.15.144:3300/0 conn(0x7f094015f2d0 0x7f09400d0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:35.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.359+0000 7f0945800640 1 -- 172.21.15.86:0/3997817315 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0940072490 msgr2=0x7f094015ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:35.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.359+0000 7f0945800640 1 --2- 172.21.15.86:0/3997817315 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0940072490 0x7f094015ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:35.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.359+0000 7f0945800640 1 -- 172.21.15.86:0/3997817315 >> v2:172.21.15.144:3300/0 conn(0x7f094015f2d0 msgr2=0x7f09400d0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:35.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.359+0000 7f0945800640 1 --2- 172.21.15.86:0/3997817315 >> v2:172.21.15.144:3300/0 conn(0x7f094015f2d0 0x7f09400d0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:35.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.359+0000 7f0945800640 1 -- 172.21.15.86:0/3997817315 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09401624b0 con 0x7f09400720b0 2024-04-20T21:27:35.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.359+0000 7f0946200640 1 --2- 172.21.15.86:0/3997817315 >> v2:172.21.15.144:3300/0 conn(0x7f094015f2d0 0x7f09400d0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:27:35.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.360+0000 7f0945800640 1 --2- 172.21.15.86:0/3997817315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09400720b0 0x7f0940162cb0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f094015c960 tx=0x7f093402b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9f5a61cb5064b72 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:35.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.360+0000 7f093fe00640 1 -- 172.21.15.86:0/3997817315 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09340317d0 con 0x7f09400720b0 2024-04-20T21:27:35.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.360+0000 7f093fe00640 1 -- 172.21.15.86:0/3997817315 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0934037070 con 0x7f09400720b0 2024-04-20T21:27:35.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.360+0000 7f093fe00640 1 -- 172.21.15.86:0/3997817315 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0934031b90 con 0x7f09400720b0 2024-04-20T21:27:35.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.361+0000 7f0946c00640 1 -- 172.21.15.86:0/3997817315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09400720b0 msgr2=0x7f0940162cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:35.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.361+0000 7f0946c00640 1 --2- 172.21.15.86:0/3997817315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09400720b0 0x7f0940162cb0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f094015c960 tx=0x7f093402b870 comp rx=0 tx=0).stop 2024-04-20T21:27:35.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.361+0000 7f0946c00640 1 -- 172.21.15.86:0/3997817315 shutdown_connections 2024-04-20T21:27:35.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.362+0000 7f0946c00640 1 --2- 172.21.15.86:0/3997817315 >> v2:172.21.15.144:3300/0 conn(0x7f094015f2d0 0x7f09400d0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:35.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.362+0000 7f0946c00640 1 --2- 172.21.15.86:0/3997817315 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0940072490 0x7f094015ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:35.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.362+0000 7f0946c00640 1 --2- 172.21.15.86:0/3997817315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09400720b0 0x7f0940162cb0 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:35.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.362+0000 7f0946c00640 1 -- 172.21.15.86:0/3997817315 >> 172.21.15.86:0/3997817315 conn(0x7f09400b2180 msgr2=0x7f09400b2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:35.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.362+0000 7f0946c00640 1 -- 172.21.15.86:0/3997817315 shutdown_connections 2024-04-20T21:27:35.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.362+0000 7f0946c00640 1 -- 172.21.15.86:0/3997817315 wait complete. 2024-04-20T21:27:35.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.364+0000 7f0946c00640 1 Processor -- start 2024-04-20T21:27:35.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.364+0000 7f0946c00640 1 -- start start 2024-04-20T21:27:35.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.366+0000 7f0946c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09400720b0 0x7f094015be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:35.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.366+0000 7f0946c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0940072490 0x7f09401f9a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:35.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.366+0000 7f0946c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f094015f2d0 0x7f09400ae140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:35.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.366+0000 7f0946c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f09400b5090 con 0x7f094015f2d0 2024-04-20T21:27:35.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.366+0000 7f0946c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f09401631f0 con 0x7f0940072490 2024-04-20T21:27:35.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.366+0000 7f0946c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f09400b5210 con 0x7f09400720b0 2024-04-20T21:27:35.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.366+0000 7f0946200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f094015f2d0 0x7f09400ae140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:35.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.366+0000 7f0946200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f094015f2d0 0x7f09400ae140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60422/0 (socket says 172.21.15.86:60422) 2024-04-20T21:27:35.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.366+0000 7f0946200640 1 -- 172.21.15.86:0/141302498 learned_addr learned my addr 172.21.15.86:0/141302498 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:35.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.367+0000 7f0944e00640 1 --2- 172.21.15.86:0/141302498 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0940072490 0x7f09401f9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:35.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.367+0000 7f0945800640 1 --2- 172.21.15.86:0/141302498 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09400720b0 0x7f094015be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:35.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.367+0000 7f0946200640 1 -- 172.21.15.86:0/141302498 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09400720b0 msgr2=0x7f094015be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:35.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.367+0000 7f0946200640 1 --2- 172.21.15.86:0/141302498 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09400720b0 0x7f094015be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:35.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.367+0000 7f0946200640 1 -- 172.21.15.86:0/141302498 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0940072490 msgr2=0x7f09401f9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:35.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.367+0000 7f0946200640 1 --2- 172.21.15.86:0/141302498 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0940072490 0x7f09401f9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:35.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.367+0000 7f0946200640 1 -- 172.21.15.86:0/141302498 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09400ad0e0 con 0x7f094015f2d0 2024-04-20T21:27:35.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.368+0000 7f0946200640 1 --2- 172.21.15.86:0/141302498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f094015f2d0 0x7f09400ae140 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f093800ced0 tx=0x7f093800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:35.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.368+0000 7f093d600640 1 -- 172.21.15.86:0/141302498 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0938003b90 con 0x7f094015f2d0 2024-04-20T21:27:35.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.368+0000 7f093d600640 1 -- 172.21.15.86:0/141302498 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0938003d50 con 0x7f094015f2d0 2024-04-20T21:27:35.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.368+0000 7f093d600640 1 -- 172.21.15.86:0/141302498 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0938004260 con 0x7f094015f2d0 2024-04-20T21:27:35.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.368+0000 7f0946c00640 1 -- 172.21.15.86:0/141302498 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f094015c350 con 0x7f094015f2d0 2024-04-20T21:27:35.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.368+0000 7f0946c00640 1 -- 172.21.15.86:0/141302498 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09400ad660 con 0x7f094015f2d0 2024-04-20T21:27:35.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.370+0000 7f0946c00640 1 -- 172.21.15.86:0/141302498 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f094004fcf0 con 0x7f094015f2d0 2024-04-20T21:27:35.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.370+0000 7f093d600640 1 -- 172.21.15.86:0/141302498 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0938004400 con 0x7f094015f2d0 2024-04-20T21:27:35.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.371+0000 7f093d600640 1 --2- 172.21.15.86:0/141302498 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f092006c8f0 0x7f092006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:35.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.371+0000 7f093d600640 1 -- 172.21.15.86:0/141302498 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5558+0+0 (secure 0 0 0) 0x7f093808c230 con 0x7f094015f2d0 2024-04-20T21:27:35.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.372+0000 7f0945800640 1 --2- 172.21.15.86:0/141302498 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f092006c8f0 0x7f092006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:35.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.373+0000 7f0945800640 1 --2- 172.21.15.86:0/141302498 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f092006c8f0 0x7f092006edb0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0934036000 tx=0x7f093403d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:35.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.376+0000 7f093d600640 1 -- 172.21.15.86:0/141302498 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f093805a130 con 0x7f094015f2d0 2024-04-20T21:27:35.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:35.576+0000 7f0946c00640 1 -- 172.21.15.86:0/141302498 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f094006f890 con 0x7f094015f2d0 2024-04-20T21:27:36.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.136+0000 7f093d600640 1 -- 172.21.15.86:0/141302498 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v92) ==== 169+0+0 (secure 0 0 0) 0x7f093805e430 con 0x7f094015f2d0 2024-04-20T21:27:36.135 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' removed 2024-04-20T21:27:36.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.140+0000 7f0946c00640 1 -- 172.21.15.86:0/141302498 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f092006c8f0 msgr2=0x7f092006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:36.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.140+0000 7f0946c00640 1 --2- 172.21.15.86:0/141302498 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f092006c8f0 0x7f092006edb0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0934036000 tx=0x7f093403d040 comp rx=0 tx=0).stop 2024-04-20T21:27:36.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.140+0000 7f0946c00640 1 -- 172.21.15.86:0/141302498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f094015f2d0 msgr2=0x7f09400ae140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:36.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.140+0000 7f0946c00640 1 --2- 172.21.15.86:0/141302498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f094015f2d0 0x7f09400ae140 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f093800ced0 tx=0x7f093800a710 comp rx=0 tx=0).stop 2024-04-20T21:27:36.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.140+0000 7f0946c00640 1 -- 172.21.15.86:0/141302498 shutdown_connections 2024-04-20T21:27:36.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.140+0000 7f0946c00640 1 --2- 172.21.15.86:0/141302498 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f092006c8f0 0x7f092006edb0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:36.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.140+0000 7f0946c00640 1 --2- 172.21.15.86:0/141302498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f094015f2d0 0x7f09400ae140 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:36.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.140+0000 7f0946c00640 1 --2- 172.21.15.86:0/141302498 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0940072490 0x7f09401f9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:36.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.140+0000 7f0946c00640 1 --2- 172.21.15.86:0/141302498 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f09400720b0 0x7f094015be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:36.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.140+0000 7f0946c00640 1 -- 172.21.15.86:0/141302498 >> 172.21.15.86:0/141302498 conn(0x7f09400b2180 msgr2=0x7f09400afbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:36.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.142+0000 7f0946c00640 1 -- 172.21.15.86:0/141302498 shutdown_connections 2024-04-20T21:27:36.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.142+0000 7f0946c00640 1 -- 172.21.15.86:0/141302498 wait complete. 2024-04-20T21:27:36.163 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-20T21:27:36.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.361+0000 7fd4dd600640 1 Processor -- start 2024-04-20T21:27:36.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.361+0000 7fd4dd600640 1 -- start start 2024-04-20T21:27:36.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.362+0000 7fd4dd600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd4d815ea80 0x7fd4d80721b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:36.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.362+0000 7fd4dd600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd4d80726f0 0x7fd4d80b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:36.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.362+0000 7fd4dd600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4d80b3800 con 0x7fd4d80b5320 2024-04-20T21:27:36.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.362+0000 7fd4dd600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd4d80b4410 con 0x7fd4d815ea80 2024-04-20T21:27:36.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.362+0000 7fd4dd600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fd4d80b3ae0 con 0x7fd4d80726f0 2024-04-20T21:27:36.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.363+0000 7fd4d6a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd4d815ea80 0x7fd4d80721b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:36.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.363+0000 7fd4d7400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd4d80b5320 0x7fd4d8162450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54222/0 (socket says 172.21.15.86:54222) 2024-04-20T21:27:36.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.363+0000 7fd4d7400640 1 -- 172.21.15.86:0/1129296353 learned_addr learned my addr 172.21.15.86:0/1129296353 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:27:36.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.363+0000 7fd4d6000640 1 --2- 172.21.15.86:0/1129296353 >> v2:172.21.15.144:3300/0 conn(0x7fd4d80726f0 0x7fd4d80b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:36.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.363+0000 7fd4d5600640 1 -- 172.21.15.86:0/1129296353 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 220026093 0 0) 0x7fd4d80b3800 con 0x7fd4d80b5320 2024-04-20T21:27:36.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.364+0000 7fd4d5600640 1 -- 172.21.15.86:0/1129296353 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd4bc003520 con 0x7fd4d80b5320 2024-04-20T21:27:36.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.364+0000 7fd4d6a00640 1 -- 172.21.15.86:0/1129296353 >> v2:172.21.15.144:3300/0 conn(0x7fd4d80726f0 msgr2=0x7fd4d80b4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:36.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.364+0000 7fd4d6a00640 1 --2- 172.21.15.86:0/1129296353 >> v2:172.21.15.144:3300/0 conn(0x7fd4d80726f0 0x7fd4d80b4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:36.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.364+0000 7fd4d6a00640 1 -- 172.21.15.86:0/1129296353 >> v1:172.21.15.86:6789/0 conn(0x7fd4d80b5320 legacy=0x7fd4d8162450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:36.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.364+0000 7fd4d6a00640 1 -- 172.21.15.86:0/1129296353 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d80b3dc0 con 0x7fd4d815ea80 2024-04-20T21:27:36.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.364+0000 7fd4d6000640 1 --2- 172.21.15.86:0/1129296353 >> v2:172.21.15.144:3300/0 conn(0x7fd4d80726f0 0x7fd4d80b4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:27:36.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.365+0000 7fd4d6a00640 1 --2- 172.21.15.86:0/1129296353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4d815ea80 0x7fd4d80721b0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fd4d80d07e0 tx=0x7fd4d002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7920396500b10aaf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:36.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.365+0000 7fd4d5600640 1 -- 172.21.15.86:0/1129296353 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d0035020 con 0x7fd4d815ea80 2024-04-20T21:27:36.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.365+0000 7fd4d5600640 1 -- 172.21.15.86:0/1129296353 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4d0031830 con 0x7fd4d815ea80 2024-04-20T21:27:36.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.365+0000 7fd4d5600640 1 -- 172.21.15.86:0/1129296353 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d0031c00 con 0x7fd4d815ea80 2024-04-20T21:27:36.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.365+0000 7fd4dd600640 1 -- 172.21.15.86:0/1129296353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4d815ea80 msgr2=0x7fd4d80721b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:36.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.365+0000 7fd4dd600640 1 --2- 172.21.15.86:0/1129296353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4d815ea80 0x7fd4d80721b0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fd4d80d07e0 tx=0x7fd4d002b8d0 comp rx=0 tx=0).stop 2024-04-20T21:27:36.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.366+0000 7fd4dd600640 1 -- 172.21.15.86:0/1129296353 shutdown_connections 2024-04-20T21:27:36.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.366+0000 7fd4dd600640 1 --2- 172.21.15.86:0/1129296353 >> v2:172.21.15.144:3300/0 conn(0x7fd4d80726f0 0x7fd4d80b4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:36.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.366+0000 7fd4dd600640 1 --2- 172.21.15.86:0/1129296353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4d815ea80 0x7fd4d80721b0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:36.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.366+0000 7fd4dd600640 1 -- 172.21.15.86:0/1129296353 >> 172.21.15.86:0/1129296353 conn(0x7fd4d80b2500 msgr2=0x7fd4d80b28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:36.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.366+0000 7fd4dd600640 1 -- 172.21.15.86:0/1129296353 shutdown_connections 2024-04-20T21:27:36.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.366+0000 7fd4dd600640 1 -- 172.21.15.86:0/1129296353 wait complete. 2024-04-20T21:27:36.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.367+0000 7fd4dd600640 1 Processor -- start 2024-04-20T21:27:36.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.367+0000 7fd4dd600640 1 -- start start 2024-04-20T21:27:36.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.368+0000 7fd4dd600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4d80726f0 0x7fd4d8138290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:36.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.368+0000 7fd4dd600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4d80b5320 0x7fd4d8150990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:36.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.368+0000 7fd4dd600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4d815ea80 0x7fd4d8150ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:36.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.368+0000 7fd4dd600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d80af180 con 0x7fd4d80726f0 2024-04-20T21:27:36.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.368+0000 7fd4dd600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d80b48a0 con 0x7fd4d80b5320 2024-04-20T21:27:36.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.368+0000 7fd4dd600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd4d806f570 con 0x7fd4d815ea80 2024-04-20T21:27:36.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.369+0000 7fd4d6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4d80726f0 0x7fd4d8138290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:36.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.369+0000 7fd4d6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4d80726f0 0x7fd4d8138290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53054/0 (socket says 172.21.15.86:53054) 2024-04-20T21:27:36.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.369+0000 7fd4d6a00640 1 -- 172.21.15.86:0/23288250 learned_addr learned my addr 172.21.15.86:0/23288250 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:36.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.369+0000 7fd4d6a00640 1 -- 172.21.15.86:0/23288250 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4d815ea80 msgr2=0x7fd4d8150ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:36.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.369+0000 7fd4d6a00640 1 --2- 172.21.15.86:0/23288250 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4d815ea80 0x7fd4d8150ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:36.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.369+0000 7fd4d6a00640 1 -- 172.21.15.86:0/23288250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4d80b5320 msgr2=0x7fd4d8150990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:36.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.369+0000 7fd4d6a00640 1 --2- 172.21.15.86:0/23288250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4d80b5320 0x7fd4d8150990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:36.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.369+0000 7fd4d6a00640 1 -- 172.21.15.86:0/23288250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d8152cd0 con 0x7fd4d80726f0 2024-04-20T21:27:36.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.369+0000 7fd4d6a00640 1 --2- 172.21.15.86:0/23288250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4d80726f0 0x7fd4d8138290 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0009890 tx=0x7fd4d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:36.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.370+0000 7fd4cea00640 1 -- 172.21.15.86:0/23288250 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d0040070 con 0x7fd4d80726f0 2024-04-20T21:27:36.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.370+0000 7fd4cea00640 1 -- 172.21.15.86:0/23288250 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4d003e070 con 0x7fd4d80726f0 2024-04-20T21:27:36.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.370+0000 7fd4cea00640 1 -- 172.21.15.86:0/23288250 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d0035040 con 0x7fd4d80726f0 2024-04-20T21:27:36.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.370+0000 7fd4dd600640 1 -- 172.21.15.86:0/23288250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4d8152f70 con 0x7fd4d80726f0 2024-04-20T21:27:36.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.370+0000 7fd4dd600640 1 -- 172.21.15.86:0/23288250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4d8153210 con 0x7fd4d80726f0 2024-04-20T21:27:36.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.371+0000 7fd4dd600640 1 -- 172.21.15.86:0/23288250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4d80689e0 con 0x7fd4d80726f0 2024-04-20T21:27:36.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.372+0000 7fd4cea00640 1 -- 172.21.15.86:0/23288250 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd4d0038020 con 0x7fd4d80726f0 2024-04-20T21:27:36.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.373+0000 7fd4cea00640 1 --2- 172.21.15.86:0/23288250 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd4b806c8f0 0x7fd4b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:36.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.374+0000 7fd4cea00640 1 -- 172.21.15.86:0/23288250 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5115+0+0 (secure 0 0 0) 0x7fd4d00ad1e0 con 0x7fd4d80726f0 2024-04-20T21:27:36.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.374+0000 7fd4d6000640 1 --2- 172.21.15.86:0/23288250 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd4b806c8f0 0x7fd4b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:36.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.374+0000 7fd4d6000640 1 --2- 172.21.15.86:0/23288250 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd4b806c8f0 0x7fd4b806edb0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8142940 tx=0x7fd4c40094a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:36.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.378+0000 7fd4cea00640 1 -- 172.21.15.86:0/23288250 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd4d007b850 con 0x7fd4d80726f0 2024-04-20T21:27:36.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.577+0000 7fd4dd600640 1 -- 172.21.15.86:0/23288250 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fd4d80b4bc0 con 0x7fd4d80726f0 2024-04-20T21:27:36.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.584+0000 7fd4cea00640 1 -- 172.21.15.86:0/23288250 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7fd4d0031800 con 0x7fd4d80726f0 2024-04-20T21:27:36.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.588+0000 7fd4dd600640 1 -- 172.21.15.86:0/23288250 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd4b806c8f0 msgr2=0x7fd4b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:36.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.588+0000 7fd4dd600640 1 --2- 172.21.15.86:0/23288250 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd4b806c8f0 0x7fd4b806edb0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8142940 tx=0x7fd4c40094a0 comp rx=0 tx=0).stop 2024-04-20T21:27:36.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.588+0000 7fd4dd600640 1 -- 172.21.15.86:0/23288250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4d80726f0 msgr2=0x7fd4d8138290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:36.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.588+0000 7fd4dd600640 1 --2- 172.21.15.86:0/23288250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4d80726f0 0x7fd4d8138290 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0009890 tx=0x7fd4d00046c0 comp rx=0 tx=0).stop 2024-04-20T21:27:36.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.589+0000 7fd4dd600640 1 -- 172.21.15.86:0/23288250 shutdown_connections 2024-04-20T21:27:36.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.589+0000 7fd4dd600640 1 --2- 172.21.15.86:0/23288250 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd4b806c8f0 0x7fd4b806edb0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:36.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.589+0000 7fd4dd600640 1 --2- 172.21.15.86:0/23288250 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4d815ea80 0x7fd4d8150ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:36.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.589+0000 7fd4dd600640 1 --2- 172.21.15.86:0/23288250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4d80b5320 0x7fd4d8150990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:36.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.589+0000 7fd4dd600640 1 --2- 172.21.15.86:0/23288250 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4d80726f0 0x7fd4d8138290 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:36.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.589+0000 7fd4dd600640 1 -- 172.21.15.86:0/23288250 >> 172.21.15.86:0/23288250 conn(0x7fd4d80b2500 msgr2=0x7fd4d81590d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:36.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.589+0000 7fd4dd600640 1 -- 172.21.15.86:0/23288250 shutdown_connections 2024-04-20T21:27:36.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.589+0000 7fd4dd600640 1 -- 172.21.15.86:0/23288250 wait complete. 2024-04-20T21:27:36.606 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-04-20T21:27:36.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.803+0000 7f40c4e00640 1 Processor -- start 2024-04-20T21:27:36.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.803+0000 7f40c4e00640 1 -- start start 2024-04-20T21:27:36.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.804+0000 7f40c4e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f40c015ca60 0x7f40c0160400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:36.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.804+0000 7f40c4e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f40c0160940 0x7f40c0166730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:36.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.804+0000 7f40c4e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40c00d0160 con 0x7f40c015c680 2024-04-20T21:27:36.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.804+0000 7f40c4e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f40c00b3f00 con 0x7f40c0160940 2024-04-20T21:27:36.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.804+0000 7f40c4e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f40c0160d90 con 0x7f40c015ca60 2024-04-20T21:27:36.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.804+0000 7f40bde00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f40c015c680 0x7f40c00cdc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54228/0 (socket says 172.21.15.86:54228) 2024-04-20T21:27:36.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.804+0000 7f40bde00640 1 -- 172.21.15.86:0/2495126277 learned_addr learned my addr 172.21.15.86:0/2495126277 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:27:36.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.804+0000 7f40be800640 1 --2- 172.21.15.86:0/2495126277 >> v2:172.21.15.86:3300/0 conn(0x7f40c0160940 0x7f40c0166730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:36.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.805+0000 7f40bd400640 1 --2- 172.21.15.86:0/2495126277 >> v2:172.21.15.144:3300/0 conn(0x7f40c015ca60 0x7f40c0160400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:36.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.805+0000 7f40bca00640 1 -- 172.21.15.86:0/2495126277 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3221627266 0 0) 0x7f40c00d0160 con 0x7f40c015c680 2024-04-20T21:27:36.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.805+0000 7f40bca00640 1 -- 172.21.15.86:0/2495126277 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f40a4003520 con 0x7f40c015c680 2024-04-20T21:27:36.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.805+0000 7f40be800640 1 -- 172.21.15.86:0/2495126277 >> v2:172.21.15.144:3300/0 conn(0x7f40c015ca60 msgr2=0x7f40c0160400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:36.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.805+0000 7f40be800640 1 --2- 172.21.15.86:0/2495126277 >> v2:172.21.15.144:3300/0 conn(0x7f40c015ca60 0x7f40c0160400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:36.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.805+0000 7f40be800640 1 -- 172.21.15.86:0/2495126277 >> v1:172.21.15.86:6789/0 conn(0x7f40c015c680 legacy=0x7f40c00cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:36.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.805+0000 7f40be800640 1 -- 172.21.15.86:0/2495126277 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40c00d0440 con 0x7f40c0160940 2024-04-20T21:27:36.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.806+0000 7f40be800640 1 --2- 172.21.15.86:0/2495126277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40c0160940 0x7f40c0166730 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f40c00735b0 tx=0x7f40b802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3013c89e059e94a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:36.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.806+0000 7f40bca00640 1 -- 172.21.15.86:0/2495126277 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b8035020 con 0x7f40c0160940 2024-04-20T21:27:36.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.806+0000 7f40bca00640 1 -- 172.21.15.86:0/2495126277 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40b8031830 con 0x7f40c0160940 2024-04-20T21:27:36.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.807+0000 7f40bca00640 1 -- 172.21.15.86:0/2495126277 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b8031be0 con 0x7f40c0160940 2024-04-20T21:27:36.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.807+0000 7f40c4e00640 1 -- 172.21.15.86:0/2495126277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40c0160940 msgr2=0x7f40c0166730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:36.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.807+0000 7f40c4e00640 1 --2- 172.21.15.86:0/2495126277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40c0160940 0x7f40c0166730 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f40c00735b0 tx=0x7f40b802b8d0 comp rx=0 tx=0).stop 2024-04-20T21:27:36.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.807+0000 7f40c4e00640 1 -- 172.21.15.86:0/2495126277 shutdown_connections 2024-04-20T21:27:36.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.807+0000 7f40c4e00640 1 --2- 172.21.15.86:0/2495126277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40c0160940 0x7f40c0166730 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:36.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.807+0000 7f40c4e00640 1 --2- 172.21.15.86:0/2495126277 >> v2:172.21.15.144:3300/0 conn(0x7f40c015ca60 0x7f40c0160400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:36.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.807+0000 7f40c4e00640 1 -- 172.21.15.86:0/2495126277 >> 172.21.15.86:0/2495126277 conn(0x7f40c00b4080 msgr2=0x7f40c00b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:36.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.807+0000 7f40c4e00640 1 -- 172.21.15.86:0/2495126277 shutdown_connections 2024-04-20T21:27:36.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.807+0000 7f40c4e00640 1 -- 172.21.15.86:0/2495126277 wait complete. 2024-04-20T21:27:36.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.809+0000 7f40c4e00640 1 Processor -- start 2024-04-20T21:27:36.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.809+0000 7f40c4e00640 1 -- start start 2024-04-20T21:27:36.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.810+0000 7f40c4e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40c015c680 0x7f40c00cf520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:36.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.810+0000 7f40c4e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40c015ca60 0x7f40c00b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:36.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.810+0000 7f40c4e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40c0160940 0x7f40c00b7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:36.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.810+0000 7f40c4e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f40c00d0a00 con 0x7f40c015ca60 2024-04-20T21:27:36.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.810+0000 7f40c4e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f40c00d1050 con 0x7f40c0160940 2024-04-20T21:27:36.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.810+0000 7f40c4e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f40c00d0ce0 con 0x7f40c015c680 2024-04-20T21:27:36.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.810+0000 7f40bd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40c015ca60 0x7f40c00b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:36.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.810+0000 7f40bd400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40c015ca60 0x7f40c00b5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53066/0 (socket says 172.21.15.86:53066) 2024-04-20T21:27:36.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.810+0000 7f40bd400640 1 -- 172.21.15.86:0/4018809952 learned_addr learned my addr 172.21.15.86:0/4018809952 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:36.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.811+0000 7f40bde00640 1 --2- 172.21.15.86:0/4018809952 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40c015c680 0x7f40c00cf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:36.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.811+0000 7f40be800640 1 --2- 172.21.15.86:0/4018809952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40c0160940 0x7f40c00b7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:36.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.811+0000 7f40bd400640 1 -- 172.21.15.86:0/4018809952 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40c015c680 msgr2=0x7f40c00cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:36.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.811+0000 7f40bd400640 1 --2- 172.21.15.86:0/4018809952 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40c015c680 0x7f40c00cf520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:36.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.811+0000 7f40bd400640 1 -- 172.21.15.86:0/4018809952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40c0160940 msgr2=0x7f40c00b7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:36.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.811+0000 7f40bd400640 1 --2- 172.21.15.86:0/4018809952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40c0160940 0x7f40c00b7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:36.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.811+0000 7f40bd400640 1 -- 172.21.15.86:0/4018809952 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40c00ca810 con 0x7f40c015ca60 2024-04-20T21:27:36.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.811+0000 7f40be800640 1 --2- 172.21.15.86:0/4018809952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40c0160940 0x7f40c00b7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:27:36.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.811+0000 7f40bd400640 1 --2- 172.21.15.86:0/4018809952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40c015ca60 0x7f40c00b5bc0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f40a8007ef0 tx=0x7f40a800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:36.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.811+0000 7f40b6000640 1 -- 172.21.15.86:0/4018809952 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a8014620 con 0x7f40c015ca60 2024-04-20T21:27:36.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.811+0000 7f40b6000640 1 -- 172.21.15.86:0/4018809952 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40a8019070 con 0x7f40c015ca60 2024-04-20T21:27:36.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.811+0000 7f40b6000640 1 -- 172.21.15.86:0/4018809952 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a8014a00 con 0x7f40c015ca60 2024-04-20T21:27:36.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.811+0000 7f40c4e00640 1 -- 172.21.15.86:0/4018809952 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40c00caaf0 con 0x7f40c015ca60 2024-04-20T21:27:36.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.812+0000 7f40c4e00640 1 -- 172.21.15.86:0/4018809952 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40c00cb0b0 con 0x7f40c015ca60 2024-04-20T21:27:36.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.813+0000 7f40c4e00640 1 -- 172.21.15.86:0/4018809952 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40c00690f0 con 0x7f40c015ca60 2024-04-20T21:27:36.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.814+0000 7f40b6000640 1 -- 172.21.15.86:0/4018809952 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f40a8014ba0 con 0x7f40c015ca60 2024-04-20T21:27:36.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.815+0000 7f40b6000640 1 --2- 172.21.15.86:0/4018809952 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f409c06c920 0x7f409c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:36.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.815+0000 7f40b6000640 1 -- 172.21.15.86:0/4018809952 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5115+0+0 (secure 0 0 0) 0x7f40a808c250 con 0x7f40c015ca60 2024-04-20T21:27:36.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.815+0000 7f40bde00640 1 --2- 172.21.15.86:0/4018809952 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f409c06c920 0x7f409c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:36.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.816+0000 7f40bde00640 1 --2- 172.21.15.86:0/4018809952 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f409c06c920 0x7f409c06ede0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f40b0005880 tx=0x7f40b0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:36.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:36.819+0000 7f40b6000640 1 -- 172.21.15.86:0/4018809952 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f40a805a8c0 con 0x7f40c015ca60 2024-04-20T21:27:37.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.095+0000 7f40c4e00640 1 -- 172.21.15.86:0/4018809952 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) -- 0x7f40c015bdf0 con 0x7f40c015ca60 2024-04-20T21:27:37.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.139+0000 7f40b6000640 1 -- 172.21.15.86:0/4018809952 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v473) ==== 167+0+0 (secure 0 0 0) 0x7f40a805a1c0 con 0x7f40c015ca60 2024-04-20T21:27:37.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.142+0000 7f40c4e00640 1 -- 172.21.15.86:0/4018809952 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f409c06c920 msgr2=0x7f409c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:37.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.142+0000 7f40c4e00640 1 --2- 172.21.15.86:0/4018809952 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f409c06c920 0x7f409c06ede0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f40b0005880 tx=0x7f40b0009040 comp rx=0 tx=0).stop 2024-04-20T21:27:37.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.142+0000 7f40c4e00640 1 -- 172.21.15.86:0/4018809952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40c015ca60 msgr2=0x7f40c00b5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:37.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.142+0000 7f40c4e00640 1 --2- 172.21.15.86:0/4018809952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40c015ca60 0x7f40c00b5bc0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f40a8007ef0 tx=0x7f40a800e4b0 comp rx=0 tx=0).stop 2024-04-20T21:27:37.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.142+0000 7f40c4e00640 1 -- 172.21.15.86:0/4018809952 shutdown_connections 2024-04-20T21:27:37.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.142+0000 7f40c4e00640 1 --2- 172.21.15.86:0/4018809952 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f409c06c920 0x7f409c06ede0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:37.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.142+0000 7f40c4e00640 1 --2- 172.21.15.86:0/4018809952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40c0160940 0x7f40c00b7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:37.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.142+0000 7f40c4e00640 1 --2- 172.21.15.86:0/4018809952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40c015ca60 0x7f40c00b5bc0 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:37.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.142+0000 7f40c4e00640 1 --2- 172.21.15.86:0/4018809952 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f40c015c680 0x7f40c00cf520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:37.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.142+0000 7f40c4e00640 1 -- 172.21.15.86:0/4018809952 >> 172.21.15.86:0/4018809952 conn(0x7f40c00b4080 msgr2=0x7f40c015a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:37.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.142+0000 7f40c4e00640 1 -- 172.21.15.86:0/4018809952 shutdown_connections 2024-04-20T21:27:37.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.142+0000 7f40c4e00640 1 -- 172.21.15.86:0/4018809952 wait complete. 2024-04-20T21:27:37.160 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-04-20T21:27:37.160 INFO:tasks.cephfs_test_runner:Starting test: test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-20T21:27:37.161 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:27:37.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.357+0000 7f6adec00640 1 Processor -- start 2024-04-20T21:27:37.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.357+0000 7f6adec00640 1 -- start start 2024-04-20T21:27:37.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.359+0000 7f6adec00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6ad8144e90 0x7f6ad815ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:37.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.359+0000 7f6adec00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ad8071d00 0x7f6ad8072190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:37.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.359+0000 7f6adec00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ad806dc30 con 0x7f6ad806f140 2024-04-20T21:27:37.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.359+0000 7f6adec00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f6ad8145270 con 0x7f6ad8144e90 2024-04-20T21:27:37.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.359+0000 7f6adec00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6ad815f720 con 0x7f6ad8071d00 2024-04-20T21:27:37.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.359+0000 7f6ad7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6ad8144e90 0x7f6ad815ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:37.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.359+0000 7f6ad7400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f6ad806f140 0x7f6ad80aeb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54234/0 (socket says 172.21.15.86:54234) 2024-04-20T21:27:37.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.360+0000 7f6ad7400640 1 -- 172.21.15.86:0/2416929419 learned_addr learned my addr 172.21.15.86:0/2416929419 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:27:37.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.360+0000 7f6ad7e00640 1 -- 172.21.15.86:0/2416929419 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ad8071d00 msgr2=0x7f6ad8072190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:37.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.360+0000 7f6ad7e00640 1 --2- 172.21.15.86:0/2416929419 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ad8071d00 0x7f6ad8072190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:37.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.360+0000 7f6ad7e00640 1 -- 172.21.15.86:0/2416929419 >> v1:172.21.15.86:6789/0 conn(0x7f6ad806f140 legacy=0x7f6ad80aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:37.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.360+0000 7f6ad7e00640 1 -- 172.21.15.86:0/2416929419 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ad815fa00 con 0x7f6ad8144e90 2024-04-20T21:27:37.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.360+0000 7f6ad6a00640 1 -- 172.21.15.86:0/2416929419 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1697403734 0 0) 0x7f6ad806dc30 con 0x7f6ad806f140 2024-04-20T21:27:37.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.361+0000 7f6ad7e00640 1 --2- 172.21.15.86:0/2416929419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ad8144e90 0x7f6ad815ee10 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f6ad81453f0 tx=0x7f6ac802b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eef375a1a7a1384 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:37.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.361+0000 7f6ad6a00640 1 -- 172.21.15.86:0/2416929419 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac8035020 con 0x7f6ad8144e90 2024-04-20T21:27:37.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.361+0000 7f6ad6a00640 1 -- 172.21.15.86:0/2416929419 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ac8031800 con 0x7f6ad8144e90 2024-04-20T21:27:37.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.361+0000 7f6ad6a00640 1 -- 172.21.15.86:0/2416929419 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac8031bd0 con 0x7f6ad8144e90 2024-04-20T21:27:37.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.362+0000 7f6adec00640 1 -- 172.21.15.86:0/2416929419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ad8144e90 msgr2=0x7f6ad815ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:37.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.362+0000 7f6adec00640 1 --2- 172.21.15.86:0/2416929419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ad8144e90 0x7f6ad815ee10 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f6ad81453f0 tx=0x7f6ac802b8a0 comp rx=0 tx=0).stop 2024-04-20T21:27:37.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.362+0000 7f6adec00640 1 -- 172.21.15.86:0/2416929419 shutdown_connections 2024-04-20T21:27:37.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.362+0000 7f6adec00640 1 --2- 172.21.15.86:0/2416929419 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ad8071d00 0x7f6ad8072190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:37.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.362+0000 7f6adec00640 1 --2- 172.21.15.86:0/2416929419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ad8144e90 0x7f6ad815ee10 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:37.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.362+0000 7f6adec00640 1 -- 172.21.15.86:0/2416929419 >> 172.21.15.86:0/2416929419 conn(0x7f6ad80b2360 msgr2=0x7f6ad80b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:37.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.362+0000 7f6adec00640 1 -- 172.21.15.86:0/2416929419 shutdown_connections 2024-04-20T21:27:37.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.362+0000 7f6adec00640 1 -- 172.21.15.86:0/2416929419 wait complete. 2024-04-20T21:27:37.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.364+0000 7f6adec00640 1 Processor -- start 2024-04-20T21:27:37.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.364+0000 7f6adec00640 1 -- start start 2024-04-20T21:27:37.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.365+0000 7f6adec00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ad806f140 0x7f6ad8141620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:37.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.365+0000 7f6adec00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ad8071d00 0x7f6ad80c53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:37.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.365+0000 7f6adec00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ad8144e90 0x7f6ad80cf230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:37.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.365+0000 7f6adec00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad815f350 con 0x7f6ad8071d00 2024-04-20T21:27:37.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.365+0000 7f6adec00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad806ded0 con 0x7f6ad806f140 2024-04-20T21:27:37.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.365+0000 7f6adec00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6ad815f4d0 con 0x7f6ad8144e90 2024-04-20T21:27:37.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.365+0000 7f6ad7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ad8071d00 0x7f6ad80c53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:37.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.365+0000 7f6ad7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ad8071d00 0x7f6ad80c53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53078/0 (socket says 172.21.15.86:53078) 2024-04-20T21:27:37.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.365+0000 7f6ad7400640 1 -- 172.21.15.86:0/165167712 learned_addr learned my addr 172.21.15.86:0/165167712 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:37.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.365+0000 7f6ad7e00640 1 --2- 172.21.15.86:0/165167712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ad806f140 0x7f6ad8141620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:37.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.365+0000 7f6adcc00640 1 --2- 172.21.15.86:0/165167712 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ad8144e90 0x7f6ad80cf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:37.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.365+0000 7f6ad7400640 1 -- 172.21.15.86:0/165167712 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ad8144e90 msgr2=0x7f6ad80cf230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:37.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.366+0000 7f6ad7400640 1 --2- 172.21.15.86:0/165167712 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ad8144e90 0x7f6ad80cf230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:37.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.366+0000 7f6ad7400640 1 -- 172.21.15.86:0/165167712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ad806f140 msgr2=0x7f6ad8141620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:37.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.366+0000 7f6ad7400640 1 --2- 172.21.15.86:0/165167712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ad806f140 0x7f6ad8141620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:37.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.366+0000 7f6ad7400640 1 -- 172.21.15.86:0/165167712 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ad814f630 con 0x7f6ad8071d00 2024-04-20T21:27:37.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.366+0000 7f6adcc00640 1 --2- 172.21.15.86:0/165167712 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ad8144e90 0x7f6ad80cf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:27:37.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.366+0000 7f6ad7e00640 1 --2- 172.21.15.86:0/165167712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ad806f140 0x7f6ad8141620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:27:37.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.366+0000 7f6ad7400640 1 --2- 172.21.15.86:0/165167712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ad8071d00 0x7f6ad80c53f0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f6ac000c430 tx=0x7f6ac000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:37.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.366+0000 7f6acfe00640 1 -- 172.21.15.86:0/165167712 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac00074e0 con 0x7f6ad8071d00 2024-04-20T21:27:37.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.366+0000 7f6acfe00640 1 -- 172.21.15.86:0/165167712 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ac00047e0 con 0x7f6ad8071d00 2024-04-20T21:27:37.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.366+0000 7f6acfe00640 1 -- 172.21.15.86:0/165167712 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac0004bd0 con 0x7f6ad8071d00 2024-04-20T21:27:37.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.366+0000 7f6adec00640 1 -- 172.21.15.86:0/165167712 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ad814f910 con 0x7f6ad8071d00 2024-04-20T21:27:37.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.366+0000 7f6adec00640 1 -- 172.21.15.86:0/165167712 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ad814fed0 con 0x7f6ad8071d00 2024-04-20T21:27:37.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.368+0000 7f6adec00640 1 -- 172.21.15.86:0/165167712 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ad80689e0 con 0x7f6ad8071d00 2024-04-20T21:27:37.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.368+0000 7f6acfe00640 1 -- 172.21.15.86:0/165167712 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6ac0011070 con 0x7f6ad8071d00 2024-04-20T21:27:37.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.369+0000 7f6acfe00640 1 --2- 172.21.15.86:0/165167712 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6ab406c8f0 0x7f6ab406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:37.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.370+0000 7f6acfe00640 1 -- 172.21.15.86:0/165167712 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5115+0+0 (secure 0 0 0) 0x7f6ac008b620 con 0x7f6ad8071d00 2024-04-20T21:27:37.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.370+0000 7f6ad7e00640 1 --2- 172.21.15.86:0/165167712 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6ab406c8f0 0x7f6ab406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:37.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.370+0000 7f6ad7e00640 1 --2- 172.21.15.86:0/165167712 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6ab406c8f0 0x7f6ab406edb0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f6ac8036000 tx=0x7f6ac8008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:37.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.374+0000 7f6acfe00640 1 -- 172.21.15.86:0/165167712 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6ac0059b70 con 0x7f6ad8071d00 2024-04-20T21:27:37.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.572+0000 7f6adec00640 1 -- 172.21.15.86:0/165167712 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6ad8160710 con 0x7f6ad8071d00 2024-04-20T21:27:37.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.574+0000 7f6acfe00640 1 -- 172.21.15.86:0/165167712 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+13287 (secure 0 0 0) 0x7f6ac0059470 con 0x7f6ad8071d00 2024-04-20T21:27:37.574 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:27:37.574 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":92,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:27:36.125490+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.144:6834/2351381466":"2024-04-21T21:27:32.961895+0000","172.21.15.144:6832/2351381466":"2024-04-21T21:27:32.961895+0000","172.21.15.144:6835/3981403733":"2024-04-21T21:26:43.869334+0000","172.21.15.144:6833/3981403733":"2024-04-21T21:26:43.869334+0000","172.21.15.144:6835/1740492227":"2024-04-21T21:25:20.040733+0000","172.21.15.144:6833/1740492227":"2024-04-21T21:25:20.040733+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:27:37.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.577+0000 7f6adec00640 1 -- 172.21.15.86:0/165167712 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6ab406c8f0 msgr2=0x7f6ab406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:37.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.577+0000 7f6adec00640 1 --2- 172.21.15.86:0/165167712 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6ab406c8f0 0x7f6ab406edb0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f6ac8036000 tx=0x7f6ac8008030 comp rx=0 tx=0).stop 2024-04-20T21:27:37.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.577+0000 7f6adec00640 1 -- 172.21.15.86:0/165167712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ad8071d00 msgr2=0x7f6ad80c53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:37.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.577+0000 7f6adec00640 1 --2- 172.21.15.86:0/165167712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ad8071d00 0x7f6ad80c53f0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f6ac000c430 tx=0x7f6ac000c8f0 comp rx=0 tx=0).stop 2024-04-20T21:27:37.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.577+0000 7f6adec00640 1 -- 172.21.15.86:0/165167712 shutdown_connections 2024-04-20T21:27:37.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.577+0000 7f6adec00640 1 --2- 172.21.15.86:0/165167712 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6ab406c8f0 0x7f6ab406edb0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:37.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.577+0000 7f6adec00640 1 --2- 172.21.15.86:0/165167712 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6ad8144e90 0x7f6ad80cf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:37.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.577+0000 7f6adec00640 1 --2- 172.21.15.86:0/165167712 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ad8071d00 0x7f6ad80c53f0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:37.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.577+0000 7f6adec00640 1 --2- 172.21.15.86:0/165167712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ad806f140 0x7f6ad8141620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:37.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.577+0000 7f6adec00640 1 -- 172.21.15.86:0/165167712 >> 172.21.15.86:0/165167712 conn(0x7f6ad80b2360 msgr2=0x7f6ad806e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:37.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.577+0000 7f6adec00640 1 -- 172.21.15.86:0/165167712 shutdown_connections 2024-04-20T21:27:37.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.577+0000 7f6adec00640 1 -- 172.21.15.86:0/165167712 wait complete. 2024-04-20T21:27:37.595 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:27:37.595 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:27:37.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.795+0000 7f7422e00640 1 Processor -- start 2024-04-20T21:27:37.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.796+0000 7f7422e00640 1 -- start start 2024-04-20T21:27:37.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.797+0000 7f7422e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f741c15ee00 0x7f741c1634c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:37.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.797+0000 7f7422e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f741c0cf830 0x7f741c15fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:37.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.797+0000 7f7422e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f741c0cff90 con 0x7f741c0d1260 2024-04-20T21:27:37.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.797+0000 7f7422e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f741c0d0550 con 0x7f741c0cf830 2024-04-20T21:27:37.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.797+0000 7f7422e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f741c0d0270 con 0x7f741c15ee00 2024-04-20T21:27:37.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.797+0000 7f741b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f741c0cf830 0x7f741c15fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:37.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.797+0000 7f741b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f741c0cf830 0x7f741c15fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53084/0 (socket says 172.21.15.86:53084) 2024-04-20T21:27:37.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.797+0000 7f741b400640 1 -- 172.21.15.86:0/786883694 learned_addr learned my addr 172.21.15.86:0/786883694 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:37.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.798+0000 7f741be00640 1 --2- 172.21.15.86:0/786883694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f741c15ee00 0x7f741c1634c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:37.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.798+0000 7f741b400640 1 -- 172.21.15.86:0/786883694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f741c15ee00 msgr2=0x7f741c1634c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:37.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.798+0000 7f741b400640 1 --2- 172.21.15.86:0/786883694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f741c15ee00 0x7f741c1634c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:37.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.798+0000 7f741b400640 1 -- 172.21.15.86:0/786883694 >> v1:172.21.15.144:6789/0 conn(0x7f741c0d1260 legacy=0x7f741c0d3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:37.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.798+0000 7f741b400640 1 -- 172.21.15.86:0/786883694 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f741c15f640 con 0x7f741c0cf830 2024-04-20T21:27:37.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.798+0000 7f741be00640 1 --2- 172.21.15.86:0/786883694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f741c15ee00 0x7f741c1634c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:27:37.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.799+0000 7f741b400640 1 --2- 172.21.15.86:0/786883694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f741c0cf830 0x7f741c15fe40 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f7404009870 tx=0x7f740402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7e543f53a0fb369 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:37.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.799+0000 7f741aa00640 1 -- 172.21.15.86:0/786883694 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7404035020 con 0x7f741c0cf830 2024-04-20T21:27:37.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.799+0000 7f741aa00640 1 -- 172.21.15.86:0/786883694 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7404031640 con 0x7f741c0cf830 2024-04-20T21:27:37.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.799+0000 7f741aa00640 1 -- 172.21.15.86:0/786883694 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740402d040 con 0x7f741c0cf830 2024-04-20T21:27:37.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.800+0000 7f7422e00640 1 -- 172.21.15.86:0/786883694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f741c0cf830 msgr2=0x7f741c15fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:37.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.800+0000 7f7422e00640 1 --2- 172.21.15.86:0/786883694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f741c0cf830 0x7f741c15fe40 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f7404009870 tx=0x7f740402b8f0 comp rx=0 tx=0).stop 2024-04-20T21:27:37.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.800+0000 7f7422e00640 1 -- 172.21.15.86:0/786883694 shutdown_connections 2024-04-20T21:27:37.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.800+0000 7f7422e00640 1 --2- 172.21.15.86:0/786883694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f741c0cf830 0x7f741c15fe40 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:37.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.800+0000 7f7422e00640 1 --2- 172.21.15.86:0/786883694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f741c15ee00 0x7f741c1634c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:37.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.800+0000 7f7422e00640 1 -- 172.21.15.86:0/786883694 >> 172.21.15.86:0/786883694 conn(0x7f741c06cc20 msgr2=0x7f741c06d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:37.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.800+0000 7f7422e00640 1 -- 172.21.15.86:0/786883694 shutdown_connections 2024-04-20T21:27:37.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.800+0000 7f7422e00640 1 -- 172.21.15.86:0/786883694 wait complete. 2024-04-20T21:27:37.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.802+0000 7f7422e00640 1 Processor -- start 2024-04-20T21:27:37.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.802+0000 7f7422e00640 1 -- start start 2024-04-20T21:27:37.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.803+0000 7f7422e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f741c0cf830 0x7f741c157d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:37.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.803+0000 7f7422e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f741c0d1260 0x7f741c0cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:37.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.803+0000 7f7422e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f741c15ee00 0x7f741c0cd200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:37.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.803+0000 7f7422e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f741c15f1e0 con 0x7f741c15ee00 2024-04-20T21:27:37.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.803+0000 7f7422e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f741c160380 con 0x7f741c0cf830 2024-04-20T21:27:37.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.803+0000 7f7422e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f741c15f360 con 0x7f741c0d1260 2024-04-20T21:27:37.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.803+0000 7f7420e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f741c15ee00 0x7f741c0cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:37.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.803+0000 7f7420e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f741c15ee00 0x7f741c0cd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53090/0 (socket says 172.21.15.86:53090) 2024-04-20T21:27:37.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.803+0000 7f7420e00640 1 -- 172.21.15.86:0/4011654018 learned_addr learned my addr 172.21.15.86:0/4011654018 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:37.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.803+0000 7f741b400640 1 --2- 172.21.15.86:0/4011654018 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f741c0d1260 0x7f741c0cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:37.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.803+0000 7f741be00640 1 --2- 172.21.15.86:0/4011654018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f741c0cf830 0x7f741c157d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:37.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.803+0000 7f7420e00640 1 -- 172.21.15.86:0/4011654018 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f741c0d1260 msgr2=0x7f741c0cc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:37.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.803+0000 7f7420e00640 1 --2- 172.21.15.86:0/4011654018 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f741c0d1260 0x7f741c0cc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:37.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.804+0000 7f7420e00640 1 -- 172.21.15.86:0/4011654018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f741c0cf830 msgr2=0x7f741c157d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:37.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.804+0000 7f7420e00640 1 --2- 172.21.15.86:0/4011654018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f741c0cf830 0x7f741c157d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:37.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.804+0000 7f7420e00640 1 -- 172.21.15.86:0/4011654018 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f741c0cba50 con 0x7f741c15ee00 2024-04-20T21:27:37.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.804+0000 7f7420e00640 1 --2- 172.21.15.86:0/4011654018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f741c15ee00 0x7f741c0cd200 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f741400c440 tx=0x7f741400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:37.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.804+0000 7f7413e00640 1 -- 172.21.15.86:0/4011654018 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7414011790 con 0x7f741c15ee00 2024-04-20T21:27:37.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.804+0000 7f7413e00640 1 -- 172.21.15.86:0/4011654018 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7414017070 con 0x7f741c15ee00 2024-04-20T21:27:37.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.804+0000 7f7413e00640 1 -- 172.21.15.86:0/4011654018 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7414011b70 con 0x7f741c15ee00 2024-04-20T21:27:37.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.804+0000 7f7422e00640 1 -- 172.21.15.86:0/4011654018 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f741c158260 con 0x7f741c15ee00 2024-04-20T21:27:37.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.804+0000 7f7422e00640 1 -- 172.21.15.86:0/4011654018 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f741c0cbfd0 con 0x7f741c15ee00 2024-04-20T21:27:37.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.805+0000 7f7422e00640 1 -- 172.21.15.86:0/4011654018 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f741c0689e0 con 0x7f741c15ee00 2024-04-20T21:27:37.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.806+0000 7f7413e00640 1 -- 172.21.15.86:0/4011654018 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7414011d10 con 0x7f741c15ee00 2024-04-20T21:27:37.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.807+0000 7f7413e00640 1 --2- 172.21.15.86:0/4011654018 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f73f806c8f0 0x7f73f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:37.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.807+0000 7f7413e00640 1 -- 172.21.15.86:0/4011654018 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5115+0+0 (secure 0 0 0) 0x7f741408b6f0 con 0x7f741c15ee00 2024-04-20T21:27:37.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.807+0000 7f741be00640 1 --2- 172.21.15.86:0/4011654018 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f73f806c8f0 0x7f73f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:37.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.808+0000 7f741be00640 1 --2- 172.21.15.86:0/4011654018 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f73f806c8f0 0x7f73f806edb0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f741c161e10 tx=0x7f740c008450 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:37.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:37.812+0000 7f7413e00640 1 -- 172.21.15.86:0/4011654018 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7414059d60 con 0x7f741c15ee00 2024-04-20T21:27:38.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.009+0000 7f7422e00640 1 -- 172.21.15.86:0/4011654018 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f741c0d39d0 con 0x7f741c15ee00 2024-04-20T21:27:38.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.010+0000 7f7413e00640 1 -- 172.21.15.86:0/4011654018 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7f7414059660 con 0x7f741c15ee00 2024-04-20T21:27:38.009 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:27:38.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.013+0000 7f7422e00640 1 -- 172.21.15.86:0/4011654018 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f73f806c8f0 msgr2=0x7f73f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:38.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.013+0000 7f7422e00640 1 --2- 172.21.15.86:0/4011654018 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f73f806c8f0 0x7f73f806edb0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f741c161e10 tx=0x7f740c008450 comp rx=0 tx=0).stop 2024-04-20T21:27:38.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.013+0000 7f7422e00640 1 -- 172.21.15.86:0/4011654018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f741c15ee00 msgr2=0x7f741c0cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:38.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.013+0000 7f7422e00640 1 --2- 172.21.15.86:0/4011654018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f741c15ee00 0x7f741c0cd200 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f741400c440 tx=0x7f741400c900 comp rx=0 tx=0).stop 2024-04-20T21:27:38.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.013+0000 7f7422e00640 1 -- 172.21.15.86:0/4011654018 shutdown_connections 2024-04-20T21:27:38.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.013+0000 7f7422e00640 1 --2- 172.21.15.86:0/4011654018 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f73f806c8f0 0x7f73f806edb0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:38.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.013+0000 7f7422e00640 1 --2- 172.21.15.86:0/4011654018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f741c15ee00 0x7f741c0cd200 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:38.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.013+0000 7f7422e00640 1 --2- 172.21.15.86:0/4011654018 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f741c0d1260 0x7f741c0cc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:38.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.013+0000 7f7422e00640 1 --2- 172.21.15.86:0/4011654018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f741c0cf830 0x7f741c157d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:38.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.013+0000 7f7422e00640 1 -- 172.21.15.86:0/4011654018 >> 172.21.15.86:0/4011654018 conn(0x7f741c06cc20 msgr2=0x7f741c0ca470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:38.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.014+0000 7f7422e00640 1 -- 172.21.15.86:0/4011654018 shutdown_connections 2024-04-20T21:27:38.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.014+0000 7f7422e00640 1 -- 172.21.15.86:0/4011654018 wait complete. 2024-04-20T21:27:38.031 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-04-20T21:27:38.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.233+0000 7fab6a800640 1 Processor -- start 2024-04-20T21:27:38.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.233+0000 7fab6a800640 1 -- start start 2024-04-20T21:27:38.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.235+0000 7fab6a800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fab6415e800 0x7fab640cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:38.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.235+0000 7fab6a800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fab6406b980 0x7fab6406bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:38.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.235+0000 7fab6a800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab640b0cf0 con 0x7fab6415ebe0 2024-04-20T21:27:38.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.235+0000 7fab6a800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fab6406c310 con 0x7fab6406b980 2024-04-20T21:27:38.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.235+0000 7fab6a800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fab640d0f10 con 0x7fab6415e800 2024-04-20T21:27:38.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.235+0000 7fab69e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fab6406b980 0x7fab6406bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:38.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.236+0000 7fab69e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fab6406b980 0x7fab6406bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53104/0 (socket says 172.21.15.86:53104) 2024-04-20T21:27:38.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.236+0000 7fab69e00640 1 -- 172.21.15.86:0/2721469287 learned_addr learned my addr 172.21.15.86:0/2721469287 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:38.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.236+0000 7fab69400640 1 --2- 172.21.15.86:0/2721469287 >> v2:172.21.15.144:3300/0 conn(0x7fab6415e800 0x7fab640cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:38.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.236+0000 7fab69e00640 1 -- 172.21.15.86:0/2721469287 >> v2:172.21.15.144:3300/0 conn(0x7fab6415e800 msgr2=0x7fab640cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:38.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.236+0000 7fab69e00640 1 --2- 172.21.15.86:0/2721469287 >> v2:172.21.15.144:3300/0 conn(0x7fab6415e800 0x7fab640cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:38.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.236+0000 7fab69e00640 1 -- 172.21.15.86:0/2721469287 >> v1:172.21.15.144:6789/0 conn(0x7fab6415ebe0 legacy=0x7fab640b05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:38.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.236+0000 7fab69e00640 1 -- 172.21.15.86:0/2721469287 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab6415f450 con 0x7fab6406b980 2024-04-20T21:27:38.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.237+0000 7fab69e00640 1 --2- 172.21.15.86:0/2721469287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab6406b980 0x7fab6406bdd0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fab5c009870 tx=0x7fab5c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e74a88997f75c841 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:38.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.237+0000 7fab63e00640 1 -- 172.21.15.86:0/2721469287 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c035020 con 0x7fab6406b980 2024-04-20T21:27:38.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.238+0000 7fab63e00640 1 -- 172.21.15.86:0/2721469287 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab5c031870 con 0x7fab6406b980 2024-04-20T21:27:38.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.238+0000 7fab63e00640 1 -- 172.21.15.86:0/2721469287 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c031c40 con 0x7fab6406b980 2024-04-20T21:27:38.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.238+0000 7fab6a800640 1 -- 172.21.15.86:0/2721469287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab6406b980 msgr2=0x7fab6406bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:38.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.238+0000 7fab6a800640 1 --2- 172.21.15.86:0/2721469287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab6406b980 0x7fab6406bdd0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fab5c009870 tx=0x7fab5c02b910 comp rx=0 tx=0).stop 2024-04-20T21:27:38.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.238+0000 7fab6a800640 1 -- 172.21.15.86:0/2721469287 shutdown_connections 2024-04-20T21:27:38.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.238+0000 7fab6a800640 1 --2- 172.21.15.86:0/2721469287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab6406b980 0x7fab6406bdd0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:38.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.238+0000 7fab6a800640 1 --2- 172.21.15.86:0/2721469287 >> v2:172.21.15.144:3300/0 conn(0x7fab6415e800 0x7fab640cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:38.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.238+0000 7fab6a800640 1 -- 172.21.15.86:0/2721469287 >> 172.21.15.86:0/2721469287 conn(0x7fab640b67a0 msgr2=0x7fab640b6bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:38.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.238+0000 7fab6a800640 1 -- 172.21.15.86:0/2721469287 shutdown_connections 2024-04-20T21:27:38.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.238+0000 7fab6a800640 1 -- 172.21.15.86:0/2721469287 wait complete. 2024-04-20T21:27:38.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.240+0000 7fab6a800640 1 Processor -- start 2024-04-20T21:27:38.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.240+0000 7fab6a800640 1 -- start start 2024-04-20T21:27:38.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.241+0000 7fab6a800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab6406b980 0x7fab640b5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:38.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.241+0000 7fab6a800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab6415e800 0x7fab64157a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:38.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.241+0000 7fab6a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab6415ebe0 0x7fab6415d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:38.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.241+0000 7fab6a800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fab6415fcf0 con 0x7fab6415ebe0 2024-04-20T21:27:38.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.241+0000 7fab6a800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fab6415fa10 con 0x7fab6415e800 2024-04-20T21:27:38.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.241+0000 7fab6a800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fab6415f070 con 0x7fab6406b980 2024-04-20T21:27:38.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.242+0000 7fab69e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab6415ebe0 0x7fab6415d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:38.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.242+0000 7fab69e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab6415ebe0 0x7fab6415d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53118/0 (socket says 172.21.15.86:53118) 2024-04-20T21:27:38.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.242+0000 7fab69e00640 1 -- 172.21.15.86:0/3026628071 learned_addr learned my addr 172.21.15.86:0/3026628071 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:38.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.242+0000 7fab68a00640 1 --2- 172.21.15.86:0/3026628071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab6415e800 0x7fab64157a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:38.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.242+0000 7fab69e00640 1 -- 172.21.15.86:0/3026628071 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab6406b980 msgr2=0x7fab640b5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:38.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.242+0000 7fab69400640 1 --2- 172.21.15.86:0/3026628071 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab6406b980 0x7fab640b5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:38.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.242+0000 7fab69e00640 1 --2- 172.21.15.86:0/3026628071 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab6406b980 0x7fab640b5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:38.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.242+0000 7fab69e00640 1 -- 172.21.15.86:0/3026628071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab6415e800 msgr2=0x7fab64157a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:38.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.242+0000 7fab69e00640 1 --2- 172.21.15.86:0/3026628071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab6415e800 0x7fab64157a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:38.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.242+0000 7fab69e00640 1 -- 172.21.15.86:0/3026628071 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab640b6190 con 0x7fab6415ebe0 2024-04-20T21:27:38.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.242+0000 7fab68a00640 1 --2- 172.21.15.86:0/3026628071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab6415e800 0x7fab64157a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:27:38.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.242+0000 7fab69e00640 1 --2- 172.21.15.86:0/3026628071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab6415ebe0 0x7fab6415d9e0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fab5c02b950 tx=0x7fab5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:38.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.243+0000 7fab61600640 1 -- 172.21.15.86:0/3026628071 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c040070 con 0x7fab6415ebe0 2024-04-20T21:27:38.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.243+0000 7fab61600640 1 -- 172.21.15.86:0/3026628071 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab5c030d00 con 0x7fab6415ebe0 2024-04-20T21:27:38.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.243+0000 7fab61600640 1 -- 172.21.15.86:0/3026628071 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c035040 con 0x7fab6415ebe0 2024-04-20T21:27:38.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.243+0000 7fab6a800640 1 -- 172.21.15.86:0/3026628071 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab640b4bb0 con 0x7fab6415ebe0 2024-04-20T21:27:38.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.243+0000 7fab6a800640 1 -- 172.21.15.86:0/3026628071 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab640b5170 con 0x7fab6415ebe0 2024-04-20T21:27:38.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.244+0000 7fab6a800640 1 -- 172.21.15.86:0/3026628071 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab6404fcf0 con 0x7fab6415ebe0 2024-04-20T21:27:38.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.245+0000 7fab61600640 1 -- 172.21.15.86:0/3026628071 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fab5c038020 con 0x7fab6415ebe0 2024-04-20T21:27:38.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.246+0000 7fab61600640 1 --2- 172.21.15.86:0/3026628071 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fab4406c8f0 0x7fab4406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:38.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.246+0000 7fab61600640 1 -- 172.21.15.86:0/3026628071 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5115+0+0 (secure 0 0 0) 0x7fab5c0ad690 con 0x7fab6415ebe0 2024-04-20T21:27:38.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.246+0000 7fab69400640 1 --2- 172.21.15.86:0/3026628071 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fab4406c8f0 0x7fab4406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:38.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.247+0000 7fab69400640 1 --2- 172.21.15.86:0/3026628071 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fab4406c8f0 0x7fab4406edb0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fab58009e90 tx=0x7fab5800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:38.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.251+0000 7fab61600640 1 -- 172.21.15.86:0/3026628071 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fab5c07bd00 con 0x7fab6415ebe0 2024-04-20T21:27:38.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:38.523+0000 7fab6a800640 1 -- 172.21.15.86:0/3026628071 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) -- 0x7fab640ce200 con 0x7fab6415ebe0 2024-04-20T21:27:39.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.154+0000 7fab61600640 1 -- 172.21.15.86:0/3026628071 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v475) ==== 149+0+0 (secure 0 0 0) 0x7fab5c07b600 con 0x7fab6415ebe0 2024-04-20T21:27:39.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.158+0000 7fab6a800640 1 -- 172.21.15.86:0/3026628071 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fab4406c8f0 msgr2=0x7fab4406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:39.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.158+0000 7fab6a800640 1 --2- 172.21.15.86:0/3026628071 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fab4406c8f0 0x7fab4406edb0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fab58009e90 tx=0x7fab5800b040 comp rx=0 tx=0).stop 2024-04-20T21:27:39.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.158+0000 7fab6a800640 1 -- 172.21.15.86:0/3026628071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab6415ebe0 msgr2=0x7fab6415d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:39.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.158+0000 7fab6a800640 1 --2- 172.21.15.86:0/3026628071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab6415ebe0 0x7fab6415d9e0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fab5c02b950 tx=0x7fab5c0046c0 comp rx=0 tx=0).stop 2024-04-20T21:27:39.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.159+0000 7fab6a800640 1 -- 172.21.15.86:0/3026628071 shutdown_connections 2024-04-20T21:27:39.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.159+0000 7fab6a800640 1 --2- 172.21.15.86:0/3026628071 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fab4406c8f0 0x7fab4406edb0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:39.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.159+0000 7fab6a800640 1 --2- 172.21.15.86:0/3026628071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab6415ebe0 0x7fab6415d9e0 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:39.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.159+0000 7fab6a800640 1 --2- 172.21.15.86:0/3026628071 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab6415e800 0x7fab64157a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:39.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.159+0000 7fab6a800640 1 --2- 172.21.15.86:0/3026628071 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab6406b980 0x7fab640b5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:39.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.159+0000 7fab6a800640 1 -- 172.21.15.86:0/3026628071 >> 172.21.15.86:0/3026628071 conn(0x7fab640b67a0 msgr2=0x7fab640b1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:39.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.159+0000 7fab6a800640 1 -- 172.21.15.86:0/3026628071 shutdown_connections 2024-04-20T21:27:39.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.159+0000 7fab6a800640 1 -- 172.21.15.86:0/3026628071 wait complete. 2024-04-20T21:27:39.182 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-20T21:27:39.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.388+0000 7f5129c00640 1 Processor -- start 2024-04-20T21:27:39.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.389+0000 7f5129c00640 1 -- start start 2024-04-20T21:27:39.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.390+0000 7f5129c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5124161c90 0x7f51240ad850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:39.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.390+0000 7f5129c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51240adee0 con 0x7f51240af410 2024-04-20T21:27:39.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.390+0000 7f5129c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51240ae1c0 con 0x7f51240aef80 2024-04-20T21:27:39.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.390+0000 7f5129c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f51240ae4a0 con 0x7f5124161c90 2024-04-20T21:27:39.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.391+0000 7f5123e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5124161c90 0x7f51240ad850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:39.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.391+0000 7f5129200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f51240af410 0x7f512406e540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54240/0 (socket says 172.21.15.86:54240) 2024-04-20T21:27:39.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.391+0000 7f5129200640 1 -- 172.21.15.86:0/2301350711 learned_addr learned my addr 172.21.15.86:0/2301350711 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:27:39.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.391+0000 7f5123e00640 1 -- 172.21.15.86:0/2301350711 >> v1:172.21.15.144:6789/0 conn(0x7f51240aef80 legacy=0x7f51240b4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:39.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.391+0000 7f5123e00640 1 -- 172.21.15.86:0/2301350711 >> v1:172.21.15.86:6789/0 conn(0x7f51240af410 legacy=0x7f512406e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:39.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.391+0000 7f5123e00640 1 -- 172.21.15.86:0/2301350711 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51240ae780 con 0x7f5124161c90 2024-04-20T21:27:39.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.392+0000 7f5123e00640 1 --2- 172.21.15.86:0/2301350711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5124161c90 0x7f51240ad850 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f51140097a0 tx=0x7f511402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b534a6ee52ae01 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:39.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.393+0000 7f5122a00640 1 -- 172.21.15.86:0/2301350711 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5114035020 con 0x7f5124161c90 2024-04-20T21:27:39.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.393+0000 7f5122a00640 1 -- 172.21.15.86:0/2301350711 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5114031830 con 0x7f5124161c90 2024-04-20T21:27:39.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.393+0000 7f5122a00640 1 -- 172.21.15.86:0/2301350711 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5114031c00 con 0x7f5124161c90 2024-04-20T21:27:39.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.393+0000 7f5129c00640 1 -- 172.21.15.86:0/2301350711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5124161c90 msgr2=0x7f51240ad850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:39.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.393+0000 7f5129c00640 1 --2- 172.21.15.86:0/2301350711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5124161c90 0x7f51240ad850 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f51140097a0 tx=0x7f511402b8d0 comp rx=0 tx=0).stop 2024-04-20T21:27:39.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.394+0000 7f5129c00640 1 -- 172.21.15.86:0/2301350711 shutdown_connections 2024-04-20T21:27:39.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.394+0000 7f5129c00640 1 --2- 172.21.15.86:0/2301350711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5124161c90 0x7f51240ad850 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:39.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.394+0000 7f5129c00640 1 -- 172.21.15.86:0/2301350711 >> 172.21.15.86:0/2301350711 conn(0x7f51240733d0 msgr2=0x7f512406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:39.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.394+0000 7f5129c00640 1 -- 172.21.15.86:0/2301350711 shutdown_connections 2024-04-20T21:27:39.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.394+0000 7f5129c00640 1 -- 172.21.15.86:0/2301350711 wait complete. 2024-04-20T21:27:39.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.396+0000 7f5129c00640 1 Processor -- start 2024-04-20T21:27:39.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.396+0000 7f5129c00640 1 -- start start 2024-04-20T21:27:39.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.397+0000 7f5129c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51240aef80 0x7f51240cd500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:39.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.397+0000 7f5129c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51240af410 0x7f5124144700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:39.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.398+0000 7f5129c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5124161c90 0x7f5124144f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:39.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.398+0000 7f5129c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5124162070 con 0x7f51240aef80 2024-04-20T21:27:39.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.398+0000 7f5129c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5124070d60 con 0x7f51240af410 2024-04-20T21:27:39.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.398+0000 7f5129c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f51240b4830 con 0x7f5124161c90 2024-04-20T21:27:39.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.398+0000 7f5123e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51240aef80 0x7f51240cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:39.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.398+0000 7f5123e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51240aef80 0x7f51240cd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53128/0 (socket says 172.21.15.86:53128) 2024-04-20T21:27:39.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.398+0000 7f5123e00640 1 -- 172.21.15.86:0/434023276 learned_addr learned my addr 172.21.15.86:0/434023276 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:39.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.398+0000 7f5123e00640 1 -- 172.21.15.86:0/434023276 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5124161c90 msgr2=0x7f5124144f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:39.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.398+0000 7f5123e00640 1 --2- 172.21.15.86:0/434023276 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5124161c90 0x7f5124144f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:39.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.398+0000 7f5123e00640 1 -- 172.21.15.86:0/434023276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51240af410 msgr2=0x7f5124144700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:39.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.398+0000 7f5123e00640 1 --2- 172.21.15.86:0/434023276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51240af410 0x7f5124144700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:39.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.398+0000 7f5123e00640 1 -- 172.21.15.86:0/434023276 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51240cdb70 con 0x7f51240aef80 2024-04-20T21:27:39.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.399+0000 7f5123e00640 1 --2- 172.21.15.86:0/434023276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51240aef80 0x7f51240cd500 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f5114000c00 tx=0x7f51140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:39.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.399+0000 7f511be00640 1 -- 172.21.15.86:0/434023276 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511403aa70 con 0x7f51240aef80 2024-04-20T21:27:39.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.399+0000 7f511be00640 1 -- 172.21.15.86:0/434023276 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f511403ac10 con 0x7f51240aef80 2024-04-20T21:27:39.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.399+0000 7f511be00640 1 -- 172.21.15.86:0/434023276 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51140443f0 con 0x7f51240aef80 2024-04-20T21:27:39.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.399+0000 7f5129c00640 1 -- 172.21.15.86:0/434023276 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51240cde50 con 0x7f51240aef80 2024-04-20T21:27:39.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.399+0000 7f5129c00640 1 -- 172.21.15.86:0/434023276 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5124151d50 con 0x7f51240aef80 2024-04-20T21:27:39.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.400+0000 7f5129c00640 1 -- 172.21.15.86:0/434023276 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f512404fd90 con 0x7f51240aef80 2024-04-20T21:27:39.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.401+0000 7f511be00640 1 -- 172.21.15.86:0/434023276 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f511404b060 con 0x7f51240aef80 2024-04-20T21:27:39.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.403+0000 7f511be00640 1 --2- 172.21.15.86:0/434023276 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f510006c8f0 0x7f510006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:39.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.403+0000 7f511be00640 1 -- 172.21.15.86:0/434023276 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5115+0+0 (secure 0 0 0) 0x7f51140ad600 con 0x7f51240aef80 2024-04-20T21:27:39.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.403+0000 7f5123400640 1 --2- 172.21.15.86:0/434023276 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f510006c8f0 0x7f510006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:39.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.404+0000 7f5123400640 1 --2- 172.21.15.86:0/434023276 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f510006c8f0 0x7f510006edb0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f51240d0500 tx=0x7f510c007990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:39.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.407+0000 7f511be00640 1 -- 172.21.15.86:0/434023276 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f511403f070 con 0x7f51240aef80 2024-04-20T21:27:39.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.606+0000 7f5129c00640 1 -- 172.21.15.86:0/434023276 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f5124157a80 con 0x7f51240aef80 2024-04-20T21:27:39.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.613+0000 7f511be00640 1 -- 172.21.15.86:0/434023276 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) ==== 122+0+0 (secure 0 0 0) 0x7f511402f2e0 con 0x7f51240aef80 2024-04-20T21:27:39.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.616+0000 7f5129c00640 1 -- 172.21.15.86:0/434023276 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f510006c8f0 msgr2=0x7f510006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:39.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.616+0000 7f5129c00640 1 --2- 172.21.15.86:0/434023276 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f510006c8f0 0x7f510006edb0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f51240d0500 tx=0x7f510c007990 comp rx=0 tx=0).stop 2024-04-20T21:27:39.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.616+0000 7f5129c00640 1 -- 172.21.15.86:0/434023276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51240aef80 msgr2=0x7f51240cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:39.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.616+0000 7f5129c00640 1 --2- 172.21.15.86:0/434023276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51240aef80 0x7f51240cd500 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f5114000c00 tx=0x7f51140046c0 comp rx=0 tx=0).stop 2024-04-20T21:27:39.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.616+0000 7f5129c00640 1 -- 172.21.15.86:0/434023276 shutdown_connections 2024-04-20T21:27:39.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.616+0000 7f5129c00640 1 --2- 172.21.15.86:0/434023276 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f510006c8f0 0x7f510006edb0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:39.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.616+0000 7f5129c00640 1 --2- 172.21.15.86:0/434023276 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5124161c90 0x7f5124144f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:39.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.616+0000 7f5129c00640 1 --2- 172.21.15.86:0/434023276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51240af410 0x7f5124144700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:39.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.616+0000 7f5129c00640 1 --2- 172.21.15.86:0/434023276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f51240aef80 0x7f51240cd500 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:39.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.616+0000 7f5129c00640 1 -- 172.21.15.86:0/434023276 >> 172.21.15.86:0/434023276 conn(0x7f51240733d0 msgr2=0x7f51240b50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:39.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.616+0000 7f5129c00640 1 -- 172.21.15.86:0/434023276 shutdown_connections 2024-04-20T21:27:39.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:39.616+0000 7f5129c00640 1 -- 172.21.15.86:0/434023276 wait complete. 2024-04-20T21:27:39.633 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-20T21:27:39.633 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T21:27:39.736 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T21:27:39.839 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-20T21:27:39.839 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:27:39.840 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:27:39.897 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052731,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248996k,nr_inodes=812249,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T21:27:39.897 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:27:39.897 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:27:39.955 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T21:27:39.955 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:27:39.955 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:27:40.013 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052731,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248996k,nr_inodes=812249,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T21:27:40.013 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:27:40.013 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:27:40.068 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T21:27:40.069 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-20T21:27:40.069 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:27:40.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.280+0000 7f4d59200640 1 Processor -- start 2024-04-20T21:27:40.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.280+0000 7f4d59200640 1 -- start start 2024-04-20T21:27:40.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.281+0000 7f4d59200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4d54144e90 0x7f4d5415ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:40.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.282+0000 7f4d59200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d54071d00 0x7f4d54072190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:40.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.282+0000 7f4d59200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d5406dc30 con 0x7f4d5406f140 2024-04-20T21:27:40.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.282+0000 7f4d59200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f4d54145270 con 0x7f4d54144e90 2024-04-20T21:27:40.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.282+0000 7f4d59200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f4d5415f720 con 0x7f4d54071d00 2024-04-20T21:27:40.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.282+0000 7f4d52800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d54071d00 0x7f4d54072190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:40.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.282+0000 7f4d51e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4d54144e90 0x7f4d5415ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:40.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.282+0000 7f4d52800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d54071d00 0x7f4d54072190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:39400/0 (socket says 172.21.15.86:39400) 2024-04-20T21:27:40.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.282+0000 7f4d52800640 1 -- 172.21.15.86:0/2248633521 learned_addr learned my addr 172.21.15.86:0/2248633521 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:40.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.283+0000 7f4d52800640 1 -- 172.21.15.86:0/2248633521 >> v2:172.21.15.144:3300/0 conn(0x7f4d54144e90 msgr2=0x7f4d5415ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:40.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.283+0000 7f4d52800640 1 --2- 172.21.15.86:0/2248633521 >> v2:172.21.15.144:3300/0 conn(0x7f4d54144e90 0x7f4d5415ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:40.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.283+0000 7f4d52800640 1 -- 172.21.15.86:0/2248633521 >> v1:172.21.15.144:6789/0 conn(0x7f4d5406f140 legacy=0x7f4d540aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:40.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.283+0000 7f4d52800640 1 -- 172.21.15.86:0/2248633521 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d5415fa00 con 0x7f4d54071d00 2024-04-20T21:27:40.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.283+0000 7f4d51e00640 1 --2- 172.21.15.86:0/2248633521 >> v2:172.21.15.144:3300/0 conn(0x7f4d54144e90 0x7f4d5415ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:27:40.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.285+0000 7f4d52800640 1 --2- 172.21.15.86:0/2248633521 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d54071d00 0x7f4d54072190 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4d541453f0 tx=0x7f4d4c02bf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=920da382329f2aef server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:40.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.285+0000 7f4d51400640 1 -- 172.21.15.86:0/2248633521 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c02cae0 con 0x7f4d54071d00 2024-04-20T21:27:40.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.285+0000 7f4d51400640 1 -- 172.21.15.86:0/2248633521 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d4c02cc80 con 0x7f4d54071d00 2024-04-20T21:27:40.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.285+0000 7f4d51400640 1 -- 172.21.15.86:0/2248633521 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c034610 con 0x7f4d54071d00 2024-04-20T21:27:40.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.286+0000 7f4d59200640 1 -- 172.21.15.86:0/2248633521 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d54071d00 msgr2=0x7f4d54072190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:40.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.286+0000 7f4d59200640 1 --2- 172.21.15.86:0/2248633521 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d54071d00 0x7f4d54072190 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4d541453f0 tx=0x7f4d4c02bf40 comp rx=0 tx=0).stop 2024-04-20T21:27:40.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.286+0000 7f4d59200640 1 -- 172.21.15.86:0/2248633521 shutdown_connections 2024-04-20T21:27:40.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.286+0000 7f4d59200640 1 --2- 172.21.15.86:0/2248633521 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d54071d00 0x7f4d54072190 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:40.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.286+0000 7f4d59200640 1 --2- 172.21.15.86:0/2248633521 >> v2:172.21.15.144:3300/0 conn(0x7f4d54144e90 0x7f4d5415ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:40.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.286+0000 7f4d59200640 1 -- 172.21.15.86:0/2248633521 >> 172.21.15.86:0/2248633521 conn(0x7f4d540b2360 msgr2=0x7f4d540b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:40.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.286+0000 7f4d59200640 1 -- 172.21.15.86:0/2248633521 shutdown_connections 2024-04-20T21:27:40.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.286+0000 7f4d59200640 1 -- 172.21.15.86:0/2248633521 wait complete. 2024-04-20T21:27:40.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.288+0000 7f4d59200640 1 Processor -- start 2024-04-20T21:27:40.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.288+0000 7f4d59200640 1 -- start start 2024-04-20T21:27:40.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.289+0000 7f4d59200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d5406f140 0x7f4d540cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:40.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.289+0000 7f4d59200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d54071d00 0x7f4d54161140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:40.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.289+0000 7f4d59200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d54144e90 0x7f4d54161680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:40.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.289+0000 7f4d59200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5415f350 con 0x7f4d54071d00 2024-04-20T21:27:40.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.289+0000 7f4d59200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5406ded0 con 0x7f4d5406f140 2024-04-20T21:27:40.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.289+0000 7f4d59200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f4d5415f4d0 con 0x7f4d54144e90 2024-04-20T21:27:40.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.289+0000 7f4d49400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d54071d00 0x7f4d54161140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:40.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.289+0000 7f4d49400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d54071d00 0x7f4d54161140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53130/0 (socket says 172.21.15.86:53130) 2024-04-20T21:27:40.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.289+0000 7f4d49400640 1 -- 172.21.15.86:0/3436242457 learned_addr learned my addr 172.21.15.86:0/3436242457 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:40.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.289+0000 7f4d51e00640 1 --2- 172.21.15.86:0/3436242457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d5406f140 0x7f4d540cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:40.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.290+0000 7f4d52800640 1 --2- 172.21.15.86:0/3436242457 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d54144e90 0x7f4d54161680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:40.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.290+0000 7f4d49400640 1 -- 172.21.15.86:0/3436242457 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d54144e90 msgr2=0x7f4d54161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:40.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.290+0000 7f4d49400640 1 --2- 172.21.15.86:0/3436242457 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d54144e90 0x7f4d54161680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:40.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.290+0000 7f4d49400640 1 -- 172.21.15.86:0/3436242457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d5406f140 msgr2=0x7f4d540cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:40.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.290+0000 7f4d49400640 1 --2- 172.21.15.86:0/3436242457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d5406f140 0x7f4d540cbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:40.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.290+0000 7f4d49400640 1 -- 172.21.15.86:0/3436242457 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d54148080 con 0x7f4d54071d00 2024-04-20T21:27:40.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.290+0000 7f4d49400640 1 --2- 172.21.15.86:0/3436242457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d54071d00 0x7f4d54161140 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c00c3f0 tx=0x7f4d3c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:40.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.290+0000 7f4d4aa00640 1 -- 172.21.15.86:0/3436242457 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c0052d0 con 0x7f4d54071d00 2024-04-20T21:27:40.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.290+0000 7f4d4aa00640 1 -- 172.21.15.86:0/3436242457 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d3c005490 con 0x7f4d54071d00 2024-04-20T21:27:40.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.290+0000 7f4d59200640 1 -- 172.21.15.86:0/3436242457 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d540c5490 con 0x7f4d54071d00 2024-04-20T21:27:40.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.290+0000 7f4d4aa00640 1 -- 172.21.15.86:0/3436242457 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c007260 con 0x7f4d54071d00 2024-04-20T21:27:40.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.291+0000 7f4d59200640 1 -- 172.21.15.86:0/3436242457 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d54148360 con 0x7f4d54071d00 2024-04-20T21:27:40.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.292+0000 7f4d59200640 1 -- 172.21.15.86:0/3436242457 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d540689e0 con 0x7f4d54071d00 2024-04-20T21:27:40.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.293+0000 7f4d4aa00640 1 -- 172.21.15.86:0/3436242457 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4d3c007400 con 0x7f4d54071d00 2024-04-20T21:27:40.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.294+0000 7f4d4aa00640 1 --2- 172.21.15.86:0/3436242457 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4d3006c8f0 0x7f4d3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:40.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.294+0000 7f4d4aa00640 1 -- 172.21.15.86:0/3436242457 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5115+0+0 (secure 0 0 0) 0x7f4d3c08adb0 con 0x7f4d54071d00 2024-04-20T21:27:40.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.294+0000 7f4d51e00640 1 --2- 172.21.15.86:0/3436242457 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4d3006c8f0 0x7f4d3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:40.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.295+0000 7f4d51e00640 1 --2- 172.21.15.86:0/3436242457 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4d3006c8f0 0x7f4d3006edb0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f4d540b3800 tx=0x7f4d44008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:40.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.298+0000 7f4d4aa00640 1 -- 172.21.15.86:0/3436242457 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4d3c08f070 con 0x7f4d54071d00 2024-04-20T21:27:40.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.537+0000 7f4d59200640 1 -- 172.21.15.86:0/3436242457 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4d5406e9b0 con 0x7f4d54071d00 2024-04-20T21:27:40.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.539+0000 7f4d4aa00640 1 -- 172.21.15.86:0/3436242457 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+3875 (secure 0 0 0) 0x7f4d3c059420 con 0x7f4d54071d00 2024-04-20T21:27:40.538 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:27:40.538 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":75,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5887,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2433619638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6833","nonce":2433619638},{"type":"v1","addr":"172.21.15.144:6835","nonce":2433619638}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":68},{"gid":5971,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6834/1313305027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":1313305027},{"type":"v1","addr":"172.21.15.144:6834","nonce":1313305027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":7506,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/4013863257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4013863257},{"type":"v1","addr":"172.21.15.86:6836","nonce":4013863257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":7509,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/2044883073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2044883073},{"type":"v1","addr":"172.21.15.86:6837","nonce":2044883073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[]} 2024-04-20T21:27:40.538 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 75 2024-04-20T21:27:40.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.541+0000 7f4d59200640 1 -- 172.21.15.86:0/3436242457 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4d3006c8f0 msgr2=0x7f4d3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:40.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.541+0000 7f4d59200640 1 --2- 172.21.15.86:0/3436242457 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4d3006c8f0 0x7f4d3006edb0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f4d540b3800 tx=0x7f4d44008040 comp rx=0 tx=0).stop 2024-04-20T21:27:40.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.541+0000 7f4d59200640 1 -- 172.21.15.86:0/3436242457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d54071d00 msgr2=0x7f4d54161140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:40.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.541+0000 7f4d59200640 1 --2- 172.21.15.86:0/3436242457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d54071d00 0x7f4d54161140 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c00c3f0 tx=0x7f4d3c00c8b0 comp rx=0 tx=0).stop 2024-04-20T21:27:40.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.541+0000 7f4d59200640 1 -- 172.21.15.86:0/3436242457 shutdown_connections 2024-04-20T21:27:40.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.541+0000 7f4d59200640 1 --2- 172.21.15.86:0/3436242457 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4d3006c8f0 0x7f4d3006edb0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:40.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.541+0000 7f4d59200640 1 --2- 172.21.15.86:0/3436242457 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d54144e90 0x7f4d54161680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:40.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.541+0000 7f4d59200640 1 --2- 172.21.15.86:0/3436242457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d54071d00 0x7f4d54161140 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:40.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.541+0000 7f4d59200640 1 --2- 172.21.15.86:0/3436242457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d5406f140 0x7f4d540cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:40.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.541+0000 7f4d59200640 1 -- 172.21.15.86:0/3436242457 >> 172.21.15.86:0/3436242457 conn(0x7f4d540b2360 msgr2=0x7f4d540b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:40.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.542+0000 7f4d59200640 1 -- 172.21.15.86:0/3436242457 shutdown_connections 2024-04-20T21:27:40.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.542+0000 7f4d59200640 1 -- 172.21.15.86:0/3436242457 wait complete. 2024-04-20T21:27:40.559 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-20T21:27:40.559 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-20T21:27:40.559 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-20T21:27:40.559 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:27:40.559 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-20T21:27:40.559 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-20T21:27:40.560 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-20T21:27:40.560 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:27:40.560 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-20T21:27:40.560 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-20T21:27:40.560 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-20T21:27:40.560 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:27:40.560 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-20T21:27:40.560 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-20T21:27:40.561 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-20T21:27:40.561 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:27:40.576 INFO:tasks.ceph.mds.c:Stopped 2024-04-20T21:27:40.576 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-20T21:27:40.577 INFO:tasks.ceph.mds.a:Stopped 2024-04-20T21:27:40.577 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-20T21:27:40.580 INFO:tasks.ceph.mds.b:Stopped 2024-04-20T21:27:40.580 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-20T21:27:40.584 INFO:tasks.ceph.mds.d:Stopped 2024-04-20T21:27:40.584 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-20T21:27:40.625 INFO:tasks.ceph.mds.c:Started 2024-04-20T21:27:40.625 DEBUG:teuthology.parallel:result is None 2024-04-20T21:27:40.631 INFO:tasks.ceph.mds.a:Started 2024-04-20T21:27:40.631 DEBUG:teuthology.parallel:result is None 2024-04-20T21:27:40.631 INFO:tasks.ceph.mds.b:Started 2024-04-20T21:27:40.632 DEBUG:teuthology.parallel:result is None 2024-04-20T21:27:40.638 INFO:tasks.ceph.mds.d:Started 2024-04-20T21:27:40.638 DEBUG:teuthology.parallel:result is None 2024-04-20T21:27:40.639 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-20T21:27:40.777 INFO:tasks.ceph.mds.a.smithi086.stdout:starting mds.a at 2024-04-20T21:27:40.777 INFO:tasks.ceph.mds.c.smithi086.stdout:starting mds.c at 2024-04-20T21:27:40.788 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2024-04-20T21:27:40.792 INFO:tasks.ceph.mds.d.smithi144.stdout:starting mds.d at 2024-04-20T21:27:40.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.818+0000 7fb8b2600640 1 Processor -- start 2024-04-20T21:27:40.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.819+0000 7fb8b2600640 1 -- start start 2024-04-20T21:27:40.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.820+0000 7fb8b2600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8ac0cd7e0 0x7fb8ac1676e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:40.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.820+0000 7fb8b2600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8ac15dcc0 con 0x7fb8ac145fe0 2024-04-20T21:27:40.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.820+0000 7fb8b2600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8ac15cf00 con 0x7fb8ac0d09e0 2024-04-20T21:27:40.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.820+0000 7fb8b2600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb8ac0d00d0 con 0x7fb8ac0cd7e0 2024-04-20T21:27:40.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.820+0000 7fb8ab400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fb8ac145fe0 0x7fb8ac0d02d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54266/0 (socket says 172.21.15.86:54266) 2024-04-20T21:27:40.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.820+0000 7fb8ab400640 1 -- 172.21.15.86:0/2296426801 learned_addr learned my addr 172.21.15.86:0/2296426801 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:27:40.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.820+0000 7fb8abe00640 1 --2- 172.21.15.86:0/2296426801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8ac0cd7e0 0x7fb8ac1676e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:40.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.821+0000 7fb8aaa00640 1 -- 172.21.15.86:0/2296426801 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3365405394 0 0) 0x7fb8ac15dcc0 con 0x7fb8ac145fe0 2024-04-20T21:27:40.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.821+0000 7fb8aaa00640 1 -- 172.21.15.86:0/2296426801 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb890003520 con 0x7fb8ac145fe0 2024-04-20T21:27:40.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.821+0000 7fb8aaa00640 1 -- 172.21.15.86:0/2296426801 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2604664528 0 0) 0x7fb8ac15cf00 con 0x7fb8ac0d09e0 2024-04-20T21:27:40.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.821+0000 7fb8aaa00640 1 -- 172.21.15.86:0/2296426801 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb8ac15dcc0 con 0x7fb8ac0d09e0 2024-04-20T21:27:40.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.821+0000 7fb8abe00640 1 -- 172.21.15.86:0/2296426801 >> v1:172.21.15.144:6789/0 conn(0x7fb8ac0d09e0 legacy=0x7fb8ac0cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:40.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.821+0000 7fb8abe00640 1 -- 172.21.15.86:0/2296426801 >> v1:172.21.15.86:6789/0 conn(0x7fb8ac145fe0 legacy=0x7fb8ac0d02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:40.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.821+0000 7fb8abe00640 1 -- 172.21.15.86:0/2296426801 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8ac15d1e0 con 0x7fb8ac0cd7e0 2024-04-20T21:27:40.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.822+0000 7fb8abe00640 1 --2- 172.21.15.86:0/2296426801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8ac0cd7e0 0x7fb8ac1676e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb8a40097d0 tx=0x7fb8a402b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=801bc99240304927 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:40.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.822+0000 7fb8aaa00640 1 -- 172.21.15.86:0/2296426801 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a4035020 con 0x7fb8ac0cd7e0 2024-04-20T21:27:40.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.822+0000 7fb8aaa00640 1 -- 172.21.15.86:0/2296426801 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8a4031890 con 0x7fb8ac0cd7e0 2024-04-20T21:27:40.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.822+0000 7fb8aaa00640 1 -- 172.21.15.86:0/2296426801 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a4031c60 con 0x7fb8ac0cd7e0 2024-04-20T21:27:40.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.823+0000 7fb8b2600640 1 -- 172.21.15.86:0/2296426801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8ac0cd7e0 msgr2=0x7fb8ac1676e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:40.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.823+0000 7fb8b2600640 1 --2- 172.21.15.86:0/2296426801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8ac0cd7e0 0x7fb8ac1676e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb8a40097d0 tx=0x7fb8a402b930 comp rx=0 tx=0).stop 2024-04-20T21:27:40.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.823+0000 7fb8b2600640 1 -- 172.21.15.86:0/2296426801 shutdown_connections 2024-04-20T21:27:40.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.823+0000 7fb8b2600640 1 --2- 172.21.15.86:0/2296426801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8ac0cd7e0 0x7fb8ac1676e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:40.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.823+0000 7fb8b2600640 1 -- 172.21.15.86:0/2296426801 >> 172.21.15.86:0/2296426801 conn(0x7fb8ac06b980 msgr2=0x7fb8ac06bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:40.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.823+0000 7fb8b2600640 1 -- 172.21.15.86:0/2296426801 shutdown_connections 2024-04-20T21:27:40.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.823+0000 7fb8b2600640 1 -- 172.21.15.86:0/2296426801 wait complete. 2024-04-20T21:27:40.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.825+0000 7fb8b2600640 1 Processor -- start 2024-04-20T21:27:40.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.825+0000 7fb8b2600640 1 -- start start 2024-04-20T21:27:40.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.826+0000 7fb8b2600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8ac0cd7e0 0x7fb8ac1405c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:40.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.826+0000 7fb8b2600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8ac0d09e0 0x7fb8ac13f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:40.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.826+0000 7fb8b2600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8ac145fe0 0x7fb8ac0b00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:40.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.826+0000 7fb8b2600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ac15d7a0 con 0x7fb8ac145fe0 2024-04-20T21:27:40.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.826+0000 7fb8b2600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ac1463c0 con 0x7fb8ac0d09e0 2024-04-20T21:27:40.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.826+0000 7fb8b2600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb8ac0cdce0 con 0x7fb8ac0cd7e0 2024-04-20T21:27:40.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.826+0000 7fb8abe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8ac145fe0 0x7fb8ac0b00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:40.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.826+0000 7fb8abe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8ac145fe0 0x7fb8ac0b00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53186/0 (socket says 172.21.15.86:53186) 2024-04-20T21:27:40.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.826+0000 7fb8abe00640 1 -- 172.21.15.86:0/2911351764 learned_addr learned my addr 172.21.15.86:0/2911351764 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:40.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.826+0000 7fb8ab400640 1 --2- 172.21.15.86:0/2911351764 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8ac0cd7e0 0x7fb8ac1405c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:40.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.826+0000 7fb8a2a00640 1 --2- 172.21.15.86:0/2911351764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8ac0d09e0 0x7fb8ac13f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:40.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.827+0000 7fb8abe00640 1 -- 172.21.15.86:0/2911351764 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8ac0cd7e0 msgr2=0x7fb8ac1405c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:40.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.827+0000 7fb8abe00640 1 --2- 172.21.15.86:0/2911351764 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8ac0cd7e0 0x7fb8ac1405c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:40.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.827+0000 7fb8abe00640 1 -- 172.21.15.86:0/2911351764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8ac0d09e0 msgr2=0x7fb8ac13f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:40.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.827+0000 7fb8abe00640 1 --2- 172.21.15.86:0/2911351764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8ac0d09e0 0x7fb8ac13f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:40.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.827+0000 7fb8abe00640 1 -- 172.21.15.86:0/2911351764 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8ac136880 con 0x7fb8ac145fe0 2024-04-20T21:27:40.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.827+0000 7fb8abe00640 1 --2- 172.21.15.86:0/2911351764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8ac145fe0 0x7fb8ac0b00a0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fb8a40097d0 tx=0x7fb8a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:40.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.827+0000 7fb8a3e00640 1 -- 172.21.15.86:0/2911351764 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a4035040 con 0x7fb8ac145fe0 2024-04-20T21:27:40.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.828+0000 7fb8a3e00640 1 -- 172.21.15.86:0/2911351764 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8a4030ca0 con 0x7fb8ac145fe0 2024-04-20T21:27:40.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.828+0000 7fb8b2600640 1 -- 172.21.15.86:0/2911351764 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8ac135d00 con 0x7fb8ac145fe0 2024-04-20T21:27:40.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.828+0000 7fb8a3e00640 1 -- 172.21.15.86:0/2911351764 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a4006e10 con 0x7fb8ac145fe0 2024-04-20T21:27:40.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.828+0000 7fb8b2600640 1 -- 172.21.15.86:0/2911351764 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8ac1362c0 con 0x7fb8ac145fe0 2024-04-20T21:27:40.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.829+0000 7fb8b2600640 1 -- 172.21.15.86:0/2911351764 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8ac0689e0 con 0x7fb8ac145fe0 2024-04-20T21:27:40.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.830+0000 7fb8a3e00640 1 -- 172.21.15.86:0/2911351764 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb8a4038020 con 0x7fb8ac145fe0 2024-04-20T21:27:40.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.830+0000 7fb8a3e00640 1 --2- 172.21.15.86:0/2911351764 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb88806c8f0 0x7fb88806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:40.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.831+0000 7fb8a3e00640 1 -- 172.21.15.86:0/2911351764 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5115+0+0 (secure 0 0 0) 0x7fb8a40aded0 con 0x7fb8ac145fe0 2024-04-20T21:27:40.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.831+0000 7fb8ab400640 1 --2- 172.21.15.86:0/2911351764 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb88806c8f0 0x7fb88806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:40.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.831+0000 7fb8ab400640 1 --2- 172.21.15.86:0/2911351764 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb88806c8f0 0x7fb88806edb0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fb89c0012e0 tx=0x7fb89c001a30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:40.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:40.835+0000 7fb8a3e00640 1 -- 172.21.15.86:0/2911351764 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb8a402d040 con 0x7fb8ac145fe0 2024-04-20T21:27:41.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.041+0000 7fb8b2600640 1 -- 172.21.15.86:0/2911351764 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fb8ac0d1020 con 0x7fb8ac145fe0 2024-04-20T21:27:41.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.178+0000 7fb8a3e00640 1 -- 172.21.15.86:0/2911351764 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v94) ==== 97+0+0 (secure 0 0 0) 0x7fb8a407be60 con 0x7fb8ac145fe0 2024-04-20T21:27:41.177 INFO:teuthology.orchestra.run.smithi086.stderr: removed all blocklist entries 2024-04-20T21:27:41.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.180+0000 7fb8b2600640 1 -- 172.21.15.86:0/2911351764 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb88806c8f0 msgr2=0x7fb88806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:41.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.181+0000 7fb8b2600640 1 --2- 172.21.15.86:0/2911351764 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb88806c8f0 0x7fb88806edb0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fb89c0012e0 tx=0x7fb89c001a30 comp rx=0 tx=0).stop 2024-04-20T21:27:41.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.181+0000 7fb8b2600640 1 -- 172.21.15.86:0/2911351764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8ac145fe0 msgr2=0x7fb8ac0b00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:41.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.181+0000 7fb8b2600640 1 --2- 172.21.15.86:0/2911351764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8ac145fe0 0x7fb8ac0b00a0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fb8a40097d0 tx=0x7fb8a40046c0 comp rx=0 tx=0).stop 2024-04-20T21:27:41.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.181+0000 7fb8b2600640 1 -- 172.21.15.86:0/2911351764 shutdown_connections 2024-04-20T21:27:41.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.181+0000 7fb8b2600640 1 --2- 172.21.15.86:0/2911351764 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb88806c8f0 0x7fb88806edb0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:41.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.181+0000 7fb8b2600640 1 --2- 172.21.15.86:0/2911351764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8ac145fe0 0x7fb8ac0b00a0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:41.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.181+0000 7fb8b2600640 1 --2- 172.21.15.86:0/2911351764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8ac0d09e0 0x7fb8ac13f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:41.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.181+0000 7fb8b2600640 1 --2- 172.21.15.86:0/2911351764 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8ac0cd7e0 0x7fb8ac1405c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:41.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.181+0000 7fb8b2600640 1 -- 172.21.15.86:0/2911351764 >> 172.21.15.86:0/2911351764 conn(0x7fb8ac06b980 msgr2=0x7fb8ac0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:41.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.181+0000 7fb8b2600640 1 -- 172.21.15.86:0/2911351764 shutdown_connections 2024-04-20T21:27:41.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.181+0000 7fb8b2600640 1 -- 172.21.15.86:0/2911351764 wait complete. 2024-04-20T21:27:41.200 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-20T21:27:41.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.395+0000 7faccae00640 1 Processor -- start 2024-04-20T21:27:41.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.395+0000 7faccae00640 1 -- start start 2024-04-20T21:27:41.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.397+0000 7faccae00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7facc40b45b0 0x7facc40b3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:41.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.397+0000 7faccae00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7facc40cf550 0x7facc40cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:41.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.397+0000 7faccae00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facc40b4b50 con 0x7facc415e870 2024-04-20T21:27:41.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.397+0000 7faccae00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7facc40b5c90 con 0x7facc40b45b0 2024-04-20T21:27:41.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.397+0000 7faccae00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7facc40b4e30 con 0x7facc40cf550 2024-04-20T21:27:41.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.397+0000 7facc9000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7facc40b45b0 0x7facc40b3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:41.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.397+0000 7facc9000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7facc40b45b0 0x7facc40b3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53188/0 (socket says 172.21.15.86:53188) 2024-04-20T21:27:41.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.397+0000 7facc9000640 1 -- 172.21.15.86:0/741650372 learned_addr learned my addr 172.21.15.86:0/741650372 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:41.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.397+0000 7facc9000640 1 -- 172.21.15.86:0/741650372 >> v2:172.21.15.144:3300/0 conn(0x7facc40cf550 msgr2=0x7facc40cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:41.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.397+0000 7facc9000640 1 --2- 172.21.15.86:0/741650372 >> v2:172.21.15.144:3300/0 conn(0x7facc40cf550 0x7facc40cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:41.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.397+0000 7facc9000640 1 -- 172.21.15.86:0/741650372 >> v1:172.21.15.144:6789/0 conn(0x7facc415e870 legacy=0x7facc40d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:41.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.397+0000 7facc9000640 1 -- 172.21.15.86:0/741650372 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facc40b5110 con 0x7facc40b45b0 2024-04-20T21:27:41.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.399+0000 7facc9000640 1 --2- 172.21.15.86:0/741650372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facc40b45b0 0x7facc40b3970 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7facb0009870 tx=0x7facb002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80943a3d84a89db6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:41.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.399+0000 7facc3e00640 1 -- 172.21.15.86:0/741650372 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb0035020 con 0x7facc40b45b0 2024-04-20T21:27:41.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.399+0000 7facc3e00640 1 -- 172.21.15.86:0/741650372 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facb0031870 con 0x7facc40b45b0 2024-04-20T21:27:41.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.399+0000 7facc3e00640 1 -- 172.21.15.86:0/741650372 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb0031c40 con 0x7facc40b45b0 2024-04-20T21:27:41.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.399+0000 7faccae00640 1 -- 172.21.15.86:0/741650372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facc40b45b0 msgr2=0x7facc40b3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:41.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.399+0000 7faccae00640 1 --2- 172.21.15.86:0/741650372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facc40b45b0 0x7facc40b3970 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7facb0009870 tx=0x7facb002b910 comp rx=0 tx=0).stop 2024-04-20T21:27:41.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.400+0000 7faccae00640 1 -- 172.21.15.86:0/741650372 shutdown_connections 2024-04-20T21:27:41.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.400+0000 7faccae00640 1 --2- 172.21.15.86:0/741650372 >> v2:172.21.15.144:3300/0 conn(0x7facc40cf550 0x7facc40cf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:41.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.400+0000 7faccae00640 1 --2- 172.21.15.86:0/741650372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facc40b45b0 0x7facc40b3970 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:41.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.400+0000 7faccae00640 1 -- 172.21.15.86:0/741650372 >> 172.21.15.86:0/741650372 conn(0x7facc409d880 msgr2=0x7facc408bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:41.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.400+0000 7faccae00640 1 -- 172.21.15.86:0/741650372 shutdown_connections 2024-04-20T21:27:41.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.400+0000 7faccae00640 1 -- 172.21.15.86:0/741650372 wait complete. 2024-04-20T21:27:41.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.401+0000 7faccae00640 1 Processor -- start 2024-04-20T21:27:41.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.402+0000 7faccae00640 1 -- start start 2024-04-20T21:27:41.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.402+0000 7faccae00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facc40b45b0 0x7facc4155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:41.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.402+0000 7faccae00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facc40cf550 0x7facc40cd210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:41.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.402+0000 7faccae00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7facc415e870 0x7facc415d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:41.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.402+0000 7faccae00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7facc40b6120 con 0x7facc40b45b0 2024-04-20T21:27:41.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.402+0000 7faccae00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7facc415ec50 con 0x7facc40cf550 2024-04-20T21:27:41.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.402+0000 7faccae00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7facc40b62a0 con 0x7facc415e870 2024-04-20T21:27:41.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.403+0000 7facc9a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facc40b45b0 0x7facc4155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:41.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.403+0000 7facc9a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facc40b45b0 0x7facc4155870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53202/0 (socket says 172.21.15.86:53202) 2024-04-20T21:27:41.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.403+0000 7facc9a00640 1 -- 172.21.15.86:0/1530407961 learned_addr learned my addr 172.21.15.86:0/1530407961 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:41.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.403+0000 7facc9a00640 1 -- 172.21.15.86:0/1530407961 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7facc415e870 msgr2=0x7facc415d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:41.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.403+0000 7facc9a00640 1 --2- 172.21.15.86:0/1530407961 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7facc415e870 0x7facc415d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:41.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.403+0000 7facc9a00640 1 -- 172.21.15.86:0/1530407961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facc40cf550 msgr2=0x7facc40cd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:41.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.403+0000 7facc9a00640 1 --2- 172.21.15.86:0/1530407961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facc40cf550 0x7facc40cd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:41.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.403+0000 7facc9a00640 1 -- 172.21.15.86:0/1530407961 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facc40cc170 con 0x7facc40b45b0 2024-04-20T21:27:41.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.403+0000 7facc9a00640 1 --2- 172.21.15.86:0/1530407961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facc40b45b0 0x7facc4155870 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7facb800b3e0 tx=0x7facb800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:41.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.404+0000 7facc1600640 1 -- 172.21.15.86:0/1530407961 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb8015020 con 0x7facc40b45b0 2024-04-20T21:27:41.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.404+0000 7faccae00640 1 -- 172.21.15.86:0/1530407961 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facc40cc450 con 0x7facc40b45b0 2024-04-20T21:27:41.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.404+0000 7facc1600640 1 -- 172.21.15.86:0/1530407961 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facb8011850 con 0x7facc40b45b0 2024-04-20T21:27:41.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.404+0000 7facc1600640 1 -- 172.21.15.86:0/1530407961 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb8011c40 con 0x7facc40b45b0 2024-04-20T21:27:41.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.404+0000 7faccae00640 1 -- 172.21.15.86:0/1530407961 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facc40cca10 con 0x7facc40b45b0 2024-04-20T21:27:41.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.405+0000 7faccae00640 1 -- 172.21.15.86:0/1530407961 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facc404fcf0 con 0x7facc40b45b0 2024-04-20T21:27:41.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.406+0000 7facc1600640 1 -- 172.21.15.86:0/1530407961 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7facb8018020 con 0x7facc40b45b0 2024-04-20T21:27:41.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.407+0000 7facc1600640 1 --2- 172.21.15.86:0/1530407961 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7faca406c8f0 0x7faca406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:41.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.407+0000 7facc1600640 1 -- 172.21.15.86:0/1530407961 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 4857+0+0 (secure 0 0 0) 0x7facb808b800 con 0x7facc40b45b0 2024-04-20T21:27:41.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.407+0000 7facc9000640 1 --2- 172.21.15.86:0/1530407961 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7faca406c8f0 0x7faca406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:41.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.408+0000 7facc9000640 1 --2- 172.21.15.86:0/1530407961 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7faca406c8f0 0x7faca406edb0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7facb0036000 tx=0x7facb0030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:41.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.411+0000 7facc1600640 1 -- 172.21.15.86:0/1530407961 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7facb801f020 con 0x7facc40b45b0 2024-04-20T21:27:41.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.680+0000 7faccae00640 1 -- 172.21.15.86:0/1530407961 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7facc40cf330 con 0x7facc40b45b0 2024-04-20T21:27:41.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.681+0000 7facc1600640 1 -- 172.21.15.86:0/1530407961 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) ==== 80+0+5304 (secure 0 0 0) 0x7facb80172c0 con 0x7facc40b45b0 2024-04-20T21:27:41.680 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:27:41.680 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:27:41.680 INFO:teuthology.orchestra.run.smithi086.stdout: "auth_dump": [ 2024-04-20T21:27:41.680 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:27:41.680 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.a", 2024-04-20T21:27:41.680 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDAMCRmOpS/LRAAPZUUQNf6TyeJ2tF7YPnIKw==", 2024-04-20T21:27:41.681 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:27:41.681 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:27:41.681 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-20T21:27:41.681 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-20T21:27:41.681 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:27:41.681 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:27:41.681 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:27:41.681 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:27:41.681 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.b", 2024-04-20T21:27:41.681 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDBMCRmCJLpDRAA/JiLRSAlQRNMpDC+qDSr0g==", 2024-04-20T21:27:41.681 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:27:41.681 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:27:41.681 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-20T21:27:41.681 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-20T21:27:41.681 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:27:41.682 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:27:41.682 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:27:41.682 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:27:41.682 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.c", 2024-04-20T21:27:41.682 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDAMCRmPHOFOxAA4tUsw/n6oSKrNmzPD+NKyw==", 2024-04-20T21:27:41.682 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:27:41.682 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:27:41.682 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-20T21:27:41.682 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-20T21:27:41.682 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:27:41.682 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:27:41.682 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:27:41.683 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:27:41.683 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.d", 2024-04-20T21:27:41.683 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDBMCRmZnPXHBAA0jXwBmGehaO7qens4boRIQ==", 2024-04-20T21:27:41.683 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:27:41.683 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:27:41.683 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-20T21:27:41.683 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-20T21:27:41.683 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:27:41.683 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:27:41.683 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:27:41.683 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:27:41.683 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.0", 2024-04-20T21:27:41.683 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDFMCRm4w9UHhAA56y4dJyDASI+IoZv2XrEoQ==", 2024-04-20T21:27:41.683 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:27:41.684 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:27:41.684 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:27:41.684 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:27:41.684 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:27:41.684 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:27:41.684 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:27:41.684 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.1", 2024-04-20T21:27:41.684 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDIMCRmFo7xKRAAou5B04+J1gZtTzW/Mo+nRQ==", 2024-04-20T21:27:41.684 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:27:41.684 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:27:41.684 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:27:41.684 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:27:41.684 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:27:41.684 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:27:41.685 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:27:41.685 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.2", 2024-04-20T21:27:41.685 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDLMCRm7/jFNRAA4KcRsapPFLaKf0NaH5MApg==", 2024-04-20T21:27:41.685 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:27:41.685 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:27:41.685 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:27:41.685 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:27:41.685 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:27:41.685 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:27:41.685 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:27:41.685 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.3", 2024-04-20T21:27:41.685 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDPMCRmUTPMExAA0/dubb4ehsWIwZPS/ao68w==", 2024-04-20T21:27:41.685 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:27:41.685 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:27:41.686 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:27:41.686 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:27:41.686 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:27:41.686 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:27:41.686 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:27:41.686 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.4", 2024-04-20T21:27:41.687 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDVMCRmB4F2KRAA8Xj0DfH5Ou8TUrO8X/BQyw==", 2024-04-20T21:27:41.687 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:27:41.687 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:27:41.687 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:27:41.687 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:27:41.687 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:27:41.687 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:27:41.688 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:27:41.688 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.5", 2024-04-20T21:27:41.688 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDYMCRmSOvYNRAAODYXmRHMGPrCAxcHwlS7Ww==", 2024-04-20T21:27:41.688 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:27:41.688 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:27:41.688 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:27:41.689 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:27:41.689 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:27:41.689 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:27:41.689 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:27:41.689 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.6", 2024-04-20T21:27:41.689 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDcMCRmENJ9BBAAi/KBUhbYGNXEyiSjSHavWQ==", 2024-04-20T21:27:41.689 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:27:41.690 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:27:41.690 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:27:41.690 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:27:41.690 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:27:41.690 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:27:41.690 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:27:41.691 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.7", 2024-04-20T21:27:41.691 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDfMCRmfTHTDhAALL9GZBSGjWYrhgFw/YI6IA==", 2024-04-20T21:27:41.691 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:27:41.691 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:27:41.691 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:27:41.691 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:27:41.691 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:27:41.692 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:27:41.692 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:27:41.692 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.0", 2024-04-20T21:27:41.692 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDBMCRmHWxyKRAANzhRi4eajp68h/jWkhFMnw==", 2024-04-20T21:27:41.693 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:27:41.693 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:27:41.693 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow r", 2024-04-20T21:27:41.693 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-20T21:27:41.693 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:27:41.693 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:27:41.694 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:27:41.694 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.1", 2024-04-20T21:27:41.694 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDBMCRmgBJAOxAADJLzO97tjEpKn6kUs8g2QA==", 2024-04-20T21:27:41.694 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:27:41.694 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:27:41.694 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow r", 2024-04-20T21:27:41.695 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow rw", 2024-04-20T21:27:41.695 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rwx" 2024-04-20T21:27:41.695 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:27:41.695 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:27:41.695 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:27:41.695 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.2", 2024-04-20T21:27:41.695 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDBMCRmQAdJMhAAiUmKhPEeuV+UPqQ3gynG9w==", 2024-04-20T21:27:41.696 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:27:41.696 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:27:41.696 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow r", 2024-04-20T21:27:41.696 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow rw", 2024-04-20T21:27:41.696 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rwx" 2024-04-20T21:27:41.696 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:27:41.697 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:27:41.697 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:27:41.697 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.3", 2024-04-20T21:27:41.697 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDCMCRm8yMSCBAAFgSVW0XYu7fa40YxHITljQ==", 2024-04-20T21:27:41.697 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:27:41.697 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:27:41.698 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow r", 2024-04-20T21:27:41.698 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow rw", 2024-04-20T21:27:41.698 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rwx" 2024-04-20T21:27:41.698 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:27:41.698 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:27:41.698 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:27:41.698 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.admin", 2024-04-20T21:27:41.699 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQC/MCRmyru0KBAApSqv1hZRcLvrRwPMnqylqQ==", 2024-04-20T21:27:41.699 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:27:41.699 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-04-20T21:27:41.699 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-20T21:27:41.699 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-20T21:27:41.699 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:27:41.700 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:27:41.700 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:27:41.700 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:27:41.700 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mgr.x", 2024-04-20T21:27:41.700 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDAMCRmvTgjIxAACeSV8gNkL5LV3xzPS6VPeg==", 2024-04-20T21:27:41.700 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:27:41.700 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-04-20T21:27:41.701 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile mgr", 2024-04-20T21:27:41.701 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:27:41.701 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:27:41.701 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:27:41.701 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:27:41.701 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mgr.y", 2024-04-20T21:27:41.702 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDAMCRmSn3VGBAA0eu3R53f3gFgscWFMRkDzg==", 2024-04-20T21:27:41.702 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:27:41.702 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-04-20T21:27:41.702 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile mgr", 2024-04-20T21:27:41.702 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:27:41.702 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:27:41.703 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:27:41.703 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-20T21:27:41.703 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:27:41.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.684+0000 7faccae00640 1 -- 172.21.15.86:0/1530407961 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7faca406c8f0 msgr2=0x7faca406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:41.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.684+0000 7faccae00640 1 --2- 172.21.15.86:0/1530407961 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7faca406c8f0 0x7faca406edb0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7facb0036000 tx=0x7facb0030000 comp rx=0 tx=0).stop 2024-04-20T21:27:41.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.684+0000 7faccae00640 1 -- 172.21.15.86:0/1530407961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facc40b45b0 msgr2=0x7facc4155870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:41.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.684+0000 7faccae00640 1 --2- 172.21.15.86:0/1530407961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facc40b45b0 0x7facc4155870 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7facb800b3e0 tx=0x7facb800b8a0 comp rx=0 tx=0).stop 2024-04-20T21:27:41.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.684+0000 7faccae00640 1 -- 172.21.15.86:0/1530407961 shutdown_connections 2024-04-20T21:27:41.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.684+0000 7faccae00640 1 --2- 172.21.15.86:0/1530407961 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7faca406c8f0 0x7faca406edb0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:41.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.684+0000 7faccae00640 1 --2- 172.21.15.86:0/1530407961 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7facc415e870 0x7facc415d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:41.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.684+0000 7faccae00640 1 --2- 172.21.15.86:0/1530407961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facc40cf550 0x7facc40cd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:41.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.684+0000 7faccae00640 1 --2- 172.21.15.86:0/1530407961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7facc40b45b0 0x7facc4155870 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:41.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.684+0000 7faccae00640 1 -- 172.21.15.86:0/1530407961 >> 172.21.15.86:0/1530407961 conn(0x7facc409d880 msgr2=0x7facc40b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:41.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.684+0000 7faccae00640 1 -- 172.21.15.86:0/1530407961 shutdown_connections 2024-04-20T21:27:41.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.684+0000 7faccae00640 1 -- 172.21.15.86:0/1530407961 wait complete. 2024-04-20T21:27:41.708 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:27:41.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.908+0000 7f0a72600640 1 Processor -- start 2024-04-20T21:27:41.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.908+0000 7f0a72600640 1 -- start start 2024-04-20T21:27:41.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.910+0000 7f0a72600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a6c06e6d0 0x7f0a6c071d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:41.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.910+0000 7f0a72600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0a6c072240 0x7f0a6c15cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:41.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.910+0000 7f0a72600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a6c06eb30 con 0x7f0a6c146130 2024-04-20T21:27:41.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.910+0000 7f0a72600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f0a6c06dc30 con 0x7f0a6c072240 2024-04-20T21:27:41.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.910+0000 7f0a72600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0a6c0cd8e0 con 0x7f0a6c06e6d0 2024-04-20T21:27:41.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.910+0000 7f0a6be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0a6c072240 0x7f0a6c15cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:41.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.910+0000 7f0a6be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0a6c072240 0x7f0a6c15cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53218/0 (socket says 172.21.15.86:53218) 2024-04-20T21:27:41.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.910+0000 7f0a6be00640 1 -- 172.21.15.86:0/2173413816 learned_addr learned my addr 172.21.15.86:0/2173413816 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:41.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.910+0000 7f0a6aa00640 1 --2- 172.21.15.86:0/2173413816 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a6c06e6d0 0x7f0a6c071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:41.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.910+0000 7f0a6be00640 1 -- 172.21.15.86:0/2173413816 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a6c06e6d0 msgr2=0x7f0a6c071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:41.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.910+0000 7f0a6be00640 1 --2- 172.21.15.86:0/2173413816 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a6c06e6d0 0x7f0a6c071d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:41.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.910+0000 7f0a6be00640 1 -- 172.21.15.86:0/2173413816 >> v1:172.21.15.144:6789/0 conn(0x7f0a6c146130 legacy=0x7f0a6c0ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:41.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.910+0000 7f0a6be00640 1 -- 172.21.15.86:0/2173413816 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a6c0cdbc0 con 0x7f0a6c072240 2024-04-20T21:27:41.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.912+0000 7f0a6be00640 1 --2- 172.21.15.86:0/2173413816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a6c072240 0x7f0a6c15cc20 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f0a64009e50 tx=0x7f0a6402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b7697ba1fedafd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:41.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.912+0000 7f0a6a000640 1 -- 172.21.15.86:0/2173413816 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6402cb00 con 0x7f0a6c072240 2024-04-20T21:27:41.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.912+0000 7f0a6a000640 1 -- 172.21.15.86:0/2173413816 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a6402cca0 con 0x7f0a6c072240 2024-04-20T21:27:41.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.912+0000 7f0a6a000640 1 -- 172.21.15.86:0/2173413816 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a64034610 con 0x7f0a6c072240 2024-04-20T21:27:41.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.913+0000 7f0a72600640 1 -- 172.21.15.86:0/2173413816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a6c072240 msgr2=0x7f0a6c15cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:41.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.913+0000 7f0a72600640 1 --2- 172.21.15.86:0/2173413816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a6c072240 0x7f0a6c15cc20 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f0a64009e50 tx=0x7f0a6402bf60 comp rx=0 tx=0).stop 2024-04-20T21:27:41.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.913+0000 7f0a72600640 1 -- 172.21.15.86:0/2173413816 shutdown_connections 2024-04-20T21:27:41.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.913+0000 7f0a72600640 1 --2- 172.21.15.86:0/2173413816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a6c072240 0x7f0a6c15cc20 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:41.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.913+0000 7f0a72600640 1 --2- 172.21.15.86:0/2173413816 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a6c06e6d0 0x7f0a6c071d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:41.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.913+0000 7f0a72600640 1 -- 172.21.15.86:0/2173413816 >> 172.21.15.86:0/2173413816 conn(0x7f0a6c0b2360 msgr2=0x7f0a6c0b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:41.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.913+0000 7f0a72600640 1 -- 172.21.15.86:0/2173413816 shutdown_connections 2024-04-20T21:27:41.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.913+0000 7f0a72600640 1 -- 172.21.15.86:0/2173413816 wait complete. 2024-04-20T21:27:41.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.915+0000 7f0a72600640 1 Processor -- start 2024-04-20T21:27:41.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.915+0000 7f0a72600640 1 -- start start 2024-04-20T21:27:41.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.916+0000 7f0a72600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a6c06e6d0 0x7f0a6c0a9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:41.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.916+0000 7f0a72600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a6c072240 0x7f0a6c13f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:41.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.916+0000 7f0a72600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a6c146130 0x7f0a6c0cb730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:41.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.916+0000 7f0a72600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0a6c06f150 con 0x7f0a6c06e6d0 2024-04-20T21:27:41.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.916+0000 7f0a72600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0a6c06e1b0 con 0x7f0a6c146130 2024-04-20T21:27:41.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.916+0000 7f0a72600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0a6c0cd4e0 con 0x7f0a6c072240 2024-04-20T21:27:41.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.916+0000 7f0a6b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a6c06e6d0 0x7f0a6c0a9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:41.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.916+0000 7f0a6b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a6c06e6d0 0x7f0a6c0a9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53230/0 (socket says 172.21.15.86:53230) 2024-04-20T21:27:41.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.916+0000 7f0a6b400640 1 -- 172.21.15.86:0/3081640535 learned_addr learned my addr 172.21.15.86:0/3081640535 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:41.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.916+0000 7f0a6b400640 1 -- 172.21.15.86:0/3081640535 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a6c072240 msgr2=0x7f0a6c13f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:41.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.916+0000 7f0a6b400640 1 --2- 172.21.15.86:0/3081640535 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a6c072240 0x7f0a6c13f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:41.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.916+0000 7f0a6b400640 1 -- 172.21.15.86:0/3081640535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a6c146130 msgr2=0x7f0a6c0cb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:41.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.916+0000 7f0a6b400640 1 --2- 172.21.15.86:0/3081640535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a6c146130 0x7f0a6c0cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:41.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.916+0000 7f0a6b400640 1 -- 172.21.15.86:0/3081640535 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a6c13f590 con 0x7f0a6c06e6d0 2024-04-20T21:27:41.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.917+0000 7f0a6b400640 1 --2- 172.21.15.86:0/3081640535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a6c06e6d0 0x7f0a6c0a9e20 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c00c460 tx=0x7f0a5c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:41.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.917+0000 7f0a63400640 1 -- 172.21.15.86:0/3081640535 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c016020 con 0x7f0a6c06e6d0 2024-04-20T21:27:41.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.917+0000 7f0a63400640 1 -- 172.21.15.86:0/3081640535 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a5c012870 con 0x7f0a6c06e6d0 2024-04-20T21:27:41.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.917+0000 7f0a63400640 1 -- 172.21.15.86:0/3081640535 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c012c60 con 0x7f0a6c06e6d0 2024-04-20T21:27:41.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.917+0000 7f0a72600640 1 -- 172.21.15.86:0/3081640535 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a6c0aa430 con 0x7f0a6c06e6d0 2024-04-20T21:27:41.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.917+0000 7f0a72600640 1 -- 172.21.15.86:0/3081640535 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a6c0aa9f0 con 0x7f0a6c06e6d0 2024-04-20T21:27:41.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.918+0000 7f0a72600640 1 -- 172.21.15.86:0/3081640535 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a6c0689e0 con 0x7f0a6c06e6d0 2024-04-20T21:27:41.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.919+0000 7f0a63400640 1 -- 172.21.15.86:0/3081640535 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0a5c019020 con 0x7f0a6c06e6d0 2024-04-20T21:27:41.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.920+0000 7f0a63400640 1 --2- 172.21.15.86:0/3081640535 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0a4c06c8f0 0x7f0a4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:41.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.920+0000 7f0a63400640 1 -- 172.21.15.86:0/3081640535 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 4857+0+0 (secure 0 0 0) 0x7f0a5c08b500 con 0x7f0a6c06e6d0 2024-04-20T21:27:41.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.920+0000 7f0a6aa00640 1 --2- 172.21.15.86:0/3081640535 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0a4c06c8f0 0x7f0a4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:41.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.921+0000 7f0a6aa00640 1 --2- 172.21.15.86:0/3081640535 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0a4c06c8f0 0x7f0a4c06edb0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f0a58006fd0 tx=0x7f0a58008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:41.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:41.924+0000 7f0a63400640 1 -- 172.21.15.86:0/3081640535 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0a5c020030 con 0x7f0a6c06e6d0 2024-04-20T21:27:42.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.122+0000 7f0a72600640 1 -- 172.21.15.86:0/3081640535 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0a6c15ff50 con 0x7f0a6c06e6d0 2024-04-20T21:27:42.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.124+0000 7f0a63400640 1 -- 172.21.15.86:0/3081640535 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+12892 (secure 0 0 0) 0x7f0a5c0182c0 con 0x7f0a6c06e6d0 2024-04-20T21:27:42.123 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:27:42.123 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":96,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:27:41.196226+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:27:42.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.126+0000 7f0a72600640 1 -- 172.21.15.86:0/3081640535 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0a4c06c8f0 msgr2=0x7f0a4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:42.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.126+0000 7f0a72600640 1 --2- 172.21.15.86:0/3081640535 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0a4c06c8f0 0x7f0a4c06edb0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f0a58006fd0 tx=0x7f0a58008040 comp rx=0 tx=0).stop 2024-04-20T21:27:42.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.126+0000 7f0a72600640 1 -- 172.21.15.86:0/3081640535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a6c06e6d0 msgr2=0x7f0a6c0a9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:42.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.126+0000 7f0a72600640 1 --2- 172.21.15.86:0/3081640535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a6c06e6d0 0x7f0a6c0a9e20 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c00c460 tx=0x7f0a5c00c920 comp rx=0 tx=0).stop 2024-04-20T21:27:42.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.127+0000 7f0a72600640 1 -- 172.21.15.86:0/3081640535 shutdown_connections 2024-04-20T21:27:42.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.127+0000 7f0a72600640 1 --2- 172.21.15.86:0/3081640535 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0a4c06c8f0 0x7f0a4c06edb0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:42.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.127+0000 7f0a72600640 1 --2- 172.21.15.86:0/3081640535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a6c146130 0x7f0a6c0cb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:42.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.127+0000 7f0a72600640 1 --2- 172.21.15.86:0/3081640535 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a6c072240 0x7f0a6c13f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:42.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.127+0000 7f0a72600640 1 --2- 172.21.15.86:0/3081640535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a6c06e6d0 0x7f0a6c0a9e20 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:42.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.127+0000 7f0a72600640 1 -- 172.21.15.86:0/3081640535 >> 172.21.15.86:0/3081640535 conn(0x7f0a6c0b2360 msgr2=0x7f0a6c06c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:42.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.127+0000 7f0a72600640 1 -- 172.21.15.86:0/3081640535 shutdown_connections 2024-04-20T21:27:42.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.127+0000 7f0a72600640 1 -- 172.21.15.86:0/3081640535 wait complete. 2024-04-20T21:27:42.144 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:27:42.144 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:27:42.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.344+0000 7f1595800640 1 Processor -- start 2024-04-20T21:27:42.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.345+0000 7f1595800640 1 -- start start 2024-04-20T21:27:42.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.346+0000 7f1595800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1590145fd0 0x7f159006b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:42.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.346+0000 7f1595800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f159015f1d0 0x7f159006bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:42.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.346+0000 7f1595800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15900b39b0 con 0x7f159009a9e0 2024-04-20T21:27:42.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.346+0000 7f1595800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f159015dd30 con 0x7f159015f1d0 2024-04-20T21:27:42.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.346+0000 7f1595800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f15900b3c90 con 0x7f1590145fd0 2024-04-20T21:27:42.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.347+0000 7f158fe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1590145fd0 0x7f159006b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:42.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.347+0000 7f158fe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1590145fd0 0x7f159006b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:39502/0 (socket says 172.21.15.86:39502) 2024-04-20T21:27:42.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.347+0000 7f158fe00640 1 -- 172.21.15.86:0/1912012729 learned_addr learned my addr 172.21.15.86:0/1912012729 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:42.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.347+0000 7f1594e00640 1 --2- 172.21.15.86:0/1912012729 >> v2:172.21.15.144:3300/0 conn(0x7f159015f1d0 0x7f159006bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:42.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.348+0000 7f158fe00640 1 -- 172.21.15.86:0/1912012729 >> v2:172.21.15.144:3300/0 conn(0x7f159015f1d0 msgr2=0x7f159006bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:42.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.348+0000 7f158fe00640 1 --2- 172.21.15.86:0/1912012729 >> v2:172.21.15.144:3300/0 conn(0x7f159015f1d0 0x7f159006bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:42.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.348+0000 7f158fe00640 1 -- 172.21.15.86:0/1912012729 >> v1:172.21.15.144:6789/0 conn(0x7f159009a9e0 legacy=0x7f1590160310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:42.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.348+0000 7f158fe00640 1 -- 172.21.15.86:0/1912012729 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f159015d9c0 con 0x7f1590145fd0 2024-04-20T21:27:42.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.349+0000 7f158fe00640 1 --2- 172.21.15.86:0/1912012729 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1590145fd0 0x7f159006b980 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1588009e50 tx=0x7f158802bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=402130a9057fd43b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:42.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.350+0000 7f158ea00640 1 -- 172.21.15.86:0/1912012729 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158802ca70 con 0x7f1590145fd0 2024-04-20T21:27:42.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.350+0000 7f158ea00640 1 -- 172.21.15.86:0/1912012729 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f158802cc10 con 0x7f1590145fd0 2024-04-20T21:27:42.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.350+0000 7f158ea00640 1 -- 172.21.15.86:0/1912012729 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15880343f0 con 0x7f1590145fd0 2024-04-20T21:27:42.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.351+0000 7f1595800640 1 -- 172.21.15.86:0/1912012729 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1590145fd0 msgr2=0x7f159006b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:42.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.351+0000 7f1595800640 1 --2- 172.21.15.86:0/1912012729 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1590145fd0 0x7f159006b980 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1588009e50 tx=0x7f158802bed0 comp rx=0 tx=0).stop 2024-04-20T21:27:42.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.351+0000 7f1595800640 1 -- 172.21.15.86:0/1912012729 shutdown_connections 2024-04-20T21:27:42.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.351+0000 7f1595800640 1 --2- 172.21.15.86:0/1912012729 >> v2:172.21.15.144:3300/0 conn(0x7f159015f1d0 0x7f159006bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:42.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.351+0000 7f1595800640 1 --2- 172.21.15.86:0/1912012729 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1590145fd0 0x7f159006b980 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:42.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.351+0000 7f1595800640 1 -- 172.21.15.86:0/1912012729 >> 172.21.15.86:0/1912012729 conn(0x7f1590073220 msgr2=0x7f15900b2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:42.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.351+0000 7f1595800640 1 -- 172.21.15.86:0/1912012729 shutdown_connections 2024-04-20T21:27:42.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.351+0000 7f1595800640 1 -- 172.21.15.86:0/1912012729 wait complete. 2024-04-20T21:27:42.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.353+0000 7f1595800640 1 Processor -- start 2024-04-20T21:27:42.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.353+0000 7f1595800640 1 -- start start 2024-04-20T21:27:42.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.354+0000 7f1595800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f159009a9e0 0x7f159015cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:42.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.354+0000 7f1595800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1590145fd0 0x7f15900cb050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:42.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.354+0000 7f1595800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f159015f1d0 0x7f15900d15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:42.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.354+0000 7f1595800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f15900b6780 con 0x7f159009a9e0 2024-04-20T21:27:42.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.354+0000 7f1595800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f15900b6600 con 0x7f1590145fd0 2024-04-20T21:27:42.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.354+0000 7f1595800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f15900ae2d0 con 0x7f159015f1d0 2024-04-20T21:27:42.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.354+0000 7f158fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f159009a9e0 0x7f159015cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:42.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.354+0000 7f158fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f159009a9e0 0x7f159015cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53234/0 (socket says 172.21.15.86:53234) 2024-04-20T21:27:42.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.354+0000 7f158fe00640 1 -- 172.21.15.86:0/3347014852 learned_addr learned my addr 172.21.15.86:0/3347014852 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:42.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.355+0000 7f158f400640 1 --2- 172.21.15.86:0/3347014852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1590145fd0 0x7f15900cb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:42.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.355+0000 7f1594e00640 1 --2- 172.21.15.86:0/3347014852 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f159015f1d0 0x7f15900d15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:42.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.355+0000 7f158fe00640 1 -- 172.21.15.86:0/3347014852 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f159015f1d0 msgr2=0x7f15900d15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:42.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.355+0000 7f158fe00640 1 --2- 172.21.15.86:0/3347014852 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f159015f1d0 0x7f15900d15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:42.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.355+0000 7f158fe00640 1 -- 172.21.15.86:0/3347014852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1590145fd0 msgr2=0x7f15900cb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:42.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.355+0000 7f158fe00640 1 --2- 172.21.15.86:0/3347014852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1590145fd0 0x7f15900cb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:42.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.355+0000 7f158fe00640 1 -- 172.21.15.86:0/3347014852 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15900cb9f0 con 0x7f159009a9e0 2024-04-20T21:27:42.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.355+0000 7f158fe00640 1 --2- 172.21.15.86:0/3347014852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f159009a9e0 0x7f159015cfa0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f1588009e50 tx=0x7f1588004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:42.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.356+0000 7f1587e00640 1 -- 172.21.15.86:0/3347014852 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1588043020 con 0x7f159009a9e0 2024-04-20T21:27:42.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.356+0000 7f1587e00640 1 -- 172.21.15.86:0/3347014852 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f158803e450 con 0x7f159009a9e0 2024-04-20T21:27:42.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.356+0000 7f1587e00640 1 -- 172.21.15.86:0/3347014852 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1588009510 con 0x7f159009a9e0 2024-04-20T21:27:42.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.356+0000 7f1595800640 1 -- 172.21.15.86:0/3347014852 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15900cbcd0 con 0x7f159009a9e0 2024-04-20T21:27:42.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.356+0000 7f1595800640 1 -- 172.21.15.86:0/3347014852 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15900cc290 con 0x7f159009a9e0 2024-04-20T21:27:42.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.357+0000 7f1595800640 1 -- 172.21.15.86:0/3347014852 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f159004fd60 con 0x7f159009a9e0 2024-04-20T21:27:42.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.358+0000 7f1587e00640 1 -- 172.21.15.86:0/3347014852 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1588046020 con 0x7f159009a9e0 2024-04-20T21:27:42.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.359+0000 7f1587e00640 1 --2- 172.21.15.86:0/3347014852 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f157006c8f0 0x7f157006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:42.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.359+0000 7f1587e00640 1 -- 172.21.15.86:0/3347014852 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 4857+0+0 (secure 0 0 0) 0x7f15880ae0c0 con 0x7f159009a9e0 2024-04-20T21:27:42.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.359+0000 7f158f400640 1 --2- 172.21.15.86:0/3347014852 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f157006c8f0 0x7f157006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:42.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.360+0000 7f158f400640 1 --2- 172.21.15.86:0/3347014852 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f157006c8f0 0x7f157006edb0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f157c004ea0 tx=0x7f157c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:42.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.364+0000 7f1587e00640 1 -- 172.21.15.86:0/3347014852 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f158804d030 con 0x7f159009a9e0 2024-04-20T21:27:42.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.568+0000 7f1595800640 1 -- 172.21.15.86:0/3347014852 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1590144020 con 0x7f159009a9e0 2024-04-20T21:27:42.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.569+0000 7f1587e00640 1 -- 172.21.15.86:0/3347014852 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) ==== 93+0+10 (secure 0 0 0) 0x7f158807c7b0 con 0x7f159009a9e0 2024-04-20T21:27:42.568 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:27:42.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.571+0000 7f1595800640 1 -- 172.21.15.86:0/3347014852 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f157006c8f0 msgr2=0x7f157006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:42.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.571+0000 7f1595800640 1 --2- 172.21.15.86:0/3347014852 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f157006c8f0 0x7f157006edb0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f157c004ea0 tx=0x7f157c009040 comp rx=0 tx=0).stop 2024-04-20T21:27:42.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.572+0000 7f1595800640 1 -- 172.21.15.86:0/3347014852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f159009a9e0 msgr2=0x7f159015cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:42.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.572+0000 7f1595800640 1 --2- 172.21.15.86:0/3347014852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f159009a9e0 0x7f159015cfa0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f1588009e50 tx=0x7f1588004a40 comp rx=0 tx=0).stop 2024-04-20T21:27:42.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.572+0000 7f1595800640 1 -- 172.21.15.86:0/3347014852 shutdown_connections 2024-04-20T21:27:42.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.572+0000 7f1595800640 1 --2- 172.21.15.86:0/3347014852 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f157006c8f0 0x7f157006edb0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:42.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.572+0000 7f1595800640 1 --2- 172.21.15.86:0/3347014852 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f159015f1d0 0x7f15900d15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:42.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.572+0000 7f1595800640 1 --2- 172.21.15.86:0/3347014852 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1590145fd0 0x7f15900cb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:42.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.572+0000 7f1595800640 1 --2- 172.21.15.86:0/3347014852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f159009a9e0 0x7f159015cfa0 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:42.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.572+0000 7f1595800640 1 -- 172.21.15.86:0/3347014852 >> 172.21.15.86:0/3347014852 conn(0x7f1590073220 msgr2=0x7f1590144540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:42.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.572+0000 7f1595800640 1 -- 172.21.15.86:0/3347014852 shutdown_connections 2024-04-20T21:27:42.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.572+0000 7f1595800640 1 -- 172.21.15.86:0/3347014852 wait complete. 2024-04-20T21:27:42.589 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-20T21:27:42.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.791+0000 7f6240200640 1 Processor -- start 2024-04-20T21:27:42.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.792+0000 7f6240200640 1 -- start start 2024-04-20T21:27:42.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.793+0000 7f6240200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6238146130 0x7f623806e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:42.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.793+0000 7f6240200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6238072410 0x7f62380cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:42.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.793+0000 7f6240200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f623806dc30 con 0x7f62380ccdd0 2024-04-20T21:27:42.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.793+0000 7f6240200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f62380cd8e0 con 0x7f6238072410 2024-04-20T21:27:42.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.793+0000 7f6240200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f62380cdbc0 con 0x7f6238146130 2024-04-20T21:27:42.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.794+0000 7f623e200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6238072410 0x7f62380cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:42.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.794+0000 7f623e200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6238072410 0x7f62380cd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53238/0 (socket says 172.21.15.86:53238) 2024-04-20T21:27:42.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.794+0000 7f623e200640 1 -- 172.21.15.86:0/146650135 learned_addr learned my addr 172.21.15.86:0/146650135 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:42.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.794+0000 7f623e200640 1 -- 172.21.15.86:0/146650135 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6238146130 msgr2=0x7f623806e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:42.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.794+0000 7f623d800640 1 --2- 172.21.15.86:0/146650135 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6238146130 0x7f623806e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:42.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.794+0000 7f623e200640 1 --2- 172.21.15.86:0/146650135 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6238146130 0x7f623806e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:42.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.794+0000 7f623e200640 1 -- 172.21.15.86:0/146650135 >> v1:172.21.15.144:6789/0 conn(0x7f62380ccdd0 legacy=0x7f6238071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:42.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.794+0000 7f623e200640 1 -- 172.21.15.86:0/146650135 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f623806ded0 con 0x7f6238072410 2024-04-20T21:27:42.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.795+0000 7f623e200640 1 --2- 172.21.15.86:0/146650135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6238072410 0x7f62380cd230 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f6230009870 tx=0x7f623002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=632e89017b910cea server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:42.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.796+0000 7f6237e00640 1 -- 172.21.15.86:0/146650135 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6230034070 con 0x7f6238072410 2024-04-20T21:27:42.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.796+0000 7f6237e00640 1 -- 172.21.15.86:0/146650135 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6230030840 con 0x7f6238072410 2024-04-20T21:27:42.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.796+0000 7f6237e00640 1 -- 172.21.15.86:0/146650135 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6230030c10 con 0x7f6238072410 2024-04-20T21:27:42.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.796+0000 7f6240200640 1 -- 172.21.15.86:0/146650135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6238072410 msgr2=0x7f62380cd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:42.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.796+0000 7f6240200640 1 --2- 172.21.15.86:0/146650135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6238072410 0x7f62380cd230 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f6230009870 tx=0x7f623002b980 comp rx=0 tx=0).stop 2024-04-20T21:27:42.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.796+0000 7f6240200640 1 -- 172.21.15.86:0/146650135 shutdown_connections 2024-04-20T21:27:42.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.796+0000 7f6240200640 1 --2- 172.21.15.86:0/146650135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6238072410 0x7f62380cd230 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:42.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.796+0000 7f6240200640 1 --2- 172.21.15.86:0/146650135 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6238146130 0x7f623806e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:42.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.796+0000 7f6240200640 1 -- 172.21.15.86:0/146650135 >> 172.21.15.86:0/146650135 conn(0x7f62380b2360 msgr2=0x7f62380b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:42.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.796+0000 7f6240200640 1 -- 172.21.15.86:0/146650135 shutdown_connections 2024-04-20T21:27:42.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.796+0000 7f6240200640 1 -- 172.21.15.86:0/146650135 wait complete. 2024-04-20T21:27:42.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.798+0000 7f6240200640 1 Processor -- start 2024-04-20T21:27:42.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.798+0000 7f6240200640 1 -- start start 2024-04-20T21:27:42.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.799+0000 7f6240200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6238072410 0x7f623815c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:42.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.799+0000 7f6240200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62380ccdd0 0x7f6238140730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:42.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.799+0000 7f6240200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6238146130 0x7f6238140c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:42.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.799+0000 7f6240200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f623815cc20 con 0x7f6238146130 2024-04-20T21:27:42.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.799+0000 7f6240200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f623806f150 con 0x7f62380ccdd0 2024-04-20T21:27:42.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.799+0000 7f6240200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f623815cda0 con 0x7f6238072410 2024-04-20T21:27:42.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.799+0000 7f623e200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6238146130 0x7f6238140c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:42.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.799+0000 7f623e200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6238146130 0x7f6238140c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53254/0 (socket says 172.21.15.86:53254) 2024-04-20T21:27:42.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.799+0000 7f623e200640 1 -- 172.21.15.86:0/1516493300 learned_addr learned my addr 172.21.15.86:0/1516493300 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:42.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.800+0000 7f623d800640 1 --2- 172.21.15.86:0/1516493300 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6238072410 0x7f623815c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:42.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.800+0000 7f623ce00640 1 --2- 172.21.15.86:0/1516493300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62380ccdd0 0x7f6238140730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:42.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.800+0000 7f623e200640 1 -- 172.21.15.86:0/1516493300 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6238072410 msgr2=0x7f623815c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:42.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.800+0000 7f623e200640 1 --2- 172.21.15.86:0/1516493300 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6238072410 0x7f623815c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:42.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.800+0000 7f623e200640 1 -- 172.21.15.86:0/1516493300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62380ccdd0 msgr2=0x7f6238140730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:42.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.800+0000 7f623e200640 1 --2- 172.21.15.86:0/1516493300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62380ccdd0 0x7f6238140730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:42.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.800+0000 7f623e200640 1 -- 172.21.15.86:0/1516493300 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62380cba30 con 0x7f6238146130 2024-04-20T21:27:42.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.800+0000 7f623ce00640 1 --2- 172.21.15.86:0/1516493300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62380ccdd0 0x7f6238140730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:27:42.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.800+0000 7f623e200640 1 --2- 172.21.15.86:0/1516493300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6238146130 0x7f6238140c70 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f623002b9c0 tx=0x7f6230004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:42.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.800+0000 7f6235600640 1 -- 172.21.15.86:0/1516493300 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6230048070 con 0x7f6238146130 2024-04-20T21:27:42.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.800+0000 7f6235600640 1 -- 172.21.15.86:0/1516493300 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6230030640 con 0x7f6238146130 2024-04-20T21:27:42.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.800+0000 7f6235600640 1 -- 172.21.15.86:0/1516493300 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6230034040 con 0x7f6238146130 2024-04-20T21:27:42.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.801+0000 7f6240200640 1 -- 172.21.15.86:0/1516493300 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62380cbd10 con 0x7f6238146130 2024-04-20T21:27:42.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.801+0000 7f6240200640 1 -- 172.21.15.86:0/1516493300 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62380cc2d0 con 0x7f6238146130 2024-04-20T21:27:42.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.802+0000 7f6240200640 1 -- 172.21.15.86:0/1516493300 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62380689e0 con 0x7f6238146130 2024-04-20T21:27:42.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.803+0000 7f6235600640 1 -- 172.21.15.86:0/1516493300 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6230037070 con 0x7f6238146130 2024-04-20T21:27:42.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.803+0000 7f6235600640 1 --2- 172.21.15.86:0/1516493300 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f621806c920 0x7f621806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:42.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.804+0000 7f6235600640 1 -- 172.21.15.86:0/1516493300 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 4857+0+0 (secure 0 0 0) 0x7f62300ad880 con 0x7f6238146130 2024-04-20T21:27:42.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.804+0000 7f623d800640 1 --2- 172.21.15.86:0/1516493300 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f621806c920 0x7f621806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:42.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.804+0000 7f623d800640 1 --2- 172.21.15.86:0/1516493300 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f621806c920 0x7f621806ede0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f6228006fd0 tx=0x7f6228008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:42.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:42.808+0000 7f6235600640 1 -- 172.21.15.86:0/1516493300 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f623007bff0 con 0x7f6238146130 2024-04-20T21:27:43.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.011+0000 7f6240200640 1 -- 172.21.15.86:0/1516493300 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f623815ff50 con 0x7f6238146130 2024-04-20T21:27:43.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.012+0000 7f6235600640 1 -- 172.21.15.86:0/1516493300 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v96) ==== 84+0+65 (secure 0 0 0) 0x7f623007b8f0 con 0x7f6238146130 2024-04-20T21:27:43.011 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:27:43.011 INFO:teuthology.orchestra.run.smithi086.stdout:[ 2024-04-20T21:27:43.011 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:27:43.012 INFO:teuthology.orchestra.run.smithi086.stdout: "poolnum": 1, 2024-04-20T21:27:43.012 INFO:teuthology.orchestra.run.smithi086.stdout: "poolname": ".mgr" 2024-04-20T21:27:43.012 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:27:43.012 INFO:teuthology.orchestra.run.smithi086.stdout:] 2024-04-20T21:27:43.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.015+0000 7f6240200640 1 -- 172.21.15.86:0/1516493300 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f621806c920 msgr2=0x7f621806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:43.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.015+0000 7f6240200640 1 --2- 172.21.15.86:0/1516493300 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f621806c920 0x7f621806ede0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f6228006fd0 tx=0x7f6228008040 comp rx=0 tx=0).stop 2024-04-20T21:27:43.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.015+0000 7f6240200640 1 -- 172.21.15.86:0/1516493300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6238146130 msgr2=0x7f6238140c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:43.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.015+0000 7f6240200640 1 --2- 172.21.15.86:0/1516493300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6238146130 0x7f6238140c70 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f623002b9c0 tx=0x7f6230004730 comp rx=0 tx=0).stop 2024-04-20T21:27:43.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.015+0000 7f6240200640 1 -- 172.21.15.86:0/1516493300 shutdown_connections 2024-04-20T21:27:43.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.015+0000 7f6240200640 1 --2- 172.21.15.86:0/1516493300 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f621806c920 0x7f621806ede0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:43.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.015+0000 7f6240200640 1 --2- 172.21.15.86:0/1516493300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6238146130 0x7f6238140c70 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:43.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.015+0000 7f6240200640 1 --2- 172.21.15.86:0/1516493300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62380ccdd0 0x7f6238140730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:43.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.015+0000 7f6240200640 1 --2- 172.21.15.86:0/1516493300 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6238072410 0x7f623815c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:43.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.015+0000 7f6240200640 1 -- 172.21.15.86:0/1516493300 >> 172.21.15.86:0/1516493300 conn(0x7f62380b2360 msgr2=0x7f6238143720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:43.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.015+0000 7f6240200640 1 -- 172.21.15.86:0/1516493300 shutdown_connections 2024-04-20T21:27:43.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.015+0000 7f6240200640 1 -- 172.21.15.86:0/1516493300 wait complete. 2024-04-20T21:27:43.032 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-20T21:27:43.032 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-20T21:27:43.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.234+0000 7f4eebc00640 1 Processor -- start 2024-04-20T21:27:43.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.234+0000 7f4eebc00640 1 -- start start 2024-04-20T21:27:43.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.236+0000 7f4eebc00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4ee4071d00 0x7f4ee4072190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:43.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.236+0000 7f4eebc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4ee415ff10 0x7f4ee4160380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:43.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.236+0000 7f4eebc00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ee406ed90 con 0x7f4ee4148150 2024-04-20T21:27:43.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.236+0000 7f4eebc00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f4ee40aec00 con 0x7f4ee415ff10 2024-04-20T21:27:43.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.236+0000 7f4eebc00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f4ee40aee30 con 0x7f4ee4071d00 2024-04-20T21:27:43.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.236+0000 7f4ee9c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4ee415ff10 0x7f4ee4160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:43.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.236+0000 7f4ee9c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4ee415ff10 0x7f4ee4160380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53258/0 (socket says 172.21.15.86:53258) 2024-04-20T21:27:43.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.236+0000 7f4ee9c00640 1 -- 172.21.15.86:0/4287539954 learned_addr learned my addr 172.21.15.86:0/4287539954 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:43.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.236+0000 7f4ee9200640 1 --1- 172.21.15.86:0/4287539954 >> v1:172.21.15.86:6789/0 conn(0x7f4ee4148150 0x7f4ee40af8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54272/0 (socket says 172.21.15.86:54272) 2024-04-20T21:27:43.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.237+0000 7f4edbe00640 1 --2- 172.21.15.86:0/4287539954 >> v2:172.21.15.144:3300/0 conn(0x7f4ee4071d00 0x7f4ee4072190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:43.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.237+0000 7f4ee9c00640 1 -- 172.21.15.86:0/4287539954 >> v2:172.21.15.144:3300/0 conn(0x7f4ee4071d00 msgr2=0x7f4ee4072190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:43.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.237+0000 7f4ee9c00640 1 --2- 172.21.15.86:0/4287539954 >> v2:172.21.15.144:3300/0 conn(0x7f4ee4071d00 0x7f4ee4072190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:43.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.237+0000 7f4ee9c00640 1 -- 172.21.15.86:0/4287539954 >> v1:172.21.15.86:6789/0 conn(0x7f4ee4148150 legacy=0x7f4ee40af8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:43.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.237+0000 7f4ee9c00640 1 -- 172.21.15.86:0/4287539954 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee406eab0 con 0x7f4ee415ff10 2024-04-20T21:27:43.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.238+0000 7f4ee9c00640 1 --2- 172.21.15.86:0/4287539954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ee415ff10 0x7f4ee4160380 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f4ee406c9b0 tx=0x7f4edc02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e9fcd1560240c40 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:43.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.238+0000 7f4ee3e00640 1 -- 172.21.15.86:0/4287539954 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4edc02cb20 con 0x7f4ee415ff10 2024-04-20T21:27:43.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.238+0000 7f4ee3e00640 1 -- 172.21.15.86:0/4287539954 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4edc02ccc0 con 0x7f4ee415ff10 2024-04-20T21:27:43.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.238+0000 7f4ee3e00640 1 -- 172.21.15.86:0/4287539954 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4edc034610 con 0x7f4ee415ff10 2024-04-20T21:27:43.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.239+0000 7f4eebc00640 1 -- 172.21.15.86:0/4287539954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ee415ff10 msgr2=0x7f4ee4160380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:43.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.239+0000 7f4eebc00640 1 --2- 172.21.15.86:0/4287539954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ee415ff10 0x7f4ee4160380 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f4ee406c9b0 tx=0x7f4edc02bf80 comp rx=0 tx=0).stop 2024-04-20T21:27:43.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.239+0000 7f4eebc00640 1 -- 172.21.15.86:0/4287539954 shutdown_connections 2024-04-20T21:27:43.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.239+0000 7f4eebc00640 1 --2- 172.21.15.86:0/4287539954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ee415ff10 0x7f4ee4160380 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:43.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.239+0000 7f4eebc00640 1 --2- 172.21.15.86:0/4287539954 >> v2:172.21.15.144:3300/0 conn(0x7f4ee4071d00 0x7f4ee4072190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:43.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.239+0000 7f4eebc00640 1 -- 172.21.15.86:0/4287539954 >> 172.21.15.86:0/4287539954 conn(0x7f4ee40b2180 msgr2=0x7f4ee40b2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:43.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.239+0000 7f4eebc00640 1 -- 172.21.15.86:0/4287539954 shutdown_connections 2024-04-20T21:27:43.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.239+0000 7f4eebc00640 1 -- 172.21.15.86:0/4287539954 wait complete. 2024-04-20T21:27:43.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.241+0000 7f4eebc00640 1 Processor -- start 2024-04-20T21:27:43.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.241+0000 7f4eebc00640 1 -- start start 2024-04-20T21:27:43.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.243+0000 7f4eebc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ee4071d00 0x7f4ee4143670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:43.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.243+0000 7f4eebc00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ee4148150 0x7f4ee40afe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:43.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.243+0000 7f4eebc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ee415ff10 0x7f4ee40cf050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:43.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.243+0000 7f4eebc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee41608c0 con 0x7f4ee4071d00 2024-04-20T21:27:43.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.243+0000 7f4eebc00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee40af3b0 con 0x7f4ee4148150 2024-04-20T21:27:43.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.243+0000 7f4eebc00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f4ee406dd80 con 0x7f4ee415ff10 2024-04-20T21:27:43.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.243+0000 7f4ee9200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ee4071d00 0x7f4ee4143670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:43.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.243+0000 7f4ee9200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ee4071d00 0x7f4ee4143670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53266/0 (socket says 172.21.15.86:53266) 2024-04-20T21:27:43.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.243+0000 7f4ee9200640 1 -- 172.21.15.86:0/1278548509 learned_addr learned my addr 172.21.15.86:0/1278548509 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:43.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.243+0000 7f4ee9c00640 1 --2- 172.21.15.86:0/1278548509 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ee415ff10 0x7f4ee40cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:43.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.243+0000 7f4edbe00640 1 --2- 172.21.15.86:0/1278548509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ee4148150 0x7f4ee40afe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:43.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.243+0000 7f4ee9200640 1 -- 172.21.15.86:0/1278548509 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ee415ff10 msgr2=0x7f4ee40cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:43.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.243+0000 7f4ee9200640 1 --2- 172.21.15.86:0/1278548509 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ee415ff10 0x7f4ee40cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:43.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.243+0000 7f4ee9200640 1 -- 172.21.15.86:0/1278548509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ee4148150 msgr2=0x7f4ee40afe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:43.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.243+0000 7f4ee9200640 1 --2- 172.21.15.86:0/1278548509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ee4148150 0x7f4ee40afe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:43.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.243+0000 7f4ee9200640 1 -- 172.21.15.86:0/1278548509 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee40ce3f0 con 0x7f4ee4071d00 2024-04-20T21:27:43.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.243+0000 7f4ee9c00640 1 --2- 172.21.15.86:0/1278548509 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ee415ff10 0x7f4ee40cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:27:43.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.243+0000 7f4edbe00640 1 --2- 172.21.15.86:0/1278548509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ee4148150 0x7f4ee40afe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:27:43.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.244+0000 7f4ee9200640 1 --2- 172.21.15.86:0/1278548509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ee4071d00 0x7f4ee4143670 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f4ed400c430 tx=0x7f4ed400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:43.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.244+0000 7f4ee1600640 1 -- 172.21.15.86:0/1278548509 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed40074e0 con 0x7f4ee4071d00 2024-04-20T21:27:43.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.244+0000 7f4ee1600640 1 -- 172.21.15.86:0/1278548509 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ed4007680 con 0x7f4ee4071d00 2024-04-20T21:27:43.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.244+0000 7f4eebc00640 1 -- 172.21.15.86:0/1278548509 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ee40ce6d0 con 0x7f4ee4071d00 2024-04-20T21:27:43.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.244+0000 7f4ee1600640 1 -- 172.21.15.86:0/1278548509 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4004a00 con 0x7f4ee4071d00 2024-04-20T21:27:43.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.244+0000 7f4eebc00640 1 -- 172.21.15.86:0/1278548509 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ee40af970 con 0x7f4ee4071d00 2024-04-20T21:27:43.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.245+0000 7f4eebc00640 1 -- 172.21.15.86:0/1278548509 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ee4158a90 con 0x7f4ee4071d00 2024-04-20T21:27:43.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.246+0000 7f4ee1600640 1 -- 172.21.15.86:0/1278548509 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4ed401a020 con 0x7f4ee4071d00 2024-04-20T21:27:43.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.247+0000 7f4ee1600640 1 --2- 172.21.15.86:0/1278548509 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4ec006c8f0 0x7f4ec006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:43.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.247+0000 7f4ee1600640 1 -- 172.21.15.86:0/1278548509 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 4857+0+0 (secure 0 0 0) 0x7f4ed408afe0 con 0x7f4ee4071d00 2024-04-20T21:27:43.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.247+0000 7f4edbe00640 1 --2- 172.21.15.86:0/1278548509 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4ec006c8f0 0x7f4ec006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:43.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.248+0000 7f4edbe00640 1 --2- 172.21.15.86:0/1278548509 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4ec006c8f0 0x7f4ec006edb0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f4ee415f9f0 tx=0x7f4ecc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:43.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.252+0000 7f4ee1600640 1 -- 172.21.15.86:0/1278548509 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4ee4158a90 con 0x7f4ee4071d00 2024-04-20T21:27:43.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:43.454+0000 7f4eebc00640 1 -- 172.21.15.86:0/1278548509 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f4ee40689e0 con 0x7f4ee4071d00 2024-04-20T21:27:44.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.212+0000 7f4ee1600640 1 -- 172.21.15.86:0/1278548509 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v97) ==== 138+0+0 (secure 0 0 0) 0x7f4ed40597b0 con 0x7f4ee4071d00 2024-04-20T21:27:44.211 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' created 2024-04-20T21:27:44.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.215+0000 7f4eebc00640 1 -- 172.21.15.86:0/1278548509 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4ec006c8f0 msgr2=0x7f4ec006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:44.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.215+0000 7f4eebc00640 1 --2- 172.21.15.86:0/1278548509 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4ec006c8f0 0x7f4ec006edb0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f4ee415f9f0 tx=0x7f4ecc008040 comp rx=0 tx=0).stop 2024-04-20T21:27:44.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.215+0000 7f4eebc00640 1 -- 172.21.15.86:0/1278548509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ee4071d00 msgr2=0x7f4ee4143670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:44.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.215+0000 7f4eebc00640 1 --2- 172.21.15.86:0/1278548509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ee4071d00 0x7f4ee4143670 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f4ed400c430 tx=0x7f4ed400c8f0 comp rx=0 tx=0).stop 2024-04-20T21:27:44.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.216+0000 7f4eebc00640 1 -- 172.21.15.86:0/1278548509 shutdown_connections 2024-04-20T21:27:44.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.216+0000 7f4eebc00640 1 --2- 172.21.15.86:0/1278548509 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4ec006c8f0 0x7f4ec006edb0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:44.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.216+0000 7f4eebc00640 1 --2- 172.21.15.86:0/1278548509 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4ee415ff10 0x7f4ee40cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:44.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.216+0000 7f4eebc00640 1 --2- 172.21.15.86:0/1278548509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ee4148150 0x7f4ee40afe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:44.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.216+0000 7f4eebc00640 1 --2- 172.21.15.86:0/1278548509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ee4071d00 0x7f4ee4143670 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:44.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.216+0000 7f4eebc00640 1 -- 172.21.15.86:0/1278548509 >> 172.21.15.86:0/1278548509 conn(0x7f4ee40b2180 msgr2=0x7f4ee415e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:44.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.216+0000 7f4eebc00640 1 -- 172.21.15.86:0/1278548509 shutdown_connections 2024-04-20T21:27:44.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.216+0000 7f4eebc00640 1 -- 172.21.15.86:0/1278548509 wait complete. 2024-04-20T21:27:44.247 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-20T21:27:44.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.441+0000 7f065f200640 1 Processor -- start 2024-04-20T21:27:44.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.441+0000 7f065f200640 1 -- start start 2024-04-20T21:27:44.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.443+0000 7f065f200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0660145250 0x7f066006efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:44.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.443+0000 7f065f200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0660145f00 0x7f0660071d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:44.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.443+0000 7f065f200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0660072280 0x7f066006dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:44.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.443+0000 7f065f200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f066006f500 con 0x7f0660145250 2024-04-20T21:27:44.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.443+0000 7f065f200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f0660160410 con 0x7f0660072280 2024-04-20T21:27:44.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.443+0000 7f065f200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f06601606f0 con 0x7f0660145f00 2024-04-20T21:27:44.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.443+0000 7f065de00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0660145250 0x7f066006efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:44.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.443+0000 7f065de00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0660145250 0x7f066006efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53282/0 (socket says 172.21.15.86:53282) 2024-04-20T21:27:44.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.444+0000 7f065de00640 1 -- 172.21.15.86:0/492018032 learned_addr learned my addr 172.21.15.86:0/492018032 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:44.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.444+0000 7f065e800640 1 --2- 172.21.15.86:0/492018032 >> v2:172.21.15.144:3300/0 conn(0x7f0660072280 0x7f066006dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:44.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.444+0000 7f0657e00640 1 --2- 172.21.15.86:0/492018032 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0660145f00 0x7f0660071d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:44.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.444+0000 7f065de00640 1 -- 172.21.15.86:0/492018032 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0660145f00 msgr2=0x7f0660071d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:44.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.444+0000 7f065de00640 1 --2- 172.21.15.86:0/492018032 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0660145f00 0x7f0660071d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:44.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.444+0000 7f065de00640 1 -- 172.21.15.86:0/492018032 >> v2:172.21.15.144:3300/0 conn(0x7f0660072280 msgr2=0x7f066006dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:44.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.444+0000 7f065de00640 1 --2- 172.21.15.86:0/492018032 >> v2:172.21.15.144:3300/0 conn(0x7f0660072280 0x7f066006dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:44.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.444+0000 7f065de00640 1 -- 172.21.15.86:0/492018032 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06601609d0 con 0x7f0660145250 2024-04-20T21:27:44.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.444+0000 7f065e800640 1 --2- 172.21.15.86:0/492018032 >> v2:172.21.15.144:3300/0 conn(0x7f0660072280 0x7f066006dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:27:44.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.444+0000 7f0657e00640 1 --2- 172.21.15.86:0/492018032 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0660145f00 0x7f0660071d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:27:44.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.445+0000 7f065de00640 1 --2- 172.21.15.86:0/492018032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0660145250 0x7f066006efc0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f06601464c0 tx=0x7f065002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c30309daecef42e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:44.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.445+0000 7f065d400640 1 -- 172.21.15.86:0/492018032 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065002e070 con 0x7f0660145250 2024-04-20T21:27:44.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.446+0000 7f065d400640 1 -- 172.21.15.86:0/492018032 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0650037070 con 0x7f0660145250 2024-04-20T21:27:44.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.446+0000 7f065d400640 1 -- 172.21.15.86:0/492018032 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0650031a00 con 0x7f0660145250 2024-04-20T21:27:44.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.446+0000 7f065f200640 1 -- 172.21.15.86:0/492018032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0660145250 msgr2=0x7f066006efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:44.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.446+0000 7f065f200640 1 --2- 172.21.15.86:0/492018032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0660145250 0x7f066006efc0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f06601464c0 tx=0x7f065002b880 comp rx=0 tx=0).stop 2024-04-20T21:27:44.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.447+0000 7f065f200640 1 -- 172.21.15.86:0/492018032 shutdown_connections 2024-04-20T21:27:44.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.447+0000 7f065f200640 1 --2- 172.21.15.86:0/492018032 >> v2:172.21.15.144:3300/0 conn(0x7f0660072280 0x7f066006dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:44.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.447+0000 7f065f200640 1 --2- 172.21.15.86:0/492018032 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0660145f00 0x7f0660071d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:44.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.447+0000 7f065f200640 1 --2- 172.21.15.86:0/492018032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0660145250 0x7f066006efc0 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:44.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.447+0000 7f065f200640 1 -- 172.21.15.86:0/492018032 >> 172.21.15.86:0/492018032 conn(0x7f0660067cf0 msgr2=0x7f066006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:44.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.447+0000 7f065f200640 1 -- 172.21.15.86:0/492018032 shutdown_connections 2024-04-20T21:27:44.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.447+0000 7f065f200640 1 -- 172.21.15.86:0/492018032 wait complete. 2024-04-20T21:27:44.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.449+0000 7f065f200640 1 Processor -- start 2024-04-20T21:27:44.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.449+0000 7f065f200640 1 -- start start 2024-04-20T21:27:44.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.450+0000 7f065f200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0660072280 0x7f066015be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:44.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.450+0000 7f065f200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0660145250 0x7f06600cc860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:44.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.450+0000 7f065f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0660145f00 0x7f06600ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:44.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.450+0000 7f065f200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f06601457b0 con 0x7f0660145f00 2024-04-20T21:27:44.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.450+0000 7f065f200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0660145630 con 0x7f0660072280 2024-04-20T21:27:44.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.450+0000 7f065f200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f066006e2e0 con 0x7f0660145250 2024-04-20T21:27:44.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.450+0000 7f065e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0660145f00 0x7f06600ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:44.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.450+0000 7f065e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0660145f00 0x7f06600ccda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53290/0 (socket says 172.21.15.86:53290) 2024-04-20T21:27:44.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.450+0000 7f065e800640 1 -- 172.21.15.86:0/35759176 learned_addr learned my addr 172.21.15.86:0/35759176 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:44.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.450+0000 7f065de00640 1 --2- 172.21.15.86:0/35759176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0660072280 0x7f066015be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:44.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.450+0000 7f0657e00640 1 --2- 172.21.15.86:0/35759176 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0660145250 0x7f06600cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:44.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.450+0000 7f065e800640 1 -- 172.21.15.86:0/35759176 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0660145250 msgr2=0x7f06600cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:44.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.450+0000 7f065e800640 1 --2- 172.21.15.86:0/35759176 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0660145250 0x7f06600cc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:44.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.450+0000 7f065e800640 1 -- 172.21.15.86:0/35759176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0660072280 msgr2=0x7f066015be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:44.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.450+0000 7f065e800640 1 --2- 172.21.15.86:0/35759176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0660072280 0x7f066015be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:44.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.451+0000 7f065e800640 1 -- 172.21.15.86:0/35759176 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06601f8a90 con 0x7f0660145f00 2024-04-20T21:27:44.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.451+0000 7f0657e00640 1 --2- 172.21.15.86:0/35759176 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0660145250 0x7f06600cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:27:44.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.451+0000 7f065e800640 1 --2- 172.21.15.86:0/35759176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0660145f00 0x7f06600ccda0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f065800cef0 tx=0x7f065800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:44.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.451+0000 7f0656000640 1 -- 172.21.15.86:0/35759176 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065801a070 con 0x7f0660145f00 2024-04-20T21:27:44.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.451+0000 7f0656000640 1 -- 172.21.15.86:0/35759176 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f065800d840 con 0x7f0660145f00 2024-04-20T21:27:44.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.451+0000 7f065f200640 1 -- 172.21.15.86:0/35759176 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06600ab460 con 0x7f0660145f00 2024-04-20T21:27:44.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.451+0000 7f0656000640 1 -- 172.21.15.86:0/35759176 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065800dc30 con 0x7f0660145f00 2024-04-20T21:27:44.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.451+0000 7f065f200640 1 -- 172.21.15.86:0/35759176 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06600aba20 con 0x7f0660145f00 2024-04-20T21:27:44.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.452+0000 7f065f200640 1 -- 172.21.15.86:0/35759176 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f066004fd90 con 0x7f0660145f00 2024-04-20T21:27:44.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.454+0000 7f0656000640 1 -- 172.21.15.86:0/35759176 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f065801d070 con 0x7f0660145f00 2024-04-20T21:27:44.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.455+0000 7f0656000640 1 --2- 172.21.15.86:0/35759176 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f063c06c8f0 0x7f063c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:44.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.455+0000 7f0656000640 1 -- 172.21.15.86:0/35759176 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5227+0+0 (secure 0 0 0) 0x7f065808c660 con 0x7f0660145f00 2024-04-20T21:27:44.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.455+0000 7f065de00640 1 --2- 172.21.15.86:0/35759176 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f063c06c8f0 0x7f063c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:44.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.456+0000 7f065de00640 1 --2- 172.21.15.86:0/35759176 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f063c06c8f0 0x7f063c06edb0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f0650036000 tx=0x7f065003d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:44.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.459+0000 7f0656000640 1 -- 172.21.15.86:0/35759176 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0658024020 con 0x7f0660145f00 2024-04-20T21:27:44.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:44.658+0000 7f065f200640 1 -- 172.21.15.86:0/35759176 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f066006f6e0 con 0x7f0660145f00 2024-04-20T21:27:45.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.218+0000 7f0656000640 1 -- 172.21.15.86:0/35759176 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v98) ==== 170+0+0 (secure 0 0 0) 0x7f065805ac50 con 0x7f0660145f00 2024-04-20T21:27:45.217 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' created 2024-04-20T21:27:45.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.222+0000 7f065f200640 1 -- 172.21.15.86:0/35759176 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f063c06c8f0 msgr2=0x7f063c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:45.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.222+0000 7f065f200640 1 --2- 172.21.15.86:0/35759176 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f063c06c8f0 0x7f063c06edb0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f0650036000 tx=0x7f065003d040 comp rx=0 tx=0).stop 2024-04-20T21:27:45.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.222+0000 7f065f200640 1 -- 172.21.15.86:0/35759176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0660145f00 msgr2=0x7f06600ccda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:45.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.222+0000 7f065f200640 1 --2- 172.21.15.86:0/35759176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0660145f00 0x7f06600ccda0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f065800cef0 tx=0x7f065800a710 comp rx=0 tx=0).stop 2024-04-20T21:27:45.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.222+0000 7f065f200640 1 -- 172.21.15.86:0/35759176 shutdown_connections 2024-04-20T21:27:45.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.222+0000 7f065f200640 1 --2- 172.21.15.86:0/35759176 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f063c06c8f0 0x7f063c06edb0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:45.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.222+0000 7f065f200640 1 --2- 172.21.15.86:0/35759176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0660145f00 0x7f06600ccda0 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:45.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.222+0000 7f065f200640 1 --2- 172.21.15.86:0/35759176 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0660145250 0x7f06600cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:45.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.222+0000 7f065f200640 1 --2- 172.21.15.86:0/35759176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0660072280 0x7f066015be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:45.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.222+0000 7f065f200640 1 -- 172.21.15.86:0/35759176 >> 172.21.15.86:0/35759176 conn(0x7f0660067cf0 msgr2=0x7f06600b3790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:45.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.223+0000 7f065f200640 1 -- 172.21.15.86:0/35759176 shutdown_connections 2024-04-20T21:27:45.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.223+0000 7f065f200640 1 -- 172.21.15.86:0/35759176 wait complete. 2024-04-20T21:27:45.261 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-20T21:27:45.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.436+0000 7fbd0d200640 1 Processor -- start 2024-04-20T21:27:45.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.437+0000 7fbd0d200640 1 -- start start 2024-04-20T21:27:45.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.438+0000 7fbd0d200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd080d1730 0x7fbd080d1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:45.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.438+0000 7fbd0d200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbd080d4b00 0x7fbd080d4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:45.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.438+0000 7fbd0d200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd080d3870 con 0x7fbd0815e850 2024-04-20T21:27:45.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.438+0000 7fbd0d200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fbd080d20d0 con 0x7fbd080d4b00 2024-04-20T21:27:45.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.438+0000 7fbd0d200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbd080d3d40 con 0x7fbd080d1730 2024-04-20T21:27:45.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.438+0000 7fbd05e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fbd0815e850 0x7fbd080b77f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54284/0 (socket says 172.21.15.86:54284) 2024-04-20T21:27:45.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.438+0000 7fbd05e00640 1 -- 172.21.15.86:0/1379862468 learned_addr learned my addr 172.21.15.86:0/1379862468 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:27:45.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.438+0000 7fbd06800640 1 --2- 172.21.15.86:0/1379862468 >> v2:172.21.15.86:3300/0 conn(0x7fbd080d4b00 0x7fbd080d4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:45.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.439+0000 7fbd05400640 1 --2- 172.21.15.86:0/1379862468 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd080d1730 0x7fbd080d1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:45.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.439+0000 7fbd06800640 1 -- 172.21.15.86:0/1379862468 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd080d1730 msgr2=0x7fbd080d1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:45.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.439+0000 7fbd06800640 1 --2- 172.21.15.86:0/1379862468 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd080d1730 0x7fbd080d1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:45.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.439+0000 7fbd04a00640 1 -- 172.21.15.86:0/1379862468 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1359789316 0 0) 0x7fbd080d3870 con 0x7fbd0815e850 2024-04-20T21:27:45.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.439+0000 7fbd06800640 1 -- 172.21.15.86:0/1379862468 >> v1:172.21.15.86:6789/0 conn(0x7fbd0815e850 legacy=0x7fbd080b77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:45.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.439+0000 7fbd06800640 1 -- 172.21.15.86:0/1379862468 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd080d4020 con 0x7fbd080d4b00 2024-04-20T21:27:45.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.439+0000 7fbd05400640 1 --2- 172.21.15.86:0/1379862468 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd080d1730 0x7fbd080d1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:27:45.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.440+0000 7fbd06800640 1 --2- 172.21.15.86:0/1379862468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd080d4b00 0x7fbd080d4f90 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fbd000097a0 tx=0x7fbd0002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b692004b7c62e37 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:45.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.440+0000 7fbd04a00640 1 -- 172.21.15.86:0/1379862468 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd00035020 con 0x7fbd080d4b00 2024-04-20T21:27:45.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.440+0000 7fbd04a00640 1 -- 172.21.15.86:0/1379862468 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd00031810 con 0x7fbd080d4b00 2024-04-20T21:27:45.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.440+0000 7fbd04a00640 1 -- 172.21.15.86:0/1379862468 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd00031be0 con 0x7fbd080d4b00 2024-04-20T21:27:45.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.441+0000 7fbd0d200640 1 -- 172.21.15.86:0/1379862468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd080d4b00 msgr2=0x7fbd080d4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:45.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.441+0000 7fbd0d200640 1 --2- 172.21.15.86:0/1379862468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd080d4b00 0x7fbd080d4f90 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fbd000097a0 tx=0x7fbd0002b8b0 comp rx=0 tx=0).stop 2024-04-20T21:27:45.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.441+0000 7fbd0d200640 1 -- 172.21.15.86:0/1379862468 shutdown_connections 2024-04-20T21:27:45.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.441+0000 7fbd0d200640 1 --2- 172.21.15.86:0/1379862468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd080d4b00 0x7fbd080d4f90 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:45.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.441+0000 7fbd0d200640 1 --2- 172.21.15.86:0/1379862468 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd080d1730 0x7fbd080d1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:45.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.441+0000 7fbd0d200640 1 -- 172.21.15.86:0/1379862468 >> 172.21.15.86:0/1379862468 conn(0x7fbd08067ae0 msgr2=0x7fbd08067ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:45.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.441+0000 7fbd0d200640 1 -- 172.21.15.86:0/1379862468 shutdown_connections 2024-04-20T21:27:45.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.441+0000 7fbd0d200640 1 -- 172.21.15.86:0/1379862468 wait complete. 2024-04-20T21:27:45.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.443+0000 7fbd0d200640 1 Processor -- start 2024-04-20T21:27:45.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.443+0000 7fbd0d200640 1 -- start start 2024-04-20T21:27:45.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.444+0000 7fbd0d200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd080d1730 0x7fbd08160ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:45.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.444+0000 7fbd0d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd080d4b00 0x7fbd08161420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:45.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.444+0000 7fbd0d200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd0815e850 0x7fbd081f44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:45.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.444+0000 7fbd0d200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbd080d54d0 con 0x7fbd080d4b00 2024-04-20T21:27:45.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.444+0000 7fbd0d200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbd080d45e0 con 0x7fbd0815e850 2024-04-20T21:27:45.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.444+0000 7fbd0d200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbd080b7f00 con 0x7fbd080d1730 2024-04-20T21:27:45.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.444+0000 7fbd05400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd080d4b00 0x7fbd08161420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:45.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.444+0000 7fbd05400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd080d4b00 0x7fbd08161420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53308/0 (socket says 172.21.15.86:53308) 2024-04-20T21:27:45.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.444+0000 7fbd05400640 1 -- 172.21.15.86:0/872555214 learned_addr learned my addr 172.21.15.86:0/872555214 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:45.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.445+0000 7fbd05e00640 1 --2- 172.21.15.86:0/872555214 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd080d1730 0x7fbd08160ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:45.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.445+0000 7fbd06800640 1 --2- 172.21.15.86:0/872555214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd0815e850 0x7fbd081f44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:45.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.445+0000 7fbd05400640 1 -- 172.21.15.86:0/872555214 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd080d1730 msgr2=0x7fbd08160ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:45.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.445+0000 7fbd05400640 1 --2- 172.21.15.86:0/872555214 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd080d1730 0x7fbd08160ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:45.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.445+0000 7fbd05400640 1 -- 172.21.15.86:0/872555214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd0815e850 msgr2=0x7fbd081f44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:45.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.445+0000 7fbd05400640 1 --2- 172.21.15.86:0/872555214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd0815e850 0x7fbd081f44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:45.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.445+0000 7fbd05400640 1 -- 172.21.15.86:0/872555214 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd081f7960 con 0x7fbd080d4b00 2024-04-20T21:27:45.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.445+0000 7fbd06800640 1 --2- 172.21.15.86:0/872555214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd0815e850 0x7fbd081f44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:27:45.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.445+0000 7fbd05400640 1 --2- 172.21.15.86:0/872555214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd080d4b00 0x7fbd08161420 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fbcf000c7f0 tx=0x7fbcf000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:45.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.445+0000 7fbcfe000640 1 -- 172.21.15.86:0/872555214 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf0015070 con 0x7fbd080d4b00 2024-04-20T21:27:45.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.445+0000 7fbcfe000640 1 -- 172.21.15.86:0/872555214 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcf0007a30 con 0x7fbd080d4b00 2024-04-20T21:27:45.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.445+0000 7fbcfe000640 1 -- 172.21.15.86:0/872555214 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf0007e00 con 0x7fbd080d4b00 2024-04-20T21:27:45.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.445+0000 7fbd0d200640 1 -- 172.21.15.86:0/872555214 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd081f7c40 con 0x7fbd080d4b00 2024-04-20T21:27:45.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.446+0000 7fbd0d200640 1 -- 172.21.15.86:0/872555214 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd081f8200 con 0x7fbd080d4b00 2024-04-20T21:27:45.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.447+0000 7fbd0d200640 1 -- 172.21.15.86:0/872555214 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd080690f0 con 0x7fbd080d4b00 2024-04-20T21:27:45.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.448+0000 7fbcfe000640 1 -- 172.21.15.86:0/872555214 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbcf0018070 con 0x7fbd080d4b00 2024-04-20T21:27:45.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.449+0000 7fbcfe000640 1 --2- 172.21.15.86:0/872555214 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbce406c8f0 0x7fbce406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:45.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.449+0000 7fbcfe000640 1 -- 172.21.15.86:0/872555214 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5609+0+0 (secure 0 0 0) 0x7fbcf008bb70 con 0x7fbd080d4b00 2024-04-20T21:27:45.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.449+0000 7fbd05e00640 1 --2- 172.21.15.86:0/872555214 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbce406c8f0 0x7fbce406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:45.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.450+0000 7fbd05e00640 1 --2- 172.21.15.86:0/872555214 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbce406c8f0 0x7fbce406edb0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8007ed0 tx=0x7fbcf8007e60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:45.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.454+0000 7fbcfe000640 1 -- 172.21.15.86:0/872555214 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbcf0059f60 con 0x7fbd080d4b00 2024-04-20T21:27:45.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:45.690+0000 7fbd0d200640 1 -- 172.21.15.86:0/872555214 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fbd080ce2e0 con 0x7fbd080d4b00 2024-04-20T21:27:46.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.266+0000 7fbcfe000640 1 -- 172.21.15.86:0/872555214 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v78) ==== 174+0+0 (secure 0 0 0) 0x7fbcf0059860 con 0x7fbd080d4b00 2024-04-20T21:27:46.269 INFO:teuthology.orchestra.run.smithi086.stderr:new fs with metadata pool 14 and data pool 15 2024-04-20T21:27:46.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.273+0000 7fbd0d200640 1 -- 172.21.15.86:0/872555214 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbce406c8f0 msgr2=0x7fbce406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:46.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.273+0000 7fbd0d200640 1 --2- 172.21.15.86:0/872555214 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbce406c8f0 0x7fbce406edb0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8007ed0 tx=0x7fbcf8007e60 comp rx=0 tx=0).stop 2024-04-20T21:27:46.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.273+0000 7fbd0d200640 1 -- 172.21.15.86:0/872555214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd080d4b00 msgr2=0x7fbd08161420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:46.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.273+0000 7fbd0d200640 1 --2- 172.21.15.86:0/872555214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd080d4b00 0x7fbd08161420 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fbcf000c7f0 tx=0x7fbcf000cbf0 comp rx=0 tx=0).stop 2024-04-20T21:27:46.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.274+0000 7fbd0d200640 1 -- 172.21.15.86:0/872555214 shutdown_connections 2024-04-20T21:27:46.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.274+0000 7fbd0d200640 1 --2- 172.21.15.86:0/872555214 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbce406c8f0 0x7fbce406edb0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:46.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.274+0000 7fbd0d200640 1 --2- 172.21.15.86:0/872555214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd0815e850 0x7fbd081f44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:46.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.274+0000 7fbd0d200640 1 --2- 172.21.15.86:0/872555214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd080d4b00 0x7fbd08161420 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:46.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.274+0000 7fbd0d200640 1 --2- 172.21.15.86:0/872555214 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd080d1730 0x7fbd08160ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:46.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.274+0000 7fbd0d200640 1 -- 172.21.15.86:0/872555214 >> 172.21.15.86:0/872555214 conn(0x7fbd08067ae0 msgr2=0x7fbd080b77f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:46.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.275+0000 7fbd0d200640 1 -- 172.21.15.86:0/872555214 shutdown_connections 2024-04-20T21:27:46.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.275+0000 7fbd0d200640 1 -- 172.21.15.86:0/872555214 wait complete. 2024-04-20T21:27:46.311 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:27:46.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.480+0000 7f98b4a00640 1 Processor -- start 2024-04-20T21:27:46.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.480+0000 7f98b4a00640 1 -- start start 2024-04-20T21:27:46.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.481+0000 7f98b4a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98b009a9e0 0x7f98b009ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:46.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.482+0000 7f98b4a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98b015efd0 con 0x7f98b00b2e60 2024-04-20T21:27:46.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.482+0000 7f98b4a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98b00b1dc0 con 0x7f98b015ebf0 2024-04-20T21:27:46.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.482+0000 7f98b4a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f98b009b380 con 0x7f98b009a9e0 2024-04-20T21:27:46.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.482+0000 7f98ae800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f98b00b2e60 0x7f98b00b32f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57356/0 (socket says 172.21.15.86:57356) 2024-04-20T21:27:46.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.482+0000 7f98ae800640 1 -- 172.21.15.86:0/2078941957 learned_addr learned my addr 172.21.15.86:0/2078941957 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:27:46.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.482+0000 7f98a5400640 1 --2- 172.21.15.86:0/2078941957 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98b009a9e0 0x7f98b009ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:46.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.483+0000 7f98ad400640 1 -- 172.21.15.86:0/2078941957 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2361865586 0 0) 0x7f98b015efd0 con 0x7f98b00b2e60 2024-04-20T21:27:46.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.483+0000 7f98ad400640 1 -- 172.21.15.86:0/2078941957 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9894003520 con 0x7f98b00b2e60 2024-04-20T21:27:46.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.483+0000 7f98ad400640 1 -- 172.21.15.86:0/2078941957 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1143631743 0 0) 0x7f9894003520 con 0x7f98b00b2e60 2024-04-20T21:27:46.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.483+0000 7f98ad400640 1 -- 172.21.15.86:0/2078941957 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98b009a9e0 msgr2=0x7f98b009ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:46.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.483+0000 7f98ad400640 1 --2- 172.21.15.86:0/2078941957 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98b009a9e0 0x7f98b009ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:46.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.483+0000 7f98ad400640 1 -- 172.21.15.86:0/2078941957 >> v1:172.21.15.144:6789/0 conn(0x7f98b015ebf0 legacy=0x7f98b00d12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:46.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.483+0000 7f98ad400640 1 -- 172.21.15.86:0/2078941957 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98b00b20a0 con 0x7f98b00b2e60 2024-04-20T21:27:46.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.483+0000 7f98ad400640 1 -- 172.21.15.86:0/2078941957 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f98a8001670 con 0x7f98b00b2e60 2024-04-20T21:27:46.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.483+0000 7f98ad400640 1 -- 172.21.15.86:0/2078941957 >> v1:172.21.15.86:6789/0 conn(0x7f98b00b2e60 legacy=0x7f98b00b32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:46.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.483+0000 7f98ad400640 1 --2- 172.21.15.86:0/2078941957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98940039e0 0x7f9894003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:46.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.483+0000 7f98ad400640 1 --2- 172.21.15.86:0/2078941957 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98b009a9e0 0x7f9894004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:46.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.483+0000 7f98ad400640 1 --2- 172.21.15.86:0/2078941957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9894004d40 0x7f98940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:46.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.483+0000 7f98ad400640 1 -- 172.21.15.86:0/2078941957 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f989400ba70 con 0x7f9894004d40 2024-04-20T21:27:46.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.483+0000 7f98ad400640 1 -- 172.21.15.86:0/2078941957 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f989400bbf0 con 0x7f98940039e0 2024-04-20T21:27:46.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.483+0000 7f98ad400640 1 -- 172.21.15.86:0/2078941957 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f989400bdf0 con 0x7f98b009a9e0 2024-04-20T21:27:46.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.484+0000 7f98ae800640 1 --2- 172.21.15.86:0/2078941957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9894004d40 0x7f98940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:46.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.484+0000 7f98a5400640 1 --2- 172.21.15.86:0/2078941957 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98b009a9e0 0x7f9894004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:46.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.484+0000 7f98ade00640 1 --2- 172.21.15.86:0/2078941957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98940039e0 0x7f9894003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:46.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.484+0000 7f98ae800640 1 -- 172.21.15.86:0/2078941957 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98b009a9e0 msgr2=0x7f9894004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:46.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.484+0000 7f98ae800640 1 --2- 172.21.15.86:0/2078941957 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98b009a9e0 0x7f9894004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:46.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.484+0000 7f98ae800640 1 -- 172.21.15.86:0/2078941957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98940039e0 msgr2=0x7f9894003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:46.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.484+0000 7f98ae800640 1 --2- 172.21.15.86:0/2078941957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98940039e0 0x7f9894003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:46.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.484+0000 7f98ae800640 1 -- 172.21.15.86:0/2078941957 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9894003520 con 0x7f9894004d40 2024-04-20T21:27:46.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.484+0000 7f98a5400640 1 --2- 172.21.15.86:0/2078941957 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98b009a9e0 0x7f9894004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:27:46.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.484+0000 7f98ade00640 1 --2- 172.21.15.86:0/2078941957 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98940039e0 0x7f9894003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:27:46.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.485+0000 7f98ae800640 1 --2- 172.21.15.86:0/2078941957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9894004d40 0x7f98940051c0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f98a8001eb0 tx=0x7f9894005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f3ff1b3812b7c90 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:46.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.485+0000 7f98ad400640 1 -- 172.21.15.86:0/2078941957 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a8007d30 con 0x7f9894004d40 2024-04-20T21:27:46.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.485+0000 7f98ad400640 1 -- 172.21.15.86:0/2078941957 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98a8030070 con 0x7f9894004d40 2024-04-20T21:27:46.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.485+0000 7f98ad400640 1 -- 172.21.15.86:0/2078941957 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a8005d30 con 0x7f9894004d40 2024-04-20T21:27:46.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.486+0000 7f98b4a00640 1 -- 172.21.15.86:0/2078941957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9894004d40 msgr2=0x7f98940051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:46.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.486+0000 7f98b4a00640 1 --2- 172.21.15.86:0/2078941957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9894004d40 0x7f98940051c0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f98a8001eb0 tx=0x7f9894005830 comp rx=0 tx=0).stop 2024-04-20T21:27:46.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.486+0000 7f98ae800640 1 -- 172.21.15.86:0/2078941957 reap_dead start 2024-04-20T21:27:46.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.486+0000 7f98b4a00640 1 -- 172.21.15.86:0/2078941957 shutdown_connections 2024-04-20T21:27:46.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.486+0000 7f98b4a00640 1 -- 172.21.15.86:0/2078941957 >> 172.21.15.86:0/2078941957 conn(0x7f98b00b3ff0 msgr2=0x7f98b00b4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:46.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.486+0000 7f98b4a00640 1 -- 172.21.15.86:0/2078941957 shutdown_connections 2024-04-20T21:27:46.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.486+0000 7f98b4a00640 1 -- 172.21.15.86:0/2078941957 wait complete. 2024-04-20T21:27:46.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.488+0000 7f98b4a00640 1 Processor -- start 2024-04-20T21:27:46.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.488+0000 7f98b4a00640 1 -- start start 2024-04-20T21:27:46.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.489+0000 7f98b4a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9894004d40 0x7f98b00cbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:46.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.489+0000 7f98b4a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98b00cce90 0x7f98b00cd310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:46.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.489+0000 7f98b4a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98b00c7bb0 0x7f98b00c8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:46.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.489+0000 7f98b4a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f98b00b2940 con 0x7f98b00cce90 2024-04-20T21:27:46.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.489+0000 7f98b4a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f98b00b2660 con 0x7f9894004d40 2024-04-20T21:27:46.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.489+0000 7f98b4a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f98b00ccb50 con 0x7f98b00c7bb0 2024-04-20T21:27:46.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.489+0000 7f98a5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98b00cce90 0x7f98b00cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:46.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.489+0000 7f98a5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98b00cce90 0x7f98b00cd310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36480/0 (socket says 172.21.15.86:36480) 2024-04-20T21:27:46.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.489+0000 7f98a5400640 1 -- 172.21.15.86:0/3672467580 learned_addr learned my addr 172.21.15.86:0/3672467580 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:46.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.490+0000 7f98a5400640 1 -- 172.21.15.86:0/3672467580 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98b00c7bb0 msgr2=0x7f98b00c8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:46.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.490+0000 7f98ae800640 1 --2- 172.21.15.86:0/3672467580 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98b00c7bb0 0x7f98b00c8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:46.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.490+0000 7f98ade00640 1 --2- 172.21.15.86:0/3672467580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9894004d40 0x7f98b00cbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:46.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.490+0000 7f98a5400640 1 --2- 172.21.15.86:0/3672467580 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98b00c7bb0 0x7f98b00c8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:46.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.490+0000 7f98a5400640 1 -- 172.21.15.86:0/3672467580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9894004d40 msgr2=0x7f98b00cbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:46.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.490+0000 7f98a5400640 1 --2- 172.21.15.86:0/3672467580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9894004d40 0x7f98b00cbfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:46.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.490+0000 7f98a5400640 1 -- 172.21.15.86:0/3672467580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98b00c8950 con 0x7f98b00cce90 2024-04-20T21:27:46.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.490+0000 7f98a5400640 1 --2- 172.21.15.86:0/3672467580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98b00cce90 0x7f98b00cd310 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f9898010910 tx=0x7f9898010dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:46.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.490+0000 7f98a6a00640 1 -- 172.21.15.86:0/3672467580 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989800f040 con 0x7f98b00cce90 2024-04-20T21:27:46.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.491+0000 7f98a6a00640 1 -- 172.21.15.86:0/3672467580 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f989800ebc0 con 0x7f98b00cce90 2024-04-20T21:27:46.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.491+0000 7f98a6a00640 1 -- 172.21.15.86:0/3672467580 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989801c400 con 0x7f98b00cce90 2024-04-20T21:27:46.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.491+0000 7f98b4a00640 1 -- 172.21.15.86:0/3672467580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98b00b5780 con 0x7f98b00cce90 2024-04-20T21:27:46.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.491+0000 7f98b4a00640 1 -- 172.21.15.86:0/3672467580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98b00b5d40 con 0x7f98b00cce90 2024-04-20T21:27:46.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.492+0000 7f98b4a00640 1 -- 172.21.15.86:0/3672467580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98b00689e0 con 0x7f98b00cce90 2024-04-20T21:27:46.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.493+0000 7f98a6a00640 1 -- 172.21.15.86:0/3672467580 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f989801c5a0 con 0x7f98b00cce90 2024-04-20T21:27:46.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.494+0000 7f98a6a00640 1 --2- 172.21.15.86:0/3672467580 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f988c06c920 0x7f988c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:46.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.494+0000 7f98a6a00640 1 -- 172.21.15.86:0/3672467580 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7f989808c170 con 0x7f98b00cce90 2024-04-20T21:27:46.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.494+0000 7f98ade00640 1 --2- 172.21.15.86:0/3672467580 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f988c06c920 0x7f988c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:46.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.495+0000 7f98ade00640 1 --2- 172.21.15.86:0/3672467580 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f988c06c920 0x7f988c06ede0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f98a0001d20 tx=0x7f98a0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:46.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.498+0000 7f98a6a00640 1 -- 172.21.15.86:0/3672467580 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9898018030 con 0x7f98b00cce90 2024-04-20T21:27:46.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.695+0000 7f98b4a00640 1 -- 172.21.15.86:0/3672467580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f98b00b1690 con 0x7f98b00cce90 2024-04-20T21:27:46.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.698+0000 7f98a6a00640 1 -- 172.21.15.86:0/3672467580 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+16621 (secure 0 0 0) 0x7f989805a5a0 con 0x7f98b00cce90 2024-04-20T21:27:46.698 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:27:46.698 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":99,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:27:45.692489+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:27:43.469685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-20T21:27:44.673505+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:27:46.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.701+0000 7f98b4a00640 1 -- 172.21.15.86:0/3672467580 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f988c06c920 msgr2=0x7f988c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:46.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.701+0000 7f98b4a00640 1 --2- 172.21.15.86:0/3672467580 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f988c06c920 0x7f988c06ede0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f98a0001d20 tx=0x7f98a0009040 comp rx=0 tx=0).stop 2024-04-20T21:27:46.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.701+0000 7f98b4a00640 1 -- 172.21.15.86:0/3672467580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98b00cce90 msgr2=0x7f98b00cd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:46.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.701+0000 7f98b4a00640 1 --2- 172.21.15.86:0/3672467580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98b00cce90 0x7f98b00cd310 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f9898010910 tx=0x7f9898010dd0 comp rx=0 tx=0).stop 2024-04-20T21:27:46.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.701+0000 7f98b4a00640 1 -- 172.21.15.86:0/3672467580 shutdown_connections 2024-04-20T21:27:46.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.701+0000 7f98b4a00640 1 --2- 172.21.15.86:0/3672467580 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f988c06c920 0x7f988c06ede0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:46.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.701+0000 7f98b4a00640 1 --2- 172.21.15.86:0/3672467580 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98b00c7bb0 0x7f98b00c8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:46.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.701+0000 7f98b4a00640 1 --2- 172.21.15.86:0/3672467580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98b00cce90 0x7f98b00cd310 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:46.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.701+0000 7f98b4a00640 1 --2- 172.21.15.86:0/3672467580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9894004d40 0x7f98b00cbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:46.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.701+0000 7f98b4a00640 1 -- 172.21.15.86:0/3672467580 >> 172.21.15.86:0/3672467580 conn(0x7f98b00b3ff0 msgr2=0x7f98b00af1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:46.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.702+0000 7f98b4a00640 1 -- 172.21.15.86:0/3672467580 shutdown_connections 2024-04-20T21:27:46.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.702+0000 7f98b4a00640 1 -- 172.21.15.86:0/3672467580 wait complete. 2024-04-20T21:27:46.717 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:27:46.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.910+0000 7f26ed600640 1 Processor -- start 2024-04-20T21:27:46.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.910+0000 7f26ed600640 1 -- start start 2024-04-20T21:27:46.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.912+0000 7f26ed600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26e8156ae0 0x7f26e815d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:46.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.912+0000 7f26ed600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f26e8160080 0x7f26e80ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:46.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.912+0000 7f26ed600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26e8160890 con 0x7f26e80ad620 2024-04-20T21:27:46.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.912+0000 7f26ed600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f26e806db20 con 0x7f26e8160080 2024-04-20T21:27:46.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.912+0000 7f26ed600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f26e8160b70 con 0x7f26e8156ae0 2024-04-20T21:27:46.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.912+0000 7f26e7400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f26e8160080 0x7f26e80ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:46.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.912+0000 7f26e7400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f26e8160080 0x7f26e80ad0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36496/0 (socket says 172.21.15.86:36496) 2024-04-20T21:27:46.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.912+0000 7f26e7400640 1 -- 172.21.15.86:0/3431320240 learned_addr learned my addr 172.21.15.86:0/3431320240 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:46.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.912+0000 7f26e7400640 1 -- 172.21.15.86:0/3431320240 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26e8156ae0 msgr2=0x7f26e815d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:46.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.912+0000 7f26e7400640 1 --2- 172.21.15.86:0/3431320240 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26e8156ae0 0x7f26e815d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:46.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.912+0000 7f26e7400640 1 -- 172.21.15.86:0/3431320240 >> v1:172.21.15.144:6789/0 conn(0x7f26e80ad620 legacy=0x7f26e8141cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:46.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.912+0000 7f26e7400640 1 -- 172.21.15.86:0/3431320240 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26e8160e50 con 0x7f26e8160080 2024-04-20T21:27:46.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.914+0000 7f26e7400640 1 --2- 172.21.15.86:0/3431320240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26e8160080 0x7f26e80ad0e0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f26d0009870 tx=0x7f26d002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29fd6df0d2848763 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:46.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.914+0000 7f26e6a00640 1 -- 172.21.15.86:0/3431320240 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d0035020 con 0x7f26e8160080 2024-04-20T21:27:46.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.914+0000 7f26e6a00640 1 -- 172.21.15.86:0/3431320240 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26d0031640 con 0x7f26e8160080 2024-04-20T21:27:46.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.914+0000 7f26e6a00640 1 -- 172.21.15.86:0/3431320240 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d002d040 con 0x7f26e8160080 2024-04-20T21:27:46.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.915+0000 7f26ed600640 1 -- 172.21.15.86:0/3431320240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26e8160080 msgr2=0x7f26e80ad0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:46.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.915+0000 7f26ed600640 1 --2- 172.21.15.86:0/3431320240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26e8160080 0x7f26e80ad0e0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f26d0009870 tx=0x7f26d002b8f0 comp rx=0 tx=0).stop 2024-04-20T21:27:46.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.915+0000 7f26ed600640 1 -- 172.21.15.86:0/3431320240 shutdown_connections 2024-04-20T21:27:46.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.915+0000 7f26ed600640 1 --2- 172.21.15.86:0/3431320240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26e8160080 0x7f26e80ad0e0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:46.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.915+0000 7f26ed600640 1 --2- 172.21.15.86:0/3431320240 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26e8156ae0 0x7f26e815d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:46.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.915+0000 7f26ed600640 1 -- 172.21.15.86:0/3431320240 >> 172.21.15.86:0/3431320240 conn(0x7f26e80d0170 msgr2=0x7f26e80d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:46.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.915+0000 7f26ed600640 1 -- 172.21.15.86:0/3431320240 shutdown_connections 2024-04-20T21:27:46.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.915+0000 7f26ed600640 1 -- 172.21.15.86:0/3431320240 wait complete. 2024-04-20T21:27:46.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.917+0000 7f26ed600640 1 Processor -- start 2024-04-20T21:27:46.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.917+0000 7f26ed600640 1 -- start start 2024-04-20T21:27:46.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.918+0000 7f26ed600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26e80ad620 0x7f26e8155850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:46.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.918+0000 7f26ed600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26e8156ae0 0x7f26e80992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:46.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.918+0000 7f26ed600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26e8160080 0x7f26e815a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:46.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.918+0000 7f26ed600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f26e8160500 con 0x7f26e8160080 2024-04-20T21:27:46.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.918+0000 7f26ed600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f26e8156ec0 con 0x7f26e8156ae0 2024-04-20T21:27:46.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.918+0000 7f26ed600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f26e806f2a0 con 0x7f26e80ad620 2024-04-20T21:27:46.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.919+0000 7f26ecc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26e8160080 0x7f26e815a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:46.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.919+0000 7f26ecc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26e8160080 0x7f26e815a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36506/0 (socket says 172.21.15.86:36506) 2024-04-20T21:27:46.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.919+0000 7f26ecc00640 1 -- 172.21.15.86:0/2647252003 learned_addr learned my addr 172.21.15.86:0/2647252003 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:46.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.919+0000 7f26e7400640 1 --2- 172.21.15.86:0/2647252003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26e8156ae0 0x7f26e80992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:46.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.919+0000 7f26e7e00640 1 --2- 172.21.15.86:0/2647252003 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26e80ad620 0x7f26e8155850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:46.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.919+0000 7f26ecc00640 1 -- 172.21.15.86:0/2647252003 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26e80ad620 msgr2=0x7f26e8155850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:46.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.919+0000 7f26ecc00640 1 --2- 172.21.15.86:0/2647252003 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26e80ad620 0x7f26e8155850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:46.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.919+0000 7f26ecc00640 1 -- 172.21.15.86:0/2647252003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26e8156ae0 msgr2=0x7f26e80992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:46.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.919+0000 7f26ecc00640 1 --2- 172.21.15.86:0/2647252003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26e8156ae0 0x7f26e80992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:46.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.919+0000 7f26ecc00640 1 -- 172.21.15.86:0/2647252003 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26e8099e10 con 0x7f26e8160080 2024-04-20T21:27:46.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.919+0000 7f26ecc00640 1 --2- 172.21.15.86:0/2647252003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26e8160080 0x7f26e815a690 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f26d800b4c0 tx=0x7f26d800b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:46.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.920+0000 7f26dfe00640 1 -- 172.21.15.86:0/2647252003 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d8015020 con 0x7f26e8160080 2024-04-20T21:27:46.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.920+0000 7f26dfe00640 1 -- 172.21.15.86:0/2647252003 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26d8011850 con 0x7f26e8160080 2024-04-20T21:27:46.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.920+0000 7f26dfe00640 1 -- 172.21.15.86:0/2647252003 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d8011c40 con 0x7f26e8160080 2024-04-20T21:27:46.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.920+0000 7f26ed600640 1 -- 172.21.15.86:0/2647252003 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26e8152f00 con 0x7f26e8160080 2024-04-20T21:27:46.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.920+0000 7f26ed600640 1 -- 172.21.15.86:0/2647252003 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26e8071dc0 con 0x7f26e8160080 2024-04-20T21:27:46.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.921+0000 7f26ed600640 1 -- 172.21.15.86:0/2647252003 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26e80b0d30 con 0x7f26e8160080 2024-04-20T21:27:46.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.922+0000 7f26dfe00640 1 -- 172.21.15.86:0/2647252003 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f26d8018020 con 0x7f26e8160080 2024-04-20T21:27:46.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.923+0000 7f26dfe00640 1 --2- 172.21.15.86:0/2647252003 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f26c406c920 0x7f26c406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:46.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.923+0000 7f26dfe00640 1 -- 172.21.15.86:0/2647252003 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5709+0+0 (secure 0 0 0) 0x7f26d808b400 con 0x7f26e8160080 2024-04-20T21:27:46.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.924+0000 7f26e7e00640 1 --2- 172.21.15.86:0/2647252003 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f26c406c920 0x7f26c406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:46.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.924+0000 7f26e7e00640 1 --2- 172.21.15.86:0/2647252003 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f26c406c920 0x7f26c406ede0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f26e0007dc0 tx=0x7f26e00072f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:46.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:46.928+0000 7f26dfe00640 1 -- 172.21.15.86:0/2647252003 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f26e80b0d30 con 0x7f26e8160080 2024-04-20T21:27:47.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.126+0000 7f26ed600640 1 -- 172.21.15.86:0/2647252003 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f26e804fcf0 con 0x7f26e8160080 2024-04-20T21:27:47.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.129+0000 7f26dfe00640 1 -- 172.21.15.86:0/2647252003 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+16621 (secure 0 0 0) 0x7f26d801f070 con 0x7f26e8160080 2024-04-20T21:27:47.129 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:27:47.130 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":99,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:27:45.692489+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:27:43.469685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-20T21:27:44.673505+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:27:47.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.132+0000 7f26ed600640 1 -- 172.21.15.86:0/2647252003 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f26c406c920 msgr2=0x7f26c406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:47.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.132+0000 7f26ed600640 1 --2- 172.21.15.86:0/2647252003 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f26c406c920 0x7f26c406ede0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f26e0007dc0 tx=0x7f26e00072f0 comp rx=0 tx=0).stop 2024-04-20T21:27:47.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.132+0000 7f26ed600640 1 -- 172.21.15.86:0/2647252003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26e8160080 msgr2=0x7f26e815a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:47.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.132+0000 7f26ed600640 1 --2- 172.21.15.86:0/2647252003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26e8160080 0x7f26e815a690 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f26d800b4c0 tx=0x7f26d800b980 comp rx=0 tx=0).stop 2024-04-20T21:27:47.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.133+0000 7f26ed600640 1 -- 172.21.15.86:0/2647252003 shutdown_connections 2024-04-20T21:27:47.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.133+0000 7f26ed600640 1 --2- 172.21.15.86:0/2647252003 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f26c406c920 0x7f26c406ede0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:47.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.133+0000 7f26ed600640 1 --2- 172.21.15.86:0/2647252003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26e8160080 0x7f26e815a690 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:47.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.133+0000 7f26ed600640 1 --2- 172.21.15.86:0/2647252003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26e8156ae0 0x7f26e80992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:47.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.133+0000 7f26ed600640 1 --2- 172.21.15.86:0/2647252003 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26e80ad620 0x7f26e8155850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:47.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.133+0000 7f26ed600640 1 -- 172.21.15.86:0/2647252003 >> 172.21.15.86:0/2647252003 conn(0x7f26e80d0170 msgr2=0x7f26e80b2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:47.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.133+0000 7f26ed600640 1 -- 172.21.15.86:0/2647252003 shutdown_connections 2024-04-20T21:27:47.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.133+0000 7f26ed600640 1 -- 172.21.15.86:0/2647252003 wait complete. 2024-04-20T21:27:47.149 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-20T21:27:47.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.345+0000 7f7884c00640 1 Processor -- start 2024-04-20T21:27:47.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.346+0000 7f7884c00640 1 -- start start 2024-04-20T21:27:47.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.347+0000 7f7884c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f788006c090 0x7f78800cd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:47.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.347+0000 7f7884c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78800b3410 0x7f78800cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:47.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.347+0000 7f7884c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78800b3910 con 0x7f7880162ed0 2024-04-20T21:27:47.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.347+0000 7f7884c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f78800b3bf0 con 0x7f788006c090 2024-04-20T21:27:47.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.347+0000 7f7884c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f78800b3ed0 con 0x7f78800b3410 2024-04-20T21:27:47.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.347+0000 7f7875400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f788006c090 0x7f78800cd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:47.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.347+0000 7f7875400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f788006c090 0x7f78800cd360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36512/0 (socket says 172.21.15.86:36512) 2024-04-20T21:27:47.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.347+0000 7f7875400640 1 -- 172.21.15.86:0/3264671322 learned_addr learned my addr 172.21.15.86:0/3264671322 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:47.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.348+0000 7f7875400640 1 -- 172.21.15.86:0/3264671322 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78800b3410 msgr2=0x7f78800cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:47.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.348+0000 7f7875400640 1 --2- 172.21.15.86:0/3264671322 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78800b3410 0x7f78800cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:47.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.348+0000 7f7875400640 1 -- 172.21.15.86:0/3264671322 >> v1:172.21.15.144:6789/0 conn(0x7f7880162ed0 legacy=0x7f788006b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:47.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.348+0000 7f7875400640 1 -- 172.21.15.86:0/3264671322 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78800b41b0 con 0x7f788006c090 2024-04-20T21:27:47.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.349+0000 7f7875400640 1 --2- 172.21.15.86:0/3264671322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f788006c090 0x7f78800cd360 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f7868009870 tx=0x7f786802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bf256333629b41e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:47.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.349+0000 7f787d400640 1 -- 172.21.15.86:0/3264671322 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7868035020 con 0x7f788006c090 2024-04-20T21:27:47.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.349+0000 7f787d400640 1 -- 172.21.15.86:0/3264671322 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7868031640 con 0x7f788006c090 2024-04-20T21:27:47.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.349+0000 7f787d400640 1 -- 172.21.15.86:0/3264671322 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786802d040 con 0x7f788006c090 2024-04-20T21:27:47.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.350+0000 7f7884c00640 1 -- 172.21.15.86:0/3264671322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f788006c090 msgr2=0x7f78800cd360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:47.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.350+0000 7f7884c00640 1 --2- 172.21.15.86:0/3264671322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f788006c090 0x7f78800cd360 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f7868009870 tx=0x7f786802b8f0 comp rx=0 tx=0).stop 2024-04-20T21:27:47.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.350+0000 7f7884c00640 1 -- 172.21.15.86:0/3264671322 shutdown_connections 2024-04-20T21:27:47.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.350+0000 7f7884c00640 1 --2- 172.21.15.86:0/3264671322 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78800b3410 0x7f78800cd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:47.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.350+0000 7f7884c00640 1 --2- 172.21.15.86:0/3264671322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f788006c090 0x7f78800cd360 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:47.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.350+0000 7f7884c00640 1 -- 172.21.15.86:0/3264671322 >> 172.21.15.86:0/3264671322 conn(0x7f78800b2110 msgr2=0x7f78800b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:47.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.350+0000 7f7884c00640 1 -- 172.21.15.86:0/3264671322 shutdown_connections 2024-04-20T21:27:47.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.350+0000 7f7884c00640 1 -- 172.21.15.86:0/3264671322 wait complete. 2024-04-20T21:27:47.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.352+0000 7f7884c00640 1 Processor -- start 2024-04-20T21:27:47.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.352+0000 7f7884c00640 1 -- start start 2024-04-20T21:27:47.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.353+0000 7f7884c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f788006c090 0x7f788015c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:47.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.353+0000 7f7884c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78800b3410 0x7f78801655f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:47.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.353+0000 7f7884c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7880162ed0 0x7f78801ecfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:47.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.353+0000 7f7884c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f788006fca0 con 0x7f788006c090 2024-04-20T21:27:47.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.353+0000 7f7884c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f788006fb20 con 0x7f7880162ed0 2024-04-20T21:27:47.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.353+0000 7f7884c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f788006e160 con 0x7f78800b3410 2024-04-20T21:27:47.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.353+0000 7f787de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f788006c090 0x7f788015c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:47.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.353+0000 7f787de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f788006c090 0x7f788015c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36524/0 (socket says 172.21.15.86:36524) 2024-04-20T21:27:47.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.353+0000 7f787de00640 1 -- 172.21.15.86:0/3408192140 learned_addr learned my addr 172.21.15.86:0/3408192140 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:47.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.354+0000 7f787e800640 1 --2- 172.21.15.86:0/3408192140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7880162ed0 0x7f78801ecfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:47.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.354+0000 7f787de00640 1 -- 172.21.15.86:0/3408192140 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78800b3410 msgr2=0x7f78801655f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:47.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.354+0000 7f7875400640 1 --2- 172.21.15.86:0/3408192140 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78800b3410 0x7f78801655f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:47.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.354+0000 7f787de00640 1 --2- 172.21.15.86:0/3408192140 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78800b3410 0x7f78801655f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:47.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.354+0000 7f787de00640 1 -- 172.21.15.86:0/3408192140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7880162ed0 msgr2=0x7f78801ecfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:47.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.354+0000 7f787de00640 1 --2- 172.21.15.86:0/3408192140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7880162ed0 0x7f78801ecfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:47.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.354+0000 7f787de00640 1 -- 172.21.15.86:0/3408192140 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7880165b30 con 0x7f788006c090 2024-04-20T21:27:47.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.354+0000 7f787e800640 1 --2- 172.21.15.86:0/3408192140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7880162ed0 0x7f78801ecfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:27:47.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.354+0000 7f787de00640 1 --2- 172.21.15.86:0/3408192140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f788006c090 0x7f788015c540 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f787000ba50 tx=0x7f787000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:47.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.354+0000 7f7876a00640 1 -- 172.21.15.86:0/3408192140 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7870019070 con 0x7f788006c090 2024-04-20T21:27:47.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.354+0000 7f7876a00640 1 -- 172.21.15.86:0/3408192140 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f787000cc10 con 0x7f788006c090 2024-04-20T21:27:47.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.354+0000 7f7884c00640 1 -- 172.21.15.86:0/3408192140 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78801ec100 con 0x7f788006c090 2024-04-20T21:27:47.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.354+0000 7f7876a00640 1 -- 172.21.15.86:0/3408192140 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7870014440 con 0x7f788006c090 2024-04-20T21:27:47.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.355+0000 7f7884c00640 1 -- 172.21.15.86:0/3408192140 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78801ec6c0 con 0x7f788006c090 2024-04-20T21:27:47.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.356+0000 7f7884c00640 1 -- 172.21.15.86:0/3408192140 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78800689e0 con 0x7f788006c090 2024-04-20T21:27:47.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.357+0000 7f7876a00640 1 -- 172.21.15.86:0/3408192140 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f787001c070 con 0x7f788006c090 2024-04-20T21:27:47.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.357+0000 7f7876a00640 1 --2- 172.21.15.86:0/3408192140 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f785c06c8f0 0x7f785c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:47.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.358+0000 7f7876a00640 1 -- 172.21.15.86:0/3408192140 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7f787008b980 con 0x7f788006c090 2024-04-20T21:27:47.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.358+0000 7f7875400640 1 --2- 172.21.15.86:0/3408192140 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f785c06c8f0 0x7f785c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:47.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.359+0000 7f7875400640 1 --2- 172.21.15.86:0/3408192140 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f785c06c8f0 0x7f785c06edb0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f7868036000 tx=0x7f786803c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:47.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.363+0000 7f7876a00640 1 -- 172.21.15.86:0/3408192140 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7880165b30 con 0x7f788006c090 2024-04-20T21:27:47.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:47.585+0000 7f7884c00640 1 -- 172.21.15.86:0/3408192140 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f7880165d40 con 0x7f788006c090 2024-04-20T21:27:47.830 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:27:47.838 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:27:48.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.281+0000 7f7876a00640 1 -- 172.21.15.86:0/3408192140 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v81) ==== 118+0+0 (secure 0 0 0) 0x7f7870059da0 con 0x7f788006c090 2024-04-20T21:27:48.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.285+0000 7f7884c00640 1 -- 172.21.15.86:0/3408192140 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f785c06c8f0 msgr2=0x7f785c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:48.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.285+0000 7f7884c00640 1 --2- 172.21.15.86:0/3408192140 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f785c06c8f0 0x7f785c06edb0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f7868036000 tx=0x7f786803c040 comp rx=0 tx=0).stop 2024-04-20T21:27:48.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.285+0000 7f7884c00640 1 -- 172.21.15.86:0/3408192140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f788006c090 msgr2=0x7f788015c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:48.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.285+0000 7f7884c00640 1 --2- 172.21.15.86:0/3408192140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f788006c090 0x7f788015c540 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f787000ba50 tx=0x7f787000bf10 comp rx=0 tx=0).stop 2024-04-20T21:27:48.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.285+0000 7f7884c00640 1 -- 172.21.15.86:0/3408192140 shutdown_connections 2024-04-20T21:27:48.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.285+0000 7f7884c00640 1 --2- 172.21.15.86:0/3408192140 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f785c06c8f0 0x7f785c06edb0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:48.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.285+0000 7f7884c00640 1 --2- 172.21.15.86:0/3408192140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7880162ed0 0x7f78801ecfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:48.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.285+0000 7f7884c00640 1 --2- 172.21.15.86:0/3408192140 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78800b3410 0x7f78801655f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:48.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.285+0000 7f7884c00640 1 --2- 172.21.15.86:0/3408192140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f788006c090 0x7f788015c540 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:48.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.285+0000 7f7884c00640 1 -- 172.21.15.86:0/3408192140 >> 172.21.15.86:0/3408192140 conn(0x7f78800b2110 msgr2=0x7f78800cd360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:48.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.285+0000 7f7884c00640 1 -- 172.21.15.86:0/3408192140 shutdown_connections 2024-04-20T21:27:48.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.285+0000 7f7884c00640 1 -- 172.21.15.86:0/3408192140 wait complete. 2024-04-20T21:27:48.306 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-20T21:27:48.306 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-20T21:27:48.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.511+0000 7f13c0e00640 1 Processor -- start 2024-04-20T21:27:48.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.511+0000 7f13c0e00640 1 -- start start 2024-04-20T21:27:48.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.512+0000 7f13c0e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f13bc0b3ba0 0x7f13bc06eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:48.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.512+0000 7f13c0e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13bc06f800 con 0x7f13bc0cfc70 2024-04-20T21:27:48.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.512+0000 7f13c0e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13bc0cddb0 con 0x7f13bc06f3f0 2024-04-20T21:27:48.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.512+0000 7f13c0e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f13bc15c8e0 con 0x7f13bc0b3ba0 2024-04-20T21:27:48.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.512+0000 7f13bb400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f13bc0cfc70 0x7f13bc0cd6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57370/0 (socket says 172.21.15.86:57370) 2024-04-20T21:27:48.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.512+0000 7f13bb400640 1 -- 172.21.15.86:0/2839919729 learned_addr learned my addr 172.21.15.86:0/2839919729 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:27:48.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.512+0000 7f13b2a00640 1 --2- 172.21.15.86:0/2839919729 >> v2:172.21.15.86:3300/0 conn(0x7f13bc0b3ba0 0x7f13bc06eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:48.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.513+0000 7f13baa00640 1 -- 172.21.15.86:0/2839919729 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4290532290 0 0) 0x7f13ac001110 con 0x7f13bc0cfc70 2024-04-20T21:27:48.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.513+0000 7f13baa00640 1 -- 172.21.15.86:0/2839919729 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f13a0005730 con 0x7f13bc0cfc70 2024-04-20T21:27:48.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.513+0000 7f13b2a00640 1 -- 172.21.15.86:0/2839919729 >> v1:172.21.15.144:6789/0 conn(0x7f13bc06f3f0 legacy=0x7f13bc071ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:48.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.513+0000 7f13b2a00640 1 -- 172.21.15.86:0/2839919729 >> v1:172.21.15.86:6789/0 conn(0x7f13bc0cfc70 legacy=0x7f13bc0cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:48.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.513+0000 7f13b2a00640 1 -- 172.21.15.86:0/2839919729 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13bc15bcd0 con 0x7f13bc0b3ba0 2024-04-20T21:27:48.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.514+0000 7f13b2a00640 1 --2- 172.21.15.86:0/2839919729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13bc0b3ba0 0x7f13bc06eeb0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f13a4009d90 tx=0x7f13a402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4605a4d3a0fb2e0f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:48.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.515+0000 7f13baa00640 1 -- 172.21.15.86:0/2839919729 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a4039070 con 0x7f13bc0b3ba0 2024-04-20T21:27:48.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.515+0000 7f13baa00640 1 -- 172.21.15.86:0/2839919729 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13a402cc60 con 0x7f13bc0b3ba0 2024-04-20T21:27:48.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.515+0000 7f13baa00640 1 -- 172.21.15.86:0/2839919729 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a4034660 con 0x7f13bc0b3ba0 2024-04-20T21:27:48.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.515+0000 7f13c0e00640 1 -- 172.21.15.86:0/2839919729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13bc0b3ba0 msgr2=0x7f13bc06eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:48.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.515+0000 7f13c0e00640 1 --2- 172.21.15.86:0/2839919729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13bc0b3ba0 0x7f13bc06eeb0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f13a4009d90 tx=0x7f13a402beb0 comp rx=0 tx=0).stop 2024-04-20T21:27:48.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.515+0000 7f13c0e00640 1 -- 172.21.15.86:0/2839919729 shutdown_connections 2024-04-20T21:27:48.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.515+0000 7f13c0e00640 1 --2- 172.21.15.86:0/2839919729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13bc0b3ba0 0x7f13bc06eeb0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:48.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.515+0000 7f13c0e00640 1 -- 172.21.15.86:0/2839919729 >> 172.21.15.86:0/2839919729 conn(0x7f13bc067cf0 msgr2=0x7f13bc06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:48.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.515+0000 7f13c0e00640 1 -- 172.21.15.86:0/2839919729 shutdown_connections 2024-04-20T21:27:48.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.516+0000 7f13c0e00640 1 -- 172.21.15.86:0/2839919729 wait complete. 2024-04-20T21:27:48.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.517+0000 7f13c0e00640 1 Processor -- start 2024-04-20T21:27:48.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.517+0000 7f13c0e00640 1 -- start start 2024-04-20T21:27:48.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.518+0000 7f13c0e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13bc06f3f0 0x7f13bc14ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:48.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.518+0000 7f13c0e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13bc0b3ba0 0x7f13bc1f02d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:48.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.518+0000 7f13c0e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13bc0cfc70 0x7f13bc1e4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:48.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.518+0000 7f13c0e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc15c570 con 0x7f13bc0cfc70 2024-04-20T21:27:48.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.518+0000 7f13c0e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc15c290 con 0x7f13bc06f3f0 2024-04-20T21:27:48.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.518+0000 7f13c0e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f13bc0b3710 con 0x7f13bc0b3ba0 2024-04-20T21:27:48.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.519+0000 7f13bbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13bc0cfc70 0x7f13bc1e4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:48.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.519+0000 7f13bbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13bc0cfc70 0x7f13bc1e4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36542/0 (socket says 172.21.15.86:36542) 2024-04-20T21:27:48.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.519+0000 7f13bbe00640 1 -- 172.21.15.86:0/2640349140 learned_addr learned my addr 172.21.15.86:0/2640349140 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:48.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.519+0000 7f13bb400640 1 --2- 172.21.15.86:0/2640349140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13bc06f3f0 0x7f13bc14ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:48.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.519+0000 7f13b2a00640 1 --2- 172.21.15.86:0/2640349140 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13bc0b3ba0 0x7f13bc1f02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:48.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.519+0000 7f13bbe00640 1 -- 172.21.15.86:0/2640349140 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13bc0b3ba0 msgr2=0x7f13bc1f02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:48.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.519+0000 7f13bbe00640 1 --2- 172.21.15.86:0/2640349140 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13bc0b3ba0 0x7f13bc1f02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:48.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.519+0000 7f13bbe00640 1 -- 172.21.15.86:0/2640349140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13bc06f3f0 msgr2=0x7f13bc14ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:48.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.519+0000 7f13bbe00640 1 --2- 172.21.15.86:0/2640349140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13bc06f3f0 0x7f13bc14ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:48.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.519+0000 7f13bbe00640 1 -- 172.21.15.86:0/2640349140 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13bc152c50 con 0x7f13bc0cfc70 2024-04-20T21:27:48.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.519+0000 7f13bbe00640 1 --2- 172.21.15.86:0/2640349140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13bc0cfc70 0x7f13bc1e4ed0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f13b400b410 tx=0x7f13b400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:48.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.520+0000 7f13b3e00640 1 -- 172.21.15.86:0/2640349140 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b4015030 con 0x7f13bc0cfc70 2024-04-20T21:27:48.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.520+0000 7f13b3e00640 1 -- 172.21.15.86:0/2640349140 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13b4004020 con 0x7f13bc0cfc70 2024-04-20T21:27:48.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.520+0000 7f13b3e00640 1 -- 172.21.15.86:0/2640349140 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b4004410 con 0x7f13bc0cfc70 2024-04-20T21:27:48.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.520+0000 7f13c0e00640 1 -- 172.21.15.86:0/2640349140 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13bc1e5a20 con 0x7f13bc0cfc70 2024-04-20T21:27:48.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.520+0000 7f13c0e00640 1 -- 172.21.15.86:0/2640349140 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13bc154420 con 0x7f13bc0cfc70 2024-04-20T21:27:48.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.521+0000 7f13c0e00640 1 -- 172.21.15.86:0/2640349140 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13bc0ce6b0 con 0x7f13bc0cfc70 2024-04-20T21:27:48.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.522+0000 7f13b3e00640 1 -- 172.21.15.86:0/2640349140 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f13b4018070 con 0x7f13bc0cfc70 2024-04-20T21:27:48.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.523+0000 7f13b3e00640 1 --2- 172.21.15.86:0/2640349140 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f139806c920 0x7f139806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:48.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.523+0000 7f13b3e00640 1 -- 172.21.15.86:0/2640349140 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7f13b408b600 con 0x7f13bc0cfc70 2024-04-20T21:27:48.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.523+0000 7f13bb400640 1 --2- 172.21.15.86:0/2640349140 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f139806c920 0x7f139806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:48.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.524+0000 7f13bb400640 1 --2- 172.21.15.86:0/2640349140 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f139806c920 0x7f139806ede0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f13ac007fd0 tx=0x7f13ac009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:48.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.528+0000 7f13b3e00640 1 -- 172.21.15.86:0/2640349140 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f13bc0ce6b0 con 0x7f13bc0cfc70 2024-04-20T21:27:48.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:48.769+0000 7f13c0e00640 1 -- 172.21.15.86:0/2640349140 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f13bc04fd90 con 0x7f13bc0cfc70 2024-04-20T21:27:49.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.284+0000 7f13b3e00640 1 -- 172.21.15.86:0/2640349140 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v82) ==== 122+0+0 (secure 0 0 0) 0x7f13b4059a20 con 0x7f13bc0cfc70 2024-04-20T21:27:49.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.288+0000 7f13c0e00640 1 -- 172.21.15.86:0/2640349140 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f139806c920 msgr2=0x7f139806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:49.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.288+0000 7f13c0e00640 1 --2- 172.21.15.86:0/2640349140 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f139806c920 0x7f139806ede0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f13ac007fd0 tx=0x7f13ac009040 comp rx=0 tx=0).stop 2024-04-20T21:27:49.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.288+0000 7f13c0e00640 1 -- 172.21.15.86:0/2640349140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13bc0cfc70 msgr2=0x7f13bc1e4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:49.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.288+0000 7f13c0e00640 1 --2- 172.21.15.86:0/2640349140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13bc0cfc70 0x7f13bc1e4ed0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f13b400b410 tx=0x7f13b400b8d0 comp rx=0 tx=0).stop 2024-04-20T21:27:49.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.289+0000 7f13c0e00640 1 -- 172.21.15.86:0/2640349140 shutdown_connections 2024-04-20T21:27:49.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.289+0000 7f13c0e00640 1 --2- 172.21.15.86:0/2640349140 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f139806c920 0x7f139806ede0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:49.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.289+0000 7f13c0e00640 1 --2- 172.21.15.86:0/2640349140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13bc0cfc70 0x7f13bc1e4ed0 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:49.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.289+0000 7f13c0e00640 1 --2- 172.21.15.86:0/2640349140 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13bc0b3ba0 0x7f13bc1f02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:49.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.289+0000 7f13c0e00640 1 --2- 172.21.15.86:0/2640349140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13bc06f3f0 0x7f13bc14ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:49.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.289+0000 7f13c0e00640 1 -- 172.21.15.86:0/2640349140 >> 172.21.15.86:0/2640349140 conn(0x7f13bc067cf0 msgr2=0x7f13bc15b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:49.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.289+0000 7f13c0e00640 1 -- 172.21.15.86:0/2640349140 shutdown_connections 2024-04-20T21:27:49.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.289+0000 7f13c0e00640 1 -- 172.21.15.86:0/2640349140 wait complete. 2024-04-20T21:27:49.310 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-20T21:27:49.310 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:27:49.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.509+0000 7f6cbda00640 1 Processor -- start 2024-04-20T21:27:49.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.509+0000 7f6cbda00640 1 -- start start 2024-04-20T21:27:49.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.510+0000 7f6cbda00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6cb8156ae0 0x7f6cb815d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:49.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.510+0000 7f6cbda00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6cb8160080 0x7f6cb80ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:49.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.511+0000 7f6cbda00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cb8160890 con 0x7f6cb80ad620 2024-04-20T21:27:49.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.511+0000 7f6cbda00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f6cb806db20 con 0x7f6cb8156ae0 2024-04-20T21:27:49.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.511+0000 7f6cbda00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6cb8160b70 con 0x7f6cb8160080 2024-04-20T21:27:49.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.511+0000 7f6cbd000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f6cb80ad620 0x7f6cb8141cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57374/0 (socket says 172.21.15.86:57374) 2024-04-20T21:27:49.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.511+0000 7f6cbd000640 1 -- 172.21.15.86:0/389841845 learned_addr learned my addr 172.21.15.86:0/389841845 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:27:49.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.511+0000 7f6cb7e00640 1 --2- 172.21.15.86:0/389841845 >> v2:172.21.15.144:3300/0 conn(0x7f6cb8156ae0 0x7f6cb815d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:49.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.511+0000 7f6caf400640 1 --2- 172.21.15.86:0/389841845 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6cb8160080 0x7f6cb80ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:49.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.511+0000 7f6cb7400640 1 -- 172.21.15.86:0/389841845 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3489751818 0 0) 0x7f6cb8160890 con 0x7f6cb80ad620 2024-04-20T21:27:49.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.512+0000 7f6cb7400640 1 -- 172.21.15.86:0/389841845 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c98003520 con 0x7f6cb80ad620 2024-04-20T21:27:49.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.512+0000 7f6cb7400640 1 -- 172.21.15.86:0/389841845 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3045989762 0 0) 0x7f6c98003520 con 0x7f6cb80ad620 2024-04-20T21:27:49.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.512+0000 7f6cb7400640 1 -- 172.21.15.86:0/389841845 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6cb8160080 msgr2=0x7f6cb80ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:49.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.512+0000 7f6cb7400640 1 --2- 172.21.15.86:0/389841845 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6cb8160080 0x7f6cb80ad0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:49.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.512+0000 7f6cb7400640 1 -- 172.21.15.86:0/389841845 >> v2:172.21.15.144:3300/0 conn(0x7f6cb8156ae0 msgr2=0x7f6cb815d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:49.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.512+0000 7f6cb7400640 1 --2- 172.21.15.86:0/389841845 >> v2:172.21.15.144:3300/0 conn(0x7f6cb8156ae0 0x7f6cb815d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:49.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.512+0000 7f6cb7400640 1 -- 172.21.15.86:0/389841845 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cb8160e50 con 0x7f6cb80ad620 2024-04-20T21:27:49.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.512+0000 7f6cb7400640 1 -- 172.21.15.86:0/389841845 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f6cb00019f0 con 0x7f6cb80ad620 2024-04-20T21:27:49.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.512+0000 7f6cb7400640 1 -- 172.21.15.86:0/389841845 >> v1:172.21.15.86:6789/0 conn(0x7f6cb80ad620 legacy=0x7f6cb8141cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:49.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.512+0000 7f6cb7400640 1 --2- 172.21.15.86:0/389841845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c980039b0 0x7f6c98003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:49.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.512+0000 7f6cb7400640 1 --2- 172.21.15.86:0/389841845 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6cb8160080 0x7f6c98004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:49.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.512+0000 7f6cb7400640 1 --2- 172.21.15.86:0/389841845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c98004d40 0x7f6c980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:49.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.512+0000 7f6cb7400640 1 -- 172.21.15.86:0/389841845 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb806db20 con 0x7f6c980039b0 2024-04-20T21:27:49.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.512+0000 7f6cb7400640 1 -- 172.21.15.86:0/389841845 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb8160b70 con 0x7f6c98004d40 2024-04-20T21:27:49.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.512+0000 7f6cb7400640 1 -- 172.21.15.86:0/389841845 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6c9800b9c0 con 0x7f6cb8160080 2024-04-20T21:27:49.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.512+0000 7f6cb7e00640 1 --2- 172.21.15.86:0/389841845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c980039b0 0x7f6c98003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:49.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.513+0000 7f6cbd000640 1 --2- 172.21.15.86:0/389841845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c98004d40 0x7f6c980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:49.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.513+0000 7f6caf400640 1 --2- 172.21.15.86:0/389841845 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6cb8160080 0x7f6c98004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:49.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.513+0000 7f6cb7e00640 1 -- 172.21.15.86:0/389841845 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6cb8160080 msgr2=0x7f6c98004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:49.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.513+0000 7f6cb7e00640 1 --2- 172.21.15.86:0/389841845 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6cb8160080 0x7f6c98004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:49.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.513+0000 7f6cb7e00640 1 -- 172.21.15.86:0/389841845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c98004d40 msgr2=0x7f6c980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:49.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.513+0000 7f6cb7e00640 1 --2- 172.21.15.86:0/389841845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c98004d40 0x7f6c980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:49.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.513+0000 7f6cb7e00640 1 -- 172.21.15.86:0/389841845 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c98003520 con 0x7f6c980039b0 2024-04-20T21:27:49.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.513+0000 7f6caf400640 1 --2- 172.21.15.86:0/389841845 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6cb8160080 0x7f6c98004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:27:49.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.514+0000 7f6cb7e00640 1 --2- 172.21.15.86:0/389841845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c980039b0 0x7f6c98003dd0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8009ac0 tx=0x7f6c98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3ab7ce9f0814340 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:49.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.514+0000 7f6cb7400640 1 -- 172.21.15.86:0/389841845 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca8039070 con 0x7f6c980039b0 2024-04-20T21:27:49.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.514+0000 7f6cb7400640 1 -- 172.21.15.86:0/389841845 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ca802c710 con 0x7f6c980039b0 2024-04-20T21:27:49.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.514+0000 7f6cb7400640 1 -- 172.21.15.86:0/389841845 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca8009690 con 0x7f6c980039b0 2024-04-20T21:27:49.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.515+0000 7f6cbda00640 1 -- 172.21.15.86:0/389841845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c980039b0 msgr2=0x7f6c98003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:49.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.515+0000 7f6cbda00640 1 --2- 172.21.15.86:0/389841845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c980039b0 0x7f6c98003dd0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8009ac0 tx=0x7f6c98005830 comp rx=0 tx=0).stop 2024-04-20T21:27:49.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.515+0000 7f6cbd000640 1 -- 172.21.15.86:0/389841845 reap_dead start 2024-04-20T21:27:49.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.515+0000 7f6cbda00640 1 -- 172.21.15.86:0/389841845 shutdown_connections 2024-04-20T21:27:49.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.515+0000 7f6cbda00640 1 -- 172.21.15.86:0/389841845 >> 172.21.15.86:0/389841845 conn(0x7f6cb80d0170 msgr2=0x7f6cb80d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:49.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.515+0000 7f6cbda00640 1 -- 172.21.15.86:0/389841845 shutdown_connections 2024-04-20T21:27:49.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.515+0000 7f6cbda00640 1 -- 172.21.15.86:0/389841845 wait complete. 2024-04-20T21:27:49.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.517+0000 7f6cbda00640 1 Processor -- start 2024-04-20T21:27:49.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.517+0000 7f6cbda00640 1 -- start start 2024-04-20T21:27:49.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.518+0000 7f6cbda00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c980039b0 0x7f6cb8099880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:49.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.518+0000 7f6cbda00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6cb815c110 0x7f6cb81549d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:49.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.518+0000 7f6cbda00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6cb8154f10 0x7f6cb815a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:49.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.518+0000 7f6cbda00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb8160500 con 0x7f6cb815c110 2024-04-20T21:27:49.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.518+0000 7f6cbda00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb8156ec0 con 0x7f6c980039b0 2024-04-20T21:27:49.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.518+0000 7f6cbda00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6cb806f2a0 con 0x7f6cb8154f10 2024-04-20T21:27:49.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.518+0000 7f6caf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6cb815c110 0x7f6cb81549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:49.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.518+0000 7f6caf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6cb815c110 0x7f6cb81549d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36558/0 (socket says 172.21.15.86:36558) 2024-04-20T21:27:49.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.518+0000 7f6caf400640 1 -- 172.21.15.86:0/2353391480 learned_addr learned my addr 172.21.15.86:0/2353391480 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:49.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.519+0000 7f6cb7e00640 1 --2- 172.21.15.86:0/2353391480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c980039b0 0x7f6cb8099880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:49.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.519+0000 7f6cbd000640 1 --2- 172.21.15.86:0/2353391480 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6cb8154f10 0x7f6cb815a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:49.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.519+0000 7f6caf400640 1 -- 172.21.15.86:0/2353391480 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6cb8154f10 msgr2=0x7f6cb815a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:49.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.519+0000 7f6caf400640 1 --2- 172.21.15.86:0/2353391480 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6cb8154f10 0x7f6cb815a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:49.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.519+0000 7f6caf400640 1 -- 172.21.15.86:0/2353391480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c980039b0 msgr2=0x7f6cb8099880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:49.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.519+0000 7f6caf400640 1 --2- 172.21.15.86:0/2353391480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c980039b0 0x7f6cb8099880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:49.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.519+0000 7f6caf400640 1 -- 172.21.15.86:0/2353391480 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cb8071dc0 con 0x7f6cb815c110 2024-04-20T21:27:49.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.519+0000 7f6caf400640 1 --2- 172.21.15.86:0/2353391480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6cb815c110 0x7f6cb81549d0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f6ca00108e0 tx=0x7f6ca0010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:49.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.519+0000 7f6cb4c00640 1 -- 172.21.15.86:0/2353391480 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca000f040 con 0x7f6cb815c110 2024-04-20T21:27:49.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.519+0000 7f6cb4c00640 1 -- 172.21.15.86:0/2353391480 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ca000ec20 con 0x7f6cb815c110 2024-04-20T21:27:49.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.519+0000 7f6cb4c00640 1 -- 172.21.15.86:0/2353391480 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca001c400 con 0x7f6cb815c110 2024-04-20T21:27:49.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.519+0000 7f6cbda00640 1 -- 172.21.15.86:0/2353391480 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cb80720a0 con 0x7f6cb815c110 2024-04-20T21:27:49.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.520+0000 7f6cbda00640 1 -- 172.21.15.86:0/2353391480 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cb8072660 con 0x7f6cb815c110 2024-04-20T21:27:49.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.521+0000 7f6cbda00640 1 -- 172.21.15.86:0/2353391480 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cb804fcf0 con 0x7f6cb815c110 2024-04-20T21:27:49.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.522+0000 7f6cb4c00640 1 -- 172.21.15.86:0/2353391480 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6ca001c5a0 con 0x7f6cb815c110 2024-04-20T21:27:49.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.523+0000 7f6cb4c00640 1 --2- 172.21.15.86:0/2353391480 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6c9406c8f0 0x7f6c9406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:49.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.523+0000 7f6cb4c00640 1 -- 172.21.15.86:0/2353391480 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7f6ca008c3d0 con 0x7f6cb815c110 2024-04-20T21:27:49.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.523+0000 7f6cb7e00640 1 --2- 172.21.15.86:0/2353391480 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6c9406c8f0 0x7f6c9406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:49.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.524+0000 7f6cb7e00640 1 --2- 172.21.15.86:0/2353391480 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6c9406c8f0 0x7f6c9406edb0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8036000 tx=0x7f6ca8030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:49.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.527+0000 7f6cb4c00640 1 -- 172.21.15.86:0/2353391480 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6ca0018030 con 0x7f6cb815c110 2024-04-20T21:27:49.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.770+0000 7f6cbda00640 1 -- 172.21.15.86:0/2353391480 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6cb81552f0 con 0x7f6cb815c110 2024-04-20T21:27:49.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.771+0000 7f6cb4c00640 1 -- 172.21.15.86:0/2353391480 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5207 (secure 0 0 0) 0x7f6ca005a760 con 0x7f6cb815c110 2024-04-20T21:27:49.770 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:27:49.770 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":82,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5525,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3902778887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3902778887},{"type":"v1","addr":"172.21.15.144:6835","nonce":3902778887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6001,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/4270988014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":4270988014},{"type":"v1","addr":"172.21.15.86:6837","nonce":4270988014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6016,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/1569462554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":1569462554},{"type":"v1","addr":"172.21.15.144:6833","nonce":1569462554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:27:45.692408+0000","modified":"2024-04-20T21:27:48.770936+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8493":{"gid":8493,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6836/1331851138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1331851138},{"type":"v1","addr":"172.21.15.86:6836","nonce":1331851138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8493,"qdb_cluster":[8493]},"id":7}]} 2024-04-20T21:27:49.770 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 82 2024-04-20T21:27:49.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.774+0000 7f6cbda00640 1 -- 172.21.15.86:0/2353391480 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6c9406c8f0 msgr2=0x7f6c9406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:49.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.774+0000 7f6cbda00640 1 --2- 172.21.15.86:0/2353391480 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6c9406c8f0 0x7f6c9406edb0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8036000 tx=0x7f6ca8030000 comp rx=0 tx=0).stop 2024-04-20T21:27:49.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.774+0000 7f6cbda00640 1 -- 172.21.15.86:0/2353391480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6cb815c110 msgr2=0x7f6cb81549d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:49.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.774+0000 7f6cbda00640 1 --2- 172.21.15.86:0/2353391480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6cb815c110 0x7f6cb81549d0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f6ca00108e0 tx=0x7f6ca0010da0 comp rx=0 tx=0).stop 2024-04-20T21:27:49.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.774+0000 7f6cbda00640 1 -- 172.21.15.86:0/2353391480 shutdown_connections 2024-04-20T21:27:49.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.774+0000 7f6cbda00640 1 --2- 172.21.15.86:0/2353391480 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6c9406c8f0 0x7f6c9406edb0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:49.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.774+0000 7f6cbda00640 1 --2- 172.21.15.86:0/2353391480 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6cb8154f10 0x7f6cb815a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:49.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.774+0000 7f6cbda00640 1 --2- 172.21.15.86:0/2353391480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6cb815c110 0x7f6cb81549d0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:49.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.774+0000 7f6cbda00640 1 --2- 172.21.15.86:0/2353391480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c980039b0 0x7f6cb8099880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:49.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.774+0000 7f6cbda00640 1 -- 172.21.15.86:0/2353391480 >> 172.21.15.86:0/2353391480 conn(0x7f6cb80d0170 msgr2=0x7f6cb80b2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:49.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.774+0000 7f6cbda00640 1 -- 172.21.15.86:0/2353391480 shutdown_connections 2024-04-20T21:27:49.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.774+0000 7f6cbda00640 1 -- 172.21.15.86:0/2353391480 wait complete. 2024-04-20T21:27:49.793 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:27:49.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.990+0000 7fe809a00640 1 Processor -- start 2024-04-20T21:27:49.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.990+0000 7fe809a00640 1 -- start start 2024-04-20T21:27:49.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.992+0000 7fe809a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe804146130 0x7fe80406e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:49.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.992+0000 7fe809a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe804072410 0x7fe8040cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:49.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.992+0000 7fe809a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe80406dc30 con 0x7fe8040ccdd0 2024-04-20T21:27:49.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.992+0000 7fe809a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe8040cd8e0 con 0x7fe804146130 2024-04-20T21:27:49.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.992+0000 7fe809a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fe8040cdbc0 con 0x7fe804072410 2024-04-20T21:27:49.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.992+0000 7fe802a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe804146130 0x7fe80406e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:49.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.992+0000 7fe7fa000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe8040ccdd0 0x7fe804071d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57378/0 (socket says 172.21.15.86:57378) 2024-04-20T21:27:49.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.992+0000 7fe7fa000640 1 -- 172.21.15.86:0/3258239915 learned_addr learned my addr 172.21.15.86:0/3258239915 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:27:49.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.993+0000 7fe803400640 1 --2- 172.21.15.86:0/3258239915 >> v2:172.21.15.144:3300/0 conn(0x7fe804072410 0x7fe8040cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:49.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.993+0000 7fe802a00640 1 -- 172.21.15.86:0/3258239915 >> v2:172.21.15.144:3300/0 conn(0x7fe804072410 msgr2=0x7fe8040cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:49.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.993+0000 7fe802a00640 1 --2- 172.21.15.86:0/3258239915 >> v2:172.21.15.144:3300/0 conn(0x7fe804072410 0x7fe8040cd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:49.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.993+0000 7fe802a00640 1 -- 172.21.15.86:0/3258239915 >> v1:172.21.15.86:6789/0 conn(0x7fe8040ccdd0 legacy=0x7fe804071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:49.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.993+0000 7fe802000640 1 -- 172.21.15.86:0/3258239915 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3151260665 0 0) 0x7fe80406dc30 con 0x7fe8040ccdd0 2024-04-20T21:27:49.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.993+0000 7fe802a00640 1 -- 172.21.15.86:0/3258239915 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe80406ded0 con 0x7fe804146130 2024-04-20T21:27:49.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.994+0000 7fe802a00640 1 --2- 172.21.15.86:0/3258239915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe804146130 0x7fe80406e6d0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fe804071c90 tx=0x7fe7f402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ded2d14c4a0e1aaa server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:49.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.995+0000 7fe802000640 1 -- 172.21.15.86:0/3258239915 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4034070 con 0x7fe804146130 2024-04-20T21:27:49.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.995+0000 7fe802000640 1 -- 172.21.15.86:0/3258239915 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7f4030610 con 0x7fe804146130 2024-04-20T21:27:49.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.995+0000 7fe802000640 1 -- 172.21.15.86:0/3258239915 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f40309e0 con 0x7fe804146130 2024-04-20T21:27:49.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.995+0000 7fe809a00640 1 -- 172.21.15.86:0/3258239915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe804146130 msgr2=0x7fe80406e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:49.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.995+0000 7fe809a00640 1 --2- 172.21.15.86:0/3258239915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe804146130 0x7fe80406e6d0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fe804071c90 tx=0x7fe7f402b960 comp rx=0 tx=0).stop 2024-04-20T21:27:49.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.995+0000 7fe809a00640 1 -- 172.21.15.86:0/3258239915 shutdown_connections 2024-04-20T21:27:49.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.995+0000 7fe809a00640 1 --2- 172.21.15.86:0/3258239915 >> v2:172.21.15.144:3300/0 conn(0x7fe804072410 0x7fe8040cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:49.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.995+0000 7fe809a00640 1 --2- 172.21.15.86:0/3258239915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe804146130 0x7fe80406e6d0 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:49.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.995+0000 7fe809a00640 1 -- 172.21.15.86:0/3258239915 >> 172.21.15.86:0/3258239915 conn(0x7fe8040b2360 msgr2=0x7fe8040b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:49.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.995+0000 7fe809a00640 1 -- 172.21.15.86:0/3258239915 shutdown_connections 2024-04-20T21:27:49.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.996+0000 7fe809a00640 1 -- 172.21.15.86:0/3258239915 wait complete. 2024-04-20T21:27:49.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.997+0000 7fe809a00640 1 Processor -- start 2024-04-20T21:27:49.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.997+0000 7fe809a00640 1 -- start start 2024-04-20T21:27:49.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.998+0000 7fe809a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe804072410 0x7fe8040ae630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:49.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.998+0000 7fe809a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8040ccdd0 0x7fe8040abda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:49.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.998+0000 7fe809a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe804146130 0x7fe8041e6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:49.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.998+0000 7fe809a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe80415cc20 con 0x7fe8040ccdd0 2024-04-20T21:27:49.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.998+0000 7fe809a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe80406f150 con 0x7fe804146130 2024-04-20T21:27:49.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.998+0000 7fe809a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe80415cda0 con 0x7fe804072410 2024-04-20T21:27:49.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.998+0000 7fe7fa000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8040ccdd0 0x7fe8040abda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:49.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.999+0000 7fe7fa000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8040ccdd0 0x7fe8040abda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36574/0 (socket says 172.21.15.86:36574) 2024-04-20T21:27:49.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.999+0000 7fe7fa000640 1 -- 172.21.15.86:0/3115213090 learned_addr learned my addr 172.21.15.86:0/3115213090 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:49.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.999+0000 7fe802a00640 1 --2- 172.21.15.86:0/3115213090 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe804072410 0x7fe8040ae630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:49.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.999+0000 7fe803400640 1 --2- 172.21.15.86:0/3115213090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe804146130 0x7fe8041e6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:49.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.999+0000 7fe7fa000640 1 -- 172.21.15.86:0/3115213090 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe804072410 msgr2=0x7fe8040ae630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:49.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.999+0000 7fe7fa000640 1 --2- 172.21.15.86:0/3115213090 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe804072410 0x7fe8040ae630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:49.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.999+0000 7fe7fa000640 1 -- 172.21.15.86:0/3115213090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe804146130 msgr2=0x7fe8041e6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:49.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.999+0000 7fe7fa000640 1 --2- 172.21.15.86:0/3115213090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe804146130 0x7fe8041e6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:49.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.999+0000 7fe7fa000640 1 -- 172.21.15.86:0/3115213090 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8040af4f0 con 0x7fe8040ccdd0 2024-04-20T21:27:49.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.999+0000 7fe802a00640 1 --2- 172.21.15.86:0/3115213090 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe804072410 0x7fe8040ae630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:27:49.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.999+0000 7fe7fa000640 1 --2- 172.21.15.86:0/3115213090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8040ccdd0 0x7fe8040abda0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec00c4d0 tx=0x7fe7ec00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:50.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:49.999+0000 7fe7fb400640 1 -- 172.21.15.86:0/3115213090 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec017070 con 0x7fe8040ccdd0 2024-04-20T21:27:50.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.000+0000 7fe7fb400640 1 -- 172.21.15.86:0/3115213090 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7ec005d00 con 0x7fe8040ccdd0 2024-04-20T21:27:50.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.000+0000 7fe7fb400640 1 -- 172.21.15.86:0/3115213090 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec0072b0 con 0x7fe8040ccdd0 2024-04-20T21:27:50.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.000+0000 7fe809a00640 1 -- 172.21.15.86:0/3115213090 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8040aef30 con 0x7fe8040ccdd0 2024-04-20T21:27:50.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.000+0000 7fe809a00640 1 -- 172.21.15.86:0/3115213090 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8041e6110 con 0x7fe8040ccdd0 2024-04-20T21:27:50.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.001+0000 7fe809a00640 1 -- 172.21.15.86:0/3115213090 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8040689e0 con 0x7fe8040ccdd0 2024-04-20T21:27:50.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.002+0000 7fe7fb400640 1 -- 172.21.15.86:0/3115213090 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe7ec01a070 con 0x7fe8040ccdd0 2024-04-20T21:27:50.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.003+0000 7fe7fb400640 1 --2- 172.21.15.86:0/3115213090 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe7e006c920 0x7fe7e006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:50.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.003+0000 7fe7fb400640 1 -- 172.21.15.86:0/3115213090 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7fe7ec08af40 con 0x7fe8040ccdd0 2024-04-20T21:27:50.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.003+0000 7fe802a00640 1 --2- 172.21.15.86:0/3115213090 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe7e006c920 0x7fe7e006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:50.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.004+0000 7fe802a00640 1 --2- 172.21.15.86:0/3115213090 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe7e006c920 0x7fe7e006ede0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4036000 tx=0x7fe7f4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:50.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.007+0000 7fe7fb400640 1 -- 172.21.15.86:0/3115213090 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe7ec0592d0 con 0x7fe8040ccdd0 2024-04-20T21:27:50.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.208+0000 7fe809a00640 1 -- 172.21.15.86:0/3115213090 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe8040afea0 con 0x7fe8040ccdd0 2024-04-20T21:27:50.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.210+0000 7fe7fb400640 1 -- 172.21.15.86:0/3115213090 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+16610 (secure 0 0 0) 0x7fe7ec058bd0 con 0x7fe8040ccdd0 2024-04-20T21:27:50.209 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:27:50.209 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":100,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:27:47.267402+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:27:43.469685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-20T21:27:44.673505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:27:50.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.212+0000 7fe809a00640 1 -- 172.21.15.86:0/3115213090 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe7e006c920 msgr2=0x7fe7e006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:50.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.212+0000 7fe809a00640 1 --2- 172.21.15.86:0/3115213090 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe7e006c920 0x7fe7e006ede0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4036000 tx=0x7fe7f4008030 comp rx=0 tx=0).stop 2024-04-20T21:27:50.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.212+0000 7fe809a00640 1 -- 172.21.15.86:0/3115213090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8040ccdd0 msgr2=0x7fe8040abda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:50.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.212+0000 7fe809a00640 1 --2- 172.21.15.86:0/3115213090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8040ccdd0 0x7fe8040abda0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec00c4d0 tx=0x7fe7ec00c990 comp rx=0 tx=0).stop 2024-04-20T21:27:50.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.213+0000 7fe809a00640 1 -- 172.21.15.86:0/3115213090 shutdown_connections 2024-04-20T21:27:50.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.213+0000 7fe809a00640 1 --2- 172.21.15.86:0/3115213090 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe7e006c920 0x7fe7e006ede0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:50.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.213+0000 7fe809a00640 1 --2- 172.21.15.86:0/3115213090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe804146130 0x7fe8041e6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:50.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.213+0000 7fe809a00640 1 --2- 172.21.15.86:0/3115213090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8040ccdd0 0x7fe8040abda0 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:50.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.213+0000 7fe809a00640 1 --2- 172.21.15.86:0/3115213090 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe804072410 0x7fe8040ae630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:50.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.213+0000 7fe809a00640 1 -- 172.21.15.86:0/3115213090 >> 172.21.15.86:0/3115213090 conn(0x7fe8040b2360 msgr2=0x7fe804144cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:50.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.213+0000 7fe809a00640 1 -- 172.21.15.86:0/3115213090 shutdown_connections 2024-04-20T21:27:50.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.213+0000 7fe809a00640 1 -- 172.21.15.86:0/3115213090 wait complete. 2024-04-20T21:27:50.229 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-20T21:27:50.230 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T21:27:50.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.432+0000 7fe9d6800640 1 Processor -- start 2024-04-20T21:27:50.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.433+0000 7fe9d6800640 1 -- start start 2024-04-20T21:27:50.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.434+0000 7fe9d6800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe9d006f140 0x7fe9d00aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:50.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.434+0000 7fe9d6800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe9d00af0b0 0x7fe9d0071d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:50.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.434+0000 7fe9d6800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9d006dc30 con 0x7fe9d0144e90 2024-04-20T21:27:50.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.434+0000 7fe9d6800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe9d0145270 con 0x7fe9d00af0b0 2024-04-20T21:27:50.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.434+0000 7fe9d6800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fe9d015f720 con 0x7fe9d006f140 2024-04-20T21:27:50.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.435+0000 7fe9cfe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe9d00af0b0 0x7fe9d0071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:50.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.435+0000 7fe9cfe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe9d00af0b0 0x7fe9d0071d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36584/0 (socket says 172.21.15.86:36584) 2024-04-20T21:27:50.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.435+0000 7fe9cfe00640 1 -- 172.21.15.86:0/500600752 learned_addr learned my addr 172.21.15.86:0/500600752 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:50.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.435+0000 7fe9c6a00640 1 --2- 172.21.15.86:0/500600752 >> v2:172.21.15.144:3300/0 conn(0x7fe9d006f140 0x7fe9d00aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:50.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.435+0000 7fe9cfe00640 1 -- 172.21.15.86:0/500600752 >> v2:172.21.15.144:3300/0 conn(0x7fe9d006f140 msgr2=0x7fe9d00aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:50.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.435+0000 7fe9cfe00640 1 --2- 172.21.15.86:0/500600752 >> v2:172.21.15.144:3300/0 conn(0x7fe9d006f140 0x7fe9d00aeb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:50.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.435+0000 7fe9cfe00640 1 -- 172.21.15.86:0/500600752 >> v1:172.21.15.144:6789/0 conn(0x7fe9d0144e90 legacy=0x7fe9d015ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:50.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.435+0000 7fe9cfe00640 1 -- 172.21.15.86:0/500600752 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9d015fa00 con 0x7fe9d00af0b0 2024-04-20T21:27:50.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.436+0000 7fe9cfe00640 1 --2- 172.21.15.86:0/500600752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9d00af0b0 0x7fe9d0071d00 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fe9d00735c0 tx=0x7fe9c802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=476417685aea4be3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:50.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.437+0000 7fe9cea00640 1 -- 172.21.15.86:0/500600752 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c802ca80 con 0x7fe9d00af0b0 2024-04-20T21:27:50.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.437+0000 7fe9cea00640 1 -- 172.21.15.86:0/500600752 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9c802cc20 con 0x7fe9d00af0b0 2024-04-20T21:27:50.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.437+0000 7fe9cea00640 1 -- 172.21.15.86:0/500600752 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c80343f0 con 0x7fe9d00af0b0 2024-04-20T21:27:50.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.437+0000 7fe9d6800640 1 -- 172.21.15.86:0/500600752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9d00af0b0 msgr2=0x7fe9d0071d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:50.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.437+0000 7fe9d6800640 1 --2- 172.21.15.86:0/500600752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9d00af0b0 0x7fe9d0071d00 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fe9d00735c0 tx=0x7fe9c802bee0 comp rx=0 tx=0).stop 2024-04-20T21:27:50.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.438+0000 7fe9d6800640 1 -- 172.21.15.86:0/500600752 shutdown_connections 2024-04-20T21:27:50.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.438+0000 7fe9d6800640 1 --2- 172.21.15.86:0/500600752 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9d00af0b0 0x7fe9d0071d00 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:50.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.438+0000 7fe9d6800640 1 --2- 172.21.15.86:0/500600752 >> v2:172.21.15.144:3300/0 conn(0x7fe9d006f140 0x7fe9d00aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:50.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.438+0000 7fe9d6800640 1 -- 172.21.15.86:0/500600752 >> 172.21.15.86:0/500600752 conn(0x7fe9d00b2360 msgr2=0x7fe9d00b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:50.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.438+0000 7fe9d6800640 1 -- 172.21.15.86:0/500600752 shutdown_connections 2024-04-20T21:27:50.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.438+0000 7fe9d6800640 1 -- 172.21.15.86:0/500600752 wait complete. 2024-04-20T21:27:50.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.439+0000 7fe9d6800640 1 Processor -- start 2024-04-20T21:27:50.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.440+0000 7fe9d6800640 1 -- start start 2024-04-20T21:27:50.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.440+0000 7fe9d6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9d006f140 0x7fe9d013be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:50.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.440+0000 7fe9d6800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9d00af0b0 0x7fe9d013f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:50.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.440+0000 7fe9d6800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9d0144e90 0x7fe9d01434d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:50.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.440+0000 7fe9d6800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d0072240 con 0x7fe9d006f140 2024-04-20T21:27:50.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.440+0000 7fe9d6800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d006ded0 con 0x7fe9d0144e90 2024-04-20T21:27:50.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.440+0000 7fe9d6800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe9d00723c0 con 0x7fe9d00af0b0 2024-04-20T21:27:50.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.441+0000 7fe9cf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9d006f140 0x7fe9d013be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:50.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.441+0000 7fe9cf400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9d006f140 0x7fe9d013be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36594/0 (socket says 172.21.15.86:36594) 2024-04-20T21:27:50.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.441+0000 7fe9cf400640 1 -- 172.21.15.86:0/2136371030 learned_addr learned my addr 172.21.15.86:0/2136371030 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:50.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.441+0000 7fe9cf400640 1 -- 172.21.15.86:0/2136371030 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9d00af0b0 msgr2=0x7fe9d013f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:50.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.441+0000 7fe9cf400640 1 --2- 172.21.15.86:0/2136371030 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9d00af0b0 0x7fe9d013f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:50.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.441+0000 7fe9cf400640 1 -- 172.21.15.86:0/2136371030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9d0144e90 msgr2=0x7fe9d01434d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:50.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.441+0000 7fe9cf400640 1 --2- 172.21.15.86:0/2136371030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9d0144e90 0x7fe9d01434d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:50.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.441+0000 7fe9cf400640 1 -- 172.21.15.86:0/2136371030 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9d00cd400 con 0x7fe9d006f140 2024-04-20T21:27:50.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.441+0000 7fe9cf400640 1 --2- 172.21.15.86:0/2136371030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9d006f140 0x7fe9d013be00 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fe9c000b3f0 tx=0x7fe9c000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:50.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.442+0000 7fe9c7e00640 1 -- 172.21.15.86:0/2136371030 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0015020 con 0x7fe9d006f140 2024-04-20T21:27:50.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.442+0000 7fe9c7e00640 1 -- 172.21.15.86:0/2136371030 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9c0011830 con 0x7fe9d006f140 2024-04-20T21:27:50.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.442+0000 7fe9c7e00640 1 -- 172.21.15.86:0/2136371030 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0011c20 con 0x7fe9d006f140 2024-04-20T21:27:50.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.442+0000 7fe9d6800640 1 -- 172.21.15.86:0/2136371030 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9d00cd6e0 con 0x7fe9d006f140 2024-04-20T21:27:50.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.442+0000 7fe9d6800640 1 -- 172.21.15.86:0/2136371030 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9d00cdca0 con 0x7fe9d006f140 2024-04-20T21:27:50.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.443+0000 7fe9d6800640 1 -- 172.21.15.86:0/2136371030 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9d00b2740 con 0x7fe9d006f140 2024-04-20T21:27:50.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.444+0000 7fe9c7e00640 1 -- 172.21.15.86:0/2136371030 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe9c0018020 con 0x7fe9d006f140 2024-04-20T21:27:50.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.445+0000 7fe9c7e00640 1 --2- 172.21.15.86:0/2136371030 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe9ac06c8f0 0x7fe9ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:50.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.445+0000 7fe9c7e00640 1 -- 172.21.15.86:0/2136371030 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7fe9c008b850 con 0x7fe9d006f140 2024-04-20T21:27:50.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.445+0000 7fe9c6a00640 1 --2- 172.21.15.86:0/2136371030 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe9ac06c8f0 0x7fe9ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:50.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.446+0000 7fe9c6a00640 1 --2- 172.21.15.86:0/2136371030 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe9ac06c8f0 0x7fe9ac06edb0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8006fd0 tx=0x7fe9b8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:50.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.450+0000 7fe9c7e00640 1 -- 172.21.15.86:0/2136371030 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe9d00b2740 con 0x7fe9d006f140 2024-04-20T21:27:50.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.648+0000 7fe9d6800640 1 -- 172.21.15.86:0/2136371030 --> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe9d00689e0 con 0x7fe9ac06c8f0 2024-04-20T21:27:50.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.668+0000 7fe9c7e00640 1 -- 172.21.15.86:0/2136371030 <== mgr.4107 v2:172.21.15.86:6832/302045240 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246768 (secure 0 0 0) 0x7fe9d00689e0 con 0x7fe9ac06c8f0 2024-04-20T21:27:50.670 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:27:50.671 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-20T21:27:50.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.673+0000 7fe9d6800640 1 -- 172.21.15.86:0/2136371030 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe9ac06c8f0 msgr2=0x7fe9ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:50.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.673+0000 7fe9d6800640 1 --2- 172.21.15.86:0/2136371030 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe9ac06c8f0 0x7fe9ac06edb0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8006fd0 tx=0x7fe9b8008040 comp rx=0 tx=0).stop 2024-04-20T21:27:50.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.673+0000 7fe9d6800640 1 -- 172.21.15.86:0/2136371030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9d006f140 msgr2=0x7fe9d013be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:50.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.673+0000 7fe9d6800640 1 --2- 172.21.15.86:0/2136371030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9d006f140 0x7fe9d013be00 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fe9c000b3f0 tx=0x7fe9c000b8b0 comp rx=0 tx=0).stop 2024-04-20T21:27:50.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.673+0000 7fe9d6800640 1 -- 172.21.15.86:0/2136371030 shutdown_connections 2024-04-20T21:27:50.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.673+0000 7fe9d6800640 1 --2- 172.21.15.86:0/2136371030 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe9ac06c8f0 0x7fe9ac06edb0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:50.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.673+0000 7fe9d6800640 1 --2- 172.21.15.86:0/2136371030 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe9d0144e90 0x7fe9d01434d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:50.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.673+0000 7fe9d6800640 1 --2- 172.21.15.86:0/2136371030 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe9d00af0b0 0x7fe9d013f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:50.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.673+0000 7fe9d6800640 1 --2- 172.21.15.86:0/2136371030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9d006f140 0x7fe9d013be00 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:50.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.673+0000 7fe9d6800640 1 -- 172.21.15.86:0/2136371030 >> 172.21.15.86:0/2136371030 conn(0x7fe9d00b2360 msgr2=0x7fe9d00b1460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:50.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.674+0000 7fe9d6800640 1 -- 172.21.15.86:0/2136371030 shutdown_connections 2024-04-20T21:27:50.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.674+0000 7fe9d6800640 1 -- 172.21.15.86:0/2136371030 wait complete. 2024-04-20T21:27:50.693 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":397,"stamp":"2024-04-20T21:27:48.842764+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":30,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260032,"kb_used_data":10584,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714688,"statfs":{"total":773094113280,"available":772827840512,"internally_reserved":0,"allocated":10838016,"data_stored":5330612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.628456"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.238919+0000","last_change":"2024-04-20T21:27:45.238919+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.238919+0000","last_undegraded":"2024-04-20T21:27:45.238919+0000","last_fullsized":"2024-04-20T21:27:45.238919+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.243025+0000","last_change":"2024-04-20T21:27:45.243025+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.243025+0000","last_undegraded":"2024-04-20T21:27:45.243025+0000","last_fullsized":"2024-04-20T21:27:45.243025+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.235891+0000","last_change":"2024-04-20T21:27:45.235891+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.235891+0000","last_undegraded":"2024-04-20T21:27:45.235891+0000","last_fullsized":"2024-04-20T21:27:45.235891+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.228960+0000","last_change":"2024-04-20T21:27:45.228960+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.228960+0000","last_undegraded":"2024-04-20T21:27:45.228960+0000","last_fullsized":"2024-04-20T21:27:45.228960+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.235632+0000","last_change":"2024-04-20T21:27:45.235632+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.235632+0000","last_undegraded":"2024-04-20T21:27:45.235632+0000","last_fullsized":"2024-04-20T21:27:45.235632+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.241913+0000","last_change":"2024-04-20T21:27:45.241913+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.241913+0000","last_undegraded":"2024-04-20T21:27:45.241913+0000","last_fullsized":"2024-04-20T21:27:45.241913+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.241757+0000","last_change":"2024-04-20T21:27:45.241757+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.241757+0000","last_undegraded":"2024-04-20T21:27:45.241757+0000","last_fullsized":"2024-04-20T21:27:45.241757+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.238197+0000","last_change":"2024-04-20T21:27:45.238197+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.238197+0000","last_undegraded":"2024-04-20T21:27:45.238197+0000","last_fullsized":"2024-04-20T21:27:45.238197+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.238424+0000","last_change":"2024-04-20T21:27:45.238424+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.238424+0000","last_undegraded":"2024-04-20T21:27:45.238424+0000","last_fullsized":"2024-04-20T21:27:45.238424+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.234642+0000","last_change":"2024-04-20T21:27:45.234642+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.234642+0000","last_undegraded":"2024-04-20T21:27:45.234642+0000","last_fullsized":"2024-04-20T21:27:45.234642+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.244845+0000","last_change":"2024-04-20T21:27:45.244845+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.244845+0000","last_undegraded":"2024-04-20T21:27:45.244845+0000","last_fullsized":"2024-04-20T21:27:45.244845+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.232143+0000","last_change":"2024-04-20T21:27:45.232143+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.232143+0000","last_undegraded":"2024-04-20T21:27:45.232143+0000","last_fullsized":"2024-04-20T21:27:45.232143+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.232008+0000","last_change":"2024-04-20T21:27:45.232008+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.232008+0000","last_undegraded":"2024-04-20T21:27:45.232008+0000","last_fullsized":"2024-04-20T21:27:45.232008+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.231743+0000","last_change":"2024-04-20T21:27:45.231743+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.231743+0000","last_undegraded":"2024-04-20T21:27:45.231743+0000","last_fullsized":"2024-04-20T21:27:45.231743+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.229621+0000","last_change":"2024-04-20T21:27:45.229621+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.229621+0000","last_undegraded":"2024-04-20T21:27:45.229621+0000","last_fullsized":"2024-04-20T21:27:45.229621+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.246898+0000","last_change":"2024-04-20T21:27:45.246898+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.246898+0000","last_undegraded":"2024-04-20T21:27:45.246898+0000","last_fullsized":"2024-04-20T21:27:45.246898+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.244936+0000","last_change":"2024-04-20T21:27:45.244936+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.244936+0000","last_undegraded":"2024-04-20T21:27:45.244936+0000","last_fullsized":"2024-04-20T21:27:45.244936+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.231623+0000","last_change":"2024-04-20T21:27:45.231623+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.231623+0000","last_undegraded":"2024-04-20T21:27:45.231623+0000","last_fullsized":"2024-04-20T21:27:45.231623+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.236300+0000","last_change":"2024-04-20T21:27:45.236300+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.236300+0000","last_undegraded":"2024-04-20T21:27:45.236300+0000","last_fullsized":"2024-04-20T21:27:45.236300+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.231041+0000","last_change":"2024-04-20T21:27:45.231041+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.231041+0000","last_undegraded":"2024-04-20T21:27:45.231041+0000","last_fullsized":"2024-04-20T21:27:45.231041+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.230799+0000","last_change":"2024-04-20T21:27:45.230799+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.230799+0000","last_undegraded":"2024-04-20T21:27:45.230799+0000","last_fullsized":"2024-04-20T21:27:45.230799+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.239045+0000","last_change":"2024-04-20T21:27:45.239045+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.239045+0000","last_undegraded":"2024-04-20T21:27:45.239045+0000","last_fullsized":"2024-04-20T21:27:45.239045+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.240004+0000","last_change":"2024-04-20T21:27:45.240004+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.240004+0000","last_undegraded":"2024-04-20T21:27:45.240004+0000","last_fullsized":"2024-04-20T21:27:45.240004+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.244779+0000","last_change":"2024-04-20T21:27:45.244779+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.244779+0000","last_undegraded":"2024-04-20T21:27:45.244779+0000","last_fullsized":"2024-04-20T21:27:45.244779+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.234234+0000","last_change":"2024-04-20T21:27:45.234234+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.234234+0000","last_undegraded":"2024-04-20T21:27:45.234234+0000","last_fullsized":"2024-04-20T21:27:45.234234+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.242621+0000","last_change":"2024-04-20T21:27:45.242621+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.242621+0000","last_undegraded":"2024-04-20T21:27:45.242621+0000","last_fullsized":"2024-04-20T21:27:45.242621+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.244137+0000","last_change":"2024-04-20T21:27:45.244137+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.244137+0000","last_undegraded":"2024-04-20T21:27:45.244137+0000","last_fullsized":"2024-04-20T21:27:45.244137+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.233903+0000","last_change":"2024-04-20T21:27:45.233903+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.233903+0000","last_undegraded":"2024-04-20T21:27:45.233903+0000","last_fullsized":"2024-04-20T21:27:45.233903+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.235958+0000","last_change":"2024-04-20T21:27:45.235958+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.235958+0000","last_undegraded":"2024-04-20T21:27:45.235958+0000","last_fullsized":"2024-04-20T21:27:45.235958+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.231264+0000","last_change":"2024-04-20T21:27:45.231264+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.231264+0000","last_undegraded":"2024-04-20T21:27:45.231264+0000","last_fullsized":"2024-04-20T21:27:45.231264+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.228957+0000","last_change":"2024-04-20T21:27:45.228957+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.228957+0000","last_undegraded":"2024-04-20T21:27:45.228957+0000","last_fullsized":"2024-04-20T21:27:45.228957+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.234740+0000","last_change":"2024-04-20T21:27:45.234740+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.234740+0000","last_undegraded":"2024-04-20T21:27:45.234740+0000","last_fullsized":"2024-04-20T21:27:45.234740+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.229491+0000","last_change":"2024-04-20T21:27:45.229491+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.229491+0000","last_undegraded":"2024-04-20T21:27:45.229491+0000","last_fullsized":"2024-04-20T21:27:45.229491+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-20T21:27:45.259614+0000","last_change":"2024-04-20T21:27:45.258515+0000","last_active":"2024-04-20T21:27:45.259614+0000","last_peered":"2024-04-20T21:27:45.259614+0000","last_clean":"2024-04-20T21:27:45.259614+0000","last_became_active":"2024-04-20T21:27:45.258254+0000","last_became_peered":"2024-04-20T21:27:45.258254+0000","last_unstale":"2024-04-20T21:27:45.259614+0000","last_undegraded":"2024-04-20T21:27:45.259614+0000","last_fullsized":"2024-04-20T21:27:45.259614+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:47:50.127355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.229531+0000","last_change":"2024-04-20T21:27:45.229531+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.229531+0000","last_undegraded":"2024-04-20T21:27:45.229531+0000","last_fullsized":"2024-04-20T21:27:45.229531+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-20T21:27:45.261889+0000","last_change":"2024-04-20T21:27:45.258703+0000","last_active":"2024-04-20T21:27:45.261889+0000","last_peered":"2024-04-20T21:27:45.261889+0000","last_clean":"2024-04-20T21:27:45.261889+0000","last_became_active":"2024-04-20T21:27:45.257807+0000","last_became_peered":"2024-04-20T21:27:45.257807+0000","last_unstale":"2024-04-20T21:27:45.261889+0000","last_undegraded":"2024-04-20T21:27:45.261889+0000","last_fullsized":"2024-04-20T21:27:45.261889+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:47:50.127355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.235063+0000","last_change":"2024-04-20T21:27:45.235063+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.235063+0000","last_undegraded":"2024-04-20T21:27:45.235063+0000","last_fullsized":"2024-04-20T21:27:45.235063+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-20T21:27:45.257746+0000","last_change":"2024-04-20T21:27:45.255784+0000","last_active":"2024-04-20T21:27:45.257746+0000","last_peered":"2024-04-20T21:27:45.257746+0000","last_clean":"2024-04-20T21:27:45.257746+0000","last_became_active":"2024-04-20T21:27:45.255462+0000","last_became_peered":"2024-04-20T21:27:45.255462+0000","last_unstale":"2024-04-20T21:27:45.257746+0000","last_undegraded":"2024-04-20T21:27:45.257746+0000","last_fullsized":"2024-04-20T21:27:45.257746+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:34:06.321344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.233292+0000","last_change":"2024-04-20T21:27:45.233292+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.233292+0000","last_undegraded":"2024-04-20T21:27:45.233292+0000","last_fullsized":"2024-04-20T21:27:45.233292+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-20T21:27:45.258635+0000","last_change":"2024-04-20T21:27:45.257220+0000","last_active":"2024-04-20T21:27:45.258635+0000","last_peered":"2024-04-20T21:27:45.258635+0000","last_clean":"2024-04-20T21:27:45.258635+0000","last_became_active":"2024-04-20T21:27:45.256799+0000","last_became_peered":"2024-04-20T21:27:45.256799+0000","last_unstale":"2024-04-20T21:27:45.258635+0000","last_undegraded":"2024-04-20T21:27:45.258635+0000","last_fullsized":"2024-04-20T21:27:45.258635+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:39:48.372277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.245199+0000","last_change":"2024-04-20T21:27:45.245199+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.245199+0000","last_undegraded":"2024-04-20T21:27:45.245199+0000","last_fullsized":"2024-04-20T21:27:45.245199+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-20T21:27:45.253111+0000","last_change":"2024-04-20T21:27:45.251360+0000","last_active":"2024-04-20T21:27:45.253111+0000","last_peered":"2024-04-20T21:27:45.253111+0000","last_clean":"2024-04-20T21:27:45.253111+0000","last_became_active":"2024-04-20T21:27:45.251076+0000","last_became_peered":"2024-04-20T21:27:45.251076+0000","last_unstale":"2024-04-20T21:27:45.253111+0000","last_undegraded":"2024-04-20T21:27:45.253111+0000","last_fullsized":"2024-04-20T21:27:45.253111+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:37:35.288484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.241597+0000","last_change":"2024-04-20T21:27:45.241597+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.241597+0000","last_undegraded":"2024-04-20T21:27:45.241597+0000","last_fullsized":"2024-04-20T21:27:45.241597+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-20T21:27:45.255084+0000","last_change":"2024-04-20T21:27:45.249660+0000","last_active":"2024-04-20T21:27:45.255084+0000","last_peered":"2024-04-20T21:27:45.255084+0000","last_clean":"2024-04-20T21:27:45.255084+0000","last_became_active":"2024-04-20T21:27:45.248485+0000","last_became_peered":"2024-04-20T21:27:45.248485+0000","last_unstale":"2024-04-20T21:27:45.255084+0000","last_undegraded":"2024-04-20T21:27:45.255084+0000","last_fullsized":"2024-04-20T21:27:45.255084+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:17:33.718683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.228770+0000","last_change":"2024-04-20T21:27:45.228770+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.228770+0000","last_undegraded":"2024-04-20T21:27:45.228770+0000","last_fullsized":"2024-04-20T21:27:45.228770+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-20T21:27:45.259008+0000","last_change":"2024-04-20T21:27:45.255056+0000","last_active":"2024-04-20T21:27:45.259008+0000","last_peered":"2024-04-20T21:27:45.259008+0000","last_clean":"2024-04-20T21:27:45.259008+0000","last_became_active":"2024-04-20T21:27:45.252809+0000","last_became_peered":"2024-04-20T21:27:45.252809+0000","last_unstale":"2024-04-20T21:27:45.259008+0000","last_undegraded":"2024-04-20T21:27:45.259008+0000","last_fullsized":"2024-04-20T21:27:45.259008+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:57:44.315268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.229882+0000","last_change":"2024-04-20T21:27:45.229882+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.229882+0000","last_undegraded":"2024-04-20T21:27:45.229882+0000","last_fullsized":"2024-04-20T21:27:45.229882+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-20T21:27:45.248003+0000","last_change":"2024-04-20T21:27:45.247265+0000","last_active":"2024-04-20T21:27:45.248003+0000","last_peered":"2024-04-20T21:27:45.248003+0000","last_clean":"2024-04-20T21:27:45.248003+0000","last_became_active":"2024-04-20T21:27:45.245173+0000","last_became_peered":"2024-04-20T21:27:45.245173+0000","last_unstale":"2024-04-20T21:27:45.248003+0000","last_undegraded":"2024-04-20T21:27:45.248003+0000","last_fullsized":"2024-04-20T21:27:45.248003+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:57:13.780436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.242177+0000","last_change":"2024-04-20T21:27:45.242177+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.242177+0000","last_undegraded":"2024-04-20T21:27:45.242177+0000","last_fullsized":"2024-04-20T21:27:45.242177+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-20T21:27:45.259154+0000","last_change":"2024-04-20T21:27:45.257108+0000","last_active":"2024-04-20T21:27:45.259154+0000","last_peered":"2024-04-20T21:27:45.259154+0000","last_clean":"2024-04-20T21:27:45.259154+0000","last_became_active":"2024-04-20T21:27:45.256824+0000","last_became_peered":"2024-04-20T21:27:45.256824+0000","last_unstale":"2024-04-20T21:27:45.259154+0000","last_undegraded":"2024-04-20T21:27:45.259154+0000","last_fullsized":"2024-04-20T21:27:45.259154+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:25:55.834101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.237302+0000","last_change":"2024-04-20T21:27:45.237302+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.237302+0000","last_undegraded":"2024-04-20T21:27:45.237302+0000","last_fullsized":"2024-04-20T21:27:45.237302+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-20T21:27:45.253472+0000","last_change":"2024-04-20T21:27:45.251360+0000","last_active":"2024-04-20T21:27:45.253472+0000","last_peered":"2024-04-20T21:27:45.253472+0000","last_clean":"2024-04-20T21:27:45.253472+0000","last_became_active":"2024-04-20T21:27:45.251057+0000","last_became_peered":"2024-04-20T21:27:45.251057+0000","last_unstale":"2024-04-20T21:27:45.253472+0000","last_undegraded":"2024-04-20T21:27:45.253472+0000","last_fullsized":"2024-04-20T21:27:45.253472+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:34:06.321344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.232581+0000","last_change":"2024-04-20T21:27:45.232581+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.232581+0000","last_undegraded":"2024-04-20T21:27:45.232581+0000","last_fullsized":"2024-04-20T21:27:45.232581+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-20T21:27:45.261910+0000","last_change":"2024-04-20T21:27:45.259658+0000","last_active":"2024-04-20T21:27:45.261910+0000","last_peered":"2024-04-20T21:27:45.261910+0000","last_clean":"2024-04-20T21:27:45.261910+0000","last_became_active":"2024-04-20T21:27:45.259277+0000","last_became_peered":"2024-04-20T21:27:45.259277+0000","last_unstale":"2024-04-20T21:27:45.261910+0000","last_undegraded":"2024-04-20T21:27:45.261910+0000","last_fullsized":"2024-04-20T21:27:45.261910+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:26:14.971064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.243379+0000","last_change":"2024-04-20T21:27:45.243379+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.243379+0000","last_undegraded":"2024-04-20T21:27:45.243379+0000","last_fullsized":"2024-04-20T21:27:45.243379+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-20T21:27:45.256007+0000","last_change":"2024-04-20T21:27:45.245837+0000","last_active":"2024-04-20T21:27:45.256007+0000","last_peered":"2024-04-20T21:27:45.256007+0000","last_clean":"2024-04-20T21:27:45.256007+0000","last_became_active":"2024-04-20T21:27:45.245491+0000","last_became_peered":"2024-04-20T21:27:45.245491+0000","last_unstale":"2024-04-20T21:27:45.256007+0000","last_undegraded":"2024-04-20T21:27:45.256007+0000","last_fullsized":"2024-04-20T21:27:45.256007+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:08:33.098011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.230167+0000","last_change":"2024-04-20T21:27:45.230167+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.230167+0000","last_undegraded":"2024-04-20T21:27:45.230167+0000","last_fullsized":"2024-04-20T21:27:45.230167+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-20T21:27:45.255937+0000","last_change":"2024-04-20T21:27:45.254404+0000","last_active":"2024-04-20T21:27:45.255937+0000","last_peered":"2024-04-20T21:27:45.255937+0000","last_clean":"2024-04-20T21:27:45.255937+0000","last_became_active":"2024-04-20T21:27:45.254086+0000","last_became_peered":"2024-04-20T21:27:45.254086+0000","last_unstale":"2024-04-20T21:27:45.255937+0000","last_undegraded":"2024-04-20T21:27:45.255937+0000","last_fullsized":"2024-04-20T21:27:45.255937+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:50:29.101884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.237297+0000","last_change":"2024-04-20T21:27:45.237297+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.237297+0000","last_undegraded":"2024-04-20T21:27:45.237297+0000","last_fullsized":"2024-04-20T21:27:45.237297+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-20T21:27:45.255297+0000","last_change":"2024-04-20T21:27:45.250257+0000","last_active":"2024-04-20T21:27:45.255297+0000","last_peered":"2024-04-20T21:27:45.255297+0000","last_clean":"2024-04-20T21:27:45.255297+0000","last_became_active":"2024-04-20T21:27:45.249980+0000","last_became_peered":"2024-04-20T21:27:45.249980+0000","last_unstale":"2024-04-20T21:27:45.255297+0000","last_undegraded":"2024-04-20T21:27:45.255297+0000","last_fullsized":"2024-04-20T21:27:45.255297+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:21:57.962556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.228869+0000","last_change":"2024-04-20T21:27:45.228869+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.228869+0000","last_undegraded":"2024-04-20T21:27:45.228869+0000","last_fullsized":"2024-04-20T21:27:45.228869+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-20T21:27:45.261936+0000","last_change":"2024-04-20T21:27:45.256573+0000","last_active":"2024-04-20T21:27:45.261936+0000","last_peered":"2024-04-20T21:27:45.261936+0000","last_clean":"2024-04-20T21:27:45.261936+0000","last_became_active":"2024-04-20T21:27:45.256161+0000","last_became_peered":"2024-04-20T21:27:45.256161+0000","last_unstale":"2024-04-20T21:27:45.261936+0000","last_undegraded":"2024-04-20T21:27:45.261936+0000","last_fullsized":"2024-04-20T21:27:45.261936+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:26:14.971064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.229054+0000","last_change":"2024-04-20T21:27:45.229054+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.229054+0000","last_undegraded":"2024-04-20T21:27:45.229054+0000","last_fullsized":"2024-04-20T21:27:45.229054+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-20T21:27:45.258848+0000","last_change":"2024-04-20T21:27:45.254036+0000","last_active":"2024-04-20T21:27:45.258848+0000","last_peered":"2024-04-20T21:27:45.258848+0000","last_clean":"2024-04-20T21:27:45.258848+0000","last_became_active":"2024-04-20T21:27:45.253759+0000","last_became_peered":"2024-04-20T21:27:45.253759+0000","last_unstale":"2024-04-20T21:27:45.258848+0000","last_undegraded":"2024-04-20T21:27:45.258848+0000","last_fullsized":"2024-04-20T21:27:45.258848+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:44:16.800178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.232052+0000","last_change":"2024-04-20T21:27:45.232052+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.232052+0000","last_undegraded":"2024-04-20T21:27:45.232052+0000","last_fullsized":"2024-04-20T21:27:45.232052+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-20T21:27:45.255490+0000","last_change":"2024-04-20T21:27:45.251308+0000","last_active":"2024-04-20T21:27:45.255490+0000","last_peered":"2024-04-20T21:27:45.255490+0000","last_clean":"2024-04-20T21:27:45.255490+0000","last_became_active":"2024-04-20T21:27:45.249643+0000","last_became_peered":"2024-04-20T21:27:45.249643+0000","last_unstale":"2024-04-20T21:27:45.255490+0000","last_undegraded":"2024-04-20T21:27:45.255490+0000","last_fullsized":"2024-04-20T21:27:45.255490+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:47:50.127355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.244834+0000","last_change":"2024-04-20T21:27:45.244834+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.244834+0000","last_undegraded":"2024-04-20T21:27:45.244834+0000","last_fullsized":"2024-04-20T21:27:45.244834+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-20T21:27:45.260230+0000","last_change":"2024-04-20T21:27:45.256516+0000","last_active":"2024-04-20T21:27:45.260230+0000","last_peered":"2024-04-20T21:27:45.260230+0000","last_clean":"2024-04-20T21:27:45.260230+0000","last_became_active":"2024-04-20T21:27:45.256184+0000","last_became_peered":"2024-04-20T21:27:45.256184+0000","last_unstale":"2024-04-20T21:27:45.260230+0000","last_undegraded":"2024-04-20T21:27:45.260230+0000","last_fullsized":"2024-04-20T21:27:45.260230+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:48:50.706623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.241325+0000","last_change":"2024-04-20T21:27:45.241325+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.241325+0000","last_undegraded":"2024-04-20T21:27:45.241325+0000","last_fullsized":"2024-04-20T21:27:45.241325+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-20T21:27:45.262952+0000","last_change":"2024-04-20T21:27:45.262413+0000","last_active":"2024-04-20T21:27:45.262952+0000","last_peered":"2024-04-20T21:27:45.262952+0000","last_clean":"2024-04-20T21:27:45.262952+0000","last_became_active":"2024-04-20T21:27:45.262069+0000","last_became_peered":"2024-04-20T21:27:45.262069+0000","last_unstale":"2024-04-20T21:27:45.262952+0000","last_undegraded":"2024-04-20T21:27:45.262952+0000","last_fullsized":"2024-04-20T21:27:45.262952+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:26:14.971064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.230866+0000","last_change":"2024-04-20T21:27:45.230866+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.230866+0000","last_undegraded":"2024-04-20T21:27:45.230866+0000","last_fullsized":"2024-04-20T21:27:45.230866+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-20T21:27:45.255099+0000","last_change":"2024-04-20T21:27:45.250116+0000","last_active":"2024-04-20T21:27:45.255099+0000","last_peered":"2024-04-20T21:27:45.255099+0000","last_clean":"2024-04-20T21:27:45.255099+0000","last_became_active":"2024-04-20T21:27:45.248147+0000","last_became_peered":"2024-04-20T21:27:45.248147+0000","last_unstale":"2024-04-20T21:27:45.255099+0000","last_undegraded":"2024-04-20T21:27:45.255099+0000","last_fullsized":"2024-04-20T21:27:45.255099+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:57:43.670428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.230221+0000","last_change":"2024-04-20T21:27:45.230221+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.230221+0000","last_undegraded":"2024-04-20T21:27:45.230221+0000","last_fullsized":"2024-04-20T21:27:45.230221+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-20T21:27:45.261184+0000","last_change":"2024-04-20T21:27:45.256259+0000","last_active":"2024-04-20T21:27:45.261184+0000","last_peered":"2024-04-20T21:27:45.261184+0000","last_clean":"2024-04-20T21:27:45.261184+0000","last_became_active":"2024-04-20T21:27:45.254959+0000","last_became_peered":"2024-04-20T21:27:45.254959+0000","last_unstale":"2024-04-20T21:27:45.261184+0000","last_undegraded":"2024-04-20T21:27:45.261184+0000","last_fullsized":"2024-04-20T21:27:45.261184+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:52:11.165181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.230673+0000","last_change":"2024-04-20T21:27:45.230673+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.230673+0000","last_undegraded":"2024-04-20T21:27:45.230673+0000","last_fullsized":"2024-04-20T21:27:45.230673+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-20T21:27:45.257616+0000","last_change":"2024-04-20T21:27:45.251046+0000","last_active":"2024-04-20T21:27:45.257616+0000","last_peered":"2024-04-20T21:27:45.257616+0000","last_clean":"2024-04-20T21:27:45.257616+0000","last_became_active":"2024-04-20T21:27:45.250764+0000","last_became_peered":"2024-04-20T21:27:45.250764+0000","last_unstale":"2024-04-20T21:27:45.257616+0000","last_undegraded":"2024-04-20T21:27:45.257616+0000","last_fullsized":"2024-04-20T21:27:45.257616+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:48:50.706623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.233777+0000","last_change":"2024-04-20T21:27:45.233777+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.233777+0000","last_undegraded":"2024-04-20T21:27:45.233777+0000","last_fullsized":"2024-04-20T21:27:45.233777+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-20T21:27:45.255168+0000","last_change":"2024-04-20T21:27:45.249470+0000","last_active":"2024-04-20T21:27:45.255168+0000","last_peered":"2024-04-20T21:27:45.255168+0000","last_clean":"2024-04-20T21:27:45.255168+0000","last_became_active":"2024-04-20T21:27:45.249115+0000","last_became_peered":"2024-04-20T21:27:45.249115+0000","last_unstale":"2024-04-20T21:27:45.255168+0000","last_undegraded":"2024-04-20T21:27:45.255168+0000","last_fullsized":"2024-04-20T21:27:45.255168+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:47:52.579441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.241767+0000","last_change":"2024-04-20T21:27:45.241767+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.241767+0000","last_undegraded":"2024-04-20T21:27:45.241767+0000","last_fullsized":"2024-04-20T21:27:45.241767+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-20T21:27:45.259224+0000","last_change":"2024-04-20T21:27:45.255106+0000","last_active":"2024-04-20T21:27:45.259224+0000","last_peered":"2024-04-20T21:27:45.259224+0000","last_clean":"2024-04-20T21:27:45.259224+0000","last_became_active":"2024-04-20T21:27:45.253372+0000","last_became_peered":"2024-04-20T21:27:45.253372+0000","last_unstale":"2024-04-20T21:27:45.259224+0000","last_undegraded":"2024-04-20T21:27:45.259224+0000","last_fullsized":"2024-04-20T21:27:45.259224+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:17:01.348366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.237729+0000","last_change":"2024-04-20T21:27:45.237729+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.237729+0000","last_undegraded":"2024-04-20T21:27:45.237729+0000","last_fullsized":"2024-04-20T21:27:45.237729+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-20T21:27:45.258145+0000","last_change":"2024-04-20T21:27:45.256313+0000","last_active":"2024-04-20T21:27:45.258145+0000","last_peered":"2024-04-20T21:27:45.258145+0000","last_clean":"2024-04-20T21:27:45.258145+0000","last_became_active":"2024-04-20T21:27:45.256016+0000","last_became_peered":"2024-04-20T21:27:45.256016+0000","last_unstale":"2024-04-20T21:27:45.258145+0000","last_undegraded":"2024-04-20T21:27:45.258145+0000","last_fullsized":"2024-04-20T21:27:45.258145+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:59:01.178395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.234320+0000","last_change":"2024-04-20T21:27:45.234320+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.234320+0000","last_undegraded":"2024-04-20T21:27:45.234320+0000","last_fullsized":"2024-04-20T21:27:45.234320+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-20T21:27:45.258287+0000","last_change":"2024-04-20T21:27:45.256398+0000","last_active":"2024-04-20T21:27:45.258287+0000","last_peered":"2024-04-20T21:27:45.258287+0000","last_clean":"2024-04-20T21:27:45.258287+0000","last_became_active":"2024-04-20T21:27:45.256100+0000","last_became_peered":"2024-04-20T21:27:45.256100+0000","last_unstale":"2024-04-20T21:27:45.258287+0000","last_undegraded":"2024-04-20T21:27:45.258287+0000","last_fullsized":"2024-04-20T21:27:45.258287+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:08:33.098011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.231418+0000","last_change":"2024-04-20T21:27:45.231418+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.231418+0000","last_undegraded":"2024-04-20T21:27:45.231418+0000","last_fullsized":"2024-04-20T21:27:45.231418+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-20T21:27:45.259076+0000","last_change":"2024-04-20T21:27:45.255282+0000","last_active":"2024-04-20T21:27:45.259076+0000","last_peered":"2024-04-20T21:27:45.259076+0000","last_clean":"2024-04-20T21:27:45.259076+0000","last_became_active":"2024-04-20T21:27:45.254976+0000","last_became_peered":"2024-04-20T21:27:45.254976+0000","last_unstale":"2024-04-20T21:27:45.259076+0000","last_undegraded":"2024-04-20T21:27:45.259076+0000","last_fullsized":"2024-04-20T21:27:45.259076+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:09:14.189073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.230404+0000","last_change":"2024-04-20T21:27:45.230404+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.230404+0000","last_undegraded":"2024-04-20T21:27:45.230404+0000","last_fullsized":"2024-04-20T21:27:45.230404+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-20T21:27:45.255075+0000","last_change":"2024-04-20T21:27:45.249896+0000","last_active":"2024-04-20T21:27:45.255075+0000","last_peered":"2024-04-20T21:27:45.255075+0000","last_clean":"2024-04-20T21:27:45.255075+0000","last_became_active":"2024-04-20T21:27:45.247789+0000","last_became_peered":"2024-04-20T21:27:45.247789+0000","last_unstale":"2024-04-20T21:27:45.255075+0000","last_undegraded":"2024-04-20T21:27:45.255075+0000","last_fullsized":"2024-04-20T21:27:45.255075+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:00:49.541790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.234899+0000","last_change":"2024-04-20T21:27:45.234899+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.234899+0000","last_undegraded":"2024-04-20T21:27:45.234899+0000","last_fullsized":"2024-04-20T21:27:45.234899+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-20T21:27:45.259111+0000","last_change":"2024-04-20T21:27:45.254136+0000","last_active":"2024-04-20T21:27:45.259111+0000","last_peered":"2024-04-20T21:27:45.259111+0000","last_clean":"2024-04-20T21:27:45.259111+0000","last_became_active":"2024-04-20T21:27:45.251225+0000","last_became_peered":"2024-04-20T21:27:45.251225+0000","last_unstale":"2024-04-20T21:27:45.259111+0000","last_undegraded":"2024-04-20T21:27:45.259111+0000","last_fullsized":"2024-04-20T21:27:45.259111+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:12:11.452106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.230117+0000","last_change":"2024-04-20T21:27:45.230117+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.230117+0000","last_undegraded":"2024-04-20T21:27:45.230117+0000","last_fullsized":"2024-04-20T21:27:45.230117+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-20T21:27:45.244344+0000","last_change":"2024-04-20T21:27:45.243432+0000","last_active":"2024-04-20T21:27:45.244344+0000","last_peered":"2024-04-20T21:27:45.244344+0000","last_clean":"2024-04-20T21:27:45.244344+0000","last_became_active":"2024-04-20T21:27:45.243095+0000","last_became_peered":"2024-04-20T21:27:45.243095+0000","last_unstale":"2024-04-20T21:27:45.244344+0000","last_undegraded":"2024-04-20T21:27:45.244344+0000","last_fullsized":"2024-04-20T21:27:45.244344+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:57:43.670428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.232192+0000","last_change":"2024-04-20T21:27:45.232192+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.232192+0000","last_undegraded":"2024-04-20T21:27:45.232192+0000","last_fullsized":"2024-04-20T21:27:45.232192+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-20T21:27:45.253122+0000","last_change":"2024-04-20T21:27:45.248068+0000","last_active":"2024-04-20T21:27:45.253122+0000","last_peered":"2024-04-20T21:27:45.253122+0000","last_clean":"2024-04-20T21:27:45.253122+0000","last_became_active":"2024-04-20T21:27:45.247799+0000","last_became_peered":"2024-04-20T21:27:45.247799+0000","last_unstale":"2024-04-20T21:27:45.253122+0000","last_undegraded":"2024-04-20T21:27:45.253122+0000","last_fullsized":"2024-04-20T21:27:45.253122+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:41:05.780325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-04-20T21:27:45.228788+0000","last_change":"2024-04-20T21:27:45.228788+0000","last_active":"2024-04-20T21:27:45.212618+0000","last_peered":"2024-04-20T21:27:45.212618+0000","last_clean":"2024-04-20T21:27:45.212618+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T21:27:45.228788+0000","last_undegraded":"2024-04-20T21:27:45.228788+0000","last_fullsized":"2024-04-20T21:27:45.228788+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-20T21:27:45.261510+0000","last_change":"2024-04-20T21:27:45.258203+0000","last_active":"2024-04-20T21:27:45.261510+0000","last_peered":"2024-04-20T21:27:45.261510+0000","last_clean":"2024-04-20T21:27:45.261510+0000","last_became_active":"2024-04-20T21:27:45.257554+0000","last_became_peered":"2024-04-20T21:27:45.257554+0000","last_unstale":"2024-04-20T21:27:45.261510+0000","last_undegraded":"2024-04-20T21:27:45.261510+0000","last_fullsized":"2024-04-20T21:27:45.261510+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:39:48.372277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":312,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-20T21:27:45.247524+0000","last_change":"2024-04-20T21:17:43.059637+0000","last_active":"2024-04-20T21:27:45.247524+0000","last_peered":"2024-04-20T21:27:45.247524+0000","last_clean":"2024-04-20T21:27:45.247524+0000","last_became_active":"2024-04-20T21:17:43.056207+0000","last_became_peered":"2024-04-20T21:17:43.056207+0000","last_unstale":"2024-04-20T21:27:45.247524+0000","last_undegraded":"2024-04-20T21:27:45.247524+0000","last_fullsized":"2024-04-20T21:27:45.247524+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:41.036290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:41.036290+0000","last_clean_scrub_stamp":"2024-04-20T21:17:41.036290+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:58:51.091337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574971,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32944,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338896,"statfs":{"total":96636764160,"available":96603029504,"internally_reserved":0,"allocated":1806336,"data_stored":1112240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574972,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1200128,"data_stored":515597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574974,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32368,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339472,"statfs":{"total":96636764160,"available":96603619328,"internally_reserved":0,"allocated":1212416,"data_stored":521872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574973,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32368,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339472,"statfs":{"total":96636764160,"available":96603619328,"internally_reserved":0,"allocated":1212416,"data_stored":521872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574973,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1200128,"data_stored":515597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574973,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32364,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339476,"statfs":{"total":96636764160,"available":96603623424,"internally_reserved":0,"allocated":1212416,"data_stored":521872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574972,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1200128,"data_stored":515597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574973,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32932,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338908,"statfs":{"total":96636764160,"available":96603041792,"internally_reserved":0,"allocated":1794048,"data_stored":1105965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T21:27:50.697 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T21:27:50.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.933+0000 7fcc82800640 1 Processor -- start 2024-04-20T21:27:50.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.933+0000 7fcc82800640 1 -- start start 2024-04-20T21:27:50.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.935+0000 7fcc82800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcc7c06e6d0 0x7fcc7c071d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:50.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.935+0000 7fcc82800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc7c06eb30 con 0x7fcc7c146130 2024-04-20T21:27:50.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.935+0000 7fcc82800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc7c06dc30 con 0x7fcc7c072240 2024-04-20T21:27:50.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.935+0000 7fcc82800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fcc7c072690 con 0x7fcc7c06e6d0 2024-04-20T21:27:50.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.935+0000 7fcc7aa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcc7c06e6d0 0x7fcc7c071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:50.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.935+0000 7fcc7aa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcc7c06e6d0 0x7fcc7c071d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36608/0 (socket says 172.21.15.86:36608) 2024-04-20T21:27:50.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.935+0000 7fcc7aa00640 1 -- 172.21.15.86:0/1467376958 learned_addr learned my addr 172.21.15.86:0/1467376958 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:50.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.935+0000 7fcc7b400640 1 --1- 172.21.15.86:0/1467376958 >> v1:172.21.15.86:6789/0 conn(0x7fcc7c146130 0x7fcc7c0ccdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57388/0 (socket says 172.21.15.86:57388) 2024-04-20T21:27:50.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.936+0000 7fcc7aa00640 1 -- 172.21.15.86:0/1467376958 >> v1:172.21.15.144:6789/0 conn(0x7fcc7c072240 legacy=0x7fcc7c15cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:50.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.936+0000 7fcc7aa00640 1 -- 172.21.15.86:0/1467376958 >> v1:172.21.15.86:6789/0 conn(0x7fcc7c146130 legacy=0x7fcc7c0ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:50.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.936+0000 7fcc7aa00640 1 -- 172.21.15.86:0/1467376958 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc7c0cd8e0 con 0x7fcc7c06e6d0 2024-04-20T21:27:50.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.937+0000 7fcc7aa00640 1 --2- 172.21.15.86:0/1467376958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc7c06e6d0 0x7fcc7c071d00 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c073120 tx=0x7fcc6802ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3aa4fed162426fd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:50.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.937+0000 7fcc7a000640 1 -- 172.21.15.86:0/1467376958 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6802f830 con 0x7fcc7c06e6d0 2024-04-20T21:27:50.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.937+0000 7fcc7a000640 1 -- 172.21.15.86:0/1467376958 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc6802f9d0 con 0x7fcc7c06e6d0 2024-04-20T21:27:50.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.937+0000 7fcc7a000640 1 -- 172.21.15.86:0/1467376958 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6802fda0 con 0x7fcc7c06e6d0 2024-04-20T21:27:50.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.938+0000 7fcc82800640 1 -- 172.21.15.86:0/1467376958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc7c06e6d0 msgr2=0x7fcc7c071d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:50.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.938+0000 7fcc82800640 1 --2- 172.21.15.86:0/1467376958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc7c06e6d0 0x7fcc7c071d00 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c073120 tx=0x7fcc6802ba10 comp rx=0 tx=0).stop 2024-04-20T21:27:50.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.938+0000 7fcc82800640 1 -- 172.21.15.86:0/1467376958 shutdown_connections 2024-04-20T21:27:50.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.938+0000 7fcc82800640 1 --2- 172.21.15.86:0/1467376958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc7c06e6d0 0x7fcc7c071d00 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:50.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.938+0000 7fcc82800640 1 -- 172.21.15.86:0/1467376958 >> 172.21.15.86:0/1467376958 conn(0x7fcc7c0b2360 msgr2=0x7fcc7c0b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:50.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.938+0000 7fcc82800640 1 -- 172.21.15.86:0/1467376958 shutdown_connections 2024-04-20T21:27:50.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.938+0000 7fcc82800640 1 -- 172.21.15.86:0/1467376958 wait complete. 2024-04-20T21:27:50.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.940+0000 7fcc82800640 1 Processor -- start 2024-04-20T21:27:50.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.940+0000 7fcc82800640 1 -- start start 2024-04-20T21:27:50.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.941+0000 7fcc82800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc7c06e6d0 0x7fcc7c15b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:50.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.941+0000 7fcc82800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc7c072240 0x7fcc7c15cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:50.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.941+0000 7fcc82800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc7c146130 0x7fcc7c15d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:50.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.941+0000 7fcc82800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcc7c06e1b0 con 0x7fcc7c072240 2024-04-20T21:27:50.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.941+0000 7fcc82800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcc7c06ded0 con 0x7fcc7c06e6d0 2024-04-20T21:27:50.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.941+0000 7fcc82800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fcc7c06f150 con 0x7fcc7c146130 2024-04-20T21:27:50.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.941+0000 7fcc7aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc7c072240 0x7fcc7c15cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:50.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.941+0000 7fcc7aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc7c072240 0x7fcc7c15cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36620/0 (socket says 172.21.15.86:36620) 2024-04-20T21:27:50.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.941+0000 7fcc7aa00640 1 -- 172.21.15.86:0/2922784362 learned_addr learned my addr 172.21.15.86:0/2922784362 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:50.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.941+0000 7fcc7aa00640 1 -- 172.21.15.86:0/2922784362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc7c146130 msgr2=0x7fcc7c15d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:50.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.941+0000 7fcc7aa00640 1 --2- 172.21.15.86:0/2922784362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc7c146130 0x7fcc7c15d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:50.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.941+0000 7fcc7aa00640 1 -- 172.21.15.86:0/2922784362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc7c06e6d0 msgr2=0x7fcc7c15b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:50.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.941+0000 7fcc7aa00640 1 --2- 172.21.15.86:0/2922784362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc7c06e6d0 0x7fcc7c15b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:50.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.941+0000 7fcc7aa00640 1 -- 172.21.15.86:0/2922784362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc7c15f720 con 0x7fcc7c072240 2024-04-20T21:27:50.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.942+0000 7fcc7aa00640 1 --2- 172.21.15.86:0/2922784362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc7c072240 0x7fcc7c15cb20 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fcc68000c00 tx=0x7fcc68004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:50.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.942+0000 7fcc73400640 1 -- 172.21.15.86:0/2922784362 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6802f800 con 0x7fcc7c072240 2024-04-20T21:27:50.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.942+0000 7fcc73400640 1 -- 172.21.15.86:0/2922784362 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc68046070 con 0x7fcc7c072240 2024-04-20T21:27:50.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.942+0000 7fcc73400640 1 -- 172.21.15.86:0/2922784362 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc68041640 con 0x7fcc7c072240 2024-04-20T21:27:50.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.942+0000 7fcc82800640 1 -- 172.21.15.86:0/2922784362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc7c15fa00 con 0x7fcc7c072240 2024-04-20T21:27:50.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.942+0000 7fcc82800640 1 -- 172.21.15.86:0/2922784362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc7c13c550 con 0x7fcc7c072240 2024-04-20T21:27:50.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.944+0000 7fcc82800640 1 -- 172.21.15.86:0/2922784362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc7c13c750 con 0x7fcc7c072240 2024-04-20T21:27:50.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.944+0000 7fcc73400640 1 -- 172.21.15.86:0/2922784362 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcc68040070 con 0x7fcc7c072240 2024-04-20T21:27:50.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.945+0000 7fcc73400640 1 --2- 172.21.15.86:0/2922784362 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcc5c06c920 0x7fcc5c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:50.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.945+0000 7fcc73400640 1 -- 172.21.15.86:0/2922784362 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7fcc680ae440 con 0x7fcc7c072240 2024-04-20T21:27:50.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.946+0000 7fcc7b400640 1 --2- 172.21.15.86:0/2922784362 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcc5c06c920 0x7fcc5c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:50.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.946+0000 7fcc7b400640 1 --2- 172.21.15.86:0/2922784362 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcc5c06c920 0x7fcc5c06ede0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c000f60 tx=0x7fcc6c007a00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:50.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:50.950+0000 7fcc73400640 1 -- 172.21.15.86:0/2922784362 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fcc7c13c750 con 0x7fcc7c072240 2024-04-20T21:27:51.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.145+0000 7fcc82800640 1 -- 172.21.15.86:0/2922784362 --> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcc7c0689e0 con 0x7fcc5c06c920 2024-04-20T21:27:51.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.152+0000 7fcc73400640 1 -- 172.21.15.86:0/2922784362 <== mgr.4107 v2:172.21.15.86:6832/302045240 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252184 (secure 0 0 0) 0x7fcc7c0689e0 con 0x7fcc5c06c920 2024-04-20T21:27:51.155 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:27:51.156 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-20T21:27:51.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.158+0000 7fcc82800640 1 -- 172.21.15.86:0/2922784362 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcc5c06c920 msgr2=0x7fcc5c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:51.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.158+0000 7fcc82800640 1 --2- 172.21.15.86:0/2922784362 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcc5c06c920 0x7fcc5c06ede0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c000f60 tx=0x7fcc6c007a00 comp rx=0 tx=0).stop 2024-04-20T21:27:51.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.158+0000 7fcc82800640 1 -- 172.21.15.86:0/2922784362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc7c072240 msgr2=0x7fcc7c15cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:51.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.158+0000 7fcc82800640 1 --2- 172.21.15.86:0/2922784362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc7c072240 0x7fcc7c15cb20 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fcc68000c00 tx=0x7fcc68004730 comp rx=0 tx=0).stop 2024-04-20T21:27:51.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.158+0000 7fcc82800640 1 -- 172.21.15.86:0/2922784362 shutdown_connections 2024-04-20T21:27:51.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.158+0000 7fcc82800640 1 --2- 172.21.15.86:0/2922784362 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcc5c06c920 0x7fcc5c06ede0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:51.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.158+0000 7fcc82800640 1 --2- 172.21.15.86:0/2922784362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcc7c146130 0x7fcc7c15d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:51.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.158+0000 7fcc82800640 1 --2- 172.21.15.86:0/2922784362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc7c072240 0x7fcc7c15cb20 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:51.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.158+0000 7fcc82800640 1 --2- 172.21.15.86:0/2922784362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc7c06e6d0 0x7fcc7c15b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:51.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.158+0000 7fcc82800640 1 -- 172.21.15.86:0/2922784362 >> 172.21.15.86:0/2922784362 conn(0x7fcc7c0b2360 msgr2=0x7fcc7c1418a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:51.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.159+0000 7fcc82800640 1 -- 172.21.15.86:0/2922784362 shutdown_connections 2024-04-20T21:27:51.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.159+0000 7fcc82800640 1 -- 172.21.15.86:0/2922784362 wait complete. 2024-04-20T21:27:51.176 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":398,"stamp":"2024-04-20T21:27:50.843488+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":184,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260360,"kb_used_data":10888,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714360,"statfs":{"total":773094113280,"available":772827504640,"internally_reserved":0,"allocated":11149312,"data_stored":5465132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002048"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:48.802455+0000","last_change":"2024-04-20T21:27:46.285677+0000","last_active":"2024-04-20T21:27:48.802455+0000","last_peered":"2024-04-20T21:27:48.802455+0000","last_clean":"2024-04-20T21:27:48.802455+0000","last_became_active":"2024-04-20T21:27:46.277673+0000","last_became_peered":"2024-04-20T21:27:46.277673+0000","last_unstale":"2024-04-20T21:27:48.802455+0000","last_undegraded":"2024-04-20T21:27:48.802455+0000","last_fullsized":"2024-04-20T21:27:48.802455+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:36:06.054648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.292100+0000","last_change":"2024-04-20T21:27:46.281687+0000","last_active":"2024-04-20T21:27:47.292100+0000","last_peered":"2024-04-20T21:27:47.292100+0000","last_clean":"2024-04-20T21:27:47.292100+0000","last_became_active":"2024-04-20T21:27:46.279975+0000","last_became_peered":"2024-04-20T21:27:46.279975+0000","last_unstale":"2024-04-20T21:27:47.292100+0000","last_undegraded":"2024-04-20T21:27:47.292100+0000","last_fullsized":"2024-04-20T21:27:47.292100+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:28:24.810799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:48.802537+0000","last_change":"2024-04-20T21:27:46.284294+0000","last_active":"2024-04-20T21:27:48.802537+0000","last_peered":"2024-04-20T21:27:48.802537+0000","last_clean":"2024-04-20T21:27:48.802537+0000","last_became_active":"2024-04-20T21:27:46.283937+0000","last_became_peered":"2024-04-20T21:27:46.283937+0000","last_unstale":"2024-04-20T21:27:48.802537+0000","last_undegraded":"2024-04-20T21:27:48.802537+0000","last_fullsized":"2024-04-20T21:27:48.802537+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:12:12.467227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:48.070820+0000","last_change":"2024-04-20T21:27:46.269146+0000","last_active":"2024-04-20T21:27:48.070820+0000","last_peered":"2024-04-20T21:27:48.070820+0000","last_clean":"2024-04-20T21:27:48.070820+0000","last_became_active":"2024-04-20T21:27:46.268870+0000","last_became_peered":"2024-04-20T21:27:46.268870+0000","last_unstale":"2024-04-20T21:27:48.070820+0000","last_undegraded":"2024-04-20T21:27:48.070820+0000","last_fullsized":"2024-04-20T21:27:48.070820+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:18:52.090184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.286366+0000","last_change":"2024-04-20T21:27:46.263670+0000","last_active":"2024-04-20T21:27:47.286366+0000","last_peered":"2024-04-20T21:27:47.286366+0000","last_clean":"2024-04-20T21:27:47.286366+0000","last_became_active":"2024-04-20T21:27:46.262841+0000","last_became_peered":"2024-04-20T21:27:46.262841+0000","last_unstale":"2024-04-20T21:27:47.286366+0000","last_undegraded":"2024-04-20T21:27:47.286366+0000","last_fullsized":"2024-04-20T21:27:47.286366+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:14:37.726956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:48.170689+0000","last_change":"2024-04-20T21:27:46.285337+0000","last_active":"2024-04-20T21:27:48.170689+0000","last_peered":"2024-04-20T21:27:48.170689+0000","last_clean":"2024-04-20T21:27:48.170689+0000","last_became_active":"2024-04-20T21:27:46.284947+0000","last_became_peered":"2024-04-20T21:27:46.284947+0000","last_unstale":"2024-04-20T21:27:48.170689+0000","last_undegraded":"2024-04-20T21:27:48.170689+0000","last_fullsized":"2024-04-20T21:27:48.170689+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:20:40.051861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:48.170618+0000","last_change":"2024-04-20T21:27:46.274942+0000","last_active":"2024-04-20T21:27:48.170618+0000","last_peered":"2024-04-20T21:27:48.170618+0000","last_clean":"2024-04-20T21:27:48.170618+0000","last_became_active":"2024-04-20T21:27:46.274618+0000","last_became_peered":"2024-04-20T21:27:46.274618+0000","last_unstale":"2024-04-20T21:27:48.170618+0000","last_undegraded":"2024-04-20T21:27:48.170618+0000","last_fullsized":"2024-04-20T21:27:48.170618+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:56:37.685697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.291917+0000","last_change":"2024-04-20T21:27:46.279306+0000","last_active":"2024-04-20T21:27:47.291917+0000","last_peered":"2024-04-20T21:27:47.291917+0000","last_clean":"2024-04-20T21:27:47.291917+0000","last_became_active":"2024-04-20T21:27:46.278928+0000","last_became_peered":"2024-04-20T21:27:46.278928+0000","last_unstale":"2024-04-20T21:27:47.291917+0000","last_undegraded":"2024-04-20T21:27:47.291917+0000","last_fullsized":"2024-04-20T21:27:47.291917+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:16:46.067641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.289157+0000","last_change":"2024-04-20T21:27:46.290161+0000","last_active":"2024-04-20T21:27:47.289157+0000","last_peered":"2024-04-20T21:27:47.289157+0000","last_clean":"2024-04-20T21:27:47.289157+0000","last_became_active":"2024-04-20T21:27:46.289804+0000","last_became_peered":"2024-04-20T21:27:46.289804+0000","last_unstale":"2024-04-20T21:27:47.289157+0000","last_undegraded":"2024-04-20T21:27:47.289157+0000","last_fullsized":"2024-04-20T21:27:47.289157+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:42:40.600684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.285850+0000","last_change":"2024-04-20T21:27:46.277866+0000","last_active":"2024-04-20T21:27:47.285850+0000","last_peered":"2024-04-20T21:27:47.285850+0000","last_clean":"2024-04-20T21:27:47.285850+0000","last_became_active":"2024-04-20T21:27:46.277536+0000","last_became_peered":"2024-04-20T21:27:46.277536+0000","last_unstale":"2024-04-20T21:27:47.285850+0000","last_undegraded":"2024-04-20T21:27:47.285850+0000","last_fullsized":"2024-04-20T21:27:47.285850+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:23:38.604661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:48.171471+0000","last_change":"2024-04-20T21:27:46.293046+0000","last_active":"2024-04-20T21:27:48.171471+0000","last_peered":"2024-04-20T21:27:48.171471+0000","last_clean":"2024-04-20T21:27:48.171471+0000","last_became_active":"2024-04-20T21:27:46.292715+0000","last_became_peered":"2024-04-20T21:27:46.292715+0000","last_unstale":"2024-04-20T21:27:48.171471+0000","last_undegraded":"2024-04-20T21:27:48.171471+0000","last_fullsized":"2024-04-20T21:27:48.171471+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:28:24.810799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.288369+0000","last_change":"2024-04-20T21:27:46.282404+0000","last_active":"2024-04-20T21:27:47.288369+0000","last_peered":"2024-04-20T21:27:47.288369+0000","last_clean":"2024-04-20T21:27:47.288369+0000","last_became_active":"2024-04-20T21:27:46.282095+0000","last_became_peered":"2024-04-20T21:27:46.282095+0000","last_unstale":"2024-04-20T21:27:47.288369+0000","last_undegraded":"2024-04-20T21:27:47.288369+0000","last_fullsized":"2024-04-20T21:27:47.288369+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:20:40.051861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:48.070604+0000","last_change":"2024-04-20T21:27:46.284783+0000","last_active":"2024-04-20T21:27:48.070604+0000","last_peered":"2024-04-20T21:27:48.070604+0000","last_clean":"2024-04-20T21:27:48.070604+0000","last_became_active":"2024-04-20T21:27:46.284508+0000","last_became_peered":"2024-04-20T21:27:46.284508+0000","last_unstale":"2024-04-20T21:27:48.070604+0000","last_undegraded":"2024-04-20T21:27:48.070604+0000","last_fullsized":"2024-04-20T21:27:48.070604+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:55:17.944959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.284309+0000","last_change":"2024-04-20T21:27:46.281471+0000","last_active":"2024-04-20T21:27:47.284309+0000","last_peered":"2024-04-20T21:27:47.284309+0000","last_clean":"2024-04-20T21:27:47.284309+0000","last_became_active":"2024-04-20T21:27:46.281142+0000","last_became_peered":"2024-04-20T21:27:46.281142+0000","last_unstale":"2024-04-20T21:27:47.284309+0000","last_undegraded":"2024-04-20T21:27:47.284309+0000","last_fullsized":"2024-04-20T21:27:47.284309+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:33:27.263551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:48.070079+0000","last_change":"2024-04-20T21:27:46.279572+0000","last_active":"2024-04-20T21:27:48.070079+0000","last_peered":"2024-04-20T21:27:48.070079+0000","last_clean":"2024-04-20T21:27:48.070079+0000","last_became_active":"2024-04-20T21:27:46.279129+0000","last_became_peered":"2024-04-20T21:27:46.279129+0000","last_unstale":"2024-04-20T21:27:48.070079+0000","last_undegraded":"2024-04-20T21:27:48.070079+0000","last_fullsized":"2024-04-20T21:27:48.070079+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:45:46.994910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:48.170612+0000","last_change":"2024-04-20T21:27:46.284190+0000","last_active":"2024-04-20T21:27:48.170612+0000","last_peered":"2024-04-20T21:27:48.170612+0000","last_clean":"2024-04-20T21:27:48.170612+0000","last_became_active":"2024-04-20T21:27:46.282873+0000","last_became_peered":"2024-04-20T21:27:46.282873+0000","last_unstale":"2024-04-20T21:27:48.170612+0000","last_undegraded":"2024-04-20T21:27:48.170612+0000","last_fullsized":"2024-04-20T21:27:48.170612+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:00:22.081761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.292481+0000","last_change":"2024-04-20T21:27:46.287351+0000","last_active":"2024-04-20T21:27:47.292481+0000","last_peered":"2024-04-20T21:27:47.292481+0000","last_clean":"2024-04-20T21:27:47.292481+0000","last_became_active":"2024-04-20T21:27:46.287014+0000","last_became_peered":"2024-04-20T21:27:46.287014+0000","last_unstale":"2024-04-20T21:27:47.292481+0000","last_undegraded":"2024-04-20T21:27:47.292481+0000","last_fullsized":"2024-04-20T21:27:47.292481+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:46:45.540553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:48.070873+0000","last_change":"2024-04-20T21:27:46.280865+0000","last_active":"2024-04-20T21:27:48.070873+0000","last_peered":"2024-04-20T21:27:48.070873+0000","last_clean":"2024-04-20T21:27:48.070873+0000","last_became_active":"2024-04-20T21:27:46.280570+0000","last_became_peered":"2024-04-20T21:27:46.280570+0000","last_unstale":"2024-04-20T21:27:48.070873+0000","last_undegraded":"2024-04-20T21:27:48.070873+0000","last_fullsized":"2024-04-20T21:27:48.070873+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:25:18.062710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.284800+0000","last_change":"2024-04-20T21:27:46.289458+0000","last_active":"2024-04-20T21:27:47.284800+0000","last_peered":"2024-04-20T21:27:47.284800+0000","last_clean":"2024-04-20T21:27:47.284800+0000","last_became_active":"2024-04-20T21:27:46.289149+0000","last_became_peered":"2024-04-20T21:27:46.289149+0000","last_unstale":"2024-04-20T21:27:47.284800+0000","last_undegraded":"2024-04-20T21:27:47.284800+0000","last_fullsized":"2024-04-20T21:27:47.284800+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:26:30.228127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.288536+0000","last_change":"2024-04-20T21:27:46.290739+0000","last_active":"2024-04-20T21:27:47.288536+0000","last_peered":"2024-04-20T21:27:47.288536+0000","last_clean":"2024-04-20T21:27:47.288536+0000","last_became_active":"2024-04-20T21:27:46.290420+0000","last_became_peered":"2024-04-20T21:27:46.290420+0000","last_unstale":"2024-04-20T21:27:47.288536+0000","last_undegraded":"2024-04-20T21:27:47.288536+0000","last_fullsized":"2024-04-20T21:27:47.288536+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:14:35.213580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.284258+0000","last_change":"2024-04-20T21:27:46.280870+0000","last_active":"2024-04-20T21:27:47.284258+0000","last_peered":"2024-04-20T21:27:47.284258+0000","last_clean":"2024-04-20T21:27:47.284258+0000","last_became_active":"2024-04-20T21:27:46.280534+0000","last_became_peered":"2024-04-20T21:27:46.280534+0000","last_unstale":"2024-04-20T21:27:47.284258+0000","last_undegraded":"2024-04-20T21:27:47.284258+0000","last_fullsized":"2024-04-20T21:27:47.284258+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:38:34.640221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.292410+0000","last_change":"2024-04-20T21:27:46.289025+0000","last_active":"2024-04-20T21:27:47.292410+0000","last_peered":"2024-04-20T21:27:47.292410+0000","last_clean":"2024-04-20T21:27:47.292410+0000","last_became_active":"2024-04-20T21:27:46.288687+0000","last_became_peered":"2024-04-20T21:27:46.288687+0000","last_unstale":"2024-04-20T21:27:47.292410+0000","last_undegraded":"2024-04-20T21:27:47.292410+0000","last_fullsized":"2024-04-20T21:27:47.292410+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:01:30.155092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.291887+0000","last_change":"2024-04-20T21:27:46.286582+0000","last_active":"2024-04-20T21:27:47.291887+0000","last_peered":"2024-04-20T21:27:47.291887+0000","last_clean":"2024-04-20T21:27:47.291887+0000","last_became_active":"2024-04-20T21:27:46.286300+0000","last_became_peered":"2024-04-20T21:27:46.286300+0000","last_unstale":"2024-04-20T21:27:47.291887+0000","last_undegraded":"2024-04-20T21:27:47.291887+0000","last_fullsized":"2024-04-20T21:27:47.291887+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:42:40.600684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:48.171335+0000","last_change":"2024-04-20T21:27:46.286534+0000","last_active":"2024-04-20T21:27:48.171335+0000","last_peered":"2024-04-20T21:27:48.171335+0000","last_clean":"2024-04-20T21:27:48.171335+0000","last_became_active":"2024-04-20T21:27:46.286026+0000","last_became_peered":"2024-04-20T21:27:46.286026+0000","last_unstale":"2024-04-20T21:27:48.171335+0000","last_undegraded":"2024-04-20T21:27:48.171335+0000","last_fullsized":"2024-04-20T21:27:48.171335+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:24:49.049445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.287646+0000","last_change":"2024-04-20T21:27:46.279842+0000","last_active":"2024-04-20T21:27:47.287646+0000","last_peered":"2024-04-20T21:27:47.287646+0000","last_clean":"2024-04-20T21:27:47.287646+0000","last_became_active":"2024-04-20T21:27:46.279441+0000","last_became_peered":"2024-04-20T21:27:46.279441+0000","last_unstale":"2024-04-20T21:27:47.287646+0000","last_undegraded":"2024-04-20T21:27:47.287646+0000","last_fullsized":"2024-04-20T21:27:47.287646+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:46:57.102949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.292305+0000","last_change":"2024-04-20T21:27:46.278582+0000","last_active":"2024-04-20T21:27:47.292305+0000","last_peered":"2024-04-20T21:27:47.292305+0000","last_clean":"2024-04-20T21:27:47.292305+0000","last_became_active":"2024-04-20T21:27:46.278249+0000","last_became_peered":"2024-04-20T21:27:46.278249+0000","last_unstale":"2024-04-20T21:27:47.292305+0000","last_undegraded":"2024-04-20T21:27:47.292305+0000","last_fullsized":"2024-04-20T21:27:47.292305+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:32:30.121850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:48.171501+0000","last_change":"2024-04-20T21:27:46.277527+0000","last_active":"2024-04-20T21:27:48.171501+0000","last_peered":"2024-04-20T21:27:48.171501+0000","last_clean":"2024-04-20T21:27:48.171501+0000","last_became_active":"2024-04-20T21:27:46.277224+0000","last_became_peered":"2024-04-20T21:27:46.277224+0000","last_unstale":"2024-04-20T21:27:48.171501+0000","last_undegraded":"2024-04-20T21:27:48.171501+0000","last_fullsized":"2024-04-20T21:27:48.171501+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:42:40.600684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.286510+0000","last_change":"2024-04-20T21:27:46.271487+0000","last_active":"2024-04-20T21:27:47.286510+0000","last_peered":"2024-04-20T21:27:47.286510+0000","last_clean":"2024-04-20T21:27:47.286510+0000","last_became_active":"2024-04-20T21:27:46.271196+0000","last_became_peered":"2024-04-20T21:27:46.271196+0000","last_unstale":"2024-04-20T21:27:47.286510+0000","last_undegraded":"2024-04-20T21:27:47.286510+0000","last_fullsized":"2024-04-20T21:27:47.286510+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:48:49.908759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.292657+0000","last_change":"2024-04-20T21:27:46.279948+0000","last_active":"2024-04-20T21:27:47.292657+0000","last_peered":"2024-04-20T21:27:47.292657+0000","last_clean":"2024-04-20T21:27:47.292657+0000","last_became_active":"2024-04-20T21:27:46.279523+0000","last_became_peered":"2024-04-20T21:27:46.279523+0000","last_unstale":"2024-04-20T21:27:47.292657+0000","last_undegraded":"2024-04-20T21:27:47.292657+0000","last_fullsized":"2024-04-20T21:27:47.292657+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:32:47.135990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.284510+0000","last_change":"2024-04-20T21:27:46.290332+0000","last_active":"2024-04-20T21:27:47.284510+0000","last_peered":"2024-04-20T21:27:47.284510+0000","last_clean":"2024-04-20T21:27:47.284510+0000","last_became_active":"2024-04-20T21:27:46.288626+0000","last_became_peered":"2024-04-20T21:27:46.288626+0000","last_unstale":"2024-04-20T21:27:47.284510+0000","last_undegraded":"2024-04-20T21:27:47.284510+0000","last_fullsized":"2024-04-20T21:27:47.284510+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:11:34.429861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:48.070438+0000","last_change":"2024-04-20T21:27:46.272614+0000","last_active":"2024-04-20T21:27:48.070438+0000","last_peered":"2024-04-20T21:27:48.070438+0000","last_clean":"2024-04-20T21:27:48.070438+0000","last_became_active":"2024-04-20T21:27:46.270964+0000","last_became_peered":"2024-04-20T21:27:46.270964+0000","last_unstale":"2024-04-20T21:27:48.070438+0000","last_undegraded":"2024-04-20T21:27:48.070438+0000","last_fullsized":"2024-04-20T21:27:48.070438+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:15:06.288125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:48.801850+0000","last_change":"2024-04-20T21:27:46.278169+0000","last_active":"2024-04-20T21:27:48.801850+0000","last_peered":"2024-04-20T21:27:48.801850+0000","last_clean":"2024-04-20T21:27:48.801850+0000","last_became_active":"2024-04-20T21:27:46.277436+0000","last_became_peered":"2024-04-20T21:27:46.277436+0000","last_unstale":"2024-04-20T21:27:48.801850+0000","last_undegraded":"2024-04-20T21:27:48.801850+0000","last_fullsized":"2024-04-20T21:27:48.801850+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:20:43.993186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.284490+0000","last_change":"2024-04-20T21:27:46.290639+0000","last_active":"2024-04-20T21:27:47.284490+0000","last_peered":"2024-04-20T21:27:47.284490+0000","last_clean":"2024-04-20T21:27:47.284490+0000","last_became_active":"2024-04-20T21:27:46.290259+0000","last_became_peered":"2024-04-20T21:27:46.290259+0000","last_unstale":"2024-04-20T21:27:47.284490+0000","last_undegraded":"2024-04-20T21:27:47.284490+0000","last_fullsized":"2024-04-20T21:27:47.284490+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:45:27.085671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"99'4","reported_seq":18,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.292389+0000","last_change":"2024-04-20T21:27:45.258515+0000","last_active":"2024-04-20T21:27:47.292389+0000","last_peered":"2024-04-20T21:27:47.292389+0000","last_clean":"2024-04-20T21:27:47.292389+0000","last_became_active":"2024-04-20T21:27:45.258254+0000","last_became_peered":"2024-04-20T21:27:45.258254+0000","last_unstale":"2024-04-20T21:27:47.292389+0000","last_undegraded":"2024-04-20T21:27:47.292389+0000","last_fullsized":"2024-04-20T21:27:47.292389+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:15:05.273003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:48.071231+0000","last_change":"2024-04-20T21:27:46.266555+0000","last_active":"2024-04-20T21:27:48.071231+0000","last_peered":"2024-04-20T21:27:48.071231+0000","last_clean":"2024-04-20T21:27:48.071231+0000","last_became_active":"2024-04-20T21:27:46.265809+0000","last_became_peered":"2024-04-20T21:27:46.265809+0000","last_unstale":"2024-04-20T21:27:48.071231+0000","last_undegraded":"2024-04-20T21:27:48.071231+0000","last_fullsized":"2024-04-20T21:27:48.071231+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:00:22.081761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"99'1","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:48.170932+0000","last_change":"2024-04-20T21:27:45.258703+0000","last_active":"2024-04-20T21:27:48.170932+0000","last_peered":"2024-04-20T21:27:48.170932+0000","last_clean":"2024-04-20T21:27:48.170932+0000","last_became_active":"2024-04-20T21:27:45.257807+0000","last_became_peered":"2024-04-20T21:27:45.257807+0000","last_unstale":"2024-04-20T21:27:48.170932+0000","last_undegraded":"2024-04-20T21:27:48.170932+0000","last_fullsized":"2024-04-20T21:27:48.170932+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:23:37.589540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:48.802280+0000","last_change":"2024-04-20T21:27:46.291327+0000","last_active":"2024-04-20T21:27:48.802280+0000","last_peered":"2024-04-20T21:27:48.802280+0000","last_clean":"2024-04-20T21:27:48.802280+0000","last_became_active":"2024-04-20T21:27:46.290947+0000","last_became_peered":"2024-04-20T21:27:46.290947+0000","last_unstale":"2024-04-20T21:27:48.802280+0000","last_undegraded":"2024-04-20T21:27:48.802280+0000","last_fullsized":"2024-04-20T21:27:48.802280+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:55:31.324268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"99'2","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.292346+0000","last_change":"2024-04-20T21:27:45.255784+0000","last_active":"2024-04-20T21:27:47.292346+0000","last_peered":"2024-04-20T21:27:47.292346+0000","last_clean":"2024-04-20T21:27:47.292346+0000","last_became_active":"2024-04-20T21:27:45.255462+0000","last_became_peered":"2024-04-20T21:27:45.255462+0000","last_unstale":"2024-04-20T21:27:47.292346+0000","last_undegraded":"2024-04-20T21:27:47.292346+0000","last_fullsized":"2024-04-20T21:27:47.292346+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:47:19.161353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.288884+0000","last_change":"2024-04-20T21:27:46.285056+0000","last_active":"2024-04-20T21:27:47.288884+0000","last_peered":"2024-04-20T21:27:47.288884+0000","last_clean":"2024-04-20T21:27:47.288884+0000","last_became_active":"2024-04-20T21:27:46.284739+0000","last_became_peered":"2024-04-20T21:27:46.284739+0000","last_unstale":"2024-04-20T21:27:47.288884+0000","last_undegraded":"2024-04-20T21:27:47.288884+0000","last_fullsized":"2024-04-20T21:27:47.288884+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:24:49.049445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"99'2","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.292079+0000","last_change":"2024-04-20T21:27:45.257220+0000","last_active":"2024-04-20T21:27:47.292079+0000","last_peered":"2024-04-20T21:27:47.292079+0000","last_clean":"2024-04-20T21:27:47.292079+0000","last_became_active":"2024-04-20T21:27:45.256799+0000","last_became_peered":"2024-04-20T21:27:45.256799+0000","last_unstale":"2024-04-20T21:27:47.292079+0000","last_undegraded":"2024-04-20T21:27:47.292079+0000","last_fullsized":"2024-04-20T21:27:47.292079+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:14:36.711834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:48.605723+0000","last_change":"2024-04-20T21:27:46.288050+0000","last_active":"2024-04-20T21:27:48.605723+0000","last_peered":"2024-04-20T21:27:48.605723+0000","last_clean":"2024-04-20T21:27:48.605723+0000","last_became_active":"2024-04-20T21:27:46.287419+0000","last_became_peered":"2024-04-20T21:27:46.287419+0000","last_unstale":"2024-04-20T21:27:48.605723+0000","last_undegraded":"2024-04-20T21:27:48.605723+0000","last_fullsized":"2024-04-20T21:27:48.605723+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:59:02.193516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"99'1","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:48.802117+0000","last_change":"2024-04-20T21:27:45.251360+0000","last_active":"2024-04-20T21:27:48.802117+0000","last_peered":"2024-04-20T21:27:48.802117+0000","last_clean":"2024-04-20T21:27:48.802117+0000","last_became_active":"2024-04-20T21:27:45.251076+0000","last_became_peered":"2024-04-20T21:27:45.251076+0000","last_unstale":"2024-04-20T21:27:48.802117+0000","last_undegraded":"2024-04-20T21:27:48.802117+0000","last_fullsized":"2024-04-20T21:27:48.802117+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:45:26.070549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:48.170770+0000","last_change":"2024-04-20T21:27:46.286341+0000","last_active":"2024-04-20T21:27:48.170770+0000","last_peered":"2024-04-20T21:27:48.170770+0000","last_clean":"2024-04-20T21:27:48.170770+0000","last_became_active":"2024-04-20T21:27:46.286027+0000","last_became_peered":"2024-04-20T21:27:46.286027+0000","last_unstale":"2024-04-20T21:27:48.170770+0000","last_undegraded":"2024-04-20T21:27:48.170770+0000","last_fullsized":"2024-04-20T21:27:48.170770+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:14:35.213580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"99'1","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.291908+0000","last_change":"2024-04-20T21:27:45.249660+0000","last_active":"2024-04-20T21:27:47.291908+0000","last_peered":"2024-04-20T21:27:47.291908+0000","last_clean":"2024-04-20T21:27:47.291908+0000","last_became_active":"2024-04-20T21:27:45.248485+0000","last_became_peered":"2024-04-20T21:27:45.248485+0000","last_unstale":"2024-04-20T21:27:47.291908+0000","last_undegraded":"2024-04-20T21:27:47.291908+0000","last_fullsized":"2024-04-20T21:27:47.291908+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:55:12.291750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:48.070037+0000","last_change":"2024-04-20T21:27:46.280355+0000","last_active":"2024-04-20T21:27:48.070037+0000","last_peered":"2024-04-20T21:27:48.070037+0000","last_clean":"2024-04-20T21:27:48.070037+0000","last_became_active":"2024-04-20T21:27:46.276200+0000","last_became_peered":"2024-04-20T21:27:46.276200+0000","last_unstale":"2024-04-20T21:27:48.070037+0000","last_undegraded":"2024-04-20T21:27:48.070037+0000","last_fullsized":"2024-04-20T21:27:48.070037+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:46:57.102949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.286089+0000","last_change":"2024-04-20T21:27:45.255056+0000","last_active":"2024-04-20T21:27:47.286089+0000","last_peered":"2024-04-20T21:27:47.286089+0000","last_clean":"2024-04-20T21:27:47.286089+0000","last_became_active":"2024-04-20T21:27:45.252809+0000","last_became_peered":"2024-04-20T21:27:45.252809+0000","last_unstale":"2024-04-20T21:27:47.286089+0000","last_undegraded":"2024-04-20T21:27:47.286089+0000","last_fullsized":"2024-04-20T21:27:47.286089+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:52:11.165181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.286996+0000","last_change":"2024-04-20T21:27:46.284191+0000","last_active":"2024-04-20T21:27:47.286996+0000","last_peered":"2024-04-20T21:27:47.286996+0000","last_clean":"2024-04-20T21:27:47.286996+0000","last_became_active":"2024-04-20T21:27:46.283711+0000","last_became_peered":"2024-04-20T21:27:46.283711+0000","last_unstale":"2024-04-20T21:27:47.286996+0000","last_undegraded":"2024-04-20T21:27:47.286996+0000","last_fullsized":"2024-04-20T21:27:47.286996+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:59:33.542734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:48.801867+0000","last_change":"2024-04-20T21:27:45.247265+0000","last_active":"2024-04-20T21:27:48.801867+0000","last_peered":"2024-04-20T21:27:48.801867+0000","last_clean":"2024-04-20T21:27:48.801867+0000","last_became_active":"2024-04-20T21:27:45.245173+0000","last_became_peered":"2024-04-20T21:27:45.245173+0000","last_unstale":"2024-04-20T21:27:48.801867+0000","last_undegraded":"2024-04-20T21:27:48.801867+0000","last_fullsized":"2024-04-20T21:27:48.801867+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:33:26.248429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.292581+0000","last_change":"2024-04-20T21:27:46.286781+0000","last_active":"2024-04-20T21:27:47.292581+0000","last_peered":"2024-04-20T21:27:47.292581+0000","last_clean":"2024-04-20T21:27:47.292581+0000","last_became_active":"2024-04-20T21:27:46.286461+0000","last_became_peered":"2024-04-20T21:27:46.286461+0000","last_unstale":"2024-04-20T21:27:47.292581+0000","last_undegraded":"2024-04-20T21:27:47.292581+0000","last_fullsized":"2024-04-20T21:27:47.292581+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:38:52.245678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:48.802781+0000","last_change":"2024-04-20T21:27:45.257108+0000","last_active":"2024-04-20T21:27:48.802781+0000","last_peered":"2024-04-20T21:27:48.802781+0000","last_clean":"2024-04-20T21:27:48.802781+0000","last_became_active":"2024-04-20T21:27:45.256824+0000","last_became_peered":"2024-04-20T21:27:45.256824+0000","last_unstale":"2024-04-20T21:27:48.802781+0000","last_undegraded":"2024-04-20T21:27:48.802781+0000","last_fullsized":"2024-04-20T21:27:48.802781+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:38:33.625100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.289113+0000","last_change":"2024-04-20T21:27:46.286378+0000","last_active":"2024-04-20T21:27:47.289113+0000","last_peered":"2024-04-20T21:27:47.289113+0000","last_clean":"2024-04-20T21:27:47.289113+0000","last_became_active":"2024-04-20T21:27:46.286071+0000","last_became_peered":"2024-04-20T21:27:46.286071+0000","last_unstale":"2024-04-20T21:27:47.289113+0000","last_undegraded":"2024-04-20T21:27:47.289113+0000","last_fullsized":"2024-04-20T21:27:47.289113+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:56:37.685697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:48.070756+0000","last_change":"2024-04-20T21:27:45.251360+0000","last_active":"2024-04-20T21:27:48.070756+0000","last_peered":"2024-04-20T21:27:48.070756+0000","last_clean":"2024-04-20T21:27:48.070756+0000","last_became_active":"2024-04-20T21:27:45.251057+0000","last_became_peered":"2024-04-20T21:27:45.251057+0000","last_unstale":"2024-04-20T21:27:48.070756+0000","last_undegraded":"2024-04-20T21:27:48.070756+0000","last_fullsized":"2024-04-20T21:27:48.070756+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:59:01.178395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.286699+0000","last_change":"2024-04-20T21:27:46.280663+0000","last_active":"2024-04-20T21:27:47.286699+0000","last_peered":"2024-04-20T21:27:47.286699+0000","last_clean":"2024-04-20T21:27:47.286699+0000","last_became_active":"2024-04-20T21:27:46.280349+0000","last_became_peered":"2024-04-20T21:27:46.280349+0000","last_unstale":"2024-04-20T21:27:47.286699+0000","last_undegraded":"2024-04-20T21:27:47.286699+0000","last_fullsized":"2024-04-20T21:27:47.286699+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:58:48.599154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"99'3","reported_seq":17,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:48.171870+0000","last_change":"2024-04-20T21:27:45.259658+0000","last_active":"2024-04-20T21:27:48.171870+0000","last_peered":"2024-04-20T21:27:48.171870+0000","last_clean":"2024-04-20T21:27:48.171870+0000","last_became_active":"2024-04-20T21:27:45.259277+0000","last_became_peered":"2024-04-20T21:27:45.259277+0000","last_unstale":"2024-04-20T21:27:48.171870+0000","last_undegraded":"2024-04-20T21:27:48.171870+0000","last_fullsized":"2024-04-20T21:27:48.171870+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:14:36.711834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:48.171740+0000","last_change":"2024-04-20T21:27:46.279199+0000","last_active":"2024-04-20T21:27:48.171740+0000","last_peered":"2024-04-20T21:27:48.171740+0000","last_clean":"2024-04-20T21:27:48.171740+0000","last_became_active":"2024-04-20T21:27:46.278830+0000","last_became_peered":"2024-04-20T21:27:46.278830+0000","last_unstale":"2024-04-20T21:27:48.171740+0000","last_undegraded":"2024-04-20T21:27:48.171740+0000","last_fullsized":"2024-04-20T21:27:48.171740+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:16:46.067641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"99'6","reported_seq":20,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.288626+0000","last_change":"2024-04-20T21:27:45.245837+0000","last_active":"2024-04-20T21:27:47.288626+0000","last_peered":"2024-04-20T21:27:47.288626+0000","last_clean":"2024-04-20T21:27:47.288626+0000","last_became_active":"2024-04-20T21:27:45.245491+0000","last_became_peered":"2024-04-20T21:27:45.245491+0000","last_unstale":"2024-04-20T21:27:47.288626+0000","last_undegraded":"2024-04-20T21:27:47.288626+0000","last_fullsized":"2024-04-20T21:27:47.288626+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:58:47.584032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:48.070831+0000","last_change":"2024-04-20T21:27:46.278694+0000","last_active":"2024-04-20T21:27:48.070831+0000","last_peered":"2024-04-20T21:27:48.070831+0000","last_clean":"2024-04-20T21:27:48.070831+0000","last_became_active":"2024-04-20T21:27:46.278413+0000","last_became_peered":"2024-04-20T21:27:46.278413+0000","last_unstale":"2024-04-20T21:27:48.070831+0000","last_undegraded":"2024-04-20T21:27:48.070831+0000","last_fullsized":"2024-04-20T21:27:48.070831+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:59:33.542734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"99'2","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.284400+0000","last_change":"2024-04-20T21:27:45.254404+0000","last_active":"2024-04-20T21:27:47.284400+0000","last_peered":"2024-04-20T21:27:47.284400+0000","last_clean":"2024-04-20T21:27:47.284400+0000","last_became_active":"2024-04-20T21:27:45.254086+0000","last_became_peered":"2024-04-20T21:27:45.254086+0000","last_unstale":"2024-04-20T21:27:47.284400+0000","last_undegraded":"2024-04-20T21:27:47.284400+0000","last_fullsized":"2024-04-20T21:27:47.284400+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:06:58.731826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:48.802175+0000","last_change":"2024-04-20T21:27:46.284009+0000","last_active":"2024-04-20T21:27:48.802175+0000","last_peered":"2024-04-20T21:27:48.802175+0000","last_clean":"2024-04-20T21:27:48.802175+0000","last_became_active":"2024-04-20T21:27:46.283732+0000","last_became_peered":"2024-04-20T21:27:46.283732+0000","last_unstale":"2024-04-20T21:27:48.802175+0000","last_undegraded":"2024-04-20T21:27:48.802175+0000","last_fullsized":"2024-04-20T21:27:48.802175+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:24:42.298789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.284831+0000","last_change":"2024-04-20T21:27:45.250257+0000","last_active":"2024-04-20T21:27:47.284831+0000","last_peered":"2024-04-20T21:27:47.284831+0000","last_clean":"2024-04-20T21:27:47.284831+0000","last_became_active":"2024-04-20T21:27:45.249980+0000","last_became_peered":"2024-04-20T21:27:45.249980+0000","last_unstale":"2024-04-20T21:27:47.284831+0000","last_undegraded":"2024-04-20T21:27:47.284831+0000","last_fullsized":"2024-04-20T21:27:47.284831+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:43:06.258958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.287953+0000","last_change":"2024-04-20T21:27:46.281992+0000","last_active":"2024-04-20T21:27:47.287953+0000","last_peered":"2024-04-20T21:27:47.287953+0000","last_clean":"2024-04-20T21:27:47.287953+0000","last_became_active":"2024-04-20T21:27:46.278877+0000","last_became_peered":"2024-04-20T21:27:46.278877+0000","last_unstale":"2024-04-20T21:27:47.287953+0000","last_undegraded":"2024-04-20T21:27:47.287953+0000","last_fullsized":"2024-04-20T21:27:47.287953+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:00:22.081761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.287081+0000","last_change":"2024-04-20T21:27:45.256573+0000","last_active":"2024-04-20T21:27:47.287081+0000","last_peered":"2024-04-20T21:27:47.287081+0000","last_clean":"2024-04-20T21:27:47.287081+0000","last_became_active":"2024-04-20T21:27:45.256161+0000","last_became_peered":"2024-04-20T21:27:45.256161+0000","last_unstale":"2024-04-20T21:27:47.287081+0000","last_undegraded":"2024-04-20T21:27:47.287081+0000","last_fullsized":"2024-04-20T21:27:47.287081+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:14:36.711834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.283977+0000","last_change":"2024-04-20T21:27:46.283521+0000","last_active":"2024-04-20T21:27:47.283977+0000","last_peered":"2024-04-20T21:27:47.283977+0000","last_clean":"2024-04-20T21:27:47.283977+0000","last_became_active":"2024-04-20T21:27:46.282867+0000","last_became_peered":"2024-04-20T21:27:46.282867+0000","last_unstale":"2024-04-20T21:27:47.283977+0000","last_undegraded":"2024-04-20T21:27:47.283977+0000","last_fullsized":"2024-04-20T21:27:47.283977+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:44:17.815299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"99'8","reported_seq":22,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:48.801975+0000","last_change":"2024-04-20T21:27:45.254036+0000","last_active":"2024-04-20T21:27:48.801975+0000","last_peered":"2024-04-20T21:27:48.801975+0000","last_clean":"2024-04-20T21:27:48.801975+0000","last_became_active":"2024-04-20T21:27:45.253759+0000","last_became_peered":"2024-04-20T21:27:45.253759+0000","last_unstale":"2024-04-20T21:27:48.801975+0000","last_undegraded":"2024-04-20T21:27:48.801975+0000","last_fullsized":"2024-04-20T21:27:48.801975+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:16:33.146477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.287631+0000","last_change":"2024-04-20T21:27:46.289686+0000","last_active":"2024-04-20T21:27:47.287631+0000","last_peered":"2024-04-20T21:27:47.287631+0000","last_clean":"2024-04-20T21:27:47.287631+0000","last_became_active":"2024-04-20T21:27:46.288542+0000","last_became_peered":"2024-04-20T21:27:46.288542+0000","last_unstale":"2024-04-20T21:27:47.287631+0000","last_undegraded":"2024-04-20T21:27:47.287631+0000","last_fullsized":"2024-04-20T21:27:47.287631+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:47:20.176474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"99'2","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.288604+0000","last_change":"2024-04-20T21:27:45.251308+0000","last_active":"2024-04-20T21:27:47.288604+0000","last_peered":"2024-04-20T21:27:47.288604+0000","last_clean":"2024-04-20T21:27:47.288604+0000","last_became_active":"2024-04-20T21:27:45.249643+0000","last_became_peered":"2024-04-20T21:27:45.249643+0000","last_unstale":"2024-04-20T21:27:47.288604+0000","last_undegraded":"2024-04-20T21:27:47.288604+0000","last_fullsized":"2024-04-20T21:27:47.288604+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:55:12.291750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:48.605640+0000","last_change":"2024-04-20T21:27:46.267162+0000","last_active":"2024-04-20T21:27:48.605640+0000","last_peered":"2024-04-20T21:27:48.605640+0000","last_clean":"2024-04-20T21:27:48.605640+0000","last_became_active":"2024-04-20T21:27:46.266376+0000","last_became_peered":"2024-04-20T21:27:46.266376+0000","last_unstale":"2024-04-20T21:27:48.605640+0000","last_undegraded":"2024-04-20T21:27:48.605640+0000","last_fullsized":"2024-04-20T21:27:48.605640+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:34:07.336466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"99'1","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.287094+0000","last_change":"2024-04-20T21:27:45.256516+0000","last_active":"2024-04-20T21:27:47.287094+0000","last_peered":"2024-04-20T21:27:47.287094+0000","last_clean":"2024-04-20T21:27:47.287094+0000","last_became_active":"2024-04-20T21:27:45.256184+0000","last_became_peered":"2024-04-20T21:27:45.256184+0000","last_unstale":"2024-04-20T21:27:47.287094+0000","last_undegraded":"2024-04-20T21:27:47.287094+0000","last_fullsized":"2024-04-20T21:27:47.287094+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:55:49.584580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:48.171893+0000","last_change":"2024-04-20T21:27:46.291432+0000","last_active":"2024-04-20T21:27:48.171893+0000","last_peered":"2024-04-20T21:27:48.171893+0000","last_clean":"2024-04-20T21:27:48.171893+0000","last_became_active":"2024-04-20T21:27:46.291070+0000","last_became_peered":"2024-04-20T21:27:46.291070+0000","last_unstale":"2024-04-20T21:27:48.171893+0000","last_undegraded":"2024-04-20T21:27:48.171893+0000","last_fullsized":"2024-04-20T21:27:48.171893+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:32:30.121850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.292227+0000","last_change":"2024-04-20T21:27:45.262413+0000","last_active":"2024-04-20T21:27:47.292227+0000","last_peered":"2024-04-20T21:27:47.292227+0000","last_clean":"2024-04-20T21:27:47.292227+0000","last_became_active":"2024-04-20T21:27:45.262069+0000","last_became_peered":"2024-04-20T21:27:45.262069+0000","last_unstale":"2024-04-20T21:27:47.292227+0000","last_undegraded":"2024-04-20T21:27:47.292227+0000","last_fullsized":"2024-04-20T21:27:47.292227+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:58:47.584032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.284564+0000","last_change":"2024-04-20T21:27:46.267434+0000","last_active":"2024-04-20T21:27:47.284564+0000","last_peered":"2024-04-20T21:27:47.284564+0000","last_clean":"2024-04-20T21:27:47.284564+0000","last_became_active":"2024-04-20T21:27:46.267160+0000","last_became_peered":"2024-04-20T21:27:46.267160+0000","last_unstale":"2024-04-20T21:27:47.284564+0000","last_undegraded":"2024-04-20T21:27:47.284564+0000","last_fullsized":"2024-04-20T21:27:47.284564+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:35:07.377012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"99'2","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.292603+0000","last_change":"2024-04-20T21:27:45.250116+0000","last_active":"2024-04-20T21:27:47.292603+0000","last_peered":"2024-04-20T21:27:47.292603+0000","last_clean":"2024-04-20T21:27:47.292603+0000","last_became_active":"2024-04-20T21:27:45.248147+0000","last_became_peered":"2024-04-20T21:27:45.248147+0000","last_unstale":"2024-04-20T21:27:47.292603+0000","last_undegraded":"2024-04-20T21:27:47.292603+0000","last_fullsized":"2024-04-20T21:27:47.292603+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:53:57.307259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.284338+0000","last_change":"2024-04-20T21:27:46.287043+0000","last_active":"2024-04-20T21:27:47.284338+0000","last_peered":"2024-04-20T21:27:47.284338+0000","last_clean":"2024-04-20T21:27:47.284338+0000","last_became_active":"2024-04-20T21:27:46.286741+0000","last_became_peered":"2024-04-20T21:27:46.286741+0000","last_unstale":"2024-04-20T21:27:47.284338+0000","last_undegraded":"2024-04-20T21:27:47.284338+0000","last_fullsized":"2024-04-20T21:27:47.284338+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:16:34.161599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:48.605738+0000","last_change":"2024-04-20T21:27:45.256259+0000","last_active":"2024-04-20T21:27:48.605738+0000","last_peered":"2024-04-20T21:27:48.605738+0000","last_clean":"2024-04-20T21:27:48.605738+0000","last_became_active":"2024-04-20T21:27:45.254959+0000","last_became_peered":"2024-04-20T21:27:45.254959+0000","last_unstale":"2024-04-20T21:27:48.605738+0000","last_undegraded":"2024-04-20T21:27:48.605738+0000","last_fullsized":"2024-04-20T21:27:48.605738+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:03:41.831352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.283950+0000","last_change":"2024-04-20T21:27:46.290267+0000","last_active":"2024-04-20T21:27:47.283950+0000","last_peered":"2024-04-20T21:27:47.283950+0000","last_clean":"2024-04-20T21:27:47.283950+0000","last_became_active":"2024-04-20T21:27:46.289950+0000","last_became_peered":"2024-04-20T21:27:46.289950+0000","last_unstale":"2024-04-20T21:27:47.283950+0000","last_undegraded":"2024-04-20T21:27:47.283950+0000","last_fullsized":"2024-04-20T21:27:47.283950+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:37:36.303605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:48.605339+0000","last_change":"2024-04-20T21:27:45.251046+0000","last_active":"2024-04-20T21:27:48.605339+0000","last_peered":"2024-04-20T21:27:48.605339+0000","last_clean":"2024-04-20T21:27:48.605339+0000","last_became_active":"2024-04-20T21:27:45.250764+0000","last_became_peered":"2024-04-20T21:27:45.250764+0000","last_unstale":"2024-04-20T21:27:48.605339+0000","last_undegraded":"2024-04-20T21:27:48.605339+0000","last_fullsized":"2024-04-20T21:27:48.605339+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:55:49.584580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.285445+0000","last_change":"2024-04-20T21:27:46.278326+0000","last_active":"2024-04-20T21:27:47.285445+0000","last_peered":"2024-04-20T21:27:47.285445+0000","last_clean":"2024-04-20T21:27:47.285445+0000","last_became_active":"2024-04-20T21:27:46.278011+0000","last_became_peered":"2024-04-20T21:27:46.278011+0000","last_unstale":"2024-04-20T21:27:47.285445+0000","last_undegraded":"2024-04-20T21:27:47.285445+0000","last_fullsized":"2024-04-20T21:27:47.285445+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:26:15.986186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.291969+0000","last_change":"2024-04-20T21:27:45.249470+0000","last_active":"2024-04-20T21:27:47.291969+0000","last_peered":"2024-04-20T21:27:47.291969+0000","last_clean":"2024-04-20T21:27:47.291969+0000","last_became_active":"2024-04-20T21:27:45.249115+0000","last_became_peered":"2024-04-20T21:27:45.249115+0000","last_unstale":"2024-04-20T21:27:47.291969+0000","last_undegraded":"2024-04-20T21:27:47.291969+0000","last_fullsized":"2024-04-20T21:27:47.291969+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:23:37.589540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.292877+0000","last_change":"2024-04-20T21:27:46.280295+0000","last_active":"2024-04-20T21:27:47.292877+0000","last_peered":"2024-04-20T21:27:47.292877+0000","last_clean":"2024-04-20T21:27:47.292877+0000","last_became_active":"2024-04-20T21:27:46.279572+0000","last_became_peered":"2024-04-20T21:27:46.279572+0000","last_unstale":"2024-04-20T21:27:47.292877+0000","last_undegraded":"2024-04-20T21:27:47.292877+0000","last_fullsized":"2024-04-20T21:27:47.292877+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:27:34.441068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"99'2","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:48.605922+0000","last_change":"2024-04-20T21:27:45.255106+0000","last_active":"2024-04-20T21:27:48.605922+0000","last_peered":"2024-04-20T21:27:48.605922+0000","last_clean":"2024-04-20T21:27:48.605922+0000","last_became_active":"2024-04-20T21:27:45.253372+0000","last_became_peered":"2024-04-20T21:27:45.253372+0000","last_unstale":"2024-04-20T21:27:48.605922+0000","last_undegraded":"2024-04-20T21:27:48.605922+0000","last_fullsized":"2024-04-20T21:27:48.605922+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:31:19.958850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:48.802620+0000","last_change":"2024-04-20T21:27:46.277404+0000","last_active":"2024-04-20T21:27:48.802620+0000","last_peered":"2024-04-20T21:27:48.802620+0000","last_clean":"2024-04-20T21:27:48.802620+0000","last_became_active":"2024-04-20T21:27:46.277108+0000","last_became_peered":"2024-04-20T21:27:46.277108+0000","last_unstale":"2024-04-20T21:27:48.802620+0000","last_undegraded":"2024-04-20T21:27:48.802620+0000","last_fullsized":"2024-04-20T21:27:48.802620+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:57:45.330389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"100'1","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.292817+0000","last_change":"2024-04-20T21:27:45.256313+0000","last_active":"2024-04-20T21:27:47.292817+0000","last_peered":"2024-04-20T21:27:47.292817+0000","last_clean":"2024-04-20T21:27:47.292817+0000","last_became_active":"2024-04-20T21:27:45.256016+0000","last_became_peered":"2024-04-20T21:27:45.256016+0000","last_unstale":"2024-04-20T21:27:47.292817+0000","last_undegraded":"2024-04-20T21:27:47.292817+0000","last_fullsized":"2024-04-20T21:27:47.292817+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:45:45.979789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:48.802458+0000","last_change":"2024-04-20T21:27:46.281113+0000","last_active":"2024-04-20T21:27:48.802458+0000","last_peered":"2024-04-20T21:27:48.802458+0000","last_clean":"2024-04-20T21:27:48.802458+0000","last_became_active":"2024-04-20T21:27:46.280820+0000","last_became_peered":"2024-04-20T21:27:46.280820+0000","last_unstale":"2024-04-20T21:27:48.802458+0000","last_undegraded":"2024-04-20T21:27:48.802458+0000","last_fullsized":"2024-04-20T21:27:48.802458+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:00:50.307892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.292282+0000","last_change":"2024-04-20T21:27:45.256398+0000","last_active":"2024-04-20T21:27:47.292282+0000","last_peered":"2024-04-20T21:27:47.292282+0000","last_clean":"2024-04-20T21:27:47.292282+0000","last_became_active":"2024-04-20T21:27:45.256100+0000","last_became_peered":"2024-04-20T21:27:45.256100+0000","last_unstale":"2024-04-20T21:27:47.292282+0000","last_undegraded":"2024-04-20T21:27:47.292282+0000","last_fullsized":"2024-04-20T21:27:47.292282+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:46:56.087827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.286512+0000","last_change":"2024-04-20T21:27:46.263227+0000","last_active":"2024-04-20T21:27:47.286512+0000","last_peered":"2024-04-20T21:27:47.286512+0000","last_clean":"2024-04-20T21:27:47.286512+0000","last_became_active":"2024-04-20T21:27:46.261925+0000","last_became_peered":"2024-04-20T21:27:46.261925+0000","last_unstale":"2024-04-20T21:27:47.286512+0000","last_undegraded":"2024-04-20T21:27:47.286512+0000","last_fullsized":"2024-04-20T21:27:47.286512+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:53:58.322380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.286955+0000","last_change":"2024-04-20T21:27:45.255282+0000","last_active":"2024-04-20T21:27:47.286955+0000","last_peered":"2024-04-20T21:27:47.286955+0000","last_clean":"2024-04-20T21:27:47.286955+0000","last_became_active":"2024-04-20T21:27:45.254976+0000","last_became_peered":"2024-04-20T21:27:45.254976+0000","last_unstale":"2024-04-20T21:27:47.286955+0000","last_undegraded":"2024-04-20T21:27:47.286955+0000","last_fullsized":"2024-04-20T21:27:47.286955+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:03:41.831352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.284767+0000","last_change":"2024-04-20T21:27:46.282324+0000","last_active":"2024-04-20T21:27:47.284767+0000","last_peered":"2024-04-20T21:27:47.284767+0000","last_clean":"2024-04-20T21:27:47.284767+0000","last_became_active":"2024-04-20T21:27:46.282001+0000","last_became_peered":"2024-04-20T21:27:46.282001+0000","last_unstale":"2024-04-20T21:27:47.284767+0000","last_undegraded":"2024-04-20T21:27:47.284767+0000","last_fullsized":"2024-04-20T21:27:47.284767+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:02:55.207130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.292827+0000","last_change":"2024-04-20T21:27:45.249896+0000","last_active":"2024-04-20T21:27:47.292827+0000","last_peered":"2024-04-20T21:27:47.292827+0000","last_clean":"2024-04-20T21:27:47.292827+0000","last_became_active":"2024-04-20T21:27:45.247789+0000","last_became_peered":"2024-04-20T21:27:45.247789+0000","last_unstale":"2024-04-20T21:27:47.292827+0000","last_undegraded":"2024-04-20T21:27:47.292827+0000","last_fullsized":"2024-04-20T21:27:47.292827+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:48:48.893638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.287893+0000","last_change":"2024-04-20T21:27:46.291983+0000","last_active":"2024-04-20T21:27:47.287893+0000","last_peered":"2024-04-20T21:27:47.287893+0000","last_clean":"2024-04-20T21:27:47.287893+0000","last_became_active":"2024-04-20T21:27:46.291036+0000","last_became_peered":"2024-04-20T21:27:46.291036+0000","last_unstale":"2024-04-20T21:27:47.287893+0000","last_undegraded":"2024-04-20T21:27:47.287893+0000","last_fullsized":"2024-04-20T21:27:47.287893+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:25:18.062710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.285344+0000","last_change":"2024-04-20T21:27:45.254136+0000","last_active":"2024-04-20T21:27:47.285344+0000","last_peered":"2024-04-20T21:27:47.285344+0000","last_clean":"2024-04-20T21:27:47.285344+0000","last_became_active":"2024-04-20T21:27:45.251225+0000","last_became_peered":"2024-04-20T21:27:45.251225+0000","last_unstale":"2024-04-20T21:27:47.285344+0000","last_undegraded":"2024-04-20T21:27:47.285344+0000","last_fullsized":"2024-04-20T21:27:47.285344+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:17:01.348366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.284074+0000","last_change":"2024-04-20T21:27:46.285631+0000","last_active":"2024-04-20T21:27:47.284074+0000","last_peered":"2024-04-20T21:27:47.284074+0000","last_clean":"2024-04-20T21:27:47.284074+0000","last_became_active":"2024-04-20T21:27:46.285368+0000","last_became_peered":"2024-04-20T21:27:46.285368+0000","last_unstale":"2024-04-20T21:27:47.284074+0000","last_undegraded":"2024-04-20T21:27:47.284074+0000","last_fullsized":"2024-04-20T21:27:47.284074+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:25:56.849223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:48.070428+0000","last_change":"2024-04-20T21:27:45.243432+0000","last_active":"2024-04-20T21:27:48.070428+0000","last_peered":"2024-04-20T21:27:48.070428+0000","last_clean":"2024-04-20T21:27:48.070428+0000","last_became_active":"2024-04-20T21:27:45.243095+0000","last_became_peered":"2024-04-20T21:27:45.243095+0000","last_unstale":"2024-04-20T21:27:48.070428+0000","last_undegraded":"2024-04-20T21:27:48.070428+0000","last_fullsized":"2024-04-20T21:27:48.070428+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:08:33.098011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.285850+0000","last_change":"2024-04-20T21:27:46.274580+0000","last_active":"2024-04-20T21:27:47.285850+0000","last_peered":"2024-04-20T21:27:47.285850+0000","last_clean":"2024-04-20T21:27:47.285850+0000","last_became_active":"2024-04-20T21:27:46.268050+0000","last_became_peered":"2024-04-20T21:27:46.268050+0000","last_unstale":"2024-04-20T21:27:47.285850+0000","last_undegraded":"2024-04-20T21:27:47.285850+0000","last_fullsized":"2024-04-20T21:27:47.285850+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:55:13.306871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.284076+0000","last_change":"2024-04-20T21:27:45.248068+0000","last_active":"2024-04-20T21:27:47.284076+0000","last_peered":"2024-04-20T21:27:47.284076+0000","last_clean":"2024-04-20T21:27:47.284076+0000","last_became_active":"2024-04-20T21:27:45.247799+0000","last_became_peered":"2024-04-20T21:27:45.247799+0000","last_unstale":"2024-04-20T21:27:47.284076+0000","last_undegraded":"2024-04-20T21:27:47.284076+0000","last_fullsized":"2024-04-20T21:27:47.284076+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:03:52.206579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:47.284595+0000","last_change":"2024-04-20T21:27:46.288072+0000","last_active":"2024-04-20T21:27:47.284595+0000","last_peered":"2024-04-20T21:27:47.284595+0000","last_clean":"2024-04-20T21:27:47.284595+0000","last_became_active":"2024-04-20T21:27:46.285764+0000","last_became_peered":"2024-04-20T21:27:46.285764+0000","last_unstale":"2024-04-20T21:27:47.284595+0000","last_undegraded":"2024-04-20T21:27:47.284595+0000","last_fullsized":"2024-04-20T21:27:47.284595+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:45.212618+0000","last_clean_scrub_stamp":"2024-04-20T21:27:45.212618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:16:06.953388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"99'1","reported_seq":15,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:48.170740+0000","last_change":"2024-04-20T21:27:45.258203+0000","last_active":"2024-04-20T21:27:48.170740+0000","last_peered":"2024-04-20T21:27:48.170740+0000","last_clean":"2024-04-20T21:27:48.170740+0000","last_became_active":"2024-04-20T21:27:45.257554+0000","last_became_peered":"2024-04-20T21:27:45.257554+0000","last_unstale":"2024-04-20T21:27:48.170740+0000","last_undegraded":"2024-04-20T21:27:48.170740+0000","last_fullsized":"2024-04-20T21:27:48.170740+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:27:44.197496+0000","last_clean_scrub_stamp":"2024-04-20T21:27:44.197496+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:55:12.291750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":316,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-20T21:27:48.171121+0000","last_change":"2024-04-20T21:17:43.059637+0000","last_active":"2024-04-20T21:27:48.171121+0000","last_peered":"2024-04-20T21:27:48.171121+0000","last_clean":"2024-04-20T21:27:48.171121+0000","last_became_active":"2024-04-20T21:17:43.056207+0000","last_became_peered":"2024-04-20T21:17:43.056207+0000","last_unstale":"2024-04-20T21:27:48.171121+0000","last_undegraded":"2024-04-20T21:27:48.171121+0000","last_fullsized":"2024-04-20T21:27:48.171121+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:41.036290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:41.036290+0000","last_clean_scrub_stamp":"2024-04-20T21:17:41.036290+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:58:51.091337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574972,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32976,"kb_used_data":1796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338864,"statfs":{"total":96636764160,"available":96602996736,"internally_reserved":0,"allocated":1839104,"data_stored":1125427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574973,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32400,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1245184,"data_stored":535599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574975,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32396,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339444,"statfs":{"total":96636764160,"available":96603590656,"internally_reserved":0,"allocated":1241088,"data_stored":535017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574974,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32412,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339428,"statfs":{"total":96636764160,"available":96603574272,"internally_reserved":0,"allocated":1257472,"data_stored":536683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574974,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32412,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339428,"statfs":{"total":96636764160,"available":96603574272,"internally_reserved":0,"allocated":1253376,"data_stored":536581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574974,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32400,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1241088,"data_stored":534981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574973,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32392,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339448,"statfs":{"total":96636764160,"available":96603594752,"internally_reserved":0,"allocated":1236992,"data_stored":534947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574974,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32972,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338868,"statfs":{"total":96636764160,"available":96603000832,"internally_reserved":0,"allocated":1835008,"data_stored":1125897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T21:27:51.180 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-20T21:27:51.180 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-20T21:27:51.184 INFO:tasks.ceph.mds.c.smithi086.stderr:2024-04-20T21:27:51.185+0000 7f1c04800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T21:27:51.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.393+0000 7f57efe00640 1 Processor -- start 2024-04-20T21:27:51.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.393+0000 7f57efe00640 1 -- start start 2024-04-20T21:27:51.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.395+0000 7f57efe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f57f00d45a0 0x7f57f00d3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:51.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.395+0000 7f57efe00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f57f015ea70 0x7f57f00d4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:51.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.395+0000 7f57efe00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57f00cf120 con 0x7f57f006d570 2024-04-20T21:27:51.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.395+0000 7f57efe00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f57f006de70 con 0x7f57f00d45a0 2024-04-20T21:27:51.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.395+0000 7f57efe00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f57f006e150 con 0x7f57f015ea70 2024-04-20T21:27:51.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.395+0000 7f57eea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f57f00d45a0 0x7f57f00d3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:51.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.395+0000 7f57eea00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f57f00d45a0 0x7f57f00d3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36634/0 (socket says 172.21.15.86:36634) 2024-04-20T21:27:51.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.395+0000 7f57eea00640 1 -- 172.21.15.86:0/2161038171 learned_addr learned my addr 172.21.15.86:0/2161038171 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:51.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.395+0000 7f57ee000640 1 --2- 172.21.15.86:0/2161038171 >> v2:172.21.15.144:3300/0 conn(0x7f57f015ea70 0x7f57f00d4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:51.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.396+0000 7f57eea00640 1 -- 172.21.15.86:0/2161038171 >> v2:172.21.15.144:3300/0 conn(0x7f57f015ea70 msgr2=0x7f57f00d4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:51.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.396+0000 7f57eea00640 1 --2- 172.21.15.86:0/2161038171 >> v2:172.21.15.144:3300/0 conn(0x7f57f015ea70 0x7f57f00d4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:51.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.396+0000 7f57eea00640 1 -- 172.21.15.86:0/2161038171 >> v1:172.21.15.144:6789/0 conn(0x7f57f006d570 legacy=0x7f57f00d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:51.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.396+0000 7f57eea00640 1 -- 172.21.15.86:0/2161038171 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57f006e430 con 0x7f57f00d45a0 2024-04-20T21:27:51.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.396+0000 7f57ee000640 1 --2- 172.21.15.86:0/2161038171 >> v2:172.21.15.144:3300/0 conn(0x7f57f015ea70 0x7f57f00d4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:27:51.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.397+0000 7f57eea00640 1 --2- 172.21.15.86:0/2161038171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f00d45a0 0x7f57f00d3260 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f57f008e910 tx=0x7f57e002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe5fd896138d7611 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:51.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.397+0000 7f57ed600640 1 -- 172.21.15.86:0/2161038171 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e00317c0 con 0x7f57f00d45a0 2024-04-20T21:27:51.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.397+0000 7f57ed600640 1 -- 172.21.15.86:0/2161038171 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57e0037070 con 0x7f57f00d45a0 2024-04-20T21:27:51.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.397+0000 7f57ed600640 1 -- 172.21.15.86:0/2161038171 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e0031b80 con 0x7f57f00d45a0 2024-04-20T21:27:51.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.398+0000 7f57efe00640 1 -- 172.21.15.86:0/2161038171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f00d45a0 msgr2=0x7f57f00d3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:51.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.398+0000 7f57efe00640 1 --2- 172.21.15.86:0/2161038171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f00d45a0 0x7f57f00d3260 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f57f008e910 tx=0x7f57e002b860 comp rx=0 tx=0).stop 2024-04-20T21:27:51.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.398+0000 7f57efe00640 1 -- 172.21.15.86:0/2161038171 shutdown_connections 2024-04-20T21:27:51.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.398+0000 7f57efe00640 1 --2- 172.21.15.86:0/2161038171 >> v2:172.21.15.144:3300/0 conn(0x7f57f015ea70 0x7f57f00d4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:51.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.398+0000 7f57efe00640 1 --2- 172.21.15.86:0/2161038171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f00d45a0 0x7f57f00d3260 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:51.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.398+0000 7f57efe00640 1 -- 172.21.15.86:0/2161038171 >> 172.21.15.86:0/2161038171 conn(0x7f57f009d9d0 msgr2=0x7f57f008bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:51.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.398+0000 7f57efe00640 1 -- 172.21.15.86:0/2161038171 shutdown_connections 2024-04-20T21:27:51.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.398+0000 7f57efe00640 1 -- 172.21.15.86:0/2161038171 wait complete. 2024-04-20T21:27:51.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.400+0000 7f57efe00640 1 Processor -- start 2024-04-20T21:27:51.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.400+0000 7f57efe00640 1 -- start start 2024-04-20T21:27:51.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.402+0000 7f57efe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57f006d570 0x7f57f00cd550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:51.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.402+0000 7f57efe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57f00d45a0 0x7f57f0157540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:51.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.402+0000 7f57efe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f015ea70 0x7f57f00c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:51.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.402+0000 7f57efe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f57f006d9a0 con 0x7f57f015ea70 2024-04-20T21:27:51.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.402+0000 7f57efe00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f57f00d4ec0 con 0x7f57f00d45a0 2024-04-20T21:27:51.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.402+0000 7f57efe00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f57f006db20 con 0x7f57f006d570 2024-04-20T21:27:51.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.402+0000 7f57ef400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f015ea70 0x7f57f00c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:51.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.402+0000 7f57ef400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f015ea70 0x7f57f00c9fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36636/0 (socket says 172.21.15.86:36636) 2024-04-20T21:27:51.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.402+0000 7f57ef400640 1 -- 172.21.15.86:0/1971748819 learned_addr learned my addr 172.21.15.86:0/1971748819 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:51.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.402+0000 7f57eea00640 1 --2- 172.21.15.86:0/1971748819 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57f006d570 0x7f57f00cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:51.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.402+0000 7f57ee000640 1 --2- 172.21.15.86:0/1971748819 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57f00d45a0 0x7f57f0157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:51.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.402+0000 7f57ef400640 1 -- 172.21.15.86:0/1971748819 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57f006d570 msgr2=0x7f57f00cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:51.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.402+0000 7f57ef400640 1 --2- 172.21.15.86:0/1971748819 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57f006d570 0x7f57f00cd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:51.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.402+0000 7f57ef400640 1 -- 172.21.15.86:0/1971748819 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57f00d45a0 msgr2=0x7f57f0157540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:51.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.402+0000 7f57ef400640 1 --2- 172.21.15.86:0/1971748819 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57f00d45a0 0x7f57f0157540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:51.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.402+0000 7f57ef400640 1 -- 172.21.15.86:0/1971748819 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57f015b820 con 0x7f57f015ea70 2024-04-20T21:27:51.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.402+0000 7f57ee000640 1 --2- 172.21.15.86:0/1971748819 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57f00d45a0 0x7f57f0157540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:27:51.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.403+0000 7f57ef400640 1 --2- 172.21.15.86:0/1971748819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f015ea70 0x7f57f00c9fe0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f57e800b970 tx=0x7f57e800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:51.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.403+0000 7f57e7400640 1 -- 172.21.15.86:0/1971748819 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e8019070 con 0x7f57f015ea70 2024-04-20T21:27:51.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.403+0000 7f57e7400640 1 -- 172.21.15.86:0/1971748819 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57e800cb40 con 0x7f57f015ea70 2024-04-20T21:27:51.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.403+0000 7f57efe00640 1 -- 172.21.15.86:0/1971748819 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57f00cc1d0 con 0x7f57f015ea70 2024-04-20T21:27:51.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.403+0000 7f57e7400640 1 -- 172.21.15.86:0/1971748819 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e8014440 con 0x7f57f015ea70 2024-04-20T21:27:51.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.403+0000 7f57efe00640 1 -- 172.21.15.86:0/1971748819 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57f00cc790 con 0x7f57f015ea70 2024-04-20T21:27:51.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.405+0000 7f57efe00640 1 -- 172.21.15.86:0/1971748819 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57f004fd90 con 0x7f57f015ea70 2024-04-20T21:27:51.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.406+0000 7f57e7400640 1 -- 172.21.15.86:0/1971748819 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f57e801c070 con 0x7f57f015ea70 2024-04-20T21:27:51.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.406+0000 7f57e7400640 1 --2- 172.21.15.86:0/1971748819 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f57d006c8f0 0x7f57d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:51.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.407+0000 7f57e7400640 1 -- 172.21.15.86:0/1971748819 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7f57e808b790 con 0x7f57f015ea70 2024-04-20T21:27:51.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.407+0000 7f57eea00640 1 --2- 172.21.15.86:0/1971748819 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f57d006c8f0 0x7f57d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:51.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.407+0000 7f57eea00640 1 --2- 172.21.15.86:0/1971748819 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f57d006c8f0 0x7f57d006edb0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f57e0036000 tx=0x7f57e003d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:51.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.411+0000 7f57e7400640 1 -- 172.21.15.86:0/1971748819 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f57e8059bb0 con 0x7f57f015ea70 2024-04-20T21:27:51.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.694+0000 7f57efe00640 1 -- 172.21.15.86:0/1971748819 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f57f00b4e40 con 0x7f57f015ea70 2024-04-20T21:27:51.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.698+0000 7f57e7400640 1 -- 172.21.15.86:0/1971748819 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) ==== 190+0+147 (secure 0 0 0) 0x7f57e80594b0 con 0x7f57f015ea70 2024-04-20T21:27:51.697 INFO:teuthology.orchestra.run.smithi086.stdout:[client.0] 2024-04-20T21:27:51.697 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQDBMCRmHWxyKRAANzhRi4eajp68h/jWkhFMnw== 2024-04-20T21:27:51.697 INFO:teuthology.orchestra.run.smithi086.stdout: caps mds = "allow" 2024-04-20T21:27:51.697 INFO:teuthology.orchestra.run.smithi086.stdout: caps mon = "allow r" 2024-04-20T21:27:51.697 INFO:teuthology.orchestra.run.smithi086.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-20T21:27:51.697 INFO:teuthology.orchestra.run.smithi086.stderr:updated caps for client.0 2024-04-20T21:27:51.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.700+0000 7f57efe00640 1 -- 172.21.15.86:0/1971748819 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f57d006c8f0 msgr2=0x7f57d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:51.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.700+0000 7f57efe00640 1 --2- 172.21.15.86:0/1971748819 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f57d006c8f0 0x7f57d006edb0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f57e0036000 tx=0x7f57e003d040 comp rx=0 tx=0).stop 2024-04-20T21:27:51.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.700+0000 7f57efe00640 1 -- 172.21.15.86:0/1971748819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f015ea70 msgr2=0x7f57f00c9fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:51.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.700+0000 7f57efe00640 1 --2- 172.21.15.86:0/1971748819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f015ea70 0x7f57f00c9fe0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f57e800b970 tx=0x7f57e800be30 comp rx=0 tx=0).stop 2024-04-20T21:27:51.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.700+0000 7f57efe00640 1 -- 172.21.15.86:0/1971748819 shutdown_connections 2024-04-20T21:27:51.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.700+0000 7f57efe00640 1 --2- 172.21.15.86:0/1971748819 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f57d006c8f0 0x7f57d006edb0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:51.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.700+0000 7f57efe00640 1 --2- 172.21.15.86:0/1971748819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f015ea70 0x7f57f00c9fe0 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:51.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.700+0000 7f57efe00640 1 --2- 172.21.15.86:0/1971748819 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57f00d45a0 0x7f57f0157540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:51.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.701+0000 7f57efe00640 1 --2- 172.21.15.86:0/1971748819 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f57f006d570 0x7f57f00cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:51.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.701+0000 7f57efe00640 1 -- 172.21.15.86:0/1971748819 >> 172.21.15.86:0/1971748819 conn(0x7f57f009d9d0 msgr2=0x7f57f00cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:51.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.701+0000 7f57efe00640 1 -- 172.21.15.86:0/1971748819 shutdown_connections 2024-04-20T21:27:51.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.701+0000 7f57efe00640 1 -- 172.21.15.86:0/1971748819 wait complete. 2024-04-20T21:27:51.718 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:27:51.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.910+0000 7f2654a00640 1 Processor -- start 2024-04-20T21:27:51.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.910+0000 7f2654a00640 1 -- start start 2024-04-20T21:27:51.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.912+0000 7f2654a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f265006e6d0 0x7f2650071d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:51.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.912+0000 7f2654a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2650072240 0x7f265015cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:51.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.912+0000 7f2654a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f265006eb30 con 0x7f2650146130 2024-04-20T21:27:51.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.912+0000 7f2654a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f265006dc30 con 0x7f265006e6d0 2024-04-20T21:27:51.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.912+0000 7f2654a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f26500cd8e0 con 0x7f2650072240 2024-04-20T21:27:51.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.912+0000 7f264de00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f2650146130 0x7f26500ccdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57392/0 (socket says 172.21.15.86:57392) 2024-04-20T21:27:51.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.912+0000 7f264de00640 1 -- 172.21.15.86:0/2855133968 learned_addr learned my addr 172.21.15.86:0/2855133968 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:27:51.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.912+0000 7f2647e00640 1 --2- 172.21.15.86:0/2855133968 >> v2:172.21.15.144:3300/0 conn(0x7f265006e6d0 0x7f2650071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:51.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.912+0000 7f264e800640 1 --2- 172.21.15.86:0/2855133968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2650072240 0x7f265015cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:51.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.912+0000 7f264d400640 1 -- 172.21.15.86:0/2855133968 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3407988952 0 0) 0x7f2640001110 con 0x7f2650146130 2024-04-20T21:27:51.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.912+0000 7f264d400640 1 -- 172.21.15.86:0/2855133968 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2634003520 con 0x7f2650146130 2024-04-20T21:27:51.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.913+0000 7f264d400640 1 -- 172.21.15.86:0/2855133968 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1493952090 0 0) 0x7f2634003520 con 0x7f2650146130 2024-04-20T21:27:51.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.913+0000 7f264d400640 1 -- 172.21.15.86:0/2855133968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2650072240 msgr2=0x7f265015cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:51.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.913+0000 7f264d400640 1 --2- 172.21.15.86:0/2855133968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2650072240 0x7f265015cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:51.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.913+0000 7f264d400640 1 -- 172.21.15.86:0/2855133968 >> v2:172.21.15.144:3300/0 conn(0x7f265006e6d0 msgr2=0x7f2650071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:51.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.913+0000 7f264d400640 1 --2- 172.21.15.86:0/2855133968 >> v2:172.21.15.144:3300/0 conn(0x7f265006e6d0 0x7f2650071d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:51.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.913+0000 7f264d400640 1 -- 172.21.15.86:0/2855133968 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26500cdbc0 con 0x7f2650146130 2024-04-20T21:27:51.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.913+0000 7f264d400640 1 -- 172.21.15.86:0/2855133968 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f2640001840 con 0x7f2650146130 2024-04-20T21:27:51.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.913+0000 7f264d400640 1 -- 172.21.15.86:0/2855133968 >> v1:172.21.15.86:6789/0 conn(0x7f2650146130 legacy=0x7f26500ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:51.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.913+0000 7f264d400640 1 --2- 172.21.15.86:0/2855133968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2650072240 0x7f26340039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:51.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.913+0000 7f264d400640 1 --2- 172.21.15.86:0/2855133968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2634003f30 0x7f2634004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:51.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.913+0000 7f264d400640 1 --2- 172.21.15.86:0/2855133968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2634004d40 0x7f26340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:51.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.913+0000 7f264d400640 1 -- 172.21.15.86:0/2855133968 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f265006dc30 con 0x7f2634004d40 2024-04-20T21:27:51.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.913+0000 7f264d400640 1 -- 172.21.15.86:0/2855133968 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f26500cd8e0 con 0x7f2634003f30 2024-04-20T21:27:51.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.913+0000 7f264d400640 1 -- 172.21.15.86:0/2855133968 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f263400b9f0 con 0x7f2650072240 2024-04-20T21:27:51.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.913+0000 7f264e800640 1 --2- 172.21.15.86:0/2855133968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2634004d40 0x7f26340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:51.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.914+0000 7f2647e00640 1 --2- 172.21.15.86:0/2855133968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2634003f30 0x7f2634004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:51.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.914+0000 7f264de00640 1 --2- 172.21.15.86:0/2855133968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2650072240 0x7f26340039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:51.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.914+0000 7f264e800640 1 -- 172.21.15.86:0/2855133968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2650072240 msgr2=0x7f26340039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:51.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.914+0000 7f264e800640 1 --2- 172.21.15.86:0/2855133968 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2650072240 0x7f26340039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:51.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.914+0000 7f264e800640 1 -- 172.21.15.86:0/2855133968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2634003f30 msgr2=0x7f2634004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:51.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.914+0000 7f264e800640 1 --2- 172.21.15.86:0/2855133968 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2634003f30 0x7f2634004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:51.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.914+0000 7f264e800640 1 -- 172.21.15.86:0/2855133968 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2634003520 con 0x7f2634004d40 2024-04-20T21:27:51.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.915+0000 7f264e800640 1 --2- 172.21.15.86:0/2855133968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2634004d40 0x7f26340051c0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f2648009fc0 tx=0x7f2634005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc0298724cec400 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:51.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.915+0000 7f264d400640 1 -- 172.21.15.86:0/2855133968 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2648039070 con 0x7f2634004d40 2024-04-20T21:27:51.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.915+0000 7f264d400640 1 -- 172.21.15.86:0/2855133968 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f264802cc20 con 0x7f2634004d40 2024-04-20T21:27:51.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.915+0000 7f264d400640 1 -- 172.21.15.86:0/2855133968 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2648009c10 con 0x7f2634004d40 2024-04-20T21:27:51.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.916+0000 7f2654a00640 1 -- 172.21.15.86:0/2855133968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2634004d40 msgr2=0x7f26340051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:51.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.916+0000 7f2654a00640 1 --2- 172.21.15.86:0/2855133968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2634004d40 0x7f26340051c0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f2648009fc0 tx=0x7f2634005830 comp rx=0 tx=0).stop 2024-04-20T21:27:51.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.916+0000 7f264e800640 1 -- 172.21.15.86:0/2855133968 reap_dead start 2024-04-20T21:27:51.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.916+0000 7f2654a00640 1 -- 172.21.15.86:0/2855133968 shutdown_connections 2024-04-20T21:27:51.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.916+0000 7f2654a00640 1 -- 172.21.15.86:0/2855133968 >> 172.21.15.86:0/2855133968 conn(0x7f26500b2360 msgr2=0x7f26500b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:51.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.916+0000 7f2654a00640 1 -- 172.21.15.86:0/2855133968 shutdown_connections 2024-04-20T21:27:51.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.916+0000 7f2654a00640 1 -- 172.21.15.86:0/2855133968 wait complete. 2024-04-20T21:27:51.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.918+0000 7f2654a00640 1 Processor -- start 2024-04-20T21:27:51.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.918+0000 7f2654a00640 1 -- start start 2024-04-20T21:27:51.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.919+0000 7f2654a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2634004d40 0x7f265015db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:51.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.919+0000 7f2654a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2650137760 0x7f265014d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:51.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.919+0000 7f2654a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2650137be0 0x7f265014d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:51.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.919+0000 7f2654a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f265006f150 con 0x7f2634004d40 2024-04-20T21:27:51.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.919+0000 7f2654a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f265006e1b0 con 0x7f2650137760 2024-04-20T21:27:51.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.919+0000 7f2654a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f26500cd4e0 con 0x7f2650137be0 2024-04-20T21:27:51.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.919+0000 7f264de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2634004d40 0x7f265015db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:51.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.919+0000 7f264de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2634004d40 0x7f265015db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36660/0 (socket says 172.21.15.86:36660) 2024-04-20T21:27:51.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.919+0000 7f264de00640 1 -- 172.21.15.86:0/526847285 learned_addr learned my addr 172.21.15.86:0/526847285 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:51.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.920+0000 7f2647e00640 1 --2- 172.21.15.86:0/526847285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2650137760 0x7f265014d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:51.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.920+0000 7f264de00640 1 -- 172.21.15.86:0/526847285 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2650137be0 msgr2=0x7f265014d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:51.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.920+0000 7f264de00640 1 --2- 172.21.15.86:0/526847285 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2650137be0 0x7f265014d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:51.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.920+0000 7f264de00640 1 -- 172.21.15.86:0/526847285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2650137760 msgr2=0x7f265014d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:51.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.920+0000 7f264de00640 1 --2- 172.21.15.86:0/526847285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2650137760 0x7f265014d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:51.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.920+0000 7f264de00640 1 -- 172.21.15.86:0/526847285 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f265015e0b0 con 0x7f2634004d40 2024-04-20T21:27:51.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.920+0000 7f2647e00640 1 --2- 172.21.15.86:0/526847285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2650137760 0x7f265014d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:27:51.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.920+0000 7f264de00640 1 --2- 172.21.15.86:0/526847285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2634004d40 0x7f265015db70 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f264000d860 tx=0x7f264000dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:51.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.920+0000 7f2646000640 1 -- 172.21.15.86:0/526847285 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2640019070 con 0x7f2634004d40 2024-04-20T21:27:51.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.920+0000 7f2646000640 1 -- 172.21.15.86:0/526847285 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2640007560 con 0x7f2634004d40 2024-04-20T21:27:51.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.920+0000 7f2646000640 1 -- 172.21.15.86:0/526847285 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2640004a00 con 0x7f2634004d40 2024-04-20T21:27:51.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.920+0000 7f2654a00640 1 -- 172.21.15.86:0/526847285 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f265015e350 con 0x7f2634004d40 2024-04-20T21:27:51.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.920+0000 7f2654a00640 1 -- 172.21.15.86:0/526847285 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f265015e910 con 0x7f2634004d40 2024-04-20T21:27:51.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.921+0000 7f2654a00640 1 -- 172.21.15.86:0/526847285 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2650072ab0 con 0x7f2634004d40 2024-04-20T21:27:51.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.922+0000 7f2646000640 1 -- 172.21.15.86:0/526847285 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2640004ba0 con 0x7f2634004d40 2024-04-20T21:27:51.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.923+0000 7f2646000640 1 --2- 172.21.15.86:0/526847285 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f262c06c8f0 0x7f262c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:51.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.923+0000 7f2646000640 1 -- 172.21.15.86:0/526847285 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7f264008b6d0 con 0x7f2634004d40 2024-04-20T21:27:51.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.924+0000 7f2647e00640 1 --2- 172.21.15.86:0/526847285 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f262c06c8f0 0x7f262c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:51.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.924+0000 7f2647e00640 1 --2- 172.21.15.86:0/526847285 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f262c06c8f0 0x7f262c06edb0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f265015f950 tx=0x7f2638008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:51.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:51.928+0000 7f2646000640 1 -- 172.21.15.86:0/526847285 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f264004fb90 con 0x7f2634004d40 2024-04-20T21:27:52.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.165+0000 7f2654a00640 1 -- 172.21.15.86:0/526847285 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f26500689e0 con 0x7f2634004d40 2024-04-20T21:27:52.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.166+0000 7f2646000640 1 -- 172.21.15.86:0/526847285 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5207 (secure 0 0 0) 0x7f2650072ab0 con 0x7f2634004d40 2024-04-20T21:27:52.168 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:27:52.168 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":82,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5525,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3902778887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3902778887},{"type":"v1","addr":"172.21.15.144:6835","nonce":3902778887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6001,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/4270988014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":4270988014},{"type":"v1","addr":"172.21.15.86:6837","nonce":4270988014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6016,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/1569462554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":1569462554},{"type":"v1","addr":"172.21.15.144:6833","nonce":1569462554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:27:45.692408+0000","modified":"2024-04-20T21:27:48.770936+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8493":{"gid":8493,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6836/1331851138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1331851138},{"type":"v1","addr":"172.21.15.86:6836","nonce":1331851138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8493,"qdb_cluster":[8493]},"id":7}]} 2024-04-20T21:27:52.169 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 82 2024-04-20T21:27:52.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.169+0000 7f2654a00640 1 -- 172.21.15.86:0/526847285 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f262c06c8f0 msgr2=0x7f262c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:52.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.169+0000 7f2654a00640 1 --2- 172.21.15.86:0/526847285 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f262c06c8f0 0x7f262c06edb0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f265015f950 tx=0x7f2638008040 comp rx=0 tx=0).stop 2024-04-20T21:27:52.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.169+0000 7f2654a00640 1 -- 172.21.15.86:0/526847285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2634004d40 msgr2=0x7f265015db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:52.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.169+0000 7f2654a00640 1 --2- 172.21.15.86:0/526847285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2634004d40 0x7f265015db70 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f264000d860 tx=0x7f264000dd20 comp rx=0 tx=0).stop 2024-04-20T21:27:52.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.169+0000 7f2654a00640 1 -- 172.21.15.86:0/526847285 shutdown_connections 2024-04-20T21:27:52.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.169+0000 7f2654a00640 1 --2- 172.21.15.86:0/526847285 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f262c06c8f0 0x7f262c06edb0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:52.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.169+0000 7f2654a00640 1 --2- 172.21.15.86:0/526847285 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2650137be0 0x7f265014d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:52.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.169+0000 7f2654a00640 1 --2- 172.21.15.86:0/526847285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2650137760 0x7f265014d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:52.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.169+0000 7f2654a00640 1 --2- 172.21.15.86:0/526847285 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2634004d40 0x7f265015db70 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:52.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.170+0000 7f2654a00640 1 -- 172.21.15.86:0/526847285 >> 172.21.15.86:0/526847285 conn(0x7f26500b2360 msgr2=0x7f26500b1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:52.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.170+0000 7f2654a00640 1 -- 172.21.15.86:0/526847285 shutdown_connections 2024-04-20T21:27:52.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.170+0000 7f2654a00640 1 -- 172.21.15.86:0/526847285 wait complete. 2024-04-20T21:27:52.590 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 82, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T21:27:45.692408+0000', 'modified': '2024-04-20T21:27:48.770936+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8493}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8493': {'gid': 8493, 'name': 'c', 'rank': 0, 'incarnation': 78, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6836/1331851138', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6834', 'nonce': 1331851138}, {'type': 'v1', 'addr': '172.21.15.86:6836', 'nonce': 1331851138}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8493, 'qdb_cluster': [8493]} 2024-04-20T21:27:52.591 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-20T21:27:52.591 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-20T21:27:52.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.802+0000 7f59b8e00640 1 Processor -- start 2024-04-20T21:27:52.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.802+0000 7f59b8e00640 1 -- start start 2024-04-20T21:27:52.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.804+0000 7f59b8e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f59b4145fe0 0x7f59b40d02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:52.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.804+0000 7f59b8e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f59b40cd7e0 0x7f59b4071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:52.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.804+0000 7f59b8e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59b415dcc0 con 0x7f59b40d0810 2024-04-20T21:27:52.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.804+0000 7f59b8e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f59b40d00d0 con 0x7f59b40cd7e0 2024-04-20T21:27:52.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.804+0000 7f59b8e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f59b40d0c70 con 0x7f59b4145fe0 2024-04-20T21:27:52.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.804+0000 7f59b2800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f59b40cd7e0 0x7f59b4071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:52.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.804+0000 7f59b2800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f59b40cd7e0 0x7f59b4071cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36668/0 (socket says 172.21.15.86:36668) 2024-04-20T21:27:52.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.804+0000 7f59b2800640 1 -- 172.21.15.86:0/1911252516 learned_addr learned my addr 172.21.15.86:0/1911252516 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:52.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.805+0000 7f59b1e00640 1 --2- 172.21.15.86:0/1911252516 >> v2:172.21.15.144:3300/0 conn(0x7f59b4145fe0 0x7f59b40d02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:52.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.805+0000 7f59b2800640 1 -- 172.21.15.86:0/1911252516 >> v2:172.21.15.144:3300/0 conn(0x7f59b4145fe0 msgr2=0x7f59b40d02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:52.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.805+0000 7f59b2800640 1 --2- 172.21.15.86:0/1911252516 >> v2:172.21.15.144:3300/0 conn(0x7f59b4145fe0 0x7f59b40d02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:52.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.805+0000 7f59b2800640 1 -- 172.21.15.86:0/1911252516 >> v1:172.21.15.144:6789/0 conn(0x7f59b40d0810 legacy=0x7f59b40cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:52.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.805+0000 7f59b2800640 1 -- 172.21.15.86:0/1911252516 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59b415cf00 con 0x7f59b40cd7e0 2024-04-20T21:27:52.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.806+0000 7f59b2800640 1 --2- 172.21.15.86:0/1911252516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59b40cd7e0 0x7f59b4071cd0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f59b406d5c0 tx=0x7f59ac02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dc7120879c7fd57 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:52.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.806+0000 7f59b0a00640 1 -- 172.21.15.86:0/1911252516 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ac02ca80 con 0x7f59b40cd7e0 2024-04-20T21:27:52.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.806+0000 7f59b0a00640 1 -- 172.21.15.86:0/1911252516 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59ac02cc20 con 0x7f59b40cd7e0 2024-04-20T21:27:52.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.806+0000 7f59b0a00640 1 -- 172.21.15.86:0/1911252516 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ac0343f0 con 0x7f59b40cd7e0 2024-04-20T21:27:52.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.807+0000 7f59b8e00640 1 -- 172.21.15.86:0/1911252516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59b40cd7e0 msgr2=0x7f59b4071cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:52.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.807+0000 7f59b8e00640 1 --2- 172.21.15.86:0/1911252516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59b40cd7e0 0x7f59b4071cd0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f59b406d5c0 tx=0x7f59ac02bee0 comp rx=0 tx=0).stop 2024-04-20T21:27:52.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.807+0000 7f59b8e00640 1 -- 172.21.15.86:0/1911252516 shutdown_connections 2024-04-20T21:27:52.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.807+0000 7f59b8e00640 1 --2- 172.21.15.86:0/1911252516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59b40cd7e0 0x7f59b4071cd0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:52.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.807+0000 7f59b8e00640 1 --2- 172.21.15.86:0/1911252516 >> v2:172.21.15.144:3300/0 conn(0x7f59b4145fe0 0x7f59b40d02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:52.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.807+0000 7f59b8e00640 1 -- 172.21.15.86:0/1911252516 >> 172.21.15.86:0/1911252516 conn(0x7f59b406b980 msgr2=0x7f59b406bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:52.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.807+0000 7f59b8e00640 1 -- 172.21.15.86:0/1911252516 shutdown_connections 2024-04-20T21:27:52.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.807+0000 7f59b8e00640 1 -- 172.21.15.86:0/1911252516 wait complete. 2024-04-20T21:27:52.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.809+0000 7f59b8e00640 1 Processor -- start 2024-04-20T21:27:52.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.810+0000 7f59b8e00640 1 -- start start 2024-04-20T21:27:52.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.811+0000 7f59b8e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59b40cd7e0 0x7f59b41315a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:52.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.811+0000 7f59b8e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59b40d0810 0x7f59b415bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:52.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.811+0000 7f59b8e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59b4145fe0 0x7f59b40b1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:52.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.811+0000 7f59b8e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f59b415d7a0 con 0x7f59b40cd7e0 2024-04-20T21:27:52.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.811+0000 7f59b8e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f59b415d4c0 con 0x7f59b4145fe0 2024-04-20T21:27:52.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.811+0000 7f59b8e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f59b41463c0 con 0x7f59b40d0810 2024-04-20T21:27:52.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.811+0000 7f59b1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59b40cd7e0 0x7f59b41315a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:52.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.811+0000 7f59b1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59b40cd7e0 0x7f59b41315a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36674/0 (socket says 172.21.15.86:36674) 2024-04-20T21:27:52.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.811+0000 7f59b1e00640 1 -- 172.21.15.86:0/3508440334 learned_addr learned my addr 172.21.15.86:0/3508440334 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:52.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.812+0000 7f59b2800640 1 --2- 172.21.15.86:0/3508440334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59b4145fe0 0x7f59b40b1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:52.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.812+0000 7f59b1400640 1 --2- 172.21.15.86:0/3508440334 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59b40d0810 0x7f59b415bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:52.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.812+0000 7f59b1e00640 1 -- 172.21.15.86:0/3508440334 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59b40d0810 msgr2=0x7f59b415bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:52.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.812+0000 7f59b1e00640 1 --2- 172.21.15.86:0/3508440334 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59b40d0810 0x7f59b415bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:52.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.812+0000 7f59b1e00640 1 -- 172.21.15.86:0/3508440334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59b4145fe0 msgr2=0x7f59b40b1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:52.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.812+0000 7f59b1e00640 1 --2- 172.21.15.86:0/3508440334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59b4145fe0 0x7f59b40b1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:52.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.812+0000 7f59b1e00640 1 -- 172.21.15.86:0/3508440334 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59b40b2da0 con 0x7f59b40cd7e0 2024-04-20T21:27:52.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.812+0000 7f59b1e00640 1 --2- 172.21.15.86:0/3508440334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59b40cd7e0 0x7f59b41315a0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f59a400d830 tx=0x7f59a400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:52.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.812+0000 7f59aa000640 1 -- 172.21.15.86:0/3508440334 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a400b6c0 con 0x7f59b40cd7e0 2024-04-20T21:27:52.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.812+0000 7f59aa000640 1 -- 172.21.15.86:0/3508440334 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59a4019070 con 0x7f59b40cd7e0 2024-04-20T21:27:52.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.812+0000 7f59b8e00640 1 -- 172.21.15.86:0/3508440334 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59b40b0a70 con 0x7f59b40cd7e0 2024-04-20T21:27:52.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.812+0000 7f59aa000640 1 -- 172.21.15.86:0/3508440334 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a400ba80 con 0x7f59b40cd7e0 2024-04-20T21:27:52.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.813+0000 7f59b8e00640 1 -- 172.21.15.86:0/3508440334 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59b40b1030 con 0x7f59b40cd7e0 2024-04-20T21:27:52.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.815+0000 7f59aa000640 1 -- 172.21.15.86:0/3508440334 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f59a401e460 con 0x7f59b40cd7e0 2024-04-20T21:27:52.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.816+0000 7f59aa000640 1 --2- 172.21.15.86:0/3508440334 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f599406c8f0 0x7f599406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:52.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.816+0000 7f59aa000640 1 -- 172.21.15.86:0/3508440334 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7f59a408c230 con 0x7f59b40cd7e0 2024-04-20T21:27:52.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.816+0000 7f59b1400640 1 --2- 172.21.15.86:0/3508440334 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f599406c8f0 0x7f599406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:52.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.817+0000 7f59bba7b740 1 Processor -- start 2024-04-20T21:27:52.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.817+0000 7f59bba7b740 1 -- start start 2024-04-20T21:27:52.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.817+0000 7f59b1400640 1 --2- 172.21.15.86:0/3508440334 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f599406c8f0 0x7f599406edb0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f59b40cf4c0 tx=0x7f59a0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:52.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.818+0000 7f59bba7b740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b3bfc94460 0x55b3bfc4ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:52.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.818+0000 7f59bba7b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3bfc4a4e0 0x55b3bfc4bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:52.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.818+0000 7f59bba7b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3bfc46930 0x55b3bfc76770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:52.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.818+0000 7f59bba7b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b3bfbb29d0 con 0x55b3bfc46930 2024-04-20T21:27:52.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.818+0000 7f59bba7b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b3bfb59270 con 0x55b3bfc4a4e0 2024-04-20T21:27:52.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.818+0000 7f59bba7b740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55b3bfbae990 con 0x55b3bfc94460 2024-04-20T21:27:52.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.818+0000 7f59b2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3bfc46930 0x55b3bfc76770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:52.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.818+0000 7f59b2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3bfc46930 0x55b3bfc76770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36682/0 (socket says 172.21.15.86:36682) 2024-04-20T21:27:52.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.818+0000 7f59b2800640 1 -- 172.21.15.86:0/4049502693 learned_addr learned my addr 172.21.15.86:0/4049502693 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:52.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.819+0000 7f59b2800640 1 -- 172.21.15.86:0/4049502693 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b3bfc94460 msgr2=0x55b3bfc4ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:52.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.819+0000 7f59b2800640 1 --2- 172.21.15.86:0/4049502693 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b3bfc94460 0x55b3bfc4ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:52.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.819+0000 7f59b1400640 1 --2- 172.21.15.86:0/4049502693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3bfc4a4e0 0x55b3bfc4bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:52.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.819+0000 7f59b2800640 1 -- 172.21.15.86:0/4049502693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3bfc4a4e0 msgr2=0x55b3bfc4bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:52.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.819+0000 7f59b2800640 1 --2- 172.21.15.86:0/4049502693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3bfc4a4e0 0x55b3bfc4bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:52.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.819+0000 7f59b2800640 1 -- 172.21.15.86:0/4049502693 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3bf8c52d0 con 0x55b3bfc46930 2024-04-20T21:27:52.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.819+0000 7f59b2800640 1 --2- 172.21.15.86:0/4049502693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3bfc46930 0x55b3bfc76770 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f59ac036040 tx=0x7f59ac02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7b850504c62e05a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:52.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.819+0000 7f599ea00640 1 -- 172.21.15.86:0/4049502693 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ac02ca80 con 0x55b3bfc46930 2024-04-20T21:27:52.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.819+0000 7f599ea00640 1 -- 172.21.15.86:0/4049502693 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59ac02cc20 con 0x55b3bfc46930 2024-04-20T21:27:52.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.819+0000 7f599ea00640 1 -- 172.21.15.86:0/4049502693 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ac03e630 con 0x55b3bfc46930 2024-04-20T21:27:52.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.820+0000 7f59bba7b740 1 -- 172.21.15.86:0/4049502693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3bfc46930 msgr2=0x55b3bfc76770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:52.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.820+0000 7f59bba7b740 1 --2- 172.21.15.86:0/4049502693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3bfc46930 0x55b3bfc76770 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f59ac036040 tx=0x7f59ac02e000 comp rx=0 tx=0).stop 2024-04-20T21:27:52.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.820+0000 7f59bba7b740 1 -- 172.21.15.86:0/4049502693 shutdown_connections 2024-04-20T21:27:52.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.820+0000 7f59bba7b740 1 --2- 172.21.15.86:0/4049502693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3bfc46930 0x55b3bfc76770 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:52.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.820+0000 7f59bba7b740 1 --2- 172.21.15.86:0/4049502693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3bfc4a4e0 0x55b3bfc4bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:52.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.820+0000 7f59bba7b740 1 --2- 172.21.15.86:0/4049502693 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b3bfc94460 0x55b3bfc4ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:52.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.820+0000 7f59bba7b740 1 -- 172.21.15.86:0/4049502693 >> 172.21.15.86:0/4049502693 conn(0x55b3bfc48bf0 msgr2=0x55b3bfc91770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:52.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.820+0000 7f59bba7b740 1 -- 172.21.15.86:0/4049502693 shutdown_connections 2024-04-20T21:27:52.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.820+0000 7f59bba7b740 1 -- 172.21.15.86:0/4049502693 wait complete. 2024-04-20T21:27:52.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.821+0000 7f59bba7b740 1 -- start start 2024-04-20T21:27:52.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.821+0000 7f59bba7b740 1 Processor -- start 2024-04-20T21:27:52.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.822+0000 7f59bba7b740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b3bfc4a4e0 0x55b3bfca2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:52.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.822+0000 7f59bba7b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3bfc94460 0x55b3bfc6c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:52.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.822+0000 7f59bba7b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3bfc6c940 0x55b3bfc97040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:52.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.822+0000 7f59bba7b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b3bfc4a9f0 con 0x55b3bfc94460 2024-04-20T21:27:52.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.822+0000 7f59bba7b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b3bfb6c5e0 con 0x55b3bfc6c940 2024-04-20T21:27:52.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.822+0000 7f59bba7b740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55b3bfc4acd0 con 0x55b3bfc4a4e0 2024-04-20T21:27:52.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.822+0000 7f59bba7b740 10 client.0 _ll_register_callbacks cb 0x55b3bf8c4ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:27:52.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.822+0000 7f59b1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3bfc94460 0x55b3bfc6c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:52.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.822+0000 7f59b1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3bfc94460 0x55b3bfc6c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36696/0 (socket says 172.21.15.86:36696) 2024-04-20T21:27:52.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.822+0000 7f59b1400640 1 -- 172.21.15.86:0/4261111298 learned_addr learned my addr 172.21.15.86:0/4261111298 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:52.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.823+0000 7f59b2800640 1 --2- 172.21.15.86:0/4261111298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3bfc6c940 0x55b3bfc97040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:52.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.823+0000 7f59b1e00640 1 --2- 172.21.15.86:0/4261111298 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b3bfc4a4e0 0x55b3bfca2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:52.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.823+0000 7f59b1400640 1 -- 172.21.15.86:0/4261111298 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b3bfc4a4e0 msgr2=0x55b3bfca2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:52.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.823+0000 7f59b1400640 1 --2- 172.21.15.86:0/4261111298 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b3bfc4a4e0 0x55b3bfca2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:52.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.823+0000 7f59b1400640 1 -- 172.21.15.86:0/4261111298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3bfc6c940 msgr2=0x55b3bfc97040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:52.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.823+0000 7f59b1400640 1 --2- 172.21.15.86:0/4261111298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3bfc6c940 0x55b3bfc97040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:52.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.823+0000 7f59b1400640 1 -- 172.21.15.86:0/4261111298 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3bf98f6b0 con 0x55b3bfc94460 2024-04-20T21:27:52.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.823+0000 7f59b2800640 1 --2- 172.21.15.86:0/4261111298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3bfc6c940 0x55b3bfc97040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:27:52.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.823+0000 7f59b1400640 1 --2- 172.21.15.86:0/4261111298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3bfc94460 0x55b3bfc6c230 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f59a0004870 tx=0x7f59a0003760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbb50d83884393f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:52.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.823+0000 7f5993400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:27:52.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.823+0000 7f5993400640 1 -- 172.21.15.86:0/4261111298 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a0025060 con 0x55b3bfc94460 2024-04-20T21:27:52.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.823+0000 7f5993400640 1 -- 172.21.15.86:0/4261111298 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7f59a000dc10 con 0x55b3bfc94460 2024-04-20T21:27:52.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.823+0000 7f59bba7b740 1 -- 172.21.15.86:0/4261111298 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59ac02cc20 con 0x55b3bfc94460 2024-04-20T21:27:52.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.824+0000 7f5993400640 1 -- 172.21.15.86:0/4261111298 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f59a002b050 con 0x55b3bfc94460 2024-04-20T21:27:52.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.824+0000 7f59bba7b740 10 client.8640 fetch_fsmap learned FSMap version 82 2024-04-20T21:27:52.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.824+0000 7f59bba7b740 1 -- 172.21.15.86:0/4261111298 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x55b3bfc96b20 con 0x55b3bfc94460 2024-04-20T21:27:52.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.824+0000 7f5993400640 1 -- 172.21.15.86:0/4261111298 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59ac02cc20 con 0x55b3bfc94460 2024-04-20T21:27:52.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.824+0000 7f5993400640 1 -- 172.21.15.86:0/4261111298 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 82) ==== 3495+0+0 (secure 0 0 0) 0x7f59a002e020 con 0x55b3bfc94460 2024-04-20T21:27:52.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.824+0000 7f59bba7b740 10 client.8640 fetch_fsmap finished waiting for FSMap version 82 2024-04-20T21:27:52.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.824+0000 7f59bba7b740 10 client.8640 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-20T21:27:52.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.824+0000 7f59bba7b740 20 client.8640 populate_metadata read hostname 'smithi086' 2024-04-20T21:27:52.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.824+0000 7f59bba7b740 1 --2- 172.21.15.86:0/4261111298 >> [v2:172.21.15.86:6834/1331851138,v1:172.21.15.86:6836/1331851138] conn(0x55b3bfc92b60 0x55b3bfc72110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:52.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.824+0000 7f59bba7b740 4 client.8640 mds_command: new command op to 8493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:27:52.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.824+0000 7f59bba7b740 1 -- 172.21.15.86:0/4261111298 --> [v2:172.21.15.86:6834/1331851138,v1:172.21.15.86:6836/1331851138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3bfbc03f0 con 0x55b3bfc92b60 2024-04-20T21:27:52.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.825+0000 7f59b1e00640 1 --2- 172.21.15.86:0/4261111298 >> [v2:172.21.15.86:6834/1331851138,v1:172.21.15.86:6836/1331851138] conn(0x55b3bfc92b60 0x55b3bfc72110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:52.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.825+0000 7f59b1e00640 1 --2- 172.21.15.86:0/4261111298 >> [v2:172.21.15.86:6834/1331851138,v1:172.21.15.86:6836/1331851138] conn(0x55b3bfc92b60 0x55b3bfc72110 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca5e8d18618bb2a7 server_cookie=17b4e1c369d0a6fb in_seq=0 out_seq=0 2024-04-20T21:27:52.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.825+0000 7f5993400640 10 client.8640 ms_handle_connect on v2:172.21.15.86:6834/1331851138 2024-04-20T21:27:52.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.826+0000 7f5993400640 1 -- 172.21.15.86:0/4261111298 <== mds.0 v2:172.21.15.86:6834/1331851138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b3bfbc03f0 con 0x55b3bfc92b60 2024-04-20T21:27:52.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.826+0000 7f5993400640 10 client.8640 handle_command_reply: tid=0 2024-04-20T21:27:52.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.826+0000 7f59bba7b740 1 client.8640 shutdown 2024-04-20T21:27:52.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.826+0000 7f59bba7b740 1 -- 172.21.15.86:0/4261111298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3bfc94460 msgr2=0x55b3bfc6c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:52.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.826+0000 7f59bba7b740 1 --2- 172.21.15.86:0/4261111298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3bfc94460 0x55b3bfc6c230 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f59a0004870 tx=0x7f59a0003760 comp rx=0 tx=0).stop 2024-04-20T21:27:52.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.826+0000 7f59bba7b740 1 -- 172.21.15.86:0/4261111298 shutdown_connections 2024-04-20T21:27:52.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.826+0000 7f59bba7b740 1 --2- 172.21.15.86:0/4261111298 >> [v2:172.21.15.86:6834/1331851138,v1:172.21.15.86:6836/1331851138] conn(0x55b3bfc92b60 0x55b3bfc72110 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:52.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.826+0000 7f59bba7b740 1 --2- 172.21.15.86:0/4261111298 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3bfc6c940 0x55b3bfc97040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:52.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.826+0000 7f59bba7b740 1 --2- 172.21.15.86:0/4261111298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3bfc94460 0x55b3bfc6c230 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:52.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.826+0000 7f59bba7b740 1 --2- 172.21.15.86:0/4261111298 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b3bfc4a4e0 0x55b3bfca2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:52.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.826+0000 7f59bba7b740 1 -- 172.21.15.86:0/4261111298 >> 172.21.15.86:0/4261111298 conn(0x55b3bfc46930 msgr2=0x55b3bfc475e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:52.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.826+0000 7f5993400640 0 client.8640 ms_handle_reset on v2:172.21.15.86:6834/1331851138 2024-04-20T21:27:52.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.827+0000 7f59bba7b740 1 -- 172.21.15.86:0/4261111298 shutdown_connections 2024-04-20T21:27:52.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.827+0000 7f59bba7b740 1 -- 172.21.15.86:0/4261111298 wait complete. 2024-04-20T21:27:52.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.827+0000 7f59bba7b740 20 client.8640 trim_cache size 0 max 16384 2024-04-20T21:27:52.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.847+0000 7f59bba7b740 1 Processor -- start 2024-04-20T21:27:52.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.847+0000 7f59bba7b740 1 -- start start 2024-04-20T21:27:52.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.848+0000 7f59bba7b740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b3bfc4a4e0 0x55b3bfc6f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:52.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.848+0000 7f59bba7b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3bfc6c940 0x55b3bfc91570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:52.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.848+0000 7f59bba7b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3bfc94460 0x55b3bfc91ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:52.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.848+0000 7f59bba7b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f59b41fff30 con 0x55b3bfc94460 2024-04-20T21:27:52.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.848+0000 7f59bba7b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f59b4144520 con 0x55b3bfc6c940 2024-04-20T21:27:52.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.848+0000 7f59bba7b740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f59b4144b70 con 0x55b3bfc4a4e0 2024-04-20T21:27:52.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.848+0000 7f59b2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3bfc94460 0x55b3bfc91ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:52.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.848+0000 7f59b2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3bfc94460 0x55b3bfc91ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36698/0 (socket says 172.21.15.86:36698) 2024-04-20T21:27:52.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.848+0000 7f59b2800640 1 -- 172.21.15.86:0/406612436 learned_addr learned my addr 172.21.15.86:0/406612436 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:52.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.849+0000 7f59b2800640 1 -- 172.21.15.86:0/406612436 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b3bfc4a4e0 msgr2=0x55b3bfc6f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:52.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.849+0000 7f59b2800640 1 --2- 172.21.15.86:0/406612436 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b3bfc4a4e0 0x55b3bfc6f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:52.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.849+0000 7f59b2800640 1 -- 172.21.15.86:0/406612436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3bfc6c940 msgr2=0x55b3bfc91570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:52.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.849+0000 7f59b1400640 1 --2- 172.21.15.86:0/406612436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3bfc6c940 0x55b3bfc91570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:52.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.849+0000 7f59b2800640 1 --2- 172.21.15.86:0/406612436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3bfc6c940 0x55b3bfc91570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:52.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.849+0000 7f59b2800640 1 -- 172.21.15.86:0/406612436 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3bfc4d7f0 con 0x55b3bfc94460 2024-04-20T21:27:52.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.849+0000 7f59b2800640 1 --2- 172.21.15.86:0/406612436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3bfc94460 0x55b3bfc91ab0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f59ac02c4f0 tx=0x7f59ac009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68b29a25a88372fa server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:52.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.849+0000 7f599e000640 1 -- 172.21.15.86:0/406612436 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ac02ca80 con 0x55b3bfc94460 2024-04-20T21:27:52.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.849+0000 7f599e000640 1 -- 172.21.15.86:0/406612436 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59ac003990 con 0x55b3bfc94460 2024-04-20T21:27:52.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.849+0000 7f599e000640 1 -- 172.21.15.86:0/406612436 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ac003d80 con 0x55b3bfc94460 2024-04-20T21:27:52.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.850+0000 7f59bba7b740 1 -- 172.21.15.86:0/406612436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3bfc94460 msgr2=0x55b3bfc91ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:52.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.850+0000 7f59bba7b740 1 --2- 172.21.15.86:0/406612436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3bfc94460 0x55b3bfc91ab0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f59ac02c4f0 tx=0x7f59ac009000 comp rx=0 tx=0).stop 2024-04-20T21:27:52.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.850+0000 7f59bba7b740 1 -- 172.21.15.86:0/406612436 shutdown_connections 2024-04-20T21:27:52.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.850+0000 7f59bba7b740 1 --2- 172.21.15.86:0/406612436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3bfc94460 0x55b3bfc91ab0 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:52.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.850+0000 7f59bba7b740 1 --2- 172.21.15.86:0/406612436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3bfc6c940 0x55b3bfc91570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:52.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.850+0000 7f59bba7b740 1 --2- 172.21.15.86:0/406612436 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b3bfc4a4e0 0x55b3bfc6f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:52.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.850+0000 7f59bba7b740 1 -- 172.21.15.86:0/406612436 >> 172.21.15.86:0/406612436 conn(0x55b3bfc46930 msgr2=0x55b3bfcb5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:52.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.850+0000 7f59bba7b740 1 -- 172.21.15.86:0/406612436 shutdown_connections 2024-04-20T21:27:52.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.850+0000 7f59bba7b740 1 -- 172.21.15.86:0/406612436 wait complete. 2024-04-20T21:27:52.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.851+0000 7f59bba7b740 1 -- start start 2024-04-20T21:27:52.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.851+0000 7f59bba7b740 1 Processor -- start 2024-04-20T21:27:52.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.852+0000 7f59bba7b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3bfc4a4e0 0x55b3bfc91700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:52.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.852+0000 7f59bba7b740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b3bfc6c940 0x55b3bfc94a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:52.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.852+0000 7f59bba7b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3bfc94460 0x55b3bfc6d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:52.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.852+0000 7f59bba7b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b3bfc929e0 con 0x55b3bfc94460 2024-04-20T21:27:52.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.852+0000 7f59bba7b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b3bfca2840 con 0x55b3bfc4a4e0 2024-04-20T21:27:52.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.852+0000 7f59bba7b740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55b3bfc92fd0 con 0x55b3bfc6c940 2024-04-20T21:27:52.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.852+0000 7f59bba7b740 10 client.0 _ll_register_callbacks cb 0x7f59a0003a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:27:52.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.852+0000 7f59b2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3bfc94460 0x55b3bfc6d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:52.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.852+0000 7f59b2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3bfc94460 0x55b3bfc6d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36702/0 (socket says 172.21.15.86:36702) 2024-04-20T21:27:52.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.852+0000 7f59b2800640 1 -- 172.21.15.86:0/1958160275 learned_addr learned my addr 172.21.15.86:0/1958160275 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:52.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.853+0000 7f59b1e00640 1 --2- 172.21.15.86:0/1958160275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3bfc4a4e0 0x55b3bfc91700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:52.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.853+0000 7f59b2800640 1 -- 172.21.15.86:0/1958160275 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b3bfc6c940 msgr2=0x55b3bfc94a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:52.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.853+0000 7f59b2800640 1 --2- 172.21.15.86:0/1958160275 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b3bfc6c940 0x55b3bfc94a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:52.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.853+0000 7f59b2800640 1 -- 172.21.15.86:0/1958160275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3bfc4a4e0 msgr2=0x55b3bfc91700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:52.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.853+0000 7f59b2800640 1 --2- 172.21.15.86:0/1958160275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3bfc4a4e0 0x55b3bfc91700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:52.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.853+0000 7f59b2800640 1 -- 172.21.15.86:0/1958160275 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3bfc6d220 con 0x55b3bfc94460 2024-04-20T21:27:52.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.853+0000 7f59b2800640 1 --2- 172.21.15.86:0/1958160275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3bfc94460 0x55b3bfc6d6f0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f59ac02bf20 tx=0x7f59ac004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6ab2c446525a875 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:52.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.853+0000 7f5993400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:27:52.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.853+0000 7f5993400640 1 -- 172.21.15.86:0/1958160275 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ac055060 con 0x55b3bfc94460 2024-04-20T21:27:52.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.853+0000 7f5993400640 1 -- 172.21.15.86:0/1958160275 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7f59ac004240 con 0x55b3bfc94460 2024-04-20T21:27:52.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.853+0000 7f59bba7b740 1 -- 172.21.15.86:0/1958160275 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59ac003990 con 0x55b3bfc94460 2024-04-20T21:27:52.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.854+0000 7f5993400640 1 -- 172.21.15.86:0/1958160275 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f59ac05b050 con 0x55b3bfc94460 2024-04-20T21:27:52.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.854+0000 7f59bba7b740 10 client.8646 fetch_fsmap learned FSMap version 82 2024-04-20T21:27:52.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.854+0000 7f59bba7b740 1 -- 172.21.15.86:0/1958160275 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x55b3bfc6d430 con 0x55b3bfc94460 2024-04-20T21:27:52.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.854+0000 7f5993400640 1 -- 172.21.15.86:0/1958160275 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59ac003990 con 0x55b3bfc94460 2024-04-20T21:27:52.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.854+0000 7f5993400640 1 -- 172.21.15.86:0/1958160275 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 82) ==== 3495+0+0 (secure 0 0 0) 0x7f59ac05e020 con 0x55b3bfc94460 2024-04-20T21:27:52.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.854+0000 7f59bba7b740 10 client.8646 fetch_fsmap finished waiting for FSMap version 82 2024-04-20T21:27:52.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.854+0000 7f59bba7b740 10 client.8646 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-20T21:27:52.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.854+0000 7f59bba7b740 20 client.8646 populate_metadata read hostname 'smithi086' 2024-04-20T21:27:52.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.854+0000 7f59bba7b740 1 --2- 172.21.15.86:0/1958160275 >> [v2:172.21.15.86:6834/1331851138,v1:172.21.15.86:6836/1331851138] conn(0x55b3bfc99010 0x55b3bfc993f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:52.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.854+0000 7f59bba7b740 4 client.8646 mds_command: new command op to 8493 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T21:27:52.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.854+0000 7f59bba7b740 1 -- 172.21.15.86:0/1958160275 --> [v2:172.21.15.86:6834/1331851138,v1:172.21.15.86:6836/1331851138] -- command(tid 0: {"prefix": "status"}) -- 0x55b3bfc99ae0 con 0x55b3bfc99010 2024-04-20T21:27:52.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.854+0000 7f59b1e00640 1 --2- 172.21.15.86:0/1958160275 >> [v2:172.21.15.86:6834/1331851138,v1:172.21.15.86:6836/1331851138] conn(0x55b3bfc99010 0x55b3bfc993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:52.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.855+0000 7f59b1e00640 1 --2- 172.21.15.86:0/1958160275 >> [v2:172.21.15.86:6834/1331851138,v1:172.21.15.86:6836/1331851138] conn(0x55b3bfc99010 0x55b3bfc993f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59f53e49ffc54243 server_cookie=1508bd148d487ca7 in_seq=0 out_seq=0 2024-04-20T21:27:52.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.855+0000 7f5993400640 10 client.8646 ms_handle_connect on v2:172.21.15.86:6834/1331851138 2024-04-20T21:27:52.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.855+0000 7f5993400640 1 -- 172.21.15.86:0/1958160275 <== mds.0 v2:172.21.15.86:6834/1331851138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55b3bfc99ae0 con 0x55b3bfc99010 2024-04-20T21:27:52.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.855+0000 7f5993400640 10 client.8646 handle_command_reply: tid=0 2024-04-20T21:27:52.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.855+0000 7f59bba7b740 1 client.8646 shutdown 2024-04-20T21:27:52.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.855+0000 7f59bba7b740 1 -- 172.21.15.86:0/1958160275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3bfc94460 msgr2=0x55b3bfc6d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:52.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.855+0000 7f59bba7b740 1 --2- 172.21.15.86:0/1958160275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3bfc94460 0x55b3bfc6d6f0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f59ac02bf20 tx=0x7f59ac004ab0 comp rx=0 tx=0).stop 2024-04-20T21:27:52.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.855+0000 7f59bba7b740 1 -- 172.21.15.86:0/1958160275 shutdown_connections 2024-04-20T21:27:52.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.855+0000 7f59bba7b740 1 --2- 172.21.15.86:0/1958160275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b3bfc94460 0x55b3bfc6d6f0 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:52.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.855+0000 7f59bba7b740 1 --2- 172.21.15.86:0/1958160275 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b3bfc6c940 0x55b3bfc94a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:52.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.855+0000 7f59bba7b740 1 --2- 172.21.15.86:0/1958160275 >> [v2:172.21.15.86:6834/1331851138,v1:172.21.15.86:6836/1331851138] conn(0x55b3bfc99010 0x55b3bfc993f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:52.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.855+0000 7f59bba7b740 1 --2- 172.21.15.86:0/1958160275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b3bfc4a4e0 0x55b3bfc91700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:52.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.855+0000 7f59bba7b740 1 -- 172.21.15.86:0/1958160275 >> 172.21.15.86:0/1958160275 conn(0x55b3bfc46930 msgr2=0x55b3bfc4b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:52.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.855+0000 7f5993400640 0 client.8646 ms_handle_reset on v2:172.21.15.86:6834/1331851138 2024-04-20T21:27:52.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.856+0000 7f59bba7b740 1 -- 172.21.15.86:0/1958160275 shutdown_connections 2024-04-20T21:27:52.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.856+0000 7f59bba7b740 1 -- 172.21.15.86:0/1958160275 wait complete. 2024-04-20T21:27:52.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.856+0000 7f59bba7b740 20 client.8646 trim_cache size 0 max 16384 2024-04-20T21:27:52.858 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:27:52.858 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "9c2dec1d-d7ae-4a83-a812-79922808e278", 2024-04-20T21:27:52.858 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-04-20T21:27:52.858 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 8493, 2024-04-20T21:27:52.859 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-20T21:27:52.859 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-20T21:27:52.859 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-20T21:27:52.859 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 6.5848857169999997, 2024-04-20T21:27:52.859 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 82, 2024-04-20T21:27:52.859 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 100, 2024-04-20T21:27:52.859 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 100, 2024-04-20T21:27:52.859 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 12.073270183 2024-04-20T21:27:52.859 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:27:52.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.856+0000 7f59b8e00640 1 -- 172.21.15.86:0/3508440334 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f599406c8f0 msgr2=0x7f599406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:52.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.856+0000 7f59b8e00640 1 --2- 172.21.15.86:0/3508440334 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f599406c8f0 0x7f599406edb0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f59b40cf4c0 tx=0x7f59a0009040 comp rx=0 tx=0).stop 2024-04-20T21:27:52.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.856+0000 7f59b8e00640 1 -- 172.21.15.86:0/3508440334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59b40cd7e0 msgr2=0x7f59b41315a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:52.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.856+0000 7f59b8e00640 1 --2- 172.21.15.86:0/3508440334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59b40cd7e0 0x7f59b41315a0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f59a400d830 tx=0x7f59a400dcf0 comp rx=0 tx=0).stop 2024-04-20T21:27:52.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.857+0000 7f59b8e00640 1 -- 172.21.15.86:0/3508440334 shutdown_connections 2024-04-20T21:27:52.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.857+0000 7f59b8e00640 1 --2- 172.21.15.86:0/3508440334 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f599406c8f0 0x7f599406edb0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:52.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.857+0000 7f59b8e00640 1 --2- 172.21.15.86:0/3508440334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59b4145fe0 0x7f59b40b1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:52.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.857+0000 7f59b8e00640 1 --2- 172.21.15.86:0/3508440334 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59b40d0810 0x7f59b415bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:52.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.857+0000 7f59b8e00640 1 --2- 172.21.15.86:0/3508440334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59b40cd7e0 0x7f59b41315a0 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:52.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.857+0000 7f59b8e00640 1 -- 172.21.15.86:0/3508440334 >> 172.21.15.86:0/3508440334 conn(0x7f59b406b980 msgr2=0x7f59b40cd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:52.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.857+0000 7f59b8e00640 1 -- 172.21.15.86:0/3508440334 shutdown_connections 2024-04-20T21:27:52.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:52.857+0000 7f59b8e00640 1 -- 172.21.15.86:0/3508440334 wait complete. 2024-04-20T21:27:52.873 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T21:27:53.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.069+0000 7efc2ae00640 1 Processor -- start 2024-04-20T21:27:53.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.069+0000 7efc2ae00640 1 -- start start 2024-04-20T21:27:53.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.070+0000 7efc2ae00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc240d11d0 0x7efc240d15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:53.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.070+0000 7efc2ae00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc240d1b10 con 0x7efc240cf410 2024-04-20T21:27:53.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.070+0000 7efc2ae00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc240d01e0 con 0x7efc24162000 2024-04-20T21:27:53.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.070+0000 7efc2ae00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7efc240cffb0 con 0x7efc240d11d0 2024-04-20T21:27:53.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.071+0000 7efc28e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7efc240cf410 0x7efc240cf8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57406/0 (socket says 172.21.15.86:57406) 2024-04-20T21:27:53.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.071+0000 7efc28e00640 1 -- 172.21.15.86:0/4187726768 learned_addr learned my addr 172.21.15.86:0/4187726768 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:27:53.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.071+0000 7efc1b400640 1 --2- 172.21.15.86:0/4187726768 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc240d11d0 0x7efc240d15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:53.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.071+0000 7efc23400640 1 -- 172.21.15.86:0/4187726768 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 31834497 0 0) 0x7efc240d1b10 con 0x7efc240cf410 2024-04-20T21:27:53.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.071+0000 7efc23400640 1 -- 172.21.15.86:0/4187726768 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efc08003520 con 0x7efc240cf410 2024-04-20T21:27:53.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.072+0000 7efc23400640 1 -- 172.21.15.86:0/4187726768 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3351858425 0 0) 0x7efc08003520 con 0x7efc240cf410 2024-04-20T21:27:53.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.072+0000 7efc23400640 1 -- 172.21.15.86:0/4187726768 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc240d11d0 msgr2=0x7efc240d15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:53.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.072+0000 7efc23400640 1 --2- 172.21.15.86:0/4187726768 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc240d11d0 0x7efc240d15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:53.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.072+0000 7efc23400640 1 -- 172.21.15.86:0/4187726768 >> v1:172.21.15.144:6789/0 conn(0x7efc24162000 legacy=0x7efc240b76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:53.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.072+0000 7efc23400640 1 -- 172.21.15.86:0/4187726768 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc240d04a0 con 0x7efc240cf410 2024-04-20T21:27:53.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.072+0000 7efc23400640 1 -- 172.21.15.86:0/4187726768 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7efc1c0019f0 con 0x7efc240cf410 2024-04-20T21:27:53.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.072+0000 7efc23400640 1 -- 172.21.15.86:0/4187726768 >> v1:172.21.15.86:6789/0 conn(0x7efc240cf410 legacy=0x7efc240cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:53.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.072+0000 7efc23400640 1 --2- 172.21.15.86:0/4187726768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc080039e0 0x7efc08003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:53.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.072+0000 7efc23400640 1 --2- 172.21.15.86:0/4187726768 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc240d11d0 0x7efc08004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:53.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.072+0000 7efc23400640 1 --2- 172.21.15.86:0/4187726768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc08004d40 0x7efc080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:53.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.072+0000 7efc23400640 1 -- 172.21.15.86:0/4187726768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efc0800bb50 con 0x7efc08004d40 2024-04-20T21:27:53.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.072+0000 7efc23400640 1 -- 172.21.15.86:0/4187726768 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efc0800bd00 con 0x7efc080039e0 2024-04-20T21:27:53.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.072+0000 7efc23400640 1 -- 172.21.15.86:0/4187726768 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7efc0800bf00 con 0x7efc240d11d0 2024-04-20T21:27:53.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.072+0000 7efc28e00640 1 --2- 172.21.15.86:0/4187726768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc08004d40 0x7efc080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:53.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.072+0000 7efc1b400640 1 --2- 172.21.15.86:0/4187726768 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc240d11d0 0x7efc08004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:53.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.072+0000 7efc23e00640 1 --2- 172.21.15.86:0/4187726768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc080039e0 0x7efc08003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:53.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.073+0000 7efc28e00640 1 -- 172.21.15.86:0/4187726768 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc240d11d0 msgr2=0x7efc08004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:53.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.073+0000 7efc28e00640 1 --2- 172.21.15.86:0/4187726768 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc240d11d0 0x7efc08004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:53.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.073+0000 7efc28e00640 1 -- 172.21.15.86:0/4187726768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc080039e0 msgr2=0x7efc08003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:53.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.073+0000 7efc28e00640 1 --2- 172.21.15.86:0/4187726768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc080039e0 0x7efc08003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:53.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.073+0000 7efc28e00640 1 -- 172.21.15.86:0/4187726768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc08003520 con 0x7efc08004d40 2024-04-20T21:27:53.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.074+0000 7efc28e00640 1 --2- 172.21.15.86:0/4187726768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc08004d40 0x7efc080051c0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7efc1c00a850 tx=0x7efc08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab893c9acfb1514d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:53.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.074+0000 7efc23400640 1 -- 172.21.15.86:0/4187726768 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c00a4a0 con 0x7efc08004d40 2024-04-20T21:27:53.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.074+0000 7efc23400640 1 -- 172.21.15.86:0/4187726768 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc1c034650 con 0x7efc08004d40 2024-04-20T21:27:53.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.074+0000 7efc23400640 1 -- 172.21.15.86:0/4187726768 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c034a40 con 0x7efc08004d40 2024-04-20T21:27:53.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.075+0000 7efc2ae00640 1 -- 172.21.15.86:0/4187726768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc08004d40 msgr2=0x7efc080051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:53.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.075+0000 7efc2ae00640 1 --2- 172.21.15.86:0/4187726768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc08004d40 0x7efc080051c0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7efc1c00a850 tx=0x7efc08005830 comp rx=0 tx=0).stop 2024-04-20T21:27:53.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.075+0000 7efc28e00640 1 -- 172.21.15.86:0/4187726768 reap_dead start 2024-04-20T21:27:53.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.075+0000 7efc2ae00640 1 -- 172.21.15.86:0/4187726768 shutdown_connections 2024-04-20T21:27:53.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.075+0000 7efc2ae00640 1 -- 172.21.15.86:0/4187726768 >> 172.21.15.86:0/4187726768 conn(0x7efc2408bf70 msgr2=0x7efc240d4870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:53.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.075+0000 7efc2ae00640 1 -- 172.21.15.86:0/4187726768 shutdown_connections 2024-04-20T21:27:53.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.075+0000 7efc2ae00640 1 -- 172.21.15.86:0/4187726768 wait complete. 2024-04-20T21:27:53.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.076+0000 7efc2ae00640 1 Processor -- start 2024-04-20T21:27:53.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.077+0000 7efc2ae00640 1 -- start start 2024-04-20T21:27:53.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.077+0000 7efc2ae00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc08004d40 0x7efc2415a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:53.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.077+0000 7efc2ae00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc240c9030 0x7efc240b9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:53.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.078+0000 7efc2ae00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc240b9ab0 0x7efc24160e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:53.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.078+0000 7efc2ae00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efc241623e0 con 0x7efc240b9ab0 2024-04-20T21:27:53.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.078+0000 7efc2ae00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7efc240d2440 con 0x7efc08004d40 2024-04-20T21:27:53.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.078+0000 7efc2ae00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7efc24162560 con 0x7efc240c9030 2024-04-20T21:27:53.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.078+0000 7efc28e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc240b9ab0 0x7efc24160e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:53.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.078+0000 7efc28e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc240b9ab0 0x7efc24160e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36706/0 (socket says 172.21.15.86:36706) 2024-04-20T21:27:53.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.078+0000 7efc28e00640 1 -- 172.21.15.86:0/3671595767 learned_addr learned my addr 172.21.15.86:0/3671595767 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:53.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.078+0000 7efc23e00640 1 --2- 172.21.15.86:0/3671595767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc08004d40 0x7efc2415a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:53.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.078+0000 7efc1b400640 1 --2- 172.21.15.86:0/3671595767 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc240c9030 0x7efc240b9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:53.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.078+0000 7efc28e00640 1 -- 172.21.15.86:0/3671595767 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc240c9030 msgr2=0x7efc240b9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:53.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.078+0000 7efc28e00640 1 --2- 172.21.15.86:0/3671595767 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc240c9030 0x7efc240b9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:53.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.078+0000 7efc28e00640 1 -- 172.21.15.86:0/3671595767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc08004d40 msgr2=0x7efc2415a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:53.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.078+0000 7efc28e00640 1 --2- 172.21.15.86:0/3671595767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc08004d40 0x7efc2415a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:53.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.078+0000 7efc28e00640 1 -- 172.21.15.86:0/3671595767 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc2415ed40 con 0x7efc240b9ab0 2024-04-20T21:27:53.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.079+0000 7efc28e00640 1 --2- 172.21.15.86:0/3671595767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc240b9ab0 0x7efc24160e40 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7efc1c00a850 tx=0x7efc1c001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:53.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.079+0000 7efc20c00640 1 -- 172.21.15.86:0/3671595767 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c0017b0 con 0x7efc240b9ab0 2024-04-20T21:27:53.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.079+0000 7efc20c00640 1 -- 172.21.15.86:0/3671595767 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc1c044070 con 0x7efc240b9ab0 2024-04-20T21:27:53.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.079+0000 7efc20c00640 1 -- 172.21.15.86:0/3671595767 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c034800 con 0x7efc240b9ab0 2024-04-20T21:27:53.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.079+0000 7efc2ae00640 1 -- 172.21.15.86:0/3671595767 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc2415f020 con 0x7efc240b9ab0 2024-04-20T21:27:53.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.079+0000 7efc2ae00640 1 -- 172.21.15.86:0/3671595767 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc2415f5e0 con 0x7efc240b9ab0 2024-04-20T21:27:53.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.080+0000 7efc2ae00640 1 -- 172.21.15.86:0/3671595767 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc240ce3e0 con 0x7efc240b9ab0 2024-04-20T21:27:53.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.081+0000 7efc20c00640 1 -- 172.21.15.86:0/3671595767 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efc1c0349a0 con 0x7efc240b9ab0 2024-04-20T21:27:53.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.082+0000 7efc20c00640 1 --2- 172.21.15.86:0/3671595767 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efc0006c8f0 0x7efc0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:53.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.082+0000 7efc20c00640 1 -- 172.21.15.86:0/3671595767 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7efc1c0ade30 con 0x7efc240b9ab0 2024-04-20T21:27:53.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.082+0000 7efc23e00640 1 --2- 172.21.15.86:0/3671595767 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efc0006c8f0 0x7efc0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:53.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.083+0000 7efc23e00640 1 --2- 172.21.15.86:0/3671595767 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efc0006c8f0 0x7efc0006edb0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7efc14007fd0 tx=0x7efc14009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:53.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.087+0000 7efc20c00640 1 -- 172.21.15.86:0/3671595767 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7efc1c07c250 con 0x7efc240b9ab0 2024-04-20T21:27:53.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.331+0000 7efc2ae00640 1 -- 172.21.15.86:0/3671595767 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7efc240689e0 con 0x7efc240b9ab0 2024-04-20T21:27:53.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.332+0000 7efc20c00640 1 -- 172.21.15.86:0/3671595767 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v82) ==== 53+0+73 (secure 0 0 0) 0x7efc1c07bb50 con 0x7efc240b9ab0 2024-04-20T21:27:53.331 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T21:27:53.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.334+0000 7efc2ae00640 1 -- 172.21.15.86:0/3671595767 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efc0006c8f0 msgr2=0x7efc0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:53.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.334+0000 7efc2ae00640 1 --2- 172.21.15.86:0/3671595767 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efc0006c8f0 0x7efc0006edb0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7efc14007fd0 tx=0x7efc14009040 comp rx=0 tx=0).stop 2024-04-20T21:27:53.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.334+0000 7efc2ae00640 1 -- 172.21.15.86:0/3671595767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc240b9ab0 msgr2=0x7efc24160e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:53.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.334+0000 7efc2ae00640 1 --2- 172.21.15.86:0/3671595767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc240b9ab0 0x7efc24160e40 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7efc1c00a850 tx=0x7efc1c001dc0 comp rx=0 tx=0).stop 2024-04-20T21:27:53.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.335+0000 7efc2ae00640 1 -- 172.21.15.86:0/3671595767 shutdown_connections 2024-04-20T21:27:53.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.335+0000 7efc2ae00640 1 --2- 172.21.15.86:0/3671595767 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7efc0006c8f0 0x7efc0006edb0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:53.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.335+0000 7efc2ae00640 1 --2- 172.21.15.86:0/3671595767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc240b9ab0 0x7efc24160e40 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:53.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.335+0000 7efc2ae00640 1 --2- 172.21.15.86:0/3671595767 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7efc240c9030 0x7efc240b9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:53.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.335+0000 7efc2ae00640 1 --2- 172.21.15.86:0/3671595767 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efc08004d40 0x7efc2415a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:53.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.335+0000 7efc2ae00640 1 -- 172.21.15.86:0/3671595767 >> 172.21.15.86:0/3671595767 conn(0x7efc2408bf70 msgr2=0x7efc2415fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:53.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.335+0000 7efc2ae00640 1 -- 172.21.15.86:0/3671595767 shutdown_connections 2024-04-20T21:27:53.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53.335+0000 7efc2ae00640 1 -- 172.21.15.86:0/3671595767 wait complete. 2024-04-20T21:27:53.353 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T21:27:53.353 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T21:27:53.353 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-04-20T21:27:53.353 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-20T21:27:53.353 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T21:27:53.353 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T21:27:53.353 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T21:27:53.353 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T21:27:53.353 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:27:53.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:27:53.354 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-04-20T21:27:53.403 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T21:27:53.403 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T21:27:53.403 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T21:27:53.404 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:27:53.404 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-20T21:27:53.404 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:27:53.404 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:27:53.404 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:27:53.404 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-04-20T21:27:53.404 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:27:53.404 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:27:53.404 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-04-20T21:27:53.404 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:27:53.404 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:27:53.404 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T21:27:53.404 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:27:53.404 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T21:27:53.404 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42082sec preferred_lft 42082sec 2024-04-20T21:27:53.405 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-04-20T21:27:53.405 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:27:53.405 INFO:teuthology.orchestra.run.smithi086.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T21:27:53.405 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 86:25:97:2c:9c:bb brd ff:ff:ff:ff:ff:ff 2024-04-20T21:27:53.405 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-20T21:27:53.405 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:27:53.405 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::a4fb:30ff:fe90:2d6a/64 scope link 2024-04-20T21:27:53.405 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:27:53.405 INFO:teuthology.orchestra.run.smithi086.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T21:27:53.405 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 86:25:97:2c:9c:bb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:27:53.405 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::8425:97ff:fe2c:9cbb/64 scope link 2024-04-20T21:27:53.405 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:27:53.405 INFO:teuthology.orchestra.run.smithi086.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T21:27:53.405 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c2:9c:8b:55:ca:84 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:27:53.405 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c09c:8bff:fe55:ca84/64 scope link 2024-04-20T21:27:53.405 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:27:53.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:27:53.407 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T21:27:53.475 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-20T21:27:53.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:27:53.479 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-04-20T21:27:53.532 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-04-20T21:27:53.532 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T21:27:53.532 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T21:27:53.532 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T21:27:53.532 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T21:27:53.532 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:27:53.532 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-20T21:27:53.532 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T21:27:53.532 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T21:27:53.533 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T21:27:53.533 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-20T21:27:53.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:27:53.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:27:53.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:27:53.739 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-20T21:27:53.786 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-20T21:27:53.787 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-20T21:27:53.788 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:27:53.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:27:53.788 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-04-20T21:27:53.859 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-20T21:27:53.859 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T21:27:53.860 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-20T21:27:53.911 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:27:53.911 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T21:27:53.911 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-20T21:27:53.958 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-04-20T21:27:54.044 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T21:27:54.098 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-04-20T21:27:54.098 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-04-20T21:27:54.098 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-04-20T21:27:54.099 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-04-20T21:27:54.099 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-04-20T21:27:54.099 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-04-20T21:27:54.099 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-04-20T21:27:54.099 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-04-20T21:27:54.099 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-04-20T21:27:54.099 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-04-20T21:27:54.099 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-04-20T21:27:54.099 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/selinux 2024-04-20T21:27:54.099 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T21:27:54.099 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-04-20T21:27:54.099 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-04-20T21:27:54.099 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-04-20T21:27:54.099 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-04-20T21:27:54.100 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-04-20T21:27:54.100 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-04-20T21:27:54.100 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T21:27:54.100 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T21:27:54.100 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T21:27:54.100 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T21:27:54.100 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-04-20T21:27:54.100 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-04-20T21:27:54.100 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T21:27:54.100 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T21:27:54.100 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T21:27:54.100 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T21:27:54.100 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-04-20T21:27:54.100 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:27:54.100 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:27:54.100 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:27:54.101 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:27:54.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:27:54.102 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-04-20T21:27:54.151 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T21:27:54.151 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-20T21:27:54.156 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-04-20T21:27:54.223 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T21:27:54.263 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-04-20T21:27:54.263 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-04-20T21:27:54.263 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-04-20T21:27:54.263 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-04-20T21:27:54.263 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-04-20T21:27:54.263 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-04-20T21:27:54.263 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-04-20T21:27:54.263 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-04-20T21:27:54.263 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-04-20T21:27:54.263 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-04-20T21:27:54.263 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-04-20T21:27:54.263 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/selinux 2024-04-20T21:27:54.263 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T21:27:54.264 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-04-20T21:27:54.264 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-04-20T21:27:54.264 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-04-20T21:27:54.264 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-04-20T21:27:54.264 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-04-20T21:27:54.264 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-04-20T21:27:54.264 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T21:27:54.264 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T21:27:54.264 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T21:27:54.264 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T21:27:54.264 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-04-20T21:27:54.264 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-04-20T21:27:54.264 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T21:27:54.264 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T21:27:54.264 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T21:27:54.265 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T21:27:54.265 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-04-20T21:27:54.265 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:27:54.265 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:27:54.265 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:27:54.265 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:27:54.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:27:54.266 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-04-20T21:27:54.323 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:2024-04-20T21:27:54.324+0000 7f17104f8480 -1 init, newargv = 0x5579dfe144b0 newargc=15 2024-04-20T21:27:54.323 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[59507]: starting ceph client 2024-04-20T21:27:54.332 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[59507]: starting fuse 2024-04-20T21:27:54.349 INFO:teuthology.orchestra.run.smithi086.stdout:41 2024-04-20T21:27:54.350 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-04-20T21:27:54.350 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer -- python3 -c ' 2024-04-20T21:27:54.350 DEBUG:teuthology.orchestra.run.smithi086:> import glob 2024-04-20T21:27:54.350 DEBUG:teuthology.orchestra.run.smithi086:> import re 2024-04-20T21:27:54.350 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:27:54.350 DEBUG:teuthology.orchestra.run.smithi086:> import subprocess 2024-04-20T21:27:54.350 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:27:54.350 DEBUG:teuthology.orchestra.run.smithi086:> def _find_admin_socket(client_name): 2024-04-20T21:27:54.350 DEBUG:teuthology.orchestra.run.smithi086:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-20T21:27:54.350 DEBUG:teuthology.orchestra.run.smithi086:> files = glob.glob(asok_path) 2024-04-20T21:27:54.350 DEBUG:teuthology.orchestra.run.smithi086:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-20T21:27:54.351 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:27:54.351 DEBUG:teuthology.orchestra.run.smithi086:> # Given a non-glob path, it better be there 2024-04-20T21:27:54.351 DEBUG:teuthology.orchestra.run.smithi086:> if "*" not in asok_path: 2024-04-20T21:27:54.351 DEBUG:teuthology.orchestra.run.smithi086:> assert(len(files) == 1) 2024-04-20T21:27:54.351 DEBUG:teuthology.orchestra.run.smithi086:> return files[0] 2024-04-20T21:27:54.351 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:27:54.351 DEBUG:teuthology.orchestra.run.smithi086:> for f in files: 2024-04-20T21:27:54.351 DEBUG:teuthology.orchestra.run.smithi086:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T21:27:54.351 DEBUG:teuthology.orchestra.run.smithi086:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T21:27:54.351 DEBUG:teuthology.orchestra.run.smithi086:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T21:27:54.351 DEBUG:teuthology.orchestra.run.smithi086:> contents = proc_f.read() 2024-04-20T21:27:54.351 DEBUG:teuthology.orchestra.run.smithi086:> if mountpoint in contents: 2024-04-20T21:27:54.351 DEBUG:teuthology.orchestra.run.smithi086:> return f 2024-04-20T21:27:54.351 DEBUG:teuthology.orchestra.run.smithi086:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T21:27:54.352 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:27:54.352 DEBUG:teuthology.orchestra.run.smithi086:> print(_find_admin_socket("client.0")) 2024-04-20T21:27:54.352 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:27:54.538 INFO:teuthology.orchestra.run.smithi086.stdout:/var/run/ceph/ceph-client.0.59507.asok 2024-04-20T21:27:54.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:27:54.555 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.59507.asok 2024-04-20T21:27:54.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:27:54.555 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.59507.asok status 2024-04-20T21:27:54.782 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:27:54.782 INFO:teuthology.orchestra.run.smithi086.stdout: "metadata": { 2024-04-20T21:27:54.782 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T21:27:54.782 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T21:27:54.783 INFO:teuthology.orchestra.run.smithi086.stdout: "entity_id": "0", 2024-04-20T21:27:54.783 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": "smithi086", 2024-04-20T21:27:54.783 INFO:teuthology.orchestra.run.smithi086.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-20T21:27:54.783 INFO:teuthology.orchestra.run.smithi086.stdout: "pid": "59507", 2024-04-20T21:27:54.783 INFO:teuthology.orchestra.run.smithi086.stdout: "root": "/" 2024-04-20T21:27:54.783 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:27:54.783 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_count": 0, 2024-04-20T21:27:54.783 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_pinned_count": 0, 2024-04-20T21:27:54.783 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 8658, 2024-04-20T21:27:54.783 INFO:teuthology.orchestra.run.smithi086.stdout: "inst": { 2024-04-20T21:27:54.783 INFO:teuthology.orchestra.run.smithi086.stdout: "name": { 2024-04-20T21:27:54.784 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "client", 2024-04-20T21:27:54.784 INFO:teuthology.orchestra.run.smithi086.stdout: "num": 8658 2024-04-20T21:27:54.784 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:27:54.784 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2024-04-20T21:27:54.784 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-04-20T21:27:54.784 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2024-04-20T21:27:54.784 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 904527058 2024-04-20T21:27:54.784 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:27:54.784 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:27:54.784 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2024-04-20T21:27:54.784 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-04-20T21:27:54.784 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2024-04-20T21:27:54.785 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 904527058 2024-04-20T21:27:54.785 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:27:54.785 INFO:teuthology.orchestra.run.smithi086.stdout: "inst_str": "client.8658 192.168.0.1:0/904527058", 2024-04-20T21:27:54.785 INFO:teuthology.orchestra.run.smithi086.stdout: "addr_str": "192.168.0.1:0/904527058", 2024-04-20T21:27:54.785 INFO:teuthology.orchestra.run.smithi086.stdout: "inode_count": 1, 2024-04-20T21:27:54.785 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_epoch": 82, 2024-04-20T21:27:54.785 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch": 100, 2024-04-20T21:27:54.785 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch_barrier": 0, 2024-04-20T21:27:54.785 INFO:teuthology.orchestra.run.smithi086.stdout: "blocklisted": false, 2024-04-20T21:27:54.785 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs" 2024-04-20T21:27:54.785 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:27:54.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:27:54.798 DEBUG:teuthology.orchestra.run.smithi086:> stat --file-system '--printf=%T 2024-04-20T21:27:54.798 DEBUG:teuthology.orchestra.run.smithi086:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-20T21:27:54.848 INFO:teuthology.orchestra.run.smithi086.stdout:fuseblk 2024-04-20T21:27:54.848 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-20T21:27:54.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:27:54.849 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-20T21:27:54.929 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:27:54.929 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:27:54.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:27:54.981 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file1_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-20T21:27:55.209 INFO:teuthology.orchestra.run.smithi086.stderr:6+0 records in 2024-04-20T21:27:55.209 INFO:teuthology.orchestra.run.smithi086.stderr:6+0 records out 2024-04-20T21:27:55.210 INFO:teuthology.orchestra.run.smithi086.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.166774 s, 37.7 MB/s 2024-04-20T21:27:55.212 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:27:55.212 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:27:55.270 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:27:55.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.468+0000 7f0b14200640 1 Processor -- start 2024-04-20T21:27:55.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.468+0000 7f0b14200640 1 -- start start 2024-04-20T21:27:55.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.470+0000 7f0b14200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0b0c144e90 0x7f0b0c0b4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:55.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.470+0000 7f0b14200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b0c0cdb50 con 0x7f0b0c06f450 2024-04-20T21:27:55.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.470+0000 7f0b14200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b0c0b3e50 con 0x7f0b0c0cd170 2024-04-20T21:27:55.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.470+0000 7f0b14200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f0b0c145270 con 0x7f0b0c144e90 2024-04-20T21:27:55.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.470+0000 7f0b12200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f0b0c06f450 0x7f0b0c15bb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57422/0 (socket says 172.21.15.86:57422) 2024-04-20T21:27:55.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.470+0000 7f0b12200640 1 -- 172.21.15.86:0/1641386296 learned_addr learned my addr 172.21.15.86:0/1641386296 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:27:55.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.471+0000 7f0b11800640 1 --2- 172.21.15.86:0/1641386296 >> v2:172.21.15.144:3300/0 conn(0x7f0b0c144e90 0x7f0b0c0b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:55.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.471+0000 7f0b0be00640 1 -- 172.21.15.86:0/1641386296 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3491486749 0 0) 0x7f0b0c0cdb50 con 0x7f0b0c06f450 2024-04-20T21:27:55.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.471+0000 7f0b0be00640 1 -- 172.21.15.86:0/1641386296 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0af4003520 con 0x7f0b0c06f450 2024-04-20T21:27:55.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.471+0000 7f0b0be00640 1 -- 172.21.15.86:0/1641386296 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3261470034 0 0) 0x7f0af4003520 con 0x7f0b0c06f450 2024-04-20T21:27:55.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.471+0000 7f0b0be00640 1 -- 172.21.15.86:0/1641386296 >> v2:172.21.15.144:3300/0 conn(0x7f0b0c144e90 msgr2=0x7f0b0c0b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:55.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.471+0000 7f0b0be00640 1 --2- 172.21.15.86:0/1641386296 >> v2:172.21.15.144:3300/0 conn(0x7f0b0c144e90 0x7f0b0c0b4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:55.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.471+0000 7f0b0be00640 1 -- 172.21.15.86:0/1641386296 >> v1:172.21.15.144:6789/0 conn(0x7f0b0c0cd170 legacy=0x7f0b0c06ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:55.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.471+0000 7f0b0be00640 1 -- 172.21.15.86:0/1641386296 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b0c0b4130 con 0x7f0b0c06f450 2024-04-20T21:27:55.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.471+0000 7f0b0be00640 1 -- 172.21.15.86:0/1641386296 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f0b040019f0 con 0x7f0b0c06f450 2024-04-20T21:27:55.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.471+0000 7f0b0be00640 1 -- 172.21.15.86:0/1641386296 >> v1:172.21.15.86:6789/0 conn(0x7f0b0c06f450 legacy=0x7f0b0c15bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:55.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.471+0000 7f0b0be00640 1 --2- 172.21.15.86:0/1641386296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0af40039b0 0x7f0af4003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:55.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.471+0000 7f0b0be00640 1 --2- 172.21.15.86:0/1641386296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0af4004800 0x7f0af4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:55.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.471+0000 7f0b0be00640 1 --2- 172.21.15.86:0/1641386296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0af40051c0 0x7f0af400dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:55.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.471+0000 7f0b0be00640 1 -- 172.21.15.86:0/1641386296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c145270 con 0x7f0af40039b0 2024-04-20T21:27:55.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.471+0000 7f0b0be00640 1 -- 172.21.15.86:0/1641386296 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0af400e2a0 con 0x7f0af40051c0 2024-04-20T21:27:55.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.471+0000 7f0b0be00640 1 -- 172.21.15.86:0/1641386296 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0af400e4a0 con 0x7f0af4004800 2024-04-20T21:27:55.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.472+0000 7f0b11800640 1 --2- 172.21.15.86:0/1641386296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0af40039b0 0x7f0af4003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:55.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.472+0000 7f0b11800640 1 -- 172.21.15.86:0/1641386296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0af4004800 msgr2=0x7f0af4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:55.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.472+0000 7f0b11800640 1 --2- 172.21.15.86:0/1641386296 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0af4004800 0x7f0af4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:55.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.472+0000 7f0b11800640 1 -- 172.21.15.86:0/1641386296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0af40051c0 msgr2=0x7f0af400dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:55.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.472+0000 7f0b11800640 1 --2- 172.21.15.86:0/1641386296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0af40051c0 0x7f0af400dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:55.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.472+0000 7f0b11800640 1 -- 172.21.15.86:0/1641386296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0af4003520 con 0x7f0af40039b0 2024-04-20T21:27:55.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.472+0000 7f0b12200640 1 -- 172.21.15.86:0/1641386296 reap_dead start 2024-04-20T21:27:55.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.473+0000 7f0b11800640 1 --2- 172.21.15.86:0/1641386296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0af40039b0 0x7f0af4003df0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f0b00009ac0 tx=0x7f0af4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e130177433eadca7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:55.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.473+0000 7f0b0be00640 1 -- 172.21.15.86:0/1641386296 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b00039070 con 0x7f0af40039b0 2024-04-20T21:27:55.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.473+0000 7f0b0be00640 1 -- 172.21.15.86:0/1641386296 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b0002c710 con 0x7f0af40039b0 2024-04-20T21:27:55.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.473+0000 7f0b0be00640 1 -- 172.21.15.86:0/1641386296 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b00009690 con 0x7f0af40039b0 2024-04-20T21:27:55.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.474+0000 7f0b14200640 1 -- 172.21.15.86:0/1641386296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0af40039b0 msgr2=0x7f0af4003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:55.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.474+0000 7f0b14200640 1 --2- 172.21.15.86:0/1641386296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0af40039b0 0x7f0af4003df0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f0b00009ac0 tx=0x7f0af4005830 comp rx=0 tx=0).stop 2024-04-20T21:27:55.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.474+0000 7f0b14200640 1 -- 172.21.15.86:0/1641386296 shutdown_connections 2024-04-20T21:27:55.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.474+0000 7f0b14200640 1 --2- 172.21.15.86:0/1641386296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0af40039b0 0x7f0af4003df0 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:55.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.474+0000 7f0b14200640 1 -- 172.21.15.86:0/1641386296 >> 172.21.15.86:0/1641386296 conn(0x7f0b0c0b2360 msgr2=0x7f0b0c0b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:55.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.474+0000 7f0b14200640 1 -- 172.21.15.86:0/1641386296 shutdown_connections 2024-04-20T21:27:55.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.474+0000 7f0b14200640 1 -- 172.21.15.86:0/1641386296 wait complete. 2024-04-20T21:27:55.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.476+0000 7f0b14200640 1 Processor -- start 2024-04-20T21:27:55.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.476+0000 7f0b14200640 1 -- start start 2024-04-20T21:27:55.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.477+0000 7f0b14200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0af40039b0 0x7f0b0c14dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:55.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.477+0000 7f0b14200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b0c1e3ae0 0x7f0b0c1e3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:55.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.477+0000 7f0b14200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b0c142400 0x7f0b0c142880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:55.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.477+0000 7f0b14200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c0d0190 con 0x7f0b0c1e3ae0 2024-04-20T21:27:55.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.477+0000 7f0b14200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c0d0010 con 0x7f0af40039b0 2024-04-20T21:27:55.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.477+0000 7f0b14200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0b0c0cd600 con 0x7f0b0c142400 2024-04-20T21:27:55.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.477+0000 7f0b10e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b0c1e3ae0 0x7f0b0c1e3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:55.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.477+0000 7f0b10e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b0c1e3ae0 0x7f0b0c1e3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36716/0 (socket says 172.21.15.86:36716) 2024-04-20T21:27:55.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.477+0000 7f0b10e00640 1 -- 172.21.15.86:0/2401833099 learned_addr learned my addr 172.21.15.86:0/2401833099 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:55.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.478+0000 7f0b12200640 1 --2- 172.21.15.86:0/2401833099 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b0c142400 0x7f0b0c142880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:55.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.478+0000 7f0b11800640 1 --2- 172.21.15.86:0/2401833099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0af40039b0 0x7f0b0c14dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:55.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.478+0000 7f0b10e00640 1 -- 172.21.15.86:0/2401833099 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b0c142400 msgr2=0x7f0b0c142880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:55.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.478+0000 7f0b10e00640 1 --2- 172.21.15.86:0/2401833099 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b0c142400 0x7f0b0c142880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:55.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.478+0000 7f0b10e00640 1 -- 172.21.15.86:0/2401833099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0af40039b0 msgr2=0x7f0b0c14dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:55.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.478+0000 7f0b10e00640 1 --2- 172.21.15.86:0/2401833099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0af40039b0 0x7f0b0c14dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:55.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.478+0000 7f0b10e00640 1 -- 172.21.15.86:0/2401833099 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b0c1430f0 con 0x7f0b0c1e3ae0 2024-04-20T21:27:55.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.478+0000 7f0b10e00640 1 --2- 172.21.15.86:0/2401833099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b0c1e3ae0 0x7f0b0c1e3f60 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f0af800c530 tx=0x7f0af800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:55.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.478+0000 7f0b09600640 1 -- 172.21.15.86:0/2401833099 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af80052d0 con 0x7f0b0c1e3ae0 2024-04-20T21:27:55.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.478+0000 7f0b09600640 1 -- 172.21.15.86:0/2401833099 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0af8005470 con 0x7f0b0c1e3ae0 2024-04-20T21:27:55.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.479+0000 7f0b14200640 1 -- 172.21.15.86:0/2401833099 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b0c1e1320 con 0x7f0b0c1e3ae0 2024-04-20T21:27:55.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.479+0000 7f0b09600640 1 -- 172.21.15.86:0/2401833099 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af8012850 con 0x7f0b0c1e3ae0 2024-04-20T21:27:55.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.479+0000 7f0b14200640 1 -- 172.21.15.86:0/2401833099 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b0c1e18e0 con 0x7f0b0c1e3ae0 2024-04-20T21:27:55.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.480+0000 7f0b14200640 1 -- 172.21.15.86:0/2401833099 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b0c0689e0 con 0x7f0b0c1e3ae0 2024-04-20T21:27:55.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.481+0000 7f0b09600640 1 -- 172.21.15.86:0/2401833099 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0af80129f0 con 0x7f0b0c1e3ae0 2024-04-20T21:27:55.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.482+0000 7f0b09600640 1 --2- 172.21.15.86:0/2401833099 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0aec06c8f0 0x7f0aec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:55.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.482+0000 7f0b09600640 1 -- 172.21.15.86:0/2401833099 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7f0af808b7b0 con 0x7f0b0c1e3ae0 2024-04-20T21:27:55.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.482+0000 7f0b11800640 1 --2- 172.21.15.86:0/2401833099 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0aec06c8f0 0x7f0aec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:55.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.483+0000 7f0b11800640 1 --2- 172.21.15.86:0/2401833099 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0aec06c8f0 0x7f0aec06edb0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0b00009ac0 tx=0x7f0b00030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:55.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.487+0000 7f0b09600640 1 -- 172.21.15.86:0/2401833099 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0af8059c70 con 0x7f0b0c1e3ae0 2024-04-20T21:27:55.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.692+0000 7f0b14200640 1 -- 172.21.15.86:0/2401833099 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0b0c0b1c30 con 0x7f0b0c1e3ae0 2024-04-20T21:27:55.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.696+0000 7f0b09600640 1 -- 172.21.15.86:0/2401833099 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+16610 (secure 0 0 0) 0x7f0af8059570 con 0x7f0b0c1e3ae0 2024-04-20T21:27:55.695 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:27:55.695 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":100,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:27:47.267402+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:27:43.469685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-20T21:27:44.673505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:27:55.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.699+0000 7f0b14200640 1 -- 172.21.15.86:0/2401833099 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0aec06c8f0 msgr2=0x7f0aec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:55.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.699+0000 7f0b14200640 1 --2- 172.21.15.86:0/2401833099 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0aec06c8f0 0x7f0aec06edb0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0b00009ac0 tx=0x7f0b00030000 comp rx=0 tx=0).stop 2024-04-20T21:27:55.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.699+0000 7f0b14200640 1 -- 172.21.15.86:0/2401833099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b0c1e3ae0 msgr2=0x7f0b0c1e3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:55.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.699+0000 7f0b14200640 1 --2- 172.21.15.86:0/2401833099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b0c1e3ae0 0x7f0b0c1e3f60 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f0af800c530 tx=0x7f0af800c9f0 comp rx=0 tx=0).stop 2024-04-20T21:27:55.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.699+0000 7f0b14200640 1 -- 172.21.15.86:0/2401833099 shutdown_connections 2024-04-20T21:27:55.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.699+0000 7f0b14200640 1 --2- 172.21.15.86:0/2401833099 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0aec06c8f0 0x7f0aec06edb0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:55.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.699+0000 7f0b14200640 1 --2- 172.21.15.86:0/2401833099 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b0c142400 0x7f0b0c142880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:55.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.699+0000 7f0b14200640 1 --2- 172.21.15.86:0/2401833099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b0c1e3ae0 0x7f0b0c1e3f60 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:55.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.699+0000 7f0b14200640 1 --2- 172.21.15.86:0/2401833099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0af40039b0 0x7f0b0c14dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:55.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.699+0000 7f0b14200640 1 -- 172.21.15.86:0/2401833099 >> 172.21.15.86:0/2401833099 conn(0x7f0b0c0b2360 msgr2=0x7f0b0c071d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:55.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.699+0000 7f0b14200640 1 -- 172.21.15.86:0/2401833099 shutdown_connections 2024-04-20T21:27:55.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.699+0000 7f0b14200640 1 -- 172.21.15.86:0/2401833099 wait complete. 2024-04-20T21:27:55.716 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:27:43.469685+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:27:44.673505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:27:55.717 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:27:55.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.919+0000 7f6a1de00640 1 Processor -- start 2024-04-20T21:27:55.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.920+0000 7f6a1de00640 1 -- start start 2024-04-20T21:27:55.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.921+0000 7f6a1de00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6a1815e870 0x7f6a1806ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:55.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.921+0000 7f6a1de00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6a180d4310 0x7f6a180d4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:55.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.921+0000 7f6a1de00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a180d4cb0 con 0x7f6a180cfb80 2024-04-20T21:27:55.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.921+0000 7f6a1de00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f6a180cf150 con 0x7f6a1815e870 2024-04-20T21:27:55.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.921+0000 7f6a1de00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f6a180d35b0 con 0x7f6a180d4310 2024-04-20T21:27:55.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.921+0000 7f6a1ca00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6a1815e870 0x7f6a1806ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:55.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.922+0000 7f6a1ca00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6a1815e870 0x7f6a1806ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52870/0 (socket says 172.21.15.86:52870) 2024-04-20T21:27:55.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.922+0000 7f6a1ca00640 1 -- 172.21.15.86:0/902820547 learned_addr learned my addr 172.21.15.86:0/902820547 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:55.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.922+0000 7f6a17e00640 1 --2- 172.21.15.86:0/902820547 >> v2:172.21.15.144:3300/0 conn(0x7f6a180d4310 0x7f6a180d4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:55.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.922+0000 7f6a1ca00640 1 -- 172.21.15.86:0/902820547 >> v2:172.21.15.144:3300/0 conn(0x7f6a180d4310 msgr2=0x7f6a180d4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:55.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.922+0000 7f6a1ca00640 1 --2- 172.21.15.86:0/902820547 >> v2:172.21.15.144:3300/0 conn(0x7f6a180d4310 0x7f6a180d4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:55.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.922+0000 7f6a1ca00640 1 -- 172.21.15.86:0/902820547 >> v1:172.21.15.144:6789/0 conn(0x7f6a180cfb80 legacy=0x7f6a180d0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:55.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.922+0000 7f6a1ca00640 1 -- 172.21.15.86:0/902820547 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a180d2ff0 con 0x7f6a1815e870 2024-04-20T21:27:55.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.922+0000 7f6a17e00640 1 --2- 172.21.15.86:0/902820547 >> v2:172.21.15.144:3300/0 conn(0x7f6a180d4310 0x7f6a180d4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:27:55.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.923+0000 7f6a1ca00640 1 --2- 172.21.15.86:0/902820547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a1815e870 0x7f6a1806ce80 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f6a180d21a0 tx=0x7f6a0802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35f5a92cce95a8a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:55.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.923+0000 7f6a17400640 1 -- 172.21.15.86:0/902820547 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a08035020 con 0x7f6a1815e870 2024-04-20T21:27:55.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.923+0000 7f6a17400640 1 -- 172.21.15.86:0/902820547 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a08031870 con 0x7f6a1815e870 2024-04-20T21:27:55.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.923+0000 7f6a17400640 1 -- 172.21.15.86:0/902820547 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a08031c40 con 0x7f6a1815e870 2024-04-20T21:27:55.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.924+0000 7f6a1de00640 1 -- 172.21.15.86:0/902820547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a1815e870 msgr2=0x7f6a1806ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:55.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.924+0000 7f6a1de00640 1 --2- 172.21.15.86:0/902820547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a1815e870 0x7f6a1806ce80 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f6a180d21a0 tx=0x7f6a0802b910 comp rx=0 tx=0).stop 2024-04-20T21:27:55.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.924+0000 7f6a1de00640 1 -- 172.21.15.86:0/902820547 shutdown_connections 2024-04-20T21:27:55.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.924+0000 7f6a1de00640 1 --2- 172.21.15.86:0/902820547 >> v2:172.21.15.144:3300/0 conn(0x7f6a180d4310 0x7f6a180d4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:55.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.924+0000 7f6a1de00640 1 --2- 172.21.15.86:0/902820547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a1815e870 0x7f6a1806ce80 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:55.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.924+0000 7f6a1de00640 1 -- 172.21.15.86:0/902820547 >> 172.21.15.86:0/902820547 conn(0x7f6a1809d9c0 msgr2=0x7f6a1808bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:55.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.924+0000 7f6a1de00640 1 -- 172.21.15.86:0/902820547 shutdown_connections 2024-04-20T21:27:55.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.924+0000 7f6a1de00640 1 -- 172.21.15.86:0/902820547 wait complete. 2024-04-20T21:27:55.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.926+0000 7f6a1de00640 1 Processor -- start 2024-04-20T21:27:55.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.926+0000 7f6a1de00640 1 -- start start 2024-04-20T21:27:55.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.927+0000 7f6a1de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a180cfb80 0x7f6a18155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:55.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.927+0000 7f6a1de00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a180d4310 0x7f6a18161060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:55.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.927+0000 7f6a1de00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a1815e870 0x7f6a181615a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:55.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.927+0000 7f6a1de00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6a180cf660 con 0x7f6a180cfb80 2024-04-20T21:27:55.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.927+0000 7f6a1de00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6a180cf380 con 0x7f6a180d4310 2024-04-20T21:27:55.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.927+0000 7f6a1de00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6a1806d3c0 con 0x7f6a1815e870 2024-04-20T21:27:55.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.927+0000 7f6a1ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a180cfb80 0x7f6a18155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:55.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.927+0000 7f6a1ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a180cfb80 0x7f6a18155ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52876/0 (socket says 172.21.15.86:52876) 2024-04-20T21:27:55.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.927+0000 7f6a1ca00640 1 -- 172.21.15.86:0/3697444210 learned_addr learned my addr 172.21.15.86:0/3697444210 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:55.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.928+0000 7f6a17e00640 1 --2- 172.21.15.86:0/3697444210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a180d4310 0x7f6a18161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:55.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.928+0000 7f6a1d400640 1 --2- 172.21.15.86:0/3697444210 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a1815e870 0x7f6a181615a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:55.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.928+0000 7f6a1ca00640 1 -- 172.21.15.86:0/3697444210 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a1815e870 msgr2=0x7f6a181615a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:55.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.928+0000 7f6a1ca00640 1 --2- 172.21.15.86:0/3697444210 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a1815e870 0x7f6a181615a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:55.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.928+0000 7f6a1ca00640 1 -- 172.21.15.86:0/3697444210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a180d4310 msgr2=0x7f6a18161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:55.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.928+0000 7f6a1ca00640 1 --2- 172.21.15.86:0/3697444210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a180d4310 0x7f6a18161060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:55.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.928+0000 7f6a1ca00640 1 -- 172.21.15.86:0/3697444210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a18161b90 con 0x7f6a180cfb80 2024-04-20T21:27:55.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.928+0000 7f6a17e00640 1 --2- 172.21.15.86:0/3697444210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a180d4310 0x7f6a18161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:27:55.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.928+0000 7f6a1ca00640 1 --2- 172.21.15.86:0/3697444210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a180cfb80 0x7f6a18155ba0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f6a08000c00 tx=0x7f6a080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:55.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.928+0000 7f6a14c00640 1 -- 172.21.15.86:0/3697444210 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a08035040 con 0x7f6a180cfb80 2024-04-20T21:27:55.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.928+0000 7f6a14c00640 1 -- 172.21.15.86:0/3697444210 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a08045480 con 0x7f6a180cfb80 2024-04-20T21:27:55.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.928+0000 7f6a14c00640 1 -- 172.21.15.86:0/3697444210 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a08045870 con 0x7f6a180cfb80 2024-04-20T21:27:55.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.928+0000 7f6a1de00640 1 -- 172.21.15.86:0/3697444210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a180cb9a0 con 0x7f6a180cfb80 2024-04-20T21:27:55.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.929+0000 7f6a1de00640 1 -- 172.21.15.86:0/3697444210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a180cbf60 con 0x7f6a180cfb80 2024-04-20T21:27:55.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.930+0000 7f6a1de00640 1 -- 172.21.15.86:0/3697444210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a180b3710 con 0x7f6a180cfb80 2024-04-20T21:27:55.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.931+0000 7f6a14c00640 1 -- 172.21.15.86:0/3697444210 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6a08038020 con 0x7f6a180cfb80 2024-04-20T21:27:55.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.931+0000 7f6a14c00640 1 --2- 172.21.15.86:0/3697444210 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f69f806c8f0 0x7f69f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:55.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.932+0000 7f6a14c00640 1 -- 172.21.15.86:0/3697444210 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7f6a080adfe0 con 0x7f6a180cfb80 2024-04-20T21:27:55.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.932+0000 7f6a17e00640 1 --2- 172.21.15.86:0/3697444210 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f69f806c8f0 0x7f69f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:55.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.933+0000 7f6a17e00640 1 --2- 172.21.15.86:0/3697444210 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f69f806c8f0 0x7f69f806edb0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f6a04006fd0 tx=0x7f6a04008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:55.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:55.936+0000 7f6a14c00640 1 -- 172.21.15.86:0/3697444210 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6a180b3710 con 0x7f6a180cfb80 2024-04-20T21:27:56.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.134+0000 7f6a1de00640 1 -- 172.21.15.86:0/3697444210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6a1806cc60 con 0x7f6a180cfb80 2024-04-20T21:27:56.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.135+0000 7f6a14c00640 1 -- 172.21.15.86:0/3697444210 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) ==== 93+0+10 (secure 0 0 0) 0x7f6a0807c460 con 0x7f6a180cfb80 2024-04-20T21:27:56.134 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:27:56.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.137+0000 7f6a1de00640 1 -- 172.21.15.86:0/3697444210 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f69f806c8f0 msgr2=0x7f69f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:56.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.137+0000 7f6a1de00640 1 --2- 172.21.15.86:0/3697444210 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f69f806c8f0 0x7f69f806edb0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f6a04006fd0 tx=0x7f6a04008040 comp rx=0 tx=0).stop 2024-04-20T21:27:56.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.137+0000 7f6a1de00640 1 -- 172.21.15.86:0/3697444210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a180cfb80 msgr2=0x7f6a18155ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:56.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.137+0000 7f6a1de00640 1 --2- 172.21.15.86:0/3697444210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a180cfb80 0x7f6a18155ba0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f6a08000c00 tx=0x7f6a080046c0 comp rx=0 tx=0).stop 2024-04-20T21:27:56.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.137+0000 7f6a1de00640 1 -- 172.21.15.86:0/3697444210 shutdown_connections 2024-04-20T21:27:56.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.137+0000 7f6a1de00640 1 --2- 172.21.15.86:0/3697444210 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f69f806c8f0 0x7f69f806edb0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:56.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.137+0000 7f6a1de00640 1 --2- 172.21.15.86:0/3697444210 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6a1815e870 0x7f6a181615a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:56.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.137+0000 7f6a1de00640 1 --2- 172.21.15.86:0/3697444210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a180d4310 0x7f6a18161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:56.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.137+0000 7f6a1de00640 1 --2- 172.21.15.86:0/3697444210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a180cfb80 0x7f6a18155ba0 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:56.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.137+0000 7f6a1de00640 1 -- 172.21.15.86:0/3697444210 >> 172.21.15.86:0/3697444210 conn(0x7f6a1809d9c0 msgr2=0x7f6a1815dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:56.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.138+0000 7f6a1de00640 1 -- 172.21.15.86:0/3697444210 shutdown_connections 2024-04-20T21:27:56.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.138+0000 7f6a1de00640 1 -- 172.21.15.86:0/3697444210 wait complete. 2024-04-20T21:27:56.155 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:27:56.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.356+0000 7f97e5200640 1 Processor -- start 2024-04-20T21:27:56.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.356+0000 7f97e5200640 1 -- start start 2024-04-20T21:27:56.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.358+0000 7f97e5200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97e00d45a0 0x7f97e00d3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:56.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.358+0000 7f97e5200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f97e015ea70 0x7f97e00d4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:56.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.358+0000 7f97e5200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97e00cf120 con 0x7f97e006d570 2024-04-20T21:27:56.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.358+0000 7f97e5200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f97e006de70 con 0x7f97e015ea70 2024-04-20T21:27:56.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.358+0000 7f97e5200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f97e006e150 con 0x7f97e00d45a0 2024-04-20T21:27:56.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.358+0000 7f97d6a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f97e015ea70 0x7f97e00d4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:56.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.358+0000 7f97dfe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f97e006d570 0x7f97e00d1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40656/0 (socket says 172.21.15.86:40656) 2024-04-20T21:27:56.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.358+0000 7f97dfe00640 1 -- 172.21.15.86:0/2751406958 learned_addr learned my addr 172.21.15.86:0/2751406958 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:27:56.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.358+0000 7f97df400640 1 --2- 172.21.15.86:0/2751406958 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97e00d45a0 0x7f97e00d3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:56.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.359+0000 7f97d6a00640 1 -- 172.21.15.86:0/2751406958 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97e00d45a0 msgr2=0x7f97e00d3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:56.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.359+0000 7f97d6a00640 1 --2- 172.21.15.86:0/2751406958 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97e00d45a0 0x7f97e00d3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:56.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.359+0000 7f97d6a00640 1 -- 172.21.15.86:0/2751406958 >> v1:172.21.15.86:6789/0 conn(0x7f97e006d570 legacy=0x7f97e00d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:56.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.359+0000 7f97d6a00640 1 -- 172.21.15.86:0/2751406958 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97e006e430 con 0x7f97e015ea70 2024-04-20T21:27:56.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.359+0000 7f97dea00640 1 -- 172.21.15.86:0/2751406958 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3759539526 0 0) 0x7f97e00cf120 con 0x7f97e006d570 2024-04-20T21:27:56.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.359+0000 7f97df400640 1 --2- 172.21.15.86:0/2751406958 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97e00d45a0 0x7f97e00d3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:27:56.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.360+0000 7f97d6a00640 1 --2- 172.21.15.86:0/2751406958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97e015ea70 0x7f97e00d4980 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f97c80097a0 tx=0x7f97c802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4479664dea7287bc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:56.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.360+0000 7f97dea00640 1 -- 172.21.15.86:0/2751406958 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c80315a0 con 0x7f97e015ea70 2024-04-20T21:27:56.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.360+0000 7f97dea00640 1 -- 172.21.15.86:0/2751406958 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97c8037070 con 0x7f97e015ea70 2024-04-20T21:27:56.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.360+0000 7f97dea00640 1 -- 172.21.15.86:0/2751406958 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c8031960 con 0x7f97e015ea70 2024-04-20T21:27:56.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.361+0000 7f97e5200640 1 -- 172.21.15.86:0/2751406958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97e015ea70 msgr2=0x7f97e00d4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:56.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.361+0000 7f97e5200640 1 --2- 172.21.15.86:0/2751406958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97e015ea70 0x7f97e00d4980 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f97c80097a0 tx=0x7f97c802b840 comp rx=0 tx=0).stop 2024-04-20T21:27:56.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.361+0000 7f97e5200640 1 -- 172.21.15.86:0/2751406958 shutdown_connections 2024-04-20T21:27:56.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.361+0000 7f97e5200640 1 --2- 172.21.15.86:0/2751406958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97e015ea70 0x7f97e00d4980 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:56.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.361+0000 7f97e5200640 1 --2- 172.21.15.86:0/2751406958 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97e00d45a0 0x7f97e00d3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:56.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.361+0000 7f97e5200640 1 -- 172.21.15.86:0/2751406958 >> 172.21.15.86:0/2751406958 conn(0x7f97e009d9d0 msgr2=0x7f97e008bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:56.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.361+0000 7f97e5200640 1 -- 172.21.15.86:0/2751406958 shutdown_connections 2024-04-20T21:27:56.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.361+0000 7f97e5200640 1 -- 172.21.15.86:0/2751406958 wait complete. 2024-04-20T21:27:56.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.362+0000 7f97e5200640 1 Processor -- start 2024-04-20T21:27:56.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.363+0000 7f97e5200640 1 -- start start 2024-04-20T21:27:56.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.363+0000 7f97e5200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97e006d570 0x7f97e0161390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:56.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.363+0000 7f97e5200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97e00d45a0 0x7f97e015aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:56.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.364+0000 7f97e5200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97e015ea70 0x7f97e01f75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:56.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.364+0000 7f97e5200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f97e006da00 con 0x7f97e00d45a0 2024-04-20T21:27:56.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.364+0000 7f97e5200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f97e00d4ec0 con 0x7f97e006d570 2024-04-20T21:27:56.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.364+0000 7f97e5200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f97e006db80 con 0x7f97e015ea70 2024-04-20T21:27:56.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.364+0000 7f97d6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97e00d45a0 0x7f97e015aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:56.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.364+0000 7f97d6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97e00d45a0 0x7f97e015aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52884/0 (socket says 172.21.15.86:52884) 2024-04-20T21:27:56.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.364+0000 7f97d6a00640 1 -- 172.21.15.86:0/3125907965 learned_addr learned my addr 172.21.15.86:0/3125907965 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:56.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.364+0000 7f97d6a00640 1 -- 172.21.15.86:0/3125907965 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97e015ea70 msgr2=0x7f97e01f75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:56.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.364+0000 7f97d6a00640 1 --2- 172.21.15.86:0/3125907965 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97e015ea70 0x7f97e01f75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:56.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.364+0000 7f97d6a00640 1 -- 172.21.15.86:0/3125907965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97e006d570 msgr2=0x7f97e0161390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:56.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.364+0000 7f97d6a00640 1 --2- 172.21.15.86:0/3125907965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97e006d570 0x7f97e0161390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:56.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.364+0000 7f97d6a00640 1 -- 172.21.15.86:0/3125907965 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97e00b7500 con 0x7f97e00d45a0 2024-04-20T21:27:56.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.365+0000 7f97d6a00640 1 --2- 172.21.15.86:0/3125907965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97e00d45a0 0x7f97e015aa30 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f97c8034040 tx=0x7f97c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:56.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.365+0000 7f97d7e00640 1 -- 172.21.15.86:0/3125907965 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c8032840 con 0x7f97e00d45a0 2024-04-20T21:27:56.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.365+0000 7f97d7e00640 1 -- 172.21.15.86:0/3125907965 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97c80329e0 con 0x7f97e00d45a0 2024-04-20T21:27:56.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.365+0000 7f97d7e00640 1 -- 172.21.15.86:0/3125907965 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c8032dd0 con 0x7f97e00d45a0 2024-04-20T21:27:56.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.365+0000 7f97e5200640 1 -- 172.21.15.86:0/3125907965 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97e00b77e0 con 0x7f97e00d45a0 2024-04-20T21:27:56.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.365+0000 7f97e5200640 1 -- 172.21.15.86:0/3125907965 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97e0160b90 con 0x7f97e00d45a0 2024-04-20T21:27:56.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.366+0000 7f97e5200640 1 -- 172.21.15.86:0/3125907965 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97e004fd90 con 0x7f97e00d45a0 2024-04-20T21:27:56.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.367+0000 7f97d7e00640 1 -- 172.21.15.86:0/3125907965 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f97c804e430 con 0x7f97e00d45a0 2024-04-20T21:27:56.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.369+0000 7f97d7e00640 1 --2- 172.21.15.86:0/3125907965 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f97bc06c8f0 0x7f97bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:56.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.369+0000 7f97d7e00640 1 -- 172.21.15.86:0/3125907965 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7f97c802f030 con 0x7f97e00d45a0 2024-04-20T21:27:56.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.369+0000 7f97df400640 1 --2- 172.21.15.86:0/3125907965 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f97bc06c8f0 0x7f97bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:56.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.370+0000 7f97df400640 1 --2- 172.21.15.86:0/3125907965 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f97bc06c8f0 0x7f97bc06edb0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f97d0008ea0 tx=0x7f97d00083d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:56.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.373+0000 7f97d7e00640 1 -- 172.21.15.86:0/3125907965 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f97c8040070 con 0x7f97e00d45a0 2024-04-20T21:27:56.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.574+0000 7f97e5200640 1 -- 172.21.15.86:0/3125907965 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f97e00b4150 con 0x7f97e00d45a0 2024-04-20T21:27:56.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.575+0000 7f97d7e00640 1 -- 172.21.15.86:0/3125907965 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v100) ==== 104+0+11 (secure 0 0 0) 0x7f97c8080e20 con 0x7f97e00d45a0 2024-04-20T21:27:56.574 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-20T21:27:56.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.577+0000 7f97e5200640 1 -- 172.21.15.86:0/3125907965 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f97bc06c8f0 msgr2=0x7f97bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:56.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.577+0000 7f97e5200640 1 --2- 172.21.15.86:0/3125907965 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f97bc06c8f0 0x7f97bc06edb0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f97d0008ea0 tx=0x7f97d00083d0 comp rx=0 tx=0).stop 2024-04-20T21:27:56.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.577+0000 7f97e5200640 1 -- 172.21.15.86:0/3125907965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97e00d45a0 msgr2=0x7f97e015aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:56.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.577+0000 7f97e5200640 1 --2- 172.21.15.86:0/3125907965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97e00d45a0 0x7f97e015aa30 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f97c8034040 tx=0x7f97c80046c0 comp rx=0 tx=0).stop 2024-04-20T21:27:56.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.577+0000 7f97e5200640 1 -- 172.21.15.86:0/3125907965 shutdown_connections 2024-04-20T21:27:56.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.577+0000 7f97e5200640 1 --2- 172.21.15.86:0/3125907965 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f97bc06c8f0 0x7f97bc06edb0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:56.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.577+0000 7f97e5200640 1 --2- 172.21.15.86:0/3125907965 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f97e015ea70 0x7f97e01f75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:56.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.577+0000 7f97e5200640 1 --2- 172.21.15.86:0/3125907965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97e00d45a0 0x7f97e015aa30 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:56.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.577+0000 7f97e5200640 1 --2- 172.21.15.86:0/3125907965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97e006d570 0x7f97e0161390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:56.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.577+0000 7f97e5200640 1 -- 172.21.15.86:0/3125907965 >> 172.21.15.86:0/3125907965 conn(0x7f97e009d9d0 msgr2=0x7f97e008bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:56.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.578+0000 7f97e5200640 1 -- 172.21.15.86:0/3125907965 shutdown_connections 2024-04-20T21:27:56.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.578+0000 7f97e5200640 1 -- 172.21.15.86:0/3125907965 wait complete. 2024-04-20T21:27:56.595 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:27:56.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.790+0000 7fc81d400640 1 Processor -- start 2024-04-20T21:27:56.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.790+0000 7fc81d400640 1 -- start start 2024-04-20T21:27:56.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.792+0000 7fc81d400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc8180d04f0 0x7fc81806e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:56.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.792+0000 7fc81d400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc818070a80 0x7fc8180af750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:56.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.792+0000 7fc81d400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc81815f300 con 0x7fc81806ea40 2024-04-20T21:27:56.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.792+0000 7fc81d400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fc8180d08d0 con 0x7fc8180d04f0 2024-04-20T21:27:56.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.792+0000 7fc81d400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc8180aecb0 con 0x7fc818070a80 2024-04-20T21:27:56.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.793+0000 7fc81ca00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc818070a80 0x7fc8180af750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:56.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.793+0000 7fc817e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc8180d04f0 0x7fc81806e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:56.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.793+0000 7fc81ca00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc818070a80 0x7fc8180af750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:57014/0 (socket says 172.21.15.86:57014) 2024-04-20T21:27:56.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.793+0000 7fc817e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc8180d04f0 0x7fc81806e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.86:40660/0 (socket says 172.21.15.86:40660) 2024-04-20T21:27:56.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.793+0000 7fc81ca00640 1 -- 172.21.15.86:0/2450419084 learned_addr learned my addr 172.21.15.86:0/2450419084 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:56.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.794+0000 7fc817e00640 1 -- 172.21.15.86:0/2450419084 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc818070a80 msgr2=0x7fc8180af750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:56.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.794+0000 7fc817e00640 1 --2- 172.21.15.86:0/2450419084 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc818070a80 0x7fc8180af750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:56.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.794+0000 7fc817e00640 1 -- 172.21.15.86:0/2450419084 >> v1:172.21.15.144:6789/0 conn(0x7fc81806ea40 legacy=0x7fc818070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:56.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.794+0000 7fc817e00640 1 -- 172.21.15.86:0/2450419084 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc81806f070 con 0x7fc8180d04f0 2024-04-20T21:27:56.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.794+0000 7fc81ca00640 1 --2- 172.21.15.86:0/2450419084 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc818070a80 0x7fc8180af750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:27:56.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.795+0000 7fc817e00640 1 --2- 172.21.15.86:0/2450419084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8180d04f0 0x7fc81806e500 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc8080097a0 tx=0x7fc80802b7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75f8a7805798ede6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:56.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.795+0000 7fc816a00640 1 -- 172.21.15.86:0/2450419084 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc808038070 con 0x7fc8180d04f0 2024-04-20T21:27:56.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.796+0000 7fc816a00640 1 -- 172.21.15.86:0/2450419084 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc808031700 con 0x7fc8180d04f0 2024-04-20T21:27:56.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.796+0000 7fc816a00640 1 -- 172.21.15.86:0/2450419084 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc808031ad0 con 0x7fc8180d04f0 2024-04-20T21:27:56.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.796+0000 7fc81d400640 1 -- 172.21.15.86:0/2450419084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8180d04f0 msgr2=0x7fc81806e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:56.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.796+0000 7fc81d400640 1 --2- 172.21.15.86:0/2450419084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8180d04f0 0x7fc81806e500 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc8080097a0 tx=0x7fc80802b7a0 comp rx=0 tx=0).stop 2024-04-20T21:27:56.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.796+0000 7fc81d400640 1 -- 172.21.15.86:0/2450419084 shutdown_connections 2024-04-20T21:27:56.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.796+0000 7fc81d400640 1 --2- 172.21.15.86:0/2450419084 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc818070a80 0x7fc8180af750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:56.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.796+0000 7fc81d400640 1 --2- 172.21.15.86:0/2450419084 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8180d04f0 0x7fc81806e500 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:56.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.796+0000 7fc81d400640 1 -- 172.21.15.86:0/2450419084 >> 172.21.15.86:0/2450419084 conn(0x7fc818073360 msgr2=0x7fc81806b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:56.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.796+0000 7fc81d400640 1 -- 172.21.15.86:0/2450419084 shutdown_connections 2024-04-20T21:27:56.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.796+0000 7fc81d400640 1 -- 172.21.15.86:0/2450419084 wait complete. 2024-04-20T21:27:56.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.798+0000 7fc81d400640 1 Processor -- start 2024-04-20T21:27:56.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.798+0000 7fc81d400640 1 -- start start 2024-04-20T21:27:56.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.799+0000 7fc81d400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc81806ea40 0x7fc8181eb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:56.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.799+0000 7fc81d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc818070a80 0x7fc8181e9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:56.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.799+0000 7fc81d400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8180d04f0 0x7fc818142300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:56.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.799+0000 7fc81d400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc8180b3c20 con 0x7fc818070a80 2024-04-20T21:27:56.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.799+0000 7fc81d400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc8180aef50 con 0x7fc8180d04f0 2024-04-20T21:27:56.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.799+0000 7fc81d400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc8180b3da0 con 0x7fc81806ea40 2024-04-20T21:27:56.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.799+0000 7fc817400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc818070a80 0x7fc8181e9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:56.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.799+0000 7fc817400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc818070a80 0x7fc8181e9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52898/0 (socket says 172.21.15.86:52898) 2024-04-20T21:27:56.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.799+0000 7fc817400640 1 -- 172.21.15.86:0/1429716021 learned_addr learned my addr 172.21.15.86:0/1429716021 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:56.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.800+0000 7fc81ca00640 1 --2- 172.21.15.86:0/1429716021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8180d04f0 0x7fc818142300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:56.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.800+0000 7fc817e00640 1 --2- 172.21.15.86:0/1429716021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc81806ea40 0x7fc8181eb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:56.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.800+0000 7fc817400640 1 -- 172.21.15.86:0/1429716021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc81806ea40 msgr2=0x7fc8181eb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:56.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.800+0000 7fc817400640 1 --2- 172.21.15.86:0/1429716021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc81806ea40 0x7fc8181eb9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:56.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.800+0000 7fc817400640 1 -- 172.21.15.86:0/1429716021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8180d04f0 msgr2=0x7fc818142300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:56.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.800+0000 7fc817400640 1 --2- 172.21.15.86:0/1429716021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8180d04f0 0x7fc818142300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:56.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.800+0000 7fc817400640 1 -- 172.21.15.86:0/1429716021 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8181ea210 con 0x7fc818070a80 2024-04-20T21:27:56.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.800+0000 7fc81ca00640 1 --2- 172.21.15.86:0/1429716021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8180d04f0 0x7fc818142300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:27:56.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.800+0000 7fc817400640 1 --2- 172.21.15.86:0/1429716021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc818070a80 0x7fc8181e9770 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fc80400c3c0 tx=0x7fc80400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:56.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.800+0000 7fc80fe00640 1 -- 172.21.15.86:0/1429716021 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8040052d0 con 0x7fc818070a80 2024-04-20T21:27:56.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.800+0000 7fc80fe00640 1 -- 172.21.15.86:0/1429716021 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc804005490 con 0x7fc818070a80 2024-04-20T21:27:56.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.800+0000 7fc81d400640 1 -- 172.21.15.86:0/1429716021 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8181ea4f0 con 0x7fc818070a80 2024-04-20T21:27:56.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.800+0000 7fc80fe00640 1 -- 172.21.15.86:0/1429716021 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc804007260 con 0x7fc818070a80 2024-04-20T21:27:56.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.800+0000 7fc81d400640 1 -- 172.21.15.86:0/1429716021 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8181eaab0 con 0x7fc818070a80 2024-04-20T21:27:56.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.801+0000 7fc81d400640 1 -- 172.21.15.86:0/1429716021 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc81815f500 con 0x7fc818070a80 2024-04-20T21:27:56.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.802+0000 7fc80fe00640 1 -- 172.21.15.86:0/1429716021 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc804007400 con 0x7fc818070a80 2024-04-20T21:27:56.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.803+0000 7fc80fe00640 1 --2- 172.21.15.86:0/1429716021 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc7f806c8f0 0x7fc7f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:56.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.803+0000 7fc80fe00640 1 -- 172.21.15.86:0/1429716021 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7fc80408b120 con 0x7fc818070a80 2024-04-20T21:27:56.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.803+0000 7fc817e00640 1 --2- 172.21.15.86:0/1429716021 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc7f806c8f0 0x7fc7f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:56.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.804+0000 7fc817e00640 1 --2- 172.21.15.86:0/1429716021 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc7f806c8f0 0x7fc7f806edb0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fc808035000 tx=0x7fc808008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:56.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:56.807+0000 7fc80fe00640 1 -- 172.21.15.86:0/1429716021 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc81815f500 con 0x7fc818070a80 2024-04-20T21:27:57.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.006+0000 7fc81d400640 1 -- 172.21.15.86:0/1429716021 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc818159290 con 0x7fc818070a80 2024-04-20T21:27:57.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.007+0000 7fc80fe00640 1 -- 172.21.15.86:0/1429716021 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v100) ==== 100+0+11 (secure 0 0 0) 0x7fc804059420 con 0x7fc818070a80 2024-04-20T21:27:57.006 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-20T21:27:57.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.009+0000 7fc81d400640 1 -- 172.21.15.86:0/1429716021 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc7f806c8f0 msgr2=0x7fc7f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:57.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.009+0000 7fc81d400640 1 --2- 172.21.15.86:0/1429716021 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc7f806c8f0 0x7fc7f806edb0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fc808035000 tx=0x7fc808008030 comp rx=0 tx=0).stop 2024-04-20T21:27:57.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.009+0000 7fc81d400640 1 -- 172.21.15.86:0/1429716021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc818070a80 msgr2=0x7fc8181e9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:57.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.009+0000 7fc81d400640 1 --2- 172.21.15.86:0/1429716021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc818070a80 0x7fc8181e9770 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fc80400c3c0 tx=0x7fc80400c880 comp rx=0 tx=0).stop 2024-04-20T21:27:57.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.010+0000 7fc81d400640 1 -- 172.21.15.86:0/1429716021 shutdown_connections 2024-04-20T21:27:57.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.010+0000 7fc81d400640 1 --2- 172.21.15.86:0/1429716021 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc7f806c8f0 0x7fc7f806edb0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:57.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.010+0000 7fc81d400640 1 --2- 172.21.15.86:0/1429716021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8180d04f0 0x7fc818142300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:57.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.010+0000 7fc81d400640 1 --2- 172.21.15.86:0/1429716021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc818070a80 0x7fc8181e9770 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:57.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.010+0000 7fc81d400640 1 --2- 172.21.15.86:0/1429716021 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc81806ea40 0x7fc8181eb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:57.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.010+0000 7fc81d400640 1 -- 172.21.15.86:0/1429716021 >> 172.21.15.86:0/1429716021 conn(0x7fc818073360 msgr2=0x7fc818160230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:57.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.010+0000 7fc81d400640 1 -- 172.21.15.86:0/1429716021 shutdown_connections 2024-04-20T21:27:57.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.010+0000 7fc81d400640 1 -- 172.21.15.86:0/1429716021 wait complete. 2024-04-20T21:27:57.026 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T21:27:57.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.231+0000 7fcf52e00640 1 Processor -- start 2024-04-20T21:27:57.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.231+0000 7fcf52e00640 1 -- start start 2024-04-20T21:27:57.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.232+0000 7fcf52e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fcf4c06f070 0x7fcf4c0b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:57.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.232+0000 7fcf52e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf4c15b9d0 con 0x7fcf4c0cff80 2024-04-20T21:27:57.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.232+0000 7fcf52e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf4c0af850 con 0x7fcf4c15b5f0 2024-04-20T21:27:57.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.232+0000 7fcf52e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fcf4c0b4650 con 0x7fcf4c06f070 2024-04-20T21:27:57.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.233+0000 7fcf50e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fcf4c0cff80 0x7fcf4c0ce130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40658/0 (socket says 172.21.15.86:40658) 2024-04-20T21:27:57.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.233+0000 7fcf50e00640 1 -- 172.21.15.86:0/2986088022 learned_addr learned my addr 172.21.15.86:0/2986088022 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:27:57.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.233+0000 7fcf43400640 1 --2- 172.21.15.86:0/2986088022 >> v2:172.21.15.144:3300/0 conn(0x7fcf4c06f070 0x7fcf4c0b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:57.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.233+0000 7fcf4b400640 1 -- 172.21.15.86:0/2986088022 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2649641752 0 0) 0x7fcf4c15b9d0 con 0x7fcf4c0cff80 2024-04-20T21:27:57.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.233+0000 7fcf4b400640 1 -- 172.21.15.86:0/2986088022 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf30003520 con 0x7fcf4c0cff80 2024-04-20T21:27:57.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.234+0000 7fcf4b400640 1 -- 172.21.15.86:0/2986088022 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4072857656 0 0) 0x7fcf30003520 con 0x7fcf4c0cff80 2024-04-20T21:27:57.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.234+0000 7fcf4b400640 1 -- 172.21.15.86:0/2986088022 >> v2:172.21.15.144:3300/0 conn(0x7fcf4c06f070 msgr2=0x7fcf4c0b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:57.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.234+0000 7fcf4b400640 1 --2- 172.21.15.86:0/2986088022 >> v2:172.21.15.144:3300/0 conn(0x7fcf4c06f070 0x7fcf4c0b3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:57.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.234+0000 7fcf4b400640 1 -- 172.21.15.86:0/2986088022 >> v1:172.21.15.144:6789/0 conn(0x7fcf4c15b5f0 legacy=0x7fcf4c0b50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:57.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.234+0000 7fcf4b400640 1 -- 172.21.15.86:0/2986088022 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf4c0afb30 con 0x7fcf4c0cff80 2024-04-20T21:27:57.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.234+0000 7fcf4b400640 1 -- 172.21.15.86:0/2986088022 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fcf440019f0 con 0x7fcf4c0cff80 2024-04-20T21:27:57.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.234+0000 7fcf4b400640 1 -- 172.21.15.86:0/2986088022 >> v1:172.21.15.86:6789/0 conn(0x7fcf4c0cff80 legacy=0x7fcf4c0ce130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:57.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.234+0000 7fcf4b400640 1 --2- 172.21.15.86:0/2986088022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf30003a00 0x7fcf30003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:57.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.234+0000 7fcf4b400640 1 --2- 172.21.15.86:0/2986088022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf30004800 0x7fcf30004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:57.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.234+0000 7fcf4b400640 1 --2- 172.21.15.86:0/2986088022 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcf300051c0 0x7fcf3000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:57.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.234+0000 7fcf4b400640 1 -- 172.21.15.86:0/2986088022 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4c0b4650 con 0x7fcf30003a00 2024-04-20T21:27:57.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.234+0000 7fcf4b400640 1 -- 172.21.15.86:0/2986088022 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3000c090 con 0x7fcf30004800 2024-04-20T21:27:57.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.234+0000 7fcf4b400640 1 -- 172.21.15.86:0/2986088022 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fcf3000c290 con 0x7fcf300051c0 2024-04-20T21:27:57.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.234+0000 7fcf4be00640 1 --2- 172.21.15.86:0/2986088022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf30003a00 0x7fcf30003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:57.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.234+0000 7fcf43400640 1 --2- 172.21.15.86:0/2986088022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf30004800 0x7fcf30004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:57.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.234+0000 7fcf4be00640 1 -- 172.21.15.86:0/2986088022 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcf300051c0 msgr2=0x7fcf3000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:57.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.234+0000 7fcf4be00640 1 --2- 172.21.15.86:0/2986088022 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcf300051c0 0x7fcf3000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:57.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.234+0000 7fcf4be00640 1 -- 172.21.15.86:0/2986088022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf30004800 msgr2=0x7fcf30004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:57.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.234+0000 7fcf4be00640 1 --2- 172.21.15.86:0/2986088022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf30004800 0x7fcf30004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:57.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.234+0000 7fcf4be00640 1 -- 172.21.15.86:0/2986088022 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf30003520 con 0x7fcf30003a00 2024-04-20T21:27:57.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.235+0000 7fcf50e00640 1 -- 172.21.15.86:0/2986088022 reap_dead start 2024-04-20T21:27:57.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.236+0000 7fcf4be00640 1 --2- 172.21.15.86:0/2986088022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf30003a00 0x7fcf30003e00 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c003db0 tx=0x7fcf30005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d6b5e71525fbdea server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:57.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.236+0000 7fcf4b400640 1 -- 172.21.15.86:0/2986088022 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c037070 con 0x7fcf30003a00 2024-04-20T21:27:57.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.236+0000 7fcf4b400640 1 -- 172.21.15.86:0/2986088022 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf3c032600 con 0x7fcf30003a00 2024-04-20T21:27:57.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.236+0000 7fcf4b400640 1 -- 172.21.15.86:0/2986088022 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c003980 con 0x7fcf30003a00 2024-04-20T21:27:57.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.236+0000 7fcf52e00640 1 -- 172.21.15.86:0/2986088022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf30003a00 msgr2=0x7fcf30003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:57.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.236+0000 7fcf52e00640 1 --2- 172.21.15.86:0/2986088022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf30003a00 0x7fcf30003e00 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c003db0 tx=0x7fcf30005830 comp rx=0 tx=0).stop 2024-04-20T21:27:57.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.237+0000 7fcf52e00640 1 -- 172.21.15.86:0/2986088022 shutdown_connections 2024-04-20T21:27:57.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.237+0000 7fcf52e00640 1 --2- 172.21.15.86:0/2986088022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf30003a00 0x7fcf30003e00 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:57.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.237+0000 7fcf52e00640 1 -- 172.21.15.86:0/2986088022 >> 172.21.15.86:0/2986088022 conn(0x7fcf4c06b980 msgr2=0x7fcf4c06bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:57.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.237+0000 7fcf52e00640 1 -- 172.21.15.86:0/2986088022 shutdown_connections 2024-04-20T21:27:57.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.237+0000 7fcf52e00640 1 -- 172.21.15.86:0/2986088022 wait complete. 2024-04-20T21:27:57.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.238+0000 7fcf52e00640 1 Processor -- start 2024-04-20T21:27:57.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.239+0000 7fcf52e00640 1 -- start start 2024-04-20T21:27:57.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.239+0000 7fcf52e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcf30003a00 0x7fcf4c14cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:57.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.239+0000 7fcf52e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf4c15d390 0x7fcf4c14d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:57.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.239+0000 7fcf52e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf4c1505f0 0x7fcf4c14de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:57.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.239+0000 7fcf52e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4c0b3d40 con 0x7fcf4c15d390 2024-04-20T21:27:57.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.239+0000 7fcf52e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4c0b4b80 con 0x7fcf4c1505f0 2024-04-20T21:27:57.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.239+0000 7fcf52e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fcf4c0b3ec0 con 0x7fcf30003a00 2024-04-20T21:27:57.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.240+0000 7fcf43400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf4c15d390 0x7fcf4c14d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:57.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.240+0000 7fcf43400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf4c15d390 0x7fcf4c14d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52916/0 (socket says 172.21.15.86:52916) 2024-04-20T21:27:57.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.240+0000 7fcf43400640 1 -- 172.21.15.86:0/3600905345 learned_addr learned my addr 172.21.15.86:0/3600905345 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:57.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.240+0000 7fcf4be00640 1 --2- 172.21.15.86:0/3600905345 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcf30003a00 0x7fcf4c14cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:57.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.240+0000 7fcf50e00640 1 --2- 172.21.15.86:0/3600905345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf4c1505f0 0x7fcf4c14de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:57.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.240+0000 7fcf43400640 1 -- 172.21.15.86:0/3600905345 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcf30003a00 msgr2=0x7fcf4c14cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:57.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.240+0000 7fcf43400640 1 --2- 172.21.15.86:0/3600905345 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcf30003a00 0x7fcf4c14cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:57.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.240+0000 7fcf43400640 1 -- 172.21.15.86:0/3600905345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf4c1505f0 msgr2=0x7fcf4c14de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:57.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.240+0000 7fcf43400640 1 --2- 172.21.15.86:0/3600905345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf4c1505f0 0x7fcf4c14de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:57.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.240+0000 7fcf43400640 1 -- 172.21.15.86:0/3600905345 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf4c15d820 con 0x7fcf4c15d390 2024-04-20T21:27:57.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.241+0000 7fcf43400640 1 --2- 172.21.15.86:0/3600905345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf4c15d390 0x7fcf4c14d920 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fcf340116d0 tx=0x7fcf34011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:57.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.241+0000 7fcf48c00640 1 -- 172.21.15.86:0/3600905345 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3400f9c0 con 0x7fcf4c15d390 2024-04-20T21:27:57.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.241+0000 7fcf48c00640 1 -- 172.21.15.86:0/3600905345 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf3400fb60 con 0x7fcf4c15d390 2024-04-20T21:27:57.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.241+0000 7fcf48c00640 1 -- 172.21.15.86:0/3600905345 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf34015460 con 0x7fcf4c15d390 2024-04-20T21:27:57.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.241+0000 7fcf52e00640 1 -- 172.21.15.86:0/3600905345 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf4c157200 con 0x7fcf4c15d390 2024-04-20T21:27:57.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.241+0000 7fcf52e00640 1 -- 172.21.15.86:0/3600905345 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf4c14d300 con 0x7fcf4c15d390 2024-04-20T21:27:57.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.242+0000 7fcf52e00640 1 -- 172.21.15.86:0/3600905345 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf4c0689e0 con 0x7fcf4c15d390 2024-04-20T21:27:57.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.243+0000 7fcf48c00640 1 -- 172.21.15.86:0/3600905345 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcf34015600 con 0x7fcf4c15d390 2024-04-20T21:27:57.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.244+0000 7fcf48c00640 1 --2- 172.21.15.86:0/3600905345 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcf2806c8f0 0x7fcf2806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:57.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.244+0000 7fcf48c00640 1 -- 172.21.15.86:0/3600905345 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7fcf3408c570 con 0x7fcf4c15d390 2024-04-20T21:27:57.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.244+0000 7fcf4be00640 1 --2- 172.21.15.86:0/3600905345 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcf2806c8f0 0x7fcf2806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:57.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.245+0000 7fcf4be00640 1 --2- 172.21.15.86:0/3600905345 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcf2806c8f0 0x7fcf2806edb0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c034000 tx=0x7fcf3c009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:57.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.248+0000 7fcf48c00640 1 -- 172.21.15.86:0/3600905345 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fcf34019030 con 0x7fcf4c15d390 2024-04-20T21:27:57.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.466+0000 7fcf52e00640 1 -- 172.21.15.86:0/3600905345 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fcf4c155430 con 0x7fcf4c15d390 2024-04-20T21:27:57.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.467+0000 7fcf48c00640 1 -- 172.21.15.86:0/3600905345 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v100) ==== 80+0+0 (secure 0 0 0) 0x7fcf3408ae20 con 0x7fcf4c15d390 2024-04-20T21:27:57.465 INFO:teuthology.orchestra.run.smithi086.stderr:listed 0 entries 2024-04-20T21:27:57.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.469+0000 7fcf52e00640 1 -- 172.21.15.86:0/3600905345 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcf2806c8f0 msgr2=0x7fcf2806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:57.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.469+0000 7fcf52e00640 1 --2- 172.21.15.86:0/3600905345 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcf2806c8f0 0x7fcf2806edb0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c034000 tx=0x7fcf3c009030 comp rx=0 tx=0).stop 2024-04-20T21:27:57.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.469+0000 7fcf52e00640 1 -- 172.21.15.86:0/3600905345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf4c15d390 msgr2=0x7fcf4c14d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:57.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.469+0000 7fcf52e00640 1 --2- 172.21.15.86:0/3600905345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf4c15d390 0x7fcf4c14d920 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fcf340116d0 tx=0x7fcf34011b90 comp rx=0 tx=0).stop 2024-04-20T21:27:57.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.469+0000 7fcf52e00640 1 -- 172.21.15.86:0/3600905345 shutdown_connections 2024-04-20T21:27:57.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.469+0000 7fcf52e00640 1 --2- 172.21.15.86:0/3600905345 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcf2806c8f0 0x7fcf2806edb0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:57.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.469+0000 7fcf52e00640 1 --2- 172.21.15.86:0/3600905345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf4c1505f0 0x7fcf4c14de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:57.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.469+0000 7fcf52e00640 1 --2- 172.21.15.86:0/3600905345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf4c15d390 0x7fcf4c14d920 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:57.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.469+0000 7fcf52e00640 1 --2- 172.21.15.86:0/3600905345 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcf30003a00 0x7fcf4c14cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:57.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.469+0000 7fcf52e00640 1 -- 172.21.15.86:0/3600905345 >> 172.21.15.86:0/3600905345 conn(0x7fcf4c06b980 msgr2=0x7fcf4c06bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:57.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.470+0000 7fcf52e00640 1 -- 172.21.15.86:0/3600905345 shutdown_connections 2024-04-20T21:27:57.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.470+0000 7fcf52e00640 1 -- 172.21.15.86:0/3600905345 wait complete. 2024-04-20T21:27:57.486 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:27:57.486 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:27:57.534 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:27:57.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.776+0000 7feaaca00640 1 Processor -- start 2024-04-20T21:27:57.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.777+0000 7feaaca00640 1 -- start start 2024-04-20T21:27:57.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.778+0000 7feaaca00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feaa80ce460 0x7feaa80d1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:57.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.778+0000 7feaaca00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7feaa80ce840 0x7feaa80d18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:57.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.778+0000 7feaaca00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7feaa806d4a0 0x7feaa806d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:57.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.778+0000 7feaaca00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7feaa809b8d0 con 0x7feaa806d4a0 2024-04-20T21:27:57.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.778+0000 7feaaca00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7feaa809bd60 con 0x7feaa80ce840 2024-04-20T21:27:57.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.778+0000 7feaaca00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7feaa809a9e0 con 0x7feaa80ce460 2024-04-20T21:27:57.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.779+0000 7feaa6800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7feaa806d4a0 0x7feaa806d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:57.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.779+0000 7feaa6800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7feaa806d4a0 0x7feaa806d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52924/0 (socket says 172.21.15.86:52924) 2024-04-20T21:27:57.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.779+0000 7feaa6800640 1 -- 172.21.15.86:0/3811602223 learned_addr learned my addr 172.21.15.86:0/3811602223 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:57.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.779+0000 7feaa6800640 1 -- 172.21.15.86:0/3811602223 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feaa80ce460 msgr2=0x7feaa80d1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:57.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.779+0000 7feaa6800640 1 --2- 172.21.15.86:0/3811602223 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feaa80ce460 0x7feaa80d1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:57.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.779+0000 7feaa6800640 1 -- 172.21.15.86:0/3811602223 >> v2:172.21.15.144:3300/0 conn(0x7feaa80ce840 msgr2=0x7feaa80d18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:57.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.779+0000 7feaa5400640 1 --2- 172.21.15.86:0/3811602223 >> v2:172.21.15.144:3300/0 conn(0x7feaa80ce840 0x7feaa80d18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:57.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.779+0000 7feaa6800640 1 --2- 172.21.15.86:0/3811602223 >> v2:172.21.15.144:3300/0 conn(0x7feaa80ce840 0x7feaa80d18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:57.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.779+0000 7feaa6800640 1 -- 172.21.15.86:0/3811602223 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaa809acc0 con 0x7feaa806d4a0 2024-04-20T21:27:57.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.780+0000 7feaa6800640 1 --2- 172.21.15.86:0/3811602223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaa806d4a0 0x7feaa806d950 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7feaa80729c0 tx=0x7feaa002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbd1b74c3b6e4186 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:57.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.781+0000 7feaa4a00640 1 -- 172.21.15.86:0/3811602223 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa002ca70 con 0x7feaa806d4a0 2024-04-20T21:27:57.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.781+0000 7feaa4a00640 1 -- 172.21.15.86:0/3811602223 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaa002cc10 con 0x7feaa806d4a0 2024-04-20T21:27:57.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.781+0000 7feaa4a00640 1 -- 172.21.15.86:0/3811602223 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa00343f0 con 0x7feaa806d4a0 2024-04-20T21:27:57.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.781+0000 7feaaca00640 1 -- 172.21.15.86:0/3811602223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaa806d4a0 msgr2=0x7feaa806d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:57.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.781+0000 7feaaca00640 1 --2- 172.21.15.86:0/3811602223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaa806d4a0 0x7feaa806d950 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7feaa80729c0 tx=0x7feaa002bed0 comp rx=0 tx=0).stop 2024-04-20T21:27:57.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.781+0000 7feaaca00640 1 -- 172.21.15.86:0/3811602223 shutdown_connections 2024-04-20T21:27:57.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.781+0000 7feaaca00640 1 --2- 172.21.15.86:0/3811602223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaa806d4a0 0x7feaa806d950 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:57.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.781+0000 7feaaca00640 1 --2- 172.21.15.86:0/3811602223 >> v2:172.21.15.144:3300/0 conn(0x7feaa80ce840 0x7feaa80d18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:57.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.781+0000 7feaaca00640 1 --2- 172.21.15.86:0/3811602223 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feaa80ce460 0x7feaa80d1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:57.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.781+0000 7feaaca00640 1 -- 172.21.15.86:0/3811602223 >> 172.21.15.86:0/3811602223 conn(0x7feaa80b4080 msgr2=0x7feaa80b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:57.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.782+0000 7feaaca00640 1 -- 172.21.15.86:0/3811602223 shutdown_connections 2024-04-20T21:27:57.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.782+0000 7feaaca00640 1 -- 172.21.15.86:0/3811602223 wait complete. 2024-04-20T21:27:57.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.783+0000 7feaaca00640 1 Processor -- start 2024-04-20T21:27:57.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.783+0000 7feaaca00640 1 -- start start 2024-04-20T21:27:57.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.784+0000 7feaaca00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feaa806d4a0 0x7feaa815aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:57.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.784+0000 7feaaca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaa80ce460 0x7feaa80cb830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:57.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.784+0000 7feaaca00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feaa80ce840 0x7feaa80cbd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:57.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.784+0000 7feaaca00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feaa809b560 con 0x7feaa80ce460 2024-04-20T21:27:57.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.784+0000 7feaaca00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7feaa809b280 con 0x7feaa80ce840 2024-04-20T21:27:57.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.784+0000 7feaaca00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7feaa815e200 con 0x7feaa806d4a0 2024-04-20T21:27:57.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.785+0000 7feaa5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaa80ce460 0x7feaa80cb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:57.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.785+0000 7feaa5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaa80ce460 0x7feaa80cb830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52930/0 (socket says 172.21.15.86:52930) 2024-04-20T21:27:57.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.785+0000 7feaa5400640 1 -- 172.21.15.86:0/1646687455 learned_addr learned my addr 172.21.15.86:0/1646687455 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:57.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.785+0000 7feaa6800640 1 --2- 172.21.15.86:0/1646687455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feaa80ce840 0x7feaa80cbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:57.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.785+0000 7feaa5e00640 1 --2- 172.21.15.86:0/1646687455 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feaa806d4a0 0x7feaa815aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:57.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.785+0000 7feaa5400640 1 -- 172.21.15.86:0/1646687455 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feaa806d4a0 msgr2=0x7feaa815aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:57.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.785+0000 7feaa5400640 1 --2- 172.21.15.86:0/1646687455 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feaa806d4a0 0x7feaa815aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:57.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.785+0000 7feaa5400640 1 -- 172.21.15.86:0/1646687455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feaa80ce840 msgr2=0x7feaa80cbd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:57.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.785+0000 7feaa5400640 1 --2- 172.21.15.86:0/1646687455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feaa80ce840 0x7feaa80cbd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:57.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.785+0000 7feaa5400640 1 -- 172.21.15.86:0/1646687455 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaa81f7c10 con 0x7feaa80ce460 2024-04-20T21:27:57.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.785+0000 7feaa5400640 1 --2- 172.21.15.86:0/1646687455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaa80ce460 0x7feaa80cb830 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fea9000c7f0 tx=0x7fea9000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:57.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.786+0000 7fea9e000640 1 -- 172.21.15.86:0/1646687455 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea90015070 con 0x7feaa80ce460 2024-04-20T21:27:57.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.786+0000 7fea9e000640 1 -- 172.21.15.86:0/1646687455 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea90004220 con 0x7feaa80ce460 2024-04-20T21:27:57.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.786+0000 7fea9e000640 1 -- 172.21.15.86:0/1646687455 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea90007620 con 0x7feaa80ce460 2024-04-20T21:27:57.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.786+0000 7feaaca00640 1 -- 172.21.15.86:0/1646687455 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaa81f7ef0 con 0x7feaa80ce460 2024-04-20T21:27:57.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.786+0000 7feaaca00640 1 -- 172.21.15.86:0/1646687455 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaa815f270 con 0x7feaa80ce460 2024-04-20T21:27:57.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.787+0000 7feaaca00640 1 -- 172.21.15.86:0/1646687455 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaa806c5f0 con 0x7feaa80ce460 2024-04-20T21:27:57.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.788+0000 7fea9e000640 1 -- 172.21.15.86:0/1646687455 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fea900079d0 con 0x7feaa80ce460 2024-04-20T21:27:57.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.789+0000 7fea9e000640 1 --2- 172.21.15.86:0/1646687455 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fea8406c920 0x7fea8406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:57.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.789+0000 7fea9e000640 1 -- 172.21.15.86:0/1646687455 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7fea9008bb60 con 0x7feaa80ce460 2024-04-20T21:27:57.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.789+0000 7feaa5e00640 1 --2- 172.21.15.86:0/1646687455 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fea8406c920 0x7fea8406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:57.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.790+0000 7feaa5e00640 1 --2- 172.21.15.86:0/1646687455 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fea8406c920 0x7fea8406ede0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fea9800a810 tx=0x7fea98008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:57.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.794+0000 7fea9e000640 1 -- 172.21.15.86:0/1646687455 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fea90059f80 con 0x7feaa80ce460 2024-04-20T21:27:57.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.994+0000 7feaaca00640 1 -- 172.21.15.86:0/1646687455 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feaa80690f0 con 0x7feaa80ce460 2024-04-20T21:27:57.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.996+0000 7fea9e000640 1 -- 172.21.15.86:0/1646687455 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+16610 (secure 0 0 0) 0x7fea90059880 con 0x7feaa80ce460 2024-04-20T21:27:57.996 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:27:57.996 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":100,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:27:47.267402+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:27:43.469685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-20T21:27:44.673505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:27:57.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.999+0000 7feaaca00640 1 -- 172.21.15.86:0/1646687455 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fea8406c920 msgr2=0x7fea8406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:57.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.999+0000 7feaaca00640 1 --2- 172.21.15.86:0/1646687455 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fea8406c920 0x7fea8406ede0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fea9800a810 tx=0x7fea98008040 comp rx=0 tx=0).stop 2024-04-20T21:27:57.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.999+0000 7feaaca00640 1 -- 172.21.15.86:0/1646687455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaa80ce460 msgr2=0x7feaa80cb830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:57.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.999+0000 7feaaca00640 1 --2- 172.21.15.86:0/1646687455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaa80ce460 0x7feaa80cb830 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fea9000c7f0 tx=0x7fea9000ccb0 comp rx=0 tx=0).stop 2024-04-20T21:27:57.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.999+0000 7feaaca00640 1 -- 172.21.15.86:0/1646687455 shutdown_connections 2024-04-20T21:27:57.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.999+0000 7feaaca00640 1 --2- 172.21.15.86:0/1646687455 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fea8406c920 0x7fea8406ede0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:57.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.999+0000 7feaaca00640 1 --2- 172.21.15.86:0/1646687455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feaa80ce840 0x7feaa80cbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:57.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.999+0000 7feaaca00640 1 --2- 172.21.15.86:0/1646687455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feaa80ce460 0x7feaa80cb830 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:57.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.999+0000 7feaaca00640 1 --2- 172.21.15.86:0/1646687455 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feaa806d4a0 0x7feaa815aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:57.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.999+0000 7feaaca00640 1 -- 172.21.15.86:0/1646687455 >> 172.21.15.86:0/1646687455 conn(0x7feaa80b4080 msgr2=0x7feaa80d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:57.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:57.999+0000 7feaaca00640 1 -- 172.21.15.86:0/1646687455 shutdown_connections 2024-04-20T21:27:57.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.000+0000 7feaaca00640 1 -- 172.21.15.86:0/1646687455 wait complete. 2024-04-20T21:27:58.016 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:27:43.469685+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:27:44.673505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:27:58.016 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:27:58.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.221+0000 7f590f200640 1 Processor -- start 2024-04-20T21:27:58.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.221+0000 7f590f200640 1 -- start start 2024-04-20T21:27:58.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.223+0000 7f590f200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5910144ce0 0x7f591015f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:58.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.223+0000 7f590f200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59101450c0 0x7f591006b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:58.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.223+0000 7f590f200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59100b2cd0 con 0x7f59100b5d50 2024-04-20T21:27:58.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.223+0000 7f590f200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f59100b6230 con 0x7f5910144ce0 2024-04-20T21:27:58.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.223+0000 7f590f200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f59100b6510 con 0x7f59101450c0 2024-04-20T21:27:58.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.223+0000 7f5907e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f59100b5d50 0x7f59100b25c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40674/0 (socket says 172.21.15.86:40674) 2024-04-20T21:27:58.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.223+0000 7f5907e00640 1 -- 172.21.15.86:0/2182780461 learned_addr learned my addr 172.21.15.86:0/2182780461 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:27:58.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.223+0000 7f590de00640 1 --2- 172.21.15.86:0/2182780461 >> v2:172.21.15.86:3300/0 conn(0x7f5910144ce0 0x7f591015f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:58.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.224+0000 7f590e800640 1 --2- 172.21.15.86:0/2182780461 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59101450c0 0x7f591006b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:58.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.224+0000 7f590de00640 1 -- 172.21.15.86:0/2182780461 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59101450c0 msgr2=0x7f591006b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:58.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.224+0000 7f590de00640 1 --2- 172.21.15.86:0/2182780461 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59101450c0 0x7f591006b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:58.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.224+0000 7f590de00640 1 -- 172.21.15.86:0/2182780461 >> v1:172.21.15.86:6789/0 conn(0x7f59100b5d50 legacy=0x7f59100b25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:58.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.224+0000 7f590de00640 1 -- 172.21.15.86:0/2182780461 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5910147260 con 0x7f5910144ce0 2024-04-20T21:27:58.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.224+0000 7f590d400640 1 -- 172.21.15.86:0/2182780461 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 88696346 0 0) 0x7f59100b2cd0 con 0x7f59100b5d50 2024-04-20T21:27:58.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.225+0000 7f590de00640 1 --2- 172.21.15.86:0/2182780461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5910144ce0 0x7f591015f080 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f59000097a0 tx=0x7f590002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=663784a5f4eefb14 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:58.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.225+0000 7f590d400640 1 -- 172.21.15.86:0/2182780461 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5900035020 con 0x7f5910144ce0 2024-04-20T21:27:58.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.225+0000 7f590d400640 1 -- 172.21.15.86:0/2182780461 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5900031810 con 0x7f5910144ce0 2024-04-20T21:27:58.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.225+0000 7f590d400640 1 -- 172.21.15.86:0/2182780461 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5900031be0 con 0x7f5910144ce0 2024-04-20T21:27:58.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.226+0000 7f590f200640 1 -- 172.21.15.86:0/2182780461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5910144ce0 msgr2=0x7f591015f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:58.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.226+0000 7f590f200640 1 --2- 172.21.15.86:0/2182780461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5910144ce0 0x7f591015f080 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f59000097a0 tx=0x7f590002b8b0 comp rx=0 tx=0).stop 2024-04-20T21:27:58.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.226+0000 7f590f200640 1 -- 172.21.15.86:0/2182780461 shutdown_connections 2024-04-20T21:27:58.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.226+0000 7f590f200640 1 --2- 172.21.15.86:0/2182780461 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59101450c0 0x7f591006b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:58.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.226+0000 7f590f200640 1 --2- 172.21.15.86:0/2182780461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5910144ce0 0x7f591015f080 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:58.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.226+0000 7f590f200640 1 -- 172.21.15.86:0/2182780461 >> 172.21.15.86:0/2182780461 conn(0x7f59100b6810 msgr2=0x7f59100b6c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:58.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.226+0000 7f590f200640 1 -- 172.21.15.86:0/2182780461 shutdown_connections 2024-04-20T21:27:58.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.226+0000 7f590f200640 1 -- 172.21.15.86:0/2182780461 wait complete. 2024-04-20T21:27:58.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.228+0000 7f590f200640 1 Processor -- start 2024-04-20T21:27:58.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.228+0000 7f590f200640 1 -- start start 2024-04-20T21:27:58.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.229+0000 7f590f200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59100b5d50 0x7f591009df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:58.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.229+0000 7f590f200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5910144ce0 0x7f59100b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:58.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.229+0000 7f590f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59101450c0 0x7f59100b1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:58.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.229+0000 7f590f200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f591015f5c0 con 0x7f59101450c0 2024-04-20T21:27:58.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.229+0000 7f590f200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5910147820 con 0x7f5910144ce0 2024-04-20T21:27:58.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.229+0000 7f590f200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f591015f740 con 0x7f59100b5d50 2024-04-20T21:27:58.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.229+0000 7f590e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59101450c0 0x7f59100b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:58.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.230+0000 7f590e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59101450c0 0x7f59100b1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52948/0 (socket says 172.21.15.86:52948) 2024-04-20T21:27:58.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.230+0000 7f590e800640 1 -- 172.21.15.86:0/2011018907 learned_addr learned my addr 172.21.15.86:0/2011018907 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:58.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.230+0000 7f590de00640 1 --2- 172.21.15.86:0/2011018907 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59100b5d50 0x7f591009df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:58.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.230+0000 7f5907e00640 1 --2- 172.21.15.86:0/2011018907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5910144ce0 0x7f59100b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:58.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.230+0000 7f590e800640 1 -- 172.21.15.86:0/2011018907 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59100b5d50 msgr2=0x7f591009df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:58.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.230+0000 7f590e800640 1 --2- 172.21.15.86:0/2011018907 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59100b5d50 0x7f591009df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:58.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.230+0000 7f590e800640 1 -- 172.21.15.86:0/2011018907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5910144ce0 msgr2=0x7f59100b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:58.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.230+0000 7f590e800640 1 --2- 172.21.15.86:0/2011018907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5910144ce0 0x7f59100b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:58.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.230+0000 7f590e800640 1 -- 172.21.15.86:0/2011018907 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59100b18d0 con 0x7f59101450c0 2024-04-20T21:27:58.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.230+0000 7f5907e00640 1 --2- 172.21.15.86:0/2011018907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5910144ce0 0x7f59100b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:27:58.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.230+0000 7f590e800640 1 --2- 172.21.15.86:0/2011018907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59101450c0 0x7f59100b1230 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f5908007e90 tx=0x7f590800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:58.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.231+0000 7f5906000640 1 -- 172.21.15.86:0/2011018907 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5908014420 con 0x7f59101450c0 2024-04-20T21:27:58.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.231+0000 7f5906000640 1 -- 172.21.15.86:0/2011018907 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5908019070 con 0x7f59101450c0 2024-04-20T21:27:58.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.231+0000 7f5906000640 1 -- 172.21.15.86:0/2011018907 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5908014800 con 0x7f59101450c0 2024-04-20T21:27:58.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.231+0000 7f590f200640 1 -- 172.21.15.86:0/2011018907 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59101328f0 con 0x7f59101450c0 2024-04-20T21:27:58.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.231+0000 7f590f200640 1 -- 172.21.15.86:0/2011018907 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5910132eb0 con 0x7f59101450c0 2024-04-20T21:27:58.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.232+0000 7f590f200640 1 -- 172.21.15.86:0/2011018907 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59101330b0 con 0x7f59101450c0 2024-04-20T21:27:58.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.233+0000 7f5906000640 1 -- 172.21.15.86:0/2011018907 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f59080149a0 con 0x7f59101450c0 2024-04-20T21:27:58.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.234+0000 7f5906000640 1 --2- 172.21.15.86:0/2011018907 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f58ec06c8f0 0x7f58ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:58.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.234+0000 7f5906000640 1 -- 172.21.15.86:0/2011018907 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7f590808c440 con 0x7f59101450c0 2024-04-20T21:27:58.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.234+0000 7f590de00640 1 --2- 172.21.15.86:0/2011018907 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f58ec06c8f0 0x7f58ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:58.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.235+0000 7f590de00640 1 --2- 172.21.15.86:0/2011018907 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f58ec06c8f0 0x7f58ec06edb0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f5900036000 tx=0x7f5900008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:58.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.238+0000 7f5906000640 1 -- 172.21.15.86:0/2011018907 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f59101330b0 con 0x7f59101450c0 2024-04-20T21:27:58.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.438+0000 7f590f200640 1 -- 172.21.15.86:0/2011018907 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f591006fe60 con 0x7f59101450c0 2024-04-20T21:27:58.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.439+0000 7f5906000640 1 -- 172.21.15.86:0/2011018907 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) ==== 93+0+10 (secure 0 0 0) 0x7f590805a7d0 con 0x7f59101450c0 2024-04-20T21:27:58.438 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:27:58.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.441+0000 7f590f200640 1 -- 172.21.15.86:0/2011018907 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f58ec06c8f0 msgr2=0x7f58ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:58.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.441+0000 7f590f200640 1 --2- 172.21.15.86:0/2011018907 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f58ec06c8f0 0x7f58ec06edb0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f5900036000 tx=0x7f5900008030 comp rx=0 tx=0).stop 2024-04-20T21:27:58.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.441+0000 7f590f200640 1 -- 172.21.15.86:0/2011018907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59101450c0 msgr2=0x7f59100b1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:58.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.441+0000 7f590f200640 1 --2- 172.21.15.86:0/2011018907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59101450c0 0x7f59100b1230 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f5908007e90 tx=0x7f590800e4b0 comp rx=0 tx=0).stop 2024-04-20T21:27:58.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.442+0000 7f590f200640 1 -- 172.21.15.86:0/2011018907 shutdown_connections 2024-04-20T21:27:58.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.442+0000 7f590f200640 1 --2- 172.21.15.86:0/2011018907 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f58ec06c8f0 0x7f58ec06edb0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:58.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.442+0000 7f590f200640 1 --2- 172.21.15.86:0/2011018907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59101450c0 0x7f59100b1230 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:58.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.442+0000 7f590f200640 1 --2- 172.21.15.86:0/2011018907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5910144ce0 0x7f59100b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:58.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.442+0000 7f590f200640 1 --2- 172.21.15.86:0/2011018907 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f59100b5d50 0x7f591009df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:58.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.442+0000 7f590f200640 1 -- 172.21.15.86:0/2011018907 >> 172.21.15.86:0/2011018907 conn(0x7f59100b6810 msgr2=0x7f5910143e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:58.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.442+0000 7f590f200640 1 -- 172.21.15.86:0/2011018907 shutdown_connections 2024-04-20T21:27:58.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.442+0000 7f590f200640 1 -- 172.21.15.86:0/2011018907 wait complete. 2024-04-20T21:27:58.460 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:27:58.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.663+0000 7ff2b2e00640 1 Processor -- start 2024-04-20T21:27:58.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.664+0000 7ff2b2e00640 1 -- start start 2024-04-20T21:27:58.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.665+0000 7ff2b2e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff2ac15c890 0x7ff2ac0ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:58.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.665+0000 7ff2b2e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2ac0cd090 0x7ff2ac09a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:58.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.665+0000 7ff2b2e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2ac0d0280 con 0x7ff2ac09af20 2024-04-20T21:27:58.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.665+0000 7ff2b2e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff2ac15cc70 con 0x7ff2ac15c890 2024-04-20T21:27:58.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.665+0000 7ff2b2e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff2ac0cd4f0 con 0x7ff2ac0cd090 2024-04-20T21:27:58.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.665+0000 7ff2b0e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7ff2ac09af20 0x7ff2ac0b76a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40684/0 (socket says 172.21.15.86:40684) 2024-04-20T21:27:58.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.666+0000 7ff2b0e00640 1 -- 172.21.15.86:0/3165837015 learned_addr learned my addr 172.21.15.86:0/3165837015 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:27:58.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.665+0000 7ff2abe00640 1 --2- 172.21.15.86:0/3165837015 >> v2:172.21.15.86:3300/0 conn(0x7ff2ac15c890 0x7ff2ac0ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:58.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.666+0000 7ff2ab400640 1 --2- 172.21.15.86:0/3165837015 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2ac0cd090 0x7ff2ac09a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:58.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.666+0000 7ff2abe00640 1 -- 172.21.15.86:0/3165837015 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2ac0cd090 msgr2=0x7ff2ac09a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:58.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.666+0000 7ff2abe00640 1 --2- 172.21.15.86:0/3165837015 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2ac0cd090 0x7ff2ac09a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:58.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.666+0000 7ff2abe00640 1 -- 172.21.15.86:0/3165837015 >> v1:172.21.15.86:6789/0 conn(0x7ff2ac09af20 legacy=0x7ff2ac0b76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:58.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.666+0000 7ff2abe00640 1 -- 172.21.15.86:0/3165837015 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2ac0d0560 con 0x7ff2ac15c890 2024-04-20T21:27:58.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.667+0000 7ff2abe00640 1 --2- 172.21.15.86:0/3165837015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2ac15c890 0x7ff2ac0ce080 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7ff2940097a0 tx=0x7ff29402b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54d909b216b44115 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:58.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.667+0000 7ff2aaa00640 1 -- 172.21.15.86:0/3165837015 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff294031780 con 0x7ff2ac15c890 2024-04-20T21:27:58.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.668+0000 7ff2aaa00640 1 -- 172.21.15.86:0/3165837015 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff294037070 con 0x7ff2ac15c890 2024-04-20T21:27:58.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.668+0000 7ff2aaa00640 1 -- 172.21.15.86:0/3165837015 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff294031b40 con 0x7ff2ac15c890 2024-04-20T21:27:58.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.668+0000 7ff2b2e00640 1 -- 172.21.15.86:0/3165837015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2ac15c890 msgr2=0x7ff2ac0ce080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:58.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.668+0000 7ff2b2e00640 1 --2- 172.21.15.86:0/3165837015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2ac15c890 0x7ff2ac0ce080 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7ff2940097a0 tx=0x7ff29402b820 comp rx=0 tx=0).stop 2024-04-20T21:27:58.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.668+0000 7ff2b2e00640 1 -- 172.21.15.86:0/3165837015 shutdown_connections 2024-04-20T21:27:58.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.668+0000 7ff2b2e00640 1 --2- 172.21.15.86:0/3165837015 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2ac0cd090 0x7ff2ac09a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:58.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.668+0000 7ff2b2e00640 1 --2- 172.21.15.86:0/3165837015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2ac15c890 0x7ff2ac0ce080 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:58.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.668+0000 7ff2b2e00640 1 -- 172.21.15.86:0/3165837015 >> 172.21.15.86:0/3165837015 conn(0x7ff2ac0b4070 msgr2=0x7ff2ac0b4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:58.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.668+0000 7ff2b2e00640 1 -- 172.21.15.86:0/3165837015 shutdown_connections 2024-04-20T21:27:58.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.668+0000 7ff2b2e00640 1 -- 172.21.15.86:0/3165837015 wait complete. 2024-04-20T21:27:58.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.670+0000 7ff2b2e00640 1 Processor -- start 2024-04-20T21:27:58.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.670+0000 7ff2b2e00640 1 -- start start 2024-04-20T21:27:58.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.671+0000 7ff2b2e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2ac09af20 0x7ff2ac15cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:58.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.671+0000 7ff2b2e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2ac0cd090 0x7ff2ac1545e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:58.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.671+0000 7ff2b2e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2ac15c890 0x7ff2ac153940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:58.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.671+0000 7ff2b2e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ac09b330 con 0x7ff2ac09af20 2024-04-20T21:27:58.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.671+0000 7ff2b2e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ac0d0e90 con 0x7ff2ac0cd090 2024-04-20T21:27:58.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.671+0000 7ff2b2e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff2ac0d0b20 con 0x7ff2ac15c890 2024-04-20T21:27:58.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.671+0000 7ff2abe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2ac09af20 0x7ff2ac15cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:58.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.672+0000 7ff2abe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2ac09af20 0x7ff2ac15cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52972/0 (socket says 172.21.15.86:52972) 2024-04-20T21:27:58.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.672+0000 7ff2abe00640 1 -- 172.21.15.86:0/2582145744 learned_addr learned my addr 172.21.15.86:0/2582145744 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:58.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.672+0000 7ff2ab400640 1 --2- 172.21.15.86:0/2582145744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2ac0cd090 0x7ff2ac1545e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:58.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.672+0000 7ff2b0e00640 1 --2- 172.21.15.86:0/2582145744 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2ac15c890 0x7ff2ac153940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:58.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.672+0000 7ff2abe00640 1 -- 172.21.15.86:0/2582145744 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2ac15c890 msgr2=0x7ff2ac153940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:58.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.672+0000 7ff2abe00640 1 --2- 172.21.15.86:0/2582145744 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2ac15c890 0x7ff2ac153940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:58.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.672+0000 7ff2abe00640 1 -- 172.21.15.86:0/2582145744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2ac0cd090 msgr2=0x7ff2ac1545e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:58.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.672+0000 7ff2abe00640 1 --2- 172.21.15.86:0/2582145744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2ac0cd090 0x7ff2ac1545e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:58.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.672+0000 7ff2abe00640 1 -- 172.21.15.86:0/2582145744 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2ac160fa0 con 0x7ff2ac09af20 2024-04-20T21:27:58.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.672+0000 7ff2ab400640 1 --2- 172.21.15.86:0/2582145744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2ac0cd090 0x7ff2ac1545e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:27:58.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.672+0000 7ff2abe00640 1 --2- 172.21.15.86:0/2582145744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2ac09af20 0x7ff2ac15cdf0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7ff294034040 tx=0x7ff2940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:58.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.673+0000 7ff2a3e00640 1 -- 172.21.15.86:0/2582145744 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff294032610 con 0x7ff2ac09af20 2024-04-20T21:27:58.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.673+0000 7ff2a3e00640 1 -- 172.21.15.86:0/2582145744 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2940327b0 con 0x7ff2ac09af20 2024-04-20T21:27:58.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.673+0000 7ff2a3e00640 1 -- 172.21.15.86:0/2582145744 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff294032ba0 con 0x7ff2ac09af20 2024-04-20T21:27:58.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.673+0000 7ff2b2e00640 1 -- 172.21.15.86:0/2582145744 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2ac0c88d0 con 0x7ff2ac09af20 2024-04-20T21:27:58.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.673+0000 7ff2b2e00640 1 -- 172.21.15.86:0/2582145744 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2ac160c30 con 0x7ff2ac09af20 2024-04-20T21:27:58.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.674+0000 7ff2b2e00640 1 -- 172.21.15.86:0/2582145744 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2ac0c8f00 con 0x7ff2ac09af20 2024-04-20T21:27:58.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.675+0000 7ff2a3e00640 1 -- 172.21.15.86:0/2582145744 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff29404c060 con 0x7ff2ac09af20 2024-04-20T21:27:58.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.676+0000 7ff2a3e00640 1 --2- 172.21.15.86:0/2582145744 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff28806c8f0 0x7ff28806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:58.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.676+0000 7ff2a3e00640 1 -- 172.21.15.86:0/2582145744 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7ff2940ad190 con 0x7ff2ac09af20 2024-04-20T21:27:58.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.676+0000 7ff2ab400640 1 --2- 172.21.15.86:0/2582145744 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff28806c8f0 0x7ff28806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:58.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.677+0000 7ff2ab400640 1 --2- 172.21.15.86:0/2582145744 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff28806c8f0 0x7ff28806edb0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac15be10 tx=0x7ff29c00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:58.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.680+0000 7ff2a3e00640 1 -- 172.21.15.86:0/2582145744 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff2ac0c8f00 con 0x7ff2ac09af20 2024-04-20T21:27:58.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.886+0000 7ff2b2e00640 1 -- 172.21.15.86:0/2582145744 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff2ac0689e0 con 0x7ff2ac09af20 2024-04-20T21:27:58.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.886+0000 7ff2a3e00640 1 -- 172.21.15.86:0/2582145744 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v100) ==== 104+0+11 (secure 0 0 0) 0x7ff294040070 con 0x7ff2ac09af20 2024-04-20T21:27:58.885 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-20T21:27:58.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.889+0000 7ff2b2e00640 1 -- 172.21.15.86:0/2582145744 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff28806c8f0 msgr2=0x7ff28806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:58.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.889+0000 7ff2b2e00640 1 --2- 172.21.15.86:0/2582145744 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff28806c8f0 0x7ff28806edb0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac15be10 tx=0x7ff29c00b040 comp rx=0 tx=0).stop 2024-04-20T21:27:58.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.889+0000 7ff2b2e00640 1 -- 172.21.15.86:0/2582145744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2ac09af20 msgr2=0x7ff2ac15cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:58.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.889+0000 7ff2b2e00640 1 --2- 172.21.15.86:0/2582145744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2ac09af20 0x7ff2ac15cdf0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7ff294034040 tx=0x7ff2940046c0 comp rx=0 tx=0).stop 2024-04-20T21:27:58.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.889+0000 7ff2b2e00640 1 -- 172.21.15.86:0/2582145744 shutdown_connections 2024-04-20T21:27:58.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.889+0000 7ff2b2e00640 1 --2- 172.21.15.86:0/2582145744 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff28806c8f0 0x7ff28806edb0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:58.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.889+0000 7ff2b2e00640 1 --2- 172.21.15.86:0/2582145744 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff2ac15c890 0x7ff2ac153940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:58.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.889+0000 7ff2b2e00640 1 --2- 172.21.15.86:0/2582145744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2ac0cd090 0x7ff2ac1545e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:58.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.889+0000 7ff2b2e00640 1 --2- 172.21.15.86:0/2582145744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff2ac09af20 0x7ff2ac15cdf0 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:58.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.889+0000 7ff2b2e00640 1 -- 172.21.15.86:0/2582145744 >> 172.21.15.86:0/2582145744 conn(0x7ff2ac0b4070 msgr2=0x7ff2ac0b76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:58.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.889+0000 7ff2b2e00640 1 -- 172.21.15.86:0/2582145744 shutdown_connections 2024-04-20T21:27:58.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:58.889+0000 7ff2b2e00640 1 -- 172.21.15.86:0/2582145744 wait complete. 2024-04-20T21:27:58.906 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:27:59.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.108+0000 7fad80a00640 1 Processor -- start 2024-04-20T21:27:59.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.109+0000 7fad80a00640 1 -- start start 2024-04-20T21:27:59.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.110+0000 7fad80a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad7c155d30 0x7fad7c0cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:59.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.110+0000 7fad80a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fad7c06b980 0x7fad7c06be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:59.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.110+0000 7fad80a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fad7c0b4ba0 0x7fad7c0b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:59.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.110+0000 7fad80a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fad7c156110 con 0x7fad7c06b980 2024-04-20T21:27:59.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.110+0000 7fad80a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fad7c0ce460 con 0x7fad7c0b4ba0 2024-04-20T21:27:59.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.110+0000 7fad80a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fad7c0ce740 con 0x7fad7c155d30 2024-04-20T21:27:59.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.110+0000 7fad79400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fad7c06b980 0x7fad7c06be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:59.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.110+0000 7fad79400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fad7c06b980 0x7fad7c06be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52976/0 (socket says 172.21.15.86:52976) 2024-04-20T21:27:59.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.110+0000 7fad79400640 1 -- 172.21.15.86:0/1174239601 learned_addr learned my addr 172.21.15.86:0/1174239601 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:59.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.111+0000 7fad7a800640 1 --2- 172.21.15.86:0/1174239601 >> v2:172.21.15.144:3300/0 conn(0x7fad7c0b4ba0 0x7fad7c0b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:59.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.111+0000 7fad79e00640 1 --2- 172.21.15.86:0/1174239601 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad7c155d30 0x7fad7c0cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:59.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.111+0000 7fad79400640 1 -- 172.21.15.86:0/1174239601 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad7c155d30 msgr2=0x7fad7c0cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:59.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.111+0000 7fad79400640 1 --2- 172.21.15.86:0/1174239601 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad7c155d30 0x7fad7c0cf500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:59.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.111+0000 7fad79400640 1 -- 172.21.15.86:0/1174239601 >> v2:172.21.15.144:3300/0 conn(0x7fad7c0b4ba0 msgr2=0x7fad7c0b4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:59.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.111+0000 7fad79400640 1 --2- 172.21.15.86:0/1174239601 >> v2:172.21.15.144:3300/0 conn(0x7fad7c0b4ba0 0x7fad7c0b4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:59.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.111+0000 7fad79400640 1 -- 172.21.15.86:0/1174239601 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad7c0cea20 con 0x7fad7c06b980 2024-04-20T21:27:59.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.112+0000 7fad79400640 1 --2- 172.21.15.86:0/1174239601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad7c06b980 0x7fad7c06be00 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fad7c06ca90 tx=0x7fad6802bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=774d8257261f4a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:59.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.112+0000 7fad78a00640 1 -- 172.21.15.86:0/1174239601 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6802c950 con 0x7fad7c06b980 2024-04-20T21:27:59.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.113+0000 7fad78a00640 1 -- 172.21.15.86:0/1174239601 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad6802caf0 con 0x7fad7c06b980 2024-04-20T21:27:59.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.113+0000 7fad78a00640 1 -- 172.21.15.86:0/1174239601 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad680343f0 con 0x7fad7c06b980 2024-04-20T21:27:59.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.113+0000 7fad80a00640 1 -- 172.21.15.86:0/1174239601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad7c06b980 msgr2=0x7fad7c06be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:59.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.113+0000 7fad80a00640 1 --2- 172.21.15.86:0/1174239601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad7c06b980 0x7fad7c06be00 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fad7c06ca90 tx=0x7fad6802bdb0 comp rx=0 tx=0).stop 2024-04-20T21:27:59.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.113+0000 7fad80a00640 1 -- 172.21.15.86:0/1174239601 shutdown_connections 2024-04-20T21:27:59.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.113+0000 7fad80a00640 1 --2- 172.21.15.86:0/1174239601 >> v2:172.21.15.144:3300/0 conn(0x7fad7c0b4ba0 0x7fad7c0b4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:59.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.113+0000 7fad80a00640 1 --2- 172.21.15.86:0/1174239601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad7c06b980 0x7fad7c06be00 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:59.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.113+0000 7fad80a00640 1 --2- 172.21.15.86:0/1174239601 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad7c155d30 0x7fad7c0cf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:59.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.113+0000 7fad80a00640 1 -- 172.21.15.86:0/1174239601 >> 172.21.15.86:0/1174239601 conn(0x7fad7c0b2110 msgr2=0x7fad7c0b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:59.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.113+0000 7fad80a00640 1 -- 172.21.15.86:0/1174239601 shutdown_connections 2024-04-20T21:27:59.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.113+0000 7fad80a00640 1 -- 172.21.15.86:0/1174239601 wait complete. 2024-04-20T21:27:59.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.115+0000 7fad80a00640 1 Processor -- start 2024-04-20T21:27:59.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.115+0000 7fad80a00640 1 -- start start 2024-04-20T21:27:59.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.116+0000 7fad80a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad7c06b980 0x7fad7c144a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:59.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.116+0000 7fad80a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad7c0b4ba0 0x7fad7c072020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:59.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.116+0000 7fad80a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad7c155d30 0x7fad7c146b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:59.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.116+0000 7fad80a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fad7c0b5510 con 0x7fad7c06b980 2024-04-20T21:27:59.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.116+0000 7fad80a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fad7c0cefe0 con 0x7fad7c0b4ba0 2024-04-20T21:27:59.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.116+0000 7fad80a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fad7c0b5690 con 0x7fad7c155d30 2024-04-20T21:27:59.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.116+0000 7fad79e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad7c06b980 0x7fad7c144a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:59.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.116+0000 7fad79e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad7c06b980 0x7fad7c144a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52992/0 (socket says 172.21.15.86:52992) 2024-04-20T21:27:59.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.116+0000 7fad79e00640 1 -- 172.21.15.86:0/3155292898 learned_addr learned my addr 172.21.15.86:0/3155292898 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:59.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.117+0000 7fad79400640 1 --2- 172.21.15.86:0/3155292898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad7c0b4ba0 0x7fad7c072020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:59.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.117+0000 7fad7a800640 1 --2- 172.21.15.86:0/3155292898 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad7c155d30 0x7fad7c146b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:59.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.117+0000 7fad79e00640 1 -- 172.21.15.86:0/3155292898 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad7c155d30 msgr2=0x7fad7c146b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:59.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.117+0000 7fad79e00640 1 --2- 172.21.15.86:0/3155292898 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad7c155d30 0x7fad7c146b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:59.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.117+0000 7fad79e00640 1 -- 172.21.15.86:0/3155292898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad7c0b4ba0 msgr2=0x7fad7c072020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:59.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.117+0000 7fad79e00640 1 --2- 172.21.15.86:0/3155292898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad7c0b4ba0 0x7fad7c072020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:59.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.117+0000 7fad79e00640 1 -- 172.21.15.86:0/3155292898 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad7c146370 con 0x7fad7c06b980 2024-04-20T21:27:59.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.117+0000 7fad79e00640 1 --2- 172.21.15.86:0/3155292898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad7c06b980 0x7fad7c144a80 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00d800 tx=0x7fad6c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:59.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.117+0000 7fad72000640 1 -- 172.21.15.86:0/3155292898 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c00b6c0 con 0x7fad7c06b980 2024-04-20T21:27:59.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.117+0000 7fad72000640 1 -- 172.21.15.86:0/3155292898 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad6c019070 con 0x7fad7c06b980 2024-04-20T21:27:59.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.117+0000 7fad72000640 1 -- 172.21.15.86:0/3155292898 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c00ba80 con 0x7fad7c06b980 2024-04-20T21:27:59.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.117+0000 7fad80a00640 1 -- 172.21.15.86:0/3155292898 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad7c1495f0 con 0x7fad7c06b980 2024-04-20T21:27:59.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.118+0000 7fad80a00640 1 -- 172.21.15.86:0/3155292898 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad7c149bb0 con 0x7fad7c06b980 2024-04-20T21:27:59.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.119+0000 7fad80a00640 1 -- 172.21.15.86:0/3155292898 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad7c149df0 con 0x7fad7c06b980 2024-04-20T21:27:59.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.120+0000 7fad72000640 1 -- 172.21.15.86:0/3155292898 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fad6c00bc20 con 0x7fad7c06b980 2024-04-20T21:27:59.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.121+0000 7fad72000640 1 --2- 172.21.15.86:0/3155292898 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fad5c06c8f0 0x7fad5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:59.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.121+0000 7fad72000640 1 -- 172.21.15.86:0/3155292898 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7fad6c08c0e0 con 0x7fad7c06b980 2024-04-20T21:27:59.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.121+0000 7fad79400640 1 --2- 172.21.15.86:0/3155292898 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fad5c06c8f0 0x7fad5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:59.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.122+0000 7fad79400640 1 --2- 172.21.15.86:0/3155292898 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fad5c06c8f0 0x7fad5c06edb0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fad68035000 tx=0x7fad68030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:59.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.125+0000 7fad72000640 1 -- 172.21.15.86:0/3155292898 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fad7c149df0 con 0x7fad7c06b980 2024-04-20T21:27:59.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.326+0000 7fad80a00640 1 -- 172.21.15.86:0/3155292898 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fad7c0689e0 con 0x7fad7c06b980 2024-04-20T21:27:59.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.327+0000 7fad72000640 1 -- 172.21.15.86:0/3155292898 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v100) ==== 100+0+11 (secure 0 0 0) 0x7fad6c01c360 con 0x7fad7c06b980 2024-04-20T21:27:59.326 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-20T21:27:59.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.329+0000 7fad80a00640 1 -- 172.21.15.86:0/3155292898 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fad5c06c8f0 msgr2=0x7fad5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:59.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.329+0000 7fad80a00640 1 --2- 172.21.15.86:0/3155292898 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fad5c06c8f0 0x7fad5c06edb0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fad68035000 tx=0x7fad68030000 comp rx=0 tx=0).stop 2024-04-20T21:27:59.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.329+0000 7fad80a00640 1 -- 172.21.15.86:0/3155292898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad7c06b980 msgr2=0x7fad7c144a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:59.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.329+0000 7fad80a00640 1 --2- 172.21.15.86:0/3155292898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad7c06b980 0x7fad7c144a80 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00d800 tx=0x7fad6c00dcc0 comp rx=0 tx=0).stop 2024-04-20T21:27:59.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.329+0000 7fad80a00640 1 -- 172.21.15.86:0/3155292898 shutdown_connections 2024-04-20T21:27:59.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.329+0000 7fad80a00640 1 --2- 172.21.15.86:0/3155292898 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fad5c06c8f0 0x7fad5c06edb0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:59.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.329+0000 7fad80a00640 1 --2- 172.21.15.86:0/3155292898 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad7c155d30 0x7fad7c146b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:59.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.329+0000 7fad80a00640 1 --2- 172.21.15.86:0/3155292898 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad7c0b4ba0 0x7fad7c072020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:59.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.329+0000 7fad80a00640 1 --2- 172.21.15.86:0/3155292898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad7c06b980 0x7fad7c144a80 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:59.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.329+0000 7fad80a00640 1 -- 172.21.15.86:0/3155292898 >> 172.21.15.86:0/3155292898 conn(0x7fad7c0b2110 msgr2=0x7fad7c06f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:59.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.329+0000 7fad80a00640 1 -- 172.21.15.86:0/3155292898 shutdown_connections 2024-04-20T21:27:59.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.330+0000 7fad80a00640 1 -- 172.21.15.86:0/3155292898 wait complete. 2024-04-20T21:27:59.349 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T21:27:59.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.541+0000 7fa5af000640 1 Processor -- start 2024-04-20T21:27:59.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.542+0000 7fa5af000640 1 -- start start 2024-04-20T21:27:59.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.543+0000 7fa5af000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa5a80d09e0 0x7fa5a80cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:59.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.543+0000 7fa5af000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5a80cda20 con 0x7fa5a8145fe0 2024-04-20T21:27:59.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.543+0000 7fa5af000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5a815dcc0 con 0x7fa5a80cd610 2024-04-20T21:27:59.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.543+0000 7fa5af000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fa5a80d00d0 con 0x7fa5a80d09e0 2024-04-20T21:27:59.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.544+0000 7fa5a7e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fa5a8145fe0 0x7fa5a80d02d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40690/0 (socket says 172.21.15.86:40690) 2024-04-20T21:27:59.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.544+0000 7fa5a7e00640 1 -- 172.21.15.86:0/2643187742 learned_addr learned my addr 172.21.15.86:0/2643187742 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:27:59.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.544+0000 7fa5a7400640 1 --2- 172.21.15.86:0/2643187742 >> v2:172.21.15.144:3300/0 conn(0x7fa5a80d09e0 0x7fa5a80cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:59.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.544+0000 7fa5a6a00640 1 -- 172.21.15.86:0/2643187742 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1937788605 0 0) 0x7fa5a80cda20 con 0x7fa5a8145fe0 2024-04-20T21:27:59.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.544+0000 7fa5a6a00640 1 -- 172.21.15.86:0/2643187742 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa58c003520 con 0x7fa5a8145fe0 2024-04-20T21:27:59.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.545+0000 7fa5a6a00640 1 -- 172.21.15.86:0/2643187742 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2456208032 0 0) 0x7fa58c003520 con 0x7fa5a8145fe0 2024-04-20T21:27:59.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.545+0000 7fa5a6a00640 1 -- 172.21.15.86:0/2643187742 >> v2:172.21.15.144:3300/0 conn(0x7fa5a80d09e0 msgr2=0x7fa5a80cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:59.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.545+0000 7fa5a6a00640 1 --2- 172.21.15.86:0/2643187742 >> v2:172.21.15.144:3300/0 conn(0x7fa5a80d09e0 0x7fa5a80cd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:59.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.545+0000 7fa5a6a00640 1 -- 172.21.15.86:0/2643187742 >> v1:172.21.15.144:6789/0 conn(0x7fa5a80cd610 legacy=0x7fa5a8071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:59.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.545+0000 7fa5a6a00640 1 -- 172.21.15.86:0/2643187742 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a815cf00 con 0x7fa5a8145fe0 2024-04-20T21:27:59.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.545+0000 7fa5a6a00640 1 -- 172.21.15.86:0/2643187742 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fa598001670 con 0x7fa5a8145fe0 2024-04-20T21:27:59.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.545+0000 7fa5a6a00640 1 -- 172.21.15.86:0/2643187742 >> v1:172.21.15.86:6789/0 conn(0x7fa5a8145fe0 legacy=0x7fa5a80d02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:59.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.545+0000 7fa5a6a00640 1 --2- 172.21.15.86:0/2643187742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa58c0039b0 0x7fa58c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:59.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.545+0000 7fa5a6a00640 1 --2- 172.21.15.86:0/2643187742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa58c004800 0x7fa58c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:59.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.545+0000 7fa5a6a00640 1 --2- 172.21.15.86:0/2643187742 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa58c0051c0 0x7fa58c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:59.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.545+0000 7fa5a6a00640 1 -- 172.21.15.86:0/2643187742 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a80d00d0 con 0x7fa58c004800 2024-04-20T21:27:59.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.545+0000 7fa5a6a00640 1 -- 172.21.15.86:0/2643187742 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa58c00c060 con 0x7fa58c0039b0 2024-04-20T21:27:59.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.545+0000 7fa5a6a00640 1 -- 172.21.15.86:0/2643187742 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa58c00c260 con 0x7fa58c0051c0 2024-04-20T21:27:59.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.545+0000 7fa5a7400640 1 --2- 172.21.15.86:0/2643187742 >> v2:172.21.15.144:3300/0 conn(0x7fa5a80d09e0 0x7fa5a80cd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:27:59.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.545+0000 7fa5a7400640 1 --2- 172.21.15.86:0/2643187742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa58c004800 0x7fa58c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:59.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.546+0000 7fa5a7e00640 1 --2- 172.21.15.86:0/2643187742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa58c0039b0 0x7fa58c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:59.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.546+0000 7fa5ad000640 1 --2- 172.21.15.86:0/2643187742 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa58c0051c0 0x7fa58c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:59.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.546+0000 7fa5a7400640 1 -- 172.21.15.86:0/2643187742 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa58c0051c0 msgr2=0x7fa58c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:59.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.546+0000 7fa5a7400640 1 --2- 172.21.15.86:0/2643187742 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa58c0051c0 0x7fa58c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:59.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.546+0000 7fa5a7400640 1 -- 172.21.15.86:0/2643187742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa58c0039b0 msgr2=0x7fa58c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:27:59.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.546+0000 7fa5a7400640 1 --2- 172.21.15.86:0/2643187742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa58c0039b0 0x7fa58c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:59.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.546+0000 7fa5a7400640 1 -- 172.21.15.86:0/2643187742 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa58c003520 con 0x7fa58c004800 2024-04-20T21:27:59.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.546+0000 7fa5a7e00640 1 --2- 172.21.15.86:0/2643187742 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa58c0039b0 0x7fa58c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:27:59.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.546+0000 7fa5ad000640 1 -- 172.21.15.86:0/2643187742 reap_dead start 2024-04-20T21:27:59.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.547+0000 7fa5a7400640 1 --2- 172.21.15.86:0/2643187742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa58c004800 0x7fa58c004c80 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fa594009c70 tx=0x7fa58c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10642c4bffa0046f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:59.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.547+0000 7fa5a6a00640 1 -- 172.21.15.86:0/2643187742 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa594009840 con 0x7fa58c004800 2024-04-20T21:27:59.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.547+0000 7fa5a6a00640 1 -- 172.21.15.86:0/2643187742 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa594004020 con 0x7fa58c004800 2024-04-20T21:27:59.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.547+0000 7fa5a6a00640 1 -- 172.21.15.86:0/2643187742 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa594004450 con 0x7fa58c004800 2024-04-20T21:27:59.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.548+0000 7fa5af000640 1 -- 172.21.15.86:0/2643187742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa58c004800 msgr2=0x7fa58c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:59.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.548+0000 7fa5af000640 1 --2- 172.21.15.86:0/2643187742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa58c004800 0x7fa58c004c80 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fa594009c70 tx=0x7fa58c005830 comp rx=0 tx=0).stop 2024-04-20T21:27:59.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.548+0000 7fa5af000640 1 -- 172.21.15.86:0/2643187742 shutdown_connections 2024-04-20T21:27:59.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.548+0000 7fa5af000640 1 --2- 172.21.15.86:0/2643187742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa58c004800 0x7fa58c004c80 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:59.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.548+0000 7fa5af000640 1 -- 172.21.15.86:0/2643187742 >> 172.21.15.86:0/2643187742 conn(0x7fa5a806b980 msgr2=0x7fa5a806bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:59.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.548+0000 7fa5af000640 1 -- 172.21.15.86:0/2643187742 shutdown_connections 2024-04-20T21:27:59.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.548+0000 7fa5af000640 1 -- 172.21.15.86:0/2643187742 wait complete. 2024-04-20T21:27:59.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.550+0000 7fa5af000640 1 Processor -- start 2024-04-20T21:27:59.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.550+0000 7fa5af000640 1 -- start start 2024-04-20T21:27:59.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.551+0000 7fa5af000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa58c004800 0x7fa5a80b1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:59.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.551+0000 7fa5af000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5a80c5460 0x7fa5a815c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:59.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.551+0000 7fa5af000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5a80b28e0 0x7fa5a80b2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:59.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.551+0000 7fa5af000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a815d7a0 con 0x7fa58c004800 2024-04-20T21:27:59.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.551+0000 7fa5af000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a815d4c0 con 0x7fa5a80b28e0 2024-04-20T21:27:59.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.551+0000 7fa5af000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa5a81463c0 con 0x7fa5a80c5460 2024-04-20T21:27:59.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.551+0000 7fa5a7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa58c004800 0x7fa5a80b1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:59.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.551+0000 7fa5a7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa58c004800 0x7fa5a80b1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53016/0 (socket says 172.21.15.86:53016) 2024-04-20T21:27:59.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.551+0000 7fa5a7e00640 1 -- 172.21.15.86:0/271946829 learned_addr learned my addr 172.21.15.86:0/271946829 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:27:59.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.552+0000 7fa5ad000640 1 --2- 172.21.15.86:0/271946829 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5a80b28e0 0x7fa5a80b2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:59.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.552+0000 7fa5a7400640 1 --2- 172.21.15.86:0/271946829 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5a80c5460 0x7fa5a815c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:59.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.552+0000 7fa5a7e00640 1 -- 172.21.15.86:0/271946829 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5a80c5460 msgr2=0x7fa5a815c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:59.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.552+0000 7fa5a7e00640 1 --2- 172.21.15.86:0/271946829 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5a80c5460 0x7fa5a815c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:59.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.552+0000 7fa5a7e00640 1 -- 172.21.15.86:0/271946829 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5a80b28e0 msgr2=0x7fa5a80b2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:59.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.552+0000 7fa5a7e00640 1 --2- 172.21.15.86:0/271946829 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5a80b28e0 0x7fa5a80b2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:59.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.552+0000 7fa5a7e00640 1 -- 172.21.15.86:0/271946829 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a80b1840 con 0x7fa58c004800 2024-04-20T21:27:59.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.552+0000 7fa5a7e00640 1 --2- 172.21.15.86:0/271946829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa58c004800 0x7fa5a80b1260 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7fa598001eb0 tx=0x7fa59800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:59.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.552+0000 7fa59fe00640 1 -- 172.21.15.86:0/271946829 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59800c6c0 con 0x7fa58c004800 2024-04-20T21:27:59.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.553+0000 7fa59fe00640 1 -- 172.21.15.86:0/271946829 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa598018070 con 0x7fa58c004800 2024-04-20T21:27:59.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.553+0000 7fa5af000640 1 -- 172.21.15.86:0/271946829 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5a80b1b20 con 0x7fa58c004800 2024-04-20T21:27:59.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.553+0000 7fa59fe00640 1 -- 172.21.15.86:0/271946829 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59800caa0 con 0x7fa58c004800 2024-04-20T21:27:59.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.553+0000 7fa5af000640 1 -- 172.21.15.86:0/271946829 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5a80b20e0 con 0x7fa58c004800 2024-04-20T21:27:59.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.554+0000 7fa5af000640 1 -- 172.21.15.86:0/271946829 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5a80689e0 con 0x7fa58c004800 2024-04-20T21:27:59.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.555+0000 7fa59fe00640 1 -- 172.21.15.86:0/271946829 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa59800cc40 con 0x7fa58c004800 2024-04-20T21:27:59.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.556+0000 7fa59fe00640 1 --2- 172.21.15.86:0/271946829 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa58806c8f0 0x7fa58806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:27:59.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.556+0000 7fa59fe00640 1 -- 172.21.15.86:0/271946829 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7fa59808b8d0 con 0x7fa58c004800 2024-04-20T21:27:59.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.556+0000 7fa5a7400640 1 --2- 172.21.15.86:0/271946829 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa58806c8f0 0x7fa58806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:27:59.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.557+0000 7fa5a7400640 1 --2- 172.21.15.86:0/271946829 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa58806c8f0 0x7fa58806edb0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fa594035000 tx=0x7fa594030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:27:59.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.561+0000 7fa59fe00640 1 -- 172.21.15.86:0/271946829 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa598059cf0 con 0x7fa58c004800 2024-04-20T21:27:59.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.773+0000 7fa5af000640 1 -- 172.21.15.86:0/271946829 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa5a815de90 con 0x7fa58c004800 2024-04-20T21:27:59.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.774+0000 7fa59fe00640 1 -- 172.21.15.86:0/271946829 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v100) ==== 80+0+0 (secure 0 0 0) 0x7fa5a815de90 con 0x7fa58c004800 2024-04-20T21:27:59.773 INFO:teuthology.orchestra.run.smithi086.stderr:listed 0 entries 2024-04-20T21:27:59.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.777+0000 7fa5af000640 1 -- 172.21.15.86:0/271946829 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa58806c8f0 msgr2=0x7fa58806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:59.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.777+0000 7fa5af000640 1 --2- 172.21.15.86:0/271946829 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa58806c8f0 0x7fa58806edb0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fa594035000 tx=0x7fa594030000 comp rx=0 tx=0).stop 2024-04-20T21:27:59.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.777+0000 7fa5af000640 1 -- 172.21.15.86:0/271946829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa58c004800 msgr2=0x7fa5a80b1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:27:59.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.777+0000 7fa5af000640 1 --2- 172.21.15.86:0/271946829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa58c004800 0x7fa5a80b1260 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7fa598001eb0 tx=0x7fa59800eb90 comp rx=0 tx=0).stop 2024-04-20T21:27:59.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.777+0000 7fa5af000640 1 -- 172.21.15.86:0/271946829 shutdown_connections 2024-04-20T21:27:59.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.777+0000 7fa5af000640 1 --2- 172.21.15.86:0/271946829 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa58806c8f0 0x7fa58806edb0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:59.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.777+0000 7fa5af000640 1 --2- 172.21.15.86:0/271946829 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5a80b28e0 0x7fa5a80b2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:59.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.777+0000 7fa5af000640 1 --2- 172.21.15.86:0/271946829 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5a80c5460 0x7fa5a815c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:59.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.777+0000 7fa5af000640 1 --2- 172.21.15.86:0/271946829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa58c004800 0x7fa5a80b1260 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:27:59.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.777+0000 7fa5af000640 1 -- 172.21.15.86:0/271946829 >> 172.21.15.86:0/271946829 conn(0x7fa5a806b980 msgr2=0x7fa5a80d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:27:59.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.777+0000 7fa5af000640 1 -- 172.21.15.86:0/271946829 shutdown_connections 2024-04-20T21:27:59.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:27:59.777+0000 7fa5af000640 1 -- 172.21.15.86:0/271946829 wait complete. 2024-04-20T21:27:59.794 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi086.front.sepia.ceph.com... 2024-04-20T21:27:59.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:27:59.794 DEBUG:teuthology.orchestra.run.smithi086:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-20T21:27:59.868 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[59507]: fuse finished with error 0 and tester_r 0 2024-04-20T21:27:59.874 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:27:59.912 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-20T21:27:59.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:27:59.912 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-20T21:28:00.010 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:28:00.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.209+0000 7fb027200640 1 Processor -- start 2024-04-20T21:28:00.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.209+0000 7fb027200640 1 -- start start 2024-04-20T21:28:00.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.210+0000 7fb027200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb0280d50c0 0x7fb0280d3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:00.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.210+0000 7fb027200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0280d3950 con 0x7fb0280cf9c0 2024-04-20T21:28:00.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.210+0000 7fb027200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0280cec00 con 0x7fb02815ece0 2024-04-20T21:28:00.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.210+0000 7fb027200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb0280ceee0 con 0x7fb0280d50c0 2024-04-20T21:28:00.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.211+0000 7fb025400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb0280d50c0 0x7fb0280d3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:00.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.211+0000 7fb025400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb0280d50c0 0x7fb0280d3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53020/0 (socket says 172.21.15.86:53020) 2024-04-20T21:28:00.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.211+0000 7fb025400640 1 -- 172.21.15.86:0/2473996087 learned_addr learned my addr 172.21.15.86:0/2473996087 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:00.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.211+0000 7fb025400640 1 -- 172.21.15.86:0/2473996087 >> v1:172.21.15.144:6789/0 conn(0x7fb02815ece0 legacy=0x7fb02815f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:00.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.211+0000 7fb025400640 1 -- 172.21.15.86:0/2473996087 >> v1:172.21.15.86:6789/0 conn(0x7fb0280cf9c0 legacy=0x7fb028160000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:00.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.211+0000 7fb025400640 1 -- 172.21.15.86:0/2473996087 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0280cf1c0 con 0x7fb0280d50c0 2024-04-20T21:28:00.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.212+0000 7fb025400640 1 --2- 172.21.15.86:0/2473996087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0280d50c0 0x7fb0280d3100 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fb0180098a0 tx=0x7fb01802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fecdbe5b6fe5b073 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:00.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.213+0000 7fb024a00640 1 -- 172.21.15.86:0/2473996087 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb018035020 con 0x7fb0280d50c0 2024-04-20T21:28:00.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.213+0000 7fb024a00640 1 -- 172.21.15.86:0/2473996087 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb018031640 con 0x7fb0280d50c0 2024-04-20T21:28:00.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.213+0000 7fb024a00640 1 -- 172.21.15.86:0/2473996087 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01802d040 con 0x7fb0280d50c0 2024-04-20T21:28:00.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.213+0000 7fb027200640 1 -- 172.21.15.86:0/2473996087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0280d50c0 msgr2=0x7fb0280d3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:00.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.213+0000 7fb027200640 1 --2- 172.21.15.86:0/2473996087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0280d50c0 0x7fb0280d3100 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fb0180098a0 tx=0x7fb01802b8f0 comp rx=0 tx=0).stop 2024-04-20T21:28:00.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.213+0000 7fb027200640 1 -- 172.21.15.86:0/2473996087 shutdown_connections 2024-04-20T21:28:00.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.213+0000 7fb027200640 1 --2- 172.21.15.86:0/2473996087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0280d50c0 0x7fb0280d3100 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:00.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.213+0000 7fb027200640 1 -- 172.21.15.86:0/2473996087 >> 172.21.15.86:0/2473996087 conn(0x7fb02808bf70 msgr2=0x7fb0280d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:00.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.213+0000 7fb027200640 1 -- 172.21.15.86:0/2473996087 shutdown_connections 2024-04-20T21:28:00.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.213+0000 7fb027200640 1 -- 172.21.15.86:0/2473996087 wait complete. 2024-04-20T21:28:00.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.215+0000 7fb027200640 1 Processor -- start 2024-04-20T21:28:00.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.215+0000 7fb027200640 1 -- start start 2024-04-20T21:28:00.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.216+0000 7fb027200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0280cf9c0 0x7fb0280b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:00.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.216+0000 7fb027200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0280d50c0 0x7fb0281517e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:00.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.216+0000 7fb027200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb02815ece0 0x7fb0280b9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:00.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.216+0000 7fb027200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb02815ea50 con 0x7fb0280cf9c0 2024-04-20T21:28:00.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.216+0000 7fb027200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb0280d3640 con 0x7fb0280d50c0 2024-04-20T21:28:00.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.216+0000 7fb027200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb0280cfe70 con 0x7fb02815ece0 2024-04-20T21:28:00.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.216+0000 7fb025e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0280cf9c0 0x7fb0280b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:00.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.216+0000 7fb025e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0280cf9c0 0x7fb0280b70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53022/0 (socket says 172.21.15.86:53022) 2024-04-20T21:28:00.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.216+0000 7fb025e00640 1 -- 172.21.15.86:0/4017583781 learned_addr learned my addr 172.21.15.86:0/4017583781 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:00.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.217+0000 7fb025400640 1 --2- 172.21.15.86:0/4017583781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0280d50c0 0x7fb0281517e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:00.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.217+0000 7fb026800640 1 --2- 172.21.15.86:0/4017583781 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb02815ece0 0x7fb0280b9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:00.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.217+0000 7fb025e00640 1 -- 172.21.15.86:0/4017583781 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb02815ece0 msgr2=0x7fb0280b9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:00.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.217+0000 7fb025e00640 1 --2- 172.21.15.86:0/4017583781 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb02815ece0 0x7fb0280b9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:00.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.217+0000 7fb025e00640 1 -- 172.21.15.86:0/4017583781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0280d50c0 msgr2=0x7fb0281517e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:00.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.217+0000 7fb025e00640 1 --2- 172.21.15.86:0/4017583781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0280d50c0 0x7fb0281517e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:00.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.217+0000 7fb025e00640 1 -- 172.21.15.86:0/4017583781 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb028151d20 con 0x7fb0280cf9c0 2024-04-20T21:28:00.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.217+0000 7fb025e00640 1 --2- 172.21.15.86:0/4017583781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0280cf9c0 0x7fb0280b70c0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fb01000c430 tx=0x7fb01000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:00.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.217+0000 7fb01e000640 1 -- 172.21.15.86:0/4017583781 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb010011790 con 0x7fb0280cf9c0 2024-04-20T21:28:00.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.218+0000 7fb01e000640 1 -- 172.21.15.86:0/4017583781 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb010017070 con 0x7fb0280cf9c0 2024-04-20T21:28:00.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.218+0000 7fb01e000640 1 -- 172.21.15.86:0/4017583781 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb010011bb0 con 0x7fb0280cf9c0 2024-04-20T21:28:00.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.218+0000 7fb027200640 1 -- 172.21.15.86:0/4017583781 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0280b7710 con 0x7fb0280cf9c0 2024-04-20T21:28:00.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.218+0000 7fb027200640 1 -- 172.21.15.86:0/4017583781 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0280b7cd0 con 0x7fb0280cf9c0 2024-04-20T21:28:00.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.219+0000 7fb027200640 1 -- 172.21.15.86:0/4017583781 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb02804fcf0 con 0x7fb0280cf9c0 2024-04-20T21:28:00.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.220+0000 7fb01e000640 1 -- 172.21.15.86:0/4017583781 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb01001c460 con 0x7fb0280cf9c0 2024-04-20T21:28:00.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.221+0000 7fb01e000640 1 --2- 172.21.15.86:0/4017583781 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb00406c8f0 0x7fb00406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:00.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.221+0000 7fb01e000640 1 -- 172.21.15.86:0/4017583781 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7fb01008bab0 con 0x7fb0280cf9c0 2024-04-20T21:28:00.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.221+0000 7fb025400640 1 --2- 172.21.15.86:0/4017583781 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb00406c8f0 0x7fb00406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:00.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.222+0000 7fb025400640 1 --2- 172.21.15.86:0/4017583781 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb00406c8f0 0x7fb00406edb0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fb018036000 tx=0x7fb01803c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:00.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.225+0000 7fb01e000640 1 -- 172.21.15.86:0/4017583781 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb010059e40 con 0x7fb0280cf9c0 2024-04-20T21:28:00.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.469+0000 7fb027200640 1 -- 172.21.15.86:0/4017583781 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb02808e320 con 0x7fb0280cf9c0 2024-04-20T21:28:00.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.470+0000 7fb01e000640 1 -- 172.21.15.86:0/4017583781 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5207 (secure 0 0 0) 0x7fb010059740 con 0x7fb0280cf9c0 2024-04-20T21:28:00.469 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:28:00.469 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":82,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5525,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3902778887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3902778887},{"type":"v1","addr":"172.21.15.144:6835","nonce":3902778887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6001,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/4270988014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":4270988014},{"type":"v1","addr":"172.21.15.86:6837","nonce":4270988014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6016,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/1569462554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":1569462554},{"type":"v1","addr":"172.21.15.144:6833","nonce":1569462554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:27:45.692408+0000","modified":"2024-04-20T21:27:48.770936+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8493":{"gid":8493,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6836/1331851138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1331851138},{"type":"v1","addr":"172.21.15.86:6836","nonce":1331851138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8493,"qdb_cluster":[8493]},"id":7}]} 2024-04-20T21:28:00.470 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 82 2024-04-20T21:28:00.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.473+0000 7fb027200640 1 -- 172.21.15.86:0/4017583781 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb00406c8f0 msgr2=0x7fb00406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:00.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.473+0000 7fb027200640 1 --2- 172.21.15.86:0/4017583781 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb00406c8f0 0x7fb00406edb0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fb018036000 tx=0x7fb01803c040 comp rx=0 tx=0).stop 2024-04-20T21:28:00.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.473+0000 7fb027200640 1 -- 172.21.15.86:0/4017583781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0280cf9c0 msgr2=0x7fb0280b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:00.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.473+0000 7fb027200640 1 --2- 172.21.15.86:0/4017583781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0280cf9c0 0x7fb0280b70c0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fb01000c430 tx=0x7fb01000c8f0 comp rx=0 tx=0).stop 2024-04-20T21:28:00.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.473+0000 7fb027200640 1 -- 172.21.15.86:0/4017583781 shutdown_connections 2024-04-20T21:28:00.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.473+0000 7fb027200640 1 --2- 172.21.15.86:0/4017583781 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb00406c8f0 0x7fb00406edb0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:00.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.473+0000 7fb027200640 1 --2- 172.21.15.86:0/4017583781 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb02815ece0 0x7fb0280b9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:00.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.473+0000 7fb027200640 1 --2- 172.21.15.86:0/4017583781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0280d50c0 0x7fb0281517e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:00.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.473+0000 7fb027200640 1 --2- 172.21.15.86:0/4017583781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0280cf9c0 0x7fb0280b70c0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:00.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.473+0000 7fb027200640 1 -- 172.21.15.86:0/4017583781 >> 172.21.15.86:0/4017583781 conn(0x7fb02808bf70 msgr2=0x7fb0281e8060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:00.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.473+0000 7fb027200640 1 -- 172.21.15.86:0/4017583781 shutdown_connections 2024-04-20T21:28:00.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.473+0000 7fb027200640 1 -- 172.21.15.86:0/4017583781 wait complete. 2024-04-20T21:28:00.491 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-20T21:28:00.719 INFO:teuthology.orchestra.run.smithi086.stdout:{"message":"","return_code":0} 2024-04-20T21:28:00.719 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-20T21:28:00.720 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:28:00.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.919+0000 7f8df3200640 1 Processor -- start 2024-04-20T21:28:00.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.919+0000 7f8df3200640 1 -- start start 2024-04-20T21:28:00.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.921+0000 7f8df3200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8dec06e6d0 0x7f8dec071d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:00.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.921+0000 7f8df3200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8dec072240 0x7f8dec15cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:00.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.921+0000 7f8df3200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8dec06eb30 con 0x7f8dec146130 2024-04-20T21:28:00.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.921+0000 7f8df3200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8dec06dc30 con 0x7f8dec072240 2024-04-20T21:28:00.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.921+0000 7f8df3200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8dec0cd8e0 con 0x7f8dec06e6d0 2024-04-20T21:28:00.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.921+0000 7f8df1200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8dec072240 0x7f8dec15cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:00.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.921+0000 7f8debe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f8dec146130 0x7f8dec0ccdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40706/0 (socket says 172.21.15.86:40706) 2024-04-20T21:28:00.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.922+0000 7f8debe00640 1 -- 172.21.15.86:0/805748517 learned_addr learned my addr 172.21.15.86:0/805748517 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:28:00.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.922+0000 7f8deb400640 1 --2- 172.21.15.86:0/805748517 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8dec06e6d0 0x7f8dec071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:00.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.922+0000 7f8df1200640 1 -- 172.21.15.86:0/805748517 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8dec06e6d0 msgr2=0x7f8dec071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:00.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.922+0000 7f8df1200640 1 --2- 172.21.15.86:0/805748517 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8dec06e6d0 0x7f8dec071d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:00.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.922+0000 7f8df1200640 1 -- 172.21.15.86:0/805748517 >> v1:172.21.15.86:6789/0 conn(0x7f8dec146130 legacy=0x7f8dec0ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:00.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.922+0000 7f8df1200640 1 -- 172.21.15.86:0/805748517 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dec0cdbc0 con 0x7f8dec072240 2024-04-20T21:28:00.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.922+0000 7f8deaa00640 1 -- 172.21.15.86:0/805748517 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2045032332 0 0) 0x7f8ddc001110 con 0x7f8dec146130 2024-04-20T21:28:00.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.923+0000 7f8df1200640 1 --2- 172.21.15.86:0/805748517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dec072240 0x7f8dec15cc20 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f8de4009d90 tx=0x7f8de402bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a372aaffeca8e133 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:00.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.924+0000 7f8deaa00640 1 -- 172.21.15.86:0/805748517 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de4039070 con 0x7f8dec072240 2024-04-20T21:28:00.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.924+0000 7f8deaa00640 1 -- 172.21.15.86:0/805748517 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8de402cce0 con 0x7f8dec072240 2024-04-20T21:28:00.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.924+0000 7f8deaa00640 1 -- 172.21.15.86:0/805748517 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de4034660 con 0x7f8dec072240 2024-04-20T21:28:00.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.924+0000 7f8df3200640 1 -- 172.21.15.86:0/805748517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dec072240 msgr2=0x7f8dec15cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:00.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.924+0000 7f8df3200640 1 --2- 172.21.15.86:0/805748517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dec072240 0x7f8dec15cc20 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f8de4009d90 tx=0x7f8de402bf30 comp rx=0 tx=0).stop 2024-04-20T21:28:00.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.924+0000 7f8df3200640 1 -- 172.21.15.86:0/805748517 shutdown_connections 2024-04-20T21:28:00.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.924+0000 7f8df3200640 1 --2- 172.21.15.86:0/805748517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dec072240 0x7f8dec15cc20 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:00.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.924+0000 7f8df3200640 1 --2- 172.21.15.86:0/805748517 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8dec06e6d0 0x7f8dec071d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:00.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.924+0000 7f8df3200640 1 -- 172.21.15.86:0/805748517 >> 172.21.15.86:0/805748517 conn(0x7f8dec0b2360 msgr2=0x7f8dec0b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:00.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.924+0000 7f8df3200640 1 -- 172.21.15.86:0/805748517 shutdown_connections 2024-04-20T21:28:00.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.924+0000 7f8df3200640 1 -- 172.21.15.86:0/805748517 wait complete. 2024-04-20T21:28:00.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.926+0000 7f8df3200640 1 Processor -- start 2024-04-20T21:28:00.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.926+0000 7f8df3200640 1 -- start start 2024-04-20T21:28:00.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.927+0000 7f8df3200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8dec06e6d0 0x7f8dec132dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:00.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.927+0000 7f8df3200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dec072240 0x7f8dec1e8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:00.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.927+0000 7f8df3200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8dec146130 0x7f8dec0cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:00.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.927+0000 7f8df3200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec06f150 con 0x7f8dec072240 2024-04-20T21:28:00.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.927+0000 7f8df3200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec06e1b0 con 0x7f8dec06e6d0 2024-04-20T21:28:00.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.927+0000 7f8df3200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8dec0cd4e0 con 0x7f8dec146130 2024-04-20T21:28:00.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.927+0000 7f8deb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dec072240 0x7f8dec1e8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:00.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.928+0000 7f8deb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dec072240 0x7f8dec1e8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53042/0 (socket says 172.21.15.86:53042) 2024-04-20T21:28:00.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.928+0000 7f8deb400640 1 -- 172.21.15.86:0/264934842 learned_addr learned my addr 172.21.15.86:0/264934842 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:00.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.928+0000 7f8debe00640 1 --2- 172.21.15.86:0/264934842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8dec06e6d0 0x7f8dec132dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:00.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.928+0000 7f8deb400640 1 -- 172.21.15.86:0/264934842 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8dec146130 msgr2=0x7f8dec0cc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:00.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.928+0000 7f8deb400640 1 --2- 172.21.15.86:0/264934842 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8dec146130 0x7f8dec0cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:00.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.928+0000 7f8deb400640 1 -- 172.21.15.86:0/264934842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8dec06e6d0 msgr2=0x7f8dec132dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:00.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.928+0000 7f8deb400640 1 --2- 172.21.15.86:0/264934842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8dec06e6d0 0x7f8dec132dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:00.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.928+0000 7f8deb400640 1 -- 172.21.15.86:0/264934842 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dec133500 con 0x7f8dec072240 2024-04-20T21:28:00.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.928+0000 7f8deb400640 1 --2- 172.21.15.86:0/264934842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dec072240 0x7f8dec1e8d50 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f8dd400d620 tx=0x7f8dd400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:00.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.929+0000 7f8de3e00640 1 -- 172.21.15.86:0/264934842 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd40125f0 con 0x7f8dec072240 2024-04-20T21:28:00.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.929+0000 7f8de3e00640 1 -- 172.21.15.86:0/264934842 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dd4019070 con 0x7f8dec072240 2024-04-20T21:28:00.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.929+0000 7f8df3200640 1 -- 172.21.15.86:0/264934842 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dec1337e0 con 0x7f8dec072240 2024-04-20T21:28:00.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.929+0000 7f8de3e00640 1 -- 172.21.15.86:0/264934842 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd40129d0 con 0x7f8dec072240 2024-04-20T21:28:00.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.929+0000 7f8df3200640 1 -- 172.21.15.86:0/264934842 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dec133da0 con 0x7f8dec072240 2024-04-20T21:28:00.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.930+0000 7f8df3200640 1 -- 172.21.15.86:0/264934842 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dec0689e0 con 0x7f8dec072240 2024-04-20T21:28:00.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.931+0000 7f8de3e00640 1 -- 172.21.15.86:0/264934842 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8dd4012b70 con 0x7f8dec072240 2024-04-20T21:28:00.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.932+0000 7f8de3e00640 1 --2- 172.21.15.86:0/264934842 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8dc806c8f0 0x7f8dc806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:00.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.932+0000 7f8de3e00640 1 -- 172.21.15.86:0/264934842 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7f8dd408bfd0 con 0x7f8dec072240 2024-04-20T21:28:00.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.932+0000 7f8debe00640 1 --2- 172.21.15.86:0/264934842 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8dc806c8f0 0x7f8dc806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:00.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.933+0000 7f8debe00640 1 --2- 172.21.15.86:0/264934842 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8dc806c8f0 0x7f8dc806edb0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc000f60 tx=0x7f8ddc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:00.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:00.937+0000 7f8de3e00640 1 -- 172.21.15.86:0/264934842 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8dd405a360 con 0x7f8dec072240 2024-04-20T21:28:01.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.175+0000 7f8df3200640 1 -- 172.21.15.86:0/264934842 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8dec0b1f60 con 0x7f8dec072240 2024-04-20T21:28:01.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.176+0000 7f8de3e00640 1 -- 172.21.15.86:0/264934842 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5207 (secure 0 0 0) 0x7f8dd4059c60 con 0x7f8dec072240 2024-04-20T21:28:01.175 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:28:01.175 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":82,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5525,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3902778887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3902778887},{"type":"v1","addr":"172.21.15.144:6835","nonce":3902778887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6001,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/4270988014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":4270988014},{"type":"v1","addr":"172.21.15.86:6837","nonce":4270988014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6016,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/1569462554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":1569462554},{"type":"v1","addr":"172.21.15.144:6833","nonce":1569462554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:27:45.692408+0000","modified":"2024-04-20T21:27:48.770936+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8493":{"gid":8493,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6836/1331851138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1331851138},{"type":"v1","addr":"172.21.15.86:6836","nonce":1331851138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8493,"qdb_cluster":[8493]},"id":7}]} 2024-04-20T21:28:01.175 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 82 2024-04-20T21:28:01.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.178+0000 7f8df3200640 1 -- 172.21.15.86:0/264934842 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8dc806c8f0 msgr2=0x7f8dc806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:01.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.178+0000 7f8df3200640 1 --2- 172.21.15.86:0/264934842 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8dc806c8f0 0x7f8dc806edb0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc000f60 tx=0x7f8ddc009040 comp rx=0 tx=0).stop 2024-04-20T21:28:01.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.178+0000 7f8df3200640 1 -- 172.21.15.86:0/264934842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dec072240 msgr2=0x7f8dec1e8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:01.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.178+0000 7f8df3200640 1 --2- 172.21.15.86:0/264934842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dec072240 0x7f8dec1e8d50 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f8dd400d620 tx=0x7f8dd400dae0 comp rx=0 tx=0).stop 2024-04-20T21:28:01.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.179+0000 7f8df3200640 1 -- 172.21.15.86:0/264934842 shutdown_connections 2024-04-20T21:28:01.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.179+0000 7f8df3200640 1 --2- 172.21.15.86:0/264934842 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8dc806c8f0 0x7f8dc806edb0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:01.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.179+0000 7f8df3200640 1 --2- 172.21.15.86:0/264934842 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8dec146130 0x7f8dec0cc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:01.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.179+0000 7f8df3200640 1 --2- 172.21.15.86:0/264934842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dec072240 0x7f8dec1e8d50 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:01.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.179+0000 7f8df3200640 1 --2- 172.21.15.86:0/264934842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8dec06e6d0 0x7f8dec132dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:01.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.179+0000 7f8df3200640 1 -- 172.21.15.86:0/264934842 >> 172.21.15.86:0/264934842 conn(0x7f8dec0b2360 msgr2=0x7f8dec145360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:01.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.179+0000 7f8df3200640 1 -- 172.21.15.86:0/264934842 shutdown_connections 2024-04-20T21:28:01.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.179+0000 7f8df3200640 1 -- 172.21.15.86:0/264934842 wait complete. 2024-04-20T21:28:01.196 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata get mds0_inotable - 2024-04-20T21:28:01.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.302+0000 7f6607c8ef40 1 Processor -- start 2024-04-20T21:28:01.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.302+0000 7f6607c8ef40 1 -- start start 2024-04-20T21:28:01.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.303+0000 7f6607c8ef40 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aaf519a4c0 0x55aaf51c2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:01.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.303+0000 7f6607c8ef40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aaf51c6400 0x55aaf526e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:01.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.303+0000 7f6607c8ef40 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aaf5265c90 0x55aaf52ff6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:01.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.303+0000 7f6607c8ef40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55aaf526f220 con 0x55aaf51c6400 2024-04-20T21:28:01.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.303+0000 7f6607c8ef40 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55aaf513dcc0 con 0x55aaf519a4c0 2024-04-20T21:28:01.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.303+0000 7f6607c8ef40 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55aaf526f4c0 con 0x55aaf5265c90 2024-04-20T21:28:01.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.303+0000 7f6604400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aaf51c6400 0x55aaf526e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:01.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.303+0000 7f6604400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aaf51c6400 0x55aaf526e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53064/0 (socket says 172.21.15.86:53064) 2024-04-20T21:28:01.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.303+0000 7f6604400640 1 -- 172.21.15.86:0/2346658115 learned_addr learned my addr 172.21.15.86:0/2346658115 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:01.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.303+0000 7f6604e00640 1 --2- 172.21.15.86:0/2346658115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aaf519a4c0 0x55aaf51c2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:01.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.303+0000 7f6604400640 1 -- 172.21.15.86:0/2346658115 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aaf5265c90 msgr2=0x55aaf52ff6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:01.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.304+0000 7f6604400640 1 --2- 172.21.15.86:0/2346658115 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aaf5265c90 0x55aaf52ff6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:01.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.304+0000 7f6604400640 1 -- 172.21.15.86:0/2346658115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aaf519a4c0 msgr2=0x55aaf51c2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:01.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.304+0000 7f6604400640 1 --2- 172.21.15.86:0/2346658115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aaf519a4c0 0x55aaf51c2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:01.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.304+0000 7f6604400640 1 -- 172.21.15.86:0/2346658115 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aaf526c420 con 0x55aaf51c6400 2024-04-20T21:28:01.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.304+0000 7f6604400640 1 --2- 172.21.15.86:0/2346658115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aaf51c6400 0x55aaf526e310 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f65f000b2e0 tx=0x7f65f000b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c94beed60b3181f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:01.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.304+0000 7f6603000640 1 -- 172.21.15.86:0/2346658115 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f0008070 con 0x55aaf51c6400 2024-04-20T21:28:01.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.304+0000 7f6603000640 1 -- 172.21.15.86:0/2346658115 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65f000d500 con 0x55aaf51c6400 2024-04-20T21:28:01.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.304+0000 7f6603000640 1 -- 172.21.15.86:0/2346658115 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f000d8f0 con 0x55aaf51c6400 2024-04-20T21:28:01.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.305+0000 7f6607c8ef40 1 -- 172.21.15.86:0/2346658115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aaf51c6400 msgr2=0x55aaf526e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:01.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.305+0000 7f6607c8ef40 1 --2- 172.21.15.86:0/2346658115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aaf51c6400 0x55aaf526e310 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f65f000b2e0 tx=0x7f65f000b7a0 comp rx=0 tx=0).stop 2024-04-20T21:28:01.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.305+0000 7f6607c8ef40 1 -- 172.21.15.86:0/2346658115 shutdown_connections 2024-04-20T21:28:01.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.305+0000 7f6607c8ef40 1 --2- 172.21.15.86:0/2346658115 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aaf5265c90 0x55aaf52ff6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:01.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.305+0000 7f6607c8ef40 1 --2- 172.21.15.86:0/2346658115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aaf51c6400 0x55aaf526e310 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:01.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.305+0000 7f6607c8ef40 1 --2- 172.21.15.86:0/2346658115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aaf519a4c0 0x55aaf51c2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:01.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.305+0000 7f6607c8ef40 1 -- 172.21.15.86:0/2346658115 >> 172.21.15.86:0/2346658115 conn(0x55aaf5194220 msgr2=0x55aaf526a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:01.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.305+0000 7f6607c8ef40 1 -- 172.21.15.86:0/2346658115 shutdown_connections 2024-04-20T21:28:01.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.305+0000 7f6607c8ef40 1 -- 172.21.15.86:0/2346658115 wait complete. 2024-04-20T21:28:01.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.306+0000 7f6607c8ef40 1 Processor -- start 2024-04-20T21:28:01.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.306+0000 7f6607c8ef40 1 -- start start 2024-04-20T21:28:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.307+0000 7f6607c8ef40 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aaf519a4c0 0x55aaf526e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.307+0000 7f6607c8ef40 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aaf51c6400 0x55aaf524bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.307+0000 7f6607c8ef40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aaf5265c90 0x55aaf524c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.307+0000 7f6607c8ef40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55aaf5194800 con 0x55aaf5265c90 2024-04-20T21:28:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.307+0000 7f6607c8ef40 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55aaf526bc00 con 0x55aaf519a4c0 2024-04-20T21:28:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.307+0000 7f6607c8ef40 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55aaf51c6140 con 0x55aaf51c6400 2024-04-20T21:28:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.307+0000 7f6605800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aaf5265c90 0x55aaf524c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.307+0000 7f6605800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aaf5265c90 0x55aaf524c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53080/0 (socket says 172.21.15.86:53080) 2024-04-20T21:28:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.307+0000 7f6605800640 1 -- 172.21.15.86:0/1095455287 learned_addr learned my addr 172.21.15.86:0/1095455287 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.307+0000 7f6604e00640 1 --2- 172.21.15.86:0/1095455287 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aaf519a4c0 0x55aaf526e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.308+0000 7f6604400640 1 --2- 172.21.15.86:0/1095455287 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aaf51c6400 0x55aaf524bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.308+0000 7f6605800640 1 -- 172.21.15.86:0/1095455287 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aaf51c6400 msgr2=0x55aaf524bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.308+0000 7f6605800640 1 --2- 172.21.15.86:0/1095455287 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aaf51c6400 0x55aaf524bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.308+0000 7f6605800640 1 -- 172.21.15.86:0/1095455287 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aaf519a4c0 msgr2=0x55aaf526e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:01.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.308+0000 7f6605800640 1 --2- 172.21.15.86:0/1095455287 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55aaf519a4c0 0x55aaf526e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:01.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.308+0000 7f6605800640 1 -- 172.21.15.86:0/1095455287 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aaf524c9f0 con 0x55aaf5265c90 2024-04-20T21:28:01.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.308+0000 7f6604400640 1 --2- 172.21.15.86:0/1095455287 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55aaf51c6400 0x55aaf524bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:28:01.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.308+0000 7f6605800640 1 --2- 172.21.15.86:0/1095455287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aaf5265c90 0x55aaf524c110 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f65fc00a1d0 tx=0x7f65fc00a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:01.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.308+0000 7f6601c00640 1 -- 172.21.15.86:0/1095455287 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65fc0115f0 con 0x55aaf5265c90 2024-04-20T21:28:01.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.308+0000 7f6601c00640 1 -- 172.21.15.86:0/1095455287 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65fc018070 con 0x55aaf5265c90 2024-04-20T21:28:01.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.308+0000 7f6607c8ef40 1 -- 172.21.15.86:0/1095455287 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55aaf524c650 con 0x55aaf5265c90 2024-04-20T21:28:01.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.308+0000 7f6601c00640 1 -- 172.21.15.86:0/1095455287 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65fc0119f0 con 0x55aaf5265c90 2024-04-20T21:28:01.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.309+0000 7f6607c8ef40 1 -- 172.21.15.86:0/1095455287 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aaf524cf40 con 0x55aaf5265c90 2024-04-20T21:28:01.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.310+0000 7f6601c00640 1 -- 172.21.15.86:0/1095455287 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f65fc011b90 con 0x55aaf5265c90 2024-04-20T21:28:01.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.311+0000 7f6601c00640 1 --2- 172.21.15.86:0/1095455287 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f65e406b2d0 0x7f65e406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:01.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.311+0000 7f6601c00640 1 -- 172.21.15.86:0/1095455287 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7f65fc08ab40 con 0x55aaf5265c90 2024-04-20T21:28:01.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.312+0000 7f6604e00640 1 --2- 172.21.15.86:0/1095455287 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f65e406b2d0 0x7f65e406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:01.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.312+0000 7f6607c8ef40 1 --2- 172.21.15.86:0/1095455287 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x55aaf526ea40 0x55aaf525d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:01.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.312+0000 7f6607c8ef40 1 -- 172.21.15.86:0/1095455287 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:1 14.13 14:c91d4a1d:::mds0_inotable:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55aaf525de40 con 0x55aaf526ea40 2024-04-20T21:28:01.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.312+0000 7f6604400640 1 --2- 172.21.15.86:0/1095455287 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x55aaf526ea40 0x55aaf525d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:01.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.312+0000 7f6604e00640 1 --2- 172.21.15.86:0/1095455287 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f65e406b2d0 0x7f65e406d790 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f65f4057000 tx=0x7f65f4051000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:01.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.313+0000 7f6604400640 1 --2- 172.21.15.86:0/1095455287 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x55aaf526ea40 0x55aaf525d900 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:01.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.313+0000 7f6604400640 1 -- 172.21.15.86:0/1095455287 <== osd.2 v2:172.21.15.86:6816/3437611468 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv3 ondisk = 0) ==== 157+0+34 (crc 0 0 0) 0x7f65f000d830 con 0x55aaf526ea40 2024-04-20T21:28:01.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.313+0000 7f6607c8ef40 1 -- 172.21.15.86:0/1095455287 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x55aaf526ea40 msgr2=0x55aaf525d900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:01.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.313+0000 7f6607c8ef40 1 --2- 172.21.15.86:0/1095455287 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x55aaf526ea40 0x55aaf525d900 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:01.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.313+0000 7f6607c8ef40 1 -- 172.21.15.86:0/1095455287 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f65e406b2d0 msgr2=0x7f65e406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:01.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.313+0000 7f6607c8ef40 1 --2- 172.21.15.86:0/1095455287 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f65e406b2d0 0x7f65e406d790 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f65f4057000 tx=0x7f65f4051000 comp rx=0 tx=0).stop 2024-04-20T21:28:01.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.313+0000 7f6607c8ef40 1 -- 172.21.15.86:0/1095455287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aaf5265c90 msgr2=0x55aaf524c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:01.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.313+0000 7f6607c8ef40 1 --2- 172.21.15.86:0/1095455287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55aaf5265c90 0x55aaf524c110 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f65fc00a1d0 tx=0x7f65fc00a690 comp rx=0 tx=0).stop 2024-04-20T21:28:01.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.313+0000 7f6605800640 1 -- 172.21.15.86:0/1095455287 reap_dead start 2024-04-20T21:28:01.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.313+0000 7f6607c8ef40 1 -- 172.21.15.86:0/1095455287 shutdown_connections 2024-04-20T21:28:01.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.313+0000 7f6607c8ef40 1 -- 172.21.15.86:0/1095455287 >> 172.21.15.86:0/1095455287 conn(0x55aaf5194220 msgr2=0x55aaf5264430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:01.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.314+0000 7f6607c8ef40 1 -- 172.21.15.86:0/1095455287 shutdown_connections 2024-04-20T21:28:01.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.314+0000 7f6607c8ef40 1 -- 172.21.15.86:0/1095455287 wait complete. 2024-04-20T21:28:01.317 INFO:teuthology.orchestra.run.smithi086.stdout:����� 2024-04-20T21:28:01.317 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T21:28:01.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.506+0000 7fb14a400640 1 Processor -- start 2024-04-20T21:28:01.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.506+0000 7fb14a400640 1 -- start start 2024-04-20T21:28:01.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.507+0000 7fb14a400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb144156ae0 0x7fb14415d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:01.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.507+0000 7fb14a400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb144160890 con 0x7fb1440ad7f0 2024-04-20T21:28:01.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.507+0000 7fb14a400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb144160b70 con 0x7fb144160080 2024-04-20T21:28:01.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.507+0000 7fb14a400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb14406db20 con 0x7fb144156ae0 2024-04-20T21:28:01.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.508+0000 7fb149a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fb1440ad7f0 0x7fb14406e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40718/0 (socket says 172.21.15.86:40718) 2024-04-20T21:28:01.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.508+0000 7fb149a00640 1 -- 172.21.15.86:0/1598069291 learned_addr learned my addr 172.21.15.86:0/1598069291 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:28:01.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.508+0000 7fb149000640 1 --2- 172.21.15.86:0/1598069291 >> v2:172.21.15.86:3300/0 conn(0x7fb144156ae0 0x7fb14415d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:01.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.508+0000 7fb143e00640 1 -- 172.21.15.86:0/1598069291 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2010390198 0 0) 0x7fb144160890 con 0x7fb1440ad7f0 2024-04-20T21:28:01.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.508+0000 7fb143e00640 1 -- 172.21.15.86:0/1598069291 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb124003520 con 0x7fb1440ad7f0 2024-04-20T21:28:01.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.508+0000 7fb149000640 1 -- 172.21.15.86:0/1598069291 >> v1:172.21.15.144:6789/0 conn(0x7fb144160080 legacy=0x7fb1440ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:01.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.508+0000 7fb149000640 1 -- 172.21.15.86:0/1598069291 >> v1:172.21.15.86:6789/0 conn(0x7fb1440ad7f0 legacy=0x7fb14406e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:01.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.508+0000 7fb149000640 1 -- 172.21.15.86:0/1598069291 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb144160e50 con 0x7fb144156ae0 2024-04-20T21:28:01.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.510+0000 7fb149000640 1 --2- 172.21.15.86:0/1598069291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb144156ae0 0x7fb14415d6a0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fb1340097a0 tx=0x7fb13402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7f90d387fc47552 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:01.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.510+0000 7fb143e00640 1 -- 172.21.15.86:0/1598069291 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb134035020 con 0x7fb144156ae0 2024-04-20T21:28:01.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.510+0000 7fb143e00640 1 -- 172.21.15.86:0/1598069291 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb134031830 con 0x7fb144156ae0 2024-04-20T21:28:01.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.510+0000 7fb143e00640 1 -- 172.21.15.86:0/1598069291 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb134031c00 con 0x7fb144156ae0 2024-04-20T21:28:01.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.511+0000 7fb14a400640 1 -- 172.21.15.86:0/1598069291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb144156ae0 msgr2=0x7fb14415d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:01.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.511+0000 7fb14a400640 1 --2- 172.21.15.86:0/1598069291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb144156ae0 0x7fb14415d6a0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fb1340097a0 tx=0x7fb13402b8d0 comp rx=0 tx=0).stop 2024-04-20T21:28:01.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.511+0000 7fb14a400640 1 -- 172.21.15.86:0/1598069291 shutdown_connections 2024-04-20T21:28:01.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.511+0000 7fb14a400640 1 --2- 172.21.15.86:0/1598069291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb144156ae0 0x7fb14415d6a0 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:01.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.511+0000 7fb14a400640 1 -- 172.21.15.86:0/1598069291 >> 172.21.15.86:0/1598069291 conn(0x7fb1440d0170 msgr2=0x7fb1440d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:01.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.511+0000 7fb14a400640 1 -- 172.21.15.86:0/1598069291 shutdown_connections 2024-04-20T21:28:01.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.511+0000 7fb14a400640 1 -- 172.21.15.86:0/1598069291 wait complete. 2024-04-20T21:28:01.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.513+0000 7fb14a400640 1 Processor -- start 2024-04-20T21:28:01.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.513+0000 7fb14a400640 1 -- start start 2024-04-20T21:28:01.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.514+0000 7fb14a400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1440ad7f0 0x7fb144158970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:01.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.514+0000 7fb14a400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb144156ae0 0x7fb144073500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:01.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.515+0000 7fb14a400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb144160080 0x7fb1441579e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:01.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.515+0000 7fb14a400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb144160500 con 0x7fb144160080 2024-04-20T21:28:01.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.515+0000 7fb14a400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb144156ec0 con 0x7fb1440ad7f0 2024-04-20T21:28:01.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.515+0000 7fb14a400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb14406e9a0 con 0x7fb144156ae0 2024-04-20T21:28:01.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.515+0000 7fb149a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb144160080 0x7fb1441579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:01.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.515+0000 7fb149a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb144160080 0x7fb1441579e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53086/0 (socket says 172.21.15.86:53086) 2024-04-20T21:28:01.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.515+0000 7fb149a00640 1 -- 172.21.15.86:0/4257196031 learned_addr learned my addr 172.21.15.86:0/4257196031 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:01.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.515+0000 7fb149000640 1 --2- 172.21.15.86:0/4257196031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1440ad7f0 0x7fb144158970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:01.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.515+0000 7fb149a00640 1 -- 172.21.15.86:0/4257196031 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb144156ae0 msgr2=0x7fb144073500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:01.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.515+0000 7fb13be00640 1 --2- 172.21.15.86:0/4257196031 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb144156ae0 0x7fb144073500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:01.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.515+0000 7fb149a00640 1 --2- 172.21.15.86:0/4257196031 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb144156ae0 0x7fb144073500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:01.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.515+0000 7fb149a00640 1 -- 172.21.15.86:0/4257196031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1440ad7f0 msgr2=0x7fb144158970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:01.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.515+0000 7fb149a00640 1 --2- 172.21.15.86:0/4257196031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1440ad7f0 0x7fb144158970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:01.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.515+0000 7fb149a00640 1 -- 172.21.15.86:0/4257196031 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1440cf6f0 con 0x7fb144160080 2024-04-20T21:28:01.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.515+0000 7fb149000640 1 --2- 172.21.15.86:0/4257196031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1440ad7f0 0x7fb144158970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:28:01.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.516+0000 7fb149a00640 1 --2- 172.21.15.86:0/4257196031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb144160080 0x7fb1441579e0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fb13c00cbe0 tx=0x7fb13c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:01.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.516+0000 7fb141600640 1 -- 172.21.15.86:0/4257196031 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13c017070 con 0x7fb144160080 2024-04-20T21:28:01.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.516+0000 7fb141600640 1 -- 172.21.15.86:0/4257196031 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb13c00ad40 con 0x7fb144160080 2024-04-20T21:28:01.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.516+0000 7fb141600640 1 -- 172.21.15.86:0/4257196031 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13c012600 con 0x7fb144160080 2024-04-20T21:28:01.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.516+0000 7fb14a400640 1 -- 172.21.15.86:0/4257196031 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb144098e10 con 0x7fb144160080 2024-04-20T21:28:01.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.516+0000 7fb14a400640 1 -- 172.21.15.86:0/4257196031 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb144158eb0 con 0x7fb144160080 2024-04-20T21:28:01.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.517+0000 7fb14a400640 1 -- 172.21.15.86:0/4257196031 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb14404fcf0 con 0x7fb144160080 2024-04-20T21:28:01.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.518+0000 7fb141600640 1 -- 172.21.15.86:0/4257196031 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb13c01a070 con 0x7fb144160080 2024-04-20T21:28:01.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.519+0000 7fb141600640 1 --2- 172.21.15.86:0/4257196031 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb12006c8f0 0x7fb12006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:01.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.519+0000 7fb141600640 1 -- 172.21.15.86:0/4257196031 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7fb13c08b8b0 con 0x7fb144160080 2024-04-20T21:28:01.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.520+0000 7fb149000640 1 --2- 172.21.15.86:0/4257196031 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb12006c8f0 0x7fb12006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:01.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.521+0000 7fb149000640 1 --2- 172.21.15.86:0/4257196031 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb12006c8f0 0x7fb12006edb0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fb134036000 tx=0x7fb134008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:01.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.524+0000 7fb141600640 1 -- 172.21.15.86:0/4257196031 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb13c0595f0 con 0x7fb144160080 2024-04-20T21:28:01.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.766+0000 7fb14a400640 1 -- 172.21.15.86:0/4257196031 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb14415c6d0 con 0x7fb144160080 2024-04-20T21:28:01.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.767+0000 7fb141600640 1 -- 172.21.15.86:0/4257196031 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v82) ==== 53+0+73 (secure 0 0 0) 0x7fb13c05d8f0 con 0x7fb144160080 2024-04-20T21:28:01.766 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T21:28:01.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.769+0000 7fb14a400640 1 -- 172.21.15.86:0/4257196031 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb12006c8f0 msgr2=0x7fb12006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:01.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.769+0000 7fb14a400640 1 --2- 172.21.15.86:0/4257196031 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb12006c8f0 0x7fb12006edb0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fb134036000 tx=0x7fb134008030 comp rx=0 tx=0).stop 2024-04-20T21:28:01.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.769+0000 7fb14a400640 1 -- 172.21.15.86:0/4257196031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb144160080 msgr2=0x7fb1441579e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:01.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.769+0000 7fb14a400640 1 --2- 172.21.15.86:0/4257196031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb144160080 0x7fb1441579e0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fb13c00cbe0 tx=0x7fb13c00a430 comp rx=0 tx=0).stop 2024-04-20T21:28:01.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.769+0000 7fb14a400640 1 -- 172.21.15.86:0/4257196031 shutdown_connections 2024-04-20T21:28:01.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.769+0000 7fb14a400640 1 --2- 172.21.15.86:0/4257196031 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb12006c8f0 0x7fb12006edb0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:01.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.769+0000 7fb14a400640 1 --2- 172.21.15.86:0/4257196031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb144160080 0x7fb1441579e0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:01.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.769+0000 7fb14a400640 1 --2- 172.21.15.86:0/4257196031 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb144156ae0 0x7fb144073500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:01.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.769+0000 7fb14a400640 1 --2- 172.21.15.86:0/4257196031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1440ad7f0 0x7fb144158970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:01.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.769+0000 7fb14a400640 1 -- 172.21.15.86:0/4257196031 >> 172.21.15.86:0/4257196031 conn(0x7fb1440d0170 msgr2=0x7fb1440b2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:01.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.770+0000 7fb14a400640 1 -- 172.21.15.86:0/4257196031 shutdown_connections 2024-04-20T21:28:01.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:01.770+0000 7fb14a400640 1 -- 172.21.15.86:0/4257196031 wait complete. 2024-04-20T21:28:01.787 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T21:28:01.787 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T21:28:01.788 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-04-20T21:28:01.788 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-20T21:28:01.788 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T21:28:01.788 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T21:28:01.788 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T21:28:01.788 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T21:28:01.788 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:28:01.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:28:01.788 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-04-20T21:28:01.842 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T21:28:01.842 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T21:28:01.842 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T21:28:01.842 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:28:01.842 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-20T21:28:01.842 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:28:01.842 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:28:01.842 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:28:01.843 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-04-20T21:28:01.843 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:28:01.843 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:28:01.843 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-04-20T21:28:01.843 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:28:01.843 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:28:01.843 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T21:28:01.843 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:28:01.843 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T21:28:01.843 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42074sec preferred_lft 42074sec 2024-04-20T21:28:01.843 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-04-20T21:28:01.843 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:28:01.843 INFO:teuthology.orchestra.run.smithi086.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T21:28:01.844 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 86:25:97:2c:9c:bb brd ff:ff:ff:ff:ff:ff 2024-04-20T21:28:01.844 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-20T21:28:01.844 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:28:01.844 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::a4fb:30ff:fe90:2d6a/64 scope link 2024-04-20T21:28:01.844 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:28:01.844 INFO:teuthology.orchestra.run.smithi086.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T21:28:01.844 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 86:25:97:2c:9c:bb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:28:01.844 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::8425:97ff:fe2c:9cbb/64 scope link 2024-04-20T21:28:01.844 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:28:01.844 INFO:teuthology.orchestra.run.smithi086.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T21:28:01.844 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c2:9c:8b:55:ca:84 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:28:01.844 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c09c:8bff:fe55:ca84/64 scope link 2024-04-20T21:28:01.844 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:28:01.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:28:01.845 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T21:28:01.919 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-20T21:28:01.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:28:01.925 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-04-20T21:28:01.976 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-04-20T21:28:01.977 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T21:28:01.977 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T21:28:01.977 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T21:28:01.977 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T21:28:01.978 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:28:01.978 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-20T21:28:01.978 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T21:28:01.978 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T21:28:01.978 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T21:28:01.978 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-20T21:28:02.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:28:02.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:28:02.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:28:02.183 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-20T21:28:02.231 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-20T21:28:02.232 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-20T21:28:02.232 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:28:02.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:28:02.232 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-04-20T21:28:02.317 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-20T21:28:02.317 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T21:28:02.317 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-20T21:28:02.369 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:28:02.370 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T21:28:02.370 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-20T21:28:02.416 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-04-20T21:28:02.546 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T21:28:02.603 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-04-20T21:28:02.604 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-04-20T21:28:02.604 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-04-20T21:28:02.604 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-04-20T21:28:02.604 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-04-20T21:28:02.604 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-04-20T21:28:02.604 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-04-20T21:28:02.604 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-04-20T21:28:02.604 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-04-20T21:28:02.604 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-04-20T21:28:02.604 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-04-20T21:28:02.604 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/selinux 2024-04-20T21:28:02.604 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T21:28:02.604 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-04-20T21:28:02.604 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-04-20T21:28:02.605 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-04-20T21:28:02.605 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-04-20T21:28:02.605 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-04-20T21:28:02.605 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-04-20T21:28:02.605 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T21:28:02.605 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T21:28:02.605 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T21:28:02.605 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T21:28:02.605 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-04-20T21:28:02.605 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-04-20T21:28:02.605 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T21:28:02.605 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T21:28:02.605 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T21:28:02.606 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T21:28:02.606 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-04-20T21:28:02.606 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:28:02.606 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:28:02.606 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:28:02.606 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:28:02.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:28:02.607 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-04-20T21:28:02.661 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T21:28:02.662 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-20T21:28:02.667 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-04-20T21:28:02.739 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T21:28:02.781 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-04-20T21:28:02.781 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-04-20T21:28:02.781 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-04-20T21:28:02.781 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-04-20T21:28:02.781 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-04-20T21:28:02.781 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-04-20T21:28:02.781 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-04-20T21:28:02.781 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-04-20T21:28:02.781 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-04-20T21:28:02.781 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-04-20T21:28:02.781 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-04-20T21:28:02.781 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/selinux 2024-04-20T21:28:02.782 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T21:28:02.782 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-04-20T21:28:02.782 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-04-20T21:28:02.782 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-04-20T21:28:02.782 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-04-20T21:28:02.782 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-04-20T21:28:02.782 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-04-20T21:28:02.782 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T21:28:02.782 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T21:28:02.782 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T21:28:02.782 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T21:28:02.783 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-04-20T21:28:02.783 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-04-20T21:28:02.783 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T21:28:02.783 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T21:28:02.783 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T21:28:02.783 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T21:28:02.783 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-04-20T21:28:02.783 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:28:02.783 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:28:02.783 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:28:02.783 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:28:02.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:28:02.784 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-04-20T21:28:02.839 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:2024-04-20T21:28:02.839+0000 7fa487741480 -1 init, newargv = 0x55d3577de510 newargc=15 2024-04-20T21:28:02.839 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[60587]: starting ceph client 2024-04-20T21:28:02.849 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[60587]: starting fuse 2024-04-20T21:28:02.871 INFO:teuthology.orchestra.run.smithi086.stdout:41 2024-04-20T21:28:02.871 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-04-20T21:28:02.871 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer -- python3 -c ' 2024-04-20T21:28:02.871 DEBUG:teuthology.orchestra.run.smithi086:> import glob 2024-04-20T21:28:02.871 DEBUG:teuthology.orchestra.run.smithi086:> import re 2024-04-20T21:28:02.871 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:28:02.871 DEBUG:teuthology.orchestra.run.smithi086:> import subprocess 2024-04-20T21:28:02.871 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:28:02.871 DEBUG:teuthology.orchestra.run.smithi086:> def _find_admin_socket(client_name): 2024-04-20T21:28:02.871 DEBUG:teuthology.orchestra.run.smithi086:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-20T21:28:02.871 DEBUG:teuthology.orchestra.run.smithi086:> files = glob.glob(asok_path) 2024-04-20T21:28:02.872 DEBUG:teuthology.orchestra.run.smithi086:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-20T21:28:02.872 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:28:02.872 DEBUG:teuthology.orchestra.run.smithi086:> # Given a non-glob path, it better be there 2024-04-20T21:28:02.872 DEBUG:teuthology.orchestra.run.smithi086:> if "*" not in asok_path: 2024-04-20T21:28:02.872 DEBUG:teuthology.orchestra.run.smithi086:> assert(len(files) == 1) 2024-04-20T21:28:02.872 DEBUG:teuthology.orchestra.run.smithi086:> return files[0] 2024-04-20T21:28:02.872 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:28:02.872 DEBUG:teuthology.orchestra.run.smithi086:> for f in files: 2024-04-20T21:28:02.872 DEBUG:teuthology.orchestra.run.smithi086:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T21:28:02.872 DEBUG:teuthology.orchestra.run.smithi086:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T21:28:02.872 DEBUG:teuthology.orchestra.run.smithi086:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T21:28:02.872 DEBUG:teuthology.orchestra.run.smithi086:> contents = proc_f.read() 2024-04-20T21:28:02.872 DEBUG:teuthology.orchestra.run.smithi086:> if mountpoint in contents: 2024-04-20T21:28:02.872 DEBUG:teuthology.orchestra.run.smithi086:> return f 2024-04-20T21:28:02.872 DEBUG:teuthology.orchestra.run.smithi086:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T21:28:02.872 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:28:02.873 DEBUG:teuthology.orchestra.run.smithi086:> print(_find_admin_socket("client.0")) 2024-04-20T21:28:02.873 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:28:03.059 INFO:teuthology.orchestra.run.smithi086.stdout:/var/run/ceph/ceph-client.0.60587.asok 2024-04-20T21:28:03.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:28:03.076 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.60587.asok 2024-04-20T21:28:03.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:28:03.076 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.60587.asok status 2024-04-20T21:28:03.302 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:28:03.303 INFO:teuthology.orchestra.run.smithi086.stdout: "metadata": { 2024-04-20T21:28:03.303 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T21:28:03.303 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T21:28:03.303 INFO:teuthology.orchestra.run.smithi086.stdout: "entity_id": "0", 2024-04-20T21:28:03.303 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": "smithi086", 2024-04-20T21:28:03.303 INFO:teuthology.orchestra.run.smithi086.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-20T21:28:03.303 INFO:teuthology.orchestra.run.smithi086.stdout: "pid": "60587", 2024-04-20T21:28:03.303 INFO:teuthology.orchestra.run.smithi086.stdout: "root": "/" 2024-04-20T21:28:03.303 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:28:03.303 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_count": 0, 2024-04-20T21:28:03.303 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_pinned_count": 0, 2024-04-20T21:28:03.303 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 8769, 2024-04-20T21:28:03.303 INFO:teuthology.orchestra.run.smithi086.stdout: "inst": { 2024-04-20T21:28:03.303 INFO:teuthology.orchestra.run.smithi086.stdout: "name": { 2024-04-20T21:28:03.304 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "client", 2024-04-20T21:28:03.304 INFO:teuthology.orchestra.run.smithi086.stdout: "num": 8769 2024-04-20T21:28:03.304 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:28:03.304 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2024-04-20T21:28:03.304 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-04-20T21:28:03.304 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2024-04-20T21:28:03.304 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 912783749 2024-04-20T21:28:03.304 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:28:03.304 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:28:03.304 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2024-04-20T21:28:03.304 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-04-20T21:28:03.304 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2024-04-20T21:28:03.304 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 912783749 2024-04-20T21:28:03.304 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:28:03.304 INFO:teuthology.orchestra.run.smithi086.stdout: "inst_str": "client.8769 192.168.0.1:0/912783749", 2024-04-20T21:28:03.304 INFO:teuthology.orchestra.run.smithi086.stdout: "addr_str": "192.168.0.1:0/912783749", 2024-04-20T21:28:03.305 INFO:teuthology.orchestra.run.smithi086.stdout: "inode_count": 1, 2024-04-20T21:28:03.305 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_epoch": 82, 2024-04-20T21:28:03.305 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch": 100, 2024-04-20T21:28:03.305 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch_barrier": 0, 2024-04-20T21:28:03.305 INFO:teuthology.orchestra.run.smithi086.stdout: "blocklisted": false, 2024-04-20T21:28:03.305 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs" 2024-04-20T21:28:03.305 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:28:03.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:28:03.317 DEBUG:teuthology.orchestra.run.smithi086:> stat --file-system '--printf=%T 2024-04-20T21:28:03.317 DEBUG:teuthology.orchestra.run.smithi086:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-20T21:28:03.372 INFO:teuthology.orchestra.run.smithi086.stdout:fuseblk 2024-04-20T21:28:03.373 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-20T21:28:03.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:28:03.373 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-20T21:28:03.455 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:28:03.455 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:28:03.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:28:03.506 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file2_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-20T21:28:03.759 INFO:teuthology.orchestra.run.smithi086.stderr:6+0 records in 2024-04-20T21:28:03.759 INFO:teuthology.orchestra.run.smithi086.stderr:6+0 records out 2024-04-20T21:28:03.759 INFO:teuthology.orchestra.run.smithi086.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.191709 s, 32.8 MB/s 2024-04-20T21:28:03.761 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:28:03.761 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:28:03.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:28:03.815 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file3_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-20T21:28:04.035 INFO:teuthology.orchestra.run.smithi086.stderr:6+0 records in 2024-04-20T21:28:04.035 INFO:teuthology.orchestra.run.smithi086.stderr:6+0 records out 2024-04-20T21:28:04.035 INFO:teuthology.orchestra.run.smithi086.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.165084 s, 38.1 MB/s 2024-04-20T21:28:04.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:28:04.037 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-20T21:28:04.100 INFO:teuthology.orchestra.run.smithi086.stdout:./ 2024-04-20T21:28:04.101 INFO:teuthology.orchestra.run.smithi086.stdout:./file2_sixmegs 2024-04-20T21:28:04.101 INFO:teuthology.orchestra.run.smithi086.stdout:./file3_sixmegs 2024-04-20T21:28:04.101 INFO:teuthology.orchestra.run.smithi086.stdout:./file1_sixmegs 2024-04-20T21:28:04.102 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-04-20T21:28:04.102 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:28:04.102 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-04-20T21:28:04.102 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:28:04.102 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-20T21:28:04.102 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:28:04.236 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-20T21:28:04.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:28:04.250 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-04-20T21:28:04.250 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:28:04.250 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-04-20T21:28:04.250 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:28:04.250 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file2_sixmegs").st_ino) 2024-04-20T21:28:04.250 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:28:04.422 INFO:teuthology.orchestra.run.smithi086.stdout:1099511627777 2024-04-20T21:28:04.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:28:04.435 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-04-20T21:28:04.435 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:28:04.435 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-04-20T21:28:04.435 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:28:04.435 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file3_sixmegs").st_ino) 2024-04-20T21:28:04.435 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:28:04.621 INFO:teuthology.orchestra.run.smithi086.stdout:1099511627778 2024-04-20T21:28:04.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:28:04.635 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-04-20T21:28:04.635 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:28:04.635 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-04-20T21:28:04.635 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:28:04.635 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file1_sixmegs").st_ino) 2024-04-20T21:28:04.635 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:28:04.820 INFO:teuthology.orchestra.run.smithi086.stdout:1099511627776 2024-04-20T21:28:04.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:28:04.834 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:28:04.834 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:28:04.930 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:28:05.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.131+0000 7f645c800640 1 Processor -- start 2024-04-20T21:28:05.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.131+0000 7f645c800640 1 -- start start 2024-04-20T21:28:05.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.133+0000 7f645c800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6454144e90 0x7f645415ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:05.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.133+0000 7f645c800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f645406f140 0x7f64540aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:05.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.133+0000 7f645c800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f645406dc30 con 0x7f64540af0b0 2024-04-20T21:28:05.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.133+0000 7f645c800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f6454145270 con 0x7f645406f140 2024-04-20T21:28:05.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.133+0000 7f645c800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f645415f720 con 0x7f6454144e90 2024-04-20T21:28:05.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.134+0000 7f645a800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f64540af0b0 0x7f6454071d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40726/0 (socket says 172.21.15.86:40726) 2024-04-20T21:28:05.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.134+0000 7f645a800640 1 -- 172.21.15.86:0/3739523852 learned_addr learned my addr 172.21.15.86:0/3739523852 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:28:05.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.134+0000 7f6459400640 1 --2- 172.21.15.86:0/3739523852 >> v2:172.21.15.86:3300/0 conn(0x7f645406f140 0x7f64540aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:05.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.134+0000 7f6459e00640 1 --2- 172.21.15.86:0/3739523852 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6454144e90 0x7f645415ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:05.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.134+0000 7f6459400640 1 -- 172.21.15.86:0/3739523852 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6454144e90 msgr2=0x7f645415ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:05.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.134+0000 7f6458a00640 1 -- 172.21.15.86:0/3739523852 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4248271925 0 0) 0x7f645406dc30 con 0x7f64540af0b0 2024-04-20T21:28:05.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.134+0000 7f6459400640 1 --2- 172.21.15.86:0/3739523852 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6454144e90 0x7f645415ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:05.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.134+0000 7f6459400640 1 -- 172.21.15.86:0/3739523852 >> v1:172.21.15.86:6789/0 conn(0x7f64540af0b0 legacy=0x7f6454071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:05.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.134+0000 7f6459400640 1 -- 172.21.15.86:0/3739523852 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f645415fa00 con 0x7f645406f140 2024-04-20T21:28:05.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.135+0000 7f6459400640 1 --2- 172.21.15.86:0/3739523852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f645406f140 0x7f64540aeb70 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f64541453f0 tx=0x7f644402b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e57946f3e460da67 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:05.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.136+0000 7f6458a00640 1 -- 172.21.15.86:0/3739523852 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6444035020 con 0x7f645406f140 2024-04-20T21:28:05.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.136+0000 7f6458a00640 1 -- 172.21.15.86:0/3739523852 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6444031800 con 0x7f645406f140 2024-04-20T21:28:05.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.136+0000 7f6458a00640 1 -- 172.21.15.86:0/3739523852 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6444031bd0 con 0x7f645406f140 2024-04-20T21:28:05.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.136+0000 7f645c800640 1 -- 172.21.15.86:0/3739523852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f645406f140 msgr2=0x7f64540aeb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:05.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.136+0000 7f645c800640 1 --2- 172.21.15.86:0/3739523852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f645406f140 0x7f64540aeb70 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f64541453f0 tx=0x7f644402b8a0 comp rx=0 tx=0).stop 2024-04-20T21:28:05.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.136+0000 7f645c800640 1 -- 172.21.15.86:0/3739523852 shutdown_connections 2024-04-20T21:28:05.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.136+0000 7f645c800640 1 --2- 172.21.15.86:0/3739523852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f645406f140 0x7f64540aeb70 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:05.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.136+0000 7f645c800640 1 --2- 172.21.15.86:0/3739523852 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6454144e90 0x7f645415ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:05.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.137+0000 7f645c800640 1 -- 172.21.15.86:0/3739523852 >> 172.21.15.86:0/3739523852 conn(0x7f64540b2360 msgr2=0x7f64540b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:05.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.137+0000 7f645c800640 1 -- 172.21.15.86:0/3739523852 shutdown_connections 2024-04-20T21:28:05.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.137+0000 7f645c800640 1 -- 172.21.15.86:0/3739523852 wait complete. 2024-04-20T21:28:05.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.139+0000 7f645c800640 1 Processor -- start 2024-04-20T21:28:05.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.139+0000 7f645c800640 1 -- start start 2024-04-20T21:28:05.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.140+0000 7f645c800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f645406f140 0x7f6454143870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:05.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.140+0000 7f645c800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64540af0b0 0x7f64540cce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:05.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.140+0000 7f645c800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6454144e90 0x7f6454133a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:05.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.140+0000 7f645c800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f645415f350 con 0x7f645406f140 2024-04-20T21:28:05.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.140+0000 7f645c800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f645406ded0 con 0x7f6454144e90 2024-04-20T21:28:05.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.140+0000 7f645c800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f645415f4d0 con 0x7f64540af0b0 2024-04-20T21:28:05.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.141+0000 7f6459e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f645406f140 0x7f6454143870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:05.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.141+0000 7f6459e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f645406f140 0x7f6454143870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53098/0 (socket says 172.21.15.86:53098) 2024-04-20T21:28:05.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.141+0000 7f6459e00640 1 -- 172.21.15.86:0/1529849997 learned_addr learned my addr 172.21.15.86:0/1529849997 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:05.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.141+0000 7f6459e00640 1 -- 172.21.15.86:0/1529849997 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64540af0b0 msgr2=0x7f64540cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:05.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.141+0000 7f6459e00640 1 --2- 172.21.15.86:0/1529849997 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64540af0b0 0x7f64540cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:05.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.141+0000 7f6459e00640 1 -- 172.21.15.86:0/1529849997 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6454144e90 msgr2=0x7f6454133a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:05.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.141+0000 7f6459e00640 1 --2- 172.21.15.86:0/1529849997 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6454144e90 0x7f6454133a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:05.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.141+0000 7f6459e00640 1 -- 172.21.15.86:0/1529849997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64540cd7d0 con 0x7f645406f140 2024-04-20T21:28:05.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.141+0000 7f6459e00640 1 --2- 172.21.15.86:0/1529849997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f645406f140 0x7f6454143870 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f644800d800 tx=0x7f644800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:05.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.142+0000 7f644e000640 1 -- 172.21.15.86:0/1529849997 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644800b6c0 con 0x7f645406f140 2024-04-20T21:28:05.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.142+0000 7f644e000640 1 -- 172.21.15.86:0/1529849997 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6448019070 con 0x7f645406f140 2024-04-20T21:28:05.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.142+0000 7f645c800640 1 -- 172.21.15.86:0/1529849997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64540cdab0 con 0x7f645406f140 2024-04-20T21:28:05.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.142+0000 7f644e000640 1 -- 172.21.15.86:0/1529849997 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644800ba80 con 0x7f645406f140 2024-04-20T21:28:05.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.142+0000 7f645c800640 1 -- 172.21.15.86:0/1529849997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64540ce350 con 0x7f645406f140 2024-04-20T21:28:05.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.143+0000 7f645c800640 1 -- 172.21.15.86:0/1529849997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64540cdc80 con 0x7f645406f140 2024-04-20T21:28:05.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.144+0000 7f644e000640 1 -- 172.21.15.86:0/1529849997 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f644800bc20 con 0x7f645406f140 2024-04-20T21:28:05.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.145+0000 7f644e000640 1 --2- 172.21.15.86:0/1529849997 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f643806c8f0 0x7f643806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:05.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.145+0000 7f644e000640 1 -- 172.21.15.86:0/1529849997 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7f644808c0e0 con 0x7f645406f140 2024-04-20T21:28:05.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.145+0000 7f6459400640 1 --2- 172.21.15.86:0/1529849997 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f643806c8f0 0x7f643806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:05.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.146+0000 7f6459400640 1 --2- 172.21.15.86:0/1529849997 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f643806c8f0 0x7f643806edb0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f6444036000 tx=0x7f6444030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:05.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.150+0000 7f644e000640 1 -- 172.21.15.86:0/1529849997 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f644801c360 con 0x7f645406f140 2024-04-20T21:28:05.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.351+0000 7f645c800640 1 -- 172.21.15.86:0/1529849997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f64540689e0 con 0x7f645406f140 2024-04-20T21:28:05.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.353+0000 7f644e000640 1 -- 172.21.15.86:0/1529849997 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+16610 (secure 0 0 0) 0x7f644805a470 con 0x7f645406f140 2024-04-20T21:28:05.353 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:28:05.353 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":100,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:27:47.267402+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:27:43.469685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-20T21:27:44.673505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:28:05.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.356+0000 7f645c800640 1 -- 172.21.15.86:0/1529849997 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f643806c8f0 msgr2=0x7f643806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:05.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.356+0000 7f645c800640 1 --2- 172.21.15.86:0/1529849997 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f643806c8f0 0x7f643806edb0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f6444036000 tx=0x7f6444030000 comp rx=0 tx=0).stop 2024-04-20T21:28:05.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.356+0000 7f645c800640 1 -- 172.21.15.86:0/1529849997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f645406f140 msgr2=0x7f6454143870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:05.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.356+0000 7f645c800640 1 --2- 172.21.15.86:0/1529849997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f645406f140 0x7f6454143870 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f644800d800 tx=0x7f644800dcc0 comp rx=0 tx=0).stop 2024-04-20T21:28:05.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.356+0000 7f645c800640 1 -- 172.21.15.86:0/1529849997 shutdown_connections 2024-04-20T21:28:05.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.356+0000 7f645c800640 1 --2- 172.21.15.86:0/1529849997 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f643806c8f0 0x7f643806edb0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:05.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.356+0000 7f645c800640 1 --2- 172.21.15.86:0/1529849997 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6454144e90 0x7f6454133a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:05.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.356+0000 7f645c800640 1 --2- 172.21.15.86:0/1529849997 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f64540af0b0 0x7f64540cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:05.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.356+0000 7f645c800640 1 --2- 172.21.15.86:0/1529849997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f645406f140 0x7f6454143870 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:05.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.356+0000 7f645c800640 1 -- 172.21.15.86:0/1529849997 >> 172.21.15.86:0/1529849997 conn(0x7f64540b2360 msgr2=0x7f64540aff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:05.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.356+0000 7f645c800640 1 -- 172.21.15.86:0/1529849997 shutdown_connections 2024-04-20T21:28:05.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.356+0000 7f645c800640 1 -- 172.21.15.86:0/1529849997 wait complete. 2024-04-20T21:28:05.375 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:27:43.469685+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:27:44.673505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:28:05.376 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:28:05.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.570+0000 7fe624e00640 1 Processor -- start 2024-04-20T21:28:05.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.570+0000 7fe624e00640 1 -- start start 2024-04-20T21:28:05.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.572+0000 7fe624e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe620155d30 0x7fe6200cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:05.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.572+0000 7fe624e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6200b52b0 0x7fe62006be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:05.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.572+0000 7fe624e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe620156110 con 0x7fe62006b980 2024-04-20T21:28:05.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.572+0000 7fe624e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe6200b56e0 con 0x7fe620155d30 2024-04-20T21:28:05.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.572+0000 7fe624e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe6200ce460 con 0x7fe6200b52b0 2024-04-20T21:28:05.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.572+0000 7fe61de00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe620155d30 0x7fe6200cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:05.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.572+0000 7fe61de00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe620155d30 0x7fe6200cf500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53106/0 (socket says 172.21.15.86:53106) 2024-04-20T21:28:05.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.572+0000 7fe61de00640 1 -- 172.21.15.86:0/2902978168 learned_addr learned my addr 172.21.15.86:0/2902978168 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:05.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.573+0000 7fe61de00640 1 -- 172.21.15.86:0/2902978168 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6200b52b0 msgr2=0x7fe62006be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:05.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.573+0000 7fe61de00640 1 --2- 172.21.15.86:0/2902978168 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6200b52b0 0x7fe62006be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:05.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.573+0000 7fe61de00640 1 -- 172.21.15.86:0/2902978168 >> v1:172.21.15.144:6789/0 conn(0x7fe62006b980 legacy=0x7fe6200b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:05.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.573+0000 7fe61de00640 1 -- 172.21.15.86:0/2902978168 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6200ce740 con 0x7fe620155d30 2024-04-20T21:28:05.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.574+0000 7fe61de00640 1 --2- 172.21.15.86:0/2902978168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe620155d30 0x7fe6200cf500 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fe6200b3150 tx=0x7fe61002d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68cffea8bb741a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:05.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.574+0000 7fe61d400640 1 -- 172.21.15.86:0/2902978168 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe610031800 con 0x7fe620155d30 2024-04-20T21:28:05.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.574+0000 7fe61d400640 1 -- 172.21.15.86:0/2902978168 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6100319a0 con 0x7fe620155d30 2024-04-20T21:28:05.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.574+0000 7fe61d400640 1 -- 172.21.15.86:0/2902978168 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe610031d70 con 0x7fe620155d30 2024-04-20T21:28:05.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.575+0000 7fe624e00640 1 -- 172.21.15.86:0/2902978168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe620155d30 msgr2=0x7fe6200cf500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:05.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.575+0000 7fe624e00640 1 --2- 172.21.15.86:0/2902978168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe620155d30 0x7fe6200cf500 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fe6200b3150 tx=0x7fe61002d9e0 comp rx=0 tx=0).stop 2024-04-20T21:28:05.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.575+0000 7fe624e00640 1 -- 172.21.15.86:0/2902978168 shutdown_connections 2024-04-20T21:28:05.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.575+0000 7fe624e00640 1 --2- 172.21.15.86:0/2902978168 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe6200b52b0 0x7fe62006be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:05.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.575+0000 7fe624e00640 1 --2- 172.21.15.86:0/2902978168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe620155d30 0x7fe6200cf500 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:05.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.575+0000 7fe624e00640 1 -- 172.21.15.86:0/2902978168 >> 172.21.15.86:0/2902978168 conn(0x7fe6200b2110 msgr2=0x7fe6200b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:05.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.575+0000 7fe624e00640 1 -- 172.21.15.86:0/2902978168 shutdown_connections 2024-04-20T21:28:05.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.575+0000 7fe624e00640 1 -- 172.21.15.86:0/2902978168 wait complete. 2024-04-20T21:28:05.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.577+0000 7fe624e00640 1 Processor -- start 2024-04-20T21:28:05.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.577+0000 7fe624e00640 1 -- start start 2024-04-20T21:28:05.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.578+0000 7fe624e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe62006b980 0x7fe620156420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:05.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.578+0000 7fe624e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6200b52b0 0x7fe6201f79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:05.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.578+0000 7fe624e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe620155d30 0x7fe62015baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:05.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.578+0000 7fe624e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe6200cefe0 con 0x7fe620155d30 2024-04-20T21:28:05.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.578+0000 7fe624e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe6200ced00 con 0x7fe6200b52b0 2024-04-20T21:28:05.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.578+0000 7fe624e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe62006ddf0 con 0x7fe62006b980 2024-04-20T21:28:05.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.578+0000 7fe61e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe620155d30 0x7fe62015baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:05.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.579+0000 7fe61e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe620155d30 0x7fe62015baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53114/0 (socket says 172.21.15.86:53114) 2024-04-20T21:28:05.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.579+0000 7fe61e800640 1 -- 172.21.15.86:0/1385403447 learned_addr learned my addr 172.21.15.86:0/1385403447 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:05.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.579+0000 7fe61de00640 1 --2- 172.21.15.86:0/1385403447 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe62006b980 0x7fe620156420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:05.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.579+0000 7fe615400640 1 --2- 172.21.15.86:0/1385403447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6200b52b0 0x7fe6201f79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:05.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.579+0000 7fe61e800640 1 -- 172.21.15.86:0/1385403447 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe62006b980 msgr2=0x7fe620156420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:05.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.579+0000 7fe61e800640 1 --2- 172.21.15.86:0/1385403447 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe62006b980 0x7fe620156420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:05.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.579+0000 7fe61e800640 1 -- 172.21.15.86:0/1385403447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6200b52b0 msgr2=0x7fe6201f79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:05.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.579+0000 7fe61e800640 1 --2- 172.21.15.86:0/1385403447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6200b52b0 0x7fe6201f79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:05.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.579+0000 7fe61e800640 1 -- 172.21.15.86:0/1385403447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6201f45d0 con 0x7fe620155d30 2024-04-20T21:28:05.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.579+0000 7fe615400640 1 --2- 172.21.15.86:0/1385403447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6200b52b0 0x7fe6201f79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:28:05.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.579+0000 7fe61e800640 1 --2- 172.21.15.86:0/1385403447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe620155d30 0x7fe62015baf0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7fe61800b970 tx=0x7fe61800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:05.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.579+0000 7fe616a00640 1 -- 172.21.15.86:0/1385403447 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe618019070 con 0x7fe620155d30 2024-04-20T21:28:05.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.580+0000 7fe616a00640 1 -- 172.21.15.86:0/1385403447 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe61800cb60 con 0x7fe620155d30 2024-04-20T21:28:05.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.580+0000 7fe624e00640 1 -- 172.21.15.86:0/1385403447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6201f8140 con 0x7fe620155d30 2024-04-20T21:28:05.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.580+0000 7fe616a00640 1 -- 172.21.15.86:0/1385403447 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe618014440 con 0x7fe620155d30 2024-04-20T21:28:05.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.580+0000 7fe624e00640 1 -- 172.21.15.86:0/1385403447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6201f8700 con 0x7fe620155d30 2024-04-20T21:28:05.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.581+0000 7fe624e00640 1 -- 172.21.15.86:0/1385403447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6200689e0 con 0x7fe620155d30 2024-04-20T21:28:05.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.582+0000 7fe616a00640 1 -- 172.21.15.86:0/1385403447 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe61801c070 con 0x7fe620155d30 2024-04-20T21:28:05.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.583+0000 7fe616a00640 1 --2- 172.21.15.86:0/1385403447 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe5fc06c8f0 0x7fe5fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:05.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.583+0000 7fe616a00640 1 -- 172.21.15.86:0/1385403447 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7fe61808b8a0 con 0x7fe620155d30 2024-04-20T21:28:05.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.583+0000 7fe61de00640 1 --2- 172.21.15.86:0/1385403447 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe5fc06c8f0 0x7fe5fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:05.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.584+0000 7fe61de00640 1 --2- 172.21.15.86:0/1385403447 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe5fc06c8f0 0x7fe5fc06edb0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fe610038000 tx=0x7fe61003b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:05.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.587+0000 7fe616a00640 1 -- 172.21.15.86:0/1385403447 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe618059cc0 con 0x7fe620155d30 2024-04-20T21:28:05.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.790+0000 7fe624e00640 1 -- 172.21.15.86:0/1385403447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe62014d4d0 con 0x7fe620155d30 2024-04-20T21:28:05.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.790+0000 7fe616a00640 1 -- 172.21.15.86:0/1385403447 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) ==== 93+0+10 (secure 0 0 0) 0x7fe6180595c0 con 0x7fe620155d30 2024-04-20T21:28:05.791 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:28:05.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.793+0000 7fe624e00640 1 -- 172.21.15.86:0/1385403447 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe5fc06c8f0 msgr2=0x7fe5fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:05.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.793+0000 7fe624e00640 1 --2- 172.21.15.86:0/1385403447 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe5fc06c8f0 0x7fe5fc06edb0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fe610038000 tx=0x7fe61003b040 comp rx=0 tx=0).stop 2024-04-20T21:28:05.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.793+0000 7fe624e00640 1 -- 172.21.15.86:0/1385403447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe620155d30 msgr2=0x7fe62015baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:05.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.793+0000 7fe624e00640 1 --2- 172.21.15.86:0/1385403447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe620155d30 0x7fe62015baf0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7fe61800b970 tx=0x7fe61800be30 comp rx=0 tx=0).stop 2024-04-20T21:28:05.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.793+0000 7fe624e00640 1 -- 172.21.15.86:0/1385403447 shutdown_connections 2024-04-20T21:28:05.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.793+0000 7fe624e00640 1 --2- 172.21.15.86:0/1385403447 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe5fc06c8f0 0x7fe5fc06edb0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:05.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.793+0000 7fe624e00640 1 --2- 172.21.15.86:0/1385403447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe620155d30 0x7fe62015baf0 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:05.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.793+0000 7fe624e00640 1 --2- 172.21.15.86:0/1385403447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6200b52b0 0x7fe6201f79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:05.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.793+0000 7fe624e00640 1 --2- 172.21.15.86:0/1385403447 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe62006b980 0x7fe620156420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:05.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.793+0000 7fe624e00640 1 -- 172.21.15.86:0/1385403447 >> 172.21.15.86:0/1385403447 conn(0x7fe6200b2110 msgr2=0x7fe62014bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:05.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.793+0000 7fe624e00640 1 -- 172.21.15.86:0/1385403447 shutdown_connections 2024-04-20T21:28:05.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:05.793+0000 7fe624e00640 1 -- 172.21.15.86:0/1385403447 wait complete. 2024-04-20T21:28:05.811 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:28:06.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.012+0000 7f8a48a00640 1 Processor -- start 2024-04-20T21:28:06.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.012+0000 7f8a48a00640 1 -- start start 2024-04-20T21:28:06.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.014+0000 7f8a48a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8a44145f00 0x7f8a44071d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:06.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.014+0000 7f8a48a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a44160410 con 0x7f8a44072280 2024-04-20T21:28:06.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.014+0000 7f8a48a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a441606f0 con 0x7f8a44145250 2024-04-20T21:28:06.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.014+0000 7f8a48a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f8a440726b0 con 0x7f8a44145f00 2024-04-20T21:28:06.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.014+0000 7f8a43200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f8a44072280 0x7f8a4415cdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:47054/0 (socket says 172.21.15.86:47054) 2024-04-20T21:28:06.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.014+0000 7f8a43200640 1 -- 172.21.15.86:0/431960691 learned_addr learned my addr 172.21.15.86:0/431960691 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:28:06.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.014+0000 7f8a41e00640 1 --2- 172.21.15.86:0/431960691 >> v2:172.21.15.144:3300/0 conn(0x7f8a44145f00 0x7f8a44071d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:06.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.015+0000 7f8a41400640 1 -- 172.21.15.86:0/431960691 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3033668339 0 0) 0x7f8a44160410 con 0x7f8a44072280 2024-04-20T21:28:06.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.015+0000 7f8a41400640 1 -- 172.21.15.86:0/431960691 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8a28003520 con 0x7f8a44072280 2024-04-20T21:28:06.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.015+0000 7f8a41400640 1 -- 172.21.15.86:0/431960691 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 126444676 0 0) 0x7f8a28003520 con 0x7f8a44072280 2024-04-20T21:28:06.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.015+0000 7f8a41400640 1 -- 172.21.15.86:0/431960691 >> v2:172.21.15.144:3300/0 conn(0x7f8a44145f00 msgr2=0x7f8a44071d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:06.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.015+0000 7f8a41400640 1 --2- 172.21.15.86:0/431960691 >> v2:172.21.15.144:3300/0 conn(0x7f8a44145f00 0x7f8a44071d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:06.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.015+0000 7f8a41400640 1 -- 172.21.15.86:0/431960691 >> v1:172.21.15.144:6789/0 conn(0x7f8a44145250 legacy=0x7f8a4406efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:06.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.015+0000 7f8a41400640 1 -- 172.21.15.86:0/431960691 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a441609d0 con 0x7f8a44072280 2024-04-20T21:28:06.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.015+0000 7f8a41400640 1 -- 172.21.15.86:0/431960691 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f8a3c0019f0 con 0x7f8a44072280 2024-04-20T21:28:06.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.015+0000 7f8a41400640 1 -- 172.21.15.86:0/431960691 >> v1:172.21.15.86:6789/0 conn(0x7f8a44072280 legacy=0x7f8a4415cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:06.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.015+0000 7f8a41400640 1 --2- 172.21.15.86:0/431960691 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a28003a70 0x7f8a28003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:06.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.015+0000 7f8a41400640 1 --2- 172.21.15.86:0/431960691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a28004800 0x7f8a28004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:06.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.015+0000 7f8a41400640 1 --2- 172.21.15.86:0/431960691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a280052e0 0x7f8a2800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:06.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.015+0000 7f8a41400640 1 -- 172.21.15.86:0/431960691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8a440726b0 con 0x7f8a28004800 2024-04-20T21:28:06.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.015+0000 7f8a41400640 1 -- 172.21.15.86:0/431960691 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2800c1b0 con 0x7f8a280052e0 2024-04-20T21:28:06.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.015+0000 7f8a41400640 1 -- 172.21.15.86:0/431960691 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8a2800c3b0 con 0x7f8a28003a70 2024-04-20T21:28:06.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.016+0000 7f8a41e00640 1 --2- 172.21.15.86:0/431960691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a28004800 0x7f8a28004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:06.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.016+0000 7f8a42800640 1 --2- 172.21.15.86:0/431960691 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a28003a70 0x7f8a28003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:06.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.016+0000 7f8a43200640 1 --2- 172.21.15.86:0/431960691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a280052e0 0x7f8a2800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:06.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.016+0000 7f8a41e00640 1 -- 172.21.15.86:0/431960691 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a28003a70 msgr2=0x7f8a28003e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:06.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.016+0000 7f8a41e00640 1 --2- 172.21.15.86:0/431960691 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a28003a70 0x7f8a28003e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:06.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.016+0000 7f8a41e00640 1 -- 172.21.15.86:0/431960691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a280052e0 msgr2=0x7f8a2800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:06.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.016+0000 7f8a41e00640 1 --2- 172.21.15.86:0/431960691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a280052e0 0x7f8a2800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:06.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.016+0000 7f8a41e00640 1 -- 172.21.15.86:0/431960691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a28003520 con 0x7f8a28004800 2024-04-20T21:28:06.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.016+0000 7f8a43200640 1 --2- 172.21.15.86:0/431960691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a280052e0 0x7f8a2800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:28:06.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.016+0000 7f8a43200640 1 -- 172.21.15.86:0/431960691 reap_dead start 2024-04-20T21:28:06.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.017+0000 7f8a41e00640 1 --2- 172.21.15.86:0/431960691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a28004800 0x7f8a28004c80 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f8a30009ac0 tx=0x7f8a28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=575cb299b031ceba server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:06.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.017+0000 7f8a41400640 1 -- 172.21.15.86:0/431960691 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a30039070 con 0x7f8a28004800 2024-04-20T21:28:06.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.017+0000 7f8a41400640 1 -- 172.21.15.86:0/431960691 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a3002c6b0 con 0x7f8a28004800 2024-04-20T21:28:06.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.017+0000 7f8a41400640 1 -- 172.21.15.86:0/431960691 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a30009690 con 0x7f8a28004800 2024-04-20T21:28:06.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.018+0000 7f8a48a00640 1 -- 172.21.15.86:0/431960691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a28004800 msgr2=0x7f8a28004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:06.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.018+0000 7f8a48a00640 1 --2- 172.21.15.86:0/431960691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a28004800 0x7f8a28004c80 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f8a30009ac0 tx=0x7f8a28005830 comp rx=0 tx=0).stop 2024-04-20T21:28:06.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.018+0000 7f8a48a00640 1 -- 172.21.15.86:0/431960691 shutdown_connections 2024-04-20T21:28:06.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.018+0000 7f8a48a00640 1 --2- 172.21.15.86:0/431960691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a28004800 0x7f8a28004c80 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:06.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.018+0000 7f8a48a00640 1 -- 172.21.15.86:0/431960691 >> 172.21.15.86:0/431960691 conn(0x7f8a44067cf0 msgr2=0x7f8a4406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:06.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.018+0000 7f8a48a00640 1 -- 172.21.15.86:0/431960691 shutdown_connections 2024-04-20T21:28:06.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.018+0000 7f8a48a00640 1 -- 172.21.15.86:0/431960691 wait complete. 2024-04-20T21:28:06.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.021+0000 7f8a48a00640 1 Processor -- start 2024-04-20T21:28:06.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.021+0000 7f8a48a00640 1 -- start start 2024-04-20T21:28:06.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.022+0000 7f8a48a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a28004800 0x7f8a4415be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:06.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.022+0000 7f8a48a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a44139360 0x7f8a4413aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:06.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.022+0000 7f8a48a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a4413b000 0x7f8a4415d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:06.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.022+0000 7f8a48a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8a441457b0 con 0x7f8a28004800 2024-04-20T21:28:06.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.022+0000 7f8a48a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8a44145630 con 0x7f8a44139360 2024-04-20T21:28:06.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.022+0000 7f8a48a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8a44071d40 con 0x7f8a4413b000 2024-04-20T21:28:06.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.022+0000 7f8a42800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a28004800 0x7f8a4415be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:06.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.022+0000 7f8a42800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a28004800 0x7f8a4415be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40812/0 (socket says 172.21.15.86:40812) 2024-04-20T21:28:06.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.022+0000 7f8a42800640 1 -- 172.21.15.86:0/592222307 learned_addr learned my addr 172.21.15.86:0/592222307 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:06.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.023+0000 7f8a43200640 1 --2- 172.21.15.86:0/592222307 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a4413b000 0x7f8a4415d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:06.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.023+0000 7f8a41e00640 1 --2- 172.21.15.86:0/592222307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a44139360 0x7f8a4413aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:06.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.023+0000 7f8a42800640 1 -- 172.21.15.86:0/592222307 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a4413b000 msgr2=0x7f8a4415d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:06.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.023+0000 7f8a42800640 1 --2- 172.21.15.86:0/592222307 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a4413b000 0x7f8a4415d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:06.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.023+0000 7f8a42800640 1 -- 172.21.15.86:0/592222307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a44139360 msgr2=0x7f8a4413aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:06.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.023+0000 7f8a42800640 1 --2- 172.21.15.86:0/592222307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a44139360 0x7f8a4413aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:06.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.023+0000 7f8a42800640 1 -- 172.21.15.86:0/592222307 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a4413b580 con 0x7f8a28004800 2024-04-20T21:28:06.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.023+0000 7f8a41e00640 1 --2- 172.21.15.86:0/592222307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a44139360 0x7f8a4413aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:28:06.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.023+0000 7f8a43200640 1 --2- 172.21.15.86:0/592222307 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a4413b000 0x7f8a4415d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:28:06.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.023+0000 7f8a42800640 1 --2- 172.21.15.86:0/592222307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a28004800 0x7f8a4415be90 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f8a3400e440 tx=0x7f8a3400e840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:06.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.023+0000 7f8a3aa00640 1 -- 172.21.15.86:0/592222307 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a34019070 con 0x7f8a28004800 2024-04-20T21:28:06.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.023+0000 7f8a3aa00640 1 -- 172.21.15.86:0/592222307 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a340052e0 con 0x7f8a28004800 2024-04-20T21:28:06.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.023+0000 7f8a3aa00640 1 -- 172.21.15.86:0/592222307 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a34003830 con 0x7f8a28004800 2024-04-20T21:28:06.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.023+0000 7f8a48a00640 1 -- 172.21.15.86:0/592222307 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a4415b3b0 con 0x7f8a28004800 2024-04-20T21:28:06.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.023+0000 7f8a48a00640 1 -- 172.21.15.86:0/592222307 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a4415b970 con 0x7f8a28004800 2024-04-20T21:28:06.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.025+0000 7f8a48a00640 1 -- 172.21.15.86:0/592222307 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a4404fd90 con 0x7f8a28004800 2024-04-20T21:28:06.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.025+0000 7f8a3aa00640 1 -- 172.21.15.86:0/592222307 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8a3401c070 con 0x7f8a28004800 2024-04-20T21:28:06.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.026+0000 7f8a3aa00640 1 --2- 172.21.15.86:0/592222307 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8a2406c920 0x7f8a2406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:06.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.026+0000 7f8a3aa00640 1 -- 172.21.15.86:0/592222307 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7f8a3408b0a0 con 0x7f8a28004800 2024-04-20T21:28:06.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.027+0000 7f8a41e00640 1 --2- 172.21.15.86:0/592222307 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8a2406c920 0x7f8a2406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:06.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.027+0000 7f8a41e00640 1 --2- 172.21.15.86:0/592222307 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8a2406c920 0x7f8a2406ede0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f8a30036000 tx=0x7f8a30030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:06.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.031+0000 7f8a3aa00640 1 -- 172.21.15.86:0/592222307 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8a34058e70 con 0x7f8a28004800 2024-04-20T21:28:06.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.228+0000 7f8a48a00640 1 -- 172.21.15.86:0/592222307 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8a44072ea0 con 0x7f8a28004800 2024-04-20T21:28:06.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.229+0000 7f8a3aa00640 1 -- 172.21.15.86:0/592222307 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v100) ==== 104+0+11 (secure 0 0 0) 0x7f8a3405d170 con 0x7f8a28004800 2024-04-20T21:28:06.229 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-20T21:28:06.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.231+0000 7f8a48a00640 1 -- 172.21.15.86:0/592222307 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8a2406c920 msgr2=0x7f8a2406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:06.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.231+0000 7f8a48a00640 1 --2- 172.21.15.86:0/592222307 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8a2406c920 0x7f8a2406ede0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f8a30036000 tx=0x7f8a30030000 comp rx=0 tx=0).stop 2024-04-20T21:28:06.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.231+0000 7f8a48a00640 1 -- 172.21.15.86:0/592222307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a28004800 msgr2=0x7f8a4415be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:06.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.231+0000 7f8a48a00640 1 --2- 172.21.15.86:0/592222307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a28004800 0x7f8a4415be90 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f8a3400e440 tx=0x7f8a3400e840 comp rx=0 tx=0).stop 2024-04-20T21:28:06.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.231+0000 7f8a48a00640 1 -- 172.21.15.86:0/592222307 shutdown_connections 2024-04-20T21:28:06.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.231+0000 7f8a48a00640 1 --2- 172.21.15.86:0/592222307 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8a2406c920 0x7f8a2406ede0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:06.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.231+0000 7f8a48a00640 1 --2- 172.21.15.86:0/592222307 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8a4413b000 0x7f8a4415d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:06.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.231+0000 7f8a48a00640 1 --2- 172.21.15.86:0/592222307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a44139360 0x7f8a4413aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:06.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.231+0000 7f8a48a00640 1 --2- 172.21.15.86:0/592222307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a28004800 0x7f8a4415be90 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:06.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.231+0000 7f8a48a00640 1 -- 172.21.15.86:0/592222307 >> 172.21.15.86:0/592222307 conn(0x7f8a44067cf0 msgr2=0x7f8a4415ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:06.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.231+0000 7f8a48a00640 1 -- 172.21.15.86:0/592222307 shutdown_connections 2024-04-20T21:28:06.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.231+0000 7f8a48a00640 1 -- 172.21.15.86:0/592222307 wait complete. 2024-04-20T21:28:06.248 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:28:06.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.443+0000 7fb515600640 1 Processor -- start 2024-04-20T21:28:06.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.444+0000 7fb515600640 1 -- start start 2024-04-20T21:28:06.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.445+0000 7fb515600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb5100d4310 0x7fb5100d4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:06.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.445+0000 7fb515600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5100d4cb0 con 0x7fb5100cfb80 2024-04-20T21:28:06.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.445+0000 7fb515600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5100d2ff0 con 0x7fb51015e870 2024-04-20T21:28:06.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.445+0000 7fb515600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb51006d590 con 0x7fb5100d4310 2024-04-20T21:28:06.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.445+0000 7fb50f400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb5100d4310 0x7fb5100d4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:06.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.445+0000 7fb514c00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fb5100cfb80 0x7fb5100d0010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:47068/0 (socket says 172.21.15.86:47068) 2024-04-20T21:28:06.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.445+0000 7fb514c00640 1 -- 172.21.15.86:0/4047745913 learned_addr learned my addr 172.21.15.86:0/4047745913 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:28:06.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.446+0000 7fb50f400640 1 -- 172.21.15.86:0/4047745913 >> v1:172.21.15.144:6789/0 conn(0x7fb51015e870 legacy=0x7fb51006ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:06.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.446+0000 7fb50f400640 1 -- 172.21.15.86:0/4047745913 >> v1:172.21.15.86:6789/0 conn(0x7fb5100cfb80 legacy=0x7fb5100d0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:06.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.446+0000 7fb50ea00640 1 -- 172.21.15.86:0/4047745913 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 913017865 0 0) 0x7fb5100d4cb0 con 0x7fb5100cfb80 2024-04-20T21:28:06.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.446+0000 7fb50f400640 1 -- 172.21.15.86:0/4047745913 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5100d32d0 con 0x7fb5100d4310 2024-04-20T21:28:06.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.447+0000 7fb50f400640 1 --2- 172.21.15.86:0/4047745913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5100d4310 0x7fb5100d4770 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8009d90 tx=0x7fb4f802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a187ab989e095676 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:06.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.447+0000 7fb50ea00640 1 -- 172.21.15.86:0/4047745913 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f8039070 con 0x7fb5100d4310 2024-04-20T21:28:06.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.447+0000 7fb50ea00640 1 -- 172.21.15.86:0/4047745913 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4f802cc60 con 0x7fb5100d4310 2024-04-20T21:28:06.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.447+0000 7fb50ea00640 1 -- 172.21.15.86:0/4047745913 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f8034660 con 0x7fb5100d4310 2024-04-20T21:28:06.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.448+0000 7fb515600640 1 -- 172.21.15.86:0/4047745913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5100d4310 msgr2=0x7fb5100d4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:06.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.448+0000 7fb515600640 1 --2- 172.21.15.86:0/4047745913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5100d4310 0x7fb5100d4770 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8009d90 tx=0x7fb4f802beb0 comp rx=0 tx=0).stop 2024-04-20T21:28:06.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.448+0000 7fb515600640 1 -- 172.21.15.86:0/4047745913 shutdown_connections 2024-04-20T21:28:06.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.448+0000 7fb515600640 1 --2- 172.21.15.86:0/4047745913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5100d4310 0x7fb5100d4770 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:06.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.448+0000 7fb515600640 1 -- 172.21.15.86:0/4047745913 >> 172.21.15.86:0/4047745913 conn(0x7fb51009d9c0 msgr2=0x7fb51008bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:06.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.448+0000 7fb515600640 1 -- 172.21.15.86:0/4047745913 shutdown_connections 2024-04-20T21:28:06.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.448+0000 7fb515600640 1 -- 172.21.15.86:0/4047745913 wait complete. 2024-04-20T21:28:06.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.450+0000 7fb515600640 1 Processor -- start 2024-04-20T21:28:06.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.450+0000 7fb515600640 1 -- start start 2024-04-20T21:28:06.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.451+0000 7fb515600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5100cfb80 0x7fb5101556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:06.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.451+0000 7fb515600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5100d4310 0x7fb5100b68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:06.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.451+0000 7fb515600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb51015e870 0x7fb5100b40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:06.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.451+0000 7fb515600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb5100d35b0 con 0x7fb5100cfb80 2024-04-20T21:28:06.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.451+0000 7fb515600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb5100cf660 con 0x7fb51015e870 2024-04-20T21:28:06.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.451+0000 7fb515600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb51015ec50 con 0x7fb5100d4310 2024-04-20T21:28:06.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.451+0000 7fb50fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5100cfb80 0x7fb5101556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:06.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.451+0000 7fb50fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5100cfb80 0x7fb5101556e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40836/0 (socket says 172.21.15.86:40836) 2024-04-20T21:28:06.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.451+0000 7fb50fe00640 1 -- 172.21.15.86:0/906053638 learned_addr learned my addr 172.21.15.86:0/906053638 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:06.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.451+0000 7fb50fe00640 1 -- 172.21.15.86:0/906053638 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5100d4310 msgr2=0x7fb5100b68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:06.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.451+0000 7fb50fe00640 1 --2- 172.21.15.86:0/906053638 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5100d4310 0x7fb5100b68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:06.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.451+0000 7fb50fe00640 1 -- 172.21.15.86:0/906053638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb51015e870 msgr2=0x7fb5100b40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:06.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.451+0000 7fb50fe00640 1 --2- 172.21.15.86:0/906053638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb51015e870 0x7fb5100b40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:06.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.451+0000 7fb50fe00640 1 -- 172.21.15.86:0/906053638 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5100b6e30 con 0x7fb5100cfb80 2024-04-20T21:28:06.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.452+0000 7fb50fe00640 1 --2- 172.21.15.86:0/906053638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5100cfb80 0x7fb5101556e0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fb50000c460 tx=0x7fb50000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:06.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.452+0000 7fb507e00640 1 -- 172.21.15.86:0/906053638 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb500016020 con 0x7fb5100cfb80 2024-04-20T21:28:06.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.452+0000 7fb507e00640 1 -- 172.21.15.86:0/906053638 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb500012870 con 0x7fb5100cfb80 2024-04-20T21:28:06.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.452+0000 7fb515600640 1 -- 172.21.15.86:0/906053638 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb510160d20 con 0x7fb5100cfb80 2024-04-20T21:28:06.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.452+0000 7fb507e00640 1 -- 172.21.15.86:0/906053638 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb500012c60 con 0x7fb5100cfb80 2024-04-20T21:28:06.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.452+0000 7fb515600640 1 -- 172.21.15.86:0/906053638 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5101612e0 con 0x7fb5100cfb80 2024-04-20T21:28:06.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.453+0000 7fb515600640 1 -- 172.21.15.86:0/906053638 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb51004fd90 con 0x7fb5100cfb80 2024-04-20T21:28:06.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.454+0000 7fb507e00640 1 -- 172.21.15.86:0/906053638 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb500019020 con 0x7fb5100cfb80 2024-04-20T21:28:06.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.455+0000 7fb507e00640 1 --2- 172.21.15.86:0/906053638 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb4ec06c8f0 0x7fb4ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:06.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.456+0000 7fb507e00640 1 -- 172.21.15.86:0/906053638 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7fb50008b510 con 0x7fb5100cfb80 2024-04-20T21:28:06.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.456+0000 7fb50f400640 1 --2- 172.21.15.86:0/906053638 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb4ec06c8f0 0x7fb4ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:06.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.456+0000 7fb50f400640 1 --2- 172.21.15.86:0/906053638 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb4ec06c8f0 0x7fb4ec06edb0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8036000 tx=0x7fb4f802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:06.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.460+0000 7fb507e00640 1 -- 172.21.15.86:0/906053638 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb500020070 con 0x7fb5100cfb80 2024-04-20T21:28:06.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.659+0000 7fb515600640 1 -- 172.21.15.86:0/906053638 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb510156330 con 0x7fb5100cfb80 2024-04-20T21:28:06.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.660+0000 7fb507e00640 1 -- 172.21.15.86:0/906053638 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v100) ==== 100+0+11 (secure 0 0 0) 0x7fb5100b6e30 con 0x7fb5100cfb80 2024-04-20T21:28:06.660 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-20T21:28:06.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.662+0000 7fb515600640 1 -- 172.21.15.86:0/906053638 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb4ec06c8f0 msgr2=0x7fb4ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:06.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.662+0000 7fb515600640 1 --2- 172.21.15.86:0/906053638 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb4ec06c8f0 0x7fb4ec06edb0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8036000 tx=0x7fb4f802e000 comp rx=0 tx=0).stop 2024-04-20T21:28:06.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.662+0000 7fb515600640 1 -- 172.21.15.86:0/906053638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5100cfb80 msgr2=0x7fb5101556e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:06.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.662+0000 7fb515600640 1 --2- 172.21.15.86:0/906053638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5100cfb80 0x7fb5101556e0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fb50000c460 tx=0x7fb50000c920 comp rx=0 tx=0).stop 2024-04-20T21:28:06.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.663+0000 7fb515600640 1 -- 172.21.15.86:0/906053638 shutdown_connections 2024-04-20T21:28:06.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.663+0000 7fb515600640 1 --2- 172.21.15.86:0/906053638 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb4ec06c8f0 0x7fb4ec06edb0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:06.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.663+0000 7fb515600640 1 --2- 172.21.15.86:0/906053638 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb51015e870 0x7fb5100b40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:06.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.663+0000 7fb515600640 1 --2- 172.21.15.86:0/906053638 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb5100d4310 0x7fb5100b68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:06.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.663+0000 7fb515600640 1 --2- 172.21.15.86:0/906053638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5100cfb80 0x7fb5101556e0 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:06.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.663+0000 7fb515600640 1 -- 172.21.15.86:0/906053638 >> 172.21.15.86:0/906053638 conn(0x7fb51009d9c0 msgr2=0x7fb5100d0010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:06.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.663+0000 7fb515600640 1 -- 172.21.15.86:0/906053638 shutdown_connections 2024-04-20T21:28:06.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.663+0000 7fb515600640 1 -- 172.21.15.86:0/906053638 wait complete. 2024-04-20T21:28:06.680 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T21:28:06.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.868+0000 7fd453e00640 1 Processor -- start 2024-04-20T21:28:06.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.868+0000 7fd453e00640 1 -- start start 2024-04-20T21:28:06.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.869+0000 7fd453e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd45415c590 0x7fd4540ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:06.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.869+0000 7fd453e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd4540af7e0 0x7fd4540cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:06.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.869+0000 7fd453e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd45406daf0 0x7fd4540d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:06.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.869+0000 7fd453e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd454160160 con 0x7fd4540af7e0 2024-04-20T21:28:06.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.869+0000 7fd453e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fd454160440 con 0x7fd45406daf0 2024-04-20T21:28:06.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.869+0000 7fd453e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd454160720 con 0x7fd45415c590 2024-04-20T21:28:06.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.870+0000 7fd44be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd4540af7e0 0x7fd4540cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:06.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.870+0000 7fd44be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd4540af7e0 0x7fd4540cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40842/0 (socket says 172.21.15.86:40842) 2024-04-20T21:28:06.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.870+0000 7fd44be00640 1 -- 172.21.15.86:0/3428574852 learned_addr learned my addr 172.21.15.86:0/3428574852 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:06.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.870+0000 7fd452a00640 1 --2- 172.21.15.86:0/3428574852 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd45415c590 0x7fd4540ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:06.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.870+0000 7fd44be00640 1 -- 172.21.15.86:0/3428574852 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd45415c590 msgr2=0x7fd4540ccb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:06.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.870+0000 7fd453400640 1 --2- 172.21.15.86:0/3428574852 >> v2:172.21.15.144:3300/0 conn(0x7fd45406daf0 0x7fd4540d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:06.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.870+0000 7fd44be00640 1 --2- 172.21.15.86:0/3428574852 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd45415c590 0x7fd4540ccb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:06.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.870+0000 7fd44be00640 1 -- 172.21.15.86:0/3428574852 >> v2:172.21.15.144:3300/0 conn(0x7fd45406daf0 msgr2=0x7fd4540d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:06.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.870+0000 7fd44be00640 1 --2- 172.21.15.86:0/3428574852 >> v2:172.21.15.144:3300/0 conn(0x7fd45406daf0 0x7fd4540d00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:06.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.870+0000 7fd44be00640 1 -- 172.21.15.86:0/3428574852 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd454160a00 con 0x7fd4540af7e0 2024-04-20T21:28:06.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.871+0000 7fd44be00640 1 --2- 172.21.15.86:0/3428574852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4540af7e0 0x7fd4540cd050 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fd4540c71a0 tx=0x7fd43c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ae419f73f39e7c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:06.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.872+0000 7fd452000640 1 -- 172.21.15.86:0/3428574852 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c035020 con 0x7fd4540af7e0 2024-04-20T21:28:06.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.872+0000 7fd452000640 1 -- 172.21.15.86:0/3428574852 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd43c031870 con 0x7fd4540af7e0 2024-04-20T21:28:06.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.872+0000 7fd452000640 1 -- 172.21.15.86:0/3428574852 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c031c40 con 0x7fd4540af7e0 2024-04-20T21:28:06.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.872+0000 7fd453e00640 1 -- 172.21.15.86:0/3428574852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4540af7e0 msgr2=0x7fd4540cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:06.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.872+0000 7fd453e00640 1 --2- 172.21.15.86:0/3428574852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4540af7e0 0x7fd4540cd050 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fd4540c71a0 tx=0x7fd43c02b910 comp rx=0 tx=0).stop 2024-04-20T21:28:06.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.872+0000 7fd453e00640 1 -- 172.21.15.86:0/3428574852 shutdown_connections 2024-04-20T21:28:06.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.872+0000 7fd453e00640 1 --2- 172.21.15.86:0/3428574852 >> v2:172.21.15.144:3300/0 conn(0x7fd45406daf0 0x7fd4540d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:06.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.872+0000 7fd453e00640 1 --2- 172.21.15.86:0/3428574852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd4540af7e0 0x7fd4540cd050 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:06.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.872+0000 7fd453e00640 1 --2- 172.21.15.86:0/3428574852 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd45415c590 0x7fd4540ccb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:06.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.872+0000 7fd453e00640 1 -- 172.21.15.86:0/3428574852 >> 172.21.15.86:0/3428574852 conn(0x7fd45406bb60 msgr2=0x7fd45406bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:06.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.873+0000 7fd453e00640 1 -- 172.21.15.86:0/3428574852 shutdown_connections 2024-04-20T21:28:06.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.873+0000 7fd453e00640 1 -- 172.21.15.86:0/3428574852 wait complete. 2024-04-20T21:28:06.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.874+0000 7fd453e00640 1 Processor -- start 2024-04-20T21:28:06.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.874+0000 7fd453e00640 1 -- start start 2024-04-20T21:28:06.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.875+0000 7fd453e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd45406daf0 0x7fd454159eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:06.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.875+0000 7fd453e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4540af7e0 0x7fd454073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:06.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.875+0000 7fd453e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd45415c590 0x7fd454155df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:06.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.875+0000 7fd453e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd45415c970 con 0x7fd45406daf0 2024-04-20T21:28:06.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.875+0000 7fd453e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd45406f3d0 con 0x7fd45415c590 2024-04-20T21:28:06.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.875+0000 7fd453e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd454160fc0 con 0x7fd4540af7e0 2024-04-20T21:28:06.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.875+0000 7fd452a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd45406daf0 0x7fd454159eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:06.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.875+0000 7fd452a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd45406daf0 0x7fd454159eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40852/0 (socket says 172.21.15.86:40852) 2024-04-20T21:28:06.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.875+0000 7fd452a00640 1 -- 172.21.15.86:0/1487834816 learned_addr learned my addr 172.21.15.86:0/1487834816 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:06.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.876+0000 7fd452a00640 1 -- 172.21.15.86:0/1487834816 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4540af7e0 msgr2=0x7fd454073480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:06.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.876+0000 7fd452a00640 1 --2- 172.21.15.86:0/1487834816 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4540af7e0 0x7fd454073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:06.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.876+0000 7fd452a00640 1 -- 172.21.15.86:0/1487834816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd45415c590 msgr2=0x7fd454155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:06.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.876+0000 7fd452a00640 1 --2- 172.21.15.86:0/1487834816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd45415c590 0x7fd454155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:06.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.876+0000 7fd452a00640 1 -- 172.21.15.86:0/1487834816 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd45415a3f0 con 0x7fd45406daf0 2024-04-20T21:28:06.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.876+0000 7fd452a00640 1 --2- 172.21.15.86:0/1487834816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd45406daf0 0x7fd454159eb0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fd44400c820 tx=0x7fd44400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:06.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.876+0000 7fd44aa00640 1 -- 172.21.15.86:0/1487834816 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd444015070 con 0x7fd45406daf0 2024-04-20T21:28:06.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.876+0000 7fd44aa00640 1 -- 172.21.15.86:0/1487834816 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd444004020 con 0x7fd45406daf0 2024-04-20T21:28:06.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.876+0000 7fd44aa00640 1 -- 172.21.15.86:0/1487834816 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4440043f0 con 0x7fd45406daf0 2024-04-20T21:28:06.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.876+0000 7fd453e00640 1 -- 172.21.15.86:0/1487834816 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd45415a690 con 0x7fd45406daf0 2024-04-20T21:28:06.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.877+0000 7fd453e00640 1 -- 172.21.15.86:0/1487834816 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd45415ac50 con 0x7fd45406daf0 2024-04-20T21:28:06.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.877+0000 7fd453e00640 1 -- 172.21.15.86:0/1487834816 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd45404fcf0 con 0x7fd45406daf0 2024-04-20T21:28:06.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.878+0000 7fd44aa00640 1 -- 172.21.15.86:0/1487834816 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd444007830 con 0x7fd45406daf0 2024-04-20T21:28:06.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.879+0000 7fd44aa00640 1 --2- 172.21.15.86:0/1487834816 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd43006c8f0 0x7fd43006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:06.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.879+0000 7fd44aa00640 1 -- 172.21.15.86:0/1487834816 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7fd44408b980 con 0x7fd45406daf0 2024-04-20T21:28:06.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.880+0000 7fd44be00640 1 --2- 172.21.15.86:0/1487834816 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd43006c8f0 0x7fd43006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:06.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.880+0000 7fd44be00640 1 --2- 172.21.15.86:0/1487834816 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd43006c8f0 0x7fd43006edb0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fd43c036000 tx=0x7fd43c030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:06.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:06.884+0000 7fd44aa00640 1 -- 172.21.15.86:0/1487834816 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd444059d10 con 0x7fd45406daf0 2024-04-20T21:28:07.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.082+0000 7fd453e00640 1 -- 172.21.15.86:0/1487834816 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd4540ce580 con 0x7fd45406daf0 2024-04-20T21:28:07.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.083+0000 7fd44aa00640 1 -- 172.21.15.86:0/1487834816 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v100) ==== 80+0+0 (secure 0 0 0) 0x7fd444059610 con 0x7fd45406daf0 2024-04-20T21:28:07.082 INFO:teuthology.orchestra.run.smithi086.stderr:listed 0 entries 2024-04-20T21:28:07.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.085+0000 7fd453e00640 1 -- 172.21.15.86:0/1487834816 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd43006c8f0 msgr2=0x7fd43006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:07.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.085+0000 7fd453e00640 1 --2- 172.21.15.86:0/1487834816 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd43006c8f0 0x7fd43006edb0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fd43c036000 tx=0x7fd43c030000 comp rx=0 tx=0).stop 2024-04-20T21:28:07.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.085+0000 7fd453e00640 1 -- 172.21.15.86:0/1487834816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd45406daf0 msgr2=0x7fd454159eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:07.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.085+0000 7fd453e00640 1 --2- 172.21.15.86:0/1487834816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd45406daf0 0x7fd454159eb0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fd44400c820 tx=0x7fd44400cce0 comp rx=0 tx=0).stop 2024-04-20T21:28:07.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.085+0000 7fd453e00640 1 -- 172.21.15.86:0/1487834816 shutdown_connections 2024-04-20T21:28:07.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.085+0000 7fd453e00640 1 --2- 172.21.15.86:0/1487834816 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd43006c8f0 0x7fd43006edb0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:07.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.085+0000 7fd453e00640 1 --2- 172.21.15.86:0/1487834816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd45415c590 0x7fd454155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:07.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.085+0000 7fd453e00640 1 --2- 172.21.15.86:0/1487834816 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd4540af7e0 0x7fd454073480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:07.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.085+0000 7fd453e00640 1 --2- 172.21.15.86:0/1487834816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd45406daf0 0x7fd454159eb0 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:07.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.085+0000 7fd453e00640 1 -- 172.21.15.86:0/1487834816 >> 172.21.15.86:0/1487834816 conn(0x7fd45406bb60 msgr2=0x7fd45406e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:07.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.086+0000 7fd453e00640 1 -- 172.21.15.86:0/1487834816 shutdown_connections 2024-04-20T21:28:07.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.086+0000 7fd453e00640 1 -- 172.21.15.86:0/1487834816 wait complete. 2024-04-20T21:28:07.104 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:28:07.104 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:28:07.150 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:28:07.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.390+0000 7f9cb6600640 1 Processor -- start 2024-04-20T21:28:07.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.390+0000 7f9cb6600640 1 -- start start 2024-04-20T21:28:07.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.391+0000 7f9cb6600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9cb00d11d0 0x7f9cb00d1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:07.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.391+0000 7f9cb6600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cb00cff70 con 0x7f9cb00cf410 2024-04-20T21:28:07.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.391+0000 7f9cb6600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cb00d01e0 con 0x7f9cb0162000 2024-04-20T21:28:07.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.391+0000 7f9cb6600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9cb00d1b40 con 0x7f9cb00d11d0 2024-04-20T21:28:07.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.392+0000 7f9cafe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f9cb00cf410 0x7f9cb00cf860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:47076/0 (socket says 172.21.15.86:47076) 2024-04-20T21:28:07.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.392+0000 7f9cafe00640 1 -- 172.21.15.86:0/2635055702 learned_addr learned my addr 172.21.15.86:0/2635055702 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:28:07.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.392+0000 7f9caea00640 1 --2- 172.21.15.86:0/2635055702 >> v2:172.21.15.86:3300/0 conn(0x7f9cb00d11d0 0x7f9cb00d1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:07.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.392+0000 7f9caea00640 1 -- 172.21.15.86:0/2635055702 >> v1:172.21.15.144:6789/0 conn(0x7f9cb0162000 legacy=0x7f9cb00b76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:07.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.392+0000 7f9caea00640 1 -- 172.21.15.86:0/2635055702 >> v1:172.21.15.86:6789/0 conn(0x7f9cb00cf410 legacy=0x7f9cb00cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:07.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.392+0000 7f9caea00640 1 -- 172.21.15.86:0/2635055702 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cb00d04a0 con 0x7f9cb00d11d0 2024-04-20T21:28:07.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.393+0000 7f9caea00640 1 --2- 172.21.15.86:0/2635055702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cb00d11d0 0x7f9cb00d1600 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f9c98009d90 tx=0x7f9c9802bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea4ee2a698cc7a1b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:07.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.394+0000 7f9cae000640 1 -- 172.21.15.86:0/2635055702 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c98039070 con 0x7f9cb00d11d0 2024-04-20T21:28:07.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.394+0000 7f9cae000640 1 -- 172.21.15.86:0/2635055702 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c9802cb40 con 0x7f9cb00d11d0 2024-04-20T21:28:07.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.394+0000 7f9cae000640 1 -- 172.21.15.86:0/2635055702 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c98034440 con 0x7f9cb00d11d0 2024-04-20T21:28:07.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.395+0000 7f9cb6600640 1 -- 172.21.15.86:0/2635055702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cb00d11d0 msgr2=0x7f9cb00d1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:07.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.395+0000 7f9cb6600640 1 --2- 172.21.15.86:0/2635055702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cb00d11d0 0x7f9cb00d1600 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f9c98009d90 tx=0x7f9c9802bd90 comp rx=0 tx=0).stop 2024-04-20T21:28:07.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.395+0000 7f9cb6600640 1 -- 172.21.15.86:0/2635055702 shutdown_connections 2024-04-20T21:28:07.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.395+0000 7f9cb6600640 1 --2- 172.21.15.86:0/2635055702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cb00d11d0 0x7f9cb00d1600 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:07.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.395+0000 7f9cb6600640 1 -- 172.21.15.86:0/2635055702 >> 172.21.15.86:0/2635055702 conn(0x7f9cb008bf70 msgr2=0x7f9cb00d4870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:07.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.395+0000 7f9cb6600640 1 -- 172.21.15.86:0/2635055702 shutdown_connections 2024-04-20T21:28:07.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.395+0000 7f9cb6600640 1 -- 172.21.15.86:0/2635055702 wait complete. 2024-04-20T21:28:07.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.396+0000 7f9cb6600640 1 Processor -- start 2024-04-20T21:28:07.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.397+0000 7f9cb6600640 1 -- start start 2024-04-20T21:28:07.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.398+0000 7f9cb6600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9cb00cf410 0x7f9cb00b8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:07.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.398+0000 7f9cb6600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cb00d11d0 0x7f9cb01479c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:07.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.398+0000 7f9cb6600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cb0162000 0x7f9cb00b9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:07.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.398+0000 7f9cb6600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb01623e0 con 0x7f9cb0162000 2024-04-20T21:28:07.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.398+0000 7f9cb6600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb00d2440 con 0x7f9cb00d11d0 2024-04-20T21:28:07.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.398+0000 7f9cb6600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9cb0162560 con 0x7f9cb00cf410 2024-04-20T21:28:07.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.398+0000 7f9cafe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cb0162000 0x7f9cb00b9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:07.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.398+0000 7f9cafe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cb0162000 0x7f9cb00b9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40876/0 (socket says 172.21.15.86:40876) 2024-04-20T21:28:07.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.398+0000 7f9cafe00640 1 -- 172.21.15.86:0/3054435160 learned_addr learned my addr 172.21.15.86:0/3054435160 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:07.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.398+0000 7f9caf400640 1 --2- 172.21.15.86:0/3054435160 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9cb00cf410 0x7f9cb00b8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:07.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.398+0000 7f9cafe00640 1 -- 172.21.15.86:0/3054435160 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9cb00cf410 msgr2=0x7f9cb00b8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:07.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.398+0000 7f9cafe00640 1 --2- 172.21.15.86:0/3054435160 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9cb00cf410 0x7f9cb00b8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:07.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.398+0000 7f9cafe00640 1 -- 172.21.15.86:0/3054435160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cb00d11d0 msgr2=0x7f9cb01479c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:07.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.398+0000 7f9cafe00640 1 --2- 172.21.15.86:0/3054435160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cb00d11d0 0x7f9cb01479c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:07.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.399+0000 7f9cafe00640 1 -- 172.21.15.86:0/3054435160 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cb0157b10 con 0x7f9cb0162000 2024-04-20T21:28:07.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.399+0000 7f9cafe00640 1 --2- 172.21.15.86:0/3054435160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cb0162000 0x7f9cb00b9170 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f9ca000c370 tx=0x7f9ca000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:07.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.399+0000 7f9ca7400640 1 -- 172.21.15.86:0/3054435160 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca0017070 con 0x7f9cb0162000 2024-04-20T21:28:07.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.400+0000 7f9ca7400640 1 -- 172.21.15.86:0/3054435160 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ca0005ce0 con 0x7f9cb0162000 2024-04-20T21:28:07.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.400+0000 7f9ca7400640 1 -- 172.21.15.86:0/3054435160 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca00072d0 con 0x7f9cb0162000 2024-04-20T21:28:07.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.400+0000 7f9cb6600640 1 -- 172.21.15.86:0/3054435160 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cb01f5070 con 0x7f9cb0162000 2024-04-20T21:28:07.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.400+0000 7f9cb6600640 1 -- 172.21.15.86:0/3054435160 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cb0159c60 con 0x7f9cb0162000 2024-04-20T21:28:07.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.401+0000 7f9cb6600640 1 -- 172.21.15.86:0/3054435160 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cb01f5460 con 0x7f9cb0162000 2024-04-20T21:28:07.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.402+0000 7f9ca7400640 1 -- 172.21.15.86:0/3054435160 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9ca001a070 con 0x7f9cb0162000 2024-04-20T21:28:07.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.403+0000 7f9ca7400640 1 --2- 172.21.15.86:0/3054435160 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9c8c06c8f0 0x7f9c8c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:07.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.403+0000 7f9ca7400640 1 -- 172.21.15.86:0/3054435160 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7f9ca008ad20 con 0x7f9cb0162000 2024-04-20T21:28:07.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.403+0000 7f9caf400640 1 --2- 172.21.15.86:0/3054435160 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9c8c06c8f0 0x7f9c8c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:07.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.404+0000 7f9caf400640 1 --2- 172.21.15.86:0/3054435160 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9c8c06c8f0 0x7f9c8c06edb0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f9cb00ba410 tx=0x7f9ca8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:07.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.408+0000 7f9ca7400640 1 -- 172.21.15.86:0/3054435160 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9ca0058b90 con 0x7f9cb0162000 2024-04-20T21:28:07.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.609+0000 7f9cb6600640 1 -- 172.21.15.86:0/3054435160 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9cb00689e0 con 0x7f9cb0162000 2024-04-20T21:28:07.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.611+0000 7f9ca7400640 1 -- 172.21.15.86:0/3054435160 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+16610 (secure 0 0 0) 0x7f9ca005ce90 con 0x7f9cb0162000 2024-04-20T21:28:07.611 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:28:07.611 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":100,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:27:47.267402+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:27:43.469685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-20T21:27:44.673505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:28:07.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.613+0000 7f9cb6600640 1 -- 172.21.15.86:0/3054435160 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9c8c06c8f0 msgr2=0x7f9c8c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:07.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.613+0000 7f9cb6600640 1 --2- 172.21.15.86:0/3054435160 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9c8c06c8f0 0x7f9c8c06edb0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f9cb00ba410 tx=0x7f9ca8009040 comp rx=0 tx=0).stop 2024-04-20T21:28:07.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.613+0000 7f9cb6600640 1 -- 172.21.15.86:0/3054435160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cb0162000 msgr2=0x7f9cb00b9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:07.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.613+0000 7f9cb6600640 1 --2- 172.21.15.86:0/3054435160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cb0162000 0x7f9cb00b9170 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f9ca000c370 tx=0x7f9ca000c830 comp rx=0 tx=0).stop 2024-04-20T21:28:07.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.613+0000 7f9cb6600640 1 -- 172.21.15.86:0/3054435160 shutdown_connections 2024-04-20T21:28:07.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.613+0000 7f9cb6600640 1 --2- 172.21.15.86:0/3054435160 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9c8c06c8f0 0x7f9c8c06edb0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:07.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.613+0000 7f9cb6600640 1 --2- 172.21.15.86:0/3054435160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9cb0162000 0x7f9cb00b9170 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:07.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.613+0000 7f9cb6600640 1 --2- 172.21.15.86:0/3054435160 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cb00d11d0 0x7f9cb01479c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:07.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.613+0000 7f9cb6600640 1 --2- 172.21.15.86:0/3054435160 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9cb00cf410 0x7f9cb00b8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:07.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.613+0000 7f9cb6600640 1 -- 172.21.15.86:0/3054435160 >> 172.21.15.86:0/3054435160 conn(0x7f9cb008bf70 msgr2=0x7f9cb00cf840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:07.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.614+0000 7f9cb6600640 1 -- 172.21.15.86:0/3054435160 shutdown_connections 2024-04-20T21:28:07.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.614+0000 7f9cb6600640 1 -- 172.21.15.86:0/3054435160 wait complete. 2024-04-20T21:28:07.630 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:27:43.469685+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:27:44.673505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:28:07.631 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:28:07.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.829+0000 7f7252800640 1 Processor -- start 2024-04-20T21:28:07.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.830+0000 7f7252800640 1 -- start start 2024-04-20T21:28:07.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.831+0000 7f7252800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f724c0af1f0 0x7f724c06ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:07.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.831+0000 7f7252800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f724c06de80 con 0x7f724c0af5d0 2024-04-20T21:28:07.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.831+0000 7f7252800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f724c06e160 con 0x7f724c06d9c0 2024-04-20T21:28:07.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.831+0000 7f7252800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f724c06f460 con 0x7f724c0af1f0 2024-04-20T21:28:07.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.831+0000 7f7251e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f724c0af5d0 0x7f724c15de70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:47086/0 (socket says 172.21.15.86:47086) 2024-04-20T21:28:07.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.831+0000 7f7251e00640 1 -- 172.21.15.86:0/2062089179 learned_addr learned my addr 172.21.15.86:0/2062089179 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:28:07.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.832+0000 7f7251400640 1 --2- 172.21.15.86:0/2062089179 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f724c0af1f0 0x7f724c06ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:07.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.832+0000 7f724be00640 1 -- 172.21.15.86:0/2062089179 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 249093021 0 0) 0x7f724c06de80 con 0x7f724c0af5d0 2024-04-20T21:28:07.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.832+0000 7f724be00640 1 -- 172.21.15.86:0/2062089179 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7234003520 con 0x7f724c0af5d0 2024-04-20T21:28:07.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.832+0000 7f724be00640 1 -- 172.21.15.86:0/2062089179 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2836483625 0 0) 0x7f7234003520 con 0x7f724c0af5d0 2024-04-20T21:28:07.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.832+0000 7f724be00640 1 -- 172.21.15.86:0/2062089179 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f724c0af1f0 msgr2=0x7f724c06ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:07.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.832+0000 7f724be00640 1 --2- 172.21.15.86:0/2062089179 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f724c0af1f0 0x7f724c06ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:07.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.832+0000 7f724be00640 1 -- 172.21.15.86:0/2062089179 >> v1:172.21.15.144:6789/0 conn(0x7f724c06d9c0 legacy=0x7f724c15d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:07.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.832+0000 7f724be00640 1 -- 172.21.15.86:0/2062089179 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f724c06e440 con 0x7f724c0af5d0 2024-04-20T21:28:07.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.832+0000 7f724be00640 1 -- 172.21.15.86:0/2062089179 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f72440019f0 con 0x7f724c0af5d0 2024-04-20T21:28:07.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.832+0000 7f724be00640 1 -- 172.21.15.86:0/2062089179 >> v1:172.21.15.86:6789/0 conn(0x7f724c0af5d0 legacy=0x7f724c15de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:07.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.832+0000 7f724be00640 1 --2- 172.21.15.86:0/2062089179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72340039e0 0x7f7234003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:07.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.832+0000 7f724be00640 1 --2- 172.21.15.86:0/2062089179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7234004800 0x7f7234004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:07.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.832+0000 7f724be00640 1 --2- 172.21.15.86:0/2062089179 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f724c0af1f0 0x7f7234005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:07.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.832+0000 7f724be00640 1 -- 172.21.15.86:0/2062089179 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f724c06f460 con 0x7f7234004800 2024-04-20T21:28:07.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.832+0000 7f724be00640 1 -- 172.21.15.86:0/2062089179 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f723400baf0 con 0x7f72340039e0 2024-04-20T21:28:07.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.832+0000 7f724be00640 1 -- 172.21.15.86:0/2062089179 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f723400bcf0 con 0x7f724c0af1f0 2024-04-20T21:28:07.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.833+0000 7f7250a00640 1 --2- 172.21.15.86:0/2062089179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7234004800 0x7f7234004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:07.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.833+0000 7f7251e00640 1 --2- 172.21.15.86:0/2062089179 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f724c0af1f0 0x7f7234005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:07.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.833+0000 7f7250a00640 1 -- 172.21.15.86:0/2062089179 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f724c0af1f0 msgr2=0x7f7234005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:07.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.833+0000 7f7251400640 1 --2- 172.21.15.86:0/2062089179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72340039e0 0x7f7234003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:07.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.833+0000 7f7250a00640 1 --2- 172.21.15.86:0/2062089179 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f724c0af1f0 0x7f7234005200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:07.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.833+0000 7f7250a00640 1 -- 172.21.15.86:0/2062089179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72340039e0 msgr2=0x7f7234003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:07.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.833+0000 7f7250a00640 1 --2- 172.21.15.86:0/2062089179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72340039e0 0x7f7234003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:07.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.833+0000 7f7250a00640 1 -- 172.21.15.86:0/2062089179 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7234003520 con 0x7f7234004800 2024-04-20T21:28:07.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.834+0000 7f7250a00640 1 --2- 172.21.15.86:0/2062089179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7234004800 0x7f7234004c80 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f7238001750 tx=0x7f7234005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=824826a214a26202 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:07.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.834+0000 7f724be00640 1 -- 172.21.15.86:0/2062089179 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7238001370 con 0x7f7234004800 2024-04-20T21:28:07.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.834+0000 7f724be00640 1 -- 172.21.15.86:0/2062089179 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72380052d0 con 0x7f7234004800 2024-04-20T21:28:07.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.834+0000 7f724be00640 1 -- 172.21.15.86:0/2062089179 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7238003830 con 0x7f7234004800 2024-04-20T21:28:07.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.835+0000 7f7252800640 1 -- 172.21.15.86:0/2062089179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7234004800 msgr2=0x7f7234004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:07.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.835+0000 7f7252800640 1 --2- 172.21.15.86:0/2062089179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7234004800 0x7f7234004c80 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f7238001750 tx=0x7f7234005830 comp rx=0 tx=0).stop 2024-04-20T21:28:07.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.835+0000 7f7251e00640 1 -- 172.21.15.86:0/2062089179 reap_dead start 2024-04-20T21:28:07.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.835+0000 7f7252800640 1 -- 172.21.15.86:0/2062089179 shutdown_connections 2024-04-20T21:28:07.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.835+0000 7f7252800640 1 -- 172.21.15.86:0/2062089179 >> 172.21.15.86:0/2062089179 conn(0x7f724c0b1f00 msgr2=0x7f724c0b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:07.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.835+0000 7f7252800640 1 -- 172.21.15.86:0/2062089179 shutdown_connections 2024-04-20T21:28:07.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.835+0000 7f7252800640 1 -- 172.21.15.86:0/2062089179 wait complete. 2024-04-20T21:28:07.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.837+0000 7f7252800640 1 Processor -- start 2024-04-20T21:28:07.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.837+0000 7f7252800640 1 -- start start 2024-04-20T21:28:07.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.838+0000 7f7252800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7234004800 0x7f724c0d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:07.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.838+0000 7f7252800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f724c1594e0 0x7f724c1f4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:07.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.838+0000 7f7252800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f724c15ecb0 0x7f724c15f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:07.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.838+0000 7f7252800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f724c0afa80 con 0x7f724c15ecb0 2024-04-20T21:28:07.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.838+0000 7f7252800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f724c06ea00 con 0x7f7234004800 2024-04-20T21:28:07.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.838+0000 7f7252800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f724c15a3c0 con 0x7f724c1594e0 2024-04-20T21:28:07.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.838+0000 7f7251e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f724c15ecb0 0x7f724c15f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:07.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.838+0000 7f7251e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f724c15ecb0 0x7f724c15f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40900/0 (socket says 172.21.15.86:40900) 2024-04-20T21:28:07.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.838+0000 7f7251e00640 1 -- 172.21.15.86:0/3143889114 learned_addr learned my addr 172.21.15.86:0/3143889114 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:07.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.839+0000 7f7251400640 1 --2- 172.21.15.86:0/3143889114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7234004800 0x7f724c0d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:07.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.839+0000 7f7251e00640 1 -- 172.21.15.86:0/3143889114 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f724c1594e0 msgr2=0x7f724c1f4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:07.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.839+0000 7f7250a00640 1 --2- 172.21.15.86:0/3143889114 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f724c1594e0 0x7f724c1f4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:07.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.839+0000 7f7251e00640 1 --2- 172.21.15.86:0/3143889114 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f724c1594e0 0x7f724c1f4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:07.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.839+0000 7f7251e00640 1 -- 172.21.15.86:0/3143889114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7234004800 msgr2=0x7f724c0d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:07.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.839+0000 7f7251e00640 1 --2- 172.21.15.86:0/3143889114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7234004800 0x7f724c0d0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:07.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.839+0000 7f7251e00640 1 -- 172.21.15.86:0/3143889114 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f724c142f10 con 0x7f724c15ecb0 2024-04-20T21:28:07.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.839+0000 7f7251e00640 1 --2- 172.21.15.86:0/3143889114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f724c15ecb0 0x7f724c15f130 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f7244007c40 tx=0x7f724400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:07.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.839+0000 7f7249600640 1 -- 172.21.15.86:0/3143889114 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724400bb20 con 0x7f724c15ecb0 2024-04-20T21:28:07.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.839+0000 7f7249600640 1 -- 172.21.15.86:0/3143889114 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f724400bce0 con 0x7f724c15ecb0 2024-04-20T21:28:07.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.839+0000 7f7249600640 1 -- 172.21.15.86:0/3143889114 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72440072b0 con 0x7f724c15ecb0 2024-04-20T21:28:07.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.839+0000 7f7252800640 1 -- 172.21.15.86:0/3143889114 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f724c0b18e0 con 0x7f724c15ecb0 2024-04-20T21:28:07.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.839+0000 7f7252800640 1 -- 172.21.15.86:0/3143889114 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f724c145f40 con 0x7f724c15ecb0 2024-04-20T21:28:07.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.841+0000 7f7252800640 1 -- 172.21.15.86:0/3143889114 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f724c04fd90 con 0x7f724c15ecb0 2024-04-20T21:28:07.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.841+0000 7f7249600640 1 -- 172.21.15.86:0/3143889114 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f724401b060 con 0x7f724c15ecb0 2024-04-20T21:28:07.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.842+0000 7f7249600640 1 --2- 172.21.15.86:0/3143889114 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f722c06c8f0 0x7f722c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:07.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.842+0000 7f7249600640 1 -- 172.21.15.86:0/3143889114 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7f724408bb90 con 0x7f724c15ecb0 2024-04-20T21:28:07.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.843+0000 7f7251400640 1 --2- 172.21.15.86:0/3143889114 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f722c06c8f0 0x7f722c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:07.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.843+0000 7f7251400640 1 --2- 172.21.15.86:0/3143889114 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f722c06c8f0 0x7f722c06edb0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f7240006fd0 tx=0x7f7240008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:07.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:07.847+0000 7f7249600640 1 -- 172.21.15.86:0/3143889114 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7244059fa0 con 0x7f724c15ecb0 2024-04-20T21:28:08.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.032+0000 7f7252800640 1 -- 172.21.15.86:0/3143889114 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f724c0b3300 con 0x7f724c15ecb0 2024-04-20T21:28:08.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.033+0000 7f7249600640 1 -- 172.21.15.86:0/3143889114 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) ==== 93+0+10 (secure 0 0 0) 0x7f72440598a0 con 0x7f724c15ecb0 2024-04-20T21:28:08.033 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:28:08.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.035+0000 7f7252800640 1 -- 172.21.15.86:0/3143889114 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f722c06c8f0 msgr2=0x7f722c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:08.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.035+0000 7f7252800640 1 --2- 172.21.15.86:0/3143889114 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f722c06c8f0 0x7f722c06edb0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f7240006fd0 tx=0x7f7240008040 comp rx=0 tx=0).stop 2024-04-20T21:28:08.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.035+0000 7f7252800640 1 -- 172.21.15.86:0/3143889114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f724c15ecb0 msgr2=0x7f724c15f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:08.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.035+0000 7f7252800640 1 --2- 172.21.15.86:0/3143889114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f724c15ecb0 0x7f724c15f130 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f7244007c40 tx=0x7f724400b4a0 comp rx=0 tx=0).stop 2024-04-20T21:28:08.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.035+0000 7f7252800640 1 -- 172.21.15.86:0/3143889114 shutdown_connections 2024-04-20T21:28:08.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.035+0000 7f7252800640 1 --2- 172.21.15.86:0/3143889114 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f722c06c8f0 0x7f722c06edb0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:08.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.035+0000 7f7252800640 1 --2- 172.21.15.86:0/3143889114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f724c15ecb0 0x7f724c15f130 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:08.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.035+0000 7f7252800640 1 --2- 172.21.15.86:0/3143889114 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f724c1594e0 0x7f724c1f4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:08.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.035+0000 7f7252800640 1 --2- 172.21.15.86:0/3143889114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7234004800 0x7f724c0d0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:08.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.035+0000 7f7252800640 1 -- 172.21.15.86:0/3143889114 >> 172.21.15.86:0/3143889114 conn(0x7f724c0b1f00 msgr2=0x7f724c0723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:08.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.035+0000 7f7252800640 1 -- 172.21.15.86:0/3143889114 shutdown_connections 2024-04-20T21:28:08.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.035+0000 7f7252800640 1 -- 172.21.15.86:0/3143889114 wait complete. 2024-04-20T21:28:08.051 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:28:08.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.242+0000 7f226c200640 1 Processor -- start 2024-04-20T21:28:08.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.242+0000 7f226c200640 1 -- start start 2024-04-20T21:28:08.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.244+0000 7f226c200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2264146210 0x7f226406e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:08.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.244+0000 7f226c200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f226406bfd0 0x7f22640cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:08.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.244+0000 7f226c200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f226415cd40 con 0x7f226406bbf0 2024-04-20T21:28:08.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.244+0000 7f226c200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f226415d000 con 0x7f226406bfd0 2024-04-20T21:28:08.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.244+0000 7f226c200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f226415d2e0 con 0x7f2264146210 2024-04-20T21:28:08.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.244+0000 7f2269800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f226406bbf0 0x7f2264145b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:47100/0 (socket says 172.21.15.86:47100) 2024-04-20T21:28:08.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.245+0000 7f2269800640 1 -- 172.21.15.86:0/510320386 learned_addr learned my addr 172.21.15.86:0/510320386 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:28:08.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.245+0000 7f2268e00640 1 --2- 172.21.15.86:0/510320386 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2264146210 0x7f226406e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:08.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.245+0000 7f226a200640 1 --2- 172.21.15.86:0/510320386 >> v2:172.21.15.144:3300/0 conn(0x7f226406bfd0 0x7f22640cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:08.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.245+0000 7f2263e00640 1 -- 172.21.15.86:0/510320386 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3033605591 0 0) 0x7f226415cd40 con 0x7f226406bbf0 2024-04-20T21:28:08.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.245+0000 7f2263e00640 1 -- 172.21.15.86:0/510320386 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f224c003520 con 0x7f226406bbf0 2024-04-20T21:28:08.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.246+0000 7f2263e00640 1 -- 172.21.15.86:0/510320386 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1610412996 0 0) 0x7f224c003520 con 0x7f226406bbf0 2024-04-20T21:28:08.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.246+0000 7f2263e00640 1 -- 172.21.15.86:0/510320386 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2264146210 msgr2=0x7f226406e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:08.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.246+0000 7f2263e00640 1 --2- 172.21.15.86:0/510320386 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2264146210 0x7f226406e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:08.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.246+0000 7f2263e00640 1 -- 172.21.15.86:0/510320386 >> v2:172.21.15.144:3300/0 conn(0x7f226406bfd0 msgr2=0x7f22640cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:08.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.246+0000 7f2263e00640 1 --2- 172.21.15.86:0/510320386 >> v2:172.21.15.144:3300/0 conn(0x7f226406bfd0 0x7f22640cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:08.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.246+0000 7f2263e00640 1 -- 172.21.15.86:0/510320386 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f226415d5c0 con 0x7f226406bbf0 2024-04-20T21:28:08.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.246+0000 7f2263e00640 1 -- 172.21.15.86:0/510320386 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f2258001670 con 0x7f226406bbf0 2024-04-20T21:28:08.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.246+0000 7f2263e00640 1 -- 172.21.15.86:0/510320386 >> v1:172.21.15.86:6789/0 conn(0x7f226406bbf0 legacy=0x7f2264145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:08.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.246+0000 7f2263e00640 1 --2- 172.21.15.86:0/510320386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f224c0039b0 0x7f224c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:08.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.246+0000 7f2263e00640 1 --2- 172.21.15.86:0/510320386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f224c004800 0x7f224c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:08.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.246+0000 7f2263e00640 1 --2- 172.21.15.86:0/510320386 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2264146210 0x7f224c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:08.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.246+0000 7f2263e00640 1 -- 172.21.15.86:0/510320386 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f226415d000 con 0x7f224c0039b0 2024-04-20T21:28:08.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.246+0000 7f2263e00640 1 -- 172.21.15.86:0/510320386 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f226415d2e0 con 0x7f224c004800 2024-04-20T21:28:08.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.246+0000 7f2263e00640 1 -- 172.21.15.86:0/510320386 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f224c00b9c0 con 0x7f2264146210 2024-04-20T21:28:08.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.246+0000 7f2269800640 1 --2- 172.21.15.86:0/510320386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f224c0039b0 0x7f224c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:08.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.246+0000 7f2268e00640 1 --2- 172.21.15.86:0/510320386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f224c004800 0x7f224c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:08.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.246+0000 7f226a200640 1 --2- 172.21.15.86:0/510320386 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2264146210 0x7f224c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:08.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.246+0000 7f2269800640 1 -- 172.21.15.86:0/510320386 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2264146210 msgr2=0x7f224c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:08.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.246+0000 7f2269800640 1 --2- 172.21.15.86:0/510320386 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2264146210 0x7f224c0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:08.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.247+0000 7f2269800640 1 -- 172.21.15.86:0/510320386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f224c004800 msgr2=0x7f224c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:08.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.247+0000 7f2269800640 1 --2- 172.21.15.86:0/510320386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f224c004800 0x7f224c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:08.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.247+0000 7f2269800640 1 -- 172.21.15.86:0/510320386 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f224c003520 con 0x7f224c0039b0 2024-04-20T21:28:08.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.247+0000 7f2268e00640 1 --2- 172.21.15.86:0/510320386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f224c004800 0x7f224c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:28:08.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.248+0000 7f2269800640 1 --2- 172.21.15.86:0/510320386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f224c0039b0 0x7f224c003dd0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f225800c240 tx=0x7f224c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd290c6a2f3bec1a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:08.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.248+0000 7f2263e00640 1 -- 172.21.15.86:0/510320386 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2258007d30 con 0x7f224c0039b0 2024-04-20T21:28:08.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.248+0000 7f2263e00640 1 -- 172.21.15.86:0/510320386 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2258030070 con 0x7f224c0039b0 2024-04-20T21:28:08.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.248+0000 7f2263e00640 1 -- 172.21.15.86:0/510320386 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225802cd60 con 0x7f224c0039b0 2024-04-20T21:28:08.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.249+0000 7f226c200640 1 -- 172.21.15.86:0/510320386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f224c0039b0 msgr2=0x7f224c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:08.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.249+0000 7f226c200640 1 --2- 172.21.15.86:0/510320386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f224c0039b0 0x7f224c003dd0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f225800c240 tx=0x7f224c005830 comp rx=0 tx=0).stop 2024-04-20T21:28:08.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.249+0000 7f226a200640 1 -- 172.21.15.86:0/510320386 reap_dead start 2024-04-20T21:28:08.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.249+0000 7f226c200640 1 -- 172.21.15.86:0/510320386 shutdown_connections 2024-04-20T21:28:08.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.249+0000 7f226c200640 1 -- 172.21.15.86:0/510320386 >> 172.21.15.86:0/510320386 conn(0x7f22640b2110 msgr2=0x7f22640b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:08.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.249+0000 7f226c200640 1 -- 172.21.15.86:0/510320386 shutdown_connections 2024-04-20T21:28:08.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.249+0000 7f226c200640 1 -- 172.21.15.86:0/510320386 wait complete. 2024-04-20T21:28:08.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.251+0000 7f226c200640 1 Processor -- start 2024-04-20T21:28:08.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.251+0000 7f226c200640 1 -- start start 2024-04-20T21:28:08.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.252+0000 7f226c200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f224c0039b0 0x7f22640ae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:08.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.252+0000 7f226c200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22641e6570 0x7f22640a9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:08.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.252+0000 7f226c200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f22640b0330 0x7f22640b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:08.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.252+0000 7f226c200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f226406e180 con 0x7f22641e6570 2024-04-20T21:28:08.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.252+0000 7f226c200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f226406e000 con 0x7f224c0039b0 2024-04-20T21:28:08.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.252+0000 7f226c200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f226406e300 con 0x7f22640b0330 2024-04-20T21:28:08.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.252+0000 7f2268e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22641e6570 0x7f22640a9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:08.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.252+0000 7f2268e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22641e6570 0x7f22640a9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40922/0 (socket says 172.21.15.86:40922) 2024-04-20T21:28:08.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.252+0000 7f2268e00640 1 -- 172.21.15.86:0/3112620149 learned_addr learned my addr 172.21.15.86:0/3112620149 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:08.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.253+0000 7f2269800640 1 --2- 172.21.15.86:0/3112620149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f224c0039b0 0x7f22640ae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:08.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.253+0000 7f226a200640 1 --2- 172.21.15.86:0/3112620149 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f22640b0330 0x7f22640b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:08.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.253+0000 7f2268e00640 1 -- 172.21.15.86:0/3112620149 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f22640b0330 msgr2=0x7f22640b07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:08.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.253+0000 7f2268e00640 1 --2- 172.21.15.86:0/3112620149 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f22640b0330 0x7f22640b07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:08.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.253+0000 7f2268e00640 1 -- 172.21.15.86:0/3112620149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f224c0039b0 msgr2=0x7f22640ae8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:08.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.253+0000 7f2268e00640 1 --2- 172.21.15.86:0/3112620149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f224c0039b0 0x7f22640ae8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:08.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.253+0000 7f2268e00640 1 -- 172.21.15.86:0/3112620149 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22641f33a0 con 0x7f22641e6570 2024-04-20T21:28:08.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.253+0000 7f2268e00640 1 --2- 172.21.15.86:0/3112620149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22641e6570 0x7f22640a9440 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f22540108c0 tx=0x7f2254010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:08.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.253+0000 7f2261600640 1 -- 172.21.15.86:0/3112620149 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225400f040 con 0x7f22641e6570 2024-04-20T21:28:08.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.254+0000 7f2261600640 1 -- 172.21.15.86:0/3112620149 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f225400ea80 con 0x7f22641e6570 2024-04-20T21:28:08.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.254+0000 7f226c200640 1 -- 172.21.15.86:0/3112620149 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22641f3680 con 0x7f22641e6570 2024-04-20T21:28:08.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.254+0000 7f2261600640 1 -- 172.21.15.86:0/3112620149 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225400ee50 con 0x7f22641e6570 2024-04-20T21:28:08.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.254+0000 7f226c200640 1 -- 172.21.15.86:0/3112620149 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22641f3c40 con 0x7f22641e6570 2024-04-20T21:28:08.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.255+0000 7f226c200640 1 -- 172.21.15.86:0/3112620149 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22640689e0 con 0x7f22641e6570 2024-04-20T21:28:08.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.256+0000 7f2261600640 1 -- 172.21.15.86:0/3112620149 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f225401c400 con 0x7f22641e6570 2024-04-20T21:28:08.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.257+0000 7f2261600640 1 --2- 172.21.15.86:0/3112620149 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f224806c8f0 0x7f224806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:08.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.257+0000 7f2261600640 1 -- 172.21.15.86:0/3112620149 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7f225408c1e0 con 0x7f22641e6570 2024-04-20T21:28:08.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.257+0000 7f2269800640 1 --2- 172.21.15.86:0/3112620149 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f224806c8f0 0x7f224806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:08.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.258+0000 7f2269800640 1 --2- 172.21.15.86:0/3112620149 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f224806c8f0 0x7f224806edb0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f2258034000 tx=0x7f2258007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:08.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.261+0000 7f2261600640 1 -- 172.21.15.86:0/3112620149 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2254018030 con 0x7f22641e6570 2024-04-20T21:28:08.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.460+0000 7f226c200640 1 -- 172.21.15.86:0/3112620149 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f22640cdcb0 con 0x7f22641e6570 2024-04-20T21:28:08.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.461+0000 7f2261600640 1 -- 172.21.15.86:0/3112620149 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v100) ==== 104+0+11 (secure 0 0 0) 0x7f225405a580 con 0x7f22641e6570 2024-04-20T21:28:08.461 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-20T21:28:08.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.464+0000 7f226c200640 1 -- 172.21.15.86:0/3112620149 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f224806c8f0 msgr2=0x7f224806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:08.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.464+0000 7f226c200640 1 --2- 172.21.15.86:0/3112620149 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f224806c8f0 0x7f224806edb0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f2258034000 tx=0x7f2258007000 comp rx=0 tx=0).stop 2024-04-20T21:28:08.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.464+0000 7f226c200640 1 -- 172.21.15.86:0/3112620149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22641e6570 msgr2=0x7f22640a9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:08.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.464+0000 7f226c200640 1 --2- 172.21.15.86:0/3112620149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22641e6570 0x7f22640a9440 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f22540108c0 tx=0x7f2254010d80 comp rx=0 tx=0).stop 2024-04-20T21:28:08.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.464+0000 7f226c200640 1 -- 172.21.15.86:0/3112620149 shutdown_connections 2024-04-20T21:28:08.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.464+0000 7f226c200640 1 --2- 172.21.15.86:0/3112620149 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f224806c8f0 0x7f224806edb0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:08.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.464+0000 7f226c200640 1 --2- 172.21.15.86:0/3112620149 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f22640b0330 0x7f22640b07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:08.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.464+0000 7f226c200640 1 --2- 172.21.15.86:0/3112620149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22641e6570 0x7f22640a9440 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:08.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.464+0000 7f226c200640 1 --2- 172.21.15.86:0/3112620149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f224c0039b0 0x7f22640ae8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:08.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.464+0000 7f226c200640 1 -- 172.21.15.86:0/3112620149 >> 172.21.15.86:0/3112620149 conn(0x7f22640b2110 msgr2=0x7f226415f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:08.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.464+0000 7f226c200640 1 -- 172.21.15.86:0/3112620149 shutdown_connections 2024-04-20T21:28:08.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.465+0000 7f226c200640 1 -- 172.21.15.86:0/3112620149 wait complete. 2024-04-20T21:28:08.483 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:28:08.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.687+0000 7fc2eb800640 1 Processor -- start 2024-04-20T21:28:08.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.687+0000 7fc2eb800640 1 -- start start 2024-04-20T21:28:08.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.689+0000 7fc2eb800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc2e40724c0 0x7fc2e40b3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:08.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.689+0000 7fc2eb800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc2e40b4400 0x7fc2e406e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:08.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.689+0000 7fc2eb800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2e40b4bf0 con 0x7fc2e4158950 2024-04-20T21:28:08.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.689+0000 7fc2eb800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc2e40b4ed0 con 0x7fc2e40b4400 2024-04-20T21:28:08.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.689+0000 7fc2eb800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fc2e406de90 con 0x7fc2e40724c0 2024-04-20T21:28:08.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.689+0000 7fc2e9800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc2e40b4400 0x7fc2e406e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:08.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.689+0000 7fc2e9800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc2e40b4400 0x7fc2e406e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40928/0 (socket says 172.21.15.86:40928) 2024-04-20T21:28:08.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.689+0000 7fc2e9800640 1 -- 172.21.15.86:0/3436267186 learned_addr learned my addr 172.21.15.86:0/3436267186 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:08.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.689+0000 7fc2dbe00640 1 --2- 172.21.15.86:0/3436267186 >> v2:172.21.15.144:3300/0 conn(0x7fc2e40724c0 0x7fc2e40b3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:08.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.690+0000 7fc2e9800640 1 -- 172.21.15.86:0/3436267186 >> v2:172.21.15.144:3300/0 conn(0x7fc2e40724c0 msgr2=0x7fc2e40b3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:08.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.690+0000 7fc2e9800640 1 --2- 172.21.15.86:0/3436267186 >> v2:172.21.15.144:3300/0 conn(0x7fc2e40724c0 0x7fc2e40b3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:08.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.690+0000 7fc2e9800640 1 -- 172.21.15.86:0/3436267186 >> v1:172.21.15.144:6789/0 conn(0x7fc2e4158950 legacy=0x7fc2e4071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:08.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.690+0000 7fc2e9800640 1 -- 172.21.15.86:0/3436267186 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2e406e170 con 0x7fc2e40b4400 2024-04-20T21:28:08.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.691+0000 7fc2e9800640 1 --2- 172.21.15.86:0/3436267186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2e40b4400 0x7fc2e406e970 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fc2e415e990 tx=0x7fc2dc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81e6bdb81fca4ffa server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:08.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.691+0000 7fc2e3e00640 1 -- 172.21.15.86:0/3436267186 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc02ca80 con 0x7fc2e40b4400 2024-04-20T21:28:08.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.691+0000 7fc2e3e00640 1 -- 172.21.15.86:0/3436267186 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2dc02cc20 con 0x7fc2e40b4400 2024-04-20T21:28:08.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.691+0000 7fc2e3e00640 1 -- 172.21.15.86:0/3436267186 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc0343f0 con 0x7fc2e40b4400 2024-04-20T21:28:08.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.692+0000 7fc2eb800640 1 -- 172.21.15.86:0/3436267186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2e40b4400 msgr2=0x7fc2e406e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:08.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.692+0000 7fc2eb800640 1 --2- 172.21.15.86:0/3436267186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2e40b4400 0x7fc2e406e970 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fc2e415e990 tx=0x7fc2dc02bee0 comp rx=0 tx=0).stop 2024-04-20T21:28:08.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.692+0000 7fc2eb800640 1 -- 172.21.15.86:0/3436267186 shutdown_connections 2024-04-20T21:28:08.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.692+0000 7fc2eb800640 1 --2- 172.21.15.86:0/3436267186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2e40b4400 0x7fc2e406e970 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:08.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.692+0000 7fc2eb800640 1 --2- 172.21.15.86:0/3436267186 >> v2:172.21.15.144:3300/0 conn(0x7fc2e40724c0 0x7fc2e40b3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:08.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.692+0000 7fc2eb800640 1 -- 172.21.15.86:0/3436267186 >> 172.21.15.86:0/3436267186 conn(0x7fc2e40b2110 msgr2=0x7fc2e40b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:08.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.692+0000 7fc2eb800640 1 -- 172.21.15.86:0/3436267186 shutdown_connections 2024-04-20T21:28:08.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.692+0000 7fc2eb800640 1 -- 172.21.15.86:0/3436267186 wait complete. 2024-04-20T21:28:08.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.694+0000 7fc2eb800640 1 Processor -- start 2024-04-20T21:28:08.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.694+0000 7fc2eb800640 1 -- start start 2024-04-20T21:28:08.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.695+0000 7fc2eb800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2e40724c0 0x7fc2e41692e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:08.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.695+0000 7fc2eb800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2e40b4400 0x7fc2e41537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:08.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.695+0000 7fc2eb800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2e4158950 0x7fc2e41542d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:08.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.695+0000 7fc2eb800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e4158d30 con 0x7fc2e40724c0 2024-04-20T21:28:08.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.695+0000 7fc2eb800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e40b48b0 con 0x7fc2e4158950 2024-04-20T21:28:08.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.695+0000 7fc2eb800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc2e4158eb0 con 0x7fc2e40b4400 2024-04-20T21:28:08.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.695+0000 7fc2e8e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2e40724c0 0x7fc2e41692e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:08.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.695+0000 7fc2e8e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2e40724c0 0x7fc2e41692e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40934/0 (socket says 172.21.15.86:40934) 2024-04-20T21:28:08.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.695+0000 7fc2e8e00640 1 -- 172.21.15.86:0/789492900 learned_addr learned my addr 172.21.15.86:0/789492900 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:08.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.695+0000 7fc2e9800640 1 --2- 172.21.15.86:0/789492900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2e4158950 0x7fc2e41542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:08.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.695+0000 7fc2e8e00640 1 -- 172.21.15.86:0/789492900 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2e40b4400 msgr2=0x7fc2e41537e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:08.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.695+0000 7fc2e8e00640 1 --2- 172.21.15.86:0/789492900 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2e40b4400 0x7fc2e41537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:08.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.695+0000 7fc2e8e00640 1 -- 172.21.15.86:0/789492900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2e4158950 msgr2=0x7fc2e41542d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:08.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.695+0000 7fc2e8e00640 1 --2- 172.21.15.86:0/789492900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2e4158950 0x7fc2e41542d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:08.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.695+0000 7fc2e8e00640 1 -- 172.21.15.86:0/789492900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2e4169820 con 0x7fc2e40724c0 2024-04-20T21:28:08.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.696+0000 7fc2e8e00640 1 --2- 172.21.15.86:0/789492900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2e40724c0 0x7fc2e41692e0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fc2d400c430 tx=0x7fc2d400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:08.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.696+0000 7fc2e1600640 1 -- 172.21.15.86:0/789492900 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d4011790 con 0x7fc2e40724c0 2024-04-20T21:28:08.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.696+0000 7fc2e1600640 1 -- 172.21.15.86:0/789492900 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2d4017070 con 0x7fc2e40724c0 2024-04-20T21:28:08.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.696+0000 7fc2e1600640 1 -- 172.21.15.86:0/789492900 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d4011bb0 con 0x7fc2e40724c0 2024-04-20T21:28:08.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.696+0000 7fc2eb800640 1 -- 172.21.15.86:0/789492900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2e4168240 con 0x7fc2e40724c0 2024-04-20T21:28:08.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.696+0000 7fc2eb800640 1 -- 172.21.15.86:0/789492900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2e4168800 con 0x7fc2e40724c0 2024-04-20T21:28:08.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.698+0000 7fc2eb800640 1 -- 172.21.15.86:0/789492900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2e40689e0 con 0x7fc2e40724c0 2024-04-20T21:28:08.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.698+0000 7fc2e1600640 1 -- 172.21.15.86:0/789492900 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc2d401c460 con 0x7fc2e40724c0 2024-04-20T21:28:08.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.699+0000 7fc2e1600640 1 --2- 172.21.15.86:0/789492900 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc2c006c920 0x7fc2c006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:08.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.700+0000 7fc2e1600640 1 -- 172.21.15.86:0/789492900 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7fc2d408ba00 con 0x7fc2e40724c0 2024-04-20T21:28:08.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.700+0000 7fc2dbe00640 1 --2- 172.21.15.86:0/789492900 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc2c006c920 0x7fc2c006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:08.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.700+0000 7fc2dbe00640 1 --2- 172.21.15.86:0/789492900 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc2c006c920 0x7fc2c006ede0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc006fd0 tx=0x7fc2cc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:08.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.704+0000 7fc2e1600640 1 -- 172.21.15.86:0/789492900 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc2d4059e20 con 0x7fc2e40724c0 2024-04-20T21:28:08.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.900+0000 7fc2eb800640 1 -- 172.21.15.86:0/789492900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc2e41485c0 con 0x7fc2e40724c0 2024-04-20T21:28:08.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.901+0000 7fc2e1600640 1 -- 172.21.15.86:0/789492900 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v100) ==== 100+0+11 (secure 0 0 0) 0x7fc2d4059720 con 0x7fc2e40724c0 2024-04-20T21:28:08.901 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-20T21:28:08.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.903+0000 7fc2eb800640 1 -- 172.21.15.86:0/789492900 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc2c006c920 msgr2=0x7fc2c006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:08.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.903+0000 7fc2eb800640 1 --2- 172.21.15.86:0/789492900 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc2c006c920 0x7fc2c006ede0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc006fd0 tx=0x7fc2cc008040 comp rx=0 tx=0).stop 2024-04-20T21:28:08.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.903+0000 7fc2eb800640 1 -- 172.21.15.86:0/789492900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2e40724c0 msgr2=0x7fc2e41692e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:08.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.903+0000 7fc2eb800640 1 --2- 172.21.15.86:0/789492900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2e40724c0 0x7fc2e41692e0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fc2d400c430 tx=0x7fc2d400c8f0 comp rx=0 tx=0).stop 2024-04-20T21:28:08.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.903+0000 7fc2eb800640 1 -- 172.21.15.86:0/789492900 shutdown_connections 2024-04-20T21:28:08.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.903+0000 7fc2eb800640 1 --2- 172.21.15.86:0/789492900 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc2c006c920 0x7fc2c006ede0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:08.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.903+0000 7fc2eb800640 1 --2- 172.21.15.86:0/789492900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2e4158950 0x7fc2e41542d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:08.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.903+0000 7fc2eb800640 1 --2- 172.21.15.86:0/789492900 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc2e40b4400 0x7fc2e41537e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:08.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.904+0000 7fc2eb800640 1 --2- 172.21.15.86:0/789492900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2e40724c0 0x7fc2e41692e0 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:08.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.904+0000 7fc2eb800640 1 -- 172.21.15.86:0/789492900 >> 172.21.15.86:0/789492900 conn(0x7fc2e40b2110 msgr2=0x7fc2e406f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:08.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.904+0000 7fc2eb800640 1 -- 172.21.15.86:0/789492900 shutdown_connections 2024-04-20T21:28:08.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:08.904+0000 7fc2eb800640 1 -- 172.21.15.86:0/789492900 wait complete. 2024-04-20T21:28:08.922 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T21:28:09.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.110+0000 7fb53e200640 1 Processor -- start 2024-04-20T21:28:09.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.111+0000 7fb53e200640 1 -- start start 2024-04-20T21:28:09.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.112+0000 7fb53e200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb53809b0f0 0x7fb5380cd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:09.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.112+0000 7fb53e200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb53815ce50 0x7fb5380c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:09.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.112+0000 7fb53e200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5380c9480 con 0x7fb53815ca70 2024-04-20T21:28:09.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.112+0000 7fb53e200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fb5380ce280 con 0x7fb53815ce50 2024-04-20T21:28:09.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.112+0000 7fb53e200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb5380ce7b0 con 0x7fb53809b0f0 2024-04-20T21:28:09.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.112+0000 7fb537e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb53815ce50 0x7fb5380c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:09.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.113+0000 7fb536a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb53809b0f0 0x7fb5380cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:09.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.113+0000 7fb537e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb53815ce50 0x7fb5380c6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.86:39886/0 (socket says 172.21.15.86:39886) 2024-04-20T21:28:09.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.113+0000 7fb537e00640 1 -- 172.21.15.86:0/1389333414 learned_addr learned my addr 172.21.15.86:0/1389333414 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:09.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.113+0000 7fb537400640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fb53815ca70 0x7fb53809a9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.86:59640/0 (socket says 172.21.15.86:59640) 2024-04-20T21:28:09.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.113+0000 7fb536a00640 1 -- 172.21.15.86:0/1389333414 >> v2:172.21.15.144:3300/0 conn(0x7fb53815ce50 msgr2=0x7fb5380c6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:09.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.113+0000 7fb536a00640 1 --2- 172.21.15.86:0/1389333414 >> v2:172.21.15.144:3300/0 conn(0x7fb53815ce50 0x7fb5380c6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:09.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.113+0000 7fb536a00640 1 -- 172.21.15.86:0/1389333414 >> v1:172.21.15.144:6789/0 conn(0x7fb53815ca70 legacy=0x7fb53809a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:09.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.113+0000 7fb536a00640 1 -- 172.21.15.86:0/1389333414 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5380ce4d0 con 0x7fb53809b0f0 2024-04-20T21:28:09.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.115+0000 7fb536a00640 1 --2- 172.21.15.86:0/1389333414 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb53809b0f0 0x7fb5380cd280 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb5280097a0 tx=0x7fb52802b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6badb79d0565c6d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:09.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.115+0000 7fb536000640 1 -- 172.21.15.86:0/1389333414 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb528035020 con 0x7fb53809b0f0 2024-04-20T21:28:09.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.115+0000 7fb536000640 1 -- 172.21.15.86:0/1389333414 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb528031810 con 0x7fb53809b0f0 2024-04-20T21:28:09.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.115+0000 7fb536000640 1 -- 172.21.15.86:0/1389333414 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb528031be0 con 0x7fb53809b0f0 2024-04-20T21:28:09.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.116+0000 7fb53e200640 1 -- 172.21.15.86:0/1389333414 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb53809b0f0 msgr2=0x7fb5380cd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:09.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.116+0000 7fb53e200640 1 --2- 172.21.15.86:0/1389333414 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb53809b0f0 0x7fb5380cd280 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb5280097a0 tx=0x7fb52802b8b0 comp rx=0 tx=0).stop 2024-04-20T21:28:09.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.116+0000 7fb53e200640 1 -- 172.21.15.86:0/1389333414 shutdown_connections 2024-04-20T21:28:09.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.116+0000 7fb53e200640 1 --2- 172.21.15.86:0/1389333414 >> v2:172.21.15.144:3300/0 conn(0x7fb53815ce50 0x7fb5380c6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:09.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.116+0000 7fb53e200640 1 --2- 172.21.15.86:0/1389333414 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb53809b0f0 0x7fb5380cd280 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:09.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.116+0000 7fb53e200640 1 -- 172.21.15.86:0/1389333414 >> 172.21.15.86:0/1389333414 conn(0x7fb5380b40b0 msgr2=0x7fb5380b44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:09.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.116+0000 7fb53e200640 1 -- 172.21.15.86:0/1389333414 shutdown_connections 2024-04-20T21:28:09.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.116+0000 7fb53e200640 1 -- 172.21.15.86:0/1389333414 wait complete. 2024-04-20T21:28:09.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.117+0000 7fb53e200640 1 Processor -- start 2024-04-20T21:28:09.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.118+0000 7fb53e200640 1 -- start start 2024-04-20T21:28:09.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.118+0000 7fb53e200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb53809b0f0 0x7fb5380cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:09.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.118+0000 7fb53e200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb53815ca70 0x7fb538152270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:09.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.118+0000 7fb53e200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb53815ce50 0x7fb5380ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:09.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.118+0000 7fb53e200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb5380c6490 con 0x7fb53815ca70 2024-04-20T21:28:09.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.118+0000 7fb53e200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb5380c6800 con 0x7fb53815ce50 2024-04-20T21:28:09.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.118+0000 7fb53e200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb53809b580 con 0x7fb53809b0f0 2024-04-20T21:28:09.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.119+0000 7fb536a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb53815ca70 0x7fb538152270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:09.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.119+0000 7fb536a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb53815ca70 0x7fb538152270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40946/0 (socket says 172.21.15.86:40946) 2024-04-20T21:28:09.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.119+0000 7fb536a00640 1 -- 172.21.15.86:0/3554041464 learned_addr learned my addr 172.21.15.86:0/3554041464 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:09.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.119+0000 7fb537400640 1 --2- 172.21.15.86:0/3554041464 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb53809b0f0 0x7fb5380cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:09.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.119+0000 7fb537e00640 1 --2- 172.21.15.86:0/3554041464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb53815ce50 0x7fb5380ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:09.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.119+0000 7fb536a00640 1 -- 172.21.15.86:0/3554041464 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb53809b0f0 msgr2=0x7fb5380cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:09.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.119+0000 7fb536a00640 1 --2- 172.21.15.86:0/3554041464 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb53809b0f0 0x7fb5380cc5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:09.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.119+0000 7fb536a00640 1 -- 172.21.15.86:0/3554041464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb53815ce50 msgr2=0x7fb5380ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:09.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.119+0000 7fb536a00640 1 --2- 172.21.15.86:0/3554041464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb53815ce50 0x7fb5380ca6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:09.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.119+0000 7fb536a00640 1 -- 172.21.15.86:0/3554041464 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5380cb050 con 0x7fb53815ca70 2024-04-20T21:28:09.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.120+0000 7fb536a00640 1 --2- 172.21.15.86:0/3554041464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb53815ca70 0x7fb538152270 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fb52802bec0 tx=0x7fb5280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:09.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.120+0000 7fb52f400640 1 -- 172.21.15.86:0/3554041464 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb528041070 con 0x7fb53815ca70 2024-04-20T21:28:09.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.120+0000 7fb52f400640 1 -- 172.21.15.86:0/3554041464 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb52803ad00 con 0x7fb53815ca70 2024-04-20T21:28:09.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.120+0000 7fb52f400640 1 -- 172.21.15.86:0/3554041464 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb528035040 con 0x7fb53815ca70 2024-04-20T21:28:09.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.120+0000 7fb53e200640 1 -- 172.21.15.86:0/3554041464 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5380cb330 con 0x7fb53815ca70 2024-04-20T21:28:09.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.120+0000 7fb53e200640 1 -- 172.21.15.86:0/3554041464 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5380b4530 con 0x7fb53815ca70 2024-04-20T21:28:09.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.121+0000 7fb53e200640 1 -- 172.21.15.86:0/3554041464 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5380690f0 con 0x7fb53815ca70 2024-04-20T21:28:09.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.122+0000 7fb52f400640 1 -- 172.21.15.86:0/3554041464 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb528038020 con 0x7fb53815ca70 2024-04-20T21:28:09.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.123+0000 7fb52f400640 1 --2- 172.21.15.86:0/3554041464 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb51406c8f0 0x7fb51406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:09.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.123+0000 7fb52f400640 1 -- 172.21.15.86:0/3554041464 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7fb5280ad810 con 0x7fb53815ca70 2024-04-20T21:28:09.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.123+0000 7fb537400640 1 --2- 172.21.15.86:0/3554041464 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb51406c8f0 0x7fb51406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:09.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.124+0000 7fb537400640 1 --2- 172.21.15.86:0/3554041464 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb51406c8f0 0x7fb51406edb0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fb520000f60 tx=0x7fb520009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:09.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.127+0000 7fb52f400640 1 -- 172.21.15.86:0/3554041464 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb52807bba0 con 0x7fb53815ca70 2024-04-20T21:28:09.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.346+0000 7fb53e200640 1 -- 172.21.15.86:0/3554041464 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb5381527b0 con 0x7fb53815ca70 2024-04-20T21:28:09.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.347+0000 7fb52f400640 1 -- 172.21.15.86:0/3554041464 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v100) ==== 80+0+0 (secure 0 0 0) 0x7fb52807b4a0 con 0x7fb53815ca70 2024-04-20T21:28:09.347 INFO:teuthology.orchestra.run.smithi086.stderr:listed 0 entries 2024-04-20T21:28:09.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.350+0000 7fb53e200640 1 -- 172.21.15.86:0/3554041464 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb51406c8f0 msgr2=0x7fb51406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:09.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.350+0000 7fb53e200640 1 --2- 172.21.15.86:0/3554041464 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb51406c8f0 0x7fb51406edb0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fb520000f60 tx=0x7fb520009040 comp rx=0 tx=0).stop 2024-04-20T21:28:09.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.350+0000 7fb53e200640 1 -- 172.21.15.86:0/3554041464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb53815ca70 msgr2=0x7fb538152270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:09.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.350+0000 7fb53e200640 1 --2- 172.21.15.86:0/3554041464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb53815ca70 0x7fb538152270 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fb52802bec0 tx=0x7fb5280046c0 comp rx=0 tx=0).stop 2024-04-20T21:28:09.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.350+0000 7fb53e200640 1 -- 172.21.15.86:0/3554041464 shutdown_connections 2024-04-20T21:28:09.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.350+0000 7fb53e200640 1 --2- 172.21.15.86:0/3554041464 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb51406c8f0 0x7fb51406edb0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:09.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.350+0000 7fb53e200640 1 --2- 172.21.15.86:0/3554041464 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb53815ce50 0x7fb5380ca6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:09.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.350+0000 7fb53e200640 1 --2- 172.21.15.86:0/3554041464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb53815ca70 0x7fb538152270 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:09.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.350+0000 7fb53e200640 1 --2- 172.21.15.86:0/3554041464 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb53809b0f0 0x7fb5380cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:09.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.350+0000 7fb53e200640 1 -- 172.21.15.86:0/3554041464 >> 172.21.15.86:0/3554041464 conn(0x7fb5380b40b0 msgr2=0x7fb53806e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:09.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.350+0000 7fb53e200640 1 -- 172.21.15.86:0/3554041464 shutdown_connections 2024-04-20T21:28:09.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:09.350+0000 7fb53e200640 1 -- 172.21.15.86:0/3554041464 wait complete. 2024-04-20T21:28:09.367 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi086.front.sepia.ceph.com... 2024-04-20T21:28:09.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:28:09.368 DEBUG:teuthology.orchestra.run.smithi086:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-20T21:28:09.432 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[60587]: fuse finished with error 0 and tester_r 0 2024-04-20T21:28:09.439 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:28:15.541 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-20T21:28:15.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:28:15.542 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-20T21:28:15.602 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:28:15.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.807+0000 7f2e9a600640 1 Processor -- start 2024-04-20T21:28:15.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.807+0000 7f2e9a600640 1 -- start start 2024-04-20T21:28:15.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.809+0000 7f2e9a600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2e9415c010 0x7f2e9415c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:15.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.809+0000 7f2e9a600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e940ae710 con 0x7f2e9406f4c0 2024-04-20T21:28:15.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.809+0000 7f2e9a600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e940ae9f0 con 0x7f2e940af7b0 2024-04-20T21:28:15.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.809+0000 7f2e9a600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f2e940af320 con 0x7f2e9415c010 2024-04-20T21:28:15.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.809+0000 7f2e99200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2e9415c010 0x7f2e9415c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:15.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.809+0000 7f2e99c00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f2e9406f4c0 0x7f2e94071b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:47108/0 (socket says 172.21.15.86:47108) 2024-04-20T21:28:15.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.809+0000 7f2e99c00640 1 -- 172.21.15.86:0/2224961000 learned_addr learned my addr 172.21.15.86:0/2224961000 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:28:15.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.809+0000 7f2e99200640 1 -- 172.21.15.86:0/2224961000 >> v1:172.21.15.144:6789/0 conn(0x7f2e940af7b0 legacy=0x7f2e940adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:15.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.809+0000 7f2e99200640 1 -- 172.21.15.86:0/2224961000 >> v1:172.21.15.86:6789/0 conn(0x7f2e9406f4c0 legacy=0x7f2e94071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:15.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.809+0000 7f2e99200640 1 -- 172.21.15.86:0/2224961000 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e940aecd0 con 0x7f2e9415c010 2024-04-20T21:28:15.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.809+0000 7f2e93400640 1 -- 172.21.15.86:0/2224961000 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2703936499 0 0) 0x7f2e940ae710 con 0x7f2e9406f4c0 2024-04-20T21:28:15.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.810+0000 7f2e99200640 1 --2- 172.21.15.86:0/2224961000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e9415c010 0x7f2e9415c3f0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f2e84009d90 tx=0x7f2e8402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdb7b9fbdb4b8d3b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:15.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.810+0000 7f2e93400640 1 -- 172.21.15.86:0/2224961000 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e84039070 con 0x7f2e9415c010 2024-04-20T21:28:15.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.810+0000 7f2e93400640 1 -- 172.21.15.86:0/2224961000 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e8402cc60 con 0x7f2e9415c010 2024-04-20T21:28:15.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.810+0000 7f2e93400640 1 -- 172.21.15.86:0/2224961000 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e84034660 con 0x7f2e9415c010 2024-04-20T21:28:15.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.811+0000 7f2e9a600640 1 -- 172.21.15.86:0/2224961000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e9415c010 msgr2=0x7f2e9415c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:15.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.811+0000 7f2e9a600640 1 --2- 172.21.15.86:0/2224961000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e9415c010 0x7f2e9415c3f0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f2e84009d90 tx=0x7f2e8402beb0 comp rx=0 tx=0).stop 2024-04-20T21:28:15.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.811+0000 7f2e9a600640 1 -- 172.21.15.86:0/2224961000 shutdown_connections 2024-04-20T21:28:15.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.811+0000 7f2e9a600640 1 --2- 172.21.15.86:0/2224961000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e9415c010 0x7f2e9415c3f0 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:15.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.811+0000 7f2e9a600640 1 -- 172.21.15.86:0/2224961000 >> 172.21.15.86:0/2224961000 conn(0x7f2e940d0170 msgr2=0x7f2e940d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:15.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.811+0000 7f2e9a600640 1 -- 172.21.15.86:0/2224961000 shutdown_connections 2024-04-20T21:28:15.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.811+0000 7f2e9a600640 1 -- 172.21.15.86:0/2224961000 wait complete. 2024-04-20T21:28:15.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.813+0000 7f2e9a600640 1 Processor -- start 2024-04-20T21:28:15.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.813+0000 7f2e9a600640 1 -- start start 2024-04-20T21:28:15.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.814+0000 7f2e9a600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e9406f4c0 0x7f2e9409b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:15.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.814+0000 7f2e9a600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e940af7b0 0x7f2e941f3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:15.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.814+0000 7f2e9a600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e9415c010 0x7f2e94150460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:15.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.814+0000 7f2e9a600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2e940ae4b0 con 0x7f2e940af7b0 2024-04-20T21:28:15.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.814+0000 7f2e9a600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2e9415c930 con 0x7f2e9406f4c0 2024-04-20T21:28:15.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.814+0000 7f2e9a600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2e9406f8f0 con 0x7f2e9415c010 2024-04-20T21:28:15.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.814+0000 7f2e93e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e940af7b0 0x7f2e941f3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:15.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.814+0000 7f2e93e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e940af7b0 0x7f2e941f3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40962/0 (socket says 172.21.15.86:40962) 2024-04-20T21:28:15.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.814+0000 7f2e93e00640 1 -- 172.21.15.86:0/2119880054 learned_addr learned my addr 172.21.15.86:0/2119880054 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:15.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.815+0000 7f2e99200640 1 --2- 172.21.15.86:0/2119880054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e9406f4c0 0x7f2e9409b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:15.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.815+0000 7f2e99c00640 1 --2- 172.21.15.86:0/2119880054 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e9415c010 0x7f2e94150460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:15.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.815+0000 7f2e93e00640 1 -- 172.21.15.86:0/2119880054 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e9415c010 msgr2=0x7f2e94150460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:15.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.815+0000 7f2e93e00640 1 --2- 172.21.15.86:0/2119880054 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e9415c010 0x7f2e94150460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:15.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.815+0000 7f2e93e00640 1 -- 172.21.15.86:0/2119880054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e9406f4c0 msgr2=0x7f2e9409b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:15.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.815+0000 7f2e93e00640 1 --2- 172.21.15.86:0/2119880054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e9406f4c0 0x7f2e9409b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:15.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.815+0000 7f2e93e00640 1 -- 172.21.15.86:0/2119880054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e9409a7d0 con 0x7f2e940af7b0 2024-04-20T21:28:15.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.815+0000 7f2e93e00640 1 --2- 172.21.15.86:0/2119880054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e940af7b0 0x7f2e941f3a60 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f2e8000c430 tx=0x7f2e8000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:15.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.815+0000 7f2e90c00640 1 -- 172.21.15.86:0/2119880054 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e80011790 con 0x7f2e940af7b0 2024-04-20T21:28:15.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.815+0000 7f2e90c00640 1 -- 172.21.15.86:0/2119880054 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e80017070 con 0x7f2e940af7b0 2024-04-20T21:28:15.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.815+0000 7f2e90c00640 1 -- 172.21.15.86:0/2119880054 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e80011bb0 con 0x7f2e940af7b0 2024-04-20T21:28:15.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.815+0000 7f2e9a600640 1 -- 172.21.15.86:0/2119880054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e9409aab0 con 0x7f2e940af7b0 2024-04-20T21:28:15.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.815+0000 7f2e9a600640 1 -- 172.21.15.86:0/2119880054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e9409b350 con 0x7f2e940af7b0 2024-04-20T21:28:15.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.816+0000 7f2e9a600640 1 -- 172.21.15.86:0/2119880054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e9409ac80 con 0x7f2e940af7b0 2024-04-20T21:28:15.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.818+0000 7f2e90c00640 1 -- 172.21.15.86:0/2119880054 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2e8001c460 con 0x7f2e940af7b0 2024-04-20T21:28:15.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.818+0000 7f2e90c00640 1 --2- 172.21.15.86:0/2119880054 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2e7406c8f0 0x7f2e7406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:15.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.819+0000 7f2e90c00640 1 -- 172.21.15.86:0/2119880054 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7f2e8008bac0 con 0x7f2e940af7b0 2024-04-20T21:28:15.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.819+0000 7f2e99200640 1 --2- 172.21.15.86:0/2119880054 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2e7406c8f0 0x7f2e7406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:15.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.819+0000 7f2e99200640 1 --2- 172.21.15.86:0/2119880054 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2e7406c8f0 0x7f2e7406edb0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f2e84036000 tx=0x7f2e84030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:15.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:15.823+0000 7f2e90c00640 1 -- 172.21.15.86:0/2119880054 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2e80059e50 con 0x7f2e940af7b0 2024-04-20T21:28:16.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.063+0000 7f2e9a600640 1 -- 172.21.15.86:0/2119880054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2e9404fcf0 con 0x7f2e940af7b0 2024-04-20T21:28:16.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.064+0000 7f2e90c00640 1 -- 172.21.15.86:0/2119880054 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5207 (secure 0 0 0) 0x7f2e80059750 con 0x7f2e940af7b0 2024-04-20T21:28:16.065 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:28:16.065 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":82,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5525,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3902778887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3902778887},{"type":"v1","addr":"172.21.15.144:6835","nonce":3902778887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6001,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/4270988014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":4270988014},{"type":"v1","addr":"172.21.15.86:6837","nonce":4270988014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6016,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/1569462554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":1569462554},{"type":"v1","addr":"172.21.15.144:6833","nonce":1569462554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:27:45.692408+0000","modified":"2024-04-20T21:27:48.770936+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8493":{"gid":8493,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6836/1331851138","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1331851138},{"type":"v1","addr":"172.21.15.86:6836","nonce":1331851138}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8493,"qdb_cluster":[8493]},"id":7}]} 2024-04-20T21:28:16.065 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 82 2024-04-20T21:28:16.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.068+0000 7f2e9a600640 1 -- 172.21.15.86:0/2119880054 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2e7406c8f0 msgr2=0x7f2e7406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:16.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.068+0000 7f2e9a600640 1 --2- 172.21.15.86:0/2119880054 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2e7406c8f0 0x7f2e7406edb0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f2e84036000 tx=0x7f2e84030000 comp rx=0 tx=0).stop 2024-04-20T21:28:16.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.068+0000 7f2e9a600640 1 -- 172.21.15.86:0/2119880054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e940af7b0 msgr2=0x7f2e941f3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:16.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.068+0000 7f2e9a600640 1 --2- 172.21.15.86:0/2119880054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e940af7b0 0x7f2e941f3a60 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f2e8000c430 tx=0x7f2e8000c8f0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.068+0000 7f2e9a600640 1 -- 172.21.15.86:0/2119880054 shutdown_connections 2024-04-20T21:28:16.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.068+0000 7f2e9a600640 1 --2- 172.21.15.86:0/2119880054 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2e7406c8f0 0x7f2e7406edb0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.068+0000 7f2e9a600640 1 --2- 172.21.15.86:0/2119880054 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2e9415c010 0x7f2e94150460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.068+0000 7f2e9a600640 1 --2- 172.21.15.86:0/2119880054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e940af7b0 0x7f2e941f3a60 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.068+0000 7f2e9a600640 1 --2- 172.21.15.86:0/2119880054 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e9406f4c0 0x7f2e9409b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.068+0000 7f2e9a600640 1 -- 172.21.15.86:0/2119880054 >> 172.21.15.86:0/2119880054 conn(0x7f2e940d0170 msgr2=0x7f2e9415dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:16.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.068+0000 7f2e9a600640 1 -- 172.21.15.86:0/2119880054 shutdown_connections 2024-04-20T21:28:16.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.068+0000 7f2e9a600640 1 -- 172.21.15.86:0/2119880054 wait complete. 2024-04-20T21:28:16.086 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-20T21:28:16.319 INFO:teuthology.orchestra.run.smithi086.stdout:{"message":"","return_code":0} 2024-04-20T21:28:16.319 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-20T21:28:16.320 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:28:16.320 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:28:16.370 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052731,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248996k,nr_inodes=812249,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T21:28:16.370 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi086.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2024-04-20T21:28:16.370 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-20T21:28:16.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:28:16.370 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-20T21:28:16.463 INFO:teuthology.orchestra.run.smithi086.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-20T21:28:16.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:28:16.464 DEBUG:teuthology.orchestra.run.smithi086:> sudo daemon-helper kill ceph --cluster ceph -w 2024-04-20T21:28:16.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:28:16.470 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-04-20T21:28:16.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.650+0000 7f9ced600640 1 Processor -- start 2024-04-20T21:28:16.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.650+0000 7f9ced600640 1 -- start start 2024-04-20T21:28:16.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.652+0000 7f9ced600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ce815e840 0x7f9ce80c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:16.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.652+0000 7f9ced600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9ce8163080 0x7f9ce80d4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:16.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.652+0000 7f9ced600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ce80d3510 con 0x7f9ce815ec20 2024-04-20T21:28:16.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.652+0000 7f9ced600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9ce8163530 con 0x7f9ce8163080 2024-04-20T21:28:16.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.652+0000 7f9ced600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9ce80cf5b0 con 0x7f9ce815e840 2024-04-20T21:28:16.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.652+0000 7f9ce7400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9ce8163080 0x7f9ce80d4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:16.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.652+0000 7f9ce7400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9ce8163080 0x7f9ce80d4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43834/0 (socket says 172.21.15.86:43834) 2024-04-20T21:28:16.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.652+0000 7f9ce7400640 1 -- 172.21.15.86:0/1508802656 learned_addr learned my addr 172.21.15.86:0/1508802656 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:16.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.653+0000 7f9ce6a00640 1 --2- 172.21.15.86:0/1508802656 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ce815e840 0x7f9ce80c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:16.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.653+0000 7f9ce7400640 1 -- 172.21.15.86:0/1508802656 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ce815e840 msgr2=0x7f9ce80c82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:16.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.653+0000 7f9ce7400640 1 --2- 172.21.15.86:0/1508802656 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ce815e840 0x7f9ce80c82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.653+0000 7f9ce7400640 1 -- 172.21.15.86:0/1508802656 >> v1:172.21.15.144:6789/0 conn(0x7f9ce815ec20 legacy=0x7f9ce80d2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:16.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.653+0000 7f9ce7400640 1 -- 172.21.15.86:0/1508802656 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ce80cf890 con 0x7f9ce8163080 2024-04-20T21:28:16.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.654+0000 7f9ce7400640 1 --2- 172.21.15.86:0/1508802656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ce8163080 0x7f9ce80d4400 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0009870 tx=0x7f9ce002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad074d0340878a8a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:16.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.654+0000 7f9ce6000640 1 -- 172.21.15.86:0/1508802656 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce0035020 con 0x7f9ce8163080 2024-04-20T21:28:16.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.654+0000 7f9ce6000640 1 -- 172.21.15.86:0/1508802656 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ce0031640 con 0x7f9ce8163080 2024-04-20T21:28:16.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.654+0000 7f9ce6000640 1 -- 172.21.15.86:0/1508802656 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce002d040 con 0x7f9ce8163080 2024-04-20T21:28:16.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.655+0000 7f9ced600640 1 -- 172.21.15.86:0/1508802656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ce8163080 msgr2=0x7f9ce80d4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:16.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.655+0000 7f9ced600640 1 --2- 172.21.15.86:0/1508802656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ce8163080 0x7f9ce80d4400 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0009870 tx=0x7f9ce002b8f0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.655+0000 7f9ced600640 1 -- 172.21.15.86:0/1508802656 shutdown_connections 2024-04-20T21:28:16.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.655+0000 7f9ced600640 1 --2- 172.21.15.86:0/1508802656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ce8163080 0x7f9ce80d4400 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.655+0000 7f9ced600640 1 --2- 172.21.15.86:0/1508802656 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ce815e840 0x7f9ce80c82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.655+0000 7f9ced600640 1 -- 172.21.15.86:0/1508802656 >> 172.21.15.86:0/1508802656 conn(0x7f9ce809da40 msgr2=0x7f9ce808bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:16.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.655+0000 7f9ced600640 1 -- 172.21.15.86:0/1508802656 shutdown_connections 2024-04-20T21:28:16.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.655+0000 7f9ced600640 1 -- 172.21.15.86:0/1508802656 wait complete. 2024-04-20T21:28:16.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.657+0000 7f9ced600640 1 Processor -- start 2024-04-20T21:28:16.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.657+0000 7f9ced600640 1 -- start start 2024-04-20T21:28:16.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.658+0000 7f9ced600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ce815e840 0x7f9ce81584f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:16.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.658+0000 7f9ced600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ce815ec20 0x7f9ce81577d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:16.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.658+0000 7f9ced600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ce8163080 0x7f9ce8160750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:16.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.658+0000 7f9ced600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce80d4940 con 0x7f9ce815e840 2024-04-20T21:28:16.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.658+0000 7f9ced600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce80cf260 con 0x7f9ce8163080 2024-04-20T21:28:16.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.658+0000 7f9ced600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9ce80d4ac0 con 0x7f9ce815ec20 2024-04-20T21:28:16.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.658+0000 7f9ce6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ce815e840 0x7f9ce81584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:16.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.658+0000 7f9ce6a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ce815e840 0x7f9ce81584f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43838/0 (socket says 172.21.15.86:43838) 2024-04-20T21:28:16.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.658+0000 7f9ce6a00640 1 -- 172.21.15.86:0/1083171950 learned_addr learned my addr 172.21.15.86:0/1083171950 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:16.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.659+0000 7f9ce6a00640 1 -- 172.21.15.86:0/1083171950 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ce815ec20 msgr2=0x7f9ce81577d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:16.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.659+0000 7f9ce6a00640 1 --2- 172.21.15.86:0/1083171950 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ce815ec20 0x7f9ce81577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.659+0000 7f9ce6a00640 1 -- 172.21.15.86:0/1083171950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ce8163080 msgr2=0x7f9ce8160750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:16.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.659+0000 7f9ce6a00640 1 --2- 172.21.15.86:0/1083171950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ce8163080 0x7f9ce8160750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.659+0000 7f9ce6a00640 1 -- 172.21.15.86:0/1083171950 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ce81f4ef0 con 0x7f9ce815e840 2024-04-20T21:28:16.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.659+0000 7f9ce6a00640 1 --2- 172.21.15.86:0/1083171950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ce815e840 0x7f9ce81584f0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f9cd800ce10 tx=0x7f9cd800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:16.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.659+0000 7f9cdf400640 1 -- 172.21.15.86:0/1083171950 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd8003b90 con 0x7f9ce815e840 2024-04-20T21:28:16.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.659+0000 7f9cdf400640 1 -- 172.21.15.86:0/1083171950 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cd8003d30 con 0x7f9ce815e840 2024-04-20T21:28:16.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.659+0000 7f9ced600640 1 -- 172.21.15.86:0/1083171950 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ce81f51d0 con 0x7f9ce815e840 2024-04-20T21:28:16.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.659+0000 7f9cdf400640 1 -- 172.21.15.86:0/1083171950 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd8004240 con 0x7f9ce815e840 2024-04-20T21:28:16.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.659+0000 7f9ced600640 1 -- 172.21.15.86:0/1083171950 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ce81f5790 con 0x7f9ce815e840 2024-04-20T21:28:16.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.662+0000 7f9cdf400640 1 -- 172.21.15.86:0/1083171950 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9cd80043e0 con 0x7f9ce815e840 2024-04-20T21:28:16.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.663+0000 7f9cdf400640 1 --2- 172.21.15.86:0/1083171950 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9cc406c8f0 0x7f9cc406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:16.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.663+0000 7f9cdf400640 1 -- 172.21.15.86:0/1083171950 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7f9cd808c640 con 0x7f9ce815e840 2024-04-20T21:28:16.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.664+0000 7f9cde000640 1 --2- 172.21.15.86:0/1083171950 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9cc406c8f0 0x7f9cc406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:16.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.664+0000 7f9cf03f8740 1 Processor -- start 2024-04-20T21:28:16.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.664+0000 7f9cf03f8740 1 -- start start 2024-04-20T21:28:16.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.664+0000 7f9cde000640 1 --2- 172.21.15.86:0/1083171950 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9cc406c8f0 0x7f9cc406edb0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f9ce81f2790 tx=0x7f9cd0007990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:16.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.665+0000 7f9cf03f8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e75e633fe0 0x55e75e606f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:16.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.665+0000 7f9cf03f8740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e75e6343c0 0x55e75e636d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:16.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.665+0000 7f9cf03f8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e75e6074c0 0x55e75e6372b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:16.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.665+0000 7f9cf03f8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e75e571d20 con 0x55e75e6074c0 2024-04-20T21:28:16.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.665+0000 7f9cf03f8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e75e5183e0 con 0x55e75e633fe0 2024-04-20T21:28:16.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.665+0000 7f9cf03f8740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55e75e56dce0 con 0x55e75e6343c0 2024-04-20T21:28:16.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.665+0000 7f9ce7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e75e6074c0 0x55e75e6372b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:16.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.665+0000 7f9ce7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e75e6074c0 0x55e75e6372b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43846/0 (socket says 172.21.15.86:43846) 2024-04-20T21:28:16.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.665+0000 7f9ce7400640 1 -- 172.21.15.86:0/2163431233 learned_addr learned my addr 172.21.15.86:0/2163431233 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:16.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.665+0000 7f9cde000640 1 --2- 172.21.15.86:0/2163431233 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e75e6343c0 0x55e75e636d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:16.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.665+0000 7f9ce6a00640 1 --2- 172.21.15.86:0/2163431233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e75e633fe0 0x55e75e606f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:16.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.665+0000 7f9ce7400640 1 -- 172.21.15.86:0/2163431233 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e75e6343c0 msgr2=0x55e75e636d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:16.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.665+0000 7f9ce7400640 1 --2- 172.21.15.86:0/2163431233 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e75e6343c0 0x55e75e636d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.665+0000 7f9ce7400640 1 -- 172.21.15.86:0/2163431233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e75e633fe0 msgr2=0x55e75e606f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:16.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.665+0000 7f9ce7400640 1 --2- 172.21.15.86:0/2163431233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e75e633fe0 0x55e75e606f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.666+0000 7f9ce7400640 1 -- 172.21.15.86:0/2163431233 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e75e537980 con 0x55e75e6074c0 2024-04-20T21:28:16.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.666+0000 7f9ce7400640 1 --2- 172.21.15.86:0/2163431233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e75e6074c0 0x55e75e6372b0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f9ce002b930 tx=0x7f9ce002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=639cc7e0859b56d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:16.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.666+0000 7f9cd7000640 1 -- 172.21.15.86:0/2163431233 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce0040070 con 0x55e75e6074c0 2024-04-20T21:28:16.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.666+0000 7f9cd7000640 1 -- 172.21.15.86:0/2163431233 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ce0031890 con 0x55e75e6074c0 2024-04-20T21:28:16.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.666+0000 7f9cd7000640 1 -- 172.21.15.86:0/2163431233 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce0035040 con 0x55e75e6074c0 2024-04-20T21:28:16.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.667+0000 7f9cf03f8740 1 -- 172.21.15.86:0/2163431233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e75e6074c0 msgr2=0x55e75e6372b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:16.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.667+0000 7f9cf03f8740 1 --2- 172.21.15.86:0/2163431233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e75e6074c0 0x55e75e6372b0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f9ce002b930 tx=0x7f9ce002e000 comp rx=0 tx=0).stop 2024-04-20T21:28:16.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.667+0000 7f9cf03f8740 1 -- 172.21.15.86:0/2163431233 shutdown_connections 2024-04-20T21:28:16.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.667+0000 7f9cf03f8740 1 --2- 172.21.15.86:0/2163431233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e75e6074c0 0x55e75e6372b0 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.667+0000 7f9cf03f8740 1 --2- 172.21.15.86:0/2163431233 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e75e6343c0 0x55e75e636d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.667+0000 7f9cf03f8740 1 --2- 172.21.15.86:0/2163431233 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e75e633fe0 0x55e75e606f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.667+0000 7f9cf03f8740 1 -- 172.21.15.86:0/2163431233 >> 172.21.15.86:0/2163431233 conn(0x55e75e6091f0 msgr2=0x55e75e651d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:16.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.667+0000 7f9cf03f8740 1 -- 172.21.15.86:0/2163431233 shutdown_connections 2024-04-20T21:28:16.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.667+0000 7f9cf03f8740 1 -- 172.21.15.86:0/2163431233 wait complete. 2024-04-20T21:28:16.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.668+0000 7f9cf03f8740 1 -- start start 2024-04-20T21:28:16.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.668+0000 7f9cf03f8740 1 Processor -- start 2024-04-20T21:28:16.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.669+0000 7f9cf03f8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e75e633fe0 0x55e75e635160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:16.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.669+0000 7f9cf03f8740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e75e6343c0 0x55e75e62fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:16.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.669+0000 7f9cf03f8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e75e62efc0 0x55e75e62cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:16.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.669+0000 7f9cf03f8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e75e60b890 con 0x55e75e62efc0 2024-04-20T21:28:16.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.669+0000 7f9cf03f8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e75e52b7a0 con 0x55e75e633fe0 2024-04-20T21:28:16.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.669+0000 7f9cf03f8740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55e75e60e0d0 con 0x55e75e6343c0 2024-04-20T21:28:16.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.669+0000 7f9cf03f8740 10 client.0 _ll_register_callbacks cb 0x55e75e52db80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:28:16.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.669+0000 7f9ce7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e75e62efc0 0x55e75e62cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:16.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.669+0000 7f9ce7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e75e62efc0 0x55e75e62cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43858/0 (socket says 172.21.15.86:43858) 2024-04-20T21:28:16.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.669+0000 7f9ce7400640 1 -- 172.21.15.86:0/19199828 learned_addr learned my addr 172.21.15.86:0/19199828 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:16.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.670+0000 7f9ce6a00640 1 --2- 172.21.15.86:0/19199828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e75e633fe0 0x55e75e635160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:16.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.670+0000 7f9cde000640 1 --2- 172.21.15.86:0/19199828 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e75e6343c0 0x55e75e62fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:16.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.670+0000 7f9ce7400640 1 -- 172.21.15.86:0/19199828 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e75e6343c0 msgr2=0x55e75e62fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:16.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.670+0000 7f9ce7400640 1 --2- 172.21.15.86:0/19199828 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e75e6343c0 0x55e75e62fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.670+0000 7f9ce7400640 1 -- 172.21.15.86:0/19199828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e75e633fe0 msgr2=0x55e75e635160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:16.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.670+0000 7f9ce7400640 1 --2- 172.21.15.86:0/19199828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e75e633fe0 0x55e75e635160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.670+0000 7f9ce7400640 1 -- 172.21.15.86:0/19199828 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e75e34c7e0 con 0x55e75e62efc0 2024-04-20T21:28:16.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.670+0000 7f9ce7400640 1 --2- 172.21.15.86:0/19199828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e75e62efc0 0x55e75e62cd80 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f9ce00090b0 tx=0x7f9ce0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edd518a64b3a19b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:16.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.670+0000 7f9ccb400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:28:16.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.670+0000 7f9ccb400640 1 -- 172.21.15.86:0/19199828 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce0053060 con 0x55e75e62efc0 2024-04-20T21:28:16.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.671+0000 7f9ccb400640 1 -- 172.21.15.86:0/19199828 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7f9ce0031c50 con 0x55e75e62efc0 2024-04-20T21:28:16.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.671+0000 7f9cf03f8740 1 -- 172.21.15.86:0/19199828 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ce0031890 con 0x55e75e62efc0 2024-04-20T21:28:16.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.671+0000 7f9ccb400640 1 -- 172.21.15.86:0/19199828 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f9ce0059050 con 0x55e75e62efc0 2024-04-20T21:28:16.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.671+0000 7f9cf03f8740 10 client.8853 fetch_fsmap learned FSMap version 82 2024-04-20T21:28:16.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.671+0000 7f9cf03f8740 1 -- 172.21.15.86:0/19199828 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x55e75e3acf70 con 0x55e75e62efc0 2024-04-20T21:28:16.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.671+0000 7f9ccb400640 1 -- 172.21.15.86:0/19199828 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ce0031890 con 0x55e75e62efc0 2024-04-20T21:28:16.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.671+0000 7f9ccb400640 1 -- 172.21.15.86:0/19199828 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 82) ==== 3495+0+0 (secure 0 0 0) 0x7f9ce005c020 con 0x55e75e62efc0 2024-04-20T21:28:16.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.671+0000 7f9cf03f8740 10 client.8853 fetch_fsmap finished waiting for FSMap version 82 2024-04-20T21:28:16.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.671+0000 7f9cf03f8740 10 client.8853 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-20T21:28:16.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.671+0000 7f9cf03f8740 20 client.8853 populate_metadata read hostname 'smithi086' 2024-04-20T21:28:16.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.671+0000 7f9cf03f8740 1 --2- 172.21.15.86:0/19199828 >> [v2:172.21.15.86:6834/1331851138,v1:172.21.15.86:6836/1331851138] conn(0x55e75e61bdf0 0x55e75e61c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:16.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.671+0000 7f9cf03f8740 4 client.8853 mds_command: new command op to 8493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:28:16.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.671+0000 7f9cf03f8740 1 -- 172.21.15.86:0/19199828 --> [v2:172.21.15.86:6834/1331851138,v1:172.21.15.86:6836/1331851138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e75e57f740 con 0x55e75e61bdf0 2024-04-20T21:28:16.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.672+0000 7f9ce6a00640 1 --2- 172.21.15.86:0/19199828 >> [v2:172.21.15.86:6834/1331851138,v1:172.21.15.86:6836/1331851138] conn(0x55e75e61bdf0 0x55e75e61c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:16.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.672+0000 7f9ce6a00640 1 --2- 172.21.15.86:0/19199828 >> [v2:172.21.15.86:6834/1331851138,v1:172.21.15.86:6836/1331851138] conn(0x55e75e61bdf0 0x55e75e61c200 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf6e5b6cceb2060f server_cookie=fa53cb663d52120a in_seq=0 out_seq=0 2024-04-20T21:28:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.672+0000 7f9ccb400640 10 client.8853 ms_handle_connect on v2:172.21.15.86:6834/1331851138 2024-04-20T21:28:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.673+0000 7f9ccb400640 1 -- 172.21.15.86:0/19199828 <== mds.0 v2:172.21.15.86:6834/1331851138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e75e57f740 con 0x55e75e61bdf0 2024-04-20T21:28:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.673+0000 7f9ccb400640 10 client.8853 handle_command_reply: tid=0 2024-04-20T21:28:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.673+0000 7f9cf03f8740 1 client.8853 shutdown 2024-04-20T21:28:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.673+0000 7f9cf03f8740 1 -- 172.21.15.86:0/19199828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e75e62efc0 msgr2=0x55e75e62cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.673+0000 7f9cf03f8740 1 --2- 172.21.15.86:0/19199828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e75e62efc0 0x55e75e62cd80 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f9ce00090b0 tx=0x7f9ce0004730 comp rx=0 tx=0).stop 2024-04-20T21:28:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.673+0000 7f9cf03f8740 1 -- 172.21.15.86:0/19199828 shutdown_connections 2024-04-20T21:28:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.673+0000 7f9cf03f8740 1 --2- 172.21.15.86:0/19199828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e75e62efc0 0x55e75e62cd80 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.673+0000 7f9cf03f8740 1 --2- 172.21.15.86:0/19199828 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e75e6343c0 0x55e75e62fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.673+0000 7f9cf03f8740 1 --2- 172.21.15.86:0/19199828 >> [v2:172.21.15.86:6834/1331851138,v1:172.21.15.86:6836/1331851138] conn(0x55e75e61bdf0 0x55e75e61c200 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.673+0000 7f9cf03f8740 1 --2- 172.21.15.86:0/19199828 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e75e633fe0 0x55e75e635160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.673+0000 7f9ccb400640 0 client.8853 ms_handle_reset on v2:172.21.15.86:6834/1331851138 2024-04-20T21:28:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.673+0000 7f9cf03f8740 1 -- 172.21.15.86:0/19199828 >> 172.21.15.86:0/19199828 conn(0x55e75e6074c0 msgr2=0x55e75e60b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.674+0000 7f9cf03f8740 1 -- 172.21.15.86:0/19199828 shutdown_connections 2024-04-20T21:28:16.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.674+0000 7f9cf03f8740 1 -- 172.21.15.86:0/19199828 wait complete. 2024-04-20T21:28:16.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.674+0000 7f9cf03f8740 20 client.8853 trim_cache size 0 max 16384 2024-04-20T21:28:16.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.679+0000 7f7e53e00640 1 Processor -- start 2024-04-20T21:28:16.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.679+0000 7f7e53e00640 1 -- start start 2024-04-20T21:28:16.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.680+0000 7f7e53e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7e5415e890 0x7f7e5406da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:16.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.680+0000 7f7e53e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7e540b6930 0x7f7e540cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:16.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.681+0000 7f7e53e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e5406e380 con 0x7f7e540d3270 2024-04-20T21:28:16.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.681+0000 7f7e53e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f7e5406e660 con 0x7f7e5415e890 2024-04-20T21:28:16.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.681+0000 7f7e53e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f7e5406e940 con 0x7f7e540b6930 2024-04-20T21:28:16.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.681+0000 7f7e52a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7e5415e890 0x7f7e5406da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:16.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.681+0000 7f7e52a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7e5415e890 0x7f7e5406da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43870/0 (socket says 172.21.15.86:43870) 2024-04-20T21:28:16.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.681+0000 7f7e52a00640 1 -- 172.21.15.86:0/3560173052 learned_addr learned my addr 172.21.15.86:0/3560173052 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:16.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.681+0000 7f7e52a00640 1 -- 172.21.15.86:0/3560173052 >> v2:172.21.15.144:3300/0 conn(0x7f7e540b6930 msgr2=0x7f7e540cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:16.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.681+0000 7f7e52a00640 1 --2- 172.21.15.86:0/3560173052 >> v2:172.21.15.144:3300/0 conn(0x7f7e540b6930 0x7f7e540cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.681+0000 7f7e52a00640 1 -- 172.21.15.86:0/3560173052 >> v1:172.21.15.144:6789/0 conn(0x7f7e540d3270 legacy=0x7f7e540b6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:16.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.681+0000 7f7e52a00640 1 -- 172.21.15.86:0/3560173052 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e540cf370 con 0x7f7e5415e890 2024-04-20T21:28:16.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.682+0000 7f7e52a00640 1 --2- 172.21.15.86:0/3560173052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e5415e890 0x7f7e5406da10 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f7e5408e980 tx=0x7f7e4402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=986b650e83c40994 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:16.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.683+0000 7f7e51600640 1 -- 172.21.15.86:0/3560173052 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e44004230 con 0x7f7e5415e890 2024-04-20T21:28:16.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.683+0000 7f7e51600640 1 -- 172.21.15.86:0/3560173052 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e440043d0 con 0x7f7e5415e890 2024-04-20T21:28:16.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.683+0000 7f7e51600640 1 -- 172.21.15.86:0/3560173052 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e44030840 con 0x7f7e5415e890 2024-04-20T21:28:16.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.683+0000 7f7e53e00640 1 -- 172.21.15.86:0/3560173052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e5415e890 msgr2=0x7f7e5406da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:16.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.683+0000 7f7e53e00640 1 --2- 172.21.15.86:0/3560173052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e5415e890 0x7f7e5406da10 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f7e5408e980 tx=0x7f7e4402b980 comp rx=0 tx=0).stop 2024-04-20T21:28:16.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.683+0000 7f7e53e00640 1 -- 172.21.15.86:0/3560173052 shutdown_connections 2024-04-20T21:28:16.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.683+0000 7f7e53e00640 1 --2- 172.21.15.86:0/3560173052 >> v2:172.21.15.144:3300/0 conn(0x7f7e540b6930 0x7f7e540cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.683+0000 7f7e53e00640 1 --2- 172.21.15.86:0/3560173052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e5415e890 0x7f7e5406da10 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.683+0000 7f7e53e00640 1 -- 172.21.15.86:0/3560173052 >> 172.21.15.86:0/3560173052 conn(0x7f7e5409dab0 msgr2=0x7f7e5408bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:16.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.683+0000 7f7e53e00640 1 -- 172.21.15.86:0/3560173052 shutdown_connections 2024-04-20T21:28:16.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.683+0000 7f7e53e00640 1 -- 172.21.15.86:0/3560173052 wait complete. 2024-04-20T21:28:16.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.685+0000 7f7e53e00640 1 Processor -- start 2024-04-20T21:28:16.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.685+0000 7f7e53e00640 1 -- start start 2024-04-20T21:28:16.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.686+0000 7f7e53e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e540b6930 0x7f7e540b2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:16.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.686+0000 7f7e53e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e540d3270 0x7f7e540ce180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:16.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.686+0000 7f7e53e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e5415e890 0x7f7e540ca630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:16.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.686+0000 7f7e53e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5406e0d0 con 0x7f7e5415e890 2024-04-20T21:28:16.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.686+0000 7f7e53e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5406df50 con 0x7f7e540d3270 2024-04-20T21:28:16.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.686+0000 7f7e53e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7e540cef00 con 0x7f7e540b6930 2024-04-20T21:28:16.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.686+0000 7f7e53400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e5415e890 0x7f7e540ca630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:16.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.687+0000 7f7e53400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e5415e890 0x7f7e540ca630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43876/0 (socket says 172.21.15.86:43876) 2024-04-20T21:28:16.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.687+0000 7f7e53400640 1 -- 172.21.15.86:0/1487035696 learned_addr learned my addr 172.21.15.86:0/1487035696 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:16.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.687+0000 7f7e52a00640 1 --2- 172.21.15.86:0/1487035696 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e540b6930 0x7f7e540b2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:16.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.687+0000 7f7e52000640 1 --2- 172.21.15.86:0/1487035696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e540d3270 0x7f7e540ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:16.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.687+0000 7f7e53400640 1 -- 172.21.15.86:0/1487035696 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e540b6930 msgr2=0x7f7e540b2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:16.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.687+0000 7f7e53400640 1 --2- 172.21.15.86:0/1487035696 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e540b6930 0x7f7e540b2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.687+0000 7f7e53400640 1 -- 172.21.15.86:0/1487035696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e540d3270 msgr2=0x7f7e540ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:16.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.687+0000 7f7e53400640 1 --2- 172.21.15.86:0/1487035696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e540d3270 0x7f7e540ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.687+0000 7f7e53400640 1 -- 172.21.15.86:0/1487035696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e540b1cd0 con 0x7f7e5415e890 2024-04-20T21:28:16.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.687+0000 7f7e53400640 1 --2- 172.21.15.86:0/1487035696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e5415e890 0x7f7e540ca630 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c00b9e0 tx=0x7f7e4c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:16.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.688+0000 7f7e4aa00640 1 -- 172.21.15.86:0/1487035696 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c019070 con 0x7f7e5415e890 2024-04-20T21:28:16.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.688+0000 7f7e4aa00640 1 -- 172.21.15.86:0/1487035696 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e4c00cb40 con 0x7f7e5415e890 2024-04-20T21:28:16.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.688+0000 7f7e4aa00640 1 -- 172.21.15.86:0/1487035696 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c014440 con 0x7f7e5415e890 2024-04-20T21:28:16.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.688+0000 7f7e53e00640 1 -- 172.21.15.86:0/1487035696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e540b1fb0 con 0x7f7e5415e890 2024-04-20T21:28:16.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.688+0000 7f7e53e00640 1 -- 172.21.15.86:0/1487035696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e540b2570 con 0x7f7e5415e890 2024-04-20T21:28:16.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.689+0000 7f7e53e00640 1 -- 172.21.15.86:0/1487035696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f7e5404fcf0 con 0x7f7e5415e890 2024-04-20T21:28:16.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.690+0000 7f7e4aa00640 1 -- 172.21.15.86:0/1487035696 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7e4c01c070 con 0x7f7e5415e890 2024-04-20T21:28:16.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.691+0000 7f7e4aa00640 1 --2- 172.21.15.86:0/1487035696 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7e3406c8f0 0x7f7e3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:16.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.691+0000 7f7e4aa00640 1 -- 172.21.15.86:0/1487035696 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7f7e4c08bf90 con 0x7f7e5415e890 2024-04-20T21:28:16.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.692+0000 7f7e52a00640 1 --2- 172.21.15.86:0/1487035696 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7e3406c8f0 0x7f7e3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:16.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.692+0000 7f7e4aa00640 1 -- 172.21.15.86:0/1487035696 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+515 (secure 0 0 0) 0x7f7e4c023070 con 0x7f7e5415e890 2024-04-20T21:28:16.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.692+0000 7f7e53e00640 1 -- 172.21.15.86:0/1487035696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7f7e540b54d0 con 0x7f7e5415e890 2024-04-20T21:28:16.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.692+0000 7f7e52a00640 1 --2- 172.21.15.86:0/1487035696 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7e3406c8f0 0x7f7e3406edb0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f7e44035000 tx=0x7f7e44008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:16.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.694+0000 7f9cf03f8740 1 Processor -- start 2024-04-20T21:28:16.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.695+0000 7f9cf03f8740 1 -- start start 2024-04-20T21:28:16.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.695+0000 7f9cf03f8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e75e62efc0 0x55e75e61ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:16.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.695+0000 7f9cf03f8740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e75e633fe0 0x55e75e61b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:16.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.695+0000 7f9cf03f8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e75e6343c0 0x55e75e652f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:16.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.695+0000 7f9cf03f8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce8200f70 con 0x55e75e6343c0 2024-04-20T21:28:16.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.696+0000 7f9cf03f8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce81f8be0 con 0x55e75e62efc0 2024-04-20T21:28:16.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.696+0000 7f9cf03f8740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9ce80cc800 con 0x55e75e633fe0 2024-04-20T21:28:16.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.696+0000 7f9ce7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e75e6343c0 0x55e75e652f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:16.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.696+0000 7f9ce7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e75e6343c0 0x55e75e652f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43892/0 (socket says 172.21.15.86:43892) 2024-04-20T21:28:16.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.696+0000 7f9ce7400640 1 -- 172.21.15.86:0/3577656216 learned_addr learned my addr 172.21.15.86:0/3577656216 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:16.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.696+0000 7f9ce7400640 1 -- 172.21.15.86:0/3577656216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e75e633fe0 msgr2=0x55e75e61b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:16.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.696+0000 7f9ce7400640 1 --2- 172.21.15.86:0/3577656216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e75e633fe0 0x55e75e61b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.696+0000 7f9ce7400640 1 -- 172.21.15.86:0/3577656216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e75e62efc0 msgr2=0x55e75e61ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:16.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.696+0000 7f9ce7400640 1 --2- 172.21.15.86:0/3577656216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e75e62efc0 0x55e75e61ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.696+0000 7f9ce7400640 1 -- 172.21.15.86:0/3577656216 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e75e244170 con 0x55e75e6343c0 2024-04-20T21:28:16.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.696+0000 7f9ce7400640 1 --2- 172.21.15.86:0/3577656216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e75e6343c0 0x55e75e652f30 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0004650 tx=0x7f9ce0033070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d991844e5acb2f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:16.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.697+0000 7f9cd6600640 1 -- 172.21.15.86:0/3577656216 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce0040040 con 0x55e75e6343c0 2024-04-20T21:28:16.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.697+0000 7f9cd6600640 1 -- 172.21.15.86:0/3577656216 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ce0003610 con 0x55e75e6343c0 2024-04-20T21:28:16.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.697+0000 7f9cd6600640 1 -- 172.21.15.86:0/3577656216 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce0035040 con 0x55e75e6343c0 2024-04-20T21:28:16.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.697+0000 7f9cf03f8740 1 -- 172.21.15.86:0/3577656216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e75e6343c0 msgr2=0x55e75e652f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:16.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.697+0000 7f9cf03f8740 1 --2- 172.21.15.86:0/3577656216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e75e6343c0 0x55e75e652f30 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0004650 tx=0x7f9ce0033070 comp rx=0 tx=0).stop 2024-04-20T21:28:16.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.698+0000 7f9cf03f8740 1 -- 172.21.15.86:0/3577656216 shutdown_connections 2024-04-20T21:28:16.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.698+0000 7f9cf03f8740 1 --2- 172.21.15.86:0/3577656216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e75e6343c0 0x55e75e652f30 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.698+0000 7f9cf03f8740 1 --2- 172.21.15.86:0/3577656216 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e75e633fe0 0x55e75e61b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.698+0000 7f9cf03f8740 1 --2- 172.21.15.86:0/3577656216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e75e62efc0 0x55e75e61ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.698+0000 7f9cf03f8740 1 -- 172.21.15.86:0/3577656216 >> 172.21.15.86:0/3577656216 conn(0x55e75e6074c0 msgr2=0x55e75e6347a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:16.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.698+0000 7f9cf03f8740 1 -- 172.21.15.86:0/3577656216 shutdown_connections 2024-04-20T21:28:16.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.698+0000 7f9cf03f8740 1 -- 172.21.15.86:0/3577656216 wait complete. 2024-04-20T21:28:16.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.698+0000 7f9cf03f8740 1 -- start start 2024-04-20T21:28:16.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.699+0000 7f9cf03f8740 1 Processor -- start 2024-04-20T21:28:16.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.699+0000 7f9cf03f8740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e75e62efc0 0x55e75e654ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:16.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.699+0000 7f9cf03f8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e75e633fe0 0x55e75e655020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:16.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.699+0000 7f9cf03f8740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e75e6343c0 0x55e75e655560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:16.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.699+0000 7f9cf03f8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e75e6360c0 con 0x55e75e633fe0 2024-04-20T21:28:16.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.699+0000 7f9cf03f8740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce81f8590 con 0x55e75e6343c0 2024-04-20T21:28:16.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.700+0000 7f9cf03f8740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55e75e636f90 con 0x55e75e62efc0 2024-04-20T21:28:16.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.700+0000 7f9cf03f8740 10 client.0 _ll_register_callbacks cb 0x7f9ce0031440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:28:16.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.700+0000 7f9cde000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e75e633fe0 0x55e75e655020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:16.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.700+0000 7f9cde000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e75e633fe0 0x55e75e655020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43898/0 (socket says 172.21.15.86:43898) 2024-04-20T21:28:16.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.700+0000 7f9cde000640 1 -- 172.21.15.86:0/1047879325 learned_addr learned my addr 172.21.15.86:0/1047879325 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:16.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.700+0000 7f9ce6a00640 1 --2- 172.21.15.86:0/1047879325 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e75e62efc0 0x55e75e654ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:16.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.700+0000 7f9ce7400640 1 --2- 172.21.15.86:0/1047879325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e75e6343c0 0x55e75e655560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:16.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.700+0000 7f9cde000640 1 -- 172.21.15.86:0/1047879325 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e75e62efc0 msgr2=0x55e75e654ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:16.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.700+0000 7f9cde000640 1 --2- 172.21.15.86:0/1047879325 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e75e62efc0 0x55e75e654ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.700+0000 7f9cde000640 1 -- 172.21.15.86:0/1047879325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e75e6343c0 msgr2=0x55e75e655560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:16.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.700+0000 7f9cde000640 1 --2- 172.21.15.86:0/1047879325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e75e6343c0 0x55e75e655560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.700+0000 7f9cde000640 1 -- 172.21.15.86:0/1047879325 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e75e606870 con 0x55e75e633fe0 2024-04-20T21:28:16.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.700+0000 7f9ce6a00640 1 --2- 172.21.15.86:0/1047879325 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e75e62efc0 0x55e75e654ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:28:16.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.701+0000 7f9cde000640 1 --2- 172.21.15.86:0/1047879325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e75e633fe0 0x55e75e655020 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f9ce80d3440 tx=0x7f9cd0003b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0050713141619f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:16.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.701+0000 7f9ccb400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:28:16.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.701+0000 7f9ccb400640 1 -- 172.21.15.86:0/1047879325 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd0015b40 con 0x55e75e633fe0 2024-04-20T21:28:16.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.701+0000 7f9ccb400640 1 -- 172.21.15.86:0/1047879325 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7f9cd0015ce0 con 0x55e75e633fe0 2024-04-20T21:28:16.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.701+0000 7f9cf03f8740 1 -- 172.21.15.86:0/1047879325 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ce0003610 con 0x55e75e633fe0 2024-04-20T21:28:16.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.701+0000 7f9ccb400640 1 -- 172.21.15.86:0/1047879325 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f9cd0007050 con 0x55e75e633fe0 2024-04-20T21:28:16.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.701+0000 7f9cf03f8740 10 client.8865 fetch_fsmap learned FSMap version 82 2024-04-20T21:28:16.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.701+0000 7f9cf03f8740 1 -- 172.21.15.86:0/1047879325 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x55e75e6cce00 con 0x55e75e633fe0 2024-04-20T21:28:16.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.701+0000 7f9ccb400640 1 -- 172.21.15.86:0/1047879325 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ce0003610 con 0x55e75e633fe0 2024-04-20T21:28:16.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.701+0000 7f9ccb400640 1 -- 172.21.15.86:0/1047879325 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 82) ==== 3495+0+0 (secure 0 0 0) 0x7f9cd00071f0 con 0x55e75e633fe0 2024-04-20T21:28:16.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.702+0000 7f9cf03f8740 10 client.8865 fetch_fsmap finished waiting for FSMap version 82 2024-04-20T21:28:16.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.702+0000 7f9cf03f8740 10 client.8865 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-20T21:28:16.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.702+0000 7f9cf03f8740 20 client.8865 populate_metadata read hostname 'smithi086' 2024-04-20T21:28:16.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.702+0000 7f9cf03f8740 1 --2- 172.21.15.86:0/1047879325 >> [v2:172.21.15.86:6834/1331851138,v1:172.21.15.86:6836/1331851138] conn(0x55e75e620dc0 0x55e75e6211a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:16.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.702+0000 7f9cf03f8740 4 client.8865 mds_command: new command op to 8493 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-04-20T21:28:16.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.702+0000 7f9cf03f8740 1 -- 172.21.15.86:0/1047879325 --> [v2:172.21.15.86:6834/1331851138,v1:172.21.15.86:6836/1331851138] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x55e75e5795b0 con 0x55e75e620dc0 2024-04-20T21:28:16.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.702+0000 7f9ce6a00640 1 --2- 172.21.15.86:0/1047879325 >> [v2:172.21.15.86:6834/1331851138,v1:172.21.15.86:6836/1331851138] conn(0x55e75e620dc0 0x55e75e6211a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:16.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.702+0000 7f9ce6a00640 1 --2- 172.21.15.86:0/1047879325 >> [v2:172.21.15.86:6834/1331851138,v1:172.21.15.86:6836/1331851138] conn(0x55e75e620dc0 0x55e75e6211a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7867e5a2f88c8dde server_cookie=650e0cf166b0c090 in_seq=0 out_seq=0 2024-04-20T21:28:16.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.702+0000 7f9ccb400640 10 client.8865 ms_handle_connect on v2:172.21.15.86:6834/1331851138 2024-04-20T21:28:16.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.703+0000 7f9ccb400640 1 -- 172.21.15.86:0/1047879325 <== mds.0 v2:172.21.15.86:6834/1331851138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e75e5795b0 con 0x55e75e620dc0 2024-04-20T21:28:16.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.703+0000 7f9ccb400640 10 client.8865 handle_command_reply: tid=0 2024-04-20T21:28:16.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.703+0000 7f9cf03f8740 1 client.8865 shutdown 2024-04-20T21:28:16.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.703+0000 7f9cf03f8740 1 -- 172.21.15.86:0/1047879325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e75e633fe0 msgr2=0x55e75e655020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:16.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.703+0000 7f9cf03f8740 1 --2- 172.21.15.86:0/1047879325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e75e633fe0 0x55e75e655020 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f9ce80d3440 tx=0x7f9cd0003b70 comp rx=0 tx=0).stop 2024-04-20T21:28:16.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.704+0000 7f9cf03f8740 1 -- 172.21.15.86:0/1047879325 shutdown_connections 2024-04-20T21:28:16.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.704+0000 7f9cf03f8740 1 --2- 172.21.15.86:0/1047879325 >> [v2:172.21.15.86:6834/1331851138,v1:172.21.15.86:6836/1331851138] conn(0x55e75e620dc0 0x55e75e6211a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.704+0000 7f9cf03f8740 1 --2- 172.21.15.86:0/1047879325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e75e6343c0 0x55e75e655560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.704+0000 7f9cf03f8740 1 --2- 172.21.15.86:0/1047879325 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e75e633fe0 0x55e75e655020 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.704+0000 7f9cf03f8740 1 --2- 172.21.15.86:0/1047879325 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e75e62efc0 0x55e75e654ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.704+0000 7f9cf03f8740 1 -- 172.21.15.86:0/1047879325 >> 172.21.15.86:0/1047879325 conn(0x55e75e6074c0 msgr2=0x55e75e63a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:16.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.704+0000 7f9ccb400640 0 client.8865 ms_handle_reset on v2:172.21.15.86:6834/1331851138 2024-04-20T21:28:16.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.704+0000 7f9cf03f8740 1 -- 172.21.15.86:0/1047879325 shutdown_connections 2024-04-20T21:28:16.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.704+0000 7f9cf03f8740 1 -- 172.21.15.86:0/1047879325 wait complete. 2024-04-20T21:28:16.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.704+0000 7f9cf03f8740 20 client.8865 trim_cache size 0 max 16384 2024-04-20T21:28:16.707 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:28:16.707 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-20T21:28:16.707 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "9c70f556-002f-4b54-97dc-75b7fdfad1ca", 2024-04-20T21:28:16.707 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-20T21:28:16.707 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:28:16.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.705+0000 7f9ced600640 1 -- 172.21.15.86:0/1083171950 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9cc406c8f0 msgr2=0x7f9cc406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:16.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.705+0000 7f9ced600640 1 --2- 172.21.15.86:0/1083171950 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9cc406c8f0 0x7f9cc406edb0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f9ce81f2790 tx=0x7f9cd0007990 comp rx=0 tx=0).stop 2024-04-20T21:28:16.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.705+0000 7f9ced600640 1 -- 172.21.15.86:0/1083171950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ce815e840 msgr2=0x7f9ce81584f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:16.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.705+0000 7f9ced600640 1 --2- 172.21.15.86:0/1083171950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ce815e840 0x7f9ce81584f0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f9cd800ce10 tx=0x7f9cd800a660 comp rx=0 tx=0).stop 2024-04-20T21:28:16.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.705+0000 7f9ced600640 1 -- 172.21.15.86:0/1083171950 shutdown_connections 2024-04-20T21:28:16.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.705+0000 7f9ced600640 1 --2- 172.21.15.86:0/1083171950 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9cc406c8f0 0x7f9cc406edb0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.705+0000 7f9ced600640 1 --2- 172.21.15.86:0/1083171950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ce8163080 0x7f9ce8160750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.705+0000 7f9ced600640 1 --2- 172.21.15.86:0/1083171950 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9ce815ec20 0x7f9ce81577d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.705+0000 7f9ced600640 1 --2- 172.21.15.86:0/1083171950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ce815e840 0x7f9ce81584f0 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.705+0000 7f9ced600640 1 -- 172.21.15.86:0/1083171950 >> 172.21.15.86:0/1083171950 conn(0x7f9ce809da40 msgr2=0x7f9ce80c9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:16.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.705+0000 7f9ced600640 1 -- 172.21.15.86:0/1083171950 shutdown_connections 2024-04-20T21:28:16.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.705+0000 7f9ced600640 1 -- 172.21.15.86:0/1083171950 wait complete. 2024-04-20T21:28:16.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:28:16.721 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-20T21:28:16.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.911+0000 7f5b62600640 1 Processor -- start 2024-04-20T21:28:16.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.911+0000 7f5b62600640 1 -- start start 2024-04-20T21:28:16.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.913+0000 7f5b62600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5b5c06bbf0 0x7f5b5c06e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:16.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.913+0000 7f5b62600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5b5c06bfd0 0x7f5b5c145b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:16.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.913+0000 7f5b62600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b5c15cd40 con 0x7f5b5c146040 2024-04-20T21:28:16.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.913+0000 7f5b62600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f5b5c1464b0 con 0x7f5b5c06bbf0 2024-04-20T21:28:16.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.913+0000 7f5b62600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f5b5c15d000 con 0x7f5b5c06bfd0 2024-04-20T21:28:16.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.913+0000 7f5b5be00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f5b5c146040 0x7f5b5c0cd540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:52582/0 (socket says 172.21.15.86:52582) 2024-04-20T21:28:16.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.913+0000 7f5b5be00640 1 -- 172.21.15.86:0/3588043884 learned_addr learned my addr 172.21.15.86:0/3588043884 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:28:16.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.914+0000 7f5b5b400640 1 --2- 172.21.15.86:0/3588043884 >> v2:172.21.15.86:3300/0 conn(0x7f5b5c06bbf0 0x7f5b5c06e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:16.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.914+0000 7f5b53e00640 1 --2- 172.21.15.86:0/3588043884 >> v2:172.21.15.144:3300/0 conn(0x7f5b5c06bfd0 0x7f5b5c145b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:16.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.914+0000 7f5b5aa00640 1 -- 172.21.15.86:0/3588043884 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 708533635 0 0) 0x7f5b5c15cd40 con 0x7f5b5c146040 2024-04-20T21:28:16.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.914+0000 7f5b5aa00640 1 -- 172.21.15.86:0/3588043884 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b40003520 con 0x7f5b5c146040 2024-04-20T21:28:16.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.914+0000 7f5b5b400640 1 -- 172.21.15.86:0/3588043884 >> v2:172.21.15.144:3300/0 conn(0x7f5b5c06bfd0 msgr2=0x7f5b5c145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:16.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.914+0000 7f5b5b400640 1 --2- 172.21.15.86:0/3588043884 >> v2:172.21.15.144:3300/0 conn(0x7f5b5c06bfd0 0x7f5b5c145b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.914+0000 7f5b5b400640 1 -- 172.21.15.86:0/3588043884 >> v1:172.21.15.86:6789/0 conn(0x7f5b5c146040 legacy=0x7f5b5c0cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:16.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.914+0000 7f5b5b400640 1 -- 172.21.15.86:0/3588043884 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b5c15d2e0 con 0x7f5b5c06bbf0 2024-04-20T21:28:16.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.915+0000 7f5b5b400640 1 --2- 172.21.15.86:0/3588043884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b5c06bbf0 0x7f5b5c06e000 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c06dcc0 tx=0x7f5b4c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54c3f55f473629a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:16.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.916+0000 7f5b5aa00640 1 -- 172.21.15.86:0/3588043884 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c035020 con 0x7f5b5c06bbf0 2024-04-20T21:28:16.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.916+0000 7f5b5aa00640 1 -- 172.21.15.86:0/3588043884 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b4c031830 con 0x7f5b5c06bbf0 2024-04-20T21:28:16.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.916+0000 7f5b5aa00640 1 -- 172.21.15.86:0/3588043884 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c031be0 con 0x7f5b5c06bbf0 2024-04-20T21:28:16.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.916+0000 7f5b62600640 1 -- 172.21.15.86:0/3588043884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b5c06bbf0 msgr2=0x7f5b5c06e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:16.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.916+0000 7f5b62600640 1 --2- 172.21.15.86:0/3588043884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b5c06bbf0 0x7f5b5c06e000 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c06dcc0 tx=0x7f5b4c02b8d0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.916+0000 7f5b62600640 1 -- 172.21.15.86:0/3588043884 shutdown_connections 2024-04-20T21:28:16.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.916+0000 7f5b62600640 1 --2- 172.21.15.86:0/3588043884 >> v2:172.21.15.144:3300/0 conn(0x7f5b5c06bfd0 0x7f5b5c145b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.916+0000 7f5b62600640 1 --2- 172.21.15.86:0/3588043884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b5c06bbf0 0x7f5b5c06e000 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.916+0000 7f5b62600640 1 -- 172.21.15.86:0/3588043884 >> 172.21.15.86:0/3588043884 conn(0x7f5b5c0b2110 msgr2=0x7f5b5c0b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:16.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.917+0000 7f5b62600640 1 -- 172.21.15.86:0/3588043884 shutdown_connections 2024-04-20T21:28:16.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.917+0000 7f5b62600640 1 -- 172.21.15.86:0/3588043884 wait complete. 2024-04-20T21:28:16.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.918+0000 7f5b62600640 1 Processor -- start 2024-04-20T21:28:16.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.918+0000 7f5b62600640 1 -- start start 2024-04-20T21:28:16.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.919+0000 7f5b62600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b5c06bbf0 0x7f5b5c1f2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:16.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.919+0000 7f5b62600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b5c06bfd0 0x7f5b5c0c55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:16.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.919+0000 7f5b62600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b5c146040 0x7f5b5c15f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:16.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.919+0000 7f5b62600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5c0cdc50 con 0x7f5b5c146040 2024-04-20T21:28:16.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.919+0000 7f5b62600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5c15d8a0 con 0x7f5b5c06bfd0 2024-04-20T21:28:16.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.919+0000 7f5b62600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5b5c0cddd0 con 0x7f5b5c06bbf0 2024-04-20T21:28:16.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.920+0000 7f5b5be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b5c146040 0x7f5b5c15f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:16.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.920+0000 7f5b5be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b5c146040 0x7f5b5c15f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43924/0 (socket says 172.21.15.86:43924) 2024-04-20T21:28:16.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.920+0000 7f5b5be00640 1 -- 172.21.15.86:0/3764372323 learned_addr learned my addr 172.21.15.86:0/3764372323 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:16.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.920+0000 7f5b5be00640 1 -- 172.21.15.86:0/3764372323 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b5c06bbf0 msgr2=0x7f5b5c1f2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:16.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.920+0000 7f5b5be00640 1 --2- 172.21.15.86:0/3764372323 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b5c06bbf0 0x7f5b5c1f2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.920+0000 7f5b5be00640 1 -- 172.21.15.86:0/3764372323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b5c06bfd0 msgr2=0x7f5b5c0c55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:16.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.920+0000 7f5b5be00640 1 --2- 172.21.15.86:0/3764372323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b5c06bfd0 0x7f5b5c0c55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.920+0000 7f5b5be00640 1 -- 172.21.15.86:0/3764372323 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b5c0c5d40 con 0x7f5b5c146040 2024-04-20T21:28:16.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.920+0000 7f5b5be00640 1 --2- 172.21.15.86:0/3764372323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b5c146040 0x7f5b5c15f2b0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f5b5400cb40 tx=0x7f5b5400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:16.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.921+0000 7f5b53400640 1 -- 172.21.15.86:0/3764372323 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b54017070 con 0x7f5b5c146040 2024-04-20T21:28:16.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.921+0000 7f5b53400640 1 -- 172.21.15.86:0/3764372323 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b5400aca0 con 0x7f5b5c146040 2024-04-20T21:28:16.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.921+0000 7f5b53400640 1 -- 172.21.15.86:0/3764372323 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b54012620 con 0x7f5b5c146040 2024-04-20T21:28:16.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.921+0000 7f5b62600640 1 -- 172.21.15.86:0/3764372323 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b5c0c6020 con 0x7f5b5c146040 2024-04-20T21:28:16.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.921+0000 7f5b62600640 1 -- 172.21.15.86:0/3764372323 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b5c0c65e0 con 0x7f5b5c146040 2024-04-20T21:28:16.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.924+0000 7f5b53400640 1 -- 172.21.15.86:0/3764372323 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5b5401a070 con 0x7f5b5c146040 2024-04-20T21:28:16.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.925+0000 7f5b53400640 1 --2- 172.21.15.86:0/3764372323 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5b3806c8f0 0x7f5b3806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:16.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.925+0000 7f5b53400640 1 -- 172.21.15.86:0/3764372323 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7f5b5408ba30 con 0x7f5b5c146040 2024-04-20T21:28:16.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.925+0000 7f5b5b400640 1 --2- 172.21.15.86:0/3764372323 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5b3806c8f0 0x7f5b3806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:16.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.926+0000 7f5b6539b740 1 Processor -- start 2024-04-20T21:28:16.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.926+0000 7f5b6539b740 1 -- start start 2024-04-20T21:28:16.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.926+0000 7f5b5b400640 1 --2- 172.21.15.86:0/3764372323 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5b3806c8f0 0x7f5b3806edb0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c036000 tx=0x7f5b4c030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:16.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.926+0000 7f5b6539b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8a989af20 0x55e8a987d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:16.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.927+0000 7f5b6539b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8a9898a40 0x55e8a987d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:16.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.927+0000 7f5b6539b740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8a987a630 0x55e8a9850d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:16.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.927+0000 7f5b6539b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e8a97b9150 con 0x55e8a989af20 2024-04-20T21:28:16.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.927+0000 7f5b6539b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e8a975f370 con 0x55e8a9898a40 2024-04-20T21:28:16.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.927+0000 7f5b6539b740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55e8a97b5110 con 0x55e8a987a630 2024-04-20T21:28:16.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.927+0000 7f5b5b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8a989af20 0x55e8a987d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:16.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.927+0000 7f5b5b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8a989af20 0x55e8a987d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43928/0 (socket says 172.21.15.86:43928) 2024-04-20T21:28:16.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.927+0000 7f5b5b400640 1 -- 172.21.15.86:0/740512679 learned_addr learned my addr 172.21.15.86:0/740512679 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:16.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.927+0000 7f5b5b400640 1 -- 172.21.15.86:0/740512679 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8a987a630 msgr2=0x55e8a9850d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:16.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.927+0000 7f5b53e00640 1 --2- 172.21.15.86:0/740512679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8a9898a40 0x55e8a987d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:16.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.927+0000 7f5b5b400640 1 --2- 172.21.15.86:0/740512679 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8a987a630 0x55e8a9850d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.927+0000 7f5b5b400640 1 -- 172.21.15.86:0/740512679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8a9898a40 msgr2=0x55e8a987d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:16.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.927+0000 7f5b5b400640 1 --2- 172.21.15.86:0/740512679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8a9898a40 0x55e8a987d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.927+0000 7f5b5b400640 1 -- 172.21.15.86:0/740512679 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8a95957d0 con 0x55e8a989af20 2024-04-20T21:28:16.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.928+0000 7f5b5b400640 1 --2- 172.21.15.86:0/740512679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8a989af20 0x55e8a987d000 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c000c00 tx=0x7f5b4c031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78d0ba4a3babbd35 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:16.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.928+0000 7f5b4be00640 1 -- 172.21.15.86:0/740512679 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c04a070 con 0x55e8a989af20 2024-04-20T21:28:16.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.928+0000 7f5b4be00640 1 -- 172.21.15.86:0/740512679 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b4c004230 con 0x55e8a989af20 2024-04-20T21:28:16.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.928+0000 7f5b4be00640 1 -- 172.21.15.86:0/740512679 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c040070 con 0x55e8a989af20 2024-04-20T21:28:16.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.929+0000 7f5b6539b740 1 -- 172.21.15.86:0/740512679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8a989af20 msgr2=0x55e8a987d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:16.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.929+0000 7f5b6539b740 1 --2- 172.21.15.86:0/740512679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8a989af20 0x55e8a987d000 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c000c00 tx=0x7f5b4c031000 comp rx=0 tx=0).stop 2024-04-20T21:28:16.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.929+0000 7f5b6539b740 1 -- 172.21.15.86:0/740512679 shutdown_connections 2024-04-20T21:28:16.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.929+0000 7f5b6539b740 1 --2- 172.21.15.86:0/740512679 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8a987a630 0x55e8a9850d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.929+0000 7f5b6539b740 1 --2- 172.21.15.86:0/740512679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8a9898a40 0x55e8a987d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.929+0000 7f5b6539b740 1 --2- 172.21.15.86:0/740512679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8a989af20 0x55e8a987d000 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.929+0000 7f5b6539b740 1 -- 172.21.15.86:0/740512679 >> 172.21.15.86:0/740512679 conn(0x55e8a984f480 msgr2=0x55e8a9898000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:16.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.929+0000 7f5b6539b740 1 -- 172.21.15.86:0/740512679 shutdown_connections 2024-04-20T21:28:16.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.929+0000 7f5b6539b740 1 -- 172.21.15.86:0/740512679 wait complete. 2024-04-20T21:28:16.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.930+0000 7f5b6539b740 1 -- start start 2024-04-20T21:28:16.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.930+0000 7f5b6539b740 1 Processor -- start 2024-04-20T21:28:16.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.931+0000 7f5b6539b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8a9898a40 0x55e8a9873b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:16.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.931+0000 7f5b6539b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8a989af20 0x55e8a9877d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:16.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.931+0000 7f5b6539b740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8a9874200 0x55e8a98a2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:16.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.931+0000 7f5b6539b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e8a984d210 con 0x55e8a9898a40 2024-04-20T21:28:16.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.931+0000 7f5b6539b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e8a9772740 con 0x55e8a989af20 2024-04-20T21:28:16.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.931+0000 7f5b6539b740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55e8a98530a0 con 0x55e8a9874200 2024-04-20T21:28:16.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.931+0000 7f5b6539b740 10 client.0 _ll_register_callbacks cb 0x55e8a976db20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:28:16.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.931+0000 7f5b5b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8a9898a40 0x55e8a9873b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:16.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.931+0000 7f5b5b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8a9898a40 0x55e8a9873b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43944/0 (socket says 172.21.15.86:43944) 2024-04-20T21:28:16.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.931+0000 7f5b5b400640 1 -- 172.21.15.86:0/3567427210 learned_addr learned my addr 172.21.15.86:0/3567427210 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:16.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.931+0000 7f5b5b400640 1 -- 172.21.15.86:0/3567427210 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8a9874200 msgr2=0x55e8a98a2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:16.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.931+0000 7f5b5be00640 1 --2- 172.21.15.86:0/3567427210 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8a9874200 0x55e8a98a2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:16.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.931+0000 7f5b5b400640 1 --2- 172.21.15.86:0/3567427210 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8a9874200 0x55e8a98a2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.932+0000 7f5b5b400640 1 -- 172.21.15.86:0/3567427210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8a989af20 msgr2=0x55e8a9877d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:16.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.932+0000 7f5b5b400640 1 --2- 172.21.15.86:0/3567427210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8a989af20 0x55e8a9877d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.932+0000 7f5b5b400640 1 -- 172.21.15.86:0/3567427210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8a94cb3f0 con 0x55e8a9898a40 2024-04-20T21:28:16.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.932+0000 7f5b5b400640 1 --2- 172.21.15.86:0/3567427210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8a9898a40 0x55e8a9873b70 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c009890 tx=0x7f5b4c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5f93f2bc562e0a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:16.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.932+0000 7f5b48c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:28:16.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.932+0000 7f5b48c00640 1 -- 172.21.15.86:0/3567427210 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c05c060 con 0x55e8a9898a40 2024-04-20T21:28:16.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.932+0000 7f5b48c00640 1 -- 172.21.15.86:0/3567427210 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7f5b4c03adc0 con 0x55e8a9898a40 2024-04-20T21:28:16.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.932+0000 7f5b6539b740 1 -- 172.21.15.86:0/3567427210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b4c004230 con 0x55e8a9898a40 2024-04-20T21:28:16.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.932+0000 7f5b48c00640 1 -- 172.21.15.86:0/3567427210 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f5b4c040050 con 0x55e8a9898a40 2024-04-20T21:28:16.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.932+0000 7f5b6539b740 10 client.8880 fetch_fsmap learned FSMap version 82 2024-04-20T21:28:16.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.932+0000 7f5b6539b740 1 -- 172.21.15.86:0/3567427210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x55e8a948b0f0 con 0x55e8a9898a40 2024-04-20T21:28:16.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.933+0000 7f5b48c00640 1 -- 172.21.15.86:0/3567427210 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b4c004230 con 0x55e8a9898a40 2024-04-20T21:28:16.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.933+0000 7f5b48c00640 1 -- 172.21.15.86:0/3567427210 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 82) ==== 3495+0+0 (secure 0 0 0) 0x7f5b4c065020 con 0x55e8a9898a40 2024-04-20T21:28:16.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.933+0000 7f5b6539b740 10 client.8880 fetch_fsmap finished waiting for FSMap version 82 2024-04-20T21:28:16.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.933+0000 7f5b6539b740 10 client.8880 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-20T21:28:16.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.933+0000 7f5b6539b740 20 client.8880 populate_metadata read hostname 'smithi086' 2024-04-20T21:28:16.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.933+0000 7f5b6539b740 1 --2- 172.21.15.86:0/3567427210 >> [v2:172.21.15.86:6834/1331851138,v1:172.21.15.86:6836/1331851138] conn(0x55e8a98782b0 0x55e8a987fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:16.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.933+0000 7f5b6539b740 4 client.8880 mds_command: new command op to 8493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:28:16.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.933+0000 7f5b6539b740 1 -- 172.21.15.86:0/3567427210 --> [v2:172.21.15.86:6834/1331851138,v1:172.21.15.86:6836/1331851138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8a97c6b70 con 0x55e8a98782b0 2024-04-20T21:28:16.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.933+0000 7f5b53e00640 1 --2- 172.21.15.86:0/3567427210 >> [v2:172.21.15.86:6834/1331851138,v1:172.21.15.86:6836/1331851138] conn(0x55e8a98782b0 0x55e8a987fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:16.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.934+0000 7f5b53e00640 1 --2- 172.21.15.86:0/3567427210 >> [v2:172.21.15.86:6834/1331851138,v1:172.21.15.86:6836/1331851138] conn(0x55e8a98782b0 0x55e8a987fe10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6face347c07e5ae server_cookie=f808a0a41cb62bf6 in_seq=0 out_seq=0 2024-04-20T21:28:16.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.934+0000 7f5b48c00640 10 client.8880 ms_handle_connect on v2:172.21.15.86:6834/1331851138 2024-04-20T21:28:16.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.934+0000 7f5b48c00640 1 -- 172.21.15.86:0/3567427210 <== mds.0 v2:172.21.15.86:6834/1331851138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e8a97c6b70 con 0x55e8a98782b0 2024-04-20T21:28:16.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.934+0000 7f5b48c00640 10 client.8880 handle_command_reply: tid=0 2024-04-20T21:28:16.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.934+0000 7f5b6539b740 1 client.8880 shutdown 2024-04-20T21:28:16.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.935+0000 7f5b6539b740 1 -- 172.21.15.86:0/3567427210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8a9898a40 msgr2=0x55e8a9873b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:16.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.935+0000 7f5b6539b740 1 --2- 172.21.15.86:0/3567427210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8a9898a40 0x55e8a9873b70 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c009890 tx=0x7f5b4c004730 comp rx=0 tx=0).stop 2024-04-20T21:28:16.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.935+0000 7f5b6539b740 1 -- 172.21.15.86:0/3567427210 shutdown_connections 2024-04-20T21:28:16.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.935+0000 7f5b6539b740 1 --2- 172.21.15.86:0/3567427210 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8a9874200 0x55e8a98a2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.935+0000 7f5b6539b740 1 --2- 172.21.15.86:0/3567427210 >> [v2:172.21.15.86:6834/1331851138,v1:172.21.15.86:6836/1331851138] conn(0x55e8a98782b0 0x55e8a987fe10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.935+0000 7f5b6539b740 1 --2- 172.21.15.86:0/3567427210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8a989af20 0x55e8a9877d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.935+0000 7f5b6539b740 1 --2- 172.21.15.86:0/3567427210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8a9898a40 0x55e8a9873b70 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.935+0000 7f5b48c00640 0 client.8880 ms_handle_reset on v2:172.21.15.86:6834/1331851138 2024-04-20T21:28:16.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.935+0000 7f5b6539b740 1 -- 172.21.15.86:0/3567427210 >> 172.21.15.86:0/3567427210 conn(0x55e8a987a630 msgr2=0x55e8a984c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:16.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.935+0000 7f5b6539b740 1 -- 172.21.15.86:0/3567427210 shutdown_connections 2024-04-20T21:28:16.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.935+0000 7f5b6539b740 1 -- 172.21.15.86:0/3567427210 wait complete. 2024-04-20T21:28:16.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.935+0000 7f5b6539b740 20 client.8880 trim_cache size 0 max 16384 2024-04-20T21:28:16.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.954+0000 7f5b6539b740 1 Processor -- start 2024-04-20T21:28:16.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.954+0000 7f5b6539b740 1 -- start start 2024-04-20T21:28:16.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.955+0000 7f5b6539b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8a9874200 0x55e8a989b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:16.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.955+0000 7f5b6539b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8a9898a40 0x55e8a9852240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:16.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.955+0000 7f5b6539b740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8a989af20 0x55e8a987d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:16.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.955+0000 7f5b6539b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5c1fc770 con 0x55e8a9898a40 2024-04-20T21:28:16.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.955+0000 7f5b6539b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5c0b48e0 con 0x55e8a9874200 2024-04-20T21:28:16.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.955+0000 7f5b6539b740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5b5c13d370 con 0x55e8a989af20 2024-04-20T21:28:16.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.955+0000 7f5b53e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8a9898a40 0x55e8a9852240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:16.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.955+0000 7f5b53e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8a9898a40 0x55e8a9852240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43948/0 (socket says 172.21.15.86:43948) 2024-04-20T21:28:16.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.955+0000 7f5b53e00640 1 -- 172.21.15.86:0/279476117 learned_addr learned my addr 172.21.15.86:0/279476117 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:16.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.955+0000 7f5b5b400640 1 --2- 172.21.15.86:0/279476117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8a9874200 0x55e8a989b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:16.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.955+0000 7f5b53e00640 1 -- 172.21.15.86:0/279476117 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8a989af20 msgr2=0x55e8a987d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:16.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.955+0000 7f5b53e00640 1 --2- 172.21.15.86:0/279476117 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8a989af20 0x55e8a987d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.956+0000 7f5b53e00640 1 -- 172.21.15.86:0/279476117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8a9874200 msgr2=0x55e8a989b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:16.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.956+0000 7f5b53e00640 1 --2- 172.21.15.86:0/279476117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8a9874200 0x55e8a989b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.956+0000 7f5b53e00640 1 -- 172.21.15.86:0/279476117 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8a984d9b0 con 0x55e8a9898a40 2024-04-20T21:28:16.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.956+0000 7f5b53e00640 1 --2- 172.21.15.86:0/279476117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8a9898a40 0x55e8a9852240 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c0ccc20 tx=0x7f5b44009d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f97439009728db server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:16.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.956+0000 7f5b4b400640 1 -- 172.21.15.86:0/279476117 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4400f040 con 0x55e8a9898a40 2024-04-20T21:28:16.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.956+0000 7f5b4b400640 1 -- 172.21.15.86:0/279476117 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b4401c030 con 0x55e8a9898a40 2024-04-20T21:28:16.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.956+0000 7f5b4b400640 1 -- 172.21.15.86:0/279476117 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b44015cf0 con 0x55e8a9898a40 2024-04-20T21:28:16.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.957+0000 7f5b6539b740 1 -- 172.21.15.86:0/279476117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8a9898a40 msgr2=0x55e8a9852240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:16.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.957+0000 7f5b6539b740 1 --2- 172.21.15.86:0/279476117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8a9898a40 0x55e8a9852240 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c0ccc20 tx=0x7f5b44009d00 comp rx=0 tx=0).stop 2024-04-20T21:28:16.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.957+0000 7f5b6539b740 1 -- 172.21.15.86:0/279476117 shutdown_connections 2024-04-20T21:28:16.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.957+0000 7f5b6539b740 1 --2- 172.21.15.86:0/279476117 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8a989af20 0x55e8a987d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.957+0000 7f5b6539b740 1 --2- 172.21.15.86:0/279476117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8a9898a40 0x55e8a9852240 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.957+0000 7f5b6539b740 1 --2- 172.21.15.86:0/279476117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8a9874200 0x55e8a989b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.957+0000 7f5b6539b740 1 -- 172.21.15.86:0/279476117 >> 172.21.15.86:0/279476117 conn(0x55e8a987a630 msgr2=0x55e8a9853410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:16.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.957+0000 7f5b6539b740 1 -- 172.21.15.86:0/279476117 shutdown_connections 2024-04-20T21:28:16.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.957+0000 7f5b6539b740 1 -- 172.21.15.86:0/279476117 wait complete. 2024-04-20T21:28:16.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.958+0000 7f5b6539b740 1 -- start start 2024-04-20T21:28:16.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.958+0000 7f5b6539b740 1 Processor -- start 2024-04-20T21:28:16.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.959+0000 7f5b6539b740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8a9874200 0x55e8a9899300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:16.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.959+0000 7f5b6539b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8a9898a40 0x55e8a9897dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:16.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.959+0000 7f5b6539b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8a989af20 0x55e8a9898310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:16.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.959+0000 7f5b6539b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5c15c640 con 0x55e8a9898a40 2024-04-20T21:28:16.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.959+0000 7f5b6539b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55e8a98780e0 con 0x55e8a989af20 2024-04-20T21:28:16.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.959+0000 7f5b6539b740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55e8a989bd40 con 0x55e8a9874200 2024-04-20T21:28:16.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.959+0000 7f5b6539b740 10 client.0 _ll_register_callbacks cb 0x7f5b4c009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:28:16.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.959+0000 7f5b53e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8a9898a40 0x55e8a9897dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:16.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.959+0000 7f5b53e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8a9898a40 0x55e8a9897dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43952/0 (socket says 172.21.15.86:43952) 2024-04-20T21:28:16.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.959+0000 7f5b53e00640 1 -- 172.21.15.86:0/923754575 learned_addr learned my addr 172.21.15.86:0/923754575 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:16.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.960+0000 7f5b53e00640 1 -- 172.21.15.86:0/923754575 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8a9874200 msgr2=0x55e8a9899300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:16.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.960+0000 7f5b53e00640 1 --2- 172.21.15.86:0/923754575 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8a9874200 0x55e8a9899300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.960+0000 7f5b53e00640 1 -- 172.21.15.86:0/923754575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8a989af20 msgr2=0x55e8a9898310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:16.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.960+0000 7f5b53e00640 1 --2- 172.21.15.86:0/923754575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8a989af20 0x55e8a9898310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.960+0000 7f5b53e00640 1 -- 172.21.15.86:0/923754575 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8a987d850 con 0x55e8a9898a40 2024-04-20T21:28:16.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.960+0000 7f5b53e00640 1 --2- 172.21.15.86:0/923754575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8a9898a40 0x55e8a9897dd0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c15e8b0 tx=0x7f5b44008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a448fee5d78bd48b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:16.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.960+0000 7f5b48c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:28:16.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.960+0000 7f5b48c00640 1 -- 172.21.15.86:0/923754575 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b44015990 con 0x55e8a9898a40 2024-04-20T21:28:16.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.960+0000 7f5b48c00640 1 -- 172.21.15.86:0/923754575 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7f5b44015b30 con 0x55e8a9898a40 2024-04-20T21:28:16.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.960+0000 7f5b6539b740 1 -- 172.21.15.86:0/923754575 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b4401c030 con 0x55e8a9898a40 2024-04-20T21:28:16.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.960+0000 7f5b48c00640 1 -- 172.21.15.86:0/923754575 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f5b44033050 con 0x55e8a9898a40 2024-04-20T21:28:16.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.960+0000 7f5b6539b740 10 client.8886 fetch_fsmap learned FSMap version 82 2024-04-20T21:28:16.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.960+0000 7f5b6539b740 1 -- 172.21.15.86:0/923754575 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x55e8a989a1a0 con 0x55e8a9898a40 2024-04-20T21:28:16.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.961+0000 7f5b48c00640 1 -- 172.21.15.86:0/923754575 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b4401c030 con 0x55e8a9898a40 2024-04-20T21:28:16.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.961+0000 7f5b48c00640 1 -- 172.21.15.86:0/923754575 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 82) ==== 3495+0+0 (secure 0 0 0) 0x7f5b44036020 con 0x55e8a9898a40 2024-04-20T21:28:16.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.961+0000 7f5b6539b740 10 client.8886 fetch_fsmap finished waiting for FSMap version 82 2024-04-20T21:28:16.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.961+0000 7f5b6539b740 10 client.8886 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-20T21:28:16.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.961+0000 7f5b6539b740 20 client.8886 populate_metadata read hostname 'smithi086' 2024-04-20T21:28:16.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.961+0000 7f5b6539b740 1 --2- 172.21.15.86:0/923754575 >> [v2:172.21.15.86:6834/1331851138,v1:172.21.15.86:6836/1331851138] conn(0x55e8a987d0e0 0x55e8a9875eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:16.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.961+0000 7f5b6539b740 4 client.8886 mds_command: new command op to 8493 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:28:16.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.961+0000 7f5b6539b740 1 -- 172.21.15.86:0/923754575 --> [v2:172.21.15.86:6834/1331851138,v1:172.21.15.86:6836/1331851138] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e8a97c09e0 con 0x55e8a987d0e0 2024-04-20T21:28:16.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.961+0000 7f5b5b400640 1 --2- 172.21.15.86:0/923754575 >> [v2:172.21.15.86:6834/1331851138,v1:172.21.15.86:6836/1331851138] conn(0x55e8a987d0e0 0x55e8a9875eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:16.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.962+0000 7f5b5b400640 1 --2- 172.21.15.86:0/923754575 >> [v2:172.21.15.86:6834/1331851138,v1:172.21.15.86:6836/1331851138] conn(0x55e8a987d0e0 0x55e8a9875eb0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37fdab2f7ded86f6 server_cookie=29d12cd2a0675a8a in_seq=0 out_seq=0 2024-04-20T21:28:16.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.962+0000 7f5b48c00640 10 client.8886 ms_handle_connect on v2:172.21.15.86:6834/1331851138 2024-04-20T21:28:16.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.962+0000 7f5b48c00640 1 -- 172.21.15.86:0/923754575 <== mds.0 v2:172.21.15.86:6834/1331851138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+260 (crc 0 0 0) 0x55e8a97c09e0 con 0x55e8a987d0e0 2024-04-20T21:28:16.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.962+0000 7f5b48c00640 10 client.8886 handle_command_reply: tid=0 2024-04-20T21:28:16.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.962+0000 7f5b6539b740 1 client.8886 shutdown 2024-04-20T21:28:16.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.962+0000 7f5b6539b740 1 -- 172.21.15.86:0/923754575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8a9898a40 msgr2=0x55e8a9897dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:16.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.962+0000 7f5b6539b740 1 --2- 172.21.15.86:0/923754575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8a9898a40 0x55e8a9897dd0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c15e8b0 tx=0x7f5b44008040 comp rx=0 tx=0).stop 2024-04-20T21:28:16.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.962+0000 7f5b6539b740 1 -- 172.21.15.86:0/923754575 shutdown_connections 2024-04-20T21:28:16.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.962+0000 7f5b6539b740 1 --2- 172.21.15.86:0/923754575 >> [v2:172.21.15.86:6834/1331851138,v1:172.21.15.86:6836/1331851138] conn(0x55e8a987d0e0 0x55e8a9875eb0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.962+0000 7f5b6539b740 1 --2- 172.21.15.86:0/923754575 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55e8a989af20 0x55e8a9898310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.962+0000 7f5b6539b740 1 --2- 172.21.15.86:0/923754575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e8a9898a40 0x55e8a9897dd0 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.962+0000 7f5b6539b740 1 --2- 172.21.15.86:0/923754575 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55e8a9874200 0x55e8a9899300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.962+0000 7f5b48c00640 0 client.8886 ms_handle_reset on v2:172.21.15.86:6834/1331851138 2024-04-20T21:28:16.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.962+0000 7f5b6539b740 1 -- 172.21.15.86:0/923754575 >> 172.21.15.86:0/923754575 conn(0x55e8a987a630 msgr2=0x55e8a987f120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:16.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.962+0000 7f5b6539b740 1 -- 172.21.15.86:0/923754575 shutdown_connections 2024-04-20T21:28:16.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.962+0000 7f5b6539b740 1 -- 172.21.15.86:0/923754575 wait complete. 2024-04-20T21:28:16.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.963+0000 7f5b6539b740 20 client.8886 trim_cache size 0 max 16384 2024-04-20T21:28:16.964 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:28:16.965 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T21:28:16.965 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-20T21:28:16.965 INFO:teuthology.orchestra.run.smithi086.stdout: "9c70f556-002f-4b54-97dc-75b7fdfad1ca": { 2024-04-20T21:28:16.965 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-20T21:28:16.965 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "9c70f556-002f-4b54-97dc-75b7fdfad1ca", 2024-04-20T21:28:16.965 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,repair" 2024-04-20T21:28:16.965 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:28:16.965 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:28:16.965 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:28:16.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.963+0000 7f5b62600640 1 -- 172.21.15.86:0/3764372323 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5b3806c8f0 msgr2=0x7f5b3806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:16.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.963+0000 7f5b62600640 1 --2- 172.21.15.86:0/3764372323 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5b3806c8f0 0x7f5b3806edb0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c036000 tx=0x7f5b4c030000 comp rx=0 tx=0).stop 2024-04-20T21:28:16.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.963+0000 7f5b62600640 1 -- 172.21.15.86:0/3764372323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b5c146040 msgr2=0x7f5b5c15f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:16.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.963+0000 7f5b62600640 1 --2- 172.21.15.86:0/3764372323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b5c146040 0x7f5b5c15f2b0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f5b5400cb40 tx=0x7f5b5400a060 comp rx=0 tx=0).stop 2024-04-20T21:28:16.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.963+0000 7f5b62600640 1 -- 172.21.15.86:0/3764372323 shutdown_connections 2024-04-20T21:28:16.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.963+0000 7f5b62600640 1 --2- 172.21.15.86:0/3764372323 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5b3806c8f0 0x7f5b3806edb0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.963+0000 7f5b62600640 1 --2- 172.21.15.86:0/3764372323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b5c146040 0x7f5b5c15f2b0 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.963+0000 7f5b62600640 1 --2- 172.21.15.86:0/3764372323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b5c06bfd0 0x7f5b5c0c55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.963+0000 7f5b62600640 1 --2- 172.21.15.86:0/3764372323 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5b5c06bbf0 0x7f5b5c1f2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:16.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.963+0000 7f5b62600640 1 -- 172.21.15.86:0/3764372323 >> 172.21.15.86:0/3764372323 conn(0x7f5b5c0b2110 msgr2=0x7f5b5c0b17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:16.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.963+0000 7f5b62600640 1 -- 172.21.15.86:0/3764372323 shutdown_connections 2024-04-20T21:28:16.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:16.963+0000 7f5b62600640 1 -- 172.21.15.86:0/3764372323 wait complete. 2024-04-20T21:28:16.980 INFO:tasks.cephfs.filesystem:scrub status for tag:9c70f556-002f-4b54-97dc-75b7fdfad1ca - {'path': '/', 'tag': '9c70f556-002f-4b54-97dc-75b7fdfad1ca', 'options': 'recursive,repair'} 2024-04-20T21:28:17.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:17.438+0000 7f7e4aa00640 1 -- 172.21.15.86:0/1487035696 <== mon.0 v2:172.21.15.86:3300/0 7 ==== log(4 entries from seq 1 at 2024-04-20T21:28:16.704794+0000) ==== 695+0+0 (secure 0 0 0) 0x7f7e4c057d50 con 0x7f7e5415e890 2024-04-20T21:28:17.437 INFO:teuthology.orchestra.run.smithi086.stdout: cluster: 2024-04-20T21:28:17.437 INFO:teuthology.orchestra.run.smithi086.stdout: id: 9c2dec1d-d7ae-4a83-a812-79922808e278 2024-04-20T21:28:17.437 INFO:teuthology.orchestra.run.smithi086.stdout: health: HEALTH_OK 2024-04-20T21:28:17.437 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:28:17.438 INFO:teuthology.orchestra.run.smithi086.stdout: services: 2024-04-20T21:28:17.438 INFO:teuthology.orchestra.run.smithi086.stdout: mon: 3 daemons, quorum a,b,c (age 10m) 2024-04-20T21:28:17.438 INFO:teuthology.orchestra.run.smithi086.stdout: mgr: y(active, since 10m), standbys: x 2024-04-20T21:28:17.438 INFO:teuthology.orchestra.run.smithi086.stdout: mds: 1/1 daemons up, 3 standby 2024-04-20T21:28:17.438 INFO:teuthology.orchestra.run.smithi086.stdout: osd: 8 osds: 8 up (since 10m), 8 in (since 10m) 2024-04-20T21:28:17.438 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:28:17.438 INFO:teuthology.orchestra.run.smithi086.stdout: data: 2024-04-20T21:28:17.438 INFO:teuthology.orchestra.run.smithi086.stdout: volumes: 1/1 healthy 2024-04-20T21:28:17.438 INFO:teuthology.orchestra.run.smithi086.stdout: pools: 3 pools, 97 pgs 2024-04-20T21:28:17.438 INFO:teuthology.orchestra.run.smithi086.stdout: objects: 30 objects, 19 MiB 2024-04-20T21:28:17.438 INFO:teuthology.orchestra.run.smithi086.stdout: usage: 290 MiB used, 720 GiB / 720 GiB avail 2024-04-20T21:28:17.438 INFO:teuthology.orchestra.run.smithi086.stdout: pgs: 97 active+clean 2024-04-20T21:28:17.438 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:28:17.438 INFO:teuthology.orchestra.run.smithi086.stdout: io: 2024-04-20T21:28:17.439 INFO:teuthology.orchestra.run.smithi086.stdout: client: 85 B/s rd, 1.0 MiB/s wr, 0 op/s rd, 1 op/s wr 2024-04-20T21:28:17.439 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:28:17.439 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:28:17.439 INFO:teuthology.orchestra.run.smithi086.stdout:2024-04-20T21:28:16.704794+0000 mds.c [INF] scrub queued for path: / 2024-04-20T21:28:17.439 INFO:teuthology.orchestra.run.smithi086.stdout:2024-04-20T21:28:16.704797+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-04-20T21:28:17.439 INFO:teuthology.orchestra.run.smithi086.stdout:2024-04-20T21:28:16.704801+0000 mds.c [INF] scrub summary: active paths [/] 2024-04-20T21:28:17.439 INFO:teuthology.orchestra.run.smithi086.stdout:2024-04-20T21:28:16.706826+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-04-20T21:28:17.960 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:28:17.968 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:28:21.466 INFO:teuthology.orchestra.run.smithi086.stdout:2024-04-20T21:28:20.787062+0000 mds.c [INF] scrub summary: idle 2024-04-20T21:28:21.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:21.466+0000 7f7e4aa00640 1 -- 172.21.15.86:0/1487035696 <== mon.0 v2:172.21.15.86:3300/0 8 ==== log(1 entries from seq 5 at 2024-04-20T21:28:20.787062+0000) ==== 189+0+0 (secure 0 0 0) 0x7f7e4c057fb0 con 0x7f7e5415e890 2024-04-20T21:28:46.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:28:46.982 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-20T21:28:47.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.195+0000 7f46f4000640 1 Processor -- start 2024-04-20T21:28:47.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.195+0000 7f46f4000640 1 -- start start 2024-04-20T21:28:47.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.197+0000 7f46f4000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f46ec0b4530 0x7f46ec169550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:47.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.197+0000 7f46f4000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46ec0adb10 con 0x7f46ec15eb30 2024-04-20T21:28:47.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.197+0000 7f46f4000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46ec0addb0 con 0x7f46ec0aee50 2024-04-20T21:28:47.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.197+0000 7f46f4000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f46ec0af7d0 con 0x7f46ec0b4530 2024-04-20T21:28:47.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.197+0000 7f46f1600640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f46ec15eb30 0x7f46ec0b3e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:45866/0 (socket says 172.21.15.86:45866) 2024-04-20T21:28:47.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.197+0000 7f46f1600640 1 -- 172.21.15.86:0/3634974301 learned_addr learned my addr 172.21.15.86:0/3634974301 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:28:47.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.197+0000 7f46f2000640 1 --2- 172.21.15.86:0/3634974301 >> v2:172.21.15.86:3300/0 conn(0x7f46ec0b4530 0x7f46ec169550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:47.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.198+0000 7f46f2000640 1 -- 172.21.15.86:0/3634974301 >> v1:172.21.15.144:6789/0 conn(0x7f46ec0aee50 legacy=0x7f46ec06b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:47.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.198+0000 7f46ebe00640 1 -- 172.21.15.86:0/3634974301 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2638850935 0 0) 0x7f46ec0adb10 con 0x7f46ec15eb30 2024-04-20T21:28:47.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.198+0000 7f46f2000640 1 -- 172.21.15.86:0/3634974301 >> v1:172.21.15.86:6789/0 conn(0x7f46ec15eb30 legacy=0x7f46ec0b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:47.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.198+0000 7f46f2000640 1 -- 172.21.15.86:0/3634974301 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46ec0ae090 con 0x7f46ec0b4530 2024-04-20T21:28:47.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.199+0000 7f46f2000640 1 --2- 172.21.15.86:0/3634974301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46ec0b4530 0x7f46ec169550 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f46e4009d90 tx=0x7f46e402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a3d3386c7100b20 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:47.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.199+0000 7f46ebe00640 1 -- 172.21.15.86:0/3634974301 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e4039070 con 0x7f46ec0b4530 2024-04-20T21:28:47.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.199+0000 7f46ebe00640 1 -- 172.21.15.86:0/3634974301 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46e402cc60 con 0x7f46ec0b4530 2024-04-20T21:28:47.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.199+0000 7f46ebe00640 1 -- 172.21.15.86:0/3634974301 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e4034660 con 0x7f46ec0b4530 2024-04-20T21:28:47.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.200+0000 7f46f4000640 1 -- 172.21.15.86:0/3634974301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46ec0b4530 msgr2=0x7f46ec169550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:47.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.200+0000 7f46f4000640 1 --2- 172.21.15.86:0/3634974301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46ec0b4530 0x7f46ec169550 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f46e4009d90 tx=0x7f46e402beb0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.200+0000 7f46f4000640 1 -- 172.21.15.86:0/3634974301 shutdown_connections 2024-04-20T21:28:47.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.200+0000 7f46f4000640 1 --2- 172.21.15.86:0/3634974301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46ec0b4530 0x7f46ec169550 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.200+0000 7f46f4000640 1 -- 172.21.15.86:0/3634974301 >> 172.21.15.86:0/3634974301 conn(0x7f46ec0b2110 msgr2=0x7f46ec0b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:47.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.200+0000 7f46f4000640 1 -- 172.21.15.86:0/3634974301 shutdown_connections 2024-04-20T21:28:47.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.200+0000 7f46f4000640 1 -- 172.21.15.86:0/3634974301 wait complete. 2024-04-20T21:28:47.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.202+0000 7f46f4000640 1 Processor -- start 2024-04-20T21:28:47.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.202+0000 7f46f4000640 1 -- start start 2024-04-20T21:28:47.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.203+0000 7f46f4000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46ec0aee50 0x7f46ec150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:47.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.203+0000 7f46f4000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46ec0b4530 0x7f46ec1e9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:47.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.203+0000 7f46f4000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46ec15eb30 0x7f46ec1f89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:47.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.203+0000 7f46f4000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f46ec0ae930 con 0x7f46ec0b4530 2024-04-20T21:28:47.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.203+0000 7f46f4000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f46ec0ae650 con 0x7f46ec0aee50 2024-04-20T21:28:47.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.203+0000 7f46f4000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f46ec06c090 con 0x7f46ec15eb30 2024-04-20T21:28:47.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.203+0000 7f46f0c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46ec0b4530 0x7f46ec1e9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:47.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.203+0000 7f46f0c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46ec0b4530 0x7f46ec1e9970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45662/0 (socket says 172.21.15.86:45662) 2024-04-20T21:28:47.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.203+0000 7f46f0c00640 1 -- 172.21.15.86:0/313238337 learned_addr learned my addr 172.21.15.86:0/313238337 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:47.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.204+0000 7f46f1600640 1 --2- 172.21.15.86:0/313238337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46ec0aee50 0x7f46ec150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:47.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.204+0000 7f46f2000640 1 --2- 172.21.15.86:0/313238337 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46ec15eb30 0x7f46ec1f89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:47.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.204+0000 7f46f0c00640 1 -- 172.21.15.86:0/313238337 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46ec15eb30 msgr2=0x7f46ec1f89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:47.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.204+0000 7f46f0c00640 1 --2- 172.21.15.86:0/313238337 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46ec15eb30 0x7f46ec1f89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.204+0000 7f46f0c00640 1 -- 172.21.15.86:0/313238337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46ec0aee50 msgr2=0x7f46ec150ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:47.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.204+0000 7f46f0c00640 1 --2- 172.21.15.86:0/313238337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46ec0aee50 0x7f46ec150ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.204+0000 7f46f0c00640 1 -- 172.21.15.86:0/313238337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46ec1f8f20 con 0x7f46ec0b4530 2024-04-20T21:28:47.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.204+0000 7f46f0c00640 1 --2- 172.21.15.86:0/313238337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46ec0b4530 0x7f46ec1e9970 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f46d800c430 tx=0x7f46d800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:47.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.204+0000 7f46e9600640 1 -- 172.21.15.86:0/313238337 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d8011790 con 0x7f46ec0b4530 2024-04-20T21:28:47.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.205+0000 7f46e9600640 1 -- 172.21.15.86:0/313238337 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46d8017070 con 0x7f46ec0b4530 2024-04-20T21:28:47.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.205+0000 7f46e9600640 1 -- 172.21.15.86:0/313238337 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d8011b70 con 0x7f46ec0b4530 2024-04-20T21:28:47.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.205+0000 7f46f4000640 1 -- 172.21.15.86:0/313238337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46ec0ccac0 con 0x7f46ec0b4530 2024-04-20T21:28:47.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.205+0000 7f46f4000640 1 -- 172.21.15.86:0/313238337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46ec0cd080 con 0x7f46ec0b4530 2024-04-20T21:28:47.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.207+0000 7f46e9600640 1 -- 172.21.15.86:0/313238337 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f46d8011d10 con 0x7f46ec0b4530 2024-04-20T21:28:47.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.208+0000 7f46e9600640 1 --2- 172.21.15.86:0/313238337 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f46cc06c8f0 0x7f46cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:47.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.208+0000 7f46e9600640 1 -- 172.21.15.86:0/313238337 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7f46d808ba60 con 0x7f46ec0b4530 2024-04-20T21:28:47.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.208+0000 7f46f1600640 1 --2- 172.21.15.86:0/313238337 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f46cc06c8f0 0x7f46cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:47.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.208+0000 7f46f6dbd740 1 Processor -- start 2024-04-20T21:28:47.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.209+0000 7f46f6dbd740 1 -- start start 2024-04-20T21:28:47.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.209+0000 7f46f1600640 1 --2- 172.21.15.86:0/313238337 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f46cc06c8f0 0x7f46cc06edb0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f46e0000f60 tx=0x7f46e0007c00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:47.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.210+0000 7f46f6dbd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5643679e21f0 0x5643679e25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:47.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.210+0000 7f46f6dbd740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5643679b2210 0x5643679df460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:47.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.210+0000 7f46f6dbd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643679b5d70 0x5643679df9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:47.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.210+0000 7f46f6dbd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56436791e150 con 0x5643679b5d70 2024-04-20T21:28:47.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.210+0000 7f46f6dbd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5643678c4370 con 0x5643679e21f0 2024-04-20T21:28:47.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.210+0000 7f46f6dbd740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x56436791a110 con 0x5643679b2210 2024-04-20T21:28:47.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.210+0000 7f46f2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643679b5d70 0x5643679df9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:47.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.210+0000 7f46f2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643679b5d70 0x5643679df9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45672/0 (socket says 172.21.15.86:45672) 2024-04-20T21:28:47.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.210+0000 7f46f2000640 1 -- 172.21.15.86:0/1265778437 learned_addr learned my addr 172.21.15.86:0/1265778437 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:47.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.210+0000 7f46f2000640 1 -- 172.21.15.86:0/1265778437 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5643679b2210 msgr2=0x5643679df460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:47.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.210+0000 7f46f2000640 1 --2- 172.21.15.86:0/1265778437 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5643679b2210 0x5643679df460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.210+0000 7f46f2000640 1 -- 172.21.15.86:0/1265778437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5643679e21f0 msgr2=0x5643679e25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:47.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.210+0000 7f46f2000640 1 --2- 172.21.15.86:0/1265778437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5643679e21f0 0x5643679e25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.210+0000 7f46f2000640 1 -- 172.21.15.86:0/1265778437 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643676fa7d0 con 0x5643679b5d70 2024-04-20T21:28:47.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.211+0000 7f46f2000640 1 --2- 172.21.15.86:0/1265778437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643679b5d70 0x5643679df9a0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f46e4009d90 tx=0x7f46e402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=760963d56ccedd89 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:47.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.211+0000 7f46ddc00640 1 -- 172.21.15.86:0/1265778437 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e4039040 con 0x5643679b5d70 2024-04-20T21:28:47.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.211+0000 7f46ddc00640 1 -- 172.21.15.86:0/1265778437 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46e403f450 con 0x5643679b5d70 2024-04-20T21:28:47.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.211+0000 7f46ddc00640 1 -- 172.21.15.86:0/1265778437 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e403f860 con 0x5643679b5d70 2024-04-20T21:28:47.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.212+0000 7f46f6dbd740 1 -- 172.21.15.86:0/1265778437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643679b5d70 msgr2=0x5643679df9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:47.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.212+0000 7f46f6dbd740 1 --2- 172.21.15.86:0/1265778437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643679b5d70 0x5643679df9a0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f46e4009d90 tx=0x7f46e402e000 comp rx=0 tx=0).stop 2024-04-20T21:28:47.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.212+0000 7f46f6dbd740 1 -- 172.21.15.86:0/1265778437 shutdown_connections 2024-04-20T21:28:47.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.212+0000 7f46f6dbd740 1 --2- 172.21.15.86:0/1265778437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643679b5d70 0x5643679df9a0 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.212+0000 7f46f6dbd740 1 --2- 172.21.15.86:0/1265778437 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5643679b2210 0x5643679df460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.212+0000 7f46f6dbd740 1 --2- 172.21.15.86:0/1265778437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5643679e21f0 0x5643679e25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.212+0000 7f46f6dbd740 1 -- 172.21.15.86:0/1265778437 >> 172.21.15.86:0/1265778437 conn(0x5643679b4480 msgr2=0x5643679fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:47.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.212+0000 7f46f6dbd740 1 -- 172.21.15.86:0/1265778437 shutdown_connections 2024-04-20T21:28:47.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.212+0000 7f46f6dbd740 1 -- 172.21.15.86:0/1265778437 wait complete. 2024-04-20T21:28:47.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.212+0000 7f46f6dbd740 1 -- start start 2024-04-20T21:28:47.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.213+0000 7f46f6dbd740 1 Processor -- start 2024-04-20T21:28:47.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.214+0000 7f46f6dbd740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5643679b5d70 0x5643679e6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:47.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.214+0000 7f46f6dbd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5643679e21f0 0x5643679cad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:47.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.214+0000 7f46f6dbd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643679daa00 0x5643679e3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:47.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.214+0000 7f46f6dbd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5643679b6f20 con 0x5643679daa00 2024-04-20T21:28:47.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.214+0000 7f46f6dbd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5643678d7740 con 0x5643679e21f0 2024-04-20T21:28:47.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.214+0000 7f46f6dbd740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5643679b6310 con 0x5643679b5d70 2024-04-20T21:28:47.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.214+0000 7f46f6dbd740 10 client.0 _ll_register_callbacks cb 0x5643678d2b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:28:47.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.214+0000 7f46f2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643679daa00 0x5643679e3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:47.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.214+0000 7f46f2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643679daa00 0x5643679e3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45678/0 (socket says 172.21.15.86:45678) 2024-04-20T21:28:47.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.214+0000 7f46f2000640 1 -- 172.21.15.86:0/2541071532 learned_addr learned my addr 172.21.15.86:0/2541071532 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:47.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.214+0000 7f46f1600640 1 --2- 172.21.15.86:0/2541071532 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5643679b5d70 0x5643679e6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:47.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.214+0000 7f46f0c00640 1 --2- 172.21.15.86:0/2541071532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5643679e21f0 0x5643679cad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:47.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.214+0000 7f46f2000640 1 -- 172.21.15.86:0/2541071532 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5643679b5d70 msgr2=0x5643679e6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:47.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.214+0000 7f46f2000640 1 --2- 172.21.15.86:0/2541071532 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5643679b5d70 0x5643679e6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.214+0000 7f46f2000640 1 -- 172.21.15.86:0/2541071532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5643679e21f0 msgr2=0x5643679cad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:47.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.214+0000 7f46f2000640 1 --2- 172.21.15.86:0/2541071532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5643679e21f0 0x5643679cad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.214+0000 7f46f2000640 1 -- 172.21.15.86:0/2541071532 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643676303f0 con 0x5643679daa00 2024-04-20T21:28:47.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.215+0000 7f46f2000640 1 --2- 172.21.15.86:0/2541071532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643679daa00 0x5643679e3330 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f46e4009d90 tx=0x7f46e4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=874450c40f15faf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:47.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.215+0000 7f46d2000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:28:47.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.215+0000 7f46d2000640 1 -- 172.21.15.86:0/2541071532 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e404f020 con 0x5643679daa00 2024-04-20T21:28:47.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.215+0000 7f46d2000640 1 -- 172.21.15.86:0/2541071532 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7f46e404b650 con 0x5643679daa00 2024-04-20T21:28:47.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.215+0000 7f46f6dbd740 1 -- 172.21.15.86:0/2541071532 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46e403f450 con 0x5643679daa00 2024-04-20T21:28:47.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.215+0000 7f46d2000640 1 -- 172.21.15.86:0/2541071532 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f46e4059050 con 0x5643679daa00 2024-04-20T21:28:47.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.215+0000 7f46f6dbd740 10 client.8901 fetch_fsmap learned FSMap version 82 2024-04-20T21:28:47.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.215+0000 7f46f6dbd740 1 -- 172.21.15.86:0/2541071532 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x5643675f00f0 con 0x5643679daa00 2024-04-20T21:28:47.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.216+0000 7f46d2000640 1 -- 172.21.15.86:0/2541071532 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46e403f450 con 0x5643679daa00 2024-04-20T21:28:47.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.216+0000 7f46d2000640 1 -- 172.21.15.86:0/2541071532 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 82) ==== 3495+0+0 (secure 0 0 0) 0x7f46e405c020 con 0x5643679daa00 2024-04-20T21:28:47.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.216+0000 7f46f6dbd740 10 client.8901 fetch_fsmap finished waiting for FSMap version 82 2024-04-20T21:28:47.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.216+0000 7f46f6dbd740 10 client.8901 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-20T21:28:47.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.216+0000 7f46f6dbd740 20 client.8901 populate_metadata read hostname 'smithi086' 2024-04-20T21:28:47.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.216+0000 7f46f6dbd740 1 --2- 172.21.15.86:0/2541071532 >> [v2:172.21.15.86:6834/1331851138,v1:172.21.15.86:6836/1331851138] conn(0x5643679dd180 0x5643679d8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:47.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.216+0000 7f46f6dbd740 4 client.8901 mds_command: new command op to 8493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:28:47.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.216+0000 7f46f6dbd740 1 -- 172.21.15.86:0/2541071532 --> [v2:172.21.15.86:6834/1331851138,v1:172.21.15.86:6836/1331851138] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56436792bb70 con 0x5643679dd180 2024-04-20T21:28:47.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.216+0000 7f46f1600640 1 --2- 172.21.15.86:0/2541071532 >> [v2:172.21.15.86:6834/1331851138,v1:172.21.15.86:6836/1331851138] conn(0x5643679dd180 0x5643679d8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:47.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.217+0000 7f46f1600640 1 --2- 172.21.15.86:0/2541071532 >> [v2:172.21.15.86:6834/1331851138,v1:172.21.15.86:6836/1331851138] conn(0x5643679dd180 0x5643679d8180 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d95902d160355a7 server_cookie=9c06d1b9174cf7b6 in_seq=0 out_seq=0 2024-04-20T21:28:47.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.217+0000 7f46d2000640 10 client.8901 ms_handle_connect on v2:172.21.15.86:6834/1331851138 2024-04-20T21:28:47.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.217+0000 7f46d2000640 1 -- 172.21.15.86:0/2541071532 <== mds.0 v2:172.21.15.86:6834/1331851138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56436792bb70 con 0x5643679dd180 2024-04-20T21:28:47.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.217+0000 7f46d2000640 10 client.8901 handle_command_reply: tid=0 2024-04-20T21:28:47.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.217+0000 7f46f6dbd740 1 client.8901 shutdown 2024-04-20T21:28:47.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.218+0000 7f46f6dbd740 1 -- 172.21.15.86:0/2541071532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643679daa00 msgr2=0x5643679e3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:47.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.218+0000 7f46f6dbd740 1 --2- 172.21.15.86:0/2541071532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643679daa00 0x5643679e3330 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f46e4009d90 tx=0x7f46e4004ab0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.218+0000 7f46f6dbd740 1 -- 172.21.15.86:0/2541071532 shutdown_connections 2024-04-20T21:28:47.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.218+0000 7f46f6dbd740 1 --2- 172.21.15.86:0/2541071532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643679daa00 0x5643679e3330 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.218+0000 7f46f6dbd740 1 --2- 172.21.15.86:0/2541071532 >> [v2:172.21.15.86:6834/1331851138,v1:172.21.15.86:6836/1331851138] conn(0x5643679dd180 0x5643679d8180 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.218+0000 7f46f6dbd740 1 --2- 172.21.15.86:0/2541071532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5643679e21f0 0x5643679cad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.218+0000 7f46f6dbd740 1 --2- 172.21.15.86:0/2541071532 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5643679b5d70 0x5643679e6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.218+0000 7f46d2000640 0 client.8901 ms_handle_reset on v2:172.21.15.86:6834/1331851138 2024-04-20T21:28:47.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.218+0000 7f46f6dbd740 1 -- 172.21.15.86:0/2541071532 >> 172.21.15.86:0/2541071532 conn(0x5643679b2210 msgr2=0x5643679fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:47.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.218+0000 7f46f6dbd740 1 -- 172.21.15.86:0/2541071532 shutdown_connections 2024-04-20T21:28:47.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.218+0000 7f46f6dbd740 1 -- 172.21.15.86:0/2541071532 wait complete. 2024-04-20T21:28:47.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.218+0000 7f46f6dbd740 20 client.8901 trim_cache size 0 max 16384 2024-04-20T21:28:47.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.237+0000 7f46f6dbd740 1 Processor -- start 2024-04-20T21:28:47.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.237+0000 7f46f6dbd740 1 -- start start 2024-04-20T21:28:47.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.238+0000 7f46f6dbd740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5643679b5d70 0x5643679fe290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:47.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.238+0000 7f46f6dbd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5643679daa00 0x5643679fe7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:47.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.238+0000 7f46f6dbd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643679e21f0 0x5643679defb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:47.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.238+0000 7f46f6dbd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f46ec200c00 con 0x5643679e21f0 2024-04-20T21:28:47.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.238+0000 7f46f6dbd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f46ec1f5310 con 0x5643679daa00 2024-04-20T21:28:47.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.238+0000 7f46f6dbd740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f46ec1ea890 con 0x5643679b5d70 2024-04-20T21:28:47.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.239+0000 7f46f2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643679e21f0 0x5643679defb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:47.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.239+0000 7f46f2000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643679e21f0 0x5643679defb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45694/0 (socket says 172.21.15.86:45694) 2024-04-20T21:28:47.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.239+0000 7f46f2000640 1 -- 172.21.15.86:0/694510937 learned_addr learned my addr 172.21.15.86:0/694510937 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:47.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.239+0000 7f46f2000640 1 -- 172.21.15.86:0/694510937 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5643679b5d70 msgr2=0x5643679fe290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:47.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.239+0000 7f46f2000640 1 --2- 172.21.15.86:0/694510937 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5643679b5d70 0x5643679fe290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.239+0000 7f46f2000640 1 -- 172.21.15.86:0/694510937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5643679daa00 msgr2=0x5643679fe7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:47.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.239+0000 7f46f2000640 1 --2- 172.21.15.86:0/694510937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5643679daa00 0x5643679fe7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.239+0000 7f46f2000640 1 -- 172.21.15.86:0/694510937 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643679e2940 con 0x5643679e21f0 2024-04-20T21:28:47.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.239+0000 7f46f2000640 1 --2- 172.21.15.86:0/694510937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643679e21f0 0x5643679defb0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f46e403f9c0 tx=0x7f46e4043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e742eede19a0dbf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:47.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.240+0000 7f46dd200640 1 -- 172.21.15.86:0/694510937 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e404f030 con 0x5643679e21f0 2024-04-20T21:28:47.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.240+0000 7f46dd200640 1 -- 172.21.15.86:0/694510937 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46e404b650 con 0x5643679e21f0 2024-04-20T21:28:47.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.240+0000 7f46dd200640 1 -- 172.21.15.86:0/694510937 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e4039050 con 0x5643679e21f0 2024-04-20T21:28:47.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.240+0000 7f46f6dbd740 1 -- 172.21.15.86:0/694510937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643679e21f0 msgr2=0x5643679defb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:47.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.240+0000 7f46f6dbd740 1 --2- 172.21.15.86:0/694510937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643679e21f0 0x5643679defb0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f46e403f9c0 tx=0x7f46e4043000 comp rx=0 tx=0).stop 2024-04-20T21:28:47.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.240+0000 7f46f6dbd740 1 -- 172.21.15.86:0/694510937 shutdown_connections 2024-04-20T21:28:47.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.240+0000 7f46f6dbd740 1 --2- 172.21.15.86:0/694510937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643679e21f0 0x5643679defb0 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.240+0000 7f46f6dbd740 1 --2- 172.21.15.86:0/694510937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5643679daa00 0x5643679fe7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.240+0000 7f46f6dbd740 1 --2- 172.21.15.86:0/694510937 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5643679b5d70 0x5643679fe290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.240+0000 7f46f6dbd740 1 -- 172.21.15.86:0/694510937 >> 172.21.15.86:0/694510937 conn(0x5643679b2210 msgr2=0x5643679b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:47.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.240+0000 7f46f6dbd740 1 -- 172.21.15.86:0/694510937 shutdown_connections 2024-04-20T21:28:47.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.241+0000 7f46f6dbd740 1 -- 172.21.15.86:0/694510937 wait complete. 2024-04-20T21:28:47.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.241+0000 7f46f6dbd740 1 -- start start 2024-04-20T21:28:47.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.241+0000 7f46f6dbd740 1 Processor -- start 2024-04-20T21:28:47.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.242+0000 7f46f6dbd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643679b5d70 0x564367a20e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:47.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.242+0000 7f46f6dbd740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5643679daa00 0x564367a21370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:47.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.242+0000 7f46f6dbd740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5643679e21f0 0x564367a0fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:47.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.242+0000 7f46f6dbd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5643679ca320 con 0x5643679b5d70 2024-04-20T21:28:47.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.242+0000 7f46f6dbd740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f46ec1f4c80 con 0x5643679e21f0 2024-04-20T21:28:47.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.242+0000 7f46f6dbd740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5643679b7060 con 0x5643679daa00 2024-04-20T21:28:47.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.242+0000 7f46f6dbd740 10 client.0 _ll_register_callbacks cb 0x7f46e4009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:28:47.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.243+0000 7f46f1600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643679b5d70 0x564367a20e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:47.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.243+0000 7f46f1600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643679b5d70 0x564367a20e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45696/0 (socket says 172.21.15.86:45696) 2024-04-20T21:28:47.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.243+0000 7f46f1600640 1 -- 172.21.15.86:0/1640261336 learned_addr learned my addr 172.21.15.86:0/1640261336 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:47.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.243+0000 7f46f1600640 1 -- 172.21.15.86:0/1640261336 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5643679daa00 msgr2=0x564367a21370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:47.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.243+0000 7f46f1600640 1 --2- 172.21.15.86:0/1640261336 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5643679daa00 0x564367a21370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.243+0000 7f46f1600640 1 -- 172.21.15.86:0/1640261336 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5643679e21f0 msgr2=0x564367a0fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:47.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.243+0000 7f46f1600640 1 --2- 172.21.15.86:0/1640261336 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5643679e21f0 0x564367a0fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.243+0000 7f46f1600640 1 -- 172.21.15.86:0/1640261336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643679b6e50 con 0x5643679b5d70 2024-04-20T21:28:47.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.243+0000 7f46f1600640 1 --2- 172.21.15.86:0/1640261336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643679b5d70 0x564367a20e30 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x5643679dd750 tx=0x7f46e0001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=692e1d0a803278ab server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:47.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.243+0000 7f46d2000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:28:47.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.244+0000 7f46d2000640 1 -- 172.21.15.86:0/1640261336 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e0027020 con 0x5643679b5d70 2024-04-20T21:28:47.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.244+0000 7f46d2000640 1 -- 172.21.15.86:0/1640261336 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7f46e0023710 con 0x5643679b5d70 2024-04-20T21:28:47.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.244+0000 7f46f6dbd740 1 -- 172.21.15.86:0/1640261336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46e404b650 con 0x5643679b5d70 2024-04-20T21:28:47.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.244+0000 7f46d2000640 1 -- 172.21.15.86:0/1640261336 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f46e0031050 con 0x5643679b5d70 2024-04-20T21:28:47.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.244+0000 7f46f6dbd740 10 client.8907 fetch_fsmap learned FSMap version 82 2024-04-20T21:28:47.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.244+0000 7f46f6dbd740 1 -- 172.21.15.86:0/1640261336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x564367a77e30 con 0x5643679b5d70 2024-04-20T21:28:47.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.244+0000 7f46d2000640 1 -- 172.21.15.86:0/1640261336 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46e404b650 con 0x5643679b5d70 2024-04-20T21:28:47.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.244+0000 7f46d2000640 1 -- 172.21.15.86:0/1640261336 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 82) ==== 3495+0+0 (secure 0 0 0) 0x7f46e0034020 con 0x5643679b5d70 2024-04-20T21:28:47.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.244+0000 7f46f6dbd740 10 client.8907 fetch_fsmap finished waiting for FSMap version 82 2024-04-20T21:28:47.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.245+0000 7f46f6dbd740 10 client.8907 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-20T21:28:47.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.245+0000 7f46f6dbd740 20 client.8907 populate_metadata read hostname 'smithi086' 2024-04-20T21:28:47.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.245+0000 7f46f6dbd740 1 --2- 172.21.15.86:0/1640261336 >> [v2:172.21.15.86:6834/1331851138,v1:172.21.15.86:6836/1331851138] conn(0x5643679e44d0 0x564367a10290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:47.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.245+0000 7f46f6dbd740 4 client.8907 mds_command: new command op to 8493 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:28:47.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.245+0000 7f46f6dbd740 1 -- 172.21.15.86:0/1640261336 --> [v2:172.21.15.86:6834/1331851138,v1:172.21.15.86:6836/1331851138] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5643679259e0 con 0x5643679e44d0 2024-04-20T21:28:47.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.245+0000 7f46f0c00640 1 --2- 172.21.15.86:0/1640261336 >> [v2:172.21.15.86:6834/1331851138,v1:172.21.15.86:6836/1331851138] conn(0x5643679e44d0 0x564367a10290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:47.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.245+0000 7f46d2000640 10 client.8907 ms_handle_connect on v2:172.21.15.86:6834/1331851138 2024-04-20T21:28:47.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.245+0000 7f46f0c00640 1 --2- 172.21.15.86:0/1640261336 >> [v2:172.21.15.86:6834/1331851138,v1:172.21.15.86:6836/1331851138] conn(0x5643679e44d0 0x564367a10290 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7812b562483b9962 server_cookie=dab2c6ee337cfd43 in_seq=0 out_seq=0 2024-04-20T21:28:47.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.246+0000 7f46d2000640 1 -- 172.21.15.86:0/1640261336 <== mds.0 v2:172.21.15.86:6834/1331851138 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5643679259e0 con 0x5643679e44d0 2024-04-20T21:28:47.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.246+0000 7f46d2000640 10 client.8907 handle_command_reply: tid=0 2024-04-20T21:28:47.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.246+0000 7f46f6dbd740 1 client.8907 shutdown 2024-04-20T21:28:47.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.246+0000 7f46f6dbd740 1 -- 172.21.15.86:0/1640261336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643679b5d70 msgr2=0x564367a20e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:47.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.246+0000 7f46f6dbd740 1 --2- 172.21.15.86:0/1640261336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643679b5d70 0x564367a20e30 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x5643679dd750 tx=0x7f46e0001080 comp rx=0 tx=0).stop 2024-04-20T21:28:47.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.246+0000 7f46f6dbd740 1 -- 172.21.15.86:0/1640261336 shutdown_connections 2024-04-20T21:28:47.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.246+0000 7f46f6dbd740 1 --2- 172.21.15.86:0/1640261336 >> [v2:172.21.15.86:6834/1331851138,v1:172.21.15.86:6836/1331851138] conn(0x5643679e44d0 0x564367a10290 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.246+0000 7f46f6dbd740 1 --2- 172.21.15.86:0/1640261336 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5643679e21f0 0x564367a0fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.246+0000 7f46f6dbd740 1 --2- 172.21.15.86:0/1640261336 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5643679daa00 0x564367a21370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.246+0000 7f46f6dbd740 1 --2- 172.21.15.86:0/1640261336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643679b5d70 0x564367a20e30 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.246+0000 7f46f6dbd740 1 -- 172.21.15.86:0/1640261336 >> 172.21.15.86:0/1640261336 conn(0x5643679b2210 msgr2=0x5643679fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:47.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.246+0000 7f46d2000640 0 client.8907 ms_handle_reset on v2:172.21.15.86:6834/1331851138 2024-04-20T21:28:47.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.246+0000 7f46f6dbd740 1 -- 172.21.15.86:0/1640261336 shutdown_connections 2024-04-20T21:28:47.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.246+0000 7f46f6dbd740 1 -- 172.21.15.86:0/1640261336 wait complete. 2024-04-20T21:28:47.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.246+0000 7f46f6dbd740 20 client.8907 trim_cache size 0 max 16384 2024-04-20T21:28:47.249 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:28:47.249 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-20T21:28:47.249 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-20T21:28:47.249 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:28:47.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.247+0000 7f46f4000640 1 -- 172.21.15.86:0/313238337 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f46cc06c8f0 msgr2=0x7f46cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:47.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.247+0000 7f46f4000640 1 --2- 172.21.15.86:0/313238337 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f46cc06c8f0 0x7f46cc06edb0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f46e0000f60 tx=0x7f46e0007c00 comp rx=0 tx=0).stop 2024-04-20T21:28:47.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.247+0000 7f46f4000640 1 -- 172.21.15.86:0/313238337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46ec0b4530 msgr2=0x7f46ec1e9970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:47.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.247+0000 7f46f4000640 1 --2- 172.21.15.86:0/313238337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46ec0b4530 0x7f46ec1e9970 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f46d800c430 tx=0x7f46d800c8f0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.247+0000 7f46f4000640 1 -- 172.21.15.86:0/313238337 shutdown_connections 2024-04-20T21:28:47.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.247+0000 7f46f4000640 1 --2- 172.21.15.86:0/313238337 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f46cc06c8f0 0x7f46cc06edb0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.247+0000 7f46f4000640 1 --2- 172.21.15.86:0/313238337 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f46ec15eb30 0x7f46ec1f89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.247+0000 7f46f4000640 1 --2- 172.21.15.86:0/313238337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46ec0b4530 0x7f46ec1e9970 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.247+0000 7f46f4000640 1 --2- 172.21.15.86:0/313238337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f46ec0aee50 0x7f46ec150ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.247+0000 7f46f4000640 1 -- 172.21.15.86:0/313238337 >> 172.21.15.86:0/313238337 conn(0x7f46ec0b2110 msgr2=0x7f46ec0b3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:47.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.247+0000 7f46f4000640 1 -- 172.21.15.86:0/313238337 shutdown_connections 2024-04-20T21:28:47.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.247+0000 7f46f4000640 1 -- 172.21.15.86:0/313238337 wait complete. 2024-04-20T21:28:47.264 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T21:28:47.365 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-20T21:28:47.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.574+0000 7f16c3000640 1 Processor -- start 2024-04-20T21:28:47.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.574+0000 7f16c3000640 1 -- start start 2024-04-20T21:28:47.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.576+0000 7f16c3000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f16bc15c590 0x7f16bc0ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:47.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.576+0000 7f16c3000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16bc0af7e0 0x7f16bc0cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:47.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.576+0000 7f16c3000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f16bc06daf0 0x7f16bc0d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:47.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.576+0000 7f16c3000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f16bc160160 con 0x7f16bc15c590 2024-04-20T21:28:47.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.576+0000 7f16c3000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f16bc160440 con 0x7f16bc06daf0 2024-04-20T21:28:47.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.576+0000 7f16c3000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f16bc160720 con 0x7f16bc0af7e0 2024-04-20T21:28:47.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.576+0000 7f16c1c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f16bc15c590 0x7f16bc0ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:47.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.576+0000 7f16c1c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f16bc15c590 0x7f16bc0ccb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45710/0 (socket says 172.21.15.86:45710) 2024-04-20T21:28:47.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.576+0000 7f16c1c00640 1 -- 172.21.15.86:0/104510560 learned_addr learned my addr 172.21.15.86:0/104510560 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:47.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.576+0000 7f16c1c00640 1 -- 172.21.15.86:0/104510560 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16bc0af7e0 msgr2=0x7f16bc0cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:47.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.576+0000 7f16c1c00640 1 --2- 172.21.15.86:0/104510560 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16bc0af7e0 0x7f16bc0cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.576+0000 7f16c1c00640 1 -- 172.21.15.86:0/104510560 >> v2:172.21.15.144:3300/0 conn(0x7f16bc06daf0 msgr2=0x7f16bc0d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:47.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.576+0000 7f16c1c00640 1 --2- 172.21.15.86:0/104510560 >> v2:172.21.15.144:3300/0 conn(0x7f16bc06daf0 0x7f16bc0d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.576+0000 7f16c1c00640 1 -- 172.21.15.86:0/104510560 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16bc160a00 con 0x7f16bc15c590 2024-04-20T21:28:47.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.578+0000 7f16c1c00640 1 --2- 172.21.15.86:0/104510560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16bc15c590 0x7f16bc0ccb10 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f16bc0c71a0 tx=0x7f16b002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bae9781cfff65c4c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:47.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.578+0000 7f16bbe00640 1 -- 172.21.15.86:0/104510560 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b0035020 con 0x7f16bc15c590 2024-04-20T21:28:47.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.578+0000 7f16bbe00640 1 -- 172.21.15.86:0/104510560 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16b0031870 con 0x7f16bc15c590 2024-04-20T21:28:47.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.578+0000 7f16bbe00640 1 -- 172.21.15.86:0/104510560 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b0031c40 con 0x7f16bc15c590 2024-04-20T21:28:47.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.579+0000 7f16c3000640 1 -- 172.21.15.86:0/104510560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16bc15c590 msgr2=0x7f16bc0ccb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:47.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.579+0000 7f16c3000640 1 --2- 172.21.15.86:0/104510560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16bc15c590 0x7f16bc0ccb10 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f16bc0c71a0 tx=0x7f16b002b910 comp rx=0 tx=0).stop 2024-04-20T21:28:47.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.579+0000 7f16c3000640 1 -- 172.21.15.86:0/104510560 shutdown_connections 2024-04-20T21:28:47.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.579+0000 7f16c3000640 1 --2- 172.21.15.86:0/104510560 >> v2:172.21.15.144:3300/0 conn(0x7f16bc06daf0 0x7f16bc0d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.579+0000 7f16c3000640 1 --2- 172.21.15.86:0/104510560 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16bc0af7e0 0x7f16bc0cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.579+0000 7f16c3000640 1 --2- 172.21.15.86:0/104510560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16bc15c590 0x7f16bc0ccb10 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.579+0000 7f16c3000640 1 -- 172.21.15.86:0/104510560 >> 172.21.15.86:0/104510560 conn(0x7f16bc06bb60 msgr2=0x7f16bc06bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:47.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.579+0000 7f16c3000640 1 -- 172.21.15.86:0/104510560 shutdown_connections 2024-04-20T21:28:47.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.579+0000 7f16c3000640 1 -- 172.21.15.86:0/104510560 wait complete. 2024-04-20T21:28:47.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.581+0000 7f16c3000640 1 Processor -- start 2024-04-20T21:28:47.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.582+0000 7f16c3000640 1 -- start start 2024-04-20T21:28:47.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.583+0000 7f16c3000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16bc06daf0 0x7f16bc09d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:47.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.583+0000 7f16c3000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16bc0af7e0 0x7f16bc1e5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:47.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.583+0000 7f16c3000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16bc15c590 0x7f16bc1e6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:47.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.583+0000 7f16c3000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc15c970 con 0x7f16bc15c590 2024-04-20T21:28:47.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.583+0000 7f16c3000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc06f3d0 con 0x7f16bc06daf0 2024-04-20T21:28:47.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.583+0000 7f16c3000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f16bc160fc0 con 0x7f16bc0af7e0 2024-04-20T21:28:47.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.583+0000 7f16c2600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16bc15c590 0x7f16bc1e6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:47.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.583+0000 7f16c2600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16bc15c590 0x7f16bc1e6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45724/0 (socket says 172.21.15.86:45724) 2024-04-20T21:28:47.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.583+0000 7f16c2600640 1 -- 172.21.15.86:0/3448811567 learned_addr learned my addr 172.21.15.86:0/3448811567 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:47.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.584+0000 7f16c1200640 1 --2- 172.21.15.86:0/3448811567 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16bc0af7e0 0x7f16bc1e5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:47.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.584+0000 7f16c1c00640 1 --2- 172.21.15.86:0/3448811567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16bc06daf0 0x7f16bc09d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:47.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.584+0000 7f16c2600640 1 -- 172.21.15.86:0/3448811567 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16bc0af7e0 msgr2=0x7f16bc1e5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:47.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.584+0000 7f16c2600640 1 --2- 172.21.15.86:0/3448811567 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16bc0af7e0 0x7f16bc1e5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.584+0000 7f16c2600640 1 -- 172.21.15.86:0/3448811567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16bc06daf0 msgr2=0x7f16bc09d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:47.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.584+0000 7f16c2600640 1 --2- 172.21.15.86:0/3448811567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16bc06daf0 0x7f16bc09d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.584+0000 7f16c2600640 1 -- 172.21.15.86:0/3448811567 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16bc15e1a0 con 0x7f16bc15c590 2024-04-20T21:28:47.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.584+0000 7f16c1c00640 1 --2- 172.21.15.86:0/3448811567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16bc06daf0 0x7f16bc09d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:28:47.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.584+0000 7f16c1200640 1 --2- 172.21.15.86:0/3448811567 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16bc0af7e0 0x7f16bc1e5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:28:47.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.584+0000 7f16c2600640 1 --2- 172.21.15.86:0/3448811567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16bc15c590 0x7f16bc1e6480 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f16b400b9e0 tx=0x7f16b400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:47.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.584+0000 7f16b9600640 1 -- 172.21.15.86:0/3448811567 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b4019070 con 0x7f16bc15c590 2024-04-20T21:28:47.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.584+0000 7f16b9600640 1 -- 172.21.15.86:0/3448811567 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16b400cb20 con 0x7f16bc15c590 2024-04-20T21:28:47.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.584+0000 7f16b9600640 1 -- 172.21.15.86:0/3448811567 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b4014440 con 0x7f16bc15c590 2024-04-20T21:28:47.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.584+0000 7f16c3000640 1 -- 172.21.15.86:0/3448811567 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16bc15e480 con 0x7f16bc15c590 2024-04-20T21:28:47.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.585+0000 7f16c3000640 1 -- 172.21.15.86:0/3448811567 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16bc15ea40 con 0x7f16bc15c590 2024-04-20T21:28:47.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.586+0000 7f16c3000640 1 -- 172.21.15.86:0/3448811567 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16bc04fcf0 con 0x7f16bc15c590 2024-04-20T21:28:47.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.587+0000 7f16b9600640 1 -- 172.21.15.86:0/3448811567 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f16b401c070 con 0x7f16bc15c590 2024-04-20T21:28:47.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.589+0000 7f16b9600640 1 --2- 172.21.15.86:0/3448811567 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f169c06c8f0 0x7f169c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:47.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.589+0000 7f16b9600640 1 -- 172.21.15.86:0/3448811567 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7f16b408b890 con 0x7f16bc15c590 2024-04-20T21:28:47.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.589+0000 7f16c1c00640 1 --2- 172.21.15.86:0/3448811567 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f169c06c8f0 0x7f169c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:47.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.590+0000 7f16c1c00640 1 --2- 172.21.15.86:0/3448811567 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f169c06c8f0 0x7f169c06edb0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f16b0036000 tx=0x7f16b003c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:47.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.595+0000 7f16b9600640 1 -- 172.21.15.86:0/3448811567 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f16b4059cb0 con 0x7f16bc15c590 2024-04-20T21:28:47.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.840+0000 7f16c3000640 1 -- 172.21.15.86:0/3448811567 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f16bc0afc60 con 0x7f16bc15c590 2024-04-20T21:28:47.840 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-04-20T21:28:47.841+0000 7f5bb9c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T21:28:47.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.845+0000 7f16b9600640 1 -- 172.21.15.86:0/3448811567 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v83) ==== 161+0+0 (secure 0 0 0) 0x7f16b40595b0 con 0x7f16bc15c590 2024-04-20T21:28:47.844 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-20T21:28:47.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.848+0000 7f16c3000640 1 -- 172.21.15.86:0/3448811567 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f169c06c8f0 msgr2=0x7f169c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:47.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.848+0000 7f16c3000640 1 --2- 172.21.15.86:0/3448811567 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f169c06c8f0 0x7f169c06edb0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f16b0036000 tx=0x7f16b003c040 comp rx=0 tx=0).stop 2024-04-20T21:28:47.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.848+0000 7f16c3000640 1 -- 172.21.15.86:0/3448811567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16bc15c590 msgr2=0x7f16bc1e6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:47.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.848+0000 7f16c3000640 1 --2- 172.21.15.86:0/3448811567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16bc15c590 0x7f16bc1e6480 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f16b400b9e0 tx=0x7f16b400bea0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.848+0000 7f16c3000640 1 -- 172.21.15.86:0/3448811567 shutdown_connections 2024-04-20T21:28:47.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.848+0000 7f16c3000640 1 --2- 172.21.15.86:0/3448811567 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f169c06c8f0 0x7f169c06edb0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.848+0000 7f16c3000640 1 --2- 172.21.15.86:0/3448811567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f16bc15c590 0x7f16bc1e6480 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.848+0000 7f16c3000640 1 --2- 172.21.15.86:0/3448811567 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f16bc0af7e0 0x7f16bc1e5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.848+0000 7f16c3000640 1 --2- 172.21.15.86:0/3448811567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f16bc06daf0 0x7f16bc09d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.848+0000 7f16c3000640 1 -- 172.21.15.86:0/3448811567 >> 172.21.15.86:0/3448811567 conn(0x7f16bc06bb60 msgr2=0x7f16bc159630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:47.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.849+0000 7f16c3000640 1 -- 172.21.15.86:0/3448811567 shutdown_connections 2024-04-20T21:28:47.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.849+0000 7f16c3000640 1 -- 172.21.15.86:0/3448811567 wait complete. 2024-04-20T21:28:47.864 INFO:tasks.ceph.mds.c.smithi086.stderr: -6361> 2024-04-20T21:27:51.185+0000 7f1c04800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T21:28:47.869 DEBUG:teuthology.orchestra.run.smithi086:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627777 summary 2024-04-20T21:28:47.921 INFO:tasks.ceph.mds.c.smithi086.stdout:starting mds.c at 2024-04-20T21:28:47.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.959+0000 7fdb84a14280 1 -- start start 2024-04-20T21:28:47.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.959+0000 7fdb84a14280 1 Processor -- start 2024-04-20T21:28:47.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.960+0000 7fdb84a14280 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b0cf602000 0x55b0ce9de580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:47.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.960+0000 7fdb84a14280 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0ce86bc00 0x55b0ce9df080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:47.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.960+0000 7fdb84a14280 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0cf602800 0x55b0ce9deb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:47.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.960+0000 7fdb84a14280 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b0cf642000 con 0x55b0ce86bc00 2024-04-20T21:28:47.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.960+0000 7fdb84a14280 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b0cf642180 con 0x55b0cf602800 2024-04-20T21:28:47.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.960+0000 7fdb84a14280 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55b0cf642300 con 0x55b0cf602000 2024-04-20T21:28:47.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.960+0000 7fdb81600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0ce86bc00 0x55b0ce9df080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:47.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.960+0000 7fdb81600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0ce86bc00 0x55b0ce9df080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45762/0 (socket says 172.21.15.86:45762) 2024-04-20T21:28:47.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.960+0000 7fdb81600640 1 -- 172.21.15.86:0/3604107914 learned_addr learned my addr 172.21.15.86:0/3604107914 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:47.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.961+0000 7fdb82000640 1 --2- 172.21.15.86:0/3604107914 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b0cf602000 0x55b0ce9de580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:47.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.961+0000 7fdb81600640 1 -- 172.21.15.86:0/3604107914 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b0cf602000 msgr2=0x55b0ce9de580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:47.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.961+0000 7fdb82a00640 1 --2- 172.21.15.86:0/3604107914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0cf602800 0x55b0ce9deb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:47.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.961+0000 7fdb81600640 1 --2- 172.21.15.86:0/3604107914 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b0cf602000 0x55b0ce9de580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.961+0000 7fdb81600640 1 -- 172.21.15.86:0/3604107914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0cf602800 msgr2=0x55b0ce9deb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:47.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.961+0000 7fdb81600640 1 --2- 172.21.15.86:0/3604107914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0cf602800 0x55b0ce9deb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.961+0000 7fdb81600640 1 -- 172.21.15.86:0/3604107914 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0ce89e780 con 0x55b0ce86bc00 2024-04-20T21:28:47.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.961+0000 7fdb81600640 1 --2- 172.21.15.86:0/3604107914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0ce86bc00 0x55b0ce9df080 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x55b0cf6994a0 tx=0x55b0cf6061e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac4df1b8b13c06c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:47.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.961+0000 7fdb7ee00640 1 -- 172.21.15.86:0/3604107914 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b0ce890d00 con 0x55b0ce86bc00 2024-04-20T21:28:47.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.961+0000 7fdb7ee00640 1 -- 172.21.15.86:0/3604107914 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55b0ce896700 con 0x55b0ce86bc00 2024-04-20T21:28:47.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.961+0000 7fdb7ee00640 1 -- 172.21.15.86:0/3604107914 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b0ce8909c0 con 0x55b0ce86bc00 2024-04-20T21:28:47.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.961+0000 7fdb84a14280 1 -- 172.21.15.86:0/3604107914 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0ce89e000 con 0x55b0ce86bc00 2024-04-20T21:28:47.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.961+0000 7fdb84a14280 1 -- 172.21.15.86:0/3604107914 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55b0ce89e960 con 0x55b0ce86bc00 2024-04-20T21:28:47.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.961+0000 7fdb84a14280 4 waiting for MDS map... 2024-04-20T21:28:47.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.961+0000 7fdb7ee00640 1 -- 172.21.15.86:0/3604107914 <== mon.0 v2:172.21.15.86:3300/0 4 ==== osd_map(101..101 src has 1..101) ==== 5795+0+0 (secure 0 0 0) 0x55b0cf644600 con 0x55b0ce86bc00 2024-04-20T21:28:47.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.962+0000 7fdb7ee00640 1 -- 172.21.15.86:0/3604107914 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 83) ==== 2886+0+0 (secure 0 0 0) 0x55b0ce8c5180 con 0x55b0ce86bc00 2024-04-20T21:28:47.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.962+0000 7fdb84a14280 4 Got MDS map 83 2024-04-20T21:28:47.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.962+0000 7fdb84a14280 10 main: JournalTool::main 2024-04-20T21:28:47.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.962+0000 7fdb84a14280 4 main: JournalTool: connecting to RADOS... 2024-04-20T21:28:47.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.963+0000 7fdb84a14280 1 Processor -- start 2024-04-20T21:28:47.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.963+0000 7fdb84a14280 1 -- start start 2024-04-20T21:28:47.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.963+0000 7fdb84a14280 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0cf67bc00 0x55b0ce9de000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:47.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.963+0000 7fdb84a14280 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0cf7a0000 0x55b0ce9df600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:47.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.963+0000 7fdb84a14280 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b0cf7a0400 0x55b0ce9dfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:47.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.963+0000 7fdb84a14280 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b0cf642480 con 0x55b0cf67bc00 2024-04-20T21:28:47.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.963+0000 7fdb84a14280 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b0cf642600 con 0x55b0cf7a0000 2024-04-20T21:28:47.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.963+0000 7fdb84a14280 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55b0cf642780 con 0x55b0cf7a0400 2024-04-20T21:28:47.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.964+0000 7fdb82a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0cf67bc00 0x55b0ce9de000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:47.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.964+0000 7fdb82a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0cf67bc00 0x55b0ce9de000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45764/0 (socket says 172.21.15.86:45764) 2024-04-20T21:28:47.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.964+0000 7fdb82a00640 1 -- 172.21.15.86:0/2779706809 learned_addr learned my addr 172.21.15.86:0/2779706809 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:47.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.964+0000 7fdb82a00640 1 -- 172.21.15.86:0/2779706809 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b0cf7a0400 msgr2=0x55b0ce9dfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:47.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.964+0000 7fdb82a00640 1 --2- 172.21.15.86:0/2779706809 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b0cf7a0400 0x55b0ce9dfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.964+0000 7fdb82a00640 1 -- 172.21.15.86:0/2779706809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0cf7a0000 msgr2=0x55b0ce9df600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:47.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.964+0000 7fdb82a00640 1 --2- 172.21.15.86:0/2779706809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0cf7a0000 0x55b0ce9df600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.964+0000 7fdb82a00640 1 -- 172.21.15.86:0/2779706809 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0ce89f680 con 0x55b0cf67bc00 2024-04-20T21:28:47.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.964+0000 7fdb82a00640 1 --2- 172.21.15.86:0/2779706809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0cf67bc00 0x55b0ce9de000 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x55b0cf74e8d0 tx=0x55b0cf754c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12148638ba9e4bad server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:47.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.964+0000 7fdb7c600640 1 -- 172.21.15.86:0/2779706809 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b0ce890d00 con 0x55b0cf67bc00 2024-04-20T21:28:47.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.964+0000 7fdb7c600640 1 -- 172.21.15.86:0/2779706809 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55b0ce896c40 con 0x55b0cf67bc00 2024-04-20T21:28:47.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.964+0000 7fdb7c600640 1 -- 172.21.15.86:0/2779706809 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b0ce8909c0 con 0x55b0cf67bc00 2024-04-20T21:28:47.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.965+0000 7fdb84a14280 1 -- 172.21.15.86:0/2779706809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0cf67bc00 msgr2=0x55b0ce9de000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:47.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.965+0000 7fdb84a14280 1 --2- 172.21.15.86:0/2779706809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0cf67bc00 0x55b0ce9de000 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x55b0cf74e8d0 tx=0x55b0cf754c00 comp rx=0 tx=0).stop 2024-04-20T21:28:47.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.965+0000 7fdb84a14280 1 -- 172.21.15.86:0/2779706809 shutdown_connections 2024-04-20T21:28:47.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.965+0000 7fdb84a14280 1 --2- 172.21.15.86:0/2779706809 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b0cf7a0400 0x55b0ce9dfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.965+0000 7fdb84a14280 1 --2- 172.21.15.86:0/2779706809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0cf7a0000 0x55b0ce9df600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.965+0000 7fdb84a14280 1 --2- 172.21.15.86:0/2779706809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0cf67bc00 0x55b0ce9de000 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.965+0000 7fdb84a14280 1 -- 172.21.15.86:0/2779706809 >> 172.21.15.86:0/2779706809 conn(0x55b0cf602400 msgr2=0x55b0cf5ef000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:47.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.965+0000 7fdb84a14280 1 -- 172.21.15.86:0/2779706809 shutdown_connections 2024-04-20T21:28:47.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.965+0000 7fdb84a14280 1 -- 172.21.15.86:0/2779706809 wait complete. 2024-04-20T21:28:47.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.966+0000 7fdb84a14280 1 Processor -- start 2024-04-20T21:28:47.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.966+0000 7fdb84a14280 1 -- start start 2024-04-20T21:28:47.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.967+0000 7fdb84a14280 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0cf67bc00 0x55b0ce9de000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:47.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.967+0000 7fdb84a14280 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b0cf7b2000 0x55b0ce9df600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:47.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.967+0000 7fdb84a14280 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0cf7b2400 0x55b0ce9dfb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:47.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.967+0000 7fdb84a14280 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b0cf643e00 con 0x55b0cf67bc00 2024-04-20T21:28:47.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.967+0000 7fdb84a14280 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b0cf643c80 con 0x55b0cf7b2400 2024-04-20T21:28:47.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.967+0000 7fdb84a14280 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55b0cf643b00 con 0x55b0cf7b2000 2024-04-20T21:28:47.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.967+0000 7fdb82a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0cf67bc00 0x55b0ce9de000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:47.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.967+0000 7fdb82a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0cf67bc00 0x55b0ce9de000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45780/0 (socket says 172.21.15.86:45780) 2024-04-20T21:28:47.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.967+0000 7fdb82a00640 1 -- 172.21.15.86:0/3769599860 learned_addr learned my addr 172.21.15.86:0/3769599860 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:47.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.967+0000 7fdb82a00640 1 -- 172.21.15.86:0/3769599860 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b0cf7b2000 msgr2=0x55b0ce9df600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:47.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.967+0000 7fdb82a00640 1 --2- 172.21.15.86:0/3769599860 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b0cf7b2000 0x55b0ce9df600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.967+0000 7fdb82a00640 1 -- 172.21.15.86:0/3769599860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0cf7b2400 msgr2=0x55b0ce9dfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:47.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.967+0000 7fdb82a00640 1 --2- 172.21.15.86:0/3769599860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0cf7b2400 0x55b0ce9dfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.967+0000 7fdb82a00640 1 -- 172.21.15.86:0/3769599860 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0ce89e780 con 0x55b0cf67bc00 2024-04-20T21:28:47.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.968+0000 7fdb82a00640 1 --2- 172.21.15.86:0/3769599860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0cf67bc00 0x55b0ce9de000 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x55b0cf74eb40 tx=0x55b0cf604ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:47.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.968+0000 7fdb7a800640 1 -- 172.21.15.86:0/3769599860 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b0ce8901a0 con 0x55b0cf67bc00 2024-04-20T21:28:47.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.968+0000 7fdb7a800640 1 -- 172.21.15.86:0/3769599860 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55b0ce8961c0 con 0x55b0cf67bc00 2024-04-20T21:28:47.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.968+0000 7fdb84a14280 1 -- 172.21.15.86:0/3769599860 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b0ce89e000 con 0x55b0cf67bc00 2024-04-20T21:28:47.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.968+0000 7fdb7a800640 1 -- 172.21.15.86:0/3769599860 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b0ce890d00 con 0x55b0cf67bc00 2024-04-20T21:28:47.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.968+0000 7fdb84a14280 1 -- 172.21.15.86:0/3769599860 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0ce89f860 con 0x55b0cf67bc00 2024-04-20T21:28:47.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.968+0000 7fdb84a14280 4 main: JournalTool: resolving pool 14 2024-04-20T21:28:47.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.970+0000 7fdb7a800640 1 -- 172.21.15.86:0/3769599860 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x55b0cf7a0c00 con 0x55b0cf67bc00 2024-04-20T21:28:47.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.970+0000 7fdb7a800640 1 --2- 172.21.15.86:0/3769599860 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x55b0cf7b4800 0x55b0ce9e0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:47.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.970+0000 7fdb7a800640 1 -- 172.21.15.86:0/3769599860 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5795+0+0 (secure 0 0 0) 0x55b0cf7aea00 con 0x55b0cf67bc00 2024-04-20T21:28:47.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.970+0000 7fdb84a14280 4 main: JournalTool: creating IoCtx.. 2024-04-20T21:28:47.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.970+0000 7fdb84a14280 4 main: Executing for rank 0 2024-04-20T21:28:47.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.970+0000 7fdb84a14280 4 Filtering by inode '1099511627777' 2024-04-20T21:28:47.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.971+0000 7fdb84a14280 1 --2- 172.21.15.86:0/3769599860 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x55b0cf7b2800 0x55b0ce9e0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:47.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.971+0000 7fdb84a14280 1 -- 172.21.15.86:0/3769599860 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:1 14.f 14:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101) -- 0x55b0cf7b2c00 con 0x55b0cf7b2800 2024-04-20T21:28:47.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.971+0000 7fdb82000640 1 --2- 172.21.15.86:0/3769599860 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x55b0cf7b4800 0x55b0ce9e0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:47.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.971+0000 7fdb82000640 1 --2- 172.21.15.86:0/3769599860 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x55b0cf7b4800 0x55b0ce9e0100 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x55b0ce94e9f0 tx=0x55b0cf605f20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:47.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.971+0000 7fdb81600640 1 --2- 172.21.15.86:0/3769599860 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x55b0cf7b2800 0x55b0ce9e0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:47.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.972+0000 7fdb81600640 1 --2- 172.21.15.86:0/3769599860 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x55b0cf7b2800 0x55b0ce9e0680 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:47.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.973+0000 7fdb81600640 1 -- 172.21.15.86:0/3769599860 <== osd.7 v2:172.21.15.144:6816/2100967057 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x55b0cf5f4000 con 0x55b0cf7b2800 2024-04-20T21:28:47.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.974+0000 7fdb84a14280 4 Pointer 400.00000000 is readable 2024-04-20T21:28:47.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.974+0000 7fdb84a14280 4 JournalScanner::scan: reading header object '200.00000000' 2024-04-20T21:28:47.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.974+0000 7fdb84a14280 1 --2- 172.21.15.86:0/3769599860 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x55b0cf7b3000 0x55b0ce9e0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:47.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.974+0000 7fdb84a14280 1 -- 172.21.15.86:0/3769599860 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:2 14.14 14:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101) -- 0x55b0cf7b3400 con 0x55b0cf7b3000 2024-04-20T21:28:47.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.974+0000 7fdb82a00640 1 --2- 172.21.15.86:0/3769599860 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x55b0cf7b3000 0x55b0ce9e0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:47.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.975+0000 7fdb82a00640 1 --2- 172.21.15.86:0/3769599860 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x55b0cf7b3000 0x55b0ce9e0c00 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:47.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.977+0000 7fdb82a00640 1 -- 172.21.15.86:0/3769599860 <== osd.4 v2:172.21.15.144:6800/4087309475 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv31 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x55b0cf5f4500 con 0x55b0cf7b3000 2024-04-20T21:28:47.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.977+0000 7fdb84a14280 10 Header 0x400000 0x405f0e 0x40628a 2024-04-20T21:28:47.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.977+0000 7fdb84a14280 10 Starting journal scan from offset 0x405f0e 2024-04-20T21:28:47.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.977+0000 7fdb84a14280 1 -- 172.21.15.86:0/3769599860 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:3 14.14 14:2e2fa760:::200.00000001:head [read 24334~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101) -- 0x55b0cf7b3800 con 0x55b0cf7b3000 2024-04-20T21:28:47.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.978+0000 7fdb82a00640 1 -- 172.21.15.86:0/3769599860 <== osd.4 v2:172.21.15.144:6800/4087309475 2 ==== osd_op_reply(3 200.00000001 [read 24334~892 out=892b] v0'0 uv27 ondisk = 0) ==== 156+0+892 (crc 0 0 0) 0x55b0cf5f4500 con 0x55b0cf7b3000 2024-04-20T21:28:47.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.978+0000 7fdb84a14280 4 Read 0x37c bytes from 200.00000001 gap=0 2024-04-20T21:28:47.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.978+0000 7fdb84a14280 10 Parsing data, 0x37c bytes available 2024-04-20T21:28:47.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.978+0000 7fdb84a14280 10 Attempting decode at 0x405f0e 2024-04-20T21:28:47.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.978+0000 7fdb84a14280 10 Consumed 0x37c bytes 2024-04-20T21:28:47.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.978+0000 7fdb84a14280 10 Valid entry at 0x405f0e 2024-04-20T21:28:47.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.978+0000 7fdb84a14280 1 --2- 172.21.15.86:0/3769599860 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x55b0cf7b3c00 0x55b0ce9e1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:47.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.978+0000 7fdb84a14280 1 -- 172.21.15.86:0/3769599860 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:4 14.1b 14:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101) -- 0x55b0cf7b4c00 con 0x55b0cf7b3c00 2024-04-20T21:28:47.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.979+0000 7fdb82000640 1 --2- 172.21.15.86:0/3769599860 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x55b0cf7b3c00 0x55b0ce9e1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:47.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.979+0000 7fdb82000640 1 --2- 172.21.15.86:0/3769599860 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x55b0cf7b3c00 0x55b0ce9e1180 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:47.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.982+0000 7fdb82000640 1 -- 172.21.15.86:0/3769599860 <== osd.1 v2:172.21.15.86:6817/3149564961 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55b0cf5f4780 con 0x55b0cf7b3c00 2024-04-20T21:28:47.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.982+0000 7fdb84a14280 4 Reached end of journal objects 2024-04-20T21:28:47.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.982+0000 7fdb84a14280 4 Scanned objects, 0 missing, 1 valid 2024-04-20T21:28:47.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.982+0000 7fdb84a14280 4 Events scanned, 0 gaps 2024-04-20T21:28:47.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.982+0000 7fdb84a14280 4 Found 1 valid events 2024-04-20T21:28:47.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.982+0000 7fdb84a14280 4 Selected 0 events events for processing 2024-04-20T21:28:47.981 INFO:teuthology.orchestra.run.smithi086.stdout:Events by type: 2024-04-20T21:28:47.981 INFO:teuthology.orchestra.run.smithi086.stdout:Errors: 0 2024-04-20T21:28:47.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.982+0000 7fdb84a14280 4 0 events 2024-04-20T21:28:47.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.982+0000 7fdb84a14280 1 -- 172.21.15.86:0/3769599860 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x55b0cf7b3c00 msgr2=0x55b0ce9e1180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:47.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.982+0000 7fdb84a14280 1 --2- 172.21.15.86:0/3769599860 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x55b0cf7b3c00 0x55b0ce9e1180 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.982+0000 7fdb84a14280 1 -- 172.21.15.86:0/3769599860 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x55b0cf7b3000 msgr2=0x55b0ce9e0c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:47.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.982+0000 7fdb84a14280 1 --2- 172.21.15.86:0/3769599860 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x55b0cf7b3000 0x55b0ce9e0c00 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.982+0000 7fdb84a14280 1 -- 172.21.15.86:0/3769599860 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x55b0cf7b2800 msgr2=0x55b0ce9e0680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:47.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.982+0000 7fdb84a14280 1 --2- 172.21.15.86:0/3769599860 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x55b0cf7b2800 0x55b0ce9e0680 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.982+0000 7fdb82000640 1 -- 172.21.15.86:0/3769599860 reap_dead start 2024-04-20T21:28:47.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.982+0000 7fdb84a14280 1 -- 172.21.15.86:0/3769599860 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x55b0cf7b4800 msgr2=0x55b0ce9e0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:47.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.982+0000 7fdb84a14280 1 --2- 172.21.15.86:0/3769599860 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x55b0cf7b4800 0x55b0ce9e0100 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x55b0ce94e9f0 tx=0x55b0cf605f20 comp rx=0 tx=0).stop 2024-04-20T21:28:47.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.982+0000 7fdb84a14280 1 -- 172.21.15.86:0/3769599860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0cf67bc00 msgr2=0x55b0ce9de000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:47.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.982+0000 7fdb84a14280 1 --2- 172.21.15.86:0/3769599860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0cf67bc00 0x55b0ce9de000 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x55b0cf74eb40 tx=0x55b0cf604ae0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.982+0000 7fdb84a14280 1 -- 172.21.15.86:0/3769599860 shutdown_connections 2024-04-20T21:28:47.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.982+0000 7fdb84a14280 1 --2- 172.21.15.86:0/3769599860 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x55b0cf7b4800 0x55b0ce9e0100 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.982+0000 7fdb84a14280 1 --2- 172.21.15.86:0/3769599860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0cf67bc00 0x55b0ce9de000 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.982+0000 7fdb84a14280 1 -- 172.21.15.86:0/3769599860 >> 172.21.15.86:0/3769599860 conn(0x55b0cf602400 msgr2=0x55b0cf5ef000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:47.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.982+0000 7fdb84a14280 1 -- 172.21.15.86:0/3769599860 shutdown_connections 2024-04-20T21:28:47.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.982+0000 7fdb84a14280 1 -- 172.21.15.86:0/3769599860 wait complete. 2024-04-20T21:28:47.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.983+0000 7fdb84a14280 1 -- 172.21.15.86:0/3604107914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0ce86bc00 msgr2=0x55b0ce9df080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:47.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.983+0000 7fdb84a14280 1 --2- 172.21.15.86:0/3604107914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0ce86bc00 0x55b0ce9df080 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x55b0cf6994a0 tx=0x55b0cf6061e0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.983+0000 7fdb84a14280 1 -- 172.21.15.86:0/3604107914 shutdown_connections 2024-04-20T21:28:47.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.983+0000 7fdb84a14280 1 --2- 172.21.15.86:0/3604107914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b0cf602800 0x55b0ce9deb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.983+0000 7fdb84a14280 1 --2- 172.21.15.86:0/3604107914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b0ce86bc00 0x55b0ce9df080 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.983+0000 7fdb84a14280 1 --2- 172.21.15.86:0/3604107914 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b0cf602000 0x55b0ce9de580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:47.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.983+0000 7fdb84a14280 1 -- 172.21.15.86:0/3604107914 >> 172.21.15.86:0/3604107914 conn(0x55b0ce86b000 msgr2=0x55b0cf5f0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:47.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.983+0000 7fdb84a14280 1 -- 172.21.15.86:0/3604107914 shutdown_connections 2024-04-20T21:28:47.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:47.983+0000 7fdb84a14280 1 -- 172.21.15.86:0/3604107914 wait complete. 2024-04-20T21:28:47.987 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627777', 'summary'] in time 0:00:00.117798, result: Events by type: Errors: 0 2024-04-20T21:28:47.987 DEBUG:teuthology.orchestra.run.smithi086:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627778 summary 2024-04-20T21:28:48.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.069+0000 7fe8c9650280 1 -- start start 2024-04-20T21:28:48.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.069+0000 7fe8c9650280 1 Processor -- start 2024-04-20T21:28:48.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.070+0000 7fe8c9650280 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a313822c00 0x55a313996000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:48.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.070+0000 7fe8c9650280 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a313823800 0x55a313996580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:48.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.070+0000 7fe8c9650280 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a313823c00 0x55a313996b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:48.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.070+0000 7fe8c9650280 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a314604180 con 0x55a313822c00 2024-04-20T21:28:48.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.070+0000 7fe8c9650280 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a314604300 con 0x55a313823800 2024-04-20T21:28:48.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.070+0000 7fe8c9650280 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55a314604480 con 0x55a313823c00 2024-04-20T21:28:48.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.071+0000 7fe8c6e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a313822c00 0x55a313996000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:48.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.071+0000 7fe8c6e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a313822c00 0x55a313996000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45794/0 (socket says 172.21.15.86:45794) 2024-04-20T21:28:48.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.071+0000 7fe8c6e00640 1 -- 172.21.15.86:0/3012877264 learned_addr learned my addr 172.21.15.86:0/3012877264 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:48.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.071+0000 7fe8c6e00640 1 -- 172.21.15.86:0/3012877264 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a313823c00 msgr2=0x55a313996b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:48.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.071+0000 7fe8c6e00640 1 --2- 172.21.15.86:0/3012877264 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a313823c00 0x55a313996b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:48.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.071+0000 7fe8c6e00640 1 -- 172.21.15.86:0/3012877264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a313823800 msgr2=0x55a313996580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:48.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.071+0000 7fe8c6e00640 1 --2- 172.21.15.86:0/3012877264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a313823800 0x55a313996580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:48.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.071+0000 7fe8c6e00640 1 -- 172.21.15.86:0/3012877264 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a313856780 con 0x55a313822c00 2024-04-20T21:28:48.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.071+0000 7fe8c6e00640 1 --2- 172.21.15.86:0/3012877264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a313822c00 0x55a313996000 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x55a314649470 tx=0x55a3145bc4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8873ac7ca2ec468 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:48.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.072+0000 7fe8c3c00640 1 -- 172.21.15.86:0/3012877264 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a313848680 con 0x55a313822c00 2024-04-20T21:28:48.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.072+0000 7fe8c3c00640 1 -- 172.21.15.86:0/3012877264 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55a31384e700 con 0x55a313822c00 2024-04-20T21:28:48.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.072+0000 7fe8c3c00640 1 -- 172.21.15.86:0/3012877264 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a313848b60 con 0x55a313822c00 2024-04-20T21:28:48.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.072+0000 7fe8c9650280 1 -- 172.21.15.86:0/3012877264 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3138574a0 con 0x55a313822c00 2024-04-20T21:28:48.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.072+0000 7fe8c9650280 1 -- 172.21.15.86:0/3012877264 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55a313856000 con 0x55a313822c00 2024-04-20T21:28:48.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.072+0000 7fe8c9650280 4 waiting for MDS map... 2024-04-20T21:28:48.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.072+0000 7fe8c3c00640 1 -- 172.21.15.86:0/3012877264 <== mon.0 v2:172.21.15.86:3300/0 4 ==== osd_map(101..101 src has 1..101) ==== 5795+0+0 (secure 0 0 0) 0x55a3146ee600 con 0x55a313822c00 2024-04-20T21:28:48.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.072+0000 7fe8c3c00640 1 -- 172.21.15.86:0/3012877264 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 83) ==== 2886+0+0 (secure 0 0 0) 0x55a31387d180 con 0x55a313822c00 2024-04-20T21:28:48.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.072+0000 7fe8c9650280 4 Got MDS map 83 2024-04-20T21:28:48.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.072+0000 7fe8c9650280 10 main: JournalTool::main 2024-04-20T21:28:48.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.072+0000 7fe8c9650280 4 main: JournalTool: connecting to RADOS... 2024-04-20T21:28:48.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.073+0000 7fe8c9650280 1 Processor -- start 2024-04-20T21:28:48.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.073+0000 7fe8c9650280 1 -- start start 2024-04-20T21:28:48.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.074+0000 7fe8c9650280 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a3146f1400 0x55a313997080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:48.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.074+0000 7fe8c9650280 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3146f1800 0x55a313997600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:48.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.074+0000 7fe8c9650280 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a3146f1c00 0x55a313997b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:48.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.074+0000 7fe8c9650280 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a314604600 con 0x55a3146f1800 2024-04-20T21:28:48.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.074+0000 7fe8c9650280 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a314604780 con 0x55a3146f1400 2024-04-20T21:28:48.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.074+0000 7fe8c9650280 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55a314604900 con 0x55a3146f1c00 2024-04-20T21:28:48.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.074+0000 7fe8c6e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3146f1800 0x55a313997600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:48.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.074+0000 7fe8c6e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3146f1800 0x55a313997600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45800/0 (socket says 172.21.15.86:45800) 2024-04-20T21:28:48.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.074+0000 7fe8c6e00640 1 -- 172.21.15.86:0/1046150194 learned_addr learned my addr 172.21.15.86:0/1046150194 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:48.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.075+0000 7fe8c6e00640 1 -- 172.21.15.86:0/1046150194 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a3146f1c00 msgr2=0x55a313997b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:48.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.075+0000 7fe8c6e00640 1 --2- 172.21.15.86:0/1046150194 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a3146f1c00 0x55a313997b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:48.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.075+0000 7fe8c6e00640 1 -- 172.21.15.86:0/1046150194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a3146f1400 msgr2=0x55a313997080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:48.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.075+0000 7fe8c6e00640 1 --2- 172.21.15.86:0/1046150194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a3146f1400 0x55a313997080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:48.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.075+0000 7fe8c6e00640 1 -- 172.21.15.86:0/1046150194 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3138563c0 con 0x55a3146f1800 2024-04-20T21:28:48.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.075+0000 7fe8c6e00640 1 --2- 172.21.15.86:0/1046150194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3146f1800 0x55a313997600 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x55a314708660 tx=0x55a3138d7320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=defaf742e11637a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:48.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.075+0000 7fe8c1400640 1 -- 172.21.15.86:0/1046150194 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a313848820 con 0x55a3146f1800 2024-04-20T21:28:48.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.075+0000 7fe8c1400640 1 -- 172.21.15.86:0/1046150194 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55a31384efc0 con 0x55a3146f1800 2024-04-20T21:28:48.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.075+0000 7fe8c1400640 1 -- 172.21.15.86:0/1046150194 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a313848680 con 0x55a3146f1800 2024-04-20T21:28:48.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.076+0000 7fe8c9650280 1 -- 172.21.15.86:0/1046150194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3146f1800 msgr2=0x55a313997600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:48.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.076+0000 7fe8c9650280 1 --2- 172.21.15.86:0/1046150194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3146f1800 0x55a313997600 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x55a314708660 tx=0x55a3138d7320 comp rx=0 tx=0).stop 2024-04-20T21:28:48.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.076+0000 7fe8c9650280 1 -- 172.21.15.86:0/1046150194 shutdown_connections 2024-04-20T21:28:48.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.076+0000 7fe8c9650280 1 --2- 172.21.15.86:0/1046150194 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a3146f1c00 0x55a313997b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:48.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.076+0000 7fe8c9650280 1 --2- 172.21.15.86:0/1046150194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3146f1800 0x55a313997600 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:48.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.076+0000 7fe8c9650280 1 --2- 172.21.15.86:0/1046150194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a3146f1400 0x55a313997080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:48.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.076+0000 7fe8c9650280 1 -- 172.21.15.86:0/1046150194 >> 172.21.15.86:0/1046150194 conn(0x55a3146f1000 msgr2=0x55a313909000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:48.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.076+0000 7fe8c9650280 1 -- 172.21.15.86:0/1046150194 shutdown_connections 2024-04-20T21:28:48.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.076+0000 7fe8c9650280 1 -- 172.21.15.86:0/1046150194 wait complete. 2024-04-20T21:28:48.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.077+0000 7fe8c9650280 1 Processor -- start 2024-04-20T21:28:48.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.077+0000 7fe8c9650280 1 -- start start 2024-04-20T21:28:48.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.077+0000 7fe8c9650280 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a3146f1400 0x55a313997080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:48.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.077+0000 7fe8c9650280 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3146f1800 0x55a313997600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:48.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.077+0000 7fe8c9650280 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a3146f1c00 0x55a313997b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:48.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.077+0000 7fe8c9650280 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a314605c80 con 0x55a3146f1800 2024-04-20T21:28:48.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.077+0000 7fe8c9650280 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a314605b00 con 0x55a3146f1400 2024-04-20T21:28:48.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.077+0000 7fe8c9650280 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55a314605980 con 0x55a3146f1c00 2024-04-20T21:28:48.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.078+0000 7fe8c6e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3146f1800 0x55a313997600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:48.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.078+0000 7fe8c6e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3146f1800 0x55a313997600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45812/0 (socket says 172.21.15.86:45812) 2024-04-20T21:28:48.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.078+0000 7fe8c6e00640 1 -- 172.21.15.86:0/2827900035 learned_addr learned my addr 172.21.15.86:0/2827900035 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:48.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.078+0000 7fe8c6e00640 1 -- 172.21.15.86:0/2827900035 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a3146f1c00 msgr2=0x55a313997b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:48.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.078+0000 7fe8c6e00640 1 --2- 172.21.15.86:0/2827900035 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a3146f1c00 0x55a313997b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:48.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.078+0000 7fe8c6e00640 1 -- 172.21.15.86:0/2827900035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a3146f1400 msgr2=0x55a313997080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:48.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.078+0000 7fe8c6e00640 1 --2- 172.21.15.86:0/2827900035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a3146f1400 0x55a313997080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:48.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.078+0000 7fe8c6e00640 1 -- 172.21.15.86:0/2827900035 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a313856780 con 0x55a3146f1800 2024-04-20T21:28:48.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.078+0000 7fe8c6e00640 1 --2- 172.21.15.86:0/2827900035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3146f1800 0x55a313997600 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x55a314709320 tx=0x55a31470ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:48.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.079+0000 7fe8bf600640 1 -- 172.21.15.86:0/2827900035 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a313848b60 con 0x55a3146f1800 2024-04-20T21:28:48.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.079+0000 7fe8bf600640 1 -- 172.21.15.86:0/2827900035 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55a31384ee00 con 0x55a3146f1800 2024-04-20T21:28:48.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.079+0000 7fe8bf600640 1 -- 172.21.15.86:0/2827900035 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a3138484e0 con 0x55a3146f1800 2024-04-20T21:28:48.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.079+0000 7fe8c9650280 1 -- 172.21.15.86:0/2827900035 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a3138574a0 con 0x55a3146f1800 2024-04-20T21:28:48.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.079+0000 7fe8c9650280 1 -- 172.21.15.86:0/2827900035 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a313856960 con 0x55a3146f1800 2024-04-20T21:28:48.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.079+0000 7fe8c9650280 4 main: JournalTool: resolving pool 14 2024-04-20T21:28:48.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.080+0000 7fe8bf600640 1 -- 172.21.15.86:0/2827900035 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x55a314778c00 con 0x55a3146f1800 2024-04-20T21:28:48.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.081+0000 7fe8bf600640 1 --2- 172.21.15.86:0/2827900035 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x55a314773000 0x55a313998100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:48.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.081+0000 7fe8bf600640 1 -- 172.21.15.86:0/2827900035 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5795+0+0 (secure 0 0 0) 0x55a31477c600 con 0x55a3146f1800 2024-04-20T21:28:48.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.081+0000 7fe8c9650280 4 main: JournalTool: creating IoCtx.. 2024-04-20T21:28:48.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.081+0000 7fe8c9650280 4 main: Executing for rank 0 2024-04-20T21:28:48.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.081+0000 7fe8c9650280 4 Filtering by inode '1099511627778' 2024-04-20T21:28:48.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.081+0000 7fe8c9650280 1 --2- 172.21.15.86:0/2827900035 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x55a314773400 0x55a313998680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:48.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.081+0000 7fe8c9650280 1 -- 172.21.15.86:0/2827900035 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:1 14.f 14:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101) -- 0x55a314773800 con 0x55a314773400 2024-04-20T21:28:48.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.081+0000 7fe8c7800640 1 --2- 172.21.15.86:0/2827900035 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x55a314773000 0x55a313998100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:48.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.082+0000 7fe8c7800640 1 --2- 172.21.15.86:0/2827900035 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x55a314773000 0x55a313998100 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x55a313853a70 tx=0x55a31470fb60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:48.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.082+0000 7fe8c6400640 1 --2- 172.21.15.86:0/2827900035 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x55a314773400 0x55a313998680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:48.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.083+0000 7fe8c6400640 1 --2- 172.21.15.86:0/2827900035 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x55a314773400 0x55a313998680 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:48.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.084+0000 7fe8c6400640 1 -- 172.21.15.86:0/2827900035 <== osd.7 v2:172.21.15.144:6816/2100967057 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x55a3145ac000 con 0x55a314773400 2024-04-20T21:28:48.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.084+0000 7fe8c9650280 4 Pointer 400.00000000 is readable 2024-04-20T21:28:48.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.084+0000 7fe8c9650280 4 JournalScanner::scan: reading header object '200.00000000' 2024-04-20T21:28:48.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.084+0000 7fe8c9650280 1 --2- 172.21.15.86:0/2827900035 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x55a314773c00 0x55a313998c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:48.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.084+0000 7fe8c9650280 1 -- 172.21.15.86:0/2827900035 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:2 14.14 14:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101) -- 0x55a313bc6000 con 0x55a314773c00 2024-04-20T21:28:48.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.084+0000 7fe8c7800640 1 --2- 172.21.15.86:0/2827900035 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x55a314773c00 0x55a313998c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:48.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.086+0000 7fe8c7800640 1 --2- 172.21.15.86:0/2827900035 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x55a314773c00 0x55a313998c00 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:48.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.086+0000 7fe8c7800640 1 -- 172.21.15.86:0/2827900035 <== osd.4 v2:172.21.15.144:6800/4087309475 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv31 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x55a3145ac500 con 0x55a314773c00 2024-04-20T21:28:48.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.086+0000 7fe8c9650280 10 Header 0x400000 0x405f0e 0x40628a 2024-04-20T21:28:48.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.086+0000 7fe8c9650280 10 Starting journal scan from offset 0x405f0e 2024-04-20T21:28:48.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.086+0000 7fe8c9650280 1 -- 172.21.15.86:0/2827900035 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:3 14.14 14:2e2fa760:::200.00000001:head [read 24334~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101) -- 0x55a313bc6400 con 0x55a314773c00 2024-04-20T21:28:48.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.087+0000 7fe8c7800640 1 -- 172.21.15.86:0/2827900035 <== osd.4 v2:172.21.15.144:6800/4087309475 2 ==== osd_op_reply(3 200.00000001 [read 24334~892 out=892b] v0'0 uv27 ondisk = 0) ==== 156+0+892 (crc 0 0 0) 0x55a3145ac500 con 0x55a314773c00 2024-04-20T21:28:48.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.087+0000 7fe8c9650280 4 Read 0x37c bytes from 200.00000001 gap=0 2024-04-20T21:28:48.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.087+0000 7fe8c9650280 10 Parsing data, 0x37c bytes available 2024-04-20T21:28:48.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.087+0000 7fe8c9650280 10 Attempting decode at 0x405f0e 2024-04-20T21:28:48.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.087+0000 7fe8c9650280 10 Consumed 0x37c bytes 2024-04-20T21:28:48.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.087+0000 7fe8c9650280 10 Valid entry at 0x405f0e 2024-04-20T21:28:48.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.087+0000 7fe8c9650280 1 --2- 172.21.15.86:0/2827900035 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x55a313bc6800 0x55a313999180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:48.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.087+0000 7fe8c9650280 1 -- 172.21.15.86:0/2827900035 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:4 14.1b 14:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101) -- 0x55a313bc6c00 con 0x55a313bc6800 2024-04-20T21:28:48.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.088+0000 7fe8c6e00640 1 --2- 172.21.15.86:0/2827900035 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x55a313bc6800 0x55a313999180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:48.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.088+0000 7fe8c6e00640 1 --2- 172.21.15.86:0/2827900035 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x55a313bc6800 0x55a313999180 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:48.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.089+0000 7fe8c6e00640 1 -- 172.21.15.86:0/2827900035 <== osd.1 v2:172.21.15.86:6817/3149564961 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55a3145ac780 con 0x55a313bc6800 2024-04-20T21:28:48.088 INFO:teuthology.orchestra.run.smithi086.stdout:Events by type: 2024-04-20T21:28:48.088 INFO:teuthology.orchestra.run.smithi086.stdout:Errors: 0 2024-04-20T21:28:48.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.089+0000 7fe8c9650280 4 Reached end of journal objects 2024-04-20T21:28:48.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.089+0000 7fe8c9650280 4 Scanned objects, 0 missing, 1 valid 2024-04-20T21:28:48.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.089+0000 7fe8c9650280 4 Events scanned, 0 gaps 2024-04-20T21:28:48.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.089+0000 7fe8c9650280 4 Found 1 valid events 2024-04-20T21:28:48.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.089+0000 7fe8c9650280 4 Selected 0 events events for processing 2024-04-20T21:28:48.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.089+0000 7fe8c9650280 4 0 events 2024-04-20T21:28:48.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.089+0000 7fe8c9650280 1 -- 172.21.15.86:0/2827900035 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x55a313bc6800 msgr2=0x55a313999180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:48.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.089+0000 7fe8c9650280 1 --2- 172.21.15.86:0/2827900035 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x55a313bc6800 0x55a313999180 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:48.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.089+0000 7fe8c9650280 1 -- 172.21.15.86:0/2827900035 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x55a314773c00 msgr2=0x55a313998c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:48.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.089+0000 7fe8c9650280 1 --2- 172.21.15.86:0/2827900035 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x55a314773c00 0x55a313998c00 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:48.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.089+0000 7fe8c9650280 1 -- 172.21.15.86:0/2827900035 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x55a314773400 msgr2=0x55a313998680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:48.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.089+0000 7fe8c9650280 1 --2- 172.21.15.86:0/2827900035 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x55a314773400 0x55a313998680 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:48.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.089+0000 7fe8c6400640 1 -- 172.21.15.86:0/2827900035 reap_dead start 2024-04-20T21:28:48.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.089+0000 7fe8c9650280 1 -- 172.21.15.86:0/2827900035 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x55a314773000 msgr2=0x55a313998100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:48.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.089+0000 7fe8c9650280 1 --2- 172.21.15.86:0/2827900035 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x55a314773000 0x55a313998100 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x55a313853a70 tx=0x55a31470fb60 comp rx=0 tx=0).stop 2024-04-20T21:28:48.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.089+0000 7fe8c9650280 1 -- 172.21.15.86:0/2827900035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3146f1800 msgr2=0x55a313997600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:48.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.089+0000 7fe8c9650280 1 --2- 172.21.15.86:0/2827900035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3146f1800 0x55a313997600 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x55a314709320 tx=0x55a31470ec00 comp rx=0 tx=0).stop 2024-04-20T21:28:48.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.089+0000 7fe8c9650280 1 -- 172.21.15.86:0/2827900035 shutdown_connections 2024-04-20T21:28:48.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.089+0000 7fe8c9650280 1 --2- 172.21.15.86:0/2827900035 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x55a314773000 0x55a313998100 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:48.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.089+0000 7fe8c9650280 1 --2- 172.21.15.86:0/2827900035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a3146f1800 0x55a313997600 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:48.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.089+0000 7fe8c9650280 1 -- 172.21.15.86:0/2827900035 >> 172.21.15.86:0/2827900035 conn(0x55a3146f1000 msgr2=0x55a313909000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:48.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.089+0000 7fe8c9650280 1 -- 172.21.15.86:0/2827900035 shutdown_connections 2024-04-20T21:28:48.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.089+0000 7fe8c9650280 1 -- 172.21.15.86:0/2827900035 wait complete. 2024-04-20T21:28:48.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.089+0000 7fe8c9650280 1 -- 172.21.15.86:0/3012877264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a313822c00 msgr2=0x55a313996000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:48.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.089+0000 7fe8c9650280 1 --2- 172.21.15.86:0/3012877264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a313822c00 0x55a313996000 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x55a314649470 tx=0x55a3145bc4e0 comp rx=0 tx=0).stop 2024-04-20T21:28:48.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.090+0000 7fe8c9650280 1 -- 172.21.15.86:0/3012877264 shutdown_connections 2024-04-20T21:28:48.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.090+0000 7fe8c9650280 1 --2- 172.21.15.86:0/3012877264 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a313823c00 0x55a313996b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:48.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.090+0000 7fe8c9650280 1 --2- 172.21.15.86:0/3012877264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a313823800 0x55a313996580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:48.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.090+0000 7fe8c9650280 1 --2- 172.21.15.86:0/3012877264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a313822c00 0x55a313996000 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:48.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.090+0000 7fe8c9650280 1 -- 172.21.15.86:0/3012877264 >> 172.21.15.86:0/3012877264 conn(0x55a313823000 msgr2=0x55a313908800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:48.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.090+0000 7fe8c9650280 1 -- 172.21.15.86:0/3012877264 shutdown_connections 2024-04-20T21:28:48.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.090+0000 7fe8c9650280 1 -- 172.21.15.86:0/3012877264 wait complete. 2024-04-20T21:28:48.093 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:28:48.094 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627778', 'summary'] in time 0:00:00.106882, result: Events by type: Errors: 0 2024-04-20T21:28:48.094 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put mds0_inotable - 2024-04-20T21:28:48.141 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:28:48.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.233+0000 7f7cd7f78f40 1 Processor -- start 2024-04-20T21:28:48.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.233+0000 7f7cd7f78f40 1 -- start start 2024-04-20T21:28:48.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.234+0000 7f7cd7f78f40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdc7f308e0 0x55cdc7ff88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:48.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.234+0000 7f7cd7f78f40 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55cdc8001da0 0x55cdc8009260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:48.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.234+0000 7f7cd7f78f40 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cdc8007e20 0x55cdc80097a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:48.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.234+0000 7f7cd7f78f40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cdc8008880 con 0x55cdc7f308e0 2024-04-20T21:28:48.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.234+0000 7f7cd7f78f40 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55cdc7ed7cc0 con 0x55cdc8007e20 2024-04-20T21:28:48.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.234+0000 7f7cd7f78f40 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55cdc8008b20 con 0x55cdc8001da0 2024-04-20T21:28:48.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.234+0000 7f7cd5200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdc7f308e0 0x55cdc7ff88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:48.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.234+0000 7f7cd5200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdc7f308e0 0x55cdc7ff88d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45836/0 (socket says 172.21.15.86:45836) 2024-04-20T21:28:48.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.234+0000 7f7cd5200640 1 -- 172.21.15.86:0/175689340 learned_addr learned my addr 172.21.15.86:0/175689340 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:48.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.235+0000 7f7cd5c00640 1 --2- 172.21.15.86:0/175689340 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cdc8007e20 0x55cdc80097a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:48.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.235+0000 7f7ccfe00640 1 --2- 172.21.15.86:0/175689340 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55cdc8001da0 0x55cdc8009260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:48.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.235+0000 7f7cd5200640 1 -- 172.21.15.86:0/175689340 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55cdc8001da0 msgr2=0x55cdc8009260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:48.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.235+0000 7f7cd5200640 1 --2- 172.21.15.86:0/175689340 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55cdc8001da0 0x55cdc8009260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:48.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.235+0000 7f7cd5200640 1 -- 172.21.15.86:0/175689340 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cdc8007e20 msgr2=0x55cdc80097a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:48.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.235+0000 7f7cd5200640 1 --2- 172.21.15.86:0/175689340 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cdc8007e20 0x55cdc80097a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:48.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.235+0000 7f7cd5200640 1 -- 172.21.15.86:0/175689340 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdc7f2e530 con 0x55cdc7f308e0 2024-04-20T21:28:48.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.235+0000 7f7ccfe00640 1 --2- 172.21.15.86:0/175689340 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55cdc8001da0 0x55cdc8009260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:28:48.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.235+0000 7f7cd5200640 1 --2- 172.21.15.86:0/175689340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdc7f308e0 0x55cdc7ff88d0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f7cc800a610 tx=0x7f7cc8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c79b394fa1359cb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:48.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.235+0000 7f7ccea00640 1 -- 172.21.15.86:0/175689340 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc8008a00 con 0x55cdc7f308e0 2024-04-20T21:28:48.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.235+0000 7f7ccea00640 1 -- 172.21.15.86:0/175689340 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cc8008ba0 con 0x55cdc7f308e0 2024-04-20T21:28:48.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.235+0000 7f7ccea00640 1 -- 172.21.15.86:0/175689340 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc8013440 con 0x55cdc7f308e0 2024-04-20T21:28:48.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.236+0000 7f7cd7f78f40 1 -- 172.21.15.86:0/175689340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdc7f308e0 msgr2=0x55cdc7ff88d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:48.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.236+0000 7f7cd7f78f40 1 --2- 172.21.15.86:0/175689340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdc7f308e0 0x55cdc7ff88d0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f7cc800a610 tx=0x7f7cc8005f90 comp rx=0 tx=0).stop 2024-04-20T21:28:48.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.236+0000 7f7cd7f78f40 1 -- 172.21.15.86:0/175689340 shutdown_connections 2024-04-20T21:28:48.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.236+0000 7f7cd7f78f40 1 --2- 172.21.15.86:0/175689340 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cdc8007e20 0x55cdc80097a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:48.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.236+0000 7f7cd7f78f40 1 --2- 172.21.15.86:0/175689340 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55cdc8001da0 0x55cdc8009260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:48.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.236+0000 7f7cd7f78f40 1 --2- 172.21.15.86:0/175689340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdc7f308e0 0x55cdc7ff88d0 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:48.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.236+0000 7f7cd7f78f40 1 -- 172.21.15.86:0/175689340 >> 172.21.15.86:0/175689340 conn(0x55cdc7f7e250 msgr2=0x55cdc8006db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:48.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.236+0000 7f7cd7f78f40 1 -- 172.21.15.86:0/175689340 shutdown_connections 2024-04-20T21:28:48.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.236+0000 7f7cd7f78f40 1 -- 172.21.15.86:0/175689340 wait complete. 2024-04-20T21:28:48.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.237+0000 7f7cd7f78f40 1 Processor -- start 2024-04-20T21:28:48.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.237+0000 7f7cd7f78f40 1 -- start start 2024-04-20T21:28:48.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.238+0000 7f7cd7f78f40 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cdc7f308e0 0x55cdc7ffce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:48.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.238+0000 7f7cd7f78f40 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55cdc8001da0 0x55cdc7f62260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:48.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.238+0000 7f7cd7f78f40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdc8007e20 0x55cdc7ff62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:48.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.238+0000 7f7cd7f78f40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cdc800ac10 con 0x55cdc8007e20 2024-04-20T21:28:48.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.238+0000 7f7cd7f78f40 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55cdc7f2e7b0 con 0x55cdc7f308e0 2024-04-20T21:28:48.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.238+0000 7f7cd7f78f40 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55cdc7fef5a0 con 0x55cdc8001da0 2024-04-20T21:28:48.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.238+0000 7f7cd5c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdc8007e20 0x55cdc7ff62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:48.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.238+0000 7f7cd5c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdc8007e20 0x55cdc7ff62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45838/0 (socket says 172.21.15.86:45838) 2024-04-20T21:28:48.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.238+0000 7f7cd5c00640 1 -- 172.21.15.86:0/1671726329 learned_addr learned my addr 172.21.15.86:0/1671726329 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:48.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.239+0000 7f7ccfe00640 1 --2- 172.21.15.86:0/1671726329 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55cdc8001da0 0x55cdc7f62260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:48.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.239+0000 7f7cd5200640 1 --2- 172.21.15.86:0/1671726329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cdc7f308e0 0x55cdc7ffce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:48.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.239+0000 7f7cd5c00640 1 -- 172.21.15.86:0/1671726329 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55cdc8001da0 msgr2=0x55cdc7f62260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:48.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.239+0000 7f7cd5c00640 1 --2- 172.21.15.86:0/1671726329 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55cdc8001da0 0x55cdc7f62260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:48.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.239+0000 7f7cd5c00640 1 -- 172.21.15.86:0/1671726329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cdc7f308e0 msgr2=0x55cdc7ffce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:48.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.239+0000 7f7cd5c00640 1 --2- 172.21.15.86:0/1671726329 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55cdc7f308e0 0x55cdc7ffce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:48.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.239+0000 7f7cd5c00640 1 -- 172.21.15.86:0/1671726329 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdc7fefdc0 con 0x55cdc8007e20 2024-04-20T21:28:48.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.239+0000 7f7cd5c00640 1 --2- 172.21.15.86:0/1671726329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdc8007e20 0x55cdc7ff62a0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f7cd000a1c0 tx=0x7f7cd000a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:48.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.239+0000 7f7ccd600640 1 -- 172.21.15.86:0/1671726329 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd0015020 con 0x55cdc8007e20 2024-04-20T21:28:48.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.239+0000 7f7ccd600640 1 -- 172.21.15.86:0/1671726329 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cd0011630 con 0x55cdc8007e20 2024-04-20T21:28:48.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.239+0000 7f7ccd600640 1 -- 172.21.15.86:0/1671726329 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd0011a00 con 0x55cdc8007e20 2024-04-20T21:28:48.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.239+0000 7f7cd7f78f40 1 -- 172.21.15.86:0/1671726329 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55cdc7fef900 con 0x55cdc8007e20 2024-04-20T21:28:48.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.240+0000 7f7cd7f78f40 1 -- 172.21.15.86:0/1671726329 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdc7ffd570 con 0x55cdc8007e20 2024-04-20T21:28:48.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.241+0000 7f7ccd600640 1 -- 172.21.15.86:0/1671726329 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7cd0018020 con 0x55cdc8007e20 2024-04-20T21:28:48.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.242+0000 7f7ccd600640 1 --2- 172.21.15.86:0/1671726329 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7cb406b2d0 0x7f7cb406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:48.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.242+0000 7f7ccd600640 1 -- 172.21.15.86:0/1671726329 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5795+0+0 (secure 0 0 0) 0x7f7cd008a980 con 0x55cdc8007e20 2024-04-20T21:28:48.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.243+0000 7f7cd5200640 1 --2- 172.21.15.86:0/1671726329 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7cb406b2d0 0x7f7cb406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:48.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.243+0000 7f7cd7f78f40 1 --2- 172.21.15.86:0/1671726329 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x55cdc7ff58d0 0x55cdc8002d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:48.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.243+0000 7f7cd7f78f40 1 -- 172.21.15.86:0/1671726329 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:1 14.13 14:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55cdc7ffdb50 con 0x55cdc7ff58d0 2024-04-20T21:28:48.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.243+0000 7f7ccfe00640 1 --2- 172.21.15.86:0/1671726329 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x55cdc7ff58d0 0x55cdc8002d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:48.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.243+0000 7f7cd5200640 1 --2- 172.21.15.86:0/1671726329 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7cb406b2d0 0x7f7cb406d790 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f7cc800c010 tx=0x7f7cc800af90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:48.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.244+0000 7f7ccfe00640 1 --2- 172.21.15.86:0/1671726329 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x55cdc7ff58d0 0x55cdc8002d00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:48.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.249+0000 7f7ccfe00640 1 -- 172.21.15.86:0/1671726329 <== osd.2 v2:172.21.15.86:6816/3437611468 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~34] v101'5 uv5 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7cc004db80 con 0x55cdc7ff58d0 2024-04-20T21:28:48.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.249+0000 7f7cd7f78f40 1 -- 172.21.15.86:0/1671726329 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x55cdc7ff58d0 msgr2=0x55cdc8002d00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:48.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.249+0000 7f7cd7f78f40 1 --2- 172.21.15.86:0/1671726329 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x55cdc7ff58d0 0x55cdc8002d00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:48.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.249+0000 7f7cd7f78f40 1 -- 172.21.15.86:0/1671726329 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7cb406b2d0 msgr2=0x7f7cb406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:48.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.249+0000 7f7cd7f78f40 1 --2- 172.21.15.86:0/1671726329 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7cb406b2d0 0x7f7cb406d790 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f7cc800c010 tx=0x7f7cc800af90 comp rx=0 tx=0).stop 2024-04-20T21:28:48.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.249+0000 7f7cd7f78f40 1 -- 172.21.15.86:0/1671726329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdc8007e20 msgr2=0x55cdc7ff62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:48.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.249+0000 7f7cd7f78f40 1 --2- 172.21.15.86:0/1671726329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdc8007e20 0x55cdc7ff62a0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f7cd000a1c0 tx=0x7f7cd000a680 comp rx=0 tx=0).stop 2024-04-20T21:28:48.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.249+0000 7f7cd5c00640 1 -- 172.21.15.86:0/1671726329 reap_dead start 2024-04-20T21:28:48.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.249+0000 7f7cd7f78f40 1 -- 172.21.15.86:0/1671726329 shutdown_connections 2024-04-20T21:28:48.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.249+0000 7f7cd7f78f40 1 -- 172.21.15.86:0/1671726329 >> 172.21.15.86:0/1671726329 conn(0x55cdc7f7e250 msgr2=0x55cdc8009ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:48.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.249+0000 7f7cd7f78f40 1 -- 172.21.15.86:0/1671726329 shutdown_connections 2024-04-20T21:28:48.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.249+0000 7f7cd7f78f40 1 -- 172.21.15.86:0/1671726329 wait complete. 2024-04-20T21:28:48.252 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-20T21:28:48.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.501+0000 7f9a23000640 1 Processor -- start 2024-04-20T21:28:48.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.502+0000 7f9a23000640 1 -- start start 2024-04-20T21:28:48.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.503+0000 7f9a23000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9a1c15ee00 0x7f9a1c1634c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:48.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.503+0000 7f9a23000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a1c0d1260 0x7f9a1c0d16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:48.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.503+0000 7f9a23000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a1c0cfce0 con 0x7f9a1c0cf830 2024-04-20T21:28:48.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.503+0000 7f9a23000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9a1c0d0550 con 0x7f9a1c15ee00 2024-04-20T21:28:48.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.503+0000 7f9a23000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9a1c0cff90 con 0x7f9a1c0d1260 2024-04-20T21:28:48.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.503+0000 7f9a1be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9a1c15ee00 0x7f9a1c1634c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:48.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.503+0000 7f9a1be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9a1c15ee00 0x7f9a1c1634c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45840/0 (socket says 172.21.15.86:45840) 2024-04-20T21:28:48.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.503+0000 7f9a1be00640 1 -- 172.21.15.86:0/162774864 learned_addr learned my addr 172.21.15.86:0/162774864 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:48.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.504+0000 7f9a1be00640 1 -- 172.21.15.86:0/162774864 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a1c0d1260 msgr2=0x7f9a1c0d16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:48.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.504+0000 7f9a1be00640 1 --2- 172.21.15.86:0/162774864 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a1c0d1260 0x7f9a1c0d16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:48.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.504+0000 7f9a1be00640 1 -- 172.21.15.86:0/162774864 >> v1:172.21.15.144:6789/0 conn(0x7f9a1c0cf830 legacy=0x7f9a1c15fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:48.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.504+0000 7f9a1be00640 1 -- 172.21.15.86:0/162774864 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a1c0d0270 con 0x7f9a1c15ee00 2024-04-20T21:28:48.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.505+0000 7f9a1be00640 1 --2- 172.21.15.86:0/162774864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a1c15ee00 0x7f9a1c1634c0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c0d1eb0 tx=0x7f9a0c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b0d3fd8f981d34f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:48.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.505+0000 7f9a1b400640 1 -- 172.21.15.86:0/162774864 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c035020 con 0x7f9a1c15ee00 2024-04-20T21:28:48.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.505+0000 7f9a1b400640 1 -- 172.21.15.86:0/162774864 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a0c031640 con 0x7f9a1c15ee00 2024-04-20T21:28:48.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.505+0000 7f9a1b400640 1 -- 172.21.15.86:0/162774864 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c02d040 con 0x7f9a1c15ee00 2024-04-20T21:28:48.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.506+0000 7f9a23000640 1 -- 172.21.15.86:0/162774864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a1c15ee00 msgr2=0x7f9a1c1634c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:48.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.506+0000 7f9a23000640 1 --2- 172.21.15.86:0/162774864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a1c15ee00 0x7f9a1c1634c0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c0d1eb0 tx=0x7f9a0c02b8f0 comp rx=0 tx=0).stop 2024-04-20T21:28:48.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.506+0000 7f9a23000640 1 -- 172.21.15.86:0/162774864 shutdown_connections 2024-04-20T21:28:48.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.506+0000 7f9a23000640 1 --2- 172.21.15.86:0/162774864 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a1c0d1260 0x7f9a1c0d16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:48.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.506+0000 7f9a23000640 1 --2- 172.21.15.86:0/162774864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a1c15ee00 0x7f9a1c1634c0 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:48.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.506+0000 7f9a23000640 1 -- 172.21.15.86:0/162774864 >> 172.21.15.86:0/162774864 conn(0x7f9a1c06cc20 msgr2=0x7f9a1c06d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:48.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.506+0000 7f9a23000640 1 -- 172.21.15.86:0/162774864 shutdown_connections 2024-04-20T21:28:48.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.506+0000 7f9a23000640 1 -- 172.21.15.86:0/162774864 wait complete. 2024-04-20T21:28:48.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.508+0000 7f9a23000640 1 Processor -- start 2024-04-20T21:28:48.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.508+0000 7f9a23000640 1 -- start start 2024-04-20T21:28:48.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.509+0000 7f9a23000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a1c0cf830 0x7f9a1c0cdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:48.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.509+0000 7f9a23000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a1c0d1260 0x7f9a1c14ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:48.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.509+0000 7f9a23000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a1c15ee00 0x7f9a1c1f4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:48.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.509+0000 7f9a23000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1c15f360 con 0x7f9a1c0cf830 2024-04-20T21:28:48.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.509+0000 7f9a23000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1c15f1e0 con 0x7f9a1c15ee00 2024-04-20T21:28:48.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.509+0000 7f9a23000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9a1c0d3040 con 0x7f9a1c0d1260 2024-04-20T21:28:48.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.509+0000 7f9a1be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a1c0cf830 0x7f9a1c0cdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:48.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.509+0000 7f9a1be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a1c0cf830 0x7f9a1c0cdfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45854/0 (socket says 172.21.15.86:45854) 2024-04-20T21:28:48.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.509+0000 7f9a1be00640 1 -- 172.21.15.86:0/4250940025 learned_addr learned my addr 172.21.15.86:0/4250940025 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:48.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.509+0000 7f9a21000640 1 --2- 172.21.15.86:0/4250940025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a1c15ee00 0x7f9a1c1f4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:48.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.510+0000 7f9a1be00640 1 -- 172.21.15.86:0/4250940025 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a1c0d1260 msgr2=0x7f9a1c14ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:48.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.510+0000 7f9a13400640 1 --2- 172.21.15.86:0/4250940025 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a1c0d1260 0x7f9a1c14ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:48.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.510+0000 7f9a1be00640 1 --2- 172.21.15.86:0/4250940025 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a1c0d1260 0x7f9a1c14ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:48.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.510+0000 7f9a1be00640 1 -- 172.21.15.86:0/4250940025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a1c15ee00 msgr2=0x7f9a1c1f4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:48.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.510+0000 7f9a1be00640 1 --2- 172.21.15.86:0/4250940025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a1c15ee00 0x7f9a1c1f4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:48.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.510+0000 7f9a1be00640 1 -- 172.21.15.86:0/4250940025 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a1c1f53b0 con 0x7f9a1c0cf830 2024-04-20T21:28:48.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.510+0000 7f9a21000640 1 --2- 172.21.15.86:0/4250940025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a1c15ee00 0x7f9a1c1f4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:28:48.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.510+0000 7f9a1be00640 1 --2- 172.21.15.86:0/4250940025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a1c0cf830 0x7f9a1c0cdfa0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c000c00 tx=0x7f9a0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:48.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.510+0000 7f9a18c00640 1 -- 172.21.15.86:0/4250940025 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c03ac60 con 0x7f9a1c0cf830 2024-04-20T21:28:48.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.510+0000 7f9a18c00640 1 -- 172.21.15.86:0/4250940025 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a0c03ae40 con 0x7f9a1c0cf830 2024-04-20T21:28:48.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.510+0000 7f9a18c00640 1 -- 172.21.15.86:0/4250940025 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c044630 con 0x7f9a1c0cf830 2024-04-20T21:28:48.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.510+0000 7f9a23000640 1 -- 172.21.15.86:0/4250940025 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a1c1f5690 con 0x7f9a1c0cf830 2024-04-20T21:28:48.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.511+0000 7f9a23000640 1 -- 172.21.15.86:0/4250940025 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a1c1f5fa0 con 0x7f9a1c0cf830 2024-04-20T21:28:48.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.512+0000 7f9a23000640 1 -- 172.21.15.86:0/4250940025 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a1c0689e0 con 0x7f9a1c0cf830 2024-04-20T21:28:48.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.513+0000 7f9a18c00640 1 -- 172.21.15.86:0/4250940025 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9a0c04b060 con 0x7f9a1c0cf830 2024-04-20T21:28:48.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.513+0000 7f9a18c00640 1 --2- 172.21.15.86:0/4250940025 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f99f806c8f0 0x7f99f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:48.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.514+0000 7f9a18c00640 1 -- 172.21.15.86:0/4250940025 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5795+0+0 (secure 0 0 0) 0x7f9a0c0ad9b0 con 0x7f9a1c0cf830 2024-04-20T21:28:48.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.514+0000 7f9a13400640 1 --2- 172.21.15.86:0/4250940025 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f99f806c8f0 0x7f99f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:48.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.514+0000 7f9a13400640 1 --2- 172.21.15.86:0/4250940025 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f99f806c8f0 0x7f99f806edb0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f9a04007fd0 tx=0x7f9a04009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:48.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.518+0000 7f9a18c00640 1 -- 172.21.15.86:0/4250940025 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9a0c03f070 con 0x7f9a1c0cf830 2024-04-20T21:28:48.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.764+0000 7f9a23000640 1 -- 172.21.15.86:0/4250940025 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f9a1c0d29c0 con 0x7f9a1c0cf830 2024-04-20T21:28:48.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.849+0000 7f9a18c00640 1 -- 172.21.15.86:0/4250940025 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v84) ==== 162+0+0 (secure 0 0 0) 0x7f9a0c031610 con 0x7f9a1c0cf830 2024-04-20T21:28:48.848 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-20T21:28:48.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.852+0000 7f9a23000640 1 -- 172.21.15.86:0/4250940025 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f99f806c8f0 msgr2=0x7f99f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:48.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.852+0000 7f9a23000640 1 --2- 172.21.15.86:0/4250940025 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f99f806c8f0 0x7f99f806edb0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f9a04007fd0 tx=0x7f9a04009040 comp rx=0 tx=0).stop 2024-04-20T21:28:48.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.852+0000 7f9a23000640 1 -- 172.21.15.86:0/4250940025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a1c0cf830 msgr2=0x7f9a1c0cdfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:48.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.852+0000 7f9a23000640 1 --2- 172.21.15.86:0/4250940025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a1c0cf830 0x7f9a1c0cdfa0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c000c00 tx=0x7f9a0c0046c0 comp rx=0 tx=0).stop 2024-04-20T21:28:48.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.852+0000 7f9a23000640 1 -- 172.21.15.86:0/4250940025 shutdown_connections 2024-04-20T21:28:48.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.852+0000 7f9a23000640 1 --2- 172.21.15.86:0/4250940025 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f99f806c8f0 0x7f99f806edb0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:48.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.852+0000 7f9a23000640 1 --2- 172.21.15.86:0/4250940025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a1c15ee00 0x7f9a1c1f4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:48.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.852+0000 7f9a23000640 1 --2- 172.21.15.86:0/4250940025 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a1c0d1260 0x7f9a1c14ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:48.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.852+0000 7f9a23000640 1 --2- 172.21.15.86:0/4250940025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a1c0cf830 0x7f9a1c0cdfa0 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:48.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.852+0000 7f9a23000640 1 -- 172.21.15.86:0/4250940025 >> 172.21.15.86:0/4250940025 conn(0x7f9a1c06cc20 msgr2=0x7f9a1c0ceb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:48.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.853+0000 7f9a23000640 1 -- 172.21.15.86:0/4250940025 shutdown_connections 2024-04-20T21:28:48.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:48.853+0000 7f9a23000640 1 -- 172.21.15.86:0/4250940025 wait complete. 2024-04-20T21:28:48.878 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:28:49.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.054+0000 7f302c000640 1 Processor -- start 2024-04-20T21:28:49.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.054+0000 7f302c000640 1 -- start start 2024-04-20T21:28:49.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.055+0000 7f302c000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3024144e90 0x7f30240b4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:49.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.055+0000 7f302c000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f30240cd170 0x7f302406ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:49.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.055+0000 7f302c000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30240cdb50 con 0x7f30240cd5d0 2024-04-20T21:28:49.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.055+0000 7f302c000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f3024145270 con 0x7f3024144e90 2024-04-20T21:28:49.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.055+0000 7f302c000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f30240b3e50 con 0x7f30240cd170 2024-04-20T21:28:49.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.056+0000 7f3029600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3024144e90 0x7f30240b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:49.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.056+0000 7f3028c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f30240cd170 0x7f302406ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:49.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.056+0000 7f3029600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3024144e90 0x7f30240b4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.86:49624/0 (socket says 172.21.15.86:49624) 2024-04-20T21:28:49.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.056+0000 7f3029600640 1 -- 172.21.15.86:0/2382093672 learned_addr learned my addr 172.21.15.86:0/2382093672 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:49.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.057+0000 7f3029600640 1 -- 172.21.15.86:0/2382093672 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f30240cd170 msgr2=0x7f302406ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:49.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.057+0000 7f3029600640 1 --2- 172.21.15.86:0/2382093672 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f30240cd170 0x7f302406ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:49.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.057+0000 7f3029600640 1 -- 172.21.15.86:0/2382093672 >> v1:172.21.15.144:6789/0 conn(0x7f30240cd5d0 legacy=0x7f302415bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:49.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.057+0000 7f3029600640 1 -- 172.21.15.86:0/2382093672 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30240b4130 con 0x7f3024144e90 2024-04-20T21:28:49.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.058+0000 7f3029600640 1 --2- 172.21.15.86:0/2382093672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3024144e90 0x7f30240b4930 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3014009870 tx=0x7f301402db00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbe5421ca11dc7e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:49.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.059+0000 7f3023e00640 1 -- 172.21.15.86:0/2382093672 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301402f8c0 con 0x7f3024144e90 2024-04-20T21:28:49.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.059+0000 7f3023e00640 1 -- 172.21.15.86:0/2382093672 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f301402fa60 con 0x7f3024144e90 2024-04-20T21:28:49.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.059+0000 7f3023e00640 1 -- 172.21.15.86:0/2382093672 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301402fe30 con 0x7f3024144e90 2024-04-20T21:28:49.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.060+0000 7f302c000640 1 -- 172.21.15.86:0/2382093672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3024144e90 msgr2=0x7f30240b4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:49.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.060+0000 7f302c000640 1 --2- 172.21.15.86:0/2382093672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3024144e90 0x7f30240b4930 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3014009870 tx=0x7f301402db00 comp rx=0 tx=0).stop 2024-04-20T21:28:49.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.060+0000 7f302c000640 1 -- 172.21.15.86:0/2382093672 shutdown_connections 2024-04-20T21:28:49.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.060+0000 7f302c000640 1 --2- 172.21.15.86:0/2382093672 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f30240cd170 0x7f302406ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:49.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.060+0000 7f302c000640 1 --2- 172.21.15.86:0/2382093672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3024144e90 0x7f30240b4930 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:49.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.060+0000 7f302c000640 1 -- 172.21.15.86:0/2382093672 >> 172.21.15.86:0/2382093672 conn(0x7f30240b2360 msgr2=0x7f30240b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:49.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.060+0000 7f302c000640 1 -- 172.21.15.86:0/2382093672 shutdown_connections 2024-04-20T21:28:49.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.060+0000 7f302c000640 1 -- 172.21.15.86:0/2382093672 wait complete. 2024-04-20T21:28:49.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.062+0000 7f302c000640 1 Processor -- start 2024-04-20T21:28:49.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.062+0000 7f302c000640 1 -- start start 2024-04-20T21:28:49.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.063+0000 7f302c000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30240cd170 0x7f3024134990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:49.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.063+0000 7f302c000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30240cd5d0 0x7f302415a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:49.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.063+0000 7f302c000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3024144e90 0x7f302415aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:49.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.063+0000 7f302c000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f30240d0010 con 0x7f30240cd5d0 2024-04-20T21:28:49.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.063+0000 7f302c000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f30240b4e70 con 0x7f30240cd170 2024-04-20T21:28:49.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.063+0000 7f302c000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f30240d0190 con 0x7f3024144e90 2024-04-20T21:28:49.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.063+0000 7f3028c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30240cd5d0 0x7f302415a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:49.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.064+0000 7f3028c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30240cd5d0 0x7f302415a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45866/0 (socket says 172.21.15.86:45866) 2024-04-20T21:28:49.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.064+0000 7f3028c00640 1 -- 172.21.15.86:0/517699362 learned_addr learned my addr 172.21.15.86:0/517699362 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:49.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.064+0000 7f3029600640 1 --2- 172.21.15.86:0/517699362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30240cd170 0x7f3024134990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:49.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.064+0000 7f3028c00640 1 -- 172.21.15.86:0/517699362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3024144e90 msgr2=0x7f302415aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:49.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.064+0000 7f3028c00640 1 --2- 172.21.15.86:0/517699362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3024144e90 0x7f302415aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:49.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.064+0000 7f3028c00640 1 -- 172.21.15.86:0/517699362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30240cd170 msgr2=0x7f3024134990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:49.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.064+0000 7f3028c00640 1 --2- 172.21.15.86:0/517699362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30240cd170 0x7f3024134990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:49.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.064+0000 7f3028c00640 1 -- 172.21.15.86:0/517699362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30241e3d00 con 0x7f30240cd5d0 2024-04-20T21:28:49.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.064+0000 7f3028c00640 1 --2- 172.21.15.86:0/517699362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30240cd5d0 0x7f302415a540 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f301800efc0 tx=0x7f301800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:49.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.065+0000 7f3021600640 1 -- 172.21.15.86:0/517699362 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3018018020 con 0x7f30240cd5d0 2024-04-20T21:28:49.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.065+0000 7f3021600640 1 -- 172.21.15.86:0/517699362 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3018014480 con 0x7f30240cd5d0 2024-04-20T21:28:49.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.065+0000 7f302c000640 1 -- 172.21.15.86:0/517699362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30241e3fe0 con 0x7f30240cd5d0 2024-04-20T21:28:49.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.065+0000 7f3021600640 1 -- 172.21.15.86:0/517699362 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3018014870 con 0x7f30240cd5d0 2024-04-20T21:28:49.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.065+0000 7f302c000640 1 -- 172.21.15.86:0/517699362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30241e45a0 con 0x7f30240cd5d0 2024-04-20T21:28:49.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.066+0000 7f302c000640 1 -- 172.21.15.86:0/517699362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30240b6750 con 0x7f30240cd5d0 2024-04-20T21:28:49.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.067+0000 7f3021600640 1 -- 172.21.15.86:0/517699362 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3018005d10 con 0x7f30240cd5d0 2024-04-20T21:28:49.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.069+0000 7f3021600640 1 --2- 172.21.15.86:0/517699362 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f300806c8f0 0x7f300806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:49.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.069+0000 7f3021600640 1 -- 172.21.15.86:0/517699362 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5795+0+0 (secure 0 0 0) 0x7f301808c820 con 0x7f30240cd5d0 2024-04-20T21:28:49.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.069+0000 7f3029600640 1 --2- 172.21.15.86:0/517699362 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f300806c8f0 0x7f300806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:49.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.070+0000 7f3029600640 1 --2- 172.21.15.86:0/517699362 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f300806c8f0 0x7f300806edb0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f3014038000 tx=0x7f3014008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:49.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.073+0000 7f3021600640 1 -- 172.21.15.86:0/517699362 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f30240b6750 con 0x7f30240cd5d0 2024-04-20T21:28:49.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.320+0000 7f302c000640 1 -- 172.21.15.86:0/517699362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30240689e0 con 0x7f30240cd5d0 2024-04-20T21:28:49.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.321+0000 7f3021600640 1 -- 172.21.15.86:0/517699362 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+5202 (secure 0 0 0) 0x7f3018022070 con 0x7f30240cd5d0 2024-04-20T21:28:49.320 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:28:49.320 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":85,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5525,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3902778887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3902778887},{"type":"v1","addr":"172.21.15.144:6835","nonce":3902778887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6001,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/4270988014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":4270988014},{"type":"v1","addr":"172.21.15.86:6837","nonce":4270988014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6016,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/1569462554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":1569462554},{"type":"v1","addr":"172.21.15.144:6833","nonce":1569462554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:27:45.692408+0000","modified":"2024-04-20T21:28:48.854764+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":101,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8922},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8922":{"gid":8922,"name":"c","rank":0,"incarnation":85,"state":"up:replay","state_seq":1,"addr":"172.21.15.86:6836/4271041811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4271041811},{"type":"v1","addr":"172.21.15.86:6836","nonce":4271041811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-04-20T21:28:49.320 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 85 2024-04-20T21:28:49.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.323+0000 7f302c000640 1 -- 172.21.15.86:0/517699362 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f300806c8f0 msgr2=0x7f300806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:49.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.323+0000 7f302c000640 1 --2- 172.21.15.86:0/517699362 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f300806c8f0 0x7f300806edb0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f3014038000 tx=0x7f3014008030 comp rx=0 tx=0).stop 2024-04-20T21:28:49.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.323+0000 7f302c000640 1 -- 172.21.15.86:0/517699362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30240cd5d0 msgr2=0x7f302415a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:49.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.323+0000 7f302c000640 1 --2- 172.21.15.86:0/517699362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30240cd5d0 0x7f302415a540 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f301800efc0 tx=0x7f301800c490 comp rx=0 tx=0).stop 2024-04-20T21:28:49.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.323+0000 7f302c000640 1 -- 172.21.15.86:0/517699362 shutdown_connections 2024-04-20T21:28:49.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.323+0000 7f302c000640 1 --2- 172.21.15.86:0/517699362 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f300806c8f0 0x7f300806edb0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:49.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.323+0000 7f302c000640 1 --2- 172.21.15.86:0/517699362 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3024144e90 0x7f302415aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:49.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.323+0000 7f302c000640 1 --2- 172.21.15.86:0/517699362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30240cd5d0 0x7f302415a540 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:49.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.323+0000 7f302c000640 1 --2- 172.21.15.86:0/517699362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30240cd170 0x7f3024134990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:49.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.323+0000 7f302c000640 1 -- 172.21.15.86:0/517699362 >> 172.21.15.86:0/517699362 conn(0x7f30240b2360 msgr2=0x7f3024142f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:49.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.324+0000 7f302c000640 1 -- 172.21.15.86:0/517699362 shutdown_connections 2024-04-20T21:28:49.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:49.324+0000 7f302c000640 1 -- 172.21.15.86:0/517699362 wait complete. 2024-04-20T21:28:49.744 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 85, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T21:27:45.692408+0000', 'modified': '2024-04-20T21:28:48.854764+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 101, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8922}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8922': {'gid': 8922, 'name': 'c', 'rank': 0, 'incarnation': 85, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.86:6836/4271041811', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6834', 'nonce': 4271041811}, {'type': 'v1', 'addr': '172.21.15.86:6836', 'nonce': 4271041811}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-20T21:28:49.744 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8922:up:replay 2024-04-20T21:28:50.744 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:28:50.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.947+0000 7fbd3f200640 1 Processor -- start 2024-04-20T21:28:50.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.947+0000 7fbd3f200640 1 -- start start 2024-04-20T21:28:50.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.949+0000 7fbd3f200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd3815c010 0x7fbd3815c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:50.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.949+0000 7fbd3f200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd380ae710 con 0x7fbd380af7b0 2024-04-20T21:28:50.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.949+0000 7fbd3f200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd380ae9f0 con 0x7fbd3806f4c0 2024-04-20T21:28:50.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.949+0000 7fbd3f200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbd380af320 con 0x7fbd3815c010 2024-04-20T21:28:50.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.949+0000 7fbd3d400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fbd380af7b0 0x7fbd380adda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:45894/0 (socket says 172.21.15.86:45894) 2024-04-20T21:28:50.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.949+0000 7fbd3d400640 1 -- 172.21.15.86:0/88277327 learned_addr learned my addr 172.21.15.86:0/88277327 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:28:50.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.950+0000 7fbd3de00640 1 --2- 172.21.15.86:0/88277327 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd3815c010 0x7fbd3815c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:50.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.950+0000 7fbd3ca00640 1 -- 172.21.15.86:0/88277327 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3156931049 0 0) 0x7fbd380ae710 con 0x7fbd380af7b0 2024-04-20T21:28:50.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.950+0000 7fbd3ca00640 1 -- 172.21.15.86:0/88277327 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd20005730 con 0x7fbd380af7b0 2024-04-20T21:28:50.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.950+0000 7fbd3ca00640 1 -- 172.21.15.86:0/88277327 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1404947226 0 0) 0x7fbd20005730 con 0x7fbd380af7b0 2024-04-20T21:28:50.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.950+0000 7fbd3ca00640 1 -- 172.21.15.86:0/88277327 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd3815c010 msgr2=0x7fbd3815c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:50.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.950+0000 7fbd3ca00640 1 --2- 172.21.15.86:0/88277327 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd3815c010 0x7fbd3815c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:50.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.950+0000 7fbd3ca00640 1 -- 172.21.15.86:0/88277327 >> v1:172.21.15.144:6789/0 conn(0x7fbd3806f4c0 legacy=0x7fbd38071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:50.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.950+0000 7fbd3ca00640 1 -- 172.21.15.86:0/88277327 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd380aecd0 con 0x7fbd380af7b0 2024-04-20T21:28:50.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.950+0000 7fbd3ca00640 1 -- 172.21.15.86:0/88277327 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fbd28001670 con 0x7fbd380af7b0 2024-04-20T21:28:50.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.950+0000 7fbd3ca00640 1 -- 172.21.15.86:0/88277327 >> v1:172.21.15.86:6789/0 conn(0x7fbd380af7b0 legacy=0x7fbd380adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:50.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.950+0000 7fbd3ca00640 1 --2- 172.21.15.86:0/88277327 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd3815c010 0x7fbd20005bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:50.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.950+0000 7fbd3ca00640 1 --2- 172.21.15.86:0/88277327 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd20006170 0x7fbd20006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:50.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.950+0000 7fbd3ca00640 1 --2- 172.21.15.86:0/88277327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd20006f50 0x7fbd200073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:50.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.950+0000 7fbd3ca00640 1 -- 172.21.15.86:0/88277327 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbd380af320 con 0x7fbd20006f50 2024-04-20T21:28:50.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.950+0000 7fbd3ca00640 1 -- 172.21.15.86:0/88277327 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2000dcb0 con 0x7fbd20006170 2024-04-20T21:28:50.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.950+0000 7fbd3ca00640 1 -- 172.21.15.86:0/88277327 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbd2000deb0 con 0x7fbd3815c010 2024-04-20T21:28:50.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.951+0000 7fbd3e800640 1 --2- 172.21.15.86:0/88277327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd20006f50 0x7fbd200073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:50.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.951+0000 7fbd3de00640 1 --2- 172.21.15.86:0/88277327 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd3815c010 0x7fbd20005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:50.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.951+0000 7fbd3e800640 1 -- 172.21.15.86:0/88277327 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd3815c010 msgr2=0x7fbd20005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:50.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.951+0000 7fbd3e800640 1 --2- 172.21.15.86:0/88277327 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd3815c010 0x7fbd20005bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:50.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.951+0000 7fbd3e800640 1 -- 172.21.15.86:0/88277327 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd20006170 msgr2=0x7fbd20006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:50.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.951+0000 7fbd3d400640 1 --2- 172.21.15.86:0/88277327 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd20006170 0x7fbd20006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:50.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.951+0000 7fbd3e800640 1 --2- 172.21.15.86:0/88277327 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd20006170 0x7fbd20006a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:50.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.951+0000 7fbd3e800640 1 -- 172.21.15.86:0/88277327 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd20005730 con 0x7fbd20006f50 2024-04-20T21:28:50.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.952+0000 7fbd3e800640 1 --2- 172.21.15.86:0/88277327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd20006f50 0x7fbd200073d0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fbd34001e50 tx=0x7fbd20007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c12d1bcfec488f1e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:50.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.952+0000 7fbd3ca00640 1 -- 172.21.15.86:0/88277327 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd34001a70 con 0x7fbd20006f50 2024-04-20T21:28:50.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.952+0000 7fbd3ca00640 1 -- 172.21.15.86:0/88277327 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd34003b70 con 0x7fbd20006f50 2024-04-20T21:28:50.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.952+0000 7fbd3ca00640 1 -- 172.21.15.86:0/88277327 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd340050d0 con 0x7fbd20006f50 2024-04-20T21:28:50.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.953+0000 7fbd3f200640 1 -- 172.21.15.86:0/88277327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd20006f50 msgr2=0x7fbd200073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:50.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.953+0000 7fbd3f200640 1 --2- 172.21.15.86:0/88277327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd20006f50 0x7fbd200073d0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fbd34001e50 tx=0x7fbd20007a40 comp rx=0 tx=0).stop 2024-04-20T21:28:50.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.953+0000 7fbd3e800640 1 -- 172.21.15.86:0/88277327 reap_dead start 2024-04-20T21:28:50.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.953+0000 7fbd3f200640 1 -- 172.21.15.86:0/88277327 shutdown_connections 2024-04-20T21:28:50.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.953+0000 7fbd3f200640 1 -- 172.21.15.86:0/88277327 >> 172.21.15.86:0/88277327 conn(0x7fbd380d0170 msgr2=0x7fbd380d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:50.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.953+0000 7fbd3f200640 1 -- 172.21.15.86:0/88277327 shutdown_connections 2024-04-20T21:28:50.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.953+0000 7fbd3f200640 1 -- 172.21.15.86:0/88277327 wait complete. 2024-04-20T21:28:50.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.955+0000 7fbd3f200640 1 Processor -- start 2024-04-20T21:28:50.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.955+0000 7fbd3f200640 1 -- start start 2024-04-20T21:28:50.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.956+0000 7fbd3f200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd20006f50 0x7fbd38099200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:50.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.956+0000 7fbd3f200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd38154d60 0x7fbd381f0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:50.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.956+0000 7fbd3f200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd38151ce0 0x7fbd38159830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:50.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.956+0000 7fbd3f200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3806f970 con 0x7fbd20006f50 2024-04-20T21:28:50.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.956+0000 7fbd3f200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbd380ae4b0 con 0x7fbd38151ce0 2024-04-20T21:28:50.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.956+0000 7fbd3f200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbd3806faf0 con 0x7fbd38154d60 2024-04-20T21:28:50.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.956+0000 7fbd3de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd20006f50 0x7fbd38099200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:50.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.956+0000 7fbd3de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd20006f50 0x7fbd38099200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45898/0 (socket says 172.21.15.86:45898) 2024-04-20T21:28:50.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.956+0000 7fbd3de00640 1 -- 172.21.15.86:0/2812996151 learned_addr learned my addr 172.21.15.86:0/2812996151 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:50.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.956+0000 7fbd3de00640 1 -- 172.21.15.86:0/2812996151 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd38154d60 msgr2=0x7fbd381f0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:50.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.956+0000 7fbd3de00640 1 --2- 172.21.15.86:0/2812996151 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd38154d60 0x7fbd381f0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:50.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.956+0000 7fbd3de00640 1 -- 172.21.15.86:0/2812996151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd38151ce0 msgr2=0x7fbd38159830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:50.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.956+0000 7fbd3e800640 1 --2- 172.21.15.86:0/2812996151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd38151ce0 0x7fbd38159830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:50.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.957+0000 7fbd3de00640 1 --2- 172.21.15.86:0/2812996151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd38151ce0 0x7fbd38159830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:50.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.957+0000 7fbd3de00640 1 -- 172.21.15.86:0/2812996151 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd38152790 con 0x7fbd20006f50 2024-04-20T21:28:50.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.957+0000 7fbd3de00640 1 --2- 172.21.15.86:0/2812996151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd20006f50 0x7fbd38099200 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c011630 tx=0x7fbd2c011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:50.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.957+0000 7fbd32000640 1 -- 172.21.15.86:0/2812996151 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c00f8c0 con 0x7fbd20006f50 2024-04-20T21:28:50.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.957+0000 7fbd32000640 1 -- 172.21.15.86:0/2812996151 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd2c00fa60 con 0x7fbd20006f50 2024-04-20T21:28:50.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.957+0000 7fbd32000640 1 -- 172.21.15.86:0/2812996151 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c00fe10 con 0x7fbd20006f50 2024-04-20T21:28:50.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.957+0000 7fbd3f200640 1 -- 172.21.15.86:0/2812996151 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd381f1180 con 0x7fbd20006f50 2024-04-20T21:28:50.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.957+0000 7fbd3f200640 1 -- 172.21.15.86:0/2812996151 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd381f0440 con 0x7fbd20006f50 2024-04-20T21:28:50.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.959+0000 7fbd3f200640 1 -- 172.21.15.86:0/2812996151 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd380ad0e0 con 0x7fbd20006f50 2024-04-20T21:28:50.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.960+0000 7fbd32000640 1 -- 172.21.15.86:0/2812996151 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbd2c015460 con 0x7fbd20006f50 2024-04-20T21:28:50.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.960+0000 7fbd32000640 1 --2- 172.21.15.86:0/2812996151 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbd1c06c920 0x7fbd1c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:50.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.961+0000 7fbd32000640 1 -- 172.21.15.86:0/2812996151 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5795+0+0 (secure 0 0 0) 0x7fbd2c08c500 con 0x7fbd20006f50 2024-04-20T21:28:50.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.961+0000 7fbd3d400640 1 --2- 172.21.15.86:0/2812996151 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbd1c06c920 0x7fbd1c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:50.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.961+0000 7fbd3d400640 1 --2- 172.21.15.86:0/2812996151 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbd1c06c920 0x7fbd1c06ede0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fbd380afe70 tx=0x7fbd28009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:50.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:50.965+0000 7fbd32000640 1 -- 172.21.15.86:0/2812996151 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbd380ad0e0 con 0x7fbd20006f50 2024-04-20T21:28:51.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:51.204+0000 7fbd3f200640 1 -- 172.21.15.86:0/2812996151 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd381f0240 con 0x7fbd20006f50 2024-04-20T21:28:51.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:51.205+0000 7fbd32000640 1 -- 172.21.15.86:0/2812996151 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) ==== 94+0+5202 (secure 0 0 0) 0x7fbd2c019030 con 0x7fbd20006f50 2024-04-20T21:28:51.206 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:28:51.206 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":87,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5525,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3902778887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3902778887},{"type":"v1","addr":"172.21.15.144:6835","nonce":3902778887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6001,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/4270988014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":4270988014},{"type":"v1","addr":"172.21.15.86:6837","nonce":4270988014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6016,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/1569462554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":1569462554},{"type":"v1","addr":"172.21.15.144:6833","nonce":1569462554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:27:45.692408+0000","modified":"2024-04-20T21:28:49.874279+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":101,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8922},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8922":{"gid":8922,"name":"c","rank":0,"incarnation":85,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.86:6836/4271041811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4271041811},{"type":"v1","addr":"172.21.15.86:6836","nonce":4271041811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-04-20T21:28:51.206 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 87 2024-04-20T21:28:51.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:51.208+0000 7fbd3f200640 1 -- 172.21.15.86:0/2812996151 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbd1c06c920 msgr2=0x7fbd1c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:51.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:51.208+0000 7fbd3f200640 1 --2- 172.21.15.86:0/2812996151 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbd1c06c920 0x7fbd1c06ede0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fbd380afe70 tx=0x7fbd28009040 comp rx=0 tx=0).stop 2024-04-20T21:28:51.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:51.208+0000 7fbd3f200640 1 -- 172.21.15.86:0/2812996151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd20006f50 msgr2=0x7fbd38099200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:51.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:51.208+0000 7fbd3f200640 1 --2- 172.21.15.86:0/2812996151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd20006f50 0x7fbd38099200 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c011630 tx=0x7fbd2c011af0 comp rx=0 tx=0).stop 2024-04-20T21:28:51.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:51.208+0000 7fbd3f200640 1 -- 172.21.15.86:0/2812996151 shutdown_connections 2024-04-20T21:28:51.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:51.208+0000 7fbd3f200640 1 --2- 172.21.15.86:0/2812996151 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbd1c06c920 0x7fbd1c06ede0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:51.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:51.208+0000 7fbd3f200640 1 --2- 172.21.15.86:0/2812996151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd38151ce0 0x7fbd38159830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:51.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:51.208+0000 7fbd3f200640 1 --2- 172.21.15.86:0/2812996151 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbd38154d60 0x7fbd381f0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:51.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:51.208+0000 7fbd3f200640 1 --2- 172.21.15.86:0/2812996151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd20006f50 0x7fbd38099200 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:51.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:51.208+0000 7fbd3f200640 1 -- 172.21.15.86:0/2812996151 >> 172.21.15.86:0/2812996151 conn(0x7fbd380d0170 msgr2=0x7fbd3815e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:51.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:51.208+0000 7fbd3f200640 1 -- 172.21.15.86:0/2812996151 shutdown_connections 2024-04-20T21:28:51.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:51.208+0000 7fbd3f200640 1 -- 172.21.15.86:0/2812996151 wait complete. 2024-04-20T21:28:51.629 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 87, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T21:27:45.692408+0000', 'modified': '2024-04-20T21:28:49.874279+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 101, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8922}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8922': {'gid': 8922, 'name': 'c', 'rank': 0, 'incarnation': 85, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.86:6836/4271041811', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6834', 'nonce': 4271041811}, {'type': 'v1', 'addr': '172.21.15.86:6836', 'nonce': 4271041811}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-20T21:28:51.629 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8922:up:rejoin 2024-04-20T21:28:52.630 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:28:52.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.842+0000 7f7a63e00640 1 Processor -- start 2024-04-20T21:28:52.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.842+0000 7f7a63e00640 1 -- start start 2024-04-20T21:28:52.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.843+0000 7f7a63e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a5c144e90 0x7f7a5c15ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:52.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.843+0000 7f7a63e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7a5c071d00 0x7f7a5c072160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:52.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.843+0000 7f7a63e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a5c06dc30 con 0x7f7a5c06f140 2024-04-20T21:28:52.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.843+0000 7f7a63e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f7a5c0726a0 con 0x7f7a5c071d00 2024-04-20T21:28:52.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.843+0000 7f7a63e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7a5c145270 con 0x7f7a5c144e90 2024-04-20T21:28:52.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.844+0000 7f7a61e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7a5c071d00 0x7f7a5c072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:52.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.844+0000 7f7a61e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7a5c071d00 0x7f7a5c072160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45910/0 (socket says 172.21.15.86:45910) 2024-04-20T21:28:52.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.844+0000 7f7a61e00640 1 -- 172.21.15.86:0/2426747255 learned_addr learned my addr 172.21.15.86:0/2426747255 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:52.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.844+0000 7f7a61400640 1 --2- 172.21.15.86:0/2426747255 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a5c144e90 0x7f7a5c15ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:52.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.844+0000 7f7a61e00640 1 -- 172.21.15.86:0/2426747255 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a5c144e90 msgr2=0x7f7a5c15ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:52.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.844+0000 7f7a61e00640 1 --2- 172.21.15.86:0/2426747255 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a5c144e90 0x7f7a5c15ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:52.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.844+0000 7f7a61e00640 1 -- 172.21.15.86:0/2426747255 >> v1:172.21.15.144:6789/0 conn(0x7f7a5c06f140 legacy=0x7f7a5c0aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:52.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.844+0000 7f7a61e00640 1 -- 172.21.15.86:0/2426747255 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a5c15f720 con 0x7f7a5c071d00 2024-04-20T21:28:52.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.845+0000 7f7a61e00640 1 --2- 172.21.15.86:0/2426747255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a5c071d00 0x7f7a5c072160 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c1453f0 tx=0x7f7a5402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b42fdb0012e91404 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:52.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.846+0000 7f7a5be00640 1 -- 172.21.15.86:0/2426747255 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5402ca50 con 0x7f7a5c071d00 2024-04-20T21:28:52.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.846+0000 7f7a5be00640 1 -- 172.21.15.86:0/2426747255 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a5402cbf0 con 0x7f7a5c071d00 2024-04-20T21:28:52.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.846+0000 7f7a5be00640 1 -- 172.21.15.86:0/2426747255 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a540343f0 con 0x7f7a5c071d00 2024-04-20T21:28:52.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.846+0000 7f7a63e00640 1 -- 172.21.15.86:0/2426747255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a5c071d00 msgr2=0x7f7a5c072160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:52.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.846+0000 7f7a63e00640 1 --2- 172.21.15.86:0/2426747255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a5c071d00 0x7f7a5c072160 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c1453f0 tx=0x7f7a5402beb0 comp rx=0 tx=0).stop 2024-04-20T21:28:52.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.847+0000 7f7a63e00640 1 -- 172.21.15.86:0/2426747255 shutdown_connections 2024-04-20T21:28:52.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.847+0000 7f7a63e00640 1 --2- 172.21.15.86:0/2426747255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a5c071d00 0x7f7a5c072160 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:52.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.847+0000 7f7a63e00640 1 --2- 172.21.15.86:0/2426747255 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a5c144e90 0x7f7a5c15ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:52.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.847+0000 7f7a63e00640 1 -- 172.21.15.86:0/2426747255 >> 172.21.15.86:0/2426747255 conn(0x7f7a5c0b2360 msgr2=0x7f7a5c0b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:52.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.847+0000 7f7a63e00640 1 -- 172.21.15.86:0/2426747255 shutdown_connections 2024-04-20T21:28:52.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.847+0000 7f7a63e00640 1 -- 172.21.15.86:0/2426747255 wait complete. 2024-04-20T21:28:52.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.848+0000 7f7a63e00640 1 Processor -- start 2024-04-20T21:28:52.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.848+0000 7f7a63e00640 1 -- start start 2024-04-20T21:28:52.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.849+0000 7f7a63e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a5c06f140 0x7f7a5c13c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:52.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.849+0000 7f7a63e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a5c071d00 0x7f7a5c0aeb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:52.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.849+0000 7f7a63e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a5c144e90 0x7f7a5c0b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:52.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.849+0000 7f7a63e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5c06ded0 con 0x7f7a5c06f140 2024-04-20T21:28:52.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.849+0000 7f7a63e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5c15fce0 con 0x7f7a5c144e90 2024-04-20T21:28:52.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.849+0000 7f7a63e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7a5c15f350 con 0x7f7a5c071d00 2024-04-20T21:28:52.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.850+0000 7f7a61400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a5c06f140 0x7f7a5c13c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:52.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.850+0000 7f7a61400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a5c06f140 0x7f7a5c13c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45914/0 (socket says 172.21.15.86:45914) 2024-04-20T21:28:52.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.850+0000 7f7a61400640 1 -- 172.21.15.86:0/1979363493 learned_addr learned my addr 172.21.15.86:0/1979363493 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:52.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.850+0000 7f7a61e00640 1 --2- 172.21.15.86:0/1979363493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a5c144e90 0x7f7a5c0b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:52.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.850+0000 7f7a61400640 1 -- 172.21.15.86:0/1979363493 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a5c071d00 msgr2=0x7f7a5c0aeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:52.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.850+0000 7f7a61400640 1 --2- 172.21.15.86:0/1979363493 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a5c071d00 0x7f7a5c0aeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:52.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.850+0000 7f7a61400640 1 -- 172.21.15.86:0/1979363493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a5c144e90 msgr2=0x7f7a5c0b37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:52.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.850+0000 7f7a61400640 1 --2- 172.21.15.86:0/1979363493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a5c144e90 0x7f7a5c0b37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:52.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.850+0000 7f7a61400640 1 -- 172.21.15.86:0/1979363493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a5c0af580 con 0x7f7a5c06f140 2024-04-20T21:28:52.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.850+0000 7f7a61400640 1 --2- 172.21.15.86:0/1979363493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a5c06f140 0x7f7a5c13c210 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f7a5000d800 tx=0x7f7a5000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:52.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.851+0000 7f7a59600640 1 -- 172.21.15.86:0/1979363493 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5000b6c0 con 0x7f7a5c06f140 2024-04-20T21:28:52.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.851+0000 7f7a59600640 1 -- 172.21.15.86:0/1979363493 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a50019070 con 0x7f7a5c06f140 2024-04-20T21:28:52.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.851+0000 7f7a63e00640 1 -- 172.21.15.86:0/1979363493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a5c0af860 con 0x7f7a5c06f140 2024-04-20T21:28:52.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.851+0000 7f7a59600640 1 -- 172.21.15.86:0/1979363493 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5000ba80 con 0x7f7a5c06f140 2024-04-20T21:28:52.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.851+0000 7f7a63e00640 1 -- 172.21.15.86:0/1979363493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a5c0afe20 con 0x7f7a5c06f140 2024-04-20T21:28:52.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.852+0000 7f7a63e00640 1 -- 172.21.15.86:0/1979363493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a5c0689e0 con 0x7f7a5c06f140 2024-04-20T21:28:52.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.853+0000 7f7a59600640 1 -- 172.21.15.86:0/1979363493 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7a5000bc20 con 0x7f7a5c06f140 2024-04-20T21:28:52.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.855+0000 7f7a59600640 1 --2- 172.21.15.86:0/1979363493 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7a4006c8f0 0x7f7a4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:52.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.855+0000 7f7a59600640 1 -- 172.21.15.86:0/1979363493 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5795+0+0 (secure 0 0 0) 0x7f7a5008c0b0 con 0x7f7a5c06f140 2024-04-20T21:28:52.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.855+0000 7f7a60a00640 1 --2- 172.21.15.86:0/1979363493 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7a4006c8f0 0x7f7a4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:52.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.856+0000 7f7a60a00640 1 --2- 172.21.15.86:0/1979363493 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7a4006c8f0 0x7f7a4006edb0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c1481f0 tx=0x7f7a4c006ba0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:52.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:52.860+0000 7f7a59600640 1 -- 172.21.15.86:0/1979363493 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7a5001c360 con 0x7f7a5c06f140 2024-04-20T21:28:53.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.098+0000 7f7a63e00640 1 -- 172.21.15.86:0/1979363493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7a5c0d01b0 con 0x7f7a5c06f140 2024-04-20T21:28:53.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.099+0000 7f7a59600640 1 -- 172.21.15.86:0/1979363493 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) ==== 94+0+5209 (secure 0 0 0) 0x7f7a5005a470 con 0x7f7a5c06f140 2024-04-20T21:28:53.099 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:28:53.100 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":88,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5525,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3902778887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3902778887},{"type":"v1","addr":"172.21.15.144:6835","nonce":3902778887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6001,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/4270988014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":4270988014},{"type":"v1","addr":"172.21.15.86:6837","nonce":4270988014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6016,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/1569462554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":1569462554},{"type":"v1","addr":"172.21.15.144:6833","nonce":1569462554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:27:45.692408+0000","modified":"2024-04-20T21:28:51.873897+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":101,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8922},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8922":{"gid":8922,"name":"c","rank":0,"incarnation":85,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6836/4271041811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4271041811},{"type":"v1","addr":"172.21.15.86:6836","nonce":4271041811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8922,"qdb_cluster":[8922]},"id":7}]} 2024-04-20T21:28:53.100 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 88 2024-04-20T21:28:53.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.102+0000 7f7a63e00640 1 -- 172.21.15.86:0/1979363493 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7a4006c8f0 msgr2=0x7f7a4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:53.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.102+0000 7f7a63e00640 1 --2- 172.21.15.86:0/1979363493 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7a4006c8f0 0x7f7a4006edb0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c1481f0 tx=0x7f7a4c006ba0 comp rx=0 tx=0).stop 2024-04-20T21:28:53.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.102+0000 7f7a63e00640 1 -- 172.21.15.86:0/1979363493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a5c06f140 msgr2=0x7f7a5c13c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:53.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.102+0000 7f7a63e00640 1 --2- 172.21.15.86:0/1979363493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a5c06f140 0x7f7a5c13c210 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f7a5000d800 tx=0x7f7a5000dcc0 comp rx=0 tx=0).stop 2024-04-20T21:28:53.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.102+0000 7f7a63e00640 1 -- 172.21.15.86:0/1979363493 shutdown_connections 2024-04-20T21:28:53.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.102+0000 7f7a63e00640 1 --2- 172.21.15.86:0/1979363493 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7a4006c8f0 0x7f7a4006edb0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:53.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.102+0000 7f7a63e00640 1 --2- 172.21.15.86:0/1979363493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a5c144e90 0x7f7a5c0b37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:53.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.102+0000 7f7a63e00640 1 --2- 172.21.15.86:0/1979363493 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7a5c071d00 0x7f7a5c0aeb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:53.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.102+0000 7f7a63e00640 1 --2- 172.21.15.86:0/1979363493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a5c06f140 0x7f7a5c13c210 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:53.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.102+0000 7f7a63e00640 1 -- 172.21.15.86:0/1979363493 >> 172.21.15.86:0/1979363493 conn(0x7f7a5c0b2360 msgr2=0x7f7a5c1429a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:53.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.102+0000 7f7a63e00640 1 -- 172.21.15.86:0/1979363493 shutdown_connections 2024-04-20T21:28:53.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.102+0000 7f7a63e00640 1 -- 172.21.15.86:0/1979363493 wait complete. 2024-04-20T21:28:53.522 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 88, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T21:27:45.692408+0000', 'modified': '2024-04-20T21:28:51.873897+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 101, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8922}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8922': {'gid': 8922, 'name': 'c', 'rank': 0, 'incarnation': 85, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.86:6836/4271041811', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6834', 'nonce': 4271041811}, {'type': 'v1', 'addr': '172.21.15.86:6836', 'nonce': 4271041811}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8922, 'qdb_cluster': [8922]} 2024-04-20T21:28:53.523 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-20T21:28:53.523 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-20T21:28:53.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.727+0000 7f9344c00640 1 Processor -- start 2024-04-20T21:28:53.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.727+0000 7f9344c00640 1 -- start start 2024-04-20T21:28:53.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.728+0000 7f9344c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f93400d16e0 0x7f93400d3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:53.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.728+0000 7f9344c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93400cf810 0x7f93400cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:53.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.728+0000 7f9344c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93400b7120 con 0x7f934015ed80 2024-04-20T21:28:53.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.728+0000 7f9344c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f93400d01c0 con 0x7f93400d16e0 2024-04-20T21:28:53.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.728+0000 7f9344c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f93400d1b80 con 0x7f93400cf810 2024-04-20T21:28:53.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.729+0000 7f933e800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93400cf810 0x7f93400cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:53.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.729+0000 7f933d400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f93400d16e0 0x7f93400d3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:53.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.729+0000 7f933e800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93400cf810 0x7f93400cfc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:39934/0 (socket says 172.21.15.86:39934) 2024-04-20T21:28:53.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.729+0000 7f933e800640 1 -- 172.21.15.86:0/3975435679 learned_addr learned my addr 172.21.15.86:0/3975435679 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:53.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.730+0000 7f933e800640 1 -- 172.21.15.86:0/3975435679 >> v2:172.21.15.144:3300/0 conn(0x7f93400d16e0 msgr2=0x7f93400d3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:53.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.730+0000 7f933e800640 1 --2- 172.21.15.86:0/3975435679 >> v2:172.21.15.144:3300/0 conn(0x7f93400d16e0 0x7f93400d3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:53.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.730+0000 7f933e800640 1 -- 172.21.15.86:0/3975435679 >> v1:172.21.15.144:6789/0 conn(0x7f934015ed80 legacy=0x7f93400d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:53.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.730+0000 7f933e800640 1 -- 172.21.15.86:0/3975435679 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93400d2e20 con 0x7f93400cf810 2024-04-20T21:28:53.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.730+0000 7f933d400640 1 --2- 172.21.15.86:0/3975435679 >> v2:172.21.15.144:3300/0 conn(0x7f93400d16e0 0x7f93400d3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:28:53.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.730+0000 7f933ca00640 1 -- 172.21.15.86:0/3975435679 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3631129853 0 0) 0x7f93400b7120 con 0x7f934015ed80 2024-04-20T21:28:53.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.732+0000 7f933e800640 1 --2- 172.21.15.86:0/3975435679 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93400cf810 0x7f93400cfc80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9338009870 tx=0x7f933802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e0bd92f9f92737a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:53.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.732+0000 7f933ca00640 1 -- 172.21.15.86:0/3975435679 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9338035020 con 0x7f93400cf810 2024-04-20T21:28:53.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.733+0000 7f933ca00640 1 -- 172.21.15.86:0/3975435679 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9338031640 con 0x7f93400cf810 2024-04-20T21:28:53.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.733+0000 7f933ca00640 1 -- 172.21.15.86:0/3975435679 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933802d040 con 0x7f93400cf810 2024-04-20T21:28:53.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.733+0000 7f9344c00640 1 -- 172.21.15.86:0/3975435679 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93400cf810 msgr2=0x7f93400cfc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:53.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.733+0000 7f9344c00640 1 --2- 172.21.15.86:0/3975435679 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93400cf810 0x7f93400cfc80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9338009870 tx=0x7f933802b8f0 comp rx=0 tx=0).stop 2024-04-20T21:28:53.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.733+0000 7f9344c00640 1 -- 172.21.15.86:0/3975435679 shutdown_connections 2024-04-20T21:28:53.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.733+0000 7f9344c00640 1 --2- 172.21.15.86:0/3975435679 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93400cf810 0x7f93400cfc80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:53.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.733+0000 7f9344c00640 1 --2- 172.21.15.86:0/3975435679 >> v2:172.21.15.144:3300/0 conn(0x7f93400d16e0 0x7f93400d3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:53.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.733+0000 7f9344c00640 1 -- 172.21.15.86:0/3975435679 >> 172.21.15.86:0/3975435679 conn(0x7f934006cc20 msgr2=0x7f934006d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:53.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.733+0000 7f9344c00640 1 -- 172.21.15.86:0/3975435679 shutdown_connections 2024-04-20T21:28:53.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.734+0000 7f9344c00640 1 -- 172.21.15.86:0/3975435679 wait complete. 2024-04-20T21:28:53.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.735+0000 7f9344c00640 1 Processor -- start 2024-04-20T21:28:53.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.735+0000 7f9344c00640 1 -- start start 2024-04-20T21:28:53.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.736+0000 7f9344c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93400cf810 0x7f93401f51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:53.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.736+0000 7f9344c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93400d16e0 0x7f93401f7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:53.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.736+0000 7f9344c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f934015ed80 0x7f93400cd370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:53.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.736+0000 7f9344c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f93400d36c0 con 0x7f93400d16e0 2024-04-20T21:28:53.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.736+0000 7f9344c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f93400d33e0 con 0x7f934015ed80 2024-04-20T21:28:53.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.736+0000 7f9344c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f934015f160 con 0x7f93400cf810 2024-04-20T21:28:53.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.737+0000 7f933d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93400d16e0 0x7f93401f7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:53.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.737+0000 7f933d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93400d16e0 0x7f93401f7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45930/0 (socket says 172.21.15.86:45930) 2024-04-20T21:28:53.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.737+0000 7f933d400640 1 -- 172.21.15.86:0/2210108185 learned_addr learned my addr 172.21.15.86:0/2210108185 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:53.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.737+0000 7f933e800640 1 --2- 172.21.15.86:0/2210108185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f934015ed80 0x7f93400cd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:53.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.737+0000 7f933de00640 1 --2- 172.21.15.86:0/2210108185 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93400cf810 0x7f93401f51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:53.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.737+0000 7f933d400640 1 -- 172.21.15.86:0/2210108185 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93400cf810 msgr2=0x7f93401f51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:53.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.737+0000 7f933d400640 1 --2- 172.21.15.86:0/2210108185 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93400cf810 0x7f93401f51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:53.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.737+0000 7f933d400640 1 -- 172.21.15.86:0/2210108185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f934015ed80 msgr2=0x7f93400cd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:53.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.737+0000 7f933d400640 1 --2- 172.21.15.86:0/2210108185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f934015ed80 0x7f93400cd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:53.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.737+0000 7f933d400640 1 -- 172.21.15.86:0/2210108185 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93401f5730 con 0x7f93400d16e0 2024-04-20T21:28:53.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.737+0000 7f933de00640 1 --2- 172.21.15.86:0/2210108185 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93400cf810 0x7f93401f51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:28:53.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.737+0000 7f933e800640 1 --2- 172.21.15.86:0/2210108185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f934015ed80 0x7f93400cd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:28:53.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.737+0000 7f933d400640 1 --2- 172.21.15.86:0/2210108185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93400d16e0 0x7f93401f7fd0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f9328009f60 tx=0x7f9328009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:53.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.738+0000 7f9336000640 1 -- 172.21.15.86:0/2210108185 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9328018020 con 0x7f93400d16e0 2024-04-20T21:28:53.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.738+0000 7f9336000640 1 -- 172.21.15.86:0/2210108185 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9328014680 con 0x7f93400d16e0 2024-04-20T21:28:53.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.738+0000 7f9336000640 1 -- 172.21.15.86:0/2210108185 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9328014a50 con 0x7f93400d16e0 2024-04-20T21:28:53.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.738+0000 7f9344c00640 1 -- 172.21.15.86:0/2210108185 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93400b86b0 con 0x7f93400d16e0 2024-04-20T21:28:53.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.738+0000 7f9344c00640 1 -- 172.21.15.86:0/2210108185 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93400b8c70 con 0x7f93400d16e0 2024-04-20T21:28:53.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.740+0000 7f9336000640 1 -- 172.21.15.86:0/2210108185 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9328005d10 con 0x7f93400d16e0 2024-04-20T21:28:53.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.741+0000 7f9336000640 1 --2- 172.21.15.86:0/2210108185 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f931c06c8f0 0x7f931c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:53.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.741+0000 7f9336000640 1 -- 172.21.15.86:0/2210108185 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5795+0+0 (secure 0 0 0) 0x7f932808cae0 con 0x7f93400d16e0 2024-04-20T21:28:53.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.741+0000 7f933de00640 1 --2- 172.21.15.86:0/2210108185 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f931c06c8f0 0x7f931c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:53.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.742+0000 7f934785c740 1 Processor -- start 2024-04-20T21:28:53.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.742+0000 7f934785c740 1 -- start start 2024-04-20T21:28:53.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.742+0000 7f933de00640 1 --2- 172.21.15.86:0/2210108185 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f931c06c8f0 0x7f931c06edb0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f9330000f60 tx=0x7f9330009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:53.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.743+0000 7f934785c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b0c258460 0x561b0c20fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:53.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.743+0000 7f934785c740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561b0c20e4e0 0x561b0c20ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:53.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.743+0000 7f934785c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b0c20a930 0x561b0c23a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:53.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.743+0000 7f934785c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561b0c1769d0 con 0x561b0c258460 2024-04-20T21:28:53.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.743+0000 7f934785c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561b0c11d270 con 0x561b0c20a930 2024-04-20T21:28:53.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.743+0000 7f934785c740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x561b0c172990 con 0x561b0c20e4e0 2024-04-20T21:28:53.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.743+0000 7f933de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b0c258460 0x561b0c20fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:53.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.743+0000 7f933de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b0c258460 0x561b0c20fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45936/0 (socket says 172.21.15.86:45936) 2024-04-20T21:28:53.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.743+0000 7f933de00640 1 -- 172.21.15.86:0/3642662787 learned_addr learned my addr 172.21.15.86:0/3642662787 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:53.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.743+0000 7f933e800640 1 --2- 172.21.15.86:0/3642662787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b0c20a930 0x561b0c23a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:53.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.743+0000 7f933de00640 1 -- 172.21.15.86:0/3642662787 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561b0c20e4e0 msgr2=0x561b0c20ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:53.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.743+0000 7f933de00640 1 --2- 172.21.15.86:0/3642662787 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561b0c20e4e0 0x561b0c20ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:53.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.743+0000 7f933de00640 1 -- 172.21.15.86:0/3642662787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b0c20a930 msgr2=0x561b0c23a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:53.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.744+0000 7f933de00640 1 --2- 172.21.15.86:0/3642662787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b0c20a930 0x561b0c23a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:53.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.744+0000 7f933de00640 1 -- 172.21.15.86:0/3642662787 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b0be892d0 con 0x561b0c258460 2024-04-20T21:28:53.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.744+0000 7f933e800640 1 --2- 172.21.15.86:0/3642662787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b0c20a930 0x561b0c23a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:28:53.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.744+0000 7f933de00640 1 --2- 172.21.15.86:0/3642662787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b0c258460 0x561b0c20fa90 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f9330005360 tx=0x7f9330010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88af31d8a92889d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:53.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.744+0000 7f932e600640 1 -- 172.21.15.86:0/3642662787 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9330003a20 con 0x561b0c258460 2024-04-20T21:28:53.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.744+0000 7f932e600640 1 -- 172.21.15.86:0/3642662787 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9330003bc0 con 0x561b0c258460 2024-04-20T21:28:53.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.744+0000 7f932e600640 1 -- 172.21.15.86:0/3642662787 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9330020450 con 0x561b0c258460 2024-04-20T21:28:53.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.745+0000 7f934785c740 1 -- 172.21.15.86:0/3642662787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b0c258460 msgr2=0x561b0c20fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:53.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.745+0000 7f934785c740 1 --2- 172.21.15.86:0/3642662787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b0c258460 0x561b0c20fa90 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f9330005360 tx=0x7f9330010000 comp rx=0 tx=0).stop 2024-04-20T21:28:53.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.745+0000 7f934785c740 1 -- 172.21.15.86:0/3642662787 shutdown_connections 2024-04-20T21:28:53.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.745+0000 7f934785c740 1 --2- 172.21.15.86:0/3642662787 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b0c20a930 0x561b0c23a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:53.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.745+0000 7f934785c740 1 --2- 172.21.15.86:0/3642662787 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561b0c20e4e0 0x561b0c20ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:53.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.745+0000 7f934785c740 1 --2- 172.21.15.86:0/3642662787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b0c258460 0x561b0c20fa90 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:53.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.745+0000 7f934785c740 1 -- 172.21.15.86:0/3642662787 >> 172.21.15.86:0/3642662787 conn(0x561b0c20cbf0 msgr2=0x561b0c255770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:53.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.745+0000 7f934785c740 1 -- 172.21.15.86:0/3642662787 shutdown_connections 2024-04-20T21:28:53.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.745+0000 7f934785c740 1 -- 172.21.15.86:0/3642662787 wait complete. 2024-04-20T21:28:53.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.746+0000 7f934785c740 1 -- start start 2024-04-20T21:28:53.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.746+0000 7f934785c740 1 Processor -- start 2024-04-20T21:28:53.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.747+0000 7f934785c740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561b0c20e4e0 0x561b0c235a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:53.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.747+0000 7f934785c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b0c258460 0x561b0c232c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:53.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.747+0000 7f934785c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b0c25a530 0x561b0c222810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:53.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.747+0000 7f934785c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561b0c20e9f0 con 0x561b0c258460 2024-04-20T21:28:53.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.747+0000 7f934785c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561b0c1305e0 con 0x561b0c25a530 2024-04-20T21:28:53.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.747+0000 7f934785c740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x561b0c20ecd0 con 0x561b0c20e4e0 2024-04-20T21:28:53.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.747+0000 7f934785c740 10 client.0 _ll_register_callbacks cb 0x561b0be88ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:28:53.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.747+0000 7f933d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b0c258460 0x561b0c232c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:53.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.747+0000 7f933d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b0c258460 0x561b0c232c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45940/0 (socket says 172.21.15.86:45940) 2024-04-20T21:28:53.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.747+0000 7f933d400640 1 -- 172.21.15.86:0/1185790503 learned_addr learned my addr 172.21.15.86:0/1185790503 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:53.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.747+0000 7f933e800640 1 --2- 172.21.15.86:0/1185790503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b0c25a530 0x561b0c222810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:53.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.747+0000 7f933d400640 1 -- 172.21.15.86:0/1185790503 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561b0c20e4e0 msgr2=0x561b0c235a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:53.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.747+0000 7f933d400640 1 --2- 172.21.15.86:0/1185790503 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561b0c20e4e0 0x561b0c235a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:53.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.747+0000 7f933d400640 1 -- 172.21.15.86:0/1185790503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b0c25a530 msgr2=0x561b0c222810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:53.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.747+0000 7f933d400640 1 --2- 172.21.15.86:0/1185790503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b0c25a530 0x561b0c222810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:53.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.747+0000 7f933d400640 1 -- 172.21.15.86:0/1185790503 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b0bf536b0 con 0x561b0c258460 2024-04-20T21:28:53.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.748+0000 7f933d400640 1 --2- 172.21.15.86:0/1185790503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b0c258460 0x561b0c232c10 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f9328062dc0 tx=0x7f932805aed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=304705664c4bb7f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:53.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.748+0000 7f9326a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:28:53.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.748+0000 7f9326a00640 1 -- 172.21.15.86:0/1185790503 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93280632f0 con 0x561b0c258460 2024-04-20T21:28:53.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.748+0000 7f9326a00640 1 -- 172.21.15.86:0/1185790503 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 5795+0+0 (secure 0 0 0) 0x7f9328059b60 con 0x561b0c258460 2024-04-20T21:28:53.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.748+0000 7f934785c740 1 -- 172.21.15.86:0/1185790503 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9330003bc0 con 0x561b0c258460 2024-04-20T21:28:53.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.748+0000 7f9326a00640 1 -- 172.21.15.86:0/1185790503 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f9328059d60 con 0x561b0c258460 2024-04-20T21:28:53.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.748+0000 7f934785c740 10 client.8985 fetch_fsmap learned FSMap version 88 2024-04-20T21:28:53.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.748+0000 7f934785c740 1 -- 172.21.15.86:0/1185790503 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x561b0c2362c0 con 0x561b0c258460 2024-04-20T21:28:53.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.748+0000 7f9326a00640 1 -- 172.21.15.86:0/1185790503 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9330003bc0 con 0x561b0c258460 2024-04-20T21:28:53.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.749+0000 7f9326a00640 1 -- 172.21.15.86:0/1185790503 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 88) ==== 3495+0+0 (secure 0 0 0) 0x7f9328056460 con 0x561b0c258460 2024-04-20T21:28:53.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.749+0000 7f934785c740 10 client.8985 fetch_fsmap finished waiting for FSMap version 88 2024-04-20T21:28:53.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.749+0000 7f934785c740 10 client.8985 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-20T21:28:53.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.749+0000 7f934785c740 20 client.8985 populate_metadata read hostname 'smithi086' 2024-04-20T21:28:53.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.749+0000 7f934785c740 1 --2- 172.21.15.86:0/1185790503 >> [v2:172.21.15.86:6834/4271041811,v1:172.21.15.86:6836/4271041811] conn(0x561b0c26a290 0x561b0c25bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:53.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.749+0000 7f934785c740 4 client.8985 mds_command: new command op to 8922 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:28:53.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.749+0000 7f934785c740 1 -- 172.21.15.86:0/1185790503 --> [v2:172.21.15.86:6834/4271041811,v1:172.21.15.86:6836/4271041811] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b0c1843f0 con 0x561b0c26a290 2024-04-20T21:28:53.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.749+0000 7f933de00640 1 --2- 172.21.15.86:0/1185790503 >> [v2:172.21.15.86:6834/4271041811,v1:172.21.15.86:6836/4271041811] conn(0x561b0c26a290 0x561b0c25bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:53.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.750+0000 7f933de00640 1 --2- 172.21.15.86:0/1185790503 >> [v2:172.21.15.86:6834/4271041811,v1:172.21.15.86:6836/4271041811] conn(0x561b0c26a290 0x561b0c25bd00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d8c9be97e6ffb25 server_cookie=9d0e4b84b6c95017 in_seq=0 out_seq=0 2024-04-20T21:28:53.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.750+0000 7f9326a00640 10 client.8985 ms_handle_connect on v2:172.21.15.86:6834/4271041811 2024-04-20T21:28:53.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.750+0000 7f9326a00640 1 -- 172.21.15.86:0/1185790503 <== mds.0 v2:172.21.15.86:6834/4271041811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561b0c1843f0 con 0x561b0c26a290 2024-04-20T21:28:53.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.750+0000 7f9326a00640 10 client.8985 handle_command_reply: tid=0 2024-04-20T21:28:53.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.750+0000 7f934785c740 1 client.8985 shutdown 2024-04-20T21:28:53.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.751+0000 7f934785c740 1 -- 172.21.15.86:0/1185790503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b0c258460 msgr2=0x561b0c232c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:53.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.751+0000 7f934785c740 1 --2- 172.21.15.86:0/1185790503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b0c258460 0x561b0c232c10 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f9328062dc0 tx=0x7f932805aed0 comp rx=0 tx=0).stop 2024-04-20T21:28:53.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.751+0000 7f934785c740 1 -- 172.21.15.86:0/1185790503 shutdown_connections 2024-04-20T21:28:53.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.751+0000 7f934785c740 1 --2- 172.21.15.86:0/1185790503 >> [v2:172.21.15.86:6834/4271041811,v1:172.21.15.86:6836/4271041811] conn(0x561b0c26a290 0x561b0c25bd00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:53.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.751+0000 7f934785c740 1 --2- 172.21.15.86:0/1185790503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b0c25a530 0x561b0c222810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:53.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.751+0000 7f9326a00640 0 client.8985 ms_handle_reset on v2:172.21.15.86:6834/4271041811 2024-04-20T21:28:53.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.751+0000 7f934785c740 1 --2- 172.21.15.86:0/1185790503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b0c258460 0x561b0c232c10 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:53.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.751+0000 7f934785c740 1 --2- 172.21.15.86:0/1185790503 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561b0c20e4e0 0x561b0c235a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:53.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.751+0000 7f934785c740 1 -- 172.21.15.86:0/1185790503 >> 172.21.15.86:0/1185790503 conn(0x561b0c20a930 msgr2=0x561b0c20f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:53.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.751+0000 7f934785c740 1 -- 172.21.15.86:0/1185790503 shutdown_connections 2024-04-20T21:28:53.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.751+0000 7f934785c740 1 -- 172.21.15.86:0/1185790503 wait complete. 2024-04-20T21:28:53.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.751+0000 7f934785c740 20 client.8985 trim_cache size 0 max 16384 2024-04-20T21:28:53.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.770+0000 7f934785c740 1 Processor -- start 2024-04-20T21:28:53.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.771+0000 7f934785c740 1 -- start start 2024-04-20T21:28:53.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.771+0000 7f934785c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b0c20e4e0 0x561b0c210790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:53.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.771+0000 7f934785c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b0c258460 0x561b0c268330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:53.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.771+0000 7f934785c740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561b0c25a530 0x561b0c268870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:53.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.771+0000 7f934785c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f934014a890 con 0x561b0c20e4e0 2024-04-20T21:28:53.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.771+0000 7f934785c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9340149bf0 con 0x561b0c258460 2024-04-20T21:28:53.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.771+0000 7f934785c740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f934014a240 con 0x561b0c25a530 2024-04-20T21:28:53.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.772+0000 7f933de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b0c20e4e0 0x561b0c210790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:53.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.772+0000 7f933de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b0c20e4e0 0x561b0c210790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45954/0 (socket says 172.21.15.86:45954) 2024-04-20T21:28:53.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.772+0000 7f933de00640 1 -- 172.21.15.86:0/2689199997 learned_addr learned my addr 172.21.15.86:0/2689199997 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:53.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.772+0000 7f933de00640 1 -- 172.21.15.86:0/2689199997 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561b0c25a530 msgr2=0x561b0c268870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:53.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.772+0000 7f933de00640 1 --2- 172.21.15.86:0/2689199997 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561b0c25a530 0x561b0c268870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:53.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.772+0000 7f933de00640 1 -- 172.21.15.86:0/2689199997 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b0c258460 msgr2=0x561b0c268330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:53.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.772+0000 7f933de00640 1 --2- 172.21.15.86:0/2689199997 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b0c258460 0x561b0c268330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:53.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.772+0000 7f933de00640 1 -- 172.21.15.86:0/2689199997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b0c211fc0 con 0x561b0c20e4e0 2024-04-20T21:28:53.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.772+0000 7f933de00640 1 --2- 172.21.15.86:0/2689199997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b0c20e4e0 0x561b0c210790 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f9330013000 tx=0x7f9330005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88aed520799b94a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:53.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.773+0000 7f932dc00640 1 -- 172.21.15.86:0/2689199997 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933002c540 con 0x561b0c20e4e0 2024-04-20T21:28:53.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.773+0000 7f932dc00640 1 -- 172.21.15.86:0/2689199997 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f933002c6e0 con 0x561b0c20e4e0 2024-04-20T21:28:53.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.773+0000 7f932dc00640 1 -- 172.21.15.86:0/2689199997 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9330020450 con 0x561b0c20e4e0 2024-04-20T21:28:53.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.773+0000 7f934785c740 1 -- 172.21.15.86:0/2689199997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b0c20e4e0 msgr2=0x561b0c210790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:53.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.773+0000 7f934785c740 1 --2- 172.21.15.86:0/2689199997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b0c20e4e0 0x561b0c210790 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f9330013000 tx=0x7f9330005000 comp rx=0 tx=0).stop 2024-04-20T21:28:53.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.773+0000 7f934785c740 1 -- 172.21.15.86:0/2689199997 shutdown_connections 2024-04-20T21:28:53.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.773+0000 7f934785c740 1 --2- 172.21.15.86:0/2689199997 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561b0c25a530 0x561b0c268870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:53.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.773+0000 7f934785c740 1 --2- 172.21.15.86:0/2689199997 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b0c258460 0x561b0c268330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:53.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.773+0000 7f934785c740 1 --2- 172.21.15.86:0/2689199997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b0c20e4e0 0x561b0c210790 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:53.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.773+0000 7f934785c740 1 -- 172.21.15.86:0/2689199997 >> 172.21.15.86:0/2689199997 conn(0x561b0c20a930 msgr2=0x561b0c238a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:53.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.773+0000 7f934785c740 1 -- 172.21.15.86:0/2689199997 shutdown_connections 2024-04-20T21:28:53.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.773+0000 7f934785c740 1 -- 172.21.15.86:0/2689199997 wait complete. 2024-04-20T21:28:53.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.774+0000 7f934785c740 1 -- start start 2024-04-20T21:28:53.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.774+0000 7f934785c740 1 Processor -- start 2024-04-20T21:28:53.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.775+0000 7f934785c740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b0c20e4e0 0x561b0c2d8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:53.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.775+0000 7f934785c740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561b0c258460 0x561b0c25bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:53.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.775+0000 7f934785c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b0c25a530 0x561b0c278fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:53.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.775+0000 7f934785c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561b0c26a110 con 0x561b0c25a530 2024-04-20T21:28:53.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.775+0000 7f934785c740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x561b0c2365f0 con 0x561b0c20e4e0 2024-04-20T21:28:53.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.775+0000 7f934785c740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x561b0c26a730 con 0x561b0c258460 2024-04-20T21:28:53.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.775+0000 7f934785c740 10 client.0 _ll_register_callbacks cb 0x7f932800cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:28:53.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.776+0000 7f933e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b0c25a530 0x561b0c278fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:53.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.776+0000 7f933e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b0c25a530 0x561b0c278fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45956/0 (socket says 172.21.15.86:45956) 2024-04-20T21:28:53.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.776+0000 7f933e800640 1 -- 172.21.15.86:0/1816744443 learned_addr learned my addr 172.21.15.86:0/1816744443 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:53.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.776+0000 7f933d400640 1 --2- 172.21.15.86:0/1816744443 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561b0c258460 0x561b0c25bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:53.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.776+0000 7f933de00640 1 --2- 172.21.15.86:0/1816744443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b0c20e4e0 0x561b0c2d8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:53.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.776+0000 7f933e800640 1 -- 172.21.15.86:0/1816744443 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561b0c258460 msgr2=0x561b0c25bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:53.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.776+0000 7f933e800640 1 --2- 172.21.15.86:0/1816744443 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561b0c258460 0x561b0c25bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:53.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.776+0000 7f933e800640 1 -- 172.21.15.86:0/1816744443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b0c20e4e0 msgr2=0x561b0c2d8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:53.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.776+0000 7f933e800640 1 --2- 172.21.15.86:0/1816744443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b0c20e4e0 0x561b0c2d8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:53.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.776+0000 7f933e800640 1 -- 172.21.15.86:0/1816744443 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b0c20b390 con 0x561b0c25a530 2024-04-20T21:28:53.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.776+0000 7f933d400640 1 --2- 172.21.15.86:0/1816744443 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561b0c258460 0x561b0c25bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:28:53.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.776+0000 7f933de00640 1 --2- 172.21.15.86:0/1816744443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b0c20e4e0 0x561b0c2d8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:28:53.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.776+0000 7f933e800640 1 --2- 172.21.15.86:0/1816744443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b0c25a530 0x561b0c278fe0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f9338009990 tx=0x7f93380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7662838ea1d14b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:53.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.776+0000 7f9326a00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:28:53.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.777+0000 7f9326a00640 1 -- 172.21.15.86:0/1816744443 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9338035040 con 0x561b0c25a530 2024-04-20T21:28:53.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.777+0000 7f934785c740 1 -- 172.21.15.86:0/1816744443 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f933002c6e0 con 0x561b0c25a530 2024-04-20T21:28:53.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.777+0000 7f9326a00640 1 -- 172.21.15.86:0/1816744443 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 5795+0+0 (secure 0 0 0) 0x7f9338003ad0 con 0x561b0c25a530 2024-04-20T21:28:53.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.777+0000 7f9326a00640 1 -- 172.21.15.86:0/1816744443 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f93380044d0 con 0x561b0c25a530 2024-04-20T21:28:53.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.777+0000 7f934785c740 10 client.8991 fetch_fsmap learned FSMap version 88 2024-04-20T21:28:53.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.777+0000 7f934785c740 1 -- 172.21.15.86:0/1816744443 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x561b0c24bb90 con 0x561b0c25a530 2024-04-20T21:28:53.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.777+0000 7f9326a00640 1 -- 172.21.15.86:0/1816744443 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f933002c6e0 con 0x561b0c25a530 2024-04-20T21:28:53.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.777+0000 7f9326a00640 1 -- 172.21.15.86:0/1816744443 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 88) ==== 3495+0+0 (secure 0 0 0) 0x7f9338042070 con 0x561b0c25a530 2024-04-20T21:28:53.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.777+0000 7f934785c740 10 client.8991 fetch_fsmap finished waiting for FSMap version 88 2024-04-20T21:28:53.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.777+0000 7f934785c740 10 client.8991 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-20T21:28:53.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.777+0000 7f934785c740 20 client.8991 populate_metadata read hostname 'smithi086' 2024-04-20T21:28:53.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.777+0000 7f934785c740 1 --2- 172.21.15.86:0/1816744443 >> [v2:172.21.15.86:6834/4271041811,v1:172.21.15.86:6836/4271041811] conn(0x561b0c2314b0 0x561b0c231890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:53.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.777+0000 7f934785c740 4 client.8991 mds_command: new command op to 8922 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T21:28:53.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.777+0000 7f934785c740 1 -- 172.21.15.86:0/1816744443 --> [v2:172.21.15.86:6834/4271041811,v1:172.21.15.86:6836/4271041811] -- command(tid 0: {"prefix": "status"}) -- 0x561b0c17e260 con 0x561b0c2314b0 2024-04-20T21:28:53.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.778+0000 7f933de00640 1 --2- 172.21.15.86:0/1816744443 >> [v2:172.21.15.86:6834/4271041811,v1:172.21.15.86:6836/4271041811] conn(0x561b0c2314b0 0x561b0c231890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:53.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.778+0000 7f933de00640 1 --2- 172.21.15.86:0/1816744443 >> [v2:172.21.15.86:6834/4271041811,v1:172.21.15.86:6836/4271041811] conn(0x561b0c2314b0 0x561b0c231890 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=176a1de691f9f1cc server_cookie=611d53de14668408 in_seq=0 out_seq=0 2024-04-20T21:28:53.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.778+0000 7f9326a00640 10 client.8991 ms_handle_connect on v2:172.21.15.86:6834/4271041811 2024-04-20T21:28:53.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.778+0000 7f9326a00640 1 -- 172.21.15.86:0/1816744443 <== mds.0 v2:172.21.15.86:6834/4271041811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x561b0c17e260 con 0x561b0c2314b0 2024-04-20T21:28:53.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.778+0000 7f9326a00640 10 client.8991 handle_command_reply: tid=0 2024-04-20T21:28:53.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.778+0000 7f934785c740 1 client.8991 shutdown 2024-04-20T21:28:53.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.779+0000 7f934785c740 1 -- 172.21.15.86:0/1816744443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b0c25a530 msgr2=0x561b0c278fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:53.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.779+0000 7f934785c740 1 --2- 172.21.15.86:0/1816744443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b0c25a530 0x561b0c278fe0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f9338009990 tx=0x7f93380046c0 comp rx=0 tx=0).stop 2024-04-20T21:28:53.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.779+0000 7f934785c740 1 -- 172.21.15.86:0/1816744443 shutdown_connections 2024-04-20T21:28:53.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.779+0000 7f934785c740 1 --2- 172.21.15.86:0/1816744443 >> [v2:172.21.15.86:6834/4271041811,v1:172.21.15.86:6836/4271041811] conn(0x561b0c2314b0 0x561b0c231890 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:53.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.779+0000 7f934785c740 1 --2- 172.21.15.86:0/1816744443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561b0c25a530 0x561b0c278fe0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:53.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.779+0000 7f934785c740 1 --2- 172.21.15.86:0/1816744443 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x561b0c258460 0x561b0c25bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:53.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.779+0000 7f9326a00640 0 client.8991 ms_handle_reset on v2:172.21.15.86:6834/4271041811 2024-04-20T21:28:53.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.779+0000 7f934785c740 1 --2- 172.21.15.86:0/1816744443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x561b0c20e4e0 0x561b0c2d8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:53.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.779+0000 7f934785c740 1 -- 172.21.15.86:0/1816744443 >> 172.21.15.86:0/1816744443 conn(0x561b0c20a930 msgr2=0x561b0c238a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:53.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.779+0000 7f934785c740 1 -- 172.21.15.86:0/1816744443 shutdown_connections 2024-04-20T21:28:53.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.779+0000 7f934785c740 1 -- 172.21.15.86:0/1816744443 wait complete. 2024-04-20T21:28:53.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.779+0000 7f934785c740 20 client.8991 trim_cache size 0 max 16384 2024-04-20T21:28:53.783 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:28:53.783 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "9c2dec1d-d7ae-4a83-a812-79922808e278", 2024-04-20T21:28:53.783 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-04-20T21:28:53.783 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 8922, 2024-04-20T21:28:53.783 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-20T21:28:53.783 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-20T21:28:53.783 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-20T21:28:53.783 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 4.9179812390000004, 2024-04-20T21:28:53.783 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 88, 2024-04-20T21:28:53.783 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 101, 2024-04-20T21:28:53.783 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 101, 2024-04-20T21:28:53.784 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 5.8532332299999998 2024-04-20T21:28:53.784 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:28:53.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.780+0000 7f9344c00640 1 -- 172.21.15.86:0/2210108185 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f931c06c8f0 msgr2=0x7f931c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:53.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.780+0000 7f9344c00640 1 --2- 172.21.15.86:0/2210108185 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f931c06c8f0 0x7f931c06edb0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f9330000f60 tx=0x7f9330009040 comp rx=0 tx=0).stop 2024-04-20T21:28:53.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.780+0000 7f9344c00640 1 -- 172.21.15.86:0/2210108185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93400d16e0 msgr2=0x7f93401f7fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:53.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.780+0000 7f9344c00640 1 --2- 172.21.15.86:0/2210108185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93400d16e0 0x7f93401f7fd0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f9328009f60 tx=0x7f9328009f90 comp rx=0 tx=0).stop 2024-04-20T21:28:53.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.780+0000 7f9344c00640 1 -- 172.21.15.86:0/2210108185 shutdown_connections 2024-04-20T21:28:53.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.780+0000 7f9344c00640 1 --2- 172.21.15.86:0/2210108185 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f931c06c8f0 0x7f931c06edb0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:53.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.780+0000 7f9344c00640 1 --2- 172.21.15.86:0/2210108185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f934015ed80 0x7f93400cd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:53.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.780+0000 7f9344c00640 1 --2- 172.21.15.86:0/2210108185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93400d16e0 0x7f93401f7fd0 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:53.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.780+0000 7f9344c00640 1 --2- 172.21.15.86:0/2210108185 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93400cf810 0x7f93401f51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:53.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.780+0000 7f9344c00640 1 -- 172.21.15.86:0/2210108185 >> 172.21.15.86:0/2210108185 conn(0x7f934006cc20 msgr2=0x7f93400ca720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:53.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.780+0000 7f9344c00640 1 -- 172.21.15.86:0/2210108185 shutdown_connections 2024-04-20T21:28:53.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.780+0000 7f9344c00640 1 -- 172.21.15.86:0/2210108185 wait complete. 2024-04-20T21:28:53.797 DEBUG:teuthology.orchestra.run.smithi086:> sudo daemon-helper kill ceph --cluster ceph -w 2024-04-20T21:28:53.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:28:53.803 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-04-20T21:28:53.863 INFO:tasks.ceph.mds.c.smithi086.stderr:2024-04-20T21:28:53.863+0000 7ff2f3600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T21:28:53.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.981+0000 7f0e59000640 1 Processor -- start 2024-04-20T21:28:53.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.981+0000 7f0e59000640 1 -- start start 2024-04-20T21:28:53.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.982+0000 7f0e59000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0e5415e840 0x7f0e540c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:53.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.982+0000 7f0e59000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0e5415ec20 0x7f0e54163080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:53.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.982+0000 7f0e59000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e540d2d30 0x7f0e540d4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:53.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.982+0000 7f0e59000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f0e540d3510 con 0x7f0e5415e840 2024-04-20T21:28:53.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.982+0000 7f0e59000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f0e540cf5b0 con 0x7f0e5415ec20 2024-04-20T21:28:53.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.982+0000 7f0e59000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0e540cf890 con 0x7f0e540d2d30 2024-04-20T21:28:53.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.983+0000 7f0e51e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0e5415e840 0x7f0e540c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:53.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.983+0000 7f0e51e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0e5415e840 0x7f0e540c82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45958/0 (socket says 172.21.15.86:45958) 2024-04-20T21:28:53.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.983+0000 7f0e51e00640 1 -- 172.21.15.86:0/1343244514 learned_addr learned my addr 172.21.15.86:0/1343244514 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:53.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.983+0000 7f0e52800640 1 --2- 172.21.15.86:0/1343244514 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e540d2d30 0x7f0e540d4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:53.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.983+0000 7f0e4be00640 1 --2- 172.21.15.86:0/1343244514 >> v2:172.21.15.144:3300/0 conn(0x7f0e5415ec20 0x7f0e54163080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:53.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.983+0000 7f0e51e00640 1 -- 172.21.15.86:0/1343244514 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e540d2d30 msgr2=0x7f0e540d4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:53.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.983+0000 7f0e51e00640 1 --2- 172.21.15.86:0/1343244514 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e540d2d30 0x7f0e540d4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:53.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.983+0000 7f0e51e00640 1 -- 172.21.15.86:0/1343244514 >> v2:172.21.15.144:3300/0 conn(0x7f0e5415ec20 msgr2=0x7f0e54163080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:53.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.983+0000 7f0e51e00640 1 --2- 172.21.15.86:0/1343244514 >> v2:172.21.15.144:3300/0 conn(0x7f0e5415ec20 0x7f0e54163080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:53.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.983+0000 7f0e51e00640 1 -- 172.21.15.86:0/1343244514 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e540cfb70 con 0x7f0e5415e840 2024-04-20T21:28:53.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.984+0000 7f0e51e00640 1 --2- 172.21.15.86:0/1343244514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e5415e840 0x7f0e540c82a0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f0e54162500 tx=0x7f0e4402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=761fa5ec570fbd46 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:53.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.985+0000 7f0e51400640 1 -- 172.21.15.86:0/1343244514 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e44035020 con 0x7f0e5415e840 2024-04-20T21:28:53.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.985+0000 7f0e51400640 1 -- 172.21.15.86:0/1343244514 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e44031870 con 0x7f0e5415e840 2024-04-20T21:28:53.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.985+0000 7f0e51400640 1 -- 172.21.15.86:0/1343244514 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e44031c40 con 0x7f0e5415e840 2024-04-20T21:28:53.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.985+0000 7f0e59000640 1 -- 172.21.15.86:0/1343244514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e5415e840 msgr2=0x7f0e540c82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:53.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.985+0000 7f0e59000640 1 --2- 172.21.15.86:0/1343244514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e5415e840 0x7f0e540c82a0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f0e54162500 tx=0x7f0e4402b910 comp rx=0 tx=0).stop 2024-04-20T21:28:53.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.985+0000 7f0e59000640 1 -- 172.21.15.86:0/1343244514 shutdown_connections 2024-04-20T21:28:53.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.985+0000 7f0e59000640 1 --2- 172.21.15.86:0/1343244514 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e540d2d30 0x7f0e540d4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:53.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.985+0000 7f0e59000640 1 --2- 172.21.15.86:0/1343244514 >> v2:172.21.15.144:3300/0 conn(0x7f0e5415ec20 0x7f0e54163080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:53.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.985+0000 7f0e59000640 1 --2- 172.21.15.86:0/1343244514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e5415e840 0x7f0e540c82a0 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:53.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.985+0000 7f0e59000640 1 -- 172.21.15.86:0/1343244514 >> 172.21.15.86:0/1343244514 conn(0x7f0e5409da40 msgr2=0x7f0e5408bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:53.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.986+0000 7f0e59000640 1 -- 172.21.15.86:0/1343244514 shutdown_connections 2024-04-20T21:28:53.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.986+0000 7f0e59000640 1 -- 172.21.15.86:0/1343244514 wait complete. 2024-04-20T21:28:53.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.987+0000 7f0e59000640 1 Processor -- start 2024-04-20T21:28:53.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.987+0000 7f0e59000640 1 -- start start 2024-04-20T21:28:53.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.988+0000 7f0e59000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e540d2d30 0x7f0e540a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:53.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.988+0000 7f0e59000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e5415e840 0x7f0e54153940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:53.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.988+0000 7f0e59000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e5415ec20 0x7f0e540b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:53.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.988+0000 7f0e59000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0e540d4940 con 0x7f0e5415ec20 2024-04-20T21:28:53.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.988+0000 7f0e59000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0e540d3180 con 0x7f0e5415e840 2024-04-20T21:28:53.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.988+0000 7f0e59000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0e540d4ac0 con 0x7f0e540d2d30 2024-04-20T21:28:53.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.989+0000 7f0e52800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e5415ec20 0x7f0e540b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:53.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.989+0000 7f0e52800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e5415ec20 0x7f0e540b1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45966/0 (socket says 172.21.15.86:45966) 2024-04-20T21:28:53.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.989+0000 7f0e52800640 1 -- 172.21.15.86:0/2693049484 learned_addr learned my addr 172.21.15.86:0/2693049484 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:53.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.989+0000 7f0e51e00640 1 --2- 172.21.15.86:0/2693049484 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e540d2d30 0x7f0e540a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:53.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.989+0000 7f0e4be00640 1 --2- 172.21.15.86:0/2693049484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e5415e840 0x7f0e54153940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:53.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.989+0000 7f0e52800640 1 -- 172.21.15.86:0/2693049484 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e540d2d30 msgr2=0x7f0e540a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:53.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.989+0000 7f0e52800640 1 --2- 172.21.15.86:0/2693049484 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e540d2d30 0x7f0e540a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:53.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.989+0000 7f0e52800640 1 -- 172.21.15.86:0/2693049484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e5415e840 msgr2=0x7f0e54153940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:53.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.989+0000 7f0e52800640 1 --2- 172.21.15.86:0/2693049484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e5415e840 0x7f0e54153940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:53.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.989+0000 7f0e52800640 1 -- 172.21.15.86:0/2693049484 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e540a03c0 con 0x7f0e5415ec20 2024-04-20T21:28:53.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.989+0000 7f0e4be00640 1 --2- 172.21.15.86:0/2693049484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e5415e840 0x7f0e54153940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:28:53.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.989+0000 7f0e52800640 1 --2- 172.21.15.86:0/2693049484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e5415ec20 0x7f0e540b1db0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c00cd70 tx=0x7f0e4c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:53.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.990+0000 7f0e4a000640 1 -- 172.21.15.86:0/2693049484 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c003b90 con 0x7f0e5415ec20 2024-04-20T21:28:53.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.990+0000 7f0e4a000640 1 -- 172.21.15.86:0/2693049484 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e4c003d30 con 0x7f0e5415ec20 2024-04-20T21:28:53.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.990+0000 7f0e4a000640 1 -- 172.21.15.86:0/2693049484 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c004240 con 0x7f0e5415ec20 2024-04-20T21:28:53.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.990+0000 7f0e59000640 1 -- 172.21.15.86:0/2693049484 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e540a0660 con 0x7f0e5415ec20 2024-04-20T21:28:53.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.990+0000 7f0e59000640 1 -- 172.21.15.86:0/2693049484 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e540a0c20 con 0x7f0e5415ec20 2024-04-20T21:28:53.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.992+0000 7f0e4a000640 1 -- 172.21.15.86:0/2693049484 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0e4c0043e0 con 0x7f0e5415ec20 2024-04-20T21:28:53.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.993+0000 7f0e4a000640 1 --2- 172.21.15.86:0/2693049484 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0e3006c8f0 0x7f0e3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:53.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.993+0000 7f0e4a000640 1 -- 172.21.15.86:0/2693049484 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5795+0+0 (secure 0 0 0) 0x7f0e4c08c730 con 0x7f0e5415ec20 2024-04-20T21:28:53.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.993+0000 7f0e51e00640 1 --2- 172.21.15.86:0/2693049484 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0e3006c8f0 0x7f0e3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:53.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.994+0000 7f0e5bdb4740 1 Processor -- start 2024-04-20T21:28:53.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.994+0000 7f0e5bdb4740 1 -- start start 2024-04-20T21:28:53.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.994+0000 7f0e51e00640 1 --2- 172.21.15.86:0/2693049484 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0e3006c8f0 0x7f0e3006edb0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f0e44036000 tx=0x7f0e4403c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:53.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.995+0000 7f0e5bdb4740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f608c48fe0 0x55f608c1bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:53.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.995+0000 7f0e5bdb4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f608c493c0 0x55f608c4bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:53.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.995+0000 7f0e5bdb4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f608c1c4c0 0x55f608c4c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:53.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.995+0000 7f0e5bdb4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f608b86d20 con 0x55f608c1c4c0 2024-04-20T21:28:53.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.995+0000 7f0e5bdb4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f608b2d3e0 con 0x55f608c493c0 2024-04-20T21:28:53.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.995+0000 7f0e5bdb4740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55f608b82ce0 con 0x55f608c48fe0 2024-04-20T21:28:53.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.995+0000 7f0e52800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f608c1c4c0 0x55f608c4c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:53.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.995+0000 7f0e52800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f608c1c4c0 0x55f608c4c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45968/0 (socket says 172.21.15.86:45968) 2024-04-20T21:28:53.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.995+0000 7f0e52800640 1 -- 172.21.15.86:0/3500329176 learned_addr learned my addr 172.21.15.86:0/3500329176 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:53.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.995+0000 7f0e51e00640 1 --2- 172.21.15.86:0/3500329176 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f608c48fe0 0x55f608c1bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:53.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.995+0000 7f0e4be00640 1 --2- 172.21.15.86:0/3500329176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f608c493c0 0x55f608c4bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:53.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.995+0000 7f0e52800640 1 -- 172.21.15.86:0/3500329176 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f608c48fe0 msgr2=0x55f608c1bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:53.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.995+0000 7f0e52800640 1 --2- 172.21.15.86:0/3500329176 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f608c48fe0 0x55f608c1bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:53.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.995+0000 7f0e52800640 1 -- 172.21.15.86:0/3500329176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f608c493c0 msgr2=0x55f608c4bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:53.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.995+0000 7f0e52800640 1 --2- 172.21.15.86:0/3500329176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f608c493c0 0x55f608c4bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:53.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.995+0000 7f0e52800640 1 -- 172.21.15.86:0/3500329176 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f608b4c980 con 0x55f608c1c4c0 2024-04-20T21:28:53.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.996+0000 7f0e4be00640 1 --2- 172.21.15.86:0/3500329176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f608c493c0 0x55f608c4bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:28:53.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.996+0000 7f0e52800640 1 --2- 172.21.15.86:0/3500329176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f608c1c4c0 0x55f608c4c2b0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c004b30 tx=0x7f0e4c05aa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a836f95ec186035 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:53.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.996+0000 7f0e42a00640 1 -- 172.21.15.86:0/3500329176 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c089070 con 0x55f608c1c4c0 2024-04-20T21:28:53.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.996+0000 7f0e42a00640 1 -- 172.21.15.86:0/3500329176 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e4c062f40 con 0x55f608c1c4c0 2024-04-20T21:28:53.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.996+0000 7f0e42a00640 1 -- 172.21.15.86:0/3500329176 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c0046d0 con 0x55f608c1c4c0 2024-04-20T21:28:54.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.997+0000 7f0e5bdb4740 1 -- 172.21.15.86:0/3500329176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f608c1c4c0 msgr2=0x55f608c4c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:54.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.997+0000 7f0e5bdb4740 1 --2- 172.21.15.86:0/3500329176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f608c1c4c0 0x55f608c4c2b0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c004b30 tx=0x7f0e4c05aa60 comp rx=0 tx=0).stop 2024-04-20T21:28:54.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.997+0000 7f0e5bdb4740 1 -- 172.21.15.86:0/3500329176 shutdown_connections 2024-04-20T21:28:54.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.997+0000 7f0e5bdb4740 1 --2- 172.21.15.86:0/3500329176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f608c1c4c0 0x55f608c4c2b0 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.997+0000 7f0e5bdb4740 1 --2- 172.21.15.86:0/3500329176 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f608c493c0 0x55f608c4bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.997+0000 7f0e5bdb4740 1 --2- 172.21.15.86:0/3500329176 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f608c48fe0 0x55f608c1bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.997+0000 7f0e5bdb4740 1 -- 172.21.15.86:0/3500329176 >> 172.21.15.86:0/3500329176 conn(0x55f608c1e1f0 msgr2=0x55f608c66d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:54.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.997+0000 7f0e5bdb4740 1 -- 172.21.15.86:0/3500329176 shutdown_connections 2024-04-20T21:28:54.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.997+0000 7f0e5bdb4740 1 -- 172.21.15.86:0/3500329176 wait complete. 2024-04-20T21:28:54.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.998+0000 7f0e5bdb4740 1 -- start start 2024-04-20T21:28:54.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.998+0000 7f0e5bdb4740 1 Processor -- start 2024-04-20T21:28:54.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.999+0000 7f0e5bdb4740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f608c48fe0 0x55f608c48090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:54.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.999+0000 7f0e5bdb4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f608c493c0 0x55f608c4ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:54.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.999+0000 7f0e5bdb4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f608c42ee0 0x55f608c77790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:54.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.999+0000 7f0e5bdb4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f608c20890 con 0x55f608c493c0 2024-04-20T21:28:54.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.999+0000 7f0e5bdb4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55f608b407a0 con 0x55f608c42ee0 2024-04-20T21:28:54.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.999+0000 7f0e5bdb4740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55f608c230d0 con 0x55f608c48fe0 2024-04-20T21:28:54.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.999+0000 7f0e5bdb4740 10 client.0 _ll_register_callbacks cb 0x55f608b42b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:28:54.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.999+0000 7f0e4be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f608c493c0 0x55f608c4ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:54.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.999+0000 7f0e4be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f608c493c0 0x55f608c4ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45984/0 (socket says 172.21.15.86:45984) 2024-04-20T21:28:54.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.999+0000 7f0e4be00640 1 -- 172.21.15.86:0/1568672409 learned_addr learned my addr 172.21.15.86:0/1568672409 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:54.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.999+0000 7f0e4be00640 1 -- 172.21.15.86:0/1568672409 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f608c48fe0 msgr2=0x55f608c48090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:54.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:53.999+0000 7f0e4be00640 1 --2- 172.21.15.86:0/1568672409 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f608c48fe0 0x55f608c48090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.000+0000 7f0e4be00640 1 -- 172.21.15.86:0/1568672409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f608c42ee0 msgr2=0x55f608c77790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:54.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.000+0000 7f0e4be00640 1 --2- 172.21.15.86:0/1568672409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f608c42ee0 0x55f608c77790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.000+0000 7f0e4be00640 1 -- 172.21.15.86:0/1568672409 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6089617e0 con 0x55f608c493c0 2024-04-20T21:28:54.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.000+0000 7f0e4be00640 1 --2- 172.21.15.86:0/1568672409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f608c493c0 0x55f608c4ea80 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c00cfd0 tx=0x7f0e3c00e740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b7992962728c76f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:54.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.000+0000 7f0e37400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:28:54.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.000+0000 7f0e37400640 1 -- 172.21.15.86:0/1568672409 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c012840 con 0x55f608c493c0 2024-04-20T21:28:54.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.000+0000 7f0e37400640 1 -- 172.21.15.86:0/1568672409 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 5795+0+0 (secure 0 0 0) 0x7f0e3c0129e0 con 0x55f608c493c0 2024-04-20T21:28:54.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.000+0000 7f0e5bdb4740 1 -- 172.21.15.86:0/1568672409 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e4c062f40 con 0x55f608c493c0 2024-04-20T21:28:54.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.000+0000 7f0e37400640 1 -- 172.21.15.86:0/1568672409 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f0e3c022050 con 0x55f608c493c0 2024-04-20T21:28:54.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.000+0000 7f0e5bdb4740 10 client.9003 fetch_fsmap learned FSMap version 88 2024-04-20T21:28:54.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.000+0000 7f0e5bdb4740 1 -- 172.21.15.86:0/1568672409 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x55f6089c1f70 con 0x55f608c493c0 2024-04-20T21:28:54.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.001+0000 7f0e37400640 1 -- 172.21.15.86:0/1568672409 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e4c062f40 con 0x55f608c493c0 2024-04-20T21:28:54.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.001+0000 7f0e37400640 1 -- 172.21.15.86:0/1568672409 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 88) ==== 3495+0+0 (secure 0 0 0) 0x7f0e3c012be0 con 0x55f608c493c0 2024-04-20T21:28:54.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.001+0000 7f0e5bdb4740 10 client.9003 fetch_fsmap finished waiting for FSMap version 88 2024-04-20T21:28:54.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.001+0000 7f0e5bdb4740 10 client.9003 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-20T21:28:54.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.001+0000 7f0e5bdb4740 20 client.9003 populate_metadata read hostname 'smithi086' 2024-04-20T21:28:54.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.001+0000 7f0e5bdb4740 1 --2- 172.21.15.86:0/1568672409 >> [v2:172.21.15.86:6834/4271041811,v1:172.21.15.86:6836/4271041811] conn(0x55f608c67350 0x55f608c41c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:54.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.001+0000 7f0e5bdb4740 4 client.9003 mds_command: new command op to 8922 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:28:54.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.001+0000 7f0e5bdb4740 1 -- 172.21.15.86:0/1568672409 --> [v2:172.21.15.86:6834/4271041811,v1:172.21.15.86:6836/4271041811] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f608b94740 con 0x55f608c67350 2024-04-20T21:28:54.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.001+0000 7f0e51e00640 1 --2- 172.21.15.86:0/1568672409 >> [v2:172.21.15.86:6834/4271041811,v1:172.21.15.86:6836/4271041811] conn(0x55f608c67350 0x55f608c41c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:54.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.002+0000 7f0e51e00640 1 --2- 172.21.15.86:0/1568672409 >> [v2:172.21.15.86:6834/4271041811,v1:172.21.15.86:6836/4271041811] conn(0x55f608c67350 0x55f608c41c30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b6a78c92897cd96 server_cookie=3513c945f7b7b9fa in_seq=0 out_seq=0 2024-04-20T21:28:54.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.002+0000 7f0e37400640 10 client.9003 ms_handle_connect on v2:172.21.15.86:6834/4271041811 2024-04-20T21:28:54.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.002+0000 7f0e37400640 1 -- 172.21.15.86:0/1568672409 <== mds.0 v2:172.21.15.86:6834/4271041811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f608b94740 con 0x55f608c67350 2024-04-20T21:28:54.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.002+0000 7f0e37400640 10 client.9003 handle_command_reply: tid=0 2024-04-20T21:28:54.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.002+0000 7f0e5bdb4740 1 client.9003 shutdown 2024-04-20T21:28:54.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.003+0000 7f0e5bdb4740 1 -- 172.21.15.86:0/1568672409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f608c493c0 msgr2=0x55f608c4ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:54.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.003+0000 7f0e5bdb4740 1 --2- 172.21.15.86:0/1568672409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f608c493c0 0x55f608c4ea80 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c00cfd0 tx=0x7f0e3c00e740 comp rx=0 tx=0).stop 2024-04-20T21:28:54.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.003+0000 7f0e5bdb4740 1 -- 172.21.15.86:0/1568672409 shutdown_connections 2024-04-20T21:28:54.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.003+0000 7f0e5bdb4740 1 --2- 172.21.15.86:0/1568672409 >> [v2:172.21.15.86:6834/4271041811,v1:172.21.15.86:6836/4271041811] conn(0x55f608c67350 0x55f608c41c30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.003+0000 7f0e5bdb4740 1 --2- 172.21.15.86:0/1568672409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f608c42ee0 0x55f608c77790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.003+0000 7f0e5bdb4740 1 --2- 172.21.15.86:0/1568672409 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f608c493c0 0x55f608c4ea80 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.003+0000 7f0e5bdb4740 1 --2- 172.21.15.86:0/1568672409 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f608c48fe0 0x55f608c48090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.003+0000 7f0e37400640 0 client.9003 ms_handle_reset on v2:172.21.15.86:6834/4271041811 2024-04-20T21:28:54.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.003+0000 7f0e5bdb4740 1 -- 172.21.15.86:0/1568672409 >> 172.21.15.86:0/1568672409 conn(0x55f608c1c4c0 msgr2=0x55f608c1ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:54.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.003+0000 7f0e5bdb4740 1 -- 172.21.15.86:0/1568672409 shutdown_connections 2024-04-20T21:28:54.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.003+0000 7f0e5bdb4740 1 -- 172.21.15.86:0/1568672409 wait complete. 2024-04-20T21:28:54.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.003+0000 7f0e5bdb4740 20 client.9003 trim_cache size 0 max 16384 2024-04-20T21:28:54.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.005+0000 7f5d25200640 1 Processor -- start 2024-04-20T21:28:54.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.005+0000 7f5d25200640 1 -- start start 2024-04-20T21:28:54.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.006+0000 7f5d25200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5d2015ea00 0x7f5d20163070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:54.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.006+0000 7f5d25200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d200d3ef0 0x7f5d200d1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:54.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.006+0000 7f5d25200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d200cf050 con 0x7f5d200d1110 2024-04-20T21:28:54.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.006+0000 7f5d25200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f5d200d1ad0 con 0x7f5d2015ea00 2024-04-20T21:28:54.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.006+0000 7f5d25200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5d201635b0 con 0x7f5d200d3ef0 2024-04-20T21:28:54.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.006+0000 7f5d17e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f5d200d1110 0x7f5d200b6270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:45904/0 (socket says 172.21.15.86:45904) 2024-04-20T21:28:54.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.006+0000 7f5d17e00640 1 -- 172.21.15.86:0/4127784258 learned_addr learned my addr 172.21.15.86:0/4127784258 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:28:54.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.006+0000 7f5d1de00640 1 --2- 172.21.15.86:0/4127784258 >> v2:172.21.15.144:3300/0 conn(0x7f5d2015ea00 0x7f5d20163070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:54.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.006+0000 7f5d1e800640 1 --2- 172.21.15.86:0/4127784258 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d200d3ef0 0x7f5d200d1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:54.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.007+0000 7f5d1d400640 1 -- 172.21.15.86:0/4127784258 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 742654090 0 0) 0x7f5d200cf050 con 0x7f5d200d1110 2024-04-20T21:28:54.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.007+0000 7f5d1d400640 1 -- 172.21.15.86:0/4127784258 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5d04003520 con 0x7f5d200d1110 2024-04-20T21:28:54.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.007+0000 7f5d1d400640 1 -- 172.21.15.86:0/4127784258 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2391978991 0 0) 0x7f5d04003520 con 0x7f5d200d1110 2024-04-20T21:28:54.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.007+0000 7f5d1d400640 1 -- 172.21.15.86:0/4127784258 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d200d3ef0 msgr2=0x7f5d200d1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:54.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.007+0000 7f5d1d400640 1 --2- 172.21.15.86:0/4127784258 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d200d3ef0 0x7f5d200d1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.007+0000 7f5d1d400640 1 -- 172.21.15.86:0/4127784258 >> v2:172.21.15.144:3300/0 conn(0x7f5d2015ea00 msgr2=0x7f5d20163070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:54.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.007+0000 7f5d1d400640 1 --2- 172.21.15.86:0/4127784258 >> v2:172.21.15.144:3300/0 conn(0x7f5d2015ea00 0x7f5d20163070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.007+0000 7f5d1d400640 1 -- 172.21.15.86:0/4127784258 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d200cf330 con 0x7f5d200d1110 2024-04-20T21:28:54.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.007+0000 7f5d1d400640 1 -- 172.21.15.86:0/4127784258 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f5d08001670 con 0x7f5d200d1110 2024-04-20T21:28:54.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.007+0000 7f5d1d400640 1 -- 172.21.15.86:0/4127784258 >> v1:172.21.15.86:6789/0 conn(0x7f5d200d1110 legacy=0x7f5d200b6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:54.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.007+0000 7f5d1d400640 1 --2- 172.21.15.86:0/4127784258 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d040039b0 0x7f5d04003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:54.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.007+0000 7f5d1d400640 1 --2- 172.21.15.86:0/4127784258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d04004800 0x7f5d04004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:54.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.007+0000 7f5d1d400640 1 --2- 172.21.15.86:0/4127784258 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d200d3ef0 0x7f5d040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:54.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.007+0000 7f5d1d400640 1 -- 172.21.15.86:0/4127784258 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5d201635b0 con 0x7f5d04004800 2024-04-20T21:28:54.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.007+0000 7f5d1d400640 1 -- 172.21.15.86:0/4127784258 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0400b9c0 con 0x7f5d040039b0 2024-04-20T21:28:54.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.007+0000 7f5d1d400640 1 -- 172.21.15.86:0/4127784258 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5d0400bb40 con 0x7f5d200d3ef0 2024-04-20T21:28:54.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.007+0000 7f5d17e00640 1 --2- 172.21.15.86:0/4127784258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d04004800 0x7f5d04004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:54.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.008+0000 7f5d1e800640 1 --2- 172.21.15.86:0/4127784258 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d200d3ef0 0x7f5d040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:54.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.008+0000 7f5d17e00640 1 -- 172.21.15.86:0/4127784258 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d200d3ef0 msgr2=0x7f5d040051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:54.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.008+0000 7f5d17e00640 1 --2- 172.21.15.86:0/4127784258 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d200d3ef0 0x7f5d040051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.008+0000 7f5d1de00640 1 --2- 172.21.15.86:0/4127784258 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d040039b0 0x7f5d04003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:54.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.008+0000 7f5d17e00640 1 -- 172.21.15.86:0/4127784258 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d040039b0 msgr2=0x7f5d04003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:54.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.008+0000 7f5d17e00640 1 --2- 172.21.15.86:0/4127784258 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d040039b0 0x7f5d04003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.008+0000 7f5d17e00640 1 -- 172.21.15.86:0/4127784258 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d04003520 con 0x7f5d04004800 2024-04-20T21:28:54.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.009+0000 7f5d17e00640 1 --2- 172.21.15.86:0/4127784258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d04004800 0x7f5d04004c80 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f5d0800c190 tx=0x7f5d04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df184d78ecaec54e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:54.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.009+0000 7f5d1d400640 1 -- 172.21.15.86:0/4127784258 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d08007c80 con 0x7f5d04004800 2024-04-20T21:28:54.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.009+0000 7f5d1d400640 1 -- 172.21.15.86:0/4127784258 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d08030070 con 0x7f5d04004800 2024-04-20T21:28:54.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.009+0000 7f5d1d400640 1 -- 172.21.15.86:0/4127784258 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d08005d30 con 0x7f5d04004800 2024-04-20T21:28:54.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.010+0000 7f5d25200640 1 -- 172.21.15.86:0/4127784258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d04004800 msgr2=0x7f5d04004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:54.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.010+0000 7f5d25200640 1 --2- 172.21.15.86:0/4127784258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d04004800 0x7f5d04004c80 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f5d0800c190 tx=0x7f5d04005830 comp rx=0 tx=0).stop 2024-04-20T21:28:54.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.010+0000 7f5d1e800640 1 -- 172.21.15.86:0/4127784258 reap_dead start 2024-04-20T21:28:54.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.010+0000 7f5d25200640 1 -- 172.21.15.86:0/4127784258 shutdown_connections 2024-04-20T21:28:54.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.010+0000 7f5d25200640 1 -- 172.21.15.86:0/4127784258 >> 172.21.15.86:0/4127784258 conn(0x7f5d2006cc90 msgr2=0x7f5d2006d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:54.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.010+0000 7f5d25200640 1 -- 172.21.15.86:0/4127784258 shutdown_connections 2024-04-20T21:28:54.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.010+0000 7f5d25200640 1 -- 172.21.15.86:0/4127784258 wait complete. 2024-04-20T21:28:54.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.012+0000 7f5d25200640 1 Processor -- start 2024-04-20T21:28:54.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.012+0000 7f5d25200640 1 -- start start 2024-04-20T21:28:54.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.013+0000 7f5d25200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d04004800 0x7f5d200d3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:54.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.013+0000 7f5d25200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d200ce4e0 0x7f5d201f5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:54.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.013+0000 7f5d25200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d200ce960 0x7f5d201f5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:54.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.013+0000 7f5d25200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5d200cf610 con 0x7f5d04004800 2024-04-20T21:28:54.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.013+0000 7f5d25200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5d200cff40 con 0x7f5d200ce4e0 2024-04-20T21:28:54.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.013+0000 7f5d25200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5d200cf8f0 con 0x7f5d200ce960 2024-04-20T21:28:54.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.013+0000 7f5d1de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d04004800 0x7f5d200d3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:54.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.013+0000 7f5d1de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d04004800 0x7f5d200d3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45996/0 (socket says 172.21.15.86:45996) 2024-04-20T21:28:54.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.013+0000 7f5d1de00640 1 -- 172.21.15.86:0/1897508667 learned_addr learned my addr 172.21.15.86:0/1897508667 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:54.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.014+0000 7f5d1de00640 1 -- 172.21.15.86:0/1897508667 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d200ce960 msgr2=0x7f5d201f5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:54.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.014+0000 7f5d1e800640 1 --2- 172.21.15.86:0/1897508667 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d200ce960 0x7f5d201f5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:54.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.014+0000 7f5d1de00640 1 --2- 172.21.15.86:0/1897508667 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5d200ce960 0x7f5d201f5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.014+0000 7f5d1de00640 1 -- 172.21.15.86:0/1897508667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d200ce4e0 msgr2=0x7f5d201f5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:54.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.014+0000 7f5d1de00640 1 --2- 172.21.15.86:0/1897508667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d200ce4e0 0x7f5d201f5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.014+0000 7f5d1de00640 1 -- 172.21.15.86:0/1897508667 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d201f4ce0 con 0x7f5d04004800 2024-04-20T21:28:54.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.014+0000 7f5d1de00640 1 --2- 172.21.15.86:0/1897508667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d04004800 0x7f5d200d3050 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f5d10010910 tx=0x7f5d10010dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:54.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.014+0000 7f5d16000640 1 -- 172.21.15.86:0/1897508667 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1000f040 con 0x7f5d04004800 2024-04-20T21:28:54.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.014+0000 7f5d16000640 1 -- 172.21.15.86:0/1897508667 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d1000ec20 con 0x7f5d04004800 2024-04-20T21:28:54.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.014+0000 7f5d25200640 1 -- 172.21.15.86:0/1897508667 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d200cda00 con 0x7f5d04004800 2024-04-20T21:28:54.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.014+0000 7f5d16000640 1 -- 172.21.15.86:0/1897508667 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1001c400 con 0x7f5d04004800 2024-04-20T21:28:54.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.014+0000 7f5d25200640 1 -- 172.21.15.86:0/1897508667 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d200cdfc0 con 0x7f5d04004800 2024-04-20T21:28:54.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.015+0000 7f5d25200640 1 -- 172.21.15.86:0/1897508667 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f5d200689e0 con 0x7f5d04004800 2024-04-20T21:28:54.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.017+0000 7f5d16000640 1 -- 172.21.15.86:0/1897508667 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5d1001c5a0 con 0x7f5d04004800 2024-04-20T21:28:54.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.018+0000 7f5d16000640 1 --2- 172.21.15.86:0/1897508667 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5cfc06c8f0 0x7f5cfc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:54.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.018+0000 7f5d16000640 1 -- 172.21.15.86:0/1897508667 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5795+0+0 (secure 0 0 0) 0x7f5d1008c7f0 con 0x7f5d04004800 2024-04-20T21:28:54.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.018+0000 7f5d17e00640 1 --2- 172.21.15.86:0/1897508667 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5cfc06c8f0 0x7f5cfc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:54.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.018+0000 7f5d16000640 1 -- 172.21.15.86:0/1897508667 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+516 (secure 0 0 0) 0x7f5d1002b070 con 0x7f5d04004800 2024-04-20T21:28:54.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.018+0000 7f5d25200640 1 -- 172.21.15.86:0/1897508667 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7f5d200d2570 con 0x7f5d04004800 2024-04-20T21:28:54.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.019+0000 7f5d17e00640 1 --2- 172.21.15.86:0/1897508667 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5cfc06c8f0 0x7f5cfc06edb0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f5d08034000 tx=0x7f5d0802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:54.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.024+0000 7f0e5bdb4740 1 Processor -- start 2024-04-20T21:28:54.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.024+0000 7f0e5bdb4740 1 -- start start 2024-04-20T21:28:54.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.025+0000 7f0e5bdb4740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f608c42ee0 0x55f608c1d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:54.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.025+0000 7f0e5bdb4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f608c48fe0 0x55f608c69ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:54.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.025+0000 7f0e5bdb4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f608c493c0 0x55f608c4b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:54.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.025+0000 7f0e5bdb4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0e54201030 con 0x55f608c48fe0 2024-04-20T21:28:54.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.025+0000 7f0e5bdb4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0e542001d0 con 0x55f608c493c0 2024-04-20T21:28:54.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.025+0000 7f0e5bdb4740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0e541f80a0 con 0x55f608c42ee0 2024-04-20T21:28:54.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.025+0000 7f0e4be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f608c48fe0 0x55f608c69ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:54.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.025+0000 7f0e4be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f608c48fe0 0x55f608c69ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46004/0 (socket says 172.21.15.86:46004) 2024-04-20T21:28:54.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.025+0000 7f0e4be00640 1 -- 172.21.15.86:0/1619391597 learned_addr learned my addr 172.21.15.86:0/1619391597 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:54.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.025+0000 7f0e4be00640 1 -- 172.21.15.86:0/1619391597 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f608c42ee0 msgr2=0x55f608c1d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:54.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.025+0000 7f0e4be00640 1 --2- 172.21.15.86:0/1619391597 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f608c42ee0 0x55f608c1d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.025+0000 7f0e4be00640 1 -- 172.21.15.86:0/1619391597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f608c493c0 msgr2=0x55f608c4b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:54.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.025+0000 7f0e4be00640 1 --2- 172.21.15.86:0/1619391597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f608c493c0 0x55f608c4b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.025+0000 7f0e4be00640 1 -- 172.21.15.86:0/1619391597 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f608c475a0 con 0x55f608c48fe0 2024-04-20T21:28:54.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.026+0000 7f0e4be00640 1 --2- 172.21.15.86:0/1619391597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f608c48fe0 0x55f608c69ae0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c00c940 tx=0x7f0e3c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af6ae23ce6ad5dc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:54.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.026+0000 7f0e42000640 1 -- 172.21.15.86:0/1619391597 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c01f070 con 0x55f608c48fe0 2024-04-20T21:28:54.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.026+0000 7f0e42000640 1 -- 172.21.15.86:0/1619391597 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e3c012860 con 0x55f608c48fe0 2024-04-20T21:28:54.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.026+0000 7f0e42000640 1 -- 172.21.15.86:0/1619391597 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c012c90 con 0x55f608c48fe0 2024-04-20T21:28:54.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.027+0000 7f0e5bdb4740 1 -- 172.21.15.86:0/1619391597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f608c48fe0 msgr2=0x55f608c69ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:54.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.027+0000 7f0e5bdb4740 1 --2- 172.21.15.86:0/1619391597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f608c48fe0 0x55f608c69ae0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c00c940 tx=0x7f0e3c013000 comp rx=0 tx=0).stop 2024-04-20T21:28:54.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.027+0000 7f0e5bdb4740 1 -- 172.21.15.86:0/1619391597 shutdown_connections 2024-04-20T21:28:54.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.027+0000 7f0e5bdb4740 1 --2- 172.21.15.86:0/1619391597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f608c493c0 0x55f608c4b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.027+0000 7f0e5bdb4740 1 --2- 172.21.15.86:0/1619391597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f608c48fe0 0x55f608c69ae0 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.027+0000 7f0e5bdb4740 1 --2- 172.21.15.86:0/1619391597 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f608c42ee0 0x55f608c1d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.027+0000 7f0e5bdb4740 1 -- 172.21.15.86:0/1619391597 >> 172.21.15.86:0/1619391597 conn(0x55f608c1c4c0 msgr2=0x55f608c774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:54.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.027+0000 7f0e5bdb4740 1 -- 172.21.15.86:0/1619391597 shutdown_connections 2024-04-20T21:28:54.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.027+0000 7f0e5bdb4740 1 -- 172.21.15.86:0/1619391597 wait complete. 2024-04-20T21:28:54.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.028+0000 7f0e5bdb4740 1 -- start start 2024-04-20T21:28:54.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.028+0000 7f0e5bdb4740 1 Processor -- start 2024-04-20T21:28:54.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.029+0000 7f0e5bdb4740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f608c42ee0 0x55f608ce2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:54.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.029+0000 7f0e5bdb4740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f608c48fe0 0x55f608c8ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:54.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.029+0000 7f0e5bdb4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f608c493c0 0x55f608c4cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:54.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.029+0000 7f0e5bdb4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f608c8b5b0 con 0x55f608c493c0 2024-04-20T21:28:54.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.029+0000 7f0e5bdb4740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0e541f86f0 con 0x55f608c42ee0 2024-04-20T21:28:54.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.029+0000 7f0e5bdb4740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55f608b92290 con 0x55f608c48fe0 2024-04-20T21:28:54.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.029+0000 7f0e5bdb4740 10 client.0 _ll_register_callbacks cb 0x7f0e3c006f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:28:54.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.029+0000 7f0e52800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f608c493c0 0x55f608c4cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:54.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.029+0000 7f0e52800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f608c493c0 0x55f608c4cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46020/0 (socket says 172.21.15.86:46020) 2024-04-20T21:28:54.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.029+0000 7f0e52800640 1 -- 172.21.15.86:0/1628511315 learned_addr learned my addr 172.21.15.86:0/1628511315 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:54.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.029+0000 7f0e52800640 1 -- 172.21.15.86:0/1628511315 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f608c48fe0 msgr2=0x55f608c8ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:54.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.029+0000 7f0e52800640 1 --2- 172.21.15.86:0/1628511315 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f608c48fe0 0x55f608c8ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.029+0000 7f0e52800640 1 -- 172.21.15.86:0/1628511315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f608c42ee0 msgr2=0x55f608ce2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:54.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.029+0000 7f0e52800640 1 --2- 172.21.15.86:0/1628511315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f608c42ee0 0x55f608ce2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.029+0000 7f0e52800640 1 -- 172.21.15.86:0/1628511315 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f608859170 con 0x55f608c493c0 2024-04-20T21:28:54.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.030+0000 7f0e52800640 1 --2- 172.21.15.86:0/1628511315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f608c493c0 0x55f608c4cd70 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c0567c0 tx=0x7f0e4c05a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d0a827405ccda09 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:54.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.030+0000 7f0e37400640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:28:54.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.030+0000 7f0e37400640 1 -- 172.21.15.86:0/1628511315 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c089040 con 0x55f608c493c0 2024-04-20T21:28:54.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.030+0000 7f0e37400640 1 -- 172.21.15.86:0/1628511315 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 5795+0+0 (secure 0 0 0) 0x7f0e4c0569d0 con 0x55f608c493c0 2024-04-20T21:28:54.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.030+0000 7f0e5bdb4740 1 -- 172.21.15.86:0/1628511315 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e3c012860 con 0x55f608c493c0 2024-04-20T21:28:54.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.030+0000 7f0e37400640 1 -- 172.21.15.86:0/1628511315 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f0e4c056600 con 0x55f608c493c0 2024-04-20T21:28:54.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.030+0000 7f0e5bdb4740 10 client.9015 fetch_fsmap learned FSMap version 88 2024-04-20T21:28:54.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.030+0000 7f0e5bdb4740 1 -- 172.21.15.86:0/1628511315 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x55f608ce2930 con 0x55f608c493c0 2024-04-20T21:28:54.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.030+0000 7f0e37400640 1 -- 172.21.15.86:0/1628511315 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e3c012860 con 0x55f608c493c0 2024-04-20T21:28:54.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.030+0000 7f0e37400640 1 -- 172.21.15.86:0/1628511315 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 88) ==== 3495+0+0 (secure 0 0 0) 0x7f0e4c062ab0 con 0x55f608c493c0 2024-04-20T21:28:54.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.030+0000 7f0e5bdb4740 10 client.9015 fetch_fsmap finished waiting for FSMap version 88 2024-04-20T21:28:54.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.030+0000 7f0e5bdb4740 10 client.9015 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-20T21:28:54.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.030+0000 7f0e5bdb4740 20 client.9015 populate_metadata read hostname 'smithi086' 2024-04-20T21:28:54.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.030+0000 7f0e5bdb4740 1 --2- 172.21.15.86:0/1628511315 >> [v2:172.21.15.86:6834/4271041811,v1:172.21.15.86:6836/4271041811] conn(0x55f608c1d250 0x55f608c4d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:54.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.030+0000 7f0e5bdb4740 4 client.9015 mds_command: new command op to 8922 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-04-20T21:28:54.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.030+0000 7f0e5bdb4740 1 -- 172.21.15.86:0/1628511315 --> [v2:172.21.15.86:6834/4271041811,v1:172.21.15.86:6836/4271041811] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x55f608b8e5b0 con 0x55f608c1d250 2024-04-20T21:28:54.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.031+0000 7f0e51e00640 1 --2- 172.21.15.86:0/1628511315 >> [v2:172.21.15.86:6834/4271041811,v1:172.21.15.86:6836/4271041811] conn(0x55f608c1d250 0x55f608c4d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:54.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.031+0000 7f0e51e00640 1 --2- 172.21.15.86:0/1628511315 >> [v2:172.21.15.86:6834/4271041811,v1:172.21.15.86:6836/4271041811] conn(0x55f608c1d250 0x55f608c4d2b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d7e6863b5f6bf68 server_cookie=f2dd521374fd7f64 in_seq=0 out_seq=0 2024-04-20T21:28:54.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.031+0000 7f0e37400640 10 client.9015 ms_handle_connect on v2:172.21.15.86:6834/4271041811 2024-04-20T21:28:54.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.032+0000 7f0e37400640 1 -- 172.21.15.86:0/1628511315 <== mds.0 v2:172.21.15.86:6834/4271041811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f608b8e5b0 con 0x55f608c1d250 2024-04-20T21:28:54.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.032+0000 7f0e37400640 10 client.9015 handle_command_reply: tid=0 2024-04-20T21:28:54.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.032+0000 7f0e5bdb4740 1 client.9015 shutdown 2024-04-20T21:28:54.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.032+0000 7f0e5bdb4740 1 -- 172.21.15.86:0/1628511315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f608c493c0 msgr2=0x55f608c4cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:54.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.032+0000 7f0e5bdb4740 1 --2- 172.21.15.86:0/1628511315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f608c493c0 0x55f608c4cd70 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c0567c0 tx=0x7f0e4c05a130 comp rx=0 tx=0).stop 2024-04-20T21:28:54.034 INFO:tasks.ceph.mds.c.smithi086.stderr:2024-04-20T21:28:54.032+0000 7ff2ee600640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-04-20T21:28:54.035 INFO:tasks.ceph.mds.c.smithi086.stderr:2024-04-20T21:28:54.032+0000 7ff2ee600640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-04-20T21:28:54.035 INFO:tasks.ceph.mds.c.smithi086.stderr:2024-04-20T21:28:54.032+0000 7ff2ee600640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-04-20T21:28:54.035 INFO:tasks.ceph.mds.c.smithi086.stderr:2024-04-20T21:28:54.032+0000 7ff2ee600640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-04-20T21:28:54.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.032+0000 7f0e5bdb4740 1 -- 172.21.15.86:0/1628511315 shutdown_connections 2024-04-20T21:28:54.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.032+0000 7f0e5bdb4740 1 --2- 172.21.15.86:0/1628511315 >> [v2:172.21.15.86:6834/4271041811,v1:172.21.15.86:6836/4271041811] conn(0x55f608c1d250 0x55f608c4d2b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.032+0000 7f0e5bdb4740 1 --2- 172.21.15.86:0/1628511315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f608c493c0 0x55f608c4cd70 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.032+0000 7f0e5bdb4740 1 --2- 172.21.15.86:0/1628511315 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55f608c48fe0 0x55f608c8ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.032+0000 7f0e5bdb4740 1 --2- 172.21.15.86:0/1628511315 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55f608c42ee0 0x55f608ce2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.032+0000 7f0e5bdb4740 1 -- 172.21.15.86:0/1628511315 >> 172.21.15.86:0/1628511315 conn(0x55f608c1c4c0 msgr2=0x55f608c80ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:54.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.032+0000 7f0e37400640 0 client.9015 ms_handle_reset on v2:172.21.15.86:6834/4271041811 2024-04-20T21:28:54.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.033+0000 7f0e5bdb4740 1 -- 172.21.15.86:0/1628511315 shutdown_connections 2024-04-20T21:28:54.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.033+0000 7f0e5bdb4740 1 -- 172.21.15.86:0/1628511315 wait complete. 2024-04-20T21:28:54.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.033+0000 7f0e5bdb4740 20 client.9015 trim_cache size 0 max 16384 2024-04-20T21:28:54.036 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:28:54.037 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-20T21:28:54.037 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "82dd1284-a0e9-4a0a-9011-7d32cd34138c", 2024-04-20T21:28:54.037 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-20T21:28:54.037 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:28:54.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.033+0000 7f0e59000640 1 -- 172.21.15.86:0/2693049484 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0e3006c8f0 msgr2=0x7f0e3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:54.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.033+0000 7f0e59000640 1 --2- 172.21.15.86:0/2693049484 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0e3006c8f0 0x7f0e3006edb0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f0e44036000 tx=0x7f0e4403c040 comp rx=0 tx=0).stop 2024-04-20T21:28:54.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.034+0000 7f0e59000640 1 -- 172.21.15.86:0/2693049484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e5415ec20 msgr2=0x7f0e540b1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:54.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.034+0000 7f0e59000640 1 --2- 172.21.15.86:0/2693049484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e5415ec20 0x7f0e540b1db0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c00cd70 tx=0x7f0e4c00a660 comp rx=0 tx=0).stop 2024-04-20T21:28:54.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.034+0000 7f0e59000640 1 -- 172.21.15.86:0/2693049484 shutdown_connections 2024-04-20T21:28:54.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.034+0000 7f0e59000640 1 --2- 172.21.15.86:0/2693049484 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0e3006c8f0 0x7f0e3006edb0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.034+0000 7f0e59000640 1 --2- 172.21.15.86:0/2693049484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e5415ec20 0x7f0e540b1db0 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.034+0000 7f0e59000640 1 --2- 172.21.15.86:0/2693049484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e5415e840 0x7f0e54153940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.034+0000 7f0e59000640 1 --2- 172.21.15.86:0/2693049484 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e540d2d30 0x7f0e540a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.034+0000 7f0e59000640 1 -- 172.21.15.86:0/2693049484 >> 172.21.15.86:0/2693049484 conn(0x7f0e5409da40 msgr2=0x7f0e540d13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:54.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.034+0000 7f0e59000640 1 -- 172.21.15.86:0/2693049484 shutdown_connections 2024-04-20T21:28:54.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.034+0000 7f0e59000640 1 -- 172.21.15.86:0/2693049484 wait complete. 2024-04-20T21:28:54.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:28:54.050 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-20T21:28:54.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.228+0000 7f8241000640 1 Processor -- start 2024-04-20T21:28:54.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.228+0000 7f8241000640 1 -- start start 2024-04-20T21:28:54.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.229+0000 7f8241000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f823c0cfc70 0x7f823c0cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:54.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.229+0000 7f8241000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f823c15bcd0 con 0x7f823c06f5c0 2024-04-20T21:28:54.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.229+0000 7f8241000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f823c15bfb0 con 0x7f823c0cdbe0 2024-04-20T21:28:54.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.229+0000 7f8241000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f823c15c8e0 con 0x7f823c0cfc70 2024-04-20T21:28:54.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.230+0000 7f823b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f823c0cfc70 0x7f823c0cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:54.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.230+0000 7f823be00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f823c06f5c0 0x7f823c071ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:45912/0 (socket says 172.21.15.86:45912) 2024-04-20T21:28:54.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.230+0000 7f823be00640 1 -- 172.21.15.86:0/3377359788 learned_addr learned my addr 172.21.15.86:0/3377359788 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:28:54.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.230+0000 7f823b400640 1 -- 172.21.15.86:0/3377359788 >> v1:172.21.15.144:6789/0 conn(0x7f823c0cdbe0 legacy=0x7f823c06eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:54.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.230+0000 7f823b400640 1 -- 172.21.15.86:0/3377359788 >> v1:172.21.15.86:6789/0 conn(0x7f823c06f5c0 legacy=0x7f823c071ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:54.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.230+0000 7f823a000640 1 -- 172.21.15.86:0/3377359788 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3857514766 0 0) 0x7f823c15bcd0 con 0x7f823c06f5c0 2024-04-20T21:28:54.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.230+0000 7f823b400640 1 -- 172.21.15.86:0/3377359788 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f823c15c290 con 0x7f823c0cfc70 2024-04-20T21:28:54.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.231+0000 7f823b400640 1 --2- 172.21.15.86:0/3377359788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f823c0cfc70 0x7f823c0cd6a0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f823c15ca60 tx=0x7f822c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b21ca803a6451f66 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:54.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.232+0000 7f823a000640 1 -- 172.21.15.86:0/3377359788 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822c035020 con 0x7f823c0cfc70 2024-04-20T21:28:54.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.232+0000 7f823a000640 1 -- 172.21.15.86:0/3377359788 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f822c031830 con 0x7f823c0cfc70 2024-04-20T21:28:54.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.232+0000 7f823a000640 1 -- 172.21.15.86:0/3377359788 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822c031c00 con 0x7f823c0cfc70 2024-04-20T21:28:54.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.232+0000 7f8241000640 1 -- 172.21.15.86:0/3377359788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f823c0cfc70 msgr2=0x7f823c0cd6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:54.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.232+0000 7f8241000640 1 --2- 172.21.15.86:0/3377359788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f823c0cfc70 0x7f823c0cd6a0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f823c15ca60 tx=0x7f822c02b8d0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.232+0000 7f8241000640 1 -- 172.21.15.86:0/3377359788 shutdown_connections 2024-04-20T21:28:54.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.232+0000 7f8241000640 1 --2- 172.21.15.86:0/3377359788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f823c0cfc70 0x7f823c0cd6a0 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.232+0000 7f8241000640 1 -- 172.21.15.86:0/3377359788 >> 172.21.15.86:0/3377359788 conn(0x7f823c067cf0 msgr2=0x7f823c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:54.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.232+0000 7f8241000640 1 -- 172.21.15.86:0/3377359788 shutdown_connections 2024-04-20T21:28:54.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.233+0000 7f8241000640 1 -- 172.21.15.86:0/3377359788 wait complete. 2024-04-20T21:28:54.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.234+0000 7f8241000640 1 Processor -- start 2024-04-20T21:28:54.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.234+0000 7f8241000640 1 -- start start 2024-04-20T21:28:54.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.235+0000 7f8241000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f823c06f5c0 0x7f823c1512a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:54.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.235+0000 7f8241000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f823c0cdbe0 0x7f823c0adfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:54.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.235+0000 7f8241000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f823c0cfc70 0x7f823c0b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:54.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.235+0000 7f8241000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f823c0d0050 con 0x7f823c0cfc70 2024-04-20T21:28:54.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.235+0000 7f8241000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f823c0b3710 con 0x7f823c0cdbe0 2024-04-20T21:28:54.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.235+0000 7f8241000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f823c0d01d0 con 0x7f823c06f5c0 2024-04-20T21:28:54.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.236+0000 7f823be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f823c0cfc70 0x7f823c0b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:54.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.236+0000 7f823be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f823c0cfc70 0x7f823c0b4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46036/0 (socket says 172.21.15.86:46036) 2024-04-20T21:28:54.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.236+0000 7f823be00640 1 -- 172.21.15.86:0/958227143 learned_addr learned my addr 172.21.15.86:0/958227143 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:54.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.236+0000 7f823aa00640 1 --2- 172.21.15.86:0/958227143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f823c0cdbe0 0x7f823c0adfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:54.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.236+0000 7f823be00640 1 -- 172.21.15.86:0/958227143 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f823c06f5c0 msgr2=0x7f823c1512a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:54.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.236+0000 7f823be00640 1 --2- 172.21.15.86:0/958227143 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f823c06f5c0 0x7f823c1512a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.236+0000 7f823be00640 1 -- 172.21.15.86:0/958227143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f823c0cdbe0 msgr2=0x7f823c0adfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:54.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.236+0000 7f823be00640 1 --2- 172.21.15.86:0/958227143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f823c0cdbe0 0x7f823c0adfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.236+0000 7f823be00640 1 -- 172.21.15.86:0/958227143 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f823c0af310 con 0x7f823c0cfc70 2024-04-20T21:28:54.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.236+0000 7f823aa00640 1 --2- 172.21.15.86:0/958227143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f823c0cdbe0 0x7f823c0adfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:28:54.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.237+0000 7f823be00640 1 --2- 172.21.15.86:0/958227143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f823c0cfc70 0x7f823c0b4e30 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f823400c780 tx=0x7f823400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:54.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.237+0000 7f8233400640 1 -- 172.21.15.86:0/958227143 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8234018070 con 0x7f823c0cfc70 2024-04-20T21:28:54.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.237+0000 7f8233400640 1 -- 172.21.15.86:0/958227143 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f823400a640 con 0x7f823c0cfc70 2024-04-20T21:28:54.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.237+0000 7f8233400640 1 -- 172.21.15.86:0/958227143 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823400aab0 con 0x7f823c0cfc70 2024-04-20T21:28:54.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.237+0000 7f8241000640 1 -- 172.21.15.86:0/958227143 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f823c1517e0 con 0x7f823c0cfc70 2024-04-20T21:28:54.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.237+0000 7f8241000640 1 -- 172.21.15.86:0/958227143 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f823c0ad500 con 0x7f823c0cfc70 2024-04-20T21:28:54.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.239+0000 7f8233400640 1 -- 172.21.15.86:0/958227143 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f823400ac50 con 0x7f823c0cfc70 2024-04-20T21:28:54.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.240+0000 7f8233400640 1 --2- 172.21.15.86:0/958227143 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f821806c8f0 0x7f821806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:54.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.240+0000 7f8233400640 1 -- 172.21.15.86:0/958227143 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5795+0+0 (secure 0 0 0) 0x7f823408b220 con 0x7f823c0cfc70 2024-04-20T21:28:54.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.240+0000 7f823b400640 1 --2- 172.21.15.86:0/958227143 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f821806c8f0 0x7f821806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:54.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.241+0000 7f824521e740 1 Processor -- start 2024-04-20T21:28:54.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.241+0000 7f824521e740 1 -- start start 2024-04-20T21:28:54.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.241+0000 7f823b400640 1 --2- 172.21.15.86:0/958227143 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f821806c8f0 0x7f821806edb0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f822c036000 tx=0x7f822c030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:54.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.242+0000 7f824521e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edcbe883d0 0x55edcbe887b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:54.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.242+0000 7f824521e740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55edcbe583f0 0x55edcbe85640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:54.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.242+0000 7f824521e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edcbe5bf50 0x55edcbe85b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:54.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.242+0000 7f824521e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55edcbdc4150 con 0x55edcbe883d0 2024-04-20T21:28:54.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.242+0000 7f824521e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55edcbd6a370 con 0x55edcbe5bf50 2024-04-20T21:28:54.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.242+0000 7f824521e740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55edcbdc0110 con 0x55edcbe583f0 2024-04-20T21:28:54.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.242+0000 7f823b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edcbe883d0 0x55edcbe887b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:54.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.242+0000 7f823b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edcbe883d0 0x55edcbe887b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46042/0 (socket says 172.21.15.86:46042) 2024-04-20T21:28:54.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.242+0000 7f823b400640 1 -- 172.21.15.86:0/4244796379 learned_addr learned my addr 172.21.15.86:0/4244796379 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:54.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.243+0000 7f823aa00640 1 --2- 172.21.15.86:0/4244796379 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55edcbe583f0 0x55edcbe85640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:54.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.243+0000 7f823b400640 1 -- 172.21.15.86:0/4244796379 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55edcbe583f0 msgr2=0x55edcbe85640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:54.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.243+0000 7f823b400640 1 --2- 172.21.15.86:0/4244796379 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55edcbe583f0 0x55edcbe85640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.243+0000 7f823be00640 1 --2- 172.21.15.86:0/4244796379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edcbe5bf50 0x55edcbe85b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:54.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.243+0000 7f823b400640 1 -- 172.21.15.86:0/4244796379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edcbe5bf50 msgr2=0x55edcbe85b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:54.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.243+0000 7f823b400640 1 --2- 172.21.15.86:0/4244796379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edcbe5bf50 0x55edcbe85b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.243+0000 7f823b400640 1 -- 172.21.15.86:0/4244796379 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edcbba07d0 con 0x55edcbe883d0 2024-04-20T21:28:54.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.243+0000 7f823be00640 1 --2- 172.21.15.86:0/4244796379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edcbe5bf50 0x55edcbe85b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:28:54.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.243+0000 7f823b400640 1 --2- 172.21.15.86:0/4244796379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edcbe883d0 0x55edcbe887b0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f822c009560 tx=0x7f822c031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6f670fc9f113b10 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:54.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.243+0000 7f822be00640 1 -- 172.21.15.86:0/4244796379 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822c03a6e0 con 0x55edcbe883d0 2024-04-20T21:28:54.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.243+0000 7f822be00640 1 -- 172.21.15.86:0/4244796379 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f822c03a880 con 0x55edcbe883d0 2024-04-20T21:28:54.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.243+0000 7f822be00640 1 -- 172.21.15.86:0/4244796379 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822c03ac30 con 0x55edcbe883d0 2024-04-20T21:28:54.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.244+0000 7f824521e740 1 -- 172.21.15.86:0/4244796379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edcbe883d0 msgr2=0x55edcbe887b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:54.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.244+0000 7f824521e740 1 --2- 172.21.15.86:0/4244796379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edcbe883d0 0x55edcbe887b0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f822c009560 tx=0x7f822c031000 comp rx=0 tx=0).stop 2024-04-20T21:28:54.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.244+0000 7f824521e740 1 -- 172.21.15.86:0/4244796379 shutdown_connections 2024-04-20T21:28:54.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.244+0000 7f824521e740 1 --2- 172.21.15.86:0/4244796379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edcbe5bf50 0x55edcbe85b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.244+0000 7f824521e740 1 --2- 172.21.15.86:0/4244796379 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55edcbe583f0 0x55edcbe85640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.244+0000 7f824521e740 1 --2- 172.21.15.86:0/4244796379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edcbe883d0 0x55edcbe887b0 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.244+0000 7f824521e740 1 -- 172.21.15.86:0/4244796379 >> 172.21.15.86:0/4244796379 conn(0x55edcbe5a660 msgr2=0x55edcbea31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:54.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.244+0000 7f824521e740 1 -- 172.21.15.86:0/4244796379 shutdown_connections 2024-04-20T21:28:54.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.244+0000 7f824521e740 1 -- 172.21.15.86:0/4244796379 wait complete. 2024-04-20T21:28:54.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.245+0000 7f824521e740 1 -- start start 2024-04-20T21:28:54.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.245+0000 7f824521e740 1 Processor -- start 2024-04-20T21:28:54.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.246+0000 7f824521e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edcbe5bf50 0x55edcbe7c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:54.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.246+0000 7f824521e740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55edcbe883d0 0x55edcbe8d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:54.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.246+0000 7f824521e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edcbe8dbf0 0x55edcbeb7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:54.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.246+0000 7f824521e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55edcbe5d100 con 0x55edcbe8dbf0 2024-04-20T21:28:54.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.246+0000 7f824521e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55edcbd7d740 con 0x55edcbe5bf50 2024-04-20T21:28:54.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.246+0000 7f824521e740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55edcbe5c4f0 con 0x55edcbe883d0 2024-04-20T21:28:54.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.246+0000 7f824521e740 10 client.0 _ll_register_callbacks cb 0x55edcbd78b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:28:54.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.247+0000 7f823be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edcbe8dbf0 0x55edcbeb7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:54.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.247+0000 7f823be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edcbe8dbf0 0x55edcbeb7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46052/0 (socket says 172.21.15.86:46052) 2024-04-20T21:28:54.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.247+0000 7f823be00640 1 -- 172.21.15.86:0/795679050 learned_addr learned my addr 172.21.15.86:0/795679050 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:54.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.247+0000 7f823b400640 1 --2- 172.21.15.86:0/795679050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edcbe5bf50 0x55edcbe7c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:54.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.247+0000 7f823aa00640 1 --2- 172.21.15.86:0/795679050 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55edcbe883d0 0x55edcbe8d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:54.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.247+0000 7f823be00640 1 -- 172.21.15.86:0/795679050 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55edcbe883d0 msgr2=0x55edcbe8d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:54.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.247+0000 7f823be00640 1 --2- 172.21.15.86:0/795679050 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55edcbe883d0 0x55edcbe8d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.247+0000 7f823be00640 1 -- 172.21.15.86:0/795679050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edcbe5bf50 msgr2=0x55edcbe7c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:54.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.247+0000 7f823be00640 1 --2- 172.21.15.86:0/795679050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edcbe5bf50 0x55edcbe7c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.247+0000 7f823be00640 1 -- 172.21.15.86:0/795679050 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edcbad63f0 con 0x55edcbe8dbf0 2024-04-20T21:28:54.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.247+0000 7f823aa00640 1 --2- 172.21.15.86:0/795679050 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55edcbe883d0 0x55edcbe8d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:28:54.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.247+0000 7f823be00640 1 --2- 172.21.15.86:0/795679050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edcbe8dbf0 0x55edcbeb7080 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f8234089680 tx=0x7f8234058840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b756883e46e1cc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:54.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.247+0000 7f8228c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:28:54.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.247+0000 7f8228c00640 1 -- 172.21.15.86:0/795679050 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8234058530 con 0x55edcbe8dbf0 2024-04-20T21:28:54.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.248+0000 7f8228c00640 1 -- 172.21.15.86:0/795679050 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 5795+0+0 (secure 0 0 0) 0x7f8234071180 con 0x55edcbe8dbf0 2024-04-20T21:28:54.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.248+0000 7f824521e740 1 -- 172.21.15.86:0/795679050 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f822c03a880 con 0x55edcbe8dbf0 2024-04-20T21:28:54.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.248+0000 7f8228c00640 1 -- 172.21.15.86:0/795679050 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f823405d290 con 0x55edcbe8dbf0 2024-04-20T21:28:54.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.248+0000 7f824521e740 10 client.9030 fetch_fsmap learned FSMap version 88 2024-04-20T21:28:54.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.248+0000 7f824521e740 1 -- 172.21.15.86:0/795679050 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x55edcba960f0 con 0x55edcbe8dbf0 2024-04-20T21:28:54.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.248+0000 7f8228c00640 1 -- 172.21.15.86:0/795679050 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f822c03a880 con 0x55edcbe8dbf0 2024-04-20T21:28:54.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.248+0000 7f8228c00640 1 -- 172.21.15.86:0/795679050 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 88) ==== 3495+0+0 (secure 0 0 0) 0x7f8234089120 con 0x55edcbe8dbf0 2024-04-20T21:28:54.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.248+0000 7f824521e740 10 client.9030 fetch_fsmap finished waiting for FSMap version 88 2024-04-20T21:28:54.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.248+0000 7f824521e740 10 client.9030 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-20T21:28:54.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.248+0000 7f824521e740 20 client.9030 populate_metadata read hostname 'smithi086' 2024-04-20T21:28:54.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.248+0000 7f824521e740 1 --2- 172.21.15.86:0/795679050 >> [v2:172.21.15.86:6834/4271041811,v1:172.21.15.86:6836/4271041811] conn(0x55edcbe8afd0 0x55edcbe8b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:54.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.248+0000 7f824521e740 4 client.9030 mds_command: new command op to 8922 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:28:54.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.248+0000 7f824521e740 1 -- 172.21.15.86:0/795679050 --> [v2:172.21.15.86:6834/4271041811,v1:172.21.15.86:6836/4271041811] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55edcbdd1b70 con 0x55edcbe8afd0 2024-04-20T21:28:54.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.249+0000 7f823b400640 1 --2- 172.21.15.86:0/795679050 >> [v2:172.21.15.86:6834/4271041811,v1:172.21.15.86:6836/4271041811] conn(0x55edcbe8afd0 0x55edcbe8b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:54.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.249+0000 7f823b400640 1 --2- 172.21.15.86:0/795679050 >> [v2:172.21.15.86:6834/4271041811,v1:172.21.15.86:6836/4271041811] conn(0x55edcbe8afd0 0x55edcbe8b3b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=693ba2066f27dd91 server_cookie=5824e122d9df5e21 in_seq=0 out_seq=0 2024-04-20T21:28:54.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.249+0000 7f8228c00640 10 client.9030 ms_handle_connect on v2:172.21.15.86:6834/4271041811 2024-04-20T21:28:54.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.250+0000 7f8228c00640 1 -- 172.21.15.86:0/795679050 <== mds.0 v2:172.21.15.86:6834/4271041811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55edcbdd1b70 con 0x55edcbe8afd0 2024-04-20T21:28:54.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.250+0000 7f8228c00640 10 client.9030 handle_command_reply: tid=0 2024-04-20T21:28:54.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.250+0000 7f824521e740 1 client.9030 shutdown 2024-04-20T21:28:54.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.250+0000 7f824521e740 1 -- 172.21.15.86:0/795679050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edcbe8dbf0 msgr2=0x55edcbeb7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:54.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.250+0000 7f824521e740 1 --2- 172.21.15.86:0/795679050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edcbe8dbf0 0x55edcbeb7080 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f8234089680 tx=0x7f8234058840 comp rx=0 tx=0).stop 2024-04-20T21:28:54.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.250+0000 7f824521e740 1 -- 172.21.15.86:0/795679050 shutdown_connections 2024-04-20T21:28:54.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.250+0000 7f824521e740 1 --2- 172.21.15.86:0/795679050 >> [v2:172.21.15.86:6834/4271041811,v1:172.21.15.86:6836/4271041811] conn(0x55edcbe8afd0 0x55edcbe8b3b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.250+0000 7f824521e740 1 --2- 172.21.15.86:0/795679050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edcbe8dbf0 0x55edcbeb7080 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.250+0000 7f824521e740 1 --2- 172.21.15.86:0/795679050 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55edcbe883d0 0x55edcbe8d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.250+0000 7f824521e740 1 --2- 172.21.15.86:0/795679050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edcbe5bf50 0x55edcbe7c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.250+0000 7f8228c00640 0 client.9030 ms_handle_reset on v2:172.21.15.86:6834/4271041811 2024-04-20T21:28:54.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.250+0000 7f824521e740 1 -- 172.21.15.86:0/795679050 >> 172.21.15.86:0/795679050 conn(0x55edcbe583f0 msgr2=0x55edcbe5d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:54.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.250+0000 7f824521e740 1 -- 172.21.15.86:0/795679050 shutdown_connections 2024-04-20T21:28:54.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.250+0000 7f824521e740 1 -- 172.21.15.86:0/795679050 wait complete. 2024-04-20T21:28:54.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.250+0000 7f824521e740 20 client.9030 trim_cache size 0 max 16384 2024-04-20T21:28:54.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.269+0000 7f824521e740 1 Processor -- start 2024-04-20T21:28:54.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.269+0000 7f824521e740 1 -- start start 2024-04-20T21:28:54.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.270+0000 7f824521e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edcbe5bf50 0x55edcbe8bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:54.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.270+0000 7f824521e740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55edcbe883d0 0x55edcbe8c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:54.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.270+0000 7f824521e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edcbe8dbf0 0x55edcbe7eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:54.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.270+0000 7f824521e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f823c200c60 con 0x55edcbe5bf50 2024-04-20T21:28:54.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.270+0000 7f824521e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f823c1ffc30 con 0x55edcbe8dbf0 2024-04-20T21:28:54.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.270+0000 7f824521e740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f823c154760 con 0x55edcbe883d0 2024-04-20T21:28:54.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.270+0000 7f823b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edcbe5bf50 0x55edcbe8bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:54.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.270+0000 7f823b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edcbe5bf50 0x55edcbe8bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46066/0 (socket says 172.21.15.86:46066) 2024-04-20T21:28:54.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.271+0000 7f823b400640 1 -- 172.21.15.86:0/918859525 learned_addr learned my addr 172.21.15.86:0/918859525 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:54.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.271+0000 7f823be00640 1 --2- 172.21.15.86:0/918859525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edcbe8dbf0 0x55edcbe7eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:54.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.271+0000 7f823aa00640 1 --2- 172.21.15.86:0/918859525 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55edcbe883d0 0x55edcbe8c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:54.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.271+0000 7f823b400640 1 -- 172.21.15.86:0/918859525 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55edcbe883d0 msgr2=0x55edcbe8c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:54.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.271+0000 7f823b400640 1 --2- 172.21.15.86:0/918859525 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55edcbe883d0 0x55edcbe8c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.271+0000 7f823b400640 1 -- 172.21.15.86:0/918859525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edcbe8dbf0 msgr2=0x55edcbe7eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:54.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.271+0000 7f823b400640 1 --2- 172.21.15.86:0/918859525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edcbe8dbf0 0x55edcbe7eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.271+0000 7f823b400640 1 -- 172.21.15.86:0/918859525 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edcbe8b720 con 0x55edcbe5bf50 2024-04-20T21:28:54.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.271+0000 7f823aa00640 1 --2- 172.21.15.86:0/918859525 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55edcbe883d0 0x55edcbe8c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:28:54.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.271+0000 7f823b400640 1 --2- 172.21.15.86:0/918859525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edcbe5bf50 0x55edcbe8bd80 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f822c009890 tx=0x7f822c03c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd2ef1c8f586969 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:54.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.272+0000 7f822b400640 1 -- 172.21.15.86:0/918859525 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822c054040 con 0x55edcbe5bf50 2024-04-20T21:28:54.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.272+0000 7f822b400640 1 -- 172.21.15.86:0/918859525 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f822c03a840 con 0x55edcbe5bf50 2024-04-20T21:28:54.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.272+0000 7f822b400640 1 -- 172.21.15.86:0/918859525 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822c03abf0 con 0x55edcbe5bf50 2024-04-20T21:28:54.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.272+0000 7f824521e740 1 -- 172.21.15.86:0/918859525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edcbe5bf50 msgr2=0x55edcbe8bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:54.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.272+0000 7f824521e740 1 --2- 172.21.15.86:0/918859525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edcbe5bf50 0x55edcbe8bd80 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f822c009890 tx=0x7f822c03c040 comp rx=0 tx=0).stop 2024-04-20T21:28:54.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.272+0000 7f824521e740 1 -- 172.21.15.86:0/918859525 shutdown_connections 2024-04-20T21:28:54.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.272+0000 7f824521e740 1 --2- 172.21.15.86:0/918859525 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edcbe8dbf0 0x55edcbe7eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.272+0000 7f824521e740 1 --2- 172.21.15.86:0/918859525 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55edcbe883d0 0x55edcbe8c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.272+0000 7f824521e740 1 --2- 172.21.15.86:0/918859525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edcbe5bf50 0x55edcbe8bd80 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.272+0000 7f824521e740 1 -- 172.21.15.86:0/918859525 >> 172.21.15.86:0/918859525 conn(0x55edcbe583f0 msgr2=0x55edcbe7c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:54.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.272+0000 7f824521e740 1 -- 172.21.15.86:0/918859525 shutdown_connections 2024-04-20T21:28:54.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.272+0000 7f824521e740 1 -- 172.21.15.86:0/918859525 wait complete. 2024-04-20T21:28:54.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.273+0000 7f824521e740 1 -- start start 2024-04-20T21:28:54.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.273+0000 7f824521e740 1 Processor -- start 2024-04-20T21:28:54.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.274+0000 7f824521e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edcbe5bf50 0x55edcbe887b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:54.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.274+0000 7f824521e740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55edcbe883d0 0x55edcbe88cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:54.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.274+0000 7f824521e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edcbe8dbf0 0x55edcbe83cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:54.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.274+0000 7f824521e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55edcbec7e50 con 0x55edcbe5bf50 2024-04-20T21:28:54.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.274+0000 7f824521e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f823c0a9c70 con 0x55edcbe8dbf0 2024-04-20T21:28:54.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.274+0000 7f824521e740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55edcbe90d70 con 0x55edcbe883d0 2024-04-20T21:28:54.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.274+0000 7f824521e740 10 client.0 _ll_register_callbacks cb 0x7f8234001800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:28:54.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.275+0000 7f823b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edcbe5bf50 0x55edcbe887b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:54.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.275+0000 7f823b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edcbe5bf50 0x55edcbe887b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46070/0 (socket says 172.21.15.86:46070) 2024-04-20T21:28:54.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.275+0000 7f823b400640 1 -- 172.21.15.86:0/1993701379 learned_addr learned my addr 172.21.15.86:0/1993701379 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:28:54.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.275+0000 7f823aa00640 1 --2- 172.21.15.86:0/1993701379 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55edcbe883d0 0x55edcbe88cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:54.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.275+0000 7f823b400640 1 -- 172.21.15.86:0/1993701379 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55edcbe883d0 msgr2=0x55edcbe88cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:54.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.275+0000 7f823b400640 1 --2- 172.21.15.86:0/1993701379 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55edcbe883d0 0x55edcbe88cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.275+0000 7f823b400640 1 -- 172.21.15.86:0/1993701379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edcbe8dbf0 msgr2=0x55edcbe83cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:28:54.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.275+0000 7f823b400640 1 --2- 172.21.15.86:0/1993701379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edcbe8dbf0 0x55edcbe83cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.275+0000 7f823b400640 1 -- 172.21.15.86:0/1993701379 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edcbebdff0 con 0x55edcbe5bf50 2024-04-20T21:28:54.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.275+0000 7f823b400640 1 --2- 172.21.15.86:0/1993701379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edcbe5bf50 0x55edcbe887b0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f822c000c00 tx=0x7f822c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efcbcbe19a411bca server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:28:54.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.275+0000 7f8228c00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:28:54.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.275+0000 7f8228c00640 1 -- 172.21.15.86:0/1993701379 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822c03a9f0 con 0x55edcbe5bf50 2024-04-20T21:28:54.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.276+0000 7f8228c00640 1 -- 172.21.15.86:0/1993701379 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 5795+0+0 (secure 0 0 0) 0x7f822c03ab90 con 0x55edcbe5bf50 2024-04-20T21:28:54.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.276+0000 7f824521e740 1 -- 172.21.15.86:0/1993701379 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f822c03a840 con 0x55edcbe5bf50 2024-04-20T21:28:54.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.276+0000 7f8228c00640 1 -- 172.21.15.86:0/1993701379 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f822c03ad90 con 0x55edcbe5bf50 2024-04-20T21:28:54.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.276+0000 7f824521e740 10 client.9036 fetch_fsmap learned FSMap version 88 2024-04-20T21:28:54.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.276+0000 7f824521e740 1 -- 172.21.15.86:0/1993701379 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x55edcbf27230 con 0x55edcbe5bf50 2024-04-20T21:28:54.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.276+0000 7f8228c00640 1 -- 172.21.15.86:0/1993701379 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f822c03a840 con 0x55edcbe5bf50 2024-04-20T21:28:54.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.276+0000 7f8228c00640 1 -- 172.21.15.86:0/1993701379 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 88) ==== 3495+0+0 (secure 0 0 0) 0x7f822c05b120 con 0x55edcbe5bf50 2024-04-20T21:28:54.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.276+0000 7f824521e740 10 client.9036 fetch_fsmap finished waiting for FSMap version 88 2024-04-20T21:28:54.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.276+0000 7f824521e740 10 client.9036 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-20T21:28:54.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.276+0000 7f824521e740 20 client.9036 populate_metadata read hostname 'smithi086' 2024-04-20T21:28:54.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.276+0000 7f824521e740 1 --2- 172.21.15.86:0/1993701379 >> [v2:172.21.15.86:6834/4271041811,v1:172.21.15.86:6836/4271041811] conn(0x55edcbe8c400 0x55edcbe84200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:28:54.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.276+0000 7f824521e740 4 client.9036 mds_command: new command op to 8922 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:28:54.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.276+0000 7f824521e740 1 -- 172.21.15.86:0/1993701379 --> [v2:172.21.15.86:6834/4271041811,v1:172.21.15.86:6836/4271041811] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55edcbdcb9e0 con 0x55edcbe8c400 2024-04-20T21:28:54.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.277+0000 7f823aa00640 1 --2- 172.21.15.86:0/1993701379 >> [v2:172.21.15.86:6834/4271041811,v1:172.21.15.86:6836/4271041811] conn(0x55edcbe8c400 0x55edcbe84200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:28:54.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.277+0000 7f823aa00640 1 --2- 172.21.15.86:0/1993701379 >> [v2:172.21.15.86:6834/4271041811,v1:172.21.15.86:6836/4271041811] conn(0x55edcbe8c400 0x55edcbe84200 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79d51739c27d29fb server_cookie=9294d8f58a0eb95d in_seq=0 out_seq=0 2024-04-20T21:28:54.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.277+0000 7f8228c00640 10 client.9036 ms_handle_connect on v2:172.21.15.86:6834/4271041811 2024-04-20T21:28:54.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.277+0000 7f8228c00640 1 -- 172.21.15.86:0/1993701379 <== mds.0 v2:172.21.15.86:6834/4271041811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+260 (crc 0 0 0) 0x55edcbdcb9e0 con 0x55edcbe8c400 2024-04-20T21:28:54.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.277+0000 7f8228c00640 10 client.9036 handle_command_reply: tid=0 2024-04-20T21:28:54.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.277+0000 7f824521e740 1 client.9036 shutdown 2024-04-20T21:28:54.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.277+0000 7f824521e740 1 -- 172.21.15.86:0/1993701379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edcbe5bf50 msgr2=0x55edcbe887b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:54.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.277+0000 7f824521e740 1 --2- 172.21.15.86:0/1993701379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edcbe5bf50 0x55edcbe887b0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f822c000c00 tx=0x7f822c004730 comp rx=0 tx=0).stop 2024-04-20T21:28:54.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.278+0000 7f824521e740 1 -- 172.21.15.86:0/1993701379 shutdown_connections 2024-04-20T21:28:54.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.278+0000 7f824521e740 1 --2- 172.21.15.86:0/1993701379 >> [v2:172.21.15.86:6834/4271041811,v1:172.21.15.86:6836/4271041811] conn(0x55edcbe8c400 0x55edcbe84200 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.278+0000 7f824521e740 1 --2- 172.21.15.86:0/1993701379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55edcbe8dbf0 0x55edcbe83cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.278+0000 7f824521e740 1 --2- 172.21.15.86:0/1993701379 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55edcbe883d0 0x55edcbe88cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.278+0000 7f824521e740 1 --2- 172.21.15.86:0/1993701379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55edcbe5bf50 0x55edcbe887b0 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.278+0000 7f824521e740 1 -- 172.21.15.86:0/1993701379 >> 172.21.15.86:0/1993701379 conn(0x55edcbe583f0 msgr2=0x55edcbeb6070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:54.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.278+0000 7f8228c00640 0 client.9036 ms_handle_reset on v2:172.21.15.86:6834/4271041811 2024-04-20T21:28:54.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.278+0000 7f824521e740 1 -- 172.21.15.86:0/1993701379 shutdown_connections 2024-04-20T21:28:54.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.278+0000 7f824521e740 1 -- 172.21.15.86:0/1993701379 wait complete. 2024-04-20T21:28:54.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.278+0000 7f824521e740 20 client.9036 trim_cache size 0 max 16384 2024-04-20T21:28:54.281 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:28:54.281 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-20T21:28:54.281 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-20T21:28:54.282 INFO:teuthology.orchestra.run.smithi086.stdout: "82dd1284-a0e9-4a0a-9011-7d32cd34138c": { 2024-04-20T21:28:54.282 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-20T21:28:54.282 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "82dd1284-a0e9-4a0a-9011-7d32cd34138c", 2024-04-20T21:28:54.282 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,repair" 2024-04-20T21:28:54.282 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:28:54.282 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:28:54.282 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:28:54.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.278+0000 7f8241000640 1 -- 172.21.15.86:0/958227143 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f821806c8f0 msgr2=0x7f821806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:54.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.278+0000 7f8241000640 1 --2- 172.21.15.86:0/958227143 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f821806c8f0 0x7f821806edb0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f822c036000 tx=0x7f822c030000 comp rx=0 tx=0).stop 2024-04-20T21:28:54.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.278+0000 7f8241000640 1 -- 172.21.15.86:0/958227143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f823c0cfc70 msgr2=0x7f823c0b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:28:54.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.278+0000 7f8241000640 1 --2- 172.21.15.86:0/958227143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f823c0cfc70 0x7f823c0b4e30 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f823400c780 tx=0x7f823400cc40 comp rx=0 tx=0).stop 2024-04-20T21:28:54.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.279+0000 7f8241000640 1 -- 172.21.15.86:0/958227143 shutdown_connections 2024-04-20T21:28:54.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.279+0000 7f8241000640 1 --2- 172.21.15.86:0/958227143 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f821806c8f0 0x7f821806edb0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.279+0000 7f8241000640 1 --2- 172.21.15.86:0/958227143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f823c0cfc70 0x7f823c0b4e30 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.279+0000 7f8241000640 1 --2- 172.21.15.86:0/958227143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f823c0cdbe0 0x7f823c0adfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.279+0000 7f8241000640 1 --2- 172.21.15.86:0/958227143 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f823c06f5c0 0x7f823c1512a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:28:54.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.279+0000 7f8241000640 1 -- 172.21.15.86:0/958227143 >> 172.21.15.86:0/958227143 conn(0x7f823c067cf0 msgr2=0x7f823c06e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:28:54.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.279+0000 7f8241000640 1 -- 172.21.15.86:0/958227143 shutdown_connections 2024-04-20T21:28:54.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:54.279+0000 7f8241000640 1 -- 172.21.15.86:0/958227143 wait complete. 2024-04-20T21:28:54.295 INFO:tasks.cephfs.filesystem:scrub status for tag:82dd1284-a0e9-4a0a-9011-7d32cd34138c - {'path': '/', 'tag': '82dd1284-a0e9-4a0a-9011-7d32cd34138c', 'options': 'recursive,repair'} 2024-04-20T21:28:55.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:55.907+0000 7f5d16000640 1 -- 172.21.15.86:0/1897508667 <== mon.0 v2:172.21.15.86:3300/0 7 ==== log(8 entries from seq 1 at 2024-04-20T21:28:54.033515+0000) ==== 1407+0+0 (secure 0 0 0) 0x7f5d10058600 con 0x7f5d04004800 2024-04-20T21:28:55.908 INFO:teuthology.orchestra.run.smithi086.stdout: cluster: 2024-04-20T21:28:55.908 INFO:teuthology.orchestra.run.smithi086.stdout: id: 9c2dec1d-d7ae-4a83-a812-79922808e278 2024-04-20T21:28:55.908 INFO:teuthology.orchestra.run.smithi086.stdout: health: HEALTH_OK 2024-04-20T21:28:55.908 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:28:55.908 INFO:teuthology.orchestra.run.smithi086.stdout: services: 2024-04-20T21:28:55.908 INFO:teuthology.orchestra.run.smithi086.stdout: mon: 3 daemons, quorum a,b,c (age 11m) 2024-04-20T21:28:55.908 INFO:teuthology.orchestra.run.smithi086.stdout: mgr: y(active, since 11m), standbys: x 2024-04-20T21:28:55.908 INFO:teuthology.orchestra.run.smithi086.stdout: mds: 1/1 daemons up, 3 standby 2024-04-20T21:28:55.908 INFO:teuthology.orchestra.run.smithi086.stdout: osd: 8 osds: 8 up (since 11m), 8 in (since 11m) 2024-04-20T21:28:55.908 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:28:55.909 INFO:teuthology.orchestra.run.smithi086.stdout: data: 2024-04-20T21:28:55.909 INFO:teuthology.orchestra.run.smithi086.stdout: volumes: 1/1 healthy 2024-04-20T21:28:55.909 INFO:teuthology.orchestra.run.smithi086.stdout: pools: 3 pools, 97 pgs 2024-04-20T21:28:55.909 INFO:teuthology.orchestra.run.smithi086.stdout: objects: 30 objects, 19 MiB 2024-04-20T21:28:55.909 INFO:teuthology.orchestra.run.smithi086.stdout: usage: 290 MiB used, 720 GiB / 720 GiB avail 2024-04-20T21:28:55.909 INFO:teuthology.orchestra.run.smithi086.stdout: pgs: 97 active+clean 2024-04-20T21:28:55.909 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:28:55.909 INFO:teuthology.orchestra.run.smithi086.stdout: io: 2024-04-20T21:28:55.909 INFO:teuthology.orchestra.run.smithi086.stdout: client: 1.8 KiB/s rd, 102 B/s wr, 1 op/s rd, 0 op/s wr 2024-04-20T21:28:55.909 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:28:55.909 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:28:55.909 INFO:teuthology.orchestra.run.smithi086.stdout:2024-04-20T21:28:54.033515+0000 mds.c [INF] scrub queued for path: / 2024-04-20T21:28:55.909 INFO:teuthology.orchestra.run.smithi086.stdout:2024-04-20T21:28:54.033518+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-04-20T21:28:55.909 INFO:teuthology.orchestra.run.smithi086.stdout:2024-04-20T21:28:54.033522+0000 mds.c [INF] scrub summary: active paths [/] 2024-04-20T21:28:55.909 INFO:teuthology.orchestra.run.smithi086.stdout:2024-04-20T21:28:54.034323+0000 mds.c [ERR] scrub: inode wrongly marked free: 0x10000000002 2024-04-20T21:28:55.909 INFO:teuthology.orchestra.run.smithi086.stdout:2024-04-20T21:28:54.034327+0000 mds.c [ERR] inode table repaired for inode: 0x10000000002 2024-04-20T21:28:55.910 INFO:teuthology.orchestra.run.smithi086.stdout:2024-04-20T21:28:54.034391+0000 mds.c [ERR] scrub: inode wrongly marked free: 0x10000000001 2024-04-20T21:28:55.910 INFO:teuthology.orchestra.run.smithi086.stdout:2024-04-20T21:28:54.034396+0000 mds.c [ERR] inode table repaired for inode: 0x10000000001 2024-04-20T21:28:55.910 INFO:teuthology.orchestra.run.smithi086.stdout:2024-04-20T21:28:54.035567+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-04-20T21:28:59.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:28:59.920+0000 7f5d16000640 1 -- 172.21.15.86:0/1897508667 <== mon.0 v2:172.21.15.86:3300/0 8 ==== log(1 entries from seq 9 at 2024-04-20T21:28:57.930132+0000) ==== 189+0+0 (secure 0 0 0) 0x7f5d10058860 con 0x7f5d04004800 2024-04-20T21:28:59.920 INFO:teuthology.orchestra.run.smithi086.stdout:2024-04-20T21:28:57.930132+0000 mds.c [INF] scrub summary: idle 2024-04-20T21:29:18.263 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:29:18.271 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:29:24.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:29:24.295 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-20T21:29:24.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.508+0000 7fe4dfa00640 1 Processor -- start 2024-04-20T21:29:24.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.508+0000 7fe4dfa00640 1 -- start start 2024-04-20T21:29:24.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.510+0000 7fe4dfa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4d8146210 0x7fe4d806e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:24.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.510+0000 7fe4dfa00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4d815cd40 con 0x7fe4d806bbf0 2024-04-20T21:29:24.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.510+0000 7fe4dfa00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4d815d000 con 0x7fe4d806bfd0 2024-04-20T21:29:24.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.510+0000 7fe4dfa00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe4d815d2e0 con 0x7fe4d8146210 2024-04-20T21:29:24.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.510+0000 7fe4dd000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe4d806bbf0 0x7fe4d8145b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41770/0 (socket says 172.21.15.86:41770) 2024-04-20T21:29:24.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.510+0000 7fe4dd000640 1 -- 172.21.15.86:0/364585682 learned_addr learned my addr 172.21.15.86:0/364585682 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:29:24.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.510+0000 7fe4d7400640 1 -- 172.21.15.86:0/364585682 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1145051805 0 0) 0x7fe4d815cd40 con 0x7fe4d806bbf0 2024-04-20T21:29:24.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.510+0000 7fe4d7e00640 1 --2- 172.21.15.86:0/364585682 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4d8146210 0x7fe4d806e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:24.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.511+0000 7fe4d7400640 1 -- 172.21.15.86:0/364585682 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe4c0003520 con 0x7fe4d806bbf0 2024-04-20T21:29:24.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.511+0000 7fe4d7400640 1 -- 172.21.15.86:0/364585682 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1094300201 0 0) 0x7fe4c0003520 con 0x7fe4d806bbf0 2024-04-20T21:29:24.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.511+0000 7fe4d7400640 1 -- 172.21.15.86:0/364585682 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4d8146210 msgr2=0x7fe4d806e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:24.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.511+0000 7fe4d7400640 1 --2- 172.21.15.86:0/364585682 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4d8146210 0x7fe4d806e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:24.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.511+0000 7fe4d7400640 1 -- 172.21.15.86:0/364585682 >> v1:172.21.15.144:6789/0 conn(0x7fe4d806bfd0 legacy=0x7fe4d80cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:24.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.511+0000 7fe4d7400640 1 -- 172.21.15.86:0/364585682 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d815d5c0 con 0x7fe4d806bbf0 2024-04-20T21:29:24.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.511+0000 7fe4d7400640 1 -- 172.21.15.86:0/364585682 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fe4cc001670 con 0x7fe4d806bbf0 2024-04-20T21:29:24.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.511+0000 7fe4d7400640 1 -- 172.21.15.86:0/364585682 >> v1:172.21.15.86:6789/0 conn(0x7fe4d806bbf0 legacy=0x7fe4d8145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:24.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.511+0000 7fe4d7400640 1 --2- 172.21.15.86:0/364585682 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4c00039e0 0x7fe4c0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:24.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.511+0000 7fe4d7400640 1 --2- 172.21.15.86:0/364585682 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4d8146210 0x7fe4c0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:24.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.511+0000 7fe4d7400640 1 --2- 172.21.15.86:0/364585682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4c0004d40 0x7fe4c00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:24.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.511+0000 7fe4d7400640 1 -- 172.21.15.86:0/364585682 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d815d2e0 con 0x7fe4c0004d40 2024-04-20T21:29:24.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.511+0000 7fe4d7400640 1 -- 172.21.15.86:0/364585682 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c000ba70 con 0x7fe4c00039e0 2024-04-20T21:29:24.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.511+0000 7fe4d7400640 1 -- 172.21.15.86:0/364585682 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe4c000bc70 con 0x7fe4d8146210 2024-04-20T21:29:24.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.512+0000 7fe4dda00640 1 --2- 172.21.15.86:0/364585682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4c0004d40 0x7fe4c00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:24.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.512+0000 7fe4dd000640 1 --2- 172.21.15.86:0/364585682 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4c00039e0 0x7fe4c0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:24.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.512+0000 7fe4d7e00640 1 --2- 172.21.15.86:0/364585682 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4d8146210 0x7fe4c0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:24.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.512+0000 7fe4dda00640 1 -- 172.21.15.86:0/364585682 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4d8146210 msgr2=0x7fe4c0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:24.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.512+0000 7fe4dda00640 1 --2- 172.21.15.86:0/364585682 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4d8146210 0x7fe4c0004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:24.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.512+0000 7fe4dda00640 1 -- 172.21.15.86:0/364585682 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4c00039e0 msgr2=0x7fe4c0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:24.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.512+0000 7fe4dda00640 1 --2- 172.21.15.86:0/364585682 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4c00039e0 0x7fe4c0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:24.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.512+0000 7fe4dda00640 1 -- 172.21.15.86:0/364585682 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4c0003520 con 0x7fe4c0004d40 2024-04-20T21:29:24.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.513+0000 7fe4dda00640 1 --2- 172.21.15.86:0/364585682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4c0004d40 0x7fe4c00051c0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fe4d806c480 tx=0x7fe4c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fa8c75a44cdb442 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:24.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.513+0000 7fe4d7400640 1 -- 172.21.15.86:0/364585682 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0001a70 con 0x7fe4c0004d40 2024-04-20T21:29:24.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.513+0000 7fe4d7400640 1 -- 172.21.15.86:0/364585682 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4d0003b70 con 0x7fe4c0004d40 2024-04-20T21:29:24.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.513+0000 7fe4d7400640 1 -- 172.21.15.86:0/364585682 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d00050d0 con 0x7fe4c0004d40 2024-04-20T21:29:24.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.514+0000 7fe4dfa00640 1 -- 172.21.15.86:0/364585682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4c0004d40 msgr2=0x7fe4c00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:24.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.514+0000 7fe4dfa00640 1 --2- 172.21.15.86:0/364585682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4c0004d40 0x7fe4c00051c0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fe4d806c480 tx=0x7fe4c0005830 comp rx=0 tx=0).stop 2024-04-20T21:29:24.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.514+0000 7fe4dda00640 1 -- 172.21.15.86:0/364585682 reap_dead start 2024-04-20T21:29:24.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.514+0000 7fe4dfa00640 1 -- 172.21.15.86:0/364585682 shutdown_connections 2024-04-20T21:29:24.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.514+0000 7fe4dfa00640 1 -- 172.21.15.86:0/364585682 >> 172.21.15.86:0/364585682 conn(0x7fe4d80b2110 msgr2=0x7fe4d80b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:24.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.514+0000 7fe4dfa00640 1 -- 172.21.15.86:0/364585682 shutdown_connections 2024-04-20T21:29:24.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.514+0000 7fe4dfa00640 1 -- 172.21.15.86:0/364585682 wait complete. 2024-04-20T21:29:24.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.516+0000 7fe4dfa00640 1 Processor -- start 2024-04-20T21:29:24.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.516+0000 7fe4dfa00640 1 -- start start 2024-04-20T21:29:24.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.517+0000 7fe4dfa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4c0004d40 0x7fe4d80c9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:24.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.517+0000 7fe4dfa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4d815c090 0x7fe4d80b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:24.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.517+0000 7fe4dfa00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4d80caa30 0x7fe4d813cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:24.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.517+0000 7fe4dfa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d80cddd0 con 0x7fe4c0004d40 2024-04-20T21:29:24.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.517+0000 7fe4dfa00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d80cdc50 con 0x7fe4d80caa30 2024-04-20T21:29:24.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.517+0000 7fe4dfa00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe4d80cdf50 con 0x7fe4d815c090 2024-04-20T21:29:24.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.517+0000 7fe4dd000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4c0004d40 0x7fe4d80c9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:24.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.517+0000 7fe4dd000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4c0004d40 0x7fe4d80c9c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46176/0 (socket says 172.21.15.86:46176) 2024-04-20T21:29:24.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.517+0000 7fe4dd000640 1 -- 172.21.15.86:0/1727149150 learned_addr learned my addr 172.21.15.86:0/1727149150 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:24.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.517+0000 7fe4d7e00640 1 --2- 172.21.15.86:0/1727149150 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4d815c090 0x7fe4d80b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:24.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.517+0000 7fe4dda00640 1 --2- 172.21.15.86:0/1727149150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4d80caa30 0x7fe4d813cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:24.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.517+0000 7fe4dd000640 1 -- 172.21.15.86:0/1727149150 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4d815c090 msgr2=0x7fe4d80b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:24.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.517+0000 7fe4dd000640 1 --2- 172.21.15.86:0/1727149150 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4d815c090 0x7fe4d80b4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:24.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.517+0000 7fe4dd000640 1 -- 172.21.15.86:0/1727149150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4d80caa30 msgr2=0x7fe4d813cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:24.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.517+0000 7fe4dd000640 1 --2- 172.21.15.86:0/1727149150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4d80caa30 0x7fe4d813cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:24.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.517+0000 7fe4dd000640 1 -- 172.21.15.86:0/1727149150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d80b3dc0 con 0x7fe4c0004d40 2024-04-20T21:29:24.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.518+0000 7fe4dd000640 1 --2- 172.21.15.86:0/1727149150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4c0004d40 0x7fe4d80c9c20 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc00e7a0 tx=0x7fe4cc00ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:24.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.518+0000 7fe4d4c00640 1 -- 172.21.15.86:0/1727149150 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc00c8d0 con 0x7fe4c0004d40 2024-04-20T21:29:24.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.518+0000 7fe4d4c00640 1 -- 172.21.15.86:0/1727149150 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4cc018070 con 0x7fe4c0004d40 2024-04-20T21:29:24.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.518+0000 7fe4d4c00640 1 -- 172.21.15.86:0/1727149150 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc00ccd0 con 0x7fe4c0004d40 2024-04-20T21:29:24.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.518+0000 7fe4dfa00640 1 -- 172.21.15.86:0/1727149150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4d80b40a0 con 0x7fe4c0004d40 2024-04-20T21:29:24.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.518+0000 7fe4dfa00640 1 -- 172.21.15.86:0/1727149150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4d80b4660 con 0x7fe4c0004d40 2024-04-20T21:29:24.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.520+0000 7fe4d4c00640 1 -- 172.21.15.86:0/1727149150 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe4cc01d460 con 0x7fe4c0004d40 2024-04-20T21:29:24.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.521+0000 7fe4d4c00640 1 --2- 172.21.15.86:0/1727149150 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe4b806c920 0x7fe4b806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:24.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.521+0000 7fe4d4c00640 1 -- 172.21.15.86:0/1727149150 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5795+0+0 (secure 0 0 0) 0x7fe4cc08bb50 con 0x7fe4c0004d40 2024-04-20T21:29:24.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.522+0000 7fe4d7e00640 1 --2- 172.21.15.86:0/1727149150 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe4b806c920 0x7fe4b806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:24.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.522+0000 7fe4e280e740 1 Processor -- start 2024-04-20T21:29:24.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.522+0000 7fe4d7e00640 1 --2- 172.21.15.86:0/1727149150 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe4b806c920 0x7fe4b806ede0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fe4d80b39c0 tx=0x7fe4c4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:24.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.522+0000 7fe4e280e740 1 -- start start 2024-04-20T21:29:24.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.523+0000 7fe4e280e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b11afb1f0 0x557b11afb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:24.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.523+0000 7fe4e280e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b11acb210 0x557b11af8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:24.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.523+0000 7fe4e280e740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557b11aced70 0x557b11af89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:24.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.523+0000 7fe4e280e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557b11a37150 con 0x557b11afb1f0 2024-04-20T21:29:24.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.523+0000 7fe4e280e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557b119dd370 con 0x557b11acb210 2024-04-20T21:29:24.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.523+0000 7fe4e280e740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x557b11a33110 con 0x557b11aced70 2024-04-20T21:29:24.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.523+0000 7fe4dd000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b11afb1f0 0x557b11afb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:24.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.524+0000 7fe4dd000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b11afb1f0 0x557b11afb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46190/0 (socket says 172.21.15.86:46190) 2024-04-20T21:29:24.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.524+0000 7fe4dd000640 1 -- 172.21.15.86:0/2795755962 learned_addr learned my addr 172.21.15.86:0/2795755962 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:24.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.524+0000 7fe4dda00640 1 --2- 172.21.15.86:0/2795755962 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557b11aced70 0x557b11af89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:24.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.524+0000 7fe4d7e00640 1 --2- 172.21.15.86:0/2795755962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b11acb210 0x557b11af8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:24.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.524+0000 7fe4dd000640 1 -- 172.21.15.86:0/2795755962 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557b11aced70 msgr2=0x557b11af89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:24.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.524+0000 7fe4dd000640 1 --2- 172.21.15.86:0/2795755962 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557b11aced70 0x557b11af89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:24.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.524+0000 7fe4dd000640 1 -- 172.21.15.86:0/2795755962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b11acb210 msgr2=0x557b11af8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:24.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.524+0000 7fe4dd000640 1 --2- 172.21.15.86:0/2795755962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b11acb210 0x557b11af8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:24.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.524+0000 7fe4dd000640 1 -- 172.21.15.86:0/2795755962 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b118137d0 con 0x557b11afb1f0 2024-04-20T21:29:24.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.524+0000 7fe4d7e00640 1 --2- 172.21.15.86:0/2795755962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b11acb210 0x557b11af8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:29:24.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.524+0000 7fe4dd000640 1 --2- 172.21.15.86:0/2795755962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b11afb1f0 0x557b11afb5d0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc061e30 tx=0x7fe4cc0590e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b9a67e5fdcf25a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:24.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.525+0000 7fe4c9600640 1 -- 172.21.15.86:0/2795755962 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc088070 con 0x557b11afb1f0 2024-04-20T21:29:24.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.525+0000 7fe4c9600640 1 -- 172.21.15.86:0/2795755962 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4cc059780 con 0x557b11afb1f0 2024-04-20T21:29:24.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.525+0000 7fe4c9600640 1 -- 172.21.15.86:0/2795755962 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc05db30 con 0x557b11afb1f0 2024-04-20T21:29:24.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.525+0000 7fe4e280e740 1 -- 172.21.15.86:0/2795755962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b11afb1f0 msgr2=0x557b11afb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:24.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.525+0000 7fe4e280e740 1 --2- 172.21.15.86:0/2795755962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b11afb1f0 0x557b11afb5d0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc061e30 tx=0x7fe4cc0590e0 comp rx=0 tx=0).stop 2024-04-20T21:29:24.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.525+0000 7fe4e280e740 1 -- 172.21.15.86:0/2795755962 shutdown_connections 2024-04-20T21:29:24.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.525+0000 7fe4e280e740 1 --2- 172.21.15.86:0/2795755962 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557b11aced70 0x557b11af89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:24.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.525+0000 7fe4e280e740 1 --2- 172.21.15.86:0/2795755962 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b11acb210 0x557b11af8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:24.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.525+0000 7fe4e280e740 1 --2- 172.21.15.86:0/2795755962 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b11afb1f0 0x557b11afb5d0 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:24.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.525+0000 7fe4e280e740 1 -- 172.21.15.86:0/2795755962 >> 172.21.15.86:0/2795755962 conn(0x557b11acd480 msgr2=0x557b11b16000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:24.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.525+0000 7fe4e280e740 1 -- 172.21.15.86:0/2795755962 shutdown_connections 2024-04-20T21:29:24.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.525+0000 7fe4e280e740 1 -- 172.21.15.86:0/2795755962 wait complete. 2024-04-20T21:29:24.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.526+0000 7fe4e280e740 1 -- start start 2024-04-20T21:29:24.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.526+0000 7fe4e280e740 1 Processor -- start 2024-04-20T21:29:24.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.527+0000 7fe4e280e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b11aced70 0x557b11aff170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:24.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.527+0000 7fe4e280e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b11afb1f0 0x557b11b265e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:24.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.527+0000 7fe4e280e740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557b11b26b20 0x557b11af69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:24.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.527+0000 7fe4e280e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557b11acff20 con 0x557b11aced70 2024-04-20T21:29:24.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.527+0000 7fe4e280e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557b119f0740 con 0x557b11afb1f0 2024-04-20T21:29:24.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.527+0000 7fe4e280e740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x557b11acf310 con 0x557b11b26b20 2024-04-20T21:29:24.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.528+0000 7fe4e280e740 10 client.0 _ll_register_callbacks cb 0x557b119ebb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:29:24.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.528+0000 7fe4dd000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b11aced70 0x557b11aff170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:24.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.528+0000 7fe4dd000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b11aced70 0x557b11aff170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46202/0 (socket says 172.21.15.86:46202) 2024-04-20T21:29:24.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.528+0000 7fe4dd000640 1 -- 172.21.15.86:0/1862341979 learned_addr learned my addr 172.21.15.86:0/1862341979 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:24.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.528+0000 7fe4dd000640 1 -- 172.21.15.86:0/1862341979 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557b11b26b20 msgr2=0x557b11af69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:24.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.528+0000 7fe4dd000640 1 --2- 172.21.15.86:0/1862341979 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557b11b26b20 0x557b11af69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:24.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.528+0000 7fe4dd000640 1 -- 172.21.15.86:0/1862341979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b11afb1f0 msgr2=0x557b11b265e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:24.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.528+0000 7fe4dd000640 1 --2- 172.21.15.86:0/1862341979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b11afb1f0 0x557b11b265e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:24.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.528+0000 7fe4dd000640 1 -- 172.21.15.86:0/1862341979 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b117493f0 con 0x557b11aced70 2024-04-20T21:29:24.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.528+0000 7fe4dd000640 1 --2- 172.21.15.86:0/1862341979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b11aced70 0x557b11aff170 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc089f80 tx=0x7fe4cc015070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8a921554cf4dbc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:24.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.528+0000 7fe4be000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:29:24.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.528+0000 7fe4be000640 1 -- 172.21.15.86:0/1862341979 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc088070 con 0x557b11aced70 2024-04-20T21:29:24.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.529+0000 7fe4be000640 1 -- 172.21.15.86:0/1862341979 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 5795+0+0 (secure 0 0 0) 0x7fe4cc055df0 con 0x557b11aced70 2024-04-20T21:29:24.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.529+0000 7fe4e280e740 1 -- 172.21.15.86:0/1862341979 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4cc059780 con 0x557b11aced70 2024-04-20T21:29:24.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.529+0000 7fe4be000640 1 -- 172.21.15.86:0/1862341979 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fe4cc05db30 con 0x557b11aced70 2024-04-20T21:29:24.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.529+0000 7fe4e280e740 10 client.9048 fetch_fsmap learned FSMap version 88 2024-04-20T21:29:24.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.529+0000 7fe4e280e740 1 -- 172.21.15.86:0/1862341979 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x557b117090f0 con 0x557b11aced70 2024-04-20T21:29:24.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.529+0000 7fe4be000640 1 -- 172.21.15.86:0/1862341979 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4cc059780 con 0x557b11aced70 2024-04-20T21:29:24.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.529+0000 7fe4be000640 1 -- 172.21.15.86:0/1862341979 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 88) ==== 3495+0+0 (secure 0 0 0) 0x7fe4cc061ed0 con 0x557b11aced70 2024-04-20T21:29:24.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.529+0000 7fe4e280e740 10 client.9048 fetch_fsmap finished waiting for FSMap version 88 2024-04-20T21:29:24.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.529+0000 7fe4e280e740 10 client.9048 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-20T21:29:24.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.529+0000 7fe4e280e740 20 client.9048 populate_metadata read hostname 'smithi086' 2024-04-20T21:29:24.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.529+0000 7fe4e280e740 1 --2- 172.21.15.86:0/1862341979 >> [v2:172.21.15.86:6834/4271041811,v1:172.21.15.86:6836/4271041811] conn(0x557b11af25e0 0x557b11af2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:24.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.529+0000 7fe4e280e740 4 client.9048 mds_command: new command op to 8922 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:29:24.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.529+0000 7fe4e280e740 1 -- 172.21.15.86:0/1862341979 --> [v2:172.21.15.86:6834/4271041811,v1:172.21.15.86:6836/4271041811] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b11a44b70 con 0x557b11af25e0 2024-04-20T21:29:24.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.530+0000 7fe4d7e00640 1 --2- 172.21.15.86:0/1862341979 >> [v2:172.21.15.86:6834/4271041811,v1:172.21.15.86:6836/4271041811] conn(0x557b11af25e0 0x557b11af2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:24.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.530+0000 7fe4d7e00640 1 --2- 172.21.15.86:0/1862341979 >> [v2:172.21.15.86:6834/4271041811,v1:172.21.15.86:6836/4271041811] conn(0x557b11af25e0 0x557b11af2a70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=278ff09b17bb89af server_cookie=90712fd32c332bb2 in_seq=0 out_seq=0 2024-04-20T21:29:24.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.530+0000 7fe4be000640 10 client.9048 ms_handle_connect on v2:172.21.15.86:6834/4271041811 2024-04-20T21:29:24.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.531+0000 7fe4be000640 1 -- 172.21.15.86:0/1862341979 <== mds.0 v2:172.21.15.86:6834/4271041811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557b11a44b70 con 0x557b11af25e0 2024-04-20T21:29:24.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.531+0000 7fe4be000640 10 client.9048 handle_command_reply: tid=0 2024-04-20T21:29:24.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.531+0000 7fe4e280e740 1 client.9048 shutdown 2024-04-20T21:29:24.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.531+0000 7fe4e280e740 1 -- 172.21.15.86:0/1862341979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b11aced70 msgr2=0x557b11aff170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:24.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.531+0000 7fe4e280e740 1 --2- 172.21.15.86:0/1862341979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b11aced70 0x557b11aff170 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc089f80 tx=0x7fe4cc015070 comp rx=0 tx=0).stop 2024-04-20T21:29:24.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.531+0000 7fe4e280e740 1 -- 172.21.15.86:0/1862341979 shutdown_connections 2024-04-20T21:29:24.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.531+0000 7fe4e280e740 1 --2- 172.21.15.86:0/1862341979 >> [v2:172.21.15.86:6834/4271041811,v1:172.21.15.86:6836/4271041811] conn(0x557b11af25e0 0x557b11af2a70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:24.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.531+0000 7fe4e280e740 1 --2- 172.21.15.86:0/1862341979 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557b11b26b20 0x557b11af69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:24.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.531+0000 7fe4e280e740 1 --2- 172.21.15.86:0/1862341979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b11afb1f0 0x557b11b265e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:24.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.531+0000 7fe4e280e740 1 --2- 172.21.15.86:0/1862341979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b11aced70 0x557b11aff170 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:24.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.531+0000 7fe4be000640 0 client.9048 ms_handle_reset on v2:172.21.15.86:6834/4271041811 2024-04-20T21:29:24.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.531+0000 7fe4e280e740 1 -- 172.21.15.86:0/1862341979 >> 172.21.15.86:0/1862341979 conn(0x557b11acb210 msgr2=0x557b11b175a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:24.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.531+0000 7fe4e280e740 1 -- 172.21.15.86:0/1862341979 shutdown_connections 2024-04-20T21:29:24.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.531+0000 7fe4e280e740 1 -- 172.21.15.86:0/1862341979 wait complete. 2024-04-20T21:29:24.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.532+0000 7fe4e280e740 20 client.9048 trim_cache size 0 max 16384 2024-04-20T21:29:24.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.550+0000 7fe4e280e740 1 Processor -- start 2024-04-20T21:29:24.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.550+0000 7fe4e280e740 1 -- start start 2024-04-20T21:29:24.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.551+0000 7fe4e280e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b11aced70 0x557b11b39d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:24.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.551+0000 7fe4e280e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b11afb1f0 0x557b11af80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:24.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.551+0000 7fe4e280e740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557b11b26b20 0x557b11afca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:24.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.551+0000 7fe4e280e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d8201620 con 0x557b11afb1f0 2024-04-20T21:29:24.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.551+0000 7fe4e280e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d8200650 con 0x557b11aced70 2024-04-20T21:29:24.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.551+0000 7fe4e280e740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe4d8143440 con 0x557b11b26b20 2024-04-20T21:29:24.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.552+0000 7fe4d7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b11afb1f0 0x557b11af80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:24.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.552+0000 7fe4d7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b11afb1f0 0x557b11af80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46206/0 (socket says 172.21.15.86:46206) 2024-04-20T21:29:24.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.552+0000 7fe4d7e00640 1 -- 172.21.15.86:0/3320007305 learned_addr learned my addr 172.21.15.86:0/3320007305 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:24.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.552+0000 7fe4dd000640 1 --2- 172.21.15.86:0/3320007305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b11aced70 0x557b11b39d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:24.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.552+0000 7fe4dda00640 1 --2- 172.21.15.86:0/3320007305 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557b11b26b20 0x557b11afca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:24.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.552+0000 7fe4d7e00640 1 -- 172.21.15.86:0/3320007305 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557b11b26b20 msgr2=0x557b11afca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:24.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.552+0000 7fe4d7e00640 1 --2- 172.21.15.86:0/3320007305 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557b11b26b20 0x557b11afca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:24.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.552+0000 7fe4d7e00640 1 -- 172.21.15.86:0/3320007305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b11aced70 msgr2=0x557b11b39d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:24.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.552+0000 7fe4d7e00640 1 --2- 172.21.15.86:0/3320007305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b11aced70 0x557b11b39d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:24.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.552+0000 7fe4d7e00640 1 -- 172.21.15.86:0/3320007305 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b11ac7190 con 0x557b11afb1f0 2024-04-20T21:29:24.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.552+0000 7fe4d7e00640 1 --2- 172.21.15.86:0/3320007305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b11afb1f0 0x557b11af80a0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8144590 tx=0x7fe4c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77aa6cdcccfa51e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:24.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.553+0000 7fe4c8c00640 1 -- 172.21.15.86:0/3320007305 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c402e070 con 0x557b11afb1f0 2024-04-20T21:29:24.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.553+0000 7fe4c8c00640 1 -- 172.21.15.86:0/3320007305 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4c401a040 con 0x557b11afb1f0 2024-04-20T21:29:24.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.553+0000 7fe4c8c00640 1 -- 172.21.15.86:0/3320007305 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c4015440 con 0x557b11afb1f0 2024-04-20T21:29:24.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.553+0000 7fe4e280e740 1 -- 172.21.15.86:0/3320007305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b11afb1f0 msgr2=0x557b11af80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:24.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.553+0000 7fe4e280e740 1 --2- 172.21.15.86:0/3320007305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b11afb1f0 0x557b11af80a0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8144590 tx=0x7fe4c40046c0 comp rx=0 tx=0).stop 2024-04-20T21:29:24.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.553+0000 7fe4e280e740 1 -- 172.21.15.86:0/3320007305 shutdown_connections 2024-04-20T21:29:24.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.553+0000 7fe4e280e740 1 --2- 172.21.15.86:0/3320007305 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557b11b26b20 0x557b11afca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:24.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.553+0000 7fe4e280e740 1 --2- 172.21.15.86:0/3320007305 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b11afb1f0 0x557b11af80a0 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:24.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.553+0000 7fe4e280e740 1 --2- 172.21.15.86:0/3320007305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b11aced70 0x557b11b39d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:24.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.553+0000 7fe4e280e740 1 -- 172.21.15.86:0/3320007305 >> 172.21.15.86:0/3320007305 conn(0x557b11acb210 msgr2=0x557b11b3a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:24.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.553+0000 7fe4e280e740 1 -- 172.21.15.86:0/3320007305 shutdown_connections 2024-04-20T21:29:24.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.553+0000 7fe4e280e740 1 -- 172.21.15.86:0/3320007305 wait complete. 2024-04-20T21:29:24.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.554+0000 7fe4e280e740 1 -- start start 2024-04-20T21:29:24.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.554+0000 7fe4e280e740 1 Processor -- start 2024-04-20T21:29:24.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.555+0000 7fe4e280e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b11aced70 0x557b11b393f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:24.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.555+0000 7fe4e280e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b11b26b20 0x557b11b2e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:24.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.555+0000 7fe4e280e740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557b11ad08f0 0x557b11b2e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:24.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.555+0000 7fe4e280e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557b11accb70 con 0x557b11aced70 2024-04-20T21:29:24.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.555+0000 7fe4e280e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d8200030 con 0x557b11b26b20 2024-04-20T21:29:24.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.555+0000 7fe4e280e740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x557b11b19dd0 con 0x557b11ad08f0 2024-04-20T21:29:24.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.555+0000 7fe4e280e740 10 client.0 _ll_register_callbacks cb 0x7fe4cc086070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:29:24.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.556+0000 7fe4dd000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b11aced70 0x557b11b393f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:24.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.556+0000 7fe4dd000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b11aced70 0x557b11b393f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46222/0 (socket says 172.21.15.86:46222) 2024-04-20T21:29:24.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.556+0000 7fe4dd000640 1 -- 172.21.15.86:0/156187364 learned_addr learned my addr 172.21.15.86:0/156187364 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:24.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.556+0000 7fe4d7e00640 1 --2- 172.21.15.86:0/156187364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b11b26b20 0x557b11b2e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:24.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.556+0000 7fe4dda00640 1 --2- 172.21.15.86:0/156187364 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557b11ad08f0 0x557b11b2e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:24.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.556+0000 7fe4dd000640 1 -- 172.21.15.86:0/156187364 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557b11ad08f0 msgr2=0x557b11b2e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:24.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.556+0000 7fe4dd000640 1 --2- 172.21.15.86:0/156187364 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557b11ad08f0 0x557b11b2e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:24.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.556+0000 7fe4dd000640 1 -- 172.21.15.86:0/156187364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b11b26b20 msgr2=0x557b11b2e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:24.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.556+0000 7fe4dd000640 1 --2- 172.21.15.86:0/156187364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b11b26b20 0x557b11b2e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:24.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.556+0000 7fe4dd000640 1 -- 172.21.15.86:0/156187364 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b11b39930 con 0x557b11aced70 2024-04-20T21:29:24.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.556+0000 7fe4dd000640 1 --2- 172.21.15.86:0/156187364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b11aced70 0x557b11b393f0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc059480 tx=0x7fe4cc00cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f38bdf01799006e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:24.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.556+0000 7fe4be000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:29:24.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.557+0000 7fe4be000640 1 -- 172.21.15.86:0/156187364 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc088040 con 0x557b11aced70 2024-04-20T21:29:24.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.557+0000 7fe4be000640 1 -- 172.21.15.86:0/156187364 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 5795+0+0 (secure 0 0 0) 0x7fe4cc055df0 con 0x557b11aced70 2024-04-20T21:29:24.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.557+0000 7fe4e280e740 1 -- 172.21.15.86:0/156187364 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4c401a040 con 0x557b11aced70 2024-04-20T21:29:24.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.557+0000 7fe4be000640 1 -- 172.21.15.86:0/156187364 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fe4cc055a20 con 0x557b11aced70 2024-04-20T21:29:24.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.557+0000 7fe4e280e740 10 client.9054 fetch_fsmap learned FSMap version 88 2024-04-20T21:29:24.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.557+0000 7fe4e280e740 1 -- 172.21.15.86:0/156187364 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x557b11b39930 con 0x557b11aced70 2024-04-20T21:29:24.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.557+0000 7fe4be000640 1 -- 172.21.15.86:0/156187364 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4c401a040 con 0x557b11aced70 2024-04-20T21:29:24.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.557+0000 7fe4be000640 1 -- 172.21.15.86:0/156187364 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 88) ==== 3495+0+0 (secure 0 0 0) 0x7fe4cc058640 con 0x557b11aced70 2024-04-20T21:29:24.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.557+0000 7fe4e280e740 10 client.9054 fetch_fsmap finished waiting for FSMap version 88 2024-04-20T21:29:24.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.557+0000 7fe4e280e740 10 client.9054 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-20T21:29:24.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.557+0000 7fe4e280e740 20 client.9054 populate_metadata read hostname 'smithi086' 2024-04-20T21:29:24.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.557+0000 7fe4e280e740 1 --2- 172.21.15.86:0/156187364 >> [v2:172.21.15.86:6834/4271041811,v1:172.21.15.86:6836/4271041811] conn(0x557b11b2ec00 0x557b11b936b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:24.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.557+0000 7fe4e280e740 4 client.9054 mds_command: new command op to 8922 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-20T21:29:24.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.557+0000 7fe4e280e740 1 -- 172.21.15.86:0/156187364 --> [v2:172.21.15.86:6834/4271041811,v1:172.21.15.86:6836/4271041811] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557b11acba10 con 0x557b11b2ec00 2024-04-20T21:29:24.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.558+0000 7fe4d7e00640 1 --2- 172.21.15.86:0/156187364 >> [v2:172.21.15.86:6834/4271041811,v1:172.21.15.86:6836/4271041811] conn(0x557b11b2ec00 0x557b11b936b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:24.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.558+0000 7fe4d7e00640 1 --2- 172.21.15.86:0/156187364 >> [v2:172.21.15.86:6834/4271041811,v1:172.21.15.86:6836/4271041811] conn(0x557b11b2ec00 0x557b11b936b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ecc7c582abeb355 server_cookie=4b8f53912f32af04 in_seq=0 out_seq=0 2024-04-20T21:29:24.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.558+0000 7fe4be000640 10 client.9054 ms_handle_connect on v2:172.21.15.86:6834/4271041811 2024-04-20T21:29:24.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.558+0000 7fe4be000640 1 -- 172.21.15.86:0/156187364 <== mds.0 v2:172.21.15.86:6834/4271041811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557b11acba10 con 0x557b11b2ec00 2024-04-20T21:29:24.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.558+0000 7fe4be000640 10 client.9054 handle_command_reply: tid=0 2024-04-20T21:29:24.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.558+0000 7fe4e280e740 1 client.9054 shutdown 2024-04-20T21:29:24.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.558+0000 7fe4e280e740 1 -- 172.21.15.86:0/156187364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b11aced70 msgr2=0x557b11b393f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:24.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.558+0000 7fe4e280e740 1 --2- 172.21.15.86:0/156187364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b11aced70 0x557b11b393f0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc059480 tx=0x7fe4cc00cef0 comp rx=0 tx=0).stop 2024-04-20T21:29:24.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.558+0000 7fe4e280e740 1 -- 172.21.15.86:0/156187364 shutdown_connections 2024-04-20T21:29:24.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.558+0000 7fe4e280e740 1 --2- 172.21.15.86:0/156187364 >> [v2:172.21.15.86:6834/4271041811,v1:172.21.15.86:6836/4271041811] conn(0x557b11b2ec00 0x557b11b936b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:24.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.559+0000 7fe4e280e740 1 --2- 172.21.15.86:0/156187364 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557b11ad08f0 0x557b11b2e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:24.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.559+0000 7fe4e280e740 1 --2- 172.21.15.86:0/156187364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557b11b26b20 0x557b11b2e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:24.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.559+0000 7fe4be000640 0 client.9054 ms_handle_reset on v2:172.21.15.86:6834/4271041811 2024-04-20T21:29:24.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.559+0000 7fe4e280e740 1 --2- 172.21.15.86:0/156187364 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557b11aced70 0x557b11b393f0 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:24.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.559+0000 7fe4e280e740 1 -- 172.21.15.86:0/156187364 >> 172.21.15.86:0/156187364 conn(0x557b11acb210 msgr2=0x557b11af2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:24.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.559+0000 7fe4e280e740 1 -- 172.21.15.86:0/156187364 shutdown_connections 2024-04-20T21:29:24.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.559+0000 7fe4e280e740 1 -- 172.21.15.86:0/156187364 wait complete. 2024-04-20T21:29:24.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.559+0000 7fe4e280e740 20 client.9054 trim_cache size 0 max 16384 2024-04-20T21:29:24.565 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:29:24.565 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-20T21:29:24.565 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-20T21:29:24.565 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:29:24.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.559+0000 7fe4dfa00640 1 -- 172.21.15.86:0/1727149150 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe4b806c920 msgr2=0x7fe4b806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:24.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.559+0000 7fe4dfa00640 1 --2- 172.21.15.86:0/1727149150 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe4b806c920 0x7fe4b806ede0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fe4d80b39c0 tx=0x7fe4c4008040 comp rx=0 tx=0).stop 2024-04-20T21:29:24.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.559+0000 7fe4dfa00640 1 -- 172.21.15.86:0/1727149150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4c0004d40 msgr2=0x7fe4d80c9c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:24.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.559+0000 7fe4dfa00640 1 --2- 172.21.15.86:0/1727149150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4c0004d40 0x7fe4d80c9c20 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc00e7a0 tx=0x7fe4cc00ec60 comp rx=0 tx=0).stop 2024-04-20T21:29:24.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.560+0000 7fe4dfa00640 1 -- 172.21.15.86:0/1727149150 shutdown_connections 2024-04-20T21:29:24.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.560+0000 7fe4dfa00640 1 --2- 172.21.15.86:0/1727149150 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe4b806c920 0x7fe4b806ede0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:24.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.560+0000 7fe4dfa00640 1 --2- 172.21.15.86:0/1727149150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4d80caa30 0x7fe4d813cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:24.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.560+0000 7fe4dfa00640 1 --2- 172.21.15.86:0/1727149150 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe4d815c090 0x7fe4d80b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:24.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.560+0000 7fe4dfa00640 1 --2- 172.21.15.86:0/1727149150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4c0004d40 0x7fe4d80c9c20 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:24.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.560+0000 7fe4dfa00640 1 -- 172.21.15.86:0/1727149150 >> 172.21.15.86:0/1727149150 conn(0x7fe4d80b2110 msgr2=0x7fe4d806e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:24.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.560+0000 7fe4dfa00640 1 -- 172.21.15.86:0/1727149150 shutdown_connections 2024-04-20T21:29:24.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.560+0000 7fe4dfa00640 1 -- 172.21.15.86:0/1727149150 wait complete. 2024-04-20T21:29:24.575 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-20T21:29:24.676 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-20T21:29:24.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.879+0000 7f025ac00640 1 Processor -- start 2024-04-20T21:29:24.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.879+0000 7f025ac00640 1 -- start start 2024-04-20T21:29:24.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.881+0000 7f025ac00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02540709d0 0x7f0254169810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:24.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.881+0000 7f025ac00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f025406ecf0 con 0x7f02540d0dd0 2024-04-20T21:29:24.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.881+0000 7f025ac00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f025406efd0 con 0x7f025406c090 2024-04-20T21:29:24.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.881+0000 7f025ac00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f02541622b0 con 0x7f02540709d0 2024-04-20T21:29:24.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.881+0000 7f0259800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f02540d0dd0 0x7f025406b980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41780/0 (socket says 172.21.15.86:41780) 2024-04-20T21:29:24.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.881+0000 7f0259800640 1 -- 172.21.15.86:0/27802287 learned_addr learned my addr 172.21.15.86:0/27802287 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:29:24.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.881+0000 7f025a200640 1 --2- 172.21.15.86:0/27802287 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02540709d0 0x7f0254169810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:24.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.882+0000 7f0253e00640 1 -- 172.21.15.86:0/27802287 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 537603952 0 0) 0x7f025406ecf0 con 0x7f02540d0dd0 2024-04-20T21:29:24.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.882+0000 7f0253e00640 1 -- 172.21.15.86:0/27802287 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f023c003520 con 0x7f02540d0dd0 2024-04-20T21:29:24.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.882+0000 7f0253e00640 1 -- 172.21.15.86:0/27802287 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1224893916 0 0) 0x7f023c003520 con 0x7f02540d0dd0 2024-04-20T21:29:24.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.882+0000 7f0253e00640 1 -- 172.21.15.86:0/27802287 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02540709d0 msgr2=0x7f0254169810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:24.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.882+0000 7f0253e00640 1 --2- 172.21.15.86:0/27802287 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02540709d0 0x7f0254169810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:24.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.882+0000 7f0253e00640 1 -- 172.21.15.86:0/27802287 >> v1:172.21.15.144:6789/0 conn(0x7f025406c090 legacy=0x7f02540702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:24.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.882+0000 7f0253e00640 1 -- 172.21.15.86:0/27802287 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f025406f2b0 con 0x7f02540d0dd0 2024-04-20T21:29:24.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.882+0000 7f0253e00640 1 -- 172.21.15.86:0/27802287 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f0248001690 con 0x7f02540d0dd0 2024-04-20T21:29:24.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.882+0000 7f0253e00640 1 -- 172.21.15.86:0/27802287 >> v1:172.21.15.86:6789/0 conn(0x7f02540d0dd0 legacy=0x7f025406b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:24.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.882+0000 7f0253e00640 1 --2- 172.21.15.86:0/27802287 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02540709d0 0x7f023c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:24.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.882+0000 7f0253e00640 1 --2- 172.21.15.86:0/27802287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f023c003fb0 0x7f023c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:24.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.882+0000 7f0253e00640 1 --2- 172.21.15.86:0/27802287 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f023c004d40 0x7f023c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:24.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.882+0000 7f0253e00640 1 -- 172.21.15.86:0/27802287 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f02541622b0 con 0x7f023c003fb0 2024-04-20T21:29:24.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.882+0000 7f0253e00640 1 -- 172.21.15.86:0/27802287 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f023c00bbb0 con 0x7f023c004d40 2024-04-20T21:29:24.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.882+0000 7f0253e00640 1 -- 172.21.15.86:0/27802287 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f023c00bdb0 con 0x7f02540709d0 2024-04-20T21:29:24.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.882+0000 7f0258e00640 1 --2- 172.21.15.86:0/27802287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f023c003fb0 0x7f023c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:24.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.883+0000 7f025a200640 1 --2- 172.21.15.86:0/27802287 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f023c004d40 0x7f023c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:24.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.883+0000 7f0258e00640 1 -- 172.21.15.86:0/27802287 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02540709d0 msgr2=0x7f023c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:24.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.883+0000 7f0259800640 1 --2- 172.21.15.86:0/27802287 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02540709d0 0x7f023c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:24.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.883+0000 7f0258e00640 1 --2- 172.21.15.86:0/27802287 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02540709d0 0x7f023c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:24.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.883+0000 7f0258e00640 1 -- 172.21.15.86:0/27802287 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f023c004d40 msgr2=0x7f023c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:24.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.883+0000 7f0258e00640 1 --2- 172.21.15.86:0/27802287 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f023c004d40 0x7f023c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:24.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.883+0000 7f0258e00640 1 -- 172.21.15.86:0/27802287 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f023c003520 con 0x7f023c003fb0 2024-04-20T21:29:24.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.883+0000 7f0259800640 1 --2- 172.21.15.86:0/27802287 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f02540709d0 0x7f023c0039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:29:24.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.884+0000 7f0258e00640 1 --2- 172.21.15.86:0/27802287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f023c003fb0 0x7f023c004800 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f0240001750 tx=0x7f023c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8616bb076448fa1b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:24.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.884+0000 7f0253e00640 1 -- 172.21.15.86:0/27802287 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0240001370 con 0x7f023c003fb0 2024-04-20T21:29:24.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.884+0000 7f0253e00640 1 -- 172.21.15.86:0/27802287 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02400052d0 con 0x7f023c003fb0 2024-04-20T21:29:24.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.884+0000 7f0253e00640 1 -- 172.21.15.86:0/27802287 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0240003830 con 0x7f023c003fb0 2024-04-20T21:29:24.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.885+0000 7f025ac00640 1 -- 172.21.15.86:0/27802287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f023c003fb0 msgr2=0x7f023c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:24.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.885+0000 7f025ac00640 1 --2- 172.21.15.86:0/27802287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f023c003fb0 0x7f023c004800 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f0240001750 tx=0x7f023c005830 comp rx=0 tx=0).stop 2024-04-20T21:29:24.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.885+0000 7f025a200640 1 -- 172.21.15.86:0/27802287 reap_dead start 2024-04-20T21:29:24.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.885+0000 7f025ac00640 1 -- 172.21.15.86:0/27802287 shutdown_connections 2024-04-20T21:29:24.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.885+0000 7f025ac00640 1 -- 172.21.15.86:0/27802287 >> 172.21.15.86:0/27802287 conn(0x7f02540d1230 msgr2=0x7f02540d1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:24.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.885+0000 7f025ac00640 1 -- 172.21.15.86:0/27802287 shutdown_connections 2024-04-20T21:29:24.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.885+0000 7f025ac00640 1 -- 172.21.15.86:0/27802287 wait complete. 2024-04-20T21:29:24.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.887+0000 7f025ac00640 1 Processor -- start 2024-04-20T21:29:24.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.887+0000 7f025ac00640 1 -- start start 2024-04-20T21:29:24.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.888+0000 7f025ac00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f023c003fb0 0x7f025415bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:24.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.888+0000 7f025ac00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02540c8400 0x7f02540b6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:24.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.888+0000 7f025ac00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02540af3c0 0x7f02540af870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:24.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.888+0000 7f025ac00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f02540ce4b0 con 0x7f02540af3c0 2024-04-20T21:29:24.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.888+0000 7f025ac00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f025406f870 con 0x7f02540c8400 2024-04-20T21:29:24.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.888+0000 7f025ac00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f02540ce630 con 0x7f023c003fb0 2024-04-20T21:29:24.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.888+0000 7f025a200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02540af3c0 0x7f02540af870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:24.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.888+0000 7f025a200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02540af3c0 0x7f02540af870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46242/0 (socket says 172.21.15.86:46242) 2024-04-20T21:29:24.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.888+0000 7f025a200640 1 -- 172.21.15.86:0/1106347376 learned_addr learned my addr 172.21.15.86:0/1106347376 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:24.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.889+0000 7f0258e00640 1 --2- 172.21.15.86:0/1106347376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02540c8400 0x7f02540b6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:24.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.889+0000 7f0259800640 1 --2- 172.21.15.86:0/1106347376 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f023c003fb0 0x7f025415bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:24.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.889+0000 7f025a200640 1 -- 172.21.15.86:0/1106347376 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f023c003fb0 msgr2=0x7f025415bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:24.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.889+0000 7f025a200640 1 --2- 172.21.15.86:0/1106347376 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f023c003fb0 0x7f025415bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:24.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.889+0000 7f025a200640 1 -- 172.21.15.86:0/1106347376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02540c8400 msgr2=0x7f02540b6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:24.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.889+0000 7f025a200640 1 --2- 172.21.15.86:0/1106347376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02540c8400 0x7f02540b6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:24.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.889+0000 7f025a200640 1 -- 172.21.15.86:0/1106347376 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f025415c040 con 0x7f02540af3c0 2024-04-20T21:29:24.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.889+0000 7f0259800640 1 --2- 172.21.15.86:0/1106347376 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f023c003fb0 0x7f025415bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:29:24.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.889+0000 7f025a200640 1 --2- 172.21.15.86:0/1106347376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02540af3c0 0x7f02540af870 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f0254071e80 tx=0x7f024c0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:24.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.889+0000 7f0251600640 1 -- 172.21.15.86:0/1106347376 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024c011e10 con 0x7f02540af3c0 2024-04-20T21:29:24.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.890+0000 7f0251600640 1 -- 172.21.15.86:0/1106347376 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f024c01b070 con 0x7f02540af3c0 2024-04-20T21:29:24.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.890+0000 7f025ac00640 1 -- 172.21.15.86:0/1106347376 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02541f4380 con 0x7f02540af3c0 2024-04-20T21:29:24.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.890+0000 7f0251600640 1 -- 172.21.15.86:0/1106347376 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024c003990 con 0x7f02540af3c0 2024-04-20T21:29:24.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.890+0000 7f025ac00640 1 -- 172.21.15.86:0/1106347376 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02541f4940 con 0x7f02540af3c0 2024-04-20T21:29:24.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.891+0000 7f025ac00640 1 -- 172.21.15.86:0/1106347376 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f025404fcf0 con 0x7f02540af3c0 2024-04-20T21:29:24.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.892+0000 7f0251600640 1 -- 172.21.15.86:0/1106347376 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f024c005d10 con 0x7f02540af3c0 2024-04-20T21:29:24.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.893+0000 7f0251600640 1 --2- 172.21.15.86:0/1106347376 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f023406c920 0x7f023406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:24.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.893+0000 7f0251600640 1 -- 172.21.15.86:0/1106347376 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5795+0+0 (secure 0 0 0) 0x7f024c08bf60 con 0x7f02540af3c0 2024-04-20T21:29:24.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.893+0000 7f0259800640 1 --2- 172.21.15.86:0/1106347376 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f023406c920 0x7f023406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:24.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.894+0000 7f0259800640 1 --2- 172.21.15.86:0/1106347376 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f023406c920 0x7f023406ede0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f0248001010 tx=0x7f0248009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:24.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:24.898+0000 7f0251600640 1 -- 172.21.15.86:0/1106347376 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f024c01d310 con 0x7f02540af3c0 2024-04-20T21:29:25.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.139+0000 7f025ac00640 1 -- 172.21.15.86:0/1106347376 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f02540b71b0 con 0x7f02540af3c0 2024-04-20T21:29:25.139 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-04-20T21:29:25.140+0000 7f5bb9c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T21:29:25.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.144+0000 7f0251600640 1 -- 172.21.15.86:0/1106347376 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v89) ==== 161+0+0 (secure 0 0 0) 0x7f024c05a330 con 0x7f02540af3c0 2024-04-20T21:29:25.143 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-20T21:29:25.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.147+0000 7f025ac00640 1 -- 172.21.15.86:0/1106347376 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f023406c920 msgr2=0x7f023406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:25.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.147+0000 7f025ac00640 1 --2- 172.21.15.86:0/1106347376 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f023406c920 0x7f023406ede0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f0248001010 tx=0x7f0248009040 comp rx=0 tx=0).stop 2024-04-20T21:29:25.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.147+0000 7f025ac00640 1 -- 172.21.15.86:0/1106347376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02540af3c0 msgr2=0x7f02540af870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:25.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.147+0000 7f025ac00640 1 --2- 172.21.15.86:0/1106347376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02540af3c0 0x7f02540af870 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f0254071e80 tx=0x7f024c0115a0 comp rx=0 tx=0).stop 2024-04-20T21:29:25.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.147+0000 7f025ac00640 1 -- 172.21.15.86:0/1106347376 shutdown_connections 2024-04-20T21:29:25.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.147+0000 7f025ac00640 1 --2- 172.21.15.86:0/1106347376 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f023406c920 0x7f023406ede0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:25.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.147+0000 7f025ac00640 1 --2- 172.21.15.86:0/1106347376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02540af3c0 0x7f02540af870 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:25.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.147+0000 7f025ac00640 1 --2- 172.21.15.86:0/1106347376 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02540c8400 0x7f02540b6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:25.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.147+0000 7f025ac00640 1 --2- 172.21.15.86:0/1106347376 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f023c003fb0 0x7f025415bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:25.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.147+0000 7f025ac00640 1 -- 172.21.15.86:0/1106347376 >> 172.21.15.86:0/1106347376 conn(0x7f02540d1230 msgr2=0x7f02540d1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:25.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.147+0000 7f025ac00640 1 -- 172.21.15.86:0/1106347376 shutdown_connections 2024-04-20T21:29:25.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.147+0000 7f025ac00640 1 -- 172.21.15.86:0/1106347376 wait complete. 2024-04-20T21:29:25.165 INFO:tasks.ceph.mds.c.smithi086.stderr: -1300> 2024-04-20T21:28:53.863+0000 7ff2f3600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T21:29:25.165 INFO:tasks.ceph.mds.c.smithi086.stderr: -1145> 2024-04-20T21:28:54.032+0000 7ff2ee600640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-04-20T21:29:25.166 INFO:tasks.ceph.mds.c.smithi086.stderr: -1142> 2024-04-20T21:28:54.032+0000 7ff2ee600640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-04-20T21:29:25.166 INFO:tasks.ceph.mds.c.smithi086.stderr: -1126> 2024-04-20T21:28:54.032+0000 7ff2ee600640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-04-20T21:29:25.166 INFO:tasks.ceph.mds.c.smithi086.stderr: -1123> 2024-04-20T21:28:54.032+0000 7ff2ee600640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-04-20T21:29:25.168 DEBUG:teuthology.orchestra.run.smithi086:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 0 show inode 2024-04-20T21:29:25.231 INFO:tasks.ceph.mds.c.smithi086.stdout:starting mds.c at 2024-04-20T21:29:25.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.257+0000 7ff0dd836280 1 -- start start 2024-04-20T21:29:25.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.257+0000 7ff0dd836280 1 Processor -- start 2024-04-20T21:29:25.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.258+0000 7ff0dd836280 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ea99e6000 0x560ea8dc0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:25.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.258+0000 7ff0dd836280 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560ea8c4fc00 0x560ea8dc1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:25.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.258+0000 7ff0dd836280 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ea99e6800 0x560ea8dc0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:25.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.258+0000 7ff0dd836280 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560ea9a3a000 con 0x560ea99e6800 2024-04-20T21:29:25.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.258+0000 7ff0dd836280 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560ea9a3a180 con 0x560ea99e6000 2024-04-20T21:29:25.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.258+0000 7ff0dd836280 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x560ea9a3a300 con 0x560ea8c4fc00 2024-04-20T21:29:25.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.259+0000 7ff0dba00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ea99e6800 0x560ea8dc0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:25.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.259+0000 7ff0dba00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ea99e6800 0x560ea8dc0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46274/0 (socket says 172.21.15.86:46274) 2024-04-20T21:29:25.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.259+0000 7ff0dba00640 1 -- 172.21.15.86:0/3483305584 learned_addr learned my addr 172.21.15.86:0/3483305584 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:25.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.259+0000 7ff0da600640 1 --2- 172.21.15.86:0/3483305584 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560ea8c4fc00 0x560ea8dc1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:25.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.259+0000 7ff0db000640 1 --2- 172.21.15.86:0/3483305584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ea99e6000 0x560ea8dc0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:25.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.259+0000 7ff0dba00640 1 -- 172.21.15.86:0/3483305584 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560ea8c4fc00 msgr2=0x560ea8dc1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:25.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.259+0000 7ff0dba00640 1 --2- 172.21.15.86:0/3483305584 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560ea8c4fc00 0x560ea8dc1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:25.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.259+0000 7ff0dba00640 1 -- 172.21.15.86:0/3483305584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ea99e6000 msgr2=0x560ea8dc0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:25.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.259+0000 7ff0dba00640 1 --2- 172.21.15.86:0/3483305584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ea99e6000 0x560ea8dc0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:25.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.259+0000 7ff0dba00640 1 -- 172.21.15.86:0/3483305584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ea8c82780 con 0x560ea99e6800 2024-04-20T21:29:25.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.259+0000 7ff0dba00640 1 --2- 172.21.15.86:0/3483305584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ea99e6800 0x560ea8dc0b00 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x560ea9a7d320 tx=0x560ea99ea120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50e24c23ae618653 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:25.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.260+0000 7ff0d7e00640 1 -- 172.21.15.86:0/3483305584 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x560ea8c749c0 con 0x560ea99e6800 2024-04-20T21:29:25.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.260+0000 7ff0d7e00640 1 -- 172.21.15.86:0/3483305584 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x560ea8c7a700 con 0x560ea99e6800 2024-04-20T21:29:25.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.260+0000 7ff0d7e00640 1 -- 172.21.15.86:0/3483305584 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x560ea8c74b60 con 0x560ea99e6800 2024-04-20T21:29:25.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.260+0000 7ff0dd836280 1 -- 172.21.15.86:0/3483305584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ea8c82000 con 0x560ea99e6800 2024-04-20T21:29:25.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.260+0000 7ff0dd836280 1 -- 172.21.15.86:0/3483305584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=0}) -- 0x560ea8c83a40 con 0x560ea99e6800 2024-04-20T21:29:25.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.260+0000 7ff0dd836280 4 waiting for MDS map... 2024-04-20T21:29:25.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.260+0000 7ff0d7e00640 1 -- 172.21.15.86:0/3483305584 <== mon.0 v2:172.21.15.86:3300/0 4 ==== osd_map(102..102 src has 1..102) ==== 5881+0+0 (secure 0 0 0) 0x560ea9b20600 con 0x560ea99e6800 2024-04-20T21:29:25.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.260+0000 7ff0d7e00640 1 -- 172.21.15.86:0/3483305584 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 89) ==== 2886+0+0 (secure 0 0 0) 0x560ea8ca9180 con 0x560ea99e6800 2024-04-20T21:29:25.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.260+0000 7ff0dd836280 4 Got MDS map 89 2024-04-20T21:29:25.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.260+0000 7ff0dd836280 10 main: main 2024-04-20T21:29:25.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.260+0000 7ff0dd836280 4 main: connecting to RADOS... 2024-04-20T21:29:25.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.261+0000 7ff0dd836280 1 Processor -- start 2024-04-20T21:29:25.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.261+0000 7ff0dd836280 1 -- start start 2024-04-20T21:29:25.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.262+0000 7ff0dd836280 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ea9b22c00 0x560ea8dc0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:25.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.262+0000 7ff0dd836280 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ea9b23000 0x560ea8dc1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:25.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.262+0000 7ff0dd836280 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560ea9b23400 0x560ea8dc1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:25.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.262+0000 7ff0dd836280 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560ea9a3a480 con 0x560ea9b22c00 2024-04-20T21:29:25.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.262+0000 7ff0dd836280 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560ea9a3a600 con 0x560ea9b23000 2024-04-20T21:29:25.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.262+0000 7ff0dd836280 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x560ea9a3a780 con 0x560ea9b23400 2024-04-20T21:29:25.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.262+0000 7ff0da600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ea9b22c00 0x560ea8dc0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:25.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.262+0000 7ff0da600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ea9b22c00 0x560ea8dc0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46290/0 (socket says 172.21.15.86:46290) 2024-04-20T21:29:25.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.262+0000 7ff0da600640 1 -- 172.21.15.86:0/1692055750 learned_addr learned my addr 172.21.15.86:0/1692055750 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:25.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.262+0000 7ff0da600640 1 -- 172.21.15.86:0/1692055750 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560ea9b23400 msgr2=0x560ea8dc1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:25.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.262+0000 7ff0da600640 1 --2- 172.21.15.86:0/1692055750 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560ea9b23400 0x560ea8dc1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:25.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.262+0000 7ff0da600640 1 -- 172.21.15.86:0/1692055750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ea9b23000 msgr2=0x560ea8dc1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:25.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.262+0000 7ff0da600640 1 --2- 172.21.15.86:0/1692055750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ea9b23000 0x560ea8dc1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:25.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.262+0000 7ff0da600640 1 -- 172.21.15.86:0/1692055750 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ea8c83860 con 0x560ea9b22c00 2024-04-20T21:29:25.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.263+0000 7ff0da600640 1 --2- 172.21.15.86:0/1692055750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ea9b22c00 0x560ea8dc0000 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x560ea9b381e0 tx=0x560ea9b3acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f0ee04617135694 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:25.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.263+0000 7ff0d5600640 1 -- 172.21.15.86:0/1692055750 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x560ea8c749c0 con 0x560ea9b22c00 2024-04-20T21:29:25.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.263+0000 7ff0d5600640 1 -- 172.21.15.86:0/1692055750 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x560ea8c7ae00 con 0x560ea9b22c00 2024-04-20T21:29:25.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.263+0000 7ff0d5600640 1 -- 172.21.15.86:0/1692055750 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x560ea8c74b60 con 0x560ea9b22c00 2024-04-20T21:29:25.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.263+0000 7ff0dd836280 1 -- 172.21.15.86:0/1692055750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ea9b22c00 msgr2=0x560ea8dc0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:25.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.263+0000 7ff0dd836280 1 --2- 172.21.15.86:0/1692055750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ea9b22c00 0x560ea8dc0000 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x560ea9b381e0 tx=0x560ea9b3acc0 comp rx=0 tx=0).stop 2024-04-20T21:29:25.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.264+0000 7ff0dd836280 1 -- 172.21.15.86:0/1692055750 shutdown_connections 2024-04-20T21:29:25.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.264+0000 7ff0dd836280 1 --2- 172.21.15.86:0/1692055750 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560ea9b23400 0x560ea8dc1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:25.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.264+0000 7ff0dd836280 1 --2- 172.21.15.86:0/1692055750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ea9b23000 0x560ea8dc1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:25.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.264+0000 7ff0dd836280 1 --2- 172.21.15.86:0/1692055750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ea9b22c00 0x560ea8dc0000 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:25.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.264+0000 7ff0dd836280 1 -- 172.21.15.86:0/1692055750 >> 172.21.15.86:0/1692055750 conn(0x560ea99e6400 msgr2=0x560ea99d3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:25.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.264+0000 7ff0dd836280 1 -- 172.21.15.86:0/1692055750 shutdown_connections 2024-04-20T21:29:25.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.264+0000 7ff0dd836280 1 -- 172.21.15.86:0/1692055750 wait complete. 2024-04-20T21:29:25.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.264+0000 7ff0dd836280 1 Processor -- start 2024-04-20T21:29:25.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.264+0000 7ff0dd836280 1 -- start start 2024-04-20T21:29:25.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.265+0000 7ff0dd836280 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560ea9b22c00 0x560ea8dc0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:25.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.265+0000 7ff0dd836280 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ea9b23000 0x560ea8dc1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:25.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.265+0000 7ff0dd836280 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ea9b23400 0x560ea8dc1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:25.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.265+0000 7ff0dd836280 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560ea9a3be00 con 0x560ea9b23400 2024-04-20T21:29:25.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.265+0000 7ff0dd836280 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x560ea9a3bc80 con 0x560ea9b23000 2024-04-20T21:29:25.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.265+0000 7ff0dd836280 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x560ea9a3bb00 con 0x560ea9b22c00 2024-04-20T21:29:25.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.265+0000 7ff0da600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ea9b23400 0x560ea8dc1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:25.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.265+0000 7ff0da600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ea9b23400 0x560ea8dc1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46292/0 (socket says 172.21.15.86:46292) 2024-04-20T21:29:25.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.265+0000 7ff0da600640 1 -- 172.21.15.86:0/3472645699 learned_addr learned my addr 172.21.15.86:0/3472645699 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:25.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.266+0000 7ff0da600640 1 --2- 172.21.15.86:0/3472645699 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560ea9b22c00 0x560ea8dc0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:25.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.266+0000 7ff0db000640 1 --2- 172.21.15.86:0/3472645699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ea9b23000 0x560ea8dc1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:25.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.266+0000 7ff0da600640 1 -- 172.21.15.86:0/3472645699 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560ea9b22c00 msgr2=0x560ea8dc0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:25.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.266+0000 7ff0da600640 1 --2- 172.21.15.86:0/3472645699 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560ea9b22c00 0x560ea8dc0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:25.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.266+0000 7ff0da600640 1 -- 172.21.15.86:0/3472645699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ea9b23000 msgr2=0x560ea8dc1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:25.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.266+0000 7ff0da600640 1 --2- 172.21.15.86:0/3472645699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ea9b23000 0x560ea8dc1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:25.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.266+0000 7ff0da600640 1 -- 172.21.15.86:0/3472645699 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ea8c82780 con 0x560ea9b23400 2024-04-20T21:29:25.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.266+0000 7ff0db000640 1 --2- 172.21.15.86:0/3472645699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ea9b23000 0x560ea8dc1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:29:25.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.266+0000 7ff0da600640 1 --2- 172.21.15.86:0/3472645699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ea9b23400 0x560ea8dc1b80 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x560ea9b39290 tx=0x560ea9b3ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:25.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.266+0000 7ff0d3800640 1 -- 172.21.15.86:0/3472645699 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x560ea8c749c0 con 0x560ea9b23400 2024-04-20T21:29:25.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.266+0000 7ff0d3800640 1 -- 172.21.15.86:0/3472645699 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x560ea8c7ac40 con 0x560ea9b23400 2024-04-20T21:29:25.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.266+0000 7ff0d3800640 1 -- 172.21.15.86:0/3472645699 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x560ea8c74b60 con 0x560ea9b23400 2024-04-20T21:29:25.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.266+0000 7ff0dd836280 1 -- 172.21.15.86:0/3472645699 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560ea8c82000 con 0x560ea9b23400 2024-04-20T21:29:25.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.266+0000 7ff0dd836280 1 -- 172.21.15.86:0/3472645699 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ea8c83680 con 0x560ea9b23400 2024-04-20T21:29:25.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.266+0000 7ff0dd836280 4 main: resolving pool 14 2024-04-20T21:29:25.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.268+0000 7ff0d3800640 1 -- 172.21.15.86:0/3472645699 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x560ea9b90c00 con 0x560ea9b23400 2024-04-20T21:29:25.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.269+0000 7ff0d3800640 1 --2- 172.21.15.86:0/3472645699 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x560ea9b92c00 0x560ea8dc2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:25.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.269+0000 7ff0d3800640 1 -- 172.21.15.86:0/3472645699 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5881+0+0 (secure 0 0 0) 0x560ea8dc4200 con 0x560ea9b23400 2024-04-20T21:29:25.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.269+0000 7ff0dd836280 4 main: creating IoCtx.. 2024-04-20T21:29:25.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.269+0000 7ff0dd836280 1 --2- 172.21.15.86:0/3472645699 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x560ea9b23800 0x560ea8dc2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:25.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.269+0000 7ff0dd836280 1 -- 172.21.15.86:0/3472645699 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:1 14.13 14:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102) -- 0x560ea9b23c00 con 0x560ea9b23800 2024-04-20T21:29:25.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.269+0000 7ff0db000640 1 --2- 172.21.15.86:0/3472645699 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x560ea9b92c00 0x560ea8dc2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:25.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.269+0000 7ff0da600640 1 --2- 172.21.15.86:0/3472645699 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x560ea9b23800 0x560ea8dc2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:25.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.270+0000 7ff0db000640 1 --2- 172.21.15.86:0/3472645699 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x560ea9b92c00 0x560ea8dc2100 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x560ea99e0b10 tx=0x560ea99e8720 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:25.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.270+0000 7ff0da600640 1 --2- 172.21.15.86:0/3472645699 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x560ea9b23800 0x560ea8dc2680 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:25.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.270+0000 7ff0da600640 1 -- 172.21.15.86:0/3472645699 <== osd.2 v2:172.21.15.86:6816/3437611468 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv7 ondisk = 0) ==== 157+0+34 (crc 0 0 0) 0x560ea99d8000 con 0x560ea9b23800 2024-04-20T21:29:25.271 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:29:25.271 INFO:teuthology.orchestra.run.smithi086.stdout: "0": { 2024-04-20T21:29:25.271 INFO:teuthology.orchestra.run.smithi086.stdout: "data": { 2024-04-20T21:29:25.271 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 5, 2024-04-20T21:29:25.271 INFO:teuthology.orchestra.run.smithi086.stdout: "inotable": { 2024-04-20T21:29:25.271 INFO:teuthology.orchestra.run.smithi086.stdout: "projected_free": [ 2024-04-20T21:29:25.271 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:29:25.271 INFO:teuthology.orchestra.run.smithi086.stdout: "start": 1099511627779, 2024-04-20T21:29:25.271 INFO:teuthology.orchestra.run.smithi086.stdout: "len": 1099511627773 2024-04-20T21:29:25.271 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:29:25.271 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-20T21:29:25.271 INFO:teuthology.orchestra.run.smithi086.stdout: "free": [ 2024-04-20T21:29:25.271 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:29:25.271 INFO:teuthology.orchestra.run.smithi086.stdout: "start": 1099511627779, 2024-04-20T21:29:25.271 INFO:teuthology.orchestra.run.smithi086.stdout: "len": 1099511627773 2024-04-20T21:29:25.272 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:29:25.272 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-20T21:29:25.272 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:29:25.272 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:29:25.272 INFO:teuthology.orchestra.run.smithi086.stdout: "result": 0 2024-04-20T21:29:25.272 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:29:25.272 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:29:25.272 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:29:25.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.270+0000 7ff0dd836280 1 -- 172.21.15.86:0/3472645699 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x560ea9b23800 msgr2=0x560ea8dc2680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:25.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.270+0000 7ff0dd836280 1 --2- 172.21.15.86:0/3472645699 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x560ea9b23800 0x560ea8dc2680 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:25.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.271+0000 7ff0dd836280 1 -- 172.21.15.86:0/3472645699 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x560ea9b92c00 msgr2=0x560ea8dc2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:25.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.271+0000 7ff0dd836280 1 --2- 172.21.15.86:0/3472645699 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x560ea9b92c00 0x560ea8dc2100 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x560ea99e0b10 tx=0x560ea99e8720 comp rx=0 tx=0).stop 2024-04-20T21:29:25.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.271+0000 7ff0dd836280 1 -- 172.21.15.86:0/3472645699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ea9b23400 msgr2=0x560ea8dc1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:25.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.271+0000 7ff0dd836280 1 --2- 172.21.15.86:0/3472645699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ea9b23400 0x560ea8dc1b80 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x560ea9b39290 tx=0x560ea9b3ac00 comp rx=0 tx=0).stop 2024-04-20T21:29:25.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.271+0000 7ff0db000640 1 -- 172.21.15.86:0/3472645699 reap_dead start 2024-04-20T21:29:25.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.271+0000 7ff0dd836280 1 -- 172.21.15.86:0/3472645699 shutdown_connections 2024-04-20T21:29:25.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.271+0000 7ff0dd836280 1 -- 172.21.15.86:0/3472645699 >> 172.21.15.86:0/3472645699 conn(0x560ea99e6400 msgr2=0x560ea99d3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:25.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.271+0000 7ff0dd836280 1 -- 172.21.15.86:0/3472645699 shutdown_connections 2024-04-20T21:29:25.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.271+0000 7ff0dd836280 1 -- 172.21.15.86:0/3472645699 wait complete. 2024-04-20T21:29:25.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.271+0000 7ff0dd836280 1 -- 172.21.15.86:0/3483305584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ea99e6800 msgr2=0x560ea8dc0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:25.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.271+0000 7ff0dd836280 1 --2- 172.21.15.86:0/3483305584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ea99e6800 0x560ea8dc0b00 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x560ea9a7d320 tx=0x560ea99ea120 comp rx=0 tx=0).stop 2024-04-20T21:29:25.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.271+0000 7ff0dd836280 1 -- 172.21.15.86:0/3483305584 shutdown_connections 2024-04-20T21:29:25.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.271+0000 7ff0dd836280 1 --2- 172.21.15.86:0/3483305584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ea99e6800 0x560ea8dc0b00 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:25.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.271+0000 7ff0dd836280 1 --2- 172.21.15.86:0/3483305584 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x560ea8c4fc00 0x560ea8dc1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:25.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.271+0000 7ff0dd836280 1 --2- 172.21.15.86:0/3483305584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x560ea99e6000 0x560ea8dc0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:25.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.271+0000 7ff0dd836280 1 -- 172.21.15.86:0/3483305584 >> 172.21.15.86:0/3483305584 conn(0x560ea8c4f000 msgr2=0x560ea99d4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:25.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.271+0000 7ff0dd836280 1 -- 172.21.15.86:0/3483305584 shutdown_connections 2024-04-20T21:29:25.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:25.271+0000 7ff0dd836280 1 -- 172.21.15.86:0/3483305584 wait complete. 2024-04-20T21:29:25.276 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '0', 'show', 'inode'] in time 0:00:00.107604, result: { "0": { "data": { "version": 5, "inotable": { "projected_free": [ { "start": 1099511627779, "len": 1099511627773 } ], "free": [ { "start": 1099511627779, "len": 1099511627773 } ] } }, "result": 0 } } 2024-04-20T21:29:25.276 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-20T21:29:25.276 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T21:29:25.356 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T21:29:25.465 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-20T21:29:25.466 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T21:29:25.466 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:29:25.466 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:29:25.524 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052731,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248996k,nr_inodes=812249,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T21:29:25.524 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-20T21:29:25.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:29:25.524 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-20T21:29:25.578 INFO:teuthology.orchestra.run.smithi086.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-20T21:29:25.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:29:25.579 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T21:29:25.579 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:29:25.579 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:29:25.639 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T21:29:25.639 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2024-04-20T21:29:25.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:29:25.639 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-20T21:29:25.692 INFO:teuthology.orchestra.run.smithi144.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-20T21:29:25.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:29:25.693 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T21:29:25.694 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:29:25.694 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:29:25.750 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052731,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248996k,nr_inodes=812249,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T21:29:25.750 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-20T21:29:25.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:29:25.751 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-20T21:29:25.797 INFO:teuthology.orchestra.run.smithi086.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-20T21:29:25.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:29:25.798 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T21:29:25.799 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:29:25.799 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:29:25.855 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T21:29:25.855 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2024-04-20T21:29:25.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:29:25.856 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-20T21:29:25.906 INFO:teuthology.orchestra.run.smithi144.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-20T21:29:25.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:29:25.908 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:29:26.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.117+0000 7ff97e000640 1 Processor -- start 2024-04-20T21:29:26.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.117+0000 7ff97e000640 1 -- start start 2024-04-20T21:29:26.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.119+0000 7ff97e000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff978146130 0x7ff97806e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:26.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.119+0000 7ff97e000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff978072410 0x7ff9780cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:26.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.119+0000 7ff97e000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff97806dc30 con 0x7ff9780ccdd0 2024-04-20T21:29:26.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.119+0000 7ff97e000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff9780cd8e0 con 0x7ff978146130 2024-04-20T21:29:26.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.119+0000 7ff97e000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff9780cdbc0 con 0x7ff978072410 2024-04-20T21:29:26.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.119+0000 7ff977400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff978146130 0x7ff97806e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:26.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.119+0000 7ff977400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff978146130 0x7ff97806e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43610/0 (socket says 172.21.15.86:43610) 2024-04-20T21:29:26.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.119+0000 7ff977400640 1 -- 172.21.15.86:0/1776428431 learned_addr learned my addr 172.21.15.86:0/1776428431 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:26.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.120+0000 7ff977e00640 1 --2- 172.21.15.86:0/1776428431 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff978072410 0x7ff9780cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:26.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.120+0000 7ff977400640 1 -- 172.21.15.86:0/1776428431 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff978072410 msgr2=0x7ff9780cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:26.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.120+0000 7ff977400640 1 --2- 172.21.15.86:0/1776428431 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff978072410 0x7ff9780cd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:26.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.120+0000 7ff977400640 1 -- 172.21.15.86:0/1776428431 >> v1:172.21.15.144:6789/0 conn(0x7ff9780ccdd0 legacy=0x7ff978071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:26.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.120+0000 7ff977400640 1 -- 172.21.15.86:0/1776428431 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff97806ded0 con 0x7ff978146130 2024-04-20T21:29:26.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.121+0000 7ff977400640 1 --2- 172.21.15.86:0/1776428431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff978146130 0x7ff97806e6d0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7ff968009870 tx=0x7ff96802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f100943ef60ae08e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:26.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.121+0000 7ff976000640 1 -- 172.21.15.86:0/1776428431 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff968034070 con 0x7ff978146130 2024-04-20T21:29:26.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.121+0000 7ff976000640 1 -- 172.21.15.86:0/1776428431 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff968030840 con 0x7ff978146130 2024-04-20T21:29:26.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.121+0000 7ff976000640 1 -- 172.21.15.86:0/1776428431 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff968030c10 con 0x7ff978146130 2024-04-20T21:29:26.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.122+0000 7ff97e000640 1 -- 172.21.15.86:0/1776428431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff978146130 msgr2=0x7ff97806e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:26.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.122+0000 7ff97e000640 1 --2- 172.21.15.86:0/1776428431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff978146130 0x7ff97806e6d0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7ff968009870 tx=0x7ff96802b980 comp rx=0 tx=0).stop 2024-04-20T21:29:26.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.122+0000 7ff97e000640 1 -- 172.21.15.86:0/1776428431 shutdown_connections 2024-04-20T21:29:26.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.122+0000 7ff97e000640 1 --2- 172.21.15.86:0/1776428431 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff978072410 0x7ff9780cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:26.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.122+0000 7ff97e000640 1 --2- 172.21.15.86:0/1776428431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff978146130 0x7ff97806e6d0 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:26.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.122+0000 7ff97e000640 1 -- 172.21.15.86:0/1776428431 >> 172.21.15.86:0/1776428431 conn(0x7ff9780b2360 msgr2=0x7ff9780b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:26.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.122+0000 7ff97e000640 1 -- 172.21.15.86:0/1776428431 shutdown_connections 2024-04-20T21:29:26.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.122+0000 7ff97e000640 1 -- 172.21.15.86:0/1776428431 wait complete. 2024-04-20T21:29:26.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.124+0000 7ff97e000640 1 Processor -- start 2024-04-20T21:29:26.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.124+0000 7ff97e000640 1 -- start start 2024-04-20T21:29:26.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.125+0000 7ff97e000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff978072410 0x7ff9780aef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:26.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.125+0000 7ff97e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9780ccdd0 0x7ff9781e7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:26.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.125+0000 7ff97e000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff978146130 0x7ff978134a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:26.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.125+0000 7ff97e000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff97815cc20 con 0x7ff9780ccdd0 2024-04-20T21:29:26.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.125+0000 7ff97e000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff97806f150 con 0x7ff978072410 2024-04-20T21:29:26.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.125+0000 7ff97e000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff97815cda0 con 0x7ff978146130 2024-04-20T21:29:26.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.125+0000 7ff976a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9780ccdd0 0x7ff9781e7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:26.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.125+0000 7ff976a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9780ccdd0 0x7ff9781e7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43624/0 (socket says 172.21.15.86:43624) 2024-04-20T21:29:26.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.125+0000 7ff976a00640 1 -- 172.21.15.86:0/2971468492 learned_addr learned my addr 172.21.15.86:0/2971468492 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:26.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.125+0000 7ff977400640 1 --2- 172.21.15.86:0/2971468492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff978072410 0x7ff9780aef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:26.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.126+0000 7ff976a00640 1 -- 172.21.15.86:0/2971468492 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff978146130 msgr2=0x7ff978134a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:26.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.126+0000 7ff976a00640 1 --2- 172.21.15.86:0/2971468492 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff978146130 0x7ff978134a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:26.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.126+0000 7ff976a00640 1 -- 172.21.15.86:0/2971468492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff978072410 msgr2=0x7ff9780aef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:26.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.126+0000 7ff976a00640 1 --2- 172.21.15.86:0/2971468492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff978072410 0x7ff9780aef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:26.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.126+0000 7ff976a00640 1 -- 172.21.15.86:0/2971468492 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9781e77d0 con 0x7ff9780ccdd0 2024-04-20T21:29:26.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.126+0000 7ff976a00640 1 --2- 172.21.15.86:0/2971468492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9780ccdd0 0x7ff9781e7050 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7ff96400c430 tx=0x7ff96400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:26.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.126+0000 7ff96f400640 1 -- 172.21.15.86:0/2971468492 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff964011790 con 0x7ff9780ccdd0 2024-04-20T21:29:26.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.126+0000 7ff96f400640 1 -- 172.21.15.86:0/2971468492 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff964017070 con 0x7ff9780ccdd0 2024-04-20T21:29:26.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.126+0000 7ff97e000640 1 -- 172.21.15.86:0/2971468492 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9781e7ab0 con 0x7ff9780ccdd0 2024-04-20T21:29:26.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.126+0000 7ff96f400640 1 -- 172.21.15.86:0/2971468492 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff964011bb0 con 0x7ff9780ccdd0 2024-04-20T21:29:26.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.126+0000 7ff97e000640 1 -- 172.21.15.86:0/2971468492 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9781e8070 con 0x7ff9780ccdd0 2024-04-20T21:29:26.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.127+0000 7ff97e000640 1 -- 172.21.15.86:0/2971468492 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9780689e0 con 0x7ff9780ccdd0 2024-04-20T21:29:26.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.128+0000 7ff96f400640 1 -- 172.21.15.86:0/2971468492 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff96401c460 con 0x7ff9780ccdd0 2024-04-20T21:29:26.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.129+0000 7ff96f400640 1 --2- 172.21.15.86:0/2971468492 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff95806c920 0x7ff95806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:26.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.130+0000 7ff96f400640 1 -- 172.21.15.86:0/2971468492 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5881+0+0 (secure 0 0 0) 0x7ff96408bb20 con 0x7ff9780ccdd0 2024-04-20T21:29:26.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.130+0000 7ff977400640 1 --2- 172.21.15.86:0/2971468492 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff95806c920 0x7ff95806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:26.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.130+0000 7ff977400640 1 --2- 172.21.15.86:0/2971468492 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff95806c920 0x7ff95806ede0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7ff968036000 tx=0x7ff96803b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:26.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.134+0000 7ff96f400640 1 -- 172.21.15.86:0/2971468492 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff964059e00 con 0x7ff9780ccdd0 2024-04-20T21:29:26.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.378+0000 7ff97e000640 1 -- 172.21.15.86:0/2971468492 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff97815ff50 con 0x7ff9780ccdd0 2024-04-20T21:29:26.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.379+0000 7ff96f400640 1 -- 172.21.15.86:0/2971468492 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+5194 (secure 0 0 0) 0x7ff964059700 con 0x7ff9780ccdd0 2024-04-20T21:29:26.378 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:29:26.378 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":90,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5525,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3902778887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3902778887},{"type":"v1","addr":"172.21.15.144:6835","nonce":3902778887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6001,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/4270988014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":4270988014},{"type":"v1","addr":"172.21.15.86:6837","nonce":4270988014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6016,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/1569462554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":1569462554},{"type":"v1","addr":"172.21.15.144:6833","nonce":1569462554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":9066,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/4050537846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4050537846},{"type":"v1","addr":"172.21.15.86:6836","nonce":4050537846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":89,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:27:45.692408+0000","modified":"2024-04-20T21:29:25.141526+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":102,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-04-20T21:29:26.378 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 90 2024-04-20T21:29:26.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.382+0000 7ff97e000640 1 -- 172.21.15.86:0/2971468492 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff95806c920 msgr2=0x7ff95806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:26.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.382+0000 7ff97e000640 1 --2- 172.21.15.86:0/2971468492 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff95806c920 0x7ff95806ede0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7ff968036000 tx=0x7ff96803b040 comp rx=0 tx=0).stop 2024-04-20T21:29:26.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.382+0000 7ff97e000640 1 -- 172.21.15.86:0/2971468492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9780ccdd0 msgr2=0x7ff9781e7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:26.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.382+0000 7ff97e000640 1 --2- 172.21.15.86:0/2971468492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9780ccdd0 0x7ff9781e7050 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7ff96400c430 tx=0x7ff96400c8f0 comp rx=0 tx=0).stop 2024-04-20T21:29:26.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.382+0000 7ff97e000640 1 -- 172.21.15.86:0/2971468492 shutdown_connections 2024-04-20T21:29:26.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.382+0000 7ff97e000640 1 --2- 172.21.15.86:0/2971468492 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff95806c920 0x7ff95806ede0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:26.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.382+0000 7ff97e000640 1 --2- 172.21.15.86:0/2971468492 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff978146130 0x7ff978134a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:26.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.382+0000 7ff97e000640 1 --2- 172.21.15.86:0/2971468492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9780ccdd0 0x7ff9781e7050 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:26.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.382+0000 7ff97e000640 1 --2- 172.21.15.86:0/2971468492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff978072410 0x7ff9780aef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:26.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.382+0000 7ff97e000640 1 -- 172.21.15.86:0/2971468492 >> 172.21.15.86:0/2971468492 conn(0x7ff9780b2360 msgr2=0x7ff978071d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:26.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.382+0000 7ff97e000640 1 -- 172.21.15.86:0/2971468492 shutdown_connections 2024-04-20T21:29:26.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.382+0000 7ff97e000640 1 -- 172.21.15.86:0/2971468492 wait complete. 2024-04-20T21:29:26.401 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:29:26.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.599+0000 7ff001200640 1 Processor -- start 2024-04-20T21:29:26.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.599+0000 7ff001200640 1 -- start start 2024-04-20T21:29:26.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.600+0000 7ff001200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feffc156ae0 0x7feffc15d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:26.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.600+0000 7ff001200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7feffc160080 0x7feffc0ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:26.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.600+0000 7ff001200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feffc160890 con 0x7feffc0ad620 2024-04-20T21:29:26.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.601+0000 7ff001200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7feffc06db20 con 0x7feffc160080 2024-04-20T21:29:26.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.601+0000 7ff001200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7feffc160b70 con 0x7feffc156ae0 2024-04-20T21:29:26.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.601+0000 7feffaa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7feffc160080 0x7feffc0ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:26.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.601+0000 7feffbe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7feffc0ad620 0x7feffc141cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51838/0 (socket says 172.21.15.86:51838) 2024-04-20T21:29:26.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.601+0000 7feffaa00640 1 --2- 172.21.15.86:0/3309335657 >> v2:172.21.15.86:3300/0 conn(0x7feffc160080 0x7feffc0ad0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43636/0 (socket says 172.21.15.86:43636) 2024-04-20T21:29:26.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.601+0000 7feffbe00640 1 -- 172.21.15.86:0/3309335657 learned_addr learned my addr 172.21.15.86:0/3309335657 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:29:26.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.601+0000 7feffb400640 1 --2- 172.21.15.86:0/3309335657 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feffc156ae0 0x7feffc15d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:26.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.601+0000 7feffaa00640 1 -- 172.21.15.86:0/3309335657 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feffc156ae0 msgr2=0x7feffc15d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:26.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.601+0000 7feffaa00640 1 --2- 172.21.15.86:0/3309335657 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feffc156ae0 0x7feffc15d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:26.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.601+0000 7feffaa00640 1 -- 172.21.15.86:0/3309335657 >> v1:172.21.15.86:6789/0 conn(0x7feffc0ad620 legacy=0x7feffc141cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:26.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.601+0000 7feffaa00640 1 -- 172.21.15.86:0/3309335657 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feffc160e50 con 0x7feffc160080 2024-04-20T21:29:26.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.601+0000 7feffb400640 1 --2- 172.21.15.86:0/3309335657 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feffc156ae0 0x7feffc15d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:29:26.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.602+0000 7feffaa00640 1 --2- 172.21.15.86:0/3309335657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feffc160080 0x7feffc0ad0e0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fefe80097a0 tx=0x7fefe802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d59ff6f09ba02d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:26.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.603+0000 7feffa000640 1 -- 172.21.15.86:0/3309335657 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe8035020 con 0x7feffc160080 2024-04-20T21:29:26.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.603+0000 7feffa000640 1 -- 172.21.15.86:0/3309335657 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefe8031810 con 0x7feffc160080 2024-04-20T21:29:26.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.603+0000 7feffa000640 1 -- 172.21.15.86:0/3309335657 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe8031be0 con 0x7feffc160080 2024-04-20T21:29:26.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.603+0000 7ff001200640 1 -- 172.21.15.86:0/3309335657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feffc160080 msgr2=0x7feffc0ad0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:26.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.603+0000 7ff001200640 1 --2- 172.21.15.86:0/3309335657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feffc160080 0x7feffc0ad0e0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fefe80097a0 tx=0x7fefe802b8b0 comp rx=0 tx=0).stop 2024-04-20T21:29:26.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.603+0000 7ff001200640 1 -- 172.21.15.86:0/3309335657 shutdown_connections 2024-04-20T21:29:26.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.603+0000 7ff001200640 1 --2- 172.21.15.86:0/3309335657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feffc160080 0x7feffc0ad0e0 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:26.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.603+0000 7ff001200640 1 --2- 172.21.15.86:0/3309335657 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feffc156ae0 0x7feffc15d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:26.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.603+0000 7ff001200640 1 -- 172.21.15.86:0/3309335657 >> 172.21.15.86:0/3309335657 conn(0x7feffc0d0170 msgr2=0x7feffc0d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:26.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.603+0000 7ff001200640 1 -- 172.21.15.86:0/3309335657 shutdown_connections 2024-04-20T21:29:26.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.603+0000 7ff001200640 1 -- 172.21.15.86:0/3309335657 wait complete. 2024-04-20T21:29:26.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.605+0000 7ff001200640 1 Processor -- start 2024-04-20T21:29:26.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.605+0000 7ff001200640 1 -- start start 2024-04-20T21:29:26.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.606+0000 7ff001200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feffc0ad620 0x7feffc158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:26.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.606+0000 7ff001200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feffc156ae0 0x7feffc158d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:26.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.606+0000 7ff001200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feffc160080 0x7feffc1592c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:26.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.606+0000 7ff001200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feffc160500 con 0x7feffc156ae0 2024-04-20T21:29:26.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.606+0000 7ff001200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7feffc156ec0 con 0x7feffc160080 2024-04-20T21:29:26.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.606+0000 7ff001200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7feffc06f2a0 con 0x7feffc0ad620 2024-04-20T21:29:26.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.606+0000 7feffaa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feffc156ae0 0x7feffc158d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:26.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.606+0000 7feffaa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feffc156ae0 0x7feffc158d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43644/0 (socket says 172.21.15.86:43644) 2024-04-20T21:29:26.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.606+0000 7feffaa00640 1 -- 172.21.15.86:0/101658503 learned_addr learned my addr 172.21.15.86:0/101658503 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:26.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.607+0000 7feffbe00640 1 --2- 172.21.15.86:0/101658503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feffc160080 0x7feffc1592c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:26.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.607+0000 7feffb400640 1 --2- 172.21.15.86:0/101658503 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feffc0ad620 0x7feffc158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:26.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.607+0000 7feffaa00640 1 -- 172.21.15.86:0/101658503 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feffc0ad620 msgr2=0x7feffc158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:26.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.607+0000 7feffaa00640 1 --2- 172.21.15.86:0/101658503 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feffc0ad620 0x7feffc158270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:26.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.607+0000 7feffaa00640 1 -- 172.21.15.86:0/101658503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feffc160080 msgr2=0x7feffc1592c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:26.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.607+0000 7feffaa00640 1 --2- 172.21.15.86:0/101658503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feffc160080 0x7feffc1592c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:26.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.607+0000 7feffaa00640 1 -- 172.21.15.86:0/101658503 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feffc15ea00 con 0x7feffc156ae0 2024-04-20T21:29:26.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.607+0000 7feffbe00640 1 --2- 172.21.15.86:0/101658503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feffc160080 0x7feffc1592c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:29:26.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.607+0000 7feffaa00640 1 --2- 172.21.15.86:0/101658503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feffc156ae0 0x7feffc158d80 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fefe802bec0 tx=0x7fefe80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:26.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.608+0000 7feff3400640 1 -- 172.21.15.86:0/101658503 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe8041070 con 0x7feffc156ae0 2024-04-20T21:29:26.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.608+0000 7feff3400640 1 -- 172.21.15.86:0/101658503 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefe803f070 con 0x7feffc156ae0 2024-04-20T21:29:26.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.608+0000 7ff001200640 1 -- 172.21.15.86:0/101658503 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feffc15ece0 con 0x7feffc156ae0 2024-04-20T21:29:26.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.608+0000 7feff3400640 1 -- 172.21.15.86:0/101658503 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe8035040 con 0x7feffc156ae0 2024-04-20T21:29:26.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.608+0000 7ff001200640 1 -- 172.21.15.86:0/101658503 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feffc15f2a0 con 0x7feffc156ae0 2024-04-20T21:29:26.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.609+0000 7ff001200640 1 -- 172.21.15.86:0/101658503 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feffc04fcf0 con 0x7feffc156ae0 2024-04-20T21:29:26.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.610+0000 7feff3400640 1 -- 172.21.15.86:0/101658503 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fefe8038020 con 0x7feffc156ae0 2024-04-20T21:29:26.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.611+0000 7feff3400640 1 --2- 172.21.15.86:0/101658503 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fefdc06c8f0 0x7fefdc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:26.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.611+0000 7feff3400640 1 -- 172.21.15.86:0/101658503 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5881+0+0 (secure 0 0 0) 0x7fefe80adbe0 con 0x7feffc156ae0 2024-04-20T21:29:26.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.611+0000 7feffb400640 1 --2- 172.21.15.86:0/101658503 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fefdc06c8f0 0x7fefdc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:26.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.612+0000 7feffb400640 1 --2- 172.21.15.86:0/101658503 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fefdc06c8f0 0x7fefdc06edb0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fefec008850 tx=0x7fefec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:26.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.615+0000 7feff3400640 1 -- 172.21.15.86:0/101658503 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fefe807be30 con 0x7feffc156ae0 2024-04-20T21:29:26.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.816+0000 7ff001200640 1 -- 172.21.15.86:0/101658503 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feffc15fa20 con 0x7feffc156ae0 2024-04-20T21:29:26.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.819+0000 7feff3400640 1 -- 172.21.15.86:0/101658503 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+16869 (secure 0 0 0) 0x7fefe807b730 con 0x7feffc156ae0 2024-04-20T21:29:26.819 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:29:26.819 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":102,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:29:25.141262+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:27:43.469685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-20T21:27:44.673505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6836/4271041811":"2024-04-21T21:29:25.141228+0000","172.21.15.86:6834/4271041811":"2024-04-21T21:29:25.141228+0000","172.21.15.86:6836/1331851138":"2024-04-21T21:28:47.841886+0000","172.21.15.86:6834/1331851138":"2024-04-21T21:28:47.841886+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:29:26.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.821+0000 7ff001200640 1 -- 172.21.15.86:0/101658503 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fefdc06c8f0 msgr2=0x7fefdc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:26.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.821+0000 7ff001200640 1 --2- 172.21.15.86:0/101658503 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fefdc06c8f0 0x7fefdc06edb0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fefec008850 tx=0x7fefec008040 comp rx=0 tx=0).stop 2024-04-20T21:29:26.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.822+0000 7ff001200640 1 -- 172.21.15.86:0/101658503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feffc156ae0 msgr2=0x7feffc158d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:26.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.822+0000 7ff001200640 1 --2- 172.21.15.86:0/101658503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feffc156ae0 0x7feffc158d80 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fefe802bec0 tx=0x7fefe80046c0 comp rx=0 tx=0).stop 2024-04-20T21:29:26.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.822+0000 7ff001200640 1 -- 172.21.15.86:0/101658503 shutdown_connections 2024-04-20T21:29:26.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.822+0000 7ff001200640 1 --2- 172.21.15.86:0/101658503 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fefdc06c8f0 0x7fefdc06edb0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:26.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.822+0000 7ff001200640 1 --2- 172.21.15.86:0/101658503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feffc160080 0x7feffc1592c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:26.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.822+0000 7ff001200640 1 --2- 172.21.15.86:0/101658503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feffc156ae0 0x7feffc158d80 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:26.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.822+0000 7ff001200640 1 --2- 172.21.15.86:0/101658503 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7feffc0ad620 0x7feffc158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:26.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.822+0000 7ff001200640 1 -- 172.21.15.86:0/101658503 >> 172.21.15.86:0/101658503 conn(0x7feffc0d0170 msgr2=0x7feffc0b07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:26.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.822+0000 7ff001200640 1 -- 172.21.15.86:0/101658503 shutdown_connections 2024-04-20T21:29:26.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:26.822+0000 7ff001200640 1 -- 172.21.15.86:0/101658503 wait complete. 2024-04-20T21:29:26.840 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:27:43.469685+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:27:44.673505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:29:26.840 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:29:27.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.037+0000 7f01d5400640 1 Processor -- start 2024-04-20T21:29:27.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.037+0000 7f01d5400640 1 -- start start 2024-04-20T21:29:27.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.039+0000 7f01d5400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01d00d05d0 0x7f01d006b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:27.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.039+0000 7f01d5400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f01d00ce6d0 0x7f01d00ae1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:27.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.039+0000 7f01d5400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01d006c010 con 0x7f01d00d0110 2024-04-20T21:29:27.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.039+0000 7f01d5400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f01d006c5d0 con 0x7f01d00ce6d0 2024-04-20T21:29:27.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.039+0000 7f01d5400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f01d00ce240 con 0x7f01d00d05d0 2024-04-20T21:29:27.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.040+0000 7f01d4a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f01d00ce6d0 0x7f01d00ae1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:27.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.040+0000 7f01c7400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f01d00d0110 0x7f01d00d09b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51852/0 (socket says 172.21.15.86:51852) 2024-04-20T21:29:27.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.040+0000 7f01c7400640 1 -- 172.21.15.86:0/993553871 learned_addr learned my addr 172.21.15.86:0/993553871 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:29:27.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.040+0000 7f01cfe00640 1 --2- 172.21.15.86:0/993553871 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01d00d05d0 0x7f01d006b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:27.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.040+0000 7f01cf400640 1 -- 172.21.15.86:0/993553871 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 75514499 0 0) 0x7f01d006c010 con 0x7f01d00d0110 2024-04-20T21:29:27.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.040+0000 7f01cf400640 1 -- 172.21.15.86:0/993553871 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f01b4003520 con 0x7f01d00d0110 2024-04-20T21:29:27.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.040+0000 7f01d4a00640 1 -- 172.21.15.86:0/993553871 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01d00d05d0 msgr2=0x7f01d006b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:27.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.040+0000 7f01d4a00640 1 --2- 172.21.15.86:0/993553871 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01d00d05d0 0x7f01d006b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:27.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.040+0000 7f01d4a00640 1 -- 172.21.15.86:0/993553871 >> v1:172.21.15.86:6789/0 conn(0x7f01d00d0110 legacy=0x7f01d00d09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:27.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.040+0000 7f01d4a00640 1 -- 172.21.15.86:0/993553871 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d006c2f0 con 0x7f01d00ce6d0 2024-04-20T21:29:27.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.040+0000 7f01cfe00640 1 --2- 172.21.15.86:0/993553871 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01d00d05d0 0x7f01d006b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:29:27.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.042+0000 7f01d4a00640 1 --2- 172.21.15.86:0/993553871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01d00ce6d0 0x7f01d00ae1d0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f01c80097a0 tx=0x7f01c802b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aa0438834e2bbec server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:27.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.042+0000 7f01cf400640 1 -- 172.21.15.86:0/993553871 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c8034070 con 0x7f01d00ce6d0 2024-04-20T21:29:27.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.042+0000 7f01cf400640 1 -- 172.21.15.86:0/993553871 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01c8030800 con 0x7f01d00ce6d0 2024-04-20T21:29:27.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.042+0000 7f01cf400640 1 -- 172.21.15.86:0/993553871 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c8030bd0 con 0x7f01d00ce6d0 2024-04-20T21:29:27.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.043+0000 7f01d5400640 1 -- 172.21.15.86:0/993553871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01d00ce6d0 msgr2=0x7f01d00ae1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:27.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.043+0000 7f01d5400640 1 --2- 172.21.15.86:0/993553871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01d00ce6d0 0x7f01d00ae1d0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f01c80097a0 tx=0x7f01c802b940 comp rx=0 tx=0).stop 2024-04-20T21:29:27.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.043+0000 7f01d5400640 1 -- 172.21.15.86:0/993553871 shutdown_connections 2024-04-20T21:29:27.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.043+0000 7f01d5400640 1 --2- 172.21.15.86:0/993553871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01d00ce6d0 0x7f01d00ae1d0 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:27.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.043+0000 7f01d5400640 1 --2- 172.21.15.86:0/993553871 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01d00d05d0 0x7f01d006b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:27.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.043+0000 7f01d5400640 1 -- 172.21.15.86:0/993553871 >> 172.21.15.86:0/993553871 conn(0x7f01d00b2ef0 msgr2=0x7f01d00b3300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:27.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.043+0000 7f01d5400640 1 -- 172.21.15.86:0/993553871 shutdown_connections 2024-04-20T21:29:27.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.043+0000 7f01d5400640 1 -- 172.21.15.86:0/993553871 wait complete. 2024-04-20T21:29:27.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.045+0000 7f01d5400640 1 Processor -- start 2024-04-20T21:29:27.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.045+0000 7f01d5400640 1 -- start start 2024-04-20T21:29:27.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.046+0000 7f01d5400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01d00ce6d0 0x7f01d0158f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:27.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.046+0000 7f01d5400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01d00d0110 0x7f01d014c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:27.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.046+0000 7f01d5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01d00d05d0 0x7f01d00c6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:27.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.046+0000 7f01d5400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f01d00ceb20 con 0x7f01d00d05d0 2024-04-20T21:29:27.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.046+0000 7f01d5400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f01d015ca90 con 0x7f01d00ce6d0 2024-04-20T21:29:27.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.046+0000 7f01d5400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f01d00cdac0 con 0x7f01d00d0110 2024-04-20T21:29:27.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.046+0000 7f01d4a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01d00d05d0 0x7f01d00c6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:27.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.046+0000 7f01d4a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01d00d05d0 0x7f01d00c6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43660/0 (socket says 172.21.15.86:43660) 2024-04-20T21:29:27.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.046+0000 7f01d4a00640 1 -- 172.21.15.86:0/508101017 learned_addr learned my addr 172.21.15.86:0/508101017 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:27.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.047+0000 7f01cfe00640 1 --2- 172.21.15.86:0/508101017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01d00ce6d0 0x7f01d0158f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:27.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.047+0000 7f01d4a00640 1 -- 172.21.15.86:0/508101017 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01d00d0110 msgr2=0x7f01d014c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:27.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.047+0000 7f01c7400640 1 --2- 172.21.15.86:0/508101017 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01d00d0110 0x7f01d014c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:27.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.047+0000 7f01d4a00640 1 --2- 172.21.15.86:0/508101017 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01d00d0110 0x7f01d014c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:27.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.047+0000 7f01d4a00640 1 -- 172.21.15.86:0/508101017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01d00ce6d0 msgr2=0x7f01d0158f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:27.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.047+0000 7f01d4a00640 1 --2- 172.21.15.86:0/508101017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01d00ce6d0 0x7f01d0158f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:27.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.047+0000 7f01d4a00640 1 -- 172.21.15.86:0/508101017 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d01595a0 con 0x7f01d00d05d0 2024-04-20T21:29:27.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.047+0000 7f01c7400640 1 --2- 172.21.15.86:0/508101017 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01d00d0110 0x7f01d014c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:29:27.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.047+0000 7f01cfe00640 1 --2- 172.21.15.86:0/508101017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01d00ce6d0 0x7f01d0158f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:29:27.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.047+0000 7f01d4a00640 1 --2- 172.21.15.86:0/508101017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01d00d05d0 0x7f01d00c6a90 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f01c80098c0 tx=0x7f01c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:27.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.047+0000 7f01ccc00640 1 -- 172.21.15.86:0/508101017 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c8034040 con 0x7f01d00d05d0 2024-04-20T21:29:27.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.047+0000 7f01ccc00640 1 -- 172.21.15.86:0/508101017 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01c8030600 con 0x7f01d00d05d0 2024-04-20T21:29:27.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.047+0000 7f01ccc00640 1 -- 172.21.15.86:0/508101017 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c8006e10 con 0x7f01d00d05d0 2024-04-20T21:29:27.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.047+0000 7f01d5400640 1 -- 172.21.15.86:0/508101017 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01d0159880 con 0x7f01d00d05d0 2024-04-20T21:29:27.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.047+0000 7f01d5400640 1 -- 172.21.15.86:0/508101017 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01d0159e40 con 0x7f01d00d05d0 2024-04-20T21:29:27.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.049+0000 7f01d5400640 1 -- 172.21.15.86:0/508101017 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01d004fd60 con 0x7f01d00d05d0 2024-04-20T21:29:27.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.049+0000 7f01ccc00640 1 -- 172.21.15.86:0/508101017 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f01c8037070 con 0x7f01d00d05d0 2024-04-20T21:29:27.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.050+0000 7f01ccc00640 1 --2- 172.21.15.86:0/508101017 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f01ac06c8f0 0x7f01ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:27.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.050+0000 7f01ccc00640 1 -- 172.21.15.86:0/508101017 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5881+0+0 (secure 0 0 0) 0x7f01c80adcd0 con 0x7f01d00d05d0 2024-04-20T21:29:27.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.051+0000 7f01cfe00640 1 --2- 172.21.15.86:0/508101017 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f01ac06c8f0 0x7f01ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:27.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.051+0000 7f01cfe00640 1 --2- 172.21.15.86:0/508101017 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f01ac06c8f0 0x7f01ac06edb0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f01c0009540 tx=0x7f01c000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:27.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.055+0000 7f01ccc00640 1 -- 172.21.15.86:0/508101017 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f01c80307d0 con 0x7f01d00d05d0 2024-04-20T21:29:27.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.263+0000 7f01d5400640 1 -- 172.21.15.86:0/508101017 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f01d006dd10 con 0x7f01d00d05d0 2024-04-20T21:29:27.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.263+0000 7f01ccc00640 1 -- 172.21.15.86:0/508101017 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v102) ==== 93+0+10 (secure 0 0 0) 0x7f01c807bb10 con 0x7f01d00d05d0 2024-04-20T21:29:27.263 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:29:27.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.266+0000 7f01d5400640 1 -- 172.21.15.86:0/508101017 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f01ac06c8f0 msgr2=0x7f01ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:27.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.266+0000 7f01d5400640 1 --2- 172.21.15.86:0/508101017 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f01ac06c8f0 0x7f01ac06edb0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f01c0009540 tx=0x7f01c000b040 comp rx=0 tx=0).stop 2024-04-20T21:29:27.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.266+0000 7f01d5400640 1 -- 172.21.15.86:0/508101017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01d00d05d0 msgr2=0x7f01d00c6a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:27.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.266+0000 7f01d5400640 1 --2- 172.21.15.86:0/508101017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01d00d05d0 0x7f01d00c6a90 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f01c80098c0 tx=0x7f01c80046c0 comp rx=0 tx=0).stop 2024-04-20T21:29:27.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.266+0000 7f01d5400640 1 -- 172.21.15.86:0/508101017 shutdown_connections 2024-04-20T21:29:27.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.266+0000 7f01d5400640 1 --2- 172.21.15.86:0/508101017 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f01ac06c8f0 0x7f01ac06edb0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:27.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.266+0000 7f01d5400640 1 --2- 172.21.15.86:0/508101017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01d00d05d0 0x7f01d00c6a90 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:27.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.266+0000 7f01d5400640 1 --2- 172.21.15.86:0/508101017 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01d00d0110 0x7f01d014c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:27.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.266+0000 7f01d5400640 1 --2- 172.21.15.86:0/508101017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01d00ce6d0 0x7f01d0158f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:27.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.266+0000 7f01d5400640 1 -- 172.21.15.86:0/508101017 >> 172.21.15.86:0/508101017 conn(0x7f01d00b2ef0 msgr2=0x7f01d00ae1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:27.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.266+0000 7f01d5400640 1 -- 172.21.15.86:0/508101017 shutdown_connections 2024-04-20T21:29:27.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.267+0000 7f01d5400640 1 -- 172.21.15.86:0/508101017 wait complete. 2024-04-20T21:29:27.284 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:29:27.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.485+0000 7f7d03200640 1 Processor -- start 2024-04-20T21:29:27.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.486+0000 7f7d03200640 1 -- start start 2024-04-20T21:29:27.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.487+0000 7f7d03200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7cfc145250 0x7f7cfc06efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:27.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.487+0000 7f7d03200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7cfc145f00 0x7f7cfc071d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:27.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.487+0000 7f7d03200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cfc06f500 con 0x7f7cfc072280 2024-04-20T21:29:27.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.487+0000 7f7d03200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f7cfc0726b0 con 0x7f7cfc145f00 2024-04-20T21:29:27.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.487+0000 7f7d03200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f7cfc160410 con 0x7f7cfc145250 2024-04-20T21:29:27.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.488+0000 7f7d01400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7cfc145f00 0x7f7cfc071d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:27.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.488+0000 7f7d01400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7cfc145f00 0x7f7cfc071d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43670/0 (socket says 172.21.15.86:43670) 2024-04-20T21:29:27.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.488+0000 7f7d01400640 1 -- 172.21.15.86:0/3705822733 learned_addr learned my addr 172.21.15.86:0/3705822733 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:27.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.488+0000 7f7d01e00640 1 --2- 172.21.15.86:0/3705822733 >> v2:172.21.15.144:3300/0 conn(0x7f7cfc145250 0x7f7cfc06efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:27.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.488+0000 7f7d01400640 1 -- 172.21.15.86:0/3705822733 >> v2:172.21.15.144:3300/0 conn(0x7f7cfc145250 msgr2=0x7f7cfc06efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:27.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.488+0000 7f7d01400640 1 --2- 172.21.15.86:0/3705822733 >> v2:172.21.15.144:3300/0 conn(0x7f7cfc145250 0x7f7cfc06efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:27.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.488+0000 7f7d01400640 1 -- 172.21.15.86:0/3705822733 >> v1:172.21.15.86:6789/0 conn(0x7f7cfc072280 legacy=0x7f7cfc06dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:27.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.488+0000 7f7d01400640 1 -- 172.21.15.86:0/3705822733 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cfc1606f0 con 0x7f7cfc145f00 2024-04-20T21:29:27.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.488+0000 7f7d01e00640 1 --2- 172.21.15.86:0/3705822733 >> v2:172.21.15.144:3300/0 conn(0x7f7cfc145250 0x7f7cfc06efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:29:27.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.489+0000 7f7d01400640 1 --2- 172.21.15.86:0/3705822733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cfc145f00 0x7f7cfc071d40 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc072bd0 tx=0x7f7cec02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b88f3354a0b6e55 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:27.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.490+0000 7f7d00a00640 1 -- 172.21.15.86:0/3705822733 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cec035020 con 0x7f7cfc145f00 2024-04-20T21:29:27.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.490+0000 7f7d00a00640 1 -- 172.21.15.86:0/3705822733 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cec031870 con 0x7f7cfc145f00 2024-04-20T21:29:27.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.490+0000 7f7d00a00640 1 -- 172.21.15.86:0/3705822733 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cec031c40 con 0x7f7cfc145f00 2024-04-20T21:29:27.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.490+0000 7f7d03200640 1 -- 172.21.15.86:0/3705822733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cfc145f00 msgr2=0x7f7cfc071d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:27.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.490+0000 7f7d03200640 1 --2- 172.21.15.86:0/3705822733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cfc145f00 0x7f7cfc071d40 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc072bd0 tx=0x7f7cec02b910 comp rx=0 tx=0).stop 2024-04-20T21:29:27.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.491+0000 7f7d03200640 1 -- 172.21.15.86:0/3705822733 shutdown_connections 2024-04-20T21:29:27.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.491+0000 7f7d03200640 1 --2- 172.21.15.86:0/3705822733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cfc145f00 0x7f7cfc071d40 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:27.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.491+0000 7f7d03200640 1 --2- 172.21.15.86:0/3705822733 >> v2:172.21.15.144:3300/0 conn(0x7f7cfc145250 0x7f7cfc06efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:27.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.491+0000 7f7d03200640 1 -- 172.21.15.86:0/3705822733 >> 172.21.15.86:0/3705822733 conn(0x7f7cfc067cf0 msgr2=0x7f7cfc06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:27.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.491+0000 7f7d03200640 1 -- 172.21.15.86:0/3705822733 shutdown_connections 2024-04-20T21:29:27.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.491+0000 7f7d03200640 1 -- 172.21.15.86:0/3705822733 wait complete. 2024-04-20T21:29:27.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.492+0000 7f7d03200640 1 Processor -- start 2024-04-20T21:29:27.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.493+0000 7f7d03200640 1 -- start start 2024-04-20T21:29:27.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.493+0000 7f7d03200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cfc072280 0x7f7cfc15e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:27.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.494+0000 7f7d03200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cfc145250 0x7f7cfc15bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:27.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.494+0000 7f7d03200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7cfc145f00 0x7f7cfc134640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:27.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.494+0000 7f7d03200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7cfc145630 con 0x7f7cfc145250 2024-04-20T21:29:27.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.494+0000 7f7d03200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7cfc160cb0 con 0x7f7cfc072280 2024-04-20T21:29:27.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.494+0000 7f7d03200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7cfc1457b0 con 0x7f7cfc145f00 2024-04-20T21:29:27.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.494+0000 7f7d01400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cfc145250 0x7f7cfc15bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:27.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.494+0000 7f7d01400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cfc145250 0x7f7cfc15bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43676/0 (socket says 172.21.15.86:43676) 2024-04-20T21:29:27.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.494+0000 7f7d01400640 1 -- 172.21.15.86:0/1883142035 learned_addr learned my addr 172.21.15.86:0/1883142035 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:27.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.494+0000 7f7d01e00640 1 --2- 172.21.15.86:0/1883142035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cfc072280 0x7f7cfc15e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:27.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.494+0000 7f7d01400640 1 -- 172.21.15.86:0/1883142035 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7cfc145f00 msgr2=0x7f7cfc134640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:27.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.494+0000 7f7d01400640 1 --2- 172.21.15.86:0/1883142035 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7cfc145f00 0x7f7cfc134640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:27.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.494+0000 7f7d01400640 1 -- 172.21.15.86:0/1883142035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cfc072280 msgr2=0x7f7cfc15e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:27.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.494+0000 7f7d01400640 1 --2- 172.21.15.86:0/1883142035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cfc072280 0x7f7cfc15e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:27.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.494+0000 7f7d01400640 1 -- 172.21.15.86:0/1883142035 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cfc0b3f60 con 0x7f7cfc145250 2024-04-20T21:29:27.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.494+0000 7f7d01e00640 1 --2- 172.21.15.86:0/1883142035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cfc072280 0x7f7cfc15e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:29:27.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.495+0000 7f7d01400640 1 --2- 172.21.15.86:0/1883142035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cfc145250 0x7f7cfc15bce0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f7cec02b950 tx=0x7f7cec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:27.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.495+0000 7f7cf6000640 1 -- 172.21.15.86:0/1883142035 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cec035040 con 0x7f7cfc145250 2024-04-20T21:29:27.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.495+0000 7f7cf6000640 1 -- 172.21.15.86:0/1883142035 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cec045480 con 0x7f7cfc145250 2024-04-20T21:29:27.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.495+0000 7f7cf6000640 1 -- 172.21.15.86:0/1883142035 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cec045870 con 0x7f7cfc145250 2024-04-20T21:29:27.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.495+0000 7f7d03200640 1 -- 172.21.15.86:0/1883142035 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cfc0cc6e0 con 0x7f7cfc145250 2024-04-20T21:29:27.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.495+0000 7f7d03200640 1 -- 172.21.15.86:0/1883142035 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cfc0b44c0 con 0x7f7cfc145250 2024-04-20T21:29:27.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.497+0000 7f7d03200640 1 -- 172.21.15.86:0/1883142035 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cfc04fd90 con 0x7f7cfc145250 2024-04-20T21:29:27.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.497+0000 7f7cf6000640 1 -- 172.21.15.86:0/1883142035 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7cec038020 con 0x7f7cfc145250 2024-04-20T21:29:27.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.498+0000 7f7cf6000640 1 --2- 172.21.15.86:0/1883142035 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7ce006c920 0x7f7ce006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:27.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.498+0000 7f7cf6000640 1 -- 172.21.15.86:0/1883142035 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5881+0+0 (secure 0 0 0) 0x7f7cec0adfe0 con 0x7f7cfc145250 2024-04-20T21:29:27.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.499+0000 7f7d01e00640 1 --2- 172.21.15.86:0/1883142035 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7ce006c920 0x7f7ce006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:27.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.499+0000 7f7d01e00640 1 --2- 172.21.15.86:0/1883142035 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7ce006c920 0x7f7ce006ede0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0006fd0 tx=0x7f7cf0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:27.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.503+0000 7f7cf6000640 1 -- 172.21.15.86:0/1883142035 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7cfc0b3f60 con 0x7f7cfc145250 2024-04-20T21:29:27.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.700+0000 7f7d03200640 1 -- 172.21.15.86:0/1883142035 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7cfc072ea0 con 0x7f7cfc145250 2024-04-20T21:29:27.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.701+0000 7f7cf6000640 1 -- 172.21.15.86:0/1883142035 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v102) ==== 104+0+11 (secure 0 0 0) 0x7f7cec07c350 con 0x7f7cfc145250 2024-04-20T21:29:27.700 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-20T21:29:27.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.703+0000 7f7d03200640 1 -- 172.21.15.86:0/1883142035 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7ce006c920 msgr2=0x7f7ce006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:27.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.703+0000 7f7d03200640 1 --2- 172.21.15.86:0/1883142035 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7ce006c920 0x7f7ce006ede0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0006fd0 tx=0x7f7cf0008040 comp rx=0 tx=0).stop 2024-04-20T21:29:27.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.703+0000 7f7d03200640 1 -- 172.21.15.86:0/1883142035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cfc145250 msgr2=0x7f7cfc15bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:27.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.703+0000 7f7d03200640 1 --2- 172.21.15.86:0/1883142035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cfc145250 0x7f7cfc15bce0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f7cec02b950 tx=0x7f7cec0046c0 comp rx=0 tx=0).stop 2024-04-20T21:29:27.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.703+0000 7f7d03200640 1 -- 172.21.15.86:0/1883142035 shutdown_connections 2024-04-20T21:29:27.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.703+0000 7f7d03200640 1 --2- 172.21.15.86:0/1883142035 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7ce006c920 0x7f7ce006ede0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:27.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.703+0000 7f7d03200640 1 --2- 172.21.15.86:0/1883142035 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7cfc145f00 0x7f7cfc134640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:27.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.703+0000 7f7d03200640 1 --2- 172.21.15.86:0/1883142035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cfc145250 0x7f7cfc15bce0 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:27.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.703+0000 7f7d03200640 1 --2- 172.21.15.86:0/1883142035 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7cfc072280 0x7f7cfc15e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:27.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.703+0000 7f7d03200640 1 -- 172.21.15.86:0/1883142035 >> 172.21.15.86:0/1883142035 conn(0x7f7cfc067cf0 msgr2=0x7f7cfc0ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:27.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.704+0000 7f7d03200640 1 -- 172.21.15.86:0/1883142035 shutdown_connections 2024-04-20T21:29:27.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.704+0000 7f7d03200640 1 -- 172.21.15.86:0/1883142035 wait complete. 2024-04-20T21:29:27.722 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:29:27.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.931+0000 7f4ca0200640 1 Processor -- start 2024-04-20T21:29:27.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.931+0000 7f4ca0200640 1 -- start start 2024-04-20T21:29:27.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.932+0000 7f4ca0200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f4c980cf500 0x7f4c980b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:27.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.932+0000 7f4ca0200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c98156110 con 0x7f4c98155d30 2024-04-20T21:29:27.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.932+0000 7f4ca0200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c980ce460 con 0x7f4c9806c090 2024-04-20T21:29:27.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.932+0000 7f4ca0200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f4c980ce740 con 0x7f4c980cf500 2024-04-20T21:29:27.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.933+0000 7f4c9d800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f4c98155d30 0x7f4c9806b980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51868/0 (socket says 172.21.15.86:51868) 2024-04-20T21:29:27.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.933+0000 7f4c9d800640 1 -- 172.21.15.86:0/2537134583 learned_addr learned my addr 172.21.15.86:0/2537134583 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:29:27.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.933+0000 7f4c97e00640 1 -- 172.21.15.86:0/2537134583 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3330042008 0 0) 0x7f4c98156110 con 0x7f4c98155d30 2024-04-20T21:29:27.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.933+0000 7f4c97e00640 1 -- 172.21.15.86:0/2537134583 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c80005730 con 0x7f4c98155d30 2024-04-20T21:29:27.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.933+0000 7f4c9e200640 1 --2- 172.21.15.86:0/2537134583 >> v2:172.21.15.144:3300/0 conn(0x7f4c980cf500 0x7f4c980b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:27.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.934+0000 7f4c97e00640 1 -- 172.21.15.86:0/2537134583 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 450134056 0 0) 0x7f4c80005730 con 0x7f4c98155d30 2024-04-20T21:29:27.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.934+0000 7f4c97e00640 1 -- 172.21.15.86:0/2537134583 >> v2:172.21.15.144:3300/0 conn(0x7f4c980cf500 msgr2=0x7f4c980b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:27.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.934+0000 7f4c97e00640 1 --2- 172.21.15.86:0/2537134583 >> v2:172.21.15.144:3300/0 conn(0x7f4c980cf500 0x7f4c980b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:27.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.934+0000 7f4c97e00640 1 -- 172.21.15.86:0/2537134583 >> v1:172.21.15.144:6789/0 conn(0x7f4c9806c090 legacy=0x7f4c980b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:27.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.934+0000 7f4c97e00640 1 -- 172.21.15.86:0/2537134583 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c980cea20 con 0x7f4c98155d30 2024-04-20T21:29:27.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.934+0000 7f4c97e00640 1 -- 172.21.15.86:0/2537134583 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f4c8c001670 con 0x7f4c98155d30 2024-04-20T21:29:27.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.934+0000 7f4c97e00640 1 -- 172.21.15.86:0/2537134583 >> v1:172.21.15.86:6789/0 conn(0x7f4c98155d30 legacy=0x7f4c9806b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:27.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.934+0000 7f4c97e00640 1 --2- 172.21.15.86:0/2537134583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c80005c10 0x7f4c80006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:27.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.934+0000 7f4c97e00640 1 --2- 172.21.15.86:0/2537134583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c80006a10 0x7f4c80006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:27.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.934+0000 7f4c97e00640 1 --2- 172.21.15.86:0/2537134583 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c800075d0 0x7f4c8000dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:27.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.934+0000 7f4c97e00640 1 -- 172.21.15.86:0/2537134583 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4c980ce740 con 0x7f4c80006a10 2024-04-20T21:29:27.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.934+0000 7f4c97e00640 1 -- 172.21.15.86:0/2537134583 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8000e490 con 0x7f4c80005c10 2024-04-20T21:29:27.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.934+0000 7f4c97e00640 1 -- 172.21.15.86:0/2537134583 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f4c8000e690 con 0x7f4c800075d0 2024-04-20T21:29:27.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.934+0000 7f4c9ce00640 1 --2- 172.21.15.86:0/2537134583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c80006a10 0x7f4c80006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:27.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.934+0000 7f4c9d800640 1 --2- 172.21.15.86:0/2537134583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c80005c10 0x7f4c80006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:27.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.934+0000 7f4c9ce00640 1 -- 172.21.15.86:0/2537134583 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c800075d0 msgr2=0x7f4c8000dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:27.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.934+0000 7f4c9ce00640 1 --2- 172.21.15.86:0/2537134583 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c800075d0 0x7f4c8000dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:27.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.934+0000 7f4c9ce00640 1 -- 172.21.15.86:0/2537134583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c80005c10 msgr2=0x7f4c80006010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:27.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.934+0000 7f4c9ce00640 1 --2- 172.21.15.86:0/2537134583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c80005c10 0x7f4c80006010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:27.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.934+0000 7f4c9ce00640 1 -- 172.21.15.86:0/2537134583 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c80005730 con 0x7f4c80006a10 2024-04-20T21:29:27.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.935+0000 7f4c9e200640 1 -- 172.21.15.86:0/2537134583 reap_dead start 2024-04-20T21:29:27.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.936+0000 7f4c9ce00640 1 --2- 172.21.15.86:0/2537134583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c80006a10 0x7f4c80006e90 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f4c980cfbe0 tx=0x7f4c80007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e9faff59134202d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:27.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.936+0000 7f4c97e00640 1 -- 172.21.15.86:0/2537134583 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c84037070 con 0x7f4c80006a10 2024-04-20T21:29:27.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.936+0000 7f4c97e00640 1 -- 172.21.15.86:0/2537134583 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c84032600 con 0x7f4c80006a10 2024-04-20T21:29:27.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.936+0000 7f4c97e00640 1 -- 172.21.15.86:0/2537134583 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c840039a0 con 0x7f4c80006a10 2024-04-20T21:29:27.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.936+0000 7f4ca0200640 1 -- 172.21.15.86:0/2537134583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c80006a10 msgr2=0x7f4c80006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:27.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.936+0000 7f4ca0200640 1 --2- 172.21.15.86:0/2537134583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c80006a10 0x7f4c80006e90 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f4c980cfbe0 tx=0x7f4c80007a40 comp rx=0 tx=0).stop 2024-04-20T21:29:27.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.937+0000 7f4ca0200640 1 -- 172.21.15.86:0/2537134583 shutdown_connections 2024-04-20T21:29:27.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.937+0000 7f4ca0200640 1 --2- 172.21.15.86:0/2537134583 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c80006a10 0x7f4c80006e90 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:27.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.937+0000 7f4ca0200640 1 -- 172.21.15.86:0/2537134583 >> 172.21.15.86:0/2537134583 conn(0x7f4c980b2110 msgr2=0x7f4c980b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:27.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.937+0000 7f4ca0200640 1 -- 172.21.15.86:0/2537134583 shutdown_connections 2024-04-20T21:29:27.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.937+0000 7f4ca0200640 1 -- 172.21.15.86:0/2537134583 wait complete. 2024-04-20T21:29:27.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.938+0000 7f4ca0200640 1 Processor -- start 2024-04-20T21:29:27.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.939+0000 7f4ca0200640 1 -- start start 2024-04-20T21:29:27.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.940+0000 7f4ca0200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c80006a10 0x7f4c981ef940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:27.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.940+0000 7f4ca0200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c9814e240 0x7f4c98150b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:27.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.940+0000 7f4ca0200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c9814e6c0 0x7f4c98151040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:27.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.940+0000 7f4ca0200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9806b980 con 0x7f4c9814e6c0 2024-04-20T21:29:27.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.940+0000 7f4ca0200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4c980cefe0 con 0x7f4c9814e240 2024-04-20T21:29:27.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.940+0000 7f4ca0200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f4c9806bb00 con 0x7f4c80006a10 2024-04-20T21:29:27.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.940+0000 7f4c9e200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c9814e6c0 0x7f4c98151040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:27.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.940+0000 7f4c9e200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c9814e6c0 0x7f4c98151040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43686/0 (socket says 172.21.15.86:43686) 2024-04-20T21:29:27.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.940+0000 7f4c9e200640 1 -- 172.21.15.86:0/278065303 learned_addr learned my addr 172.21.15.86:0/278065303 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:27.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.940+0000 7f4c9d800640 1 --2- 172.21.15.86:0/278065303 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c80006a10 0x7f4c981ef940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:27.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.940+0000 7f4c9e200640 1 -- 172.21.15.86:0/278065303 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c80006a10 msgr2=0x7f4c981ef940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:27.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.940+0000 7f4c9e200640 1 --2- 172.21.15.86:0/278065303 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c80006a10 0x7f4c981ef940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:27.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.940+0000 7f4c9ce00640 1 --2- 172.21.15.86:0/278065303 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c9814e240 0x7f4c98150b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:27.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.940+0000 7f4c9e200640 1 -- 172.21.15.86:0/278065303 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c9814e240 msgr2=0x7f4c98150b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:27.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.940+0000 7f4c9e200640 1 --2- 172.21.15.86:0/278065303 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c9814e240 0x7f4c98150b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:27.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.940+0000 7f4c9e200640 1 -- 172.21.15.86:0/278065303 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c9814ebb0 con 0x7f4c9814e6c0 2024-04-20T21:29:27.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.941+0000 7f4c9e200640 1 --2- 172.21.15.86:0/278065303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c9814e6c0 0x7f4c98151040 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f4c980ccfd0 tx=0x7f4c9000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:27.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.941+0000 7f4c95600640 1 -- 172.21.15.86:0/278065303 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9000ed90 con 0x7f4c9814e6c0 2024-04-20T21:29:27.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.941+0000 7f4c95600640 1 -- 172.21.15.86:0/278065303 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c90019070 con 0x7f4c9814e6c0 2024-04-20T21:29:27.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.941+0000 7f4c95600640 1 -- 172.21.15.86:0/278065303 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c90014660 con 0x7f4c9814e6c0 2024-04-20T21:29:27.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.941+0000 7f4ca0200640 1 -- 172.21.15.86:0/278065303 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c9814ee90 con 0x7f4c9814e6c0 2024-04-20T21:29:27.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.941+0000 7f4ca0200640 1 -- 172.21.15.86:0/278065303 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c9814f450 con 0x7f4c9814e6c0 2024-04-20T21:29:27.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.943+0000 7f4ca0200640 1 -- 172.21.15.86:0/278065303 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c980689e0 con 0x7f4c9814e6c0 2024-04-20T21:29:27.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.943+0000 7f4c95600640 1 -- 172.21.15.86:0/278065303 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4c90014800 con 0x7f4c9814e6c0 2024-04-20T21:29:27.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.944+0000 7f4c95600640 1 --2- 172.21.15.86:0/278065303 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4c7806c8f0 0x7f4c7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:27.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.944+0000 7f4c95600640 1 -- 172.21.15.86:0/278065303 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5881+0+0 (secure 0 0 0) 0x7f4c9008c170 con 0x7f4c9814e6c0 2024-04-20T21:29:27.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.945+0000 7f4c9d800640 1 --2- 172.21.15.86:0/278065303 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4c7806c8f0 0x7f4c7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:27.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.945+0000 7f4c9d800640 1 --2- 172.21.15.86:0/278065303 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4c7806c8f0 0x7f4c7806edb0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c001c10 tx=0x7f4c8c001ba0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:27.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:27.949+0000 7f4c95600640 1 -- 172.21.15.86:0/278065303 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4c9005a4f0 con 0x7f4c9814e6c0 2024-04-20T21:29:28.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.147+0000 7f4ca0200640 1 -- 172.21.15.86:0/278065303 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4c980691e0 con 0x7f4c9814e6c0 2024-04-20T21:29:28.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.148+0000 7f4c95600640 1 -- 172.21.15.86:0/278065303 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v102) ==== 100+0+11 (secure 0 0 0) 0x7f4c90059df0 con 0x7f4c9814e6c0 2024-04-20T21:29:28.147 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-20T21:29:28.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.151+0000 7f4ca0200640 1 -- 172.21.15.86:0/278065303 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4c7806c8f0 msgr2=0x7f4c7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:28.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.151+0000 7f4ca0200640 1 --2- 172.21.15.86:0/278065303 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4c7806c8f0 0x7f4c7806edb0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c001c10 tx=0x7f4c8c001ba0 comp rx=0 tx=0).stop 2024-04-20T21:29:28.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.151+0000 7f4ca0200640 1 -- 172.21.15.86:0/278065303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c9814e6c0 msgr2=0x7f4c98151040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:28.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.151+0000 7f4ca0200640 1 --2- 172.21.15.86:0/278065303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c9814e6c0 0x7f4c98151040 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f4c980ccfd0 tx=0x7f4c9000e4b0 comp rx=0 tx=0).stop 2024-04-20T21:29:28.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.151+0000 7f4ca0200640 1 -- 172.21.15.86:0/278065303 shutdown_connections 2024-04-20T21:29:28.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.151+0000 7f4ca0200640 1 --2- 172.21.15.86:0/278065303 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4c7806c8f0 0x7f4c7806edb0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:28.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.151+0000 7f4ca0200640 1 --2- 172.21.15.86:0/278065303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c9814e6c0 0x7f4c98151040 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:28.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.151+0000 7f4ca0200640 1 --2- 172.21.15.86:0/278065303 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4c9814e240 0x7f4c98150b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:28.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.151+0000 7f4ca0200640 1 --2- 172.21.15.86:0/278065303 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4c80006a10 0x7f4c981ef940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:28.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.151+0000 7f4ca0200640 1 -- 172.21.15.86:0/278065303 >> 172.21.15.86:0/278065303 conn(0x7f4c980b2110 msgr2=0x7f4c980b4040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:28.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.151+0000 7f4ca0200640 1 -- 172.21.15.86:0/278065303 shutdown_connections 2024-04-20T21:29:28.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.151+0000 7f4ca0200640 1 -- 172.21.15.86:0/278065303 wait complete. 2024-04-20T21:29:28.170 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:29:28.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.363+0000 7fa565200640 1 Processor -- start 2024-04-20T21:29:28.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.363+0000 7fa565200640 1 -- start start 2024-04-20T21:29:28.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.365+0000 7fa565200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa56015d6a0 0x7fa5600ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:28.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.365+0000 7fa565200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa5600ad620 0x7fa56006e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:28.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.365+0000 7fa565200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa560160890 con 0x7fa560156ae0 2024-04-20T21:29:28.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.365+0000 7fa565200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa56006db20 con 0x7fa5600ad620 2024-04-20T21:29:28.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.365+0000 7fa565200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa560160b70 con 0x7fa56015d6a0 2024-04-20T21:29:28.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.365+0000 7fa55fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa5600ad620 0x7fa56006e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:28.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.365+0000 7fa55fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa5600ad620 0x7fa56006e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43702/0 (socket says 172.21.15.86:43702) 2024-04-20T21:29:28.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.365+0000 7fa55fe00640 1 -- 172.21.15.86:0/4025100249 learned_addr learned my addr 172.21.15.86:0/4025100249 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:28.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.366+0000 7fa55fe00640 1 -- 172.21.15.86:0/4025100249 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa56015d6a0 msgr2=0x7fa5600ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:28.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.365+0000 7fa55ea00640 1 --2- 172.21.15.86:0/4025100249 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa56015d6a0 0x7fa5600ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:28.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.366+0000 7fa55fe00640 1 --2- 172.21.15.86:0/4025100249 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa56015d6a0 0x7fa5600ad0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:28.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.366+0000 7fa55fe00640 1 -- 172.21.15.86:0/4025100249 >> v1:172.21.15.144:6789/0 conn(0x7fa560156ae0 legacy=0x7fa560160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:28.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.366+0000 7fa55fe00640 1 -- 172.21.15.86:0/4025100249 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa560160e50 con 0x7fa5600ad620 2024-04-20T21:29:28.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.367+0000 7fa55fe00640 1 --2- 172.21.15.86:0/4025100249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5600ad620 0x7fa56006e290 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fa558009e50 tx=0x7fa55802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fea6f63b0c6ce7c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:28.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.367+0000 7fa55e000640 1 -- 172.21.15.86:0/4025100249 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55802cb00 con 0x7fa5600ad620 2024-04-20T21:29:28.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.367+0000 7fa55e000640 1 -- 172.21.15.86:0/4025100249 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa55802cca0 con 0x7fa5600ad620 2024-04-20T21:29:28.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.367+0000 7fa55e000640 1 -- 172.21.15.86:0/4025100249 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa558034610 con 0x7fa5600ad620 2024-04-20T21:29:28.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.368+0000 7fa565200640 1 -- 172.21.15.86:0/4025100249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5600ad620 msgr2=0x7fa56006e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:28.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.368+0000 7fa565200640 1 --2- 172.21.15.86:0/4025100249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5600ad620 0x7fa56006e290 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fa558009e50 tx=0x7fa55802bf60 comp rx=0 tx=0).stop 2024-04-20T21:29:28.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.368+0000 7fa565200640 1 -- 172.21.15.86:0/4025100249 shutdown_connections 2024-04-20T21:29:28.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.368+0000 7fa565200640 1 --2- 172.21.15.86:0/4025100249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5600ad620 0x7fa56006e290 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:28.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.368+0000 7fa565200640 1 --2- 172.21.15.86:0/4025100249 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa56015d6a0 0x7fa5600ad0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:28.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.368+0000 7fa565200640 1 -- 172.21.15.86:0/4025100249 >> 172.21.15.86:0/4025100249 conn(0x7fa5600d0170 msgr2=0x7fa5600d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:28.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.368+0000 7fa565200640 1 -- 172.21.15.86:0/4025100249 shutdown_connections 2024-04-20T21:29:28.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.368+0000 7fa565200640 1 -- 172.21.15.86:0/4025100249 wait complete. 2024-04-20T21:29:28.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.370+0000 7fa565200640 1 Processor -- start 2024-04-20T21:29:28.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.370+0000 7fa565200640 1 -- start start 2024-04-20T21:29:28.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.371+0000 7fa565200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5600ad620 0x7fa5601f4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:28.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.371+0000 7fa565200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa560156ae0 0x7fa5601587b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:28.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.371+0000 7fa565200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa56015d6a0 0x7fa5600ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:28.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.371+0000 7fa565200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa56006e7d0 con 0x7fa560156ae0 2024-04-20T21:29:28.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.371+0000 7fa565200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa560156ec0 con 0x7fa5600ad620 2024-04-20T21:29:28.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.371+0000 7fa565200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa56006e950 con 0x7fa56015d6a0 2024-04-20T21:29:28.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.371+0000 7fa55ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa560156ae0 0x7fa5601587b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:28.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.371+0000 7fa55ea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa560156ae0 0x7fa5601587b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43708/0 (socket says 172.21.15.86:43708) 2024-04-20T21:29:28.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.371+0000 7fa55ea00640 1 -- 172.21.15.86:0/3660657643 learned_addr learned my addr 172.21.15.86:0/3660657643 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:28.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.371+0000 7fa55f400640 1 --2- 172.21.15.86:0/3660657643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5600ad620 0x7fa5601f4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:28.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.371+0000 7fa55ea00640 1 -- 172.21.15.86:0/3660657643 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa56015d6a0 msgr2=0x7fa5600ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:28.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.371+0000 7fa55fe00640 1 --2- 172.21.15.86:0/3660657643 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa56015d6a0 0x7fa5600ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:28.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.371+0000 7fa55ea00640 1 --2- 172.21.15.86:0/3660657643 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa56015d6a0 0x7fa5600ce6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:28.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.371+0000 7fa55ea00640 1 -- 172.21.15.86:0/3660657643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5600ad620 msgr2=0x7fa5601f4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:28.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.371+0000 7fa55ea00640 1 --2- 172.21.15.86:0/3660657643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5600ad620 0x7fa5601f4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:28.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.372+0000 7fa55ea00640 1 -- 172.21.15.86:0/3660657643 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa560159290 con 0x7fa560156ae0 2024-04-20T21:29:28.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.372+0000 7fa55f400640 1 --2- 172.21.15.86:0/3660657643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5600ad620 0x7fa5601f4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:29:28.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.372+0000 7fa55ea00640 1 --2- 172.21.15.86:0/3660657643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa560156ae0 0x7fa5601587b0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fa54c00c7f0 tx=0x7fa54c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:28.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.372+0000 7fa557400640 1 -- 172.21.15.86:0/3660657643 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54c015070 con 0x7fa560156ae0 2024-04-20T21:29:28.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.372+0000 7fa557400640 1 -- 172.21.15.86:0/3660657643 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa54c004220 con 0x7fa560156ae0 2024-04-20T21:29:28.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.372+0000 7fa557400640 1 -- 172.21.15.86:0/3660657643 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54c007620 con 0x7fa560156ae0 2024-04-20T21:29:28.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.372+0000 7fa565200640 1 -- 172.21.15.86:0/3660657643 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa560159570 con 0x7fa560156ae0 2024-04-20T21:29:28.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.372+0000 7fa565200640 1 -- 172.21.15.86:0/3660657643 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa560159b30 con 0x7fa560156ae0 2024-04-20T21:29:28.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.373+0000 7fa565200640 1 -- 172.21.15.86:0/3660657643 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa56004fcf0 con 0x7fa560156ae0 2024-04-20T21:29:28.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.374+0000 7fa557400640 1 -- 172.21.15.86:0/3660657643 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa54c0079d0 con 0x7fa560156ae0 2024-04-20T21:29:28.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.376+0000 7fa557400640 1 --2- 172.21.15.86:0/3660657643 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa54006c8f0 0x7fa54006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:28.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.376+0000 7fa557400640 1 -- 172.21.15.86:0/3660657643 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5881+0+0 (secure 0 0 0) 0x7fa54c08bc70 con 0x7fa560156ae0 2024-04-20T21:29:28.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.376+0000 7fa55f400640 1 --2- 172.21.15.86:0/3660657643 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa54006c8f0 0x7fa54006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:28.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.377+0000 7fa55f400640 1 --2- 172.21.15.86:0/3660657643 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa54006c8f0 0x7fa54006edb0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fa550001680 tx=0x7fa550009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:28.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.380+0000 7fa557400640 1 -- 172.21.15.86:0/3660657643 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa54c059fe0 con 0x7fa560156ae0 2024-04-20T21:29:28.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.623+0000 7fa565200640 1 -- 172.21.15.86:0/3660657643 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa5600b1300 con 0x7fa560156ae0 2024-04-20T21:29:28.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.624+0000 7fa557400640 1 -- 172.21.15.86:0/3660657643 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+5194 (secure 0 0 0) 0x7fa54c05ee20 con 0x7fa560156ae0 2024-04-20T21:29:28.623 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:29:28.623 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":90,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5525,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3902778887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3902778887},{"type":"v1","addr":"172.21.15.144:6835","nonce":3902778887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6001,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/4270988014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":4270988014},{"type":"v1","addr":"172.21.15.86:6837","nonce":4270988014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6016,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/1569462554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":1569462554},{"type":"v1","addr":"172.21.15.144:6833","nonce":1569462554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":9066,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/4050537846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4050537846},{"type":"v1","addr":"172.21.15.86:6836","nonce":4050537846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":89,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:27:45.692408+0000","modified":"2024-04-20T21:29:25.141526+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":102,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-04-20T21:29:28.623 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 90 2024-04-20T21:29:28.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.627+0000 7fa565200640 1 -- 172.21.15.86:0/3660657643 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa54006c8f0 msgr2=0x7fa54006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:28.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.627+0000 7fa565200640 1 --2- 172.21.15.86:0/3660657643 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa54006c8f0 0x7fa54006edb0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fa550001680 tx=0x7fa550009040 comp rx=0 tx=0).stop 2024-04-20T21:29:28.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.627+0000 7fa565200640 1 -- 172.21.15.86:0/3660657643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa560156ae0 msgr2=0x7fa5601587b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:28.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.627+0000 7fa565200640 1 --2- 172.21.15.86:0/3660657643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa560156ae0 0x7fa5601587b0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fa54c00c7f0 tx=0x7fa54c00ccb0 comp rx=0 tx=0).stop 2024-04-20T21:29:28.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.627+0000 7fa565200640 1 -- 172.21.15.86:0/3660657643 shutdown_connections 2024-04-20T21:29:28.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.627+0000 7fa565200640 1 --2- 172.21.15.86:0/3660657643 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa54006c8f0 0x7fa54006edb0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:28.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.627+0000 7fa565200640 1 --2- 172.21.15.86:0/3660657643 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa56015d6a0 0x7fa5600ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:28.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.627+0000 7fa565200640 1 --2- 172.21.15.86:0/3660657643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa560156ae0 0x7fa5601587b0 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:28.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.627+0000 7fa565200640 1 --2- 172.21.15.86:0/3660657643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5600ad620 0x7fa5601f4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:28.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.627+0000 7fa565200640 1 -- 172.21.15.86:0/3660657643 >> 172.21.15.86:0/3660657643 conn(0x7fa5600d0170 msgr2=0x7fa5600afd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:28.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.627+0000 7fa565200640 1 -- 172.21.15.86:0/3660657643 shutdown_connections 2024-04-20T21:29:28.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.627+0000 7fa565200640 1 -- 172.21.15.86:0/3660657643 wait complete. 2024-04-20T21:29:28.645 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:29:28.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.843+0000 7f6736600640 1 Processor -- start 2024-04-20T21:29:28.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.843+0000 7f6736600640 1 -- start start 2024-04-20T21:29:28.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.845+0000 7f6736600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6730144e90 0x7f673015ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:28.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.845+0000 7f6736600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f673006f140 0x7f67300aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:28.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.845+0000 7f6736600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67300af0b0 0x7f6730071d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:28.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.845+0000 7f6736600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f6730145270 con 0x7f673006f140 2024-04-20T21:29:28.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.845+0000 7f6736600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f673006dc30 con 0x7f6730144e90 2024-04-20T21:29:28.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.845+0000 7f6736600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f673015f720 con 0x7f67300af0b0 2024-04-20T21:29:28.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.845+0000 7f6727e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f673006f140 0x7f67300aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:28.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.845+0000 7f6727e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f673006f140 0x7f67300aeb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43718/0 (socket says 172.21.15.86:43718) 2024-04-20T21:29:28.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.845+0000 7f6727e00640 1 -- 172.21.15.86:0/4181710774 learned_addr learned my addr 172.21.15.86:0/4181710774 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:28.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.846+0000 7f672f400640 1 --2- 172.21.15.86:0/4181710774 >> v2:172.21.15.144:3300/0 conn(0x7f6730144e90 0x7f673015ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:28.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.846+0000 7f672fe00640 1 --2- 172.21.15.86:0/4181710774 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67300af0b0 0x7f6730071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:28.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.846+0000 7f6727e00640 1 -- 172.21.15.86:0/4181710774 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67300af0b0 msgr2=0x7f6730071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:28.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.846+0000 7f6727e00640 1 --2- 172.21.15.86:0/4181710774 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67300af0b0 0x7f6730071d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:28.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.846+0000 7f6727e00640 1 -- 172.21.15.86:0/4181710774 >> v2:172.21.15.144:3300/0 conn(0x7f6730144e90 msgr2=0x7f673015ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:28.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.846+0000 7f6727e00640 1 --2- 172.21.15.86:0/4181710774 >> v2:172.21.15.144:3300/0 conn(0x7f6730144e90 0x7f673015ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:28.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.846+0000 7f6727e00640 1 -- 172.21.15.86:0/4181710774 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f673015fa00 con 0x7f673006f140 2024-04-20T21:29:28.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.847+0000 7f6727e00640 1 --2- 172.21.15.86:0/4181710774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f673006f140 0x7f67300aeb70 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f673006dbb0 tx=0x7f671802b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b436e07e2ce511e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:28.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.847+0000 7f672ea00640 1 -- 172.21.15.86:0/4181710774 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6718035020 con 0x7f673006f140 2024-04-20T21:29:28.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.847+0000 7f672ea00640 1 -- 172.21.15.86:0/4181710774 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6718031880 con 0x7f673006f140 2024-04-20T21:29:28.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.847+0000 7f672ea00640 1 -- 172.21.15.86:0/4181710774 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6718031c50 con 0x7f673006f140 2024-04-20T21:29:28.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.848+0000 7f6736600640 1 -- 172.21.15.86:0/4181710774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f673006f140 msgr2=0x7f67300aeb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:28.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.848+0000 7f6736600640 1 --2- 172.21.15.86:0/4181710774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f673006f140 0x7f67300aeb70 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f673006dbb0 tx=0x7f671802b920 comp rx=0 tx=0).stop 2024-04-20T21:29:28.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.848+0000 7f6736600640 1 -- 172.21.15.86:0/4181710774 shutdown_connections 2024-04-20T21:29:28.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.848+0000 7f6736600640 1 --2- 172.21.15.86:0/4181710774 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f67300af0b0 0x7f6730071d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:28.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.848+0000 7f6736600640 1 --2- 172.21.15.86:0/4181710774 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f673006f140 0x7f67300aeb70 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:28.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.848+0000 7f6736600640 1 --2- 172.21.15.86:0/4181710774 >> v2:172.21.15.144:3300/0 conn(0x7f6730144e90 0x7f673015ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:28.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.848+0000 7f6736600640 1 -- 172.21.15.86:0/4181710774 >> 172.21.15.86:0/4181710774 conn(0x7f67300b2360 msgr2=0x7f67300b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:28.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.848+0000 7f6736600640 1 -- 172.21.15.86:0/4181710774 shutdown_connections 2024-04-20T21:29:28.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.848+0000 7f6736600640 1 -- 172.21.15.86:0/4181710774 wait complete. 2024-04-20T21:29:28.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.850+0000 7f6736600640 1 Processor -- start 2024-04-20T21:29:28.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.850+0000 7f6736600640 1 -- start start 2024-04-20T21:29:28.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.851+0000 7f6736600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f673006f140 0x7f67300cefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:28.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.851+0000 7f6736600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67300af0b0 0x7f67300cf970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:28.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.851+0000 7f6736600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6730144e90 0x7f673015dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:28.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.851+0000 7f6736600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f673015f350 con 0x7f6730144e90 2024-04-20T21:29:28.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.851+0000 7f6736600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f673006ded0 con 0x7f67300af0b0 2024-04-20T21:29:28.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.851+0000 7f6736600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f673015f4d0 con 0x7f673006f140 2024-04-20T21:29:28.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.851+0000 7f672fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6730144e90 0x7f673015dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:28.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.852+0000 7f672fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6730144e90 0x7f673015dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43734/0 (socket says 172.21.15.86:43734) 2024-04-20T21:29:28.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.852+0000 7f672fe00640 1 -- 172.21.15.86:0/3489718099 learned_addr learned my addr 172.21.15.86:0/3489718099 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:28.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.852+0000 7f672f400640 1 --2- 172.21.15.86:0/3489718099 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f673006f140 0x7f67300cefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:28.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.852+0000 7f6727e00640 1 --2- 172.21.15.86:0/3489718099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67300af0b0 0x7f67300cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:28.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.852+0000 7f672fe00640 1 -- 172.21.15.86:0/3489718099 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f673006f140 msgr2=0x7f67300cefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:28.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.852+0000 7f672fe00640 1 --2- 172.21.15.86:0/3489718099 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f673006f140 0x7f67300cefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:28.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.852+0000 7f672fe00640 1 -- 172.21.15.86:0/3489718099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67300af0b0 msgr2=0x7f67300cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:28.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.852+0000 7f672fe00640 1 --2- 172.21.15.86:0/3489718099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67300af0b0 0x7f67300cf970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:28.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.852+0000 7f672fe00640 1 -- 172.21.15.86:0/3489718099 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6730161260 con 0x7f6730144e90 2024-04-20T21:29:28.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.852+0000 7f672fe00640 1 --2- 172.21.15.86:0/3489718099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6730144e90 0x7f673015dc40 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f6728007e60 tx=0x7f672800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:28.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.853+0000 7f6727400640 1 -- 172.21.15.86:0/3489718099 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6728014420 con 0x7f6730144e90 2024-04-20T21:29:28.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.853+0000 7f6727400640 1 -- 172.21.15.86:0/3489718099 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6728019070 con 0x7f6730144e90 2024-04-20T21:29:28.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.853+0000 7f6727400640 1 -- 172.21.15.86:0/3489718099 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6728014820 con 0x7f6730144e90 2024-04-20T21:29:28.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.853+0000 7f6736600640 1 -- 172.21.15.86:0/3489718099 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6730161540 con 0x7f6730144e90 2024-04-20T21:29:28.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.853+0000 7f6736600640 1 -- 172.21.15.86:0/3489718099 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6730146490 con 0x7f6730144e90 2024-04-20T21:29:28.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.854+0000 7f6736600640 1 -- 172.21.15.86:0/3489718099 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6730161710 con 0x7f6730144e90 2024-04-20T21:29:28.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.855+0000 7f6727400640 1 -- 172.21.15.86:0/3489718099 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f67280149c0 con 0x7f6730144e90 2024-04-20T21:29:28.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.856+0000 7f6727400640 1 --2- 172.21.15.86:0/3489718099 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f670c06c8f0 0x7f670c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:28.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.857+0000 7f6727400640 1 -- 172.21.15.86:0/3489718099 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5881+0+0 (secure 0 0 0) 0x7f672808c500 con 0x7f6730144e90 2024-04-20T21:29:28.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.857+0000 7f672f400640 1 --2- 172.21.15.86:0/3489718099 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f670c06c8f0 0x7f670c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:28.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.857+0000 7f672f400640 1 --2- 172.21.15.86:0/3489718099 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f670c06c8f0 0x7f670c06edb0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f6720006fd0 tx=0x7f6720008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:28.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:28.861+0000 7f6727400640 1 -- 172.21.15.86:0/3489718099 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f672805a7e0 con 0x7f6730144e90 2024-04-20T21:29:29.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.058+0000 7f6736600640 1 -- 172.21.15.86:0/3489718099 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f67300689e0 con 0x7f6730144e90 2024-04-20T21:29:29.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.061+0000 7f6727400640 1 -- 172.21.15.86:0/3489718099 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+16869 (secure 0 0 0) 0x7f672805a0e0 con 0x7f6730144e90 2024-04-20T21:29:29.061 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:29:29.061 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":102,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:29:25.141262+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:27:43.469685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-20T21:27:44.673505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6836/4271041811":"2024-04-21T21:29:25.141228+0000","172.21.15.86:6834/4271041811":"2024-04-21T21:29:25.141228+0000","172.21.15.86:6836/1331851138":"2024-04-21T21:28:47.841886+0000","172.21.15.86:6834/1331851138":"2024-04-21T21:28:47.841886+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:29:29.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.064+0000 7f6736600640 1 -- 172.21.15.86:0/3489718099 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f670c06c8f0 msgr2=0x7f670c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:29.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.064+0000 7f6736600640 1 --2- 172.21.15.86:0/3489718099 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f670c06c8f0 0x7f670c06edb0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f6720006fd0 tx=0x7f6720008040 comp rx=0 tx=0).stop 2024-04-20T21:29:29.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.064+0000 7f6736600640 1 -- 172.21.15.86:0/3489718099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6730144e90 msgr2=0x7f673015dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:29.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.064+0000 7f6736600640 1 --2- 172.21.15.86:0/3489718099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6730144e90 0x7f673015dc40 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f6728007e60 tx=0x7f672800e4b0 comp rx=0 tx=0).stop 2024-04-20T21:29:29.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.064+0000 7f6736600640 1 -- 172.21.15.86:0/3489718099 shutdown_connections 2024-04-20T21:29:29.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.064+0000 7f6736600640 1 --2- 172.21.15.86:0/3489718099 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f670c06c8f0 0x7f670c06edb0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:29.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.064+0000 7f6736600640 1 --2- 172.21.15.86:0/3489718099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6730144e90 0x7f673015dc40 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:29.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.064+0000 7f6736600640 1 --2- 172.21.15.86:0/3489718099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67300af0b0 0x7f67300cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:29.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.064+0000 7f6736600640 1 --2- 172.21.15.86:0/3489718099 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f673006f140 0x7f67300cefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:29.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.064+0000 7f6736600640 1 -- 172.21.15.86:0/3489718099 >> 172.21.15.86:0/3489718099 conn(0x7f67300b2360 msgr2=0x7f67300b0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:29.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.065+0000 7f6736600640 1 -- 172.21.15.86:0/3489718099 shutdown_connections 2024-04-20T21:29:29.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.065+0000 7f6736600640 1 -- 172.21.15.86:0/3489718099 wait complete. 2024-04-20T21:29:29.081 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-20T21:29:29.082 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:29:29.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.277+0000 7fcbe9400640 1 Processor -- start 2024-04-20T21:29:29.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.277+0000 7fcbe9400640 1 -- start start 2024-04-20T21:29:29.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.279+0000 7fcbe9400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcbe415e750 0x7fcbe40b4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:29.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.279+0000 7fcbe9400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbe40b3c50 0x7fcbe406fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:29.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.279+0000 7fcbe9400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fcbe4070170 0x7fcbe406c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:29.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.279+0000 7fcbe9400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fcbe406f7a0 con 0x7fcbe415e750 2024-04-20T21:29:29.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.279+0000 7fcbe9400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fcbe40b4d40 con 0x7fcbe4070170 2024-04-20T21:29:29.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.279+0000 7fcbe9400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fcbe40b40b0 con 0x7fcbe40b3c50 2024-04-20T21:29:29.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.279+0000 7fcbe2800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcbe415e750 0x7fcbe40b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:29.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.280+0000 7fcbe2800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcbe415e750 0x7fcbe40b4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43742/0 (socket says 172.21.15.86:43742) 2024-04-20T21:29:29.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.280+0000 7fcbe2800640 1 -- 172.21.15.86:0/889404737 learned_addr learned my addr 172.21.15.86:0/889404737 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:29.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.280+0000 7fcbdbe00640 1 --2- 172.21.15.86:0/889404737 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbe40b3c50 0x7fcbe406fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:29.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.280+0000 7fcbe3200640 1 --2- 172.21.15.86:0/889404737 >> v2:172.21.15.144:3300/0 conn(0x7fcbe4070170 0x7fcbe406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:29.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.280+0000 7fcbe2800640 1 -- 172.21.15.86:0/889404737 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbe40b3c50 msgr2=0x7fcbe406fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:29.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.280+0000 7fcbe2800640 1 --2- 172.21.15.86:0/889404737 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbe40b3c50 0x7fcbe406fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:29.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.280+0000 7fcbe2800640 1 -- 172.21.15.86:0/889404737 >> v2:172.21.15.144:3300/0 conn(0x7fcbe4070170 msgr2=0x7fcbe406c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:29.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.280+0000 7fcbe2800640 1 --2- 172.21.15.86:0/889404737 >> v2:172.21.15.144:3300/0 conn(0x7fcbe4070170 0x7fcbe406c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:29.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.280+0000 7fcbe2800640 1 -- 172.21.15.86:0/889404737 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbe406ee70 con 0x7fcbe415e750 2024-04-20T21:29:29.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.281+0000 7fcbe2800640 1 --2- 172.21.15.86:0/889404737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbe415e750 0x7fcbe40b4640 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fcbe40739c0 tx=0x7fcbd402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c16abd2387d930 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:29.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.282+0000 7fcbe1e00640 1 -- 172.21.15.86:0/889404737 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd4035020 con 0x7fcbe415e750 2024-04-20T21:29:29.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.282+0000 7fcbe1e00640 1 -- 172.21.15.86:0/889404737 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbd4031870 con 0x7fcbe415e750 2024-04-20T21:29:29.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.282+0000 7fcbe1e00640 1 -- 172.21.15.86:0/889404737 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd4031c40 con 0x7fcbe415e750 2024-04-20T21:29:29.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.282+0000 7fcbe9400640 1 -- 172.21.15.86:0/889404737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbe415e750 msgr2=0x7fcbe40b4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:29.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.282+0000 7fcbe9400640 1 --2- 172.21.15.86:0/889404737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbe415e750 0x7fcbe40b4640 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fcbe40739c0 tx=0x7fcbd402b910 comp rx=0 tx=0).stop 2024-04-20T21:29:29.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.282+0000 7fcbe9400640 1 -- 172.21.15.86:0/889404737 shutdown_connections 2024-04-20T21:29:29.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.282+0000 7fcbe9400640 1 --2- 172.21.15.86:0/889404737 >> v2:172.21.15.144:3300/0 conn(0x7fcbe4070170 0x7fcbe406c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:29.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.282+0000 7fcbe9400640 1 --2- 172.21.15.86:0/889404737 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbe40b3c50 0x7fcbe406fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:29.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.282+0000 7fcbe9400640 1 --2- 172.21.15.86:0/889404737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbe415e750 0x7fcbe40b4640 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:29.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.282+0000 7fcbe9400640 1 -- 172.21.15.86:0/889404737 >> 172.21.15.86:0/889404737 conn(0x7fcbe4072540 msgr2=0x7fcbe4072950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:29.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.282+0000 7fcbe9400640 1 -- 172.21.15.86:0/889404737 shutdown_connections 2024-04-20T21:29:29.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.283+0000 7fcbe9400640 1 -- 172.21.15.86:0/889404737 wait complete. 2024-04-20T21:29:29.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.284+0000 7fcbe9400640 1 Processor -- start 2024-04-20T21:29:29.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.284+0000 7fcbe9400640 1 -- start start 2024-04-20T21:29:29.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.285+0000 7fcbe9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbe4070170 0x7fcbe41f7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:29.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.285+0000 7fcbe9400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbe40b3c50 0x7fcbe4160e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:29.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.285+0000 7fcbe9400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbe415e750 0x7fcbe41613b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:29.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.285+0000 7fcbe9400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe415eb30 con 0x7fcbe4070170 2024-04-20T21:29:29.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.285+0000 7fcbe9400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe406f430 con 0x7fcbe415e750 2024-04-20T21:29:29.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.285+0000 7fcbe9400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fcbe415ecb0 con 0x7fcbe40b3c50 2024-04-20T21:29:29.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.286+0000 7fcbe2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbe4070170 0x7fcbe41f7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:29.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.286+0000 7fcbe2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbe4070170 0x7fcbe41f7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43752/0 (socket says 172.21.15.86:43752) 2024-04-20T21:29:29.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.286+0000 7fcbe2800640 1 -- 172.21.15.86:0/181778447 learned_addr learned my addr 172.21.15.86:0/181778447 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:29.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.286+0000 7fcbdbe00640 1 --2- 172.21.15.86:0/181778447 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbe40b3c50 0x7fcbe4160e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:29.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.286+0000 7fcbe3200640 1 --2- 172.21.15.86:0/181778447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbe415e750 0x7fcbe41613b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:29.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.286+0000 7fcbe2800640 1 -- 172.21.15.86:0/181778447 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbe40b3c50 msgr2=0x7fcbe4160e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:29.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.286+0000 7fcbe2800640 1 --2- 172.21.15.86:0/181778447 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbe40b3c50 0x7fcbe4160e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:29.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.286+0000 7fcbe2800640 1 -- 172.21.15.86:0/181778447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbe415e750 msgr2=0x7fcbe41613b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:29.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.286+0000 7fcbe2800640 1 --2- 172.21.15.86:0/181778447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbe415e750 0x7fcbe41613b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:29.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.286+0000 7fcbe2800640 1 -- 172.21.15.86:0/181778447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbe41e7da0 con 0x7fcbe4070170 2024-04-20T21:29:29.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.286+0000 7fcbe3200640 1 --2- 172.21.15.86:0/181778447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbe415e750 0x7fcbe41613b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:29:29.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.286+0000 7fcbe2800640 1 --2- 172.21.15.86:0/181778447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbe4070170 0x7fcbe41f7620 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4000c00 tx=0x7fcbd40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:29.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.287+0000 7fcbdaa00640 1 -- 172.21.15.86:0/181778447 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd403ad60 con 0x7fcbe4070170 2024-04-20T21:29:29.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.287+0000 7fcbdaa00640 1 -- 172.21.15.86:0/181778447 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbd40443f0 con 0x7fcbe4070170 2024-04-20T21:29:29.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.287+0000 7fcbdaa00640 1 -- 172.21.15.86:0/181778447 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd40447c0 con 0x7fcbe4070170 2024-04-20T21:29:29.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.287+0000 7fcbe9400640 1 -- 172.21.15.86:0/181778447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbe41e8080 con 0x7fcbe4070170 2024-04-20T21:29:29.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.287+0000 7fcbe9400640 1 -- 172.21.15.86:0/181778447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbe41e8640 con 0x7fcbe4070170 2024-04-20T21:29:29.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.288+0000 7fcbe9400640 1 -- 172.21.15.86:0/181778447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbe40689e0 con 0x7fcbe4070170 2024-04-20T21:29:29.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.289+0000 7fcbdaa00640 1 -- 172.21.15.86:0/181778447 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcbd404b060 con 0x7fcbe4070170 2024-04-20T21:29:29.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.290+0000 7fcbdaa00640 1 --2- 172.21.15.86:0/181778447 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcbc006c940 0x7fcbc006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:29.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.290+0000 7fcbdaa00640 1 -- 172.21.15.86:0/181778447 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5881+0+0 (secure 0 0 0) 0x7fcbd40adbe0 con 0x7fcbe4070170 2024-04-20T21:29:29.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.290+0000 7fcbdbe00640 1 --2- 172.21.15.86:0/181778447 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcbc006c940 0x7fcbc006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:29.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.291+0000 7fcbdbe00640 1 --2- 172.21.15.86:0/181778447 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcbc006c940 0x7fcbc006ee00 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fcbe4161d90 tx=0x7fcbcc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:29.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.295+0000 7fcbdaa00640 1 -- 172.21.15.86:0/181778447 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fcbd403f070 con 0x7fcbe4070170 2024-04-20T21:29:29.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.531+0000 7fcbe9400640 1 -- 172.21.15.86:0/181778447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcbe4072c00 con 0x7fcbe4070170 2024-04-20T21:29:29.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.533+0000 7fcbdaa00640 1 -- 172.21.15.86:0/181778447 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+5194 (secure 0 0 0) 0x7fcbd402f320 con 0x7fcbe4070170 2024-04-20T21:29:29.532 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:29:29.532 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":90,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5525,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3902778887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3902778887},{"type":"v1","addr":"172.21.15.144:6835","nonce":3902778887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6001,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/4270988014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":4270988014},{"type":"v1","addr":"172.21.15.86:6837","nonce":4270988014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6016,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/1569462554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":1569462554},{"type":"v1","addr":"172.21.15.144:6833","nonce":1569462554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":9066,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/4050537846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4050537846},{"type":"v1","addr":"172.21.15.86:6836","nonce":4050537846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":89,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:27:45.692408+0000","modified":"2024-04-20T21:29:25.141526+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":102,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-04-20T21:29:29.532 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 90 2024-04-20T21:29:29.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.535+0000 7fcbe9400640 1 -- 172.21.15.86:0/181778447 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcbc006c940 msgr2=0x7fcbc006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:29.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.535+0000 7fcbe9400640 1 --2- 172.21.15.86:0/181778447 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcbc006c940 0x7fcbc006ee00 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fcbe4161d90 tx=0x7fcbcc008040 comp rx=0 tx=0).stop 2024-04-20T21:29:29.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.535+0000 7fcbe9400640 1 -- 172.21.15.86:0/181778447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbe4070170 msgr2=0x7fcbe41f7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:29.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.535+0000 7fcbe9400640 1 --2- 172.21.15.86:0/181778447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbe4070170 0x7fcbe41f7620 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4000c00 tx=0x7fcbd40046c0 comp rx=0 tx=0).stop 2024-04-20T21:29:29.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.535+0000 7fcbe9400640 1 -- 172.21.15.86:0/181778447 shutdown_connections 2024-04-20T21:29:29.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.535+0000 7fcbe9400640 1 --2- 172.21.15.86:0/181778447 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcbc006c940 0x7fcbc006ee00 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:29.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.535+0000 7fcbe9400640 1 --2- 172.21.15.86:0/181778447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcbe415e750 0x7fcbe41613b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:29.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.535+0000 7fcbe9400640 1 --2- 172.21.15.86:0/181778447 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcbe40b3c50 0x7fcbe4160e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:29.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.535+0000 7fcbe9400640 1 --2- 172.21.15.86:0/181778447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcbe4070170 0x7fcbe41f7620 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:29.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.535+0000 7fcbe9400640 1 -- 172.21.15.86:0/181778447 >> 172.21.15.86:0/181778447 conn(0x7fcbe4072540 msgr2=0x7fcbe40af2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:29.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.536+0000 7fcbe9400640 1 -- 172.21.15.86:0/181778447 shutdown_connections 2024-04-20T21:29:29.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.536+0000 7fcbe9400640 1 -- 172.21.15.86:0/181778447 wait complete. 2024-04-20T21:29:29.552 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:29:29.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.747+0000 7fdaf8e00640 1 Processor -- start 2024-04-20T21:29:29.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.747+0000 7fdaf8e00640 1 -- start start 2024-04-20T21:29:29.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.748+0000 7fdaf8e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdaf415ac10 0x7fdaf40b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:29.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.748+0000 7fdaf8e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdaf4072ca0 0x7fdaf40cea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:29.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.748+0000 7fdaf8e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdaf406dcd0 con 0x7fdaf415f210 2024-04-20T21:29:29.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.748+0000 7fdaf8e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fdaf406e8e0 con 0x7fdaf4072ca0 2024-04-20T21:29:29.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.748+0000 7fdaf8e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fdaf406dfb0 con 0x7fdaf415ac10 2024-04-20T21:29:29.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.749+0000 7fdaf2800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdaf4072ca0 0x7fdaf40cea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:29.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.749+0000 7fdaf2800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdaf4072ca0 0x7fdaf40cea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43756/0 (socket says 172.21.15.86:43756) 2024-04-20T21:29:29.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.749+0000 7fdaf2800640 1 -- 172.21.15.86:0/973168452 learned_addr learned my addr 172.21.15.86:0/973168452 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:29.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.749+0000 7fdaf1e00640 1 --2- 172.21.15.86:0/973168452 >> v2:172.21.15.144:3300/0 conn(0x7fdaf415ac10 0x7fdaf40b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:29.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.749+0000 7fdaf2800640 1 -- 172.21.15.86:0/973168452 >> v2:172.21.15.144:3300/0 conn(0x7fdaf415ac10 msgr2=0x7fdaf40b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:29.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.749+0000 7fdaf2800640 1 --2- 172.21.15.86:0/973168452 >> v2:172.21.15.144:3300/0 conn(0x7fdaf415ac10 0x7fdaf40b53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:29.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.749+0000 7fdaf2800640 1 -- 172.21.15.86:0/973168452 >> v1:172.21.15.144:6789/0 conn(0x7fdaf415f210 legacy=0x7fdaf4072590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:29.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.749+0000 7fdaf2800640 1 -- 172.21.15.86:0/973168452 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf406e290 con 0x7fdaf4072ca0 2024-04-20T21:29:29.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.749+0000 7fdaf1e00640 1 --2- 172.21.15.86:0/973168452 >> v2:172.21.15.144:3300/0 conn(0x7fdaf415ac10 0x7fdaf40b53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:29:29.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.750+0000 7fdaf2800640 1 --2- 172.21.15.86:0/973168452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf4072ca0 0x7fdaf40cea10 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fdaf40700d0 tx=0x7fdaec02cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=191f802461ae1379 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:29.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.751+0000 7fdaf1400640 1 -- 172.21.15.86:0/973168452 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec02d900 con 0x7fdaf4072ca0 2024-04-20T21:29:29.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.751+0000 7fdaf1400640 1 -- 172.21.15.86:0/973168452 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaec02daa0 con 0x7fdaf4072ca0 2024-04-20T21:29:29.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.751+0000 7fdaf1400640 1 -- 172.21.15.86:0/973168452 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec008500 con 0x7fdaf4072ca0 2024-04-20T21:29:29.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.751+0000 7fdaf8e00640 1 -- 172.21.15.86:0/973168452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf4072ca0 msgr2=0x7fdaf40cea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:29.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.751+0000 7fdaf8e00640 1 --2- 172.21.15.86:0/973168452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf4072ca0 0x7fdaf40cea10 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fdaf40700d0 tx=0x7fdaec02cf60 comp rx=0 tx=0).stop 2024-04-20T21:29:29.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.751+0000 7fdaf8e00640 1 -- 172.21.15.86:0/973168452 shutdown_connections 2024-04-20T21:29:29.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.751+0000 7fdaf8e00640 1 --2- 172.21.15.86:0/973168452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf4072ca0 0x7fdaf40cea10 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:29.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.751+0000 7fdaf8e00640 1 --2- 172.21.15.86:0/973168452 >> v2:172.21.15.144:3300/0 conn(0x7fdaf415ac10 0x7fdaf40b53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:29.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.751+0000 7fdaf8e00640 1 -- 172.21.15.86:0/973168452 >> 172.21.15.86:0/973168452 conn(0x7fdaf40b2360 msgr2=0x7fdaf40b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:29.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.752+0000 7fdaf8e00640 1 -- 172.21.15.86:0/973168452 shutdown_connections 2024-04-20T21:29:29.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.752+0000 7fdaf8e00640 1 -- 172.21.15.86:0/973168452 wait complete. 2024-04-20T21:29:29.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.754+0000 7fdaf8e00640 1 Processor -- start 2024-04-20T21:29:29.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.754+0000 7fdaf8e00640 1 -- start start 2024-04-20T21:29:29.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.755+0000 7fdaf8e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdaf4072ca0 0x7fdaf41eff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:29.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.755+0000 7fdaf8e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdaf415ac10 0x7fdaf41f2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:29.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.755+0000 7fdaf8e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf415f210 0x7fdaf414f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:29.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.755+0000 7fdaf8e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf415f650 con 0x7fdaf415f210 2024-04-20T21:29:29.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.755+0000 7fdaf8e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf415aff0 con 0x7fdaf415ac10 2024-04-20T21:29:29.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.755+0000 7fdaf8e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdaf415f7d0 con 0x7fdaf4072ca0 2024-04-20T21:29:29.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.755+0000 7fdaf2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf415f210 0x7fdaf414f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:29.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.755+0000 7fdaf2800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf415f210 0x7fdaf414f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43758/0 (socket says 172.21.15.86:43758) 2024-04-20T21:29:29.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.755+0000 7fdaf2800640 1 -- 172.21.15.86:0/359592934 learned_addr learned my addr 172.21.15.86:0/359592934 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:29.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.756+0000 7fdaf1e00640 1 --2- 172.21.15.86:0/359592934 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdaf4072ca0 0x7fdaf41eff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:29.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.756+0000 7fdae9400640 1 --2- 172.21.15.86:0/359592934 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdaf415ac10 0x7fdaf41f2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:29.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.756+0000 7fdaf2800640 1 -- 172.21.15.86:0/359592934 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdaf4072ca0 msgr2=0x7fdaf41eff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:29.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.756+0000 7fdaf2800640 1 --2- 172.21.15.86:0/359592934 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdaf4072ca0 0x7fdaf41eff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:29.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.756+0000 7fdaf2800640 1 -- 172.21.15.86:0/359592934 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdaf415ac10 msgr2=0x7fdaf41f2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:29.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.756+0000 7fdaf2800640 1 --2- 172.21.15.86:0/359592934 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdaf415ac10 0x7fdaf41f2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:29.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.756+0000 7fdaf2800640 1 -- 172.21.15.86:0/359592934 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf414b2d0 con 0x7fdaf415f210 2024-04-20T21:29:29.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.756+0000 7fdaf2800640 1 --2- 172.21.15.86:0/359592934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf415f210 0x7fdaf414f9a0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fdaec00c140 tx=0x7fdaec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:29.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.756+0000 7fdaeaa00640 1 -- 172.21.15.86:0/359592934 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec044020 con 0x7fdaf415f210 2024-04-20T21:29:29.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.757+0000 7fdaeaa00640 1 -- 172.21.15.86:0/359592934 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaec009c50 con 0x7fdaf415f210 2024-04-20T21:29:29.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.757+0000 7fdaeaa00640 1 -- 172.21.15.86:0/359592934 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec040470 con 0x7fdaf415f210 2024-04-20T21:29:29.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.757+0000 7fdaf8e00640 1 -- 172.21.15.86:0/359592934 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaf414cbf0 con 0x7fdaf415f210 2024-04-20T21:29:29.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.757+0000 7fdaf8e00640 1 -- 172.21.15.86:0/359592934 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaf414d1b0 con 0x7fdaf415f210 2024-04-20T21:29:29.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.758+0000 7fdaf8e00640 1 -- 172.21.15.86:0/359592934 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaf40689e0 con 0x7fdaf415f210 2024-04-20T21:29:29.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.759+0000 7fdaeaa00640 1 -- 172.21.15.86:0/359592934 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdaec047020 con 0x7fdaf415f210 2024-04-20T21:29:29.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.759+0000 7fdaeaa00640 1 --2- 172.21.15.86:0/359592934 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdad006c8f0 0x7fdad006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:29.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.760+0000 7fdaeaa00640 1 -- 172.21.15.86:0/359592934 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5881+0+0 (secure 0 0 0) 0x7fdaec0aea80 con 0x7fdaf415f210 2024-04-20T21:29:29.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.760+0000 7fdaf1e00640 1 --2- 172.21.15.86:0/359592934 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdad006c8f0 0x7fdad006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:29.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.760+0000 7fdaf1e00640 1 --2- 172.21.15.86:0/359592934 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdad006c8f0 0x7fdad006edb0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fdae400a990 tx=0x7fdae4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:29.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.764+0000 7fdaeaa00640 1 -- 172.21.15.86:0/359592934 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fdaec0b0e20 con 0x7fdaf415f210 2024-04-20T21:29:29.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.988+0000 7fdaf8e00640 1 -- 172.21.15.86:0/359592934 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdaf406f450 con 0x7fdaf415f210 2024-04-20T21:29:29.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.989+0000 7fdaeaa00640 1 -- 172.21.15.86:0/359592934 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+5194 (secure 0 0 0) 0x7fdaec04e030 con 0x7fdaf415f210 2024-04-20T21:29:29.988 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:29:29.988 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":90,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5525,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3902778887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3902778887},{"type":"v1","addr":"172.21.15.144:6835","nonce":3902778887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6001,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/4270988014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":4270988014},{"type":"v1","addr":"172.21.15.86:6837","nonce":4270988014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6016,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/1569462554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":1569462554},{"type":"v1","addr":"172.21.15.144:6833","nonce":1569462554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":9066,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/4050537846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4050537846},{"type":"v1","addr":"172.21.15.86:6836","nonce":4050537846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":89,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:27:45.692408+0000","modified":"2024-04-20T21:29:25.141526+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":102,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-04-20T21:29:29.988 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 90 2024-04-20T21:29:29.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.991+0000 7fdaf8e00640 1 -- 172.21.15.86:0/359592934 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdad006c8f0 msgr2=0x7fdad006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:29.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.991+0000 7fdaf8e00640 1 --2- 172.21.15.86:0/359592934 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdad006c8f0 0x7fdad006edb0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fdae400a990 tx=0x7fdae4008040 comp rx=0 tx=0).stop 2024-04-20T21:29:29.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.991+0000 7fdaf8e00640 1 -- 172.21.15.86:0/359592934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf415f210 msgr2=0x7fdaf414f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:29.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.991+0000 7fdaf8e00640 1 --2- 172.21.15.86:0/359592934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf415f210 0x7fdaf414f9a0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fdaec00c140 tx=0x7fdaec004a40 comp rx=0 tx=0).stop 2024-04-20T21:29:29.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.991+0000 7fdaf8e00640 1 -- 172.21.15.86:0/359592934 shutdown_connections 2024-04-20T21:29:29.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.991+0000 7fdaf8e00640 1 --2- 172.21.15.86:0/359592934 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdad006c8f0 0x7fdad006edb0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:29.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.991+0000 7fdaf8e00640 1 --2- 172.21.15.86:0/359592934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdaf415f210 0x7fdaf414f9a0 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:29.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.991+0000 7fdaf8e00640 1 --2- 172.21.15.86:0/359592934 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdaf415ac10 0x7fdaf41f2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:29.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.991+0000 7fdaf8e00640 1 --2- 172.21.15.86:0/359592934 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdaf4072ca0 0x7fdaf41eff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:29.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.992+0000 7fdaf8e00640 1 -- 172.21.15.86:0/359592934 >> 172.21.15.86:0/359592934 conn(0x7fdaf40b2360 msgr2=0x7fdaf40b3b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:29.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.992+0000 7fdaf8e00640 1 -- 172.21.15.86:0/359592934 shutdown_connections 2024-04-20T21:29:29.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:29.992+0000 7fdaf8e00640 1 -- 172.21.15.86:0/359592934 wait complete. 2024-04-20T21:29:30.008 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:29:30.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.202+0000 7fc057800640 1 Processor -- start 2024-04-20T21:29:30.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.202+0000 7fc057800640 1 -- start start 2024-04-20T21:29:30.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.204+0000 7fc057800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc050071d00 0x7fc050072190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:30.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.204+0000 7fc057800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc05006dc30 con 0x7fc05006f140 2024-04-20T21:29:30.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.204+0000 7fc057800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc05015f720 con 0x7fc050144e90 2024-04-20T21:29:30.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.204+0000 7fc057800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc050145270 con 0x7fc050071d00 2024-04-20T21:29:30.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.204+0000 7fc047e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fc05006f140 0x7fc0500aeb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51884/0 (socket says 172.21.15.86:51884) 2024-04-20T21:29:30.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.204+0000 7fc047e00640 1 -- 172.21.15.86:0/1365348186 learned_addr learned my addr 172.21.15.86:0/1365348186 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:29:30.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.205+0000 7fc055800640 1 --2- 172.21.15.86:0/1365348186 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc050071d00 0x7fc050072190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:30.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.205+0000 7fc04fe00640 1 -- 172.21.15.86:0/1365348186 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4026990943 0 0) 0x7fc05006dc30 con 0x7fc05006f140 2024-04-20T21:29:30.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.205+0000 7fc04fe00640 1 -- 172.21.15.86:0/1365348186 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc034003520 con 0x7fc05006f140 2024-04-20T21:29:30.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.205+0000 7fc04fe00640 1 -- 172.21.15.86:0/1365348186 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1780949871 0 0) 0x7fc034003520 con 0x7fc05006f140 2024-04-20T21:29:30.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.205+0000 7fc04fe00640 1 -- 172.21.15.86:0/1365348186 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc050071d00 msgr2=0x7fc050072190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:30.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.205+0000 7fc04fe00640 1 --2- 172.21.15.86:0/1365348186 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc050071d00 0x7fc050072190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:30.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.205+0000 7fc04fe00640 1 -- 172.21.15.86:0/1365348186 >> v1:172.21.15.144:6789/0 conn(0x7fc050144e90 legacy=0x7fc05015ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:30.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.205+0000 7fc04fe00640 1 -- 172.21.15.86:0/1365348186 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc05015fa00 con 0x7fc05006f140 2024-04-20T21:29:30.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.205+0000 7fc04fe00640 1 -- 172.21.15.86:0/1365348186 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fc038001670 con 0x7fc05006f140 2024-04-20T21:29:30.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.205+0000 7fc04fe00640 1 -- 172.21.15.86:0/1365348186 >> v1:172.21.15.86:6789/0 conn(0x7fc05006f140 legacy=0x7fc0500aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:30.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.205+0000 7fc04fe00640 1 --2- 172.21.15.86:0/1365348186 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc050071d00 0x7fc0340039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:30.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.205+0000 7fc04fe00640 1 --2- 172.21.15.86:0/1365348186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc034003fb0 0x7fc034004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:30.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.206+0000 7fc04fe00640 1 --2- 172.21.15.86:0/1365348186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc034004d40 0x7fc0340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:30.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.206+0000 7fc04fe00640 1 -- 172.21.15.86:0/1365348186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc050145270 con 0x7fc034003fb0 2024-04-20T21:29:30.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.206+0000 7fc04fe00640 1 -- 172.21.15.86:0/1365348186 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc03400bb00 con 0x7fc034004d40 2024-04-20T21:29:30.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.206+0000 7fc04fe00640 1 -- 172.21.15.86:0/1365348186 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc03400bd00 con 0x7fc050071d00 2024-04-20T21:29:30.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.206+0000 7fc047e00640 1 --2- 172.21.15.86:0/1365348186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc034003fb0 0x7fc034004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:30.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.206+0000 7fc047e00640 1 -- 172.21.15.86:0/1365348186 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc050071d00 msgr2=0x7fc0340039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:30.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.206+0000 7fc047e00640 1 --2- 172.21.15.86:0/1365348186 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc050071d00 0x7fc0340039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:30.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.206+0000 7fc047e00640 1 -- 172.21.15.86:0/1365348186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc034004d40 msgr2=0x7fc0340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:30.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.206+0000 7fc055800640 1 --2- 172.21.15.86:0/1365348186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc034004d40 0x7fc0340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:30.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.206+0000 7fc047e00640 1 --2- 172.21.15.86:0/1365348186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc034004d40 0x7fc0340051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:30.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.206+0000 7fc047e00640 1 -- 172.21.15.86:0/1365348186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc034003520 con 0x7fc034003fb0 2024-04-20T21:29:30.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.207+0000 7fc047e00640 1 --2- 172.21.15.86:0/1365348186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc034003fb0 0x7fc034004800 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fc03800c110 tx=0x7fc034005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=317b7b62fdd609ac server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:30.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.207+0000 7fc04fe00640 1 -- 172.21.15.86:0/1365348186 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc038007c40 con 0x7fc034003fb0 2024-04-20T21:29:30.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.208+0000 7fc04fe00640 1 -- 172.21.15.86:0/1365348186 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc038030070 con 0x7fc034003fb0 2024-04-20T21:29:30.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.208+0000 7fc04fe00640 1 -- 172.21.15.86:0/1365348186 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc038005d70 con 0x7fc034003fb0 2024-04-20T21:29:30.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.208+0000 7fc057800640 1 -- 172.21.15.86:0/1365348186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc034003fb0 msgr2=0x7fc034004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:30.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.208+0000 7fc057800640 1 --2- 172.21.15.86:0/1365348186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc034003fb0 0x7fc034004800 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fc03800c110 tx=0x7fc034005830 comp rx=0 tx=0).stop 2024-04-20T21:29:30.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.208+0000 7fc055800640 1 -- 172.21.15.86:0/1365348186 reap_dead start 2024-04-20T21:29:30.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.208+0000 7fc057800640 1 -- 172.21.15.86:0/1365348186 shutdown_connections 2024-04-20T21:29:30.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.208+0000 7fc057800640 1 -- 172.21.15.86:0/1365348186 >> 172.21.15.86:0/1365348186 conn(0x7fc0500b2360 msgr2=0x7fc0500b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:30.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.208+0000 7fc057800640 1 -- 172.21.15.86:0/1365348186 shutdown_connections 2024-04-20T21:29:30.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.208+0000 7fc057800640 1 -- 172.21.15.86:0/1365348186 wait complete. 2024-04-20T21:29:30.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.210+0000 7fc057800640 1 Processor -- start 2024-04-20T21:29:30.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.210+0000 7fc057800640 1 -- start start 2024-04-20T21:29:30.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.211+0000 7fc057800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc034003fb0 0x7fc0500cf980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:30.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.211+0000 7fc057800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0500cce60 0x7fc05015e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:30.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.211+0000 7fc057800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0500cd890 0x7fc0500cdd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:30.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.211+0000 7fc057800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc0500af280 con 0x7fc0500cce60 2024-04-20T21:29:30.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.211+0000 7fc057800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc05006ded0 con 0x7fc0500cd890 2024-04-20T21:29:30.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.211+0000 7fc057800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc0500af400 con 0x7fc034003fb0 2024-04-20T21:29:30.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.211+0000 7fc047e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0500cce60 0x7fc05015e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:30.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.211+0000 7fc047e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0500cce60 0x7fc05015e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43786/0 (socket says 172.21.15.86:43786) 2024-04-20T21:29:30.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.211+0000 7fc047e00640 1 -- 172.21.15.86:0/885750346 learned_addr learned my addr 172.21.15.86:0/885750346 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:30.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.212+0000 7fc054e00640 1 --2- 172.21.15.86:0/885750346 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc034003fb0 0x7fc0500cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:30.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.212+0000 7fc055800640 1 --2- 172.21.15.86:0/885750346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0500cd890 0x7fc0500cdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:30.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.212+0000 7fc047e00640 1 -- 172.21.15.86:0/885750346 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc034003fb0 msgr2=0x7fc0500cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:30.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.212+0000 7fc047e00640 1 --2- 172.21.15.86:0/885750346 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc034003fb0 0x7fc0500cf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:30.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.212+0000 7fc047e00640 1 -- 172.21.15.86:0/885750346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0500cd890 msgr2=0x7fc0500cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:30.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.212+0000 7fc047e00640 1 --2- 172.21.15.86:0/885750346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0500cd890 0x7fc0500cdd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:30.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.212+0000 7fc047e00640 1 -- 172.21.15.86:0/885750346 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0500cd2c0 con 0x7fc0500cce60 2024-04-20T21:29:30.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.212+0000 7fc055800640 1 --2- 172.21.15.86:0/885750346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0500cd890 0x7fc0500cdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:29:30.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.212+0000 7fc047e00640 1 --2- 172.21.15.86:0/885750346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0500cce60 0x7fc05015e230 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fc03802c3e0 tx=0x7fc05015f8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:30.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.212+0000 7fc04d600640 1 -- 172.21.15.86:0/885750346 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc038007480 con 0x7fc0500cce60 2024-04-20T21:29:30.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.212+0000 7fc04d600640 1 -- 172.21.15.86:0/885750346 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc038000ed0 con 0x7fc0500cce60 2024-04-20T21:29:30.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.212+0000 7fc057800640 1 -- 172.21.15.86:0/885750346 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0500cff80 con 0x7fc0500cce60 2024-04-20T21:29:30.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.212+0000 7fc04d600640 1 -- 172.21.15.86:0/885750346 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03802c770 con 0x7fc0500cce60 2024-04-20T21:29:30.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.213+0000 7fc057800640 1 -- 172.21.15.86:0/885750346 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc050146ca0 con 0x7fc0500cce60 2024-04-20T21:29:30.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.214+0000 7fc057800640 1 -- 172.21.15.86:0/885750346 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0500689e0 con 0x7fc0500cce60 2024-04-20T21:29:30.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.214+0000 7fc04d600640 1 -- 172.21.15.86:0/885750346 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc038047020 con 0x7fc0500cce60 2024-04-20T21:29:30.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.215+0000 7fc04d600640 1 --2- 172.21.15.86:0/885750346 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc02c06c8f0 0x7fc02c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:30.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.215+0000 7fc04d600640 1 -- 172.21.15.86:0/885750346 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5881+0+0 (secure 0 0 0) 0x7fc0380acfe0 con 0x7fc0500cce60 2024-04-20T21:29:30.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.216+0000 7fc054e00640 1 --2- 172.21.15.86:0/885750346 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc02c06c8f0 0x7fc02c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:30.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.216+0000 7fc054e00640 1 --2- 172.21.15.86:0/885750346 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc02c06c8f0 0x7fc02c06edb0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fc0400016f0 tx=0x7fc040009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:30.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.220+0000 7fc04d600640 1 -- 172.21.15.86:0/885750346 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc03807b2c0 con 0x7fc0500cce60 2024-04-20T21:29:30.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.415+0000 7fc057800640 1 -- 172.21.15.86:0/885750346 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc0500b4380 con 0x7fc0500cce60 2024-04-20T21:29:30.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.419+0000 7fc04d600640 1 -- 172.21.15.86:0/885750346 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+16869 (secure 0 0 0) 0x7fc03807abc0 con 0x7fc0500cce60 2024-04-20T21:29:30.418 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:29:30.418 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":102,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:29:25.141262+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:27:43.469685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-20T21:27:44.673505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6836/4271041811":"2024-04-21T21:29:25.141228+0000","172.21.15.86:6834/4271041811":"2024-04-21T21:29:25.141228+0000","172.21.15.86:6836/1331851138":"2024-04-21T21:28:47.841886+0000","172.21.15.86:6834/1331851138":"2024-04-21T21:28:47.841886+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:29:30.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.421+0000 7fc057800640 1 -- 172.21.15.86:0/885750346 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc02c06c8f0 msgr2=0x7fc02c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:30.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.421+0000 7fc057800640 1 --2- 172.21.15.86:0/885750346 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc02c06c8f0 0x7fc02c06edb0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fc0400016f0 tx=0x7fc040009040 comp rx=0 tx=0).stop 2024-04-20T21:29:30.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.421+0000 7fc057800640 1 -- 172.21.15.86:0/885750346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0500cce60 msgr2=0x7fc05015e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:30.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.421+0000 7fc057800640 1 --2- 172.21.15.86:0/885750346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0500cce60 0x7fc05015e230 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fc03802c3e0 tx=0x7fc05015f8f0 comp rx=0 tx=0).stop 2024-04-20T21:29:30.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.421+0000 7fc057800640 1 -- 172.21.15.86:0/885750346 shutdown_connections 2024-04-20T21:29:30.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.421+0000 7fc057800640 1 --2- 172.21.15.86:0/885750346 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc02c06c8f0 0x7fc02c06edb0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:30.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.421+0000 7fc057800640 1 --2- 172.21.15.86:0/885750346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0500cd890 0x7fc0500cdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:30.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.421+0000 7fc057800640 1 --2- 172.21.15.86:0/885750346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0500cce60 0x7fc05015e230 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:30.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.421+0000 7fc057800640 1 --2- 172.21.15.86:0/885750346 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc034003fb0 0x7fc0500cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:30.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.421+0000 7fc057800640 1 -- 172.21.15.86:0/885750346 >> 172.21.15.86:0/885750346 conn(0x7fc0500b2360 msgr2=0x7fc050144130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:30.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.422+0000 7fc057800640 1 -- 172.21.15.86:0/885750346 shutdown_connections 2024-04-20T21:29:30.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.422+0000 7fc057800640 1 -- 172.21.15.86:0/885750346 wait complete. 2024-04-20T21:29:30.439 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-20T21:29:30.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.632+0000 7f4872000640 1 Processor -- start 2024-04-20T21:29:30.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.632+0000 7f4872000640 1 -- start start 2024-04-20T21:29:30.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.634+0000 7f4872000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f486c162810 0x7f486c0ae240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:30.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.634+0000 7f4872000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f486c163250 con 0x7f486c1623d0 2024-04-20T21:29:30.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.634+0000 7f4872000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f486c0b34b0 con 0x7f486c15e740 2024-04-20T21:29:30.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.634+0000 7f4872000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f486c0b3790 con 0x7f486c162810 2024-04-20T21:29:30.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.634+0000 7f4862a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f486c1623d0 0x7f486c0adb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51900/0 (socket says 172.21.15.86:51900) 2024-04-20T21:29:30.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.634+0000 7f4862a00640 1 -- 172.21.15.86:0/1285562801 learned_addr learned my addr 172.21.15.86:0/1285562801 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:29:30.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.635+0000 7f486be00640 1 --2- 172.21.15.86:0/1285562801 >> v2:172.21.15.144:3300/0 conn(0x7f486c162810 0x7f486c0ae240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:30.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.635+0000 7f486aa00640 1 -- 172.21.15.86:0/1285562801 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1632395433 0 0) 0x7f486c163250 con 0x7f486c1623d0 2024-04-20T21:29:30.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.635+0000 7f486aa00640 1 -- 172.21.15.86:0/1285562801 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4850003520 con 0x7f486c1623d0 2024-04-20T21:29:30.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.635+0000 7f486aa00640 1 -- 172.21.15.86:0/1285562801 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 862864442 0 0) 0x7f4850003520 con 0x7f486c1623d0 2024-04-20T21:29:30.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.635+0000 7f486aa00640 1 -- 172.21.15.86:0/1285562801 >> v2:172.21.15.144:3300/0 conn(0x7f486c162810 msgr2=0x7f486c0ae240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:30.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.635+0000 7f486aa00640 1 --2- 172.21.15.86:0/1285562801 >> v2:172.21.15.144:3300/0 conn(0x7f486c162810 0x7f486c0ae240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:30.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.635+0000 7f486aa00640 1 -- 172.21.15.86:0/1285562801 >> v1:172.21.15.144:6789/0 conn(0x7f486c15e740 legacy=0x7f486c0d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:30.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.635+0000 7f486aa00640 1 -- 172.21.15.86:0/1285562801 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f486c0b3a70 con 0x7f486c1623d0 2024-04-20T21:29:30.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.635+0000 7f486aa00640 1 -- 172.21.15.86:0/1285562801 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f4854001670 con 0x7f486c1623d0 2024-04-20T21:29:30.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.636+0000 7f486aa00640 1 -- 172.21.15.86:0/1285562801 >> v1:172.21.15.86:6789/0 conn(0x7f486c1623d0 legacy=0x7f486c0adb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:30.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.636+0000 7f486aa00640 1 --2- 172.21.15.86:0/1285562801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48500039b0 0x7f4850003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:30.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.636+0000 7f486aa00640 1 --2- 172.21.15.86:0/1285562801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4850004800 0x7f4850004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:30.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.636+0000 7f486aa00640 1 --2- 172.21.15.86:0/1285562801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f48500051c0 0x7f485000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:30.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.636+0000 7f486aa00640 1 -- 172.21.15.86:0/1285562801 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f486c0b3790 con 0x7f4850004800 2024-04-20T21:29:30.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.636+0000 7f486aa00640 1 -- 172.21.15.86:0/1285562801 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f485000c060 con 0x7f48500039b0 2024-04-20T21:29:30.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.636+0000 7f486aa00640 1 -- 172.21.15.86:0/1285562801 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f485000c260 con 0x7f48500051c0 2024-04-20T21:29:30.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.636+0000 7f4862a00640 1 --2- 172.21.15.86:0/1285562801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4850004800 0x7f4850004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:30.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.636+0000 7f486be00640 1 --2- 172.21.15.86:0/1285562801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f48500051c0 0x7f485000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:30.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.636+0000 7f4862a00640 1 -- 172.21.15.86:0/1285562801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f48500051c0 msgr2=0x7f485000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:30.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.636+0000 7f486b400640 1 --2- 172.21.15.86:0/1285562801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48500039b0 0x7f4850003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:30.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.636+0000 7f4862a00640 1 --2- 172.21.15.86:0/1285562801 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f48500051c0 0x7f485000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:30.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.637+0000 7f4862a00640 1 -- 172.21.15.86:0/1285562801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48500039b0 msgr2=0x7f4850003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:30.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.637+0000 7f4862a00640 1 --2- 172.21.15.86:0/1285562801 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48500039b0 0x7f4850003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:30.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.637+0000 7f486be00640 1 -- 172.21.15.86:0/1285562801 reap_dead start 2024-04-20T21:29:30.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.637+0000 7f4862a00640 1 -- 172.21.15.86:0/1285562801 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4850003520 con 0x7f4850004800 2024-04-20T21:29:30.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.638+0000 7f4862a00640 1 --2- 172.21.15.86:0/1285562801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4850004800 0x7f4850004c80 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f485400c240 tx=0x7f4850005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b125844967aeb7f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:30.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.638+0000 7f486aa00640 1 -- 172.21.15.86:0/1285562801 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4854007cf0 con 0x7f4850004800 2024-04-20T21:29:30.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.638+0000 7f486aa00640 1 -- 172.21.15.86:0/1285562801 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4854030070 con 0x7f4850004800 2024-04-20T21:29:30.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.638+0000 7f486aa00640 1 -- 172.21.15.86:0/1285562801 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4854005d70 con 0x7f4850004800 2024-04-20T21:29:30.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.639+0000 7f4872000640 1 -- 172.21.15.86:0/1285562801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4850004800 msgr2=0x7f4850004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:30.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.639+0000 7f4872000640 1 --2- 172.21.15.86:0/1285562801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4850004800 0x7f4850004c80 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f485400c240 tx=0x7f4850005830 comp rx=0 tx=0).stop 2024-04-20T21:29:30.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.639+0000 7f4872000640 1 -- 172.21.15.86:0/1285562801 shutdown_connections 2024-04-20T21:29:30.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.639+0000 7f4872000640 1 --2- 172.21.15.86:0/1285562801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4850004800 0x7f4850004c80 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:30.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.639+0000 7f4872000640 1 -- 172.21.15.86:0/1285562801 >> 172.21.15.86:0/1285562801 conn(0x7f486c06b980 msgr2=0x7f486c06bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:30.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.639+0000 7f4872000640 1 -- 172.21.15.86:0/1285562801 shutdown_connections 2024-04-20T21:29:30.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.639+0000 7f4872000640 1 -- 172.21.15.86:0/1285562801 wait complete. 2024-04-20T21:29:30.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.641+0000 7f4872000640 1 Processor -- start 2024-04-20T21:29:30.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.641+0000 7f4872000640 1 -- start start 2024-04-20T21:29:30.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.642+0000 7f4872000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4850004800 0x7f486c155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:30.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.642+0000 7f4872000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f486c1f1930 0x7f486c1534b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:30.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.642+0000 7f4872000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f486c1f79f0 0x7f486c1f7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:30.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.642+0000 7f4872000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f486c162c80 con 0x7f4850004800 2024-04-20T21:29:30.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.642+0000 7f4872000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f486c0b4030 con 0x7f486c1f79f0 2024-04-20T21:29:30.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.642+0000 7f4872000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f486c0b4550 con 0x7f486c1f1930 2024-04-20T21:29:30.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.642+0000 7f486b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4850004800 0x7f486c155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:30.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.642+0000 7f486b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4850004800 0x7f486c155aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43814/0 (socket says 172.21.15.86:43814) 2024-04-20T21:29:30.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.642+0000 7f486b400640 1 -- 172.21.15.86:0/1971223458 learned_addr learned my addr 172.21.15.86:0/1971223458 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:30.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.642+0000 7f486be00640 1 --2- 172.21.15.86:0/1971223458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f486c1f79f0 0x7f486c1f7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:30.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.642+0000 7f4862a00640 1 --2- 172.21.15.86:0/1971223458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f486c1f1930 0x7f486c1534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:30.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.642+0000 7f486b400640 1 -- 172.21.15.86:0/1971223458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f486c1f1930 msgr2=0x7f486c1534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:30.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.642+0000 7f486b400640 1 --2- 172.21.15.86:0/1971223458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f486c1f1930 0x7f486c1534b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:30.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.643+0000 7f486b400640 1 -- 172.21.15.86:0/1971223458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f486c1f79f0 msgr2=0x7f486c1f7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:30.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.643+0000 7f486b400640 1 --2- 172.21.15.86:0/1971223458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f486c1f79f0 0x7f486c1f7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:30.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.643+0000 7f486b400640 1 -- 172.21.15.86:0/1971223458 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f486c156640 con 0x7f4850004800 2024-04-20T21:29:30.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.643+0000 7f486be00640 1 --2- 172.21.15.86:0/1971223458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f486c1f79f0 0x7f486c1f7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:29:30.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.643+0000 7f486b400640 1 --2- 172.21.15.86:0/1971223458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4850004800 0x7f486c155aa0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f485c00d4a0 tx=0x7f485c00d960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:30.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.643+0000 7f4863e00640 1 -- 172.21.15.86:0/1971223458 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485c005480 con 0x7f4850004800 2024-04-20T21:29:30.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.643+0000 7f4863e00640 1 -- 172.21.15.86:0/1971223458 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f485c017070 con 0x7f4850004800 2024-04-20T21:29:30.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.643+0000 7f4863e00640 1 -- 172.21.15.86:0/1971223458 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485c003810 con 0x7f4850004800 2024-04-20T21:29:30.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.643+0000 7f4872000640 1 -- 172.21.15.86:0/1971223458 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f486c156080 con 0x7f4850004800 2024-04-20T21:29:30.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.643+0000 7f4872000640 1 -- 172.21.15.86:0/1971223458 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f486c0b1200 con 0x7f4850004800 2024-04-20T21:29:30.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.644+0000 7f4872000640 1 -- 172.21.15.86:0/1971223458 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f486c0689e0 con 0x7f4850004800 2024-04-20T21:29:30.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.645+0000 7f4863e00640 1 -- 172.21.15.86:0/1971223458 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f485c0039b0 con 0x7f4850004800 2024-04-20T21:29:30.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.646+0000 7f4863e00640 1 --2- 172.21.15.86:0/1971223458 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f484806c8f0 0x7f484806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:30.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.646+0000 7f4863e00640 1 -- 172.21.15.86:0/1971223458 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5881+0+0 (secure 0 0 0) 0x7f485c08b930 con 0x7f4850004800 2024-04-20T21:29:30.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.647+0000 7f4862a00640 1 --2- 172.21.15.86:0/1971223458 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f484806c8f0 0x7f484806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:30.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.647+0000 7f4862a00640 1 --2- 172.21.15.86:0/1971223458 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f484806c8f0 0x7f484806edb0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f4854034000 tx=0x7f4854039040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:30.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.651+0000 7f4863e00640 1 -- 172.21.15.86:0/1971223458 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f485c00f2f0 con 0x7f4850004800 2024-04-20T21:29:30.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:30.876+0000 7f4872000640 1 -- 172.21.15.86:0/1971223458 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f486c06e840 con 0x7f4850004800 2024-04-20T21:29:31.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.185+0000 7f4863e00640 1 -- 172.21.15.86:0/1971223458 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v91) ==== 161+0+0 (secure 0 0 0) 0x7f485c059ca0 con 0x7f4850004800 2024-04-20T21:29:31.185 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-20T21:29:31.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.189+0000 7f4872000640 1 -- 172.21.15.86:0/1971223458 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f484806c8f0 msgr2=0x7f484806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:31.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.189+0000 7f4872000640 1 --2- 172.21.15.86:0/1971223458 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f484806c8f0 0x7f484806edb0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f4854034000 tx=0x7f4854039040 comp rx=0 tx=0).stop 2024-04-20T21:29:31.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.189+0000 7f4872000640 1 -- 172.21.15.86:0/1971223458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4850004800 msgr2=0x7f486c155aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:31.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.189+0000 7f4872000640 1 --2- 172.21.15.86:0/1971223458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4850004800 0x7f486c155aa0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f485c00d4a0 tx=0x7f485c00d960 comp rx=0 tx=0).stop 2024-04-20T21:29:31.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.189+0000 7f4872000640 1 -- 172.21.15.86:0/1971223458 shutdown_connections 2024-04-20T21:29:31.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.189+0000 7f4872000640 1 --2- 172.21.15.86:0/1971223458 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f484806c8f0 0x7f484806edb0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:31.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.189+0000 7f4872000640 1 --2- 172.21.15.86:0/1971223458 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f486c1f79f0 0x7f486c1f7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:31.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.189+0000 7f4872000640 1 --2- 172.21.15.86:0/1971223458 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f486c1f1930 0x7f486c1534b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:31.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.190+0000 7f4872000640 1 --2- 172.21.15.86:0/1971223458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4850004800 0x7f486c155aa0 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:31.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.190+0000 7f4872000640 1 -- 172.21.15.86:0/1971223458 >> 172.21.15.86:0/1971223458 conn(0x7f486c06b980 msgr2=0x7f486c15eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:31.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.190+0000 7f4872000640 1 -- 172.21.15.86:0/1971223458 shutdown_connections 2024-04-20T21:29:31.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.190+0000 7f4872000640 1 -- 172.21.15.86:0/1971223458 wait complete. 2024-04-20T21:29:31.210 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-20T21:29:31.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.411+0000 7fb2cb800640 1 Processor -- start 2024-04-20T21:29:31.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.411+0000 7fb2cb800640 1 -- start start 2024-04-20T21:29:31.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.412+0000 7fb2cb800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2c415eb30 0x7fb2c40b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:31.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.412+0000 7fb2cb800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb2c406c090 0x7fb2c40b4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:31.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.412+0000 7fb2cb800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2c40adb10 con 0x7fb2c40aee50 2024-04-20T21:29:31.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.412+0000 7fb2cb800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb2c40af7d0 con 0x7fb2c406c090 2024-04-20T21:29:31.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.412+0000 7fb2cb800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb2c40addb0 con 0x7fb2c415eb30 2024-04-20T21:29:31.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.413+0000 7fb2c9800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb2c406c090 0x7fb2c40b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:31.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.413+0000 7fb2c9800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb2c406c090 0x7fb2c40b4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43818/0 (socket says 172.21.15.86:43818) 2024-04-20T21:29:31.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.413+0000 7fb2c9800640 1 -- 172.21.15.86:0/1281469110 learned_addr learned my addr 172.21.15.86:0/1281469110 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:31.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.413+0000 7fb2c9800640 1 -- 172.21.15.86:0/1281469110 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2c415eb30 msgr2=0x7fb2c40b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:31.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.413+0000 7fb2c9800640 1 --2- 172.21.15.86:0/1281469110 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2c415eb30 0x7fb2c40b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:31.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.413+0000 7fb2c9800640 1 -- 172.21.15.86:0/1281469110 >> v1:172.21.15.144:6789/0 conn(0x7fb2c40aee50 legacy=0x7fb2c406b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:31.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.413+0000 7fb2c9800640 1 -- 172.21.15.86:0/1281469110 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c40ae090 con 0x7fb2c406c090 2024-04-20T21:29:31.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.414+0000 7fb2c9800640 1 --2- 172.21.15.86:0/1281469110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2c406c090 0x7fb2c40b4360 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc009e50 tx=0x7fb2bc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42e5d73ef3963d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:31.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.415+0000 7fb2c3e00640 1 -- 172.21.15.86:0/1281469110 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc02ca70 con 0x7fb2c406c090 2024-04-20T21:29:31.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.415+0000 7fb2c3e00640 1 -- 172.21.15.86:0/1281469110 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2bc02cc10 con 0x7fb2c406c090 2024-04-20T21:29:31.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.415+0000 7fb2c3e00640 1 -- 172.21.15.86:0/1281469110 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc0343f0 con 0x7fb2c406c090 2024-04-20T21:29:31.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.415+0000 7fb2cb800640 1 -- 172.21.15.86:0/1281469110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2c406c090 msgr2=0x7fb2c40b4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:31.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.415+0000 7fb2cb800640 1 --2- 172.21.15.86:0/1281469110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2c406c090 0x7fb2c40b4360 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc009e50 tx=0x7fb2bc02bed0 comp rx=0 tx=0).stop 2024-04-20T21:29:31.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.416+0000 7fb2cb800640 1 -- 172.21.15.86:0/1281469110 shutdown_connections 2024-04-20T21:29:31.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.416+0000 7fb2cb800640 1 --2- 172.21.15.86:0/1281469110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2c406c090 0x7fb2c40b4360 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:31.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.416+0000 7fb2cb800640 1 --2- 172.21.15.86:0/1281469110 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2c415eb30 0x7fb2c40b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:31.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.416+0000 7fb2cb800640 1 -- 172.21.15.86:0/1281469110 >> 172.21.15.86:0/1281469110 conn(0x7fb2c40b2110 msgr2=0x7fb2c40b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:31.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.416+0000 7fb2cb800640 1 -- 172.21.15.86:0/1281469110 shutdown_connections 2024-04-20T21:29:31.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.416+0000 7fb2cb800640 1 -- 172.21.15.86:0/1281469110 wait complete. 2024-04-20T21:29:31.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.417+0000 7fb2cb800640 1 Processor -- start 2024-04-20T21:29:31.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.417+0000 7fb2cb800640 1 -- start start 2024-04-20T21:29:31.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.418+0000 7fb2cb800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2c406c090 0x7fb2c414b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:31.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.418+0000 7fb2cb800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2c40aee50 0x7fb2c40cc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:31.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.418+0000 7fb2cb800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2c415eb30 0x7fb2c40ccf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:31.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.418+0000 7fb2cb800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c40ae930 con 0x7fb2c406c090 2024-04-20T21:29:31.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.418+0000 7fb2cb800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c40ae650 con 0x7fb2c40aee50 2024-04-20T21:29:31.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.418+0000 7fb2cb800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb2c406e160 con 0x7fb2c415eb30 2024-04-20T21:29:31.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.419+0000 7fb2c8e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2c406c090 0x7fb2c414b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:31.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.419+0000 7fb2c8e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2c406c090 0x7fb2c414b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43824/0 (socket says 172.21.15.86:43824) 2024-04-20T21:29:31.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.419+0000 7fb2c8e00640 1 -- 172.21.15.86:0/404756117 learned_addr learned my addr 172.21.15.86:0/404756117 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:31.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.419+0000 7fb2c9800640 1 --2- 172.21.15.86:0/404756117 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2c415eb30 0x7fb2c40ccf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:31.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.419+0000 7fb2bbe00640 1 --2- 172.21.15.86:0/404756117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2c40aee50 0x7fb2c40cc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:31.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.419+0000 7fb2c8e00640 1 -- 172.21.15.86:0/404756117 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2c415eb30 msgr2=0x7fb2c40ccf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:31.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.419+0000 7fb2c8e00640 1 --2- 172.21.15.86:0/404756117 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2c415eb30 0x7fb2c40ccf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:31.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.419+0000 7fb2c8e00640 1 -- 172.21.15.86:0/404756117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2c40aee50 msgr2=0x7fb2c40cc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:31.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.419+0000 7fb2c8e00640 1 --2- 172.21.15.86:0/404756117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2c40aee50 0x7fb2c40cc9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:31.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.419+0000 7fb2c8e00640 1 -- 172.21.15.86:0/404756117 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c40cd550 con 0x7fb2c406c090 2024-04-20T21:29:31.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.420+0000 7fb2c8e00640 1 --2- 172.21.15.86:0/404756117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2c406c090 0x7fb2c414b180 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fb2b400b410 tx=0x7fb2b400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:31.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.420+0000 7fb2c1600640 1 -- 172.21.15.86:0/404756117 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b4015020 con 0x7fb2c406c090 2024-04-20T21:29:31.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.420+0000 7fb2c1600640 1 -- 172.21.15.86:0/404756117 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2b4011850 con 0x7fb2c406c090 2024-04-20T21:29:31.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.420+0000 7fb2c1600640 1 -- 172.21.15.86:0/404756117 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b4011c40 con 0x7fb2c406c090 2024-04-20T21:29:31.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.420+0000 7fb2cb800640 1 -- 172.21.15.86:0/404756117 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2c40cd830 con 0x7fb2c406c090 2024-04-20T21:29:31.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.420+0000 7fb2cb800640 1 -- 172.21.15.86:0/404756117 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2c4159bb0 con 0x7fb2c406c090 2024-04-20T21:29:31.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.421+0000 7fb2cb800640 1 -- 172.21.15.86:0/404756117 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2c40689e0 con 0x7fb2c406c090 2024-04-20T21:29:31.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.422+0000 7fb2c1600640 1 -- 172.21.15.86:0/404756117 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb2b4018020 con 0x7fb2c406c090 2024-04-20T21:29:31.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.423+0000 7fb2c1600640 1 --2- 172.21.15.86:0/404756117 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb2a006c8f0 0x7fb2a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:31.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.424+0000 7fb2c1600640 1 -- 172.21.15.86:0/404756117 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5881+0+0 (secure 0 0 0) 0x7fb2b408bb20 con 0x7fb2c406c090 2024-04-20T21:29:31.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.424+0000 7fb2bbe00640 1 --2- 172.21.15.86:0/404756117 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb2a006c8f0 0x7fb2a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:31.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.424+0000 7fb2bbe00640 1 --2- 172.21.15.86:0/404756117 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb2a006c8f0 0x7fb2a006edb0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fb2c40ce010 tx=0x7fb2ac009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:31.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.428+0000 7fb2c1600640 1 -- 172.21.15.86:0/404756117 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb2b401f020 con 0x7fb2c406c090 2024-04-20T21:29:31.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:31.650+0000 7fb2cb800640 1 -- 172.21.15.86:0/404756117 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fb2c4159f90 con 0x7fb2c406c090 2024-04-20T21:29:32.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.198+0000 7fb2c1600640 1 -- 172.21.15.86:0/404756117 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v92) ==== 104+0+0 (secure 0 0 0) 0x7fb2b40172c0 con 0x7fb2c406c090 2024-04-20T21:29:32.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.203+0000 7fb2cb800640 1 -- 172.21.15.86:0/404756117 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb2a006c8f0 msgr2=0x7fb2a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:32.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.203+0000 7fb2cb800640 1 --2- 172.21.15.86:0/404756117 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb2a006c8f0 0x7fb2a006edb0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fb2c40ce010 tx=0x7fb2ac009040 comp rx=0 tx=0).stop 2024-04-20T21:29:32.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.203+0000 7fb2cb800640 1 -- 172.21.15.86:0/404756117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2c406c090 msgr2=0x7fb2c414b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:32.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.203+0000 7fb2cb800640 1 --2- 172.21.15.86:0/404756117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2c406c090 0x7fb2c414b180 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fb2b400b410 tx=0x7fb2b400b8d0 comp rx=0 tx=0).stop 2024-04-20T21:29:32.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.203+0000 7fb2cb800640 1 -- 172.21.15.86:0/404756117 shutdown_connections 2024-04-20T21:29:32.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.203+0000 7fb2cb800640 1 --2- 172.21.15.86:0/404756117 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb2a006c8f0 0x7fb2a006edb0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:32.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.203+0000 7fb2cb800640 1 --2- 172.21.15.86:0/404756117 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2c415eb30 0x7fb2c40ccf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:32.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.203+0000 7fb2cb800640 1 --2- 172.21.15.86:0/404756117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2c40aee50 0x7fb2c40cc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:32.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.203+0000 7fb2cb800640 1 --2- 172.21.15.86:0/404756117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2c406c090 0x7fb2c414b180 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:32.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.203+0000 7fb2cb800640 1 -- 172.21.15.86:0/404756117 >> 172.21.15.86:0/404756117 conn(0x7fb2c40b2110 msgr2=0x7fb2c40b0040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:32.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.203+0000 7fb2cb800640 1 -- 172.21.15.86:0/404756117 shutdown_connections 2024-04-20T21:29:32.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.203+0000 7fb2cb800640 1 -- 172.21.15.86:0/404756117 wait complete. 2024-04-20T21:29:32.224 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-20T21:29:32.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.431+0000 7f271de00640 1 Processor -- start 2024-04-20T21:29:32.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.432+0000 7f271de00640 1 -- start start 2024-04-20T21:29:32.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.433+0000 7f271de00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f271815e840 0x7f27180c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:32.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.433+0000 7f271de00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f271815ec20 0x7f2718163080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:32.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.433+0000 7f271de00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f27180d2d30 0x7f27180d4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:32.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.433+0000 7f271de00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f27180d3510 con 0x7f271815e840 2024-04-20T21:29:32.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.433+0000 7f271de00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f27180cf5b0 con 0x7f271815ec20 2024-04-20T21:29:32.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.433+0000 7f271de00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f27180cf890 con 0x7f27180d2d30 2024-04-20T21:29:32.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.434+0000 7f2716a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f271815e840 0x7f27180c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:32.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.434+0000 7f2716a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f271815e840 0x7f27180c82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43840/0 (socket says 172.21.15.86:43840) 2024-04-20T21:29:32.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.434+0000 7f2716a00640 1 -- 172.21.15.86:0/1978476323 learned_addr learned my addr 172.21.15.86:0/1978476323 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:32.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.434+0000 7f2716a00640 1 -- 172.21.15.86:0/1978476323 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f27180d2d30 msgr2=0x7f27180d4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:32.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.434+0000 7f2716a00640 1 --2- 172.21.15.86:0/1978476323 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f27180d2d30 0x7f27180d4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:32.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.434+0000 7f2716a00640 1 -- 172.21.15.86:0/1978476323 >> v2:172.21.15.144:3300/0 conn(0x7f271815ec20 msgr2=0x7f2718163080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:32.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.434+0000 7f2716a00640 1 --2- 172.21.15.86:0/1978476323 >> v2:172.21.15.144:3300/0 conn(0x7f271815ec20 0x7f2718163080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:32.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.434+0000 7f2716a00640 1 -- 172.21.15.86:0/1978476323 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27180cfb70 con 0x7f271815e840 2024-04-20T21:29:32.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.435+0000 7f2716a00640 1 --2- 172.21.15.86:0/1978476323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f271815e840 0x7f27180c82a0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f2718162500 tx=0x7f270802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ca6d32526459862 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:32.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.436+0000 7f2716000640 1 -- 172.21.15.86:0/1978476323 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2708035020 con 0x7f271815e840 2024-04-20T21:29:32.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.436+0000 7f2716000640 1 -- 172.21.15.86:0/1978476323 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f270802d040 con 0x7f271815e840 2024-04-20T21:29:32.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.436+0000 7f2716000640 1 -- 172.21.15.86:0/1978476323 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2708031a80 con 0x7f271815e840 2024-04-20T21:29:32.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.436+0000 7f271de00640 1 -- 172.21.15.86:0/1978476323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f271815e840 msgr2=0x7f27180c82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:32.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.436+0000 7f271de00640 1 --2- 172.21.15.86:0/1978476323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f271815e840 0x7f27180c82a0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f2718162500 tx=0x7f270802b900 comp rx=0 tx=0).stop 2024-04-20T21:29:32.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.436+0000 7f271de00640 1 -- 172.21.15.86:0/1978476323 shutdown_connections 2024-04-20T21:29:32.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.436+0000 7f271de00640 1 --2- 172.21.15.86:0/1978476323 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f27180d2d30 0x7f27180d4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:32.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.436+0000 7f271de00640 1 --2- 172.21.15.86:0/1978476323 >> v2:172.21.15.144:3300/0 conn(0x7f271815ec20 0x7f2718163080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:32.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.436+0000 7f271de00640 1 --2- 172.21.15.86:0/1978476323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f271815e840 0x7f27180c82a0 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:32.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.436+0000 7f271de00640 1 -- 172.21.15.86:0/1978476323 >> 172.21.15.86:0/1978476323 conn(0x7f271809da40 msgr2=0x7f271808bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:32.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.436+0000 7f271de00640 1 -- 172.21.15.86:0/1978476323 shutdown_connections 2024-04-20T21:29:32.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.437+0000 7f271de00640 1 -- 172.21.15.86:0/1978476323 wait complete. 2024-04-20T21:29:32.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.438+0000 7f271de00640 1 Processor -- start 2024-04-20T21:29:32.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.438+0000 7f271de00640 1 -- start start 2024-04-20T21:29:32.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.439+0000 7f271de00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27180d2d30 0x7f271815bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:32.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.439+0000 7f271de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f271815e840 0x7f27180d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:32.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.439+0000 7f271de00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f271815ec20 0x7f27180d1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:32.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.439+0000 7f271de00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f27180d4940 con 0x7f271815e840 2024-04-20T21:29:32.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.439+0000 7f271de00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f27180d3180 con 0x7f27180d2d30 2024-04-20T21:29:32.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.439+0000 7f271de00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f27180d4ac0 con 0x7f271815ec20 2024-04-20T21:29:32.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.440+0000 7f270e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f271815e840 0x7f27180d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:32.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.440+0000 7f270e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f271815e840 0x7f27180d0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43844/0 (socket says 172.21.15.86:43844) 2024-04-20T21:29:32.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.440+0000 7f270e000640 1 -- 172.21.15.86:0/2017789402 learned_addr learned my addr 172.21.15.86:0/2017789402 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:32.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.440+0000 7f2716a00640 1 --2- 172.21.15.86:0/2017789402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27180d2d30 0x7f271815bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:32.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.440+0000 7f270e000640 1 -- 172.21.15.86:0/2017789402 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f271815ec20 msgr2=0x7f27180d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:32.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.440+0000 7f270e000640 1 --2- 172.21.15.86:0/2017789402 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f271815ec20 0x7f27180d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:32.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.440+0000 7f270e000640 1 -- 172.21.15.86:0/2017789402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27180d2d30 msgr2=0x7f271815bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:32.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.440+0000 7f270e000640 1 --2- 172.21.15.86:0/2017789402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27180d2d30 0x7f271815bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:32.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.440+0000 7f270e000640 1 -- 172.21.15.86:0/2017789402 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27181f49e0 con 0x7f271815e840 2024-04-20T21:29:32.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.440+0000 7f270e000640 1 --2- 172.21.15.86:0/2017789402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f271815e840 0x7f27180d0d10 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f270000b410 tx=0x7f270000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:32.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.441+0000 7f270fe00640 1 -- 172.21.15.86:0/2017789402 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2700015020 con 0x7f271815e840 2024-04-20T21:29:32.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.441+0000 7f270fe00640 1 -- 172.21.15.86:0/2017789402 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2700011850 con 0x7f271815e840 2024-04-20T21:29:32.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.441+0000 7f270fe00640 1 -- 172.21.15.86:0/2017789402 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2700011c40 con 0x7f271815e840 2024-04-20T21:29:32.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.441+0000 7f271de00640 1 -- 172.21.15.86:0/2017789402 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27181f4cc0 con 0x7f271815e840 2024-04-20T21:29:32.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.441+0000 7f271de00640 1 -- 172.21.15.86:0/2017789402 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27181f5280 con 0x7f271815e840 2024-04-20T21:29:32.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.442+0000 7f271de00640 1 -- 172.21.15.86:0/2017789402 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27180689e0 con 0x7f271815e840 2024-04-20T21:29:32.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.443+0000 7f270fe00640 1 -- 172.21.15.86:0/2017789402 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2700018020 con 0x7f271815e840 2024-04-20T21:29:32.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.444+0000 7f270fe00640 1 --2- 172.21.15.86:0/2017789402 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f26f406c8f0 0x7f26f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:32.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.444+0000 7f270fe00640 1 -- 172.21.15.86:0/2017789402 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5881+0+0 (secure 0 0 0) 0x7f270008bc10 con 0x7f271815e840 2024-04-20T21:29:32.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.444+0000 7f2716a00640 1 --2- 172.21.15.86:0/2017789402 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f26f406c8f0 0x7f26f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:32.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.445+0000 7f2716a00640 1 --2- 172.21.15.86:0/2017789402 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f26f406c8f0 0x7f26f406edb0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f2708036000 tx=0x7f270803c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:32.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.449+0000 7f270fe00640 1 -- 172.21.15.86:0/2017789402 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f270001f020 con 0x7f271815e840 2024-04-20T21:29:32.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:32.655+0000 7f271de00640 1 -- 172.21.15.86:0/2017789402 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f27180d5640 con 0x7f271815e840 2024-04-20T21:29:33.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.204+0000 7f270fe00640 1 -- 172.21.15.86:0/2017789402 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v103) ==== 181+0+0 (secure 0 0 0) 0x7f27000172c0 con 0x7f271815e840 2024-04-20T21:29:33.203 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' removed 2024-04-20T21:29:33.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.208+0000 7f271de00640 1 -- 172.21.15.86:0/2017789402 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f26f406c8f0 msgr2=0x7f26f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:33.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.208+0000 7f271de00640 1 --2- 172.21.15.86:0/2017789402 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f26f406c8f0 0x7f26f406edb0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f2708036000 tx=0x7f270803c040 comp rx=0 tx=0).stop 2024-04-20T21:29:33.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.208+0000 7f271de00640 1 -- 172.21.15.86:0/2017789402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f271815e840 msgr2=0x7f27180d0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:33.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.208+0000 7f271de00640 1 --2- 172.21.15.86:0/2017789402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f271815e840 0x7f27180d0d10 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f270000b410 tx=0x7f270000b8d0 comp rx=0 tx=0).stop 2024-04-20T21:29:33.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.208+0000 7f271de00640 1 -- 172.21.15.86:0/2017789402 shutdown_connections 2024-04-20T21:29:33.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.208+0000 7f271de00640 1 --2- 172.21.15.86:0/2017789402 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f26f406c8f0 0x7f26f406edb0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:33.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.208+0000 7f271de00640 1 --2- 172.21.15.86:0/2017789402 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f271815ec20 0x7f27180d1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:33.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.208+0000 7f271de00640 1 --2- 172.21.15.86:0/2017789402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f271815e840 0x7f27180d0d10 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:33.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.208+0000 7f271de00640 1 --2- 172.21.15.86:0/2017789402 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27180d2d30 0x7f271815bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:33.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.208+0000 7f271de00640 1 -- 172.21.15.86:0/2017789402 >> 172.21.15.86:0/2017789402 conn(0x7f271809da40 msgr2=0x7f27180d0090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:33.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.209+0000 7f271de00640 1 -- 172.21.15.86:0/2017789402 shutdown_connections 2024-04-20T21:29:33.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.209+0000 7f271de00640 1 -- 172.21.15.86:0/2017789402 wait complete. 2024-04-20T21:29:33.234 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-20T21:29:33.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.433+0000 7fc1f2200640 1 Processor -- start 2024-04-20T21:29:33.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.433+0000 7fc1f2200640 1 -- start start 2024-04-20T21:29:33.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.434+0000 7fc1f2200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc1ec071d00 0x7fc1ec06f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:33.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.435+0000 7fc1f2200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc1ec0720e0 0x7fc1ec145a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:33.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.435+0000 7fc1f2200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1ec145fd0 0x7fc1ec15cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:33.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.435+0000 7fc1f2200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc1ec06f5f0 con 0x7fc1ec071d00 2024-04-20T21:29:33.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.435+0000 7fc1f2200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fc1ec06e010 con 0x7fc1ec0720e0 2024-04-20T21:29:33.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.435+0000 7fc1f2200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc1ec06e2f0 con 0x7fc1ec145fd0 2024-04-20T21:29:33.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.435+0000 7fc1eb400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc1ec071d00 0x7fc1ec06f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:33.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.435+0000 7fc1eb400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc1ec071d00 0x7fc1ec06f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43860/0 (socket says 172.21.15.86:43860) 2024-04-20T21:29:33.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.435+0000 7fc1eb400640 1 -- 172.21.15.86:0/2527926709 learned_addr learned my addr 172.21.15.86:0/2527926709 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:33.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.435+0000 7fc1e3e00640 1 --2- 172.21.15.86:0/2527926709 >> v2:172.21.15.144:3300/0 conn(0x7fc1ec0720e0 0x7fc1ec145a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:33.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.435+0000 7fc1ebe00640 1 --2- 172.21.15.86:0/2527926709 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1ec145fd0 0x7fc1ec15cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:33.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.435+0000 7fc1eb400640 1 -- 172.21.15.86:0/2527926709 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1ec145fd0 msgr2=0x7fc1ec15cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:33.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.435+0000 7fc1eb400640 1 --2- 172.21.15.86:0/2527926709 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1ec145fd0 0x7fc1ec15cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:33.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.435+0000 7fc1eb400640 1 -- 172.21.15.86:0/2527926709 >> v2:172.21.15.144:3300/0 conn(0x7fc1ec0720e0 msgr2=0x7fc1ec145a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:33.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.435+0000 7fc1eb400640 1 --2- 172.21.15.86:0/2527926709 >> v2:172.21.15.144:3300/0 conn(0x7fc1ec0720e0 0x7fc1ec145a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:33.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.435+0000 7fc1eb400640 1 -- 172.21.15.86:0/2527926709 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1ec06e5d0 con 0x7fc1ec071d00 2024-04-20T21:29:33.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.435+0000 7fc1e3e00640 1 --2- 172.21.15.86:0/2527926709 >> v2:172.21.15.144:3300/0 conn(0x7fc1ec0720e0 0x7fc1ec145a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:29:33.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.437+0000 7fc1eb400640 1 --2- 172.21.15.86:0/2527926709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1ec071d00 0x7fc1ec06f0b0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec146550 tx=0x7fc1dc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aef33e750b4e51c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:33.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.437+0000 7fc1eaa00640 1 -- 172.21.15.86:0/2527926709 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1dc035020 con 0x7fc1ec071d00 2024-04-20T21:29:33.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.437+0000 7fc1eaa00640 1 -- 172.21.15.86:0/2527926709 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1dc031870 con 0x7fc1ec071d00 2024-04-20T21:29:33.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.437+0000 7fc1eaa00640 1 -- 172.21.15.86:0/2527926709 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1dc031c40 con 0x7fc1ec071d00 2024-04-20T21:29:33.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.438+0000 7fc1f2200640 1 -- 172.21.15.86:0/2527926709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1ec071d00 msgr2=0x7fc1ec06f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:33.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.438+0000 7fc1f2200640 1 --2- 172.21.15.86:0/2527926709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1ec071d00 0x7fc1ec06f0b0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec146550 tx=0x7fc1dc02b910 comp rx=0 tx=0).stop 2024-04-20T21:29:33.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.438+0000 7fc1f2200640 1 -- 172.21.15.86:0/2527926709 shutdown_connections 2024-04-20T21:29:33.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.438+0000 7fc1f2200640 1 --2- 172.21.15.86:0/2527926709 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1ec145fd0 0x7fc1ec15cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:33.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.438+0000 7fc1f2200640 1 --2- 172.21.15.86:0/2527926709 >> v2:172.21.15.144:3300/0 conn(0x7fc1ec0720e0 0x7fc1ec145a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:33.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.438+0000 7fc1f2200640 1 --2- 172.21.15.86:0/2527926709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1ec071d00 0x7fc1ec06f0b0 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:33.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.438+0000 7fc1f2200640 1 -- 172.21.15.86:0/2527926709 >> 172.21.15.86:0/2527926709 conn(0x7fc1ec0b2180 msgr2=0x7fc1ec0b2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:33.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.438+0000 7fc1f2200640 1 -- 172.21.15.86:0/2527926709 shutdown_connections 2024-04-20T21:29:33.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.438+0000 7fc1f2200640 1 -- 172.21.15.86:0/2527926709 wait complete. 2024-04-20T21:29:33.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.440+0000 7fc1f2200640 1 Processor -- start 2024-04-20T21:29:33.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.440+0000 7fc1f2200640 1 -- start start 2024-04-20T21:29:33.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.441+0000 7fc1f2200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1ec071d00 0x7fc1ec13eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:33.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.441+0000 7fc1f2200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1ec0720e0 0x7fc1ec139c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:33.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.441+0000 7fc1f2200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1ec145fd0 0x7fc1ec0b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:33.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.441+0000 7fc1f2200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ec072540 con 0x7fc1ec071d00 2024-04-20T21:29:33.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.441+0000 7fc1f2200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ec06eb90 con 0x7fc1ec145fd0 2024-04-20T21:29:33.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.441+0000 7fc1f2200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc1ec06dc30 con 0x7fc1ec0720e0 2024-04-20T21:29:33.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.441+0000 7fc1eb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1ec071d00 0x7fc1ec13eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:33.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.441+0000 7fc1eb400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1ec071d00 0x7fc1ec13eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43878/0 (socket says 172.21.15.86:43878) 2024-04-20T21:29:33.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.441+0000 7fc1eb400640 1 -- 172.21.15.86:0/2469902017 learned_addr learned my addr 172.21.15.86:0/2469902017 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:33.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.441+0000 7fc1eb400640 1 -- 172.21.15.86:0/2469902017 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1ec0720e0 msgr2=0x7fc1ec139c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:33.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.441+0000 7fc1e3e00640 1 --2- 172.21.15.86:0/2469902017 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1ec0720e0 0x7fc1ec139c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:33.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.441+0000 7fc1ebe00640 1 --2- 172.21.15.86:0/2469902017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1ec145fd0 0x7fc1ec0b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:33.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.441+0000 7fc1eb400640 1 --2- 172.21.15.86:0/2469902017 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1ec0720e0 0x7fc1ec139c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:33.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.441+0000 7fc1eb400640 1 -- 172.21.15.86:0/2469902017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1ec145fd0 msgr2=0x7fc1ec0b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:33.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.441+0000 7fc1eb400640 1 --2- 172.21.15.86:0/2469902017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1ec145fd0 0x7fc1ec0b4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:33.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.441+0000 7fc1eb400640 1 -- 172.21.15.86:0/2469902017 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1ec0adb00 con 0x7fc1ec071d00 2024-04-20T21:29:33.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.442+0000 7fc1eb400640 1 --2- 172.21.15.86:0/2469902017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1ec071d00 0x7fc1ec13eda0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc000c00 tx=0x7fc1dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:33.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.442+0000 7fc1e3400640 1 -- 172.21.15.86:0/2469902017 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1dc03ad60 con 0x7fc1ec071d00 2024-04-20T21:29:33.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.442+0000 7fc1e3400640 1 -- 172.21.15.86:0/2469902017 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1dc0443f0 con 0x7fc1ec071d00 2024-04-20T21:29:33.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.442+0000 7fc1e3400640 1 -- 172.21.15.86:0/2469902017 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1dc0447e0 con 0x7fc1ec071d00 2024-04-20T21:29:33.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.442+0000 7fc1f2200640 1 -- 172.21.15.86:0/2469902017 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1ec13a1d0 con 0x7fc1ec071d00 2024-04-20T21:29:33.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.442+0000 7fc1f2200640 1 -- 172.21.15.86:0/2469902017 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1ec0ae080 con 0x7fc1ec071d00 2024-04-20T21:29:33.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.443+0000 7fc1f2200640 1 -- 172.21.15.86:0/2469902017 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1ec0689e0 con 0x7fc1ec071d00 2024-04-20T21:29:33.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.444+0000 7fc1e3400640 1 -- 172.21.15.86:0/2469902017 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc1dc04b060 con 0x7fc1ec071d00 2024-04-20T21:29:33.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.445+0000 7fc1e3400640 1 --2- 172.21.15.86:0/2469902017 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc1c806c8f0 0x7fc1c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:33.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.445+0000 7fc1e3400640 1 -- 172.21.15.86:0/2469902017 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5443+0+0 (secure 0 0 0) 0x7fc1dc0adc90 con 0x7fc1ec071d00 2024-04-20T21:29:33.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.446+0000 7fc1e3e00640 1 --2- 172.21.15.86:0/2469902017 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc1c806c8f0 0x7fc1c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:33.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.446+0000 7fc1e3e00640 1 --2- 172.21.15.86:0/2469902017 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc1c806c8f0 0x7fc1c806edb0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec0b3640 tx=0x7fc1d4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:33.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.450+0000 7fc1e3400640 1 -- 172.21.15.86:0/2469902017 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc1dc03f070 con 0x7fc1ec071d00 2024-04-20T21:29:33.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:33.659+0000 7fc1f2200640 1 -- 172.21.15.86:0/2469902017 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc1ec0b2950 con 0x7fc1ec071d00 2024-04-20T21:29:34.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.222+0000 7fc1e3400640 1 -- 172.21.15.86:0/2469902017 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v104) ==== 169+0+0 (secure 0 0 0) 0x7fc1dc02f320 con 0x7fc1ec071d00 2024-04-20T21:29:34.221 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' removed 2024-04-20T21:29:34.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.226+0000 7fc1f2200640 1 -- 172.21.15.86:0/2469902017 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc1c806c8f0 msgr2=0x7fc1c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:34.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.226+0000 7fc1f2200640 1 --2- 172.21.15.86:0/2469902017 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc1c806c8f0 0x7fc1c806edb0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec0b3640 tx=0x7fc1d4008040 comp rx=0 tx=0).stop 2024-04-20T21:29:34.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.226+0000 7fc1f2200640 1 -- 172.21.15.86:0/2469902017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1ec071d00 msgr2=0x7fc1ec13eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:34.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.226+0000 7fc1f2200640 1 --2- 172.21.15.86:0/2469902017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1ec071d00 0x7fc1ec13eda0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc000c00 tx=0x7fc1dc0046c0 comp rx=0 tx=0).stop 2024-04-20T21:29:34.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.226+0000 7fc1f2200640 1 -- 172.21.15.86:0/2469902017 shutdown_connections 2024-04-20T21:29:34.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.226+0000 7fc1f2200640 1 --2- 172.21.15.86:0/2469902017 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc1c806c8f0 0x7fc1c806edb0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:34.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.226+0000 7fc1f2200640 1 --2- 172.21.15.86:0/2469902017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1ec145fd0 0x7fc1ec0b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:34.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.226+0000 7fc1f2200640 1 --2- 172.21.15.86:0/2469902017 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc1ec0720e0 0x7fc1ec139c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:34.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.226+0000 7fc1f2200640 1 --2- 172.21.15.86:0/2469902017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc1ec071d00 0x7fc1ec13eda0 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:34.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.226+0000 7fc1f2200640 1 -- 172.21.15.86:0/2469902017 >> 172.21.15.86:0/2469902017 conn(0x7fc1ec0b2180 msgr2=0x7fc1ec143a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:34.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.226+0000 7fc1f2200640 1 -- 172.21.15.86:0/2469902017 shutdown_connections 2024-04-20T21:29:34.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.227+0000 7fc1f2200640 1 -- 172.21.15.86:0/2469902017 wait complete. 2024-04-20T21:29:34.250 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-20T21:29:34.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.446+0000 7fdfd9e00640 1 Processor -- start 2024-04-20T21:29:34.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.446+0000 7fdfd9e00640 1 -- start start 2024-04-20T21:29:34.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.448+0000 7fdfd9e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfd40b4530 0x7fdfd4169550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:34.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.448+0000 7fdfd9e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfd40adb10 con 0x7fdfd415eb30 2024-04-20T21:29:34.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.448+0000 7fdfd9e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfd40addb0 con 0x7fdfd40aee50 2024-04-20T21:29:34.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.448+0000 7fdfd9e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdfd40af7d0 con 0x7fdfd40b4530 2024-04-20T21:29:34.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.448+0000 7fdfd2a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fdfd415eb30 0x7fdfd40b3e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51914/0 (socket says 172.21.15.86:51914) 2024-04-20T21:29:34.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.448+0000 7fdfd2a00640 1 -- 172.21.15.86:0/3275554591 learned_addr learned my addr 172.21.15.86:0/3275554591 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:29:34.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.448+0000 7fdfd1600640 1 -- 172.21.15.86:0/3275554591 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1050105780 0 0) 0x7fdfd40adb10 con 0x7fdfd415eb30 2024-04-20T21:29:34.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.449+0000 7fdfd1600640 1 -- 172.21.15.86:0/3275554591 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdfb8003520 con 0x7fdfd415eb30 2024-04-20T21:29:34.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.449+0000 7fdfd3400640 1 --2- 172.21.15.86:0/3275554591 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfd40b4530 0x7fdfd4169550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:34.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.449+0000 7fdfd1600640 1 -- 172.21.15.86:0/3275554591 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 80855062 0 0) 0x7fdfb8003520 con 0x7fdfd415eb30 2024-04-20T21:29:34.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.449+0000 7fdfd1600640 1 -- 172.21.15.86:0/3275554591 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfd40b4530 msgr2=0x7fdfd4169550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:34.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.449+0000 7fdfd1600640 1 --2- 172.21.15.86:0/3275554591 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfd40b4530 0x7fdfd4169550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:34.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.449+0000 7fdfd1600640 1 -- 172.21.15.86:0/3275554591 >> v1:172.21.15.144:6789/0 conn(0x7fdfd40aee50 legacy=0x7fdfd406b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:34.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.449+0000 7fdfd1600640 1 -- 172.21.15.86:0/3275554591 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfd40ae090 con 0x7fdfd415eb30 2024-04-20T21:29:34.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.449+0000 7fdfd1600640 1 -- 172.21.15.86:0/3275554591 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fdfc4001670 con 0x7fdfd415eb30 2024-04-20T21:29:34.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.449+0000 7fdfd1600640 1 -- 172.21.15.86:0/3275554591 >> v1:172.21.15.86:6789/0 conn(0x7fdfd415eb30 legacy=0x7fdfd40b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:34.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.449+0000 7fdfd1600640 1 --2- 172.21.15.86:0/3275554591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfb80039e0 0x7fdfb8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:34.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.449+0000 7fdfd1600640 1 --2- 172.21.15.86:0/3275554591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfb8004800 0x7fdfb8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:34.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.449+0000 7fdfd1600640 1 --2- 172.21.15.86:0/3275554591 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfd40b4530 0x7fdfb8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:34.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.449+0000 7fdfd1600640 1 -- 172.21.15.86:0/3275554591 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd40af7d0 con 0x7fdfb8004800 2024-04-20T21:29:34.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.449+0000 7fdfd1600640 1 -- 172.21.15.86:0/3275554591 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb800baf0 con 0x7fdfb80039e0 2024-04-20T21:29:34.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.449+0000 7fdfd1600640 1 -- 172.21.15.86:0/3275554591 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdfb800bcf0 con 0x7fdfd40b4530 2024-04-20T21:29:34.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.449+0000 7fdfd2000640 1 --2- 172.21.15.86:0/3275554591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfb8004800 0x7fdfb8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:34.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.450+0000 7fdfd2a00640 1 --2- 172.21.15.86:0/3275554591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfb80039e0 0x7fdfb8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:34.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.450+0000 7fdfd2000640 1 -- 172.21.15.86:0/3275554591 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfd40b4530 msgr2=0x7fdfb8005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:34.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.450+0000 7fdfd2000640 1 --2- 172.21.15.86:0/3275554591 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfd40b4530 0x7fdfb8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:34.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.450+0000 7fdfd2000640 1 -- 172.21.15.86:0/3275554591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfb80039e0 msgr2=0x7fdfb8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:34.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.450+0000 7fdfd2000640 1 --2- 172.21.15.86:0/3275554591 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfb80039e0 0x7fdfb8003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:34.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.450+0000 7fdfd2000640 1 -- 172.21.15.86:0/3275554591 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfb8003520 con 0x7fdfb8004800 2024-04-20T21:29:34.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.451+0000 7fdfd2000640 1 --2- 172.21.15.86:0/3275554591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfb8004800 0x7fdfb8004c80 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0001750 tx=0x7fdfb8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=271b9e416ff7eed9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:34.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.451+0000 7fdfd1600640 1 -- 172.21.15.86:0/3275554591 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc0001370 con 0x7fdfb8004800 2024-04-20T21:29:34.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.451+0000 7fdfd1600640 1 -- 172.21.15.86:0/3275554591 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfc00052d0 con 0x7fdfb8004800 2024-04-20T21:29:34.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.451+0000 7fdfd1600640 1 -- 172.21.15.86:0/3275554591 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc0003830 con 0x7fdfb8004800 2024-04-20T21:29:34.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.452+0000 7fdfd9e00640 1 -- 172.21.15.86:0/3275554591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfb8004800 msgr2=0x7fdfb8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:34.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.452+0000 7fdfd9e00640 1 --2- 172.21.15.86:0/3275554591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfb8004800 0x7fdfb8004c80 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0001750 tx=0x7fdfb8005830 comp rx=0 tx=0).stop 2024-04-20T21:29:34.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.452+0000 7fdfd3400640 1 -- 172.21.15.86:0/3275554591 reap_dead start 2024-04-20T21:29:34.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.452+0000 7fdfd9e00640 1 -- 172.21.15.86:0/3275554591 shutdown_connections 2024-04-20T21:29:34.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.452+0000 7fdfd9e00640 1 -- 172.21.15.86:0/3275554591 >> 172.21.15.86:0/3275554591 conn(0x7fdfd40b2110 msgr2=0x7fdfd40b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:34.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.452+0000 7fdfd9e00640 1 -- 172.21.15.86:0/3275554591 shutdown_connections 2024-04-20T21:29:34.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.452+0000 7fdfd9e00640 1 -- 172.21.15.86:0/3275554591 wait complete. 2024-04-20T21:29:34.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.454+0000 7fdfd9e00640 1 Processor -- start 2024-04-20T21:29:34.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.454+0000 7fdfd9e00640 1 -- start start 2024-04-20T21:29:34.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.455+0000 7fdfd9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfb8004800 0x7fdfd4156890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:34.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.455+0000 7fdfd9e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfd41f5690 0x7fdfd40cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:34.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.455+0000 7fdfd9e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfd40ce1e0 0x7fdfd415a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:34.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.455+0000 7fdfd9e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd40ae930 con 0x7fdfb8004800 2024-04-20T21:29:34.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.455+0000 7fdfd9e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd40ae650 con 0x7fdfd41f5690 2024-04-20T21:29:34.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.455+0000 7fdfd9e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdfd406c090 con 0x7fdfd40ce1e0 2024-04-20T21:29:34.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.455+0000 7fdfd2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfb8004800 0x7fdfd4156890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:34.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.455+0000 7fdfd2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfb8004800 0x7fdfd4156890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43906/0 (socket says 172.21.15.86:43906) 2024-04-20T21:29:34.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.455+0000 7fdfd2a00640 1 -- 172.21.15.86:0/1970367429 learned_addr learned my addr 172.21.15.86:0/1970367429 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:34.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.455+0000 7fdfd2a00640 1 -- 172.21.15.86:0/1970367429 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfd40ce1e0 msgr2=0x7fdfd415a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:34.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.455+0000 7fdfd2a00640 1 --2- 172.21.15.86:0/1970367429 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfd40ce1e0 0x7fdfd415a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:34.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.455+0000 7fdfd2000640 1 --2- 172.21.15.86:0/1970367429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfd41f5690 0x7fdfd40cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:34.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.455+0000 7fdfd2a00640 1 -- 172.21.15.86:0/1970367429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfd41f5690 msgr2=0x7fdfd40cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:34.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.455+0000 7fdfd2a00640 1 --2- 172.21.15.86:0/1970367429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfd41f5690 0x7fdfd40cdca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:34.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.455+0000 7fdfd2a00640 1 -- 172.21.15.86:0/1970367429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfd4157190 con 0x7fdfb8004800 2024-04-20T21:29:34.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.456+0000 7fdfd2a00640 1 --2- 172.21.15.86:0/1970367429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfb8004800 0x7fdfd4156890 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fdfc400e850 tx=0x7fdfc400ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:34.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.456+0000 7fdfcb400640 1 -- 172.21.15.86:0/1970367429 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc400c870 con 0x7fdfb8004800 2024-04-20T21:29:34.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.456+0000 7fdfcb400640 1 -- 172.21.15.86:0/1970367429 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfc4018070 con 0x7fdfb8004800 2024-04-20T21:29:34.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.456+0000 7fdfcb400640 1 -- 172.21.15.86:0/1970367429 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc400cc50 con 0x7fdfb8004800 2024-04-20T21:29:34.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.456+0000 7fdfd9e00640 1 -- 172.21.15.86:0/1970367429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfd4157470 con 0x7fdfb8004800 2024-04-20T21:29:34.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.456+0000 7fdfd9e00640 1 -- 172.21.15.86:0/1970367429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfd4156dd0 con 0x7fdfb8004800 2024-04-20T21:29:34.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.457+0000 7fdfd9e00640 1 -- 172.21.15.86:0/1970367429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfd40689e0 con 0x7fdfb8004800 2024-04-20T21:29:34.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.458+0000 7fdfcb400640 1 -- 172.21.15.86:0/1970367429 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdfc401d460 con 0x7fdfb8004800 2024-04-20T21:29:34.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.459+0000 7fdfcb400640 1 --2- 172.21.15.86:0/1970367429 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdfb406c8f0 0x7fdfb406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:34.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.459+0000 7fdfcb400640 1 -- 172.21.15.86:0/1970367429 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5029+0+0 (secure 0 0 0) 0x7fdfc408b840 con 0x7fdfb8004800 2024-04-20T21:29:34.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.459+0000 7fdfd2000640 1 --2- 172.21.15.86:0/1970367429 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdfb406c8f0 0x7fdfb406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:34.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.460+0000 7fdfd2000640 1 --2- 172.21.15.86:0/1970367429 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdfb406c8f0 0x7fdfb406edb0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0033000 tx=0x7fdfc0009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:34.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.464+0000 7fdfcb400640 1 -- 172.21.15.86:0/1970367429 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fdfc4059f00 con 0x7fdfb8004800 2024-04-20T21:29:34.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.664+0000 7fdfd9e00640 1 -- 172.21.15.86:0/1970367429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fdfd415c670 con 0x7fdfb8004800 2024-04-20T21:29:34.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.671+0000 7fdfcb400640 1 -- 172.21.15.86:0/1970367429 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) ==== 104+0+0 (secure 0 0 0) 0x7fdfc4059800 con 0x7fdfb8004800 2024-04-20T21:29:34.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.674+0000 7fdfd9e00640 1 -- 172.21.15.86:0/1970367429 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdfb406c8f0 msgr2=0x7fdfb406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:34.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.674+0000 7fdfd9e00640 1 --2- 172.21.15.86:0/1970367429 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdfb406c8f0 0x7fdfb406edb0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0033000 tx=0x7fdfc0009030 comp rx=0 tx=0).stop 2024-04-20T21:29:34.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.674+0000 7fdfd9e00640 1 -- 172.21.15.86:0/1970367429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfb8004800 msgr2=0x7fdfd4156890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:34.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.674+0000 7fdfd9e00640 1 --2- 172.21.15.86:0/1970367429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfb8004800 0x7fdfd4156890 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fdfc400e850 tx=0x7fdfc400ed10 comp rx=0 tx=0).stop 2024-04-20T21:29:34.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.674+0000 7fdfd9e00640 1 -- 172.21.15.86:0/1970367429 shutdown_connections 2024-04-20T21:29:34.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.674+0000 7fdfd9e00640 1 --2- 172.21.15.86:0/1970367429 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdfb406c8f0 0x7fdfb406edb0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:34.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.674+0000 7fdfd9e00640 1 --2- 172.21.15.86:0/1970367429 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdfd40ce1e0 0x7fdfd415a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:34.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.674+0000 7fdfd9e00640 1 --2- 172.21.15.86:0/1970367429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdfd41f5690 0x7fdfd40cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:34.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.674+0000 7fdfd9e00640 1 --2- 172.21.15.86:0/1970367429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfb8004800 0x7fdfd4156890 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:34.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.674+0000 7fdfd9e00640 1 -- 172.21.15.86:0/1970367429 >> 172.21.15.86:0/1970367429 conn(0x7fdfd40b2110 msgr2=0x7fdfd40d0420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:34.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.674+0000 7fdfd9e00640 1 -- 172.21.15.86:0/1970367429 shutdown_connections 2024-04-20T21:29:34.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.674+0000 7fdfd9e00640 1 -- 172.21.15.86:0/1970367429 wait complete. 2024-04-20T21:29:34.691 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-04-20T21:29:34.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.893+0000 7fbe9b800640 1 Processor -- start 2024-04-20T21:29:34.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.893+0000 7fbe9b800640 1 -- start start 2024-04-20T21:29:34.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.894+0000 7fbe9b800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbe940cd170 0x7fbe9406ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:34.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.894+0000 7fbe9b800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbe940cd600 0x7fbe9406f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:34.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.894+0000 7fbe9b800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe940cdb50 con 0x7fbe94144e90 2024-04-20T21:29:34.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.894+0000 7fbe9b800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fbe94145270 con 0x7fbe940cd600 2024-04-20T21:29:34.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.894+0000 7fbe9b800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fbe940b3e50 con 0x7fbe940cd170 2024-04-20T21:29:34.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.895+0000 7fbe99800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbe940cd600 0x7fbe9406f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:34.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.895+0000 7fbe99800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbe940cd600 0x7fbe9406f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43914/0 (socket says 172.21.15.86:43914) 2024-04-20T21:29:34.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.895+0000 7fbe99800640 1 -- 172.21.15.86:0/2159766413 learned_addr learned my addr 172.21.15.86:0/2159766413 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:34.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.895+0000 7fbe93e00640 1 --2- 172.21.15.86:0/2159766413 >> v2:172.21.15.144:3300/0 conn(0x7fbe940cd170 0x7fbe9406ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:34.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.895+0000 7fbe99800640 1 -- 172.21.15.86:0/2159766413 >> v2:172.21.15.144:3300/0 conn(0x7fbe940cd170 msgr2=0x7fbe9406ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:34.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.895+0000 7fbe99800640 1 --2- 172.21.15.86:0/2159766413 >> v2:172.21.15.144:3300/0 conn(0x7fbe940cd170 0x7fbe9406ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:34.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.895+0000 7fbe99800640 1 -- 172.21.15.86:0/2159766413 >> v1:172.21.15.144:6789/0 conn(0x7fbe94144e90 legacy=0x7fbe940b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:34.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.895+0000 7fbe99800640 1 -- 172.21.15.86:0/2159766413 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe940b4130 con 0x7fbe940cd600 2024-04-20T21:29:34.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.897+0000 7fbe99800640 1 --2- 172.21.15.86:0/2159766413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe940cd600 0x7fbe9406f280 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7fbe9406fa20 tx=0x7fbe8c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f6e6970d5e01207 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:34.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.897+0000 7fbe93400640 1 -- 172.21.15.86:0/2159766413 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c02e070 con 0x7fbe940cd600 2024-04-20T21:29:34.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.897+0000 7fbe93400640 1 -- 172.21.15.86:0/2159766413 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe8c037070 con 0x7fbe940cd600 2024-04-20T21:29:34.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.897+0000 7fbe93400640 1 -- 172.21.15.86:0/2159766413 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c031a00 con 0x7fbe940cd600 2024-04-20T21:29:34.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.898+0000 7fbe9b800640 1 -- 172.21.15.86:0/2159766413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe940cd600 msgr2=0x7fbe9406f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:34.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.898+0000 7fbe9b800640 1 --2- 172.21.15.86:0/2159766413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe940cd600 0x7fbe9406f280 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7fbe9406fa20 tx=0x7fbe8c02b880 comp rx=0 tx=0).stop 2024-04-20T21:29:34.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.898+0000 7fbe9b800640 1 -- 172.21.15.86:0/2159766413 shutdown_connections 2024-04-20T21:29:34.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.898+0000 7fbe9b800640 1 --2- 172.21.15.86:0/2159766413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe940cd600 0x7fbe9406f280 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:34.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.898+0000 7fbe9b800640 1 --2- 172.21.15.86:0/2159766413 >> v2:172.21.15.144:3300/0 conn(0x7fbe940cd170 0x7fbe9406ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:34.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.898+0000 7fbe9b800640 1 -- 172.21.15.86:0/2159766413 >> 172.21.15.86:0/2159766413 conn(0x7fbe940b2360 msgr2=0x7fbe940b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:34.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.898+0000 7fbe9b800640 1 -- 172.21.15.86:0/2159766413 shutdown_connections 2024-04-20T21:29:34.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.898+0000 7fbe9b800640 1 -- 172.21.15.86:0/2159766413 wait complete. 2024-04-20T21:29:34.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.900+0000 7fbe9b800640 1 Processor -- start 2024-04-20T21:29:34.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.900+0000 7fbe9b800640 1 -- start start 2024-04-20T21:29:34.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.901+0000 7fbe9b800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe940cd170 0x7fbe9415b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:34.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.901+0000 7fbe9b800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe940cd600 0x7fbe940a9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:34.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.901+0000 7fbe9b800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe94144e90 0x7fbe940aad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:34.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.901+0000 7fbe9b800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbe940d0190 con 0x7fbe94144e90 2024-04-20T21:29:34.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.901+0000 7fbe9b800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbe940d0010 con 0x7fbe940cd170 2024-04-20T21:29:34.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.901+0000 7fbe9b800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbe9415bb50 con 0x7fbe940cd600 2024-04-20T21:29:34.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.901+0000 7fbe99800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe94144e90 0x7fbe940aad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:34.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.901+0000 7fbe99800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe94144e90 0x7fbe940aad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43916/0 (socket says 172.21.15.86:43916) 2024-04-20T21:29:34.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.901+0000 7fbe99800640 1 -- 172.21.15.86:0/2390142210 learned_addr learned my addr 172.21.15.86:0/2390142210 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:34.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.902+0000 7fbe98e00640 1 --2- 172.21.15.86:0/2390142210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe940cd170 0x7fbe9415b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:34.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.902+0000 7fbe93e00640 1 --2- 172.21.15.86:0/2390142210 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe940cd600 0x7fbe940a9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:34.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.902+0000 7fbe99800640 1 -- 172.21.15.86:0/2390142210 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe940cd600 msgr2=0x7fbe940a9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:34.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.902+0000 7fbe99800640 1 --2- 172.21.15.86:0/2390142210 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe940cd600 0x7fbe940a9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:34.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.902+0000 7fbe99800640 1 -- 172.21.15.86:0/2390142210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe940cd170 msgr2=0x7fbe9415b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:34.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.902+0000 7fbe99800640 1 --2- 172.21.15.86:0/2390142210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe940cd170 0x7fbe9415b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:34.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.902+0000 7fbe99800640 1 -- 172.21.15.86:0/2390142210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe940a9d40 con 0x7fbe94144e90 2024-04-20T21:29:34.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.902+0000 7fbe93e00640 1 --2- 172.21.15.86:0/2390142210 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe940cd600 0x7fbe940a9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:29:34.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.902+0000 7fbe99800640 1 --2- 172.21.15.86:0/2390142210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe94144e90 0x7fbe940aad70 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c02b8c0 tx=0x7fbe8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:34.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.902+0000 7fbe90c00640 1 -- 172.21.15.86:0/2390142210 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c032670 con 0x7fbe94144e90 2024-04-20T21:29:34.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.902+0000 7fbe90c00640 1 -- 172.21.15.86:0/2390142210 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe8c032810 con 0x7fbe94144e90 2024-04-20T21:29:34.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.902+0000 7fbe9b800640 1 -- 172.21.15.86:0/2390142210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe940aa020 con 0x7fbe94144e90 2024-04-20T21:29:34.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.902+0000 7fbe90c00640 1 -- 172.21.15.86:0/2390142210 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c032c00 con 0x7fbe94144e90 2024-04-20T21:29:34.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.902+0000 7fbe9b800640 1 -- 172.21.15.86:0/2390142210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe940aa5e0 con 0x7fbe94144e90 2024-04-20T21:29:34.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.904+0000 7fbe9b800640 1 -- 172.21.15.86:0/2390142210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe940689e0 con 0x7fbe94144e90 2024-04-20T21:29:34.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.905+0000 7fbe90c00640 1 -- 172.21.15.86:0/2390142210 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbe8c04d060 con 0x7fbe94144e90 2024-04-20T21:29:34.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.906+0000 7fbe90c00640 1 --2- 172.21.15.86:0/2390142210 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbe7006c920 0x7fbe7006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:34.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.906+0000 7fbe90c00640 1 -- 172.21.15.86:0/2390142210 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5029+0+0 (secure 0 0 0) 0x7fbe8c0ad8b0 con 0x7fbe94144e90 2024-04-20T21:29:34.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.906+0000 7fbe98e00640 1 --2- 172.21.15.86:0/2390142210 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbe7006c920 0x7fbe7006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:34.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.906+0000 7fbe98e00640 1 --2- 172.21.15.86:0/2390142210 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbe7006c920 0x7fbe7006ede0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fbe940cb980 tx=0x7fbe840094f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:34.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:34.910+0000 7fbe90c00640 1 -- 172.21.15.86:0/2390142210 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbe8c040070 con 0x7fbe94144e90 2024-04-20T21:29:35.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.168+0000 7fbe9b800640 1 -- 172.21.15.86:0/2390142210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) -- 0x7fbe940b00f0 con 0x7fbe94144e90 2024-04-20T21:29:35.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.226+0000 7fbe90c00640 1 -- 172.21.15.86:0/2390142210 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v555) ==== 146+0+0 (secure 0 0 0) 0x7fbe8c07bf80 con 0x7fbe94144e90 2024-04-20T21:29:35.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.228+0000 7fbe9b800640 1 -- 172.21.15.86:0/2390142210 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbe7006c920 msgr2=0x7fbe7006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:35.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.228+0000 7fbe9b800640 1 --2- 172.21.15.86:0/2390142210 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbe7006c920 0x7fbe7006ede0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fbe940cb980 tx=0x7fbe840094f0 comp rx=0 tx=0).stop 2024-04-20T21:29:35.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.228+0000 7fbe9b800640 1 -- 172.21.15.86:0/2390142210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe94144e90 msgr2=0x7fbe940aad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:35.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.228+0000 7fbe9b800640 1 --2- 172.21.15.86:0/2390142210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe94144e90 0x7fbe940aad70 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c02b8c0 tx=0x7fbe8c0046c0 comp rx=0 tx=0).stop 2024-04-20T21:29:35.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.228+0000 7fbe9b800640 1 -- 172.21.15.86:0/2390142210 shutdown_connections 2024-04-20T21:29:35.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.228+0000 7fbe9b800640 1 --2- 172.21.15.86:0/2390142210 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbe7006c920 0x7fbe7006ede0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:35.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.228+0000 7fbe9b800640 1 --2- 172.21.15.86:0/2390142210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe94144e90 0x7fbe940aad70 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:35.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.228+0000 7fbe9b800640 1 --2- 172.21.15.86:0/2390142210 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbe940cd600 0x7fbe940a9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:35.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.228+0000 7fbe9b800640 1 --2- 172.21.15.86:0/2390142210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbe940cd170 0x7fbe9415b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:35.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.228+0000 7fbe9b800640 1 -- 172.21.15.86:0/2390142210 >> 172.21.15.86:0/2390142210 conn(0x7fbe940b2360 msgr2=0x7fbe940b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:35.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.229+0000 7fbe9b800640 1 -- 172.21.15.86:0/2390142210 shutdown_connections 2024-04-20T21:29:35.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.229+0000 7fbe9b800640 1 -- 172.21.15.86:0/2390142210 wait complete. 2024-04-20T21:29:35.247 INFO:tasks.cephfs_test_runner:test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-04-20T21:29:35.247 INFO:tasks.cephfs_test_runner:Starting test: test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-20T21:29:35.247 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:29:35.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.430+0000 7f3e1b600640 1 Processor -- start 2024-04-20T21:29:35.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.430+0000 7f3e1b600640 1 -- start start 2024-04-20T21:29:35.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.432+0000 7f3e1b600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3e14144e90 0x7f3e1415ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:35.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.432+0000 7f3e1b600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3e1406f140 0x7f3e140aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:35.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.432+0000 7f3e1b600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e140af0b0 0x7f3e14071d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:35.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.432+0000 7f3e1b600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f3e14145270 con 0x7f3e1406f140 2024-04-20T21:29:35.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.432+0000 7f3e1b600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f3e1406dc30 con 0x7f3e14144e90 2024-04-20T21:29:35.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.432+0000 7f3e1b600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3e1415f720 con 0x7f3e140af0b0 2024-04-20T21:29:35.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.432+0000 7f3e13e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3e1406f140 0x7f3e140aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:35.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.432+0000 7f3e13e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3e1406f140 0x7f3e140aeb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43930/0 (socket says 172.21.15.86:43930) 2024-04-20T21:29:35.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.432+0000 7f3e13e00640 1 -- 172.21.15.86:0/3541118207 learned_addr learned my addr 172.21.15.86:0/3541118207 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:35.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.432+0000 7f3e19600640 1 --2- 172.21.15.86:0/3541118207 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e140af0b0 0x7f3e14071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:35.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.433+0000 7f3e18c00640 1 --2- 172.21.15.86:0/3541118207 >> v2:172.21.15.144:3300/0 conn(0x7f3e14144e90 0x7f3e1415ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:35.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.433+0000 7f3e13e00640 1 -- 172.21.15.86:0/3541118207 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e140af0b0 msgr2=0x7f3e14071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:35.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.433+0000 7f3e13e00640 1 --2- 172.21.15.86:0/3541118207 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e140af0b0 0x7f3e14071d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:35.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.433+0000 7f3e13e00640 1 -- 172.21.15.86:0/3541118207 >> v2:172.21.15.144:3300/0 conn(0x7f3e14144e90 msgr2=0x7f3e1415ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:35.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.433+0000 7f3e13e00640 1 --2- 172.21.15.86:0/3541118207 >> v2:172.21.15.144:3300/0 conn(0x7f3e14144e90 0x7f3e1415ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:35.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.433+0000 7f3e13e00640 1 -- 172.21.15.86:0/3541118207 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e1415fa00 con 0x7f3e1406f140 2024-04-20T21:29:35.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.433+0000 7f3e18c00640 1 --2- 172.21.15.86:0/3541118207 >> v2:172.21.15.144:3300/0 conn(0x7f3e14144e90 0x7f3e1415ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:29:35.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.433+0000 7f3e19600640 1 --2- 172.21.15.86:0/3541118207 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e140af0b0 0x7f3e14071d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:29:35.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.434+0000 7f3e13e00640 1 --2- 172.21.15.86:0/3541118207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e1406f140 0x7f3e140aeb70 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f3e1406dbb0 tx=0x7f3e0002b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=288fd32ed2a483d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:35.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.434+0000 7f3e13400640 1 -- 172.21.15.86:0/3541118207 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e00035020 con 0x7f3e1406f140 2024-04-20T21:29:35.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.434+0000 7f3e13400640 1 -- 172.21.15.86:0/3541118207 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e00031880 con 0x7f3e1406f140 2024-04-20T21:29:35.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.434+0000 7f3e13400640 1 -- 172.21.15.86:0/3541118207 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e00031c50 con 0x7f3e1406f140 2024-04-20T21:29:35.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.435+0000 7f3e1b600640 1 -- 172.21.15.86:0/3541118207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e1406f140 msgr2=0x7f3e140aeb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:35.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.435+0000 7f3e1b600640 1 --2- 172.21.15.86:0/3541118207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e1406f140 0x7f3e140aeb70 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f3e1406dbb0 tx=0x7f3e0002b920 comp rx=0 tx=0).stop 2024-04-20T21:29:35.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.435+0000 7f3e1b600640 1 -- 172.21.15.86:0/3541118207 shutdown_connections 2024-04-20T21:29:35.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.435+0000 7f3e1b600640 1 --2- 172.21.15.86:0/3541118207 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e140af0b0 0x7f3e14071d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:35.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.435+0000 7f3e1b600640 1 --2- 172.21.15.86:0/3541118207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e1406f140 0x7f3e140aeb70 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:35.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.435+0000 7f3e1b600640 1 --2- 172.21.15.86:0/3541118207 >> v2:172.21.15.144:3300/0 conn(0x7f3e14144e90 0x7f3e1415ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:35.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.435+0000 7f3e1b600640 1 -- 172.21.15.86:0/3541118207 >> 172.21.15.86:0/3541118207 conn(0x7f3e140b2360 msgr2=0x7f3e140b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:35.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.435+0000 7f3e1b600640 1 -- 172.21.15.86:0/3541118207 shutdown_connections 2024-04-20T21:29:35.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.435+0000 7f3e1b600640 1 -- 172.21.15.86:0/3541118207 wait complete. 2024-04-20T21:29:35.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.437+0000 7f3e1b600640 1 Processor -- start 2024-04-20T21:29:35.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.437+0000 7f3e1b600640 1 -- start start 2024-04-20T21:29:35.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.438+0000 7f3e1b600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e1406f140 0x7f3e140ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:35.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.438+0000 7f3e1b600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e140af0b0 0x7f3e140cd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:35.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.438+0000 7f3e1b600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e14144e90 0x7f3e141611e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:35.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.438+0000 7f3e1b600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1415f350 con 0x7f3e14144e90 2024-04-20T21:29:35.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.438+0000 7f3e1b600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1406ded0 con 0x7f3e1406f140 2024-04-20T21:29:35.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.438+0000 7f3e1b600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3e1415f4d0 con 0x7f3e140af0b0 2024-04-20T21:29:35.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.438+0000 7f3e19600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e14144e90 0x7f3e141611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:35.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.438+0000 7f3e19600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e14144e90 0x7f3e141611e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43940/0 (socket says 172.21.15.86:43940) 2024-04-20T21:29:35.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.438+0000 7f3e19600640 1 -- 172.21.15.86:0/3524553696 learned_addr learned my addr 172.21.15.86:0/3524553696 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:35.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.439+0000 7f3e13e00640 1 --2- 172.21.15.86:0/3524553696 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e140af0b0 0x7f3e140cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:35.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.439+0000 7f3e18c00640 1 --2- 172.21.15.86:0/3524553696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e1406f140 0x7f3e140ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:35.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.439+0000 7f3e19600640 1 -- 172.21.15.86:0/3524553696 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e140af0b0 msgr2=0x7f3e140cd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:35.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.439+0000 7f3e19600640 1 --2- 172.21.15.86:0/3524553696 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e140af0b0 0x7f3e140cd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:35.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.439+0000 7f3e19600640 1 -- 172.21.15.86:0/3524553696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e1406f140 msgr2=0x7f3e140ce7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:35.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.439+0000 7f3e19600640 1 --2- 172.21.15.86:0/3524553696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e1406f140 0x7f3e140ce7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:35.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.439+0000 7f3e19600640 1 -- 172.21.15.86:0/3524553696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e14142520 con 0x7f3e14144e90 2024-04-20T21:29:35.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.439+0000 7f3e13e00640 1 --2- 172.21.15.86:0/3524553696 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e140af0b0 0x7f3e140cd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:29:35.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.439+0000 7f3e19600640 1 --2- 172.21.15.86:0/3524553696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e14144e90 0x7f3e141611e0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c00cd50 tx=0x7f3e0c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:35.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.439+0000 7f3e10c00640 1 -- 172.21.15.86:0/3524553696 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c00f680 con 0x7f3e14144e90 2024-04-20T21:29:35.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.439+0000 7f3e10c00640 1 -- 172.21.15.86:0/3524553696 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e0c00f840 con 0x7f3e14144e90 2024-04-20T21:29:35.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.439+0000 7f3e10c00640 1 -- 172.21.15.86:0/3524553696 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c00fc30 con 0x7f3e14144e90 2024-04-20T21:29:35.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.439+0000 7f3e1b600640 1 -- 172.21.15.86:0/3524553696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e14142860 con 0x7f3e14144e90 2024-04-20T21:29:35.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.439+0000 7f3e1b600640 1 -- 172.21.15.86:0/3524553696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e140cddb0 con 0x7f3e14144e90 2024-04-20T21:29:35.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.441+0000 7f3e1b600640 1 -- 172.21.15.86:0/3524553696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e140689e0 con 0x7f3e14144e90 2024-04-20T21:29:35.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.441+0000 7f3e10c00640 1 -- 172.21.15.86:0/3524553696 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3e0c00e3e0 con 0x7f3e14144e90 2024-04-20T21:29:35.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.442+0000 7f3e10c00640 1 --2- 172.21.15.86:0/3524553696 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3df406c8f0 0x7f3df406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:35.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.442+0000 7f3e10c00640 1 -- 172.21.15.86:0/3524553696 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5029+0+0 (secure 0 0 0) 0x7f3e0c08c600 con 0x7f3e14144e90 2024-04-20T21:29:35.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.443+0000 7f3e18c00640 1 --2- 172.21.15.86:0/3524553696 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3df406c8f0 0x7f3df406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:35.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.443+0000 7f3e18c00640 1 --2- 172.21.15.86:0/3524553696 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3df406c8f0 0x7f3df406edb0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f3e0800a8d0 tx=0x7f3e08008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:35.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.447+0000 7f3e10c00640 1 -- 172.21.15.86:0/3524553696 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3e14142860 con 0x7f3e14144e90 2024-04-20T21:29:35.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.666+0000 7f3e1b600640 1 -- 172.21.15.86:0/3524553696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3e140b28e0 con 0x7f3e14144e90 2024-04-20T21:29:35.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.668+0000 7f3e10c00640 1 -- 172.21.15.86:0/3524553696 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) ==== 74+0+13152 (secure 0 0 0) 0x7f3e0c014030 con 0x7f3e14144e90 2024-04-20T21:29:35.667 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:29:35.667 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":104,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:29:34.219088+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6836/4271041811":"2024-04-21T21:29:25.141228+0000","172.21.15.86:6834/4271041811":"2024-04-21T21:29:25.141228+0000","172.21.15.86:6836/1331851138":"2024-04-21T21:28:47.841886+0000","172.21.15.86:6834/1331851138":"2024-04-21T21:28:47.841886+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:29:35.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.671+0000 7f3e1b600640 1 -- 172.21.15.86:0/3524553696 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3df406c8f0 msgr2=0x7f3df406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:35.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.671+0000 7f3e1b600640 1 --2- 172.21.15.86:0/3524553696 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3df406c8f0 0x7f3df406edb0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f3e0800a8d0 tx=0x7f3e08008040 comp rx=0 tx=0).stop 2024-04-20T21:29:35.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.671+0000 7f3e1b600640 1 -- 172.21.15.86:0/3524553696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e14144e90 msgr2=0x7f3e141611e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:35.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.671+0000 7f3e1b600640 1 --2- 172.21.15.86:0/3524553696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e14144e90 0x7f3e141611e0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c00cd50 tx=0x7f3e0c00a650 comp rx=0 tx=0).stop 2024-04-20T21:29:35.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.671+0000 7f3e1b600640 1 -- 172.21.15.86:0/3524553696 shutdown_connections 2024-04-20T21:29:35.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.671+0000 7f3e1b600640 1 --2- 172.21.15.86:0/3524553696 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3df406c8f0 0x7f3df406edb0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:35.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.672+0000 7f3e1b600640 1 --2- 172.21.15.86:0/3524553696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3e14144e90 0x7f3e141611e0 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:35.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.672+0000 7f3e1b600640 1 --2- 172.21.15.86:0/3524553696 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3e140af0b0 0x7f3e140cd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:35.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.672+0000 7f3e1b600640 1 --2- 172.21.15.86:0/3524553696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e1406f140 0x7f3e140ce7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:35.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.672+0000 7f3e1b600640 1 -- 172.21.15.86:0/3524553696 >> 172.21.15.86:0/3524553696 conn(0x7f3e140b2360 msgr2=0x7f3e1406e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:35.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.672+0000 7f3e1b600640 1 -- 172.21.15.86:0/3524553696 shutdown_connections 2024-04-20T21:29:35.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.672+0000 7f3e1b600640 1 -- 172.21.15.86:0/3524553696 wait complete. 2024-04-20T21:29:35.689 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:29:35.690 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:29:35.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.884+0000 7fdeba800640 1 Processor -- start 2024-04-20T21:29:35.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.884+0000 7fdeba800640 1 -- start start 2024-04-20T21:29:35.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.886+0000 7fdeba800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdeb406bbf0 0x7fdeb406e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:35.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.886+0000 7fdeba800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdeb406bfd0 0x7fdeb4145b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:35.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.886+0000 7fdeba800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdeb415cd40 con 0x7fdeb4146040 2024-04-20T21:29:35.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.886+0000 7fdeba800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fdeb41464b0 con 0x7fdeb406bbf0 2024-04-20T21:29:35.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.886+0000 7fdeba800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fdeb415d000 con 0x7fdeb406bfd0 2024-04-20T21:29:35.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.886+0000 7fdeb3400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdeb406bbf0 0x7fdeb406e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:35.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.886+0000 7fdeb3400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdeb406bbf0 0x7fdeb406e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58416/0 (socket says 172.21.15.86:58416) 2024-04-20T21:29:35.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.886+0000 7fdeb3400640 1 -- 172.21.15.86:0/646381916 learned_addr learned my addr 172.21.15.86:0/646381916 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:35.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.887+0000 7fdeb2a00640 1 --2- 172.21.15.86:0/646381916 >> v2:172.21.15.144:3300/0 conn(0x7fdeb406bfd0 0x7fdeb4145b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:35.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.887+0000 7fdeb3400640 1 -- 172.21.15.86:0/646381916 >> v2:172.21.15.144:3300/0 conn(0x7fdeb406bfd0 msgr2=0x7fdeb4145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:35.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.887+0000 7fdeb3400640 1 --2- 172.21.15.86:0/646381916 >> v2:172.21.15.144:3300/0 conn(0x7fdeb406bfd0 0x7fdeb4145b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:35.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.887+0000 7fdeb3400640 1 -- 172.21.15.86:0/646381916 >> v1:172.21.15.144:6789/0 conn(0x7fdeb4146040 legacy=0x7fdeb40cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:35.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.887+0000 7fdeb3400640 1 -- 172.21.15.86:0/646381916 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeb415d2e0 con 0x7fdeb406bbf0 2024-04-20T21:29:35.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.889+0000 7fdeb3400640 1 --2- 172.21.15.86:0/646381916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdeb406bbf0 0x7fdeb406e000 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fdeb406dcc0 tx=0x7fdea402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dce67bf0cc0f99be server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:35.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.889+0000 7fdeb2000640 1 -- 172.21.15.86:0/646381916 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea4035020 con 0x7fdeb406bbf0 2024-04-20T21:29:35.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.889+0000 7fdeb2000640 1 -- 172.21.15.86:0/646381916 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdea4031870 con 0x7fdeb406bbf0 2024-04-20T21:29:35.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.889+0000 7fdeb2000640 1 -- 172.21.15.86:0/646381916 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea4031c40 con 0x7fdeb406bbf0 2024-04-20T21:29:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.890+0000 7fdeba800640 1 -- 172.21.15.86:0/646381916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdeb406bbf0 msgr2=0x7fdeb406e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.890+0000 7fdeba800640 1 --2- 172.21.15.86:0/646381916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdeb406bbf0 0x7fdeb406e000 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fdeb406dcc0 tx=0x7fdea402b910 comp rx=0 tx=0).stop 2024-04-20T21:29:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.890+0000 7fdeba800640 1 -- 172.21.15.86:0/646381916 shutdown_connections 2024-04-20T21:29:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.890+0000 7fdeba800640 1 --2- 172.21.15.86:0/646381916 >> v2:172.21.15.144:3300/0 conn(0x7fdeb406bfd0 0x7fdeb4145b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.890+0000 7fdeba800640 1 --2- 172.21.15.86:0/646381916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdeb406bbf0 0x7fdeb406e000 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.890+0000 7fdeba800640 1 -- 172.21.15.86:0/646381916 >> 172.21.15.86:0/646381916 conn(0x7fdeb40b2110 msgr2=0x7fdeb40b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.890+0000 7fdeba800640 1 -- 172.21.15.86:0/646381916 shutdown_connections 2024-04-20T21:29:35.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.890+0000 7fdeba800640 1 -- 172.21.15.86:0/646381916 wait complete. 2024-04-20T21:29:35.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.892+0000 7fdeba800640 1 Processor -- start 2024-04-20T21:29:35.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.892+0000 7fdeba800640 1 -- start start 2024-04-20T21:29:35.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.894+0000 7fdeba800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdeb406bbf0 0x7fdeb4142170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:35.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.894+0000 7fdeba800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdeb406bfd0 0x7fdeb40ca580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:35.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.894+0000 7fdeba800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdeb4146040 0x7fdeb41e7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:35.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.894+0000 7fdeba800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb40cdc50 con 0x7fdeb4146040 2024-04-20T21:29:35.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.894+0000 7fdeba800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb415d8a0 con 0x7fdeb406bbf0 2024-04-20T21:29:35.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.894+0000 7fdeba800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdeb40cddd0 con 0x7fdeb406bfd0 2024-04-20T21:29:35.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.894+0000 7fdeb3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdeb4146040 0x7fdeb41e7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:35.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.894+0000 7fdeb3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdeb4146040 0x7fdeb41e7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58422/0 (socket says 172.21.15.86:58422) 2024-04-20T21:29:35.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.894+0000 7fdeb3e00640 1 -- 172.21.15.86:0/739098771 learned_addr learned my addr 172.21.15.86:0/739098771 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:35.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.894+0000 7fdeb3400640 1 --2- 172.21.15.86:0/739098771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdeb406bbf0 0x7fdeb4142170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:35.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.894+0000 7fdeb3e00640 1 -- 172.21.15.86:0/739098771 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdeb406bfd0 msgr2=0x7fdeb40ca580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:35.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.894+0000 7fdeb3e00640 1 --2- 172.21.15.86:0/739098771 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdeb406bfd0 0x7fdeb40ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:35.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.894+0000 7fdeb3e00640 1 -- 172.21.15.86:0/739098771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdeb406bbf0 msgr2=0x7fdeb4142170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:35.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.894+0000 7fdeb3e00640 1 --2- 172.21.15.86:0/739098771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdeb406bbf0 0x7fdeb4142170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:35.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.894+0000 7fdeb3e00640 1 -- 172.21.15.86:0/739098771 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeb40cafd0 con 0x7fdeb4146040 2024-04-20T21:29:35.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.894+0000 7fdeb3400640 1 --2- 172.21.15.86:0/739098771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdeb406bbf0 0x7fdeb4142170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:29:35.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.895+0000 7fdeb3e00640 1 --2- 172.21.15.86:0/739098771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdeb4146040 0x7fdeb41e7f10 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fdeac00c7b0 tx=0x7fdeac00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:35.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.895+0000 7fdeab400640 1 -- 172.21.15.86:0/739098771 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeac004b60 con 0x7fdeb4146040 2024-04-20T21:29:35.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.895+0000 7fdeab400640 1 -- 172.21.15.86:0/739098771 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdeac004d00 con 0x7fdeb4146040 2024-04-20T21:29:35.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.895+0000 7fdeba800640 1 -- 172.21.15.86:0/739098771 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdeb40cb2b0 con 0x7fdeb4146040 2024-04-20T21:29:35.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.895+0000 7fdeab400640 1 -- 172.21.15.86:0/739098771 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeac00aa90 con 0x7fdeb4146040 2024-04-20T21:29:35.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.895+0000 7fdeba800640 1 -- 172.21.15.86:0/739098771 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdeb40cb870 con 0x7fdeb4146040 2024-04-20T21:29:35.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.896+0000 7fdeba800640 1 -- 172.21.15.86:0/739098771 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdeb40689e0 con 0x7fdeb4146040 2024-04-20T21:29:35.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.897+0000 7fdeab400640 1 -- 172.21.15.86:0/739098771 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdeac00ac30 con 0x7fdeb4146040 2024-04-20T21:29:35.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.898+0000 7fdeab400640 1 --2- 172.21.15.86:0/739098771 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fde9406c920 0x7fde9406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:35.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.898+0000 7fdeab400640 1 -- 172.21.15.86:0/739098771 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5029+0+0 (secure 0 0 0) 0x7fdeac08bc60 con 0x7fdeb4146040 2024-04-20T21:29:35.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.899+0000 7fdeb3400640 1 --2- 172.21.15.86:0/739098771 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fde9406c920 0x7fde9406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:35.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.900+0000 7fdeb3400640 1 --2- 172.21.15.86:0/739098771 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fde9406c920 0x7fde9406ede0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fdea4036000 tx=0x7fdea4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:35.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:35.903+0000 7fdeab400640 1 -- 172.21.15.86:0/739098771 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fdeac020070 con 0x7fdeb4146040 2024-04-20T21:29:36.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.100+0000 7fdeba800640 1 -- 172.21.15.86:0/739098771 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdeb406c780 con 0x7fdeb4146040 2024-04-20T21:29:36.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.100+0000 7fdeab400640 1 -- 172.21.15.86:0/739098771 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v104) ==== 93+0+10 (secure 0 0 0) 0x7fdeac05a240 con 0x7fdeb4146040 2024-04-20T21:29:36.099 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:29:36.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.103+0000 7fdeba800640 1 -- 172.21.15.86:0/739098771 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fde9406c920 msgr2=0x7fde9406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:36.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.103+0000 7fdeba800640 1 --2- 172.21.15.86:0/739098771 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fde9406c920 0x7fde9406ede0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fdea4036000 tx=0x7fdea4008030 comp rx=0 tx=0).stop 2024-04-20T21:29:36.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.103+0000 7fdeba800640 1 -- 172.21.15.86:0/739098771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdeb4146040 msgr2=0x7fdeb41e7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:36.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.103+0000 7fdeba800640 1 --2- 172.21.15.86:0/739098771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdeb4146040 0x7fdeb41e7f10 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fdeac00c7b0 tx=0x7fdeac00cc70 comp rx=0 tx=0).stop 2024-04-20T21:29:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.103+0000 7fdeba800640 1 -- 172.21.15.86:0/739098771 shutdown_connections 2024-04-20T21:29:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.103+0000 7fdeba800640 1 --2- 172.21.15.86:0/739098771 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fde9406c920 0x7fde9406ede0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.103+0000 7fdeba800640 1 --2- 172.21.15.86:0/739098771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdeb4146040 0x7fdeb41e7f10 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.103+0000 7fdeba800640 1 --2- 172.21.15.86:0/739098771 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdeb406bfd0 0x7fdeb40ca580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.103+0000 7fdeba800640 1 --2- 172.21.15.86:0/739098771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdeb406bbf0 0x7fdeb4142170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.103+0000 7fdeba800640 1 -- 172.21.15.86:0/739098771 >> 172.21.15.86:0/739098771 conn(0x7fdeb40b2110 msgr2=0x7fdeb415dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.103+0000 7fdeba800640 1 -- 172.21.15.86:0/739098771 shutdown_connections 2024-04-20T21:29:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.104+0000 7fdeba800640 1 -- 172.21.15.86:0/739098771 wait complete. 2024-04-20T21:29:36.120 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-04-20T21:29:36.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.308+0000 7f3c60400640 1 Processor -- start 2024-04-20T21:29:36.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.308+0000 7f3c60400640 1 -- start start 2024-04-20T21:29:36.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.310+0000 7f3c60400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3c58146130 0x7f3c5806e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:36.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.310+0000 7f3c60400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3c580ccdd0 0x7f3c580cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:36.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.310+0000 7f3c60400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c58071d00 0x7f3c58072190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:36.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.310+0000 7f3c60400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f3c5806dc30 con 0x7f3c580ccdd0 2024-04-20T21:29:36.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.310+0000 7f3c60400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f3c580cd8e0 con 0x7f3c58146130 2024-04-20T21:29:36.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.310+0000 7f3c60400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3c580cdbc0 con 0x7f3c58071d00 2024-04-20T21:29:36.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.310+0000 7f3c5d000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3c580ccdd0 0x7f3c580cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:36.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.310+0000 7f3c5d000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3c580ccdd0 0x7f3c580cd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58436/0 (socket says 172.21.15.86:58436) 2024-04-20T21:29:36.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.310+0000 7f3c5d000640 1 -- 172.21.15.86:0/1124665261 learned_addr learned my addr 172.21.15.86:0/1124665261 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:36.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.311+0000 7f3c5da00640 1 --2- 172.21.15.86:0/1124665261 >> v2:172.21.15.144:3300/0 conn(0x7f3c58146130 0x7f3c5806e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:36.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.311+0000 7f3c5e400640 1 --2- 172.21.15.86:0/1124665261 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c58071d00 0x7f3c58072190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:36.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.311+0000 7f3c5d000640 1 -- 172.21.15.86:0/1124665261 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c58071d00 msgr2=0x7f3c58072190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:36.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.311+0000 7f3c5d000640 1 --2- 172.21.15.86:0/1124665261 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c58071d00 0x7f3c58072190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:36.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.311+0000 7f3c5d000640 1 -- 172.21.15.86:0/1124665261 >> v2:172.21.15.144:3300/0 conn(0x7f3c58146130 msgr2=0x7f3c5806e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:36.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.311+0000 7f3c5d000640 1 --2- 172.21.15.86:0/1124665261 >> v2:172.21.15.144:3300/0 conn(0x7f3c58146130 0x7f3c5806e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:36.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.311+0000 7f3c5d000640 1 -- 172.21.15.86:0/1124665261 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c5806ded0 con 0x7f3c580ccdd0 2024-04-20T21:29:36.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.312+0000 7f3c5d000640 1 --2- 172.21.15.86:0/1124665261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c580ccdd0 0x7f3c580cd230 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f3c580b5950 tx=0x7f3c4402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7890db85fa36caa6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:36.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.312+0000 7f3c57e00640 1 -- 172.21.15.86:0/1124665261 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c44034070 con 0x7f3c580ccdd0 2024-04-20T21:29:36.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.312+0000 7f3c57e00640 1 -- 172.21.15.86:0/1124665261 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c44030860 con 0x7f3c580ccdd0 2024-04-20T21:29:36.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.312+0000 7f3c57e00640 1 -- 172.21.15.86:0/1124665261 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c44030c30 con 0x7f3c580ccdd0 2024-04-20T21:29:36.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.313+0000 7f3c60400640 1 -- 172.21.15.86:0/1124665261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c580ccdd0 msgr2=0x7f3c580cd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:36.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.313+0000 7f3c60400640 1 --2- 172.21.15.86:0/1124665261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c580ccdd0 0x7f3c580cd230 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f3c580b5950 tx=0x7f3c4402b9a0 comp rx=0 tx=0).stop 2024-04-20T21:29:36.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.313+0000 7f3c60400640 1 -- 172.21.15.86:0/1124665261 shutdown_connections 2024-04-20T21:29:36.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.313+0000 7f3c60400640 1 --2- 172.21.15.86:0/1124665261 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c58071d00 0x7f3c58072190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:36.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.313+0000 7f3c60400640 1 --2- 172.21.15.86:0/1124665261 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c580ccdd0 0x7f3c580cd230 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:36.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.313+0000 7f3c60400640 1 --2- 172.21.15.86:0/1124665261 >> v2:172.21.15.144:3300/0 conn(0x7f3c58146130 0x7f3c5806e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:36.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.313+0000 7f3c60400640 1 -- 172.21.15.86:0/1124665261 >> 172.21.15.86:0/1124665261 conn(0x7f3c580b2360 msgr2=0x7f3c580b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:36.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.313+0000 7f3c60400640 1 -- 172.21.15.86:0/1124665261 shutdown_connections 2024-04-20T21:29:36.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.313+0000 7f3c60400640 1 -- 172.21.15.86:0/1124665261 wait complete. 2024-04-20T21:29:36.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.315+0000 7f3c60400640 1 Processor -- start 2024-04-20T21:29:36.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.315+0000 7f3c60400640 1 -- start start 2024-04-20T21:29:36.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.316+0000 7f3c60400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c58071d00 0x7f3c580adc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:36.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.316+0000 7f3c60400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c580ccdd0 0x7f3c5815bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:36.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.316+0000 7f3c60400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c58146130 0x7f3c580b1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:36.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.316+0000 7f3c60400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5815cc20 con 0x7f3c58146130 2024-04-20T21:29:36.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.316+0000 7f3c60400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5806f150 con 0x7f3c580ccdd0 2024-04-20T21:29:36.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.316+0000 7f3c60400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3c5815cda0 con 0x7f3c58071d00 2024-04-20T21:29:36.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.317+0000 7f3c5e400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c58146130 0x7f3c580b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:36.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.317+0000 7f3c5e400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c58146130 0x7f3c580b1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58446/0 (socket says 172.21.15.86:58446) 2024-04-20T21:29:36.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.317+0000 7f3c5e400640 1 -- 172.21.15.86:0/2697167844 learned_addr learned my addr 172.21.15.86:0/2697167844 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:36.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.317+0000 7f3c5d000640 1 --2- 172.21.15.86:0/2697167844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c580ccdd0 0x7f3c5815bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:36.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.317+0000 7f3c5da00640 1 --2- 172.21.15.86:0/2697167844 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c58071d00 0x7f3c580adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:36.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.317+0000 7f3c5e400640 1 -- 172.21.15.86:0/2697167844 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c58071d00 msgr2=0x7f3c580adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:36.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.317+0000 7f3c5e400640 1 --2- 172.21.15.86:0/2697167844 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c58071d00 0x7f3c580adc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:36.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.317+0000 7f3c5e400640 1 -- 172.21.15.86:0/2697167844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c580ccdd0 msgr2=0x7f3c5815bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:36.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.317+0000 7f3c5e400640 1 --2- 172.21.15.86:0/2697167844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c580ccdd0 0x7f3c5815bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:36.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.317+0000 7f3c5e400640 1 -- 172.21.15.86:0/2697167844 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c580af070 con 0x7f3c58146130 2024-04-20T21:29:36.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.317+0000 7f3c5e400640 1 --2- 172.21.15.86:0/2697167844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c58146130 0x7f3c580b1450 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f3c50007e60 tx=0x7f3c5000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:36.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.318+0000 7f3c55600640 1 -- 172.21.15.86:0/2697167844 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c50014620 con 0x7f3c58146130 2024-04-20T21:29:36.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.318+0000 7f3c55600640 1 -- 172.21.15.86:0/2697167844 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c50019070 con 0x7f3c58146130 2024-04-20T21:29:36.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.318+0000 7f3c55600640 1 -- 172.21.15.86:0/2697167844 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c50014a20 con 0x7f3c58146130 2024-04-20T21:29:36.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.318+0000 7f3c60400640 1 -- 172.21.15.86:0/2697167844 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c580ae4f0 con 0x7f3c58146130 2024-04-20T21:29:36.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.318+0000 7f3c60400640 1 -- 172.21.15.86:0/2697167844 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c580aeab0 con 0x7f3c58146130 2024-04-20T21:29:36.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.319+0000 7f3c60400640 1 -- 172.21.15.86:0/2697167844 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c580689e0 con 0x7f3c58146130 2024-04-20T21:29:36.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.320+0000 7f3c55600640 1 -- 172.21.15.86:0/2697167844 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3c50014bc0 con 0x7f3c58146130 2024-04-20T21:29:36.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.321+0000 7f3c55600640 1 --2- 172.21.15.86:0/2697167844 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3c3806c8f0 0x7f3c3806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:36.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.321+0000 7f3c55600640 1 -- 172.21.15.86:0/2697167844 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5029+0+0 (secure 0 0 0) 0x7f3c5008c160 con 0x7f3c58146130 2024-04-20T21:29:36.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.321+0000 7f3c5da00640 1 --2- 172.21.15.86:0/2697167844 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3c3806c8f0 0x7f3c3806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:36.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.322+0000 7f3c5da00640 1 --2- 172.21.15.86:0/2697167844 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3c3806c8f0 0x7f3c3806edb0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c006fd0 tx=0x7f3c4c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:36.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.330+0000 7f3c55600640 1 -- 172.21.15.86:0/2697167844 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3c50011260 con 0x7f3c58146130 2024-04-20T21:29:36.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:36.595+0000 7f3c60400640 1 -- 172.21.15.86:0/2697167844 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) -- 0x7f3c5815ff50 con 0x7f3c58146130 2024-04-20T21:29:37.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.237+0000 7f3c55600640 1 -- 172.21.15.86:0/2697167844 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v557) ==== 147+0+0 (secure 0 0 0) 0x7f3c5005a820 con 0x7f3c58146130 2024-04-20T21:29:37.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.240+0000 7f3c60400640 1 -- 172.21.15.86:0/2697167844 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3c3806c8f0 msgr2=0x7f3c3806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:37.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.240+0000 7f3c60400640 1 --2- 172.21.15.86:0/2697167844 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3c3806c8f0 0x7f3c3806edb0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c006fd0 tx=0x7f3c4c008040 comp rx=0 tx=0).stop 2024-04-20T21:29:37.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.240+0000 7f3c60400640 1 -- 172.21.15.86:0/2697167844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c58146130 msgr2=0x7f3c580b1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:37.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.240+0000 7f3c60400640 1 --2- 172.21.15.86:0/2697167844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c58146130 0x7f3c580b1450 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f3c50007e60 tx=0x7f3c5000e4b0 comp rx=0 tx=0).stop 2024-04-20T21:29:37.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.241+0000 7f3c60400640 1 -- 172.21.15.86:0/2697167844 shutdown_connections 2024-04-20T21:29:37.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.241+0000 7f3c60400640 1 --2- 172.21.15.86:0/2697167844 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3c3806c8f0 0x7f3c3806edb0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:37.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.241+0000 7f3c60400640 1 --2- 172.21.15.86:0/2697167844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c58146130 0x7f3c580b1450 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:37.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.241+0000 7f3c60400640 1 --2- 172.21.15.86:0/2697167844 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c580ccdd0 0x7f3c5815bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:37.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.241+0000 7f3c60400640 1 --2- 172.21.15.86:0/2697167844 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3c58071d00 0x7f3c580adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:37.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.241+0000 7f3c60400640 1 -- 172.21.15.86:0/2697167844 >> 172.21.15.86:0/2697167844 conn(0x7f3c580b2360 msgr2=0x7f3c5815e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:37.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.241+0000 7f3c60400640 1 -- 172.21.15.86:0/2697167844 shutdown_connections 2024-04-20T21:29:37.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.241+0000 7f3c60400640 1 -- 172.21.15.86:0/2697167844 wait complete. 2024-04-20T21:29:37.263 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-20T21:29:37.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.458+0000 7ff5b3200640 1 Processor -- start 2024-04-20T21:29:37.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.459+0000 7ff5b3200640 1 -- start start 2024-04-20T21:29:37.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.460+0000 7ff5b3200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff5b4162dd0 0x7ff5b40ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:37.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.460+0000 7ff5b3200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff5b4072120 0x7ff5b406e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:37.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.460+0000 7ff5b3200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5b40cf550 con 0x7ff5b4071d40 2024-04-20T21:29:37.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.460+0000 7ff5b3200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff5b40ce6f0 con 0x7ff5b4162dd0 2024-04-20T21:29:37.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.460+0000 7ff5b3200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7ff5b40ce940 con 0x7ff5b4072120 2024-04-20T21:29:37.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.460+0000 7ff5b1400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff5b4162dd0 0x7ff5b40ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:37.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.460+0000 7ff5b1400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff5b4162dd0 0x7ff5b40ce1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58458/0 (socket says 172.21.15.86:58458) 2024-04-20T21:29:37.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.460+0000 7ff5b1400640 1 -- 172.21.15.86:0/1853108040 learned_addr learned my addr 172.21.15.86:0/1853108040 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:37.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.461+0000 7ff5b1400640 1 -- 172.21.15.86:0/1853108040 >> v2:172.21.15.144:3300/0 conn(0x7ff5b4072120 msgr2=0x7ff5b406e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:37.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.461+0000 7ff5b1400640 1 --2- 172.21.15.86:0/1853108040 >> v2:172.21.15.144:3300/0 conn(0x7ff5b4072120 0x7ff5b406e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:37.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.461+0000 7ff5b1400640 1 -- 172.21.15.86:0/1853108040 >> v1:172.21.15.144:6789/0 conn(0x7ff5b4071d40 legacy=0x7ff5b40b3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:37.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.461+0000 7ff5b1400640 1 -- 172.21.15.86:0/1853108040 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5b40cec20 con 0x7ff5b4162dd0 2024-04-20T21:29:37.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.462+0000 7ff5b1400640 1 --2- 172.21.15.86:0/1853108040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5b4162dd0 0x7ff5b40ce1b0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7ff5b40700e0 tx=0x7ff59c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2051794f88a163ee server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:37.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.462+0000 7ff5b0a00640 1 -- 172.21.15.86:0/1853108040 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c035020 con 0x7ff5b4162dd0 2024-04-20T21:29:37.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.462+0000 7ff5b0a00640 1 -- 172.21.15.86:0/1853108040 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff59c031870 con 0x7ff5b4162dd0 2024-04-20T21:29:37.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.462+0000 7ff5b0a00640 1 -- 172.21.15.86:0/1853108040 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c031c40 con 0x7ff5b4162dd0 2024-04-20T21:29:37.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.463+0000 7ff5b3200640 1 -- 172.21.15.86:0/1853108040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5b4162dd0 msgr2=0x7ff5b40ce1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:37.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.463+0000 7ff5b3200640 1 --2- 172.21.15.86:0/1853108040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5b4162dd0 0x7ff5b40ce1b0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7ff5b40700e0 tx=0x7ff59c02b910 comp rx=0 tx=0).stop 2024-04-20T21:29:37.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.463+0000 7ff5b3200640 1 -- 172.21.15.86:0/1853108040 shutdown_connections 2024-04-20T21:29:37.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.463+0000 7ff5b3200640 1 --2- 172.21.15.86:0/1853108040 >> v2:172.21.15.144:3300/0 conn(0x7ff5b4072120 0x7ff5b406e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:37.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.463+0000 7ff5b3200640 1 --2- 172.21.15.86:0/1853108040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5b4162dd0 0x7ff5b40ce1b0 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:37.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.463+0000 7ff5b3200640 1 -- 172.21.15.86:0/1853108040 >> 172.21.15.86:0/1853108040 conn(0x7ff5b4067cf0 msgr2=0x7ff5b406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:37.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.463+0000 7ff5b3200640 1 -- 172.21.15.86:0/1853108040 shutdown_connections 2024-04-20T21:29:37.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.463+0000 7ff5b3200640 1 -- 172.21.15.86:0/1853108040 wait complete. 2024-04-20T21:29:37.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.465+0000 7ff5b3200640 1 Processor -- start 2024-04-20T21:29:37.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.465+0000 7ff5b3200640 1 -- start start 2024-04-20T21:29:37.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.466+0000 7ff5b3200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5b4071d40 0x7ff5b414a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:37.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.466+0000 7ff5b3200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5b4072120 0x7ff5b4158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:37.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.466+0000 7ff5b3200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5b4162dd0 0x7ff5b415d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:37.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.466+0000 7ff5b3200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b4072570 con 0x7ff5b4071d40 2024-04-20T21:29:37.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.466+0000 7ff5b3200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff5b40cf1e0 con 0x7ff5b4072120 2024-04-20T21:29:37.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.466+0000 7ff5b3200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff5b40726f0 con 0x7ff5b4162dd0 2024-04-20T21:29:37.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.466+0000 7ff5b1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5b4071d40 0x7ff5b414a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:37.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.466+0000 7ff5b1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5b4071d40 0x7ff5b414a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58464/0 (socket says 172.21.15.86:58464) 2024-04-20T21:29:37.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.466+0000 7ff5b1e00640 1 -- 172.21.15.86:0/3518304610 learned_addr learned my addr 172.21.15.86:0/3518304610 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:37.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.467+0000 7ff5b1400640 1 --2- 172.21.15.86:0/3518304610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5b4072120 0x7ff5b4158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:37.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.467+0000 7ff5b1e00640 1 -- 172.21.15.86:0/3518304610 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5b4162dd0 msgr2=0x7ff5b415d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:37.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.467+0000 7ff5b1e00640 1 --2- 172.21.15.86:0/3518304610 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5b4162dd0 0x7ff5b415d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:37.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.467+0000 7ff5b1e00640 1 -- 172.21.15.86:0/3518304610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5b4072120 msgr2=0x7ff5b4158100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:37.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.467+0000 7ff5b1e00640 1 --2- 172.21.15.86:0/3518304610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5b4072120 0x7ff5b4158100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:37.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.467+0000 7ff5b1e00640 1 -- 172.21.15.86:0/3518304610 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5b414aab0 con 0x7ff5b4071d40 2024-04-20T21:29:37.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.467+0000 7ff5b1e00640 1 --2- 172.21.15.86:0/3518304610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5b4071d40 0x7ff5b414a570 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7ff5a400c7a0 tx=0x7ff5a400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:37.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.467+0000 7ff5aa000640 1 -- 172.21.15.86:0/3518304610 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a4017070 con 0x7ff5b4071d40 2024-04-20T21:29:37.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.467+0000 7ff5aa000640 1 -- 172.21.15.86:0/3518304610 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5a4003da0 con 0x7ff5b4071d40 2024-04-20T21:29:37.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.467+0000 7ff5aa000640 1 -- 172.21.15.86:0/3518304610 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a40052b0 con 0x7ff5b4071d40 2024-04-20T21:29:37.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.467+0000 7ff5b3200640 1 -- 172.21.15.86:0/3518304610 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5b415acc0 con 0x7ff5b4071d40 2024-04-20T21:29:37.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.468+0000 7ff5b3200640 1 -- 172.21.15.86:0/3518304610 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5b4158e10 con 0x7ff5b4071d40 2024-04-20T21:29:37.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.469+0000 7ff5b3200640 1 -- 172.21.15.86:0/3518304610 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5b404fd90 con 0x7ff5b4071d40 2024-04-20T21:29:37.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.470+0000 7ff5aa000640 1 -- 172.21.15.86:0/3518304610 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff5a4005450 con 0x7ff5b4071d40 2024-04-20T21:29:37.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.470+0000 7ff5aa000640 1 --2- 172.21.15.86:0/3518304610 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff59006c8f0 0x7ff59006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:37.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.471+0000 7ff5aa000640 1 -- 172.21.15.86:0/3518304610 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5029+0+0 (secure 0 0 0) 0x7ff5a408aee0 con 0x7ff5b4071d40 2024-04-20T21:29:37.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.471+0000 7ff5b1400640 1 --2- 172.21.15.86:0/3518304610 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff59006c8f0 0x7ff59006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:37.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.471+0000 7ff5b1400640 1 --2- 172.21.15.86:0/3518304610 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff59006c8f0 0x7ff59006edb0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ff59c036000 tx=0x7ff59c03c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:37.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.475+0000 7ff5aa000640 1 -- 172.21.15.86:0/3518304610 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff5b414aab0 con 0x7ff5b4071d40 2024-04-20T21:29:37.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.672+0000 7ff5b3200640 1 -- 172.21.15.86:0/3518304610 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7ff5b40cf900 con 0x7ff5b4071d40 2024-04-20T21:29:37.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.681+0000 7ff5aa000640 1 -- 172.21.15.86:0/3518304610 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) ==== 122+0+0 (secure 0 0 0) 0x7ff5a4059510 con 0x7ff5b4071d40 2024-04-20T21:29:37.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.684+0000 7ff5b3200640 1 -- 172.21.15.86:0/3518304610 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff59006c8f0 msgr2=0x7ff59006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:37.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.684+0000 7ff5b3200640 1 --2- 172.21.15.86:0/3518304610 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff59006c8f0 0x7ff59006edb0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ff59c036000 tx=0x7ff59c03c040 comp rx=0 tx=0).stop 2024-04-20T21:29:37.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.684+0000 7ff5b3200640 1 -- 172.21.15.86:0/3518304610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5b4071d40 msgr2=0x7ff5b414a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:37.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.684+0000 7ff5b3200640 1 --2- 172.21.15.86:0/3518304610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5b4071d40 0x7ff5b414a570 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7ff5a400c7a0 tx=0x7ff5a400cc60 comp rx=0 tx=0).stop 2024-04-20T21:29:37.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.684+0000 7ff5b3200640 1 -- 172.21.15.86:0/3518304610 shutdown_connections 2024-04-20T21:29:37.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.684+0000 7ff5b3200640 1 --2- 172.21.15.86:0/3518304610 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff59006c8f0 0x7ff59006edb0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:37.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.684+0000 7ff5b3200640 1 --2- 172.21.15.86:0/3518304610 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff5b4162dd0 0x7ff5b415d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:37.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.684+0000 7ff5b3200640 1 --2- 172.21.15.86:0/3518304610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5b4072120 0x7ff5b4158100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:37.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.684+0000 7ff5b3200640 1 --2- 172.21.15.86:0/3518304610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5b4071d40 0x7ff5b414a570 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:37.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.684+0000 7ff5b3200640 1 -- 172.21.15.86:0/3518304610 >> 172.21.15.86:0/3518304610 conn(0x7ff5b4067cf0 msgr2=0x7ff5b40b29f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:37.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.684+0000 7ff5b3200640 1 -- 172.21.15.86:0/3518304610 shutdown_connections 2024-04-20T21:29:37.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:37.684+0000 7ff5b3200640 1 -- 172.21.15.86:0/3518304610 wait complete. 2024-04-20T21:29:37.703 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-20T21:29:37.703 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T21:29:37.800 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T21:29:37.906 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-20T21:29:37.906 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:29:37.907 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:29:37.962 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052731,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248996k,nr_inodes=812249,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T21:29:37.963 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:29:37.963 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:29:38.020 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T21:29:38.020 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:29:38.020 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:29:38.077 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052731,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248996k,nr_inodes=812249,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T21:29:38.077 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:29:38.077 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:29:38.135 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T21:29:38.135 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-20T21:29:38.135 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:29:38.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.342+0000 7f628d200640 1 Processor -- start 2024-04-20T21:29:38.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.342+0000 7f628d200640 1 -- start start 2024-04-20T21:29:38.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.344+0000 7f628d200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f62880ad490 0x7f62880b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:38.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.344+0000 7f628d200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f62880b40a0 0x7f628806efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:38.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.344+0000 7f628d200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62880b44b0 con 0x7f628815aa40 2024-04-20T21:29:38.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.344+0000 7f628d200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f6288073ea0 con 0x7f62880b40a0 2024-04-20T21:29:38.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.344+0000 7f628d200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f6288073bc0 con 0x7f62880ad490 2024-04-20T21:29:38.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.344+0000 7f6287400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f628815aa40 0x7f6288073100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40412/0 (socket says 172.21.15.86:40412) 2024-04-20T21:29:38.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.344+0000 7f6287400640 1 -- 172.21.15.86:0/2408147304 learned_addr learned my addr 172.21.15.86:0/2408147304 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:29:38.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.344+0000 7f6287e00640 1 --2- 172.21.15.86:0/2408147304 >> v2:172.21.15.86:3300/0 conn(0x7f62880b40a0 0x7f628806efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:38.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.344+0000 7f6286a00640 1 --2- 172.21.15.86:0/2408147304 >> v2:172.21.15.144:3300/0 conn(0x7f62880ad490 0x7f62880b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:38.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.344+0000 7f6286000640 1 -- 172.21.15.86:0/2408147304 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 802579805 0 0) 0x7f62880b44b0 con 0x7f628815aa40 2024-04-20T21:29:38.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.345+0000 7f6286000640 1 -- 172.21.15.86:0/2408147304 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f626c003520 con 0x7f628815aa40 2024-04-20T21:29:38.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.345+0000 7f6287e00640 1 -- 172.21.15.86:0/2408147304 >> v2:172.21.15.144:3300/0 conn(0x7f62880ad490 msgr2=0x7f62880b3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:38.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.345+0000 7f6287e00640 1 --2- 172.21.15.86:0/2408147304 >> v2:172.21.15.144:3300/0 conn(0x7f62880ad490 0x7f62880b3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:38.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.345+0000 7f6287e00640 1 -- 172.21.15.86:0/2408147304 >> v1:172.21.15.86:6789/0 conn(0x7f628815aa40 legacy=0x7f6288073100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:38.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.345+0000 7f6287e00640 1 -- 172.21.15.86:0/2408147304 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f628806e4e0 con 0x7f62880b40a0 2024-04-20T21:29:38.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.346+0000 7f6287e00640 1 --2- 172.21.15.86:0/2408147304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62880b40a0 0x7f628806efc0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f6288074020 tx=0x7f628002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80993709937614b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:38.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.346+0000 7f6286000640 1 -- 172.21.15.86:0/2408147304 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62800315a0 con 0x7f62880b40a0 2024-04-20T21:29:38.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.346+0000 7f6286000640 1 -- 172.21.15.86:0/2408147304 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6280037070 con 0x7f62880b40a0 2024-04-20T21:29:38.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.346+0000 7f6286000640 1 -- 172.21.15.86:0/2408147304 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6280031960 con 0x7f62880b40a0 2024-04-20T21:29:38.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.347+0000 7f628d200640 1 -- 172.21.15.86:0/2408147304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62880b40a0 msgr2=0x7f628806efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:38.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.347+0000 7f628d200640 1 --2- 172.21.15.86:0/2408147304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62880b40a0 0x7f628806efc0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f6288074020 tx=0x7f628002b840 comp rx=0 tx=0).stop 2024-04-20T21:29:38.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.347+0000 7f628d200640 1 -- 172.21.15.86:0/2408147304 shutdown_connections 2024-04-20T21:29:38.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.347+0000 7f628d200640 1 --2- 172.21.15.86:0/2408147304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62880b40a0 0x7f628806efc0 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:38.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.347+0000 7f628d200640 1 --2- 172.21.15.86:0/2408147304 >> v2:172.21.15.144:3300/0 conn(0x7f62880ad490 0x7f62880b3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:38.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.347+0000 7f628d200640 1 -- 172.21.15.86:0/2408147304 >> 172.21.15.86:0/2408147304 conn(0x7f62880d0170 msgr2=0x7f62880d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:38.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.347+0000 7f628d200640 1 -- 172.21.15.86:0/2408147304 shutdown_connections 2024-04-20T21:29:38.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.347+0000 7f628d200640 1 -- 172.21.15.86:0/2408147304 wait complete. 2024-04-20T21:29:38.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.348+0000 7f628d200640 1 Processor -- start 2024-04-20T21:29:38.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.349+0000 7f628d200640 1 -- start start 2024-04-20T21:29:38.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.349+0000 7f628d200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62880ad490 0x7f62880998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:38.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.349+0000 7f628d200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f62880b40a0 0x7f6288158030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:38.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.350+0000 7f628d200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f628815aa40 0x7f6288158570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:38.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.350+0000 7f628d200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f62880ad920 con 0x7f62880ad490 2024-04-20T21:29:38.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.350+0000 7f628d200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f628806eaa0 con 0x7f628815aa40 2024-04-20T21:29:38.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.350+0000 7f628d200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f62880b33a0 con 0x7f62880b40a0 2024-04-20T21:29:38.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.350+0000 7f6287400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62880ad490 0x7f62880998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:38.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.350+0000 7f6287400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62880ad490 0x7f62880998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58482/0 (socket says 172.21.15.86:58482) 2024-04-20T21:29:38.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.350+0000 7f6287400640 1 -- 172.21.15.86:0/569522676 learned_addr learned my addr 172.21.15.86:0/569522676 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:38.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.350+0000 7f6287400640 1 -- 172.21.15.86:0/569522676 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f62880b40a0 msgr2=0x7f6288158030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:38.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.350+0000 7f6287400640 1 --2- 172.21.15.86:0/569522676 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f62880b40a0 0x7f6288158030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:38.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.350+0000 7f6287400640 1 -- 172.21.15.86:0/569522676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f628815aa40 msgr2=0x7f6288158570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:38.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.350+0000 7f6287400640 1 --2- 172.21.15.86:0/569522676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f628815aa40 0x7f6288158570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:38.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.350+0000 7f6287400640 1 -- 172.21.15.86:0/569522676 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6288150fe0 con 0x7f62880ad490 2024-04-20T21:29:38.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.351+0000 7f6287400640 1 --2- 172.21.15.86:0/569522676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62880ad490 0x7f62880998c0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f627800cad0 tx=0x7f627800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:38.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.351+0000 7f627f400640 1 -- 172.21.15.86:0/569522676 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6278017070 con 0x7f62880ad490 2024-04-20T21:29:38.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.351+0000 7f627f400640 1 -- 172.21.15.86:0/569522676 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f627800ac30 con 0x7f62880ad490 2024-04-20T21:29:38.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.351+0000 7f627f400640 1 -- 172.21.15.86:0/569522676 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6278012400 con 0x7f62880ad490 2024-04-20T21:29:38.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.351+0000 7f628d200640 1 -- 172.21.15.86:0/569522676 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6288158c00 con 0x7f62880ad490 2024-04-20T21:29:38.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.351+0000 7f628d200640 1 -- 172.21.15.86:0/569522676 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6288145410 con 0x7f62880ad490 2024-04-20T21:29:38.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.352+0000 7f628d200640 1 -- 172.21.15.86:0/569522676 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f628804fcf0 con 0x7f62880ad490 2024-04-20T21:29:38.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.353+0000 7f627f400640 1 -- 172.21.15.86:0/569522676 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f627801a070 con 0x7f62880ad490 2024-04-20T21:29:38.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.354+0000 7f627f400640 1 --2- 172.21.15.86:0/569522676 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f626406c8f0 0x7f626406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:38.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.354+0000 7f627f400640 1 -- 172.21.15.86:0/569522676 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5029+0+0 (secure 0 0 0) 0x7f627808b760 con 0x7f62880ad490 2024-04-20T21:29:38.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.354+0000 7f6286a00640 1 --2- 172.21.15.86:0/569522676 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f626406c8f0 0x7f626406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:38.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.355+0000 7f6286a00640 1 --2- 172.21.15.86:0/569522676 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f626406c8f0 0x7f626406edb0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f628815a210 tx=0x7f6270008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:38.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.359+0000 7f627f400640 1 -- 172.21.15.86:0/569522676 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6278059d90 con 0x7f62880ad490 2024-04-20T21:29:38.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.596+0000 7f628d200640 1 -- 172.21.15.86:0/569522676 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f62881678d0 con 0x7f62880ad490 2024-04-20T21:29:38.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.598+0000 7f627f400640 1 -- 172.21.15.86:0/569522676 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+3875 (secure 0 0 0) 0x7f6278059690 con 0x7f62880ad490 2024-04-20T21:29:38.598 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:29:38.598 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":92,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5525,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/3902778887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":3902778887},{"type":"v1","addr":"172.21.15.144:6835","nonce":3902778887}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6001,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/4270988014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":4270988014},{"type":"v1","addr":"172.21.15.86:6837","nonce":4270988014}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6016,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/1569462554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":1569462554},{"type":"v1","addr":"172.21.15.144:6833","nonce":1569462554}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":9066,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/4050537846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4050537846},{"type":"v1","addr":"172.21.15.86:6836","nonce":4050537846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90}],"filesystems":[]} 2024-04-20T21:29:38.598 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 92 2024-04-20T21:29:38.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.600+0000 7f628d200640 1 -- 172.21.15.86:0/569522676 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f626406c8f0 msgr2=0x7f626406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:38.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.600+0000 7f628d200640 1 --2- 172.21.15.86:0/569522676 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f626406c8f0 0x7f626406edb0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f628815a210 tx=0x7f6270008040 comp rx=0 tx=0).stop 2024-04-20T21:29:38.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.600+0000 7f628d200640 1 -- 172.21.15.86:0/569522676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62880ad490 msgr2=0x7f62880998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:38.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.600+0000 7f628d200640 1 --2- 172.21.15.86:0/569522676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62880ad490 0x7f62880998c0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f627800cad0 tx=0x7f627800cf90 comp rx=0 tx=0).stop 2024-04-20T21:29:38.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.600+0000 7f628d200640 1 -- 172.21.15.86:0/569522676 shutdown_connections 2024-04-20T21:29:38.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.600+0000 7f628d200640 1 --2- 172.21.15.86:0/569522676 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f626406c8f0 0x7f626406edb0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:38.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.600+0000 7f628d200640 1 --2- 172.21.15.86:0/569522676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f628815aa40 0x7f6288158570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:38.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.600+0000 7f628d200640 1 --2- 172.21.15.86:0/569522676 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f62880b40a0 0x7f6288158030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:38.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.600+0000 7f628d200640 1 --2- 172.21.15.86:0/569522676 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f62880ad490 0x7f62880998c0 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:38.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.600+0000 7f628d200640 1 -- 172.21.15.86:0/569522676 >> 172.21.15.86:0/569522676 conn(0x7f62880d0170 msgr2=0x7f62881539a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:38.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.601+0000 7f628d200640 1 -- 172.21.15.86:0/569522676 shutdown_connections 2024-04-20T21:29:38.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.601+0000 7f628d200640 1 -- 172.21.15.86:0/569522676 wait complete. 2024-04-20T21:29:38.620 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-20T21:29:38.621 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-20T21:29:38.621 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-20T21:29:38.621 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:29:38.621 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-20T21:29:38.621 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-20T21:29:38.621 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-20T21:29:38.621 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:29:38.622 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-20T21:29:38.622 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-20T21:29:38.622 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-20T21:29:38.622 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:29:38.622 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-20T21:29:38.622 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-20T21:29:38.622 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-20T21:29:38.623 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:29:38.640 INFO:tasks.ceph.mds.c:Stopped 2024-04-20T21:29:38.640 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-20T21:29:38.645 INFO:tasks.ceph.mds.a:Stopped 2024-04-20T21:29:38.645 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-20T21:29:38.647 INFO:tasks.ceph.mds.b:Stopped 2024-04-20T21:29:38.647 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-20T21:29:38.651 INFO:tasks.ceph.mds.d:Stopped 2024-04-20T21:29:38.652 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-20T21:29:38.692 INFO:tasks.ceph.mds.c:Started 2024-04-20T21:29:38.693 DEBUG:teuthology.parallel:result is None 2024-04-20T21:29:38.699 INFO:tasks.ceph.mds.b:Started 2024-04-20T21:29:38.699 DEBUG:teuthology.parallel:result is None 2024-04-20T21:29:38.700 INFO:tasks.ceph.mds.a:Started 2024-04-20T21:29:38.700 DEBUG:teuthology.parallel:result is None 2024-04-20T21:29:38.705 INFO:tasks.ceph.mds.d:Started 2024-04-20T21:29:38.705 DEBUG:teuthology.parallel:result is None 2024-04-20T21:29:38.706 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-20T21:29:38.842 INFO:tasks.ceph.mds.a.smithi086.stdout:starting mds.a at 2024-04-20T21:29:38.842 INFO:tasks.ceph.mds.c.smithi086.stdout:starting mds.c at 2024-04-20T21:29:38.857 INFO:tasks.ceph.mds.d.smithi144.stdout:starting mds.d at 2024-04-20T21:29:38.859 INFO:tasks.ceph.mds.b.smithi144.stdout:starting mds.b at 2024-04-20T21:29:38.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.892+0000 7f7026e00640 1 Processor -- start 2024-04-20T21:29:38.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.892+0000 7f7026e00640 1 -- start start 2024-04-20T21:29:38.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.893+0000 7f7026e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f702015ec30 0x7f70200b61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:38.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.893+0000 7f7026e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f70200b6720 0x7f7020168870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:38.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.893+0000 7f7026e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70200d24d0 con 0x7f702015e850 2024-04-20T21:29:38.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.893+0000 7f7026e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f70200d22a0 con 0x7f70200b6720 2024-04-20T21:29:38.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.893+0000 7f7026e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f70200b6b70 con 0x7f702015ec30 2024-04-20T21:29:38.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.894+0000 7f7024e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f70200b6720 0x7f7020168870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:38.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.894+0000 7f7024e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f70200b6720 0x7f7020168870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58538/0 (socket says 172.21.15.86:58538) 2024-04-20T21:29:38.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.894+0000 7f7024e00640 1 -- 172.21.15.86:0/1731352535 learned_addr learned my addr 172.21.15.86:0/1731352535 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:38.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.894+0000 7f701fe00640 1 --1- 172.21.15.86:0/1731352535 >> v1:172.21.15.144:6789/0 conn(0x7f702015e850 0x7f70200cfcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.86:42534/0 (socket says 172.21.15.86:42534) 2024-04-20T21:29:38.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.894+0000 7f7017400640 1 --2- 172.21.15.86:0/1731352535 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f702015ec30 0x7f70200b61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:38.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.894+0000 7f7024e00640 1 -- 172.21.15.86:0/1731352535 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f702015ec30 msgr2=0x7f70200b61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:38.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.894+0000 7f7024e00640 1 --2- 172.21.15.86:0/1731352535 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f702015ec30 0x7f70200b61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:38.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.894+0000 7f7024e00640 1 -- 172.21.15.86:0/1731352535 >> v1:172.21.15.144:6789/0 conn(0x7f702015e850 legacy=0x7f70200cfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:38.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.894+0000 7f7024e00640 1 -- 172.21.15.86:0/1731352535 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70200d27b0 con 0x7f70200b6720 2024-04-20T21:29:38.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.895+0000 7f7024e00640 1 --2- 172.21.15.86:0/1731352535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70200b6720 0x7f7020168870 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f7018009e50 tx=0x7f701802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66ce4df44988163a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:38.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.896+0000 7f701f400640 1 -- 172.21.15.86:0/1731352535 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701802cb00 con 0x7f70200b6720 2024-04-20T21:29:38.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.896+0000 7f701f400640 1 -- 172.21.15.86:0/1731352535 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f701802cca0 con 0x7f70200b6720 2024-04-20T21:29:38.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.896+0000 7f701f400640 1 -- 172.21.15.86:0/1731352535 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7018034610 con 0x7f70200b6720 2024-04-20T21:29:38.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.896+0000 7f7026e00640 1 -- 172.21.15.86:0/1731352535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70200b6720 msgr2=0x7f7020168870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:38.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.896+0000 7f7026e00640 1 --2- 172.21.15.86:0/1731352535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70200b6720 0x7f7020168870 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f7018009e50 tx=0x7f701802bf60 comp rx=0 tx=0).stop 2024-04-20T21:29:38.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.896+0000 7f7026e00640 1 -- 172.21.15.86:0/1731352535 shutdown_connections 2024-04-20T21:29:38.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.896+0000 7f7026e00640 1 --2- 172.21.15.86:0/1731352535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70200b6720 0x7f7020168870 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:38.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.896+0000 7f7026e00640 1 --2- 172.21.15.86:0/1731352535 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f702015ec30 0x7f70200b61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:38.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.896+0000 7f7026e00640 1 -- 172.21.15.86:0/1731352535 >> 172.21.15.86:0/1731352535 conn(0x7f702008bf70 msgr2=0x7f70200d4760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:38.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.896+0000 7f7026e00640 1 -- 172.21.15.86:0/1731352535 shutdown_connections 2024-04-20T21:29:38.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.896+0000 7f7026e00640 1 -- 172.21.15.86:0/1731352535 wait complete. 2024-04-20T21:29:38.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.898+0000 7f7026e00640 1 Processor -- start 2024-04-20T21:29:38.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.898+0000 7f7026e00640 1 -- start start 2024-04-20T21:29:38.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.899+0000 7f7026e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70200b6720 0x7f7020158c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:38.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.899+0000 7f7026e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f702015e850 0x7f702015f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:38.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.899+0000 7f7026e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f702015ec30 0x7f702015f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:38.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.899+0000 7f7026e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f70200d3050 con 0x7f70200b6720 2024-04-20T21:29:38.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.899+0000 7f7026e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f70200d2d70 con 0x7f702015e850 2024-04-20T21:29:38.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.899+0000 7f7026e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f70200d3570 con 0x7f702015ec30 2024-04-20T21:29:38.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.899+0000 7f701fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70200b6720 0x7f7020158c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:38.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.899+0000 7f701fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70200b6720 0x7f7020158c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58546/0 (socket says 172.21.15.86:58546) 2024-04-20T21:29:38.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.899+0000 7f701fe00640 1 -- 172.21.15.86:0/2132390161 learned_addr learned my addr 172.21.15.86:0/2132390161 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:38.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.900+0000 7f7024e00640 1 --2- 172.21.15.86:0/2132390161 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f702015ec30 0x7f702015f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:38.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.900+0000 7f7017400640 1 --2- 172.21.15.86:0/2132390161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f702015e850 0x7f702015f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:38.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.900+0000 7f701fe00640 1 -- 172.21.15.86:0/2132390161 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f702015ec30 msgr2=0x7f702015f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:38.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.900+0000 7f701fe00640 1 --2- 172.21.15.86:0/2132390161 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f702015ec30 0x7f702015f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:38.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.900+0000 7f701fe00640 1 -- 172.21.15.86:0/2132390161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f702015e850 msgr2=0x7f702015f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:38.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.900+0000 7f701fe00640 1 --2- 172.21.15.86:0/2132390161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f702015e850 0x7f702015f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:38.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.900+0000 7f701fe00640 1 -- 172.21.15.86:0/2132390161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70200b77b0 con 0x7f70200b6720 2024-04-20T21:29:38.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.900+0000 7f7024e00640 1 --2- 172.21.15.86:0/2132390161 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f702015ec30 0x7f702015f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:29:38.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.900+0000 7f7017400640 1 --2- 172.21.15.86:0/2132390161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f702015e850 0x7f702015f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:29:38.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.900+0000 7f701fe00640 1 --2- 172.21.15.86:0/2132390161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70200b6720 0x7f7020158c40 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f701000c3c0 tx=0x7f701000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:38.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.900+0000 7f701cc00640 1 -- 172.21.15.86:0/2132390161 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7010017070 con 0x7f70200b6720 2024-04-20T21:29:38.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.901+0000 7f701cc00640 1 -- 172.21.15.86:0/2132390161 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7010005c70 con 0x7f70200b6720 2024-04-20T21:29:38.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.901+0000 7f701cc00640 1 -- 172.21.15.86:0/2132390161 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7010005e60 con 0x7f70200b6720 2024-04-20T21:29:38.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.901+0000 7f7026e00640 1 -- 172.21.15.86:0/2132390161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70200b7a90 con 0x7f70200b6720 2024-04-20T21:29:38.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.901+0000 7f7026e00640 1 -- 172.21.15.86:0/2132390161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70200b8050 con 0x7f70200b6720 2024-04-20T21:29:38.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.902+0000 7f7026e00640 1 -- 172.21.15.86:0/2132390161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70200b8480 con 0x7f70200b6720 2024-04-20T21:29:38.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.903+0000 7f701cc00640 1 -- 172.21.15.86:0/2132390161 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f701001a070 con 0x7f70200b6720 2024-04-20T21:29:38.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.904+0000 7f701cc00640 1 --2- 172.21.15.86:0/2132390161 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6ffc06c920 0x7f6ffc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:38.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.904+0000 7f701cc00640 1 -- 172.21.15.86:0/2132390161 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5029+0+0 (secure 0 0 0) 0x7f701008aad0 con 0x7f70200b6720 2024-04-20T21:29:38.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.904+0000 7f7017400640 1 --2- 172.21.15.86:0/2132390161 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6ffc06c920 0x7f6ffc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:38.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.905+0000 7f7017400640 1 --2- 172.21.15.86:0/2132390161 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6ffc06c920 0x7f6ffc06ede0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f70201f7900 tx=0x7f7008008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:38.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:38.909+0000 7f701cc00640 1 -- 172.21.15.86:0/2132390161 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7010058b50 con 0x7f70200b6720 2024-04-20T21:29:39.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.126+0000 7f7026e00640 1 -- 172.21.15.86:0/2132390161 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f70200689e0 con 0x7f70200b6720 2024-04-20T21:29:39.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.262+0000 7f701cc00640 1 -- 172.21.15.86:0/2132390161 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v106) ==== 97+0+0 (secure 0 0 0) 0x7f701005ce50 con 0x7f70200b6720 2024-04-20T21:29:39.262 INFO:teuthology.orchestra.run.smithi086.stderr: removed all blocklist entries 2024-04-20T21:29:39.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.265+0000 7f7026e00640 1 -- 172.21.15.86:0/2132390161 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6ffc06c920 msgr2=0x7f6ffc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:39.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.265+0000 7f7026e00640 1 --2- 172.21.15.86:0/2132390161 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6ffc06c920 0x7f6ffc06ede0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f70201f7900 tx=0x7f7008008040 comp rx=0 tx=0).stop 2024-04-20T21:29:39.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.265+0000 7f7026e00640 1 -- 172.21.15.86:0/2132390161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70200b6720 msgr2=0x7f7020158c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:39.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.265+0000 7f7026e00640 1 --2- 172.21.15.86:0/2132390161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70200b6720 0x7f7020158c40 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f701000c3c0 tx=0x7f701000c880 comp rx=0 tx=0).stop 2024-04-20T21:29:39.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.266+0000 7f7026e00640 1 -- 172.21.15.86:0/2132390161 shutdown_connections 2024-04-20T21:29:39.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.266+0000 7f7026e00640 1 --2- 172.21.15.86:0/2132390161 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6ffc06c920 0x7f6ffc06ede0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:39.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.266+0000 7f7026e00640 1 --2- 172.21.15.86:0/2132390161 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f702015ec30 0x7f702015f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:39.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.266+0000 7f7026e00640 1 --2- 172.21.15.86:0/2132390161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f702015e850 0x7f702015f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:39.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.266+0000 7f7026e00640 1 --2- 172.21.15.86:0/2132390161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70200b6720 0x7f7020158c40 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:39.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.266+0000 7f7026e00640 1 -- 172.21.15.86:0/2132390161 >> 172.21.15.86:0/2132390161 conn(0x7f702008bf70 msgr2=0x7f70200cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:39.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.266+0000 7f7026e00640 1 -- 172.21.15.86:0/2132390161 shutdown_connections 2024-04-20T21:29:39.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.266+0000 7f7026e00640 1 -- 172.21.15.86:0/2132390161 wait complete. 2024-04-20T21:29:39.290 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-20T21:29:39.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.482+0000 7f47f3800640 1 Processor -- start 2024-04-20T21:29:39.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.482+0000 7f47f3800640 1 -- start start 2024-04-20T21:29:39.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.484+0000 7f47f3800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f47ec06eeb0 0x7f47ec0b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:39.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.484+0000 7f47f3800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47ec15f160 con 0x7f47ec15bbb0 2024-04-20T21:29:39.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.484+0000 7f47f3800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47ec15f440 con 0x7f47ec06f290 2024-04-20T21:29:39.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.484+0000 7f47f3800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f47ec15f720 con 0x7f47ec06eeb0 2024-04-20T21:29:39.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.484+0000 7f47f0e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f47ec06eeb0 0x7f47ec0b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:39.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.484+0000 7f47f1800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f47ec15bbb0 0x7f47ec15bfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40432/0 (socket says 172.21.15.86:40432) 2024-04-20T21:29:39.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.484+0000 7f47f1800640 1 -- 172.21.15.86:0/4161490477 learned_addr learned my addr 172.21.15.86:0/4161490477 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:29:39.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.485+0000 7f47f0e00640 1 -- 172.21.15.86:0/4161490477 >> v1:172.21.15.144:6789/0 conn(0x7f47ec06f290 legacy=0x7f47ec0cd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:39.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.485+0000 7f47f0e00640 1 -- 172.21.15.86:0/4161490477 >> v1:172.21.15.86:6789/0 conn(0x7f47ec15bbb0 legacy=0x7f47ec15bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:39.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.485+0000 7f47ebe00640 1 -- 172.21.15.86:0/4161490477 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1934092429 0 0) 0x7f47ec15f160 con 0x7f47ec15bbb0 2024-04-20T21:29:39.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.485+0000 7f47f0e00640 1 -- 172.21.15.86:0/4161490477 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47ec15fa00 con 0x7f47ec06eeb0 2024-04-20T21:29:39.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.486+0000 7f47f0e00640 1 --2- 172.21.15.86:0/4161490477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47ec06eeb0 0x7f47ec0b3c10 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f47dc0097a0 tx=0x7f47dc02b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbf5fbaa54e6607d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:39.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.486+0000 7f47ebe00640 1 -- 172.21.15.86:0/4161490477 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47dc038070 con 0x7f47ec06eeb0 2024-04-20T21:29:39.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.486+0000 7f47ebe00640 1 -- 172.21.15.86:0/4161490477 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47dc031700 con 0x7f47ec06eeb0 2024-04-20T21:29:39.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.486+0000 7f47ebe00640 1 -- 172.21.15.86:0/4161490477 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47dc031ad0 con 0x7f47ec06eeb0 2024-04-20T21:29:39.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.487+0000 7f47f3800640 1 -- 172.21.15.86:0/4161490477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47ec06eeb0 msgr2=0x7f47ec0b3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:39.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.487+0000 7f47f3800640 1 --2- 172.21.15.86:0/4161490477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47ec06eeb0 0x7f47ec0b3c10 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f47dc0097a0 tx=0x7f47dc02b7a0 comp rx=0 tx=0).stop 2024-04-20T21:29:39.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.487+0000 7f47f3800640 1 -- 172.21.15.86:0/4161490477 shutdown_connections 2024-04-20T21:29:39.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.487+0000 7f47f3800640 1 --2- 172.21.15.86:0/4161490477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47ec06eeb0 0x7f47ec0b3c10 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:39.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.487+0000 7f47f3800640 1 -- 172.21.15.86:0/4161490477 >> 172.21.15.86:0/4161490477 conn(0x7f47ec06b980 msgr2=0x7f47ec06bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:39.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.487+0000 7f47f3800640 1 -- 172.21.15.86:0/4161490477 shutdown_connections 2024-04-20T21:29:39.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.487+0000 7f47f3800640 1 -- 172.21.15.86:0/4161490477 wait complete. 2024-04-20T21:29:39.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.489+0000 7f47f3800640 1 Processor -- start 2024-04-20T21:29:39.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.489+0000 7f47f3800640 1 -- start start 2024-04-20T21:29:39.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.490+0000 7f47f3800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47ec06eeb0 0x7f47ec14c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:39.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.490+0000 7f47f3800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47ec06f290 0x7f47ec0b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:39.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.490+0000 7f47f3800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47ec15bbb0 0x7f47ec150d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:39.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.490+0000 7f47f3800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f47ec06f700 con 0x7f47ec06eeb0 2024-04-20T21:29:39.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.490+0000 7f47f3800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f47ec0cdd40 con 0x7f47ec06f290 2024-04-20T21:29:39.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.490+0000 7f47f3800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f47ec0cff90 con 0x7f47ec15bbb0 2024-04-20T21:29:39.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.490+0000 7f47f0e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47ec06eeb0 0x7f47ec14c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:39.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.490+0000 7f47f0e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47ec06eeb0 0x7f47ec14c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58552/0 (socket says 172.21.15.86:58552) 2024-04-20T21:29:39.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.490+0000 7f47f0e00640 1 -- 172.21.15.86:0/850210811 learned_addr learned my addr 172.21.15.86:0/850210811 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:39.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.491+0000 7f47f0e00640 1 -- 172.21.15.86:0/850210811 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47ec15bbb0 msgr2=0x7f47ec150d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:39.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.491+0000 7f47f0e00640 1 --2- 172.21.15.86:0/850210811 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47ec15bbb0 0x7f47ec150d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:39.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.491+0000 7f47f0e00640 1 -- 172.21.15.86:0/850210811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47ec06f290 msgr2=0x7f47ec0b2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:39.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.491+0000 7f47f0e00640 1 --2- 172.21.15.86:0/850210811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47ec06f290 0x7f47ec0b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:39.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.491+0000 7f47f0e00640 1 -- 172.21.15.86:0/850210811 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47ec0b1c70 con 0x7f47ec06eeb0 2024-04-20T21:29:39.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.491+0000 7f47f0e00640 1 --2- 172.21.15.86:0/850210811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47ec06eeb0 0x7f47ec14c520 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f47dc0097a0 tx=0x7f47dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:39.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.491+0000 7f47e9600640 1 -- 172.21.15.86:0/850210811 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47dc043070 con 0x7f47ec06eeb0 2024-04-20T21:29:39.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.491+0000 7f47e9600640 1 -- 172.21.15.86:0/850210811 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47dc033a30 con 0x7f47ec06eeb0 2024-04-20T21:29:39.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.492+0000 7f47e9600640 1 -- 172.21.15.86:0/850210811 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47dc038040 con 0x7f47ec06eeb0 2024-04-20T21:29:39.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.492+0000 7f47f3800640 1 -- 172.21.15.86:0/850210811 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47ec0b1f50 con 0x7f47ec06eeb0 2024-04-20T21:29:39.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.492+0000 7f47f3800640 1 -- 172.21.15.86:0/850210811 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47ec0b2510 con 0x7f47ec06eeb0 2024-04-20T21:29:39.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.493+0000 7f47f3800640 1 -- 172.21.15.86:0/850210811 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47ec0689e0 con 0x7f47ec06eeb0 2024-04-20T21:29:39.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.494+0000 7f47e9600640 1 -- 172.21.15.86:0/850210811 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f47dc046070 con 0x7f47ec06eeb0 2024-04-20T21:29:39.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.494+0000 7f47e9600640 1 --2- 172.21.15.86:0/850210811 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f47c806c920 0x7f47c806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:39.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.495+0000 7f47e9600640 1 -- 172.21.15.86:0/850210811 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 4857+0+0 (secure 0 0 0) 0x7f47dc0ad410 con 0x7f47ec06eeb0 2024-04-20T21:29:39.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.495+0000 7f47e3e00640 1 --2- 172.21.15.86:0/850210811 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f47c806c920 0x7f47c806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:39.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.495+0000 7f47e3e00640 1 --2- 172.21.15.86:0/850210811 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f47c806c920 0x7f47c806ede0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f47d4006fd0 tx=0x7f47d4006bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:39.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.499+0000 7f47e9600640 1 -- 172.21.15.86:0/850210811 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f47dc030070 con 0x7f47ec06eeb0 2024-04-20T21:29:39.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.769+0000 7f47f3800640 1 -- 172.21.15.86:0/850210811 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f47ec14ca60 con 0x7f47ec06eeb0 2024-04-20T21:29:39.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.771+0000 7f47e9600640 1 -- 172.21.15.86:0/850210811 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) ==== 80+0+5304 (secure 0 0 0) 0x7f47dc07bb80 con 0x7f47ec06eeb0 2024-04-20T21:29:39.771 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:29:39.771 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:29:39.771 INFO:teuthology.orchestra.run.smithi086.stdout: "auth_dump": [ 2024-04-20T21:29:39.771 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:29:39.771 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.a", 2024-04-20T21:29:39.771 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDAMCRmOpS/LRAAPZUUQNf6TyeJ2tF7YPnIKw==", 2024-04-20T21:29:39.771 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:29:39.771 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:29:39.771 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-20T21:29:39.771 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-20T21:29:39.771 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:29:39.771 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:29:39.771 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:29:39.772 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:29:39.772 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.b", 2024-04-20T21:29:39.772 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDBMCRmCJLpDRAA/JiLRSAlQRNMpDC+qDSr0g==", 2024-04-20T21:29:39.772 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:29:39.772 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:29:39.772 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-20T21:29:39.772 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-20T21:29:39.772 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:29:39.772 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:29:39.772 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:29:39.772 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:29:39.772 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.c", 2024-04-20T21:29:39.772 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDAMCRmPHOFOxAA4tUsw/n6oSKrNmzPD+NKyw==", 2024-04-20T21:29:39.772 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:29:39.772 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:29:39.773 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-20T21:29:39.773 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-20T21:29:39.773 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:29:39.773 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:29:39.773 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:29:39.773 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:29:39.773 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mds.d", 2024-04-20T21:29:39.773 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDBMCRmZnPXHBAA0jXwBmGehaO7qens4boRIQ==", 2024-04-20T21:29:39.773 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:29:39.773 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:29:39.773 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-20T21:29:39.773 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-20T21:29:39.773 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:29:39.773 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:29:39.774 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:29:39.774 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:29:39.774 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.0", 2024-04-20T21:29:39.774 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDFMCRm4w9UHhAA56y4dJyDASI+IoZv2XrEoQ==", 2024-04-20T21:29:39.774 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:29:39.774 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:29:39.775 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:29:39.775 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:29:39.775 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:29:39.775 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:29:39.775 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:29:39.775 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.1", 2024-04-20T21:29:39.775 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDIMCRmFo7xKRAAou5B04+J1gZtTzW/Mo+nRQ==", 2024-04-20T21:29:39.776 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:29:39.776 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:29:39.776 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:29:39.776 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:29:39.776 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:29:39.777 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:29:39.777 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:29:39.777 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.2", 2024-04-20T21:29:39.777 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDLMCRm7/jFNRAA4KcRsapPFLaKf0NaH5MApg==", 2024-04-20T21:29:39.777 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:29:39.777 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:29:39.778 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:29:39.778 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:29:39.778 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:29:39.778 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:29:39.778 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:29:39.779 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.3", 2024-04-20T21:29:39.779 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDPMCRmUTPMExAA0/dubb4ehsWIwZPS/ao68w==", 2024-04-20T21:29:39.779 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:29:39.779 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:29:39.779 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:29:39.779 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:29:39.780 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:29:39.780 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:29:39.780 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:29:39.780 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.4", 2024-04-20T21:29:39.780 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDVMCRmB4F2KRAA8Xj0DfH5Ou8TUrO8X/BQyw==", 2024-04-20T21:29:39.780 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:29:39.781 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:29:39.781 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:29:39.781 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:29:39.781 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:29:39.781 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:29:39.782 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:29:39.782 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.5", 2024-04-20T21:29:39.782 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDYMCRmSOvYNRAAODYXmRHMGPrCAxcHwlS7Ww==", 2024-04-20T21:29:39.782 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:29:39.782 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:29:39.783 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:29:39.783 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:29:39.783 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:29:39.783 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:29:39.783 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:29:39.783 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.6", 2024-04-20T21:29:39.784 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDcMCRmENJ9BBAAi/KBUhbYGNXEyiSjSHavWQ==", 2024-04-20T21:29:39.784 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:29:39.784 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:29:39.784 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:29:39.784 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:29:39.785 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:29:39.785 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:29:39.785 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:29:39.785 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "osd.7", 2024-04-20T21:29:39.785 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDfMCRmfTHTDhAALL9GZBSGjWYrhgFw/YI6IA==", 2024-04-20T21:29:39.785 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:29:39.786 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow profile osd", 2024-04-20T21:29:39.786 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile osd", 2024-04-20T21:29:39.786 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:29:39.786 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:29:39.786 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:29:39.787 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:29:39.787 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.0", 2024-04-20T21:29:39.787 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDBMCRmHWxyKRAANzhRi4eajp68h/jWkhFMnw==", 2024-04-20T21:29:39.787 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:29:39.787 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:29:39.787 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow r", 2024-04-20T21:29:39.788 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-20T21:29:39.788 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:29:39.788 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:29:39.788 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:29:39.788 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.1", 2024-04-20T21:29:39.788 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDBMCRmgBJAOxAADJLzO97tjEpKn6kUs8g2QA==", 2024-04-20T21:29:39.789 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:29:39.789 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:29:39.789 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow r", 2024-04-20T21:29:39.789 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow rw", 2024-04-20T21:29:39.789 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rwx" 2024-04-20T21:29:39.790 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:29:39.790 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:29:39.790 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:29:39.790 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.2", 2024-04-20T21:29:39.790 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDBMCRmQAdJMhAAiUmKhPEeuV+UPqQ3gynG9w==", 2024-04-20T21:29:39.790 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:29:39.791 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:29:39.791 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow r", 2024-04-20T21:29:39.791 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow rw", 2024-04-20T21:29:39.791 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rwx" 2024-04-20T21:29:39.791 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:29:39.792 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:29:39.792 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:29:39.792 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.3", 2024-04-20T21:29:39.792 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDCMCRm8yMSCBAAFgSVW0XYu7fa40YxHITljQ==", 2024-04-20T21:29:39.792 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:29:39.792 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow", 2024-04-20T21:29:39.793 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow r", 2024-04-20T21:29:39.793 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow rw", 2024-04-20T21:29:39.793 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow rwx" 2024-04-20T21:29:39.793 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:29:39.793 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:29:39.793 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:29:39.794 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "client.admin", 2024-04-20T21:29:39.794 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQC/MCRmyru0KBAApSqv1hZRcLvrRwPMnqylqQ==", 2024-04-20T21:29:39.794 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:29:39.794 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-04-20T21:29:39.794 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr": "allow *", 2024-04-20T21:29:39.795 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow *", 2024-04-20T21:29:39.795 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:29:39.795 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:29:39.795 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:29:39.795 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:29:39.795 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mgr.x", 2024-04-20T21:29:39.796 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDAMCRmvTgjIxAACeSV8gNkL5LV3xzPS6VPeg==", 2024-04-20T21:29:39.796 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:29:39.796 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-04-20T21:29:39.796 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile mgr", 2024-04-20T21:29:39.796 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:29:39.796 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:29:39.797 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:29:39.797 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:29:39.797 INFO:teuthology.orchestra.run.smithi086.stdout: "entity": "mgr.y", 2024-04-20T21:29:39.797 INFO:teuthology.orchestra.run.smithi086.stdout: "key": "AQDAMCRmSn3VGBAA0eu3R53f3gFgscWFMRkDzg==", 2024-04-20T21:29:39.797 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": { 2024-04-20T21:29:39.797 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": "allow *", 2024-04-20T21:29:39.798 INFO:teuthology.orchestra.run.smithi086.stdout: "mon": "allow profile mgr", 2024-04-20T21:29:39.798 INFO:teuthology.orchestra.run.smithi086.stdout: "osd": "allow *" 2024-04-20T21:29:39.798 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:29:39.798 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:29:39.798 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-20T21:29:39.798 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:29:39.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.773+0000 7f47f3800640 1 -- 172.21.15.86:0/850210811 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f47c806c920 msgr2=0x7f47c806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:39.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.773+0000 7f47f3800640 1 --2- 172.21.15.86:0/850210811 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f47c806c920 0x7f47c806ede0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f47d4006fd0 tx=0x7f47d4006bc0 comp rx=0 tx=0).stop 2024-04-20T21:29:39.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.773+0000 7f47f3800640 1 -- 172.21.15.86:0/850210811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47ec06eeb0 msgr2=0x7f47ec14c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:39.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.773+0000 7f47f3800640 1 --2- 172.21.15.86:0/850210811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47ec06eeb0 0x7f47ec14c520 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f47dc0097a0 tx=0x7f47dc0046c0 comp rx=0 tx=0).stop 2024-04-20T21:29:39.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.773+0000 7f47f3800640 1 -- 172.21.15.86:0/850210811 shutdown_connections 2024-04-20T21:29:39.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.773+0000 7f47f3800640 1 --2- 172.21.15.86:0/850210811 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f47c806c920 0x7f47c806ede0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:39.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.773+0000 7f47f3800640 1 --2- 172.21.15.86:0/850210811 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f47ec15bbb0 0x7f47ec150d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:39.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.773+0000 7f47f3800640 1 --2- 172.21.15.86:0/850210811 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47ec06f290 0x7f47ec0b2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:39.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.773+0000 7f47f3800640 1 --2- 172.21.15.86:0/850210811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47ec06eeb0 0x7f47ec14c520 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:39.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.773+0000 7f47f3800640 1 -- 172.21.15.86:0/850210811 >> 172.21.15.86:0/850210811 conn(0x7f47ec06b980 msgr2=0x7f47ec1557f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:39.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.774+0000 7f47f3800640 1 -- 172.21.15.86:0/850210811 shutdown_connections 2024-04-20T21:29:39.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.774+0000 7f47f3800640 1 -- 172.21.15.86:0/850210811 wait complete. 2024-04-20T21:29:39.803 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:29:39.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:39.999+0000 7f7500200640 1 Processor -- start 2024-04-20T21:29:40.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.000+0000 7f7500200640 1 -- start start 2024-04-20T21:29:40.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.001+0000 7f7500200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f74f8071d00 0x7f74f8072180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:40.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.001+0000 7f7500200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74f806fb70 con 0x7f74f8163200 2024-04-20T21:29:40.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.001+0000 7f7500200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74f815d6a0 con 0x7f74f815d2c0 2024-04-20T21:29:40.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.001+0000 7f7500200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f74f8166900 con 0x7f74f8071d00 2024-04-20T21:29:40.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.002+0000 7f74fce00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f74f8071d00 0x7f74f8072180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:40.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.002+0000 7f74fe200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f74f8163200 0x7f74f8163650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40436/0 (socket says 172.21.15.86:40436) 2024-04-20T21:29:40.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.002+0000 7f74fe200640 1 -- 172.21.15.86:0/3451050042 learned_addr learned my addr 172.21.15.86:0/3451050042 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:29:40.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.002+0000 7f74fce00640 1 -- 172.21.15.86:0/3451050042 >> v1:172.21.15.144:6789/0 conn(0x7f74f815d2c0 legacy=0x7f74f806f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:40.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.002+0000 7f74fce00640 1 -- 172.21.15.86:0/3451050042 >> v1:172.21.15.86:6789/0 conn(0x7f74f8163200 legacy=0x7f74f8163650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:40.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.002+0000 7f74fce00640 1 -- 172.21.15.86:0/3451050042 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74f8166be0 con 0x7f74f8071d00 2024-04-20T21:29:40.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.002+0000 7f74f7e00640 1 -- 172.21.15.86:0/3451050042 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1134462009 0 0) 0x7f74f806fb70 con 0x7f74f8163200 2024-04-20T21:29:40.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.003+0000 7f74fce00640 1 --2- 172.21.15.86:0/3451050042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74f8071d00 0x7f74f8072180 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f74e80097a0 tx=0x7f74e802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce479061bf9504c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:40.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.003+0000 7f74f7e00640 1 -- 172.21.15.86:0/3451050042 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e8035020 con 0x7f74f8071d00 2024-04-20T21:29:40.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.003+0000 7f74f7e00640 1 -- 172.21.15.86:0/3451050042 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74e8031830 con 0x7f74f8071d00 2024-04-20T21:29:40.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.003+0000 7f74f7e00640 1 -- 172.21.15.86:0/3451050042 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e8031c00 con 0x7f74f8071d00 2024-04-20T21:29:40.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.004+0000 7f7500200640 1 -- 172.21.15.86:0/3451050042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74f8071d00 msgr2=0x7f74f8072180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:40.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.004+0000 7f7500200640 1 --2- 172.21.15.86:0/3451050042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74f8071d00 0x7f74f8072180 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f74e80097a0 tx=0x7f74e802b8d0 comp rx=0 tx=0).stop 2024-04-20T21:29:40.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.004+0000 7f7500200640 1 -- 172.21.15.86:0/3451050042 shutdown_connections 2024-04-20T21:29:40.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.004+0000 7f7500200640 1 --2- 172.21.15.86:0/3451050042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74f8071d00 0x7f74f8072180 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:40.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.004+0000 7f7500200640 1 -- 172.21.15.86:0/3451050042 >> 172.21.15.86:0/3451050042 conn(0x7f74f80b2360 msgr2=0x7f74f80b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:40.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.004+0000 7f7500200640 1 -- 172.21.15.86:0/3451050042 shutdown_connections 2024-04-20T21:29:40.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.004+0000 7f7500200640 1 -- 172.21.15.86:0/3451050042 wait complete. 2024-04-20T21:29:40.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.006+0000 7f7500200640 1 Processor -- start 2024-04-20T21:29:40.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.006+0000 7f7500200640 1 -- start start 2024-04-20T21:29:40.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.007+0000 7f7500200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74f8071d00 0x7f74f81535e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:40.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.007+0000 7f7500200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74f815d2c0 0x7f74f815c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:40.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.007+0000 7f7500200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74f8163200 0x7f74f806dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:40.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.007+0000 7f7500200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f74f81676c0 con 0x7f74f8163200 2024-04-20T21:29:40.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.007+0000 7f7500200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f74f81671a0 con 0x7f74f815d2c0 2024-04-20T21:29:40.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.007+0000 7f7500200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f74f80b42c0 con 0x7f74f8071d00 2024-04-20T21:29:40.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.007+0000 7f74fe200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74f8163200 0x7f74f806dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:40.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.007+0000 7f74fe200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74f8163200 0x7f74f806dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58576/0 (socket says 172.21.15.86:58576) 2024-04-20T21:29:40.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.007+0000 7f74fe200640 1 -- 172.21.15.86:0/1780970650 learned_addr learned my addr 172.21.15.86:0/1780970650 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:40.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.008+0000 7f74fce00640 1 --2- 172.21.15.86:0/1780970650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74f815d2c0 0x7f74f815c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:40.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.007+0000 7f74fd800640 1 --2- 172.21.15.86:0/1780970650 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74f8071d00 0x7f74f81535e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:40.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.008+0000 7f74fe200640 1 -- 172.21.15.86:0/1780970650 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74f8071d00 msgr2=0x7f74f81535e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:40.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.008+0000 7f74fe200640 1 --2- 172.21.15.86:0/1780970650 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74f8071d00 0x7f74f81535e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:40.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.008+0000 7f74fe200640 1 -- 172.21.15.86:0/1780970650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74f815d2c0 msgr2=0x7f74f815c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:40.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.008+0000 7f74fe200640 1 --2- 172.21.15.86:0/1780970650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74f815d2c0 0x7f74f815c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:40.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.008+0000 7f74fe200640 1 -- 172.21.15.86:0/1780970650 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74f8153b20 con 0x7f74f8163200 2024-04-20T21:29:40.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.008+0000 7f74fd800640 1 --2- 172.21.15.86:0/1780970650 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74f8071d00 0x7f74f81535e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:29:40.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.008+0000 7f74fe200640 1 --2- 172.21.15.86:0/1780970650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74f8163200 0x7f74f806dbe0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f74f000c700 tx=0x7f74f000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:40.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.008+0000 7f74f5600640 1 -- 172.21.15.86:0/1780970650 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f0018070 con 0x7f74f8163200 2024-04-20T21:29:40.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.008+0000 7f74f5600640 1 -- 172.21.15.86:0/1780970650 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74f000ce10 con 0x7f74f8163200 2024-04-20T21:29:40.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.008+0000 7f7500200640 1 -- 172.21.15.86:0/1780970650 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74f81f9680 con 0x7f74f8163200 2024-04-20T21:29:40.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.008+0000 7f74f5600640 1 -- 172.21.15.86:0/1780970650 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f000a880 con 0x7f74f8163200 2024-04-20T21:29:40.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.008+0000 7f7500200640 1 -- 172.21.15.86:0/1780970650 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74f815bc40 con 0x7f74f8163200 2024-04-20T21:29:40.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.009+0000 7f7500200640 1 -- 172.21.15.86:0/1780970650 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74f80689e0 con 0x7f74f8163200 2024-04-20T21:29:40.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.010+0000 7f74f5600640 1 -- 172.21.15.86:0/1780970650 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f74f001b070 con 0x7f74f8163200 2024-04-20T21:29:40.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.011+0000 7f74f5600640 1 --2- 172.21.15.86:0/1780970650 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f74dc06c8f0 0x7f74dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:40.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.011+0000 7f74f5600640 1 -- 172.21.15.86:0/1780970650 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 4857+0+0 (secure 0 0 0) 0x7f74f008ad80 con 0x7f74f8163200 2024-04-20T21:29:40.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.012+0000 7f74fd800640 1 --2- 172.21.15.86:0/1780970650 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f74dc06c8f0 0x7f74dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:40.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.012+0000 7f74fd800640 1 --2- 172.21.15.86:0/1780970650 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f74dc06c8f0 0x7f74dc06edb0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f74ec003eb0 tx=0x7f74ec009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:40.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.016+0000 7f74f5600640 1 -- 172.21.15.86:0/1780970650 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f74f8153b20 con 0x7f74f8163200 2024-04-20T21:29:40.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.199+0000 7f7500200640 1 -- 172.21.15.86:0/1780970650 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f74f806ec60 con 0x7f74f8163200 2024-04-20T21:29:40.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.200+0000 7f74f5600640 1 -- 172.21.15.86:0/1780970650 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) ==== 74+0+12893 (secure 0 0 0) 0x7f74f0059500 con 0x7f74f8163200 2024-04-20T21:29:40.200 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:29:40.200 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":108,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:29:39.277828+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:29:40.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.203+0000 7f7500200640 1 -- 172.21.15.86:0/1780970650 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f74dc06c8f0 msgr2=0x7f74dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:40.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.203+0000 7f7500200640 1 --2- 172.21.15.86:0/1780970650 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f74dc06c8f0 0x7f74dc06edb0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f74ec003eb0 tx=0x7f74ec009040 comp rx=0 tx=0).stop 2024-04-20T21:29:40.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.203+0000 7f7500200640 1 -- 172.21.15.86:0/1780970650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74f8163200 msgr2=0x7f74f806dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:40.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.203+0000 7f7500200640 1 --2- 172.21.15.86:0/1780970650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74f8163200 0x7f74f806dbe0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f74f000c700 tx=0x7f74f000cbc0 comp rx=0 tx=0).stop 2024-04-20T21:29:40.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.203+0000 7f7500200640 1 -- 172.21.15.86:0/1780970650 shutdown_connections 2024-04-20T21:29:40.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.203+0000 7f7500200640 1 --2- 172.21.15.86:0/1780970650 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f74dc06c8f0 0x7f74dc06edb0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:40.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.203+0000 7f7500200640 1 --2- 172.21.15.86:0/1780970650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74f8163200 0x7f74f806dbe0 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:40.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.203+0000 7f7500200640 1 --2- 172.21.15.86:0/1780970650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74f815d2c0 0x7f74f815c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:40.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.203+0000 7f7500200640 1 --2- 172.21.15.86:0/1780970650 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f74f8071d00 0x7f74f81535e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:40.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.203+0000 7f7500200640 1 -- 172.21.15.86:0/1780970650 >> 172.21.15.86:0/1780970650 conn(0x7f74f80b2360 msgr2=0x7f74f8165f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:40.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.203+0000 7f7500200640 1 -- 172.21.15.86:0/1780970650 shutdown_connections 2024-04-20T21:29:40.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.203+0000 7f7500200640 1 -- 172.21.15.86:0/1780970650 wait complete. 2024-04-20T21:29:40.222 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:29:40.222 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:29:40.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.416+0000 7fc009c00640 1 Processor -- start 2024-04-20T21:29:40.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.416+0000 7fc009c00640 1 -- start start 2024-04-20T21:29:40.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.418+0000 7fc009c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc00406bbf0 0x7fc00406e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:40.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.418+0000 7fc009c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc00406bfd0 0x7fc004145b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:40.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.418+0000 7fc009c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc00415cd40 con 0x7fc004146040 2024-04-20T21:29:40.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.418+0000 7fc009c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc0041464b0 con 0x7fc00406bfd0 2024-04-20T21:29:40.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.418+0000 7fc009c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fc00415d000 con 0x7fc00406bbf0 2024-04-20T21:29:40.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.418+0000 7fbffa000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc00406bfd0 0x7fc004145b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:40.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.418+0000 7fc003400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fc004146040 0x7fc0040cd540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40446/0 (socket says 172.21.15.86:40446) 2024-04-20T21:29:40.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.418+0000 7fc003400640 1 -- 172.21.15.86:0/3215117037 learned_addr learned my addr 172.21.15.86:0/3215117037 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:29:40.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.419+0000 7fc002a00640 1 --2- 172.21.15.86:0/3215117037 >> v2:172.21.15.144:3300/0 conn(0x7fc00406bbf0 0x7fc00406e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:40.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.419+0000 7fbffa000640 1 -- 172.21.15.86:0/3215117037 >> v2:172.21.15.144:3300/0 conn(0x7fc00406bbf0 msgr2=0x7fc00406e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:40.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.419+0000 7fc002000640 1 -- 172.21.15.86:0/3215117037 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1978554365 0 0) 0x7fc00415cd40 con 0x7fc004146040 2024-04-20T21:29:40.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.419+0000 7fbffa000640 1 --2- 172.21.15.86:0/3215117037 >> v2:172.21.15.144:3300/0 conn(0x7fc00406bbf0 0x7fc00406e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:40.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.419+0000 7fbffa000640 1 -- 172.21.15.86:0/3215117037 >> v1:172.21.15.86:6789/0 conn(0x7fc004146040 legacy=0x7fc0040cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:40.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.419+0000 7fbffa000640 1 -- 172.21.15.86:0/3215117037 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc00415d2e0 con 0x7fc00406bfd0 2024-04-20T21:29:40.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.420+0000 7fbffa000640 1 --2- 172.21.15.86:0/3215117037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc00406bfd0 0x7fc004145b00 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fc004073150 tx=0x7fbfec02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5ef51f68e219c90 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:40.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.421+0000 7fc002000640 1 -- 172.21.15.86:0/3215117037 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec035020 con 0x7fc00406bfd0 2024-04-20T21:29:40.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.421+0000 7fc002000640 1 -- 172.21.15.86:0/3215117037 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfec031830 con 0x7fc00406bfd0 2024-04-20T21:29:40.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.421+0000 7fc002000640 1 -- 172.21.15.86:0/3215117037 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec031c00 con 0x7fc00406bfd0 2024-04-20T21:29:40.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.421+0000 7fc009c00640 1 -- 172.21.15.86:0/3215117037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc00406bfd0 msgr2=0x7fc004145b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:40.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.421+0000 7fc009c00640 1 --2- 172.21.15.86:0/3215117037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc00406bfd0 0x7fc004145b00 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fc004073150 tx=0x7fbfec02b8d0 comp rx=0 tx=0).stop 2024-04-20T21:29:40.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.421+0000 7fc009c00640 1 -- 172.21.15.86:0/3215117037 shutdown_connections 2024-04-20T21:29:40.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.421+0000 7fc009c00640 1 --2- 172.21.15.86:0/3215117037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc00406bfd0 0x7fc004145b00 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:40.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.421+0000 7fc009c00640 1 --2- 172.21.15.86:0/3215117037 >> v2:172.21.15.144:3300/0 conn(0x7fc00406bbf0 0x7fc00406e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:40.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.421+0000 7fc009c00640 1 -- 172.21.15.86:0/3215117037 >> 172.21.15.86:0/3215117037 conn(0x7fc0040b2110 msgr2=0x7fc0040b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:40.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.421+0000 7fc009c00640 1 -- 172.21.15.86:0/3215117037 shutdown_connections 2024-04-20T21:29:40.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.421+0000 7fc009c00640 1 -- 172.21.15.86:0/3215117037 wait complete. 2024-04-20T21:29:40.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.423+0000 7fc009c00640 1 Processor -- start 2024-04-20T21:29:40.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.423+0000 7fc009c00640 1 -- start start 2024-04-20T21:29:40.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.424+0000 7fc009c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc00406bbf0 0x7fc004152040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:40.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.424+0000 7fc009c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc00406bfd0 0x7fc00413c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:40.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.424+0000 7fc009c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc004146040 0x7fc0041f3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:40.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.424+0000 7fc009c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc0040cdc50 con 0x7fc004146040 2024-04-20T21:29:40.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.424+0000 7fc009c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc00415d8a0 con 0x7fc00406bfd0 2024-04-20T21:29:40.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.424+0000 7fc009c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc0040cddd0 con 0x7fc00406bbf0 2024-04-20T21:29:40.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.424+0000 7fc003400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc004146040 0x7fc0041f3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:40.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.425+0000 7fc003400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc004146040 0x7fc0041f3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58592/0 (socket says 172.21.15.86:58592) 2024-04-20T21:29:40.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.425+0000 7fc003400640 1 -- 172.21.15.86:0/1223768694 learned_addr learned my addr 172.21.15.86:0/1223768694 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:40.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.425+0000 7fc002a00640 1 --2- 172.21.15.86:0/1223768694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc00406bbf0 0x7fc004152040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:40.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.425+0000 7fbffa000640 1 --2- 172.21.15.86:0/1223768694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc00406bfd0 0x7fc00413c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:40.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.425+0000 7fc003400640 1 -- 172.21.15.86:0/1223768694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc00406bbf0 msgr2=0x7fc004152040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:40.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.425+0000 7fc003400640 1 --2- 172.21.15.86:0/1223768694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc00406bbf0 0x7fc004152040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:40.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.425+0000 7fc003400640 1 -- 172.21.15.86:0/1223768694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc00406bfd0 msgr2=0x7fc00413c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:40.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.425+0000 7fc003400640 1 --2- 172.21.15.86:0/1223768694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc00406bfd0 0x7fc00413c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:40.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.425+0000 7fc003400640 1 -- 172.21.15.86:0/1223768694 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc004152580 con 0x7fc004146040 2024-04-20T21:29:40.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.425+0000 7fc003400640 1 --2- 172.21.15.86:0/1223768694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc004146040 0x7fc0041f3fa0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7fbffc00c480 tx=0x7fbffc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:40.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.425+0000 7fbffb400640 1 -- 172.21.15.86:0/1223768694 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc018070 con 0x7fc004146040 2024-04-20T21:29:40.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.425+0000 7fbffb400640 1 -- 172.21.15.86:0/1223768694 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbffc007500 con 0x7fc004146040 2024-04-20T21:29:40.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.425+0000 7fbffb400640 1 -- 172.21.15.86:0/1223768694 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc004a20 con 0x7fc004146040 2024-04-20T21:29:40.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.426+0000 7fc009c00640 1 -- 172.21.15.86:0/1223768694 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc00413ebb0 con 0x7fc004146040 2024-04-20T21:29:40.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.426+0000 7fc009c00640 1 -- 172.21.15.86:0/1223768694 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc00413f450 con 0x7fc004146040 2024-04-20T21:29:40.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.427+0000 7fc009c00640 1 -- 172.21.15.86:0/1223768694 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0040689e0 con 0x7fc004146040 2024-04-20T21:29:40.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.428+0000 7fbffb400640 1 -- 172.21.15.86:0/1223768694 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbffc004bc0 con 0x7fc004146040 2024-04-20T21:29:40.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.429+0000 7fbffb400640 1 --2- 172.21.15.86:0/1223768694 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbfe006c8f0 0x7fbfe006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:40.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.429+0000 7fbffb400640 1 -- 172.21.15.86:0/1223768694 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 4857+0+0 (secure 0 0 0) 0x7fbffc08aa80 con 0x7fc004146040 2024-04-20T21:29:40.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.429+0000 7fc002a00640 1 --2- 172.21.15.86:0/1223768694 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbfe006c8f0 0x7fbfe006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:40.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.429+0000 7fc002a00640 1 --2- 172.21.15.86:0/1223768694 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbfe006c8f0 0x7fbfe006edb0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fbff4009e90 tx=0x7fbff400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:40.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.433+0000 7fbffb400640 1 -- 172.21.15.86:0/1223768694 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbffc08ce20 con 0x7fc004146040 2024-04-20T21:29:40.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.632+0000 7fc009c00640 1 -- 172.21.15.86:0/1223768694 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc00406c780 con 0x7fc004146040 2024-04-20T21:29:40.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.633+0000 7fbffb400640 1 -- 172.21.15.86:0/1223768694 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v108) ==== 93+0+10 (secure 0 0 0) 0x7fc004152580 con 0x7fc004146040 2024-04-20T21:29:40.633 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:29:40.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.636+0000 7fc009c00640 1 -- 172.21.15.86:0/1223768694 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbfe006c8f0 msgr2=0x7fbfe006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:40.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.636+0000 7fc009c00640 1 --2- 172.21.15.86:0/1223768694 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbfe006c8f0 0x7fbfe006edb0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fbff4009e90 tx=0x7fbff400b040 comp rx=0 tx=0).stop 2024-04-20T21:29:40.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.636+0000 7fc009c00640 1 -- 172.21.15.86:0/1223768694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc004146040 msgr2=0x7fc0041f3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:40.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.636+0000 7fc009c00640 1 --2- 172.21.15.86:0/1223768694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc004146040 0x7fc0041f3fa0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7fbffc00c480 tx=0x7fbffc00c940 comp rx=0 tx=0).stop 2024-04-20T21:29:40.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.636+0000 7fc009c00640 1 -- 172.21.15.86:0/1223768694 shutdown_connections 2024-04-20T21:29:40.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.636+0000 7fc009c00640 1 --2- 172.21.15.86:0/1223768694 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbfe006c8f0 0x7fbfe006edb0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:40.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.636+0000 7fc009c00640 1 --2- 172.21.15.86:0/1223768694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc004146040 0x7fc0041f3fa0 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:40.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.636+0000 7fc009c00640 1 --2- 172.21.15.86:0/1223768694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc00406bfd0 0x7fc00413c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:40.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.636+0000 7fc009c00640 1 --2- 172.21.15.86:0/1223768694 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc00406bbf0 0x7fc004152040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:40.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.636+0000 7fc009c00640 1 -- 172.21.15.86:0/1223768694 >> 172.21.15.86:0/1223768694 conn(0x7fc0040b2110 msgr2=0x7fc0040749f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:40.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.636+0000 7fc009c00640 1 -- 172.21.15.86:0/1223768694 shutdown_connections 2024-04-20T21:29:40.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.636+0000 7fc009c00640 1 -- 172.21.15.86:0/1223768694 wait complete. 2024-04-20T21:29:40.655 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-20T21:29:40.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.849+0000 7fc5a4000640 1 Processor -- start 2024-04-20T21:29:40.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.849+0000 7fc5a4000640 1 -- start start 2024-04-20T21:29:40.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.851+0000 7fc5a4000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc59c06e6d0 0x7fc59c071d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:40.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.851+0000 7fc5a4000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc59c06eb30 con 0x7fc59c146130 2024-04-20T21:29:40.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.851+0000 7fc5a4000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc59c06dc30 con 0x7fc59c072240 2024-04-20T21:29:40.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.851+0000 7fc5a4000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fc59c072690 con 0x7fc59c06e6d0 2024-04-20T21:29:40.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.851+0000 7fc5a1600640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fc59c146130 0x7fc59c0ccdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40462/0 (socket says 172.21.15.86:40462) 2024-04-20T21:29:40.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.851+0000 7fc5a1600640 1 -- 172.21.15.86:0/1075282313 learned_addr learned my addr 172.21.15.86:0/1075282313 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:29:40.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.851+0000 7fc59be00640 1 -- 172.21.15.86:0/1075282313 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4138360581 0 0) 0x7fc590001110 con 0x7fc59c146130 2024-04-20T21:29:40.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.851+0000 7fc59be00640 1 -- 172.21.15.86:0/1075282313 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc584003520 con 0x7fc59c146130 2024-04-20T21:29:40.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.851+0000 7fc5a0c00640 1 --2- 172.21.15.86:0/1075282313 >> v2:172.21.15.144:3300/0 conn(0x7fc59c06e6d0 0x7fc59c071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:40.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.852+0000 7fc59be00640 1 -- 172.21.15.86:0/1075282313 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1518081852 0 0) 0x7fc584003520 con 0x7fc59c146130 2024-04-20T21:29:40.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.852+0000 7fc59be00640 1 -- 172.21.15.86:0/1075282313 >> v2:172.21.15.144:3300/0 conn(0x7fc59c06e6d0 msgr2=0x7fc59c071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:40.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.852+0000 7fc59be00640 1 --2- 172.21.15.86:0/1075282313 >> v2:172.21.15.144:3300/0 conn(0x7fc59c06e6d0 0x7fc59c071d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:40.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.852+0000 7fc59be00640 1 -- 172.21.15.86:0/1075282313 >> v1:172.21.15.144:6789/0 conn(0x7fc59c072240 legacy=0x7fc59c15cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:40.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.852+0000 7fc59be00640 1 -- 172.21.15.86:0/1075282313 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc59c0cd8e0 con 0x7fc59c146130 2024-04-20T21:29:40.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.852+0000 7fc59be00640 1 -- 172.21.15.86:0/1075282313 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fc590001840 con 0x7fc59c146130 2024-04-20T21:29:40.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.852+0000 7fc59be00640 1 -- 172.21.15.86:0/1075282313 >> v1:172.21.15.86:6789/0 conn(0x7fc59c146130 legacy=0x7fc59c0ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:40.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.852+0000 7fc59be00640 1 --2- 172.21.15.86:0/1075282313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5840039b0 0x7fc584003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:40.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.852+0000 7fc59be00640 1 --2- 172.21.15.86:0/1075282313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc584004800 0x7fc584004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:40.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.852+0000 7fc59be00640 1 --2- 172.21.15.86:0/1075282313 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc5840051c0 0x7fc58400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:40.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.852+0000 7fc59be00640 1 -- 172.21.15.86:0/1075282313 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc59c072690 con 0x7fc584004800 2024-04-20T21:29:40.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.852+0000 7fc59be00640 1 -- 172.21.15.86:0/1075282313 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc58400c0f0 con 0x7fc5840039b0 2024-04-20T21:29:40.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.852+0000 7fc59be00640 1 -- 172.21.15.86:0/1075282313 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc58400c2f0 con 0x7fc5840051c0 2024-04-20T21:29:40.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.852+0000 7fc5a0c00640 1 --2- 172.21.15.86:0/1075282313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc584004800 0x7fc584004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:40.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.852+0000 7fc5a0c00640 1 -- 172.21.15.86:0/1075282313 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc5840051c0 msgr2=0x7fc58400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:40.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.852+0000 7fc5a0c00640 1 --2- 172.21.15.86:0/1075282313 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc5840051c0 0x7fc58400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:40.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.852+0000 7fc5a1600640 1 --2- 172.21.15.86:0/1075282313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5840039b0 0x7fc584003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:40.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.852+0000 7fc5a0c00640 1 -- 172.21.15.86:0/1075282313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5840039b0 msgr2=0x7fc584003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:40.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.853+0000 7fc5a0c00640 1 --2- 172.21.15.86:0/1075282313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5840039b0 0x7fc584003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:40.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.853+0000 7fc5a0c00640 1 -- 172.21.15.86:0/1075282313 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc584003520 con 0x7fc584004800 2024-04-20T21:29:40.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.853+0000 7fc5a2000640 1 -- 172.21.15.86:0/1075282313 reap_dead start 2024-04-20T21:29:40.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.853+0000 7fc5a1600640 1 --2- 172.21.15.86:0/1075282313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc5840039b0 0x7fc584003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:29:40.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.854+0000 7fc5a0c00640 1 --2- 172.21.15.86:0/1075282313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc584004800 0x7fc584004c80 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7fc58800ba10 tx=0x7fc584005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=932c9c395ed8f7a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:40.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.854+0000 7fc59be00640 1 -- 172.21.15.86:0/1075282313 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc588036020 con 0x7fc584004800 2024-04-20T21:29:40.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.854+0000 7fc59be00640 1 -- 172.21.15.86:0/1075282313 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc588032630 con 0x7fc584004800 2024-04-20T21:29:40.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.854+0000 7fc59be00640 1 -- 172.21.15.86:0/1075282313 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58800b5e0 con 0x7fc584004800 2024-04-20T21:29:40.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.854+0000 7fc5a4000640 1 -- 172.21.15.86:0/1075282313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc584004800 msgr2=0x7fc584004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:40.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.854+0000 7fc5a4000640 1 --2- 172.21.15.86:0/1075282313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc584004800 0x7fc584004c80 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7fc58800ba10 tx=0x7fc584005830 comp rx=0 tx=0).stop 2024-04-20T21:29:40.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.854+0000 7fc5a4000640 1 -- 172.21.15.86:0/1075282313 shutdown_connections 2024-04-20T21:29:40.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.854+0000 7fc5a4000640 1 --2- 172.21.15.86:0/1075282313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc584004800 0x7fc584004c80 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:40.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.854+0000 7fc5a4000640 1 -- 172.21.15.86:0/1075282313 >> 172.21.15.86:0/1075282313 conn(0x7fc59c0b2360 msgr2=0x7fc59c0b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:40.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.855+0000 7fc5a4000640 1 -- 172.21.15.86:0/1075282313 shutdown_connections 2024-04-20T21:29:40.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.855+0000 7fc5a4000640 1 -- 172.21.15.86:0/1075282313 wait complete. 2024-04-20T21:29:40.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.856+0000 7fc5a4000640 1 Processor -- start 2024-04-20T21:29:40.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.856+0000 7fc5a4000640 1 -- start start 2024-04-20T21:29:40.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.857+0000 7fc5a4000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc584004800 0x7fc59c15b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:40.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.857+0000 7fc5a4000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc59c15bd80 0x7fc59c0ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:40.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.857+0000 7fc5a4000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc59c13bd30 0x7fc59c0aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:40.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.857+0000 7fc5a4000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc59c06e1b0 con 0x7fc59c15bd80 2024-04-20T21:29:40.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.857+0000 7fc5a4000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc59c06ded0 con 0x7fc584004800 2024-04-20T21:29:40.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.857+0000 7fc5a4000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc59c06f150 con 0x7fc59c13bd30 2024-04-20T21:29:40.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.858+0000 7fc5a0c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc59c15bd80 0x7fc59c0ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:40.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.858+0000 7fc5a0c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc59c15bd80 0x7fc59c0ae8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58596/0 (socket says 172.21.15.86:58596) 2024-04-20T21:29:40.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.858+0000 7fc5a0c00640 1 -- 172.21.15.86:0/2166665723 learned_addr learned my addr 172.21.15.86:0/2166665723 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:40.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.858+0000 7fc5a1600640 1 --2- 172.21.15.86:0/2166665723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc584004800 0x7fc59c15b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:40.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.858+0000 7fc5a2000640 1 --2- 172.21.15.86:0/2166665723 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc59c13bd30 0x7fc59c0aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:40.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.858+0000 7fc5a0c00640 1 -- 172.21.15.86:0/2166665723 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc59c13bd30 msgr2=0x7fc59c0aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:40.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.858+0000 7fc5a0c00640 1 --2- 172.21.15.86:0/2166665723 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc59c13bd30 0x7fc59c0aede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:40.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.858+0000 7fc5a0c00640 1 -- 172.21.15.86:0/2166665723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc584004800 msgr2=0x7fc59c15b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:40.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.858+0000 7fc5a0c00640 1 --2- 172.21.15.86:0/2166665723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc584004800 0x7fc59c15b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:40.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.858+0000 7fc5a0c00640 1 -- 172.21.15.86:0/2166665723 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc59c15bb80 con 0x7fc59c15bd80 2024-04-20T21:29:40.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.858+0000 7fc5a1600640 1 --2- 172.21.15.86:0/2166665723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc584004800 0x7fc59c15b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:29:40.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.858+0000 7fc5a0c00640 1 --2- 172.21.15.86:0/2166665723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc59c15bd80 0x7fc59c0ae8a0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fc58800bb00 tx=0x7fc588004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:40.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.859+0000 7fc599600640 1 -- 172.21.15.86:0/2166665723 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc588036040 con 0x7fc59c15bd80 2024-04-20T21:29:40.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.859+0000 7fc599600640 1 -- 172.21.15.86:0/2166665723 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc588030a70 con 0x7fc59c15bd80 2024-04-20T21:29:40.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.859+0000 7fc599600640 1 -- 172.21.15.86:0/2166665723 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58800b5e0 con 0x7fc59c15bd80 2024-04-20T21:29:40.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.859+0000 7fc5a4000640 1 -- 172.21.15.86:0/2166665723 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc59c13c310 con 0x7fc59c15bd80 2024-04-20T21:29:40.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.859+0000 7fc5a4000640 1 -- 172.21.15.86:0/2166665723 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc59c13c5b0 con 0x7fc59c15bd80 2024-04-20T21:29:40.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.860+0000 7fc5a4000640 1 -- 172.21.15.86:0/2166665723 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc59c0689e0 con 0x7fc59c15bd80 2024-04-20T21:29:40.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.861+0000 7fc599600640 1 -- 172.21.15.86:0/2166665723 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc588039020 con 0x7fc59c15bd80 2024-04-20T21:29:40.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.862+0000 7fc599600640 1 --2- 172.21.15.86:0/2166665723 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc57c06c8f0 0x7fc57c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:40.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.862+0000 7fc599600640 1 -- 172.21.15.86:0/2166665723 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 4857+0+0 (secure 0 0 0) 0x7fc5880adf20 con 0x7fc59c15bd80 2024-04-20T21:29:40.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.862+0000 7fc5a1600640 1 --2- 172.21.15.86:0/2166665723 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc57c06c8f0 0x7fc57c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:40.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.863+0000 7fc5a1600640 1 --2- 172.21.15.86:0/2166665723 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc57c06c8f0 0x7fc57c06edb0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc590000ff0 tx=0x7fc590009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:40.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:40.867+0000 7fc599600640 1 -- 172.21.15.86:0/2166665723 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc59c15bb80 con 0x7fc59c15bd80 2024-04-20T21:29:41.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.069+0000 7fc5a4000640 1 -- 172.21.15.86:0/2166665723 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fc59c15ff50 con 0x7fc59c15bd80 2024-04-20T21:29:41.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.070+0000 7fc599600640 1 -- 172.21.15.86:0/2166665723 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v108) ==== 84+0+65 (secure 0 0 0) 0x7fc58807c570 con 0x7fc59c15bd80 2024-04-20T21:29:41.070 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:29:41.070 INFO:teuthology.orchestra.run.smithi086.stdout:[ 2024-04-20T21:29:41.070 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-20T21:29:41.070 INFO:teuthology.orchestra.run.smithi086.stdout: "poolnum": 1, 2024-04-20T21:29:41.070 INFO:teuthology.orchestra.run.smithi086.stdout: "poolname": ".mgr" 2024-04-20T21:29:41.070 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:29:41.070 INFO:teuthology.orchestra.run.smithi086.stdout:] 2024-04-20T21:29:41.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.072+0000 7fc5a4000640 1 -- 172.21.15.86:0/2166665723 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc57c06c8f0 msgr2=0x7fc57c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:41.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.072+0000 7fc5a4000640 1 --2- 172.21.15.86:0/2166665723 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc57c06c8f0 0x7fc57c06edb0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc590000ff0 tx=0x7fc590009040 comp rx=0 tx=0).stop 2024-04-20T21:29:41.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.072+0000 7fc5a4000640 1 -- 172.21.15.86:0/2166665723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc59c15bd80 msgr2=0x7fc59c0ae8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:41.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.072+0000 7fc5a4000640 1 --2- 172.21.15.86:0/2166665723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc59c15bd80 0x7fc59c0ae8a0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fc58800bb00 tx=0x7fc588004730 comp rx=0 tx=0).stop 2024-04-20T21:29:41.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.073+0000 7fc5a4000640 1 -- 172.21.15.86:0/2166665723 shutdown_connections 2024-04-20T21:29:41.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.073+0000 7fc5a4000640 1 --2- 172.21.15.86:0/2166665723 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc57c06c8f0 0x7fc57c06edb0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:41.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.073+0000 7fc5a4000640 1 --2- 172.21.15.86:0/2166665723 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc59c13bd30 0x7fc59c0aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:41.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.073+0000 7fc5a4000640 1 --2- 172.21.15.86:0/2166665723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc59c15bd80 0x7fc59c0ae8a0 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:41.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.073+0000 7fc5a4000640 1 --2- 172.21.15.86:0/2166665723 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc584004800 0x7fc59c15b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:41.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.073+0000 7fc5a4000640 1 -- 172.21.15.86:0/2166665723 >> 172.21.15.86:0/2166665723 conn(0x7fc59c0b2360 msgr2=0x7fc59c1448b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:41.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.073+0000 7fc5a4000640 1 -- 172.21.15.86:0/2166665723 shutdown_connections 2024-04-20T21:29:41.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.073+0000 7fc5a4000640 1 -- 172.21.15.86:0/2166665723 wait complete. 2024-04-20T21:29:41.091 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-20T21:29:41.091 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-20T21:29:41.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.296+0000 7f0fc1600640 1 Processor -- start 2024-04-20T21:29:41.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.296+0000 7f0fc1600640 1 -- start start 2024-04-20T21:29:41.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.297+0000 7f0fc1600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fbc15cc70 0x7f0fbc15d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:41.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.297+0000 7f0fc1600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fbc06e010 con 0x7f0fbc071d00 2024-04-20T21:29:41.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.298+0000 7f0fc1600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fbc06e2f0 con 0x7f0fbc06f0b0 2024-04-20T21:29:41.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.298+0000 7f0fc1600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0fbc06e5d0 con 0x7f0fbc15cc70 2024-04-20T21:29:41.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.298+0000 7f0fbaa00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f0fbc071d00 0x7f0fbc0720e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40476/0 (socket says 172.21.15.86:40476) 2024-04-20T21:29:41.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.298+0000 7f0fbaa00640 1 -- 172.21.15.86:0/808705041 learned_addr learned my addr 172.21.15.86:0/808705041 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:29:41.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.298+0000 7f0fb9600640 1 -- 172.21.15.86:0/808705041 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4218467007 0 0) 0x7f0fbc06e010 con 0x7f0fbc071d00 2024-04-20T21:29:41.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.298+0000 7f0fbb400640 1 --2- 172.21.15.86:0/808705041 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fbc15cc70 0x7f0fbc15d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:41.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.298+0000 7f0fb9600640 1 -- 172.21.15.86:0/808705041 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0fa0003520 con 0x7f0fbc071d00 2024-04-20T21:29:41.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.299+0000 7f0fb9600640 1 -- 172.21.15.86:0/808705041 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 222367231 0 0) 0x7f0fa0003520 con 0x7f0fbc071d00 2024-04-20T21:29:41.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.299+0000 7f0fb9600640 1 -- 172.21.15.86:0/808705041 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fbc15cc70 msgr2=0x7f0fbc15d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:41.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.299+0000 7f0fb9600640 1 --2- 172.21.15.86:0/808705041 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fbc15cc70 0x7f0fbc15d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:41.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.299+0000 7f0fb9600640 1 -- 172.21.15.86:0/808705041 >> v1:172.21.15.144:6789/0 conn(0x7f0fbc06f0b0 legacy=0x7f0fbc145a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:41.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.299+0000 7f0fb9600640 1 -- 172.21.15.86:0/808705041 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fbc06e8b0 con 0x7f0fbc071d00 2024-04-20T21:29:41.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.299+0000 7f0fb9600640 1 -- 172.21.15.86:0/808705041 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f0fac001670 con 0x7f0fbc071d00 2024-04-20T21:29:41.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.299+0000 7f0fb9600640 1 -- 172.21.15.86:0/808705041 >> v1:172.21.15.86:6789/0 conn(0x7f0fbc071d00 legacy=0x7f0fbc0720e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:41.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.299+0000 7f0fb9600640 1 --2- 172.21.15.86:0/808705041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fa00039e0 0x7f0fa0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:41.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.299+0000 7f0fb9600640 1 --2- 172.21.15.86:0/808705041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fa0004800 0x7f0fa0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:41.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.299+0000 7f0fb9600640 1 --2- 172.21.15.86:0/808705041 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fbc15cc70 0x7f0fa00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:41.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.299+0000 7f0fb9600640 1 -- 172.21.15.86:0/808705041 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0fbc06e5d0 con 0x7f0fa0004800 2024-04-20T21:29:41.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.299+0000 7f0fb9600640 1 -- 172.21.15.86:0/808705041 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa000baa0 con 0x7f0fa00039e0 2024-04-20T21:29:41.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.299+0000 7f0fb9600640 1 -- 172.21.15.86:0/808705041 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0fa000bca0 con 0x7f0fbc15cc70 2024-04-20T21:29:41.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.299+0000 7f0fba000640 1 --2- 172.21.15.86:0/808705041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fa0004800 0x7f0fa0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:41.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.299+0000 7f0fbaa00640 1 --2- 172.21.15.86:0/808705041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fa00039e0 0x7f0fa0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:41.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.300+0000 7f0fbb400640 1 --2- 172.21.15.86:0/808705041 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fbc15cc70 0x7f0fa00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:41.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.300+0000 7f0fba000640 1 -- 172.21.15.86:0/808705041 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fbc15cc70 msgr2=0x7f0fa00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:41.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.300+0000 7f0fba000640 1 --2- 172.21.15.86:0/808705041 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fbc15cc70 0x7f0fa00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:41.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.300+0000 7f0fba000640 1 -- 172.21.15.86:0/808705041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fa00039e0 msgr2=0x7f0fa0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:41.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.300+0000 7f0fba000640 1 --2- 172.21.15.86:0/808705041 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fa00039e0 0x7f0fa0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:41.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.300+0000 7f0fba000640 1 -- 172.21.15.86:0/808705041 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fa0003520 con 0x7f0fa0004800 2024-04-20T21:29:41.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.300+0000 7f0fbb400640 1 --2- 172.21.15.86:0/808705041 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fbc15cc70 0x7f0fa00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:29:41.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.301+0000 7f0fba000640 1 --2- 172.21.15.86:0/808705041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fa0004800 0x7f0fa0004c80 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8007fd0 tx=0x7f0fa0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0f28674c0ec861c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:41.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.301+0000 7f0fb9600640 1 -- 172.21.15.86:0/808705041 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa80013c0 con 0x7f0fa0004800 2024-04-20T21:29:41.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.301+0000 7f0fb9600640 1 -- 172.21.15.86:0/808705041 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fa80052d0 con 0x7f0fa0004800 2024-04-20T21:29:41.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.301+0000 7f0fb9600640 1 -- 172.21.15.86:0/808705041 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa8003640 con 0x7f0fa0004800 2024-04-20T21:29:41.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.302+0000 7f0fc1600640 1 -- 172.21.15.86:0/808705041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fa0004800 msgr2=0x7f0fa0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:41.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.302+0000 7f0fc1600640 1 --2- 172.21.15.86:0/808705041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fa0004800 0x7f0fa0004c80 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8007fd0 tx=0x7f0fa0005830 comp rx=0 tx=0).stop 2024-04-20T21:29:41.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.302+0000 7f0fbb400640 1 -- 172.21.15.86:0/808705041 reap_dead start 2024-04-20T21:29:41.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.302+0000 7f0fc1600640 1 -- 172.21.15.86:0/808705041 shutdown_connections 2024-04-20T21:29:41.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.302+0000 7f0fc1600640 1 -- 172.21.15.86:0/808705041 >> 172.21.15.86:0/808705041 conn(0x7f0fbc0b2180 msgr2=0x7f0fbc0b2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:41.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.302+0000 7f0fc1600640 1 -- 172.21.15.86:0/808705041 shutdown_connections 2024-04-20T21:29:41.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.302+0000 7f0fc1600640 1 -- 172.21.15.86:0/808705041 wait complete. 2024-04-20T21:29:41.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.304+0000 7f0fc1600640 1 Processor -- start 2024-04-20T21:29:41.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.304+0000 7f0fc1600640 1 -- start start 2024-04-20T21:29:41.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.305+0000 7f0fc1600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fa0004800 0x7f0fbc15f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:41.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.305+0000 7f0fc1600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fbc13fc60 0x7f0fbc13b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:41.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.305+0000 7f0fc1600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fbc1400e0 0x7f0fbc13b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:41.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.305+0000 7f0fc1600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0fbc1461a0 con 0x7f0fbc13fc60 2024-04-20T21:29:41.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.305+0000 7f0fc1600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0fbc06f510 con 0x7f0fbc1400e0 2024-04-20T21:29:41.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.305+0000 7f0fc1600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0fbc06dc30 con 0x7f0fa0004800 2024-04-20T21:29:41.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.305+0000 7f0fba000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fbc13fc60 0x7f0fbc13b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:41.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.305+0000 7f0fba000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fbc13fc60 0x7f0fbc13b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58616/0 (socket says 172.21.15.86:58616) 2024-04-20T21:29:41.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.305+0000 7f0fba000640 1 -- 172.21.15.86:0/2813090271 learned_addr learned my addr 172.21.15.86:0/2813090271 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:41.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.306+0000 7f0fbaa00640 1 --2- 172.21.15.86:0/2813090271 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fa0004800 0x7f0fbc15f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:41.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.306+0000 7f0fba000640 1 -- 172.21.15.86:0/2813090271 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fa0004800 msgr2=0x7f0fbc15f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:41.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.306+0000 7f0fbb400640 1 --2- 172.21.15.86:0/2813090271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fbc1400e0 0x7f0fbc13b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:41.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.306+0000 7f0fba000640 1 --2- 172.21.15.86:0/2813090271 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fa0004800 0x7f0fbc15f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:41.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.306+0000 7f0fba000640 1 -- 172.21.15.86:0/2813090271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fbc1400e0 msgr2=0x7f0fbc13b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:41.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.306+0000 7f0fba000640 1 --2- 172.21.15.86:0/2813090271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fbc1400e0 0x7f0fbc13b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:41.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.306+0000 7f0fba000640 1 -- 172.21.15.86:0/2813090271 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fbc13ebc0 con 0x7f0fbc13fc60 2024-04-20T21:29:41.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.306+0000 7f0fba000640 1 --2- 172.21.15.86:0/2813090271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fbc13fc60 0x7f0fbc13b4a0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8001830 tx=0x7f0fa8001170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:41.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.306+0000 7f0fb2a00640 1 -- 172.21.15.86:0/2813090271 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa8041020 con 0x7f0fbc13fc60 2024-04-20T21:29:41.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.306+0000 7f0fb2a00640 1 -- 172.21.15.86:0/2813090271 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fa80050d0 con 0x7f0fbc13fc60 2024-04-20T21:29:41.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.306+0000 7f0fc1600640 1 -- 172.21.15.86:0/2813090271 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fbc13eea0 con 0x7f0fbc13fc60 2024-04-20T21:29:41.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.306+0000 7f0fb2a00640 1 -- 172.21.15.86:0/2813090271 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa8007490 con 0x7f0fbc13fc60 2024-04-20T21:29:41.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.307+0000 7f0fc1600640 1 -- 172.21.15.86:0/2813090271 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fbc13f460 con 0x7f0fbc13fc60 2024-04-20T21:29:41.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.308+0000 7f0fc1600640 1 -- 172.21.15.86:0/2813090271 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fbc0681f0 con 0x7f0fbc13fc60 2024-04-20T21:29:41.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.308+0000 7f0fb2a00640 1 -- 172.21.15.86:0/2813090271 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0fa8044020 con 0x7f0fbc13fc60 2024-04-20T21:29:41.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.310+0000 7f0fb2a00640 1 --2- 172.21.15.86:0/2813090271 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0f9c06c920 0x7f0f9c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:41.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.310+0000 7f0fb2a00640 1 -- 172.21.15.86:0/2813090271 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 4857+0+0 (secure 0 0 0) 0x7f0fa80ad580 con 0x7f0fbc13fc60 2024-04-20T21:29:41.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.310+0000 7f0fbaa00640 1 --2- 172.21.15.86:0/2813090271 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0f9c06c920 0x7f0f9c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:41.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.311+0000 7f0fbaa00640 1 --2- 172.21.15.86:0/2813090271 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0f9c06c920 0x7f0f9c06ede0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f0fac001200 tx=0x7f0fac000ed0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:41.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.314+0000 7f0fb2a00640 1 -- 172.21.15.86:0/2813090271 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0fbc0681f0 con 0x7f0fbc13fc60 2024-04-20T21:29:41.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:41.513+0000 7f0fc1600640 1 -- 172.21.15.86:0/2813090271 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f0fbc0ce600 con 0x7f0fbc13fc60 2024-04-20T21:29:42.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.291+0000 7f0fb2a00640 1 -- 172.21.15.86:0/2813090271 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v109) ==== 138+0+0 (secure 0 0 0) 0x7f0fa804b020 con 0x7f0fbc13fc60 2024-04-20T21:29:42.292 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' created 2024-04-20T21:29:42.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.295+0000 7f0fc1600640 1 -- 172.21.15.86:0/2813090271 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0f9c06c920 msgr2=0x7f0f9c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:42.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.296+0000 7f0fc1600640 1 --2- 172.21.15.86:0/2813090271 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0f9c06c920 0x7f0f9c06ede0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f0fac001200 tx=0x7f0fac000ed0 comp rx=0 tx=0).stop 2024-04-20T21:29:42.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.296+0000 7f0fc1600640 1 -- 172.21.15.86:0/2813090271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fbc13fc60 msgr2=0x7f0fbc13b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:42.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.296+0000 7f0fc1600640 1 --2- 172.21.15.86:0/2813090271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fbc13fc60 0x7f0fbc13b4a0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8001830 tx=0x7f0fa8001170 comp rx=0 tx=0).stop 2024-04-20T21:29:42.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.296+0000 7f0fc1600640 1 -- 172.21.15.86:0/2813090271 shutdown_connections 2024-04-20T21:29:42.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.296+0000 7f0fc1600640 1 --2- 172.21.15.86:0/2813090271 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0f9c06c920 0x7f0f9c06ede0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:42.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.296+0000 7f0fc1600640 1 --2- 172.21.15.86:0/2813090271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fbc1400e0 0x7f0fbc13b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:42.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.296+0000 7f0fc1600640 1 --2- 172.21.15.86:0/2813090271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fbc13fc60 0x7f0fbc13b4a0 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:42.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.296+0000 7f0fc1600640 1 --2- 172.21.15.86:0/2813090271 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0fa0004800 0x7f0fbc15f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:42.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.296+0000 7f0fc1600640 1 -- 172.21.15.86:0/2813090271 >> 172.21.15.86:0/2813090271 conn(0x7f0fbc0b2180 msgr2=0x7f0fbc0b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:42.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.296+0000 7f0fc1600640 1 -- 172.21.15.86:0/2813090271 shutdown_connections 2024-04-20T21:29:42.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.297+0000 7f0fc1600640 1 -- 172.21.15.86:0/2813090271 wait complete. 2024-04-20T21:29:42.323 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-20T21:29:42.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.516+0000 7f08f8200640 1 Processor -- start 2024-04-20T21:29:42.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.516+0000 7f08f8200640 1 -- start start 2024-04-20T21:29:42.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.518+0000 7f08f8200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f08f015eb30 0x7f08f00b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:42.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.518+0000 7f08f8200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f08f006c090 0x7f08f00b4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:42.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.518+0000 7f08f8200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08f00adb10 con 0x7f08f00aee50 2024-04-20T21:29:42.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.518+0000 7f08f8200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f08f00af7d0 con 0x7f08f015eb30 2024-04-20T21:29:42.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.518+0000 7f08f8200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f08f00addb0 con 0x7f08f006c090 2024-04-20T21:29:42.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.518+0000 7f08f5800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f08f015eb30 0x7f08f00b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:42.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.518+0000 7f08f5800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f08f015eb30 0x7f08f00b3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58632/0 (socket says 172.21.15.86:58632) 2024-04-20T21:29:42.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.518+0000 7f08f5800640 1 -- 172.21.15.86:0/1847410719 learned_addr learned my addr 172.21.15.86:0/1847410719 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:42.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.519+0000 7f08f6200640 1 --2- 172.21.15.86:0/1847410719 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f08f006c090 0x7f08f00b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:42.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.519+0000 7f08f5800640 1 -- 172.21.15.86:0/1847410719 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f08f006c090 msgr2=0x7f08f00b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:42.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.519+0000 7f08f5800640 1 --2- 172.21.15.86:0/1847410719 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f08f006c090 0x7f08f00b4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:42.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.519+0000 7f08f5800640 1 -- 172.21.15.86:0/1847410719 >> v1:172.21.15.144:6789/0 conn(0x7f08f00aee50 legacy=0x7f08f006b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:42.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.519+0000 7f08f5800640 1 -- 172.21.15.86:0/1847410719 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08f00ae090 con 0x7f08f015eb30 2024-04-20T21:29:42.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.520+0000 7f08f5800640 1 --2- 172.21.15.86:0/1847410719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08f015eb30 0x7f08f00b3e20 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f08e4009870 tx=0x7f08e402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77d0865cd70a3107 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:42.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.520+0000 7f08efe00640 1 -- 172.21.15.86:0/1847410719 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e4035020 con 0x7f08f015eb30 2024-04-20T21:29:42.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.520+0000 7f08efe00640 1 -- 172.21.15.86:0/1847410719 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08e4031640 con 0x7f08f015eb30 2024-04-20T21:29:42.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.520+0000 7f08efe00640 1 -- 172.21.15.86:0/1847410719 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e402d040 con 0x7f08f015eb30 2024-04-20T21:29:42.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.521+0000 7f08f8200640 1 -- 172.21.15.86:0/1847410719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08f015eb30 msgr2=0x7f08f00b3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:42.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.521+0000 7f08f8200640 1 --2- 172.21.15.86:0/1847410719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08f015eb30 0x7f08f00b3e20 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f08e4009870 tx=0x7f08e402b8f0 comp rx=0 tx=0).stop 2024-04-20T21:29:42.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.521+0000 7f08f8200640 1 -- 172.21.15.86:0/1847410719 shutdown_connections 2024-04-20T21:29:42.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.521+0000 7f08f8200640 1 --2- 172.21.15.86:0/1847410719 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f08f006c090 0x7f08f00b4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:42.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.521+0000 7f08f8200640 1 --2- 172.21.15.86:0/1847410719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08f015eb30 0x7f08f00b3e20 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:42.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.521+0000 7f08f8200640 1 -- 172.21.15.86:0/1847410719 >> 172.21.15.86:0/1847410719 conn(0x7f08f00b2110 msgr2=0x7f08f00b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:42.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.521+0000 7f08f8200640 1 -- 172.21.15.86:0/1847410719 shutdown_connections 2024-04-20T21:29:42.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.521+0000 7f08f8200640 1 -- 172.21.15.86:0/1847410719 wait complete. 2024-04-20T21:29:42.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.523+0000 7f08f8200640 1 Processor -- start 2024-04-20T21:29:42.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.523+0000 7f08f8200640 1 -- start start 2024-04-20T21:29:42.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.524+0000 7f08f8200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f08f006c090 0x7f08f01f4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:42.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.524+0000 7f08f8200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08f00aee50 0x7f08f00cd400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:42.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.524+0000 7f08f8200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08f015eb30 0x7f08f015d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:42.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.524+0000 7f08f8200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f08f00ae930 con 0x7f08f00aee50 2024-04-20T21:29:42.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.524+0000 7f08f8200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f08f00ae650 con 0x7f08f015eb30 2024-04-20T21:29:42.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.524+0000 7f08f8200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f08f006e160 con 0x7f08f006c090 2024-04-20T21:29:42.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.524+0000 7f08f4e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08f00aee50 0x7f08f00cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:42.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.524+0000 7f08f4e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08f00aee50 0x7f08f00cd400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58642/0 (socket says 172.21.15.86:58642) 2024-04-20T21:29:42.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.524+0000 7f08f4e00640 1 -- 172.21.15.86:0/477757475 learned_addr learned my addr 172.21.15.86:0/477757475 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:42.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.525+0000 7f08f5800640 1 --2- 172.21.15.86:0/477757475 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f08f006c090 0x7f08f01f4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:42.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.525+0000 7f08f4e00640 1 -- 172.21.15.86:0/477757475 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f08f006c090 msgr2=0x7f08f01f4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:42.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.525+0000 7f08f4e00640 1 --2- 172.21.15.86:0/477757475 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f08f006c090 0x7f08f01f4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:42.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.525+0000 7f08f4e00640 1 -- 172.21.15.86:0/477757475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08f015eb30 msgr2=0x7f08f015d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:42.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.525+0000 7f08f6200640 1 --2- 172.21.15.86:0/477757475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08f015eb30 0x7f08f015d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:42.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.525+0000 7f08f4e00640 1 --2- 172.21.15.86:0/477757475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08f015eb30 0x7f08f015d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:42.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.525+0000 7f08f4e00640 1 -- 172.21.15.86:0/477757475 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08f01f48a0 con 0x7f08f00aee50 2024-04-20T21:29:42.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.525+0000 7f08f4e00640 1 --2- 172.21.15.86:0/477757475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08f00aee50 0x7f08f00cd400 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00c430 tx=0x7f08dc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:42.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.525+0000 7f08ed600640 1 -- 172.21.15.86:0/477757475 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08dc011790 con 0x7f08f00aee50 2024-04-20T21:29:42.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.526+0000 7f08ed600640 1 -- 172.21.15.86:0/477757475 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08dc017070 con 0x7f08f00aee50 2024-04-20T21:29:42.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.526+0000 7f08ed600640 1 -- 172.21.15.86:0/477757475 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08dc011bb0 con 0x7f08f00aee50 2024-04-20T21:29:42.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.526+0000 7f08f8200640 1 -- 172.21.15.86:0/477757475 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08f00ceaa0 con 0x7f08f00aee50 2024-04-20T21:29:42.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.526+0000 7f08f8200640 1 -- 172.21.15.86:0/477757475 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08f00cf060 con 0x7f08f00aee50 2024-04-20T21:29:42.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.527+0000 7f08f8200640 1 -- 172.21.15.86:0/477757475 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08f006ebd0 con 0x7f08f00aee50 2024-04-20T21:29:42.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.528+0000 7f08ed600640 1 -- 172.21.15.86:0/477757475 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f08dc01c460 con 0x7f08f00aee50 2024-04-20T21:29:42.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.529+0000 7f08ed600640 1 --2- 172.21.15.86:0/477757475 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f08d006c8f0 0x7f08d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:42.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.529+0000 7f08ed600640 1 -- 172.21.15.86:0/477757475 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5227+0+0 (secure 0 0 0) 0x7f08dc08b8a0 con 0x7f08f00aee50 2024-04-20T21:29:42.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.530+0000 7f08f5800640 1 --2- 172.21.15.86:0/477757475 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f08d006c8f0 0x7f08d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:42.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.530+0000 7f08f5800640 1 --2- 172.21.15.86:0/477757475 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f08d006c8f0 0x7f08d006edb0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f08e4036000 tx=0x7f08e403c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:42.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.534+0000 7f08ed600640 1 -- 172.21.15.86:0/477757475 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f08f006ebd0 con 0x7f08f00aee50 2024-04-20T21:29:42.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:42.734+0000 7f08f8200640 1 -- 172.21.15.86:0/477757475 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f08f0068a10 con 0x7f08f00aee50 2024-04-20T21:29:43.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.298+0000 7f08ed600640 1 -- 172.21.15.86:0/477757475 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v110) ==== 170+0+0 (secure 0 0 0) 0x7f08f0068a10 con 0x7f08f00aee50 2024-04-20T21:29:43.298 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' created 2024-04-20T21:29:43.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.303+0000 7f08f8200640 1 -- 172.21.15.86:0/477757475 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f08d006c8f0 msgr2=0x7f08d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:43.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.303+0000 7f08f8200640 1 --2- 172.21.15.86:0/477757475 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f08d006c8f0 0x7f08d006edb0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f08e4036000 tx=0x7f08e403c040 comp rx=0 tx=0).stop 2024-04-20T21:29:43.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.303+0000 7f08f8200640 1 -- 172.21.15.86:0/477757475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08f00aee50 msgr2=0x7f08f00cd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:43.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.303+0000 7f08f8200640 1 --2- 172.21.15.86:0/477757475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08f00aee50 0x7f08f00cd400 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00c430 tx=0x7f08dc00c8f0 comp rx=0 tx=0).stop 2024-04-20T21:29:43.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.303+0000 7f08f8200640 1 -- 172.21.15.86:0/477757475 shutdown_connections 2024-04-20T21:29:43.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.303+0000 7f08f8200640 1 --2- 172.21.15.86:0/477757475 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f08d006c8f0 0x7f08d006edb0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:43.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.303+0000 7f08f8200640 1 --2- 172.21.15.86:0/477757475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08f015eb30 0x7f08f015d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:43.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.303+0000 7f08f8200640 1 --2- 172.21.15.86:0/477757475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f08f00aee50 0x7f08f00cd400 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:43.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.303+0000 7f08f8200640 1 --2- 172.21.15.86:0/477757475 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f08f006c090 0x7f08f01f4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:43.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.303+0000 7f08f8200640 1 -- 172.21.15.86:0/477757475 >> 172.21.15.86:0/477757475 conn(0x7f08f00b2110 msgr2=0x7f08f006e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:43.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.304+0000 7f08f8200640 1 -- 172.21.15.86:0/477757475 shutdown_connections 2024-04-20T21:29:43.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.304+0000 7f08f8200640 1 -- 172.21.15.86:0/477757475 wait complete. 2024-04-20T21:29:43.341 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-20T21:29:43.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.515+0000 7f0b81200640 1 Processor -- start 2024-04-20T21:29:43.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.515+0000 7f0b81200640 1 -- start start 2024-04-20T21:29:43.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.516+0000 7f0b81200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0b7c145f00 0x7f0b7c071d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:43.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.516+0000 7f0b81200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b7c072280 0x7f0b7c15cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:43.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.516+0000 7f0b81200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b7c160410 con 0x7f0b7c145250 2024-04-20T21:29:43.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.516+0000 7f0b81200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f0b7c0726f0 con 0x7f0b7c145f00 2024-04-20T21:29:43.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.516+0000 7f0b81200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0b7c1606f0 con 0x7f0b7c072280 2024-04-20T21:29:43.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.517+0000 7f0b7aa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0b7c145f00 0x7f0b7c071d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:43.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.517+0000 7f0b7aa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0b7c145f00 0x7f0b7c071d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58646/0 (socket says 172.21.15.86:58646) 2024-04-20T21:29:43.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.517+0000 7f0b7aa00640 1 -- 172.21.15.86:0/2039236186 learned_addr learned my addr 172.21.15.86:0/2039236186 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:43.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.517+0000 7f0b7be00640 1 --2- 172.21.15.86:0/2039236186 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b7c072280 0x7f0b7c15cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:43.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.517+0000 7f0b7aa00640 1 -- 172.21.15.86:0/2039236186 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b7c072280 msgr2=0x7f0b7c15cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:43.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.517+0000 7f0b7aa00640 1 --2- 172.21.15.86:0/2039236186 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b7c072280 0x7f0b7c15cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:43.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.517+0000 7f0b7aa00640 1 -- 172.21.15.86:0/2039236186 >> v1:172.21.15.144:6789/0 conn(0x7f0b7c145250 legacy=0x7f0b7c06efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:43.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.517+0000 7f0b7aa00640 1 -- 172.21.15.86:0/2039236186 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b7c1609d0 con 0x7f0b7c145f00 2024-04-20T21:29:43.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.517+0000 7f0b7be00640 1 --2- 172.21.15.86:0/2039236186 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b7c072280 0x7f0b7c15cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:29:43.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.518+0000 7f0b7aa00640 1 --2- 172.21.15.86:0/2039236186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b7c145f00 0x7f0b7c071d40 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f0b68009870 tx=0x7f0b6802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d684775dcb77085 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:43.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.519+0000 7f0b7a000640 1 -- 172.21.15.86:0/2039236186 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b680317c0 con 0x7f0b7c145f00 2024-04-20T21:29:43.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.519+0000 7f0b7a000640 1 -- 172.21.15.86:0/2039236186 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b68037070 con 0x7f0b7c145f00 2024-04-20T21:29:43.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.519+0000 7f0b7a000640 1 -- 172.21.15.86:0/2039236186 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b68031b80 con 0x7f0b7c145f00 2024-04-20T21:29:43.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.519+0000 7f0b81200640 1 -- 172.21.15.86:0/2039236186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b7c145f00 msgr2=0x7f0b7c071d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:43.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.519+0000 7f0b81200640 1 --2- 172.21.15.86:0/2039236186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b7c145f00 0x7f0b7c071d40 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f0b68009870 tx=0x7f0b6802b860 comp rx=0 tx=0).stop 2024-04-20T21:29:43.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.519+0000 7f0b81200640 1 -- 172.21.15.86:0/2039236186 shutdown_connections 2024-04-20T21:29:43.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.519+0000 7f0b81200640 1 --2- 172.21.15.86:0/2039236186 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b7c072280 0x7f0b7c15cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:43.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.519+0000 7f0b81200640 1 --2- 172.21.15.86:0/2039236186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b7c145f00 0x7f0b7c071d40 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:43.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.519+0000 7f0b81200640 1 -- 172.21.15.86:0/2039236186 >> 172.21.15.86:0/2039236186 conn(0x7f0b7c067cf0 msgr2=0x7f0b7c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:43.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.520+0000 7f0b81200640 1 -- 172.21.15.86:0/2039236186 shutdown_connections 2024-04-20T21:29:43.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.520+0000 7f0b81200640 1 -- 172.21.15.86:0/2039236186 wait complete. 2024-04-20T21:29:43.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.521+0000 7f0b81200640 1 Processor -- start 2024-04-20T21:29:43.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.521+0000 7f0b81200640 1 -- start start 2024-04-20T21:29:43.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.522+0000 7f0b81200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b7c072280 0x7f0b7c13e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:43.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.522+0000 7f0b81200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b7c145250 0x7f0b7c0cb810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:43.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.522+0000 7f0b81200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b7c145f00 0x7f0b7c0cbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:43.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.522+0000 7f0b81200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c1457b0 con 0x7f0b7c072280 2024-04-20T21:29:43.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.522+0000 7f0b81200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c145630 con 0x7f0b7c145250 2024-04-20T21:29:43.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.522+0000 7f0b81200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0b7c15d330 con 0x7f0b7c145f00 2024-04-20T21:29:43.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.523+0000 7f0b7b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b7c072280 0x7f0b7c13e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:43.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.523+0000 7f0b7b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b7c072280 0x7f0b7c13e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58658/0 (socket says 172.21.15.86:58658) 2024-04-20T21:29:43.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.523+0000 7f0b7b400640 1 -- 172.21.15.86:0/2253880839 learned_addr learned my addr 172.21.15.86:0/2253880839 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:43.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.523+0000 7f0b7b400640 1 -- 172.21.15.86:0/2253880839 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b7c145f00 msgr2=0x7f0b7c0cbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:43.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.523+0000 7f0b7aa00640 1 --2- 172.21.15.86:0/2253880839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b7c145250 0x7f0b7c0cb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:43.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.523+0000 7f0b7be00640 1 --2- 172.21.15.86:0/2253880839 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b7c145f00 0x7f0b7c0cbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:43.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.523+0000 7f0b7b400640 1 --2- 172.21.15.86:0/2253880839 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b7c145f00 0x7f0b7c0cbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:43.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.523+0000 7f0b7b400640 1 -- 172.21.15.86:0/2253880839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b7c145250 msgr2=0x7f0b7c0cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:43.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.523+0000 7f0b7b400640 1 --2- 172.21.15.86:0/2253880839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b7c145250 0x7f0b7c0cb810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:43.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.523+0000 7f0b7b400640 1 -- 172.21.15.86:0/2253880839 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b7c13f130 con 0x7f0b7c072280 2024-04-20T21:29:43.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.524+0000 7f0b7b400640 1 --2- 172.21.15.86:0/2253880839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b7c072280 0x7f0b7c13e620 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00c430 tx=0x7f0b6c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:43.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.524+0000 7f0b73400640 1 -- 172.21.15.86:0/2253880839 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c011790 con 0x7f0b7c072280 2024-04-20T21:29:43.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.524+0000 7f0b73400640 1 -- 172.21.15.86:0/2253880839 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b6c017070 con 0x7f0b7c072280 2024-04-20T21:29:43.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.524+0000 7f0b73400640 1 -- 172.21.15.86:0/2253880839 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c011bb0 con 0x7f0b7c072280 2024-04-20T21:29:43.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.524+0000 7f0b81200640 1 -- 172.21.15.86:0/2253880839 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b7c141940 con 0x7f0b7c072280 2024-04-20T21:29:43.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.524+0000 7f0b81200640 1 -- 172.21.15.86:0/2253880839 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b7c141f00 con 0x7f0b7c072280 2024-04-20T21:29:43.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.525+0000 7f0b81200640 1 -- 172.21.15.86:0/2253880839 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b7c04fd90 con 0x7f0b7c072280 2024-04-20T21:29:43.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.526+0000 7f0b73400640 1 -- 172.21.15.86:0/2253880839 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0b6c01c460 con 0x7f0b7c072280 2024-04-20T21:29:43.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.527+0000 7f0b73400640 1 --2- 172.21.15.86:0/2253880839 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0b5c06c8f0 0x7f0b5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:43.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.527+0000 7f0b73400640 1 -- 172.21.15.86:0/2253880839 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5609+0+0 (secure 0 0 0) 0x7f0b6c08bab0 con 0x7f0b7c072280 2024-04-20T21:29:43.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.527+0000 7f0b7aa00640 1 --2- 172.21.15.86:0/2253880839 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0b5c06c8f0 0x7f0b5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:43.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.528+0000 7f0b7aa00640 1 --2- 172.21.15.86:0/2253880839 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0b5c06c8f0 0x7f0b5c06edb0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f0b68036000 tx=0x7f0b68008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:43.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.533+0000 7f0b73400640 1 -- 172.21.15.86:0/2253880839 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0b6c059ea0 con 0x7f0b7c072280 2024-04-20T21:29:43.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:43.767+0000 7f0b81200640 1 -- 172.21.15.86:0/2253880839 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f0b7c072ea0 con 0x7f0b7c072280 2024-04-20T21:29:44.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.346+0000 7f0b73400640 1 -- 172.21.15.86:0/2253880839 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v95) ==== 174+0+0 (secure 0 0 0) 0x7f0b6c0597a0 con 0x7f0b7c072280 2024-04-20T21:29:44.353 INFO:teuthology.orchestra.run.smithi086.stderr:new fs with metadata pool 16 and data pool 17 2024-04-20T21:29:44.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.362+0000 7f0b81200640 1 -- 172.21.15.86:0/2253880839 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0b5c06c8f0 msgr2=0x7f0b5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:44.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.362+0000 7f0b81200640 1 --2- 172.21.15.86:0/2253880839 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0b5c06c8f0 0x7f0b5c06edb0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f0b68036000 tx=0x7f0b68008030 comp rx=0 tx=0).stop 2024-04-20T21:29:44.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.362+0000 7f0b81200640 1 -- 172.21.15.86:0/2253880839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b7c072280 msgr2=0x7f0b7c13e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:44.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.362+0000 7f0b81200640 1 --2- 172.21.15.86:0/2253880839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b7c072280 0x7f0b7c13e620 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00c430 tx=0x7f0b6c00c8f0 comp rx=0 tx=0).stop 2024-04-20T21:29:44.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.363+0000 7f0b81200640 1 -- 172.21.15.86:0/2253880839 shutdown_connections 2024-04-20T21:29:44.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.363+0000 7f0b81200640 1 --2- 172.21.15.86:0/2253880839 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0b5c06c8f0 0x7f0b5c06edb0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:44.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.363+0000 7f0b81200640 1 --2- 172.21.15.86:0/2253880839 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0b7c145f00 0x7f0b7c0cbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:44.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.363+0000 7f0b81200640 1 --2- 172.21.15.86:0/2253880839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0b7c145250 0x7f0b7c0cb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:44.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.363+0000 7f0b81200640 1 --2- 172.21.15.86:0/2253880839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b7c072280 0x7f0b7c13e620 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:44.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.363+0000 7f0b81200640 1 -- 172.21.15.86:0/2253880839 >> 172.21.15.86:0/2253880839 conn(0x7f0b7c067cf0 msgr2=0x7f0b7c15d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:44.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.364+0000 7f0b81200640 1 -- 172.21.15.86:0/2253880839 shutdown_connections 2024-04-20T21:29:44.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.365+0000 7f0b81200640 1 -- 172.21.15.86:0/2253880839 wait complete. 2024-04-20T21:29:44.395 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:29:44.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.563+0000 7fb89e400640 1 Processor -- start 2024-04-20T21:29:44.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.563+0000 7fb89e400640 1 -- start start 2024-04-20T21:29:44.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.565+0000 7fb89e400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb8980af8d0 0x7fb898141f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:44.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.565+0000 7fb89e400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb898071c10 con 0x7fb8980b3810 2024-04-20T21:29:44.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.565+0000 7fb89e400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb89815eaf0 con 0x7fb898162c40 2024-04-20T21:29:44.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.565+0000 7fb89e400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fb898142490 con 0x7fb8980af8d0 2024-04-20T21:29:44.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.565+0000 7fb89da00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fb8980b3810 0x7fb898072990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40486/0 (socket says 172.21.15.86:40486) 2024-04-20T21:29:44.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.565+0000 7fb89da00640 1 -- 172.21.15.86:0/3339928245 learned_addr learned my addr 172.21.15.86:0/3339928245 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:29:44.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.565+0000 7fb89d000640 1 --2- 172.21.15.86:0/3339928245 >> v2:172.21.15.144:3300/0 conn(0x7fb8980af8d0 0x7fb898141f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:44.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.566+0000 7fb897e00640 1 -- 172.21.15.86:0/3339928245 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2328281744 0 0) 0x7fb898071c10 con 0x7fb8980b3810 2024-04-20T21:29:44.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.566+0000 7fb897e00640 1 -- 172.21.15.86:0/3339928245 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb87c003520 con 0x7fb8980b3810 2024-04-20T21:29:44.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.566+0000 7fb897e00640 1 -- 172.21.15.86:0/3339928245 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1320293129 0 0) 0x7fb87c003520 con 0x7fb8980b3810 2024-04-20T21:29:44.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.566+0000 7fb897e00640 1 -- 172.21.15.86:0/3339928245 >> v2:172.21.15.144:3300/0 conn(0x7fb8980af8d0 msgr2=0x7fb898141f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:44.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.566+0000 7fb897e00640 1 --2- 172.21.15.86:0/3339928245 >> v2:172.21.15.144:3300/0 conn(0x7fb8980af8d0 0x7fb898141f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:44.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.566+0000 7fb897e00640 1 -- 172.21.15.86:0/3339928245 >> v1:172.21.15.144:6789/0 conn(0x7fb898162c40 legacy=0x7fb8980b3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:44.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.566+0000 7fb897e00640 1 -- 172.21.15.86:0/3339928245 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb898071eb0 con 0x7fb8980b3810 2024-04-20T21:29:44.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.566+0000 7fb897e00640 1 -- 172.21.15.86:0/3339928245 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fb890001a10 con 0x7fb8980b3810 2024-04-20T21:29:44.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.566+0000 7fb897e00640 1 -- 172.21.15.86:0/3339928245 >> v1:172.21.15.86:6789/0 conn(0x7fb8980b3810 legacy=0x7fb898072990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:44.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.566+0000 7fb897e00640 1 --2- 172.21.15.86:0/3339928245 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb87c0039b0 0x7fb87c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:44.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.566+0000 7fb897e00640 1 --2- 172.21.15.86:0/3339928245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb87c004800 0x7fb87c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:44.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.566+0000 7fb897e00640 1 --2- 172.21.15.86:0/3339928245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb87c0051c0 0x7fb87c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:44.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.566+0000 7fb897e00640 1 -- 172.21.15.86:0/3339928245 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c00c130 con 0x7fb87c004800 2024-04-20T21:29:44.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.566+0000 7fb897e00640 1 -- 172.21.15.86:0/3339928245 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c00c2e0 con 0x7fb87c0051c0 2024-04-20T21:29:44.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.566+0000 7fb897e00640 1 -- 172.21.15.86:0/3339928245 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb87c00c4e0 con 0x7fb87c0039b0 2024-04-20T21:29:44.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.566+0000 7fb88fe00640 1 --2- 172.21.15.86:0/3339928245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb87c004800 0x7fb87c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:44.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.567+0000 7fb89d000640 1 --2- 172.21.15.86:0/3339928245 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb87c0039b0 0x7fb87c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:44.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.567+0000 7fb88fe00640 1 -- 172.21.15.86:0/3339928245 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb87c0039b0 msgr2=0x7fb87c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:44.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.567+0000 7fb88fe00640 1 --2- 172.21.15.86:0/3339928245 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb87c0039b0 0x7fb87c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:44.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.567+0000 7fb88fe00640 1 -- 172.21.15.86:0/3339928245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb87c0051c0 msgr2=0x7fb87c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:44.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.567+0000 7fb88fe00640 1 --2- 172.21.15.86:0/3339928245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb87c0051c0 0x7fb87c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:44.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.567+0000 7fb88fe00640 1 -- 172.21.15.86:0/3339928245 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb87c003520 con 0x7fb87c004800 2024-04-20T21:29:44.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.567+0000 7fb89da00640 1 -- 172.21.15.86:0/3339928245 reap_dead start 2024-04-20T21:29:44.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.568+0000 7fb88fe00640 1 --2- 172.21.15.86:0/3339928245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb87c004800 0x7fb87c004c80 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fb880001750 tx=0x7fb87c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb8c86b13f8ee371 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:44.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.568+0000 7fb897e00640 1 -- 172.21.15.86:0/3339928245 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb880001370 con 0x7fb87c004800 2024-04-20T21:29:44.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.568+0000 7fb897e00640 1 -- 172.21.15.86:0/3339928245 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8800052d0 con 0x7fb87c004800 2024-04-20T21:29:44.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.568+0000 7fb897e00640 1 -- 172.21.15.86:0/3339928245 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb880003830 con 0x7fb87c004800 2024-04-20T21:29:44.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.569+0000 7fb89e400640 1 -- 172.21.15.86:0/3339928245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb87c004800 msgr2=0x7fb87c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:44.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.569+0000 7fb89e400640 1 --2- 172.21.15.86:0/3339928245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb87c004800 0x7fb87c004c80 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fb880001750 tx=0x7fb87c005830 comp rx=0 tx=0).stop 2024-04-20T21:29:44.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.569+0000 7fb89e400640 1 -- 172.21.15.86:0/3339928245 shutdown_connections 2024-04-20T21:29:44.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.569+0000 7fb89e400640 1 --2- 172.21.15.86:0/3339928245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb87c004800 0x7fb87c004c80 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:44.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.569+0000 7fb89e400640 1 -- 172.21.15.86:0/3339928245 >> 172.21.15.86:0/3339928245 conn(0x7fb8980d0170 msgr2=0x7fb8980d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:44.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.569+0000 7fb89e400640 1 -- 172.21.15.86:0/3339928245 shutdown_connections 2024-04-20T21:29:44.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.569+0000 7fb89e400640 1 -- 172.21.15.86:0/3339928245 wait complete. 2024-04-20T21:29:44.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.571+0000 7fb89e400640 1 Processor -- start 2024-04-20T21:29:44.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.571+0000 7fb89e400640 1 -- start start 2024-04-20T21:29:44.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.572+0000 7fb89e400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb87c004800 0x7fb89815fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:44.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.572+0000 7fb89e400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8981463e0 0x7fb898159330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:44.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.572+0000 7fb89e400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb898146860 0x7fb89814c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:44.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.572+0000 7fb89e400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb898163100 con 0x7fb898146860 2024-04-20T21:29:44.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.572+0000 7fb89e400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb898072470 con 0x7fb87c004800 2024-04-20T21:29:44.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.572+0000 7fb89e400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb898148050 con 0x7fb8981463e0 2024-04-20T21:29:44.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.572+0000 7fb89da00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb898146860 0x7fb89814c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:44.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.572+0000 7fb89da00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb898146860 0x7fb89814c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58666/0 (socket says 172.21.15.86:58666) 2024-04-20T21:29:44.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.572+0000 7fb89da00640 1 -- 172.21.15.86:0/2872124177 learned_addr learned my addr 172.21.15.86:0/2872124177 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:44.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.572+0000 7fb88fe00640 1 --2- 172.21.15.86:0/2872124177 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8981463e0 0x7fb898159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:44.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.572+0000 7fb89d000640 1 --2- 172.21.15.86:0/2872124177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb87c004800 0x7fb89815fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:44.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.572+0000 7fb89da00640 1 -- 172.21.15.86:0/2872124177 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8981463e0 msgr2=0x7fb898159330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:44.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.572+0000 7fb89da00640 1 --2- 172.21.15.86:0/2872124177 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8981463e0 0x7fb898159330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:44.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.572+0000 7fb89da00640 1 -- 172.21.15.86:0/2872124177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb87c004800 msgr2=0x7fb89815fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:44.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.572+0000 7fb89da00640 1 --2- 172.21.15.86:0/2872124177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb87c004800 0x7fb89815fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:44.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.572+0000 7fb89da00640 1 -- 172.21.15.86:0/2872124177 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb898159d80 con 0x7fb898146860 2024-04-20T21:29:44.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.572+0000 7fb88fe00640 1 --2- 172.21.15.86:0/2872124177 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8981463e0 0x7fb898159330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:29:44.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.573+0000 7fb89da00640 1 --2- 172.21.15.86:0/2872124177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb898146860 0x7fb89814c750 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fb890001560 tx=0x7fb89000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:44.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.573+0000 7fb895600640 1 -- 172.21.15.86:0/2872124177 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89000a610 con 0x7fb898146860 2024-04-20T21:29:44.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.573+0000 7fb895600640 1 -- 172.21.15.86:0/2872124177 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb89000a7b0 con 0x7fb898146860 2024-04-20T21:29:44.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.573+0000 7fb895600640 1 -- 172.21.15.86:0/2872124177 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89000aba0 con 0x7fb898146860 2024-04-20T21:29:44.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.573+0000 7fb89e400640 1 -- 172.21.15.86:0/2872124177 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb89815ef80 con 0x7fb898146860 2024-04-20T21:29:44.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.573+0000 7fb89e400640 1 -- 172.21.15.86:0/2872124177 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb89815ffa0 con 0x7fb898146860 2024-04-20T21:29:44.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.574+0000 7fb89e400640 1 -- 172.21.15.86:0/2872124177 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb89804fcf0 con 0x7fb898146860 2024-04-20T21:29:44.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.575+0000 7fb895600640 1 -- 172.21.15.86:0/2872124177 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb890011070 con 0x7fb898146860 2024-04-20T21:29:44.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.576+0000 7fb895600640 1 --2- 172.21.15.86:0/2872124177 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb87406c920 0x7fb87406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:44.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.576+0000 7fb895600640 1 -- 172.21.15.86:0/2872124177 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5709+0+0 (secure 0 0 0) 0x7fb89008b2c0 con 0x7fb898146860 2024-04-20T21:29:44.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.576+0000 7fb89d000640 1 --2- 172.21.15.86:0/2872124177 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb87406c920 0x7fb87406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:44.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.577+0000 7fb89d000640 1 --2- 172.21.15.86:0/2872124177 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb87406c920 0x7fb87406ede0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fb89815df40 tx=0x7fb888008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:44.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.581+0000 7fb895600640 1 -- 172.21.15.86:0/2872124177 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb89001a060 con 0x7fb898146860 2024-04-20T21:29:44.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.781+0000 7fb89e400640 1 -- 172.21.15.86:0/2872124177 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb8981622e0 con 0x7fb898146860 2024-04-20T21:29:44.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.782+0000 7fb895600640 1 -- 172.21.15.86:0/2872124177 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) ==== 74+0+16640 (secure 0 0 0) 0x7fb89000f260 con 0x7fb898146860 2024-04-20T21:29:44.782 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:29:44.783 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":111,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:29:43.769676+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:29:41.532824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-20T21:29:42.749687+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:29:44.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.785+0000 7fb89e400640 1 -- 172.21.15.86:0/2872124177 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb87406c920 msgr2=0x7fb87406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:44.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.785+0000 7fb89e400640 1 --2- 172.21.15.86:0/2872124177 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb87406c920 0x7fb87406ede0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fb89815df40 tx=0x7fb888008040 comp rx=0 tx=0).stop 2024-04-20T21:29:44.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.785+0000 7fb89e400640 1 -- 172.21.15.86:0/2872124177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb898146860 msgr2=0x7fb89814c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:44.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.785+0000 7fb89e400640 1 --2- 172.21.15.86:0/2872124177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb898146860 0x7fb89814c750 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fb890001560 tx=0x7fb89000ccb0 comp rx=0 tx=0).stop 2024-04-20T21:29:44.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.785+0000 7fb89e400640 1 -- 172.21.15.86:0/2872124177 shutdown_connections 2024-04-20T21:29:44.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.785+0000 7fb89e400640 1 --2- 172.21.15.86:0/2872124177 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb87406c920 0x7fb87406ede0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:44.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.785+0000 7fb89e400640 1 --2- 172.21.15.86:0/2872124177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb898146860 0x7fb89814c750 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:44.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.785+0000 7fb89e400640 1 --2- 172.21.15.86:0/2872124177 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb8981463e0 0x7fb898159330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:44.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.785+0000 7fb89e400640 1 --2- 172.21.15.86:0/2872124177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb87c004800 0x7fb89815fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:44.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.785+0000 7fb89e400640 1 -- 172.21.15.86:0/2872124177 >> 172.21.15.86:0/2872124177 conn(0x7fb8980d0170 msgr2=0x7fb89815cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:44.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.785+0000 7fb89e400640 1 -- 172.21.15.86:0/2872124177 shutdown_connections 2024-04-20T21:29:44.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.785+0000 7fb89e400640 1 -- 172.21.15.86:0/2872124177 wait complete. 2024-04-20T21:29:44.803 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:29:44.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.990+0000 7fb4afe00640 1 Processor -- start 2024-04-20T21:29:44.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.990+0000 7fb4afe00640 1 -- start start 2024-04-20T21:29:44.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.992+0000 7fb4afe00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb4b0156ae0 0x7fb4b015d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:44.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.992+0000 7fb4afe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4b0160080 0x7fb4b00ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:44.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.992+0000 7fb4afe00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4b0160890 con 0x7fb4b00ad620 2024-04-20T21:29:44.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.992+0000 7fb4afe00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fb4b006db20 con 0x7fb4b0156ae0 2024-04-20T21:29:44.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.992+0000 7fb4afe00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb4b0160b70 con 0x7fb4b0160080 2024-04-20T21:29:44.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.992+0000 7fb4af400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fb4b00ad620 0x7fb4b0141cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:40490/0 (socket says 172.21.15.86:40490) 2024-04-20T21:29:44.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.992+0000 7fb4af400640 1 -- 172.21.15.86:0/2107270835 learned_addr learned my addr 172.21.15.86:0/2107270835 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:29:44.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.992+0000 7fb4ae000640 1 --2- 172.21.15.86:0/2107270835 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4b0160080 0x7fb4b00ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:44.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.992+0000 7fb4aea00640 1 --2- 172.21.15.86:0/2107270835 >> v2:172.21.15.144:3300/0 conn(0x7fb4b0156ae0 0x7fb4b015d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:44.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.992+0000 7fb4ad600640 1 -- 172.21.15.86:0/2107270835 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2679979574 0 0) 0x7fb4b0160890 con 0x7fb4b00ad620 2024-04-20T21:29:44.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.992+0000 7fb4ad600640 1 -- 172.21.15.86:0/2107270835 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb494003520 con 0x7fb4b00ad620 2024-04-20T21:29:44.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.993+0000 7fb4ad600640 1 -- 172.21.15.86:0/2107270835 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3581978352 0 0) 0x7fb494003520 con 0x7fb4b00ad620 2024-04-20T21:29:44.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.993+0000 7fb4ad600640 1 -- 172.21.15.86:0/2107270835 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4b0160080 msgr2=0x7fb4b00ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:44.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.993+0000 7fb4ad600640 1 --2- 172.21.15.86:0/2107270835 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4b0160080 0x7fb4b00ad0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:44.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.993+0000 7fb4ad600640 1 -- 172.21.15.86:0/2107270835 >> v2:172.21.15.144:3300/0 conn(0x7fb4b0156ae0 msgr2=0x7fb4b015d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:44.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.993+0000 7fb4ad600640 1 --2- 172.21.15.86:0/2107270835 >> v2:172.21.15.144:3300/0 conn(0x7fb4b0156ae0 0x7fb4b015d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:44.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.993+0000 7fb4ad600640 1 -- 172.21.15.86:0/2107270835 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4b0160e50 con 0x7fb4b00ad620 2024-04-20T21:29:44.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.993+0000 7fb4ad600640 1 -- 172.21.15.86:0/2107270835 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fb4a80019f0 con 0x7fb4b00ad620 2024-04-20T21:29:44.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.993+0000 7fb4ad600640 1 -- 172.21.15.86:0/2107270835 >> v1:172.21.15.86:6789/0 conn(0x7fb4b00ad620 legacy=0x7fb4b0141cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:44.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.993+0000 7fb4ad600640 1 --2- 172.21.15.86:0/2107270835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4940039b0 0x7fb494003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:44.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.993+0000 7fb4ad600640 1 --2- 172.21.15.86:0/2107270835 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4b0160080 0x7fb494004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:44.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.993+0000 7fb4ad600640 1 --2- 172.21.15.86:0/2107270835 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb494004d40 0x7fb4940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:44.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.993+0000 7fb4ad600640 1 -- 172.21.15.86:0/2107270835 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b006db20 con 0x7fb4940039b0 2024-04-20T21:29:44.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.993+0000 7fb4ad600640 1 -- 172.21.15.86:0/2107270835 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b0160b70 con 0x7fb494004d40 2024-04-20T21:29:44.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.993+0000 7fb4ad600640 1 -- 172.21.15.86:0/2107270835 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb49400ba70 con 0x7fb4b0160080 2024-04-20T21:29:44.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.993+0000 7fb4aea00640 1 --2- 172.21.15.86:0/2107270835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4940039b0 0x7fb494003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:44.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.993+0000 7fb4ae000640 1 --2- 172.21.15.86:0/2107270835 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4b0160080 0x7fb494004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:44.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.994+0000 7fb4af400640 1 --2- 172.21.15.86:0/2107270835 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb494004d40 0x7fb4940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:44.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.994+0000 7fb4aea00640 1 -- 172.21.15.86:0/2107270835 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4b0160080 msgr2=0x7fb494004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:44.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.994+0000 7fb4aea00640 1 --2- 172.21.15.86:0/2107270835 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4b0160080 0x7fb494004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:44.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.994+0000 7fb4aea00640 1 -- 172.21.15.86:0/2107270835 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb494004d40 msgr2=0x7fb4940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:44.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.994+0000 7fb4aea00640 1 --2- 172.21.15.86:0/2107270835 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb494004d40 0x7fb4940051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:44.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.994+0000 7fb4aea00640 1 -- 172.21.15.86:0/2107270835 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb494003520 con 0x7fb4940039b0 2024-04-20T21:29:44.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.995+0000 7fb4aea00640 1 --2- 172.21.15.86:0/2107270835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4940039b0 0x7fb494003dd0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fb4a000ba60 tx=0x7fb494005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a28f4a555077af81 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:45.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.995+0000 7fb4ad600640 1 -- 172.21.15.86:0/2107270835 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a0036020 con 0x7fb4940039b0 2024-04-20T21:29:45.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.995+0000 7fb4ad600640 1 -- 172.21.15.86:0/2107270835 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4a0032680 con 0x7fb4940039b0 2024-04-20T21:29:45.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.995+0000 7fb4ad600640 1 -- 172.21.15.86:0/2107270835 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a000b600 con 0x7fb4940039b0 2024-04-20T21:29:45.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.995+0000 7fb4afe00640 1 -- 172.21.15.86:0/2107270835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4940039b0 msgr2=0x7fb494003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:45.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.995+0000 7fb4afe00640 1 --2- 172.21.15.86:0/2107270835 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4940039b0 0x7fb494003dd0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fb4a000ba60 tx=0x7fb494005830 comp rx=0 tx=0).stop 2024-04-20T21:29:45.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.995+0000 7fb4af400640 1 -- 172.21.15.86:0/2107270835 reap_dead start 2024-04-20T21:29:45.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.996+0000 7fb4afe00640 1 -- 172.21.15.86:0/2107270835 shutdown_connections 2024-04-20T21:29:45.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.996+0000 7fb4afe00640 1 -- 172.21.15.86:0/2107270835 >> 172.21.15.86:0/2107270835 conn(0x7fb4b00d0170 msgr2=0x7fb4b00d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:45.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.996+0000 7fb4afe00640 1 -- 172.21.15.86:0/2107270835 shutdown_connections 2024-04-20T21:29:45.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.996+0000 7fb4afe00640 1 -- 172.21.15.86:0/2107270835 wait complete. 2024-04-20T21:29:45.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.997+0000 7fb4afe00640 1 Processor -- start 2024-04-20T21:29:45.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.998+0000 7fb4afe00640 1 -- start start 2024-04-20T21:29:45.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.998+0000 7fb4afe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4940039b0 0x7fb4b01f3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:45.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.998+0000 7fb4afe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4b00b4070 0x7fb4b00b44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:45.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.998+0000 7fb4afe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4b00b4a30 0x7fb4b015ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:45.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.998+0000 7fb4afe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b0160500 con 0x7fb4b00b4070 2024-04-20T21:29:45.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.998+0000 7fb4afe00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b0156ec0 con 0x7fb4940039b0 2024-04-20T21:29:45.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.998+0000 7fb4afe00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb4b006f2a0 con 0x7fb4b00b4a30 2024-04-20T21:29:45.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.999+0000 7fb4ae000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4b00b4070 0x7fb4b00b44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:45.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.999+0000 7fb4ae000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4b00b4070 0x7fb4b00b44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58678/0 (socket says 172.21.15.86:58678) 2024-04-20T21:29:45.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.999+0000 7fb4ae000640 1 -- 172.21.15.86:0/3551717183 learned_addr learned my addr 172.21.15.86:0/3551717183 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:45.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.999+0000 7fb4af400640 1 --2- 172.21.15.86:0/3551717183 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4b00b4a30 0x7fb4b015ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:45.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.999+0000 7fb4ae000640 1 -- 172.21.15.86:0/3551717183 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4b00b4a30 msgr2=0x7fb4b015ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:45.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.999+0000 7fb4ae000640 1 --2- 172.21.15.86:0/3551717183 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4b00b4a30 0x7fb4b015ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:45.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.999+0000 7fb4ae000640 1 -- 172.21.15.86:0/3551717183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4940039b0 msgr2=0x7fb4b01f3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:45.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.999+0000 7fb4aea00640 1 --2- 172.21.15.86:0/3551717183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4940039b0 0x7fb4b01f3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:45.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.999+0000 7fb4ae000640 1 --2- 172.21.15.86:0/3551717183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4940039b0 0x7fb4b01f3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:45.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.999+0000 7fb4ae000640 1 -- 172.21.15.86:0/3551717183 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4b00b4eb0 con 0x7fb4b00b4070 2024-04-20T21:29:45.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:44.999+0000 7fb4ae000640 1 --2- 172.21.15.86:0/3551717183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4b00b4070 0x7fb4b00b44f0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7fb49c00e810 tx=0x7fb49c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:45.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.000+0000 7fb4a6a00640 1 -- 172.21.15.86:0/3551717183 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49c017070 con 0x7fb4b00b4070 2024-04-20T21:29:45.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.000+0000 7fb4a6a00640 1 -- 172.21.15.86:0/3551717183 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb49c00c670 con 0x7fb4b00b4070 2024-04-20T21:29:45.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.000+0000 7fb4a6a00640 1 -- 172.21.15.86:0/3551717183 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49c00f040 con 0x7fb4b00b4070 2024-04-20T21:29:45.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.000+0000 7fb4afe00640 1 -- 172.21.15.86:0/3551717183 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4b01588a0 con 0x7fb4b00b4070 2024-04-20T21:29:45.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.000+0000 7fb4afe00640 1 -- 172.21.15.86:0/3551717183 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4b01582e0 con 0x7fb4b00b4070 2024-04-20T21:29:45.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.001+0000 7fb4afe00640 1 -- 172.21.15.86:0/3551717183 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4b004fcf0 con 0x7fb4b00b4070 2024-04-20T21:29:45.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.002+0000 7fb4a6a00640 1 -- 172.21.15.86:0/3551717183 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb49c01a070 con 0x7fb4b00b4070 2024-04-20T21:29:45.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.003+0000 7fb4a6a00640 1 --2- 172.21.15.86:0/3551717183 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb49006c8f0 0x7fb49006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:45.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.003+0000 7fb4a6a00640 1 -- 172.21.15.86:0/3551717183 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5709+0+0 (secure 0 0 0) 0x7fb49c08b780 con 0x7fb4b00b4070 2024-04-20T21:29:45.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.003+0000 7fb4aea00640 1 --2- 172.21.15.86:0/3551717183 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb49006c8f0 0x7fb49006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:45.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.004+0000 7fb4aea00640 1 --2- 172.21.15.86:0/3551717183 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb49006c8f0 0x7fb49006edb0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fb4a000ba30 tx=0x7fb4a0030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:45.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.007+0000 7fb4a6a00640 1 -- 172.21.15.86:0/3551717183 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb49c059ba0 con 0x7fb4b00b4070 2024-04-20T21:29:45.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.192+0000 7fb4afe00640 1 -- 172.21.15.86:0/3551717183 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb4b0072560 con 0x7fb4b00b4070 2024-04-20T21:29:45.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.194+0000 7fb4a6a00640 1 -- 172.21.15.86:0/3551717183 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) ==== 74+0+16640 (secure 0 0 0) 0x7fb49c0594a0 con 0x7fb4b00b4070 2024-04-20T21:29:45.194 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:29:45.194 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":111,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:29:43.769676+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:29:41.532824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-20T21:29:42.749687+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:29:45.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.196+0000 7fb4afe00640 1 -- 172.21.15.86:0/3551717183 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb49006c8f0 msgr2=0x7fb49006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:45.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.196+0000 7fb4afe00640 1 --2- 172.21.15.86:0/3551717183 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb49006c8f0 0x7fb49006edb0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fb4a000ba30 tx=0x7fb4a0030000 comp rx=0 tx=0).stop 2024-04-20T21:29:45.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.196+0000 7fb4afe00640 1 -- 172.21.15.86:0/3551717183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4b00b4070 msgr2=0x7fb4b00b44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:45.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.196+0000 7fb4afe00640 1 --2- 172.21.15.86:0/3551717183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4b00b4070 0x7fb4b00b44f0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7fb49c00e810 tx=0x7fb49c00ecd0 comp rx=0 tx=0).stop 2024-04-20T21:29:45.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.196+0000 7fb4afe00640 1 -- 172.21.15.86:0/3551717183 shutdown_connections 2024-04-20T21:29:45.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.196+0000 7fb4afe00640 1 --2- 172.21.15.86:0/3551717183 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb49006c8f0 0x7fb49006edb0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:45.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.196+0000 7fb4afe00640 1 --2- 172.21.15.86:0/3551717183 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb4b00b4a30 0x7fb4b015ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:45.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.196+0000 7fb4afe00640 1 --2- 172.21.15.86:0/3551717183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb4b00b4070 0x7fb4b00b44f0 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:45.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.196+0000 7fb4afe00640 1 --2- 172.21.15.86:0/3551717183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4940039b0 0x7fb4b01f3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:45.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.196+0000 7fb4afe00640 1 -- 172.21.15.86:0/3551717183 >> 172.21.15.86:0/3551717183 conn(0x7fb4b00d0170 msgr2=0x7fb4b015cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:45.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.197+0000 7fb4afe00640 1 -- 172.21.15.86:0/3551717183 shutdown_connections 2024-04-20T21:29:45.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.197+0000 7fb4afe00640 1 -- 172.21.15.86:0/3551717183 wait complete. 2024-04-20T21:29:45.213 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-20T21:29:45.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.418+0000 7f650d400640 1 Processor -- start 2024-04-20T21:29:45.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.419+0000 7f650d400640 1 -- start start 2024-04-20T21:29:45.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.420+0000 7f650d400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f65080d45a0 0x7f65080d3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:45.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.420+0000 7f650d400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f650815ea70 0x7f65080d4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:45.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.420+0000 7f650d400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f650806d570 0x7f65080d1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:45.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.420+0000 7f650d400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f65080cf120 con 0x7f65080d45a0 2024-04-20T21:29:45.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.420+0000 7f650d400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f650806de70 con 0x7f650815ea70 2024-04-20T21:29:45.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.420+0000 7f650d400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f650806e150 con 0x7f650806d570 2024-04-20T21:29:45.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.420+0000 7f6507e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f65080d45a0 0x7f65080d3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:45.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.420+0000 7f6507e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f65080d45a0 0x7f65080d3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58694/0 (socket says 172.21.15.86:58694) 2024-04-20T21:29:45.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.420+0000 7f6507e00640 1 -- 172.21.15.86:0/2196104753 learned_addr learned my addr 172.21.15.86:0/2196104753 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:45.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.421+0000 7f6507e00640 1 -- 172.21.15.86:0/2196104753 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f650806d570 msgr2=0x7f65080d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:45.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.421+0000 7f6507e00640 1 --2- 172.21.15.86:0/2196104753 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f650806d570 0x7f65080d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:45.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.421+0000 7f6507e00640 1 -- 172.21.15.86:0/2196104753 >> v2:172.21.15.144:3300/0 conn(0x7f650815ea70 msgr2=0x7f65080d4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:45.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.421+0000 7f6507e00640 1 --2- 172.21.15.86:0/2196104753 >> v2:172.21.15.144:3300/0 conn(0x7f650815ea70 0x7f65080d4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:45.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.421+0000 7f6507e00640 1 -- 172.21.15.86:0/2196104753 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f650806e430 con 0x7f65080d45a0 2024-04-20T21:29:45.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.422+0000 7f6507e00640 1 --2- 172.21.15.86:0/2196104753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65080d45a0 0x7f65080d3260 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f65080b4120 tx=0x7f64f802b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cc4cc27255ba97e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:45.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.422+0000 7f6506a00640 1 -- 172.21.15.86:0/2196104753 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f80317d0 con 0x7f65080d45a0 2024-04-20T21:29:45.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.422+0000 7f6506a00640 1 -- 172.21.15.86:0/2196104753 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64f8037070 con 0x7f65080d45a0 2024-04-20T21:29:45.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.422+0000 7f6506a00640 1 -- 172.21.15.86:0/2196104753 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f8031b90 con 0x7f65080d45a0 2024-04-20T21:29:45.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.422+0000 7f650d400640 1 -- 172.21.15.86:0/2196104753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65080d45a0 msgr2=0x7f65080d3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:45.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.423+0000 7f650d400640 1 --2- 172.21.15.86:0/2196104753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65080d45a0 0x7f65080d3260 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f65080b4120 tx=0x7f64f802b870 comp rx=0 tx=0).stop 2024-04-20T21:29:45.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.423+0000 7f650d400640 1 -- 172.21.15.86:0/2196104753 shutdown_connections 2024-04-20T21:29:45.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.423+0000 7f650d400640 1 --2- 172.21.15.86:0/2196104753 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f650806d570 0x7f65080d1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:45.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.423+0000 7f650d400640 1 --2- 172.21.15.86:0/2196104753 >> v2:172.21.15.144:3300/0 conn(0x7f650815ea70 0x7f65080d4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:45.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.423+0000 7f650d400640 1 --2- 172.21.15.86:0/2196104753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f65080d45a0 0x7f65080d3260 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:45.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.423+0000 7f650d400640 1 -- 172.21.15.86:0/2196104753 >> 172.21.15.86:0/2196104753 conn(0x7f650809d9d0 msgr2=0x7f650808bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:45.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.423+0000 7f650d400640 1 -- 172.21.15.86:0/2196104753 shutdown_connections 2024-04-20T21:29:45.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.423+0000 7f650d400640 1 -- 172.21.15.86:0/2196104753 wait complete. 2024-04-20T21:29:45.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.424+0000 7f650d400640 1 Processor -- start 2024-04-20T21:29:45.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.424+0000 7f650d400640 1 -- start start 2024-04-20T21:29:45.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.425+0000 7f650d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f650806d570 0x7f6508160420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:45.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.425+0000 7f650d400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65080d45a0 0x7f65080c9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:45.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.425+0000 7f650d400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f650815ea70 0x7f65080ca730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:45.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.425+0000 7f650d400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f650806d9e0 con 0x7f650806d570 2024-04-20T21:29:45.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.425+0000 7f650d400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f65080d4ec0 con 0x7f65080d45a0 2024-04-20T21:29:45.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.425+0000 7f650d400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f650806db60 con 0x7f650815ea70 2024-04-20T21:29:45.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.426+0000 7f6507e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f650806d570 0x7f6508160420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:45.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.426+0000 7f6507e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f650806d570 0x7f6508160420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58700/0 (socket says 172.21.15.86:58700) 2024-04-20T21:29:45.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.426+0000 7f6507e00640 1 -- 172.21.15.86:0/280308961 learned_addr learned my addr 172.21.15.86:0/280308961 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:45.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.426+0000 7f6507e00640 1 -- 172.21.15.86:0/280308961 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f650815ea70 msgr2=0x7f65080ca730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:45.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.426+0000 7f6507400640 1 --2- 172.21.15.86:0/280308961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65080d45a0 0x7f65080c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:45.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.426+0000 7f6507e00640 1 --2- 172.21.15.86:0/280308961 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f650815ea70 0x7f65080ca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:45.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.426+0000 7f6507e00640 1 -- 172.21.15.86:0/280308961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65080d45a0 msgr2=0x7f65080c9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:45.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.426+0000 7f6507e00640 1 --2- 172.21.15.86:0/280308961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65080d45a0 0x7f65080c9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:45.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.426+0000 7f6507e00640 1 -- 172.21.15.86:0/280308961 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65080c8350 con 0x7f650806d570 2024-04-20T21:29:45.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.426+0000 7f6507e00640 1 --2- 172.21.15.86:0/280308961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f650806d570 0x7f6508160420 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f64f802b8b0 tx=0x7f64f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:45.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.427+0000 7f64ffe00640 1 -- 172.21.15.86:0/280308961 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f8032870 con 0x7f650806d570 2024-04-20T21:29:45.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.427+0000 7f64ffe00640 1 -- 172.21.15.86:0/280308961 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64f8032a10 con 0x7f650806d570 2024-04-20T21:29:45.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.427+0000 7f64ffe00640 1 -- 172.21.15.86:0/280308961 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f8032e00 con 0x7f650806d570 2024-04-20T21:29:45.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.427+0000 7f650d400640 1 -- 172.21.15.86:0/280308961 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65080c8630 con 0x7f650806d570 2024-04-20T21:29:45.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.427+0000 7f650d400640 1 -- 172.21.15.86:0/280308961 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6508161b00 con 0x7f650806d570 2024-04-20T21:29:45.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.428+0000 7f650d400640 1 -- 172.21.15.86:0/280308961 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f650804fd90 con 0x7f650806d570 2024-04-20T21:29:45.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.429+0000 7f64ffe00640 1 -- 172.21.15.86:0/280308961 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f64f804c060 con 0x7f650806d570 2024-04-20T21:29:45.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.430+0000 7f64ffe00640 1 --2- 172.21.15.86:0/280308961 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f64e806c920 0x7f64e806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:45.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.430+0000 7f64ffe00640 1 -- 172.21.15.86:0/280308961 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7f64f80ad710 con 0x7f650806d570 2024-04-20T21:29:45.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.430+0000 7f6507400640 1 --2- 172.21.15.86:0/280308961 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f64e806c920 0x7f64e806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:45.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.431+0000 7f6507400640 1 --2- 172.21.15.86:0/280308961 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f64e806c920 0x7f64e806ede0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f65081f2480 tx=0x7f64f4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:45.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.434+0000 7f64ffe00640 1 -- 172.21.15.86:0/280308961 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f64f8040070 con 0x7f650806d570 2024-04-20T21:29:45.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:45.656+0000 7f650d400640 1 -- 172.21.15.86:0/280308961 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f65080ca510 con 0x7f650806d570 2024-04-20T21:29:46.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.360+0000 7f64ffe00640 1 -- 172.21.15.86:0/280308961 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v98) ==== 118+0+0 (secure 0 0 0) 0x7f64f807bb30 con 0x7f650806d570 2024-04-20T21:29:46.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.363+0000 7f650d400640 1 -- 172.21.15.86:0/280308961 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f64e806c920 msgr2=0x7f64e806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:46.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.363+0000 7f650d400640 1 --2- 172.21.15.86:0/280308961 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f64e806c920 0x7f64e806ede0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f65081f2480 tx=0x7f64f4008040 comp rx=0 tx=0).stop 2024-04-20T21:29:46.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.363+0000 7f650d400640 1 -- 172.21.15.86:0/280308961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f650806d570 msgr2=0x7f6508160420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:46.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.363+0000 7f650d400640 1 --2- 172.21.15.86:0/280308961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f650806d570 0x7f6508160420 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f64f802b8b0 tx=0x7f64f80046c0 comp rx=0 tx=0).stop 2024-04-20T21:29:46.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.364+0000 7f650d400640 1 -- 172.21.15.86:0/280308961 shutdown_connections 2024-04-20T21:29:46.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.364+0000 7f650d400640 1 --2- 172.21.15.86:0/280308961 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f64e806c920 0x7f64e806ede0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:46.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.364+0000 7f650d400640 1 --2- 172.21.15.86:0/280308961 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f650815ea70 0x7f65080ca730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:46.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.364+0000 7f650d400640 1 --2- 172.21.15.86:0/280308961 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65080d45a0 0x7f65080c9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:46.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.364+0000 7f650d400640 1 --2- 172.21.15.86:0/280308961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f650806d570 0x7f6508160420 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:46.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.364+0000 7f650d400640 1 -- 172.21.15.86:0/280308961 >> 172.21.15.86:0/280308961 conn(0x7f650809d9d0 msgr2=0x7f65080b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:46.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.364+0000 7f650d400640 1 -- 172.21.15.86:0/280308961 shutdown_connections 2024-04-20T21:29:46.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.364+0000 7f650d400640 1 -- 172.21.15.86:0/280308961 wait complete. 2024-04-20T21:29:46.386 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-20T21:29:46.387 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-20T21:29:46.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.593+0000 7fd801400640 1 Processor -- start 2024-04-20T21:29:46.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.594+0000 7fd801400640 1 -- start start 2024-04-20T21:29:46.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.595+0000 7fd801400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd7fc15f990 0x7fd7fc06dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:46.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.595+0000 7fd801400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7fc071d40 0x7fd7fc0721d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:46.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.595+0000 7fd801400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7fc0aecd0 con 0x7fd7fc161c60 2024-04-20T21:29:46.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.595+0000 7fd801400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fd7fc06e2e0 con 0x7fd7fc15f990 2024-04-20T21:29:46.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.595+0000 7fd801400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd7fc06e580 con 0x7fd7fc071d40 2024-04-20T21:29:46.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.596+0000 7fd7f3400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd7fc15f990 0x7fd7fc06dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:46.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.596+0000 7fd7fbe00640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fd7fc161c60 0x7fd7fc15ff10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.86:46068/0 (socket says 172.21.15.86:46068) 2024-04-20T21:29:46.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.596+0000 7fd7fbe00640 1 -- 172.21.15.86:0/3493420386 learned_addr learned my addr 172.21.15.86:0/3493420386 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:29:46.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.596+0000 7fd7f3400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fd7fc15f990 0x7fd7fc06dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.86:42824/0 (socket says 172.21.15.86:42824) 2024-04-20T21:29:46.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.596+0000 7fd800a00640 1 --2- 172.21.15.86:0/3493420386 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7fc071d40 0x7fd7fc0721d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:46.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.597+0000 7fd7fb400640 1 -- 172.21.15.86:0/3493420386 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3274859323 0 0) 0x7fd7fc0aecd0 con 0x7fd7fc161c60 2024-04-20T21:29:46.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.597+0000 7fd7fb400640 1 -- 172.21.15.86:0/3493420386 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd7e0003520 con 0x7fd7fc161c60 2024-04-20T21:29:46.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.597+0000 7fd800a00640 1 -- 172.21.15.86:0/3493420386 >> v2:172.21.15.144:3300/0 conn(0x7fd7fc15f990 msgr2=0x7fd7fc06dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:46.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.597+0000 7fd800a00640 1 --2- 172.21.15.86:0/3493420386 >> v2:172.21.15.144:3300/0 conn(0x7fd7fc15f990 0x7fd7fc06dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:46.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.597+0000 7fd800a00640 1 -- 172.21.15.86:0/3493420386 >> v1:172.21.15.144:6789/0 conn(0x7fd7fc161c60 legacy=0x7fd7fc15ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:46.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.597+0000 7fd800a00640 1 -- 172.21.15.86:0/3493420386 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7fc06e860 con 0x7fd7fc071d40 2024-04-20T21:29:46.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.599+0000 7fd800a00640 1 --2- 172.21.15.86:0/3493420386 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7fc071d40 0x7fd7fc0721d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4009d90 tx=0x7fd7f402bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fd0848306160b36 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:46.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.599+0000 7fd7fb400640 1 -- 172.21.15.86:0/3493420386 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f4039070 con 0x7fd7fc071d40 2024-04-20T21:29:46.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.599+0000 7fd7fb400640 1 -- 172.21.15.86:0/3493420386 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7f402cc50 con 0x7fd7fc071d40 2024-04-20T21:29:46.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.599+0000 7fd7fb400640 1 -- 172.21.15.86:0/3493420386 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f4034440 con 0x7fd7fc071d40 2024-04-20T21:29:46.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.600+0000 7fd801400640 1 -- 172.21.15.86:0/3493420386 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7fc071d40 msgr2=0x7fd7fc0721d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:46.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.600+0000 7fd801400640 1 --2- 172.21.15.86:0/3493420386 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7fc071d40 0x7fd7fc0721d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4009d90 tx=0x7fd7f402bea0 comp rx=0 tx=0).stop 2024-04-20T21:29:46.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.600+0000 7fd801400640 1 -- 172.21.15.86:0/3493420386 shutdown_connections 2024-04-20T21:29:46.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.600+0000 7fd801400640 1 --2- 172.21.15.86:0/3493420386 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7fc071d40 0x7fd7fc0721d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:46.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.600+0000 7fd801400640 1 --2- 172.21.15.86:0/3493420386 >> v2:172.21.15.144:3300/0 conn(0x7fd7fc15f990 0x7fd7fc06dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:46.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.600+0000 7fd801400640 1 -- 172.21.15.86:0/3493420386 >> 172.21.15.86:0/3493420386 conn(0x7fd7fc067cf0 msgr2=0x7fd7fc06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:46.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.600+0000 7fd801400640 1 -- 172.21.15.86:0/3493420386 shutdown_connections 2024-04-20T21:29:46.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.600+0000 7fd801400640 1 -- 172.21.15.86:0/3493420386 wait complete. 2024-04-20T21:29:46.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.602+0000 7fd801400640 1 Processor -- start 2024-04-20T21:29:46.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.603+0000 7fd801400640 1 -- start start 2024-04-20T21:29:46.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.604+0000 7fd801400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7fc071d40 0x7fd7fc13bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:46.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.604+0000 7fd801400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7fc15f990 0x7fd7fc1593f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:46.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.604+0000 7fd801400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7fc161c60 0x7fd7fc0cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:46.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.604+0000 7fd801400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd7fc162040 con 0x7fd7fc15f990 2024-04-20T21:29:46.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.604+0000 7fd801400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd7fc06ee20 con 0x7fd7fc161c60 2024-04-20T21:29:46.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.604+0000 7fd801400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd7fc0af1b0 con 0x7fd7fc071d40 2024-04-20T21:29:46.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.604+0000 7fd7f3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7fc15f990 0x7fd7fc1593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:46.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.604+0000 7fd7f3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7fc15f990 0x7fd7fc1593f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49102/0 (socket says 172.21.15.86:49102) 2024-04-20T21:29:46.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.604+0000 7fd7f3400640 1 -- 172.21.15.86:0/79435615 learned_addr learned my addr 172.21.15.86:0/79435615 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:46.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.604+0000 7fd7fbe00640 1 --2- 172.21.15.86:0/79435615 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7fc071d40 0x7fd7fc13bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:46.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.605+0000 7fd800a00640 1 --2- 172.21.15.86:0/79435615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7fc161c60 0x7fd7fc0cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:46.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.605+0000 7fd7f3400640 1 -- 172.21.15.86:0/79435615 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7fc071d40 msgr2=0x7fd7fc13bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:46.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.605+0000 7fd7f3400640 1 --2- 172.21.15.86:0/79435615 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7fc071d40 0x7fd7fc13bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:46.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.605+0000 7fd7f3400640 1 -- 172.21.15.86:0/79435615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7fc161c60 msgr2=0x7fd7fc0cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:46.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.605+0000 7fd7f3400640 1 --2- 172.21.15.86:0/79435615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7fc161c60 0x7fd7fc0cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:46.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.605+0000 7fd7f3400640 1 -- 172.21.15.86:0/79435615 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7fc159dd0 con 0x7fd7fc15f990 2024-04-20T21:29:46.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.605+0000 7fd800a00640 1 --2- 172.21.15.86:0/79435615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7fc161c60 0x7fd7fc0cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:29:46.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.605+0000 7fd7f3400640 1 --2- 172.21.15.86:0/79435615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7fc15f990 0x7fd7fc1593f0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fd7e400e9e0 tx=0x7fd7e400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:46.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.605+0000 7fd7f8c00640 1 -- 172.21.15.86:0/79435615 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e4017070 con 0x7fd7fc15f990 2024-04-20T21:29:46.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.605+0000 7fd7f8c00640 1 -- 172.21.15.86:0/79435615 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7e400c9d0 con 0x7fd7fc15f990 2024-04-20T21:29:46.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.605+0000 7fd7f8c00640 1 -- 172.21.15.86:0/79435615 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e400cdc0 con 0x7fd7fc15f990 2024-04-20T21:29:46.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.605+0000 7fd801400640 1 -- 172.21.15.86:0/79435615 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7fc15a0b0 con 0x7fd7fc15f990 2024-04-20T21:29:46.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.606+0000 7fd801400640 1 -- 172.21.15.86:0/79435615 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7fc15b0a0 con 0x7fd7fc15f990 2024-04-20T21:29:46.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.607+0000 7fd801400640 1 -- 172.21.15.86:0/79435615 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7fc04fd90 con 0x7fd7fc15f990 2024-04-20T21:29:46.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.608+0000 7fd7f8c00640 1 -- 172.21.15.86:0/79435615 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd7e401a070 con 0x7fd7fc15f990 2024-04-20T21:29:46.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.609+0000 7fd7f8c00640 1 --2- 172.21.15.86:0/79435615 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd7d806c8f0 0x7fd7d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:46.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.609+0000 7fd7f8c00640 1 -- 172.21.15.86:0/79435615 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7fd7e408c080 con 0x7fd7fc15f990 2024-04-20T21:29:46.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.609+0000 7fd7fbe00640 1 --2- 172.21.15.86:0/79435615 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd7d806c8f0 0x7fd7d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:46.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.610+0000 7fd7fbe00640 1 --2- 172.21.15.86:0/79435615 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd7d806c8f0 0x7fd7d806edb0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec0011e0 tx=0x7fd7ec009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:46.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.614+0000 7fd7f8c00640 1 -- 172.21.15.86:0/79435615 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd7e405a390 con 0x7fd7fc15f990 2024-04-20T21:29:46.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:46.856+0000 7fd801400640 1 -- 172.21.15.86:0/79435615 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fd7fc06cd20 con 0x7fd7fc15f990 2024-04-20T21:29:47.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.371+0000 7fd7f8c00640 1 -- 172.21.15.86:0/79435615 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v99) ==== 122+0+0 (secure 0 0 0) 0x7fd7e4059c90 con 0x7fd7fc15f990 2024-04-20T21:29:47.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.375+0000 7fd801400640 1 -- 172.21.15.86:0/79435615 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd7d806c8f0 msgr2=0x7fd7d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:47.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.375+0000 7fd801400640 1 --2- 172.21.15.86:0/79435615 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd7d806c8f0 0x7fd7d806edb0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec0011e0 tx=0x7fd7ec009040 comp rx=0 tx=0).stop 2024-04-20T21:29:47.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.375+0000 7fd801400640 1 -- 172.21.15.86:0/79435615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7fc15f990 msgr2=0x7fd7fc1593f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:47.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.375+0000 7fd801400640 1 --2- 172.21.15.86:0/79435615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7fc15f990 0x7fd7fc1593f0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fd7e400e9e0 tx=0x7fd7e400eea0 comp rx=0 tx=0).stop 2024-04-20T21:29:47.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.375+0000 7fd801400640 1 -- 172.21.15.86:0/79435615 shutdown_connections 2024-04-20T21:29:47.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.375+0000 7fd801400640 1 --2- 172.21.15.86:0/79435615 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd7d806c8f0 0x7fd7d806edb0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:47.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.375+0000 7fd801400640 1 --2- 172.21.15.86:0/79435615 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd7fc161c60 0x7fd7fc0cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:47.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.375+0000 7fd801400640 1 --2- 172.21.15.86:0/79435615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7fc15f990 0x7fd7fc1593f0 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:47.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.375+0000 7fd801400640 1 --2- 172.21.15.86:0/79435615 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7fc071d40 0x7fd7fc13bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:47.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.375+0000 7fd801400640 1 -- 172.21.15.86:0/79435615 >> 172.21.15.86:0/79435615 conn(0x7fd7fc067cf0 msgr2=0x7fd7fc0af5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:47.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.375+0000 7fd801400640 1 -- 172.21.15.86:0/79435615 shutdown_connections 2024-04-20T21:29:47.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.375+0000 7fd801400640 1 -- 172.21.15.86:0/79435615 wait complete. 2024-04-20T21:29:47.397 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-20T21:29:47.397 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:29:47.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.596+0000 7f431b200640 1 Processor -- start 2024-04-20T21:29:47.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.596+0000 7f431b200640 1 -- start start 2024-04-20T21:29:47.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.598+0000 7f431b200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f431415ebf0 0x7f43140d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:47.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.598+0000 7f431b200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f431409b0f0 0x7f43140b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:47.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.598+0000 7f431b200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f431415efd0 con 0x7f43140d1820 2024-04-20T21:29:47.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.598+0000 7f431b200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f43140b1dc0 con 0x7f431409b0f0 2024-04-20T21:29:47.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.598+0000 7f431b200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f43140b20a0 con 0x7f431415ebf0 2024-04-20T21:29:47.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.599+0000 7f4319200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f431409b0f0 0x7f43140b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:47.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.599+0000 7f4313e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f431415ebf0 0x7f43140d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:47.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.599+0000 7f430b400640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7f43140d1820 0x7f431409a9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.86:46078/0 (socket says 172.21.15.86:46078) 2024-04-20T21:29:47.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.599+0000 7f430b400640 1 -- 172.21.15.86:0/2363498857 learned_addr learned my addr 172.21.15.86:0/2363498857 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:29:47.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.600+0000 7f4319200640 1 -- 172.21.15.86:0/2363498857 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f431415ebf0 msgr2=0x7f43140d12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:47.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.600+0000 7f4319200640 1 --2- 172.21.15.86:0/2363498857 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f431415ebf0 0x7f43140d12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:47.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.600+0000 7f4319200640 1 -- 172.21.15.86:0/2363498857 >> v1:172.21.15.144:6789/0 conn(0x7f43140d1820 legacy=0x7f431409a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:47.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.600+0000 7f4319200640 1 -- 172.21.15.86:0/2363498857 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43140b2380 con 0x7f431409b0f0 2024-04-20T21:29:47.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.601+0000 7f4319200640 1 --2- 172.21.15.86:0/2363498857 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f431409b0f0 0x7f43140b2e60 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f430c009d90 tx=0x7f430c02beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48bc0d08c1086f29 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:47.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.602+0000 7f4313400640 1 -- 172.21.15.86:0/2363498857 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c039070 con 0x7f431409b0f0 2024-04-20T21:29:47.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.602+0000 7f4313400640 1 -- 172.21.15.86:0/2363498857 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f430c02cc60 con 0x7f431409b0f0 2024-04-20T21:29:47.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.602+0000 7f4313400640 1 -- 172.21.15.86:0/2363498857 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c034660 con 0x7f431409b0f0 2024-04-20T21:29:47.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.603+0000 7f431b200640 1 -- 172.21.15.86:0/2363498857 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f431409b0f0 msgr2=0x7f43140b2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:47.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.603+0000 7f431b200640 1 --2- 172.21.15.86:0/2363498857 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f431409b0f0 0x7f43140b2e60 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f430c009d90 tx=0x7f430c02beb0 comp rx=0 tx=0).stop 2024-04-20T21:29:47.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.603+0000 7f431b200640 1 -- 172.21.15.86:0/2363498857 shutdown_connections 2024-04-20T21:29:47.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.603+0000 7f431b200640 1 --2- 172.21.15.86:0/2363498857 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f431409b0f0 0x7f43140b2e60 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:47.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.603+0000 7f431b200640 1 --2- 172.21.15.86:0/2363498857 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f431415ebf0 0x7f43140d12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:47.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.603+0000 7f431b200640 1 -- 172.21.15.86:0/2363498857 >> 172.21.15.86:0/2363498857 conn(0x7f43140b3ff0 msgr2=0x7f43140b4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:47.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.603+0000 7f431b200640 1 -- 172.21.15.86:0/2363498857 shutdown_connections 2024-04-20T21:29:47.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.603+0000 7f431b200640 1 -- 172.21.15.86:0/2363498857 wait complete. 2024-04-20T21:29:47.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.605+0000 7f431b200640 1 Processor -- start 2024-04-20T21:29:47.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.605+0000 7f431b200640 1 -- start start 2024-04-20T21:29:47.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.607+0000 7f431b200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f431409b0f0 0x7f4314150fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:47.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.607+0000 7f431b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43140d1820 0x7f4314150750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:47.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.607+0000 7f431b200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f431415ebf0 0x7f43140b5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:47.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.607+0000 7f431b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f43140ccb50 con 0x7f43140d1820 2024-04-20T21:29:47.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.607+0000 7f431b200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f43140b2940 con 0x7f431409b0f0 2024-04-20T21:29:47.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.607+0000 7f431b200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f43140cccd0 con 0x7f431415ebf0 2024-04-20T21:29:47.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.607+0000 7f430b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43140d1820 0x7f4314150750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:47.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.607+0000 7f430b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43140d1820 0x7f4314150750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49106/0 (socket says 172.21.15.86:49106) 2024-04-20T21:29:47.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.607+0000 7f430b400640 1 -- 172.21.15.86:0/1707302208 learned_addr learned my addr 172.21.15.86:0/1707302208 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:47.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.607+0000 7f4313e00640 1 --2- 172.21.15.86:0/1707302208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f431409b0f0 0x7f4314150fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:47.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.607+0000 7f4319200640 1 --2- 172.21.15.86:0/1707302208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f431415ebf0 0x7f43140b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:47.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.608+0000 7f430b400640 1 -- 172.21.15.86:0/1707302208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f431415ebf0 msgr2=0x7f43140b5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:47.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.608+0000 7f430b400640 1 --2- 172.21.15.86:0/1707302208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f431415ebf0 0x7f43140b5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:47.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.608+0000 7f430b400640 1 -- 172.21.15.86:0/1707302208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f431409b0f0 msgr2=0x7f4314150fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:47.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.608+0000 7f430b400640 1 --2- 172.21.15.86:0/1707302208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f431409b0f0 0x7f4314150fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:47.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.608+0000 7f430b400640 1 -- 172.21.15.86:0/1707302208 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43140b8110 con 0x7f43140d1820 2024-04-20T21:29:47.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.608+0000 7f430b400640 1 --2- 172.21.15.86:0/1707302208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43140d1820 0x7f4314150750 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f42fc00c400 tx=0x7f42fc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:47.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.608+0000 7f4310c00640 1 -- 172.21.15.86:0/1707302208 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42fc017070 con 0x7f43140d1820 2024-04-20T21:29:47.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.608+0000 7f4310c00640 1 -- 172.21.15.86:0/1707302208 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42fc005ce0 con 0x7f43140d1820 2024-04-20T21:29:47.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.608+0000 7f4310c00640 1 -- 172.21.15.86:0/1707302208 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42fc0072f0 con 0x7f43140d1820 2024-04-20T21:29:47.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.608+0000 7f431b200640 1 -- 172.21.15.86:0/1707302208 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43140ce0c0 con 0x7f43140d1820 2024-04-20T21:29:47.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.608+0000 7f431b200640 1 -- 172.21.15.86:0/1707302208 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4314150c90 con 0x7f43140d1820 2024-04-20T21:29:47.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.609+0000 7f431b200640 1 -- 172.21.15.86:0/1707302208 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43140689e0 con 0x7f43140d1820 2024-04-20T21:29:47.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.610+0000 7f4310c00640 1 -- 172.21.15.86:0/1707302208 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f42fc01a070 con 0x7f43140d1820 2024-04-20T21:29:47.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.611+0000 7f4310c00640 1 --2- 172.21.15.86:0/1707302208 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f42f006c8f0 0x7f42f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:47.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.611+0000 7f4310c00640 1 -- 172.21.15.86:0/1707302208 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7f42fc08af60 con 0x7f43140d1820 2024-04-20T21:29:47.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.612+0000 7f4313e00640 1 --2- 172.21.15.86:0/1707302208 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f42f006c8f0 0x7f42f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:47.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.612+0000 7f4313e00640 1 --2- 172.21.15.86:0/1707302208 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f42f006c8f0 0x7f42f006edb0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f431415de70 tx=0x7f4304008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:47.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.616+0000 7f4310c00640 1 -- 172.21.15.86:0/1707302208 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f42fc059390 con 0x7f43140d1820 2024-04-20T21:29:47.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.849+0000 7f431b200640 1 -- 172.21.15.86:0/1707302208 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f431406d5a0 con 0x7f43140d1820 2024-04-20T21:29:47.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.850+0000 7f4310c00640 1 -- 172.21.15.86:0/1707302208 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) ==== 94+0+5204 (secure 0 0 0) 0x7f42fc058c90 con 0x7f43140d1820 2024-04-20T21:29:47.850 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:29:47.850 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":99,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6253,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3627230007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3627230007},{"type":"v1","addr":"172.21.15.144:6833","nonce":3627230007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":6256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2498297133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2498297133},{"type":"v1","addr":"172.21.15.144:6835","nonce":2498297133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":9228,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/1120740475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1120740475},{"type":"v1","addr":"172.21.15.86:6836","nonce":1120740475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:29:43.769581+0000","modified":"2024-04-20T21:29:46.859349+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9231},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9231":{"gid":9231,"name":"a","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/289590924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":289590924},{"type":"v1","addr":"172.21.15.86:6837","nonce":289590924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9231,"qdb_cluster":[9231]},"id":8}]} 2024-04-20T21:29:47.851 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 99 2024-04-20T21:29:47.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.853+0000 7f431b200640 1 -- 172.21.15.86:0/1707302208 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f42f006c8f0 msgr2=0x7f42f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:47.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.853+0000 7f431b200640 1 --2- 172.21.15.86:0/1707302208 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f42f006c8f0 0x7f42f006edb0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f431415de70 tx=0x7f4304008040 comp rx=0 tx=0).stop 2024-04-20T21:29:47.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.853+0000 7f431b200640 1 -- 172.21.15.86:0/1707302208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43140d1820 msgr2=0x7f4314150750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:47.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.853+0000 7f431b200640 1 --2- 172.21.15.86:0/1707302208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43140d1820 0x7f4314150750 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f42fc00c400 tx=0x7f42fc00c8c0 comp rx=0 tx=0).stop 2024-04-20T21:29:47.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.853+0000 7f431b200640 1 -- 172.21.15.86:0/1707302208 shutdown_connections 2024-04-20T21:29:47.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.853+0000 7f431b200640 1 --2- 172.21.15.86:0/1707302208 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f42f006c8f0 0x7f42f006edb0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:47.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.853+0000 7f431b200640 1 --2- 172.21.15.86:0/1707302208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f431415ebf0 0x7f43140b5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:47.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.853+0000 7f431b200640 1 --2- 172.21.15.86:0/1707302208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43140d1820 0x7f4314150750 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:47.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.853+0000 7f431b200640 1 --2- 172.21.15.86:0/1707302208 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f431409b0f0 0x7f4314150fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:47.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.853+0000 7f431b200640 1 -- 172.21.15.86:0/1707302208 >> 172.21.15.86:0/1707302208 conn(0x7f43140b3ff0 msgr2=0x7f43140b43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:47.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.853+0000 7f431b200640 1 -- 172.21.15.86:0/1707302208 shutdown_connections 2024-04-20T21:29:47.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:47.853+0000 7f431b200640 1 -- 172.21.15.86:0/1707302208 wait complete. 2024-04-20T21:29:47.871 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:29:48.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.068+0000 7f11c4600640 1 Processor -- start 2024-04-20T21:29:48.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.068+0000 7f11c4600640 1 -- start start 2024-04-20T21:29:48.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.070+0000 7f11c4600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f11bc144e90 0x7f11bc15ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:48.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.070+0000 7f11c4600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11bc06dc30 con 0x7f11bc06f140 2024-04-20T21:29:48.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.070+0000 7f11c4600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11bc15f720 con 0x7f11bc071d00 2024-04-20T21:29:48.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.070+0000 7f11c4600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f11bc145270 con 0x7f11bc144e90 2024-04-20T21:29:48.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.070+0000 7f11c1200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f11bc06f140 0x7f11bc0aeb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:33846/0 (socket says 172.21.15.86:33846) 2024-04-20T21:29:48.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.070+0000 7f11c1200640 1 -- 172.21.15.86:0/114922392 learned_addr learned my addr 172.21.15.86:0/114922392 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:29:48.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.070+0000 7f11c1c00640 1 --2- 172.21.15.86:0/114922392 >> v2:172.21.15.86:3300/0 conn(0x7f11bc144e90 0x7f11bc15ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:48.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.071+0000 7f11bbe00640 1 -- 172.21.15.86:0/114922392 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3678588002 0 0) 0x7f11bc06dc30 con 0x7f11bc06f140 2024-04-20T21:29:48.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.071+0000 7f11bbe00640 1 -- 172.21.15.86:0/114922392 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f11a4003520 con 0x7f11bc06f140 2024-04-20T21:29:48.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.071+0000 7f11c1c00640 1 -- 172.21.15.86:0/114922392 >> v1:172.21.15.144:6789/0 conn(0x7f11bc071d00 legacy=0x7f11bc06e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:48.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.071+0000 7f11c1c00640 1 -- 172.21.15.86:0/114922392 >> v1:172.21.15.86:6789/0 conn(0x7f11bc06f140 legacy=0x7f11bc0aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:48.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.071+0000 7f11c1c00640 1 -- 172.21.15.86:0/114922392 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11bc15fa00 con 0x7f11bc144e90 2024-04-20T21:29:48.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.072+0000 7f11c1c00640 1 --2- 172.21.15.86:0/114922392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11bc144e90 0x7f11bc15ee10 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f11b00097a0 tx=0x7f11b002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71180869fd4268e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:48.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.073+0000 7f11bbe00640 1 -- 172.21.15.86:0/114922392 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b0035020 con 0x7f11bc144e90 2024-04-20T21:29:48.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.073+0000 7f11bbe00640 1 -- 172.21.15.86:0/114922392 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11b0031830 con 0x7f11bc144e90 2024-04-20T21:29:48.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.073+0000 7f11bbe00640 1 -- 172.21.15.86:0/114922392 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b0031be0 con 0x7f11bc144e90 2024-04-20T21:29:48.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.073+0000 7f11c4600640 1 -- 172.21.15.86:0/114922392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11bc144e90 msgr2=0x7f11bc15ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:48.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.073+0000 7f11c4600640 1 --2- 172.21.15.86:0/114922392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11bc144e90 0x7f11bc15ee10 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f11b00097a0 tx=0x7f11b002b8d0 comp rx=0 tx=0).stop 2024-04-20T21:29:48.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.074+0000 7f11c4600640 1 -- 172.21.15.86:0/114922392 shutdown_connections 2024-04-20T21:29:48.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.074+0000 7f11c4600640 1 --2- 172.21.15.86:0/114922392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11bc144e90 0x7f11bc15ee10 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:48.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.074+0000 7f11c4600640 1 -- 172.21.15.86:0/114922392 >> 172.21.15.86:0/114922392 conn(0x7f11bc0b2360 msgr2=0x7f11bc0b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:48.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.074+0000 7f11c4600640 1 -- 172.21.15.86:0/114922392 shutdown_connections 2024-04-20T21:29:48.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.074+0000 7f11c4600640 1 -- 172.21.15.86:0/114922392 wait complete. 2024-04-20T21:29:48.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.075+0000 7f11c4600640 1 Processor -- start 2024-04-20T21:29:48.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.075+0000 7f11c4600640 1 -- start start 2024-04-20T21:29:48.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.076+0000 7f11c4600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11bc06f140 0x7f11bc15e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:48.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.076+0000 7f11c4600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11bc071d00 0x7f11bc1313f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:48.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.076+0000 7f11c4600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11bc144e90 0x7f11bc161480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:48.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.076+0000 7f11c4600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc15f350 con 0x7f11bc06f140 2024-04-20T21:29:48.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.076+0000 7f11c4600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc06ded0 con 0x7f11bc144e90 2024-04-20T21:29:48.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.076+0000 7f11c4600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f11bc15f4d0 con 0x7f11bc071d00 2024-04-20T21:29:48.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.076+0000 7f11c1c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11bc06f140 0x7f11bc15e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:48.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.077+0000 7f11c1c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11bc06f140 0x7f11bc15e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49124/0 (socket says 172.21.15.86:49124) 2024-04-20T21:29:48.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.077+0000 7f11c1c00640 1 -- 172.21.15.86:0/1027636831 learned_addr learned my addr 172.21.15.86:0/1027636831 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:48.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.077+0000 7f11c1c00640 1 -- 172.21.15.86:0/1027636831 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11bc071d00 msgr2=0x7f11bc1313f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:48.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.077+0000 7f11c1c00640 1 --2- 172.21.15.86:0/1027636831 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11bc071d00 0x7f11bc1313f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:48.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.077+0000 7f11c1c00640 1 -- 172.21.15.86:0/1027636831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11bc144e90 msgr2=0x7f11bc161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:48.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.077+0000 7f11c1c00640 1 --2- 172.21.15.86:0/1027636831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11bc144e90 0x7f11bc161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:48.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.077+0000 7f11c1c00640 1 -- 172.21.15.86:0/1027636831 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11bc12f460 con 0x7f11bc06f140 2024-04-20T21:29:48.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.077+0000 7f11c1c00640 1 --2- 172.21.15.86:0/1027636831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11bc06f140 0x7f11bc15e4e0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f11b00098c0 tx=0x7f11b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:48.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.078+0000 7f11b9600640 1 -- 172.21.15.86:0/1027636831 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b0041070 con 0x7f11bc06f140 2024-04-20T21:29:48.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.078+0000 7f11b9600640 1 -- 172.21.15.86:0/1027636831 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11b003f070 con 0x7f11bc06f140 2024-04-20T21:29:48.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.078+0000 7f11b9600640 1 -- 172.21.15.86:0/1027636831 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b0035040 con 0x7f11bc06f140 2024-04-20T21:29:48.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.078+0000 7f11c4600640 1 -- 172.21.15.86:0/1027636831 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11bc131d60 con 0x7f11bc06f140 2024-04-20T21:29:48.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.078+0000 7f11c4600640 1 -- 172.21.15.86:0/1027636831 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11bc132320 con 0x7f11bc06f140 2024-04-20T21:29:48.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.079+0000 7f11c4600640 1 -- 172.21.15.86:0/1027636831 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11bc0689e0 con 0x7f11bc06f140 2024-04-20T21:29:48.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.080+0000 7f11b9600640 1 -- 172.21.15.86:0/1027636831 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f11b0038020 con 0x7f11bc06f140 2024-04-20T21:29:48.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.081+0000 7f11b9600640 1 --2- 172.21.15.86:0/1027636831 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f119c06c8f0 0x7f119c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:48.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.081+0000 7f11b9600640 1 -- 172.21.15.86:0/1027636831 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7f11b00ad510 con 0x7f11bc06f140 2024-04-20T21:29:48.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.081+0000 7f11c1200640 1 --2- 172.21.15.86:0/1027636831 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f119c06c8f0 0x7f119c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:48.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.082+0000 7f11c1200640 1 --2- 172.21.15.86:0/1027636831 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f119c06c8f0 0x7f119c06edb0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f11a8000f60 tx=0x7f11a80078f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:48.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.085+0000 7f11b9600640 1 -- 172.21.15.86:0/1027636831 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f11b0031800 con 0x7f11bc06f140 2024-04-20T21:29:48.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.274+0000 7f11c4600640 1 -- 172.21.15.86:0/1027636831 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f11bc0b0d90 con 0x7f11bc06f140 2024-04-20T21:29:48.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.277+0000 7f11b9600640 1 -- 172.21.15.86:0/1027636831 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) ==== 74+0+16627 (secure 0 0 0) 0x7f11b007b930 con 0x7f11bc06f140 2024-04-20T21:29:48.278 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:29:48.278 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":112,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:29:45.322441+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:29:41.532824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-20T21:29:42.749687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:29:48.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.280+0000 7f11c4600640 1 -- 172.21.15.86:0/1027636831 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f119c06c8f0 msgr2=0x7f119c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:48.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.280+0000 7f11c4600640 1 --2- 172.21.15.86:0/1027636831 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f119c06c8f0 0x7f119c06edb0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f11a8000f60 tx=0x7f11a80078f0 comp rx=0 tx=0).stop 2024-04-20T21:29:48.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.280+0000 7f11c4600640 1 -- 172.21.15.86:0/1027636831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11bc06f140 msgr2=0x7f11bc15e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:48.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.280+0000 7f11c4600640 1 --2- 172.21.15.86:0/1027636831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11bc06f140 0x7f11bc15e4e0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f11b00098c0 tx=0x7f11b00046c0 comp rx=0 tx=0).stop 2024-04-20T21:29:48.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.280+0000 7f11c4600640 1 -- 172.21.15.86:0/1027636831 shutdown_connections 2024-04-20T21:29:48.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.280+0000 7f11c4600640 1 --2- 172.21.15.86:0/1027636831 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f119c06c8f0 0x7f119c06edb0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:48.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.280+0000 7f11c4600640 1 --2- 172.21.15.86:0/1027636831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11bc144e90 0x7f11bc161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:48.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.280+0000 7f11c4600640 1 --2- 172.21.15.86:0/1027636831 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11bc071d00 0x7f11bc1313f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:48.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.280+0000 7f11c4600640 1 --2- 172.21.15.86:0/1027636831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11bc06f140 0x7f11bc15e4e0 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:48.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.280+0000 7f11c4600640 1 -- 172.21.15.86:0/1027636831 >> 172.21.15.86:0/1027636831 conn(0x7f11bc0b2360 msgr2=0x7f11bc072150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:48.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.281+0000 7f11c4600640 1 -- 172.21.15.86:0/1027636831 shutdown_connections 2024-04-20T21:29:48.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.281+0000 7f11c4600640 1 -- 172.21.15.86:0/1027636831 wait complete. 2024-04-20T21:29:48.298 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-20T21:29:48.299 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T21:29:48.392 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:29:48.399 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:29:48.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.493+0000 7fcadda00640 1 Processor -- start 2024-04-20T21:29:48.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.494+0000 7fcadda00640 1 -- start start 2024-04-20T21:29:48.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.495+0000 7fcadda00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fcad8072410 0x7fcad815cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:48.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.495+0000 7fcadda00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcad806eb30 con 0x7fcad806e6d0 2024-04-20T21:29:48.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.495+0000 7fcadda00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcad806dc30 con 0x7fcad8146130 2024-04-20T21:29:48.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.495+0000 7fcadda00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fcad80cd8e0 con 0x7fcad8072410 2024-04-20T21:29:48.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.495+0000 7fcacfe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fcad806e6d0 0x7fcad8071d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:33854/0 (socket says 172.21.15.86:33854) 2024-04-20T21:29:48.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.496+0000 7fcacfe00640 1 -- 172.21.15.86:0/3775019790 learned_addr learned my addr 172.21.15.86:0/3775019790 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:29:48.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.496+0000 7fcad7400640 1 --2- 172.21.15.86:0/3775019790 >> v2:172.21.15.144:3300/0 conn(0x7fcad8072410 0x7fcad815cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:48.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.496+0000 7fcad6000640 1 -- 172.21.15.86:0/3775019790 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2648340199 0 0) 0x7fcac0001110 con 0x7fcad806e6d0 2024-04-20T21:29:48.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.496+0000 7fcad6000640 1 -- 172.21.15.86:0/3775019790 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcabc003520 con 0x7fcad806e6d0 2024-04-20T21:29:48.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.496+0000 7fcad6000640 1 -- 172.21.15.86:0/3775019790 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 96090912 0 0) 0x7fcabc003520 con 0x7fcad806e6d0 2024-04-20T21:29:48.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.496+0000 7fcad6000640 1 -- 172.21.15.86:0/3775019790 >> v2:172.21.15.144:3300/0 conn(0x7fcad8072410 msgr2=0x7fcad815cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:48.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.496+0000 7fcad6000640 1 --2- 172.21.15.86:0/3775019790 >> v2:172.21.15.144:3300/0 conn(0x7fcad8072410 0x7fcad815cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:48.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.496+0000 7fcad6000640 1 -- 172.21.15.86:0/3775019790 >> v1:172.21.15.144:6789/0 conn(0x7fcad8146130 legacy=0x7fcad80ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:48.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.496+0000 7fcad6000640 1 -- 172.21.15.86:0/3775019790 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad80cdbc0 con 0x7fcad806e6d0 2024-04-20T21:29:48.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.496+0000 7fcad6000640 1 -- 172.21.15.86:0/3775019790 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fcac0001840 con 0x7fcad806e6d0 2024-04-20T21:29:48.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.496+0000 7fcad6000640 1 -- 172.21.15.86:0/3775019790 >> v1:172.21.15.86:6789/0 conn(0x7fcad806e6d0 legacy=0x7fcad8071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:48.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.496+0000 7fcad6000640 1 --2- 172.21.15.86:0/3775019790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcabc0039b0 0x7fcabc003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:48.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.497+0000 7fcad6000640 1 --2- 172.21.15.86:0/3775019790 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcabc004800 0x7fcabc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:48.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.497+0000 7fcad6000640 1 --2- 172.21.15.86:0/3775019790 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcabc0051c0 0x7fcabc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:48.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.497+0000 7fcad6000640 1 -- 172.21.15.86:0/3775019790 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcad80cd8e0 con 0x7fcabc0039b0 2024-04-20T21:29:48.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.497+0000 7fcad6000640 1 -- 172.21.15.86:0/3775019790 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc00c040 con 0x7fcabc0051c0 2024-04-20T21:29:48.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.497+0000 7fcad6000640 1 -- 172.21.15.86:0/3775019790 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fcabc00c240 con 0x7fcabc004800 2024-04-20T21:29:48.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.497+0000 7fcad6a00640 1 --2- 172.21.15.86:0/3775019790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcabc0039b0 0x7fcabc003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:48.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.497+0000 7fcacfe00640 1 --2- 172.21.15.86:0/3775019790 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcabc004800 0x7fcabc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:48.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.497+0000 7fcad6a00640 1 -- 172.21.15.86:0/3775019790 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcabc004800 msgr2=0x7fcabc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:48.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.497+0000 7fcad6a00640 1 --2- 172.21.15.86:0/3775019790 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcabc004800 0x7fcabc004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:48.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.497+0000 7fcad6a00640 1 -- 172.21.15.86:0/3775019790 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcabc0051c0 msgr2=0x7fcabc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:48.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.497+0000 7fcad6a00640 1 --2- 172.21.15.86:0/3775019790 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcabc0051c0 0x7fcabc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:48.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.497+0000 7fcad6a00640 1 -- 172.21.15.86:0/3775019790 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcabc003520 con 0x7fcabc0039b0 2024-04-20T21:29:48.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.497+0000 7fcad7400640 1 -- 172.21.15.86:0/3775019790 reap_dead start 2024-04-20T21:29:48.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.498+0000 7fcad6a00640 1 --2- 172.21.15.86:0/3775019790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcabc0039b0 0x7fcabc003db0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7fcac8001620 tx=0x7fcabc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aef48065b063ce7d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:48.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.499+0000 7fcad6000640 1 -- 172.21.15.86:0/3775019790 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac8001220 con 0x7fcabc0039b0 2024-04-20T21:29:48.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.499+0000 7fcad6000640 1 -- 172.21.15.86:0/3775019790 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcac8005290 con 0x7fcabc0039b0 2024-04-20T21:29:48.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.499+0000 7fcad6000640 1 -- 172.21.15.86:0/3775019790 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac8003850 con 0x7fcabc0039b0 2024-04-20T21:29:48.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.499+0000 7fcadda00640 1 -- 172.21.15.86:0/3775019790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcabc0039b0 msgr2=0x7fcabc003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:48.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.499+0000 7fcadda00640 1 --2- 172.21.15.86:0/3775019790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcabc0039b0 0x7fcabc003db0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7fcac8001620 tx=0x7fcabc005830 comp rx=0 tx=0).stop 2024-04-20T21:29:48.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.499+0000 7fcadda00640 1 -- 172.21.15.86:0/3775019790 shutdown_connections 2024-04-20T21:29:48.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.499+0000 7fcadda00640 1 --2- 172.21.15.86:0/3775019790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcabc0039b0 0x7fcabc003db0 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:48.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.499+0000 7fcadda00640 1 -- 172.21.15.86:0/3775019790 >> 172.21.15.86:0/3775019790 conn(0x7fcad80b2360 msgr2=0x7fcad80b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:48.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.499+0000 7fcadda00640 1 -- 172.21.15.86:0/3775019790 shutdown_connections 2024-04-20T21:29:48.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.499+0000 7fcadda00640 1 -- 172.21.15.86:0/3775019790 wait complete. 2024-04-20T21:29:48.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.501+0000 7fcadda00640 1 Processor -- start 2024-04-20T21:29:48.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.501+0000 7fcadda00640 1 -- start start 2024-04-20T21:29:48.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.502+0000 7fcadda00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcabc0039b0 0x7fcad81ee6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:48.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.502+0000 7fcadda00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcad813f540 0x7fcad80af3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:48.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.502+0000 7fcadda00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcad80af930 0x7fcad813cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:48.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.502+0000 7fcadda00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcad806f150 con 0x7fcad813f540 2024-04-20T21:29:48.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.502+0000 7fcadda00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcad806e1b0 con 0x7fcabc0039b0 2024-04-20T21:29:48.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.502+0000 7fcadda00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fcad8071d00 con 0x7fcad80af930 2024-04-20T21:29:48.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.502+0000 7fcacfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcad813f540 0x7fcad80af3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:48.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.503+0000 7fcacfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcad813f540 0x7fcad80af3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49148/0 (socket says 172.21.15.86:49148) 2024-04-20T21:29:48.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.503+0000 7fcacfe00640 1 -- 172.21.15.86:0/3636023501 learned_addr learned my addr 172.21.15.86:0/3636023501 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:48.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.503+0000 7fcad6a00640 1 --2- 172.21.15.86:0/3636023501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcabc0039b0 0x7fcad81ee6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:48.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.503+0000 7fcad7400640 1 --2- 172.21.15.86:0/3636023501 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcad80af930 0x7fcad813cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:48.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.503+0000 7fcacfe00640 1 -- 172.21.15.86:0/3636023501 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcad80af930 msgr2=0x7fcad813cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:48.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.503+0000 7fcacfe00640 1 --2- 172.21.15.86:0/3636023501 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcad80af930 0x7fcad813cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:48.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.503+0000 7fcacfe00640 1 -- 172.21.15.86:0/3636023501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcabc0039b0 msgr2=0x7fcad81ee6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:48.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.503+0000 7fcacfe00640 1 --2- 172.21.15.86:0/3636023501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcabc0039b0 0x7fcad81ee6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:48.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.503+0000 7fcacfe00640 1 -- 172.21.15.86:0/3636023501 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad81eec20 con 0x7fcad813f540 2024-04-20T21:29:48.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.503+0000 7fcacfe00640 1 --2- 172.21.15.86:0/3636023501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcad813f540 0x7fcad80af3f0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7fcac000d770 tx=0x7fcac000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:48.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.503+0000 7fcacea00640 1 -- 172.21.15.86:0/3636023501 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac0019070 con 0x7fcad813f540 2024-04-20T21:29:48.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.503+0000 7fcacea00640 1 -- 172.21.15.86:0/3636023501 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcac00074e0 con 0x7fcad813f540 2024-04-20T21:29:48.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.503+0000 7fcacea00640 1 -- 172.21.15.86:0/3636023501 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac0004a00 con 0x7fcad813f540 2024-04-20T21:29:48.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.504+0000 7fcadda00640 1 -- 172.21.15.86:0/3636023501 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcad813d910 con 0x7fcad813f540 2024-04-20T21:29:48.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.504+0000 7fcadda00640 1 -- 172.21.15.86:0/3636023501 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcad813ded0 con 0x7fcad813f540 2024-04-20T21:29:48.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.505+0000 7fcadda00640 1 -- 172.21.15.86:0/3636023501 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcad80689e0 con 0x7fcad813f540 2024-04-20T21:29:48.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.506+0000 7fcacea00640 1 -- 172.21.15.86:0/3636023501 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcac0004ba0 con 0x7fcad813f540 2024-04-20T21:29:48.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.506+0000 7fcacea00640 1 --2- 172.21.15.86:0/3636023501 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcab406c8f0 0x7fcab406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:48.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.507+0000 7fcacea00640 1 -- 172.21.15.86:0/3636023501 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7fcac008b460 con 0x7fcad813f540 2024-04-20T21:29:48.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.507+0000 7fcad6a00640 1 --2- 172.21.15.86:0/3636023501 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcab406c8f0 0x7fcab406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:48.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.507+0000 7fcad6a00640 1 --2- 172.21.15.86:0/3636023501 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcab406c8f0 0x7fcab406edb0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fcac8001620 tx=0x7fcac8009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:48.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.511+0000 7fcacea00640 1 -- 172.21.15.86:0/3636023501 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fcad806eb30 con 0x7fcad813f540 2024-04-20T21:29:48.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.705+0000 7fcadda00640 1 -- 172.21.15.86:0/3636023501 --> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcad80afd10 con 0x7fcab406c8f0 2024-04-20T21:29:48.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.724+0000 7fcacea00640 1 -- 172.21.15.86:0/3636023501 <== mgr.4107 v2:172.21.15.86:6832/302045240 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252480 (secure 0 0 0) 0x7fcad80afd10 con 0x7fcab406c8f0 2024-04-20T21:29:48.728 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:29:48.729 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-20T21:29:48.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.731+0000 7fcadda00640 1 -- 172.21.15.86:0/3636023501 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcab406c8f0 msgr2=0x7fcab406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:48.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.731+0000 7fcadda00640 1 --2- 172.21.15.86:0/3636023501 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcab406c8f0 0x7fcab406edb0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fcac8001620 tx=0x7fcac8009030 comp rx=0 tx=0).stop 2024-04-20T21:29:48.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.731+0000 7fcadda00640 1 -- 172.21.15.86:0/3636023501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcad813f540 msgr2=0x7fcad80af3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:48.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.731+0000 7fcadda00640 1 --2- 172.21.15.86:0/3636023501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcad813f540 0x7fcad80af3f0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7fcac000d770 tx=0x7fcac000dc30 comp rx=0 tx=0).stop 2024-04-20T21:29:48.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.731+0000 7fcadda00640 1 -- 172.21.15.86:0/3636023501 shutdown_connections 2024-04-20T21:29:48.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.731+0000 7fcadda00640 1 --2- 172.21.15.86:0/3636023501 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcab406c8f0 0x7fcab406edb0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:48.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.731+0000 7fcadda00640 1 --2- 172.21.15.86:0/3636023501 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcad80af930 0x7fcad813cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:48.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.731+0000 7fcadda00640 1 --2- 172.21.15.86:0/3636023501 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcad813f540 0x7fcad80af3f0 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:48.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.731+0000 7fcadda00640 1 --2- 172.21.15.86:0/3636023501 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcabc0039b0 0x7fcad81ee6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:48.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.731+0000 7fcadda00640 1 -- 172.21.15.86:0/3636023501 >> 172.21.15.86:0/3636023501 conn(0x7fcad80b2360 msgr2=0x7fcad8142240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:48.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.731+0000 7fcadda00640 1 -- 172.21.15.86:0/3636023501 shutdown_connections 2024-04-20T21:29:48.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.731+0000 7fcadda00640 1 -- 172.21.15.86:0/3636023501 wait complete. 2024-04-20T21:29:48.751 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":468,"stamp":"2024-04-20T21:29:46.884617+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":113,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261520,"kb_used_data":12044,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713200,"statfs":{"total":773094113280,"available":772826316800,"internally_reserved":0,"allocated":12333056,"data_stored":5991192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.588595"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.669947+0000","last_change":"2024-04-20T21:29:44.377305+0000","last_active":"2024-04-20T21:29:45.669947+0000","last_peered":"2024-04-20T21:29:45.669947+0000","last_clean":"2024-04-20T21:29:45.669947+0000","last_became_active":"2024-04-20T21:29:44.376455+0000","last_became_peered":"2024-04-20T21:29:44.376455+0000","last_unstale":"2024-04-20T21:29:45.669947+0000","last_undegraded":"2024-04-20T21:29:45.669947+0000","last_fullsized":"2024-04-20T21:29:45.669947+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:25:32.489738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.336198+0000","last_change":"2024-04-20T21:29:44.367475+0000","last_active":"2024-04-20T21:29:45.336198+0000","last_peered":"2024-04-20T21:29:45.336198+0000","last_clean":"2024-04-20T21:29:45.336198+0000","last_became_active":"2024-04-20T21:29:44.367125+0000","last_became_peered":"2024-04-20T21:29:44.367125+0000","last_unstale":"2024-04-20T21:29:45.336198+0000","last_undegraded":"2024-04-20T21:29:45.336198+0000","last_fullsized":"2024-04-20T21:29:45.336198+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:29:37.924593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.669789+0000","last_change":"2024-04-20T21:29:44.366397+0000","last_active":"2024-04-20T21:29:45.669789+0000","last_peered":"2024-04-20T21:29:45.669789+0000","last_clean":"2024-04-20T21:29:45.669789+0000","last_became_active":"2024-04-20T21:29:44.362782+0000","last_became_peered":"2024-04-20T21:29:44.362782+0000","last_unstale":"2024-04-20T21:29:45.669789+0000","last_undegraded":"2024-04-20T21:29:45.669789+0000","last_fullsized":"2024-04-20T21:29:45.669789+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:29:37.924593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.668296+0000","last_change":"2024-04-20T21:29:44.366936+0000","last_active":"2024-04-20T21:29:45.668296+0000","last_peered":"2024-04-20T21:29:45.668296+0000","last_clean":"2024-04-20T21:29:45.668296+0000","last_became_active":"2024-04-20T21:29:44.365338+0000","last_became_peered":"2024-04-20T21:29:44.365338+0000","last_unstale":"2024-04-20T21:29:45.668296+0000","last_undegraded":"2024-04-20T21:29:45.668296+0000","last_fullsized":"2024-04-20T21:29:45.668296+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:26:34.559712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.670497+0000","last_change":"2024-04-20T21:29:44.345490+0000","last_active":"2024-04-20T21:29:45.670497+0000","last_peered":"2024-04-20T21:29:45.670497+0000","last_clean":"2024-04-20T21:29:45.670497+0000","last_became_active":"2024-04-20T21:29:44.344747+0000","last_became_peered":"2024-04-20T21:29:44.344747+0000","last_unstale":"2024-04-20T21:29:45.670497+0000","last_undegraded":"2024-04-20T21:29:45.670497+0000","last_fullsized":"2024-04-20T21:29:45.670497+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:48:32.350465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.670671+0000","last_change":"2024-04-20T21:29:44.365055+0000","last_active":"2024-04-20T21:29:45.670671+0000","last_peered":"2024-04-20T21:29:45.670671+0000","last_clean":"2024-04-20T21:29:45.670671+0000","last_became_active":"2024-04-20T21:29:44.363410+0000","last_became_peered":"2024-04-20T21:29:44.363410+0000","last_unstale":"2024-04-20T21:29:45.670671+0000","last_undegraded":"2024-04-20T21:29:45.670671+0000","last_fullsized":"2024-04-20T21:29:45.670671+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:07:44.240776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.364499+0000","last_change":"2024-04-20T21:29:44.343308+0000","last_active":"2024-04-20T21:29:45.364499+0000","last_peered":"2024-04-20T21:29:45.364499+0000","last_clean":"2024-04-20T21:29:45.364499+0000","last_became_active":"2024-04-20T21:29:44.342567+0000","last_became_peered":"2024-04-20T21:29:44.342567+0000","last_unstale":"2024-04-20T21:29:45.364499+0000","last_undegraded":"2024-04-20T21:29:45.364499+0000","last_fullsized":"2024-04-20T21:29:45.364499+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:20:22.961359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.669633+0000","last_change":"2024-04-20T21:29:44.367727+0000","last_active":"2024-04-20T21:29:45.669633+0000","last_peered":"2024-04-20T21:29:45.669633+0000","last_clean":"2024-04-20T21:29:45.669633+0000","last_became_active":"2024-04-20T21:29:44.367299+0000","last_became_peered":"2024-04-20T21:29:44.367299+0000","last_unstale":"2024-04-20T21:29:45.669633+0000","last_undegraded":"2024-04-20T21:29:45.669633+0000","last_fullsized":"2024-04-20T21:29:45.669633+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:08:50.141052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.670381+0000","last_change":"2024-04-20T21:29:44.380655+0000","last_active":"2024-04-20T21:29:45.670381+0000","last_peered":"2024-04-20T21:29:45.670381+0000","last_clean":"2024-04-20T21:29:45.670381+0000","last_became_active":"2024-04-20T21:29:44.380411+0000","last_became_peered":"2024-04-20T21:29:44.380411+0000","last_unstale":"2024-04-20T21:29:45.670381+0000","last_undegraded":"2024-04-20T21:29:45.670381+0000","last_fullsized":"2024-04-20T21:29:45.670381+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:12:58.114645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.668331+0000","last_change":"2024-04-20T21:29:44.367403+0000","last_active":"2024-04-20T21:29:45.668331+0000","last_peered":"2024-04-20T21:29:45.668331+0000","last_clean":"2024-04-20T21:29:45.668331+0000","last_became_active":"2024-04-20T21:29:44.367139+0000","last_became_peered":"2024-04-20T21:29:44.367139+0000","last_unstale":"2024-04-20T21:29:45.668331+0000","last_undegraded":"2024-04-20T21:29:45.668331+0000","last_fullsized":"2024-04-20T21:29:45.668331+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:50:28.523525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.336847+0000","last_change":"2024-04-20T21:29:44.364870+0000","last_active":"2024-04-20T21:29:45.336847+0000","last_peered":"2024-04-20T21:29:45.336847+0000","last_clean":"2024-04-20T21:29:45.336847+0000","last_became_active":"2024-04-20T21:29:44.363085+0000","last_became_peered":"2024-04-20T21:29:44.363085+0000","last_unstale":"2024-04-20T21:29:45.336847+0000","last_undegraded":"2024-04-20T21:29:45.336847+0000","last_fullsized":"2024-04-20T21:29:45.336847+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:20:17.475536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.668425+0000","last_change":"2024-04-20T21:29:44.366107+0000","last_active":"2024-04-20T21:29:45.668425+0000","last_peered":"2024-04-20T21:29:45.668425+0000","last_clean":"2024-04-20T21:29:45.668425+0000","last_became_active":"2024-04-20T21:29:44.365324+0000","last_became_peered":"2024-04-20T21:29:44.365324+0000","last_unstale":"2024-04-20T21:29:45.668425+0000","last_undegraded":"2024-04-20T21:29:45.668425+0000","last_fullsized":"2024-04-20T21:29:45.668425+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:44:38.679019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.669982+0000","last_change":"2024-04-20T21:29:44.365441+0000","last_active":"2024-04-20T21:29:45.669982+0000","last_peered":"2024-04-20T21:29:45.669982+0000","last_clean":"2024-04-20T21:29:45.669982+0000","last_became_active":"2024-04-20T21:29:44.363953+0000","last_became_peered":"2024-04-20T21:29:44.363953+0000","last_unstale":"2024-04-20T21:29:45.669982+0000","last_undegraded":"2024-04-20T21:29:45.669982+0000","last_fullsized":"2024-04-20T21:29:45.669982+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:17:30.350963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.669906+0000","last_change":"2024-04-20T21:29:44.361572+0000","last_active":"2024-04-20T21:29:45.669906+0000","last_peered":"2024-04-20T21:29:45.669906+0000","last_clean":"2024-04-20T21:29:45.669906+0000","last_became_active":"2024-04-20T21:29:44.360587+0000","last_became_peered":"2024-04-20T21:29:44.360587+0000","last_unstale":"2024-04-20T21:29:45.669906+0000","last_undegraded":"2024-04-20T21:29:45.669906+0000","last_fullsized":"2024-04-20T21:29:45.669906+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:58:30.280453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.670509+0000","last_change":"2024-04-20T21:29:44.358772+0000","last_active":"2024-04-20T21:29:45.670509+0000","last_peered":"2024-04-20T21:29:45.670509+0000","last_clean":"2024-04-20T21:29:45.670509+0000","last_became_active":"2024-04-20T21:29:44.358453+0000","last_became_peered":"2024-04-20T21:29:44.358453+0000","last_unstale":"2024-04-20T21:29:45.670509+0000","last_undegraded":"2024-04-20T21:29:45.670509+0000","last_fullsized":"2024-04-20T21:29:45.670509+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:39:26.243434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.667928+0000","last_change":"2024-04-20T21:29:44.373334+0000","last_active":"2024-04-20T21:29:45.667928+0000","last_peered":"2024-04-20T21:29:45.667928+0000","last_clean":"2024-04-20T21:29:45.667928+0000","last_became_active":"2024-04-20T21:29:44.373066+0000","last_became_peered":"2024-04-20T21:29:44.373066+0000","last_unstale":"2024-04-20T21:29:45.667928+0000","last_undegraded":"2024-04-20T21:29:45.667928+0000","last_fullsized":"2024-04-20T21:29:45.667928+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:00:47.628001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.336591+0000","last_change":"2024-04-20T21:29:44.364331+0000","last_active":"2024-04-20T21:29:45.336591+0000","last_peered":"2024-04-20T21:29:45.336591+0000","last_clean":"2024-04-20T21:29:45.336591+0000","last_became_active":"2024-04-20T21:29:44.363169+0000","last_became_peered":"2024-04-20T21:29:44.363169+0000","last_unstale":"2024-04-20T21:29:45.336591+0000","last_undegraded":"2024-04-20T21:29:45.336591+0000","last_fullsized":"2024-04-20T21:29:45.336591+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:17:30.350963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.334327+0000","last_change":"2024-04-20T21:29:44.354526+0000","last_active":"2024-04-20T21:29:45.334327+0000","last_peered":"2024-04-20T21:29:45.334327+0000","last_clean":"2024-04-20T21:29:45.334327+0000","last_became_active":"2024-04-20T21:29:44.353732+0000","last_became_peered":"2024-04-20T21:29:44.353732+0000","last_unstale":"2024-04-20T21:29:45.334327+0000","last_undegraded":"2024-04-20T21:29:45.334327+0000","last_fullsized":"2024-04-20T21:29:45.334327+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:09:23.816382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.669863+0000","last_change":"2024-04-20T21:29:44.367165+0000","last_active":"2024-04-20T21:29:45.669863+0000","last_peered":"2024-04-20T21:29:45.669863+0000","last_clean":"2024-04-20T21:29:45.669863+0000","last_became_active":"2024-04-20T21:29:44.366572+0000","last_became_peered":"2024-04-20T21:29:44.366572+0000","last_unstale":"2024-04-20T21:29:45.669863+0000","last_undegraded":"2024-04-20T21:29:45.669863+0000","last_fullsized":"2024-04-20T21:29:45.669863+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:57:47.970601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.336433+0000","last_change":"2024-04-20T21:29:44.361500+0000","last_active":"2024-04-20T21:29:45.336433+0000","last_peered":"2024-04-20T21:29:45.336433+0000","last_clean":"2024-04-20T21:29:45.336433+0000","last_became_active":"2024-04-20T21:29:44.361027+0000","last_became_peered":"2024-04-20T21:29:44.361027+0000","last_unstale":"2024-04-20T21:29:45.336433+0000","last_undegraded":"2024-04-20T21:29:45.336433+0000","last_fullsized":"2024-04-20T21:29:45.336433+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:58:30.280453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.336815+0000","last_change":"2024-04-20T21:29:44.365077+0000","last_active":"2024-04-20T21:29:45.336815+0000","last_peered":"2024-04-20T21:29:45.336815+0000","last_clean":"2024-04-20T21:29:45.336815+0000","last_became_active":"2024-04-20T21:29:44.364568+0000","last_became_peered":"2024-04-20T21:29:44.364568+0000","last_unstale":"2024-04-20T21:29:45.336815+0000","last_undegraded":"2024-04-20T21:29:45.336815+0000","last_fullsized":"2024-04-20T21:29:45.336815+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:45:50.712038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.332934+0000","last_change":"2024-04-20T21:29:44.347781+0000","last_active":"2024-04-20T21:29:45.332934+0000","last_peered":"2024-04-20T21:29:45.332934+0000","last_clean":"2024-04-20T21:29:45.332934+0000","last_became_active":"2024-04-20T21:29:44.345629+0000","last_became_peered":"2024-04-20T21:29:44.345629+0000","last_unstale":"2024-04-20T21:29:45.332934+0000","last_undegraded":"2024-04-20T21:29:45.332934+0000","last_fullsized":"2024-04-20T21:29:45.332934+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:32:33.898830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.335234+0000","last_change":"2024-04-20T21:29:44.361086+0000","last_active":"2024-04-20T21:29:45.335234+0000","last_peered":"2024-04-20T21:29:45.335234+0000","last_clean":"2024-04-20T21:29:45.335234+0000","last_became_active":"2024-04-20T21:29:44.360629+0000","last_became_peered":"2024-04-20T21:29:44.360629+0000","last_unstale":"2024-04-20T21:29:45.335234+0000","last_undegraded":"2024-04-20T21:29:45.335234+0000","last_fullsized":"2024-04-20T21:29:45.335234+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:32:33.898830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.332227+0000","last_change":"2024-04-20T21:29:44.362905+0000","last_active":"2024-04-20T21:29:45.332227+0000","last_peered":"2024-04-20T21:29:45.332227+0000","last_clean":"2024-04-20T21:29:45.332227+0000","last_became_active":"2024-04-20T21:29:44.358406+0000","last_became_peered":"2024-04-20T21:29:44.358406+0000","last_unstale":"2024-04-20T21:29:45.332227+0000","last_undegraded":"2024-04-20T21:29:45.332227+0000","last_fullsized":"2024-04-20T21:29:45.332227+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:11:45.891059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.336900+0000","last_change":"2024-04-20T21:29:44.361775+0000","last_active":"2024-04-20T21:29:45.336900+0000","last_peered":"2024-04-20T21:29:45.336900+0000","last_clean":"2024-04-20T21:29:45.336900+0000","last_became_active":"2024-04-20T21:29:44.360651+0000","last_became_peered":"2024-04-20T21:29:44.360651+0000","last_unstale":"2024-04-20T21:29:45.336900+0000","last_undegraded":"2024-04-20T21:29:45.336900+0000","last_fullsized":"2024-04-20T21:29:45.336900+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:07:26.881957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.668458+0000","last_change":"2024-04-20T21:29:44.366648+0000","last_active":"2024-04-20T21:29:45.668458+0000","last_peered":"2024-04-20T21:29:45.668458+0000","last_clean":"2024-04-20T21:29:45.668458+0000","last_became_active":"2024-04-20T21:29:44.365895+0000","last_became_peered":"2024-04-20T21:29:44.365895+0000","last_unstale":"2024-04-20T21:29:45.668458+0000","last_undegraded":"2024-04-20T21:29:45.668458+0000","last_fullsized":"2024-04-20T21:29:45.668458+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:29:34.228709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.671440+0000","last_change":"2024-04-20T21:29:44.360975+0000","last_active":"2024-04-20T21:29:45.671440+0000","last_peered":"2024-04-20T21:29:45.671440+0000","last_clean":"2024-04-20T21:29:45.671440+0000","last_became_active":"2024-04-20T21:29:44.360681+0000","last_became_peered":"2024-04-20T21:29:44.360681+0000","last_unstale":"2024-04-20T21:29:45.671440+0000","last_undegraded":"2024-04-20T21:29:45.671440+0000","last_fullsized":"2024-04-20T21:29:45.671440+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:45:09.165513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.333081+0000","last_change":"2024-04-20T21:29:44.371691+0000","last_active":"2024-04-20T21:29:45.333081+0000","last_peered":"2024-04-20T21:29:45.333081+0000","last_clean":"2024-04-20T21:29:45.333081+0000","last_became_active":"2024-04-20T21:29:44.371386+0000","last_became_peered":"2024-04-20T21:29:44.371386+0000","last_unstale":"2024-04-20T21:29:45.333081+0000","last_undegraded":"2024-04-20T21:29:45.333081+0000","last_fullsized":"2024-04-20T21:29:45.333081+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:42:22.098265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.335791+0000","last_change":"2024-04-20T21:29:44.364204+0000","last_active":"2024-04-20T21:29:45.335791+0000","last_peered":"2024-04-20T21:29:45.335791+0000","last_clean":"2024-04-20T21:29:45.335791+0000","last_became_active":"2024-04-20T21:29:44.363498+0000","last_became_peered":"2024-04-20T21:29:44.363498+0000","last_unstale":"2024-04-20T21:29:45.335791+0000","last_undegraded":"2024-04-20T21:29:45.335791+0000","last_fullsized":"2024-04-20T21:29:45.335791+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:42:22.098265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.364509+0000","last_change":"2024-04-20T21:29:44.345912+0000","last_active":"2024-04-20T21:29:45.364509+0000","last_peered":"2024-04-20T21:29:45.364509+0000","last_clean":"2024-04-20T21:29:45.364509+0000","last_became_active":"2024-04-20T21:29:44.345213+0000","last_became_peered":"2024-04-20T21:29:44.345213+0000","last_unstale":"2024-04-20T21:29:45.364509+0000","last_undegraded":"2024-04-20T21:29:45.364509+0000","last_fullsized":"2024-04-20T21:29:45.364509+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:34:35.212237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.668571+0000","last_change":"2024-04-20T21:29:44.370474+0000","last_active":"2024-04-20T21:29:45.668571+0000","last_peered":"2024-04-20T21:29:45.668571+0000","last_clean":"2024-04-20T21:29:45.668571+0000","last_became_active":"2024-04-20T21:29:44.370158+0000","last_became_peered":"2024-04-20T21:29:44.370158+0000","last_unstale":"2024-04-20T21:29:45.668571+0000","last_undegraded":"2024-04-20T21:29:45.668571+0000","last_fullsized":"2024-04-20T21:29:45.668571+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:26:41.276861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.669086+0000","last_change":"2024-04-20T21:29:44.364089+0000","last_active":"2024-04-20T21:29:45.669086+0000","last_peered":"2024-04-20T21:29:45.669086+0000","last_clean":"2024-04-20T21:29:45.669086+0000","last_became_active":"2024-04-20T21:29:44.363835+0000","last_became_peered":"2024-04-20T21:29:44.363835+0000","last_unstale":"2024-04-20T21:29:45.669086+0000","last_undegraded":"2024-04-20T21:29:45.669086+0000","last_fullsized":"2024-04-20T21:29:45.669086+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:36:00.505654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.671338+0000","last_change":"2024-04-20T21:29:44.363440+0000","last_active":"2024-04-20T21:29:45.671338+0000","last_peered":"2024-04-20T21:29:45.671338+0000","last_clean":"2024-04-20T21:29:45.671338+0000","last_became_active":"2024-04-20T21:29:44.363116+0000","last_became_peered":"2024-04-20T21:29:44.363116+0000","last_unstale":"2024-04-20T21:29:45.671338+0000","last_undegraded":"2024-04-20T21:29:45.671338+0000","last_fullsized":"2024-04-20T21:29:45.671338+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:26:47.318941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"111'4","reported_seq":18,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.365557+0000","last_change":"2024-04-20T21:29:43.339073+0000","last_active":"2024-04-20T21:29:45.365557+0000","last_peered":"2024-04-20T21:29:45.365557+0000","last_clean":"2024-04-20T21:29:45.365557+0000","last_became_active":"2024-04-20T21:29:43.338573+0000","last_became_peered":"2024-04-20T21:29:43.338573+0000","last_unstale":"2024-04-20T21:29:45.365557+0000","last_undegraded":"2024-04-20T21:29:45.365557+0000","last_fullsized":"2024-04-20T21:29:45.365557+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:07:43.224631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.335130+0000","last_change":"2024-04-20T21:29:44.352707+0000","last_active":"2024-04-20T21:29:45.335130+0000","last_peered":"2024-04-20T21:29:45.335130+0000","last_clean":"2024-04-20T21:29:45.335130+0000","last_became_active":"2024-04-20T21:29:44.352281+0000","last_became_peered":"2024-04-20T21:29:44.352281+0000","last_unstale":"2024-04-20T21:29:45.335130+0000","last_undegraded":"2024-04-20T21:29:45.335130+0000","last_fullsized":"2024-04-20T21:29:45.335130+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:50:27.154529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"111'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.670218+0000","last_change":"2024-04-20T21:29:43.344751+0000","last_active":"2024-04-20T21:29:45.670218+0000","last_peered":"2024-04-20T21:29:45.670218+0000","last_clean":"2024-04-20T21:29:45.670218+0000","last_became_active":"2024-04-20T21:29:43.344392+0000","last_became_peered":"2024-04-20T21:29:43.344392+0000","last_unstale":"2024-04-20T21:29:45.670218+0000","last_undegraded":"2024-04-20T21:29:45.670218+0000","last_fullsized":"2024-04-20T21:29:45.670218+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:50:26.138384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.671118+0000","last_change":"2024-04-20T21:29:44.356394+0000","last_active":"2024-04-20T21:29:45.671118+0000","last_peered":"2024-04-20T21:29:45.671118+0000","last_clean":"2024-04-20T21:29:45.671118+0000","last_became_active":"2024-04-20T21:29:44.356071+0000","last_became_peered":"2024-04-20T21:29:44.356071+0000","last_unstale":"2024-04-20T21:29:45.671118+0000","last_undegraded":"2024-04-20T21:29:45.671118+0000","last_fullsized":"2024-04-20T21:29:45.671118+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:54:02.287967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"111'2","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.669861+0000","last_change":"2024-04-20T21:29:43.321851+0000","last_active":"2024-04-20T21:29:45.669861+0000","last_peered":"2024-04-20T21:29:45.669861+0000","last_clean":"2024-04-20T21:29:45.669861+0000","last_became_active":"2024-04-20T21:29:43.321480+0000","last_became_peered":"2024-04-20T21:29:43.321480+0000","last_unstale":"2024-04-20T21:29:45.669861+0000","last_undegraded":"2024-04-20T21:29:45.669861+0000","last_fullsized":"2024-04-20T21:29:45.669861+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:40:49.310112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.670231+0000","last_change":"2024-04-20T21:29:44.368108+0000","last_active":"2024-04-20T21:29:45.670231+0000","last_peered":"2024-04-20T21:29:45.670231+0000","last_clean":"2024-04-20T21:29:45.670231+0000","last_became_active":"2024-04-20T21:29:44.366711+0000","last_became_peered":"2024-04-20T21:29:44.366711+0000","last_unstale":"2024-04-20T21:29:45.670231+0000","last_undegraded":"2024-04-20T21:29:45.670231+0000","last_fullsized":"2024-04-20T21:29:45.670231+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:07:26.881957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"111'2","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.668923+0000","last_change":"2024-04-20T21:29:43.321794+0000","last_active":"2024-04-20T21:29:45.668923+0000","last_peered":"2024-04-20T21:29:45.668923+0000","last_clean":"2024-04-20T21:29:45.668923+0000","last_became_active":"2024-04-20T21:29:43.321437+0000","last_became_peered":"2024-04-20T21:29:43.321437+0000","last_unstale":"2024-04-20T21:29:45.668923+0000","last_undegraded":"2024-04-20T21:29:45.668923+0000","last_fullsized":"2024-04-20T21:29:45.668923+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:48:42.604987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.668878+0000","last_change":"2024-04-20T21:29:44.377619+0000","last_active":"2024-04-20T21:29:45.668878+0000","last_peered":"2024-04-20T21:29:45.668878+0000","last_clean":"2024-04-20T21:29:45.668878+0000","last_became_active":"2024-04-20T21:29:44.377338+0000","last_became_peered":"2024-04-20T21:29:44.377338+0000","last_unstale":"2024-04-20T21:29:45.668878+0000","last_undegraded":"2024-04-20T21:29:45.668878+0000","last_fullsized":"2024-04-20T21:29:45.668878+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:33:51.266811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"111'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.365001+0000","last_change":"2024-04-20T21:29:43.337340+0000","last_active":"2024-04-20T21:29:45.365001+0000","last_peered":"2024-04-20T21:29:45.365001+0000","last_clean":"2024-04-20T21:29:45.365001+0000","last_became_active":"2024-04-20T21:29:43.335975+0000","last_became_peered":"2024-04-20T21:29:43.335975+0000","last_unstale":"2024-04-20T21:29:45.365001+0000","last_undegraded":"2024-04-20T21:29:45.365001+0000","last_fullsized":"2024-04-20T21:29:45.365001+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:35:15.956972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.336245+0000","last_change":"2024-04-20T21:29:44.364846+0000","last_active":"2024-04-20T21:29:45.336245+0000","last_peered":"2024-04-20T21:29:45.336245+0000","last_clean":"2024-04-20T21:29:45.336245+0000","last_became_active":"2024-04-20T21:29:44.363910+0000","last_became_peered":"2024-04-20T21:29:44.363910+0000","last_unstale":"2024-04-20T21:29:45.336245+0000","last_undegraded":"2024-04-20T21:29:45.336245+0000","last_fullsized":"2024-04-20T21:29:45.336245+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:14:23.820951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"111'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.333229+0000","last_change":"2024-04-20T21:29:43.331564+0000","last_active":"2024-04-20T21:29:45.333229+0000","last_peered":"2024-04-20T21:29:45.333229+0000","last_clean":"2024-04-20T21:29:45.333229+0000","last_became_active":"2024-04-20T21:29:43.331204+0000","last_became_peered":"2024-04-20T21:29:43.331204+0000","last_unstale":"2024-04-20T21:29:45.333229+0000","last_undegraded":"2024-04-20T21:29:45.333229+0000","last_fullsized":"2024-04-20T21:29:45.333229+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:39:19.105304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.669151+0000","last_change":"2024-04-20T21:29:44.364222+0000","last_active":"2024-04-20T21:29:45.669151+0000","last_peered":"2024-04-20T21:29:45.669151+0000","last_clean":"2024-04-20T21:29:45.669151+0000","last_became_active":"2024-04-20T21:29:44.359967+0000","last_became_peered":"2024-04-20T21:29:44.359967+0000","last_unstale":"2024-04-20T21:29:45.669151+0000","last_undegraded":"2024-04-20T21:29:45.669151+0000","last_fullsized":"2024-04-20T21:29:45.669151+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:15:01.087200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.335574+0000","last_change":"2024-04-20T21:29:43.336906+0000","last_active":"2024-04-20T21:29:45.335574+0000","last_peered":"2024-04-20T21:29:45.335574+0000","last_clean":"2024-04-20T21:29:45.335574+0000","last_became_active":"2024-04-20T21:29:43.336305+0000","last_became_peered":"2024-04-20T21:29:43.336305+0000","last_unstale":"2024-04-20T21:29:45.335574+0000","last_undegraded":"2024-04-20T21:29:45.335574+0000","last_fullsized":"2024-04-20T21:29:45.335574+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:50:26.138384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.364283+0000","last_change":"2024-04-20T21:29:44.349585+0000","last_active":"2024-04-20T21:29:45.364283+0000","last_peered":"2024-04-20T21:29:45.364283+0000","last_clean":"2024-04-20T21:29:45.364283+0000","last_became_active":"2024-04-20T21:29:44.349120+0000","last_became_peered":"2024-04-20T21:29:44.349120+0000","last_unstale":"2024-04-20T21:29:45.364283+0000","last_undegraded":"2024-04-20T21:29:45.364283+0000","last_fullsized":"2024-04-20T21:29:45.364283+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:14:49.871482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.669099+0000","last_change":"2024-04-20T21:29:43.337896+0000","last_active":"2024-04-20T21:29:45.669099+0000","last_peered":"2024-04-20T21:29:45.669099+0000","last_clean":"2024-04-20T21:29:45.669099+0000","last_became_active":"2024-04-20T21:29:43.337440+0000","last_became_peered":"2024-04-20T21:29:43.337440+0000","last_unstale":"2024-04-20T21:29:45.669099+0000","last_undegraded":"2024-04-20T21:29:45.669099+0000","last_fullsized":"2024-04-20T21:29:45.669099+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:42:21.082120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.365353+0000","last_change":"2024-04-20T21:29:44.344994+0000","last_active":"2024-04-20T21:29:45.365353+0000","last_peered":"2024-04-20T21:29:45.365353+0000","last_clean":"2024-04-20T21:29:45.365353+0000","last_became_active":"2024-04-20T21:29:44.344226+0000","last_became_peered":"2024-04-20T21:29:44.344226+0000","last_unstale":"2024-04-20T21:29:45.365353+0000","last_undegraded":"2024-04-20T21:29:45.365353+0000","last_fullsized":"2024-04-20T21:29:45.365353+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:09:23.816382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.669497+0000","last_change":"2024-04-20T21:29:43.326187+0000","last_active":"2024-04-20T21:29:45.669497+0000","last_peered":"2024-04-20T21:29:45.669497+0000","last_clean":"2024-04-20T21:29:45.669497+0000","last_became_active":"2024-04-20T21:29:43.325900+0000","last_became_peered":"2024-04-20T21:29:43.325900+0000","last_unstale":"2024-04-20T21:29:45.669497+0000","last_undegraded":"2024-04-20T21:29:45.669497+0000","last_fullsized":"2024-04-20T21:29:45.669497+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:28:02.279050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.335788+0000","last_change":"2024-04-20T21:29:44.372545+0000","last_active":"2024-04-20T21:29:45.335788+0000","last_peered":"2024-04-20T21:29:45.335788+0000","last_clean":"2024-04-20T21:29:45.335788+0000","last_became_active":"2024-04-20T21:29:44.372190+0000","last_became_peered":"2024-04-20T21:29:44.372190+0000","last_unstale":"2024-04-20T21:29:45.335788+0000","last_undegraded":"2024-04-20T21:29:45.335788+0000","last_fullsized":"2024-04-20T21:29:45.335788+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:49:14.412117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.332734+0000","last_change":"2024-04-20T21:29:43.340366+0000","last_active":"2024-04-20T21:29:45.332734+0000","last_peered":"2024-04-20T21:29:45.332734+0000","last_clean":"2024-04-20T21:29:45.332734+0000","last_became_active":"2024-04-20T21:29:43.340048+0000","last_became_peered":"2024-04-20T21:29:43.340048+0000","last_unstale":"2024-04-20T21:29:45.332734+0000","last_undegraded":"2024-04-20T21:29:45.332734+0000","last_fullsized":"2024-04-20T21:29:45.332734+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:34:34.196092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.333235+0000","last_change":"2024-04-20T21:29:44.355980+0000","last_active":"2024-04-20T21:29:45.333235+0000","last_peered":"2024-04-20T21:29:45.333235+0000","last_clean":"2024-04-20T21:29:45.333235+0000","last_became_active":"2024-04-20T21:29:44.355690+0000","last_became_peered":"2024-04-20T21:29:44.355690+0000","last_unstale":"2024-04-20T21:29:45.333235+0000","last_undegraded":"2024-04-20T21:29:45.333235+0000","last_fullsized":"2024-04-20T21:29:45.333235+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:29:37.924593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"111'3","reported_seq":17,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.668721+0000","last_change":"2024-04-20T21:29:43.332776+0000","last_active":"2024-04-20T21:29:45.668721+0000","last_peered":"2024-04-20T21:29:45.668721+0000","last_clean":"2024-04-20T21:29:45.668721+0000","last_became_active":"2024-04-20T21:29:43.332469+0000","last_became_peered":"2024-04-20T21:29:43.332469+0000","last_unstale":"2024-04-20T21:29:45.668721+0000","last_undegraded":"2024-04-20T21:29:45.668721+0000","last_fullsized":"2024-04-20T21:29:45.668721+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:32:32.882685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.365139+0000","last_change":"2024-04-20T21:29:44.355468+0000","last_active":"2024-04-20T21:29:45.365139+0000","last_peered":"2024-04-20T21:29:45.365139+0000","last_clean":"2024-04-20T21:29:45.365139+0000","last_became_active":"2024-04-20T21:29:44.355136+0000","last_became_peered":"2024-04-20T21:29:44.355136+0000","last_unstale":"2024-04-20T21:29:45.365139+0000","last_undegraded":"2024-04-20T21:29:45.365139+0000","last_fullsized":"2024-04-20T21:29:45.365139+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:15:29.469663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"111'6","reported_seq":20,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.668658+0000","last_change":"2024-04-20T21:29:43.338112+0000","last_active":"2024-04-20T21:29:45.668658+0000","last_peered":"2024-04-20T21:29:45.668658+0000","last_clean":"2024-04-20T21:29:45.668658+0000","last_became_active":"2024-04-20T21:29:43.337051+0000","last_became_peered":"2024-04-20T21:29:43.337051+0000","last_unstale":"2024-04-20T21:29:45.668658+0000","last_undegraded":"2024-04-20T21:29:45.668658+0000","last_fullsized":"2024-04-20T21:29:45.668658+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:18:02.087371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.332998+0000","last_change":"2024-04-20T21:29:44.351822+0000","last_active":"2024-04-20T21:29:45.332998+0000","last_peered":"2024-04-20T21:29:45.332998+0000","last_clean":"2024-04-20T21:29:45.332998+0000","last_became_active":"2024-04-20T21:29:44.351479+0000","last_became_peered":"2024-04-20T21:29:44.351479+0000","last_unstale":"2024-04-20T21:29:45.332998+0000","last_undegraded":"2024-04-20T21:29:45.332998+0000","last_fullsized":"2024-04-20T21:29:45.332998+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:08:50.141052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"111'2","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.670897+0000","last_change":"2024-04-20T21:29:43.335170+0000","last_active":"2024-04-20T21:29:45.670897+0000","last_peered":"2024-04-20T21:29:45.670897+0000","last_clean":"2024-04-20T21:29:45.670897+0000","last_became_active":"2024-04-20T21:29:43.334553+0000","last_became_peered":"2024-04-20T21:29:43.334553+0000","last_unstale":"2024-04-20T21:29:45.670897+0000","last_undegraded":"2024-04-20T21:29:45.670897+0000","last_fullsized":"2024-04-20T21:29:45.670897+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:29:31.505502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.332742+0000","last_change":"2024-04-20T21:29:44.362433+0000","last_active":"2024-04-20T21:29:45.332742+0000","last_peered":"2024-04-20T21:29:45.332742+0000","last_clean":"2024-04-20T21:29:45.332742+0000","last_became_active":"2024-04-20T21:29:44.361615+0000","last_became_peered":"2024-04-20T21:29:44.361615+0000","last_unstale":"2024-04-20T21:29:45.332742+0000","last_undegraded":"2024-04-20T21:29:45.332742+0000","last_fullsized":"2024-04-20T21:29:45.332742+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:18:03.103516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.335828+0000","last_change":"2024-04-20T21:29:43.332508+0000","last_active":"2024-04-20T21:29:45.335828+0000","last_peered":"2024-04-20T21:29:45.335828+0000","last_clean":"2024-04-20T21:29:45.335828+0000","last_became_active":"2024-04-20T21:29:43.329900+0000","last_became_peered":"2024-04-20T21:29:43.329900+0000","last_unstale":"2024-04-20T21:29:45.335828+0000","last_undegraded":"2024-04-20T21:29:45.335828+0000","last_fullsized":"2024-04-20T21:29:45.335828+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:32:48.807084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.332206+0000","last_change":"2024-04-20T21:29:44.353877+0000","last_active":"2024-04-20T21:29:45.332206+0000","last_peered":"2024-04-20T21:29:45.332206+0000","last_clean":"2024-04-20T21:29:45.332206+0000","last_became_active":"2024-04-20T21:29:44.353534+0000","last_became_peered":"2024-04-20T21:29:44.353534+0000","last_unstale":"2024-04-20T21:29:45.332206+0000","last_undegraded":"2024-04-20T21:29:45.332206+0000","last_fullsized":"2024-04-20T21:29:45.332206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:12:00.136226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"111'8","reported_seq":22,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.668245+0000","last_change":"2024-04-20T21:29:43.332779+0000","last_active":"2024-04-20T21:29:45.668245+0000","last_peered":"2024-04-20T21:29:45.668245+0000","last_clean":"2024-04-20T21:29:45.668245+0000","last_became_active":"2024-04-20T21:29:43.332394+0000","last_became_peered":"2024-04-20T21:29:43.332394+0000","last_unstale":"2024-04-20T21:29:45.668245+0000","last_undegraded":"2024-04-20T21:29:45.668245+0000","last_fullsized":"2024-04-20T21:29:45.668245+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:52:53.484893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.669431+0000","last_change":"2024-04-20T21:29:44.360500+0000","last_active":"2024-04-20T21:29:45.669431+0000","last_peered":"2024-04-20T21:29:45.669431+0000","last_clean":"2024-04-20T21:29:45.669431+0000","last_became_active":"2024-04-20T21:29:44.360215+0000","last_became_peered":"2024-04-20T21:29:44.360215+0000","last_unstale":"2024-04-20T21:29:45.669431+0000","last_undegraded":"2024-04-20T21:29:45.669431+0000","last_fullsized":"2024-04-20T21:29:45.669431+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:19:51.279597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"111'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.671092+0000","last_change":"2024-04-20T21:29:43.337255+0000","last_active":"2024-04-20T21:29:45.671092+0000","last_peered":"2024-04-20T21:29:45.671092+0000","last_clean":"2024-04-20T21:29:45.671092+0000","last_became_active":"2024-04-20T21:29:43.336803+0000","last_became_peered":"2024-04-20T21:29:43.336803+0000","last_unstale":"2024-04-20T21:29:45.671092+0000","last_undegraded":"2024-04-20T21:29:45.671092+0000","last_fullsized":"2024-04-20T21:29:45.671092+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:07:25.865812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.671354+0000","last_change":"2024-04-20T21:29:44.378497+0000","last_active":"2024-04-20T21:29:45.671354+0000","last_peered":"2024-04-20T21:29:45.671354+0000","last_clean":"2024-04-20T21:29:45.671354+0000","last_became_active":"2024-04-20T21:29:44.378254+0000","last_became_peered":"2024-04-20T21:29:44.378254+0000","last_unstale":"2024-04-20T21:29:45.671354+0000","last_undegraded":"2024-04-20T21:29:45.671354+0000","last_fullsized":"2024-04-20T21:29:45.671354+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:35:16.973117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"111'2","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.669603+0000","last_change":"2024-04-20T21:29:43.326712+0000","last_active":"2024-04-20T21:29:45.669603+0000","last_peered":"2024-04-20T21:29:45.669603+0000","last_clean":"2024-04-20T21:29:45.669603+0000","last_became_active":"2024-04-20T21:29:43.324815+0000","last_became_peered":"2024-04-20T21:29:43.324815+0000","last_unstale":"2024-04-20T21:29:45.669603+0000","last_undegraded":"2024-04-20T21:29:45.669603+0000","last_fullsized":"2024-04-20T21:29:45.669603+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:29:31.505502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.669753+0000","last_change":"2024-04-20T21:29:44.371208+0000","last_active":"2024-04-20T21:29:45.669753+0000","last_peered":"2024-04-20T21:29:45.669753+0000","last_clean":"2024-04-20T21:29:45.669753+0000","last_became_active":"2024-04-20T21:29:44.370774+0000","last_became_peered":"2024-04-20T21:29:44.370774+0000","last_unstale":"2024-04-20T21:29:45.669753+0000","last_undegraded":"2024-04-20T21:29:45.669753+0000","last_fullsized":"2024-04-20T21:29:45.669753+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:42:22.098265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"111'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.334772+0000","last_change":"2024-04-20T21:29:43.332872+0000","last_active":"2024-04-20T21:29:45.334772+0000","last_peered":"2024-04-20T21:29:45.334772+0000","last_clean":"2024-04-20T21:29:45.334772+0000","last_became_active":"2024-04-20T21:29:43.330579+0000","last_became_peered":"2024-04-20T21:29:43.330579+0000","last_unstale":"2024-04-20T21:29:45.334772+0000","last_undegraded":"2024-04-20T21:29:45.334772+0000","last_fullsized":"2024-04-20T21:29:45.334772+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:51:29.472135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.670322+0000","last_change":"2024-04-20T21:29:44.367410+0000","last_active":"2024-04-20T21:29:45.670322+0000","last_peered":"2024-04-20T21:29:45.670322+0000","last_clean":"2024-04-20T21:29:45.670322+0000","last_became_active":"2024-04-20T21:29:44.366896+0000","last_became_peered":"2024-04-20T21:29:44.366896+0000","last_unstale":"2024-04-20T21:29:45.670322+0000","last_undegraded":"2024-04-20T21:29:45.670322+0000","last_fullsized":"2024-04-20T21:29:45.670322+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:30:57.290323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.364556+0000","last_change":"2024-04-20T21:29:43.336071+0000","last_active":"2024-04-20T21:29:45.364556+0000","last_peered":"2024-04-20T21:29:45.364556+0000","last_clean":"2024-04-20T21:29:45.364556+0000","last_became_active":"2024-04-20T21:29:43.335598+0000","last_became_peered":"2024-04-20T21:29:43.335598+0000","last_unstale":"2024-04-20T21:29:45.364556+0000","last_undegraded":"2024-04-20T21:29:45.364556+0000","last_fullsized":"2024-04-20T21:29:45.364556+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:26:46.302795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.670088+0000","last_change":"2024-04-20T21:29:44.361150+0000","last_active":"2024-04-20T21:29:45.670088+0000","last_peered":"2024-04-20T21:29:45.670088+0000","last_clean":"2024-04-20T21:29:45.670088+0000","last_became_active":"2024-04-20T21:29:44.360872+0000","last_became_peered":"2024-04-20T21:29:44.360872+0000","last_unstale":"2024-04-20T21:29:45.670088+0000","last_undegraded":"2024-04-20T21:29:45.670088+0000","last_fullsized":"2024-04-20T21:29:45.670088+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:45:50.712038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"111'2","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.670873+0000","last_change":"2024-04-20T21:29:43.328115+0000","last_active":"2024-04-20T21:29:45.670873+0000","last_peered":"2024-04-20T21:29:45.670873+0000","last_clean":"2024-04-20T21:29:45.670873+0000","last_became_active":"2024-04-20T21:29:43.327815+0000","last_became_peered":"2024-04-20T21:29:43.327815+0000","last_unstale":"2024-04-20T21:29:45.670873+0000","last_undegraded":"2024-04-20T21:29:45.670873+0000","last_fullsized":"2024-04-20T21:29:45.670873+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:34:44.200425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.668131+0000","last_change":"2024-04-20T21:29:44.364751+0000","last_active":"2024-04-20T21:29:45.668131+0000","last_peered":"2024-04-20T21:29:45.668131+0000","last_clean":"2024-04-20T21:29:45.668131+0000","last_became_active":"2024-04-20T21:29:44.358983+0000","last_became_peered":"2024-04-20T21:29:44.358983+0000","last_unstale":"2024-04-20T21:29:45.668131+0000","last_undegraded":"2024-04-20T21:29:45.668131+0000","last_fullsized":"2024-04-20T21:29:45.668131+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:47:48.597492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.669060+0000","last_change":"2024-04-20T21:29:43.326076+0000","last_active":"2024-04-20T21:29:45.669060+0000","last_peered":"2024-04-20T21:29:45.669060+0000","last_clean":"2024-04-20T21:29:45.669060+0000","last_became_active":"2024-04-20T21:29:43.325775+0000","last_became_peered":"2024-04-20T21:29:43.325775+0000","last_unstale":"2024-04-20T21:29:45.669060+0000","last_undegraded":"2024-04-20T21:29:45.669060+0000","last_fullsized":"2024-04-20T21:29:45.669060+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:54:50.130284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.669852+0000","last_change":"2024-04-20T21:29:44.364962+0000","last_active":"2024-04-20T21:29:45.669852+0000","last_peered":"2024-04-20T21:29:45.669852+0000","last_clean":"2024-04-20T21:29:45.669852+0000","last_became_active":"2024-04-20T21:29:44.363713+0000","last_became_peered":"2024-04-20T21:29:44.363713+0000","last_unstale":"2024-04-20T21:29:45.669852+0000","last_undegraded":"2024-04-20T21:29:45.669852+0000","last_fullsized":"2024-04-20T21:29:45.669852+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:14:23.820951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.670484+0000","last_change":"2024-04-20T21:29:43.330185+0000","last_active":"2024-04-20T21:29:45.670484+0000","last_peered":"2024-04-20T21:29:45.670484+0000","last_clean":"2024-04-20T21:29:45.670484+0000","last_became_active":"2024-04-20T21:29:43.329621+0000","last_became_peered":"2024-04-20T21:29:43.329621+0000","last_unstale":"2024-04-20T21:29:45.670484+0000","last_undegraded":"2024-04-20T21:29:45.670484+0000","last_fullsized":"2024-04-20T21:29:45.670484+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:40:49.310112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.332338+0000","last_change":"2024-04-20T21:29:44.372765+0000","last_active":"2024-04-20T21:29:45.332338+0000","last_peered":"2024-04-20T21:29:45.332338+0000","last_clean":"2024-04-20T21:29:45.332338+0000","last_became_active":"2024-04-20T21:29:44.372462+0000","last_became_peered":"2024-04-20T21:29:44.372462+0000","last_unstale":"2024-04-20T21:29:45.332338+0000","last_undegraded":"2024-04-20T21:29:45.332338+0000","last_fullsized":"2024-04-20T21:29:45.332338+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:22:50.948614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.669187+0000","last_change":"2024-04-20T21:29:43.335494+0000","last_active":"2024-04-20T21:29:45.669187+0000","last_peered":"2024-04-20T21:29:45.669187+0000","last_clean":"2024-04-20T21:29:45.669187+0000","last_became_active":"2024-04-20T21:29:43.335179+0000","last_became_peered":"2024-04-20T21:29:43.335179+0000","last_unstale":"2024-04-20T21:29:45.669187+0000","last_undegraded":"2024-04-20T21:29:45.669187+0000","last_fullsized":"2024-04-20T21:29:45.669187+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:29:36.908448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.334738+0000","last_change":"2024-04-20T21:29:44.359210+0000","last_active":"2024-04-20T21:29:45.334738+0000","last_peered":"2024-04-20T21:29:45.334738+0000","last_clean":"2024-04-20T21:29:45.334738+0000","last_became_active":"2024-04-20T21:29:44.358734+0000","last_became_peered":"2024-04-20T21:29:44.358734+0000","last_unstale":"2024-04-20T21:29:45.334738+0000","last_undegraded":"2024-04-20T21:29:45.334738+0000","last_fullsized":"2024-04-20T21:29:45.334738+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:28:24.144297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"111'2","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.333469+0000","last_change":"2024-04-20T21:29:43.338251+0000","last_active":"2024-04-20T21:29:45.333469+0000","last_peered":"2024-04-20T21:29:45.333469+0000","last_clean":"2024-04-20T21:29:45.333469+0000","last_became_active":"2024-04-20T21:29:43.337958+0000","last_became_peered":"2024-04-20T21:29:43.337958+0000","last_unstale":"2024-04-20T21:29:45.333469+0000","last_undegraded":"2024-04-20T21:29:45.333469+0000","last_fullsized":"2024-04-20T21:29:45.333469+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:15:28.453518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.670785+0000","last_change":"2024-04-20T21:29:44.371871+0000","last_active":"2024-04-20T21:29:45.670785+0000","last_peered":"2024-04-20T21:29:45.670785+0000","last_clean":"2024-04-20T21:29:45.670785+0000","last_became_active":"2024-04-20T21:29:44.371590+0000","last_became_peered":"2024-04-20T21:29:44.371590+0000","last_unstale":"2024-04-20T21:29:45.670785+0000","last_undegraded":"2024-04-20T21:29:45.670785+0000","last_fullsized":"2024-04-20T21:29:45.670785+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:27:39.892247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"112'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.363049+0000","last_change":"2024-04-20T21:29:43.334484+0000","last_active":"2024-04-20T21:29:45.363049+0000","last_peered":"2024-04-20T21:29:45.363049+0000","last_clean":"2024-04-20T21:29:45.363049+0000","last_became_active":"2024-04-20T21:29:43.334168+0000","last_became_peered":"2024-04-20T21:29:43.334168+0000","last_unstale":"2024-04-20T21:29:45.363049+0000","last_undegraded":"2024-04-20T21:29:45.363049+0000","last_fullsized":"2024-04-20T21:29:45.363049+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:17:45.018510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.668561+0000","last_change":"2024-04-20T21:29:44.366320+0000","last_active":"2024-04-20T21:29:45.668561+0000","last_peered":"2024-04-20T21:29:45.668561+0000","last_clean":"2024-04-20T21:29:45.668561+0000","last_became_active":"2024-04-20T21:29:44.365791+0000","last_became_peered":"2024-04-20T21:29:44.365791+0000","last_unstale":"2024-04-20T21:29:45.668561+0000","last_undegraded":"2024-04-20T21:29:45.668561+0000","last_fullsized":"2024-04-20T21:29:45.668561+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:32:45.366534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.668860+0000","last_change":"2024-04-20T21:29:43.330567+0000","last_active":"2024-04-20T21:29:45.668860+0000","last_peered":"2024-04-20T21:29:45.668860+0000","last_clean":"2024-04-20T21:29:45.668860+0000","last_became_active":"2024-04-20T21:29:43.330292+0000","last_became_peered":"2024-04-20T21:29:43.330292+0000","last_unstale":"2024-04-20T21:29:45.668860+0000","last_undegraded":"2024-04-20T21:29:45.668860+0000","last_fullsized":"2024-04-20T21:29:45.668860+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:08:49.124907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.334492+0000","last_change":"2024-04-20T21:29:44.357498+0000","last_active":"2024-04-20T21:29:45.334492+0000","last_peered":"2024-04-20T21:29:45.334492+0000","last_clean":"2024-04-20T21:29:45.334492+0000","last_became_active":"2024-04-20T21:29:44.357172+0000","last_became_peered":"2024-04-20T21:29:44.357172+0000","last_unstale":"2024-04-20T21:29:45.334492+0000","last_undegraded":"2024-04-20T21:29:45.334492+0000","last_fullsized":"2024-04-20T21:29:45.334492+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:39:20.121449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.333402+0000","last_change":"2024-04-20T21:29:43.338913+0000","last_active":"2024-04-20T21:29:45.333402+0000","last_peered":"2024-04-20T21:29:45.333402+0000","last_clean":"2024-04-20T21:29:45.333402+0000","last_became_active":"2024-04-20T21:29:43.338565+0000","last_became_peered":"2024-04-20T21:29:43.338565+0000","last_unstale":"2024-04-20T21:29:45.333402+0000","last_undegraded":"2024-04-20T21:29:45.333402+0000","last_fullsized":"2024-04-20T21:29:45.333402+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:09:22.800237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.335639+0000","last_change":"2024-04-20T21:29:44.363937+0000","last_active":"2024-04-20T21:29:45.335639+0000","last_peered":"2024-04-20T21:29:45.335639+0000","last_clean":"2024-04-20T21:29:45.335639+0000","last_became_active":"2024-04-20T21:29:44.363236+0000","last_became_peered":"2024-04-20T21:29:44.363236+0000","last_unstale":"2024-04-20T21:29:45.335639+0000","last_undegraded":"2024-04-20T21:29:45.335639+0000","last_fullsized":"2024-04-20T21:29:45.335639+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:08:50.141052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.365005+0000","last_change":"2024-04-20T21:29:43.337585+0000","last_active":"2024-04-20T21:29:45.365005+0000","last_peered":"2024-04-20T21:29:45.365005+0000","last_clean":"2024-04-20T21:29:45.365005+0000","last_became_active":"2024-04-20T21:29:43.337026+0000","last_became_peered":"2024-04-20T21:29:43.337026+0000","last_unstale":"2024-04-20T21:29:45.365005+0000","last_undegraded":"2024-04-20T21:29:45.365005+0000","last_fullsized":"2024-04-20T21:29:45.365005+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:54:01.271822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.336411+0000","last_change":"2024-04-20T21:29:44.358657+0000","last_active":"2024-04-20T21:29:45.336411+0000","last_peered":"2024-04-20T21:29:45.336411+0000","last_clean":"2024-04-20T21:29:45.336411+0000","last_became_active":"2024-04-20T21:29:44.358367+0000","last_became_peered":"2024-04-20T21:29:44.358367+0000","last_unstale":"2024-04-20T21:29:45.336411+0000","last_undegraded":"2024-04-20T21:29:45.336411+0000","last_fullsized":"2024-04-20T21:29:45.336411+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:57:47.970601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.335654+0000","last_change":"2024-04-20T21:29:43.332201+0000","last_active":"2024-04-20T21:29:45.335654+0000","last_peered":"2024-04-20T21:29:45.335654+0000","last_clean":"2024-04-20T21:29:45.335654+0000","last_became_active":"2024-04-20T21:29:43.331922+0000","last_became_peered":"2024-04-20T21:29:43.331922+0000","last_unstale":"2024-04-20T21:29:45.335654+0000","last_undegraded":"2024-04-20T21:29:45.335654+0000","last_fullsized":"2024-04-20T21:29:45.335654+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:28:23.128152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.670094+0000","last_change":"2024-04-20T21:29:44.365565+0000","last_active":"2024-04-20T21:29:45.670094+0000","last_peered":"2024-04-20T21:29:45.670094+0000","last_clean":"2024-04-20T21:29:45.670094+0000","last_became_active":"2024-04-20T21:29:44.365250+0000","last_became_peered":"2024-04-20T21:29:44.365250+0000","last_unstale":"2024-04-20T21:29:45.670094+0000","last_undegraded":"2024-04-20T21:29:45.670094+0000","last_fullsized":"2024-04-20T21:29:45.670094+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:20:17.475536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.668501+0000","last_change":"2024-04-20T21:29:43.322753+0000","last_active":"2024-04-20T21:29:45.668501+0000","last_peered":"2024-04-20T21:29:45.668501+0000","last_clean":"2024-04-20T21:29:45.668501+0000","last_became_active":"2024-04-20T21:29:43.322124+0000","last_became_peered":"2024-04-20T21:29:43.322124+0000","last_unstale":"2024-04-20T21:29:45.668501+0000","last_undegraded":"2024-04-20T21:29:45.668501+0000","last_fullsized":"2024-04-20T21:29:45.668501+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:34:44.200425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.668715+0000","last_change":"2024-04-20T21:29:44.378575+0000","last_active":"2024-04-20T21:29:45.668715+0000","last_peered":"2024-04-20T21:29:45.668715+0000","last_clean":"2024-04-20T21:29:45.668715+0000","last_became_active":"2024-04-20T21:29:44.378100+0000","last_became_peered":"2024-04-20T21:29:44.378100+0000","last_unstale":"2024-04-20T21:29:45.668715+0000","last_undegraded":"2024-04-20T21:29:45.668715+0000","last_fullsized":"2024-04-20T21:29:45.668715+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:22:41.884105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.669297+0000","last_change":"2024-04-20T21:29:43.330047+0000","last_active":"2024-04-20T21:29:45.669297+0000","last_peered":"2024-04-20T21:29:45.669297+0000","last_clean":"2024-04-20T21:29:45.669297+0000","last_became_active":"2024-04-20T21:29:43.329729+0000","last_became_peered":"2024-04-20T21:29:43.329729+0000","last_unstale":"2024-04-20T21:29:45.669297+0000","last_undegraded":"2024-04-20T21:29:45.669297+0000","last_fullsized":"2024-04-20T21:29:45.669297+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:14:22.804806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.332494+0000","last_change":"2024-04-20T21:29:44.355138+0000","last_active":"2024-04-20T21:29:45.332494+0000","last_peered":"2024-04-20T21:29:45.332494+0000","last_clean":"2024-04-20T21:29:45.332494+0000","last_became_active":"2024-04-20T21:29:44.354878+0000","last_became_peered":"2024-04-20T21:29:44.354878+0000","last_unstale":"2024-04-20T21:29:45.332494+0000","last_undegraded":"2024-04-20T21:29:45.332494+0000","last_fullsized":"2024-04-20T21:29:45.332494+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:30:49.193474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.364547+0000","last_change":"2024-04-20T21:29:43.334963+0000","last_active":"2024-04-20T21:29:45.364547+0000","last_peered":"2024-04-20T21:29:45.364547+0000","last_clean":"2024-04-20T21:29:45.364547+0000","last_became_active":"2024-04-20T21:29:43.334654+0000","last_became_peered":"2024-04-20T21:29:43.334654+0000","last_unstale":"2024-04-20T21:29:45.364547+0000","last_undegraded":"2024-04-20T21:29:45.364547+0000","last_fullsized":"2024-04-20T21:29:45.364547+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:45:08.149368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":338,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.670332+0000","last_change":"2024-04-20T21:17:43.059637+0000","last_active":"2024-04-20T21:29:45.670332+0000","last_peered":"2024-04-20T21:29:45.670332+0000","last_clean":"2024-04-20T21:29:45.670332+0000","last_became_active":"2024-04-20T21:17:43.056207+0000","last_became_peered":"2024-04-20T21:17:43.056207+0000","last_unstale":"2024-04-20T21:29:45.670332+0000","last_undegraded":"2024-04-20T21:29:45.670332+0000","last_fullsized":"2024-04-20T21:29:45.670332+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:41.036290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:41.036290+0000","last_clean_scrub_stamp":"2024-04-20T21:17:41.036290+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:58:51.091337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574995,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33124,"kb_used_data":1936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338716,"statfs":{"total":96636764160,"available":96602845184,"internally_reserved":0,"allocated":1982464,"data_stored":1189326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574996,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32536,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339304,"statfs":{"total":96636764160,"available":96603447296,"internally_reserved":0,"allocated":1384448,"data_stored":598412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574998,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32552,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339288,"statfs":{"total":96636764160,"available":96603430912,"internally_reserved":0,"allocated":1400832,"data_stored":605246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574997,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32548,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339292,"statfs":{"total":96636764160,"available":96603435008,"internally_reserved":0,"allocated":1392640,"data_stored":604642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574997,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32548,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339292,"statfs":{"total":96636764160,"available":96603435008,"internally_reserved":0,"allocated":1396736,"data_stored":600024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574997,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1380352,"data_stored":598342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574996,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32556,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339284,"statfs":{"total":96636764160,"available":96603426816,"internally_reserved":0,"allocated":1400832,"data_stored":604766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574997,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33124,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338716,"statfs":{"total":96636764160,"available":96602845184,"internally_reserved":0,"allocated":1994752,"data_stored":1190434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T21:29:48.755 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T21:29:48.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.993+0000 7f83d7c00640 1 Processor -- start 2024-04-20T21:29:48.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.993+0000 7f83d7c00640 1 -- start start 2024-04-20T21:29:48.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.995+0000 7f83d7c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f83d006f140 0x7f83d00aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:48.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.995+0000 7f83d7c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f83d00af0b0 0x7f83d0071d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:48.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.995+0000 7f83d7c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83d006dc30 con 0x7f83d0144e90 2024-04-20T21:29:48.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.995+0000 7f83d7c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f83d0145270 con 0x7f83d006f140 2024-04-20T21:29:48.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.995+0000 7f83d7c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f83d015f720 con 0x7f83d00af0b0 2024-04-20T21:29:48.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.995+0000 7f83cfe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f83d006f140 0x7f83d00aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:48.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.995+0000 7f83d5200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f83d0144e90 0x7f83d015ee10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:33866/0 (socket says 172.21.15.86:33866) 2024-04-20T21:29:48.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.995+0000 7f83d5200640 1 -- 172.21.15.86:0/250341184 learned_addr learned my addr 172.21.15.86:0/250341184 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:29:48.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.996+0000 7f83d5c00640 1 --2- 172.21.15.86:0/250341184 >> v2:172.21.15.144:3300/0 conn(0x7f83d00af0b0 0x7f83d0071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:48.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.996+0000 7f83cfe00640 1 -- 172.21.15.86:0/250341184 >> v2:172.21.15.144:3300/0 conn(0x7f83d00af0b0 msgr2=0x7f83d0071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:48.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.996+0000 7f83cfe00640 1 --2- 172.21.15.86:0/250341184 >> v2:172.21.15.144:3300/0 conn(0x7f83d00af0b0 0x7f83d0071d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:48.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.996+0000 7f83cfe00640 1 -- 172.21.15.86:0/250341184 >> v1:172.21.15.86:6789/0 conn(0x7f83d0144e90 legacy=0x7f83d015ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:48.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.996+0000 7f83cfe00640 1 -- 172.21.15.86:0/250341184 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d015fa00 con 0x7f83d006f140 2024-04-20T21:29:48.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.997+0000 7f83cfe00640 1 --2- 172.21.15.86:0/250341184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83d006f140 0x7f83d00aeb70 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f83d00b2110 tx=0x7f83b802bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c7337906ebdc293 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:48.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.997+0000 7f83cf400640 1 -- 172.21.15.86:0/250341184 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b8039070 con 0x7f83d006f140 2024-04-20T21:29:48.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.997+0000 7f83cf400640 1 -- 172.21.15.86:0/250341184 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83b802cc70 con 0x7f83d006f140 2024-04-20T21:29:48.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.997+0000 7f83cf400640 1 -- 172.21.15.86:0/250341184 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b8034660 con 0x7f83d006f140 2024-04-20T21:29:48.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.998+0000 7f83d7c00640 1 -- 172.21.15.86:0/250341184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83d006f140 msgr2=0x7f83d00aeb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:48.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.998+0000 7f83d7c00640 1 --2- 172.21.15.86:0/250341184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83d006f140 0x7f83d00aeb70 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f83d00b2110 tx=0x7f83b802bec0 comp rx=0 tx=0).stop 2024-04-20T21:29:48.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.998+0000 7f83d7c00640 1 -- 172.21.15.86:0/250341184 shutdown_connections 2024-04-20T21:29:48.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.998+0000 7f83d7c00640 1 --2- 172.21.15.86:0/250341184 >> v2:172.21.15.144:3300/0 conn(0x7f83d00af0b0 0x7f83d0071d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:48.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.998+0000 7f83d7c00640 1 --2- 172.21.15.86:0/250341184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83d006f140 0x7f83d00aeb70 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:48.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.998+0000 7f83d7c00640 1 -- 172.21.15.86:0/250341184 >> 172.21.15.86:0/250341184 conn(0x7f83d00b2360 msgr2=0x7f83d00b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:48.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.998+0000 7f83d7c00640 1 -- 172.21.15.86:0/250341184 shutdown_connections 2024-04-20T21:29:48.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:48.998+0000 7f83d7c00640 1 -- 172.21.15.86:0/250341184 wait complete. 2024-04-20T21:29:48.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.000+0000 7f83d7c00640 1 Processor -- start 2024-04-20T21:29:48.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.000+0000 7f83d7c00640 1 -- start start 2024-04-20T21:29:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.001+0000 7f83d7c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83d006f140 0x7f83d013e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.001+0000 7f83d7c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83d00af0b0 0x7f83d00cd790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.001+0000 7f83d7c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83d0144e90 0x7f83d01f81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.001+0000 7f83d7c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f83d0072240 con 0x7f83d006f140 2024-04-20T21:29:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.001+0000 7f83d7c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f83d006ded0 con 0x7f83d0144e90 2024-04-20T21:29:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.001+0000 7f83d7c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f83d00723c0 con 0x7f83d00af0b0 2024-04-20T21:29:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.001+0000 7f83d5200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83d006f140 0x7f83d013e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.001+0000 7f83d5200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83d006f140 0x7f83d013e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49168/0 (socket says 172.21.15.86:49168) 2024-04-20T21:29:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.001+0000 7f83d5200640 1 -- 172.21.15.86:0/3172701588 learned_addr learned my addr 172.21.15.86:0/3172701588 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:49.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.002+0000 7f83d5200640 1 -- 172.21.15.86:0/3172701588 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83d00af0b0 msgr2=0x7f83d00cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:49.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.002+0000 7f83d5200640 1 --2- 172.21.15.86:0/3172701588 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83d00af0b0 0x7f83d00cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:49.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.002+0000 7f83d5c00640 1 --2- 172.21.15.86:0/3172701588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83d0144e90 0x7f83d01f81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:49.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.002+0000 7f83d5200640 1 -- 172.21.15.86:0/3172701588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83d0144e90 msgr2=0x7f83d01f81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:49.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.002+0000 7f83d5200640 1 --2- 172.21.15.86:0/3172701588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83d0144e90 0x7f83d01f81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:49.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.002+0000 7f83d5200640 1 -- 172.21.15.86:0/3172701588 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d01f8b90 con 0x7f83d006f140 2024-04-20T21:29:49.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.002+0000 7f83d5c00640 1 --2- 172.21.15.86:0/3172701588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83d0144e90 0x7f83d01f81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:29:49.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.002+0000 7f83d5200640 1 --2- 172.21.15.86:0/3172701588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83d006f140 0x7f83d013e150 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f83c000c3c0 tx=0x7f83c000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:49.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.002+0000 7f83ccc00640 1 -- 172.21.15.86:0/3172701588 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c0017070 con 0x7f83d006f140 2024-04-20T21:29:49.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.002+0000 7f83ccc00640 1 -- 172.21.15.86:0/3172701588 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83c0005ce0 con 0x7f83d006f140 2024-04-20T21:29:49.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.002+0000 7f83ccc00640 1 -- 172.21.15.86:0/3172701588 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c00072f0 con 0x7f83d006f140 2024-04-20T21:29:49.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.002+0000 7f83d7c00640 1 -- 172.21.15.86:0/3172701588 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83d01f8e70 con 0x7f83d006f140 2024-04-20T21:29:49.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.002+0000 7f83d7c00640 1 -- 172.21.15.86:0/3172701588 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83d00cd270 con 0x7f83d006f140 2024-04-20T21:29:49.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.003+0000 7f83d7c00640 1 -- 172.21.15.86:0/3172701588 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83d00b4d10 con 0x7f83d006f140 2024-04-20T21:29:49.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.004+0000 7f83ccc00640 1 -- 172.21.15.86:0/3172701588 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f83c001a070 con 0x7f83d006f140 2024-04-20T21:29:49.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.005+0000 7f83ccc00640 1 --2- 172.21.15.86:0/3172701588 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f83ac06c8f0 0x7f83ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:49.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.005+0000 7f83ccc00640 1 -- 172.21.15.86:0/3172701588 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7f83c008af00 con 0x7f83d006f140 2024-04-20T21:29:49.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.006+0000 7f83cfe00640 1 --2- 172.21.15.86:0/3172701588 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f83ac06c8f0 0x7f83ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:49.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.006+0000 7f83cfe00640 1 --2- 172.21.15.86:0/3172701588 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f83ac06c8f0 0x7f83ac06edb0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f83b8036000 tx=0x7f83b802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:49.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.009+0000 7f83ccc00640 1 -- 172.21.15.86:0/3172701588 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f83d00b4d10 con 0x7f83d006f140 2024-04-20T21:29:49.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.206+0000 7f83d7c00640 1 -- 172.21.15.86:0/3172701588 --> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f83d00cd4b0 con 0x7f83ac06c8f0 2024-04-20T21:29:49.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.218+0000 7f83ccc00640 1 -- 172.21.15.86:0/3172701588 <== mgr.4107 v2:172.21.15.86:6832/302045240 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252480 (secure 0 0 0) 0x7f83b803a040 con 0x7f83ac06c8f0 2024-04-20T21:29:49.222 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:29:49.223 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-20T21:29:49.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.225+0000 7f83d7c00640 1 -- 172.21.15.86:0/3172701588 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f83ac06c8f0 msgr2=0x7f83ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:49.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.225+0000 7f83d7c00640 1 --2- 172.21.15.86:0/3172701588 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f83ac06c8f0 0x7f83ac06edb0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f83b8036000 tx=0x7f83b802e000 comp rx=0 tx=0).stop 2024-04-20T21:29:49.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.225+0000 7f83d7c00640 1 -- 172.21.15.86:0/3172701588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83d006f140 msgr2=0x7f83d013e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:49.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.225+0000 7f83d7c00640 1 --2- 172.21.15.86:0/3172701588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83d006f140 0x7f83d013e150 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f83c000c3c0 tx=0x7f83c000c880 comp rx=0 tx=0).stop 2024-04-20T21:29:49.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.225+0000 7f83d7c00640 1 -- 172.21.15.86:0/3172701588 shutdown_connections 2024-04-20T21:29:49.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.225+0000 7f83d7c00640 1 --2- 172.21.15.86:0/3172701588 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f83ac06c8f0 0x7f83ac06edb0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:49.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.225+0000 7f83d7c00640 1 --2- 172.21.15.86:0/3172701588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83d0144e90 0x7f83d01f81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:49.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.225+0000 7f83d7c00640 1 --2- 172.21.15.86:0/3172701588 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f83d00af0b0 0x7f83d00cd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:49.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.225+0000 7f83d7c00640 1 --2- 172.21.15.86:0/3172701588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83d006f140 0x7f83d013e150 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:49.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.225+0000 7f83d7c00640 1 -- 172.21.15.86:0/3172701588 >> 172.21.15.86:0/3172701588 conn(0x7f83d00b2360 msgr2=0x7f83d0161280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:49.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.225+0000 7f83d7c00640 1 -- 172.21.15.86:0/3172701588 shutdown_connections 2024-04-20T21:29:49.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.225+0000 7f83d7c00640 1 -- 172.21.15.86:0/3172701588 wait complete. 2024-04-20T21:29:49.243 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":469,"stamp":"2024-04-20T21:29:48.885277+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":113,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261520,"kb_used_data":12044,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713200,"statfs":{"total":773094113280,"available":772826316800,"internally_reserved":0,"allocated":12333056,"data_stored":5991192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001831"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.669947+0000","last_change":"2024-04-20T21:29:44.377305+0000","last_active":"2024-04-20T21:29:45.669947+0000","last_peered":"2024-04-20T21:29:45.669947+0000","last_clean":"2024-04-20T21:29:45.669947+0000","last_became_active":"2024-04-20T21:29:44.376455+0000","last_became_peered":"2024-04-20T21:29:44.376455+0000","last_unstale":"2024-04-20T21:29:45.669947+0000","last_undegraded":"2024-04-20T21:29:45.669947+0000","last_fullsized":"2024-04-20T21:29:45.669947+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:25:32.489738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.336198+0000","last_change":"2024-04-20T21:29:44.367475+0000","last_active":"2024-04-20T21:29:45.336198+0000","last_peered":"2024-04-20T21:29:45.336198+0000","last_clean":"2024-04-20T21:29:45.336198+0000","last_became_active":"2024-04-20T21:29:44.367125+0000","last_became_peered":"2024-04-20T21:29:44.367125+0000","last_unstale":"2024-04-20T21:29:45.336198+0000","last_undegraded":"2024-04-20T21:29:45.336198+0000","last_fullsized":"2024-04-20T21:29:45.336198+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:29:37.924593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.669789+0000","last_change":"2024-04-20T21:29:44.366397+0000","last_active":"2024-04-20T21:29:45.669789+0000","last_peered":"2024-04-20T21:29:45.669789+0000","last_clean":"2024-04-20T21:29:45.669789+0000","last_became_active":"2024-04-20T21:29:44.362782+0000","last_became_peered":"2024-04-20T21:29:44.362782+0000","last_unstale":"2024-04-20T21:29:45.669789+0000","last_undegraded":"2024-04-20T21:29:45.669789+0000","last_fullsized":"2024-04-20T21:29:45.669789+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:29:37.924593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.668296+0000","last_change":"2024-04-20T21:29:44.366936+0000","last_active":"2024-04-20T21:29:45.668296+0000","last_peered":"2024-04-20T21:29:45.668296+0000","last_clean":"2024-04-20T21:29:45.668296+0000","last_became_active":"2024-04-20T21:29:44.365338+0000","last_became_peered":"2024-04-20T21:29:44.365338+0000","last_unstale":"2024-04-20T21:29:45.668296+0000","last_undegraded":"2024-04-20T21:29:45.668296+0000","last_fullsized":"2024-04-20T21:29:45.668296+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:26:34.559712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.670497+0000","last_change":"2024-04-20T21:29:44.345490+0000","last_active":"2024-04-20T21:29:45.670497+0000","last_peered":"2024-04-20T21:29:45.670497+0000","last_clean":"2024-04-20T21:29:45.670497+0000","last_became_active":"2024-04-20T21:29:44.344747+0000","last_became_peered":"2024-04-20T21:29:44.344747+0000","last_unstale":"2024-04-20T21:29:45.670497+0000","last_undegraded":"2024-04-20T21:29:45.670497+0000","last_fullsized":"2024-04-20T21:29:45.670497+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:48:32.350465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.670671+0000","last_change":"2024-04-20T21:29:44.365055+0000","last_active":"2024-04-20T21:29:45.670671+0000","last_peered":"2024-04-20T21:29:45.670671+0000","last_clean":"2024-04-20T21:29:45.670671+0000","last_became_active":"2024-04-20T21:29:44.363410+0000","last_became_peered":"2024-04-20T21:29:44.363410+0000","last_unstale":"2024-04-20T21:29:45.670671+0000","last_undegraded":"2024-04-20T21:29:45.670671+0000","last_fullsized":"2024-04-20T21:29:45.670671+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:07:44.240776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.364499+0000","last_change":"2024-04-20T21:29:44.343308+0000","last_active":"2024-04-20T21:29:45.364499+0000","last_peered":"2024-04-20T21:29:45.364499+0000","last_clean":"2024-04-20T21:29:45.364499+0000","last_became_active":"2024-04-20T21:29:44.342567+0000","last_became_peered":"2024-04-20T21:29:44.342567+0000","last_unstale":"2024-04-20T21:29:45.364499+0000","last_undegraded":"2024-04-20T21:29:45.364499+0000","last_fullsized":"2024-04-20T21:29:45.364499+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:20:22.961359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.669633+0000","last_change":"2024-04-20T21:29:44.367727+0000","last_active":"2024-04-20T21:29:45.669633+0000","last_peered":"2024-04-20T21:29:45.669633+0000","last_clean":"2024-04-20T21:29:45.669633+0000","last_became_active":"2024-04-20T21:29:44.367299+0000","last_became_peered":"2024-04-20T21:29:44.367299+0000","last_unstale":"2024-04-20T21:29:45.669633+0000","last_undegraded":"2024-04-20T21:29:45.669633+0000","last_fullsized":"2024-04-20T21:29:45.669633+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:08:50.141052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.670381+0000","last_change":"2024-04-20T21:29:44.380655+0000","last_active":"2024-04-20T21:29:45.670381+0000","last_peered":"2024-04-20T21:29:45.670381+0000","last_clean":"2024-04-20T21:29:45.670381+0000","last_became_active":"2024-04-20T21:29:44.380411+0000","last_became_peered":"2024-04-20T21:29:44.380411+0000","last_unstale":"2024-04-20T21:29:45.670381+0000","last_undegraded":"2024-04-20T21:29:45.670381+0000","last_fullsized":"2024-04-20T21:29:45.670381+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:12:58.114645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.668331+0000","last_change":"2024-04-20T21:29:44.367403+0000","last_active":"2024-04-20T21:29:45.668331+0000","last_peered":"2024-04-20T21:29:45.668331+0000","last_clean":"2024-04-20T21:29:45.668331+0000","last_became_active":"2024-04-20T21:29:44.367139+0000","last_became_peered":"2024-04-20T21:29:44.367139+0000","last_unstale":"2024-04-20T21:29:45.668331+0000","last_undegraded":"2024-04-20T21:29:45.668331+0000","last_fullsized":"2024-04-20T21:29:45.668331+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:50:28.523525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.336847+0000","last_change":"2024-04-20T21:29:44.364870+0000","last_active":"2024-04-20T21:29:45.336847+0000","last_peered":"2024-04-20T21:29:45.336847+0000","last_clean":"2024-04-20T21:29:45.336847+0000","last_became_active":"2024-04-20T21:29:44.363085+0000","last_became_peered":"2024-04-20T21:29:44.363085+0000","last_unstale":"2024-04-20T21:29:45.336847+0000","last_undegraded":"2024-04-20T21:29:45.336847+0000","last_fullsized":"2024-04-20T21:29:45.336847+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:20:17.475536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.668425+0000","last_change":"2024-04-20T21:29:44.366107+0000","last_active":"2024-04-20T21:29:45.668425+0000","last_peered":"2024-04-20T21:29:45.668425+0000","last_clean":"2024-04-20T21:29:45.668425+0000","last_became_active":"2024-04-20T21:29:44.365324+0000","last_became_peered":"2024-04-20T21:29:44.365324+0000","last_unstale":"2024-04-20T21:29:45.668425+0000","last_undegraded":"2024-04-20T21:29:45.668425+0000","last_fullsized":"2024-04-20T21:29:45.668425+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:44:38.679019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.669982+0000","last_change":"2024-04-20T21:29:44.365441+0000","last_active":"2024-04-20T21:29:45.669982+0000","last_peered":"2024-04-20T21:29:45.669982+0000","last_clean":"2024-04-20T21:29:45.669982+0000","last_became_active":"2024-04-20T21:29:44.363953+0000","last_became_peered":"2024-04-20T21:29:44.363953+0000","last_unstale":"2024-04-20T21:29:45.669982+0000","last_undegraded":"2024-04-20T21:29:45.669982+0000","last_fullsized":"2024-04-20T21:29:45.669982+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:17:30.350963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.669906+0000","last_change":"2024-04-20T21:29:44.361572+0000","last_active":"2024-04-20T21:29:45.669906+0000","last_peered":"2024-04-20T21:29:45.669906+0000","last_clean":"2024-04-20T21:29:45.669906+0000","last_became_active":"2024-04-20T21:29:44.360587+0000","last_became_peered":"2024-04-20T21:29:44.360587+0000","last_unstale":"2024-04-20T21:29:45.669906+0000","last_undegraded":"2024-04-20T21:29:45.669906+0000","last_fullsized":"2024-04-20T21:29:45.669906+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:58:30.280453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.670509+0000","last_change":"2024-04-20T21:29:44.358772+0000","last_active":"2024-04-20T21:29:45.670509+0000","last_peered":"2024-04-20T21:29:45.670509+0000","last_clean":"2024-04-20T21:29:45.670509+0000","last_became_active":"2024-04-20T21:29:44.358453+0000","last_became_peered":"2024-04-20T21:29:44.358453+0000","last_unstale":"2024-04-20T21:29:45.670509+0000","last_undegraded":"2024-04-20T21:29:45.670509+0000","last_fullsized":"2024-04-20T21:29:45.670509+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:39:26.243434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.667928+0000","last_change":"2024-04-20T21:29:44.373334+0000","last_active":"2024-04-20T21:29:45.667928+0000","last_peered":"2024-04-20T21:29:45.667928+0000","last_clean":"2024-04-20T21:29:45.667928+0000","last_became_active":"2024-04-20T21:29:44.373066+0000","last_became_peered":"2024-04-20T21:29:44.373066+0000","last_unstale":"2024-04-20T21:29:45.667928+0000","last_undegraded":"2024-04-20T21:29:45.667928+0000","last_fullsized":"2024-04-20T21:29:45.667928+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:00:47.628001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.336591+0000","last_change":"2024-04-20T21:29:44.364331+0000","last_active":"2024-04-20T21:29:45.336591+0000","last_peered":"2024-04-20T21:29:45.336591+0000","last_clean":"2024-04-20T21:29:45.336591+0000","last_became_active":"2024-04-20T21:29:44.363169+0000","last_became_peered":"2024-04-20T21:29:44.363169+0000","last_unstale":"2024-04-20T21:29:45.336591+0000","last_undegraded":"2024-04-20T21:29:45.336591+0000","last_fullsized":"2024-04-20T21:29:45.336591+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:17:30.350963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.334327+0000","last_change":"2024-04-20T21:29:44.354526+0000","last_active":"2024-04-20T21:29:45.334327+0000","last_peered":"2024-04-20T21:29:45.334327+0000","last_clean":"2024-04-20T21:29:45.334327+0000","last_became_active":"2024-04-20T21:29:44.353732+0000","last_became_peered":"2024-04-20T21:29:44.353732+0000","last_unstale":"2024-04-20T21:29:45.334327+0000","last_undegraded":"2024-04-20T21:29:45.334327+0000","last_fullsized":"2024-04-20T21:29:45.334327+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:09:23.816382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.669863+0000","last_change":"2024-04-20T21:29:44.367165+0000","last_active":"2024-04-20T21:29:45.669863+0000","last_peered":"2024-04-20T21:29:45.669863+0000","last_clean":"2024-04-20T21:29:45.669863+0000","last_became_active":"2024-04-20T21:29:44.366572+0000","last_became_peered":"2024-04-20T21:29:44.366572+0000","last_unstale":"2024-04-20T21:29:45.669863+0000","last_undegraded":"2024-04-20T21:29:45.669863+0000","last_fullsized":"2024-04-20T21:29:45.669863+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:57:47.970601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.336433+0000","last_change":"2024-04-20T21:29:44.361500+0000","last_active":"2024-04-20T21:29:45.336433+0000","last_peered":"2024-04-20T21:29:45.336433+0000","last_clean":"2024-04-20T21:29:45.336433+0000","last_became_active":"2024-04-20T21:29:44.361027+0000","last_became_peered":"2024-04-20T21:29:44.361027+0000","last_unstale":"2024-04-20T21:29:45.336433+0000","last_undegraded":"2024-04-20T21:29:45.336433+0000","last_fullsized":"2024-04-20T21:29:45.336433+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:58:30.280453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.336815+0000","last_change":"2024-04-20T21:29:44.365077+0000","last_active":"2024-04-20T21:29:45.336815+0000","last_peered":"2024-04-20T21:29:45.336815+0000","last_clean":"2024-04-20T21:29:45.336815+0000","last_became_active":"2024-04-20T21:29:44.364568+0000","last_became_peered":"2024-04-20T21:29:44.364568+0000","last_unstale":"2024-04-20T21:29:45.336815+0000","last_undegraded":"2024-04-20T21:29:45.336815+0000","last_fullsized":"2024-04-20T21:29:45.336815+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:45:50.712038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.332934+0000","last_change":"2024-04-20T21:29:44.347781+0000","last_active":"2024-04-20T21:29:45.332934+0000","last_peered":"2024-04-20T21:29:45.332934+0000","last_clean":"2024-04-20T21:29:45.332934+0000","last_became_active":"2024-04-20T21:29:44.345629+0000","last_became_peered":"2024-04-20T21:29:44.345629+0000","last_unstale":"2024-04-20T21:29:45.332934+0000","last_undegraded":"2024-04-20T21:29:45.332934+0000","last_fullsized":"2024-04-20T21:29:45.332934+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:32:33.898830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.335234+0000","last_change":"2024-04-20T21:29:44.361086+0000","last_active":"2024-04-20T21:29:45.335234+0000","last_peered":"2024-04-20T21:29:45.335234+0000","last_clean":"2024-04-20T21:29:45.335234+0000","last_became_active":"2024-04-20T21:29:44.360629+0000","last_became_peered":"2024-04-20T21:29:44.360629+0000","last_unstale":"2024-04-20T21:29:45.335234+0000","last_undegraded":"2024-04-20T21:29:45.335234+0000","last_fullsized":"2024-04-20T21:29:45.335234+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:32:33.898830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.332227+0000","last_change":"2024-04-20T21:29:44.362905+0000","last_active":"2024-04-20T21:29:45.332227+0000","last_peered":"2024-04-20T21:29:45.332227+0000","last_clean":"2024-04-20T21:29:45.332227+0000","last_became_active":"2024-04-20T21:29:44.358406+0000","last_became_peered":"2024-04-20T21:29:44.358406+0000","last_unstale":"2024-04-20T21:29:45.332227+0000","last_undegraded":"2024-04-20T21:29:45.332227+0000","last_fullsized":"2024-04-20T21:29:45.332227+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:11:45.891059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.336900+0000","last_change":"2024-04-20T21:29:44.361775+0000","last_active":"2024-04-20T21:29:45.336900+0000","last_peered":"2024-04-20T21:29:45.336900+0000","last_clean":"2024-04-20T21:29:45.336900+0000","last_became_active":"2024-04-20T21:29:44.360651+0000","last_became_peered":"2024-04-20T21:29:44.360651+0000","last_unstale":"2024-04-20T21:29:45.336900+0000","last_undegraded":"2024-04-20T21:29:45.336900+0000","last_fullsized":"2024-04-20T21:29:45.336900+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:07:26.881957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.668458+0000","last_change":"2024-04-20T21:29:44.366648+0000","last_active":"2024-04-20T21:29:45.668458+0000","last_peered":"2024-04-20T21:29:45.668458+0000","last_clean":"2024-04-20T21:29:45.668458+0000","last_became_active":"2024-04-20T21:29:44.365895+0000","last_became_peered":"2024-04-20T21:29:44.365895+0000","last_unstale":"2024-04-20T21:29:45.668458+0000","last_undegraded":"2024-04-20T21:29:45.668458+0000","last_fullsized":"2024-04-20T21:29:45.668458+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:29:34.228709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.671440+0000","last_change":"2024-04-20T21:29:44.360975+0000","last_active":"2024-04-20T21:29:45.671440+0000","last_peered":"2024-04-20T21:29:45.671440+0000","last_clean":"2024-04-20T21:29:45.671440+0000","last_became_active":"2024-04-20T21:29:44.360681+0000","last_became_peered":"2024-04-20T21:29:44.360681+0000","last_unstale":"2024-04-20T21:29:45.671440+0000","last_undegraded":"2024-04-20T21:29:45.671440+0000","last_fullsized":"2024-04-20T21:29:45.671440+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:45:09.165513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.333081+0000","last_change":"2024-04-20T21:29:44.371691+0000","last_active":"2024-04-20T21:29:45.333081+0000","last_peered":"2024-04-20T21:29:45.333081+0000","last_clean":"2024-04-20T21:29:45.333081+0000","last_became_active":"2024-04-20T21:29:44.371386+0000","last_became_peered":"2024-04-20T21:29:44.371386+0000","last_unstale":"2024-04-20T21:29:45.333081+0000","last_undegraded":"2024-04-20T21:29:45.333081+0000","last_fullsized":"2024-04-20T21:29:45.333081+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:42:22.098265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.335791+0000","last_change":"2024-04-20T21:29:44.364204+0000","last_active":"2024-04-20T21:29:45.335791+0000","last_peered":"2024-04-20T21:29:45.335791+0000","last_clean":"2024-04-20T21:29:45.335791+0000","last_became_active":"2024-04-20T21:29:44.363498+0000","last_became_peered":"2024-04-20T21:29:44.363498+0000","last_unstale":"2024-04-20T21:29:45.335791+0000","last_undegraded":"2024-04-20T21:29:45.335791+0000","last_fullsized":"2024-04-20T21:29:45.335791+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:42:22.098265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.364509+0000","last_change":"2024-04-20T21:29:44.345912+0000","last_active":"2024-04-20T21:29:45.364509+0000","last_peered":"2024-04-20T21:29:45.364509+0000","last_clean":"2024-04-20T21:29:45.364509+0000","last_became_active":"2024-04-20T21:29:44.345213+0000","last_became_peered":"2024-04-20T21:29:44.345213+0000","last_unstale":"2024-04-20T21:29:45.364509+0000","last_undegraded":"2024-04-20T21:29:45.364509+0000","last_fullsized":"2024-04-20T21:29:45.364509+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:34:35.212237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.668571+0000","last_change":"2024-04-20T21:29:44.370474+0000","last_active":"2024-04-20T21:29:45.668571+0000","last_peered":"2024-04-20T21:29:45.668571+0000","last_clean":"2024-04-20T21:29:45.668571+0000","last_became_active":"2024-04-20T21:29:44.370158+0000","last_became_peered":"2024-04-20T21:29:44.370158+0000","last_unstale":"2024-04-20T21:29:45.668571+0000","last_undegraded":"2024-04-20T21:29:45.668571+0000","last_fullsized":"2024-04-20T21:29:45.668571+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:26:41.276861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.669086+0000","last_change":"2024-04-20T21:29:44.364089+0000","last_active":"2024-04-20T21:29:45.669086+0000","last_peered":"2024-04-20T21:29:45.669086+0000","last_clean":"2024-04-20T21:29:45.669086+0000","last_became_active":"2024-04-20T21:29:44.363835+0000","last_became_peered":"2024-04-20T21:29:44.363835+0000","last_unstale":"2024-04-20T21:29:45.669086+0000","last_undegraded":"2024-04-20T21:29:45.669086+0000","last_fullsized":"2024-04-20T21:29:45.669086+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:36:00.505654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.671338+0000","last_change":"2024-04-20T21:29:44.363440+0000","last_active":"2024-04-20T21:29:45.671338+0000","last_peered":"2024-04-20T21:29:45.671338+0000","last_clean":"2024-04-20T21:29:45.671338+0000","last_became_active":"2024-04-20T21:29:44.363116+0000","last_became_peered":"2024-04-20T21:29:44.363116+0000","last_unstale":"2024-04-20T21:29:45.671338+0000","last_undegraded":"2024-04-20T21:29:45.671338+0000","last_fullsized":"2024-04-20T21:29:45.671338+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:26:47.318941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"111'4","reported_seq":18,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.365557+0000","last_change":"2024-04-20T21:29:43.339073+0000","last_active":"2024-04-20T21:29:45.365557+0000","last_peered":"2024-04-20T21:29:45.365557+0000","last_clean":"2024-04-20T21:29:45.365557+0000","last_became_active":"2024-04-20T21:29:43.338573+0000","last_became_peered":"2024-04-20T21:29:43.338573+0000","last_unstale":"2024-04-20T21:29:45.365557+0000","last_undegraded":"2024-04-20T21:29:45.365557+0000","last_fullsized":"2024-04-20T21:29:45.365557+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:07:43.224631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.335130+0000","last_change":"2024-04-20T21:29:44.352707+0000","last_active":"2024-04-20T21:29:45.335130+0000","last_peered":"2024-04-20T21:29:45.335130+0000","last_clean":"2024-04-20T21:29:45.335130+0000","last_became_active":"2024-04-20T21:29:44.352281+0000","last_became_peered":"2024-04-20T21:29:44.352281+0000","last_unstale":"2024-04-20T21:29:45.335130+0000","last_undegraded":"2024-04-20T21:29:45.335130+0000","last_fullsized":"2024-04-20T21:29:45.335130+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:50:27.154529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"111'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.670218+0000","last_change":"2024-04-20T21:29:43.344751+0000","last_active":"2024-04-20T21:29:45.670218+0000","last_peered":"2024-04-20T21:29:45.670218+0000","last_clean":"2024-04-20T21:29:45.670218+0000","last_became_active":"2024-04-20T21:29:43.344392+0000","last_became_peered":"2024-04-20T21:29:43.344392+0000","last_unstale":"2024-04-20T21:29:45.670218+0000","last_undegraded":"2024-04-20T21:29:45.670218+0000","last_fullsized":"2024-04-20T21:29:45.670218+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:50:26.138384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.671118+0000","last_change":"2024-04-20T21:29:44.356394+0000","last_active":"2024-04-20T21:29:45.671118+0000","last_peered":"2024-04-20T21:29:45.671118+0000","last_clean":"2024-04-20T21:29:45.671118+0000","last_became_active":"2024-04-20T21:29:44.356071+0000","last_became_peered":"2024-04-20T21:29:44.356071+0000","last_unstale":"2024-04-20T21:29:45.671118+0000","last_undegraded":"2024-04-20T21:29:45.671118+0000","last_fullsized":"2024-04-20T21:29:45.671118+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:54:02.287967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"111'2","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.669861+0000","last_change":"2024-04-20T21:29:43.321851+0000","last_active":"2024-04-20T21:29:45.669861+0000","last_peered":"2024-04-20T21:29:45.669861+0000","last_clean":"2024-04-20T21:29:45.669861+0000","last_became_active":"2024-04-20T21:29:43.321480+0000","last_became_peered":"2024-04-20T21:29:43.321480+0000","last_unstale":"2024-04-20T21:29:45.669861+0000","last_undegraded":"2024-04-20T21:29:45.669861+0000","last_fullsized":"2024-04-20T21:29:45.669861+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:40:49.310112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.670231+0000","last_change":"2024-04-20T21:29:44.368108+0000","last_active":"2024-04-20T21:29:45.670231+0000","last_peered":"2024-04-20T21:29:45.670231+0000","last_clean":"2024-04-20T21:29:45.670231+0000","last_became_active":"2024-04-20T21:29:44.366711+0000","last_became_peered":"2024-04-20T21:29:44.366711+0000","last_unstale":"2024-04-20T21:29:45.670231+0000","last_undegraded":"2024-04-20T21:29:45.670231+0000","last_fullsized":"2024-04-20T21:29:45.670231+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:07:26.881957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"111'2","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.668923+0000","last_change":"2024-04-20T21:29:43.321794+0000","last_active":"2024-04-20T21:29:45.668923+0000","last_peered":"2024-04-20T21:29:45.668923+0000","last_clean":"2024-04-20T21:29:45.668923+0000","last_became_active":"2024-04-20T21:29:43.321437+0000","last_became_peered":"2024-04-20T21:29:43.321437+0000","last_unstale":"2024-04-20T21:29:45.668923+0000","last_undegraded":"2024-04-20T21:29:45.668923+0000","last_fullsized":"2024-04-20T21:29:45.668923+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:48:42.604987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.668878+0000","last_change":"2024-04-20T21:29:44.377619+0000","last_active":"2024-04-20T21:29:45.668878+0000","last_peered":"2024-04-20T21:29:45.668878+0000","last_clean":"2024-04-20T21:29:45.668878+0000","last_became_active":"2024-04-20T21:29:44.377338+0000","last_became_peered":"2024-04-20T21:29:44.377338+0000","last_unstale":"2024-04-20T21:29:45.668878+0000","last_undegraded":"2024-04-20T21:29:45.668878+0000","last_fullsized":"2024-04-20T21:29:45.668878+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:33:51.266811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"111'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.365001+0000","last_change":"2024-04-20T21:29:43.337340+0000","last_active":"2024-04-20T21:29:45.365001+0000","last_peered":"2024-04-20T21:29:45.365001+0000","last_clean":"2024-04-20T21:29:45.365001+0000","last_became_active":"2024-04-20T21:29:43.335975+0000","last_became_peered":"2024-04-20T21:29:43.335975+0000","last_unstale":"2024-04-20T21:29:45.365001+0000","last_undegraded":"2024-04-20T21:29:45.365001+0000","last_fullsized":"2024-04-20T21:29:45.365001+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:35:15.956972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.336245+0000","last_change":"2024-04-20T21:29:44.364846+0000","last_active":"2024-04-20T21:29:45.336245+0000","last_peered":"2024-04-20T21:29:45.336245+0000","last_clean":"2024-04-20T21:29:45.336245+0000","last_became_active":"2024-04-20T21:29:44.363910+0000","last_became_peered":"2024-04-20T21:29:44.363910+0000","last_unstale":"2024-04-20T21:29:45.336245+0000","last_undegraded":"2024-04-20T21:29:45.336245+0000","last_fullsized":"2024-04-20T21:29:45.336245+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:14:23.820951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"111'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.333229+0000","last_change":"2024-04-20T21:29:43.331564+0000","last_active":"2024-04-20T21:29:45.333229+0000","last_peered":"2024-04-20T21:29:45.333229+0000","last_clean":"2024-04-20T21:29:45.333229+0000","last_became_active":"2024-04-20T21:29:43.331204+0000","last_became_peered":"2024-04-20T21:29:43.331204+0000","last_unstale":"2024-04-20T21:29:45.333229+0000","last_undegraded":"2024-04-20T21:29:45.333229+0000","last_fullsized":"2024-04-20T21:29:45.333229+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:39:19.105304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.669151+0000","last_change":"2024-04-20T21:29:44.364222+0000","last_active":"2024-04-20T21:29:45.669151+0000","last_peered":"2024-04-20T21:29:45.669151+0000","last_clean":"2024-04-20T21:29:45.669151+0000","last_became_active":"2024-04-20T21:29:44.359967+0000","last_became_peered":"2024-04-20T21:29:44.359967+0000","last_unstale":"2024-04-20T21:29:45.669151+0000","last_undegraded":"2024-04-20T21:29:45.669151+0000","last_fullsized":"2024-04-20T21:29:45.669151+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:15:01.087200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.335574+0000","last_change":"2024-04-20T21:29:43.336906+0000","last_active":"2024-04-20T21:29:45.335574+0000","last_peered":"2024-04-20T21:29:45.335574+0000","last_clean":"2024-04-20T21:29:45.335574+0000","last_became_active":"2024-04-20T21:29:43.336305+0000","last_became_peered":"2024-04-20T21:29:43.336305+0000","last_unstale":"2024-04-20T21:29:45.335574+0000","last_undegraded":"2024-04-20T21:29:45.335574+0000","last_fullsized":"2024-04-20T21:29:45.335574+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:50:26.138384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.364283+0000","last_change":"2024-04-20T21:29:44.349585+0000","last_active":"2024-04-20T21:29:45.364283+0000","last_peered":"2024-04-20T21:29:45.364283+0000","last_clean":"2024-04-20T21:29:45.364283+0000","last_became_active":"2024-04-20T21:29:44.349120+0000","last_became_peered":"2024-04-20T21:29:44.349120+0000","last_unstale":"2024-04-20T21:29:45.364283+0000","last_undegraded":"2024-04-20T21:29:45.364283+0000","last_fullsized":"2024-04-20T21:29:45.364283+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:14:49.871482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.669099+0000","last_change":"2024-04-20T21:29:43.337896+0000","last_active":"2024-04-20T21:29:45.669099+0000","last_peered":"2024-04-20T21:29:45.669099+0000","last_clean":"2024-04-20T21:29:45.669099+0000","last_became_active":"2024-04-20T21:29:43.337440+0000","last_became_peered":"2024-04-20T21:29:43.337440+0000","last_unstale":"2024-04-20T21:29:45.669099+0000","last_undegraded":"2024-04-20T21:29:45.669099+0000","last_fullsized":"2024-04-20T21:29:45.669099+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:42:21.082120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.365353+0000","last_change":"2024-04-20T21:29:44.344994+0000","last_active":"2024-04-20T21:29:45.365353+0000","last_peered":"2024-04-20T21:29:45.365353+0000","last_clean":"2024-04-20T21:29:45.365353+0000","last_became_active":"2024-04-20T21:29:44.344226+0000","last_became_peered":"2024-04-20T21:29:44.344226+0000","last_unstale":"2024-04-20T21:29:45.365353+0000","last_undegraded":"2024-04-20T21:29:45.365353+0000","last_fullsized":"2024-04-20T21:29:45.365353+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:09:23.816382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.669497+0000","last_change":"2024-04-20T21:29:43.326187+0000","last_active":"2024-04-20T21:29:45.669497+0000","last_peered":"2024-04-20T21:29:45.669497+0000","last_clean":"2024-04-20T21:29:45.669497+0000","last_became_active":"2024-04-20T21:29:43.325900+0000","last_became_peered":"2024-04-20T21:29:43.325900+0000","last_unstale":"2024-04-20T21:29:45.669497+0000","last_undegraded":"2024-04-20T21:29:45.669497+0000","last_fullsized":"2024-04-20T21:29:45.669497+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:28:02.279050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.335788+0000","last_change":"2024-04-20T21:29:44.372545+0000","last_active":"2024-04-20T21:29:45.335788+0000","last_peered":"2024-04-20T21:29:45.335788+0000","last_clean":"2024-04-20T21:29:45.335788+0000","last_became_active":"2024-04-20T21:29:44.372190+0000","last_became_peered":"2024-04-20T21:29:44.372190+0000","last_unstale":"2024-04-20T21:29:45.335788+0000","last_undegraded":"2024-04-20T21:29:45.335788+0000","last_fullsized":"2024-04-20T21:29:45.335788+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:49:14.412117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.332734+0000","last_change":"2024-04-20T21:29:43.340366+0000","last_active":"2024-04-20T21:29:45.332734+0000","last_peered":"2024-04-20T21:29:45.332734+0000","last_clean":"2024-04-20T21:29:45.332734+0000","last_became_active":"2024-04-20T21:29:43.340048+0000","last_became_peered":"2024-04-20T21:29:43.340048+0000","last_unstale":"2024-04-20T21:29:45.332734+0000","last_undegraded":"2024-04-20T21:29:45.332734+0000","last_fullsized":"2024-04-20T21:29:45.332734+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:34:34.196092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.333235+0000","last_change":"2024-04-20T21:29:44.355980+0000","last_active":"2024-04-20T21:29:45.333235+0000","last_peered":"2024-04-20T21:29:45.333235+0000","last_clean":"2024-04-20T21:29:45.333235+0000","last_became_active":"2024-04-20T21:29:44.355690+0000","last_became_peered":"2024-04-20T21:29:44.355690+0000","last_unstale":"2024-04-20T21:29:45.333235+0000","last_undegraded":"2024-04-20T21:29:45.333235+0000","last_fullsized":"2024-04-20T21:29:45.333235+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:29:37.924593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"111'3","reported_seq":17,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.668721+0000","last_change":"2024-04-20T21:29:43.332776+0000","last_active":"2024-04-20T21:29:45.668721+0000","last_peered":"2024-04-20T21:29:45.668721+0000","last_clean":"2024-04-20T21:29:45.668721+0000","last_became_active":"2024-04-20T21:29:43.332469+0000","last_became_peered":"2024-04-20T21:29:43.332469+0000","last_unstale":"2024-04-20T21:29:45.668721+0000","last_undegraded":"2024-04-20T21:29:45.668721+0000","last_fullsized":"2024-04-20T21:29:45.668721+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:32:32.882685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.365139+0000","last_change":"2024-04-20T21:29:44.355468+0000","last_active":"2024-04-20T21:29:45.365139+0000","last_peered":"2024-04-20T21:29:45.365139+0000","last_clean":"2024-04-20T21:29:45.365139+0000","last_became_active":"2024-04-20T21:29:44.355136+0000","last_became_peered":"2024-04-20T21:29:44.355136+0000","last_unstale":"2024-04-20T21:29:45.365139+0000","last_undegraded":"2024-04-20T21:29:45.365139+0000","last_fullsized":"2024-04-20T21:29:45.365139+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:15:29.469663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"111'6","reported_seq":20,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.668658+0000","last_change":"2024-04-20T21:29:43.338112+0000","last_active":"2024-04-20T21:29:45.668658+0000","last_peered":"2024-04-20T21:29:45.668658+0000","last_clean":"2024-04-20T21:29:45.668658+0000","last_became_active":"2024-04-20T21:29:43.337051+0000","last_became_peered":"2024-04-20T21:29:43.337051+0000","last_unstale":"2024-04-20T21:29:45.668658+0000","last_undegraded":"2024-04-20T21:29:45.668658+0000","last_fullsized":"2024-04-20T21:29:45.668658+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:18:02.087371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.332998+0000","last_change":"2024-04-20T21:29:44.351822+0000","last_active":"2024-04-20T21:29:45.332998+0000","last_peered":"2024-04-20T21:29:45.332998+0000","last_clean":"2024-04-20T21:29:45.332998+0000","last_became_active":"2024-04-20T21:29:44.351479+0000","last_became_peered":"2024-04-20T21:29:44.351479+0000","last_unstale":"2024-04-20T21:29:45.332998+0000","last_undegraded":"2024-04-20T21:29:45.332998+0000","last_fullsized":"2024-04-20T21:29:45.332998+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:08:50.141052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"111'2","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.670897+0000","last_change":"2024-04-20T21:29:43.335170+0000","last_active":"2024-04-20T21:29:45.670897+0000","last_peered":"2024-04-20T21:29:45.670897+0000","last_clean":"2024-04-20T21:29:45.670897+0000","last_became_active":"2024-04-20T21:29:43.334553+0000","last_became_peered":"2024-04-20T21:29:43.334553+0000","last_unstale":"2024-04-20T21:29:45.670897+0000","last_undegraded":"2024-04-20T21:29:45.670897+0000","last_fullsized":"2024-04-20T21:29:45.670897+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:29:31.505502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.332742+0000","last_change":"2024-04-20T21:29:44.362433+0000","last_active":"2024-04-20T21:29:45.332742+0000","last_peered":"2024-04-20T21:29:45.332742+0000","last_clean":"2024-04-20T21:29:45.332742+0000","last_became_active":"2024-04-20T21:29:44.361615+0000","last_became_peered":"2024-04-20T21:29:44.361615+0000","last_unstale":"2024-04-20T21:29:45.332742+0000","last_undegraded":"2024-04-20T21:29:45.332742+0000","last_fullsized":"2024-04-20T21:29:45.332742+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:18:03.103516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.335828+0000","last_change":"2024-04-20T21:29:43.332508+0000","last_active":"2024-04-20T21:29:45.335828+0000","last_peered":"2024-04-20T21:29:45.335828+0000","last_clean":"2024-04-20T21:29:45.335828+0000","last_became_active":"2024-04-20T21:29:43.329900+0000","last_became_peered":"2024-04-20T21:29:43.329900+0000","last_unstale":"2024-04-20T21:29:45.335828+0000","last_undegraded":"2024-04-20T21:29:45.335828+0000","last_fullsized":"2024-04-20T21:29:45.335828+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T07:32:48.807084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.332206+0000","last_change":"2024-04-20T21:29:44.353877+0000","last_active":"2024-04-20T21:29:45.332206+0000","last_peered":"2024-04-20T21:29:45.332206+0000","last_clean":"2024-04-20T21:29:45.332206+0000","last_became_active":"2024-04-20T21:29:44.353534+0000","last_became_peered":"2024-04-20T21:29:44.353534+0000","last_unstale":"2024-04-20T21:29:45.332206+0000","last_undegraded":"2024-04-20T21:29:45.332206+0000","last_fullsized":"2024-04-20T21:29:45.332206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:12:00.136226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"111'8","reported_seq":22,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.668245+0000","last_change":"2024-04-20T21:29:43.332779+0000","last_active":"2024-04-20T21:29:45.668245+0000","last_peered":"2024-04-20T21:29:45.668245+0000","last_clean":"2024-04-20T21:29:45.668245+0000","last_became_active":"2024-04-20T21:29:43.332394+0000","last_became_peered":"2024-04-20T21:29:43.332394+0000","last_unstale":"2024-04-20T21:29:45.668245+0000","last_undegraded":"2024-04-20T21:29:45.668245+0000","last_fullsized":"2024-04-20T21:29:45.668245+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:52:53.484893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.669431+0000","last_change":"2024-04-20T21:29:44.360500+0000","last_active":"2024-04-20T21:29:45.669431+0000","last_peered":"2024-04-20T21:29:45.669431+0000","last_clean":"2024-04-20T21:29:45.669431+0000","last_became_active":"2024-04-20T21:29:44.360215+0000","last_became_peered":"2024-04-20T21:29:44.360215+0000","last_unstale":"2024-04-20T21:29:45.669431+0000","last_undegraded":"2024-04-20T21:29:45.669431+0000","last_fullsized":"2024-04-20T21:29:45.669431+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:19:51.279597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"111'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.671092+0000","last_change":"2024-04-20T21:29:43.337255+0000","last_active":"2024-04-20T21:29:45.671092+0000","last_peered":"2024-04-20T21:29:45.671092+0000","last_clean":"2024-04-20T21:29:45.671092+0000","last_became_active":"2024-04-20T21:29:43.336803+0000","last_became_peered":"2024-04-20T21:29:43.336803+0000","last_unstale":"2024-04-20T21:29:45.671092+0000","last_undegraded":"2024-04-20T21:29:45.671092+0000","last_fullsized":"2024-04-20T21:29:45.671092+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:07:25.865812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.671354+0000","last_change":"2024-04-20T21:29:44.378497+0000","last_active":"2024-04-20T21:29:45.671354+0000","last_peered":"2024-04-20T21:29:45.671354+0000","last_clean":"2024-04-20T21:29:45.671354+0000","last_became_active":"2024-04-20T21:29:44.378254+0000","last_became_peered":"2024-04-20T21:29:44.378254+0000","last_unstale":"2024-04-20T21:29:45.671354+0000","last_undegraded":"2024-04-20T21:29:45.671354+0000","last_fullsized":"2024-04-20T21:29:45.671354+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:35:16.973117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"111'2","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.669603+0000","last_change":"2024-04-20T21:29:43.326712+0000","last_active":"2024-04-20T21:29:45.669603+0000","last_peered":"2024-04-20T21:29:45.669603+0000","last_clean":"2024-04-20T21:29:45.669603+0000","last_became_active":"2024-04-20T21:29:43.324815+0000","last_became_peered":"2024-04-20T21:29:43.324815+0000","last_unstale":"2024-04-20T21:29:45.669603+0000","last_undegraded":"2024-04-20T21:29:45.669603+0000","last_fullsized":"2024-04-20T21:29:45.669603+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:29:31.505502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.669753+0000","last_change":"2024-04-20T21:29:44.371208+0000","last_active":"2024-04-20T21:29:45.669753+0000","last_peered":"2024-04-20T21:29:45.669753+0000","last_clean":"2024-04-20T21:29:45.669753+0000","last_became_active":"2024-04-20T21:29:44.370774+0000","last_became_peered":"2024-04-20T21:29:44.370774+0000","last_unstale":"2024-04-20T21:29:45.669753+0000","last_undegraded":"2024-04-20T21:29:45.669753+0000","last_fullsized":"2024-04-20T21:29:45.669753+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:42:22.098265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"111'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.334772+0000","last_change":"2024-04-20T21:29:43.332872+0000","last_active":"2024-04-20T21:29:45.334772+0000","last_peered":"2024-04-20T21:29:45.334772+0000","last_clean":"2024-04-20T21:29:45.334772+0000","last_became_active":"2024-04-20T21:29:43.330579+0000","last_became_peered":"2024-04-20T21:29:43.330579+0000","last_unstale":"2024-04-20T21:29:45.334772+0000","last_undegraded":"2024-04-20T21:29:45.334772+0000","last_fullsized":"2024-04-20T21:29:45.334772+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:51:29.472135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.670322+0000","last_change":"2024-04-20T21:29:44.367410+0000","last_active":"2024-04-20T21:29:45.670322+0000","last_peered":"2024-04-20T21:29:45.670322+0000","last_clean":"2024-04-20T21:29:45.670322+0000","last_became_active":"2024-04-20T21:29:44.366896+0000","last_became_peered":"2024-04-20T21:29:44.366896+0000","last_unstale":"2024-04-20T21:29:45.670322+0000","last_undegraded":"2024-04-20T21:29:45.670322+0000","last_fullsized":"2024-04-20T21:29:45.670322+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:30:57.290323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.364556+0000","last_change":"2024-04-20T21:29:43.336071+0000","last_active":"2024-04-20T21:29:45.364556+0000","last_peered":"2024-04-20T21:29:45.364556+0000","last_clean":"2024-04-20T21:29:45.364556+0000","last_became_active":"2024-04-20T21:29:43.335598+0000","last_became_peered":"2024-04-20T21:29:43.335598+0000","last_unstale":"2024-04-20T21:29:45.364556+0000","last_undegraded":"2024-04-20T21:29:45.364556+0000","last_fullsized":"2024-04-20T21:29:45.364556+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:26:46.302795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.670088+0000","last_change":"2024-04-20T21:29:44.361150+0000","last_active":"2024-04-20T21:29:45.670088+0000","last_peered":"2024-04-20T21:29:45.670088+0000","last_clean":"2024-04-20T21:29:45.670088+0000","last_became_active":"2024-04-20T21:29:44.360872+0000","last_became_peered":"2024-04-20T21:29:44.360872+0000","last_unstale":"2024-04-20T21:29:45.670088+0000","last_undegraded":"2024-04-20T21:29:45.670088+0000","last_fullsized":"2024-04-20T21:29:45.670088+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:45:50.712038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"111'2","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.670873+0000","last_change":"2024-04-20T21:29:43.328115+0000","last_active":"2024-04-20T21:29:45.670873+0000","last_peered":"2024-04-20T21:29:45.670873+0000","last_clean":"2024-04-20T21:29:45.670873+0000","last_became_active":"2024-04-20T21:29:43.327815+0000","last_became_peered":"2024-04-20T21:29:43.327815+0000","last_unstale":"2024-04-20T21:29:45.670873+0000","last_undegraded":"2024-04-20T21:29:45.670873+0000","last_fullsized":"2024-04-20T21:29:45.670873+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:34:44.200425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.668131+0000","last_change":"2024-04-20T21:29:44.364751+0000","last_active":"2024-04-20T21:29:45.668131+0000","last_peered":"2024-04-20T21:29:45.668131+0000","last_clean":"2024-04-20T21:29:45.668131+0000","last_became_active":"2024-04-20T21:29:44.358983+0000","last_became_peered":"2024-04-20T21:29:44.358983+0000","last_unstale":"2024-04-20T21:29:45.668131+0000","last_undegraded":"2024-04-20T21:29:45.668131+0000","last_fullsized":"2024-04-20T21:29:45.668131+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T02:47:48.597492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.669060+0000","last_change":"2024-04-20T21:29:43.326076+0000","last_active":"2024-04-20T21:29:45.669060+0000","last_peered":"2024-04-20T21:29:45.669060+0000","last_clean":"2024-04-20T21:29:45.669060+0000","last_became_active":"2024-04-20T21:29:43.325775+0000","last_became_peered":"2024-04-20T21:29:43.325775+0000","last_unstale":"2024-04-20T21:29:45.669060+0000","last_undegraded":"2024-04-20T21:29:45.669060+0000","last_fullsized":"2024-04-20T21:29:45.669060+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:54:50.130284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.669852+0000","last_change":"2024-04-20T21:29:44.364962+0000","last_active":"2024-04-20T21:29:45.669852+0000","last_peered":"2024-04-20T21:29:45.669852+0000","last_clean":"2024-04-20T21:29:45.669852+0000","last_became_active":"2024-04-20T21:29:44.363713+0000","last_became_peered":"2024-04-20T21:29:44.363713+0000","last_unstale":"2024-04-20T21:29:45.669852+0000","last_undegraded":"2024-04-20T21:29:45.669852+0000","last_fullsized":"2024-04-20T21:29:45.669852+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:14:23.820951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.670484+0000","last_change":"2024-04-20T21:29:43.330185+0000","last_active":"2024-04-20T21:29:45.670484+0000","last_peered":"2024-04-20T21:29:45.670484+0000","last_clean":"2024-04-20T21:29:45.670484+0000","last_became_active":"2024-04-20T21:29:43.329621+0000","last_became_peered":"2024-04-20T21:29:43.329621+0000","last_unstale":"2024-04-20T21:29:45.670484+0000","last_undegraded":"2024-04-20T21:29:45.670484+0000","last_fullsized":"2024-04-20T21:29:45.670484+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:40:49.310112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.332338+0000","last_change":"2024-04-20T21:29:44.372765+0000","last_active":"2024-04-20T21:29:45.332338+0000","last_peered":"2024-04-20T21:29:45.332338+0000","last_clean":"2024-04-20T21:29:45.332338+0000","last_became_active":"2024-04-20T21:29:44.372462+0000","last_became_peered":"2024-04-20T21:29:44.372462+0000","last_unstale":"2024-04-20T21:29:45.332338+0000","last_undegraded":"2024-04-20T21:29:45.332338+0000","last_fullsized":"2024-04-20T21:29:45.332338+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:22:50.948614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.669187+0000","last_change":"2024-04-20T21:29:43.335494+0000","last_active":"2024-04-20T21:29:45.669187+0000","last_peered":"2024-04-20T21:29:45.669187+0000","last_clean":"2024-04-20T21:29:45.669187+0000","last_became_active":"2024-04-20T21:29:43.335179+0000","last_became_peered":"2024-04-20T21:29:43.335179+0000","last_unstale":"2024-04-20T21:29:45.669187+0000","last_undegraded":"2024-04-20T21:29:45.669187+0000","last_fullsized":"2024-04-20T21:29:45.669187+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:29:36.908448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.334738+0000","last_change":"2024-04-20T21:29:44.359210+0000","last_active":"2024-04-20T21:29:45.334738+0000","last_peered":"2024-04-20T21:29:45.334738+0000","last_clean":"2024-04-20T21:29:45.334738+0000","last_became_active":"2024-04-20T21:29:44.358734+0000","last_became_peered":"2024-04-20T21:29:44.358734+0000","last_unstale":"2024-04-20T21:29:45.334738+0000","last_undegraded":"2024-04-20T21:29:45.334738+0000","last_fullsized":"2024-04-20T21:29:45.334738+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:28:24.144297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"111'2","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.333469+0000","last_change":"2024-04-20T21:29:43.338251+0000","last_active":"2024-04-20T21:29:45.333469+0000","last_peered":"2024-04-20T21:29:45.333469+0000","last_clean":"2024-04-20T21:29:45.333469+0000","last_became_active":"2024-04-20T21:29:43.337958+0000","last_became_peered":"2024-04-20T21:29:43.337958+0000","last_unstale":"2024-04-20T21:29:45.333469+0000","last_undegraded":"2024-04-20T21:29:45.333469+0000","last_fullsized":"2024-04-20T21:29:45.333469+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:15:28.453518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.670785+0000","last_change":"2024-04-20T21:29:44.371871+0000","last_active":"2024-04-20T21:29:45.670785+0000","last_peered":"2024-04-20T21:29:45.670785+0000","last_clean":"2024-04-20T21:29:45.670785+0000","last_became_active":"2024-04-20T21:29:44.371590+0000","last_became_peered":"2024-04-20T21:29:44.371590+0000","last_unstale":"2024-04-20T21:29:45.670785+0000","last_undegraded":"2024-04-20T21:29:45.670785+0000","last_fullsized":"2024-04-20T21:29:45.670785+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:27:39.892247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"112'1","reported_seq":15,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.363049+0000","last_change":"2024-04-20T21:29:43.334484+0000","last_active":"2024-04-20T21:29:45.363049+0000","last_peered":"2024-04-20T21:29:45.363049+0000","last_clean":"2024-04-20T21:29:45.363049+0000","last_became_active":"2024-04-20T21:29:43.334168+0000","last_became_peered":"2024-04-20T21:29:43.334168+0000","last_unstale":"2024-04-20T21:29:45.363049+0000","last_undegraded":"2024-04-20T21:29:45.363049+0000","last_fullsized":"2024-04-20T21:29:45.363049+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:17:45.018510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.668561+0000","last_change":"2024-04-20T21:29:44.366320+0000","last_active":"2024-04-20T21:29:45.668561+0000","last_peered":"2024-04-20T21:29:45.668561+0000","last_clean":"2024-04-20T21:29:45.668561+0000","last_became_active":"2024-04-20T21:29:44.365791+0000","last_became_peered":"2024-04-20T21:29:44.365791+0000","last_unstale":"2024-04-20T21:29:45.668561+0000","last_undegraded":"2024-04-20T21:29:45.668561+0000","last_fullsized":"2024-04-20T21:29:45.668561+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T01:32:45.366534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.668860+0000","last_change":"2024-04-20T21:29:43.330567+0000","last_active":"2024-04-20T21:29:45.668860+0000","last_peered":"2024-04-20T21:29:45.668860+0000","last_clean":"2024-04-20T21:29:45.668860+0000","last_became_active":"2024-04-20T21:29:43.330292+0000","last_became_peered":"2024-04-20T21:29:43.330292+0000","last_unstale":"2024-04-20T21:29:45.668860+0000","last_undegraded":"2024-04-20T21:29:45.668860+0000","last_fullsized":"2024-04-20T21:29:45.668860+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:08:49.124907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.334492+0000","last_change":"2024-04-20T21:29:44.357498+0000","last_active":"2024-04-20T21:29:45.334492+0000","last_peered":"2024-04-20T21:29:45.334492+0000","last_clean":"2024-04-20T21:29:45.334492+0000","last_became_active":"2024-04-20T21:29:44.357172+0000","last_became_peered":"2024-04-20T21:29:44.357172+0000","last_unstale":"2024-04-20T21:29:45.334492+0000","last_undegraded":"2024-04-20T21:29:45.334492+0000","last_fullsized":"2024-04-20T21:29:45.334492+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:39:20.121449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.333402+0000","last_change":"2024-04-20T21:29:43.338913+0000","last_active":"2024-04-20T21:29:45.333402+0000","last_peered":"2024-04-20T21:29:45.333402+0000","last_clean":"2024-04-20T21:29:45.333402+0000","last_became_active":"2024-04-20T21:29:43.338565+0000","last_became_peered":"2024-04-20T21:29:43.338565+0000","last_unstale":"2024-04-20T21:29:45.333402+0000","last_undegraded":"2024-04-20T21:29:45.333402+0000","last_fullsized":"2024-04-20T21:29:45.333402+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:09:22.800237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.335639+0000","last_change":"2024-04-20T21:29:44.363937+0000","last_active":"2024-04-20T21:29:45.335639+0000","last_peered":"2024-04-20T21:29:45.335639+0000","last_clean":"2024-04-20T21:29:45.335639+0000","last_became_active":"2024-04-20T21:29:44.363236+0000","last_became_peered":"2024-04-20T21:29:44.363236+0000","last_unstale":"2024-04-20T21:29:45.335639+0000","last_undegraded":"2024-04-20T21:29:45.335639+0000","last_fullsized":"2024-04-20T21:29:45.335639+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:08:50.141052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.365005+0000","last_change":"2024-04-20T21:29:43.337585+0000","last_active":"2024-04-20T21:29:45.365005+0000","last_peered":"2024-04-20T21:29:45.365005+0000","last_clean":"2024-04-20T21:29:45.365005+0000","last_became_active":"2024-04-20T21:29:43.337026+0000","last_became_peered":"2024-04-20T21:29:43.337026+0000","last_unstale":"2024-04-20T21:29:45.365005+0000","last_undegraded":"2024-04-20T21:29:45.365005+0000","last_fullsized":"2024-04-20T21:29:45.365005+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:54:01.271822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.336411+0000","last_change":"2024-04-20T21:29:44.358657+0000","last_active":"2024-04-20T21:29:45.336411+0000","last_peered":"2024-04-20T21:29:45.336411+0000","last_clean":"2024-04-20T21:29:45.336411+0000","last_became_active":"2024-04-20T21:29:44.358367+0000","last_became_peered":"2024-04-20T21:29:44.358367+0000","last_unstale":"2024-04-20T21:29:45.336411+0000","last_undegraded":"2024-04-20T21:29:45.336411+0000","last_fullsized":"2024-04-20T21:29:45.336411+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T04:57:47.970601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.335654+0000","last_change":"2024-04-20T21:29:43.332201+0000","last_active":"2024-04-20T21:29:45.335654+0000","last_peered":"2024-04-20T21:29:45.335654+0000","last_clean":"2024-04-20T21:29:45.335654+0000","last_became_active":"2024-04-20T21:29:43.331922+0000","last_became_peered":"2024-04-20T21:29:43.331922+0000","last_unstale":"2024-04-20T21:29:45.335654+0000","last_undegraded":"2024-04-20T21:29:45.335654+0000","last_fullsized":"2024-04-20T21:29:45.335654+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T23:28:23.128152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.670094+0000","last_change":"2024-04-20T21:29:44.365565+0000","last_active":"2024-04-20T21:29:45.670094+0000","last_peered":"2024-04-20T21:29:45.670094+0000","last_clean":"2024-04-20T21:29:45.670094+0000","last_became_active":"2024-04-20T21:29:44.365250+0000","last_became_peered":"2024-04-20T21:29:44.365250+0000","last_unstale":"2024-04-20T21:29:45.670094+0000","last_undegraded":"2024-04-20T21:29:45.670094+0000","last_fullsized":"2024-04-20T21:29:45.670094+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:20:17.475536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.668501+0000","last_change":"2024-04-20T21:29:43.322753+0000","last_active":"2024-04-20T21:29:45.668501+0000","last_peered":"2024-04-20T21:29:45.668501+0000","last_clean":"2024-04-20T21:29:45.668501+0000","last_became_active":"2024-04-20T21:29:43.322124+0000","last_became_peered":"2024-04-20T21:29:43.322124+0000","last_unstale":"2024-04-20T21:29:45.668501+0000","last_undegraded":"2024-04-20T21:29:45.668501+0000","last_fullsized":"2024-04-20T21:29:45.668501+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T00:34:44.200425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.668715+0000","last_change":"2024-04-20T21:29:44.378575+0000","last_active":"2024-04-20T21:29:45.668715+0000","last_peered":"2024-04-20T21:29:45.668715+0000","last_clean":"2024-04-20T21:29:45.668715+0000","last_became_active":"2024-04-20T21:29:44.378100+0000","last_became_peered":"2024-04-20T21:29:44.378100+0000","last_unstale":"2024-04-20T21:29:45.668715+0000","last_undegraded":"2024-04-20T21:29:45.668715+0000","last_fullsized":"2024-04-20T21:29:45.668715+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:22:41.884105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.669297+0000","last_change":"2024-04-20T21:29:43.330047+0000","last_active":"2024-04-20T21:29:45.669297+0000","last_peered":"2024-04-20T21:29:45.669297+0000","last_clean":"2024-04-20T21:29:45.669297+0000","last_became_active":"2024-04-20T21:29:43.329729+0000","last_became_peered":"2024-04-20T21:29:43.329729+0000","last_unstale":"2024-04-20T21:29:45.669297+0000","last_undegraded":"2024-04-20T21:29:45.669297+0000","last_fullsized":"2024-04-20T21:29:45.669297+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T06:14:22.804806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.332494+0000","last_change":"2024-04-20T21:29:44.355138+0000","last_active":"2024-04-20T21:29:45.332494+0000","last_peered":"2024-04-20T21:29:45.332494+0000","last_clean":"2024-04-20T21:29:45.332494+0000","last_became_active":"2024-04-20T21:29:44.354878+0000","last_became_peered":"2024-04-20T21:29:44.354878+0000","last_unstale":"2024-04-20T21:29:45.332494+0000","last_undegraded":"2024-04-20T21:29:45.332494+0000","last_fullsized":"2024-04-20T21:29:45.332494+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:43.293197+0000","last_clean_scrub_stamp":"2024-04-20T21:29:43.293197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T03:30:49.193474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.364547+0000","last_change":"2024-04-20T21:29:43.334963+0000","last_active":"2024-04-20T21:29:45.364547+0000","last_peered":"2024-04-20T21:29:45.364547+0000","last_clean":"2024-04-20T21:29:45.364547+0000","last_became_active":"2024-04-20T21:29:43.334654+0000","last_became_peered":"2024-04-20T21:29:43.334654+0000","last_unstale":"2024-04-20T21:29:45.364547+0000","last_undegraded":"2024-04-20T21:29:45.364547+0000","last_fullsized":"2024-04-20T21:29:45.364547+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:29:42.277052+0000","last_clean_scrub_stamp":"2024-04-20T21:29:42.277052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:45:08.149368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":338,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-04-20T21:29:45.670332+0000","last_change":"2024-04-20T21:17:43.059637+0000","last_active":"2024-04-20T21:29:45.670332+0000","last_peered":"2024-04-20T21:29:45.670332+0000","last_clean":"2024-04-20T21:29:45.670332+0000","last_became_active":"2024-04-20T21:17:43.056207+0000","last_became_peered":"2024-04-20T21:17:43.056207+0000","last_unstale":"2024-04-20T21:29:45.670332+0000","last_undegraded":"2024-04-20T21:29:45.670332+0000","last_fullsized":"2024-04-20T21:29:45.670332+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:41.036290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:41.036290+0000","last_clean_scrub_stamp":"2024-04-20T21:17:41.036290+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:58:51.091337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574995,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33124,"kb_used_data":1936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338716,"statfs":{"total":96636764160,"available":96602845184,"internally_reserved":0,"allocated":1982464,"data_stored":1189326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574996,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32536,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339304,"statfs":{"total":96636764160,"available":96603447296,"internally_reserved":0,"allocated":1384448,"data_stored":598412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574998,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32552,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339288,"statfs":{"total":96636764160,"available":96603430912,"internally_reserved":0,"allocated":1400832,"data_stored":605246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574997,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32548,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339292,"statfs":{"total":96636764160,"available":96603435008,"internally_reserved":0,"allocated":1392640,"data_stored":604642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574997,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32548,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339292,"statfs":{"total":96636764160,"available":96603435008,"internally_reserved":0,"allocated":1396736,"data_stored":600024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574997,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1380352,"data_stored":598342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574996,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32556,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339284,"statfs":{"total":96636764160,"available":96603426816,"internally_reserved":0,"allocated":1400832,"data_stored":604766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574997,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33124,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338716,"statfs":{"total":96636764160,"available":96602845184,"internally_reserved":0,"allocated":1994752,"data_stored":1190434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T21:29:49.248 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-20T21:29:49.248 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-20T21:29:49.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.503+0000 7f26c7200640 1 Processor -- start 2024-04-20T21:29:49.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.503+0000 7f26c7200640 1 -- start start 2024-04-20T21:29:49.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.504+0000 7f26c7200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f26c006bbf0 0x7f26c006e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:49.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.504+0000 7f26c7200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26c006bfd0 0x7f26c0145b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:49.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.504+0000 7f26c7200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f26c0146040 0x7f26c00cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:49.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.504+0000 7f26c7200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f26c015cd40 con 0x7f26c0146040 2024-04-20T21:29:49.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.504+0000 7f26c7200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f26c015d000 con 0x7f26c006bbf0 2024-04-20T21:29:49.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.504+0000 7f26c7200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f26c015d2e0 con 0x7f26c006bfd0 2024-04-20T21:29:49.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.505+0000 7f26c5200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f26c0146040 0x7f26c00cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:49.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.505+0000 7f26c5200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f26c0146040 0x7f26c00cd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49172/0 (socket says 172.21.15.86:49172) 2024-04-20T21:29:49.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.505+0000 7f26c5200640 1 -- 172.21.15.86:0/223567898 learned_addr learned my addr 172.21.15.86:0/223567898 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:49.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.505+0000 7f26b7400640 1 --2- 172.21.15.86:0/223567898 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26c006bfd0 0x7f26c0145b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:49.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.505+0000 7f26c5200640 1 -- 172.21.15.86:0/223567898 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26c006bfd0 msgr2=0x7f26c0145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:49.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.505+0000 7f26c5200640 1 --2- 172.21.15.86:0/223567898 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26c006bfd0 0x7f26c0145b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:49.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.505+0000 7f26bfe00640 1 --2- 172.21.15.86:0/223567898 >> v2:172.21.15.144:3300/0 conn(0x7f26c006bbf0 0x7f26c006e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:49.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.505+0000 7f26c5200640 1 -- 172.21.15.86:0/223567898 >> v2:172.21.15.144:3300/0 conn(0x7f26c006bbf0 msgr2=0x7f26c006e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:49.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.505+0000 7f26c5200640 1 --2- 172.21.15.86:0/223567898 >> v2:172.21.15.144:3300/0 conn(0x7f26c006bbf0 0x7f26c006e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:49.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.505+0000 7f26c5200640 1 -- 172.21.15.86:0/223567898 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26c015d5c0 con 0x7f26c0146040 2024-04-20T21:29:49.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.505+0000 7f26bfe00640 1 --2- 172.21.15.86:0/223567898 >> v2:172.21.15.144:3300/0 conn(0x7f26c006bbf0 0x7f26c006e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:29:49.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.507+0000 7f26c5200640 1 --2- 172.21.15.86:0/223567898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26c0146040 0x7f26c00cd540 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f26c0073150 tx=0x7f26b802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=632926b438b4f9fd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:49.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.507+0000 7f26bf400640 1 -- 172.21.15.86:0/223567898 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b802ca80 con 0x7f26c0146040 2024-04-20T21:29:49.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.507+0000 7f26bf400640 1 -- 172.21.15.86:0/223567898 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26b802cc20 con 0x7f26c0146040 2024-04-20T21:29:49.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.507+0000 7f26bf400640 1 -- 172.21.15.86:0/223567898 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b80343f0 con 0x7f26c0146040 2024-04-20T21:29:49.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.508+0000 7f26c7200640 1 -- 172.21.15.86:0/223567898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26c0146040 msgr2=0x7f26c00cd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:49.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.508+0000 7f26c7200640 1 --2- 172.21.15.86:0/223567898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26c0146040 0x7f26c00cd540 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f26c0073150 tx=0x7f26b802bee0 comp rx=0 tx=0).stop 2024-04-20T21:29:49.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.508+0000 7f26c7200640 1 -- 172.21.15.86:0/223567898 shutdown_connections 2024-04-20T21:29:49.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.508+0000 7f26c7200640 1 --2- 172.21.15.86:0/223567898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26c0146040 0x7f26c00cd540 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:49.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.508+0000 7f26c7200640 1 --2- 172.21.15.86:0/223567898 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26c006bfd0 0x7f26c0145b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:49.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.508+0000 7f26c7200640 1 --2- 172.21.15.86:0/223567898 >> v2:172.21.15.144:3300/0 conn(0x7f26c006bbf0 0x7f26c006e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:49.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.508+0000 7f26c7200640 1 -- 172.21.15.86:0/223567898 >> 172.21.15.86:0/223567898 conn(0x7f26c00b2110 msgr2=0x7f26c00b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:49.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.508+0000 7f26c7200640 1 -- 172.21.15.86:0/223567898 shutdown_connections 2024-04-20T21:29:49.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.508+0000 7f26c7200640 1 -- 172.21.15.86:0/223567898 wait complete. 2024-04-20T21:29:49.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.510+0000 7f26c7200640 1 Processor -- start 2024-04-20T21:29:49.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.510+0000 7f26c7200640 1 -- start start 2024-04-20T21:29:49.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.511+0000 7f26c7200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26c006bbf0 0x7f26c0141930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:49.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.511+0000 7f26c7200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26c006bfd0 0x7f26c014f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:49.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.511+0000 7f26c7200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26c0146040 0x7f26c01445c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:49.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.511+0000 7f26c7200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f26c00cdc00 con 0x7f26c006bfd0 2024-04-20T21:29:49.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.511+0000 7f26c7200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f26c00cda80 con 0x7f26c0146040 2024-04-20T21:29:49.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.511+0000 7f26c7200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f26c00cdd80 con 0x7f26c006bbf0 2024-04-20T21:29:49.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.511+0000 7f26b7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26c006bfd0 0x7f26c014f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:49.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.511+0000 7f26b7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26c006bfd0 0x7f26c014f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49178/0 (socket says 172.21.15.86:49178) 2024-04-20T21:29:49.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.511+0000 7f26b7400640 1 -- 172.21.15.86:0/3866132906 learned_addr learned my addr 172.21.15.86:0/3866132906 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:49.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.511+0000 7f26bfe00640 1 --2- 172.21.15.86:0/3866132906 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26c006bbf0 0x7f26c0141930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:49.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.511+0000 7f26c5200640 1 --2- 172.21.15.86:0/3866132906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26c0146040 0x7f26c01445c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:49.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.512+0000 7f26b7400640 1 -- 172.21.15.86:0/3866132906 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26c006bbf0 msgr2=0x7f26c0141930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:49.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.512+0000 7f26b7400640 1 --2- 172.21.15.86:0/3866132906 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26c006bbf0 0x7f26c0141930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:49.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.512+0000 7f26b7400640 1 -- 172.21.15.86:0/3866132906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26c0146040 msgr2=0x7f26c01445c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:49.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.512+0000 7f26b7400640 1 --2- 172.21.15.86:0/3866132906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26c0146040 0x7f26c01445c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:49.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.512+0000 7f26b7400640 1 -- 172.21.15.86:0/3866132906 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26c01f7ce0 con 0x7f26c006bfd0 2024-04-20T21:29:49.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.512+0000 7f26c5200640 1 --2- 172.21.15.86:0/3866132906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26c0146040 0x7f26c01445c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:29:49.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.512+0000 7f26b7400640 1 --2- 172.21.15.86:0/3866132906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26c006bfd0 0x7f26c014f650 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f26a800c820 tx=0x7f26a800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:49.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.512+0000 7f26bcc00640 1 -- 172.21.15.86:0/3866132906 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a8015070 con 0x7f26c006bfd0 2024-04-20T21:29:49.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.512+0000 7f26bcc00640 1 -- 172.21.15.86:0/3866132906 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26a8004020 con 0x7f26c006bfd0 2024-04-20T21:29:49.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.512+0000 7f26bcc00640 1 -- 172.21.15.86:0/3866132906 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a80043f0 con 0x7f26c006bfd0 2024-04-20T21:29:49.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.512+0000 7f26c7200640 1 -- 172.21.15.86:0/3866132906 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26c01f7fc0 con 0x7f26c006bfd0 2024-04-20T21:29:49.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.512+0000 7f26c7200640 1 -- 172.21.15.86:0/3866132906 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26c01f8580 con 0x7f26c006bfd0 2024-04-20T21:29:49.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.513+0000 7f26c7200640 1 -- 172.21.15.86:0/3866132906 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26c00689e0 con 0x7f26c006bfd0 2024-04-20T21:29:49.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.514+0000 7f26bcc00640 1 -- 172.21.15.86:0/3866132906 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f26a8007830 con 0x7f26c006bfd0 2024-04-20T21:29:49.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.515+0000 7f26bcc00640 1 --2- 172.21.15.86:0/3866132906 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f269c06c920 0x7f269c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:49.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.515+0000 7f26bcc00640 1 -- 172.21.15.86:0/3866132906 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7f26a808b920 con 0x7f26c006bfd0 2024-04-20T21:29:49.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.516+0000 7f26bfe00640 1 --2- 172.21.15.86:0/3866132906 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f269c06c920 0x7f269c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:49.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.516+0000 7f26bfe00640 1 --2- 172.21.15.86:0/3866132906 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f269c06c920 0x7f269c06ede0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f26b000a980 tx=0x7f26b0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:49.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.520+0000 7f26bcc00640 1 -- 172.21.15.86:0/3866132906 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f26a8059d40 con 0x7f26c006bfd0 2024-04-20T21:29:49.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.797+0000 7f26c7200640 1 -- 172.21.15.86:0/3866132906 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f26c006c780 con 0x7f26c006bfd0 2024-04-20T21:29:49.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.803+0000 7f26bcc00640 1 -- 172.21.15.86:0/3866132906 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) ==== 190+0+147 (secure 0 0 0) 0x7f26a8059640 con 0x7f26c006bfd0 2024-04-20T21:29:49.803 INFO:teuthology.orchestra.run.smithi086.stdout:[client.0] 2024-04-20T21:29:49.803 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQDBMCRmHWxyKRAANzhRi4eajp68h/jWkhFMnw== 2024-04-20T21:29:49.803 INFO:teuthology.orchestra.run.smithi086.stdout: caps mds = "allow" 2024-04-20T21:29:49.803 INFO:teuthology.orchestra.run.smithi086.stdout: caps mon = "allow r" 2024-04-20T21:29:49.803 INFO:teuthology.orchestra.run.smithi086.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-20T21:29:49.803 INFO:teuthology.orchestra.run.smithi086.stderr:updated caps for client.0 2024-04-20T21:29:49.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.805+0000 7f26c7200640 1 -- 172.21.15.86:0/3866132906 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f269c06c920 msgr2=0x7f269c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:49.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.805+0000 7f26c7200640 1 --2- 172.21.15.86:0/3866132906 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f269c06c920 0x7f269c06ede0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f26b000a980 tx=0x7f26b0008040 comp rx=0 tx=0).stop 2024-04-20T21:29:49.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.805+0000 7f26c7200640 1 -- 172.21.15.86:0/3866132906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26c006bfd0 msgr2=0x7f26c014f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:49.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.805+0000 7f26c7200640 1 --2- 172.21.15.86:0/3866132906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26c006bfd0 0x7f26c014f650 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f26a800c820 tx=0x7f26a800cce0 comp rx=0 tx=0).stop 2024-04-20T21:29:49.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.805+0000 7f26c7200640 1 -- 172.21.15.86:0/3866132906 shutdown_connections 2024-04-20T21:29:49.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.805+0000 7f26c7200640 1 --2- 172.21.15.86:0/3866132906 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f269c06c920 0x7f269c06ede0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:49.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.805+0000 7f26c7200640 1 --2- 172.21.15.86:0/3866132906 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26c0146040 0x7f26c01445c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:49.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.805+0000 7f26c7200640 1 --2- 172.21.15.86:0/3866132906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26c006bfd0 0x7f26c014f650 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:49.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.805+0000 7f26c7200640 1 --2- 172.21.15.86:0/3866132906 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f26c006bbf0 0x7f26c0141930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:49.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.805+0000 7f26c7200640 1 -- 172.21.15.86:0/3866132906 >> 172.21.15.86:0/3866132906 conn(0x7f26c00b2110 msgr2=0x7f26c00727f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:49.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.806+0000 7f26c7200640 1 -- 172.21.15.86:0/3866132906 shutdown_connections 2024-04-20T21:29:49.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:49.806+0000 7f26c7200640 1 -- 172.21.15.86:0/3866132906 wait complete. 2024-04-20T21:29:49.824 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:29:50.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.024+0000 7ff12ca00640 1 Processor -- start 2024-04-20T21:29:50.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.024+0000 7ff12ca00640 1 -- start start 2024-04-20T21:29:50.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.025+0000 7ff12ca00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff12815d6a0 0x7ff1280ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:50.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.025+0000 7ff12ca00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff128160890 con 0x7ff1280ad620 2024-04-20T21:29:50.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.025+0000 7ff12ca00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff128160b70 con 0x7ff128156ae0 2024-04-20T21:29:50.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.025+0000 7ff12ca00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff12806db20 con 0x7ff12815d6a0 2024-04-20T21:29:50.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.026+0000 7ff127e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7ff1280ad620 0x7ff12806e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:33876/0 (socket says 172.21.15.86:33876) 2024-04-20T21:29:50.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.026+0000 7ff127e00640 1 -- 172.21.15.86:0/2909959946 learned_addr learned my addr 172.21.15.86:0/2909959946 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:29:50.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.026+0000 7ff11ea00640 1 --2- 172.21.15.86:0/2909959946 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff12815d6a0 0x7ff1280ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:50.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.026+0000 7ff126a00640 1 -- 172.21.15.86:0/2909959946 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2527207337 0 0) 0x7ff128160890 con 0x7ff1280ad620 2024-04-20T21:29:50.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.026+0000 7ff126a00640 1 -- 172.21.15.86:0/2909959946 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff10c003520 con 0x7ff1280ad620 2024-04-20T21:29:50.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.027+0000 7ff126a00640 1 -- 172.21.15.86:0/2909959946 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3116525588 0 0) 0x7ff10c003520 con 0x7ff1280ad620 2024-04-20T21:29:50.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.027+0000 7ff126a00640 1 -- 172.21.15.86:0/2909959946 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff12815d6a0 msgr2=0x7ff1280ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:50.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.027+0000 7ff126a00640 1 --2- 172.21.15.86:0/2909959946 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff12815d6a0 0x7ff1280ad0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:50.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.027+0000 7ff126a00640 1 -- 172.21.15.86:0/2909959946 >> v1:172.21.15.144:6789/0 conn(0x7ff128156ae0 legacy=0x7ff128160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:50.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.027+0000 7ff126a00640 1 -- 172.21.15.86:0/2909959946 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff128160e50 con 0x7ff1280ad620 2024-04-20T21:29:50.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.027+0000 7ff126a00640 1 -- 172.21.15.86:0/2909959946 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7ff120001670 con 0x7ff1280ad620 2024-04-20T21:29:50.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.027+0000 7ff126a00640 1 -- 172.21.15.86:0/2909959946 >> v1:172.21.15.86:6789/0 conn(0x7ff1280ad620 legacy=0x7ff12806e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:50.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.027+0000 7ff126a00640 1 --2- 172.21.15.86:0/2909959946 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff12815d6a0 0x7ff10c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:50.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.027+0000 7ff126a00640 1 --2- 172.21.15.86:0/2909959946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff10c003f60 0x7ff10c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:50.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.027+0000 7ff126a00640 1 --2- 172.21.15.86:0/2909959946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff10c004d40 0x7ff10c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:50.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.027+0000 7ff126a00640 1 -- 172.21.15.86:0/2909959946 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff12806db20 con 0x7ff10c003f60 2024-04-20T21:29:50.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.027+0000 7ff126a00640 1 -- 172.21.15.86:0/2909959946 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff10c00baa0 con 0x7ff10c004d40 2024-04-20T21:29:50.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.027+0000 7ff126a00640 1 -- 172.21.15.86:0/2909959946 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff10c00bca0 con 0x7ff12815d6a0 2024-04-20T21:29:50.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.027+0000 7ff11ea00640 1 --2- 172.21.15.86:0/2909959946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff10c003f60 0x7ff10c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:50.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.028+0000 7ff127e00640 1 --2- 172.21.15.86:0/2909959946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff10c004d40 0x7ff10c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:50.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.028+0000 7ff127400640 1 --2- 172.21.15.86:0/2909959946 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff12815d6a0 0x7ff10c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:50.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.028+0000 7ff11ea00640 1 -- 172.21.15.86:0/2909959946 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff12815d6a0 msgr2=0x7ff10c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:50.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.028+0000 7ff11ea00640 1 --2- 172.21.15.86:0/2909959946 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff12815d6a0 0x7ff10c0039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:50.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.028+0000 7ff11ea00640 1 -- 172.21.15.86:0/2909959946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff10c004d40 msgr2=0x7ff10c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:50.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.028+0000 7ff11ea00640 1 --2- 172.21.15.86:0/2909959946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff10c004d40 0x7ff10c0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:50.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.028+0000 7ff11ea00640 1 -- 172.21.15.86:0/2909959946 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff10c003520 con 0x7ff10c003f60 2024-04-20T21:29:50.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.029+0000 7ff11ea00640 1 --2- 172.21.15.86:0/2909959946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff10c003f60 0x7ff10c004800 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7ff110009ac0 tx=0x7ff10c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eddb793c109e939 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:50.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.029+0000 7ff126a00640 1 -- 172.21.15.86:0/2909959946 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff110039070 con 0x7ff10c003f60 2024-04-20T21:29:50.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.029+0000 7ff126a00640 1 -- 172.21.15.86:0/2909959946 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff11002c6b0 con 0x7ff10c003f60 2024-04-20T21:29:50.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.029+0000 7ff126a00640 1 -- 172.21.15.86:0/2909959946 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff110009690 con 0x7ff10c003f60 2024-04-20T21:29:50.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.029+0000 7ff12ca00640 1 -- 172.21.15.86:0/2909959946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff10c003f60 msgr2=0x7ff10c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:50.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.029+0000 7ff12ca00640 1 --2- 172.21.15.86:0/2909959946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff10c003f60 0x7ff10c004800 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7ff110009ac0 tx=0x7ff10c005830 comp rx=0 tx=0).stop 2024-04-20T21:29:50.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.030+0000 7ff127e00640 1 -- 172.21.15.86:0/2909959946 reap_dead start 2024-04-20T21:29:50.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.030+0000 7ff12ca00640 1 -- 172.21.15.86:0/2909959946 shutdown_connections 2024-04-20T21:29:50.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.030+0000 7ff12ca00640 1 -- 172.21.15.86:0/2909959946 >> 172.21.15.86:0/2909959946 conn(0x7ff1280d0170 msgr2=0x7ff1280d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:50.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.030+0000 7ff12ca00640 1 -- 172.21.15.86:0/2909959946 shutdown_connections 2024-04-20T21:29:50.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.030+0000 7ff12ca00640 1 -- 172.21.15.86:0/2909959946 wait complete. 2024-04-20T21:29:50.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.031+0000 7ff12ca00640 1 Processor -- start 2024-04-20T21:29:50.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.032+0000 7ff12ca00640 1 -- start start 2024-04-20T21:29:50.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.033+0000 7ff12ca00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff10c003f60 0x7ff1280b0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:50.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.033+0000 7ff12ca00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff12808bf70 0x7ff128099080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:50.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.033+0000 7ff12ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff12815ead0 0x7ff12815ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:50.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.033+0000 7ff12ca00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff1280ad0e0 con 0x7ff12815ead0 2024-04-20T21:29:50.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.033+0000 7ff12ca00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff128156ec0 con 0x7ff12808bf70 2024-04-20T21:29:50.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.033+0000 7ff12ca00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff1280ad260 con 0x7ff10c003f60 2024-04-20T21:29:50.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.033+0000 7ff127e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff12815ead0 0x7ff12815ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:50.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.033+0000 7ff127e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff12815ead0 0x7ff12815ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49202/0 (socket says 172.21.15.86:49202) 2024-04-20T21:29:50.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.033+0000 7ff127e00640 1 -- 172.21.15.86:0/2181667880 learned_addr learned my addr 172.21.15.86:0/2181667880 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:50.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.033+0000 7ff127400640 1 --2- 172.21.15.86:0/2181667880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff10c003f60 0x7ff1280b0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:50.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.033+0000 7ff11ea00640 1 --2- 172.21.15.86:0/2181667880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff12808bf70 0x7ff128099080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:50.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.033+0000 7ff127e00640 1 -- 172.21.15.86:0/2181667880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff10c003f60 msgr2=0x7ff1280b0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:50.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.033+0000 7ff127e00640 1 --2- 172.21.15.86:0/2181667880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff10c003f60 0x7ff1280b0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:50.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.033+0000 7ff127e00640 1 -- 172.21.15.86:0/2181667880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff12808bf70 msgr2=0x7ff128099080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:50.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.033+0000 7ff127e00640 1 --2- 172.21.15.86:0/2181667880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff12808bf70 0x7ff128099080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:50.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.033+0000 7ff127e00640 1 -- 172.21.15.86:0/2181667880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff12808c3f0 con 0x7ff12815ead0 2024-04-20T21:29:50.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.033+0000 7ff11ea00640 1 --2- 172.21.15.86:0/2181667880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff12808bf70 0x7ff128099080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:29:50.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.034+0000 7ff127e00640 1 --2- 172.21.15.86:0/2181667880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff12815ead0 0x7ff12815ef50 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7ff120000fb0 tx=0x7ff12000d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:50.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.034+0000 7ff11fe00640 1 -- 172.21.15.86:0/2181667880 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff120007500 con 0x7ff12815ead0 2024-04-20T21:29:50.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.034+0000 7ff11fe00640 1 -- 172.21.15.86:0/2181667880 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1200047e0 con 0x7ff12815ead0 2024-04-20T21:29:50.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.034+0000 7ff11fe00640 1 -- 172.21.15.86:0/2181667880 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff120004bd0 con 0x7ff12815ead0 2024-04-20T21:29:50.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.034+0000 7ff12ca00640 1 -- 172.21.15.86:0/2181667880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1281421a0 con 0x7ff12815ead0 2024-04-20T21:29:50.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.034+0000 7ff12ca00640 1 -- 172.21.15.86:0/2181667880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff12808c960 con 0x7ff12815ead0 2024-04-20T21:29:50.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.035+0000 7ff12ca00640 1 -- 172.21.15.86:0/2181667880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff12804fcf0 con 0x7ff12815ead0 2024-04-20T21:29:50.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.036+0000 7ff11fe00640 1 -- 172.21.15.86:0/2181667880 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff12001b020 con 0x7ff12815ead0 2024-04-20T21:29:50.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.038+0000 7ff11fe00640 1 --2- 172.21.15.86:0/2181667880 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff10406c8f0 0x7ff10406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:50.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.038+0000 7ff11fe00640 1 -- 172.21.15.86:0/2181667880 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7ff12008b690 con 0x7ff12815ead0 2024-04-20T21:29:50.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.038+0000 7ff127400640 1 --2- 172.21.15.86:0/2181667880 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff10406c8f0 0x7ff10406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:50.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.039+0000 7ff127400640 1 --2- 172.21.15.86:0/2181667880 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff10406c8f0 0x7ff10406edb0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7ff118007fd0 tx=0x7ff118009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:50.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.042+0000 7ff11fe00640 1 -- 172.21.15.86:0/2181667880 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff12000f2f0 con 0x7ff12815ead0 2024-04-20T21:29:50.272 INFO:tasks.ceph.mds.a.smithi086.stderr:2024-04-20T21:29:50.272+0000 7f4d3f600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T21:29:50.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.281+0000 7ff12ca00640 1 -- 172.21.15.86:0/2181667880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1280b13a0 con 0x7ff12815ead0 2024-04-20T21:29:50.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.282+0000 7ff11fe00640 1 -- 172.21.15.86:0/2181667880 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) ==== 94+0+5204 (secure 0 0 0) 0x7ff120059a20 con 0x7ff12815ead0 2024-04-20T21:29:50.282 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:29:50.282 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":99,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6253,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3627230007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3627230007},{"type":"v1","addr":"172.21.15.144:6833","nonce":3627230007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":6256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2498297133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2498297133},{"type":"v1","addr":"172.21.15.144:6835","nonce":2498297133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":9228,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/1120740475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1120740475},{"type":"v1","addr":"172.21.15.86:6836","nonce":1120740475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:29:43.769581+0000","modified":"2024-04-20T21:29:46.859349+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9231},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9231":{"gid":9231,"name":"a","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.86:6837/289590924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":289590924},{"type":"v1","addr":"172.21.15.86:6837","nonce":289590924}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9231,"qdb_cluster":[9231]},"id":8}]} 2024-04-20T21:29:50.282 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 99 2024-04-20T21:29:50.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.284+0000 7ff12ca00640 1 -- 172.21.15.86:0/2181667880 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff10406c8f0 msgr2=0x7ff10406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:50.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.284+0000 7ff12ca00640 1 --2- 172.21.15.86:0/2181667880 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff10406c8f0 0x7ff10406edb0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7ff118007fd0 tx=0x7ff118009040 comp rx=0 tx=0).stop 2024-04-20T21:29:50.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.284+0000 7ff12ca00640 1 -- 172.21.15.86:0/2181667880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff12815ead0 msgr2=0x7ff12815ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:50.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.284+0000 7ff12ca00640 1 --2- 172.21.15.86:0/2181667880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff12815ead0 0x7ff12815ef50 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7ff120000fb0 tx=0x7ff12000d9d0 comp rx=0 tx=0).stop 2024-04-20T21:29:50.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.284+0000 7ff12ca00640 1 -- 172.21.15.86:0/2181667880 shutdown_connections 2024-04-20T21:29:50.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.284+0000 7ff12ca00640 1 --2- 172.21.15.86:0/2181667880 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff10406c8f0 0x7ff10406edb0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:50.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.284+0000 7ff12ca00640 1 --2- 172.21.15.86:0/2181667880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff12815ead0 0x7ff12815ef50 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:50.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.284+0000 7ff12ca00640 1 --2- 172.21.15.86:0/2181667880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff12808bf70 0x7ff128099080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:50.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.284+0000 7ff12ca00640 1 --2- 172.21.15.86:0/2181667880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff10c003f60 0x7ff1280b0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:50.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.284+0000 7ff12ca00640 1 -- 172.21.15.86:0/2181667880 >> 172.21.15.86:0/2181667880 conn(0x7ff1280d0170 msgr2=0x7ff1280b4030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:50.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.285+0000 7ff12ca00640 1 -- 172.21.15.86:0/2181667880 shutdown_connections 2024-04-20T21:29:50.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.285+0000 7ff12ca00640 1 -- 172.21.15.86:0/2181667880 wait complete. 2024-04-20T21:29:50.706 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 99, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T21:29:43.769581+0000', 'modified': '2024-04-20T21:29:46.859349+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9231}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9231': {'gid': 9231, 'name': 'a', 'rank': 0, 'incarnation': 95, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.86:6837/289590924', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6835', 'nonce': 289590924}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 289590924}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9231, 'qdb_cluster': [9231]} 2024-04-20T21:29:50.707 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-20T21:29:50.707 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-20T21:29:50.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.918+0000 7f38bcc00640 1 Processor -- start 2024-04-20T21:29:50.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.919+0000 7f38bcc00640 1 -- start start 2024-04-20T21:29:50.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.920+0000 7f38bcc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38b80b2870 0x7f38b80cd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:50.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.920+0000 7f38bcc00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f38b80b0c40 0x7f38b80cd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:50.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.920+0000 7f38bcc00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38b80b1120 con 0x7f38b815c880 2024-04-20T21:29:50.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.920+0000 7f38bcc00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f38b80b1400 con 0x7f38b80b0c40 2024-04-20T21:29:50.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.920+0000 7f38bcc00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f38b80b16e0 con 0x7f38b80b2870 2024-04-20T21:29:50.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.921+0000 7f38b6800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f38b80b0c40 0x7f38b80cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:50.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.921+0000 7f38b5e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f38b815c880 0x7f38b80b2160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:33878/0 (socket says 172.21.15.86:33878) 2024-04-20T21:29:50.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.921+0000 7f38b5e00640 1 -- 172.21.15.86:0/2774733393 learned_addr learned my addr 172.21.15.86:0/2774733393 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:29:50.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.921+0000 7f38b5400640 1 --2- 172.21.15.86:0/2774733393 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38b80b2870 0x7f38b80cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:50.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.921+0000 7f38b6800640 1 -- 172.21.15.86:0/2774733393 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38b80b2870 msgr2=0x7f38b80cd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:50.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.921+0000 7f38b6800640 1 --2- 172.21.15.86:0/2774733393 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38b80b2870 0x7f38b80cd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:50.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.921+0000 7f38b6800640 1 -- 172.21.15.86:0/2774733393 >> v1:172.21.15.86:6789/0 conn(0x7f38b815c880 legacy=0x7f38b80b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:50.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.921+0000 7f38b6800640 1 -- 172.21.15.86:0/2774733393 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38b80b19c0 con 0x7f38b80b0c40 2024-04-20T21:29:50.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.921+0000 7f38b4a00640 1 -- 172.21.15.86:0/2774733393 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3481015114 0 0) 0x7f38b80b1120 con 0x7f38b815c880 2024-04-20T21:29:50.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.922+0000 7f38b6800640 1 --2- 172.21.15.86:0/2774733393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38b80b0c40 0x7f38b80cd940 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f38b00097a0 tx=0x7f38b002b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd7703d22b5fa87a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:50.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.923+0000 7f38b4a00640 1 -- 172.21.15.86:0/2774733393 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b0031780 con 0x7f38b80b0c40 2024-04-20T21:29:50.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.923+0000 7f38b4a00640 1 -- 172.21.15.86:0/2774733393 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38b0037070 con 0x7f38b80b0c40 2024-04-20T21:29:50.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.923+0000 7f38b4a00640 1 -- 172.21.15.86:0/2774733393 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b0031b40 con 0x7f38b80b0c40 2024-04-20T21:29:50.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.923+0000 7f38bcc00640 1 -- 172.21.15.86:0/2774733393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38b80b0c40 msgr2=0x7f38b80cd940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:50.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.923+0000 7f38bcc00640 1 --2- 172.21.15.86:0/2774733393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38b80b0c40 0x7f38b80cd940 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f38b00097a0 tx=0x7f38b002b820 comp rx=0 tx=0).stop 2024-04-20T21:29:50.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.923+0000 7f38bcc00640 1 -- 172.21.15.86:0/2774733393 shutdown_connections 2024-04-20T21:29:50.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.923+0000 7f38bcc00640 1 --2- 172.21.15.86:0/2774733393 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38b80b0c40 0x7f38b80cd940 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:50.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.923+0000 7f38bcc00640 1 --2- 172.21.15.86:0/2774733393 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38b80b2870 0x7f38b80cd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:50.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.923+0000 7f38bcc00640 1 -- 172.21.15.86:0/2774733393 >> 172.21.15.86:0/2774733393 conn(0x7f38b809a9e0 msgr2=0x7f38b80b2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:50.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.923+0000 7f38bcc00640 1 -- 172.21.15.86:0/2774733393 shutdown_connections 2024-04-20T21:29:50.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.923+0000 7f38bcc00640 1 -- 172.21.15.86:0/2774733393 wait complete. 2024-04-20T21:29:50.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.925+0000 7f38bcc00640 1 Processor -- start 2024-04-20T21:29:50.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.925+0000 7f38bcc00640 1 -- start start 2024-04-20T21:29:50.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.926+0000 7f38bcc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38b80b0c40 0x7f38b80c7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:50.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.926+0000 7f38bcc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38b80b2870 0x7f38b80c5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:50.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.926+0000 7f38bcc00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38b815c880 0x7f38b80b5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:50.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.926+0000 7f38bcc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f38b806b980 con 0x7f38b80b0c40 2024-04-20T21:29:50.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.926+0000 7f38bcc00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f38b80b3d70 con 0x7f38b815c880 2024-04-20T21:29:50.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.926+0000 7f38bcc00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f38b806bb00 con 0x7f38b80b2870 2024-04-20T21:29:50.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.926+0000 7f38b5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38b80b0c40 0x7f38b80c7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:50.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.926+0000 7f38b5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38b80b0c40 0x7f38b80c7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49222/0 (socket says 172.21.15.86:49222) 2024-04-20T21:29:50.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.926+0000 7f38b5e00640 1 -- 172.21.15.86:0/2386765319 learned_addr learned my addr 172.21.15.86:0/2386765319 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:50.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.927+0000 7f38b5400640 1 --2- 172.21.15.86:0/2386765319 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38b80b2870 0x7f38b80c5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:50.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.927+0000 7f38b6800640 1 --2- 172.21.15.86:0/2386765319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38b815c880 0x7f38b80b5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:50.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.927+0000 7f38b5e00640 1 -- 172.21.15.86:0/2386765319 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38b80b2870 msgr2=0x7f38b80c5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:50.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.927+0000 7f38b5e00640 1 --2- 172.21.15.86:0/2386765319 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38b80b2870 0x7f38b80c5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:50.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.927+0000 7f38b5e00640 1 -- 172.21.15.86:0/2386765319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38b815c880 msgr2=0x7f38b80b5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:50.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.927+0000 7f38b5e00640 1 --2- 172.21.15.86:0/2386765319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38b815c880 0x7f38b80b5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:50.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.927+0000 7f38b5e00640 1 -- 172.21.15.86:0/2386765319 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38b80c7bb0 con 0x7f38b80b0c40 2024-04-20T21:29:50.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.927+0000 7f38b5e00640 1 --2- 172.21.15.86:0/2386765319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38b80b0c40 0x7f38b80c7670 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f38a000c4d0 tx=0x7f38a000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:50.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.927+0000 7f38ae000640 1 -- 172.21.15.86:0/2386765319 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a0018070 con 0x7f38b80b0c40 2024-04-20T21:29:50.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.927+0000 7f38ae000640 1 -- 172.21.15.86:0/2386765319 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38a0007500 con 0x7f38b80b0c40 2024-04-20T21:29:50.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.927+0000 7f38ae000640 1 -- 172.21.15.86:0/2386765319 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a0004a20 con 0x7f38b80b0c40 2024-04-20T21:29:50.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.927+0000 7f38bcc00640 1 -- 172.21.15.86:0/2386765319 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38b80c4b60 con 0x7f38b80b0c40 2024-04-20T21:29:50.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.928+0000 7f38bcc00640 1 -- 172.21.15.86:0/2386765319 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38b80c5120 con 0x7f38b80b0c40 2024-04-20T21:29:50.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.931+0000 7f38ae000640 1 -- 172.21.15.86:0/2386765319 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f38a0004bc0 con 0x7f38b80b0c40 2024-04-20T21:29:50.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.931+0000 7f38ae000640 1 --2- 172.21.15.86:0/2386765319 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f389406c8f0 0x7f389406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:50.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.931+0000 7f38ae000640 1 -- 172.21.15.86:0/2386765319 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7f38a008ae50 con 0x7f38b80b0c40 2024-04-20T21:29:50.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.931+0000 7f38b5400640 1 --2- 172.21.15.86:0/2386765319 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f389406c8f0 0x7f389406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:50.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.931+0000 7f38bfa1b740 1 Processor -- start 2024-04-20T21:29:50.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.931+0000 7f38bfa1b740 1 -- start start 2024-04-20T21:29:50.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.931+0000 7f38b5400640 1 --2- 172.21.15.86:0/2386765319 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f389406c8f0 0x7f389406edb0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f38b80c7ee0 tx=0x7f38a800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:50.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.932+0000 7f38bfa1b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca4a099460 0x55ca4a050a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:50.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.932+0000 7f38bfa1b740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ca4a04f4e0 0x55ca4a050fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:50.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.932+0000 7f38bfa1b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca4a04b930 0x55ca4a07b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:50.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.932+0000 7f38bfa1b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ca49fb79d0 con 0x55ca4a099460 2024-04-20T21:29:50.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.932+0000 7f38bfa1b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ca49f5e270 con 0x55ca4a04b930 2024-04-20T21:29:50.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.932+0000 7f38bfa1b740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55ca49fb3990 con 0x55ca4a04f4e0 2024-04-20T21:29:50.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.932+0000 7f38b5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca4a099460 0x55ca4a050a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:50.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.932+0000 7f38b5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca4a099460 0x55ca4a050a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49224/0 (socket says 172.21.15.86:49224) 2024-04-20T21:29:50.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.932+0000 7f38b5e00640 1 -- 172.21.15.86:0/4229941744 learned_addr learned my addr 172.21.15.86:0/4229941744 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:50.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.933+0000 7f38b5e00640 1 -- 172.21.15.86:0/4229941744 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ca4a04f4e0 msgr2=0x55ca4a050fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:50.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.933+0000 7f38b5e00640 1 --2- 172.21.15.86:0/4229941744 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ca4a04f4e0 0x55ca4a050fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:50.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.933+0000 7f38b5e00640 1 -- 172.21.15.86:0/4229941744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca4a04b930 msgr2=0x55ca4a07b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:50.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.933+0000 7f38b5e00640 1 --2- 172.21.15.86:0/4229941744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca4a04b930 0x55ca4a07b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:50.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.933+0000 7f38b5e00640 1 -- 172.21.15.86:0/4229941744 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca49cca2d0 con 0x55ca4a099460 2024-04-20T21:29:50.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.933+0000 7f38b5e00640 1 --2- 172.21.15.86:0/4229941744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca4a099460 0x55ca4a050a90 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f38a0004fd0 tx=0x7f38a0058ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9268376aa67de601 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:50.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.933+0000 7f38a6600640 1 -- 172.21.15.86:0/4229941744 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a0056de0 con 0x55ca4a099460 2024-04-20T21:29:50.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.933+0000 7f38a6600640 1 -- 172.21.15.86:0/4229941744 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38a0055150 con 0x55ca4a099460 2024-04-20T21:29:50.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.933+0000 7f38a6600640 1 -- 172.21.15.86:0/4229941744 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a0058170 con 0x55ca4a099460 2024-04-20T21:29:50.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.934+0000 7f38bfa1b740 1 -- 172.21.15.86:0/4229941744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca4a099460 msgr2=0x55ca4a050a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:50.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.934+0000 7f38bfa1b740 1 --2- 172.21.15.86:0/4229941744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca4a099460 0x55ca4a050a90 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f38a0004fd0 tx=0x7f38a0058ae0 comp rx=0 tx=0).stop 2024-04-20T21:29:50.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.934+0000 7f38bfa1b740 1 -- 172.21.15.86:0/4229941744 shutdown_connections 2024-04-20T21:29:50.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.934+0000 7f38bfa1b740 1 --2- 172.21.15.86:0/4229941744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca4a04b930 0x55ca4a07b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:50.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.934+0000 7f38bfa1b740 1 --2- 172.21.15.86:0/4229941744 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ca4a04f4e0 0x55ca4a050fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:50.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.934+0000 7f38bfa1b740 1 --2- 172.21.15.86:0/4229941744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca4a099460 0x55ca4a050a90 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:50.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.934+0000 7f38bfa1b740 1 -- 172.21.15.86:0/4229941744 >> 172.21.15.86:0/4229941744 conn(0x55ca4a04dbf0 msgr2=0x55ca4a096770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:50.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.934+0000 7f38bfa1b740 1 -- 172.21.15.86:0/4229941744 shutdown_connections 2024-04-20T21:29:50.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.934+0000 7f38bfa1b740 1 -- 172.21.15.86:0/4229941744 wait complete. 2024-04-20T21:29:50.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.935+0000 7f38bfa1b740 1 -- start start 2024-04-20T21:29:50.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.935+0000 7f38bfa1b740 1 Processor -- start 2024-04-20T21:29:50.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.936+0000 7f38bfa1b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca4a04f4e0 0x55ca4a071670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:50.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.936+0000 7f38bfa1b740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ca4a099460 0x55ca4a05dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:50.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.936+0000 7f38bfa1b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca4a073880 0x55ca4a073d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:50.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.936+0000 7f38bfa1b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ca4a04f9f0 con 0x55ca4a04f4e0 2024-04-20T21:29:50.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.936+0000 7f38bfa1b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ca49f715e0 con 0x55ca4a073880 2024-04-20T21:29:50.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.936+0000 7f38bfa1b740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55ca4a04fcd0 con 0x55ca4a099460 2024-04-20T21:29:50.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.936+0000 7f38bfa1b740 10 client.0 _ll_register_callbacks cb 0x55ca49cc9ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:29:50.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.936+0000 7f38b5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca4a04f4e0 0x55ca4a071670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:50.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.936+0000 7f38b5e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca4a04f4e0 0x55ca4a071670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49232/0 (socket says 172.21.15.86:49232) 2024-04-20T21:29:50.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.936+0000 7f38b5e00640 1 -- 172.21.15.86:0/63742069 learned_addr learned my addr 172.21.15.86:0/63742069 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:50.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.937+0000 7f38b5e00640 1 -- 172.21.15.86:0/63742069 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ca4a099460 msgr2=0x55ca4a05dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:50.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.937+0000 7f38b5e00640 1 --2- 172.21.15.86:0/63742069 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ca4a099460 0x55ca4a05dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:50.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.937+0000 7f38b5e00640 1 -- 172.21.15.86:0/63742069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca4a073880 msgr2=0x55ca4a073d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:50.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.937+0000 7f38b6800640 1 --2- 172.21.15.86:0/63742069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca4a073880 0x55ca4a073d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:50.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.937+0000 7f38b5e00640 1 --2- 172.21.15.86:0/63742069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca4a073880 0x55ca4a073d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:50.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.937+0000 7f38b5e00640 1 -- 172.21.15.86:0/63742069 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca49d946b0 con 0x55ca4a04f4e0 2024-04-20T21:29:50.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.937+0000 7f38b5e00640 1 --2- 172.21.15.86:0/63742069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca4a04f4e0 0x55ca4a071670 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f38a005d050 tx=0x7f38a0000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a23dc42e81450d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:50.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.937+0000 7f389aa00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:29:50.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.937+0000 7f389aa00640 1 -- 172.21.15.86:0/63742069 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a0054a30 con 0x55ca4a04f4e0 2024-04-20T21:29:50.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.937+0000 7f389aa00640 1 -- 172.21.15.86:0/63742069 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7f38a005f050 con 0x55ca4a04f4e0 2024-04-20T21:29:50.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.937+0000 7f38bfa1b740 1 -- 172.21.15.86:0/63742069 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38a0055150 con 0x55ca4a04f4e0 2024-04-20T21:29:50.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.937+0000 7f389aa00640 1 -- 172.21.15.86:0/63742069 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f38a007e050 con 0x55ca4a04f4e0 2024-04-20T21:29:50.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.937+0000 7f38bfa1b740 10 client.9378 fetch_fsmap learned FSMap version 99 2024-04-20T21:29:50.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.937+0000 7f38bfa1b740 1 -- 172.21.15.86:0/63742069 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) -- 0x55ca4a05ee90 con 0x55ca4a04f4e0 2024-04-20T21:29:50.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.938+0000 7f389aa00640 1 -- 172.21.15.86:0/63742069 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38a0055150 con 0x55ca4a04f4e0 2024-04-20T21:29:50.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.938+0000 7f389aa00640 1 -- 172.21.15.86:0/63742069 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 99) ==== 3495+0+0 (secure 0 0 0) 0x7f38a0081020 con 0x55ca4a04f4e0 2024-04-20T21:29:50.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.938+0000 7f38bfa1b740 10 client.9378 fetch_fsmap finished waiting for FSMap version 99 2024-04-20T21:29:50.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.938+0000 7f38bfa1b740 10 client.9378 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-20T21:29:50.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.938+0000 7f38bfa1b740 20 client.9378 populate_metadata read hostname 'smithi086' 2024-04-20T21:29:50.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.938+0000 7f38bfa1b740 1 --2- 172.21.15.86:0/63742069 >> [v2:172.21.15.86:6835/289590924,v1:172.21.15.86:6837/289590924] conn(0x55ca4a076b90 0x55ca4a076f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:50.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.938+0000 7f38bfa1b740 4 client.9378 mds_command: new command op to 9231 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:29:50.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.938+0000 7f38bfa1b740 1 -- 172.21.15.86:0/63742069 --> [v2:172.21.15.86:6835/289590924,v1:172.21.15.86:6837/289590924] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca49fc53f0 con 0x55ca4a076b90 2024-04-20T21:29:50.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.938+0000 7f38b5400640 1 --2- 172.21.15.86:0/63742069 >> [v2:172.21.15.86:6835/289590924,v1:172.21.15.86:6837/289590924] conn(0x55ca4a076b90 0x55ca4a076f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:50.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.939+0000 7f389aa00640 10 client.9378 ms_handle_connect on v2:172.21.15.86:6835/289590924 2024-04-20T21:29:50.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.939+0000 7f38b5400640 1 --2- 172.21.15.86:0/63742069 >> [v2:172.21.15.86:6835/289590924,v1:172.21.15.86:6837/289590924] conn(0x55ca4a076b90 0x55ca4a076f70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51b9df99bd27caf2 server_cookie=64019308b6b388c8 in_seq=0 out_seq=0 2024-04-20T21:29:50.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.939+0000 7f389aa00640 1 -- 172.21.15.86:0/63742069 <== mds.0 v2:172.21.15.86:6835/289590924 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ca49fc53f0 con 0x55ca4a076b90 2024-04-20T21:29:50.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.940+0000 7f389aa00640 10 client.9378 handle_command_reply: tid=0 2024-04-20T21:29:50.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.940+0000 7f38bfa1b740 1 client.9378 shutdown 2024-04-20T21:29:50.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.940+0000 7f38bfa1b740 1 -- 172.21.15.86:0/63742069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca4a04f4e0 msgr2=0x55ca4a071670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:50.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.940+0000 7f38bfa1b740 1 --2- 172.21.15.86:0/63742069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca4a04f4e0 0x55ca4a071670 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f38a005d050 tx=0x7f38a0000ed0 comp rx=0 tx=0).stop 2024-04-20T21:29:50.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.940+0000 7f38bfa1b740 1 -- 172.21.15.86:0/63742069 shutdown_connections 2024-04-20T21:29:50.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.940+0000 7f38bfa1b740 1 --2- 172.21.15.86:0/63742069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca4a073880 0x55ca4a073d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:50.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.940+0000 7f38bfa1b740 1 --2- 172.21.15.86:0/63742069 >> [v2:172.21.15.86:6835/289590924,v1:172.21.15.86:6837/289590924] conn(0x55ca4a076b90 0x55ca4a076f70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:50.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.940+0000 7f38bfa1b740 1 --2- 172.21.15.86:0/63742069 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ca4a099460 0x55ca4a05dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:50.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.940+0000 7f38bfa1b740 1 --2- 172.21.15.86:0/63742069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca4a04f4e0 0x55ca4a071670 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:50.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.940+0000 7f389aa00640 0 client.9378 ms_handle_reset on v2:172.21.15.86:6835/289590924 2024-04-20T21:29:50.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.940+0000 7f38bfa1b740 1 -- 172.21.15.86:0/63742069 >> 172.21.15.86:0/63742069 conn(0x55ca4a04b930 msgr2=0x55ca4a096570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:50.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.940+0000 7f38bfa1b740 1 -- 172.21.15.86:0/63742069 shutdown_connections 2024-04-20T21:29:50.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.940+0000 7f38bfa1b740 1 -- 172.21.15.86:0/63742069 wait complete. 2024-04-20T21:29:50.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.940+0000 7f38bfa1b740 20 client.9378 trim_cache size 0 max 16384 2024-04-20T21:29:50.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.959+0000 7f38bfa1b740 1 Processor -- start 2024-04-20T21:29:50.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.959+0000 7f38bfa1b740 1 -- start start 2024-04-20T21:29:50.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.960+0000 7f38bfa1b740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ca4a04f4e0 0x55ca4a07f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:50.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.960+0000 7f38bfa1b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca4a073880 0x55ca4a05d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:50.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.960+0000 7f38bfa1b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca4a099460 0x55ca4a05db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:50.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.960+0000 7f38bfa1b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f38b81ffce0 con 0x55ca4a099460 2024-04-20T21:29:50.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.960+0000 7f38bfa1b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f38b80b4e50 con 0x55ca4a073880 2024-04-20T21:29:50.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.960+0000 7f38bfa1b740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f38b80b54a0 con 0x55ca4a04f4e0 2024-04-20T21:29:50.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.960+0000 7f38b6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca4a099460 0x55ca4a05db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:50.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.960+0000 7f38b6800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca4a099460 0x55ca4a05db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49236/0 (socket says 172.21.15.86:49236) 2024-04-20T21:29:50.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.960+0000 7f38b6800640 1 -- 172.21.15.86:0/2304061826 learned_addr learned my addr 172.21.15.86:0/2304061826 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:50.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.960+0000 7f38b5e00640 1 --2- 172.21.15.86:0/2304061826 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ca4a04f4e0 0x55ca4a07f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:50.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.960+0000 7f38b6800640 1 -- 172.21.15.86:0/2304061826 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ca4a04f4e0 msgr2=0x55ca4a07f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:50.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.960+0000 7f38b6800640 1 --2- 172.21.15.86:0/2304061826 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ca4a04f4e0 0x55ca4a07f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:50.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.960+0000 7f38b6800640 1 -- 172.21.15.86:0/2304061826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca4a073880 msgr2=0x55ca4a05d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:50.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.960+0000 7f38b6800640 1 --2- 172.21.15.86:0/2304061826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca4a073880 0x55ca4a05d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:50.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.960+0000 7f38b6800640 1 -- 172.21.15.86:0/2304061826 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca4a05f110 con 0x55ca4a099460 2024-04-20T21:29:50.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.961+0000 7f38b6800640 1 --2- 172.21.15.86:0/2304061826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca4a099460 0x55ca4a05db20 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f38b0009890 tx=0x7f38b0030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccd08c5fec211654 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:50.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.961+0000 7f38a5c00640 1 -- 172.21.15.86:0/2304061826 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b004a070 con 0x55ca4a099460 2024-04-20T21:29:50.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.961+0000 7f38a5c00640 1 -- 172.21.15.86:0/2304061826 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38b0037040 con 0x55ca4a099460 2024-04-20T21:29:50.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.961+0000 7f38a5c00640 1 -- 172.21.15.86:0/2304061826 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b00317c0 con 0x55ca4a099460 2024-04-20T21:29:50.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.962+0000 7f38bfa1b740 1 -- 172.21.15.86:0/2304061826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca4a099460 msgr2=0x55ca4a05db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:50.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.962+0000 7f38bfa1b740 1 --2- 172.21.15.86:0/2304061826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca4a099460 0x55ca4a05db20 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f38b0009890 tx=0x7f38b0030000 comp rx=0 tx=0).stop 2024-04-20T21:29:50.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.962+0000 7f38bfa1b740 1 -- 172.21.15.86:0/2304061826 shutdown_connections 2024-04-20T21:29:50.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.962+0000 7f38bfa1b740 1 --2- 172.21.15.86:0/2304061826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca4a099460 0x55ca4a05db20 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:50.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.962+0000 7f38bfa1b740 1 --2- 172.21.15.86:0/2304061826 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca4a073880 0x55ca4a05d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:50.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.962+0000 7f38bfa1b740 1 --2- 172.21.15.86:0/2304061826 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ca4a04f4e0 0x55ca4a07f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:50.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.962+0000 7f38bfa1b740 1 -- 172.21.15.86:0/2304061826 >> 172.21.15.86:0/2304061826 conn(0x55ca4a04b930 msgr2=0x55ca4a04bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:50.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.962+0000 7f38bfa1b740 1 -- 172.21.15.86:0/2304061826 shutdown_connections 2024-04-20T21:29:50.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.962+0000 7f38bfa1b740 1 -- 172.21.15.86:0/2304061826 wait complete. 2024-04-20T21:29:50.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.962+0000 7f38bfa1b740 1 -- start start 2024-04-20T21:29:50.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.963+0000 7f38bfa1b740 1 Processor -- start 2024-04-20T21:29:50.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.963+0000 7f38bfa1b740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ca4a04f4e0 0x55ca4a072990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:50.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.963+0000 7f38bfa1b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca4a073880 0x55ca4a094730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:50.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.964+0000 7f38bfa1b740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca4a099460 0x55ca4a094c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:50.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.964+0000 7f38bfa1b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ca4a04f9f0 con 0x55ca4a073880 2024-04-20T21:29:50.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.964+0000 7f38bfa1b740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ca49f715e0 con 0x55ca4a099460 2024-04-20T21:29:50.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.964+0000 7f38bfa1b740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55ca4a05f910 con 0x55ca4a04f4e0 2024-04-20T21:29:50.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.964+0000 7f38bfa1b740 10 client.0 _ll_register_callbacks cb 0x7f38a0070070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:29:50.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.964+0000 7f38b5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca4a073880 0x55ca4a094730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:50.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.964+0000 7f38b5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca4a073880 0x55ca4a094730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49246/0 (socket says 172.21.15.86:49246) 2024-04-20T21:29:50.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.964+0000 7f38b5400640 1 -- 172.21.15.86:0/268844818 learned_addr learned my addr 172.21.15.86:0/268844818 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:50.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.964+0000 7f38b5400640 1 -- 172.21.15.86:0/268844818 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ca4a04f4e0 msgr2=0x55ca4a072990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:50.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.964+0000 7f38b5400640 1 --2- 172.21.15.86:0/268844818 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ca4a04f4e0 0x55ca4a072990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:50.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.964+0000 7f38b5400640 1 -- 172.21.15.86:0/268844818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca4a099460 msgr2=0x55ca4a094c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:50.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.964+0000 7f38b5400640 1 --2- 172.21.15.86:0/268844818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca4a099460 0x55ca4a094c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:50.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.964+0000 7f38b5400640 1 -- 172.21.15.86:0/268844818 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca4a072ed0 con 0x55ca4a073880 2024-04-20T21:29:50.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.964+0000 7f38b5400640 1 --2- 172.21.15.86:0/268844818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca4a073880 0x55ca4a094730 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f38a8007300 tx=0x7f38a8018c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12e9330acc8a062 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:50.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.964+0000 7f389aa00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:29:50.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.965+0000 7f389aa00640 1 -- 172.21.15.86:0/268844818 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a80182d0 con 0x55ca4a073880 2024-04-20T21:29:50.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.965+0000 7f389aa00640 1 -- 172.21.15.86:0/268844818 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7f38a80148a0 con 0x55ca4a073880 2024-04-20T21:29:50.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.965+0000 7f38bfa1b740 1 -- 172.21.15.86:0/268844818 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38b0037040 con 0x55ca4a073880 2024-04-20T21:29:50.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.965+0000 7f389aa00640 1 -- 172.21.15.86:0/268844818 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f38a8034050 con 0x55ca4a073880 2024-04-20T21:29:50.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.965+0000 7f38bfa1b740 10 client.9384 fetch_fsmap learned FSMap version 99 2024-04-20T21:29:50.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.965+0000 7f38bfa1b740 1 -- 172.21.15.86:0/268844818 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) -- 0x55ca4a06fc70 con 0x55ca4a073880 2024-04-20T21:29:50.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.965+0000 7f389aa00640 1 -- 172.21.15.86:0/268844818 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38b0037040 con 0x55ca4a073880 2024-04-20T21:29:50.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.965+0000 7f389aa00640 1 -- 172.21.15.86:0/268844818 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 99) ==== 3495+0+0 (secure 0 0 0) 0x7f38a8037020 con 0x55ca4a073880 2024-04-20T21:29:50.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.965+0000 7f38bfa1b740 10 client.9384 fetch_fsmap finished waiting for FSMap version 99 2024-04-20T21:29:50.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.965+0000 7f38bfa1b740 10 client.9384 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-20T21:29:50.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.965+0000 7f38bfa1b740 20 client.9384 populate_metadata read hostname 'smithi086' 2024-04-20T21:29:50.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.965+0000 7f38bfa1b740 1 --2- 172.21.15.86:0/268844818 >> [v2:172.21.15.86:6835/289590924,v1:172.21.15.86:6837/289590924] conn(0x55ca4a0b1530 0x55ca4a11e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:50.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.965+0000 7f38bfa1b740 4 client.9384 mds_command: new command op to 9231 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T21:29:50.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.965+0000 7f38bfa1b740 1 -- 172.21.15.86:0/268844818 --> [v2:172.21.15.86:6835/289590924,v1:172.21.15.86:6837/289590924] -- command(tid 0: {"prefix": "status"}) -- 0x55ca49fbf260 con 0x55ca4a0b1530 2024-04-20T21:29:50.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.966+0000 7f38b5e00640 1 --2- 172.21.15.86:0/268844818 >> [v2:172.21.15.86:6835/289590924,v1:172.21.15.86:6837/289590924] conn(0x55ca4a0b1530 0x55ca4a11e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:50.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.966+0000 7f389aa00640 10 client.9384 ms_handle_connect on v2:172.21.15.86:6835/289590924 2024-04-20T21:29:50.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.966+0000 7f38b5e00640 1 --2- 172.21.15.86:0/268844818 >> [v2:172.21.15.86:6835/289590924,v1:172.21.15.86:6837/289590924] conn(0x55ca4a0b1530 0x55ca4a11e2f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5863f6f8b448303e server_cookie=dc8eb7e0c95cbcc4 in_seq=0 out_seq=0 2024-04-20T21:29:50.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.966+0000 7f389aa00640 1 -- 172.21.15.86:0/268844818 <== mds.0 v2:172.21.15.86:6835/289590924 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x55ca49fbf260 con 0x55ca4a0b1530 2024-04-20T21:29:50.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.966+0000 7f389aa00640 10 client.9384 handle_command_reply: tid=0 2024-04-20T21:29:50.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.966+0000 7f38bfa1b740 1 client.9384 shutdown 2024-04-20T21:29:50.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.966+0000 7f38bfa1b740 1 -- 172.21.15.86:0/268844818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca4a073880 msgr2=0x55ca4a094730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:50.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.966+0000 7f38bfa1b740 1 --2- 172.21.15.86:0/268844818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca4a073880 0x55ca4a094730 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f38a8007300 tx=0x7f38a8018c00 comp rx=0 tx=0).stop 2024-04-20T21:29:50.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.967+0000 7f38bfa1b740 1 -- 172.21.15.86:0/268844818 shutdown_connections 2024-04-20T21:29:50.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.967+0000 7f38bfa1b740 1 --2- 172.21.15.86:0/268844818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ca4a099460 0x55ca4a094c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:50.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.967+0000 7f38bfa1b740 1 --2- 172.21.15.86:0/268844818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ca4a073880 0x55ca4a094730 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:50.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.967+0000 7f38bfa1b740 1 --2- 172.21.15.86:0/268844818 >> [v2:172.21.15.86:6835/289590924,v1:172.21.15.86:6837/289590924] conn(0x55ca4a0b1530 0x55ca4a11e2f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:50.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.967+0000 7f38bfa1b740 1 --2- 172.21.15.86:0/268844818 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ca4a04f4e0 0x55ca4a072990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:50.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.967+0000 7f389aa00640 0 client.9384 ms_handle_reset on v2:172.21.15.86:6835/289590924 2024-04-20T21:29:50.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.967+0000 7f38bfa1b740 1 -- 172.21.15.86:0/268844818 >> 172.21.15.86:0/268844818 conn(0x55ca4a04b930 msgr2=0x55ca4a07f670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:50.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.967+0000 7f38bfa1b740 1 -- 172.21.15.86:0/268844818 shutdown_connections 2024-04-20T21:29:50.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.967+0000 7f38bfa1b740 1 -- 172.21.15.86:0/268844818 wait complete. 2024-04-20T21:29:50.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.967+0000 7f38bfa1b740 20 client.9384 trim_cache size 0 max 16384 2024-04-20T21:29:50.971 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:29:50.971 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "9c2dec1d-d7ae-4a83-a812-79922808e278", 2024-04-20T21:29:50.971 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-04-20T21:29:50.971 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 9231, 2024-04-20T21:29:50.971 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-20T21:29:50.971 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-20T21:29:50.971 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-20T21:29:50.971 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 6.6185682899999998, 2024-04-20T21:29:50.971 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 99, 2024-04-20T21:29:50.972 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 112, 2024-04-20T21:29:50.972 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 112, 2024-04-20T21:29:50.972 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 12.119820219999999 2024-04-20T21:29:50.972 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:29:50.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.967+0000 7f38bcc00640 1 -- 172.21.15.86:0/2386765319 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f389406c8f0 msgr2=0x7f389406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:50.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.967+0000 7f38bcc00640 1 --2- 172.21.15.86:0/2386765319 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f389406c8f0 0x7f389406edb0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f38b80c7ee0 tx=0x7f38a800b040 comp rx=0 tx=0).stop 2024-04-20T21:29:50.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.967+0000 7f38bcc00640 1 -- 172.21.15.86:0/2386765319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38b80b0c40 msgr2=0x7f38b80c7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:50.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.967+0000 7f38bcc00640 1 --2- 172.21.15.86:0/2386765319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38b80b0c40 0x7f38b80c7670 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f38a000c4d0 tx=0x7f38a000c990 comp rx=0 tx=0).stop 2024-04-20T21:29:50.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.968+0000 7f38bcc00640 1 -- 172.21.15.86:0/2386765319 shutdown_connections 2024-04-20T21:29:50.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.968+0000 7f38bcc00640 1 --2- 172.21.15.86:0/2386765319 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f389406c8f0 0x7f389406edb0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:50.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.968+0000 7f38bcc00640 1 --2- 172.21.15.86:0/2386765319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38b815c880 0x7f38b80b5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:50.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.968+0000 7f38bcc00640 1 --2- 172.21.15.86:0/2386765319 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f38b80b2870 0x7f38b80c5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:50.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.968+0000 7f38bcc00640 1 --2- 172.21.15.86:0/2386765319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38b80b0c40 0x7f38b80c7670 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:50.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.968+0000 7f38bcc00640 1 -- 172.21.15.86:0/2386765319 >> 172.21.15.86:0/2386765319 conn(0x7f38b809a9e0 msgr2=0x7f38b80d0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:50.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.968+0000 7f38bcc00640 1 -- 172.21.15.86:0/2386765319 shutdown_connections 2024-04-20T21:29:50.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:50.968+0000 7f38bcc00640 1 -- 172.21.15.86:0/2386765319 wait complete. 2024-04-20T21:29:50.982 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T21:29:51.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.167+0000 7f05c1c00640 1 Processor -- start 2024-04-20T21:29:51.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.167+0000 7f05c1c00640 1 -- start start 2024-04-20T21:29:51.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.168+0000 7f05c1c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f05bc1629e0 0x7f05bc0b3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:51.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.168+0000 7f05c1c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f05bc06f6e0 0x7f05bc0b3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:51.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.168+0000 7f05c1c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05bc06e640 con 0x7f05bc162dc0 2024-04-20T21:29:51.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.169+0000 7f05c1c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f05bc06e920 con 0x7f05bc06f6e0 2024-04-20T21:29:51.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.169+0000 7f05c1c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f05bc06ec00 con 0x7f05bc1629e0 2024-04-20T21:29:51.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.169+0000 7f05b3400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f05bc162dc0 0x7f05bc0ad0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:33894/0 (socket says 172.21.15.86:33894) 2024-04-20T21:29:51.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.169+0000 7f05b3400640 1 -- 172.21.15.86:0/2600398701 learned_addr learned my addr 172.21.15.86:0/2600398701 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:29:51.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.169+0000 7f05c1200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f05bc06f6e0 0x7f05bc0b3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:51.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.169+0000 7f05bbe00640 1 --2- 172.21.15.86:0/2600398701 >> v2:172.21.15.144:3300/0 conn(0x7f05bc1629e0 0x7f05bc0b3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:51.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.169+0000 7f05bb400640 1 -- 172.21.15.86:0/2600398701 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1460911219 0 0) 0x7f05bc06e640 con 0x7f05bc162dc0 2024-04-20T21:29:51.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.169+0000 7f05c1200640 1 -- 172.21.15.86:0/2600398701 >> v2:172.21.15.144:3300/0 conn(0x7f05bc1629e0 msgr2=0x7f05bc0b3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:51.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.169+0000 7f05c1200640 1 --2- 172.21.15.86:0/2600398701 >> v2:172.21.15.144:3300/0 conn(0x7f05bc1629e0 0x7f05bc0b3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:51.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.169+0000 7f05c1200640 1 -- 172.21.15.86:0/2600398701 >> v1:172.21.15.86:6789/0 conn(0x7f05bc162dc0 legacy=0x7f05bc0ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:51.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.169+0000 7f05c1200640 1 -- 172.21.15.86:0/2600398701 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05bc06eee0 con 0x7f05bc06f6e0 2024-04-20T21:29:51.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.170+0000 7f05c1200640 1 --2- 172.21.15.86:0/2600398701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05bc06f6e0 0x7f05bc0b3640 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f05bc073bc0 tx=0x7f05b402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfb1cc4684f08569 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:51.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.171+0000 7f05bb400640 1 -- 172.21.15.86:0/2600398701 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b4039070 con 0x7f05bc06f6e0 2024-04-20T21:29:51.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.171+0000 7f05bb400640 1 -- 172.21.15.86:0/2600398701 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05b402cc80 con 0x7f05bc06f6e0 2024-04-20T21:29:51.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.171+0000 7f05bb400640 1 -- 172.21.15.86:0/2600398701 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b4034660 con 0x7f05bc06f6e0 2024-04-20T21:29:51.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.171+0000 7f05c1c00640 1 -- 172.21.15.86:0/2600398701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05bc06f6e0 msgr2=0x7f05bc0b3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:51.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.171+0000 7f05c1c00640 1 --2- 172.21.15.86:0/2600398701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05bc06f6e0 0x7f05bc0b3640 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f05bc073bc0 tx=0x7f05b402bed0 comp rx=0 tx=0).stop 2024-04-20T21:29:51.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.171+0000 7f05c1c00640 1 -- 172.21.15.86:0/2600398701 shutdown_connections 2024-04-20T21:29:51.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.171+0000 7f05c1c00640 1 --2- 172.21.15.86:0/2600398701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05bc06f6e0 0x7f05bc0b3640 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:51.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.171+0000 7f05c1c00640 1 --2- 172.21.15.86:0/2600398701 >> v2:172.21.15.144:3300/0 conn(0x7f05bc1629e0 0x7f05bc0b3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:51.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.171+0000 7f05c1c00640 1 -- 172.21.15.86:0/2600398701 >> 172.21.15.86:0/2600398701 conn(0x7f05bc0d0170 msgr2=0x7f05bc0d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:51.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.172+0000 7f05c1c00640 1 -- 172.21.15.86:0/2600398701 shutdown_connections 2024-04-20T21:29:51.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.172+0000 7f05c1c00640 1 -- 172.21.15.86:0/2600398701 wait complete. 2024-04-20T21:29:51.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.173+0000 7f05c1c00640 1 Processor -- start 2024-04-20T21:29:51.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.173+0000 7f05c1c00640 1 -- start start 2024-04-20T21:29:51.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.174+0000 7f05c1c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05bc06f6e0 0x7f05bc155e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:51.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.174+0000 7f05c1c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05bc1629e0 0x7f05bc0ae290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:51.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.174+0000 7f05c1c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f05bc0ae7d0 0x7f05bc0ce010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:51.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.174+0000 7f05c1c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f05bc0ad970 con 0x7f05bc1629e0 2024-04-20T21:29:51.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.174+0000 7f05c1c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f05bc0ad7f0 con 0x7f05bc06f6e0 2024-04-20T21:29:51.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.174+0000 7f05c1c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f05bc0adaf0 con 0x7f05bc0ae7d0 2024-04-20T21:29:51.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.175+0000 7f05b3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05bc1629e0 0x7f05bc0ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:51.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.175+0000 7f05b3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05bc1629e0 0x7f05bc0ae290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49262/0 (socket says 172.21.15.86:49262) 2024-04-20T21:29:51.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.175+0000 7f05b3400640 1 -- 172.21.15.86:0/1662777423 learned_addr learned my addr 172.21.15.86:0/1662777423 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:51.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.175+0000 7f05bbe00640 1 --2- 172.21.15.86:0/1662777423 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05bc06f6e0 0x7f05bc155e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:51.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.175+0000 7f05c1200640 1 --2- 172.21.15.86:0/1662777423 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f05bc0ae7d0 0x7f05bc0ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:51.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.175+0000 7f05b3400640 1 -- 172.21.15.86:0/1662777423 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f05bc0ae7d0 msgr2=0x7f05bc0ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:51.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.175+0000 7f05b3400640 1 --2- 172.21.15.86:0/1662777423 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f05bc0ae7d0 0x7f05bc0ce010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:51.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.175+0000 7f05b3400640 1 -- 172.21.15.86:0/1662777423 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05bc06f6e0 msgr2=0x7f05bc155e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:51.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.175+0000 7f05b3400640 1 --2- 172.21.15.86:0/1662777423 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05bc06f6e0 0x7f05bc155e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:51.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.175+0000 7f05b3400640 1 -- 172.21.15.86:0/1662777423 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05bc1fbde0 con 0x7f05bc1629e0 2024-04-20T21:29:51.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.175+0000 7f05b3400640 1 --2- 172.21.15.86:0/1662777423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05bc1629e0 0x7f05bc0ae290 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f05a400c430 tx=0x7f05a400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:51.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.176+0000 7f05b8c00640 1 -- 172.21.15.86:0/1662777423 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a40074e0 con 0x7f05bc1629e0 2024-04-20T21:29:51.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.176+0000 7f05b8c00640 1 -- 172.21.15.86:0/1662777423 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05a40047e0 con 0x7f05bc1629e0 2024-04-20T21:29:51.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.176+0000 7f05b8c00640 1 -- 172.21.15.86:0/1662777423 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a4004bd0 con 0x7f05bc1629e0 2024-04-20T21:29:51.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.176+0000 7f05c1c00640 1 -- 172.21.15.86:0/1662777423 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05bc1fc0c0 con 0x7f05bc1629e0 2024-04-20T21:29:51.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.176+0000 7f05c1c00640 1 -- 172.21.15.86:0/1662777423 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05bc156380 con 0x7f05bc1629e0 2024-04-20T21:29:51.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.177+0000 7f05c1c00640 1 -- 172.21.15.86:0/1662777423 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05bc0aebe0 con 0x7f05bc1629e0 2024-04-20T21:29:51.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.178+0000 7f05b8c00640 1 -- 172.21.15.86:0/1662777423 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f05a4011070 con 0x7f05bc1629e0 2024-04-20T21:29:51.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.179+0000 7f05b8c00640 1 --2- 172.21.15.86:0/1662777423 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f059806c8f0 0x7f059806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:51.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.179+0000 7f05b8c00640 1 -- 172.21.15.86:0/1662777423 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7f05a408b840 con 0x7f05bc1629e0 2024-04-20T21:29:51.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.179+0000 7f05bbe00640 1 --2- 172.21.15.86:0/1662777423 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f059806c8f0 0x7f059806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:51.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.180+0000 7f05bbe00640 1 --2- 172.21.15.86:0/1662777423 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f059806c8f0 0x7f059806edb0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f05ac006fd0 tx=0x7f05ac008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:51.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.183+0000 7f05b8c00640 1 -- 172.21.15.86:0/1662777423 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f05bc0aebe0 con 0x7f05bc1629e0 2024-04-20T21:29:51.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.425+0000 7f05c1c00640 1 -- 172.21.15.86:0/1662777423 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f05bc04fcf0 con 0x7f05bc1629e0 2024-04-20T21:29:51.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.426+0000 7f05b8c00640 1 -- 172.21.15.86:0/1662777423 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v99) ==== 53+0+73 (secure 0 0 0) 0x7f05a4059c60 con 0x7f05bc1629e0 2024-04-20T21:29:51.426 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T21:29:51.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.429+0000 7f05c1c00640 1 -- 172.21.15.86:0/1662777423 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f059806c8f0 msgr2=0x7f059806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:51.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.429+0000 7f05c1c00640 1 --2- 172.21.15.86:0/1662777423 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f059806c8f0 0x7f059806edb0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f05ac006fd0 tx=0x7f05ac008040 comp rx=0 tx=0).stop 2024-04-20T21:29:51.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.429+0000 7f05c1c00640 1 -- 172.21.15.86:0/1662777423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05bc1629e0 msgr2=0x7f05bc0ae290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:51.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.429+0000 7f05c1c00640 1 --2- 172.21.15.86:0/1662777423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05bc1629e0 0x7f05bc0ae290 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f05a400c430 tx=0x7f05a400c8f0 comp rx=0 tx=0).stop 2024-04-20T21:29:51.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.429+0000 7f05c1c00640 1 -- 172.21.15.86:0/1662777423 shutdown_connections 2024-04-20T21:29:51.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.429+0000 7f05c1c00640 1 --2- 172.21.15.86:0/1662777423 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f059806c8f0 0x7f059806edb0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:51.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.429+0000 7f05c1c00640 1 --2- 172.21.15.86:0/1662777423 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f05bc0ae7d0 0x7f05bc0ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:51.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.429+0000 7f05c1c00640 1 --2- 172.21.15.86:0/1662777423 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05bc1629e0 0x7f05bc0ae290 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:51.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.429+0000 7f05c1c00640 1 --2- 172.21.15.86:0/1662777423 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05bc06f6e0 0x7f05bc155e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:51.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.429+0000 7f05c1c00640 1 -- 172.21.15.86:0/1662777423 >> 172.21.15.86:0/1662777423 conn(0x7f05bc0d0170 msgr2=0x7f05bc0b4790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:51.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.429+0000 7f05c1c00640 1 -- 172.21.15.86:0/1662777423 shutdown_connections 2024-04-20T21:29:51.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51.429+0000 7f05c1c00640 1 -- 172.21.15.86:0/1662777423 wait complete. 2024-04-20T21:29:51.447 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T21:29:51.447 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T21:29:51.447 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-04-20T21:29:51.448 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-20T21:29:51.448 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T21:29:51.448 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T21:29:51.448 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T21:29:51.448 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T21:29:51.448 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:29:51.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:29:51.448 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-04-20T21:29:51.498 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T21:29:51.498 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T21:29:51.498 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T21:29:51.499 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:29:51.499 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-20T21:29:51.499 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:29:51.499 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:29:51.499 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:29:51.499 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-04-20T21:29:51.499 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:29:51.499 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:29:51.499 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-04-20T21:29:51.499 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:29:51.499 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:29:51.499 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T21:29:51.499 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:29:51.499 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T21:29:51.500 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 41964sec preferred_lft 41964sec 2024-04-20T21:29:51.500 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-04-20T21:29:51.500 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:29:51.500 INFO:teuthology.orchestra.run.smithi086.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T21:29:51.500 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 86:25:97:2c:9c:bb brd ff:ff:ff:ff:ff:ff 2024-04-20T21:29:51.500 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-20T21:29:51.500 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:29:51.500 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::a4fb:30ff:fe90:2d6a/64 scope link 2024-04-20T21:29:51.500 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:29:51.500 INFO:teuthology.orchestra.run.smithi086.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T21:29:51.500 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 86:25:97:2c:9c:bb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:29:51.500 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::8425:97ff:fe2c:9cbb/64 scope link 2024-04-20T21:29:51.500 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:29:51.500 INFO:teuthology.orchestra.run.smithi086.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T21:29:51.500 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c2:9c:8b:55:ca:84 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:29:51.501 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c09c:8bff:fe55:ca84/64 scope link 2024-04-20T21:29:51.501 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:29:51.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:29:51.501 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T21:29:51.569 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-20T21:29:51.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:29:51.573 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-04-20T21:29:51.622 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-04-20T21:29:51.623 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T21:29:51.623 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T21:29:51.623 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T21:29:51.623 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T21:29:51.623 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:29:51.623 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-20T21:29:51.623 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T21:29:51.623 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T21:29:51.624 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T21:29:51.624 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-20T21:29:51.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:29:51.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:29:51.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:29:51.829 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-20T21:29:51.877 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-20T21:29:51.877 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-20T21:29:51.878 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:29:51.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:29:51.878 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-04-20T21:29:51.958 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-20T21:29:51.958 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T21:29:51.958 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-20T21:29:52.009 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:29:52.009 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T21:29:52.010 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-20T21:29:52.059 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-04-20T21:29:52.142 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T21:29:52.195 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-04-20T21:29:52.195 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-04-20T21:29:52.195 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-04-20T21:29:52.195 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-04-20T21:29:52.195 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-04-20T21:29:52.195 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-04-20T21:29:52.195 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-04-20T21:29:52.195 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-04-20T21:29:52.196 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-04-20T21:29:52.196 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-04-20T21:29:52.196 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-04-20T21:29:52.196 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/selinux 2024-04-20T21:29:52.196 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T21:29:52.196 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-04-20T21:29:52.196 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-04-20T21:29:52.196 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-04-20T21:29:52.196 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-04-20T21:29:52.196 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-04-20T21:29:52.196 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-04-20T21:29:52.196 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T21:29:52.196 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T21:29:52.196 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T21:29:52.197 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T21:29:52.197 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-04-20T21:29:52.197 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-04-20T21:29:52.197 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T21:29:52.197 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T21:29:52.197 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T21:29:52.197 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T21:29:52.197 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-04-20T21:29:52.197 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:29:52.197 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:29:52.197 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:29:52.197 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:29:52.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:29:52.198 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-04-20T21:29:52.247 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T21:29:52.247 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-20T21:29:52.252 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-04-20T21:29:52.318 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T21:29:52.364 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-04-20T21:29:52.364 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-04-20T21:29:52.364 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-04-20T21:29:52.364 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-04-20T21:29:52.365 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-04-20T21:29:52.365 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-04-20T21:29:52.365 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-04-20T21:29:52.365 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-04-20T21:29:52.365 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-04-20T21:29:52.366 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-04-20T21:29:52.366 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-04-20T21:29:52.366 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/selinux 2024-04-20T21:29:52.366 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T21:29:52.366 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-04-20T21:29:52.366 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-04-20T21:29:52.366 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-04-20T21:29:52.367 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-04-20T21:29:52.367 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-04-20T21:29:52.367 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-04-20T21:29:52.367 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T21:29:52.367 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T21:29:52.367 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T21:29:52.367 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T21:29:52.368 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-04-20T21:29:52.368 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-04-20T21:29:52.368 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T21:29:52.368 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T21:29:52.368 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T21:29:52.368 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T21:29:52.369 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-04-20T21:29:52.369 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:29:52.369 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:29:52.369 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:29:52.369 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:29:52.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:29:52.371 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-04-20T21:29:52.424 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:2024-04-20T21:29:52.425+0000 7f6bfb63e480 -1 init, newargv = 0x564067229540 newargc=15 2024-04-20T21:29:52.424 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[64761]: starting ceph client 2024-04-20T21:29:52.432 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[64761]: starting fuse 2024-04-20T21:29:52.456 INFO:teuthology.orchestra.run.smithi086.stdout:41 2024-04-20T21:29:52.456 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-04-20T21:29:52.457 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer -- python3 -c ' 2024-04-20T21:29:52.457 DEBUG:teuthology.orchestra.run.smithi086:> import glob 2024-04-20T21:29:52.457 DEBUG:teuthology.orchestra.run.smithi086:> import re 2024-04-20T21:29:52.457 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:29:52.457 DEBUG:teuthology.orchestra.run.smithi086:> import subprocess 2024-04-20T21:29:52.458 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:29:52.458 DEBUG:teuthology.orchestra.run.smithi086:> def _find_admin_socket(client_name): 2024-04-20T21:29:52.458 DEBUG:teuthology.orchestra.run.smithi086:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-20T21:29:52.458 DEBUG:teuthology.orchestra.run.smithi086:> files = glob.glob(asok_path) 2024-04-20T21:29:52.458 DEBUG:teuthology.orchestra.run.smithi086:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-20T21:29:52.458 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:29:52.459 DEBUG:teuthology.orchestra.run.smithi086:> # Given a non-glob path, it better be there 2024-04-20T21:29:52.459 DEBUG:teuthology.orchestra.run.smithi086:> if "*" not in asok_path: 2024-04-20T21:29:52.459 DEBUG:teuthology.orchestra.run.smithi086:> assert(len(files) == 1) 2024-04-20T21:29:52.459 DEBUG:teuthology.orchestra.run.smithi086:> return files[0] 2024-04-20T21:29:52.459 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:29:52.459 DEBUG:teuthology.orchestra.run.smithi086:> for f in files: 2024-04-20T21:29:52.460 DEBUG:teuthology.orchestra.run.smithi086:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T21:29:52.460 DEBUG:teuthology.orchestra.run.smithi086:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T21:29:52.460 DEBUG:teuthology.orchestra.run.smithi086:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T21:29:52.460 DEBUG:teuthology.orchestra.run.smithi086:> contents = proc_f.read() 2024-04-20T21:29:52.460 DEBUG:teuthology.orchestra.run.smithi086:> if mountpoint in contents: 2024-04-20T21:29:52.460 DEBUG:teuthology.orchestra.run.smithi086:> return f 2024-04-20T21:29:52.461 DEBUG:teuthology.orchestra.run.smithi086:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T21:29:52.461 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:29:52.461 DEBUG:teuthology.orchestra.run.smithi086:> print(_find_admin_socket("client.0")) 2024-04-20T21:29:52.461 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:29:52.648 INFO:teuthology.orchestra.run.smithi086.stdout:/var/run/ceph/ceph-client.0.64761.asok 2024-04-20T21:29:52.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:29:52.664 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.64761.asok 2024-04-20T21:29:52.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:29:52.665 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.64761.asok status 2024-04-20T21:29:52.894 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:29:52.895 INFO:teuthology.orchestra.run.smithi086.stdout: "metadata": { 2024-04-20T21:29:52.895 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T21:29:52.895 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T21:29:52.895 INFO:teuthology.orchestra.run.smithi086.stdout: "entity_id": "0", 2024-04-20T21:29:52.895 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": "smithi086", 2024-04-20T21:29:52.895 INFO:teuthology.orchestra.run.smithi086.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-20T21:29:52.895 INFO:teuthology.orchestra.run.smithi086.stdout: "pid": "64761", 2024-04-20T21:29:52.895 INFO:teuthology.orchestra.run.smithi086.stdout: "root": "/" 2024-04-20T21:29:52.895 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:29:52.895 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_count": 0, 2024-04-20T21:29:52.895 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_pinned_count": 0, 2024-04-20T21:29:52.895 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 9399, 2024-04-20T21:29:52.896 INFO:teuthology.orchestra.run.smithi086.stdout: "inst": { 2024-04-20T21:29:52.896 INFO:teuthology.orchestra.run.smithi086.stdout: "name": { 2024-04-20T21:29:52.896 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "client", 2024-04-20T21:29:52.896 INFO:teuthology.orchestra.run.smithi086.stdout: "num": 9399 2024-04-20T21:29:52.896 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:29:52.896 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2024-04-20T21:29:52.896 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-04-20T21:29:52.896 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2024-04-20T21:29:52.896 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 488537776 2024-04-20T21:29:52.896 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:29:52.896 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:29:52.896 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2024-04-20T21:29:52.896 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-04-20T21:29:52.896 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2024-04-20T21:29:52.896 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 488537776 2024-04-20T21:29:52.897 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:29:52.897 INFO:teuthology.orchestra.run.smithi086.stdout: "inst_str": "client.9399 192.168.0.1:0/488537776", 2024-04-20T21:29:52.897 INFO:teuthology.orchestra.run.smithi086.stdout: "addr_str": "192.168.0.1:0/488537776", 2024-04-20T21:29:52.897 INFO:teuthology.orchestra.run.smithi086.stdout: "inode_count": 1, 2024-04-20T21:29:52.897 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_epoch": 99, 2024-04-20T21:29:52.897 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch": 112, 2024-04-20T21:29:52.897 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch_barrier": 0, 2024-04-20T21:29:52.897 INFO:teuthology.orchestra.run.smithi086.stdout: "blocklisted": false, 2024-04-20T21:29:52.897 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs" 2024-04-20T21:29:52.897 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:29:52.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:29:52.910 DEBUG:teuthology.orchestra.run.smithi086:> stat --file-system '--printf=%T 2024-04-20T21:29:52.910 DEBUG:teuthology.orchestra.run.smithi086:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-20T21:29:52.963 INFO:teuthology.orchestra.run.smithi086.stdout:fuseblk 2024-04-20T21:29:52.963 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-20T21:29:52.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:29:52.964 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-20T21:29:53.042 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-04-20T21:29:53.042 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:29:53.042 DEBUG:teuthology.orchestra.run.smithi086:> mount_point = "/home/ubuntu/cephtest/mnt.0" 2024-04-20T21:29:53.042 DEBUG:teuthology.orchestra.run.smithi086:> parent = os.path.join(mount_point, "parent") 2024-04-20T21:29:53.042 DEBUG:teuthology.orchestra.run.smithi086:> os.mkdir(parent) 2024-04-20T21:29:53.042 DEBUG:teuthology.orchestra.run.smithi086:> flushed = os.path.join(parent, "flushed") 2024-04-20T21:29:53.042 DEBUG:teuthology.orchestra.run.smithi086:> os.mkdir(flushed) 2024-04-20T21:29:53.042 DEBUG:teuthology.orchestra.run.smithi086:> for f in ["alpha", "bravo", "charlie"]: 2024-04-20T21:29:53.042 DEBUG:teuthology.orchestra.run.smithi086:> open(os.path.join(flushed, f), '"'"'w'"'"').write(f) 2024-04-20T21:29:53.042 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:29:53.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:29:53.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:29:53.232 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-20T21:29:53.350 INFO:teuthology.orchestra.run.smithi086.stdout:./ 2024-04-20T21:29:53.350 INFO:teuthology.orchestra.run.smithi086.stdout:./parent 2024-04-20T21:29:53.350 INFO:teuthology.orchestra.run.smithi086.stdout:./parent/flushed 2024-04-20T21:29:53.350 INFO:teuthology.orchestra.run.smithi086.stdout:./parent/flushed/charlie 2024-04-20T21:29:53.350 INFO:teuthology.orchestra.run.smithi086.stdout:./parent/flushed/alpha 2024-04-20T21:29:53.350 INFO:teuthology.orchestra.run.smithi086.stdout:./parent/flushed/bravo 2024-04-20T21:29:53.352 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-04-20T21:29:53.352 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:29:53.352 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-04-20T21:29:53.352 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:29:53.352 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-20T21:29:53.352 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:29:53.483 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-20T21:29:53.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:29:53.496 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-04-20T21:29:53.496 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:29:53.496 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-04-20T21:29:53.496 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:29:53.496 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-04-20T21:29:53.497 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:29:53.683 INFO:teuthology.orchestra.run.smithi086.stdout:1099511627776 2024-04-20T21:29:53.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:29:53.698 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-04-20T21:29:53.698 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:29:53.698 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-04-20T21:29:53.698 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:29:53.698 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-04-20T21:29:53.698 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:29:53.884 INFO:teuthology.orchestra.run.smithi086.stdout:1099511627777 2024-04-20T21:29:53.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:29:53.897 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-04-20T21:29:53.897 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:29:53.897 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-04-20T21:29:53.897 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:29:53.898 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-04-20T21:29:53.898 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:29:54.083 INFO:teuthology.orchestra.run.smithi086.stdout:1099511628280 2024-04-20T21:29:54.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:29:54.097 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-04-20T21:29:54.097 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:29:54.097 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-04-20T21:29:54.097 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:29:54.097 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-04-20T21:29:54.098 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:29:54.282 INFO:teuthology.orchestra.run.smithi086.stdout:1099511627778 2024-04-20T21:29:54.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:29:54.296 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-04-20T21:29:54.296 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:29:54.296 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-04-20T21:29:54.296 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:29:54.296 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-04-20T21:29:54.296 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:29:54.482 INFO:teuthology.orchestra.run.smithi086.stdout:1099511628279 2024-04-20T21:29:54.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:29:54.495 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:29:54.495 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:29:54.593 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:29:54.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.795+0000 7f92eb200640 1 Processor -- start 2024-04-20T21:29:54.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.795+0000 7f92eb200640 1 -- start start 2024-04-20T21:29:54.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.796+0000 7f92eb200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f92ec06f2a0 0x7f92ec167410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:54.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.796+0000 7f92eb200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92ec160890 con 0x7f92ec15d6a0 2024-04-20T21:29:54.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.796+0000 7f92eb200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92ec160b70 con 0x7f92ec156ae0 2024-04-20T21:29:54.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.796+0000 7f92eb200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f92ec06db20 con 0x7f92ec06f2a0 2024-04-20T21:29:54.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.797+0000 7f92e1400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f92ec15d6a0 0x7f92ec0ad0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:33898/0 (socket says 172.21.15.86:33898) 2024-04-20T21:29:54.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.797+0000 7f92e1400640 1 -- 172.21.15.86:0/2717943905 learned_addr learned my addr 172.21.15.86:0/2717943905 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:29:54.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.797+0000 7f92ea800640 1 --2- 172.21.15.86:0/2717943905 >> v2:172.21.15.144:3300/0 conn(0x7f92ec06f2a0 0x7f92ec167410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:54.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.797+0000 7f92e9400640 1 -- 172.21.15.86:0/2717943905 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1995641884 0 0) 0x7f92ec160890 con 0x7f92ec15d6a0 2024-04-20T21:29:54.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.797+0000 7f92e9400640 1 -- 172.21.15.86:0/2717943905 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f92d0003520 con 0x7f92ec15d6a0 2024-04-20T21:29:54.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.798+0000 7f92e9400640 1 -- 172.21.15.86:0/2717943905 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2108122042 0 0) 0x7f92d0003520 con 0x7f92ec15d6a0 2024-04-20T21:29:54.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.798+0000 7f92e9400640 1 -- 172.21.15.86:0/2717943905 >> v2:172.21.15.144:3300/0 conn(0x7f92ec06f2a0 msgr2=0x7f92ec167410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:54.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.798+0000 7f92e9400640 1 --2- 172.21.15.86:0/2717943905 >> v2:172.21.15.144:3300/0 conn(0x7f92ec06f2a0 0x7f92ec167410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:54.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.798+0000 7f92e9400640 1 -- 172.21.15.86:0/2717943905 >> v1:172.21.15.144:6789/0 conn(0x7f92ec156ae0 legacy=0x7f92ec160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:54.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.798+0000 7f92e9400640 1 -- 172.21.15.86:0/2717943905 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92ec160e50 con 0x7f92ec15d6a0 2024-04-20T21:29:54.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.798+0000 7f92e9400640 1 -- 172.21.15.86:0/2717943905 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f92d4001670 con 0x7f92ec15d6a0 2024-04-20T21:29:54.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.798+0000 7f92e9400640 1 -- 172.21.15.86:0/2717943905 >> v1:172.21.15.86:6789/0 conn(0x7f92ec15d6a0 legacy=0x7f92ec0ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:54.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.798+0000 7f92e9400640 1 --2- 172.21.15.86:0/2717943905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92d00039b0 0x7f92d0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:54.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.798+0000 7f92e9400640 1 --2- 172.21.15.86:0/2717943905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92d0004800 0x7f92d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:54.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.798+0000 7f92e9400640 1 --2- 172.21.15.86:0/2717943905 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92d00051c0 0x7f92d000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:54.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.798+0000 7f92e9400640 1 -- 172.21.15.86:0/2717943905 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f92ec06db20 con 0x7f92d0004800 2024-04-20T21:29:54.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.798+0000 7f92e9400640 1 -- 172.21.15.86:0/2717943905 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f92d000c060 con 0x7f92d00039b0 2024-04-20T21:29:54.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.798+0000 7f92e9400640 1 -- 172.21.15.86:0/2717943905 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f92d000c260 con 0x7f92d00051c0 2024-04-20T21:29:54.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.798+0000 7f92e1400640 1 --2- 172.21.15.86:0/2717943905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92d0004800 0x7f92d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:54.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.798+0000 7f92e1400640 1 -- 172.21.15.86:0/2717943905 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92d00051c0 msgr2=0x7f92d000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:54.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.798+0000 7f92e1400640 1 --2- 172.21.15.86:0/2717943905 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92d00051c0 0x7f92d000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:54.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.798+0000 7f92e1400640 1 -- 172.21.15.86:0/2717943905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92d00039b0 msgr2=0x7f92d0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:54.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.798+0000 7f92e9e00640 1 --2- 172.21.15.86:0/2717943905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92d00039b0 0x7f92d0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:54.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.798+0000 7f92e1400640 1 --2- 172.21.15.86:0/2717943905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92d00039b0 0x7f92d0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:54.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.798+0000 7f92e1400640 1 -- 172.21.15.86:0/2717943905 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92d0003520 con 0x7f92d0004800 2024-04-20T21:29:54.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.798+0000 7f92ea800640 1 -- 172.21.15.86:0/2717943905 reap_dead start 2024-04-20T21:29:54.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.798+0000 7f92e9e00640 1 --2- 172.21.15.86:0/2717943905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92d00039b0 0x7f92d0003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:29:54.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.799+0000 7f92e1400640 1 --2- 172.21.15.86:0/2717943905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92d0004800 0x7f92d0004c80 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f92d400c210 tx=0x7f92d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=233cd7a3d1efa734 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:54.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.800+0000 7f92e9400640 1 -- 172.21.15.86:0/2717943905 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d4007d30 con 0x7f92d0004800 2024-04-20T21:29:54.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.800+0000 7f92e9400640 1 -- 172.21.15.86:0/2717943905 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92d4030070 con 0x7f92d0004800 2024-04-20T21:29:54.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.800+0000 7f92e9400640 1 -- 172.21.15.86:0/2717943905 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d4005d70 con 0x7f92d0004800 2024-04-20T21:29:54.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.800+0000 7f92eb200640 1 -- 172.21.15.86:0/2717943905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92d0004800 msgr2=0x7f92d0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:54.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.800+0000 7f92eb200640 1 --2- 172.21.15.86:0/2717943905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92d0004800 0x7f92d0004c80 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f92d400c210 tx=0x7f92d0005830 comp rx=0 tx=0).stop 2024-04-20T21:29:54.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.800+0000 7f92eb200640 1 -- 172.21.15.86:0/2717943905 shutdown_connections 2024-04-20T21:29:54.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.800+0000 7f92eb200640 1 --2- 172.21.15.86:0/2717943905 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92d0004800 0x7f92d0004c80 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:54.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.800+0000 7f92eb200640 1 -- 172.21.15.86:0/2717943905 >> 172.21.15.86:0/2717943905 conn(0x7f92ec0d0170 msgr2=0x7f92ec0d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:54.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.801+0000 7f92eb200640 1 -- 172.21.15.86:0/2717943905 shutdown_connections 2024-04-20T21:29:54.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.801+0000 7f92eb200640 1 -- 172.21.15.86:0/2717943905 wait complete. 2024-04-20T21:29:54.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.802+0000 7f92eb200640 1 Processor -- start 2024-04-20T21:29:54.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.802+0000 7f92eb200640 1 -- start start 2024-04-20T21:29:54.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.803+0000 7f92eb200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92d0004800 0x7f92ec153c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:54.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.803+0000 7f92eb200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92ec143790 0x7f92ec099800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:54.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.803+0000 7f92eb200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92ec157040 0x7f92ec144f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:54.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.803+0000 7f92eb200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f92ec160080 con 0x7f92ec143790 2024-04-20T21:29:54.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.803+0000 7f92eb200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f92ec156ec0 con 0x7f92ec157040 2024-04-20T21:29:54.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.803+0000 7f92eb200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f92ec160200 con 0x7f92d0004800 2024-04-20T21:29:54.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.804+0000 7f92e1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92ec143790 0x7f92ec099800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:54.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.804+0000 7f92e1400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92ec143790 0x7f92ec099800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49286/0 (socket says 172.21.15.86:49286) 2024-04-20T21:29:54.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.804+0000 7f92e1400640 1 -- 172.21.15.86:0/3771656734 learned_addr learned my addr 172.21.15.86:0/3771656734 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:54.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.804+0000 7f92e9e00640 1 --2- 172.21.15.86:0/3771656734 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92d0004800 0x7f92ec153c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:54.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.804+0000 7f92e1400640 1 -- 172.21.15.86:0/3771656734 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92d0004800 msgr2=0x7f92ec153c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:54.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.804+0000 7f92e1400640 1 --2- 172.21.15.86:0/3771656734 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92d0004800 0x7f92ec153c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:54.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.804+0000 7f92e1400640 1 -- 172.21.15.86:0/3771656734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92ec157040 msgr2=0x7f92ec144f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:54.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.804+0000 7f92ea800640 1 --2- 172.21.15.86:0/3771656734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92ec157040 0x7f92ec144f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:54.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.804+0000 7f92e1400640 1 --2- 172.21.15.86:0/3771656734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92ec157040 0x7f92ec144f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:54.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.804+0000 7f92e1400640 1 -- 172.21.15.86:0/3771656734 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92ec1539d0 con 0x7f92ec143790 2024-04-20T21:29:54.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.804+0000 7f92e1400640 1 --2- 172.21.15.86:0/3771656734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92ec143790 0x7f92ec099800 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f92d400c210 tx=0x7f92d402e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:54.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.805+0000 7f92e2a00640 1 -- 172.21.15.86:0/3771656734 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d4046070 con 0x7f92ec143790 2024-04-20T21:29:54.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.805+0000 7f92e2a00640 1 -- 172.21.15.86:0/3771656734 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92d4004bf0 con 0x7f92ec143790 2024-04-20T21:29:54.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.805+0000 7f92e2a00640 1 -- 172.21.15.86:0/3771656734 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d4007d10 con 0x7f92ec143790 2024-04-20T21:29:54.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.805+0000 7f92eb200640 1 -- 172.21.15.86:0/3771656734 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92ec072320 con 0x7f92ec143790 2024-04-20T21:29:54.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.805+0000 7f92eb200640 1 -- 172.21.15.86:0/3771656734 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92ec0afe70 con 0x7f92ec143790 2024-04-20T21:29:54.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.806+0000 7f92eb200640 1 -- 172.21.15.86:0/3771656734 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92ec04fcf0 con 0x7f92ec143790 2024-04-20T21:29:54.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.807+0000 7f92e2a00640 1 -- 172.21.15.86:0/3771656734 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f92d4007070 con 0x7f92ec143790 2024-04-20T21:29:54.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.808+0000 7f92e2a00640 1 --2- 172.21.15.86:0/3771656734 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f92c806c8f0 0x7f92c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:54.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.808+0000 7f92e2a00640 1 -- 172.21.15.86:0/3771656734 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7f92d40acff0 con 0x7f92ec143790 2024-04-20T21:29:54.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.808+0000 7f92e9e00640 1 --2- 172.21.15.86:0/3771656734 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f92c806c8f0 0x7f92c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:54.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.809+0000 7f92e9e00640 1 --2- 172.21.15.86:0/3771656734 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f92c806c8f0 0x7f92c806edb0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f92ec0706a0 tx=0x7f92dc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:54.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:54.813+0000 7f92e2a00640 1 -- 172.21.15.86:0/3771656734 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f92d407b350 con 0x7f92ec143790 2024-04-20T21:29:55.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.015+0000 7f92eb200640 1 -- 172.21.15.86:0/3771656734 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f92ec0cf600 con 0x7f92ec143790 2024-04-20T21:29:55.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.018+0000 7f92e2a00640 1 -- 172.21.15.86:0/3771656734 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) ==== 74+0+16627 (secure 0 0 0) 0x7f92d407ac50 con 0x7f92ec143790 2024-04-20T21:29:55.025 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:29:55.026 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":112,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:29:45.322441+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:29:41.532824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-20T21:29:42.749687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:29:55.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.021+0000 7f92eb200640 1 -- 172.21.15.86:0/3771656734 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f92c806c8f0 msgr2=0x7f92c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:55.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.021+0000 7f92eb200640 1 --2- 172.21.15.86:0/3771656734 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f92c806c8f0 0x7f92c806edb0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f92ec0706a0 tx=0x7f92dc009040 comp rx=0 tx=0).stop 2024-04-20T21:29:55.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.021+0000 7f92eb200640 1 -- 172.21.15.86:0/3771656734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92ec143790 msgr2=0x7f92ec099800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:55.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.021+0000 7f92eb200640 1 --2- 172.21.15.86:0/3771656734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92ec143790 0x7f92ec099800 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f92d400c210 tx=0x7f92d402e070 comp rx=0 tx=0).stop 2024-04-20T21:29:55.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.021+0000 7f92eb200640 1 -- 172.21.15.86:0/3771656734 shutdown_connections 2024-04-20T21:29:55.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.021+0000 7f92eb200640 1 --2- 172.21.15.86:0/3771656734 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f92c806c8f0 0x7f92c806edb0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:55.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.021+0000 7f92eb200640 1 --2- 172.21.15.86:0/3771656734 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92ec157040 0x7f92ec144f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:55.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.021+0000 7f92eb200640 1 --2- 172.21.15.86:0/3771656734 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92ec143790 0x7f92ec099800 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:55.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.021+0000 7f92eb200640 1 --2- 172.21.15.86:0/3771656734 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f92d0004800 0x7f92ec153c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:55.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.021+0000 7f92eb200640 1 -- 172.21.15.86:0/3771656734 >> 172.21.15.86:0/3771656734 conn(0x7f92ec0d0170 msgr2=0x7f92ec0af0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:55.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.021+0000 7f92eb200640 1 -- 172.21.15.86:0/3771656734 shutdown_connections 2024-04-20T21:29:55.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.021+0000 7f92eb200640 1 -- 172.21.15.86:0/3771656734 wait complete. 2024-04-20T21:29:55.037 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:29:41.532824+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:29:42.749687+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:29:55.038 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:29:55.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.238+0000 7f2c09400640 1 Processor -- start 2024-04-20T21:29:55.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.239+0000 7f2c09400640 1 -- start start 2024-04-20T21:29:55.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.240+0000 7f2c09400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c04155d30 0x7f2c040cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:55.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.240+0000 7f2c09400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2c0406b980 0x7f2c0406be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:55.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.240+0000 7f2c09400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c04156110 con 0x7f2c040b4ba0 2024-04-20T21:29:55.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.240+0000 7f2c09400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f2c040b56e0 con 0x7f2c0406b980 2024-04-20T21:29:55.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.240+0000 7f2c09400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2c040ce460 con 0x7f2c04155d30 2024-04-20T21:29:55.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.241+0000 7f2c01e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2c0406b980 0x7f2c0406be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:55.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.241+0000 7f2c01e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f2c0406b980 0x7f2c0406be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49290/0 (socket says 172.21.15.86:49290) 2024-04-20T21:29:55.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.241+0000 7f2c01e00640 1 -- 172.21.15.86:0/4272631816 learned_addr learned my addr 172.21.15.86:0/4272631816 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:55.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.241+0000 7f2c02800640 1 --2- 172.21.15.86:0/4272631816 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c04155d30 0x7f2c040cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:55.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.241+0000 7f2c01e00640 1 -- 172.21.15.86:0/4272631816 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c04155d30 msgr2=0x7f2c040cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:55.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.241+0000 7f2c01e00640 1 --2- 172.21.15.86:0/4272631816 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c04155d30 0x7f2c040cf500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:55.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.241+0000 7f2c01e00640 1 -- 172.21.15.86:0/4272631816 >> v1:172.21.15.144:6789/0 conn(0x7f2c040b4ba0 legacy=0x7f2c040b4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:55.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.241+0000 7f2c01e00640 1 -- 172.21.15.86:0/4272631816 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c040ce740 con 0x7f2c0406b980 2024-04-20T21:29:55.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.242+0000 7f2c01e00640 1 --2- 172.21.15.86:0/4272631816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c0406b980 0x7f2c0406be00 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0009870 tx=0x7f2bf002d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ce66b3243784095 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:55.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.243+0000 7f2c01400640 1 -- 172.21.15.86:0/4272631816 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf0031810 con 0x7f2c0406b980 2024-04-20T21:29:55.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.243+0000 7f2c01400640 1 -- 172.21.15.86:0/4272631816 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bf00319b0 con 0x7f2c0406b980 2024-04-20T21:29:55.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.243+0000 7f2c01400640 1 -- 172.21.15.86:0/4272631816 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf0031d80 con 0x7f2c0406b980 2024-04-20T21:29:55.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.243+0000 7f2c09400640 1 -- 172.21.15.86:0/4272631816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c0406b980 msgr2=0x7f2c0406be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:55.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.243+0000 7f2c09400640 1 --2- 172.21.15.86:0/4272631816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c0406b980 0x7f2c0406be00 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0009870 tx=0x7f2bf002d9f0 comp rx=0 tx=0).stop 2024-04-20T21:29:55.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.243+0000 7f2c09400640 1 -- 172.21.15.86:0/4272631816 shutdown_connections 2024-04-20T21:29:55.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.243+0000 7f2c09400640 1 --2- 172.21.15.86:0/4272631816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c0406b980 0x7f2c0406be00 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:55.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.243+0000 7f2c09400640 1 --2- 172.21.15.86:0/4272631816 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c04155d30 0x7f2c040cf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:55.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.243+0000 7f2c09400640 1 -- 172.21.15.86:0/4272631816 >> 172.21.15.86:0/4272631816 conn(0x7f2c040b2110 msgr2=0x7f2c040b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:55.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.243+0000 7f2c09400640 1 -- 172.21.15.86:0/4272631816 shutdown_connections 2024-04-20T21:29:55.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.244+0000 7f2c09400640 1 -- 172.21.15.86:0/4272631816 wait complete. 2024-04-20T21:29:55.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.245+0000 7f2c09400640 1 Processor -- start 2024-04-20T21:29:55.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.245+0000 7f2c09400640 1 -- start start 2024-04-20T21:29:55.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.246+0000 7f2c09400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c0406b980 0x7f2c0414ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:55.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.246+0000 7f2c09400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c040b4ba0 0x7f2c0415a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:55.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.246+0000 7f2c09400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c04155d30 0x7f2c0415ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:55.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.246+0000 7f2c09400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2c040cefe0 con 0x7f2c040b4ba0 2024-04-20T21:29:55.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.246+0000 7f2c09400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f2c040ced00 con 0x7f2c0406b980 2024-04-20T21:29:55.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.246+0000 7f2c09400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f2c0406ddf0 con 0x7f2c04155d30 2024-04-20T21:29:55.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.247+0000 7f2c01e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c040b4ba0 0x7f2c0415a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:55.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.247+0000 7f2c01e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c040b4ba0 0x7f2c0415a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49294/0 (socket says 172.21.15.86:49294) 2024-04-20T21:29:55.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.247+0000 7f2c01e00640 1 -- 172.21.15.86:0/1465902317 learned_addr learned my addr 172.21.15.86:0/1465902317 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:55.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.247+0000 7f2c02800640 1 --2- 172.21.15.86:0/1465902317 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c0406b980 0x7f2c0414ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:55.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.247+0000 7f2c03200640 1 --2- 172.21.15.86:0/1465902317 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c04155d30 0x7f2c0415ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:55.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.247+0000 7f2c01e00640 1 -- 172.21.15.86:0/1465902317 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c04155d30 msgr2=0x7f2c0415ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:55.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.247+0000 7f2c01e00640 1 --2- 172.21.15.86:0/1465902317 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c04155d30 0x7f2c0415ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:55.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.247+0000 7f2c01e00640 1 -- 172.21.15.86:0/1465902317 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c0406b980 msgr2=0x7f2c0414ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:55.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.247+0000 7f2c01e00640 1 --2- 172.21.15.86:0/1465902317 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c0406b980 0x7f2c0414ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:55.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.247+0000 7f2c01e00640 1 -- 172.21.15.86:0/1465902317 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c0414f530 con 0x7f2c040b4ba0 2024-04-20T21:29:55.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.247+0000 7f2c02800640 1 --2- 172.21.15.86:0/1465902317 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c0406b980 0x7f2c0414ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:29:55.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.247+0000 7f2c03200640 1 --2- 172.21.15.86:0/1465902317 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c04155d30 0x7f2c0415ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:29:55.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.247+0000 7f2c01e00640 1 --2- 172.21.15.86:0/1465902317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c040b4ba0 0x7f2c0415a730 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0009870 tx=0x7f2bf00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:55.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.248+0000 7f2bfaa00640 1 -- 172.21.15.86:0/1465902317 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf0039810 con 0x7f2c040b4ba0 2024-04-20T21:29:55.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.248+0000 7f2bfaa00640 1 -- 172.21.15.86:0/1465902317 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bf00399b0 con 0x7f2c040b4ba0 2024-04-20T21:29:55.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.248+0000 7f2bfaa00640 1 -- 172.21.15.86:0/1465902317 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf0039da0 con 0x7f2c040b4ba0 2024-04-20T21:29:55.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.248+0000 7f2c09400640 1 -- 172.21.15.86:0/1465902317 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c0414da80 con 0x7f2c040b4ba0 2024-04-20T21:29:55.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.248+0000 7f2c09400640 1 -- 172.21.15.86:0/1465902317 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c0414f100 con 0x7f2c040b4ba0 2024-04-20T21:29:55.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.249+0000 7f2c09400640 1 -- 172.21.15.86:0/1465902317 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c040689e0 con 0x7f2c040b4ba0 2024-04-20T21:29:55.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.250+0000 7f2bfaa00640 1 -- 172.21.15.86:0/1465902317 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2bf004a060 con 0x7f2c040b4ba0 2024-04-20T21:29:55.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.251+0000 7f2bfaa00640 1 --2- 172.21.15.86:0/1465902317 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2be406c940 0x7f2be406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:55.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.251+0000 7f2bfaa00640 1 -- 172.21.15.86:0/1465902317 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7f2bf00ad470 con 0x7f2c040b4ba0 2024-04-20T21:29:55.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.251+0000 7f2c02800640 1 --2- 172.21.15.86:0/1465902317 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2be406c940 0x7f2be406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:55.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.252+0000 7f2c02800640 1 --2- 172.21.15.86:0/1465902317 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2be406c940 0x7f2be406ee00 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4009f50 tx=0x7f2bf400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:55.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.256+0000 7f2bfaa00640 1 -- 172.21.15.86:0/1465902317 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2bf003e070 con 0x7f2c040b4ba0 2024-04-20T21:29:55.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.455+0000 7f2c09400640 1 -- 172.21.15.86:0/1465902317 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2c040721c0 con 0x7f2c040b4ba0 2024-04-20T21:29:55.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.456+0000 7f2bfaa00640 1 -- 172.21.15.86:0/1465902317 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) ==== 93+0+10 (secure 0 0 0) 0x7f2bf0035030 con 0x7f2c040b4ba0 2024-04-20T21:29:55.455 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:29:55.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.459+0000 7f2c09400640 1 -- 172.21.15.86:0/1465902317 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2be406c940 msgr2=0x7f2be406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:55.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.459+0000 7f2c09400640 1 --2- 172.21.15.86:0/1465902317 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2be406c940 0x7f2be406ee00 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4009f50 tx=0x7f2bf400b040 comp rx=0 tx=0).stop 2024-04-20T21:29:55.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.459+0000 7f2c09400640 1 -- 172.21.15.86:0/1465902317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c040b4ba0 msgr2=0x7f2c0415a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:55.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.459+0000 7f2c09400640 1 --2- 172.21.15.86:0/1465902317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c040b4ba0 0x7f2c0415a730 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0009870 tx=0x7f2bf00046c0 comp rx=0 tx=0).stop 2024-04-20T21:29:55.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.459+0000 7f2c09400640 1 -- 172.21.15.86:0/1465902317 shutdown_connections 2024-04-20T21:29:55.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.459+0000 7f2c09400640 1 --2- 172.21.15.86:0/1465902317 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f2be406c940 0x7f2be406ee00 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:55.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.459+0000 7f2c09400640 1 --2- 172.21.15.86:0/1465902317 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2c04155d30 0x7f2c0415ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:55.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.459+0000 7f2c09400640 1 --2- 172.21.15.86:0/1465902317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c040b4ba0 0x7f2c0415a730 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:55.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.459+0000 7f2c09400640 1 --2- 172.21.15.86:0/1465902317 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c0406b980 0x7f2c0414ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:55.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.459+0000 7f2c09400640 1 -- 172.21.15.86:0/1465902317 >> 172.21.15.86:0/1465902317 conn(0x7f2c040b2110 msgr2=0x7f2c040726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:55.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.459+0000 7f2c09400640 1 -- 172.21.15.86:0/1465902317 shutdown_connections 2024-04-20T21:29:55.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.459+0000 7f2c09400640 1 -- 172.21.15.86:0/1465902317 wait complete. 2024-04-20T21:29:55.478 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:29:55.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.684+0000 7f7878c00640 1 Processor -- start 2024-04-20T21:29:55.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.684+0000 7f7878c00640 1 -- start start 2024-04-20T21:29:55.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.685+0000 7f7878c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f787415ea70 0x7f78740d3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:55.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.685+0000 7f7878c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f787406d570 0x7f78740cf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:55.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.686+0000 7f7878c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78740cf120 con 0x7f78740d45a0 2024-04-20T21:29:55.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.686+0000 7f7878c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f787406de70 con 0x7f787415ea70 2024-04-20T21:29:55.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.686+0000 7f7878c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f787406e150 con 0x7f787406d570 2024-04-20T21:29:55.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.686+0000 7f786aa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f787415ea70 0x7f78740d3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:55.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.686+0000 7f786aa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f787415ea70 0x7f78740d3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49310/0 (socket says 172.21.15.86:49310) 2024-04-20T21:29:55.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.686+0000 7f786aa00640 1 -- 172.21.15.86:0/3416603097 learned_addr learned my addr 172.21.15.86:0/3416603097 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:55.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.686+0000 7f7873e00640 1 --2- 172.21.15.86:0/3416603097 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f787406d570 0x7f78740cf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:55.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.686+0000 7f786aa00640 1 -- 172.21.15.86:0/3416603097 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f787406d570 msgr2=0x7f78740cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:55.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.686+0000 7f786aa00640 1 --2- 172.21.15.86:0/3416603097 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f787406d570 0x7f78740cf8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:55.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.686+0000 7f786aa00640 1 -- 172.21.15.86:0/3416603097 >> v1:172.21.15.144:6789/0 conn(0x7f78740d45a0 legacy=0x7f78740d4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:55.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.686+0000 7f786aa00640 1 -- 172.21.15.86:0/3416603097 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f787406e430 con 0x7f787415ea70 2024-04-20T21:29:55.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.688+0000 7f786aa00640 1 --2- 172.21.15.86:0/3416603097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f787415ea70 0x7f78740d3260 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f78740b4120 tx=0x7f785c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dc807aab7bf7b1a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:55.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.688+0000 7f7872a00640 1 -- 172.21.15.86:0/3416603097 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785c0315a0 con 0x7f787415ea70 2024-04-20T21:29:55.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.688+0000 7f7872a00640 1 -- 172.21.15.86:0/3416603097 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f785c037070 con 0x7f787415ea70 2024-04-20T21:29:55.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.688+0000 7f7872a00640 1 -- 172.21.15.86:0/3416603097 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785c031960 con 0x7f787415ea70 2024-04-20T21:29:55.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.688+0000 7f7878c00640 1 -- 172.21.15.86:0/3416603097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f787415ea70 msgr2=0x7f78740d3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:55.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.688+0000 7f7878c00640 1 --2- 172.21.15.86:0/3416603097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f787415ea70 0x7f78740d3260 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f78740b4120 tx=0x7f785c02b840 comp rx=0 tx=0).stop 2024-04-20T21:29:55.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.689+0000 7f7878c00640 1 -- 172.21.15.86:0/3416603097 shutdown_connections 2024-04-20T21:29:55.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.689+0000 7f7878c00640 1 --2- 172.21.15.86:0/3416603097 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f787406d570 0x7f78740cf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:55.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.689+0000 7f7878c00640 1 --2- 172.21.15.86:0/3416603097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f787415ea70 0x7f78740d3260 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:55.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.689+0000 7f7878c00640 1 -- 172.21.15.86:0/3416603097 >> 172.21.15.86:0/3416603097 conn(0x7f787409d9d0 msgr2=0x7f787408bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:55.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.689+0000 7f7878c00640 1 -- 172.21.15.86:0/3416603097 shutdown_connections 2024-04-20T21:29:55.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.689+0000 7f7878c00640 1 -- 172.21.15.86:0/3416603097 wait complete. 2024-04-20T21:29:55.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.690+0000 7f7878c00640 1 Processor -- start 2024-04-20T21:29:55.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.691+0000 7f7878c00640 1 -- start start 2024-04-20T21:29:55.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.691+0000 7f7878c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f787406d570 0x7f78740cce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:55.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.691+0000 7f7878c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78740d45a0 0x7f78741591f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:55.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.691+0000 7f7878c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f787415ea70 0x7f7874159730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:55.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.691+0000 7f7878c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f787406db60 con 0x7f78740d45a0 2024-04-20T21:29:55.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.691+0000 7f7878c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f787406d9e0 con 0x7f787415ea70 2024-04-20T21:29:55.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.691+0000 7f7878c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f78740cfe20 con 0x7f787406d570 2024-04-20T21:29:55.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.692+0000 7f786aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78740d45a0 0x7f78741591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:55.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.692+0000 7f786aa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78740d45a0 0x7f78741591f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49326/0 (socket says 172.21.15.86:49326) 2024-04-20T21:29:55.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.692+0000 7f786aa00640 1 -- 172.21.15.86:0/1260902617 learned_addr learned my addr 172.21.15.86:0/1260902617 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:55.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.692+0000 7f7873400640 1 --2- 172.21.15.86:0/1260902617 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f787406d570 0x7f78740cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:55.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.692+0000 7f786aa00640 1 -- 172.21.15.86:0/1260902617 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f787406d570 msgr2=0x7f78740cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:55.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.692+0000 7f786aa00640 1 --2- 172.21.15.86:0/1260902617 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f787406d570 0x7f78740cce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:55.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.692+0000 7f7873e00640 1 --2- 172.21.15.86:0/1260902617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f787415ea70 0x7f7874159730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:55.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.692+0000 7f786aa00640 1 -- 172.21.15.86:0/1260902617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f787415ea70 msgr2=0x7f7874159730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:55.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.692+0000 7f786aa00640 1 --2- 172.21.15.86:0/1260902617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f787415ea70 0x7f7874159730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:55.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.692+0000 7f786aa00640 1 -- 172.21.15.86:0/1260902617 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78740cbdd0 con 0x7f78740d45a0 2024-04-20T21:29:55.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.693+0000 7f786aa00640 1 --2- 172.21.15.86:0/1260902617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78740d45a0 0x7f78741591f0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f785c02b880 tx=0x7f785c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:55.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.693+0000 7f786be00640 1 -- 172.21.15.86:0/1260902617 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785c032630 con 0x7f78740d45a0 2024-04-20T21:29:55.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.693+0000 7f786be00640 1 -- 172.21.15.86:0/1260902617 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f785c0327f0 con 0x7f78740d45a0 2024-04-20T21:29:55.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.693+0000 7f7878c00640 1 -- 172.21.15.86:0/1260902617 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78740cc0b0 con 0x7f78740d45a0 2024-04-20T21:29:55.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.693+0000 7f786be00640 1 -- 172.21.15.86:0/1260902617 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785c032be0 con 0x7f78740d45a0 2024-04-20T21:29:55.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.693+0000 7f7878c00640 1 -- 172.21.15.86:0/1260902617 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78740cc670 con 0x7f78740d45a0 2024-04-20T21:29:55.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.694+0000 7f7878c00640 1 -- 172.21.15.86:0/1260902617 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f787404fd90 con 0x7f78740d45a0 2024-04-20T21:29:55.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.695+0000 7f786be00640 1 -- 172.21.15.86:0/1260902617 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f785c03b3f0 con 0x7f78740d45a0 2024-04-20T21:29:55.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.696+0000 7f786be00640 1 --2- 172.21.15.86:0/1260902617 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f785006c8f0 0x7f785006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:55.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.697+0000 7f786be00640 1 -- 172.21.15.86:0/1260902617 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7f785c02f030 con 0x7f78740d45a0 2024-04-20T21:29:55.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.697+0000 7f7873400640 1 --2- 172.21.15.86:0/1260902617 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f785006c8f0 0x7f785006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:55.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.697+0000 7f7873400640 1 --2- 172.21.15.86:0/1260902617 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f785006c8f0 0x7f785006edb0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f78740cb9f0 tx=0x7f7864009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:55.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.701+0000 7f786be00640 1 -- 172.21.15.86:0/1260902617 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f785c040070 con 0x7f78740d45a0 2024-04-20T21:29:55.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.905+0000 7f7878c00640 1 -- 172.21.15.86:0/1260902617 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f787415da30 con 0x7f78740d45a0 2024-04-20T21:29:55.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.906+0000 7f786be00640 1 -- 172.21.15.86:0/1260902617 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v112) ==== 104+0+11 (secure 0 0 0) 0x7f785c07be00 con 0x7f78740d45a0 2024-04-20T21:29:55.905 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-20T21:29:55.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.909+0000 7f7878c00640 1 -- 172.21.15.86:0/1260902617 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f785006c8f0 msgr2=0x7f785006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:55.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.909+0000 7f7878c00640 1 --2- 172.21.15.86:0/1260902617 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f785006c8f0 0x7f785006edb0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f78740cb9f0 tx=0x7f7864009040 comp rx=0 tx=0).stop 2024-04-20T21:29:55.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.909+0000 7f7878c00640 1 -- 172.21.15.86:0/1260902617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78740d45a0 msgr2=0x7f78741591f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:55.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.909+0000 7f7878c00640 1 --2- 172.21.15.86:0/1260902617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78740d45a0 0x7f78741591f0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f785c02b880 tx=0x7f785c0046c0 comp rx=0 tx=0).stop 2024-04-20T21:29:55.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.909+0000 7f7878c00640 1 -- 172.21.15.86:0/1260902617 shutdown_connections 2024-04-20T21:29:55.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.909+0000 7f7878c00640 1 --2- 172.21.15.86:0/1260902617 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f785006c8f0 0x7f785006edb0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:55.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.909+0000 7f7878c00640 1 --2- 172.21.15.86:0/1260902617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f787415ea70 0x7f7874159730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:55.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.909+0000 7f7878c00640 1 --2- 172.21.15.86:0/1260902617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78740d45a0 0x7f78741591f0 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:55.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.909+0000 7f7878c00640 1 --2- 172.21.15.86:0/1260902617 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f787406d570 0x7f78740cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:55.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.909+0000 7f7878c00640 1 -- 172.21.15.86:0/1260902617 >> 172.21.15.86:0/1260902617 conn(0x7f787409d9d0 msgr2=0x7f78740b14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:55.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.909+0000 7f7878c00640 1 -- 172.21.15.86:0/1260902617 shutdown_connections 2024-04-20T21:29:55.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:55.909+0000 7f7878c00640 1 -- 172.21.15.86:0/1260902617 wait complete. 2024-04-20T21:29:55.927 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:29:56.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.127+0000 7f194ac00640 1 Processor -- start 2024-04-20T21:29:56.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.127+0000 7f194ac00640 1 -- start start 2024-04-20T21:29:56.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.129+0000 7f194ac00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1944146210 0x7f194406e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:56.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.129+0000 7f194ac00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f194406bfd0 0x7f19440cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:56.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.129+0000 7f194ac00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f194415cd40 con 0x7f194406bbf0 2024-04-20T21:29:56.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.129+0000 7f194ac00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f194415d000 con 0x7f194406bfd0 2024-04-20T21:29:56.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.129+0000 7f194ac00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f194415d2e0 con 0x7f1944146210 2024-04-20T21:29:56.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.130+0000 7f1948c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f194406bfd0 0x7f19440cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:56.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.130+0000 7f1948c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f194406bfd0 0x7f19440cd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38734/0 (socket says 172.21.15.86:38734) 2024-04-20T21:29:56.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.130+0000 7f1948c00640 1 -- 172.21.15.86:0/4064524351 learned_addr learned my addr 172.21.15.86:0/4064524351 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:56.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.130+0000 7f193b400640 1 --2- 172.21.15.86:0/4064524351 >> v2:172.21.15.144:3300/0 conn(0x7f1944146210 0x7f194406e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:56.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.130+0000 7f1948c00640 1 -- 172.21.15.86:0/4064524351 >> v2:172.21.15.144:3300/0 conn(0x7f1944146210 msgr2=0x7f194406e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:56.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.130+0000 7f1948c00640 1 --2- 172.21.15.86:0/4064524351 >> v2:172.21.15.144:3300/0 conn(0x7f1944146210 0x7f194406e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:56.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.130+0000 7f1948c00640 1 -- 172.21.15.86:0/4064524351 >> v1:172.21.15.144:6789/0 conn(0x7f194406bbf0 legacy=0x7f1944145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:56.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.130+0000 7f1948c00640 1 -- 172.21.15.86:0/4064524351 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f194415d5c0 con 0x7f194406bfd0 2024-04-20T21:29:56.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.132+0000 7f1948c00640 1 --2- 172.21.15.86:0/4064524351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f194406bfd0 0x7f19440cd540 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f194406dcc0 tx=0x7f193c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d65bb9efe6acfbbe server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:56.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.132+0000 7f1943400640 1 -- 172.21.15.86:0/4064524351 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193c035020 con 0x7f194406bfd0 2024-04-20T21:29:56.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.132+0000 7f1943400640 1 -- 172.21.15.86:0/4064524351 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f193c031870 con 0x7f194406bfd0 2024-04-20T21:29:56.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.132+0000 7f1943400640 1 -- 172.21.15.86:0/4064524351 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193c031c40 con 0x7f194406bfd0 2024-04-20T21:29:56.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.133+0000 7f194ac00640 1 -- 172.21.15.86:0/4064524351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f194406bfd0 msgr2=0x7f19440cd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:56.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.133+0000 7f194ac00640 1 --2- 172.21.15.86:0/4064524351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f194406bfd0 0x7f19440cd540 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f194406dcc0 tx=0x7f193c02b910 comp rx=0 tx=0).stop 2024-04-20T21:29:56.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.133+0000 7f194ac00640 1 -- 172.21.15.86:0/4064524351 shutdown_connections 2024-04-20T21:29:56.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.133+0000 7f194ac00640 1 --2- 172.21.15.86:0/4064524351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f194406bfd0 0x7f19440cd540 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:56.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.133+0000 7f194ac00640 1 --2- 172.21.15.86:0/4064524351 >> v2:172.21.15.144:3300/0 conn(0x7f1944146210 0x7f194406e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:56.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.133+0000 7f194ac00640 1 -- 172.21.15.86:0/4064524351 >> 172.21.15.86:0/4064524351 conn(0x7f19440b2110 msgr2=0x7f19440b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:56.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.133+0000 7f194ac00640 1 -- 172.21.15.86:0/4064524351 shutdown_connections 2024-04-20T21:29:56.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.133+0000 7f194ac00640 1 -- 172.21.15.86:0/4064524351 wait complete. 2024-04-20T21:29:56.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.135+0000 7f194ac00640 1 Processor -- start 2024-04-20T21:29:56.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.135+0000 7f194ac00640 1 -- start start 2024-04-20T21:29:56.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.136+0000 7f194ac00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f194406bbf0 0x7f19441f89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:56.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.136+0000 7f194ac00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f194406bfd0 0x7f19440b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:56.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.136+0000 7f194ac00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1944146210 0x7f194406e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:56.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.136+0000 7f194ac00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f19440cdc00 con 0x7f1944146210 2024-04-20T21:29:56.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.136+0000 7f194ac00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f19440cda80 con 0x7f194406bbf0 2024-04-20T21:29:56.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.136+0000 7f194ac00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f19440cdd80 con 0x7f194406bfd0 2024-04-20T21:29:56.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.136+0000 7f1948c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1944146210 0x7f194406e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:56.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.137+0000 7f1948c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1944146210 0x7f194406e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38738/0 (socket says 172.21.15.86:38738) 2024-04-20T21:29:56.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.137+0000 7f1948c00640 1 -- 172.21.15.86:0/1159850246 learned_addr learned my addr 172.21.15.86:0/1159850246 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:56.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.137+0000 7f1943e00640 1 --2- 172.21.15.86:0/1159850246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f194406bbf0 0x7f19441f89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:56.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.137+0000 7f1948c00640 1 -- 172.21.15.86:0/1159850246 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f194406bfd0 msgr2=0x7f19440b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:56.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.137+0000 7f193b400640 1 --2- 172.21.15.86:0/1159850246 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f194406bfd0 0x7f19440b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:56.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.137+0000 7f1948c00640 1 --2- 172.21.15.86:0/1159850246 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f194406bfd0 0x7f19440b4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:56.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.137+0000 7f1948c00640 1 -- 172.21.15.86:0/1159850246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f194406bbf0 msgr2=0x7f19441f89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:56.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.137+0000 7f1948c00640 1 --2- 172.21.15.86:0/1159850246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f194406bbf0 0x7f19441f89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:56.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.137+0000 7f1948c00640 1 -- 172.21.15.86:0/1159850246 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f194413cdc0 con 0x7f1944146210 2024-04-20T21:29:56.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.137+0000 7f1943e00640 1 --2- 172.21.15.86:0/1159850246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f194406bbf0 0x7f19441f89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:29:56.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.137+0000 7f1948c00640 1 --2- 172.21.15.86:0/1159850246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1944146210 0x7f194406e270 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f193c02b950 tx=0x7f193c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:56.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.138+0000 7f1940c00640 1 -- 172.21.15.86:0/1159850246 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193c040070 con 0x7f1944146210 2024-04-20T21:29:56.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.138+0000 7f1940c00640 1 -- 172.21.15.86:0/1159850246 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f193c030d00 con 0x7f1944146210 2024-04-20T21:29:56.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.138+0000 7f194ac00640 1 -- 172.21.15.86:0/1159850246 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19441f7ed0 con 0x7f1944146210 2024-04-20T21:29:56.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.138+0000 7f1940c00640 1 -- 172.21.15.86:0/1159850246 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193c035040 con 0x7f1944146210 2024-04-20T21:29:56.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.138+0000 7f194ac00640 1 -- 172.21.15.86:0/1159850246 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19441f8490 con 0x7f1944146210 2024-04-20T21:29:56.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.139+0000 7f194ac00640 1 -- 172.21.15.86:0/1159850246 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19440689e0 con 0x7f1944146210 2024-04-20T21:29:56.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.140+0000 7f1940c00640 1 -- 172.21.15.86:0/1159850246 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f193c038020 con 0x7f1944146210 2024-04-20T21:29:56.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.141+0000 7f1940c00640 1 --2- 172.21.15.86:0/1159850246 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f192006c8f0 0x7f192006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:56.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.141+0000 7f1940c00640 1 -- 172.21.15.86:0/1159850246 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7f193c0ada30 con 0x7f1944146210 2024-04-20T21:29:56.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.141+0000 7f1943e00640 1 --2- 172.21.15.86:0/1159850246 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f192006c8f0 0x7f192006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:56.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.142+0000 7f1943e00640 1 --2- 172.21.15.86:0/1159850246 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f192006c8f0 0x7f192006edb0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f1944144b10 tx=0x7f1934009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:56.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.146+0000 7f1940c00640 1 -- 172.21.15.86:0/1159850246 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f193c07bdc0 con 0x7f1944146210 2024-04-20T21:29:56.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.345+0000 7f194ac00640 1 -- 172.21.15.86:0/1159850246 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f19440af4a0 con 0x7f1944146210 2024-04-20T21:29:56.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.347+0000 7f1940c00640 1 -- 172.21.15.86:0/1159850246 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v112) ==== 100+0+11 (secure 0 0 0) 0x7f193c07b6c0 con 0x7f1944146210 2024-04-20T21:29:56.346 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-20T21:29:56.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.349+0000 7f194ac00640 1 -- 172.21.15.86:0/1159850246 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f192006c8f0 msgr2=0x7f192006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:56.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.349+0000 7f194ac00640 1 --2- 172.21.15.86:0/1159850246 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f192006c8f0 0x7f192006edb0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f1944144b10 tx=0x7f1934009040 comp rx=0 tx=0).stop 2024-04-20T21:29:56.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.349+0000 7f194ac00640 1 -- 172.21.15.86:0/1159850246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1944146210 msgr2=0x7f194406e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:56.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.349+0000 7f194ac00640 1 --2- 172.21.15.86:0/1159850246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1944146210 0x7f194406e270 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f193c02b950 tx=0x7f193c0046c0 comp rx=0 tx=0).stop 2024-04-20T21:29:56.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.349+0000 7f194ac00640 1 -- 172.21.15.86:0/1159850246 shutdown_connections 2024-04-20T21:29:56.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.349+0000 7f194ac00640 1 --2- 172.21.15.86:0/1159850246 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f192006c8f0 0x7f192006edb0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:56.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.349+0000 7f194ac00640 1 --2- 172.21.15.86:0/1159850246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1944146210 0x7f194406e270 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:56.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.349+0000 7f194ac00640 1 --2- 172.21.15.86:0/1159850246 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f194406bfd0 0x7f19440b4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:56.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.349+0000 7f194ac00640 1 --2- 172.21.15.86:0/1159850246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f194406bbf0 0x7f19441f89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:56.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.349+0000 7f194ac00640 1 -- 172.21.15.86:0/1159850246 >> 172.21.15.86:0/1159850246 conn(0x7f19440b2110 msgr2=0x7f194415dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:56.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.349+0000 7f194ac00640 1 -- 172.21.15.86:0/1159850246 shutdown_connections 2024-04-20T21:29:56.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.349+0000 7f194ac00640 1 -- 172.21.15.86:0/1159850246 wait complete. 2024-04-20T21:29:56.366 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T21:29:56.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.568+0000 7f60cf800640 1 Processor -- start 2024-04-20T21:29:56.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.568+0000 7f60cf800640 1 -- start start 2024-04-20T21:29:56.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.570+0000 7f60cf800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f60c815b5f0 0x7f60c80b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:56.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.570+0000 7f60cf800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f60c806f070 0x7f60c80b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:56.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.570+0000 7f60cf800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f60c80cff80 0x7f60c80d0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:56.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.570+0000 7f60cf800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f60c80b4650 con 0x7f60c815b5f0 2024-04-20T21:29:56.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.570+0000 7f60cf800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f60c815b9d0 con 0x7f60c806f070 2024-04-20T21:29:56.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.570+0000 7f60cf800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f60c80af850 con 0x7f60c80cff80 2024-04-20T21:29:56.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.570+0000 7f60cce00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f60c815b5f0 0x7f60c80b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:56.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.570+0000 7f60cce00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f60c815b5f0 0x7f60c80b3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38742/0 (socket says 172.21.15.86:38742) 2024-04-20T21:29:56.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.570+0000 7f60cce00640 1 -- 172.21.15.86:0/1332517882 learned_addr learned my addr 172.21.15.86:0/1332517882 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:56.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.571+0000 7f60cd800640 1 --2- 172.21.15.86:0/1332517882 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f60c80cff80 0x7f60c80d0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:56.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.571+0000 7f60bfe00640 1 --2- 172.21.15.86:0/1332517882 >> v2:172.21.15.144:3300/0 conn(0x7f60c806f070 0x7f60c80b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:56.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.571+0000 7f60cce00640 1 -- 172.21.15.86:0/1332517882 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f60c80cff80 msgr2=0x7f60c80d0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:56.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.571+0000 7f60cce00640 1 --2- 172.21.15.86:0/1332517882 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f60c80cff80 0x7f60c80d0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:56.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.571+0000 7f60cce00640 1 -- 172.21.15.86:0/1332517882 >> v2:172.21.15.144:3300/0 conn(0x7f60c806f070 msgr2=0x7f60c80b50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:56.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.571+0000 7f60cce00640 1 --2- 172.21.15.86:0/1332517882 >> v2:172.21.15.144:3300/0 conn(0x7f60c806f070 0x7f60c80b50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:56.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.571+0000 7f60cce00640 1 -- 172.21.15.86:0/1332517882 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60c80afb30 con 0x7f60c815b5f0 2024-04-20T21:29:56.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.571+0000 7f60bfe00640 1 --2- 172.21.15.86:0/1332517882 >> v2:172.21.15.144:3300/0 conn(0x7f60c806f070 0x7f60c80b50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:29:56.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.572+0000 7f60cce00640 1 --2- 172.21.15.86:0/1332517882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60c815b5f0 0x7f60c80b3d40 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f60c806ffe0 tx=0x7f60b802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a4a35ef330d92c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:56.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.572+0000 7f60c7e00640 1 -- 172.21.15.86:0/1332517882 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b8035020 con 0x7f60c815b5f0 2024-04-20T21:29:56.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.572+0000 7f60c7e00640 1 -- 172.21.15.86:0/1332517882 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60b8031870 con 0x7f60c815b5f0 2024-04-20T21:29:56.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.572+0000 7f60c7e00640 1 -- 172.21.15.86:0/1332517882 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b8031c40 con 0x7f60c815b5f0 2024-04-20T21:29:56.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.573+0000 7f60cf800640 1 -- 172.21.15.86:0/1332517882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60c815b5f0 msgr2=0x7f60c80b3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:56.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.573+0000 7f60cf800640 1 --2- 172.21.15.86:0/1332517882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60c815b5f0 0x7f60c80b3d40 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f60c806ffe0 tx=0x7f60b802b910 comp rx=0 tx=0).stop 2024-04-20T21:29:56.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.573+0000 7f60cf800640 1 -- 172.21.15.86:0/1332517882 shutdown_connections 2024-04-20T21:29:56.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.573+0000 7f60cf800640 1 --2- 172.21.15.86:0/1332517882 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f60c80cff80 0x7f60c80d0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:56.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.573+0000 7f60cf800640 1 --2- 172.21.15.86:0/1332517882 >> v2:172.21.15.144:3300/0 conn(0x7f60c806f070 0x7f60c80b50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:56.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.573+0000 7f60cf800640 1 --2- 172.21.15.86:0/1332517882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60c815b5f0 0x7f60c80b3d40 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:56.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.573+0000 7f60cf800640 1 -- 172.21.15.86:0/1332517882 >> 172.21.15.86:0/1332517882 conn(0x7f60c806b980 msgr2=0x7f60c806bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:56.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.573+0000 7f60cf800640 1 -- 172.21.15.86:0/1332517882 shutdown_connections 2024-04-20T21:29:56.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.573+0000 7f60cf800640 1 -- 172.21.15.86:0/1332517882 wait complete. 2024-04-20T21:29:56.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.575+0000 7f60cf800640 1 Processor -- start 2024-04-20T21:29:56.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.575+0000 7f60cf800640 1 -- start start 2024-04-20T21:29:56.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.576+0000 7f60cf800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f60c806f070 0x7f60c81f5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:56.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.576+0000 7f60cf800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60c80cff80 0x7f60c80afd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:56.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.576+0000 7f60cf800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60c815b5f0 0x7f60c815c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:56.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.576+0000 7f60cf800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f60c80b55e0 con 0x7f60c80cff80 2024-04-20T21:29:56.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.576+0000 7f60cf800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f60c80b4b80 con 0x7f60c815b5f0 2024-04-20T21:29:56.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.576+0000 7f60cf800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f60c80ce130 con 0x7f60c806f070 2024-04-20T21:29:56.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.576+0000 7f60bfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60c80cff80 0x7f60c80afd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:56.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.576+0000 7f60bfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60c80cff80 0x7f60c80afd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38746/0 (socket says 172.21.15.86:38746) 2024-04-20T21:29:56.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.576+0000 7f60bfe00640 1 -- 172.21.15.86:0/1725820822 learned_addr learned my addr 172.21.15.86:0/1725820822 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:56.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.577+0000 7f60cd800640 1 --2- 172.21.15.86:0/1725820822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60c815b5f0 0x7f60c815c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:56.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.577+0000 7f60cce00640 1 --2- 172.21.15.86:0/1725820822 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f60c806f070 0x7f60c81f5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:56.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.577+0000 7f60bfe00640 1 -- 172.21.15.86:0/1725820822 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f60c806f070 msgr2=0x7f60c81f5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:56.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.577+0000 7f60bfe00640 1 --2- 172.21.15.86:0/1725820822 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f60c806f070 0x7f60c81f5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:56.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.577+0000 7f60bfe00640 1 -- 172.21.15.86:0/1725820822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60c815b5f0 msgr2=0x7f60c815c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:56.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.577+0000 7f60bfe00640 1 --2- 172.21.15.86:0/1725820822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60c815b5f0 0x7f60c815c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:56.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.577+0000 7f60bfe00640 1 -- 172.21.15.86:0/1725820822 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60c81f5ff0 con 0x7f60c80cff80 2024-04-20T21:29:56.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.577+0000 7f60bfe00640 1 --2- 172.21.15.86:0/1725820822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60c80cff80 0x7f60c80afd70 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f60b000e760 tx=0x7f60b000ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:56.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.577+0000 7f60c5600640 1 -- 172.21.15.86:0/1725820822 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b000f040 con 0x7f60c80cff80 2024-04-20T21:29:56.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.577+0000 7f60c5600640 1 -- 172.21.15.86:0/1725820822 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60b0004230 con 0x7f60c80cff80 2024-04-20T21:29:56.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.577+0000 7f60cf800640 1 -- 172.21.15.86:0/1725820822 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60c81f62d0 con 0x7f60c80cff80 2024-04-20T21:29:56.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.577+0000 7f60c5600640 1 -- 172.21.15.86:0/1725820822 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b000ee70 con 0x7f60c80cff80 2024-04-20T21:29:56.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.578+0000 7f60cf800640 1 -- 172.21.15.86:0/1725820822 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60c814e610 con 0x7f60c80cff80 2024-04-20T21:29:56.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.579+0000 7f60cf800640 1 -- 172.21.15.86:0/1725820822 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60c80689e0 con 0x7f60c80cff80 2024-04-20T21:29:56.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.580+0000 7f60c5600640 1 -- 172.21.15.86:0/1725820822 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f60b000c6d0 con 0x7f60c80cff80 2024-04-20T21:29:56.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.581+0000 7f60c5600640 1 --2- 172.21.15.86:0/1725820822 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f60a406c8f0 0x7f60a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:56.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.581+0000 7f60c5600640 1 -- 172.21.15.86:0/1725820822 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7f60b008b850 con 0x7f60c80cff80 2024-04-20T21:29:56.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.581+0000 7f60cce00640 1 --2- 172.21.15.86:0/1725820822 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f60a406c8f0 0x7f60a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:56.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.582+0000 7f60cce00640 1 --2- 172.21.15.86:0/1725820822 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f60a406c8f0 0x7f60a406edb0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f60b8036000 tx=0x7f60b803c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:56.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.585+0000 7f60c5600640 1 -- 172.21.15.86:0/1725820822 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f60b0017020 con 0x7f60c80cff80 2024-04-20T21:29:56.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.798+0000 7f60cf800640 1 -- 172.21.15.86:0/1725820822 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f60c80b3eb0 con 0x7f60c80cff80 2024-04-20T21:29:56.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.799+0000 7f60c5600640 1 -- 172.21.15.86:0/1725820822 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v112) ==== 80+0+0 (secure 0 0 0) 0x7f60b0059c80 con 0x7f60c80cff80 2024-04-20T21:29:56.798 INFO:teuthology.orchestra.run.smithi086.stderr:listed 0 entries 2024-04-20T21:29:56.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.802+0000 7f60cf800640 1 -- 172.21.15.86:0/1725820822 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f60a406c8f0 msgr2=0x7f60a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:56.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.802+0000 7f60cf800640 1 --2- 172.21.15.86:0/1725820822 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f60a406c8f0 0x7f60a406edb0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f60b8036000 tx=0x7f60b803c040 comp rx=0 tx=0).stop 2024-04-20T21:29:56.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.802+0000 7f60cf800640 1 -- 172.21.15.86:0/1725820822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60c80cff80 msgr2=0x7f60c80afd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:56.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.802+0000 7f60cf800640 1 --2- 172.21.15.86:0/1725820822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60c80cff80 0x7f60c80afd70 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f60b000e760 tx=0x7f60b000ec20 comp rx=0 tx=0).stop 2024-04-20T21:29:56.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.802+0000 7f60cf800640 1 -- 172.21.15.86:0/1725820822 shutdown_connections 2024-04-20T21:29:56.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.802+0000 7f60cf800640 1 --2- 172.21.15.86:0/1725820822 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f60a406c8f0 0x7f60a406edb0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:56.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.802+0000 7f60cf800640 1 --2- 172.21.15.86:0/1725820822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60c815b5f0 0x7f60c815c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:56.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.802+0000 7f60cf800640 1 --2- 172.21.15.86:0/1725820822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60c80cff80 0x7f60c80afd70 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:56.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.802+0000 7f60cf800640 1 --2- 172.21.15.86:0/1725820822 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f60c806f070 0x7f60c81f5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:56.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.802+0000 7f60cf800640 1 -- 172.21.15.86:0/1725820822 >> 172.21.15.86:0/1725820822 conn(0x7f60c806b980 msgr2=0x7f60c815e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:56.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.802+0000 7f60cf800640 1 -- 172.21.15.86:0/1725820822 shutdown_connections 2024-04-20T21:29:56.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:56.802+0000 7f60cf800640 1 -- 172.21.15.86:0/1725820822 wait complete. 2024-04-20T21:29:56.820 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:29:56.820 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:29:56.869 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:29:57.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.118+0000 7fa8ec800640 1 Processor -- start 2024-04-20T21:29:57.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.118+0000 7fa8ec800640 1 -- start start 2024-04-20T21:29:57.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.120+0000 7fa8ec800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa8e4146130 0x7fa8e406e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:57.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.120+0000 7fa8ec800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa8e4072410 0x7fa8e40cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:57.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.120+0000 7fa8ec800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8e406dc30 con 0x7fa8e40ccdd0 2024-04-20T21:29:57.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.120+0000 7fa8ec800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa8e40cd8e0 con 0x7fa8e4072410 2024-04-20T21:29:57.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.120+0000 7fa8ec800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fa8e40cdbc0 con 0x7fa8e4146130 2024-04-20T21:29:57.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.120+0000 7fa8ea800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa8e4072410 0x7fa8e40cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:57.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.120+0000 7fa8e9400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fa8e40ccdd0 0x7fa8e4071d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39314/0 (socket says 172.21.15.86:39314) 2024-04-20T21:29:57.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.120+0000 7fa8e9400640 1 -- 172.21.15.86:0/3011067700 learned_addr learned my addr 172.21.15.86:0/3011067700 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:29:57.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.120+0000 7fa8ea800640 1 --2- 172.21.15.86:0/3011067700 >> v2:172.21.15.86:3300/0 conn(0x7fa8e4072410 0x7fa8e40cd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38758/0 (socket says 172.21.15.86:38758) 2024-04-20T21:29:57.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.120+0000 7fa8e9e00640 1 --2- 172.21.15.86:0/3011067700 >> v2:172.21.15.144:3300/0 conn(0x7fa8e4146130 0x7fa8e406e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:57.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.121+0000 7fa8ea800640 1 -- 172.21.15.86:0/3011067700 >> v2:172.21.15.144:3300/0 conn(0x7fa8e4146130 msgr2=0x7fa8e406e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:57.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.121+0000 7fa8ea800640 1 --2- 172.21.15.86:0/3011067700 >> v2:172.21.15.144:3300/0 conn(0x7fa8e4146130 0x7fa8e406e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:57.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.121+0000 7fa8ea800640 1 -- 172.21.15.86:0/3011067700 >> v1:172.21.15.86:6789/0 conn(0x7fa8e40ccdd0 legacy=0x7fa8e4071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:57.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.121+0000 7fa8e8a00640 1 -- 172.21.15.86:0/3011067700 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2873218726 0 0) 0x7fa8e406dc30 con 0x7fa8e40ccdd0 2024-04-20T21:29:57.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.121+0000 7fa8ea800640 1 -- 172.21.15.86:0/3011067700 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8e406ded0 con 0x7fa8e4072410 2024-04-20T21:29:57.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.122+0000 7fa8ea800640 1 --2- 172.21.15.86:0/3011067700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8e4072410 0x7fa8e40cd230 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fa8e40b5950 tx=0x7fa8e002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48f9ab64dae20f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:57.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.122+0000 7fa8e8a00640 1 -- 172.21.15.86:0/3011067700 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0039070 con 0x7fa8e4072410 2024-04-20T21:29:57.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.122+0000 7fa8e8a00640 1 -- 172.21.15.86:0/3011067700 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8e002cd00 con 0x7fa8e4072410 2024-04-20T21:29:57.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.122+0000 7fa8e8a00640 1 -- 172.21.15.86:0/3011067700 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0034660 con 0x7fa8e4072410 2024-04-20T21:29:57.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.123+0000 7fa8ec800640 1 -- 172.21.15.86:0/3011067700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8e4072410 msgr2=0x7fa8e40cd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:57.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.123+0000 7fa8ec800640 1 --2- 172.21.15.86:0/3011067700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8e4072410 0x7fa8e40cd230 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fa8e40b5950 tx=0x7fa8e002bf50 comp rx=0 tx=0).stop 2024-04-20T21:29:57.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.123+0000 7fa8ec800640 1 -- 172.21.15.86:0/3011067700 shutdown_connections 2024-04-20T21:29:57.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.123+0000 7fa8ec800640 1 --2- 172.21.15.86:0/3011067700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8e4072410 0x7fa8e40cd230 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:57.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.123+0000 7fa8ec800640 1 --2- 172.21.15.86:0/3011067700 >> v2:172.21.15.144:3300/0 conn(0x7fa8e4146130 0x7fa8e406e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:57.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.123+0000 7fa8ec800640 1 -- 172.21.15.86:0/3011067700 >> 172.21.15.86:0/3011067700 conn(0x7fa8e40b2360 msgr2=0x7fa8e40b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:57.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.123+0000 7fa8ec800640 1 -- 172.21.15.86:0/3011067700 shutdown_connections 2024-04-20T21:29:57.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.123+0000 7fa8ec800640 1 -- 172.21.15.86:0/3011067700 wait complete. 2024-04-20T21:29:57.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.124+0000 7fa8ec800640 1 Processor -- start 2024-04-20T21:29:57.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.125+0000 7fa8ec800640 1 -- start start 2024-04-20T21:29:57.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.125+0000 7fa8ec800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8e4072410 0x7fa8e40a9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:57.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.125+0000 7fa8ec800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8e40ccdd0 0x7fa8e4140bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:57.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.126+0000 7fa8ec800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8e4146130 0x7fa8e40ab540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:57.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.126+0000 7fa8ec800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e415cc20 con 0x7fa8e40ccdd0 2024-04-20T21:29:57.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.126+0000 7fa8ec800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e406f150 con 0x7fa8e4072410 2024-04-20T21:29:57.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.126+0000 7fa8ec800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa8e415cda0 con 0x7fa8e4146130 2024-04-20T21:29:57.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.126+0000 7fa8e9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8e40ccdd0 0x7fa8e4140bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:57.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.126+0000 7fa8e9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8e40ccdd0 0x7fa8e4140bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38762/0 (socket says 172.21.15.86:38762) 2024-04-20T21:29:57.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.126+0000 7fa8e9400640 1 -- 172.21.15.86:0/687882821 learned_addr learned my addr 172.21.15.86:0/687882821 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:57.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.126+0000 7fa8e9400640 1 -- 172.21.15.86:0/687882821 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8e4146130 msgr2=0x7fa8e40ab540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:57.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.126+0000 7fa8e9400640 1 --2- 172.21.15.86:0/687882821 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8e4146130 0x7fa8e40ab540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:57.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.126+0000 7fa8e9400640 1 -- 172.21.15.86:0/687882821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8e4072410 msgr2=0x7fa8e40a9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:57.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.126+0000 7fa8e9e00640 1 --2- 172.21.15.86:0/687882821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8e4072410 0x7fa8e40a9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:57.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.126+0000 7fa8e9400640 1 --2- 172.21.15.86:0/687882821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8e4072410 0x7fa8e40a9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:57.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.126+0000 7fa8e9400640 1 -- 172.21.15.86:0/687882821 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8e40aa4d0 con 0x7fa8e40ccdd0 2024-04-20T21:29:57.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.127+0000 7fa8e9400640 1 --2- 172.21.15.86:0/687882821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8e40ccdd0 0x7fa8e4140bc0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fa8d400c4d0 tx=0x7fa8d400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:57.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.127+0000 7fa8de000640 1 -- 172.21.15.86:0/687882821 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4017070 con 0x7fa8e40ccdd0 2024-04-20T21:29:57.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.127+0000 7fa8de000640 1 -- 172.21.15.86:0/687882821 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8d4005d00 con 0x7fa8e40ccdd0 2024-04-20T21:29:57.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.127+0000 7fa8ec800640 1 -- 172.21.15.86:0/687882821 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8e40aa7b0 con 0x7fa8e40ccdd0 2024-04-20T21:29:57.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.127+0000 7fa8de000640 1 -- 172.21.15.86:0/687882821 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d40072b0 con 0x7fa8e40ccdd0 2024-04-20T21:29:57.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.127+0000 7fa8ec800640 1 -- 172.21.15.86:0/687882821 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8e41e6a20 con 0x7fa8e40ccdd0 2024-04-20T21:29:57.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.128+0000 7fa8ec800640 1 -- 172.21.15.86:0/687882821 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8e40689e0 con 0x7fa8e40ccdd0 2024-04-20T21:29:57.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.129+0000 7fa8de000640 1 -- 172.21.15.86:0/687882821 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa8d401a070 con 0x7fa8e40ccdd0 2024-04-20T21:29:57.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.131+0000 7fa8de000640 1 --2- 172.21.15.86:0/687882821 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa8c806c920 0x7fa8c806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:57.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.131+0000 7fa8de000640 1 -- 172.21.15.86:0/687882821 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7fa8d408aeb0 con 0x7fa8e40ccdd0 2024-04-20T21:29:57.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.131+0000 7fa8e9e00640 1 --2- 172.21.15.86:0/687882821 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa8c806c920 0x7fa8c806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:57.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.132+0000 7fa8e9e00640 1 --2- 172.21.15.86:0/687882821 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa8c806c920 0x7fa8c806ede0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8006fd0 tx=0x7fa8d8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:57.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.135+0000 7fa8de000640 1 -- 172.21.15.86:0/687882821 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa8d40592d0 con 0x7fa8e40ccdd0 2024-04-20T21:29:57.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.334+0000 7fa8ec800640 1 -- 172.21.15.86:0/687882821 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa8e40aa980 con 0x7fa8e40ccdd0 2024-04-20T21:29:57.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.336+0000 7fa8de000640 1 -- 172.21.15.86:0/687882821 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) ==== 74+0+16627 (secure 0 0 0) 0x7fa8d4058bd0 con 0x7fa8e40ccdd0 2024-04-20T21:29:57.336 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:29:57.336 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":112,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:29:45.322441+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:29:41.532824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-20T21:29:42.749687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:29:57.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.339+0000 7fa8ec800640 1 -- 172.21.15.86:0/687882821 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa8c806c920 msgr2=0x7fa8c806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:57.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.339+0000 7fa8ec800640 1 --2- 172.21.15.86:0/687882821 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa8c806c920 0x7fa8c806ede0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8006fd0 tx=0x7fa8d8008040 comp rx=0 tx=0).stop 2024-04-20T21:29:57.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.339+0000 7fa8ec800640 1 -- 172.21.15.86:0/687882821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8e40ccdd0 msgr2=0x7fa8e4140bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:57.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.339+0000 7fa8ec800640 1 --2- 172.21.15.86:0/687882821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8e40ccdd0 0x7fa8e4140bc0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fa8d400c4d0 tx=0x7fa8d400c990 comp rx=0 tx=0).stop 2024-04-20T21:29:57.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.339+0000 7fa8ec800640 1 -- 172.21.15.86:0/687882821 shutdown_connections 2024-04-20T21:29:57.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.339+0000 7fa8ec800640 1 --2- 172.21.15.86:0/687882821 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa8c806c920 0x7fa8c806ede0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:57.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.339+0000 7fa8ec800640 1 --2- 172.21.15.86:0/687882821 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa8e4146130 0x7fa8e40ab540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:57.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.339+0000 7fa8ec800640 1 --2- 172.21.15.86:0/687882821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8e40ccdd0 0x7fa8e4140bc0 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:57.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.339+0000 7fa8ec800640 1 --2- 172.21.15.86:0/687882821 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8e4072410 0x7fa8e40a9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:57.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.339+0000 7fa8ec800640 1 -- 172.21.15.86:0/687882821 >> 172.21.15.86:0/687882821 conn(0x7fa8e40b2360 msgr2=0x7fa8e4071d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:57.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.339+0000 7fa8ec800640 1 -- 172.21.15.86:0/687882821 shutdown_connections 2024-04-20T21:29:57.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.340+0000 7fa8ec800640 1 -- 172.21.15.86:0/687882821 wait complete. 2024-04-20T21:29:57.358 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:29:41.532824+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:29:42.749687+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:29:57.360 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:29:57.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.565+0000 7f808fe00640 1 Processor -- start 2024-04-20T21:29:57.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.565+0000 7f808fe00640 1 -- start start 2024-04-20T21:29:57.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.567+0000 7f808fe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f809015ef50 0x7f809015f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:57.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.567+0000 7f808fe00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80900729f0 con 0x7f809015b4b0 2024-04-20T21:29:57.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.567+0000 7f808fe00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8090072cd0 con 0x7f80900ae060 2024-04-20T21:29:57.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.567+0000 7f808fe00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f809015fe10 con 0x7f809015ef50 2024-04-20T21:29:57.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.567+0000 7f808ea00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f809015b4b0 0x7f80900cc3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39318/0 (socket says 172.21.15.86:39318) 2024-04-20T21:29:57.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.567+0000 7f808ea00640 1 -- 172.21.15.86:0/3072315057 learned_addr learned my addr 172.21.15.86:0/3072315057 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:29:57.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.568+0000 7f8086000640 1 --2- 172.21.15.86:0/3072315057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f809015ef50 0x7f809015f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:57.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.568+0000 7f808e000640 1 -- 172.21.15.86:0/3072315057 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3574997447 0 0) 0x7f80900729f0 con 0x7f809015b4b0 2024-04-20T21:29:57.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.568+0000 7f808e000640 1 -- 172.21.15.86:0/3072315057 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8074003520 con 0x7f809015b4b0 2024-04-20T21:29:57.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.568+0000 7f808e000640 1 -- 172.21.15.86:0/3072315057 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 902166852 0 0) 0x7f8074003520 con 0x7f809015b4b0 2024-04-20T21:29:57.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.568+0000 7f808e000640 1 -- 172.21.15.86:0/3072315057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f809015ef50 msgr2=0x7f809015f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:57.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.568+0000 7f808e000640 1 --2- 172.21.15.86:0/3072315057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f809015ef50 0x7f809015f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:57.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.568+0000 7f808e000640 1 -- 172.21.15.86:0/3072315057 >> v1:172.21.15.144:6789/0 conn(0x7f80900ae060 legacy=0x7f80900ae4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:57.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.568+0000 7f808e000640 1 -- 172.21.15.86:0/3072315057 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8090072fb0 con 0x7f809015b4b0 2024-04-20T21:29:57.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.568+0000 7f808e000640 1 -- 172.21.15.86:0/3072315057 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f8080001670 con 0x7f809015b4b0 2024-04-20T21:29:57.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.568+0000 7f808e000640 1 -- 172.21.15.86:0/3072315057 >> v1:172.21.15.86:6789/0 conn(0x7f809015b4b0 legacy=0x7f80900cc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:57.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.568+0000 7f808e000640 1 --2- 172.21.15.86:0/3072315057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80740039e0 0x7f8074003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:57.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.568+0000 7f808e000640 1 --2- 172.21.15.86:0/3072315057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8074004800 0x7f8074004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:57.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.568+0000 7f808e000640 1 --2- 172.21.15.86:0/3072315057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f809015ef50 0x7f80740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:57.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.568+0000 7f808e000640 1 -- 172.21.15.86:0/3072315057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f809015fe10 con 0x7f8074004800 2024-04-20T21:29:57.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.568+0000 7f808e000640 1 -- 172.21.15.86:0/3072315057 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f807400baa0 con 0x7f80740039e0 2024-04-20T21:29:57.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.568+0000 7f808e000640 1 -- 172.21.15.86:0/3072315057 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f807400bca0 con 0x7f809015ef50 2024-04-20T21:29:57.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.569+0000 7f8086000640 1 --2- 172.21.15.86:0/3072315057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8074004800 0x7f8074004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:57.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.569+0000 7f808ea00640 1 --2- 172.21.15.86:0/3072315057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80740039e0 0x7f8074003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:57.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.569+0000 7f8086000640 1 -- 172.21.15.86:0/3072315057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f809015ef50 msgr2=0x7f80740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:57.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.569+0000 7f8086000640 1 --2- 172.21.15.86:0/3072315057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f809015ef50 0x7f80740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:57.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.569+0000 7f8086000640 1 -- 172.21.15.86:0/3072315057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80740039e0 msgr2=0x7f8074003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:57.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.569+0000 7f8086000640 1 --2- 172.21.15.86:0/3072315057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f80740039e0 0x7f8074003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:57.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.569+0000 7f8086000640 1 -- 172.21.15.86:0/3072315057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8074003520 con 0x7f8074004800 2024-04-20T21:29:57.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.570+0000 7f8086000640 1 --2- 172.21.15.86:0/3072315057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8074004800 0x7f8074004c80 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f807800bd30 tx=0x7f8074005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21a0cdc681c8b4ac server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:57.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.570+0000 7f808e000640 1 -- 172.21.15.86:0/3072315057 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807800b930 con 0x7f8074004800 2024-04-20T21:29:57.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.571+0000 7f808e000640 1 -- 172.21.15.86:0/3072315057 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80780306e0 con 0x7f8074004800 2024-04-20T21:29:57.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.571+0000 7f808e000640 1 -- 172.21.15.86:0/3072315057 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8078030ab0 con 0x7f8074004800 2024-04-20T21:29:57.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.571+0000 7f808fe00640 1 -- 172.21.15.86:0/3072315057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8074004800 msgr2=0x7f8074004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:57.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.571+0000 7f808fe00640 1 --2- 172.21.15.86:0/3072315057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8074004800 0x7f8074004c80 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f807800bd30 tx=0x7f8074005830 comp rx=0 tx=0).stop 2024-04-20T21:29:57.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.571+0000 7f808f400640 1 -- 172.21.15.86:0/3072315057 reap_dead start 2024-04-20T21:29:57.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.571+0000 7f808fe00640 1 -- 172.21.15.86:0/3072315057 shutdown_connections 2024-04-20T21:29:57.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.571+0000 7f808fe00640 1 -- 172.21.15.86:0/3072315057 >> 172.21.15.86:0/3072315057 conn(0x7f80900b1f00 msgr2=0x7f80900b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:57.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.571+0000 7f808fe00640 1 -- 172.21.15.86:0/3072315057 shutdown_connections 2024-04-20T21:29:57.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.571+0000 7f808fe00640 1 -- 172.21.15.86:0/3072315057 wait complete. 2024-04-20T21:29:57.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.573+0000 7f808fe00640 1 Processor -- start 2024-04-20T21:29:57.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.573+0000 7f808fe00640 1 -- start start 2024-04-20T21:29:57.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.574+0000 7f808fe00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8074004800 0x7f809015e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:57.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.574+0000 7f808fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f809009e090 0x7f8090159c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:57.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.574+0000 7f808fe00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f809009bed0 0x7f8090150de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:57.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.574+0000 7f808fe00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8090073570 con 0x7f809009e090 2024-04-20T21:29:57.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.574+0000 7f808fe00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f80900ccb00 con 0x7f8074004800 2024-04-20T21:29:57.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.574+0000 7f808fe00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8090072650 con 0x7f809009bed0 2024-04-20T21:29:57.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.574+0000 7f8086000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f809009e090 0x7f8090159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:57.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.574+0000 7f8086000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f809009e090 0x7f8090159c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38788/0 (socket says 172.21.15.86:38788) 2024-04-20T21:29:57.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.574+0000 7f8086000640 1 -- 172.21.15.86:0/2566365457 learned_addr learned my addr 172.21.15.86:0/2566365457 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:57.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.575+0000 7f808ea00640 1 --2- 172.21.15.86:0/2566365457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8074004800 0x7f809015e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:57.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.575+0000 7f808f400640 1 --2- 172.21.15.86:0/2566365457 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f809009bed0 0x7f8090150de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:57.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.575+0000 7f8086000640 1 -- 172.21.15.86:0/2566365457 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f809009bed0 msgr2=0x7f8090150de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:57.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.575+0000 7f8086000640 1 --2- 172.21.15.86:0/2566365457 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f809009bed0 0x7f8090150de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:57.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.575+0000 7f8086000640 1 -- 172.21.15.86:0/2566365457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8074004800 msgr2=0x7f809015e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:57.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.575+0000 7f8086000640 1 --2- 172.21.15.86:0/2566365457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8074004800 0x7f809015e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:57.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.575+0000 7f8086000640 1 -- 172.21.15.86:0/2566365457 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f809015d420 con 0x7f809009e090 2024-04-20T21:29:57.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.575+0000 7f808f400640 1 --2- 172.21.15.86:0/2566365457 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f809009bed0 0x7f8090150de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:29:57.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.575+0000 7f8086000640 1 --2- 172.21.15.86:0/2566365457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f809009e090 0x7f8090159c20 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f807800be50 tx=0x7f8078004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:57.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.575+0000 7f8087400640 1 -- 172.21.15.86:0/2566365457 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8078047020 con 0x7f809009e090 2024-04-20T21:29:57.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.575+0000 7f8087400640 1 -- 172.21.15.86:0/2566365457 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80780044c0 con 0x7f809009e090 2024-04-20T21:29:57.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.575+0000 7f8087400640 1 -- 172.21.15.86:0/2566365457 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807800b930 con 0x7f809009e090 2024-04-20T21:29:57.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.575+0000 7f808fe00640 1 -- 172.21.15.86:0/2566365457 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f809015d700 con 0x7f809009e090 2024-04-20T21:29:57.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.576+0000 7f808fe00640 1 -- 172.21.15.86:0/2566365457 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f809015dcc0 con 0x7f809009e090 2024-04-20T21:29:57.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.577+0000 7f808fe00640 1 -- 172.21.15.86:0/2566365457 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f809004fd90 con 0x7f809009e090 2024-04-20T21:29:57.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.578+0000 7f8087400640 1 -- 172.21.15.86:0/2566365457 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f807804a020 con 0x7f809009e090 2024-04-20T21:29:57.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.579+0000 7f8087400640 1 --2- 172.21.15.86:0/2566365457 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f806c06c920 0x7f806c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:57.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.579+0000 7f8087400640 1 -- 172.21.15.86:0/2566365457 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7f80780b1110 con 0x7f809009e090 2024-04-20T21:29:57.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.579+0000 7f808ea00640 1 --2- 172.21.15.86:0/2566365457 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f806c06c920 0x7f806c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:57.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.580+0000 7f808ea00640 1 --2- 172.21.15.86:0/2566365457 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f806c06c920 0x7f806c06ede0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f8080001050 tx=0x7f8080009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:57.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.583+0000 7f8087400640 1 -- 172.21.15.86:0/2566365457 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f807807f530 con 0x7f809009e090 2024-04-20T21:29:57.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.784+0000 7f808fe00640 1 -- 172.21.15.86:0/2566365457 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f809015fbf0 con 0x7f809009e090 2024-04-20T21:29:57.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.785+0000 7f8087400640 1 -- 172.21.15.86:0/2566365457 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) ==== 93+0+10 (secure 0 0 0) 0x7f807807ee30 con 0x7f809009e090 2024-04-20T21:29:57.784 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:29:57.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.788+0000 7f808fe00640 1 -- 172.21.15.86:0/2566365457 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f806c06c920 msgr2=0x7f806c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:57.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.788+0000 7f808fe00640 1 --2- 172.21.15.86:0/2566365457 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f806c06c920 0x7f806c06ede0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f8080001050 tx=0x7f8080009040 comp rx=0 tx=0).stop 2024-04-20T21:29:57.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.788+0000 7f808fe00640 1 -- 172.21.15.86:0/2566365457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f809009e090 msgr2=0x7f8090159c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:57.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.788+0000 7f808fe00640 1 --2- 172.21.15.86:0/2566365457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f809009e090 0x7f8090159c20 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f807800be50 tx=0x7f8078004730 comp rx=0 tx=0).stop 2024-04-20T21:29:57.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.788+0000 7f808fe00640 1 -- 172.21.15.86:0/2566365457 shutdown_connections 2024-04-20T21:29:57.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.788+0000 7f808fe00640 1 --2- 172.21.15.86:0/2566365457 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f806c06c920 0x7f806c06ede0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:57.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.788+0000 7f808fe00640 1 --2- 172.21.15.86:0/2566365457 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f809009bed0 0x7f8090150de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:57.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.788+0000 7f808fe00640 1 --2- 172.21.15.86:0/2566365457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f809009e090 0x7f8090159c20 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:57.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.788+0000 7f808fe00640 1 --2- 172.21.15.86:0/2566365457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8074004800 0x7f809015e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:57.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.788+0000 7f808fe00640 1 -- 172.21.15.86:0/2566365457 >> 172.21.15.86:0/2566365457 conn(0x7f80900b1f00 msgr2=0x7f80900b07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:57.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.788+0000 7f808fe00640 1 -- 172.21.15.86:0/2566365457 shutdown_connections 2024-04-20T21:29:57.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:57.788+0000 7f808fe00640 1 -- 172.21.15.86:0/2566365457 wait complete. 2024-04-20T21:29:57.806 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:29:58.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.002+0000 7fd34c000640 1 Processor -- start 2024-04-20T21:29:58.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.002+0000 7fd34c000640 1 -- start start 2024-04-20T21:29:58.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.004+0000 7fd34c000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd34406bbf0 0x7fd34406e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:58.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.004+0000 7fd34c000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd34406bfd0 0x7fd344145b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:58.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.004+0000 7fd34c000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd34415cd40 con 0x7fd344146040 2024-04-20T21:29:58.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.004+0000 7fd34c000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd34415d000 con 0x7fd34406bfd0 2024-04-20T21:29:58.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.004+0000 7fd34c000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd34415d2e0 con 0x7fd34406bbf0 2024-04-20T21:29:58.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.004+0000 7fd348c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd34406bfd0 0x7fd344145b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:58.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.004+0000 7fd34a000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd344146040 0x7fd3440cd540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39328/0 (socket says 172.21.15.86:39328) 2024-04-20T21:29:58.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.004+0000 7fd34a000640 1 -- 172.21.15.86:0/3224237539 learned_addr learned my addr 172.21.15.86:0/3224237539 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:29:58.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.005+0000 7fd348c00640 1 -- 172.21.15.86:0/3224237539 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd34406bbf0 msgr2=0x7fd34406e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:58.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.005+0000 7fd349600640 1 --2- 172.21.15.86:0/3224237539 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd34406bbf0 0x7fd34406e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:58.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.005+0000 7fd343e00640 1 -- 172.21.15.86:0/3224237539 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2962205441 0 0) 0x7fd34415cd40 con 0x7fd344146040 2024-04-20T21:29:58.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.005+0000 7fd348c00640 1 --2- 172.21.15.86:0/3224237539 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd34406bbf0 0x7fd34406e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:58.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.005+0000 7fd348c00640 1 -- 172.21.15.86:0/3224237539 >> v1:172.21.15.86:6789/0 conn(0x7fd344146040 legacy=0x7fd3440cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:58.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.005+0000 7fd348c00640 1 -- 172.21.15.86:0/3224237539 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd34415d5c0 con 0x7fd34406bfd0 2024-04-20T21:29:58.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.005+0000 7fd349600640 1 --2- 172.21.15.86:0/3224237539 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd34406bbf0 0x7fd34406e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:29:58.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.006+0000 7fd348c00640 1 --2- 172.21.15.86:0/3224237539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd34406bfd0 0x7fd344145b00 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7fd3300097a0 tx=0x7fd33002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6004ac5d258f149e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:58.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.006+0000 7fd343e00640 1 -- 172.21.15.86:0/3224237539 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd330035020 con 0x7fd34406bfd0 2024-04-20T21:29:58.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.006+0000 7fd343e00640 1 -- 172.21.15.86:0/3224237539 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd330031810 con 0x7fd34406bfd0 2024-04-20T21:29:58.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.006+0000 7fd343e00640 1 -- 172.21.15.86:0/3224237539 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd330031be0 con 0x7fd34406bfd0 2024-04-20T21:29:58.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.007+0000 7fd34c000640 1 -- 172.21.15.86:0/3224237539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd34406bfd0 msgr2=0x7fd344145b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:58.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.007+0000 7fd34c000640 1 --2- 172.21.15.86:0/3224237539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd34406bfd0 0x7fd344145b00 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7fd3300097a0 tx=0x7fd33002b8b0 comp rx=0 tx=0).stop 2024-04-20T21:29:58.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.007+0000 7fd34c000640 1 -- 172.21.15.86:0/3224237539 shutdown_connections 2024-04-20T21:29:58.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.007+0000 7fd34c000640 1 --2- 172.21.15.86:0/3224237539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd34406bfd0 0x7fd344145b00 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:58.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.007+0000 7fd34c000640 1 --2- 172.21.15.86:0/3224237539 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd34406bbf0 0x7fd34406e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:58.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.007+0000 7fd34c000640 1 -- 172.21.15.86:0/3224237539 >> 172.21.15.86:0/3224237539 conn(0x7fd3440b2110 msgr2=0x7fd3440b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:58.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.007+0000 7fd34c000640 1 -- 172.21.15.86:0/3224237539 shutdown_connections 2024-04-20T21:29:58.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.007+0000 7fd34c000640 1 -- 172.21.15.86:0/3224237539 wait complete. 2024-04-20T21:29:58.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.009+0000 7fd34c000640 1 Processor -- start 2024-04-20T21:29:58.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.009+0000 7fd34c000640 1 -- start start 2024-04-20T21:29:58.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.010+0000 7fd34c000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd34406bbf0 0x7fd34415c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:58.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.010+0000 7fd34c000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd34406bfd0 0x7fd3441e5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:58.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.010+0000 7fd34c000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd344146040 0x7fd3440cb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:58.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.010+0000 7fd34c000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd3440cddd0 con 0x7fd34406bbf0 2024-04-20T21:29:58.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.010+0000 7fd34c000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd3440cdc50 con 0x7fd344146040 2024-04-20T21:29:58.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.010+0000 7fd34c000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd3440cdf50 con 0x7fd34406bfd0 2024-04-20T21:29:58.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.010+0000 7fd349600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd34406bbf0 0x7fd34415c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:58.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.010+0000 7fd349600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd34406bbf0 0x7fd34415c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38802/0 (socket says 172.21.15.86:38802) 2024-04-20T21:29:58.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.010+0000 7fd349600640 1 -- 172.21.15.86:0/478029960 learned_addr learned my addr 172.21.15.86:0/478029960 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:58.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.011+0000 7fd349600640 1 -- 172.21.15.86:0/478029960 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd34406bfd0 msgr2=0x7fd3441e5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:58.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.011+0000 7fd349600640 1 --2- 172.21.15.86:0/478029960 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd34406bfd0 0x7fd3441e5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:58.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.011+0000 7fd349600640 1 -- 172.21.15.86:0/478029960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd344146040 msgr2=0x7fd3440cb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:58.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.011+0000 7fd349600640 1 --2- 172.21.15.86:0/478029960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd344146040 0x7fd3440cb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:58.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.011+0000 7fd349600640 1 -- 172.21.15.86:0/478029960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3441f5f20 con 0x7fd34406bbf0 2024-04-20T21:29:58.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.011+0000 7fd349600640 1 --2- 172.21.15.86:0/478029960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd34406bbf0 0x7fd34415c2a0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7fd33800c820 tx=0x7fd33800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:58.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.011+0000 7fd341600640 1 -- 172.21.15.86:0/478029960 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd338015070 con 0x7fd34406bbf0 2024-04-20T21:29:58.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.011+0000 7fd341600640 1 -- 172.21.15.86:0/478029960 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd338007a30 con 0x7fd34406bbf0 2024-04-20T21:29:58.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.011+0000 7fd34c000640 1 -- 172.21.15.86:0/478029960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3441f6200 con 0x7fd34406bbf0 2024-04-20T21:29:58.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.011+0000 7fd341600640 1 -- 172.21.15.86:0/478029960 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd338007e00 con 0x7fd34406bbf0 2024-04-20T21:29:58.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.011+0000 7fd34c000640 1 -- 172.21.15.86:0/478029960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3441f67c0 con 0x7fd34406bbf0 2024-04-20T21:29:58.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.013+0000 7fd34c000640 1 -- 172.21.15.86:0/478029960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3440689e0 con 0x7fd34406bbf0 2024-04-20T21:29:58.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.014+0000 7fd341600640 1 -- 172.21.15.86:0/478029960 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd338018070 con 0x7fd34406bbf0 2024-04-20T21:29:58.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.015+0000 7fd341600640 1 --2- 172.21.15.86:0/478029960 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd32406c8f0 0x7fd32406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:58.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.015+0000 7fd341600640 1 -- 172.21.15.86:0/478029960 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7fd33808bb30 con 0x7fd34406bbf0 2024-04-20T21:29:58.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.015+0000 7fd348c00640 1 --2- 172.21.15.86:0/478029960 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd32406c8f0 0x7fd32406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:58.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.016+0000 7fd348c00640 1 --2- 172.21.15.86:0/478029960 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd32406c8f0 0x7fd32406edb0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fd330036000 tx=0x7fd330030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:58.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.020+0000 7fd341600640 1 -- 172.21.15.86:0/478029960 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd338059f50 con 0x7fd34406bbf0 2024-04-20T21:29:58.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.218+0000 7fd34c000640 1 -- 172.21.15.86:0/478029960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd3440cf3e0 con 0x7fd34406bbf0 2024-04-20T21:29:58.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.219+0000 7fd341600640 1 -- 172.21.15.86:0/478029960 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v112) ==== 104+0+11 (secure 0 0 0) 0x7fd338059850 con 0x7fd34406bbf0 2024-04-20T21:29:58.218 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-20T21:29:58.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.221+0000 7fd34c000640 1 -- 172.21.15.86:0/478029960 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd32406c8f0 msgr2=0x7fd32406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:58.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.221+0000 7fd34c000640 1 --2- 172.21.15.86:0/478029960 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd32406c8f0 0x7fd32406edb0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fd330036000 tx=0x7fd330030000 comp rx=0 tx=0).stop 2024-04-20T21:29:58.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.221+0000 7fd34c000640 1 -- 172.21.15.86:0/478029960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd34406bbf0 msgr2=0x7fd34415c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:58.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.221+0000 7fd34c000640 1 --2- 172.21.15.86:0/478029960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd34406bbf0 0x7fd34415c2a0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7fd33800c820 tx=0x7fd33800cc20 comp rx=0 tx=0).stop 2024-04-20T21:29:58.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.222+0000 7fd34c000640 1 -- 172.21.15.86:0/478029960 shutdown_connections 2024-04-20T21:29:58.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.222+0000 7fd34c000640 1 --2- 172.21.15.86:0/478029960 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd32406c8f0 0x7fd32406edb0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:58.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.222+0000 7fd34c000640 1 --2- 172.21.15.86:0/478029960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd344146040 0x7fd3440cb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:58.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.222+0000 7fd34c000640 1 --2- 172.21.15.86:0/478029960 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd34406bfd0 0x7fd3441e5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:58.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.222+0000 7fd34c000640 1 --2- 172.21.15.86:0/478029960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd34406bbf0 0x7fd34415c2a0 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:58.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.222+0000 7fd34c000640 1 -- 172.21.15.86:0/478029960 >> 172.21.15.86:0/478029960 conn(0x7fd3440b2110 msgr2=0x7fd3440b3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:58.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.222+0000 7fd34c000640 1 -- 172.21.15.86:0/478029960 shutdown_connections 2024-04-20T21:29:58.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.222+0000 7fd34c000640 1 -- 172.21.15.86:0/478029960 wait complete. 2024-04-20T21:29:58.239 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:29:58.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.446+0000 7fc899c00640 1 Processor -- start 2024-04-20T21:29:58.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.447+0000 7fc899c00640 1 -- start start 2024-04-20T21:29:58.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.448+0000 7fc899c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc894159360 0x7fc8940b3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:58.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.448+0000 7fc899c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc8940cca40 0x7fc8940cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:58.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.448+0000 7fc899c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8940ccfe0 con 0x7fc8940d0150 2024-04-20T21:29:58.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.448+0000 7fc899c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc8940cd2c0 con 0x7fc894159360 2024-04-20T21:29:58.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.448+0000 7fc899c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fc8940cf130 con 0x7fc8940cca40 2024-04-20T21:29:58.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.448+0000 7fc892a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc894159360 0x7fc8940b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:58.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.448+0000 7fc892a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc894159360 0x7fc8940b3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38816/0 (socket says 172.21.15.86:38816) 2024-04-20T21:29:58.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.449+0000 7fc892a00640 1 -- 172.21.15.86:0/2564024633 learned_addr learned my addr 172.21.15.86:0/2564024633 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:58.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.449+0000 7fc892000640 1 --2- 172.21.15.86:0/2564024633 >> v2:172.21.15.144:3300/0 conn(0x7fc8940cca40 0x7fc8940cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:58.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.449+0000 7fc892a00640 1 -- 172.21.15.86:0/2564024633 >> v2:172.21.15.144:3300/0 conn(0x7fc8940cca40 msgr2=0x7fc8940cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:58.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.449+0000 7fc892a00640 1 --2- 172.21.15.86:0/2564024633 >> v2:172.21.15.144:3300/0 conn(0x7fc8940cca40 0x7fc8940cfc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:58.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.449+0000 7fc892a00640 1 -- 172.21.15.86:0/2564024633 >> v1:172.21.15.144:6789/0 conn(0x7fc8940d0150 legacy=0x7fc89415f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:58.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.449+0000 7fc892a00640 1 -- 172.21.15.86:0/2564024633 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8940cf410 con 0x7fc894159360 2024-04-20T21:29:58.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.450+0000 7fc892a00640 1 --2- 172.21.15.86:0/2564024633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc894159360 0x7fc8940b3830 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7fc8940d25c0 tx=0x7fc88402e0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfebed1d348eca9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:58.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.450+0000 7fc891600640 1 -- 172.21.15.86:0/2564024633 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88402ea90 con 0x7fc894159360 2024-04-20T21:29:58.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.451+0000 7fc891600640 1 -- 172.21.15.86:0/2564024633 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc88402ec30 con 0x7fc894159360 2024-04-20T21:29:58.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.451+0000 7fc891600640 1 -- 172.21.15.86:0/2564024633 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc884009600 con 0x7fc894159360 2024-04-20T21:29:58.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.452+0000 7fc899c00640 1 -- 172.21.15.86:0/2564024633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc894159360 msgr2=0x7fc8940b3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:58.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.452+0000 7fc899c00640 1 --2- 172.21.15.86:0/2564024633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc894159360 0x7fc8940b3830 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7fc8940d25c0 tx=0x7fc88402e0f0 comp rx=0 tx=0).stop 2024-04-20T21:29:58.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.452+0000 7fc899c00640 1 -- 172.21.15.86:0/2564024633 shutdown_connections 2024-04-20T21:29:58.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.452+0000 7fc899c00640 1 --2- 172.21.15.86:0/2564024633 >> v2:172.21.15.144:3300/0 conn(0x7fc8940cca40 0x7fc8940cfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:58.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.452+0000 7fc899c00640 1 --2- 172.21.15.86:0/2564024633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc894159360 0x7fc8940b3830 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:58.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.452+0000 7fc899c00640 1 -- 172.21.15.86:0/2564024633 >> 172.21.15.86:0/2564024633 conn(0x7fc89409a9e0 msgr2=0x7fc8940b2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:58.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.452+0000 7fc899c00640 1 -- 172.21.15.86:0/2564024633 shutdown_connections 2024-04-20T21:29:58.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.452+0000 7fc899c00640 1 -- 172.21.15.86:0/2564024633 wait complete. 2024-04-20T21:29:58.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.454+0000 7fc899c00640 1 Processor -- start 2024-04-20T21:29:58.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.454+0000 7fc899c00640 1 -- start start 2024-04-20T21:29:58.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.455+0000 7fc899c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8940cca40 0x7fc894152a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:58.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.455+0000 7fc899c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8940d0150 0x7fc89414ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:58.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.456+0000 7fc899c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc894159360 0x7fc8940c8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:58.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.456+0000 7fc899c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc89415f980 con 0x7fc894159360 2024-04-20T21:29:58.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.456+0000 7fc899c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc8940b3d70 con 0x7fc8940d0150 2024-04-20T21:29:58.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.456+0000 7fc899c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc89415fb00 con 0x7fc8940cca40 2024-04-20T21:29:58.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.456+0000 7fc893400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc894159360 0x7fc8940c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:58.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.456+0000 7fc893400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc894159360 0x7fc8940c8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38824/0 (socket says 172.21.15.86:38824) 2024-04-20T21:29:58.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.456+0000 7fc893400640 1 -- 172.21.15.86:0/3055345647 learned_addr learned my addr 172.21.15.86:0/3055345647 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:58.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.456+0000 7fc892a00640 1 --2- 172.21.15.86:0/3055345647 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8940cca40 0x7fc894152a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:58.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.456+0000 7fc892000640 1 --2- 172.21.15.86:0/3055345647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8940d0150 0x7fc89414ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:58.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.456+0000 7fc893400640 1 -- 172.21.15.86:0/3055345647 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8940cca40 msgr2=0x7fc894152a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:58.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.456+0000 7fc893400640 1 --2- 172.21.15.86:0/3055345647 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8940cca40 0x7fc894152a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:58.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.456+0000 7fc893400640 1 -- 172.21.15.86:0/3055345647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8940d0150 msgr2=0x7fc89414ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:58.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.456+0000 7fc893400640 1 --2- 172.21.15.86:0/3055345647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8940d0150 0x7fc89414ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:58.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.456+0000 7fc893400640 1 -- 172.21.15.86:0/3055345647 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8940c8ad0 con 0x7fc894159360 2024-04-20T21:29:58.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.457+0000 7fc893400640 1 --2- 172.21.15.86:0/3055345647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc894159360 0x7fc8940c8590 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7fc88c00b410 tx=0x7fc88c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:58.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.457+0000 7fc88aa00640 1 -- 172.21.15.86:0/3055345647 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88c015030 con 0x7fc894159360 2024-04-20T21:29:58.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.457+0000 7fc88aa00640 1 -- 172.21.15.86:0/3055345647 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc88c004020 con 0x7fc894159360 2024-04-20T21:29:58.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.457+0000 7fc88aa00640 1 -- 172.21.15.86:0/3055345647 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88c004410 con 0x7fc894159360 2024-04-20T21:29:58.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.457+0000 7fc899c00640 1 -- 172.21.15.86:0/3055345647 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc89415b3b0 con 0x7fc894159360 2024-04-20T21:29:58.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.457+0000 7fc899c00640 1 -- 172.21.15.86:0/3055345647 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc894156880 con 0x7fc894159360 2024-04-20T21:29:58.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.458+0000 7fc899c00640 1 -- 172.21.15.86:0/3055345647 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8940690f0 con 0x7fc894159360 2024-04-20T21:29:58.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.460+0000 7fc88aa00640 1 -- 172.21.15.86:0/3055345647 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc88c018070 con 0x7fc894159360 2024-04-20T21:29:58.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.461+0000 7fc88aa00640 1 --2- 172.21.15.86:0/3055345647 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc87006c8f0 0x7fc87006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:58.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.461+0000 7fc88aa00640 1 -- 172.21.15.86:0/3055345647 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7fc88c08b690 con 0x7fc894159360 2024-04-20T21:29:58.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.461+0000 7fc892a00640 1 --2- 172.21.15.86:0/3055345647 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc87006c8f0 0x7fc87006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:58.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.462+0000 7fc892a00640 1 --2- 172.21.15.86:0/3055345647 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc87006c8f0 0x7fc87006edb0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fc884037000 tx=0x7fc884032000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:58.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.465+0000 7fc88aa00640 1 -- 172.21.15.86:0/3055345647 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc88c059a20 con 0x7fc894159360 2024-04-20T21:29:58.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.666+0000 7fc899c00640 1 -- 172.21.15.86:0/3055345647 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc89415c650 con 0x7fc894159360 2024-04-20T21:29:58.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.666+0000 7fc88aa00640 1 -- 172.21.15.86:0/3055345647 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v112) ==== 100+0+11 (secure 0 0 0) 0x7fc88c059320 con 0x7fc894159360 2024-04-20T21:29:58.665 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-20T21:29:58.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.670+0000 7fc899c00640 1 -- 172.21.15.86:0/3055345647 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc87006c8f0 msgr2=0x7fc87006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:58.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.670+0000 7fc899c00640 1 --2- 172.21.15.86:0/3055345647 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc87006c8f0 0x7fc87006edb0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fc884037000 tx=0x7fc884032000 comp rx=0 tx=0).stop 2024-04-20T21:29:58.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.670+0000 7fc899c00640 1 -- 172.21.15.86:0/3055345647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc894159360 msgr2=0x7fc8940c8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:58.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.670+0000 7fc899c00640 1 --2- 172.21.15.86:0/3055345647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc894159360 0x7fc8940c8590 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7fc88c00b410 tx=0x7fc88c00b8d0 comp rx=0 tx=0).stop 2024-04-20T21:29:58.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.670+0000 7fc899c00640 1 -- 172.21.15.86:0/3055345647 shutdown_connections 2024-04-20T21:29:58.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.670+0000 7fc899c00640 1 --2- 172.21.15.86:0/3055345647 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc87006c8f0 0x7fc87006edb0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:58.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.670+0000 7fc899c00640 1 --2- 172.21.15.86:0/3055345647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc894159360 0x7fc8940c8590 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:58.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.670+0000 7fc899c00640 1 --2- 172.21.15.86:0/3055345647 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc8940d0150 0x7fc89414ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:58.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.670+0000 7fc899c00640 1 --2- 172.21.15.86:0/3055345647 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc8940cca40 0x7fc894152a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:58.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.670+0000 7fc899c00640 1 -- 172.21.15.86:0/3055345647 >> 172.21.15.86:0/3055345647 conn(0x7fc89409a9e0 msgr2=0x7fc8940749f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:58.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.670+0000 7fc899c00640 1 -- 172.21.15.86:0/3055345647 shutdown_connections 2024-04-20T21:29:58.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.670+0000 7fc899c00640 1 -- 172.21.15.86:0/3055345647 wait complete. 2024-04-20T21:29:58.688 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T21:29:58.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.889+0000 7f8f6da00640 1 Processor -- start 2024-04-20T21:29:58.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.889+0000 7f8f6da00640 1 -- start start 2024-04-20T21:29:58.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.890+0000 7f8f6da00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8f6815b5f0 0x7f8f680b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:58.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.890+0000 7f8f6da00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8f6806f070 0x7f8f680b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:58.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.890+0000 7f8f6da00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f6815b9d0 con 0x7f8f680cff80 2024-04-20T21:29:58.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.890+0000 7f8f6da00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8f680b4650 con 0x7f8f6815b5f0 2024-04-20T21:29:58.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.890+0000 7f8f6da00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f8f680af850 con 0x7f8f6806f070 2024-04-20T21:29:58.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.891+0000 7f8f66a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8f6815b5f0 0x7f8f680b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:58.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.891+0000 7f8f66a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8f6815b5f0 0x7f8f680b3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38832/0 (socket says 172.21.15.86:38832) 2024-04-20T21:29:58.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.891+0000 7f8f66a00640 1 -- 172.21.15.86:0/582771121 learned_addr learned my addr 172.21.15.86:0/582771121 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:58.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.891+0000 7f8f67400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f8f680cff80 0x7f8f680ce130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39332/0 (socket says 172.21.15.86:39332) 2024-04-20T21:29:58.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.891+0000 7f8f66000640 1 --2- 172.21.15.86:0/582771121 >> v2:172.21.15.144:3300/0 conn(0x7f8f6806f070 0x7f8f680b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:58.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.891+0000 7f8f66a00640 1 -- 172.21.15.86:0/582771121 >> v2:172.21.15.144:3300/0 conn(0x7f8f6806f070 msgr2=0x7f8f680b50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:29:58.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.891+0000 7f8f66a00640 1 --2- 172.21.15.86:0/582771121 >> v2:172.21.15.144:3300/0 conn(0x7f8f6806f070 0x7f8f680b50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:58.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.891+0000 7f8f66a00640 1 -- 172.21.15.86:0/582771121 >> v1:172.21.15.86:6789/0 conn(0x7f8f680cff80 legacy=0x7f8f680ce130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:58.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.891+0000 7f8f66a00640 1 -- 172.21.15.86:0/582771121 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f680afb30 con 0x7f8f6815b5f0 2024-04-20T21:29:58.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.891+0000 7f8f65600640 1 -- 172.21.15.86:0/582771121 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1332726023 0 0) 0x7f8f6815b9d0 con 0x7f8f680cff80 2024-04-20T21:29:58.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.892+0000 7f8f66a00640 1 --2- 172.21.15.86:0/582771121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f6815b5f0 0x7f8f680b3d40 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f8f6806ffe0 tx=0x7f8f5802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ae92b5b58982145 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:58.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.893+0000 7f8f65600640 1 -- 172.21.15.86:0/582771121 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f58034070 con 0x7f8f6815b5f0 2024-04-20T21:29:58.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.893+0000 7f8f65600640 1 -- 172.21.15.86:0/582771121 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f58030860 con 0x7f8f6815b5f0 2024-04-20T21:29:58.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.893+0000 7f8f65600640 1 -- 172.21.15.86:0/582771121 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f58030c30 con 0x7f8f6815b5f0 2024-04-20T21:29:58.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.893+0000 7f8f6da00640 1 -- 172.21.15.86:0/582771121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f6815b5f0 msgr2=0x7f8f680b3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:58.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.893+0000 7f8f6da00640 1 --2- 172.21.15.86:0/582771121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f6815b5f0 0x7f8f680b3d40 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f8f6806ffe0 tx=0x7f8f5802b9a0 comp rx=0 tx=0).stop 2024-04-20T21:29:58.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.893+0000 7f8f6da00640 1 -- 172.21.15.86:0/582771121 shutdown_connections 2024-04-20T21:29:58.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.893+0000 7f8f6da00640 1 --2- 172.21.15.86:0/582771121 >> v2:172.21.15.144:3300/0 conn(0x7f8f6806f070 0x7f8f680b50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:58.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.893+0000 7f8f6da00640 1 --2- 172.21.15.86:0/582771121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f6815b5f0 0x7f8f680b3d40 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:58.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.893+0000 7f8f6da00640 1 -- 172.21.15.86:0/582771121 >> 172.21.15.86:0/582771121 conn(0x7f8f6806b980 msgr2=0x7f8f6806bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:58.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.893+0000 7f8f6da00640 1 -- 172.21.15.86:0/582771121 shutdown_connections 2024-04-20T21:29:58.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.893+0000 7f8f6da00640 1 -- 172.21.15.86:0/582771121 wait complete. 2024-04-20T21:29:58.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.895+0000 7f8f6da00640 1 Processor -- start 2024-04-20T21:29:58.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.895+0000 7f8f6da00640 1 -- start start 2024-04-20T21:29:58.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.896+0000 7f8f6da00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8f6806f070 0x7f8f68144ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:58.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.896+0000 7f8f6da00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f680cff80 0x7f8f68145b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:58.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.896+0000 7f8f6da00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f6815b5f0 0x7f8f6815cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:58.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.896+0000 7f8f6da00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8f680b55e0 con 0x7f8f680cff80 2024-04-20T21:29:58.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.896+0000 7f8f6da00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8f680b4b80 con 0x7f8f6815b5f0 2024-04-20T21:29:58.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.896+0000 7f8f6da00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8f680d03d0 con 0x7f8f6806f070 2024-04-20T21:29:58.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.896+0000 7f8f66000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f680cff80 0x7f8f68145b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:58.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.896+0000 7f8f66000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f680cff80 0x7f8f68145b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38834/0 (socket says 172.21.15.86:38834) 2024-04-20T21:29:58.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.896+0000 7f8f66000640 1 -- 172.21.15.86:0/2347441174 learned_addr learned my addr 172.21.15.86:0/2347441174 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:29:58.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.897+0000 7f8f67400640 1 --2- 172.21.15.86:0/2347441174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f6815b5f0 0x7f8f6815cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:58.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.897+0000 7f8f66a00640 1 --2- 172.21.15.86:0/2347441174 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8f6806f070 0x7f8f68144ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:58.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.897+0000 7f8f66000640 1 -- 172.21.15.86:0/2347441174 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8f6806f070 msgr2=0x7f8f68144ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:58.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.897+0000 7f8f66000640 1 --2- 172.21.15.86:0/2347441174 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8f6806f070 0x7f8f68144ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:58.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.897+0000 7f8f66000640 1 -- 172.21.15.86:0/2347441174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f6815b5f0 msgr2=0x7f8f6815cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:58.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.897+0000 7f8f66000640 1 --2- 172.21.15.86:0/2347441174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f6815b5f0 0x7f8f6815cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:58.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.897+0000 7f8f66000640 1 -- 172.21.15.86:0/2347441174 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f6815bf10 con 0x7f8f680cff80 2024-04-20T21:29:58.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.897+0000 7f8f67400640 1 --2- 172.21.15.86:0/2347441174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f6815b5f0 0x7f8f6815cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:29:58.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.897+0000 7f8f66000640 1 --2- 172.21.15.86:0/2347441174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f680cff80 0x7f8f68145b00 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f8f5400c7f0 tx=0x7f8f5400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:58.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.897+0000 7f8f5ea00640 1 -- 172.21.15.86:0/2347441174 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f54015070 con 0x7f8f680cff80 2024-04-20T21:29:58.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.897+0000 7f8f5ea00640 1 -- 172.21.15.86:0/2347441174 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f54004240 con 0x7f8f680cff80 2024-04-20T21:29:58.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.897+0000 7f8f5ea00640 1 -- 172.21.15.86:0/2347441174 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f54007620 con 0x7f8f680cff80 2024-04-20T21:29:58.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.897+0000 7f8f6da00640 1 -- 172.21.15.86:0/2347441174 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f68146660 con 0x7f8f680cff80 2024-04-20T21:29:58.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.898+0000 7f8f6da00640 1 -- 172.21.15.86:0/2347441174 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f6815c490 con 0x7f8f680cff80 2024-04-20T21:29:58.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.899+0000 7f8f6da00640 1 -- 172.21.15.86:0/2347441174 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f680689e0 con 0x7f8f680cff80 2024-04-20T21:29:58.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.900+0000 7f8f5ea00640 1 -- 172.21.15.86:0/2347441174 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8f540079d0 con 0x7f8f680cff80 2024-04-20T21:29:58.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.901+0000 7f8f5ea00640 1 --2- 172.21.15.86:0/2347441174 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8f4806c8f0 0x7f8f4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:29:58.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.901+0000 7f8f5ea00640 1 -- 172.21.15.86:0/2347441174 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7f8f5408bf70 con 0x7f8f680cff80 2024-04-20T21:29:58.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.901+0000 7f8f66a00640 1 --2- 172.21.15.86:0/2347441174 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8f4806c8f0 0x7f8f4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:29:58.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.902+0000 7f8f66a00640 1 --2- 172.21.15.86:0/2347441174 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8f4806c8f0 0x7f8f4806edb0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f8f58036000 tx=0x7f8f58008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:29:58.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:58.905+0000 7f8f5ea00640 1 -- 172.21.15.86:0/2347441174 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8f5405a390 con 0x7f8f680cff80 2024-04-20T21:29:59.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:59.119+0000 7f8f6da00640 1 -- 172.21.15.86:0/2347441174 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8f6814ef40 con 0x7f8f680cff80 2024-04-20T21:29:59.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:59.120+0000 7f8f5ea00640 1 -- 172.21.15.86:0/2347441174 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v112) ==== 80+0+0 (secure 0 0 0) 0x7f8f54059c90 con 0x7f8f680cff80 2024-04-20T21:29:59.119 INFO:teuthology.orchestra.run.smithi086.stderr:listed 0 entries 2024-04-20T21:29:59.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:59.122+0000 7f8f6da00640 1 -- 172.21.15.86:0/2347441174 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8f4806c8f0 msgr2=0x7f8f4806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:59.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:59.122+0000 7f8f6da00640 1 --2- 172.21.15.86:0/2347441174 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8f4806c8f0 0x7f8f4806edb0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f8f58036000 tx=0x7f8f58008030 comp rx=0 tx=0).stop 2024-04-20T21:29:59.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:59.122+0000 7f8f6da00640 1 -- 172.21.15.86:0/2347441174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f680cff80 msgr2=0x7f8f68145b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:29:59.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:59.122+0000 7f8f6da00640 1 --2- 172.21.15.86:0/2347441174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f680cff80 0x7f8f68145b00 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f8f5400c7f0 tx=0x7f8f5400ccb0 comp rx=0 tx=0).stop 2024-04-20T21:29:59.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:59.123+0000 7f8f6da00640 1 -- 172.21.15.86:0/2347441174 shutdown_connections 2024-04-20T21:29:59.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:59.123+0000 7f8f6da00640 1 --2- 172.21.15.86:0/2347441174 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8f4806c8f0 0x7f8f4806edb0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:59.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:59.123+0000 7f8f6da00640 1 --2- 172.21.15.86:0/2347441174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f6815b5f0 0x7f8f6815cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:59.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:59.123+0000 7f8f6da00640 1 --2- 172.21.15.86:0/2347441174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8f680cff80 0x7f8f68145b00 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:59.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:59.123+0000 7f8f6da00640 1 --2- 172.21.15.86:0/2347441174 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8f6806f070 0x7f8f68144ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:29:59.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:59.123+0000 7f8f6da00640 1 -- 172.21.15.86:0/2347441174 >> 172.21.15.86:0/2347441174 conn(0x7f8f6806b980 msgr2=0x7f8f68071cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:29:59.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:59.123+0000 7f8f6da00640 1 -- 172.21.15.86:0/2347441174 shutdown_connections 2024-04-20T21:29:59.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:29:59.123+0000 7f8f6da00640 1 -- 172.21.15.86:0/2347441174 wait complete. 2024-04-20T21:29:59.141 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi086.front.sepia.ceph.com... 2024-04-20T21:29:59.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:29:59.141 DEBUG:teuthology.orchestra.run.smithi086:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-20T21:29:59.210 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[64761]: fuse finished with error 0 and tester_r 0 2024-04-20T21:29:59.215 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:30:05.316 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-20T21:30:05.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:30:05.317 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-20T21:30:05.377 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-04-20T21:30:05.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.587+0000 7f285b400640 1 Processor -- start 2024-04-20T21:30:05.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.587+0000 7f285b400640 1 -- start start 2024-04-20T21:30:05.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.589+0000 7f285b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f285406bbf0 0x7f285406e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:05.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.589+0000 7f285b400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2854146210 0x7f28540cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:05.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.589+0000 7f285b400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f285415cd40 con 0x7f285406bfd0 2024-04-20T21:30:05.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.589+0000 7f285b400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f285415d000 con 0x7f285406bbf0 2024-04-20T21:30:05.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.589+0000 7f285b400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f285415d2e0 con 0x7f2854146210 2024-04-20T21:30:05.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.589+0000 7f2858a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f285406bbf0 0x7f285406e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:05.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.589+0000 7f284be00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f285406bfd0 0x7f2854145b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:39348/0 (socket says 172.21.15.86:39348) 2024-04-20T21:30:05.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.589+0000 7f284be00640 1 -- 172.21.15.86:0/2411531450 learned_addr learned my addr 172.21.15.86:0/2411531450 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:30:05.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.590+0000 7f2859400640 1 --2- 172.21.15.86:0/2411531450 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2854146210 0x7f28540cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:05.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.590+0000 7f2853e00640 1 -- 172.21.15.86:0/2411531450 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2433232653 0 0) 0x7f285415cd40 con 0x7f285406bfd0 2024-04-20T21:30:05.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.590+0000 7f2853e00640 1 -- 172.21.15.86:0/2411531450 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2838003520 con 0x7f285406bfd0 2024-04-20T21:30:05.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.590+0000 7f2858a00640 1 -- 172.21.15.86:0/2411531450 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2854146210 msgr2=0x7f28540cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:05.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.590+0000 7f2858a00640 1 --2- 172.21.15.86:0/2411531450 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2854146210 0x7f28540cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:05.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.590+0000 7f2858a00640 1 -- 172.21.15.86:0/2411531450 >> v1:172.21.15.86:6789/0 conn(0x7f285406bfd0 legacy=0x7f2854145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:05.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.590+0000 7f2858a00640 1 -- 172.21.15.86:0/2411531450 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f285415d5c0 con 0x7f285406bbf0 2024-04-20T21:30:05.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.590+0000 7f2859400640 1 --2- 172.21.15.86:0/2411531450 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2854146210 0x7f28540cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:30:05.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.591+0000 7f2858a00640 1 --2- 172.21.15.86:0/2411531450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f285406bbf0 0x7f285406e000 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f28440097a0 tx=0x7f284402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa493bddd2d1bb0a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:05.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.591+0000 7f2853e00640 1 -- 172.21.15.86:0/2411531450 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2844035020 con 0x7f285406bbf0 2024-04-20T21:30:05.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.591+0000 7f2853e00640 1 -- 172.21.15.86:0/2411531450 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2844031810 con 0x7f285406bbf0 2024-04-20T21:30:05.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.592+0000 7f2853e00640 1 -- 172.21.15.86:0/2411531450 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2844031be0 con 0x7f285406bbf0 2024-04-20T21:30:05.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.592+0000 7f285b400640 1 -- 172.21.15.86:0/2411531450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f285406bbf0 msgr2=0x7f285406e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:05.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.592+0000 7f285b400640 1 --2- 172.21.15.86:0/2411531450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f285406bbf0 0x7f285406e000 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f28440097a0 tx=0x7f284402b8b0 comp rx=0 tx=0).stop 2024-04-20T21:30:05.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.593+0000 7f285b400640 1 -- 172.21.15.86:0/2411531450 shutdown_connections 2024-04-20T21:30:05.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.593+0000 7f285b400640 1 --2- 172.21.15.86:0/2411531450 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f2854146210 0x7f28540cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:05.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.593+0000 7f285b400640 1 --2- 172.21.15.86:0/2411531450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f285406bbf0 0x7f285406e000 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:05.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.593+0000 7f285b400640 1 -- 172.21.15.86:0/2411531450 >> 172.21.15.86:0/2411531450 conn(0x7f28540b2110 msgr2=0x7f28540b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:05.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.593+0000 7f285b400640 1 -- 172.21.15.86:0/2411531450 shutdown_connections 2024-04-20T21:30:05.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.593+0000 7f285b400640 1 -- 172.21.15.86:0/2411531450 wait complete. 2024-04-20T21:30:05.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.595+0000 7f285b400640 1 Processor -- start 2024-04-20T21:30:05.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.595+0000 7f285b400640 1 -- start start 2024-04-20T21:30:05.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.596+0000 7f285b400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f285406bbf0 0x7f2854133e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:05.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.596+0000 7f285b400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f285406bfd0 0x7f2854131d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:05.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.596+0000 7f285b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2854146210 0x7f2854132250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:05.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.596+0000 7f285b400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f28540cdc00 con 0x7f2854146210 2024-04-20T21:30:05.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.596+0000 7f285b400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f28540cda80 con 0x7f285406bbf0 2024-04-20T21:30:05.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.596+0000 7f285b400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f28540cdd80 con 0x7f285406bfd0 2024-04-20T21:30:05.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.596+0000 7f2859400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2854146210 0x7f2854132250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:05.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.596+0000 7f2859400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2854146210 0x7f2854132250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38840/0 (socket says 172.21.15.86:38840) 2024-04-20T21:30:05.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.596+0000 7f2859400640 1 -- 172.21.15.86:0/3907758880 learned_addr learned my addr 172.21.15.86:0/3907758880 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:05.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.596+0000 7f2858a00640 1 --2- 172.21.15.86:0/3907758880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f285406bbf0 0x7f2854133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:05.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.596+0000 7f284be00640 1 --2- 172.21.15.86:0/3907758880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f285406bfd0 0x7f2854131d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:05.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.596+0000 7f2859400640 1 -- 172.21.15.86:0/3907758880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f285406bfd0 msgr2=0x7f2854131d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:05.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.596+0000 7f2859400640 1 --2- 172.21.15.86:0/3907758880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f285406bfd0 0x7f2854131d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:05.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.596+0000 7f2859400640 1 -- 172.21.15.86:0/3907758880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f285406bbf0 msgr2=0x7f2854133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:05.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.596+0000 7f2859400640 1 --2- 172.21.15.86:0/3907758880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f285406bbf0 0x7f2854133e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:05.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.597+0000 7f2859400640 1 -- 172.21.15.86:0/3907758880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2854132a80 con 0x7f2854146210 2024-04-20T21:30:05.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.597+0000 7f2859400640 1 --2- 172.21.15.86:0/3907758880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2854146210 0x7f2854132250 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f284c00cef0 tx=0x7f284c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:05.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.597+0000 7f2851600640 1 -- 172.21.15.86:0/3907758880 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284c01a070 con 0x7f2854146210 2024-04-20T21:30:05.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.597+0000 7f2851600640 1 -- 172.21.15.86:0/3907758880 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f284c00d860 con 0x7f2854146210 2024-04-20T21:30:05.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.597+0000 7f285b400640 1 -- 172.21.15.86:0/3907758880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2854132d60 con 0x7f2854146210 2024-04-20T21:30:05.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.597+0000 7f2851600640 1 -- 172.21.15.86:0/3907758880 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284c00dc50 con 0x7f2854146210 2024-04-20T21:30:05.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.597+0000 7f285b400640 1 -- 172.21.15.86:0/3907758880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2854133600 con 0x7f2854146210 2024-04-20T21:30:05.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.599+0000 7f2851600640 1 -- 172.21.15.86:0/3907758880 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f284c01d070 con 0x7f2854146210 2024-04-20T21:30:05.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.600+0000 7f2851600640 1 --2- 172.21.15.86:0/3907758880 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f283006c8f0 0x7f283006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:05.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.600+0000 7f2851600640 1 -- 172.21.15.86:0/3907758880 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7f284c08c430 con 0x7f2854146210 2024-04-20T21:30:05.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.601+0000 7f2858a00640 1 --2- 172.21.15.86:0/3907758880 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f283006c8f0 0x7f283006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:05.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.601+0000 7f285e060740 1 Processor -- start 2024-04-20T21:30:05.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.601+0000 7f2858a00640 1 --2- 172.21.15.86:0/3907758880 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f283006c8f0 0x7f283006edb0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f2844036000 tx=0x7f2844008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:05.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.601+0000 7f285e060740 1 -- start start 2024-04-20T21:30:05.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.602+0000 7f285e060740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea9cc52860 0x55ea9cc5efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:05.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.602+0000 7f285e060740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea9cc52c40 0x55ea9cc16320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:05.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.602+0000 7f285e060740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea9cc16860 0x55ea9cc16ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:05.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.602+0000 7f285e060740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ea9cb7d150 con 0x55ea9cc16860 2024-04-20T21:30:05.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.602+0000 7f285e060740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ea9cb23370 con 0x55ea9cc52c40 2024-04-20T21:30:05.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.602+0000 7f285e060740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55ea9cb79110 con 0x55ea9cc52860 2024-04-20T21:30:05.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.603+0000 7f2859400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea9cc16860 0x55ea9cc16ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:05.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.603+0000 7f2859400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea9cc16860 0x55ea9cc16ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38856/0 (socket says 172.21.15.86:38856) 2024-04-20T21:30:05.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.603+0000 7f2859400640 1 -- 172.21.15.86:0/3055265124 learned_addr learned my addr 172.21.15.86:0/3055265124 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:05.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.603+0000 7f284be00640 1 --2- 172.21.15.86:0/3055265124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea9cc52c40 0x55ea9cc16320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:05.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.603+0000 7f2858a00640 1 --2- 172.21.15.86:0/3055265124 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea9cc52860 0x55ea9cc5efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:05.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.603+0000 7f2859400640 1 -- 172.21.15.86:0/3055265124 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea9cc52860 msgr2=0x55ea9cc5efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:05.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.603+0000 7f2859400640 1 --2- 172.21.15.86:0/3055265124 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea9cc52860 0x55ea9cc5efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:05.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.603+0000 7f2859400640 1 -- 172.21.15.86:0/3055265124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea9cc52c40 msgr2=0x55ea9cc16320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:05.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.603+0000 7f2859400640 1 --2- 172.21.15.86:0/3055265124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea9cc52c40 0x55ea9cc16320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:05.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.603+0000 7f2859400640 1 -- 172.21.15.86:0/3055265124 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea9c9597d0 con 0x55ea9cc16860 2024-04-20T21:30:05.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.603+0000 7f2859400640 1 --2- 172.21.15.86:0/3055265124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea9cc16860 0x55ea9cc16ce0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f284c056670 tx=0x7f284c05a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90fade5dc850c16e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:05.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.604+0000 7f2849200640 1 -- 172.21.15.86:0/3055265124 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284c062df0 con 0x55ea9cc16860 2024-04-20T21:30:05.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.604+0000 7f2849200640 1 -- 172.21.15.86:0/3055265124 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f284c058510 con 0x55ea9cc16860 2024-04-20T21:30:05.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.604+0000 7f2849200640 1 -- 172.21.15.86:0/3055265124 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284c056880 con 0x55ea9cc16860 2024-04-20T21:30:05.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.604+0000 7f285e060740 1 -- 172.21.15.86:0/3055265124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea9cc16860 msgr2=0x55ea9cc16ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:05.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.604+0000 7f285e060740 1 --2- 172.21.15.86:0/3055265124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea9cc16860 0x55ea9cc16ce0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f284c056670 tx=0x7f284c05a060 comp rx=0 tx=0).stop 2024-04-20T21:30:05.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.604+0000 7f285e060740 1 -- 172.21.15.86:0/3055265124 shutdown_connections 2024-04-20T21:30:05.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.604+0000 7f285e060740 1 --2- 172.21.15.86:0/3055265124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea9cc16860 0x55ea9cc16ce0 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:05.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.604+0000 7f285e060740 1 --2- 172.21.15.86:0/3055265124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea9cc52c40 0x55ea9cc16320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:05.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.604+0000 7f285e060740 1 --2- 172.21.15.86:0/3055265124 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea9cc52860 0x55ea9cc5efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:05.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.604+0000 7f285e060740 1 -- 172.21.15.86:0/3055265124 >> 172.21.15.86:0/3055265124 conn(0x55ea9cc13480 msgr2=0x55ea9cc5c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:05.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.605+0000 7f285e060740 1 -- 172.21.15.86:0/3055265124 shutdown_connections 2024-04-20T21:30:05.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.605+0000 7f285e060740 1 -- 172.21.15.86:0/3055265124 wait complete. 2024-04-20T21:30:05.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.605+0000 7f285e060740 1 -- start start 2024-04-20T21:30:05.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.605+0000 7f285e060740 1 Processor -- start 2024-04-20T21:30:05.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.606+0000 7f285e060740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea9cc52860 0x55ea9cc3fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:05.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.606+0000 7f285e060740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea9cc52c40 0x55ea9cc43c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:05.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.606+0000 7f285e060740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea9cc40240 0x55ea9cc441d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:05.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.606+0000 7f285e060740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ea9cc11ab0 con 0x55ea9cc52c40 2024-04-20T21:30:05.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.606+0000 7f285e060740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ea9cb36740 con 0x55ea9cc40240 2024-04-20T21:30:05.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.606+0000 7f285e060740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55ea9cc158b0 con 0x55ea9cc52860 2024-04-20T21:30:05.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.606+0000 7f285e060740 10 client.0 _ll_register_callbacks cb 0x55ea9cb31b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:30:05.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.607+0000 7f284be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea9cc52c40 0x55ea9cc43c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:05.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.607+0000 7f284be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea9cc52c40 0x55ea9cc43c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38864/0 (socket says 172.21.15.86:38864) 2024-04-20T21:30:05.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.607+0000 7f284be00640 1 -- 172.21.15.86:0/4032193072 learned_addr learned my addr 172.21.15.86:0/4032193072 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:05.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.607+0000 7f284be00640 1 -- 172.21.15.86:0/4032193072 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea9cc52860 msgr2=0x55ea9cc3fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:05.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.607+0000 7f284be00640 1 --2- 172.21.15.86:0/4032193072 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea9cc52860 0x55ea9cc3fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:05.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.607+0000 7f284be00640 1 -- 172.21.15.86:0/4032193072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea9cc40240 msgr2=0x55ea9cc441d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:05.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.607+0000 7f284be00640 1 --2- 172.21.15.86:0/4032193072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea9cc40240 0x55ea9cc441d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:05.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.607+0000 7f284be00640 1 -- 172.21.15.86:0/4032193072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea9c88f3f0 con 0x55ea9cc52c40 2024-04-20T21:30:05.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.607+0000 7f284be00640 1 --2- 172.21.15.86:0/4032193072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea9cc52c40 0x55ea9cc43c90 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f283c001de0 tx=0x7f283c00f5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=324660718bb2ac2d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:05.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.607+0000 7f2841600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:30:05.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.608+0000 7f2841600640 1 -- 172.21.15.86:0/4032193072 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283c0058b0 con 0x55ea9cc52c40 2024-04-20T21:30:05.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.608+0000 7f2841600640 1 -- 172.21.15.86:0/4032193072 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7f283c005a50 con 0x55ea9cc52c40 2024-04-20T21:30:05.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.608+0000 7f285e060740 1 -- 172.21.15.86:0/4032193072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f284c058510 con 0x55ea9cc52c40 2024-04-20T21:30:05.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.608+0000 7f2841600640 1 -- 172.21.15.86:0/4032193072 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f283c0011b0 con 0x55ea9cc52c40 2024-04-20T21:30:05.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.608+0000 7f285e060740 10 client.9483 fetch_fsmap learned FSMap version 99 2024-04-20T21:30:05.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.608+0000 7f285e060740 1 -- 172.21.15.86:0/4032193072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) -- 0x55ea9c84f0f0 con 0x55ea9cc52c40 2024-04-20T21:30:05.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.608+0000 7f2841600640 1 -- 172.21.15.86:0/4032193072 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f284c058510 con 0x55ea9cc52c40 2024-04-20T21:30:05.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.608+0000 7f2841600640 1 -- 172.21.15.86:0/4032193072 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 99) ==== 3495+0+0 (secure 0 0 0) 0x7f283c023020 con 0x55ea9cc52c40 2024-04-20T21:30:05.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.608+0000 7f285e060740 10 client.9483 fetch_fsmap finished waiting for FSMap version 99 2024-04-20T21:30:05.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.608+0000 7f285e060740 10 client.9483 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-20T21:30:05.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.608+0000 7f285e060740 20 client.9483 populate_metadata read hostname 'smithi086' 2024-04-20T21:30:05.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.608+0000 7f285e060740 1 --2- 172.21.15.86:0/4032193072 >> [v2:172.21.15.86:6835/289590924,v1:172.21.15.86:6837/289590924] conn(0x55ea9cc3df80 0x55ea9cc3e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:05.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.608+0000 7f285e060740 4 client.9483 mds_command: new command op to 9231 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:30:05.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.609+0000 7f285e060740 1 -- 172.21.15.86:0/4032193072 --> [v2:172.21.15.86:6835/289590924,v1:172.21.15.86:6837/289590924] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea9cb8ab70 con 0x55ea9cc3df80 2024-04-20T21:30:05.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.609+0000 7f2858a00640 1 --2- 172.21.15.86:0/4032193072 >> [v2:172.21.15.86:6835/289590924,v1:172.21.15.86:6837/289590924] conn(0x55ea9cc3df80 0x55ea9cc3e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:05.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.609+0000 7f2858a00640 1 --2- 172.21.15.86:0/4032193072 >> [v2:172.21.15.86:6835/289590924,v1:172.21.15.86:6837/289590924] conn(0x55ea9cc3df80 0x55ea9cc3e3e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b90f316c44b199e1 server_cookie=3fc88f951173c671 in_seq=0 out_seq=0 2024-04-20T21:30:05.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.609+0000 7f2841600640 10 client.9483 ms_handle_connect on v2:172.21.15.86:6835/289590924 2024-04-20T21:30:05.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.610+0000 7f2841600640 1 -- 172.21.15.86:0/4032193072 <== mds.0 v2:172.21.15.86:6835/289590924 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ea9cb8ab70 con 0x55ea9cc3df80 2024-04-20T21:30:05.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.610+0000 7f2841600640 10 client.9483 handle_command_reply: tid=0 2024-04-20T21:30:05.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.610+0000 7f285e060740 1 client.9483 shutdown 2024-04-20T21:30:05.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.610+0000 7f285e060740 1 -- 172.21.15.86:0/4032193072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea9cc52c40 msgr2=0x55ea9cc43c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:05.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.610+0000 7f285e060740 1 --2- 172.21.15.86:0/4032193072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea9cc52c40 0x55ea9cc43c90 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f283c001de0 tx=0x7f283c00f5a0 comp rx=0 tx=0).stop 2024-04-20T21:30:05.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.611+0000 7f285e060740 1 -- 172.21.15.86:0/4032193072 shutdown_connections 2024-04-20T21:30:05.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.611+0000 7f285e060740 1 --2- 172.21.15.86:0/4032193072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea9cc40240 0x55ea9cc441d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:05.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.611+0000 7f285e060740 1 --2- 172.21.15.86:0/4032193072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea9cc52c40 0x55ea9cc43c90 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:05.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.611+0000 7f285e060740 1 --2- 172.21.15.86:0/4032193072 >> [v2:172.21.15.86:6835/289590924,v1:172.21.15.86:6837/289590924] conn(0x55ea9cc3df80 0x55ea9cc3e3e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:05.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.611+0000 7f285e060740 1 --2- 172.21.15.86:0/4032193072 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea9cc52860 0x55ea9cc3fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:05.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.611+0000 7f2841600640 0 client.9483 ms_handle_reset on v2:172.21.15.86:6835/289590924 2024-04-20T21:30:05.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.611+0000 7f285e060740 1 -- 172.21.15.86:0/4032193072 >> 172.21.15.86:0/4032193072 conn(0x55ea9cc16860 msgr2=0x55ea9cc10030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:05.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.611+0000 7f285e060740 1 -- 172.21.15.86:0/4032193072 shutdown_connections 2024-04-20T21:30:05.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.611+0000 7f285e060740 1 -- 172.21.15.86:0/4032193072 wait complete. 2024-04-20T21:30:05.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.611+0000 7f285e060740 20 client.9483 trim_cache size 0 max 16384 2024-04-20T21:30:05.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.627+0000 7f285e060740 1 Processor -- start 2024-04-20T21:30:05.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.627+0000 7f285e060740 1 -- start start 2024-04-20T21:30:05.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.628+0000 7f285e060740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea9cc40240 0x55ea9cc40f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:05.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.628+0000 7f285e060740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea9cc52860 0x55ea9cc5d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:05.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.628+0000 7f285e060740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea9cc52c40 0x55ea9cc10220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:05.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.628+0000 7f285e060740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f28540c7160 con 0x55ea9cc52c40 2024-04-20T21:30:05.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.628+0000 7f285e060740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f28541e8e00 con 0x55ea9cc52860 2024-04-20T21:30:05.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.628+0000 7f285e060740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f28541e5c20 con 0x55ea9cc40240 2024-04-20T21:30:05.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.628+0000 7f2859400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea9cc52c40 0x55ea9cc10220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:05.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.628+0000 7f2859400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea9cc52c40 0x55ea9cc10220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38876/0 (socket says 172.21.15.86:38876) 2024-04-20T21:30:05.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.628+0000 7f2859400640 1 -- 172.21.15.86:0/620023832 learned_addr learned my addr 172.21.15.86:0/620023832 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:05.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.629+0000 7f2859400640 1 -- 172.21.15.86:0/620023832 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea9cc40240 msgr2=0x55ea9cc40f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:05.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.629+0000 7f2859400640 1 --2- 172.21.15.86:0/620023832 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea9cc40240 0x55ea9cc40f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:05.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.629+0000 7f2859400640 1 -- 172.21.15.86:0/620023832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea9cc52860 msgr2=0x55ea9cc5d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:05.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.629+0000 7f284be00640 1 --2- 172.21.15.86:0/620023832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea9cc52860 0x55ea9cc5d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:05.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.629+0000 7f2859400640 1 --2- 172.21.15.86:0/620023832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea9cc52860 0x55ea9cc5d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:05.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.629+0000 7f2859400640 1 -- 172.21.15.86:0/620023832 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea9cc5d020 con 0x55ea9cc52c40 2024-04-20T21:30:05.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.629+0000 7f2859400640 1 --2- 172.21.15.86:0/620023832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea9cc52c40 0x55ea9cc10220 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f284c05a1a0 tx=0x7f284c021070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb5d8bc507930353 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:05.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.629+0000 7f2843e00640 1 -- 172.21.15.86:0/620023832 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284c0564b0 con 0x55ea9cc52c40 2024-04-20T21:30:05.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.629+0000 7f2843e00640 1 -- 172.21.15.86:0/620023832 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f284c062df0 con 0x55ea9cc52c40 2024-04-20T21:30:05.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.629+0000 7f2843e00640 1 -- 172.21.15.86:0/620023832 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284c056880 con 0x55ea9cc52c40 2024-04-20T21:30:05.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.630+0000 7f285e060740 1 -- 172.21.15.86:0/620023832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea9cc52c40 msgr2=0x55ea9cc10220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:05.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.630+0000 7f285e060740 1 --2- 172.21.15.86:0/620023832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea9cc52c40 0x55ea9cc10220 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f284c05a1a0 tx=0x7f284c021070 comp rx=0 tx=0).stop 2024-04-20T21:30:05.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.630+0000 7f285e060740 1 -- 172.21.15.86:0/620023832 shutdown_connections 2024-04-20T21:30:05.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.630+0000 7f285e060740 1 --2- 172.21.15.86:0/620023832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea9cc52c40 0x55ea9cc10220 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:05.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.630+0000 7f285e060740 1 --2- 172.21.15.86:0/620023832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea9cc52860 0x55ea9cc5d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:05.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.630+0000 7f285e060740 1 --2- 172.21.15.86:0/620023832 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea9cc40240 0x55ea9cc40f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:05.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.630+0000 7f285e060740 1 -- 172.21.15.86:0/620023832 >> 172.21.15.86:0/620023832 conn(0x55ea9cc16860 msgr2=0x55ea9cc53020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:05.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.630+0000 7f285e060740 1 -- 172.21.15.86:0/620023832 shutdown_connections 2024-04-20T21:30:05.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.630+0000 7f285e060740 1 -- 172.21.15.86:0/620023832 wait complete. 2024-04-20T21:30:05.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.631+0000 7f285e060740 1 -- start start 2024-04-20T21:30:05.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.631+0000 7f285e060740 1 Processor -- start 2024-04-20T21:30:05.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.632+0000 7f285e060740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea9cc40240 0x55ea9cc10fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:05.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.632+0000 7f285e060740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea9cc52860 0x55ea9cc10220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:05.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.632+0000 7f285e060740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea9cc52c40 0x55ea9cc10760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:05.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.632+0000 7f285e060740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ea9cc3b5e0 con 0x55ea9cc40240 2024-04-20T21:30:05.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.632+0000 7f285e060740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f28541e87b0 con 0x55ea9cc52c40 2024-04-20T21:30:05.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.632+0000 7f285e060740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55ea9cc41460 con 0x55ea9cc52860 2024-04-20T21:30:05.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.632+0000 7f285e060740 10 client.0 _ll_register_callbacks cb 0x7f283c007dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:30:05.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.632+0000 7f2858a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea9cc40240 0x55ea9cc10fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:05.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.632+0000 7f2858a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea9cc40240 0x55ea9cc10fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38882/0 (socket says 172.21.15.86:38882) 2024-04-20T21:30:05.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.632+0000 7f2858a00640 1 -- 172.21.15.86:0/75807481 learned_addr learned my addr 172.21.15.86:0/75807481 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:05.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.633+0000 7f2858a00640 1 -- 172.21.15.86:0/75807481 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea9cc52860 msgr2=0x55ea9cc10220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:05.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.633+0000 7f2858a00640 1 --2- 172.21.15.86:0/75807481 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea9cc52860 0x55ea9cc10220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:05.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.633+0000 7f2858a00640 1 -- 172.21.15.86:0/75807481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea9cc52c40 msgr2=0x55ea9cc10760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:05.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.633+0000 7f2858a00640 1 --2- 172.21.15.86:0/75807481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea9cc52c40 0x55ea9cc10760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:05.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.633+0000 7f2858a00640 1 -- 172.21.15.86:0/75807481 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea9cc44170 con 0x55ea9cc40240 2024-04-20T21:30:05.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.633+0000 7f2858a00640 1 --2- 172.21.15.86:0/75807481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea9cc40240 0x55ea9cc10fb0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f2844009560 tx=0x7f28440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6320a8b75dc817d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:05.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.633+0000 7f2841600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:30:05.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.633+0000 7f2841600640 1 -- 172.21.15.86:0/75807481 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2844055060 con 0x55ea9cc40240 2024-04-20T21:30:05.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.633+0000 7f2841600640 1 -- 172.21.15.86:0/75807481 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7f284404f670 con 0x55ea9cc40240 2024-04-20T21:30:05.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.633+0000 7f285e060740 1 -- 172.21.15.86:0/75807481 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f284c062df0 con 0x55ea9cc40240 2024-04-20T21:30:05.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.633+0000 7f2841600640 1 -- 172.21.15.86:0/75807481 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f284405b050 con 0x55ea9cc40240 2024-04-20T21:30:05.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.634+0000 7f285e060740 10 client.9489 fetch_fsmap learned FSMap version 99 2024-04-20T21:30:05.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.634+0000 7f285e060740 1 -- 172.21.15.86:0/75807481 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) -- 0x55ea9cc59a30 con 0x55ea9cc40240 2024-04-20T21:30:05.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.634+0000 7f2841600640 1 -- 172.21.15.86:0/75807481 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f284c062df0 con 0x55ea9cc40240 2024-04-20T21:30:05.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.634+0000 7f2841600640 1 -- 172.21.15.86:0/75807481 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 99) ==== 3495+0+0 (secure 0 0 0) 0x7f284403a420 con 0x55ea9cc40240 2024-04-20T21:30:05.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.634+0000 7f285e060740 10 client.9489 fetch_fsmap finished waiting for FSMap version 99 2024-04-20T21:30:05.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.634+0000 7f285e060740 10 client.9489 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-20T21:30:05.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.634+0000 7f285e060740 20 client.9489 populate_metadata read hostname 'smithi086' 2024-04-20T21:30:05.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.634+0000 7f285e060740 1 --2- 172.21.15.86:0/75807481 >> [v2:172.21.15.86:6835/289590924,v1:172.21.15.86:6837/289590924] conn(0x55ea9ccc4b80 0x55ea9cc50530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:05.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.634+0000 7f285e060740 4 client.9489 mds_command: new command op to 9231 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-20T21:30:05.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.634+0000 7f285e060740 1 -- 172.21.15.86:0/75807481 --> [v2:172.21.15.86:6835/289590924,v1:172.21.15.86:6837/289590924] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ea9cc12b70 con 0x55ea9ccc4b80 2024-04-20T21:30:05.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.634+0000 7f284be00640 1 --2- 172.21.15.86:0/75807481 >> [v2:172.21.15.86:6835/289590924,v1:172.21.15.86:6837/289590924] conn(0x55ea9ccc4b80 0x55ea9cc50530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:05.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.635+0000 7f2841600640 10 client.9489 ms_handle_connect on v2:172.21.15.86:6835/289590924 2024-04-20T21:30:05.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.635+0000 7f284be00640 1 --2- 172.21.15.86:0/75807481 >> [v2:172.21.15.86:6835/289590924,v1:172.21.15.86:6837/289590924] conn(0x55ea9ccc4b80 0x55ea9cc50530 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e4b1be2d0fcf1ff server_cookie=5a47526a97cbf722 in_seq=0 out_seq=0 2024-04-20T21:30:05.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.648+0000 7f2841600640 1 -- 172.21.15.86:0/75807481 <== mds.0 v2:172.21.15.86:6835/289590924 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ea9cc12b70 con 0x55ea9ccc4b80 2024-04-20T21:30:05.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.648+0000 7f2841600640 10 client.9489 handle_command_reply: tid=0 2024-04-20T21:30:05.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.648+0000 7f285e060740 1 client.9489 shutdown 2024-04-20T21:30:05.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.648+0000 7f285e060740 1 -- 172.21.15.86:0/75807481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea9cc40240 msgr2=0x55ea9cc10fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:05.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.648+0000 7f285e060740 1 --2- 172.21.15.86:0/75807481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea9cc40240 0x55ea9cc10fb0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f2844009560 tx=0x7f28440046c0 comp rx=0 tx=0).stop 2024-04-20T21:30:05.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.648+0000 7f285e060740 1 -- 172.21.15.86:0/75807481 shutdown_connections 2024-04-20T21:30:05.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.648+0000 7f285e060740 1 --2- 172.21.15.86:0/75807481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea9cc52c40 0x55ea9cc10760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:05.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.648+0000 7f285e060740 1 --2- 172.21.15.86:0/75807481 >> [v2:172.21.15.86:6835/289590924,v1:172.21.15.86:6837/289590924] conn(0x55ea9ccc4b80 0x55ea9cc50530 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:05.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.648+0000 7f285e060740 1 --2- 172.21.15.86:0/75807481 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea9cc52860 0x55ea9cc10220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:05.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.648+0000 7f285e060740 1 --2- 172.21.15.86:0/75807481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea9cc40240 0x55ea9cc10fb0 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:05.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.648+0000 7f285e060740 1 -- 172.21.15.86:0/75807481 >> 172.21.15.86:0/75807481 conn(0x55ea9cc16860 msgr2=0x55ea9cc5d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:05.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.648+0000 7f2841600640 0 client.9489 ms_handle_reset on v2:172.21.15.86:6835/289590924 2024-04-20T21:30:05.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.649+0000 7f285e060740 1 -- 172.21.15.86:0/75807481 shutdown_connections 2024-04-20T21:30:05.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.649+0000 7f285e060740 1 -- 172.21.15.86:0/75807481 wait complete. 2024-04-20T21:30:05.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.649+0000 7f285e060740 20 client.9489 trim_cache size 0 max 16384 2024-04-20T21:30:05.649 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:30:05.649 INFO:teuthology.orchestra.run.smithi086.stdout: "message": "", 2024-04-20T21:30:05.649 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0 2024-04-20T21:30:05.649 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:30:05.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.649+0000 7f285b400640 1 -- 172.21.15.86:0/3907758880 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f283006c8f0 msgr2=0x7f283006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:05.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.649+0000 7f285b400640 1 --2- 172.21.15.86:0/3907758880 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f283006c8f0 0x7f283006edb0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f2844036000 tx=0x7f2844008030 comp rx=0 tx=0).stop 2024-04-20T21:30:05.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.649+0000 7f285b400640 1 -- 172.21.15.86:0/3907758880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2854146210 msgr2=0x7f2854132250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:05.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.650+0000 7f285b400640 1 --2- 172.21.15.86:0/3907758880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2854146210 0x7f2854132250 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f284c00cef0 tx=0x7f284c00a710 comp rx=0 tx=0).stop 2024-04-20T21:30:05.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.650+0000 7f285b400640 1 -- 172.21.15.86:0/3907758880 shutdown_connections 2024-04-20T21:30:05.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.650+0000 7f285b400640 1 --2- 172.21.15.86:0/3907758880 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f283006c8f0 0x7f283006edb0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:05.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.650+0000 7f285b400640 1 --2- 172.21.15.86:0/3907758880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2854146210 0x7f2854132250 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:05.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.650+0000 7f285b400640 1 --2- 172.21.15.86:0/3907758880 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f285406bfd0 0x7f2854131d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:05.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.650+0000 7f285b400640 1 --2- 172.21.15.86:0/3907758880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f285406bbf0 0x7f2854133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:05.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.650+0000 7f285b400640 1 -- 172.21.15.86:0/3907758880 >> 172.21.15.86:0/3907758880 conn(0x7f28540b2110 msgr2=0x7f2854071f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:05.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.650+0000 7f285b400640 1 -- 172.21.15.86:0/3907758880 shutdown_connections 2024-04-20T21:30:05.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.650+0000 7f285b400640 1 -- 172.21.15.86:0/3907758880 wait complete. 2024-04-20T21:30:05.668 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T21:30:05.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.860+0000 7fd79e800640 1 Processor -- start 2024-04-20T21:30:05.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.861+0000 7fd79e800640 1 -- start start 2024-04-20T21:30:05.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.862+0000 7fd79e800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7980cd170 0x7fd79806ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:05.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.862+0000 7fd79e800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd7980cd5d0 0x7fd79806f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:05.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.862+0000 7fd79e800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7980cdb50 con 0x7fd798144e90 2024-04-20T21:30:05.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.862+0000 7fd79e800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fd798145270 con 0x7fd7980cd5d0 2024-04-20T21:30:05.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.862+0000 7fd79e800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd7980b3e50 con 0x7fd7980cd170 2024-04-20T21:30:05.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.862+0000 7fd797e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd7980cd5d0 0x7fd79806f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:05.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.862+0000 7fd797e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd7980cd5d0 0x7fd79806f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59900/0 (socket says 172.21.15.86:59900) 2024-04-20T21:30:05.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.862+0000 7fd797e00640 1 -- 172.21.15.86:0/2284033878 learned_addr learned my addr 172.21.15.86:0/2284033878 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:05.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.863+0000 7fd78ea00640 1 --2- 172.21.15.86:0/2284033878 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7980cd170 0x7fd79806ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:05.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.863+0000 7fd797e00640 1 -- 172.21.15.86:0/2284033878 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7980cd170 msgr2=0x7fd79806ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:05.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.863+0000 7fd797e00640 1 --2- 172.21.15.86:0/2284033878 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7980cd170 0x7fd79806ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:05.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.863+0000 7fd797e00640 1 -- 172.21.15.86:0/2284033878 >> v1:172.21.15.144:6789/0 conn(0x7fd798144e90 legacy=0x7fd7980b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:05.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.863+0000 7fd797e00640 1 -- 172.21.15.86:0/2284033878 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7980b4130 con 0x7fd7980cd5d0 2024-04-20T21:30:05.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.864+0000 7fd797e00640 1 --2- 172.21.15.86:0/2284033878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7980cd5d0 0x7fd79806f280 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7fd790009e50 tx=0x7fd79002bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49c113a21edce514 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:05.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.865+0000 7fd796a00640 1 -- 172.21.15.86:0/2284033878 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79002c950 con 0x7fd7980cd5d0 2024-04-20T21:30:05.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.865+0000 7fd796a00640 1 -- 172.21.15.86:0/2284033878 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd79002caf0 con 0x7fd7980cd5d0 2024-04-20T21:30:05.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.865+0000 7fd796a00640 1 -- 172.21.15.86:0/2284033878 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7900343f0 con 0x7fd7980cd5d0 2024-04-20T21:30:05.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.865+0000 7fd79e800640 1 -- 172.21.15.86:0/2284033878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7980cd5d0 msgr2=0x7fd79806f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:05.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.865+0000 7fd79e800640 1 --2- 172.21.15.86:0/2284033878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7980cd5d0 0x7fd79806f280 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7fd790009e50 tx=0x7fd79002bdb0 comp rx=0 tx=0).stop 2024-04-20T21:30:05.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.865+0000 7fd79e800640 1 -- 172.21.15.86:0/2284033878 shutdown_connections 2024-04-20T21:30:05.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.865+0000 7fd79e800640 1 --2- 172.21.15.86:0/2284033878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7980cd5d0 0x7fd79806f280 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:05.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.865+0000 7fd79e800640 1 --2- 172.21.15.86:0/2284033878 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7980cd170 0x7fd79806ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:05.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.865+0000 7fd79e800640 1 -- 172.21.15.86:0/2284033878 >> 172.21.15.86:0/2284033878 conn(0x7fd7980b2360 msgr2=0x7fd7980b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:05.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.865+0000 7fd79e800640 1 -- 172.21.15.86:0/2284033878 shutdown_connections 2024-04-20T21:30:05.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.865+0000 7fd79e800640 1 -- 172.21.15.86:0/2284033878 wait complete. 2024-04-20T21:30:05.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.867+0000 7fd79e800640 1 Processor -- start 2024-04-20T21:30:05.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.867+0000 7fd79e800640 1 -- start start 2024-04-20T21:30:05.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.868+0000 7fd79e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7980cd170 0x7fd79813a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:05.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.868+0000 7fd79e800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7980cd5d0 0x7fd79813eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:05.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.868+0000 7fd79e800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd798144e90 0x7fd79813f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:05.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.868+0000 7fd79e800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd7980d0190 con 0x7fd7980cd170 2024-04-20T21:30:05.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.868+0000 7fd79e800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd7980d0010 con 0x7fd798144e90 2024-04-20T21:30:05.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.868+0000 7fd79e800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd79815bb50 con 0x7fd7980cd5d0 2024-04-20T21:30:05.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.869+0000 7fd797400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7980cd170 0x7fd79813a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:05.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.869+0000 7fd797400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7980cd170 0x7fd79813a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59914/0 (socket says 172.21.15.86:59914) 2024-04-20T21:30:05.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.869+0000 7fd797400640 1 -- 172.21.15.86:0/2198476188 learned_addr learned my addr 172.21.15.86:0/2198476188 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:05.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.869+0000 7fd797e00640 1 --2- 172.21.15.86:0/2198476188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd798144e90 0x7fd79813f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:05.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.869+0000 7fd797400640 1 -- 172.21.15.86:0/2198476188 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7980cd5d0 msgr2=0x7fd79813eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:05.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.869+0000 7fd797400640 1 --2- 172.21.15.86:0/2198476188 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7980cd5d0 0x7fd79813eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:05.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.869+0000 7fd797400640 1 -- 172.21.15.86:0/2198476188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd798144e90 msgr2=0x7fd79813f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:05.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.869+0000 7fd797400640 1 --2- 172.21.15.86:0/2198476188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd798144e90 0x7fd79813f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:05.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.869+0000 7fd797400640 1 -- 172.21.15.86:0/2198476188 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd798141f60 con 0x7fd7980cd170 2024-04-20T21:30:05.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.869+0000 7fd797400640 1 --2- 172.21.15.86:0/2198476188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7980cd170 0x7fd79813a5e0 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7fd78800b420 tx=0x7fd78800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:05.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.870+0000 7fd78fe00640 1 -- 172.21.15.86:0/2198476188 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd788011830 con 0x7fd7980cd170 2024-04-20T21:30:05.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.870+0000 7fd78fe00640 1 -- 172.21.15.86:0/2198476188 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd788016070 con 0x7fd7980cd170 2024-04-20T21:30:05.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.870+0000 7fd78fe00640 1 -- 172.21.15.86:0/2198476188 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd788011c10 con 0x7fd7980cd170 2024-04-20T21:30:05.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.870+0000 7fd79e800640 1 -- 172.21.15.86:0/2198476188 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd798142240 con 0x7fd7980cd170 2024-04-20T21:30:05.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.870+0000 7fd79e800640 1 -- 172.21.15.86:0/2198476188 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd798142800 con 0x7fd7980cd170 2024-04-20T21:30:05.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.871+0000 7fd79e800640 1 -- 172.21.15.86:0/2198476188 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7980689e0 con 0x7fd7980cd170 2024-04-20T21:30:05.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.872+0000 7fd78fe00640 1 -- 172.21.15.86:0/2198476188 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd78801b460 con 0x7fd7980cd170 2024-04-20T21:30:05.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.873+0000 7fd78fe00640 1 --2- 172.21.15.86:0/2198476188 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd77406c8f0 0x7fd77406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:05.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.873+0000 7fd78fe00640 1 -- 172.21.15.86:0/2198476188 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7fd78808b6f0 con 0x7fd7980cd170 2024-04-20T21:30:05.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.873+0000 7fd78ea00640 1 --2- 172.21.15.86:0/2198476188 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd77406c8f0 0x7fd77406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:05.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.874+0000 7fd78ea00640 1 --2- 172.21.15.86:0/2198476188 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd77406c8f0 0x7fd77406edb0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fd79806f320 tx=0x7fd780008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:05.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:05.877+0000 7fd78fe00640 1 -- 172.21.15.86:0/2198476188 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd788020020 con 0x7fd7980cd170 2024-04-20T21:30:06.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:06.118+0000 7fd79e800640 1 -- 172.21.15.86:0/2198476188 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd7980b00f0 con 0x7fd7980cd170 2024-04-20T21:30:06.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:06.119+0000 7fd78fe00640 1 -- 172.21.15.86:0/2198476188 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v99) ==== 53+0+73 (secure 0 0 0) 0x7fd788059a80 con 0x7fd7980cd170 2024-04-20T21:30:06.118 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T21:30:06.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:06.122+0000 7fd79e800640 1 -- 172.21.15.86:0/2198476188 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd77406c8f0 msgr2=0x7fd77406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:06.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:06.122+0000 7fd79e800640 1 --2- 172.21.15.86:0/2198476188 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd77406c8f0 0x7fd77406edb0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fd79806f320 tx=0x7fd780008040 comp rx=0 tx=0).stop 2024-04-20T21:30:06.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:06.122+0000 7fd79e800640 1 -- 172.21.15.86:0/2198476188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7980cd170 msgr2=0x7fd79813a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:06.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:06.122+0000 7fd79e800640 1 --2- 172.21.15.86:0/2198476188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7980cd170 0x7fd79813a5e0 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7fd78800b420 tx=0x7fd78800b8e0 comp rx=0 tx=0).stop 2024-04-20T21:30:06.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:06.122+0000 7fd79e800640 1 -- 172.21.15.86:0/2198476188 shutdown_connections 2024-04-20T21:30:06.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:06.122+0000 7fd79e800640 1 --2- 172.21.15.86:0/2198476188 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd77406c8f0 0x7fd77406edb0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:06.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:06.122+0000 7fd79e800640 1 --2- 172.21.15.86:0/2198476188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd798144e90 0x7fd79813f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:06.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:06.122+0000 7fd79e800640 1 --2- 172.21.15.86:0/2198476188 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd7980cd5d0 0x7fd79813eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:06.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:06.122+0000 7fd79e800640 1 --2- 172.21.15.86:0/2198476188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7980cd170 0x7fd79813a5e0 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:06.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:06.122+0000 7fd79e800640 1 -- 172.21.15.86:0/2198476188 >> 172.21.15.86:0/2198476188 conn(0x7fd7980b2360 msgr2=0x7fd7980b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:06.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:06.122+0000 7fd79e800640 1 -- 172.21.15.86:0/2198476188 shutdown_connections 2024-04-20T21:30:06.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:06.122+0000 7fd79e800640 1 -- 172.21.15.86:0/2198476188 wait complete. 2024-04-20T21:30:06.140 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T21:30:06.140 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T21:30:06.140 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-04-20T21:30:06.140 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-20T21:30:06.140 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T21:30:06.140 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T21:30:06.140 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T21:30:06.140 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T21:30:06.141 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:30:06.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:30:06.141 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-04-20T21:30:06.188 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T21:30:06.188 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T21:30:06.188 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T21:30:06.188 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:30:06.188 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-20T21:30:06.189 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:30:06.189 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:30:06.189 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:30:06.189 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-04-20T21:30:06.189 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:30:06.189 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:30:06.189 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-04-20T21:30:06.189 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:30:06.189 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:30:06.189 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T21:30:06.189 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:30:06.189 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T21:30:06.189 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 41950sec preferred_lft 41950sec 2024-04-20T21:30:06.189 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-04-20T21:30:06.189 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:30:06.189 INFO:teuthology.orchestra.run.smithi086.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T21:30:06.190 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 86:25:97:2c:9c:bb brd ff:ff:ff:ff:ff:ff 2024-04-20T21:30:06.190 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-20T21:30:06.190 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:30:06.190 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::a4fb:30ff:fe90:2d6a/64 scope link 2024-04-20T21:30:06.190 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:30:06.190 INFO:teuthology.orchestra.run.smithi086.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T21:30:06.190 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 86:25:97:2c:9c:bb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:30:06.190 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::8425:97ff:fe2c:9cbb/64 scope link 2024-04-20T21:30:06.190 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:30:06.190 INFO:teuthology.orchestra.run.smithi086.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T21:30:06.190 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c2:9c:8b:55:ca:84 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:30:06.190 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c09c:8bff:fe55:ca84/64 scope link 2024-04-20T21:30:06.190 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:30:06.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:30:06.191 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T21:30:06.311 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-20T21:30:06.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:30:06.315 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-04-20T21:30:06.366 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-04-20T21:30:06.367 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T21:30:06.367 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T21:30:06.367 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T21:30:06.367 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T21:30:06.367 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:30:06.367 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-20T21:30:06.367 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T21:30:06.367 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T21:30:06.367 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T21:30:06.368 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-20T21:30:06.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:30:06.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:30:06.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:30:06.566 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-20T21:30:06.610 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-20T21:30:06.610 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-20T21:30:06.611 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:30:06.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:30:06.612 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-04-20T21:30:06.739 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-20T21:30:06.739 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T21:30:06.739 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-20T21:30:06.792 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:30:06.792 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T21:30:06.792 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-20T21:30:06.841 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-04-20T21:30:06.918 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T21:30:06.973 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-04-20T21:30:06.973 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-04-20T21:30:06.973 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-04-20T21:30:06.974 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-04-20T21:30:06.974 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-04-20T21:30:06.974 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-04-20T21:30:06.974 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-04-20T21:30:06.974 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-04-20T21:30:06.974 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-04-20T21:30:06.974 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-04-20T21:30:06.974 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-04-20T21:30:06.974 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/selinux 2024-04-20T21:30:06.974 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T21:30:06.974 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-04-20T21:30:06.975 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-04-20T21:30:06.975 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-04-20T21:30:06.975 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-04-20T21:30:06.975 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-04-20T21:30:06.975 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-04-20T21:30:06.975 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T21:30:06.975 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T21:30:06.975 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T21:30:06.975 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T21:30:06.975 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-04-20T21:30:06.975 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-04-20T21:30:06.975 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T21:30:06.976 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T21:30:06.976 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T21:30:06.976 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T21:30:06.976 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-04-20T21:30:06.976 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:30:06.976 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:30:06.976 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:30:06.976 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:30:06.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:30:06.977 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-04-20T21:30:07.027 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T21:30:07.028 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-20T21:30:07.032 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-04-20T21:30:07.103 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T21:30:07.146 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-04-20T21:30:07.147 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-04-20T21:30:07.147 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-04-20T21:30:07.147 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-04-20T21:30:07.147 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-04-20T21:30:07.147 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-04-20T21:30:07.147 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-04-20T21:30:07.147 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-04-20T21:30:07.147 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-04-20T21:30:07.147 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-04-20T21:30:07.147 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-04-20T21:30:07.147 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/selinux 2024-04-20T21:30:07.148 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T21:30:07.148 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-04-20T21:30:07.148 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-04-20T21:30:07.148 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-04-20T21:30:07.148 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-04-20T21:30:07.148 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-04-20T21:30:07.148 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-04-20T21:30:07.148 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T21:30:07.148 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T21:30:07.148 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T21:30:07.148 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T21:30:07.148 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-04-20T21:30:07.148 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-04-20T21:30:07.149 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T21:30:07.149 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T21:30:07.149 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T21:30:07.149 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T21:30:07.149 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-04-20T21:30:07.149 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:30:07.149 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:30:07.149 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:30:07.149 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:30:07.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:30:07.150 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-04-20T21:30:07.202 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:2024-04-20T21:30:07.204+0000 7fa44ef4d480 -1 init, newargv = 0x564cde1d3250 newargc=15 2024-04-20T21:30:07.203 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[65871]: starting ceph client 2024-04-20T21:30:07.210 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[65871]: starting fuse 2024-04-20T21:30:07.234 INFO:teuthology.orchestra.run.smithi086.stdout:41 2024-04-20T21:30:07.234 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-04-20T21:30:07.234 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer -- python3 -c ' 2024-04-20T21:30:07.234 DEBUG:teuthology.orchestra.run.smithi086:> import glob 2024-04-20T21:30:07.235 DEBUG:teuthology.orchestra.run.smithi086:> import re 2024-04-20T21:30:07.235 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:30:07.235 DEBUG:teuthology.orchestra.run.smithi086:> import subprocess 2024-04-20T21:30:07.235 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:30:07.235 DEBUG:teuthology.orchestra.run.smithi086:> def _find_admin_socket(client_name): 2024-04-20T21:30:07.235 DEBUG:teuthology.orchestra.run.smithi086:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-20T21:30:07.235 DEBUG:teuthology.orchestra.run.smithi086:> files = glob.glob(asok_path) 2024-04-20T21:30:07.235 DEBUG:teuthology.orchestra.run.smithi086:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-20T21:30:07.235 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:30:07.235 DEBUG:teuthology.orchestra.run.smithi086:> # Given a non-glob path, it better be there 2024-04-20T21:30:07.235 DEBUG:teuthology.orchestra.run.smithi086:> if "*" not in asok_path: 2024-04-20T21:30:07.235 DEBUG:teuthology.orchestra.run.smithi086:> assert(len(files) == 1) 2024-04-20T21:30:07.236 DEBUG:teuthology.orchestra.run.smithi086:> return files[0] 2024-04-20T21:30:07.236 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:30:07.236 DEBUG:teuthology.orchestra.run.smithi086:> for f in files: 2024-04-20T21:30:07.236 DEBUG:teuthology.orchestra.run.smithi086:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T21:30:07.236 DEBUG:teuthology.orchestra.run.smithi086:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T21:30:07.236 DEBUG:teuthology.orchestra.run.smithi086:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T21:30:07.236 DEBUG:teuthology.orchestra.run.smithi086:> contents = proc_f.read() 2024-04-20T21:30:07.236 DEBUG:teuthology.orchestra.run.smithi086:> if mountpoint in contents: 2024-04-20T21:30:07.236 DEBUG:teuthology.orchestra.run.smithi086:> return f 2024-04-20T21:30:07.236 DEBUG:teuthology.orchestra.run.smithi086:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T21:30:07.236 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:30:07.236 DEBUG:teuthology.orchestra.run.smithi086:> print(_find_admin_socket("client.0")) 2024-04-20T21:30:07.236 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:30:07.435 INFO:teuthology.orchestra.run.smithi086.stdout:/var/run/ceph/ceph-client.0.65871.asok 2024-04-20T21:30:07.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:30:07.452 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.65871.asok 2024-04-20T21:30:07.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:30:07.452 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.65871.asok status 2024-04-20T21:30:07.680 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:30:07.680 INFO:teuthology.orchestra.run.smithi086.stdout: "metadata": { 2024-04-20T21:30:07.680 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T21:30:07.680 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T21:30:07.681 INFO:teuthology.orchestra.run.smithi086.stdout: "entity_id": "0", 2024-04-20T21:30:07.681 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": "smithi086", 2024-04-20T21:30:07.681 INFO:teuthology.orchestra.run.smithi086.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-20T21:30:07.681 INFO:teuthology.orchestra.run.smithi086.stdout: "pid": "65871", 2024-04-20T21:30:07.681 INFO:teuthology.orchestra.run.smithi086.stdout: "root": "/" 2024-04-20T21:30:07.681 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:30:07.681 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_count": 0, 2024-04-20T21:30:07.681 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_pinned_count": 0, 2024-04-20T21:30:07.681 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 9501, 2024-04-20T21:30:07.681 INFO:teuthology.orchestra.run.smithi086.stdout: "inst": { 2024-04-20T21:30:07.681 INFO:teuthology.orchestra.run.smithi086.stdout: "name": { 2024-04-20T21:30:07.681 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "client", 2024-04-20T21:30:07.681 INFO:teuthology.orchestra.run.smithi086.stdout: "num": 9501 2024-04-20T21:30:07.681 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:30:07.681 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2024-04-20T21:30:07.682 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-04-20T21:30:07.682 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2024-04-20T21:30:07.682 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 1703760452 2024-04-20T21:30:07.682 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:30:07.682 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:30:07.682 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2024-04-20T21:30:07.682 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-04-20T21:30:07.682 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2024-04-20T21:30:07.682 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 1703760452 2024-04-20T21:30:07.682 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:30:07.682 INFO:teuthology.orchestra.run.smithi086.stdout: "inst_str": "client.9501 192.168.0.1:0/1703760452", 2024-04-20T21:30:07.682 INFO:teuthology.orchestra.run.smithi086.stdout: "addr_str": "192.168.0.1:0/1703760452", 2024-04-20T21:30:07.682 INFO:teuthology.orchestra.run.smithi086.stdout: "inode_count": 1, 2024-04-20T21:30:07.682 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_epoch": 99, 2024-04-20T21:30:07.682 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch": 112, 2024-04-20T21:30:07.682 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch_barrier": 0, 2024-04-20T21:30:07.683 INFO:teuthology.orchestra.run.smithi086.stdout: "blocklisted": false, 2024-04-20T21:30:07.683 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs" 2024-04-20T21:30:07.683 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:30:07.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:30:07.695 DEBUG:teuthology.orchestra.run.smithi086:> stat --file-system '--printf=%T 2024-04-20T21:30:07.695 DEBUG:teuthology.orchestra.run.smithi086:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-20T21:30:07.748 INFO:teuthology.orchestra.run.smithi086.stdout:fuseblk 2024-04-20T21:30:07.748 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-20T21:30:07.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:30:07.749 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-20T21:30:07.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:30:07.833 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent/unflushed) 2024-04-20T21:30:07.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:30:07.893 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=./parent/unflushed/jfile bs=1M count=8) 2024-04-20T21:30:08.032 INFO:teuthology.orchestra.run.smithi086.stderr:8+0 records in 2024-04-20T21:30:08.032 INFO:teuthology.orchestra.run.smithi086.stderr:8+0 records out 2024-04-20T21:30:08.032 INFO:teuthology.orchestra.run.smithi086.stderr:8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.0886909 s, 94.6 MB/s 2024-04-20T21:30:08.034 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-04-20T21:30:08.034 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:30:08.034 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-04-20T21:30:08.034 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:30:08.034 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-04-20T21:30:08.034 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:30:08.169 INFO:teuthology.orchestra.run.smithi086.stdout:1099511627779 2024-04-20T21:30:08.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:30:08.182 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-04-20T21:30:08.182 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:30:08.182 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-04-20T21:30:08.182 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:30:08.182 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-04-20T21:30:08.182 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:30:08.353 INFO:teuthology.orchestra.run.smithi086.stdout:1099511627780 2024-04-20T21:30:08.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:30:08.369 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:30:08.369 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:30:08.467 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:30:08.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.661+0000 7f0e3a200640 1 Processor -- start 2024-04-20T21:30:08.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.662+0000 7f0e3a200640 1 -- start start 2024-04-20T21:30:08.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.663+0000 7f0e3a200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e34156ae0 0x7f0e3415d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:08.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.663+0000 7f0e3a200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0e34160080 0x7f0e340ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:08.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.663+0000 7f0e3a200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e34160890 con 0x7f0e340ad620 2024-04-20T21:30:08.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.663+0000 7f0e3a200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f0e3406db20 con 0x7f0e34160080 2024-04-20T21:30:08.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.663+0000 7f0e3a200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0e34160b70 con 0x7f0e34156ae0 2024-04-20T21:30:08.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.663+0000 7f0e39800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f0e340ad620 0x7f0e34141cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51250/0 (socket says 172.21.15.86:51250) 2024-04-20T21:30:08.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.663+0000 7f0e39800640 1 -- 172.21.15.86:0/2168809232 learned_addr learned my addr 172.21.15.86:0/2168809232 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:30:08.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.664+0000 7f0e38e00640 1 --2- 172.21.15.86:0/2168809232 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e34156ae0 0x7f0e3415d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:08.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.664+0000 7f0e33e00640 1 --2- 172.21.15.86:0/2168809232 >> v2:172.21.15.144:3300/0 conn(0x7f0e34160080 0x7f0e340ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:08.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.664+0000 7f0e33400640 1 -- 172.21.15.86:0/2168809232 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2683528313 0 0) 0x7f0e34160890 con 0x7f0e340ad620 2024-04-20T21:30:08.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.664+0000 7f0e33400640 1 -- 172.21.15.86:0/2168809232 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e1c003520 con 0x7f0e340ad620 2024-04-20T21:30:08.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.664+0000 7f0e33400640 1 -- 172.21.15.86:0/2168809232 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2422667083 0 0) 0x7f0e1c003520 con 0x7f0e340ad620 2024-04-20T21:30:08.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.664+0000 7f0e33400640 1 -- 172.21.15.86:0/2168809232 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e34156ae0 msgr2=0x7f0e3415d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:08.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.664+0000 7f0e33400640 1 --2- 172.21.15.86:0/2168809232 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e34156ae0 0x7f0e3415d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:08.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.664+0000 7f0e33400640 1 -- 172.21.15.86:0/2168809232 >> v2:172.21.15.144:3300/0 conn(0x7f0e34160080 msgr2=0x7f0e340ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:08.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.664+0000 7f0e33400640 1 --2- 172.21.15.86:0/2168809232 >> v2:172.21.15.144:3300/0 conn(0x7f0e34160080 0x7f0e340ad0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:08.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.664+0000 7f0e33400640 1 -- 172.21.15.86:0/2168809232 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e34160e50 con 0x7f0e340ad620 2024-04-20T21:30:08.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.664+0000 7f0e33400640 1 -- 172.21.15.86:0/2168809232 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f0e2c0019f0 con 0x7f0e340ad620 2024-04-20T21:30:08.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.664+0000 7f0e33400640 1 -- 172.21.15.86:0/2168809232 >> v1:172.21.15.86:6789/0 conn(0x7f0e340ad620 legacy=0x7f0e34141cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:08.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.664+0000 7f0e33400640 1 --2- 172.21.15.86:0/2168809232 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e34156ae0 0x7f0e1c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:08.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.665+0000 7f0e33400640 1 --2- 172.21.15.86:0/2168809232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e1c003f30 0x7f0e1c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:08.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.665+0000 7f0e33400640 1 --2- 172.21.15.86:0/2168809232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e1c004d40 0x7f0e1c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:08.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.665+0000 7f0e33400640 1 -- 172.21.15.86:0/2168809232 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3406db20 con 0x7f0e1c003f30 2024-04-20T21:30:08.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.665+0000 7f0e33400640 1 -- 172.21.15.86:0/2168809232 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0e34160b70 con 0x7f0e1c004d40 2024-04-20T21:30:08.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.665+0000 7f0e33400640 1 -- 172.21.15.86:0/2168809232 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0e1c00b9c0 con 0x7f0e34156ae0 2024-04-20T21:30:08.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.665+0000 7f0e33e00640 1 --2- 172.21.15.86:0/2168809232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e1c003f30 0x7f0e1c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:08.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.665+0000 7f0e33e00640 1 -- 172.21.15.86:0/2168809232 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e34156ae0 msgr2=0x7f0e1c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:08.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.665+0000 7f0e33e00640 1 --2- 172.21.15.86:0/2168809232 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e34156ae0 0x7f0e1c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:08.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.665+0000 7f0e33e00640 1 -- 172.21.15.86:0/2168809232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e1c004d40 msgr2=0x7f0e1c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:08.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.665+0000 7f0e33e00640 1 --2- 172.21.15.86:0/2168809232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e1c004d40 0x7f0e1c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:08.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.665+0000 7f0e33e00640 1 -- 172.21.15.86:0/2168809232 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e1c003520 con 0x7f0e1c003f30 2024-04-20T21:30:08.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.666+0000 7f0e33e00640 1 --2- 172.21.15.86:0/2168809232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e1c003f30 0x7f0e1c004800 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f0e20009ac0 tx=0x7f0e1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70b7c4c21ecfe1f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:08.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.666+0000 7f0e33400640 1 -- 172.21.15.86:0/2168809232 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e20039070 con 0x7f0e1c003f30 2024-04-20T21:30:08.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.666+0000 7f0e33400640 1 -- 172.21.15.86:0/2168809232 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e2002c6d0 con 0x7f0e1c003f30 2024-04-20T21:30:08.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.666+0000 7f0e33400640 1 -- 172.21.15.86:0/2168809232 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e20009690 con 0x7f0e1c003f30 2024-04-20T21:30:08.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.667+0000 7f0e3a200640 1 -- 172.21.15.86:0/2168809232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e1c003f30 msgr2=0x7f0e1c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:08.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.667+0000 7f0e3a200640 1 --2- 172.21.15.86:0/2168809232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e1c003f30 0x7f0e1c004800 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f0e20009ac0 tx=0x7f0e1c005830 comp rx=0 tx=0).stop 2024-04-20T21:30:08.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.667+0000 7f0e39800640 1 -- 172.21.15.86:0/2168809232 reap_dead start 2024-04-20T21:30:08.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.667+0000 7f0e3a200640 1 -- 172.21.15.86:0/2168809232 shutdown_connections 2024-04-20T21:30:08.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.667+0000 7f0e3a200640 1 -- 172.21.15.86:0/2168809232 >> 172.21.15.86:0/2168809232 conn(0x7f0e340d0170 msgr2=0x7f0e340d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:08.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.667+0000 7f0e3a200640 1 -- 172.21.15.86:0/2168809232 shutdown_connections 2024-04-20T21:30:08.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.667+0000 7f0e3a200640 1 -- 172.21.15.86:0/2168809232 wait complete. 2024-04-20T21:30:08.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.669+0000 7f0e3a200640 1 Processor -- start 2024-04-20T21:30:08.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.669+0000 7f0e3a200640 1 -- start start 2024-04-20T21:30:08.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.670+0000 7f0e3a200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e1c003f30 0x7f0e3415f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:08.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.670+0000 7f0e3a200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e34072f10 0x7f0e341f4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:08.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.670+0000 7f0e3a200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e341f48d0 0x7f0e3415ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:08.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.670+0000 7f0e3a200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0e34160500 con 0x7f0e1c003f30 2024-04-20T21:30:08.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.670+0000 7f0e3a200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0e34156ec0 con 0x7f0e341f48d0 2024-04-20T21:30:08.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.670+0000 7f0e3a200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0e3406f2a0 con 0x7f0e34072f10 2024-04-20T21:30:08.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.671+0000 7f0e38e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e1c003f30 0x7f0e3415f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:08.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.671+0000 7f0e38e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e1c003f30 0x7f0e3415f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59934/0 (socket says 172.21.15.86:59934) 2024-04-20T21:30:08.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.671+0000 7f0e38e00640 1 -- 172.21.15.86:0/3195001119 learned_addr learned my addr 172.21.15.86:0/3195001119 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:08.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.671+0000 7f0e33e00640 1 --2- 172.21.15.86:0/3195001119 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e34072f10 0x7f0e341f4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:08.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.671+0000 7f0e39800640 1 --2- 172.21.15.86:0/3195001119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e341f48d0 0x7f0e3415ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:08.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.671+0000 7f0e38e00640 1 -- 172.21.15.86:0/3195001119 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e34072f10 msgr2=0x7f0e341f4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:08.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.671+0000 7f0e38e00640 1 --2- 172.21.15.86:0/3195001119 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e34072f10 0x7f0e341f4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:08.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.671+0000 7f0e38e00640 1 -- 172.21.15.86:0/3195001119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e341f48d0 msgr2=0x7f0e3415ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:08.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.671+0000 7f0e38e00640 1 --2- 172.21.15.86:0/3195001119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e341f48d0 0x7f0e3415ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:08.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.671+0000 7f0e38e00640 1 -- 172.21.15.86:0/3195001119 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e341556d0 con 0x7f0e1c003f30 2024-04-20T21:30:08.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.671+0000 7f0e38e00640 1 --2- 172.21.15.86:0/3195001119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e1c003f30 0x7f0e3415f9c0 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f0e2800e7c0 tx=0x7f0e2800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:08.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.672+0000 7f0e30c00640 1 -- 172.21.15.86:0/3195001119 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e28017070 con 0x7f0e1c003f30 2024-04-20T21:30:08.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.672+0000 7f0e30c00640 1 -- 172.21.15.86:0/3195001119 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e2800c940 con 0x7f0e1c003f30 2024-04-20T21:30:08.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.672+0000 7f0e3a200640 1 -- 172.21.15.86:0/3195001119 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e34142e10 con 0x7f0e1c003f30 2024-04-20T21:30:08.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.672+0000 7f0e30c00640 1 -- 172.21.15.86:0/3195001119 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2800cd10 con 0x7f0e1c003f30 2024-04-20T21:30:08.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.672+0000 7f0e3a200640 1 -- 172.21.15.86:0/3195001119 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e341433d0 con 0x7f0e1c003f30 2024-04-20T21:30:08.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.673+0000 7f0e3a200640 1 -- 172.21.15.86:0/3195001119 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e34143810 con 0x7f0e1c003f30 2024-04-20T21:30:08.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.674+0000 7f0e30c00640 1 -- 172.21.15.86:0/3195001119 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0e2801a070 con 0x7f0e1c003f30 2024-04-20T21:30:08.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.675+0000 7f0e30c00640 1 --2- 172.21.15.86:0/3195001119 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0e1406c920 0x7f0e1406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:08.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.676+0000 7f0e30c00640 1 -- 172.21.15.86:0/3195001119 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7f0e2808bce0 con 0x7f0e1c003f30 2024-04-20T21:30:08.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.676+0000 7f0e33e00640 1 --2- 172.21.15.86:0/3195001119 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0e1406c920 0x7f0e1406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:08.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.676+0000 7f0e33e00640 1 --2- 172.21.15.86:0/3195001119 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0e1406c920 0x7f0e1406ede0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f0e20036000 tx=0x7f0e20030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:08.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.680+0000 7f0e30c00640 1 -- 172.21.15.86:0/3195001119 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0e2805a100 con 0x7f0e1c003f30 2024-04-20T21:30:08.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.881+0000 7f0e3a200640 1 -- 172.21.15.86:0/3195001119 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0e34071b70 con 0x7f0e1c003f30 2024-04-20T21:30:08.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.883+0000 7f0e30c00640 1 -- 172.21.15.86:0/3195001119 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) ==== 74+0+16627 (secure 0 0 0) 0x7f0e28059a00 con 0x7f0e1c003f30 2024-04-20T21:30:08.883 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:08.883 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":112,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:29:45.322441+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:29:41.532824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-20T21:29:42.749687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:30:08.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.886+0000 7f0e3a200640 1 -- 172.21.15.86:0/3195001119 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0e1406c920 msgr2=0x7f0e1406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:08.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.886+0000 7f0e3a200640 1 --2- 172.21.15.86:0/3195001119 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0e1406c920 0x7f0e1406ede0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f0e20036000 tx=0x7f0e20030000 comp rx=0 tx=0).stop 2024-04-20T21:30:08.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.886+0000 7f0e3a200640 1 -- 172.21.15.86:0/3195001119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e1c003f30 msgr2=0x7f0e3415f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:08.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.886+0000 7f0e3a200640 1 --2- 172.21.15.86:0/3195001119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e1c003f30 0x7f0e3415f9c0 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f0e2800e7c0 tx=0x7f0e2800ec80 comp rx=0 tx=0).stop 2024-04-20T21:30:08.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.886+0000 7f0e3a200640 1 -- 172.21.15.86:0/3195001119 shutdown_connections 2024-04-20T21:30:08.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.886+0000 7f0e3a200640 1 --2- 172.21.15.86:0/3195001119 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0e1406c920 0x7f0e1406ede0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:08.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.886+0000 7f0e3a200640 1 --2- 172.21.15.86:0/3195001119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e341f48d0 0x7f0e3415ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:08.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.886+0000 7f0e3a200640 1 --2- 172.21.15.86:0/3195001119 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0e34072f10 0x7f0e341f4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:08.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.886+0000 7f0e3a200640 1 --2- 172.21.15.86:0/3195001119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e1c003f30 0x7f0e3415f9c0 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:08.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.886+0000 7f0e3a200640 1 -- 172.21.15.86:0/3195001119 >> 172.21.15.86:0/3195001119 conn(0x7f0e340d0170 msgr2=0x7f0e3415cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:08.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.886+0000 7f0e3a200640 1 -- 172.21.15.86:0/3195001119 shutdown_connections 2024-04-20T21:30:08.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:08.887+0000 7f0e3a200640 1 -- 172.21.15.86:0/3195001119 wait complete. 2024-04-20T21:30:08.905 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:29:41.532824+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:29:42.749687+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:30:08.905 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:30:09.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.112+0000 7ff8ada00640 1 Processor -- start 2024-04-20T21:30:09.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.112+0000 7ff8ada00640 1 -- start start 2024-04-20T21:30:09.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.114+0000 7ff8ada00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff8a80af1f0 0x7ff8a806ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:09.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.114+0000 7ff8ada00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff8a80af5d0 0x7ff8a815de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:09.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.114+0000 7ff8ada00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8a806de80 con 0x7ff8a806d9c0 2024-04-20T21:30:09.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.114+0000 7ff8ada00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff8a806f460 con 0x7ff8a80af1f0 2024-04-20T21:30:09.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.114+0000 7ff8ada00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7ff8a806e160 con 0x7ff8a80af5d0 2024-04-20T21:30:09.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.114+0000 7ff89f400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7ff8a806d9c0 0x7ff8a815d520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51266/0 (socket says 172.21.15.86:51266) 2024-04-20T21:30:09.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.114+0000 7ff89f400640 1 -- 172.21.15.86:0/3950951551 learned_addr learned my addr 172.21.15.86:0/3950951551 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:30:09.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.114+0000 7ff8a7e00640 1 --2- 172.21.15.86:0/3950951551 >> v2:172.21.15.86:3300/0 conn(0x7ff8a80af1f0 0x7ff8a806ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:09.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.115+0000 7ff8ad000640 1 --2- 172.21.15.86:0/3950951551 >> v2:172.21.15.144:3300/0 conn(0x7ff8a80af5d0 0x7ff8a815de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:09.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.115+0000 7ff8a7400640 1 -- 172.21.15.86:0/3950951551 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2983522147 0 0) 0x7ff8a806de80 con 0x7ff8a806d9c0 2024-04-20T21:30:09.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.115+0000 7ff8a7400640 1 -- 172.21.15.86:0/3950951551 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff88c003520 con 0x7ff8a806d9c0 2024-04-20T21:30:09.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.115+0000 7ff8a7e00640 1 -- 172.21.15.86:0/3950951551 >> v2:172.21.15.144:3300/0 conn(0x7ff8a80af5d0 msgr2=0x7ff8a815de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:09.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.115+0000 7ff8a7e00640 1 --2- 172.21.15.86:0/3950951551 >> v2:172.21.15.144:3300/0 conn(0x7ff8a80af5d0 0x7ff8a815de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:09.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.115+0000 7ff8a7e00640 1 -- 172.21.15.86:0/3950951551 >> v1:172.21.15.86:6789/0 conn(0x7ff8a806d9c0 legacy=0x7ff8a815d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:09.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.115+0000 7ff8a7e00640 1 -- 172.21.15.86:0/3950951551 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8a806e440 con 0x7ff8a80af1f0 2024-04-20T21:30:09.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.117+0000 7ff8a7e00640 1 --2- 172.21.15.86:0/3950951551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8a80af1f0 0x7ff8a806ef20 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7ff8a80b2f60 tx=0x7ff89802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aeb4020aa0284 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:09.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.117+0000 7ff8a7400640 1 -- 172.21.15.86:0/3950951551 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff898035020 con 0x7ff8a80af1f0 2024-04-20T21:30:09.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.117+0000 7ff8a7400640 1 -- 172.21.15.86:0/3950951551 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff898031830 con 0x7ff8a80af1f0 2024-04-20T21:30:09.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.117+0000 7ff8a7400640 1 -- 172.21.15.86:0/3950951551 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff898031c00 con 0x7ff8a80af1f0 2024-04-20T21:30:09.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.118+0000 7ff8ada00640 1 -- 172.21.15.86:0/3950951551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8a80af1f0 msgr2=0x7ff8a806ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:09.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.118+0000 7ff8ada00640 1 --2- 172.21.15.86:0/3950951551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8a80af1f0 0x7ff8a806ef20 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7ff8a80b2f60 tx=0x7ff89802b8d0 comp rx=0 tx=0).stop 2024-04-20T21:30:09.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.118+0000 7ff8ada00640 1 -- 172.21.15.86:0/3950951551 shutdown_connections 2024-04-20T21:30:09.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.118+0000 7ff8ada00640 1 --2- 172.21.15.86:0/3950951551 >> v2:172.21.15.144:3300/0 conn(0x7ff8a80af5d0 0x7ff8a815de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:09.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.118+0000 7ff8ada00640 1 --2- 172.21.15.86:0/3950951551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8a80af1f0 0x7ff8a806ef20 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:09.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.118+0000 7ff8ada00640 1 -- 172.21.15.86:0/3950951551 >> 172.21.15.86:0/3950951551 conn(0x7ff8a80b1f00 msgr2=0x7ff8a80b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:09.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.118+0000 7ff8ada00640 1 -- 172.21.15.86:0/3950951551 shutdown_connections 2024-04-20T21:30:09.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.118+0000 7ff8ada00640 1 -- 172.21.15.86:0/3950951551 wait complete. 2024-04-20T21:30:09.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.120+0000 7ff8ada00640 1 Processor -- start 2024-04-20T21:30:09.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.120+0000 7ff8ada00640 1 -- start start 2024-04-20T21:30:09.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.121+0000 7ff8ada00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8a806d9c0 0x7ff8a81f5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:09.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.121+0000 7ff8ada00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8a80af1f0 0x7ff8a80735c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:09.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.121+0000 7ff8ada00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8a80af5d0 0x7ff8a815c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:09.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.121+0000 7ff8ada00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a80afa40 con 0x7ff8a80af5d0 2024-04-20T21:30:09.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.121+0000 7ff8ada00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a806ea00 con 0x7ff8a806d9c0 2024-04-20T21:30:09.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.121+0000 7ff8ada00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff8a815a3c0 con 0x7ff8a80af1f0 2024-04-20T21:30:09.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.121+0000 7ff8ad000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8a80af5d0 0x7ff8a815c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:09.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.122+0000 7ff8ad000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8a80af5d0 0x7ff8a815c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59960/0 (socket says 172.21.15.86:59960) 2024-04-20T21:30:09.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.122+0000 7ff8ad000640 1 -- 172.21.15.86:0/2965855304 learned_addr learned my addr 172.21.15.86:0/2965855304 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:09.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.122+0000 7ff89f400640 1 --2- 172.21.15.86:0/2965855304 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8a80af1f0 0x7ff8a80735c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:09.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.122+0000 7ff8a7e00640 1 --2- 172.21.15.86:0/2965855304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8a806d9c0 0x7ff8a81f5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:09.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.122+0000 7ff8ad000640 1 -- 172.21.15.86:0/2965855304 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8a80af1f0 msgr2=0x7ff8a80735c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:09.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.122+0000 7ff8ad000640 1 --2- 172.21.15.86:0/2965855304 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8a80af1f0 0x7ff8a80735c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:09.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.122+0000 7ff8ad000640 1 -- 172.21.15.86:0/2965855304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8a806d9c0 msgr2=0x7ff8a81f5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:09.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.122+0000 7ff8ad000640 1 --2- 172.21.15.86:0/2965855304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8a806d9c0 0x7ff8a81f5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:09.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.122+0000 7ff8ad000640 1 -- 172.21.15.86:0/2965855304 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8a815ee00 con 0x7ff8a80af5d0 2024-04-20T21:30:09.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.122+0000 7ff8a7e00640 1 --2- 172.21.15.86:0/2965855304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8a806d9c0 0x7ff8a81f5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:30:09.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.122+0000 7ff8ad000640 1 --2- 172.21.15.86:0/2965855304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8a80af5d0 0x7ff8a815c9a0 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7ff8a000efc0 tx=0x7ff8a000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:09.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.122+0000 7ff8a4c00640 1 -- 172.21.15.86:0/2965855304 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a0018020 con 0x7ff8a80af5d0 2024-04-20T21:30:09.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.123+0000 7ff8a4c00640 1 -- 172.21.15.86:0/2965855304 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8a0014480 con 0x7ff8a80af5d0 2024-04-20T21:30:09.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.123+0000 7ff8ada00640 1 -- 172.21.15.86:0/2965855304 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8a815f0e0 con 0x7ff8a80af5d0 2024-04-20T21:30:09.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.123+0000 7ff8a4c00640 1 -- 172.21.15.86:0/2965855304 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a0014870 con 0x7ff8a80af5d0 2024-04-20T21:30:09.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.123+0000 7ff8ada00640 1 -- 172.21.15.86:0/2965855304 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8a815f6a0 con 0x7ff8a80af5d0 2024-04-20T21:30:09.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.124+0000 7ff8ada00640 1 -- 172.21.15.86:0/2965855304 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8a804fd90 con 0x7ff8a80af5d0 2024-04-20T21:30:09.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.125+0000 7ff8a4c00640 1 -- 172.21.15.86:0/2965855304 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff8a0005d10 con 0x7ff8a80af5d0 2024-04-20T21:30:09.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.126+0000 7ff8a4c00640 1 --2- 172.21.15.86:0/2965855304 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff88406c8f0 0x7ff88406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:09.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.126+0000 7ff8a4c00640 1 -- 172.21.15.86:0/2965855304 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7ff8a008c780 con 0x7ff8a80af5d0 2024-04-20T21:30:09.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.126+0000 7ff8a7e00640 1 --2- 172.21.15.86:0/2965855304 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff88406c8f0 0x7ff88406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:09.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.127+0000 7ff8a7e00640 1 --2- 172.21.15.86:0/2965855304 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff88406c8f0 0x7ff88406edb0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7ff898036000 tx=0x7ff898008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:09.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.130+0000 7ff8a4c00640 1 -- 172.21.15.86:0/2965855304 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff8a0022070 con 0x7ff8a80af5d0 2024-04-20T21:30:09.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.329+0000 7ff8ada00640 1 -- 172.21.15.86:0/2965855304 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff8a806f880 con 0x7ff8a80af5d0 2024-04-20T21:30:09.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.329+0000 7ff8a4c00640 1 -- 172.21.15.86:0/2965855304 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) ==== 93+0+10 (secure 0 0 0) 0x7ff8a0010040 con 0x7ff8a80af5d0 2024-04-20T21:30:09.328 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:30:09.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.332+0000 7ff8ada00640 1 -- 172.21.15.86:0/2965855304 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff88406c8f0 msgr2=0x7ff88406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:09.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.332+0000 7ff8ada00640 1 --2- 172.21.15.86:0/2965855304 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff88406c8f0 0x7ff88406edb0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7ff898036000 tx=0x7ff898008030 comp rx=0 tx=0).stop 2024-04-20T21:30:09.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.332+0000 7ff8ada00640 1 -- 172.21.15.86:0/2965855304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8a80af5d0 msgr2=0x7ff8a815c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:09.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.332+0000 7ff8ada00640 1 --2- 172.21.15.86:0/2965855304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8a80af5d0 0x7ff8a815c9a0 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7ff8a000efc0 tx=0x7ff8a000c490 comp rx=0 tx=0).stop 2024-04-20T21:30:09.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.332+0000 7ff8ada00640 1 -- 172.21.15.86:0/2965855304 shutdown_connections 2024-04-20T21:30:09.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.332+0000 7ff8ada00640 1 --2- 172.21.15.86:0/2965855304 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff88406c8f0 0x7ff88406edb0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:09.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.332+0000 7ff8ada00640 1 --2- 172.21.15.86:0/2965855304 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8a80af5d0 0x7ff8a815c9a0 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:09.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.332+0000 7ff8ada00640 1 --2- 172.21.15.86:0/2965855304 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff8a80af1f0 0x7ff8a80735c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:09.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.332+0000 7ff8ada00640 1 --2- 172.21.15.86:0/2965855304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8a806d9c0 0x7ff8a81f5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:09.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.332+0000 7ff8ada00640 1 -- 172.21.15.86:0/2965855304 >> 172.21.15.86:0/2965855304 conn(0x7ff8a80b1f00 msgr2=0x7ff8a80d00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:09.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.333+0000 7ff8ada00640 1 -- 172.21.15.86:0/2965855304 shutdown_connections 2024-04-20T21:30:09.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.333+0000 7ff8ada00640 1 -- 172.21.15.86:0/2965855304 wait complete. 2024-04-20T21:30:09.349 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:30:09.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.555+0000 7ff761400640 1 Processor -- start 2024-04-20T21:30:09.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.555+0000 7ff761400640 1 -- start start 2024-04-20T21:30:09.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.556+0000 7ff761400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff75c15add0 0x7ff75c0cf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:09.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.557+0000 7ff761400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff75c0725d0 0x7ff75c0729e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:09.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.557+0000 7ff761400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff75c072f20 con 0x7ff75c06f480 2024-04-20T21:30:09.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.557+0000 7ff761400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff75c06ee90 con 0x7ff75c15add0 2024-04-20T21:30:09.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.557+0000 7ff761400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7ff75c06fc60 con 0x7ff75c0725d0 2024-04-20T21:30:09.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.557+0000 7ff75be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff75c15add0 0x7ff75c0cf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:09.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.557+0000 7ff75be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff75c15add0 0x7ff75c0cf790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59976/0 (socket says 172.21.15.86:59976) 2024-04-20T21:30:09.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.557+0000 7ff75be00640 1 -- 172.21.15.86:0/2840401531 learned_addr learned my addr 172.21.15.86:0/2840401531 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:09.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.557+0000 7ff753400640 1 --1- 172.21.15.86:0/2840401531 >> v1:172.21.15.86:6789/0 conn(0x7ff75c06f480 0x7ff75c06e780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51276/0 (socket says 172.21.15.86:51276) 2024-04-20T21:30:09.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.557+0000 7ff760a00640 1 --2- 172.21.15.86:0/2840401531 >> v2:172.21.15.144:3300/0 conn(0x7ff75c0725d0 0x7ff75c0729e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:09.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.557+0000 7ff75be00640 1 -- 172.21.15.86:0/2840401531 >> v2:172.21.15.144:3300/0 conn(0x7ff75c0725d0 msgr2=0x7ff75c0729e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:09.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.557+0000 7ff75be00640 1 --2- 172.21.15.86:0/2840401531 >> v2:172.21.15.144:3300/0 conn(0x7ff75c0725d0 0x7ff75c0729e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:09.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.557+0000 7ff75be00640 1 -- 172.21.15.86:0/2840401531 >> v1:172.21.15.86:6789/0 conn(0x7ff75c06f480 legacy=0x7ff75c06e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:09.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.558+0000 7ff75be00640 1 -- 172.21.15.86:0/2840401531 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff75c0cea10 con 0x7ff75c15add0 2024-04-20T21:30:09.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.558+0000 7ff760a00640 1 --2- 172.21.15.86:0/2840401531 >> v2:172.21.15.144:3300/0 conn(0x7ff75c0725d0 0x7ff75c0729e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:30:09.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.559+0000 7ff75be00640 1 --2- 172.21.15.86:0/2840401531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff75c15add0 0x7ff75c0cf790 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7ff75c15f850 tx=0x7ff74c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3350a56999f584ee server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:09.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.559+0000 7ff75b400640 1 -- 172.21.15.86:0/2840401531 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74c034070 con 0x7ff75c15add0 2024-04-20T21:30:09.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.559+0000 7ff75b400640 1 -- 172.21.15.86:0/2840401531 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff74c030860 con 0x7ff75c15add0 2024-04-20T21:30:09.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.559+0000 7ff75b400640 1 -- 172.21.15.86:0/2840401531 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74c030c30 con 0x7ff75c15add0 2024-04-20T21:30:09.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.560+0000 7ff761400640 1 -- 172.21.15.86:0/2840401531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff75c15add0 msgr2=0x7ff75c0cf790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:09.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.560+0000 7ff761400640 1 --2- 172.21.15.86:0/2840401531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff75c15add0 0x7ff75c0cf790 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7ff75c15f850 tx=0x7ff74c02b9a0 comp rx=0 tx=0).stop 2024-04-20T21:30:09.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.560+0000 7ff761400640 1 -- 172.21.15.86:0/2840401531 shutdown_connections 2024-04-20T21:30:09.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.560+0000 7ff761400640 1 --2- 172.21.15.86:0/2840401531 >> v2:172.21.15.144:3300/0 conn(0x7ff75c0725d0 0x7ff75c0729e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:09.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.560+0000 7ff761400640 1 --2- 172.21.15.86:0/2840401531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff75c15add0 0x7ff75c0cf790 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:09.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.560+0000 7ff761400640 1 -- 172.21.15.86:0/2840401531 >> 172.21.15.86:0/2840401531 conn(0x7ff75c067cf0 msgr2=0x7ff75c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:09.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.560+0000 7ff761400640 1 -- 172.21.15.86:0/2840401531 shutdown_connections 2024-04-20T21:30:09.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.560+0000 7ff761400640 1 -- 172.21.15.86:0/2840401531 wait complete. 2024-04-20T21:30:09.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.562+0000 7ff761400640 1 Processor -- start 2024-04-20T21:30:09.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.562+0000 7ff761400640 1 -- start start 2024-04-20T21:30:09.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.563+0000 7ff761400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff75c06f480 0x7ff75c14ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:09.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.563+0000 7ff761400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff75c0725d0 0x7ff75c155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:09.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.563+0000 7ff761400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff75c156210 0x7ff75c15db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:09.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.563+0000 7ff761400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff75c0cef90 con 0x7ff75c156210 2024-04-20T21:30:09.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.563+0000 7ff761400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff75c0cecb0 con 0x7ff75c06f480 2024-04-20T21:30:09.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.563+0000 7ff761400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff75c0cf270 con 0x7ff75c0725d0 2024-04-20T21:30:09.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.563+0000 7ff760a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff75c156210 0x7ff75c15db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:09.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.563+0000 7ff760a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff75c156210 0x7ff75c15db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59982/0 (socket says 172.21.15.86:59982) 2024-04-20T21:30:09.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.563+0000 7ff760a00640 1 -- 172.21.15.86:0/3710217578 learned_addr learned my addr 172.21.15.86:0/3710217578 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:09.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.563+0000 7ff753400640 1 --2- 172.21.15.86:0/3710217578 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff75c0725d0 0x7ff75c155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:09.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.563+0000 7ff75be00640 1 --2- 172.21.15.86:0/3710217578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff75c06f480 0x7ff75c14ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:09.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.563+0000 7ff760a00640 1 -- 172.21.15.86:0/3710217578 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff75c0725d0 msgr2=0x7ff75c155cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:09.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.563+0000 7ff760a00640 1 --2- 172.21.15.86:0/3710217578 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff75c0725d0 0x7ff75c155cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:09.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.563+0000 7ff760a00640 1 -- 172.21.15.86:0/3710217578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff75c06f480 msgr2=0x7ff75c14ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:09.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.563+0000 7ff760a00640 1 --2- 172.21.15.86:0/3710217578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff75c06f480 0x7ff75c14ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:09.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.563+0000 7ff760a00640 1 -- 172.21.15.86:0/3710217578 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff75c14f350 con 0x7ff75c156210 2024-04-20T21:30:09.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.564+0000 7ff760a00640 1 --2- 172.21.15.86:0/3710217578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff75c156210 0x7ff75c15db60 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7ff75400c7f0 tx=0x7ff75400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:09.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.564+0000 7ff758c00640 1 -- 172.21.15.86:0/3710217578 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff754015070 con 0x7ff75c156210 2024-04-20T21:30:09.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.564+0000 7ff758c00640 1 -- 172.21.15.86:0/3710217578 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff754007820 con 0x7ff75c156210 2024-04-20T21:30:09.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.564+0000 7ff758c00640 1 -- 172.21.15.86:0/3710217578 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff754007c10 con 0x7ff75c156210 2024-04-20T21:30:09.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.564+0000 7ff761400640 1 -- 172.21.15.86:0/3710217578 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff75c158380 con 0x7ff75c156210 2024-04-20T21:30:09.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.564+0000 7ff761400640 1 -- 172.21.15.86:0/3710217578 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff75c158940 con 0x7ff75c156210 2024-04-20T21:30:09.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.566+0000 7ff761400640 1 -- 172.21.15.86:0/3710217578 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff75c156690 con 0x7ff75c156210 2024-04-20T21:30:09.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.566+0000 7ff758c00640 1 -- 172.21.15.86:0/3710217578 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff754018070 con 0x7ff75c156210 2024-04-20T21:30:09.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.567+0000 7ff758c00640 1 --2- 172.21.15.86:0/3710217578 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff73806c8f0 0x7ff73806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:09.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.567+0000 7ff758c00640 1 -- 172.21.15.86:0/3710217578 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7ff75408bc20 con 0x7ff75c156210 2024-04-20T21:30:09.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.568+0000 7ff75be00640 1 --2- 172.21.15.86:0/3710217578 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff73806c8f0 0x7ff73806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:09.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.568+0000 7ff75be00640 1 --2- 172.21.15.86:0/3710217578 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff73806c8f0 0x7ff73806edb0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7ff74c036000 tx=0x7ff74c03b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:09.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.572+0000 7ff758c00640 1 -- 172.21.15.86:0/3710217578 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff75c14f350 con 0x7ff75c156210 2024-04-20T21:30:09.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.772+0000 7ff761400640 1 -- 172.21.15.86:0/3710217578 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff75c04fd90 con 0x7ff75c156210 2024-04-20T21:30:09.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.773+0000 7ff758c00640 1 -- 172.21.15.86:0/3710217578 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v112) ==== 104+0+11 (secure 0 0 0) 0x7ff75405a0e0 con 0x7ff75c156210 2024-04-20T21:30:09.772 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-20T21:30:09.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.776+0000 7ff761400640 1 -- 172.21.15.86:0/3710217578 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff73806c8f0 msgr2=0x7ff73806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:09.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.776+0000 7ff761400640 1 --2- 172.21.15.86:0/3710217578 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff73806c8f0 0x7ff73806edb0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7ff74c036000 tx=0x7ff74c03b040 comp rx=0 tx=0).stop 2024-04-20T21:30:09.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.776+0000 7ff761400640 1 -- 172.21.15.86:0/3710217578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff75c156210 msgr2=0x7ff75c15db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:09.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.776+0000 7ff761400640 1 --2- 172.21.15.86:0/3710217578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff75c156210 0x7ff75c15db60 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7ff75400c7f0 tx=0x7ff75400ccb0 comp rx=0 tx=0).stop 2024-04-20T21:30:09.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.776+0000 7ff761400640 1 -- 172.21.15.86:0/3710217578 shutdown_connections 2024-04-20T21:30:09.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.776+0000 7ff761400640 1 --2- 172.21.15.86:0/3710217578 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff73806c8f0 0x7ff73806edb0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:09.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.776+0000 7ff761400640 1 --2- 172.21.15.86:0/3710217578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff75c156210 0x7ff75c15db60 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:09.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.776+0000 7ff761400640 1 --2- 172.21.15.86:0/3710217578 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff75c0725d0 0x7ff75c155cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:09.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.776+0000 7ff761400640 1 --2- 172.21.15.86:0/3710217578 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff75c06f480 0x7ff75c14ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:09.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.776+0000 7ff761400640 1 -- 172.21.15.86:0/3710217578 >> 172.21.15.86:0/3710217578 conn(0x7ff75c067cf0 msgr2=0x7ff75c150bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:09.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.776+0000 7ff761400640 1 -- 172.21.15.86:0/3710217578 shutdown_connections 2024-04-20T21:30:09.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.776+0000 7ff761400640 1 -- 172.21.15.86:0/3710217578 wait complete. 2024-04-20T21:30:09.794 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:30:09.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.997+0000 7f9ac4600640 1 Processor -- start 2024-04-20T21:30:09.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.998+0000 7f9ac4600640 1 -- start start 2024-04-20T21:30:09.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.999+0000 7f9ac4600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9abc15ee00 0x7f9abc1634c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:09.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.999+0000 7f9ac4600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9abc0d1260 0x7f9abc0d16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:09.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.999+0000 7f9ac4600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9abc0cff90 con 0x7f9abc0cf830 2024-04-20T21:30:09.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.999+0000 7f9ac4600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f9abc0d0550 con 0x7f9abc0d1260 2024-04-20T21:30:09.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:09.999+0000 7f9ac4600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9abc0d0270 con 0x7f9abc15ee00 2024-04-20T21:30:09.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.000+0000 7f9ac1c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9abc15ee00 0x7f9abc1634c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:09.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.000+0000 7f9ac1c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9abc15ee00 0x7f9abc1634c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:57330/0 (socket says 172.21.15.86:57330) 2024-04-20T21:30:09.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.000+0000 7f9ac1c00640 1 -- 172.21.15.86:0/3242290445 learned_addr learned my addr 172.21.15.86:0/3242290445 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:09.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.000+0000 7f9ac2600640 1 --2- 172.21.15.86:0/3242290445 >> v2:172.21.15.144:3300/0 conn(0x7f9abc0d1260 0x7f9abc0d16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:10.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.001+0000 7f9ac1c00640 1 -- 172.21.15.86:0/3242290445 >> v2:172.21.15.144:3300/0 conn(0x7f9abc0d1260 msgr2=0x7f9abc0d16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:10.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.001+0000 7f9ac1c00640 1 --2- 172.21.15.86:0/3242290445 >> v2:172.21.15.144:3300/0 conn(0x7f9abc0d1260 0x7f9abc0d16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:10.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.001+0000 7f9ac1c00640 1 -- 172.21.15.86:0/3242290445 >> v1:172.21.15.144:6789/0 conn(0x7f9abc0cf830 legacy=0x7f9abc15fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:10.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.001+0000 7f9abbe00640 1 -- 172.21.15.86:0/3242290445 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 934473543 0 0) 0x7f9abc0cff90 con 0x7f9abc0cf830 2024-04-20T21:30:10.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.001+0000 7f9ac1c00640 1 -- 172.21.15.86:0/3242290445 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9abc15f640 con 0x7f9abc15ee00 2024-04-20T21:30:10.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.001+0000 7f9ac2600640 1 --2- 172.21.15.86:0/3242290445 >> v2:172.21.15.144:3300/0 conn(0x7f9abc0d1260 0x7f9abc0d16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:30:10.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.003+0000 7f9ac1c00640 1 --2- 172.21.15.86:0/3242290445 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9abc15ee00 0x7f9abc1634c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9ab0009870 tx=0x7f9ab002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d391029edc458877 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:10.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.003+0000 7f9abbe00640 1 -- 172.21.15.86:0/3242290445 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab0034070 con 0x7f9abc15ee00 2024-04-20T21:30:10.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.003+0000 7f9abbe00640 1 -- 172.21.15.86:0/3242290445 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ab0030840 con 0x7f9abc15ee00 2024-04-20T21:30:10.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.003+0000 7f9abbe00640 1 -- 172.21.15.86:0/3242290445 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab0030c10 con 0x7f9abc15ee00 2024-04-20T21:30:10.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.004+0000 7f9ac4600640 1 -- 172.21.15.86:0/3242290445 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9abc15ee00 msgr2=0x7f9abc1634c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:10.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.004+0000 7f9ac4600640 1 --2- 172.21.15.86:0/3242290445 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9abc15ee00 0x7f9abc1634c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9ab0009870 tx=0x7f9ab002b980 comp rx=0 tx=0).stop 2024-04-20T21:30:10.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.004+0000 7f9ac4600640 1 -- 172.21.15.86:0/3242290445 shutdown_connections 2024-04-20T21:30:10.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.004+0000 7f9ac4600640 1 --2- 172.21.15.86:0/3242290445 >> v2:172.21.15.144:3300/0 conn(0x7f9abc0d1260 0x7f9abc0d16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:10.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.004+0000 7f9ac4600640 1 --2- 172.21.15.86:0/3242290445 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9abc15ee00 0x7f9abc1634c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:10.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.004+0000 7f9ac4600640 1 -- 172.21.15.86:0/3242290445 >> 172.21.15.86:0/3242290445 conn(0x7f9abc06cc20 msgr2=0x7f9abc06d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:10.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.004+0000 7f9ac4600640 1 -- 172.21.15.86:0/3242290445 shutdown_connections 2024-04-20T21:30:10.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.004+0000 7f9ac4600640 1 -- 172.21.15.86:0/3242290445 wait complete. 2024-04-20T21:30:10.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.006+0000 7f9ac4600640 1 Processor -- start 2024-04-20T21:30:10.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.007+0000 7f9ac4600640 1 -- start start 2024-04-20T21:30:10.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.008+0000 7f9ac4600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9abc0cf830 0x7f9abc152a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:10.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.008+0000 7f9ac4600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9abc0d1260 0x7f9abc0b8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:10.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.008+0000 7f9ac4600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9abc15ee00 0x7f9abc0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:10.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.008+0000 7f9ac4600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9abc15f360 con 0x7f9abc0cf830 2024-04-20T21:30:10.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.008+0000 7f9ac4600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9abc15f1e0 con 0x7f9abc0d1260 2024-04-20T21:30:10.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.008+0000 7f9ac4600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9abc0d3040 con 0x7f9abc15ee00 2024-04-20T21:30:10.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.008+0000 7f9ac1c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9abc0cf830 0x7f9abc152a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:10.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.008+0000 7f9ac1c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9abc0cf830 0x7f9abc152a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59992/0 (socket says 172.21.15.86:59992) 2024-04-20T21:30:10.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.008+0000 7f9ac1c00640 1 -- 172.21.15.86:0/3936105636 learned_addr learned my addr 172.21.15.86:0/3936105636 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:10.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.009+0000 7f9ac2600640 1 --2- 172.21.15.86:0/3936105636 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9abc15ee00 0x7f9abc0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:10.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.009+0000 7f9ac1200640 1 --2- 172.21.15.86:0/3936105636 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9abc0d1260 0x7f9abc0b8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:10.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.009+0000 7f9ac1c00640 1 -- 172.21.15.86:0/3936105636 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9abc15ee00 msgr2=0x7f9abc0b8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:10.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.009+0000 7f9ac1c00640 1 --2- 172.21.15.86:0/3936105636 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9abc15ee00 0x7f9abc0b8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:10.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.009+0000 7f9ac1c00640 1 -- 172.21.15.86:0/3936105636 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9abc0d1260 msgr2=0x7f9abc0b8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:10.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.009+0000 7f9ac1c00640 1 --2- 172.21.15.86:0/3936105636 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9abc0d1260 0x7f9abc0b8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:10.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.009+0000 7f9ac1c00640 1 -- 172.21.15.86:0/3936105636 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9abc156b40 con 0x7f9abc0cf830 2024-04-20T21:30:10.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.009+0000 7f9ac2600640 1 --2- 172.21.15.86:0/3936105636 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9abc15ee00 0x7f9abc0b8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:30:10.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.009+0000 7f9ac1200640 1 --2- 172.21.15.86:0/3936105636 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9abc0d1260 0x7f9abc0b8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:30:10.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.009+0000 7f9ac1c00640 1 --2- 172.21.15.86:0/3936105636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9abc0cf830 0x7f9abc152a30 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f9ab0000c00 tx=0x7f9ab00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:10.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.009+0000 7f9ab9600640 1 -- 172.21.15.86:0/3936105636 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab0034040 con 0x7f9abc0cf830 2024-04-20T21:30:10.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.009+0000 7f9ab9600640 1 -- 172.21.15.86:0/3936105636 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ab003ab90 con 0x7f9abc0cf830 2024-04-20T21:30:10.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.009+0000 7f9ab9600640 1 -- 172.21.15.86:0/3936105636 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab0004030 con 0x7f9abc0cf830 2024-04-20T21:30:10.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.009+0000 7f9ac4600640 1 -- 172.21.15.86:0/3936105636 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9abc156860 con 0x7f9abc0cf830 2024-04-20T21:30:10.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.010+0000 7f9ac4600640 1 -- 172.21.15.86:0/3936105636 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9abc0ce3e0 con 0x7f9abc0cf830 2024-04-20T21:30:10.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.011+0000 7f9ac4600640 1 -- 172.21.15.86:0/3936105636 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9abc0689e0 con 0x7f9abc0cf830 2024-04-20T21:30:10.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.012+0000 7f9ab9600640 1 -- 172.21.15.86:0/3936105636 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9ab004b060 con 0x7f9abc0cf830 2024-04-20T21:30:10.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.013+0000 7f9ab9600640 1 --2- 172.21.15.86:0/3936105636 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9aa006c8f0 0x7f9aa006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:10.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.013+0000 7f9ab9600640 1 -- 172.21.15.86:0/3936105636 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7f9ab00ad900 con 0x7f9abc0cf830 2024-04-20T21:30:10.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.014+0000 7f9ac1200640 1 --2- 172.21.15.86:0/3936105636 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9aa006c8f0 0x7f9aa006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:10.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.014+0000 7f9ac1200640 1 --2- 172.21.15.86:0/3936105636 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9aa006c8f0 0x7f9aa006edb0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f9abc0ba4c0 tx=0x7f9aac009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:10.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.017+0000 7f9ab9600640 1 -- 172.21.15.86:0/3936105636 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9ab003f070 con 0x7f9abc0cf830 2024-04-20T21:30:10.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.214+0000 7f9ac4600640 1 -- 172.21.15.86:0/3936105636 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9abc0d29c0 con 0x7f9abc0cf830 2024-04-20T21:30:10.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.215+0000 7f9ab9600640 1 -- 172.21.15.86:0/3936105636 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v112) ==== 100+0+11 (secure 0 0 0) 0x7f9ab00ad020 con 0x7f9abc0cf830 2024-04-20T21:30:10.214 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-20T21:30:10.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.218+0000 7f9ac4600640 1 -- 172.21.15.86:0/3936105636 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9aa006c8f0 msgr2=0x7f9aa006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:10.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.218+0000 7f9ac4600640 1 --2- 172.21.15.86:0/3936105636 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9aa006c8f0 0x7f9aa006edb0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f9abc0ba4c0 tx=0x7f9aac009040 comp rx=0 tx=0).stop 2024-04-20T21:30:10.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.218+0000 7f9ac4600640 1 -- 172.21.15.86:0/3936105636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9abc0cf830 msgr2=0x7f9abc152a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:10.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.218+0000 7f9ac4600640 1 --2- 172.21.15.86:0/3936105636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9abc0cf830 0x7f9abc152a30 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f9ab0000c00 tx=0x7f9ab00046c0 comp rx=0 tx=0).stop 2024-04-20T21:30:10.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.218+0000 7f9ac4600640 1 -- 172.21.15.86:0/3936105636 shutdown_connections 2024-04-20T21:30:10.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.218+0000 7f9ac4600640 1 --2- 172.21.15.86:0/3936105636 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9aa006c8f0 0x7f9aa006edb0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:10.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.218+0000 7f9ac4600640 1 --2- 172.21.15.86:0/3936105636 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9abc15ee00 0x7f9abc0b8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:10.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.218+0000 7f9ac4600640 1 --2- 172.21.15.86:0/3936105636 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9abc0d1260 0x7f9abc0b8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:10.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.218+0000 7f9ac4600640 1 --2- 172.21.15.86:0/3936105636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9abc0cf830 0x7f9abc152a30 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:10.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.218+0000 7f9ac4600640 1 -- 172.21.15.86:0/3936105636 >> 172.21.15.86:0/3936105636 conn(0x7f9abc06cc20 msgr2=0x7f9abc074ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:10.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.218+0000 7f9ac4600640 1 -- 172.21.15.86:0/3936105636 shutdown_connections 2024-04-20T21:30:10.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.218+0000 7f9ac4600640 1 -- 172.21.15.86:0/3936105636 wait complete. 2024-04-20T21:30:10.235 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T21:30:10.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.441+0000 7f9c16000640 1 Processor -- start 2024-04-20T21:30:10.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.442+0000 7f9c16000640 1 -- start start 2024-04-20T21:30:10.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.443+0000 7f9c16000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9c100d0ee0 0x7f9c100d1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:10.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.443+0000 7f9c16000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c100d3100 con 0x7f9c100d25a0 2024-04-20T21:30:10.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.443+0000 7f9c16000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c100d33b0 con 0x7f9c1015e690 2024-04-20T21:30:10.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.443+0000 7f9c16000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9c100d18b0 con 0x7f9c100d0ee0 2024-04-20T21:30:10.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.443+0000 7f9c0fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9c100d0ee0 0x7f9c100d1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:10.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.443+0000 7f9c0fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9c100d0ee0 0x7f9c100d1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60000/0 (socket says 172.21.15.86:60000) 2024-04-20T21:30:10.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.443+0000 7f9c15600640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f9c100d25a0 0x7f9c100d29f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51286/0 (socket says 172.21.15.86:51286) 2024-04-20T21:30:10.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.443+0000 7f9c0fe00640 1 -- 172.21.15.86:0/2134811953 learned_addr learned my addr 172.21.15.86:0/2134811953 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:10.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.444+0000 7f9c0fe00640 1 -- 172.21.15.86:0/2134811953 >> v1:172.21.15.144:6789/0 conn(0x7f9c1015e690 legacy=0x7f9c1006ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:10.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.444+0000 7f9c0fe00640 1 -- 172.21.15.86:0/2134811953 >> v1:172.21.15.86:6789/0 conn(0x7f9c100d25a0 legacy=0x7f9c100d29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:10.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.444+0000 7f9c0fe00640 1 -- 172.21.15.86:0/2134811953 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c100d3690 con 0x7f9c100d0ee0 2024-04-20T21:30:10.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.445+0000 7f9c0fe00640 1 --2- 172.21.15.86:0/2134811953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c100d0ee0 0x7f9c100d1370 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8009850 tx=0x7f9bf802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da3539884fd6eb46 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:10.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.445+0000 7f9c0f400640 1 -- 172.21.15.86:0/2134811953 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf8004230 con 0x7f9c100d0ee0 2024-04-20T21:30:10.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.445+0000 7f9c0f400640 1 -- 172.21.15.86:0/2134811953 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bf80043d0 con 0x7f9c100d0ee0 2024-04-20T21:30:10.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.445+0000 7f9c0f400640 1 -- 172.21.15.86:0/2134811953 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf8030840 con 0x7f9c100d0ee0 2024-04-20T21:30:10.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.446+0000 7f9c16000640 1 -- 172.21.15.86:0/2134811953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c100d0ee0 msgr2=0x7f9c100d1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:10.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.446+0000 7f9c16000640 1 --2- 172.21.15.86:0/2134811953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c100d0ee0 0x7f9c100d1370 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8009850 tx=0x7f9bf802b980 comp rx=0 tx=0).stop 2024-04-20T21:30:10.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.446+0000 7f9c16000640 1 -- 172.21.15.86:0/2134811953 shutdown_connections 2024-04-20T21:30:10.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.446+0000 7f9c16000640 1 --2- 172.21.15.86:0/2134811953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c100d0ee0 0x7f9c100d1370 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:10.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.446+0000 7f9c16000640 1 -- 172.21.15.86:0/2134811953 >> 172.21.15.86:0/2134811953 conn(0x7f9c1009d8a0 msgr2=0x7f9c1008bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:10.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.446+0000 7f9c16000640 1 -- 172.21.15.86:0/2134811953 shutdown_connections 2024-04-20T21:30:10.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.446+0000 7f9c16000640 1 -- 172.21.15.86:0/2134811953 wait complete. 2024-04-20T21:30:10.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.448+0000 7f9c16000640 1 Processor -- start 2024-04-20T21:30:10.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.448+0000 7f9c16000640 1 -- start start 2024-04-20T21:30:10.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.449+0000 7f9c16000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9c100d0ee0 0x7f9c10156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:10.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.449+0000 7f9c16000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c100d25a0 0x7f9c10150670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:10.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.449+0000 7f9c16000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c1015e690 0x7f9c100b2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:10.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.449+0000 7f9c16000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9c100cef00 con 0x7f9c1015e690 2024-04-20T21:30:10.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.449+0000 7f9c16000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9c100cec20 con 0x7f9c100d25a0 2024-04-20T21:30:10.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.449+0000 7f9c16000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9c1015ea70 con 0x7f9c100d0ee0 2024-04-20T21:30:10.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.449+0000 7f9c15600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c1015e690 0x7f9c100b2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:10.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.449+0000 7f9c15600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c1015e690 0x7f9c100b2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60006/0 (socket says 172.21.15.86:60006) 2024-04-20T21:30:10.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.449+0000 7f9c15600640 1 -- 172.21.15.86:0/608029672 learned_addr learned my addr 172.21.15.86:0/608029672 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:10.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.449+0000 7f9c0fe00640 1 --2- 172.21.15.86:0/608029672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c100d25a0 0x7f9c10150670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:10.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.449+0000 7f9c15600640 1 -- 172.21.15.86:0/608029672 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9c100d0ee0 msgr2=0x7f9c10156ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:10.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.449+0000 7f9c15600640 1 --2- 172.21.15.86:0/608029672 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9c100d0ee0 0x7f9c10156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:10.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.449+0000 7f9c15600640 1 -- 172.21.15.86:0/608029672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c100d25a0 msgr2=0x7f9c10150670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:10.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.449+0000 7f9c15600640 1 --2- 172.21.15.86:0/608029672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c100d25a0 0x7f9c10150670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:10.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.449+0000 7f9c15600640 1 -- 172.21.15.86:0/608029672 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c10150bb0 con 0x7f9c1015e690 2024-04-20T21:30:10.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.450+0000 7f9c15600640 1 --2- 172.21.15.86:0/608029672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c1015e690 0x7f9c100b2d90 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800c400 tx=0x7f9c0800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:10.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.450+0000 7f9c0cc00640 1 -- 172.21.15.86:0/608029672 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c080074e0 con 0x7f9c1015e690 2024-04-20T21:30:10.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.450+0000 7f9c0cc00640 1 -- 172.21.15.86:0/608029672 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c08007680 con 0x7f9c1015e690 2024-04-20T21:30:10.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.450+0000 7f9c0cc00640 1 -- 172.21.15.86:0/608029672 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c080049e0 con 0x7f9c1015e690 2024-04-20T21:30:10.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.450+0000 7f9c16000640 1 -- 172.21.15.86:0/608029672 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c10155de0 con 0x7f9c1015e690 2024-04-20T21:30:10.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.450+0000 7f9c16000640 1 -- 172.21.15.86:0/608029672 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c101563a0 con 0x7f9c1015e690 2024-04-20T21:30:10.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.452+0000 7f9c16000640 1 -- 172.21.15.86:0/608029672 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c1004fcf0 con 0x7f9c1015e690 2024-04-20T21:30:10.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.452+0000 7f9c0cc00640 1 -- 172.21.15.86:0/608029672 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9c0801a020 con 0x7f9c1015e690 2024-04-20T21:30:10.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.453+0000 7f9c0cc00640 1 --2- 172.21.15.86:0/608029672 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9bec06c920 0x7f9bec06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:10.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.453+0000 7f9c0cc00640 1 -- 172.21.15.86:0/608029672 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7f9c0808b2a0 con 0x7f9c1015e690 2024-04-20T21:30:10.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.454+0000 7f9c14c00640 1 --2- 172.21.15.86:0/608029672 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9bec06c920 0x7f9bec06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:10.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.455+0000 7f9c14c00640 1 --2- 172.21.15.86:0/608029672 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9bec06c920 0x7f9bec06ede0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f9c100b40b0 tx=0x7f9c00009230 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:10.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.458+0000 7f9c0cc00640 1 -- 172.21.15.86:0/608029672 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9c08059760 con 0x7f9c1015e690 2024-04-20T21:30:10.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.667+0000 7f9c16000640 1 -- 172.21.15.86:0/608029672 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9c1015bc80 con 0x7f9c1015e690 2024-04-20T21:30:10.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.668+0000 7f9c0cc00640 1 -- 172.21.15.86:0/608029672 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v112) ==== 80+0+0 (secure 0 0 0) 0x7f9c08059060 con 0x7f9c1015e690 2024-04-20T21:30:10.667 INFO:teuthology.orchestra.run.smithi086.stderr:listed 0 entries 2024-04-20T21:30:10.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.671+0000 7f9c16000640 1 -- 172.21.15.86:0/608029672 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9bec06c920 msgr2=0x7f9bec06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:10.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.671+0000 7f9c16000640 1 --2- 172.21.15.86:0/608029672 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9bec06c920 0x7f9bec06ede0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f9c100b40b0 tx=0x7f9c00009230 comp rx=0 tx=0).stop 2024-04-20T21:30:10.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.671+0000 7f9c16000640 1 -- 172.21.15.86:0/608029672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c1015e690 msgr2=0x7f9c100b2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:10.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.671+0000 7f9c16000640 1 --2- 172.21.15.86:0/608029672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c1015e690 0x7f9c100b2d90 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800c400 tx=0x7f9c0800c8c0 comp rx=0 tx=0).stop 2024-04-20T21:30:10.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.671+0000 7f9c16000640 1 -- 172.21.15.86:0/608029672 shutdown_connections 2024-04-20T21:30:10.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.671+0000 7f9c16000640 1 --2- 172.21.15.86:0/608029672 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9bec06c920 0x7f9bec06ede0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:10.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.671+0000 7f9c16000640 1 --2- 172.21.15.86:0/608029672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c1015e690 0x7f9c100b2d90 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:10.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.671+0000 7f9c16000640 1 --2- 172.21.15.86:0/608029672 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c100d25a0 0x7f9c10150670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:10.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.671+0000 7f9c16000640 1 --2- 172.21.15.86:0/608029672 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9c100d0ee0 0x7f9c10156ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:10.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.671+0000 7f9c16000640 1 -- 172.21.15.86:0/608029672 >> 172.21.15.86:0/608029672 conn(0x7f9c1009d8a0 msgr2=0x7f9c1006e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:10.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.671+0000 7f9c16000640 1 -- 172.21.15.86:0/608029672 shutdown_connections 2024-04-20T21:30:10.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.671+0000 7f9c16000640 1 -- 172.21.15.86:0/608029672 wait complete. 2024-04-20T21:30:10.688 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:30:10.688 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:30:10.737 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:30:10.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.980+0000 7f063da00640 1 Processor -- start 2024-04-20T21:30:10.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.980+0000 7f063da00640 1 -- start start 2024-04-20T21:30:10.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.981+0000 7f063da00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0638162dc0 0x7f0638072b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:10.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.982+0000 7f063da00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f063806e640 con 0x7f06380b3810 2024-04-20T21:30:10.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.982+0000 7f063da00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f063806e920 con 0x7f06381629e0 2024-04-20T21:30:10.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.982+0000 7f063da00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f063806ec00 con 0x7f0638162dc0 2024-04-20T21:30:10.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.982+0000 7f063d000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f06380b3810 0x7f06380ad0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51302/0 (socket says 172.21.15.86:51302) 2024-04-20T21:30:10.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.982+0000 7f063d000640 1 -- 172.21.15.86:0/2165781052 learned_addr learned my addr 172.21.15.86:0/2165781052 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:30:10.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.982+0000 7f062fe00640 1 --2- 172.21.15.86:0/2165781052 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0638162dc0 0x7f0638072b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:10.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.982+0000 7f0637400640 1 -- 172.21.15.86:0/2165781052 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2728126107 0 0) 0x7f063806e640 con 0x7f06380b3810 2024-04-20T21:30:10.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.982+0000 7f0637400640 1 -- 172.21.15.86:0/2165781052 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f061c003520 con 0x7f06380b3810 2024-04-20T21:30:10.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.983+0000 7f0637400640 1 -- 172.21.15.86:0/2165781052 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3492864791 0 0) 0x7f061c003520 con 0x7f06380b3810 2024-04-20T21:30:10.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.983+0000 7f0637400640 1 -- 172.21.15.86:0/2165781052 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0638162dc0 msgr2=0x7f0638072b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:10.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.983+0000 7f0637400640 1 --2- 172.21.15.86:0/2165781052 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0638162dc0 0x7f0638072b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:10.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.983+0000 7f0637400640 1 -- 172.21.15.86:0/2165781052 >> v1:172.21.15.144:6789/0 conn(0x7f06381629e0 legacy=0x7f06380b3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:10.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.983+0000 7f0637400640 1 -- 172.21.15.86:0/2165781052 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f063806eee0 con 0x7f06380b3810 2024-04-20T21:30:10.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.983+0000 7f0637400640 1 -- 172.21.15.86:0/2165781052 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f06300019f0 con 0x7f06380b3810 2024-04-20T21:30:10.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.983+0000 7f0637400640 1 -- 172.21.15.86:0/2165781052 >> v1:172.21.15.86:6789/0 conn(0x7f06380b3810 legacy=0x7f06380ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:10.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.983+0000 7f0637400640 1 --2- 172.21.15.86:0/2165781052 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f061c0039e0 0x7f061c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:10.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.983+0000 7f0637400640 1 --2- 172.21.15.86:0/2165781052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f061c004340 0x7f061c004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:10.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.983+0000 7f0637400640 1 --2- 172.21.15.86:0/2165781052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f061c005020 0x7f061c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:10.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.983+0000 7f0637400640 1 -- 172.21.15.86:0/2165781052 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f063806ec00 con 0x7f061c005020 2024-04-20T21:30:10.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.983+0000 7f0637400640 1 -- 172.21.15.86:0/2165781052 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f061c00bf30 con 0x7f061c004340 2024-04-20T21:30:10.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.983+0000 7f0637400640 1 -- 172.21.15.86:0/2165781052 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f061c00c130 con 0x7f061c0039e0 2024-04-20T21:30:10.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.983+0000 7f063d000640 1 --2- 172.21.15.86:0/2165781052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f061c005020 0x7f061c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:10.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.983+0000 7f0637e00640 1 --2- 172.21.15.86:0/2165781052 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f061c0039e0 0x7f061c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:10.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.984+0000 7f062fe00640 1 --2- 172.21.15.86:0/2165781052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f061c004340 0x7f061c004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:10.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.984+0000 7f063d000640 1 -- 172.21.15.86:0/2165781052 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f061c0039e0 msgr2=0x7f061c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:10.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.984+0000 7f063d000640 1 --2- 172.21.15.86:0/2165781052 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f061c0039e0 0x7f061c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:10.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.984+0000 7f063d000640 1 -- 172.21.15.86:0/2165781052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f061c004340 msgr2=0x7f061c004820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:10.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.984+0000 7f063d000640 1 --2- 172.21.15.86:0/2165781052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f061c004340 0x7f061c004820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:10.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.984+0000 7f063d000640 1 -- 172.21.15.86:0/2165781052 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f061c003520 con 0x7f061c005020 2024-04-20T21:30:10.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.984+0000 7f062fe00640 1 --2- 172.21.15.86:0/2165781052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f061c004340 0x7f061c004820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:30:10.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.985+0000 7f063d000640 1 --2- 172.21.15.86:0/2165781052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f061c005020 0x7f061c00b9c0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f063000a840 tx=0x7f061c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a584094268fb9a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:10.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.985+0000 7f0637400640 1 -- 172.21.15.86:0/2165781052 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063000a460 con 0x7f061c005020 2024-04-20T21:30:10.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.985+0000 7f0637400640 1 -- 172.21.15.86:0/2165781052 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0630034650 con 0x7f061c005020 2024-04-20T21:30:10.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.985+0000 7f0637400640 1 -- 172.21.15.86:0/2165781052 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0630034a80 con 0x7f061c005020 2024-04-20T21:30:10.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.986+0000 7f063da00640 1 -- 172.21.15.86:0/2165781052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f061c005020 msgr2=0x7f061c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:10.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.986+0000 7f063da00640 1 --2- 172.21.15.86:0/2165781052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f061c005020 0x7f061c00b9c0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f063000a840 tx=0x7f061c005830 comp rx=0 tx=0).stop 2024-04-20T21:30:10.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.986+0000 7f063d000640 1 -- 172.21.15.86:0/2165781052 reap_dead start 2024-04-20T21:30:10.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.986+0000 7f063da00640 1 -- 172.21.15.86:0/2165781052 shutdown_connections 2024-04-20T21:30:10.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.986+0000 7f063da00640 1 -- 172.21.15.86:0/2165781052 >> 172.21.15.86:0/2165781052 conn(0x7f06380d0170 msgr2=0x7f06380d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:10.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.986+0000 7f063da00640 1 -- 172.21.15.86:0/2165781052 shutdown_connections 2024-04-20T21:30:10.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.986+0000 7f063da00640 1 -- 172.21.15.86:0/2165781052 wait complete. 2024-04-20T21:30:10.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.988+0000 7f063da00640 1 Processor -- start 2024-04-20T21:30:10.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.988+0000 7f063da00640 1 -- start start 2024-04-20T21:30:10.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.989+0000 7f063da00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f061c005020 0x7f063815d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:10.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.989+0000 7f063da00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06381f9850 0x7f06381fb950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:10.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.989+0000 7f063da00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06381f9cd0 0x7f06381fbe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:10.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.989+0000 7f063da00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f063806f860 con 0x7f06381f9850 2024-04-20T21:30:10.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.989+0000 7f063da00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f063806f6e0 con 0x7f061c005020 2024-04-20T21:30:10.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.989+0000 7f063da00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f063806f9e0 con 0x7f06381f9cd0 2024-04-20T21:30:10.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.989+0000 7f062fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06381f9850 0x7f06381fb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:10.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.989+0000 7f062fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06381f9850 0x7f06381fb950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60034/0 (socket says 172.21.15.86:60034) 2024-04-20T21:30:10.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.989+0000 7f062fe00640 1 -- 172.21.15.86:0/2815610394 learned_addr learned my addr 172.21.15.86:0/2815610394 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:10.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.990+0000 7f063d000640 1 --2- 172.21.15.86:0/2815610394 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06381f9cd0 0x7f06381fbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:10.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.990+0000 7f0637e00640 1 --2- 172.21.15.86:0/2815610394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f061c005020 0x7f063815d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:10.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.990+0000 7f062fe00640 1 -- 172.21.15.86:0/2815610394 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06381f9cd0 msgr2=0x7f06381fbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:10.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.990+0000 7f062fe00640 1 --2- 172.21.15.86:0/2815610394 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06381f9cd0 0x7f06381fbe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:10.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.990+0000 7f062fe00640 1 -- 172.21.15.86:0/2815610394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f061c005020 msgr2=0x7f063815d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:10.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.990+0000 7f062fe00640 1 --2- 172.21.15.86:0/2815610394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f061c005020 0x7f063815d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:10.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.990+0000 7f062fe00640 1 -- 172.21.15.86:0/2815610394 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0638163060 con 0x7f06381f9850 2024-04-20T21:30:10.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.990+0000 7f062fe00640 1 --2- 172.21.15.86:0/2815610394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06381f9850 0x7f06381fb950 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f06200108f0 tx=0x7f0620010cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:10.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.990+0000 7f0634c00640 1 -- 172.21.15.86:0/2815610394 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062000f040 con 0x7f06381f9850 2024-04-20T21:30:10.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.990+0000 7f0634c00640 1 -- 172.21.15.86:0/2815610394 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f062000eb60 con 0x7f06381f9850 2024-04-20T21:30:10.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.990+0000 7f063da00640 1 -- 172.21.15.86:0/2815610394 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0638163320 con 0x7f06381f9850 2024-04-20T21:30:10.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.990+0000 7f0634c00640 1 -- 172.21.15.86:0/2815610394 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062001c400 con 0x7f06381f9850 2024-04-20T21:30:10.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.990+0000 7f063da00640 1 -- 172.21.15.86:0/2815610394 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06381638e0 con 0x7f06381f9850 2024-04-20T21:30:10.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.992+0000 7f063da00640 1 -- 172.21.15.86:0/2815610394 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f063804fcf0 con 0x7f06381f9850 2024-04-20T21:30:10.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.992+0000 7f0634c00640 1 -- 172.21.15.86:0/2815610394 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f062001c5a0 con 0x7f06381f9850 2024-04-20T21:30:10.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.993+0000 7f0634c00640 1 --2- 172.21.15.86:0/2815610394 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f061406c8f0 0x7f061406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:10.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.993+0000 7f0634c00640 1 -- 172.21.15.86:0/2815610394 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7f062008c180 con 0x7f06381f9850 2024-04-20T21:30:10.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.994+0000 7f0637e00640 1 --2- 172.21.15.86:0/2815610394 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f061406c8f0 0x7f061406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:10.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.994+0000 7f0637e00640 1 --2- 172.21.15.86:0/2815610394 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f061406c8f0 0x7f061406edb0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f0628007fd0 tx=0x7f0628009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:10.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:10.998+0000 7f0634c00640 1 -- 172.21.15.86:0/2815610394 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0620018030 con 0x7f06381f9850 2024-04-20T21:30:11.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.186+0000 7f063da00640 1 -- 172.21.15.86:0/2815610394 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0638071eb0 con 0x7f06381f9850 2024-04-20T21:30:11.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.189+0000 7f0634c00640 1 -- 172.21.15.86:0/2815610394 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) ==== 74+0+16627 (secure 0 0 0) 0x7f062005a5a0 con 0x7f06381f9850 2024-04-20T21:30:11.188 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:11.188 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":112,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:29:45.322441+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:29:41.532824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-20T21:29:42.749687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:30:11.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.191+0000 7f063da00640 1 -- 172.21.15.86:0/2815610394 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f061406c8f0 msgr2=0x7f061406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:11.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.191+0000 7f063da00640 1 --2- 172.21.15.86:0/2815610394 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f061406c8f0 0x7f061406edb0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f0628007fd0 tx=0x7f0628009040 comp rx=0 tx=0).stop 2024-04-20T21:30:11.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.191+0000 7f063da00640 1 -- 172.21.15.86:0/2815610394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06381f9850 msgr2=0x7f06381fb950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:11.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.191+0000 7f063da00640 1 --2- 172.21.15.86:0/2815610394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06381f9850 0x7f06381fb950 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f06200108f0 tx=0x7f0620010cf0 comp rx=0 tx=0).stop 2024-04-20T21:30:11.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.192+0000 7f063da00640 1 -- 172.21.15.86:0/2815610394 shutdown_connections 2024-04-20T21:30:11.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.192+0000 7f063da00640 1 --2- 172.21.15.86:0/2815610394 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f061406c8f0 0x7f061406edb0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:11.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.192+0000 7f063da00640 1 --2- 172.21.15.86:0/2815610394 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f06381f9cd0 0x7f06381fbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:11.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.192+0000 7f063da00640 1 --2- 172.21.15.86:0/2815610394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06381f9850 0x7f06381fb950 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:11.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.192+0000 7f063da00640 1 --2- 172.21.15.86:0/2815610394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f061c005020 0x7f063815d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:11.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.192+0000 7f063da00640 1 -- 172.21.15.86:0/2815610394 >> 172.21.15.86:0/2815610394 conn(0x7f06380d0170 msgr2=0x7f063815f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:11.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.192+0000 7f063da00640 1 -- 172.21.15.86:0/2815610394 shutdown_connections 2024-04-20T21:30:11.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.192+0000 7f063da00640 1 -- 172.21.15.86:0/2815610394 wait complete. 2024-04-20T21:30:11.210 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:29:41.532824+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:29:42.749687+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:30:11.211 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:30:11.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.413+0000 7fcb92000640 1 Processor -- start 2024-04-20T21:30:11.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.413+0000 7fcb92000640 1 -- start start 2024-04-20T21:30:11.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.415+0000 7fcb92000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fcb8c0af980 0x7fcb8c147a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:11.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.415+0000 7fcb92000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcb8c0b3200 0x7fcb8c0b3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:11.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.415+0000 7fcb92000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb8c06e840 con 0x7fcb8c143f10 2024-04-20T21:30:11.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.415+0000 7fcb92000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fcb8c0b3ba0 con 0x7fcb8c0b3200 2024-04-20T21:30:11.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.415+0000 7fcb92000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fcb8c06f450 con 0x7fcb8c0af980 2024-04-20T21:30:11.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.415+0000 7fcb83e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcb8c0b3200 0x7fcb8c0b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:11.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.415+0000 7fcb83e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcb8c0b3200 0x7fcb8c0b3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60036/0 (socket says 172.21.15.86:60036) 2024-04-20T21:30:11.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.415+0000 7fcb83e00640 1 -- 172.21.15.86:0/30958412 learned_addr learned my addr 172.21.15.86:0/30958412 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:11.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.415+0000 7fcb91600640 1 --1- 172.21.15.86:0/30958412 >> v1:172.21.15.86:6789/0 conn(0x7fcb8c143f10 0x7fcb8c144380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51312/0 (socket says 172.21.15.86:51312) 2024-04-20T21:30:11.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.416+0000 7fcb90c00640 1 --2- 172.21.15.86:0/30958412 >> v2:172.21.15.144:3300/0 conn(0x7fcb8c0af980 0x7fcb8c147a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:11.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.416+0000 7fcb83e00640 1 -- 172.21.15.86:0/30958412 >> v2:172.21.15.144:3300/0 conn(0x7fcb8c0af980 msgr2=0x7fcb8c147a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:11.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.416+0000 7fcb83e00640 1 --2- 172.21.15.86:0/30958412 >> v2:172.21.15.144:3300/0 conn(0x7fcb8c0af980 0x7fcb8c147a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:11.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.416+0000 7fcb83e00640 1 -- 172.21.15.86:0/30958412 >> v1:172.21.15.86:6789/0 conn(0x7fcb8c143f10 legacy=0x7fcb8c144380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:11.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.416+0000 7fcb83e00640 1 -- 172.21.15.86:0/30958412 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb8c06eb20 con 0x7fcb8c0b3200 2024-04-20T21:30:11.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.417+0000 7fcb83e00640 1 --2- 172.21.15.86:0/30958412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb8c0b3200 0x7fcb8c0b3660 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c06d3d0 tx=0x7fcb7c02b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=250a500efbf37aa1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:11.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.417+0000 7fcb8be00640 1 -- 172.21.15.86:0/30958412 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c034070 con 0x7fcb8c0b3200 2024-04-20T21:30:11.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.417+0000 7fcb8be00640 1 -- 172.21.15.86:0/30958412 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb7c030870 con 0x7fcb8c0b3200 2024-04-20T21:30:11.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.417+0000 7fcb8be00640 1 -- 172.21.15.86:0/30958412 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c030c40 con 0x7fcb8c0b3200 2024-04-20T21:30:11.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.418+0000 7fcb92000640 1 -- 172.21.15.86:0/30958412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb8c0b3200 msgr2=0x7fcb8c0b3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:11.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.418+0000 7fcb92000640 1 --2- 172.21.15.86:0/30958412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb8c0b3200 0x7fcb8c0b3660 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c06d3d0 tx=0x7fcb7c02b9b0 comp rx=0 tx=0).stop 2024-04-20T21:30:11.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.418+0000 7fcb92000640 1 -- 172.21.15.86:0/30958412 shutdown_connections 2024-04-20T21:30:11.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.418+0000 7fcb92000640 1 --2- 172.21.15.86:0/30958412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb8c0b3200 0x7fcb8c0b3660 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:11.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.418+0000 7fcb92000640 1 --2- 172.21.15.86:0/30958412 >> v2:172.21.15.144:3300/0 conn(0x7fcb8c0af980 0x7fcb8c147a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:11.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.418+0000 7fcb92000640 1 -- 172.21.15.86:0/30958412 >> 172.21.15.86:0/30958412 conn(0x7fcb8c0b1f00 msgr2=0x7fcb8c0b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:11.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.418+0000 7fcb92000640 1 -- 172.21.15.86:0/30958412 shutdown_connections 2024-04-20T21:30:11.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.418+0000 7fcb92000640 1 -- 172.21.15.86:0/30958412 wait complete. 2024-04-20T21:30:11.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.420+0000 7fcb92000640 1 Processor -- start 2024-04-20T21:30:11.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.420+0000 7fcb92000640 1 -- start start 2024-04-20T21:30:11.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.421+0000 7fcb92000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb8c0af980 0x7fcb8c14c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:11.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.421+0000 7fcb92000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb8c0b3200 0x7fcb8c1606d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:11.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.421+0000 7fcb92000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb8c143f10 0x7fcb8c155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:11.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.421+0000 7fcb92000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8c06e3d0 con 0x7fcb8c0b3200 2024-04-20T21:30:11.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.421+0000 7fcb92000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8c06f0e0 con 0x7fcb8c0af980 2024-04-20T21:30:11.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.421+0000 7fcb92000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fcb8c06e550 con 0x7fcb8c143f10 2024-04-20T21:30:11.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.421+0000 7fcb83e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb8c0b3200 0x7fcb8c1606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:11.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.421+0000 7fcb83e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb8c0b3200 0x7fcb8c1606d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60038/0 (socket says 172.21.15.86:60038) 2024-04-20T21:30:11.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.421+0000 7fcb83e00640 1 -- 172.21.15.86:0/109601979 learned_addr learned my addr 172.21.15.86:0/109601979 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:11.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.421+0000 7fcb90c00640 1 --2- 172.21.15.86:0/109601979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb8c0af980 0x7fcb8c14c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:11.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.421+0000 7fcb83e00640 1 -- 172.21.15.86:0/109601979 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb8c143f10 msgr2=0x7fcb8c155820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:11.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.421+0000 7fcb83e00640 1 --2- 172.21.15.86:0/109601979 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb8c143f10 0x7fcb8c155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:11.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.421+0000 7fcb83e00640 1 -- 172.21.15.86:0/109601979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb8c0af980 msgr2=0x7fcb8c14c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:11.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.421+0000 7fcb83e00640 1 --2- 172.21.15.86:0/109601979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb8c0af980 0x7fcb8c14c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:11.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.421+0000 7fcb83e00640 1 -- 172.21.15.86:0/109601979 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb8c160c10 con 0x7fcb8c0b3200 2024-04-20T21:30:11.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.421+0000 7fcb90c00640 1 --2- 172.21.15.86:0/109601979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb8c0af980 0x7fcb8c14c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:30:11.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.422+0000 7fcb83e00640 1 --2- 172.21.15.86:0/109601979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb8c0b3200 0x7fcb8c1606d0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c02b9f0 tx=0x7fcb7c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:11.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.422+0000 7fcb89600640 1 -- 172.21.15.86:0/109601979 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c048070 con 0x7fcb8c0b3200 2024-04-20T21:30:11.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.422+0000 7fcb89600640 1 -- 172.21.15.86:0/109601979 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb7c030670 con 0x7fcb8c0b3200 2024-04-20T21:30:11.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.422+0000 7fcb89600640 1 -- 172.21.15.86:0/109601979 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c034040 con 0x7fcb8c0b3200 2024-04-20T21:30:11.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.422+0000 7fcb92000640 1 -- 172.21.15.86:0/109601979 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb8c160eb0 con 0x7fcb8c0b3200 2024-04-20T21:30:11.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.422+0000 7fcb92000640 1 -- 172.21.15.86:0/109601979 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb8c161470 con 0x7fcb8c0b3200 2024-04-20T21:30:11.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.423+0000 7fcb92000640 1 -- 172.21.15.86:0/109601979 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb8c04fd90 con 0x7fcb8c0b3200 2024-04-20T21:30:11.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.424+0000 7fcb89600640 1 -- 172.21.15.86:0/109601979 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcb7c04b070 con 0x7fcb8c0b3200 2024-04-20T21:30:11.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.425+0000 7fcb89600640 1 --2- 172.21.15.86:0/109601979 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcb6806c8f0 0x7fcb6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:11.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.425+0000 7fcb89600640 1 -- 172.21.15.86:0/109601979 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7fcb7c0add90 con 0x7fcb8c0b3200 2024-04-20T21:30:11.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.426+0000 7fcb90c00640 1 --2- 172.21.15.86:0/109601979 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcb6806c8f0 0x7fcb6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:11.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.426+0000 7fcb90c00640 1 --2- 172.21.15.86:0/109601979 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcb6806c8f0 0x7fcb6806edb0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fcb74006fd0 tx=0x7fcb74008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:11.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.430+0000 7fcb89600640 1 -- 172.21.15.86:0/109601979 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fcb7c07c130 con 0x7fcb8c0b3200 2024-04-20T21:30:11.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.629+0000 7fcb92000640 1 -- 172.21.15.86:0/109601979 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcb8c142bc0 con 0x7fcb8c0b3200 2024-04-20T21:30:11.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.630+0000 7fcb89600640 1 -- 172.21.15.86:0/109601979 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) ==== 93+0+10 (secure 0 0 0) 0x7fcb7c07ba30 con 0x7fcb8c0b3200 2024-04-20T21:30:11.629 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:30:11.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.632+0000 7fcb92000640 1 -- 172.21.15.86:0/109601979 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcb6806c8f0 msgr2=0x7fcb6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:11.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.632+0000 7fcb92000640 1 --2- 172.21.15.86:0/109601979 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcb6806c8f0 0x7fcb6806edb0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fcb74006fd0 tx=0x7fcb74008040 comp rx=0 tx=0).stop 2024-04-20T21:30:11.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.632+0000 7fcb92000640 1 -- 172.21.15.86:0/109601979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb8c0b3200 msgr2=0x7fcb8c1606d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:11.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.632+0000 7fcb92000640 1 --2- 172.21.15.86:0/109601979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb8c0b3200 0x7fcb8c1606d0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c02b9f0 tx=0x7fcb7c004730 comp rx=0 tx=0).stop 2024-04-20T21:30:11.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.633+0000 7fcb92000640 1 -- 172.21.15.86:0/109601979 shutdown_connections 2024-04-20T21:30:11.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.633+0000 7fcb92000640 1 --2- 172.21.15.86:0/109601979 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fcb6806c8f0 0x7fcb6806edb0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:11.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.633+0000 7fcb92000640 1 --2- 172.21.15.86:0/109601979 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fcb8c143f10 0x7fcb8c155820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:11.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.633+0000 7fcb92000640 1 --2- 172.21.15.86:0/109601979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb8c0b3200 0x7fcb8c1606d0 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:11.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.633+0000 7fcb92000640 1 --2- 172.21.15.86:0/109601979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb8c0af980 0x7fcb8c14c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:11.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.633+0000 7fcb92000640 1 -- 172.21.15.86:0/109601979 >> 172.21.15.86:0/109601979 conn(0x7fcb8c0b1f00 msgr2=0x7fcb8c142240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:11.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.633+0000 7fcb92000640 1 -- 172.21.15.86:0/109601979 shutdown_connections 2024-04-20T21:30:11.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.633+0000 7fcb92000640 1 -- 172.21.15.86:0/109601979 wait complete. 2024-04-20T21:30:11.650 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:30:11.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.845+0000 7f7195600640 1 Processor -- start 2024-04-20T21:30:11.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.845+0000 7f7195600640 1 -- start start 2024-04-20T21:30:11.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.847+0000 7f7195600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7190072450 0x7f7190167670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:11.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.847+0000 7f7195600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7190160410 con 0x7f7190145f00 2024-04-20T21:30:11.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.847+0000 7f7195600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71901606f0 con 0x7f7190145250 2024-04-20T21:30:11.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.847+0000 7f7195600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f71901609d0 con 0x7f7190072450 2024-04-20T21:30:11.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.847+0000 7f7194c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7190072450 0x7f7190167670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:11.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.847+0000 7f718f400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f7190145f00 0x7f7190071d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51328/0 (socket says 172.21.15.86:51328) 2024-04-20T21:30:11.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.847+0000 7f7194c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7190072450 0x7f7190167670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60048/0 (socket says 172.21.15.86:60048) 2024-04-20T21:30:11.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.847+0000 7f718f400640 1 -- 172.21.15.86:0/3465898582 learned_addr learned my addr 172.21.15.86:0/3465898582 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:30:11.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.848+0000 7f7194c00640 1 -- 172.21.15.86:0/3465898582 >> v1:172.21.15.144:6789/0 conn(0x7f7190145250 legacy=0x7f719006efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:11.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.848+0000 7f7194c00640 1 -- 172.21.15.86:0/3465898582 >> v1:172.21.15.86:6789/0 conn(0x7f7190145f00 legacy=0x7f7190071d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:11.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.848+0000 7f7194c00640 1 -- 172.21.15.86:0/3465898582 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7190160cb0 con 0x7f7190072450 2024-04-20T21:30:11.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.848+0000 7f718ea00640 1 -- 172.21.15.86:0/3465898582 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 141336710 0 0) 0x7f7190160410 con 0x7f7190145f00 2024-04-20T21:30:11.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.849+0000 7f7194c00640 1 --2- 172.21.15.86:0/3465898582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7190072450 0x7f7190167670 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f7188009d90 tx=0x7f718802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab8fb969236f1085 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:11.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.849+0000 7f718ea00640 1 -- 172.21.15.86:0/3465898582 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7188039070 con 0x7f7190072450 2024-04-20T21:30:11.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.849+0000 7f718ea00640 1 -- 172.21.15.86:0/3465898582 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f718802cbd0 con 0x7f7190072450 2024-04-20T21:30:11.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.849+0000 7f718ea00640 1 -- 172.21.15.86:0/3465898582 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7188034440 con 0x7f7190072450 2024-04-20T21:30:11.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.850+0000 7f7195600640 1 -- 172.21.15.86:0/3465898582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7190072450 msgr2=0x7f7190167670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:11.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.850+0000 7f7195600640 1 --2- 172.21.15.86:0/3465898582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7190072450 0x7f7190167670 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f7188009d90 tx=0x7f718802be20 comp rx=0 tx=0).stop 2024-04-20T21:30:11.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.850+0000 7f7195600640 1 -- 172.21.15.86:0/3465898582 shutdown_connections 2024-04-20T21:30:11.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.850+0000 7f7195600640 1 --2- 172.21.15.86:0/3465898582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7190072450 0x7f7190167670 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:11.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.850+0000 7f7195600640 1 -- 172.21.15.86:0/3465898582 >> 172.21.15.86:0/3465898582 conn(0x7f7190067cf0 msgr2=0x7f719006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:11.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.850+0000 7f7195600640 1 -- 172.21.15.86:0/3465898582 shutdown_connections 2024-04-20T21:30:11.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.850+0000 7f7195600640 1 -- 172.21.15.86:0/3465898582 wait complete. 2024-04-20T21:30:11.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.852+0000 7f7195600640 1 Processor -- start 2024-04-20T21:30:11.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.852+0000 7f7195600640 1 -- start start 2024-04-20T21:30:11.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.853+0000 7f7195600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7190072450 0x7f719013af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:11.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.853+0000 7f7195600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7190145250 0x7f71900c5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:11.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.853+0000 7f7195600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7190145f00 0x7f71900c6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:11.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.853+0000 7f7195600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f719015df80 con 0x7f7190072450 2024-04-20T21:30:11.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.853+0000 7f7195600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f719015de00 con 0x7f7190145f00 2024-04-20T21:30:11.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.853+0000 7f7195600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f719015e100 con 0x7f7190145250 2024-04-20T21:30:11.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.853+0000 7f718fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7190072450 0x7f719013af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:11.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.853+0000 7f718fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7190072450 0x7f719013af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60054/0 (socket says 172.21.15.86:60054) 2024-04-20T21:30:11.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.853+0000 7f718fe00640 1 -- 172.21.15.86:0/2510288729 learned_addr learned my addr 172.21.15.86:0/2510288729 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:11.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.854+0000 7f7194c00640 1 --2- 172.21.15.86:0/2510288729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7190145f00 0x7f71900c6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:11.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.854+0000 7f718f400640 1 --2- 172.21.15.86:0/2510288729 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7190145250 0x7f71900c5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:11.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.854+0000 7f718fe00640 1 -- 172.21.15.86:0/2510288729 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7190145250 msgr2=0x7f71900c5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:11.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.854+0000 7f718fe00640 1 --2- 172.21.15.86:0/2510288729 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7190145250 0x7f71900c5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:11.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.854+0000 7f718fe00640 1 -- 172.21.15.86:0/2510288729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7190145f00 msgr2=0x7f71900c6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:11.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.854+0000 7f718fe00640 1 --2- 172.21.15.86:0/2510288729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7190145f00 0x7f71900c6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:11.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.854+0000 7f718fe00640 1 -- 172.21.15.86:0/2510288729 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f719013b8f0 con 0x7f7190072450 2024-04-20T21:30:11.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.854+0000 7f7194c00640 1 --2- 172.21.15.86:0/2510288729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7190145f00 0x7f71900c6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:30:11.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.854+0000 7f718fe00640 1 --2- 172.21.15.86:0/2510288729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7190072450 0x7f719013af10 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f718000c430 tx=0x7f718000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:11.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.854+0000 7f7187e00640 1 -- 172.21.15.86:0/2510288729 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7180011790 con 0x7f7190072450 2024-04-20T21:30:11.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.854+0000 7f7187e00640 1 -- 172.21.15.86:0/2510288729 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7180017070 con 0x7f7190072450 2024-04-20T21:30:11.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.854+0000 7f7187e00640 1 -- 172.21.15.86:0/2510288729 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7180011bb0 con 0x7f7190072450 2024-04-20T21:30:11.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.854+0000 7f7195600640 1 -- 172.21.15.86:0/2510288729 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f719013bbd0 con 0x7f7190072450 2024-04-20T21:30:11.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.855+0000 7f7195600640 1 -- 172.21.15.86:0/2510288729 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f719013c190 con 0x7f7190072450 2024-04-20T21:30:11.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.856+0000 7f7195600640 1 -- 172.21.15.86:0/2510288729 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f719004fd90 con 0x7f7190072450 2024-04-20T21:30:11.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.857+0000 7f7187e00640 1 -- 172.21.15.86:0/2510288729 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f718001c460 con 0x7f7190072450 2024-04-20T21:30:11.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.857+0000 7f7187e00640 1 --2- 172.21.15.86:0/2510288729 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f717006c920 0x7f717006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:11.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.858+0000 7f7187e00640 1 -- 172.21.15.86:0/2510288729 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7f718008b9d0 con 0x7f7190072450 2024-04-20T21:30:11.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.858+0000 7f718f400640 1 --2- 172.21.15.86:0/2510288729 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f717006c920 0x7f717006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:11.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.858+0000 7f718f400640 1 --2- 172.21.15.86:0/2510288729 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f717006c920 0x7f717006ede0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f71900b2d90 tx=0x7f717c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:11.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:11.862+0000 7f7187e00640 1 -- 172.21.15.86:0/2510288729 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7180059e00 con 0x7f7190072450 2024-04-20T21:30:12.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.060+0000 7f7195600640 1 -- 172.21.15.86:0/2510288729 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7190072ea0 con 0x7f7190072450 2024-04-20T21:30:12.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.061+0000 7f7187e00640 1 -- 172.21.15.86:0/2510288729 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v112) ==== 104+0+11 (secure 0 0 0) 0x7f7180059700 con 0x7f7190072450 2024-04-20T21:30:12.060 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-20T21:30:12.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.063+0000 7f7195600640 1 -- 172.21.15.86:0/2510288729 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f717006c920 msgr2=0x7f717006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:12.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.063+0000 7f7195600640 1 --2- 172.21.15.86:0/2510288729 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f717006c920 0x7f717006ede0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f71900b2d90 tx=0x7f717c009040 comp rx=0 tx=0).stop 2024-04-20T21:30:12.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.063+0000 7f7195600640 1 -- 172.21.15.86:0/2510288729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7190072450 msgr2=0x7f719013af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:12.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.063+0000 7f7195600640 1 --2- 172.21.15.86:0/2510288729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7190072450 0x7f719013af10 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f718000c430 tx=0x7f718000c8f0 comp rx=0 tx=0).stop 2024-04-20T21:30:12.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.063+0000 7f7195600640 1 -- 172.21.15.86:0/2510288729 shutdown_connections 2024-04-20T21:30:12.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.063+0000 7f7195600640 1 --2- 172.21.15.86:0/2510288729 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f717006c920 0x7f717006ede0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:12.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.063+0000 7f7195600640 1 --2- 172.21.15.86:0/2510288729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7190145f00 0x7f71900c6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:12.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.063+0000 7f7195600640 1 --2- 172.21.15.86:0/2510288729 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7190145250 0x7f71900c5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:12.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.063+0000 7f7195600640 1 --2- 172.21.15.86:0/2510288729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7190072450 0x7f719013af10 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:12.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.063+0000 7f7195600640 1 -- 172.21.15.86:0/2510288729 >> 172.21.15.86:0/2510288729 conn(0x7f7190067cf0 msgr2=0x7f719015e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:12.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.063+0000 7f7195600640 1 -- 172.21.15.86:0/2510288729 shutdown_connections 2024-04-20T21:30:12.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.063+0000 7f7195600640 1 -- 172.21.15.86:0/2510288729 wait complete. 2024-04-20T21:30:12.081 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:30:12.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.286+0000 7f87b1800640 1 Processor -- start 2024-04-20T21:30:12.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.286+0000 7f87b1800640 1 -- start start 2024-04-20T21:30:12.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.288+0000 7f87b1800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f87ac06bbf0 0x7f87ac06e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:12.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.288+0000 7f87b1800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f87ac06bfd0 0x7f87ac145b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:12.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.288+0000 7f87b1800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87ac15cd40 con 0x7f87ac146040 2024-04-20T21:30:12.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.288+0000 7f87b1800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f87ac1464b0 con 0x7f87ac06bbf0 2024-04-20T21:30:12.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.288+0000 7f87b1800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f87ac15d000 con 0x7f87ac06bfd0 2024-04-20T21:30:12.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.288+0000 7f87aaa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f87ac06bbf0 0x7f87ac06e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:12.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.288+0000 7f87aaa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f87ac06bbf0 0x7f87ac06e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60068/0 (socket says 172.21.15.86:60068) 2024-04-20T21:30:12.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.288+0000 7f87aaa00640 1 -- 172.21.15.86:0/3111760842 learned_addr learned my addr 172.21.15.86:0/3111760842 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:12.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.289+0000 7f87a3e00640 1 --2- 172.21.15.86:0/3111760842 >> v2:172.21.15.144:3300/0 conn(0x7f87ac06bfd0 0x7f87ac145b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:12.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.289+0000 7f87aaa00640 1 -- 172.21.15.86:0/3111760842 >> v2:172.21.15.144:3300/0 conn(0x7f87ac06bfd0 msgr2=0x7f87ac145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:12.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.289+0000 7f87aaa00640 1 --2- 172.21.15.86:0/3111760842 >> v2:172.21.15.144:3300/0 conn(0x7f87ac06bfd0 0x7f87ac145b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:12.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.289+0000 7f87aaa00640 1 -- 172.21.15.86:0/3111760842 >> v1:172.21.15.144:6789/0 conn(0x7f87ac146040 legacy=0x7f87ac0cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:12.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.289+0000 7f87aaa00640 1 -- 172.21.15.86:0/3111760842 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87ac15d2e0 con 0x7f87ac06bbf0 2024-04-20T21:30:12.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.291+0000 7f87aaa00640 1 --2- 172.21.15.86:0/3111760842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87ac06bbf0 0x7f87ac06e000 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f87ac06dcc0 tx=0x7f879c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33dc01a3926fdc41 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:12.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.291+0000 7f87aa000640 1 -- 172.21.15.86:0/3111760842 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879c035020 con 0x7f87ac06bbf0 2024-04-20T21:30:12.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.291+0000 7f87aa000640 1 -- 172.21.15.86:0/3111760842 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f879c031870 con 0x7f87ac06bbf0 2024-04-20T21:30:12.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.291+0000 7f87aa000640 1 -- 172.21.15.86:0/3111760842 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879c031c40 con 0x7f87ac06bbf0 2024-04-20T21:30:12.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.292+0000 7f87b1800640 1 -- 172.21.15.86:0/3111760842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87ac06bbf0 msgr2=0x7f87ac06e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:12.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.292+0000 7f87b1800640 1 --2- 172.21.15.86:0/3111760842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87ac06bbf0 0x7f87ac06e000 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f87ac06dcc0 tx=0x7f879c02b910 comp rx=0 tx=0).stop 2024-04-20T21:30:12.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.292+0000 7f87b1800640 1 -- 172.21.15.86:0/3111760842 shutdown_connections 2024-04-20T21:30:12.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.292+0000 7f87b1800640 1 --2- 172.21.15.86:0/3111760842 >> v2:172.21.15.144:3300/0 conn(0x7f87ac06bfd0 0x7f87ac145b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:12.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.292+0000 7f87b1800640 1 --2- 172.21.15.86:0/3111760842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87ac06bbf0 0x7f87ac06e000 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:12.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.292+0000 7f87b1800640 1 -- 172.21.15.86:0/3111760842 >> 172.21.15.86:0/3111760842 conn(0x7f87ac0b2110 msgr2=0x7f87ac0b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:12.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.292+0000 7f87b1800640 1 -- 172.21.15.86:0/3111760842 shutdown_connections 2024-04-20T21:30:12.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.293+0000 7f87b1800640 1 -- 172.21.15.86:0/3111760842 wait complete. 2024-04-20T21:30:12.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.294+0000 7f87b1800640 1 Processor -- start 2024-04-20T21:30:12.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.294+0000 7f87b1800640 1 -- start start 2024-04-20T21:30:12.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.295+0000 7f87b1800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87ac06bbf0 0x7f87ac142170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:12.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.295+0000 7f87b1800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f87ac06bfd0 0x7f87ac0ca580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:12.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.295+0000 7f87b1800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87ac146040 0x7f87ac1e7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:12.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.295+0000 7f87b1800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f87ac0cdc50 con 0x7f87ac146040 2024-04-20T21:30:12.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.295+0000 7f87b1800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f87ac15d8a0 con 0x7f87ac06bbf0 2024-04-20T21:30:12.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.295+0000 7f87b1800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f87ac0cddd0 con 0x7f87ac06bfd0 2024-04-20T21:30:12.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.296+0000 7f87ab400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87ac146040 0x7f87ac1e7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:12.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.296+0000 7f87ab400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87ac146040 0x7f87ac1e7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60078/0 (socket says 172.21.15.86:60078) 2024-04-20T21:30:12.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.296+0000 7f87ab400640 1 -- 172.21.15.86:0/219230682 learned_addr learned my addr 172.21.15.86:0/219230682 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:12.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.296+0000 7f87aaa00640 1 --2- 172.21.15.86:0/219230682 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87ac06bbf0 0x7f87ac142170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:12.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.296+0000 7f87a3e00640 1 --2- 172.21.15.86:0/219230682 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f87ac06bfd0 0x7f87ac0ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:12.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.296+0000 7f87ab400640 1 -- 172.21.15.86:0/219230682 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f87ac06bfd0 msgr2=0x7f87ac0ca580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:12.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.296+0000 7f87ab400640 1 --2- 172.21.15.86:0/219230682 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f87ac06bfd0 0x7f87ac0ca580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:12.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.296+0000 7f87ab400640 1 -- 172.21.15.86:0/219230682 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87ac06bbf0 msgr2=0x7f87ac142170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:12.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.296+0000 7f87ab400640 1 --2- 172.21.15.86:0/219230682 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87ac06bbf0 0x7f87ac142170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:12.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.296+0000 7f87ab400640 1 -- 172.21.15.86:0/219230682 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87ac0cafd0 con 0x7f87ac146040 2024-04-20T21:30:12.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.296+0000 7f87aaa00640 1 --2- 172.21.15.86:0/219230682 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87ac06bbf0 0x7f87ac142170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:30:12.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.297+0000 7f87ab400640 1 --2- 172.21.15.86:0/219230682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87ac146040 0x7f87ac1e7f10 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f87a400c7b0 tx=0x7f87a400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:12.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.297+0000 7f87a2a00640 1 -- 172.21.15.86:0/219230682 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a4004b60 con 0x7f87ac146040 2024-04-20T21:30:12.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.297+0000 7f87a2a00640 1 -- 172.21.15.86:0/219230682 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87a4004d00 con 0x7f87ac146040 2024-04-20T21:30:12.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.297+0000 7f87b1800640 1 -- 172.21.15.86:0/219230682 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87ac0cb2b0 con 0x7f87ac146040 2024-04-20T21:30:12.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.297+0000 7f87a2a00640 1 -- 172.21.15.86:0/219230682 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a400aa90 con 0x7f87ac146040 2024-04-20T21:30:12.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.297+0000 7f87b1800640 1 -- 172.21.15.86:0/219230682 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87ac0cb870 con 0x7f87ac146040 2024-04-20T21:30:12.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.298+0000 7f87b1800640 1 -- 172.21.15.86:0/219230682 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87ac0689e0 con 0x7f87ac146040 2024-04-20T21:30:12.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.299+0000 7f87a2a00640 1 -- 172.21.15.86:0/219230682 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f87a400ac30 con 0x7f87ac146040 2024-04-20T21:30:12.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.300+0000 7f87a2a00640 1 --2- 172.21.15.86:0/219230682 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f878806c920 0x7f878806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:12.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.300+0000 7f87a2a00640 1 -- 172.21.15.86:0/219230682 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7f87a408bec0 con 0x7f87ac146040 2024-04-20T21:30:12.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.300+0000 7f87aaa00640 1 --2- 172.21.15.86:0/219230682 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f878806c920 0x7f878806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:12.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.301+0000 7f87aaa00640 1 --2- 172.21.15.86:0/219230682 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f878806c920 0x7f878806ede0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f879c036000 tx=0x7f879c03c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:12.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.305+0000 7f87a2a00640 1 -- 172.21.15.86:0/219230682 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f87a4020070 con 0x7f87ac146040 2024-04-20T21:30:12.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.503+0000 7f87b1800640 1 -- 172.21.15.86:0/219230682 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f87ac06c780 con 0x7f87ac146040 2024-04-20T21:30:12.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.504+0000 7f87a2a00640 1 -- 172.21.15.86:0/219230682 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v112) ==== 100+0+11 (secure 0 0 0) 0x7f87a405a260 con 0x7f87ac146040 2024-04-20T21:30:12.503 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-20T21:30:12.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.507+0000 7f87b1800640 1 -- 172.21.15.86:0/219230682 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f878806c920 msgr2=0x7f878806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:12.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.507+0000 7f87b1800640 1 --2- 172.21.15.86:0/219230682 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f878806c920 0x7f878806ede0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f879c036000 tx=0x7f879c03c040 comp rx=0 tx=0).stop 2024-04-20T21:30:12.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.507+0000 7f87b1800640 1 -- 172.21.15.86:0/219230682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87ac146040 msgr2=0x7f87ac1e7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:12.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.507+0000 7f87b1800640 1 --2- 172.21.15.86:0/219230682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87ac146040 0x7f87ac1e7f10 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f87a400c7b0 tx=0x7f87a400cc70 comp rx=0 tx=0).stop 2024-04-20T21:30:12.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.507+0000 7f87b1800640 1 -- 172.21.15.86:0/219230682 shutdown_connections 2024-04-20T21:30:12.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.507+0000 7f87b1800640 1 --2- 172.21.15.86:0/219230682 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f878806c920 0x7f878806ede0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:12.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.507+0000 7f87b1800640 1 --2- 172.21.15.86:0/219230682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87ac146040 0x7f87ac1e7f10 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:12.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.507+0000 7f87b1800640 1 --2- 172.21.15.86:0/219230682 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f87ac06bfd0 0x7f87ac0ca580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:12.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.507+0000 7f87b1800640 1 --2- 172.21.15.86:0/219230682 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87ac06bbf0 0x7f87ac142170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:12.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.507+0000 7f87b1800640 1 -- 172.21.15.86:0/219230682 >> 172.21.15.86:0/219230682 conn(0x7f87ac0b2110 msgr2=0x7f87ac15dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:12.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.507+0000 7f87b1800640 1 -- 172.21.15.86:0/219230682 shutdown_connections 2024-04-20T21:30:12.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.507+0000 7f87b1800640 1 -- 172.21.15.86:0/219230682 wait complete. 2024-04-20T21:30:12.525 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T21:30:12.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.716+0000 7f44edc00640 1 Processor -- start 2024-04-20T21:30:12.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.716+0000 7f44edc00640 1 -- start start 2024-04-20T21:30:12.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.717+0000 7f44edc00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f44e80af7e0 0x7f44e80b4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:12.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.717+0000 7f44edc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44e80cd220 0x7f44e80b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:12.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.718+0000 7f44edc00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44e8160160 con 0x7f44e815c590 2024-04-20T21:30:12.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.718+0000 7f44edc00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f44e8160440 con 0x7f44e80af7e0 2024-04-20T21:30:12.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.718+0000 7f44edc00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f44e8160720 con 0x7f44e80cd220 2024-04-20T21:30:12.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.718+0000 7f44e7e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f44e815c590 0x7f44e80ccb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:51334/0 (socket says 172.21.15.86:51334) 2024-04-20T21:30:12.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.718+0000 7f44e7e00640 1 -- 172.21.15.86:0/3495010225 learned_addr learned my addr 172.21.15.86:0/3495010225 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:30:12.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.718+0000 7f44ed200640 1 --2- 172.21.15.86:0/3495010225 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44e80cd220 0x7f44e80b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:12.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.718+0000 7f44e7400640 1 -- 172.21.15.86:0/3495010225 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2277578272 0 0) 0x7f44e8160160 con 0x7f44e815c590 2024-04-20T21:30:12.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.718+0000 7f44df400640 1 --2- 172.21.15.86:0/3495010225 >> v2:172.21.15.144:3300/0 conn(0x7f44e80af7e0 0x7f44e80b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:12.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.718+0000 7f44e7400640 1 -- 172.21.15.86:0/3495010225 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f44cc003520 con 0x7f44e815c590 2024-04-20T21:30:12.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.719+0000 7f44e7400640 1 -- 172.21.15.86:0/3495010225 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2439751290 0 0) 0x7f44cc003520 con 0x7f44e815c590 2024-04-20T21:30:12.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.719+0000 7f44e7400640 1 -- 172.21.15.86:0/3495010225 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44e80cd220 msgr2=0x7f44e80b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:12.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.719+0000 7f44e7400640 1 --2- 172.21.15.86:0/3495010225 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44e80cd220 0x7f44e80b46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:12.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.719+0000 7f44e7400640 1 -- 172.21.15.86:0/3495010225 >> v2:172.21.15.144:3300/0 conn(0x7f44e80af7e0 msgr2=0x7f44e80b4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:12.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.719+0000 7f44e7400640 1 --2- 172.21.15.86:0/3495010225 >> v2:172.21.15.144:3300/0 conn(0x7f44e80af7e0 0x7f44e80b4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:12.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.719+0000 7f44e7400640 1 -- 172.21.15.86:0/3495010225 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44e8160a00 con 0x7f44e815c590 2024-04-20T21:30:12.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.719+0000 7f44e7400640 1 -- 172.21.15.86:0/3495010225 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f44d8001670 con 0x7f44e815c590 2024-04-20T21:30:12.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.719+0000 7f44df400640 1 --2- 172.21.15.86:0/3495010225 >> v2:172.21.15.144:3300/0 conn(0x7f44e80af7e0 0x7f44e80b4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:30:12.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.719+0000 7f44e7400640 1 -- 172.21.15.86:0/3495010225 >> v1:172.21.15.86:6789/0 conn(0x7f44e815c590 legacy=0x7f44e80ccb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:12.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.719+0000 7f44e7400640 1 --2- 172.21.15.86:0/3495010225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44cc0039b0 0x7f44cc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:12.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.719+0000 7f44e7400640 1 --2- 172.21.15.86:0/3495010225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44cc004800 0x7f44cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:12.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.719+0000 7f44e7400640 1 --2- 172.21.15.86:0/3495010225 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44e80cd220 0x7f44cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:12.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.719+0000 7f44e7400640 1 -- 172.21.15.86:0/3495010225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f44e8160440 con 0x7f44cc0039b0 2024-04-20T21:30:12.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.719+0000 7f44e7400640 1 -- 172.21.15.86:0/3495010225 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f44e8160720 con 0x7f44cc004800 2024-04-20T21:30:12.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.719+0000 7f44e7400640 1 -- 172.21.15.86:0/3495010225 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f44cc00b9f0 con 0x7f44e80cd220 2024-04-20T21:30:12.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.719+0000 7f44e7e00640 1 --2- 172.21.15.86:0/3495010225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44cc0039b0 0x7f44cc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:12.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.720+0000 7f44df400640 1 --2- 172.21.15.86:0/3495010225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44cc004800 0x7f44cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:12.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.720+0000 7f44e7e00640 1 -- 172.21.15.86:0/3495010225 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44e80cd220 msgr2=0x7f44cc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:12.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.720+0000 7f44e7e00640 1 --2- 172.21.15.86:0/3495010225 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44e80cd220 0x7f44cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:12.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.720+0000 7f44e7e00640 1 -- 172.21.15.86:0/3495010225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44cc004800 msgr2=0x7f44cc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:12.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.720+0000 7f44e7e00640 1 --2- 172.21.15.86:0/3495010225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44cc004800 0x7f44cc004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:12.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.720+0000 7f44e7e00640 1 -- 172.21.15.86:0/3495010225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44cc003520 con 0x7f44cc0039b0 2024-04-20T21:30:12.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.721+0000 7f44e7e00640 1 --2- 172.21.15.86:0/3495010225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44cc0039b0 0x7f44cc003dd0 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f44d800c290 tx=0x7f44cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c156f23b61198cdc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:12.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.721+0000 7f44e7400640 1 -- 172.21.15.86:0/3495010225 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d8007d50 con 0x7f44cc0039b0 2024-04-20T21:30:12.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.721+0000 7f44e7400640 1 -- 172.21.15.86:0/3495010225 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44d8030070 con 0x7f44cc0039b0 2024-04-20T21:30:12.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.721+0000 7f44e7400640 1 -- 172.21.15.86:0/3495010225 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d802cd80 con 0x7f44cc0039b0 2024-04-20T21:30:12.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.722+0000 7f44edc00640 1 -- 172.21.15.86:0/3495010225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44cc0039b0 msgr2=0x7f44cc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:12.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.722+0000 7f44edc00640 1 --2- 172.21.15.86:0/3495010225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44cc0039b0 0x7f44cc003dd0 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f44d800c290 tx=0x7f44cc005830 comp rx=0 tx=0).stop 2024-04-20T21:30:12.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.722+0000 7f44ed200640 1 -- 172.21.15.86:0/3495010225 reap_dead start 2024-04-20T21:30:12.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.722+0000 7f44edc00640 1 -- 172.21.15.86:0/3495010225 shutdown_connections 2024-04-20T21:30:12.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.722+0000 7f44edc00640 1 -- 172.21.15.86:0/3495010225 >> 172.21.15.86:0/3495010225 conn(0x7f44e806bb60 msgr2=0x7f44e806bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:12.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.722+0000 7f44edc00640 1 -- 172.21.15.86:0/3495010225 shutdown_connections 2024-04-20T21:30:12.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.722+0000 7f44edc00640 1 -- 172.21.15.86:0/3495010225 wait complete. 2024-04-20T21:30:12.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.724+0000 7f44edc00640 1 Processor -- start 2024-04-20T21:30:12.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.724+0000 7f44edc00640 1 -- start start 2024-04-20T21:30:12.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.725+0000 7f44edc00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44cc0039b0 0x7f44e80ae290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:12.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.725+0000 7f44edc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44e815ed50 0x7f44e815f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:12.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.725+0000 7f44edc00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44e81f60d0 0x7f44e81f6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:12.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.725+0000 7f44edc00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f44e815c970 con 0x7f44e815ed50 2024-04-20T21:30:12.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.725+0000 7f44edc00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f44e806f3d0 con 0x7f44e81f60d0 2024-04-20T21:30:12.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.725+0000 7f44edc00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f44e8160fc0 con 0x7f44cc0039b0 2024-04-20T21:30:12.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.726+0000 7f44df400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44e815ed50 0x7f44e815f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:12.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.726+0000 7f44df400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44e815ed50 0x7f44e815f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60098/0 (socket says 172.21.15.86:60098) 2024-04-20T21:30:12.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.726+0000 7f44df400640 1 -- 172.21.15.86:0/1722240540 learned_addr learned my addr 172.21.15.86:0/1722240540 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:12.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.726+0000 7f44e7e00640 1 --2- 172.21.15.86:0/1722240540 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44cc0039b0 0x7f44e80ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:12.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.726+0000 7f44ed200640 1 --2- 172.21.15.86:0/1722240540 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44e81f60d0 0x7f44e81f6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:12.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.726+0000 7f44df400640 1 -- 172.21.15.86:0/1722240540 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44cc0039b0 msgr2=0x7f44e80ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:12.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.726+0000 7f44df400640 1 --2- 172.21.15.86:0/1722240540 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44cc0039b0 0x7f44e80ae290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:12.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.726+0000 7f44df400640 1 -- 172.21.15.86:0/1722240540 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44e81f60d0 msgr2=0x7f44e81f6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:12.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.726+0000 7f44df400640 1 --2- 172.21.15.86:0/1722240540 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44e81f60d0 0x7f44e81f6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:12.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.726+0000 7f44df400640 1 -- 172.21.15.86:0/1722240540 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44e81f2ae0 con 0x7f44e815ed50 2024-04-20T21:30:12.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.726+0000 7f44df400640 1 --2- 172.21.15.86:0/1722240540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44e815ed50 0x7f44e815f1d0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f44d0009fb0 tx=0x7f44d000e5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:12.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.727+0000 7f44e4c00640 1 -- 172.21.15.86:0/1722240540 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d0015610 con 0x7f44e815ed50 2024-04-20T21:30:12.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.727+0000 7f44e4c00640 1 -- 172.21.15.86:0/1722240540 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44d001a070 con 0x7f44e815ed50 2024-04-20T21:30:12.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.727+0000 7f44e4c00640 1 -- 172.21.15.86:0/1722240540 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d00159f0 con 0x7f44e815ed50 2024-04-20T21:30:12.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.727+0000 7f44edc00640 1 -- 172.21.15.86:0/1722240540 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44e815f710 con 0x7f44e815ed50 2024-04-20T21:30:12.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.727+0000 7f44edc00640 1 -- 172.21.15.86:0/1722240540 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44e81f2d60 con 0x7f44e815ed50 2024-04-20T21:30:12.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.728+0000 7f44edc00640 1 -- 172.21.15.86:0/1722240540 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44e804fcf0 con 0x7f44e815ed50 2024-04-20T21:30:12.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.729+0000 7f44e4c00640 1 -- 172.21.15.86:0/1722240540 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f44d0012030 con 0x7f44e815ed50 2024-04-20T21:30:12.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.730+0000 7f44e4c00640 1 --2- 172.21.15.86:0/1722240540 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f44c406c920 0x7f44c406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:12.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.730+0000 7f44e4c00640 1 -- 172.21.15.86:0/1722240540 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7f44d008c5a0 con 0x7f44e815ed50 2024-04-20T21:30:12.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.730+0000 7f44e7e00640 1 --2- 172.21.15.86:0/1722240540 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f44c406c920 0x7f44c406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:12.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.731+0000 7f44e7e00640 1 --2- 172.21.15.86:0/1722240540 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f44c406c920 0x7f44c406ede0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f44d8034000 tx=0x7f44d8007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:12.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.735+0000 7f44e4c00640 1 -- 172.21.15.86:0/1722240540 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f44e81f2ae0 con 0x7f44e815ed50 2024-04-20T21:30:12.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.946+0000 7f44edc00640 1 -- 172.21.15.86:0/1722240540 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f44e806e360 con 0x7f44e815ed50 2024-04-20T21:30:12.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.947+0000 7f44e4c00640 1 -- 172.21.15.86:0/1722240540 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v112) ==== 80+0+0 (secure 0 0 0) 0x7f44d0010040 con 0x7f44e815ed50 2024-04-20T21:30:12.946 INFO:teuthology.orchestra.run.smithi086.stderr:listed 0 entries 2024-04-20T21:30:12.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.949+0000 7f44edc00640 1 -- 172.21.15.86:0/1722240540 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f44c406c920 msgr2=0x7f44c406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:12.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.949+0000 7f44edc00640 1 --2- 172.21.15.86:0/1722240540 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f44c406c920 0x7f44c406ede0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f44d8034000 tx=0x7f44d8007000 comp rx=0 tx=0).stop 2024-04-20T21:30:12.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.949+0000 7f44edc00640 1 -- 172.21.15.86:0/1722240540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44e815ed50 msgr2=0x7f44e815f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:12.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.949+0000 7f44edc00640 1 --2- 172.21.15.86:0/1722240540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44e815ed50 0x7f44e815f1d0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f44d0009fb0 tx=0x7f44d000e5d0 comp rx=0 tx=0).stop 2024-04-20T21:30:12.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.950+0000 7f44edc00640 1 -- 172.21.15.86:0/1722240540 shutdown_connections 2024-04-20T21:30:12.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.950+0000 7f44edc00640 1 --2- 172.21.15.86:0/1722240540 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f44c406c920 0x7f44c406ede0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:12.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.950+0000 7f44edc00640 1 --2- 172.21.15.86:0/1722240540 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f44e81f60d0 0x7f44e81f6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:12.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.950+0000 7f44edc00640 1 --2- 172.21.15.86:0/1722240540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44e815ed50 0x7f44e815f1d0 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:12.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.950+0000 7f44edc00640 1 --2- 172.21.15.86:0/1722240540 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f44cc0039b0 0x7f44e80ae290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:12.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.950+0000 7f44edc00640 1 -- 172.21.15.86:0/1722240540 >> 172.21.15.86:0/1722240540 conn(0x7f44e806bb60 msgr2=0x7f44e80d0090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:12.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.950+0000 7f44edc00640 1 -- 172.21.15.86:0/1722240540 shutdown_connections 2024-04-20T21:30:12.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:12.950+0000 7f44edc00640 1 -- 172.21.15.86:0/1722240540 wait complete. 2024-04-20T21:30:12.966 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi086.front.sepia.ceph.com... 2024-04-20T21:30:12.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:30:12.966 DEBUG:teuthology.orchestra.run.smithi086:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-20T21:30:13.038 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[65871]: fuse finished with error 0 and tester_r 0 2024-04-20T21:30:13.044 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:30:18.521 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:30:18.529 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:30:19.146 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-20T21:30:19.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:30:19.146 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-20T21:30:19.206 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:30:19.206 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:30:19.262 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052731,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248996k,nr_inodes=812249,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T21:30:19.262 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi086.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2024-04-20T21:30:19.262 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-20T21:30:19.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:30:19.263 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-20T21:30:19.317 INFO:teuthology.orchestra.run.smithi086.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-20T21:30:19.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:30:19.319 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-20T21:30:19.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.525+0000 7fa5a8600640 1 Processor -- start 2024-04-20T21:30:19.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.525+0000 7fa5a8600640 1 -- start start 2024-04-20T21:30:19.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.527+0000 7fa5a8600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa5a0145fe0 0x7fa5a00d02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:19.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.527+0000 7fa5a8600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5a00d0810 0x7fa5a00cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:19.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.527+0000 7fa5a8600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5a015dcc0 con 0x7fa5a00cd610 2024-04-20T21:30:19.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.527+0000 7fa5a8600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fa5a00d00d0 con 0x7fa5a0145fe0 2024-04-20T21:30:19.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.527+0000 7fa5a8600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa5a00d0c70 con 0x7fa5a00d0810 2024-04-20T21:30:19.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.528+0000 7fa5a6600640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fa5a00cd610 0x7fa5a0160350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54122/0 (socket says 172.21.15.86:54122) 2024-04-20T21:30:19.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.528+0000 7fa5a6600640 1 -- 172.21.15.86:0/3753941726 learned_addr learned my addr 172.21.15.86:0/3753941726 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:30:19.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.528+0000 7fa5a5200640 1 --2- 172.21.15.86:0/3753941726 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5a00d0810 0x7fa5a00cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:19.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.528+0000 7fa5a5c00640 1 --2- 172.21.15.86:0/3753941726 >> v2:172.21.15.144:3300/0 conn(0x7fa5a0145fe0 0x7fa5a00d02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:19.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.528+0000 7fa59fe00640 1 -- 172.21.15.86:0/3753941726 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1225143971 0 0) 0x7fa5a015dcc0 con 0x7fa5a00cd610 2024-04-20T21:30:19.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.528+0000 7fa59fe00640 1 -- 172.21.15.86:0/3753941726 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa588003520 con 0x7fa5a00cd610 2024-04-20T21:30:19.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.528+0000 7fa59fe00640 1 -- 172.21.15.86:0/3753941726 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 418645376 0 0) 0x7fa588003520 con 0x7fa5a00cd610 2024-04-20T21:30:19.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.528+0000 7fa59fe00640 1 -- 172.21.15.86:0/3753941726 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5a00d0810 msgr2=0x7fa5a00cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:19.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.528+0000 7fa59fe00640 1 --2- 172.21.15.86:0/3753941726 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5a00d0810 0x7fa5a00cd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:19.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.528+0000 7fa59fe00640 1 -- 172.21.15.86:0/3753941726 >> v2:172.21.15.144:3300/0 conn(0x7fa5a0145fe0 msgr2=0x7fa5a00d02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:19.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.528+0000 7fa59fe00640 1 --2- 172.21.15.86:0/3753941726 >> v2:172.21.15.144:3300/0 conn(0x7fa5a0145fe0 0x7fa5a00d02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:19.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.528+0000 7fa59fe00640 1 -- 172.21.15.86:0/3753941726 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a015cf00 con 0x7fa5a00cd610 2024-04-20T21:30:19.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.528+0000 7fa59fe00640 1 -- 172.21.15.86:0/3753941726 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fa5980019f0 con 0x7fa5a00cd610 2024-04-20T21:30:19.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.528+0000 7fa59fe00640 1 -- 172.21.15.86:0/3753941726 >> v1:172.21.15.86:6789/0 conn(0x7fa5a00cd610 legacy=0x7fa5a0160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:19.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.529+0000 7fa59fe00640 1 --2- 172.21.15.86:0/3753941726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5880039b0 0x7fa588003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:19.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.529+0000 7fa59fe00640 1 --2- 172.21.15.86:0/3753941726 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa588004800 0x7fa588004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:19.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.529+0000 7fa59fe00640 1 --2- 172.21.15.86:0/3753941726 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5a00d0810 0x7fa5880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:19.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.529+0000 7fa59fe00640 1 -- 172.21.15.86:0/3753941726 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a00d00d0 con 0x7fa5880039b0 2024-04-20T21:30:19.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.529+0000 7fa59fe00640 1 -- 172.21.15.86:0/3753941726 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa58800b9c0 con 0x7fa588004800 2024-04-20T21:30:19.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.529+0000 7fa59fe00640 1 -- 172.21.15.86:0/3753941726 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa58800bb40 con 0x7fa5a00d0810 2024-04-20T21:30:19.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.529+0000 7fa5a5c00640 1 --2- 172.21.15.86:0/3753941726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5880039b0 0x7fa588003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:19.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.529+0000 7fa5a5c00640 1 -- 172.21.15.86:0/3753941726 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5a00d0810 msgr2=0x7fa5880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:19.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.529+0000 7fa5a5c00640 1 --2- 172.21.15.86:0/3753941726 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5a00d0810 0x7fa5880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:19.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.529+0000 7fa5a5c00640 1 -- 172.21.15.86:0/3753941726 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa588004800 msgr2=0x7fa588004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:19.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.529+0000 7fa5a5c00640 1 --2- 172.21.15.86:0/3753941726 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa588004800 0x7fa588004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:19.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.529+0000 7fa5a5c00640 1 -- 172.21.15.86:0/3753941726 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa588003520 con 0x7fa5880039b0 2024-04-20T21:30:19.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.530+0000 7fa5a5c00640 1 --2- 172.21.15.86:0/3753941726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5880039b0 0x7fa588003dd0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7fa594009ac0 tx=0x7fa588005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bda19a9b02d6d182 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:19.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.531+0000 7fa59fe00640 1 -- 172.21.15.86:0/3753941726 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa594039070 con 0x7fa5880039b0 2024-04-20T21:30:19.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.531+0000 7fa59fe00640 1 -- 172.21.15.86:0/3753941726 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa59402c6d0 con 0x7fa5880039b0 2024-04-20T21:30:19.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.531+0000 7fa59fe00640 1 -- 172.21.15.86:0/3753941726 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa594009690 con 0x7fa5880039b0 2024-04-20T21:30:19.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.531+0000 7fa5a8600640 1 -- 172.21.15.86:0/3753941726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5880039b0 msgr2=0x7fa588003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:19.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.531+0000 7fa5a8600640 1 --2- 172.21.15.86:0/3753941726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5880039b0 0x7fa588003dd0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7fa594009ac0 tx=0x7fa588005830 comp rx=0 tx=0).stop 2024-04-20T21:30:19.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.531+0000 7fa5a6600640 1 -- 172.21.15.86:0/3753941726 reap_dead start 2024-04-20T21:30:19.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.531+0000 7fa5a8600640 1 -- 172.21.15.86:0/3753941726 shutdown_connections 2024-04-20T21:30:19.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.531+0000 7fa5a8600640 1 -- 172.21.15.86:0/3753941726 >> 172.21.15.86:0/3753941726 conn(0x7fa5a006b980 msgr2=0x7fa5a006bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:19.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.531+0000 7fa5a8600640 1 -- 172.21.15.86:0/3753941726 shutdown_connections 2024-04-20T21:30:19.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.531+0000 7fa5a8600640 1 -- 172.21.15.86:0/3753941726 wait complete. 2024-04-20T21:30:19.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.533+0000 7fa5a8600640 1 Processor -- start 2024-04-20T21:30:19.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.533+0000 7fa5a8600640 1 -- start start 2024-04-20T21:30:19.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.534+0000 7fa5a8600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5880039b0 0x7fa5a015c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:19.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.534+0000 7fa5a8600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5a0142420 0x7fa5a00b0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:19.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.534+0000 7fa5a8600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5a00b1430 0x7fa5a0140190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:19.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.534+0000 7fa5a8600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a00cdaa0 con 0x7fa5880039b0 2024-04-20T21:30:19.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.534+0000 7fa5a8600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a015d4c0 con 0x7fa5a00b1430 2024-04-20T21:30:19.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.534+0000 7fa5a8600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa5a015d7a0 con 0x7fa5a0142420 2024-04-20T21:30:19.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.534+0000 7fa5a5c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5880039b0 0x7fa5a015c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:19.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.534+0000 7fa5a5c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5880039b0 0x7fa5a015c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51748/0 (socket says 172.21.15.86:51748) 2024-04-20T21:30:19.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.534+0000 7fa5a5c00640 1 -- 172.21.15.86:0/82681933 learned_addr learned my addr 172.21.15.86:0/82681933 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:19.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.535+0000 7fa5a5200640 1 --2- 172.21.15.86:0/82681933 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5a0142420 0x7fa5a00b0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:19.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.535+0000 7fa5a6600640 1 --2- 172.21.15.86:0/82681933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5a00b1430 0x7fa5a0140190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:19.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.535+0000 7fa5a5c00640 1 -- 172.21.15.86:0/82681933 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5a0142420 msgr2=0x7fa5a00b0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:19.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.535+0000 7fa5a5c00640 1 --2- 172.21.15.86:0/82681933 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5a0142420 0x7fa5a00b0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:19.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.535+0000 7fa5a5c00640 1 -- 172.21.15.86:0/82681933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5a00b1430 msgr2=0x7fa5a0140190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:19.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.535+0000 7fa5a5c00640 1 --2- 172.21.15.86:0/82681933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5a00b1430 0x7fa5a0140190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:19.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.535+0000 7fa5a5c00640 1 -- 172.21.15.86:0/82681933 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a00b18d0 con 0x7fa5880039b0 2024-04-20T21:30:19.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.535+0000 7fa5a5c00640 1 --2- 172.21.15.86:0/82681933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5880039b0 0x7fa5a015c180 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7fa594037040 tx=0x7fa594004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:19.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.536+0000 7fa59d600640 1 -- 172.21.15.86:0/82681933 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa594039050 con 0x7fa5880039b0 2024-04-20T21:30:19.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.536+0000 7fa59d600640 1 -- 172.21.15.86:0/82681933 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa594045070 con 0x7fa5880039b0 2024-04-20T21:30:19.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.536+0000 7fa59d600640 1 -- 172.21.15.86:0/82681933 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa594003a50 con 0x7fa5880039b0 2024-04-20T21:30:19.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.536+0000 7fa5a8600640 1 -- 172.21.15.86:0/82681933 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5a00b1bb0 con 0x7fa5880039b0 2024-04-20T21:30:19.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.536+0000 7fa5a8600640 1 -- 172.21.15.86:0/82681933 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5a00b2170 con 0x7fa5880039b0 2024-04-20T21:30:19.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.537+0000 7fa5a8600640 1 -- 172.21.15.86:0/82681933 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5a00689e0 con 0x7fa5880039b0 2024-04-20T21:30:19.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.538+0000 7fa59d600640 1 -- 172.21.15.86:0/82681933 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa594003bf0 con 0x7fa5880039b0 2024-04-20T21:30:19.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.539+0000 7fa59d600640 1 --2- 172.21.15.86:0/82681933 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa58006c940 0x7fa58006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:19.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.539+0000 7fa59d600640 1 -- 172.21.15.86:0/82681933 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5709+0+0 (secure 0 0 0) 0x7fa5940ad660 con 0x7fa5880039b0 2024-04-20T21:30:19.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.539+0000 7fa5a5200640 1 --2- 172.21.15.86:0/82681933 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa58006c940 0x7fa58006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:19.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.540+0000 7fa5a5200640 1 --2- 172.21.15.86:0/82681933 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa58006c940 0x7fa58006ee00 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fa5a013d110 tx=0x7fa58c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:19.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.544+0000 7fa59d600640 1 -- 172.21.15.86:0/82681933 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa59404e020 con 0x7fa5880039b0 2024-04-20T21:30:19.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.783+0000 7fa5a8600640 1 -- 172.21.15.86:0/82681933 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa5a00d1000 con 0x7fa5880039b0 2024-04-20T21:30:19.782 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-04-20T21:30:19.784+0000 7f5bb9c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T21:30:19.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.787+0000 7fa59d600640 1 -- 172.21.15.86:0/82681933 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v100) ==== 161+0+0 (secure 0 0 0) 0x7fa59407ba70 con 0x7fa5880039b0 2024-04-20T21:30:19.786 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-20T21:30:19.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.790+0000 7fa5a8600640 1 -- 172.21.15.86:0/82681933 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa58006c940 msgr2=0x7fa58006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:19.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.790+0000 7fa5a8600640 1 --2- 172.21.15.86:0/82681933 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa58006c940 0x7fa58006ee00 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fa5a013d110 tx=0x7fa58c008040 comp rx=0 tx=0).stop 2024-04-20T21:30:19.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.790+0000 7fa5a8600640 1 -- 172.21.15.86:0/82681933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5880039b0 msgr2=0x7fa5a015c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:19.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.790+0000 7fa5a8600640 1 --2- 172.21.15.86:0/82681933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5880039b0 0x7fa5a015c180 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7fa594037040 tx=0x7fa594004730 comp rx=0 tx=0).stop 2024-04-20T21:30:19.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.790+0000 7fa5a8600640 1 -- 172.21.15.86:0/82681933 shutdown_connections 2024-04-20T21:30:19.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.790+0000 7fa5a8600640 1 --2- 172.21.15.86:0/82681933 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa58006c940 0x7fa58006ee00 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:19.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.790+0000 7fa5a8600640 1 --2- 172.21.15.86:0/82681933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5a00b1430 0x7fa5a0140190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:19.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.790+0000 7fa5a8600640 1 --2- 172.21.15.86:0/82681933 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa5a0142420 0x7fa5a00b0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:19.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.790+0000 7fa5a8600640 1 --2- 172.21.15.86:0/82681933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5880039b0 0x7fa5a015c180 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:19.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.790+0000 7fa5a8600640 1 -- 172.21.15.86:0/82681933 >> 172.21.15.86:0/82681933 conn(0x7fa5a006b980 msgr2=0x7fa5a015f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:19.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.790+0000 7fa5a8600640 1 -- 172.21.15.86:0/82681933 shutdown_connections 2024-04-20T21:30:19.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:19.791+0000 7fa5a8600640 1 -- 172.21.15.86:0/82681933 wait complete. 2024-04-20T21:30:19.810 INFO:tasks.ceph.mds.a.smithi086.stderr: -8237> 2024-04-20T21:29:50.272+0000 7f4d3f600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T21:30:19.817 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-20T21:30:19.825 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-20T21:30:19.876 INFO:tasks.ceph.mds.a.smithi086.stdout:starting mds.a at 2024-04-20T21:30:19.937 INFO:teuthology.orchestra.run.smithi086.stdout:[global] 2024-04-20T21:30:19.937 INFO:teuthology.orchestra.run.smithi086.stdout: chdir = "" 2024-04-20T21:30:19.937 INFO:teuthology.orchestra.run.smithi086.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-20T21:30:19.937 INFO:teuthology.orchestra.run.smithi086.stdout: auth supported = cephx 2024-04-20T21:30:19.937 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:19.937 INFO:teuthology.orchestra.run.smithi086.stdout: filestore xattr use omap = true 2024-04-20T21:30:19.937 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:19.937 INFO:teuthology.orchestra.run.smithi086.stdout: mon clock drift allowed = 1.000 2024-04-20T21:30:19.937 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:19.937 INFO:teuthology.orchestra.run.smithi086.stdout: osd crush chooseleaf type = 0 2024-04-20T21:30:19.937 INFO:teuthology.orchestra.run.smithi086.stdout: auth debug = true 2024-04-20T21:30:19.938 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:19.938 INFO:teuthology.orchestra.run.smithi086.stdout: ms die on old message = true 2024-04-20T21:30:19.938 INFO:teuthology.orchestra.run.smithi086.stdout: ms die on bug = true 2024-04-20T21:30:19.938 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:19.938 INFO:teuthology.orchestra.run.smithi086.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-20T21:30:19.938 INFO:teuthology.orchestra.run.smithi086.stdout: mon pg warn max object skew = 0 2024-04-20T21:30:19.938 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:19.938 INFO:teuthology.orchestra.run.smithi086.stdout: # disable pg_autoscaler by default for new pools 2024-04-20T21:30:19.938 INFO:teuthology.orchestra.run.smithi086.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-20T21:30:19.938 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:19.938 INFO:teuthology.orchestra.run.smithi086.stdout: osd pool default size = 2 2024-04-20T21:30:19.938 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:19.938 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd allow primary affinity = true 2024-04-20T21:30:19.939 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd allow pg remap = true 2024-04-20T21:30:19.939 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on legacy crush tunables = false 2024-04-20T21:30:19.939 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on crush straw calc version zero = false 2024-04-20T21:30:19.939 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on no sortbitwise = false 2024-04-20T21:30:19.939 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on osd down out interval zero = false 2024-04-20T21:30:19.939 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on too few osds = false 2024-04-20T21:30:19.939 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-20T21:30:19.939 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-20T21:30:19.939 INFO:teuthology.orchestra.run.smithi086.stdout: mon_allow_pool_size_one = true 2024-04-20T21:30:19.939 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:19.939 INFO:teuthology.orchestra.run.smithi086.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-20T21:30:19.939 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:19.939 INFO:teuthology.orchestra.run.smithi086.stdout: osd default data pool replay window = 5 2024-04-20T21:30:19.939 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:19.939 INFO:teuthology.orchestra.run.smithi086.stdout: mon allow pool delete = true 2024-04-20T21:30:19.940 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:19.940 INFO:teuthology.orchestra.run.smithi086.stdout: mon cluster log file level = debug 2024-04-20T21:30:19.940 INFO:teuthology.orchestra.run.smithi086.stdout: debug asserts on shutdown = true 2024-04-20T21:30:19.940 INFO:teuthology.orchestra.run.smithi086.stdout: mon health detail to clog = false 2024-04-20T21:30:19.940 INFO:teuthology.orchestra.run.smithi086.stdout: mon host = "172.21.15.86,172.21.15.144,[v2:172.21.15.144:3301,v1:172.21.15.144:6790]" 2024-04-20T21:30:19.940 INFO:teuthology.orchestra.run.smithi086.stdout: fsid = 9c2dec1d-d7ae-4a83-a812-79922808e278 2024-04-20T21:30:19.940 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:19.940 INFO:teuthology.orchestra.run.smithi086.stdout:[osd] 2024-04-20T21:30:19.940 INFO:teuthology.orchestra.run.smithi086.stdout: osd journal size = 100 2024-04-20T21:30:19.940 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:19.940 INFO:teuthology.orchestra.run.smithi086.stdout: osd scrub load threshold = 5.0 2024-04-20T21:30:19.940 INFO:teuthology.orchestra.run.smithi086.stdout: osd scrub max interval = 600 2024-04-20T21:30:19.940 INFO:teuthology.orchestra.run.smithi086.stdout: osd mclock profile = high_recovery_ops 2024-04-20T21:30:19.940 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:19.940 INFO:teuthology.orchestra.run.smithi086.stdout: osd recover clone overlap = true 2024-04-20T21:30:19.941 INFO:teuthology.orchestra.run.smithi086.stdout: osd recovery max chunk = 1048576 2024-04-20T21:30:19.941 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:19.941 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug shutdown = true 2024-04-20T21:30:19.941 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug op order = true 2024-04-20T21:30:19.941 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug verify stray on activate = true 2024-04-20T21:30:19.941 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug trim objects = true 2024-04-20T21:30:19.941 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:19.941 INFO:teuthology.orchestra.run.smithi086.stdout: osd open classes on start = true 2024-04-20T21:30:19.941 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug pg log writeout = true 2024-04-20T21:30:19.941 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:19.941 INFO:teuthology.orchestra.run.smithi086.stdout: osd deep scrub update digest min age = 30 2024-04-20T21:30:19.941 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:19.941 INFO:teuthology.orchestra.run.smithi086.stdout: osd map max advance = 10 2024-04-20T21:30:19.941 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:19.942 INFO:teuthology.orchestra.run.smithi086.stdout: journal zero on create = true 2024-04-20T21:30:19.942 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:19.942 INFO:teuthology.orchestra.run.smithi086.stdout: filestore ondisk finisher threads = 3 2024-04-20T21:30:19.942 INFO:teuthology.orchestra.run.smithi086.stdout: filestore apply finisher threads = 3 2024-04-20T21:30:19.942 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:19.942 INFO:teuthology.orchestra.run.smithi086.stdout: bdev debug aio = true 2024-04-20T21:30:19.942 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug misdirected ops = true 2024-04-20T21:30:19.942 INFO:teuthology.orchestra.run.smithi086.stdout: bdev async discard = True 2024-04-20T21:30:19.942 INFO:teuthology.orchestra.run.smithi086.stdout: bdev enable discard = True 2024-04-20T21:30:19.942 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore allocator = bitmap 2024-04-20T21:30:19.942 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore block size = 96636764160 2024-04-20T21:30:19.942 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore fsck on mount = True 2024-04-20T21:30:19.942 INFO:teuthology.orchestra.run.smithi086.stdout: debug bluefs = 1/20 2024-04-20T21:30:19.943 INFO:teuthology.orchestra.run.smithi086.stdout: debug bluestore = 1/20 2024-04-20T21:30:19.943 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-20T21:30:19.943 INFO:teuthology.orchestra.run.smithi086.stdout: debug osd = 20 2024-04-20T21:30:19.943 INFO:teuthology.orchestra.run.smithi086.stdout: debug rocksdb = 4/10 2024-04-20T21:30:19.943 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd backfillfull_ratio = 0.85 2024-04-20T21:30:19.943 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd full ratio = 0.9 2024-04-20T21:30:19.943 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd nearfull ratio = 0.8 2024-04-20T21:30:19.943 INFO:teuthology.orchestra.run.smithi086.stdout: osd failsafe full ratio = 0.95 2024-04-20T21:30:19.943 INFO:teuthology.orchestra.run.smithi086.stdout: osd objectstore = bluestore 2024-04-20T21:30:19.943 INFO:teuthology.orchestra.run.smithi086.stdout: osd op complaint time = 180 2024-04-20T21:30:19.943 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:19.943 INFO:teuthology.orchestra.run.smithi086.stdout:[mgr] 2024-04-20T21:30:19.943 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-20T21:30:19.943 INFO:teuthology.orchestra.run.smithi086.stdout: debug mgr = 20 2024-04-20T21:30:19.944 INFO:teuthology.orchestra.run.smithi086.stdout: debug mon = 20 2024-04-20T21:30:19.944 INFO:teuthology.orchestra.run.smithi086.stdout: debug auth = 20 2024-04-20T21:30:19.944 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min pgs per osd = 4 2024-04-20T21:30:19.944 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min bytes per osd = 10 2024-04-20T21:30:19.944 INFO:teuthology.orchestra.run.smithi086.stdout: mgr/telemetry/nag = false 2024-04-20T21:30:19.944 INFO:teuthology.orchestra.run.smithi086.stdout: debug client = 20 2024-04-20T21:30:19.944 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:19.944 INFO:teuthology.orchestra.run.smithi086.stdout:[mon] 2024-04-20T21:30:19.944 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-20T21:30:19.944 INFO:teuthology.orchestra.run.smithi086.stdout: debug mon = 20 2024-04-20T21:30:19.944 INFO:teuthology.orchestra.run.smithi086.stdout: debug paxos = 20 2024-04-20T21:30:19.944 INFO:teuthology.orchestra.run.smithi086.stdout: debug auth = 20 2024-04-20T21:30:19.944 INFO:teuthology.orchestra.run.smithi086.stdout: mon data avail warn = 5 2024-04-20T21:30:19.944 INFO:teuthology.orchestra.run.smithi086.stdout: mon mgr mkfs grace = 240 2024-04-20T21:30:19.945 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min pgs per osd = 4 2024-04-20T21:30:19.945 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd reporter subtree level = osd 2024-04-20T21:30:19.945 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd prime pg temp = true 2024-04-20T21:30:19.945 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min bytes per osd = 10 2024-04-20T21:30:19.945 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:19.945 INFO:teuthology.orchestra.run.smithi086.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-20T21:30:19.945 INFO:teuthology.orchestra.run.smithi086.stdout: auth mon ticket ttl = 660 # 11m 2024-04-20T21:30:19.945 INFO:teuthology.orchestra.run.smithi086.stdout: auth service ticket ttl = 240 # 4m 2024-04-20T21:30:19.945 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:19.945 INFO:teuthology.orchestra.run.smithi086.stdout: # don't complain about insecure global_id in the test suite 2024-04-20T21:30:19.945 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-20T21:30:19.945 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-20T21:30:19.945 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:19.945 INFO:teuthology.orchestra.run.smithi086.stdout: # 1m isn't quite enough 2024-04-20T21:30:19.946 INFO:teuthology.orchestra.run.smithi086.stdout: mon_down_mkfs_grace = 2m 2024-04-20T21:30:19.946 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:19.946 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_filestore_osds = false 2024-04-20T21:30:19.946 INFO:teuthology.orchestra.run.smithi086.stdout: mon down mkfs grace = 300 2024-04-20T21:30:19.946 INFO:teuthology.orchestra.run.smithi086.stdout: mon op complaint time = 120 2024-04-20T21:30:19.946 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:19.946 INFO:teuthology.orchestra.run.smithi086.stdout:[client] 2024-04-20T21:30:19.946 INFO:teuthology.orchestra.run.smithi086.stdout: rgw cache enabled = true 2024-04-20T21:30:19.946 INFO:teuthology.orchestra.run.smithi086.stdout: rgw enable ops log = true 2024-04-20T21:30:19.946 INFO:teuthology.orchestra.run.smithi086.stdout: rgw enable usage log = true 2024-04-20T21:30:19.946 INFO:teuthology.orchestra.run.smithi086.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-20T21:30:19.946 INFO:teuthology.orchestra.run.smithi086.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-20T21:30:19.946 INFO:teuthology.orchestra.run.smithi086.stdout: client mount timeout = 600 2024-04-20T21:30:19.946 INFO:teuthology.orchestra.run.smithi086.stdout: debug client = 20 2024-04-20T21:30:19.947 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-20T21:30:19.947 INFO:teuthology.orchestra.run.smithi086.stdout: pid file = "" 2024-04-20T21:30:19.947 INFO:teuthology.orchestra.run.smithi086.stdout: rados mon op timeout = 900 2024-04-20T21:30:19.947 INFO:teuthology.orchestra.run.smithi086.stdout: rados osd op timeout = 900 2024-04-20T21:30:19.947 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.a] 2024-04-20T21:30:19.947 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.b] 2024-04-20T21:30:19.947 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.c] 2024-04-20T21:30:19.947 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.a] 2024-04-20T21:30:19.947 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.c] 2024-04-20T21:30:19.947 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.b] 2024-04-20T21:30:19.947 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.d] 2024-04-20T21:30:19.947 INFO:teuthology.orchestra.run.smithi086.stdout:[mds] 2024-04-20T21:30:19.947 INFO:teuthology.orchestra.run.smithi086.stdout: debug mds = 20 2024-04-20T21:30:19.947 INFO:teuthology.orchestra.run.smithi086.stdout: debug mds balancer = 20 2024-04-20T21:30:19.947 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-20T21:30:19.948 INFO:teuthology.orchestra.run.smithi086.stdout: mds debug frag = True 2024-04-20T21:30:19.948 INFO:teuthology.orchestra.run.smithi086.stdout: mds debug scatterstat = True 2024-04-20T21:30:19.948 INFO:teuthology.orchestra.run.smithi086.stdout: mds op complaint time = 180 2024-04-20T21:30:19.948 INFO:teuthology.orchestra.run.smithi086.stdout: mds verify scatter = False 2024-04-20T21:30:19.948 INFO:teuthology.orchestra.run.smithi086.stdout: osd op complaint time = 180 2024-04-20T21:30:19.948 INFO:teuthology.orchestra.run.smithi086.stdout: rados mon op timeout = 900 2024-04-20T21:30:19.948 INFO:teuthology.orchestra.run.smithi086.stdout: rados osd op timeout = 900 2024-04-20T21:30:19.970 INFO:teuthology.orchestra.run.smithi144.stdout:[global] 2024-04-20T21:30:19.970 INFO:teuthology.orchestra.run.smithi144.stdout: chdir = "" 2024-04-20T21:30:19.970 INFO:teuthology.orchestra.run.smithi144.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-20T21:30:19.970 INFO:teuthology.orchestra.run.smithi144.stdout: auth supported = cephx 2024-04-20T21:30:19.970 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:19.970 INFO:teuthology.orchestra.run.smithi144.stdout: filestore xattr use omap = true 2024-04-20T21:30:19.970 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:19.970 INFO:teuthology.orchestra.run.smithi144.stdout: mon clock drift allowed = 1.000 2024-04-20T21:30:19.970 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:19.970 INFO:teuthology.orchestra.run.smithi144.stdout: osd crush chooseleaf type = 0 2024-04-20T21:30:19.970 INFO:teuthology.orchestra.run.smithi144.stdout: auth debug = true 2024-04-20T21:30:19.971 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:19.971 INFO:teuthology.orchestra.run.smithi144.stdout: ms die on old message = true 2024-04-20T21:30:19.971 INFO:teuthology.orchestra.run.smithi144.stdout: ms die on bug = true 2024-04-20T21:30:19.971 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:19.971 INFO:teuthology.orchestra.run.smithi144.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-20T21:30:19.971 INFO:teuthology.orchestra.run.smithi144.stdout: mon pg warn max object skew = 0 2024-04-20T21:30:19.971 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:19.971 INFO:teuthology.orchestra.run.smithi144.stdout: # disable pg_autoscaler by default for new pools 2024-04-20T21:30:19.971 INFO:teuthology.orchestra.run.smithi144.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-20T21:30:19.971 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:19.971 INFO:teuthology.orchestra.run.smithi144.stdout: osd pool default size = 2 2024-04-20T21:30:19.971 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:19.971 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd allow primary affinity = true 2024-04-20T21:30:19.971 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd allow pg remap = true 2024-04-20T21:30:19.971 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on legacy crush tunables = false 2024-04-20T21:30:19.971 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on crush straw calc version zero = false 2024-04-20T21:30:19.972 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on no sortbitwise = false 2024-04-20T21:30:19.972 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on osd down out interval zero = false 2024-04-20T21:30:19.972 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on too few osds = false 2024-04-20T21:30:19.972 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-20T21:30:19.972 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-20T21:30:19.972 INFO:teuthology.orchestra.run.smithi144.stdout: mon_allow_pool_size_one = true 2024-04-20T21:30:19.972 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:19.972 INFO:teuthology.orchestra.run.smithi144.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-20T21:30:19.972 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:19.972 INFO:teuthology.orchestra.run.smithi144.stdout: osd default data pool replay window = 5 2024-04-20T21:30:19.972 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:19.972 INFO:teuthology.orchestra.run.smithi144.stdout: mon allow pool delete = true 2024-04-20T21:30:19.972 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:19.972 INFO:teuthology.orchestra.run.smithi144.stdout: mon cluster log file level = debug 2024-04-20T21:30:19.973 INFO:teuthology.orchestra.run.smithi144.stdout: debug asserts on shutdown = true 2024-04-20T21:30:19.973 INFO:teuthology.orchestra.run.smithi144.stdout: mon health detail to clog = false 2024-04-20T21:30:19.973 INFO:teuthology.orchestra.run.smithi144.stdout: mon host = "172.21.15.86,172.21.15.144,[v2:172.21.15.144:3301,v1:172.21.15.144:6790]" 2024-04-20T21:30:19.973 INFO:teuthology.orchestra.run.smithi144.stdout: fsid = 9c2dec1d-d7ae-4a83-a812-79922808e278 2024-04-20T21:30:19.973 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:19.973 INFO:teuthology.orchestra.run.smithi144.stdout:[osd] 2024-04-20T21:30:19.973 INFO:teuthology.orchestra.run.smithi144.stdout: osd journal size = 100 2024-04-20T21:30:19.973 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:19.973 INFO:teuthology.orchestra.run.smithi144.stdout: osd scrub load threshold = 5.0 2024-04-20T21:30:19.973 INFO:teuthology.orchestra.run.smithi144.stdout: osd scrub max interval = 600 2024-04-20T21:30:19.973 INFO:teuthology.orchestra.run.smithi144.stdout: osd mclock profile = high_recovery_ops 2024-04-20T21:30:19.974 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:19.974 INFO:teuthology.orchestra.run.smithi144.stdout: osd recover clone overlap = true 2024-04-20T21:30:19.974 INFO:teuthology.orchestra.run.smithi144.stdout: osd recovery max chunk = 1048576 2024-04-20T21:30:19.974 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:19.974 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug shutdown = true 2024-04-20T21:30:19.974 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug op order = true 2024-04-20T21:30:19.974 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug verify stray on activate = true 2024-04-20T21:30:19.974 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug trim objects = true 2024-04-20T21:30:19.974 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:19.974 INFO:teuthology.orchestra.run.smithi144.stdout: osd open classes on start = true 2024-04-20T21:30:19.974 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug pg log writeout = true 2024-04-20T21:30:19.974 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:19.975 INFO:teuthology.orchestra.run.smithi144.stdout: osd deep scrub update digest min age = 30 2024-04-20T21:30:19.975 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:19.975 INFO:teuthology.orchestra.run.smithi144.stdout: osd map max advance = 10 2024-04-20T21:30:19.975 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:19.975 INFO:teuthology.orchestra.run.smithi144.stdout: journal zero on create = true 2024-04-20T21:30:19.975 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:19.975 INFO:teuthology.orchestra.run.smithi144.stdout: filestore ondisk finisher threads = 3 2024-04-20T21:30:19.975 INFO:teuthology.orchestra.run.smithi144.stdout: filestore apply finisher threads = 3 2024-04-20T21:30:19.975 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:19.975 INFO:teuthology.orchestra.run.smithi144.stdout: bdev debug aio = true 2024-04-20T21:30:19.975 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug misdirected ops = true 2024-04-20T21:30:19.975 INFO:teuthology.orchestra.run.smithi144.stdout: bdev async discard = True 2024-04-20T21:30:19.975 INFO:teuthology.orchestra.run.smithi144.stdout: bdev enable discard = True 2024-04-20T21:30:19.975 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore allocator = bitmap 2024-04-20T21:30:19.976 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore block size = 96636764160 2024-04-20T21:30:19.976 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore fsck on mount = True 2024-04-20T21:30:19.976 INFO:teuthology.orchestra.run.smithi144.stdout: debug bluefs = 1/20 2024-04-20T21:30:19.976 INFO:teuthology.orchestra.run.smithi144.stdout: debug bluestore = 1/20 2024-04-20T21:30:19.976 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-04-20T21:30:19.976 INFO:teuthology.orchestra.run.smithi144.stdout: debug osd = 20 2024-04-20T21:30:19.976 INFO:teuthology.orchestra.run.smithi144.stdout: debug rocksdb = 4/10 2024-04-20T21:30:19.976 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd backfillfull_ratio = 0.85 2024-04-20T21:30:19.976 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd full ratio = 0.9 2024-04-20T21:30:19.976 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd nearfull ratio = 0.8 2024-04-20T21:30:19.976 INFO:teuthology.orchestra.run.smithi144.stdout: osd failsafe full ratio = 0.95 2024-04-20T21:30:19.976 INFO:teuthology.orchestra.run.smithi144.stdout: osd objectstore = bluestore 2024-04-20T21:30:19.976 INFO:teuthology.orchestra.run.smithi144.stdout: osd op complaint time = 180 2024-04-20T21:30:19.977 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:19.977 INFO:teuthology.orchestra.run.smithi144.stdout:[mgr] 2024-04-20T21:30:19.977 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-04-20T21:30:19.977 INFO:teuthology.orchestra.run.smithi144.stdout: debug mgr = 20 2024-04-20T21:30:19.977 INFO:teuthology.orchestra.run.smithi144.stdout: debug mon = 20 2024-04-20T21:30:19.977 INFO:teuthology.orchestra.run.smithi144.stdout: debug auth = 20 2024-04-20T21:30:19.977 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min pgs per osd = 4 2024-04-20T21:30:19.977 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min bytes per osd = 10 2024-04-20T21:30:19.977 INFO:teuthology.orchestra.run.smithi144.stdout: mgr/telemetry/nag = false 2024-04-20T21:30:19.977 INFO:teuthology.orchestra.run.smithi144.stdout: debug client = 20 2024-04-20T21:30:19.977 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:19.977 INFO:teuthology.orchestra.run.smithi144.stdout:[mon] 2024-04-20T21:30:19.977 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-04-20T21:30:19.977 INFO:teuthology.orchestra.run.smithi144.stdout: debug mon = 20 2024-04-20T21:30:19.978 INFO:teuthology.orchestra.run.smithi144.stdout: debug paxos = 20 2024-04-20T21:30:19.978 INFO:teuthology.orchestra.run.smithi144.stdout: debug auth = 20 2024-04-20T21:30:19.978 INFO:teuthology.orchestra.run.smithi144.stdout: mon data avail warn = 5 2024-04-20T21:30:19.978 INFO:teuthology.orchestra.run.smithi144.stdout: mon mgr mkfs grace = 240 2024-04-20T21:30:19.978 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min pgs per osd = 4 2024-04-20T21:30:19.978 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd reporter subtree level = osd 2024-04-20T21:30:19.978 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd prime pg temp = true 2024-04-20T21:30:19.978 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min bytes per osd = 10 2024-04-20T21:30:19.978 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:19.978 INFO:teuthology.orchestra.run.smithi144.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-20T21:30:19.978 INFO:teuthology.orchestra.run.smithi144.stdout: auth mon ticket ttl = 660 # 11m 2024-04-20T21:30:19.978 INFO:teuthology.orchestra.run.smithi144.stdout: auth service ticket ttl = 240 # 4m 2024-04-20T21:30:19.979 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:19.979 INFO:teuthology.orchestra.run.smithi144.stdout: # don't complain about insecure global_id in the test suite 2024-04-20T21:30:19.979 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-20T21:30:19.979 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-20T21:30:19.979 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:19.979 INFO:teuthology.orchestra.run.smithi144.stdout: # 1m isn't quite enough 2024-04-20T21:30:19.979 INFO:teuthology.orchestra.run.smithi144.stdout: mon_down_mkfs_grace = 2m 2024-04-20T21:30:19.979 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:19.979 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_filestore_osds = false 2024-04-20T21:30:19.979 INFO:teuthology.orchestra.run.smithi144.stdout: mon down mkfs grace = 300 2024-04-20T21:30:19.979 INFO:teuthology.orchestra.run.smithi144.stdout: mon op complaint time = 120 2024-04-20T21:30:19.979 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:19.979 INFO:teuthology.orchestra.run.smithi144.stdout:[client] 2024-04-20T21:30:19.979 INFO:teuthology.orchestra.run.smithi144.stdout: rgw cache enabled = true 2024-04-20T21:30:19.980 INFO:teuthology.orchestra.run.smithi144.stdout: rgw enable ops log = true 2024-04-20T21:30:19.980 INFO:teuthology.orchestra.run.smithi144.stdout: rgw enable usage log = true 2024-04-20T21:30:19.980 INFO:teuthology.orchestra.run.smithi144.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-20T21:30:19.980 INFO:teuthology.orchestra.run.smithi144.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-20T21:30:19.980 INFO:teuthology.orchestra.run.smithi144.stdout: client mount timeout = 600 2024-04-20T21:30:19.980 INFO:teuthology.orchestra.run.smithi144.stdout: debug client = 20 2024-04-20T21:30:19.980 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-04-20T21:30:19.980 INFO:teuthology.orchestra.run.smithi144.stdout: pid file = "" 2024-04-20T21:30:19.980 INFO:teuthology.orchestra.run.smithi144.stdout: rados mon op timeout = 900 2024-04-20T21:30:19.980 INFO:teuthology.orchestra.run.smithi144.stdout: rados osd op timeout = 900 2024-04-20T21:30:19.980 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.a] 2024-04-20T21:30:19.980 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.b] 2024-04-20T21:30:19.980 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.c] 2024-04-20T21:30:19.980 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.a] 2024-04-20T21:30:19.980 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.c] 2024-04-20T21:30:19.981 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.b] 2024-04-20T21:30:19.981 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.d] 2024-04-20T21:30:19.981 INFO:teuthology.orchestra.run.smithi144.stdout:[mds] 2024-04-20T21:30:19.981 INFO:teuthology.orchestra.run.smithi144.stdout: debug mds = 20 2024-04-20T21:30:19.981 INFO:teuthology.orchestra.run.smithi144.stdout: debug mds balancer = 20 2024-04-20T21:30:19.981 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-04-20T21:30:19.981 INFO:teuthology.orchestra.run.smithi144.stdout: mds debug frag = True 2024-04-20T21:30:19.981 INFO:teuthology.orchestra.run.smithi144.stdout: mds debug scatterstat = True 2024-04-20T21:30:19.981 INFO:teuthology.orchestra.run.smithi144.stdout: mds op complaint time = 180 2024-04-20T21:30:19.981 INFO:teuthology.orchestra.run.smithi144.stdout: mds verify scatter = False 2024-04-20T21:30:19.981 INFO:teuthology.orchestra.run.smithi144.stdout: osd op complaint time = 180 2024-04-20T21:30:19.981 INFO:teuthology.orchestra.run.smithi144.stdout: rados mon op timeout = 900 2024-04-20T21:30:19.982 INFO:teuthology.orchestra.run.smithi144.stdout: rados osd op timeout = 900 2024-04-20T21:30:20.004 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-20T21:30:20.011 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-20T21:30:20.135 INFO:teuthology.orchestra.run.smithi086.stdout:[global] 2024-04-20T21:30:20.135 INFO:teuthology.orchestra.run.smithi086.stdout: chdir = "" 2024-04-20T21:30:20.135 INFO:teuthology.orchestra.run.smithi086.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-20T21:30:20.135 INFO:teuthology.orchestra.run.smithi086.stdout: auth supported = cephx 2024-04-20T21:30:20.135 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:20.136 INFO:teuthology.orchestra.run.smithi086.stdout: filestore xattr use omap = true 2024-04-20T21:30:20.136 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:20.136 INFO:teuthology.orchestra.run.smithi086.stdout: mon clock drift allowed = 1.000 2024-04-20T21:30:20.136 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:20.136 INFO:teuthology.orchestra.run.smithi086.stdout: osd crush chooseleaf type = 0 2024-04-20T21:30:20.136 INFO:teuthology.orchestra.run.smithi086.stdout: auth debug = true 2024-04-20T21:30:20.136 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:20.136 INFO:teuthology.orchestra.run.smithi086.stdout: ms die on old message = true 2024-04-20T21:30:20.136 INFO:teuthology.orchestra.run.smithi086.stdout: ms die on bug = true 2024-04-20T21:30:20.136 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:20.136 INFO:teuthology.orchestra.run.smithi086.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-20T21:30:20.136 INFO:teuthology.orchestra.run.smithi086.stdout: mon pg warn max object skew = 0 2024-04-20T21:30:20.136 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:20.136 INFO:teuthology.orchestra.run.smithi086.stdout: # disable pg_autoscaler by default for new pools 2024-04-20T21:30:20.137 INFO:teuthology.orchestra.run.smithi086.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-20T21:30:20.137 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:20.137 INFO:teuthology.orchestra.run.smithi086.stdout: osd pool default size = 2 2024-04-20T21:30:20.137 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:20.137 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd allow primary affinity = true 2024-04-20T21:30:20.137 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd allow pg remap = true 2024-04-20T21:30:20.137 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on legacy crush tunables = false 2024-04-20T21:30:20.137 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on crush straw calc version zero = false 2024-04-20T21:30:20.137 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on no sortbitwise = false 2024-04-20T21:30:20.137 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on osd down out interval zero = false 2024-04-20T21:30:20.137 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on too few osds = false 2024-04-20T21:30:20.137 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-20T21:30:20.137 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-20T21:30:20.137 INFO:teuthology.orchestra.run.smithi086.stdout: mon_allow_pool_size_one = true 2024-04-20T21:30:20.137 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:20.138 INFO:teuthology.orchestra.run.smithi086.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-20T21:30:20.138 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:20.138 INFO:teuthology.orchestra.run.smithi086.stdout: osd default data pool replay window = 5 2024-04-20T21:30:20.138 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:20.138 INFO:teuthology.orchestra.run.smithi086.stdout: mon allow pool delete = true 2024-04-20T21:30:20.138 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:20.138 INFO:teuthology.orchestra.run.smithi086.stdout: mon cluster log file level = debug 2024-04-20T21:30:20.138 INFO:teuthology.orchestra.run.smithi086.stdout: debug asserts on shutdown = true 2024-04-20T21:30:20.138 INFO:teuthology.orchestra.run.smithi086.stdout: mon health detail to clog = false 2024-04-20T21:30:20.138 INFO:teuthology.orchestra.run.smithi086.stdout: mon host = "172.21.15.86,172.21.15.144,[v2:172.21.15.144:3301,v1:172.21.15.144:6790]" 2024-04-20T21:30:20.138 INFO:teuthology.orchestra.run.smithi086.stdout: fsid = 9c2dec1d-d7ae-4a83-a812-79922808e278 2024-04-20T21:30:20.138 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:20.139 INFO:teuthology.orchestra.run.smithi086.stdout:[osd] 2024-04-20T21:30:20.139 INFO:teuthology.orchestra.run.smithi086.stdout: osd journal size = 100 2024-04-20T21:30:20.139 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:20.139 INFO:teuthology.orchestra.run.smithi086.stdout: osd scrub load threshold = 5.0 2024-04-20T21:30:20.139 INFO:teuthology.orchestra.run.smithi086.stdout: osd scrub max interval = 600 2024-04-20T21:30:20.139 INFO:teuthology.orchestra.run.smithi086.stdout: osd mclock profile = high_recovery_ops 2024-04-20T21:30:20.139 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:20.139 INFO:teuthology.orchestra.run.smithi086.stdout: osd recover clone overlap = true 2024-04-20T21:30:20.139 INFO:teuthology.orchestra.run.smithi086.stdout: osd recovery max chunk = 1048576 2024-04-20T21:30:20.139 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:20.139 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug shutdown = true 2024-04-20T21:30:20.139 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug op order = true 2024-04-20T21:30:20.139 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug verify stray on activate = true 2024-04-20T21:30:20.139 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug trim objects = true 2024-04-20T21:30:20.140 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:20.140 INFO:teuthology.orchestra.run.smithi086.stdout: osd open classes on start = true 2024-04-20T21:30:20.140 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug pg log writeout = true 2024-04-20T21:30:20.140 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:20.140 INFO:teuthology.orchestra.run.smithi086.stdout: osd deep scrub update digest min age = 30 2024-04-20T21:30:20.140 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:20.140 INFO:teuthology.orchestra.run.smithi086.stdout: osd map max advance = 10 2024-04-20T21:30:20.140 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:20.140 INFO:teuthology.orchestra.run.smithi086.stdout: journal zero on create = true 2024-04-20T21:30:20.140 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:20.140 INFO:teuthology.orchestra.run.smithi086.stdout: filestore ondisk finisher threads = 3 2024-04-20T21:30:20.140 INFO:teuthology.orchestra.run.smithi086.stdout: filestore apply finisher threads = 3 2024-04-20T21:30:20.141 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:20.141 INFO:teuthology.orchestra.run.smithi086.stdout: bdev debug aio = true 2024-04-20T21:30:20.141 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug misdirected ops = true 2024-04-20T21:30:20.141 INFO:teuthology.orchestra.run.smithi086.stdout: bdev async discard = True 2024-04-20T21:30:20.141 INFO:teuthology.orchestra.run.smithi086.stdout: bdev enable discard = True 2024-04-20T21:30:20.141 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore allocator = bitmap 2024-04-20T21:30:20.141 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore block size = 96636764160 2024-04-20T21:30:20.141 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore fsck on mount = True 2024-04-20T21:30:20.141 INFO:teuthology.orchestra.run.smithi086.stdout: debug bluefs = 1/20 2024-04-20T21:30:20.141 INFO:teuthology.orchestra.run.smithi086.stdout: debug bluestore = 1/20 2024-04-20T21:30:20.141 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-20T21:30:20.141 INFO:teuthology.orchestra.run.smithi086.stdout: debug osd = 20 2024-04-20T21:30:20.141 INFO:teuthology.orchestra.run.smithi086.stdout: debug rocksdb = 4/10 2024-04-20T21:30:20.141 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd backfillfull_ratio = 0.85 2024-04-20T21:30:20.141 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd full ratio = 0.9 2024-04-20T21:30:20.142 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd nearfull ratio = 0.8 2024-04-20T21:30:20.142 INFO:teuthology.orchestra.run.smithi086.stdout: osd failsafe full ratio = 0.95 2024-04-20T21:30:20.142 INFO:teuthology.orchestra.run.smithi086.stdout: osd objectstore = bluestore 2024-04-20T21:30:20.142 INFO:teuthology.orchestra.run.smithi086.stdout: osd op complaint time = 180 2024-04-20T21:30:20.142 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:20.142 INFO:teuthology.orchestra.run.smithi086.stdout:[mgr] 2024-04-20T21:30:20.142 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-20T21:30:20.142 INFO:teuthology.orchestra.run.smithi086.stdout: debug mgr = 20 2024-04-20T21:30:20.142 INFO:teuthology.orchestra.run.smithi086.stdout: debug mon = 20 2024-04-20T21:30:20.142 INFO:teuthology.orchestra.run.smithi086.stdout: debug auth = 20 2024-04-20T21:30:20.142 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min pgs per osd = 4 2024-04-20T21:30:20.142 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min bytes per osd = 10 2024-04-20T21:30:20.142 INFO:teuthology.orchestra.run.smithi086.stdout: mgr/telemetry/nag = false 2024-04-20T21:30:20.143 INFO:teuthology.orchestra.run.smithi086.stdout: debug client = 20 2024-04-20T21:30:20.143 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:20.143 INFO:teuthology.orchestra.run.smithi086.stdout:[mon] 2024-04-20T21:30:20.143 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-20T21:30:20.143 INFO:teuthology.orchestra.run.smithi086.stdout: debug mon = 20 2024-04-20T21:30:20.143 INFO:teuthology.orchestra.run.smithi086.stdout: debug paxos = 20 2024-04-20T21:30:20.143 INFO:teuthology.orchestra.run.smithi086.stdout: debug auth = 20 2024-04-20T21:30:20.143 INFO:teuthology.orchestra.run.smithi086.stdout: mon data avail warn = 5 2024-04-20T21:30:20.143 INFO:teuthology.orchestra.run.smithi086.stdout: mon mgr mkfs grace = 240 2024-04-20T21:30:20.143 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min pgs per osd = 4 2024-04-20T21:30:20.143 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd reporter subtree level = osd 2024-04-20T21:30:20.143 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd prime pg temp = true 2024-04-20T21:30:20.143 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min bytes per osd = 10 2024-04-20T21:30:20.143 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:20.144 INFO:teuthology.orchestra.run.smithi086.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-20T21:30:20.144 INFO:teuthology.orchestra.run.smithi086.stdout: auth mon ticket ttl = 660 # 11m 2024-04-20T21:30:20.144 INFO:teuthology.orchestra.run.smithi086.stdout: auth service ticket ttl = 240 # 4m 2024-04-20T21:30:20.144 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:20.144 INFO:teuthology.orchestra.run.smithi086.stdout: # don't complain about insecure global_id in the test suite 2024-04-20T21:30:20.144 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-20T21:30:20.144 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-20T21:30:20.144 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:20.144 INFO:teuthology.orchestra.run.smithi086.stdout: # 1m isn't quite enough 2024-04-20T21:30:20.144 INFO:teuthology.orchestra.run.smithi086.stdout: mon_down_mkfs_grace = 2m 2024-04-20T21:30:20.144 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:20.144 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_filestore_osds = false 2024-04-20T21:30:20.144 INFO:teuthology.orchestra.run.smithi086.stdout: mon down mkfs grace = 300 2024-04-20T21:30:20.145 INFO:teuthology.orchestra.run.smithi086.stdout: mon op complaint time = 120 2024-04-20T21:30:20.145 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:20.145 INFO:teuthology.orchestra.run.smithi086.stdout:[client] 2024-04-20T21:30:20.145 INFO:teuthology.orchestra.run.smithi086.stdout: rgw cache enabled = true 2024-04-20T21:30:20.145 INFO:teuthology.orchestra.run.smithi086.stdout: rgw enable ops log = true 2024-04-20T21:30:20.145 INFO:teuthology.orchestra.run.smithi086.stdout: rgw enable usage log = true 2024-04-20T21:30:20.145 INFO:teuthology.orchestra.run.smithi086.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-20T21:30:20.145 INFO:teuthology.orchestra.run.smithi086.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-20T21:30:20.145 INFO:teuthology.orchestra.run.smithi086.stdout: client mount timeout = 600 2024-04-20T21:30:20.145 INFO:teuthology.orchestra.run.smithi086.stdout: debug client = 20 2024-04-20T21:30:20.145 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-20T21:30:20.145 INFO:teuthology.orchestra.run.smithi086.stdout: pid file = "" 2024-04-20T21:30:20.145 INFO:teuthology.orchestra.run.smithi086.stdout: rados mon op timeout = 900 2024-04-20T21:30:20.145 INFO:teuthology.orchestra.run.smithi086.stdout: rados osd op timeout = 900 2024-04-20T21:30:20.146 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.a] 2024-04-20T21:30:20.146 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.b] 2024-04-20T21:30:20.146 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.c] 2024-04-20T21:30:20.146 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.a] 2024-04-20T21:30:20.146 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.c] 2024-04-20T21:30:20.146 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.b] 2024-04-20T21:30:20.146 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.d] 2024-04-20T21:30:20.146 INFO:teuthology.orchestra.run.smithi086.stdout:[mds] 2024-04-20T21:30:20.146 INFO:teuthology.orchestra.run.smithi086.stdout: debug mds = 20 2024-04-20T21:30:20.146 INFO:teuthology.orchestra.run.smithi086.stdout: debug mds balancer = 20 2024-04-20T21:30:20.146 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-20T21:30:20.146 INFO:teuthology.orchestra.run.smithi086.stdout: mds debug frag = True 2024-04-20T21:30:20.146 INFO:teuthology.orchestra.run.smithi086.stdout: mds debug scatterstat = False 2024-04-20T21:30:20.147 INFO:teuthology.orchestra.run.smithi086.stdout: mds op complaint time = 180 2024-04-20T21:30:20.147 INFO:teuthology.orchestra.run.smithi086.stdout: mds verify scatter = False 2024-04-20T21:30:20.147 INFO:teuthology.orchestra.run.smithi086.stdout: osd op complaint time = 180 2024-04-20T21:30:20.147 INFO:teuthology.orchestra.run.smithi086.stdout: rados mon op timeout = 900 2024-04-20T21:30:20.147 INFO:teuthology.orchestra.run.smithi086.stdout: rados osd op timeout = 900 2024-04-20T21:30:20.147 INFO:teuthology.orchestra.run.smithi144.stdout:[global] 2024-04-20T21:30:20.147 INFO:teuthology.orchestra.run.smithi144.stdout: chdir = "" 2024-04-20T21:30:20.147 INFO:teuthology.orchestra.run.smithi144.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-20T21:30:20.148 INFO:teuthology.orchestra.run.smithi144.stdout: auth supported = cephx 2024-04-20T21:30:20.148 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:20.148 INFO:teuthology.orchestra.run.smithi144.stdout: filestore xattr use omap = true 2024-04-20T21:30:20.148 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:20.148 INFO:teuthology.orchestra.run.smithi144.stdout: mon clock drift allowed = 1.000 2024-04-20T21:30:20.148 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:20.148 INFO:teuthology.orchestra.run.smithi144.stdout: osd crush chooseleaf type = 0 2024-04-20T21:30:20.148 INFO:teuthology.orchestra.run.smithi144.stdout: auth debug = true 2024-04-20T21:30:20.148 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:20.148 INFO:teuthology.orchestra.run.smithi144.stdout: ms die on old message = true 2024-04-20T21:30:20.148 INFO:teuthology.orchestra.run.smithi144.stdout: ms die on bug = true 2024-04-20T21:30:20.148 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:20.148 INFO:teuthology.orchestra.run.smithi144.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-20T21:30:20.148 INFO:teuthology.orchestra.run.smithi144.stdout: mon pg warn max object skew = 0 2024-04-20T21:30:20.148 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:20.149 INFO:teuthology.orchestra.run.smithi144.stdout: # disable pg_autoscaler by default for new pools 2024-04-20T21:30:20.149 INFO:teuthology.orchestra.run.smithi144.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-20T21:30:20.149 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:20.149 INFO:teuthology.orchestra.run.smithi144.stdout: osd pool default size = 2 2024-04-20T21:30:20.149 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:20.149 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd allow primary affinity = true 2024-04-20T21:30:20.149 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd allow pg remap = true 2024-04-20T21:30:20.149 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on legacy crush tunables = false 2024-04-20T21:30:20.149 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on crush straw calc version zero = false 2024-04-20T21:30:20.149 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on no sortbitwise = false 2024-04-20T21:30:20.149 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on osd down out interval zero = false 2024-04-20T21:30:20.149 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on too few osds = false 2024-04-20T21:30:20.149 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-20T21:30:20.149 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-20T21:30:20.150 INFO:teuthology.orchestra.run.smithi144.stdout: mon_allow_pool_size_one = true 2024-04-20T21:30:20.150 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:20.150 INFO:teuthology.orchestra.run.smithi144.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-20T21:30:20.150 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:20.150 INFO:teuthology.orchestra.run.smithi144.stdout: osd default data pool replay window = 5 2024-04-20T21:30:20.150 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:20.150 INFO:teuthology.orchestra.run.smithi144.stdout: mon allow pool delete = true 2024-04-20T21:30:20.150 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:20.150 INFO:teuthology.orchestra.run.smithi144.stdout: mon cluster log file level = debug 2024-04-20T21:30:20.150 INFO:teuthology.orchestra.run.smithi144.stdout: debug asserts on shutdown = true 2024-04-20T21:30:20.150 INFO:teuthology.orchestra.run.smithi144.stdout: mon health detail to clog = false 2024-04-20T21:30:20.150 INFO:teuthology.orchestra.run.smithi144.stdout: mon host = "172.21.15.86,172.21.15.144,[v2:172.21.15.144:3301,v1:172.21.15.144:6790]" 2024-04-20T21:30:20.150 INFO:teuthology.orchestra.run.smithi144.stdout: fsid = 9c2dec1d-d7ae-4a83-a812-79922808e278 2024-04-20T21:30:20.151 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:20.151 INFO:teuthology.orchestra.run.smithi144.stdout:[osd] 2024-04-20T21:30:20.151 INFO:teuthology.orchestra.run.smithi144.stdout: osd journal size = 100 2024-04-20T21:30:20.151 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:20.151 INFO:teuthology.orchestra.run.smithi144.stdout: osd scrub load threshold = 5.0 2024-04-20T21:30:20.151 INFO:teuthology.orchestra.run.smithi144.stdout: osd scrub max interval = 600 2024-04-20T21:30:20.151 INFO:teuthology.orchestra.run.smithi144.stdout: osd mclock profile = high_recovery_ops 2024-04-20T21:30:20.151 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:20.151 INFO:teuthology.orchestra.run.smithi144.stdout: osd recover clone overlap = true 2024-04-20T21:30:20.151 INFO:teuthology.orchestra.run.smithi144.stdout: osd recovery max chunk = 1048576 2024-04-20T21:30:20.151 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:20.151 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug shutdown = true 2024-04-20T21:30:20.151 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug op order = true 2024-04-20T21:30:20.151 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug verify stray on activate = true 2024-04-20T21:30:20.151 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug trim objects = true 2024-04-20T21:30:20.152 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:20.152 INFO:teuthology.orchestra.run.smithi144.stdout: osd open classes on start = true 2024-04-20T21:30:20.152 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug pg log writeout = true 2024-04-20T21:30:20.152 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:20.152 INFO:teuthology.orchestra.run.smithi144.stdout: osd deep scrub update digest min age = 30 2024-04-20T21:30:20.152 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:20.152 INFO:teuthology.orchestra.run.smithi144.stdout: osd map max advance = 10 2024-04-20T21:30:20.152 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:20.152 INFO:teuthology.orchestra.run.smithi144.stdout: journal zero on create = true 2024-04-20T21:30:20.152 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:20.152 INFO:teuthology.orchestra.run.smithi144.stdout: filestore ondisk finisher threads = 3 2024-04-20T21:30:20.152 INFO:teuthology.orchestra.run.smithi144.stdout: filestore apply finisher threads = 3 2024-04-20T21:30:20.152 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:20.152 INFO:teuthology.orchestra.run.smithi144.stdout: bdev debug aio = true 2024-04-20T21:30:20.153 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug misdirected ops = true 2024-04-20T21:30:20.153 INFO:teuthology.orchestra.run.smithi144.stdout: bdev async discard = True 2024-04-20T21:30:20.153 INFO:teuthology.orchestra.run.smithi144.stdout: bdev enable discard = True 2024-04-20T21:30:20.153 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore allocator = bitmap 2024-04-20T21:30:20.153 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore block size = 96636764160 2024-04-20T21:30:20.153 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore fsck on mount = True 2024-04-20T21:30:20.153 INFO:teuthology.orchestra.run.smithi144.stdout: debug bluefs = 1/20 2024-04-20T21:30:20.153 INFO:teuthology.orchestra.run.smithi144.stdout: debug bluestore = 1/20 2024-04-20T21:30:20.153 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-04-20T21:30:20.153 INFO:teuthology.orchestra.run.smithi144.stdout: debug osd = 20 2024-04-20T21:30:20.153 INFO:teuthology.orchestra.run.smithi144.stdout: debug rocksdb = 4/10 2024-04-20T21:30:20.153 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd backfillfull_ratio = 0.85 2024-04-20T21:30:20.153 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd full ratio = 0.9 2024-04-20T21:30:20.153 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd nearfull ratio = 0.8 2024-04-20T21:30:20.154 INFO:teuthology.orchestra.run.smithi144.stdout: osd failsafe full ratio = 0.95 2024-04-20T21:30:20.154 INFO:teuthology.orchestra.run.smithi144.stdout: osd objectstore = bluestore 2024-04-20T21:30:20.154 INFO:teuthology.orchestra.run.smithi144.stdout: osd op complaint time = 180 2024-04-20T21:30:20.154 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:20.154 INFO:teuthology.orchestra.run.smithi144.stdout:[mgr] 2024-04-20T21:30:20.154 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-04-20T21:30:20.154 INFO:teuthology.orchestra.run.smithi144.stdout: debug mgr = 20 2024-04-20T21:30:20.154 INFO:teuthology.orchestra.run.smithi144.stdout: debug mon = 20 2024-04-20T21:30:20.154 INFO:teuthology.orchestra.run.smithi144.stdout: debug auth = 20 2024-04-20T21:30:20.154 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min pgs per osd = 4 2024-04-20T21:30:20.154 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min bytes per osd = 10 2024-04-20T21:30:20.154 INFO:teuthology.orchestra.run.smithi144.stdout: mgr/telemetry/nag = false 2024-04-20T21:30:20.154 INFO:teuthology.orchestra.run.smithi144.stdout: debug client = 20 2024-04-20T21:30:20.154 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:20.155 INFO:teuthology.orchestra.run.smithi144.stdout:[mon] 2024-04-20T21:30:20.155 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-04-20T21:30:20.155 INFO:teuthology.orchestra.run.smithi144.stdout: debug mon = 20 2024-04-20T21:30:20.155 INFO:teuthology.orchestra.run.smithi144.stdout: debug paxos = 20 2024-04-20T21:30:20.155 INFO:teuthology.orchestra.run.smithi144.stdout: debug auth = 20 2024-04-20T21:30:20.155 INFO:teuthology.orchestra.run.smithi144.stdout: mon data avail warn = 5 2024-04-20T21:30:20.155 INFO:teuthology.orchestra.run.smithi144.stdout: mon mgr mkfs grace = 240 2024-04-20T21:30:20.155 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min pgs per osd = 4 2024-04-20T21:30:20.155 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd reporter subtree level = osd 2024-04-20T21:30:20.155 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd prime pg temp = true 2024-04-20T21:30:20.155 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min bytes per osd = 10 2024-04-20T21:30:20.155 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:20.155 INFO:teuthology.orchestra.run.smithi144.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-20T21:30:20.155 INFO:teuthology.orchestra.run.smithi144.stdout: auth mon ticket ttl = 660 # 11m 2024-04-20T21:30:20.155 INFO:teuthology.orchestra.run.smithi144.stdout: auth service ticket ttl = 240 # 4m 2024-04-20T21:30:20.156 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:20.156 INFO:teuthology.orchestra.run.smithi144.stdout: # don't complain about insecure global_id in the test suite 2024-04-20T21:30:20.156 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-20T21:30:20.156 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-20T21:30:20.156 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:20.156 INFO:teuthology.orchestra.run.smithi144.stdout: # 1m isn't quite enough 2024-04-20T21:30:20.156 INFO:teuthology.orchestra.run.smithi144.stdout: mon_down_mkfs_grace = 2m 2024-04-20T21:30:20.156 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:20.156 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_filestore_osds = false 2024-04-20T21:30:20.156 INFO:teuthology.orchestra.run.smithi144.stdout: mon down mkfs grace = 300 2024-04-20T21:30:20.156 INFO:teuthology.orchestra.run.smithi144.stdout: mon op complaint time = 120 2024-04-20T21:30:20.156 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:20.156 INFO:teuthology.orchestra.run.smithi144.stdout:[client] 2024-04-20T21:30:20.156 INFO:teuthology.orchestra.run.smithi144.stdout: rgw cache enabled = true 2024-04-20T21:30:20.156 INFO:teuthology.orchestra.run.smithi144.stdout: rgw enable ops log = true 2024-04-20T21:30:20.156 INFO:teuthology.orchestra.run.smithi144.stdout: rgw enable usage log = true 2024-04-20T21:30:20.157 INFO:teuthology.orchestra.run.smithi144.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-20T21:30:20.157 INFO:teuthology.orchestra.run.smithi144.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-20T21:30:20.157 INFO:teuthology.orchestra.run.smithi144.stdout: client mount timeout = 600 2024-04-20T21:30:20.157 INFO:teuthology.orchestra.run.smithi144.stdout: debug client = 20 2024-04-20T21:30:20.157 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-04-20T21:30:20.157 INFO:teuthology.orchestra.run.smithi144.stdout: pid file = "" 2024-04-20T21:30:20.157 INFO:teuthology.orchestra.run.smithi144.stdout: rados mon op timeout = 900 2024-04-20T21:30:20.157 INFO:teuthology.orchestra.run.smithi144.stdout: rados osd op timeout = 900 2024-04-20T21:30:20.157 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.a] 2024-04-20T21:30:20.157 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.b] 2024-04-20T21:30:20.157 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.c] 2024-04-20T21:30:20.157 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.a] 2024-04-20T21:30:20.157 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.c] 2024-04-20T21:30:20.157 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.b] 2024-04-20T21:30:20.157 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.d] 2024-04-20T21:30:20.158 INFO:teuthology.orchestra.run.smithi144.stdout:[mds] 2024-04-20T21:30:20.158 INFO:teuthology.orchestra.run.smithi144.stdout: debug mds = 20 2024-04-20T21:30:20.158 INFO:teuthology.orchestra.run.smithi144.stdout: debug mds balancer = 20 2024-04-20T21:30:20.158 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-04-20T21:30:20.158 INFO:teuthology.orchestra.run.smithi144.stdout: mds debug frag = True 2024-04-20T21:30:20.158 INFO:teuthology.orchestra.run.smithi144.stdout: mds debug scatterstat = False 2024-04-20T21:30:20.158 INFO:teuthology.orchestra.run.smithi144.stdout: mds op complaint time = 180 2024-04-20T21:30:20.158 INFO:teuthology.orchestra.run.smithi144.stdout: mds verify scatter = False 2024-04-20T21:30:20.158 INFO:teuthology.orchestra.run.smithi144.stdout: osd op complaint time = 180 2024-04-20T21:30:20.158 INFO:teuthology.orchestra.run.smithi144.stdout: rados mon op timeout = 900 2024-04-20T21:30:20.158 INFO:teuthology.orchestra.run.smithi144.stdout: rados osd op timeout = 900 2024-04-20T21:30:20.178 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmomapkey 10000000001.00000000 bravo_head 2024-04-20T21:30:20.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.285+0000 7f1a06cb5f40 1 Processor -- start 2024-04-20T21:30:20.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.286+0000 7f1a06cb5f40 1 -- start start 2024-04-20T21:30:20.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.286+0000 7f1a06cb5f40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56512273ecf0 0x56512273f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:20.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.286+0000 7f1a06cb5f40 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x565122740350 0x5651227c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:20.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.286+0000 7f1a06cb5f40 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651227c9220 0x5651227c9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:20.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.286+0000 7f1a06cb5f40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56512275acf0 con 0x56512273ecf0 2024-04-20T21:30:20.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.286+0000 7f1a06cb5f40 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5651226b5cc0 con 0x5651227c9220 2024-04-20T21:30:20.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.286+0000 7f1a06cb5f40 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x56512275af90 con 0x565122740350 2024-04-20T21:30:20.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.287+0000 7f1a03e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56512273ecf0 0x56512273f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:20.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.287+0000 7f1a03e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56512273ecf0 0x56512273f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51780/0 (socket says 172.21.15.86:51780) 2024-04-20T21:30:20.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.287+0000 7f1a03e00640 1 -- 172.21.15.86:0/1717990865 learned_addr learned my addr 172.21.15.86:0/1717990865 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:20.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.287+0000 7f1a03e00640 1 -- 172.21.15.86:0/1717990865 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x565122740350 msgr2=0x5651227c8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:20.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.287+0000 7f1a03400640 1 --2- 172.21.15.86:0/1717990865 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x565122740350 0x5651227c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:20.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.287+0000 7f1a04800640 1 --2- 172.21.15.86:0/1717990865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651227c9220 0x5651227c9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:20.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.287+0000 7f1a03e00640 1 --2- 172.21.15.86:0/1717990865 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x565122740350 0x5651227c8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:20.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.287+0000 7f1a03e00640 1 -- 172.21.15.86:0/1717990865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651227c9220 msgr2=0x5651227c9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:20.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.287+0000 7f1a03e00640 1 --2- 172.21.15.86:0/1717990865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651227c9220 0x5651227c9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:20.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.287+0000 7f1a03e00640 1 -- 172.21.15.86:0/1717990865 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651227ce710 con 0x56512273ecf0 2024-04-20T21:30:20.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.288+0000 7f1a03e00640 1 --2- 172.21.15.86:0/1717990865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56512273ecf0 0x56512273f0d0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f19f4044d50 tx=0x7f19f404d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b1877db3a6a93e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:20.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.288+0000 7f1a02000640 1 -- 172.21.15.86:0/1717990865 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f4001ab0 con 0x56512273ecf0 2024-04-20T21:30:20.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.288+0000 7f1a02000640 1 -- 172.21.15.86:0/1717990865 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19f4006070 con 0x56512273ecf0 2024-04-20T21:30:20.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.288+0000 7f1a02000640 1 -- 172.21.15.86:0/1717990865 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f4003510 con 0x56512273ecf0 2024-04-20T21:30:20.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.289+0000 7f1a06cb5f40 1 -- 172.21.15.86:0/1717990865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56512273ecf0 msgr2=0x56512273f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:20.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.289+0000 7f1a06cb5f40 1 --2- 172.21.15.86:0/1717990865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56512273ecf0 0x56512273f0d0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f19f4044d50 tx=0x7f19f404d000 comp rx=0 tx=0).stop 2024-04-20T21:30:20.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.289+0000 7f1a06cb5f40 1 -- 172.21.15.86:0/1717990865 shutdown_connections 2024-04-20T21:30:20.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.289+0000 7f1a06cb5f40 1 --2- 172.21.15.86:0/1717990865 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5651227c9220 0x5651227c9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:20.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.289+0000 7f1a06cb5f40 1 --2- 172.21.15.86:0/1717990865 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x565122740350 0x5651227c8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:20.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.289+0000 7f1a06cb5f40 1 --2- 172.21.15.86:0/1717990865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56512273ecf0 0x56512273f0d0 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:20.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.289+0000 7f1a06cb5f40 1 -- 172.21.15.86:0/1717990865 >> 172.21.15.86:0/1717990865 conn(0x565122713950 msgr2=0x5651227e4a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:20.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.289+0000 7f1a06cb5f40 1 -- 172.21.15.86:0/1717990865 shutdown_connections 2024-04-20T21:30:20.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.289+0000 7f1a06cb5f40 1 -- 172.21.15.86:0/1717990865 wait complete. 2024-04-20T21:30:20.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.289+0000 7f1a06cb5f40 1 Processor -- start 2024-04-20T21:30:20.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.290+0000 7f1a06cb5f40 1 -- start start 2024-04-20T21:30:20.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.290+0000 7f1a06cb5f40 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56512273ecf0 0x5651227c7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:20.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.291+0000 7f1a06cb5f40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565122740350 0x5651227e4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:20.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.291+0000 7f1a06cb5f40 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5651227c9220 0x5651227ea520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:20.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.291+0000 7f1a06cb5f40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5651227ca130 con 0x565122740350 2024-04-20T21:30:20.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.291+0000 7f1a06cb5f40 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5651227c9e90 con 0x56512273ecf0 2024-04-20T21:30:20.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.291+0000 7f1a06cb5f40 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x56512273f7d0 con 0x5651227c9220 2024-04-20T21:30:20.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.291+0000 7f1a03400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565122740350 0x5651227e4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:20.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.291+0000 7f1a03400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565122740350 0x5651227e4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51794/0 (socket says 172.21.15.86:51794) 2024-04-20T21:30:20.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.291+0000 7f1a03400640 1 -- 172.21.15.86:0/3838007652 learned_addr learned my addr 172.21.15.86:0/3838007652 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:20.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.291+0000 7f1a03e00640 1 --2- 172.21.15.86:0/3838007652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56512273ecf0 0x5651227c7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:20.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.291+0000 7f1a04800640 1 --2- 172.21.15.86:0/3838007652 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5651227c9220 0x5651227ea520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:20.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.291+0000 7f1a03400640 1 -- 172.21.15.86:0/3838007652 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5651227c9220 msgr2=0x5651227ea520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:20.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.291+0000 7f1a03400640 1 --2- 172.21.15.86:0/3838007652 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5651227c9220 0x5651227ea520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:20.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.291+0000 7f1a03400640 1 -- 172.21.15.86:0/3838007652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56512273ecf0 msgr2=0x5651227c7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:20.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.291+0000 7f1a03400640 1 --2- 172.21.15.86:0/3838007652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56512273ecf0 0x5651227c7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:20.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.291+0000 7f1a03400640 1 -- 172.21.15.86:0/3838007652 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565122757d90 con 0x565122740350 2024-04-20T21:30:20.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.292+0000 7f1a03400640 1 --2- 172.21.15.86:0/3838007652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565122740350 0x5651227e4f40 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f19ec00bdb0 tx=0x7f19ec00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:20.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.292+0000 7f1a00c00640 1 -- 172.21.15.86:0/3838007652 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ec019070 con 0x565122740350 2024-04-20T21:30:20.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.292+0000 7f1a00c00640 1 -- 172.21.15.86:0/3838007652 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19ec007a60 con 0x565122740350 2024-04-20T21:30:20.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.292+0000 7f1a00c00640 1 -- 172.21.15.86:0/3838007652 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ec007e50 con 0x565122740350 2024-04-20T21:30:20.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.292+0000 7f1a06cb5f40 1 -- 172.21.15.86:0/3838007652 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x565122758070 con 0x565122740350 2024-04-20T21:30:20.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.292+0000 7f1a06cb5f40 1 -- 172.21.15.86:0/3838007652 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651227c6e10 con 0x565122740350 2024-04-20T21:30:20.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.294+0000 7f1a00c00640 1 -- 172.21.15.86:0/3838007652 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f19ec01c070 con 0x565122740350 2024-04-20T21:30:20.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.295+0000 7f1a00c00640 1 --2- 172.21.15.86:0/3838007652 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f19e006b470 0x7f19e006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:20.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.295+0000 7f1a00c00640 1 -- 172.21.15.86:0/3838007652 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5795+0+0 (secure 0 0 0) 0x7f19ec089a50 con 0x565122740350 2024-04-20T21:30:20.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.295+0000 7f1a03e00640 1 --2- 172.21.15.86:0/3838007652 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f19e006b470 0x7f19e006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:20.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.295+0000 7f1a06cb5f40 1 --2- 172.21.15.86:0/3838007652 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x5651227e82c0 0x56512287f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:20.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.295+0000 7f1a06cb5f40 1 -- 172.21.15.86:0/3838007652 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:1 16.7 16:e2e07930:::10000000001.00000000:head [omap-rm-keys in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56512287f550 con 0x5651227e82c0 2024-04-20T21:30:20.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.296+0000 7f1a04800640 1 --2- 172.21.15.86:0/3838007652 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x5651227e82c0 0x56512287f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:20.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.296+0000 7f1a03e00640 1 --2- 172.21.15.86:0/3838007652 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f19e006b470 0x7f19e006d930 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f19f405d000 tx=0x7f19f4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:20.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.296+0000 7f1a04800640 1 --2- 172.21.15.86:0/3838007652 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x5651227e82c0 0x56512287f010 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:20.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.298+0000 7f1a04800640 1 -- 172.21.15.86:0/3838007652 <== osd.3 v2:172.21.15.86:6808/794135519 1 ==== osd_op_reply(1 10000000001.00000000 [omap-rm-keys] v113'5 uv5 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f19fc0073f0 con 0x5651227e82c0 2024-04-20T21:30:20.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.298+0000 7f1a06cb5f40 1 -- 172.21.15.86:0/3838007652 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x5651227e82c0 msgr2=0x56512287f010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:20.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.298+0000 7f1a06cb5f40 1 --2- 172.21.15.86:0/3838007652 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x5651227e82c0 0x56512287f010 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:20.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.298+0000 7f1a06cb5f40 1 -- 172.21.15.86:0/3838007652 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f19e006b470 msgr2=0x7f19e006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:20.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.298+0000 7f1a06cb5f40 1 --2- 172.21.15.86:0/3838007652 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f19e006b470 0x7f19e006d930 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f19f405d000 tx=0x7f19f4008030 comp rx=0 tx=0).stop 2024-04-20T21:30:20.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.298+0000 7f1a06cb5f40 1 -- 172.21.15.86:0/3838007652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565122740350 msgr2=0x5651227e4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:20.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.298+0000 7f1a06cb5f40 1 --2- 172.21.15.86:0/3838007652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565122740350 0x5651227e4f40 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f19ec00bdb0 tx=0x7f19ec00bea0 comp rx=0 tx=0).stop 2024-04-20T21:30:20.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.298+0000 7f1a04800640 1 -- 172.21.15.86:0/3838007652 reap_dead start 2024-04-20T21:30:20.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.298+0000 7f1a06cb5f40 1 -- 172.21.15.86:0/3838007652 shutdown_connections 2024-04-20T21:30:20.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.298+0000 7f1a06cb5f40 1 -- 172.21.15.86:0/3838007652 >> 172.21.15.86:0/3838007652 conn(0x565122713950 msgr2=0x5651227c4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:20.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.299+0000 7f1a06cb5f40 1 -- 172.21.15.86:0/3838007652 shutdown_connections 2024-04-20T21:30:20.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.299+0000 7f1a06cb5f40 1 -- 172.21.15.86:0/3838007652 wait complete. 2024-04-20T21:30:20.302 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-20T21:30:20.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.490+0000 7fb2e6c00640 1 Processor -- start 2024-04-20T21:30:20.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.491+0000 7fb2e6c00640 1 -- start start 2024-04-20T21:30:20.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.492+0000 7fb2e6c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2e006bbf0 0x7fb2e006e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:20.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.492+0000 7fb2e6c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fb2e006bfd0 0x7fb2e0145b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:20.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.492+0000 7fb2e6c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb2e0146040 0x7fb2e00cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:20.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.492+0000 7fb2e6c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb2e015cd40 con 0x7fb2e0146040 2024-04-20T21:30:20.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.492+0000 7fb2e6c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fb2e015d000 con 0x7fb2e006bfd0 2024-04-20T21:30:20.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.492+0000 7fb2e6c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb2e015d2e0 con 0x7fb2e006bbf0 2024-04-20T21:30:20.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.493+0000 7fb2e4c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb2e0146040 0x7fb2e00cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:20.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.493+0000 7fb2e4c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb2e0146040 0x7fb2e00cd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51804/0 (socket says 172.21.15.86:51804) 2024-04-20T21:30:20.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.493+0000 7fb2e4c00640 1 -- 172.21.15.86:0/238919842 learned_addr learned my addr 172.21.15.86:0/238919842 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:20.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.493+0000 7fb2df400640 1 --2- 172.21.15.86:0/238919842 >> v2:172.21.15.144:3300/0 conn(0x7fb2e006bfd0 0x7fb2e0145b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:20.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.493+0000 7fb2dfe00640 1 --2- 172.21.15.86:0/238919842 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2e006bbf0 0x7fb2e006e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:20.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.493+0000 7fb2e4c00640 1 -- 172.21.15.86:0/238919842 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2e006bbf0 msgr2=0x7fb2e006e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:20.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.493+0000 7fb2e4c00640 1 --2- 172.21.15.86:0/238919842 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2e006bbf0 0x7fb2e006e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:20.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.493+0000 7fb2e4c00640 1 -- 172.21.15.86:0/238919842 >> v2:172.21.15.144:3300/0 conn(0x7fb2e006bfd0 msgr2=0x7fb2e0145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:20.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.493+0000 7fb2e4c00640 1 --2- 172.21.15.86:0/238919842 >> v2:172.21.15.144:3300/0 conn(0x7fb2e006bfd0 0x7fb2e0145b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:20.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.493+0000 7fb2e4c00640 1 -- 172.21.15.86:0/238919842 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2e015d5c0 con 0x7fb2e0146040 2024-04-20T21:30:20.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.495+0000 7fb2e4c00640 1 --2- 172.21.15.86:0/238919842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2e0146040 0x7fb2e00cd540 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7fb2e00b57d0 tx=0x7fb2d802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c68dc1b8ef1f06a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:20.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.495+0000 7fb2dea00640 1 -- 172.21.15.86:0/238919842 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d8035020 con 0x7fb2e0146040 2024-04-20T21:30:20.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.495+0000 7fb2dea00640 1 -- 172.21.15.86:0/238919842 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2d8031870 con 0x7fb2e0146040 2024-04-20T21:30:20.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.495+0000 7fb2dea00640 1 -- 172.21.15.86:0/238919842 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d8031c40 con 0x7fb2e0146040 2024-04-20T21:30:20.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.496+0000 7fb2e6c00640 1 -- 172.21.15.86:0/238919842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2e0146040 msgr2=0x7fb2e00cd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:20.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.496+0000 7fb2e6c00640 1 --2- 172.21.15.86:0/238919842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2e0146040 0x7fb2e00cd540 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7fb2e00b57d0 tx=0x7fb2d802b910 comp rx=0 tx=0).stop 2024-04-20T21:30:20.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.496+0000 7fb2e6c00640 1 -- 172.21.15.86:0/238919842 shutdown_connections 2024-04-20T21:30:20.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.496+0000 7fb2e6c00640 1 --2- 172.21.15.86:0/238919842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2e0146040 0x7fb2e00cd540 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:20.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.496+0000 7fb2e6c00640 1 --2- 172.21.15.86:0/238919842 >> v2:172.21.15.144:3300/0 conn(0x7fb2e006bfd0 0x7fb2e0145b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:20.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.496+0000 7fb2e6c00640 1 --2- 172.21.15.86:0/238919842 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2e006bbf0 0x7fb2e006e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:20.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.496+0000 7fb2e6c00640 1 -- 172.21.15.86:0/238919842 >> 172.21.15.86:0/238919842 conn(0x7fb2e00b2110 msgr2=0x7fb2e00b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:20.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.496+0000 7fb2e6c00640 1 -- 172.21.15.86:0/238919842 shutdown_connections 2024-04-20T21:30:20.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.496+0000 7fb2e6c00640 1 -- 172.21.15.86:0/238919842 wait complete. 2024-04-20T21:30:20.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.498+0000 7fb2e6c00640 1 Processor -- start 2024-04-20T21:30:20.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.498+0000 7fb2e6c00640 1 -- start start 2024-04-20T21:30:20.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.499+0000 7fb2e6c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2e006bbf0 0x7fb2e0152e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:20.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.499+0000 7fb2e6c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2e006bfd0 0x7fb2e00aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:20.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.499+0000 7fb2e6c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2e0146040 0x7fb2e00af320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:20.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.499+0000 7fb2e6c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e00cdc00 con 0x7fb2e0146040 2024-04-20T21:30:20.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.499+0000 7fb2e6c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e00cda80 con 0x7fb2e006bbf0 2024-04-20T21:30:20.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.499+0000 7fb2e6c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb2e00cdd80 con 0x7fb2e006bfd0 2024-04-20T21:30:20.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.499+0000 7fb2e4c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2e0146040 0x7fb2e00af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:20.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.499+0000 7fb2e4c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2e0146040 0x7fb2e00af320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51812/0 (socket says 172.21.15.86:51812) 2024-04-20T21:30:20.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.499+0000 7fb2e4c00640 1 -- 172.21.15.86:0/3039054063 learned_addr learned my addr 172.21.15.86:0/3039054063 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:20.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.499+0000 7fb2e4c00640 1 -- 172.21.15.86:0/3039054063 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2e006bfd0 msgr2=0x7fb2e00aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:20.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.499+0000 7fb2df400640 1 --2- 172.21.15.86:0/3039054063 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2e006bfd0 0x7fb2e00aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:20.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.499+0000 7fb2dfe00640 1 --2- 172.21.15.86:0/3039054063 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2e006bbf0 0x7fb2e0152e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:20.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.499+0000 7fb2e4c00640 1 --2- 172.21.15.86:0/3039054063 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2e006bfd0 0x7fb2e00aede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:20.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.499+0000 7fb2e4c00640 1 -- 172.21.15.86:0/3039054063 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2e006bbf0 msgr2=0x7fb2e0152e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:20.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.499+0000 7fb2e4c00640 1 --2- 172.21.15.86:0/3039054063 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2e006bbf0 0x7fb2e0152e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:20.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.499+0000 7fb2e4c00640 1 -- 172.21.15.86:0/3039054063 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2e01401c0 con 0x7fb2e0146040 2024-04-20T21:30:20.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.500+0000 7fb2e4c00640 1 --2- 172.21.15.86:0/3039054063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2e0146040 0x7fb2e00af320 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7fb2d802b950 tx=0x7fb2d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:20.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.500+0000 7fb2d7e00640 1 -- 172.21.15.86:0/3039054063 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d8040070 con 0x7fb2e0146040 2024-04-20T21:30:20.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.500+0000 7fb2d7e00640 1 -- 172.21.15.86:0/3039054063 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2d8030c50 con 0x7fb2e0146040 2024-04-20T21:30:20.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.500+0000 7fb2e6c00640 1 -- 172.21.15.86:0/3039054063 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2e01404a0 con 0x7fb2e0146040 2024-04-20T21:30:20.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.500+0000 7fb2d7e00640 1 -- 172.21.15.86:0/3039054063 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d8035040 con 0x7fb2e0146040 2024-04-20T21:30:20.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.500+0000 7fb2e6c00640 1 -- 172.21.15.86:0/3039054063 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2e0140a60 con 0x7fb2e0146040 2024-04-20T21:30:20.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.501+0000 7fb2e6c00640 1 -- 172.21.15.86:0/3039054063 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2e00689e0 con 0x7fb2e0146040 2024-04-20T21:30:20.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.502+0000 7fb2d7e00640 1 -- 172.21.15.86:0/3039054063 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb2d8038020 con 0x7fb2e0146040 2024-04-20T21:30:20.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.503+0000 7fb2d7e00640 1 --2- 172.21.15.86:0/3039054063 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb2bc06c8f0 0x7fb2bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:20.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.503+0000 7fb2d7e00640 1 -- 172.21.15.86:0/3039054063 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5795+0+0 (secure 0 0 0) 0x7fb2d80ada30 con 0x7fb2e0146040 2024-04-20T21:30:20.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.503+0000 7fb2dfe00640 1 --2- 172.21.15.86:0/3039054063 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb2bc06c8f0 0x7fb2bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:20.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.504+0000 7fb2dfe00640 1 --2- 172.21.15.86:0/3039054063 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb2bc06c8f0 0x7fb2bc06edb0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0006fd0 tx=0x7fb2d0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:20.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.507+0000 7fb2d7e00640 1 -- 172.21.15.86:0/3039054063 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb2d807bd60 con 0x7fb2e0146040 2024-04-20T21:30:20.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.749+0000 7fb2e6c00640 1 -- 172.21.15.86:0/3039054063 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fb2e006c780 con 0x7fb2e0146040 2024-04-20T21:30:20.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.790+0000 7fb2d7e00640 1 -- 172.21.15.86:0/3039054063 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v101) ==== 162+0+0 (secure 0 0 0) 0x7fb2d807b660 con 0x7fb2e0146040 2024-04-20T21:30:20.789 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-20T21:30:20.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.793+0000 7fb2e6c00640 1 -- 172.21.15.86:0/3039054063 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb2bc06c8f0 msgr2=0x7fb2bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:20.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.793+0000 7fb2e6c00640 1 --2- 172.21.15.86:0/3039054063 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb2bc06c8f0 0x7fb2bc06edb0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0006fd0 tx=0x7fb2d0008040 comp rx=0 tx=0).stop 2024-04-20T21:30:20.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.793+0000 7fb2e6c00640 1 -- 172.21.15.86:0/3039054063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2e0146040 msgr2=0x7fb2e00af320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:20.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.793+0000 7fb2e6c00640 1 --2- 172.21.15.86:0/3039054063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2e0146040 0x7fb2e00af320 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7fb2d802b950 tx=0x7fb2d80046c0 comp rx=0 tx=0).stop 2024-04-20T21:30:20.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.793+0000 7fb2e6c00640 1 -- 172.21.15.86:0/3039054063 shutdown_connections 2024-04-20T21:30:20.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.793+0000 7fb2e6c00640 1 --2- 172.21.15.86:0/3039054063 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb2bc06c8f0 0x7fb2bc06edb0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:20.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.793+0000 7fb2e6c00640 1 --2- 172.21.15.86:0/3039054063 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2e0146040 0x7fb2e00af320 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:20.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.793+0000 7fb2e6c00640 1 --2- 172.21.15.86:0/3039054063 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2e006bfd0 0x7fb2e00aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:20.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.793+0000 7fb2e6c00640 1 --2- 172.21.15.86:0/3039054063 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2e006bbf0 0x7fb2e0152e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:20.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.793+0000 7fb2e6c00640 1 -- 172.21.15.86:0/3039054063 >> 172.21.15.86:0/3039054063 conn(0x7fb2e00b2110 msgr2=0x7fb2e006e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:20.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.793+0000 7fb2e6c00640 1 -- 172.21.15.86:0/3039054063 shutdown_connections 2024-04-20T21:30:20.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.793+0000 7fb2e6c00640 1 -- 172.21.15.86:0/3039054063 wait complete. 2024-04-20T21:30:20.814 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:30:20.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.991+0000 7f7497000640 1 Processor -- start 2024-04-20T21:30:20.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.991+0000 7f7497000640 1 -- start start 2024-04-20T21:30:20.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.993+0000 7f7497000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f749015e950 0x7f74900ce8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:20.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.993+0000 7f7497000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f749006c090 0x7f74900b2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:20.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.993+0000 7f7497000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74900cd840 con 0x7f74900b24d0 2024-04-20T21:30:20.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.993+0000 7f7497000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f74900cd610 con 0x7f749015e950 2024-04-20T21:30:20.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.993+0000 7f7497000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f74900cf350 con 0x7f749006c090 2024-04-20T21:30:20.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.993+0000 7f748fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f749015e950 0x7f74900ce8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:20.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.993+0000 7f748f400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f74900b24d0 0x7f749006b980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54146/0 (socket says 172.21.15.86:54146) 2024-04-20T21:30:20.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.993+0000 7f748f400640 1 -- 172.21.15.86:0/2054022277 learned_addr learned my addr 172.21.15.86:0/2054022277 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:30:20.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.994+0000 7f7495000640 1 --2- 172.21.15.86:0/2054022277 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f749006c090 0x7f74900b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:20.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.994+0000 7f748fe00640 1 -- 172.21.15.86:0/2054022277 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f749006c090 msgr2=0x7f74900b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:20.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.994+0000 7f748fe00640 1 --2- 172.21.15.86:0/2054022277 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f749006c090 0x7f74900b2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:20.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.994+0000 7f748fe00640 1 -- 172.21.15.86:0/2054022277 >> v1:172.21.15.86:6789/0 conn(0x7f74900b24d0 legacy=0x7f749006b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:20.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.994+0000 7f748fe00640 1 -- 172.21.15.86:0/2054022277 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74900cdb20 con 0x7f749015e950 2024-04-20T21:30:20.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.994+0000 7f7495000640 1 --2- 172.21.15.86:0/2054022277 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f749006c090 0x7f74900b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:30:20.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.995+0000 7f748fe00640 1 --2- 172.21.15.86:0/2054022277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f749015e950 0x7f74900ce8e0 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f74800097a0 tx=0x7f748002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19c00c8cb7f023d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:20.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.995+0000 7f748ea00640 1 -- 172.21.15.86:0/2054022277 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7480035020 con 0x7f749015e950 2024-04-20T21:30:20.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.995+0000 7f748ea00640 1 -- 172.21.15.86:0/2054022277 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7480031810 con 0x7f749015e950 2024-04-20T21:30:20.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.995+0000 7f748ea00640 1 -- 172.21.15.86:0/2054022277 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7480031be0 con 0x7f749015e950 2024-04-20T21:30:20.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.996+0000 7f7497000640 1 -- 172.21.15.86:0/2054022277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f749015e950 msgr2=0x7f74900ce8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:20.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.996+0000 7f7497000640 1 --2- 172.21.15.86:0/2054022277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f749015e950 0x7f74900ce8e0 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f74800097a0 tx=0x7f748002b8b0 comp rx=0 tx=0).stop 2024-04-20T21:30:20.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.996+0000 7f7497000640 1 -- 172.21.15.86:0/2054022277 shutdown_connections 2024-04-20T21:30:20.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.996+0000 7f7497000640 1 --2- 172.21.15.86:0/2054022277 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f749006c090 0x7f74900b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:20.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.996+0000 7f7497000640 1 --2- 172.21.15.86:0/2054022277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f749015e950 0x7f74900ce8e0 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:20.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.996+0000 7f7497000640 1 -- 172.21.15.86:0/2054022277 >> 172.21.15.86:0/2054022277 conn(0x7f749009a9e0 msgr2=0x7f74900b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:20.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.996+0000 7f7497000640 1 -- 172.21.15.86:0/2054022277 shutdown_connections 2024-04-20T21:30:20.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.996+0000 7f7497000640 1 -- 172.21.15.86:0/2054022277 wait complete. 2024-04-20T21:30:20.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.998+0000 7f7497000640 1 Processor -- start 2024-04-20T21:30:20.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.998+0000 7f7497000640 1 -- start start 2024-04-20T21:30:20.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.999+0000 7f7497000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f749006c090 0x7f74901525d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:20.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.999+0000 7f7497000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74900b24d0 0x7f7490152bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:20.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.999+0000 7f7497000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f749015e950 0x7f74900c60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:20.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.999+0000 7f7497000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f74900ce3c0 con 0x7f74900b24d0 2024-04-20T21:30:20.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.999+0000 7f7497000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f74900ce0e0 con 0x7f749006c090 2024-04-20T21:30:20.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.999+0000 7f7497000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f749015ed30 con 0x7f749015e950 2024-04-20T21:30:20.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:20.999+0000 7f748f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74900b24d0 0x7f7490152bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:20.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:21.000+0000 7f748f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74900b24d0 0x7f7490152bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51836/0 (socket says 172.21.15.86:51836) 2024-04-20T21:30:20.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:21.000+0000 7f748f400640 1 -- 172.21.15.86:0/2307000789 learned_addr learned my addr 172.21.15.86:0/2307000789 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:20.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:21.000+0000 7f748f400640 1 -- 172.21.15.86:0/2307000789 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f749015e950 msgr2=0x7f74900c60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:20.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:21.000+0000 7f748f400640 1 --2- 172.21.15.86:0/2307000789 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f749015e950 0x7f74900c60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:20.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:21.000+0000 7f748f400640 1 -- 172.21.15.86:0/2307000789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f749006c090 msgr2=0x7f74901525d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:20.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:21.000+0000 7f748f400640 1 --2- 172.21.15.86:0/2307000789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f749006c090 0x7f74901525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:21.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:21.000+0000 7f748f400640 1 -- 172.21.15.86:0/2307000789 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74900c5030 con 0x7f74900b24d0 2024-04-20T21:30:21.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:21.000+0000 7f748f400640 1 --2- 172.21.15.86:0/2307000789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74900b24d0 0x7f7490152bf0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f747800c400 tx=0x7f747800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:21.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:21.000+0000 7f7487e00640 1 -- 172.21.15.86:0/2307000789 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7478017070 con 0x7f74900b24d0 2024-04-20T21:30:21.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:21.000+0000 7f7487e00640 1 -- 172.21.15.86:0/2307000789 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7478005ce0 con 0x7f74900b24d0 2024-04-20T21:30:21.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:21.000+0000 7f7487e00640 1 -- 172.21.15.86:0/2307000789 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74780072f0 con 0x7f74900b24d0 2024-04-20T21:30:21.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:21.001+0000 7f7497000640 1 -- 172.21.15.86:0/2307000789 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74900c5310 con 0x7f74900b24d0 2024-04-20T21:30:21.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:21.001+0000 7f7497000640 1 -- 172.21.15.86:0/2307000789 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74900c58d0 con 0x7f74900b24d0 2024-04-20T21:30:21.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:21.002+0000 7f7497000640 1 -- 172.21.15.86:0/2307000789 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74900689e0 con 0x7f74900b24d0 2024-04-20T21:30:21.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:21.003+0000 7f7487e00640 1 -- 172.21.15.86:0/2307000789 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f747801a070 con 0x7f74900b24d0 2024-04-20T21:30:21.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:21.004+0000 7f7487e00640 1 --2- 172.21.15.86:0/2307000789 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f746c06c8f0 0x7f746c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:21.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:21.004+0000 7f7487e00640 1 -- 172.21.15.86:0/2307000789 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5795+0+0 (secure 0 0 0) 0x7f747808aff0 con 0x7f74900b24d0 2024-04-20T21:30:21.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:21.004+0000 7f748fe00640 1 --2- 172.21.15.86:0/2307000789 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f746c06c8f0 0x7f746c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:21.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:21.005+0000 7f748fe00640 1 --2- 172.21.15.86:0/2307000789 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f746c06c8f0 0x7f746c06edb0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f7480036000 tx=0x7f7480030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:21.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:21.008+0000 7f7487e00640 1 -- 172.21.15.86:0/2307000789 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7478059380 con 0x7f74900b24d0 2024-04-20T21:30:21.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:21.248+0000 7f7497000640 1 -- 172.21.15.86:0/2307000789 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f74900b52d0 con 0x7f74900b24d0 2024-04-20T21:30:21.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:21.249+0000 7f7487e00640 1 -- 172.21.15.86:0/2307000789 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) ==== 95+0+5205 (secure 0 0 0) 0x7f7478058c80 con 0x7f74900b24d0 2024-04-20T21:30:21.248 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:21.248 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":102,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6253,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3627230007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3627230007},{"type":"v1","addr":"172.21.15.144:6833","nonce":3627230007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":6256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2498297133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2498297133},{"type":"v1","addr":"172.21.15.144:6835","nonce":2498297133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":9228,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/1120740475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1120740475},{"type":"v1","addr":"172.21.15.86:6836","nonce":1120740475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:29:43.769581+0000","modified":"2024-04-20T21:30:20.795743+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":113,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9585},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9585":{"gid":9585,"name":"a","rank":0,"incarnation":102,"state":"up:replay","state_seq":1,"addr":"172.21.15.86:6837/1453621918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":1453621918},{"type":"v1","addr":"172.21.15.86:6837","nonce":1453621918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-04-20T21:30:21.248 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 102 2024-04-20T21:30:21.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:21.252+0000 7f7497000640 1 -- 172.21.15.86:0/2307000789 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f746c06c8f0 msgr2=0x7f746c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:21.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:21.252+0000 7f7497000640 1 --2- 172.21.15.86:0/2307000789 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f746c06c8f0 0x7f746c06edb0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f7480036000 tx=0x7f7480030000 comp rx=0 tx=0).stop 2024-04-20T21:30:21.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:21.252+0000 7f7497000640 1 -- 172.21.15.86:0/2307000789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74900b24d0 msgr2=0x7f7490152bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:21.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:21.252+0000 7f7497000640 1 --2- 172.21.15.86:0/2307000789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74900b24d0 0x7f7490152bf0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f747800c400 tx=0x7f747800c8c0 comp rx=0 tx=0).stop 2024-04-20T21:30:21.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:21.252+0000 7f7497000640 1 -- 172.21.15.86:0/2307000789 shutdown_connections 2024-04-20T21:30:21.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:21.252+0000 7f7497000640 1 --2- 172.21.15.86:0/2307000789 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f746c06c8f0 0x7f746c06edb0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:21.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:21.252+0000 7f7497000640 1 --2- 172.21.15.86:0/2307000789 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f749015e950 0x7f74900c60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:21.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:21.252+0000 7f7497000640 1 --2- 172.21.15.86:0/2307000789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74900b24d0 0x7f7490152bf0 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:21.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:21.252+0000 7f7497000640 1 --2- 172.21.15.86:0/2307000789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f749006c090 0x7f74901525d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:21.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:21.252+0000 7f7497000640 1 -- 172.21.15.86:0/2307000789 >> 172.21.15.86:0/2307000789 conn(0x7f749009a9e0 msgr2=0x7f749015da40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:21.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:21.252+0000 7f7497000640 1 -- 172.21.15.86:0/2307000789 shutdown_connections 2024-04-20T21:30:21.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:21.252+0000 7f7497000640 1 -- 172.21.15.86:0/2307000789 wait complete. 2024-04-20T21:30:21.671 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 102, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T21:29:43.769581+0000', 'modified': '2024-04-20T21:30:20.795743+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 113, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9585}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9585': {'gid': 9585, 'name': 'a', 'rank': 0, 'incarnation': 102, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.86:6837/1453621918', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6835', 'nonce': 1453621918}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 1453621918}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-20T21:30:21.671 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9585:up:replay 2024-04-20T21:30:22.671 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:30:22.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.883+0000 7f7e33000640 1 Processor -- start 2024-04-20T21:30:22.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.883+0000 7f7e33000640 1 -- start start 2024-04-20T21:30:22.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.884+0000 7f7e33000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7e2c0b4030 0x7f7e2c0d17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:22.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.884+0000 7f7e33000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7e2c0d1d00 0x7f7e2c0d2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:22.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.884+0000 7f7e33000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e2c15eed0 con 0x7f7e2c15eaf0 2024-04-20T21:30:22.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.884+0000 7f7e33000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f7e2c163110 con 0x7f7e2c0b4030 2024-04-20T21:30:22.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.884+0000 7f7e33000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f7e2c0b7a00 con 0x7f7e2c0d1d00 2024-04-20T21:30:22.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.884+0000 7f7e2b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7e2c0b4030 0x7f7e2c0d17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:22.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.884+0000 7f7e2b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7e2c0b4030 0x7f7e2c0d17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51848/0 (socket says 172.21.15.86:51848) 2024-04-20T21:30:22.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.884+0000 7f7e2b400640 1 -- 172.21.15.86:0/3363964229 learned_addr learned my addr 172.21.15.86:0/3363964229 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:22.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.885+0000 7f7e2b400640 1 -- 172.21.15.86:0/3363964229 >> v2:172.21.15.144:3300/0 conn(0x7f7e2c0d1d00 msgr2=0x7f7e2c0d2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:22.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.885+0000 7f7e31000640 1 --2- 172.21.15.86:0/3363964229 >> v2:172.21.15.144:3300/0 conn(0x7f7e2c0d1d00 0x7f7e2c0d2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:22.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.885+0000 7f7e2b400640 1 --2- 172.21.15.86:0/3363964229 >> v2:172.21.15.144:3300/0 conn(0x7f7e2c0d1d00 0x7f7e2c0d2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:22.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.885+0000 7f7e2b400640 1 -- 172.21.15.86:0/3363964229 >> v1:172.21.15.144:6789/0 conn(0x7f7e2c15eaf0 legacy=0x7f7e2c0ba1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:22.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.885+0000 7f7e2b400640 1 -- 172.21.15.86:0/3363964229 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e2c0d2110 con 0x7f7e2c0b4030 2024-04-20T21:30:22.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.885+0000 7f7e31000640 1 --2- 172.21.15.86:0/3363964229 >> v2:172.21.15.144:3300/0 conn(0x7f7e2c0d1d00 0x7f7e2c0d2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:30:22.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.886+0000 7f7e2b400640 1 --2- 172.21.15.86:0/3363964229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e2c0b4030 0x7f7e2c0d17c0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c06d8b0 tx=0x7f7e1402b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b0e6bfbf3801fd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:22.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.886+0000 7f7e2aa00640 1 -- 172.21.15.86:0/3363964229 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e14034070 con 0x7f7e2c0b4030 2024-04-20T21:30:22.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.886+0000 7f7e2aa00640 1 -- 172.21.15.86:0/3363964229 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e14030850 con 0x7f7e2c0b4030 2024-04-20T21:30:22.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.886+0000 7f7e2aa00640 1 -- 172.21.15.86:0/3363964229 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e14030c20 con 0x7f7e2c0b4030 2024-04-20T21:30:22.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.887+0000 7f7e33000640 1 -- 172.21.15.86:0/3363964229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e2c0b4030 msgr2=0x7f7e2c0d17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:22.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.887+0000 7f7e33000640 1 --2- 172.21.15.86:0/3363964229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e2c0b4030 0x7f7e2c0d17c0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c06d8b0 tx=0x7f7e1402b990 comp rx=0 tx=0).stop 2024-04-20T21:30:22.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.887+0000 7f7e33000640 1 -- 172.21.15.86:0/3363964229 shutdown_connections 2024-04-20T21:30:22.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.887+0000 7f7e33000640 1 --2- 172.21.15.86:0/3363964229 >> v2:172.21.15.144:3300/0 conn(0x7f7e2c0d1d00 0x7f7e2c0d2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:22.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.887+0000 7f7e33000640 1 --2- 172.21.15.86:0/3363964229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e2c0b4030 0x7f7e2c0d17c0 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:22.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.887+0000 7f7e33000640 1 -- 172.21.15.86:0/3363964229 >> 172.21.15.86:0/3363964229 conn(0x7f7e2c0678a0 msgr2=0x7f7e2c067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:22.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.887+0000 7f7e33000640 1 -- 172.21.15.86:0/3363964229 shutdown_connections 2024-04-20T21:30:22.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.887+0000 7f7e33000640 1 -- 172.21.15.86:0/3363964229 wait complete. 2024-04-20T21:30:22.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.889+0000 7f7e33000640 1 Processor -- start 2024-04-20T21:30:22.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.889+0000 7f7e33000640 1 -- start start 2024-04-20T21:30:22.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.890+0000 7f7e33000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e2c0b4030 0x7f7e2c0ce600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:22.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.890+0000 7f7e33000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e2c0d1d00 0x7f7e2c0b6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:22.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.890+0000 7f7e33000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e2c15eaf0 0x7f7e2c158180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:22.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.890+0000 7f7e33000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2c162da0 con 0x7f7e2c15eaf0 2024-04-20T21:30:22.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.890+0000 7f7e33000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2c162ac0 con 0x7f7e2c0b4030 2024-04-20T21:30:22.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.890+0000 7f7e33000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f7e2c0d3270 con 0x7f7e2c0d1d00 2024-04-20T21:30:22.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.890+0000 7f7e31000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e2c15eaf0 0x7f7e2c158180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:22.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.890+0000 7f7e31000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e2c15eaf0 0x7f7e2c158180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51862/0 (socket says 172.21.15.86:51862) 2024-04-20T21:30:22.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.890+0000 7f7e31000640 1 -- 172.21.15.86:0/1657925463 learned_addr learned my addr 172.21.15.86:0/1657925463 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:22.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.890+0000 7f7e2be00640 1 --2- 172.21.15.86:0/1657925463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e2c0b4030 0x7f7e2c0ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:22.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.891+0000 7f7e31000640 1 -- 172.21.15.86:0/1657925463 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e2c0d1d00 msgr2=0x7f7e2c0b6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:22.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.891+0000 7f7e2b400640 1 --2- 172.21.15.86:0/1657925463 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e2c0d1d00 0x7f7e2c0b6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:22.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.891+0000 7f7e31000640 1 --2- 172.21.15.86:0/1657925463 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e2c0d1d00 0x7f7e2c0b6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:22.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.891+0000 7f7e31000640 1 -- 172.21.15.86:0/1657925463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e2c0b4030 msgr2=0x7f7e2c0ce600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:22.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.891+0000 7f7e31000640 1 --2- 172.21.15.86:0/1657925463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e2c0b4030 0x7f7e2c0ce600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:22.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.891+0000 7f7e31000640 1 -- 172.21.15.86:0/1657925463 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e2c0b5c00 con 0x7f7e2c15eaf0 2024-04-20T21:30:22.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.891+0000 7f7e2be00640 1 --2- 172.21.15.86:0/1657925463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e2c0b4030 0x7f7e2c0ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:30:22.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.891+0000 7f7e31000640 1 --2- 172.21.15.86:0/1657925463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e2c15eaf0 0x7f7e2c158180 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f7e2400cef0 tx=0x7f7e2400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:22.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.891+0000 7f7e23e00640 1 -- 172.21.15.86:0/1657925463 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2401a070 con 0x7f7e2c15eaf0 2024-04-20T21:30:22.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.891+0000 7f7e23e00640 1 -- 172.21.15.86:0/1657925463 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e2400d860 con 0x7f7e2c15eaf0 2024-04-20T21:30:22.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.891+0000 7f7e33000640 1 -- 172.21.15.86:0/1657925463 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e2c0b5ee0 con 0x7f7e2c15eaf0 2024-04-20T21:30:22.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.891+0000 7f7e23e00640 1 -- 172.21.15.86:0/1657925463 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2400dc50 con 0x7f7e2c15eaf0 2024-04-20T21:30:22.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.891+0000 7f7e33000640 1 -- 172.21.15.86:0/1657925463 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e2c0b64a0 con 0x7f7e2c15eaf0 2024-04-20T21:30:22.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.892+0000 7f7e33000640 1 -- 172.21.15.86:0/1657925463 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e2c0689e0 con 0x7f7e2c15eaf0 2024-04-20T21:30:22.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.893+0000 7f7e23e00640 1 -- 172.21.15.86:0/1657925463 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7e2401d070 con 0x7f7e2c15eaf0 2024-04-20T21:30:22.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.894+0000 7f7e23e00640 1 --2- 172.21.15.86:0/1657925463 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7e0806c920 0x7f7e0806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:22.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.894+0000 7f7e23e00640 1 -- 172.21.15.86:0/1657925463 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5795+0+0 (secure 0 0 0) 0x7f7e2408c570 con 0x7f7e2c15eaf0 2024-04-20T21:30:22.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.895+0000 7f7e2be00640 1 --2- 172.21.15.86:0/1657925463 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7e0806c920 0x7f7e0806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:22.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.895+0000 7f7e2be00640 1 --2- 172.21.15.86:0/1657925463 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7e0806c920 0x7f7e0806ede0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c00a810 tx=0x7f7e1c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:22.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:22.899+0000 7f7e23e00640 1 -- 172.21.15.86:0/1657925463 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7e2405a8d0 con 0x7f7e2c15eaf0 2024-04-20T21:30:23.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:23.140+0000 7f7e33000640 1 -- 172.21.15.86:0/1657925463 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e2c06dda0 con 0x7f7e2c15eaf0 2024-04-20T21:30:23.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:23.141+0000 7f7e23e00640 1 -- 172.21.15.86:0/1657925463 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) ==== 95+0+5205 (secure 0 0 0) 0x7f7e2405a1d0 con 0x7f7e2c15eaf0 2024-04-20T21:30:23.140 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:23.140 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":104,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6253,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3627230007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3627230007},{"type":"v1","addr":"172.21.15.144:6833","nonce":3627230007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":6256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2498297133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2498297133},{"type":"v1","addr":"172.21.15.144:6835","nonce":2498297133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":9228,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/1120740475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1120740475},{"type":"v1","addr":"172.21.15.86:6836","nonce":1120740475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:29:43.769581+0000","modified":"2024-04-20T21:30:21.813237+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":113,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9585},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9585":{"gid":9585,"name":"a","rank":0,"incarnation":102,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.86:6837/1453621918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":1453621918},{"type":"v1","addr":"172.21.15.86:6837","nonce":1453621918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-04-20T21:30:23.140 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 104 2024-04-20T21:30:23.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:23.143+0000 7f7e33000640 1 -- 172.21.15.86:0/1657925463 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7e0806c920 msgr2=0x7f7e0806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:23.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:23.143+0000 7f7e33000640 1 --2- 172.21.15.86:0/1657925463 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7e0806c920 0x7f7e0806ede0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c00a810 tx=0x7f7e1c008040 comp rx=0 tx=0).stop 2024-04-20T21:30:23.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:23.143+0000 7f7e33000640 1 -- 172.21.15.86:0/1657925463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e2c15eaf0 msgr2=0x7f7e2c158180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:23.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:23.143+0000 7f7e33000640 1 --2- 172.21.15.86:0/1657925463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e2c15eaf0 0x7f7e2c158180 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f7e2400cef0 tx=0x7f7e2400a710 comp rx=0 tx=0).stop 2024-04-20T21:30:23.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:23.143+0000 7f7e33000640 1 -- 172.21.15.86:0/1657925463 shutdown_connections 2024-04-20T21:30:23.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:23.143+0000 7f7e33000640 1 --2- 172.21.15.86:0/1657925463 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f7e0806c920 0x7f7e0806ede0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:23.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:23.143+0000 7f7e33000640 1 --2- 172.21.15.86:0/1657925463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7e2c15eaf0 0x7f7e2c158180 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:23.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:23.143+0000 7f7e33000640 1 --2- 172.21.15.86:0/1657925463 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7e2c0d1d00 0x7f7e2c0b6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:23.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:23.143+0000 7f7e33000640 1 --2- 172.21.15.86:0/1657925463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e2c0b4030 0x7f7e2c0ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:23.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:23.143+0000 7f7e33000640 1 -- 172.21.15.86:0/1657925463 >> 172.21.15.86:0/1657925463 conn(0x7f7e2c0678a0 msgr2=0x7f7e2c15f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:23.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:23.143+0000 7f7e33000640 1 -- 172.21.15.86:0/1657925463 shutdown_connections 2024-04-20T21:30:23.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:23.143+0000 7f7e33000640 1 -- 172.21.15.86:0/1657925463 wait complete. 2024-04-20T21:30:23.562 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 104, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T21:29:43.769581+0000', 'modified': '2024-04-20T21:30:21.813237+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 113, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9585}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9585': {'gid': 9585, 'name': 'a', 'rank': 0, 'incarnation': 102, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.86:6837/1453621918', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6835', 'nonce': 1453621918}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 1453621918}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-20T21:30:23.563 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9585:up:rejoin 2024-04-20T21:30:24.564 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:30:24.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.774+0000 7fb278c00640 1 Processor -- start 2024-04-20T21:30:24.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.774+0000 7fb278c00640 1 -- start start 2024-04-20T21:30:24.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.776+0000 7fb278c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb2740cd170 0x7fb27406ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:24.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.776+0000 7fb278c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2740cd600 0x7fb27406f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:24.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.776+0000 7fb278c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2740cdb50 con 0x7fb274144e90 2024-04-20T21:30:24.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.776+0000 7fb278c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fb274145270 con 0x7fb2740cd170 2024-04-20T21:30:24.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.776+0000 7fb278c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb2740b3e50 con 0x7fb2740cd600 2024-04-20T21:30:24.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.776+0000 7fb271400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb2740cd170 0x7fb27406ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:24.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.776+0000 7fb271400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb2740cd170 0x7fb27406ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51864/0 (socket says 172.21.15.86:51864) 2024-04-20T21:30:24.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.776+0000 7fb271400640 1 -- 172.21.15.86:0/2187264499 learned_addr learned my addr 172.21.15.86:0/2187264499 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:24.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.776+0000 7fb271400640 1 -- 172.21.15.86:0/2187264499 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2740cd600 msgr2=0x7fb27406f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:24.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.776+0000 7fb271400640 1 --2- 172.21.15.86:0/2187264499 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2740cd600 0x7fb27406f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:24.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.776+0000 7fb271400640 1 -- 172.21.15.86:0/2187264499 >> v1:172.21.15.144:6789/0 conn(0x7fb274144e90 legacy=0x7fb2740b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:24.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.776+0000 7fb271400640 1 -- 172.21.15.86:0/2187264499 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2740b4130 con 0x7fb2740cd170 2024-04-20T21:30:24.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.778+0000 7fb271400640 1 --2- 172.21.15.86:0/2187264499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2740cd170 0x7fb27406ed40 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7fb260009870 tx=0x7fb26002da70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=458652a95521adc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:24.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.778+0000 7fb270a00640 1 -- 172.21.15.86:0/2187264499 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb26002f830 con 0x7fb2740cd170 2024-04-20T21:30:24.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.778+0000 7fb270a00640 1 -- 172.21.15.86:0/2187264499 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb26002f9d0 con 0x7fb2740cd170 2024-04-20T21:30:24.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.778+0000 7fb270a00640 1 -- 172.21.15.86:0/2187264499 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb26002fda0 con 0x7fb2740cd170 2024-04-20T21:30:24.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.779+0000 7fb278c00640 1 -- 172.21.15.86:0/2187264499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2740cd170 msgr2=0x7fb27406ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:24.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.779+0000 7fb278c00640 1 --2- 172.21.15.86:0/2187264499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2740cd170 0x7fb27406ed40 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7fb260009870 tx=0x7fb26002da70 comp rx=0 tx=0).stop 2024-04-20T21:30:24.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.779+0000 7fb278c00640 1 -- 172.21.15.86:0/2187264499 shutdown_connections 2024-04-20T21:30:24.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.779+0000 7fb278c00640 1 --2- 172.21.15.86:0/2187264499 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb2740cd600 0x7fb27406f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:24.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.779+0000 7fb278c00640 1 --2- 172.21.15.86:0/2187264499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2740cd170 0x7fb27406ed40 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:24.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.779+0000 7fb278c00640 1 -- 172.21.15.86:0/2187264499 >> 172.21.15.86:0/2187264499 conn(0x7fb2740b2360 msgr2=0x7fb2740b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:24.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.779+0000 7fb278c00640 1 -- 172.21.15.86:0/2187264499 shutdown_connections 2024-04-20T21:30:24.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.779+0000 7fb278c00640 1 -- 172.21.15.86:0/2187264499 wait complete. 2024-04-20T21:30:24.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.781+0000 7fb278c00640 1 Processor -- start 2024-04-20T21:30:24.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.781+0000 7fb278c00640 1 -- start start 2024-04-20T21:30:24.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.782+0000 7fb278c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2740cd170 0x7fb2741f4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:24.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.782+0000 7fb278c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2740cd600 0x7fb27415a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:24.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.782+0000 7fb278c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb274144e90 0x7fb2741e5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:24.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.783+0000 7fb278c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb2740d0190 con 0x7fb2740cd170 2024-04-20T21:30:24.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.783+0000 7fb278c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fb2740d0010 con 0x7fb2740cd600 2024-04-20T21:30:24.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.783+0000 7fb278c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fb27415bb50 con 0x7fb274144e90 2024-04-20T21:30:24.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.783+0000 7fb271e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2740cd170 0x7fb2741f4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:24.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.783+0000 7fb271e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2740cd170 0x7fb2741f4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51866/0 (socket says 172.21.15.86:51866) 2024-04-20T21:30:24.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.783+0000 7fb271e00640 1 -- 172.21.15.86:0/4064266802 learned_addr learned my addr 172.21.15.86:0/4064266802 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:24.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.783+0000 7fb271400640 1 --2- 172.21.15.86:0/4064266802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2740cd600 0x7fb27415a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:24.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.783+0000 7fb271e00640 1 -- 172.21.15.86:0/4064266802 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb274144e90 msgr2=0x7fb2741e5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:24.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.783+0000 7fb272800640 1 --2- 172.21.15.86:0/4064266802 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb274144e90 0x7fb2741e5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:24.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.783+0000 7fb271e00640 1 --2- 172.21.15.86:0/4064266802 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb274144e90 0x7fb2741e5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:24.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.783+0000 7fb271e00640 1 -- 172.21.15.86:0/4064266802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2740cd600 msgr2=0x7fb27415a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:24.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.783+0000 7fb271e00640 1 --2- 172.21.15.86:0/4064266802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2740cd600 0x7fb27415a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:24.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.783+0000 7fb271e00640 1 -- 172.21.15.86:0/4064266802 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2740ab270 con 0x7fb2740cd170 2024-04-20T21:30:24.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.783+0000 7fb271400640 1 --2- 172.21.15.86:0/4064266802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2740cd600 0x7fb27415a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:30:24.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.784+0000 7fb271e00640 1 --2- 172.21.15.86:0/4064266802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2740cd170 0x7fb2741f4a30 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7fb26400b410 tx=0x7fb26400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:24.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.784+0000 7fb26a000640 1 -- 172.21.15.86:0/4064266802 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb264015020 con 0x7fb2740cd170 2024-04-20T21:30:24.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.784+0000 7fb26a000640 1 -- 172.21.15.86:0/4064266802 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb264011830 con 0x7fb2740cd170 2024-04-20T21:30:24.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.784+0000 7fb278c00640 1 -- 172.21.15.86:0/4064266802 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2741f5150 con 0x7fb2740cd170 2024-04-20T21:30:24.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.784+0000 7fb26a000640 1 -- 172.21.15.86:0/4064266802 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb264011c20 con 0x7fb2740cd170 2024-04-20T21:30:24.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.784+0000 7fb278c00640 1 -- 172.21.15.86:0/4064266802 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2741f5710 con 0x7fb2740cd170 2024-04-20T21:30:24.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.785+0000 7fb278c00640 1 -- 172.21.15.86:0/4064266802 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2740689e0 con 0x7fb2740cd170 2024-04-20T21:30:24.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.786+0000 7fb26a000640 1 -- 172.21.15.86:0/4064266802 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb264018020 con 0x7fb2740cd170 2024-04-20T21:30:24.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.787+0000 7fb26a000640 1 --2- 172.21.15.86:0/4064266802 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb25406c8f0 0x7fb25406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:24.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.787+0000 7fb26a000640 1 -- 172.21.15.86:0/4064266802 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5795+0+0 (secure 0 0 0) 0x7fb26408bad0 con 0x7fb2740cd170 2024-04-20T21:30:24.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.788+0000 7fb271400640 1 --2- 172.21.15.86:0/4064266802 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb25406c8f0 0x7fb25406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:24.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.788+0000 7fb271400640 1 --2- 172.21.15.86:0/4064266802 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb25406c8f0 0x7fb25406edb0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fb260038000 tx=0x7fb260008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:24.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:24.792+0000 7fb26a000640 1 -- 172.21.15.86:0/4064266802 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb26401f020 con 0x7fb2740cd170 2024-04-20T21:30:25.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.034+0000 7fb278c00640 1 -- 172.21.15.86:0/4064266802 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb2740ccf30 con 0x7fb2740cd170 2024-04-20T21:30:25.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.035+0000 7fb26a000640 1 -- 172.21.15.86:0/4064266802 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) ==== 95+0+5212 (secure 0 0 0) 0x7fb2640172c0 con 0x7fb2740cd170 2024-04-20T21:30:25.035 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:25.035 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":105,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6253,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3627230007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3627230007},{"type":"v1","addr":"172.21.15.144:6833","nonce":3627230007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":6256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2498297133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2498297133},{"type":"v1","addr":"172.21.15.144:6835","nonce":2498297133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":9228,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/1120740475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1120740475},{"type":"v1","addr":"172.21.15.86:6836","nonce":1120740475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:29:43.769581+0000","modified":"2024-04-20T21:30:23.820570+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":113,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9585},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9585":{"gid":9585,"name":"a","rank":0,"incarnation":102,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6837/1453621918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":1453621918},{"type":"v1","addr":"172.21.15.86:6837","nonce":1453621918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9585,"qdb_cluster":[9585]},"id":8}]} 2024-04-20T21:30:25.036 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 105 2024-04-20T21:30:25.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.038+0000 7fb278c00640 1 -- 172.21.15.86:0/4064266802 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb25406c8f0 msgr2=0x7fb25406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:25.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.038+0000 7fb278c00640 1 --2- 172.21.15.86:0/4064266802 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb25406c8f0 0x7fb25406edb0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fb260038000 tx=0x7fb260008030 comp rx=0 tx=0).stop 2024-04-20T21:30:25.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.038+0000 7fb278c00640 1 -- 172.21.15.86:0/4064266802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2740cd170 msgr2=0x7fb2741f4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:25.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.038+0000 7fb278c00640 1 --2- 172.21.15.86:0/4064266802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2740cd170 0x7fb2741f4a30 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7fb26400b410 tx=0x7fb26400b8d0 comp rx=0 tx=0).stop 2024-04-20T21:30:25.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.038+0000 7fb278c00640 1 -- 172.21.15.86:0/4064266802 shutdown_connections 2024-04-20T21:30:25.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.038+0000 7fb278c00640 1 --2- 172.21.15.86:0/4064266802 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb25406c8f0 0x7fb25406edb0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:25.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.038+0000 7fb278c00640 1 --2- 172.21.15.86:0/4064266802 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fb274144e90 0x7fb2741e5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:25.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.038+0000 7fb278c00640 1 --2- 172.21.15.86:0/4064266802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2740cd600 0x7fb27415a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:25.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.038+0000 7fb278c00640 1 --2- 172.21.15.86:0/4064266802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2740cd170 0x7fb2741f4a30 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:25.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.038+0000 7fb278c00640 1 -- 172.21.15.86:0/4064266802 >> 172.21.15.86:0/4064266802 conn(0x7fb2740b2360 msgr2=0x7fb27406c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:25.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.038+0000 7fb278c00640 1 -- 172.21.15.86:0/4064266802 shutdown_connections 2024-04-20T21:30:25.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.038+0000 7fb278c00640 1 -- 172.21.15.86:0/4064266802 wait complete. 2024-04-20T21:30:25.460 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 105, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T21:29:43.769581+0000', 'modified': '2024-04-20T21:30:23.820570+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 113, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9585}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9585': {'gid': 9585, 'name': 'a', 'rank': 0, 'incarnation': 102, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.86:6837/1453621918', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6835', 'nonce': 1453621918}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 1453621918}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9585, 'qdb_cluster': [9585]} 2024-04-20T21:30:25.460 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-20T21:30:25.460 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-20T21:30:25.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.671+0000 7f4d2c400640 1 Processor -- start 2024-04-20T21:30:25.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.672+0000 7f4d2c400640 1 -- start start 2024-04-20T21:30:25.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.673+0000 7f4d2c400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d2415e740 0x7f4d240d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:25.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.673+0000 7f4d2c400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4d240d0fe0 0x7f4d241623d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:25.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.673+0000 7f4d2c400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d24163250 con 0x7f4d24162910 2024-04-20T21:30:25.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.673+0000 7f4d2c400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f4d240b34b0 con 0x7f4d240d0fe0 2024-04-20T21:30:25.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.673+0000 7f4d2c400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f4d240b3790 con 0x7f4d2415e740 2024-04-20T21:30:25.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.674+0000 7f4d29000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4d240d0fe0 0x7f4d241623d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:25.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.674+0000 7f4d2a400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f4d24162910 0x7f4d24071cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:54154/0 (socket says 172.21.15.86:54154) 2024-04-20T21:30:25.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.674+0000 7f4d2a400640 1 -- 172.21.15.86:0/2644317587 learned_addr learned my addr 172.21.15.86:0/2644317587 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:30:25.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.674+0000 7f4d29a00640 1 --2- 172.21.15.86:0/2644317587 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d2415e740 0x7f4d240d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:25.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.674+0000 7f4d29000640 1 -- 172.21.15.86:0/2644317587 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d2415e740 msgr2=0x7f4d240d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:25.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.674+0000 7f4d29000640 1 --2- 172.21.15.86:0/2644317587 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d2415e740 0x7f4d240d0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:25.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.674+0000 7f4d29000640 1 -- 172.21.15.86:0/2644317587 >> v1:172.21.15.86:6789/0 conn(0x7f4d24162910 legacy=0x7f4d24071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:25.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.674+0000 7f4d29000640 1 -- 172.21.15.86:0/2644317587 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d240b3a70 con 0x7f4d240d0fe0 2024-04-20T21:30:25.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.674+0000 7f4d29a00640 1 --2- 172.21.15.86:0/2644317587 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d2415e740 0x7f4d240d0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:30:25.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.674+0000 7f4d23e00640 1 -- 172.21.15.86:0/2644317587 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1499405983 0 0) 0x7f4d24163250 con 0x7f4d24162910 2024-04-20T21:30:25.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.675+0000 7f4d29000640 1 --2- 172.21.15.86:0/2644317587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d240d0fe0 0x7f4d241623d0 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f4d100097a0 tx=0x7f4d1002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb2893f07d737e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:25.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.676+0000 7f4d23e00640 1 -- 172.21.15.86:0/2644317587 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d10035020 con 0x7f4d240d0fe0 2024-04-20T21:30:25.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.676+0000 7f4d23e00640 1 -- 172.21.15.86:0/2644317587 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d10031810 con 0x7f4d240d0fe0 2024-04-20T21:30:25.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.676+0000 7f4d23e00640 1 -- 172.21.15.86:0/2644317587 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d10031be0 con 0x7f4d240d0fe0 2024-04-20T21:30:25.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.676+0000 7f4d2c400640 1 -- 172.21.15.86:0/2644317587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d240d0fe0 msgr2=0x7f4d241623d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:25.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.676+0000 7f4d2c400640 1 --2- 172.21.15.86:0/2644317587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d240d0fe0 0x7f4d241623d0 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f4d100097a0 tx=0x7f4d1002b8b0 comp rx=0 tx=0).stop 2024-04-20T21:30:25.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.676+0000 7f4d2c400640 1 -- 172.21.15.86:0/2644317587 shutdown_connections 2024-04-20T21:30:25.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.676+0000 7f4d2c400640 1 --2- 172.21.15.86:0/2644317587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d240d0fe0 0x7f4d241623d0 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:25.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.676+0000 7f4d2c400640 1 --2- 172.21.15.86:0/2644317587 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d2415e740 0x7f4d240d0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:25.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.676+0000 7f4d2c400640 1 -- 172.21.15.86:0/2644317587 >> 172.21.15.86:0/2644317587 conn(0x7f4d2406b980 msgr2=0x7f4d2406bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:25.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.677+0000 7f4d2c400640 1 -- 172.21.15.86:0/2644317587 shutdown_connections 2024-04-20T21:30:25.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.677+0000 7f4d2c400640 1 -- 172.21.15.86:0/2644317587 wait complete. 2024-04-20T21:30:25.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.678+0000 7f4d2c400640 1 Processor -- start 2024-04-20T21:30:25.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.678+0000 7f4d2c400640 1 -- start start 2024-04-20T21:30:25.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.679+0000 7f4d2c400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d240d0fe0 0x7f4d2414b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:25.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.679+0000 7f4d2c400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d2415e740 0x7f4d24147d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:25.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.679+0000 7f4d2c400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d24162910 0x7f4d2415abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:25.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.679+0000 7f4d2c400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4d240b4550 con 0x7f4d2415e740 2024-04-20T21:30:25.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.679+0000 7f4d2c400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4d240b4030 con 0x7f4d240d0fe0 2024-04-20T21:30:25.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.679+0000 7f4d2c400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f4d240aeb40 con 0x7f4d24162910 2024-04-20T21:30:25.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.680+0000 7f4d29000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d2415e740 0x7f4d24147d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:25.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.680+0000 7f4d29000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d2415e740 0x7f4d24147d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51890/0 (socket says 172.21.15.86:51890) 2024-04-20T21:30:25.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.680+0000 7f4d29000640 1 -- 172.21.15.86:0/2194493251 learned_addr learned my addr 172.21.15.86:0/2194493251 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:25.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.680+0000 7f4d29a00640 1 --2- 172.21.15.86:0/2194493251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d240d0fe0 0x7f4d2414b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:25.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.680+0000 7f4d2a400640 1 --2- 172.21.15.86:0/2194493251 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d24162910 0x7f4d2415abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:25.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.680+0000 7f4d29000640 1 -- 172.21.15.86:0/2194493251 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d24162910 msgr2=0x7f4d2415abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:25.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.680+0000 7f4d29000640 1 --2- 172.21.15.86:0/2194493251 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d24162910 0x7f4d2415abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:25.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.680+0000 7f4d29000640 1 -- 172.21.15.86:0/2194493251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d240d0fe0 msgr2=0x7f4d2414b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:25.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.680+0000 7f4d29000640 1 --2- 172.21.15.86:0/2194493251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d240d0fe0 0x7f4d2414b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:25.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.680+0000 7f4d29000640 1 -- 172.21.15.86:0/2194493251 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d2415f6b0 con 0x7f4d2415e740 2024-04-20T21:30:25.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.680+0000 7f4d29000640 1 --2- 172.21.15.86:0/2194493251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d2415e740 0x7f4d24147d70 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f4d10009890 tx=0x7f4d100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:25.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.681+0000 7f4d21600640 1 -- 172.21.15.86:0/2194493251 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d10041070 con 0x7f4d2415e740 2024-04-20T21:30:25.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.681+0000 7f4d21600640 1 -- 172.21.15.86:0/2194493251 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d1003f070 con 0x7f4d2415e740 2024-04-20T21:30:25.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.681+0000 7f4d21600640 1 -- 172.21.15.86:0/2194493251 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d10035040 con 0x7f4d2415e740 2024-04-20T21:30:25.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.681+0000 7f4d2c400640 1 -- 172.21.15.86:0/2194493251 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d2415f990 con 0x7f4d2415e740 2024-04-20T21:30:25.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.681+0000 7f4d2c400640 1 -- 172.21.15.86:0/2194493251 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d2415ff50 con 0x7f4d2415e740 2024-04-20T21:30:25.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.684+0000 7f4d21600640 1 -- 172.21.15.86:0/2194493251 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4d10038020 con 0x7f4d2415e740 2024-04-20T21:30:25.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.685+0000 7f4d21600640 1 --2- 172.21.15.86:0/2194493251 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4d0406c8f0 0x7f4d0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:25.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.685+0000 7f4d21600640 1 -- 172.21.15.86:0/2194493251 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5795+0+0 (secure 0 0 0) 0x7f4d100adac0 con 0x7f4d2415e740 2024-04-20T21:30:25.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.685+0000 7f4d29a00640 1 --2- 172.21.15.86:0/2194493251 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4d0406c8f0 0x7f4d0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:25.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.685+0000 7f4d2f19d740 1 Processor -- start 2024-04-20T21:30:25.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.685+0000 7f4d2f19d740 1 -- start start 2024-04-20T21:30:25.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.686+0000 7f4d29a00640 1 --2- 172.21.15.86:0/2194493251 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4d0406c8f0 0x7f4d0406edb0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f4d18008850 tx=0x7f4d18008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:25.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.686+0000 7f4d2f19d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55856f6a5460 0x55856f65ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:25.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.686+0000 7f4d2f19d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55856f65b4e0 0x55856f65cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:25.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.686+0000 7f4d2f19d740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55856f657930 0x55856f687770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:25.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.686+0000 7f4d2f19d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55856f5c39d0 con 0x55856f6a5460 2024-04-20T21:30:25.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.686+0000 7f4d2f19d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55856f56a270 con 0x55856f65b4e0 2024-04-20T21:30:25.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.686+0000 7f4d2f19d740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55856f5bf990 con 0x55856f657930 2024-04-20T21:30:25.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.686+0000 7f4d29a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55856f6a5460 0x55856f65ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:25.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.687+0000 7f4d29a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55856f6a5460 0x55856f65ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51896/0 (socket says 172.21.15.86:51896) 2024-04-20T21:30:25.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.687+0000 7f4d29a00640 1 -- 172.21.15.86:0/1792645735 learned_addr learned my addr 172.21.15.86:0/1792645735 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:25.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.687+0000 7f4d2a400640 1 --2- 172.21.15.86:0/1792645735 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55856f657930 0x55856f687770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:25.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.687+0000 7f4d29000640 1 --2- 172.21.15.86:0/1792645735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55856f65b4e0 0x55856f65cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:25.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.687+0000 7f4d29a00640 1 -- 172.21.15.86:0/1792645735 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55856f657930 msgr2=0x55856f687770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:25.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.687+0000 7f4d29a00640 1 --2- 172.21.15.86:0/1792645735 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55856f657930 0x55856f687770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:25.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.687+0000 7f4d29a00640 1 -- 172.21.15.86:0/1792645735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55856f65b4e0 msgr2=0x55856f65cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:25.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.687+0000 7f4d29a00640 1 --2- 172.21.15.86:0/1792645735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55856f65b4e0 0x55856f65cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:25.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.687+0000 7f4d29a00640 1 -- 172.21.15.86:0/1792645735 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55856f2d62d0 con 0x55856f6a5460 2024-04-20T21:30:25.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.687+0000 7f4d29a00640 1 --2- 172.21.15.86:0/1792645735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55856f6a5460 0x55856f65ca90 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f4d1800dab0 tx=0x7f4d18010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=645208eafa488414 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:25.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.688+0000 7f4d16000640 1 -- 172.21.15.86:0/1792645735 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d18019050 con 0x55856f6a5460 2024-04-20T21:30:25.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.688+0000 7f4d16000640 1 -- 172.21.15.86:0/1792645735 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d18023070 con 0x55856f6a5460 2024-04-20T21:30:25.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.688+0000 7f4d16000640 1 -- 172.21.15.86:0/1792645735 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d18012c10 con 0x55856f6a5460 2024-04-20T21:30:25.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.688+0000 7f4d2f19d740 1 -- 172.21.15.86:0/1792645735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55856f6a5460 msgr2=0x55856f65ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:25.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.688+0000 7f4d2f19d740 1 --2- 172.21.15.86:0/1792645735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55856f6a5460 0x55856f65ca90 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f4d1800dab0 tx=0x7f4d18010000 comp rx=0 tx=0).stop 2024-04-20T21:30:25.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.688+0000 7f4d2f19d740 1 -- 172.21.15.86:0/1792645735 shutdown_connections 2024-04-20T21:30:25.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.688+0000 7f4d2f19d740 1 --2- 172.21.15.86:0/1792645735 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55856f657930 0x55856f687770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:25.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.688+0000 7f4d2f19d740 1 --2- 172.21.15.86:0/1792645735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55856f65b4e0 0x55856f65cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:25.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.688+0000 7f4d2f19d740 1 --2- 172.21.15.86:0/1792645735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55856f6a5460 0x55856f65ca90 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:25.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.688+0000 7f4d2f19d740 1 -- 172.21.15.86:0/1792645735 >> 172.21.15.86:0/1792645735 conn(0x55856f659bf0 msgr2=0x55856f6a2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:25.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.688+0000 7f4d2f19d740 1 -- 172.21.15.86:0/1792645735 shutdown_connections 2024-04-20T21:30:25.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.688+0000 7f4d2f19d740 1 -- 172.21.15.86:0/1792645735 wait complete. 2024-04-20T21:30:25.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.689+0000 7f4d2f19d740 1 -- start start 2024-04-20T21:30:25.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.689+0000 7f4d2f19d740 1 Processor -- start 2024-04-20T21:30:25.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.690+0000 7f4d2f19d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55856f65b4e0 0x55856f6aa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:25.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.690+0000 7f4d2f19d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55856f6a5460 0x55856f67f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:25.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.690+0000 7f4d2f19d740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55856f6a8d50 0x55856f686060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:25.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.690+0000 7f4d2f19d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55856f65b9f0 con 0x55856f65b4e0 2024-04-20T21:30:25.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.690+0000 7f4d2f19d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55856f57d5e0 con 0x55856f6a5460 2024-04-20T21:30:25.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.690+0000 7f4d2f19d740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55856f65bcd0 con 0x55856f6a8d50 2024-04-20T21:30:25.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.690+0000 7f4d2f19d740 10 client.0 _ll_register_callbacks cb 0x55856f2d5ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:30:25.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.691+0000 7f4d29a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55856f65b4e0 0x55856f6aa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:25.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.691+0000 7f4d29a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55856f65b4e0 0x55856f6aa8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51904/0 (socket says 172.21.15.86:51904) 2024-04-20T21:30:25.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.691+0000 7f4d29a00640 1 -- 172.21.15.86:0/4004130612 learned_addr learned my addr 172.21.15.86:0/4004130612 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:25.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.691+0000 7f4d29a00640 1 -- 172.21.15.86:0/4004130612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55856f6a8d50 msgr2=0x55856f686060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:25.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.691+0000 7f4d29a00640 1 --2- 172.21.15.86:0/4004130612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55856f6a8d50 0x55856f686060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:25.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.691+0000 7f4d29a00640 1 -- 172.21.15.86:0/4004130612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55856f6a5460 msgr2=0x55856f67f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:25.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.691+0000 7f4d29a00640 1 --2- 172.21.15.86:0/4004130612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55856f6a5460 0x55856f67f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:25.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.691+0000 7f4d29a00640 1 -- 172.21.15.86:0/4004130612 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55856f3a06b0 con 0x55856f65b4e0 2024-04-20T21:30:25.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.691+0000 7f4d29a00640 1 --2- 172.21.15.86:0/4004130612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55856f65b4e0 0x55856f6aa8b0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f4d1800d590 tx=0x7f4d18004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=413989cecc27cab1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:25.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.691+0000 7f4d0aa00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:30:25.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.692+0000 7f4d0aa00640 1 -- 172.21.15.86:0/4004130612 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1801e700 con 0x55856f65b4e0 2024-04-20T21:30:25.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.692+0000 7f4d0aa00640 1 -- 172.21.15.86:0/4004130612 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 5795+0+0 (secure 0 0 0) 0x7f4d18032b40 con 0x55856f65b4e0 2024-04-20T21:30:25.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.692+0000 7f4d2f19d740 1 -- 172.21.15.86:0/4004130612 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d18023070 con 0x55856f65b4e0 2024-04-20T21:30:25.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.692+0000 7f4d0aa00640 1 -- 172.21.15.86:0/4004130612 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) ==== 24+0+0 (secure 0 0 0) 0x7f4d18032d40 con 0x55856f65b4e0 2024-04-20T21:30:25.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.692+0000 7f4d2f19d740 10 client.9636 fetch_fsmap learned FSMap version 105 2024-04-20T21:30:25.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.692+0000 7f4d2f19d740 1 -- 172.21.15.86:0/4004130612 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) -- 0x55856f6aa390 con 0x55856f65b4e0 2024-04-20T21:30:25.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.692+0000 7f4d0aa00640 1 -- 172.21.15.86:0/4004130612 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d18023070 con 0x55856f65b4e0 2024-04-20T21:30:25.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.692+0000 7f4d0aa00640 1 -- 172.21.15.86:0/4004130612 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 105) ==== 3495+0+0 (secure 0 0 0) 0x7f4d18039020 con 0x55856f65b4e0 2024-04-20T21:30:25.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.692+0000 7f4d2f19d740 10 client.9636 fetch_fsmap finished waiting for FSMap version 105 2024-04-20T21:30:25.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.692+0000 7f4d2f19d740 10 client.9636 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-20T21:30:25.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.692+0000 7f4d2f19d740 20 client.9636 populate_metadata read hostname 'smithi086' 2024-04-20T21:30:25.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.692+0000 7f4d2f19d740 1 --2- 172.21.15.86:0/4004130612 >> [v2:172.21.15.86:6835/1453621918,v1:172.21.15.86:6837/1453621918] conn(0x55856f6b3840 0x55856f6a7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:25.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.692+0000 7f4d2f19d740 4 client.9636 mds_command: new command op to 9585 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:30:25.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.692+0000 7f4d2f19d740 1 -- 172.21.15.86:0/4004130612 --> [v2:172.21.15.86:6835/1453621918,v1:172.21.15.86:6837/1453621918] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55856f5d13f0 con 0x55856f6b3840 2024-04-20T21:30:25.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.693+0000 7f4d29000640 1 --2- 172.21.15.86:0/4004130612 >> [v2:172.21.15.86:6835/1453621918,v1:172.21.15.86:6837/1453621918] conn(0x55856f6b3840 0x55856f6a7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:25.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.693+0000 7f4d29000640 1 --2- 172.21.15.86:0/4004130612 >> [v2:172.21.15.86:6835/1453621918,v1:172.21.15.86:6837/1453621918] conn(0x55856f6b3840 0x55856f6a7750 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3639518f017fb7d server_cookie=9ab71b576bee1809 in_seq=0 out_seq=0 2024-04-20T21:30:25.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.693+0000 7f4d0aa00640 10 client.9636 ms_handle_connect on v2:172.21.15.86:6835/1453621918 2024-04-20T21:30:25.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.694+0000 7f4d0aa00640 1 -- 172.21.15.86:0/4004130612 <== mds.0 v2:172.21.15.86:6835/1453621918 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55856f5d13f0 con 0x55856f6b3840 2024-04-20T21:30:25.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.694+0000 7f4d0aa00640 10 client.9636 handle_command_reply: tid=0 2024-04-20T21:30:25.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.694+0000 7f4d2f19d740 1 client.9636 shutdown 2024-04-20T21:30:25.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.694+0000 7f4d2f19d740 1 -- 172.21.15.86:0/4004130612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55856f65b4e0 msgr2=0x55856f6aa8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:25.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.694+0000 7f4d2f19d740 1 --2- 172.21.15.86:0/4004130612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55856f65b4e0 0x55856f6aa8b0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f4d1800d590 tx=0x7f4d18004730 comp rx=0 tx=0).stop 2024-04-20T21:30:25.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.694+0000 7f4d2f19d740 1 -- 172.21.15.86:0/4004130612 shutdown_connections 2024-04-20T21:30:25.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.694+0000 7f4d2f19d740 1 --2- 172.21.15.86:0/4004130612 >> [v2:172.21.15.86:6835/1453621918,v1:172.21.15.86:6837/1453621918] conn(0x55856f6b3840 0x55856f6a7750 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:25.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.694+0000 7f4d2f19d740 1 --2- 172.21.15.86:0/4004130612 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55856f6a8d50 0x55856f686060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:25.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.694+0000 7f4d2f19d740 1 --2- 172.21.15.86:0/4004130612 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55856f6a5460 0x55856f67f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:25.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.694+0000 7f4d2f19d740 1 --2- 172.21.15.86:0/4004130612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55856f65b4e0 0x55856f6aa8b0 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:25.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.694+0000 7f4d0aa00640 0 client.9636 ms_handle_reset on v2:172.21.15.86:6835/1453621918 2024-04-20T21:30:25.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.694+0000 7f4d2f19d740 1 -- 172.21.15.86:0/4004130612 >> 172.21.15.86:0/4004130612 conn(0x55856f657930 msgr2=0x55856f65d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:25.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.695+0000 7f4d2f19d740 1 -- 172.21.15.86:0/4004130612 shutdown_connections 2024-04-20T21:30:25.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.695+0000 7f4d2f19d740 1 -- 172.21.15.86:0/4004130612 wait complete. 2024-04-20T21:30:25.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.695+0000 7f4d2f19d740 20 client.9636 trim_cache size 0 max 16384 2024-04-20T21:30:25.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.714+0000 7f4d2f19d740 1 Processor -- start 2024-04-20T21:30:25.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.714+0000 7f4d2f19d740 1 -- start start 2024-04-20T21:30:25.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.715+0000 7f4d2f19d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55856f65b4e0 0x55856f6a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:25.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.715+0000 7f4d2f19d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55856f6a5460 0x55856f6a3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:25.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.715+0000 7f4d2f19d740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55856f6a8d50 0x55856f67ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:25.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.715+0000 7f4d2f19d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4d241ffe60 con 0x55856f65b4e0 2024-04-20T21:30:25.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.715+0000 7f4d2f19d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f4d24159960 con 0x55856f6a5460 2024-04-20T21:30:25.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.715+0000 7f4d2f19d740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f4d24159fb0 con 0x55856f6a8d50 2024-04-20T21:30:25.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.715+0000 7f4d29a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55856f65b4e0 0x55856f6a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:25.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.716+0000 7f4d29a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55856f65b4e0 0x55856f6a39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51914/0 (socket says 172.21.15.86:51914) 2024-04-20T21:30:25.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.716+0000 7f4d29a00640 1 -- 172.21.15.86:0/2365604474 learned_addr learned my addr 172.21.15.86:0/2365604474 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:25.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.716+0000 7f4d29a00640 1 -- 172.21.15.86:0/2365604474 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55856f6a8d50 msgr2=0x55856f67ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:25.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.716+0000 7f4d29a00640 1 --2- 172.21.15.86:0/2365604474 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55856f6a8d50 0x55856f67ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:25.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.716+0000 7f4d29a00640 1 -- 172.21.15.86:0/2365604474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55856f6a5460 msgr2=0x55856f6a3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:25.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.716+0000 7f4d29a00640 1 --2- 172.21.15.86:0/2365604474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55856f6a5460 0x55856f6a3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:25.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.716+0000 7f4d29a00640 1 -- 172.21.15.86:0/2365604474 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55856f67fda0 con 0x55856f65b4e0 2024-04-20T21:30:25.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.716+0000 7f4d29a00640 1 --2- 172.21.15.86:0/2365604474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55856f65b4e0 0x55856f6a39e0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f4d18008e30 tx=0x7f4d1802a070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bb8effde64519fb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:25.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.716+0000 7f4d15600640 1 -- 172.21.15.86:0/2365604474 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d18019070 con 0x55856f65b4e0 2024-04-20T21:30:25.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.716+0000 7f4d15600640 1 -- 172.21.15.86:0/2365604474 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d18023040 con 0x55856f65b4e0 2024-04-20T21:30:25.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.716+0000 7f4d15600640 1 -- 172.21.15.86:0/2365604474 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1801e9e0 con 0x55856f65b4e0 2024-04-20T21:30:25.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.717+0000 7f4d2f19d740 1 -- 172.21.15.86:0/2365604474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55856f65b4e0 msgr2=0x55856f6a39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:25.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.717+0000 7f4d2f19d740 1 --2- 172.21.15.86:0/2365604474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55856f65b4e0 0x55856f6a39e0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f4d18008e30 tx=0x7f4d1802a070 comp rx=0 tx=0).stop 2024-04-20T21:30:25.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.717+0000 7f4d2f19d740 1 -- 172.21.15.86:0/2365604474 shutdown_connections 2024-04-20T21:30:25.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.717+0000 7f4d2f19d740 1 --2- 172.21.15.86:0/2365604474 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55856f6a8d50 0x55856f67ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:25.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.717+0000 7f4d2f19d740 1 --2- 172.21.15.86:0/2365604474 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55856f6a5460 0x55856f6a3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:25.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.717+0000 7f4d2f19d740 1 --2- 172.21.15.86:0/2365604474 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55856f65b4e0 0x55856f6a39e0 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:25.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.717+0000 7f4d2f19d740 1 -- 172.21.15.86:0/2365604474 >> 172.21.15.86:0/2365604474 conn(0x55856f657930 msgr2=0x55856f658810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:25.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.717+0000 7f4d2f19d740 1 -- 172.21.15.86:0/2365604474 shutdown_connections 2024-04-20T21:30:25.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.717+0000 7f4d2f19d740 1 -- 172.21.15.86:0/2365604474 wait complete. 2024-04-20T21:30:25.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.718+0000 7f4d2f19d740 1 -- start start 2024-04-20T21:30:25.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.718+0000 7f4d2f19d740 1 Processor -- start 2024-04-20T21:30:25.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.719+0000 7f4d2f19d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55856f65b4e0 0x55856f6a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:25.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.719+0000 7f4d2f19d740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55856f6a5460 0x55856f6a3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:25.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.719+0000 7f4d2f19d740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55856f6a8d50 0x55856f71cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:25.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.719+0000 7f4d2f19d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55856f6b36c0 con 0x55856f65b4e0 2024-04-20T21:30:25.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.719+0000 7f4d2f19d740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55856f67fff0 con 0x55856f6a8d50 2024-04-20T21:30:25.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.719+0000 7f4d2f19d740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55856f6b3d10 con 0x55856f6a5460 2024-04-20T21:30:25.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.719+0000 7f4d2f19d740 10 client.0 _ll_register_callbacks cb 0x7f4d18008690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:30:25.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.719+0000 7f4d29a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55856f65b4e0 0x55856f6a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:25.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.719+0000 7f4d29a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55856f65b4e0 0x55856f6a39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51928/0 (socket says 172.21.15.86:51928) 2024-04-20T21:30:25.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.719+0000 7f4d29a00640 1 -- 172.21.15.86:0/2687729242 learned_addr learned my addr 172.21.15.86:0/2687729242 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:25.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.720+0000 7f4d29a00640 1 -- 172.21.15.86:0/2687729242 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55856f6a5460 msgr2=0x55856f6a3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:25.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.720+0000 7f4d29a00640 1 --2- 172.21.15.86:0/2687729242 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55856f6a5460 0x55856f6a3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:25.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.720+0000 7f4d29a00640 1 -- 172.21.15.86:0/2687729242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55856f6a8d50 msgr2=0x55856f71cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:25.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.720+0000 7f4d29a00640 1 --2- 172.21.15.86:0/2687729242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55856f6a8d50 0x55856f71cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:25.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.720+0000 7f4d29a00640 1 -- 172.21.15.86:0/2687729242 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55856f658e00 con 0x55856f65b4e0 2024-04-20T21:30:25.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.720+0000 7f4d29a00640 1 --2- 172.21.15.86:0/2687729242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55856f65b4e0 0x55856f6a39e0 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f4d18020040 tx=0x7f4d18013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=839434c45fd9df2a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:25.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.720+0000 7f4d0aa00640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:30:25.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.720+0000 7f4d0aa00640 1 -- 172.21.15.86:0/2687729242 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d18049060 con 0x55856f65b4e0 2024-04-20T21:30:25.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.720+0000 7f4d0aa00640 1 -- 172.21.15.86:0/2687729242 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 5795+0+0 (secure 0 0 0) 0x7f4d1801eb80 con 0x55856f65b4e0 2024-04-20T21:30:25.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.720+0000 7f4d2f19d740 1 -- 172.21.15.86:0/2687729242 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d18023040 con 0x55856f65b4e0 2024-04-20T21:30:25.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.720+0000 7f4d0aa00640 1 -- 172.21.15.86:0/2687729242 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) ==== 24+0+0 (secure 0 0 0) 0x7f4d1804f050 con 0x55856f65b4e0 2024-04-20T21:30:25.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.720+0000 7f4d2f19d740 10 client.9642 fetch_fsmap learned FSMap version 105 2024-04-20T21:30:25.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.720+0000 7f4d2f19d740 1 -- 172.21.15.86:0/2687729242 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) -- 0x55856f71d930 con 0x55856f65b4e0 2024-04-20T21:30:25.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.721+0000 7f4d0aa00640 1 -- 172.21.15.86:0/2687729242 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d18023040 con 0x55856f65b4e0 2024-04-20T21:30:25.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.721+0000 7f4d0aa00640 1 -- 172.21.15.86:0/2687729242 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 105) ==== 3495+0+0 (secure 0 0 0) 0x7f4d18052020 con 0x55856f65b4e0 2024-04-20T21:30:25.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.721+0000 7f4d2f19d740 10 client.9642 fetch_fsmap finished waiting for FSMap version 105 2024-04-20T21:30:25.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.721+0000 7f4d2f19d740 10 client.9642 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-20T21:30:25.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.721+0000 7f4d2f19d740 20 client.9642 populate_metadata read hostname 'smithi086' 2024-04-20T21:30:25.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.721+0000 7f4d2f19d740 1 --2- 172.21.15.86:0/2687729242 >> [v2:172.21.15.86:6835/1453621918,v1:172.21.15.86:6837/1453621918] conn(0x55856f6a7e70 0x55856f6bbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:25.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.721+0000 7f4d2f19d740 4 client.9642 mds_command: new command op to 9585 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T21:30:25.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.721+0000 7f4d2f19d740 1 -- 172.21.15.86:0/2687729242 --> [v2:172.21.15.86:6835/1453621918,v1:172.21.15.86:6837/1453621918] -- command(tid 0: {"prefix": "status"}) -- 0x55856f6a52b0 con 0x55856f6a7e70 2024-04-20T21:30:25.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.721+0000 7f4d29000640 1 --2- 172.21.15.86:0/2687729242 >> [v2:172.21.15.86:6835/1453621918,v1:172.21.15.86:6837/1453621918] conn(0x55856f6a7e70 0x55856f6bbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:25.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.722+0000 7f4d29000640 1 --2- 172.21.15.86:0/2687729242 >> [v2:172.21.15.86:6835/1453621918,v1:172.21.15.86:6837/1453621918] conn(0x55856f6a7e70 0x55856f6bbfa0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4352c80ed24e8b58 server_cookie=e8bab5d2d107074 in_seq=0 out_seq=0 2024-04-20T21:30:25.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.722+0000 7f4d0aa00640 10 client.9642 ms_handle_connect on v2:172.21.15.86:6835/1453621918 2024-04-20T21:30:25.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.722+0000 7f4d0aa00640 1 -- 172.21.15.86:0/2687729242 <== mds.0 v2:172.21.15.86:6835/1453621918 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x55856f6a52b0 con 0x55856f6a7e70 2024-04-20T21:30:25.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.722+0000 7f4d0aa00640 10 client.9642 handle_command_reply: tid=0 2024-04-20T21:30:25.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.722+0000 7f4d2f19d740 1 client.9642 shutdown 2024-04-20T21:30:25.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.722+0000 7f4d2f19d740 1 -- 172.21.15.86:0/2687729242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55856f65b4e0 msgr2=0x55856f6a39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:25.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.722+0000 7f4d2f19d740 1 --2- 172.21.15.86:0/2687729242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55856f65b4e0 0x55856f6a39e0 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f4d18020040 tx=0x7f4d18013000 comp rx=0 tx=0).stop 2024-04-20T21:30:25.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.722+0000 7f4d2f19d740 1 -- 172.21.15.86:0/2687729242 shutdown_connections 2024-04-20T21:30:25.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.722+0000 7f4d2f19d740 1 --2- 172.21.15.86:0/2687729242 >> [v2:172.21.15.86:6835/1453621918,v1:172.21.15.86:6837/1453621918] conn(0x55856f6a7e70 0x55856f6bbfa0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:25.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.722+0000 7f4d2f19d740 1 --2- 172.21.15.86:0/2687729242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55856f6a8d50 0x55856f71cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:25.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.722+0000 7f4d2f19d740 1 --2- 172.21.15.86:0/2687729242 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55856f6a5460 0x55856f6a3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:25.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.722+0000 7f4d2f19d740 1 --2- 172.21.15.86:0/2687729242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55856f65b4e0 0x55856f6a39e0 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:25.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.722+0000 7f4d2f19d740 1 -- 172.21.15.86:0/2687729242 >> 172.21.15.86:0/2687729242 conn(0x55856f657930 msgr2=0x55856f65bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:25.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.722+0000 7f4d0aa00640 0 client.9642 ms_handle_reset on v2:172.21.15.86:6835/1453621918 2024-04-20T21:30:25.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.722+0000 7f4d2f19d740 1 -- 172.21.15.86:0/2687729242 shutdown_connections 2024-04-20T21:30:25.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.722+0000 7f4d2f19d740 1 -- 172.21.15.86:0/2687729242 wait complete. 2024-04-20T21:30:25.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.723+0000 7f4d2f19d740 20 client.9642 trim_cache size 0 max 16384 2024-04-20T21:30:25.765 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:30:25.765 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "9c2dec1d-d7ae-4a83-a812-79922808e278", 2024-04-20T21:30:25.765 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-04-20T21:30:25.765 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 9585, 2024-04-20T21:30:25.765 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-20T21:30:25.765 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-20T21:30:25.765 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-20T21:30:25.765 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 4.9167296609999998, 2024-04-20T21:30:25.766 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 105, 2024-04-20T21:30:25.766 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 113, 2024-04-20T21:30:25.766 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 113, 2024-04-20T21:30:25.766 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 5.8413816230000002 2024-04-20T21:30:25.766 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:30:25.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.723+0000 7f4d2c400640 1 -- 172.21.15.86:0/2194493251 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4d0406c8f0 msgr2=0x7f4d0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:25.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.723+0000 7f4d2c400640 1 --2- 172.21.15.86:0/2194493251 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4d0406c8f0 0x7f4d0406edb0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f4d18008850 tx=0x7f4d18008040 comp rx=0 tx=0).stop 2024-04-20T21:30:25.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.723+0000 7f4d2c400640 1 -- 172.21.15.86:0/2194493251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d2415e740 msgr2=0x7f4d24147d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:25.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.723+0000 7f4d2c400640 1 --2- 172.21.15.86:0/2194493251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d2415e740 0x7f4d24147d70 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f4d10009890 tx=0x7f4d100046c0 comp rx=0 tx=0).stop 2024-04-20T21:30:25.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.723+0000 7f4d2c400640 1 -- 172.21.15.86:0/2194493251 shutdown_connections 2024-04-20T21:30:25.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.723+0000 7f4d2c400640 1 --2- 172.21.15.86:0/2194493251 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f4d0406c8f0 0x7f4d0406edb0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:25.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.723+0000 7f4d2c400640 1 --2- 172.21.15.86:0/2194493251 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f4d24162910 0x7f4d2415abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:25.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.723+0000 7f4d2c400640 1 --2- 172.21.15.86:0/2194493251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4d2415e740 0x7f4d24147d70 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:25.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.723+0000 7f4d2c400640 1 --2- 172.21.15.86:0/2194493251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d240d0fe0 0x7f4d2414b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:25.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.723+0000 7f4d2c400640 1 -- 172.21.15.86:0/2194493251 >> 172.21.15.86:0/2194493251 conn(0x7f4d2406b980 msgr2=0x7f4d2415be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:25.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.723+0000 7f4d2c400640 1 -- 172.21.15.86:0/2194493251 shutdown_connections 2024-04-20T21:30:25.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.723+0000 7f4d2c400640 1 -- 172.21.15.86:0/2194493251 wait complete. 2024-04-20T21:30:25.768 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T21:30:25.808 INFO:tasks.ceph.mds.a.smithi086.stderr:2024-04-20T21:30:25.808+0000 7f9fe5e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T21:30:25.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.967+0000 7f6472800640 1 Processor -- start 2024-04-20T21:30:25.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.967+0000 7f6472800640 1 -- start start 2024-04-20T21:30:25.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.969+0000 7f6472800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f646c146130 0x7f646c06e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:25.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.969+0000 7f6472800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f646c06dc30 con 0x7f646c0ccdd0 2024-04-20T21:30:25.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.969+0000 7f6472800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f646c0cd8e0 con 0x7f646c072410 2024-04-20T21:30:25.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.969+0000 7f6472800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f646c0cdbc0 con 0x7f646c146130 2024-04-20T21:30:25.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.969+0000 7f646b400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f646c146130 0x7f646c06e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:25.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.969+0000 7f646aa00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f646c0ccdd0 0x7f646c071d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:32864/0 (socket says 172.21.15.86:32864) 2024-04-20T21:30:25.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.969+0000 7f646aa00640 1 -- 172.21.15.86:0/1999767721 learned_addr learned my addr 172.21.15.86:0/1999767721 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:30:25.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.970+0000 7f646a000640 1 -- 172.21.15.86:0/1999767721 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1575233024 0 0) 0x7f646c06dc30 con 0x7f646c0ccdd0 2024-04-20T21:30:25.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.970+0000 7f646a000640 1 -- 172.21.15.86:0/1999767721 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6450003520 con 0x7f646c0ccdd0 2024-04-20T21:30:25.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.970+0000 7f646b400640 1 -- 172.21.15.86:0/1999767721 >> v1:172.21.15.144:6789/0 conn(0x7f646c072410 legacy=0x7f646c15cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:25.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.970+0000 7f646b400640 1 -- 172.21.15.86:0/1999767721 >> v1:172.21.15.86:6789/0 conn(0x7f646c0ccdd0 legacy=0x7f646c071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:25.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.970+0000 7f646b400640 1 -- 172.21.15.86:0/1999767721 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f646c06ded0 con 0x7f646c146130 2024-04-20T21:30:25.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.971+0000 7f646b400640 1 --2- 172.21.15.86:0/1999767721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f646c146130 0x7f646c06e6d0 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f645c0097a0 tx=0x7f645c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29134210ba6037b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:25.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.972+0000 7f646a000640 1 -- 172.21.15.86:0/1999767721 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645c034070 con 0x7f646c146130 2024-04-20T21:30:25.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.972+0000 7f646a000640 1 -- 172.21.15.86:0/1999767721 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f645c030610 con 0x7f646c146130 2024-04-20T21:30:25.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.972+0000 7f646a000640 1 -- 172.21.15.86:0/1999767721 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645c0309e0 con 0x7f646c146130 2024-04-20T21:30:25.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.972+0000 7f6472800640 1 -- 172.21.15.86:0/1999767721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f646c146130 msgr2=0x7f646c06e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:25.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.972+0000 7f6472800640 1 --2- 172.21.15.86:0/1999767721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f646c146130 0x7f646c06e6d0 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f645c0097a0 tx=0x7f645c02b960 comp rx=0 tx=0).stop 2024-04-20T21:30:25.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.972+0000 7f6472800640 1 -- 172.21.15.86:0/1999767721 shutdown_connections 2024-04-20T21:30:25.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.972+0000 7f6472800640 1 --2- 172.21.15.86:0/1999767721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f646c146130 0x7f646c06e6d0 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:25.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.972+0000 7f6472800640 1 -- 172.21.15.86:0/1999767721 >> 172.21.15.86:0/1999767721 conn(0x7f646c0b2360 msgr2=0x7f646c0b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:25.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.972+0000 7f6472800640 1 -- 172.21.15.86:0/1999767721 shutdown_connections 2024-04-20T21:30:25.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.972+0000 7f6472800640 1 -- 172.21.15.86:0/1999767721 wait complete. 2024-04-20T21:30:25.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.974+0000 7f6472800640 1 Processor -- start 2024-04-20T21:30:25.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.974+0000 7f6472800640 1 -- start start 2024-04-20T21:30:25.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.975+0000 7f6472800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f646c072410 0x7f646c15f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:25.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.975+0000 7f6472800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f646c0ccdd0 0x7f646c13fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:25.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.975+0000 7f6472800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f646c146130 0x7f646c140330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:25.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.975+0000 7f6472800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f646c0cd230 con 0x7f646c146130 2024-04-20T21:30:25.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.975+0000 7f6472800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f646c06f150 con 0x7f646c0ccdd0 2024-04-20T21:30:25.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.975+0000 7f6472800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f646c0cd3b0 con 0x7f646c072410 2024-04-20T21:30:25.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.975+0000 7f646be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f646c146130 0x7f646c140330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:25.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.976+0000 7f646be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f646c146130 0x7f646c140330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53472/0 (socket says 172.21.15.86:53472) 2024-04-20T21:30:25.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.976+0000 7f646be00640 1 -- 172.21.15.86:0/974590186 learned_addr learned my addr 172.21.15.86:0/974590186 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:25.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.976+0000 7f646be00640 1 -- 172.21.15.86:0/974590186 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f646c072410 msgr2=0x7f646c15f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:25.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.976+0000 7f646be00640 1 --2- 172.21.15.86:0/974590186 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f646c072410 0x7f646c15f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:25.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.976+0000 7f646be00640 1 -- 172.21.15.86:0/974590186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f646c0ccdd0 msgr2=0x7f646c13fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:25.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.976+0000 7f646be00640 1 --2- 172.21.15.86:0/974590186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f646c0ccdd0 0x7f646c13fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:25.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.976+0000 7f646be00640 1 -- 172.21.15.86:0/974590186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f646c1e96f0 con 0x7f646c146130 2024-04-20T21:30:25.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.976+0000 7f646be00640 1 --2- 172.21.15.86:0/974590186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f646c146130 0x7f646c140330 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f646400c3e0 tx=0x7f646400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:25.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.976+0000 7f6463400640 1 -- 172.21.15.86:0/974590186 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6464011790 con 0x7f646c146130 2024-04-20T21:30:25.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.977+0000 7f6463400640 1 -- 172.21.15.86:0/974590186 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6464017070 con 0x7f646c146130 2024-04-20T21:30:25.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.977+0000 7f6463400640 1 -- 172.21.15.86:0/974590186 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6464011bb0 con 0x7f646c146130 2024-04-20T21:30:25.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.977+0000 7f6472800640 1 -- 172.21.15.86:0/974590186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f646c1e99d0 con 0x7f646c146130 2024-04-20T21:30:25.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.977+0000 7f6472800640 1 -- 172.21.15.86:0/974590186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f646c0b3be0 con 0x7f646c146130 2024-04-20T21:30:25.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.978+0000 7f6472800640 1 -- 172.21.15.86:0/974590186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f646c0689e0 con 0x7f646c146130 2024-04-20T21:30:25.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.979+0000 7f6463400640 1 -- 172.21.15.86:0/974590186 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f646401c460 con 0x7f646c146130 2024-04-20T21:30:25.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.980+0000 7f6463400640 1 --2- 172.21.15.86:0/974590186 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f644c06c920 0x7f644c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:25.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.980+0000 7f6463400640 1 -- 172.21.15.86:0/974590186 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5795+0+0 (secure 0 0 0) 0x7f646408b950 con 0x7f646c146130 2024-04-20T21:30:25.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.980+0000 7f646b400640 1 --2- 172.21.15.86:0/974590186 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f644c06c920 0x7f644c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:25.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.981+0000 7f646b400640 1 --2- 172.21.15.86:0/974590186 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f644c06c920 0x7f644c06ede0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f645c036000 tx=0x7f645c030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:25.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:25.984+0000 7f6463400640 1 -- 172.21.15.86:0/974590186 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6464059d10 con 0x7f646c146130 2024-04-20T21:30:26.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:26.225+0000 7f6472800640 1 -- 172.21.15.86:0/974590186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f646c15ff50 con 0x7f646c146130 2024-04-20T21:30:26.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:26.226+0000 7f6463400640 1 -- 172.21.15.86:0/974590186 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v105) ==== 53+0+73 (secure 0 0 0) 0x7f6464059610 con 0x7f646c146130 2024-04-20T21:30:26.226 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T21:30:26.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:26.229+0000 7f6472800640 1 -- 172.21.15.86:0/974590186 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f644c06c920 msgr2=0x7f644c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:26.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:26.229+0000 7f6472800640 1 --2- 172.21.15.86:0/974590186 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f644c06c920 0x7f644c06ede0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f645c036000 tx=0x7f645c030000 comp rx=0 tx=0).stop 2024-04-20T21:30:26.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:26.229+0000 7f6472800640 1 -- 172.21.15.86:0/974590186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f646c146130 msgr2=0x7f646c140330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:26.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:26.229+0000 7f6472800640 1 --2- 172.21.15.86:0/974590186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f646c146130 0x7f646c140330 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f646400c3e0 tx=0x7f646400c8a0 comp rx=0 tx=0).stop 2024-04-20T21:30:26.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:26.229+0000 7f6472800640 1 -- 172.21.15.86:0/974590186 shutdown_connections 2024-04-20T21:30:26.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:26.229+0000 7f6472800640 1 --2- 172.21.15.86:0/974590186 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f644c06c920 0x7f644c06ede0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:26.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:26.229+0000 7f6472800640 1 --2- 172.21.15.86:0/974590186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f646c146130 0x7f646c140330 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:26.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:26.229+0000 7f6472800640 1 --2- 172.21.15.86:0/974590186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f646c0ccdd0 0x7f646c13fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:26.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:26.229+0000 7f6472800640 1 --2- 172.21.15.86:0/974590186 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f646c072410 0x7f646c15f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:26.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:26.229+0000 7f6472800640 1 -- 172.21.15.86:0/974590186 >> 172.21.15.86:0/974590186 conn(0x7f646c0b2360 msgr2=0x7f646c145460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:26.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:26.230+0000 7f6472800640 1 -- 172.21.15.86:0/974590186 shutdown_connections 2024-04-20T21:30:26.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:26.230+0000 7f6472800640 1 -- 172.21.15.86:0/974590186 wait complete. 2024-04-20T21:30:26.248 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T21:30:26.248 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T21:30:26.248 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-04-20T21:30:26.248 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-20T21:30:26.248 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T21:30:26.248 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T21:30:26.248 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T21:30:26.248 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T21:30:26.248 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:30:26.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:30:26.249 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-04-20T21:30:26.346 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T21:30:26.347 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T21:30:26.347 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T21:30:26.347 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:30:26.347 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-20T21:30:26.347 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:30:26.347 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:30:26.348 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:30:26.348 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-04-20T21:30:26.348 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:30:26.348 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:30:26.348 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-04-20T21:30:26.349 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:30:26.349 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:30:26.349 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T21:30:26.349 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:30:26.349 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T21:30:26.349 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 41930sec preferred_lft 41930sec 2024-04-20T21:30:26.350 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-04-20T21:30:26.350 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:30:26.350 INFO:teuthology.orchestra.run.smithi086.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T21:30:26.350 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 86:25:97:2c:9c:bb brd ff:ff:ff:ff:ff:ff 2024-04-20T21:30:26.350 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-20T21:30:26.350 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:30:26.351 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::a4fb:30ff:fe90:2d6a/64 scope link 2024-04-20T21:30:26.351 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:30:26.351 INFO:teuthology.orchestra.run.smithi086.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T21:30:26.351 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 86:25:97:2c:9c:bb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:30:26.351 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::8425:97ff:fe2c:9cbb/64 scope link 2024-04-20T21:30:26.351 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:30:26.352 INFO:teuthology.orchestra.run.smithi086.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T21:30:26.352 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c2:9c:8b:55:ca:84 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:30:26.352 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c09c:8bff:fe55:ca84/64 scope link 2024-04-20T21:30:26.352 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:30:26.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:30:26.353 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T21:30:26.434 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-20T21:30:26.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:30:26.439 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-04-20T21:30:26.490 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-04-20T21:30:26.490 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T21:30:26.490 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T21:30:26.490 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T21:30:26.490 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T21:30:26.491 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:30:26.491 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-20T21:30:26.491 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T21:30:26.491 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T21:30:26.491 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T21:30:26.491 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-20T21:30:26.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:30:26.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:30:26.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:30:26.701 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-20T21:30:26.744 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-20T21:30:26.744 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-20T21:30:26.745 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:30:26.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:30:26.745 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-04-20T21:30:26.875 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-20T21:30:26.875 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T21:30:26.875 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-20T21:30:26.927 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:30:26.927 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T21:30:26.927 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-20T21:30:26.976 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-04-20T21:30:27.060 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T21:30:27.113 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-04-20T21:30:27.113 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-04-20T21:30:27.113 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-04-20T21:30:27.114 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-04-20T21:30:27.114 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-04-20T21:30:27.114 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-04-20T21:30:27.114 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-04-20T21:30:27.114 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-04-20T21:30:27.114 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-04-20T21:30:27.114 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-04-20T21:30:27.114 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-04-20T21:30:27.114 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/selinux 2024-04-20T21:30:27.114 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T21:30:27.114 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-04-20T21:30:27.114 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-04-20T21:30:27.114 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-04-20T21:30:27.115 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-04-20T21:30:27.115 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-04-20T21:30:27.115 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-04-20T21:30:27.115 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T21:30:27.115 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T21:30:27.115 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T21:30:27.115 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T21:30:27.115 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-04-20T21:30:27.115 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-04-20T21:30:27.115 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T21:30:27.115 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T21:30:27.115 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T21:30:27.115 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T21:30:27.115 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-04-20T21:30:27.115 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:30:27.116 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:30:27.116 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:30:27.116 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:30:27.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:30:27.117 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-04-20T21:30:27.163 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T21:30:27.164 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-20T21:30:27.211 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-04-20T21:30:27.283 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T21:30:27.325 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-04-20T21:30:27.325 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-04-20T21:30:27.325 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-04-20T21:30:27.325 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-04-20T21:30:27.325 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-04-20T21:30:27.326 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-04-20T21:30:27.326 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-04-20T21:30:27.326 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-04-20T21:30:27.326 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-04-20T21:30:27.326 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-04-20T21:30:27.326 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-04-20T21:30:27.326 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/selinux 2024-04-20T21:30:27.326 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T21:30:27.326 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-04-20T21:30:27.326 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-04-20T21:30:27.326 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-04-20T21:30:27.326 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-04-20T21:30:27.327 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-04-20T21:30:27.327 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-04-20T21:30:27.327 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T21:30:27.327 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T21:30:27.327 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T21:30:27.327 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T21:30:27.327 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-04-20T21:30:27.327 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-04-20T21:30:27.327 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T21:30:27.327 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T21:30:27.327 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T21:30:27.327 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T21:30:27.327 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-04-20T21:30:27.328 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:30:27.328 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:30:27.328 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:30:27.328 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:30:27.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:30:27.329 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-04-20T21:30:27.388 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:2024-04-20T21:30:27.388+0000 7f738c387480 -1 init, newargv = 0x561151663e10 newargc=15 2024-04-20T21:30:27.388 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[67298]: starting ceph client 2024-04-20T21:30:27.397 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[67298]: starting fuse 2024-04-20T21:30:27.414 INFO:teuthology.orchestra.run.smithi086.stdout:41 2024-04-20T21:30:27.415 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-04-20T21:30:27.415 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer -- python3 -c ' 2024-04-20T21:30:27.415 DEBUG:teuthology.orchestra.run.smithi086:> import glob 2024-04-20T21:30:27.415 DEBUG:teuthology.orchestra.run.smithi086:> import re 2024-04-20T21:30:27.415 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:30:27.415 DEBUG:teuthology.orchestra.run.smithi086:> import subprocess 2024-04-20T21:30:27.415 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:30:27.415 DEBUG:teuthology.orchestra.run.smithi086:> def _find_admin_socket(client_name): 2024-04-20T21:30:27.415 DEBUG:teuthology.orchestra.run.smithi086:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-20T21:30:27.415 DEBUG:teuthology.orchestra.run.smithi086:> files = glob.glob(asok_path) 2024-04-20T21:30:27.415 DEBUG:teuthology.orchestra.run.smithi086:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-20T21:30:27.416 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:30:27.416 DEBUG:teuthology.orchestra.run.smithi086:> # Given a non-glob path, it better be there 2024-04-20T21:30:27.416 DEBUG:teuthology.orchestra.run.smithi086:> if "*" not in asok_path: 2024-04-20T21:30:27.416 DEBUG:teuthology.orchestra.run.smithi086:> assert(len(files) == 1) 2024-04-20T21:30:27.416 DEBUG:teuthology.orchestra.run.smithi086:> return files[0] 2024-04-20T21:30:27.416 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:30:27.416 DEBUG:teuthology.orchestra.run.smithi086:> for f in files: 2024-04-20T21:30:27.416 DEBUG:teuthology.orchestra.run.smithi086:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T21:30:27.416 DEBUG:teuthology.orchestra.run.smithi086:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T21:30:27.416 DEBUG:teuthology.orchestra.run.smithi086:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T21:30:27.416 DEBUG:teuthology.orchestra.run.smithi086:> contents = proc_f.read() 2024-04-20T21:30:27.416 DEBUG:teuthology.orchestra.run.smithi086:> if mountpoint in contents: 2024-04-20T21:30:27.416 DEBUG:teuthology.orchestra.run.smithi086:> return f 2024-04-20T21:30:27.416 DEBUG:teuthology.orchestra.run.smithi086:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T21:30:27.416 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:30:27.416 DEBUG:teuthology.orchestra.run.smithi086:> print(_find_admin_socket("client.0")) 2024-04-20T21:30:27.417 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:30:27.603 INFO:teuthology.orchestra.run.smithi086.stdout:/var/run/ceph/ceph-client.0.67298.asok 2024-04-20T21:30:27.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:30:27.620 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.67298.asok 2024-04-20T21:30:27.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:30:27.620 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.67298.asok status 2024-04-20T21:30:27.848 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:30:27.848 INFO:teuthology.orchestra.run.smithi086.stdout: "metadata": { 2024-04-20T21:30:27.848 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T21:30:27.848 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T21:30:27.848 INFO:teuthology.orchestra.run.smithi086.stdout: "entity_id": "0", 2024-04-20T21:30:27.848 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": "smithi086", 2024-04-20T21:30:27.848 INFO:teuthology.orchestra.run.smithi086.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-20T21:30:27.849 INFO:teuthology.orchestra.run.smithi086.stdout: "pid": "67298", 2024-04-20T21:30:27.849 INFO:teuthology.orchestra.run.smithi086.stdout: "root": "/" 2024-04-20T21:30:27.849 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:30:27.849 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_count": 0, 2024-04-20T21:30:27.849 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_pinned_count": 0, 2024-04-20T21:30:27.849 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 9657, 2024-04-20T21:30:27.849 INFO:teuthology.orchestra.run.smithi086.stdout: "inst": { 2024-04-20T21:30:27.849 INFO:teuthology.orchestra.run.smithi086.stdout: "name": { 2024-04-20T21:30:27.849 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "client", 2024-04-20T21:30:27.849 INFO:teuthology.orchestra.run.smithi086.stdout: "num": 9657 2024-04-20T21:30:27.849 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:30:27.849 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2024-04-20T21:30:27.849 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-04-20T21:30:27.849 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2024-04-20T21:30:27.849 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 2778092853 2024-04-20T21:30:27.850 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:30:27.850 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:30:27.850 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2024-04-20T21:30:27.850 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-04-20T21:30:27.850 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2024-04-20T21:30:27.850 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 2778092853 2024-04-20T21:30:27.850 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:30:27.850 INFO:teuthology.orchestra.run.smithi086.stdout: "inst_str": "client.9657 192.168.0.1:0/2778092853", 2024-04-20T21:30:27.850 INFO:teuthology.orchestra.run.smithi086.stdout: "addr_str": "192.168.0.1:0/2778092853", 2024-04-20T21:30:27.850 INFO:teuthology.orchestra.run.smithi086.stdout: "inode_count": 1, 2024-04-20T21:30:27.850 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_epoch": 105, 2024-04-20T21:30:27.850 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch": 113, 2024-04-20T21:30:27.850 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch_barrier": 0, 2024-04-20T21:30:27.851 INFO:teuthology.orchestra.run.smithi086.stdout: "blocklisted": false, 2024-04-20T21:30:27.851 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs" 2024-04-20T21:30:27.851 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:30:27.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:30:27.863 DEBUG:teuthology.orchestra.run.smithi086:> stat --file-system '--printf=%T 2024-04-20T21:30:27.863 DEBUG:teuthology.orchestra.run.smithi086:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-20T21:30:27.913 INFO:teuthology.orchestra.run.smithi086.stdout:fuseblk 2024-04-20T21:30:27.913 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-20T21:30:27.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:30:27.913 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-20T21:30:27.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:30:27.990 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-20T21:30:28.058 INFO:teuthology.orchestra.run.smithi086.stdout:./ 2024-04-20T21:30:28.058 INFO:teuthology.orchestra.run.smithi086.stdout:./parent 2024-04-20T21:30:28.058 INFO:teuthology.orchestra.run.smithi086.stdout:./parent/flushed 2024-04-20T21:30:28.058 INFO:teuthology.orchestra.run.smithi086.stdout:./parent/flushed/charlie 2024-04-20T21:30:28.058 INFO:teuthology.orchestra.run.smithi086.stdout:./parent/flushed/alpha 2024-04-20T21:30:28.058 INFO:teuthology.orchestra.run.smithi086.stdout:./parent/unflushed 2024-04-20T21:30:28.058 INFO:teuthology.orchestra.run.smithi086.stdout:./parent/unflushed/jfile 2024-04-20T21:30:28.059 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-04-20T21:30:28.059 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:30:28.060 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-04-20T21:30:28.060 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:30:28.060 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-20T21:30:28.060 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:30:28.177 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-20T21:30:28.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:30:28.190 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-04-20T21:30:28.191 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:30:28.191 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-04-20T21:30:28.191 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:30:28.191 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-04-20T21:30:28.191 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:30:28.371 INFO:teuthology.orchestra.run.smithi086.stdout:1099511627776 2024-04-20T21:30:28.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:30:28.385 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-04-20T21:30:28.385 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:30:28.385 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-04-20T21:30:28.385 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:30:28.385 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-04-20T21:30:28.385 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:30:28.592 INFO:teuthology.orchestra.run.smithi086.stdout:1099511627777 2024-04-20T21:30:28.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:30:28.605 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-04-20T21:30:28.605 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:30:28.605 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-04-20T21:30:28.605 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:30:28.606 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-04-20T21:30:28.606 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:30:28.791 INFO:teuthology.orchestra.run.smithi086.stdout:1099511628280 2024-04-20T21:30:28.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:30:28.805 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-04-20T21:30:28.805 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:30:28.805 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-04-20T21:30:28.805 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:30:28.805 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-04-20T21:30:28.805 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:30:28.993 INFO:teuthology.orchestra.run.smithi086.stdout:1099511627778 2024-04-20T21:30:28.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:30:29.007 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-04-20T21:30:29.007 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:30:29.008 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-04-20T21:30:29.008 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:30:29.008 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-04-20T21:30:29.008 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:30:29.189 INFO:teuthology.orchestra.run.smithi086.stdout:1099511627779 2024-04-20T21:30:29.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:30:29.203 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-04-20T21:30:29.203 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:30:29.203 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-04-20T21:30:29.203 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:30:29.204 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-04-20T21:30:29.204 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:30:29.379 INFO:teuthology.orchestra.run.smithi086.stdout:1099511627780 2024-04-20T21:30:29.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:30:29.392 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:30:29.392 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:30:29.490 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:30:29.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.692+0000 7fc383800640 1 Processor -- start 2024-04-20T21:30:29.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.692+0000 7fc383800640 1 -- start start 2024-04-20T21:30:29.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.694+0000 7fc383800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc37c06e6d0 0x7fc37c071d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:29.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.694+0000 7fc383800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc37c072240 0x7fc37c15cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:29.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.694+0000 7fc383800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc37c06eb30 con 0x7fc37c146130 2024-04-20T21:30:29.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.694+0000 7fc383800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fc37c072690 con 0x7fc37c06e6d0 2024-04-20T21:30:29.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.694+0000 7fc383800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fc37c06dc30 con 0x7fc37c072240 2024-04-20T21:30:29.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.694+0000 7fc380e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fc37c146130 0x7fc37c0ccdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:32874/0 (socket says 172.21.15.86:32874) 2024-04-20T21:30:29.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.694+0000 7fc380e00640 1 -- 172.21.15.86:0/1345620838 learned_addr learned my addr 172.21.15.86:0/1345620838 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:30:29.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.694+0000 7fc37be00640 1 --2- 172.21.15.86:0/1345620838 >> v2:172.21.15.86:3300/0 conn(0x7fc37c06e6d0 0x7fc37c071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:29.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.694+0000 7fc37be00640 1 -- 172.21.15.86:0/1345620838 >> v2:172.21.15.144:3300/0 conn(0x7fc37c072240 msgr2=0x7fc37c15cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:29.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.694+0000 7fc381800640 1 --2- 172.21.15.86:0/1345620838 >> v2:172.21.15.144:3300/0 conn(0x7fc37c072240 0x7fc37c15cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:29.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.695+0000 7fc37be00640 1 --2- 172.21.15.86:0/1345620838 >> v2:172.21.15.144:3300/0 conn(0x7fc37c072240 0x7fc37c15cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:29.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.695+0000 7fc37be00640 1 -- 172.21.15.86:0/1345620838 >> v1:172.21.15.86:6789/0 conn(0x7fc37c146130 legacy=0x7fc37c0ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:29.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.695+0000 7fc37b400640 1 -- 172.21.15.86:0/1345620838 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1653175580 0 0) 0x7fc370001110 con 0x7fc37c146130 2024-04-20T21:30:29.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.695+0000 7fc37be00640 1 -- 172.21.15.86:0/1345620838 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc37c0cd8e0 con 0x7fc37c06e6d0 2024-04-20T21:30:29.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.695+0000 7fc381800640 1 --2- 172.21.15.86:0/1345620838 >> v2:172.21.15.144:3300/0 conn(0x7fc37c072240 0x7fc37c15cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:30:29.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.696+0000 7fc37be00640 1 --2- 172.21.15.86:0/1345620838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc37c06e6d0 0x7fc37c071d00 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7fc37c071c90 tx=0x7fc36802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5cdf8c4d3da3428 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:29.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.696+0000 7fc37b400640 1 -- 172.21.15.86:0/1345620838 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc368034070 con 0x7fc37c06e6d0 2024-04-20T21:30:29.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.696+0000 7fc37b400640 1 -- 172.21.15.86:0/1345620838 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc368030610 con 0x7fc37c06e6d0 2024-04-20T21:30:29.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.696+0000 7fc37b400640 1 -- 172.21.15.86:0/1345620838 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3680309e0 con 0x7fc37c06e6d0 2024-04-20T21:30:29.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.697+0000 7fc383800640 1 -- 172.21.15.86:0/1345620838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc37c06e6d0 msgr2=0x7fc37c071d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:29.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.697+0000 7fc383800640 1 --2- 172.21.15.86:0/1345620838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc37c06e6d0 0x7fc37c071d00 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7fc37c071c90 tx=0x7fc36802b960 comp rx=0 tx=0).stop 2024-04-20T21:30:29.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.697+0000 7fc383800640 1 -- 172.21.15.86:0/1345620838 shutdown_connections 2024-04-20T21:30:29.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.697+0000 7fc383800640 1 --2- 172.21.15.86:0/1345620838 >> v2:172.21.15.144:3300/0 conn(0x7fc37c072240 0x7fc37c15cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:29.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.697+0000 7fc383800640 1 --2- 172.21.15.86:0/1345620838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc37c06e6d0 0x7fc37c071d00 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:29.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.697+0000 7fc383800640 1 -- 172.21.15.86:0/1345620838 >> 172.21.15.86:0/1345620838 conn(0x7fc37c0b2360 msgr2=0x7fc37c0b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:29.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.697+0000 7fc383800640 1 -- 172.21.15.86:0/1345620838 shutdown_connections 2024-04-20T21:30:29.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.697+0000 7fc383800640 1 -- 172.21.15.86:0/1345620838 wait complete. 2024-04-20T21:30:29.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.698+0000 7fc383800640 1 Processor -- start 2024-04-20T21:30:29.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.699+0000 7fc383800640 1 -- start start 2024-04-20T21:30:29.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.699+0000 7fc383800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc37c06e6d0 0x7fc37c15ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:29.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.699+0000 7fc383800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc37c072240 0x7fc37c12d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:29.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.700+0000 7fc383800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc37c146130 0x7fc37c15bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:29.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.700+0000 7fc383800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c06e1b0 con 0x7fc37c072240 2024-04-20T21:30:29.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.700+0000 7fc383800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c06ded0 con 0x7fc37c06e6d0 2024-04-20T21:30:29.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.700+0000 7fc383800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc37c06f150 con 0x7fc37c146130 2024-04-20T21:30:29.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.700+0000 7fc37be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc37c072240 0x7fc37c12d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:29.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.700+0000 7fc37be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc37c072240 0x7fc37c12d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53502/0 (socket says 172.21.15.86:53502) 2024-04-20T21:30:29.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.700+0000 7fc37be00640 1 -- 172.21.15.86:0/3253997753 learned_addr learned my addr 172.21.15.86:0/3253997753 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:29.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.700+0000 7fc37be00640 1 -- 172.21.15.86:0/3253997753 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc37c146130 msgr2=0x7fc37c15bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:29.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.700+0000 7fc37be00640 1 --2- 172.21.15.86:0/3253997753 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc37c146130 0x7fc37c15bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:29.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.700+0000 7fc37be00640 1 -- 172.21.15.86:0/3253997753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc37c06e6d0 msgr2=0x7fc37c15ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:29.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.700+0000 7fc37be00640 1 --2- 172.21.15.86:0/3253997753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc37c06e6d0 0x7fc37c15ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:29.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.700+0000 7fc37be00640 1 -- 172.21.15.86:0/3253997753 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc37c12de60 con 0x7fc37c072240 2024-04-20T21:30:29.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.701+0000 7fc37be00640 1 --2- 172.21.15.86:0/3253997753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc37c072240 0x7fc37c12d3a0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7fc368009890 tx=0x7fc3680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:29.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.701+0000 7fc378c00640 1 -- 172.21.15.86:0/3253997753 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc368034040 con 0x7fc37c072240 2024-04-20T21:30:29.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.701+0000 7fc378c00640 1 -- 172.21.15.86:0/3253997753 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc36803acc0 con 0x7fc37c072240 2024-04-20T21:30:29.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.701+0000 7fc378c00640 1 -- 172.21.15.86:0/3253997753 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc368044640 con 0x7fc37c072240 2024-04-20T21:30:29.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.701+0000 7fc383800640 1 -- 172.21.15.86:0/3253997753 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc37c12e140 con 0x7fc37c072240 2024-04-20T21:30:29.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.701+0000 7fc383800640 1 -- 172.21.15.86:0/3253997753 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc37c12e700 con 0x7fc37c072240 2024-04-20T21:30:29.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.702+0000 7fc383800640 1 -- 172.21.15.86:0/3253997753 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc37c0689e0 con 0x7fc37c072240 2024-04-20T21:30:29.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.703+0000 7fc378c00640 1 -- 172.21.15.86:0/3253997753 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc368037070 con 0x7fc37c072240 2024-04-20T21:30:29.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.705+0000 7fc378c00640 1 --2- 172.21.15.86:0/3253997753 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc35c06c8f0 0x7fc35c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:29.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.705+0000 7fc378c00640 1 -- 172.21.15.86:0/3253997753 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5795+0+0 (secure 0 0 0) 0x7fc3680ad480 con 0x7fc37c072240 2024-04-20T21:30:29.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.705+0000 7fc380e00640 1 --2- 172.21.15.86:0/3253997753 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc35c06c8f0 0x7fc35c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:29.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.705+0000 7fc380e00640 1 --2- 172.21.15.86:0/3253997753 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc35c06c8f0 0x7fc35c06edb0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fc370001620 tx=0x7fc370001310 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:29.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.709+0000 7fc378c00640 1 -- 172.21.15.86:0/3253997753 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc36803f070 con 0x7fc37c072240 2024-04-20T21:30:29.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.911+0000 7fc383800640 1 -- 172.21.15.86:0/3253997753 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc37c15ff50 con 0x7fc37c072240 2024-04-20T21:30:29.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.913+0000 7fc378c00640 1 -- 172.21.15.86:0/3253997753 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) ==== 74+0+16754 (secure 0 0 0) 0x7fc36807b7b0 con 0x7fc37c072240 2024-04-20T21:30:29.914 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:29.914 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":113,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:30:19.784426+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:29:41.532824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-20T21:29:42.749687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6837/289590924":"2024-04-21T21:30:19.784403+0000","172.21.15.86:6835/289590924":"2024-04-21T21:30:19.784403+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:30:29.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.916+0000 7fc383800640 1 -- 172.21.15.86:0/3253997753 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc35c06c8f0 msgr2=0x7fc35c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:29.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.916+0000 7fc383800640 1 --2- 172.21.15.86:0/3253997753 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc35c06c8f0 0x7fc35c06edb0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fc370001620 tx=0x7fc370001310 comp rx=0 tx=0).stop 2024-04-20T21:30:29.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.916+0000 7fc383800640 1 -- 172.21.15.86:0/3253997753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc37c072240 msgr2=0x7fc37c12d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:29.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.916+0000 7fc383800640 1 --2- 172.21.15.86:0/3253997753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc37c072240 0x7fc37c12d3a0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7fc368009890 tx=0x7fc3680046c0 comp rx=0 tx=0).stop 2024-04-20T21:30:29.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.916+0000 7fc383800640 1 -- 172.21.15.86:0/3253997753 shutdown_connections 2024-04-20T21:30:29.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.916+0000 7fc383800640 1 --2- 172.21.15.86:0/3253997753 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc35c06c8f0 0x7fc35c06edb0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:29.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.916+0000 7fc383800640 1 --2- 172.21.15.86:0/3253997753 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc37c146130 0x7fc37c15bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:29.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.916+0000 7fc383800640 1 --2- 172.21.15.86:0/3253997753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc37c072240 0x7fc37c12d3a0 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:29.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.916+0000 7fc383800640 1 --2- 172.21.15.86:0/3253997753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc37c06e6d0 0x7fc37c15ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:29.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.916+0000 7fc383800640 1 -- 172.21.15.86:0/3253997753 >> 172.21.15.86:0/3253997753 conn(0x7fc37c0b2360 msgr2=0x7fc37c145420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:29.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.916+0000 7fc383800640 1 -- 172.21.15.86:0/3253997753 shutdown_connections 2024-04-20T21:30:29.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:29.916+0000 7fc383800640 1 -- 172.21.15.86:0/3253997753 wait complete. 2024-04-20T21:30:29.935 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:29:41.532824+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:29:42.749687+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:30:29.936 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:30:30.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.129+0000 7fbfaaa00640 1 Processor -- start 2024-04-20T21:30:30.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.129+0000 7fbfaaa00640 1 -- start start 2024-04-20T21:30:30.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.131+0000 7fbfaaa00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbfa4162ed0 0x7fbfa406b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:30.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.131+0000 7fbfaaa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbfa40b3410 0x7fbfa406bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:30.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.131+0000 7fbfaaa00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfa40b3910 con 0x7fbfa406e160 2024-04-20T21:30:30.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.131+0000 7fbfaaa00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fbfa40b3bf0 con 0x7fbfa4162ed0 2024-04-20T21:30:30.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.131+0000 7fbfaaa00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbfa40b3ed0 con 0x7fbfa40b3410 2024-04-20T21:30:30.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.131+0000 7fbfa8a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fbfa406e160 0x7fbfa4071db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:32886/0 (socket says 172.21.15.86:32886) 2024-04-20T21:30:30.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.131+0000 7fbfa8a00640 1 -- 172.21.15.86:0/2446655887 learned_addr learned my addr 172.21.15.86:0/2446655887 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:30:30.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.131+0000 7fbf9be00640 1 --2- 172.21.15.86:0/2446655887 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbfa40b3410 0x7fbfa406bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:30.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.131+0000 7fbfa3400640 1 -- 172.21.15.86:0/2446655887 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3030681657 0 0) 0x7fbfa40b3910 con 0x7fbfa406e160 2024-04-20T21:30:30.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.132+0000 7fbfa3400640 1 -- 172.21.15.86:0/2446655887 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbf88003520 con 0x7fbfa406e160 2024-04-20T21:30:30.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.132+0000 7fbfa3e00640 1 --2- 172.21.15.86:0/2446655887 >> v2:172.21.15.144:3300/0 conn(0x7fbfa4162ed0 0x7fbfa406b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:30.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.132+0000 7fbfa3400640 1 -- 172.21.15.86:0/2446655887 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1489992508 0 0) 0x7fbf88003520 con 0x7fbfa406e160 2024-04-20T21:30:30.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.132+0000 7fbfa3400640 1 -- 172.21.15.86:0/2446655887 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbfa40b3410 msgr2=0x7fbfa406bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:30.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.132+0000 7fbfa3400640 1 --2- 172.21.15.86:0/2446655887 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbfa40b3410 0x7fbfa406bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:30.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.132+0000 7fbfa3400640 1 -- 172.21.15.86:0/2446655887 >> v2:172.21.15.144:3300/0 conn(0x7fbfa4162ed0 msgr2=0x7fbfa406b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:30.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.132+0000 7fbfa3400640 1 --2- 172.21.15.86:0/2446655887 >> v2:172.21.15.144:3300/0 conn(0x7fbfa4162ed0 0x7fbfa406b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:30.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.132+0000 7fbfa3400640 1 -- 172.21.15.86:0/2446655887 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfa40b41b0 con 0x7fbfa406e160 2024-04-20T21:30:30.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.132+0000 7fbfa3400640 1 -- 172.21.15.86:0/2446655887 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fbf9c0019f0 con 0x7fbfa406e160 2024-04-20T21:30:30.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.132+0000 7fbfa3e00640 1 --2- 172.21.15.86:0/2446655887 >> v2:172.21.15.144:3300/0 conn(0x7fbfa4162ed0 0x7fbfa406b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:30:30.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.132+0000 7fbfa3400640 1 -- 172.21.15.86:0/2446655887 >> v1:172.21.15.86:6789/0 conn(0x7fbfa406e160 legacy=0x7fbfa4071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:30.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.132+0000 7fbfa3400640 1 --2- 172.21.15.86:0/2446655887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf880039b0 0x7fbf88003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:30.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.132+0000 7fbfa3400640 1 --2- 172.21.15.86:0/2446655887 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbfa40b3410 0x7fbf88004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:30.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.132+0000 7fbfa3400640 1 --2- 172.21.15.86:0/2446655887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf88004d40 0x7fbf880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:30.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.132+0000 7fbfa3400640 1 -- 172.21.15.86:0/2446655887 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa40b3bf0 con 0x7fbf880039b0 2024-04-20T21:30:30.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.132+0000 7fbfa3400640 1 -- 172.21.15.86:0/2446655887 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa40b3ed0 con 0x7fbf88004d40 2024-04-20T21:30:30.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.132+0000 7fbfa3400640 1 -- 172.21.15.86:0/2446655887 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbf8800ba70 con 0x7fbfa40b3410 2024-04-20T21:30:30.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.132+0000 7fbfa3e00640 1 --2- 172.21.15.86:0/2446655887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf880039b0 0x7fbf88003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:30.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.133+0000 7fbf9be00640 1 --2- 172.21.15.86:0/2446655887 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbfa40b3410 0x7fbf88004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:30.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.133+0000 7fbfa3e00640 1 -- 172.21.15.86:0/2446655887 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbfa40b3410 msgr2=0x7fbf88004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:30.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.133+0000 7fbfa3e00640 1 --2- 172.21.15.86:0/2446655887 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbfa40b3410 0x7fbf88004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:30.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.133+0000 7fbfa3e00640 1 -- 172.21.15.86:0/2446655887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf88004d40 msgr2=0x7fbf880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:30.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.133+0000 7fbfa8a00640 1 --2- 172.21.15.86:0/2446655887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf88004d40 0x7fbf880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:30.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.133+0000 7fbfa3e00640 1 --2- 172.21.15.86:0/2446655887 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf88004d40 0x7fbf880051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:30.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.133+0000 7fbfa3e00640 1 -- 172.21.15.86:0/2446655887 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf88003520 con 0x7fbf880039b0 2024-04-20T21:30:30.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.134+0000 7fbfa3e00640 1 --2- 172.21.15.86:0/2446655887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf880039b0 0x7fbf88003dd0 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7fbf940098a0 tx=0x7fbf88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=208da71c657646c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:30.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.134+0000 7fbfa3400640 1 -- 172.21.15.86:0/2446655887 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9400f040 con 0x7fbf880039b0 2024-04-20T21:30:30.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.134+0000 7fbfa3400640 1 -- 172.21.15.86:0/2446655887 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf94004230 con 0x7fbf880039b0 2024-04-20T21:30:30.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.134+0000 7fbfa3400640 1 -- 172.21.15.86:0/2446655887 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf94003610 con 0x7fbf880039b0 2024-04-20T21:30:30.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.135+0000 7fbfaaa00640 1 -- 172.21.15.86:0/2446655887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf880039b0 msgr2=0x7fbf88003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:30.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.135+0000 7fbfaaa00640 1 --2- 172.21.15.86:0/2446655887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf880039b0 0x7fbf88003dd0 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7fbf940098a0 tx=0x7fbf88005830 comp rx=0 tx=0).stop 2024-04-20T21:30:30.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.135+0000 7fbfa8a00640 1 -- 172.21.15.86:0/2446655887 reap_dead start 2024-04-20T21:30:30.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.135+0000 7fbfaaa00640 1 -- 172.21.15.86:0/2446655887 shutdown_connections 2024-04-20T21:30:30.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.135+0000 7fbfaaa00640 1 -- 172.21.15.86:0/2446655887 >> 172.21.15.86:0/2446655887 conn(0x7fbfa40b2110 msgr2=0x7fbfa40b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:30.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.135+0000 7fbfaaa00640 1 -- 172.21.15.86:0/2446655887 shutdown_connections 2024-04-20T21:30:30.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.135+0000 7fbfaaa00640 1 -- 172.21.15.86:0/2446655887 wait complete. 2024-04-20T21:30:30.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.137+0000 7fbfaaa00640 1 Processor -- start 2024-04-20T21:30:30.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.137+0000 7fbfaaa00640 1 -- start start 2024-04-20T21:30:30.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.138+0000 7fbfaaa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf880039b0 0x7fbfa4165e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:30.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.138+0000 7fbfaaa00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbfa40aaf30 0x7fbfa40a9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:30.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.138+0000 7fbfaaa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbfa415ab30 0x7fbfa40b1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:30.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.138+0000 7fbfaaa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa406fca0 con 0x7fbf880039b0 2024-04-20T21:30:30.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.138+0000 7fbfaaa00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa406fb20 con 0x7fbfa40aaf30 2024-04-20T21:30:30.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.138+0000 7fbfaaa00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbfa40ce370 con 0x7fbfa415ab30 2024-04-20T21:30:30.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.138+0000 7fbfa3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf880039b0 0x7fbfa4165e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:30.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.138+0000 7fbfa3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf880039b0 0x7fbfa4165e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53514/0 (socket says 172.21.15.86:53514) 2024-04-20T21:30:30.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.138+0000 7fbfa3e00640 1 -- 172.21.15.86:0/2616784637 learned_addr learned my addr 172.21.15.86:0/2616784637 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:30.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.139+0000 7fbfa3e00640 1 -- 172.21.15.86:0/2616784637 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbfa415ab30 msgr2=0x7fbfa40b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:30.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.139+0000 7fbfa3e00640 1 --2- 172.21.15.86:0/2616784637 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbfa415ab30 0x7fbfa40b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:30.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.139+0000 7fbfa3e00640 1 -- 172.21.15.86:0/2616784637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbfa40aaf30 msgr2=0x7fbfa40a9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:30.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.139+0000 7fbfa3e00640 1 --2- 172.21.15.86:0/2616784637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbfa40aaf30 0x7fbfa40a9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:30.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.139+0000 7fbfa3e00640 1 -- 172.21.15.86:0/2616784637 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfa41663a0 con 0x7fbf880039b0 2024-04-20T21:30:30.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.139+0000 7fbfa3e00640 1 --2- 172.21.15.86:0/2616784637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf880039b0 0x7fbfa4165e60 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7fbf9402bff0 tx=0x7fbf94004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:30.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.139+0000 7fbfa0c00640 1 -- 172.21.15.86:0/2616784637 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf94047070 con 0x7fbf880039b0 2024-04-20T21:30:30.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.139+0000 7fbfa0c00640 1 -- 172.21.15.86:0/2616784637 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf94042400 con 0x7fbf880039b0 2024-04-20T21:30:30.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.139+0000 7fbfa0c00640 1 -- 172.21.15.86:0/2616784637 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf940427d0 con 0x7fbf880039b0 2024-04-20T21:30:30.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.139+0000 7fbfaaa00640 1 -- 172.21.15.86:0/2616784637 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfa40a9c90 con 0x7fbf880039b0 2024-04-20T21:30:30.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.140+0000 7fbfaaa00640 1 -- 172.21.15.86:0/2616784637 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfa40aa250 con 0x7fbf880039b0 2024-04-20T21:30:30.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.141+0000 7fbfaaa00640 1 -- 172.21.15.86:0/2616784637 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfa40689e0 con 0x7fbf880039b0 2024-04-20T21:30:30.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.142+0000 7fbfa0c00640 1 -- 172.21.15.86:0/2616784637 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbf9404a070 con 0x7fbf880039b0 2024-04-20T21:30:30.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.142+0000 7fbfa0c00640 1 --2- 172.21.15.86:0/2616784637 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbf8006c920 0x7fbf8006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:30.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.143+0000 7fbfa0c00640 1 -- 172.21.15.86:0/2616784637 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5795+0+0 (secure 0 0 0) 0x7fbf940ad420 con 0x7fbf880039b0 2024-04-20T21:30:30.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.143+0000 7fbf9be00640 1 --2- 172.21.15.86:0/2616784637 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbf8006c920 0x7fbf8006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:30.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.143+0000 7fbf9be00640 1 --2- 172.21.15.86:0/2616784637 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbf8006c920 0x7fbf8006ede0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fbfa40b1f30 tx=0x7fbf8c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:30.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.147+0000 7fbfa0c00640 1 -- 172.21.15.86:0/2616784637 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbf94033070 con 0x7fbf880039b0 2024-04-20T21:30:30.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.350+0000 7fbfaaa00640 1 -- 172.21.15.86:0/2616784637 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbfa4162020 con 0x7fbf880039b0 2024-04-20T21:30:30.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.351+0000 7fbfa0c00640 1 -- 172.21.15.86:0/2616784637 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) ==== 93+0+10 (secure 0 0 0) 0x7fbf9407b750 con 0x7fbf880039b0 2024-04-20T21:30:30.351 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:30:30.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.353+0000 7fbfaaa00640 1 -- 172.21.15.86:0/2616784637 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbf8006c920 msgr2=0x7fbf8006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:30.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.353+0000 7fbfaaa00640 1 --2- 172.21.15.86:0/2616784637 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbf8006c920 0x7fbf8006ede0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fbfa40b1f30 tx=0x7fbf8c008040 comp rx=0 tx=0).stop 2024-04-20T21:30:30.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.353+0000 7fbfaaa00640 1 -- 172.21.15.86:0/2616784637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf880039b0 msgr2=0x7fbfa4165e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:30.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.353+0000 7fbfaaa00640 1 --2- 172.21.15.86:0/2616784637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf880039b0 0x7fbfa4165e60 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7fbf9402bff0 tx=0x7fbf94004730 comp rx=0 tx=0).stop 2024-04-20T21:30:30.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.354+0000 7fbfaaa00640 1 -- 172.21.15.86:0/2616784637 shutdown_connections 2024-04-20T21:30:30.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.354+0000 7fbfaaa00640 1 --2- 172.21.15.86:0/2616784637 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbf8006c920 0x7fbf8006ede0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:30.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.354+0000 7fbfaaa00640 1 --2- 172.21.15.86:0/2616784637 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbfa415ab30 0x7fbfa40b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:30.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.354+0000 7fbfaaa00640 1 --2- 172.21.15.86:0/2616784637 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbfa40aaf30 0x7fbfa40a9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:30.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.354+0000 7fbfaaa00640 1 --2- 172.21.15.86:0/2616784637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf880039b0 0x7fbfa4165e60 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:30.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.354+0000 7fbfaaa00640 1 -- 172.21.15.86:0/2616784637 >> 172.21.15.86:0/2616784637 conn(0x7fbfa40b2110 msgr2=0x7fbfa40b0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:30.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.354+0000 7fbfaaa00640 1 -- 172.21.15.86:0/2616784637 shutdown_connections 2024-04-20T21:30:30.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.354+0000 7fbfaaa00640 1 -- 172.21.15.86:0/2616784637 wait complete. 2024-04-20T21:30:30.373 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:30:30.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.577+0000 7f8632a00640 1 Processor -- start 2024-04-20T21:30:30.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.578+0000 7f8632a00640 1 -- start start 2024-04-20T21:30:30.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.579+0000 7f8632a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f862c06bbf0 0x7f862c06e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:30.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.579+0000 7f8632a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f862c06bfd0 0x7f862c145b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:30.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.579+0000 7f8632a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f862c146040 0x7f862c0cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:30.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.579+0000 7f8632a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f862c15cd40 con 0x7f862c146040 2024-04-20T21:30:30.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.579+0000 7f8632a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f862c15d000 con 0x7f862c06bbf0 2024-04-20T21:30:30.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.579+0000 7f8632a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f862c15d2e0 con 0x7f862c06bfd0 2024-04-20T21:30:30.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.580+0000 7f8630a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f862c146040 0x7f862c0cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:30.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.580+0000 7f8630a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f862c146040 0x7f862c0cd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53524/0 (socket says 172.21.15.86:53524) 2024-04-20T21:30:30.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.580+0000 7f8630a00640 1 -- 172.21.15.86:0/1049056580 learned_addr learned my addr 172.21.15.86:0/1049056580 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:30.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.580+0000 7f8623400640 1 --2- 172.21.15.86:0/1049056580 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f862c06bfd0 0x7f862c145b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:30.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.580+0000 7f862be00640 1 --2- 172.21.15.86:0/1049056580 >> v2:172.21.15.144:3300/0 conn(0x7f862c06bbf0 0x7f862c06e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:30.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.580+0000 7f8630a00640 1 -- 172.21.15.86:0/1049056580 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f862c06bfd0 msgr2=0x7f862c145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:30.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.580+0000 7f8630a00640 1 --2- 172.21.15.86:0/1049056580 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f862c06bfd0 0x7f862c145b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:30.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.580+0000 7f8630a00640 1 -- 172.21.15.86:0/1049056580 >> v2:172.21.15.144:3300/0 conn(0x7f862c06bbf0 msgr2=0x7f862c06e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:30.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.580+0000 7f8630a00640 1 --2- 172.21.15.86:0/1049056580 >> v2:172.21.15.144:3300/0 conn(0x7f862c06bbf0 0x7f862c06e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:30.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.580+0000 7f8630a00640 1 -- 172.21.15.86:0/1049056580 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f862c15d5c0 con 0x7f862c146040 2024-04-20T21:30:30.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.581+0000 7f8630a00640 1 --2- 172.21.15.86:0/1049056580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f862c146040 0x7f862c0cd540 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f862c073150 tx=0x7f862402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db7a13b269617ab7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:30.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.582+0000 7f862b400640 1 -- 172.21.15.86:0/1049056580 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862402ca70 con 0x7f862c146040 2024-04-20T21:30:30.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.582+0000 7f862b400640 1 -- 172.21.15.86:0/1049056580 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f862402cc10 con 0x7f862c146040 2024-04-20T21:30:30.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.582+0000 7f862b400640 1 -- 172.21.15.86:0/1049056580 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86240343f0 con 0x7f862c146040 2024-04-20T21:30:30.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.582+0000 7f8632a00640 1 -- 172.21.15.86:0/1049056580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f862c146040 msgr2=0x7f862c0cd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:30.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.582+0000 7f8632a00640 1 --2- 172.21.15.86:0/1049056580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f862c146040 0x7f862c0cd540 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f862c073150 tx=0x7f862402bed0 comp rx=0 tx=0).stop 2024-04-20T21:30:30.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.583+0000 7f8632a00640 1 -- 172.21.15.86:0/1049056580 shutdown_connections 2024-04-20T21:30:30.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.583+0000 7f8632a00640 1 --2- 172.21.15.86:0/1049056580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f862c146040 0x7f862c0cd540 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:30.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.583+0000 7f8632a00640 1 --2- 172.21.15.86:0/1049056580 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f862c06bfd0 0x7f862c145b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:30.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.583+0000 7f8632a00640 1 --2- 172.21.15.86:0/1049056580 >> v2:172.21.15.144:3300/0 conn(0x7f862c06bbf0 0x7f862c06e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:30.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.583+0000 7f8632a00640 1 -- 172.21.15.86:0/1049056580 >> 172.21.15.86:0/1049056580 conn(0x7f862c0b2110 msgr2=0x7f862c0b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:30.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.583+0000 7f8632a00640 1 -- 172.21.15.86:0/1049056580 shutdown_connections 2024-04-20T21:30:30.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.583+0000 7f8632a00640 1 -- 172.21.15.86:0/1049056580 wait complete. 2024-04-20T21:30:30.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.584+0000 7f8632a00640 1 Processor -- start 2024-04-20T21:30:30.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.585+0000 7f8632a00640 1 -- start start 2024-04-20T21:30:30.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.585+0000 7f8632a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f862c06bbf0 0x7f862c1e5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:30.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.585+0000 7f8632a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f862c06bfd0 0x7f862c15c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:30.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.586+0000 7f8632a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f862c146040 0x7f862c144150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:30.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.586+0000 7f8632a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f862c0cdc00 con 0x7f862c06bfd0 2024-04-20T21:30:30.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.586+0000 7f8632a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f862c0cda80 con 0x7f862c146040 2024-04-20T21:30:30.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.586+0000 7f8632a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f862c0cdd80 con 0x7f862c06bbf0 2024-04-20T21:30:30.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.586+0000 7f8623400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f862c06bfd0 0x7f862c15c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:30.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.586+0000 7f8623400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f862c06bfd0 0x7f862c15c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53538/0 (socket says 172.21.15.86:53538) 2024-04-20T21:30:30.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.586+0000 7f8623400640 1 -- 172.21.15.86:0/2480813229 learned_addr learned my addr 172.21.15.86:0/2480813229 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:30.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.586+0000 7f8630a00640 1 --2- 172.21.15.86:0/2480813229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f862c146040 0x7f862c144150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:30.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.586+0000 7f862be00640 1 --2- 172.21.15.86:0/2480813229 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f862c06bbf0 0x7f862c1e5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:30.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.586+0000 7f8623400640 1 -- 172.21.15.86:0/2480813229 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f862c06bbf0 msgr2=0x7f862c1e5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:30.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.586+0000 7f8623400640 1 --2- 172.21.15.86:0/2480813229 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f862c06bbf0 0x7f862c1e5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:30.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.586+0000 7f8623400640 1 -- 172.21.15.86:0/2480813229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f862c146040 msgr2=0x7f862c144150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:30.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.586+0000 7f8623400640 1 --2- 172.21.15.86:0/2480813229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f862c146040 0x7f862c144150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:30.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.586+0000 7f8623400640 1 -- 172.21.15.86:0/2480813229 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f862c1e5050 con 0x7f862c06bfd0 2024-04-20T21:30:30.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.587+0000 7f8623400640 1 --2- 172.21.15.86:0/2480813229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f862c06bfd0 0x7f862c15c570 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f861400d5f0 tx=0x7f861400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:30.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.587+0000 7f8628c00640 1 -- 172.21.15.86:0/2480813229 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86140125f0 con 0x7f862c06bfd0 2024-04-20T21:30:30.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.587+0000 7f8628c00640 1 -- 172.21.15.86:0/2480813229 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8614019070 con 0x7f862c06bfd0 2024-04-20T21:30:30.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.587+0000 7f8632a00640 1 -- 172.21.15.86:0/2480813229 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f862c144b30 con 0x7f862c06bfd0 2024-04-20T21:30:30.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.587+0000 7f8628c00640 1 -- 172.21.15.86:0/2480813229 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86140129d0 con 0x7f862c06bfd0 2024-04-20T21:30:30.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.587+0000 7f8632a00640 1 -- 172.21.15.86:0/2480813229 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f862c15cab0 con 0x7f862c06bfd0 2024-04-20T21:30:30.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.588+0000 7f8632a00640 1 -- 172.21.15.86:0/2480813229 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f862c0689e0 con 0x7f862c06bfd0 2024-04-20T21:30:30.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.589+0000 7f8628c00640 1 -- 172.21.15.86:0/2480813229 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8614012b70 con 0x7f862c06bfd0 2024-04-20T21:30:30.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.591+0000 7f8628c00640 1 --2- 172.21.15.86:0/2480813229 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f860806c8f0 0x7f860806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:30.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.591+0000 7f8628c00640 1 -- 172.21.15.86:0/2480813229 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5795+0+0 (secure 0 0 0) 0x7f861408bff0 con 0x7f862c06bfd0 2024-04-20T21:30:30.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.591+0000 7f862be00640 1 --2- 172.21.15.86:0/2480813229 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f860806c8f0 0x7f860806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:30.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.592+0000 7f862be00640 1 --2- 172.21.15.86:0/2480813229 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f860806c8f0 0x7f860806edb0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f861c006fd0 tx=0x7f861c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:30.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.595+0000 7f8628c00640 1 -- 172.21.15.86:0/2480813229 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f861405a380 con 0x7f862c06bfd0 2024-04-20T21:30:30.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.793+0000 7f8632a00640 1 -- 172.21.15.86:0/2480813229 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f862c06c780 con 0x7f862c06bfd0 2024-04-20T21:30:30.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.794+0000 7f8628c00640 1 -- 172.21.15.86:0/2480813229 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v113) ==== 104+0+11 (secure 0 0 0) 0x7f8614059c80 con 0x7f862c06bfd0 2024-04-20T21:30:30.794 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-20T21:30:30.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.796+0000 7f8632a00640 1 -- 172.21.15.86:0/2480813229 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f860806c8f0 msgr2=0x7f860806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:30.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.796+0000 7f8632a00640 1 --2- 172.21.15.86:0/2480813229 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f860806c8f0 0x7f860806edb0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f861c006fd0 tx=0x7f861c008040 comp rx=0 tx=0).stop 2024-04-20T21:30:30.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.796+0000 7f8632a00640 1 -- 172.21.15.86:0/2480813229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f862c06bfd0 msgr2=0x7f862c15c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:30.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.796+0000 7f8632a00640 1 --2- 172.21.15.86:0/2480813229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f862c06bfd0 0x7f862c15c570 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f861400d5f0 tx=0x7f861400dab0 comp rx=0 tx=0).stop 2024-04-20T21:30:30.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.796+0000 7f8632a00640 1 -- 172.21.15.86:0/2480813229 shutdown_connections 2024-04-20T21:30:30.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.796+0000 7f8632a00640 1 --2- 172.21.15.86:0/2480813229 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f860806c8f0 0x7f860806edb0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:30.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.796+0000 7f8632a00640 1 --2- 172.21.15.86:0/2480813229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f862c146040 0x7f862c144150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:30.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.796+0000 7f8632a00640 1 --2- 172.21.15.86:0/2480813229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f862c06bfd0 0x7f862c15c570 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:30.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.796+0000 7f8632a00640 1 --2- 172.21.15.86:0/2480813229 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f862c06bbf0 0x7f862c1e5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:30.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.796+0000 7f8632a00640 1 -- 172.21.15.86:0/2480813229 >> 172.21.15.86:0/2480813229 conn(0x7f862c0b2110 msgr2=0x7f862c0adf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:30.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.796+0000 7f8632a00640 1 -- 172.21.15.86:0/2480813229 shutdown_connections 2024-04-20T21:30:30.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:30.797+0000 7f8632a00640 1 -- 172.21.15.86:0/2480813229 wait complete. 2024-04-20T21:30:30.814 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:30:31.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.000+0000 7f9a88000640 1 Processor -- start 2024-04-20T21:30:31.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.000+0000 7f9a88000640 1 -- start start 2024-04-20T21:30:31.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.002+0000 7f9a88000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9a8006c090 0x7f9a800cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:31.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.002+0000 7f9a88000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f9a800b4ba0 0x7f9a800b5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:31.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.002+0000 7f9a88000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a80156110 con 0x7f9a80155d30 2024-04-20T21:30:31.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.002+0000 7f9a88000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f9a800ce460 con 0x7f9a8006c090 2024-04-20T21:30:31.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.002+0000 7f9a88000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f9a800ce740 con 0x7f9a800b4ba0 2024-04-20T21:30:31.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.002+0000 7f9a84c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9a8006c090 0x7f9a800cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:31.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.002+0000 7f9a84c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9a8006c090 0x7f9a800cf500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53540/0 (socket says 172.21.15.86:53540) 2024-04-20T21:30:31.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.002+0000 7f9a84c00640 1 -- 172.21.15.86:0/1081047931 learned_addr learned my addr 172.21.15.86:0/1081047931 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:31.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.003+0000 7f9a84c00640 1 -- 172.21.15.86:0/1081047931 >> v2:172.21.15.144:3300/0 conn(0x7f9a800b4ba0 msgr2=0x7f9a800b5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:31.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.003+0000 7f9a86000640 1 --2- 172.21.15.86:0/1081047931 >> v2:172.21.15.144:3300/0 conn(0x7f9a800b4ba0 0x7f9a800b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:31.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.003+0000 7f9a84c00640 1 --2- 172.21.15.86:0/1081047931 >> v2:172.21.15.144:3300/0 conn(0x7f9a800b4ba0 0x7f9a800b5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:31.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.003+0000 7f9a84c00640 1 -- 172.21.15.86:0/1081047931 >> v1:172.21.15.144:6789/0 conn(0x7f9a80155d30 legacy=0x7f9a8006b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:31.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.003+0000 7f9a84c00640 1 -- 172.21.15.86:0/1081047931 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a800cea20 con 0x7f9a8006c090 2024-04-20T21:30:31.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.003+0000 7f9a86000640 1 --2- 172.21.15.86:0/1081047931 >> v2:172.21.15.144:3300/0 conn(0x7f9a800b4ba0 0x7f9a800b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:30:31.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.004+0000 7f9a84c00640 1 --2- 172.21.15.86:0/1081047931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a8006c090 0x7f9a800cf500 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f9a800b4960 tx=0x7f9a6c02b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=800d9246f5c8aac5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:31.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.004+0000 7f9a7fe00640 1 -- 172.21.15.86:0/1081047931 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c031760 con 0x7f9a8006c090 2024-04-20T21:30:31.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.004+0000 7f9a7fe00640 1 -- 172.21.15.86:0/1081047931 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a6c037070 con 0x7f9a8006c090 2024-04-20T21:30:31.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.004+0000 7f9a7fe00640 1 -- 172.21.15.86:0/1081047931 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c031b20 con 0x7f9a8006c090 2024-04-20T21:30:31.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.005+0000 7f9a88000640 1 -- 172.21.15.86:0/1081047931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a8006c090 msgr2=0x7f9a800cf500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:31.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.005+0000 7f9a88000640 1 --2- 172.21.15.86:0/1081047931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a8006c090 0x7f9a800cf500 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f9a800b4960 tx=0x7f9a6c02b800 comp rx=0 tx=0).stop 2024-04-20T21:30:31.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.005+0000 7f9a88000640 1 -- 172.21.15.86:0/1081047931 shutdown_connections 2024-04-20T21:30:31.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.005+0000 7f9a88000640 1 --2- 172.21.15.86:0/1081047931 >> v2:172.21.15.144:3300/0 conn(0x7f9a800b4ba0 0x7f9a800b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:31.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.005+0000 7f9a88000640 1 --2- 172.21.15.86:0/1081047931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a8006c090 0x7f9a800cf500 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:31.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.005+0000 7f9a88000640 1 -- 172.21.15.86:0/1081047931 >> 172.21.15.86:0/1081047931 conn(0x7f9a800b2110 msgr2=0x7f9a800b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:31.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.005+0000 7f9a88000640 1 -- 172.21.15.86:0/1081047931 shutdown_connections 2024-04-20T21:30:31.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.005+0000 7f9a88000640 1 -- 172.21.15.86:0/1081047931 wait complete. 2024-04-20T21:30:31.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.007+0000 7f9a88000640 1 Processor -- start 2024-04-20T21:30:31.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.007+0000 7f9a88000640 1 -- start start 2024-04-20T21:30:31.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.008+0000 7f9a88000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a8006c090 0x7f9a800b3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:31.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.008+0000 7f9a88000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a800b4ba0 0x7f9a8014af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:31.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.008+0000 7f9a88000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a80155d30 0x7f9a8014ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:31.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.008+0000 7f9a88000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9a800b5570 con 0x7f9a8006c090 2024-04-20T21:30:31.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.008+0000 7f9a88000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f9a800cefe0 con 0x7f9a80155d30 2024-04-20T21:30:31.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.008+0000 7f9a88000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9a800b56f0 con 0x7f9a800b4ba0 2024-04-20T21:30:31.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.008+0000 7f9a85600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a8006c090 0x7f9a800b3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:31.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.008+0000 7f9a85600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a8006c090 0x7f9a800b3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53556/0 (socket says 172.21.15.86:53556) 2024-04-20T21:30:31.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.008+0000 7f9a85600640 1 -- 172.21.15.86:0/4031791491 learned_addr learned my addr 172.21.15.86:0/4031791491 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:31.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.009+0000 7f9a85600640 1 -- 172.21.15.86:0/4031791491 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a800b4ba0 msgr2=0x7f9a8014af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:31.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.009+0000 7f9a85600640 1 --2- 172.21.15.86:0/4031791491 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a800b4ba0 0x7f9a8014af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:31.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.009+0000 7f9a85600640 1 -- 172.21.15.86:0/4031791491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a80155d30 msgr2=0x7f9a8014ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:31.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.009+0000 7f9a85600640 1 --2- 172.21.15.86:0/4031791491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a80155d30 0x7f9a8014ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:31.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.009+0000 7f9a85600640 1 -- 172.21.15.86:0/4031791491 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a8014eaf0 con 0x7f9a8006c090 2024-04-20T21:30:31.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.009+0000 7f9a85600640 1 --2- 172.21.15.86:0/4031791491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a8006c090 0x7f9a800b3f10 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f9a7400c410 tx=0x7f9a7400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:31.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.009+0000 7f9a7d600640 1 -- 172.21.15.86:0/4031791491 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a74017070 con 0x7f9a8006c090 2024-04-20T21:30:31.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.009+0000 7f9a7d600640 1 -- 172.21.15.86:0/4031791491 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a740035f0 con 0x7f9a8006c090 2024-04-20T21:30:31.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.009+0000 7f9a88000640 1 -- 172.21.15.86:0/4031791491 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a8014edd0 con 0x7f9a8006c090 2024-04-20T21:30:31.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.009+0000 7f9a7d600640 1 -- 172.21.15.86:0/4031791491 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a74003a20 con 0x7f9a8006c090 2024-04-20T21:30:31.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.009+0000 7f9a88000640 1 -- 172.21.15.86:0/4031791491 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a8014f390 con 0x7f9a8006c090 2024-04-20T21:30:31.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.010+0000 7f9a88000640 1 -- 172.21.15.86:0/4031791491 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a8006be40 con 0x7f9a8006c090 2024-04-20T21:30:31.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.011+0000 7f9a7d600640 1 -- 172.21.15.86:0/4031791491 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9a7401a070 con 0x7f9a8006c090 2024-04-20T21:30:31.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.012+0000 7f9a7d600640 1 --2- 172.21.15.86:0/4031791491 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9a6006c940 0x7f9a6006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:31.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.012+0000 7f9a7d600640 1 -- 172.21.15.86:0/4031791491 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5795+0+0 (secure 0 0 0) 0x7f9a7408b0d0 con 0x7f9a8006c090 2024-04-20T21:30:31.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.013+0000 7f9a84c00640 1 --2- 172.21.15.86:0/4031791491 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9a6006c940 0x7f9a6006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:31.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.013+0000 7f9a84c00640 1 --2- 172.21.15.86:0/4031791491 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9a6006c940 0x7f9a6006ee00 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c036000 tx=0x7f9a6c030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:31.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.017+0000 7f9a7d600640 1 -- 172.21.15.86:0/4031791491 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9a8006be40 con 0x7f9a8006c090 2024-04-20T21:30:31.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.214+0000 7f9a88000640 1 -- 172.21.15.86:0/4031791491 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9a800689e0 con 0x7f9a8006c090 2024-04-20T21:30:31.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.215+0000 7f9a7d600640 1 -- 172.21.15.86:0/4031791491 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v113) ==== 100+0+11 (secure 0 0 0) 0x7f9a74059500 con 0x7f9a8006c090 2024-04-20T21:30:31.215 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-20T21:30:31.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.218+0000 7f9a88000640 1 -- 172.21.15.86:0/4031791491 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9a6006c940 msgr2=0x7f9a6006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:31.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.218+0000 7f9a88000640 1 --2- 172.21.15.86:0/4031791491 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9a6006c940 0x7f9a6006ee00 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c036000 tx=0x7f9a6c030000 comp rx=0 tx=0).stop 2024-04-20T21:30:31.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.218+0000 7f9a88000640 1 -- 172.21.15.86:0/4031791491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a8006c090 msgr2=0x7f9a800b3f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:31.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.218+0000 7f9a88000640 1 --2- 172.21.15.86:0/4031791491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a8006c090 0x7f9a800b3f10 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f9a7400c410 tx=0x7f9a7400c8d0 comp rx=0 tx=0).stop 2024-04-20T21:30:31.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.218+0000 7f9a88000640 1 -- 172.21.15.86:0/4031791491 shutdown_connections 2024-04-20T21:30:31.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.218+0000 7f9a88000640 1 --2- 172.21.15.86:0/4031791491 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f9a6006c940 0x7f9a6006ee00 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:31.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.218+0000 7f9a88000640 1 --2- 172.21.15.86:0/4031791491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a80155d30 0x7f9a8014ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:31.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.218+0000 7f9a88000640 1 --2- 172.21.15.86:0/4031791491 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f9a800b4ba0 0x7f9a8014af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:31.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.218+0000 7f9a88000640 1 --2- 172.21.15.86:0/4031791491 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a8006c090 0x7f9a800b3f10 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:31.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.218+0000 7f9a88000640 1 -- 172.21.15.86:0/4031791491 >> 172.21.15.86:0/4031791491 conn(0x7f9a800b2110 msgr2=0x7f9a80154ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:31.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.218+0000 7f9a88000640 1 -- 172.21.15.86:0/4031791491 shutdown_connections 2024-04-20T21:30:31.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.218+0000 7f9a88000640 1 -- 172.21.15.86:0/4031791491 wait complete. 2024-04-20T21:30:31.235 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T21:30:31.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.426+0000 7f8fe8a00640 1 Processor -- start 2024-04-20T21:30:31.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.426+0000 7f8fe8a00640 1 -- start start 2024-04-20T21:30:31.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.427+0000 7f8fe8a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fe406e1b0 0x7f8fe4071b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:31.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.428+0000 7f8fe8a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8fe40720b0 0x7f8fe406e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:31.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.428+0000 7f8fe8a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fe40b49a0 con 0x7f8fe415e9b0 2024-04-20T21:30:31.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.428+0000 7f8fe8a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8fe4160030 con 0x7f8fe40720b0 2024-04-20T21:30:31.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.428+0000 7f8fe8a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8fe406dd20 con 0x7f8fe406e1b0 2024-04-20T21:30:31.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.428+0000 7f8fe3e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8fe40720b0 0x7f8fe406e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:31.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.428+0000 7f8fe3e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8fe40720b0 0x7f8fe406e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53570/0 (socket says 172.21.15.86:53570) 2024-04-20T21:30:31.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.428+0000 7f8fe3e00640 1 -- 172.21.15.86:0/3876820584 learned_addr learned my addr 172.21.15.86:0/3876820584 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:31.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.428+0000 7f8fe2a00640 1 --2- 172.21.15.86:0/3876820584 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fe406e1b0 0x7f8fe4071b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:31.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.428+0000 7f8fe3e00640 1 -- 172.21.15.86:0/3876820584 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fe406e1b0 msgr2=0x7f8fe4071b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:31.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.428+0000 7f8fe3e00640 1 --2- 172.21.15.86:0/3876820584 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fe406e1b0 0x7f8fe4071b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:31.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.428+0000 7f8fe3e00640 1 -- 172.21.15.86:0/3876820584 >> v1:172.21.15.144:6789/0 conn(0x7f8fe415e9b0 legacy=0x7f8fe415f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:31.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.428+0000 7f8fe3e00640 1 -- 172.21.15.86:0/3876820584 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fe40b4c40 con 0x7f8fe40720b0 2024-04-20T21:30:31.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.428+0000 7f8fe2a00640 1 --2- 172.21.15.86:0/3876820584 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fe406e1b0 0x7f8fe4071b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:30:31.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.429+0000 7f8fe3e00640 1 --2- 172.21.15.86:0/3876820584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fe40720b0 0x7f8fe406e700 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc009e50 tx=0x7f8fdc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c510bf408a99fb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:31.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.430+0000 7f8fe2000640 1 -- 172.21.15.86:0/3876820584 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc02ca70 con 0x7f8fe40720b0 2024-04-20T21:30:31.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.430+0000 7f8fe2000640 1 -- 172.21.15.86:0/3876820584 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fdc02cc10 con 0x7f8fe40720b0 2024-04-20T21:30:31.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.430+0000 7f8fe2000640 1 -- 172.21.15.86:0/3876820584 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc0343f0 con 0x7f8fe40720b0 2024-04-20T21:30:31.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.430+0000 7f8fe8a00640 1 -- 172.21.15.86:0/3876820584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fe40720b0 msgr2=0x7f8fe406e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:31.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.430+0000 7f8fe8a00640 1 --2- 172.21.15.86:0/3876820584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fe40720b0 0x7f8fe406e700 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc009e50 tx=0x7f8fdc02bed0 comp rx=0 tx=0).stop 2024-04-20T21:30:31.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.430+0000 7f8fe8a00640 1 -- 172.21.15.86:0/3876820584 shutdown_connections 2024-04-20T21:30:31.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.430+0000 7f8fe8a00640 1 --2- 172.21.15.86:0/3876820584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fe40720b0 0x7f8fe406e700 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:31.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.430+0000 7f8fe8a00640 1 --2- 172.21.15.86:0/3876820584 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fe406e1b0 0x7f8fe4071b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:31.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.431+0000 7f8fe8a00640 1 -- 172.21.15.86:0/3876820584 >> 172.21.15.86:0/3876820584 conn(0x7f8fe406bb60 msgr2=0x7f8fe406bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:31.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.431+0000 7f8fe8a00640 1 -- 172.21.15.86:0/3876820584 shutdown_connections 2024-04-20T21:30:31.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.431+0000 7f8fe8a00640 1 -- 172.21.15.86:0/3876820584 wait complete. 2024-04-20T21:30:31.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.432+0000 7f8fe8a00640 1 Processor -- start 2024-04-20T21:30:31.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.432+0000 7f8fe8a00640 1 -- start start 2024-04-20T21:30:31.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.433+0000 7f8fe8a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fe406e1b0 0x7f8fe40ca750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:31.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.433+0000 7f8fe8a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fe40720b0 0x7f8fe415d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:31.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.433+0000 7f8fe8a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fe415e9b0 0x7f8fe415d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:31.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.433+0000 7f8fe8a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe415f400 con 0x7f8fe406e1b0 2024-04-20T21:30:31.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.433+0000 7f8fe8a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe415f120 con 0x7f8fe415e9b0 2024-04-20T21:30:31.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.433+0000 7f8fe8a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8fe4072540 con 0x7f8fe40720b0 2024-04-20T21:30:31.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.433+0000 7f8fe3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fe406e1b0 0x7f8fe40ca750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:31.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.433+0000 7f8fe3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fe406e1b0 0x7f8fe40ca750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53586/0 (socket says 172.21.15.86:53586) 2024-04-20T21:30:31.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.433+0000 7f8fe3400640 1 -- 172.21.15.86:0/378858259 learned_addr learned my addr 172.21.15.86:0/378858259 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:31.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.434+0000 7f8fe2a00640 1 --2- 172.21.15.86:0/378858259 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fe40720b0 0x7f8fe415d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:31.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.434+0000 7f8fe3400640 1 -- 172.21.15.86:0/378858259 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fe40720b0 msgr2=0x7f8fe415d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:31.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.434+0000 7f8fe3400640 1 --2- 172.21.15.86:0/378858259 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fe40720b0 0x7f8fe415d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:31.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.434+0000 7f8fe3400640 1 -- 172.21.15.86:0/378858259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fe415e9b0 msgr2=0x7f8fe415d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:31.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.434+0000 7f8fe3400640 1 --2- 172.21.15.86:0/378858259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fe415e9b0 0x7f8fe415d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:31.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.434+0000 7f8fe3400640 1 -- 172.21.15.86:0/378858259 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fe40cc470 con 0x7f8fe406e1b0 2024-04-20T21:30:31.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.434+0000 7f8fe2a00640 1 --2- 172.21.15.86:0/378858259 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fe40720b0 0x7f8fe415d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:30:31.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.434+0000 7f8fe3400640 1 --2- 172.21.15.86:0/378858259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fe406e1b0 0x7f8fe40ca750 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f8fd400c430 tx=0x7f8fd400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:31.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.434+0000 7f8fdb400640 1 -- 172.21.15.86:0/378858259 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd4011790 con 0x7f8fe406e1b0 2024-04-20T21:30:31.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.434+0000 7f8fdb400640 1 -- 172.21.15.86:0/378858259 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fd4017070 con 0x7f8fe406e1b0 2024-04-20T21:30:31.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.434+0000 7f8fdb400640 1 -- 172.21.15.86:0/378858259 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd4011bb0 con 0x7f8fe406e1b0 2024-04-20T21:30:31.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.435+0000 7f8fe8a00640 1 -- 172.21.15.86:0/378858259 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fe40cc710 con 0x7f8fe406e1b0 2024-04-20T21:30:31.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.435+0000 7f8fe8a00640 1 -- 172.21.15.86:0/378858259 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fe40cccd0 con 0x7f8fe406e1b0 2024-04-20T21:30:31.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.436+0000 7f8fe8a00640 1 -- 172.21.15.86:0/378858259 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fe404fcf0 con 0x7f8fe406e1b0 2024-04-20T21:30:31.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.437+0000 7f8fdb400640 1 -- 172.21.15.86:0/378858259 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8fd401c460 con 0x7f8fe406e1b0 2024-04-20T21:30:31.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.437+0000 7f8fdb400640 1 --2- 172.21.15.86:0/378858259 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8fc406c8f0 0x7f8fc406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:31.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.437+0000 7f8fdb400640 1 -- 172.21.15.86:0/378858259 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5795+0+0 (secure 0 0 0) 0x7f8fd408bae0 con 0x7f8fe406e1b0 2024-04-20T21:30:31.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.438+0000 7f8fe2a00640 1 --2- 172.21.15.86:0/378858259 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8fc406c8f0 0x7f8fc406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:31.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.438+0000 7f8fe2a00640 1 --2- 172.21.15.86:0/378858259 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8fc406c8f0 0x7f8fc406edb0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0006fd0 tx=0x7f8fd0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:31.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.442+0000 7f8fdb400640 1 -- 172.21.15.86:0/378858259 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8fd4059ea0 con 0x7f8fe406e1b0 2024-04-20T21:30:31.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.652+0000 7f8fe8a00640 1 -- 172.21.15.86:0/378858259 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8fe40ce730 con 0x7f8fe406e1b0 2024-04-20T21:30:31.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.653+0000 7f8fdb400640 1 -- 172.21.15.86:0/378858259 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v113) ==== 80+0+120 (secure 0 0 0) 0x7f8fd40597a0 con 0x7f8fe406e1b0 2024-04-20T21:30:31.653 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6837/289590924 2024-04-21T21:30:19.784403+0000 2024-04-20T21:30:31.653 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6835/289590924 2024-04-21T21:30:19.784403+0000 2024-04-20T21:30:31.653 INFO:teuthology.orchestra.run.smithi086.stderr:listed 2 entries 2024-04-20T21:30:31.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.655+0000 7f8fe8a00640 1 -- 172.21.15.86:0/378858259 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8fc406c8f0 msgr2=0x7f8fc406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:31.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.655+0000 7f8fe8a00640 1 --2- 172.21.15.86:0/378858259 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8fc406c8f0 0x7f8fc406edb0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0006fd0 tx=0x7f8fd0008040 comp rx=0 tx=0).stop 2024-04-20T21:30:31.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.656+0000 7f8fe8a00640 1 -- 172.21.15.86:0/378858259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fe406e1b0 msgr2=0x7f8fe40ca750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:31.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.656+0000 7f8fe8a00640 1 --2- 172.21.15.86:0/378858259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fe406e1b0 0x7f8fe40ca750 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f8fd400c430 tx=0x7f8fd400c8f0 comp rx=0 tx=0).stop 2024-04-20T21:30:31.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.656+0000 7f8fe8a00640 1 -- 172.21.15.86:0/378858259 shutdown_connections 2024-04-20T21:30:31.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.656+0000 7f8fe8a00640 1 --2- 172.21.15.86:0/378858259 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8fc406c8f0 0x7f8fc406edb0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:31.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.656+0000 7f8fe8a00640 1 --2- 172.21.15.86:0/378858259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fe415e9b0 0x7f8fe415d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:31.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.656+0000 7f8fe8a00640 1 --2- 172.21.15.86:0/378858259 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8fe40720b0 0x7f8fe415d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:31.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.656+0000 7f8fe8a00640 1 --2- 172.21.15.86:0/378858259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fe406e1b0 0x7f8fe40ca750 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:31.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.656+0000 7f8fe8a00640 1 -- 172.21.15.86:0/378858259 >> 172.21.15.86:0/378858259 conn(0x7f8fe406bb60 msgr2=0x7f8fe406bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:31.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.656+0000 7f8fe8a00640 1 -- 172.21.15.86:0/378858259 shutdown_connections 2024-04-20T21:30:31.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.656+0000 7f8fe8a00640 1 -- 172.21.15.86:0/378858259 wait complete. 2024-04-20T21:30:31.673 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:30:31.673 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:30:31.719 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:30:31.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.956+0000 7fc053e00640 1 Processor -- start 2024-04-20T21:30:31.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.956+0000 7fc053e00640 1 -- start start 2024-04-20T21:30:31.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.957+0000 7fc053e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc054156ae0 0x7fc05415d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:31.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.957+0000 7fc053e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc054160080 0x7fc0540ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:31.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.957+0000 7fc053e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc054160890 con 0x7fc0540ad620 2024-04-20T21:30:31.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.957+0000 7fc053e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fc05406db20 con 0x7fc054156ae0 2024-04-20T21:30:31.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.957+0000 7fc053e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc054160b70 con 0x7fc054160080 2024-04-20T21:30:31.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.958+0000 7fc04a000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc054160080 0x7fc0540ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:31.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.958+0000 7fc04a000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc054160080 0x7fc0540ad0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:52650/0 (socket says 172.21.15.86:52650) 2024-04-20T21:30:31.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.958+0000 7fc04a000640 1 -- 172.21.15.86:0/2353964844 learned_addr learned my addr 172.21.15.86:0/2353964844 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:31.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.958+0000 7fc052a00640 1 --2- 172.21.15.86:0/2353964844 >> v2:172.21.15.144:3300/0 conn(0x7fc054156ae0 0x7fc05415d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:31.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.959+0000 7fc04a000640 1 -- 172.21.15.86:0/2353964844 >> v2:172.21.15.144:3300/0 conn(0x7fc054156ae0 msgr2=0x7fc05415d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:31.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.959+0000 7fc04a000640 1 --2- 172.21.15.86:0/2353964844 >> v2:172.21.15.144:3300/0 conn(0x7fc054156ae0 0x7fc05415d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:31.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.959+0000 7fc04a000640 1 -- 172.21.15.86:0/2353964844 >> v1:172.21.15.144:6789/0 conn(0x7fc0540ad620 legacy=0x7fc054141cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:31.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.959+0000 7fc04a000640 1 -- 172.21.15.86:0/2353964844 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc054160e50 con 0x7fc054160080 2024-04-20T21:30:31.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.959+0000 7fc052a00640 1 --2- 172.21.15.86:0/2353964844 >> v2:172.21.15.144:3300/0 conn(0x7fc054156ae0 0x7fc05415d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:30:31.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.960+0000 7fc04a000640 1 --2- 172.21.15.86:0/2353964844 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc054160080 0x7fc0540ad0e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc03c009e50 tx=0x7fc03c02bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f02b5f439e5cc2c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:31.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.961+0000 7fc052000640 1 -- 172.21.15.86:0/2353964844 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03c02cb00 con 0x7fc054160080 2024-04-20T21:30:31.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.961+0000 7fc052000640 1 -- 172.21.15.86:0/2353964844 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc03c02cca0 con 0x7fc054160080 2024-04-20T21:30:31.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.961+0000 7fc052000640 1 -- 172.21.15.86:0/2353964844 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03c034610 con 0x7fc054160080 2024-04-20T21:30:31.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.962+0000 7fc053e00640 1 -- 172.21.15.86:0/2353964844 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc054160080 msgr2=0x7fc0540ad0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:31.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.962+0000 7fc053e00640 1 --2- 172.21.15.86:0/2353964844 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc054160080 0x7fc0540ad0e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc03c009e50 tx=0x7fc03c02bf60 comp rx=0 tx=0).stop 2024-04-20T21:30:31.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.962+0000 7fc053e00640 1 -- 172.21.15.86:0/2353964844 shutdown_connections 2024-04-20T21:30:31.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.962+0000 7fc053e00640 1 --2- 172.21.15.86:0/2353964844 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc054160080 0x7fc0540ad0e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:31.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.962+0000 7fc053e00640 1 --2- 172.21.15.86:0/2353964844 >> v2:172.21.15.144:3300/0 conn(0x7fc054156ae0 0x7fc05415d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:31.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.962+0000 7fc053e00640 1 -- 172.21.15.86:0/2353964844 >> 172.21.15.86:0/2353964844 conn(0x7fc0540d0170 msgr2=0x7fc0540d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:31.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.962+0000 7fc053e00640 1 -- 172.21.15.86:0/2353964844 shutdown_connections 2024-04-20T21:30:31.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.962+0000 7fc053e00640 1 -- 172.21.15.86:0/2353964844 wait complete. 2024-04-20T21:30:31.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.964+0000 7fc053e00640 1 Processor -- start 2024-04-20T21:30:31.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.964+0000 7fc053e00640 1 -- start start 2024-04-20T21:30:31.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.965+0000 7fc053e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0540ad620 0x7fc054072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:31.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.965+0000 7fc053e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc054156ae0 0x7fc054154310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:31.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.965+0000 7fc053e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc054160080 0x7fc0541f73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:31.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.965+0000 7fc053e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc054160500 con 0x7fc054156ae0 2024-04-20T21:30:31.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.965+0000 7fc053e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc054156ec0 con 0x7fc054160080 2024-04-20T21:30:31.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.965+0000 7fc053e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc05406f2a0 con 0x7fc0540ad620 2024-04-20T21:30:31.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.965+0000 7fc04a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc054156ae0 0x7fc054154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:31.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.965+0000 7fc04a000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc054156ae0 0x7fc054154310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53590/0 (socket says 172.21.15.86:53590) 2024-04-20T21:30:31.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.965+0000 7fc04a000640 1 -- 172.21.15.86:0/160508482 learned_addr learned my addr 172.21.15.86:0/160508482 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:31.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.966+0000 7fc052a00640 1 --2- 172.21.15.86:0/160508482 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0540ad620 0x7fc054072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:31.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.966+0000 7fc053400640 1 --2- 172.21.15.86:0/160508482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc054160080 0x7fc0541f73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:31.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.966+0000 7fc04a000640 1 -- 172.21.15.86:0/160508482 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0540ad620 msgr2=0x7fc054072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:31.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.966+0000 7fc04a000640 1 --2- 172.21.15.86:0/160508482 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0540ad620 0x7fc054072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:31.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.966+0000 7fc04a000640 1 -- 172.21.15.86:0/160508482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc054160080 msgr2=0x7fc0541f73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:31.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.966+0000 7fc04a000640 1 --2- 172.21.15.86:0/160508482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc054160080 0x7fc0541f73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:31.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.966+0000 7fc04a000640 1 -- 172.21.15.86:0/160508482 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc054153120 con 0x7fc054156ae0 2024-04-20T21:30:31.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.966+0000 7fc04a000640 1 --2- 172.21.15.86:0/160508482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc054156ae0 0x7fc054154310 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7fc03c000f80 tx=0x7fc03c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:31.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.966+0000 7fc04b400640 1 -- 172.21.15.86:0/160508482 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03c043020 con 0x7fc054156ae0 2024-04-20T21:30:31.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.967+0000 7fc04b400640 1 -- 172.21.15.86:0/160508482 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc03c02ce50 con 0x7fc054156ae0 2024-04-20T21:30:31.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.967+0000 7fc04b400640 1 -- 172.21.15.86:0/160508482 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03c009510 con 0x7fc054156ae0 2024-04-20T21:30:31.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.967+0000 7fc053e00640 1 -- 172.21.15.86:0/160508482 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc054153400 con 0x7fc054156ae0 2024-04-20T21:30:31.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.967+0000 7fc053e00640 1 -- 172.21.15.86:0/160508482 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0541575b0 con 0x7fc054156ae0 2024-04-20T21:30:31.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.968+0000 7fc053e00640 1 -- 172.21.15.86:0/160508482 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc05415d6a0 con 0x7fc054156ae0 2024-04-20T21:30:31.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.969+0000 7fc04b400640 1 -- 172.21.15.86:0/160508482 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc03c046020 con 0x7fc054156ae0 2024-04-20T21:30:31.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.970+0000 7fc04b400640 1 --2- 172.21.15.86:0/160508482 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc03006c920 0x7fc03006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:31.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.970+0000 7fc04b400640 1 -- 172.21.15.86:0/160508482 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5795+0+0 (secure 0 0 0) 0x7fc03c0ae210 con 0x7fc054156ae0 2024-04-20T21:30:31.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.970+0000 7fc052a00640 1 --2- 172.21.15.86:0/160508482 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc03006c920 0x7fc03006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:31.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.971+0000 7fc052a00640 1 --2- 172.21.15.86:0/160508482 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc03006c920 0x7fc03006ede0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fc044009560 tx=0x7fc044008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:31.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:31.976+0000 7fc04b400640 1 -- 172.21.15.86:0/160508482 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc05415d6a0 con 0x7fc054156ae0 2024-04-20T21:30:32.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.179+0000 7fc053e00640 1 -- 172.21.15.86:0/160508482 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc05406f8a0 con 0x7fc054156ae0 2024-04-20T21:30:32.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.181+0000 7fc04b400640 1 -- 172.21.15.86:0/160508482 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) ==== 74+0+16754 (secure 0 0 0) 0x7fc05406f8a0 con 0x7fc054156ae0 2024-04-20T21:30:32.181 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:32.181 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":113,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:30:19.784426+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:29:41.532824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-20T21:29:42.749687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6837/289590924":"2024-04-21T21:30:19.784403+0000","172.21.15.86:6835/289590924":"2024-04-21T21:30:19.784403+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:30:32.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.183+0000 7fc053e00640 1 -- 172.21.15.86:0/160508482 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc03006c920 msgr2=0x7fc03006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:32.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.183+0000 7fc053e00640 1 --2- 172.21.15.86:0/160508482 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc03006c920 0x7fc03006ede0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fc044009560 tx=0x7fc044008040 comp rx=0 tx=0).stop 2024-04-20T21:30:32.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.183+0000 7fc053e00640 1 -- 172.21.15.86:0/160508482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc054156ae0 msgr2=0x7fc054154310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:32.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.183+0000 7fc053e00640 1 --2- 172.21.15.86:0/160508482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc054156ae0 0x7fc054154310 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7fc03c000f80 tx=0x7fc03c004a40 comp rx=0 tx=0).stop 2024-04-20T21:30:32.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.183+0000 7fc053e00640 1 -- 172.21.15.86:0/160508482 shutdown_connections 2024-04-20T21:30:32.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.183+0000 7fc053e00640 1 --2- 172.21.15.86:0/160508482 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc03006c920 0x7fc03006ede0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:32.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.183+0000 7fc053e00640 1 --2- 172.21.15.86:0/160508482 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc054160080 0x7fc0541f73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:32.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.183+0000 7fc053e00640 1 --2- 172.21.15.86:0/160508482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc054156ae0 0x7fc054154310 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:32.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.183+0000 7fc053e00640 1 --2- 172.21.15.86:0/160508482 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0540ad620 0x7fc054072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:32.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.183+0000 7fc053e00640 1 -- 172.21.15.86:0/160508482 >> 172.21.15.86:0/160508482 conn(0x7fc0540d0170 msgr2=0x7fc0540b00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:32.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.184+0000 7fc053e00640 1 -- 172.21.15.86:0/160508482 shutdown_connections 2024-04-20T21:30:32.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.184+0000 7fc053e00640 1 -- 172.21.15.86:0/160508482 wait complete. 2024-04-20T21:30:32.202 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:29:41.532824+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:29:42.749687+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:30:32.202 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:30:32.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.407+0000 7f0ab7a00640 1 Processor -- start 2024-04-20T21:30:32.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.407+0000 7f0ab7a00640 1 -- start start 2024-04-20T21:30:32.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.409+0000 7f0ab7a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ab0155d30 0x7f0ab00cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:32.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.409+0000 7f0ab7a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0ab006b980 0x7f0ab006be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:32.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.409+0000 7f0ab7a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ab0156110 con 0x7f0ab00b4ba0 2024-04-20T21:30:32.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.409+0000 7f0ab7a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f0ab00b56e0 con 0x7f0ab006b980 2024-04-20T21:30:32.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.409+0000 7f0ab7a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0ab00ce460 con 0x7f0ab0155d30 2024-04-20T21:30:32.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.410+0000 7f0aa7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0ab006b980 0x7f0ab006be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:32.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.410+0000 7f0aa7e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0ab006b980 0x7f0ab006be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53592/0 (socket says 172.21.15.86:53592) 2024-04-20T21:30:32.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.410+0000 7f0aa7e00640 1 -- 172.21.15.86:0/4230555941 learned_addr learned my addr 172.21.15.86:0/4230555941 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:32.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.410+0000 7f0ab5000640 1 --2- 172.21.15.86:0/4230555941 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ab0155d30 0x7f0ab00cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:32.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.410+0000 7f0aa7e00640 1 -- 172.21.15.86:0/4230555941 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ab0155d30 msgr2=0x7f0ab00cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:32.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.410+0000 7f0aa7e00640 1 --2- 172.21.15.86:0/4230555941 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ab0155d30 0x7f0ab00cf500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:32.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.410+0000 7f0aa7e00640 1 -- 172.21.15.86:0/4230555941 >> v1:172.21.15.144:6789/0 conn(0x7f0ab00b4ba0 legacy=0x7f0ab00b4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:32.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.410+0000 7f0aa7e00640 1 -- 172.21.15.86:0/4230555941 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ab00ce740 con 0x7f0ab006b980 2024-04-20T21:30:32.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.411+0000 7f0aa7e00640 1 --2- 172.21.15.86:0/4230555941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ab006b980 0x7f0ab006be00 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f0a9800bb10 tx=0x7f0a9802dfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e4615dd5d736db4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:32.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.412+0000 7f0aafe00640 1 -- 172.21.15.86:0/4230555941 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a9803b070 con 0x7f0ab006b980 2024-04-20T21:30:32.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.412+0000 7f0aafe00640 1 -- 172.21.15.86:0/4230555941 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a9802ed50 con 0x7f0ab006b980 2024-04-20T21:30:32.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.412+0000 7f0aafe00640 1 -- 172.21.15.86:0/4230555941 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a98036660 con 0x7f0ab006b980 2024-04-20T21:30:32.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.413+0000 7f0ab7a00640 1 -- 172.21.15.86:0/4230555941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ab006b980 msgr2=0x7f0ab006be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:32.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.413+0000 7f0ab7a00640 1 --2- 172.21.15.86:0/4230555941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ab006b980 0x7f0ab006be00 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f0a9800bb10 tx=0x7f0a9802dfa0 comp rx=0 tx=0).stop 2024-04-20T21:30:32.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.413+0000 7f0ab7a00640 1 -- 172.21.15.86:0/4230555941 shutdown_connections 2024-04-20T21:30:32.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.413+0000 7f0ab7a00640 1 --2- 172.21.15.86:0/4230555941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ab006b980 0x7f0ab006be00 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:32.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.413+0000 7f0ab7a00640 1 --2- 172.21.15.86:0/4230555941 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ab0155d30 0x7f0ab00cf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:32.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.413+0000 7f0ab7a00640 1 -- 172.21.15.86:0/4230555941 >> 172.21.15.86:0/4230555941 conn(0x7f0ab00b2110 msgr2=0x7f0ab00b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:32.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.413+0000 7f0ab7a00640 1 -- 172.21.15.86:0/4230555941 shutdown_connections 2024-04-20T21:30:32.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.413+0000 7f0ab7a00640 1 -- 172.21.15.86:0/4230555941 wait complete. 2024-04-20T21:30:32.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.414+0000 7f0ab7a00640 1 Processor -- start 2024-04-20T21:30:32.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.415+0000 7f0ab7a00640 1 -- start start 2024-04-20T21:30:32.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.415+0000 7f0ab7a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ab006b980 0x7f0ab014ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:32.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.415+0000 7f0ab7a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ab00b4ba0 0x7f0ab015a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:32.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.415+0000 7f0ab7a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ab0155d30 0x7f0ab015ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:32.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.416+0000 7f0ab7a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab00cefe0 con 0x7f0ab00b4ba0 2024-04-20T21:30:32.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.416+0000 7f0ab7a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab00ced00 con 0x7f0ab0155d30 2024-04-20T21:30:32.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.416+0000 7f0ab7a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0ab006ddf0 con 0x7f0ab006b980 2024-04-20T21:30:32.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.416+0000 7f0aa7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ab00b4ba0 0x7f0ab015a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:32.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.416+0000 7f0aa7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ab00b4ba0 0x7f0ab015a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53604/0 (socket says 172.21.15.86:53604) 2024-04-20T21:30:32.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.416+0000 7f0aa7e00640 1 -- 172.21.15.86:0/3809060096 learned_addr learned my addr 172.21.15.86:0/3809060096 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:32.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.416+0000 7f0aa7e00640 1 -- 172.21.15.86:0/3809060096 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ab006b980 msgr2=0x7f0ab014ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:32.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.416+0000 7f0ab5000640 1 --2- 172.21.15.86:0/3809060096 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ab006b980 0x7f0ab014ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:32.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.416+0000 7f0ab5a00640 1 --2- 172.21.15.86:0/3809060096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ab0155d30 0x7f0ab015ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:32.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.416+0000 7f0aa7e00640 1 --2- 172.21.15.86:0/3809060096 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ab006b980 0x7f0ab014ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:32.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.416+0000 7f0aa7e00640 1 -- 172.21.15.86:0/3809060096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ab0155d30 msgr2=0x7f0ab015ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:32.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.416+0000 7f0aa7e00640 1 --2- 172.21.15.86:0/3809060096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ab0155d30 0x7f0ab015ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:32.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.416+0000 7f0aa7e00640 1 -- 172.21.15.86:0/3809060096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ab014f530 con 0x7f0ab00b4ba0 2024-04-20T21:30:32.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.416+0000 7f0ab5a00640 1 --2- 172.21.15.86:0/3809060096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ab0155d30 0x7f0ab015ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:30:32.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.417+0000 7f0aa7e00640 1 --2- 172.21.15.86:0/3809060096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ab00b4ba0 0x7f0ab015a730 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f0a9800b2c0 tx=0x7f0a980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:32.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.417+0000 7f0aad600640 1 -- 172.21.15.86:0/3809060096 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a98044070 con 0x7f0ab00b4ba0 2024-04-20T21:30:32.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.417+0000 7f0aad600640 1 -- 172.21.15.86:0/3809060096 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a9803f600 con 0x7f0ab00b4ba0 2024-04-20T21:30:32.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.417+0000 7f0aad600640 1 -- 172.21.15.86:0/3809060096 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a9803b040 con 0x7f0ab00b4ba0 2024-04-20T21:30:32.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.417+0000 7f0ab7a00640 1 -- 172.21.15.86:0/3809060096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ab014da80 con 0x7f0ab00b4ba0 2024-04-20T21:30:32.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.417+0000 7f0ab7a00640 1 -- 172.21.15.86:0/3809060096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ab014f100 con 0x7f0ab00b4ba0 2024-04-20T21:30:32.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.418+0000 7f0ab7a00640 1 -- 172.21.15.86:0/3809060096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ab00689e0 con 0x7f0ab00b4ba0 2024-04-20T21:30:32.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.419+0000 7f0aad600640 1 -- 172.21.15.86:0/3809060096 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0a98047070 con 0x7f0ab00b4ba0 2024-04-20T21:30:32.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.420+0000 7f0aad600640 1 --2- 172.21.15.86:0/3809060096 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0a8c06c940 0x7f0a8c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:32.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.420+0000 7f0aad600640 1 -- 172.21.15.86:0/3809060096 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5795+0+0 (secure 0 0 0) 0x7f0a980ad170 con 0x7f0ab00b4ba0 2024-04-20T21:30:32.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.421+0000 7f0ab5000640 1 --2- 172.21.15.86:0/3809060096 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0a8c06c940 0x7f0a8c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:32.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.421+0000 7f0ab5000640 1 --2- 172.21.15.86:0/3809060096 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0a8c06c940 0x7f0a8c06ee00 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0006fd0 tx=0x7f0aa0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:32.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.425+0000 7f0aad600640 1 -- 172.21.15.86:0/3809060096 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0a9807b5b0 con 0x7f0ab00b4ba0 2024-04-20T21:30:32.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.623+0000 7f0ab7a00640 1 -- 172.21.15.86:0/3809060096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0ab00721c0 con 0x7f0ab00b4ba0 2024-04-20T21:30:32.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.624+0000 7f0aad600640 1 -- 172.21.15.86:0/3809060096 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) ==== 93+0+10 (secure 0 0 0) 0x7f0a9807aeb0 con 0x7f0ab00b4ba0 2024-04-20T21:30:32.624 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:30:32.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.627+0000 7f0ab7a00640 1 -- 172.21.15.86:0/3809060096 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0a8c06c940 msgr2=0x7f0a8c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:32.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.627+0000 7f0ab7a00640 1 --2- 172.21.15.86:0/3809060096 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0a8c06c940 0x7f0a8c06ee00 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0006fd0 tx=0x7f0aa0008040 comp rx=0 tx=0).stop 2024-04-20T21:30:32.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.627+0000 7f0ab7a00640 1 -- 172.21.15.86:0/3809060096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ab00b4ba0 msgr2=0x7f0ab015a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:32.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.627+0000 7f0ab7a00640 1 --2- 172.21.15.86:0/3809060096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ab00b4ba0 0x7f0ab015a730 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f0a9800b2c0 tx=0x7f0a980046c0 comp rx=0 tx=0).stop 2024-04-20T21:30:32.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.627+0000 7f0ab7a00640 1 -- 172.21.15.86:0/3809060096 shutdown_connections 2024-04-20T21:30:32.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.627+0000 7f0ab7a00640 1 --2- 172.21.15.86:0/3809060096 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0a8c06c940 0x7f0a8c06ee00 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:32.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.627+0000 7f0ab7a00640 1 --2- 172.21.15.86:0/3809060096 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ab0155d30 0x7f0ab015ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:32.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.627+0000 7f0ab7a00640 1 --2- 172.21.15.86:0/3809060096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ab00b4ba0 0x7f0ab015a730 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:32.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.627+0000 7f0ab7a00640 1 --2- 172.21.15.86:0/3809060096 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0ab006b980 0x7f0ab014ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:32.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.627+0000 7f0ab7a00640 1 -- 172.21.15.86:0/3809060096 >> 172.21.15.86:0/3809060096 conn(0x7f0ab00b2110 msgr2=0x7f0ab00726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:32.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.627+0000 7f0ab7a00640 1 -- 172.21.15.86:0/3809060096 shutdown_connections 2024-04-20T21:30:32.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.627+0000 7f0ab7a00640 1 -- 172.21.15.86:0/3809060096 wait complete. 2024-04-20T21:30:32.647 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:30:32.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.850+0000 7fe5a3e00640 1 Processor -- start 2024-04-20T21:30:32.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.850+0000 7fe5a3e00640 1 -- start start 2024-04-20T21:30:32.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.852+0000 7fe5a3e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5a409b5a0 0x7fe5a40d0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:32.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.852+0000 7fe5a3e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe5a41625b0 0x7fe5a415eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:32.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.852+0000 7fe5a3e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5a40d0f70 con 0x7fe5a409a9e0 2024-04-20T21:30:32.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.852+0000 7fe5a3e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fe5a40d2010 con 0x7fe5a41625b0 2024-04-20T21:30:32.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.852+0000 7fe5a3e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe5a40d1250 con 0x7fe5a409b5a0 2024-04-20T21:30:32.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.853+0000 7fe5a2a00640 1 --1- >> v1:172.21.15.144:6789/0 conn(0x7fe5a409a9e0 0x7fe5a409adc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.144:6789/0 says I am v1:172.21.15.86:35946/0 (socket says 172.21.15.86:35946) 2024-04-20T21:30:32.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.853+0000 7fe5a2a00640 1 -- 172.21.15.86:0/4219408583 learned_addr learned my addr 172.21.15.86:0/4219408583 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:30:32.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.853+0000 7fe5a3400640 1 --2- 172.21.15.86:0/4219408583 >> v2:172.21.15.144:3300/0 conn(0x7fe5a41625b0 0x7fe5a415eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:32.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.853+0000 7fe5a2000640 1 --2- 172.21.15.86:0/4219408583 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5a409b5a0 0x7fe5a40d0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:32.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.854+0000 7fe5a1600640 1 -- 172.21.15.86:0/4219408583 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1417895862 0 0) 0x7fe5a40d0f70 con 0x7fe5a409a9e0 2024-04-20T21:30:32.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.854+0000 7fe5a1600640 1 -- 172.21.15.86:0/4219408583 --> v1:172.21.15.144:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe588003520 con 0x7fe5a409a9e0 2024-04-20T21:30:32.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.854+0000 7fe5a3400640 1 -- 172.21.15.86:0/4219408583 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5a409b5a0 msgr2=0x7fe5a40d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:32.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.854+0000 7fe5a3400640 1 --2- 172.21.15.86:0/4219408583 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5a409b5a0 0x7fe5a40d0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:32.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.854+0000 7fe5a3400640 1 -- 172.21.15.86:0/4219408583 >> v1:172.21.15.144:6789/0 conn(0x7fe5a409a9e0 legacy=0x7fe5a409adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:32.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.854+0000 7fe5a3400640 1 -- 172.21.15.86:0/4219408583 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5a40d1530 con 0x7fe5a41625b0 2024-04-20T21:30:32.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.854+0000 7fe5a2000640 1 --2- 172.21.15.86:0/4219408583 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5a409b5a0 0x7fe5a40d0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:30:32.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.855+0000 7fe5a3400640 1 --2- 172.21.15.86:0/4219408583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5a41625b0 0x7fe5a415eca0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe59c009d90 tx=0x7fe59c02be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb2155b3868163b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:32.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.856+0000 7fe5a1600640 1 -- 172.21.15.86:0/4219408583 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c039070 con 0x7fe5a41625b0 2024-04-20T21:30:32.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.856+0000 7fe5a1600640 1 -- 172.21.15.86:0/4219408583 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe59c02cbc0 con 0x7fe5a41625b0 2024-04-20T21:30:32.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.856+0000 7fe5a1600640 1 -- 172.21.15.86:0/4219408583 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c034440 con 0x7fe5a41625b0 2024-04-20T21:30:32.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.856+0000 7fe5a3e00640 1 -- 172.21.15.86:0/4219408583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5a41625b0 msgr2=0x7fe5a415eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:32.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.856+0000 7fe5a3e00640 1 --2- 172.21.15.86:0/4219408583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5a41625b0 0x7fe5a415eca0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe59c009d90 tx=0x7fe59c02be10 comp rx=0 tx=0).stop 2024-04-20T21:30:32.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.856+0000 7fe5a3e00640 1 -- 172.21.15.86:0/4219408583 shutdown_connections 2024-04-20T21:30:32.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.856+0000 7fe5a3e00640 1 --2- 172.21.15.86:0/4219408583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5a41625b0 0x7fe5a415eca0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:32.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.856+0000 7fe5a3e00640 1 --2- 172.21.15.86:0/4219408583 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5a409b5a0 0x7fe5a40d0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:32.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.857+0000 7fe5a3e00640 1 -- 172.21.15.86:0/4219408583 >> 172.21.15.86:0/4219408583 conn(0x7fe5a40b4010 msgr2=0x7fe5a40b4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:32.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.857+0000 7fe5a3e00640 1 -- 172.21.15.86:0/4219408583 shutdown_connections 2024-04-20T21:30:32.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.857+0000 7fe5a3e00640 1 -- 172.21.15.86:0/4219408583 wait complete. 2024-04-20T21:30:32.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.858+0000 7fe5a3e00640 1 Processor -- start 2024-04-20T21:30:32.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.859+0000 7fe5a3e00640 1 -- start start 2024-04-20T21:30:32.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.860+0000 7fe5a3e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5a409a9e0 0x7fe5a40af2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:32.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.860+0000 7fe5a3e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5a409b5a0 0x7fe5a41f3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:32.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.860+0000 7fe5a3e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5a41625b0 0x7fe5a40c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:32.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.860+0000 7fe5a3e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a406daa0 con 0x7fe5a41625b0 2024-04-20T21:30:32.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.860+0000 7fe5a3e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a40d1af0 con 0x7fe5a409a9e0 2024-04-20T21:30:32.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.860+0000 7fe5a3e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe5a406dc20 con 0x7fe5a409b5a0 2024-04-20T21:30:32.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.860+0000 7fe5a3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5a41625b0 0x7fe5a40c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:32.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.860+0000 7fe5a3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5a41625b0 0x7fe5a40c66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53612/0 (socket says 172.21.15.86:53612) 2024-04-20T21:30:32.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.860+0000 7fe5a3400640 1 -- 172.21.15.86:0/1161716383 learned_addr learned my addr 172.21.15.86:0/1161716383 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:32.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.860+0000 7fe5a2a00640 1 --2- 172.21.15.86:0/1161716383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5a409a9e0 0x7fe5a40af2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:32.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.860+0000 7fe5a2000640 1 --2- 172.21.15.86:0/1161716383 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5a409b5a0 0x7fe5a41f3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:32.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.860+0000 7fe5a3400640 1 -- 172.21.15.86:0/1161716383 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5a409b5a0 msgr2=0x7fe5a41f3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:32.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.860+0000 7fe5a3400640 1 --2- 172.21.15.86:0/1161716383 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5a409b5a0 0x7fe5a41f3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:32.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.860+0000 7fe5a3400640 1 -- 172.21.15.86:0/1161716383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5a409a9e0 msgr2=0x7fe5a40af2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:32.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.860+0000 7fe5a3400640 1 --2- 172.21.15.86:0/1161716383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5a409a9e0 0x7fe5a40af2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:32.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.860+0000 7fe5a3400640 1 -- 172.21.15.86:0/1161716383 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5a40afb20 con 0x7fe5a41625b0 2024-04-20T21:30:32.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.860+0000 7fe5a2a00640 1 --2- 172.21.15.86:0/1161716383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5a409a9e0 0x7fe5a40af2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:30:32.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.860+0000 7fe5a2000640 1 --2- 172.21.15.86:0/1161716383 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5a409b5a0 0x7fe5a41f3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:30:32.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.861+0000 7fe5a3400640 1 --2- 172.21.15.86:0/1161716383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5a41625b0 0x7fe5a40c66f0 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7fe59c037040 tx=0x7fe59c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:32.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.861+0000 7fe59aa00640 1 -- 172.21.15.86:0/1161716383 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c039050 con 0x7fe5a41625b0 2024-04-20T21:30:32.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.861+0000 7fe59aa00640 1 -- 172.21.15.86:0/1161716383 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe59c03a040 con 0x7fe5a41625b0 2024-04-20T21:30:32.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.861+0000 7fe59aa00640 1 -- 172.21.15.86:0/1161716383 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c009450 con 0x7fe5a41625b0 2024-04-20T21:30:32.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.861+0000 7fe5a3e00640 1 -- 172.21.15.86:0/1161716383 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5a40afe00 con 0x7fe5a41625b0 2024-04-20T21:30:32.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.861+0000 7fe5a3e00640 1 -- 172.21.15.86:0/1161716383 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5a40c61d0 con 0x7fe5a41625b0 2024-04-20T21:30:32.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.863+0000 7fe5a3e00640 1 -- 172.21.15.86:0/1161716383 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5a404fd90 con 0x7fe5a41625b0 2024-04-20T21:30:32.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.863+0000 7fe59aa00640 1 -- 172.21.15.86:0/1161716383 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe59c047020 con 0x7fe5a41625b0 2024-04-20T21:30:32.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.864+0000 7fe59aa00640 1 --2- 172.21.15.86:0/1161716383 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe58406c8f0 0x7fe58406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:32.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.864+0000 7fe59aa00640 1 -- 172.21.15.86:0/1161716383 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5795+0+0 (secure 0 0 0) 0x7fe59c0adb20 con 0x7fe5a41625b0 2024-04-20T21:30:32.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.865+0000 7fe5a2a00640 1 --2- 172.21.15.86:0/1161716383 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe58406c8f0 0x7fe58406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:32.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.865+0000 7fe5a2a00640 1 --2- 172.21.15.86:0/1161716383 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe58406c8f0 0x7fe58406edb0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fe594005880 tx=0x7fe594009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:32.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:32.869+0000 7fe59aa00640 1 -- 172.21.15.86:0/1161716383 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe59c04e020 con 0x7fe5a41625b0 2024-04-20T21:30:33.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.069+0000 7fe5a3e00640 1 -- 172.21.15.86:0/1161716383 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe5a40b84a0 con 0x7fe5a41625b0 2024-04-20T21:30:33.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.070+0000 7fe59aa00640 1 -- 172.21.15.86:0/1161716383 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v113) ==== 104+0+11 (secure 0 0 0) 0x7fe59c07bfa0 con 0x7fe5a41625b0 2024-04-20T21:30:33.069 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-20T21:30:33.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.072+0000 7fe5a3e00640 1 -- 172.21.15.86:0/1161716383 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe58406c8f0 msgr2=0x7fe58406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:33.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.072+0000 7fe5a3e00640 1 --2- 172.21.15.86:0/1161716383 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe58406c8f0 0x7fe58406edb0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fe594005880 tx=0x7fe594009040 comp rx=0 tx=0).stop 2024-04-20T21:30:33.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.072+0000 7fe5a3e00640 1 -- 172.21.15.86:0/1161716383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5a41625b0 msgr2=0x7fe5a40c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:33.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.072+0000 7fe5a3e00640 1 --2- 172.21.15.86:0/1161716383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5a41625b0 0x7fe5a40c66f0 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7fe59c037040 tx=0x7fe59c004a40 comp rx=0 tx=0).stop 2024-04-20T21:30:33.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.073+0000 7fe5a3e00640 1 -- 172.21.15.86:0/1161716383 shutdown_connections 2024-04-20T21:30:33.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.073+0000 7fe5a3e00640 1 --2- 172.21.15.86:0/1161716383 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe58406c8f0 0x7fe58406edb0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:33.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.073+0000 7fe5a3e00640 1 --2- 172.21.15.86:0/1161716383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5a41625b0 0x7fe5a40c66f0 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:33.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.073+0000 7fe5a3e00640 1 --2- 172.21.15.86:0/1161716383 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe5a409b5a0 0x7fe5a41f3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:33.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.073+0000 7fe5a3e00640 1 --2- 172.21.15.86:0/1161716383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5a409a9e0 0x7fe5a40af2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:33.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.073+0000 7fe5a3e00640 1 -- 172.21.15.86:0/1161716383 >> 172.21.15.86:0/1161716383 conn(0x7fe5a40b4010 msgr2=0x7fe5a40ceda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:33.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.073+0000 7fe5a3e00640 1 -- 172.21.15.86:0/1161716383 shutdown_connections 2024-04-20T21:30:33.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.073+0000 7fe5a3e00640 1 -- 172.21.15.86:0/1161716383 wait complete. 2024-04-20T21:30:33.092 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:30:33.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.288+0000 7fdec4400640 1 Processor -- start 2024-04-20T21:30:33.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.288+0000 7fdec4400640 1 -- start start 2024-04-20T21:30:33.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.290+0000 7fdec4400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fdebc158950 0x7fdebc071db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:33.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.290+0000 7fdec4400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdebc0722f0 0x7fdebc0b3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:33.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.290+0000 7fdec4400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdebc0b4400 0x7fdebc15ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:33.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.290+0000 7fdec4400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fdebc0b4bf0 con 0x7fdebc0722f0 2024-04-20T21:30:33.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.290+0000 7fdec4400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fdebc0b4ed0 con 0x7fdebc158950 2024-04-20T21:30:33.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.290+0000 7fdec4400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdebc06de90 con 0x7fdebc0b4400 2024-04-20T21:30:33.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.290+0000 7fdec1000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdebc0722f0 0x7fdebc0b3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:33.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.290+0000 7fdec1000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdebc0722f0 0x7fdebc0b3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53620/0 (socket says 172.21.15.86:53620) 2024-04-20T21:30:33.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.290+0000 7fdec1000640 1 -- 172.21.15.86:0/929846331 learned_addr learned my addr 172.21.15.86:0/929846331 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:33.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.290+0000 7fdec1a00640 1 --2- 172.21.15.86:0/929846331 >> v2:172.21.15.144:3300/0 conn(0x7fdebc158950 0x7fdebc071db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:33.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.291+0000 7fdec1000640 1 -- 172.21.15.86:0/929846331 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdebc0b4400 msgr2=0x7fdebc15ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:33.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.291+0000 7fdec1000640 1 --2- 172.21.15.86:0/929846331 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdebc0b4400 0x7fdebc15ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:33.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.291+0000 7fdec1000640 1 -- 172.21.15.86:0/929846331 >> v2:172.21.15.144:3300/0 conn(0x7fdebc158950 msgr2=0x7fdebc071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:33.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.291+0000 7fdec1000640 1 --2- 172.21.15.86:0/929846331 >> v2:172.21.15.144:3300/0 conn(0x7fdebc158950 0x7fdebc071db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:33.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.291+0000 7fdec1000640 1 -- 172.21.15.86:0/929846331 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdebc06e170 con 0x7fdebc0722f0 2024-04-20T21:30:33.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.291+0000 7fdec1a00640 1 --2- 172.21.15.86:0/929846331 >> v2:172.21.15.144:3300/0 conn(0x7fdebc158950 0x7fdebc071db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:30:33.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.292+0000 7fdec1000640 1 --2- 172.21.15.86:0/929846331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdebc0722f0 0x7fdebc0b3ec0 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7fdebc06fed0 tx=0x7fdea802b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff41aa65633109ce server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:33.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.292+0000 7fdebbe00640 1 -- 172.21.15.86:0/929846331 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea8035020 con 0x7fdebc0722f0 2024-04-20T21:30:33.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.292+0000 7fdebbe00640 1 -- 172.21.15.86:0/929846331 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdea8031880 con 0x7fdebc0722f0 2024-04-20T21:30:33.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.292+0000 7fdebbe00640 1 -- 172.21.15.86:0/929846331 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea8031c50 con 0x7fdebc0722f0 2024-04-20T21:30:33.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.293+0000 7fdec4400640 1 -- 172.21.15.86:0/929846331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdebc0722f0 msgr2=0x7fdebc0b3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:33.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.293+0000 7fdec4400640 1 --2- 172.21.15.86:0/929846331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdebc0722f0 0x7fdebc0b3ec0 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7fdebc06fed0 tx=0x7fdea802b920 comp rx=0 tx=0).stop 2024-04-20T21:30:33.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.293+0000 7fdec4400640 1 -- 172.21.15.86:0/929846331 shutdown_connections 2024-04-20T21:30:33.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.293+0000 7fdec4400640 1 --2- 172.21.15.86:0/929846331 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdebc0b4400 0x7fdebc15ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:33.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.293+0000 7fdec4400640 1 --2- 172.21.15.86:0/929846331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdebc0722f0 0x7fdebc0b3ec0 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:33.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.293+0000 7fdec4400640 1 --2- 172.21.15.86:0/929846331 >> v2:172.21.15.144:3300/0 conn(0x7fdebc158950 0x7fdebc071db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:33.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.293+0000 7fdec4400640 1 -- 172.21.15.86:0/929846331 >> 172.21.15.86:0/929846331 conn(0x7fdebc0b2110 msgr2=0x7fdebc0b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:33.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.293+0000 7fdec4400640 1 -- 172.21.15.86:0/929846331 shutdown_connections 2024-04-20T21:30:33.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.293+0000 7fdec4400640 1 -- 172.21.15.86:0/929846331 wait complete. 2024-04-20T21:30:33.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.295+0000 7fdec4400640 1 Processor -- start 2024-04-20T21:30:33.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.295+0000 7fdec4400640 1 -- start start 2024-04-20T21:30:33.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.297+0000 7fdec4400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdebc0722f0 0x7fdebc14eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:33.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.297+0000 7fdec4400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdebc0b4400 0x7fdebc0ae260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:33.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.297+0000 7fdec4400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdebc158950 0x7fdebc150890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:33.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.297+0000 7fdec4400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc0b4840 con 0x7fdebc0722f0 2024-04-20T21:30:33.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.297+0000 7fdec4400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc158d30 con 0x7fdebc0b4400 2024-04-20T21:30:33.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.297+0000 7fdec4400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdebc0ae7a0 con 0x7fdebc158950 2024-04-20T21:30:33.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.297+0000 7fdec1a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdebc0722f0 0x7fdebc14eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:33.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.297+0000 7fdec1a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdebc0722f0 0x7fdebc14eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53622/0 (socket says 172.21.15.86:53622) 2024-04-20T21:30:33.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.297+0000 7fdec1a00640 1 -- 172.21.15.86:0/266871825 learned_addr learned my addr 172.21.15.86:0/266871825 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:33.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.297+0000 7fdec1000640 1 --2- 172.21.15.86:0/266871825 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdebc0b4400 0x7fdebc0ae260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:33.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.297+0000 7fdec2400640 1 --2- 172.21.15.86:0/266871825 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdebc158950 0x7fdebc150890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:33.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.297+0000 7fdec1a00640 1 -- 172.21.15.86:0/266871825 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdebc158950 msgr2=0x7fdebc150890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:33.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.297+0000 7fdec1a00640 1 --2- 172.21.15.86:0/266871825 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdebc158950 0x7fdebc150890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:33.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.297+0000 7fdec1a00640 1 -- 172.21.15.86:0/266871825 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdebc0b4400 msgr2=0x7fdebc0ae260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:33.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.297+0000 7fdec1a00640 1 --2- 172.21.15.86:0/266871825 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdebc0b4400 0x7fdebc0ae260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:33.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.297+0000 7fdec1a00640 1 -- 172.21.15.86:0/266871825 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdebc0cde10 con 0x7fdebc0722f0 2024-04-20T21:30:33.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.297+0000 7fdec1000640 1 --2- 172.21.15.86:0/266871825 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdebc0b4400 0x7fdebc0ae260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:30:33.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.298+0000 7fdec1a00640 1 --2- 172.21.15.86:0/266871825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdebc0722f0 0x7fdebc14eaf0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7fdeb000c7f0 tx=0x7fdeb000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:33.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.298+0000 7fdeb9600640 1 -- 172.21.15.86:0/266871825 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb0015070 con 0x7fdebc0722f0 2024-04-20T21:30:33.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.298+0000 7fdeb9600640 1 -- 172.21.15.86:0/266871825 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdeb00077c0 con 0x7fdebc0722f0 2024-04-20T21:30:33.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.298+0000 7fdeb9600640 1 -- 172.21.15.86:0/266871825 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb0007b70 con 0x7fdebc0722f0 2024-04-20T21:30:33.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.298+0000 7fdec4400640 1 -- 172.21.15.86:0/266871825 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdebc0ce0f0 con 0x7fdebc0722f0 2024-04-20T21:30:33.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.298+0000 7fdec4400640 1 -- 172.21.15.86:0/266871825 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdebc0ce6b0 con 0x7fdebc0722f0 2024-04-20T21:30:33.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.299+0000 7fdec4400640 1 -- 172.21.15.86:0/266871825 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdebc068250 con 0x7fdebc0722f0 2024-04-20T21:30:33.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.300+0000 7fdeb9600640 1 -- 172.21.15.86:0/266871825 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdeb0018070 con 0x7fdebc0722f0 2024-04-20T21:30:33.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.301+0000 7fdeb9600640 1 --2- 172.21.15.86:0/266871825 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fde9c06c8f0 0x7fde9c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:33.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.301+0000 7fdeb9600640 1 -- 172.21.15.86:0/266871825 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5795+0+0 (secure 0 0 0) 0x7fdeb008bd30 con 0x7fdebc0722f0 2024-04-20T21:30:33.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.302+0000 7fdec1000640 1 --2- 172.21.15.86:0/266871825 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fde9c06c8f0 0x7fde9c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:33.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.302+0000 7fdec1000640 1 --2- 172.21.15.86:0/266871825 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fde9c06c8f0 0x7fde9c06edb0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fdea8036000 tx=0x7fdea8008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:33.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.306+0000 7fdeb9600640 1 -- 172.21.15.86:0/266871825 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fdebc068250 con 0x7fdebc0722f0 2024-04-20T21:30:33.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.503+0000 7fdec4400640 1 -- 172.21.15.86:0/266871825 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdebc0689e0 con 0x7fdebc0722f0 2024-04-20T21:30:33.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.504+0000 7fdeb9600640 1 -- 172.21.15.86:0/266871825 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v113) ==== 100+0+11 (secure 0 0 0) 0x7fdeb005a060 con 0x7fdebc0722f0 2024-04-20T21:30:33.504 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-20T21:30:33.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.507+0000 7fdec4400640 1 -- 172.21.15.86:0/266871825 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fde9c06c8f0 msgr2=0x7fde9c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:33.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.507+0000 7fdec4400640 1 --2- 172.21.15.86:0/266871825 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fde9c06c8f0 0x7fde9c06edb0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fdea8036000 tx=0x7fdea8008030 comp rx=0 tx=0).stop 2024-04-20T21:30:33.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.507+0000 7fdec4400640 1 -- 172.21.15.86:0/266871825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdebc0722f0 msgr2=0x7fdebc14eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:33.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.507+0000 7fdec4400640 1 --2- 172.21.15.86:0/266871825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdebc0722f0 0x7fdebc14eaf0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7fdeb000c7f0 tx=0x7fdeb000ccb0 comp rx=0 tx=0).stop 2024-04-20T21:30:33.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.507+0000 7fdec4400640 1 -- 172.21.15.86:0/266871825 shutdown_connections 2024-04-20T21:30:33.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.507+0000 7fdec4400640 1 --2- 172.21.15.86:0/266871825 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fde9c06c8f0 0x7fde9c06edb0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:33.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.507+0000 7fdec4400640 1 --2- 172.21.15.86:0/266871825 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdebc158950 0x7fdebc150890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:33.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.507+0000 7fdec4400640 1 --2- 172.21.15.86:0/266871825 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdebc0b4400 0x7fdebc0ae260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:33.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.507+0000 7fdec4400640 1 --2- 172.21.15.86:0/266871825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdebc0722f0 0x7fdebc14eaf0 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:33.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.507+0000 7fdec4400640 1 -- 172.21.15.86:0/266871825 >> 172.21.15.86:0/266871825 conn(0x7fdebc0b2110 msgr2=0x7fdebc15c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:33.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.507+0000 7fdec4400640 1 -- 172.21.15.86:0/266871825 shutdown_connections 2024-04-20T21:30:33.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.507+0000 7fdec4400640 1 -- 172.21.15.86:0/266871825 wait complete. 2024-04-20T21:30:33.523 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T21:30:33.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.714+0000 7f286ee00640 1 Processor -- start 2024-04-20T21:30:33.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.714+0000 7f286ee00640 1 -- start start 2024-04-20T21:30:33.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.716+0000 7f286ee00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f286815e930 0x7f28680d3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:33.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.716+0000 7f286ee00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28680cfda0 0x7f28680d0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:33.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.716+0000 7f286ee00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28680d2540 con 0x7f28680d0200 2024-04-20T21:30:33.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.716+0000 7f286ee00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f28680d22f0 con 0x7f286815e930 2024-04-20T21:30:33.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.716+0000 7f286ee00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f28680d2820 con 0x7f28680cfda0 2024-04-20T21:30:33.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.716+0000 7f2867e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f286815e930 0x7f28680d3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:33.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.716+0000 7f2867e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f286815e930 0x7f28680d3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53626/0 (socket says 172.21.15.86:53626) 2024-04-20T21:30:33.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.717+0000 7f2867e00640 1 -- 172.21.15.86:0/4004169919 learned_addr learned my addr 172.21.15.86:0/4004169919 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:33.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.716+0000 7f286ce00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f28680d0200 0x7f28680b7570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:32894/0 (socket says 172.21.15.86:32894) 2024-04-20T21:30:33.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.717+0000 7f2867e00640 1 -- 172.21.15.86:0/4004169919 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28680cfda0 msgr2=0x7f28680d0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:33.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.717+0000 7f2867e00640 1 --2- 172.21.15.86:0/4004169919 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28680cfda0 0x7f28680d0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:33.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.717+0000 7f2867e00640 1 -- 172.21.15.86:0/4004169919 >> v1:172.21.15.86:6789/0 conn(0x7f28680d0200 legacy=0x7f28680b7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:33.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.717+0000 7f2867e00640 1 -- 172.21.15.86:0/4004169919 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28680d2b00 con 0x7f286815e930 2024-04-20T21:30:33.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.718+0000 7f2867e00640 1 --2- 172.21.15.86:0/4004169919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f286815e930 0x7f28680d3300 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f2858009870 tx=0x7f285802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d31aa289a6e2c146 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:33.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.718+0000 7f2867400640 1 -- 172.21.15.86:0/4004169919 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2858035020 con 0x7f286815e930 2024-04-20T21:30:33.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.718+0000 7f2867400640 1 -- 172.21.15.86:0/4004169919 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2858031640 con 0x7f286815e930 2024-04-20T21:30:33.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.718+0000 7f2867400640 1 -- 172.21.15.86:0/4004169919 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285802d040 con 0x7f286815e930 2024-04-20T21:30:33.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.719+0000 7f286ee00640 1 -- 172.21.15.86:0/4004169919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f286815e930 msgr2=0x7f28680d3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:33.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.719+0000 7f286ee00640 1 --2- 172.21.15.86:0/4004169919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f286815e930 0x7f28680d3300 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f2858009870 tx=0x7f285802b8f0 comp rx=0 tx=0).stop 2024-04-20T21:30:33.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.719+0000 7f286ee00640 1 -- 172.21.15.86:0/4004169919 shutdown_connections 2024-04-20T21:30:33.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.719+0000 7f286ee00640 1 --2- 172.21.15.86:0/4004169919 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28680cfda0 0x7f28680d0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:33.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.719+0000 7f286ee00640 1 --2- 172.21.15.86:0/4004169919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f286815e930 0x7f28680d3300 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:33.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.719+0000 7f286ee00640 1 -- 172.21.15.86:0/4004169919 >> 172.21.15.86:0/4004169919 conn(0x7f286808bf70 msgr2=0x7f28680d4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:33.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.719+0000 7f286ee00640 1 -- 172.21.15.86:0/4004169919 shutdown_connections 2024-04-20T21:30:33.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.719+0000 7f286ee00640 1 -- 172.21.15.86:0/4004169919 wait complete. 2024-04-20T21:30:33.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.721+0000 7f286ee00640 1 Processor -- start 2024-04-20T21:30:33.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.721+0000 7f286ee00640 1 -- start start 2024-04-20T21:30:33.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.722+0000 7f286ee00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28680cfda0 0x7f28680c8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:33.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.722+0000 7f286ee00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28680d0200 0x7f28680cdf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:33.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.722+0000 7f286ee00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f286815e930 0x7f28680cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:33.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.722+0000 7f286ee00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f28680d4ff0 con 0x7f28680d0200 2024-04-20T21:30:33.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.722+0000 7f286ee00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f286815ed10 con 0x7f286815e930 2024-04-20T21:30:33.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.722+0000 7f286ee00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f28680d5170 con 0x7f28680cfda0 2024-04-20T21:30:33.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.722+0000 7f285fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28680d0200 0x7f28680cdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:33.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.723+0000 7f285fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28680d0200 0x7f28680cdf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53628/0 (socket says 172.21.15.86:53628) 2024-04-20T21:30:33.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.723+0000 7f285fe00640 1 -- 172.21.15.86:0/458706541 learned_addr learned my addr 172.21.15.86:0/458706541 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:33.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.723+0000 7f2867e00640 1 --2- 172.21.15.86:0/458706541 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28680cfda0 0x7f28680c8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:33.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.723+0000 7f286ce00640 1 --2- 172.21.15.86:0/458706541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f286815e930 0x7f28680cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:33.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.723+0000 7f285fe00640 1 -- 172.21.15.86:0/458706541 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28680cfda0 msgr2=0x7f28680c8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:33.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.723+0000 7f285fe00640 1 --2- 172.21.15.86:0/458706541 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28680cfda0 0x7f28680c8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:33.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.723+0000 7f285fe00640 1 -- 172.21.15.86:0/458706541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f286815e930 msgr2=0x7f28680cb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:33.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.723+0000 7f285fe00640 1 --2- 172.21.15.86:0/458706541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f286815e930 0x7f28680cb7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:33.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.723+0000 7f285fe00640 1 -- 172.21.15.86:0/458706541 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28680c9e30 con 0x7f28680d0200 2024-04-20T21:30:33.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.723+0000 7f285fe00640 1 --2- 172.21.15.86:0/458706541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28680d0200 0x7f28680cdf70 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f285000b410 tx=0x7f285000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:33.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.724+0000 7f2864c00640 1 -- 172.21.15.86:0/458706541 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2850015020 con 0x7f28680d0200 2024-04-20T21:30:33.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.724+0000 7f2864c00640 1 -- 172.21.15.86:0/458706541 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2850011850 con 0x7f28680d0200 2024-04-20T21:30:33.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.724+0000 7f2864c00640 1 -- 172.21.15.86:0/458706541 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2850011c40 con 0x7f28680d0200 2024-04-20T21:30:33.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.724+0000 7f286ee00640 1 -- 172.21.15.86:0/458706541 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28680c8d90 con 0x7f28680d0200 2024-04-20T21:30:33.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.724+0000 7f286ee00640 1 -- 172.21.15.86:0/458706541 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28680c9350 con 0x7f28680d0200 2024-04-20T21:30:33.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.725+0000 7f286ee00640 1 -- 172.21.15.86:0/458706541 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28680c9b10 con 0x7f28680d0200 2024-04-20T21:30:33.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.726+0000 7f2864c00640 1 -- 172.21.15.86:0/458706541 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2850018020 con 0x7f28680d0200 2024-04-20T21:30:33.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.727+0000 7f2864c00640 1 --2- 172.21.15.86:0/458706541 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f284406c8f0 0x7f284406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:33.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.727+0000 7f2864c00640 1 -- 172.21.15.86:0/458706541 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5795+0+0 (secure 0 0 0) 0x7f285008bb70 con 0x7f28680d0200 2024-04-20T21:30:33.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.727+0000 7f2867e00640 1 --2- 172.21.15.86:0/458706541 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f284406c8f0 0x7f284406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:33.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.728+0000 7f2867e00640 1 --2- 172.21.15.86:0/458706541 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f284406c8f0 0x7f284406edb0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f2858036000 tx=0x7f285803c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:33.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.732+0000 7f2864c00640 1 -- 172.21.15.86:0/458706541 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f28680c9b10 con 0x7f28680d0200 2024-04-20T21:30:33.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.943+0000 7f286ee00640 1 -- 172.21.15.86:0/458706541 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f286808e810 con 0x7f28680d0200 2024-04-20T21:30:33.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.944+0000 7f2864c00640 1 -- 172.21.15.86:0/458706541 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v113) ==== 80+0+120 (secure 0 0 0) 0x7f285001f020 con 0x7f28680d0200 2024-04-20T21:30:33.944 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6837/289590924 2024-04-21T21:30:19.784403+0000 2024-04-20T21:30:33.944 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6835/289590924 2024-04-21T21:30:19.784403+0000 2024-04-20T21:30:33.945 INFO:teuthology.orchestra.run.smithi086.stderr:listed 2 entries 2024-04-20T21:30:33.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.947+0000 7f286ee00640 1 -- 172.21.15.86:0/458706541 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f284406c8f0 msgr2=0x7f284406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:33.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.947+0000 7f286ee00640 1 --2- 172.21.15.86:0/458706541 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f284406c8f0 0x7f284406edb0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f2858036000 tx=0x7f285803c040 comp rx=0 tx=0).stop 2024-04-20T21:30:33.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.947+0000 7f286ee00640 1 -- 172.21.15.86:0/458706541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28680d0200 msgr2=0x7f28680cdf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:33.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.947+0000 7f286ee00640 1 --2- 172.21.15.86:0/458706541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28680d0200 0x7f28680cdf70 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f285000b410 tx=0x7f285000b8d0 comp rx=0 tx=0).stop 2024-04-20T21:30:33.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.947+0000 7f286ee00640 1 -- 172.21.15.86:0/458706541 shutdown_connections 2024-04-20T21:30:33.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.947+0000 7f286ee00640 1 --2- 172.21.15.86:0/458706541 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f284406c8f0 0x7f284406edb0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:33.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.947+0000 7f286ee00640 1 --2- 172.21.15.86:0/458706541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f286815e930 0x7f28680cb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:33.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.947+0000 7f286ee00640 1 --2- 172.21.15.86:0/458706541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28680d0200 0x7f28680cdf70 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:33.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.947+0000 7f286ee00640 1 --2- 172.21.15.86:0/458706541 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f28680cfda0 0x7f28680c8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:33.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.947+0000 7f286ee00640 1 -- 172.21.15.86:0/458706541 >> 172.21.15.86:0/458706541 conn(0x7f286808bf70 msgr2=0x7f28680b5670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:33.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.947+0000 7f286ee00640 1 -- 172.21.15.86:0/458706541 shutdown_connections 2024-04-20T21:30:33.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:33.947+0000 7f286ee00640 1 -- 172.21.15.86:0/458706541 wait complete. 2024-04-20T21:30:33.965 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi086.front.sepia.ceph.com... 2024-04-20T21:30:33.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:30:33.966 DEBUG:teuthology.orchestra.run.smithi086:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-20T21:30:34.034 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[67298]: fuse finished with error 0 and tester_r 0 2024-04-20T21:30:34.039 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:30:40.141 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-20T21:30:40.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:30:40.141 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-20T21:30:40.200 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:30:40.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.413+0000 7fe646400640 1 Processor -- start 2024-04-20T21:30:40.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.413+0000 7fe646400640 1 -- start start 2024-04-20T21:30:40.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.414+0000 7fe646400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe64006e6d0 0x7fe640071d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:40.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.414+0000 7fe646400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe64006eb30 con 0x7fe640072240 2024-04-20T21:30:40.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.414+0000 7fe646400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe64006dc30 con 0x7fe640146130 2024-04-20T21:30:40.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.414+0000 7fe646400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe6400cd8e0 con 0x7fe64006e6d0 2024-04-20T21:30:40.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.415+0000 7fe63fe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe640072240 0x7fe64015cc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41174/0 (socket says 172.21.15.86:41174) 2024-04-20T21:30:40.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.415+0000 7fe63fe00640 1 -- 172.21.15.86:0/2786838284 learned_addr learned my addr 172.21.15.86:0/2786838284 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:30:40.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.415+0000 7fe637e00640 1 --2- 172.21.15.86:0/2786838284 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe64006e6d0 0x7fe640071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:40.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.416+0000 7fe63ea00640 1 -- 172.21.15.86:0/2786838284 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3697925167 0 0) 0x7fe638001490 con 0x7fe640072240 2024-04-20T21:30:40.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.416+0000 7fe63ea00640 1 -- 172.21.15.86:0/2786838284 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe624003520 con 0x7fe640072240 2024-04-20T21:30:40.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.416+0000 7fe63ea00640 1 -- 172.21.15.86:0/2786838284 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 872251887 0 0) 0x7fe624003520 con 0x7fe640072240 2024-04-20T21:30:40.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.416+0000 7fe63ea00640 1 -- 172.21.15.86:0/2786838284 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe64006e6d0 msgr2=0x7fe640071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:40.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.416+0000 7fe63ea00640 1 --2- 172.21.15.86:0/2786838284 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe64006e6d0 0x7fe640071d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:40.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.416+0000 7fe63ea00640 1 -- 172.21.15.86:0/2786838284 >> v1:172.21.15.144:6789/0 conn(0x7fe640146130 legacy=0x7fe6400ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:40.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.416+0000 7fe63ea00640 1 -- 172.21.15.86:0/2786838284 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6400cdbc0 con 0x7fe640072240 2024-04-20T21:30:40.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.416+0000 7fe63ea00640 1 -- 172.21.15.86:0/2786838284 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fe638001bc0 con 0x7fe640072240 2024-04-20T21:30:40.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.416+0000 7fe63ea00640 1 -- 172.21.15.86:0/2786838284 >> v1:172.21.15.86:6789/0 conn(0x7fe640072240 legacy=0x7fe64015cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:40.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.416+0000 7fe63ea00640 1 --2- 172.21.15.86:0/2786838284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6240039e0 0x7fe624003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:40.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.416+0000 7fe63ea00640 1 --2- 172.21.15.86:0/2786838284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe624004800 0x7fe624004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:40.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.416+0000 7fe63ea00640 1 --2- 172.21.15.86:0/2786838284 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe64006e6d0 0x7fe6240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:40.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.416+0000 7fe63ea00640 1 -- 172.21.15.86:0/2786838284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe6400cd8e0 con 0x7fe6240039e0 2024-04-20T21:30:40.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.416+0000 7fe63ea00640 1 -- 172.21.15.86:0/2786838284 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe62400bb00 con 0x7fe624004800 2024-04-20T21:30:40.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.416+0000 7fe63ea00640 1 -- 172.21.15.86:0/2786838284 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe62400bd00 con 0x7fe64006e6d0 2024-04-20T21:30:40.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.416+0000 7fe63f400640 1 --2- 172.21.15.86:0/2786838284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6240039e0 0x7fe624003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:40.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.417+0000 7fe637e00640 1 --2- 172.21.15.86:0/2786838284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe624004800 0x7fe624004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:40.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.417+0000 7fe63fe00640 1 --2- 172.21.15.86:0/2786838284 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe64006e6d0 0x7fe6240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:40.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.417+0000 7fe63f400640 1 -- 172.21.15.86:0/2786838284 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe64006e6d0 msgr2=0x7fe6240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:40.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.417+0000 7fe63f400640 1 --2- 172.21.15.86:0/2786838284 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe64006e6d0 0x7fe6240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:40.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.417+0000 7fe63f400640 1 -- 172.21.15.86:0/2786838284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe624004800 msgr2=0x7fe624004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:40.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.417+0000 7fe63f400640 1 --2- 172.21.15.86:0/2786838284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe624004800 0x7fe624004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:40.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.417+0000 7fe63f400640 1 -- 172.21.15.86:0/2786838284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe624003520 con 0x7fe6240039e0 2024-04-20T21:30:40.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.418+0000 7fe63f400640 1 --2- 172.21.15.86:0/2786838284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6240039e0 0x7fe624003e00 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7fe630001720 tx=0x7fe624005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de8fedb17c3ffe97 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:40.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.418+0000 7fe63ea00640 1 -- 172.21.15.86:0/2786838284 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe630001370 con 0x7fe6240039e0 2024-04-20T21:30:40.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.418+0000 7fe63ea00640 1 -- 172.21.15.86:0/2786838284 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6300052d0 con 0x7fe6240039e0 2024-04-20T21:30:40.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.418+0000 7fe63ea00640 1 -- 172.21.15.86:0/2786838284 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe630003830 con 0x7fe6240039e0 2024-04-20T21:30:40.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.419+0000 7fe646400640 1 -- 172.21.15.86:0/2786838284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6240039e0 msgr2=0x7fe624003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:40.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.419+0000 7fe646400640 1 --2- 172.21.15.86:0/2786838284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6240039e0 0x7fe624003e00 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7fe630001720 tx=0x7fe624005830 comp rx=0 tx=0).stop 2024-04-20T21:30:40.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.419+0000 7fe63fe00640 1 -- 172.21.15.86:0/2786838284 reap_dead start 2024-04-20T21:30:40.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.419+0000 7fe646400640 1 -- 172.21.15.86:0/2786838284 shutdown_connections 2024-04-20T21:30:40.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.419+0000 7fe646400640 1 -- 172.21.15.86:0/2786838284 >> 172.21.15.86:0/2786838284 conn(0x7fe6400b2360 msgr2=0x7fe6400b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:40.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.419+0000 7fe646400640 1 -- 172.21.15.86:0/2786838284 shutdown_connections 2024-04-20T21:30:40.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.419+0000 7fe646400640 1 -- 172.21.15.86:0/2786838284 wait complete. 2024-04-20T21:30:40.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.421+0000 7fe646400640 1 Processor -- start 2024-04-20T21:30:40.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.422+0000 7fe646400640 1 -- start start 2024-04-20T21:30:40.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.422+0000 7fe646400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6240039e0 0x7fe6401f6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:40.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.423+0000 7fe646400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6400a9cc0 0x7fe6401319a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:40.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.423+0000 7fe646400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe640140710 0x7fe640140b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:40.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.423+0000 7fe646400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe64006f150 con 0x7fe6240039e0 2024-04-20T21:30:40.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.423+0000 7fe646400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe64006e1b0 con 0x7fe6400a9cc0 2024-04-20T21:30:40.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.423+0000 7fe646400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe6400cd4e0 con 0x7fe640140710 2024-04-20T21:30:40.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.423+0000 7fe63f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6240039e0 0x7fe6401f6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:40.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.423+0000 7fe63f400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6240039e0 0x7fe6401f6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58566/0 (socket says 172.21.15.86:58566) 2024-04-20T21:30:40.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.423+0000 7fe63f400640 1 -- 172.21.15.86:0/2524613922 learned_addr learned my addr 172.21.15.86:0/2524613922 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:40.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.423+0000 7fe63fe00640 1 --2- 172.21.15.86:0/2524613922 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe640140710 0x7fe640140b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:40.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.423+0000 7fe63f400640 1 -- 172.21.15.86:0/2524613922 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe640140710 msgr2=0x7fe640140b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:40.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.423+0000 7fe637e00640 1 --2- 172.21.15.86:0/2524613922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6400a9cc0 0x7fe6401319a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:40.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.423+0000 7fe63f400640 1 --2- 172.21.15.86:0/2524613922 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe640140710 0x7fe640140b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:40.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.423+0000 7fe63f400640 1 -- 172.21.15.86:0/2524613922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6400a9cc0 msgr2=0x7fe6401319a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:40.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.423+0000 7fe63f400640 1 --2- 172.21.15.86:0/2524613922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6400a9cc0 0x7fe6401319a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:40.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.423+0000 7fe63f400640 1 -- 172.21.15.86:0/2524613922 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6401f29b0 con 0x7fe6240039e0 2024-04-20T21:30:40.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.424+0000 7fe63f400640 1 --2- 172.21.15.86:0/2524613922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6240039e0 0x7fe6401f6870 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7fe630008fd0 tx=0x7fe630001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:40.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.424+0000 7fe637400640 1 -- 172.21.15.86:0/2524613922 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe630041070 con 0x7fe6240039e0 2024-04-20T21:30:40.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.424+0000 7fe637400640 1 -- 172.21.15.86:0/2524613922 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe630000ed0 con 0x7fe6240039e0 2024-04-20T21:30:40.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.424+0000 7fe637400640 1 -- 172.21.15.86:0/2524613922 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe630007a30 con 0x7fe6240039e0 2024-04-20T21:30:40.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.424+0000 7fe646400640 1 -- 172.21.15.86:0/2524613922 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6401f2c90 con 0x7fe6240039e0 2024-04-20T21:30:40.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.424+0000 7fe646400640 1 -- 172.21.15.86:0/2524613922 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6401f3250 con 0x7fe6240039e0 2024-04-20T21:30:40.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.426+0000 7fe646400640 1 -- 172.21.15.86:0/2524613922 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6400690f0 con 0x7fe6240039e0 2024-04-20T21:30:40.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.427+0000 7fe637400640 1 -- 172.21.15.86:0/2524613922 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe630044070 con 0x7fe6240039e0 2024-04-20T21:30:40.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.427+0000 7fe637400640 1 --2- 172.21.15.86:0/2524613922 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe61c06c920 0x7fe61c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:40.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.428+0000 7fe637400640 1 -- 172.21.15.86:0/2524613922 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5853+0+0 (secure 0 0 0) 0x7fe6300ad730 con 0x7fe6240039e0 2024-04-20T21:30:40.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.428+0000 7fe637e00640 1 --2- 172.21.15.86:0/2524613922 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe61c06c920 0x7fe61c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:40.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.428+0000 7fe637e00640 1 --2- 172.21.15.86:0/2524613922 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe61c06c920 0x7fe61c06ede0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fe6400aff60 tx=0x7fe628008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:40.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.432+0000 7fe637400640 1 -- 172.21.15.86:0/2524613922 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe6400690f0 con 0x7fe6240039e0 2024-04-20T21:30:40.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.675+0000 7fe646400640 1 -- 172.21.15.86:0/2524613922 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6400689e0 con 0x7fe6240039e0 2024-04-20T21:30:40.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.676+0000 7fe637400640 1 -- 172.21.15.86:0/2524613922 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) ==== 95+0+5212 (secure 0 0 0) 0x7fe63007bac0 con 0x7fe6240039e0 2024-04-20T21:30:40.675 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:40.675 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":105,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6253,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3627230007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3627230007},{"type":"v1","addr":"172.21.15.144:6833","nonce":3627230007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":6256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2498297133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2498297133},{"type":"v1","addr":"172.21.15.144:6835","nonce":2498297133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":9228,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/1120740475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1120740475},{"type":"v1","addr":"172.21.15.86:6836","nonce":1120740475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:29:43.769581+0000","modified":"2024-04-20T21:30:23.820570+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":113,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9585},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9585":{"gid":9585,"name":"a","rank":0,"incarnation":102,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6837/1453621918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":1453621918},{"type":"v1","addr":"172.21.15.86:6837","nonce":1453621918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9585,"qdb_cluster":[9585]},"id":8}]} 2024-04-20T21:30:40.676 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 105 2024-04-20T21:30:40.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.679+0000 7fe646400640 1 -- 172.21.15.86:0/2524613922 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe61c06c920 msgr2=0x7fe61c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:40.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.679+0000 7fe646400640 1 --2- 172.21.15.86:0/2524613922 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe61c06c920 0x7fe61c06ede0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fe6400aff60 tx=0x7fe628008040 comp rx=0 tx=0).stop 2024-04-20T21:30:40.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.679+0000 7fe646400640 1 -- 172.21.15.86:0/2524613922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6240039e0 msgr2=0x7fe6401f6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:40.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.679+0000 7fe646400640 1 --2- 172.21.15.86:0/2524613922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6240039e0 0x7fe6401f6870 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7fe630008fd0 tx=0x7fe630001120 comp rx=0 tx=0).stop 2024-04-20T21:30:40.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.679+0000 7fe646400640 1 -- 172.21.15.86:0/2524613922 shutdown_connections 2024-04-20T21:30:40.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.679+0000 7fe646400640 1 --2- 172.21.15.86:0/2524613922 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe61c06c920 0x7fe61c06ede0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:40.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.679+0000 7fe646400640 1 --2- 172.21.15.86:0/2524613922 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe640140710 0x7fe640140b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:40.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.679+0000 7fe646400640 1 --2- 172.21.15.86:0/2524613922 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6400a9cc0 0x7fe6401319a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:40.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.679+0000 7fe646400640 1 --2- 172.21.15.86:0/2524613922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6240039e0 0x7fe6401f6870 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:40.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.679+0000 7fe646400640 1 -- 172.21.15.86:0/2524613922 >> 172.21.15.86:0/2524613922 conn(0x7fe6400b2360 msgr2=0x7fe64015e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:40.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.679+0000 7fe646400640 1 -- 172.21.15.86:0/2524613922 shutdown_connections 2024-04-20T21:30:40.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:40.679+0000 7fe646400640 1 -- 172.21.15.86:0/2524613922 wait complete. 2024-04-20T21:30:40.697 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json tag path /parent mytag123 2024-04-20T21:30:40.929 INFO:teuthology.orchestra.run.smithi086.stdout:{"return_code":0,"scrub_tag":"mytag123","mode":"asynchronous"} 2024-04-20T21:30:40.929 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag123" } 2024-04-20T21:30:40.930 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-04-20T21:30:41.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.034+0000 7f2285457f40 1 Processor -- start 2024-04-20T21:30:41.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.034+0000 7f2285457f40 1 -- start start 2024-04-20T21:30:41.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.035+0000 7f2285457f40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56342d16be30 0x56342d14eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.035+0000 7f2285457f40 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56342d1de7e0 0x56342d1dc870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.035+0000 7f2285457f40 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56342d1e1ed0 0x56342d1d7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.035+0000 7f2285457f40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56342d16da50 con 0x56342d16be30 2024-04-20T21:30:41.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.035+0000 7f2285457f40 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56342d0c8cc0 con 0x56342d1e1ed0 2024-04-20T21:30:41.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.035+0000 7f2285457f40 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x56342d16dcf0 con 0x56342d1de7e0 2024-04-20T21:30:41.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.036+0000 7f2282600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56342d16be30 0x56342d14eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:41.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.036+0000 7f2282600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56342d16be30 0x56342d14eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58576/0 (socket says 172.21.15.86:58576) 2024-04-20T21:30:41.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.036+0000 7f2282600640 1 -- 172.21.15.86:0/2062479440 learned_addr learned my addr 172.21.15.86:0/2062479440 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:41.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.036+0000 7f227be00640 1 --2- 172.21.15.86:0/2062479440 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56342d1de7e0 0x56342d1dc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:41.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.036+0000 7f2283000640 1 --2- 172.21.15.86:0/2062479440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56342d1e1ed0 0x56342d1d7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:41.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.036+0000 7f2282600640 1 -- 172.21.15.86:0/2062479440 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56342d1de7e0 msgr2=0x56342d1dc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:41.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.036+0000 7f2282600640 1 --2- 172.21.15.86:0/2062479440 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56342d1de7e0 0x56342d1dc870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.036+0000 7f2282600640 1 -- 172.21.15.86:0/2062479440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56342d1e1ed0 msgr2=0x56342d1d7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:41.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.036+0000 7f2282600640 1 --2- 172.21.15.86:0/2062479440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56342d1e1ed0 0x56342d1d7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.036+0000 7f2282600640 1 -- 172.21.15.86:0/2062479440 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56342d1e1530 con 0x56342d16be30 2024-04-20T21:30:41.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.036+0000 7f227be00640 1 --2- 172.21.15.86:0/2062479440 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56342d1de7e0 0x56342d1dc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:30:41.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.037+0000 7f2282600640 1 --2- 172.21.15.86:0/2062479440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56342d16be30 0x56342d14eaa0 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f227400a5c0 tx=0x7f227400aa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5582101a460a9d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:41.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.038+0000 7f2281200640 1 -- 172.21.15.86:0/2062479440 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22740089f0 con 0x56342d16be30 2024-04-20T21:30:41.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.038+0000 7f2281200640 1 -- 172.21.15.86:0/2062479440 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f227400f070 con 0x56342d16be30 2024-04-20T21:30:41.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.038+0000 7f2281200640 1 -- 172.21.15.86:0/2062479440 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2274008df0 con 0x56342d16be30 2024-04-20T21:30:41.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.039+0000 7f2285457f40 1 -- 172.21.15.86:0/2062479440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56342d16be30 msgr2=0x56342d14eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:41.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.039+0000 7f2285457f40 1 --2- 172.21.15.86:0/2062479440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56342d16be30 0x56342d14eaa0 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f227400a5c0 tx=0x7f227400aa80 comp rx=0 tx=0).stop 2024-04-20T21:30:41.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.039+0000 7f2285457f40 1 -- 172.21.15.86:0/2062479440 shutdown_connections 2024-04-20T21:30:41.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.039+0000 7f2285457f40 1 --2- 172.21.15.86:0/2062479440 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56342d1e1ed0 0x56342d1d7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.039+0000 7f2285457f40 1 --2- 172.21.15.86:0/2062479440 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56342d1de7e0 0x56342d1dc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.039+0000 7f2285457f40 1 --2- 172.21.15.86:0/2062479440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56342d16be30 0x56342d14eaa0 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.039+0000 7f2285457f40 1 -- 172.21.15.86:0/2062479440 >> 172.21.15.86:0/2062479440 conn(0x56342d126950 msgr2=0x56342d120060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:41.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.039+0000 7f2285457f40 1 -- 172.21.15.86:0/2062479440 shutdown_connections 2024-04-20T21:30:41.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.039+0000 7f2285457f40 1 -- 172.21.15.86:0/2062479440 wait complete. 2024-04-20T21:30:41.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.040+0000 7f2285457f40 1 Processor -- start 2024-04-20T21:30:41.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.040+0000 7f2285457f40 1 -- start start 2024-04-20T21:30:41.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.041+0000 7f2285457f40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56342d16be30 0x56342d1f6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.041+0000 7f2285457f40 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56342d1de7e0 0x56342d1d61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.041+0000 7f2285457f40 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56342d1e1ed0 0x56342d1d6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.041+0000 7f2285457f40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56342d1528c0 con 0x56342d16be30 2024-04-20T21:30:41.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.041+0000 7f2285457f40 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56342d14efe0 con 0x56342d1e1ed0 2024-04-20T21:30:41.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.041+0000 7f2285457f40 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x56342d1e12e0 con 0x56342d1de7e0 2024-04-20T21:30:41.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.041+0000 7f2282600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56342d16be30 0x56342d1f6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:41.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.041+0000 7f2282600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56342d16be30 0x56342d1f6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58584/0 (socket says 172.21.15.86:58584) 2024-04-20T21:30:41.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.041+0000 7f2282600640 1 -- 172.21.15.86:0/3109023473 learned_addr learned my addr 172.21.15.86:0/3109023473 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:41.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.042+0000 7f2283000640 1 --2- 172.21.15.86:0/3109023473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56342d1e1ed0 0x56342d1d6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:41.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.042+0000 7f2282600640 1 -- 172.21.15.86:0/3109023473 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56342d1de7e0 msgr2=0x56342d1d61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:41.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.042+0000 7f2282600640 1 --2- 172.21.15.86:0/3109023473 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56342d1de7e0 0x56342d1d61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.042+0000 7f2282600640 1 -- 172.21.15.86:0/3109023473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56342d1e1ed0 msgr2=0x56342d1d6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:41.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.042+0000 7f2282600640 1 --2- 172.21.15.86:0/3109023473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56342d1e1ed0 0x56342d1d6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.042+0000 7f2282600640 1 -- 172.21.15.86:0/3109023473 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56342d1e0b00 con 0x56342d16be30 2024-04-20T21:30:41.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.042+0000 7f2282600640 1 --2- 172.21.15.86:0/3109023473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56342d16be30 0x56342d1f6930 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f227400aac0 tx=0x7f2274002fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:41.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.045+0000 7f227aa00640 1 -- 172.21.15.86:0/3109023473 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2274014680 con 0x56342d16be30 2024-04-20T21:30:41.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.045+0000 7f227aa00640 1 -- 172.21.15.86:0/3109023473 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2274014840 con 0x56342d16be30 2024-04-20T21:30:41.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.045+0000 7f227aa00640 1 -- 172.21.15.86:0/3109023473 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2274014a70 con 0x56342d16be30 2024-04-20T21:30:41.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.045+0000 7f2285457f40 1 -- 172.21.15.86:0/3109023473 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56342d1def30 con 0x56342d16be30 2024-04-20T21:30:41.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.045+0000 7f2285457f40 1 -- 172.21.15.86:0/3109023473 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56342d1df1d0 con 0x56342d16be30 2024-04-20T21:30:41.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.047+0000 7f227aa00640 1 -- 172.21.15.86:0/3109023473 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2274026020 con 0x56342d16be30 2024-04-20T21:30:41.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.048+0000 7f227aa00640 1 --2- 172.21.15.86:0/3109023473 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f226006b2d0 0x7f226006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.048+0000 7f227aa00640 1 -- 172.21.15.86:0/3109023473 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5853+0+0 (secure 0 0 0) 0x7f227408c090 con 0x56342d16be30 2024-04-20T21:30:41.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.048+0000 7f227be00640 1 --2- 172.21.15.86:0/3109023473 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f226006b2d0 0x7f226006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:41.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.048+0000 7f2285457f40 1 --2- 172.21.15.86:0/3109023473 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x56342d1e17e0 0x56342d1fa250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.048+0000 7f2285457f40 1 -- 172.21.15.86:0/3109023473 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:1 17.36 17:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x56342d1d98f0 con 0x56342d1e17e0 2024-04-20T21:30:41.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.049+0000 7f2283000640 1 --2- 172.21.15.86:0/3109023473 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x56342d1e17e0 0x56342d1fa250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:41.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.049+0000 7f227be00640 1 --2- 172.21.15.86:0/3109023473 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f226006b2d0 0x7f226006d790 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x56342d1fb230 tx=0x7f226c0055f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:41.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.050+0000 7f2283000640 1 --2- 172.21.15.86:0/3109023473 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x56342d1e17e0 0x56342d1fa250 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:41.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.050+0000 7f2283000640 1 -- 172.21.15.86:0/3109023473 <== osd.0 v2:172.21.15.86:6800/2471076035 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) ==== 164+0+0 (crc 0 0 0) 0x7f227c0525a0 con 0x56342d1e17e0 2024-04-20T21:30:41.050 INFO:teuthology.orchestra.run.smithi086.stderr:error getting xattr cephfs_data/100000001f7.00000000/scrub_tag: (61) No data available 2024-04-20T21:30:41.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.050+0000 7f2285457f40 1 -- 172.21.15.86:0/3109023473 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x56342d1e17e0 msgr2=0x56342d1fa250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:41.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.050+0000 7f2285457f40 1 --2- 172.21.15.86:0/3109023473 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x56342d1e17e0 0x56342d1fa250 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.050+0000 7f2285457f40 1 -- 172.21.15.86:0/3109023473 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f226006b2d0 msgr2=0x7f226006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:41.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.050+0000 7f2285457f40 1 --2- 172.21.15.86:0/3109023473 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f226006b2d0 0x7f226006d790 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x56342d1fb230 tx=0x7f226c0055f0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.050+0000 7f2285457f40 1 -- 172.21.15.86:0/3109023473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56342d16be30 msgr2=0x56342d1f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:41.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.050+0000 7f2285457f40 1 --2- 172.21.15.86:0/3109023473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56342d16be30 0x56342d1f6930 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f227400aac0 tx=0x7f2274002fd0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.050+0000 7f2283000640 1 -- 172.21.15.86:0/3109023473 reap_dead start 2024-04-20T21:30:41.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.050+0000 7f2285457f40 1 -- 172.21.15.86:0/3109023473 shutdown_connections 2024-04-20T21:30:41.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.050+0000 7f2285457f40 1 -- 172.21.15.86:0/3109023473 >> 172.21.15.86:0/3109023473 conn(0x56342d126950 msgr2=0x56342d120060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:41.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.050+0000 7f2285457f40 1 -- 172.21.15.86:0/3109023473 shutdown_connections 2024-04-20T21:30:41.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.050+0000 7f2285457f40 1 -- 172.21.15.86:0/3109023473 wait complete. 2024-04-20T21:30:41.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:30:41.055 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-04-20T21:30:41.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.141+0000 7f7045778f40 1 Processor -- start 2024-04-20T21:30:41.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.141+0000 7f7045778f40 1 -- start start 2024-04-20T21:30:41.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.142+0000 7f7045778f40 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70280030f0 0x55a275511830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.142+0000 7f7045778f40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2754379a0 0x55a27546a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.142+0000 7f7045778f40 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a27550fae0 0x55a27546ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.142+0000 7f7045778f40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a2753e0cc0 con 0x55a2754379a0 2024-04-20T21:30:41.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.142+0000 7f7045778f40 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a2753e4d00 con 0x7f70280030f0 2024-04-20T21:30:41.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.142+0000 7f7045778f40 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55a275485f90 con 0x55a27550fae0 2024-04-20T21:30:41.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.143+0000 7f7042000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2754379a0 0x55a27546a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:41.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.143+0000 7f7042000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2754379a0 0x55a27546a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58600/0 (socket says 172.21.15.86:58600) 2024-04-20T21:30:41.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.143+0000 7f7042000640 1 -- 172.21.15.86:0/3490413 learned_addr learned my addr 172.21.15.86:0/3490413 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:41.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.143+0000 7f7042a00640 1 --2- 172.21.15.86:0/3490413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70280030f0 0x55a275511830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:41.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.143+0000 7f7043400640 1 --2- 172.21.15.86:0/3490413 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a27550fae0 0x55a27546ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:41.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.143+0000 7f7042000640 1 -- 172.21.15.86:0/3490413 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a27550fae0 msgr2=0x55a27546ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:41.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.143+0000 7f7042000640 1 --2- 172.21.15.86:0/3490413 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a27550fae0 0x55a27546ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.143+0000 7f7042000640 1 -- 172.21.15.86:0/3490413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70280030f0 msgr2=0x55a275511830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:41.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.143+0000 7f7042000640 1 --2- 172.21.15.86:0/3490413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70280030f0 0x55a275511830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.143+0000 7f7042000640 1 -- 172.21.15.86:0/3490413 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a27546a060 con 0x55a2754379a0 2024-04-20T21:30:41.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.143+0000 7f7042a00640 1 --2- 172.21.15.86:0/3490413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70280030f0 0x55a275511830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:30:41.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.143+0000 7f7042000640 1 --2- 172.21.15.86:0/3490413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2754379a0 0x55a27546a6c0 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f7030000c00 tx=0x7f703004c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d75771687e52af15 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:41.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.144+0000 7f7040c00640 1 -- 172.21.15.86:0/3490413 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7030057040 con 0x55a2754379a0 2024-04-20T21:30:41.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.144+0000 7f7040c00640 1 -- 172.21.15.86:0/3490413 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7030061070 con 0x55a2754379a0 2024-04-20T21:30:41.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.144+0000 7f7040c00640 1 -- 172.21.15.86:0/3490413 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703005c810 con 0x55a2754379a0 2024-04-20T21:30:41.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.144+0000 7f7045778f40 1 -- 172.21.15.86:0/3490413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2754379a0 msgr2=0x55a27546a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:41.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.144+0000 7f7045778f40 1 --2- 172.21.15.86:0/3490413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2754379a0 0x55a27546a6c0 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f7030000c00 tx=0x7f703004c070 comp rx=0 tx=0).stop 2024-04-20T21:30:41.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.144+0000 7f7045778f40 1 -- 172.21.15.86:0/3490413 shutdown_connections 2024-04-20T21:30:41.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.144+0000 7f7045778f40 1 --2- 172.21.15.86:0/3490413 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55a27550fae0 0x55a27546ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.144+0000 7f7045778f40 1 --2- 172.21.15.86:0/3490413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a2754379a0 0x55a27546a6c0 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.144+0000 7f7045778f40 1 --2- 172.21.15.86:0/3490413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70280030f0 0x55a275511830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.144+0000 7f7045778f40 1 -- 172.21.15.86:0/3490413 >> 172.21.15.86:0/3490413 conn(0x55a27543e950 msgr2=0x55a275437290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:41.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.144+0000 7f7045778f40 1 -- 172.21.15.86:0/3490413 shutdown_connections 2024-04-20T21:30:41.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.145+0000 7f7045778f40 1 -- 172.21.15.86:0/3490413 wait complete. 2024-04-20T21:30:41.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.145+0000 7f7045778f40 1 Processor -- start 2024-04-20T21:30:41.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.145+0000 7f7045778f40 1 -- start start 2024-04-20T21:30:41.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.146+0000 7f7045778f40 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2754379a0 0x55a2754f1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.146+0000 7f7045778f40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a27550fae0 0x55a27550f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.146+0000 7f7045778f40 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70280030f0 0x55a2754f4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.146+0000 7f7045778f40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a2755101c0 con 0x55a27550fae0 2024-04-20T21:30:41.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.146+0000 7f7045778f40 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55a27546a300 con 0x55a2754379a0 2024-04-20T21:30:41.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.146+0000 7f7045778f40 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55a2754f6160 con 0x7f70280030f0 2024-04-20T21:30:41.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.147+0000 7f7042000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a27550fae0 0x55a27550f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:41.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.147+0000 7f7042000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a27550fae0 0x55a27550f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58614/0 (socket says 172.21.15.86:58614) 2024-04-20T21:30:41.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.147+0000 7f7042000640 1 -- 172.21.15.86:0/4045544399 learned_addr learned my addr 172.21.15.86:0/4045544399 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:41.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.147+0000 7f7043400640 1 --2- 172.21.15.86:0/4045544399 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70280030f0 0x55a2754f4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:41.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.147+0000 7f7042000640 1 -- 172.21.15.86:0/4045544399 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70280030f0 msgr2=0x55a2754f4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:41.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.147+0000 7f7042000640 1 --2- 172.21.15.86:0/4045544399 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f70280030f0 0x55a2754f4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.147+0000 7f7042000640 1 -- 172.21.15.86:0/4045544399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2754379a0 msgr2=0x55a2754f1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:41.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.147+0000 7f7042a00640 1 --2- 172.21.15.86:0/4045544399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2754379a0 0x55a2754f1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:41.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.147+0000 7f7042000640 1 --2- 172.21.15.86:0/4045544399 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55a2754379a0 0x55a2754f1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.147+0000 7f7042000640 1 -- 172.21.15.86:0/4045544399 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2754f2050 con 0x55a27550fae0 2024-04-20T21:30:41.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.147+0000 7f7042000640 1 --2- 172.21.15.86:0/4045544399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a27550fae0 0x55a27550f260 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7f7030001d90 tx=0x7f7028004120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:41.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.148+0000 7f703b400640 1 -- 172.21.15.86:0/4045544399 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703004a8b0 con 0x55a27550fae0 2024-04-20T21:30:41.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.148+0000 7f703b400640 1 -- 172.21.15.86:0/4045544399 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f703006b030 con 0x55a27550fae0 2024-04-20T21:30:41.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.148+0000 7f703b400640 1 -- 172.21.15.86:0/4045544399 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7030070420 con 0x55a27550fae0 2024-04-20T21:30:41.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.148+0000 7f7045778f40 1 -- 172.21.15.86:0/4045544399 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a2754f88d0 con 0x55a27550fae0 2024-04-20T21:30:41.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.148+0000 7f7045778f40 1 -- 172.21.15.86:0/4045544399 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2754f25f0 con 0x55a27550fae0 2024-04-20T21:30:41.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.150+0000 7f703b400640 1 -- 172.21.15.86:0/4045544399 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7030067410 con 0x55a27550fae0 2024-04-20T21:30:41.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.150+0000 7f703b400640 1 --2- 172.21.15.86:0/4045544399 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f702406b300 0x7f702406d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.151+0000 7f703b400640 1 -- 172.21.15.86:0/4045544399 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5853+0+0 (secure 0 0 0) 0x7f70300cd6c0 con 0x55a27550fae0 2024-04-20T21:30:41.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.151+0000 7f7042a00640 1 --2- 172.21.15.86:0/4045544399 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f702406b300 0x7f702406d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:41.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.151+0000 7f7045778f40 1 --2- 172.21.15.86:0/4045544399 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x55a2754f1390 0x55a2754f05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.151+0000 7f7045778f40 1 -- 172.21.15.86:0/4045544399 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:1 17.12 17:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x55a27546a850 con 0x55a2754f1390 2024-04-20T21:30:41.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.151+0000 7f7043400640 1 --2- 172.21.15.86:0/4045544399 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x55a2754f1390 0x55a2754f05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:41.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.151+0000 7f7042a00640 1 --2- 172.21.15.86:0/4045544399 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f702406b300 0x7f702406d7c0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f7034001ed0 tx=0x7f70340014c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:41.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.152+0000 7f7043400640 1 --2- 172.21.15.86:0/4045544399 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x55a2754f1390 0x55a2754f05b0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:41.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.152+0000 7f7043400640 1 -- 172.21.15.86:0/4045544399 <== osd.3 v2:172.21.15.86:6808/794135519 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr (12) out=12b] v0'0 uv22 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7f703c011040 con 0x55a2754f1390 2024-04-20T21:30:41.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.152+0000 7f7045778f40 1 -- 172.21.15.86:0/4045544399 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x55a2754f1390 msgr2=0x55a2754f05b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:41.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.152+0000 7f7045778f40 1 --2- 172.21.15.86:0/4045544399 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x55a2754f1390 0x55a2754f05b0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.152+0000 7f7045778f40 1 -- 172.21.15.86:0/4045544399 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f702406b300 msgr2=0x7f702406d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:41.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.152+0000 7f7045778f40 1 --2- 172.21.15.86:0/4045544399 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f702406b300 0x7f702406d7c0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f7034001ed0 tx=0x7f70340014c0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.152+0000 7f7045778f40 1 -- 172.21.15.86:0/4045544399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a27550fae0 msgr2=0x55a27550f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:41.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.152+0000 7f7045778f40 1 --2- 172.21.15.86:0/4045544399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a27550fae0 0x55a27550f260 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7f7030001d90 tx=0x7f7028004120 comp rx=0 tx=0).stop 2024-04-20T21:30:41.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.152+0000 7f7043400640 1 -- 172.21.15.86:0/4045544399 reap_dead start 2024-04-20T21:30:41.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.152+0000 7f7045778f40 1 -- 172.21.15.86:0/4045544399 shutdown_connections 2024-04-20T21:30:41.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.152+0000 7f7045778f40 1 -- 172.21.15.86:0/4045544399 >> 172.21.15.86:0/4045544399 conn(0x55a27543e950 msgr2=0x55a27546b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:41.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.152+0000 7f7045778f40 1 -- 172.21.15.86:0/4045544399 shutdown_connections 2024-04-20T21:30:41.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.153+0000 7f7045778f40 1 -- 172.21.15.86:0/4045544399 wait complete. 2024-04-20T21:30:41.155 INFO:teuthology.orchestra.run.smithi086.stdout:mytag123 2024-04-20T21:30:41.156 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-04-20T21:30:41.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.245+0000 7f9424036f40 1 Processor -- start 2024-04-20T21:30:41.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.245+0000 7f9424036f40 1 -- start start 2024-04-20T21:30:41.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.246+0000 7f9424036f40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9e3effcf0 0x55c9e3f9e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.246+0000 7f9424036f40 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9e3f14f60 0x55c9e3f18af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.246+0000 7f9424036f40 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c9e3f1a720 0x55c9e3f9b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.246+0000 7f9424036f40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c9e3e77cc0 con 0x55c9e3effcf0 2024-04-20T21:30:41.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.246+0000 7f9424036f40 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c9e3fa2150 con 0x55c9e3f14f60 2024-04-20T21:30:41.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.246+0000 7f9424036f40 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55c9e3f1b340 con 0x55c9e3f1a720 2024-04-20T21:30:41.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.247+0000 7f9420a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9e3effcf0 0x55c9e3f9e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:41.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.247+0000 7f9420a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9e3effcf0 0x55c9e3f9e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58628/0 (socket says 172.21.15.86:58628) 2024-04-20T21:30:41.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.247+0000 7f9420a00640 1 -- 172.21.15.86:0/1561830620 learned_addr learned my addr 172.21.15.86:0/1561830620 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:41.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.247+0000 7f9421400640 1 --2- 172.21.15.86:0/1561830620 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c9e3f1a720 0x55c9e3f9b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:41.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.247+0000 7f9420000640 1 --2- 172.21.15.86:0/1561830620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9e3f14f60 0x55c9e3f18af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:41.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.247+0000 7f9420a00640 1 -- 172.21.15.86:0/1561830620 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c9e3f1a720 msgr2=0x55c9e3f9b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:41.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.247+0000 7f9420a00640 1 --2- 172.21.15.86:0/1561830620 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c9e3f1a720 0x55c9e3f9b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.247+0000 7f9420a00640 1 -- 172.21.15.86:0/1561830620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9e3f14f60 msgr2=0x55c9e3f18af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:41.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.247+0000 7f9420a00640 1 --2- 172.21.15.86:0/1561830620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9e3f14f60 0x55c9e3f18af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.247+0000 7f9420a00640 1 -- 172.21.15.86:0/1561830620 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9e3ece980 con 0x55c9e3effcf0 2024-04-20T21:30:41.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.247+0000 7f9420a00640 1 --2- 172.21.15.86:0/1561830620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9e3effcf0 0x55c9e3f9e8e0 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f9410007000 tx=0x7f9410051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71a120704c2eaed2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:41.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.248+0000 7f941ec00640 1 -- 172.21.15.86:0/1561830620 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941005b040 con 0x55c9e3effcf0 2024-04-20T21:30:41.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.248+0000 7f941ec00640 1 -- 172.21.15.86:0/1561830620 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9410066070 con 0x55c9e3effcf0 2024-04-20T21:30:41.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.248+0000 7f941ec00640 1 -- 172.21.15.86:0/1561830620 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941004ee40 con 0x55c9e3effcf0 2024-04-20T21:30:41.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.248+0000 7f9424036f40 1 -- 172.21.15.86:0/1561830620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9e3effcf0 msgr2=0x55c9e3f9e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:41.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.248+0000 7f9424036f40 1 --2- 172.21.15.86:0/1561830620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9e3effcf0 0x55c9e3f9e8e0 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f9410007000 tx=0x7f9410051000 comp rx=0 tx=0).stop 2024-04-20T21:30:41.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.248+0000 7f9424036f40 1 -- 172.21.15.86:0/1561830620 shutdown_connections 2024-04-20T21:30:41.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.248+0000 7f9424036f40 1 --2- 172.21.15.86:0/1561830620 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c9e3f1a720 0x55c9e3f9b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.248+0000 7f9424036f40 1 --2- 172.21.15.86:0/1561830620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9e3f14f60 0x55c9e3f18af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.248+0000 7f9424036f40 1 --2- 172.21.15.86:0/1561830620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9e3effcf0 0x55c9e3f9e8e0 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.248+0000 7f9424036f40 1 -- 172.21.15.86:0/1561830620 >> 172.21.15.86:0/1561830620 conn(0x55c9e3ece220 msgr2=0x55c9e3efc120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:41.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.249+0000 7f9424036f40 1 -- 172.21.15.86:0/1561830620 shutdown_connections 2024-04-20T21:30:41.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.249+0000 7f9424036f40 1 -- 172.21.15.86:0/1561830620 wait complete. 2024-04-20T21:30:41.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.249+0000 7f9424036f40 1 Processor -- start 2024-04-20T21:30:41.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.249+0000 7f9424036f40 1 -- start start 2024-04-20T21:30:41.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.250+0000 7f9424036f40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9e3effcf0 0x55c9e3fa66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.250+0000 7f9424036f40 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9e3f14f60 0x55c9e3efc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.250+0000 7f9424036f40 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c9e3f1a720 0x55c9e3f8f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.250+0000 7f9424036f40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c9e3fa0300 con 0x55c9e3effcf0 2024-04-20T21:30:41.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.250+0000 7f9424036f40 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55c9e3f1b570 con 0x55c9e3f14f60 2024-04-20T21:30:41.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.250+0000 7f9424036f40 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55c9e3f1b810 con 0x55c9e3f1a720 2024-04-20T21:30:41.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.251+0000 7f9420a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9e3effcf0 0x55c9e3fa66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:41.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.251+0000 7f9420a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9e3effcf0 0x55c9e3fa66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58630/0 (socket says 172.21.15.86:58630) 2024-04-20T21:30:41.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.251+0000 7f9420a00640 1 -- 172.21.15.86:0/559836106 learned_addr learned my addr 172.21.15.86:0/559836106 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:41.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.251+0000 7f9421400640 1 --2- 172.21.15.86:0/559836106 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c9e3f1a720 0x55c9e3f8f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:41.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.251+0000 7f9420000640 1 --2- 172.21.15.86:0/559836106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9e3f14f60 0x55c9e3efc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:41.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.251+0000 7f9420a00640 1 -- 172.21.15.86:0/559836106 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c9e3f1a720 msgr2=0x55c9e3f8f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:41.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.251+0000 7f9420a00640 1 --2- 172.21.15.86:0/559836106 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55c9e3f1a720 0x55c9e3f8f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.251+0000 7f9420a00640 1 -- 172.21.15.86:0/559836106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9e3f14f60 msgr2=0x55c9e3efc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:41.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.251+0000 7f9420a00640 1 --2- 172.21.15.86:0/559836106 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55c9e3f14f60 0x55c9e3efc4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.251+0000 7f9420a00640 1 -- 172.21.15.86:0/559836106 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9e3f8fe30 con 0x55c9e3effcf0 2024-04-20T21:30:41.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.251+0000 7f9420a00640 1 --2- 172.21.15.86:0/559836106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9e3effcf0 0x55c9e3fa66d0 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f9410063040 tx=0x7f9410003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:41.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.252+0000 7f941d800640 1 -- 172.21.15.86:0/559836106 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94100617b0 con 0x55c9e3effcf0 2024-04-20T21:30:41.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.252+0000 7f941d800640 1 -- 172.21.15.86:0/559836106 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9410061990 con 0x55c9e3effcf0 2024-04-20T21:30:41.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.252+0000 7f941d800640 1 -- 172.21.15.86:0/559836106 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9410061d80 con 0x55c9e3effcf0 2024-04-20T21:30:41.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.252+0000 7f9424036f40 1 -- 172.21.15.86:0/559836106 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c9e403e6e0 con 0x55c9e3effcf0 2024-04-20T21:30:41.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.252+0000 7f9424036f40 1 -- 172.21.15.86:0/559836106 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9e3fa38e0 con 0x55c9e3effcf0 2024-04-20T21:30:41.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.254+0000 7f941d800640 1 -- 172.21.15.86:0/559836106 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9410056420 con 0x55c9e3effcf0 2024-04-20T21:30:41.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.255+0000 7f941d800640 1 --2- 172.21.15.86:0/559836106 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f940006b2d0 0x7f940006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.255+0000 7f941d800640 1 -- 172.21.15.86:0/559836106 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5853+0+0 (secure 0 0 0) 0x7f94100d0f30 con 0x55c9e3effcf0 2024-04-20T21:30:41.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.255+0000 7f9420000640 1 --2- 172.21.15.86:0/559836106 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f940006b2d0 0x7f940006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:41.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.255+0000 7f9424036f40 1 --2- 172.21.15.86:0/559836106 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x55c9e3fa43c0 0x55c9e3fa4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.255+0000 7f9424036f40 1 -- 172.21.15.86:0/559836106 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:1 17.19 17:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x55c9e3fa4d50 con 0x55c9e3fa43c0 2024-04-20T21:30:41.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.256+0000 7f9420000640 1 --2- 172.21.15.86:0/559836106 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f940006b2d0 0x7f940006d790 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x55c9e3f19270 tx=0x7f940c0084b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:41.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.256+0000 7f9421400640 1 --2- 172.21.15.86:0/559836106 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x55c9e3fa43c0 0x55c9e3fa4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:41.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.256+0000 7f9421400640 1 --2- 172.21.15.86:0/559836106 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x55c9e3fa43c0 0x55c9e3fa4810 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:41.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.257+0000 7f9421400640 1 -- 172.21.15.86:0/559836106 <== osd.5 v2:172.21.15.144:6808/1635974704 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (12) out=12b] v0'0 uv23 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7f9418009660 con 0x55c9e3fa43c0 2024-04-20T21:30:41.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.257+0000 7f9424036f40 1 -- 172.21.15.86:0/559836106 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x55c9e3fa43c0 msgr2=0x55c9e3fa4810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:41.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.257+0000 7f9424036f40 1 --2- 172.21.15.86:0/559836106 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x55c9e3fa43c0 0x55c9e3fa4810 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.258+0000 7f9424036f40 1 -- 172.21.15.86:0/559836106 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f940006b2d0 msgr2=0x7f940006d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:41.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.258+0000 7f9424036f40 1 --2- 172.21.15.86:0/559836106 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f940006b2d0 0x7f940006d790 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x55c9e3f19270 tx=0x7f940c0084b0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.258+0000 7f9424036f40 1 -- 172.21.15.86:0/559836106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9e3effcf0 msgr2=0x55c9e3fa66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:41.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.258+0000 7f9424036f40 1 --2- 172.21.15.86:0/559836106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c9e3effcf0 0x55c9e3fa66d0 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f9410063040 tx=0x7f9410003330 comp rx=0 tx=0).stop 2024-04-20T21:30:41.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.258+0000 7f9421400640 1 -- 172.21.15.86:0/559836106 reap_dead start 2024-04-20T21:30:41.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.258+0000 7f9424036f40 1 -- 172.21.15.86:0/559836106 shutdown_connections 2024-04-20T21:30:41.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.258+0000 7f9424036f40 1 -- 172.21.15.86:0/559836106 >> 172.21.15.86:0/559836106 conn(0x55c9e3ece220 msgr2=0x55c9e3efcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:41.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.258+0000 7f9424036f40 1 -- 172.21.15.86:0/559836106 shutdown_connections 2024-04-20T21:30:41.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.258+0000 7f9424036f40 1 -- 172.21.15.86:0/559836106 wait complete. 2024-04-20T21:30:41.261 INFO:teuthology.orchestra.run.smithi086.stdout:mytag123 2024-04-20T21:30:41.261 DEBUG:teuthology.orchestra.run.smithi086:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000004.00000000 scrub_tag 2024-04-20T21:30:41.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.353+0000 7fb518272f40 1 Processor -- start 2024-04-20T21:30:41.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.353+0000 7fb518272f40 1 -- start start 2024-04-20T21:30:41.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.354+0000 7fb518272f40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56544e1cae30 0x56544e2571d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.354+0000 7fb518272f40 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56544e23d7e0 0x56544e239b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.354+0000 7fb518272f40 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56544e240ed0 0x56544e23fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.354+0000 7fb518272f40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56544e127cc0 con 0x56544e1cae30 2024-04-20T21:30:41.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.354+0000 7fb518272f40 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56544e12bd00 con 0x56544e23d7e0 2024-04-20T21:30:41.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.354+0000 7fb518272f40 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x56544e1cccf0 con 0x56544e240ed0 2024-04-20T21:30:41.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.354+0000 7fb515400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56544e1cae30 0x56544e2571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:41.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.355+0000 7fb515400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56544e1cae30 0x56544e2571d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58656/0 (socket says 172.21.15.86:58656) 2024-04-20T21:30:41.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.355+0000 7fb515400640 1 -- 172.21.15.86:0/2953394745 learned_addr learned my addr 172.21.15.86:0/2953394745 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:41.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.355+0000 7fb515400640 1 -- 172.21.15.86:0/2953394745 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56544e240ed0 msgr2=0x56544e23fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:41.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.355+0000 7fb515400640 1 --2- 172.21.15.86:0/2953394745 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56544e240ed0 0x56544e23fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.355+0000 7fb514a00640 1 --2- 172.21.15.86:0/2953394745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56544e23d7e0 0x56544e239b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:41.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.355+0000 7fb515400640 1 -- 172.21.15.86:0/2953394745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56544e23d7e0 msgr2=0x56544e239b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:41.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.355+0000 7fb515400640 1 --2- 172.21.15.86:0/2953394745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56544e23d7e0 0x56544e239b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.355+0000 7fb515400640 1 -- 172.21.15.86:0/2953394745 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56544e23ee40 con 0x56544e1cae30 2024-04-20T21:30:41.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.355+0000 7fb515400640 1 --2- 172.21.15.86:0/2953394745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56544e1cae30 0x56544e2571d0 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7fb50000b2e0 tx=0x7fb50000b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5fb71b245dffcf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:41.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.355+0000 7fb50f400640 1 -- 172.21.15.86:0/2953394745 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb500008070 con 0x56544e1cae30 2024-04-20T21:30:41.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.356+0000 7fb50f400640 1 -- 172.21.15.86:0/2953394745 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb50000d500 con 0x56544e1cae30 2024-04-20T21:30:41.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.356+0000 7fb50f400640 1 -- 172.21.15.86:0/2953394745 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50000d8f0 con 0x56544e1cae30 2024-04-20T21:30:41.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.356+0000 7fb518272f40 1 -- 172.21.15.86:0/2953394745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56544e1cae30 msgr2=0x56544e2571d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:41.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.356+0000 7fb518272f40 1 --2- 172.21.15.86:0/2953394745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56544e1cae30 0x56544e2571d0 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7fb50000b2e0 tx=0x7fb50000b7a0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.356+0000 7fb518272f40 1 -- 172.21.15.86:0/2953394745 shutdown_connections 2024-04-20T21:30:41.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.356+0000 7fb518272f40 1 --2- 172.21.15.86:0/2953394745 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56544e240ed0 0x56544e23fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.356+0000 7fb518272f40 1 --2- 172.21.15.86:0/2953394745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56544e23d7e0 0x56544e239b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.356+0000 7fb518272f40 1 --2- 172.21.15.86:0/2953394745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56544e1cae30 0x56544e2571d0 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.356+0000 7fb518272f40 1 -- 172.21.15.86:0/2953394745 >> 172.21.15.86:0/2953394745 conn(0x56544e185950 msgr2=0x56544e23de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:41.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.356+0000 7fb518272f40 1 -- 172.21.15.86:0/2953394745 shutdown_connections 2024-04-20T21:30:41.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.356+0000 7fb518272f40 1 -- 172.21.15.86:0/2953394745 wait complete. 2024-04-20T21:30:41.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.357+0000 7fb518272f40 1 Processor -- start 2024-04-20T21:30:41.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.357+0000 7fb518272f40 1 -- start start 2024-04-20T21:30:41.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.358+0000 7fb518272f40 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56544e1cae30 0x56544e23fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.358+0000 7fb518272f40 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56544e23d7e0 0x56544e240160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.358+0000 7fb518272f40 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56544e240ed0 0x56544e254720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.358+0000 7fb518272f40 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56544e2396b0 con 0x56544e1cae30 2024-04-20T21:30:41.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.358+0000 7fb518272f40 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x56544e238d00 con 0x56544e23d7e0 2024-04-20T21:30:41.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.358+0000 7fb518272f40 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x56544e23f0e0 con 0x56544e240ed0 2024-04-20T21:30:41.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.359+0000 7fb515400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56544e1cae30 0x56544e23fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:41.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.359+0000 7fb515400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56544e1cae30 0x56544e23fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58668/0 (socket says 172.21.15.86:58668) 2024-04-20T21:30:41.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.359+0000 7fb515400640 1 -- 172.21.15.86:0/3287288873 learned_addr learned my addr 172.21.15.86:0/3287288873 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:41.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.359+0000 7fb514a00640 1 --2- 172.21.15.86:0/3287288873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56544e23d7e0 0x56544e240160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:41.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.359+0000 7fb515400640 1 -- 172.21.15.86:0/3287288873 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56544e240ed0 msgr2=0x56544e254720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:41.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.359+0000 7fb515400640 1 --2- 172.21.15.86:0/3287288873 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x56544e240ed0 0x56544e254720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.359+0000 7fb515400640 1 -- 172.21.15.86:0/3287288873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56544e23d7e0 msgr2=0x56544e240160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:41.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.359+0000 7fb515400640 1 --2- 172.21.15.86:0/3287288873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x56544e23d7e0 0x56544e240160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.359+0000 7fb515400640 1 -- 172.21.15.86:0/3287288873 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56544e2337e0 con 0x56544e1cae30 2024-04-20T21:30:41.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.359+0000 7fb515400640 1 --2- 172.21.15.86:0/3287288873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56544e1cae30 0x56544e23fc20 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7fb500000c00 tx=0x7fb500000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:41.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.360+0000 7fb50e000640 1 -- 172.21.15.86:0/3287288873 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb500021030 con 0x56544e1cae30 2024-04-20T21:30:41.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.360+0000 7fb50e000640 1 -- 172.21.15.86:0/3287288873 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb50001c650 con 0x56544e1cae30 2024-04-20T21:30:41.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.360+0000 7fb518272f40 1 -- 172.21.15.86:0/3287288873 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56544e233ac0 con 0x56544e1cae30 2024-04-20T21:30:41.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.360+0000 7fb50e000640 1 -- 172.21.15.86:0/3287288873 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50001c8c0 con 0x56544e1cae30 2024-04-20T21:30:41.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.360+0000 7fb518272f40 1 -- 172.21.15.86:0/3287288873 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56544e253f20 con 0x56544e1cae30 2024-04-20T21:30:41.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.362+0000 7fb50e000640 1 -- 172.21.15.86:0/3287288873 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb500024070 con 0x56544e1cae30 2024-04-20T21:30:41.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.363+0000 7fb50e000640 1 --2- 172.21.15.86:0/3287288873 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb4f406b2d0 0x7fb4f406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.363+0000 7fb50e000640 1 -- 172.21.15.86:0/3287288873 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5853+0+0 (secure 0 0 0) 0x7fb50008bdb0 con 0x56544e1cae30 2024-04-20T21:30:41.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.363+0000 7fb514a00640 1 --2- 172.21.15.86:0/3287288873 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb4f406b2d0 0x7fb4f406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:41.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.363+0000 7fb518272f40 1 --2- 172.21.15.86:0/3287288873 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x56544e2311a0 0x56544e254c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.363+0000 7fb518272f40 1 -- 172.21.15.86:0/3287288873 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:1 17.5 17:a367a3e8:::10000000004.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e115) -- 0x56544e253a30 con 0x56544e2311a0 2024-04-20T21:30:41.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.364+0000 7fb514a00640 1 --2- 172.21.15.86:0/3287288873 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb4f406b2d0 0x7fb4f406d790 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fb508053000 tx=0x7fb50804e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:41.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.364+0000 7fb515e00640 1 --2- 172.21.15.86:0/3287288873 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x56544e2311a0 0x56544e254c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:41.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.365+0000 7fb515e00640 1 --2- 172.21.15.86:0/3287288873 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x56544e2311a0 0x56544e254c60 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:41.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.366+0000 7fb515e00640 1 -- 172.21.15.86:0/3287288873 <== osd.6 v2:172.21.15.144:6824/1893803079 1 ==== osd_op_reply(1 10000000004.00000000 [getxattr (12) out=12b] v0'0 uv12 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7fb51000b420 con 0x56544e2311a0 2024-04-20T21:30:41.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.366+0000 7fb518272f40 1 -- 172.21.15.86:0/3287288873 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x56544e2311a0 msgr2=0x56544e254c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:41.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.366+0000 7fb518272f40 1 --2- 172.21.15.86:0/3287288873 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x56544e2311a0 0x56544e254c60 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.366+0000 7fb518272f40 1 -- 172.21.15.86:0/3287288873 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb4f406b2d0 msgr2=0x7fb4f406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:41.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.366+0000 7fb518272f40 1 --2- 172.21.15.86:0/3287288873 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fb4f406b2d0 0x7fb4f406d790 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fb508053000 tx=0x7fb50804e000 comp rx=0 tx=0).stop 2024-04-20T21:30:41.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.366+0000 7fb518272f40 1 -- 172.21.15.86:0/3287288873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56544e1cae30 msgr2=0x56544e23fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:41.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.366+0000 7fb518272f40 1 --2- 172.21.15.86:0/3287288873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56544e1cae30 0x56544e23fc20 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7fb500000c00 tx=0x7fb500000ed0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.366+0000 7fb515e00640 1 -- 172.21.15.86:0/3287288873 reap_dead start 2024-04-20T21:30:41.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.366+0000 7fb518272f40 1 -- 172.21.15.86:0/3287288873 shutdown_connections 2024-04-20T21:30:41.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.366+0000 7fb518272f40 1 -- 172.21.15.86:0/3287288873 >> 172.21.15.86:0/3287288873 conn(0x56544e185950 msgr2=0x56544e23de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:41.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.366+0000 7fb518272f40 1 -- 172.21.15.86:0/3287288873 shutdown_connections 2024-04-20T21:30:41.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.366+0000 7fb518272f40 1 -- 172.21.15.86:0/3287288873 wait complete. 2024-04-20T21:30:41.370 INFO:teuthology.orchestra.run.smithi086.stdout:mytag123 2024-04-20T21:30:41.370 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-04-20T21:30:41.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.560+0000 7ff665a00640 1 Processor -- start 2024-04-20T21:30:41.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.560+0000 7ff665a00640 1 -- start start 2024-04-20T21:30:41.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.561+0000 7ff665a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff6600b3ba0 0x7ff66006eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.561+0000 7ff665a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff66006f3f0 0x7ff660071ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.561+0000 7ff665a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff66006f800 con 0x7ff6600cfc70 2024-04-20T21:30:41.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.562+0000 7ff665a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff66015c8e0 con 0x7ff66006f3f0 2024-04-20T21:30:41.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.562+0000 7ff665a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7ff6600cddb0 con 0x7ff6600b3ba0 2024-04-20T21:30:41.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.562+0000 7ff665000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff66006f3f0 0x7ff660071ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:41.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.562+0000 7ff665000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff66006f3f0 0x7ff660071ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58672/0 (socket says 172.21.15.86:58672) 2024-04-20T21:30:41.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.562+0000 7ff665000640 1 -- 172.21.15.86:0/1091673515 learned_addr learned my addr 172.21.15.86:0/1091673515 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:41.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.562+0000 7ff657400640 1 --2- 172.21.15.86:0/1091673515 >> v2:172.21.15.144:3300/0 conn(0x7ff6600b3ba0 0x7ff66006eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:41.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.562+0000 7ff665000640 1 -- 172.21.15.86:0/1091673515 >> v2:172.21.15.144:3300/0 conn(0x7ff6600b3ba0 msgr2=0x7ff66006eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:41.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.562+0000 7ff665000640 1 --2- 172.21.15.86:0/1091673515 >> v2:172.21.15.144:3300/0 conn(0x7ff6600b3ba0 0x7ff66006eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.562+0000 7ff665000640 1 -- 172.21.15.86:0/1091673515 >> v1:172.21.15.144:6789/0 conn(0x7ff6600cfc70 legacy=0x7ff6600cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:41.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.562+0000 7ff665000640 1 -- 172.21.15.86:0/1091673515 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff66015bcd0 con 0x7ff66006f3f0 2024-04-20T21:30:41.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.563+0000 7ff665000640 1 --2- 172.21.15.86:0/1091673515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff66006f3f0 0x7ff660071ef0 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7ff66006fb20 tx=0x7ff65802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de0731595c58849b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:41.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.564+0000 7ff65f400640 1 -- 172.21.15.86:0/1091673515 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65802ca80 con 0x7ff66006f3f0 2024-04-20T21:30:41.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.564+0000 7ff65f400640 1 -- 172.21.15.86:0/1091673515 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff65802cc20 con 0x7ff66006f3f0 2024-04-20T21:30:41.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.564+0000 7ff65f400640 1 -- 172.21.15.86:0/1091673515 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6580343f0 con 0x7ff66006f3f0 2024-04-20T21:30:41.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.565+0000 7ff665a00640 1 -- 172.21.15.86:0/1091673515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff66006f3f0 msgr2=0x7ff660071ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:41.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.565+0000 7ff665a00640 1 --2- 172.21.15.86:0/1091673515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff66006f3f0 0x7ff660071ef0 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7ff66006fb20 tx=0x7ff65802bee0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.565+0000 7ff665a00640 1 -- 172.21.15.86:0/1091673515 shutdown_connections 2024-04-20T21:30:41.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.565+0000 7ff665a00640 1 --2- 172.21.15.86:0/1091673515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff66006f3f0 0x7ff660071ef0 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.565+0000 7ff665a00640 1 --2- 172.21.15.86:0/1091673515 >> v2:172.21.15.144:3300/0 conn(0x7ff6600b3ba0 0x7ff66006eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.565+0000 7ff665a00640 1 -- 172.21.15.86:0/1091673515 >> 172.21.15.86:0/1091673515 conn(0x7ff660067cf0 msgr2=0x7ff66006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:41.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.565+0000 7ff665a00640 1 -- 172.21.15.86:0/1091673515 shutdown_connections 2024-04-20T21:30:41.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.565+0000 7ff665a00640 1 -- 172.21.15.86:0/1091673515 wait complete. 2024-04-20T21:30:41.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.567+0000 7ff665a00640 1 Processor -- start 2024-04-20T21:30:41.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.567+0000 7ff665a00640 1 -- start start 2024-04-20T21:30:41.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.568+0000 7ff665a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff66006f3f0 0x7ff6601556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.568+0000 7ff665a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6600b3ba0 0x7ff660072160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.568+0000 7ff665a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6600cfc70 0x7ff660149820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.568+0000 7ff665a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff66015c570 con 0x7ff6600cfc70 2024-04-20T21:30:41.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.568+0000 7ff665a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff66015c290 con 0x7ff66006f3f0 2024-04-20T21:30:41.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.568+0000 7ff665a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff6600b3710 con 0x7ff6600b3ba0 2024-04-20T21:30:41.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.568+0000 7ff665000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6600cfc70 0x7ff660149820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:41.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.568+0000 7ff665000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6600cfc70 0x7ff660149820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58680/0 (socket says 172.21.15.86:58680) 2024-04-20T21:30:41.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.568+0000 7ff665000640 1 -- 172.21.15.86:0/2599508019 learned_addr learned my addr 172.21.15.86:0/2599508019 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:41.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.569+0000 7ff657400640 1 --2- 172.21.15.86:0/2599508019 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6600b3ba0 0x7ff660072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:41.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.569+0000 7ff65fe00640 1 --2- 172.21.15.86:0/2599508019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff66006f3f0 0x7ff6601556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:41.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.569+0000 7ff665000640 1 -- 172.21.15.86:0/2599508019 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6600b3ba0 msgr2=0x7ff660072160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:41.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.569+0000 7ff665000640 1 --2- 172.21.15.86:0/2599508019 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6600b3ba0 0x7ff660072160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.569+0000 7ff665000640 1 -- 172.21.15.86:0/2599508019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff66006f3f0 msgr2=0x7ff6601556b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:41.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.569+0000 7ff665000640 1 --2- 172.21.15.86:0/2599508019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff66006f3f0 0x7ff6601556b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.569+0000 7ff665000640 1 -- 172.21.15.86:0/2599508019 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6601f78b0 con 0x7ff6600cfc70 2024-04-20T21:30:41.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.569+0000 7ff665000640 1 --2- 172.21.15.86:0/2599508019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6600cfc70 0x7ff660149820 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7ff658036040 tx=0x7ff658004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:41.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.569+0000 7ff65cc00640 1 -- 172.21.15.86:0/2599508019 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff658043020 con 0x7ff6600cfc70 2024-04-20T21:30:41.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.569+0000 7ff65cc00640 1 -- 172.21.15.86:0/2599508019 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff65803e450 con 0x7ff6600cfc70 2024-04-20T21:30:41.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.569+0000 7ff65cc00640 1 -- 172.21.15.86:0/2599508019 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff658009510 con 0x7ff6600cfc70 2024-04-20T21:30:41.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.570+0000 7ff665a00640 1 -- 172.21.15.86:0/2599508019 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6601f7b50 con 0x7ff6600cfc70 2024-04-20T21:30:41.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.570+0000 7ff665a00640 1 -- 172.21.15.86:0/2599508019 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6601f8110 con 0x7ff6600cfc70 2024-04-20T21:30:41.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.572+0000 7ff65cc00640 1 -- 172.21.15.86:0/2599508019 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff658046020 con 0x7ff6600cfc70 2024-04-20T21:30:41.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.573+0000 7ff65cc00640 1 --2- 172.21.15.86:0/2599508019 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff63c06c8f0 0x7ff63c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.573+0000 7ff65cc00640 1 -- 172.21.15.86:0/2599508019 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5853+0+0 (secure 0 0 0) 0x7ff6580ae500 con 0x7ff6600cfc70 2024-04-20T21:30:41.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.573+0000 7ff65fe00640 1 --2- 172.21.15.86:0/2599508019 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff63c06c8f0 0x7ff63c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:41.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.573+0000 7ff669bfe740 1 Processor -- start 2024-04-20T21:30:41.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.574+0000 7ff669bfe740 1 -- start start 2024-04-20T21:30:41.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.574+0000 7ff65fe00640 1 --2- 172.21.15.86:0/2599508019 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff63c06c8f0 0x7ff63c06edb0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7ff650004ea0 tx=0x7ff650009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:41.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.574+0000 7ff669bfe740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638b4dc83d0 0x5638b4dc87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.574+0000 7ff669bfe740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b4d983f0 0x5638b4dc5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.575+0000 7ff669bfe740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5638b4d9bf50 0x5638b4dc5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.575+0000 7ff669bfe740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5638b4d04150 con 0x5638b4d983f0 2024-04-20T21:30:41.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.575+0000 7ff669bfe740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5638b4caa370 con 0x5638b4dc83d0 2024-04-20T21:30:41.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.575+0000 7ff669bfe740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5638b4d00110 con 0x5638b4d9bf50 2024-04-20T21:30:41.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.575+0000 7ff657400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b4d983f0 0x5638b4dc5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:41.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.575+0000 7ff657400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b4d983f0 0x5638b4dc5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58682/0 (socket says 172.21.15.86:58682) 2024-04-20T21:30:41.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.575+0000 7ff657400640 1 -- 172.21.15.86:0/2159795313 learned_addr learned my addr 172.21.15.86:0/2159795313 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:41.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.575+0000 7ff657400640 1 -- 172.21.15.86:0/2159795313 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5638b4d9bf50 msgr2=0x5638b4dc5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:41.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.575+0000 7ff657400640 1 --2- 172.21.15.86:0/2159795313 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5638b4d9bf50 0x5638b4dc5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.575+0000 7ff657400640 1 -- 172.21.15.86:0/2159795313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638b4dc83d0 msgr2=0x5638b4dc87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:41.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.575+0000 7ff657400640 1 --2- 172.21.15.86:0/2159795313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638b4dc83d0 0x5638b4dc87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.575+0000 7ff657400640 1 -- 172.21.15.86:0/2159795313 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638b4ae07d0 con 0x5638b4d983f0 2024-04-20T21:30:41.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.576+0000 7ff657400640 1 --2- 172.21.15.86:0/2159795313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b4d983f0 0x5638b4dc5640 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7ff64800b8a0 tx=0x7ff64800bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8b2bba139b1605d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:41.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.576+0000 7ff654c00640 1 -- 172.21.15.86:0/2159795313 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff648018020 con 0x5638b4d983f0 2024-04-20T21:30:41.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.576+0000 7ff654c00640 1 -- 172.21.15.86:0/2159795313 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff64800f9c0 con 0x5638b4d983f0 2024-04-20T21:30:41.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.576+0000 7ff654c00640 1 -- 172.21.15.86:0/2159795313 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64800fd90 con 0x5638b4d983f0 2024-04-20T21:30:41.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.576+0000 7ff669bfe740 1 -- 172.21.15.86:0/2159795313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b4d983f0 msgr2=0x5638b4dc5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:41.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.576+0000 7ff669bfe740 1 --2- 172.21.15.86:0/2159795313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b4d983f0 0x5638b4dc5640 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7ff64800b8a0 tx=0x7ff64800bd60 comp rx=0 tx=0).stop 2024-04-20T21:30:41.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.577+0000 7ff669bfe740 1 -- 172.21.15.86:0/2159795313 shutdown_connections 2024-04-20T21:30:41.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.577+0000 7ff669bfe740 1 --2- 172.21.15.86:0/2159795313 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5638b4d9bf50 0x5638b4dc5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.577+0000 7ff669bfe740 1 --2- 172.21.15.86:0/2159795313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b4d983f0 0x5638b4dc5640 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.577+0000 7ff669bfe740 1 --2- 172.21.15.86:0/2159795313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638b4dc83d0 0x5638b4dc87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.577+0000 7ff669bfe740 1 -- 172.21.15.86:0/2159795313 >> 172.21.15.86:0/2159795313 conn(0x5638b4d9a660 msgr2=0x5638b4de31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:41.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.577+0000 7ff669bfe740 1 -- 172.21.15.86:0/2159795313 shutdown_connections 2024-04-20T21:30:41.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.577+0000 7ff669bfe740 1 -- 172.21.15.86:0/2159795313 wait complete. 2024-04-20T21:30:41.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.577+0000 7ff669bfe740 1 -- start start 2024-04-20T21:30:41.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.578+0000 7ff669bfe740 1 Processor -- start 2024-04-20T21:30:41.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.579+0000 7ff669bfe740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b4d9bf50 0x5638b4dd9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.579+0000 7ff669bfe740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638b4dc83d0 0x5638b4ddb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.579+0000 7ff669bfe740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5638b4dc4ca0 0x5638b4ddc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.579+0000 7ff669bfe740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5638b4d9d100 con 0x5638b4d9bf50 2024-04-20T21:30:41.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.579+0000 7ff669bfe740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5638b4cbd740 con 0x5638b4dc83d0 2024-04-20T21:30:41.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.579+0000 7ff669bfe740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5638b4d9c4f0 con 0x5638b4dc4ca0 2024-04-20T21:30:41.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.579+0000 7ff669bfe740 10 client.0 _ll_register_callbacks cb 0x5638b4cb8b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:30:41.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.579+0000 7ff65fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b4d9bf50 0x5638b4dd9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:41.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.579+0000 7ff65fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b4d9bf50 0x5638b4dd9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58686/0 (socket says 172.21.15.86:58686) 2024-04-20T21:30:41.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.579+0000 7ff65fe00640 1 -- 172.21.15.86:0/3910687353 learned_addr learned my addr 172.21.15.86:0/3910687353 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:41.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.579+0000 7ff65fe00640 1 -- 172.21.15.86:0/3910687353 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5638b4dc4ca0 msgr2=0x5638b4ddc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:41.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.579+0000 7ff65fe00640 1 --2- 172.21.15.86:0/3910687353 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5638b4dc4ca0 0x5638b4ddc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.579+0000 7ff65fe00640 1 -- 172.21.15.86:0/3910687353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638b4dc83d0 msgr2=0x5638b4ddb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:41.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.579+0000 7ff65fe00640 1 --2- 172.21.15.86:0/3910687353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638b4dc83d0 0x5638b4ddb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.579+0000 7ff65fe00640 1 -- 172.21.15.86:0/3910687353 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638b4a163f0 con 0x5638b4d9bf50 2024-04-20T21:30:41.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.580+0000 7ff65fe00640 1 --2- 172.21.15.86:0/3910687353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b4d9bf50 0x5638b4dd9dc0 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7ff65000edc0 tx=0x7ff650015800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c17502a6cdbd82 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:41.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.580+0000 7ff64d600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:30:41.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.580+0000 7ff64d600640 1 -- 172.21.15.86:0/3910687353 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff650025060 con 0x5638b4d9bf50 2024-04-20T21:30:41.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.580+0000 7ff64d600640 1 -- 172.21.15.86:0/3910687353 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 5853+0+0 (secure 0 0 0) 0x7ff650020400 con 0x5638b4d9bf50 2024-04-20T21:30:41.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.580+0000 7ff669bfe740 1 -- 172.21.15.86:0/3910687353 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff64800f9c0 con 0x5638b4d9bf50 2024-04-20T21:30:41.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.580+0000 7ff64d600640 1 -- 172.21.15.86:0/3910687353 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) ==== 24+0+0 (secure 0 0 0) 0x7ff65002b050 con 0x5638b4d9bf50 2024-04-20T21:30:41.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.580+0000 7ff669bfe740 10 client.9771 fetch_fsmap learned FSMap version 105 2024-04-20T21:30:41.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.580+0000 7ff669bfe740 1 -- 172.21.15.86:0/3910687353 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) -- 0x5638b49d60f0 con 0x5638b4d9bf50 2024-04-20T21:30:41.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.580+0000 7ff64d600640 1 -- 172.21.15.86:0/3910687353 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff64800f9c0 con 0x5638b4d9bf50 2024-04-20T21:30:41.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.580+0000 7ff64d600640 1 -- 172.21.15.86:0/3910687353 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 105) ==== 3495+0+0 (secure 0 0 0) 0x7ff65002e020 con 0x5638b4d9bf50 2024-04-20T21:30:41.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.580+0000 7ff669bfe740 10 client.9771 fetch_fsmap finished waiting for FSMap version 105 2024-04-20T21:30:41.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.580+0000 7ff669bfe740 10 client.9771 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-20T21:30:41.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.581+0000 7ff669bfe740 20 client.9771 populate_metadata read hostname 'smithi086' 2024-04-20T21:30:41.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.581+0000 7ff669bfe740 1 --2- 172.21.15.86:0/3910687353 >> [v2:172.21.15.86:6835/1453621918,v1:172.21.15.86:6837/1453621918] conn(0x5638b4db5810 0x5638b4dc9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.581+0000 7ff669bfe740 4 client.9771 mds_command: new command op to 9585 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:30:41.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.581+0000 7ff669bfe740 1 -- 172.21.15.86:0/3910687353 --> [v2:172.21.15.86:6835/1453621918,v1:172.21.15.86:6837/1453621918] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638b4d11b70 con 0x5638b4db5810 2024-04-20T21:30:41.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.581+0000 7ff657400640 1 --2- 172.21.15.86:0/3910687353 >> [v2:172.21.15.86:6835/1453621918,v1:172.21.15.86:6837/1453621918] conn(0x5638b4db5810 0x5638b4dc9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:41.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.581+0000 7ff657400640 1 --2- 172.21.15.86:0/3910687353 >> [v2:172.21.15.86:6835/1453621918,v1:172.21.15.86:6837/1453621918] conn(0x5638b4db5810 0x5638b4dc9620 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d96e649cd768dbe7 server_cookie=8e6bd4ec5922ef7f in_seq=0 out_seq=0 2024-04-20T21:30:41.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.581+0000 7ff64d600640 10 client.9771 ms_handle_connect on v2:172.21.15.86:6835/1453621918 2024-04-20T21:30:41.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.582+0000 7ff64d600640 1 -- 172.21.15.86:0/3910687353 <== mds.0 v2:172.21.15.86:6835/1453621918 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5638b4d11b70 con 0x5638b4db5810 2024-04-20T21:30:41.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.582+0000 7ff64d600640 10 client.9771 handle_command_reply: tid=0 2024-04-20T21:30:41.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.582+0000 7ff669bfe740 1 client.9771 shutdown 2024-04-20T21:30:41.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.582+0000 7ff669bfe740 1 -- 172.21.15.86:0/3910687353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b4d9bf50 msgr2=0x5638b4dd9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:41.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.582+0000 7ff669bfe740 1 --2- 172.21.15.86:0/3910687353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b4d9bf50 0x5638b4dd9dc0 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7ff65000edc0 tx=0x7ff650015800 comp rx=0 tx=0).stop 2024-04-20T21:30:41.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.582+0000 7ff669bfe740 1 -- 172.21.15.86:0/3910687353 shutdown_connections 2024-04-20T21:30:41.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.582+0000 7ff669bfe740 1 --2- 172.21.15.86:0/3910687353 >> [v2:172.21.15.86:6835/1453621918,v1:172.21.15.86:6837/1453621918] conn(0x5638b4db5810 0x5638b4dc9620 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.582+0000 7ff669bfe740 1 --2- 172.21.15.86:0/3910687353 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5638b4dc4ca0 0x5638b4ddc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.582+0000 7ff669bfe740 1 --2- 172.21.15.86:0/3910687353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638b4dc83d0 0x5638b4ddb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.582+0000 7ff669bfe740 1 --2- 172.21.15.86:0/3910687353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b4d9bf50 0x5638b4dd9dc0 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.582+0000 7ff64d600640 0 client.9771 ms_handle_reset on v2:172.21.15.86:6835/1453621918 2024-04-20T21:30:41.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.582+0000 7ff669bfe740 1 -- 172.21.15.86:0/3910687353 >> 172.21.15.86:0/3910687353 conn(0x5638b4d983f0 msgr2=0x5638b4d9d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:41.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.583+0000 7ff669bfe740 1 -- 172.21.15.86:0/3910687353 shutdown_connections 2024-04-20T21:30:41.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.583+0000 7ff669bfe740 1 -- 172.21.15.86:0/3910687353 wait complete. 2024-04-20T21:30:41.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.583+0000 7ff669bfe740 20 client.9771 trim_cache size 0 max 16384 2024-04-20T21:30:41.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.599+0000 7ff669bfe740 1 Processor -- start 2024-04-20T21:30:41.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.599+0000 7ff669bfe740 1 -- start start 2024-04-20T21:30:41.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.600+0000 7ff669bfe740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5638b4d9bf50 0x5638b4ddb720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.600+0000 7ff669bfe740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b4dc4ca0 0x5638b4dcb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.600+0000 7ff669bfe740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638b4dc83d0 0x5638b4dc5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.600+0000 7ff669bfe740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff660200e90 con 0x5638b4dc4ca0 2024-04-20T21:30:41.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.600+0000 7ff669bfe740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff6601ffe90 con 0x5638b4dc83d0 2024-04-20T21:30:41.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.600+0000 7ff669bfe740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff6600a9ef0 con 0x5638b4d9bf50 2024-04-20T21:30:41.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.600+0000 7ff657400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b4dc4ca0 0x5638b4dcb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:41.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.600+0000 7ff657400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b4dc4ca0 0x5638b4dcb810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58690/0 (socket says 172.21.15.86:58690) 2024-04-20T21:30:41.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.600+0000 7ff657400640 1 -- 172.21.15.86:0/3393193833 learned_addr learned my addr 172.21.15.86:0/3393193833 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:41.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.601+0000 7ff657400640 1 -- 172.21.15.86:0/3393193833 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5638b4d9bf50 msgr2=0x5638b4ddb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:41.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.601+0000 7ff657400640 1 --2- 172.21.15.86:0/3393193833 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5638b4d9bf50 0x5638b4ddb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.601+0000 7ff657400640 1 -- 172.21.15.86:0/3393193833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638b4dc83d0 msgr2=0x5638b4dc5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:41.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.601+0000 7ff657400640 1 --2- 172.21.15.86:0/3393193833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638b4dc83d0 0x5638b4dc5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.601+0000 7ff657400640 1 -- 172.21.15.86:0/3393193833 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638b4d9dd50 con 0x5638b4dc4ca0 2024-04-20T21:30:41.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.601+0000 7ff657400640 1 --2- 172.21.15.86:0/3393193833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b4dc4ca0 0x5638b4dcb810 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7ff648000c00 tx=0x7ff648009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63585d616f554f2f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:41.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.601+0000 7ff64fe00640 1 -- 172.21.15.86:0/3393193833 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff648018070 con 0x5638b4dc4ca0 2024-04-20T21:30:41.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.601+0000 7ff64fe00640 1 -- 172.21.15.86:0/3393193833 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff64802d070 con 0x5638b4dc4ca0 2024-04-20T21:30:41.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.601+0000 7ff64fe00640 1 -- 172.21.15.86:0/3393193833 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64800fe30 con 0x5638b4dc4ca0 2024-04-20T21:30:41.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.602+0000 7ff669bfe740 1 -- 172.21.15.86:0/3393193833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b4dc4ca0 msgr2=0x5638b4dcb810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:41.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.602+0000 7ff669bfe740 1 --2- 172.21.15.86:0/3393193833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b4dc4ca0 0x5638b4dcb810 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7ff648000c00 tx=0x7ff648009000 comp rx=0 tx=0).stop 2024-04-20T21:30:41.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.602+0000 7ff669bfe740 1 -- 172.21.15.86:0/3393193833 shutdown_connections 2024-04-20T21:30:41.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.602+0000 7ff669bfe740 1 --2- 172.21.15.86:0/3393193833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638b4dc83d0 0x5638b4dc5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.602+0000 7ff669bfe740 1 --2- 172.21.15.86:0/3393193833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b4dc4ca0 0x5638b4dcb810 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.602+0000 7ff669bfe740 1 --2- 172.21.15.86:0/3393193833 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5638b4d9bf50 0x5638b4ddb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.602+0000 7ff669bfe740 1 -- 172.21.15.86:0/3393193833 >> 172.21.15.86:0/3393193833 conn(0x5638b4d983f0 msgr2=0x5638b4ddc500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:41.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.602+0000 7ff669bfe740 1 -- 172.21.15.86:0/3393193833 shutdown_connections 2024-04-20T21:30:41.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.602+0000 7ff669bfe740 1 -- 172.21.15.86:0/3393193833 wait complete. 2024-04-20T21:30:41.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.603+0000 7ff669bfe740 1 -- start start 2024-04-20T21:30:41.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.603+0000 7ff669bfe740 1 Processor -- start 2024-04-20T21:30:41.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.604+0000 7ff669bfe740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5638b4d9bf50 0x5638b4e43b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.604+0000 7ff669bfe740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b4dc4ca0 0x5638b4dc3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.604+0000 7ff669bfe740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638b4dc83d0 0x5638b4dcb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.604+0000 7ff669bfe740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5638b4cca060 con 0x5638b4dc4ca0 2024-04-20T21:30:41.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.604+0000 7ff669bfe740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff6600aa540 con 0x5638b4dc83d0 2024-04-20T21:30:41.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.604+0000 7ff669bfe740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5638b4db5190 con 0x5638b4d9bf50 2024-04-20T21:30:41.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.604+0000 7ff669bfe740 10 client.0 _ll_register_callbacks cb 0x7ff650007e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:30:41.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.604+0000 7ff657400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b4dc4ca0 0x5638b4dc3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:41.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.604+0000 7ff657400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b4dc4ca0 0x5638b4dc3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58702/0 (socket says 172.21.15.86:58702) 2024-04-20T21:30:41.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.604+0000 7ff657400640 1 -- 172.21.15.86:0/402328492 learned_addr learned my addr 172.21.15.86:0/402328492 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:41.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.604+0000 7ff657400640 1 -- 172.21.15.86:0/402328492 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5638b4d9bf50 msgr2=0x5638b4e43b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:41.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.604+0000 7ff657400640 1 --2- 172.21.15.86:0/402328492 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5638b4d9bf50 0x5638b4e43b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.604+0000 7ff657400640 1 -- 172.21.15.86:0/402328492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638b4dc83d0 msgr2=0x5638b4dcb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:41.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.604+0000 7ff657400640 1 --2- 172.21.15.86:0/402328492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638b4dc83d0 0x5638b4dcb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.604+0000 7ff657400640 1 -- 172.21.15.86:0/402328492 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638b4e46fa0 con 0x5638b4dc4ca0 2024-04-20T21:30:41.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.605+0000 7ff657400640 1 --2- 172.21.15.86:0/402328492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b4dc4ca0 0x5638b4dc3900 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7ff64802a040 tx=0x7ff648004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84a55717735e86e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:41.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.605+0000 7ff64d600640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:30:41.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.605+0000 7ff64d600640 1 -- 172.21.15.86:0/402328492 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff648035020 con 0x5638b4dc4ca0 2024-04-20T21:30:41.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.605+0000 7ff64d600640 1 -- 172.21.15.86:0/402328492 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 5853+0+0 (secure 0 0 0) 0x7ff648004040 con 0x5638b4dc4ca0 2024-04-20T21:30:41.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.605+0000 7ff669bfe740 1 -- 172.21.15.86:0/402328492 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff64802d070 con 0x5638b4dc4ca0 2024-04-20T21:30:41.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.605+0000 7ff64d600640 1 -- 172.21.15.86:0/402328492 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) ==== 24+0+0 (secure 0 0 0) 0x7ff64803f050 con 0x5638b4dc4ca0 2024-04-20T21:30:41.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.605+0000 7ff669bfe740 10 client.9777 fetch_fsmap learned FSMap version 105 2024-04-20T21:30:41.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.605+0000 7ff669bfe740 1 -- 172.21.15.86:0/402328492 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) -- 0x5638b4e61510 con 0x5638b4dc4ca0 2024-04-20T21:30:41.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.606+0000 7ff64d600640 1 -- 172.21.15.86:0/402328492 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff64802d070 con 0x5638b4dc4ca0 2024-04-20T21:30:41.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.606+0000 7ff64d600640 1 -- 172.21.15.86:0/402328492 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 105) ==== 3495+0+0 (secure 0 0 0) 0x7ff648042020 con 0x5638b4dc4ca0 2024-04-20T21:30:41.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.606+0000 7ff669bfe740 10 client.9777 fetch_fsmap finished waiting for FSMap version 105 2024-04-20T21:30:41.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.606+0000 7ff669bfe740 10 client.9777 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-20T21:30:41.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.606+0000 7ff669bfe740 20 client.9777 populate_metadata read hostname 'smithi086' 2024-04-20T21:30:41.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.606+0000 7ff669bfe740 1 --2- 172.21.15.86:0/402328492 >> [v2:172.21.15.86:6835/1453621918,v1:172.21.15.86:6837/1453621918] conn(0x5638b4de93e0 0x5638b4dcbf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.606+0000 7ff669bfe740 4 client.9777 mds_command: new command op to 9585 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-20T21:30:41.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.606+0000 7ff669bfe740 1 -- 172.21.15.86:0/402328492 --> [v2:172.21.15.86:6835/1453621918,v1:172.21.15.86:6837/1453621918] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5638b4d0b9e0 con 0x5638b4de93e0 2024-04-20T21:30:41.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.606+0000 7ff65fe00640 1 --2- 172.21.15.86:0/402328492 >> [v2:172.21.15.86:6835/1453621918,v1:172.21.15.86:6837/1453621918] conn(0x5638b4de93e0 0x5638b4dcbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:41.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.607+0000 7ff65fe00640 1 --2- 172.21.15.86:0/402328492 >> [v2:172.21.15.86:6835/1453621918,v1:172.21.15.86:6837/1453621918] conn(0x5638b4de93e0 0x5638b4dcbf00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=403f016ecff33df0 server_cookie=4fa5fd9ef6c50542 in_seq=0 out_seq=0 2024-04-20T21:30:41.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.607+0000 7ff64d600640 10 client.9777 ms_handle_connect on v2:172.21.15.86:6835/1453621918 2024-04-20T21:30:41.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.619+0000 7ff64d600640 1 -- 172.21.15.86:0/402328492 <== mds.0 v2:172.21.15.86:6835/1453621918 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5638b4d0b9e0 con 0x5638b4de93e0 2024-04-20T21:30:41.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.619+0000 7ff64d600640 10 client.9777 handle_command_reply: tid=0 2024-04-20T21:30:41.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.619+0000 7ff669bfe740 1 client.9777 shutdown 2024-04-20T21:30:41.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.619+0000 7ff669bfe740 1 -- 172.21.15.86:0/402328492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b4dc4ca0 msgr2=0x5638b4dc3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:41.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.619+0000 7ff669bfe740 1 --2- 172.21.15.86:0/402328492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b4dc4ca0 0x5638b4dc3900 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7ff64802a040 tx=0x7ff648004730 comp rx=0 tx=0).stop 2024-04-20T21:30:41.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.619+0000 7ff669bfe740 1 -- 172.21.15.86:0/402328492 shutdown_connections 2024-04-20T21:30:41.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.619+0000 7ff669bfe740 1 --2- 172.21.15.86:0/402328492 >> [v2:172.21.15.86:6835/1453621918,v1:172.21.15.86:6837/1453621918] conn(0x5638b4de93e0 0x5638b4dcbf00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.619+0000 7ff669bfe740 1 --2- 172.21.15.86:0/402328492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5638b4dc83d0 0x5638b4dcb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.619+0000 7ff669bfe740 1 --2- 172.21.15.86:0/402328492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5638b4dc4ca0 0x5638b4dc3900 unknown :-1 s=CLOSED pgs=2031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.619+0000 7ff669bfe740 1 --2- 172.21.15.86:0/402328492 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5638b4d9bf50 0x5638b4e43b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.619+0000 7ff669bfe740 1 -- 172.21.15.86:0/402328492 >> 172.21.15.86:0/402328492 conn(0x5638b4d983f0 msgr2=0x5638b4dc53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:41.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.619+0000 7ff64d600640 0 client.9777 ms_handle_reset on v2:172.21.15.86:6835/1453621918 2024-04-20T21:30:41.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.619+0000 7ff669bfe740 1 -- 172.21.15.86:0/402328492 shutdown_connections 2024-04-20T21:30:41.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.619+0000 7ff669bfe740 1 -- 172.21.15.86:0/402328492 wait complete. 2024-04-20T21:30:41.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.619+0000 7ff669bfe740 20 client.9777 trim_cache size 0 max 16384 2024-04-20T21:30:41.619 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:30:41.620 INFO:teuthology.orchestra.run.smithi086.stdout: "message": "", 2024-04-20T21:30:41.620 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0 2024-04-20T21:30:41.620 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:30:41.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.620+0000 7ff665a00640 1 -- 172.21.15.86:0/2599508019 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff63c06c8f0 msgr2=0x7ff63c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:41.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.620+0000 7ff665a00640 1 --2- 172.21.15.86:0/2599508019 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff63c06c8f0 0x7ff63c06edb0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7ff650004ea0 tx=0x7ff650009040 comp rx=0 tx=0).stop 2024-04-20T21:30:41.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.620+0000 7ff665a00640 1 -- 172.21.15.86:0/2599508019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6600cfc70 msgr2=0x7ff660149820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:41.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.620+0000 7ff665a00640 1 --2- 172.21.15.86:0/2599508019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6600cfc70 0x7ff660149820 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7ff658036040 tx=0x7ff658004a40 comp rx=0 tx=0).stop 2024-04-20T21:30:41.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.620+0000 7ff665a00640 1 -- 172.21.15.86:0/2599508019 shutdown_connections 2024-04-20T21:30:41.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.620+0000 7ff665a00640 1 --2- 172.21.15.86:0/2599508019 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff63c06c8f0 0x7ff63c06edb0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.620+0000 7ff665a00640 1 --2- 172.21.15.86:0/2599508019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff6600cfc70 0x7ff660149820 unknown :-1 s=CLOSED pgs=2027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.620+0000 7ff665a00640 1 --2- 172.21.15.86:0/2599508019 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff6600b3ba0 0x7ff660072160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.620+0000 7ff665a00640 1 --2- 172.21.15.86:0/2599508019 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff66006f3f0 0x7ff6601556b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.620+0000 7ff665a00640 1 -- 172.21.15.86:0/2599508019 >> 172.21.15.86:0/2599508019 conn(0x7ff660067cf0 msgr2=0x7ff66006ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:41.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.620+0000 7ff665a00640 1 -- 172.21.15.86:0/2599508019 shutdown_connections 2024-04-20T21:30:41.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.620+0000 7ff665a00640 1 -- 172.21.15.86:0/2599508019 wait complete. 2024-04-20T21:30:41.636 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-20T21:30:41.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.823+0000 7f8270600640 1 Processor -- start 2024-04-20T21:30:41.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.823+0000 7f8270600640 1 -- start start 2024-04-20T21:30:41.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.825+0000 7f8270600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8268162bd0 0x7f82680cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.825+0000 7f8270600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f826806e1e0 0x7f826806e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.825+0000 7f8270600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f826806eb20 con 0x7f82680b3330 2024-04-20T21:30:41.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.825+0000 7f8270600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f82680cd5e0 con 0x7f8268162bd0 2024-04-20T21:30:41.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.825+0000 7f8270600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f82680b3830 con 0x7f826806e1e0 2024-04-20T21:30:41.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.825+0000 7f826d200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f82680b3330 0x7f82680d0ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41200/0 (socket says 172.21.15.86:41200) 2024-04-20T21:30:41.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.825+0000 7f826d200640 1 -- 172.21.15.86:0/2258781773 learned_addr learned my addr 172.21.15.86:0/2258781773 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:30:41.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.826+0000 7f826dc00640 1 --2- 172.21.15.86:0/2258781773 >> v2:172.21.15.86:3300/0 conn(0x7f8268162bd0 0x7f82680cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:41.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.826+0000 7f826e600640 1 --2- 172.21.15.86:0/2258781773 >> v2:172.21.15.144:3300/0 conn(0x7f826806e1e0 0x7f826806e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:41.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.826+0000 7f8267e00640 1 -- 172.21.15.86:0/2258781773 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 580562839 0 0) 0x7f826806eb20 con 0x7f82680b3330 2024-04-20T21:30:41.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.826+0000 7f8267e00640 1 -- 172.21.15.86:0/2258781773 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8250003520 con 0x7f82680b3330 2024-04-20T21:30:41.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.826+0000 7f826dc00640 1 -- 172.21.15.86:0/2258781773 >> v2:172.21.15.144:3300/0 conn(0x7f826806e1e0 msgr2=0x7f826806e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:41.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.826+0000 7f826dc00640 1 --2- 172.21.15.86:0/2258781773 >> v2:172.21.15.144:3300/0 conn(0x7f826806e1e0 0x7f826806e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.826+0000 7f826dc00640 1 -- 172.21.15.86:0/2258781773 >> v1:172.21.15.86:6789/0 conn(0x7f82680b3330 legacy=0x7f82680d0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:41.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.826+0000 7f826dc00640 1 -- 172.21.15.86:0/2258781773 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82680b3fc0 con 0x7f8268162bd0 2024-04-20T21:30:41.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.827+0000 7f826dc00640 1 --2- 172.21.15.86:0/2258781773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8268162bd0 0x7f82680cd0a0 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f82680aff40 tx=0x7f825c02b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7685c872b87831d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:41.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.828+0000 7f8267e00640 1 -- 172.21.15.86:0/2258781773 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c038070 con 0x7f8268162bd0 2024-04-20T21:30:41.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.828+0000 7f8267e00640 1 -- 172.21.15.86:0/2258781773 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f825c031710 con 0x7f8268162bd0 2024-04-20T21:30:41.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.828+0000 7f8267e00640 1 -- 172.21.15.86:0/2258781773 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c031ae0 con 0x7f8268162bd0 2024-04-20T21:30:41.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.828+0000 7f8270600640 1 -- 172.21.15.86:0/2258781773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8268162bd0 msgr2=0x7f82680cd0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:41.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.828+0000 7f8270600640 1 --2- 172.21.15.86:0/2258781773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8268162bd0 0x7f82680cd0a0 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f82680aff40 tx=0x7f825c02b7b0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.828+0000 7f8270600640 1 -- 172.21.15.86:0/2258781773 shutdown_connections 2024-04-20T21:30:41.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.828+0000 7f8270600640 1 --2- 172.21.15.86:0/2258781773 >> v2:172.21.15.144:3300/0 conn(0x7f826806e1e0 0x7f826806e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.828+0000 7f8270600640 1 --2- 172.21.15.86:0/2258781773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8268162bd0 0x7f82680cd0a0 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.828+0000 7f8270600640 1 -- 172.21.15.86:0/2258781773 >> 172.21.15.86:0/2258781773 conn(0x7f826806b980 msgr2=0x7f826806bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:41.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.828+0000 7f8270600640 1 -- 172.21.15.86:0/2258781773 shutdown_connections 2024-04-20T21:30:41.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.828+0000 7f8270600640 1 -- 172.21.15.86:0/2258781773 wait complete. 2024-04-20T21:30:41.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.830+0000 7f8270600640 1 Processor -- start 2024-04-20T21:30:41.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.830+0000 7f8270600640 1 -- start start 2024-04-20T21:30:41.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.831+0000 7f8270600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f826806e1e0 0x7f826815e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.831+0000 7f8270600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82680b3330 0x7f8268161ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.831+0000 7f8270600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8268162bd0 0x7f826814cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.831+0000 7f8270600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8268162fb0 con 0x7f8268162bd0 2024-04-20T21:30:41.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.831+0000 7f8270600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f82680b4580 con 0x7f826806e1e0 2024-04-20T21:30:41.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.831+0000 7f8270600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8268163130 con 0x7f82680b3330 2024-04-20T21:30:41.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.831+0000 7f826e600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8268162bd0 0x7f826814cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:41.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.831+0000 7f826e600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8268162bd0 0x7f826814cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58714/0 (socket says 172.21.15.86:58714) 2024-04-20T21:30:41.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.831+0000 7f826e600640 1 -- 172.21.15.86:0/143228904 learned_addr learned my addr 172.21.15.86:0/143228904 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:41.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.832+0000 7f826d200640 1 --2- 172.21.15.86:0/143228904 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82680b3330 0x7f8268161ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:41.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.832+0000 7f826dc00640 1 --2- 172.21.15.86:0/143228904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f826806e1e0 0x7f826815e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:41.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.832+0000 7f826e600640 1 -- 172.21.15.86:0/143228904 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82680b3330 msgr2=0x7f8268161ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:41.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.832+0000 7f826e600640 1 --2- 172.21.15.86:0/143228904 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82680b3330 0x7f8268161ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.832+0000 7f826e600640 1 -- 172.21.15.86:0/143228904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f826806e1e0 msgr2=0x7f826815e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:41.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.832+0000 7f826e600640 1 --2- 172.21.15.86:0/143228904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f826806e1e0 0x7f826815e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:41.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.832+0000 7f826e600640 1 -- 172.21.15.86:0/143228904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8268162430 con 0x7f8268162bd0 2024-04-20T21:30:41.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.832+0000 7f826d200640 1 --2- 172.21.15.86:0/143228904 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82680b3330 0x7f8268161ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:30:41.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.832+0000 7f826e600640 1 --2- 172.21.15.86:0/143228904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8268162bd0 0x7f826814cee0 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f8260007e70 tx=0x7f826000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:41.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.832+0000 7f8265600640 1 -- 172.21.15.86:0/143228904 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8260014620 con 0x7f8268162bd0 2024-04-20T21:30:41.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.832+0000 7f8265600640 1 -- 172.21.15.86:0/143228904 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8260019070 con 0x7f8268162bd0 2024-04-20T21:30:41.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.832+0000 7f8270600640 1 -- 172.21.15.86:0/143228904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f826815ec10 con 0x7f8268162bd0 2024-04-20T21:30:41.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.832+0000 7f8265600640 1 -- 172.21.15.86:0/143228904 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8260014a20 con 0x7f8268162bd0 2024-04-20T21:30:41.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.833+0000 7f8270600640 1 -- 172.21.15.86:0/143228904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f826815f4b0 con 0x7f8268162bd0 2024-04-20T21:30:41.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.834+0000 7f8270600640 1 -- 172.21.15.86:0/143228904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82680689e0 con 0x7f8268162bd0 2024-04-20T21:30:41.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.835+0000 7f8265600640 1 -- 172.21.15.86:0/143228904 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8260014bc0 con 0x7f8268162bd0 2024-04-20T21:30:41.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.836+0000 7f8265600640 1 --2- 172.21.15.86:0/143228904 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f824806c8f0 0x7f824806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:41.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.836+0000 7f8265600640 1 -- 172.21.15.86:0/143228904 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5853+0+0 (secure 0 0 0) 0x7f826008c530 con 0x7f8268162bd0 2024-04-20T21:30:41.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.836+0000 7f826dc00640 1 --2- 172.21.15.86:0/143228904 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f824806c8f0 0x7f824806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:41.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.837+0000 7f826dc00640 1 --2- 172.21.15.86:0/143228904 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f824806c8f0 0x7f824806edb0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f825c035000 tx=0x7f825c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:41.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:41.841+0000 7f8265600640 1 -- 172.21.15.86:0/143228904 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8268162430 con 0x7f8268162bd0 2024-04-20T21:30:42.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.085+0000 7f8270600640 1 -- 172.21.15.86:0/143228904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f826814d420 con 0x7f8268162bd0 2024-04-20T21:30:42.085 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-04-20T21:30:42.086+0000 7f5bb9c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T21:30:42.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.090+0000 7f8265600640 1 -- 172.21.15.86:0/143228904 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v106) ==== 161+0+0 (secure 0 0 0) 0x7f8260011260 con 0x7f8268162bd0 2024-04-20T21:30:42.089 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-20T21:30:42.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.093+0000 7f8270600640 1 -- 172.21.15.86:0/143228904 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f824806c8f0 msgr2=0x7f824806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.093+0000 7f8270600640 1 --2- 172.21.15.86:0/143228904 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f824806c8f0 0x7f824806edb0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f825c035000 tx=0x7f825c008030 comp rx=0 tx=0).stop 2024-04-20T21:30:42.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.093+0000 7f8270600640 1 -- 172.21.15.86:0/143228904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8268162bd0 msgr2=0x7f826814cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.093+0000 7f8270600640 1 --2- 172.21.15.86:0/143228904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8268162bd0 0x7f826814cee0 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f8260007e70 tx=0x7f826000e4b0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.094+0000 7f8270600640 1 -- 172.21.15.86:0/143228904 shutdown_connections 2024-04-20T21:30:42.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.094+0000 7f8270600640 1 --2- 172.21.15.86:0/143228904 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f824806c8f0 0x7f824806edb0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.094+0000 7f8270600640 1 --2- 172.21.15.86:0/143228904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8268162bd0 0x7f826814cee0 unknown :-1 s=CLOSED pgs=2034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.094+0000 7f8270600640 1 --2- 172.21.15.86:0/143228904 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f82680b3330 0x7f8268161ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.094+0000 7f8270600640 1 --2- 172.21.15.86:0/143228904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f826806e1e0 0x7f826815e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.094+0000 7f8270600640 1 -- 172.21.15.86:0/143228904 >> 172.21.15.86:0/143228904 conn(0x7f826806b980 msgr2=0x7f82680cc520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:42.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.094+0000 7f8270600640 1 -- 172.21.15.86:0/143228904 shutdown_connections 2024-04-20T21:30:42.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.094+0000 7f8270600640 1 -- 172.21.15.86:0/143228904 wait complete. 2024-04-20T21:30:42.101 INFO:tasks.ceph.mds.a.smithi086.stderr: -2479> 2024-04-20T21:30:25.808+0000 7f9fe5e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T21:30:42.113 DEBUG:teuthology.orchestra.run.smithi086:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --force 2024-04-20T21:30:42.151 INFO:tasks.ceph.mds.a.smithi086.stdout:starting mds.a at 2024-04-20T21:30:42.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.202+0000 7f5adbc38280 1 -- start start 2024-04-20T21:30:42.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.202+0000 7f5adbc38280 1 Processor -- start 2024-04-20T21:30:42.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.203+0000 7f5adbc38280 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a77a0e000 0x557a77a05080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.203+0000 7f5adbc38280 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557a77899c00 0x557a77a04b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.203+0000 7f5adbc38280 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a77a0e800 0x557a77a05b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.203+0000 7f5adbc38280 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557a78684480 con 0x557a77a0e800 2024-04-20T21:30:42.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.203+0000 7f5adbc38280 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557a78684600 con 0x557a77a0e000 2024-04-20T21:30:42.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.203+0000 7f5adbc38280 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x557a78684780 con 0x557a77899c00 2024-04-20T21:30:42.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.204+0000 7f5ad9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a77a0e800 0x557a77a05b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.204+0000 7f5ad9e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a77a0e800 0x557a77a05b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58750/0 (socket says 172.21.15.86:58750) 2024-04-20T21:30:42.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.204+0000 7f5ad9e00640 1 -- 172.21.15.86:0/656182916 learned_addr learned my addr 172.21.15.86:0/656182916 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:42.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.204+0000 7f5ad9400640 1 --2- 172.21.15.86:0/656182916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a77a0e000 0x557a77a05080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.204+0000 7f5ad8a00640 1 --2- 172.21.15.86:0/656182916 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557a77899c00 0x557a77a04b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.204+0000 7f5ad9e00640 1 -- 172.21.15.86:0/656182916 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557a77899c00 msgr2=0x557a77a04b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:42.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.204+0000 7f5ad9e00640 1 --2- 172.21.15.86:0/656182916 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557a77899c00 0x557a77a04b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.204+0000 7f5ad9e00640 1 -- 172.21.15.86:0/656182916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a77a0e000 msgr2=0x557a77a05080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:42.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.204+0000 7f5ad9e00640 1 --2- 172.21.15.86:0/656182916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a77a0e000 0x557a77a05080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.204+0000 7f5ad9e00640 1 -- 172.21.15.86:0/656182916 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a778cc780 con 0x557a77a0e800 2024-04-20T21:30:42.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.204+0000 7f5ad9e00640 1 --2- 172.21.15.86:0/656182916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a77a0e800 0x557a77a05b80 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x557a786c3380 tx=0x557a786341e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5058aa9598ed28ba server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:42.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.204+0000 7f5ad6200640 1 -- 172.21.15.86:0/656182916 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x557a778bed00 con 0x557a77a0e800 2024-04-20T21:30:42.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.205+0000 7f5ad6200640 1 -- 172.21.15.86:0/656182916 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x557a778c4fc0 con 0x557a77a0e800 2024-04-20T21:30:42.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.205+0000 7f5ad6200640 1 -- 172.21.15.86:0/656182916 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x557a778be680 con 0x557a77a0e800 2024-04-20T21:30:42.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.205+0000 7f5adbc38280 1 -- 172.21.15.86:0/656182916 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a778cc000 con 0x557a77a0e800 2024-04-20T21:30:42.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.205+0000 7f5adbc38280 1 -- 172.21.15.86:0/656182916 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=0}) -- 0x557a778cd680 con 0x557a77a0e800 2024-04-20T21:30:42.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.205+0000 7f5adbc38280 4 waiting for MDS map... 2024-04-20T21:30:42.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.205+0000 7f5ad6200640 1 -- 172.21.15.86:0/656182916 <== mon.0 v2:172.21.15.86:3300/0 4 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x557a7876a600 con 0x557a77a0e800 2024-04-20T21:30:42.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.205+0000 7f5ad6200640 1 -- 172.21.15.86:0/656182916 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 106) ==== 2886+0+0 (secure 0 0 0) 0x557a778f3180 con 0x557a77a0e800 2024-04-20T21:30:42.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.205+0000 7f5adbc38280 4 Got MDS map 106 2024-04-20T21:30:42.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.205+0000 7f5adbc38280 10 main: JournalTool::main 2024-04-20T21:30:42.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.205+0000 7f5adbc38280 4 main: JournalTool: connecting to RADOS... 2024-04-20T21:30:42.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.206+0000 7f5adbc38280 1 Processor -- start 2024-04-20T21:30:42.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.206+0000 7f5adbc38280 1 -- start start 2024-04-20T21:30:42.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.207+0000 7f5adbc38280 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557a7876cc00 0x557a77a05600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.207+0000 7f5adbc38280 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a7876d000 0x557a77a04580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.207+0000 7f5adbc38280 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a7876d400 0x557a77a06100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.207+0000 7f5adbc38280 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557a778fa180 con 0x557a7876d400 2024-04-20T21:30:42.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.207+0000 7f5adbc38280 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557a78684f00 con 0x557a7876d000 2024-04-20T21:30:42.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.207+0000 7f5adbc38280 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x557a78685080 con 0x557a7876cc00 2024-04-20T21:30:42.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.207+0000 7f5ad8a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a7876d400 0x557a77a06100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.207+0000 7f5ad8a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a7876d400 0x557a77a06100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58764/0 (socket says 172.21.15.86:58764) 2024-04-20T21:30:42.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.207+0000 7f5ad8a00640 1 -- 172.21.15.86:0/3951771999 learned_addr learned my addr 172.21.15.86:0/3951771999 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:42.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.207+0000 7f5ad8a00640 1 --2- 172.21.15.86:0/3951771999 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557a7876cc00 0x557a77a05600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.208+0000 7f5ad9400640 1 --2- 172.21.15.86:0/3951771999 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a7876d000 0x557a77a04580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.208+0000 7f5ad8a00640 1 -- 172.21.15.86:0/3951771999 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557a7876cc00 msgr2=0x557a77a05600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:42.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.208+0000 7f5ad8a00640 1 --2- 172.21.15.86:0/3951771999 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557a7876cc00 0x557a77a05600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.208+0000 7f5ad8a00640 1 -- 172.21.15.86:0/3951771999 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a7876d000 msgr2=0x557a77a04580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:42.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.208+0000 7f5ad8a00640 1 --2- 172.21.15.86:0/3951771999 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a7876d000 0x557a77a04580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.208+0000 7f5ad8a00640 1 -- 172.21.15.86:0/3951771999 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a778cd860 con 0x557a7876d400 2024-04-20T21:30:42.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.208+0000 7f5ad8a00640 1 --2- 172.21.15.86:0/3951771999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a7876d400 0x557a77a06100 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x557a786c2db0 tx=0x557a78632900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffa8247cf14a5b82 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:42.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.208+0000 7f5ad3a00640 1 -- 172.21.15.86:0/3951771999 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x557a778bed00 con 0x557a7876d400 2024-04-20T21:30:42.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.208+0000 7f5ad3a00640 1 -- 172.21.15.86:0/3951771999 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x557a778c4700 con 0x557a7876d400 2024-04-20T21:30:42.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.208+0000 7f5ad3a00640 1 -- 172.21.15.86:0/3951771999 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x557a778be680 con 0x557a7876d400 2024-04-20T21:30:42.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.209+0000 7f5adbc38280 1 -- 172.21.15.86:0/3951771999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a7876d400 msgr2=0x557a77a06100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.209+0000 7f5adbc38280 1 --2- 172.21.15.86:0/3951771999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a7876d400 0x557a77a06100 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x557a786c2db0 tx=0x557a78632900 comp rx=0 tx=0).stop 2024-04-20T21:30:42.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.209+0000 7f5adbc38280 1 -- 172.21.15.86:0/3951771999 shutdown_connections 2024-04-20T21:30:42.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.209+0000 7f5adbc38280 1 --2- 172.21.15.86:0/3951771999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a7876d400 0x557a77a06100 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.209+0000 7f5adbc38280 1 --2- 172.21.15.86:0/3951771999 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a7876d000 0x557a77a04580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.209+0000 7f5adbc38280 1 --2- 172.21.15.86:0/3951771999 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557a7876cc00 0x557a77a05600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.209+0000 7f5adbc38280 1 -- 172.21.15.86:0/3951771999 >> 172.21.15.86:0/3951771999 conn(0x557a77a0e400 msgr2=0x557a78639800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:42.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.209+0000 7f5adbc38280 1 -- 172.21.15.86:0/3951771999 shutdown_connections 2024-04-20T21:30:42.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.209+0000 7f5adbc38280 1 -- 172.21.15.86:0/3951771999 wait complete. 2024-04-20T21:30:42.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.209+0000 7f5adbc38280 1 Processor -- start 2024-04-20T21:30:42.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.210+0000 7f5adbc38280 1 -- start start 2024-04-20T21:30:42.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.210+0000 7f5adbc38280 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a7876cc00 0x557a77a07180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.210+0000 7f5adbc38280 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a7876d000 0x557a77a06c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.210+0000 7f5adbc38280 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557a7876d400 0x557a77a06680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.210+0000 7f5adbc38280 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557a78685380 con 0x557a7876cc00 2024-04-20T21:30:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.210+0000 7f5adbc38280 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557a78685200 con 0x557a7876d000 2024-04-20T21:30:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.210+0000 7f5adbc38280 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x557a78685500 con 0x557a7876d400 2024-04-20T21:30:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.210+0000 7f5ad8a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a7876cc00 0x557a77a07180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.211+0000 7f5ad8a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a7876cc00 0x557a77a07180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58770/0 (socket says 172.21.15.86:58770) 2024-04-20T21:30:42.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.211+0000 7f5ad8a00640 1 -- 172.21.15.86:0/3271227996 learned_addr learned my addr 172.21.15.86:0/3271227996 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:42.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.211+0000 7f5ad8a00640 1 -- 172.21.15.86:0/3271227996 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557a7876d400 msgr2=0x557a77a06680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.211+0000 7f5ad8a00640 1 --2- 172.21.15.86:0/3271227996 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557a7876d400 0x557a77a06680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.211+0000 7f5ad8a00640 1 -- 172.21.15.86:0/3271227996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a7876d000 msgr2=0x557a77a06c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.211+0000 7f5ad8a00640 1 --2- 172.21.15.86:0/3271227996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a7876d000 0x557a77a06c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.211+0000 7f5ad8a00640 1 -- 172.21.15.86:0/3271227996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a778cc3c0 con 0x557a7876cc00 2024-04-20T21:30:42.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.211+0000 7f5ad8a00640 1 --2- 172.21.15.86:0/3271227996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a7876cc00 0x557a77a07180 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x557a787827e0 tx=0x557a786340c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:42.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.211+0000 7f5ad1c00640 1 -- 172.21.15.86:0/3271227996 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x557a778be820 con 0x557a7876cc00 2024-04-20T21:30:42.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.211+0000 7f5ad1c00640 1 -- 172.21.15.86:0/3271227996 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x557a778c4e00 con 0x557a7876cc00 2024-04-20T21:30:42.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.211+0000 7f5adbc38280 1 -- 172.21.15.86:0/3271227996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557a778cc780 con 0x557a7876cc00 2024-04-20T21:30:42.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.211+0000 7f5ad1c00640 1 -- 172.21.15.86:0/3271227996 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x557a778bed00 con 0x557a7876cc00 2024-04-20T21:30:42.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.211+0000 7f5adbc38280 1 -- 172.21.15.86:0/3271227996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a778cc000 con 0x557a7876cc00 2024-04-20T21:30:42.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.211+0000 7f5adbc38280 4 main: JournalTool: resolving pool 16 2024-04-20T21:30:42.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.213+0000 7f5ad1c00640 1 -- 172.21.15.86:0/3271227996 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x557a7869ec00 con 0x557a7876cc00 2024-04-20T21:30:42.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.213+0000 7f5ad1c00640 1 --2- 172.21.15.86:0/3271227996 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x557a77c02000 0x557a77a07700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.214+0000 7f5ad1c00640 1 -- 172.21.15.86:0/3271227996 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x557a7869ca00 con 0x557a7876cc00 2024-04-20T21:30:42.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.214+0000 7f5adbc38280 4 main: JournalTool: creating IoCtx.. 2024-04-20T21:30:42.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.214+0000 7f5adbc38280 4 main: Executing for rank 0 2024-04-20T21:30:42.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.214+0000 7f5ad9400640 1 --2- 172.21.15.86:0/3271227996 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x557a77c02000 0x557a77a07700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.215+0000 7f5ad9400640 1 --2- 172.21.15.86:0/3271227996 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x557a77c02000 0x557a77a07700 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x557a7862a7e0 tx=0x557a77b8d800 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:42.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.215+0000 7f5adbc38280 1 -- start start 2024-04-20T21:30:42.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.215+0000 7f5adbc38280 1 Processor -- start 2024-04-20T21:30:42.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.216+0000 7f5adbc38280 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a7876dc00 0x557a77a06100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.216+0000 7f5adbc38280 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a77c03000 0x557a77a04580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.216+0000 7f5adbc38280 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557a77c03400 0x557a77a05600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.216+0000 7f5adbc38280 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557a78685800 con 0x557a77c03000 2024-04-20T21:30:42.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.216+0000 7f5adbc38280 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x557a78685980 con 0x557a7876dc00 2024-04-20T21:30:42.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.216+0000 7f5adbc38280 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x557a78685b00 con 0x557a77c03400 2024-04-20T21:30:42.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.216+0000 7f5ad9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a77c03000 0x557a77a04580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.216+0000 7f5ad9400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a77c03000 0x557a77a04580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58782/0 (socket says 172.21.15.86:58782) 2024-04-20T21:30:42.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.216+0000 7f5ad9400640 1 -- 172.21.15.86:0/3250918588 learned_addr learned my addr 172.21.15.86:0/3250918588 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:42.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.216+0000 7f5ad9e00640 1 --2- 172.21.15.86:0/3250918588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a7876dc00 0x557a77a06100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.217+0000 7f5ad9400640 1 -- 172.21.15.86:0/3250918588 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557a77c03400 msgr2=0x557a77a05600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:42.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.217+0000 7f5ad9400640 1 --2- 172.21.15.86:0/3250918588 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557a77c03400 0x557a77a05600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.217+0000 7f5ad9400640 1 -- 172.21.15.86:0/3250918588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a7876dc00 msgr2=0x557a77a06100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.217+0000 7f5ad9400640 1 --2- 172.21.15.86:0/3250918588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a7876dc00 0x557a77a06100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.217+0000 7f5ad9400640 1 -- 172.21.15.86:0/3250918588 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a778cda40 con 0x557a77c03000 2024-04-20T21:30:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.217+0000 7f5ad9400640 1 --2- 172.21.15.86:0/3250918588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a77c03000 0x557a77a04580 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x557a77bb7950 tx=0x557a78632ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caa3116c8510ef35 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.217+0000 7f5ace000640 1 -- 172.21.15.86:0/3250918588 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x557a77b8bba0 con 0x557a77c03000 2024-04-20T21:30:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.217+0000 7f5ace000640 1 -- 172.21.15.86:0/3250918588 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x557a778c48c0 con 0x557a77c03000 2024-04-20T21:30:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.217+0000 7f5ace000640 1 -- 172.21.15.86:0/3250918588 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x557a77b8ba00 con 0x557a77c03000 2024-04-20T21:30:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.217+0000 7f5adbc38280 1 -- 172.21.15.86:0/3250918588 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a778cd4a0 con 0x557a77c03000 2024-04-20T21:30:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.217+0000 7f5adbc38280 1 -- 172.21.15.86:0/3250918588 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=0}) -- 0x557a778cc960 con 0x557a77c03000 2024-04-20T21:30:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.217+0000 7f5adbc38280 4 waiting for MDS map... 2024-04-20T21:30:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.217+0000 7f5ace000640 1 -- 172.21.15.86:0/3250918588 <== mon.0 v2:172.21.15.86:3300/0 4 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x557a77c6c600 con 0x557a77c03000 2024-04-20T21:30:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.217+0000 7f5ace000640 1 -- 172.21.15.86:0/3250918588 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 106) ==== 2886+0+0 (secure 0 0 0) 0x557a778f3500 con 0x557a77c03000 2024-04-20T21:30:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.218+0000 7f5adbc38280 4 Got MDS map 106 2024-04-20T21:30:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.218+0000 7f5adbc38280 1 --2- 172.21.15.86:0/3250918588 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x557a77c03800 0x557a77c8a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.218+0000 7f5adbc38280 1 -- 172.21.15.86:0/3250918588 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:1 16.f 16:f1f69726:::400.00000000:head [writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x557a77c03c00 con 0x557a77c03800 2024-04-20T21:30:42.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.218+0000 7f5ad9e00640 1 --2- 172.21.15.86:0/3250918588 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x557a77c03800 0x557a77c8a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.219+0000 7f5ad9e00640 1 --2- 172.21.15.86:0/3250918588 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x557a77c03800 0x557a77c8a000 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=6cc8af9da3e213db server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:42.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.225+0000 7f5ad9e00640 1 -- 172.21.15.86:0/3250918588 <== osd.0 v2:172.21.15.86:6800/2471076035 1 ==== osd_op_reply(1 400.00000000 [writefull 0~22] v116'2 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x557a78622000 con 0x557a77c03800 2024-04-20T21:30:42.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.225+0000 7f5adbc38280 1 -- 172.21.15.86:0/3250918588 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:2 16.14 16:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x557a77c6f400 con 0x557a77c03800 2024-04-20T21:30:42.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.227+0000 7f5ad9e00640 1 -- 172.21.15.86:0/3250918588 <== osd.0 v2:172.21.15.86:6800/2471076035 2 ==== osd_op_reply(2 200.00000000 [writefull 0~90] v116'42 uv42 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x557a78622000 con 0x557a77c03800 2024-04-20T21:30:42.226 INFO:teuthology.orchestra.run.smithi086.stdout:writing EResetJournal entry 2024-04-20T21:30:42.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.227+0000 7f5adbc38280 1 -- 172.21.15.86:0/3250918588 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:3 16.14 16:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x557a77c6f800 con 0x557a77c03800 2024-04-20T21:30:42.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.227+0000 7f5adbc38280 1 --2- 172.21.15.86:0/3250918588 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x557a77c6fc00 0x557a77c8a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.227+0000 7f5adbc38280 1 -- 172.21.15.86:0/3250918588 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:4 16.1b 16:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x557a77c9a000 con 0x557a77c6fc00 2024-04-20T21:30:42.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.227+0000 7f5adbc38280 1 --2- 172.21.15.86:0/3250918588 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x557a77c9a400 0x557a77c8ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.227+0000 7f5adbc38280 1 -- 172.21.15.86:0/3250918588 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:5 16.b 16:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x557a77c9a800 con 0x557a77c9a400 2024-04-20T21:30:42.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.227+0000 7f5adbc38280 1 -- 172.21.15.86:0/3250918588 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:6 16.1f 16:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x557a77c9ac00 con 0x557a77c6fc00 2024-04-20T21:30:42.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.227+0000 7f5adbc38280 1 --2- 172.21.15.86:0/3250918588 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x557a77c9b000 0x557a77c8b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.227+0000 7f5adbc38280 1 -- 172.21.15.86:0/3250918588 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:7 16.1a 16:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x557a77c9b400 con 0x557a77c9b000 2024-04-20T21:30:42.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.227+0000 7f5adbc38280 1 --2- 172.21.15.86:0/3250918588 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x557a77c9b800 0x557a77c8b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.227+0000 7f5adbc38280 1 -- 172.21.15.86:0/3250918588 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:8 16.1e 16:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x557a77c9bc00 con 0x557a77c9b800 2024-04-20T21:30:42.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.227+0000 7f5ad9e00640 1 --2- 172.21.15.86:0/3250918588 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x557a77c9a400 0x557a77c8ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.227+0000 7f5ad9400640 1 --2- 172.21.15.86:0/3250918588 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x557a77c9b000 0x557a77c8b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.228+0000 7f5ad8a00640 1 --2- 172.21.15.86:0/3250918588 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x557a77c6fc00 0x557a77c8a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.228+0000 7f5ad8a00640 1 --2- 172.21.15.86:0/3250918588 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x557a77c9b800 0x557a77c8b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.228+0000 7f5ad9e00640 1 --2- 172.21.15.86:0/3250918588 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x557a77c9a400 0x557a77c8ab00 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=8b096d5183aca838 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:42.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.228+0000 7f5ad9400640 1 --2- 172.21.15.86:0/3250918588 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x557a77c9b000 0x557a77c8b080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=20f5d456d2b8499c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:42.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.229+0000 7f5ad9e00640 1 -- 172.21.15.86:0/3250918588 <== osd.0 v2:172.21.15.86:6800/2471076035 3 ==== osd_op_reply(3 200.00000001 [delete] v116'43 uv43 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x557a78622000 con 0x557a77c03800 2024-04-20T21:30:42.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.229+0000 7f5ad8a00640 1 --2- 172.21.15.86:0/3250918588 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x557a77c6fc00 0x557a77c8a580 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=3f67bf48eae06a70 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:42.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.229+0000 7f5ad8a00640 1 --2- 172.21.15.86:0/3250918588 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x557a77c9b800 0x557a77c8b600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=2145a92343b0728b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:42.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.231+0000 7f5ad9400640 1 -- 172.21.15.86:0/3250918588 <== osd.3 v2:172.21.15.86:6808/794135519 1 ==== osd_op_reply(7 200.00000005 [delete] v116'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x557a78622500 con 0x557a77c9b000 2024-04-20T21:30:42.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.232+0000 7f5ad9e00640 1 -- 172.21.15.86:0/3250918588 <== osd.1 v2:172.21.15.86:6817/3149564961 1 ==== osd_op_reply(5 200.00000003 [delete] v116'6 uv5 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x557a78622000 con 0x557a77c9a400 2024-04-20T21:30:42.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.232+0000 7f5ad8a00640 1 -- 172.21.15.86:0/3250918588 <== osd.5 v2:172.21.15.144:6808/1635974704 1 ==== osd_op_reply(4 200.00000002 [delete] v116'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x557a78622780 con 0x557a77c6fc00 2024-04-20T21:30:42.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.233+0000 7f5acc200640 1 -- 172.21.15.86:0/3250918588 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:9 16.14 16:2e2fa760:::200.00000001:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x557a77c03c00 con 0x557a77c03800 2024-04-20T21:30:42.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.233+0000 7f5ad8a00640 1 -- 172.21.15.86:0/3250918588 <== osd.5 v2:172.21.15.144:6808/1635974704 2 ==== osd_op_reply(6 200.00000004 [delete] v116'9 uv8 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x557a78622780 con 0x557a77c6fc00 2024-04-20T21:30:42.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.234+0000 7f5ad9e00640 1 -- 172.21.15.86:0/3250918588 <== osd.0 v2:172.21.15.86:6800/2471076035 4 ==== osd_op_reply(9 200.00000001 [write 0~48] v116'44 uv44 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x557a78622000 con 0x557a77c03800 2024-04-20T21:30:42.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.239+0000 7f5ad8a00640 1 -- 172.21.15.86:0/3250918588 <== osd.7 v2:172.21.15.144:6816/2100967057 1 ==== osd_op_reply(8 200.00000006 [delete] v116'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x557a78622780 con 0x557a77c9b800 2024-04-20T21:30:42.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.239+0000 7f5adbc38280 4 Successfully wrote new journal pointer and header for rank 8:0 2024-04-20T21:30:42.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.239+0000 7f5adbc38280 1 -- 172.21.15.86:0/3250918588 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x557a77c03800 msgr2=0x557a77c8a000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.239+0000 7f5adbc38280 1 --2- 172.21.15.86:0/3250918588 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x557a77c03800 0x557a77c8a000 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.239+0000 7f5adbc38280 1 -- 172.21.15.86:0/3250918588 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x557a77c9a400 msgr2=0x557a77c8ab00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.239+0000 7f5adbc38280 1 --2- 172.21.15.86:0/3250918588 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x557a77c9a400 0x557a77c8ab00 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.239+0000 7f5adbc38280 1 -- 172.21.15.86:0/3250918588 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x557a77c9b000 msgr2=0x557a77c8b080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.239+0000 7f5adbc38280 1 --2- 172.21.15.86:0/3250918588 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x557a77c9b000 0x557a77c8b080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.239+0000 7f5adbc38280 1 -- 172.21.15.86:0/3250918588 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x557a77c6fc00 msgr2=0x557a77c8a580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.239+0000 7f5adbc38280 1 --2- 172.21.15.86:0/3250918588 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x557a77c6fc00 0x557a77c8a580 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.239+0000 7f5ad8a00640 1 -- 172.21.15.86:0/3250918588 reap_dead start 2024-04-20T21:30:42.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.239+0000 7f5adbc38280 1 -- 172.21.15.86:0/3250918588 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x557a77c9b800 msgr2=0x557a77c8b600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.239+0000 7f5adbc38280 1 --2- 172.21.15.86:0/3250918588 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x557a77c9b800 0x557a77c8b600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.239+0000 7f5adbc38280 1 -- 172.21.15.86:0/3250918588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a77c03000 msgr2=0x557a77a04580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.239+0000 7f5adbc38280 1 --2- 172.21.15.86:0/3250918588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a77c03000 0x557a77a04580 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x557a77bb7950 tx=0x557a78632ae0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.239+0000 7f5adbc38280 1 -- 172.21.15.86:0/3250918588 shutdown_connections 2024-04-20T21:30:42.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.239+0000 7f5adbc38280 1 --2- 172.21.15.86:0/3250918588 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x557a77c6fc00 0x557a77c8a580 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.239+0000 7f5adbc38280 1 --2- 172.21.15.86:0/3250918588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a77c03000 0x557a77a04580 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.239+0000 7f5adbc38280 1 --2- 172.21.15.86:0/3250918588 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x557a77c9b800 0x557a77c8b600 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.239+0000 7f5adbc38280 1 -- 172.21.15.86:0/3250918588 >> 172.21.15.86:0/3250918588 conn(0x557a7876d800 msgr2=0x557a77c60000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:42.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.239+0000 7f5adbc38280 1 -- 172.21.15.86:0/3250918588 shutdown_connections 2024-04-20T21:30:42.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.240+0000 7f5adbc38280 1 -- 172.21.15.86:0/3250918588 wait complete. 2024-04-20T21:30:42.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.240+0000 7f5adbc38280 1 -- 172.21.15.86:0/3271227996 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x557a77c02000 msgr2=0x557a77a07700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.240+0000 7f5adbc38280 1 --2- 172.21.15.86:0/3271227996 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x557a77c02000 0x557a77a07700 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x557a7862a7e0 tx=0x557a77b8d800 comp rx=0 tx=0).stop 2024-04-20T21:30:42.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.240+0000 7f5adbc38280 1 -- 172.21.15.86:0/3271227996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a7876cc00 msgr2=0x557a77a07180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.240+0000 7f5adbc38280 1 --2- 172.21.15.86:0/3271227996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a7876cc00 0x557a77a07180 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x557a787827e0 tx=0x557a786340c0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.240+0000 7f5adbc38280 1 -- 172.21.15.86:0/3271227996 shutdown_connections 2024-04-20T21:30:42.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.240+0000 7f5adbc38280 1 --2- 172.21.15.86:0/3271227996 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x557a77c02000 0x557a77a07700 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.240+0000 7f5adbc38280 1 --2- 172.21.15.86:0/3271227996 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557a7876d400 0x557a77a06680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.240+0000 7f5adbc38280 1 --2- 172.21.15.86:0/3271227996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a7876d000 0x557a77a06c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.240+0000 7f5adbc38280 1 --2- 172.21.15.86:0/3271227996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a7876cc00 0x557a77a07180 unknown :-1 s=CLOSED pgs=2042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.240+0000 7f5adbc38280 1 -- 172.21.15.86:0/3271227996 >> 172.21.15.86:0/3271227996 conn(0x557a77a0e400 msgr2=0x557a78639800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:42.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.240+0000 7f5adbc38280 1 -- 172.21.15.86:0/3271227996 shutdown_connections 2024-04-20T21:30:42.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.240+0000 7f5adbc38280 1 -- 172.21.15.86:0/3271227996 wait complete. 2024-04-20T21:30:42.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.241+0000 7f5adbc38280 1 -- 172.21.15.86:0/656182916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a77a0e800 msgr2=0x557a77a05b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.241+0000 7f5adbc38280 1 --2- 172.21.15.86:0/656182916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a77a0e800 0x557a77a05b80 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x557a786c3380 tx=0x557a786341e0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.241+0000 7f5adbc38280 1 -- 172.21.15.86:0/656182916 shutdown_connections 2024-04-20T21:30:42.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.241+0000 7f5adbc38280 1 --2- 172.21.15.86:0/656182916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a77a0e800 0x557a77a05b80 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.241+0000 7f5adbc38280 1 --2- 172.21.15.86:0/656182916 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x557a77899c00 0x557a77a04b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.241+0000 7f5adbc38280 1 --2- 172.21.15.86:0/656182916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x557a77a0e000 0x557a77a05080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.241+0000 7f5adbc38280 1 -- 172.21.15.86:0/656182916 >> 172.21.15.86:0/656182916 conn(0x557a77899000 msgr2=0x557a78639000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:42.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.241+0000 7f5adbc38280 1 -- 172.21.15.86:0/656182916 shutdown_connections 2024-04-20T21:30:42.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.241+0000 7f5adbc38280 1 -- 172.21.15.86:0/656182916 wait complete. 2024-04-20T21:30:42.245 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--force'] in time 0:00:00.132310, result: writing EResetJournal entry 2024-04-20T21:30:42.245 DEBUG:teuthology.orchestra.run.smithi086:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-04-20T21:30:42.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.381+0000 7f5a4324e740 1 -- start start 2024-04-20T21:30:42.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.381+0000 7f5a4324e740 1 Processor -- start 2024-04-20T21:30:42.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.382+0000 7f5a4324e740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea95c56c00 0x55ea969cc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.382+0000 7f5a4324e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea95c57800 0x55ea969cc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.382+0000 7f5a4324e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea95c57c00 0x55ea969ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.382+0000 7f5a4324e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ea969d0180 con 0x55ea95c57c00 2024-04-20T21:30:42.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.382+0000 7f5a4324e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ea95cb6900 con 0x55ea95c57800 2024-04-20T21:30:42.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.382+0000 7f5a4324e740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55ea95cb6480 con 0x55ea95c56c00 2024-04-20T21:30:42.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.382+0000 7f5a41600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea95c57c00 0x55ea969ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.383+0000 7f5a41600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea95c57c00 0x55ea969ccb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58806/0 (socket says 172.21.15.86:58806) 2024-04-20T21:30:42.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.383+0000 7f5a41600640 1 -- 172.21.15.86:0/984722530 learned_addr learned my addr 172.21.15.86:0/984722530 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:42.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.383+0000 7f5a41600640 1 -- 172.21.15.86:0/984722530 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea95c56c00 msgr2=0x55ea969cc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:42.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.383+0000 7f5a41600640 1 --2- 172.21.15.86:0/984722530 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea95c56c00 0x55ea969cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.383+0000 7f5a41600640 1 -- 172.21.15.86:0/984722530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea95c57800 msgr2=0x55ea969cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:42.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.383+0000 7f5a41600640 1 --2- 172.21.15.86:0/984722530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea95c57800 0x55ea969cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.383+0000 7f5a41600640 1 -- 172.21.15.86:0/984722530 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea95c83680 con 0x55ea95c57c00 2024-04-20T21:30:42.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.383+0000 7f5a41600640 1 --2- 172.21.15.86:0/984722530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea95c57c00 0x55ea969ccb00 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x55ea96a79c50 tx=0x55ea969f65a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cec7618f4c920693 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:42.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.384+0000 7f5a3da00640 1 -- 172.21.15.86:0/984722530 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ea95c7c680 con 0x55ea95c57c00 2024-04-20T21:30:42.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.384+0000 7f5a3da00640 1 -- 172.21.15.86:0/984722530 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55ea95ca01c0 con 0x55ea95c57c00 2024-04-20T21:30:42.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.384+0000 7f5a3da00640 1 -- 172.21.15.86:0/984722530 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ea95c7cb60 con 0x55ea95c57c00 2024-04-20T21:30:42.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.384+0000 7f5a4324e740 1 -- 172.21.15.86:0/984722530 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea95c83860 con 0x55ea95c57c00 2024-04-20T21:30:42.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.384+0000 7f5a4324e740 1 -- 172.21.15.86:0/984722530 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55ea95c832c0 con 0x55ea95c57c00 2024-04-20T21:30:42.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.384+0000 7f5a4324e740 4 waiting for MDS map... 2024-04-20T21:30:42.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.384+0000 7f5a3da00640 1 -- 172.21.15.86:0/984722530 <== mon.0 v2:172.21.15.86:3300/0 4 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x55ea96b2c600 con 0x55ea95c57c00 2024-04-20T21:30:42.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.384+0000 7f5a3da00640 1 -- 172.21.15.86:0/984722530 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 106) ==== 2886+0+0 (secure 0 0 0) 0x55ea95caf180 con 0x55ea95c57c00 2024-04-20T21:30:42.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.384+0000 7f5a4324e740 4 Got MDS map 106 2024-04-20T21:30:42.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.384+0000 7f5a4324e740 4 datascan.main: Using metadata pool output 2024-04-20T21:30:42.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.384+0000 7f5a4324e740 4 datascan.main: connecting to RADOS... 2024-04-20T21:30:42.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.385+0000 7f5a4324e740 1 Processor -- start 2024-04-20T21:30:42.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.385+0000 7f5a4324e740 1 -- start start 2024-04-20T21:30:42.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.386+0000 7f5a4324e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea96b2f400 0x55ea969cd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.386+0000 7f5a4324e740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea96b2f800 0x55ea969cd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.386+0000 7f5a4324e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea96b2fc00 0x55ea969cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.386+0000 7f5a4324e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ea95cb6180 con 0x55ea96b2fc00 2024-04-20T21:30:42.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.386+0000 7f5a4324e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ea969d0d80 con 0x55ea96b2f400 2024-04-20T21:30:42.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.386+0000 7f5a4324e740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55ea969d0f00 con 0x55ea96b2f800 2024-04-20T21:30:42.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.386+0000 7f5a40200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea96b2fc00 0x55ea969cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.386+0000 7f5a40200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea96b2fc00 0x55ea969cdb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58822/0 (socket says 172.21.15.86:58822) 2024-04-20T21:30:42.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.386+0000 7f5a40200640 1 -- 172.21.15.86:0/66919123 learned_addr learned my addr 172.21.15.86:0/66919123 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:42.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.387+0000 7f5a40200640 1 --2- 172.21.15.86:0/66919123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea96b2f400 0x55ea969cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.387+0000 7f5a40c00640 1 --2- 172.21.15.86:0/66919123 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea96b2f800 0x55ea969cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.387+0000 7f5a40200640 1 -- 172.21.15.86:0/66919123 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea96b2f800 msgr2=0x55ea969cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:42.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.387+0000 7f5a40200640 1 --2- 172.21.15.86:0/66919123 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea96b2f800 0x55ea969cd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.387+0000 7f5a40200640 1 -- 172.21.15.86:0/66919123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea96b2f400 msgr2=0x55ea969cd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:42.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.387+0000 7f5a40200640 1 --2- 172.21.15.86:0/66919123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea96b2f400 0x55ea969cd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.387+0000 7f5a40200640 1 -- 172.21.15.86:0/66919123 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea95c834a0 con 0x55ea96b2fc00 2024-04-20T21:30:42.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.387+0000 7f5a40c00640 1 --2- 172.21.15.86:0/66919123 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea96b2f800 0x55ea969cd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:30:42.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.387+0000 7f5a40200640 1 --2- 172.21.15.86:0/66919123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea96b2fc00 0x55ea969cdb80 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x55ea96b44ba0 tx=0x55ea96b50d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=759792d60df5b4b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:42.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.387+0000 7f5a3b200640 1 -- 172.21.15.86:0/66919123 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ea95c7c680 con 0x55ea96b2fc00 2024-04-20T21:30:42.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.387+0000 7f5a3b200640 1 -- 172.21.15.86:0/66919123 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55ea95ca0e00 con 0x55ea96b2fc00 2024-04-20T21:30:42.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.387+0000 7f5a3b200640 1 -- 172.21.15.86:0/66919123 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ea95c7cb60 con 0x55ea96b2fc00 2024-04-20T21:30:42.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.388+0000 7f5a4324e740 1 -- 172.21.15.86:0/66919123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea96b2fc00 msgr2=0x55ea969cdb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.388+0000 7f5a4324e740 1 --2- 172.21.15.86:0/66919123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea96b2fc00 0x55ea969cdb80 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x55ea96b44ba0 tx=0x55ea96b50d20 comp rx=0 tx=0).stop 2024-04-20T21:30:42.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.388+0000 7f5a4324e740 1 -- 172.21.15.86:0/66919123 shutdown_connections 2024-04-20T21:30:42.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.388+0000 7f5a4324e740 1 --2- 172.21.15.86:0/66919123 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea96b2fc00 0x55ea969cdb80 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.388+0000 7f5a4324e740 1 --2- 172.21.15.86:0/66919123 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea96b2f800 0x55ea969cd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.388+0000 7f5a4324e740 1 --2- 172.21.15.86:0/66919123 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea96b2f400 0x55ea969cd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.388+0000 7f5a4324e740 1 -- 172.21.15.86:0/66919123 >> 172.21.15.86:0/66919123 conn(0x55ea96b2f000 msgr2=0x55ea95d43000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:42.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.388+0000 7f5a4324e740 1 -- 172.21.15.86:0/66919123 shutdown_connections 2024-04-20T21:30:42.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.388+0000 7f5a4324e740 1 -- 172.21.15.86:0/66919123 wait complete. 2024-04-20T21:30:42.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.389+0000 7f5a4324e740 1 Processor -- start 2024-04-20T21:30:42.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.389+0000 7f5a4324e740 1 -- start start 2024-04-20T21:30:42.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.389+0000 7f5a4324e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea96b2f400 0x55ea969cd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.389+0000 7f5a4324e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea96b2f800 0x55ea969cd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.390+0000 7f5a4324e740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea96b2fc00 0x55ea969cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.390+0000 7f5a4324e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ea969d1200 con 0x55ea96b2f400 2024-04-20T21:30:42.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.390+0000 7f5a4324e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55ea969d1080 con 0x55ea96b2f800 2024-04-20T21:30:42.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.390+0000 7f5a4324e740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55ea969d1380 con 0x55ea96b2fc00 2024-04-20T21:30:42.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.390+0000 7f5a40200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea96b2f400 0x55ea969cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.390+0000 7f5a40200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea96b2f400 0x55ea969cd080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58838/0 (socket says 172.21.15.86:58838) 2024-04-20T21:30:42.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.390+0000 7f5a40200640 1 -- 172.21.15.86:0/4116617820 learned_addr learned my addr 172.21.15.86:0/4116617820 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:42.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.390+0000 7f5a40200640 1 -- 172.21.15.86:0/4116617820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea96b2fc00 msgr2=0x55ea969cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.390+0000 7f5a40200640 1 --2- 172.21.15.86:0/4116617820 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea96b2fc00 0x55ea969cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.390+0000 7f5a40200640 1 -- 172.21.15.86:0/4116617820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea96b2f800 msgr2=0x55ea969cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.390+0000 7f5a40200640 1 --2- 172.21.15.86:0/4116617820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea96b2f800 0x55ea969cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.390+0000 7f5a40200640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea95c83680 con 0x55ea96b2f400 2024-04-20T21:30:42.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.390+0000 7f5a40200640 1 --2- 172.21.15.86:0/4116617820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea96b2f400 0x55ea969cd080 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x55ea96b45860 tx=0x55ea96b50c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:42.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.391+0000 7f5a39400640 1 -- 172.21.15.86:0/4116617820 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ea95c7c4e0 con 0x55ea96b2f400 2024-04-20T21:30:42.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.391+0000 7f5a39400640 1 -- 172.21.15.86:0/4116617820 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55ea95ca0540 con 0x55ea96b2f400 2024-04-20T21:30:42.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.391+0000 7f5a39400640 1 -- 172.21.15.86:0/4116617820 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ea95c7c680 con 0x55ea96b2f400 2024-04-20T21:30:42.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.391+0000 7f5a4324e740 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ea95c83860 con 0x55ea96b2f400 2024-04-20T21:30:42.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.391+0000 7f5a4324e740 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea95c830e0 con 0x55ea96b2f400 2024-04-20T21:30:42.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.391+0000 7f5a4324e740 4 datascan.init: resolving metadata pool 16 2024-04-20T21:30:42.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.392+0000 7f5a39400640 1 -- 172.21.15.86:0/4116617820 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x55ea96bb2c00 con 0x55ea96b2f400 2024-04-20T21:30:42.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.393+0000 7f5a39400640 1 --2- 172.21.15.86:0/4116617820 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x55ea96baec00 0x55ea969ce100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.393+0000 7f5a39400640 1 -- 172.21.15.86:0/4116617820 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x55ea96bb0a00 con 0x55ea96b2f400 2024-04-20T21:30:42.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.393+0000 7f5a4324e740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-20T21:30:42.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.393+0000 7f5a4324e740 4 datascan.main: data pool 'cephfs_data' has ID 17 2024-04-20T21:30:42.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.393+0000 7f5a4324e740 4 datascan.main: opening data pool 'cephfs_data' 2024-04-20T21:30:42.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.393+0000 7f5a4324e740 1 --2- 172.21.15.86:0/4116617820 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x55ea96baf000 0x55ea969ce680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.393+0000 7f5a40c00640 1 --2- 172.21.15.86:0/4116617820 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x55ea96baec00 0x55ea969ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.393+0000 7f5a4324e740 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:1 17.0 17:00000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea96baf400 con 0x55ea96baf000 2024-04-20T21:30:42.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.394+0000 7f5a40c00640 1 --2- 172.21.15.86:0/4116617820 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x55ea96baec00 0x55ea969ce100 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x55ea969f8150 tx=0x55ea96b50f00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:42.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.394+0000 7f5a40200640 1 --2- 172.21.15.86:0/4116617820 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x55ea96baf000 0x55ea969ce680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.395+0000 7f5a40200640 1 --2- 172.21.15.86:0/4116617820 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x55ea96baf000 0x55ea969ce680 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:42.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.396+0000 7f5a40200640 1 -- 172.21.15.86:0/4116617820 <== osd.7 v2:172.21.15.144:6816/2100967057 1 ==== osd_op_reply(1 [pgnls start_epoch 116 out=49b] v112'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4000 con 0x55ea96baf000 2024-04-20T21:30:42.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.396+0000 7f5a3a800640 1 --2- 172.21.15.86:0/4116617820 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x55ea95fee000 0x55ea95ff4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.396+0000 7f5a3a800640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:2 17.20 17:04000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea95fee400 con 0x55ea95fee000 2024-04-20T21:30:42.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.396+0000 7f5a41600640 1 --2- 172.21.15.86:0/4116617820 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x55ea95fee000 0x55ea95ff4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.397+0000 7f5a41600640 1 --2- 172.21.15.86:0/4116617820 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x55ea95fee000 0x55ea95ff4000 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:42.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.397+0000 7f5a41600640 1 -- 172.21.15.86:0/4116617820 <== osd.1 v2:172.21.15.86:6817/3149564961 1 ==== osd_op_reply(2 [pgnls start_epoch 116 out=49b] v112'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4500 con 0x55ea95fee000 2024-04-20T21:30:42.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.397+0000 7f5a3b200640 1 --2- 172.21.15.86:0/4116617820 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x55ea95feec00 0x55ea95ff4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.397+0000 7f5a3b200640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:3 17.10 17:08000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea95fef000 con 0x55ea95feec00 2024-04-20T21:30:42.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.398+0000 7f5a40c00640 1 --2- 172.21.15.86:0/4116617820 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x55ea95feec00 0x55ea95ff4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.398+0000 7f5a40c00640 1 --2- 172.21.15.86:0/4116617820 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x55ea95feec00 0x55ea95ff4580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:42.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.398+0000 7f5a40c00640 1 -- 172.21.15.86:0/4116617820 <== osd.3 v2:172.21.15.86:6808/794135519 1 ==== osd_op_reply(3 [pgnls start_epoch 116 out=49b] v112'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4780 con 0x55ea95feec00 2024-04-20T21:30:42.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.399+0000 7f5a3a800640 1 --2- 172.21.15.86:0/4116617820 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x55ea95fee800 0x55ea95ff4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.399+0000 7f5a3a800640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:4 17.30 17:0c000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea95fef800 con 0x55ea95fee800 2024-04-20T21:30:42.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.399+0000 7f5a40200640 1 --2- 172.21.15.86:0/4116617820 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x55ea95fee800 0x55ea95ff4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.399+0000 7f5a40200640 1 --2- 172.21.15.86:0/4116617820 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x55ea95fee800 0x55ea95ff4b00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:42.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.400+0000 7f5a40200640 1 -- 172.21.15.86:0/4116617820 <== osd.0 v2:172.21.15.86:6800/2471076035 1 ==== osd_op_reply(4 [pgnls start_epoch 116 out=49b] v112'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4000 con 0x55ea95fee800 2024-04-20T21:30:42.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.400+0000 7f5a3b200640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:5 17.8 17:10000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea95fef400 con 0x55ea96baf000 2024-04-20T21:30:42.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.401+0000 7f5a40200640 1 -- 172.21.15.86:0/4116617820 <== osd.7 v2:172.21.15.144:6816/2100967057 2 ==== osd_op_reply(5 [pgnls start_epoch 116 out=49b] v112'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4000 con 0x55ea96baf000 2024-04-20T21:30:42.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.401+0000 7f5a3a800640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:6 17.28 17:14000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea95fefc00 con 0x55ea95feec00 2024-04-20T21:30:42.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.401+0000 7f5a40c00640 1 -- 172.21.15.86:0/4116617820 <== osd.3 v2:172.21.15.86:6808/794135519 2 ==== osd_op_reply(6 [pgnls start_epoch 116 out=49b] v112'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4780 con 0x55ea95feec00 2024-04-20T21:30:42.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.401+0000 7f5a3b200640 1 --2- 172.21.15.86:0/4116617820 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x55ea96004000 0x55ea95ff5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.401+0000 7f5a3b200640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:7 17.18 17:18000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea96004400 con 0x55ea96004000 2024-04-20T21:30:42.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.401+0000 7f5a41600640 1 --2- 172.21.15.86:0/4116617820 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x55ea96004000 0x55ea95ff5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.402+0000 7f5a41600640 1 --2- 172.21.15.86:0/4116617820 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x55ea96004000 0x55ea95ff5600 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:42.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.402+0000 7f5a41600640 1 -- 172.21.15.86:0/4116617820 <== osd.2 v2:172.21.15.86:6816/3437611468 1 ==== osd_op_reply(7 [pgnls start_epoch 116 out=49b] v112'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4500 con 0x55ea96004000 2024-04-20T21:30:42.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.402+0000 7f5a3a800640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:8 17.38 17:1c000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea96004c00 con 0x55ea96baf000 2024-04-20T21:30:42.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.403+0000 7f5a40200640 1 -- 172.21.15.86:0/4116617820 <== osd.7 v2:172.21.15.144:6816/2100967057 3 ==== osd_op_reply(8 [pgnls start_epoch 116 out=49b] v112'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4000 con 0x55ea96baf000 2024-04-20T21:30:42.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.403+0000 7f5a3b200640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:9 17.4 17:20000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea96004800 con 0x55ea95fee800 2024-04-20T21:30:42.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.404+0000 7f5a40200640 1 -- 172.21.15.86:0/4116617820 <== osd.0 v2:172.21.15.86:6800/2471076035 2 ==== osd_op_reply(9 [pgnls start_epoch 116 out=49b] v112'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4000 con 0x55ea95fee800 2024-04-20T21:30:42.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.404+0000 7f5a3a800640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:10 17.24 17:24000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea96005000 con 0x55ea95feec00 2024-04-20T21:30:42.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.404+0000 7f5a40c00640 1 -- 172.21.15.86:0/4116617820 <== osd.3 v2:172.21.15.86:6808/794135519 3 ==== osd_op_reply(10 [pgnls start_epoch 116 out=49b] v112'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4780 con 0x55ea95feec00 2024-04-20T21:30:42.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.404+0000 7f5a3b200640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:11 17.14 17:28000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea95fef000 con 0x55ea95feec00 2024-04-20T21:30:42.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.404+0000 7f5a40c00640 1 -- 172.21.15.86:0/4116617820 <== osd.3 v2:172.21.15.86:6808/794135519 4 ==== osd_op_reply(11 [pgnls start_epoch 116 out=49b] v112'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4780 con 0x55ea95feec00 2024-04-20T21:30:42.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.404+0000 7f5a3a800640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:12 17.34 17:2c000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea96005400 con 0x55ea95fee800 2024-04-20T21:30:42.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.405+0000 7f5a40200640 1 -- 172.21.15.86:0/4116617820 <== osd.0 v2:172.21.15.86:6800/2471076035 3 ==== osd_op_reply(12 [pgnls start_epoch 116 out=49b] v112'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4000 con 0x55ea95fee800 2024-04-20T21:30:42.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.405+0000 7f5a3b200640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:13 17.c 17:30000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea96baf400 con 0x55ea96baf000 2024-04-20T21:30:42.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.406+0000 7f5a40200640 1 -- 172.21.15.86:0/4116617820 <== osd.7 v2:172.21.15.144:6816/2100967057 4 ==== osd_op_reply(13 [pgnls start_epoch 116 out=49b] v112'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4000 con 0x55ea96baf000 2024-04-20T21:30:42.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.406+0000 7f5a3a800640 1 --2- 172.21.15.86:0/4116617820 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x55ea96005800 0x55ea95ff5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.406+0000 7f5a3a800640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:14 17.2c 17:34000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea96005000 con 0x55ea96005800 2024-04-20T21:30:42.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.406+0000 7f5a40c00640 1 --2- 172.21.15.86:0/4116617820 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x55ea96005800 0x55ea95ff5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.408+0000 7f5a40c00640 1 --2- 172.21.15.86:0/4116617820 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x55ea96005800 0x55ea95ff5080 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:42.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.414+0000 7f5a40c00640 1 -- 172.21.15.86:0/4116617820 <== osd.4 v2:172.21.15.144:6800/4087309475 1 ==== osd_op_reply(14 [pgnls start_epoch 116 out=49b] v112'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4780 con 0x55ea96005800 2024-04-20T21:30:42.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.414+0000 7f5a3b200640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:15 17.1c 17:38000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea969f5400 con 0x55ea95fee000 2024-04-20T21:30:42.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.415+0000 7f5a41600640 1 -- 172.21.15.86:0/4116617820 <== osd.1 v2:172.21.15.86:6817/3149564961 2 ==== osd_op_reply(15 [pgnls start_epoch 116 out=49b] v112'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4500 con 0x55ea95fee000 2024-04-20T21:30:42.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.415+0000 7f5a3a800640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:16 17.3c 17:3c000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea96004c00 con 0x55ea95fee800 2024-04-20T21:30:42.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.416+0000 7f5a40200640 1 -- 172.21.15.86:0/4116617820 <== osd.0 v2:172.21.15.86:6800/2471076035 4 ==== osd_op_reply(16 [pgnls start_epoch 116 out=49b] v112'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4000 con 0x55ea95fee800 2024-04-20T21:30:42.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.416+0000 7f5a3b200640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:17 17.2 17:40000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea969f5000 con 0x55ea96005800 2024-04-20T21:30:42.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.417+0000 7f5a40c00640 1 -- 172.21.15.86:0/4116617820 <== osd.4 v2:172.21.15.144:6800/4087309475 2 ==== osd_op_reply(17 [pgnls start_epoch 116 out=49b] v112'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4780 con 0x55ea96005800 2024-04-20T21:30:42.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.417+0000 7f5a3a800640 1 --2- 172.21.15.86:0/4116617820 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x55ea95fefc00 0x55ea95ff6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.417+0000 7f5a3a800640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:18 17.22 17:44000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea95fef800 con 0x55ea95fefc00 2024-04-20T21:30:42.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.418+0000 7f5a40200640 1 --2- 172.21.15.86:0/4116617820 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x55ea95fefc00 0x55ea95ff6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.419+0000 7f5a40200640 1 --2- 172.21.15.86:0/4116617820 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x55ea95fefc00 0x55ea95ff6100 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:42.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.420+0000 7f5a40200640 1 -- 172.21.15.86:0/4116617820 <== osd.5 v2:172.21.15.144:6808/1635974704 1 ==== osd_op_reply(18 [pgnls start_epoch 116 out=49b] v112'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4000 con 0x55ea95fefc00 2024-04-20T21:30:42.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.421+0000 7f5a3b200640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:19 17.12 17:48000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea96004800 con 0x55ea95feec00 2024-04-20T21:30:42.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.421+0000 7f5a40c00640 1 -- 172.21.15.86:0/4116617820 <== osd.3 v2:172.21.15.86:6808/794135519 5 ==== osd_op_reply(19 [pgnls start_epoch 116 out=81b] v115'22 uv22 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55ea969e4780 con 0x55ea95feec00 2024-04-20T21:30:42.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.421+0000 7f5a4324e740 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:20 17.12 17:4bec3886:::10000000002.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x55ea96baf800 con 0x55ea95feec00 2024-04-20T21:30:42.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.422+0000 7f5a40c00640 1 -- 172.21.15.86:0/4116617820 <== osd.3 v2:172.21.15.86:6808/794135519 6 ==== osd_op_reply(20 10000000002.00000000 [stat out=16b] v0'0 uv22 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x55ea969e4780 con 0x55ea95feec00 2024-04-20T21:30:42.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.422+0000 7f5a4324e740 10 datascan.operator(): handling object 1099511627778.0 2024-04-20T21:30:42.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.422+0000 7f5a4324e740 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:21 17.12 17:4bec3886:::10000000002.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55ea96bafc00 con 0x55ea95feec00 2024-04-20T21:30:42.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.425+0000 7f5a40c00640 1 -- 172.21.15.86:0/4116617820 <== osd.3 v2:172.21.15.86:6808/794135519 7 ==== osd_op_reply(21 10000000002.00000000 [call] v116'23 uv23 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55ea969e4780 con 0x55ea95feec00 2024-04-20T21:30:42.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.425+0000 7f5a4324e740 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:22 17.32 17:4c000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea96005000 con 0x55ea96baf000 2024-04-20T21:30:42.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.426+0000 7f5a40200640 1 -- 172.21.15.86:0/4116617820 <== osd.7 v2:172.21.15.144:6816/2100967057 5 ==== osd_op_reply(22 [pgnls start_epoch 116 out=49b] v112'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4000 con 0x55ea96baf000 2024-04-20T21:30:42.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.426+0000 7f5a3b200640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:23 17.a 17:50000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea96005400 con 0x55ea96baf000 2024-04-20T21:30:42.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.427+0000 7f5a40200640 1 -- 172.21.15.86:0/4116617820 <== osd.7 v2:172.21.15.144:6816/2100967057 6 ==== osd_op_reply(23 [pgnls start_epoch 116 out=49b] v112'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4000 con 0x55ea96baf000 2024-04-20T21:30:42.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.427+0000 7f5a3a800640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:24 17.2a 17:54000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea95fef400 con 0x55ea95feec00 2024-04-20T21:30:42.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.428+0000 7f5a40c00640 1 -- 172.21.15.86:0/4116617820 <== osd.3 v2:172.21.15.86:6808/794135519 8 ==== osd_op_reply(24 [pgnls start_epoch 116 out=49b] v112'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4780 con 0x55ea95feec00 2024-04-20T21:30:42.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.428+0000 7f5a3b200640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:25 17.1a 17:58000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea96004c00 con 0x55ea95fee800 2024-04-20T21:30:42.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.428+0000 7f5a40200640 1 -- 172.21.15.86:0/4116617820 <== osd.0 v2:172.21.15.86:6800/2471076035 5 ==== osd_op_reply(25 [pgnls start_epoch 116 out=49b] v112'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4000 con 0x55ea95fee800 2024-04-20T21:30:42.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.428+0000 7f5a3a800640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:26 17.3a 17:5c000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea96005c00 con 0x55ea95fee000 2024-04-20T21:30:42.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.429+0000 7f5a41600640 1 -- 172.21.15.86:0/4116617820 <== osd.1 v2:172.21.15.86:6817/3149564961 3 ==== osd_op_reply(26 [pgnls start_epoch 116 out=49b] v112'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4500 con 0x55ea95fee000 2024-04-20T21:30:42.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.429+0000 7f5a3b200640 1 --2- 172.21.15.86:0/4116617820 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x55ea95fef800 0x55ea95ff5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.429+0000 7f5a3b200640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:27 17.6 17:60000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea96baf400 con 0x55ea95fef800 2024-04-20T21:30:42.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.430+0000 7f5a41600640 1 --2- 172.21.15.86:0/4116617820 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x55ea95fef800 0x55ea95ff5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.431+0000 7f5a41600640 1 --2- 172.21.15.86:0/4116617820 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x55ea95fef800 0x55ea95ff5b80 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:42.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.432+0000 7f5a41600640 1 -- 172.21.15.86:0/4116617820 <== osd.6 v2:172.21.15.144:6824/1893803079 1 ==== osd_op_reply(27 [pgnls start_epoch 116 out=49b] v112'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4500 con 0x55ea95fef800 2024-04-20T21:30:42.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.432+0000 7f5a3a800640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:28 17.26 17:64000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea96004400 con 0x55ea95fee800 2024-04-20T21:30:42.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.433+0000 7f5a40200640 1 -- 172.21.15.86:0/4116617820 <== osd.0 v2:172.21.15.86:6800/2471076035 6 ==== osd_op_reply(28 [pgnls start_epoch 116 out=49b] v112'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4000 con 0x55ea95fee800 2024-04-20T21:30:42.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.433+0000 7f5a3b200640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:29 17.16 17:68000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea9601c000 con 0x55ea95fefc00 2024-04-20T21:30:42.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.434+0000 7f5a40200640 1 -- 172.21.15.86:0/4116617820 <== osd.5 v2:172.21.15.144:6808/1635974704 2 ==== osd_op_reply(29 [pgnls start_epoch 116 out=49b] v112'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4000 con 0x55ea95fefc00 2024-04-20T21:30:42.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.434+0000 7f5a3a800640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:30 17.36 17:6c000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea96005c00 con 0x55ea95fee800 2024-04-20T21:30:42.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.434+0000 7f5a40200640 1 -- 172.21.15.86:0/4116617820 <== osd.0 v2:172.21.15.86:6800/2471076035 7 ==== osd_op_reply(30 [pgnls start_epoch 116 out=81b] v112'33 uv17 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55ea969e4000 con 0x55ea95fee800 2024-04-20T21:30:42.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.434+0000 7f5a4324e740 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:31 17.36 17:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x55ea96004800 con 0x55ea95fee800 2024-04-20T21:30:42.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.435+0000 7f5a40200640 1 -- 172.21.15.86:0/4116617820 <== osd.0 v2:172.21.15.86:6800/2471076035 8 ==== osd_op_reply(31 100000001f7.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x55ea969e4000 con 0x55ea95fee800 2024-04-20T21:30:42.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.435+0000 7f5a4324e740 10 datascan.operator(): handling object 1099511628279.0 2024-04-20T21:30:42.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.435+0000 7f5a4324e740 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:32 17.36 17:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55ea96baf800 con 0x55ea95fee800 2024-04-20T21:30:42.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.438+0000 7f5a40200640 1 -- 172.21.15.86:0/4116617820 <== osd.0 v2:172.21.15.86:6800/2471076035 9 ==== osd_op_reply(32 100000001f7.00000000 [call] v116'34 uv34 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55ea969e4000 con 0x55ea95fee800 2024-04-20T21:30:42.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.438+0000 7f5a4324e740 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:33 17.e 17:70000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea96bafc00 con 0x55ea96004000 2024-04-20T21:30:42.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.439+0000 7f5a41600640 1 -- 172.21.15.86:0/4116617820 <== osd.2 v2:172.21.15.86:6816/3437611468 2 ==== osd_op_reply(33 [pgnls start_epoch 116 out=49b] v112'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4500 con 0x55ea96004000 2024-04-20T21:30:42.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.439+0000 7f5a3a800640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:34 17.2e 17:74000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea969f5400 con 0x55ea95fef800 2024-04-20T21:30:42.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.440+0000 7f5a41600640 1 -- 172.21.15.86:0/4116617820 <== osd.6 v2:172.21.15.144:6824/1893803079 2 ==== osd_op_reply(34 [pgnls start_epoch 116 out=49b] v112'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4500 con 0x55ea95fef800 2024-04-20T21:30:42.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.440+0000 7f5a3b200640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:35 17.1e 17:78000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea9601c400 con 0x55ea95fee000 2024-04-20T21:30:42.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.441+0000 7f5a41600640 1 -- 172.21.15.86:0/4116617820 <== osd.1 v2:172.21.15.86:6817/3149564961 4 ==== osd_op_reply(35 [pgnls start_epoch 116 out=49b] v112'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4500 con 0x55ea95fee000 2024-04-20T21:30:42.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.441+0000 7f5a3a800640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:36 17.3e 17:7c000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea95fee400 con 0x55ea96005800 2024-04-20T21:30:42.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.442+0000 7f5a40c00640 1 -- 172.21.15.86:0/4116617820 <== osd.4 v2:172.21.15.144:6800/4087309475 3 ==== osd_op_reply(36 [pgnls start_epoch 116 out=49b] v112'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4780 con 0x55ea96005800 2024-04-20T21:30:42.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.442+0000 7f5a3b200640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:37 17.1 17:80000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea9601c800 con 0x55ea95fee800 2024-04-20T21:30:42.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.442+0000 7f5a40200640 1 -- 172.21.15.86:0/4116617820 <== osd.0 v2:172.21.15.86:6800/2471076035 10 ==== osd_op_reply(37 [pgnls start_epoch 116 out=49b] v112'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4000 con 0x55ea95fee800 2024-04-20T21:30:42.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.442+0000 7f5a3a800640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:38 17.21 17:84000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea96b2e800 con 0x55ea95fee800 2024-04-20T21:30:42.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.443+0000 7f5a40200640 1 -- 172.21.15.86:0/4116617820 <== osd.0 v2:172.21.15.86:6800/2471076035 11 ==== osd_op_reply(38 [pgnls start_epoch 116 out=49b] v112'24 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4000 con 0x55ea95fee800 2024-04-20T21:30:42.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.443+0000 7f5a3b200640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:39 17.11 17:88000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea9601cc00 con 0x55ea95feec00 2024-04-20T21:30:42.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.444+0000 7f5a40c00640 1 -- 172.21.15.86:0/4116617820 <== osd.3 v2:172.21.15.86:6808/794135519 9 ==== osd_op_reply(39 [pgnls start_epoch 116 out=49b] v112'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4780 con 0x55ea95feec00 2024-04-20T21:30:42.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.444+0000 7f5a3a800640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:40 17.31 17:8c000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea9601c000 con 0x55ea95fee000 2024-04-20T21:30:42.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.445+0000 7f5a41600640 1 -- 172.21.15.86:0/4116617820 <== osd.1 v2:172.21.15.86:6817/3149564961 5 ==== osd_op_reply(40 [pgnls start_epoch 116 out=81b] v112'15 uv8 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55ea969e4500 con 0x55ea95fee000 2024-04-20T21:30:42.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.445+0000 7f5a4324e740 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:41 17.31 17:8de29a03:::10000000004.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x55ea969f5000 con 0x55ea95fee000 2024-04-20T21:30:42.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.445+0000 7f5a41600640 1 -- 172.21.15.86:0/4116617820 <== osd.1 v2:172.21.15.86:6817/3149564961 6 ==== osd_op_reply(41 10000000004.00000001 [stat out=16b] v0'0 uv8 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x55ea969e4500 con 0x55ea95fee000 2024-04-20T21:30:42.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.445+0000 7f5a4324e740 10 datascan.operator(): handling object 1099511627780.1 2024-04-20T21:30:42.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.445+0000 7f5a4324e740 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:42 17.5 17:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55ea95fef000 con 0x55ea95fef800 2024-04-20T21:30:42.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.449+0000 7f5a41600640 1 -- 172.21.15.86:0/4116617820 <== osd.6 v2:172.21.15.144:6824/1893803079 3 ==== osd_op_reply(42 10000000004.00000000 [call] v116'14 uv14 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55ea969e4500 con 0x55ea95fef800 2024-04-20T21:30:42.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.449+0000 7f5a4324e740 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:43 17.9 17:90000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea969f5400 con 0x55ea95feec00 2024-04-20T21:30:42.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.450+0000 7f5a40c00640 1 -- 172.21.15.86:0/4116617820 <== osd.3 v2:172.21.15.86:6808/794135519 10 ==== osd_op_reply(43 [pgnls start_epoch 116 out=49b] v112'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4780 con 0x55ea95feec00 2024-04-20T21:30:42.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.450+0000 7f5a3a800640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:44 17.29 17:94000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea9601d800 con 0x55ea96005800 2024-04-20T21:30:42.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.451+0000 7f5a40c00640 1 -- 172.21.15.86:0/4116617820 <== osd.4 v2:172.21.15.144:6800/4087309475 4 ==== osd_op_reply(44 [pgnls start_epoch 116 out=49b] v112'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4780 con 0x55ea96005800 2024-04-20T21:30:42.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.451+0000 7f5a3b200640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:45 17.19 17:98000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea9601d000 con 0x55ea95fefc00 2024-04-20T21:30:42.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.453+0000 7f5a40200640 1 -- 172.21.15.86:0/4116617820 <== osd.5 v2:172.21.15.144:6808/1635974704 3 ==== osd_op_reply(45 [pgnls start_epoch 116 out=81b] v115'23 uv23 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55ea969e4000 con 0x55ea95fefc00 2024-04-20T21:30:42.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.453+0000 7f5a4324e740 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:46 17.19 17:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x55ea969f5000 con 0x55ea95fefc00 2024-04-20T21:30:42.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.454+0000 7f5a40200640 1 -- 172.21.15.86:0/4116617820 <== osd.5 v2:172.21.15.144:6808/1635974704 4 ==== osd_op_reply(46 100000001f8.00000000 [stat out=16b] v0'0 uv23 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x55ea969e4000 con 0x55ea95fefc00 2024-04-20T21:30:42.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.454+0000 7f5a4324e740 10 datascan.operator(): handling object 1099511628280.0 2024-04-20T21:30:42.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.454+0000 7f5a4324e740 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:47 17.19 17:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55ea9601c400 con 0x55ea95fefc00 2024-04-20T21:30:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.458+0000 7f5a40200640 1 -- 172.21.15.86:0/4116617820 <== osd.5 v2:172.21.15.144:6808/1635974704 5 ==== osd_op_reply(47 100000001f8.00000000 [call] v116'24 uv24 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55ea969e4000 con 0x55ea95fefc00 2024-04-20T21:30:42.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.458+0000 7f5a4324e740 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:48 17.39 17:9c000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea9601c000 con 0x55ea95fefc00 2024-04-20T21:30:42.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.459+0000 7f5a40200640 1 -- 172.21.15.86:0/4116617820 <== osd.5 v2:172.21.15.144:6808/1635974704 6 ==== osd_op_reply(48 [pgnls start_epoch 116 out=49b] v112'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4000 con 0x55ea95fefc00 2024-04-20T21:30:42.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.459+0000 7f5a3b200640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:49 17.5 17:a0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea9601d400 con 0x55ea95fef800 2024-04-20T21:30:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.460+0000 7f5a41600640 1 -- 172.21.15.86:0/4116617820 <== osd.6 v2:172.21.15.144:6824/1893803079 4 ==== osd_op_reply(49 [pgnls start_epoch 116 out=81b] v116'14 uv14 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55ea969e4500 con 0x55ea95fef800 2024-04-20T21:30:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.460+0000 7f5a4324e740 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:50 17.5 17:a367a3e8:::10000000004.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x55ea96baf400 con 0x55ea95fef800 2024-04-20T21:30:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.461+0000 7f5a41600640 1 -- 172.21.15.86:0/4116617820 <== osd.6 v2:172.21.15.144:6824/1893803079 5 ==== osd_op_reply(50 10000000004.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x55ea969e4500 con 0x55ea95fef800 2024-04-20T21:30:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.461+0000 7f5a4324e740 10 datascan.operator(): handling object 1099511627780.0 2024-04-20T21:30:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.461+0000 7f5a4324e740 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:51 17.5 17:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55ea96bafc00 con 0x55ea95fef800 2024-04-20T21:30:42.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.465+0000 7f5a41600640 1 -- 172.21.15.86:0/4116617820 <== osd.6 v2:172.21.15.144:6824/1893803079 6 ==== osd_op_reply(51 10000000004.00000000 [call] v116'15 uv15 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55ea969e4500 con 0x55ea95fef800 2024-04-20T21:30:42.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.465+0000 7f5a4324e740 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:52 17.25 17:a4000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea9601c800 con 0x55ea95fee000 2024-04-20T21:30:42.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.466+0000 7f5a41600640 1 -- 172.21.15.86:0/4116617820 <== osd.1 v2:172.21.15.86:6817/3149564961 7 ==== osd_op_reply(52 [pgnls start_epoch 116 out=49b] v112'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4500 con 0x55ea95fee000 2024-04-20T21:30:42.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.466+0000 7f5a3b200640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:53 17.15 17:a8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea96024000 con 0x55ea95fefc00 2024-04-20T21:30:42.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.467+0000 7f5a40200640 1 -- 172.21.15.86:0/4116617820 <== osd.5 v2:172.21.15.144:6808/1635974704 7 ==== osd_op_reply(53 [pgnls start_epoch 116 out=49b] v112'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4000 con 0x55ea95fefc00 2024-04-20T21:30:42.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.467+0000 7f5a3a800640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:54 17.35 17:ac000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea9601dc00 con 0x55ea96005800 2024-04-20T21:30:42.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.469+0000 7f5a40c00640 1 -- 172.21.15.86:0/4116617820 <== osd.4 v2:172.21.15.144:6800/4087309475 5 ==== osd_op_reply(54 [pgnls start_epoch 116 out=49b] v112'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4780 con 0x55ea96005800 2024-04-20T21:30:42.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.469+0000 7f5a3b200640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:55 17.d 17:b0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea96024400 con 0x55ea96baf000 2024-04-20T21:30:42.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.470+0000 7f5a40200640 1 -- 172.21.15.86:0/4116617820 <== osd.7 v2:172.21.15.144:6816/2100967057 7 ==== osd_op_reply(55 [pgnls start_epoch 116 out=49b] v112'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4000 con 0x55ea96baf000 2024-04-20T21:30:42.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.470+0000 7f5a3a800640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:56 17.2d 17:b4000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea96005000 con 0x55ea96baf000 2024-04-20T21:30:42.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.471+0000 7f5a40200640 1 -- 172.21.15.86:0/4116617820 <== osd.7 v2:172.21.15.144:6816/2100967057 8 ==== osd_op_reply(56 [pgnls start_epoch 116 out=49b] v112'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4000 con 0x55ea96baf000 2024-04-20T21:30:42.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.472+0000 7f5a3b200640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:57 17.1d 17:b8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea96024800 con 0x55ea96baf000 2024-04-20T21:30:42.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.473+0000 7f5a40200640 1 -- 172.21.15.86:0/4116617820 <== osd.7 v2:172.21.15.144:6816/2100967057 9 ==== osd_op_reply(57 [pgnls start_epoch 116 out=49b] v112'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4000 con 0x55ea96baf000 2024-04-20T21:30:42.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.473+0000 7f5a3a800640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:58 17.3d 17:bc000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea96005400 con 0x55ea96baf000 2024-04-20T21:30:42.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.474+0000 7f5a40200640 1 -- 172.21.15.86:0/4116617820 <== osd.7 v2:172.21.15.144:6816/2100967057 10 ==== osd_op_reply(58 [pgnls start_epoch 116 out=49b] v112'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4000 con 0x55ea96baf000 2024-04-20T21:30:42.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.474+0000 7f5a3b200640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:59 17.3 17:c0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea96024c00 con 0x55ea96005800 2024-04-20T21:30:42.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.475+0000 7f5a40c00640 1 -- 172.21.15.86:0/4116617820 <== osd.4 v2:172.21.15.144:6800/4087309475 6 ==== osd_op_reply(59 [pgnls start_epoch 116 out=49b] v112'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4780 con 0x55ea96005800 2024-04-20T21:30:42.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.476+0000 7f5a3a800640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:60 17.23 17:c4000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea96004c00 con 0x55ea96005800 2024-04-20T21:30:42.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.477+0000 7f5a40c00640 1 -- 172.21.15.86:0/4116617820 <== osd.4 v2:172.21.15.144:6800/4087309475 7 ==== osd_op_reply(60 [pgnls start_epoch 116 out=49b] v112'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4780 con 0x55ea96005800 2024-04-20T21:30:42.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.477+0000 7f5a3b200640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:61 17.13 17:c8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea96025000 con 0x55ea95feec00 2024-04-20T21:30:42.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.478+0000 7f5a40c00640 1 -- 172.21.15.86:0/4116617820 <== osd.3 v2:172.21.15.86:6808/794135519 11 ==== osd_op_reply(61 [pgnls start_epoch 116 out=49b] v112'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4780 con 0x55ea95feec00 2024-04-20T21:30:42.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.478+0000 7f5a3a800640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:62 17.33 17:cc000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea96004400 con 0x55ea96baf000 2024-04-20T21:30:42.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.479+0000 7f5a40200640 1 -- 172.21.15.86:0/4116617820 <== osd.7 v2:172.21.15.144:6816/2100967057 11 ==== osd_op_reply(62 [pgnls start_epoch 116 out=49b] v112'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4000 con 0x55ea96baf000 2024-04-20T21:30:42.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.479+0000 7f5a3b200640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:63 17.b 17:d0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea96025400 con 0x55ea95fee800 2024-04-20T21:30:42.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.480+0000 7f5a40200640 1 -- 172.21.15.86:0/4116617820 <== osd.0 v2:172.21.15.86:6800/2471076035 12 ==== osd_op_reply(63 [pgnls start_epoch 116 out=49b] v112'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4000 con 0x55ea95fee800 2024-04-20T21:30:42.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.480+0000 7f5a3a800640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:64 17.2b 17:d4000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea96005c00 con 0x55ea96005800 2024-04-20T21:30:42.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.481+0000 7f5a40c00640 1 -- 172.21.15.86:0/4116617820 <== osd.4 v2:172.21.15.144:6800/4087309475 8 ==== osd_op_reply(64 [pgnls start_epoch 116 out=49b] v112'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4780 con 0x55ea96005800 2024-04-20T21:30:42.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.481+0000 7f5a3b200640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:65 17.1b 17:d8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea96025800 con 0x55ea96005800 2024-04-20T21:30:42.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.483+0000 7f5a40c00640 1 -- 172.21.15.86:0/4116617820 <== osd.4 v2:172.21.15.144:6800/4087309475 9 ==== osd_op_reply(65 [pgnls start_epoch 116 out=49b] v112'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4780 con 0x55ea96005800 2024-04-20T21:30:42.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.483+0000 7f5a3a800640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:66 17.3b 17:dc000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea9601c000 con 0x55ea96004000 2024-04-20T21:30:42.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.484+0000 7f5a41600640 1 -- 172.21.15.86:0/4116617820 <== osd.2 v2:172.21.15.86:6816/3437611468 3 ==== osd_op_reply(66 [pgnls start_epoch 116 out=49b] v112'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4500 con 0x55ea96004000 2024-04-20T21:30:42.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.484+0000 7f5a3b200640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:67 17.7 17:e0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea96024000 con 0x55ea95fee800 2024-04-20T21:30:42.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.485+0000 7f5a40200640 1 -- 172.21.15.86:0/4116617820 <== osd.0 v2:172.21.15.86:6800/2471076035 13 ==== osd_op_reply(67 [pgnls start_epoch 116 out=49b] v112'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4000 con 0x55ea95fee800 2024-04-20T21:30:42.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.485+0000 7f5a3a800640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:68 17.27 17:e4000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea9601cc00 con 0x55ea96005800 2024-04-20T21:30:42.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.486+0000 7f5a40c00640 1 -- 172.21.15.86:0/4116617820 <== osd.4 v2:172.21.15.144:6800/4087309475 10 ==== osd_op_reply(68 [pgnls start_epoch 116 out=49b] v112'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4780 con 0x55ea96005800 2024-04-20T21:30:42.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.486+0000 7f5a3b200640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:69 17.17 17:e8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea96024400 con 0x55ea95fef800 2024-04-20T21:30:42.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.488+0000 7f5a41600640 1 -- 172.21.15.86:0/4116617820 <== osd.6 v2:172.21.15.144:6824/1893803079 7 ==== osd_op_reply(69 [pgnls start_epoch 116 out=49b] v112'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4500 con 0x55ea95fef800 2024-04-20T21:30:42.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.488+0000 7f5a3a800640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:70 17.37 17:ec000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea9601d800 con 0x55ea95fee800 2024-04-20T21:30:42.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.489+0000 7f5a40200640 1 -- 172.21.15.86:0/4116617820 <== osd.0 v2:172.21.15.86:6800/2471076035 14 ==== osd_op_reply(70 [pgnls start_epoch 116 out=49b] v112'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4000 con 0x55ea95fee800 2024-04-20T21:30:42.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.489+0000 7f5a3b200640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:71 17.f 17:f0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea96024800 con 0x55ea95fee000 2024-04-20T21:30:42.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.490+0000 7f5a41600640 1 -- 172.21.15.86:0/4116617820 <== osd.1 v2:172.21.15.86:6817/3149564961 8 ==== osd_op_reply(71 [pgnls start_epoch 116 out=49b] v112'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4500 con 0x55ea95fee000 2024-04-20T21:30:42.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.490+0000 7f5a3a800640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:72 17.2f 17:f4000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea969f5400 con 0x55ea96005800 2024-04-20T21:30:42.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.491+0000 7f5a40c00640 1 -- 172.21.15.86:0/4116617820 <== osd.4 v2:172.21.15.144:6800/4087309475 11 ==== osd_op_reply(72 [pgnls start_epoch 116 out=49b] v112'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4780 con 0x55ea96005800 2024-04-20T21:30:42.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.491+0000 7f5a3b200640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:73 17.1f 17:f8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea96025400 con 0x55ea95fef800 2024-04-20T21:30:42.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.493+0000 7f5a41600640 1 -- 172.21.15.86:0/4116617820 <== osd.6 v2:172.21.15.144:6824/1893803079 8 ==== osd_op_reply(73 [pgnls start_epoch 116 out=49b] v112'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4500 con 0x55ea95fef800 2024-04-20T21:30:42.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.493+0000 7f5a3a800640 1 -- 172.21.15.86:0/4116617820 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:74 17.3f 17:fc000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x55ea95fef400 con 0x55ea95fee800 2024-04-20T21:30:42.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.494+0000 7f5a40200640 1 -- 172.21.15.86:0/4116617820 <== osd.0 v2:172.21.15.86:6800/2471076035 15 ==== osd_op_reply(74 [pgnls start_epoch 116 out=49b] v112'17 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55ea969e4000 con 0x55ea95fee800 2024-04-20T21:30:42.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.494+0000 7f5a4324e740 1 -- 172.21.15.86:0/4116617820 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x55ea95fee800 msgr2=0x55ea95ff4b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.494+0000 7f5a4324e740 1 --2- 172.21.15.86:0/4116617820 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x55ea95fee800 0x55ea95ff4b00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.494+0000 7f5a4324e740 1 -- 172.21.15.86:0/4116617820 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x55ea95fee000 msgr2=0x55ea95ff4000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.494+0000 7f5a4324e740 1 --2- 172.21.15.86:0/4116617820 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x55ea95fee000 0x55ea95ff4000 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.494+0000 7f5a4324e740 1 -- 172.21.15.86:0/4116617820 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x55ea96004000 msgr2=0x55ea95ff5600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.494+0000 7f5a4324e740 1 --2- 172.21.15.86:0/4116617820 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x55ea96004000 0x55ea95ff5600 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.494+0000 7f5a4324e740 1 -- 172.21.15.86:0/4116617820 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x55ea95feec00 msgr2=0x55ea95ff4580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.494+0000 7f5a4324e740 1 --2- 172.21.15.86:0/4116617820 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x55ea95feec00 0x55ea95ff4580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.494+0000 7f5a4324e740 1 -- 172.21.15.86:0/4116617820 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x55ea96005800 msgr2=0x55ea95ff5080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.494+0000 7f5a4324e740 1 --2- 172.21.15.86:0/4116617820 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x55ea96005800 0x55ea95ff5080 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.494+0000 7f5a4324e740 1 -- 172.21.15.86:0/4116617820 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x55ea95fefc00 msgr2=0x55ea95ff6100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.494+0000 7f5a4324e740 1 --2- 172.21.15.86:0/4116617820 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x55ea95fefc00 0x55ea95ff6100 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.494+0000 7f5a40c00640 1 -- 172.21.15.86:0/4116617820 reap_dead start 2024-04-20T21:30:42.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.494+0000 7f5a4324e740 1 -- 172.21.15.86:0/4116617820 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x55ea95fef800 msgr2=0x55ea95ff5b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.494+0000 7f5a4324e740 1 --2- 172.21.15.86:0/4116617820 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x55ea95fef800 0x55ea95ff5b80 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.494+0000 7f5a40c00640 1 -- 172.21.15.86:0/4116617820 reap_dead start 2024-04-20T21:30:42.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.494+0000 7f5a40c00640 1 -- 172.21.15.86:0/4116617820 reap_dead start 2024-04-20T21:30:42.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.494+0000 7f5a40c00640 1 -- 172.21.15.86:0/4116617820 reap_dead start 2024-04-20T21:30:42.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.494+0000 7f5a4324e740 1 -- 172.21.15.86:0/4116617820 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x55ea96baf000 msgr2=0x55ea969ce680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.495+0000 7f5a4324e740 1 --2- 172.21.15.86:0/4116617820 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x55ea96baf000 0x55ea969ce680 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.495+0000 7f5a4324e740 1 -- 172.21.15.86:0/4116617820 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x55ea96baec00 msgr2=0x55ea969ce100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.495+0000 7f5a4324e740 1 --2- 172.21.15.86:0/4116617820 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x55ea96baec00 0x55ea969ce100 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x55ea969f8150 tx=0x55ea96b50f00 comp rx=0 tx=0).stop 2024-04-20T21:30:42.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.495+0000 7f5a4324e740 1 -- 172.21.15.86:0/4116617820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea96b2f400 msgr2=0x55ea969cd080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.495+0000 7f5a4324e740 1 --2- 172.21.15.86:0/4116617820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea96b2f400 0x55ea969cd080 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x55ea96b45860 tx=0x55ea96b50c00 comp rx=0 tx=0).stop 2024-04-20T21:30:42.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.495+0000 7f5a4324e740 1 -- 172.21.15.86:0/4116617820 shutdown_connections 2024-04-20T21:30:42.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.495+0000 7f5a4324e740 1 --2- 172.21.15.86:0/4116617820 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x55ea95fef800 0x55ea95ff5b80 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.495+0000 7f5a4324e740 1 --2- 172.21.15.86:0/4116617820 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x55ea96baec00 0x55ea969ce100 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.495+0000 7f5a4324e740 1 --2- 172.21.15.86:0/4116617820 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x55ea96baf000 0x55ea969ce680 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.495+0000 7f5a4324e740 1 --2- 172.21.15.86:0/4116617820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea96b2f400 0x55ea969cd080 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.495+0000 7f5a4324e740 1 -- 172.21.15.86:0/4116617820 >> 172.21.15.86:0/4116617820 conn(0x55ea96b2f000 msgr2=0x55ea95d43000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:42.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.496+0000 7f5a4324e740 1 -- 172.21.15.86:0/4116617820 shutdown_connections 2024-04-20T21:30:42.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.496+0000 7f5a4324e740 1 -- 172.21.15.86:0/4116617820 wait complete. 2024-04-20T21:30:42.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.496+0000 7f5a4324e740 1 -- 172.21.15.86:0/984722530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea95c57c00 msgr2=0x55ea969ccb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.496+0000 7f5a4324e740 1 --2- 172.21.15.86:0/984722530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea95c57c00 0x55ea969ccb00 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x55ea96a79c50 tx=0x55ea969f65a0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.497+0000 7f5a4324e740 1 -- 172.21.15.86:0/984722530 shutdown_connections 2024-04-20T21:30:42.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.497+0000 7f5a4324e740 1 --2- 172.21.15.86:0/984722530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ea95c57c00 0x55ea969ccb00 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.497+0000 7f5a4324e740 1 --2- 172.21.15.86:0/984722530 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55ea95c57800 0x55ea969cc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.497+0000 7f5a4324e740 1 --2- 172.21.15.86:0/984722530 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55ea95c56c00 0x55ea969cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.497+0000 7f5a4324e740 1 -- 172.21.15.86:0/984722530 >> 172.21.15.86:0/984722530 conn(0x55ea95c57000 msgr2=0x55ea95d42800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:42.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.497+0000 7f5a4324e740 1 -- 172.21.15.86:0/984722530 shutdown_connections 2024-04-20T21:30:42.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.497+0000 7f5a4324e740 1 -- 172.21.15.86:0/984722530 wait complete. 2024-04-20T21:30:42.507 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.262194, result: 2024-04-20T21:30:42.508 DEBUG:teuthology.orchestra.run.smithi086:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --filter-tag mytag123 cephfs_data 2024-04-20T21:30:42.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.616+0000 7f178e613740 1 -- start start 2024-04-20T21:30:42.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.616+0000 7f178e613740 1 Processor -- start 2024-04-20T21:30:42.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.617+0000 7f178e613740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ee157ac00 0x562ee22e6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.617+0000 7f178e613740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562ee157b800 0x562ee22e6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.617+0000 7f178e613740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ee157bc00 0x562ee22e6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.617+0000 7f178e613740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562ee2434180 con 0x562ee157bc00 2024-04-20T21:30:42.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.617+0000 7f178e613740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562ee2434300 con 0x562ee157ac00 2024-04-20T21:30:42.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.617+0000 7f178e613740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x562ee2434480 con 0x562ee157b800 2024-04-20T21:30:42.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.617+0000 7f178c600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ee157bc00 0x562ee22e6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.617+0000 7f178c600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ee157bc00 0x562ee22e6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58860/0 (socket says 172.21.15.86:58860) 2024-04-20T21:30:42.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.617+0000 7f178c600640 1 -- 172.21.15.86:0/2540827816 learned_addr learned my addr 172.21.15.86:0/2540827816 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:42.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.618+0000 7f178bc00640 1 --2- 172.21.15.86:0/2540827816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ee157ac00 0x562ee22e6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.618+0000 7f178b200640 1 --2- 172.21.15.86:0/2540827816 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562ee157b800 0x562ee22e6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.618+0000 7f178c600640 1 -- 172.21.15.86:0/2540827816 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562ee157b800 msgr2=0x562ee22e6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:42.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.618+0000 7f178c600640 1 --2- 172.21.15.86:0/2540827816 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562ee157b800 0x562ee22e6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.618+0000 7f178c600640 1 -- 172.21.15.86:0/2540827816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ee157ac00 msgr2=0x562ee22e6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:42.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.618+0000 7f178c600640 1 --2- 172.21.15.86:0/2540827816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ee157ac00 0x562ee22e6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.618+0000 7f178c600640 1 -- 172.21.15.86:0/2540827816 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ee15a7680 con 0x562ee157bc00 2024-04-20T21:30:42.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.618+0000 7f178bc00640 1 --2- 172.21.15.86:0/2540827816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ee157ac00 0x562ee22e6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:30:42.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.618+0000 7f178c600640 1 --2- 172.21.15.86:0/2540827816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ee157bc00 0x562ee22e6b00 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x562ee2395cb0 tx=0x562ee231e120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5346e61eda41bdfb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:42.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.619+0000 7f1788a00640 1 -- 172.21.15.86:0/2540827816 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562ee15a0680 con 0x562ee157bc00 2024-04-20T21:30:42.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.619+0000 7f1788a00640 1 -- 172.21.15.86:0/2540827816 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x562ee15c41c0 con 0x562ee157bc00 2024-04-20T21:30:42.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.619+0000 7f1788a00640 1 -- 172.21.15.86:0/2540827816 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562ee15a0b60 con 0x562ee157bc00 2024-04-20T21:30:42.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.619+0000 7f178e613740 1 -- 172.21.15.86:0/2540827816 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ee15a7860 con 0x562ee157bc00 2024-04-20T21:30:42.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.619+0000 7f178e613740 1 -- 172.21.15.86:0/2540827816 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=0}) -- 0x562ee15a72c0 con 0x562ee157bc00 2024-04-20T21:30:42.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.619+0000 7f178e613740 4 waiting for MDS map... 2024-04-20T21:30:42.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.619+0000 7f1788a00640 1 -- 172.21.15.86:0/2540827816 <== mon.0 v2:172.21.15.86:3300/0 4 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x562ee2456600 con 0x562ee157bc00 2024-04-20T21:30:42.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.619+0000 7f1788a00640 1 -- 172.21.15.86:0/2540827816 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 106) ==== 2886+0+0 (secure 0 0 0) 0x562ee15d3180 con 0x562ee157bc00 2024-04-20T21:30:42.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.619+0000 7f178e613740 4 Got MDS map 106 2024-04-20T21:30:42.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.619+0000 7f178e613740 10 datascan.parse_kwarg: Applying tag filter: 'mytag123' 2024-04-20T21:30:42.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.619+0000 7f178e613740 4 datascan.main: Using metadata pool output 2024-04-20T21:30:42.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.619+0000 7f178e613740 4 datascan.main: connecting to RADOS... 2024-04-20T21:30:42.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.620+0000 7f178e613740 1 Processor -- start 2024-04-20T21:30:42.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.620+0000 7f178e613740 1 -- start start 2024-04-20T21:30:42.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.621+0000 7f178e613740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ee2373400 0x562ee22e7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.621+0000 7f178e613740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562ee2373800 0x562ee22e7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.621+0000 7f178e613740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ee2373c00 0x562ee22e7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.621+0000 7f178e613740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562ee2434600 con 0x562ee2373c00 2024-04-20T21:30:42.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.621+0000 7f178e613740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562ee2434780 con 0x562ee2373400 2024-04-20T21:30:42.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.621+0000 7f178e613740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x562ee2434900 con 0x562ee2373800 2024-04-20T21:30:42.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.621+0000 7f178b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ee2373c00 0x562ee22e7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.621+0000 7f178b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ee2373c00 0x562ee22e7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58868/0 (socket says 172.21.15.86:58868) 2024-04-20T21:30:42.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.621+0000 7f178b200640 1 -- 172.21.15.86:0/3027685229 learned_addr learned my addr 172.21.15.86:0/3027685229 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:42.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.622+0000 7f178bc00640 1 --2- 172.21.15.86:0/3027685229 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562ee2373800 0x562ee22e7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.622+0000 7f178b200640 1 --2- 172.21.15.86:0/3027685229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ee2373400 0x562ee22e7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.622+0000 7f178b200640 1 -- 172.21.15.86:0/3027685229 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562ee2373800 msgr2=0x562ee22e7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:42.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.622+0000 7f178b200640 1 --2- 172.21.15.86:0/3027685229 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562ee2373800 0x562ee22e7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.622+0000 7f178b200640 1 -- 172.21.15.86:0/3027685229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ee2373400 msgr2=0x562ee22e7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:42.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.622+0000 7f178b200640 1 --2- 172.21.15.86:0/3027685229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ee2373400 0x562ee22e7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.622+0000 7f178b200640 1 -- 172.21.15.86:0/3027685229 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ee15a74a0 con 0x562ee2373c00 2024-04-20T21:30:42.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.622+0000 7f178bc00640 1 --2- 172.21.15.86:0/3027685229 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562ee2373800 0x562ee22e7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:30:42.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.622+0000 7f178b200640 1 --2- 172.21.15.86:0/3027685229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ee2373c00 0x562ee22e7b80 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x562ee23951a0 tx=0x562ee231a900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=772bebe1ec6897d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:42.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.622+0000 7f1786200640 1 -- 172.21.15.86:0/3027685229 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562ee15a09c0 con 0x562ee2373c00 2024-04-20T21:30:42.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.622+0000 7f1786200640 1 -- 172.21.15.86:0/3027685229 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x562ee15c4e00 con 0x562ee2373c00 2024-04-20T21:30:42.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.622+0000 7f1786200640 1 -- 172.21.15.86:0/3027685229 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562ee15a0680 con 0x562ee2373c00 2024-04-20T21:30:42.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.623+0000 7f178e613740 1 -- 172.21.15.86:0/3027685229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ee2373c00 msgr2=0x562ee22e7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.623+0000 7f178e613740 1 --2- 172.21.15.86:0/3027685229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ee2373c00 0x562ee22e7b80 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x562ee23951a0 tx=0x562ee231a900 comp rx=0 tx=0).stop 2024-04-20T21:30:42.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.623+0000 7f178e613740 1 -- 172.21.15.86:0/3027685229 shutdown_connections 2024-04-20T21:30:42.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.623+0000 7f178e613740 1 --2- 172.21.15.86:0/3027685229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ee2373c00 0x562ee22e7b80 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.623+0000 7f178e613740 1 --2- 172.21.15.86:0/3027685229 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562ee2373800 0x562ee22e7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.623+0000 7f178e613740 1 --2- 172.21.15.86:0/3027685229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ee2373400 0x562ee22e7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.623+0000 7f178e613740 1 -- 172.21.15.86:0/3027685229 >> 172.21.15.86:0/3027685229 conn(0x562ee2373000 msgr2=0x562ee1649000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:42.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.623+0000 7f178e613740 1 -- 172.21.15.86:0/3027685229 shutdown_connections 2024-04-20T21:30:42.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.623+0000 7f178e613740 1 -- 172.21.15.86:0/3027685229 wait complete. 2024-04-20T21:30:42.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.624+0000 7f178e613740 1 Processor -- start 2024-04-20T21:30:42.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.624+0000 7f178e613740 1 -- start start 2024-04-20T21:30:42.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.625+0000 7f178e613740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ee2373400 0x562ee22e7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.625+0000 7f178e613740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ee2373800 0x562ee22e7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.625+0000 7f178e613740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562ee2373c00 0x562ee22e8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.625+0000 7f178e613740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562ee2435980 con 0x562ee2373800 2024-04-20T21:30:42.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.625+0000 7f178e613740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x562ee2435800 con 0x562ee2373400 2024-04-20T21:30:42.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.625+0000 7f178e613740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x562ee2435680 con 0x562ee2373c00 2024-04-20T21:30:42.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.625+0000 7f178bc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ee2373800 0x562ee22e7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.625+0000 7f178bc00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ee2373800 0x562ee22e7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58876/0 (socket says 172.21.15.86:58876) 2024-04-20T21:30:42.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.625+0000 7f178bc00640 1 -- 172.21.15.86:0/1122026518 learned_addr learned my addr 172.21.15.86:0/1122026518 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:42.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.625+0000 7f178bc00640 1 -- 172.21.15.86:0/1122026518 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562ee2373c00 msgr2=0x562ee22e8100 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-20T21:30:42.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.625+0000 7f178bc00640 1 --2- 172.21.15.86:0/1122026518 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562ee2373c00 0x562ee22e8100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.625+0000 7f178bc00640 1 -- 172.21.15.86:0/1122026518 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ee2373400 msgr2=0x562ee22e7080 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-20T21:30:42.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.625+0000 7f178bc00640 1 --2- 172.21.15.86:0/1122026518 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ee2373400 0x562ee22e7080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.625+0000 7f178bc00640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ee15a7680 con 0x562ee2373800 2024-04-20T21:30:42.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.626+0000 7f178bc00640 1 --2- 172.21.15.86:0/1122026518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ee2373800 0x562ee22e7600 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x562ee2443ad0 tx=0x562ee2466c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:42.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.626+0000 7f1784400640 1 -- 172.21.15.86:0/1122026518 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562ee15a09c0 con 0x562ee2373800 2024-04-20T21:30:42.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.626+0000 7f1784400640 1 -- 172.21.15.86:0/1122026518 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x562ee15c4c40 con 0x562ee2373800 2024-04-20T21:30:42.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.626+0000 7f178e613740 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562ee15a7860 con 0x562ee2373800 2024-04-20T21:30:42.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.626+0000 7f1784400640 1 -- 172.21.15.86:0/1122026518 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562ee15a0680 con 0x562ee2373800 2024-04-20T21:30:42.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.626+0000 7f178e613740 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ee15a70e0 con 0x562ee2373800 2024-04-20T21:30:42.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.626+0000 7f178e613740 4 datascan.init: resolving metadata pool 16 2024-04-20T21:30:42.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.628+0000 7f1784400640 1 -- 172.21.15.86:0/1122026518 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x562ee244ec00 con 0x562ee2373800 2024-04-20T21:30:42.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.628+0000 7f1784400640 1 --2- 172.21.15.86:0/1122026518 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x562ee2450c00 0x562ee22e9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.628+0000 7f1784400640 1 -- 172.21.15.86:0/1122026518 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x562ee24bea00 con 0x562ee2373800 2024-04-20T21:30:42.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.628+0000 7f178e613740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-20T21:30:42.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.629+0000 7f178e613740 4 datascan.main: data pool 'cephfs_data' has ID 17 2024-04-20T21:30:42.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.629+0000 7f178e613740 4 datascan.main: opening data pool 'cephfs_data' 2024-04-20T21:30:42.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.629+0000 7f178e613740 1 --2- 172.21.15.86:0/1122026518 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x562ee2319400 0x562ee22e8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.629+0000 7f178b200640 1 --2- 172.21.15.86:0/1122026518 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x562ee2450c00 0x562ee22e9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.629+0000 7f178e613740 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:1 16.14 16:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x562ee2319000 con 0x562ee2319400 2024-04-20T21:30:42.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.629+0000 7f178b200640 1 --2- 172.21.15.86:0/1122026518 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x562ee2319400 0x562ee22e8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.630+0000 7f178b200640 1 --2- 172.21.15.86:0/1122026518 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x562ee2450c00 0x562ee22e9700 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x562ee23957d0 tx=0x562ee231a960 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:42.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.630+0000 7f178b200640 1 --2- 172.21.15.86:0/1122026518 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x562ee2319400 0x562ee22e8680 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:42.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.630+0000 7f178b200640 1 -- 172.21.15.86:0/1122026518 <== osd.0 v2:172.21.15.86:6800/2471076035 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv38 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x562ee2308000 con 0x562ee2319400 2024-04-20T21:30:42.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.630+0000 7f178e613740 1 --2- 172.21.15.86:0/1122026518 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x562ee2318c00 0x562ee22e8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.630+0000 7f178e613740 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:2 16.1f 16:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x562ee2318800 con 0x562ee2318c00 2024-04-20T21:30:42.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.631+0000 7f178c600640 1 --2- 172.21.15.86:0/1122026518 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x562ee2318c00 0x562ee22e8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.632+0000 7f178c600640 1 --2- 172.21.15.86:0/1122026518 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x562ee2318c00 0x562ee22e8c00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:42.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.633+0000 7f178c600640 1 -- 172.21.15.86:0/1122026518 <== osd.5 v2:172.21.15.144:6808/1635974704 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) ==== 162+0+16 (crc 0 0 0) 0x562ee2308500 con 0x562ee2318c00 2024-04-20T21:30:42.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.633+0000 7f178e613740 1 --2- 172.21.15.86:0/1122026518 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x562ee1910000 0x562ee22e9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.633+0000 7f178e613740 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:3 17.0 17:00000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee1910400 con 0x562ee1910000 2024-04-20T21:30:42.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.633+0000 7f178bc00640 1 --2- 172.21.15.86:0/1122026518 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x562ee1910000 0x562ee22e9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.635+0000 7f178bc00640 1 --2- 172.21.15.86:0/1122026518 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x562ee1910000 0x562ee22e9180 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:42.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.635+0000 7f178bc00640 1 -- 172.21.15.86:0/1122026518 <== osd.7 v2:172.21.15.144:6816/2100967057 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 116 out=49b] v112'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308780 con 0x562ee1910000 2024-04-20T21:30:42.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.636+0000 7f1785800640 1 --2- 172.21.15.86:0/1122026518 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x562ee1911800 0x562ee1926000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.636+0000 7f1785800640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:4 17.20 17:04000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee1911c00 con 0x562ee1911800 2024-04-20T21:30:42.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.636+0000 7f178b200640 1 --2- 172.21.15.86:0/1122026518 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x562ee1911800 0x562ee1926000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.636+0000 7f178b200640 1 --2- 172.21.15.86:0/1122026518 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x562ee1911800 0x562ee1926000 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:42.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.637+0000 7f178b200640 1 -- 172.21.15.86:0/1122026518 <== osd.1 v2:172.21.15.86:6817/3149564961 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 116 out=49b] v112'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308000 con 0x562ee1911800 2024-04-20T21:30:42.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.637+0000 7f1786200640 1 --2- 172.21.15.86:0/1122026518 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x562ee192c000 0x562ee1926580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.637+0000 7f1786200640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:5 17.10 17:08000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee192c400 con 0x562ee192c000 2024-04-20T21:30:42.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.637+0000 7f178c600640 1 --2- 172.21.15.86:0/1122026518 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x562ee192c000 0x562ee1926580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.638+0000 7f178c600640 1 --2- 172.21.15.86:0/1122026518 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x562ee192c000 0x562ee1926580 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:42.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.638+0000 7f178c600640 1 -- 172.21.15.86:0/1122026518 <== osd.3 v2:172.21.15.86:6808/794135519 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 116 out=49b] v112'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308500 con 0x562ee192c000 2024-04-20T21:30:42.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.638+0000 7f1785800640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:6 17.30 17:0c000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee192cc00 con 0x562ee2319400 2024-04-20T21:30:42.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.638+0000 7f178b200640 1 -- 172.21.15.86:0/1122026518 <== osd.0 v2:172.21.15.86:6800/2471076035 2 ==== osd_op_reply(6 [pgnls-filter start_epoch 116 out=49b] v112'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308000 con 0x562ee2319400 2024-04-20T21:30:42.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.639+0000 7f1786200640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:7 17.8 17:10000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee192c800 con 0x562ee1910000 2024-04-20T21:30:42.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.639+0000 7f178bc00640 1 -- 172.21.15.86:0/1122026518 <== osd.7 v2:172.21.15.144:6816/2100967057 2 ==== osd_op_reply(7 [pgnls-filter start_epoch 116 out=49b] v112'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308780 con 0x562ee1910000 2024-04-20T21:30:42.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.639+0000 7f1785800640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:8 17.28 17:14000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee192d000 con 0x562ee192c000 2024-04-20T21:30:42.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.640+0000 7f178c600640 1 -- 172.21.15.86:0/1122026518 <== osd.3 v2:172.21.15.86:6808/794135519 2 ==== osd_op_reply(8 [pgnls-filter start_epoch 116 out=49b] v112'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308500 con 0x562ee192c000 2024-04-20T21:30:42.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.640+0000 7f1786200640 1 --2- 172.21.15.86:0/1122026518 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x562ee192d800 0x562ee1926b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.640+0000 7f1786200640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:9 17.18 17:18000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee192dc00 con 0x562ee192d800 2024-04-20T21:30:42.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.640+0000 7f178bc00640 1 --2- 172.21.15.86:0/1122026518 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x562ee192d800 0x562ee1926b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.641+0000 7f178bc00640 1 --2- 172.21.15.86:0/1122026518 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x562ee192d800 0x562ee1926b00 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:42.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.641+0000 7f178bc00640 1 -- 172.21.15.86:0/1122026518 <== osd.2 v2:172.21.15.86:6816/3437611468 1 ==== osd_op_reply(9 [pgnls-filter start_epoch 116 out=49b] v112'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308780 con 0x562ee192d800 2024-04-20T21:30:42.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.641+0000 7f1785800640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:10 17.38 17:1c000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee192d400 con 0x562ee1910000 2024-04-20T21:30:42.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.642+0000 7f178bc00640 1 -- 172.21.15.86:0/1122026518 <== osd.7 v2:172.21.15.144:6816/2100967057 3 ==== osd_op_reply(10 [pgnls-filter start_epoch 116 out=49b] v112'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308780 con 0x562ee1910000 2024-04-20T21:30:42.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.642+0000 7f1786200640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:11 17.4 17:20000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee192c800 con 0x562ee2319400 2024-04-20T21:30:42.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.642+0000 7f178b200640 1 -- 172.21.15.86:0/1122026518 <== osd.0 v2:172.21.15.86:6800/2471076035 3 ==== osd_op_reply(11 [pgnls-filter start_epoch 116 out=49b] v112'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308000 con 0x562ee2319400 2024-04-20T21:30:42.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.642+0000 7f1785800640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:12 17.24 17:24000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee1932000 con 0x562ee192c000 2024-04-20T21:30:42.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.643+0000 7f178c600640 1 -- 172.21.15.86:0/1122026518 <== osd.3 v2:172.21.15.86:6808/794135519 3 ==== osd_op_reply(12 [pgnls-filter start_epoch 116 out=49b] v112'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308500 con 0x562ee192c000 2024-04-20T21:30:42.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.643+0000 7f1786200640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:13 17.14 17:28000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee192dc00 con 0x562ee192c000 2024-04-20T21:30:42.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.643+0000 7f178c600640 1 -- 172.21.15.86:0/1122026518 <== osd.3 v2:172.21.15.86:6808/794135519 4 ==== osd_op_reply(13 [pgnls-filter start_epoch 116 out=49b] v112'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308500 con 0x562ee192c000 2024-04-20T21:30:42.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.643+0000 7f1785800640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:14 17.34 17:2c000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee1932400 con 0x562ee2319400 2024-04-20T21:30:42.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.644+0000 7f178b200640 1 -- 172.21.15.86:0/1122026518 <== osd.0 v2:172.21.15.86:6800/2471076035 4 ==== osd_op_reply(14 [pgnls-filter start_epoch 116 out=49b] v112'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308000 con 0x562ee2319400 2024-04-20T21:30:42.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.644+0000 7f1786200640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:15 17.c 17:30000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee192d400 con 0x562ee1910000 2024-04-20T21:30:42.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.645+0000 7f178bc00640 1 -- 172.21.15.86:0/1122026518 <== osd.7 v2:172.21.15.144:6816/2100967057 4 ==== osd_op_reply(15 [pgnls-filter start_epoch 116 out=49b] v112'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308780 con 0x562ee1910000 2024-04-20T21:30:42.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.645+0000 7f1785800640 1 --2- 172.21.15.86:0/1122026518 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x562ee1932800 0x562ee1927600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.645+0000 7f1785800640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:16 17.2c 17:34000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee1932c00 con 0x562ee1932800 2024-04-20T21:30:42.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.645+0000 7f178b200640 1 --2- 172.21.15.86:0/1122026518 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x562ee1932800 0x562ee1927600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.646+0000 7f178b200640 1 --2- 172.21.15.86:0/1122026518 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x562ee1932800 0x562ee1927600 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:42.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.647+0000 7f178b200640 1 -- 172.21.15.86:0/1122026518 <== osd.4 v2:172.21.15.144:6800/4087309475 1 ==== osd_op_reply(16 [pgnls-filter start_epoch 116 out=49b] v112'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308000 con 0x562ee1932800 2024-04-20T21:30:42.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.647+0000 7f1786200640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:17 17.1c 17:38000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee1910400 con 0x562ee1911800 2024-04-20T21:30:42.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.648+0000 7f178b200640 1 -- 172.21.15.86:0/1122026518 <== osd.1 v2:172.21.15.86:6817/3149564961 2 ==== osd_op_reply(17 [pgnls-filter start_epoch 116 out=49b] v112'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308000 con 0x562ee1911800 2024-04-20T21:30:42.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.648+0000 7f1785800640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:18 17.3c 17:3c000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee2319000 con 0x562ee2319400 2024-04-20T21:30:42.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.648+0000 7f178b200640 1 -- 172.21.15.86:0/1122026518 <== osd.0 v2:172.21.15.86:6800/2471076035 5 ==== osd_op_reply(18 [pgnls-filter start_epoch 116 out=49b] v112'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308000 con 0x562ee2319400 2024-04-20T21:30:42.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.648+0000 7f1786200640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:19 17.2 17:40000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee192cc00 con 0x562ee1932800 2024-04-20T21:30:42.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.649+0000 7f178b200640 1 -- 172.21.15.86:0/1122026518 <== osd.4 v2:172.21.15.144:6800/4087309475 2 ==== osd_op_reply(19 [pgnls-filter start_epoch 116 out=49b] v112'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308000 con 0x562ee1932800 2024-04-20T21:30:42.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.649+0000 7f1785800640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:20 17.22 17:44000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee2372800 con 0x562ee2318c00 2024-04-20T21:30:42.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.650+0000 7f178c600640 1 -- 172.21.15.86:0/1122026518 <== osd.5 v2:172.21.15.144:6808/1635974704 2 ==== osd_op_reply(20 [pgnls-filter start_epoch 116 out=49b] v112'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308500 con 0x562ee2318c00 2024-04-20T21:30:42.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.650+0000 7f1786200640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:21 17.12 17:48000000::::head [pgnls-filter start_epoch 116 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee192c800 con 0x562ee192c000 2024-04-20T21:30:42.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.651+0000 7f178c600640 1 -- 172.21.15.86:0/1122026518 <== osd.3 v2:172.21.15.86:6808/794135519 5 ==== osd_op_reply(21 [pgnls-filter start_epoch 116 out=81b] v116'23 uv23 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x562ee2308500 con 0x562ee192c000 2024-04-20T21:30:42.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.651+0000 7f178e613740 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:22 17.0 17:00000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee1910800 con 0x562ee1910000 2024-04-20T21:30:42.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.652+0000 7f178bc00640 1 -- 172.21.15.86:0/1122026518 <== osd.7 v2:172.21.15.144:6816/2100967057 5 ==== osd_op_reply(22 [pgnls-filter start_epoch 116 out=49b] v112'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308780 con 0x562ee1910000 2024-04-20T21:30:42.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.652+0000 7f1786200640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:23 17.20 17:04000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee192c400 con 0x562ee1911800 2024-04-20T21:30:42.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.653+0000 7f178b200640 1 -- 172.21.15.86:0/1122026518 <== osd.1 v2:172.21.15.86:6817/3149564961 3 ==== osd_op_reply(23 [pgnls-filter start_epoch 116 out=49b] v112'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308000 con 0x562ee1911800 2024-04-20T21:30:42.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.653+0000 7f1785800640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:24 17.10 17:08000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee1911c00 con 0x562ee192c000 2024-04-20T21:30:42.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.653+0000 7f178c600640 1 -- 172.21.15.86:0/1122026518 <== osd.3 v2:172.21.15.86:6808/794135519 6 ==== osd_op_reply(24 [pgnls-filter start_epoch 116 out=49b] v112'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308500 con 0x562ee192c000 2024-04-20T21:30:42.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.653+0000 7f1786200640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:25 17.30 17:0c000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee192d000 con 0x562ee2319400 2024-04-20T21:30:42.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.654+0000 7f178b200640 1 -- 172.21.15.86:0/1122026518 <== osd.0 v2:172.21.15.86:6800/2471076035 6 ==== osd_op_reply(25 [pgnls-filter start_epoch 116 out=49b] v112'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308000 con 0x562ee2319400 2024-04-20T21:30:42.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.654+0000 7f1785800640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:26 17.8 17:10000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee2451c00 con 0x562ee1910000 2024-04-20T21:30:42.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.655+0000 7f178bc00640 1 -- 172.21.15.86:0/1122026518 <== osd.7 v2:172.21.15.144:6816/2100967057 6 ==== osd_op_reply(26 [pgnls-filter start_epoch 116 out=49b] v112'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308780 con 0x562ee1910000 2024-04-20T21:30:42.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.655+0000 7f1786200640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:27 17.28 17:14000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee2318800 con 0x562ee192c000 2024-04-20T21:30:42.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.655+0000 7f178c600640 1 -- 172.21.15.86:0/1122026518 <== osd.3 v2:172.21.15.86:6808/794135519 7 ==== osd_op_reply(27 [pgnls-filter start_epoch 116 out=49b] v112'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308500 con 0x562ee192c000 2024-04-20T21:30:42.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.655+0000 7f1785800640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:28 17.18 17:18000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee2451800 con 0x562ee192d800 2024-04-20T21:30:42.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.656+0000 7f178bc00640 1 -- 172.21.15.86:0/1122026518 <== osd.2 v2:172.21.15.86:6816/3437611468 2 ==== osd_op_reply(28 [pgnls-filter start_epoch 116 out=49b] v112'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308780 con 0x562ee192d800 2024-04-20T21:30:42.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.656+0000 7f1786200640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:29 17.38 17:1c000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee1932400 con 0x562ee1910000 2024-04-20T21:30:42.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.657+0000 7f178bc00640 1 -- 172.21.15.86:0/1122026518 <== osd.7 v2:172.21.15.144:6816/2100967057 7 ==== osd_op_reply(29 [pgnls-filter start_epoch 116 out=49b] v112'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308780 con 0x562ee1910000 2024-04-20T21:30:42.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.657+0000 7f1785800640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:30 17.4 17:20000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee1932400 con 0x562ee2319400 2024-04-20T21:30:42.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.658+0000 7f178b200640 1 -- 172.21.15.86:0/1122026518 <== osd.0 v2:172.21.15.86:6800/2471076035 7 ==== osd_op_reply(30 [pgnls-filter start_epoch 116 out=49b] v112'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308000 con 0x562ee2319400 2024-04-20T21:30:42.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.658+0000 7f1786200640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:31 17.24 17:24000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee1932000 con 0x562ee192c000 2024-04-20T21:30:42.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.658+0000 7f178c600640 1 -- 172.21.15.86:0/1122026518 <== osd.3 v2:172.21.15.86:6808/794135519 8 ==== osd_op_reply(31 [pgnls-filter start_epoch 116 out=49b] v112'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308500 con 0x562ee192c000 2024-04-20T21:30:42.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.658+0000 7f1785800640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:32 17.14 17:28000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee2451c00 con 0x562ee192c000 2024-04-20T21:30:42.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.659+0000 7f178c600640 1 -- 172.21.15.86:0/1122026518 <== osd.3 v2:172.21.15.86:6808/794135519 9 ==== osd_op_reply(32 [pgnls-filter start_epoch 116 out=49b] v112'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308500 con 0x562ee192c000 2024-04-20T21:30:42.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.659+0000 7f1786200640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:33 17.34 17:2c000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee1933000 con 0x562ee2319400 2024-04-20T21:30:42.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.659+0000 7f178b200640 1 -- 172.21.15.86:0/1122026518 <== osd.0 v2:172.21.15.86:6800/2471076035 8 ==== osd_op_reply(33 [pgnls-filter start_epoch 116 out=49b] v112'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308000 con 0x562ee2319400 2024-04-20T21:30:42.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.659+0000 7f1785800640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:34 17.c 17:30000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee2451800 con 0x562ee1910000 2024-04-20T21:30:42.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.661+0000 7f178bc00640 1 -- 172.21.15.86:0/1122026518 <== osd.7 v2:172.21.15.144:6816/2100967057 8 ==== osd_op_reply(34 [pgnls-filter start_epoch 116 out=49b] v112'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308780 con 0x562ee1910000 2024-04-20T21:30:42.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.661+0000 7f1786200640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:35 17.2c 17:34000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee1933400 con 0x562ee1932800 2024-04-20T21:30:42.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.662+0000 7f178b200640 1 -- 172.21.15.86:0/1122026518 <== osd.4 v2:172.21.15.144:6800/4087309475 3 ==== osd_op_reply(35 [pgnls-filter start_epoch 116 out=49b] v112'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308000 con 0x562ee1932800 2024-04-20T21:30:42.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.662+0000 7f1785800640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:36 17.1c 17:38000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee1910800 con 0x562ee1911800 2024-04-20T21:30:42.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.663+0000 7f178b200640 1 -- 172.21.15.86:0/1122026518 <== osd.1 v2:172.21.15.86:6817/3149564961 4 ==== osd_op_reply(36 [pgnls-filter start_epoch 116 out=49b] v112'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308000 con 0x562ee1911800 2024-04-20T21:30:42.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.663+0000 7f1786200640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:37 17.3c 17:3c000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee1933800 con 0x562ee2319400 2024-04-20T21:30:42.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.663+0000 7f178b200640 1 -- 172.21.15.86:0/1122026518 <== osd.0 v2:172.21.15.86:6800/2471076035 9 ==== osd_op_reply(37 [pgnls-filter start_epoch 116 out=49b] v112'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308000 con 0x562ee2319400 2024-04-20T21:30:42.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.663+0000 7f1785800640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:38 17.2 17:40000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee192d400 con 0x562ee1932800 2024-04-20T21:30:42.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.664+0000 7f178b200640 1 -- 172.21.15.86:0/1122026518 <== osd.4 v2:172.21.15.144:6800/4087309475 4 ==== osd_op_reply(38 [pgnls-filter start_epoch 116 out=49b] v112'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308000 con 0x562ee1932800 2024-04-20T21:30:42.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.665+0000 7f1786200640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:39 17.22 17:44000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee1933c00 con 0x562ee2318c00 2024-04-20T21:30:42.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.666+0000 7f178c600640 1 -- 172.21.15.86:0/1122026518 <== osd.5 v2:172.21.15.144:6808/1635974704 3 ==== osd_op_reply(39 [pgnls-filter start_epoch 116 out=49b] v112'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308500 con 0x562ee2318c00 2024-04-20T21:30:42.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.666+0000 7f1785800640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:40 17.12 17:48000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee1910400 con 0x562ee192c000 2024-04-20T21:30:42.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.666+0000 7f178c600640 1 -- 172.21.15.86:0/1122026518 <== osd.3 v2:172.21.15.86:6808/794135519 10 ==== osd_op_reply(40 [pgnls-filter start_epoch 116 out=49b] v116'23 uv23 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308500 con 0x562ee192c000 2024-04-20T21:30:42.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.666+0000 7f1786200640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:41 17.32 17:4c000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee1932c00 con 0x562ee1910000 2024-04-20T21:30:42.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.668+0000 7f178bc00640 1 -- 172.21.15.86:0/1122026518 <== osd.7 v2:172.21.15.144:6816/2100967057 9 ==== osd_op_reply(41 [pgnls-filter start_epoch 116 out=49b] v112'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308780 con 0x562ee1910000 2024-04-20T21:30:42.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.668+0000 7f1785800640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:42 17.a 17:50000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee192cc00 con 0x562ee1910000 2024-04-20T21:30:42.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.669+0000 7f178bc00640 1 -- 172.21.15.86:0/1122026518 <== osd.7 v2:172.21.15.144:6816/2100967057 10 ==== osd_op_reply(42 [pgnls-filter start_epoch 116 out=49b] v112'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308780 con 0x562ee1910000 2024-04-20T21:30:42.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.669+0000 7f1786200640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:43 17.2a 17:54000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee1932400 con 0x562ee192c000 2024-04-20T21:30:42.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.670+0000 7f178c600640 1 -- 172.21.15.86:0/1122026518 <== osd.3 v2:172.21.15.86:6808/794135519 11 ==== osd_op_reply(43 [pgnls-filter start_epoch 116 out=49b] v112'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308500 con 0x562ee192c000 2024-04-20T21:30:42.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.670+0000 7f1785800640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:44 17.1a 17:58000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee192c400 con 0x562ee2319400 2024-04-20T21:30:42.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.670+0000 7f178b200640 1 -- 172.21.15.86:0/1122026518 <== osd.0 v2:172.21.15.86:6800/2471076035 10 ==== osd_op_reply(44 [pgnls-filter start_epoch 116 out=49b] v112'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308000 con 0x562ee2319400 2024-04-20T21:30:42.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.670+0000 7f1786200640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:45 17.3a 17:5c000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee1933000 con 0x562ee1911800 2024-04-20T21:30:42.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.671+0000 7f178b200640 1 -- 172.21.15.86:0/1122026518 <== osd.1 v2:172.21.15.86:6817/3149564961 5 ==== osd_op_reply(45 [pgnls-filter start_epoch 116 out=49b] v112'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308000 con 0x562ee1911800 2024-04-20T21:30:42.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.671+0000 7f1785800640 1 --2- 172.21.15.86:0/1122026518 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x562ee192d000 0x562ee1927b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.671+0000 7f1785800640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:46 17.6 17:60000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee192dc00 con 0x562ee192d000 2024-04-20T21:30:42.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.672+0000 7f178c600640 1 --2- 172.21.15.86:0/1122026518 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x562ee192d000 0x562ee1927b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.673+0000 7f178c600640 1 --2- 172.21.15.86:0/1122026518 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x562ee192d000 0x562ee1927b80 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:42.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.674+0000 7f178c600640 1 -- 172.21.15.86:0/1122026518 <== osd.6 v2:172.21.15.144:6824/1893803079 1 ==== osd_op_reply(46 [pgnls-filter start_epoch 116 out=49b] v112'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308500 con 0x562ee192d000 2024-04-20T21:30:42.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.674+0000 7f1786200640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:47 17.26 17:64000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee1932000 con 0x562ee2319400 2024-04-20T21:30:42.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.675+0000 7f178b200640 1 -- 172.21.15.86:0/1122026518 <== osd.0 v2:172.21.15.86:6800/2471076035 11 ==== osd_op_reply(47 [pgnls-filter start_epoch 116 out=49b] v112'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308000 con 0x562ee2319400 2024-04-20T21:30:42.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.675+0000 7f1785800640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:48 17.16 17:68000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee192c800 con 0x562ee2318c00 2024-04-20T21:30:42.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.676+0000 7f178c600640 1 -- 172.21.15.86:0/1122026518 <== osd.5 v2:172.21.15.144:6808/1635974704 4 ==== osd_op_reply(48 [pgnls-filter start_epoch 116 out=49b] v112'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308500 con 0x562ee2318c00 2024-04-20T21:30:42.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.676+0000 7f1786200640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:49 17.36 17:6c000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee2319000 con 0x562ee2319400 2024-04-20T21:30:42.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.677+0000 7f178b200640 1 -- 172.21.15.86:0/1122026518 <== osd.0 v2:172.21.15.86:6800/2471076035 12 ==== osd_op_reply(49 [pgnls-filter start_epoch 116 out=81b] v116'34 uv34 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x562ee2308000 con 0x562ee2319400 2024-04-20T21:30:42.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.677+0000 7f178e613740 20 datascan.forall_objects: OSD matched oid 100000001f7.00000000 2024-04-20T21:30:42.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.677+0000 7f178e613740 10 datascan.operator(): handling object 100000001f7.0 2024-04-20T21:30:42.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.677+0000 7f178e613740 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:50 17.36 17:6fe38dc2:::100000001f7.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x562ee1911000 con 0x562ee2319400 2024-04-20T21:30:42.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.677+0000 7f178b200640 1 -- 172.21.15.86:0/1122026518 <== osd.0 v2:172.21.15.86:6800/2471076035 13 ==== osd_op_reply(50 100000001f7.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (126) out=126b,getxattr (30) out=30b,getxattr] v0'0 uv34 ondisk = 0) ==== 416+0+194 (crc 0 0 0) 0x562ee2308000 con 0x562ee2319400 2024-04-20T21:30:42.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.677+0000 7f178e613740 10 datascan.inject_with_backtrace: inode: 0x0x100000001f7 2024-04-20T21:30:42.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.677+0000 7f178e613740 10 datascan.inject_with_backtrace: backptr: 0x0x10000000001/bravo 2024-04-20T21:30:42.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.677+0000 7f178e613740 20 datascan.get_frag_of: dirino=0x10000000001 target_dname=bravo 2024-04-20T21:30:42.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.678+0000 7f178e613740 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:51 16.7 16:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x562ee1911400 con 0x562ee192c000 2024-04-20T21:30:42.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.678+0000 7f178c600640 1 -- 172.21.15.86:0/1122026518 <== osd.3 v2:172.21.15.86:6808/794135519 12 ==== osd_op_reply(51 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv7 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x562ee2308500 con 0x562ee192c000 2024-04-20T21:30:42.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.678+0000 7f178e613740 20 datascan.get_frag_of: got backtrace parent 0x10000000000/flushed 2024-04-20T21:30:42.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.678+0000 7f178e613740 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:52 16.10 16:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x562ee1933c00 con 0x562ee192d800 2024-04-20T21:30:42.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.679+0000 7f178bc00640 1 -- 172.21.15.86:0/1122026518 <== osd.2 v2:172.21.15.86:6816/3437611468 3 ==== osd_op_reply(52 10000000000.00000000 [omap-get-vals-by-keys out=524b] v0'0 uv12 ondisk = 0) ==== 164+0+524 (crc 0 0 0) 0x562ee2308780 con 0x562ee192d800 2024-04-20T21:30:42.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.679+0000 7f178e613740 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed frag=*, r=0 2024-04-20T21:30:42.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.679+0000 7f178e613740 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*) 2024-04-20T21:30:42.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.679+0000 7f178e613740 20 datascan.get_frag_of: frag is * 2024-04-20T21:30:42.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.679+0000 7f178e613740 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:53 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x562ee1932400 con 0x562ee192c000 2024-04-20T21:30:42.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.680+0000 7f178c600640 1 -- 172.21.15.86:0/1122026518 <== osd.3 v2:172.21.15.86:6808/794135519 13 ==== osd_op_reply(53 10000000001.00000000 [omap-get-header out=274b] v0'0 uv7 ondisk = 0) ==== 164+0+274 (crc 0 0 0) 0x562ee2308500 con 0x562ee192c000 2024-04-20T21:30:42.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.680+0000 7f178e613740 10 datascan.find_or_create_dirfrag: read_version = 7 2024-04-20T21:30:42.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.680+0000 7f178e613740 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000001 * 2024-04-20T21:30:42.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.680+0000 7f178e613740 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:54 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x562ee192dc00 con 0x562ee192c000 2024-04-20T21:30:42.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.680+0000 7f178c600640 1 -- 172.21.15.86:0/1122026518 <== osd.3 v2:172.21.15.86:6808/794135519 14 ==== osd_op_reply(54 10000000001.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv7 ondisk = 0) ==== 164+0+4 (crc 0 0 0) 0x562ee2308500 con 0x562ee192c000 2024-04-20T21:30:42.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.680+0000 7f178e613740 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-04-20T21:30:42.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.680+0000 7f178e613740 20 datascan.read_dentry: bravo_head not found in result 2024-04-20T21:30:42.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.680+0000 7f178e613740 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f7 at 0x0x10000000001/bravo with size=5 bytes 2024-04-20T21:30:42.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.680+0000 7f178e613740 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:55 16.7 16:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x562ee192c800 con 0x562ee192c000 2024-04-20T21:30:42.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.683+0000 7f178c600640 1 -- 172.21.15.86:0/1122026518 <== osd.3 v2:172.21.15.86:6808/794135519 15 ==== osd_op_reply(55 10000000001.00000000 [omap-set-vals] v116'9 uv9 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x562ee2308500 con 0x562ee192c000 2024-04-20T21:30:42.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.683+0000 7f178e613740 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2024-04-20T21:30:42.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.683+0000 7f178e613740 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:56 17.e 17:70000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee192d400 con 0x562ee192d800 2024-04-20T21:30:42.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.684+0000 7f178bc00640 1 -- 172.21.15.86:0/1122026518 <== osd.2 v2:172.21.15.86:6816/3437611468 4 ==== osd_op_reply(56 [pgnls-filter start_epoch 116 out=49b] v112'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308780 con 0x562ee192d800 2024-04-20T21:30:42.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.684+0000 7f1786200640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:57 17.2e 17:74000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee2318800 con 0x562ee192d000 2024-04-20T21:30:42.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.685+0000 7f178c600640 1 -- 172.21.15.86:0/1122026518 <== osd.6 v2:172.21.15.144:6824/1893803079 2 ==== osd_op_reply(57 [pgnls-filter start_epoch 116 out=49b] v112'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308500 con 0x562ee192d000 2024-04-20T21:30:42.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.685+0000 7f1785800640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:58 17.1e 17:78000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee2372800 con 0x562ee1911800 2024-04-20T21:30:42.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.686+0000 7f178b200640 1 -- 172.21.15.86:0/1122026518 <== osd.1 v2:172.21.15.86:6817/3149564961 6 ==== osd_op_reply(58 [pgnls-filter start_epoch 116 out=49b] v112'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308000 con 0x562ee1911800 2024-04-20T21:30:42.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.686+0000 7f1786200640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:59 17.3e 17:7c000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee1911c00 con 0x562ee1932800 2024-04-20T21:30:42.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.687+0000 7f178b200640 1 -- 172.21.15.86:0/1122026518 <== osd.4 v2:172.21.15.144:6800/4087309475 5 ==== osd_op_reply(59 [pgnls-filter start_epoch 116 out=49b] v112'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308000 con 0x562ee1932800 2024-04-20T21:30:42.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.687+0000 7f1785800640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:60 17.1 17:80000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee1910400 con 0x562ee2319400 2024-04-20T21:30:42.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.687+0000 7f178b200640 1 -- 172.21.15.86:0/1122026518 <== osd.0 v2:172.21.15.86:6800/2471076035 14 ==== osd_op_reply(60 [pgnls-filter start_epoch 116 out=49b] v112'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308000 con 0x562ee2319400 2024-04-20T21:30:42.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.687+0000 7f1786200640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:61 17.21 17:84000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee1911000 con 0x562ee2319400 2024-04-20T21:30:42.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.688+0000 7f178b200640 1 -- 172.21.15.86:0/1122026518 <== osd.0 v2:172.21.15.86:6800/2471076035 15 ==== osd_op_reply(61 [pgnls-filter start_epoch 116 out=49b] v112'24 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308000 con 0x562ee2319400 2024-04-20T21:30:42.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.688+0000 7f1785800640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:62 17.11 17:88000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee1911400 con 0x562ee192c000 2024-04-20T21:30:42.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.689+0000 7f178c600640 1 -- 172.21.15.86:0/1122026518 <== osd.3 v2:172.21.15.86:6808/794135519 16 ==== osd_op_reply(62 [pgnls-filter start_epoch 116 out=49b] v112'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308500 con 0x562ee192c000 2024-04-20T21:30:42.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.689+0000 7f1786200640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:63 17.31 17:8c000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee194c000 con 0x562ee1911800 2024-04-20T21:30:42.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.690+0000 7f178b200640 1 -- 172.21.15.86:0/1122026518 <== osd.1 v2:172.21.15.86:6817/3149564961 7 ==== osd_op_reply(63 [pgnls-filter start_epoch 116 out=49b] v112'15 uv8 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308000 con 0x562ee1911800 2024-04-20T21:30:42.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.690+0000 7f1785800640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:64 17.9 17:90000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee1910800 con 0x562ee192c000 2024-04-20T21:30:42.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.691+0000 7f178c600640 1 -- 172.21.15.86:0/1122026518 <== osd.3 v2:172.21.15.86:6808/794135519 17 ==== osd_op_reply(64 [pgnls-filter start_epoch 116 out=49b] v112'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308500 con 0x562ee192c000 2024-04-20T21:30:42.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.691+0000 7f1786200640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:65 17.29 17:94000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee194c400 con 0x562ee1932800 2024-04-20T21:30:42.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.692+0000 7f178b200640 1 -- 172.21.15.86:0/1122026518 <== osd.4 v2:172.21.15.144:6800/4087309475 6 ==== osd_op_reply(65 [pgnls-filter start_epoch 116 out=49b] v112'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308000 con 0x562ee1932800 2024-04-20T21:30:42.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.692+0000 7f1785800640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:66 17.19 17:98000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee2319000 con 0x562ee2318c00 2024-04-20T21:30:42.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.694+0000 7f178c600640 1 -- 172.21.15.86:0/1122026518 <== osd.5 v2:172.21.15.144:6808/1635974704 5 ==== osd_op_reply(66 [pgnls-filter start_epoch 116 out=49b] v116'24 uv24 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308500 con 0x562ee2318c00 2024-04-20T21:30:42.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.694+0000 7f1786200640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:67 17.39 17:9c000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee194c800 con 0x562ee2318c00 2024-04-20T21:30:42.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.695+0000 7f178c600640 1 -- 172.21.15.86:0/1122026518 <== osd.5 v2:172.21.15.144:6808/1635974704 6 ==== osd_op_reply(67 [pgnls-filter start_epoch 116 out=49b] v112'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308500 con 0x562ee2318c00 2024-04-20T21:30:42.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.695+0000 7f1785800640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:68 17.5 17:a0000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee1933400 con 0x562ee192d000 2024-04-20T21:30:42.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.696+0000 7f178c600640 1 -- 172.21.15.86:0/1122026518 <== osd.6 v2:172.21.15.144:6824/1893803079 3 ==== osd_op_reply(68 [pgnls-filter start_epoch 116 out=49b] v116'15 uv15 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308500 con 0x562ee192d000 2024-04-20T21:30:42.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.696+0000 7f1786200640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:69 17.25 17:a4000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee194cc00 con 0x562ee1911800 2024-04-20T21:30:42.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.697+0000 7f178b200640 1 -- 172.21.15.86:0/1122026518 <== osd.1 v2:172.21.15.86:6817/3149564961 8 ==== osd_op_reply(69 [pgnls-filter start_epoch 116 out=49b] v112'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308000 con 0x562ee1911800 2024-04-20T21:30:42.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.697+0000 7f1785800640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:70 17.15 17:a8000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee1933800 con 0x562ee2318c00 2024-04-20T21:30:42.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.698+0000 7f178c600640 1 -- 172.21.15.86:0/1122026518 <== osd.5 v2:172.21.15.144:6808/1635974704 7 ==== osd_op_reply(70 [pgnls-filter start_epoch 116 out=49b] v112'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308500 con 0x562ee2318c00 2024-04-20T21:30:42.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.698+0000 7f1786200640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:71 17.35 17:ac000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee194d000 con 0x562ee1932800 2024-04-20T21:30:42.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.699+0000 7f178b200640 1 -- 172.21.15.86:0/1122026518 <== osd.4 v2:172.21.15.144:6800/4087309475 7 ==== osd_op_reply(71 [pgnls-filter start_epoch 116 out=49b] v112'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308000 con 0x562ee1932800 2024-04-20T21:30:42.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.700+0000 7f1785800640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:72 17.d 17:b0000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee1933000 con 0x562ee1910000 2024-04-20T21:30:42.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.701+0000 7f178bc00640 1 -- 172.21.15.86:0/1122026518 <== osd.7 v2:172.21.15.144:6816/2100967057 11 ==== osd_op_reply(72 [pgnls-filter start_epoch 116 out=49b] v112'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308780 con 0x562ee1910000 2024-04-20T21:30:42.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.701+0000 7f1786200640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:73 17.2d 17:b4000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee194d400 con 0x562ee1910000 2024-04-20T21:30:42.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.702+0000 7f178bc00640 1 -- 172.21.15.86:0/1122026518 <== osd.7 v2:172.21.15.144:6816/2100967057 12 ==== osd_op_reply(73 [pgnls-filter start_epoch 116 out=49b] v112'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308780 con 0x562ee1910000 2024-04-20T21:30:42.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.702+0000 7f1785800640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:74 17.1d 17:b8000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee1932000 con 0x562ee1910000 2024-04-20T21:30:42.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.704+0000 7f178bc00640 1 -- 172.21.15.86:0/1122026518 <== osd.7 v2:172.21.15.144:6816/2100967057 13 ==== osd_op_reply(74 [pgnls-filter start_epoch 116 out=49b] v112'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308780 con 0x562ee1910000 2024-04-20T21:30:42.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.704+0000 7f1786200640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:75 17.3d 17:bc000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee194d800 con 0x562ee1910000 2024-04-20T21:30:42.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.705+0000 7f178bc00640 1 -- 172.21.15.86:0/1122026518 <== osd.7 v2:172.21.15.144:6816/2100967057 14 ==== osd_op_reply(75 [pgnls-filter start_epoch 116 out=49b] v112'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308780 con 0x562ee1910000 2024-04-20T21:30:42.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.705+0000 7f1785800640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:76 17.3 17:c0000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee192cc00 con 0x562ee1932800 2024-04-20T21:30:42.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.706+0000 7f178b200640 1 -- 172.21.15.86:0/1122026518 <== osd.4 v2:172.21.15.144:6800/4087309475 8 ==== osd_op_reply(76 [pgnls-filter start_epoch 116 out=49b] v112'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308000 con 0x562ee1932800 2024-04-20T21:30:42.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.706+0000 7f1786200640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:77 17.23 17:c4000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee194dc00 con 0x562ee1932800 2024-04-20T21:30:42.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.707+0000 7f178b200640 1 -- 172.21.15.86:0/1122026518 <== osd.4 v2:172.21.15.144:6800/4087309475 9 ==== osd_op_reply(77 [pgnls-filter start_epoch 116 out=49b] v112'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308000 con 0x562ee1932800 2024-04-20T21:30:42.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.708+0000 7f1785800640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:78 17.13 17:c8000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee192d400 con 0x562ee192c000 2024-04-20T21:30:42.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.708+0000 7f178c600640 1 -- 172.21.15.86:0/1122026518 <== osd.3 v2:172.21.15.86:6808/794135519 18 ==== osd_op_reply(78 [pgnls-filter start_epoch 116 out=49b] v112'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308500 con 0x562ee192c000 2024-04-20T21:30:42.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.709+0000 7f1786200640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:79 17.33 17:cc000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee2318800 con 0x562ee1910000 2024-04-20T21:30:42.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.710+0000 7f178bc00640 1 -- 172.21.15.86:0/1122026518 <== osd.7 v2:172.21.15.144:6816/2100967057 15 ==== osd_op_reply(79 [pgnls-filter start_epoch 116 out=49b] v112'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308780 con 0x562ee1910000 2024-04-20T21:30:42.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.710+0000 7f1785800640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:80 17.b 17:d0000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee192dc00 con 0x562ee2319400 2024-04-20T21:30:42.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.711+0000 7f178b200640 1 -- 172.21.15.86:0/1122026518 <== osd.0 v2:172.21.15.86:6800/2471076035 16 ==== osd_op_reply(80 [pgnls-filter start_epoch 116 out=49b] v112'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308000 con 0x562ee2319400 2024-04-20T21:30:42.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.711+0000 7f1786200640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:81 17.2b 17:d4000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee2319000 con 0x562ee1932800 2024-04-20T21:30:42.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.712+0000 7f178b200640 1 -- 172.21.15.86:0/1122026518 <== osd.4 v2:172.21.15.144:6800/4087309475 10 ==== osd_op_reply(81 [pgnls-filter start_epoch 116 out=49b] v112'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308000 con 0x562ee1932800 2024-04-20T21:30:42.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.712+0000 7f1785800640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:82 17.1b 17:d8000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee192c800 con 0x562ee1932800 2024-04-20T21:30:42.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.713+0000 7f178b200640 1 -- 172.21.15.86:0/1122026518 <== osd.4 v2:172.21.15.144:6800/4087309475 11 ==== osd_op_reply(82 [pgnls-filter start_epoch 116 out=49b] v112'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308000 con 0x562ee1932800 2024-04-20T21:30:42.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.713+0000 7f1786200640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:83 17.3b 17:dc000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee1910c00 con 0x562ee192d800 2024-04-20T21:30:42.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.714+0000 7f178bc00640 1 -- 172.21.15.86:0/1122026518 <== osd.2 v2:172.21.15.86:6816/3437611468 5 ==== osd_op_reply(83 [pgnls-filter start_epoch 116 out=49b] v112'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308780 con 0x562ee192d800 2024-04-20T21:30:42.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.714+0000 7f1785800640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:84 17.7 17:e0000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee1932c00 con 0x562ee2319400 2024-04-20T21:30:42.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.715+0000 7f178b200640 1 -- 172.21.15.86:0/1122026518 <== osd.0 v2:172.21.15.86:6800/2471076035 17 ==== osd_op_reply(84 [pgnls-filter start_epoch 116 out=49b] v112'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308000 con 0x562ee2319400 2024-04-20T21:30:42.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.716+0000 7f1786200640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:85 17.27 17:e4000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee1911c00 con 0x562ee1932800 2024-04-20T21:30:42.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.717+0000 7f178b200640 1 -- 172.21.15.86:0/1122026518 <== osd.4 v2:172.21.15.144:6800/4087309475 12 ==== osd_op_reply(85 [pgnls-filter start_epoch 116 out=49b] v112'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308000 con 0x562ee1932800 2024-04-20T21:30:42.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.717+0000 7f1785800640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:86 17.17 17:e8000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee1933c00 con 0x562ee192d000 2024-04-20T21:30:42.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.718+0000 7f178c600640 1 -- 172.21.15.86:0/1122026518 <== osd.6 v2:172.21.15.144:6824/1893803079 4 ==== osd_op_reply(86 [pgnls-filter start_epoch 116 out=49b] v112'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308500 con 0x562ee192d000 2024-04-20T21:30:42.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.718+0000 7f1786200640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:87 17.37 17:ec000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee1910400 con 0x562ee2319400 2024-04-20T21:30:42.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.719+0000 7f178b200640 1 -- 172.21.15.86:0/1122026518 <== osd.0 v2:172.21.15.86:6800/2471076035 18 ==== osd_op_reply(87 [pgnls-filter start_epoch 116 out=49b] v112'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308000 con 0x562ee2319400 2024-04-20T21:30:42.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.719+0000 7f1785800640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:88 17.f 17:f0000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee1933000 con 0x562ee1911800 2024-04-20T21:30:42.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.720+0000 7f178b200640 1 -- 172.21.15.86:0/1122026518 <== osd.1 v2:172.21.15.86:6817/3149564961 9 ==== osd_op_reply(88 [pgnls-filter start_epoch 116 out=49b] v112'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308000 con 0x562ee1911800 2024-04-20T21:30:42.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.720+0000 7f1786200640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:89 17.2f 17:f4000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee1911000 con 0x562ee1932800 2024-04-20T21:30:42.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.721+0000 7f178b200640 1 -- 172.21.15.86:0/1122026518 <== osd.4 v2:172.21.15.144:6800/4087309475 13 ==== osd_op_reply(89 [pgnls-filter start_epoch 116 out=49b] v112'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308000 con 0x562ee1932800 2024-04-20T21:30:42.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.721+0000 7f1785800640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:90 17.1f 17:f8000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee1932400 con 0x562ee192d000 2024-04-20T21:30:42.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.723+0000 7f178c600640 1 -- 172.21.15.86:0/1122026518 <== osd.6 v2:172.21.15.144:6824/1893803079 5 ==== osd_op_reply(90 [pgnls-filter start_epoch 116 out=49b] v112'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308500 con 0x562ee192d000 2024-04-20T21:30:42.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.723+0000 7f1786200640 1 -- 172.21.15.86:0/1122026518 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:91 17.3f 17:fc000000::::head [pgnls-filter start_epoch 116 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x562ee1911400 con 0x562ee2319400 2024-04-20T21:30:42.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.724+0000 7f178b200640 1 -- 172.21.15.86:0/1122026518 <== osd.0 v2:172.21.15.86:6800/2471076035 19 ==== osd_op_reply(91 [pgnls-filter start_epoch 116 out=49b] v112'17 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ee2308000 con 0x562ee2319400 2024-04-20T21:30:42.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.724+0000 7f178e613740 1 -- 172.21.15.86:0/1122026518 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x562ee2319400 msgr2=0x562ee22e8680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.724+0000 7f178e613740 1 --2- 172.21.15.86:0/1122026518 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x562ee2319400 0x562ee22e8680 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.724+0000 7f178e613740 1 -- 172.21.15.86:0/1122026518 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x562ee1911800 msgr2=0x562ee1926000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.724+0000 7f178e613740 1 --2- 172.21.15.86:0/1122026518 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x562ee1911800 0x562ee1926000 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.724+0000 7f178e613740 1 -- 172.21.15.86:0/1122026518 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x562ee192d800 msgr2=0x562ee1926b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.724+0000 7f178e613740 1 --2- 172.21.15.86:0/1122026518 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x562ee192d800 0x562ee1926b00 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.724+0000 7f178e613740 1 -- 172.21.15.86:0/1122026518 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x562ee192c000 msgr2=0x562ee1926580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.724+0000 7f178e613740 1 --2- 172.21.15.86:0/1122026518 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x562ee192c000 0x562ee1926580 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.724+0000 7f178bc00640 1 -- 172.21.15.86:0/1122026518 reap_dead start 2024-04-20T21:30:42.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.724+0000 7f178e613740 1 -- 172.21.15.86:0/1122026518 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x562ee1932800 msgr2=0x562ee1927600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.724+0000 7f178e613740 1 --2- 172.21.15.86:0/1122026518 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x562ee1932800 0x562ee1927600 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.724+0000 7f178e613740 1 -- 172.21.15.86:0/1122026518 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x562ee2318c00 msgr2=0x562ee22e8c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.724+0000 7f178e613740 1 --2- 172.21.15.86:0/1122026518 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x562ee2318c00 0x562ee22e8c00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.724+0000 7f178e613740 1 -- 172.21.15.86:0/1122026518 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x562ee192d000 msgr2=0x562ee1927b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.724+0000 7f178e613740 1 --2- 172.21.15.86:0/1122026518 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x562ee192d000 0x562ee1927b80 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.724+0000 7f178e613740 1 -- 172.21.15.86:0/1122026518 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x562ee1910000 msgr2=0x562ee22e9180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.724+0000 7f178e613740 1 --2- 172.21.15.86:0/1122026518 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x562ee1910000 0x562ee22e9180 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.725+0000 7f178bc00640 1 -- 172.21.15.86:0/1122026518 reap_dead start 2024-04-20T21:30:42.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.725+0000 7f178e613740 1 -- 172.21.15.86:0/1122026518 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x562ee2450c00 msgr2=0x562ee22e9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.725+0000 7f178e613740 1 --2- 172.21.15.86:0/1122026518 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x562ee2450c00 0x562ee22e9700 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x562ee23957d0 tx=0x562ee231a960 comp rx=0 tx=0).stop 2024-04-20T21:30:42.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.725+0000 7f178e613740 1 -- 172.21.15.86:0/1122026518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ee2373800 msgr2=0x562ee22e7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.725+0000 7f178e613740 1 --2- 172.21.15.86:0/1122026518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ee2373800 0x562ee22e7600 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x562ee2443ad0 tx=0x562ee2466c00 comp rx=0 tx=0).stop 2024-04-20T21:30:42.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.725+0000 7f178e613740 1 -- 172.21.15.86:0/1122026518 shutdown_connections 2024-04-20T21:30:42.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.726+0000 7f178e613740 1 --2- 172.21.15.86:0/1122026518 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x562ee2450c00 0x562ee22e9700 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.726+0000 7f178e613740 1 --2- 172.21.15.86:0/1122026518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ee2373800 0x562ee22e7600 unknown :-1 s=CLOSED pgs=2051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.726+0000 7f178e613740 1 -- 172.21.15.86:0/1122026518 >> 172.21.15.86:0/1122026518 conn(0x562ee2373000 msgr2=0x562ee1649000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:42.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.726+0000 7f178e613740 1 -- 172.21.15.86:0/1122026518 shutdown_connections 2024-04-20T21:30:42.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.726+0000 7f178e613740 1 -- 172.21.15.86:0/1122026518 wait complete. 2024-04-20T21:30:42.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.727+0000 7f178e613740 1 -- 172.21.15.86:0/2540827816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ee157bc00 msgr2=0x562ee22e6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.727+0000 7f178e613740 1 --2- 172.21.15.86:0/2540827816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ee157bc00 0x562ee22e6b00 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x562ee2395cb0 tx=0x562ee231e120 comp rx=0 tx=0).stop 2024-04-20T21:30:42.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.728+0000 7f178e613740 1 -- 172.21.15.86:0/2540827816 shutdown_connections 2024-04-20T21:30:42.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.728+0000 7f178e613740 1 --2- 172.21.15.86:0/2540827816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562ee157bc00 0x562ee22e6b00 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.728+0000 7f178e613740 1 --2- 172.21.15.86:0/2540827816 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x562ee157b800 0x562ee22e6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.728+0000 7f178e613740 1 --2- 172.21.15.86:0/2540827816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x562ee157ac00 0x562ee22e6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.728+0000 7f178e613740 1 -- 172.21.15.86:0/2540827816 >> 172.21.15.86:0/2540827816 conn(0x562ee157b000 msgr2=0x562ee1648800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:42.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.728+0000 7f178e613740 1 -- 172.21.15.86:0/2540827816 shutdown_connections 2024-04-20T21:30:42.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.728+0000 7f178e613740 1 -- 172.21.15.86:0/2540827816 wait complete. 2024-04-20T21:30:42.739 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--filter-tag', 'mytag123', 'cephfs_data'] in time 0:00:00.231152, result: 2024-04-20T21:30:42.739 DEBUG:teuthology.orchestra.run.smithi086:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-04-20T21:30:42.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.837+0000 7fea9324e740 1 -- start start 2024-04-20T21:30:42.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.837+0000 7fea9324e740 1 Processor -- start 2024-04-20T21:30:42.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.838+0000 7fea9324e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609d3608c00 0x5609d4370000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.838+0000 7fea9324e740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5609d3609800 0x5609d4370580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.838+0000 7fea9324e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609d3609c00 0x5609d4370b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.838+0000 7fea9324e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5609d4378600 con 0x5609d3609c00 2024-04-20T21:30:42.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.838+0000 7fea9324e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5609d4378480 con 0x5609d3608c00 2024-04-20T21:30:42.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.838+0000 7fea9324e740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5609d4378300 con 0x5609d3609800 2024-04-20T21:30:42.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.839+0000 7fea91600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609d3609c00 0x5609d4370b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.839+0000 7fea91600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609d3609c00 0x5609d4370b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58894/0 (socket says 172.21.15.86:58894) 2024-04-20T21:30:42.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.839+0000 7fea91600640 1 -- 172.21.15.86:0/779686845 learned_addr learned my addr 172.21.15.86:0/779686845 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:42.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.839+0000 7fea90c00640 1 --2- 172.21.15.86:0/779686845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609d3608c00 0x5609d4370000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.839+0000 7fea90200640 1 --2- 172.21.15.86:0/779686845 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5609d3609800 0x5609d4370580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.839+0000 7fea91600640 1 -- 172.21.15.86:0/779686845 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5609d3609800 msgr2=0x5609d4370580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:42.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.839+0000 7fea91600640 1 --2- 172.21.15.86:0/779686845 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5609d3609800 0x5609d4370580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.839+0000 7fea91600640 1 -- 172.21.15.86:0/779686845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609d3608c00 msgr2=0x5609d4370000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:42.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.839+0000 7fea91600640 1 --2- 172.21.15.86:0/779686845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609d3608c00 0x5609d4370000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.839+0000 7fea91600640 1 -- 172.21.15.86:0/779686845 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609d3635680 con 0x5609d3609c00 2024-04-20T21:30:42.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.839+0000 7fea90200640 1 --2- 172.21.15.86:0/779686845 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5609d3609800 0x5609d4370580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:30:42.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.840+0000 7fea91600640 1 --2- 172.21.15.86:0/779686845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609d3609c00 0x5609d4370b00 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x5609d442b4a0 tx=0x5609d43acf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2065304f24f9085d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:42.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.840+0000 7fea8da00640 1 -- 172.21.15.86:0/779686845 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5609d362e680 con 0x5609d3609c00 2024-04-20T21:30:42.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.840+0000 7fea8da00640 1 -- 172.21.15.86:0/779686845 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5609d3652c40 con 0x5609d3609c00 2024-04-20T21:30:42.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.840+0000 7fea8da00640 1 -- 172.21.15.86:0/779686845 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5609d362eb60 con 0x5609d3609c00 2024-04-20T21:30:42.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.840+0000 7fea9324e740 1 -- 172.21.15.86:0/779686845 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609d36354a0 con 0x5609d3609c00 2024-04-20T21:30:42.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.840+0000 7fea9324e740 1 -- 172.21.15.86:0/779686845 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5609d3635a40 con 0x5609d3609c00 2024-04-20T21:30:42.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.840+0000 7fea9324e740 4 waiting for MDS map... 2024-04-20T21:30:42.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.840+0000 7fea8da00640 1 -- 172.21.15.86:0/779686845 <== mon.0 v2:172.21.15.86:3300/0 4 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x5609d44e0600 con 0x5609d3609c00 2024-04-20T21:30:42.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.841+0000 7fea8da00640 1 -- 172.21.15.86:0/779686845 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 106) ==== 2886+0+0 (secure 0 0 0) 0x5609d3661180 con 0x5609d3609c00 2024-04-20T21:30:42.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.841+0000 7fea9324e740 4 Got MDS map 106 2024-04-20T21:30:42.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.841+0000 7fea9324e740 4 datascan.main: Using metadata pool output 2024-04-20T21:30:42.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.841+0000 7fea9324e740 4 datascan.main: connecting to RADOS... 2024-04-20T21:30:42.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.841+0000 7fea9324e740 1 Processor -- start 2024-04-20T21:30:42.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.842+0000 7fea9324e740 1 -- start start 2024-04-20T21:30:42.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.842+0000 7fea9324e740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5609d44e3400 0x5609d4371080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.842+0000 7fea9324e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609d44e3800 0x5609d4371600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.842+0000 7fea9324e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609d44e3c00 0x5609d4371b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.842+0000 7fea9324e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5609d4378180 con 0x5609d44e3c00 2024-04-20T21:30:42.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.842+0000 7fea9324e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5609d3668480 con 0x5609d44e3800 2024-04-20T21:30:42.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.842+0000 7fea9324e740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5609d3668900 con 0x5609d44e3400 2024-04-20T21:30:42.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.843+0000 7fea90200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609d44e3c00 0x5609d4371b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.843+0000 7fea90200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609d44e3c00 0x5609d4371b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58898/0 (socket says 172.21.15.86:58898) 2024-04-20T21:30:42.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.843+0000 7fea90200640 1 -- 172.21.15.86:0/1942580950 learned_addr learned my addr 172.21.15.86:0/1942580950 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:42.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.843+0000 7fea90c00640 1 --2- 172.21.15.86:0/1942580950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609d44e3800 0x5609d4371600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.843+0000 7fea90200640 1 --2- 172.21.15.86:0/1942580950 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5609d44e3400 0x5609d4371080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.843+0000 7fea90200640 1 -- 172.21.15.86:0/1942580950 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5609d44e3400 msgr2=0x5609d4371080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:42.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.843+0000 7fea90200640 1 --2- 172.21.15.86:0/1942580950 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5609d44e3400 0x5609d4371080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.843+0000 7fea90200640 1 -- 172.21.15.86:0/1942580950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609d44e3800 msgr2=0x5609d4371600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:42.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.843+0000 7fea90200640 1 --2- 172.21.15.86:0/1942580950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609d44e3800 0x5609d4371600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.843+0000 7fea90200640 1 -- 172.21.15.86:0/1942580950 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609d3635860 con 0x5609d44e3c00 2024-04-20T21:30:42.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.844+0000 7fea90200640 1 --2- 172.21.15.86:0/1942580950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609d44e3c00 0x5609d4371b80 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x5609d44f6930 tx=0x5609d44fccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b65323f53f71cdad server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:42.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.844+0000 7fea8b200640 1 -- 172.21.15.86:0/1942580950 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5609d362e680 con 0x5609d44e3c00 2024-04-20T21:30:42.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.844+0000 7fea8b200640 1 -- 172.21.15.86:0/1942580950 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5609d36521c0 con 0x5609d44e3c00 2024-04-20T21:30:42.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.844+0000 7fea8b200640 1 -- 172.21.15.86:0/1942580950 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5609d362eb60 con 0x5609d44e3c00 2024-04-20T21:30:42.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.844+0000 7fea9324e740 1 -- 172.21.15.86:0/1942580950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609d44e3c00 msgr2=0x5609d4371b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.844+0000 7fea9324e740 1 --2- 172.21.15.86:0/1942580950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609d44e3c00 0x5609d4371b80 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x5609d44f6930 tx=0x5609d44fccc0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.844+0000 7fea9324e740 1 -- 172.21.15.86:0/1942580950 shutdown_connections 2024-04-20T21:30:42.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.844+0000 7fea9324e740 1 --2- 172.21.15.86:0/1942580950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609d44e3c00 0x5609d4371b80 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.844+0000 7fea9324e740 1 --2- 172.21.15.86:0/1942580950 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609d44e3800 0x5609d4371600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.844+0000 7fea9324e740 1 --2- 172.21.15.86:0/1942580950 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5609d44e3400 0x5609d4371080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.844+0000 7fea9324e740 1 -- 172.21.15.86:0/1942580950 >> 172.21.15.86:0/1942580950 conn(0x5609d44e3000 msgr2=0x5609d36f3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:42.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.844+0000 7fea9324e740 1 -- 172.21.15.86:0/1942580950 shutdown_connections 2024-04-20T21:30:42.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.844+0000 7fea9324e740 1 -- 172.21.15.86:0/1942580950 wait complete. 2024-04-20T21:30:42.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.845+0000 7fea9324e740 1 Processor -- start 2024-04-20T21:30:42.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.845+0000 7fea9324e740 1 -- start start 2024-04-20T21:30:42.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.846+0000 7fea9324e740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5609d44e3400 0x5609d4371080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.846+0000 7fea9324e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609d44e3800 0x5609d4371600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.846+0000 7fea9324e740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609d44e3c00 0x5609d4371b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.846+0000 7fea9324e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5609d4379380 con 0x5609d44e3800 2024-04-20T21:30:42.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.846+0000 7fea9324e740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x5609d4378180 con 0x5609d44e3c00 2024-04-20T21:30:42.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.846+0000 7fea9324e740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x5609d4379500 con 0x5609d44e3400 2024-04-20T21:30:42.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.846+0000 7fea90c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609d44e3800 0x5609d4371600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.846+0000 7fea90c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609d44e3800 0x5609d4371600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58904/0 (socket says 172.21.15.86:58904) 2024-04-20T21:30:42.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.846+0000 7fea90c00640 1 -- 172.21.15.86:0/2846374639 learned_addr learned my addr 172.21.15.86:0/2846374639 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:42.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.847+0000 7fea90200640 1 --2- 172.21.15.86:0/2846374639 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5609d44e3400 0x5609d4371080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.847+0000 7fea90c00640 1 -- 172.21.15.86:0/2846374639 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5609d44e3400 msgr2=0x5609d4371080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.847+0000 7fea90200640 1 --2- 172.21.15.86:0/2846374639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609d44e3c00 0x5609d4371b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.847+0000 7fea90c00640 1 --2- 172.21.15.86:0/2846374639 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5609d44e3400 0x5609d4371080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.847+0000 7fea90c00640 1 -- 172.21.15.86:0/2846374639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609d44e3c00 msgr2=0x5609d4371b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.847+0000 7fea90c00640 1 --2- 172.21.15.86:0/2846374639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609d44e3c00 0x5609d4371b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.847+0000 7fea90c00640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609d3635680 con 0x5609d44e3800 2024-04-20T21:30:42.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.847+0000 7fea90200640 1 --2- 172.21.15.86:0/2846374639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609d44e3c00 0x5609d4371b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:30:42.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.847+0000 7fea90c00640 1 --2- 172.21.15.86:0/2846374639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609d44e3800 0x5609d4371600 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x5609d44f7bf0 tx=0x5609d43acd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:42.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.847+0000 7fea89400640 1 -- 172.21.15.86:0/2846374639 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5609d362e9c0 con 0x5609d44e3800 2024-04-20T21:30:42.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.847+0000 7fea89400640 1 -- 172.21.15.86:0/2846374639 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5609d3653340 con 0x5609d44e3800 2024-04-20T21:30:42.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.847+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5609d36354a0 con 0x5609d44e3800 2024-04-20T21:30:42.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.847+0000 7fea89400640 1 -- 172.21.15.86:0/2846374639 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5609d362e680 con 0x5609d44e3800 2024-04-20T21:30:42.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.848+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609d3635a40 con 0x5609d44e3800 2024-04-20T21:30:42.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.848+0000 7fea9324e740 4 datascan.init: resolving metadata pool 16 2024-04-20T21:30:42.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.849+0000 7fea89400640 1 -- 172.21.15.86:0/2846374639 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x5609d4512c00 con 0x5609d44e3800 2024-04-20T21:30:42.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.850+0000 7fea89400640 1 --2- 172.21.15.86:0/2846374639 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x5609d455ec00 0x5609d4372100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.850+0000 7fea89400640 1 -- 172.21.15.86:0/2846374639 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x5609d44d0a00 con 0x5609d44e3800 2024-04-20T21:30:42.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.850+0000 7fea9324e740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-20T21:30:42.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.850+0000 7fea9324e740 4 datascan.main: resolving metadata pool 16 2024-04-20T21:30:42.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.850+0000 7fea9324e740 1 --2- 172.21.15.86:0/2846374639 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x5609d455f000 0x5609d4372680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.850+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:1 16.0 16:00000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d455f400 con 0x5609d455f000 2024-04-20T21:30:42.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.850+0000 7fea90200640 1 --2- 172.21.15.86:0/2846374639 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x5609d455ec00 0x5609d4372100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.851+0000 7fea90200640 1 --2- 172.21.15.86:0/2846374639 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x5609d455f000 0x5609d4372680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.851+0000 7fea90200640 1 --2- 172.21.15.86:0/2846374639 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x5609d455ec00 0x5609d4372100 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x5609d44f6a80 tx=0x5609d44fd140 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:42.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.851+0000 7fea90200640 1 --2- 172.21.15.86:0/2846374639 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x5609d455f000 0x5609d4372680 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:42.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.852+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 <== osd.0 v2:172.21.15.86:6800/2471076035 1 ==== osd_op_reply(1 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5609d4396000 con 0x5609d455f000 2024-04-20T21:30:42.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.852+0000 7fea90200640 1 --2- 172.21.15.86:0/2846374639 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x5609d455f400 0x5609d3994000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.852+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:2 16.10 16:08000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d455e800 con 0x5609d455f400 2024-04-20T21:30:42.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.852+0000 7fea91600640 1 --2- 172.21.15.86:0/2846374639 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x5609d455f400 0x5609d3994000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.853+0000 7fea91600640 1 --2- 172.21.15.86:0/2846374639 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x5609d455f400 0x5609d3994000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:42.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.853+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 <== osd.2 v2:172.21.15.86:6816/3437611468 1 ==== osd_op_reply(2 [pgnls start_epoch 116 out=177b] v115'16 uv16 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x5609d4396500 con 0x5609d455f400 2024-04-20T21:30:42.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.853+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:3 16.8 16:10000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d455e800 con 0x5609d455f000 2024-04-20T21:30:42.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.853+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 <== osd.0 v2:172.21.15.86:6800/2471076035 2 ==== osd_op_reply(3 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5609d4396000 con 0x5609d455f000 2024-04-20T21:30:42.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.854+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:4 16.18 16:18000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d455e800 con 0x5609d455f000 2024-04-20T21:30:42.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.854+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 <== osd.0 v2:172.21.15.86:6800/2471076035 3 ==== osd_op_reply(4 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5609d4396000 con 0x5609d455f000 2024-04-20T21:30:42.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.854+0000 7fea90200640 1 --2- 172.21.15.86:0/2846374639 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x5609d455e800 0x5609d3994580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.854+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:5 16.4 16:20000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d39a4000 con 0x5609d455e800 2024-04-20T21:30:42.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.854+0000 7fea90c00640 1 --2- 172.21.15.86:0/2846374639 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x5609d455e800 0x5609d3994580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.855+0000 7fea90c00640 1 --2- 172.21.15.86:0/2846374639 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x5609d455e800 0x5609d3994580 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:42.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.855+0000 7fea90c00640 1 -- 172.21.15.86:0/2846374639 <== osd.3 v2:172.21.15.86:6808/794135519 1 ==== osd_op_reply(5 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5609d4396780 con 0x5609d455e800 2024-04-20T21:30:42.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.855+0000 7fea90c00640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:6 16.14 16:28000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d39a4000 con 0x5609d455f000 2024-04-20T21:30:42.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.856+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 <== osd.0 v2:172.21.15.86:6800/2471076035 4 ==== osd_op_reply(6 [pgnls start_epoch 116 out=149b] v116'44 uv44 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x5609d4396000 con 0x5609d455f000 2024-04-20T21:30:42.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.856+0000 7fea90200640 1 --2- 172.21.15.86:0/2846374639 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x5609d39a4000 0x5609d3994b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.856+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:7 16.c 16:30000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d39a4400 con 0x5609d39a4000 2024-04-20T21:30:42.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.856+0000 7fea90200640 1 --2- 172.21.15.86:0/2846374639 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x5609d39a4000 0x5609d3994b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.858+0000 7fea90200640 1 --2- 172.21.15.86:0/2846374639 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x5609d39a4000 0x5609d3994b00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:42.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.858+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 <== osd.5 v2:172.21.15.144:6808/1635974704 1 ==== osd_op_reply(7 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5609d4396000 con 0x5609d39a4000 2024-04-20T21:30:42.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.858+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:8 16.1c 16:38000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d39a4400 con 0x5609d455f400 2024-04-20T21:30:42.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.859+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 <== osd.2 v2:172.21.15.86:6816/3437611468 2 ==== osd_op_reply(8 [pgnls start_epoch 116 out=73b] v115'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5609d4396500 con 0x5609d455f400 2024-04-20T21:30:42.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.859+0000 7fea91600640 1 --2- 172.21.15.86:0/2846374639 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x5609d39a4400 0x5609d3995080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.859+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:9 16.2 16:40000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d44e2800 con 0x5609d39a4400 2024-04-20T21:30:42.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.860+0000 7fea91600640 1 --2- 172.21.15.86:0/2846374639 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x5609d39a4400 0x5609d3995080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.861+0000 7fea91600640 1 --2- 172.21.15.86:0/2846374639 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x5609d39a4400 0x5609d3995080 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:42.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.862+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 <== osd.4 v2:172.21.15.144:6800/4087309475 1 ==== osd_op_reply(9 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5609d4396500 con 0x5609d39a4400 2024-04-20T21:30:42.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.862+0000 7fea91600640 1 --2- 172.21.15.86:0/2846374639 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x5609d44e2800 0x5609d3995600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.862+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:10 16.12 16:48000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d39a4c00 con 0x5609d44e2800 2024-04-20T21:30:42.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.863+0000 7fea90c00640 1 --2- 172.21.15.86:0/2846374639 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x5609d44e2800 0x5609d3995600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.864+0000 7fea90c00640 1 --2- 172.21.15.86:0/2846374639 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x5609d44e2800 0x5609d3995600 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:42.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.865+0000 7fea90c00640 1 -- 172.21.15.86:0/2846374639 <== osd.6 v2:172.21.15.144:6824/1893803079 1 ==== osd_op_reply(10 [pgnls start_epoch 116 out=81b] v115'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5609d4396780 con 0x5609d44e2800 2024-04-20T21:30:42.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.865+0000 7fea90c00640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:11 16.a 16:50000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d39a4c00 con 0x5609d455f400 2024-04-20T21:30:42.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.865+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 <== osd.2 v2:172.21.15.86:6816/3437611468 3 ==== osd_op_reply(11 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5609d4396500 con 0x5609d455f400 2024-04-20T21:30:42.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.865+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:12 16.1a 16:58000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d39a4c00 con 0x5609d455e800 2024-04-20T21:30:42.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.866+0000 7fea90c00640 1 -- 172.21.15.86:0/2846374639 <== osd.3 v2:172.21.15.86:6808/794135519 2 ==== osd_op_reply(12 [pgnls start_epoch 116 out=49b] v116'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5609d4396780 con 0x5609d455e800 2024-04-20T21:30:42.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.866+0000 7fea90c00640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:13 16.6 16:60000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d39a4c00 con 0x5609d44e2800 2024-04-20T21:30:42.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.867+0000 7fea90c00640 1 -- 172.21.15.86:0/2846374639 <== osd.6 v2:172.21.15.144:6824/1893803079 2 ==== osd_op_reply(13 [pgnls start_epoch 116 out=77b] v115'10 uv10 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x5609d4396780 con 0x5609d44e2800 2024-04-20T21:30:42.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.867+0000 7fea90c00640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:14 16.16 16:68000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d39a4c00 con 0x5609d455e800 2024-04-20T21:30:42.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.867+0000 7fea90c00640 1 -- 172.21.15.86:0/2846374639 <== osd.3 v2:172.21.15.86:6808/794135519 3 ==== osd_op_reply(14 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5609d4396780 con 0x5609d455e800 2024-04-20T21:30:42.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.867+0000 7fea90c00640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:15 16.e 16:70000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d39a4c00 con 0x5609d455f400 2024-04-20T21:30:42.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.868+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 <== osd.2 v2:172.21.15.86:6816/3437611468 4 ==== osd_op_reply(15 [pgnls start_epoch 116 out=73b] v115'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5609d4396500 con 0x5609d455f400 2024-04-20T21:30:42.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.868+0000 7fea91600640 1 --2- 172.21.15.86:0/2846374639 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x5609d39a4c00 0x5609d3995b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.868+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:16 16.1e 16:78000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d39a5000 con 0x5609d39a4c00 2024-04-20T21:30:42.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.869+0000 7fea90200640 1 --2- 172.21.15.86:0/2846374639 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x5609d39a4c00 0x5609d3995b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.870+0000 7fea90200640 1 --2- 172.21.15.86:0/2846374639 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x5609d39a4c00 0x5609d3995b80 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:42.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.871+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 <== osd.7 v2:172.21.15.144:6816/2100967057 1 ==== osd_op_reply(16 [pgnls start_epoch 116 out=49b] v116'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5609d4396000 con 0x5609d39a4c00 2024-04-20T21:30:42.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.871+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:17 16.1 16:80000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d39a5000 con 0x5609d455f400 2024-04-20T21:30:42.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.871+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 <== osd.2 v2:172.21.15.86:6816/3437611468 5 ==== osd_op_reply(17 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5609d4396500 con 0x5609d455f400 2024-04-20T21:30:42.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.871+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:18 16.11 16:88000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d39a5000 con 0x5609d39a4000 2024-04-20T21:30:42.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.872+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 <== osd.5 v2:172.21.15.144:6808/1635974704 2 ==== osd_op_reply(18 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5609d4396000 con 0x5609d39a4000 2024-04-20T21:30:42.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.873+0000 7fea90200640 1 --2- 172.21.15.86:0/2846374639 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x5609d39a5000 0x5609d3996100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:42.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.873+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:19 16.9 16:90000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d39a4800 con 0x5609d39a5000 2024-04-20T21:30:42.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.873+0000 7fea91600640 1 --2- 172.21.15.86:0/2846374639 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x5609d39a5000 0x5609d3996100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:42.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.874+0000 7fea91600640 1 --2- 172.21.15.86:0/2846374639 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x5609d39a5000 0x5609d3996100 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:42.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.874+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 <== osd.1 v2:172.21.15.86:6817/3149564961 1 ==== osd_op_reply(19 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5609d4396500 con 0x5609d39a5000 2024-04-20T21:30:42.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.874+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:20 16.19 16:98000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d39a4800 con 0x5609d39a4400 2024-04-20T21:30:42.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.875+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 <== osd.4 v2:172.21.15.144:6800/4087309475 2 ==== osd_op_reply(20 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5609d4396500 con 0x5609d39a4400 2024-04-20T21:30:42.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.875+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:21 16.5 16:a0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d39a4800 con 0x5609d455f000 2024-04-20T21:30:42.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.876+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 <== osd.0 v2:172.21.15.86:6800/2471076035 5 ==== osd_op_reply(21 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5609d4396000 con 0x5609d455f000 2024-04-20T21:30:42.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.876+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:22 16.15 16:a8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d39a4800 con 0x5609d455f000 2024-04-20T21:30:42.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.876+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 <== osd.0 v2:172.21.15.86:6800/2471076035 6 ==== osd_op_reply(22 [pgnls start_epoch 116 out=97b] v115'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x5609d4396000 con 0x5609d455f000 2024-04-20T21:30:42.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.877+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:23 16.d 16:b0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d39a4800 con 0x5609d44e2800 2024-04-20T21:30:42.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.878+0000 7fea90c00640 1 -- 172.21.15.86:0/2846374639 <== osd.6 v2:172.21.15.144:6824/1893803079 3 ==== osd_op_reply(23 [pgnls start_epoch 116 out=74b] v111'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x5609d4396780 con 0x5609d44e2800 2024-04-20T21:30:42.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.878+0000 7fea90c00640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:24 16.1d 16:b8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d39a4800 con 0x5609d455f400 2024-04-20T21:30:42.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.878+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 <== osd.2 v2:172.21.15.86:6816/3437611468 6 ==== osd_op_reply(24 [pgnls start_epoch 116 out=73b] v115'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5609d4396500 con 0x5609d455f400 2024-04-20T21:30:42.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.878+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:25 16.3 16:c0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d39a4800 con 0x5609d39a4000 2024-04-20T21:30:42.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.879+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 <== osd.5 v2:172.21.15.144:6808/1635974704 3 ==== osd_op_reply(25 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5609d4396000 con 0x5609d39a4000 2024-04-20T21:30:42.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.879+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:26 16.13 16:c8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d39a4800 con 0x5609d39a5000 2024-04-20T21:30:42.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.880+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 <== osd.1 v2:172.21.15.86:6817/3149564961 2 ==== osd_op_reply(26 [pgnls start_epoch 116 out=98b] v115'5 uv5 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x5609d4396500 con 0x5609d39a5000 2024-04-20T21:30:42.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.880+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:27 16.b 16:d0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d39a4800 con 0x5609d39a5000 2024-04-20T21:30:42.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.880+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 <== osd.1 v2:172.21.15.86:6817/3149564961 3 ==== osd_op_reply(27 [pgnls start_epoch 116 out=76b] v116'6 uv5 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x5609d4396500 con 0x5609d39a5000 2024-04-20T21:30:42.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.880+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:28 16.1b 16:d8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d39a4800 con 0x5609d39a4000 2024-04-20T21:30:42.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.881+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 <== osd.5 v2:172.21.15.144:6808/1635974704 4 ==== osd_op_reply(28 [pgnls start_epoch 116 out=49b] v116'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5609d4396000 con 0x5609d39a4000 2024-04-20T21:30:42.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.881+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:29 16.7 16:e0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d39a4800 con 0x5609d455e800 2024-04-20T21:30:42.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.882+0000 7fea90c00640 1 -- 172.21.15.86:0/2846374639 <== osd.3 v2:172.21.15.86:6808/794135519 4 ==== osd_op_reply(29 [pgnls start_epoch 116 out=105b] v116'9 uv9 ondisk = 1) ==== 144+0+105 (crc 0 0 0) 0x5609d4396780 con 0x5609d455e800 2024-04-20T21:30:42.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.882+0000 7fea90c00640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:30 16.17 16:e8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d39a4800 con 0x5609d455f400 2024-04-20T21:30:42.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.883+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 <== osd.2 v2:172.21.15.86:6816/3437611468 7 ==== osd_op_reply(30 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5609d4396500 con 0x5609d455f400 2024-04-20T21:30:42.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.883+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:31 16.f 16:f0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d39a4800 con 0x5609d455f000 2024-04-20T21:30:42.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.883+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 <== osd.0 v2:172.21.15.86:6800/2471076035 7 ==== osd_op_reply(31 [pgnls start_epoch 116 out=73b] v116'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5609d4396000 con 0x5609d455f000 2024-04-20T21:30:42.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.883+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:32 16.1f 16:f8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d39a4800 con 0x5609d39a4000 2024-04-20T21:30:42.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.885+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 <== osd.5 v2:172.21.15.144:6808/1635974704 5 ==== osd_op_reply(32 [pgnls start_epoch 116 out=101b] v116'9 uv8 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x5609d4396000 con 0x5609d39a4000 2024-04-20T21:30:42.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.885+0000 7fea9324e740 10 datascan.scan_links: step 1: handling object 601.00000000 2024-04-20T21:30:42.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.885+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:33 16.10 16:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x5609d455f800 con 0x5609d455f400 2024-04-20T21:30:42.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.885+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 <== osd.2 v2:172.21.15.86:6816/3437611468 8 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5609d4396500 con 0x5609d455f400 2024-04-20T21:30:42.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.885+0000 7fea9324e740 10 datascan.scan_links: step 1: handling object 602.00000000 2024-04-20T21:30:42.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.885+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:34 16.10 16:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x5609d455fc00 con 0x5609d455f400 2024-04-20T21:30:42.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.886+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 <== osd.2 v2:172.21.15.86:6816/3437611468 9 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5609d4396500 con 0x5609d455f400 2024-04-20T21:30:42.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.886+0000 7fea9324e740 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-04-20T21:30:42.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.886+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:35 16.10 16:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x5609d39a5c00 con 0x5609d455f400 2024-04-20T21:30:42.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.886+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 <== osd.2 v2:172.21.15.86:6816/3437611468 10 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=1047b] v0'0 uv12 ondisk = 0) ==== 164+0+1047 (crc 0 0 0) 0x5609d4396500 con 0x5609d455f400 2024-04-20T21:30:42.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.886+0000 7fea9324e740 10 datascan.scan_links: step 1: handling object 600.00000000 2024-04-20T21:30:42.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.886+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:36 16.10 16:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x5609d39c2000 con 0x5609d455f400 2024-04-20T21:30:42.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.887+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 <== osd.2 v2:172.21.15.86:6816/3437611468 11 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5609d4396500 con 0x5609d455f400 2024-04-20T21:30:42.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.887+0000 7fea9324e740 10 datascan.scan_links: step 1: handling object 603.00000000 2024-04-20T21:30:42.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.887+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:37 16.10 16:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x5609d39c2400 con 0x5609d455f400 2024-04-20T21:30:42.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.887+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 <== osd.2 v2:172.21.15.86:6816/3437611468 12 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5609d4396500 con 0x5609d455f400 2024-04-20T21:30:42.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.887+0000 7fea9324e740 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-04-20T21:30:42.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.887+0000 7fea9324e740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-20T21:30:42.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.887+0000 7fea9324e740 10 datascan.scan_links: step 1: handling object 200.00000000 2024-04-20T21:30:42.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.887+0000 7fea9324e740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-20T21:30:42.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.887+0000 7fea9324e740 10 datascan.scan_links: step 1: handling object 200.00000001 2024-04-20T21:30:42.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.887+0000 7fea9324e740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-20T21:30:42.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.887+0000 7fea9324e740 10 datascan.scan_links: step 1: handling object 606.00000000 2024-04-20T21:30:42.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.887+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:38 16.14 16:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x5609d39c2800 con 0x5609d455f000 2024-04-20T21:30:42.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.888+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 <== osd.0 v2:172.21.15.86:6800/2471076035 8 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv39 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5609d4396000 con 0x5609d455f000 2024-04-20T21:30:42.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.888+0000 7fea9324e740 10 datascan.scan_links: step 1: handling object 607.00000000 2024-04-20T21:30:42.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.888+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:39 16.1c 16:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x5609d39c2c00 con 0x5609d455f400 2024-04-20T21:30:42.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.889+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 <== osd.2 v2:172.21.15.86:6816/3437611468 13 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5609d4396500 con 0x5609d455f400 2024-04-20T21:30:42.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.889+0000 7fea9324e740 10 datascan.scan_links: step 1: handling object 10000000003.00000000 2024-04-20T21:30:42.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.889+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:40 16.12 16:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x5609d39c3000 con 0x5609d44e2800 2024-04-20T21:30:42.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.890+0000 7fea90c00640 1 -- 172.21.15.86:0/2846374639 <== osd.6 v2:172.21.15.144:6824/1893803079 4 ==== osd_op_reply(40 10000000003.00000000 [omap-get-vals out=523b] v0'0 uv3 ondisk = 0) ==== 164+0+523 (crc 0 0 0) 0x5609d4396780 con 0x5609d44e2800 2024-04-20T21:30:42.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.890+0000 7fea9324e740 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-04-20T21:30:42.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.890+0000 7fea9324e740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-20T21:30:42.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.890+0000 7fea9324e740 10 datascan.scan_links: step 1: handling object 608.00000000 2024-04-20T21:30:42.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.890+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:41 16.e 16:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x5609d39c3400 con 0x5609d455f400 2024-04-20T21:30:42.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.891+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 <== osd.2 v2:172.21.15.86:6816/3437611468 14 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5609d4396500 con 0x5609d455f400 2024-04-20T21:30:42.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.891+0000 7fea9324e740 10 datascan.scan_links: step 1: handling object 604.00000000 2024-04-20T21:30:42.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.891+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:42 16.15 16:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x5609d39c3800 con 0x5609d455f000 2024-04-20T21:30:42.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.891+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 <== osd.0 v2:172.21.15.86:6800/2471076035 9 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5609d4396000 con 0x5609d455f000 2024-04-20T21:30:42.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.891+0000 7fea9324e740 10 datascan.scan_links: step 1: handling object 500.00000000 2024-04-20T21:30:42.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.891+0000 7fea9324e740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-20T21:30:42.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.891+0000 7fea9324e740 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-04-20T21:30:42.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.891+0000 7fea9324e740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-20T21:30:42.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.891+0000 7fea9324e740 10 datascan.scan_links: step 1: handling object 605.00000000 2024-04-20T21:30:42.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.892+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:43 16.1d 16:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x5609d39c2000 con 0x5609d455f400 2024-04-20T21:30:42.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.892+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 <== osd.2 v2:172.21.15.86:6816/3437611468 15 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5609d4396500 con 0x5609d455f400 2024-04-20T21:30:42.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.892+0000 7fea9324e740 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-04-20T21:30:42.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.892+0000 7fea9324e740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-20T21:30:42.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.892+0000 7fea9324e740 10 datascan.scan_links: step 1: handling object 100.00000000 2024-04-20T21:30:42.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.892+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:44 16.13 16:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x5609d39c3c00 con 0x5609d39a5000 2024-04-20T21:30:42.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.893+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 <== osd.1 v2:172.21.15.86:6817/3149564961 4 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x5609d4396500 con 0x5609d39a5000 2024-04-20T21:30:42.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.893+0000 7fea9324e740 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-04-20T21:30:42.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.893+0000 7fea9324e740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-20T21:30:42.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.893+0000 7fea9324e740 10 datascan.scan_links: step 1: handling object 609.00000000 2024-04-20T21:30:42.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.893+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:45 16.7 16:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x5609d39a5800 con 0x5609d455e800 2024-04-20T21:30:42.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.894+0000 7fea90c00640 1 -- 172.21.15.86:0/2846374639 <== osd.3 v2:172.21.15.86:6808/794135519 5 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv8 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5609d4396780 con 0x5609d455e800 2024-04-20T21:30:42.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.894+0000 7fea9324e740 10 datascan.scan_links: step 1: handling object 10000000001.00000000 2024-04-20T21:30:42.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.894+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:46 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x5609d39a5400 con 0x5609d455e800 2024-04-20T21:30:42.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.895+0000 7fea90c00640 1 -- 172.21.15.86:0/2846374639 <== osd.3 v2:172.21.15.86:6808/794135519 6 ==== osd_op_reply(46 10000000001.00000000 [omap-get-vals out=1545b] v0'0 uv9 ondisk = 0) ==== 164+0+1545 (crc 0 0 0) 0x5609d4396780 con 0x5609d455e800 2024-04-20T21:30:42.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.895+0000 7fea9324e740 20 datascan.scan_links: injected ino detected 2024-04-20T21:30:42.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.895+0000 7fea9324e740 10 datascan.scan_links: step 1: handling object 400.00000000 2024-04-20T21:30:42.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.895+0000 7fea9324e740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-20T21:30:42.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.895+0000 7fea9324e740 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-04-20T21:30:42.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.895+0000 7fea9324e740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-20T21:30:42.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.895+0000 7fea9324e740 10 datascan.scan_links: step 1: handling object 1.00000000 2024-04-20T21:30:42.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.895+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:47 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x5609d39a5c00 con 0x5609d39a4000 2024-04-20T21:30:42.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.896+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 <== osd.5 v2:172.21.15.144:6808/1635974704 6 ==== osd_op_reply(47 1.00000000 [omap-get-vals out=524b] v0'0 uv8 ondisk = 0) ==== 154+0+524 (crc 0 0 0) 0x5609d4396000 con 0x5609d39a4000 2024-04-20T21:30:42.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.896+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:48 16.0 16:00000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d455f800 con 0x5609d455f000 2024-04-20T21:30:42.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.897+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 <== osd.0 v2:172.21.15.86:6800/2471076035 10 ==== osd_op_reply(48 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5609d4396000 con 0x5609d455f000 2024-04-20T21:30:42.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.897+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:49 16.10 16:08000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d455f800 con 0x5609d455f400 2024-04-20T21:30:42.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.898+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 <== osd.2 v2:172.21.15.86:6816/3437611468 16 ==== osd_op_reply(49 [pgnls start_epoch 116 out=177b] v115'16 uv16 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x5609d4396500 con 0x5609d455f400 2024-04-20T21:30:42.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.898+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:50 16.8 16:10000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d455f800 con 0x5609d455f000 2024-04-20T21:30:42.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.899+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 <== osd.0 v2:172.21.15.86:6800/2471076035 11 ==== osd_op_reply(50 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5609d4396000 con 0x5609d455f000 2024-04-20T21:30:42.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.899+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:51 16.18 16:18000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d455f800 con 0x5609d455f000 2024-04-20T21:30:42.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.899+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 <== osd.0 v2:172.21.15.86:6800/2471076035 12 ==== osd_op_reply(51 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5609d4396000 con 0x5609d455f000 2024-04-20T21:30:42.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.899+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:52 16.4 16:20000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d455f800 con 0x5609d455e800 2024-04-20T21:30:42.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.900+0000 7fea90c00640 1 -- 172.21.15.86:0/2846374639 <== osd.3 v2:172.21.15.86:6808/794135519 7 ==== osd_op_reply(52 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5609d4396780 con 0x5609d455e800 2024-04-20T21:30:42.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.900+0000 7fea90c00640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:53 16.14 16:28000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d455f800 con 0x5609d455f000 2024-04-20T21:30:42.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.901+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 <== osd.0 v2:172.21.15.86:6800/2471076035 13 ==== osd_op_reply(53 [pgnls start_epoch 116 out=149b] v116'44 uv44 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x5609d4396000 con 0x5609d455f000 2024-04-20T21:30:42.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.901+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:54 16.c 16:30000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d455f800 con 0x5609d39a4000 2024-04-20T21:30:42.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.902+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 <== osd.5 v2:172.21.15.144:6808/1635974704 7 ==== osd_op_reply(54 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5609d4396000 con 0x5609d39a4000 2024-04-20T21:30:42.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.902+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:55 16.1c 16:38000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d455f800 con 0x5609d455f400 2024-04-20T21:30:42.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.903+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 <== osd.2 v2:172.21.15.86:6816/3437611468 17 ==== osd_op_reply(55 [pgnls start_epoch 116 out=73b] v115'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5609d4396500 con 0x5609d455f400 2024-04-20T21:30:42.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.903+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:56 16.2 16:40000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d455f800 con 0x5609d39a4400 2024-04-20T21:30:42.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.904+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 <== osd.4 v2:172.21.15.144:6800/4087309475 3 ==== osd_op_reply(56 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5609d4396500 con 0x5609d39a4400 2024-04-20T21:30:42.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.904+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:57 16.12 16:48000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d455f800 con 0x5609d44e2800 2024-04-20T21:30:42.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.905+0000 7fea90c00640 1 -- 172.21.15.86:0/2846374639 <== osd.6 v2:172.21.15.144:6824/1893803079 5 ==== osd_op_reply(57 [pgnls start_epoch 116 out=81b] v115'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5609d4396780 con 0x5609d44e2800 2024-04-20T21:30:42.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.905+0000 7fea90c00640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:58 16.a 16:50000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d455f800 con 0x5609d455f400 2024-04-20T21:30:42.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.905+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 <== osd.2 v2:172.21.15.86:6816/3437611468 18 ==== osd_op_reply(58 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5609d4396500 con 0x5609d455f400 2024-04-20T21:30:42.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.905+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:59 16.1a 16:58000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d455f800 con 0x5609d455e800 2024-04-20T21:30:42.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.906+0000 7fea90c00640 1 -- 172.21.15.86:0/2846374639 <== osd.3 v2:172.21.15.86:6808/794135519 8 ==== osd_op_reply(59 [pgnls start_epoch 116 out=49b] v116'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5609d4396780 con 0x5609d455e800 2024-04-20T21:30:42.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.906+0000 7fea90c00640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:60 16.6 16:60000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d455f800 con 0x5609d44e2800 2024-04-20T21:30:42.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.907+0000 7fea90c00640 1 -- 172.21.15.86:0/2846374639 <== osd.6 v2:172.21.15.144:6824/1893803079 6 ==== osd_op_reply(60 [pgnls start_epoch 116 out=77b] v115'10 uv10 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x5609d4396780 con 0x5609d44e2800 2024-04-20T21:30:42.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.907+0000 7fea90c00640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:61 16.16 16:68000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d455f800 con 0x5609d455e800 2024-04-20T21:30:42.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.908+0000 7fea90c00640 1 -- 172.21.15.86:0/2846374639 <== osd.3 v2:172.21.15.86:6808/794135519 9 ==== osd_op_reply(61 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5609d4396780 con 0x5609d455e800 2024-04-20T21:30:42.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.908+0000 7fea90c00640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:62 16.e 16:70000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d455f800 con 0x5609d455f400 2024-04-20T21:30:42.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.909+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 <== osd.2 v2:172.21.15.86:6816/3437611468 19 ==== osd_op_reply(62 [pgnls start_epoch 116 out=73b] v115'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5609d4396500 con 0x5609d455f400 2024-04-20T21:30:42.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.909+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- osd_op(unknown.0.0:63 16.1e 16:78000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d455f800 con 0x5609d39a4c00 2024-04-20T21:30:42.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.910+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 <== osd.7 v2:172.21.15.144:6816/2100967057 2 ==== osd_op_reply(63 [pgnls start_epoch 116 out=49b] v116'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5609d4396000 con 0x5609d39a4c00 2024-04-20T21:30:42.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.910+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:64 16.1 16:80000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d455f800 con 0x5609d455f400 2024-04-20T21:30:42.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.911+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 <== osd.2 v2:172.21.15.86:6816/3437611468 20 ==== osd_op_reply(64 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5609d4396500 con 0x5609d455f400 2024-04-20T21:30:42.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.911+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:65 16.11 16:88000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d455f800 con 0x5609d39a4000 2024-04-20T21:30:42.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.912+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 <== osd.5 v2:172.21.15.144:6808/1635974704 8 ==== osd_op_reply(65 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5609d4396000 con 0x5609d39a4000 2024-04-20T21:30:42.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.912+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:66 16.9 16:90000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d455f800 con 0x5609d39a5000 2024-04-20T21:30:42.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.913+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 <== osd.1 v2:172.21.15.86:6817/3149564961 5 ==== osd_op_reply(66 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5609d4396500 con 0x5609d39a5000 2024-04-20T21:30:42.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.913+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- osd_op(unknown.0.0:67 16.19 16:98000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d455f800 con 0x5609d39a4400 2024-04-20T21:30:42.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.914+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 <== osd.4 v2:172.21.15.144:6800/4087309475 4 ==== osd_op_reply(67 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5609d4396500 con 0x5609d39a4400 2024-04-20T21:30:42.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.914+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:68 16.5 16:a0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d455f800 con 0x5609d455f000 2024-04-20T21:30:42.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.915+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 <== osd.0 v2:172.21.15.86:6800/2471076035 14 ==== osd_op_reply(68 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5609d4396000 con 0x5609d455f000 2024-04-20T21:30:42.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.915+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:69 16.15 16:a8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d455f800 con 0x5609d455f000 2024-04-20T21:30:42.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.916+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 <== osd.0 v2:172.21.15.86:6800/2471076035 15 ==== osd_op_reply(69 [pgnls start_epoch 116 out=97b] v115'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x5609d4396000 con 0x5609d455f000 2024-04-20T21:30:42.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.916+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:70 16.d 16:b0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d455f800 con 0x5609d44e2800 2024-04-20T21:30:42.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.918+0000 7fea90c00640 1 -- 172.21.15.86:0/2846374639 <== osd.6 v2:172.21.15.144:6824/1893803079 7 ==== osd_op_reply(70 [pgnls start_epoch 116 out=74b] v111'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x5609d4396780 con 0x5609d44e2800 2024-04-20T21:30:42.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.918+0000 7fea90c00640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:71 16.1d 16:b8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d455f800 con 0x5609d455f400 2024-04-20T21:30:42.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.918+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 <== osd.2 v2:172.21.15.86:6816/3437611468 21 ==== osd_op_reply(71 [pgnls start_epoch 116 out=73b] v115'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5609d4396500 con 0x5609d455f400 2024-04-20T21:30:42.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.918+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:72 16.3 16:c0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d455f800 con 0x5609d39a4000 2024-04-20T21:30:42.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.920+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 <== osd.5 v2:172.21.15.144:6808/1635974704 9 ==== osd_op_reply(72 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5609d4396000 con 0x5609d39a4000 2024-04-20T21:30:42.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.920+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:73 16.13 16:c8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d455f800 con 0x5609d39a5000 2024-04-20T21:30:42.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.921+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 <== osd.1 v2:172.21.15.86:6817/3149564961 6 ==== osd_op_reply(73 [pgnls start_epoch 116 out=98b] v115'5 uv5 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x5609d4396500 con 0x5609d39a5000 2024-04-20T21:30:42.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.921+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:74 16.b 16:d0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d455f800 con 0x5609d39a5000 2024-04-20T21:30:42.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.922+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 <== osd.1 v2:172.21.15.86:6817/3149564961 7 ==== osd_op_reply(74 [pgnls start_epoch 116 out=76b] v116'6 uv5 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x5609d4396500 con 0x5609d39a5000 2024-04-20T21:30:42.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.922+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:75 16.1b 16:d8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d455f800 con 0x5609d39a4000 2024-04-20T21:30:42.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.923+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 <== osd.5 v2:172.21.15.144:6808/1635974704 10 ==== osd_op_reply(75 [pgnls start_epoch 116 out=49b] v116'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5609d4396000 con 0x5609d39a4000 2024-04-20T21:30:42.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.923+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:76 16.7 16:e0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d455f800 con 0x5609d455e800 2024-04-20T21:30:42.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.924+0000 7fea90c00640 1 -- 172.21.15.86:0/2846374639 <== osd.3 v2:172.21.15.86:6808/794135519 10 ==== osd_op_reply(76 [pgnls start_epoch 116 out=105b] v116'9 uv9 ondisk = 1) ==== 144+0+105 (crc 0 0 0) 0x5609d4396780 con 0x5609d455e800 2024-04-20T21:30:42.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.924+0000 7fea90c00640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:77 16.17 16:e8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d455f800 con 0x5609d455f400 2024-04-20T21:30:42.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.925+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 <== osd.2 v2:172.21.15.86:6816/3437611468 22 ==== osd_op_reply(77 [pgnls start_epoch 116 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5609d4396500 con 0x5609d455f400 2024-04-20T21:30:42.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.925+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:78 16.f 16:f0000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d455f800 con 0x5609d455f000 2024-04-20T21:30:42.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.926+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 <== osd.0 v2:172.21.15.86:6800/2471076035 16 ==== osd_op_reply(78 [pgnls start_epoch 116 out=73b] v116'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5609d4396000 con 0x5609d455f000 2024-04-20T21:30:42.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.926+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:79 16.1f 16:f8000000::::head [pgnls start_epoch 116 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e116) -- 0x5609d455f800 con 0x5609d39a4000 2024-04-20T21:30:42.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.927+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 <== osd.5 v2:172.21.15.144:6808/1635974704 11 ==== osd_op_reply(79 [pgnls start_epoch 116 out=101b] v116'9 uv8 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x5609d4396000 con 0x5609d39a4000 2024-04-20T21:30:42.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.927+0000 7fea9324e740 10 datascan.scan_links: step 2: handling object 601.00000000 2024-04-20T21:30:42.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.927+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:80 16.10 16:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x5609d455fc00 con 0x5609d455f400 2024-04-20T21:30:42.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.928+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 <== osd.2 v2:172.21.15.86:6816/3437611468 23 ==== osd_op_reply(80 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5609d4396500 con 0x5609d455f400 2024-04-20T21:30:42.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.928+0000 7fea9324e740 10 datascan.scan_links: step 2: handling object 602.00000000 2024-04-20T21:30:42.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.928+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:81 16.10 16:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x5609d39cc000 con 0x5609d455f400 2024-04-20T21:30:42.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.929+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 <== osd.2 v2:172.21.15.86:6816/3437611468 24 ==== osd_op_reply(81 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5609d4396500 con 0x5609d455f400 2024-04-20T21:30:42.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.929+0000 7fea9324e740 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-04-20T21:30:42.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.930+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:82 16.10 16:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x5609d39cc400 con 0x5609d455f400 2024-04-20T21:30:42.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.931+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 <== osd.2 v2:172.21.15.86:6816/3437611468 25 ==== osd_op_reply(82 10000000000.00000000 [omap-get-vals out=1047b] v0'0 uv12 ondisk = 0) ==== 164+0+1047 (crc 0 0 0) 0x5609d4396500 con 0x5609d455f400 2024-04-20T21:30:42.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.931+0000 7fea9324e740 10 datascan.scan_links: step 2: handling object 600.00000000 2024-04-20T21:30:42.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.931+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:83 16.10 16:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x5609d39cc800 con 0x5609d455f400 2024-04-20T21:30:42.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.931+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 <== osd.2 v2:172.21.15.86:6816/3437611468 26 ==== osd_op_reply(83 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5609d4396500 con 0x5609d455f400 2024-04-20T21:30:42.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.931+0000 7fea9324e740 10 datascan.scan_links: step 2: handling object 603.00000000 2024-04-20T21:30:42.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.931+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:84 16.10 16:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x5609d39ccc00 con 0x5609d455f400 2024-04-20T21:30:42.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.932+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 <== osd.2 v2:172.21.15.86:6816/3437611468 27 ==== osd_op_reply(84 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5609d4396500 con 0x5609d455f400 2024-04-20T21:30:42.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.932+0000 7fea9324e740 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-04-20T21:30:42.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.932+0000 7fea9324e740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-20T21:30:42.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.932+0000 7fea9324e740 10 datascan.scan_links: step 2: handling object 200.00000000 2024-04-20T21:30:42.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.932+0000 7fea9324e740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-20T21:30:42.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.932+0000 7fea9324e740 10 datascan.scan_links: step 2: handling object 200.00000001 2024-04-20T21:30:42.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.932+0000 7fea9324e740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-20T21:30:42.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.932+0000 7fea9324e740 10 datascan.scan_links: step 2: handling object 606.00000000 2024-04-20T21:30:42.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.932+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:85 16.14 16:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x5609d39cd000 con 0x5609d455f000 2024-04-20T21:30:42.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.933+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 <== osd.0 v2:172.21.15.86:6800/2471076035 17 ==== osd_op_reply(85 606.00000000 [omap-get-vals out=5b] v0'0 uv39 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5609d4396000 con 0x5609d455f000 2024-04-20T21:30:42.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.933+0000 7fea9324e740 10 datascan.scan_links: step 2: handling object 607.00000000 2024-04-20T21:30:42.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.933+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:86 16.1c 16:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x5609d39cd400 con 0x5609d455f400 2024-04-20T21:30:42.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.934+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 <== osd.2 v2:172.21.15.86:6816/3437611468 28 ==== osd_op_reply(86 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5609d4396500 con 0x5609d455f400 2024-04-20T21:30:42.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.934+0000 7fea9324e740 10 datascan.scan_links: step 2: handling object 10000000003.00000000 2024-04-20T21:30:42.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.935+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:87 16.12 16:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x5609d39cd800 con 0x5609d44e2800 2024-04-20T21:30:42.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.936+0000 7fea90c00640 1 -- 172.21.15.86:0/2846374639 <== osd.6 v2:172.21.15.144:6824/1893803079 8 ==== osd_op_reply(87 10000000003.00000000 [omap-get-vals out=523b] v0'0 uv3 ondisk = 0) ==== 164+0+523 (crc 0 0 0) 0x5609d4396780 con 0x5609d44e2800 2024-04-20T21:30:42.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.936+0000 7fea9324e740 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-04-20T21:30:42.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.936+0000 7fea9324e740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-20T21:30:42.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.936+0000 7fea9324e740 10 datascan.scan_links: step 2: handling object 608.00000000 2024-04-20T21:30:42.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.936+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:88 16.e 16:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x5609d39cdc00 con 0x5609d455f400 2024-04-20T21:30:42.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.937+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 <== osd.2 v2:172.21.15.86:6816/3437611468 29 ==== osd_op_reply(88 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5609d4396500 con 0x5609d455f400 2024-04-20T21:30:42.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.937+0000 7fea9324e740 10 datascan.scan_links: step 2: handling object 604.00000000 2024-04-20T21:30:42.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.937+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- osd_op(unknown.0.0:89 16.15 16:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x5609d39cc000 con 0x5609d455f000 2024-04-20T21:30:42.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.938+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 <== osd.0 v2:172.21.15.86:6800/2471076035 18 ==== osd_op_reply(89 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5609d4396000 con 0x5609d455f000 2024-04-20T21:30:42.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.938+0000 7fea9324e740 10 datascan.scan_links: step 2: handling object 500.00000000 2024-04-20T21:30:42.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.938+0000 7fea9324e740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-20T21:30:42.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.938+0000 7fea9324e740 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-04-20T21:30:42.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.938+0000 7fea9324e740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-20T21:30:42.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.938+0000 7fea9324e740 10 datascan.scan_links: step 2: handling object 605.00000000 2024-04-20T21:30:42.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.938+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- osd_op(unknown.0.0:90 16.1d 16:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x5609d455fc00 con 0x5609d455f400 2024-04-20T21:30:42.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.939+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 <== osd.2 v2:172.21.15.86:6816/3437611468 30 ==== osd_op_reply(90 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5609d4396500 con 0x5609d455f400 2024-04-20T21:30:42.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.939+0000 7fea9324e740 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-04-20T21:30:42.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.939+0000 7fea9324e740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-20T21:30:42.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.939+0000 7fea9324e740 10 datascan.scan_links: step 2: handling object 100.00000000 2024-04-20T21:30:42.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.939+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:91 16.13 16:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x5609d39c2400 con 0x5609d39a5000 2024-04-20T21:30:42.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.941+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 <== osd.1 v2:172.21.15.86:6817/3149564961 8 ==== osd_op_reply(91 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x5609d4396500 con 0x5609d39a5000 2024-04-20T21:30:42.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.941+0000 7fea9324e740 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-04-20T21:30:42.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.941+0000 7fea9324e740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-20T21:30:42.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.941+0000 7fea9324e740 10 datascan.scan_links: step 2: handling object 609.00000000 2024-04-20T21:30:42.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.941+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:92 16.7 16:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x5609d39c2c00 con 0x5609d455e800 2024-04-20T21:30:42.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.942+0000 7fea90c00640 1 -- 172.21.15.86:0/2846374639 <== osd.3 v2:172.21.15.86:6808/794135519 11 ==== osd_op_reply(92 609.00000000 [omap-get-vals out=5b] v0'0 uv8 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5609d4396780 con 0x5609d455e800 2024-04-20T21:30:42.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.942+0000 7fea9324e740 10 datascan.scan_links: step 2: handling object 10000000001.00000000 2024-04-20T21:30:42.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.942+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:93 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x5609d39c3400 con 0x5609d455e800 2024-04-20T21:30:42.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.943+0000 7fea90c00640 1 -- 172.21.15.86:0/2846374639 <== osd.3 v2:172.21.15.86:6808/794135519 12 ==== osd_op_reply(93 10000000001.00000000 [omap-get-vals out=1545b] v0'0 uv9 ondisk = 0) ==== 164+0+1545 (crc 0 0 0) 0x5609d4396780 con 0x5609d455e800 2024-04-20T21:30:42.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.943+0000 7fea9324e740 20 datascan.scan_links: injected ino detected 2024-04-20T21:30:42.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.943+0000 7fea9324e740 20 datascan.scan_links: adding 0x100000001f7 for future processing to fix dnfirst 2024-04-20T21:30:42.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.943+0000 7fea9324e740 10 datascan.scan_links: step 2: handling object 400.00000000 2024-04-20T21:30:42.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.943+0000 7fea9324e740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-20T21:30:42.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.943+0000 7fea9324e740 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-04-20T21:30:42.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.943+0000 7fea9324e740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-20T21:30:42.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.943+0000 7fea9324e740 10 datascan.scan_links: step 2: handling object 1.00000000 2024-04-20T21:30:42.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.943+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- osd_op(unknown.0.0:94 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x5609d39c2000 con 0x5609d39a4000 2024-04-20T21:30:42.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.945+0000 7fea90200640 1 -- 172.21.15.86:0/2846374639 <== osd.5 v2:172.21.15.144:6808/1635974704 12 ==== osd_op_reply(94 1.00000000 [omap-get-vals out=524b] v0'0 uv8 ondisk = 0) ==== 154+0+524 (crc 0 0 0) 0x5609d4396000 con 0x5609d39a4000 2024-04-20T21:30:42.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.945+0000 7fea9324e740 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2024-04-20T21:30:42.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.945+0000 7fea9324e740 10 datascan.scan_links: removing dup dentries from 0 objects 2024-04-20T21:30:42.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.945+0000 7fea9324e740 10 datascan.scan_links: processing 0 bad_nlink_inos 2024-04-20T21:30:42.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.945+0000 7fea9324e740 10 datascan.scan_links: processing 1 injected_inos 2024-04-20T21:30:42.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.945+0000 7fea9324e740 10 datascan.scan_links: handling injected_ino 0x100000001f7 2024-04-20T21:30:42.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.945+0000 7fea9324e740 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000001:bravo 2024-04-20T21:30:42.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.945+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:95 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x5609d39c3c00 con 0x5609d455e800 2024-04-20T21:30:42.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.946+0000 7fea90c00640 1 -- 172.21.15.86:0/2846374639 <== osd.3 v2:172.21.15.86:6808/794135519 13 ==== osd_op_reply(95 10000000001.00000000 [omap-get-vals-by-keys out=506b] v0'0 uv9 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x5609d4396780 con 0x5609d455e800 2024-04-20T21:30:42.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.946+0000 7fea9324e740 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-04-20T21:30:42.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.946+0000 7fea9324e740 20 datascan.scan_links: first is now 2 2024-04-20T21:30:42.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.946+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- osd_op(unknown.0.0:96 16.7 16:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x5609d455f800 con 0x5609d455e800 2024-04-20T21:30:42.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.950+0000 7fea90c00640 1 -- 172.21.15.86:0/2846374639 <== osd.3 v2:172.21.15.86:6808/794135519 14 ==== osd_op_reply(96 10000000001.00000000 [omap-set-vals] v116'10 uv10 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5609d4396780 con 0x5609d455e800 2024-04-20T21:30:42.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.950+0000 7fea9324e740 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2024-04-20T21:30:42.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.950+0000 7fea9324e740 10 datascan.scan_links: updating inotable 2024-04-20T21:30:42.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.950+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:97 16.13 16:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x5609d39a5400 con 0x5609d39a5000 2024-04-20T21:30:42.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.951+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 <== osd.1 v2:172.21.15.86:6817/3149564961 9 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv4 ondisk = 0) ==== 157+0+50 (crc 0 0 0) 0x5609d4396500 con 0x5609d39a5000 2024-04-20T21:30:42.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.951+0000 7fea9324e740 10 mds.0.inotable: skip_inos was [0x10000000005~0x1f2,0x100000001f9~0xfffffffe07] 2024-04-20T21:30:42.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.951+0000 7fea9324e740 10 mds.0.inotable: skip_inos now [0x100000001f9~0xfffffffe07] 2024-04-20T21:30:42.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.951+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- osd_op(unknown.0.0:98 16.13 16:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x5609d39a5800 con 0x5609d39a5000 2024-04-20T21:30:42.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.955+0000 7fea91600640 1 -- 172.21.15.86:0/2846374639 <== osd.1 v2:172.21.15.86:6817/3149564961 10 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v116'6 uv6 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x5609d4396500 con 0x5609d39a5000 2024-04-20T21:30:42.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.955+0000 7fea9324e740 10 datascan.scan_links: updating snaptable 2024-04-20T21:30:42.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.955+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- osd_op(unknown.0.0:99 16.d 16:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e116) -- 0x5609d39a5c00 con 0x5609d44e2800 2024-04-20T21:30:42.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.957+0000 7fea90c00640 1 -- 172.21.15.86:0/2846374639 <== osd.6 v2:172.21.15.144:6824/1893803079 9 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) ==== 157+0+70 (crc 0 0 0) 0x5609d4396780 con 0x5609d44e2800 2024-04-20T21:30:42.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.957+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x5609d455f000 msgr2=0x5609d4372680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.957+0000 7fea9324e740 1 --2- 172.21.15.86:0/2846374639 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x5609d455f000 0x5609d4372680 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.957+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x5609d39a5000 msgr2=0x5609d3996100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.957+0000 7fea9324e740 1 --2- 172.21.15.86:0/2846374639 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x5609d39a5000 0x5609d3996100 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.957+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x5609d455f400 msgr2=0x5609d3994000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.957+0000 7fea9324e740 1 --2- 172.21.15.86:0/2846374639 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x5609d455f400 0x5609d3994000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.957+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x5609d455e800 msgr2=0x5609d3994580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.957+0000 7fea9324e740 1 --2- 172.21.15.86:0/2846374639 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x5609d455e800 0x5609d3994580 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.957+0000 7fea90c00640 1 -- 172.21.15.86:0/2846374639 reap_dead start 2024-04-20T21:30:42.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.957+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x5609d39a4400 msgr2=0x5609d3995080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.957+0000 7fea9324e740 1 --2- 172.21.15.86:0/2846374639 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x5609d39a4400 0x5609d3995080 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.957+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x5609d39a4000 msgr2=0x5609d3994b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.957+0000 7fea9324e740 1 --2- 172.21.15.86:0/2846374639 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x5609d39a4000 0x5609d3994b00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.958+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x5609d44e2800 msgr2=0x5609d3995600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.958+0000 7fea9324e740 1 --2- 172.21.15.86:0/2846374639 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x5609d44e2800 0x5609d3995600 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.958+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x5609d39a4c00 msgr2=0x5609d3995b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.958+0000 7fea9324e740 1 --2- 172.21.15.86:0/2846374639 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x5609d39a4c00 0x5609d3995b80 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.958+0000 7fea90c00640 1 -- 172.21.15.86:0/2846374639 reap_dead start 2024-04-20T21:30:42.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.958+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x5609d455ec00 msgr2=0x5609d4372100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.958+0000 7fea9324e740 1 --2- 172.21.15.86:0/2846374639 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x5609d455ec00 0x5609d4372100 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x5609d44f6a80 tx=0x5609d44fd140 comp rx=0 tx=0).stop 2024-04-20T21:30:42.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.958+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609d44e3800 msgr2=0x5609d4371600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.958+0000 7fea9324e740 1 --2- 172.21.15.86:0/2846374639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609d44e3800 0x5609d4371600 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x5609d44f7bf0 tx=0x5609d43acd20 comp rx=0 tx=0).stop 2024-04-20T21:30:42.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.958+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 shutdown_connections 2024-04-20T21:30:42.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.958+0000 7fea9324e740 1 --2- 172.21.15.86:0/2846374639 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x5609d455ec00 0x5609d4372100 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.958+0000 7fea9324e740 1 --2- 172.21.15.86:0/2846374639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609d44e3800 0x5609d4371600 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.958+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 >> 172.21.15.86:0/2846374639 conn(0x5609d44e3000 msgr2=0x5609d36f3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:42.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.959+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 shutdown_connections 2024-04-20T21:30:42.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.959+0000 7fea9324e740 1 -- 172.21.15.86:0/2846374639 wait complete. 2024-04-20T21:30:42.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.960+0000 7fea9324e740 1 -- 172.21.15.86:0/779686845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609d3609c00 msgr2=0x5609d4370b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:42.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.960+0000 7fea9324e740 1 --2- 172.21.15.86:0/779686845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609d3609c00 0x5609d4370b00 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x5609d442b4a0 tx=0x5609d43acf60 comp rx=0 tx=0).stop 2024-04-20T21:30:42.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.960+0000 7fea9324e740 1 -- 172.21.15.86:0/779686845 shutdown_connections 2024-04-20T21:30:42.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.960+0000 7fea9324e740 1 --2- 172.21.15.86:0/779686845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5609d3609c00 0x5609d4370b00 unknown :-1 s=CLOSED pgs=2054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.960+0000 7fea9324e740 1 --2- 172.21.15.86:0/779686845 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x5609d3609800 0x5609d4370580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.960+0000 7fea9324e740 1 --2- 172.21.15.86:0/779686845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x5609d3608c00 0x5609d4370000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:42.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.960+0000 7fea9324e740 1 -- 172.21.15.86:0/779686845 >> 172.21.15.86:0/779686845 conn(0x5609d3609000 msgr2=0x5609d36f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:42.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.960+0000 7fea9324e740 1 -- 172.21.15.86:0/779686845 shutdown_connections 2024-04-20T21:30:42.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:42.960+0000 7fea9324e740 1 -- 172.21.15.86:0/779686845 wait complete. 2024-04-20T21:30:42.968 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.228802, result: 2024-04-20T21:30:42.969 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-20T21:30:42.977 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-20T21:30:43.106 INFO:teuthology.orchestra.run.smithi086.stdout:[global] 2024-04-20T21:30:43.106 INFO:teuthology.orchestra.run.smithi086.stdout: chdir = "" 2024-04-20T21:30:43.106 INFO:teuthology.orchestra.run.smithi086.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-20T21:30:43.106 INFO:teuthology.orchestra.run.smithi086.stdout: auth supported = cephx 2024-04-20T21:30:43.106 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.106 INFO:teuthology.orchestra.run.smithi086.stdout: filestore xattr use omap = true 2024-04-20T21:30:43.106 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.106 INFO:teuthology.orchestra.run.smithi086.stdout: mon clock drift allowed = 1.000 2024-04-20T21:30:43.106 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.107 INFO:teuthology.orchestra.run.smithi086.stdout: osd crush chooseleaf type = 0 2024-04-20T21:30:43.107 INFO:teuthology.orchestra.run.smithi086.stdout: auth debug = true 2024-04-20T21:30:43.107 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.107 INFO:teuthology.orchestra.run.smithi086.stdout: ms die on old message = true 2024-04-20T21:30:43.107 INFO:teuthology.orchestra.run.smithi086.stdout: ms die on bug = true 2024-04-20T21:30:43.107 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.107 INFO:teuthology.orchestra.run.smithi086.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-20T21:30:43.107 INFO:teuthology.orchestra.run.smithi086.stdout: mon pg warn max object skew = 0 2024-04-20T21:30:43.107 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.107 INFO:teuthology.orchestra.run.smithi086.stdout: # disable pg_autoscaler by default for new pools 2024-04-20T21:30:43.107 INFO:teuthology.orchestra.run.smithi086.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-20T21:30:43.107 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.107 INFO:teuthology.orchestra.run.smithi086.stdout: osd pool default size = 2 2024-04-20T21:30:43.107 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.107 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd allow primary affinity = true 2024-04-20T21:30:43.107 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd allow pg remap = true 2024-04-20T21:30:43.108 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on legacy crush tunables = false 2024-04-20T21:30:43.108 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on crush straw calc version zero = false 2024-04-20T21:30:43.108 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on no sortbitwise = false 2024-04-20T21:30:43.108 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on osd down out interval zero = false 2024-04-20T21:30:43.108 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on too few osds = false 2024-04-20T21:30:43.108 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-20T21:30:43.108 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-20T21:30:43.108 INFO:teuthology.orchestra.run.smithi086.stdout: mon_allow_pool_size_one = true 2024-04-20T21:30:43.108 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.108 INFO:teuthology.orchestra.run.smithi086.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-20T21:30:43.108 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.108 INFO:teuthology.orchestra.run.smithi086.stdout: osd default data pool replay window = 5 2024-04-20T21:30:43.108 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.108 INFO:teuthology.orchestra.run.smithi086.stdout: mon allow pool delete = true 2024-04-20T21:30:43.109 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.109 INFO:teuthology.orchestra.run.smithi086.stdout: mon cluster log file level = debug 2024-04-20T21:30:43.109 INFO:teuthology.orchestra.run.smithi086.stdout: debug asserts on shutdown = true 2024-04-20T21:30:43.109 INFO:teuthology.orchestra.run.smithi086.stdout: mon health detail to clog = false 2024-04-20T21:30:43.109 INFO:teuthology.orchestra.run.smithi086.stdout: mon host = "172.21.15.86,172.21.15.144,[v2:172.21.15.144:3301,v1:172.21.15.144:6790]" 2024-04-20T21:30:43.109 INFO:teuthology.orchestra.run.smithi086.stdout: fsid = 9c2dec1d-d7ae-4a83-a812-79922808e278 2024-04-20T21:30:43.109 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.109 INFO:teuthology.orchestra.run.smithi086.stdout:[osd] 2024-04-20T21:30:43.109 INFO:teuthology.orchestra.run.smithi086.stdout: osd journal size = 100 2024-04-20T21:30:43.109 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.109 INFO:teuthology.orchestra.run.smithi086.stdout: osd scrub load threshold = 5.0 2024-04-20T21:30:43.109 INFO:teuthology.orchestra.run.smithi086.stdout: osd scrub max interval = 600 2024-04-20T21:30:43.109 INFO:teuthology.orchestra.run.smithi086.stdout: osd mclock profile = high_recovery_ops 2024-04-20T21:30:43.110 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.110 INFO:teuthology.orchestra.run.smithi086.stdout: osd recover clone overlap = true 2024-04-20T21:30:43.110 INFO:teuthology.orchestra.run.smithi086.stdout: osd recovery max chunk = 1048576 2024-04-20T21:30:43.110 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.110 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug shutdown = true 2024-04-20T21:30:43.110 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug op order = true 2024-04-20T21:30:43.110 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug verify stray on activate = true 2024-04-20T21:30:43.110 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug trim objects = true 2024-04-20T21:30:43.110 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.110 INFO:teuthology.orchestra.run.smithi086.stdout: osd open classes on start = true 2024-04-20T21:30:43.110 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug pg log writeout = true 2024-04-20T21:30:43.110 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.110 INFO:teuthology.orchestra.run.smithi086.stdout: osd deep scrub update digest min age = 30 2024-04-20T21:30:43.110 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.111 INFO:teuthology.orchestra.run.smithi086.stdout: osd map max advance = 10 2024-04-20T21:30:43.111 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.111 INFO:teuthology.orchestra.run.smithi086.stdout: journal zero on create = true 2024-04-20T21:30:43.111 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.111 INFO:teuthology.orchestra.run.smithi086.stdout: filestore ondisk finisher threads = 3 2024-04-20T21:30:43.111 INFO:teuthology.orchestra.run.smithi086.stdout: filestore apply finisher threads = 3 2024-04-20T21:30:43.111 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.111 INFO:teuthology.orchestra.run.smithi086.stdout: bdev debug aio = true 2024-04-20T21:30:43.111 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug misdirected ops = true 2024-04-20T21:30:43.111 INFO:teuthology.orchestra.run.smithi086.stdout: bdev async discard = True 2024-04-20T21:30:43.111 INFO:teuthology.orchestra.run.smithi086.stdout: bdev enable discard = True 2024-04-20T21:30:43.111 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore allocator = bitmap 2024-04-20T21:30:43.111 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore block size = 96636764160 2024-04-20T21:30:43.111 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore fsck on mount = True 2024-04-20T21:30:43.111 INFO:teuthology.orchestra.run.smithi086.stdout: debug bluefs = 1/20 2024-04-20T21:30:43.111 INFO:teuthology.orchestra.run.smithi086.stdout: debug bluestore = 1/20 2024-04-20T21:30:43.112 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-20T21:30:43.112 INFO:teuthology.orchestra.run.smithi086.stdout: debug osd = 20 2024-04-20T21:30:43.112 INFO:teuthology.orchestra.run.smithi086.stdout: debug rocksdb = 4/10 2024-04-20T21:30:43.112 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd backfillfull_ratio = 0.85 2024-04-20T21:30:43.112 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd full ratio = 0.9 2024-04-20T21:30:43.112 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd nearfull ratio = 0.8 2024-04-20T21:30:43.112 INFO:teuthology.orchestra.run.smithi086.stdout: osd failsafe full ratio = 0.95 2024-04-20T21:30:43.112 INFO:teuthology.orchestra.run.smithi086.stdout: osd objectstore = bluestore 2024-04-20T21:30:43.112 INFO:teuthology.orchestra.run.smithi086.stdout: osd op complaint time = 180 2024-04-20T21:30:43.112 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.112 INFO:teuthology.orchestra.run.smithi086.stdout:[mgr] 2024-04-20T21:30:43.112 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-20T21:30:43.112 INFO:teuthology.orchestra.run.smithi086.stdout: debug mgr = 20 2024-04-20T21:30:43.112 INFO:teuthology.orchestra.run.smithi086.stdout: debug mon = 20 2024-04-20T21:30:43.112 INFO:teuthology.orchestra.run.smithi086.stdout: debug auth = 20 2024-04-20T21:30:43.113 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min pgs per osd = 4 2024-04-20T21:30:43.113 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min bytes per osd = 10 2024-04-20T21:30:43.113 INFO:teuthology.orchestra.run.smithi086.stdout: mgr/telemetry/nag = false 2024-04-20T21:30:43.113 INFO:teuthology.orchestra.run.smithi086.stdout: debug client = 20 2024-04-20T21:30:43.113 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.113 INFO:teuthology.orchestra.run.smithi086.stdout:[mon] 2024-04-20T21:30:43.113 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-20T21:30:43.113 INFO:teuthology.orchestra.run.smithi086.stdout: debug mon = 20 2024-04-20T21:30:43.113 INFO:teuthology.orchestra.run.smithi086.stdout: debug paxos = 20 2024-04-20T21:30:43.113 INFO:teuthology.orchestra.run.smithi086.stdout: debug auth = 20 2024-04-20T21:30:43.113 INFO:teuthology.orchestra.run.smithi086.stdout: mon data avail warn = 5 2024-04-20T21:30:43.113 INFO:teuthology.orchestra.run.smithi086.stdout: mon mgr mkfs grace = 240 2024-04-20T21:30:43.113 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min pgs per osd = 4 2024-04-20T21:30:43.113 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd reporter subtree level = osd 2024-04-20T21:30:43.114 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd prime pg temp = true 2024-04-20T21:30:43.114 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min bytes per osd = 10 2024-04-20T21:30:43.114 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.114 INFO:teuthology.orchestra.run.smithi086.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-20T21:30:43.114 INFO:teuthology.orchestra.run.smithi086.stdout: auth mon ticket ttl = 660 # 11m 2024-04-20T21:30:43.114 INFO:teuthology.orchestra.run.smithi086.stdout: auth service ticket ttl = 240 # 4m 2024-04-20T21:30:43.114 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.114 INFO:teuthology.orchestra.run.smithi086.stdout: # don't complain about insecure global_id in the test suite 2024-04-20T21:30:43.114 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-20T21:30:43.114 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-20T21:30:43.114 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.114 INFO:teuthology.orchestra.run.smithi086.stdout: # 1m isn't quite enough 2024-04-20T21:30:43.114 INFO:teuthology.orchestra.run.smithi086.stdout: mon_down_mkfs_grace = 2m 2024-04-20T21:30:43.114 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.115 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_filestore_osds = false 2024-04-20T21:30:43.115 INFO:teuthology.orchestra.run.smithi086.stdout: mon down mkfs grace = 300 2024-04-20T21:30:43.115 INFO:teuthology.orchestra.run.smithi086.stdout: mon op complaint time = 120 2024-04-20T21:30:43.115 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.115 INFO:teuthology.orchestra.run.smithi086.stdout:[client] 2024-04-20T21:30:43.115 INFO:teuthology.orchestra.run.smithi086.stdout: rgw cache enabled = true 2024-04-20T21:30:43.115 INFO:teuthology.orchestra.run.smithi086.stdout: rgw enable ops log = true 2024-04-20T21:30:43.115 INFO:teuthology.orchestra.run.smithi086.stdout: rgw enable usage log = true 2024-04-20T21:30:43.115 INFO:teuthology.orchestra.run.smithi086.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-20T21:30:43.115 INFO:teuthology.orchestra.run.smithi086.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-20T21:30:43.115 INFO:teuthology.orchestra.run.smithi086.stdout: client mount timeout = 600 2024-04-20T21:30:43.115 INFO:teuthology.orchestra.run.smithi086.stdout: debug client = 20 2024-04-20T21:30:43.115 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-20T21:30:43.115 INFO:teuthology.orchestra.run.smithi086.stdout: pid file = "" 2024-04-20T21:30:43.115 INFO:teuthology.orchestra.run.smithi086.stdout: rados mon op timeout = 900 2024-04-20T21:30:43.116 INFO:teuthology.orchestra.run.smithi086.stdout: rados osd op timeout = 900 2024-04-20T21:30:43.116 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.a] 2024-04-20T21:30:43.116 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.b] 2024-04-20T21:30:43.116 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.c] 2024-04-20T21:30:43.116 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.a] 2024-04-20T21:30:43.116 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.c] 2024-04-20T21:30:43.116 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.b] 2024-04-20T21:30:43.116 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.d] 2024-04-20T21:30:43.116 INFO:teuthology.orchestra.run.smithi086.stdout:[mds] 2024-04-20T21:30:43.116 INFO:teuthology.orchestra.run.smithi086.stdout: debug mds = 20 2024-04-20T21:30:43.116 INFO:teuthology.orchestra.run.smithi086.stdout: debug mds balancer = 20 2024-04-20T21:30:43.116 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-20T21:30:43.116 INFO:teuthology.orchestra.run.smithi086.stdout: mds debug frag = True 2024-04-20T21:30:43.116 INFO:teuthology.orchestra.run.smithi086.stdout: mds debug scatterstat = False 2024-04-20T21:30:43.116 INFO:teuthology.orchestra.run.smithi086.stdout: mds op complaint time = 180 2024-04-20T21:30:43.117 INFO:teuthology.orchestra.run.smithi086.stdout: mds verify scatter = True 2024-04-20T21:30:43.117 INFO:teuthology.orchestra.run.smithi086.stdout: osd op complaint time = 180 2024-04-20T21:30:43.117 INFO:teuthology.orchestra.run.smithi086.stdout: rados mon op timeout = 900 2024-04-20T21:30:43.117 INFO:teuthology.orchestra.run.smithi086.stdout: rados osd op timeout = 900 2024-04-20T21:30:43.121 INFO:teuthology.orchestra.run.smithi144.stdout:[global] 2024-04-20T21:30:43.121 INFO:teuthology.orchestra.run.smithi144.stdout: chdir = "" 2024-04-20T21:30:43.121 INFO:teuthology.orchestra.run.smithi144.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-20T21:30:43.121 INFO:teuthology.orchestra.run.smithi144.stdout: auth supported = cephx 2024-04-20T21:30:43.121 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.121 INFO:teuthology.orchestra.run.smithi144.stdout: filestore xattr use omap = true 2024-04-20T21:30:43.121 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.121 INFO:teuthology.orchestra.run.smithi144.stdout: mon clock drift allowed = 1.000 2024-04-20T21:30:43.121 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.122 INFO:teuthology.orchestra.run.smithi144.stdout: osd crush chooseleaf type = 0 2024-04-20T21:30:43.122 INFO:teuthology.orchestra.run.smithi144.stdout: auth debug = true 2024-04-20T21:30:43.122 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.122 INFO:teuthology.orchestra.run.smithi144.stdout: ms die on old message = true 2024-04-20T21:30:43.122 INFO:teuthology.orchestra.run.smithi144.stdout: ms die on bug = true 2024-04-20T21:30:43.122 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.122 INFO:teuthology.orchestra.run.smithi144.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-20T21:30:43.122 INFO:teuthology.orchestra.run.smithi144.stdout: mon pg warn max object skew = 0 2024-04-20T21:30:43.122 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.122 INFO:teuthology.orchestra.run.smithi144.stdout: # disable pg_autoscaler by default for new pools 2024-04-20T21:30:43.122 INFO:teuthology.orchestra.run.smithi144.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-20T21:30:43.122 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.122 INFO:teuthology.orchestra.run.smithi144.stdout: osd pool default size = 2 2024-04-20T21:30:43.122 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.123 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd allow primary affinity = true 2024-04-20T21:30:43.123 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd allow pg remap = true 2024-04-20T21:30:43.123 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on legacy crush tunables = false 2024-04-20T21:30:43.123 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on crush straw calc version zero = false 2024-04-20T21:30:43.123 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on no sortbitwise = false 2024-04-20T21:30:43.123 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on osd down out interval zero = false 2024-04-20T21:30:43.123 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on too few osds = false 2024-04-20T21:30:43.123 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-20T21:30:43.123 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-20T21:30:43.123 INFO:teuthology.orchestra.run.smithi144.stdout: mon_allow_pool_size_one = true 2024-04-20T21:30:43.123 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.123 INFO:teuthology.orchestra.run.smithi144.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-20T21:30:43.123 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.123 INFO:teuthology.orchestra.run.smithi144.stdout: osd default data pool replay window = 5 2024-04-20T21:30:43.123 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.124 INFO:teuthology.orchestra.run.smithi144.stdout: mon allow pool delete = true 2024-04-20T21:30:43.124 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.124 INFO:teuthology.orchestra.run.smithi144.stdout: mon cluster log file level = debug 2024-04-20T21:30:43.124 INFO:teuthology.orchestra.run.smithi144.stdout: debug asserts on shutdown = true 2024-04-20T21:30:43.124 INFO:teuthology.orchestra.run.smithi144.stdout: mon health detail to clog = false 2024-04-20T21:30:43.124 INFO:teuthology.orchestra.run.smithi144.stdout: mon host = "172.21.15.86,172.21.15.144,[v2:172.21.15.144:3301,v1:172.21.15.144:6790]" 2024-04-20T21:30:43.124 INFO:teuthology.orchestra.run.smithi144.stdout: fsid = 9c2dec1d-d7ae-4a83-a812-79922808e278 2024-04-20T21:30:43.124 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.124 INFO:teuthology.orchestra.run.smithi144.stdout:[osd] 2024-04-20T21:30:43.124 INFO:teuthology.orchestra.run.smithi144.stdout: osd journal size = 100 2024-04-20T21:30:43.124 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.124 INFO:teuthology.orchestra.run.smithi144.stdout: osd scrub load threshold = 5.0 2024-04-20T21:30:43.124 INFO:teuthology.orchestra.run.smithi144.stdout: osd scrub max interval = 600 2024-04-20T21:30:43.124 INFO:teuthology.orchestra.run.smithi144.stdout: osd mclock profile = high_recovery_ops 2024-04-20T21:30:43.124 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.125 INFO:teuthology.orchestra.run.smithi144.stdout: osd recover clone overlap = true 2024-04-20T21:30:43.125 INFO:teuthology.orchestra.run.smithi144.stdout: osd recovery max chunk = 1048576 2024-04-20T21:30:43.125 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.125 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug shutdown = true 2024-04-20T21:30:43.125 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug op order = true 2024-04-20T21:30:43.125 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug verify stray on activate = true 2024-04-20T21:30:43.125 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug trim objects = true 2024-04-20T21:30:43.125 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.125 INFO:teuthology.orchestra.run.smithi144.stdout: osd open classes on start = true 2024-04-20T21:30:43.125 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug pg log writeout = true 2024-04-20T21:30:43.125 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.125 INFO:teuthology.orchestra.run.smithi144.stdout: osd deep scrub update digest min age = 30 2024-04-20T21:30:43.125 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.125 INFO:teuthology.orchestra.run.smithi144.stdout: osd map max advance = 10 2024-04-20T21:30:43.125 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.125 INFO:teuthology.orchestra.run.smithi144.stdout: journal zero on create = true 2024-04-20T21:30:43.126 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.126 INFO:teuthology.orchestra.run.smithi144.stdout: filestore ondisk finisher threads = 3 2024-04-20T21:30:43.126 INFO:teuthology.orchestra.run.smithi144.stdout: filestore apply finisher threads = 3 2024-04-20T21:30:43.126 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.126 INFO:teuthology.orchestra.run.smithi144.stdout: bdev debug aio = true 2024-04-20T21:30:43.126 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug misdirected ops = true 2024-04-20T21:30:43.126 INFO:teuthology.orchestra.run.smithi144.stdout: bdev async discard = True 2024-04-20T21:30:43.126 INFO:teuthology.orchestra.run.smithi144.stdout: bdev enable discard = True 2024-04-20T21:30:43.126 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore allocator = bitmap 2024-04-20T21:30:43.126 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore block size = 96636764160 2024-04-20T21:30:43.126 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore fsck on mount = True 2024-04-20T21:30:43.126 INFO:teuthology.orchestra.run.smithi144.stdout: debug bluefs = 1/20 2024-04-20T21:30:43.126 INFO:teuthology.orchestra.run.smithi144.stdout: debug bluestore = 1/20 2024-04-20T21:30:43.126 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-04-20T21:30:43.126 INFO:teuthology.orchestra.run.smithi144.stdout: debug osd = 20 2024-04-20T21:30:43.127 INFO:teuthology.orchestra.run.smithi144.stdout: debug rocksdb = 4/10 2024-04-20T21:30:43.127 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd backfillfull_ratio = 0.85 2024-04-20T21:30:43.127 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd full ratio = 0.9 2024-04-20T21:30:43.127 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd nearfull ratio = 0.8 2024-04-20T21:30:43.127 INFO:teuthology.orchestra.run.smithi144.stdout: osd failsafe full ratio = 0.95 2024-04-20T21:30:43.127 INFO:teuthology.orchestra.run.smithi144.stdout: osd objectstore = bluestore 2024-04-20T21:30:43.127 INFO:teuthology.orchestra.run.smithi144.stdout: osd op complaint time = 180 2024-04-20T21:30:43.127 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.127 INFO:teuthology.orchestra.run.smithi144.stdout:[mgr] 2024-04-20T21:30:43.127 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-04-20T21:30:43.127 INFO:teuthology.orchestra.run.smithi144.stdout: debug mgr = 20 2024-04-20T21:30:43.127 INFO:teuthology.orchestra.run.smithi144.stdout: debug mon = 20 2024-04-20T21:30:43.127 INFO:teuthology.orchestra.run.smithi144.stdout: debug auth = 20 2024-04-20T21:30:43.127 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min pgs per osd = 4 2024-04-20T21:30:43.127 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min bytes per osd = 10 2024-04-20T21:30:43.127 INFO:teuthology.orchestra.run.smithi144.stdout: mgr/telemetry/nag = false 2024-04-20T21:30:43.128 INFO:teuthology.orchestra.run.smithi144.stdout: debug client = 20 2024-04-20T21:30:43.128 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.128 INFO:teuthology.orchestra.run.smithi144.stdout:[mon] 2024-04-20T21:30:43.128 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-04-20T21:30:43.128 INFO:teuthology.orchestra.run.smithi144.stdout: debug mon = 20 2024-04-20T21:30:43.128 INFO:teuthology.orchestra.run.smithi144.stdout: debug paxos = 20 2024-04-20T21:30:43.128 INFO:teuthology.orchestra.run.smithi144.stdout: debug auth = 20 2024-04-20T21:30:43.128 INFO:teuthology.orchestra.run.smithi144.stdout: mon data avail warn = 5 2024-04-20T21:30:43.128 INFO:teuthology.orchestra.run.smithi144.stdout: mon mgr mkfs grace = 240 2024-04-20T21:30:43.128 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min pgs per osd = 4 2024-04-20T21:30:43.128 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd reporter subtree level = osd 2024-04-20T21:30:43.128 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd prime pg temp = true 2024-04-20T21:30:43.128 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min bytes per osd = 10 2024-04-20T21:30:43.128 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.128 INFO:teuthology.orchestra.run.smithi144.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-20T21:30:43.128 INFO:teuthology.orchestra.run.smithi144.stdout: auth mon ticket ttl = 660 # 11m 2024-04-20T21:30:43.129 INFO:teuthology.orchestra.run.smithi144.stdout: auth service ticket ttl = 240 # 4m 2024-04-20T21:30:43.129 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.129 INFO:teuthology.orchestra.run.smithi144.stdout: # don't complain about insecure global_id in the test suite 2024-04-20T21:30:43.129 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-20T21:30:43.129 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-20T21:30:43.129 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.129 INFO:teuthology.orchestra.run.smithi144.stdout: # 1m isn't quite enough 2024-04-20T21:30:43.129 INFO:teuthology.orchestra.run.smithi144.stdout: mon_down_mkfs_grace = 2m 2024-04-20T21:30:43.129 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.129 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_filestore_osds = false 2024-04-20T21:30:43.129 INFO:teuthology.orchestra.run.smithi144.stdout: mon down mkfs grace = 300 2024-04-20T21:30:43.129 INFO:teuthology.orchestra.run.smithi144.stdout: mon op complaint time = 120 2024-04-20T21:30:43.129 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.129 INFO:teuthology.orchestra.run.smithi144.stdout:[client] 2024-04-20T21:30:43.129 INFO:teuthology.orchestra.run.smithi144.stdout: rgw cache enabled = true 2024-04-20T21:30:43.129 INFO:teuthology.orchestra.run.smithi144.stdout: rgw enable ops log = true 2024-04-20T21:30:43.130 INFO:teuthology.orchestra.run.smithi144.stdout: rgw enable usage log = true 2024-04-20T21:30:43.130 INFO:teuthology.orchestra.run.smithi144.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-20T21:30:43.130 INFO:teuthology.orchestra.run.smithi144.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-20T21:30:43.130 INFO:teuthology.orchestra.run.smithi144.stdout: client mount timeout = 600 2024-04-20T21:30:43.130 INFO:teuthology.orchestra.run.smithi144.stdout: debug client = 20 2024-04-20T21:30:43.130 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-04-20T21:30:43.130 INFO:teuthology.orchestra.run.smithi144.stdout: pid file = "" 2024-04-20T21:30:43.130 INFO:teuthology.orchestra.run.smithi144.stdout: rados mon op timeout = 900 2024-04-20T21:30:43.130 INFO:teuthology.orchestra.run.smithi144.stdout: rados osd op timeout = 900 2024-04-20T21:30:43.130 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.a] 2024-04-20T21:30:43.130 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.b] 2024-04-20T21:30:43.130 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.c] 2024-04-20T21:30:43.130 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.a] 2024-04-20T21:30:43.130 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.c] 2024-04-20T21:30:43.130 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.b] 2024-04-20T21:30:43.131 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.d] 2024-04-20T21:30:43.131 INFO:teuthology.orchestra.run.smithi144.stdout:[mds] 2024-04-20T21:30:43.131 INFO:teuthology.orchestra.run.smithi144.stdout: debug mds = 20 2024-04-20T21:30:43.131 INFO:teuthology.orchestra.run.smithi144.stdout: debug mds balancer = 20 2024-04-20T21:30:43.131 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-04-20T21:30:43.131 INFO:teuthology.orchestra.run.smithi144.stdout: mds debug frag = True 2024-04-20T21:30:43.131 INFO:teuthology.orchestra.run.smithi144.stdout: mds debug scatterstat = False 2024-04-20T21:30:43.131 INFO:teuthology.orchestra.run.smithi144.stdout: mds op complaint time = 180 2024-04-20T21:30:43.131 INFO:teuthology.orchestra.run.smithi144.stdout: mds verify scatter = True 2024-04-20T21:30:43.131 INFO:teuthology.orchestra.run.smithi144.stdout: osd op complaint time = 180 2024-04-20T21:30:43.131 INFO:teuthology.orchestra.run.smithi144.stdout: rados mon op timeout = 900 2024-04-20T21:30:43.131 INFO:teuthology.orchestra.run.smithi144.stdout: rados osd op timeout = 900 2024-04-20T21:30:43.153 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-20T21:30:43.158 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-20T21:30:43.281 INFO:teuthology.orchestra.run.smithi086.stdout:[global] 2024-04-20T21:30:43.281 INFO:teuthology.orchestra.run.smithi086.stdout: chdir = "" 2024-04-20T21:30:43.281 INFO:teuthology.orchestra.run.smithi086.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-20T21:30:43.281 INFO:teuthology.orchestra.run.smithi086.stdout: auth supported = cephx 2024-04-20T21:30:43.281 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.282 INFO:teuthology.orchestra.run.smithi086.stdout: filestore xattr use omap = true 2024-04-20T21:30:43.282 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.282 INFO:teuthology.orchestra.run.smithi086.stdout: mon clock drift allowed = 1.000 2024-04-20T21:30:43.282 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.282 INFO:teuthology.orchestra.run.smithi086.stdout: osd crush chooseleaf type = 0 2024-04-20T21:30:43.282 INFO:teuthology.orchestra.run.smithi086.stdout: auth debug = true 2024-04-20T21:30:43.282 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.282 INFO:teuthology.orchestra.run.smithi086.stdout: ms die on old message = true 2024-04-20T21:30:43.282 INFO:teuthology.orchestra.run.smithi086.stdout: ms die on bug = true 2024-04-20T21:30:43.282 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.282 INFO:teuthology.orchestra.run.smithi086.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-20T21:30:43.282 INFO:teuthology.orchestra.run.smithi086.stdout: mon pg warn max object skew = 0 2024-04-20T21:30:43.282 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.283 INFO:teuthology.orchestra.run.smithi086.stdout: # disable pg_autoscaler by default for new pools 2024-04-20T21:30:43.283 INFO:teuthology.orchestra.run.smithi086.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-20T21:30:43.283 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.283 INFO:teuthology.orchestra.run.smithi086.stdout: osd pool default size = 2 2024-04-20T21:30:43.283 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.283 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd allow primary affinity = true 2024-04-20T21:30:43.283 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd allow pg remap = true 2024-04-20T21:30:43.283 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on legacy crush tunables = false 2024-04-20T21:30:43.283 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on crush straw calc version zero = false 2024-04-20T21:30:43.283 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on no sortbitwise = false 2024-04-20T21:30:43.283 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on osd down out interval zero = false 2024-04-20T21:30:43.283 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on too few osds = false 2024-04-20T21:30:43.283 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-20T21:30:43.283 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-20T21:30:43.283 INFO:teuthology.orchestra.run.smithi086.stdout: mon_allow_pool_size_one = true 2024-04-20T21:30:43.283 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.283 INFO:teuthology.orchestra.run.smithi086.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-20T21:30:43.284 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.284 INFO:teuthology.orchestra.run.smithi086.stdout: osd default data pool replay window = 5 2024-04-20T21:30:43.284 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.284 INFO:teuthology.orchestra.run.smithi086.stdout: mon allow pool delete = true 2024-04-20T21:30:43.284 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.284 INFO:teuthology.orchestra.run.smithi086.stdout: mon cluster log file level = debug 2024-04-20T21:30:43.284 INFO:teuthology.orchestra.run.smithi086.stdout: debug asserts on shutdown = true 2024-04-20T21:30:43.284 INFO:teuthology.orchestra.run.smithi086.stdout: mon health detail to clog = false 2024-04-20T21:30:43.284 INFO:teuthology.orchestra.run.smithi086.stdout: mon host = "172.21.15.86,172.21.15.144,[v2:172.21.15.144:3301,v1:172.21.15.144:6790]" 2024-04-20T21:30:43.284 INFO:teuthology.orchestra.run.smithi086.stdout: fsid = 9c2dec1d-d7ae-4a83-a812-79922808e278 2024-04-20T21:30:43.284 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.284 INFO:teuthology.orchestra.run.smithi086.stdout:[osd] 2024-04-20T21:30:43.284 INFO:teuthology.orchestra.run.smithi086.stdout: osd journal size = 100 2024-04-20T21:30:43.284 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.285 INFO:teuthology.orchestra.run.smithi086.stdout: osd scrub load threshold = 5.0 2024-04-20T21:30:43.285 INFO:teuthology.orchestra.run.smithi086.stdout: osd scrub max interval = 600 2024-04-20T21:30:43.285 INFO:teuthology.orchestra.run.smithi086.stdout: osd mclock profile = high_recovery_ops 2024-04-20T21:30:43.285 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.285 INFO:teuthology.orchestra.run.smithi086.stdout: osd recover clone overlap = true 2024-04-20T21:30:43.285 INFO:teuthology.orchestra.run.smithi086.stdout: osd recovery max chunk = 1048576 2024-04-20T21:30:43.285 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.285 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug shutdown = true 2024-04-20T21:30:43.285 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug op order = true 2024-04-20T21:30:43.285 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug verify stray on activate = true 2024-04-20T21:30:43.285 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug trim objects = true 2024-04-20T21:30:43.285 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.285 INFO:teuthology.orchestra.run.smithi086.stdout: osd open classes on start = true 2024-04-20T21:30:43.285 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug pg log writeout = true 2024-04-20T21:30:43.285 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.285 INFO:teuthology.orchestra.run.smithi086.stdout: osd deep scrub update digest min age = 30 2024-04-20T21:30:43.286 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.286 INFO:teuthology.orchestra.run.smithi086.stdout: osd map max advance = 10 2024-04-20T21:30:43.286 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.286 INFO:teuthology.orchestra.run.smithi086.stdout: journal zero on create = true 2024-04-20T21:30:43.286 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.286 INFO:teuthology.orchestra.run.smithi086.stdout: filestore ondisk finisher threads = 3 2024-04-20T21:30:43.286 INFO:teuthology.orchestra.run.smithi086.stdout: filestore apply finisher threads = 3 2024-04-20T21:30:43.286 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.286 INFO:teuthology.orchestra.run.smithi086.stdout: bdev debug aio = true 2024-04-20T21:30:43.286 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug misdirected ops = true 2024-04-20T21:30:43.286 INFO:teuthology.orchestra.run.smithi086.stdout: bdev async discard = True 2024-04-20T21:30:43.286 INFO:teuthology.orchestra.run.smithi086.stdout: bdev enable discard = True 2024-04-20T21:30:43.286 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore allocator = bitmap 2024-04-20T21:30:43.286 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore block size = 96636764160 2024-04-20T21:30:43.286 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore fsck on mount = True 2024-04-20T21:30:43.287 INFO:teuthology.orchestra.run.smithi086.stdout: debug bluefs = 1/20 2024-04-20T21:30:43.287 INFO:teuthology.orchestra.run.smithi086.stdout: debug bluestore = 1/20 2024-04-20T21:30:43.287 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-20T21:30:43.287 INFO:teuthology.orchestra.run.smithi086.stdout: debug osd = 20 2024-04-20T21:30:43.287 INFO:teuthology.orchestra.run.smithi086.stdout: debug rocksdb = 4/10 2024-04-20T21:30:43.287 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd backfillfull_ratio = 0.85 2024-04-20T21:30:43.287 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd full ratio = 0.9 2024-04-20T21:30:43.287 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd nearfull ratio = 0.8 2024-04-20T21:30:43.287 INFO:teuthology.orchestra.run.smithi086.stdout: osd failsafe full ratio = 0.95 2024-04-20T21:30:43.287 INFO:teuthology.orchestra.run.smithi086.stdout: osd objectstore = bluestore 2024-04-20T21:30:43.287 INFO:teuthology.orchestra.run.smithi086.stdout: osd op complaint time = 180 2024-04-20T21:30:43.287 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.287 INFO:teuthology.orchestra.run.smithi086.stdout:[mgr] 2024-04-20T21:30:43.287 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-20T21:30:43.287 INFO:teuthology.orchestra.run.smithi086.stdout: debug mgr = 20 2024-04-20T21:30:43.288 INFO:teuthology.orchestra.run.smithi086.stdout: debug mon = 20 2024-04-20T21:30:43.288 INFO:teuthology.orchestra.run.smithi086.stdout: debug auth = 20 2024-04-20T21:30:43.288 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min pgs per osd = 4 2024-04-20T21:30:43.288 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min bytes per osd = 10 2024-04-20T21:30:43.288 INFO:teuthology.orchestra.run.smithi086.stdout: mgr/telemetry/nag = false 2024-04-20T21:30:43.288 INFO:teuthology.orchestra.run.smithi086.stdout: debug client = 20 2024-04-20T21:30:43.288 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.288 INFO:teuthology.orchestra.run.smithi086.stdout:[mon] 2024-04-20T21:30:43.288 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-20T21:30:43.288 INFO:teuthology.orchestra.run.smithi086.stdout: debug mon = 20 2024-04-20T21:30:43.288 INFO:teuthology.orchestra.run.smithi086.stdout: debug paxos = 20 2024-04-20T21:30:43.288 INFO:teuthology.orchestra.run.smithi086.stdout: debug auth = 20 2024-04-20T21:30:43.288 INFO:teuthology.orchestra.run.smithi086.stdout: mon data avail warn = 5 2024-04-20T21:30:43.288 INFO:teuthology.orchestra.run.smithi086.stdout: mon mgr mkfs grace = 240 2024-04-20T21:30:43.288 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min pgs per osd = 4 2024-04-20T21:30:43.288 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd reporter subtree level = osd 2024-04-20T21:30:43.289 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd prime pg temp = true 2024-04-20T21:30:43.289 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min bytes per osd = 10 2024-04-20T21:30:43.289 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.289 INFO:teuthology.orchestra.run.smithi086.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-20T21:30:43.289 INFO:teuthology.orchestra.run.smithi086.stdout: auth mon ticket ttl = 660 # 11m 2024-04-20T21:30:43.289 INFO:teuthology.orchestra.run.smithi086.stdout: auth service ticket ttl = 240 # 4m 2024-04-20T21:30:43.289 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.289 INFO:teuthology.orchestra.run.smithi086.stdout: # don't complain about insecure global_id in the test suite 2024-04-20T21:30:43.289 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-20T21:30:43.289 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-20T21:30:43.289 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.289 INFO:teuthology.orchestra.run.smithi086.stdout: # 1m isn't quite enough 2024-04-20T21:30:43.289 INFO:teuthology.orchestra.run.smithi086.stdout: mon_down_mkfs_grace = 2m 2024-04-20T21:30:43.289 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.289 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_filestore_osds = false 2024-04-20T21:30:43.290 INFO:teuthology.orchestra.run.smithi086.stdout: mon down mkfs grace = 300 2024-04-20T21:30:43.290 INFO:teuthology.orchestra.run.smithi086.stdout: mon op complaint time = 120 2024-04-20T21:30:43.290 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:43.290 INFO:teuthology.orchestra.run.smithi086.stdout:[client] 2024-04-20T21:30:43.290 INFO:teuthology.orchestra.run.smithi086.stdout: rgw cache enabled = true 2024-04-20T21:30:43.290 INFO:teuthology.orchestra.run.smithi086.stdout: rgw enable ops log = true 2024-04-20T21:30:43.290 INFO:teuthology.orchestra.run.smithi086.stdout: rgw enable usage log = true 2024-04-20T21:30:43.290 INFO:teuthology.orchestra.run.smithi086.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-20T21:30:43.290 INFO:teuthology.orchestra.run.smithi086.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-20T21:30:43.290 INFO:teuthology.orchestra.run.smithi086.stdout: client mount timeout = 600 2024-04-20T21:30:43.290 INFO:teuthology.orchestra.run.smithi086.stdout: debug client = 20 2024-04-20T21:30:43.290 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-20T21:30:43.290 INFO:teuthology.orchestra.run.smithi086.stdout: pid file = "" 2024-04-20T21:30:43.290 INFO:teuthology.orchestra.run.smithi086.stdout: rados mon op timeout = 900 2024-04-20T21:30:43.291 INFO:teuthology.orchestra.run.smithi086.stdout: rados osd op timeout = 900 2024-04-20T21:30:43.291 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.a] 2024-04-20T21:30:43.291 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.b] 2024-04-20T21:30:43.291 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.c] 2024-04-20T21:30:43.291 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.a] 2024-04-20T21:30:43.291 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.c] 2024-04-20T21:30:43.291 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.b] 2024-04-20T21:30:43.291 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.d] 2024-04-20T21:30:43.291 INFO:teuthology.orchestra.run.smithi086.stdout:[mds] 2024-04-20T21:30:43.291 INFO:teuthology.orchestra.run.smithi086.stdout: debug mds = 20 2024-04-20T21:30:43.291 INFO:teuthology.orchestra.run.smithi086.stdout: debug mds balancer = 20 2024-04-20T21:30:43.291 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2024-04-20T21:30:43.291 INFO:teuthology.orchestra.run.smithi086.stdout: mds debug frag = True 2024-04-20T21:30:43.291 INFO:teuthology.orchestra.run.smithi086.stdout: mds debug scatterstat = True 2024-04-20T21:30:43.291 INFO:teuthology.orchestra.run.smithi086.stdout: mds op complaint time = 180 2024-04-20T21:30:43.292 INFO:teuthology.orchestra.run.smithi086.stdout: mds verify scatter = True 2024-04-20T21:30:43.292 INFO:teuthology.orchestra.run.smithi086.stdout: osd op complaint time = 180 2024-04-20T21:30:43.292 INFO:teuthology.orchestra.run.smithi086.stdout: rados mon op timeout = 900 2024-04-20T21:30:43.292 INFO:teuthology.orchestra.run.smithi086.stdout: rados osd op timeout = 900 2024-04-20T21:30:43.292 INFO:teuthology.orchestra.run.smithi144.stdout:[global] 2024-04-20T21:30:43.292 INFO:teuthology.orchestra.run.smithi144.stdout: chdir = "" 2024-04-20T21:30:43.292 INFO:teuthology.orchestra.run.smithi144.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-20T21:30:43.292 INFO:teuthology.orchestra.run.smithi144.stdout: auth supported = cephx 2024-04-20T21:30:43.292 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.293 INFO:teuthology.orchestra.run.smithi144.stdout: filestore xattr use omap = true 2024-04-20T21:30:43.293 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.293 INFO:teuthology.orchestra.run.smithi144.stdout: mon clock drift allowed = 1.000 2024-04-20T21:30:43.293 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.293 INFO:teuthology.orchestra.run.smithi144.stdout: osd crush chooseleaf type = 0 2024-04-20T21:30:43.293 INFO:teuthology.orchestra.run.smithi144.stdout: auth debug = true 2024-04-20T21:30:43.293 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.293 INFO:teuthology.orchestra.run.smithi144.stdout: ms die on old message = true 2024-04-20T21:30:43.293 INFO:teuthology.orchestra.run.smithi144.stdout: ms die on bug = true 2024-04-20T21:30:43.293 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.293 INFO:teuthology.orchestra.run.smithi144.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-20T21:30:43.293 INFO:teuthology.orchestra.run.smithi144.stdout: mon pg warn max object skew = 0 2024-04-20T21:30:43.293 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.293 INFO:teuthology.orchestra.run.smithi144.stdout: # disable pg_autoscaler by default for new pools 2024-04-20T21:30:43.293 INFO:teuthology.orchestra.run.smithi144.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-20T21:30:43.294 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.294 INFO:teuthology.orchestra.run.smithi144.stdout: osd pool default size = 2 2024-04-20T21:30:43.294 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.294 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd allow primary affinity = true 2024-04-20T21:30:43.294 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd allow pg remap = true 2024-04-20T21:30:43.294 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on legacy crush tunables = false 2024-04-20T21:30:43.294 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on crush straw calc version zero = false 2024-04-20T21:30:43.294 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on no sortbitwise = false 2024-04-20T21:30:43.294 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on osd down out interval zero = false 2024-04-20T21:30:43.294 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on too few osds = false 2024-04-20T21:30:43.294 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-20T21:30:43.294 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-20T21:30:43.294 INFO:teuthology.orchestra.run.smithi144.stdout: mon_allow_pool_size_one = true 2024-04-20T21:30:43.295 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.295 INFO:teuthology.orchestra.run.smithi144.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-20T21:30:43.295 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.295 INFO:teuthology.orchestra.run.smithi144.stdout: osd default data pool replay window = 5 2024-04-20T21:30:43.295 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.295 INFO:teuthology.orchestra.run.smithi144.stdout: mon allow pool delete = true 2024-04-20T21:30:43.295 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.295 INFO:teuthology.orchestra.run.smithi144.stdout: mon cluster log file level = debug 2024-04-20T21:30:43.295 INFO:teuthology.orchestra.run.smithi144.stdout: debug asserts on shutdown = true 2024-04-20T21:30:43.295 INFO:teuthology.orchestra.run.smithi144.stdout: mon health detail to clog = false 2024-04-20T21:30:43.295 INFO:teuthology.orchestra.run.smithi144.stdout: mon host = "172.21.15.86,172.21.15.144,[v2:172.21.15.144:3301,v1:172.21.15.144:6790]" 2024-04-20T21:30:43.295 INFO:teuthology.orchestra.run.smithi144.stdout: fsid = 9c2dec1d-d7ae-4a83-a812-79922808e278 2024-04-20T21:30:43.295 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.295 INFO:teuthology.orchestra.run.smithi144.stdout:[osd] 2024-04-20T21:30:43.296 INFO:teuthology.orchestra.run.smithi144.stdout: osd journal size = 100 2024-04-20T21:30:43.296 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.296 INFO:teuthology.orchestra.run.smithi144.stdout: osd scrub load threshold = 5.0 2024-04-20T21:30:43.296 INFO:teuthology.orchestra.run.smithi144.stdout: osd scrub max interval = 600 2024-04-20T21:30:43.296 INFO:teuthology.orchestra.run.smithi144.stdout: osd mclock profile = high_recovery_ops 2024-04-20T21:30:43.296 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.296 INFO:teuthology.orchestra.run.smithi144.stdout: osd recover clone overlap = true 2024-04-20T21:30:43.296 INFO:teuthology.orchestra.run.smithi144.stdout: osd recovery max chunk = 1048576 2024-04-20T21:30:43.296 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.296 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug shutdown = true 2024-04-20T21:30:43.296 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug op order = true 2024-04-20T21:30:43.296 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug verify stray on activate = true 2024-04-20T21:30:43.296 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug trim objects = true 2024-04-20T21:30:43.296 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.297 INFO:teuthology.orchestra.run.smithi144.stdout: osd open classes on start = true 2024-04-20T21:30:43.297 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug pg log writeout = true 2024-04-20T21:30:43.297 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.297 INFO:teuthology.orchestra.run.smithi144.stdout: osd deep scrub update digest min age = 30 2024-04-20T21:30:43.297 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.297 INFO:teuthology.orchestra.run.smithi144.stdout: osd map max advance = 10 2024-04-20T21:30:43.297 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.297 INFO:teuthology.orchestra.run.smithi144.stdout: journal zero on create = true 2024-04-20T21:30:43.297 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.297 INFO:teuthology.orchestra.run.smithi144.stdout: filestore ondisk finisher threads = 3 2024-04-20T21:30:43.297 INFO:teuthology.orchestra.run.smithi144.stdout: filestore apply finisher threads = 3 2024-04-20T21:30:43.297 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.297 INFO:teuthology.orchestra.run.smithi144.stdout: bdev debug aio = true 2024-04-20T21:30:43.297 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug misdirected ops = true 2024-04-20T21:30:43.298 INFO:teuthology.orchestra.run.smithi144.stdout: bdev async discard = True 2024-04-20T21:30:43.298 INFO:teuthology.orchestra.run.smithi144.stdout: bdev enable discard = True 2024-04-20T21:30:43.298 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore allocator = bitmap 2024-04-20T21:30:43.298 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore block size = 96636764160 2024-04-20T21:30:43.298 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore fsck on mount = True 2024-04-20T21:30:43.298 INFO:teuthology.orchestra.run.smithi144.stdout: debug bluefs = 1/20 2024-04-20T21:30:43.298 INFO:teuthology.orchestra.run.smithi144.stdout: debug bluestore = 1/20 2024-04-20T21:30:43.298 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-04-20T21:30:43.298 INFO:teuthology.orchestra.run.smithi144.stdout: debug osd = 20 2024-04-20T21:30:43.298 INFO:teuthology.orchestra.run.smithi144.stdout: debug rocksdb = 4/10 2024-04-20T21:30:43.298 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd backfillfull_ratio = 0.85 2024-04-20T21:30:43.298 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd full ratio = 0.9 2024-04-20T21:30:43.298 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd nearfull ratio = 0.8 2024-04-20T21:30:43.298 INFO:teuthology.orchestra.run.smithi144.stdout: osd failsafe full ratio = 0.95 2024-04-20T21:30:43.298 INFO:teuthology.orchestra.run.smithi144.stdout: osd objectstore = bluestore 2024-04-20T21:30:43.299 INFO:teuthology.orchestra.run.smithi144.stdout: osd op complaint time = 180 2024-04-20T21:30:43.299 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.299 INFO:teuthology.orchestra.run.smithi144.stdout:[mgr] 2024-04-20T21:30:43.299 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-04-20T21:30:43.299 INFO:teuthology.orchestra.run.smithi144.stdout: debug mgr = 20 2024-04-20T21:30:43.299 INFO:teuthology.orchestra.run.smithi144.stdout: debug mon = 20 2024-04-20T21:30:43.299 INFO:teuthology.orchestra.run.smithi144.stdout: debug auth = 20 2024-04-20T21:30:43.299 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min pgs per osd = 4 2024-04-20T21:30:43.299 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min bytes per osd = 10 2024-04-20T21:30:43.299 INFO:teuthology.orchestra.run.smithi144.stdout: mgr/telemetry/nag = false 2024-04-20T21:30:43.299 INFO:teuthology.orchestra.run.smithi144.stdout: debug client = 20 2024-04-20T21:30:43.301 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.301 INFO:teuthology.orchestra.run.smithi144.stdout:[mon] 2024-04-20T21:30:43.301 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-04-20T21:30:43.301 INFO:teuthology.orchestra.run.smithi144.stdout: debug mon = 20 2024-04-20T21:30:43.301 INFO:teuthology.orchestra.run.smithi144.stdout: debug paxos = 20 2024-04-20T21:30:43.301 INFO:teuthology.orchestra.run.smithi144.stdout: debug auth = 20 2024-04-20T21:30:43.301 INFO:teuthology.orchestra.run.smithi144.stdout: mon data avail warn = 5 2024-04-20T21:30:43.301 INFO:teuthology.orchestra.run.smithi144.stdout: mon mgr mkfs grace = 240 2024-04-20T21:30:43.301 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min pgs per osd = 4 2024-04-20T21:30:43.301 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd reporter subtree level = osd 2024-04-20T21:30:43.301 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd prime pg temp = true 2024-04-20T21:30:43.301 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min bytes per osd = 10 2024-04-20T21:30:43.302 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.302 INFO:teuthology.orchestra.run.smithi144.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-20T21:30:43.302 INFO:teuthology.orchestra.run.smithi144.stdout: auth mon ticket ttl = 660 # 11m 2024-04-20T21:30:43.302 INFO:teuthology.orchestra.run.smithi144.stdout: auth service ticket ttl = 240 # 4m 2024-04-20T21:30:43.302 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.302 INFO:teuthology.orchestra.run.smithi144.stdout: # don't complain about insecure global_id in the test suite 2024-04-20T21:30:43.302 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-20T21:30:43.302 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-20T21:30:43.302 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.302 INFO:teuthology.orchestra.run.smithi144.stdout: # 1m isn't quite enough 2024-04-20T21:30:43.302 INFO:teuthology.orchestra.run.smithi144.stdout: mon_down_mkfs_grace = 2m 2024-04-20T21:30:43.302 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.302 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_filestore_osds = false 2024-04-20T21:30:43.302 INFO:teuthology.orchestra.run.smithi144.stdout: mon down mkfs grace = 300 2024-04-20T21:30:43.302 INFO:teuthology.orchestra.run.smithi144.stdout: mon op complaint time = 120 2024-04-20T21:30:43.303 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-04-20T21:30:43.303 INFO:teuthology.orchestra.run.smithi144.stdout:[client] 2024-04-20T21:30:43.303 INFO:teuthology.orchestra.run.smithi144.stdout: rgw cache enabled = true 2024-04-20T21:30:43.303 INFO:teuthology.orchestra.run.smithi144.stdout: rgw enable ops log = true 2024-04-20T21:30:43.303 INFO:teuthology.orchestra.run.smithi144.stdout: rgw enable usage log = true 2024-04-20T21:30:43.303 INFO:teuthology.orchestra.run.smithi144.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-20T21:30:43.303 INFO:teuthology.orchestra.run.smithi144.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-20T21:30:43.303 INFO:teuthology.orchestra.run.smithi144.stdout: client mount timeout = 600 2024-04-20T21:30:43.303 INFO:teuthology.orchestra.run.smithi144.stdout: debug client = 20 2024-04-20T21:30:43.303 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-04-20T21:30:43.303 INFO:teuthology.orchestra.run.smithi144.stdout: pid file = "" 2024-04-20T21:30:43.303 INFO:teuthology.orchestra.run.smithi144.stdout: rados mon op timeout = 900 2024-04-20T21:30:43.303 INFO:teuthology.orchestra.run.smithi144.stdout: rados osd op timeout = 900 2024-04-20T21:30:43.303 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.a] 2024-04-20T21:30:43.303 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.b] 2024-04-20T21:30:43.303 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.c] 2024-04-20T21:30:43.304 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.a] 2024-04-20T21:30:43.304 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.c] 2024-04-20T21:30:43.304 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.b] 2024-04-20T21:30:43.304 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.d] 2024-04-20T21:30:43.304 INFO:teuthology.orchestra.run.smithi144.stdout:[mds] 2024-04-20T21:30:43.304 INFO:teuthology.orchestra.run.smithi144.stdout: debug mds = 20 2024-04-20T21:30:43.304 INFO:teuthology.orchestra.run.smithi144.stdout: debug mds balancer = 20 2024-04-20T21:30:43.304 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2024-04-20T21:30:43.304 INFO:teuthology.orchestra.run.smithi144.stdout: mds debug frag = True 2024-04-20T21:30:43.304 INFO:teuthology.orchestra.run.smithi144.stdout: mds debug scatterstat = True 2024-04-20T21:30:43.304 INFO:teuthology.orchestra.run.smithi144.stdout: mds op complaint time = 180 2024-04-20T21:30:43.304 INFO:teuthology.orchestra.run.smithi144.stdout: mds verify scatter = True 2024-04-20T21:30:43.304 INFO:teuthology.orchestra.run.smithi144.stdout: osd op complaint time = 180 2024-04-20T21:30:43.304 INFO:teuthology.orchestra.run.smithi144.stdout: rados mon op timeout = 900 2024-04-20T21:30:43.304 INFO:teuthology.orchestra.run.smithi144.stdout: rados osd op timeout = 900 2024-04-20T21:30:43.317 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-20T21:30:43.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.519+0000 7fee7be00640 1 Processor -- start 2024-04-20T21:30:43.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.519+0000 7fee7be00640 1 -- start start 2024-04-20T21:30:43.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.521+0000 7fee7be00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fee7c0b3ba0 0x7fee7c06eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:43.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.521+0000 7fee7be00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee7c06f3f0 0x7fee7c071ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:43.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.521+0000 7fee7be00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee7c06f800 con 0x7fee7c0cfc70 2024-04-20T21:30:43.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.521+0000 7fee7be00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fee7c15c8e0 con 0x7fee7c0b3ba0 2024-04-20T21:30:43.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.521+0000 7fee7be00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fee7c0cddb0 con 0x7fee7c06f3f0 2024-04-20T21:30:43.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.521+0000 7fee7a000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fee7c0b3ba0 0x7fee7c06eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:43.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.522+0000 7fee7a000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fee7c0b3ba0 0x7fee7c06eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.86:59154/0 (socket says 172.21.15.86:59154) 2024-04-20T21:30:43.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.522+0000 7fee7a000640 1 -- 172.21.15.86:0/2411325476 learned_addr learned my addr 172.21.15.86:0/2411325476 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:43.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.522+0000 7fee7b400640 1 --2- 172.21.15.86:0/2411325476 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee7c06f3f0 0x7fee7c071ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:43.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.523+0000 7fee7a000640 1 -- 172.21.15.86:0/2411325476 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee7c06f3f0 msgr2=0x7fee7c071ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:43.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.523+0000 7fee7a000640 1 --2- 172.21.15.86:0/2411325476 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee7c06f3f0 0x7fee7c071ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:43.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.523+0000 7fee7a000640 1 -- 172.21.15.86:0/2411325476 >> v1:172.21.15.144:6789/0 conn(0x7fee7c0cfc70 legacy=0x7fee7c0cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:43.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.523+0000 7fee7a000640 1 -- 172.21.15.86:0/2411325476 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee7c15bcd0 con 0x7fee7c0b3ba0 2024-04-20T21:30:43.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.525+0000 7fee7a000640 1 --2- 172.21.15.86:0/2411325476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee7c0b3ba0 0x7fee7c06eeb0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fee6800bb10 tx=0x7fee6802e160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce6879c42085e2de server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:43.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.525+0000 7fee79600640 1 -- 172.21.15.86:0/2411325476 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6803b070 con 0x7fee7c0b3ba0 2024-04-20T21:30:43.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.525+0000 7fee79600640 1 -- 172.21.15.86:0/2411325476 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee68036440 con 0x7fee7c0b3ba0 2024-04-20T21:30:43.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.525+0000 7fee79600640 1 -- 172.21.15.86:0/2411325476 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee68036810 con 0x7fee7c0b3ba0 2024-04-20T21:30:43.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.526+0000 7fee7be00640 1 -- 172.21.15.86:0/2411325476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee7c0b3ba0 msgr2=0x7fee7c06eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:43.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.526+0000 7fee7be00640 1 --2- 172.21.15.86:0/2411325476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee7c0b3ba0 0x7fee7c06eeb0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fee6800bb10 tx=0x7fee6802e160 comp rx=0 tx=0).stop 2024-04-20T21:30:43.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.526+0000 7fee7be00640 1 -- 172.21.15.86:0/2411325476 shutdown_connections 2024-04-20T21:30:43.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.526+0000 7fee7be00640 1 --2- 172.21.15.86:0/2411325476 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee7c06f3f0 0x7fee7c071ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:43.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.526+0000 7fee7be00640 1 --2- 172.21.15.86:0/2411325476 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee7c0b3ba0 0x7fee7c06eeb0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:43.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.526+0000 7fee7be00640 1 -- 172.21.15.86:0/2411325476 >> 172.21.15.86:0/2411325476 conn(0x7fee7c067cf0 msgr2=0x7fee7c06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:43.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.526+0000 7fee7be00640 1 -- 172.21.15.86:0/2411325476 shutdown_connections 2024-04-20T21:30:43.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.526+0000 7fee7be00640 1 -- 172.21.15.86:0/2411325476 wait complete. 2024-04-20T21:30:43.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.528+0000 7fee7be00640 1 Processor -- start 2024-04-20T21:30:43.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.528+0000 7fee7be00640 1 -- start start 2024-04-20T21:30:43.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.529+0000 7fee7be00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee7c06f3f0 0x7fee7c1f4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:43.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.529+0000 7fee7be00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee7c0b3ba0 0x7fee7c15d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:43.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.529+0000 7fee7be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee7c0cfc70 0x7fee7c1f6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:43.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.529+0000 7fee7be00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fee7c15c570 con 0x7fee7c0cfc70 2024-04-20T21:30:43.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.529+0000 7fee7be00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fee7c15c290 con 0x7fee7c0b3ba0 2024-04-20T21:30:43.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.529+0000 7fee7be00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fee7c0b3710 con 0x7fee7c06f3f0 2024-04-20T21:30:43.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.530+0000 7fee7b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee7c0cfc70 0x7fee7c1f6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:43.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.530+0000 7fee7b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee7c0cfc70 0x7fee7c1f6d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58920/0 (socket says 172.21.15.86:58920) 2024-04-20T21:30:43.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.530+0000 7fee7b400640 1 -- 172.21.15.86:0/354827187 learned_addr learned my addr 172.21.15.86:0/354827187 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:43.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.530+0000 7fee7a000640 1 --2- 172.21.15.86:0/354827187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee7c0b3ba0 0x7fee7c15d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:43.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.530+0000 7fee7aa00640 1 --2- 172.21.15.86:0/354827187 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee7c06f3f0 0x7fee7c1f4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:43.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.530+0000 7fee7b400640 1 -- 172.21.15.86:0/354827187 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee7c06f3f0 msgr2=0x7fee7c1f4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:43.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.530+0000 7fee7b400640 1 --2- 172.21.15.86:0/354827187 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee7c06f3f0 0x7fee7c1f4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:43.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.530+0000 7fee7b400640 1 -- 172.21.15.86:0/354827187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee7c0b3ba0 msgr2=0x7fee7c15d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:43.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.530+0000 7fee7b400640 1 --2- 172.21.15.86:0/354827187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee7c0b3ba0 0x7fee7c15d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:43.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.530+0000 7fee7b400640 1 -- 172.21.15.86:0/354827187 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee7c14f3e0 con 0x7fee7c0cfc70 2024-04-20T21:30:43.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.530+0000 7fee7a000640 1 --2- 172.21.15.86:0/354827187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee7c0b3ba0 0x7fee7c15d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:30:43.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.531+0000 7fee7b400640 1 --2- 172.21.15.86:0/354827187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee7c0cfc70 0x7fee7c1f6d50 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7fee7400efc0 tx=0x7fee7400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:43.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.531+0000 7fee72a00640 1 -- 172.21.15.86:0/354827187 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee74018020 con 0x7fee7c0cfc70 2024-04-20T21:30:43.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.531+0000 7fee72a00640 1 -- 172.21.15.86:0/354827187 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee74014480 con 0x7fee7c0cfc70 2024-04-20T21:30:43.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.531+0000 7fee72a00640 1 -- 172.21.15.86:0/354827187 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee74014870 con 0x7fee7c0cfc70 2024-04-20T21:30:43.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.531+0000 7fee7be00640 1 -- 172.21.15.86:0/354827187 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee7c1f7ce0 con 0x7fee7c0cfc70 2024-04-20T21:30:43.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.531+0000 7fee7be00640 1 -- 172.21.15.86:0/354827187 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee7c1f82a0 con 0x7fee7c0cfc70 2024-04-20T21:30:43.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.532+0000 7fee7be00640 1 -- 172.21.15.86:0/354827187 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee7c0cebf0 con 0x7fee7c0cfc70 2024-04-20T21:30:43.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.534+0000 7fee72a00640 1 -- 172.21.15.86:0/354827187 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fee74005d10 con 0x7fee7c0cfc70 2024-04-20T21:30:43.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.535+0000 7fee72a00640 1 --2- 172.21.15.86:0/354827187 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fee5c06c8f0 0x7fee5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:43.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.535+0000 7fee72a00640 1 -- 172.21.15.86:0/354827187 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7fee7408cea0 con 0x7fee7c0cfc70 2024-04-20T21:30:43.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.535+0000 7fee7aa00640 1 --2- 172.21.15.86:0/354827187 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fee5c06c8f0 0x7fee5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:43.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.536+0000 7fee7aa00640 1 --2- 172.21.15.86:0/354827187 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fee5c06c8f0 0x7fee5c06edb0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fee7c15dde0 tx=0x7fee6c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:43.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.539+0000 7fee72a00640 1 -- 172.21.15.86:0/354827187 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fee7c0cebf0 con 0x7fee7c0cfc70 2024-04-20T21:30:43.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:43.779+0000 7fee7be00640 1 -- 172.21.15.86:0/354827187 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fee7c04fd90 con 0x7fee7c0cfc70 2024-04-20T21:30:44.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.105+0000 7fee72a00640 1 -- 172.21.15.86:0/354827187 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v108) ==== 162+0+0 (secure 0 0 0) 0x7fee74022070 con 0x7fee7c0cfc70 2024-04-20T21:30:44.104 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-20T21:30:44.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.110+0000 7fee7be00640 1 -- 172.21.15.86:0/354827187 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fee5c06c8f0 msgr2=0x7fee5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:44.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.110+0000 7fee7be00640 1 --2- 172.21.15.86:0/354827187 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fee5c06c8f0 0x7fee5c06edb0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fee7c15dde0 tx=0x7fee6c009040 comp rx=0 tx=0).stop 2024-04-20T21:30:44.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.110+0000 7fee7be00640 1 -- 172.21.15.86:0/354827187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee7c0cfc70 msgr2=0x7fee7c1f6d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:44.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.110+0000 7fee7be00640 1 --2- 172.21.15.86:0/354827187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee7c0cfc70 0x7fee7c1f6d50 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7fee7400efc0 tx=0x7fee7400c490 comp rx=0 tx=0).stop 2024-04-20T21:30:44.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.110+0000 7fee7be00640 1 -- 172.21.15.86:0/354827187 shutdown_connections 2024-04-20T21:30:44.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.110+0000 7fee7be00640 1 --2- 172.21.15.86:0/354827187 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fee5c06c8f0 0x7fee5c06edb0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:44.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.110+0000 7fee7be00640 1 --2- 172.21.15.86:0/354827187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee7c0cfc70 0x7fee7c1f6d50 unknown :-1 s=CLOSED pgs=2057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:44.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.110+0000 7fee7be00640 1 --2- 172.21.15.86:0/354827187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fee7c0b3ba0 0x7fee7c15d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:44.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.110+0000 7fee7be00640 1 --2- 172.21.15.86:0/354827187 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fee7c06f3f0 0x7fee7c1f4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:44.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.110+0000 7fee7be00640 1 -- 172.21.15.86:0/354827187 >> 172.21.15.86:0/354827187 conn(0x7fee7c067cf0 msgr2=0x7fee7c072660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:44.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.111+0000 7fee7be00640 1 -- 172.21.15.86:0/354827187 shutdown_connections 2024-04-20T21:30:44.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.111+0000 7fee7be00640 1 -- 172.21.15.86:0/354827187 wait complete. 2024-04-20T21:30:44.133 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:30:44.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.309+0000 7fa84f800640 1 Processor -- start 2024-04-20T21:30:44.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.310+0000 7fa84f800640 1 -- start start 2024-04-20T21:30:44.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.311+0000 7fa84f800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa84815eaf0 0x7fa8480ba1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:44.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.311+0000 7fa84f800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa8480b4030 0x7fa8480d17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:44.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.311+0000 7fa84f800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa84815eed0 con 0x7fa8480d1d00 2024-04-20T21:30:44.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.311+0000 7fa84f800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa848163110 con 0x7fa8480b4030 2024-04-20T21:30:44.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.311+0000 7fa84f800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa8480d2170 con 0x7fa84815eaf0 2024-04-20T21:30:44.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.311+0000 7fa84d800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fa8480d1d00 0x7fa8480b6480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:41240/0 (socket says 172.21.15.86:41240) 2024-04-20T21:30:44.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.311+0000 7fa83fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa8480b4030 0x7fa8480d17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:44.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.311+0000 7fa84d800640 1 -- 172.21.15.86:0/528978496 learned_addr learned my addr 172.21.15.86:0/528978496 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:30:44.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.311+0000 7fa84ce00640 1 --2- 172.21.15.86:0/528978496 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa84815eaf0 0x7fa8480ba1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:44.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.312+0000 7fa83fe00640 1 -- 172.21.15.86:0/528978496 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa84815eaf0 msgr2=0x7fa8480ba1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:44.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.312+0000 7fa83fe00640 1 --2- 172.21.15.86:0/528978496 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa84815eaf0 0x7fa8480ba1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:44.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.312+0000 7fa83fe00640 1 -- 172.21.15.86:0/528978496 >> v1:172.21.15.86:6789/0 conn(0x7fa8480d1d00 legacy=0x7fa8480b6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:44.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.312+0000 7fa83fe00640 1 -- 172.21.15.86:0/528978496 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8481627e0 con 0x7fa8480b4030 2024-04-20T21:30:44.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.312+0000 7fa847e00640 1 -- 172.21.15.86:0/528978496 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2445153941 0 0) 0x7fa84815eed0 con 0x7fa8480d1d00 2024-04-20T21:30:44.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.313+0000 7fa83fe00640 1 --2- 172.21.15.86:0/528978496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8480b4030 0x7fa8480d17c0 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7fa8300097a0 tx=0x7fa83002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebe469e5923ad0f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:44.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.313+0000 7fa847e00640 1 -- 172.21.15.86:0/528978496 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa830034070 con 0x7fa8480b4030 2024-04-20T21:30:44.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.313+0000 7fa847e00640 1 -- 172.21.15.86:0/528978496 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa830030610 con 0x7fa8480b4030 2024-04-20T21:30:44.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.313+0000 7fa847e00640 1 -- 172.21.15.86:0/528978496 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8300309e0 con 0x7fa8480b4030 2024-04-20T21:30:44.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.314+0000 7fa84f800640 1 -- 172.21.15.86:0/528978496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8480b4030 msgr2=0x7fa8480d17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:44.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.314+0000 7fa84f800640 1 --2- 172.21.15.86:0/528978496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8480b4030 0x7fa8480d17c0 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7fa8300097a0 tx=0x7fa83002b960 comp rx=0 tx=0).stop 2024-04-20T21:30:44.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.314+0000 7fa84f800640 1 -- 172.21.15.86:0/528978496 shutdown_connections 2024-04-20T21:30:44.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.314+0000 7fa84f800640 1 --2- 172.21.15.86:0/528978496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8480b4030 0x7fa8480d17c0 unknown :-1 s=CLOSED pgs=2059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:44.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.314+0000 7fa84f800640 1 --2- 172.21.15.86:0/528978496 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa84815eaf0 0x7fa8480ba1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:44.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.314+0000 7fa84f800640 1 -- 172.21.15.86:0/528978496 >> 172.21.15.86:0/528978496 conn(0x7fa8480678a0 msgr2=0x7fa848067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:44.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.314+0000 7fa84f800640 1 -- 172.21.15.86:0/528978496 shutdown_connections 2024-04-20T21:30:44.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.314+0000 7fa84f800640 1 -- 172.21.15.86:0/528978496 wait complete. 2024-04-20T21:30:44.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.317+0000 7fa84f800640 1 Processor -- start 2024-04-20T21:30:44.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.317+0000 7fa84f800640 1 -- start start 2024-04-20T21:30:44.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.318+0000 7fa84f800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8480b4030 0x7fa848159270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:44.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.319+0000 7fa84f800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8480d1d00 0x7fa848160960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:44.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.319+0000 7fa84f800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa84815eaf0 0x7fa84815b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:44.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.319+0000 7fa84f800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa848162da0 con 0x7fa8480d1d00 2024-04-20T21:30:44.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.319+0000 7fa84f800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa8480b7a00 con 0x7fa8480b4030 2024-04-20T21:30:44.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.319+0000 7fa84f800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa8480d3d40 con 0x7fa84815eaf0 2024-04-20T21:30:44.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.319+0000 7fa83fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8480d1d00 0x7fa848160960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:44.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.319+0000 7fa83fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8480d1d00 0x7fa848160960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58934/0 (socket says 172.21.15.86:58934) 2024-04-20T21:30:44.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.319+0000 7fa83fe00640 1 -- 172.21.15.86:0/1946982859 learned_addr learned my addr 172.21.15.86:0/1946982859 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:44.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.319+0000 7fa83fe00640 1 -- 172.21.15.86:0/1946982859 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa84815eaf0 msgr2=0x7fa84815b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:44.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.319+0000 7fa84ce00640 1 --2- 172.21.15.86:0/1946982859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8480b4030 0x7fa848159270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:44.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.319+0000 7fa83fe00640 1 --2- 172.21.15.86:0/1946982859 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa84815eaf0 0x7fa84815b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:44.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.319+0000 7fa83fe00640 1 -- 172.21.15.86:0/1946982859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8480b4030 msgr2=0x7fa848159270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:44.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.319+0000 7fa83fe00640 1 --2- 172.21.15.86:0/1946982859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8480b4030 0x7fa848159270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:44.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.319+0000 7fa83fe00640 1 -- 172.21.15.86:0/1946982859 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8481f4cc0 con 0x7fa8480d1d00 2024-04-20T21:30:44.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.320+0000 7fa83fe00640 1 --2- 172.21.15.86:0/1946982859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8480d1d00 0x7fa848160960 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7fa83002bf70 tx=0x7fa8300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:44.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.320+0000 7fa845600640 1 -- 172.21.15.86:0/1946982859 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa830034040 con 0x7fa8480d1d00 2024-04-20T21:30:44.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.320+0000 7fa845600640 1 -- 172.21.15.86:0/1946982859 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa830039a10 con 0x7fa8480d1d00 2024-04-20T21:30:44.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.320+0000 7fa845600640 1 -- 172.21.15.86:0/1946982859 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa830039e20 con 0x7fa8480d1d00 2024-04-20T21:30:44.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.320+0000 7fa84f800640 1 -- 172.21.15.86:0/1946982859 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8481f4fa0 con 0x7fa8480d1d00 2024-04-20T21:30:44.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.320+0000 7fa84f800640 1 -- 172.21.15.86:0/1946982859 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8481f5560 con 0x7fa8480d1d00 2024-04-20T21:30:44.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.322+0000 7fa84f800640 1 -- 172.21.15.86:0/1946982859 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8480689e0 con 0x7fa8480d1d00 2024-04-20T21:30:44.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.322+0000 7fa845600640 1 -- 172.21.15.86:0/1946982859 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa830037070 con 0x7fa8480d1d00 2024-04-20T21:30:44.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.323+0000 7fa845600640 1 --2- 172.21.15.86:0/1946982859 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa82406c8f0 0x7fa82406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:44.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.324+0000 7fa845600640 1 -- 172.21.15.86:0/1946982859 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7fa8300add70 con 0x7fa8480d1d00 2024-04-20T21:30:44.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.324+0000 7fa84ce00640 1 --2- 172.21.15.86:0/1946982859 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa82406c8f0 0x7fa82406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:44.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.324+0000 7fa84ce00640 1 --2- 172.21.15.86:0/1946982859 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa82406c8f0 0x7fa82406edb0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fa838006fd0 tx=0x7fa838008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:44.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.328+0000 7fa845600640 1 -- 172.21.15.86:0/1946982859 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa83007c010 con 0x7fa8480d1d00 2024-04-20T21:30:44.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.562+0000 7fa84f800640 1 -- 172.21.15.86:0/1946982859 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa84806dda0 con 0x7fa8480d1d00 2024-04-20T21:30:44.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.563+0000 7fa845600640 1 -- 172.21.15.86:0/1946982859 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) ==== 95+0+5205 (secure 0 0 0) 0x7fa83007b910 con 0x7fa8480d1d00 2024-04-20T21:30:44.562 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:44.562 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":109,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6253,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3627230007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3627230007},{"type":"v1","addr":"172.21.15.144:6833","nonce":3627230007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":6256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2498297133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2498297133},{"type":"v1","addr":"172.21.15.144:6835","nonce":2498297133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":9228,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/1120740475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1120740475},{"type":"v1","addr":"172.21.15.86:6836","nonce":1120740475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:29:43.769581+0000","modified":"2024-04-20T21:30:44.109975+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9792},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9792":{"gid":9792,"name":"a","rank":0,"incarnation":109,"state":"up:replay","state_seq":1,"addr":"172.21.15.86:6837/2019839930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2019839930},{"type":"v1","addr":"172.21.15.86:6837","nonce":2019839930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-04-20T21:30:44.563 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 109 2024-04-20T21:30:44.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.566+0000 7fa84f800640 1 -- 172.21.15.86:0/1946982859 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa82406c8f0 msgr2=0x7fa82406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:44.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.566+0000 7fa84f800640 1 --2- 172.21.15.86:0/1946982859 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa82406c8f0 0x7fa82406edb0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fa838006fd0 tx=0x7fa838008040 comp rx=0 tx=0).stop 2024-04-20T21:30:44.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.566+0000 7fa84f800640 1 -- 172.21.15.86:0/1946982859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8480d1d00 msgr2=0x7fa848160960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:44.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.566+0000 7fa84f800640 1 --2- 172.21.15.86:0/1946982859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8480d1d00 0x7fa848160960 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7fa83002bf70 tx=0x7fa8300046c0 comp rx=0 tx=0).stop 2024-04-20T21:30:44.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.566+0000 7fa84f800640 1 -- 172.21.15.86:0/1946982859 shutdown_connections 2024-04-20T21:30:44.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.566+0000 7fa84f800640 1 --2- 172.21.15.86:0/1946982859 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa82406c8f0 0x7fa82406edb0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:44.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.566+0000 7fa84f800640 1 --2- 172.21.15.86:0/1946982859 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa84815eaf0 0x7fa84815b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:44.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.566+0000 7fa84f800640 1 --2- 172.21.15.86:0/1946982859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8480d1d00 0x7fa848160960 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:44.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.566+0000 7fa84f800640 1 --2- 172.21.15.86:0/1946982859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8480b4030 0x7fa848159270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:44.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.566+0000 7fa84f800640 1 -- 172.21.15.86:0/1946982859 >> 172.21.15.86:0/1946982859 conn(0x7fa8480678a0 msgr2=0x7fa8480d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:44.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.566+0000 7fa84f800640 1 -- 172.21.15.86:0/1946982859 shutdown_connections 2024-04-20T21:30:44.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:44.566+0000 7fa84f800640 1 -- 172.21.15.86:0/1946982859 wait complete. 2024-04-20T21:30:44.985 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 109, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T21:29:43.769581+0000', 'modified': '2024-04-20T21:30:44.109975+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 116, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9792}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9792': {'gid': 9792, 'name': 'a', 'rank': 0, 'incarnation': 109, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.86:6837/2019839930', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6835', 'nonce': 2019839930}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 2019839930}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-20T21:30:44.986 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9792:up:replay 2024-04-20T21:30:45.986 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:30:46.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.196+0000 7f98cea00640 1 Processor -- start 2024-04-20T21:30:46.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.196+0000 7f98cea00640 1 -- start start 2024-04-20T21:30:46.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.198+0000 7f98cea00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98c8144e90 0x7f98c80b4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:46.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.198+0000 7f98cea00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98c80cdb50 con 0x7f98c80cd170 2024-04-20T21:30:46.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.198+0000 7f98cea00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98c80b3e50 con 0x7f98c806f450 2024-04-20T21:30:46.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.198+0000 7f98cea00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f98c8145270 con 0x7f98c8144e90 2024-04-20T21:30:46.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.199+0000 7f98c7400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f98c80cd170 0x7f98c806ed40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57070/0 (socket says 172.21.15.86:57070) 2024-04-20T21:30:46.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.199+0000 7f98c7400640 1 -- 172.21.15.86:0/2262052674 learned_addr learned my addr 172.21.15.86:0/2262052674 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:30:46.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.199+0000 7f98c7e00640 1 --2- 172.21.15.86:0/2262052674 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98c8144e90 0x7f98c80b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:46.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.199+0000 7f98c6a00640 1 -- 172.21.15.86:0/2262052674 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 785145341 0 0) 0x7f98c80cdb50 con 0x7f98c80cd170 2024-04-20T21:30:46.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.199+0000 7f98c6a00640 1 -- 172.21.15.86:0/2262052674 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f98ac005730 con 0x7f98c80cd170 2024-04-20T21:30:46.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.199+0000 7f98c6a00640 1 -- 172.21.15.86:0/2262052674 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1065957419 0 0) 0x7f98ac005730 con 0x7f98c80cd170 2024-04-20T21:30:46.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.200+0000 7f98c6a00640 1 -- 172.21.15.86:0/2262052674 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98c8144e90 msgr2=0x7f98c80b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:46.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.200+0000 7f98c6a00640 1 --2- 172.21.15.86:0/2262052674 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98c8144e90 0x7f98c80b4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:46.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.200+0000 7f98c6a00640 1 -- 172.21.15.86:0/2262052674 >> v1:172.21.15.144:6789/0 conn(0x7f98c806f450 legacy=0x7f98c815bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:46.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.200+0000 7f98c6a00640 1 -- 172.21.15.86:0/2262052674 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98c80b4130 con 0x7f98c80cd170 2024-04-20T21:30:46.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.200+0000 7f98c6a00640 1 -- 172.21.15.86:0/2262052674 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f98b4001670 con 0x7f98c80cd170 2024-04-20T21:30:46.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.200+0000 7f98c6a00640 1 -- 172.21.15.86:0/2262052674 >> v1:172.21.15.86:6789/0 conn(0x7f98c80cd170 legacy=0x7f98c806ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:46.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.200+0000 7f98c6a00640 1 --2- 172.21.15.86:0/2262052674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98ac005bf0 0x7f98ac006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:46.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.200+0000 7f98c6a00640 1 --2- 172.21.15.86:0/2262052674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98ac006550 0x7f98ac006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:46.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.200+0000 7f98c6a00640 1 --2- 172.21.15.86:0/2262052674 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98c8144e90 0x7f98ac007250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:46.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.200+0000 7f98c6a00640 1 -- 172.21.15.86:0/2262052674 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f98c8145270 con 0x7f98ac006550 2024-04-20T21:30:46.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.200+0000 7f98c6a00640 1 -- 172.21.15.86:0/2262052674 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f98ac00ffd0 con 0x7f98ac005bf0 2024-04-20T21:30:46.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.200+0000 7f98c6a00640 1 -- 172.21.15.86:0/2262052674 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f98ac010150 con 0x7f98c8144e90 2024-04-20T21:30:46.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.200+0000 7f98c7400640 1 --2- 172.21.15.86:0/2262052674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98ac006550 0x7f98ac006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:46.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.200+0000 7f98c7e00640 1 --2- 172.21.15.86:0/2262052674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98ac005bf0 0x7f98ac006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:46.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.200+0000 7f98c7400640 1 -- 172.21.15.86:0/2262052674 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98c8144e90 msgr2=0x7f98ac007250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:46.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.200+0000 7f98cca00640 1 --2- 172.21.15.86:0/2262052674 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98c8144e90 0x7f98ac007250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:46.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.200+0000 7f98c7400640 1 --2- 172.21.15.86:0/2262052674 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98c8144e90 0x7f98ac007250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:46.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.200+0000 7f98c7400640 1 -- 172.21.15.86:0/2262052674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98ac005bf0 msgr2=0x7f98ac006010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:46.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.200+0000 7f98c7400640 1 --2- 172.21.15.86:0/2262052674 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98ac005bf0 0x7f98ac006010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:46.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.201+0000 7f98c7400640 1 -- 172.21.15.86:0/2262052674 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98ac005730 con 0x7f98ac006550 2024-04-20T21:30:46.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.202+0000 7f98c7400640 1 --2- 172.21.15.86:0/2262052674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98ac006550 0x7f98ac006a10 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f98b400c240 tx=0x7f98ac007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=418bfbde145ab201 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:46.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.202+0000 7f98c6a00640 1 -- 172.21.15.86:0/2262052674 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b4007cf0 con 0x7f98ac006550 2024-04-20T21:30:46.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.202+0000 7f98c6a00640 1 -- 172.21.15.86:0/2262052674 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98b4030070 con 0x7f98ac006550 2024-04-20T21:30:46.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.202+0000 7f98c6a00640 1 -- 172.21.15.86:0/2262052674 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b4005d30 con 0x7f98ac006550 2024-04-20T21:30:46.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.202+0000 7f98cea00640 1 -- 172.21.15.86:0/2262052674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98ac006550 msgr2=0x7f98ac006a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:46.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.202+0000 7f98cea00640 1 --2- 172.21.15.86:0/2262052674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98ac006550 0x7f98ac006a10 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f98b400c240 tx=0x7f98ac007a40 comp rx=0 tx=0).stop 2024-04-20T21:30:46.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.202+0000 7f98cca00640 1 -- 172.21.15.86:0/2262052674 reap_dead start 2024-04-20T21:30:46.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.203+0000 7f98cea00640 1 -- 172.21.15.86:0/2262052674 shutdown_connections 2024-04-20T21:30:46.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.203+0000 7f98cea00640 1 -- 172.21.15.86:0/2262052674 >> 172.21.15.86:0/2262052674 conn(0x7f98c80b2360 msgr2=0x7f98c80b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:46.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.203+0000 7f98cea00640 1 -- 172.21.15.86:0/2262052674 shutdown_connections 2024-04-20T21:30:46.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.203+0000 7f98cea00640 1 -- 172.21.15.86:0/2262052674 wait complete. 2024-04-20T21:30:46.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.205+0000 7f98cea00640 1 Processor -- start 2024-04-20T21:30:46.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.205+0000 7f98cea00640 1 -- start start 2024-04-20T21:30:46.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.206+0000 7f98cea00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98ac006550 0x7f98c81e5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:46.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.206+0000 7f98cea00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98c813f020 0x7f98c813f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:46.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.206+0000 7f98cea00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98c81f2b70 0x7f98c81f2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:46.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.206+0000 7f98cea00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f98c80d0190 con 0x7f98c81f2b70 2024-04-20T21:30:46.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.206+0000 7f98cea00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f98c80d0010 con 0x7f98ac006550 2024-04-20T21:30:46.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.206+0000 7f98cea00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f98c80cd5d0 con 0x7f98c813f020 2024-04-20T21:30:46.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.206+0000 7f98cca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98c81f2b70 0x7f98c81f2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:46.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.207+0000 7f98cca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98c81f2b70 0x7f98c81f2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49070/0 (socket says 172.21.15.86:49070) 2024-04-20T21:30:46.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.207+0000 7f98cca00640 1 -- 172.21.15.86:0/2252057124 learned_addr learned my addr 172.21.15.86:0/2252057124 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:46.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.207+0000 7f98c7400640 1 --2- 172.21.15.86:0/2252057124 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98c813f020 0x7f98c813f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:46.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.207+0000 7f98c7e00640 1 --2- 172.21.15.86:0/2252057124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98ac006550 0x7f98c81e5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:46.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.207+0000 7f98cca00640 1 -- 172.21.15.86:0/2252057124 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98c813f020 msgr2=0x7f98c813f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:46.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.207+0000 7f98cca00640 1 --2- 172.21.15.86:0/2252057124 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98c813f020 0x7f98c813f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:46.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.207+0000 7f98cca00640 1 -- 172.21.15.86:0/2252057124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98ac006550 msgr2=0x7f98c81e5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:46.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.207+0000 7f98cca00640 1 --2- 172.21.15.86:0/2252057124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98ac006550 0x7f98c81e5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:46.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.207+0000 7f98cca00640 1 -- 172.21.15.86:0/2252057124 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98c813e430 con 0x7f98c81f2b70 2024-04-20T21:30:46.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.207+0000 7f98c7e00640 1 --2- 172.21.15.86:0/2252057124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98ac006550 0x7f98c81e5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:30:46.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.207+0000 7f98cca00640 1 --2- 172.21.15.86:0/2252057124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98c81f2b70 0x7f98c81f2fe0 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f98c80cdfc0 tx=0x7f98c000d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:46.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.208+0000 7f98bfe00640 1 -- 172.21.15.86:0/2252057124 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c0018070 con 0x7f98c81f2b70 2024-04-20T21:30:46.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.208+0000 7f98bfe00640 1 -- 172.21.15.86:0/2252057124 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98c00052f0 con 0x7f98c81f2b70 2024-04-20T21:30:46.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.208+0000 7f98bfe00640 1 -- 172.21.15.86:0/2252057124 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c0012840 con 0x7f98c81f2b70 2024-04-20T21:30:46.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.208+0000 7f98cea00640 1 -- 172.21.15.86:0/2252057124 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98c814b8a0 con 0x7f98c81f2b70 2024-04-20T21:30:46.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.208+0000 7f98cea00640 1 -- 172.21.15.86:0/2252057124 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98c814be60 con 0x7f98c81f2b70 2024-04-20T21:30:46.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.209+0000 7f98cea00640 1 -- 172.21.15.86:0/2252057124 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98c80cebe0 con 0x7f98c81f2b70 2024-04-20T21:30:46.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.210+0000 7f98bfe00640 1 -- 172.21.15.86:0/2252057124 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f98c001b070 con 0x7f98c81f2b70 2024-04-20T21:30:46.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.211+0000 7f98bfe00640 1 --2- 172.21.15.86:0/2252057124 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f98a806c8f0 0x7f98a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:46.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.211+0000 7f98bfe00640 1 -- 172.21.15.86:0/2252057124 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7f98c008ae80 con 0x7f98c81f2b70 2024-04-20T21:30:46.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.212+0000 7f98c7e00640 1 --2- 172.21.15.86:0/2252057124 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f98a806c8f0 0x7f98a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:46.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.212+0000 7f98c7e00640 1 --2- 172.21.15.86:0/2252057124 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f98a806c8f0 0x7f98a806edb0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f98c81400e0 tx=0x7f98b800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:46.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.216+0000 7f98bfe00640 1 -- 172.21.15.86:0/2252057124 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f98c80cebe0 con 0x7f98c81f2b70 2024-04-20T21:30:46.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.453+0000 7f98cea00640 1 -- 172.21.15.86:0/2252057124 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f98c80689e0 con 0x7f98c81f2b70 2024-04-20T21:30:46.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.454+0000 7f98bfe00640 1 -- 172.21.15.86:0/2252057124 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) ==== 95+0+5205 (secure 0 0 0) 0x7f98c0059120 con 0x7f98c81f2b70 2024-04-20T21:30:46.454 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:46.454 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":111,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6253,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3627230007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3627230007},{"type":"v1","addr":"172.21.15.144:6833","nonce":3627230007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":6256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2498297133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2498297133},{"type":"v1","addr":"172.21.15.144:6835","nonce":2498297133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":9228,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/1120740475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1120740475},{"type":"v1","addr":"172.21.15.86:6836","nonce":1120740475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:29:43.769581+0000","modified":"2024-04-20T21:30:45.128369+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9792},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9792":{"gid":9792,"name":"a","rank":0,"incarnation":109,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.86:6837/2019839930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2019839930},{"type":"v1","addr":"172.21.15.86:6837","nonce":2019839930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-04-20T21:30:46.454 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 111 2024-04-20T21:30:46.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.457+0000 7f98cea00640 1 -- 172.21.15.86:0/2252057124 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f98a806c8f0 msgr2=0x7f98a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:46.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.457+0000 7f98cea00640 1 --2- 172.21.15.86:0/2252057124 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f98a806c8f0 0x7f98a806edb0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f98c81400e0 tx=0x7f98b800b040 comp rx=0 tx=0).stop 2024-04-20T21:30:46.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.457+0000 7f98cea00640 1 -- 172.21.15.86:0/2252057124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98c81f2b70 msgr2=0x7f98c81f2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:46.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.457+0000 7f98cea00640 1 --2- 172.21.15.86:0/2252057124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98c81f2b70 0x7f98c81f2fe0 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f98c80cdfc0 tx=0x7f98c000d890 comp rx=0 tx=0).stop 2024-04-20T21:30:46.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.457+0000 7f98cea00640 1 -- 172.21.15.86:0/2252057124 shutdown_connections 2024-04-20T21:30:46.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.457+0000 7f98cea00640 1 --2- 172.21.15.86:0/2252057124 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f98a806c8f0 0x7f98a806edb0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:46.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.457+0000 7f98cea00640 1 --2- 172.21.15.86:0/2252057124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98c81f2b70 0x7f98c81f2fe0 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:46.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.457+0000 7f98cea00640 1 --2- 172.21.15.86:0/2252057124 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f98c813f020 0x7f98c813f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:46.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.457+0000 7f98cea00640 1 --2- 172.21.15.86:0/2252057124 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98ac006550 0x7f98c81e5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:46.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.457+0000 7f98cea00640 1 -- 172.21.15.86:0/2252057124 >> 172.21.15.86:0/2252057124 conn(0x7f98c80b2360 msgr2=0x7f98c8071d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:46.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.458+0000 7f98cea00640 1 -- 172.21.15.86:0/2252057124 shutdown_connections 2024-04-20T21:30:46.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:46.458+0000 7f98cea00640 1 -- 172.21.15.86:0/2252057124 wait complete. 2024-04-20T21:30:46.879 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 111, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T21:29:43.769581+0000', 'modified': '2024-04-20T21:30:45.128369+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 116, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9792}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9792': {'gid': 9792, 'name': 'a', 'rank': 0, 'incarnation': 109, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.86:6837/2019839930', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6835', 'nonce': 2019839930}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 2019839930}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-20T21:30:46.879 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9792:up:rejoin 2024-04-20T21:30:47.880 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:30:48.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.091+0000 7f11aaa00640 1 Processor -- start 2024-04-20T21:30:48.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.091+0000 7f11aaa00640 1 -- start start 2024-04-20T21:30:48.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.093+0000 7f11aaa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11a4144e90 0x7f11a40b4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:48.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.093+0000 7f11aaa00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f11a40cd170 0x7f11a406ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:48.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.093+0000 7f11aaa00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f11a40cd5d0 0x7f11a406f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:48.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.093+0000 7f11aaa00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f11a4145270 con 0x7f11a40cd170 2024-04-20T21:30:48.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.093+0000 7f11aaa00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f11a40cdb50 con 0x7f11a40cd5d0 2024-04-20T21:30:48.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.093+0000 7f11aaa00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f11a40b3e50 con 0x7f11a4144e90 2024-04-20T21:30:48.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.093+0000 7f11a3400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f11a40cd170 0x7f11a406ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:48.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.093+0000 7f11a3400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f11a40cd170 0x7f11a406ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49080/0 (socket says 172.21.15.86:49080) 2024-04-20T21:30:48.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.093+0000 7f11a3400640 1 -- 172.21.15.86:0/1303424247 learned_addr learned my addr 172.21.15.86:0/1303424247 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:48.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.093+0000 7f11a3400640 1 -- 172.21.15.86:0/1303424247 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11a4144e90 msgr2=0x7f11a40b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:48.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.093+0000 7f11a3e00640 1 --2- 172.21.15.86:0/1303424247 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11a4144e90 0x7f11a40b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:48.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.093+0000 7f11a3400640 1 --2- 172.21.15.86:0/1303424247 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11a4144e90 0x7f11a40b4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:48.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.093+0000 7f11a3400640 1 -- 172.21.15.86:0/1303424247 >> v2:172.21.15.144:3300/0 conn(0x7f11a40cd5d0 msgr2=0x7f11a406f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:48.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.093+0000 7f11a8a00640 1 --2- 172.21.15.86:0/1303424247 >> v2:172.21.15.144:3300/0 conn(0x7f11a40cd5d0 0x7f11a406f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:48.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.093+0000 7f11a3400640 1 --2- 172.21.15.86:0/1303424247 >> v2:172.21.15.144:3300/0 conn(0x7f11a40cd5d0 0x7f11a406f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:48.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.093+0000 7f11a3400640 1 -- 172.21.15.86:0/1303424247 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a40b4130 con 0x7f11a40cd170 2024-04-20T21:30:48.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.095+0000 7f11a3400640 1 --2- 172.21.15.86:0/1303424247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11a40cd170 0x7f11a406ed40 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f11a4071870 tx=0x7f119002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8922001d89e5793c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:48.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.095+0000 7f11a2a00640 1 -- 172.21.15.86:0/1303424247 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119002e070 con 0x7f11a40cd170 2024-04-20T21:30:48.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.095+0000 7f11a2a00640 1 -- 172.21.15.86:0/1303424247 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1190037070 con 0x7f11a40cd170 2024-04-20T21:30:48.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.095+0000 7f11a2a00640 1 -- 172.21.15.86:0/1303424247 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1190031a00 con 0x7f11a40cd170 2024-04-20T21:30:48.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.096+0000 7f11aaa00640 1 -- 172.21.15.86:0/1303424247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11a40cd170 msgr2=0x7f11a406ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:48.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.096+0000 7f11aaa00640 1 --2- 172.21.15.86:0/1303424247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11a40cd170 0x7f11a406ed40 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f11a4071870 tx=0x7f119002b880 comp rx=0 tx=0).stop 2024-04-20T21:30:48.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.096+0000 7f11aaa00640 1 -- 172.21.15.86:0/1303424247 shutdown_connections 2024-04-20T21:30:48.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.096+0000 7f11aaa00640 1 --2- 172.21.15.86:0/1303424247 >> v2:172.21.15.144:3300/0 conn(0x7f11a40cd5d0 0x7f11a406f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:48.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.096+0000 7f11aaa00640 1 --2- 172.21.15.86:0/1303424247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11a40cd170 0x7f11a406ed40 unknown :-1 s=CLOSED pgs=2064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:48.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.096+0000 7f11aaa00640 1 --2- 172.21.15.86:0/1303424247 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11a4144e90 0x7f11a40b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:48.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.096+0000 7f11aaa00640 1 -- 172.21.15.86:0/1303424247 >> 172.21.15.86:0/1303424247 conn(0x7f11a40b2360 msgr2=0x7f11a40b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:48.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.096+0000 7f11aaa00640 1 -- 172.21.15.86:0/1303424247 shutdown_connections 2024-04-20T21:30:48.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.096+0000 7f11aaa00640 1 -- 172.21.15.86:0/1303424247 wait complete. 2024-04-20T21:30:48.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.098+0000 7f11aaa00640 1 Processor -- start 2024-04-20T21:30:48.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.098+0000 7f11aaa00640 1 -- start start 2024-04-20T21:30:48.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.099+0000 7f11aaa00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11a40cd170 0x7f11a41f7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:48.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.099+0000 7f11aaa00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11a40cd5d0 0x7f11a41f3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:48.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.099+0000 7f11aaa00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11a4144e90 0x7f11a406e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:48.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.100+0000 7f11aaa00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f11a40d0010 con 0x7f11a40cd5d0 2024-04-20T21:30:48.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.100+0000 7f11aaa00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f11a40b4e70 con 0x7f11a4144e90 2024-04-20T21:30:48.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.100+0000 7f11aaa00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f11a40d0190 con 0x7f11a40cd170 2024-04-20T21:30:48.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.100+0000 7f11a3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11a40cd5d0 0x7f11a41f3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:48.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.100+0000 7f11a3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11a40cd5d0 0x7f11a41f3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49086/0 (socket says 172.21.15.86:49086) 2024-04-20T21:30:48.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.100+0000 7f11a3400640 1 -- 172.21.15.86:0/2262639175 learned_addr learned my addr 172.21.15.86:0/2262639175 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:48.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.100+0000 7f11a3e00640 1 --2- 172.21.15.86:0/2262639175 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11a40cd170 0x7f11a41f7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:48.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.100+0000 7f11a3400640 1 -- 172.21.15.86:0/2262639175 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11a40cd170 msgr2=0x7f11a41f7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:48.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.100+0000 7f11a3400640 1 --2- 172.21.15.86:0/2262639175 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11a40cd170 0x7f11a41f7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:48.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.100+0000 7f11a3400640 1 -- 172.21.15.86:0/2262639175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11a4144e90 msgr2=0x7f11a406e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:48.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.100+0000 7f11a3400640 1 --2- 172.21.15.86:0/2262639175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11a4144e90 0x7f11a406e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:48.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.100+0000 7f11a3400640 1 -- 172.21.15.86:0/2262639175 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a41f84d0 con 0x7f11a40cd5d0 2024-04-20T21:30:48.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.101+0000 7f11a3400640 1 --2- 172.21.15.86:0/2262639175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11a40cd5d0 0x7f11a41f3880 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f119002b8c0 tx=0x7f11900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:48.100 INFO:tasks.ceph.mds.a.smithi086.stderr:2024-04-20T21:30:48.101+0000 7fd3b3000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T21:30:48.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.101+0000 7f119be00640 1 -- 172.21.15.86:0/2262639175 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1190032880 con 0x7f11a40cd5d0 2024-04-20T21:30:48.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.101+0000 7f119be00640 1 -- 172.21.15.86:0/2262639175 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1190032a20 con 0x7f11a40cd5d0 2024-04-20T21:30:48.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.101+0000 7f119be00640 1 -- 172.21.15.86:0/2262639175 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1190032e10 con 0x7f11a40cd5d0 2024-04-20T21:30:48.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.101+0000 7f11aaa00640 1 -- 172.21.15.86:0/2262639175 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11a41e1600 con 0x7f11a40cd5d0 2024-04-20T21:30:48.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.101+0000 7f11aaa00640 1 -- 172.21.15.86:0/2262639175 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11a413d9b0 con 0x7f11a40cd5d0 2024-04-20T21:30:48.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.102+0000 7f11aaa00640 1 -- 172.21.15.86:0/2262639175 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11a40689e0 con 0x7f11a40cd5d0 2024-04-20T21:30:48.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.103+0000 7f119be00640 1 -- 172.21.15.86:0/2262639175 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f119004c060 con 0x7f11a40cd5d0 2024-04-20T21:30:48.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.104+0000 7f119be00640 1 --2- 172.21.15.86:0/2262639175 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f118406c8f0 0x7f118406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:48.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.104+0000 7f119be00640 1 -- 172.21.15.86:0/2262639175 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7f11900ad9a0 con 0x7f11a40cd5d0 2024-04-20T21:30:48.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.104+0000 7f11a3e00640 1 --2- 172.21.15.86:0/2262639175 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f118406c8f0 0x7f118406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:48.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.105+0000 7f11a3e00640 1 --2- 172.21.15.86:0/2262639175 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f118406c8f0 0x7f118406edb0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f1194006fd0 tx=0x7f1194008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:48.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.109+0000 7f119be00640 1 -- 172.21.15.86:0/2262639175 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f11a41f84d0 con 0x7f11a40cd5d0 2024-04-20T21:30:48.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.349+0000 7f11aaa00640 1 -- 172.21.15.86:0/2262639175 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f11a41f3dc0 con 0x7f11a40cd5d0 2024-04-20T21:30:48.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.351+0000 7f119be00640 1 -- 172.21.15.86:0/2262639175 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5212 (secure 0 0 0) 0x7f1190040070 con 0x7f11a40cd5d0 2024-04-20T21:30:48.350 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:48.350 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":112,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6253,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3627230007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3627230007},{"type":"v1","addr":"172.21.15.144:6833","nonce":3627230007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":6256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2498297133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2498297133},{"type":"v1","addr":"172.21.15.144:6835","nonce":2498297133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":9228,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/1120740475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1120740475},{"type":"v1","addr":"172.21.15.86:6836","nonce":1120740475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:29:43.769581+0000","modified":"2024-04-20T21:30:47.137068+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9792},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9792":{"gid":9792,"name":"a","rank":0,"incarnation":109,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6837/2019839930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2019839930},{"type":"v1","addr":"172.21.15.86:6837","nonce":2019839930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9792,"qdb_cluster":[9792]},"id":8}]} 2024-04-20T21:30:48.350 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 112 2024-04-20T21:30:48.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.353+0000 7f11aaa00640 1 -- 172.21.15.86:0/2262639175 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f118406c8f0 msgr2=0x7f118406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:48.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.353+0000 7f11aaa00640 1 --2- 172.21.15.86:0/2262639175 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f118406c8f0 0x7f118406edb0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f1194006fd0 tx=0x7f1194008040 comp rx=0 tx=0).stop 2024-04-20T21:30:48.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.353+0000 7f11aaa00640 1 -- 172.21.15.86:0/2262639175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11a40cd5d0 msgr2=0x7f11a41f3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:48.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.353+0000 7f11aaa00640 1 --2- 172.21.15.86:0/2262639175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11a40cd5d0 0x7f11a41f3880 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f119002b8c0 tx=0x7f11900046c0 comp rx=0 tx=0).stop 2024-04-20T21:30:48.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.353+0000 7f11aaa00640 1 -- 172.21.15.86:0/2262639175 shutdown_connections 2024-04-20T21:30:48.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.353+0000 7f11aaa00640 1 --2- 172.21.15.86:0/2262639175 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f118406c8f0 0x7f118406edb0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:48.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.353+0000 7f11aaa00640 1 --2- 172.21.15.86:0/2262639175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11a4144e90 0x7f11a406e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:48.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.353+0000 7f11aaa00640 1 --2- 172.21.15.86:0/2262639175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11a40cd5d0 0x7f11a41f3880 unknown :-1 s=CLOSED pgs=2065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:48.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.353+0000 7f11aaa00640 1 --2- 172.21.15.86:0/2262639175 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f11a40cd170 0x7f11a41f7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:48.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.353+0000 7f11aaa00640 1 -- 172.21.15.86:0/2262639175 >> 172.21.15.86:0/2262639175 conn(0x7f11a40b2360 msgr2=0x7f11a40b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:48.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.354+0000 7f11aaa00640 1 -- 172.21.15.86:0/2262639175 shutdown_connections 2024-04-20T21:30:48.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.354+0000 7f11aaa00640 1 -- 172.21.15.86:0/2262639175 wait complete. 2024-04-20T21:30:48.684 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:30:48.692 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:30:48.777 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 112, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T21:29:43.769581+0000', 'modified': '2024-04-20T21:30:47.137068+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 116, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9792}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9792': {'gid': 9792, 'name': 'a', 'rank': 0, 'incarnation': 109, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.86:6837/2019839930', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6835', 'nonce': 2019839930}, {'type': 'v1', 'addr': '172.21.15.86:6837', 'nonce': 2019839930}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9792, 'qdb_cluster': [9792]} 2024-04-20T21:30:48.778 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-20T21:30:48.778 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-20T21:30:48.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.983+0000 7f9047800640 1 Processor -- start 2024-04-20T21:30:48.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.983+0000 7f9047800640 1 -- start start 2024-04-20T21:30:48.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.985+0000 7f9047800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f90400b3330 0x7f90400d0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:48.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.985+0000 7f9047800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f904006e1e0 0x7f904006e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:48.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.985+0000 7f9047800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90400d1220 con 0x7f9040162bd0 2024-04-20T21:30:48.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.985+0000 7f9047800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f904006eb50 con 0x7f90400b3330 2024-04-20T21:30:48.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.985+0000 7f9047800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f90400b3830 con 0x7f904006e1e0 2024-04-20T21:30:48.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.985+0000 7f9037e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f90400b3330 0x7f90400d0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:48.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.985+0000 7f9037e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f90400b3330 0x7f90400d0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49102/0 (socket says 172.21.15.86:49102) 2024-04-20T21:30:48.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.985+0000 7f9037e00640 1 -- 172.21.15.86:0/4031310284 learned_addr learned my addr 172.21.15.86:0/4031310284 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:48.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.986+0000 7f9037e00640 1 -- 172.21.15.86:0/4031310284 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f904006e1e0 msgr2=0x7f904006e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:48.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.986+0000 7f9037e00640 1 --2- 172.21.15.86:0/4031310284 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f904006e1e0 0x7f904006e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:48.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.986+0000 7f9037e00640 1 -- 172.21.15.86:0/4031310284 >> v1:172.21.15.86:6789/0 conn(0x7f9040162bd0 legacy=0x7f90400cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:48.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.986+0000 7f9037e00640 1 -- 172.21.15.86:0/4031310284 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90400b3fc0 con 0x7f90400b3330 2024-04-20T21:30:48.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.987+0000 7f9037e00640 1 --2- 172.21.15.86:0/4031310284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90400b3330 0x7f90400d0ce0 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f9028009870 tx=0x7f902802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b96ef77a62ba044f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:48.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.987+0000 7f903fe00640 1 -- 172.21.15.86:0/4031310284 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9028035020 con 0x7f90400b3330 2024-04-20T21:30:48.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.987+0000 7f903fe00640 1 -- 172.21.15.86:0/4031310284 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9028031640 con 0x7f90400b3330 2024-04-20T21:30:48.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.987+0000 7f903fe00640 1 -- 172.21.15.86:0/4031310284 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902802d040 con 0x7f90400b3330 2024-04-20T21:30:48.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.988+0000 7f9047800640 1 -- 172.21.15.86:0/4031310284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90400b3330 msgr2=0x7f90400d0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:48.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.988+0000 7f9047800640 1 --2- 172.21.15.86:0/4031310284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90400b3330 0x7f90400d0ce0 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f9028009870 tx=0x7f902802b8f0 comp rx=0 tx=0).stop 2024-04-20T21:30:48.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.988+0000 7f9047800640 1 -- 172.21.15.86:0/4031310284 shutdown_connections 2024-04-20T21:30:48.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.988+0000 7f9047800640 1 --2- 172.21.15.86:0/4031310284 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f904006e1e0 0x7f904006e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:48.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.988+0000 7f9047800640 1 --2- 172.21.15.86:0/4031310284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90400b3330 0x7f90400d0ce0 unknown :-1 s=CLOSED pgs=2067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:48.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.988+0000 7f9047800640 1 -- 172.21.15.86:0/4031310284 >> 172.21.15.86:0/4031310284 conn(0x7f904006b980 msgr2=0x7f904006bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:48.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.988+0000 7f9047800640 1 -- 172.21.15.86:0/4031310284 shutdown_connections 2024-04-20T21:30:48.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.988+0000 7f9047800640 1 -- 172.21.15.86:0/4031310284 wait complete. 2024-04-20T21:30:48.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.990+0000 7f9047800640 1 Processor -- start 2024-04-20T21:30:48.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.990+0000 7f9047800640 1 -- start start 2024-04-20T21:30:48.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.991+0000 7f9047800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f904006e1e0 0x7f90401651d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:48.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.991+0000 7f9047800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90400b3330 0x7f90400b2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:48.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.991+0000 7f9047800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9040162bd0 0x7f9040146140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:48.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.991+0000 7f9047800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9040162fb0 con 0x7f90400b3330 2024-04-20T21:30:48.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.991+0000 7f9047800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f90400b4580 con 0x7f9040162bd0 2024-04-20T21:30:48.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.991+0000 7f9047800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f9040163130 con 0x7f904006e1e0 2024-04-20T21:30:48.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.991+0000 7f9037e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90400b3330 0x7f90400b2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:48.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.991+0000 7f9037e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90400b3330 0x7f90400b2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49106/0 (socket says 172.21.15.86:49106) 2024-04-20T21:30:48.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.991+0000 7f9037e00640 1 -- 172.21.15.86:0/1499614373 learned_addr learned my addr 172.21.15.86:0/1499614373 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:48.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.992+0000 7f9044e00640 1 --2- 172.21.15.86:0/1499614373 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f904006e1e0 0x7f90401651d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:48.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.992+0000 7f9045800640 1 --2- 172.21.15.86:0/1499614373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9040162bd0 0x7f9040146140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:48.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.992+0000 7f9037e00640 1 -- 172.21.15.86:0/1499614373 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f904006e1e0 msgr2=0x7f90401651d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:48.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.992+0000 7f9037e00640 1 --2- 172.21.15.86:0/1499614373 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f904006e1e0 0x7f90401651d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:48.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.992+0000 7f9037e00640 1 -- 172.21.15.86:0/1499614373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9040162bd0 msgr2=0x7f9040146140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:48.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.992+0000 7f9037e00640 1 --2- 172.21.15.86:0/1499614373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9040162bd0 0x7f9040146140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:48.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.992+0000 7f9037e00640 1 -- 172.21.15.86:0/1499614373 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90400b16b0 con 0x7f90400b3330 2024-04-20T21:30:48.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.992+0000 7f9044e00640 1 --2- 172.21.15.86:0/1499614373 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f904006e1e0 0x7f90401651d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:30:48.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.992+0000 7f9037e00640 1 --2- 172.21.15.86:0/1499614373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90400b3330 0x7f90400b2750 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f902802b930 tx=0x7f90280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:48.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.993+0000 7f903d600640 1 -- 172.21.15.86:0/1499614373 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9028040070 con 0x7f90400b3330 2024-04-20T21:30:48.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.993+0000 7f903d600640 1 -- 172.21.15.86:0/1499614373 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9028039c10 con 0x7f90400b3330 2024-04-20T21:30:48.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.993+0000 7f9047800640 1 -- 172.21.15.86:0/1499614373 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90400b1990 con 0x7f90400b3330 2024-04-20T21:30:48.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.993+0000 7f903d600640 1 -- 172.21.15.86:0/1499614373 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9028035040 con 0x7f90400b3330 2024-04-20T21:30:48.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.993+0000 7f9047800640 1 -- 172.21.15.86:0/1499614373 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90400b1f50 con 0x7f90400b3330 2024-04-20T21:30:48.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.995+0000 7f903d600640 1 -- 172.21.15.86:0/1499614373 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9028038020 con 0x7f90400b3330 2024-04-20T21:30:48.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.996+0000 7f903d600640 1 --2- 172.21.15.86:0/1499614373 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f901c06c940 0x7f901c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:48.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.996+0000 7f903d600640 1 -- 172.21.15.86:0/1499614373 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7f90280ad820 con 0x7f90400b3330 2024-04-20T21:30:48.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.997+0000 7f9044e00640 1 --2- 172.21.15.86:0/1499614373 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f901c06c940 0x7f901c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:48.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.997+0000 7f904a585740 1 Processor -- start 2024-04-20T21:30:48.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.997+0000 7f904a585740 1 -- start start 2024-04-20T21:30:48.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.997+0000 7f9044e00640 1 --2- 172.21.15.86:0/1499614373 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f901c06c940 0x7f901c06ee00 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f90401fc160 tx=0x7f9030009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:48.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.998+0000 7f904a585740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b40ebee460 0x55b40eba5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:48.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.998+0000 7f904a585740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b40eba44e0 0x55b40eba5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:48.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.998+0000 7f904a585740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b40eba0930 0x55b40ebd0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:48.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.998+0000 7f904a585740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b40eb0c9d0 con 0x55b40eba0930 2024-04-20T21:30:48.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.998+0000 7f904a585740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b40eab3270 con 0x55b40eba44e0 2024-04-20T21:30:48.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.998+0000 7f904a585740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55b40eb08990 con 0x55b40ebee460 2024-04-20T21:30:48.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.998+0000 7f9045800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b40eba0930 0x55b40ebd0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:48.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.998+0000 7f9045800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b40eba0930 0x55b40ebd0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49122/0 (socket says 172.21.15.86:49122) 2024-04-20T21:30:48.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.998+0000 7f9045800640 1 -- 172.21.15.86:0/1772811340 learned_addr learned my addr 172.21.15.86:0/1772811340 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:48.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.999+0000 7f9037e00640 1 --2- 172.21.15.86:0/1772811340 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b40eba44e0 0x55b40eba5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:48.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.999+0000 7f9044e00640 1 --2- 172.21.15.86:0/1772811340 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b40ebee460 0x55b40eba5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:48.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.999+0000 7f9045800640 1 -- 172.21.15.86:0/1772811340 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b40ebee460 msgr2=0x55b40eba5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:48.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.999+0000 7f9045800640 1 --2- 172.21.15.86:0/1772811340 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b40ebee460 0x55b40eba5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:48.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.999+0000 7f9045800640 1 -- 172.21.15.86:0/1772811340 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b40eba44e0 msgr2=0x55b40eba5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:48.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.999+0000 7f9045800640 1 --2- 172.21.15.86:0/1772811340 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b40eba44e0 0x55b40eba5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:48.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.999+0000 7f9045800640 1 -- 172.21.15.86:0/1772811340 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b40e81f2d0 con 0x55b40eba0930 2024-04-20T21:30:48.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.999+0000 7f9044e00640 1 --2- 172.21.15.86:0/1772811340 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b40ebee460 0x55b40eba5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:30:48.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.999+0000 7f9045800640 1 --2- 172.21.15.86:0/1772811340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b40eba0930 0x55b40ebd0770 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f9038009f20 tx=0x7f903800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fd148b2aab4ba29 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:49.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:48.999+0000 7f9035600640 1 -- 172.21.15.86:0/1772811340 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903801a070 con 0x55b40eba0930 2024-04-20T21:30:49.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.000+0000 7f9035600640 1 -- 172.21.15.86:0/1772811340 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9038015410 con 0x55b40eba0930 2024-04-20T21:30:49.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.000+0000 7f9035600640 1 -- 172.21.15.86:0/1772811340 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90380157c0 con 0x55b40eba0930 2024-04-20T21:30:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.000+0000 7f904a585740 1 -- 172.21.15.86:0/1772811340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b40eba0930 msgr2=0x55b40ebd0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.000+0000 7f904a585740 1 --2- 172.21.15.86:0/1772811340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b40eba0930 0x55b40ebd0770 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f9038009f20 tx=0x7f903800c8b0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.000+0000 7f904a585740 1 -- 172.21.15.86:0/1772811340 shutdown_connections 2024-04-20T21:30:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.000+0000 7f904a585740 1 --2- 172.21.15.86:0/1772811340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b40eba0930 0x55b40ebd0770 unknown :-1 s=CLOSED pgs=2069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.000+0000 7f904a585740 1 --2- 172.21.15.86:0/1772811340 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b40eba44e0 0x55b40eba5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.000+0000 7f904a585740 1 --2- 172.21.15.86:0/1772811340 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b40ebee460 0x55b40eba5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.000+0000 7f904a585740 1 -- 172.21.15.86:0/1772811340 >> 172.21.15.86:0/1772811340 conn(0x55b40eba2bf0 msgr2=0x55b40ebeb770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.000+0000 7f904a585740 1 -- 172.21.15.86:0/1772811340 shutdown_connections 2024-04-20T21:30:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.001+0000 7f904a585740 1 -- 172.21.15.86:0/1772811340 wait complete. 2024-04-20T21:30:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.001+0000 7f904a585740 1 -- start start 2024-04-20T21:30:49.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.001+0000 7f904a585740 1 Processor -- start 2024-04-20T21:30:49.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.002+0000 7f904a585740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b40eba44e0 0x55b40ebf34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:49.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.002+0000 7f904a585740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b40ebee460 0x55b40ebb8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:49.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.002+0000 7f904a585740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b40ebffbc0 0x55b40ebc71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:49.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.002+0000 7f904a585740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b40eba49f0 con 0x55b40ebee460 2024-04-20T21:30:49.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.002+0000 7f904a585740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b40eac65e0 con 0x55b40eba44e0 2024-04-20T21:30:49.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.002+0000 7f904a585740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55b40eba4cd0 con 0x55b40ebffbc0 2024-04-20T21:30:49.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.002+0000 7f904a585740 10 client.0 _ll_register_callbacks cb 0x55b40e81eac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:30:49.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.003+0000 7f9037e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b40ebee460 0x55b40ebb8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:49.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.003+0000 7f9037e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b40ebee460 0x55b40ebb8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49124/0 (socket says 172.21.15.86:49124) 2024-04-20T21:30:49.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.003+0000 7f9037e00640 1 -- 172.21.15.86:0/2519501029 learned_addr learned my addr 172.21.15.86:0/2519501029 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:49.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.003+0000 7f9044e00640 1 --2- 172.21.15.86:0/2519501029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b40eba44e0 0x55b40ebf34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:49.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.003+0000 7f9045800640 1 --2- 172.21.15.86:0/2519501029 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b40ebffbc0 0x55b40ebc71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:49.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.003+0000 7f9037e00640 1 -- 172.21.15.86:0/2519501029 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b40ebffbc0 msgr2=0x55b40ebc71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:49.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.003+0000 7f9037e00640 1 --2- 172.21.15.86:0/2519501029 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b40ebffbc0 0x55b40ebc71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.003+0000 7f9037e00640 1 -- 172.21.15.86:0/2519501029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b40eba44e0 msgr2=0x55b40ebf34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:49.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.003+0000 7f9037e00640 1 --2- 172.21.15.86:0/2519501029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b40eba44e0 0x55b40ebf34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.003+0000 7f9037e00640 1 -- 172.21.15.86:0/2519501029 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b40e8e96b0 con 0x55b40ebee460 2024-04-20T21:30:49.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.003+0000 7f9037e00640 1 --2- 172.21.15.86:0/2519501029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b40ebee460 0x55b40ebb8cd0 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f902802bf00 tx=0x7f902807bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11855b064e20f701 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:49.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.003+0000 7f902e000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:30:49.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.004+0000 7f902e000640 1 -- 172.21.15.86:0/2519501029 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9028084020 con 0x55b40ebee460 2024-04-20T21:30:49.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.004+0000 7f902e000640 1 -- 172.21.15.86:0/2519501029 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7f90280a6d40 con 0x55b40ebee460 2024-04-20T21:30:49.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.004+0000 7f904a585740 1 -- 172.21.15.86:0/2519501029 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9038015410 con 0x55b40ebee460 2024-04-20T21:30:49.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.004+0000 7f902e000640 1 -- 172.21.15.86:0/2519501029 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) ==== 24+0+0 (secure 0 0 0) 0x7f9028083b90 con 0x55b40ebee460 2024-04-20T21:30:49.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.004+0000 7f904a585740 10 client.9885 fetch_fsmap learned FSMap version 112 2024-04-20T21:30:49.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.004+0000 7f904a585740 1 -- 172.21.15.86:0/2519501029 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) -- 0x55b40ec006e0 con 0x55b40ebee460 2024-04-20T21:30:49.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.004+0000 7f902e000640 1 -- 172.21.15.86:0/2519501029 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9038015410 con 0x55b40ebee460 2024-04-20T21:30:49.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.004+0000 7f902e000640 1 -- 172.21.15.86:0/2519501029 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 112) ==== 3495+0+0 (secure 0 0 0) 0x7f902807a890 con 0x55b40ebee460 2024-04-20T21:30:49.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.004+0000 7f904a585740 10 client.9885 fetch_fsmap finished waiting for FSMap version 112 2024-04-20T21:30:49.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.004+0000 7f904a585740 10 client.9885 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-20T21:30:49.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.004+0000 7f904a585740 20 client.9885 populate_metadata read hostname 'smithi086' 2024-04-20T21:30:49.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.004+0000 7f904a585740 1 --2- 172.21.15.86:0/2519501029 >> [v2:172.21.15.86:6835/2019839930,v1:172.21.15.86:6837/2019839930] conn(0x55b40ebcd6d0 0x55b40ebf1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:49.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.004+0000 7f904a585740 4 client.9885 mds_command: new command op to 9792 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T21:30:49.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.004+0000 7f904a585740 1 -- 172.21.15.86:0/2519501029 --> [v2:172.21.15.86:6835/2019839930,v1:172.21.15.86:6837/2019839930] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b40eb1a3f0 con 0x55b40ebcd6d0 2024-04-20T21:30:49.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.005+0000 7f9044e00640 1 --2- 172.21.15.86:0/2519501029 >> [v2:172.21.15.86:6835/2019839930,v1:172.21.15.86:6837/2019839930] conn(0x55b40ebcd6d0 0x55b40ebf1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:49.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.005+0000 7f9044e00640 1 --2- 172.21.15.86:0/2519501029 >> [v2:172.21.15.86:6835/2019839930,v1:172.21.15.86:6837/2019839930] conn(0x55b40ebcd6d0 0x55b40ebf1550 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=478c7c2f12294293 server_cookie=39c85b99bacb0716 in_seq=0 out_seq=0 2024-04-20T21:30:49.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.005+0000 7f902e000640 10 client.9885 ms_handle_connect on v2:172.21.15.86:6835/2019839930 2024-04-20T21:30:49.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.006+0000 7f902e000640 1 -- 172.21.15.86:0/2519501029 <== mds.0 v2:172.21.15.86:6835/2019839930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b40eb1a3f0 con 0x55b40ebcd6d0 2024-04-20T21:30:49.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.006+0000 7f902e000640 10 client.9885 handle_command_reply: tid=0 2024-04-20T21:30:49.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.006+0000 7f904a585740 1 client.9885 shutdown 2024-04-20T21:30:49.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.006+0000 7f904a585740 1 -- 172.21.15.86:0/2519501029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b40ebee460 msgr2=0x55b40ebb8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:49.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.006+0000 7f904a585740 1 --2- 172.21.15.86:0/2519501029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b40ebee460 0x55b40ebb8cd0 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f902802bf00 tx=0x7f902807bb40 comp rx=0 tx=0).stop 2024-04-20T21:30:49.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.006+0000 7f904a585740 1 -- 172.21.15.86:0/2519501029 shutdown_connections 2024-04-20T21:30:49.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.006+0000 7f904a585740 1 --2- 172.21.15.86:0/2519501029 >> [v2:172.21.15.86:6835/2019839930,v1:172.21.15.86:6837/2019839930] conn(0x55b40ebcd6d0 0x55b40ebf1550 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.006+0000 7f904a585740 1 --2- 172.21.15.86:0/2519501029 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b40ebffbc0 0x55b40ebc71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.006+0000 7f904a585740 1 --2- 172.21.15.86:0/2519501029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b40ebee460 0x55b40ebb8cd0 unknown :-1 s=CLOSED pgs=2070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.006+0000 7f904a585740 1 --2- 172.21.15.86:0/2519501029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b40eba44e0 0x55b40ebf34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.006+0000 7f904a585740 1 -- 172.21.15.86:0/2519501029 >> 172.21.15.86:0/2519501029 conn(0x55b40eba0930 msgr2=0x55b40eba65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:49.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.006+0000 7f902e000640 0 client.9885 ms_handle_reset on v2:172.21.15.86:6835/2019839930 2024-04-20T21:30:49.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.007+0000 7f904a585740 1 -- 172.21.15.86:0/2519501029 shutdown_connections 2024-04-20T21:30:49.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.007+0000 7f904a585740 1 -- 172.21.15.86:0/2519501029 wait complete. 2024-04-20T21:30:49.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.007+0000 7f904a585740 20 client.9885 trim_cache size 0 max 16384 2024-04-20T21:30:49.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.027+0000 7f904a585740 1 Processor -- start 2024-04-20T21:30:49.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.027+0000 7f904a585740 1 -- start start 2024-04-20T21:30:49.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.028+0000 7f904a585740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b40eba44e0 0x55b40ebf1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:49.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.028+0000 7f904a585740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b40ebee460 0x55b40ebfe6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:49.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.028+0000 7f904a585740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b40ebffbc0 0x55b40ebfec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:49.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.028+0000 7f904a585740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f90402045c0 con 0x55b40eba44e0 2024-04-20T21:30:49.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.028+0000 7f904a585740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f90400b0d60 con 0x55b40ebffbc0 2024-04-20T21:30:49.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.028+0000 7f904a585740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f90400ce710 con 0x55b40ebee460 2024-04-20T21:30:49.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.028+0000 7f9044e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b40eba44e0 0x55b40ebf1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:49.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.028+0000 7f9044e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b40eba44e0 0x55b40ebf1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49130/0 (socket says 172.21.15.86:49130) 2024-04-20T21:30:49.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.028+0000 7f9044e00640 1 -- 172.21.15.86:0/3122834048 learned_addr learned my addr 172.21.15.86:0/3122834048 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:49.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.028+0000 7f9044e00640 1 -- 172.21.15.86:0/3122834048 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b40ebee460 msgr2=0x55b40ebfe6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:49.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.028+0000 7f9044e00640 1 --2- 172.21.15.86:0/3122834048 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b40ebee460 0x55b40ebfe6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.028+0000 7f9044e00640 1 -- 172.21.15.86:0/3122834048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b40ebffbc0 msgr2=0x55b40ebfec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:49.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.028+0000 7f9044e00640 1 --2- 172.21.15.86:0/3122834048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b40ebffbc0 0x55b40ebfec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.028+0000 7f9044e00640 1 -- 172.21.15.86:0/3122834048 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b40ebc7560 con 0x55b40eba44e0 2024-04-20T21:30:49.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.029+0000 7f9044e00640 1 --2- 172.21.15.86:0/3122834048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b40eba44e0 0x55b40ebf1710 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f90400aff70 tx=0x7f9030012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc53504cd2300fbf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:49.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.029+0000 7f9034c00640 1 -- 172.21.15.86:0/3122834048 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90300053d0 con 0x55b40eba44e0 2024-04-20T21:30:49.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.029+0000 7f9034c00640 1 -- 172.21.15.86:0/3122834048 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9030029020 con 0x55b40eba44e0 2024-04-20T21:30:49.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.029+0000 7f9034c00640 1 -- 172.21.15.86:0/3122834048 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9030024420 con 0x55b40eba44e0 2024-04-20T21:30:49.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.030+0000 7f904a585740 1 -- 172.21.15.86:0/3122834048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b40eba44e0 msgr2=0x55b40ebf1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:49.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.030+0000 7f904a585740 1 --2- 172.21.15.86:0/3122834048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b40eba44e0 0x55b40ebf1710 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f90400aff70 tx=0x7f9030012000 comp rx=0 tx=0).stop 2024-04-20T21:30:49.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.030+0000 7f904a585740 1 -- 172.21.15.86:0/3122834048 shutdown_connections 2024-04-20T21:30:49.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.030+0000 7f904a585740 1 --2- 172.21.15.86:0/3122834048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b40ebffbc0 0x55b40ebfec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.030+0000 7f904a585740 1 --2- 172.21.15.86:0/3122834048 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b40ebee460 0x55b40ebfe6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.030+0000 7f904a585740 1 --2- 172.21.15.86:0/3122834048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b40eba44e0 0x55b40ebf1710 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.030+0000 7f904a585740 1 -- 172.21.15.86:0/3122834048 >> 172.21.15.86:0/3122834048 conn(0x55b40eba0930 msgr2=0x55b40ebb8110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:49.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.030+0000 7f904a585740 1 -- 172.21.15.86:0/3122834048 shutdown_connections 2024-04-20T21:30:49.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.030+0000 7f904a585740 1 -- 172.21.15.86:0/3122834048 wait complete. 2024-04-20T21:30:49.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.030+0000 7f904a585740 1 -- start start 2024-04-20T21:30:49.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.031+0000 7f904a585740 1 Processor -- start 2024-04-20T21:30:49.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.032+0000 7f904a585740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b40eba44e0 0x55b40ec66530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:49.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.032+0000 7f904a585740 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b40ebee460 0x55b40ebfe6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:49.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.032+0000 7f904a585740 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b40ebffbc0 0x55b40ebef010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:49.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.032+0000 7f904a585740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b40ebf2070 con 0x55b40eba44e0 2024-04-20T21:30:49.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.032+0000 7f904a585740 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x55b40ec00960 con 0x55b40ebffbc0 2024-04-20T21:30:49.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.032+0000 7f904a585740 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x55b40ebcdba0 con 0x55b40ebee460 2024-04-20T21:30:49.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.032+0000 7f904a585740 10 client.0 _ll_register_callbacks cb 0x7f90280094d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T21:30:49.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.032+0000 7f9044e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b40eba44e0 0x55b40ec66530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:49.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.032+0000 7f9044e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b40eba44e0 0x55b40ec66530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49142/0 (socket says 172.21.15.86:49142) 2024-04-20T21:30:49.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.032+0000 7f9044e00640 1 -- 172.21.15.86:0/4248234981 learned_addr learned my addr 172.21.15.86:0/4248234981 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:49.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.032+0000 7f9044e00640 1 -- 172.21.15.86:0/4248234981 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b40ebee460 msgr2=0x55b40ebfe6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:49.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.032+0000 7f9044e00640 1 --2- 172.21.15.86:0/4248234981 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b40ebee460 0x55b40ebfe6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.032+0000 7f9044e00640 1 -- 172.21.15.86:0/4248234981 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b40ebffbc0 msgr2=0x55b40ebef010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:49.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.032+0000 7f9044e00640 1 --2- 172.21.15.86:0/4248234981 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b40ebffbc0 0x55b40ebef010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.032+0000 7f9044e00640 1 -- 172.21.15.86:0/4248234981 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b40ebfec10 con 0x55b40eba44e0 2024-04-20T21:30:49.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.033+0000 7f9044e00640 1 --2- 172.21.15.86:0/4248234981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b40eba44e0 0x55b40ec66530 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f9030009010 tx=0x7f9030000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5da69a1d53070bb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:49.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.033+0000 7f902e000640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-20T21:30:49.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.033+0000 7f902e000640 1 -- 172.21.15.86:0/4248234981 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9030028040 con 0x55b40eba44e0 2024-04-20T21:30:49.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.033+0000 7f902e000640 1 -- 172.21.15.86:0/4248234981 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7f9030038bd0 con 0x55b40eba44e0 2024-04-20T21:30:49.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.033+0000 7f904a585740 1 -- 172.21.15.86:0/4248234981 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9030029020 con 0x55b40eba44e0 2024-04-20T21:30:49.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.033+0000 7f902e000640 1 -- 172.21.15.86:0/4248234981 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) ==== 24+0+0 (secure 0 0 0) 0x7f9030038dd0 con 0x55b40eba44e0 2024-04-20T21:30:49.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.033+0000 7f904a585740 10 client.9891 fetch_fsmap learned FSMap version 112 2024-04-20T21:30:49.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.033+0000 7f904a585740 1 -- 172.21.15.86:0/4248234981 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) -- 0x55b40ebeed60 con 0x55b40eba44e0 2024-04-20T21:30:49.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.034+0000 7f902e000640 1 -- 172.21.15.86:0/4248234981 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9030029020 con 0x55b40eba44e0 2024-04-20T21:30:49.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.034+0000 7f902e000640 1 -- 172.21.15.86:0/4248234981 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 112) ==== 3495+0+0 (secure 0 0 0) 0x7f90300035f0 con 0x55b40eba44e0 2024-04-20T21:30:49.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.034+0000 7f904a585740 10 client.9891 fetch_fsmap finished waiting for FSMap version 112 2024-04-20T21:30:49.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.034+0000 7f904a585740 10 client.9891 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-20T21:30:49.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.034+0000 7f904a585740 20 client.9891 populate_metadata read hostname 'smithi086' 2024-04-20T21:30:49.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.034+0000 7f904a585740 1 --2- 172.21.15.86:0/4248234981 >> [v2:172.21.15.86:6835/2019839930,v1:172.21.15.86:6837/2019839930] conn(0x55b40ebec9e0 0x55b40ebecdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:49.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.034+0000 7f904a585740 4 client.9891 mds_command: new command op to 9792 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T21:30:49.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.034+0000 7f904a585740 1 -- 172.21.15.86:0/4248234981 --> [v2:172.21.15.86:6835/2019839930,v1:172.21.15.86:6837/2019839930] -- command(tid 0: {"prefix": "status"}) -- 0x55b40eb14260 con 0x55b40ebec9e0 2024-04-20T21:30:49.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.034+0000 7f9037e00640 1 --2- 172.21.15.86:0/4248234981 >> [v2:172.21.15.86:6835/2019839930,v1:172.21.15.86:6837/2019839930] conn(0x55b40ebec9e0 0x55b40ebecdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:49.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.035+0000 7f9037e00640 1 --2- 172.21.15.86:0/4248234981 >> [v2:172.21.15.86:6835/2019839930,v1:172.21.15.86:6837/2019839930] conn(0x55b40ebec9e0 0x55b40ebecdc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b76e110a58adade server_cookie=5fd7f98eb923b93f in_seq=0 out_seq=0 2024-04-20T21:30:49.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.035+0000 7f902e000640 10 client.9891 ms_handle_connect on v2:172.21.15.86:6835/2019839930 2024-04-20T21:30:49.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.035+0000 7f902e000640 1 -- 172.21.15.86:0/4248234981 <== mds.0 v2:172.21.15.86:6835/2019839930 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55b40eb14260 con 0x55b40ebec9e0 2024-04-20T21:30:49.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.035+0000 7f902e000640 10 client.9891 handle_command_reply: tid=0 2024-04-20T21:30:49.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.035+0000 7f904a585740 1 client.9891 shutdown 2024-04-20T21:30:49.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.035+0000 7f904a585740 1 -- 172.21.15.86:0/4248234981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b40eba44e0 msgr2=0x55b40ec66530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:49.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.035+0000 7f904a585740 1 --2- 172.21.15.86:0/4248234981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b40eba44e0 0x55b40ec66530 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f9030009010 tx=0x7f9030000bc0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.035+0000 7f904a585740 1 -- 172.21.15.86:0/4248234981 shutdown_connections 2024-04-20T21:30:49.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.035+0000 7f904a585740 1 --2- 172.21.15.86:0/4248234981 >> [v2:172.21.15.86:6835/2019839930,v1:172.21.15.86:6837/2019839930] conn(0x55b40ebec9e0 0x55b40ebecdc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.035+0000 7f904a585740 1 --2- 172.21.15.86:0/4248234981 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x55b40ebffbc0 0x55b40ebef010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.035+0000 7f904a585740 1 --2- 172.21.15.86:0/4248234981 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x55b40ebee460 0x55b40ebfe6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.035+0000 7f904a585740 1 --2- 172.21.15.86:0/4248234981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b40eba44e0 0x55b40ec66530 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.035+0000 7f902e000640 0 client.9891 ms_handle_reset on v2:172.21.15.86:6835/2019839930 2024-04-20T21:30:49.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.035+0000 7f904a585740 1 -- 172.21.15.86:0/4248234981 >> 172.21.15.86:0/4248234981 conn(0x55b40eba0930 msgr2=0x55b40eba4c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:49.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.035+0000 7f904a585740 1 -- 172.21.15.86:0/4248234981 shutdown_connections 2024-04-20T21:30:49.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.035+0000 7f904a585740 1 -- 172.21.15.86:0/4248234981 wait complete. 2024-04-20T21:30:49.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.035+0000 7f904a585740 20 client.9891 trim_cache size 0 max 16384 2024-04-20T21:30:49.039 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:30:49.039 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "9c2dec1d-d7ae-4a83-a812-79922808e278", 2024-04-20T21:30:49.039 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-04-20T21:30:49.040 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 9792, 2024-04-20T21:30:49.040 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-20T21:30:49.040 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-20T21:30:49.040 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-20T21:30:49.040 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 4.92156737, 2024-04-20T21:30:49.040 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 112, 2024-04-20T21:30:49.040 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 116, 2024-04-20T21:30:49.040 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 116, 2024-04-20T21:30:49.040 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 6.8784290209999996 2024-04-20T21:30:49.040 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:30:49.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.036+0000 7f9047800640 1 -- 172.21.15.86:0/1499614373 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f901c06c940 msgr2=0x7f901c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:49.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.036+0000 7f9047800640 1 --2- 172.21.15.86:0/1499614373 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f901c06c940 0x7f901c06ee00 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f90401fc160 tx=0x7f9030009040 comp rx=0 tx=0).stop 2024-04-20T21:30:49.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.036+0000 7f9047800640 1 -- 172.21.15.86:0/1499614373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90400b3330 msgr2=0x7f90400b2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:49.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.036+0000 7f9047800640 1 --2- 172.21.15.86:0/1499614373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90400b3330 0x7f90400b2750 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f902802b930 tx=0x7f90280046c0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.036+0000 7f9047800640 1 -- 172.21.15.86:0/1499614373 shutdown_connections 2024-04-20T21:30:49.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.036+0000 7f9047800640 1 --2- 172.21.15.86:0/1499614373 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f901c06c940 0x7f901c06ee00 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.036+0000 7f9047800640 1 --2- 172.21.15.86:0/1499614373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9040162bd0 0x7f9040146140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.036+0000 7f9047800640 1 --2- 172.21.15.86:0/1499614373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f90400b3330 0x7f90400b2750 unknown :-1 s=CLOSED pgs=2068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.036+0000 7f9047800640 1 --2- 172.21.15.86:0/1499614373 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f904006e1e0 0x7f90401651d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.036+0000 7f9047800640 1 -- 172.21.15.86:0/1499614373 >> 172.21.15.86:0/1499614373 conn(0x7f904006b980 msgr2=0x7f90401601c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:49.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.036+0000 7f9047800640 1 -- 172.21.15.86:0/1499614373 shutdown_connections 2024-04-20T21:30:49.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.036+0000 7f9047800640 1 -- 172.21.15.86:0/1499614373 wait complete. 2024-04-20T21:30:49.052 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T21:30:49.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.238+0000 7fdc81000640 1 Processor -- start 2024-04-20T21:30:49.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.238+0000 7fdc81000640 1 -- start start 2024-04-20T21:30:49.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.240+0000 7fdc81000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdc7c06e6d0 0x7fdc7c071d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:49.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.240+0000 7fdc81000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc7c06eb30 con 0x7fdc7c146130 2024-04-20T21:30:49.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.240+0000 7fdc81000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc7c06dc30 con 0x7fdc7c072240 2024-04-20T21:30:49.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.240+0000 7fdc81000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fdc7c072690 con 0x7fdc7c06e6d0 2024-04-20T21:30:49.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.240+0000 7fdc79400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdc7c06e6d0 0x7fdc7c071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:49.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.240+0000 7fdc79e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fdc7c146130 0x7fdc7c0ccdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57082/0 (socket says 172.21.15.86:57082) 2024-04-20T21:30:49.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.240+0000 7fdc79e00640 1 -- 172.21.15.86:0/217003271 learned_addr learned my addr 172.21.15.86:0/217003271 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:30:49.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.241+0000 7fdc79400640 1 -- 172.21.15.86:0/217003271 >> v1:172.21.15.144:6789/0 conn(0x7fdc7c072240 legacy=0x7fdc7c15cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:49.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.241+0000 7fdc79400640 1 -- 172.21.15.86:0/217003271 >> v1:172.21.15.86:6789/0 conn(0x7fdc7c146130 legacy=0x7fdc7c0ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:49.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.241+0000 7fdc79400640 1 -- 172.21.15.86:0/217003271 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc7c0cd8e0 con 0x7fdc7c06e6d0 2024-04-20T21:30:49.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.241+0000 7fdc78a00640 1 -- 172.21.15.86:0/217003271 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3654803005 0 0) 0x7fdc6c001490 con 0x7fdc7c146130 2024-04-20T21:30:49.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.242+0000 7fdc79400640 1 --2- 172.21.15.86:0/217003271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc7c06e6d0 0x7fdc7c071d00 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c073120 tx=0x7fdc6802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=446534fb0f5f0e4f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:49.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.242+0000 7fdc78a00640 1 -- 172.21.15.86:0/217003271 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc68034070 con 0x7fdc7c06e6d0 2024-04-20T21:30:49.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.242+0000 7fdc78a00640 1 -- 172.21.15.86:0/217003271 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc68030610 con 0x7fdc7c06e6d0 2024-04-20T21:30:49.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.242+0000 7fdc78a00640 1 -- 172.21.15.86:0/217003271 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc680309e0 con 0x7fdc7c06e6d0 2024-04-20T21:30:49.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.243+0000 7fdc81000640 1 -- 172.21.15.86:0/217003271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc7c06e6d0 msgr2=0x7fdc7c071d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:49.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.243+0000 7fdc81000640 1 --2- 172.21.15.86:0/217003271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc7c06e6d0 0x7fdc7c071d00 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c073120 tx=0x7fdc6802b960 comp rx=0 tx=0).stop 2024-04-20T21:30:49.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.243+0000 7fdc81000640 1 -- 172.21.15.86:0/217003271 shutdown_connections 2024-04-20T21:30:49.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.243+0000 7fdc81000640 1 --2- 172.21.15.86:0/217003271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc7c06e6d0 0x7fdc7c071d00 unknown :-1 s=CLOSED pgs=2074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.243+0000 7fdc81000640 1 -- 172.21.15.86:0/217003271 >> 172.21.15.86:0/217003271 conn(0x7fdc7c0b2360 msgr2=0x7fdc7c0b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:49.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.243+0000 7fdc81000640 1 -- 172.21.15.86:0/217003271 shutdown_connections 2024-04-20T21:30:49.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.243+0000 7fdc81000640 1 -- 172.21.15.86:0/217003271 wait complete. 2024-04-20T21:30:49.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.245+0000 7fdc81000640 1 Processor -- start 2024-04-20T21:30:49.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.245+0000 7fdc81000640 1 -- start start 2024-04-20T21:30:49.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.246+0000 7fdc81000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc7c06e6d0 0x7fdc7c140190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:49.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.246+0000 7fdc81000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc7c072240 0x7fdc7c0a9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:49.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.246+0000 7fdc81000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc7c146130 0x7fdc7c13e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:49.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.246+0000 7fdc81000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdc7c06e1b0 con 0x7fdc7c072240 2024-04-20T21:30:49.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.246+0000 7fdc81000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fdc7c06ded0 con 0x7fdc7c146130 2024-04-20T21:30:49.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.246+0000 7fdc81000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fdc7c06f150 con 0x7fdc7c06e6d0 2024-04-20T21:30:49.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.246+0000 7fdc79400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc7c072240 0x7fdc7c0a9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:49.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.246+0000 7fdc79400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc7c072240 0x7fdc7c0a9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49160/0 (socket says 172.21.15.86:49160) 2024-04-20T21:30:49.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.246+0000 7fdc79400640 1 -- 172.21.15.86:0/2125800765 learned_addr learned my addr 172.21.15.86:0/2125800765 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:49.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.247+0000 7fdc7a800640 1 --2- 172.21.15.86:0/2125800765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc7c146130 0x7fdc7c13e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:49.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.247+0000 7fdc79e00640 1 --2- 172.21.15.86:0/2125800765 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc7c06e6d0 0x7fdc7c140190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:49.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.247+0000 7fdc79400640 1 -- 172.21.15.86:0/2125800765 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc7c06e6d0 msgr2=0x7fdc7c140190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:49.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.247+0000 7fdc79400640 1 --2- 172.21.15.86:0/2125800765 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc7c06e6d0 0x7fdc7c140190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.247+0000 7fdc79400640 1 -- 172.21.15.86:0/2125800765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc7c146130 msgr2=0x7fdc7c13e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:49.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.247+0000 7fdc79400640 1 --2- 172.21.15.86:0/2125800765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc7c146130 0x7fdc7c13e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.247+0000 7fdc79400640 1 -- 172.21.15.86:0/2125800765 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc7c13e3e0 con 0x7fdc7c072240 2024-04-20T21:30:49.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.247+0000 7fdc79400640 1 --2- 172.21.15.86:0/2125800765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc7c072240 0x7fdc7c0a9670 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7fdc68009890 tx=0x7fdc680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:49.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.248+0000 7fdc72000640 1 -- 172.21.15.86:0/2125800765 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc68034040 con 0x7fdc7c072240 2024-04-20T21:30:49.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.248+0000 7fdc72000640 1 -- 172.21.15.86:0/2125800765 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc68004030 con 0x7fdc7c072240 2024-04-20T21:30:49.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.248+0000 7fdc72000640 1 -- 172.21.15.86:0/2125800765 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc68004400 con 0x7fdc7c072240 2024-04-20T21:30:49.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.248+0000 7fdc81000640 1 -- 172.21.15.86:0/2125800765 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc7c13ee50 con 0x7fdc7c072240 2024-04-20T21:30:49.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.248+0000 7fdc81000640 1 -- 172.21.15.86:0/2125800765 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc7c13f3d0 con 0x7fdc7c072240 2024-04-20T21:30:49.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.249+0000 7fdc81000640 1 -- 172.21.15.86:0/2125800765 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc7c0689e0 con 0x7fdc7c072240 2024-04-20T21:30:49.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.250+0000 7fdc72000640 1 -- 172.21.15.86:0/2125800765 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdc68037070 con 0x7fdc7c072240 2024-04-20T21:30:49.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.251+0000 7fdc72000640 1 --2- 172.21.15.86:0/2125800765 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdc5c06c920 0x7fdc5c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:49.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.251+0000 7fdc72000640 1 -- 172.21.15.86:0/2125800765 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7fdc680adab0 con 0x7fdc7c072240 2024-04-20T21:30:49.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.251+0000 7fdc79e00640 1 --2- 172.21.15.86:0/2125800765 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdc5c06c920 0x7fdc5c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:49.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.252+0000 7fdc79e00640 1 --2- 172.21.15.86:0/2125800765 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdc5c06c920 0x7fdc5c06ede0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c007ed0 tx=0x7fdc6c007e60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:49.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.255+0000 7fdc72000640 1 -- 172.21.15.86:0/2125800765 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fdc6807be70 con 0x7fdc7c072240 2024-04-20T21:30:49.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.496+0000 7fdc81000640 1 -- 172.21.15.86:0/2125800765 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fdc7c15ff50 con 0x7fdc7c072240 2024-04-20T21:30:49.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.497+0000 7fdc72000640 1 -- 172.21.15.86:0/2125800765 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v112) ==== 53+0+73 (secure 0 0 0) 0x7fdc6807b770 con 0x7fdc7c072240 2024-04-20T21:30:49.496 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T21:30:49.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.501+0000 7fdc81000640 1 -- 172.21.15.86:0/2125800765 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdc5c06c920 msgr2=0x7fdc5c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:49.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.501+0000 7fdc81000640 1 --2- 172.21.15.86:0/2125800765 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdc5c06c920 0x7fdc5c06ede0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c007ed0 tx=0x7fdc6c007e60 comp rx=0 tx=0).stop 2024-04-20T21:30:49.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.501+0000 7fdc81000640 1 -- 172.21.15.86:0/2125800765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc7c072240 msgr2=0x7fdc7c0a9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:49.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.501+0000 7fdc81000640 1 --2- 172.21.15.86:0/2125800765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc7c072240 0x7fdc7c0a9670 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7fdc68009890 tx=0x7fdc680046c0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.502+0000 7fdc81000640 1 -- 172.21.15.86:0/2125800765 shutdown_connections 2024-04-20T21:30:49.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.502+0000 7fdc81000640 1 --2- 172.21.15.86:0/2125800765 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fdc5c06c920 0x7fdc5c06ede0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.502+0000 7fdc81000640 1 --2- 172.21.15.86:0/2125800765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc7c146130 0x7fdc7c13e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.502+0000 7fdc81000640 1 --2- 172.21.15.86:0/2125800765 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc7c072240 0x7fdc7c0a9670 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.502+0000 7fdc81000640 1 --2- 172.21.15.86:0/2125800765 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fdc7c06e6d0 0x7fdc7c140190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:49.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.502+0000 7fdc81000640 1 -- 172.21.15.86:0/2125800765 >> 172.21.15.86:0/2125800765 conn(0x7fdc7c0b2360 msgr2=0x7fdc7c0b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:49.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.502+0000 7fdc81000640 1 -- 172.21.15.86:0/2125800765 shutdown_connections 2024-04-20T21:30:49.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49.502+0000 7fdc81000640 1 -- 172.21.15.86:0/2125800765 wait complete. 2024-04-20T21:30:49.520 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T21:30:49.520 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T21:30:49.520 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-04-20T21:30:49.520 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-20T21:30:49.520 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T21:30:49.520 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T21:30:49.520 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T21:30:49.520 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T21:30:49.520 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:30:49.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:30:49.521 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-04-20T21:30:49.570 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T21:30:49.570 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T21:30:49.571 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T21:30:49.571 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:30:49.571 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-20T21:30:49.571 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:30:49.571 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:30:49.571 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:30:49.571 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-04-20T21:30:49.571 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:30:49.571 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:30:49.571 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-04-20T21:30:49.571 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T21:30:49.571 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:30:49.571 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T21:30:49.571 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-04-20T21:30:49.571 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T21:30:49.572 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 41906sec preferred_lft 41906sec 2024-04-20T21:30:49.572 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2024-04-20T21:30:49.572 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:30:49.572 INFO:teuthology.orchestra.run.smithi086.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T21:30:49.572 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 86:25:97:2c:9c:bb brd ff:ff:ff:ff:ff:ff 2024-04-20T21:30:49.572 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-20T21:30:49.572 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:30:49.572 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::a4fb:30ff:fe90:2d6a/64 scope link 2024-04-20T21:30:49.572 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:30:49.572 INFO:teuthology.orchestra.run.smithi086.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T21:30:49.572 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 86:25:97:2c:9c:bb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:30:49.572 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::8425:97ff:fe2c:9cbb/64 scope link 2024-04-20T21:30:49.572 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:30:49.572 INFO:teuthology.orchestra.run.smithi086.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T21:30:49.573 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether c2:9c:8b:55:ca:84 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:30:49.573 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c09c:8bff:fe55:ca84/64 scope link 2024-04-20T21:30:49.573 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-20T21:30:49.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:30:49.574 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T21:30:49.652 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-20T21:30:49.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:30:49.656 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-04-20T21:30:49.711 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-04-20T21:30:49.711 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T21:30:49.711 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T21:30:49.711 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T21:30:49.711 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T21:30:49.711 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T21:30:49.711 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-20T21:30:49.711 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T21:30:49.712 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T21:30:49.712 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T21:30:49.712 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-20T21:30:49.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T21:30:49.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:30:49.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:30:49.922 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-20T21:30:49.969 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-20T21:30:49.969 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-20T21:30:49.969 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:30:49.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:30:49.970 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-04-20T21:30:50.054 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-20T21:30:50.054 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T21:30:50.054 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-20T21:30:50.107 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-20T21:30:50.107 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T21:30:50.107 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-20T21:30:50.157 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-04-20T21:30:50.233 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T21:30:50.288 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-04-20T21:30:50.288 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-04-20T21:30:50.289 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-04-20T21:30:50.289 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-04-20T21:30:50.289 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-04-20T21:30:50.289 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-04-20T21:30:50.289 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-04-20T21:30:50.289 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-04-20T21:30:50.289 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-04-20T21:30:50.289 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-04-20T21:30:50.289 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-04-20T21:30:50.289 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/selinux 2024-04-20T21:30:50.289 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T21:30:50.289 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-04-20T21:30:50.289 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-04-20T21:30:50.290 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-04-20T21:30:50.290 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-04-20T21:30:50.290 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-04-20T21:30:50.290 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-04-20T21:30:50.290 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T21:30:50.290 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T21:30:50.290 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T21:30:50.290 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T21:30:50.290 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-04-20T21:30:50.290 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-04-20T21:30:50.290 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T21:30:50.290 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T21:30:50.290 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T21:30:50.290 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T21:30:50.291 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-04-20T21:30:50.291 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:30:50.291 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:30:50.291 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:30:50.291 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:30:50.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:30:50.292 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-04-20T21:30:50.342 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T21:30:50.342 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-20T21:30:50.348 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-04-20T21:30:50.417 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T21:30:50.458 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-04-20T21:30:50.458 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-04-20T21:30:50.458 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-04-20T21:30:50.458 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-04-20T21:30:50.458 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-04-20T21:30:50.458 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-04-20T21:30:50.458 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-04-20T21:30:50.458 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-04-20T21:30:50.458 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-04-20T21:30:50.458 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-04-20T21:30:50.458 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-04-20T21:30:50.459 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/selinux 2024-04-20T21:30:50.459 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T21:30:50.459 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-04-20T21:30:50.459 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-04-20T21:30:50.459 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-04-20T21:30:50.459 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-04-20T21:30:50.459 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-04-20T21:30:50.459 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-04-20T21:30:50.459 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T21:30:50.459 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T21:30:50.459 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T21:30:50.459 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T21:30:50.459 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-04-20T21:30:50.459 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-04-20T21:30:50.460 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T21:30:50.460 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T21:30:50.460 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T21:30:50.460 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T21:30:50.460 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-04-20T21:30:50.460 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:30:50.460 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:30:50.460 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:30:50.460 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:30:50.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:30:50.461 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-04-20T21:30:50.515 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:2024-04-20T21:30:50.517+0000 7ff253e8f480 -1 init, newargv = 0x55f4bbb8c4e0 newargc=15 2024-04-20T21:30:50.516 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[69175]: starting ceph client 2024-04-20T21:30:50.524 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[69175]: starting fuse 2024-04-20T21:30:50.543 INFO:teuthology.orchestra.run.smithi086.stdout:41 2024-04-20T21:30:50.543 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-04-20T21:30:50.544 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer -- python3 -c ' 2024-04-20T21:30:50.544 DEBUG:teuthology.orchestra.run.smithi086:> import glob 2024-04-20T21:30:50.544 DEBUG:teuthology.orchestra.run.smithi086:> import re 2024-04-20T21:30:50.544 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:30:50.544 DEBUG:teuthology.orchestra.run.smithi086:> import subprocess 2024-04-20T21:30:50.544 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:30:50.544 DEBUG:teuthology.orchestra.run.smithi086:> def _find_admin_socket(client_name): 2024-04-20T21:30:50.544 DEBUG:teuthology.orchestra.run.smithi086:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-20T21:30:50.544 DEBUG:teuthology.orchestra.run.smithi086:> files = glob.glob(asok_path) 2024-04-20T21:30:50.544 DEBUG:teuthology.orchestra.run.smithi086:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-20T21:30:50.544 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:30:50.544 DEBUG:teuthology.orchestra.run.smithi086:> # Given a non-glob path, it better be there 2024-04-20T21:30:50.544 DEBUG:teuthology.orchestra.run.smithi086:> if "*" not in asok_path: 2024-04-20T21:30:50.544 DEBUG:teuthology.orchestra.run.smithi086:> assert(len(files) == 1) 2024-04-20T21:30:50.545 DEBUG:teuthology.orchestra.run.smithi086:> return files[0] 2024-04-20T21:30:50.545 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:30:50.545 DEBUG:teuthology.orchestra.run.smithi086:> for f in files: 2024-04-20T21:30:50.545 DEBUG:teuthology.orchestra.run.smithi086:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T21:30:50.545 DEBUG:teuthology.orchestra.run.smithi086:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T21:30:50.545 DEBUG:teuthology.orchestra.run.smithi086:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T21:30:50.545 DEBUG:teuthology.orchestra.run.smithi086:> contents = proc_f.read() 2024-04-20T21:30:50.545 DEBUG:teuthology.orchestra.run.smithi086:> if mountpoint in contents: 2024-04-20T21:30:50.545 DEBUG:teuthology.orchestra.run.smithi086:> return f 2024-04-20T21:30:50.545 DEBUG:teuthology.orchestra.run.smithi086:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T21:30:50.545 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:30:50.545 DEBUG:teuthology.orchestra.run.smithi086:> print(_find_admin_socket("client.0")) 2024-04-20T21:30:50.545 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:30:50.727 INFO:teuthology.orchestra.run.smithi086.stdout:/var/run/ceph/ceph-client.0.69175.asok 2024-04-20T21:30:50.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:30:50.742 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.69175.asok 2024-04-20T21:30:50.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:30:50.743 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.69175.asok status 2024-04-20T21:30:50.960 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:30:50.961 INFO:teuthology.orchestra.run.smithi086.stdout: "metadata": { 2024-04-20T21:30:50.961 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T21:30:50.961 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T21:30:50.961 INFO:teuthology.orchestra.run.smithi086.stdout: "entity_id": "0", 2024-04-20T21:30:50.961 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": "smithi086", 2024-04-20T21:30:50.961 INFO:teuthology.orchestra.run.smithi086.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-20T21:30:50.961 INFO:teuthology.orchestra.run.smithi086.stdout: "pid": "69175", 2024-04-20T21:30:50.961 INFO:teuthology.orchestra.run.smithi086.stdout: "root": "/" 2024-04-20T21:30:50.961 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:30:50.961 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_count": 0, 2024-04-20T21:30:50.961 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_pinned_count": 0, 2024-04-20T21:30:50.961 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 9906, 2024-04-20T21:30:50.962 INFO:teuthology.orchestra.run.smithi086.stdout: "inst": { 2024-04-20T21:30:50.962 INFO:teuthology.orchestra.run.smithi086.stdout: "name": { 2024-04-20T21:30:50.962 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "client", 2024-04-20T21:30:50.962 INFO:teuthology.orchestra.run.smithi086.stdout: "num": 9906 2024-04-20T21:30:50.962 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:30:50.962 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2024-04-20T21:30:50.962 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-04-20T21:30:50.962 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2024-04-20T21:30:50.962 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 1562456788 2024-04-20T21:30:50.962 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-20T21:30:50.962 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:30:50.962 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2024-04-20T21:30:50.962 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-04-20T21:30:50.962 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2024-04-20T21:30:50.962 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 1562456788 2024-04-20T21:30:50.963 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-20T21:30:50.963 INFO:teuthology.orchestra.run.smithi086.stdout: "inst_str": "client.9906 192.168.0.1:0/1562456788", 2024-04-20T21:30:50.963 INFO:teuthology.orchestra.run.smithi086.stdout: "addr_str": "192.168.0.1:0/1562456788", 2024-04-20T21:30:50.963 INFO:teuthology.orchestra.run.smithi086.stdout: "inode_count": 1, 2024-04-20T21:30:50.963 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_epoch": 112, 2024-04-20T21:30:50.963 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch": 116, 2024-04-20T21:30:50.963 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch_barrier": 0, 2024-04-20T21:30:50.963 INFO:teuthology.orchestra.run.smithi086.stdout: "blocklisted": false, 2024-04-20T21:30:50.963 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs" 2024-04-20T21:30:50.963 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:30:50.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:30:50.975 DEBUG:teuthology.orchestra.run.smithi086:> stat --file-system '--printf=%T 2024-04-20T21:30:50.975 DEBUG:teuthology.orchestra.run.smithi086:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-20T21:30:51.027 INFO:teuthology.orchestra.run.smithi086.stdout:fuseblk 2024-04-20T21:30:51.028 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-20T21:30:51.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:30:51.028 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-20T21:30:51.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:30:51.109 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-20T21:30:51.183 INFO:teuthology.orchestra.run.smithi086.stdout:./ 2024-04-20T21:30:51.183 INFO:teuthology.orchestra.run.smithi086.stdout:./parent 2024-04-20T21:30:51.183 INFO:teuthology.orchestra.run.smithi086.stdout:./parent/flushed 2024-04-20T21:30:51.183 INFO:teuthology.orchestra.run.smithi086.stdout:./parent/flushed/charlie 2024-04-20T21:30:51.183 INFO:teuthology.orchestra.run.smithi086.stdout:./parent/flushed/alpha 2024-04-20T21:30:51.183 INFO:teuthology.orchestra.run.smithi086.stdout:./parent/flushed/bravo 2024-04-20T21:30:51.183 INFO:teuthology.orchestra.run.smithi086.stdout:./parent/unflushed 2024-04-20T21:30:51.183 INFO:teuthology.orchestra.run.smithi086.stdout:./parent/unflushed/jfile 2024-04-20T21:30:51.184 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-04-20T21:30:51.184 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:30:51.185 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-04-20T21:30:51.185 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:30:51.185 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-20T21:30:51.185 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:30:51.318 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-20T21:30:51.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:30:51.331 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-04-20T21:30:51.331 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:30:51.331 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-04-20T21:30:51.331 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:30:51.332 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-04-20T21:30:51.332 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:30:51.514 INFO:teuthology.orchestra.run.smithi086.stdout:1099511627776 2024-04-20T21:30:51.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:30:51.528 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-04-20T21:30:51.528 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:30:51.528 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-04-20T21:30:51.528 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:30:51.528 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-04-20T21:30:51.528 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:30:51.706 INFO:teuthology.orchestra.run.smithi086.stdout:1099511627777 2024-04-20T21:30:51.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:30:51.720 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-04-20T21:30:51.720 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:30:51.720 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-04-20T21:30:51.720 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:30:51.720 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-04-20T21:30:51.720 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:30:51.904 INFO:teuthology.orchestra.run.smithi086.stdout:1099511628280 2024-04-20T21:30:51.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:30:51.919 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-04-20T21:30:51.919 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:30:51.919 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-04-20T21:30:51.919 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:30:51.919 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-04-20T21:30:51.919 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:30:52.101 INFO:teuthology.orchestra.run.smithi086.stdout:1099511627778 2024-04-20T21:30:52.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:30:52.115 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-04-20T21:30:52.116 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:30:52.116 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-04-20T21:30:52.116 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:30:52.116 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-04-20T21:30:52.116 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:30:52.293 INFO:teuthology.orchestra.run.smithi086.stdout:1099511628279 2024-04-20T21:30:52.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:30:52.307 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-04-20T21:30:52.307 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:30:52.307 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-04-20T21:30:52.307 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:30:52.307 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-04-20T21:30:52.307 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:30:52.490 INFO:teuthology.orchestra.run.smithi086.stdout:1099511627779 2024-04-20T21:30:52.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:30:52.504 DEBUG:teuthology.orchestra.run.smithi086:> stdin-killer -- python3 -c ' 2024-04-20T21:30:52.504 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-20T21:30:52.504 DEBUG:teuthology.orchestra.run.smithi086:> import stat 2024-04-20T21:30:52.504 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-20T21:30:52.505 DEBUG:teuthology.orchestra.run.smithi086:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-04-20T21:30:52.505 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-20T21:30:52.690 INFO:teuthology.orchestra.run.smithi086.stdout:1099511627780 2024-04-20T21:30:52.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T21:30:52.706 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-20T21:30:52.706 DEBUG:teuthology.orchestra.run.smithi086:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T21:30:52.869 DEBUG:teuthology.orchestra.run.smithi144:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T21:30:52.985 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-20T21:30:52.986 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T21:30:52.986 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:30:52.986 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:30:53.043 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:30:53.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.249+0000 7f49fec00640 1 Processor -- start 2024-04-20T21:30:53.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.249+0000 7f49fec00640 1 -- start start 2024-04-20T21:30:53.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.250+0000 7f49fec00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f49f80ad490 0x7f49f80b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:53.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.250+0000 7f49fec00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f49f80b40a0 0x7f49f806efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:53.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.250+0000 7f49fec00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49f80b44b0 con 0x7f49f815aa40 2024-04-20T21:30:53.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.250+0000 7f49fec00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f49f8073ea0 con 0x7f49f80b40a0 2024-04-20T21:30:53.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.250+0000 7f49fec00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f49f8073bc0 con 0x7f49f80ad490 2024-04-20T21:30:53.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.251+0000 7f49fe200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f49f80b40a0 0x7f49f806efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:53.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.251+0000 7f49fe200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f49f80b40a0 0x7f49f806efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49164/0 (socket says 172.21.15.86:49164) 2024-04-20T21:30:53.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.251+0000 7f49fe200640 1 -- 172.21.15.86:0/4066293296 learned_addr learned my addr 172.21.15.86:0/4066293296 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:53.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.251+0000 7f49fce00640 1 --2- 172.21.15.86:0/4066293296 >> v2:172.21.15.144:3300/0 conn(0x7f49f80ad490 0x7f49f80b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:53.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.251+0000 7f49fe200640 1 -- 172.21.15.86:0/4066293296 >> v2:172.21.15.144:3300/0 conn(0x7f49f80ad490 msgr2=0x7f49f80b3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:53.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.251+0000 7f49fe200640 1 --2- 172.21.15.86:0/4066293296 >> v2:172.21.15.144:3300/0 conn(0x7f49f80ad490 0x7f49f80b3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:53.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.251+0000 7f49fe200640 1 -- 172.21.15.86:0/4066293296 >> v1:172.21.15.144:6789/0 conn(0x7f49f815aa40 legacy=0x7f49f8073100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:53.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.251+0000 7f49fe200640 1 -- 172.21.15.86:0/4066293296 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f806e4e0 con 0x7f49f80b40a0 2024-04-20T21:30:53.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.253+0000 7f49fe200640 1 --2- 172.21.15.86:0/4066293296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49f80b40a0 0x7f49f806efc0 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f49f8074020 tx=0x7f49f002da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4b78ae5c924f11c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:53.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.253+0000 7f49f7e00640 1 -- 172.21.15.86:0/4066293296 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f002f850 con 0x7f49f80b40a0 2024-04-20T21:30:53.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.253+0000 7f49f7e00640 1 -- 172.21.15.86:0/4066293296 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49f002f9f0 con 0x7f49f80b40a0 2024-04-20T21:30:53.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.253+0000 7f49f7e00640 1 -- 172.21.15.86:0/4066293296 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f002fdc0 con 0x7f49f80b40a0 2024-04-20T21:30:53.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.253+0000 7f49fec00640 1 -- 172.21.15.86:0/4066293296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49f80b40a0 msgr2=0x7f49f806efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:53.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.253+0000 7f49fec00640 1 --2- 172.21.15.86:0/4066293296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49f80b40a0 0x7f49f806efc0 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f49f8074020 tx=0x7f49f002da90 comp rx=0 tx=0).stop 2024-04-20T21:30:53.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.254+0000 7f49fec00640 1 -- 172.21.15.86:0/4066293296 shutdown_connections 2024-04-20T21:30:53.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.254+0000 7f49fec00640 1 --2- 172.21.15.86:0/4066293296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49f80b40a0 0x7f49f806efc0 unknown :-1 s=CLOSED pgs=2078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:53.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.254+0000 7f49fec00640 1 --2- 172.21.15.86:0/4066293296 >> v2:172.21.15.144:3300/0 conn(0x7f49f80ad490 0x7f49f80b3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:53.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.254+0000 7f49fec00640 1 -- 172.21.15.86:0/4066293296 >> 172.21.15.86:0/4066293296 conn(0x7f49f80d0170 msgr2=0x7f49f80d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:53.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.254+0000 7f49fec00640 1 -- 172.21.15.86:0/4066293296 shutdown_connections 2024-04-20T21:30:53.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.254+0000 7f49fec00640 1 -- 172.21.15.86:0/4066293296 wait complete. 2024-04-20T21:30:53.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.256+0000 7f49fec00640 1 Processor -- start 2024-04-20T21:30:53.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.256+0000 7f49fec00640 1 -- start start 2024-04-20T21:30:53.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.257+0000 7f49fec00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49f80ad490 0x7f49f81492f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:53.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.257+0000 7f49fec00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49f80b40a0 0x7f49f81f4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:53.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.257+0000 7f49fec00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f49f815aa40 0x7f49f8150590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:53.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.257+0000 7f49fec00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f49f80ad920 con 0x7f49f80b40a0 2024-04-20T21:30:53.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.257+0000 7f49fec00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f49f806eaa0 con 0x7f49f80ad490 2024-04-20T21:30:53.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.257+0000 7f49fec00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f49f80b33a0 con 0x7f49f815aa40 2024-04-20T21:30:53.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.257+0000 7f49fce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49f80b40a0 0x7f49f81f4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:53.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.257+0000 7f49fce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49f80b40a0 0x7f49f81f4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49172/0 (socket says 172.21.15.86:49172) 2024-04-20T21:30:53.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.257+0000 7f49fce00640 1 -- 172.21.15.86:0/629788868 learned_addr learned my addr 172.21.15.86:0/629788868 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:53.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.257+0000 7f49fce00640 1 -- 172.21.15.86:0/629788868 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f49f815aa40 msgr2=0x7f49f8150590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:53.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.257+0000 7f49fce00640 1 --2- 172.21.15.86:0/629788868 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f49f815aa40 0x7f49f8150590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:53.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.257+0000 7f49fce00640 1 -- 172.21.15.86:0/629788868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49f80ad490 msgr2=0x7f49f81492f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:53.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.257+0000 7f49fd800640 1 --2- 172.21.15.86:0/629788868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49f80ad490 0x7f49f81492f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:53.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.258+0000 7f49fce00640 1 --2- 172.21.15.86:0/629788868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49f80ad490 0x7f49f81492f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:53.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.258+0000 7f49fce00640 1 -- 172.21.15.86:0/629788868 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f814f6c0 con 0x7f49f80b40a0 2024-04-20T21:30:53.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.258+0000 7f49fce00640 1 --2- 172.21.15.86:0/629788868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49f80b40a0 0x7f49f81f4740 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f49e400d800 tx=0x7f49e400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:53.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.258+0000 7f49f5600640 1 -- 172.21.15.86:0/629788868 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e400b6c0 con 0x7f49f80b40a0 2024-04-20T21:30:53.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.258+0000 7f49f5600640 1 -- 172.21.15.86:0/629788868 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49e4019070 con 0x7f49f80b40a0 2024-04-20T21:30:53.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.258+0000 7f49fec00640 1 -- 172.21.15.86:0/629788868 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49f814f9a0 con 0x7f49f80b40a0 2024-04-20T21:30:53.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.258+0000 7f49f5600640 1 -- 172.21.15.86:0/629788868 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e400ba80 con 0x7f49f80b40a0 2024-04-20T21:30:53.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.258+0000 7f49fec00640 1 -- 172.21.15.86:0/629788868 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49f814ff60 con 0x7f49f80b40a0 2024-04-20T21:30:53.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.259+0000 7f49fec00640 1 -- 172.21.15.86:0/629788868 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49f81501a0 con 0x7f49f80b40a0 2024-04-20T21:30:53.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.260+0000 7f49f5600640 1 -- 172.21.15.86:0/629788868 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f49e400bc20 con 0x7f49f80b40a0 2024-04-20T21:30:53.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.261+0000 7f49f5600640 1 --2- 172.21.15.86:0/629788868 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f49d806c8f0 0x7f49d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:53.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.261+0000 7f49f5600640 1 -- 172.21.15.86:0/629788868 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7f49e408c1d0 con 0x7f49f80b40a0 2024-04-20T21:30:53.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.262+0000 7f49fd800640 1 --2- 172.21.15.86:0/629788868 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f49d806c8f0 0x7f49d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:53.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.262+0000 7f49fd800640 1 --2- 172.21.15.86:0/629788868 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f49d806c8f0 0x7f49d806edb0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f49ec007dc0 tx=0x7f49ec0072f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:53.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.266+0000 7f49f5600640 1 -- 172.21.15.86:0/629788868 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f49f81501a0 con 0x7f49f80b40a0 2024-04-20T21:30:53.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.467+0000 7f49fec00640 1 -- 172.21.15.86:0/629788868 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f49f8151530 con 0x7f49f80b40a0 2024-04-20T21:30:53.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.470+0000 7f49f5600640 1 -- 172.21.15.86:0/629788868 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) ==== 74+0+17037 (secure 0 0 0) 0x7f49e401c360 con 0x7f49f80b40a0 2024-04-20T21:30:53.469 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:53.470 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":116,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:30:42.086516+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:29:41.532824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-20T21:29:42.749687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.f","mappings":[{"from":0,"to":5}]},{"pgid":"17.18","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6837/1453621918":"2024-04-21T21:30:42.086474+0000","172.21.15.86:6835/1453621918":"2024-04-21T21:30:42.086474+0000","172.21.15.86:6837/289590924":"2024-04-21T21:30:19.784403+0000","172.21.15.86:6835/289590924":"2024-04-21T21:30:19.784403+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:30:53.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.473+0000 7f49fec00640 1 -- 172.21.15.86:0/629788868 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f49d806c8f0 msgr2=0x7f49d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:53.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.473+0000 7f49fec00640 1 --2- 172.21.15.86:0/629788868 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f49d806c8f0 0x7f49d806edb0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f49ec007dc0 tx=0x7f49ec0072f0 comp rx=0 tx=0).stop 2024-04-20T21:30:53.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.473+0000 7f49fec00640 1 -- 172.21.15.86:0/629788868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49f80b40a0 msgr2=0x7f49f81f4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:53.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.473+0000 7f49fec00640 1 --2- 172.21.15.86:0/629788868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49f80b40a0 0x7f49f81f4740 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f49e400d800 tx=0x7f49e400dcc0 comp rx=0 tx=0).stop 2024-04-20T21:30:53.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.473+0000 7f49fec00640 1 -- 172.21.15.86:0/629788868 shutdown_connections 2024-04-20T21:30:53.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.473+0000 7f49fec00640 1 --2- 172.21.15.86:0/629788868 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f49d806c8f0 0x7f49d806edb0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:53.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.473+0000 7f49fec00640 1 --2- 172.21.15.86:0/629788868 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f49f815aa40 0x7f49f8150590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:53.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.473+0000 7f49fec00640 1 --2- 172.21.15.86:0/629788868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49f80b40a0 0x7f49f81f4740 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:53.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.473+0000 7f49fec00640 1 --2- 172.21.15.86:0/629788868 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49f80ad490 0x7f49f81492f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:53.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.473+0000 7f49fec00640 1 -- 172.21.15.86:0/629788868 >> 172.21.15.86:0/629788868 conn(0x7f49f80d0170 msgr2=0x7f49f8158ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:53.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.473+0000 7f49fec00640 1 -- 172.21.15.86:0/629788868 shutdown_connections 2024-04-20T21:30:53.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.473+0000 7f49fec00640 1 -- 172.21.15.86:0/629788868 wait complete. 2024-04-20T21:30:53.491 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:29:41.532824+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:29:42.749687+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:30:53.491 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:30:53.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.692+0000 7ff4f7200640 1 Processor -- start 2024-04-20T21:30:53.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.692+0000 7ff4f7200640 1 -- start start 2024-04-20T21:30:53.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.694+0000 7ff4f7200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ff4f006bbf0 0x7ff4f006e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:53.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.694+0000 7ff4f7200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4f006bfd0 0x7ff4f0145b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:53.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.694+0000 7ff4f7200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4f015cd40 con 0x7ff4f0146040 2024-04-20T21:30:53.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.694+0000 7ff4f7200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7ff4f015d000 con 0x7ff4f006bbf0 2024-04-20T21:30:53.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.694+0000 7ff4f7200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff4f015d2e0 con 0x7ff4f006bfd0 2024-04-20T21:30:53.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.695+0000 7ff4f5200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7ff4f0146040 0x7ff4f00cd540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57086/0 (socket says 172.21.15.86:57086) 2024-04-20T21:30:53.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.695+0000 7ff4f5200640 1 -- 172.21.15.86:0/4001555208 learned_addr learned my addr 172.21.15.86:0/4001555208 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:30:53.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.695+0000 7ff4e7400640 1 --2- 172.21.15.86:0/4001555208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4f006bfd0 0x7ff4f0145b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:53.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.695+0000 7ff4efe00640 1 --2- 172.21.15.86:0/4001555208 >> v2:172.21.15.144:3300/0 conn(0x7ff4f006bbf0 0x7ff4f006e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:53.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.695+0000 7ff4ef400640 1 -- 172.21.15.86:0/4001555208 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3348698012 0 0) 0x7ff4f015cd40 con 0x7ff4f0146040 2024-04-20T21:30:53.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.695+0000 7ff4ef400640 1 -- 172.21.15.86:0/4001555208 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff4d4003520 con 0x7ff4f0146040 2024-04-20T21:30:53.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.696+0000 7ff4ef400640 1 -- 172.21.15.86:0/4001555208 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1332776723 0 0) 0x7ff4d4003520 con 0x7ff4f0146040 2024-04-20T21:30:53.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.696+0000 7ff4e7400640 1 -- 172.21.15.86:0/4001555208 >> v2:172.21.15.144:3300/0 conn(0x7ff4f006bbf0 msgr2=0x7ff4f006e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:53.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.696+0000 7ff4e7400640 1 --2- 172.21.15.86:0/4001555208 >> v2:172.21.15.144:3300/0 conn(0x7ff4f006bbf0 0x7ff4f006e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:53.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.696+0000 7ff4e7400640 1 -- 172.21.15.86:0/4001555208 >> v1:172.21.15.86:6789/0 conn(0x7ff4f0146040 legacy=0x7ff4f00cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:53.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.696+0000 7ff4e7400640 1 -- 172.21.15.86:0/4001555208 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4f015d5c0 con 0x7ff4f006bfd0 2024-04-20T21:30:53.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.696+0000 7ff4efe00640 1 --2- 172.21.15.86:0/4001555208 >> v2:172.21.15.144:3300/0 conn(0x7ff4f006bbf0 0x7ff4f006e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:30:53.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.697+0000 7ff4e7400640 1 --2- 172.21.15.86:0/4001555208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4f006bfd0 0x7ff4f0145b00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff4d80097a0 tx=0x7ff4d802b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62a6e9cfede985c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:53.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.697+0000 7ff4ef400640 1 -- 172.21.15.86:0/4001555208 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d8037070 con 0x7ff4f006bfd0 2024-04-20T21:30:53.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.697+0000 7ff4ef400640 1 -- 172.21.15.86:0/4001555208 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4d8030510 con 0x7ff4f006bfd0 2024-04-20T21:30:53.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.698+0000 7ff4ef400640 1 -- 172.21.15.86:0/4001555208 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8001670 con 0x7ff4f006bfd0 2024-04-20T21:30:53.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.698+0000 7ff4f7200640 1 -- 172.21.15.86:0/4001555208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4f006bfd0 msgr2=0x7ff4f0145b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:53.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.698+0000 7ff4f7200640 1 --2- 172.21.15.86:0/4001555208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4f006bfd0 0x7ff4f0145b00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff4d80097a0 tx=0x7ff4d802b790 comp rx=0 tx=0).stop 2024-04-20T21:30:53.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.698+0000 7ff4f7200640 1 -- 172.21.15.86:0/4001555208 shutdown_connections 2024-04-20T21:30:53.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.698+0000 7ff4f7200640 1 --2- 172.21.15.86:0/4001555208 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4f006bfd0 0x7ff4f0145b00 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:53.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.698+0000 7ff4f7200640 1 --2- 172.21.15.86:0/4001555208 >> v2:172.21.15.144:3300/0 conn(0x7ff4f006bbf0 0x7ff4f006e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:53.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.698+0000 7ff4f7200640 1 -- 172.21.15.86:0/4001555208 >> 172.21.15.86:0/4001555208 conn(0x7ff4f00b2110 msgr2=0x7ff4f00b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:53.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.698+0000 7ff4f7200640 1 -- 172.21.15.86:0/4001555208 shutdown_connections 2024-04-20T21:30:53.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.698+0000 7ff4f7200640 1 -- 172.21.15.86:0/4001555208 wait complete. 2024-04-20T21:30:53.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.700+0000 7ff4f7200640 1 Processor -- start 2024-04-20T21:30:53.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.700+0000 7ff4f7200640 1 -- start start 2024-04-20T21:30:53.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.701+0000 7ff4f7200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4f006bbf0 0x7ff4f0142060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:53.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.701+0000 7ff4f7200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4f006bfd0 0x7ff4f01e9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:53.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.701+0000 7ff4f7200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4f0146040 0x7ff4f01e7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:53.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.701+0000 7ff4f7200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f00cddd0 con 0x7ff4f006bfd0 2024-04-20T21:30:53.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.701+0000 7ff4f7200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f00cdc50 con 0x7ff4f006bbf0 2024-04-20T21:30:53.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.701+0000 7ff4f7200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff4f00cdf50 con 0x7ff4f0146040 2024-04-20T21:30:53.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.702+0000 7ff4e7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4f006bfd0 0x7ff4f01e9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:53.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.702+0000 7ff4e7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4f006bfd0 0x7ff4f01e9240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49186/0 (socket says 172.21.15.86:49186) 2024-04-20T21:30:53.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.702+0000 7ff4e7400640 1 -- 172.21.15.86:0/4198839997 learned_addr learned my addr 172.21.15.86:0/4198839997 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:53.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.702+0000 7ff4efe00640 1 --2- 172.21.15.86:0/4198839997 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4f006bbf0 0x7ff4f0142060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:53.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.702+0000 7ff4f5200640 1 --2- 172.21.15.86:0/4198839997 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4f0146040 0x7ff4f01e7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:53.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.702+0000 7ff4e7400640 1 -- 172.21.15.86:0/4198839997 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4f0146040 msgr2=0x7ff4f01e7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:53.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.702+0000 7ff4e7400640 1 --2- 172.21.15.86:0/4198839997 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4f0146040 0x7ff4f01e7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:53.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.702+0000 7ff4e7400640 1 -- 172.21.15.86:0/4198839997 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4f006bbf0 msgr2=0x7ff4f0142060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:53.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.702+0000 7ff4e7400640 1 --2- 172.21.15.86:0/4198839997 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4f006bbf0 0x7ff4f0142060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:53.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.702+0000 7ff4e7400640 1 -- 172.21.15.86:0/4198839997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4f01e9cb0 con 0x7ff4f006bfd0 2024-04-20T21:30:53.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.703+0000 7ff4e7400640 1 --2- 172.21.15.86:0/4198839997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4f006bfd0 0x7ff4f01e9240 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7ff4d8000c00 tx=0x7ff4d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:53.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.703+0000 7ff4ecc00640 1 -- 172.21.15.86:0/4198839997 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d8041020 con 0x7ff4f006bfd0 2024-04-20T21:30:53.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.703+0000 7ff4ecc00640 1 -- 172.21.15.86:0/4198839997 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4d803c870 con 0x7ff4f006bfd0 2024-04-20T21:30:53.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.703+0000 7ff4ecc00640 1 -- 172.21.15.86:0/4198839997 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d8037040 con 0x7ff4f006bfd0 2024-04-20T21:30:53.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.703+0000 7ff4f7200640 1 -- 172.21.15.86:0/4198839997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4f01e9f90 con 0x7ff4f006bfd0 2024-04-20T21:30:53.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.703+0000 7ff4f7200640 1 -- 172.21.15.86:0/4198839997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4f01ea550 con 0x7ff4f006bfd0 2024-04-20T21:30:53.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.704+0000 7ff4f7200640 1 -- 172.21.15.86:0/4198839997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4f01ea930 con 0x7ff4f006bfd0 2024-04-20T21:30:53.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.705+0000 7ff4ecc00640 1 -- 172.21.15.86:0/4198839997 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff4d8044020 con 0x7ff4f006bfd0 2024-04-20T21:30:53.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.706+0000 7ff4ecc00640 1 --2- 172.21.15.86:0/4198839997 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff4cc06c920 0x7ff4cc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:53.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.706+0000 7ff4ecc00640 1 -- 172.21.15.86:0/4198839997 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7ff4d80ad440 con 0x7ff4f006bfd0 2024-04-20T21:30:53.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.706+0000 7ff4efe00640 1 --2- 172.21.15.86:0/4198839997 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff4cc06c920 0x7ff4cc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:53.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.707+0000 7ff4efe00640 1 --2- 172.21.15.86:0/4198839997 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff4cc06c920 0x7ff4cc06ede0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ff4f00cb2a0 tx=0x7ff4e000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:53.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.710+0000 7ff4ecc00640 1 -- 172.21.15.86:0/4198839997 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff4f01ea930 con 0x7ff4f006bfd0 2024-04-20T21:30:53.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.911+0000 7ff4f7200640 1 -- 172.21.15.86:0/4198839997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff4f00689e0 con 0x7ff4f006bfd0 2024-04-20T21:30:53.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.912+0000 7ff4ecc00640 1 -- 172.21.15.86:0/4198839997 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v116) ==== 93+0+10 (secure 0 0 0) 0x7ff4d804b070 con 0x7ff4f006bfd0 2024-04-20T21:30:53.911 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:30:53.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.914+0000 7ff4f7200640 1 -- 172.21.15.86:0/4198839997 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff4cc06c920 msgr2=0x7ff4cc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:53.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.914+0000 7ff4f7200640 1 --2- 172.21.15.86:0/4198839997 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff4cc06c920 0x7ff4cc06ede0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ff4f00cb2a0 tx=0x7ff4e000b040 comp rx=0 tx=0).stop 2024-04-20T21:30:53.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.914+0000 7ff4f7200640 1 -- 172.21.15.86:0/4198839997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4f006bfd0 msgr2=0x7ff4f01e9240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:53.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.914+0000 7ff4f7200640 1 --2- 172.21.15.86:0/4198839997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4f006bfd0 0x7ff4f01e9240 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7ff4d8000c00 tx=0x7ff4d80046c0 comp rx=0 tx=0).stop 2024-04-20T21:30:53.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.915+0000 7ff4f7200640 1 -- 172.21.15.86:0/4198839997 shutdown_connections 2024-04-20T21:30:53.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.915+0000 7ff4f7200640 1 --2- 172.21.15.86:0/4198839997 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff4cc06c920 0x7ff4cc06ede0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:53.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.915+0000 7ff4f7200640 1 --2- 172.21.15.86:0/4198839997 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4f0146040 0x7ff4f01e7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:53.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.915+0000 7ff4f7200640 1 --2- 172.21.15.86:0/4198839997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4f006bfd0 0x7ff4f01e9240 unknown :-1 s=CLOSED pgs=2081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:53.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.915+0000 7ff4f7200640 1 --2- 172.21.15.86:0/4198839997 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4f006bbf0 0x7ff4f0142060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:53.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.915+0000 7ff4f7200640 1 -- 172.21.15.86:0/4198839997 >> 172.21.15.86:0/4198839997 conn(0x7ff4f00b2110 msgr2=0x7ff4f015f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:53.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.915+0000 7ff4f7200640 1 -- 172.21.15.86:0/4198839997 shutdown_connections 2024-04-20T21:30:53.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:53.915+0000 7ff4f7200640 1 -- 172.21.15.86:0/4198839997 wait complete. 2024-04-20T21:30:53.933 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:30:54.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.136+0000 7f01eb200640 1 Processor -- start 2024-04-20T21:30:54.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.137+0000 7f01eb200640 1 -- start start 2024-04-20T21:30:54.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.138+0000 7f01eb200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01ec145250 0x7f01ec06efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:54.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.138+0000 7f01eb200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f01ec145f00 0x7f01ec071d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:54.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.138+0000 7f01eb200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01ec06f500 con 0x7f01ec072280 2024-04-20T21:30:54.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.138+0000 7f01eb200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f01ec160410 con 0x7f01ec145f00 2024-04-20T21:30:54.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.138+0000 7f01eb200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f01ec1606f0 con 0x7f01ec145250 2024-04-20T21:30:54.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.138+0000 7f01e9400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f01ec145f00 0x7f01ec071d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:54.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.138+0000 7f01ea800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f01ec072280 0x7f01ec06dda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57092/0 (socket says 172.21.15.86:57092) 2024-04-20T21:30:54.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.138+0000 7f01ea800640 1 -- 172.21.15.86:0/2929510898 learned_addr learned my addr 172.21.15.86:0/2929510898 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:30:54.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.139+0000 7f01e9e00640 1 --2- 172.21.15.86:0/2929510898 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01ec145250 0x7f01ec06efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:54.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.139+0000 7f01e9400640 1 -- 172.21.15.86:0/2929510898 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01ec145250 msgr2=0x7f01ec06efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:54.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.139+0000 7f01e9400640 1 --2- 172.21.15.86:0/2929510898 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01ec145250 0x7f01ec06efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:54.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.139+0000 7f01e9400640 1 -- 172.21.15.86:0/2929510898 >> v1:172.21.15.86:6789/0 conn(0x7f01ec072280 legacy=0x7f01ec06dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:54.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.139+0000 7f01e8a00640 1 -- 172.21.15.86:0/2929510898 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3665616883 0 0) 0x7f01e4001490 con 0x7f01ec072280 2024-04-20T21:30:54.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.139+0000 7f01e9400640 1 -- 172.21.15.86:0/2929510898 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01ec1609d0 con 0x7f01ec145f00 2024-04-20T21:30:54.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.140+0000 7f01e9400640 1 --2- 172.21.15.86:0/2929510898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01ec145f00 0x7f01ec071d40 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f01d80097a0 tx=0x7f01d802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a11e368108b46a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:54.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.140+0000 7f01e8a00640 1 -- 172.21.15.86:0/2929510898 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d80315a0 con 0x7f01ec145f00 2024-04-20T21:30:54.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.140+0000 7f01e8a00640 1 -- 172.21.15.86:0/2929510898 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01d8037070 con 0x7f01ec145f00 2024-04-20T21:30:54.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.140+0000 7f01e8a00640 1 -- 172.21.15.86:0/2929510898 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d8031960 con 0x7f01ec145f00 2024-04-20T21:30:54.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.141+0000 7f01eb200640 1 -- 172.21.15.86:0/2929510898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01ec145f00 msgr2=0x7f01ec071d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:54.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.141+0000 7f01eb200640 1 --2- 172.21.15.86:0/2929510898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01ec145f00 0x7f01ec071d40 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f01d80097a0 tx=0x7f01d802b840 comp rx=0 tx=0).stop 2024-04-20T21:30:54.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.141+0000 7f01eb200640 1 -- 172.21.15.86:0/2929510898 shutdown_connections 2024-04-20T21:30:54.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.141+0000 7f01eb200640 1 --2- 172.21.15.86:0/2929510898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01ec145f00 0x7f01ec071d40 unknown :-1 s=CLOSED pgs=2083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:54.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.141+0000 7f01eb200640 1 --2- 172.21.15.86:0/2929510898 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01ec145250 0x7f01ec06efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:54.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.141+0000 7f01eb200640 1 -- 172.21.15.86:0/2929510898 >> 172.21.15.86:0/2929510898 conn(0x7f01ec067cf0 msgr2=0x7f01ec06b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:54.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.141+0000 7f01eb200640 1 -- 172.21.15.86:0/2929510898 shutdown_connections 2024-04-20T21:30:54.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.141+0000 7f01eb200640 1 -- 172.21.15.86:0/2929510898 wait complete. 2024-04-20T21:30:54.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.143+0000 7f01eb200640 1 Processor -- start 2024-04-20T21:30:54.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.143+0000 7f01eb200640 1 -- start start 2024-04-20T21:30:54.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.144+0000 7f01eb200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01ec072280 0x7f01ec1395b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:54.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.144+0000 7f01eb200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01ec145250 0x7f01ec0b45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:54.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.144+0000 7f01eb200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01ec145f00 0x7f01ec0b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:54.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.144+0000 7f01eb200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f01ec1457b0 con 0x7f01ec145f00 2024-04-20T21:30:54.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.144+0000 7f01eb200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f01ec145630 con 0x7f01ec145250 2024-04-20T21:30:54.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.144+0000 7f01eb200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f01ec15de00 con 0x7f01ec072280 2024-04-20T21:30:54.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.144+0000 7f01ea800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01ec145f00 0x7f01ec0b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:54.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.144+0000 7f01ea800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01ec145f00 0x7f01ec0b4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49210/0 (socket says 172.21.15.86:49210) 2024-04-20T21:30:54.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.144+0000 7f01ea800640 1 -- 172.21.15.86:0/3959289 learned_addr learned my addr 172.21.15.86:0/3959289 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:54.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.145+0000 7f01e9400640 1 --2- 172.21.15.86:0/3959289 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01ec145250 0x7f01ec0b45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:54.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.145+0000 7f01e9e00640 1 --2- 172.21.15.86:0/3959289 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01ec072280 0x7f01ec1395b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:54.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.145+0000 7f01ea800640 1 -- 172.21.15.86:0/3959289 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01ec072280 msgr2=0x7f01ec1395b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:54.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.145+0000 7f01ea800640 1 --2- 172.21.15.86:0/3959289 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01ec072280 0x7f01ec1395b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:54.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.145+0000 7f01ea800640 1 -- 172.21.15.86:0/3959289 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01ec145250 msgr2=0x7f01ec0b45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:54.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.145+0000 7f01ea800640 1 --2- 172.21.15.86:0/3959289 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01ec145250 0x7f01ec0b45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:54.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.145+0000 7f01ea800640 1 -- 172.21.15.86:0/3959289 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01ec1f7c30 con 0x7f01ec145f00 2024-04-20T21:30:54.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.145+0000 7f01e9400640 1 --2- 172.21.15.86:0/3959289 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01ec145250 0x7f01ec0b45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:30:54.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.145+0000 7f01ea800640 1 --2- 172.21.15.86:0/3959289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01ec145f00 0x7f01ec0b4b10 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f01e400c7d0 tx=0x7f01e400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:54.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.145+0000 7f01e2000640 1 -- 172.21.15.86:0/3959289 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e4018070 con 0x7f01ec145f00 2024-04-20T21:30:54.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.145+0000 7f01e2000640 1 -- 172.21.15.86:0/3959289 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01e400a640 con 0x7f01ec145f00 2024-04-20T21:30:54.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.145+0000 7f01eb200640 1 -- 172.21.15.86:0/3959289 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01ec1f7ef0 con 0x7f01ec145f00 2024-04-20T21:30:54.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.145+0000 7f01e2000640 1 -- 172.21.15.86:0/3959289 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e400aab0 con 0x7f01ec145f00 2024-04-20T21:30:54.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.146+0000 7f01eb200640 1 -- 172.21.15.86:0/3959289 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01ec1f84b0 con 0x7f01ec145f00 2024-04-20T21:30:54.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.147+0000 7f01eb200640 1 -- 172.21.15.86:0/3959289 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01ec04fd90 con 0x7f01ec145f00 2024-04-20T21:30:54.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.148+0000 7f01e2000640 1 -- 172.21.15.86:0/3959289 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f01e401b070 con 0x7f01ec145f00 2024-04-20T21:30:54.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.149+0000 7f01e2000640 1 --2- 172.21.15.86:0/3959289 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f01cc06c8f0 0x7f01cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:54.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.149+0000 7f01e2000640 1 -- 172.21.15.86:0/3959289 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7f01e408b280 con 0x7f01ec145f00 2024-04-20T21:30:54.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.149+0000 7f01e9e00640 1 --2- 172.21.15.86:0/3959289 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f01cc06c8f0 0x7f01cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:54.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.150+0000 7f01e9e00640 1 --2- 172.21.15.86:0/3959289 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f01cc06c8f0 0x7f01cc06edb0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f01dc006fd0 tx=0x7f01dc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:54.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.154+0000 7f01e2000640 1 -- 172.21.15.86:0/3959289 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f01ec06f500 con 0x7f01ec145f00 2024-04-20T21:30:54.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.350+0000 7f01eb200640 1 -- 172.21.15.86:0/3959289 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f01ec15fc80 con 0x7f01ec145f00 2024-04-20T21:30:54.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.351+0000 7f01e2000640 1 -- 172.21.15.86:0/3959289 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v116) ==== 104+0+11 (secure 0 0 0) 0x7f01e40595c0 con 0x7f01ec145f00 2024-04-20T21:30:54.350 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-20T21:30:54.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.353+0000 7f01eb200640 1 -- 172.21.15.86:0/3959289 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f01cc06c8f0 msgr2=0x7f01cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:54.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.353+0000 7f01eb200640 1 --2- 172.21.15.86:0/3959289 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f01cc06c8f0 0x7f01cc06edb0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f01dc006fd0 tx=0x7f01dc008040 comp rx=0 tx=0).stop 2024-04-20T21:30:54.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.353+0000 7f01eb200640 1 -- 172.21.15.86:0/3959289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01ec145f00 msgr2=0x7f01ec0b4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:54.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.353+0000 7f01eb200640 1 --2- 172.21.15.86:0/3959289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01ec145f00 0x7f01ec0b4b10 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f01e400c7d0 tx=0x7f01e400cc90 comp rx=0 tx=0).stop 2024-04-20T21:30:54.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.354+0000 7f01eb200640 1 -- 172.21.15.86:0/3959289 shutdown_connections 2024-04-20T21:30:54.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.354+0000 7f01eb200640 1 --2- 172.21.15.86:0/3959289 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f01cc06c8f0 0x7f01cc06edb0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:54.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.354+0000 7f01eb200640 1 --2- 172.21.15.86:0/3959289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f01ec145f00 0x7f01ec0b4b10 unknown :-1 s=CLOSED pgs=2084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:54.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.354+0000 7f01eb200640 1 --2- 172.21.15.86:0/3959289 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01ec145250 0x7f01ec0b45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:54.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.354+0000 7f01eb200640 1 --2- 172.21.15.86:0/3959289 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f01ec072280 0x7f01ec1395b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:54.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.354+0000 7f01eb200640 1 -- 172.21.15.86:0/3959289 >> 172.21.15.86:0/3959289 conn(0x7f01ec067cf0 msgr2=0x7f01ec1433a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:54.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.354+0000 7f01eb200640 1 -- 172.21.15.86:0/3959289 shutdown_connections 2024-04-20T21:30:54.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.354+0000 7f01eb200640 1 -- 172.21.15.86:0/3959289 wait complete. 2024-04-20T21:30:54.371 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:30:54.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.575+0000 7f915f600640 1 Processor -- start 2024-04-20T21:30:54.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.575+0000 7f915f600640 1 -- start start 2024-04-20T21:30:54.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.577+0000 7f915f600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f915806bbf0 0x7f915806e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:54.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.577+0000 7f915f600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9158146210 0x7f91580cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:54.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.577+0000 7f915f600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f915815cd40 con 0x7f915806bfd0 2024-04-20T21:30:54.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.577+0000 7f915f600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f915815d000 con 0x7f9158146210 2024-04-20T21:30:54.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.577+0000 7f915f600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f915815d2e0 con 0x7f915806bbf0 2024-04-20T21:30:54.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.577+0000 7f915d600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9158146210 0x7f91580cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:54.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.577+0000 7f915d600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9158146210 0x7f91580cd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49214/0 (socket says 172.21.15.86:49214) 2024-04-20T21:30:54.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.577+0000 7f915d600640 1 -- 172.21.15.86:0/1487690578 learned_addr learned my addr 172.21.15.86:0/1487690578 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:54.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.577+0000 7f915cc00640 1 --2- 172.21.15.86:0/1487690578 >> v2:172.21.15.144:3300/0 conn(0x7f915806bbf0 0x7f915806e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:54.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.577+0000 7f915d600640 1 -- 172.21.15.86:0/1487690578 >> v2:172.21.15.144:3300/0 conn(0x7f915806bbf0 msgr2=0x7f915806e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:54.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.577+0000 7f915d600640 1 --2- 172.21.15.86:0/1487690578 >> v2:172.21.15.144:3300/0 conn(0x7f915806bbf0 0x7f915806e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:54.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.577+0000 7f915d600640 1 -- 172.21.15.86:0/1487690578 >> v1:172.21.15.144:6789/0 conn(0x7f915806bfd0 legacy=0x7f9158145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:54.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.577+0000 7f915d600640 1 -- 172.21.15.86:0/1487690578 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f915815d5c0 con 0x7f9158146210 2024-04-20T21:30:54.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.579+0000 7f915d600640 1 --2- 172.21.15.86:0/1487690578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9158146210 0x7f91580cd540 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f9158073150 tx=0x7f915002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66f726c03c47255c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:54.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.579+0000 7f9157400640 1 -- 172.21.15.86:0/1487690578 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915002ca80 con 0x7f9158146210 2024-04-20T21:30:54.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.579+0000 7f9157400640 1 -- 172.21.15.86:0/1487690578 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f915002cc20 con 0x7f9158146210 2024-04-20T21:30:54.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.579+0000 7f9157400640 1 -- 172.21.15.86:0/1487690578 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91500343f0 con 0x7f9158146210 2024-04-20T21:30:54.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.580+0000 7f915f600640 1 -- 172.21.15.86:0/1487690578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9158146210 msgr2=0x7f91580cd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:54.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.580+0000 7f915f600640 1 --2- 172.21.15.86:0/1487690578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9158146210 0x7f91580cd540 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f9158073150 tx=0x7f915002bee0 comp rx=0 tx=0).stop 2024-04-20T21:30:54.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.580+0000 7f915f600640 1 -- 172.21.15.86:0/1487690578 shutdown_connections 2024-04-20T21:30:54.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.580+0000 7f915f600640 1 --2- 172.21.15.86:0/1487690578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9158146210 0x7f91580cd540 unknown :-1 s=CLOSED pgs=2085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:54.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.580+0000 7f915f600640 1 --2- 172.21.15.86:0/1487690578 >> v2:172.21.15.144:3300/0 conn(0x7f915806bbf0 0x7f915806e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:54.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.580+0000 7f915f600640 1 -- 172.21.15.86:0/1487690578 >> 172.21.15.86:0/1487690578 conn(0x7f91580b2110 msgr2=0x7f91580b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:54.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.580+0000 7f915f600640 1 -- 172.21.15.86:0/1487690578 shutdown_connections 2024-04-20T21:30:54.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.580+0000 7f915f600640 1 -- 172.21.15.86:0/1487690578 wait complete. 2024-04-20T21:30:54.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.581+0000 7f915f600640 1 Processor -- start 2024-04-20T21:30:54.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.582+0000 7f915f600640 1 -- start start 2024-04-20T21:30:54.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.582+0000 7f915f600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f915806bbf0 0x7f91581e8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:54.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.583+0000 7f915f600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f915806bfd0 0x7f915815c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:54.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.583+0000 7f915f600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9158146210 0x7f91581e9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:54.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.583+0000 7f915f600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f91580cdc00 con 0x7f9158146210 2024-04-20T21:30:54.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.583+0000 7f915f600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f91580cda80 con 0x7f915806bbf0 2024-04-20T21:30:54.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.583+0000 7f915f600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f91580cdd80 con 0x7f915806bfd0 2024-04-20T21:30:54.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.583+0000 7f915d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9158146210 0x7f91581e9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:54.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.583+0000 7f915d600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9158146210 0x7f91581e9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49216/0 (socket says 172.21.15.86:49216) 2024-04-20T21:30:54.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.583+0000 7f915d600640 1 -- 172.21.15.86:0/2683042977 learned_addr learned my addr 172.21.15.86:0/2683042977 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:54.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.583+0000 7f915d600640 1 -- 172.21.15.86:0/2683042977 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f915806bfd0 msgr2=0x7f915815c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:54.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.583+0000 7f915d600640 1 --2- 172.21.15.86:0/2683042977 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f915806bfd0 0x7f915815c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:54.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.583+0000 7f915d600640 1 -- 172.21.15.86:0/2683042977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f915806bbf0 msgr2=0x7f91581e8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:54.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.583+0000 7f915d600640 1 --2- 172.21.15.86:0/2683042977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f915806bbf0 0x7f91581e8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:54.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.583+0000 7f915d600640 1 -- 172.21.15.86:0/2683042977 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91581e87c0 con 0x7f9158146210 2024-04-20T21:30:54.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.584+0000 7f915d600640 1 --2- 172.21.15.86:0/2683042977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9158146210 0x7f91581e9860 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f9150009f40 tx=0x7f9150004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:54.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.584+0000 7f9154c00640 1 -- 172.21.15.86:0/2683042977 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9150043020 con 0x7f9158146210 2024-04-20T21:30:54.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.584+0000 7f9154c00640 1 -- 172.21.15.86:0/2683042977 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f915003e450 con 0x7f9158146210 2024-04-20T21:30:54.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.584+0000 7f9154c00640 1 -- 172.21.15.86:0/2683042977 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915003e820 con 0x7f9158146210 2024-04-20T21:30:54.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.584+0000 7f915f600640 1 -- 172.21.15.86:0/2683042977 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91581e8aa0 con 0x7f9158146210 2024-04-20T21:30:54.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.584+0000 7f915f600640 1 -- 172.21.15.86:0/2683042977 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91581e9060 con 0x7f9158146210 2024-04-20T21:30:54.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.585+0000 7f915f600640 1 -- 172.21.15.86:0/2683042977 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91580cf3e0 con 0x7f9158146210 2024-04-20T21:30:54.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.586+0000 7f9154c00640 1 -- 172.21.15.86:0/2683042977 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9150046020 con 0x7f9158146210 2024-04-20T21:30:54.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.587+0000 7f9154c00640 1 --2- 172.21.15.86:0/2683042977 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f913806c8f0 0x7f913806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:54.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.587+0000 7f9154c00640 1 -- 172.21.15.86:0/2683042977 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7f91500ae540 con 0x7f9158146210 2024-04-20T21:30:54.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.587+0000 7f915cc00640 1 --2- 172.21.15.86:0/2683042977 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f913806c8f0 0x7f913806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:54.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.588+0000 7f915cc00640 1 --2- 172.21.15.86:0/2683042977 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f913806c8f0 0x7f913806edb0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f914c00a810 tx=0x7f914c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:54.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.592+0000 7f9154c00640 1 -- 172.21.15.86:0/2683042977 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f91580cf3e0 con 0x7f9158146210 2024-04-20T21:30:54.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.790+0000 7f915f600640 1 -- 172.21.15.86:0/2683042977 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f91580689e0 con 0x7f9158146210 2024-04-20T21:30:54.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.791+0000 7f9154c00640 1 -- 172.21.15.86:0/2683042977 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v116) ==== 100+0+11 (secure 0 0 0) 0x7f915004d030 con 0x7f9158146210 2024-04-20T21:30:54.790 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-20T21:30:54.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.793+0000 7f915f600640 1 -- 172.21.15.86:0/2683042977 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f913806c8f0 msgr2=0x7f913806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:54.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.793+0000 7f915f600640 1 --2- 172.21.15.86:0/2683042977 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f913806c8f0 0x7f913806edb0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f914c00a810 tx=0x7f914c008040 comp rx=0 tx=0).stop 2024-04-20T21:30:54.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.793+0000 7f915f600640 1 -- 172.21.15.86:0/2683042977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9158146210 msgr2=0x7f91581e9860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:54.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.793+0000 7f915f600640 1 --2- 172.21.15.86:0/2683042977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9158146210 0x7f91581e9860 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f9150009f40 tx=0x7f9150004a40 comp rx=0 tx=0).stop 2024-04-20T21:30:54.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.794+0000 7f915f600640 1 -- 172.21.15.86:0/2683042977 shutdown_connections 2024-04-20T21:30:54.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.794+0000 7f915f600640 1 --2- 172.21.15.86:0/2683042977 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f913806c8f0 0x7f913806edb0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:54.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.794+0000 7f915f600640 1 --2- 172.21.15.86:0/2683042977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9158146210 0x7f91581e9860 unknown :-1 s=CLOSED pgs=2086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:54.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.794+0000 7f915f600640 1 --2- 172.21.15.86:0/2683042977 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f915806bfd0 0x7f915815c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:54.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.794+0000 7f915f600640 1 --2- 172.21.15.86:0/2683042977 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f915806bbf0 0x7f91581e8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:54.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.794+0000 7f915f600640 1 -- 172.21.15.86:0/2683042977 >> 172.21.15.86:0/2683042977 conn(0x7f91580b2110 msgr2=0x7f915806e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:54.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.794+0000 7f915f600640 1 -- 172.21.15.86:0/2683042977 shutdown_connections 2024-04-20T21:30:54.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:54.794+0000 7f915f600640 1 -- 172.21.15.86:0/2683042977 wait complete. 2024-04-20T21:30:54.812 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T21:30:55.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.016+0000 7f0a6ca00640 1 Processor -- start 2024-04-20T21:30:55.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.016+0000 7f0a6ca00640 1 -- start start 2024-04-20T21:30:55.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.017+0000 7f0a6ca00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0a68073540 0x7f0a680d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:55.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.017+0000 7f0a6ca00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a680cfa00 0x7f0a680d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:55.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.018+0000 7f0a6ca00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a680ce1c0 con 0x7f0a680af870 2024-04-20T21:30:55.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.018+0000 7f0a6ca00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f0a680ce4a0 con 0x7f0a68073540 2024-04-20T21:30:55.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.018+0000 7f0a6ca00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0a680ce780 con 0x7f0a680cfa00 2024-04-20T21:30:55.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.018+0000 7f0a66800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0a68073540 0x7f0a680d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:55.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.018+0000 7f0a66800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0a68073540 0x7f0a680d00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49226/0 (socket says 172.21.15.86:49226) 2024-04-20T21:30:55.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.018+0000 7f0a66800640 1 -- 172.21.15.86:0/2477408013 learned_addr learned my addr 172.21.15.86:0/2477408013 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:55.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.018+0000 7f0a67200640 1 --2- 172.21.15.86:0/2477408013 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a680cfa00 0x7f0a680d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:55.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.018+0000 7f0a66800640 1 -- 172.21.15.86:0/2477408013 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a680cfa00 msgr2=0x7f0a680d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:55.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.018+0000 7f0a66800640 1 --2- 172.21.15.86:0/2477408013 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a680cfa00 0x7f0a680d05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:55.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.018+0000 7f0a66800640 1 -- 172.21.15.86:0/2477408013 >> v1:172.21.15.86:6789/0 conn(0x7f0a680af870 legacy=0x7f0a6815f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:55.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.018+0000 7f0a66800640 1 -- 172.21.15.86:0/2477408013 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a680cea60 con 0x7f0a68073540 2024-04-20T21:30:55.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.018+0000 7f0a67200640 1 --2- 172.21.15.86:0/2477408013 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a680cfa00 0x7f0a680d05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:30:55.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.018+0000 7f0a65400640 1 -- 172.21.15.86:0/2477408013 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1471374960 0 0) 0x7f0a680ce1c0 con 0x7f0a680af870 2024-04-20T21:30:55.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.020+0000 7f0a66800640 1 --2- 172.21.15.86:0/2477408013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a68073540 0x7f0a680d00b0 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7f0a5800bb10 tx=0x7f0a5802e150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36fee3d0d4428920 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:55.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.020+0000 7f0a65400640 1 -- 172.21.15.86:0/2477408013 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a5803b070 con 0x7f0a68073540 2024-04-20T21:30:55.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.020+0000 7f0a65400640 1 -- 172.21.15.86:0/2477408013 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a58036440 con 0x7f0a68073540 2024-04-20T21:30:55.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.020+0000 7f0a65400640 1 -- 172.21.15.86:0/2477408013 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a58036810 con 0x7f0a68073540 2024-04-20T21:30:55.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.020+0000 7f0a6ca00640 1 -- 172.21.15.86:0/2477408013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a68073540 msgr2=0x7f0a680d00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:55.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.020+0000 7f0a6ca00640 1 --2- 172.21.15.86:0/2477408013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a68073540 0x7f0a680d00b0 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7f0a5800bb10 tx=0x7f0a5802e150 comp rx=0 tx=0).stop 2024-04-20T21:30:55.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.020+0000 7f0a6ca00640 1 -- 172.21.15.86:0/2477408013 shutdown_connections 2024-04-20T21:30:55.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.021+0000 7f0a6ca00640 1 --2- 172.21.15.86:0/2477408013 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a680cfa00 0x7f0a680d05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:55.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.021+0000 7f0a6ca00640 1 --2- 172.21.15.86:0/2477408013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a68073540 0x7f0a680d00b0 unknown :-1 s=CLOSED pgs=2088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:55.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.021+0000 7f0a6ca00640 1 -- 172.21.15.86:0/2477408013 >> 172.21.15.86:0/2477408013 conn(0x7f0a6806bb60 msgr2=0x7f0a6806bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:55.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.021+0000 7f0a6ca00640 1 -- 172.21.15.86:0/2477408013 shutdown_connections 2024-04-20T21:30:55.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.021+0000 7f0a6ca00640 1 -- 172.21.15.86:0/2477408013 wait complete. 2024-04-20T21:30:55.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.022+0000 7f0a6ca00640 1 Processor -- start 2024-04-20T21:30:55.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.022+0000 7f0a6ca00640 1 -- start start 2024-04-20T21:30:55.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.023+0000 7f0a6ca00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a68073540 0x7f0a6815b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:55.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.023+0000 7f0a6ca00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a680cfa00 0x7f0a680c5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:55.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.023+0000 7f0a6ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a680c8530 0x7f0a6806e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:55.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.023+0000 7f0a6ca00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0a68072b80 con 0x7f0a680c8530 2024-04-20T21:30:55.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.023+0000 7f0a6ca00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0a68073920 con 0x7f0a68073540 2024-04-20T21:30:55.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.023+0000 7f0a6ca00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0a68099830 con 0x7f0a680cfa00 2024-04-20T21:30:55.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.024+0000 7f0a67200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a680c8530 0x7f0a6806e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:55.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.024+0000 7f0a67200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a680c8530 0x7f0a6806e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49240/0 (socket says 172.21.15.86:49240) 2024-04-20T21:30:55.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.024+0000 7f0a67200640 1 -- 172.21.15.86:0/2872882444 learned_addr learned my addr 172.21.15.86:0/2872882444 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:55.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.024+0000 7f0a67200640 1 -- 172.21.15.86:0/2872882444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a680cfa00 msgr2=0x7f0a680c5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:55.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.024+0000 7f0a67200640 1 --2- 172.21.15.86:0/2872882444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a680cfa00 0x7f0a680c5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:55.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.024+0000 7f0a67200640 1 -- 172.21.15.86:0/2872882444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a68073540 msgr2=0x7f0a6815b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:55.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.024+0000 7f0a67200640 1 --2- 172.21.15.86:0/2872882444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a68073540 0x7f0a6815b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:55.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.024+0000 7f0a67200640 1 -- 172.21.15.86:0/2872882444 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a680c9b70 con 0x7f0a680c8530 2024-04-20T21:30:55.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.024+0000 7f0a67200640 1 --2- 172.21.15.86:0/2872882444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a680c8530 0x7f0a6806e2b0 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f0a6000cdb0 tx=0x7f0a6000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:55.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.025+0000 7f0a5ea00640 1 -- 172.21.15.86:0/2872882444 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6000f680 con 0x7f0a680c8530 2024-04-20T21:30:55.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.025+0000 7f0a5ea00640 1 -- 172.21.15.86:0/2872882444 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a6000f840 con 0x7f0a680c8530 2024-04-20T21:30:55.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.025+0000 7f0a5ea00640 1 -- 172.21.15.86:0/2872882444 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6000fc30 con 0x7f0a680c8530 2024-04-20T21:30:55.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.025+0000 7f0a6ca00640 1 -- 172.21.15.86:0/2872882444 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a680c8ad0 con 0x7f0a680c8530 2024-04-20T21:30:55.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.025+0000 7f0a6ca00640 1 -- 172.21.15.86:0/2872882444 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a680c9090 con 0x7f0a680c8530 2024-04-20T21:30:55.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.026+0000 7f0a6ca00640 1 -- 172.21.15.86:0/2872882444 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a6804fcf0 con 0x7f0a680c8530 2024-04-20T21:30:55.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.027+0000 7f0a5ea00640 1 -- 172.21.15.86:0/2872882444 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0a6000e3e0 con 0x7f0a680c8530 2024-04-20T21:30:55.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.028+0000 7f0a5ea00640 1 --2- 172.21.15.86:0/2872882444 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0a4806c8f0 0x7f0a4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:55.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.028+0000 7f0a5ea00640 1 -- 172.21.15.86:0/2872882444 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7f0a6008c9c0 con 0x7f0a680c8530 2024-04-20T21:30:55.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.028+0000 7f0a66800640 1 --2- 172.21.15.86:0/2872882444 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0a4806c8f0 0x7f0a4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:55.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.029+0000 7f0a66800640 1 --2- 172.21.15.86:0/2872882444 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0a4806c8f0 0x7f0a4806edb0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f0a58038000 tx=0x7f0a58030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:55.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.033+0000 7f0a5ea00640 1 -- 172.21.15.86:0/2872882444 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0a60014030 con 0x7f0a680c8530 2024-04-20T21:30:55.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.250+0000 7f0a6ca00640 1 -- 172.21.15.86:0/2872882444 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0a680cf7e0 con 0x7f0a680c8530 2024-04-20T21:30:55.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.251+0000 7f0a5ea00640 1 -- 172.21.15.86:0/2872882444 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v116) ==== 80+0+242 (secure 0 0 0) 0x7f0a600152b0 con 0x7f0a680c8530 2024-04-20T21:30:55.250 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6837/1453621918 2024-04-21T21:30:42.086474+0000 2024-04-20T21:30:55.251 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6835/1453621918 2024-04-21T21:30:42.086474+0000 2024-04-20T21:30:55.251 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6837/289590924 2024-04-21T21:30:19.784403+0000 2024-04-20T21:30:55.251 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6835/289590924 2024-04-21T21:30:19.784403+0000 2024-04-20T21:30:55.251 INFO:teuthology.orchestra.run.smithi086.stderr:listed 4 entries 2024-04-20T21:30:55.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.254+0000 7f0a6ca00640 1 -- 172.21.15.86:0/2872882444 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0a4806c8f0 msgr2=0x7f0a4806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:55.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.254+0000 7f0a6ca00640 1 --2- 172.21.15.86:0/2872882444 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0a4806c8f0 0x7f0a4806edb0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f0a58038000 tx=0x7f0a58030000 comp rx=0 tx=0).stop 2024-04-20T21:30:55.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.254+0000 7f0a6ca00640 1 -- 172.21.15.86:0/2872882444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a680c8530 msgr2=0x7f0a6806e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:55.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.254+0000 7f0a6ca00640 1 --2- 172.21.15.86:0/2872882444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a680c8530 0x7f0a6806e2b0 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f0a6000cdb0 tx=0x7f0a6000a650 comp rx=0 tx=0).stop 2024-04-20T21:30:55.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.254+0000 7f0a6ca00640 1 -- 172.21.15.86:0/2872882444 shutdown_connections 2024-04-20T21:30:55.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.254+0000 7f0a6ca00640 1 --2- 172.21.15.86:0/2872882444 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0a4806c8f0 0x7f0a4806edb0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:55.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.254+0000 7f0a6ca00640 1 --2- 172.21.15.86:0/2872882444 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a680c8530 0x7f0a6806e2b0 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:55.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.254+0000 7f0a6ca00640 1 --2- 172.21.15.86:0/2872882444 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0a680cfa00 0x7f0a680c5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:55.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.254+0000 7f0a6ca00640 1 --2- 172.21.15.86:0/2872882444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a68073540 0x7f0a6815b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:55.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.254+0000 7f0a6ca00640 1 -- 172.21.15.86:0/2872882444 >> 172.21.15.86:0/2872882444 conn(0x7f0a6806bb60 msgr2=0x7f0a68071a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:55.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.254+0000 7f0a6ca00640 1 -- 172.21.15.86:0/2872882444 shutdown_connections 2024-04-20T21:30:55.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:55.254+0000 7f0a6ca00640 1 -- 172.21.15.86:0/2872882444 wait complete. 2024-04-20T21:30:55.272 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi086.front.sepia.ceph.com... 2024-04-20T21:30:55.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:30:55.272 DEBUG:teuthology.orchestra.run.smithi086:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-20T21:30:55.356 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi086.stderr:ceph-fuse[69175]: fuse finished with error 0 and tester_r 0 2024-04-20T21:30:55.362 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-20T21:30:55.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:30:55.363 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-20T21:30:55.517 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T21:30:55.518 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:30:55.518 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:30:55.576 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T21:30:55.577 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2024-04-20T21:30:55.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:30:55.577 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-20T21:30:55.632 INFO:teuthology.orchestra.run.smithi144.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-20T21:30:55.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:30:55.634 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T21:30:55.634 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:30:55.634 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:30:55.692 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052731,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248996k,nr_inodes=812249,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T21:30:55.692 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-20T21:30:55.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:30:55.693 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-20T21:30:55.744 INFO:teuthology.orchestra.run.smithi086.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-20T21:30:55.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:30:55.745 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T21:30:55.745 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:30:55.746 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:30:55.804 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T21:30:55.804 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi144.front.sepia.ceph.com 2024-04-20T21:30:55.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:30:55.805 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-20T21:30:55.857 INFO:teuthology.orchestra.run.smithi144.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-20T21:30:55.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:30:55.858 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:30:56.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.068+0000 7f1ce5200640 1 Processor -- start 2024-04-20T21:30:56.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.068+0000 7f1ce5200640 1 -- start start 2024-04-20T21:30:56.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.070+0000 7f1ce5200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ce0146130 0x7f1ce006e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:56.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.070+0000 7f1ce5200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1ce00ccdd0 0x7f1ce00cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:56.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.070+0000 7f1ce5200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1ce0071d00 0x7f1ce0072190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:56.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.070+0000 7f1ce5200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f1ce006dc30 con 0x7f1ce00ccdd0 2024-04-20T21:30:56.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.070+0000 7f1ce5200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f1ce00cd8e0 con 0x7f1ce0071d00 2024-04-20T21:30:56.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.070+0000 7f1ce5200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1ce00cdbc0 con 0x7f1ce0146130 2024-04-20T21:30:56.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.070+0000 7f1cdd400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1ce00ccdd0 0x7f1ce00cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:56.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.070+0000 7f1cdd400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1ce00ccdd0 0x7f1ce00cd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46478/0 (socket says 172.21.15.86:46478) 2024-04-20T21:30:56.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.070+0000 7f1cdd400640 1 -- 172.21.15.86:0/833333382 learned_addr learned my addr 172.21.15.86:0/833333382 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:56.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.071+0000 7f1cdd400640 1 -- 172.21.15.86:0/833333382 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ce0146130 msgr2=0x7f1ce006e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:56.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.071+0000 7f1cde800640 1 --2- 172.21.15.86:0/833333382 >> v2:172.21.15.144:3300/0 conn(0x7f1ce0071d00 0x7f1ce0072190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:56.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.071+0000 7f1cdde00640 1 --2- 172.21.15.86:0/833333382 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ce0146130 0x7f1ce006e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:56.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.071+0000 7f1cdd400640 1 --2- 172.21.15.86:0/833333382 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ce0146130 0x7f1ce006e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:56.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.071+0000 7f1cdd400640 1 -- 172.21.15.86:0/833333382 >> v2:172.21.15.144:3300/0 conn(0x7f1ce0071d00 msgr2=0x7f1ce0072190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:56.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.071+0000 7f1cdd400640 1 --2- 172.21.15.86:0/833333382 >> v2:172.21.15.144:3300/0 conn(0x7f1ce0071d00 0x7f1ce0072190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:56.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.071+0000 7f1cdd400640 1 -- 172.21.15.86:0/833333382 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ce006ded0 con 0x7f1ce00ccdd0 2024-04-20T21:30:56.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.071+0000 7f1cde800640 1 --2- 172.21.15.86:0/833333382 >> v2:172.21.15.144:3300/0 conn(0x7f1ce0071d00 0x7f1ce0072190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:30:56.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.072+0000 7f1cdd400640 1 --2- 172.21.15.86:0/833333382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ce00ccdd0 0x7f1ce00cd230 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f1ce0070070 tx=0x7f1ccc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a815c6bfbb1b0b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:56.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.072+0000 7f1cdca00640 1 -- 172.21.15.86:0/833333382 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc034070 con 0x7f1ce00ccdd0 2024-04-20T21:30:56.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.072+0000 7f1cdca00640 1 -- 172.21.15.86:0/833333382 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ccc030860 con 0x7f1ce00ccdd0 2024-04-20T21:30:56.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.072+0000 7f1cdca00640 1 -- 172.21.15.86:0/833333382 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc030c30 con 0x7f1ce00ccdd0 2024-04-20T21:30:56.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.073+0000 7f1ce5200640 1 -- 172.21.15.86:0/833333382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ce00ccdd0 msgr2=0x7f1ce00cd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:56.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.073+0000 7f1ce5200640 1 --2- 172.21.15.86:0/833333382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ce00ccdd0 0x7f1ce00cd230 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f1ce0070070 tx=0x7f1ccc02b9a0 comp rx=0 tx=0).stop 2024-04-20T21:30:56.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.073+0000 7f1ce5200640 1 -- 172.21.15.86:0/833333382 shutdown_connections 2024-04-20T21:30:56.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.073+0000 7f1ce5200640 1 --2- 172.21.15.86:0/833333382 >> v2:172.21.15.144:3300/0 conn(0x7f1ce0071d00 0x7f1ce0072190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:56.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.073+0000 7f1ce5200640 1 --2- 172.21.15.86:0/833333382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ce00ccdd0 0x7f1ce00cd230 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:56.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.073+0000 7f1ce5200640 1 --2- 172.21.15.86:0/833333382 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ce0146130 0x7f1ce006e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:56.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.073+0000 7f1ce5200640 1 -- 172.21.15.86:0/833333382 >> 172.21.15.86:0/833333382 conn(0x7f1ce00b2360 msgr2=0x7f1ce00b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:56.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.073+0000 7f1ce5200640 1 -- 172.21.15.86:0/833333382 shutdown_connections 2024-04-20T21:30:56.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.073+0000 7f1ce5200640 1 -- 172.21.15.86:0/833333382 wait complete. 2024-04-20T21:30:56.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.075+0000 7f1ce5200640 1 Processor -- start 2024-04-20T21:30:56.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.075+0000 7f1ce5200640 1 -- start start 2024-04-20T21:30:56.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.076+0000 7f1ce5200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ce0071d00 0x7f1ce015c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:56.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.076+0000 7f1ce5200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ce00ccdd0 0x7f1ce0140fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:56.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.076+0000 7f1ce5200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ce0146130 0x7f1ce01414f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:56.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.076+0000 7f1ce5200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce015cc20 con 0x7f1ce0146130 2024-04-20T21:30:56.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.076+0000 7f1ce5200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce006f150 con 0x7f1ce0071d00 2024-04-20T21:30:56.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.076+0000 7f1ce5200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1ce015cda0 con 0x7f1ce00ccdd0 2024-04-20T21:30:56.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.076+0000 7f1cde800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ce0146130 0x7f1ce01414f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:56.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.076+0000 7f1cde800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ce0146130 0x7f1ce01414f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46480/0 (socket says 172.21.15.86:46480) 2024-04-20T21:30:56.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.076+0000 7f1cde800640 1 -- 172.21.15.86:0/2246125057 learned_addr learned my addr 172.21.15.86:0/2246125057 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:56.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.077+0000 7f1cdde00640 1 --2- 172.21.15.86:0/2246125057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ce0071d00 0x7f1ce015c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:56.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.077+0000 7f1cdd400640 1 --2- 172.21.15.86:0/2246125057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ce00ccdd0 0x7f1ce0140fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:56.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.077+0000 7f1cde800640 1 -- 172.21.15.86:0/2246125057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ce00ccdd0 msgr2=0x7f1ce0140fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:56.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.077+0000 7f1cde800640 1 --2- 172.21.15.86:0/2246125057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ce00ccdd0 0x7f1ce0140fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:56.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.077+0000 7f1cde800640 1 -- 172.21.15.86:0/2246125057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ce0071d00 msgr2=0x7f1ce015c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:56.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.077+0000 7f1cde800640 1 --2- 172.21.15.86:0/2246125057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ce0071d00 0x7f1ce015c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:56.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.077+0000 7f1cde800640 1 -- 172.21.15.86:0/2246125057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ce00cc2e0 con 0x7f1ce0146130 2024-04-20T21:30:56.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.077+0000 7f1cdde00640 1 --2- 172.21.15.86:0/2246125057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ce0071d00 0x7f1ce015c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:30:56.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.077+0000 7f1cde800640 1 --2- 172.21.15.86:0/2246125057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ce0146130 0x7f1ce01414f0 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f1cd800c820 tx=0x7f1cd800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:56.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.077+0000 7f1cd6000640 1 -- 172.21.15.86:0/2246125057 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd8015070 con 0x7f1ce0146130 2024-04-20T21:30:56.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.077+0000 7f1cd6000640 1 -- 172.21.15.86:0/2246125057 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cd8007a30 con 0x7f1ce0146130 2024-04-20T21:30:56.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.077+0000 7f1cd6000640 1 -- 172.21.15.86:0/2246125057 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd8007e20 con 0x7f1ce0146130 2024-04-20T21:30:56.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.077+0000 7f1ce5200640 1 -- 172.21.15.86:0/2246125057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ce00cc5c0 con 0x7f1ce0146130 2024-04-20T21:30:56.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.078+0000 7f1ce5200640 1 -- 172.21.15.86:0/2246125057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ce00cb5d0 con 0x7f1ce0146130 2024-04-20T21:30:56.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.078+0000 7f1ce5200640 1 -- 172.21.15.86:0/2246125057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ce00689e0 con 0x7f1ce0146130 2024-04-20T21:30:56.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.080+0000 7f1cd6000640 1 -- 172.21.15.86:0/2246125057 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1cd8018070 con 0x7f1ce0146130 2024-04-20T21:30:56.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.080+0000 7f1cd6000640 1 --2- 172.21.15.86:0/2246125057 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1cc006c8f0 0x7f1cc006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:56.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.081+0000 7f1cd6000640 1 -- 172.21.15.86:0/2246125057 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7f1cd808ba90 con 0x7f1ce0146130 2024-04-20T21:30:56.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.081+0000 7f1cdde00640 1 --2- 172.21.15.86:0/2246125057 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1cc006c8f0 0x7f1cc006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:56.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.081+0000 7f1cdde00640 1 --2- 172.21.15.86:0/2246125057 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1cc006c8f0 0x7f1cc006edb0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0007eb0 tx=0x7f1cd0007400 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:56.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.085+0000 7f1cd6000640 1 -- 172.21.15.86:0/2246125057 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1cd8059f00 con 0x7f1ce0146130 2024-04-20T21:30:56.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.326+0000 7f1ce5200640 1 -- 172.21.15.86:0/2246125057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ce015ff50 con 0x7f1ce0146130 2024-04-20T21:30:56.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.327+0000 7f1cd6000640 1 -- 172.21.15.86:0/2246125057 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5212 (secure 0 0 0) 0x7f1cd80600d0 con 0x7f1ce0146130 2024-04-20T21:30:56.325 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:56.326 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":112,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6253,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3627230007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3627230007},{"type":"v1","addr":"172.21.15.144:6833","nonce":3627230007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":6256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2498297133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2498297133},{"type":"v1","addr":"172.21.15.144:6835","nonce":2498297133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":9228,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/1120740475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1120740475},{"type":"v1","addr":"172.21.15.86:6836","nonce":1120740475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:29:43.769581+0000","modified":"2024-04-20T21:30:47.137068+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9792},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9792":{"gid":9792,"name":"a","rank":0,"incarnation":109,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6837/2019839930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2019839930},{"type":"v1","addr":"172.21.15.86:6837","nonce":2019839930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9792,"qdb_cluster":[9792]},"id":8}]} 2024-04-20T21:30:56.326 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 112 2024-04-20T21:30:56.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.329+0000 7f1ce5200640 1 -- 172.21.15.86:0/2246125057 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1cc006c8f0 msgr2=0x7f1cc006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:56.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.329+0000 7f1ce5200640 1 --2- 172.21.15.86:0/2246125057 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1cc006c8f0 0x7f1cc006edb0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0007eb0 tx=0x7f1cd0007400 comp rx=0 tx=0).stop 2024-04-20T21:30:56.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.329+0000 7f1ce5200640 1 -- 172.21.15.86:0/2246125057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ce0146130 msgr2=0x7f1ce01414f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:56.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.329+0000 7f1ce5200640 1 --2- 172.21.15.86:0/2246125057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ce0146130 0x7f1ce01414f0 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f1cd800c820 tx=0x7f1cd800cce0 comp rx=0 tx=0).stop 2024-04-20T21:30:56.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.329+0000 7f1ce5200640 1 -- 172.21.15.86:0/2246125057 shutdown_connections 2024-04-20T21:30:56.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.329+0000 7f1ce5200640 1 --2- 172.21.15.86:0/2246125057 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1cc006c8f0 0x7f1cc006edb0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:56.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.329+0000 7f1ce5200640 1 --2- 172.21.15.86:0/2246125057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ce0146130 0x7f1ce01414f0 unknown :-1 s=CLOSED pgs=2091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:56.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.329+0000 7f1ce5200640 1 --2- 172.21.15.86:0/2246125057 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1ce00ccdd0 0x7f1ce0140fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:56.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.329+0000 7f1ce5200640 1 --2- 172.21.15.86:0/2246125057 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ce0071d00 0x7f1ce015c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:56.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.329+0000 7f1ce5200640 1 -- 172.21.15.86:0/2246125057 >> 172.21.15.86:0/2246125057 conn(0x7f1ce00b2360 msgr2=0x7f1ce0143a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:56.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.330+0000 7f1ce5200640 1 -- 172.21.15.86:0/2246125057 shutdown_connections 2024-04-20T21:30:56.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.330+0000 7f1ce5200640 1 -- 172.21.15.86:0/2246125057 wait complete. 2024-04-20T21:30:56.348 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:30:56.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.547+0000 7f0c2e400640 1 Processor -- start 2024-04-20T21:30:56.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.547+0000 7f0c2e400640 1 -- start start 2024-04-20T21:30:56.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.549+0000 7f0c2e400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f0c2815ebf0 0x7f0c280d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:56.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.549+0000 7f0c2e400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0c280d1820 0x7f0c2809a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:56.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.549+0000 7f0c2e400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c2815efd0 con 0x7f0c2809af20 2024-04-20T21:30:56.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.549+0000 7f0c2e400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f0c280b3e70 con 0x7f0c280d1820 2024-04-20T21:30:56.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.549+0000 7f0c2e400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f0c280b1dc0 con 0x7f0c2815ebf0 2024-04-20T21:30:56.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.549+0000 7f0c26a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0c280d1820 0x7f0c2809a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:56.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.549+0000 7f0c26a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0c280d1820 0x7f0c2809a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46492/0 (socket says 172.21.15.86:46492) 2024-04-20T21:30:56.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.549+0000 7f0c26a00640 1 -- 172.21.15.86:0/3152436926 learned_addr learned my addr 172.21.15.86:0/3152436926 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:56.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.550+0000 7f0c27400640 1 --2- 172.21.15.86:0/3152436926 >> v2:172.21.15.144:3300/0 conn(0x7f0c2815ebf0 0x7f0c280d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:56.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.550+0000 7f0c26a00640 1 -- 172.21.15.86:0/3152436926 >> v2:172.21.15.144:3300/0 conn(0x7f0c2815ebf0 msgr2=0x7f0c280d12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:56.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.550+0000 7f0c26a00640 1 --2- 172.21.15.86:0/3152436926 >> v2:172.21.15.144:3300/0 conn(0x7f0c2815ebf0 0x7f0c280d12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:56.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.550+0000 7f0c26a00640 1 -- 172.21.15.86:0/3152436926 >> v1:172.21.15.144:6789/0 conn(0x7f0c2809af20 legacy=0x7f0c2806cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:56.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.550+0000 7f0c26a00640 1 -- 172.21.15.86:0/3152436926 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c280b20a0 con 0x7f0c280d1820 2024-04-20T21:30:56.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.551+0000 7f0c26a00640 1 --2- 172.21.15.86:0/3152436926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c280d1820 0x7f0c2809a9e0 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7f0c280b50f0 tx=0x7f0c1002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2fbee8a27e75d44 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:56.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.551+0000 7f0c26000640 1 -- 172.21.15.86:0/3152436926 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1002ca80 con 0x7f0c280d1820 2024-04-20T21:30:56.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.551+0000 7f0c26000640 1 -- 172.21.15.86:0/3152436926 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c1002cc20 con 0x7f0c280d1820 2024-04-20T21:30:56.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.551+0000 7f0c26000640 1 -- 172.21.15.86:0/3152436926 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c100343f0 con 0x7f0c280d1820 2024-04-20T21:30:56.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.552+0000 7f0c2e400640 1 -- 172.21.15.86:0/3152436926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c280d1820 msgr2=0x7f0c2809a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:56.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.552+0000 7f0c2e400640 1 --2- 172.21.15.86:0/3152436926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c280d1820 0x7f0c2809a9e0 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7f0c280b50f0 tx=0x7f0c1002bee0 comp rx=0 tx=0).stop 2024-04-20T21:30:56.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.552+0000 7f0c2e400640 1 -- 172.21.15.86:0/3152436926 shutdown_connections 2024-04-20T21:30:56.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.552+0000 7f0c2e400640 1 --2- 172.21.15.86:0/3152436926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c280d1820 0x7f0c2809a9e0 unknown :-1 s=CLOSED pgs=2092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:56.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.552+0000 7f0c2e400640 1 --2- 172.21.15.86:0/3152436926 >> v2:172.21.15.144:3300/0 conn(0x7f0c2815ebf0 0x7f0c280d12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:56.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.552+0000 7f0c2e400640 1 -- 172.21.15.86:0/3152436926 >> 172.21.15.86:0/3152436926 conn(0x7f0c280b3ff0 msgr2=0x7f0c280b4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:56.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.552+0000 7f0c2e400640 1 -- 172.21.15.86:0/3152436926 shutdown_connections 2024-04-20T21:30:56.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.552+0000 7f0c2e400640 1 -- 172.21.15.86:0/3152436926 wait complete. 2024-04-20T21:30:56.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.554+0000 7f0c2e400640 1 Processor -- start 2024-04-20T21:30:56.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.554+0000 7f0c2e400640 1 -- start start 2024-04-20T21:30:56.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.555+0000 7f0c2e400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c2809af20 0x7f0c28156ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:56.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.555+0000 7f0c2e400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c280d1820 0x7f0c280cb170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:56.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.555+0000 7f0c2e400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c2815ebf0 0x7f0c280cb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:56.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.555+0000 7f0c2e400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0c280b2940 con 0x7f0c2815ebf0 2024-04-20T21:30:56.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.555+0000 7f0c2e400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f0c280b2660 con 0x7f0c280d1820 2024-04-20T21:30:56.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.555+0000 7f0c2e400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f0c280ccb50 con 0x7f0c2809af20 2024-04-20T21:30:56.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.556+0000 7f0c27e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c2815ebf0 0x7f0c280cb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:56.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.556+0000 7f0c27e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c2815ebf0 0x7f0c280cb6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46498/0 (socket says 172.21.15.86:46498) 2024-04-20T21:30:56.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.556+0000 7f0c27e00640 1 -- 172.21.15.86:0/1794588284 learned_addr learned my addr 172.21.15.86:0/1794588284 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:56.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.556+0000 7f0c26a00640 1 --2- 172.21.15.86:0/1794588284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c280d1820 0x7f0c280cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:56.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.556+0000 7f0c27400640 1 --2- 172.21.15.86:0/1794588284 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c2809af20 0x7f0c28156ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:56.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.556+0000 7f0c27e00640 1 -- 172.21.15.86:0/1794588284 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c2809af20 msgr2=0x7f0c28156ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:56.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.556+0000 7f0c27e00640 1 --2- 172.21.15.86:0/1794588284 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c2809af20 0x7f0c28156ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:56.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.556+0000 7f0c27e00640 1 -- 172.21.15.86:0/1794588284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c280d1820 msgr2=0x7f0c280cb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:56.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.556+0000 7f0c27e00640 1 --2- 172.21.15.86:0/1794588284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c280d1820 0x7f0c280cb170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:56.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.556+0000 7f0c27e00640 1 -- 172.21.15.86:0/1794588284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c280cd7c0 con 0x7f0c2815ebf0 2024-04-20T21:30:56.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.556+0000 7f0c27e00640 1 --2- 172.21.15.86:0/1794588284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c2815ebf0 0x7f0c280cb6b0 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f0c2000b490 tx=0x7f0c2000b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:56.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.557+0000 7f0c1f400640 1 -- 172.21.15.86:0/1794588284 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20015020 con 0x7f0c2815ebf0 2024-04-20T21:30:56.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.557+0000 7f0c1f400640 1 -- 172.21.15.86:0/1794588284 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c20011850 con 0x7f0c2815ebf0 2024-04-20T21:30:56.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.557+0000 7f0c1f400640 1 -- 172.21.15.86:0/1794588284 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20011c40 con 0x7f0c2815ebf0 2024-04-20T21:30:56.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.557+0000 7f0c2e400640 1 -- 172.21.15.86:0/1794588284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c280cdaa0 con 0x7f0c2815ebf0 2024-04-20T21:30:56.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.557+0000 7f0c2e400640 1 -- 172.21.15.86:0/1794588284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c280c75b0 con 0x7f0c2815ebf0 2024-04-20T21:30:56.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.558+0000 7f0c2e400640 1 -- 172.21.15.86:0/1794588284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c280689e0 con 0x7f0c2815ebf0 2024-04-20T21:30:56.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.559+0000 7f0c1f400640 1 -- 172.21.15.86:0/1794588284 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0c20018020 con 0x7f0c2815ebf0 2024-04-20T21:30:56.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.560+0000 7f0c1f400640 1 --2- 172.21.15.86:0/1794588284 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0c0406c8f0 0x7f0c0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:56.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.560+0000 7f0c1f400640 1 -- 172.21.15.86:0/1794588284 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7f0c2008baf0 con 0x7f0c2815ebf0 2024-04-20T21:30:56.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.560+0000 7f0c27400640 1 --2- 172.21.15.86:0/1794588284 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0c0406c8f0 0x7f0c0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:56.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.561+0000 7f0c27400640 1 --2- 172.21.15.86:0/1794588284 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0c0406c8f0 0x7f0c0406edb0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f0c1800a970 tx=0x7f0c18008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:56.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.564+0000 7f0c1f400640 1 -- 172.21.15.86:0/1794588284 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0c2001f020 con 0x7f0c2815ebf0 2024-04-20T21:30:56.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.766+0000 7f0c2e400640 1 -- 172.21.15.86:0/1794588284 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0c280af250 con 0x7f0c2815ebf0 2024-04-20T21:30:56.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.768+0000 7f0c1f400640 1 -- 172.21.15.86:0/1794588284 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) ==== 74+0+17037 (secure 0 0 0) 0x7f0c200172c0 con 0x7f0c2815ebf0 2024-04-20T21:30:56.767 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:56.767 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":116,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:30:42.086516+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:29:41.532824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-20T21:29:42.749687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.f","mappings":[{"from":0,"to":5}]},{"pgid":"17.18","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6837/1453621918":"2024-04-21T21:30:42.086474+0000","172.21.15.86:6835/1453621918":"2024-04-21T21:30:42.086474+0000","172.21.15.86:6837/289590924":"2024-04-21T21:30:19.784403+0000","172.21.15.86:6835/289590924":"2024-04-21T21:30:19.784403+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:30:56.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.770+0000 7f0c2e400640 1 -- 172.21.15.86:0/1794588284 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0c0406c8f0 msgr2=0x7f0c0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:56.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.770+0000 7f0c2e400640 1 --2- 172.21.15.86:0/1794588284 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0c0406c8f0 0x7f0c0406edb0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f0c1800a970 tx=0x7f0c18008040 comp rx=0 tx=0).stop 2024-04-20T21:30:56.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.770+0000 7f0c2e400640 1 -- 172.21.15.86:0/1794588284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c2815ebf0 msgr2=0x7f0c280cb6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:56.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.770+0000 7f0c2e400640 1 --2- 172.21.15.86:0/1794588284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c2815ebf0 0x7f0c280cb6b0 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f0c2000b490 tx=0x7f0c2000b950 comp rx=0 tx=0).stop 2024-04-20T21:30:56.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.770+0000 7f0c2e400640 1 -- 172.21.15.86:0/1794588284 shutdown_connections 2024-04-20T21:30:56.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.770+0000 7f0c2e400640 1 --2- 172.21.15.86:0/1794588284 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f0c0406c8f0 0x7f0c0406edb0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:56.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.770+0000 7f0c2e400640 1 --2- 172.21.15.86:0/1794588284 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c2815ebf0 0x7f0c280cb6b0 unknown :-1 s=CLOSED pgs=2093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:56.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.770+0000 7f0c2e400640 1 --2- 172.21.15.86:0/1794588284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c280d1820 0x7f0c280cb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:56.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.770+0000 7f0c2e400640 1 --2- 172.21.15.86:0/1794588284 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f0c2809af20 0x7f0c28156ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:56.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.770+0000 7f0c2e400640 1 -- 172.21.15.86:0/1794588284 >> 172.21.15.86:0/1794588284 conn(0x7f0c280b3ff0 msgr2=0x7f0c280afa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:56.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.771+0000 7f0c2e400640 1 -- 172.21.15.86:0/1794588284 shutdown_connections 2024-04-20T21:30:56.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.771+0000 7f0c2e400640 1 -- 172.21.15.86:0/1794588284 wait complete. 2024-04-20T21:30:56.787 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T21:17:40.663693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T21:29:41.532824+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T21:29:42.749687+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T21:30:56.788 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T21:30:56.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.987+0000 7f78e2a00640 1 Processor -- start 2024-04-20T21:30:56.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.988+0000 7f78e2a00640 1 -- start start 2024-04-20T21:30:56.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.989+0000 7f78e2a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78dc155d30 0x7f78dc0cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:56.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.989+0000 7f78e2a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f78dc06b980 0x7f78dc06be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:56.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.989+0000 7f78e2a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f78dc0b4ba0 0x7f78dc0b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:56.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.989+0000 7f78e2a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f78dc156110 con 0x7f78dc06b980 2024-04-20T21:30:56.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.989+0000 7f78e2a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f78dc0ce460 con 0x7f78dc0b4ba0 2024-04-20T21:30:56.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.989+0000 7f78e2a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f78dc0ce740 con 0x7f78dc155d30 2024-04-20T21:30:56.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.990+0000 7f78d3400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f78dc06b980 0x7f78dc06be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:56.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.990+0000 7f78d3400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f78dc06b980 0x7f78dc06be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46514/0 (socket says 172.21.15.86:46514) 2024-04-20T21:30:56.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.990+0000 7f78d3400640 1 -- 172.21.15.86:0/857039481 learned_addr learned my addr 172.21.15.86:0/857039481 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:56.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.990+0000 7f78dbe00640 1 --2- 172.21.15.86:0/857039481 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78dc155d30 0x7f78dc0cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:56.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.990+0000 7f78e0a00640 1 --2- 172.21.15.86:0/857039481 >> v2:172.21.15.144:3300/0 conn(0x7f78dc0b4ba0 0x7f78dc0b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:56.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.990+0000 7f78d3400640 1 -- 172.21.15.86:0/857039481 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78dc155d30 msgr2=0x7f78dc0cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:56.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.990+0000 7f78d3400640 1 --2- 172.21.15.86:0/857039481 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78dc155d30 0x7f78dc0cf500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:56.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.990+0000 7f78d3400640 1 -- 172.21.15.86:0/857039481 >> v2:172.21.15.144:3300/0 conn(0x7f78dc0b4ba0 msgr2=0x7f78dc0b4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:56.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.990+0000 7f78d3400640 1 --2- 172.21.15.86:0/857039481 >> v2:172.21.15.144:3300/0 conn(0x7f78dc0b4ba0 0x7f78dc0b4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:56.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.990+0000 7f78d3400640 1 -- 172.21.15.86:0/857039481 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78dc0cea20 con 0x7f78dc06b980 2024-04-20T21:30:56.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.991+0000 7f78d3400640 1 --2- 172.21.15.86:0/857039481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78dc06b980 0x7f78dc06be00 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f78dc06ca90 tx=0x7f78c402b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2220b9de1c92fa7a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:56.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.992+0000 7f78db400640 1 -- 172.21.15.86:0/857039481 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c4038070 con 0x7f78dc06b980 2024-04-20T21:30:56.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.992+0000 7f78db400640 1 -- 172.21.15.86:0/857039481 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78c402f050 con 0x7f78dc06b980 2024-04-20T21:30:56.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.992+0000 7f78db400640 1 -- 172.21.15.86:0/857039481 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c4031970 con 0x7f78dc06b980 2024-04-20T21:30:56.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.993+0000 7f78e2a00640 1 -- 172.21.15.86:0/857039481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78dc06b980 msgr2=0x7f78dc06be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:56.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.993+0000 7f78e2a00640 1 --2- 172.21.15.86:0/857039481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78dc06b980 0x7f78dc06be00 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f78dc06ca90 tx=0x7f78c402b7f0 comp rx=0 tx=0).stop 2024-04-20T21:30:56.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.993+0000 7f78e2a00640 1 -- 172.21.15.86:0/857039481 shutdown_connections 2024-04-20T21:30:56.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.993+0000 7f78e2a00640 1 --2- 172.21.15.86:0/857039481 >> v2:172.21.15.144:3300/0 conn(0x7f78dc0b4ba0 0x7f78dc0b4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:56.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.993+0000 7f78e2a00640 1 --2- 172.21.15.86:0/857039481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78dc06b980 0x7f78dc06be00 unknown :-1 s=CLOSED pgs=2094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:56.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.993+0000 7f78e2a00640 1 --2- 172.21.15.86:0/857039481 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78dc155d30 0x7f78dc0cf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:56.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.993+0000 7f78e2a00640 1 -- 172.21.15.86:0/857039481 >> 172.21.15.86:0/857039481 conn(0x7f78dc0b2110 msgr2=0x7f78dc0b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:56.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.993+0000 7f78e2a00640 1 -- 172.21.15.86:0/857039481 shutdown_connections 2024-04-20T21:30:56.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.993+0000 7f78e2a00640 1 -- 172.21.15.86:0/857039481 wait complete. 2024-04-20T21:30:56.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.994+0000 7f78e2a00640 1 Processor -- start 2024-04-20T21:30:56.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.995+0000 7f78e2a00640 1 -- start start 2024-04-20T21:30:56.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.996+0000 7f78e2a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78dc06b980 0x7f78dc1f5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:56.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.996+0000 7f78e2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78dc0b4ba0 0x7f78dc1431a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:56.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.996+0000 7f78e2a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78dc155d30 0x7f78dc14f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:56.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.996+0000 7f78e2a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f78dc0b5510 con 0x7f78dc0b4ba0 2024-04-20T21:30:56.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.996+0000 7f78e2a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f78dc0cefe0 con 0x7f78dc155d30 2024-04-20T21:30:56.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.996+0000 7f78e2a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f78dc0b5690 con 0x7f78dc06b980 2024-04-20T21:30:56.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.996+0000 7f78d3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78dc0b4ba0 0x7f78dc1431a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:56.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.996+0000 7f78d3400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78dc0b4ba0 0x7f78dc1431a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46516/0 (socket says 172.21.15.86:46516) 2024-04-20T21:30:56.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.996+0000 7f78d3400640 1 -- 172.21.15.86:0/2903397625 learned_addr learned my addr 172.21.15.86:0/2903397625 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:56.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.996+0000 7f78dbe00640 1 --2- 172.21.15.86:0/2903397625 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78dc06b980 0x7f78dc1f5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:56.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.996+0000 7f78e0a00640 1 --2- 172.21.15.86:0/2903397625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78dc155d30 0x7f78dc14f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:56.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.996+0000 7f78d3400640 1 -- 172.21.15.86:0/2903397625 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78dc06b980 msgr2=0x7f78dc1f5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:56.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.996+0000 7f78d3400640 1 --2- 172.21.15.86:0/2903397625 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78dc06b980 0x7f78dc1f5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:56.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.996+0000 7f78d3400640 1 -- 172.21.15.86:0/2903397625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78dc155d30 msgr2=0x7f78dc14f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:56.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.996+0000 7f78d3400640 1 --2- 172.21.15.86:0/2903397625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78dc155d30 0x7f78dc14f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:56.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.996+0000 7f78d3400640 1 -- 172.21.15.86:0/2903397625 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78dc150850 con 0x7f78dc0b4ba0 2024-04-20T21:30:56.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.997+0000 7f78d3400640 1 --2- 172.21.15.86:0/2903397625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78dc0b4ba0 0x7f78dc1431a0 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f78c4000c00 tx=0x7f78c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:56.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.997+0000 7f78d8c00640 1 -- 172.21.15.86:0/2903397625 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c4042070 con 0x7f78dc0b4ba0 2024-04-20T21:30:56.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.997+0000 7f78d8c00640 1 -- 172.21.15.86:0/2903397625 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78c403d890 con 0x7f78dc0b4ba0 2024-04-20T21:30:56.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.997+0000 7f78d8c00640 1 -- 172.21.15.86:0/2903397625 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c4038040 con 0x7f78dc0b4ba0 2024-04-20T21:30:56.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.997+0000 7f78e2a00640 1 -- 172.21.15.86:0/2903397625 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78dc150b30 con 0x7f78dc0b4ba0 2024-04-20T21:30:56.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.997+0000 7f78e2a00640 1 -- 172.21.15.86:0/2903397625 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78dc14ba50 con 0x7f78dc0b4ba0 2024-04-20T21:30:56.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.998+0000 7f78e2a00640 1 -- 172.21.15.86:0/2903397625 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78dc0689e0 con 0x7f78dc0b4ba0 2024-04-20T21:30:56.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:56.999+0000 7f78d8c00640 1 -- 172.21.15.86:0/2903397625 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f78c4045070 con 0x7f78dc0b4ba0 2024-04-20T21:30:56.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.000+0000 7f78d8c00640 1 --2- 172.21.15.86:0/2903397625 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f78b806c920 0x7f78b806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:56.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.000+0000 7f78d8c00640 1 -- 172.21.15.86:0/2903397625 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7f78c40ad880 con 0x7f78dc0b4ba0 2024-04-20T21:30:56.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.000+0000 7f78dbe00640 1 --2- 172.21.15.86:0/2903397625 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f78b806c920 0x7f78b806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:57.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.001+0000 7f78dbe00640 1 --2- 172.21.15.86:0/2903397625 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f78b806c920 0x7f78b806ede0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f78cc0099c0 tx=0x7f78cc003990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:57.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.005+0000 7f78d8c00640 1 -- 172.21.15.86:0/2903397625 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f78c407bbb0 con 0x7f78dc0b4ba0 2024-04-20T21:30:57.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.207+0000 7f78e2a00640 1 -- 172.21.15.86:0/2903397625 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f78dc072af0 con 0x7f78dc0b4ba0 2024-04-20T21:30:57.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.208+0000 7f78d8c00640 1 -- 172.21.15.86:0/2903397625 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v116) ==== 93+0+10 (secure 0 0 0) 0x7f78c407b4b0 con 0x7f78dc0b4ba0 2024-04-20T21:30:57.207 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-20T21:30:57.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.210+0000 7f78e2a00640 1 -- 172.21.15.86:0/2903397625 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f78b806c920 msgr2=0x7f78b806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:57.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.210+0000 7f78e2a00640 1 --2- 172.21.15.86:0/2903397625 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f78b806c920 0x7f78b806ede0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f78cc0099c0 tx=0x7f78cc003990 comp rx=0 tx=0).stop 2024-04-20T21:30:57.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.210+0000 7f78e2a00640 1 -- 172.21.15.86:0/2903397625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78dc0b4ba0 msgr2=0x7f78dc1431a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:57.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.210+0000 7f78e2a00640 1 --2- 172.21.15.86:0/2903397625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78dc0b4ba0 0x7f78dc1431a0 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f78c4000c00 tx=0x7f78c40046c0 comp rx=0 tx=0).stop 2024-04-20T21:30:57.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.210+0000 7f78e2a00640 1 -- 172.21.15.86:0/2903397625 shutdown_connections 2024-04-20T21:30:57.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.210+0000 7f78e2a00640 1 --2- 172.21.15.86:0/2903397625 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f78b806c920 0x7f78b806ede0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:57.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.211+0000 7f78e2a00640 1 --2- 172.21.15.86:0/2903397625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78dc155d30 0x7f78dc14f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:57.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.211+0000 7f78e2a00640 1 --2- 172.21.15.86:0/2903397625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78dc0b4ba0 0x7f78dc1431a0 unknown :-1 s=CLOSED pgs=2095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:57.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.211+0000 7f78e2a00640 1 --2- 172.21.15.86:0/2903397625 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f78dc06b980 0x7f78dc1f5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:57.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.211+0000 7f78e2a00640 1 -- 172.21.15.86:0/2903397625 >> 172.21.15.86:0/2903397625 conn(0x7f78dc0b2110 msgr2=0x7f78dc15e430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:57.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.211+0000 7f78e2a00640 1 -- 172.21.15.86:0/2903397625 shutdown_connections 2024-04-20T21:30:57.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.211+0000 7f78e2a00640 1 -- 172.21.15.86:0/2903397625 wait complete. 2024-04-20T21:30:57.227 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T21:30:57.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.433+0000 7f5fed800640 1 Processor -- start 2024-04-20T21:30:57.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.433+0000 7f5fed800640 1 -- start start 2024-04-20T21:30:57.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.434+0000 7f5fed800640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5fe809b0f0 0x7f5fe80cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:57.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.434+0000 7f5fed800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5fe8070e50 0x7f5fe80ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:57.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.434+0000 7f5fed800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fe8070090 con 0x7f5fe80ce640 2024-04-20T21:30:57.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.434+0000 7f5fed800640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f5fe809bec0 con 0x7f5fe809b0f0 2024-04-20T21:30:57.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.434+0000 7f5fed800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5fe80709c0 con 0x7f5fe8070e50 2024-04-20T21:30:57.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.435+0000 7f5fe7e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f5fe80ce640 0x7f5fe809a9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57250/0 (socket says 172.21.15.86:57250) 2024-04-20T21:30:57.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.435+0000 7f5fe7e00640 1 -- 172.21.15.86:0/3112673526 learned_addr learned my addr 172.21.15.86:0/3112673526 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:30:57.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.435+0000 7f5fece00640 1 --2- 172.21.15.86:0/3112673526 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5fe8070e50 0x7f5fe80ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:57.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.435+0000 7f5fe6a00640 1 -- 172.21.15.86:0/3112673526 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3840089510 0 0) 0x7f5fe8070090 con 0x7f5fe80ce640 2024-04-20T21:30:57.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.435+0000 7f5fe6a00640 1 -- 172.21.15.86:0/3112673526 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5fcc003520 con 0x7f5fe80ce640 2024-04-20T21:30:57.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.435+0000 7f5fe7400640 1 --2- 172.21.15.86:0/3112673526 >> v2:172.21.15.144:3300/0 conn(0x7f5fe809b0f0 0x7f5fe80cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:57.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.436+0000 7f5fe6a00640 1 -- 172.21.15.86:0/3112673526 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1192885930 0 0) 0x7f5fcc003520 con 0x7f5fe80ce640 2024-04-20T21:30:57.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.436+0000 7f5fe6a00640 1 -- 172.21.15.86:0/3112673526 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5fe8070e50 msgr2=0x7f5fe80ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:57.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.436+0000 7f5fe6a00640 1 --2- 172.21.15.86:0/3112673526 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5fe8070e50 0x7f5fe80ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:57.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.436+0000 7f5fe6a00640 1 -- 172.21.15.86:0/3112673526 >> v2:172.21.15.144:3300/0 conn(0x7f5fe809b0f0 msgr2=0x7f5fe80cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:57.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.436+0000 7f5fe6a00640 1 --2- 172.21.15.86:0/3112673526 >> v2:172.21.15.144:3300/0 conn(0x7f5fe809b0f0 0x7f5fe80cea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:57.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.436+0000 7f5fe6a00640 1 -- 172.21.15.86:0/3112673526 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe8070370 con 0x7f5fe80ce640 2024-04-20T21:30:57.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.436+0000 7f5fe6a00640 1 -- 172.21.15.86:0/3112673526 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f5fe0001670 con 0x7f5fe80ce640 2024-04-20T21:30:57.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.436+0000 7f5fe6a00640 1 -- 172.21.15.86:0/3112673526 >> v1:172.21.15.86:6789/0 conn(0x7f5fe80ce640 legacy=0x7f5fe809a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:57.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.436+0000 7f5fe6a00640 1 --2- 172.21.15.86:0/3112673526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fcc003a20 0x7f5fcc003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:57.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.436+0000 7f5fe6a00640 1 --2- 172.21.15.86:0/3112673526 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5fe8070e50 0x7f5fcc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:57.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.436+0000 7f5fe6a00640 1 --2- 172.21.15.86:0/3112673526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fcc004d40 0x7f5fcc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:57.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.436+0000 7f5fe6a00640 1 -- 172.21.15.86:0/3112673526 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe809bec0 con 0x7f5fcc003a20 2024-04-20T21:30:57.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.436+0000 7f5fe6a00640 1 -- 172.21.15.86:0/3112673526 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe80709c0 con 0x7f5fcc004d40 2024-04-20T21:30:57.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.436+0000 7f5fe6a00640 1 -- 172.21.15.86:0/3112673526 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5fcc00baa0 con 0x7f5fe8070e50 2024-04-20T21:30:57.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.436+0000 7f5fe7e00640 1 --2- 172.21.15.86:0/3112673526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fcc003a20 0x7f5fcc003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:57.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.436+0000 7f5fe7400640 1 --2- 172.21.15.86:0/3112673526 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5fe8070e50 0x7f5fcc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:57.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.436+0000 7f5fe7e00640 1 -- 172.21.15.86:0/3112673526 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5fe8070e50 msgr2=0x7f5fcc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:57.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.437+0000 7f5fe7e00640 1 --2- 172.21.15.86:0/3112673526 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5fe8070e50 0x7f5fcc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:57.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.437+0000 7f5fe7e00640 1 -- 172.21.15.86:0/3112673526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fcc004d40 msgr2=0x7f5fcc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:57.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.437+0000 7f5fe7e00640 1 --2- 172.21.15.86:0/3112673526 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fcc004d40 0x7f5fcc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:57.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.437+0000 7f5fe7e00640 1 -- 172.21.15.86:0/3112673526 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fcc003520 con 0x7f5fcc003a20 2024-04-20T21:30:57.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.437+0000 7f5fe7400640 1 --2- 172.21.15.86:0/3112673526 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5fe8070e50 0x7f5fcc004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:30:57.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.438+0000 7f5fe7e00640 1 --2- 172.21.15.86:0/3112673526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fcc003a20 0x7f5fcc003e40 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f5fe000c210 tx=0x7f5fcc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=858df91dc1866f90 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:57.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.438+0000 7f5fe6a00640 1 -- 172.21.15.86:0/3112673526 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe0007d50 con 0x7f5fcc003a20 2024-04-20T21:30:57.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.438+0000 7f5fe6a00640 1 -- 172.21.15.86:0/3112673526 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fe0030070 con 0x7f5fcc003a20 2024-04-20T21:30:57.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.438+0000 7f5fe6a00640 1 -- 172.21.15.86:0/3112673526 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe002cd80 con 0x7f5fcc003a20 2024-04-20T21:30:57.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.438+0000 7f5fed800640 1 -- 172.21.15.86:0/3112673526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fcc003a20 msgr2=0x7f5fcc003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:57.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.438+0000 7f5fed800640 1 --2- 172.21.15.86:0/3112673526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fcc003a20 0x7f5fcc003e40 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f5fe000c210 tx=0x7f5fcc005830 comp rx=0 tx=0).stop 2024-04-20T21:30:57.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.439+0000 7f5fece00640 1 -- 172.21.15.86:0/3112673526 reap_dead start 2024-04-20T21:30:57.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.439+0000 7f5fed800640 1 -- 172.21.15.86:0/3112673526 shutdown_connections 2024-04-20T21:30:57.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.439+0000 7f5fed800640 1 -- 172.21.15.86:0/3112673526 >> 172.21.15.86:0/3112673526 conn(0x7f5fe80b3ff0 msgr2=0x7f5fe80b4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:57.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.439+0000 7f5fed800640 1 -- 172.21.15.86:0/3112673526 shutdown_connections 2024-04-20T21:30:57.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.439+0000 7f5fed800640 1 -- 172.21.15.86:0/3112673526 wait complete. 2024-04-20T21:30:57.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.440+0000 7f5fed800640 1 Processor -- start 2024-04-20T21:30:57.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.441+0000 7f5fed800640 1 -- start start 2024-04-20T21:30:57.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.441+0000 7f5fed800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fcc003a20 0x7f5fe80c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:57.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.442+0000 7f5fed800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5fe80c9b10 0x7f5fe80c7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:57.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.442+0000 7f5fed800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fe80c75b0 0x7f5fe8157430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:57.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.442+0000 7f5fed800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe8071fa0 con 0x7f5fcc003a20 2024-04-20T21:30:57.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.442+0000 7f5fed800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe8071e20 con 0x7f5fe80c75b0 2024-04-20T21:30:57.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.442+0000 7f5fed800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5fe809b950 con 0x7f5fe80c9b10 2024-04-20T21:30:57.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.442+0000 7f5fe7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fcc003a20 0x7f5fe80c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:57.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.442+0000 7f5fe7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fcc003a20 0x7f5fe80c95d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46536/0 (socket says 172.21.15.86:46536) 2024-04-20T21:30:57.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.442+0000 7f5fe7e00640 1 -- 172.21.15.86:0/358845179 learned_addr learned my addr 172.21.15.86:0/358845179 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:57.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.442+0000 7f5fe7400640 1 --2- 172.21.15.86:0/358845179 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5fe80c9b10 0x7f5fe80c7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:57.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.442+0000 7f5fe7e00640 1 -- 172.21.15.86:0/358845179 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5fe80c9b10 msgr2=0x7f5fe80c7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:57.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.442+0000 7f5fece00640 1 --2- 172.21.15.86:0/358845179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fe80c75b0 0x7f5fe8157430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:57.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.442+0000 7f5fe7e00640 1 --2- 172.21.15.86:0/358845179 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5fe80c9b10 0x7f5fe80c7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:57.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.442+0000 7f5fe7e00640 1 -- 172.21.15.86:0/358845179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fe80c75b0 msgr2=0x7f5fe8157430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:57.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.442+0000 7f5fe7e00640 1 --2- 172.21.15.86:0/358845179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fe80c75b0 0x7f5fe8157430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:57.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.442+0000 7f5fe7e00640 1 -- 172.21.15.86:0/358845179 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe80c7c50 con 0x7f5fcc003a20 2024-04-20T21:30:57.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.442+0000 7f5fe7400640 1 --2- 172.21.15.86:0/358845179 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5fe80c9b10 0x7f5fe80c7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:30:57.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.443+0000 7f5fe7e00640 1 --2- 172.21.15.86:0/358845179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fcc003a20 0x7f5fe80c95d0 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f5fe000c330 tx=0x7f5fe0001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:57.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.443+0000 7f5fdfe00640 1 -- 172.21.15.86:0/358845179 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe0007d50 con 0x7f5fcc003a20 2024-04-20T21:30:57.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.443+0000 7f5fdfe00640 1 -- 172.21.15.86:0/358845179 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fe0004bf0 con 0x7f5fcc003a20 2024-04-20T21:30:57.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.443+0000 7f5fdfe00640 1 -- 172.21.15.86:0/358845179 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe0001c90 con 0x7f5fcc003a20 2024-04-20T21:30:57.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.443+0000 7f5fed800640 1 -- 172.21.15.86:0/358845179 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fe80c7f30 con 0x7f5fcc003a20 2024-04-20T21:30:57.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.443+0000 7f5fed800640 1 -- 172.21.15.86:0/358845179 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fe80c84f0 con 0x7f5fcc003a20 2024-04-20T21:30:57.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.444+0000 7f5fed800640 1 -- 172.21.15.86:0/358845179 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fe804fd90 con 0x7f5fcc003a20 2024-04-20T21:30:57.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.445+0000 7f5fdfe00640 1 -- 172.21.15.86:0/358845179 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5fe0047020 con 0x7f5fcc003a20 2024-04-20T21:30:57.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.446+0000 7f5fdfe00640 1 --2- 172.21.15.86:0/358845179 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5fc406c8f0 0x7f5fc406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:57.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.447+0000 7f5fdfe00640 1 -- 172.21.15.86:0/358845179 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7f5fe00ace20 con 0x7f5fcc003a20 2024-04-20T21:30:57.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.447+0000 7f5fe7400640 1 --2- 172.21.15.86:0/358845179 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5fc406c8f0 0x7f5fc406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:57.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.447+0000 7f5fe7400640 1 --2- 172.21.15.86:0/358845179 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5fc406c8f0 0x7f5fc406edb0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f5fe80c8d60 tx=0x7f5fd000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:57.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.451+0000 7f5fdfe00640 1 -- 172.21.15.86:0/358845179 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5fe007b140 con 0x7f5fcc003a20 2024-04-20T21:30:57.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.651+0000 7f5fed800640 1 -- 172.21.15.86:0/358845179 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5fe8069300 con 0x7f5fcc003a20 2024-04-20T21:30:57.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.652+0000 7f5fdfe00640 1 -- 172.21.15.86:0/358845179 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v116) ==== 104+0+11 (secure 0 0 0) 0x7f5fe007aa40 con 0x7f5fcc003a20 2024-04-20T21:30:57.651 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-20T21:30:57.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.655+0000 7f5fed800640 1 -- 172.21.15.86:0/358845179 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5fc406c8f0 msgr2=0x7f5fc406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:57.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.655+0000 7f5fed800640 1 --2- 172.21.15.86:0/358845179 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5fc406c8f0 0x7f5fc406edb0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f5fe80c8d60 tx=0x7f5fd000b040 comp rx=0 tx=0).stop 2024-04-20T21:30:57.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.655+0000 7f5fed800640 1 -- 172.21.15.86:0/358845179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fcc003a20 msgr2=0x7f5fe80c95d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:57.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.655+0000 7f5fed800640 1 --2- 172.21.15.86:0/358845179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fcc003a20 0x7f5fe80c95d0 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f5fe000c330 tx=0x7f5fe0001a40 comp rx=0 tx=0).stop 2024-04-20T21:30:57.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.655+0000 7f5fed800640 1 -- 172.21.15.86:0/358845179 shutdown_connections 2024-04-20T21:30:57.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.655+0000 7f5fed800640 1 --2- 172.21.15.86:0/358845179 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5fc406c8f0 0x7f5fc406edb0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:57.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.655+0000 7f5fed800640 1 --2- 172.21.15.86:0/358845179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5fe80c75b0 0x7f5fe8157430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:57.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.655+0000 7f5fed800640 1 --2- 172.21.15.86:0/358845179 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5fe80c9b10 0x7f5fe80c7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:57.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.655+0000 7f5fed800640 1 --2- 172.21.15.86:0/358845179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5fcc003a20 0x7f5fe80c95d0 unknown :-1 s=CLOSED pgs=2098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:57.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.655+0000 7f5fed800640 1 -- 172.21.15.86:0/358845179 >> 172.21.15.86:0/358845179 conn(0x7f5fe80b3ff0 msgr2=0x7f5fe80b1200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:57.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.655+0000 7f5fed800640 1 -- 172.21.15.86:0/358845179 shutdown_connections 2024-04-20T21:30:57.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.656+0000 7f5fed800640 1 -- 172.21.15.86:0/358845179 wait complete. 2024-04-20T21:30:57.674 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T21:30:57.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.876+0000 7ffb52400640 1 Processor -- start 2024-04-20T21:30:57.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.876+0000 7ffb52400640 1 -- start start 2024-04-20T21:30:57.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.877+0000 7ffb52400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb4c0af1f0 0x7ffb4c06ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:57.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.877+0000 7ffb52400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7ffb4c06d9c0 0x7ffb4c15d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:57.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.877+0000 7ffb52400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ffb4c0af5d0 0x7ffb4c15de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:57.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.877+0000 7ffb52400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ffb4c06f460 con 0x7ffb4c0af5d0 2024-04-20T21:30:57.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.877+0000 7ffb52400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7ffb4c15da60 con 0x7ffb4c06d9c0 2024-04-20T21:30:57.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.877+0000 7ffb52400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ffb4c06de80 con 0x7ffb4c0af1f0 2024-04-20T21:30:57.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.878+0000 7ffb51a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ffb4c0af5d0 0x7ffb4c15de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:57.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.878+0000 7ffb51a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ffb4c0af5d0 0x7ffb4c15de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46548/0 (socket says 172.21.15.86:46548) 2024-04-20T21:30:57.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.878+0000 7ffb51a00640 1 -- 172.21.15.86:0/567295203 learned_addr learned my addr 172.21.15.86:0/567295203 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:57.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.878+0000 7ffb51000640 1 --2- 172.21.15.86:0/567295203 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb4c0af1f0 0x7ffb4c06ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:57.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.878+0000 7ffb4be00640 1 --2- 172.21.15.86:0/567295203 >> v2:172.21.15.144:3300/0 conn(0x7ffb4c06d9c0 0x7ffb4c15d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:57.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.879+0000 7ffb51a00640 1 -- 172.21.15.86:0/567295203 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb4c0af1f0 msgr2=0x7ffb4c06ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:57.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.879+0000 7ffb51a00640 1 --2- 172.21.15.86:0/567295203 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb4c0af1f0 0x7ffb4c06ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:57.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.879+0000 7ffb51a00640 1 -- 172.21.15.86:0/567295203 >> v2:172.21.15.144:3300/0 conn(0x7ffb4c06d9c0 msgr2=0x7ffb4c15d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:57.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.879+0000 7ffb51a00640 1 --2- 172.21.15.86:0/567295203 >> v2:172.21.15.144:3300/0 conn(0x7ffb4c06d9c0 0x7ffb4c15d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:57.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.879+0000 7ffb51a00640 1 -- 172.21.15.86:0/567295203 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb4c06e160 con 0x7ffb4c0af5d0 2024-04-20T21:30:57.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.879+0000 7ffb51000640 1 --2- 172.21.15.86:0/567295203 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb4c0af1f0 0x7ffb4c06ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:30:57.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.879+0000 7ffb4be00640 1 --2- 172.21.15.86:0/567295203 >> v2:172.21.15.144:3300/0 conn(0x7ffb4c06d9c0 0x7ffb4c15d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:30:57.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.880+0000 7ffb51a00640 1 --2- 172.21.15.86:0/567295203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb4c0af5d0 0x7ffb4c15de70 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c074160 tx=0x7ffb4402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f50afddd752d88a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:57.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.880+0000 7ffb4b400640 1 -- 172.21.15.86:0/567295203 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4402ca80 con 0x7ffb4c0af5d0 2024-04-20T21:30:57.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.880+0000 7ffb4b400640 1 -- 172.21.15.86:0/567295203 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb4402cc20 con 0x7ffb4c0af5d0 2024-04-20T21:30:57.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.880+0000 7ffb4b400640 1 -- 172.21.15.86:0/567295203 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb440343f0 con 0x7ffb4c0af5d0 2024-04-20T21:30:57.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.881+0000 7ffb52400640 1 -- 172.21.15.86:0/567295203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb4c0af5d0 msgr2=0x7ffb4c15de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:57.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.881+0000 7ffb52400640 1 --2- 172.21.15.86:0/567295203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb4c0af5d0 0x7ffb4c15de70 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c074160 tx=0x7ffb4402bee0 comp rx=0 tx=0).stop 2024-04-20T21:30:57.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.881+0000 7ffb52400640 1 -- 172.21.15.86:0/567295203 shutdown_connections 2024-04-20T21:30:57.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.881+0000 7ffb52400640 1 --2- 172.21.15.86:0/567295203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb4c0af5d0 0x7ffb4c15de70 unknown :-1 s=CLOSED pgs=2099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:57.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.881+0000 7ffb52400640 1 --2- 172.21.15.86:0/567295203 >> v2:172.21.15.144:3300/0 conn(0x7ffb4c06d9c0 0x7ffb4c15d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:57.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.881+0000 7ffb52400640 1 --2- 172.21.15.86:0/567295203 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb4c0af1f0 0x7ffb4c06ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:57.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.881+0000 7ffb52400640 1 -- 172.21.15.86:0/567295203 >> 172.21.15.86:0/567295203 conn(0x7ffb4c0b1f00 msgr2=0x7ffb4c0b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:57.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.881+0000 7ffb52400640 1 -- 172.21.15.86:0/567295203 shutdown_connections 2024-04-20T21:30:57.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.881+0000 7ffb52400640 1 -- 172.21.15.86:0/567295203 wait complete. 2024-04-20T21:30:57.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.883+0000 7ffb52400640 1 Processor -- start 2024-04-20T21:30:57.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.884+0000 7ffb52400640 1 -- start start 2024-04-20T21:30:57.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.885+0000 7ffb52400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb4c06d9c0 0x7ffb4c1f4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:57.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.885+0000 7ffb52400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb4c0af1f0 0x7ffb4c155570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:57.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.885+0000 7ffb52400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb4c0af5d0 0x7ffb4c143010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:57.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.885+0000 7ffb52400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffb4c06ea00 con 0x7ffb4c06d9c0 2024-04-20T21:30:57.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.885+0000 7ffb52400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ffb4c06e720 con 0x7ffb4c0af1f0 2024-04-20T21:30:57.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.885+0000 7ffb52400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ffb4c0afa80 con 0x7ffb4c0af5d0 2024-04-20T21:30:57.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.885+0000 7ffb51000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb4c06d9c0 0x7ffb4c1f4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:57.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.885+0000 7ffb51000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb4c06d9c0 0x7ffb4c1f4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46564/0 (socket says 172.21.15.86:46564) 2024-04-20T21:30:57.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.885+0000 7ffb51000640 1 -- 172.21.15.86:0/876950884 learned_addr learned my addr 172.21.15.86:0/876950884 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:57.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.885+0000 7ffb4be00640 1 --2- 172.21.15.86:0/876950884 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb4c0af1f0 0x7ffb4c155570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:57.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.885+0000 7ffb51000640 1 -- 172.21.15.86:0/876950884 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb4c0af5d0 msgr2=0x7ffb4c143010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:57.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.885+0000 7ffb51a00640 1 --2- 172.21.15.86:0/876950884 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb4c0af5d0 0x7ffb4c143010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:57.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.886+0000 7ffb51000640 1 --2- 172.21.15.86:0/876950884 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb4c0af5d0 0x7ffb4c143010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:57.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.886+0000 7ffb51000640 1 -- 172.21.15.86:0/876950884 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb4c0af1f0 msgr2=0x7ffb4c155570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:57.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.886+0000 7ffb51000640 1 --2- 172.21.15.86:0/876950884 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb4c0af1f0 0x7ffb4c155570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:57.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.886+0000 7ffb51000640 1 -- 172.21.15.86:0/876950884 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb4c152bb0 con 0x7ffb4c06d9c0 2024-04-20T21:30:57.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.886+0000 7ffb51000640 1 --2- 172.21.15.86:0/876950884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb4c06d9c0 0x7ffb4c1f4220 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7ffb4000d7d0 tx=0x7ffb4000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:57.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.886+0000 7ffb48c00640 1 -- 172.21.15.86:0/876950884 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb40004030 con 0x7ffb4c06d9c0 2024-04-20T21:30:57.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.886+0000 7ffb48c00640 1 -- 172.21.15.86:0/876950884 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb400041d0 con 0x7ffb4c06d9c0 2024-04-20T21:30:57.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.886+0000 7ffb52400640 1 -- 172.21.15.86:0/876950884 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb4c152e90 con 0x7ffb4c06d9c0 2024-04-20T21:30:57.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.886+0000 7ffb48c00640 1 -- 172.21.15.86:0/876950884 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb400096a0 con 0x7ffb4c06d9c0 2024-04-20T21:30:57.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.886+0000 7ffb52400640 1 -- 172.21.15.86:0/876950884 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb4c153450 con 0x7ffb4c06d9c0 2024-04-20T21:30:57.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.888+0000 7ffb52400640 1 -- 172.21.15.86:0/876950884 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb4c0adbf0 con 0x7ffb4c06d9c0 2024-04-20T21:30:57.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.889+0000 7ffb48c00640 1 -- 172.21.15.86:0/876950884 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffb40009840 con 0x7ffb4c06d9c0 2024-04-20T21:30:57.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.889+0000 7ffb48c00640 1 --2- 172.21.15.86:0/876950884 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ffb3006c8f0 0x7ffb3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:57.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.890+0000 7ffb48c00640 1 -- 172.21.15.86:0/876950884 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7ffb4008bfc0 con 0x7ffb4c06d9c0 2024-04-20T21:30:57.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.890+0000 7ffb4be00640 1 --2- 172.21.15.86:0/876950884 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ffb3006c8f0 0x7ffb3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:57.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.890+0000 7ffb4be00640 1 --2- 172.21.15.86:0/876950884 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ffb3006c8f0 0x7ffb3006edb0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c006fd0 tx=0x7ffb3c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:57.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:57.895+0000 7ffb48c00640 1 -- 172.21.15.86:0/876950884 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ffb4c0adbf0 con 0x7ffb4c06d9c0 2024-04-20T21:30:58.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.096+0000 7ffb52400640 1 -- 172.21.15.86:0/876950884 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ffb4c0681f0 con 0x7ffb4c06d9c0 2024-04-20T21:30:58.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.097+0000 7ffb48c00640 1 -- 172.21.15.86:0/876950884 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v116) ==== 100+0+11 (secure 0 0 0) 0x7ffb4c0681f0 con 0x7ffb4c06d9c0 2024-04-20T21:30:58.096 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-20T21:30:58.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.100+0000 7ffb52400640 1 -- 172.21.15.86:0/876950884 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ffb3006c8f0 msgr2=0x7ffb3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:58.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.100+0000 7ffb52400640 1 --2- 172.21.15.86:0/876950884 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ffb3006c8f0 0x7ffb3006edb0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c006fd0 tx=0x7ffb3c008040 comp rx=0 tx=0).stop 2024-04-20T21:30:58.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.100+0000 7ffb52400640 1 -- 172.21.15.86:0/876950884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb4c06d9c0 msgr2=0x7ffb4c1f4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:58.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.100+0000 7ffb52400640 1 --2- 172.21.15.86:0/876950884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb4c06d9c0 0x7ffb4c1f4220 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7ffb4000d7d0 tx=0x7ffb4000dc90 comp rx=0 tx=0).stop 2024-04-20T21:30:58.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.100+0000 7ffb52400640 1 -- 172.21.15.86:0/876950884 shutdown_connections 2024-04-20T21:30:58.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.100+0000 7ffb52400640 1 --2- 172.21.15.86:0/876950884 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ffb3006c8f0 0x7ffb3006edb0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:58.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.100+0000 7ffb52400640 1 --2- 172.21.15.86:0/876950884 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ffb4c0af5d0 0x7ffb4c143010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:58.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.100+0000 7ffb52400640 1 --2- 172.21.15.86:0/876950884 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffb4c0af1f0 0x7ffb4c155570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:58.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.100+0000 7ffb52400640 1 --2- 172.21.15.86:0/876950884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb4c06d9c0 0x7ffb4c1f4220 unknown :-1 s=CLOSED pgs=2100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:58.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.100+0000 7ffb52400640 1 -- 172.21.15.86:0/876950884 >> 172.21.15.86:0/876950884 conn(0x7ffb4c0b1f00 msgr2=0x7ffb4c0d00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:58.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.100+0000 7ffb52400640 1 -- 172.21.15.86:0/876950884 shutdown_connections 2024-04-20T21:30:58.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.100+0000 7ffb52400640 1 -- 172.21.15.86:0/876950884 wait complete. 2024-04-20T21:30:58.117 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:30:58.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.315+0000 7f3447200640 1 Processor -- start 2024-04-20T21:30:58.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.315+0000 7f3447200640 1 -- start start 2024-04-20T21:30:58.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.316+0000 7f3447200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3440156ae0 0x7f344015d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:58.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.316+0000 7f3447200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3440160890 con 0x7f34400ad7f0 2024-04-20T21:30:58.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.316+0000 7f3447200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3440160b70 con 0x7f3440160080 2024-04-20T21:30:58.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.316+0000 7f3447200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f344006db20 con 0x7f3440156ae0 2024-04-20T21:30:58.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.316+0000 7f3446800640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f34400ad7f0 0x7f344006e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57260/0 (socket says 172.21.15.86:57260) 2024-04-20T21:30:58.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.316+0000 7f3446800640 1 -- 172.21.15.86:0/2717401068 learned_addr learned my addr 172.21.15.86:0/2717401068 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:30:58.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.317+0000 7f3445e00640 1 --2- 172.21.15.86:0/2717401068 >> v2:172.21.15.144:3300/0 conn(0x7f3440156ae0 0x7f344015d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:58.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.317+0000 7f3444a00640 1 -- 172.21.15.86:0/2717401068 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 707548116 0 0) 0x7f3440160890 con 0x7f34400ad7f0 2024-04-20T21:30:58.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.317+0000 7f3444a00640 1 -- 172.21.15.86:0/2717401068 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3428003520 con 0x7f34400ad7f0 2024-04-20T21:30:58.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.317+0000 7f3444a00640 1 -- 172.21.15.86:0/2717401068 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 250571970 0 0) 0x7f3428003520 con 0x7f34400ad7f0 2024-04-20T21:30:58.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.317+0000 7f3444a00640 1 -- 172.21.15.86:0/2717401068 >> v2:172.21.15.144:3300/0 conn(0x7f3440156ae0 msgr2=0x7f344015d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:58.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.317+0000 7f3444a00640 1 --2- 172.21.15.86:0/2717401068 >> v2:172.21.15.144:3300/0 conn(0x7f3440156ae0 0x7f344015d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:58.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.317+0000 7f3444a00640 1 -- 172.21.15.86:0/2717401068 >> v1:172.21.15.144:6789/0 conn(0x7f3440160080 legacy=0x7f34400ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:58.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.317+0000 7f3444a00640 1 -- 172.21.15.86:0/2717401068 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3440160e50 con 0x7f34400ad7f0 2024-04-20T21:30:58.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.317+0000 7f3444a00640 1 -- 172.21.15.86:0/2717401068 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f343c0019f0 con 0x7f34400ad7f0 2024-04-20T21:30:58.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.318+0000 7f3444a00640 1 -- 172.21.15.86:0/2717401068 >> v1:172.21.15.86:6789/0 conn(0x7f34400ad7f0 legacy=0x7f344006e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:58.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.318+0000 7f3444a00640 1 --2- 172.21.15.86:0/2717401068 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34280039b0 0x7f3428003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:58.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.318+0000 7f3444a00640 1 --2- 172.21.15.86:0/2717401068 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3428004800 0x7f3428004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:58.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.318+0000 7f3444a00640 1 --2- 172.21.15.86:0/2717401068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34280051c0 0x7f342800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:58.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.318+0000 7f3444a00640 1 -- 172.21.15.86:0/2717401068 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f344006db20 con 0x7f34280051c0 2024-04-20T21:30:58.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.318+0000 7f3444a00640 1 -- 172.21.15.86:0/2717401068 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f342800c040 con 0x7f34280039b0 2024-04-20T21:30:58.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.318+0000 7f3444a00640 1 -- 172.21.15.86:0/2717401068 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f342800c240 con 0x7f3428004800 2024-04-20T21:30:58.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.318+0000 7f3446800640 1 --2- 172.21.15.86:0/2717401068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34280051c0 0x7f342800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:58.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.318+0000 7f3445400640 1 --2- 172.21.15.86:0/2717401068 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3428004800 0x7f3428004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:58.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.318+0000 7f3446800640 1 -- 172.21.15.86:0/2717401068 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3428004800 msgr2=0x7f3428004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:58.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.318+0000 7f3446800640 1 --2- 172.21.15.86:0/2717401068 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3428004800 0x7f3428004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:58.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.318+0000 7f3445e00640 1 --2- 172.21.15.86:0/2717401068 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34280039b0 0x7f3428003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:58.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.318+0000 7f3446800640 1 -- 172.21.15.86:0/2717401068 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34280039b0 msgr2=0x7f3428003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:58.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.318+0000 7f3446800640 1 --2- 172.21.15.86:0/2717401068 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34280039b0 0x7f3428003db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:58.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.319+0000 7f3446800640 1 -- 172.21.15.86:0/2717401068 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3428003520 con 0x7f34280051c0 2024-04-20T21:30:58.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.319+0000 7f3445400640 1 --2- 172.21.15.86:0/2717401068 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3428004800 0x7f3428004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:30:58.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.319+0000 7f3446800640 1 -- 172.21.15.86:0/2717401068 reap_dead start 2024-04-20T21:30:58.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.320+0000 7f3446800640 1 --2- 172.21.15.86:0/2717401068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34280051c0 0x7f342800b9c0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7f343c00a8f0 tx=0x7f3428005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9ec29a9979999fd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:58.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.320+0000 7f3444a00640 1 -- 172.21.15.86:0/2717401068 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343c00a4c0 con 0x7f34280051c0 2024-04-20T21:30:58.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.320+0000 7f3444a00640 1 -- 172.21.15.86:0/2717401068 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f343c034650 con 0x7f34280051c0 2024-04-20T21:30:58.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.320+0000 7f3444a00640 1 -- 172.21.15.86:0/2717401068 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343c034a40 con 0x7f34280051c0 2024-04-20T21:30:58.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.320+0000 7f3447200640 1 -- 172.21.15.86:0/2717401068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34280051c0 msgr2=0x7f342800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:58.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.320+0000 7f3447200640 1 --2- 172.21.15.86:0/2717401068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34280051c0 0x7f342800b9c0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7f343c00a8f0 tx=0x7f3428005830 comp rx=0 tx=0).stop 2024-04-20T21:30:58.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.321+0000 7f3447200640 1 -- 172.21.15.86:0/2717401068 shutdown_connections 2024-04-20T21:30:58.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.321+0000 7f3447200640 1 --2- 172.21.15.86:0/2717401068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f34280051c0 0x7f342800b9c0 unknown :-1 s=CLOSED pgs=2102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:58.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.321+0000 7f3447200640 1 -- 172.21.15.86:0/2717401068 >> 172.21.15.86:0/2717401068 conn(0x7f34400d0170 msgr2=0x7f34400d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:58.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.321+0000 7f3447200640 1 -- 172.21.15.86:0/2717401068 shutdown_connections 2024-04-20T21:30:58.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.321+0000 7f3447200640 1 -- 172.21.15.86:0/2717401068 wait complete. 2024-04-20T21:30:58.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.322+0000 7f3447200640 1 Processor -- start 2024-04-20T21:30:58.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.322+0000 7f3447200640 1 -- start start 2024-04-20T21:30:58.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.323+0000 7f3447200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34280051c0 0x7f3440159b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:58.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.323+0000 7f3447200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f344015c110 0x7f34401f3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:58.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.323+0000 7f3447200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f34401452f0 0x7f3440145770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:58.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.323+0000 7f3447200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3440160500 con 0x7f344015c110 2024-04-20T21:30:58.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.323+0000 7f3447200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3440156ec0 con 0x7f34280051c0 2024-04-20T21:30:58.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.323+0000 7f3447200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f344015d6a0 con 0x7f34401452f0 2024-04-20T21:30:58.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.324+0000 7f3445400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f344015c110 0x7f34401f3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:58.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.324+0000 7f3445400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f344015c110 0x7f34401f3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46586/0 (socket says 172.21.15.86:46586) 2024-04-20T21:30:58.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.324+0000 7f3445400640 1 -- 172.21.15.86:0/2116636045 learned_addr learned my addr 172.21.15.86:0/2116636045 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:58.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.324+0000 7f3445e00640 1 --2- 172.21.15.86:0/2116636045 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34280051c0 0x7f3440159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:58.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.324+0000 7f3445400640 1 -- 172.21.15.86:0/2116636045 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f34401452f0 msgr2=0x7f3440145770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:58.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.324+0000 7f3445400640 1 --2- 172.21.15.86:0/2116636045 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f34401452f0 0x7f3440145770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:58.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.324+0000 7f3445400640 1 -- 172.21.15.86:0/2116636045 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34280051c0 msgr2=0x7f3440159b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:58.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.324+0000 7f3445400640 1 --2- 172.21.15.86:0/2116636045 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34280051c0 0x7f3440159b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:58.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.324+0000 7f3445400640 1 -- 172.21.15.86:0/2116636045 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f344015c960 con 0x7f344015c110 2024-04-20T21:30:58.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.324+0000 7f3445400640 1 --2- 172.21.15.86:0/2116636045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f344015c110 0x7f34401f3c00 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f343000d5f0 tx=0x7f343000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:58.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.325+0000 7f343a000640 1 -- 172.21.15.86:0/2116636045 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3430016070 con 0x7f344015c110 2024-04-20T21:30:58.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.325+0000 7f343a000640 1 -- 172.21.15.86:0/2116636045 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3430007240 con 0x7f344015c110 2024-04-20T21:30:58.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.325+0000 7f343a000640 1 -- 172.21.15.86:0/2116636045 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3430007630 con 0x7f344015c110 2024-04-20T21:30:58.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.325+0000 7f3447200640 1 -- 172.21.15.86:0/2116636045 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f344015cc40 con 0x7f344015c110 2024-04-20T21:30:58.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.325+0000 7f3447200640 1 -- 172.21.15.86:0/2116636045 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3440142f20 con 0x7f344015c110 2024-04-20T21:30:58.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.326+0000 7f3447200640 1 -- 172.21.15.86:0/2116636045 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3440143120 con 0x7f344015c110 2024-04-20T21:30:58.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.327+0000 7f343a000640 1 -- 172.21.15.86:0/2116636045 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f34300067d0 con 0x7f344015c110 2024-04-20T21:30:58.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.328+0000 7f343a000640 1 --2- 172.21.15.86:0/2116636045 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f342406c920 0x7f342406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:58.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.328+0000 7f343a000640 1 -- 172.21.15.86:0/2116636045 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7f343008ba90 con 0x7f344015c110 2024-04-20T21:30:58.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.328+0000 7f3445e00640 1 --2- 172.21.15.86:0/2116636045 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f342406c920 0x7f342406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:58.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.329+0000 7f3445e00640 1 --2- 172.21.15.86:0/2116636045 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f342406c920 0x7f342406ede0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f3434009910 tx=0x7f3434008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:58.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.332+0000 7f343a000640 1 -- 172.21.15.86:0/2116636045 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3440143120 con 0x7f344015c110 2024-04-20T21:30:58.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.567+0000 7f3447200640 1 -- 172.21.15.86:0/2116636045 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f344004fcf0 con 0x7f344015c110 2024-04-20T21:30:58.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.568+0000 7f343a000640 1 -- 172.21.15.86:0/2116636045 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5212 (secure 0 0 0) 0x7f3430059dc0 con 0x7f344015c110 2024-04-20T21:30:58.567 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:58.567 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":112,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6253,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3627230007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3627230007},{"type":"v1","addr":"172.21.15.144:6833","nonce":3627230007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":6256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2498297133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2498297133},{"type":"v1","addr":"172.21.15.144:6835","nonce":2498297133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":9228,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/1120740475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1120740475},{"type":"v1","addr":"172.21.15.86:6836","nonce":1120740475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:29:43.769581+0000","modified":"2024-04-20T21:30:47.137068+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9792},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9792":{"gid":9792,"name":"a","rank":0,"incarnation":109,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6837/2019839930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2019839930},{"type":"v1","addr":"172.21.15.86:6837","nonce":2019839930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9792,"qdb_cluster":[9792]},"id":8}]} 2024-04-20T21:30:58.568 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 112 2024-04-20T21:30:58.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.571+0000 7f3447200640 1 -- 172.21.15.86:0/2116636045 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f342406c920 msgr2=0x7f342406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:58.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.571+0000 7f3447200640 1 --2- 172.21.15.86:0/2116636045 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f342406c920 0x7f342406ede0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f3434009910 tx=0x7f3434008040 comp rx=0 tx=0).stop 2024-04-20T21:30:58.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.571+0000 7f3447200640 1 -- 172.21.15.86:0/2116636045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f344015c110 msgr2=0x7f34401f3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:58.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.571+0000 7f3447200640 1 --2- 172.21.15.86:0/2116636045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f344015c110 0x7f34401f3c00 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f343000d5f0 tx=0x7f343000dab0 comp rx=0 tx=0).stop 2024-04-20T21:30:58.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.571+0000 7f3447200640 1 -- 172.21.15.86:0/2116636045 shutdown_connections 2024-04-20T21:30:58.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.571+0000 7f3447200640 1 --2- 172.21.15.86:0/2116636045 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f342406c920 0x7f342406ede0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:58.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.571+0000 7f3447200640 1 --2- 172.21.15.86:0/2116636045 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f34401452f0 0x7f3440145770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:58.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.571+0000 7f3447200640 1 --2- 172.21.15.86:0/2116636045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f344015c110 0x7f34401f3c00 unknown :-1 s=CLOSED pgs=2103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:58.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.571+0000 7f3447200640 1 --2- 172.21.15.86:0/2116636045 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34280051c0 0x7f3440159b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:58.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.571+0000 7f3447200640 1 -- 172.21.15.86:0/2116636045 >> 172.21.15.86:0/2116636045 conn(0x7f34400d0170 msgr2=0x7f34400b1360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:58.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.571+0000 7f3447200640 1 -- 172.21.15.86:0/2116636045 shutdown_connections 2024-04-20T21:30:58.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.571+0000 7f3447200640 1 -- 172.21.15.86:0/2116636045 wait complete. 2024-04-20T21:30:58.587 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:30:58.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.786+0000 7f8c34200640 1 Processor -- start 2024-04-20T21:30:58.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.786+0000 7f8c34200640 1 -- start start 2024-04-20T21:30:58.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.788+0000 7f8c34200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8c2c146130 0x7f8c2c06e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:58.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.788+0000 7f8c34200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8c2c0ccdd0 0x7f8c2c0cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:58.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.788+0000 7f8c34200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c2c06dc30 con 0x7f8c2c071d00 2024-04-20T21:30:58.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.788+0000 7f8c34200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f8c2c0cd8e0 con 0x7f8c2c146130 2024-04-20T21:30:58.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.788+0000 7f8c34200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f8c2c0cdbc0 con 0x7f8c2c0ccdd0 2024-04-20T21:30:58.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.788+0000 7f8c32200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f8c2c071d00 0x7f8c2c15cc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57268/0 (socket says 172.21.15.86:57268) 2024-04-20T21:30:58.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.788+0000 7f8c31800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8c2c146130 0x7f8c2c06e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:58.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.788+0000 7f8c32200640 1 -- 172.21.15.86:0/363955407 learned_addr learned my addr 172.21.15.86:0/363955407 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:30:58.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.789+0000 7f8c30e00640 1 --2- 172.21.15.86:0/363955407 >> v2:172.21.15.144:3300/0 conn(0x7f8c2c0ccdd0 0x7f8c2c0cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:58.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.789+0000 7f8c31800640 1 -- 172.21.15.86:0/363955407 >> v2:172.21.15.144:3300/0 conn(0x7f8c2c0ccdd0 msgr2=0x7f8c2c0cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:58.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.789+0000 7f8c31800640 1 --2- 172.21.15.86:0/363955407 >> v2:172.21.15.144:3300/0 conn(0x7f8c2c0ccdd0 0x7f8c2c0cd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:58.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.789+0000 7f8c31800640 1 -- 172.21.15.86:0/363955407 >> v1:172.21.15.86:6789/0 conn(0x7f8c2c071d00 legacy=0x7f8c2c15cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:58.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.789+0000 7f8c2be00640 1 -- 172.21.15.86:0/363955407 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1076801364 0 0) 0x7f8c2c06dc30 con 0x7f8c2c071d00 2024-04-20T21:30:58.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.789+0000 7f8c31800640 1 -- 172.21.15.86:0/363955407 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c2c06ded0 con 0x7f8c2c146130 2024-04-20T21:30:58.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.789+0000 7f8c30e00640 1 --2- 172.21.15.86:0/363955407 >> v2:172.21.15.144:3300/0 conn(0x7f8c2c0ccdd0 0x7f8c2c0cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:30:58.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.790+0000 7f8c31800640 1 --2- 172.21.15.86:0/363955407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c2c146130 0x7f8c2c06e6d0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c06de00 tx=0x7f8c2002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ee00db4f3dfbf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:58.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.790+0000 7f8c2be00640 1 -- 172.21.15.86:0/363955407 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c20034070 con 0x7f8c2c146130 2024-04-20T21:30:58.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.790+0000 7f8c2be00640 1 -- 172.21.15.86:0/363955407 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c20030610 con 0x7f8c2c146130 2024-04-20T21:30:58.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.790+0000 7f8c2be00640 1 -- 172.21.15.86:0/363955407 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c200309e0 con 0x7f8c2c146130 2024-04-20T21:30:58.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.791+0000 7f8c34200640 1 -- 172.21.15.86:0/363955407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c2c146130 msgr2=0x7f8c2c06e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:58.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.791+0000 7f8c34200640 1 --2- 172.21.15.86:0/363955407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c2c146130 0x7f8c2c06e6d0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c06de00 tx=0x7f8c2002b960 comp rx=0 tx=0).stop 2024-04-20T21:30:58.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.791+0000 7f8c34200640 1 -- 172.21.15.86:0/363955407 shutdown_connections 2024-04-20T21:30:58.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.791+0000 7f8c34200640 1 --2- 172.21.15.86:0/363955407 >> v2:172.21.15.144:3300/0 conn(0x7f8c2c0ccdd0 0x7f8c2c0cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:58.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.791+0000 7f8c34200640 1 --2- 172.21.15.86:0/363955407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c2c146130 0x7f8c2c06e6d0 unknown :-1 s=CLOSED pgs=2105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:58.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.791+0000 7f8c34200640 1 -- 172.21.15.86:0/363955407 >> 172.21.15.86:0/363955407 conn(0x7f8c2c0b2360 msgr2=0x7f8c2c0b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:58.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.791+0000 7f8c34200640 1 -- 172.21.15.86:0/363955407 shutdown_connections 2024-04-20T21:30:58.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.791+0000 7f8c34200640 1 -- 172.21.15.86:0/363955407 wait complete. 2024-04-20T21:30:58.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.793+0000 7f8c34200640 1 Processor -- start 2024-04-20T21:30:58.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.793+0000 7f8c34200640 1 -- start start 2024-04-20T21:30:58.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.794+0000 7f8c34200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c2c071d00 0x7f8c2c15cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:58.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.794+0000 7f8c34200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c2c0ccdd0 0x7f8c2c15c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:58.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.794+0000 7f8c34200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c2c146130 0x7f8c2c15f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:58.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.794+0000 7f8c34200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2c072150 con 0x7f8c2c146130 2024-04-20T21:30:58.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.794+0000 7f8c34200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2c06f150 con 0x7f8c2c071d00 2024-04-20T21:30:58.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.794+0000 7f8c34200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8c2c0722d0 con 0x7f8c2c0ccdd0 2024-04-20T21:30:58.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.794+0000 7f8c32200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c2c146130 0x7f8c2c15f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:58.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.795+0000 7f8c32200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c2c146130 0x7f8c2c15f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46614/0 (socket says 172.21.15.86:46614) 2024-04-20T21:30:58.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.795+0000 7f8c32200640 1 -- 172.21.15.86:0/3753780521 learned_addr learned my addr 172.21.15.86:0/3753780521 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:58.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.795+0000 7f8c31800640 1 --2- 172.21.15.86:0/3753780521 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c2c071d00 0x7f8c2c15cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:58.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.795+0000 7f8c30e00640 1 --2- 172.21.15.86:0/3753780521 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c2c0ccdd0 0x7f8c2c15c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:58.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.795+0000 7f8c32200640 1 -- 172.21.15.86:0/3753780521 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c2c0ccdd0 msgr2=0x7f8c2c15c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:58.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.795+0000 7f8c32200640 1 --2- 172.21.15.86:0/3753780521 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c2c0ccdd0 0x7f8c2c15c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:58.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.795+0000 7f8c32200640 1 -- 172.21.15.86:0/3753780521 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c2c071d00 msgr2=0x7f8c2c15cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:58.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.795+0000 7f8c32200640 1 --2- 172.21.15.86:0/3753780521 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c2c071d00 0x7f8c2c15cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:58.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.795+0000 7f8c32200640 1 -- 172.21.15.86:0/3753780521 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c2c15c620 con 0x7f8c2c146130 2024-04-20T21:30:58.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.795+0000 7f8c32200640 1 --2- 172.21.15.86:0/3753780521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c2c146130 0x7f8c2c15f050 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f8c2400c7a0 tx=0x7f8c2400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:58.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.796+0000 7f8c29600640 1 -- 172.21.15.86:0/3753780521 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c24018070 con 0x7f8c2c146130 2024-04-20T21:30:58.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.796+0000 7f8c29600640 1 -- 172.21.15.86:0/3753780521 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c2400a640 con 0x7f8c2c146130 2024-04-20T21:30:58.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.796+0000 7f8c29600640 1 -- 172.21.15.86:0/3753780521 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2400aab0 con 0x7f8c2c146130 2024-04-20T21:30:58.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.796+0000 7f8c34200640 1 -- 172.21.15.86:0/3753780521 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c2c15d1e0 con 0x7f8c2c146130 2024-04-20T21:30:58.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.796+0000 7f8c34200640 1 -- 172.21.15.86:0/3753780521 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c2c15d7a0 con 0x7f8c2c146130 2024-04-20T21:30:58.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.797+0000 7f8c34200640 1 -- 172.21.15.86:0/3753780521 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c2c0689e0 con 0x7f8c2c146130 2024-04-20T21:30:58.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.798+0000 7f8c29600640 1 -- 172.21.15.86:0/3753780521 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8c2400ac50 con 0x7f8c2c146130 2024-04-20T21:30:58.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.799+0000 7f8c29600640 1 --2- 172.21.15.86:0/3753780521 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8c0c06c8f0 0x7f8c0c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:58.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.799+0000 7f8c29600640 1 -- 172.21.15.86:0/3753780521 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7f8c2408b360 con 0x7f8c2c146130 2024-04-20T21:30:58.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.799+0000 7f8c31800640 1 --2- 172.21.15.86:0/3753780521 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8c0c06c8f0 0x7f8c0c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:58.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.800+0000 7f8c31800640 1 --2- 172.21.15.86:0/3753780521 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8c0c06c8f0 0x7f8c0c06edb0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f8c20036000 tx=0x7f8c20008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:58.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:58.803+0000 7f8c29600640 1 -- 172.21.15.86:0/3753780521 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8c2c15c620 con 0x7f8c2c146130 2024-04-20T21:30:59.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.006+0000 7f8c34200640 1 -- 172.21.15.86:0/3753780521 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8c2c15ff50 con 0x7f8c2c146130 2024-04-20T21:30:59.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.008+0000 7f8c29600640 1 -- 172.21.15.86:0/3753780521 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) ==== 74+0+17037 (secure 0 0 0) 0x7f8c24059600 con 0x7f8c2c146130 2024-04-20T21:30:59.007 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:59.008 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":116,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:30:42.086516+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:29:41.532824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-20T21:29:42.749687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.f","mappings":[{"from":0,"to":5}]},{"pgid":"17.18","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6837/1453621918":"2024-04-21T21:30:42.086474+0000","172.21.15.86:6835/1453621918":"2024-04-21T21:30:42.086474+0000","172.21.15.86:6837/289590924":"2024-04-21T21:30:19.784403+0000","172.21.15.86:6835/289590924":"2024-04-21T21:30:19.784403+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:30:59.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.010+0000 7f8c34200640 1 -- 172.21.15.86:0/3753780521 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8c0c06c8f0 msgr2=0x7f8c0c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:59.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.010+0000 7f8c34200640 1 --2- 172.21.15.86:0/3753780521 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8c0c06c8f0 0x7f8c0c06edb0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f8c20036000 tx=0x7f8c20008030 comp rx=0 tx=0).stop 2024-04-20T21:30:59.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.010+0000 7f8c34200640 1 -- 172.21.15.86:0/3753780521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c2c146130 msgr2=0x7f8c2c15f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:59.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.010+0000 7f8c34200640 1 --2- 172.21.15.86:0/3753780521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c2c146130 0x7f8c2c15f050 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f8c2400c7a0 tx=0x7f8c2400cc60 comp rx=0 tx=0).stop 2024-04-20T21:30:59.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.011+0000 7f8c34200640 1 -- 172.21.15.86:0/3753780521 shutdown_connections 2024-04-20T21:30:59.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.011+0000 7f8c34200640 1 --2- 172.21.15.86:0/3753780521 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f8c0c06c8f0 0x7f8c0c06edb0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:59.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.011+0000 7f8c34200640 1 --2- 172.21.15.86:0/3753780521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c2c146130 0x7f8c2c15f050 unknown :-1 s=CLOSED pgs=2106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:59.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.011+0000 7f8c34200640 1 --2- 172.21.15.86:0/3753780521 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8c2c0ccdd0 0x7f8c2c15c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:59.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.011+0000 7f8c34200640 1 --2- 172.21.15.86:0/3753780521 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c2c071d00 0x7f8c2c15cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:59.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.011+0000 7f8c34200640 1 -- 172.21.15.86:0/3753780521 >> 172.21.15.86:0/3753780521 conn(0x7f8c2c0b2360 msgr2=0x7f8c2c0b4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:59.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.011+0000 7f8c34200640 1 -- 172.21.15.86:0/3753780521 shutdown_connections 2024-04-20T21:30:59.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.011+0000 7f8c34200640 1 -- 172.21.15.86:0/3753780521 wait complete. 2024-04-20T21:30:59.029 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-20T21:30:59.029 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:30:59.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.216+0000 7fc0d9600640 1 Processor -- start 2024-04-20T21:30:59.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.216+0000 7fc0d9600640 1 -- start start 2024-04-20T21:30:59.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.217+0000 7fc0d9600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0d4144e90 0x7fc0d40b4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:59.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.217+0000 7fc0d9600640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc0d40cd170 0x7fc0d406ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:59.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.218+0000 7fc0d9600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0d40cdb50 con 0x7fc0d40cd5d0 2024-04-20T21:30:59.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.218+0000 7fc0d9600640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fc0d4145270 con 0x7fc0d40cd170 2024-04-20T21:30:59.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.218+0000 7fc0d9600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc0d40b3e50 con 0x7fc0d4144e90 2024-04-20T21:30:59.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.218+0000 7fc0d2a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0d4144e90 0x7fc0d40b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:59.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.218+0000 7fc0d2000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fc0d40cd170 0x7fc0d406ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:59.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.218+0000 7fc0d2a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0d4144e90 0x7fc0d40b4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:51196/0 (socket says 172.21.15.86:51196) 2024-04-20T21:30:59.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.218+0000 7fc0d2a00640 1 -- 172.21.15.86:0/555606301 learned_addr learned my addr 172.21.15.86:0/555606301 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:59.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.219+0000 7fc0d2a00640 1 -- 172.21.15.86:0/555606301 >> v2:172.21.15.144:3300/0 conn(0x7fc0d40cd170 msgr2=0x7fc0d406ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:30:59.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.219+0000 7fc0d2a00640 1 --2- 172.21.15.86:0/555606301 >> v2:172.21.15.144:3300/0 conn(0x7fc0d40cd170 0x7fc0d406ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:59.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.219+0000 7fc0d2a00640 1 -- 172.21.15.86:0/555606301 >> v1:172.21.15.144:6789/0 conn(0x7fc0d40cd5d0 legacy=0x7fc0d415bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:59.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.219+0000 7fc0d2a00640 1 -- 172.21.15.86:0/555606301 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0d40b4130 con 0x7fc0d4144e90 2024-04-20T21:30:59.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.220+0000 7fc0d2a00640 1 --2- 172.21.15.86:0/555606301 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0d4144e90 0x7fc0d40b4930 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4009870 tx=0x7fc0c402db00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd1837048d2e1e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:59.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.221+0000 7fc0d1600640 1 -- 172.21.15.86:0/555606301 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c402f8c0 con 0x7fc0d4144e90 2024-04-20T21:30:59.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.221+0000 7fc0d1600640 1 -- 172.21.15.86:0/555606301 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0c402fa60 con 0x7fc0d4144e90 2024-04-20T21:30:59.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.221+0000 7fc0d1600640 1 -- 172.21.15.86:0/555606301 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c402fe30 con 0x7fc0d4144e90 2024-04-20T21:30:59.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.222+0000 7fc0d9600640 1 -- 172.21.15.86:0/555606301 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0d4144e90 msgr2=0x7fc0d40b4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:59.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.222+0000 7fc0d9600640 1 --2- 172.21.15.86:0/555606301 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0d4144e90 0x7fc0d40b4930 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4009870 tx=0x7fc0c402db00 comp rx=0 tx=0).stop 2024-04-20T21:30:59.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.222+0000 7fc0d9600640 1 -- 172.21.15.86:0/555606301 shutdown_connections 2024-04-20T21:30:59.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.222+0000 7fc0d9600640 1 --2- 172.21.15.86:0/555606301 >> v2:172.21.15.144:3300/0 conn(0x7fc0d40cd170 0x7fc0d406ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:59.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.222+0000 7fc0d9600640 1 --2- 172.21.15.86:0/555606301 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0d4144e90 0x7fc0d40b4930 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:59.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.222+0000 7fc0d9600640 1 -- 172.21.15.86:0/555606301 >> 172.21.15.86:0/555606301 conn(0x7fc0d40b2360 msgr2=0x7fc0d40b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:59.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.222+0000 7fc0d9600640 1 -- 172.21.15.86:0/555606301 shutdown_connections 2024-04-20T21:30:59.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.222+0000 7fc0d9600640 1 -- 172.21.15.86:0/555606301 wait complete. 2024-04-20T21:30:59.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.224+0000 7fc0d9600640 1 Processor -- start 2024-04-20T21:30:59.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.224+0000 7fc0d9600640 1 -- start start 2024-04-20T21:30:59.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.225+0000 7fc0d9600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0d40cd170 0x7fc0d40ab880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:59.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.225+0000 7fc0d9600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0d40cd5d0 0x7fc0d40adea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:59.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.225+0000 7fc0d9600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0d4144e90 0x7fc0d41fc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:59.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.225+0000 7fc0d9600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d40d0010 con 0x7fc0d40cd170 2024-04-20T21:30:59.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.225+0000 7fc0d9600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d40b4e70 con 0x7fc0d40cd5d0 2024-04-20T21:30:59.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.225+0000 7fc0d9600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fc0d40d0190 con 0x7fc0d4144e90 2024-04-20T21:30:59.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.226+0000 7fc0d2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0d40cd170 0x7fc0d40ab880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:59.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.226+0000 7fc0d2a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0d40cd170 0x7fc0d40ab880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46620/0 (socket says 172.21.15.86:46620) 2024-04-20T21:30:59.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.226+0000 7fc0d2a00640 1 -- 172.21.15.86:0/3750828838 learned_addr learned my addr 172.21.15.86:0/3750828838 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:59.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.226+0000 7fc0d2a00640 1 -- 172.21.15.86:0/3750828838 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0d4144e90 msgr2=0x7fc0d41fc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:59.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.226+0000 7fc0d3400640 1 --2- 172.21.15.86:0/3750828838 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0d4144e90 0x7fc0d41fc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:59.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.226+0000 7fc0d2a00640 1 --2- 172.21.15.86:0/3750828838 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0d4144e90 0x7fc0d41fc3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:59.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.226+0000 7fc0d2a00640 1 -- 172.21.15.86:0/3750828838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0d40cd5d0 msgr2=0x7fc0d40adea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:59.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.226+0000 7fc0d2000640 1 --2- 172.21.15.86:0/3750828838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0d40cd5d0 0x7fc0d40adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:59.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.226+0000 7fc0d2a00640 1 --2- 172.21.15.86:0/3750828838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0d40cd5d0 0x7fc0d40adea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:59.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.226+0000 7fc0d2a00640 1 -- 172.21.15.86:0/3750828838 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0d40ac270 con 0x7fc0d40cd170 2024-04-20T21:30:59.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.226+0000 7fc0d2a00640 1 --2- 172.21.15.86:0/3750828838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0d40cd170 0x7fc0d40ab880 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7fc0c402db40 tx=0x7fc0c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:59.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.227+0000 7fc0caa00640 1 -- 172.21.15.86:0/3750828838 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c4046020 con 0x7fc0d40cd170 2024-04-20T21:30:59.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.227+0000 7fc0caa00640 1 -- 172.21.15.86:0/3750828838 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0c40428a0 con 0x7fc0d40cd170 2024-04-20T21:30:59.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.227+0000 7fc0caa00640 1 -- 172.21.15.86:0/3750828838 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c4042c90 con 0x7fc0d40cd170 2024-04-20T21:30:59.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.227+0000 7fc0d9600640 1 -- 172.21.15.86:0/3750828838 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0d40ac550 con 0x7fc0d40cd170 2024-04-20T21:30:59.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.227+0000 7fc0d9600640 1 -- 172.21.15.86:0/3750828838 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0d40acb10 con 0x7fc0d40cd170 2024-04-20T21:30:59.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.228+0000 7fc0d9600640 1 -- 172.21.15.86:0/3750828838 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0d40acef0 con 0x7fc0d40cd170 2024-04-20T21:30:59.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.229+0000 7fc0caa00640 1 -- 172.21.15.86:0/3750828838 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc0c4049020 con 0x7fc0d40cd170 2024-04-20T21:30:59.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.230+0000 7fc0caa00640 1 --2- 172.21.15.86:0/3750828838 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc0b406c8f0 0x7fc0b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:59.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.230+0000 7fc0caa00640 1 -- 172.21.15.86:0/3750828838 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7fc0c40ae3c0 con 0x7fc0d40cd170 2024-04-20T21:30:59.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.230+0000 7fc0d2000640 1 --2- 172.21.15.86:0/3750828838 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc0b406c8f0 0x7fc0b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:59.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.231+0000 7fc0d2000640 1 --2- 172.21.15.86:0/3750828838 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc0b406c8f0 0x7fc0b406edb0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fc0d406f390 tx=0x7fc0c0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:59.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.236+0000 7fc0caa00640 1 -- 172.21.15.86:0/3750828838 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc0d40acef0 con 0x7fc0d40cd170 2024-04-20T21:30:59.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.456+0000 7fc0d9600640 1 -- 172.21.15.86:0/3750828838 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0d40689e0 con 0x7fc0d40cd170 2024-04-20T21:30:59.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.457+0000 7fc0caa00640 1 -- 172.21.15.86:0/3750828838 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5212 (secure 0 0 0) 0x7fc0c4050030 con 0x7fc0d40cd170 2024-04-20T21:30:59.456 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:59.456 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":112,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6253,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3627230007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3627230007},{"type":"v1","addr":"172.21.15.144:6833","nonce":3627230007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":6256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2498297133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2498297133},{"type":"v1","addr":"172.21.15.144:6835","nonce":2498297133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":9228,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/1120740475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1120740475},{"type":"v1","addr":"172.21.15.86:6836","nonce":1120740475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:29:43.769581+0000","modified":"2024-04-20T21:30:47.137068+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9792},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9792":{"gid":9792,"name":"a","rank":0,"incarnation":109,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6837/2019839930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2019839930},{"type":"v1","addr":"172.21.15.86:6837","nonce":2019839930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9792,"qdb_cluster":[9792]},"id":8}]} 2024-04-20T21:30:59.457 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 112 2024-04-20T21:30:59.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.460+0000 7fc0d9600640 1 -- 172.21.15.86:0/3750828838 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc0b406c8f0 msgr2=0x7fc0b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:59.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.460+0000 7fc0d9600640 1 --2- 172.21.15.86:0/3750828838 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc0b406c8f0 0x7fc0b406edb0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fc0d406f390 tx=0x7fc0c0008040 comp rx=0 tx=0).stop 2024-04-20T21:30:59.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.460+0000 7fc0d9600640 1 -- 172.21.15.86:0/3750828838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0d40cd170 msgr2=0x7fc0d40ab880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:59.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.460+0000 7fc0d9600640 1 --2- 172.21.15.86:0/3750828838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0d40cd170 0x7fc0d40ab880 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7fc0c402db40 tx=0x7fc0c40046c0 comp rx=0 tx=0).stop 2024-04-20T21:30:59.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.460+0000 7fc0d9600640 1 -- 172.21.15.86:0/3750828838 shutdown_connections 2024-04-20T21:30:59.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.460+0000 7fc0d9600640 1 --2- 172.21.15.86:0/3750828838 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fc0b406c8f0 0x7fc0b406edb0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:59.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.460+0000 7fc0d9600640 1 --2- 172.21.15.86:0/3750828838 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fc0d4144e90 0x7fc0d41fc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:59.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.460+0000 7fc0d9600640 1 --2- 172.21.15.86:0/3750828838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0d40cd5d0 0x7fc0d40adea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:59.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.460+0000 7fc0d9600640 1 --2- 172.21.15.86:0/3750828838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc0d40cd170 0x7fc0d40ab880 unknown :-1 s=CLOSED pgs=2107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:59.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.460+0000 7fc0d9600640 1 -- 172.21.15.86:0/3750828838 >> 172.21.15.86:0/3750828838 conn(0x7fc0d40b2360 msgr2=0x7fc0d40b0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:59.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.460+0000 7fc0d9600640 1 -- 172.21.15.86:0/3750828838 shutdown_connections 2024-04-20T21:30:59.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.460+0000 7fc0d9600640 1 -- 172.21.15.86:0/3750828838 wait complete. 2024-04-20T21:30:59.476 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:30:59.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.670+0000 7fe28e800640 1 Processor -- start 2024-04-20T21:30:59.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.670+0000 7fe28e800640 1 -- start start 2024-04-20T21:30:59.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.671+0000 7fe28e800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe28806f2a0 0x7fe288167410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:59.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.671+0000 7fe28e800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe288160890 con 0x7fe28815d6a0 2024-04-20T21:30:59.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.671+0000 7fe28e800640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe288160b70 con 0x7fe288156ae0 2024-04-20T21:30:59.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.671+0000 7fe28e800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe28806db20 con 0x7fe28806f2a0 2024-04-20T21:30:59.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.672+0000 7fe28ca00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe28815d6a0 0x7fe2880ad0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57282/0 (socket says 172.21.15.86:57282) 2024-04-20T21:30:59.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.672+0000 7fe28ca00640 1 -- 172.21.15.86:0/1679225511 learned_addr learned my addr 172.21.15.86:0/1679225511 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:30:59.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.672+0000 7fe28de00640 1 --2- 172.21.15.86:0/1679225511 >> v2:172.21.15.86:3300/0 conn(0x7fe28806f2a0 0x7fe288167410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:59.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.672+0000 7fe287e00640 1 -- 172.21.15.86:0/1679225511 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2026768441 0 0) 0x7fe288160890 con 0x7fe28815d6a0 2024-04-20T21:30:59.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.672+0000 7fe287e00640 1 -- 172.21.15.86:0/1679225511 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe270003520 con 0x7fe28815d6a0 2024-04-20T21:30:59.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.673+0000 7fe28de00640 1 -- 172.21.15.86:0/1679225511 >> v1:172.21.15.144:6789/0 conn(0x7fe288156ae0 legacy=0x7fe288160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:59.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.673+0000 7fe28de00640 1 -- 172.21.15.86:0/1679225511 >> v1:172.21.15.86:6789/0 conn(0x7fe28815d6a0 legacy=0x7fe2880ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:59.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.673+0000 7fe28de00640 1 -- 172.21.15.86:0/1679225511 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe288160e50 con 0x7fe28806f2a0 2024-04-20T21:30:59.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.674+0000 7fe28de00640 1 --2- 172.21.15.86:0/1679225511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe28806f2a0 0x7fe288167410 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7fe280009d90 tx=0x7fe28002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a40181562da11a47 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:59.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.674+0000 7fe287e00640 1 -- 172.21.15.86:0/1679225511 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe280039070 con 0x7fe28806f2a0 2024-04-20T21:30:59.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.674+0000 7fe287e00640 1 -- 172.21.15.86:0/1679225511 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe28002cd00 con 0x7fe28806f2a0 2024-04-20T21:30:59.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.674+0000 7fe287e00640 1 -- 172.21.15.86:0/1679225511 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe280034640 con 0x7fe28806f2a0 2024-04-20T21:30:59.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.674+0000 7fe28e800640 1 -- 172.21.15.86:0/1679225511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe28806f2a0 msgr2=0x7fe288167410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:59.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.674+0000 7fe28e800640 1 --2- 172.21.15.86:0/1679225511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe28806f2a0 0x7fe288167410 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7fe280009d90 tx=0x7fe28002bf50 comp rx=0 tx=0).stop 2024-04-20T21:30:59.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.675+0000 7fe28e800640 1 -- 172.21.15.86:0/1679225511 shutdown_connections 2024-04-20T21:30:59.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.675+0000 7fe28e800640 1 --2- 172.21.15.86:0/1679225511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe28806f2a0 0x7fe288167410 unknown :-1 s=CLOSED pgs=2109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:59.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.675+0000 7fe28e800640 1 -- 172.21.15.86:0/1679225511 >> 172.21.15.86:0/1679225511 conn(0x7fe2880d0170 msgr2=0x7fe2880d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:59.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.675+0000 7fe28e800640 1 -- 172.21.15.86:0/1679225511 shutdown_connections 2024-04-20T21:30:59.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.675+0000 7fe28e800640 1 -- 172.21.15.86:0/1679225511 wait complete. 2024-04-20T21:30:59.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.676+0000 7fe28e800640 1 Processor -- start 2024-04-20T21:30:59.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.676+0000 7fe28e800640 1 -- start start 2024-04-20T21:30:59.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.677+0000 7fe28e800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe28806f2a0 0x7fe2881f2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:59.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.677+0000 7fe28e800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe288156ae0 0x7fe28815b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:59.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.677+0000 7fe28e800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe28815d6a0 0x7fe28815ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:59.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.677+0000 7fe28e800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe2880ad7f0 con 0x7fe28815d6a0 2024-04-20T21:30:59.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.677+0000 7fe28e800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe288156ec0 con 0x7fe28806f2a0 2024-04-20T21:30:59.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.677+0000 7fe28e800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe2880ad970 con 0x7fe288156ae0 2024-04-20T21:30:59.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.678+0000 7fe28de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe28815d6a0 0x7fe28815ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:59.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.678+0000 7fe28de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe28815d6a0 0x7fe28815ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46638/0 (socket says 172.21.15.86:46638) 2024-04-20T21:30:59.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.678+0000 7fe28de00640 1 -- 172.21.15.86:0/3744443508 learned_addr learned my addr 172.21.15.86:0/3744443508 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:30:59.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.678+0000 7fe28ca00640 1 --2- 172.21.15.86:0/3744443508 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe288156ae0 0x7fe28815b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:59.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.678+0000 7fe28de00640 1 -- 172.21.15.86:0/3744443508 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe288156ae0 msgr2=0x7fe28815b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:59.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.678+0000 7fe28d400640 1 --2- 172.21.15.86:0/3744443508 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe28806f2a0 0x7fe2881f2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:59.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.678+0000 7fe28de00640 1 --2- 172.21.15.86:0/3744443508 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe288156ae0 0x7fe28815b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:59.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.678+0000 7fe28de00640 1 -- 172.21.15.86:0/3744443508 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe28806f2a0 msgr2=0x7fe2881f2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:59.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.678+0000 7fe28de00640 1 --2- 172.21.15.86:0/3744443508 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe28806f2a0 0x7fe2881f2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:59.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.678+0000 7fe28de00640 1 -- 172.21.15.86:0/3744443508 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe288158c90 con 0x7fe28815d6a0 2024-04-20T21:30:59.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.678+0000 7fe28de00640 1 --2- 172.21.15.86:0/3744443508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe28815d6a0 0x7fe28815ba70 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7fe280009d90 tx=0x7fe280004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:59.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.679+0000 7fe285600640 1 -- 172.21.15.86:0/3744443508 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe280044020 con 0x7fe28815d6a0 2024-04-20T21:30:59.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.679+0000 7fe285600640 1 -- 172.21.15.86:0/3744443508 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe28003a040 con 0x7fe28815d6a0 2024-04-20T21:30:59.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.679+0000 7fe285600640 1 -- 172.21.15.86:0/3744443508 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe280039050 con 0x7fe28815d6a0 2024-04-20T21:30:59.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.679+0000 7fe28e800640 1 -- 172.21.15.86:0/3744443508 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe288158f70 con 0x7fe28815d6a0 2024-04-20T21:30:59.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.679+0000 7fe28e800640 1 -- 172.21.15.86:0/3744443508 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe288159530 con 0x7fe28815d6a0 2024-04-20T21:30:59.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.680+0000 7fe28e800640 1 -- 172.21.15.86:0/3744443508 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2880aeb80 con 0x7fe28815d6a0 2024-04-20T21:30:59.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.681+0000 7fe285600640 1 -- 172.21.15.86:0/3744443508 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe280003bd0 con 0x7fe28815d6a0 2024-04-20T21:30:59.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.682+0000 7fe285600640 1 --2- 172.21.15.86:0/3744443508 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe26806c8f0 0x7fe26806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:30:59.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.682+0000 7fe285600640 1 -- 172.21.15.86:0/3744443508 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7fe2800ae2c0 con 0x7fe28815d6a0 2024-04-20T21:30:59.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.683+0000 7fe28d400640 1 --2- 172.21.15.86:0/3744443508 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe26806c8f0 0x7fe26806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:30:59.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.683+0000 7fe28d400640 1 --2- 172.21.15.86:0/3744443508 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe26806c8f0 0x7fe26806edb0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fe27c0015a0 tx=0x7fe27c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:30:59.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.687+0000 7fe285600640 1 -- 172.21.15.86:0/3744443508 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe28004e060 con 0x7fe28815d6a0 2024-04-20T21:30:59.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.931+0000 7fe28e800640 1 -- 172.21.15.86:0/3744443508 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe28804fcf0 con 0x7fe28815d6a0 2024-04-20T21:30:59.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.933+0000 7fe285600640 1 -- 172.21.15.86:0/3744443508 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5212 (secure 0 0 0) 0x7fe28007c5f0 con 0x7fe28815d6a0 2024-04-20T21:30:59.932 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:30:59.932 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":112,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6253,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3627230007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3627230007},{"type":"v1","addr":"172.21.15.144:6833","nonce":3627230007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":6256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2498297133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2498297133},{"type":"v1","addr":"172.21.15.144:6835","nonce":2498297133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":9228,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/1120740475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1120740475},{"type":"v1","addr":"172.21.15.86:6836","nonce":1120740475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T21:29:43.769581+0000","modified":"2024-04-20T21:30:47.137068+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9792},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9792":{"gid":9792,"name":"a","rank":0,"incarnation":109,"state":"up:active","state_seq":4,"addr":"172.21.15.86:6837/2019839930","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":2019839930},{"type":"v1","addr":"172.21.15.86:6837","nonce":2019839930}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9792,"qdb_cluster":[9792]},"id":8}]} 2024-04-20T21:30:59.932 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 112 2024-04-20T21:30:59.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.935+0000 7fe28e800640 1 -- 172.21.15.86:0/3744443508 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe26806c8f0 msgr2=0x7fe26806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:59.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.935+0000 7fe28e800640 1 --2- 172.21.15.86:0/3744443508 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe26806c8f0 0x7fe26806edb0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fe27c0015a0 tx=0x7fe27c009040 comp rx=0 tx=0).stop 2024-04-20T21:30:59.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.935+0000 7fe28e800640 1 -- 172.21.15.86:0/3744443508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe28815d6a0 msgr2=0x7fe28815ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:30:59.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.935+0000 7fe28e800640 1 --2- 172.21.15.86:0/3744443508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe28815d6a0 0x7fe28815ba70 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7fe280009d90 tx=0x7fe280004a40 comp rx=0 tx=0).stop 2024-04-20T21:30:59.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.936+0000 7fe28e800640 1 -- 172.21.15.86:0/3744443508 shutdown_connections 2024-04-20T21:30:59.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.936+0000 7fe28e800640 1 --2- 172.21.15.86:0/3744443508 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe26806c8f0 0x7fe26806edb0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:59.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.936+0000 7fe28e800640 1 --2- 172.21.15.86:0/3744443508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe28815d6a0 0x7fe28815ba70 unknown :-1 s=CLOSED pgs=2110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:59.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.936+0000 7fe28e800640 1 --2- 172.21.15.86:0/3744443508 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe288156ae0 0x7fe28815b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:59.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.936+0000 7fe28e800640 1 --2- 172.21.15.86:0/3744443508 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe28806f2a0 0x7fe2881f2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:30:59.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.936+0000 7fe28e800640 1 -- 172.21.15.86:0/3744443508 >> 172.21.15.86:0/3744443508 conn(0x7fe2880d0170 msgr2=0x7fe28806e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:30:59.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.936+0000 7fe28e800640 1 -- 172.21.15.86:0/3744443508 shutdown_connections 2024-04-20T21:30:59.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:30:59.936+0000 7fe28e800640 1 -- 172.21.15.86:0/3744443508 wait complete. 2024-04-20T21:30:59.952 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:31:00.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.142+0000 7fab86a00640 1 Processor -- start 2024-04-20T21:31:00.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.142+0000 7fab86a00640 1 -- start start 2024-04-20T21:31:00.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.144+0000 7fab86a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fab800cdc60 0x7fab800b05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:00.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.144+0000 7fab86a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab8015f450 con 0x7fab8006b980 2024-04-20T21:31:00.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.144+0000 7fab86a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab8015f730 con 0x7fab8015e800 2024-04-20T21:31:00.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.144+0000 7fab86a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fab800d0f10 con 0x7fab800cdc60 2024-04-20T21:31:00.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.144+0000 7fab84c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fab800cdc60 0x7fab800b05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:00.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.144+0000 7fab86000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fab8006b980 0x7fab800d1380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57290/0 (socket says 172.21.15.86:57290) 2024-04-20T21:31:00.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.144+0000 7fab86000640 1 -- 172.21.15.86:0/2920890116 learned_addr learned my addr 172.21.15.86:0/2920890116 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:31:00.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.145+0000 7fab84c00640 1 -- 172.21.15.86:0/2920890116 >> v1:172.21.15.144:6789/0 conn(0x7fab8015e800 legacy=0x7fab8015ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:00.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.145+0000 7fab84c00640 1 -- 172.21.15.86:0/2920890116 >> v1:172.21.15.86:6789/0 conn(0x7fab8006b980 legacy=0x7fab800d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:00.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.145+0000 7fab84c00640 1 -- 172.21.15.86:0/2920890116 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab8015fa10 con 0x7fab800cdc60 2024-04-20T21:31:00.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.146+0000 7fab84c00640 1 --2- 172.21.15.86:0/2920890116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab800cdc60 0x7fab800b05e0 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7fab6c009dc0 tx=0x7fab6c02be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=634501190183ca4c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:00.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.146+0000 7fab7fe00640 1 -- 172.21.15.86:0/2920890116 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c039070 con 0x7fab800cdc60 2024-04-20T21:31:00.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.146+0000 7fab7fe00640 1 -- 172.21.15.86:0/2920890116 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab6c02cc40 con 0x7fab800cdc60 2024-04-20T21:31:00.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.146+0000 7fab7fe00640 1 -- 172.21.15.86:0/2920890116 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c034440 con 0x7fab800cdc60 2024-04-20T21:31:00.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.147+0000 7fab86a00640 1 -- 172.21.15.86:0/2920890116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab800cdc60 msgr2=0x7fab800b05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:00.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.147+0000 7fab86a00640 1 --2- 172.21.15.86:0/2920890116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab800cdc60 0x7fab800b05e0 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7fab6c009dc0 tx=0x7fab6c02be90 comp rx=0 tx=0).stop 2024-04-20T21:31:00.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.147+0000 7fab86a00640 1 -- 172.21.15.86:0/2920890116 shutdown_connections 2024-04-20T21:31:00.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.147+0000 7fab86a00640 1 --2- 172.21.15.86:0/2920890116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab800cdc60 0x7fab800b05e0 unknown :-1 s=CLOSED pgs=2112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:00.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.147+0000 7fab86a00640 1 -- 172.21.15.86:0/2920890116 >> 172.21.15.86:0/2920890116 conn(0x7fab800b67a0 msgr2=0x7fab800b6bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:00.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.147+0000 7fab86a00640 1 -- 172.21.15.86:0/2920890116 shutdown_connections 2024-04-20T21:31:00.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.147+0000 7fab86a00640 1 -- 172.21.15.86:0/2920890116 wait complete. 2024-04-20T21:31:00.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.148+0000 7fab86a00640 1 Processor -- start 2024-04-20T21:31:00.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.149+0000 7fab86a00640 1 -- start start 2024-04-20T21:31:00.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.150+0000 7fab86a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab8006b980 0x7fab800ae260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:00.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.150+0000 7fab86a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab800cdc60 0x7fab801469f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:00.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.150+0000 7fab86a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab8015e800 0x7fab800afaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:00.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.150+0000 7fab86a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fab800b0ca0 con 0x7fab800cdc60 2024-04-20T21:31:00.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.150+0000 7fab86a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fab800b0b20 con 0x7fab8006b980 2024-04-20T21:31:00.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.150+0000 7fab86a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fab8006bdd0 con 0x7fab8015e800 2024-04-20T21:31:00.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.150+0000 7fab84c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab800cdc60 0x7fab801469f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:00.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.150+0000 7fab84c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab800cdc60 0x7fab801469f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46658/0 (socket says 172.21.15.86:46658) 2024-04-20T21:31:00.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.150+0000 7fab84c00640 1 -- 172.21.15.86:0/3570913224 learned_addr learned my addr 172.21.15.86:0/3570913224 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:31:00.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.150+0000 7fab85600640 1 --2- 172.21.15.86:0/3570913224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab8006b980 0x7fab800ae260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:00.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.150+0000 7fab86000640 1 --2- 172.21.15.86:0/3570913224 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab8015e800 0x7fab800afaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:00.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.150+0000 7fab84c00640 1 -- 172.21.15.86:0/3570913224 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab8015e800 msgr2=0x7fab800afaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:00.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.150+0000 7fab84c00640 1 --2- 172.21.15.86:0/3570913224 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab8015e800 0x7fab800afaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:00.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.150+0000 7fab84c00640 1 -- 172.21.15.86:0/3570913224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab8006b980 msgr2=0x7fab800ae260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:00.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.150+0000 7fab84c00640 1 --2- 172.21.15.86:0/3570913224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab8006b980 0x7fab800ae260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:00.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.150+0000 7fab84c00640 1 -- 172.21.15.86:0/3570913224 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab800aea00 con 0x7fab800cdc60 2024-04-20T21:31:00.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.150+0000 7fab85600640 1 --2- 172.21.15.86:0/3570913224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab8006b980 0x7fab800ae260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:31:00.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.151+0000 7fab84c00640 1 --2- 172.21.15.86:0/3570913224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab800cdc60 0x7fab801469f0 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7fab6c037040 tx=0x7fab6c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:00.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.151+0000 7fab7d600640 1 -- 172.21.15.86:0/3570913224 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c039050 con 0x7fab800cdc60 2024-04-20T21:31:00.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.151+0000 7fab7d600640 1 -- 172.21.15.86:0/3570913224 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab6c045070 con 0x7fab800cdc60 2024-04-20T21:31:00.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.151+0000 7fab86a00640 1 -- 172.21.15.86:0/3570913224 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab800aece0 con 0x7fab800cdc60 2024-04-20T21:31:00.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.151+0000 7fab7d600640 1 -- 172.21.15.86:0/3570913224 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c03e850 con 0x7fab800cdc60 2024-04-20T21:31:00.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.151+0000 7fab86a00640 1 -- 172.21.15.86:0/3570913224 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab800af2a0 con 0x7fab800cdc60 2024-04-20T21:31:00.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.152+0000 7fab86a00640 1 -- 172.21.15.86:0/3570913224 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab8004fcf0 con 0x7fab800cdc60 2024-04-20T21:31:00.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.153+0000 7fab7d600640 1 -- 172.21.15.86:0/3570913224 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fab6c03e9f0 con 0x7fab800cdc60 2024-04-20T21:31:00.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.154+0000 7fab7d600640 1 --2- 172.21.15.86:0/3570913224 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fab6006c8f0 0x7fab6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:00.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.154+0000 7fab7d600640 1 -- 172.21.15.86:0/3570913224 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7fab6c0ae920 con 0x7fab800cdc60 2024-04-20T21:31:00.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.154+0000 7fab85600640 1 --2- 172.21.15.86:0/3570913224 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fab6006c8f0 0x7fab6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:00.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.155+0000 7fab85600640 1 --2- 172.21.15.86:0/3570913224 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fab6006c8f0 0x7fab6006edb0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fab78003eb0 tx=0x7fab78009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:00.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.158+0000 7fab7d600640 1 -- 172.21.15.86:0/3570913224 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fab6c07cbc0 con 0x7fab800cdc60 2024-04-20T21:31:00.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.364+0000 7fab86a00640 1 -- 172.21.15.86:0/3570913224 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fab8015e490 con 0x7fab800cdc60 2024-04-20T21:31:00.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.367+0000 7fab7d600640 1 -- 172.21.15.86:0/3570913224 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) ==== 74+0+17037 (secure 0 0 0) 0x7fab6c07c4c0 con 0x7fab800cdc60 2024-04-20T21:31:00.366 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:31:00.366 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":116,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:30:42.086516+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-20T21:29:41.532824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-20T21:29:42.749687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.f","mappings":[{"from":0,"to":5}]},{"pgid":"17.18","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6837/1453621918":"2024-04-21T21:30:42.086474+0000","172.21.15.86:6835/1453621918":"2024-04-21T21:30:42.086474+0000","172.21.15.86:6837/289590924":"2024-04-21T21:30:19.784403+0000","172.21.15.86:6835/289590924":"2024-04-21T21:30:19.784403+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:31:00.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.369+0000 7fab86a00640 1 -- 172.21.15.86:0/3570913224 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fab6006c8f0 msgr2=0x7fab6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:00.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.369+0000 7fab86a00640 1 --2- 172.21.15.86:0/3570913224 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fab6006c8f0 0x7fab6006edb0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fab78003eb0 tx=0x7fab78009040 comp rx=0 tx=0).stop 2024-04-20T21:31:00.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.369+0000 7fab86a00640 1 -- 172.21.15.86:0/3570913224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab800cdc60 msgr2=0x7fab801469f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:00.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.369+0000 7fab86a00640 1 --2- 172.21.15.86:0/3570913224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab800cdc60 0x7fab801469f0 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7fab6c037040 tx=0x7fab6c004a40 comp rx=0 tx=0).stop 2024-04-20T21:31:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.370+0000 7fab86a00640 1 -- 172.21.15.86:0/3570913224 shutdown_connections 2024-04-20T21:31:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.370+0000 7fab86a00640 1 --2- 172.21.15.86:0/3570913224 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fab6006c8f0 0x7fab6006edb0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.370+0000 7fab86a00640 1 --2- 172.21.15.86:0/3570913224 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fab8015e800 0x7fab800afaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.370+0000 7fab86a00640 1 --2- 172.21.15.86:0/3570913224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab800cdc60 0x7fab801469f0 unknown :-1 s=CLOSED pgs=2113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.370+0000 7fab86a00640 1 --2- 172.21.15.86:0/3570913224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab8006b980 0x7fab800ae260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.370+0000 7fab86a00640 1 -- 172.21.15.86:0/3570913224 >> 172.21.15.86:0/3570913224 conn(0x7fab800b67a0 msgr2=0x7fab8015ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.370+0000 7fab86a00640 1 -- 172.21.15.86:0/3570913224 shutdown_connections 2024-04-20T21:31:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.370+0000 7fab86a00640 1 -- 172.21.15.86:0/3570913224 wait complete. 2024-04-20T21:31:00.389 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-20T21:31:00.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.576+0000 7fe831400640 1 Processor -- start 2024-04-20T21:31:00.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.576+0000 7fe831400640 1 -- start start 2024-04-20T21:31:00.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.578+0000 7fe831400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe82c15e690 0x7fe82c06ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:00.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.578+0000 7fe831400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe82c0d2cb0 0x7fe82c0d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:00.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.578+0000 7fe831400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe82c0d33b0 con 0x7fe82c0d0ee0 2024-04-20T21:31:00.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.578+0000 7fe831400640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fe82c0d1880 con 0x7fe82c0d2cb0 2024-04-20T21:31:00.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.578+0000 7fe831400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe82c0d3690 con 0x7fe82c15e690 2024-04-20T21:31:00.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.578+0000 7fe830a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe82c0d2cb0 0x7fe82c0d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:00.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.578+0000 7fe830a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe82c0d2cb0 0x7fe82c0d1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46672/0 (socket says 172.21.15.86:46672) 2024-04-20T21:31:00.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.578+0000 7fe830a00640 1 -- 172.21.15.86:0/3752762034 learned_addr learned my addr 172.21.15.86:0/3752762034 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:31:00.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.579+0000 7fe830a00640 1 -- 172.21.15.86:0/3752762034 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe82c15e690 msgr2=0x7fe82c06ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:00.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.579+0000 7fe82be00640 1 --2- 172.21.15.86:0/3752762034 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe82c15e690 0x7fe82c06ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:00.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.579+0000 7fe830a00640 1 --2- 172.21.15.86:0/3752762034 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe82c15e690 0x7fe82c06ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:00.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.579+0000 7fe830a00640 1 -- 172.21.15.86:0/3752762034 >> v1:172.21.15.144:6789/0 conn(0x7fe82c0d0ee0 legacy=0x7fe82c0d25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:00.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.579+0000 7fe830a00640 1 -- 172.21.15.86:0/3752762034 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe82c0d3970 con 0x7fe82c0d2cb0 2024-04-20T21:31:00.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.580+0000 7fe830a00640 1 --2- 172.21.15.86:0/3752762034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe82c0d2cb0 0x7fe82c0d1340 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7fe824009870 tx=0x7fe82402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=979e60d35f569eb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.580+0000 7fe82aa00640 1 -- 172.21.15.86:0/3752762034 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe824035020 con 0x7fe82c0d2cb0 2024-04-20T21:31:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.580+0000 7fe82aa00640 1 -- 172.21.15.86:0/3752762034 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe824031640 con 0x7fe82c0d2cb0 2024-04-20T21:31:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.580+0000 7fe82aa00640 1 -- 172.21.15.86:0/3752762034 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82402d040 con 0x7fe82c0d2cb0 2024-04-20T21:31:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.581+0000 7fe831400640 1 -- 172.21.15.86:0/3752762034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe82c0d2cb0 msgr2=0x7fe82c0d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.581+0000 7fe831400640 1 --2- 172.21.15.86:0/3752762034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe82c0d2cb0 0x7fe82c0d1340 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7fe824009870 tx=0x7fe82402b8f0 comp rx=0 tx=0).stop 2024-04-20T21:31:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.581+0000 7fe831400640 1 -- 172.21.15.86:0/3752762034 shutdown_connections 2024-04-20T21:31:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.581+0000 7fe831400640 1 --2- 172.21.15.86:0/3752762034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe82c0d2cb0 0x7fe82c0d1340 unknown :-1 s=CLOSED pgs=2114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.581+0000 7fe831400640 1 --2- 172.21.15.86:0/3752762034 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe82c15e690 0x7fe82c06ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.581+0000 7fe831400640 1 -- 172.21.15.86:0/3752762034 >> 172.21.15.86:0/3752762034 conn(0x7fe82c09d8a0 msgr2=0x7fe82c08bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.581+0000 7fe831400640 1 -- 172.21.15.86:0/3752762034 shutdown_connections 2024-04-20T21:31:00.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.581+0000 7fe831400640 1 -- 172.21.15.86:0/3752762034 wait complete. 2024-04-20T21:31:00.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.583+0000 7fe831400640 1 Processor -- start 2024-04-20T21:31:00.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.583+0000 7fe831400640 1 -- start start 2024-04-20T21:31:00.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.584+0000 7fe831400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe82c0d0ee0 0x7fe82c0ca890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:00.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.584+0000 7fe831400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe82c0d2cb0 0x7fe82c0cde00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:00.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.584+0000 7fe831400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe82c15e690 0x7fe82c153b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:00.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.584+0000 7fe831400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe82c06d360 con 0x7fe82c0d2cb0 2024-04-20T21:31:00.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.584+0000 7fe831400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe82c0cef00 con 0x7fe82c15e690 2024-04-20T21:31:00.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.584+0000 7fe831400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe82c0d30d0 con 0x7fe82c0d0ee0 2024-04-20T21:31:00.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.584+0000 7fe82b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe82c0d2cb0 0x7fe82c0cde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:00.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.584+0000 7fe82b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe82c0d2cb0 0x7fe82c0cde00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46686/0 (socket says 172.21.15.86:46686) 2024-04-20T21:31:00.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.584+0000 7fe82b400640 1 -- 172.21.15.86:0/3844172689 learned_addr learned my addr 172.21.15.86:0/3844172689 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:31:00.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.584+0000 7fe82be00640 1 --2- 172.21.15.86:0/3844172689 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe82c0d0ee0 0x7fe82c0ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:00.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.584+0000 7fe830a00640 1 --2- 172.21.15.86:0/3844172689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe82c15e690 0x7fe82c153b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:00.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.584+0000 7fe82b400640 1 -- 172.21.15.86:0/3844172689 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe82c0d0ee0 msgr2=0x7fe82c0ca890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:00.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.584+0000 7fe82b400640 1 --2- 172.21.15.86:0/3844172689 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe82c0d0ee0 0x7fe82c0ca890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:00.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.584+0000 7fe82b400640 1 -- 172.21.15.86:0/3844172689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe82c15e690 msgr2=0x7fe82c153b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:00.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.584+0000 7fe82b400640 1 --2- 172.21.15.86:0/3844172689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe82c15e690 0x7fe82c153b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:00.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.584+0000 7fe82b400640 1 -- 172.21.15.86:0/3844172689 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe82c0b41a0 con 0x7fe82c0d2cb0 2024-04-20T21:31:00.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.585+0000 7fe82b400640 1 --2- 172.21.15.86:0/3844172689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe82c0d2cb0 0x7fe82c0cde00 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7fe81400c7a0 tx=0x7fe81400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:00.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.585+0000 7fe823e00640 1 -- 172.21.15.86:0/3844172689 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe814016020 con 0x7fe82c0d2cb0 2024-04-20T21:31:00.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.585+0000 7fe823e00640 1 -- 172.21.15.86:0/3844172689 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe81400aa50 con 0x7fe82c0d2cb0 2024-04-20T21:31:00.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.585+0000 7fe831400640 1 -- 172.21.15.86:0/3844172689 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe82c0b4480 con 0x7fe82c0d2cb0 2024-04-20T21:31:00.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.585+0000 7fe823e00640 1 -- 172.21.15.86:0/3844172689 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe814012420 con 0x7fe82c0d2cb0 2024-04-20T21:31:00.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.585+0000 7fe831400640 1 -- 172.21.15.86:0/3844172689 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe82c0b4a40 con 0x7fe82c0d2cb0 2024-04-20T21:31:00.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.586+0000 7fe831400640 1 -- 172.21.15.86:0/3844172689 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe82c04fcf0 con 0x7fe82c0d2cb0 2024-04-20T21:31:00.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.587+0000 7fe823e00640 1 -- 172.21.15.86:0/3844172689 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe814019020 con 0x7fe82c0d2cb0 2024-04-20T21:31:00.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.588+0000 7fe823e00640 1 --2- 172.21.15.86:0/3844172689 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe80806c8f0 0x7fe80806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:00.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.588+0000 7fe823e00640 1 -- 172.21.15.86:0/3844172689 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5939+0+0 (secure 0 0 0) 0x7fe81408c0b0 con 0x7fe82c0d2cb0 2024-04-20T21:31:00.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.588+0000 7fe82be00640 1 --2- 172.21.15.86:0/3844172689 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe80806c8f0 0x7fe80806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:00.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.589+0000 7fe82be00640 1 --2- 172.21.15.86:0/3844172689 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe80806c8f0 0x7fe80806edb0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fe82c15b680 tx=0x7fe81c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:00.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.593+0000 7fe823e00640 1 -- 172.21.15.86:0/3844172689 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe81405a350 con 0x7fe82c0d2cb0 2024-04-20T21:31:00.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.814+0000 7fe831400640 1 -- 172.21.15.86:0/3844172689 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe82c06cc00 con 0x7fe82c0d2cb0 2024-04-20T21:31:00.814 INFO:tasks.ceph.mon.a.smithi086.stderr:2024-04-20T21:31:00.815+0000 7f5bb9c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T21:31:00.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.818+0000 7fe823e00640 1 -- 172.21.15.86:0/3844172689 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v113) ==== 161+0+0 (secure 0 0 0) 0x7fe814059c50 con 0x7fe82c0d2cb0 2024-04-20T21:31:00.817 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-20T21:31:00.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.821+0000 7fe831400640 1 -- 172.21.15.86:0/3844172689 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe80806c8f0 msgr2=0x7fe80806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:00.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.821+0000 7fe831400640 1 --2- 172.21.15.86:0/3844172689 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe80806c8f0 0x7fe80806edb0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fe82c15b680 tx=0x7fe81c008040 comp rx=0 tx=0).stop 2024-04-20T21:31:00.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.821+0000 7fe831400640 1 -- 172.21.15.86:0/3844172689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe82c0d2cb0 msgr2=0x7fe82c0cde00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:00.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.821+0000 7fe831400640 1 --2- 172.21.15.86:0/3844172689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe82c0d2cb0 0x7fe82c0cde00 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7fe81400c7a0 tx=0x7fe81400cc60 comp rx=0 tx=0).stop 2024-04-20T21:31:00.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.822+0000 7fe831400640 1 -- 172.21.15.86:0/3844172689 shutdown_connections 2024-04-20T21:31:00.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.822+0000 7fe831400640 1 --2- 172.21.15.86:0/3844172689 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe80806c8f0 0x7fe80806edb0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:00.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.822+0000 7fe831400640 1 --2- 172.21.15.86:0/3844172689 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe82c15e690 0x7fe82c153b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:00.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.822+0000 7fe831400640 1 --2- 172.21.15.86:0/3844172689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe82c0d2cb0 0x7fe82c0cde00 unknown :-1 s=CLOSED pgs=2115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:00.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.822+0000 7fe831400640 1 --2- 172.21.15.86:0/3844172689 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe82c0d0ee0 0x7fe82c0ca890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:00.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.822+0000 7fe831400640 1 -- 172.21.15.86:0/3844172689 >> 172.21.15.86:0/3844172689 conn(0x7fe82c09d8a0 msgr2=0x7fe82c06e010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:00.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.822+0000 7fe831400640 1 -- 172.21.15.86:0/3844172689 shutdown_connections 2024-04-20T21:31:00.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:00.822+0000 7fe831400640 1 -- 172.21.15.86:0/3844172689 wait complete. 2024-04-20T21:31:00.828 INFO:tasks.ceph.mds.a.smithi086.stderr: -1524> 2024-04-20T21:30:48.101+0000 7fd3b3000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T21:31:00.845 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-20T21:31:00.882 INFO:tasks.ceph.mds.a.smithi086.stdout:starting mds.a at 2024-04-20T21:31:01.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.022+0000 7fad24c00640 1 Processor -- start 2024-04-20T21:31:01.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.022+0000 7fad24c00640 1 -- start start 2024-04-20T21:31:01.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.024+0000 7fad24c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fad2015ecc0 0x7fad200d4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:01.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.024+0000 7fad24c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fad200d4ba0 0x7fad200cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:01.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.024+0000 7fad24c00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad200d1f60 con 0x7fad2015e8e0 2024-04-20T21:31:01.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.024+0000 7fad24c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fad200d4ff0 con 0x7fad200d4ba0 2024-04-20T21:31:01.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.024+0000 7fad24c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fad200cf7f0 con 0x7fad2015ecc0 2024-04-20T21:31:01.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.024+0000 7fad1fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fad200d4ba0 0x7fad200cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:01.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.024+0000 7fad1fe00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fad200d4ba0 0x7fad200cfc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46708/0 (socket says 172.21.15.86:46708) 2024-04-20T21:31:01.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.024+0000 7fad1fe00640 1 -- 172.21.15.86:0/397164443 learned_addr learned my addr 172.21.15.86:0/397164443 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:31:01.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.025+0000 7fad1ea00640 1 --2- 172.21.15.86:0/397164443 >> v2:172.21.15.144:3300/0 conn(0x7fad2015ecc0 0x7fad200d4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:01.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.025+0000 7fad1fe00640 1 -- 172.21.15.86:0/397164443 >> v2:172.21.15.144:3300/0 conn(0x7fad2015ecc0 msgr2=0x7fad200d4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:01.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.025+0000 7fad1fe00640 1 --2- 172.21.15.86:0/397164443 >> v2:172.21.15.144:3300/0 conn(0x7fad2015ecc0 0x7fad200d4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:01.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.025+0000 7fad1fe00640 1 -- 172.21.15.86:0/397164443 >> v1:172.21.15.144:6789/0 conn(0x7fad2015e8e0 legacy=0x7fad2006d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:01.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.025+0000 7fad1fe00640 1 -- 172.21.15.86:0/397164443 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad200d19a0 con 0x7fad200d4ba0 2024-04-20T21:31:01.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.025+0000 7fad1ea00640 1 --2- 172.21.15.86:0/397164443 >> v2:172.21.15.144:3300/0 conn(0x7fad2015ecc0 0x7fad200d4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:31:01.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.026+0000 7fad1fe00640 1 --2- 172.21.15.86:0/397164443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad200d4ba0 0x7fad200cfc80 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7fad2008c790 tx=0x7fad1802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e88e76291fa8e388 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:01.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.026+0000 7fad1e000640 1 -- 172.21.15.86:0/397164443 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad18035020 con 0x7fad200d4ba0 2024-04-20T21:31:01.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.026+0000 7fad1e000640 1 -- 172.21.15.86:0/397164443 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad18031870 con 0x7fad200d4ba0 2024-04-20T21:31:01.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.026+0000 7fad1e000640 1 -- 172.21.15.86:0/397164443 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad18031c40 con 0x7fad200d4ba0 2024-04-20T21:31:01.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.027+0000 7fad24c00640 1 -- 172.21.15.86:0/397164443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad200d4ba0 msgr2=0x7fad200cfc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:01.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.027+0000 7fad24c00640 1 --2- 172.21.15.86:0/397164443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad200d4ba0 0x7fad200cfc80 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7fad2008c790 tx=0x7fad1802b910 comp rx=0 tx=0).stop 2024-04-20T21:31:01.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.027+0000 7fad24c00640 1 -- 172.21.15.86:0/397164443 shutdown_connections 2024-04-20T21:31:01.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.027+0000 7fad24c00640 1 --2- 172.21.15.86:0/397164443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad200d4ba0 0x7fad200cfc80 unknown :-1 s=CLOSED pgs=2119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:01.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.027+0000 7fad24c00640 1 --2- 172.21.15.86:0/397164443 >> v2:172.21.15.144:3300/0 conn(0x7fad2015ecc0 0x7fad200d4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:01.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.027+0000 7fad24c00640 1 -- 172.21.15.86:0/397164443 >> 172.21.15.86:0/397164443 conn(0x7fad2009d9d0 msgr2=0x7fad2008bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:01.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.027+0000 7fad24c00640 1 -- 172.21.15.86:0/397164443 shutdown_connections 2024-04-20T21:31:01.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.027+0000 7fad24c00640 1 -- 172.21.15.86:0/397164443 wait complete. 2024-04-20T21:31:01.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.029+0000 7fad24c00640 1 Processor -- start 2024-04-20T21:31:01.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.029+0000 7fad24c00640 1 -- start start 2024-04-20T21:31:01.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.030+0000 7fad24c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad200d4ba0 0x7fad20160d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:01.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.030+0000 7fad24c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad2015e8e0 0x7fad200b9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:01.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.030+0000 7fad24c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad200d0550 0x7fad200d09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:01.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.030+0000 7fad24c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fad2006d910 con 0x7fad200d0550 2024-04-20T21:31:01.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.030+0000 7fad24c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fad200cf480 con 0x7fad200d4ba0 2024-04-20T21:31:01.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.030+0000 7fad24c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fad200d01c0 con 0x7fad2015e8e0 2024-04-20T21:31:01.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.030+0000 7fad1fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad200d0550 0x7fad200d09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:01.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.030+0000 7fad1fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad200d0550 0x7fad200d09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46710/0 (socket says 172.21.15.86:46710) 2024-04-20T21:31:01.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.030+0000 7fad1fe00640 1 -- 172.21.15.86:0/2446841363 learned_addr learned my addr 172.21.15.86:0/2446841363 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:31:01.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.030+0000 7fad1fe00640 1 -- 172.21.15.86:0/2446841363 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad2015e8e0 msgr2=0x7fad200b9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:01.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.030+0000 7fad1fe00640 1 --2- 172.21.15.86:0/2446841363 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad2015e8e0 0x7fad200b9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:01.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.030+0000 7fad1fe00640 1 -- 172.21.15.86:0/2446841363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad200d4ba0 msgr2=0x7fad20160d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:01.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.030+0000 7fad1fe00640 1 --2- 172.21.15.86:0/2446841363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad200d4ba0 0x7fad20160d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:01.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.030+0000 7fad1fe00640 1 -- 172.21.15.86:0/2446841363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad20160280 con 0x7fad200d0550 2024-04-20T21:31:01.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.031+0000 7fad1fe00640 1 --2- 172.21.15.86:0/2446841363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad200d0550 0x7fad200d09e0 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7fad18009930 tx=0x7fad180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:01.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.031+0000 7fad17400640 1 -- 172.21.15.86:0/2446841363 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad18040070 con 0x7fad200d0550 2024-04-20T21:31:01.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.031+0000 7fad17400640 1 -- 172.21.15.86:0/2446841363 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad18030d00 con 0x7fad200d0550 2024-04-20T21:31:01.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.031+0000 7fad24c00640 1 -- 172.21.15.86:0/2446841363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad200ba040 con 0x7fad200d0550 2024-04-20T21:31:01.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.031+0000 7fad17400640 1 -- 172.21.15.86:0/2446841363 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad18035040 con 0x7fad200d0550 2024-04-20T21:31:01.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.031+0000 7fad24c00640 1 -- 172.21.15.86:0/2446841363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad20160540 con 0x7fad200d0550 2024-04-20T21:31:01.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.032+0000 7fad24c00640 1 -- 172.21.15.86:0/2446841363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad2004fd90 con 0x7fad200d0550 2024-04-20T21:31:01.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.033+0000 7fad17400640 1 -- 172.21.15.86:0/2446841363 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fad18038020 con 0x7fad200d0550 2024-04-20T21:31:01.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.034+0000 7fad17400640 1 --2- 172.21.15.86:0/2446841363 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7facfc06c920 0x7facfc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:01.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.034+0000 7fad17400640 1 -- 172.21.15.86:0/2446841363 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 6025+0+0 (secure 0 0 0) 0x7fad180adc80 con 0x7fad200d0550 2024-04-20T21:31:01.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.034+0000 7fad1f400640 1 --2- 172.21.15.86:0/2446841363 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7facfc06c920 0x7facfc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:01.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.035+0000 7fad1f400640 1 --2- 172.21.15.86:0/2446841363 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7facfc06c920 0x7facfc06ede0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fad2015c5c0 tx=0x7fad10007d10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:01.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.039+0000 7fad17400640 1 -- 172.21.15.86:0/2446841363 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fad1807bed0 con 0x7fad200d0550 2024-04-20T21:31:01.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.271+0000 7fad24c00640 1 -- 172.21.15.86:0/2446841363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fad2006cd10 con 0x7fad200d0550 2024-04-20T21:31:01.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.831+0000 7fad17400640 1 -- 172.21.15.86:0/2446841363 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v114) ==== 104+0+0 (secure 0 0 0) 0x7fad1807b7d0 con 0x7fad200d0550 2024-04-20T21:31:01.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.835+0000 7fad24c00640 1 -- 172.21.15.86:0/2446841363 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7facfc06c920 msgr2=0x7facfc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:01.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.835+0000 7fad24c00640 1 --2- 172.21.15.86:0/2446841363 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7facfc06c920 0x7facfc06ede0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fad2015c5c0 tx=0x7fad10007d10 comp rx=0 tx=0).stop 2024-04-20T21:31:01.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.835+0000 7fad24c00640 1 -- 172.21.15.86:0/2446841363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad200d0550 msgr2=0x7fad200d09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:01.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.835+0000 7fad24c00640 1 --2- 172.21.15.86:0/2446841363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad200d0550 0x7fad200d09e0 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7fad18009930 tx=0x7fad180046c0 comp rx=0 tx=0).stop 2024-04-20T21:31:01.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.835+0000 7fad24c00640 1 -- 172.21.15.86:0/2446841363 shutdown_connections 2024-04-20T21:31:01.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.835+0000 7fad24c00640 1 --2- 172.21.15.86:0/2446841363 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7facfc06c920 0x7facfc06ede0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:01.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.835+0000 7fad24c00640 1 --2- 172.21.15.86:0/2446841363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad200d0550 0x7fad200d09e0 unknown :-1 s=CLOSED pgs=2120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:01.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.835+0000 7fad24c00640 1 --2- 172.21.15.86:0/2446841363 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fad2015e8e0 0x7fad200b9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:01.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.835+0000 7fad24c00640 1 --2- 172.21.15.86:0/2446841363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad200d4ba0 0x7fad20160d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:01.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.835+0000 7fad24c00640 1 -- 172.21.15.86:0/2446841363 >> 172.21.15.86:0/2446841363 conn(0x7fad2009d9d0 msgr2=0x7fad2008bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:01.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.836+0000 7fad24c00640 1 -- 172.21.15.86:0/2446841363 shutdown_connections 2024-04-20T21:31:01.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:01.836+0000 7fad24c00640 1 -- 172.21.15.86:0/2446841363 wait complete. 2024-04-20T21:31:01.856 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-20T21:31:02.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.106+0000 7fa800a00640 1 Processor -- start 2024-04-20T21:31:02.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.106+0000 7fa800a00640 1 -- start start 2024-04-20T21:31:02.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.107+0000 7fa800a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fa7fc158ca0 0x7fa7fc0ae910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:02.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.107+0000 7fa800a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa7fc0d0d00 0x7fa7fc072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:02.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.107+0000 7fa800a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7fc15bc10 con 0x7fa7fc159080 2024-04-20T21:31:02.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.107+0000 7fa800a00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fa7fc071c50 con 0x7fa7fc0d0d00 2024-04-20T21:31:02.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.107+0000 7fa800a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fa7fc0af3b0 con 0x7fa7fc158ca0 2024-04-20T21:31:02.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.108+0000 7fa7fb200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa7fc0d0d00 0x7fa7fc072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:02.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.108+0000 7fa7f3e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fa7fc159080 0x7fa7fc0d05f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57312/0 (socket says 172.21.15.86:57312) 2024-04-20T21:31:02.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.108+0000 7fa7f3e00640 1 -- 172.21.15.86:0/1465805652 learned_addr learned my addr 172.21.15.86:0/1465805652 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:31:02.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.108+0000 7fa7fa800640 1 --2- 172.21.15.86:0/1465805652 >> v2:172.21.15.144:3300/0 conn(0x7fa7fc158ca0 0x7fa7fc0ae910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:02.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.108+0000 7fa7fb200640 1 -- 172.21.15.86:0/1465805652 >> v2:172.21.15.144:3300/0 conn(0x7fa7fc158ca0 msgr2=0x7fa7fc0ae910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:02.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.108+0000 7fa7fb200640 1 --2- 172.21.15.86:0/1465805652 >> v2:172.21.15.144:3300/0 conn(0x7fa7fc158ca0 0x7fa7fc0ae910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:02.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.108+0000 7fa7fb200640 1 -- 172.21.15.86:0/1465805652 >> v1:172.21.15.86:6789/0 conn(0x7fa7fc159080 legacy=0x7fa7fc0d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:02.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.108+0000 7fa7fb200640 1 -- 172.21.15.86:0/1465805652 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7fc0af690 con 0x7fa7fc0d0d00 2024-04-20T21:31:02.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.109+0000 7fa7fb200640 1 --2- 172.21.15.86:0/1465805652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7fc0d0d00 0x7fa7fc072150 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc0b2dd0 tx=0x7fa7f402bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ce32429952a516d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:02.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.110+0000 7fa7f9e00640 1 -- 172.21.15.86:0/1465805652 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f4039070 con 0x7fa7fc0d0d00 2024-04-20T21:31:02.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.110+0000 7fa7f9e00640 1 -- 172.21.15.86:0/1465805652 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7f402cc70 con 0x7fa7fc0d0d00 2024-04-20T21:31:02.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.110+0000 7fa7f9e00640 1 -- 172.21.15.86:0/1465805652 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f4034660 con 0x7fa7fc0d0d00 2024-04-20T21:31:02.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.110+0000 7fa800a00640 1 -- 172.21.15.86:0/1465805652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7fc0d0d00 msgr2=0x7fa7fc072150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:02.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.110+0000 7fa800a00640 1 --2- 172.21.15.86:0/1465805652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7fc0d0d00 0x7fa7fc072150 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc0b2dd0 tx=0x7fa7f402bec0 comp rx=0 tx=0).stop 2024-04-20T21:31:02.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.111+0000 7fa800a00640 1 -- 172.21.15.86:0/1465805652 shutdown_connections 2024-04-20T21:31:02.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.111+0000 7fa800a00640 1 --2- 172.21.15.86:0/1465805652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7fc0d0d00 0x7fa7fc072150 unknown :-1 s=CLOSED pgs=2122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:02.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.111+0000 7fa800a00640 1 --2- 172.21.15.86:0/1465805652 >> v2:172.21.15.144:3300/0 conn(0x7fa7fc158ca0 0x7fa7fc0ae910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:02.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.111+0000 7fa800a00640 1 -- 172.21.15.86:0/1465805652 >> 172.21.15.86:0/1465805652 conn(0x7fa7fc0b2220 msgr2=0x7fa7fc0b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:02.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.111+0000 7fa800a00640 1 -- 172.21.15.86:0/1465805652 shutdown_connections 2024-04-20T21:31:02.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.111+0000 7fa800a00640 1 -- 172.21.15.86:0/1465805652 wait complete. 2024-04-20T21:31:02.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.112+0000 7fa800a00640 1 Processor -- start 2024-04-20T21:31:02.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.112+0000 7fa800a00640 1 -- start start 2024-04-20T21:31:02.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.113+0000 7fa800a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7fc0d0d00 0x7fa7fc09dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:02.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.113+0000 7fa800a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7fc158ca0 0x7fa7fc156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:02.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.113+0000 7fa800a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7fc159080 0x7fa7fc09ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:02.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.113+0000 7fa800a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc0aefd0 con 0x7fa7fc159080 2024-04-20T21:31:02.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.113+0000 7fa800a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc0aee50 con 0x7fa7fc0d0d00 2024-04-20T21:31:02.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.113+0000 7fa800a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fa7fc0af150 con 0x7fa7fc158ca0 2024-04-20T21:31:02.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.114+0000 7fa7fb200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7fc159080 0x7fa7fc09ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:02.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.114+0000 7fa7fb200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7fc159080 0x7fa7fc09ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46716/0 (socket says 172.21.15.86:46716) 2024-04-20T21:31:02.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.114+0000 7fa7fb200640 1 -- 172.21.15.86:0/3441514506 learned_addr learned my addr 172.21.15.86:0/3441514506 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:31:02.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.114+0000 7fa7fa800640 1 --2- 172.21.15.86:0/3441514506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7fc0d0d00 0x7fa7fc09dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:02.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.114+0000 7fa7fb200640 1 -- 172.21.15.86:0/3441514506 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7fc158ca0 msgr2=0x7fa7fc156950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:02.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.114+0000 7fa7fb200640 1 --2- 172.21.15.86:0/3441514506 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7fc158ca0 0x7fa7fc156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:02.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.114+0000 7fa7fb200640 1 -- 172.21.15.86:0/3441514506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7fc0d0d00 msgr2=0x7fa7fc09dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:02.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.114+0000 7fa7fb200640 1 --2- 172.21.15.86:0/3441514506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7fc0d0d00 0x7fa7fc09dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:02.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.114+0000 7fa7fb200640 1 -- 172.21.15.86:0/3441514506 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7fc154c80 con 0x7fa7fc159080 2024-04-20T21:31:02.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.114+0000 7fa7fb200640 1 --2- 172.21.15.86:0/3441514506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7fc159080 0x7fa7fc09ba20 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4009e80 tx=0x7fa7f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:02.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.115+0000 7fa7f2a00640 1 -- 172.21.15.86:0/3441514506 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f4039050 con 0x7fa7fc159080 2024-04-20T21:31:02.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.115+0000 7fa7f2a00640 1 -- 172.21.15.86:0/3441514506 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7f4045070 con 0x7fa7fc159080 2024-04-20T21:31:02.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.115+0000 7fa7f2a00640 1 -- 172.21.15.86:0/3441514506 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f403e830 con 0x7fa7fc159080 2024-04-20T21:31:02.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.115+0000 7fa800a00640 1 -- 172.21.15.86:0/3441514506 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7fc154f00 con 0x7fa7fc159080 2024-04-20T21:31:02.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.115+0000 7fa800a00640 1 -- 172.21.15.86:0/3441514506 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7fc1554c0 con 0x7fa7fc159080 2024-04-20T21:31:02.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.116+0000 7fa800a00640 1 -- 172.21.15.86:0/3441514506 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7fc04fcf0 con 0x7fa7fc159080 2024-04-20T21:31:02.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.117+0000 7fa7f2a00640 1 -- 172.21.15.86:0/3441514506 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa7f403e9d0 con 0x7fa7fc159080 2024-04-20T21:31:02.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.118+0000 7fa7f2a00640 1 --2- 172.21.15.86:0/3441514506 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa7d806c8f0 0x7fa7d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:02.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.118+0000 7fa7f2a00640 1 -- 172.21.15.86:0/3441514506 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 6025+0+0 (secure 0 0 0) 0x7fa7f40ae9a0 con 0x7fa7fc159080 2024-04-20T21:31:02.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.118+0000 7fa7fa800640 1 --2- 172.21.15.86:0/3441514506 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa7d806c8f0 0x7fa7d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:02.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.119+0000 7fa7fa800640 1 --2- 172.21.15.86:0/3441514506 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa7d806c8f0 0x7fa7d806edb0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec006fd0 tx=0x7fa7ec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:02.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.122+0000 7fa7f2a00640 1 -- 172.21.15.86:0/3441514506 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa7fc154c80 con 0x7fa7fc159080 2024-04-20T21:31:02.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:02.321+0000 7fa800a00640 1 -- 172.21.15.86:0/3441514506 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa7fc15c830 con 0x7fa7fc159080 2024-04-20T21:31:03.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.229+0000 7fa7f2a00640 1 -- 172.21.15.86:0/3441514506 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v118) ==== 181+0+0 (secure 0 0 0) 0x7fa7f402ce20 con 0x7fa7fc159080 2024-04-20T21:31:03.228 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' removed 2024-04-20T21:31:03.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.233+0000 7fa800a00640 1 -- 172.21.15.86:0/3441514506 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa7d806c8f0 msgr2=0x7fa7d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:03.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.233+0000 7fa800a00640 1 --2- 172.21.15.86:0/3441514506 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa7d806c8f0 0x7fa7d806edb0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec006fd0 tx=0x7fa7ec008040 comp rx=0 tx=0).stop 2024-04-20T21:31:03.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.233+0000 7fa800a00640 1 -- 172.21.15.86:0/3441514506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7fc159080 msgr2=0x7fa7fc09ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:03.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.233+0000 7fa800a00640 1 --2- 172.21.15.86:0/3441514506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7fc159080 0x7fa7fc09ba20 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4009e80 tx=0x7fa7f4004a40 comp rx=0 tx=0).stop 2024-04-20T21:31:03.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.234+0000 7fa800a00640 1 -- 172.21.15.86:0/3441514506 shutdown_connections 2024-04-20T21:31:03.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.234+0000 7fa800a00640 1 --2- 172.21.15.86:0/3441514506 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fa7d806c8f0 0x7fa7d806edb0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:03.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.234+0000 7fa800a00640 1 --2- 172.21.15.86:0/3441514506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7fc159080 0x7fa7fc09ba20 unknown :-1 s=CLOSED pgs=2123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:03.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.234+0000 7fa800a00640 1 --2- 172.21.15.86:0/3441514506 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fa7fc158ca0 0x7fa7fc156950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:03.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.234+0000 7fa800a00640 1 --2- 172.21.15.86:0/3441514506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7fc0d0d00 0x7fa7fc09dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:03.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.234+0000 7fa800a00640 1 -- 172.21.15.86:0/3441514506 >> 172.21.15.86:0/3441514506 conn(0x7fa7fc0b2220 msgr2=0x7fa7fc06da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:03.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.234+0000 7fa800a00640 1 -- 172.21.15.86:0/3441514506 shutdown_connections 2024-04-20T21:31:03.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.235+0000 7fa800a00640 1 -- 172.21.15.86:0/3441514506 wait complete. 2024-04-20T21:31:03.256 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-20T21:31:03.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.460+0000 7f135d000640 1 Processor -- start 2024-04-20T21:31:03.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.460+0000 7f135d000640 1 -- start start 2024-04-20T21:31:03.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.462+0000 7f135d000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13581666f0 0x7f135806e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:03.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.462+0000 7f135d000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1358166b70 con 0x7f135806e730 2024-04-20T21:31:03.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.462+0000 7f135d000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1358072800 con 0x7f13580b49f0 2024-04-20T21:31:03.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.462+0000 7f135d000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f13581676a0 con 0x7f13581666f0 2024-04-20T21:31:03.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.462+0000 7f1357e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f135806e730 0x7f13580b3670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57316/0 (socket says 172.21.15.86:57316) 2024-04-20T21:31:03.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.462+0000 7f1357e00640 1 -- 172.21.15.86:0/1163140165 learned_addr learned my addr 172.21.15.86:0/1163140165 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:31:03.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.463+0000 7f1356a00640 1 --2- 172.21.15.86:0/1163140165 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13581666f0 0x7f135806e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:03.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.463+0000 7f1356000640 1 -- 172.21.15.86:0/1163140165 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1693086132 0 0) 0x7f1358166b70 con 0x7f135806e730 2024-04-20T21:31:03.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.463+0000 7f1356000640 1 -- 172.21.15.86:0/1163140165 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f133c003520 con 0x7f135806e730 2024-04-20T21:31:03.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.463+0000 7f1356000640 1 -- 172.21.15.86:0/1163140165 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1656135504 0 0) 0x7f133c003520 con 0x7f135806e730 2024-04-20T21:31:03.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.463+0000 7f1356000640 1 -- 172.21.15.86:0/1163140165 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13581666f0 msgr2=0x7f135806e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:03.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.463+0000 7f1356000640 1 --2- 172.21.15.86:0/1163140165 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13581666f0 0x7f135806e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:03.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.463+0000 7f1356000640 1 -- 172.21.15.86:0/1163140165 >> v1:172.21.15.144:6789/0 conn(0x7f13580b49f0 legacy=0x7f13580b4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:03.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.463+0000 7f1356000640 1 -- 172.21.15.86:0/1163140165 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1358166e00 con 0x7f135806e730 2024-04-20T21:31:03.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.463+0000 7f1356000640 1 -- 172.21.15.86:0/1163140165 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f13500019f0 con 0x7f135806e730 2024-04-20T21:31:03.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.463+0000 7f1356000640 1 -- 172.21.15.86:0/1163140165 >> v1:172.21.15.86:6789/0 conn(0x7f135806e730 legacy=0x7f13580b3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:03.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.463+0000 7f1356000640 1 --2- 172.21.15.86:0/1163140165 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13581666f0 0x7f133c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:03.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.463+0000 7f1356000640 1 --2- 172.21.15.86:0/1163140165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f133c003f60 0x7f133c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:03.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.463+0000 7f1356000640 1 --2- 172.21.15.86:0/1163140165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f133c004d40 0x7f133c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:03.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.463+0000 7f1356000640 1 -- 172.21.15.86:0/1163140165 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f13581676a0 con 0x7f133c004d40 2024-04-20T21:31:03.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.463+0000 7f1356000640 1 -- 172.21.15.86:0/1163140165 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f133c00bb50 con 0x7f133c003f60 2024-04-20T21:31:03.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.463+0000 7f1356000640 1 -- 172.21.15.86:0/1163140165 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f133c00bd50 con 0x7f13581666f0 2024-04-20T21:31:03.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.464+0000 7f1357e00640 1 --2- 172.21.15.86:0/1163140165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f133c004d40 0x7f133c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:03.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.464+0000 7f1356a00640 1 --2- 172.21.15.86:0/1163140165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f133c003f60 0x7f133c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:03.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.464+0000 7f1357400640 1 --2- 172.21.15.86:0/1163140165 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13581666f0 0x7f133c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:03.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.464+0000 7f1357e00640 1 -- 172.21.15.86:0/1163140165 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13581666f0 msgr2=0x7f133c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:03.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.464+0000 7f1357e00640 1 --2- 172.21.15.86:0/1163140165 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13581666f0 0x7f133c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:03.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.464+0000 7f1357e00640 1 -- 172.21.15.86:0/1163140165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f133c003f60 msgr2=0x7f133c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:03.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.464+0000 7f1357e00640 1 --2- 172.21.15.86:0/1163140165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f133c003f60 0x7f133c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:03.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.464+0000 7f1357e00640 1 -- 172.21.15.86:0/1163140165 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f133c003520 con 0x7f133c004d40 2024-04-20T21:31:03.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.465+0000 7f1357e00640 1 --2- 172.21.15.86:0/1163140165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f133c004d40 0x7f133c0051c0 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7f135000a860 tx=0x7f133c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b9b4082ece61a06 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:03.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.465+0000 7f1356000640 1 -- 172.21.15.86:0/1163140165 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135000a480 con 0x7f133c004d40 2024-04-20T21:31:03.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.466+0000 7f1356000640 1 -- 172.21.15.86:0/1163140165 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1350034630 con 0x7f133c004d40 2024-04-20T21:31:03.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.466+0000 7f1356000640 1 -- 172.21.15.86:0/1163140165 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1350034a40 con 0x7f133c004d40 2024-04-20T21:31:03.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.466+0000 7f135d000640 1 -- 172.21.15.86:0/1163140165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f133c004d40 msgr2=0x7f133c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:03.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.466+0000 7f135d000640 1 --2- 172.21.15.86:0/1163140165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f133c004d40 0x7f133c0051c0 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7f135000a860 tx=0x7f133c005830 comp rx=0 tx=0).stop 2024-04-20T21:31:03.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.466+0000 7f1357e00640 1 -- 172.21.15.86:0/1163140165 reap_dead start 2024-04-20T21:31:03.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.466+0000 7f135d000640 1 -- 172.21.15.86:0/1163140165 shutdown_connections 2024-04-20T21:31:03.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.466+0000 7f135d000640 1 -- 172.21.15.86:0/1163140165 >> 172.21.15.86:0/1163140165 conn(0x7f13580b2180 msgr2=0x7f13580b2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:03.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.466+0000 7f135d000640 1 -- 172.21.15.86:0/1163140165 shutdown_connections 2024-04-20T21:31:03.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.466+0000 7f135d000640 1 -- 172.21.15.86:0/1163140165 wait complete. 2024-04-20T21:31:03.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.468+0000 7f135d000640 1 Processor -- start 2024-04-20T21:31:03.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.468+0000 7f135d000640 1 -- start start 2024-04-20T21:31:03.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.469+0000 7f135d000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f133c004d40 0x7f135809b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:03.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.469+0000 7f135d000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13581656e0 0x7f135809d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:03.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.469+0000 7f135d000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13581640b0 0x7f1358164530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:03.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.469+0000 7f135d000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f135806dcd0 con 0x7f133c004d40 2024-04-20T21:31:03.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.469+0000 7f135d000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f13581673c0 con 0x7f13581656e0 2024-04-20T21:31:03.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.469+0000 7f135d000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f13580b3d80 con 0x7f13581640b0 2024-04-20T21:31:03.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.469+0000 7f1357400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f133c004d40 0x7f135809b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:03.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.469+0000 7f1357400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f133c004d40 0x7f135809b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46736/0 (socket says 172.21.15.86:46736) 2024-04-20T21:31:03.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.469+0000 7f1357400640 1 -- 172.21.15.86:0/713740274 learned_addr learned my addr 172.21.15.86:0/713740274 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:31:03.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.470+0000 7f1356a00640 1 --2- 172.21.15.86:0/713740274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13581656e0 0x7f135809d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:03.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.470+0000 7f1357400640 1 -- 172.21.15.86:0/713740274 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13581640b0 msgr2=0x7f1358164530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:03.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.470+0000 7f1357400640 1 --2- 172.21.15.86:0/713740274 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13581640b0 0x7f1358164530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:03.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.470+0000 7f1357400640 1 -- 172.21.15.86:0/713740274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13581656e0 msgr2=0x7f135809d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:03.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.470+0000 7f1357400640 1 --2- 172.21.15.86:0/713740274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13581656e0 0x7f135809d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:03.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.470+0000 7f1357400640 1 -- 172.21.15.86:0/713740274 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f135809b900 con 0x7f133c004d40 2024-04-20T21:31:03.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.470+0000 7f1356a00640 1 --2- 172.21.15.86:0/713740274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13581656e0 0x7f135809d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:31:03.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.470+0000 7f1357400640 1 --2- 172.21.15.86:0/713740274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f133c004d40 0x7f135809b0d0 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f134800e550 tx=0x7f134800ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:03.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.470+0000 7f134f400640 1 -- 172.21.15.86:0/713740274 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13480052e0 con 0x7f133c004d40 2024-04-20T21:31:03.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.470+0000 7f134f400640 1 -- 172.21.15.86:0/713740274 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1348018070 con 0x7f133c004d40 2024-04-20T21:31:03.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.470+0000 7f134f400640 1 -- 172.21.15.86:0/713740274 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1348003810 con 0x7f133c004d40 2024-04-20T21:31:03.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.470+0000 7f135d000640 1 -- 172.21.15.86:0/713740274 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f135809bbe0 con 0x7f133c004d40 2024-04-20T21:31:03.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.471+0000 7f135d000640 1 -- 172.21.15.86:0/713740274 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f135809c1a0 con 0x7f133c004d40 2024-04-20T21:31:03.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.472+0000 7f135d000640 1 -- 172.21.15.86:0/713740274 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f135804fcf0 con 0x7f133c004d40 2024-04-20T21:31:03.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.473+0000 7f134f400640 1 -- 172.21.15.86:0/713740274 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f13480039b0 con 0x7f133c004d40 2024-04-20T21:31:03.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.473+0000 7f134f400640 1 --2- 172.21.15.86:0/713740274 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f133806c920 0x7f133806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:03.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.474+0000 7f134f400640 1 -- 172.21.15.86:0/713740274 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5587+0+0 (secure 0 0 0) 0x7f134808b700 con 0x7f133c004d40 2024-04-20T21:31:03.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.474+0000 7f1356a00640 1 --2- 172.21.15.86:0/713740274 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f133806c920 0x7f133806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:03.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.474+0000 7f1356a00640 1 --2- 172.21.15.86:0/713740274 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f133806c920 0x7f133806ede0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f13580cf130 tx=0x7f1344008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:03.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.478+0000 7f134f400640 1 -- 172.21.15.86:0/713740274 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1348059c10 con 0x7f133c004d40 2024-04-20T21:31:03.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:03.662+0000 7f135d000640 1 -- 172.21.15.86:0/713740274 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f135806f890 con 0x7f133c004d40 2024-04-20T21:31:04.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.232+0000 7f134f400640 1 -- 172.21.15.86:0/713740274 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v119) ==== 169+0+0 (secure 0 0 0) 0x7f1348059510 con 0x7f133c004d40 2024-04-20T21:31:04.232 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' removed 2024-04-20T21:31:04.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.237+0000 7f135d000640 1 -- 172.21.15.86:0/713740274 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f133806c920 msgr2=0x7f133806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:04.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.237+0000 7f135d000640 1 --2- 172.21.15.86:0/713740274 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f133806c920 0x7f133806ede0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f13580cf130 tx=0x7f1344008040 comp rx=0 tx=0).stop 2024-04-20T21:31:04.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.237+0000 7f135d000640 1 -- 172.21.15.86:0/713740274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f133c004d40 msgr2=0x7f135809b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:04.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.237+0000 7f135d000640 1 --2- 172.21.15.86:0/713740274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f133c004d40 0x7f135809b0d0 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f134800e550 tx=0x7f134800ea10 comp rx=0 tx=0).stop 2024-04-20T21:31:04.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.238+0000 7f135d000640 1 -- 172.21.15.86:0/713740274 shutdown_connections 2024-04-20T21:31:04.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.238+0000 7f135d000640 1 --2- 172.21.15.86:0/713740274 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f133806c920 0x7f133806ede0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:04.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.238+0000 7f135d000640 1 --2- 172.21.15.86:0/713740274 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f13581640b0 0x7f1358164530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:04.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.238+0000 7f135d000640 1 --2- 172.21.15.86:0/713740274 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13581656e0 0x7f135809d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:04.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.238+0000 7f135d000640 1 --2- 172.21.15.86:0/713740274 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f133c004d40 0x7f135809b0d0 unknown :-1 s=CLOSED pgs=2126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:04.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.238+0000 7f135d000640 1 -- 172.21.15.86:0/713740274 >> 172.21.15.86:0/713740274 conn(0x7f13580b2180 msgr2=0x7f13580d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:04.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.239+0000 7f135d000640 1 -- 172.21.15.86:0/713740274 shutdown_connections 2024-04-20T21:31:04.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.239+0000 7f135d000640 1 -- 172.21.15.86:0/713740274 wait complete. 2024-04-20T21:31:04.263 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-20T21:31:04.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.454+0000 7f1c06c00640 1 Processor -- start 2024-04-20T21:31:04.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.455+0000 7f1c06c00640 1 -- start start 2024-04-20T21:31:04.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.456+0000 7f1c06c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c00145250 0x7f1c0006efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:04.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.456+0000 7f1c06c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1c00145f00 0x7f1c00071d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:04.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.456+0000 7f1c06c00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1c00072280 0x7f1c0006dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:04.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.456+0000 7f1c06c00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f1c0006f500 con 0x7f1c00072280 2024-04-20T21:31:04.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.456+0000 7f1c06c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f1c00160410 con 0x7f1c00145f00 2024-04-20T21:31:04.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.456+0000 7f1c06c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1c001606f0 con 0x7f1c00145250 2024-04-20T21:31:04.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.457+0000 7f1c06200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1c00072280 0x7f1c0006dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:04.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.457+0000 7f1c06200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1c00072280 0x7f1c0006dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46750/0 (socket says 172.21.15.86:46750) 2024-04-20T21:31:04.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.457+0000 7f1c06200640 1 -- 172.21.15.86:0/1163289672 learned_addr learned my addr 172.21.15.86:0/1163289672 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:31:04.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.457+0000 7f1c05800640 1 --2- 172.21.15.86:0/1163289672 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c00145250 0x7f1c0006efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:04.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.457+0000 7f1c04e00640 1 --2- 172.21.15.86:0/1163289672 >> v2:172.21.15.144:3300/0 conn(0x7f1c00145f00 0x7f1c00071d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:04.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.457+0000 7f1c06200640 1 -- 172.21.15.86:0/1163289672 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c00145250 msgr2=0x7f1c0006efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:04.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.457+0000 7f1c06200640 1 --2- 172.21.15.86:0/1163289672 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c00145250 0x7f1c0006efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:04.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.457+0000 7f1c06200640 1 -- 172.21.15.86:0/1163289672 >> v2:172.21.15.144:3300/0 conn(0x7f1c00145f00 msgr2=0x7f1c00071d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:04.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.457+0000 7f1c06200640 1 --2- 172.21.15.86:0/1163289672 >> v2:172.21.15.144:3300/0 conn(0x7f1c00145f00 0x7f1c00071d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:04.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.457+0000 7f1c06200640 1 -- 172.21.15.86:0/1163289672 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c001609d0 con 0x7f1c00072280 2024-04-20T21:31:04.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.458+0000 7f1c06200640 1 --2- 172.21.15.86:0/1163289672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c00072280 0x7f1c0006dda0 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f1c000b58d0 tx=0x7f1bf802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3d2ed444a6516e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:04.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.459+0000 7f1bffe00640 1 -- 172.21.15.86:0/1163289672 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf802c9f0 con 0x7f1c00072280 2024-04-20T21:31:04.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.459+0000 7f1bffe00640 1 -- 172.21.15.86:0/1163289672 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bf802cb90 con 0x7f1c00072280 2024-04-20T21:31:04.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.459+0000 7f1bffe00640 1 -- 172.21.15.86:0/1163289672 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf80343f0 con 0x7f1c00072280 2024-04-20T21:31:04.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.459+0000 7f1c06c00640 1 -- 172.21.15.86:0/1163289672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c00072280 msgr2=0x7f1c0006dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:04.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.459+0000 7f1c06c00640 1 --2- 172.21.15.86:0/1163289672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c00072280 0x7f1c0006dda0 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f1c000b58d0 tx=0x7f1bf802be50 comp rx=0 tx=0).stop 2024-04-20T21:31:04.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.459+0000 7f1c06c00640 1 -- 172.21.15.86:0/1163289672 shutdown_connections 2024-04-20T21:31:04.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.459+0000 7f1c06c00640 1 --2- 172.21.15.86:0/1163289672 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c00072280 0x7f1c0006dda0 unknown :-1 s=CLOSED pgs=2127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:04.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.459+0000 7f1c06c00640 1 --2- 172.21.15.86:0/1163289672 >> v2:172.21.15.144:3300/0 conn(0x7f1c00145f00 0x7f1c00071d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:04.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.459+0000 7f1c06c00640 1 --2- 172.21.15.86:0/1163289672 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c00145250 0x7f1c0006efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:04.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.459+0000 7f1c06c00640 1 -- 172.21.15.86:0/1163289672 >> 172.21.15.86:0/1163289672 conn(0x7f1c00067cf0 msgr2=0x7f1c0006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:04.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.460+0000 7f1c06c00640 1 -- 172.21.15.86:0/1163289672 shutdown_connections 2024-04-20T21:31:04.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.460+0000 7f1c06c00640 1 -- 172.21.15.86:0/1163289672 wait complete. 2024-04-20T21:31:04.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.461+0000 7f1c06c00640 1 Processor -- start 2024-04-20T21:31:04.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.461+0000 7f1c06c00640 1 -- start start 2024-04-20T21:31:04.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.462+0000 7f1c06c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c00072280 0x7f1c000c7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:04.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.462+0000 7f1c06c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c00145250 0x7f1c000b2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:04.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.462+0000 7f1c06c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c00145f00 0x7f1c001f59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:04.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.462+0000 7f1c06c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1c001457b0 con 0x7f1c00145f00 2024-04-20T21:31:04.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.462+0000 7f1c06c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1c00145630 con 0x7f1c00072280 2024-04-20T21:31:04.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.462+0000 7f1c06c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1c0006e2e0 con 0x7f1c00145250 2024-04-20T21:31:04.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.463+0000 7f1c06200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c00145f00 0x7f1c001f59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:04.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.463+0000 7f1c06200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c00145f00 0x7f1c001f59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46764/0 (socket says 172.21.15.86:46764) 2024-04-20T21:31:04.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.463+0000 7f1c06200640 1 -- 172.21.15.86:0/1850403567 learned_addr learned my addr 172.21.15.86:0/1850403567 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:31:04.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.463+0000 7f1c05800640 1 --2- 172.21.15.86:0/1850403567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c00072280 0x7f1c000c7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:04.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.463+0000 7f1c04e00640 1 --2- 172.21.15.86:0/1850403567 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c00145250 0x7f1c000b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:04.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.463+0000 7f1c06200640 1 -- 172.21.15.86:0/1850403567 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c00145250 msgr2=0x7f1c000b2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:04.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.463+0000 7f1c06200640 1 --2- 172.21.15.86:0/1850403567 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c00145250 0x7f1c000b2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:04.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.463+0000 7f1c06200640 1 -- 172.21.15.86:0/1850403567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c00072280 msgr2=0x7f1c000c7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:04.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.463+0000 7f1c06200640 1 --2- 172.21.15.86:0/1850403567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c00072280 0x7f1c000c7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:04.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.463+0000 7f1c06200640 1 -- 172.21.15.86:0/1850403567 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c000c8490 con 0x7f1c00145f00 2024-04-20T21:31:04.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.463+0000 7f1c06200640 1 --2- 172.21.15.86:0/1850403567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c00145f00 0x7f1c001f59a0 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8036040 tx=0x7f1bf8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:04.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.464+0000 7f1bfd600640 1 -- 172.21.15.86:0/1850403567 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf8043020 con 0x7f1c00145f00 2024-04-20T21:31:04.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.464+0000 7f1bfd600640 1 -- 172.21.15.86:0/1850403567 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bf803e450 con 0x7f1c00145f00 2024-04-20T21:31:04.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.464+0000 7f1bfd600640 1 -- 172.21.15.86:0/1850403567 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf8009510 con 0x7f1c00145f00 2024-04-20T21:31:04.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.464+0000 7f1c06c00640 1 -- 172.21.15.86:0/1850403567 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c000c8770 con 0x7f1c00145f00 2024-04-20T21:31:04.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.464+0000 7f1c06c00640 1 -- 172.21.15.86:0/1850403567 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c000c8d30 con 0x7f1c00145f00 2024-04-20T21:31:04.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.465+0000 7f1c06c00640 1 -- 172.21.15.86:0/1850403567 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c0004fd90 con 0x7f1c00145f00 2024-04-20T21:31:04.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.466+0000 7f1bfd600640 1 -- 172.21.15.86:0/1850403567 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1bf8046020 con 0x7f1c00145f00 2024-04-20T21:31:04.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.467+0000 7f1bfd600640 1 --2- 172.21.15.86:0/1850403567 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1be006c8f0 0x7f1be006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:04.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.468+0000 7f1bfd600640 1 -- 172.21.15.86:0/1850403567 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f1bf80ae2f0 con 0x7f1c00145f00 2024-04-20T21:31:04.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.468+0000 7f1c05800640 1 --2- 172.21.15.86:0/1850403567 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1be006c8f0 0x7f1be006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:04.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.468+0000 7f1c05800640 1 --2- 172.21.15.86:0/1850403567 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1be006c8f0 0x7f1be006edb0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f1bf400a990 tx=0x7f1bf4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:04.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.472+0000 7f1bfd600640 1 -- 172.21.15.86:0/1850403567 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1bf804d030 con 0x7f1c00145f00 2024-04-20T21:31:04.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.676+0000 7f1c06c00640 1 -- 172.21.15.86:0/1850403567 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f1c00142ae0 con 0x7f1c00145f00 2024-04-20T21:31:04.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.681+0000 7f1bfd600640 1 -- 172.21.15.86:0/1850403567 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) ==== 104+0+0 (secure 0 0 0) 0x7f1bf807c8d0 con 0x7f1c00145f00 2024-04-20T21:31:04.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.684+0000 7f1c06c00640 1 -- 172.21.15.86:0/1850403567 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1be006c8f0 msgr2=0x7f1be006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:04.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.684+0000 7f1c06c00640 1 --2- 172.21.15.86:0/1850403567 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1be006c8f0 0x7f1be006edb0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f1bf400a990 tx=0x7f1bf4008040 comp rx=0 tx=0).stop 2024-04-20T21:31:04.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.684+0000 7f1c06c00640 1 -- 172.21.15.86:0/1850403567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c00145f00 msgr2=0x7f1c001f59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:04.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.684+0000 7f1c06c00640 1 --2- 172.21.15.86:0/1850403567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c00145f00 0x7f1c001f59a0 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8036040 tx=0x7f1bf8004a40 comp rx=0 tx=0).stop 2024-04-20T21:31:04.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.684+0000 7f1c06c00640 1 -- 172.21.15.86:0/1850403567 shutdown_connections 2024-04-20T21:31:04.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.684+0000 7f1c06c00640 1 --2- 172.21.15.86:0/1850403567 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1be006c8f0 0x7f1be006edb0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:04.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.684+0000 7f1c06c00640 1 --2- 172.21.15.86:0/1850403567 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1c00145f00 0x7f1c001f59a0 unknown :-1 s=CLOSED pgs=2128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:04.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.684+0000 7f1c06c00640 1 --2- 172.21.15.86:0/1850403567 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1c00145250 0x7f1c000b2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:04.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.684+0000 7f1c06c00640 1 --2- 172.21.15.86:0/1850403567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c00072280 0x7f1c000c7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:04.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.684+0000 7f1c06c00640 1 -- 172.21.15.86:0/1850403567 >> 172.21.15.86:0/1850403567 conn(0x7f1c00067cf0 msgr2=0x7f1c000cd680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:04.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.684+0000 7f1c06c00640 1 -- 172.21.15.86:0/1850403567 shutdown_connections 2024-04-20T21:31:04.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.684+0000 7f1c06c00640 1 -- 172.21.15.86:0/1850403567 wait complete. 2024-04-20T21:31:04.701 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-04-20T21:31:04.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.907+0000 7fbac7a00640 1 Processor -- start 2024-04-20T21:31:04.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.907+0000 7fbac7a00640 1 -- start start 2024-04-20T21:31:04.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.908+0000 7fbac7a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fbac0072410 0x7fbac015cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:04.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.908+0000 7fbac7a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbac006eb30 con 0x7fbac0146130 2024-04-20T21:31:04.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.908+0000 7fbac7a00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbac006dc30 con 0x7fbac006e6d0 2024-04-20T21:31:04.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.908+0000 7fbac7a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fbac00cd8e0 con 0x7fbac0072410 2024-04-20T21:31:04.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.908+0000 7fbac5000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fbac0146130 0x7fbac00ccdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57332/0 (socket says 172.21.15.86:57332) 2024-04-20T21:31:04.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.908+0000 7fbac5000640 1 -- 172.21.15.86:0/3277214241 learned_addr learned my addr 172.21.15.86:0/3277214241 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:31:04.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.909+0000 7fbabf400640 1 -- 172.21.15.86:0/3277214241 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 78472343 0 0) 0x7fbab0001490 con 0x7fbac0146130 2024-04-20T21:31:04.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.909+0000 7fbabf400640 1 -- 172.21.15.86:0/3277214241 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbaa4003520 con 0x7fbac0146130 2024-04-20T21:31:04.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.909+0000 7fbac5a00640 1 --2- 172.21.15.86:0/3277214241 >> v2:172.21.15.144:3300/0 conn(0x7fbac0072410 0x7fbac015cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:04.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.909+0000 7fbabf400640 1 -- 172.21.15.86:0/3277214241 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2474204515 0 0) 0x7fbaa4003520 con 0x7fbac0146130 2024-04-20T21:31:04.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.909+0000 7fbabf400640 1 -- 172.21.15.86:0/3277214241 >> v2:172.21.15.144:3300/0 conn(0x7fbac0072410 msgr2=0x7fbac015cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:04.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.909+0000 7fbabf400640 1 --2- 172.21.15.86:0/3277214241 >> v2:172.21.15.144:3300/0 conn(0x7fbac0072410 0x7fbac015cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:04.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.909+0000 7fbabf400640 1 -- 172.21.15.86:0/3277214241 >> v1:172.21.15.144:6789/0 conn(0x7fbac006e6d0 legacy=0x7fbac0071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:04.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.909+0000 7fbabf400640 1 -- 172.21.15.86:0/3277214241 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbac00cdbc0 con 0x7fbac0146130 2024-04-20T21:31:04.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.909+0000 7fbabf400640 1 -- 172.21.15.86:0/3277214241 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fbab0001bc0 con 0x7fbac0146130 2024-04-20T21:31:04.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.909+0000 7fbabf400640 1 -- 172.21.15.86:0/3277214241 >> v1:172.21.15.86:6789/0 conn(0x7fbac0146130 legacy=0x7fbac00ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:04.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.909+0000 7fbabf400640 1 --2- 172.21.15.86:0/3277214241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbaa40039b0 0x7fbaa4003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:04.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.909+0000 7fbabf400640 1 --2- 172.21.15.86:0/3277214241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbaa4004800 0x7fbaa4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:04.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.910+0000 7fbabf400640 1 --2- 172.21.15.86:0/3277214241 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbaa40051c0 0x7fbaa400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:04.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.910+0000 7fbabf400640 1 -- 172.21.15.86:0/3277214241 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbac00cd8e0 con 0x7fbaa40039b0 2024-04-20T21:31:04.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.910+0000 7fbabf400640 1 -- 172.21.15.86:0/3277214241 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa400c0d0 con 0x7fbaa4004800 2024-04-20T21:31:04.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.910+0000 7fbabf400640 1 -- 172.21.15.86:0/3277214241 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbaa400c2d0 con 0x7fbaa40051c0 2024-04-20T21:31:04.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.910+0000 7fbac5000640 1 --2- 172.21.15.86:0/3277214241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbaa40039b0 0x7fbaa4003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:04.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.910+0000 7fbac5a00640 1 --2- 172.21.15.86:0/3277214241 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbaa40051c0 0x7fbaa400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:04.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.910+0000 7fbac5000640 1 -- 172.21.15.86:0/3277214241 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbaa40051c0 msgr2=0x7fbaa400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:04.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.910+0000 7fbac5000640 1 --2- 172.21.15.86:0/3277214241 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbaa40051c0 0x7fbaa400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:04.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.910+0000 7fbabfe00640 1 --2- 172.21.15.86:0/3277214241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbaa4004800 0x7fbaa4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:04.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.910+0000 7fbac5000640 1 -- 172.21.15.86:0/3277214241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbaa4004800 msgr2=0x7fbaa4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:04.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.910+0000 7fbac5000640 1 --2- 172.21.15.86:0/3277214241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbaa4004800 0x7fbaa4004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:04.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.910+0000 7fbac5000640 1 -- 172.21.15.86:0/3277214241 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaa4003520 con 0x7fbaa40039b0 2024-04-20T21:31:04.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.910+0000 7fbac5a00640 1 -- 172.21.15.86:0/3277214241 reap_dead start 2024-04-20T21:31:04.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.911+0000 7fbac5000640 1 --2- 172.21.15.86:0/3277214241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbaa40039b0 0x7fbaa4003db0 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7fbab000aa50 tx=0x7fbaa4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2801acd5ebee2432 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:04.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.912+0000 7fbabf400640 1 -- 172.21.15.86:0/3277214241 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab0038070 con 0x7fbaa40039b0 2024-04-20T21:31:04.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.912+0000 7fbabf400640 1 -- 172.21.15.86:0/3277214241 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbab0033600 con 0x7fbaa40039b0 2024-04-20T21:31:04.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.912+0000 7fbabf400640 1 -- 172.21.15.86:0/3277214241 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab000a670 con 0x7fbaa40039b0 2024-04-20T21:31:04.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.912+0000 7fbac7a00640 1 -- 172.21.15.86:0/3277214241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbaa40039b0 msgr2=0x7fbaa4003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:04.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.912+0000 7fbac7a00640 1 --2- 172.21.15.86:0/3277214241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbaa40039b0 0x7fbaa4003db0 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7fbab000aa50 tx=0x7fbaa4005830 comp rx=0 tx=0).stop 2024-04-20T21:31:04.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.912+0000 7fbac7a00640 1 -- 172.21.15.86:0/3277214241 shutdown_connections 2024-04-20T21:31:04.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.912+0000 7fbac7a00640 1 --2- 172.21.15.86:0/3277214241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbaa40039b0 0x7fbaa4003db0 unknown :-1 s=CLOSED pgs=2130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:04.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.912+0000 7fbac7a00640 1 -- 172.21.15.86:0/3277214241 >> 172.21.15.86:0/3277214241 conn(0x7fbac00b2360 msgr2=0x7fbac00b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:04.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.912+0000 7fbac7a00640 1 -- 172.21.15.86:0/3277214241 shutdown_connections 2024-04-20T21:31:04.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.912+0000 7fbac7a00640 1 -- 172.21.15.86:0/3277214241 wait complete. 2024-04-20T21:31:04.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.914+0000 7fbac7a00640 1 Processor -- start 2024-04-20T21:31:04.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.914+0000 7fbac7a00640 1 -- start start 2024-04-20T21:31:04.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.915+0000 7fbac7a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbaa40039b0 0x7fbac015cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:04.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.915+0000 7fbac7a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbac015e310 0x7fbac015b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:04.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.915+0000 7fbac7a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbac015bda0 0x7fbac015f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:04.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.915+0000 7fbac7a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbac006f150 con 0x7fbac015e310 2024-04-20T21:31:04.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.915+0000 7fbac7a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbac006e1b0 con 0x7fbaa40039b0 2024-04-20T21:31:04.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.915+0000 7fbac7a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbac0071d00 con 0x7fbac015bda0 2024-04-20T21:31:04.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.915+0000 7fbabfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbac015e310 0x7fbac015b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:04.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.916+0000 7fbabfe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbac015e310 0x7fbac015b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46786/0 (socket says 172.21.15.86:46786) 2024-04-20T21:31:04.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.916+0000 7fbabfe00640 1 -- 172.21.15.86:0/4164667317 learned_addr learned my addr 172.21.15.86:0/4164667317 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:31:04.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.916+0000 7fbac5000640 1 --2- 172.21.15.86:0/4164667317 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbaa40039b0 0x7fbac015cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:04.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.916+0000 7fbac5a00640 1 --2- 172.21.15.86:0/4164667317 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbac015bda0 0x7fbac015f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:04.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.916+0000 7fbabfe00640 1 -- 172.21.15.86:0/4164667317 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbac015bda0 msgr2=0x7fbac015f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:04.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.916+0000 7fbabfe00640 1 --2- 172.21.15.86:0/4164667317 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbac015bda0 0x7fbac015f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:04.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.916+0000 7fbabfe00640 1 -- 172.21.15.86:0/4164667317 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbaa40039b0 msgr2=0x7fbac015cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:04.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.916+0000 7fbabfe00640 1 --2- 172.21.15.86:0/4164667317 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbaa40039b0 0x7fbac015cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:04.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.916+0000 7fbabfe00640 1 -- 172.21.15.86:0/4164667317 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbac015d270 con 0x7fbac015e310 2024-04-20T21:31:04.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.916+0000 7fbabfe00640 1 --2- 172.21.15.86:0/4164667317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbac015e310 0x7fbac015b590 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7fbaac00d430 tx=0x7fbaac00d8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:04.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.917+0000 7fbabcc00640 1 -- 172.21.15.86:0/4164667317 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaac018070 con 0x7fbac015e310 2024-04-20T21:31:04.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.917+0000 7fbabcc00640 1 -- 172.21.15.86:0/4164667317 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaac0035f0 con 0x7fbac015e310 2024-04-20T21:31:04.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.917+0000 7fbabcc00640 1 -- 172.21.15.86:0/4164667317 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaac0039e0 con 0x7fbac015e310 2024-04-20T21:31:04.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.917+0000 7fbac7a00640 1 -- 172.21.15.86:0/4164667317 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbac015d550 con 0x7fbac015e310 2024-04-20T21:31:04.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.917+0000 7fbac7a00640 1 -- 172.21.15.86:0/4164667317 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbac015db10 con 0x7fbac015e310 2024-04-20T21:31:04.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.918+0000 7fbac7a00640 1 -- 172.21.15.86:0/4164667317 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbac00689e0 con 0x7fbac015e310 2024-04-20T21:31:04.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.919+0000 7fbabcc00640 1 -- 172.21.15.86:0/4164667317 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbaac01b070 con 0x7fbac015e310 2024-04-20T21:31:04.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.920+0000 7fbabcc00640 1 --2- 172.21.15.86:0/4164667317 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbaa006c8f0 0x7fbaa006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:04.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.920+0000 7fbabcc00640 1 -- 172.21.15.86:0/4164667317 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7fbaac08b1f0 con 0x7fbac015e310 2024-04-20T21:31:04.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.920+0000 7fbac5000640 1 --2- 172.21.15.86:0/4164667317 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbaa006c8f0 0x7fbaa006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:04.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.921+0000 7fbac5000640 1 --2- 172.21.15.86:0/4164667317 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbaa006c8f0 0x7fbaa006edb0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fbab0035000 tx=0x7fbab0030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:04.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:04.925+0000 7fbabcc00640 1 -- 172.21.15.86:0/4164667317 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbaac059740 con 0x7fbac015e310 2024-04-20T21:31:05.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:05.197+0000 7fbac7a00640 1 -- 172.21.15.86:0/4164667317 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) -- 0x7fbac015ff50 con 0x7fbac015e310 2024-04-20T21:31:05.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:05.249+0000 7fbabcc00640 1 -- 172.21.15.86:0/4164667317 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v631) ==== 144+0+0 (secure 0 0 0) 0x7fbaac059040 con 0x7fbac015e310 2024-04-20T21:31:05.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:05.252+0000 7fbac7a00640 1 -- 172.21.15.86:0/4164667317 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbaa006c8f0 msgr2=0x7fbaa006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:05.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:05.252+0000 7fbac7a00640 1 --2- 172.21.15.86:0/4164667317 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbaa006c8f0 0x7fbaa006edb0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fbab0035000 tx=0x7fbab0030000 comp rx=0 tx=0).stop 2024-04-20T21:31:05.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:05.252+0000 7fbac7a00640 1 -- 172.21.15.86:0/4164667317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbac015e310 msgr2=0x7fbac015b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:05.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:05.252+0000 7fbac7a00640 1 --2- 172.21.15.86:0/4164667317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbac015e310 0x7fbac015b590 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7fbaac00d430 tx=0x7fbaac00d8f0 comp rx=0 tx=0).stop 2024-04-20T21:31:05.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:05.252+0000 7fbac7a00640 1 -- 172.21.15.86:0/4164667317 shutdown_connections 2024-04-20T21:31:05.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:05.252+0000 7fbac7a00640 1 --2- 172.21.15.86:0/4164667317 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbaa006c8f0 0x7fbaa006edb0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:05.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:05.252+0000 7fbac7a00640 1 --2- 172.21.15.86:0/4164667317 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbac015bda0 0x7fbac015f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:05.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:05.252+0000 7fbac7a00640 1 --2- 172.21.15.86:0/4164667317 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbac015e310 0x7fbac015b590 unknown :-1 s=CLOSED pgs=2131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:05.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:05.252+0000 7fbac7a00640 1 --2- 172.21.15.86:0/4164667317 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbaa40039b0 0x7fbac015cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:05.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:05.252+0000 7fbac7a00640 1 -- 172.21.15.86:0/4164667317 >> 172.21.15.86:0/4164667317 conn(0x7fbac00b2360 msgr2=0x7fbac00ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:05.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:05.252+0000 7fbac7a00640 1 -- 172.21.15.86:0/4164667317 shutdown_connections 2024-04-20T21:31:05.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:05.252+0000 7fbac7a00640 1 -- 172.21.15.86:0/4164667317 wait complete. 2024-04-20T21:31:05.268 INFO:tasks.cephfs_test_runner:test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-04-20T21:31:05.268 INFO:tasks.cephfs_test_runner: 2024-04-20T21:31:05.268 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-20T21:31:05.268 INFO:tasks.cephfs_test_runner:Ran 7 tests in 753.802s 2024-04-20T21:31:05.268 INFO:tasks.cephfs_test_runner: 2024-04-20T21:31:05.269 INFO:tasks.cephfs_test_runner:OK 2024-04-20T21:31:05.269 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-20T21:31:05.279 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-20T21:31:05.291 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-20T21:31:05.291 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:31:05.291 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:31:05.342 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052731,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248996k,nr_inodes=812249,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T21:31:05.342 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-20T21:31:05.342 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:31:05.391 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052731,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248996k,nr_inodes=812249,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498000k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T21:31:05.391 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:31:05.391 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:31:05.450 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T21:31:05.450 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-04-20T21:31:05.450 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T21:31:05.500 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T21:31:05.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:31:05.500 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list 2024-04-20T21:31:05.546 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-20T21:31:05.546 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-20T21:31:05.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:31:05.547 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T21:31:05.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:31:05.677 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T21:31:05.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:31:05.754 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link delete ceph-brx 2024-04-20T21:31:05.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:31:05.885 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-20T21:31:05.941 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-20T21:31:05.941 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-20T21:31:05.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:31:05.942 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T21:31:06.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:31:06.031 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T21:31:06.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T21:31:06.099 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link delete ceph-brx 2024-04-20T21:31:06.251 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-20T21:31:06.260 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-20T21:31:06.260 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T21:31:06.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.430+0000 7f93d3c00640 1 Processor -- start 2024-04-20T21:31:06.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.430+0000 7f93d3c00640 1 -- start start 2024-04-20T21:31:06.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.431+0000 7f93d3c00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f93cc144e90 0x7f93cc15ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:06.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.432+0000 7f93d3c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93cc071d00 0x7f93cc072190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:06.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.432+0000 7f93d3c00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93cc06dc30 con 0x7f93cc06f140 2024-04-20T21:31:06.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.432+0000 7f93d3c00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f93cc145270 con 0x7f93cc144e90 2024-04-20T21:31:06.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.432+0000 7f93d3c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f93cc15f720 con 0x7f93cc071d00 2024-04-20T21:31:06.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.432+0000 7f93cbe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f93cc06f140 0x7f93cc0aeb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:53074/0 (socket says 172.21.15.86:53074) 2024-04-20T21:31:06.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.432+0000 7f93cbe00640 1 -- 172.21.15.86:0/3127142301 learned_addr learned my addr 172.21.15.86:0/3127142301 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:31:06.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.432+0000 7f93d1c00640 1 --2- 172.21.15.86:0/3127142301 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93cc071d00 0x7f93cc072190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:06.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.432+0000 7f93d1200640 1 --2- 172.21.15.86:0/3127142301 >> v2:172.21.15.144:3300/0 conn(0x7f93cc144e90 0x7f93cc15ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:06.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.432+0000 7f93cb400640 1 -- 172.21.15.86:0/3127142301 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2457750040 0 0) 0x7f93cc06dc30 con 0x7f93cc06f140 2024-04-20T21:31:06.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.433+0000 7f93cb400640 1 -- 172.21.15.86:0/3127142301 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f93b4003520 con 0x7f93cc06f140 2024-04-20T21:31:06.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.433+0000 7f93cb400640 1 -- 172.21.15.86:0/3127142301 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4129141506 0 0) 0x7f93b4003520 con 0x7f93cc06f140 2024-04-20T21:31:06.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.433+0000 7f93cb400640 1 -- 172.21.15.86:0/3127142301 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93cc071d00 msgr2=0x7f93cc072190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:06.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.433+0000 7f93cb400640 1 --2- 172.21.15.86:0/3127142301 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93cc071d00 0x7f93cc072190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:06.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.433+0000 7f93cb400640 1 -- 172.21.15.86:0/3127142301 >> v2:172.21.15.144:3300/0 conn(0x7f93cc144e90 msgr2=0x7f93cc15ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:06.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.433+0000 7f93cb400640 1 --2- 172.21.15.86:0/3127142301 >> v2:172.21.15.144:3300/0 conn(0x7f93cc144e90 0x7f93cc15ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:06.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.433+0000 7f93cb400640 1 -- 172.21.15.86:0/3127142301 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93cc15fa00 con 0x7f93cc06f140 2024-04-20T21:31:06.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.433+0000 7f93cb400640 1 -- 172.21.15.86:0/3127142301 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f93b8001670 con 0x7f93cc06f140 2024-04-20T21:31:06.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.433+0000 7f93d1200640 1 --2- 172.21.15.86:0/3127142301 >> v2:172.21.15.144:3300/0 conn(0x7f93cc144e90 0x7f93cc15ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:31:06.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.433+0000 7f93cb400640 1 -- 172.21.15.86:0/3127142301 >> v1:172.21.15.86:6789/0 conn(0x7f93cc06f140 legacy=0x7f93cc0aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:06.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.433+0000 7f93cb400640 1 --2- 172.21.15.86:0/3127142301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93b40039b0 0x7f93b4003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:06.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.433+0000 7f93cb400640 1 --2- 172.21.15.86:0/3127142301 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93cc071d00 0x7f93b4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:06.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.433+0000 7f93cb400640 1 --2- 172.21.15.86:0/3127142301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93b4004d40 0x7f93b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:06.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.433+0000 7f93cb400640 1 -- 172.21.15.86:0/3127142301 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f93cc145270 con 0x7f93b4004d40 2024-04-20T21:31:06.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.433+0000 7f93cb400640 1 -- 172.21.15.86:0/3127142301 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f93cc15f720 con 0x7f93b40039b0 2024-04-20T21:31:06.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.433+0000 7f93cb400640 1 -- 172.21.15.86:0/3127142301 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f93b400ba40 con 0x7f93cc071d00 2024-04-20T21:31:06.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.433+0000 7f93d1c00640 1 --2- 172.21.15.86:0/3127142301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93b4004d40 0x7f93b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:06.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.434+0000 7f93d1200640 1 --2- 172.21.15.86:0/3127142301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93b40039b0 0x7f93b4003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:06.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.434+0000 7f93cbe00640 1 --2- 172.21.15.86:0/3127142301 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93cc071d00 0x7f93b4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:06.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.434+0000 7f93d1c00640 1 -- 172.21.15.86:0/3127142301 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93cc071d00 msgr2=0x7f93b4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:06.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.434+0000 7f93d1c00640 1 --2- 172.21.15.86:0/3127142301 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93cc071d00 0x7f93b4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:06.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.434+0000 7f93d1c00640 1 -- 172.21.15.86:0/3127142301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93b40039b0 msgr2=0x7f93b4003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:06.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.434+0000 7f93d1c00640 1 --2- 172.21.15.86:0/3127142301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93b40039b0 0x7f93b4003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:06.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.434+0000 7f93d1c00640 1 -- 172.21.15.86:0/3127142301 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93b4003520 con 0x7f93b4004d40 2024-04-20T21:31:06.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.434+0000 7f93d1200640 1 --2- 172.21.15.86:0/3127142301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93b40039b0 0x7f93b4003df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:31:06.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.435+0000 7f93d1c00640 1 --2- 172.21.15.86:0/3127142301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93b4004d40 0x7f93b40051c0 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f93c4009fc0 tx=0x7f93b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cae17c9e07f9d76 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:06.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.435+0000 7f93cb400640 1 -- 172.21.15.86:0/3127142301 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c4039070 con 0x7f93b4004d40 2024-04-20T21:31:06.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.435+0000 7f93cb400640 1 -- 172.21.15.86:0/3127142301 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93c402cc20 con 0x7f93b4004d40 2024-04-20T21:31:06.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.435+0000 7f93cb400640 1 -- 172.21.15.86:0/3127142301 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c4009c10 con 0x7f93b4004d40 2024-04-20T21:31:06.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.436+0000 7f93d3c00640 1 -- 172.21.15.86:0/3127142301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93b4004d40 msgr2=0x7f93b40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:06.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.436+0000 7f93d3c00640 1 --2- 172.21.15.86:0/3127142301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93b4004d40 0x7f93b40051c0 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f93c4009fc0 tx=0x7f93b4005830 comp rx=0 tx=0).stop 2024-04-20T21:31:06.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.436+0000 7f93d1c00640 1 -- 172.21.15.86:0/3127142301 reap_dead start 2024-04-20T21:31:06.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.436+0000 7f93d3c00640 1 -- 172.21.15.86:0/3127142301 shutdown_connections 2024-04-20T21:31:06.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.436+0000 7f93d3c00640 1 -- 172.21.15.86:0/3127142301 >> 172.21.15.86:0/3127142301 conn(0x7f93cc0b2360 msgr2=0x7f93cc0b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:06.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.436+0000 7f93d3c00640 1 -- 172.21.15.86:0/3127142301 shutdown_connections 2024-04-20T21:31:06.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.436+0000 7f93d3c00640 1 -- 172.21.15.86:0/3127142301 wait complete. 2024-04-20T21:31:06.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.438+0000 7f93d3c00640 1 Processor -- start 2024-04-20T21:31:06.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.438+0000 7f93d3c00640 1 -- start start 2024-04-20T21:31:06.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.439+0000 7f93d3c00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93b4004d40 0x7f93cc148080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:06.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.439+0000 7f93d3c00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93cc15e370 0x7f93cc0b0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:06.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.439+0000 7f93d3c00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93cc145910 0x7f93cc145d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:06.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.439+0000 7f93d3c00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f93cc15f350 con 0x7f93cc15e370 2024-04-20T21:31:06.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.439+0000 7f93d3c00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f93cc06ded0 con 0x7f93cc145910 2024-04-20T21:31:06.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.439+0000 7f93d3c00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f93cc15f4d0 con 0x7f93b4004d40 2024-04-20T21:31:06.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.439+0000 7f93cbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93cc15e370 0x7f93cc0b0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:06.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.439+0000 7f93cbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93cc15e370 0x7f93cc0b0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33460/0 (socket says 172.21.15.86:33460) 2024-04-20T21:31:06.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.439+0000 7f93cbe00640 1 -- 172.21.15.86:0/2831130927 learned_addr learned my addr 172.21.15.86:0/2831130927 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:31:06.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.440+0000 7f93d1c00640 1 --2- 172.21.15.86:0/2831130927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93cc145910 0x7f93cc145d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:06.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.440+0000 7f93cbe00640 1 -- 172.21.15.86:0/2831130927 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93b4004d40 msgr2=0x7f93cc148080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:06.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.440+0000 7f93d1200640 1 --2- 172.21.15.86:0/2831130927 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93b4004d40 0x7f93cc148080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:06.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.440+0000 7f93cbe00640 1 --2- 172.21.15.86:0/2831130927 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93b4004d40 0x7f93cc148080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:06.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.440+0000 7f93cbe00640 1 -- 172.21.15.86:0/2831130927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93cc145910 msgr2=0x7f93cc145d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:06.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.440+0000 7f93cbe00640 1 --2- 172.21.15.86:0/2831130927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93cc145910 0x7f93cc145d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:06.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.440+0000 7f93cbe00640 1 -- 172.21.15.86:0/2831130927 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93cc0b1550 con 0x7f93cc15e370 2024-04-20T21:31:06.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.440+0000 7f93cbe00640 1 --2- 172.21.15.86:0/2831130927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93cc15e370 0x7f93cc0b0d00 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f93b800e7b0 tx=0x7f93b800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:06.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.440+0000 7f93c8c00640 1 -- 172.21.15.86:0/2831130927 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b800c870 con 0x7f93cc15e370 2024-04-20T21:31:06.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.440+0000 7f93c8c00640 1 -- 172.21.15.86:0/2831130927 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93b8018070 con 0x7f93cc15e370 2024-04-20T21:31:06.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.440+0000 7f93d3c00640 1 -- 172.21.15.86:0/2831130927 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93cc0b1830 con 0x7f93cc15e370 2024-04-20T21:31:06.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.440+0000 7f93c8c00640 1 -- 172.21.15.86:0/2831130927 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b800cc50 con 0x7f93cc15e370 2024-04-20T21:31:06.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.441+0000 7f93d3c00640 1 -- 172.21.15.86:0/2831130927 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93cc0b1df0 con 0x7f93cc15e370 2024-04-20T21:31:06.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.442+0000 7f93d3c00640 1 -- 172.21.15.86:0/2831130927 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93cc0689e0 con 0x7f93cc15e370 2024-04-20T21:31:06.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.443+0000 7f93c8c00640 1 -- 172.21.15.86:0/2831130927 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f93b8013420 con 0x7f93cc15e370 2024-04-20T21:31:06.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.443+0000 7f93c8c00640 1 --2- 172.21.15.86:0/2831130927 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f93ac06c920 0x7f93ac06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:06.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.444+0000 7f93c8c00640 1 -- 172.21.15.86:0/2831130927 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f93b808b450 con 0x7f93cc15e370 2024-04-20T21:31:06.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.444+0000 7f93d1200640 1 --2- 172.21.15.86:0/2831130927 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f93ac06c920 0x7f93ac06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:06.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.444+0000 7f93d1200640 1 --2- 172.21.15.86:0/2831130927 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f93ac06c920 0x7f93ac06ede0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f93c000ba20 tx=0x7f93c0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:06.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.448+0000 7f93c8c00640 1 -- 172.21.15.86:0/2831130927 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f93b8059ac0 con 0x7f93cc15e370 2024-04-20T21:31:06.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.643+0000 7f93d3c00640 1 -- 172.21.15.86:0/2831130927 --> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f93cc161be0 con 0x7f93ac06c920 2024-04-20T21:31:06.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.644+0000 7f93c8c00640 1 -- 172.21.15.86:0/2831130927 <== mgr.4107 v2:172.21.15.86:6832/302045240 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13435 (secure 0 0 0) 0x7f93cc161be0 con 0x7f93ac06c920 2024-04-20T21:31:06.643 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:31:06.644 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-20T21:31:06.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.646+0000 7f93d3c00640 1 -- 172.21.15.86:0/2831130927 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f93ac06c920 msgr2=0x7f93ac06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:06.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.646+0000 7f93d3c00640 1 --2- 172.21.15.86:0/2831130927 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f93ac06c920 0x7f93ac06ede0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f93c000ba20 tx=0x7f93c0008040 comp rx=0 tx=0).stop 2024-04-20T21:31:06.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.646+0000 7f93d3c00640 1 -- 172.21.15.86:0/2831130927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93cc15e370 msgr2=0x7f93cc0b0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:06.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.646+0000 7f93d3c00640 1 --2- 172.21.15.86:0/2831130927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93cc15e370 0x7f93cc0b0d00 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f93b800e7b0 tx=0x7f93b800ec70 comp rx=0 tx=0).stop 2024-04-20T21:31:06.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.647+0000 7f93d3c00640 1 -- 172.21.15.86:0/2831130927 shutdown_connections 2024-04-20T21:31:06.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.647+0000 7f93d3c00640 1 --2- 172.21.15.86:0/2831130927 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f93ac06c920 0x7f93ac06ede0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:06.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.647+0000 7f93d3c00640 1 --2- 172.21.15.86:0/2831130927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93cc145910 0x7f93cc145d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:06.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.647+0000 7f93d3c00640 1 --2- 172.21.15.86:0/2831130927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93cc15e370 0x7f93cc0b0d00 unknown :-1 s=CLOSED pgs=2134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:06.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.647+0000 7f93d3c00640 1 --2- 172.21.15.86:0/2831130927 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f93b4004d40 0x7f93cc148080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:06.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.647+0000 7f93d3c00640 1 -- 172.21.15.86:0/2831130927 >> 172.21.15.86:0/2831130927 conn(0x7f93cc0b2360 msgr2=0x7f93cc06e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:06.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.647+0000 7f93d3c00640 1 -- 172.21.15.86:0/2831130927 shutdown_connections 2024-04-20T21:31:06.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.647+0000 7f93d3c00640 1 -- 172.21.15.86:0/2831130927 wait complete. 2024-04-20T21:31:06.663 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":514,"stamp":"2024-04-20T21:31:04.911299+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":278316,"kb_used_data":28896,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754696404,"statfs":{"total":773094113280,"available":772809117696,"internally_reserved":0,"allocated":29589504,"data_stored":22995210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390725,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-264,"num_write":-177,"num_write_kb":-8296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001514"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":346,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-04-20T21:30:42.236741+0000","last_change":"2024-04-20T21:17:43.059637+0000","last_active":"2024-04-20T21:30:42.236741+0000","last_peered":"2024-04-20T21:30:42.236741+0000","last_clean":"2024-04-20T21:30:42.236741+0000","last_became_active":"2024-04-20T21:17:43.056207+0000","last_became_peered":"2024-04-20T21:17:43.056207+0000","last_unstale":"2024-04-20T21:30:42.236741+0000","last_undegraded":"2024-04-20T21:30:42.236741+0000","last_fullsized":"2024-04-20T21:30:42.236741+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:41.036290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:41.036290+0000","last_clean_scrub_stamp":"2024-04-20T21:17:41.036290+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:58:51.091337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575010,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33176,"kb_used_data":1996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338664,"statfs":{"total":96636764160,"available":96602791936,"internally_reserved":0,"allocated":2043904,"data_stored":1220112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575011,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36688,"kb_used_data":5512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335152,"statfs":{"total":96636764160,"available":96599195648,"internally_reserved":0,"allocated":5644288,"data_stored":4823509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575013,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32600,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339240,"statfs":{"total":96636764160,"available":96603381760,"internally_reserved":0,"allocated":1458176,"data_stored":629744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575012,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32588,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1441792,"data_stored":629128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575012,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36700,"kb_used_data":5524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335140,"statfs":{"total":96636764160,"available":96599183360,"internally_reserved":0,"allocated":5656576,"data_stored":4824809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575012,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32592,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1445888,"data_stored":629133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575011,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36688,"kb_used_data":5512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335152,"statfs":{"total":96636764160,"available":96599195648,"internally_reserved":0,"allocated":5644288,"data_stored":4823556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575012,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37284,"kb_used_data":6108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334556,"statfs":{"total":96636764160,"available":96598585344,"internally_reserved":0,"allocated":6254592,"data_stored":5415219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T21:31:06.664 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T21:31:06.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.867+0000 7f5e81000640 1 Processor -- start 2024-04-20T21:31:06.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.867+0000 7f5e81000640 1 -- start start 2024-04-20T21:31:06.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.868+0000 7f5e81000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5e7c0af7b0 0x7f5e7c15c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:06.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.868+0000 7f5e81000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5e7c06f4c0 0x7f5e7c06f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:06.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.868+0000 7f5e81000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e7c0ae710 con 0x7f5e7c15c010 2024-04-20T21:31:06.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.868+0000 7f5e81000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f5e7c0af320 con 0x7f5e7c06f4c0 2024-04-20T21:31:06.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.868+0000 7f5e81000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f5e7c0ae4b0 con 0x7f5e7c0af7b0 2024-04-20T21:31:06.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.869+0000 7f5e7be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5e7c06f4c0 0x7f5e7c06f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:06.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.869+0000 7f5e7be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5e7c06f4c0 0x7f5e7c06f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33470/0 (socket says 172.21.15.86:33470) 2024-04-20T21:31:06.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.869+0000 7f5e7be00640 1 -- 172.21.15.86:0/4112646836 learned_addr learned my addr 172.21.15.86:0/4112646836 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:31:06.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.869+0000 7f5e73e00640 1 --2- 172.21.15.86:0/4112646836 >> v2:172.21.15.144:3300/0 conn(0x7f5e7c0af7b0 0x7f5e7c15c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:06.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.869+0000 7f5e7be00640 1 -- 172.21.15.86:0/4112646836 >> v2:172.21.15.144:3300/0 conn(0x7f5e7c0af7b0 msgr2=0x7f5e7c15c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:06.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.869+0000 7f5e7be00640 1 --2- 172.21.15.86:0/4112646836 >> v2:172.21.15.144:3300/0 conn(0x7f5e7c0af7b0 0x7f5e7c15c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:06.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.869+0000 7f5e7be00640 1 -- 172.21.15.86:0/4112646836 >> v1:172.21.15.144:6789/0 conn(0x7f5e7c15c010 legacy=0x7f5e7c0adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:06.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.869+0000 7f5e7be00640 1 -- 172.21.15.86:0/4112646836 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e7c0ae9f0 con 0x7f5e7c06f4c0 2024-04-20T21:31:06.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.869+0000 7f5e73e00640 1 --2- 172.21.15.86:0/4112646836 >> v2:172.21.15.144:3300/0 conn(0x7f5e7c0af7b0 0x7f5e7c15c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:31:06.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.870+0000 7f5e7be00640 1 --2- 172.21.15.86:0/4112646836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e7c06f4c0 0x7f5e7c06f8f0 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c0af4a0 tx=0x7f5e7402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1949475b4eb2ce8a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:06.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.871+0000 7f5e7aa00640 1 -- 172.21.15.86:0/4112646836 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7402ca70 con 0x7f5e7c06f4c0 2024-04-20T21:31:06.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.871+0000 7f5e7aa00640 1 -- 172.21.15.86:0/4112646836 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e7402cc10 con 0x7f5e7c06f4c0 2024-04-20T21:31:06.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.871+0000 7f5e7aa00640 1 -- 172.21.15.86:0/4112646836 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e740343f0 con 0x7f5e7c06f4c0 2024-04-20T21:31:06.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.871+0000 7f5e81000640 1 -- 172.21.15.86:0/4112646836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e7c06f4c0 msgr2=0x7f5e7c06f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:06.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.871+0000 7f5e81000640 1 --2- 172.21.15.86:0/4112646836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e7c06f4c0 0x7f5e7c06f8f0 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c0af4a0 tx=0x7f5e7402bed0 comp rx=0 tx=0).stop 2024-04-20T21:31:06.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.871+0000 7f5e81000640 1 -- 172.21.15.86:0/4112646836 shutdown_connections 2024-04-20T21:31:06.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.872+0000 7f5e81000640 1 --2- 172.21.15.86:0/4112646836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e7c06f4c0 0x7f5e7c06f8f0 unknown :-1 s=CLOSED pgs=2135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:06.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.872+0000 7f5e81000640 1 --2- 172.21.15.86:0/4112646836 >> v2:172.21.15.144:3300/0 conn(0x7f5e7c0af7b0 0x7f5e7c15c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:06.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.872+0000 7f5e81000640 1 -- 172.21.15.86:0/4112646836 >> 172.21.15.86:0/4112646836 conn(0x7f5e7c0d0170 msgr2=0x7f5e7c0d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:06.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.872+0000 7f5e81000640 1 -- 172.21.15.86:0/4112646836 shutdown_connections 2024-04-20T21:31:06.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.872+0000 7f5e81000640 1 -- 172.21.15.86:0/4112646836 wait complete. 2024-04-20T21:31:06.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.873+0000 7f5e81000640 1 Processor -- start 2024-04-20T21:31:06.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.873+0000 7f5e81000640 1 -- start start 2024-04-20T21:31:06.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.874+0000 7f5e81000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e7c06f4c0 0x7f5e7c1670c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:06.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.874+0000 7f5e81000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e7c0af7b0 0x7f5e7c164c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:06.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.874+0000 7f5e81000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e7c15c010 0x7f5e7c15a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:06.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.874+0000 7f5e81000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5e7c06fe30 con 0x7f5e7c06f4c0 2024-04-20T21:31:06.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.874+0000 7f5e81000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5e7c15c930 con 0x7f5e7c15c010 2024-04-20T21:31:06.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.874+0000 7f5e81000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5e7c0aefb0 con 0x7f5e7c0af7b0 2024-04-20T21:31:06.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.875+0000 7f5e7b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e7c06f4c0 0x7f5e7c1670c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:06.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.875+0000 7f5e7b400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e7c06f4c0 0x7f5e7c1670c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33474/0 (socket says 172.21.15.86:33474) 2024-04-20T21:31:06.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.875+0000 7f5e7b400640 1 -- 172.21.15.86:0/3366701791 learned_addr learned my addr 172.21.15.86:0/3366701791 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:31:06.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.875+0000 7f5e7be00640 1 --2- 172.21.15.86:0/3366701791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e7c15c010 0x7f5e7c15a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:06.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.875+0000 7f5e73e00640 1 --2- 172.21.15.86:0/3366701791 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e7c0af7b0 0x7f5e7c164c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:06.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.875+0000 7f5e7b400640 1 -- 172.21.15.86:0/3366701791 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e7c0af7b0 msgr2=0x7f5e7c164c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:06.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.875+0000 7f5e7b400640 1 --2- 172.21.15.86:0/3366701791 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e7c0af7b0 0x7f5e7c164c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:06.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.875+0000 7f5e7b400640 1 -- 172.21.15.86:0/3366701791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e7c15c010 msgr2=0x7f5e7c15a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:06.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.875+0000 7f5e7b400640 1 --2- 172.21.15.86:0/3366701791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e7c15c010 0x7f5e7c15a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:06.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.875+0000 7f5e7b400640 1 -- 172.21.15.86:0/3366701791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e7c1653a0 con 0x7f5e7c06f4c0 2024-04-20T21:31:06.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.875+0000 7f5e7b400640 1 --2- 172.21.15.86:0/3366701791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e7c06f4c0 0x7f5e7c1670c0 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c00b3e0 tx=0x7f5e6c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:06.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.876+0000 7f5e73400640 1 -- 172.21.15.86:0/3366701791 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c015020 con 0x7f5e7c06f4c0 2024-04-20T21:31:06.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.876+0000 7f5e73400640 1 -- 172.21.15.86:0/3366701791 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e6c011850 con 0x7f5e7c06f4c0 2024-04-20T21:31:06.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.876+0000 7f5e81000640 1 -- 172.21.15.86:0/3366701791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e7c165680 con 0x7f5e7c06f4c0 2024-04-20T21:31:06.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.876+0000 7f5e73400640 1 -- 172.21.15.86:0/3366701791 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c011c40 con 0x7f5e7c06f4c0 2024-04-20T21:31:06.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.876+0000 7f5e81000640 1 -- 172.21.15.86:0/3366701791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e7c09d640 con 0x7f5e7c06f4c0 2024-04-20T21:31:06.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.877+0000 7f5e81000640 1 -- 172.21.15.86:0/3366701791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e7c04fcf0 con 0x7f5e7c06f4c0 2024-04-20T21:31:06.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.878+0000 7f5e73400640 1 -- 172.21.15.86:0/3366701791 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5e6c018020 con 0x7f5e7c06f4c0 2024-04-20T21:31:06.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.879+0000 7f5e73400640 1 --2- 172.21.15.86:0/3366701791 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5e5806c8f0 0x7f5e5806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:06.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.879+0000 7f5e73400640 1 -- 172.21.15.86:0/3366701791 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f5e6c08b960 con 0x7f5e7c06f4c0 2024-04-20T21:31:06.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.879+0000 7f5e73e00640 1 --2- 172.21.15.86:0/3366701791 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5e5806c8f0 0x7f5e5806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:06.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.880+0000 7f5e73e00640 1 --2- 172.21.15.86:0/3366701791 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5e5806c8f0 0x7f5e5806edb0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c167f50 tx=0x7f5e64008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:06.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:06.884+0000 7f5e73400640 1 -- 172.21.15.86:0/3366701791 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5e6c01f020 con 0x7f5e7c06f4c0 2024-04-20T21:31:07.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.077+0000 7f5e81000640 1 -- 172.21.15.86:0/3366701791 --> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5e7c09d840 con 0x7f5e5806c8f0 2024-04-20T21:31:07.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.080+0000 7f5e73400640 1 -- 172.21.15.86:0/3366701791 <== mgr.4107 v2:172.21.15.86:6832/302045240 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13516 (secure 0 0 0) 0x7f5e7c09d840 con 0x7f5e5806c8f0 2024-04-20T21:31:07.079 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:31:07.079 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-20T21:31:07.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.082+0000 7f5e81000640 1 -- 172.21.15.86:0/3366701791 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5e5806c8f0 msgr2=0x7f5e5806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:07.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.082+0000 7f5e81000640 1 --2- 172.21.15.86:0/3366701791 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5e5806c8f0 0x7f5e5806edb0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c167f50 tx=0x7f5e64008040 comp rx=0 tx=0).stop 2024-04-20T21:31:07.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.082+0000 7f5e81000640 1 -- 172.21.15.86:0/3366701791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e7c06f4c0 msgr2=0x7f5e7c1670c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:07.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.082+0000 7f5e81000640 1 --2- 172.21.15.86:0/3366701791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e7c06f4c0 0x7f5e7c1670c0 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c00b3e0 tx=0x7f5e6c00b8a0 comp rx=0 tx=0).stop 2024-04-20T21:31:07.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.082+0000 7f5e81000640 1 -- 172.21.15.86:0/3366701791 shutdown_connections 2024-04-20T21:31:07.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.082+0000 7f5e81000640 1 --2- 172.21.15.86:0/3366701791 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5e5806c8f0 0x7f5e5806edb0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:07.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.082+0000 7f5e81000640 1 --2- 172.21.15.86:0/3366701791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e7c15c010 0x7f5e7c15a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:07.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.082+0000 7f5e81000640 1 --2- 172.21.15.86:0/3366701791 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5e7c0af7b0 0x7f5e7c164c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:07.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.082+0000 7f5e81000640 1 --2- 172.21.15.86:0/3366701791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e7c06f4c0 0x7f5e7c1670c0 unknown :-1 s=CLOSED pgs=2136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:07.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.082+0000 7f5e81000640 1 -- 172.21.15.86:0/3366701791 >> 172.21.15.86:0/3366701791 conn(0x7f5e7c0d0170 msgr2=0x7f5e7c0b1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:07.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.083+0000 7f5e81000640 1 -- 172.21.15.86:0/3366701791 shutdown_connections 2024-04-20T21:31:07.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.083+0000 7f5e81000640 1 -- 172.21.15.86:0/3366701791 wait complete. 2024-04-20T21:31:07.101 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":515,"stamp":"2024-04-20T21:31:06.911665+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":116,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":262236,"kb_used_data":12776,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712484,"statfs":{"total":773094113280,"available":772825583616,"internally_reserved":0,"allocated":13082624,"data_stored":6360280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390725,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-264,"num_write":-179,"num_write_kb":-8296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.091637"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":352,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-20T21:31:04.800301+0000","last_change":"2024-04-20T21:17:43.059637+0000","last_active":"2024-04-20T21:31:04.800301+0000","last_peered":"2024-04-20T21:31:04.800301+0000","last_clean":"2024-04-20T21:31:04.800301+0000","last_became_active":"2024-04-20T21:17:43.056207+0000","last_became_peered":"2024-04-20T21:17:43.056207+0000","last_unstale":"2024-04-20T21:31:04.800301+0000","last_undegraded":"2024-04-20T21:31:04.800301+0000","last_fullsized":"2024-04-20T21:31:04.800301+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:41.036290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:41.036290+0000","last_clean_scrub_stamp":"2024-04-20T21:17:41.036290+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:58:51.091337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575011,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33216,"kb_used_data":2032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338624,"statfs":{"total":96636764160,"available":96602750976,"internally_reserved":0,"allocated":2080768,"data_stored":1237811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575012,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32632,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1486848,"data_stored":647443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575014,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32636,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339204,"statfs":{"total":96636764160,"available":96603344896,"internally_reserved":0,"allocated":1486848,"data_stored":647443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575013,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32632,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1486848,"data_stored":647443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575013,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32636,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339204,"statfs":{"total":96636764160,"available":96603344896,"internally_reserved":0,"allocated":1486848,"data_stored":647443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575013,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32636,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339204,"statfs":{"total":96636764160,"available":96603344896,"internally_reserved":0,"allocated":1486848,"data_stored":647443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575012,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32632,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1486848,"data_stored":647443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575013,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33216,"kb_used_data":2032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338624,"statfs":{"total":96636764160,"available":96602750976,"internally_reserved":0,"allocated":2080768,"data_stored":1237811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T21:31:07.101 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-20T21:31:07.101 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T21:31:07.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.302+0000 7f187ca00640 1 Processor -- start 2024-04-20T21:31:07.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.302+0000 7f187ca00640 1 -- start start 2024-04-20T21:31:07.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.304+0000 7f187ca00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f187806e6d0 0x7f1878071d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:07.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.304+0000 7f187ca00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1878072240 0x7f187815cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:07.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.304+0000 7f187ca00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f187806eb30 con 0x7f1878146130 2024-04-20T21:31:07.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.304+0000 7f187ca00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f187806dc30 con 0x7f187806e6d0 2024-04-20T21:31:07.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.304+0000 7f187ca00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f18780cd8e0 con 0x7f1878072240 2024-04-20T21:31:07.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.304+0000 7f1875e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f1878146130 0x7f18780ccdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:53080/0 (socket says 172.21.15.86:53080) 2024-04-20T21:31:07.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.304+0000 7f1875e00640 1 -- 172.21.15.86:0/655461101 learned_addr learned my addr 172.21.15.86:0/655461101 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:31:07.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.304+0000 7f1876800640 1 --2- 172.21.15.86:0/655461101 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1878072240 0x7f187815cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:07.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.304+0000 7f1874a00640 1 -- 172.21.15.86:0/655461101 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 534189813 0 0) 0x7f1868001110 con 0x7f1878146130 2024-04-20T21:31:07.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.304+0000 7f1874a00640 1 -- 172.21.15.86:0/655461101 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f185c003520 con 0x7f1878146130 2024-04-20T21:31:07.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.304+0000 7f1875400640 1 --2- 172.21.15.86:0/655461101 >> v2:172.21.15.144:3300/0 conn(0x7f187806e6d0 0x7f1878071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:07.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.305+0000 7f1874a00640 1 -- 172.21.15.86:0/655461101 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3892252765 0 0) 0x7f185c003520 con 0x7f1878146130 2024-04-20T21:31:07.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.305+0000 7f1874a00640 1 -- 172.21.15.86:0/655461101 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1878072240 msgr2=0x7f187815cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:07.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.305+0000 7f1874a00640 1 --2- 172.21.15.86:0/655461101 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1878072240 0x7f187815cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:07.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.305+0000 7f1874a00640 1 -- 172.21.15.86:0/655461101 >> v2:172.21.15.144:3300/0 conn(0x7f187806e6d0 msgr2=0x7f1878071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:07.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.305+0000 7f1874a00640 1 --2- 172.21.15.86:0/655461101 >> v2:172.21.15.144:3300/0 conn(0x7f187806e6d0 0x7f1878071d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:07.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.305+0000 7f1874a00640 1 -- 172.21.15.86:0/655461101 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18780cdbc0 con 0x7f1878146130 2024-04-20T21:31:07.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.305+0000 7f1874a00640 1 -- 172.21.15.86:0/655461101 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f1868001840 con 0x7f1878146130 2024-04-20T21:31:07.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.305+0000 7f1874a00640 1 -- 172.21.15.86:0/655461101 >> v1:172.21.15.86:6789/0 conn(0x7f1878146130 legacy=0x7f18780ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:07.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.305+0000 7f1874a00640 1 --2- 172.21.15.86:0/655461101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f185c0039b0 0x7f185c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:07.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.305+0000 7f1874a00640 1 --2- 172.21.15.86:0/655461101 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1878072240 0x7f185c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:07.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.305+0000 7f1874a00640 1 --2- 172.21.15.86:0/655461101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f185c004d40 0x7f185c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:07.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.305+0000 7f1874a00640 1 -- 172.21.15.86:0/655461101 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f187806dc30 con 0x7f185c0039b0 2024-04-20T21:31:07.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.305+0000 7f1874a00640 1 -- 172.21.15.86:0/655461101 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f18780cd8e0 con 0x7f185c004d40 2024-04-20T21:31:07.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.305+0000 7f1874a00640 1 -- 172.21.15.86:0/655461101 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f185c00ba80 con 0x7f1878072240 2024-04-20T21:31:07.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.305+0000 7f1875e00640 1 --2- 172.21.15.86:0/655461101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f185c0039b0 0x7f185c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:07.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.305+0000 7f1875e00640 1 -- 172.21.15.86:0/655461101 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1878072240 msgr2=0x7f185c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:07.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.305+0000 7f1875e00640 1 --2- 172.21.15.86:0/655461101 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1878072240 0x7f185c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:07.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.305+0000 7f1875e00640 1 -- 172.21.15.86:0/655461101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f185c004d40 msgr2=0x7f185c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:07.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.305+0000 7f1876800640 1 --2- 172.21.15.86:0/655461101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f185c004d40 0x7f185c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:07.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.306+0000 7f1875e00640 1 --2- 172.21.15.86:0/655461101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f185c004d40 0x7f185c0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:07.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.306+0000 7f1875e00640 1 -- 172.21.15.86:0/655461101 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f185c003520 con 0x7f185c0039b0 2024-04-20T21:31:07.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.307+0000 7f1875e00640 1 --2- 172.21.15.86:0/655461101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f185c0039b0 0x7f185c003dd0 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7f186800c3e0 tx=0x7f185c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59e2e72c17e3b984 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:07.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.307+0000 7f1874a00640 1 -- 172.21.15.86:0/655461101 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186800c0e0 con 0x7f185c0039b0 2024-04-20T21:31:07.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.307+0000 7f1874a00640 1 -- 172.21.15.86:0/655461101 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1868030070 con 0x7f185c0039b0 2024-04-20T21:31:07.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.307+0000 7f1874a00640 1 -- 172.21.15.86:0/655461101 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18680058d0 con 0x7f185c0039b0 2024-04-20T21:31:07.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.307+0000 7f187ca00640 1 -- 172.21.15.86:0/655461101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f185c0039b0 msgr2=0x7f185c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:07.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.307+0000 7f187ca00640 1 --2- 172.21.15.86:0/655461101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f185c0039b0 0x7f185c003dd0 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7f186800c3e0 tx=0x7f185c005830 comp rx=0 tx=0).stop 2024-04-20T21:31:07.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.307+0000 7f1876800640 1 -- 172.21.15.86:0/655461101 reap_dead start 2024-04-20T21:31:07.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.308+0000 7f187ca00640 1 -- 172.21.15.86:0/655461101 shutdown_connections 2024-04-20T21:31:07.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.308+0000 7f187ca00640 1 -- 172.21.15.86:0/655461101 >> 172.21.15.86:0/655461101 conn(0x7f18780b2360 msgr2=0x7f18780b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:07.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.308+0000 7f187ca00640 1 -- 172.21.15.86:0/655461101 shutdown_connections 2024-04-20T21:31:07.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.308+0000 7f187ca00640 1 -- 172.21.15.86:0/655461101 wait complete. 2024-04-20T21:31:07.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.309+0000 7f187ca00640 1 Processor -- start 2024-04-20T21:31:07.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.310+0000 7f187ca00640 1 -- start start 2024-04-20T21:31:07.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.310+0000 7f187ca00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f185c0039b0 0x7f187813bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:07.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.310+0000 7f187ca00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18781f2330 0x7f187813c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:07.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.310+0000 7f187ca00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f187813c670 0x7f187815db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:07.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.310+0000 7f187ca00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f187806f150 con 0x7f185c0039b0 2024-04-20T21:31:07.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.310+0000 7f187ca00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f187806e1b0 con 0x7f18781f2330 2024-04-20T21:31:07.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.310+0000 7f187ca00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f18780cd4e0 con 0x7f187813c670 2024-04-20T21:31:07.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.311+0000 7f1875e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f185c0039b0 0x7f187813bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:07.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.311+0000 7f1875e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f185c0039b0 0x7f187813bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33488/0 (socket says 172.21.15.86:33488) 2024-04-20T21:31:07.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.311+0000 7f1875e00640 1 -- 172.21.15.86:0/1303454542 learned_addr learned my addr 172.21.15.86:0/1303454542 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:31:07.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.311+0000 7f1875400640 1 --2- 172.21.15.86:0/1303454542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18781f2330 0x7f187813c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:07.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.311+0000 7f1876800640 1 --2- 172.21.15.86:0/1303454542 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f187813c670 0x7f187815db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:07.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.311+0000 7f1875e00640 1 -- 172.21.15.86:0/1303454542 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f187813c670 msgr2=0x7f187815db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:07.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.311+0000 7f1875e00640 1 --2- 172.21.15.86:0/1303454542 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f187813c670 0x7f187815db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:07.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.311+0000 7f1875e00640 1 -- 172.21.15.86:0/1303454542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18781f2330 msgr2=0x7f187813c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:07.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.311+0000 7f1875e00640 1 --2- 172.21.15.86:0/1303454542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18781f2330 0x7f187813c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:07.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.311+0000 7f1875e00640 1 -- 172.21.15.86:0/1303454542 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f187813ab50 con 0x7f185c0039b0 2024-04-20T21:31:07.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.311+0000 7f1875400640 1 --2- 172.21.15.86:0/1303454542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18781f2330 0x7f187813c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:31:07.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.312+0000 7f1875e00640 1 --2- 172.21.15.86:0/1303454542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f185c0039b0 0x7f187813bbf0 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7f186802c770 tx=0x7f1878146510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:07.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.312+0000 7f186e000640 1 -- 172.21.15.86:0/1303454542 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1868004bf0 con 0x7f185c0039b0 2024-04-20T21:31:07.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.312+0000 7f186e000640 1 -- 172.21.15.86:0/1303454542 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1868004d90 con 0x7f185c0039b0 2024-04-20T21:31:07.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.312+0000 7f186e000640 1 -- 172.21.15.86:0/1303454542 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1868007270 con 0x7f185c0039b0 2024-04-20T21:31:07.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.312+0000 7f187ca00640 1 -- 172.21.15.86:0/1303454542 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f187813ae30 con 0x7f185c0039b0 2024-04-20T21:31:07.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.312+0000 7f187ca00640 1 -- 172.21.15.86:0/1303454542 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f187813b3f0 con 0x7f185c0039b0 2024-04-20T21:31:07.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.313+0000 7f187ca00640 1 -- 172.21.15.86:0/1303454542 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18780689e0 con 0x7f185c0039b0 2024-04-20T21:31:07.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.314+0000 7f186e000640 1 -- 172.21.15.86:0/1303454542 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1868007410 con 0x7f185c0039b0 2024-04-20T21:31:07.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.315+0000 7f186e000640 1 --2- 172.21.15.86:0/1303454542 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f185806c920 0x7f185806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:07.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.315+0000 7f186e000640 1 -- 172.21.15.86:0/1303454542 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f18680ad210 con 0x7f185c0039b0 2024-04-20T21:31:07.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.315+0000 7f1875400640 1 --2- 172.21.15.86:0/1303454542 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f185806c920 0x7f185806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:07.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.316+0000 7f1875400640 1 --2- 172.21.15.86:0/1303454542 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f185806c920 0x7f185806ede0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f187815c7b0 tx=0x7f1864008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:07.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.320+0000 7f186e000640 1 -- 172.21.15.86:0/1303454542 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f187806eb30 con 0x7f185c0039b0 2024-04-20T21:31:07.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.515+0000 7f187ca00640 1 -- 172.21.15.86:0/1303454542 --> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f187815ff50 con 0x7f185806c920 2024-04-20T21:31:07.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.517+0000 7f186e000640 1 -- 172.21.15.86:0/1303454542 <== mgr.4107 v2:172.21.15.86:6832/302045240 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13516 (secure 0 0 0) 0x7f187815ff50 con 0x7f185806c920 2024-04-20T21:31:07.516 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:31:07.516 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-20T21:31:07.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.520+0000 7f187ca00640 1 -- 172.21.15.86:0/1303454542 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f185806c920 msgr2=0x7f185806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:07.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.520+0000 7f187ca00640 1 --2- 172.21.15.86:0/1303454542 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f185806c920 0x7f185806ede0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f187815c7b0 tx=0x7f1864008040 comp rx=0 tx=0).stop 2024-04-20T21:31:07.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.520+0000 7f187ca00640 1 -- 172.21.15.86:0/1303454542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f185c0039b0 msgr2=0x7f187813bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:07.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.520+0000 7f187ca00640 1 --2- 172.21.15.86:0/1303454542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f185c0039b0 0x7f187813bbf0 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7f186802c770 tx=0x7f1878146510 comp rx=0 tx=0).stop 2024-04-20T21:31:07.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.520+0000 7f187ca00640 1 -- 172.21.15.86:0/1303454542 shutdown_connections 2024-04-20T21:31:07.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.520+0000 7f187ca00640 1 --2- 172.21.15.86:0/1303454542 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f185806c920 0x7f185806ede0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:07.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.520+0000 7f187ca00640 1 --2- 172.21.15.86:0/1303454542 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f187813c670 0x7f187815db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:07.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.520+0000 7f187ca00640 1 --2- 172.21.15.86:0/1303454542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f18781f2330 0x7f187813c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:07.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.520+0000 7f187ca00640 1 --2- 172.21.15.86:0/1303454542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f185c0039b0 0x7f187813bbf0 unknown :-1 s=CLOSED pgs=2139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:07.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.520+0000 7f187ca00640 1 -- 172.21.15.86:0/1303454542 >> 172.21.15.86:0/1303454542 conn(0x7f18780b2360 msgr2=0x7f1878144010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:07.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.520+0000 7f187ca00640 1 -- 172.21.15.86:0/1303454542 shutdown_connections 2024-04-20T21:31:07.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.520+0000 7f187ca00640 1 -- 172.21.15.86:0/1303454542 wait complete. 2024-04-20T21:31:07.537 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":515,"stamp":"2024-04-20T21:31:06.911665+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":116,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":262236,"kb_used_data":12776,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712484,"statfs":{"total":773094113280,"available":772825583616,"internally_reserved":0,"allocated":13082624,"data_stored":6360280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390725,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-264,"num_write":-179,"num_write_kb":-8296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.091637"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":352,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-20T21:31:04.800301+0000","last_change":"2024-04-20T21:17:43.059637+0000","last_active":"2024-04-20T21:31:04.800301+0000","last_peered":"2024-04-20T21:31:04.800301+0000","last_clean":"2024-04-20T21:31:04.800301+0000","last_became_active":"2024-04-20T21:17:43.056207+0000","last_became_peered":"2024-04-20T21:17:43.056207+0000","last_unstale":"2024-04-20T21:31:04.800301+0000","last_undegraded":"2024-04-20T21:31:04.800301+0000","last_fullsized":"2024-04-20T21:31:04.800301+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:41.036290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:41.036290+0000","last_clean_scrub_stamp":"2024-04-20T21:17:41.036290+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:58:51.091337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575011,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33216,"kb_used_data":2032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338624,"statfs":{"total":96636764160,"available":96602750976,"internally_reserved":0,"allocated":2080768,"data_stored":1237811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575012,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32632,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1486848,"data_stored":647443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575014,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32636,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339204,"statfs":{"total":96636764160,"available":96603344896,"internally_reserved":0,"allocated":1486848,"data_stored":647443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575013,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32632,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1486848,"data_stored":647443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575013,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32636,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339204,"statfs":{"total":96636764160,"available":96603344896,"internally_reserved":0,"allocated":1486848,"data_stored":647443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575013,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32636,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339204,"statfs":{"total":96636764160,"available":96603344896,"internally_reserved":0,"allocated":1486848,"data_stored":647443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575012,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32632,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1486848,"data_stored":647443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575013,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33216,"kb_used_data":2032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338624,"statfs":{"total":96636764160,"available":96602750976,"internally_reserved":0,"allocated":2080768,"data_stored":1237811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T21:31:07.538 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T21:31:07.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.736+0000 7f3ee9200640 1 Processor -- start 2024-04-20T21:31:07.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.737+0000 7f3ee9200640 1 -- start start 2024-04-20T21:31:07.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.738+0000 7f3ee9200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f3ee40cd170 0x7f3ee406ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:07.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.738+0000 7f3ee9200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3ee40cd600 0x7f3ee406f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:07.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.738+0000 7f3ee9200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ee40cdb50 con 0x7f3ee4144e90 2024-04-20T21:31:07.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.738+0000 7f3ee9200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f3ee4145270 con 0x7f3ee40cd170 2024-04-20T21:31:07.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.738+0000 7f3ee9200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3ee40b3e50 con 0x7f3ee40cd600 2024-04-20T21:31:07.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.738+0000 7f3ee1e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f3ee4144e90 0x7f3ee40b4930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:53092/0 (socket says 172.21.15.86:53092) 2024-04-20T21:31:07.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.738+0000 7f3ee1e00640 1 -- 172.21.15.86:0/491965543 learned_addr learned my addr 172.21.15.86:0/491965543 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:31:07.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.739+0000 7f3ee2800640 1 --2- 172.21.15.86:0/491965543 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3ee40cd600 0x7f3ee406f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:07.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.739+0000 7f3ed9400640 1 --2- 172.21.15.86:0/491965543 >> v2:172.21.15.144:3300/0 conn(0x7f3ee40cd170 0x7f3ee406ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:07.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.739+0000 7f3ee1400640 1 -- 172.21.15.86:0/491965543 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 665862802 0 0) 0x7f3ee40cdb50 con 0x7f3ee4144e90 2024-04-20T21:31:07.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.739+0000 7f3ee1400640 1 -- 172.21.15.86:0/491965543 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ec8003520 con 0x7f3ee4144e90 2024-04-20T21:31:07.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.739+0000 7f3ee1400640 1 -- 172.21.15.86:0/491965543 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3975116327 0 0) 0x7f3ec8003520 con 0x7f3ee4144e90 2024-04-20T21:31:07.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.739+0000 7f3ee1400640 1 -- 172.21.15.86:0/491965543 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3ee40cd600 msgr2=0x7f3ee406f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:07.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.739+0000 7f3ee1400640 1 --2- 172.21.15.86:0/491965543 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3ee40cd600 0x7f3ee406f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:07.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.739+0000 7f3ee1400640 1 -- 172.21.15.86:0/491965543 >> v2:172.21.15.144:3300/0 conn(0x7f3ee40cd170 msgr2=0x7f3ee406ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:07.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.739+0000 7f3ee1400640 1 --2- 172.21.15.86:0/491965543 >> v2:172.21.15.144:3300/0 conn(0x7f3ee40cd170 0x7f3ee406ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:07.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.739+0000 7f3ee1400640 1 -- 172.21.15.86:0/491965543 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ee40b4130 con 0x7f3ee4144e90 2024-04-20T21:31:07.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.739+0000 7f3ee1400640 1 -- 172.21.15.86:0/491965543 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f3ed4001670 con 0x7f3ee4144e90 2024-04-20T21:31:07.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.739+0000 7f3ee1400640 1 -- 172.21.15.86:0/491965543 >> v1:172.21.15.86:6789/0 conn(0x7f3ee4144e90 legacy=0x7f3ee40b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:07.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.739+0000 7f3ee1400640 1 --2- 172.21.15.86:0/491965543 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3ee40cd600 0x7f3ec80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:07.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.739+0000 7f3ee1400640 1 --2- 172.21.15.86:0/491965543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ec8003f30 0x7f3ec8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:07.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.739+0000 7f3ee1400640 1 --2- 172.21.15.86:0/491965543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ec8004d40 0x7f3ec80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:07.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.739+0000 7f3ee1400640 1 -- 172.21.15.86:0/491965543 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee4145270 con 0x7f3ec8003f30 2024-04-20T21:31:07.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.739+0000 7f3ee1400640 1 -- 172.21.15.86:0/491965543 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee40b3e50 con 0x7f3ec8004d40 2024-04-20T21:31:07.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.739+0000 7f3ee1400640 1 -- 172.21.15.86:0/491965543 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3ec800ba10 con 0x7f3ee40cd600 2024-04-20T21:31:07.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.740+0000 7f3ed9400640 1 --2- 172.21.15.86:0/491965543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ec8003f30 0x7f3ec8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:07.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.740+0000 7f3ee1e00640 1 --2- 172.21.15.86:0/491965543 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3ee40cd600 0x7f3ec80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:07.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.740+0000 7f3ee2800640 1 --2- 172.21.15.86:0/491965543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ec8004d40 0x7f3ec80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:07.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.740+0000 7f3ed9400640 1 -- 172.21.15.86:0/491965543 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3ee40cd600 msgr2=0x7f3ec80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:07.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.740+0000 7f3ed9400640 1 --2- 172.21.15.86:0/491965543 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3ee40cd600 0x7f3ec80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:07.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.740+0000 7f3ed9400640 1 -- 172.21.15.86:0/491965543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ec8004d40 msgr2=0x7f3ec80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:07.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.740+0000 7f3ed9400640 1 --2- 172.21.15.86:0/491965543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ec8004d40 0x7f3ec80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:07.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.740+0000 7f3ed9400640 1 -- 172.21.15.86:0/491965543 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ec8003520 con 0x7f3ec8003f30 2024-04-20T21:31:07.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.740+0000 7f3ee2800640 1 --2- 172.21.15.86:0/491965543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ec8004d40 0x7f3ec80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:31:07.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.741+0000 7f3ed9400640 1 --2- 172.21.15.86:0/491965543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ec8003f30 0x7f3ec8004800 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc009af0 tx=0x7f3ec8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5023ae256ca685e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:07.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.741+0000 7f3ee1400640 1 -- 172.21.15.86:0/491965543 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc039070 con 0x7f3ec8003f30 2024-04-20T21:31:07.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.741+0000 7f3ee1400640 1 -- 172.21.15.86:0/491965543 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ecc02c710 con 0x7f3ec8003f30 2024-04-20T21:31:07.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.741+0000 7f3ee1400640 1 -- 172.21.15.86:0/491965543 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc009690 con 0x7f3ec8003f30 2024-04-20T21:31:07.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.742+0000 7f3ee9200640 1 -- 172.21.15.86:0/491965543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ec8003f30 msgr2=0x7f3ec8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:07.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.742+0000 7f3ee9200640 1 --2- 172.21.15.86:0/491965543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ec8003f30 0x7f3ec8004800 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc009af0 tx=0x7f3ec8005830 comp rx=0 tx=0).stop 2024-04-20T21:31:07.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.742+0000 7f3ee2800640 1 -- 172.21.15.86:0/491965543 reap_dead start 2024-04-20T21:31:07.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.742+0000 7f3ee9200640 1 -- 172.21.15.86:0/491965543 shutdown_connections 2024-04-20T21:31:07.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.742+0000 7f3ee9200640 1 -- 172.21.15.86:0/491965543 >> 172.21.15.86:0/491965543 conn(0x7f3ee40b2360 msgr2=0x7f3ee40b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:07.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.742+0000 7f3ee9200640 1 -- 172.21.15.86:0/491965543 shutdown_connections 2024-04-20T21:31:07.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.742+0000 7f3ee9200640 1 -- 172.21.15.86:0/491965543 wait complete. 2024-04-20T21:31:07.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.744+0000 7f3ee9200640 1 Processor -- start 2024-04-20T21:31:07.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.744+0000 7f3ee9200640 1 -- start start 2024-04-20T21:31:07.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.745+0000 7f3ee9200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ec8003f30 0x7f3ee40a9e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:07.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.745+0000 7f3ee9200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3ee41f6650 0x7f3ee41f6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:07.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.745+0000 7f3ee9200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ee41f7010 0x7f3ee4141fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:07.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.745+0000 7f3ee9200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee40d0190 con 0x7f3ec8003f30 2024-04-20T21:31:07.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.745+0000 7f3ee9200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee40d0010 con 0x7f3ee41f7010 2024-04-20T21:31:07.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.745+0000 7f3ee9200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3ee406f280 con 0x7f3ee41f6650 2024-04-20T21:31:07.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.745+0000 7f3ee1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ec8003f30 0x7f3ee40a9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:07.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.745+0000 7f3ee1e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ec8003f30 0x7f3ee40a9e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33506/0 (socket says 172.21.15.86:33506) 2024-04-20T21:31:07.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.745+0000 7f3ee1e00640 1 -- 172.21.15.86:0/3923284219 learned_addr learned my addr 172.21.15.86:0/3923284219 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:31:07.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.746+0000 7f3ee2800640 1 --2- 172.21.15.86:0/3923284219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ee41f7010 0x7f3ee4141fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:07.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.746+0000 7f3ee1e00640 1 -- 172.21.15.86:0/3923284219 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3ee41f6650 msgr2=0x7f3ee41f6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:07.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.746+0000 7f3ee1e00640 1 --2- 172.21.15.86:0/3923284219 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3ee41f6650 0x7f3ee41f6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:07.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.746+0000 7f3ee1e00640 1 -- 172.21.15.86:0/3923284219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ee41f7010 msgr2=0x7f3ee4141fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:07.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.746+0000 7f3ee1e00640 1 --2- 172.21.15.86:0/3923284219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ee41f7010 0x7f3ee4141fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:07.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.746+0000 7f3ee1e00640 1 -- 172.21.15.86:0/3923284219 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ee40a9690 con 0x7f3ec8003f30 2024-04-20T21:31:07.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.746+0000 7f3ee1e00640 1 --2- 172.21.15.86:0/3923284219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ec8003f30 0x7f3ee40a9e30 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f3ed400e7e0 tx=0x7f3ed400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:07.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.746+0000 7f3edaa00640 1 -- 172.21.15.86:0/3923284219 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed400c8d0 con 0x7f3ec8003f30 2024-04-20T21:31:07.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.746+0000 7f3edaa00640 1 -- 172.21.15.86:0/3923284219 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ed4018070 con 0x7f3ec8003f30 2024-04-20T21:31:07.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.746+0000 7f3ee9200640 1 -- 172.21.15.86:0/3923284219 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ee40a9970 con 0x7f3ec8003f30 2024-04-20T21:31:07.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.746+0000 7f3edaa00640 1 -- 172.21.15.86:0/3923284219 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed400ccd0 con 0x7f3ec8003f30 2024-04-20T21:31:07.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.746+0000 7f3ee9200640 1 -- 172.21.15.86:0/3923284219 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ee4141ae0 con 0x7f3ec8003f30 2024-04-20T21:31:07.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.747+0000 7f3ee9200640 1 -- 172.21.15.86:0/3923284219 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ee40689e0 con 0x7f3ec8003f30 2024-04-20T21:31:07.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.748+0000 7f3edaa00640 1 -- 172.21.15.86:0/3923284219 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3ed401d460 con 0x7f3ec8003f30 2024-04-20T21:31:07.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.749+0000 7f3edaa00640 1 --2- 172.21.15.86:0/3923284219 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3ec006c8f0 0x7f3ec006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:07.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.749+0000 7f3edaa00640 1 -- 172.21.15.86:0/3923284219 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f3ed408b820 con 0x7f3ec8003f30 2024-04-20T21:31:07.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.749+0000 7f3ed9400640 1 --2- 172.21.15.86:0/3923284219 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3ec006c8f0 0x7f3ec006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:07.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.750+0000 7f3ed9400640 1 --2- 172.21.15.86:0/3923284219 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3ec006c8f0 0x7f3ec006edb0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc036000 tx=0x7f3ecc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:07.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.754+0000 7f3edaa00640 1 -- 172.21.15.86:0/3923284219 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3ed4059e90 con 0x7f3ec8003f30 2024-04-20T21:31:07.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.953+0000 7f3ee9200640 1 -- 172.21.15.86:0/3923284219 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3ee4143da0 con 0x7f3ec8003f30 2024-04-20T21:31:07.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.955+0000 7f3edaa00640 1 -- 172.21.15.86:0/3923284219 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) ==== 74+0+13288 (secure 0 0 0) 0x7f3ed4059790 con 0x7f3ec8003f30 2024-04-20T21:31:07.955 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:31:07.955 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":119,"fsid":"9c2dec1d-d7ae-4a83-a812-79922808e278","created":"2024-04-20T21:17:27.498955+0000","modified":"2024-04-20T21:31:04.229503+0000","last_up_change":"2024-04-20T21:17:40.011304+0000","last_in_change":"2024-04-20T21:17:30.752446+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T21:17:40.663693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"87a2269d-c3c3-4e14-9d35-03ea51d17677","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6801","nonce":2471076035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6803","nonce":2471076035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6807","nonce":2471076035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2471076035},{"type":"v1","addr":"172.21.15.86:6805","nonce":2471076035}]},"public_addr":"172.21.15.86:6801/2471076035","cluster_addr":"172.21.15.86:6803/2471076035","heartbeat_back_addr":"172.21.15.86:6807/2471076035","heartbeat_front_addr":"172.21.15.86:6805/2471076035","state":["exists","up"]},{"osd":1,"uuid":"52e5994e-f24d-4f0c-bbea-e064c89e8af9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6819","nonce":3149564961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6821","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6822","nonce":3149564961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6827","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6829","nonce":3149564961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":3149564961},{"type":"v1","addr":"172.21.15.86:6826","nonce":3149564961}]},"public_addr":"172.21.15.86:6819/3149564961","cluster_addr":"172.21.15.86:6822/3149564961","heartbeat_back_addr":"172.21.15.86:6829/3149564961","heartbeat_front_addr":"172.21.15.86:6826/3149564961","state":["exists","up"]},{"osd":2,"uuid":"1fe89fb0-bf19-49ae-80ef-13880cf7144c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6818","nonce":3437611468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6823","nonce":3437611468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6831","nonce":3437611468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6825","nonce":3437611468},{"type":"v1","addr":"172.21.15.86:6828","nonce":3437611468}]},"public_addr":"172.21.15.86:6818/3437611468","cluster_addr":"172.21.15.86:6823/3437611468","heartbeat_back_addr":"172.21.15.86:6831/3437611468","heartbeat_front_addr":"172.21.15.86:6828/3437611468","state":["exists","up"]},{"osd":3,"uuid":"728f9832-283e-4e71-a815-33afa71d1344","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6809","nonce":794135519}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6811","nonce":794135519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6815","nonce":794135519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":794135519},{"type":"v1","addr":"172.21.15.86:6813","nonce":794135519}]},"public_addr":"172.21.15.86:6809/794135519","cluster_addr":"172.21.15.86:6811/794135519","heartbeat_back_addr":"172.21.15.86:6815/794135519","heartbeat_front_addr":"172.21.15.86:6813/794135519","state":["exists","up"]},{"osd":4,"uuid":"38b3b5f6-6c5a-4145-a43d-f88b190e50b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6801","nonce":4087309475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6803","nonce":4087309475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6807","nonce":4087309475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":4087309475},{"type":"v1","addr":"172.21.15.144:6805","nonce":4087309475}]},"public_addr":"172.21.15.144:6801/4087309475","cluster_addr":"172.21.15.144:6803/4087309475","heartbeat_back_addr":"172.21.15.144:6807/4087309475","heartbeat_front_addr":"172.21.15.144:6805/4087309475","state":["exists","up"]},{"osd":5,"uuid":"05a16fec-9d93-4955-a3b1-5d1742bdb8fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6809","nonce":1635974704}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6811","nonce":1635974704}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6815","nonce":1635974704}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1635974704},{"type":"v1","addr":"172.21.15.144:6813","nonce":1635974704}]},"public_addr":"172.21.15.144:6809/1635974704","cluster_addr":"172.21.15.144:6811/1635974704","heartbeat_back_addr":"172.21.15.144:6815/1635974704","heartbeat_front_addr":"172.21.15.144:6813/1635974704","state":["exists","up"]},{"osd":6,"uuid":"f6e35481-399e-4da1-be5c-bfe52662d6be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6825","nonce":1893803079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6827","nonce":1893803079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6831","nonce":1893803079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1893803079},{"type":"v1","addr":"172.21.15.144:6829","nonce":1893803079}]},"public_addr":"172.21.15.144:6825/1893803079","cluster_addr":"172.21.15.144:6827/1893803079","heartbeat_back_addr":"172.21.15.144:6831/1893803079","heartbeat_front_addr":"172.21.15.144:6829/1893803079","state":["exists","up"]},{"osd":7,"uuid":"d965de5d-e7cf-40fc-99b3-42ea692295a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6817","nonce":2100967057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6819","nonce":2100967057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6823","nonce":2100967057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":2100967057},{"type":"v1","addr":"172.21.15.144:6821","nonce":2100967057}]},"public_addr":"172.21.15.144:6817/2100967057","cluster_addr":"172.21.15.144:6819/2100967057","heartbeat_back_addr":"172.21.15.144:6823/2100967057","heartbeat_front_addr":"172.21.15.144:6821/2100967057","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.023672+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:35.860994+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.121802+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.145063+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.366758+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.097461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.157947+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T21:17:36.403135+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6835/2019839930":"2024-04-21T21:31:00.815525+0000","172.21.15.86:6837/2019839930":"2024-04-21T21:31:00.815525+0000","172.21.15.86:6837/1453621918":"2024-04-21T21:30:42.086474+0000","172.21.15.86:6835/1453621918":"2024-04-21T21:30:42.086474+0000","172.21.15.86:6837/289590924":"2024-04-21T21:30:19.784403+0000","172.21.15.86:6835/289590924":"2024-04-21T21:30:19.784403+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T21:31:07.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.958+0000 7f3ee9200640 1 -- 172.21.15.86:0/3923284219 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3ec006c8f0 msgr2=0x7f3ec006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:07.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.958+0000 7f3ee9200640 1 --2- 172.21.15.86:0/3923284219 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3ec006c8f0 0x7f3ec006edb0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc036000 tx=0x7f3ecc02e000 comp rx=0 tx=0).stop 2024-04-20T21:31:07.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.958+0000 7f3ee9200640 1 -- 172.21.15.86:0/3923284219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ec8003f30 msgr2=0x7f3ee40a9e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:07.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.958+0000 7f3ee9200640 1 --2- 172.21.15.86:0/3923284219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ec8003f30 0x7f3ee40a9e30 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f3ed400e7e0 tx=0x7f3ed400eca0 comp rx=0 tx=0).stop 2024-04-20T21:31:07.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.958+0000 7f3ee9200640 1 -- 172.21.15.86:0/3923284219 shutdown_connections 2024-04-20T21:31:07.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.958+0000 7f3ee9200640 1 --2- 172.21.15.86:0/3923284219 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3ec006c8f0 0x7f3ec006edb0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:07.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.958+0000 7f3ee9200640 1 --2- 172.21.15.86:0/3923284219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ee41f7010 0x7f3ee4141fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:07.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.958+0000 7f3ee9200640 1 --2- 172.21.15.86:0/3923284219 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3ee41f6650 0x7f3ee41f6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:07.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.958+0000 7f3ee9200640 1 --2- 172.21.15.86:0/3923284219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ec8003f30 0x7f3ee40a9e30 unknown :-1 s=CLOSED pgs=2142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:07.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.958+0000 7f3ee9200640 1 -- 172.21.15.86:0/3923284219 >> 172.21.15.86:0/3923284219 conn(0x7f3ee40b2360 msgr2=0x7f3ee40727c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:07.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.959+0000 7f3ee9200640 1 -- 172.21.15.86:0/3923284219 shutdown_connections 2024-04-20T21:31:07.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:07.959+0000 7f3ee9200640 1 -- 172.21.15.86:0/3923284219 wait complete. 2024-04-20T21:31:08.976 INFO:tasks.ceph:Scrubbing osd.0 2024-04-20T21:31:08.976 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-20T21:31:09.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.192+0000 7f1f65400640 1 Processor -- start 2024-04-20T21:31:09.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.192+0000 7f1f65400640 1 -- start start 2024-04-20T21:31:09.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.194+0000 7f1f65400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f600b6770 0x7f1f600cee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:09.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.194+0000 7f1f65400640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1f6006d340 0x7f1f6006d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:09.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.194+0000 7f1f65400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f601622d0 con 0x7f1f60161ef0 2024-04-20T21:31:09.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.194+0000 7f1f65400640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f1f600d4f70 con 0x7f1f6006d340 2024-04-20T21:31:09.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.194+0000 7f1f65400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1f600b3e20 con 0x7f1f600b6770 2024-04-20T21:31:09.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.194+0000 7f1f5fe00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f1f60161ef0 0x7f1f600b6060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:53098/0 (socket says 172.21.15.86:53098) 2024-04-20T21:31:09.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.194+0000 7f1f5fe00640 1 -- 172.21.15.86:0/2552328029 learned_addr learned my addr 172.21.15.86:0/2552328029 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:31:09.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.194+0000 7f1f64a00640 1 --2- 172.21.15.86:0/2552328029 >> v2:172.21.15.144:3300/0 conn(0x7f1f6006d340 0x7f1f6006d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:09.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.194+0000 7f1f5f400640 1 --2- 172.21.15.86:0/2552328029 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f600b6770 0x7f1f600cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:09.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.195+0000 7f1f5ea00640 1 -- 172.21.15.86:0/2552328029 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3112830773 0 0) 0x7f1f601622d0 con 0x7f1f60161ef0 2024-04-20T21:31:09.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.195+0000 7f1f5ea00640 1 -- 172.21.15.86:0/2552328029 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f44003520 con 0x7f1f60161ef0 2024-04-20T21:31:09.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.195+0000 7f1f5ea00640 1 -- 172.21.15.86:0/2552328029 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 230796577 0 0) 0x7f1f44003520 con 0x7f1f60161ef0 2024-04-20T21:31:09.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.195+0000 7f1f5ea00640 1 -- 172.21.15.86:0/2552328029 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f600b6770 msgr2=0x7f1f600cee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:09.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.195+0000 7f1f5ea00640 1 --2- 172.21.15.86:0/2552328029 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f600b6770 0x7f1f600cee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:09.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.195+0000 7f1f5ea00640 1 -- 172.21.15.86:0/2552328029 >> v2:172.21.15.144:3300/0 conn(0x7f1f6006d340 msgr2=0x7f1f6006d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:09.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.195+0000 7f1f5ea00640 1 --2- 172.21.15.86:0/2552328029 >> v2:172.21.15.144:3300/0 conn(0x7f1f6006d340 0x7f1f6006d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:09.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.195+0000 7f1f5ea00640 1 -- 172.21.15.86:0/2552328029 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f600b3860 con 0x7f1f60161ef0 2024-04-20T21:31:09.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.195+0000 7f1f5ea00640 1 -- 172.21.15.86:0/2552328029 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f1f50001670 con 0x7f1f60161ef0 2024-04-20T21:31:09.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.195+0000 7f1f5ea00640 1 -- 172.21.15.86:0/2552328029 >> v1:172.21.15.86:6789/0 conn(0x7f1f60161ef0 legacy=0x7f1f600b6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:09.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.195+0000 7f1f5ea00640 1 --2- 172.21.15.86:0/2552328029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f440039b0 0x7f1f44003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:09.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.195+0000 7f1f5ea00640 1 --2- 172.21.15.86:0/2552328029 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f600b6770 0x7f1f44004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:09.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.195+0000 7f1f5ea00640 1 --2- 172.21.15.86:0/2552328029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f44004d40 0x7f1f440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:09.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.195+0000 7f1f5ea00640 1 -- 172.21.15.86:0/2552328029 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1f600d4f70 con 0x7f1f440039b0 2024-04-20T21:31:09.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.195+0000 7f1f5ea00640 1 -- 172.21.15.86:0/2552328029 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1f600b3e20 con 0x7f1f44004d40 2024-04-20T21:31:09.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.195+0000 7f1f5ea00640 1 -- 172.21.15.86:0/2552328029 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1f4400b9c0 con 0x7f1f600b6770 2024-04-20T21:31:09.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.196+0000 7f1f5fe00640 1 --2- 172.21.15.86:0/2552328029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f440039b0 0x7f1f44003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:09.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.196+0000 7f1f5f400640 1 --2- 172.21.15.86:0/2552328029 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f600b6770 0x7f1f44004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:09.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.196+0000 7f1f64a00640 1 --2- 172.21.15.86:0/2552328029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f44004d40 0x7f1f440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:09.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.196+0000 7f1f5fe00640 1 -- 172.21.15.86:0/2552328029 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f600b6770 msgr2=0x7f1f44004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:09.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.196+0000 7f1f5fe00640 1 --2- 172.21.15.86:0/2552328029 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f600b6770 0x7f1f44004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:09.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.196+0000 7f1f5fe00640 1 -- 172.21.15.86:0/2552328029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f44004d40 msgr2=0x7f1f440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:09.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.196+0000 7f1f5fe00640 1 --2- 172.21.15.86:0/2552328029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f44004d40 0x7f1f440051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:09.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.196+0000 7f1f5fe00640 1 -- 172.21.15.86:0/2552328029 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f44003520 con 0x7f1f440039b0 2024-04-20T21:31:09.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.196+0000 7f1f64a00640 1 --2- 172.21.15.86:0/2552328029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f44004d40 0x7f1f440051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:31:09.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.197+0000 7f1f5fe00640 1 --2- 172.21.15.86:0/2552328029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f440039b0 0x7f1f44003dd0 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f1f5000c210 tx=0x7f1f44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d033f9a6c96e95 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:09.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.197+0000 7f1f5ea00640 1 -- 172.21.15.86:0/2552328029 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f50007d50 con 0x7f1f440039b0 2024-04-20T21:31:09.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.197+0000 7f1f5ea00640 1 -- 172.21.15.86:0/2552328029 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f50030070 con 0x7f1f440039b0 2024-04-20T21:31:09.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.197+0000 7f1f5ea00640 1 -- 172.21.15.86:0/2552328029 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5002cd80 con 0x7f1f440039b0 2024-04-20T21:31:09.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.198+0000 7f1f65400640 1 -- 172.21.15.86:0/2552328029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f440039b0 msgr2=0x7f1f44003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:09.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.198+0000 7f1f65400640 1 --2- 172.21.15.86:0/2552328029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f440039b0 0x7f1f44003dd0 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f1f5000c210 tx=0x7f1f44005830 comp rx=0 tx=0).stop 2024-04-20T21:31:09.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.198+0000 7f1f64a00640 1 -- 172.21.15.86:0/2552328029 reap_dead start 2024-04-20T21:31:09.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.198+0000 7f1f65400640 1 -- 172.21.15.86:0/2552328029 shutdown_connections 2024-04-20T21:31:09.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.198+0000 7f1f65400640 1 -- 172.21.15.86:0/2552328029 >> 172.21.15.86:0/2552328029 conn(0x7f1f6008bf70 msgr2=0x7f1f600d45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:09.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.198+0000 7f1f65400640 1 -- 172.21.15.86:0/2552328029 shutdown_connections 2024-04-20T21:31:09.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.198+0000 7f1f65400640 1 -- 172.21.15.86:0/2552328029 wait complete. 2024-04-20T21:31:09.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.200+0000 7f1f65400640 1 Processor -- start 2024-04-20T21:31:09.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.200+0000 7f1f65400640 1 -- start start 2024-04-20T21:31:09.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.201+0000 7f1f65400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f440039b0 0x7f1f600cb620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:09.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.201+0000 7f1f65400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f600ce200 0x7f1f601543a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:09.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.201+0000 7f1f65400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f601548e0 0x7f1f600b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:09.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.201+0000 7f1f65400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6006ce20 con 0x7f1f440039b0 2024-04-20T21:31:09.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.201+0000 7f1f65400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6006cb40 con 0x7f1f600ce200 2024-04-20T21:31:09.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.201+0000 7f1f65400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1f600b34c0 con 0x7f1f601548e0 2024-04-20T21:31:09.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.201+0000 7f1f5fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f440039b0 0x7f1f600cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:09.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.201+0000 7f1f5fe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f440039b0 0x7f1f600cb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33528/0 (socket says 172.21.15.86:33528) 2024-04-20T21:31:09.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.201+0000 7f1f5fe00640 1 -- 172.21.15.86:0/930167832 learned_addr learned my addr 172.21.15.86:0/930167832 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:31:09.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.202+0000 7f1f64a00640 1 --2- 172.21.15.86:0/930167832 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f601548e0 0x7f1f600b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:09.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.202+0000 7f1f5fe00640 1 -- 172.21.15.86:0/930167832 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f601548e0 msgr2=0x7f1f600b4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:09.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.202+0000 7f1f5f400640 1 --2- 172.21.15.86:0/930167832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f600ce200 0x7f1f601543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:09.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.202+0000 7f1f5fe00640 1 --2- 172.21.15.86:0/930167832 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1f601548e0 0x7f1f600b4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:09.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.202+0000 7f1f5fe00640 1 -- 172.21.15.86:0/930167832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f600ce200 msgr2=0x7f1f601543a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:09.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.202+0000 7f1f5fe00640 1 --2- 172.21.15.86:0/930167832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f600ce200 0x7f1f601543a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:09.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.202+0000 7f1f5fe00640 1 -- 172.21.15.86:0/930167832 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f600ce680 con 0x7f1f440039b0 2024-04-20T21:31:09.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.202+0000 7f1f5fe00640 1 --2- 172.21.15.86:0/930167832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f440039b0 0x7f1f600cb620 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7f1f5000c210 tx=0x7f1f50001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:09.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.202+0000 7f1f57e00640 1 -- 172.21.15.86:0/930167832 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f50007d50 con 0x7f1f440039b0 2024-04-20T21:31:09.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.202+0000 7f1f57e00640 1 -- 172.21.15.86:0/930167832 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f50004bf0 con 0x7f1f440039b0 2024-04-20T21:31:09.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.202+0000 7f1f65400640 1 -- 172.21.15.86:0/930167832 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f601f8150 con 0x7f1f440039b0 2024-04-20T21:31:09.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.202+0000 7f1f57e00640 1 -- 172.21.15.86:0/930167832 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f50001c90 con 0x7f1f440039b0 2024-04-20T21:31:09.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.203+0000 7f1f65400640 1 -- 172.21.15.86:0/930167832 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f601f8710 con 0x7f1f440039b0 2024-04-20T21:31:09.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.203+0000 7f1f65400640 1 -- 172.21.15.86:0/930167832 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1f6004fcf0 con 0x7f1f440039b0 2024-04-20T21:31:09.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.205+0000 7f1f57e00640 1 -- 172.21.15.86:0/930167832 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1f50047020 con 0x7f1f440039b0 2024-04-20T21:31:09.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.206+0000 7f1f57e00640 1 --2- 172.21.15.86:0/930167832 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1f4006c8f0 0x7f1f4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:09.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.206+0000 7f1f57e00640 1 -- 172.21.15.86:0/930167832 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f1f500ad3f0 con 0x7f1f440039b0 2024-04-20T21:31:09.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.206+0000 7f1f5f400640 1 --2- 172.21.15.86:0/930167832 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1f4006c8f0 0x7f1f4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:09.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.206+0000 7f1f57e00640 1 --2- 172.21.15.86:0/930167832 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x7f1f400728f0 0x7f1f40074d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:09.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.206+0000 7f1f57e00640 1 -- 172.21.15.86:0/930167832 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1f400753c0 con 0x7f1f400728f0 2024-04-20T21:31:09.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.206+0000 7f1f57e00640 1 -- 172.21.15.86:0/930167832 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=119) ==== 24+0+0 (secure 0 0 0) 0x7f1f500ad7a0 con 0x7f1f440039b0 2024-04-20T21:31:09.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.206+0000 7f1f64a00640 1 --2- 172.21.15.86:0/930167832 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x7f1f400728f0 0x7f1f40074d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:09.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.207+0000 7f1f5f400640 1 --2- 172.21.15.86:0/930167832 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1f4006c8f0 0x7f1f4006edb0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f1f60158d30 tx=0x7f1f4c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:09.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.207+0000 7f1f64a00640 1 --2- 172.21.15.86:0/930167832 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x7f1f400728f0 0x7f1f40074d10 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:09.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.208+0000 7f1f57e00640 1 -- 172.21.15.86:0/930167832 <== osd.0 v2:172.21.15.86:6800/2471076035 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1f400753c0 con 0x7f1f400728f0 2024-04-20T21:31:09.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.222+0000 7f1f65400640 1 -- 172.21.15.86:0/930167832 --> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f1f600b6cd0 con 0x7f1f400728f0 2024-04-20T21:31:09.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.223+0000 7f1f57e00640 1 -- 172.21.15.86:0/930167832 <== osd.0 v2:172.21.15.86:6800/2471076035 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f1f600b6cd0 con 0x7f1f400728f0 2024-04-20T21:31:09.221 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:31:09.221 INFO:teuthology.orchestra.run.smithi086.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T21:31:09.222 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:31:09.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.223+0000 7f1f65400640 1 -- 172.21.15.86:0/930167832 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x7f1f400728f0 msgr2=0x7f1f40074d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:09.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.223+0000 7f1f65400640 1 --2- 172.21.15.86:0/930167832 >> [v2:172.21.15.86:6800/2471076035,v1:172.21.15.86:6801/2471076035] conn(0x7f1f400728f0 0x7f1f40074d10 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:09.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.223+0000 7f1f65400640 1 -- 172.21.15.86:0/930167832 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1f4006c8f0 msgr2=0x7f1f4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:09.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.223+0000 7f1f65400640 1 --2- 172.21.15.86:0/930167832 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1f4006c8f0 0x7f1f4006edb0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f1f60158d30 tx=0x7f1f4c008040 comp rx=0 tx=0).stop 2024-04-20T21:31:09.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.223+0000 7f1f65400640 1 -- 172.21.15.86:0/930167832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f440039b0 msgr2=0x7f1f600cb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:09.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.223+0000 7f1f65400640 1 --2- 172.21.15.86:0/930167832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f440039b0 0x7f1f600cb620 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7f1f5000c210 tx=0x7f1f50001a40 comp rx=0 tx=0).stop 2024-04-20T21:31:09.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.224+0000 7f1f64a00640 1 -- 172.21.15.86:0/930167832 reap_dead start 2024-04-20T21:31:09.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.224+0000 7f1f65400640 1 -- 172.21.15.86:0/930167832 shutdown_connections 2024-04-20T21:31:09.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.224+0000 7f1f65400640 1 -- 172.21.15.86:0/930167832 >> 172.21.15.86:0/930167832 conn(0x7f1f6008bf70 msgr2=0x7f1f600b2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:09.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.224+0000 7f1f65400640 1 -- 172.21.15.86:0/930167832 shutdown_connections 2024-04-20T21:31:09.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.224+0000 7f1f65400640 1 -- 172.21.15.86:0/930167832 wait complete. 2024-04-20T21:31:09.239 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-20T21:31:09.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.441+0000 7f6c2ce00640 1 Processor -- start 2024-04-20T21:31:09.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.441+0000 7f6c2ce00640 1 -- start start 2024-04-20T21:31:09.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.443+0000 7f6c2ce00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6c28145fe0 0x7f6c280d02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:09.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.443+0000 7f6c2ce00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c280d0810 0x7f6c280cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:09.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.443+0000 7f6c2ce00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c2815dcc0 con 0x7f6c280cd610 2024-04-20T21:31:09.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.443+0000 7f6c2ce00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f6c280d00d0 con 0x7f6c28145fe0 2024-04-20T21:31:09.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.443+0000 7f6c2ce00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6c280d0c70 con 0x7f6c280d0810 2024-04-20T21:31:09.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.444+0000 7f6c25400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c280d0810 0x7f6c280cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:09.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.444+0000 7f6c25e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f6c28145fe0 0x7f6c280d02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:09.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.444+0000 7f6c25400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c280d0810 0x7f6c280cd0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:49246/0 (socket says 172.21.15.86:49246) 2024-04-20T21:31:09.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.444+0000 7f6c25400640 1 -- 172.21.15.86:0/3606576839 learned_addr learned my addr 172.21.15.86:0/3606576839 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:31:09.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.445+0000 7f6c25400640 1 -- 172.21.15.86:0/3606576839 >> v2:172.21.15.144:3300/0 conn(0x7f6c28145fe0 msgr2=0x7f6c280d02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:09.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.445+0000 7f6c25400640 1 --2- 172.21.15.86:0/3606576839 >> v2:172.21.15.144:3300/0 conn(0x7f6c28145fe0 0x7f6c280d02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:09.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.445+0000 7f6c25400640 1 -- 172.21.15.86:0/3606576839 >> v1:172.21.15.144:6789/0 conn(0x7f6c280cd610 legacy=0x7f6c28160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:09.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.445+0000 7f6c25400640 1 -- 172.21.15.86:0/3606576839 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c2815cf00 con 0x7f6c280d0810 2024-04-20T21:31:09.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.446+0000 7f6c25400640 1 --2- 172.21.15.86:0/3606576839 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c280d0810 0x7f6c280cd0d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6c10009870 tx=0x7f6c1002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57e0c2c36aa3b7c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:09.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.447+0000 7f6c24a00640 1 -- 172.21.15.86:0/3606576839 <== mon.2 v2:172.21.15.144:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c10034070 con 0x7f6c280d0810 2024-04-20T21:31:09.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.447+0000 7f6c24a00640 1 -- 172.21.15.86:0/3606576839 <== mon.2 v2:172.21.15.144:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c10030840 con 0x7f6c280d0810 2024-04-20T21:31:09.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.447+0000 7f6c24a00640 1 -- 172.21.15.86:0/3606576839 <== mon.2 v2:172.21.15.144:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c10030c10 con 0x7f6c280d0810 2024-04-20T21:31:09.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.447+0000 7f6c2ce00640 1 -- 172.21.15.86:0/3606576839 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c280d0810 msgr2=0x7f6c280cd0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:09.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.447+0000 7f6c2ce00640 1 --2- 172.21.15.86:0/3606576839 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c280d0810 0x7f6c280cd0d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6c10009870 tx=0x7f6c1002b980 comp rx=0 tx=0).stop 2024-04-20T21:31:09.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.447+0000 7f6c2ce00640 1 -- 172.21.15.86:0/3606576839 shutdown_connections 2024-04-20T21:31:09.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.447+0000 7f6c2ce00640 1 --2- 172.21.15.86:0/3606576839 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c280d0810 0x7f6c280cd0d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:09.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.447+0000 7f6c2ce00640 1 --2- 172.21.15.86:0/3606576839 >> v2:172.21.15.144:3300/0 conn(0x7f6c28145fe0 0x7f6c280d02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:09.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.447+0000 7f6c2ce00640 1 -- 172.21.15.86:0/3606576839 >> 172.21.15.86:0/3606576839 conn(0x7f6c2806b980 msgr2=0x7f6c2806bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:09.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.448+0000 7f6c2ce00640 1 -- 172.21.15.86:0/3606576839 shutdown_connections 2024-04-20T21:31:09.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.448+0000 7f6c2ce00640 1 -- 172.21.15.86:0/3606576839 wait complete. 2024-04-20T21:31:09.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.449+0000 7f6c2ce00640 1 Processor -- start 2024-04-20T21:31:09.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.449+0000 7f6c2ce00640 1 -- start start 2024-04-20T21:31:09.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.450+0000 7f6c2ce00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c280cd610 0x7f6c280b1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:09.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.450+0000 7f6c2ce00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c280d0810 0x7f6c281e8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:09.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.450+0000 7f6c2ce00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c28145fe0 0x7f6c281e7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:09.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.450+0000 7f6c2ce00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6c280cdaa0 con 0x7f6c28145fe0 2024-04-20T21:31:09.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.450+0000 7f6c2ce00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2815d4c0 con 0x7f6c280d0810 2024-04-20T21:31:09.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.450+0000 7f6c2ce00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6c2815d7a0 con 0x7f6c280cd610 2024-04-20T21:31:09.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.451+0000 7f6c26800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c28145fe0 0x7f6c281e7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:09.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.451+0000 7f6c26800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c28145fe0 0x7f6c281e7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33538/0 (socket says 172.21.15.86:33538) 2024-04-20T21:31:09.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.451+0000 7f6c26800640 1 -- 172.21.15.86:0/2452600140 learned_addr learned my addr 172.21.15.86:0/2452600140 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:31:09.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.451+0000 7f6c25e00640 1 --2- 172.21.15.86:0/2452600140 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c280cd610 0x7f6c280b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:09.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.451+0000 7f6c25400640 1 --2- 172.21.15.86:0/2452600140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c280d0810 0x7f6c281e8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:09.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.451+0000 7f6c26800640 1 -- 172.21.15.86:0/2452600140 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c280cd610 msgr2=0x7f6c280b1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:09.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.451+0000 7f6c26800640 1 --2- 172.21.15.86:0/2452600140 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c280cd610 0x7f6c280b1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:09.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.451+0000 7f6c26800640 1 -- 172.21.15.86:0/2452600140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c280d0810 msgr2=0x7f6c281e8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:09.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.451+0000 7f6c26800640 1 --2- 172.21.15.86:0/2452600140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c280d0810 0x7f6c281e8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:09.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.451+0000 7f6c26800640 1 -- 172.21.15.86:0/2452600140 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c281e7e90 con 0x7f6c28145fe0 2024-04-20T21:31:09.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.451+0000 7f6c26800640 1 --2- 172.21.15.86:0/2452600140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c28145fe0 0x7f6c281e7440 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7f6c20001eb0 tx=0x7f6c2000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:09.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.452+0000 7f6c1e000640 1 -- 172.21.15.86:0/2452600140 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c20017070 con 0x7f6c28145fe0 2024-04-20T21:31:09.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.452+0000 7f6c1e000640 1 -- 172.21.15.86:0/2452600140 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c200052d0 con 0x7f6c28145fe0 2024-04-20T21:31:09.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.452+0000 7f6c1e000640 1 -- 172.21.15.86:0/2452600140 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c20003810 con 0x7f6c28145fe0 2024-04-20T21:31:09.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.452+0000 7f6c2ce00640 1 -- 172.21.15.86:0/2452600140 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c281e8170 con 0x7f6c28145fe0 2024-04-20T21:31:09.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.452+0000 7f6c2ce00640 1 -- 172.21.15.86:0/2452600140 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c281e8730 con 0x7f6c28145fe0 2024-04-20T21:31:09.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.453+0000 7f6c2ce00640 1 -- 172.21.15.86:0/2452600140 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c280689e0 con 0x7f6c28145fe0 2024-04-20T21:31:09.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.454+0000 7f6c1e000640 1 -- 172.21.15.86:0/2452600140 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6c2001a070 con 0x7f6c28145fe0 2024-04-20T21:31:09.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.455+0000 7f6c1e000640 1 --2- 172.21.15.86:0/2452600140 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6c0406c8f0 0x7f6c0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:09.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.455+0000 7f6c1e000640 1 -- 172.21.15.86:0/2452600140 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f6c2008ad30 con 0x7f6c28145fe0 2024-04-20T21:31:09.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.455+0000 7f6c25e00640 1 --2- 172.21.15.86:0/2452600140 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6c0406c8f0 0x7f6c0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:09.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.456+0000 7f6c25e00640 1 --2- 172.21.15.86:0/2452600140 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6c0406c8f0 0x7f6c0406edb0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f6c18007380 tx=0x7f6c1800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:09.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.459+0000 7f6c1e000640 1 -- 172.21.15.86:0/2452600140 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6c20059310 con 0x7f6c28145fe0 2024-04-20T21:31:09.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.660+0000 7f6c2ce00640 1 -- 172.21.15.86:0/2452600140 --> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f6c280d1020 con 0x7f6c0406c8f0 2024-04-20T21:31:09.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.661+0000 7f6c1e000640 1 -- 172.21.15.86:0/2452600140 <== mgr.4107 v2:172.21.15.86:6832/302045240 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f6c280d1020 con 0x7f6c0406c8f0 2024-04-20T21:31:09.660 INFO:teuthology.orchestra.run.smithi086.stderr:instructed osd(s) 0 to deep-scrub 2024-04-20T21:31:09.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.664+0000 7f6c2ce00640 1 -- 172.21.15.86:0/2452600140 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6c0406c8f0 msgr2=0x7f6c0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:09.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.664+0000 7f6c2ce00640 1 --2- 172.21.15.86:0/2452600140 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6c0406c8f0 0x7f6c0406edb0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f6c18007380 tx=0x7f6c1800b040 comp rx=0 tx=0).stop 2024-04-20T21:31:09.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.664+0000 7f6c2ce00640 1 -- 172.21.15.86:0/2452600140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c28145fe0 msgr2=0x7f6c281e7440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:09.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.664+0000 7f6c2ce00640 1 --2- 172.21.15.86:0/2452600140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c28145fe0 0x7f6c281e7440 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7f6c20001eb0 tx=0x7f6c2000c890 comp rx=0 tx=0).stop 2024-04-20T21:31:09.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.664+0000 7f6c2ce00640 1 -- 172.21.15.86:0/2452600140 shutdown_connections 2024-04-20T21:31:09.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.664+0000 7f6c2ce00640 1 --2- 172.21.15.86:0/2452600140 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6c0406c8f0 0x7f6c0406edb0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:09.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.664+0000 7f6c2ce00640 1 --2- 172.21.15.86:0/2452600140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c28145fe0 0x7f6c281e7440 unknown :-1 s=CLOSED pgs=2146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:09.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.664+0000 7f6c2ce00640 1 --2- 172.21.15.86:0/2452600140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c280d0810 0x7f6c281e8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:09.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.664+0000 7f6c2ce00640 1 --2- 172.21.15.86:0/2452600140 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6c280cd610 0x7f6c280b1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:09.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.664+0000 7f6c2ce00640 1 -- 172.21.15.86:0/2452600140 >> 172.21.15.86:0/2452600140 conn(0x7f6c2806b980 msgr2=0x7f6c280aeb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:09.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.664+0000 7f6c2ce00640 1 -- 172.21.15.86:0/2452600140 shutdown_connections 2024-04-20T21:31:09.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.664+0000 7f6c2ce00640 1 -- 172.21.15.86:0/2452600140 wait complete. 2024-04-20T21:31:09.682 INFO:tasks.ceph:Scrubbing osd.1 2024-04-20T21:31:09.682 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-20T21:31:09.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.875+0000 7f3f3b200640 1 Processor -- start 2024-04-20T21:31:09.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.876+0000 7f3f3b200640 1 -- start start 2024-04-20T21:31:09.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.877+0000 7f3f3b200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3f3c0b6770 0x7f3f3c0cee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:09.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.877+0000 7f3f3b200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f3c1622d0 con 0x7f3f3c161ef0 2024-04-20T21:31:09.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.877+0000 7f3f3b200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f3c0b3e20 con 0x7f3f3c06d340 2024-04-20T21:31:09.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.877+0000 7f3f3b200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f3f3c0d4f70 con 0x7f3f3c0b6770 2024-04-20T21:31:09.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.877+0000 7f3f39e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f3f3c161ef0 0x7f3f3c0b6060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:53102/0 (socket says 172.21.15.86:53102) 2024-04-20T21:31:09.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.877+0000 7f3f39400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3f3c0b6770 0x7f3f3c0cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:09.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.877+0000 7f3f39e00640 1 -- 172.21.15.86:0/122775597 learned_addr learned my addr 172.21.15.86:0/122775597 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:31:09.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.878+0000 7f3f39400640 1 -- 172.21.15.86:0/122775597 >> v1:172.21.15.144:6789/0 conn(0x7f3f3c06d340 legacy=0x7f3f3c06d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:09.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.878+0000 7f3f39400640 1 -- 172.21.15.86:0/122775597 >> v1:172.21.15.86:6789/0 conn(0x7f3f3c161ef0 legacy=0x7f3f3c0b6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:09.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.878+0000 7f3f38a00640 1 -- 172.21.15.86:0/122775597 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1766717294 0 0) 0x7f3f3c1622d0 con 0x7f3f3c161ef0 2024-04-20T21:31:09.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.878+0000 7f3f39400640 1 -- 172.21.15.86:0/122775597 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f3c0b3860 con 0x7f3f3c0b6770 2024-04-20T21:31:09.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.879+0000 7f3f39400640 1 --2- 172.21.15.86:0/122775597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f3c0b6770 0x7f3f3c0cee60 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7f3f24009790 tx=0x7f3f2402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b7fd1a6fb3cc5c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:09.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.879+0000 7f3f38a00640 1 -- 172.21.15.86:0/122775597 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f24035020 con 0x7f3f3c0b6770 2024-04-20T21:31:09.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.880+0000 7f3f38a00640 1 -- 172.21.15.86:0/122775597 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f24031830 con 0x7f3f3c0b6770 2024-04-20T21:31:09.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.880+0000 7f3f38a00640 1 -- 172.21.15.86:0/122775597 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f24031c00 con 0x7f3f3c0b6770 2024-04-20T21:31:09.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.880+0000 7f3f3b200640 1 -- 172.21.15.86:0/122775597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f3c0b6770 msgr2=0x7f3f3c0cee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:09.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.880+0000 7f3f3b200640 1 --2- 172.21.15.86:0/122775597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f3c0b6770 0x7f3f3c0cee60 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7f3f24009790 tx=0x7f3f2402b8d0 comp rx=0 tx=0).stop 2024-04-20T21:31:09.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.880+0000 7f3f3b200640 1 -- 172.21.15.86:0/122775597 shutdown_connections 2024-04-20T21:31:09.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.880+0000 7f3f3b200640 1 --2- 172.21.15.86:0/122775597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f3c0b6770 0x7f3f3c0cee60 unknown :-1 s=CLOSED pgs=2148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:09.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.880+0000 7f3f3b200640 1 -- 172.21.15.86:0/122775597 >> 172.21.15.86:0/122775597 conn(0x7f3f3c08bf70 msgr2=0x7f3f3c0d45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:09.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.880+0000 7f3f3b200640 1 -- 172.21.15.86:0/122775597 shutdown_connections 2024-04-20T21:31:09.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.880+0000 7f3f3b200640 1 -- 172.21.15.86:0/122775597 wait complete. 2024-04-20T21:31:09.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.882+0000 7f3f3b200640 1 Processor -- start 2024-04-20T21:31:09.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.882+0000 7f3f3b200640 1 -- start start 2024-04-20T21:31:09.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.883+0000 7f3f3b200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f3c06d340 0x7f3f3c154d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:09.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.883+0000 7f3f3b200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f3c0b6770 0x7f3f3c0b7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:09.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.883+0000 7f3f3b200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f3c161ef0 0x7f3f3c15dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:09.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.883+0000 7f3f3b200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3c06ce20 con 0x7f3f3c161ef0 2024-04-20T21:31:09.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.883+0000 7f3f3b200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3c06cb40 con 0x7f3f3c06d340 2024-04-20T21:31:09.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.883+0000 7f3f3b200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f3f3c0b34c0 con 0x7f3f3c0b6770 2024-04-20T21:31:09.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.884+0000 7f3f3a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f3c161ef0 0x7f3f3c15dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:09.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.884+0000 7f3f3a800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f3c161ef0 0x7f3f3c15dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33564/0 (socket says 172.21.15.86:33564) 2024-04-20T21:31:09.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.884+0000 7f3f3a800640 1 -- 172.21.15.86:0/1612429388 learned_addr learned my addr 172.21.15.86:0/1612429388 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:31:09.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.884+0000 7f3f39e00640 1 --2- 172.21.15.86:0/1612429388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f3c06d340 0x7f3f3c154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:09.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.884+0000 7f3f39400640 1 --2- 172.21.15.86:0/1612429388 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f3c0b6770 0x7f3f3c0b7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:09.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.884+0000 7f3f3a800640 1 -- 172.21.15.86:0/1612429388 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f3c0b6770 msgr2=0x7f3f3c0b7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:09.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.884+0000 7f3f3a800640 1 --2- 172.21.15.86:0/1612429388 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f3f3c0b6770 0x7f3f3c0b7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:09.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.884+0000 7f3f3a800640 1 -- 172.21.15.86:0/1612429388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f3c06d340 msgr2=0x7f3f3c154d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:09.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.884+0000 7f3f3a800640 1 --2- 172.21.15.86:0/1612429388 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f3c06d340 0x7f3f3c154d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:09.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.884+0000 7f3f3a800640 1 -- 172.21.15.86:0/1612429388 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f3c1f8040 con 0x7f3f3c161ef0 2024-04-20T21:31:09.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.885+0000 7f3f3a800640 1 --2- 172.21.15.86:0/1612429388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f3c161ef0 0x7f3f3c15dcb0 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7f3f3400c440 tx=0x7f3f3400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:09.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.885+0000 7f3f32000640 1 -- 172.21.15.86:0/1612429388 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f34011790 con 0x7f3f3c161ef0 2024-04-20T21:31:09.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.885+0000 7f3f32000640 1 -- 172.21.15.86:0/1612429388 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f34017070 con 0x7f3f3c161ef0 2024-04-20T21:31:09.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.885+0000 7f3f3b200640 1 -- 172.21.15.86:0/1612429388 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f3c1f8320 con 0x7f3f3c161ef0 2024-04-20T21:31:09.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.885+0000 7f3f32000640 1 -- 172.21.15.86:0/1612429388 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f34011b70 con 0x7f3f3c161ef0 2024-04-20T21:31:09.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.885+0000 7f3f3b200640 1 -- 172.21.15.86:0/1612429388 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f3c1f88e0 con 0x7f3f3c161ef0 2024-04-20T21:31:09.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.886+0000 7f3f3b200640 1 -- 172.21.15.86:0/1612429388 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3f3c04fcf0 con 0x7f3f3c161ef0 2024-04-20T21:31:09.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.888+0000 7f3f32000640 1 -- 172.21.15.86:0/1612429388 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3f34011d10 con 0x7f3f3c161ef0 2024-04-20T21:31:09.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.888+0000 7f3f32000640 1 --2- 172.21.15.86:0/1612429388 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3f1806c920 0x7f3f1806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:09.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.889+0000 7f3f32000640 1 -- 172.21.15.86:0/1612429388 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f3f3408c060 con 0x7f3f3c161ef0 2024-04-20T21:31:09.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.889+0000 7f3f39e00640 1 --2- 172.21.15.86:0/1612429388 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3f1806c920 0x7f3f1806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:09.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.889+0000 7f3f32000640 1 --2- 172.21.15.86:0/1612429388 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x7f3f18072920 0x7f3f18074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:09.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.889+0000 7f3f32000640 1 -- 172.21.15.86:0/1612429388 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3f180753f0 con 0x7f3f18072920 2024-04-20T21:31:09.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.889+0000 7f3f32000640 1 -- 172.21.15.86:0/1612429388 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=119) ==== 24+0+0 (secure 0 0 0) 0x7f3f3408c410 con 0x7f3f3c161ef0 2024-04-20T21:31:09.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.889+0000 7f3f39400640 1 --2- 172.21.15.86:0/1612429388 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x7f3f18072920 0x7f3f18074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:09.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.890+0000 7f3f39e00640 1 --2- 172.21.15.86:0/1612429388 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3f1806c920 0x7f3f1806ede0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c007ed0 tx=0x7f3f2c007e60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:09.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.890+0000 7f3f39400640 1 --2- 172.21.15.86:0/1612429388 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x7f3f18072920 0x7f3f18074d40 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:09.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.891+0000 7f3f32000640 1 -- 172.21.15.86:0/1612429388 <== osd.1 v2:172.21.15.86:6817/3149564961 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3f180753f0 con 0x7f3f18072920 2024-04-20T21:31:09.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.905+0000 7f3f3b200640 1 -- 172.21.15.86:0/1612429388 --> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f3f3c0b6cd0 con 0x7f3f18072920 2024-04-20T21:31:09.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.905+0000 7f3f32000640 1 -- 172.21.15.86:0/1612429388 <== osd.1 v2:172.21.15.86:6817/3149564961 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f3f3c0b6cd0 con 0x7f3f18072920 2024-04-20T21:31:09.904 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:31:09.904 INFO:teuthology.orchestra.run.smithi086.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T21:31:09.904 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:31:09.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.906+0000 7f3f3b200640 1 -- 172.21.15.86:0/1612429388 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x7f3f18072920 msgr2=0x7f3f18074d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:09.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.906+0000 7f3f3b200640 1 --2- 172.21.15.86:0/1612429388 >> [v2:172.21.15.86:6817/3149564961,v1:172.21.15.86:6819/3149564961] conn(0x7f3f18072920 0x7f3f18074d40 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:09.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.906+0000 7f3f3b200640 1 -- 172.21.15.86:0/1612429388 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3f1806c920 msgr2=0x7f3f1806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:09.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.906+0000 7f3f3b200640 1 --2- 172.21.15.86:0/1612429388 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f3f1806c920 0x7f3f1806ede0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c007ed0 tx=0x7f3f2c007e60 comp rx=0 tx=0).stop 2024-04-20T21:31:09.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.906+0000 7f3f3b200640 1 -- 172.21.15.86:0/1612429388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f3c161ef0 msgr2=0x7f3f3c15dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:09.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.906+0000 7f3f3b200640 1 --2- 172.21.15.86:0/1612429388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3f3c161ef0 0x7f3f3c15dcb0 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7f3f3400c440 tx=0x7f3f3400c900 comp rx=0 tx=0).stop 2024-04-20T21:31:09.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.906+0000 7f3f3a800640 1 -- 172.21.15.86:0/1612429388 reap_dead start 2024-04-20T21:31:09.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.906+0000 7f3f3b200640 1 -- 172.21.15.86:0/1612429388 shutdown_connections 2024-04-20T21:31:09.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.906+0000 7f3f3b200640 1 -- 172.21.15.86:0/1612429388 >> 172.21.15.86:0/1612429388 conn(0x7f3f3c08bf70 msgr2=0x7f3f3c06d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:09.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.906+0000 7f3f3b200640 1 -- 172.21.15.86:0/1612429388 shutdown_connections 2024-04-20T21:31:09.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:09.906+0000 7f3f3b200640 1 -- 172.21.15.86:0/1612429388 wait complete. 2024-04-20T21:31:09.922 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-20T21:31:10.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.112+0000 7ff4a8200640 1 Processor -- start 2024-04-20T21:31:10.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.113+0000 7ff4a8200640 1 -- start start 2024-04-20T21:31:10.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.114+0000 7ff4a8200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4a006ef20 0x7ff4a00b4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:10.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.114+0000 7ff4a8200640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4a00b3cc0 con 0x7ff4a00723e0 2024-04-20T21:31:10.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.114+0000 7ff4a8200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4a00b3f60 con 0x7ff4a015ad50 2024-04-20T21:31:10.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.114+0000 7ff4a8200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff4a00b4240 con 0x7ff4a006ef20 2024-04-20T21:31:10.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.115+0000 7ff4a6200640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7ff4a00723e0 0x7ff4a00cd8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:53116/0 (socket says 172.21.15.86:53116) 2024-04-20T21:31:10.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.115+0000 7ff4a6200640 1 -- 172.21.15.86:0/2988104277 learned_addr learned my addr 172.21.15.86:0/2988104277 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:31:10.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.115+0000 7ff4a4e00640 1 --2- 172.21.15.86:0/2988104277 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4a006ef20 0x7ff4a00b4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:10.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.115+0000 7ff49fe00640 1 -- 172.21.15.86:0/2988104277 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2288437651 0 0) 0x7ff4a00b3cc0 con 0x7ff4a00723e0 2024-04-20T21:31:10.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.115+0000 7ff49fe00640 1 -- 172.21.15.86:0/2988104277 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff488003520 con 0x7ff4a00723e0 2024-04-20T21:31:10.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.115+0000 7ff49fe00640 1 -- 172.21.15.86:0/2988104277 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3503798360 0 0) 0x7ff488003520 con 0x7ff4a00723e0 2024-04-20T21:31:10.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.115+0000 7ff49fe00640 1 -- 172.21.15.86:0/2988104277 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4a006ef20 msgr2=0x7ff4a00b4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:10.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.115+0000 7ff49fe00640 1 --2- 172.21.15.86:0/2988104277 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4a006ef20 0x7ff4a00b4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:10.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.115+0000 7ff49fe00640 1 -- 172.21.15.86:0/2988104277 >> v1:172.21.15.144:6789/0 conn(0x7ff4a015ad50 legacy=0x7ff4a0071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:10.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.115+0000 7ff49fe00640 1 -- 172.21.15.86:0/2988104277 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4a015f860 con 0x7ff4a00723e0 2024-04-20T21:31:10.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.115+0000 7ff49fe00640 1 -- 172.21.15.86:0/2988104277 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7ff498001a10 con 0x7ff4a00723e0 2024-04-20T21:31:10.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.115+0000 7ff49fe00640 1 -- 172.21.15.86:0/2988104277 >> v1:172.21.15.86:6789/0 conn(0x7ff4a00723e0 legacy=0x7ff4a00cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:10.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.116+0000 7ff49fe00640 1 --2- 172.21.15.86:0/2988104277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4880039e0 0x7ff488003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:10.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.116+0000 7ff49fe00640 1 --2- 172.21.15.86:0/2988104277 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4a006ef20 0x7ff488004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:10.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.116+0000 7ff49fe00640 1 --2- 172.21.15.86:0/2988104277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff488004d40 0x7ff4880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:10.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.116+0000 7ff49fe00640 1 -- 172.21.15.86:0/2988104277 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a00b4240 con 0x7ff4880039e0 2024-04-20T21:31:10.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.116+0000 7ff49fe00640 1 -- 172.21.15.86:0/2988104277 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff48800dc80 con 0x7ff488004d40 2024-04-20T21:31:10.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.116+0000 7ff49fe00640 1 -- 172.21.15.86:0/2988104277 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff48800de80 con 0x7ff4a006ef20 2024-04-20T21:31:10.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.116+0000 7ff4a5800640 1 --2- 172.21.15.86:0/2988104277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4880039e0 0x7ff488003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:10.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.116+0000 7ff4a4e00640 1 --2- 172.21.15.86:0/2988104277 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4a006ef20 0x7ff488004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:10.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.116+0000 7ff4a6200640 1 --2- 172.21.15.86:0/2988104277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff488004d40 0x7ff4880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:10.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.116+0000 7ff4a5800640 1 -- 172.21.15.86:0/2988104277 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4a006ef20 msgr2=0x7ff488004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:10.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.116+0000 7ff4a5800640 1 --2- 172.21.15.86:0/2988104277 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4a006ef20 0x7ff488004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:10.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.116+0000 7ff4a5800640 1 -- 172.21.15.86:0/2988104277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff488004d40 msgr2=0x7ff4880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:10.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.116+0000 7ff4a5800640 1 --2- 172.21.15.86:0/2988104277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff488004d40 0x7ff4880051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:10.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.116+0000 7ff4a5800640 1 -- 172.21.15.86:0/2988104277 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff488003520 con 0x7ff4880039e0 2024-04-20T21:31:10.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.117+0000 7ff4a5800640 1 --2- 172.21.15.86:0/2988104277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4880039e0 0x7ff488003e00 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7ff494001720 tx=0x7ff488005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e135c21f9bf5e83 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:10.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.118+0000 7ff49fe00640 1 -- 172.21.15.86:0/2988104277 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff494001370 con 0x7ff4880039e0 2024-04-20T21:31:10.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.118+0000 7ff49fe00640 1 -- 172.21.15.86:0/2988104277 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4940052d0 con 0x7ff4880039e0 2024-04-20T21:31:10.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.118+0000 7ff49fe00640 1 -- 172.21.15.86:0/2988104277 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff494003830 con 0x7ff4880039e0 2024-04-20T21:31:10.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.118+0000 7ff4a8200640 1 -- 172.21.15.86:0/2988104277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4880039e0 msgr2=0x7ff488003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:10.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.118+0000 7ff4a8200640 1 --2- 172.21.15.86:0/2988104277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4880039e0 0x7ff488003e00 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7ff494001720 tx=0x7ff488005830 comp rx=0 tx=0).stop 2024-04-20T21:31:10.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.118+0000 7ff4a6200640 1 -- 172.21.15.86:0/2988104277 reap_dead start 2024-04-20T21:31:10.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.118+0000 7ff4a8200640 1 -- 172.21.15.86:0/2988104277 shutdown_connections 2024-04-20T21:31:10.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.118+0000 7ff4a8200640 1 -- 172.21.15.86:0/2988104277 >> 172.21.15.86:0/2988104277 conn(0x7ff4a006b980 msgr2=0x7ff4a006bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:10.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.119+0000 7ff4a8200640 1 -- 172.21.15.86:0/2988104277 shutdown_connections 2024-04-20T21:31:10.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.119+0000 7ff4a8200640 1 -- 172.21.15.86:0/2988104277 wait complete. 2024-04-20T21:31:10.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.120+0000 7ff4a8200640 1 Processor -- start 2024-04-20T21:31:10.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.120+0000 7ff4a8200640 1 -- start start 2024-04-20T21:31:10.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.121+0000 7ff4a8200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4880039e0 0x7ff4a00b2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:10.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.121+0000 7ff4a8200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4a0150d70 0x7ff4a01511d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:10.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.121+0000 7ff4a8200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4a00b0e80 0x7ff4a00b1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:10.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.121+0000 7ff4a8200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a015f520 con 0x7ff4a0150d70 2024-04-20T21:31:10.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.121+0000 7ff4a8200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a015f3a0 con 0x7ff4a00b0e80 2024-04-20T21:31:10.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.121+0000 7ff4a8200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff4a006f370 con 0x7ff4880039e0 2024-04-20T21:31:10.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.122+0000 7ff4a4e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4a0150d70 0x7ff4a01511d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:10.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.122+0000 7ff4a4e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4a0150d70 0x7ff4a01511d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33592/0 (socket says 172.21.15.86:33592) 2024-04-20T21:31:10.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.122+0000 7ff4a4e00640 1 -- 172.21.15.86:0/851407066 learned_addr learned my addr 172.21.15.86:0/851407066 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:31:10.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.122+0000 7ff4a5800640 1 --2- 172.21.15.86:0/851407066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4880039e0 0x7ff4a00b2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:10.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.122+0000 7ff4a4e00640 1 -- 172.21.15.86:0/851407066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4880039e0 msgr2=0x7ff4a00b2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:10.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.122+0000 7ff4a6200640 1 --2- 172.21.15.86:0/851407066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4a00b0e80 0x7ff4a00b1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:10.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.122+0000 7ff4a4e00640 1 --2- 172.21.15.86:0/851407066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4880039e0 0x7ff4a00b2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:10.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.122+0000 7ff4a4e00640 1 -- 172.21.15.86:0/851407066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4a00b0e80 msgr2=0x7ff4a00b1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:10.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.122+0000 7ff4a4e00640 1 --2- 172.21.15.86:0/851407066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4a00b0e80 0x7ff4a00b1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:10.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.122+0000 7ff4a4e00640 1 -- 172.21.15.86:0/851407066 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4a00c53d0 con 0x7ff4a0150d70 2024-04-20T21:31:10.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.122+0000 7ff4a4e00640 1 --2- 172.21.15.86:0/851407066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4a0150d70 0x7ff4a01511d0 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7ff48c011630 tx=0x7ff48c011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:10.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.123+0000 7ff49d600640 1 -- 172.21.15.86:0/851407066 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48c00f920 con 0x7ff4a0150d70 2024-04-20T21:31:10.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.123+0000 7ff49d600640 1 -- 172.21.15.86:0/851407066 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff48c00fac0 con 0x7ff4a0150d70 2024-04-20T21:31:10.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.123+0000 7ff49d600640 1 -- 172.21.15.86:0/851407066 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48c00fe70 con 0x7ff4a0150d70 2024-04-20T21:31:10.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.123+0000 7ff4a8200640 1 -- 172.21.15.86:0/851407066 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4a00c5670 con 0x7ff4a0150d70 2024-04-20T21:31:10.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.123+0000 7ff4a8200640 1 -- 172.21.15.86:0/851407066 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4a00c5c30 con 0x7ff4a0150d70 2024-04-20T21:31:10.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.124+0000 7ff4a8200640 1 -- 172.21.15.86:0/851407066 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4a00689e0 con 0x7ff4a0150d70 2024-04-20T21:31:10.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.125+0000 7ff49d600640 1 -- 172.21.15.86:0/851407066 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff48c015460 con 0x7ff4a0150d70 2024-04-20T21:31:10.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.126+0000 7ff49d600640 1 --2- 172.21.15.86:0/851407066 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff48006c8f0 0x7ff48006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:10.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.126+0000 7ff49d600640 1 -- 172.21.15.86:0/851407066 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7ff48c08bd70 con 0x7ff4a0150d70 2024-04-20T21:31:10.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.126+0000 7ff4a5800640 1 --2- 172.21.15.86:0/851407066 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff48006c8f0 0x7ff48006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:10.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.127+0000 7ff4a5800640 1 --2- 172.21.15.86:0/851407066 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff48006c8f0 0x7ff48006edb0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7ff494033000 tx=0x7ff494009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:10.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.130+0000 7ff49d600640 1 -- 172.21.15.86:0/851407066 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff48c019030 con 0x7ff4a0150d70 2024-04-20T21:31:10.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.329+0000 7ff4a8200640 1 -- 172.21.15.86:0/851407066 --> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7ff4a00721c0 con 0x7ff48006c8f0 2024-04-20T21:31:10.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.330+0000 7ff49d600640 1 -- 172.21.15.86:0/851407066 <== mgr.4107 v2:172.21.15.86:6832/302045240 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ff4a00721c0 con 0x7ff48006c8f0 2024-04-20T21:31:10.328 INFO:teuthology.orchestra.run.smithi086.stderr:instructed osd(s) 1 to deep-scrub 2024-04-20T21:31:10.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.332+0000 7ff4a8200640 1 -- 172.21.15.86:0/851407066 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff48006c8f0 msgr2=0x7ff48006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:10.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.332+0000 7ff4a8200640 1 --2- 172.21.15.86:0/851407066 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff48006c8f0 0x7ff48006edb0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7ff494033000 tx=0x7ff494009030 comp rx=0 tx=0).stop 2024-04-20T21:31:10.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.332+0000 7ff4a8200640 1 -- 172.21.15.86:0/851407066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4a0150d70 msgr2=0x7ff4a01511d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:10.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.332+0000 7ff4a8200640 1 --2- 172.21.15.86:0/851407066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4a0150d70 0x7ff4a01511d0 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7ff48c011630 tx=0x7ff48c011af0 comp rx=0 tx=0).stop 2024-04-20T21:31:10.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.333+0000 7ff4a8200640 1 -- 172.21.15.86:0/851407066 shutdown_connections 2024-04-20T21:31:10.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.333+0000 7ff4a8200640 1 --2- 172.21.15.86:0/851407066 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff48006c8f0 0x7ff48006edb0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:10.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.333+0000 7ff4a8200640 1 --2- 172.21.15.86:0/851407066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff4a00b0e80 0x7ff4a00b1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:10.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.333+0000 7ff4a8200640 1 --2- 172.21.15.86:0/851407066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4a0150d70 0x7ff4a01511d0 unknown :-1 s=CLOSED pgs=2152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:10.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.333+0000 7ff4a8200640 1 --2- 172.21.15.86:0/851407066 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff4880039e0 0x7ff4a00b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:10.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.333+0000 7ff4a8200640 1 -- 172.21.15.86:0/851407066 >> 172.21.15.86:0/851407066 conn(0x7ff4a006b980 msgr2=0x7ff4a015d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:10.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.333+0000 7ff4a8200640 1 -- 172.21.15.86:0/851407066 shutdown_connections 2024-04-20T21:31:10.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.333+0000 7ff4a8200640 1 -- 172.21.15.86:0/851407066 wait complete. 2024-04-20T21:31:10.349 INFO:tasks.ceph:Scrubbing osd.2 2024-04-20T21:31:10.349 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-20T21:31:10.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.547+0000 7fca57e00640 1 Processor -- start 2024-04-20T21:31:10.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.547+0000 7fca57e00640 1 -- start start 2024-04-20T21:31:10.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.548+0000 7fca57e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca580af870 0x7fca5806dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:10.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.548+0000 7fca57e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca5815cbe0 con 0x7fca58153590 2024-04-20T21:31:10.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.548+0000 7fca57e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca5815cea0 con 0x7fca580d0d20 2024-04-20T21:31:10.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.548+0000 7fca57e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fca58153970 con 0x7fca580af870 2024-04-20T21:31:10.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.549+0000 7fca56a00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fca58153590 0x7fca580d0610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:53126/0 (socket says 172.21.15.86:53126) 2024-04-20T21:31:10.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.549+0000 7fca56a00640 1 -- 172.21.15.86:0/3353196472 learned_addr learned my addr 172.21.15.86:0/3353196472 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:31:10.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.549+0000 7fca57400640 1 --2- 172.21.15.86:0/3353196472 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca580af870 0x7fca5806dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:10.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.549+0000 7fca55600640 1 -- 172.21.15.86:0/3353196472 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4126421426 0 0) 0x7fca5815cbe0 con 0x7fca58153590 2024-04-20T21:31:10.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.549+0000 7fca55600640 1 -- 172.21.15.86:0/3353196472 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fca3c003520 con 0x7fca58153590 2024-04-20T21:31:10.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.550+0000 7fca55600640 1 -- 172.21.15.86:0/3353196472 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1972646362 0 0) 0x7fca3c003520 con 0x7fca58153590 2024-04-20T21:31:10.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.550+0000 7fca55600640 1 -- 172.21.15.86:0/3353196472 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca580af870 msgr2=0x7fca5806dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:10.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.550+0000 7fca55600640 1 --2- 172.21.15.86:0/3353196472 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca580af870 0x7fca5806dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:10.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.550+0000 7fca55600640 1 -- 172.21.15.86:0/3353196472 >> v1:172.21.15.144:6789/0 conn(0x7fca580d0d20 legacy=0x7fca580af160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:10.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.550+0000 7fca55600640 1 -- 172.21.15.86:0/3353196472 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca5815d180 con 0x7fca58153590 2024-04-20T21:31:10.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.550+0000 7fca55600640 1 -- 172.21.15.86:0/3353196472 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fca480019f0 con 0x7fca58153590 2024-04-20T21:31:10.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.550+0000 7fca55600640 1 -- 172.21.15.86:0/3353196472 >> v1:172.21.15.86:6789/0 conn(0x7fca58153590 legacy=0x7fca580d0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:10.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.550+0000 7fca55600640 1 --2- 172.21.15.86:0/3353196472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca3c0039e0 0x7fca3c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:10.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.550+0000 7fca55600640 1 --2- 172.21.15.86:0/3353196472 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca580af870 0x7fca3c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:10.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.550+0000 7fca55600640 1 --2- 172.21.15.86:0/3353196472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca3c004d40 0x7fca3c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:10.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.550+0000 7fca55600640 1 -- 172.21.15.86:0/3353196472 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fca58153970 con 0x7fca3c0039e0 2024-04-20T21:31:10.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.550+0000 7fca55600640 1 -- 172.21.15.86:0/3353196472 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fca3c00ba70 con 0x7fca3c004d40 2024-04-20T21:31:10.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.550+0000 7fca55600640 1 -- 172.21.15.86:0/3353196472 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fca3c00bc70 con 0x7fca580af870 2024-04-20T21:31:10.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.550+0000 7fca56a00640 1 --2- 172.21.15.86:0/3353196472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca3c0039e0 0x7fca3c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:10.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.550+0000 7fca57400640 1 --2- 172.21.15.86:0/3353196472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca3c004d40 0x7fca3c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:10.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.550+0000 7fca56a00640 1 -- 172.21.15.86:0/3353196472 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca580af870 msgr2=0x7fca3c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:10.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.550+0000 7fca56000640 1 --2- 172.21.15.86:0/3353196472 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca580af870 0x7fca3c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:10.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.550+0000 7fca56a00640 1 --2- 172.21.15.86:0/3353196472 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca580af870 0x7fca3c004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:10.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.550+0000 7fca56a00640 1 -- 172.21.15.86:0/3353196472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca3c004d40 msgr2=0x7fca3c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:10.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.550+0000 7fca56a00640 1 --2- 172.21.15.86:0/3353196472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca3c004d40 0x7fca3c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:10.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.551+0000 7fca56a00640 1 -- 172.21.15.86:0/3353196472 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca3c003520 con 0x7fca3c0039e0 2024-04-20T21:31:10.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.551+0000 7fca57400640 1 --2- 172.21.15.86:0/3353196472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca3c004d40 0x7fca3c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:31:10.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.551+0000 7fca56a00640 1 --2- 172.21.15.86:0/3353196472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca3c0039e0 0x7fca3c003e00 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7fca4800a830 tx=0x7fca3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eca6972727846b27 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:10.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.552+0000 7fca55600640 1 -- 172.21.15.86:0/3353196472 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4800a480 con 0x7fca3c0039e0 2024-04-20T21:31:10.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.552+0000 7fca55600640 1 -- 172.21.15.86:0/3353196472 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca48034630 con 0x7fca3c0039e0 2024-04-20T21:31:10.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.552+0000 7fca55600640 1 -- 172.21.15.86:0/3353196472 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca48034a60 con 0x7fca3c0039e0 2024-04-20T21:31:10.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.552+0000 7fca57e00640 1 -- 172.21.15.86:0/3353196472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca3c0039e0 msgr2=0x7fca3c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:10.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.552+0000 7fca57e00640 1 --2- 172.21.15.86:0/3353196472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca3c0039e0 0x7fca3c003e00 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7fca4800a830 tx=0x7fca3c005830 comp rx=0 tx=0).stop 2024-04-20T21:31:10.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.552+0000 7fca57400640 1 -- 172.21.15.86:0/3353196472 reap_dead start 2024-04-20T21:31:10.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.552+0000 7fca57e00640 1 -- 172.21.15.86:0/3353196472 shutdown_connections 2024-04-20T21:31:10.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.552+0000 7fca57e00640 1 -- 172.21.15.86:0/3353196472 >> 172.21.15.86:0/3353196472 conn(0x7fca580b2180 msgr2=0x7fca580b2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:10.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.552+0000 7fca57e00640 1 -- 172.21.15.86:0/3353196472 shutdown_connections 2024-04-20T21:31:10.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.552+0000 7fca57e00640 1 -- 172.21.15.86:0/3353196472 wait complete. 2024-04-20T21:31:10.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.554+0000 7fca57e00640 1 Processor -- start 2024-04-20T21:31:10.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.554+0000 7fca57e00640 1 -- start start 2024-04-20T21:31:10.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.555+0000 7fca57e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca3c0039e0 0x7fca581f6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:10.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.555+0000 7fca57e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca581f7490 0x7fca58158fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:10.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.555+0000 7fca57e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca581543e0 0x7fca5815b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:10.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.555+0000 7fca57e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fca58071ad0 con 0x7fca581543e0 2024-04-20T21:31:10.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.555+0000 7fca57e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fca5815d740 con 0x7fca581f7490 2024-04-20T21:31:10.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.555+0000 7fca57e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fca58071c50 con 0x7fca3c0039e0 2024-04-20T21:31:10.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.555+0000 7fca57400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca581543e0 0x7fca5815b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:10.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.555+0000 7fca57400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca581543e0 0x7fca5815b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33612/0 (socket says 172.21.15.86:33612) 2024-04-20T21:31:10.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.556+0000 7fca57400640 1 -- 172.21.15.86:0/1202590565 learned_addr learned my addr 172.21.15.86:0/1202590565 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:31:10.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.556+0000 7fca56a00640 1 --2- 172.21.15.86:0/1202590565 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca3c0039e0 0x7fca581f6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:10.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.556+0000 7fca56000640 1 --2- 172.21.15.86:0/1202590565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca581f7490 0x7fca58158fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:10.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.556+0000 7fca57400640 1 -- 172.21.15.86:0/1202590565 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca3c0039e0 msgr2=0x7fca581f6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:10.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.556+0000 7fca57400640 1 --2- 172.21.15.86:0/1202590565 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fca3c0039e0 0x7fca581f6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:10.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.556+0000 7fca57400640 1 -- 172.21.15.86:0/1202590565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca581f7490 msgr2=0x7fca58158fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:10.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.556+0000 7fca57400640 1 --2- 172.21.15.86:0/1202590565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca581f7490 0x7fca58158fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:10.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.556+0000 7fca57400640 1 -- 172.21.15.86:0/1202590565 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca581594f0 con 0x7fca581543e0 2024-04-20T21:31:10.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.556+0000 7fca57400640 1 --2- 172.21.15.86:0/1202590565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca581543e0 0x7fca5815b5b0 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7fca50010790 tx=0x7fca50010c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:10.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.556+0000 7fca4f400640 1 -- 172.21.15.86:0/1202590565 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5000f040 con 0x7fca581543e0 2024-04-20T21:31:10.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.556+0000 7fca4f400640 1 -- 172.21.15.86:0/1202590565 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca5000e870 con 0x7fca581543e0 2024-04-20T21:31:10.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.556+0000 7fca4f400640 1 -- 172.21.15.86:0/1202590565 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5000ec40 con 0x7fca581543e0 2024-04-20T21:31:10.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.556+0000 7fca57e00640 1 -- 172.21.15.86:0/1202590565 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca581f7a10 con 0x7fca581543e0 2024-04-20T21:31:10.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.557+0000 7fca57e00640 1 -- 172.21.15.86:0/1202590565 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca581f7fd0 con 0x7fca581543e0 2024-04-20T21:31:10.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.557+0000 7fca57e00640 1 -- 172.21.15.86:0/1202590565 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fca5804fcf0 con 0x7fca581543e0 2024-04-20T21:31:10.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.558+0000 7fca4f400640 1 -- 172.21.15.86:0/1202590565 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fca500253f0 con 0x7fca581543e0 2024-04-20T21:31:10.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.559+0000 7fca4f400640 1 --2- 172.21.15.86:0/1202590565 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fca3806c920 0x7fca3806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:10.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.560+0000 7fca4f400640 1 -- 172.21.15.86:0/1202590565 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7fca5008ba60 con 0x7fca581543e0 2024-04-20T21:31:10.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.560+0000 7fca56a00640 1 --2- 172.21.15.86:0/1202590565 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fca3806c920 0x7fca3806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:10.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.560+0000 7fca4f400640 1 --2- 172.21.15.86:0/1202590565 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x7fca38072920 0x7fca38074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:10.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.560+0000 7fca4f400640 1 -- 172.21.15.86:0/1202590565 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fca50001d30 con 0x7fca38072920 2024-04-20T21:31:10.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.560+0000 7fca4f400640 1 -- 172.21.15.86:0/1202590565 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=119) ==== 24+0+0 (secure 0 0 0) 0x7fca500902a0 con 0x7fca581543e0 2024-04-20T21:31:10.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.560+0000 7fca56000640 1 --2- 172.21.15.86:0/1202590565 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x7fca38072920 0x7fca38074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:10.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.560+0000 7fca56a00640 1 --2- 172.21.15.86:0/1202590565 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fca3806c920 0x7fca3806ede0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fca48034000 tx=0x7fca4802f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:10.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.561+0000 7fca56000640 1 --2- 172.21.15.86:0/1202590565 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x7fca38072920 0x7fca38074d40 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:10.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.561+0000 7fca4f400640 1 -- 172.21.15.86:0/1202590565 <== osd.2 v2:172.21.15.86:6816/3437611468 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fca50001d30 con 0x7fca38072920 2024-04-20T21:31:10.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.576+0000 7fca57e00640 1 -- 172.21.15.86:0/1202590565 --> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fca580b47a0 con 0x7fca38072920 2024-04-20T21:31:10.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.576+0000 7fca4f400640 1 -- 172.21.15.86:0/1202590565 <== osd.2 v2:172.21.15.86:6816/3437611468 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fca580b47a0 con 0x7fca38072920 2024-04-20T21:31:10.575 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:31:10.575 INFO:teuthology.orchestra.run.smithi086.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T21:31:10.575 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:31:10.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.576+0000 7fca57e00640 1 -- 172.21.15.86:0/1202590565 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x7fca38072920 msgr2=0x7fca38074d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:10.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.576+0000 7fca57e00640 1 --2- 172.21.15.86:0/1202590565 >> [v2:172.21.15.86:6816/3437611468,v1:172.21.15.86:6818/3437611468] conn(0x7fca38072920 0x7fca38074d40 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:10.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.577+0000 7fca57e00640 1 -- 172.21.15.86:0/1202590565 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fca3806c920 msgr2=0x7fca3806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:10.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.577+0000 7fca57e00640 1 --2- 172.21.15.86:0/1202590565 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fca3806c920 0x7fca3806ede0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fca48034000 tx=0x7fca4802f000 comp rx=0 tx=0).stop 2024-04-20T21:31:10.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.577+0000 7fca57e00640 1 -- 172.21.15.86:0/1202590565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca581543e0 msgr2=0x7fca5815b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:10.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.577+0000 7fca57e00640 1 --2- 172.21.15.86:0/1202590565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca581543e0 0x7fca5815b5b0 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7fca50010790 tx=0x7fca50010c50 comp rx=0 tx=0).stop 2024-04-20T21:31:10.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.577+0000 7fca57400640 1 -- 172.21.15.86:0/1202590565 reap_dead start 2024-04-20T21:31:10.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.577+0000 7fca57e00640 1 -- 172.21.15.86:0/1202590565 shutdown_connections 2024-04-20T21:31:10.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.577+0000 7fca57e00640 1 -- 172.21.15.86:0/1202590565 >> 172.21.15.86:0/1202590565 conn(0x7fca580b2180 msgr2=0x7fca580d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:10.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.577+0000 7fca57e00640 1 -- 172.21.15.86:0/1202590565 shutdown_connections 2024-04-20T21:31:10.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.577+0000 7fca57e00640 1 -- 172.21.15.86:0/1202590565 wait complete. 2024-04-20T21:31:10.593 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-20T21:31:10.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.786+0000 7ff1be600640 1 Processor -- start 2024-04-20T21:31:10.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.786+0000 7ff1be600640 1 -- start start 2024-04-20T21:31:10.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.788+0000 7ff1be600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff1b8162bd0 0x7ff1b80cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:10.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.788+0000 7ff1be600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1b80cd5e0 con 0x7ff1b806e1e0 2024-04-20T21:31:10.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.788+0000 7ff1be600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1b80b3830 con 0x7ff1b80b3330 2024-04-20T21:31:10.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.788+0000 7ff1be600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7ff1b80b3fc0 con 0x7ff1b8162bd0 2024-04-20T21:31:10.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.788+0000 7ff1b7e00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7ff1b806e1e0 0x7ff1b806e5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:53130/0 (socket says 172.21.15.86:53130) 2024-04-20T21:31:10.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.788+0000 7ff1b7400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff1b8162bd0 0x7ff1b80cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:10.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.788+0000 7ff1b7e00640 1 -- 172.21.15.86:0/1073503449 learned_addr learned my addr 172.21.15.86:0/1073503449 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:31:10.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.789+0000 7ff1b7400640 1 -- 172.21.15.86:0/1073503449 >> v1:172.21.15.144:6789/0 conn(0x7ff1b80b3330 legacy=0x7ff1b80d0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:10.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.789+0000 7ff1b7400640 1 -- 172.21.15.86:0/1073503449 >> v1:172.21.15.86:6789/0 conn(0x7ff1b806e1e0 legacy=0x7ff1b806e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:10.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.789+0000 7ff1b7400640 1 -- 172.21.15.86:0/1073503449 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1b80b42a0 con 0x7ff1b8162bd0 2024-04-20T21:31:10.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.789+0000 7ff1b6000640 1 -- 172.21.15.86:0/1073503449 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 472068937 0 0) 0x7ff1b80cd5e0 con 0x7ff1b806e1e0 2024-04-20T21:31:10.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.790+0000 7ff1b7400640 1 --2- 172.21.15.86:0/1073503449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1b8162bd0 0x7ff1b80cd0a0 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7ff1a80097a0 tx=0x7ff1a802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69bd898b8e2dd988 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:10.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.790+0000 7ff1b6000640 1 -- 172.21.15.86:0/1073503449 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a80315a0 con 0x7ff1b8162bd0 2024-04-20T21:31:10.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.790+0000 7ff1b6000640 1 -- 172.21.15.86:0/1073503449 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1a8037070 con 0x7ff1b8162bd0 2024-04-20T21:31:10.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.790+0000 7ff1b6000640 1 -- 172.21.15.86:0/1073503449 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8031960 con 0x7ff1b8162bd0 2024-04-20T21:31:10.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.791+0000 7ff1be600640 1 -- 172.21.15.86:0/1073503449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1b8162bd0 msgr2=0x7ff1b80cd0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:10.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.791+0000 7ff1be600640 1 --2- 172.21.15.86:0/1073503449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1b8162bd0 0x7ff1b80cd0a0 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7ff1a80097a0 tx=0x7ff1a802b840 comp rx=0 tx=0).stop 2024-04-20T21:31:10.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.791+0000 7ff1be600640 1 -- 172.21.15.86:0/1073503449 shutdown_connections 2024-04-20T21:31:10.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.791+0000 7ff1be600640 1 --2- 172.21.15.86:0/1073503449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1b8162bd0 0x7ff1b80cd0a0 unknown :-1 s=CLOSED pgs=2157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:10.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.791+0000 7ff1be600640 1 -- 172.21.15.86:0/1073503449 >> 172.21.15.86:0/1073503449 conn(0x7ff1b806b980 msgr2=0x7ff1b806bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:10.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.791+0000 7ff1be600640 1 -- 172.21.15.86:0/1073503449 shutdown_connections 2024-04-20T21:31:10.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.791+0000 7ff1be600640 1 -- 172.21.15.86:0/1073503449 wait complete. 2024-04-20T21:31:10.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.793+0000 7ff1be600640 1 Processor -- start 2024-04-20T21:31:10.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.793+0000 7ff1be600640 1 -- start start 2024-04-20T21:31:10.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.794+0000 7ff1be600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1b806e1e0 0x7ff1b81532d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:10.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.794+0000 7ff1be600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1b80b3330 0x7ff1b8157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:10.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.794+0000 7ff1be600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1b8162bd0 0x7ff1b81560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:10.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.794+0000 7ff1be600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b8163130 con 0x7ff1b806e1e0 2024-04-20T21:31:10.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.794+0000 7ff1be600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b8162fb0 con 0x7ff1b80b3330 2024-04-20T21:31:10.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.794+0000 7ff1be600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7ff1b8072040 con 0x7ff1b8162bd0 2024-04-20T21:31:10.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.794+0000 7ff1b7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1b806e1e0 0x7ff1b81532d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:10.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.794+0000 7ff1b7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1b806e1e0 0x7ff1b81532d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33644/0 (socket says 172.21.15.86:33644) 2024-04-20T21:31:10.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.794+0000 7ff1b7400640 1 -- 172.21.15.86:0/2035629334 learned_addr learned my addr 172.21.15.86:0/2035629334 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:31:10.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.794+0000 7ff1b7400640 1 -- 172.21.15.86:0/2035629334 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1b8162bd0 msgr2=0x7ff1b81560b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:10.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.794+0000 7ff1b7400640 1 --2- 172.21.15.86:0/2035629334 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1b8162bd0 0x7ff1b81560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:10.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.794+0000 7ff1b7400640 1 -- 172.21.15.86:0/2035629334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1b80b3330 msgr2=0x7ff1b8157fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:10.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.794+0000 7ff1b6a00640 1 --2- 172.21.15.86:0/2035629334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1b80b3330 0x7ff1b8157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:10.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.794+0000 7ff1b7400640 1 --2- 172.21.15.86:0/2035629334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1b80b3330 0x7ff1b8157fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:10.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.794+0000 7ff1b7400640 1 -- 172.21.15.86:0/2035629334 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1b8155010 con 0x7ff1b806e1e0 2024-04-20T21:31:10.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.795+0000 7ff1b7400640 1 --2- 172.21.15.86:0/2035629334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1b806e1e0 0x7ff1b81532d0 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7ff1a80098c0 tx=0x7ff1a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:10.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.795+0000 7ff1af400640 1 -- 172.21.15.86:0/2035629334 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8032840 con 0x7ff1b806e1e0 2024-04-20T21:31:10.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.795+0000 7ff1af400640 1 -- 172.21.15.86:0/2035629334 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1a80329e0 con 0x7ff1b806e1e0 2024-04-20T21:31:10.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.795+0000 7ff1be600640 1 -- 172.21.15.86:0/2035629334 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1b81552f0 con 0x7ff1b806e1e0 2024-04-20T21:31:10.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.795+0000 7ff1af400640 1 -- 172.21.15.86:0/2035629334 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8032dd0 con 0x7ff1b806e1e0 2024-04-20T21:31:10.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.795+0000 7ff1be600640 1 -- 172.21.15.86:0/2035629334 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1b81558b0 con 0x7ff1b806e1e0 2024-04-20T21:31:10.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.796+0000 7ff1be600640 1 -- 172.21.15.86:0/2035629334 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1b80689e0 con 0x7ff1b806e1e0 2024-04-20T21:31:10.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.797+0000 7ff1af400640 1 -- 172.21.15.86:0/2035629334 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff1a803b3f0 con 0x7ff1b806e1e0 2024-04-20T21:31:10.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.798+0000 7ff1af400640 1 --2- 172.21.15.86:0/2035629334 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff19806c8f0 0x7ff19806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:10.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.798+0000 7ff1af400640 1 -- 172.21.15.86:0/2035629334 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7ff1a802f030 con 0x7ff1b806e1e0 2024-04-20T21:31:10.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.798+0000 7ff1b6a00640 1 --2- 172.21.15.86:0/2035629334 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff19806c8f0 0x7ff19806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:10.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.799+0000 7ff1b6a00640 1 --2- 172.21.15.86:0/2035629334 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff19806c8f0 0x7ff19806edb0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7ff1b80cc530 tx=0x7ff1a4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:10.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:10.803+0000 7ff1af400640 1 -- 172.21.15.86:0/2035629334 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff1a8040070 con 0x7ff1b806e1e0 2024-04-20T21:31:11.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.002+0000 7ff1be600640 1 -- 172.21.15.86:0/2035629334 --> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7ff1b80ce670 con 0x7ff19806c8f0 2024-04-20T21:31:11.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.003+0000 7ff1af400640 1 -- 172.21.15.86:0/2035629334 <== mgr.4107 v2:172.21.15.86:6832/302045240 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ff1b80ce670 con 0x7ff19806c8f0 2024-04-20T21:31:11.002 INFO:teuthology.orchestra.run.smithi086.stderr:instructed osd(s) 2 to deep-scrub 2024-04-20T21:31:11.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.005+0000 7ff1be600640 1 -- 172.21.15.86:0/2035629334 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff19806c8f0 msgr2=0x7ff19806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:11.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.005+0000 7ff1be600640 1 --2- 172.21.15.86:0/2035629334 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff19806c8f0 0x7ff19806edb0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7ff1b80cc530 tx=0x7ff1a4009040 comp rx=0 tx=0).stop 2024-04-20T21:31:11.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.005+0000 7ff1be600640 1 -- 172.21.15.86:0/2035629334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1b806e1e0 msgr2=0x7ff1b81532d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:11.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.005+0000 7ff1be600640 1 --2- 172.21.15.86:0/2035629334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1b806e1e0 0x7ff1b81532d0 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7ff1a80098c0 tx=0x7ff1a80046c0 comp rx=0 tx=0).stop 2024-04-20T21:31:11.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.005+0000 7ff1be600640 1 -- 172.21.15.86:0/2035629334 shutdown_connections 2024-04-20T21:31:11.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.006+0000 7ff1be600640 1 --2- 172.21.15.86:0/2035629334 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7ff19806c8f0 0x7ff19806edb0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:11.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.006+0000 7ff1be600640 1 --2- 172.21.15.86:0/2035629334 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7ff1b8162bd0 0x7ff1b81560b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:11.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.006+0000 7ff1be600640 1 --2- 172.21.15.86:0/2035629334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1b80b3330 0x7ff1b8157fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:11.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.006+0000 7ff1be600640 1 --2- 172.21.15.86:0/2035629334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1b806e1e0 0x7ff1b81532d0 unknown :-1 s=CLOSED pgs=2158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:11.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.006+0000 7ff1be600640 1 -- 172.21.15.86:0/2035629334 >> 172.21.15.86:0/2035629334 conn(0x7ff1b806b980 msgr2=0x7ff1b806e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:11.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.006+0000 7ff1be600640 1 -- 172.21.15.86:0/2035629334 shutdown_connections 2024-04-20T21:31:11.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.006+0000 7ff1be600640 1 -- 172.21.15.86:0/2035629334 wait complete. 2024-04-20T21:31:11.024 INFO:tasks.ceph:Scrubbing osd.3 2024-04-20T21:31:11.024 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-20T21:31:11.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.229+0000 7f6824e00640 1 Processor -- start 2024-04-20T21:31:11.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.229+0000 7f6824e00640 1 -- start start 2024-04-20T21:31:11.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.231+0000 7f6824e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6820072410 0x7f6820167340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:11.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.231+0000 7f6824e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f682006f810 con 0x7f6820162bb0 2024-04-20T21:31:11.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.231+0000 7f6824e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68200b4840 con 0x7f682006f070 2024-04-20T21:31:11.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.231+0000 7f6824e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f68200b4b20 con 0x7f6820072410 2024-04-20T21:31:11.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.231+0000 7f681de00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f6820162bb0 0x7f6820071d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:53140/0 (socket says 172.21.15.86:53140) 2024-04-20T21:31:11.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.231+0000 7f681de00640 1 -- 172.21.15.86:0/2733573353 learned_addr learned my addr 172.21.15.86:0/2733573353 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:31:11.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.232+0000 7f681d400640 1 --2- 172.21.15.86:0/2733573353 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6820072410 0x7f6820167340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:11.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.232+0000 7f681ca00640 1 -- 172.21.15.86:0/2733573353 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2449384076 0 0) 0x7f682006f810 con 0x7f6820162bb0 2024-04-20T21:31:11.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.232+0000 7f681ca00640 1 -- 172.21.15.86:0/2733573353 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6800003520 con 0x7f6820162bb0 2024-04-20T21:31:11.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.232+0000 7f681ca00640 1 -- 172.21.15.86:0/2733573353 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2703638442 0 0) 0x7f6800003520 con 0x7f6820162bb0 2024-04-20T21:31:11.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.232+0000 7f681ca00640 1 -- 172.21.15.86:0/2733573353 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6820072410 msgr2=0x7f6820167340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:11.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.232+0000 7f681ca00640 1 --2- 172.21.15.86:0/2733573353 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6820072410 0x7f6820167340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:11.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.232+0000 7f681ca00640 1 -- 172.21.15.86:0/2733573353 >> v1:172.21.15.144:6789/0 conn(0x7f682006f070 legacy=0x7f682006dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:11.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.232+0000 7f681ca00640 1 -- 172.21.15.86:0/2733573353 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68200b4e00 con 0x7f6820162bb0 2024-04-20T21:31:11.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.232+0000 7f681ca00640 1 -- 172.21.15.86:0/2733573353 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f6810001670 con 0x7f6820162bb0 2024-04-20T21:31:11.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.232+0000 7f681ca00640 1 -- 172.21.15.86:0/2733573353 >> v1:172.21.15.86:6789/0 conn(0x7f6820162bb0 legacy=0x7f6820071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:11.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.232+0000 7f681ca00640 1 --2- 172.21.15.86:0/2733573353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68000039e0 0x7f6800003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:11.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.233+0000 7f681ca00640 1 --2- 172.21.15.86:0/2733573353 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6820072410 0x7f6800004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:11.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.233+0000 7f681ca00640 1 --2- 172.21.15.86:0/2733573353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6800004d40 0x7f68000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:11.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.233+0000 7f681ca00640 1 -- 172.21.15.86:0/2733573353 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f68200b4b20 con 0x7f68000039e0 2024-04-20T21:31:11.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.233+0000 7f681ca00640 1 -- 172.21.15.86:0/2733573353 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f680000baa0 con 0x7f6800004d40 2024-04-20T21:31:11.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.233+0000 7f681ca00640 1 -- 172.21.15.86:0/2733573353 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f680000bca0 con 0x7f6820072410 2024-04-20T21:31:11.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.233+0000 7f681de00640 1 --2- 172.21.15.86:0/2733573353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68000039e0 0x7f6800003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:11.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.233+0000 7f681de00640 1 -- 172.21.15.86:0/2733573353 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6820072410 msgr2=0x7f6800004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:11.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.233+0000 7f681de00640 1 --2- 172.21.15.86:0/2733573353 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6820072410 0x7f6800004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:11.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.233+0000 7f681de00640 1 -- 172.21.15.86:0/2733573353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6800004d40 msgr2=0x7f68000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:11.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.233+0000 7f681de00640 1 --2- 172.21.15.86:0/2733573353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6800004d40 0x7f68000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:11.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.233+0000 7f681de00640 1 -- 172.21.15.86:0/2733573353 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6800003520 con 0x7f68000039e0 2024-04-20T21:31:11.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.234+0000 7f681de00640 1 --2- 172.21.15.86:0/2733573353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68000039e0 0x7f6800003e00 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f681000c210 tx=0x7f6800005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3ce08be480b7cba server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:11.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.234+0000 7f681ca00640 1 -- 172.21.15.86:0/2733573353 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6810007d10 con 0x7f68000039e0 2024-04-20T21:31:11.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.234+0000 7f681ca00640 1 -- 172.21.15.86:0/2733573353 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6810030070 con 0x7f68000039e0 2024-04-20T21:31:11.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.234+0000 7f681ca00640 1 -- 172.21.15.86:0/2733573353 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6810005d30 con 0x7f68000039e0 2024-04-20T21:31:11.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.235+0000 7f6824e00640 1 -- 172.21.15.86:0/2733573353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68000039e0 msgr2=0x7f6800003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:11.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.235+0000 7f6824e00640 1 --2- 172.21.15.86:0/2733573353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68000039e0 0x7f6800003e00 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f681000c210 tx=0x7f6800005830 comp rx=0 tx=0).stop 2024-04-20T21:31:11.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.235+0000 7f681e800640 1 -- 172.21.15.86:0/2733573353 reap_dead start 2024-04-20T21:31:11.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.235+0000 7f6824e00640 1 -- 172.21.15.86:0/2733573353 shutdown_connections 2024-04-20T21:31:11.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.235+0000 7f6824e00640 1 -- 172.21.15.86:0/2733573353 >> 172.21.15.86:0/2733573353 conn(0x7f68200b2180 msgr2=0x7f68200b2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:11.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.235+0000 7f6824e00640 1 -- 172.21.15.86:0/2733573353 shutdown_connections 2024-04-20T21:31:11.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.235+0000 7f6824e00640 1 -- 172.21.15.86:0/2733573353 wait complete. 2024-04-20T21:31:11.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.237+0000 7f6824e00640 1 Processor -- start 2024-04-20T21:31:11.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.237+0000 7f6824e00640 1 -- start start 2024-04-20T21:31:11.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.238+0000 7f6824e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f68000039e0 0x7f6820152c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:11.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.238+0000 7f6824e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6820158870 0x7f68201611f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:11.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.238+0000 7f6824e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6820158cf0 0x7f6820161730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:11.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.238+0000 7f6824e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f68200b5600 con 0x7f6820158870 2024-04-20T21:31:11.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.238+0000 7f6824e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f682006f4e0 con 0x7f6820158cf0 2024-04-20T21:31:11.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.238+0000 7f6824e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6820162f90 con 0x7f68000039e0 2024-04-20T21:31:11.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.238+0000 7f681d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6820158870 0x7f68201611f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:11.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.238+0000 7f681d400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6820158870 0x7f68201611f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33664/0 (socket says 172.21.15.86:33664) 2024-04-20T21:31:11.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.238+0000 7f681d400640 1 -- 172.21.15.86:0/1344564549 learned_addr learned my addr 172.21.15.86:0/1344564549 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:31:11.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.239+0000 7f681de00640 1 --2- 172.21.15.86:0/1344564549 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f68000039e0 0x7f6820152c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:11.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.239+0000 7f681e800640 1 --2- 172.21.15.86:0/1344564549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6820158cf0 0x7f6820161730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:11.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.239+0000 7f681d400640 1 -- 172.21.15.86:0/1344564549 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f68000039e0 msgr2=0x7f6820152c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:11.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.239+0000 7f681d400640 1 --2- 172.21.15.86:0/1344564549 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f68000039e0 0x7f6820152c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:11.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.239+0000 7f681d400640 1 -- 172.21.15.86:0/1344564549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6820158cf0 msgr2=0x7f6820161730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:11.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.239+0000 7f681d400640 1 --2- 172.21.15.86:0/1344564549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6820158cf0 0x7f6820161730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:11.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.239+0000 7f681d400640 1 -- 172.21.15.86:0/1344564549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6820159250 con 0x7f6820158870 2024-04-20T21:31:11.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.239+0000 7f681de00640 1 --2- 172.21.15.86:0/1344564549 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f68000039e0 0x7f6820152c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:31:11.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.239+0000 7f681d400640 1 --2- 172.21.15.86:0/1344564549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6820158870 0x7f68201611f0 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7f680800e7f0 tx=0x7f680800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:11.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.239+0000 7f6816000640 1 -- 172.21.15.86:0/1344564549 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6808017070 con 0x7f6820158870 2024-04-20T21:31:11.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.239+0000 7f6816000640 1 -- 172.21.15.86:0/1344564549 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f680800c940 con 0x7f6820158870 2024-04-20T21:31:11.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.239+0000 7f6816000640 1 -- 172.21.15.86:0/1344564549 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680800cd30 con 0x7f6820158870 2024-04-20T21:31:11.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.239+0000 7f6824e00640 1 -- 172.21.15.86:0/1344564549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68201ec260 con 0x7f6820158870 2024-04-20T21:31:11.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.239+0000 7f6824e00640 1 -- 172.21.15.86:0/1344564549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68201ec820 con 0x7f6820158870 2024-04-20T21:31:11.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.240+0000 7f6824e00640 1 -- 172.21.15.86:0/1344564549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f68201ecc00 con 0x7f6820158870 2024-04-20T21:31:11.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.242+0000 7f6816000640 1 -- 172.21.15.86:0/1344564549 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f680801a070 con 0x7f6820158870 2024-04-20T21:31:11.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.242+0000 7f6816000640 1 --2- 172.21.15.86:0/1344564549 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f67fc06c920 0x7f67fc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:11.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.243+0000 7f6816000640 1 -- 172.21.15.86:0/1344564549 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f680808b6a0 con 0x7f6820158870 2024-04-20T21:31:11.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.243+0000 7f681de00640 1 --2- 172.21.15.86:0/1344564549 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f67fc06c920 0x7f67fc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:11.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.243+0000 7f6816000640 1 --2- 172.21.15.86:0/1344564549 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x7f67fc072920 0x7f67fc074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:11.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.243+0000 7f6816000640 1 -- 172.21.15.86:0/1344564549 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6808013e50 con 0x7f67fc072920 2024-04-20T21:31:11.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.243+0000 7f6816000640 1 -- 172.21.15.86:0/1344564549 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=119) ==== 24+0+0 (secure 0 0 0) 0x7f6808092050 con 0x7f6820158870 2024-04-20T21:31:11.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.243+0000 7f681e800640 1 --2- 172.21.15.86:0/1344564549 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x7f67fc072920 0x7f67fc074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:11.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.243+0000 7f681de00640 1 --2- 172.21.15.86:0/1344564549 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f67fc06c920 0x7f67fc06ede0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f6810034000 tx=0x7f6810009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:11.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.244+0000 7f681e800640 1 --2- 172.21.15.86:0/1344564549 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x7f67fc072920 0x7f67fc074d40 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:11.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.244+0000 7f6816000640 1 -- 172.21.15.86:0/1344564549 <== osd.3 v2:172.21.15.86:6808/794135519 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6808013e50 con 0x7f67fc072920 2024-04-20T21:31:11.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.259+0000 7f6824e00640 1 -- 172.21.15.86:0/1344564549 --> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f68200689e0 con 0x7f67fc072920 2024-04-20T21:31:11.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.259+0000 7f6816000640 1 -- 172.21.15.86:0/1344564549 <== osd.3 v2:172.21.15.86:6808/794135519 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f68200689e0 con 0x7f67fc072920 2024-04-20T21:31:11.258 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:31:11.258 INFO:teuthology.orchestra.run.smithi086.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T21:31:11.258 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:31:11.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.260+0000 7f6824e00640 1 -- 172.21.15.86:0/1344564549 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x7f67fc072920 msgr2=0x7f67fc074d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:11.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.260+0000 7f6824e00640 1 --2- 172.21.15.86:0/1344564549 >> [v2:172.21.15.86:6808/794135519,v1:172.21.15.86:6809/794135519] conn(0x7f67fc072920 0x7f67fc074d40 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:11.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.260+0000 7f6824e00640 1 -- 172.21.15.86:0/1344564549 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f67fc06c920 msgr2=0x7f67fc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:11.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.260+0000 7f6824e00640 1 --2- 172.21.15.86:0/1344564549 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f67fc06c920 0x7f67fc06ede0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f6810034000 tx=0x7f6810009030 comp rx=0 tx=0).stop 2024-04-20T21:31:11.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.260+0000 7f6824e00640 1 -- 172.21.15.86:0/1344564549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6820158870 msgr2=0x7f68201611f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:11.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.260+0000 7f6824e00640 1 --2- 172.21.15.86:0/1344564549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6820158870 0x7f68201611f0 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7f680800e7f0 tx=0x7f680800ecb0 comp rx=0 tx=0).stop 2024-04-20T21:31:11.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.260+0000 7f681e800640 1 -- 172.21.15.86:0/1344564549 reap_dead start 2024-04-20T21:31:11.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.260+0000 7f6824e00640 1 -- 172.21.15.86:0/1344564549 shutdown_connections 2024-04-20T21:31:11.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.260+0000 7f6824e00640 1 -- 172.21.15.86:0/1344564549 >> 172.21.15.86:0/1344564549 conn(0x7f68200b2180 msgr2=0x7f682015d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:11.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.260+0000 7f6824e00640 1 -- 172.21.15.86:0/1344564549 shutdown_connections 2024-04-20T21:31:11.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.260+0000 7f6824e00640 1 -- 172.21.15.86:0/1344564549 wait complete. 2024-04-20T21:31:11.279 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-20T21:31:11.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.474+0000 7f238c600640 1 Processor -- start 2024-04-20T21:31:11.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.475+0000 7f238c600640 1 -- start start 2024-04-20T21:31:11.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.476+0000 7f238c600640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f238406d4a0 0x7f238406d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:11.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.476+0000 7f238c600640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f238409ae10 con 0x7f238415e160 2024-04-20T21:31:11.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.476+0000 7f238c600640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23840d1ae0 con 0x7f23840ce790 2024-04-20T21:31:11.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.476+0000 7f238c600640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f238406de30 con 0x7f238406d4a0 2024-04-20T21:31:11.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.477+0000 7f2389200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f238406d4a0 0x7f238406d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:11.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.477+0000 7f2389200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f238406d4a0 0x7f238406d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33680/0 (socket says 172.21.15.86:33680) 2024-04-20T21:31:11.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.477+0000 7f2389200640 1 -- 172.21.15.86:0/1486780452 learned_addr learned my addr 172.21.15.86:0/1486780452 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:31:11.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.477+0000 7f2389200640 1 -- 172.21.15.86:0/1486780452 >> v1:172.21.15.144:6789/0 conn(0x7f23840ce790 legacy=0x7f23840d13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:11.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.477+0000 7f2389200640 1 -- 172.21.15.86:0/1486780452 >> v1:172.21.15.86:6789/0 conn(0x7f238415e160 legacy=0x7f238415e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:11.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.477+0000 7f2389200640 1 -- 172.21.15.86:0/1486780452 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f238409b0d0 con 0x7f238406d4a0 2024-04-20T21:31:11.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.478+0000 7f2389200640 1 --2- 172.21.15.86:0/1486780452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f238406d4a0 0x7f238406d8f0 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7f23700098a0 tx=0x7f237002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ab56003c8919345 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:11.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.479+0000 7f2383e00640 1 -- 172.21.15.86:0/1486780452 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2370034070 con 0x7f238406d4a0 2024-04-20T21:31:11.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.479+0000 7f2383e00640 1 -- 172.21.15.86:0/1486780452 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2370030840 con 0x7f238406d4a0 2024-04-20T21:31:11.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.479+0000 7f2383e00640 1 -- 172.21.15.86:0/1486780452 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2370030c10 con 0x7f238406d4a0 2024-04-20T21:31:11.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.479+0000 7f238c600640 1 -- 172.21.15.86:0/1486780452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f238406d4a0 msgr2=0x7f238406d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:11.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.479+0000 7f238c600640 1 --2- 172.21.15.86:0/1486780452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f238406d4a0 0x7f238406d8f0 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7f23700098a0 tx=0x7f237002b980 comp rx=0 tx=0).stop 2024-04-20T21:31:11.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.479+0000 7f238c600640 1 -- 172.21.15.86:0/1486780452 shutdown_connections 2024-04-20T21:31:11.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.479+0000 7f238c600640 1 --2- 172.21.15.86:0/1486780452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f238406d4a0 0x7f238406d8f0 unknown :-1 s=CLOSED pgs=2163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:11.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.479+0000 7f238c600640 1 -- 172.21.15.86:0/1486780452 >> 172.21.15.86:0/1486780452 conn(0x7f23840b3fe0 msgr2=0x7f23840b43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:11.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.480+0000 7f238c600640 1 -- 172.21.15.86:0/1486780452 shutdown_connections 2024-04-20T21:31:11.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.480+0000 7f238c600640 1 -- 172.21.15.86:0/1486780452 wait complete. 2024-04-20T21:31:11.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.481+0000 7f238c600640 1 Processor -- start 2024-04-20T21:31:11.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.481+0000 7f238c600640 1 -- start start 2024-04-20T21:31:11.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.482+0000 7f238c600640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f238406d4a0 0x7f23840c8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:11.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.482+0000 7f238c600640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23840ce790 0x7f23840ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:11.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.482+0000 7f238c600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f238415e160 0x7f23840ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:11.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.482+0000 7f238c600640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f238409b970 con 0x7f238415e160 2024-04-20T21:31:11.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.482+0000 7f238c600640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f238409b690 con 0x7f23840ce790 2024-04-20T21:31:11.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.482+0000 7f238c600640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f238415ecc0 con 0x7f238406d4a0 2024-04-20T21:31:11.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.483+0000 7f238a600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f238415e160 0x7f23840ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:11.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.483+0000 7f238a600640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f238415e160 0x7f23840ccb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33684/0 (socket says 172.21.15.86:33684) 2024-04-20T21:31:11.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.483+0000 7f238a600640 1 -- 172.21.15.86:0/381178095 learned_addr learned my addr 172.21.15.86:0/381178095 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:31:11.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.483+0000 7f2389c00640 1 --2- 172.21.15.86:0/381178095 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f238406d4a0 0x7f23840c8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:11.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.483+0000 7f2389200640 1 --2- 172.21.15.86:0/381178095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23840ce790 0x7f23840ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:11.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.483+0000 7f238a600640 1 -- 172.21.15.86:0/381178095 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f238406d4a0 msgr2=0x7f23840c8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:11.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.483+0000 7f238a600640 1 --2- 172.21.15.86:0/381178095 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f238406d4a0 0x7f23840c8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:11.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.483+0000 7f238a600640 1 -- 172.21.15.86:0/381178095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23840ce790 msgr2=0x7f23840ca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:11.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.483+0000 7f238a600640 1 --2- 172.21.15.86:0/381178095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23840ce790 0x7f23840ca4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:11.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.483+0000 7f238a600640 1 -- 172.21.15.86:0/381178095 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23840cb5d0 con 0x7f238415e160 2024-04-20T21:31:11.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.483+0000 7f2389c00640 1 --2- 172.21.15.86:0/381178095 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f238406d4a0 0x7f23840c8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:31:11.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.484+0000 7f238a600640 1 --2- 172.21.15.86:0/381178095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f238415e160 0x7f23840ccb20 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7f237c00c740 tx=0x7f237c00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:11.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.484+0000 7f2381600640 1 -- 172.21.15.86:0/381178095 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237c016020 con 0x7f238415e160 2024-04-20T21:31:11.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.484+0000 7f2381600640 1 -- 172.21.15.86:0/381178095 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f237c00ce50 con 0x7f238415e160 2024-04-20T21:31:11.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.484+0000 7f2381600640 1 -- 172.21.15.86:0/381178095 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237c003970 con 0x7f238415e160 2024-04-20T21:31:11.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.484+0000 7f238c600640 1 -- 172.21.15.86:0/381178095 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23840cad30 con 0x7f238415e160 2024-04-20T21:31:11.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.484+0000 7f238c600640 1 -- 172.21.15.86:0/381178095 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23840cb2f0 con 0x7f238415e160 2024-04-20T21:31:11.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.485+0000 7f238c600640 1 -- 172.21.15.86:0/381178095 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23840689e0 con 0x7f238415e160 2024-04-20T21:31:11.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.486+0000 7f2381600640 1 -- 172.21.15.86:0/381178095 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f237c003b10 con 0x7f238415e160 2024-04-20T21:31:11.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.487+0000 7f2381600640 1 --2- 172.21.15.86:0/381178095 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f236406c8f0 0x7f236406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:11.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.488+0000 7f2381600640 1 -- 172.21.15.86:0/381178095 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f237c08b5e0 con 0x7f238415e160 2024-04-20T21:31:11.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.488+0000 7f2389c00640 1 --2- 172.21.15.86:0/381178095 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f236406c8f0 0x7f236406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:11.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.489+0000 7f2389c00640 1 --2- 172.21.15.86:0/381178095 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f236406c8f0 0x7f236406edb0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f237800a830 tx=0x7f2378008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:11.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.492+0000 7f2381600640 1 -- 172.21.15.86:0/381178095 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f237c020020 con 0x7f238415e160 2024-04-20T21:31:11.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.695+0000 7f238c600640 1 -- 172.21.15.86:0/381178095 --> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f23840d0280 con 0x7f236406c8f0 2024-04-20T21:31:11.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.696+0000 7f2381600640 1 -- 172.21.15.86:0/381178095 <== mgr.4107 v2:172.21.15.86:6832/302045240 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f23840d0280 con 0x7f236406c8f0 2024-04-20T21:31:11.694 INFO:teuthology.orchestra.run.smithi086.stderr:instructed osd(s) 3 to deep-scrub 2024-04-20T21:31:11.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.699+0000 7f238c600640 1 -- 172.21.15.86:0/381178095 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f236406c8f0 msgr2=0x7f236406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:11.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.699+0000 7f238c600640 1 --2- 172.21.15.86:0/381178095 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f236406c8f0 0x7f236406edb0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f237800a830 tx=0x7f2378008040 comp rx=0 tx=0).stop 2024-04-20T21:31:11.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.699+0000 7f238c600640 1 -- 172.21.15.86:0/381178095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f238415e160 msgr2=0x7f23840ccb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:11.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.699+0000 7f238c600640 1 --2- 172.21.15.86:0/381178095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f238415e160 0x7f23840ccb20 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7f237c00c740 tx=0x7f237c00cc00 comp rx=0 tx=0).stop 2024-04-20T21:31:11.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.699+0000 7f238c600640 1 -- 172.21.15.86:0/381178095 shutdown_connections 2024-04-20T21:31:11.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.699+0000 7f238c600640 1 --2- 172.21.15.86:0/381178095 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f236406c8f0 0x7f236406edb0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:11.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.699+0000 7f238c600640 1 --2- 172.21.15.86:0/381178095 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f238415e160 0x7f23840ccb20 unknown :-1 s=CLOSED pgs=2164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:11.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.699+0000 7f238c600640 1 --2- 172.21.15.86:0/381178095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23840ce790 0x7f23840ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:11.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.699+0000 7f238c600640 1 --2- 172.21.15.86:0/381178095 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f238406d4a0 0x7f23840c8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:11.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.699+0000 7f238c600640 1 -- 172.21.15.86:0/381178095 >> 172.21.15.86:0/381178095 conn(0x7f23840b3fe0 msgr2=0x7f238415e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:11.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.699+0000 7f238c600640 1 -- 172.21.15.86:0/381178095 shutdown_connections 2024-04-20T21:31:11.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.699+0000 7f238c600640 1 -- 172.21.15.86:0/381178095 wait complete. 2024-04-20T21:31:11.716 INFO:tasks.ceph:Scrubbing osd.4 2024-04-20T21:31:11.716 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-20T21:31:11.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.923+0000 7fd78ae00640 1 Processor -- start 2024-04-20T21:31:11.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.923+0000 7fd78ae00640 1 -- start start 2024-04-20T21:31:11.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.924+0000 7fd78ae00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd78415cc70 0x7fd78415d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:11.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.924+0000 7fd78ae00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd78406e010 con 0x7fd78406f0b0 2024-04-20T21:31:11.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.924+0000 7fd78ae00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd78406e2f0 con 0x7fd784071d00 2024-04-20T21:31:11.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.925+0000 7fd78ae00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd78406e5d0 con 0x7fd78415cc70 2024-04-20T21:31:11.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.925+0000 7fd77b400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fd78406f0b0 0x7fd784145a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:53162/0 (socket says 172.21.15.86:53162) 2024-04-20T21:31:11.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.925+0000 7fd77b400640 1 -- 172.21.15.86:0/3589326623 learned_addr learned my addr 172.21.15.86:0/3589326623 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:31:11.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.925+0000 7fd788e00640 1 --2- 172.21.15.86:0/3589326623 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd78415cc70 0x7fd78415d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:11.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.925+0000 7fd783400640 1 -- 172.21.15.86:0/3589326623 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4215033779 0 0) 0x7fd78406e010 con 0x7fd78406f0b0 2024-04-20T21:31:11.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.925+0000 7fd783400640 1 -- 172.21.15.86:0/3589326623 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd768003520 con 0x7fd78406f0b0 2024-04-20T21:31:11.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.926+0000 7fd783400640 1 -- 172.21.15.86:0/3589326623 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3889573516 0 0) 0x7fd768003520 con 0x7fd78406f0b0 2024-04-20T21:31:11.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.926+0000 7fd783400640 1 -- 172.21.15.86:0/3589326623 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd78415cc70 msgr2=0x7fd78415d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:11.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.926+0000 7fd783400640 1 --2- 172.21.15.86:0/3589326623 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd78415cc70 0x7fd78415d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:11.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.926+0000 7fd783400640 1 -- 172.21.15.86:0/3589326623 >> v1:172.21.15.144:6789/0 conn(0x7fd784071d00 legacy=0x7fd7840720e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:11.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.926+0000 7fd783400640 1 -- 172.21.15.86:0/3589326623 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd78406e8b0 con 0x7fd78406f0b0 2024-04-20T21:31:11.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.926+0000 7fd783400640 1 -- 172.21.15.86:0/3589326623 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fd76c0017f0 con 0x7fd78406f0b0 2024-04-20T21:31:11.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.926+0000 7fd783400640 1 -- 172.21.15.86:0/3589326623 >> v1:172.21.15.86:6789/0 conn(0x7fd78406f0b0 legacy=0x7fd784145a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:11.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.926+0000 7fd783400640 1 --2- 172.21.15.86:0/3589326623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7680039e0 0x7fd768003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:11.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.926+0000 7fd783400640 1 --2- 172.21.15.86:0/3589326623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd768004800 0x7fd768004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:11.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.926+0000 7fd783400640 1 --2- 172.21.15.86:0/3589326623 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd78415cc70 0x7fd7680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:11.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.926+0000 7fd783400640 1 -- 172.21.15.86:0/3589326623 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd78406e5d0 con 0x7fd7680039e0 2024-04-20T21:31:11.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.926+0000 7fd783400640 1 -- 172.21.15.86:0/3589326623 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd76800bad0 con 0x7fd768004800 2024-04-20T21:31:11.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.926+0000 7fd783400640 1 -- 172.21.15.86:0/3589326623 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd76800bcd0 con 0x7fd78415cc70 2024-04-20T21:31:11.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.926+0000 7fd783e00640 1 --2- 172.21.15.86:0/3589326623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7680039e0 0x7fd768003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:11.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.926+0000 7fd77b400640 1 --2- 172.21.15.86:0/3589326623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd768004800 0x7fd768004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:11.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.926+0000 7fd783e00640 1 -- 172.21.15.86:0/3589326623 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd78415cc70 msgr2=0x7fd7680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:11.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.926+0000 7fd783e00640 1 --2- 172.21.15.86:0/3589326623 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd78415cc70 0x7fd7680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:11.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.926+0000 7fd783e00640 1 -- 172.21.15.86:0/3589326623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd768004800 msgr2=0x7fd768004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:11.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.926+0000 7fd783e00640 1 --2- 172.21.15.86:0/3589326623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd768004800 0x7fd768004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:11.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.927+0000 7fd783e00640 1 -- 172.21.15.86:0/3589326623 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd768003520 con 0x7fd7680039e0 2024-04-20T21:31:11.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.928+0000 7fd783e00640 1 --2- 172.21.15.86:0/3589326623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7680039e0 0x7fd768003e00 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7fd774001720 tx=0x7fd768005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eb02e6906aeceb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:11.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.928+0000 7fd783400640 1 -- 172.21.15.86:0/3589326623 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd774001370 con 0x7fd7680039e0 2024-04-20T21:31:11.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.928+0000 7fd783400640 1 -- 172.21.15.86:0/3589326623 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7740052d0 con 0x7fd7680039e0 2024-04-20T21:31:11.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.928+0000 7fd783400640 1 -- 172.21.15.86:0/3589326623 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd774003830 con 0x7fd7680039e0 2024-04-20T21:31:11.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.928+0000 7fd78ae00640 1 -- 172.21.15.86:0/3589326623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7680039e0 msgr2=0x7fd768003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:11.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.928+0000 7fd78ae00640 1 --2- 172.21.15.86:0/3589326623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7680039e0 0x7fd768003e00 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7fd774001720 tx=0x7fd768005830 comp rx=0 tx=0).stop 2024-04-20T21:31:11.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.928+0000 7fd788e00640 1 -- 172.21.15.86:0/3589326623 reap_dead start 2024-04-20T21:31:11.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.928+0000 7fd78ae00640 1 -- 172.21.15.86:0/3589326623 shutdown_connections 2024-04-20T21:31:11.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.928+0000 7fd78ae00640 1 -- 172.21.15.86:0/3589326623 >> 172.21.15.86:0/3589326623 conn(0x7fd7840b2180 msgr2=0x7fd7840b2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:11.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.929+0000 7fd78ae00640 1 -- 172.21.15.86:0/3589326623 shutdown_connections 2024-04-20T21:31:11.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.929+0000 7fd78ae00640 1 -- 172.21.15.86:0/3589326623 wait complete. 2024-04-20T21:31:11.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.930+0000 7fd78ae00640 1 Processor -- start 2024-04-20T21:31:11.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.930+0000 7fd78ae00640 1 -- start start 2024-04-20T21:31:11.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.931+0000 7fd78ae00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7680039e0 0x7fd7840b4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:11.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.931+0000 7fd78ae00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd78413ca90 0x7fd784150bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:11.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.931+0000 7fd78ae00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd784151110 0x7fd7841515a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:11.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.931+0000 7fd78ae00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd7841461a0 con 0x7fd7680039e0 2024-04-20T21:31:11.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.931+0000 7fd78ae00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fd78406f510 con 0x7fd78413ca90 2024-04-20T21:31:11.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.931+0000 7fd78ae00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fd78406dc30 con 0x7fd784151110 2024-04-20T21:31:11.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.932+0000 7fd783e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7680039e0 0x7fd7840b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:11.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.932+0000 7fd783e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7680039e0 0x7fd7840b4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33710/0 (socket says 172.21.15.86:33710) 2024-04-20T21:31:11.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.932+0000 7fd783e00640 1 -- 172.21.15.86:0/4091609625 learned_addr learned my addr 172.21.15.86:0/4091609625 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:31:11.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.932+0000 7fd783e00640 1 -- 172.21.15.86:0/4091609625 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd784151110 msgr2=0x7fd7841515a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:11.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.932+0000 7fd783e00640 1 --2- 172.21.15.86:0/4091609625 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fd784151110 0x7fd7841515a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:11.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.932+0000 7fd783e00640 1 -- 172.21.15.86:0/4091609625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd78413ca90 msgr2=0x7fd784150bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:11.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.932+0000 7fd783e00640 1 --2- 172.21.15.86:0/4091609625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd78413ca90 0x7fd784150bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:11.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.932+0000 7fd783e00640 1 -- 172.21.15.86:0/4091609625 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7840b49a0 con 0x7fd7680039e0 2024-04-20T21:31:11.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.932+0000 7fd783e00640 1 --2- 172.21.15.86:0/4091609625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7680039e0 0x7fd7840b4030 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7fd774008fd0 tx=0x7fd774001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:11.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.933+0000 7fd780c00640 1 -- 172.21.15.86:0/4091609625 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd774041070 con 0x7fd7680039e0 2024-04-20T21:31:11.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.933+0000 7fd780c00640 1 -- 172.21.15.86:0/4091609625 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd774005480 con 0x7fd7680039e0 2024-04-20T21:31:11.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.933+0000 7fd780c00640 1 -- 172.21.15.86:0/4091609625 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd774007a30 con 0x7fd7680039e0 2024-04-20T21:31:11.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.933+0000 7fd78ae00640 1 -- 172.21.15.86:0/4091609625 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7840b4c80 con 0x7fd7680039e0 2024-04-20T21:31:11.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.933+0000 7fd78ae00640 1 -- 172.21.15.86:0/4091609625 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7840b5240 con 0x7fd7680039e0 2024-04-20T21:31:11.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.934+0000 7fd78ae00640 1 -- 172.21.15.86:0/4091609625 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd7840689e0 con 0x7fd7680039e0 2024-04-20T21:31:11.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.935+0000 7fd780c00640 1 -- 172.21.15.86:0/4091609625 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd774044070 con 0x7fd7680039e0 2024-04-20T21:31:11.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.936+0000 7fd780c00640 1 --2- 172.21.15.86:0/4091609625 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd76006c940 0x7fd76006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:11.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.936+0000 7fd780c00640 1 -- 172.21.15.86:0/4091609625 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7fd7740ad490 con 0x7fd7680039e0 2024-04-20T21:31:11.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.936+0000 7fd780c00640 1 --2- 172.21.15.86:0/4091609625 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x7fd760072940 0x7fd760074d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:11.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.936+0000 7fd780c00640 1 -- 172.21.15.86:0/4091609625 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd7740052d0 con 0x7fd760072940 2024-04-20T21:31:11.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.936+0000 7fd780c00640 1 -- 172.21.15.86:0/4091609625 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=119) ==== 24+0+0 (secure 0 0 0) 0x7fd7740b4050 con 0x7fd7680039e0 2024-04-20T21:31:11.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.936+0000 7fd77b400640 1 --2- 172.21.15.86:0/4091609625 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd76006c940 0x7fd76006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:11.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.937+0000 7fd788e00640 1 --2- 172.21.15.86:0/4091609625 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x7fd760072940 0x7fd760074d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:11.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.937+0000 7fd77b400640 1 --2- 172.21.15.86:0/4091609625 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd76006c940 0x7fd76006ee00 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fd78413ceb0 tx=0x7fd76c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:11.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.938+0000 7fd788e00640 1 --2- 172.21.15.86:0/4091609625 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x7fd760072940 0x7fd760074d60 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:11.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.940+0000 7fd780c00640 1 -- 172.21.15.86:0/4091609625 <== osd.4 v2:172.21.15.144:6800/4087309475 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd7740052d0 con 0x7fd760072940 2024-04-20T21:31:11.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.954+0000 7fd78ae00640 1 -- 172.21.15.86:0/4091609625 --> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd7840b4570 con 0x7fd760072940 2024-04-20T21:31:11.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.955+0000 7fd780c00640 1 -- 172.21.15.86:0/4091609625 <== osd.4 v2:172.21.15.144:6800/4087309475 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fd7840b4570 con 0x7fd760072940 2024-04-20T21:31:11.954 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:31:11.955 INFO:teuthology.orchestra.run.smithi086.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T21:31:11.955 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:31:11.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.956+0000 7fd78ae00640 1 -- 172.21.15.86:0/4091609625 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x7fd760072940 msgr2=0x7fd760074d60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:11.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.956+0000 7fd78ae00640 1 --2- 172.21.15.86:0/4091609625 >> [v2:172.21.15.144:6800/4087309475,v1:172.21.15.144:6801/4087309475] conn(0x7fd760072940 0x7fd760074d60 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:11.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.956+0000 7fd78ae00640 1 -- 172.21.15.86:0/4091609625 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd76006c940 msgr2=0x7fd76006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:11.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.956+0000 7fd78ae00640 1 --2- 172.21.15.86:0/4091609625 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fd76006c940 0x7fd76006ee00 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fd78413ceb0 tx=0x7fd76c009040 comp rx=0 tx=0).stop 2024-04-20T21:31:11.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.956+0000 7fd78ae00640 1 -- 172.21.15.86:0/4091609625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7680039e0 msgr2=0x7fd7840b4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:11.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.956+0000 7fd78ae00640 1 --2- 172.21.15.86:0/4091609625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd7680039e0 0x7fd7840b4030 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7fd774008fd0 tx=0x7fd774001120 comp rx=0 tx=0).stop 2024-04-20T21:31:11.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.956+0000 7fd788e00640 1 -- 172.21.15.86:0/4091609625 reap_dead start 2024-04-20T21:31:11.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.956+0000 7fd78ae00640 1 -- 172.21.15.86:0/4091609625 shutdown_connections 2024-04-20T21:31:11.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.957+0000 7fd78ae00640 1 -- 172.21.15.86:0/4091609625 >> 172.21.15.86:0/4091609625 conn(0x7fd7840b2180 msgr2=0x7fd78415dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:11.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.957+0000 7fd78ae00640 1 -- 172.21.15.86:0/4091609625 shutdown_connections 2024-04-20T21:31:11.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:11.957+0000 7fd78ae00640 1 -- 172.21.15.86:0/4091609625 wait complete. 2024-04-20T21:31:11.974 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-20T21:31:12.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.175+0000 7f5bd5800640 1 Processor -- start 2024-04-20T21:31:12.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.176+0000 7f5bd5800640 1 -- start start 2024-04-20T21:31:12.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.177+0000 7f5bd5800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bd0073540 0x7f5bd00d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:12.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.177+0000 7f5bd5800640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5bd00cfa00 0x7f5bd00d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:12.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.177+0000 7f5bd5800640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bd00ce1c0 con 0x7f5bd00af870 2024-04-20T21:31:12.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.177+0000 7f5bd5800640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f5bd00ce4a0 con 0x7f5bd00cfa00 2024-04-20T21:31:12.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.177+0000 7f5bd5800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5bd00ce780 con 0x7f5bd0073540 2024-04-20T21:31:12.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.177+0000 7f5bc7400640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f5bd00af870 0x7f5bd015f160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:53174/0 (socket says 172.21.15.86:53174) 2024-04-20T21:31:12.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.177+0000 7f5bd4e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5bd00cfa00 0x7f5bd00d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:12.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.177+0000 7f5bc7400640 1 -- 172.21.15.86:0/3732875998 learned_addr learned my addr 172.21.15.86:0/3732875998 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:31:12.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.178+0000 7f5bcfe00640 1 --2- 172.21.15.86:0/3732875998 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bd0073540 0x7f5bd00d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:12.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.178+0000 7f5bcf400640 1 -- 172.21.15.86:0/3732875998 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 149472396 0 0) 0x7f5bd00ce1c0 con 0x7f5bd00af870 2024-04-20T21:31:12.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.178+0000 7f5bcf400640 1 -- 172.21.15.86:0/3732875998 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5bb4003520 con 0x7f5bd00af870 2024-04-20T21:31:12.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.178+0000 7f5bd4e00640 1 -- 172.21.15.86:0/3732875998 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bd0073540 msgr2=0x7f5bd00d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:12.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.178+0000 7f5bd4e00640 1 --2- 172.21.15.86:0/3732875998 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bd0073540 0x7f5bd00d00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:12.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.178+0000 7f5bd4e00640 1 -- 172.21.15.86:0/3732875998 >> v1:172.21.15.86:6789/0 conn(0x7f5bd00af870 legacy=0x7f5bd015f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:12.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.178+0000 7f5bd4e00640 1 -- 172.21.15.86:0/3732875998 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bd00cea60 con 0x7f5bd00cfa00 2024-04-20T21:31:12.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.179+0000 7f5bd4e00640 1 --2- 172.21.15.86:0/3732875998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bd00cfa00 0x7f5bd00d05f0 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f5bc80097a0 tx=0x7f5bc802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0227151e9f52e19 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:12.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.179+0000 7f5bcf400640 1 -- 172.21.15.86:0/3732875998 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc8035020 con 0x7f5bd00cfa00 2024-04-20T21:31:12.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.179+0000 7f5bcf400640 1 -- 172.21.15.86:0/3732875998 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bc8031810 con 0x7f5bd00cfa00 2024-04-20T21:31:12.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.179+0000 7f5bcf400640 1 -- 172.21.15.86:0/3732875998 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc8031be0 con 0x7f5bd00cfa00 2024-04-20T21:31:12.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.180+0000 7f5bd5800640 1 -- 172.21.15.86:0/3732875998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bd00cfa00 msgr2=0x7f5bd00d05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:12.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.180+0000 7f5bd5800640 1 --2- 172.21.15.86:0/3732875998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bd00cfa00 0x7f5bd00d05f0 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f5bc80097a0 tx=0x7f5bc802b8b0 comp rx=0 tx=0).stop 2024-04-20T21:31:12.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.180+0000 7f5bd5800640 1 -- 172.21.15.86:0/3732875998 shutdown_connections 2024-04-20T21:31:12.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.180+0000 7f5bd5800640 1 --2- 172.21.15.86:0/3732875998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bd00cfa00 0x7f5bd00d05f0 unknown :-1 s=CLOSED pgs=2169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:12.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.180+0000 7f5bd5800640 1 --2- 172.21.15.86:0/3732875998 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bd0073540 0x7f5bd00d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:12.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.180+0000 7f5bd5800640 1 -- 172.21.15.86:0/3732875998 >> 172.21.15.86:0/3732875998 conn(0x7f5bd006bb60 msgr2=0x7f5bd006bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:12.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.180+0000 7f5bd5800640 1 -- 172.21.15.86:0/3732875998 shutdown_connections 2024-04-20T21:31:12.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.180+0000 7f5bd5800640 1 -- 172.21.15.86:0/3732875998 wait complete. 2024-04-20T21:31:12.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.182+0000 7f5bd5800640 1 Processor -- start 2024-04-20T21:31:12.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.182+0000 7f5bd5800640 1 -- start start 2024-04-20T21:31:12.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.183+0000 7f5bd5800640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bd0073540 0x7f5bd00c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:12.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.183+0000 7f5bd5800640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bd00cfa00 0x7f5bd0071a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:12.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.183+0000 7f5bd5800640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bd00c8370 0x7f5bd0071fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:12.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.183+0000 7f5bd5800640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd0072b80 con 0x7f5bd00cfa00 2024-04-20T21:31:12.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.183+0000 7f5bd5800640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd0073920 con 0x7f5bd0073540 2024-04-20T21:31:12.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.183+0000 7f5bd5800640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5bd0099830 con 0x7f5bd00c8370 2024-04-20T21:31:12.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.183+0000 7f5bc7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bd00cfa00 0x7f5bd0071a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:12.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.183+0000 7f5bc7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bd00cfa00 0x7f5bd0071a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33716/0 (socket says 172.21.15.86:33716) 2024-04-20T21:31:12.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.183+0000 7f5bc7400640 1 -- 172.21.15.86:0/4114750993 learned_addr learned my addr 172.21.15.86:0/4114750993 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:31:12.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.184+0000 7f5bd4e00640 1 --2- 172.21.15.86:0/4114750993 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bd00c8370 0x7f5bd0071fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:12.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.184+0000 7f5bcfe00640 1 --2- 172.21.15.86:0/4114750993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bd0073540 0x7f5bd00c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:12.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.184+0000 7f5bc7400640 1 -- 172.21.15.86:0/4114750993 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bd00c8370 msgr2=0x7f5bd0071fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:12.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.184+0000 7f5bc7400640 1 --2- 172.21.15.86:0/4114750993 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bd00c8370 0x7f5bd0071fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:12.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.184+0000 7f5bc7400640 1 -- 172.21.15.86:0/4114750993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bd0073540 msgr2=0x7f5bd00c7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:12.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.184+0000 7f5bc7400640 1 --2- 172.21.15.86:0/4114750993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bd0073540 0x7f5bd00c7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:12.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.184+0000 7f5bc7400640 1 -- 172.21.15.86:0/4114750993 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bd015dc20 con 0x7f5bd00cfa00 2024-04-20T21:31:12.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.184+0000 7f5bcfe00640 1 --2- 172.21.15.86:0/4114750993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bd0073540 0x7f5bd00c7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:31:12.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.184+0000 7f5bc7400640 1 --2- 172.21.15.86:0/4114750993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bd00cfa00 0x7f5bd0071a70 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7f5bb800c430 tx=0x7f5bb800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:12.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.184+0000 7f5bccc00640 1 -- 172.21.15.86:0/4114750993 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8017070 con 0x7f5bd00cfa00 2024-04-20T21:31:12.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.184+0000 7f5bccc00640 1 -- 172.21.15.86:0/4114750993 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bb8005ce0 con 0x7f5bd00cfa00 2024-04-20T21:31:12.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.184+0000 7f5bccc00640 1 -- 172.21.15.86:0/4114750993 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb80072f0 con 0x7f5bd00cfa00 2024-04-20T21:31:12.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.184+0000 7f5bd5800640 1 -- 172.21.15.86:0/4114750993 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bd00c7390 con 0x7f5bd00cfa00 2024-04-20T21:31:12.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.184+0000 7f5bd5800640 1 -- 172.21.15.86:0/4114750993 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bd00c7630 con 0x7f5bd00cfa00 2024-04-20T21:31:12.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.186+0000 7f5bd5800640 1 -- 172.21.15.86:0/4114750993 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bd00730f0 con 0x7f5bd00cfa00 2024-04-20T21:31:12.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.187+0000 7f5bccc00640 1 -- 172.21.15.86:0/4114750993 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5bb801a070 con 0x7f5bd00cfa00 2024-04-20T21:31:12.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.188+0000 7f5bccc00640 1 --2- 172.21.15.86:0/4114750993 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5bac06c8f0 0x7f5bac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:12.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.188+0000 7f5bccc00640 1 -- 172.21.15.86:0/4114750993 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f5bb808adf0 con 0x7f5bd00cfa00 2024-04-20T21:31:12.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.188+0000 7f5bcfe00640 1 --2- 172.21.15.86:0/4114750993 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5bac06c8f0 0x7f5bac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:12.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.189+0000 7f5bcfe00640 1 --2- 172.21.15.86:0/4114750993 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5bac06c8f0 0x7f5bac06edb0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0007330 tx=0x7f5bc000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:12.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.192+0000 7f5bccc00640 1 -- 172.21.15.86:0/4114750993 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5bd00730f0 con 0x7f5bd00cfa00 2024-04-20T21:31:12.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.393+0000 7f5bd5800640 1 -- 172.21.15.86:0/4114750993 --> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f5bd004fcf0 con 0x7f5bac06c8f0 2024-04-20T21:31:12.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.394+0000 7f5bccc00640 1 -- 172.21.15.86:0/4114750993 <== mgr.4107 v2:172.21.15.86:6832/302045240 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f5bd004fcf0 con 0x7f5bac06c8f0 2024-04-20T21:31:12.393 INFO:teuthology.orchestra.run.smithi086.stderr:instructed osd(s) 4 to deep-scrub 2024-04-20T21:31:12.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.396+0000 7f5bd5800640 1 -- 172.21.15.86:0/4114750993 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5bac06c8f0 msgr2=0x7f5bac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:12.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.396+0000 7f5bd5800640 1 --2- 172.21.15.86:0/4114750993 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5bac06c8f0 0x7f5bac06edb0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0007330 tx=0x7f5bc000b040 comp rx=0 tx=0).stop 2024-04-20T21:31:12.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.396+0000 7f5bd5800640 1 -- 172.21.15.86:0/4114750993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bd00cfa00 msgr2=0x7f5bd0071a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:12.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.396+0000 7f5bd5800640 1 --2- 172.21.15.86:0/4114750993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bd00cfa00 0x7f5bd0071a70 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7f5bb800c430 tx=0x7f5bb800c8f0 comp rx=0 tx=0).stop 2024-04-20T21:31:12.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.396+0000 7f5bd5800640 1 -- 172.21.15.86:0/4114750993 shutdown_connections 2024-04-20T21:31:12.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.396+0000 7f5bd5800640 1 --2- 172.21.15.86:0/4114750993 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5bac06c8f0 0x7f5bac06edb0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:12.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.396+0000 7f5bd5800640 1 --2- 172.21.15.86:0/4114750993 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5bd00c8370 0x7f5bd0071fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:12.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.396+0000 7f5bd5800640 1 --2- 172.21.15.86:0/4114750993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5bd00cfa00 0x7f5bd0071a70 unknown :-1 s=CLOSED pgs=2170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:12.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.396+0000 7f5bd5800640 1 --2- 172.21.15.86:0/4114750993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bd0073540 0x7f5bd00c7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:12.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.396+0000 7f5bd5800640 1 -- 172.21.15.86:0/4114750993 >> 172.21.15.86:0/4114750993 conn(0x7f5bd006bb60 msgr2=0x7f5bd00b12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:12.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.397+0000 7f5bd5800640 1 -- 172.21.15.86:0/4114750993 shutdown_connections 2024-04-20T21:31:12.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.397+0000 7f5bd5800640 1 -- 172.21.15.86:0/4114750993 wait complete. 2024-04-20T21:31:12.415 INFO:tasks.ceph:Scrubbing osd.5 2024-04-20T21:31:12.415 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-20T21:31:12.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.618+0000 7fbc87e00640 1 Processor -- start 2024-04-20T21:31:12.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.619+0000 7fbc87e00640 1 -- start start 2024-04-20T21:31:12.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.620+0000 7fbc87e00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbc88153590 0x7fbc880d0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:12.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.620+0000 7fbc87e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc8815cbe0 con 0x7fbc88071ad0 2024-04-20T21:31:12.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.620+0000 7fbc87e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc8815cea0 con 0x7fbc880af870 2024-04-20T21:31:12.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.620+0000 7fbc87e00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7fbc88153970 con 0x7fbc88153590 2024-04-20T21:31:12.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.620+0000 7fbc7e000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fbc88071ad0 0x7fbc880af160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:53186/0 (socket says 172.21.15.86:53186) 2024-04-20T21:31:12.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.620+0000 7fbc86a00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbc88153590 0x7fbc880d0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:12.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.620+0000 7fbc7e000640 1 -- 172.21.15.86:0/1901587442 learned_addr learned my addr 172.21.15.86:0/1901587442 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:31:12.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.621+0000 7fbc86a00640 1 -- 172.21.15.86:0/1901587442 >> v1:172.21.15.144:6789/0 conn(0x7fbc880af870 legacy=0x7fbc8806dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:12.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.621+0000 7fbc86a00640 1 -- 172.21.15.86:0/1901587442 >> v1:172.21.15.86:6789/0 conn(0x7fbc88071ad0 legacy=0x7fbc880af160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:12.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.621+0000 7fbc86a00640 1 -- 172.21.15.86:0/1901587442 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc8815d180 con 0x7fbc88153590 2024-04-20T21:31:12.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.622+0000 7fbc86a00640 1 --2- 172.21.15.86:0/1901587442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc88153590 0x7fbc880d0610 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7fbc780097d0 tx=0x7fbc7802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddf344a32474f5eb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:12.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.622+0000 7fbc86000640 1 -- 172.21.15.86:0/1901587442 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc78035020 con 0x7fbc88153590 2024-04-20T21:31:12.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.622+0000 7fbc86000640 1 -- 172.21.15.86:0/1901587442 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc78031810 con 0x7fbc88153590 2024-04-20T21:31:12.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.622+0000 7fbc86000640 1 -- 172.21.15.86:0/1901587442 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc78031be0 con 0x7fbc88153590 2024-04-20T21:31:12.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.623+0000 7fbc87e00640 1 -- 172.21.15.86:0/1901587442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc88153590 msgr2=0x7fbc880d0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:12.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.623+0000 7fbc87e00640 1 --2- 172.21.15.86:0/1901587442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc88153590 0x7fbc880d0610 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7fbc780097d0 tx=0x7fbc7802b8b0 comp rx=0 tx=0).stop 2024-04-20T21:31:12.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.623+0000 7fbc87e00640 1 -- 172.21.15.86:0/1901587442 shutdown_connections 2024-04-20T21:31:12.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.623+0000 7fbc87e00640 1 --2- 172.21.15.86:0/1901587442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc88153590 0x7fbc880d0610 unknown :-1 s=CLOSED pgs=2172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:12.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.623+0000 7fbc87e00640 1 -- 172.21.15.86:0/1901587442 >> 172.21.15.86:0/1901587442 conn(0x7fbc880b2180 msgr2=0x7fbc880b2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:12.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.623+0000 7fbc87e00640 1 -- 172.21.15.86:0/1901587442 shutdown_connections 2024-04-20T21:31:12.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.623+0000 7fbc87e00640 1 -- 172.21.15.86:0/1901587442 wait complete. 2024-04-20T21:31:12.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.625+0000 7fbc87e00640 1 Processor -- start 2024-04-20T21:31:12.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.625+0000 7fbc87e00640 1 -- start start 2024-04-20T21:31:12.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.626+0000 7fbc87e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc88071ad0 0x7fbc8814d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:12.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.626+0000 7fbc87e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc880af870 0x7fbc88153af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:12.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.626+0000 7fbc87e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc88153590 0x7fbc88154030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:12.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.626+0000 7fbc87e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbc880d0b50 con 0x7fbc88071ad0 2024-04-20T21:31:12.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.626+0000 7fbc87e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8815d740 con 0x7fbc880af870 2024-04-20T21:31:12.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.626+0000 7fbc87e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fbc880d0cd0 con 0x7fbc88153590 2024-04-20T21:31:12.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.626+0000 7fbc86a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc88071ad0 0x7fbc8814d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:12.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.626+0000 7fbc86a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc88071ad0 0x7fbc8814d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33740/0 (socket says 172.21.15.86:33740) 2024-04-20T21:31:12.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.626+0000 7fbc86a00640 1 -- 172.21.15.86:0/2755712040 learned_addr learned my addr 172.21.15.86:0/2755712040 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:31:12.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.627+0000 7fbc86a00640 1 -- 172.21.15.86:0/2755712040 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc88153590 msgr2=0x7fbc88154030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:12.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.627+0000 7fbc86a00640 1 --2- 172.21.15.86:0/2755712040 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fbc88153590 0x7fbc88154030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:12.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.627+0000 7fbc86a00640 1 -- 172.21.15.86:0/2755712040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc880af870 msgr2=0x7fbc88153af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:12.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.627+0000 7fbc86a00640 1 --2- 172.21.15.86:0/2755712040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc880af870 0x7fbc88153af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:12.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.627+0000 7fbc86a00640 1 -- 172.21.15.86:0/2755712040 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc8814c7a0 con 0x7fbc88071ad0 2024-04-20T21:31:12.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.627+0000 7fbc86a00640 1 --2- 172.21.15.86:0/2755712040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc88071ad0 0x7fbc8814d840 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7fbc780097d0 tx=0x7fbc780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:12.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.627+0000 7fbc7f400640 1 -- 172.21.15.86:0/2755712040 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7803ad60 con 0x7fbc88071ad0 2024-04-20T21:31:12.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.627+0000 7fbc7f400640 1 -- 172.21.15.86:0/2755712040 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc780443f0 con 0x7fbc88071ad0 2024-04-20T21:31:12.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.627+0000 7fbc7f400640 1 -- 172.21.15.86:0/2755712040 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc780447e0 con 0x7fbc88071ad0 2024-04-20T21:31:12.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.627+0000 7fbc87e00640 1 -- 172.21.15.86:0/2755712040 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc8814ca80 con 0x7fbc88071ad0 2024-04-20T21:31:12.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.627+0000 7fbc87e00640 1 -- 172.21.15.86:0/2755712040 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc8814d040 con 0x7fbc88071ad0 2024-04-20T21:31:12.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.628+0000 7fbc87e00640 1 -- 172.21.15.86:0/2755712040 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbc8804fcf0 con 0x7fbc88071ad0 2024-04-20T21:31:12.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.630+0000 7fbc7f400640 1 -- 172.21.15.86:0/2755712040 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbc7804b060 con 0x7fbc88071ad0 2024-04-20T21:31:12.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.630+0000 7fbc7f400640 1 --2- 172.21.15.86:0/2755712040 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbc6406c920 0x7fbc6406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:12.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.631+0000 7fbc7f400640 1 -- 172.21.15.86:0/2755712040 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7fbc780adfd0 con 0x7fbc88071ad0 2024-04-20T21:31:12.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.631+0000 7fbc7e000640 1 --2- 172.21.15.86:0/2755712040 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbc6406c920 0x7fbc6406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:12.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.631+0000 7fbc7f400640 1 --2- 172.21.15.86:0/2755712040 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x7fbc64072920 0x7fbc64074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:12.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.631+0000 7fbc7f400640 1 -- 172.21.15.86:0/2755712040 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbc78038ec0 con 0x7fbc64072920 2024-04-20T21:31:12.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.631+0000 7fbc7f400640 1 -- 172.21.15.86:0/2755712040 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=119) ==== 24+0+0 (secure 0 0 0) 0x7fbc780ae380 con 0x7fbc88071ad0 2024-04-20T21:31:12.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.631+0000 7fbc7e000640 1 --2- 172.21.15.86:0/2755712040 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbc6406c920 0x7fbc6406ede0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fbc70000f60 tx=0x7fbc70007a80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:12.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.631+0000 7fbc87400640 1 --2- 172.21.15.86:0/2755712040 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x7fbc64072920 0x7fbc64074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:12.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.633+0000 7fbc87400640 1 --2- 172.21.15.86:0/2755712040 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x7fbc64072920 0x7fbc64074d40 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:12.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.635+0000 7fbc7f400640 1 -- 172.21.15.86:0/2755712040 <== osd.5 v2:172.21.15.144:6808/1635974704 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbc78038ec0 con 0x7fbc64072920 2024-04-20T21:31:12.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.649+0000 7fbc87e00640 1 -- 172.21.15.86:0/2755712040 --> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fbc881507c0 con 0x7fbc64072920 2024-04-20T21:31:12.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.650+0000 7fbc7f400640 1 -- 172.21.15.86:0/2755712040 <== osd.5 v2:172.21.15.144:6808/1635974704 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fbc881507c0 con 0x7fbc64072920 2024-04-20T21:31:12.650 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:31:12.650 INFO:teuthology.orchestra.run.smithi086.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T21:31:12.650 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:31:12.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.651+0000 7fbc87e00640 1 -- 172.21.15.86:0/2755712040 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x7fbc64072920 msgr2=0x7fbc64074d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:12.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.651+0000 7fbc87e00640 1 --2- 172.21.15.86:0/2755712040 >> [v2:172.21.15.144:6808/1635974704,v1:172.21.15.144:6809/1635974704] conn(0x7fbc64072920 0x7fbc64074d40 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:12.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.651+0000 7fbc87e00640 1 -- 172.21.15.86:0/2755712040 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbc6406c920 msgr2=0x7fbc6406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:12.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.651+0000 7fbc87e00640 1 --2- 172.21.15.86:0/2755712040 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fbc6406c920 0x7fbc6406ede0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fbc70000f60 tx=0x7fbc70007a80 comp rx=0 tx=0).stop 2024-04-20T21:31:12.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.651+0000 7fbc87e00640 1 -- 172.21.15.86:0/2755712040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc88071ad0 msgr2=0x7fbc8814d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:12.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.651+0000 7fbc87e00640 1 --2- 172.21.15.86:0/2755712040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbc88071ad0 0x7fbc8814d840 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7fbc780097d0 tx=0x7fbc780046c0 comp rx=0 tx=0).stop 2024-04-20T21:31:12.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.651+0000 7fbc87400640 1 -- 172.21.15.86:0/2755712040 reap_dead start 2024-04-20T21:31:12.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.651+0000 7fbc87e00640 1 -- 172.21.15.86:0/2755712040 shutdown_connections 2024-04-20T21:31:12.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.651+0000 7fbc87e00640 1 -- 172.21.15.86:0/2755712040 >> 172.21.15.86:0/2755712040 conn(0x7fbc880b2180 msgr2=0x7fbc880ad040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:12.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.651+0000 7fbc87e00640 1 -- 172.21.15.86:0/2755712040 shutdown_connections 2024-04-20T21:31:12.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.651+0000 7fbc87e00640 1 -- 172.21.15.86:0/2755712040 wait complete. 2024-04-20T21:31:12.668 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-20T21:31:12.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.870+0000 7f1d52200640 1 Processor -- start 2024-04-20T21:31:12.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.870+0000 7f1d52200640 1 -- start start 2024-04-20T21:31:12.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.871+0000 7f1d52200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1d4c145fe0 0x7f1d4c0d02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:12.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.871+0000 7f1d52200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f1d4c0d0810 0x7f1d4c0cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:12.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.872+0000 7f1d52200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1d4c0cd610 0x7f1d4c160350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:12.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.872+0000 7f1d52200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f1d4c0d00d0 con 0x7f1d4c0cd610 2024-04-20T21:31:12.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.872+0000 7f1d52200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f1d4c0d0c70 con 0x7f1d4c0d0810 2024-04-20T21:31:12.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.872+0000 7f1d52200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1d4c0cdaa0 con 0x7f1d4c145fe0 2024-04-20T21:31:12.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.872+0000 7f1d4be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1d4c0cd610 0x7f1d4c160350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:12.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.872+0000 7f1d4be00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1d4c0cd610 0x7f1d4c160350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33756/0 (socket says 172.21.15.86:33756) 2024-04-20T21:31:12.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.872+0000 7f1d4be00640 1 -- 172.21.15.86:0/3811500799 learned_addr learned my addr 172.21.15.86:0/3811500799 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:31:12.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.872+0000 7f1d4b400640 1 --2- 172.21.15.86:0/3811500799 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1d4c145fe0 0x7f1d4c0d02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:12.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.872+0000 7f1d4aa00640 1 --2- 172.21.15.86:0/3811500799 >> v2:172.21.15.144:3300/0 conn(0x7f1d4c0d0810 0x7f1d4c0cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:12.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.873+0000 7f1d4be00640 1 -- 172.21.15.86:0/3811500799 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1d4c145fe0 msgr2=0x7f1d4c0d02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:12.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.873+0000 7f1d4be00640 1 --2- 172.21.15.86:0/3811500799 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1d4c145fe0 0x7f1d4c0d02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:12.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.873+0000 7f1d4be00640 1 -- 172.21.15.86:0/3811500799 >> v2:172.21.15.144:3300/0 conn(0x7f1d4c0d0810 msgr2=0x7f1d4c0cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:12.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.873+0000 7f1d4be00640 1 --2- 172.21.15.86:0/3811500799 >> v2:172.21.15.144:3300/0 conn(0x7f1d4c0d0810 0x7f1d4c0cd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:12.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.873+0000 7f1d4be00640 1 -- 172.21.15.86:0/3811500799 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d4c15dcc0 con 0x7f1d4c0cd610 2024-04-20T21:31:12.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.874+0000 7f1d4be00640 1 --2- 172.21.15.86:0/3811500799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d4c0cd610 0x7f1d4c160350 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c0d1440 tx=0x7f1d4402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce3892fc0f94d4dd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:12.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.874+0000 7f1d4a000640 1 -- 172.21.15.86:0/3811500799 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d44035020 con 0x7f1d4c0cd610 2024-04-20T21:31:12.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.874+0000 7f1d4a000640 1 -- 172.21.15.86:0/3811500799 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d44031870 con 0x7f1d4c0cd610 2024-04-20T21:31:12.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.874+0000 7f1d4a000640 1 -- 172.21.15.86:0/3811500799 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d44031c40 con 0x7f1d4c0cd610 2024-04-20T21:31:12.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.875+0000 7f1d52200640 1 -- 172.21.15.86:0/3811500799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d4c0cd610 msgr2=0x7f1d4c160350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:12.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.875+0000 7f1d52200640 1 --2- 172.21.15.86:0/3811500799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d4c0cd610 0x7f1d4c160350 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c0d1440 tx=0x7f1d4402b910 comp rx=0 tx=0).stop 2024-04-20T21:31:12.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.875+0000 7f1d52200640 1 -- 172.21.15.86:0/3811500799 shutdown_connections 2024-04-20T21:31:12.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.875+0000 7f1d52200640 1 --2- 172.21.15.86:0/3811500799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d4c0cd610 0x7f1d4c160350 unknown :-1 s=CLOSED pgs=2174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:12.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.875+0000 7f1d52200640 1 --2- 172.21.15.86:0/3811500799 >> v2:172.21.15.144:3300/0 conn(0x7f1d4c0d0810 0x7f1d4c0cd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:12.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.875+0000 7f1d52200640 1 --2- 172.21.15.86:0/3811500799 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1d4c145fe0 0x7f1d4c0d02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:12.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.875+0000 7f1d52200640 1 -- 172.21.15.86:0/3811500799 >> 172.21.15.86:0/3811500799 conn(0x7f1d4c06b980 msgr2=0x7f1d4c06bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:12.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.875+0000 7f1d52200640 1 -- 172.21.15.86:0/3811500799 shutdown_connections 2024-04-20T21:31:12.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.875+0000 7f1d52200640 1 -- 172.21.15.86:0/3811500799 wait complete. 2024-04-20T21:31:12.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.877+0000 7f1d52200640 1 Processor -- start 2024-04-20T21:31:12.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.877+0000 7f1d52200640 1 -- start start 2024-04-20T21:31:12.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.877+0000 7f1d52200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1d4c0cd610 0x7f1d4c0b17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:12.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.878+0000 7f1d52200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d4c0d0810 0x7f1d4c1406b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:12.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.878+0000 7f1d52200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d4c145fe0 0x7f1d4c140bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:12.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.878+0000 7f1d52200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4c15d4c0 con 0x7f1d4c145fe0 2024-04-20T21:31:12.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.878+0000 7f1d52200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4c15d1e0 con 0x7f1d4c0d0810 2024-04-20T21:31:12.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.878+0000 7f1d52200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f1d4c15d7a0 con 0x7f1d4c0cd610 2024-04-20T21:31:12.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.878+0000 7f1d4be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d4c145fe0 0x7f1d4c140bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:12.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.878+0000 7f1d4be00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d4c145fe0 0x7f1d4c140bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33764/0 (socket says 172.21.15.86:33764) 2024-04-20T21:31:12.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.878+0000 7f1d4be00640 1 -- 172.21.15.86:0/2332161056 learned_addr learned my addr 172.21.15.86:0/2332161056 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:31:12.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.878+0000 7f1d4be00640 1 -- 172.21.15.86:0/2332161056 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1d4c0cd610 msgr2=0x7f1d4c0b17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:12.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.878+0000 7f1d4be00640 1 --2- 172.21.15.86:0/2332161056 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1d4c0cd610 0x7f1d4c0b17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:12.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.878+0000 7f1d4be00640 1 -- 172.21.15.86:0/2332161056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d4c0d0810 msgr2=0x7f1d4c1406b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:12.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.878+0000 7f1d4be00640 1 --2- 172.21.15.86:0/2332161056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d4c0d0810 0x7f1d4c1406b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:12.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.878+0000 7f1d4be00640 1 -- 172.21.15.86:0/2332161056 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d4c1e6ec0 con 0x7f1d4c145fe0 2024-04-20T21:31:12.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.879+0000 7f1d4be00640 1 --2- 172.21.15.86:0/2332161056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d4c145fe0 0x7f1d4c140bf0 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7f1d44009960 tx=0x7f1d440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:12.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.879+0000 7f1d43400640 1 -- 172.21.15.86:0/2332161056 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d44040070 con 0x7f1d4c145fe0 2024-04-20T21:31:12.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.879+0000 7f1d43400640 1 -- 172.21.15.86:0/2332161056 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d44030c80 con 0x7f1d4c145fe0 2024-04-20T21:31:12.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.879+0000 7f1d52200640 1 -- 172.21.15.86:0/2332161056 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d4c0ac480 con 0x7f1d4c145fe0 2024-04-20T21:31:12.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.879+0000 7f1d43400640 1 -- 172.21.15.86:0/2332161056 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d44035040 con 0x7f1d4c145fe0 2024-04-20T21:31:12.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.879+0000 7f1d52200640 1 -- 172.21.15.86:0/2332161056 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d4c0aca40 con 0x7f1d4c145fe0 2024-04-20T21:31:12.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.880+0000 7f1d52200640 1 -- 172.21.15.86:0/2332161056 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d4c0689e0 con 0x7f1d4c145fe0 2024-04-20T21:31:12.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.881+0000 7f1d43400640 1 -- 172.21.15.86:0/2332161056 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1d44038020 con 0x7f1d4c145fe0 2024-04-20T21:31:12.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.883+0000 7f1d43400640 1 --2- 172.21.15.86:0/2332161056 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1d2c06c8f0 0x7f1d2c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:12.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.883+0000 7f1d43400640 1 -- 172.21.15.86:0/2332161056 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f1d440ad760 con 0x7f1d4c145fe0 2024-04-20T21:31:12.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.883+0000 7f1d4b400640 1 --2- 172.21.15.86:0/2332161056 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1d2c06c8f0 0x7f1d2c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:12.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.884+0000 7f1d4b400640 1 --2- 172.21.15.86:0/2332161056 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1d2c06c8f0 0x7f1d2c06edb0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c0070b0 tx=0x7f1d3c007040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:12.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:12.888+0000 7f1d43400640 1 -- 172.21.15.86:0/2332161056 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1d4407bd40 con 0x7f1d4c145fe0 2024-04-20T21:31:13.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.095+0000 7f1d52200640 1 -- 172.21.15.86:0/2332161056 --> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f1d4c0acc80 con 0x7f1d2c06c8f0 2024-04-20T21:31:13.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.096+0000 7f1d43400640 1 -- 172.21.15.86:0/2332161056 <== mgr.4107 v2:172.21.15.86:6832/302045240 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f1d3c007e10 con 0x7f1d2c06c8f0 2024-04-20T21:31:13.096 INFO:teuthology.orchestra.run.smithi086.stderr:instructed osd(s) 5 to deep-scrub 2024-04-20T21:31:13.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.099+0000 7f1d52200640 1 -- 172.21.15.86:0/2332161056 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1d2c06c8f0 msgr2=0x7f1d2c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:13.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.099+0000 7f1d52200640 1 --2- 172.21.15.86:0/2332161056 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1d2c06c8f0 0x7f1d2c06edb0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c0070b0 tx=0x7f1d3c007040 comp rx=0 tx=0).stop 2024-04-20T21:31:13.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.099+0000 7f1d52200640 1 -- 172.21.15.86:0/2332161056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d4c145fe0 msgr2=0x7f1d4c140bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:13.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.099+0000 7f1d52200640 1 --2- 172.21.15.86:0/2332161056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d4c145fe0 0x7f1d4c140bf0 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7f1d44009960 tx=0x7f1d440046c0 comp rx=0 tx=0).stop 2024-04-20T21:31:13.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.099+0000 7f1d52200640 1 -- 172.21.15.86:0/2332161056 shutdown_connections 2024-04-20T21:31:13.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.099+0000 7f1d52200640 1 --2- 172.21.15.86:0/2332161056 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f1d2c06c8f0 0x7f1d2c06edb0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:13.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.099+0000 7f1d52200640 1 --2- 172.21.15.86:0/2332161056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d4c145fe0 0x7f1d4c140bf0 unknown :-1 s=CLOSED pgs=2175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:13.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.099+0000 7f1d52200640 1 --2- 172.21.15.86:0/2332161056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d4c0d0810 0x7f1d4c1406b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:13.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.099+0000 7f1d52200640 1 --2- 172.21.15.86:0/2332161056 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f1d4c0cd610 0x7f1d4c0b17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:13.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.099+0000 7f1d52200640 1 -- 172.21.15.86:0/2332161056 >> 172.21.15.86:0/2332161056 conn(0x7f1d4c06b980 msgr2=0x7f1d4c15f490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:13.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.100+0000 7f1d52200640 1 -- 172.21.15.86:0/2332161056 shutdown_connections 2024-04-20T21:31:13.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.100+0000 7f1d52200640 1 -- 172.21.15.86:0/2332161056 wait complete. 2024-04-20T21:31:13.120 INFO:tasks.ceph:Scrubbing osd.6 2024-04-20T21:31:13.120 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-20T21:31:13.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.324+0000 7f2203a00640 1 Processor -- start 2024-04-20T21:31:13.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.324+0000 7f2203a00640 1 -- start start 2024-04-20T21:31:13.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.326+0000 7f2203a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21fc06f0b0 0x7f21fc145a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:13.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.326+0000 7f2203a00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f21fc145fd0 0x7f21fc15cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:13.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.326+0000 7f2203a00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21fc06e010 con 0x7f21fc071d00 2024-04-20T21:31:13.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.326+0000 7f2203a00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f21fc06e2f0 con 0x7f21fc145fd0 2024-04-20T21:31:13.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.326+0000 7f2203a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f21fc06e5d0 con 0x7f21fc06f0b0 2024-04-20T21:31:13.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.326+0000 7f2201000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f21fc071d00 0x7f21fc0720e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:53200/0 (socket says 172.21.15.86:53200) 2024-04-20T21:31:13.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.326+0000 7f2201000640 1 -- 172.21.15.86:0/844742523 learned_addr learned my addr 172.21.15.86:0/844742523 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:31:13.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.327+0000 7f2201a00640 1 --2- 172.21.15.86:0/844742523 >> v2:172.21.15.144:3300/0 conn(0x7f21fc145fd0 0x7f21fc15cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:13.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.327+0000 7f21fbe00640 1 --2- 172.21.15.86:0/844742523 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21fc06f0b0 0x7f21fc145a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:13.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.327+0000 7f21fb400640 1 -- 172.21.15.86:0/844742523 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1147339818 0 0) 0x7f21fc06e010 con 0x7f21fc071d00 2024-04-20T21:31:13.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.327+0000 7f21fb400640 1 -- 172.21.15.86:0/844742523 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f21e0003520 con 0x7f21fc071d00 2024-04-20T21:31:13.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.327+0000 7f21fb400640 1 -- 172.21.15.86:0/844742523 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 185159426 0 0) 0x7f21e0003520 con 0x7f21fc071d00 2024-04-20T21:31:13.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.327+0000 7f21fb400640 1 -- 172.21.15.86:0/844742523 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21fc06f0b0 msgr2=0x7f21fc145a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:13.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.327+0000 7f21fb400640 1 --2- 172.21.15.86:0/844742523 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21fc06f0b0 0x7f21fc145a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:13.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.327+0000 7f21fb400640 1 -- 172.21.15.86:0/844742523 >> v2:172.21.15.144:3300/0 conn(0x7f21fc145fd0 msgr2=0x7f21fc15cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:13.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.327+0000 7f21fb400640 1 --2- 172.21.15.86:0/844742523 >> v2:172.21.15.144:3300/0 conn(0x7f21fc145fd0 0x7f21fc15cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:13.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.327+0000 7f21fb400640 1 -- 172.21.15.86:0/844742523 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21fc06e8b0 con 0x7f21fc071d00 2024-04-20T21:31:13.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.327+0000 7f21fb400640 1 -- 172.21.15.86:0/844742523 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f21ec001670 con 0x7f21fc071d00 2024-04-20T21:31:13.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.327+0000 7f21fb400640 1 -- 172.21.15.86:0/844742523 >> v1:172.21.15.86:6789/0 conn(0x7f21fc071d00 legacy=0x7f21fc0720e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:13.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.327+0000 7f21fb400640 1 --2- 172.21.15.86:0/844742523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21e00039b0 0x7f21e0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:13.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.327+0000 7f21fb400640 1 --2- 172.21.15.86:0/844742523 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21e0004800 0x7f21e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:13.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.327+0000 7f21fb400640 1 --2- 172.21.15.86:0/844742523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21e00051c0 0x7f21e000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:13.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.327+0000 7f21fb400640 1 -- 172.21.15.86:0/844742523 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f21fc06e2f0 con 0x7f21e00039b0 2024-04-20T21:31:13.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.327+0000 7f21fb400640 1 -- 172.21.15.86:0/844742523 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f21fc06e5d0 con 0x7f21e00051c0 2024-04-20T21:31:13.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.328+0000 7f21fb400640 1 -- 172.21.15.86:0/844742523 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f21e000bf30 con 0x7f21e0004800 2024-04-20T21:31:13.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.328+0000 7f21fbe00640 1 --2- 172.21.15.86:0/844742523 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21fc06f0b0 0x7f21fc145a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:31:13.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.328+0000 7f2201000640 1 --2- 172.21.15.86:0/844742523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21e00039b0 0x7f21e0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:13.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.328+0000 7f2201a00640 1 --2- 172.21.15.86:0/844742523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21e00051c0 0x7f21e000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:13.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.328+0000 7f2201000640 1 -- 172.21.15.86:0/844742523 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21e0004800 msgr2=0x7f21e0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:13.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.328+0000 7f2201000640 1 --2- 172.21.15.86:0/844742523 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21e0004800 0x7f21e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:13.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.328+0000 7f2201000640 1 -- 172.21.15.86:0/844742523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21e00051c0 msgr2=0x7f21e000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:13.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.328+0000 7f2201000640 1 --2- 172.21.15.86:0/844742523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21e00051c0 0x7f21e000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:13.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.328+0000 7f2201000640 1 -- 172.21.15.86:0/844742523 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21e0003520 con 0x7f21e00039b0 2024-04-20T21:31:13.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.330+0000 7f2201000640 1 --2- 172.21.15.86:0/844742523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21e00039b0 0x7f21e0003dd0 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7f21ec00c210 tx=0x7f21e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff9771f8611cf3a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:13.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.330+0000 7f21fb400640 1 -- 172.21.15.86:0/844742523 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec007d50 con 0x7f21e00039b0 2024-04-20T21:31:13.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.330+0000 7f21fb400640 1 -- 172.21.15.86:0/844742523 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21ec030070 con 0x7f21e00039b0 2024-04-20T21:31:13.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.330+0000 7f21fb400640 1 -- 172.21.15.86:0/844742523 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec02cd80 con 0x7f21e00039b0 2024-04-20T21:31:13.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.331+0000 7f2203a00640 1 -- 172.21.15.86:0/844742523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21e00039b0 msgr2=0x7f21e0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:13.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.331+0000 7f2203a00640 1 --2- 172.21.15.86:0/844742523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21e00039b0 0x7f21e0003dd0 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7f21ec00c210 tx=0x7f21e0005830 comp rx=0 tx=0).stop 2024-04-20T21:31:13.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.331+0000 7f2201a00640 1 -- 172.21.15.86:0/844742523 reap_dead start 2024-04-20T21:31:13.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.331+0000 7f2203a00640 1 -- 172.21.15.86:0/844742523 shutdown_connections 2024-04-20T21:31:13.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.331+0000 7f2203a00640 1 -- 172.21.15.86:0/844742523 >> 172.21.15.86:0/844742523 conn(0x7f21fc0b2180 msgr2=0x7f21fc0b2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:13.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.331+0000 7f2203a00640 1 -- 172.21.15.86:0/844742523 shutdown_connections 2024-04-20T21:31:13.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.331+0000 7f2203a00640 1 -- 172.21.15.86:0/844742523 wait complete. 2024-04-20T21:31:13.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.333+0000 7f2203a00640 1 Processor -- start 2024-04-20T21:31:13.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.333+0000 7f2203a00640 1 -- start start 2024-04-20T21:31:13.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.334+0000 7f2203a00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21e00039b0 0x7f21fc1e6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:13.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.334+0000 7f2203a00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21fc1e8750 0x7f21fc15c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:13.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.334+0000 7f2203a00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21fc1e5470 0x7f21fc1e7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:13.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.334+0000 7f2203a00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f21fc06dc30 con 0x7f21fc1e8750 2024-04-20T21:31:13.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.334+0000 7f2203a00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f21fc06f510 con 0x7f21fc1e5470 2024-04-20T21:31:13.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.334+0000 7f2203a00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f21fc06ddb0 con 0x7f21e00039b0 2024-04-20T21:31:13.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.334+0000 7f21fbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21fc1e8750 0x7f21fc15c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:13.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.334+0000 7f21fbe00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21fc1e8750 0x7f21fc15c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33790/0 (socket says 172.21.15.86:33790) 2024-04-20T21:31:13.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.334+0000 7f21fbe00640 1 -- 172.21.15.86:0/3661446969 learned_addr learned my addr 172.21.15.86:0/3661446969 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:31:13.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.334+0000 7f2201000640 1 --2- 172.21.15.86:0/3661446969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21e00039b0 0x7f21fc1e6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:13.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.334+0000 7f2201a00640 1 --2- 172.21.15.86:0/3661446969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21fc1e5470 0x7f21fc1e7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:13.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.335+0000 7f21fbe00640 1 -- 172.21.15.86:0/3661446969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21e00039b0 msgr2=0x7f21fc1e6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:13.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.335+0000 7f21fbe00640 1 --2- 172.21.15.86:0/3661446969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21e00039b0 0x7f21fc1e6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:13.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.335+0000 7f21fbe00640 1 -- 172.21.15.86:0/3661446969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21fc1e5470 msgr2=0x7f21fc1e7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:13.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.335+0000 7f21fbe00640 1 --2- 172.21.15.86:0/3661446969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21fc1e5470 0x7f21fc1e7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:13.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.335+0000 7f21fbe00640 1 -- 172.21.15.86:0/3661446969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21fc1e5de0 con 0x7f21fc1e8750 2024-04-20T21:31:13.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.335+0000 7f2201000640 1 --2- 172.21.15.86:0/3661446969 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f21e00039b0 0x7f21fc1e6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:31:13.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.335+0000 7f2201a00640 1 --2- 172.21.15.86:0/3661446969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21fc1e5470 0x7f21fc1e7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T21:31:13.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.335+0000 7f21fbe00640 1 --2- 172.21.15.86:0/3661446969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21fc1e8750 0x7f21fc15c150 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7f21e400e560 tx=0x7f21e400ea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:13.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.335+0000 7f21f8c00640 1 -- 172.21.15.86:0/3661446969 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e4017020 con 0x7f21fc1e8750 2024-04-20T21:31:13.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.335+0000 7f21f8c00640 1 -- 172.21.15.86:0/3661446969 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21e4005b90 con 0x7f21fc1e8750 2024-04-20T21:31:13.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.335+0000 7f2203a00640 1 -- 172.21.15.86:0/3661446969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21fc1e60c0 con 0x7f21fc1e8750 2024-04-20T21:31:13.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.335+0000 7f21f8c00640 1 -- 172.21.15.86:0/3661446969 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e400f040 con 0x7f21fc1e8750 2024-04-20T21:31:13.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.335+0000 7f2203a00640 1 -- 172.21.15.86:0/3661446969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21fc1e6680 con 0x7f21fc1e8750 2024-04-20T21:31:13.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.336+0000 7f2203a00640 1 -- 172.21.15.86:0/3661446969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f21fc0689e0 con 0x7f21fc1e8750 2024-04-20T21:31:13.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.337+0000 7f21f8c00640 1 -- 172.21.15.86:0/3661446969 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f21e401a020 con 0x7f21fc1e8750 2024-04-20T21:31:13.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.338+0000 7f21f8c00640 1 --2- 172.21.15.86:0/3661446969 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f21d806c8f0 0x7f21d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:13.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.338+0000 7f21f8c00640 1 -- 172.21.15.86:0/3661446969 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f21e408b580 con 0x7f21fc1e8750 2024-04-20T21:31:13.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.338+0000 7f2201000640 1 --2- 172.21.15.86:0/3661446969 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f21d806c8f0 0x7f21d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:13.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.339+0000 7f21f8c00640 1 --2- 172.21.15.86:0/3661446969 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x7f21d80728f0 0x7f21d8074d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:13.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.339+0000 7f21f8c00640 1 -- 172.21.15.86:0/3661446969 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f21d80753c0 con 0x7f21d80728f0 2024-04-20T21:31:13.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.339+0000 7f21f8c00640 1 -- 172.21.15.86:0/3661446969 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=119) ==== 24+0+0 (secure 0 0 0) 0x7f21e4092050 con 0x7f21fc1e8750 2024-04-20T21:31:13.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.339+0000 7f2201000640 1 --2- 172.21.15.86:0/3661446969 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f21d806c8f0 0x7f21d806edb0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f21ec00c210 tx=0x7f21ec007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:13.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.339+0000 7f2201a00640 1 --2- 172.21.15.86:0/3661446969 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x7f21d80728f0 0x7f21d8074d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:13.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.340+0000 7f2201a00640 1 --2- 172.21.15.86:0/3661446969 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x7f21d80728f0 0x7f21d8074d10 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:13.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.342+0000 7f21f8c00640 1 -- 172.21.15.86:0/3661446969 <== osd.6 v2:172.21.15.144:6824/1893803079 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f21d80753c0 con 0x7f21d80728f0 2024-04-20T21:31:13.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.357+0000 7f2203a00640 1 -- 172.21.15.86:0/3661446969 --> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f21fc144a40 con 0x7f21d80728f0 2024-04-20T21:31:13.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.358+0000 7f21f8c00640 1 -- 172.21.15.86:0/3661446969 <== osd.6 v2:172.21.15.144:6824/1893803079 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f21fc144a40 con 0x7f21d80728f0 2024-04-20T21:31:13.358 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:31:13.358 INFO:teuthology.orchestra.run.smithi086.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T21:31:13.358 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:31:13.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.358+0000 7f2203a00640 1 -- 172.21.15.86:0/3661446969 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x7f21d80728f0 msgr2=0x7f21d8074d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:13.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.358+0000 7f2203a00640 1 --2- 172.21.15.86:0/3661446969 >> [v2:172.21.15.144:6824/1893803079,v1:172.21.15.144:6825/1893803079] conn(0x7f21d80728f0 0x7f21d8074d10 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:13.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.359+0000 7f2203a00640 1 -- 172.21.15.86:0/3661446969 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f21d806c8f0 msgr2=0x7f21d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:13.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.359+0000 7f2203a00640 1 --2- 172.21.15.86:0/3661446969 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f21d806c8f0 0x7f21d806edb0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f21ec00c210 tx=0x7f21ec007000 comp rx=0 tx=0).stop 2024-04-20T21:31:13.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.359+0000 7f2203a00640 1 -- 172.21.15.86:0/3661446969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21fc1e8750 msgr2=0x7f21fc15c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:13.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.359+0000 7f2203a00640 1 --2- 172.21.15.86:0/3661446969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21fc1e8750 0x7f21fc15c150 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7f21e400e560 tx=0x7f21e400ea20 comp rx=0 tx=0).stop 2024-04-20T21:31:13.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.359+0000 7f2201a00640 1 -- 172.21.15.86:0/3661446969 reap_dead start 2024-04-20T21:31:13.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.359+0000 7f2203a00640 1 -- 172.21.15.86:0/3661446969 shutdown_connections 2024-04-20T21:31:13.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.359+0000 7f2203a00640 1 -- 172.21.15.86:0/3661446969 >> 172.21.15.86:0/3661446969 conn(0x7f21fc0b2180 msgr2=0x7f21fc142bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:13.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.359+0000 7f2203a00640 1 -- 172.21.15.86:0/3661446969 shutdown_connections 2024-04-20T21:31:13.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.359+0000 7f2203a00640 1 -- 172.21.15.86:0/3661446969 wait complete. 2024-04-20T21:31:13.376 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-20T21:31:13.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.576+0000 7f96a7200640 1 Processor -- start 2024-04-20T21:31:13.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.576+0000 7f96a7200640 1 -- start start 2024-04-20T21:31:13.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.578+0000 7f96a7200640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f96a00af7e0 0x7f96a00b4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:13.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.578+0000 7f96a7200640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f96a00cd220 0x7f96a00b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:13.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.578+0000 7f96a7200640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96a0160160 con 0x7f96a015c590 2024-04-20T21:31:13.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.578+0000 7f96a7200640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f96a0160440 con 0x7f96a00af7e0 2024-04-20T21:31:13.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.578+0000 7f96a7200640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f96a0160720 con 0x7f96a00cd220 2024-04-20T21:31:13.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.578+0000 7f96a5400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f96a00af7e0 0x7f96a00b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:13.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.578+0000 7f96a5400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f96a00af7e0 0x7f96a00b4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33804/0 (socket says 172.21.15.86:33804) 2024-04-20T21:31:13.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.578+0000 7f96a5400640 1 -- 172.21.15.86:0/1347245141 learned_addr learned my addr 172.21.15.86:0/1347245141 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:31:13.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.578+0000 7f96a6800640 1 --2- 172.21.15.86:0/1347245141 >> v2:172.21.15.144:3300/0 conn(0x7f96a00cd220 0x7f96a00b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:13.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.579+0000 7f96a5400640 1 -- 172.21.15.86:0/1347245141 >> v2:172.21.15.144:3300/0 conn(0x7f96a00cd220 msgr2=0x7f96a00b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:13.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.579+0000 7f96a5400640 1 --2- 172.21.15.86:0/1347245141 >> v2:172.21.15.144:3300/0 conn(0x7f96a00cd220 0x7f96a00b46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:13.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.579+0000 7f96a5400640 1 -- 172.21.15.86:0/1347245141 >> v1:172.21.15.144:6789/0 conn(0x7f96a015c590 legacy=0x7f96a00ccb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:13.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.579+0000 7f96a5400640 1 -- 172.21.15.86:0/1347245141 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96a0160a00 con 0x7f96a00af7e0 2024-04-20T21:31:13.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.580+0000 7f96a5400640 1 --2- 172.21.15.86:0/1347245141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96a00af7e0 0x7f96a00b4190 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7f96a00c7170 tx=0x7f969002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=561e9b8a8e74001 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:13.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.580+0000 7f96a4a00640 1 -- 172.21.15.86:0/1347245141 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9690035020 con 0x7f96a00af7e0 2024-04-20T21:31:13.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.580+0000 7f96a4a00640 1 -- 172.21.15.86:0/1347245141 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9690031870 con 0x7f96a00af7e0 2024-04-20T21:31:13.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.580+0000 7f96a4a00640 1 -- 172.21.15.86:0/1347245141 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9690031c40 con 0x7f96a00af7e0 2024-04-20T21:31:13.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.580+0000 7f96a7200640 1 -- 172.21.15.86:0/1347245141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96a00af7e0 msgr2=0x7f96a00b4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:13.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.580+0000 7f96a7200640 1 --2- 172.21.15.86:0/1347245141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96a00af7e0 0x7f96a00b4190 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7f96a00c7170 tx=0x7f969002b910 comp rx=0 tx=0).stop 2024-04-20T21:31:13.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.581+0000 7f96a7200640 1 -- 172.21.15.86:0/1347245141 shutdown_connections 2024-04-20T21:31:13.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.581+0000 7f96a7200640 1 --2- 172.21.15.86:0/1347245141 >> v2:172.21.15.144:3300/0 conn(0x7f96a00cd220 0x7f96a00b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:13.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.581+0000 7f96a7200640 1 --2- 172.21.15.86:0/1347245141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96a00af7e0 0x7f96a00b4190 unknown :-1 s=CLOSED pgs=2179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:13.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.581+0000 7f96a7200640 1 -- 172.21.15.86:0/1347245141 >> 172.21.15.86:0/1347245141 conn(0x7f96a006bb60 msgr2=0x7f96a006bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:13.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.581+0000 7f96a7200640 1 -- 172.21.15.86:0/1347245141 shutdown_connections 2024-04-20T21:31:13.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.581+0000 7f96a7200640 1 -- 172.21.15.86:0/1347245141 wait complete. 2024-04-20T21:31:13.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.582+0000 7f96a7200640 1 Processor -- start 2024-04-20T21:31:13.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.582+0000 7f96a7200640 1 -- start start 2024-04-20T21:31:13.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.583+0000 7f96a7200640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96a00af7e0 0x7f96a00ae290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:13.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.583+0000 7f96a7200640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96a00cd220 0x7f96a015caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:13.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.583+0000 7f96a7200640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96a015c590 0x7f96a00c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:13.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.583+0000 7f96a7200640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f96a015c970 con 0x7f96a00cd220 2024-04-20T21:31:13.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.583+0000 7f96a7200640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f96a006f3d0 con 0x7f96a015c590 2024-04-20T21:31:13.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.583+0000 7f96a7200640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f96a0160fc0 con 0x7f96a00af7e0 2024-04-20T21:31:13.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.584+0000 7f96a5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96a00cd220 0x7f96a015caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:13.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.584+0000 7f96a5400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96a00cd220 0x7f96a015caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33808/0 (socket says 172.21.15.86:33808) 2024-04-20T21:31:13.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.584+0000 7f96a5400640 1 -- 172.21.15.86:0/3380530115 learned_addr learned my addr 172.21.15.86:0/3380530115 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:31:13.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.584+0000 7f96a5400640 1 -- 172.21.15.86:0/3380530115 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96a00af7e0 msgr2=0x7f96a00ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:13.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.584+0000 7f96a5400640 1 --2- 172.21.15.86:0/3380530115 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96a00af7e0 0x7f96a00ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:13.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.584+0000 7f96a5400640 1 -- 172.21.15.86:0/3380530115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96a015c590 msgr2=0x7f96a00c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:13.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.584+0000 7f96a5400640 1 --2- 172.21.15.86:0/3380530115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96a015c590 0x7f96a00c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:13.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.584+0000 7f96a5400640 1 -- 172.21.15.86:0/3380530115 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96a00ca420 con 0x7f96a00cd220 2024-04-20T21:31:13.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.584+0000 7f96a5400640 1 --2- 172.21.15.86:0/3380530115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96a00cd220 0x7f96a015caf0 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7f9690000c00 tx=0x7f96900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:13.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.585+0000 7f969a000640 1 -- 172.21.15.86:0/3380530115 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9690040070 con 0x7f96a00cd220 2024-04-20T21:31:13.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.585+0000 7f969a000640 1 -- 172.21.15.86:0/3380530115 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9690030d00 con 0x7f96a00cd220 2024-04-20T21:31:13.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.585+0000 7f96a7200640 1 -- 172.21.15.86:0/3380530115 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96a00ca700 con 0x7f96a00cd220 2024-04-20T21:31:13.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.585+0000 7f969a000640 1 -- 172.21.15.86:0/3380530115 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9690035040 con 0x7f96a00cd220 2024-04-20T21:31:13.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.585+0000 7f96a7200640 1 -- 172.21.15.86:0/3380530115 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96a00cafa0 con 0x7f96a00cd220 2024-04-20T21:31:13.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.586+0000 7f96a7200640 1 -- 172.21.15.86:0/3380530115 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96a00ca8d0 con 0x7f96a00cd220 2024-04-20T21:31:13.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.587+0000 7f969a000640 1 -- 172.21.15.86:0/3380530115 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f969004b060 con 0x7f96a00cd220 2024-04-20T21:31:13.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.588+0000 7f969a000640 1 --2- 172.21.15.86:0/3380530115 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f968406c920 0x7f968406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:13.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.588+0000 7f969a000640 1 -- 172.21.15.86:0/3380530115 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f96900ad720 con 0x7f96a00cd220 2024-04-20T21:31:13.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.588+0000 7f96a5e00640 1 --2- 172.21.15.86:0/3380530115 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f968406c920 0x7f968406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:13.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.589+0000 7f96a5e00640 1 --2- 172.21.15.86:0/3380530115 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f968406c920 0x7f968406ede0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f9694006fd0 tx=0x7f9694006bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:13.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.593+0000 7f969a000640 1 -- 172.21.15.86:0/3380530115 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f969007bd90 con 0x7f96a00cd220 2024-04-20T21:31:13.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.793+0000 7f96a7200640 1 -- 172.21.15.86:0/3380530115 --> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f96a00ce580 con 0x7f968406c920 2024-04-20T21:31:13.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.794+0000 7f969a000640 1 -- 172.21.15.86:0/3380530115 <== mgr.4107 v2:172.21.15.86:6832/302045240 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f96a00ce580 con 0x7f968406c920 2024-04-20T21:31:13.794 INFO:teuthology.orchestra.run.smithi086.stderr:instructed osd(s) 6 to deep-scrub 2024-04-20T21:31:13.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.796+0000 7f96a7200640 1 -- 172.21.15.86:0/3380530115 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f968406c920 msgr2=0x7f968406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:13.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.796+0000 7f96a7200640 1 --2- 172.21.15.86:0/3380530115 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f968406c920 0x7f968406ede0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f9694006fd0 tx=0x7f9694006bc0 comp rx=0 tx=0).stop 2024-04-20T21:31:13.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.796+0000 7f96a7200640 1 -- 172.21.15.86:0/3380530115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96a00cd220 msgr2=0x7f96a015caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:13.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.796+0000 7f96a7200640 1 --2- 172.21.15.86:0/3380530115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96a00cd220 0x7f96a015caf0 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7f9690000c00 tx=0x7f96900046c0 comp rx=0 tx=0).stop 2024-04-20T21:31:13.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.796+0000 7f96a7200640 1 -- 172.21.15.86:0/3380530115 shutdown_connections 2024-04-20T21:31:13.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.796+0000 7f96a7200640 1 --2- 172.21.15.86:0/3380530115 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f968406c920 0x7f968406ede0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:13.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.797+0000 7f96a7200640 1 --2- 172.21.15.86:0/3380530115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96a015c590 0x7f96a00c7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:13.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.797+0000 7f96a7200640 1 --2- 172.21.15.86:0/3380530115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f96a00cd220 0x7f96a015caf0 unknown :-1 s=CLOSED pgs=2180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:13.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.797+0000 7f96a7200640 1 --2- 172.21.15.86:0/3380530115 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f96a00af7e0 0x7f96a00ae290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:13.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.797+0000 7f96a7200640 1 -- 172.21.15.86:0/3380530115 >> 172.21.15.86:0/3380530115 conn(0x7f96a006bb60 msgr2=0x7f96a00b0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:13.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.797+0000 7f96a7200640 1 -- 172.21.15.86:0/3380530115 shutdown_connections 2024-04-20T21:31:13.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:13.797+0000 7f96a7200640 1 -- 172.21.15.86:0/3380530115 wait complete. 2024-04-20T21:31:13.814 INFO:tasks.ceph:Scrubbing osd.7 2024-04-20T21:31:13.815 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-20T21:31:14.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.018+0000 7f5ef7e00640 1 Processor -- start 2024-04-20T21:31:14.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.018+0000 7f5ef7e00640 1 -- start start 2024-04-20T21:31:14.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.020+0000 7f5ef7e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ef8161ef0 0x7f5ef80cee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:14.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.020+0000 7f5ef7e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f5ef806d340 0x7f5ef806d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:14.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.020+0000 7f5ef7e00640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ef81622d0 con 0x7f5ef80b6060 2024-04-20T21:31:14.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.020+0000 7f5ef7e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f5ef80d4f70 con 0x7f5ef806d340 2024-04-20T21:31:14.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.020+0000 7f5ef7e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5ef80b3e20 con 0x7f5ef8161ef0 2024-04-20T21:31:14.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.020+0000 7f5ef6000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f5ef80b6060 0x7f5ef80b6480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:53204/0 (socket says 172.21.15.86:53204) 2024-04-20T21:31:14.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.020+0000 7f5ef6000640 1 -- 172.21.15.86:0/9385497 learned_addr learned my addr 172.21.15.86:0/9385497 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:31:14.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.021+0000 7f5ef7400640 1 --2- 172.21.15.86:0/9385497 >> v2:172.21.15.144:3300/0 conn(0x7f5ef806d340 0x7f5ef806d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:14.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.021+0000 7f5ef6a00640 1 --2- 172.21.15.86:0/9385497 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ef8161ef0 0x7f5ef80cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:14.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.021+0000 7f5ef5600640 1 -- 172.21.15.86:0/9385497 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3450573218 0 0) 0x7f5ef81622d0 con 0x7f5ef80b6060 2024-04-20T21:31:14.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.021+0000 7f5ef5600640 1 -- 172.21.15.86:0/9385497 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5edc003520 con 0x7f5ef80b6060 2024-04-20T21:31:14.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.021+0000 7f5ef5600640 1 -- 172.21.15.86:0/9385497 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2536176189 0 0) 0x7f5edc003520 con 0x7f5ef80b6060 2024-04-20T21:31:14.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.021+0000 7f5ef5600640 1 -- 172.21.15.86:0/9385497 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ef8161ef0 msgr2=0x7f5ef80cee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:14.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.021+0000 7f5ef5600640 1 --2- 172.21.15.86:0/9385497 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ef8161ef0 0x7f5ef80cee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:14.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.021+0000 7f5ef5600640 1 -- 172.21.15.86:0/9385497 >> v2:172.21.15.144:3300/0 conn(0x7f5ef806d340 msgr2=0x7f5ef806d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:14.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.021+0000 7f5ef5600640 1 --2- 172.21.15.86:0/9385497 >> v2:172.21.15.144:3300/0 conn(0x7f5ef806d340 0x7f5ef806d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:14.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.021+0000 7f5ef5600640 1 -- 172.21.15.86:0/9385497 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ef80b3860 con 0x7f5ef80b6060 2024-04-20T21:31:14.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.021+0000 7f5ef5600640 1 -- 172.21.15.86:0/9385497 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f5ee40017f0 con 0x7f5ef80b6060 2024-04-20T21:31:14.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.021+0000 7f5ef5600640 1 -- 172.21.15.86:0/9385497 >> v1:172.21.15.86:6789/0 conn(0x7f5ef80b6060 legacy=0x7f5ef80b6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:14.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.022+0000 7f5ef5600640 1 --2- 172.21.15.86:0/9385497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5edc0039b0 0x7f5edc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:14.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.022+0000 7f5ef5600640 1 --2- 172.21.15.86:0/9385497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5edc004800 0x7f5edc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:14.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.022+0000 7f5ef5600640 1 --2- 172.21.15.86:0/9385497 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ef8161ef0 0x7f5edc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:14.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.022+0000 7f5ef5600640 1 -- 172.21.15.86:0/9385497 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef80d4f70 con 0x7f5edc0039b0 2024-04-20T21:31:14.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.022+0000 7f5ef5600640 1 -- 172.21.15.86:0/9385497 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef80b3e20 con 0x7f5edc004800 2024-04-20T21:31:14.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.022+0000 7f5ef5600640 1 -- 172.21.15.86:0/9385497 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5edc00b9c0 con 0x7f5ef8161ef0 2024-04-20T21:31:14.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.022+0000 7f5ef6a00640 1 --2- 172.21.15.86:0/9385497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5edc0039b0 0x7f5edc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:14.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.022+0000 7f5ef7400640 1 --2- 172.21.15.86:0/9385497 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ef8161ef0 0x7f5edc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:14.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.022+0000 7f5ef6a00640 1 -- 172.21.15.86:0/9385497 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ef8161ef0 msgr2=0x7f5edc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:14.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.022+0000 7f5ef6a00640 1 --2- 172.21.15.86:0/9385497 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ef8161ef0 0x7f5edc0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:14.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.022+0000 7f5ef6a00640 1 -- 172.21.15.86:0/9385497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5edc004800 msgr2=0x7f5edc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:14.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.022+0000 7f5ef6a00640 1 --2- 172.21.15.86:0/9385497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5edc004800 0x7f5edc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:14.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.022+0000 7f5ef6a00640 1 -- 172.21.15.86:0/9385497 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5edc003520 con 0x7f5edc0039b0 2024-04-20T21:31:14.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.023+0000 7f5ef6a00640 1 --2- 172.21.15.86:0/9385497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5edc0039b0 0x7f5edc003dd0 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8009ac0 tx=0x7f5edc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe0e81775c7569c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:14.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.024+0000 7f5ef5600640 1 -- 172.21.15.86:0/9385497 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee8039070 con 0x7f5edc0039b0 2024-04-20T21:31:14.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.024+0000 7f5ef5600640 1 -- 172.21.15.86:0/9385497 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ee802c6b0 con 0x7f5edc0039b0 2024-04-20T21:31:14.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.024+0000 7f5ef5600640 1 -- 172.21.15.86:0/9385497 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee8009690 con 0x7f5edc0039b0 2024-04-20T21:31:14.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.024+0000 7f5ef7e00640 1 -- 172.21.15.86:0/9385497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5edc0039b0 msgr2=0x7f5edc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:14.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.024+0000 7f5ef7e00640 1 --2- 172.21.15.86:0/9385497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5edc0039b0 0x7f5edc003dd0 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8009ac0 tx=0x7f5edc005830 comp rx=0 tx=0).stop 2024-04-20T21:31:14.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.024+0000 7f5ef7400640 1 -- 172.21.15.86:0/9385497 reap_dead start 2024-04-20T21:31:14.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.024+0000 7f5ef7e00640 1 -- 172.21.15.86:0/9385497 shutdown_connections 2024-04-20T21:31:14.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.024+0000 7f5ef7e00640 1 -- 172.21.15.86:0/9385497 >> 172.21.15.86:0/9385497 conn(0x7f5ef808bf70 msgr2=0x7f5ef80d45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:14.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.024+0000 7f5ef7e00640 1 -- 172.21.15.86:0/9385497 shutdown_connections 2024-04-20T21:31:14.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.025+0000 7f5ef7e00640 1 -- 172.21.15.86:0/9385497 wait complete. 2024-04-20T21:31:14.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.026+0000 7f5ef7e00640 1 Processor -- start 2024-04-20T21:31:14.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.026+0000 7f5ef7e00640 1 -- start start 2024-04-20T21:31:14.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.027+0000 7f5ef7e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5edc0039b0 0x7f5ef8156af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:14.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.027+0000 7f5ef7e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ef80ca6f0 0x7f5ef80b7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:14.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.027+0000 7f5ef7e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ef80b83a0 0x7f5ef806da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:14.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.027+0000 7f5ef7e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef806ce20 con 0x7f5ef80b83a0 2024-04-20T21:31:14.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.027+0000 7f5ef7e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef806cb40 con 0x7f5edc0039b0 2024-04-20T21:31:14.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.027+0000 7f5ef7e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f5ef80b34c0 con 0x7f5ef80ca6f0 2024-04-20T21:31:14.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.028+0000 7f5ef7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ef80b83a0 0x7f5ef806da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:14.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.028+0000 7f5ef7400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ef80b83a0 0x7f5ef806da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33826/0 (socket says 172.21.15.86:33826) 2024-04-20T21:31:14.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.028+0000 7f5ef7400640 1 -- 172.21.15.86:0/4189769783 learned_addr learned my addr 172.21.15.86:0/4189769783 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:31:14.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.028+0000 7f5ef7400640 1 -- 172.21.15.86:0/4189769783 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ef80ca6f0 msgr2=0x7f5ef80b7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:14.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.028+0000 7f5ef7400640 1 --2- 172.21.15.86:0/4189769783 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f5ef80ca6f0 0x7f5ef80b7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:14.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.028+0000 7f5ef7400640 1 -- 172.21.15.86:0/4189769783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5edc0039b0 msgr2=0x7f5ef8156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:14.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.028+0000 7f5ef7400640 1 --2- 172.21.15.86:0/4189769783 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5edc0039b0 0x7f5ef8156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:14.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.028+0000 7f5ef7400640 1 -- 172.21.15.86:0/4189769783 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ef80c9650 con 0x7f5ef80b83a0 2024-04-20T21:31:14.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.028+0000 7f5ef7400640 1 --2- 172.21.15.86:0/4189769783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ef80b83a0 0x7f5ef806da90 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7f5ef80c8030 tx=0x7f5ef000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:14.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.029+0000 7f5eeea00640 1 -- 172.21.15.86:0/4189769783 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef0003990 con 0x7f5ef80b83a0 2024-04-20T21:31:14.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.029+0000 7f5eeea00640 1 -- 172.21.15.86:0/4189769783 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ef0003b50 con 0x7f5ef80b83a0 2024-04-20T21:31:14.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.029+0000 7f5eeea00640 1 -- 172.21.15.86:0/4189769783 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef0004020 con 0x7f5ef80b83a0 2024-04-20T21:31:14.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.029+0000 7f5ef7e00640 1 -- 172.21.15.86:0/4189769783 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ef80c9930 con 0x7f5ef80b83a0 2024-04-20T21:31:14.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.029+0000 7f5ef7e00640 1 -- 172.21.15.86:0/4189769783 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ef80c9ef0 con 0x7f5ef80b83a0 2024-04-20T21:31:14.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.030+0000 7f5ef7e00640 1 -- 172.21.15.86:0/4189769783 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5ef804fcf0 con 0x7f5ef80b83a0 2024-04-20T21:31:14.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.031+0000 7f5eeea00640 1 -- 172.21.15.86:0/4189769783 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5ef0005d10 con 0x7f5ef80b83a0 2024-04-20T21:31:14.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.032+0000 7f5eeea00640 1 --2- 172.21.15.86:0/4189769783 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5ed806c920 0x7f5ed806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:14.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.032+0000 7f5eeea00640 1 -- 172.21.15.86:0/4189769783 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f5ef008bb10 con 0x7f5ef80b83a0 2024-04-20T21:31:14.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.032+0000 7f5ef6a00640 1 --2- 172.21.15.86:0/4189769783 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5ed806c920 0x7f5ed806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:14.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.032+0000 7f5eeea00640 1 --2- 172.21.15.86:0/4189769783 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x7f5ed8072920 0x7f5ed8074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:14.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.032+0000 7f5eeea00640 1 -- 172.21.15.86:0/4189769783 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5ef0016e50 con 0x7f5ed8072920 2024-04-20T21:31:14.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.032+0000 7f5eeea00640 1 -- 172.21.15.86:0/4189769783 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=119) ==== 24+0+0 (secure 0 0 0) 0x7f5ef00902c0 con 0x7f5ef80b83a0 2024-04-20T21:31:14.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.033+0000 7f5ef6000640 1 --2- 172.21.15.86:0/4189769783 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x7f5ed8072920 0x7f5ed8074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:14.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.033+0000 7f5ef6a00640 1 --2- 172.21.15.86:0/4189769783 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5ed806c920 0x7f5ed806ede0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8036000 tx=0x7f5ee802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:14.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.034+0000 7f5ef6000640 1 --2- 172.21.15.86:0/4189769783 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x7f5ed8072920 0x7f5ed8074d40 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:14.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.036+0000 7f5eeea00640 1 -- 172.21.15.86:0/4189769783 <== osd.7 v2:172.21.15.144:6816/2100967057 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5ef0016e50 con 0x7f5ed8072920 2024-04-20T21:31:14.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.050+0000 7f5ef7e00640 1 -- 172.21.15.86:0/4189769783 --> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f5ef80b6cd0 con 0x7f5ed8072920 2024-04-20T21:31:14.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.052+0000 7f5eeea00640 1 -- 172.21.15.86:0/4189769783 <== osd.7 v2:172.21.15.144:6816/2100967057 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f5ef80b6cd0 con 0x7f5ed8072920 2024-04-20T21:31:14.052 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-20T21:31:14.052 INFO:teuthology.orchestra.run.smithi086.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T21:31:14.052 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-20T21:31:14.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.052+0000 7f5ef7e00640 1 -- 172.21.15.86:0/4189769783 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x7f5ed8072920 msgr2=0x7f5ed8074d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:14.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.052+0000 7f5ef7e00640 1 --2- 172.21.15.86:0/4189769783 >> [v2:172.21.15.144:6816/2100967057,v1:172.21.15.144:6817/2100967057] conn(0x7f5ed8072920 0x7f5ed8074d40 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:14.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.052+0000 7f5ef7e00640 1 -- 172.21.15.86:0/4189769783 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5ed806c920 msgr2=0x7f5ed806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:14.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.052+0000 7f5ef7e00640 1 --2- 172.21.15.86:0/4189769783 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f5ed806c920 0x7f5ed806ede0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8036000 tx=0x7f5ee802e000 comp rx=0 tx=0).stop 2024-04-20T21:31:14.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.052+0000 7f5ef7e00640 1 -- 172.21.15.86:0/4189769783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ef80b83a0 msgr2=0x7f5ef806da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:14.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.052+0000 7f5ef7e00640 1 --2- 172.21.15.86:0/4189769783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ef80b83a0 0x7f5ef806da90 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7f5ef80c8030 tx=0x7f5ef000f4a0 comp rx=0 tx=0).stop 2024-04-20T21:31:14.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.053+0000 7f5ef7400640 1 -- 172.21.15.86:0/4189769783 reap_dead start 2024-04-20T21:31:14.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.053+0000 7f5ef7e00640 1 -- 172.21.15.86:0/4189769783 shutdown_connections 2024-04-20T21:31:14.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.053+0000 7f5ef7e00640 1 -- 172.21.15.86:0/4189769783 >> 172.21.15.86:0/4189769783 conn(0x7f5ef808bf70 msgr2=0x7f5ef80d0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:14.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.053+0000 7f5ef7e00640 1 -- 172.21.15.86:0/4189769783 shutdown_connections 2024-04-20T21:31:14.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.053+0000 7f5ef7e00640 1 -- 172.21.15.86:0/4189769783 wait complete. 2024-04-20T21:31:14.069 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-20T21:31:14.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.267+0000 7f750f000640 1 Processor -- start 2024-04-20T21:31:14.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.267+0000 7f750f000640 1 -- start start 2024-04-20T21:31:14.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.269+0000 7f750f000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f7508145fe0 0x7f75080d02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:14.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.269+0000 7f750f000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f75080d0810 0x7f75080cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:14.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.269+0000 7f750f000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75080cda40 con 0x7f75080cd610 2024-04-20T21:31:14.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.269+0000 7f750f000640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f75080d00d0 con 0x7f75080d0810 2024-04-20T21:31:14.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.269+0000 7f750f000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f75080d0c70 con 0x7f7508145fe0 2024-04-20T21:31:14.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.269+0000 7f7507400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f75080d0810 0x7f75080cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:14.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.269+0000 7f7507400640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f75080d0810 0x7f75080cd0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33836/0 (socket says 172.21.15.86:33836) 2024-04-20T21:31:14.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.269+0000 7f7507400640 1 -- 172.21.15.86:0/3316471933 learned_addr learned my addr 172.21.15.86:0/3316471933 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:31:14.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.269+0000 7f7507e00640 1 --2- 172.21.15.86:0/3316471933 >> v2:172.21.15.144:3300/0 conn(0x7f7508145fe0 0x7f75080d02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:14.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.270+0000 7f7507400640 1 -- 172.21.15.86:0/3316471933 >> v2:172.21.15.144:3300/0 conn(0x7f7508145fe0 msgr2=0x7f75080d02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:14.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.270+0000 7f7507400640 1 --2- 172.21.15.86:0/3316471933 >> v2:172.21.15.144:3300/0 conn(0x7f7508145fe0 0x7f75080d02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:14.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.270+0000 7f7507400640 1 -- 172.21.15.86:0/3316471933 >> v1:172.21.15.144:6789/0 conn(0x7f75080cd610 legacy=0x7f7508160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:14.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.270+0000 7f7507400640 1 -- 172.21.15.86:0/3316471933 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f750815dcc0 con 0x7f75080d0810 2024-04-20T21:31:14.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.271+0000 7f7507400640 1 --2- 172.21.15.86:0/3316471933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75080d0810 0x7f75080cd0d0 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7f750806d5c0 tx=0x7f74f402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5658687e33084f3c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:14.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.271+0000 7f7506a00640 1 -- 172.21.15.86:0/3316471933 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f4035020 con 0x7f75080d0810 2024-04-20T21:31:14.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.271+0000 7f7506a00640 1 -- 172.21.15.86:0/3316471933 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74f4031870 con 0x7f75080d0810 2024-04-20T21:31:14.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.271+0000 7f7506a00640 1 -- 172.21.15.86:0/3316471933 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f4031c40 con 0x7f75080d0810 2024-04-20T21:31:14.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.272+0000 7f750f000640 1 -- 172.21.15.86:0/3316471933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75080d0810 msgr2=0x7f75080cd0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:14.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.272+0000 7f750f000640 1 --2- 172.21.15.86:0/3316471933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75080d0810 0x7f75080cd0d0 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7f750806d5c0 tx=0x7f74f402b910 comp rx=0 tx=0).stop 2024-04-20T21:31:14.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.272+0000 7f750f000640 1 -- 172.21.15.86:0/3316471933 shutdown_connections 2024-04-20T21:31:14.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.272+0000 7f750f000640 1 --2- 172.21.15.86:0/3316471933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75080d0810 0x7f75080cd0d0 unknown :-1 s=CLOSED pgs=2184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:14.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.272+0000 7f750f000640 1 --2- 172.21.15.86:0/3316471933 >> v2:172.21.15.144:3300/0 conn(0x7f7508145fe0 0x7f75080d02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:14.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.272+0000 7f750f000640 1 -- 172.21.15.86:0/3316471933 >> 172.21.15.86:0/3316471933 conn(0x7f750806b980 msgr2=0x7f750806bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:14.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.272+0000 7f750f000640 1 -- 172.21.15.86:0/3316471933 shutdown_connections 2024-04-20T21:31:14.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.272+0000 7f750f000640 1 -- 172.21.15.86:0/3316471933 wait complete. 2024-04-20T21:31:14.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.274+0000 7f750f000640 1 Processor -- start 2024-04-20T21:31:14.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.274+0000 7f750f000640 1 -- start start 2024-04-20T21:31:14.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.275+0000 7f750f000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75080cd610 0x7f75080b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:14.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.275+0000 7f750f000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75080d0810 0x7f75080b2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:14.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.275+0000 7f750f000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7508145fe0 0x7f7508135c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:14.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.275+0000 7f750f000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f750815d1e0 con 0x7f75080cd610 2024-04-20T21:31:14.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.275+0000 7f750f000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f750815d4c0 con 0x7f75080d0810 2024-04-20T21:31:14.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.275+0000 7f750f000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f750815d7a0 con 0x7f7508145fe0 2024-04-20T21:31:14.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.275+0000 7f7507e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75080cd610 0x7f75080b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:14.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.275+0000 7f7507e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75080cd610 0x7f75080b2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33844/0 (socket says 172.21.15.86:33844) 2024-04-20T21:31:14.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.275+0000 7f7507e00640 1 -- 172.21.15.86:0/2520267715 learned_addr learned my addr 172.21.15.86:0/2520267715 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:31:14.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.275+0000 7f7507e00640 1 -- 172.21.15.86:0/2520267715 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7508145fe0 msgr2=0x7f7508135c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:14.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.275+0000 7f7507e00640 1 --2- 172.21.15.86:0/2520267715 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7508145fe0 0x7f7508135c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:14.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.275+0000 7f7507e00640 1 -- 172.21.15.86:0/2520267715 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75080d0810 msgr2=0x7f75080b2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:14.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.275+0000 7f7507e00640 1 --2- 172.21.15.86:0/2520267715 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75080d0810 0x7f75080b2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:14.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.275+0000 7f7507e00640 1 -- 172.21.15.86:0/2520267715 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75081430d0 con 0x7f75080cd610 2024-04-20T21:31:14.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.276+0000 7f7507e00640 1 --2- 172.21.15.86:0/2520267715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75080cd610 0x7f75080b2170 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7f74f800d5f0 tx=0x7f74f800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:14.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.276+0000 7f74ffe00640 1 -- 172.21.15.86:0/2520267715 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f80125f0 con 0x7f75080cd610 2024-04-20T21:31:14.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.276+0000 7f74ffe00640 1 -- 172.21.15.86:0/2520267715 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74f8019070 con 0x7f75080cd610 2024-04-20T21:31:14.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.276+0000 7f74ffe00640 1 -- 172.21.15.86:0/2520267715 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f80129d0 con 0x7f75080cd610 2024-04-20T21:31:14.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.276+0000 7f750f000640 1 -- 172.21.15.86:0/2520267715 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75081f7d70 con 0x7f75080cd610 2024-04-20T21:31:14.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.276+0000 7f750f000640 1 -- 172.21.15.86:0/2520267715 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75081f8330 con 0x7f75080cd610 2024-04-20T21:31:14.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.277+0000 7f750f000640 1 -- 172.21.15.86:0/2520267715 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75080689e0 con 0x7f75080cd610 2024-04-20T21:31:14.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.278+0000 7f74ffe00640 1 -- 172.21.15.86:0/2520267715 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f74f8012b70 con 0x7f75080cd610 2024-04-20T21:31:14.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.279+0000 7f74ffe00640 1 --2- 172.21.15.86:0/2520267715 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f74e806c8f0 0x7f74e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:14.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.279+0000 7f74ffe00640 1 -- 172.21.15.86:0/2520267715 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f74f808bdf0 con 0x7f75080cd610 2024-04-20T21:31:14.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.280+0000 7f7507400640 1 --2- 172.21.15.86:0/2520267715 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f74e806c8f0 0x7f74e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:14.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.280+0000 7f7507400640 1 --2- 172.21.15.86:0/2520267715 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f74e806c8f0 0x7f74e806edb0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f74f4036000 tx=0x7f74f4030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:14.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.284+0000 7f74ffe00640 1 -- 172.21.15.86:0/2520267715 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f75081430d0 con 0x7f75080cd610 2024-04-20T21:31:14.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.486+0000 7f750f000640 1 -- 172.21.15.86:0/2520267715 --> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f75080cdf20 con 0x7f74e806c8f0 2024-04-20T21:31:14.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.488+0000 7f74ffe00640 1 -- 172.21.15.86:0/2520267715 <== mgr.4107 v2:172.21.15.86:6832/302045240 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f75080cdf20 con 0x7f74e806c8f0 2024-04-20T21:31:14.487 INFO:teuthology.orchestra.run.smithi086.stderr:instructed osd(s) 7 to deep-scrub 2024-04-20T21:31:14.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.490+0000 7f750f000640 1 -- 172.21.15.86:0/2520267715 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f74e806c8f0 msgr2=0x7f74e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:14.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.490+0000 7f750f000640 1 --2- 172.21.15.86:0/2520267715 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f74e806c8f0 0x7f74e806edb0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f74f4036000 tx=0x7f74f4030000 comp rx=0 tx=0).stop 2024-04-20T21:31:14.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.490+0000 7f750f000640 1 -- 172.21.15.86:0/2520267715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75080cd610 msgr2=0x7f75080b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:14.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.490+0000 7f750f000640 1 --2- 172.21.15.86:0/2520267715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75080cd610 0x7f75080b2170 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7f74f800d5f0 tx=0x7f74f800dab0 comp rx=0 tx=0).stop 2024-04-20T21:31:14.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.490+0000 7f750f000640 1 -- 172.21.15.86:0/2520267715 shutdown_connections 2024-04-20T21:31:14.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.490+0000 7f750f000640 1 --2- 172.21.15.86:0/2520267715 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f74e806c8f0 0x7f74e806edb0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:14.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.490+0000 7f750f000640 1 --2- 172.21.15.86:0/2520267715 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f7508145fe0 0x7f7508135c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:14.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.490+0000 7f750f000640 1 --2- 172.21.15.86:0/2520267715 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75080d0810 0x7f75080b2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:14.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.490+0000 7f750f000640 1 --2- 172.21.15.86:0/2520267715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75080cd610 0x7f75080b2170 unknown :-1 s=CLOSED pgs=2185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:14.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.490+0000 7f750f000640 1 -- 172.21.15.86:0/2520267715 >> 172.21.15.86:0/2520267715 conn(0x7f750806b980 msgr2=0x7f75080b4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:14.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.491+0000 7f750f000640 1 -- 172.21.15.86:0/2520267715 shutdown_connections 2024-04-20T21:31:14.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.491+0000 7f750f000640 1 -- 172.21.15.86:0/2520267715 wait complete. 2024-04-20T21:31:14.509 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T21:31:14.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.709+0000 7f83a0400640 1 Processor -- start 2024-04-20T21:31:14.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.709+0000 7f83a0400640 1 -- start start 2024-04-20T21:31:14.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.711+0000 7f83a0400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8398072120 0x7f83980adb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:14.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.711+0000 7f83a0400640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83981485c0 con 0x7f83980b4210 2024-04-20T21:31:14.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.711+0000 7f83a0400640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8398072770 con 0x7f83980d0450 2024-04-20T21:31:14.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.711+0000 7f83a0400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8398072550 con 0x7f8398072120 2024-04-20T21:31:14.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.711+0000 7f839d000640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f83980b4210 0x7f839806eb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:53220/0 (socket says 172.21.15.86:53220) 2024-04-20T21:31:14.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.711+0000 7f839d000640 1 -- 172.21.15.86:0/2871044682 learned_addr learned my addr 172.21.15.86:0/2871044682 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:31:14.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.712+0000 7f839e400640 1 --2- 172.21.15.86:0/2871044682 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8398072120 0x7f83980adb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:14.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.712+0000 7f8397e00640 1 -- 172.21.15.86:0/2871044682 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4254990165 0 0) 0x7f83981485c0 con 0x7f83980b4210 2024-04-20T21:31:14.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.712+0000 7f8397e00640 1 -- 172.21.15.86:0/2871044682 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8380003520 con 0x7f83980b4210 2024-04-20T21:31:14.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.712+0000 7f8397e00640 1 -- 172.21.15.86:0/2871044682 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2962479839 0 0) 0x7f8380003520 con 0x7f83980b4210 2024-04-20T21:31:14.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.712+0000 7f8397e00640 1 -- 172.21.15.86:0/2871044682 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8398072120 msgr2=0x7f83980adb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:14.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.712+0000 7f8397e00640 1 --2- 172.21.15.86:0/2871044682 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8398072120 0x7f83980adb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:14.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.712+0000 7f8397e00640 1 -- 172.21.15.86:0/2871044682 >> v1:172.21.15.144:6789/0 conn(0x7f83980d0450 legacy=0x7f83980d0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:14.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.712+0000 7f8397e00640 1 -- 172.21.15.86:0/2871044682 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8398072a50 con 0x7f83980b4210 2024-04-20T21:31:14.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.712+0000 7f8397e00640 1 -- 172.21.15.86:0/2871044682 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7f8384001670 con 0x7f83980b4210 2024-04-20T21:31:14.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.712+0000 7f8397e00640 1 -- 172.21.15.86:0/2871044682 >> v1:172.21.15.86:6789/0 conn(0x7f83980b4210 legacy=0x7f839806eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:14.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.712+0000 7f8397e00640 1 --2- 172.21.15.86:0/2871044682 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83800039e0 0x7f8380003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:14.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.712+0000 7f8397e00640 1 --2- 172.21.15.86:0/2871044682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8380004800 0x7f8380004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:14.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.712+0000 7f8397e00640 1 --2- 172.21.15.86:0/2871044682 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8398072120 0x7f83800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:14.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.712+0000 7f8397e00640 1 -- 172.21.15.86:0/2871044682 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8398072550 con 0x7f8380004800 2024-04-20T21:31:14.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.712+0000 7f8397e00640 1 -- 172.21.15.86:0/2871044682 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f838000ba70 con 0x7f83800039e0 2024-04-20T21:31:14.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.712+0000 7f8397e00640 1 -- 172.21.15.86:0/2871044682 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f838000bc70 con 0x7f8398072120 2024-04-20T21:31:14.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.713+0000 7f839d000640 1 --2- 172.21.15.86:0/2871044682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8380004800 0x7f8380004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:14.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.713+0000 7f839da00640 1 --2- 172.21.15.86:0/2871044682 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83800039e0 0x7f8380003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:14.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.713+0000 7f839d000640 1 -- 172.21.15.86:0/2871044682 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8398072120 msgr2=0x7f83800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:14.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.713+0000 7f839d000640 1 --2- 172.21.15.86:0/2871044682 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8398072120 0x7f83800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:14.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.713+0000 7f839d000640 1 -- 172.21.15.86:0/2871044682 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83800039e0 msgr2=0x7f8380003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:14.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.713+0000 7f839d000640 1 --2- 172.21.15.86:0/2871044682 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83800039e0 0x7f8380003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:14.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.713+0000 7f839d000640 1 -- 172.21.15.86:0/2871044682 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8380003520 con 0x7f8380004800 2024-04-20T21:31:14.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.714+0000 7f839d000640 1 --2- 172.21.15.86:0/2871044682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8380004800 0x7f8380004c80 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7f8384001eb0 tx=0x7f8380005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=687bea611cc31675 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:14.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.714+0000 7f8397e00640 1 -- 172.21.15.86:0/2871044682 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8384007d30 con 0x7f8380004800 2024-04-20T21:31:14.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.714+0000 7f8397e00640 1 -- 172.21.15.86:0/2871044682 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8384030070 con 0x7f8380004800 2024-04-20T21:31:14.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.714+0000 7f8397e00640 1 -- 172.21.15.86:0/2871044682 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8384005d30 con 0x7f8380004800 2024-04-20T21:31:14.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.715+0000 7f83a0400640 1 -- 172.21.15.86:0/2871044682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8380004800 msgr2=0x7f8380004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:14.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.715+0000 7f83a0400640 1 --2- 172.21.15.86:0/2871044682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8380004800 0x7f8380004c80 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7f8384001eb0 tx=0x7f8380005830 comp rx=0 tx=0).stop 2024-04-20T21:31:14.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.715+0000 7f839e400640 1 -- 172.21.15.86:0/2871044682 reap_dead start 2024-04-20T21:31:14.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.715+0000 7f83a0400640 1 -- 172.21.15.86:0/2871044682 shutdown_connections 2024-04-20T21:31:14.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.715+0000 7f83a0400640 1 -- 172.21.15.86:0/2871044682 >> 172.21.15.86:0/2871044682 conn(0x7f83980b2400 msgr2=0x7f83980b2810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:14.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.715+0000 7f83a0400640 1 -- 172.21.15.86:0/2871044682 shutdown_connections 2024-04-20T21:31:14.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.715+0000 7f83a0400640 1 -- 172.21.15.86:0/2871044682 wait complete. 2024-04-20T21:31:14.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.717+0000 7f83a0400640 1 Processor -- start 2024-04-20T21:31:14.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.717+0000 7f83a0400640 1 -- start start 2024-04-20T21:31:14.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.718+0000 7f83a0400640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8380004800 0x7f83980c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:14.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.718+0000 7f83a0400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f839813f0b0 0x7f83980c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:14.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.718+0000 7f83a0400640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83980c7ab0 0x7f83980c7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:14.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.718+0000 7f83a0400640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f839806e630 con 0x7f8380004800 2024-04-20T21:31:14.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.718+0000 7f83a0400640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8398073010 con 0x7f83980c7ab0 2024-04-20T21:31:14.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.718+0000 7f83a0400640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f83980d11b0 con 0x7f839813f0b0 2024-04-20T21:31:14.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.718+0000 7f839da00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8380004800 0x7f83980c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:14.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.718+0000 7f839da00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8380004800 0x7f83980c8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33856/0 (socket says 172.21.15.86:33856) 2024-04-20T21:31:14.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.718+0000 7f839da00640 1 -- 172.21.15.86:0/956306214 learned_addr learned my addr 172.21.15.86:0/956306214 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:31:14.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.719+0000 7f839d000640 1 --2- 172.21.15.86:0/956306214 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f839813f0b0 0x7f83980c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:14.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.719+0000 7f839e400640 1 --2- 172.21.15.86:0/956306214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83980c7ab0 0x7f83980c7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:14.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.719+0000 7f839da00640 1 -- 172.21.15.86:0/956306214 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f839813f0b0 msgr2=0x7f83980c90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:14.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.719+0000 7f839da00640 1 --2- 172.21.15.86:0/956306214 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f839813f0b0 0x7f83980c90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:14.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.719+0000 7f839da00640 1 -- 172.21.15.86:0/956306214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83980c7ab0 msgr2=0x7f83980c7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:14.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.719+0000 7f839da00640 1 --2- 172.21.15.86:0/956306214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83980c7ab0 0x7f83980c7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:14.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.719+0000 7f839da00640 1 -- 172.21.15.86:0/956306214 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83980ca7c0 con 0x7f8380004800 2024-04-20T21:31:14.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.719+0000 7f839da00640 1 --2- 172.21.15.86:0/956306214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8380004800 0x7f83980c8b60 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7f838c00e8c0 tx=0x7f838c00ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:14.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.719+0000 7f8395600640 1 -- 172.21.15.86:0/956306214 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838c003b80 con 0x7f8380004800 2024-04-20T21:31:14.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.719+0000 7f8395600640 1 -- 172.21.15.86:0/956306214 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f838c018070 con 0x7f8380004800 2024-04-20T21:31:14.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.719+0000 7f8395600640 1 -- 172.21.15.86:0/956306214 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838c0050d0 con 0x7f8380004800 2024-04-20T21:31:14.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.719+0000 7f83a0400640 1 -- 172.21.15.86:0/956306214 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83980caa80 con 0x7f8380004800 2024-04-20T21:31:14.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.719+0000 7f83a0400640 1 -- 172.21.15.86:0/956306214 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83980cb040 con 0x7f8380004800 2024-04-20T21:31:14.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.721+0000 7f83a0400640 1 -- 172.21.15.86:0/956306214 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83980689e0 con 0x7f8380004800 2024-04-20T21:31:14.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.721+0000 7f8395600640 1 -- 172.21.15.86:0/956306214 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f838c005270 con 0x7f8380004800 2024-04-20T21:31:14.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.722+0000 7f8395600640 1 --2- 172.21.15.86:0/956306214 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f837806c8f0 0x7f837806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:14.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.722+0000 7f8395600640 1 -- 172.21.15.86:0/956306214 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f838c08b960 con 0x7f8380004800 2024-04-20T21:31:14.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.723+0000 7f839d000640 1 --2- 172.21.15.86:0/956306214 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f837806c8f0 0x7f837806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:14.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.723+0000 7f839d000640 1 --2- 172.21.15.86:0/956306214 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f837806c8f0 0x7f837806edb0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f8384034000 tx=0x7f8384009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:14.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.727+0000 7f8395600640 1 -- 172.21.15.86:0/956306214 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f838c01a310 con 0x7f8380004800 2024-04-20T21:31:14.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.923+0000 7f83a0400640 1 -- 172.21.15.86:0/956306214 --> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f839813e9b0 con 0x7f837806c8f0 2024-04-20T21:31:14.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.925+0000 7f8395600640 1 -- 172.21.15.86:0/956306214 <== mgr.4107 v2:172.21.15.86:6832/302045240 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13370 (secure 0 0 0) 0x7f839813e9b0 con 0x7f837806c8f0 2024-04-20T21:31:14.925 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:31:14.926 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-20T21:31:14.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.928+0000 7f83a0400640 1 -- 172.21.15.86:0/956306214 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f837806c8f0 msgr2=0x7f837806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:14.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.928+0000 7f83a0400640 1 --2- 172.21.15.86:0/956306214 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f837806c8f0 0x7f837806edb0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f8384034000 tx=0x7f8384009030 comp rx=0 tx=0).stop 2024-04-20T21:31:14.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.928+0000 7f83a0400640 1 -- 172.21.15.86:0/956306214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8380004800 msgr2=0x7f83980c8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:14.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.928+0000 7f83a0400640 1 --2- 172.21.15.86:0/956306214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8380004800 0x7f83980c8b60 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7f838c00e8c0 tx=0x7f838c00ed80 comp rx=0 tx=0).stop 2024-04-20T21:31:14.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.928+0000 7f83a0400640 1 -- 172.21.15.86:0/956306214 shutdown_connections 2024-04-20T21:31:14.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.928+0000 7f83a0400640 1 --2- 172.21.15.86:0/956306214 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f837806c8f0 0x7f837806edb0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:14.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.928+0000 7f83a0400640 1 --2- 172.21.15.86:0/956306214 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f83980c7ab0 0x7f83980c7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:14.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.928+0000 7f83a0400640 1 --2- 172.21.15.86:0/956306214 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f839813f0b0 0x7f83980c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:14.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.928+0000 7f83a0400640 1 --2- 172.21.15.86:0/956306214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8380004800 0x7f83980c8b60 unknown :-1 s=CLOSED pgs=2188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:14.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.928+0000 7f83a0400640 1 -- 172.21.15.86:0/956306214 >> 172.21.15.86:0/956306214 conn(0x7f83980b2400 msgr2=0x7f83980ae910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:14.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.928+0000 7f83a0400640 1 -- 172.21.15.86:0/956306214 shutdown_connections 2024-04-20T21:31:14.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:14.928+0000 7f83a0400640 1 -- 172.21.15.86:0/956306214 wait complete. 2024-04-20T21:31:14.948 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":519,"stamp":"2024-04-20T21:31:14.913514+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":262236,"kb_used_data":12776,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712484,"statfs":{"total":773094113280,"available":772825583616,"internally_reserved":0,"allocated":13082624,"data_stored":6360280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.678920"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":352,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-20T21:31:04.800301+0000","last_change":"2024-04-20T21:17:43.059637+0000","last_active":"2024-04-20T21:31:04.800301+0000","last_peered":"2024-04-20T21:31:04.800301+0000","last_clean":"2024-04-20T21:31:04.800301+0000","last_became_active":"2024-04-20T21:17:43.056207+0000","last_became_peered":"2024-04-20T21:17:43.056207+0000","last_unstale":"2024-04-20T21:31:04.800301+0000","last_undegraded":"2024-04-20T21:31:04.800301+0000","last_fullsized":"2024-04-20T21:31:04.800301+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T21:17:41.036290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T21:17:41.036290+0000","last_clean_scrub_stamp":"2024-04-20T21:17:41.036290+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T05:58:51.091337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575012,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33216,"kb_used_data":2032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338624,"statfs":{"total":96636764160,"available":96602750976,"internally_reserved":0,"allocated":2080768,"data_stored":1237811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575013,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32632,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1486848,"data_stored":647443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575015,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32636,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339204,"statfs":{"total":96636764160,"available":96603344896,"internally_reserved":0,"allocated":1486848,"data_stored":647443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575014,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32632,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1486848,"data_stored":647443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575014,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32636,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339204,"statfs":{"total":96636764160,"available":96603344896,"internally_reserved":0,"allocated":1486848,"data_stored":647443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575014,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32636,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339204,"statfs":{"total":96636764160,"available":96603344896,"internally_reserved":0,"allocated":1486848,"data_stored":647443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575013,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32632,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1486848,"data_stored":647443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575014,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33216,"kb_used_data":2032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338624,"statfs":{"total":96636764160,"available":96602750976,"internally_reserved":0,"allocated":2080768,"data_stored":1237811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T21:31:14.952 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-20T21:17:41.036290+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=21, tm_min=17, tm_sec=41, tm_wday=5, tm_yday=111, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=20, tm_hour=21, tm_min=31, tm_sec=7, tm_wday=5, tm_yday=111, tm_isdst=0) 2024-04-20T21:31:14.952 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-20T21:31:18.815 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:31:18.823 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:31:34.953 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T21:31:35.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.162+0000 7f8728e00640 1 Processor -- start 2024-04-20T21:31:35.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.162+0000 7f8728e00640 1 -- start start 2024-04-20T21:31:35.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.164+0000 7f8728e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8724156ae0 0x7f872415d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:35.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.164+0000 7f8728e00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8724160080 0x7f87240ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:35.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.164+0000 7f8728e00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8724160890 con 0x7f87240ad620 2024-04-20T21:31:35.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.164+0000 7f8728e00640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7f872406db20 con 0x7f8724160080 2024-04-20T21:31:35.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.164+0000 7f8728e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f8724160b70 con 0x7f8724156ae0 2024-04-20T21:31:35.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.165+0000 7f871be00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8724160080 0x7f87240ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:35.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.165+0000 7f8723400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8724156ae0 0x7f872415d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:35.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.165+0000 7f871be00640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7f8724160080 0x7f87240ad0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.86:38582/0 (socket says 172.21.15.86:38582) 2024-04-20T21:31:35.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.165+0000 7f8723400640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8724156ae0 0x7f872415d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3301/0 says I am v2:172.21.15.86:52436/0 (socket says 172.21.15.86:52436) 2024-04-20T21:31:35.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.165+0000 7f871be00640 1 -- 172.21.15.86:0/3657122253 learned_addr learned my addr 172.21.15.86:0/3657122253 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:31:35.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.166+0000 7f871be00640 1 -- 172.21.15.86:0/3657122253 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8724156ae0 msgr2=0x7f872415d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:35.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.166+0000 7f871be00640 1 --2- 172.21.15.86:0/3657122253 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8724156ae0 0x7f872415d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.166+0000 7f871be00640 1 -- 172.21.15.86:0/3657122253 >> v1:172.21.15.144:6789/0 conn(0x7f87240ad620 legacy=0x7f8724141cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:35.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.166+0000 7f871be00640 1 -- 172.21.15.86:0/3657122253 --> v2:172.21.15.144:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8724160e50 con 0x7f8724160080 2024-04-20T21:31:35.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.166+0000 7f8723400640 1 --2- 172.21.15.86:0/3657122253 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8724156ae0 0x7f872415d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T21:31:35.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.166+0000 7f8722a00640 1 -- 172.21.15.86:0/3657122253 <== mon.1 v1:172.21.15.144:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1396119570 0 0) 0x7f8724160890 con 0x7f87240ad620 2024-04-20T21:31:35.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.168+0000 7f871be00640 1 --2- 172.21.15.86:0/3657122253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8724160080 0x7f87240ad0e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f870c009870 tx=0x7f870c02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67e66354ed6ee864 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:35.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.169+0000 7f8722a00640 1 -- 172.21.15.86:0/3657122253 <== mon.1 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870c034070 con 0x7f8724160080 2024-04-20T21:31:35.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.169+0000 7f8722a00640 1 -- 172.21.15.86:0/3657122253 <== mon.1 v2:172.21.15.144:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f870c030840 con 0x7f8724160080 2024-04-20T21:31:35.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.169+0000 7f8722a00640 1 -- 172.21.15.86:0/3657122253 <== mon.1 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870c030c10 con 0x7f8724160080 2024-04-20T21:31:35.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.169+0000 7f8728e00640 1 -- 172.21.15.86:0/3657122253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8724160080 msgr2=0x7f87240ad0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:35.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.169+0000 7f8728e00640 1 --2- 172.21.15.86:0/3657122253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8724160080 0x7f87240ad0e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f870c009870 tx=0x7f870c02b980 comp rx=0 tx=0).stop 2024-04-20T21:31:35.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.170+0000 7f8728e00640 1 -- 172.21.15.86:0/3657122253 shutdown_connections 2024-04-20T21:31:35.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.170+0000 7f8728e00640 1 --2- 172.21.15.86:0/3657122253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8724160080 0x7f87240ad0e0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.170+0000 7f8728e00640 1 --2- 172.21.15.86:0/3657122253 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f8724156ae0 0x7f872415d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.170+0000 7f8728e00640 1 -- 172.21.15.86:0/3657122253 >> 172.21.15.86:0/3657122253 conn(0x7f87240d0170 msgr2=0x7f87240d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:35.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.170+0000 7f8728e00640 1 -- 172.21.15.86:0/3657122253 shutdown_connections 2024-04-20T21:31:35.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.170+0000 7f8728e00640 1 -- 172.21.15.86:0/3657122253 wait complete. 2024-04-20T21:31:35.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.172+0000 7f8728e00640 1 Processor -- start 2024-04-20T21:31:35.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.172+0000 7f8728e00640 1 -- start start 2024-04-20T21:31:35.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.173+0000 7f8728e00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f87240ad620 0x7f8724099910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:35.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.173+0000 7f8728e00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8724156ae0 0x7f8724154930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:35.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.173+0000 7f8728e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8724160080 0x7f8724154e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:35.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.173+0000 7f8728e00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8724160500 con 0x7f8724160080 2024-04-20T21:31:35.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.173+0000 7f8728e00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f8724156ec0 con 0x7f8724156ae0 2024-04-20T21:31:35.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.173+0000 7f8728e00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f872406f2a0 con 0x7f87240ad620 2024-04-20T21:31:35.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.173+0000 7f8723e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8724160080 0x7f8724154e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:35.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.173+0000 7f8723e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8724160080 0x7f8724154e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48002/0 (socket says 172.21.15.86:48002) 2024-04-20T21:31:35.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.173+0000 7f8723e00640 1 -- 172.21.15.86:0/515070183 learned_addr learned my addr 172.21.15.86:0/515070183 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:31:35.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.173+0000 7f871be00640 1 --2- 172.21.15.86:0/515070183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8724156ae0 0x7f8724154930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:35.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.173+0000 7f8723400640 1 --2- 172.21.15.86:0/515070183 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f87240ad620 0x7f8724099910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:35.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.174+0000 7f8723e00640 1 -- 172.21.15.86:0/515070183 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f87240ad620 msgr2=0x7f8724099910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:35.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.174+0000 7f8723e00640 1 --2- 172.21.15.86:0/515070183 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f87240ad620 0x7f8724099910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.174+0000 7f8723e00640 1 -- 172.21.15.86:0/515070183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8724156ae0 msgr2=0x7f8724154930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:35.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.174+0000 7f8723e00640 1 --2- 172.21.15.86:0/515070183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8724156ae0 0x7f8724154930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.174+0000 7f8723e00640 1 -- 172.21.15.86:0/515070183 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8724099e50 con 0x7f8724160080 2024-04-20T21:31:35.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.174+0000 7f8723e00640 1 --2- 172.21.15.86:0/515070183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8724160080 0x7f8724154e70 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7f871c00c780 tx=0x7f871c00cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:35.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.174+0000 7f871b400640 1 -- 172.21.15.86:0/515070183 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871c003b70 con 0x7f8724160080 2024-04-20T21:31:35.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.174+0000 7f871b400640 1 -- 172.21.15.86:0/515070183 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f871c003d10 con 0x7f8724160080 2024-04-20T21:31:35.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.174+0000 7f871b400640 1 -- 172.21.15.86:0/515070183 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871c005270 con 0x7f8724160080 2024-04-20T21:31:35.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.174+0000 7f8728e00640 1 -- 172.21.15.86:0/515070183 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8724144c70 con 0x7f8724160080 2024-04-20T21:31:35.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.174+0000 7f8728e00640 1 -- 172.21.15.86:0/515070183 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8724145230 con 0x7f8724160080 2024-04-20T21:31:35.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.176+0000 7f8728e00640 1 -- 172.21.15.86:0/515070183 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f872404fcf0 con 0x7f8724160080 2024-04-20T21:31:35.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.177+0000 7f871b400640 1 -- 172.21.15.86:0/515070183 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f871c005410 con 0x7f8724160080 2024-04-20T21:31:35.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.177+0000 7f871b400640 1 --2- 172.21.15.86:0/515070183 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f870006c8f0 0x7f870006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:35.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.178+0000 7f871b400640 1 -- 172.21.15.86:0/515070183 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f871c08b870 con 0x7f8724160080 2024-04-20T21:31:35.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.178+0000 7f8723400640 1 --2- 172.21.15.86:0/515070183 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f870006c8f0 0x7f870006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:35.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.178+0000 7f8723400640 1 --2- 172.21.15.86:0/515070183 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f870006c8f0 0x7f870006edb0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f8714009a30 tx=0x7f8714008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:35.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.182+0000 7f871b400640 1 -- 172.21.15.86:0/515070183 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f871c059e00 con 0x7f8724160080 2024-04-20T21:31:35.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.380+0000 7f8728e00640 1 -- 172.21.15.86:0/515070183 --> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f87240b1bb0 con 0x7f870006c8f0 2024-04-20T21:31:35.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.381+0000 7f871b400640 1 -- 172.21.15.86:0/515070183 <== mgr.4107 v2:172.21.15.86:6832/302045240 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13379 (secure 0 0 0) 0x7f87240b1bb0 con 0x7f870006c8f0 2024-04-20T21:31:35.380 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-20T21:31:35.381 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:31:35.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.384+0000 7f8728e00640 1 -- 172.21.15.86:0/515070183 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f870006c8f0 msgr2=0x7f870006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:35.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.384+0000 7f8728e00640 1 --2- 172.21.15.86:0/515070183 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f870006c8f0 0x7f870006edb0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f8714009a30 tx=0x7f8714008040 comp rx=0 tx=0).stop 2024-04-20T21:31:35.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.384+0000 7f8728e00640 1 -- 172.21.15.86:0/515070183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8724160080 msgr2=0x7f8724154e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:35.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.384+0000 7f8728e00640 1 --2- 172.21.15.86:0/515070183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8724160080 0x7f8724154e70 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7f871c00c780 tx=0x7f871c00cb80 comp rx=0 tx=0).stop 2024-04-20T21:31:35.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.384+0000 7f8728e00640 1 -- 172.21.15.86:0/515070183 shutdown_connections 2024-04-20T21:31:35.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.384+0000 7f8728e00640 1 --2- 172.21.15.86:0/515070183 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f870006c8f0 0x7f870006edb0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.384+0000 7f8728e00640 1 --2- 172.21.15.86:0/515070183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8724160080 0x7f8724154e70 unknown :-1 s=CLOSED pgs=2189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.384+0000 7f8728e00640 1 --2- 172.21.15.86:0/515070183 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8724156ae0 0x7f8724154930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.384+0000 7f8728e00640 1 --2- 172.21.15.86:0/515070183 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f87240ad620 0x7f8724099910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.384+0000 7f8728e00640 1 -- 172.21.15.86:0/515070183 >> 172.21.15.86:0/515070183 conn(0x7f87240d0170 msgr2=0x7f872406e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:35.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.384+0000 7f8728e00640 1 -- 172.21.15.86:0/515070183 shutdown_connections 2024-04-20T21:31:35.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.384+0000 7f8728e00640 1 -- 172.21.15.86:0/515070183 wait complete. 2024-04-20T21:31:35.402 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":529,"stamp":"2024-04-20T21:31:34.919216+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":262220,"kb_used_data":12776,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712500,"statfs":{"total":773094113280,"available":772825600000,"internally_reserved":0,"allocated":13082624,"data_stored":6360280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003308"},"pg_stats":[{"pgid":"1.0","version":"119'119","reported_seq":363,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-20T21:31:15.458427+0000","last_change":"2024-04-20T21:31:15.458502+0000","last_active":"2024-04-20T21:31:15.458427+0000","last_peered":"2024-04-20T21:31:15.458427+0000","last_clean":"2024-04-20T21:31:15.458427+0000","last_became_active":"2024-04-20T21:17:43.056207+0000","last_became_peered":"2024-04-20T21:17:43.056207+0000","last_unstale":"2024-04-20T21:31:15.458427+0000","last_undegraded":"2024-04-20T21:31:15.458427+0000","last_fullsized":"2024-04-20T21:31:15.458427+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'119","last_scrub_stamp":"2024-04-20T21:31:15.458154+0000","last_deep_scrub":"119'119","last_deep_scrub_stamp":"2024-04-20T21:31:15.458154+0000","last_clean_scrub_stamp":"2024-04-20T21:31:15.458154+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:27:34.306165+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575016,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33208,"kb_used_data":2032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338632,"statfs":{"total":96636764160,"available":96602759168,"internally_reserved":0,"allocated":2080768,"data_stored":1237811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575017,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32632,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1486848,"data_stored":647443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575019,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32636,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339204,"statfs":{"total":96636764160,"available":96603344896,"internally_reserved":0,"allocated":1486848,"data_stored":647443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575018,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32632,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1486848,"data_stored":647443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575018,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32636,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339204,"statfs":{"total":96636764160,"available":96603344896,"internally_reserved":0,"allocated":1486848,"data_stored":647443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575018,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32636,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339204,"statfs":{"total":96636764160,"available":96603344896,"internally_reserved":0,"allocated":1486848,"data_stored":647443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575017,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32632,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1486848,"data_stored":647443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575018,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33208,"kb_used_data":2032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338632,"statfs":{"total":96636764160,"available":96602759168,"internally_reserved":0,"allocated":2080768,"data_stored":1237811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T21:31:35.403 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-20T21:31:35.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.582+0000 7fe42e000640 1 Processor -- start 2024-04-20T21:31:35.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.582+0000 7fe42e000640 1 -- start start 2024-04-20T21:31:35.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.584+0000 7fe42e000640 1 --2- >> v2:172.21.15.144:3300/0 conn(0x7fe42815ea70 0x7fe4280d3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:35.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.584+0000 7fe42e000640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4280cf120 con 0x7fe4280d45a0 2024-04-20T21:31:35.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.584+0000 7fe42e000640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe42806de70 con 0x7fe42806d570 2024-04-20T21:31:35.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.584+0000 7fe42e000640 1 -- --> v2:172.21.15.144:3300/0 -- mon_getmap magic: 0 -- 0x7fe42806e150 con 0x7fe42815ea70 2024-04-20T21:31:35.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.584+0000 7fe42cc00640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe4280d45a0 0x7fe4280d4980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:57928/0 (socket says 172.21.15.86:57928) 2024-04-20T21:31:35.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.584+0000 7fe42cc00640 1 -- 172.21.15.86:0/785153740 learned_addr learned my addr 172.21.15.86:0/785153740 (peer_addr_for_me v1:172.21.15.86:0/0) 2024-04-20T21:31:35.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.585+0000 7fe427e00640 1 --2- 172.21.15.86:0/785153740 >> v2:172.21.15.144:3300/0 conn(0x7fe42815ea70 0x7fe4280d3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:35.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.585+0000 7fe427400640 1 -- 172.21.15.86:0/785153740 <== mon.0 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2067776286 0 0) 0x7fe4280cf120 con 0x7fe4280d45a0 2024-04-20T21:31:35.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.585+0000 7fe427400640 1 -- 172.21.15.86:0/785153740 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe40c003520 con 0x7fe4280d45a0 2024-04-20T21:31:35.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.585+0000 7fe427400640 1 -- 172.21.15.86:0/785153740 <== mon.0 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1794800984 0 0) 0x7fe40c003520 con 0x7fe4280d45a0 2024-04-20T21:31:35.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.585+0000 7fe427400640 1 -- 172.21.15.86:0/785153740 >> v2:172.21.15.144:3300/0 conn(0x7fe42815ea70 msgr2=0x7fe4280d3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:35.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.585+0000 7fe427400640 1 --2- 172.21.15.86:0/785153740 >> v2:172.21.15.144:3300/0 conn(0x7fe42815ea70 0x7fe4280d3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.585+0000 7fe427400640 1 -- 172.21.15.86:0/785153740 >> v1:172.21.15.144:6789/0 conn(0x7fe42806d570 legacy=0x7fe4280cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:35.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.585+0000 7fe427400640 1 -- 172.21.15.86:0/785153740 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe42806e430 con 0x7fe4280d45a0 2024-04-20T21:31:35.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.585+0000 7fe427400640 1 -- 172.21.15.86:0/785153740 <== mon.0 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2894900828 0 0) 0x7fe418001690 con 0x7fe4280d45a0 2024-04-20T21:31:35.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.585+0000 7fe427400640 1 -- 172.21.15.86:0/785153740 >> v1:172.21.15.86:6789/0 conn(0x7fe4280d45a0 legacy=0x7fe4280d4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:35.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.585+0000 7fe427400640 1 --2- 172.21.15.86:0/785153740 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe40c003a20 0x7fe40c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:35.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.585+0000 7fe427400640 1 --2- 172.21.15.86:0/785153740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe40c004800 0x7fe40c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:35.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.585+0000 7fe427400640 1 --2- 172.21.15.86:0/785153740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe40c0052e0 0x7fe40c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:35.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.585+0000 7fe427400640 1 -- 172.21.15.86:0/785153740 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe42806e150 con 0x7fe40c0052e0 2024-04-20T21:31:35.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.585+0000 7fe427400640 1 -- 172.21.15.86:0/785153740 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe40c00c230 con 0x7fe40c004800 2024-04-20T21:31:35.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.585+0000 7fe427400640 1 -- 172.21.15.86:0/785153740 --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe40c00c430 con 0x7fe40c003a20 2024-04-20T21:31:35.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.585+0000 7fe427e00640 1 --2- 172.21.15.86:0/785153740 >> v2:172.21.15.144:3300/0 conn(0x7fe42815ea70 0x7fe4280d3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T21:31:35.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.585+0000 7fe42d600640 1 --2- 172.21.15.86:0/785153740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe40c0052e0 0x7fe40c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:35.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.586+0000 7fe427e00640 1 --2- 172.21.15.86:0/785153740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe40c004800 0x7fe40c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:35.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.586+0000 7fe42cc00640 1 --2- 172.21.15.86:0/785153740 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe40c003a20 0x7fe40c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:35.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.586+0000 7fe42d600640 1 -- 172.21.15.86:0/785153740 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe40c003a20 msgr2=0x7fe40c003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:35.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.586+0000 7fe42d600640 1 --2- 172.21.15.86:0/785153740 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe40c003a20 0x7fe40c003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.586+0000 7fe42d600640 1 -- 172.21.15.86:0/785153740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe40c004800 msgr2=0x7fe40c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:35.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.586+0000 7fe42d600640 1 --2- 172.21.15.86:0/785153740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe40c004800 0x7fe40c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.586+0000 7fe42d600640 1 -- 172.21.15.86:0/785153740 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe40c003520 con 0x7fe40c0052e0 2024-04-20T21:31:35.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.587+0000 7fe42d600640 1 -- 172.21.15.86:0/785153740 reap_dead start 2024-04-20T21:31:35.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.587+0000 7fe42d600640 1 --2- 172.21.15.86:0/785153740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe40c0052e0 0x7fe40c00b9c0 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7fe420001720 tx=0x7fe40c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=660315ab90fcb6f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:35.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.587+0000 7fe427400640 1 -- 172.21.15.86:0/785153740 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe420001370 con 0x7fe40c0052e0 2024-04-20T21:31:35.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.587+0000 7fe427400640 1 -- 172.21.15.86:0/785153740 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4200052d0 con 0x7fe40c0052e0 2024-04-20T21:31:35.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.587+0000 7fe427400640 1 -- 172.21.15.86:0/785153740 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe420003830 con 0x7fe40c0052e0 2024-04-20T21:31:35.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.588+0000 7fe42e000640 1 -- 172.21.15.86:0/785153740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe40c0052e0 msgr2=0x7fe40c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:35.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.588+0000 7fe42e000640 1 --2- 172.21.15.86:0/785153740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe40c0052e0 0x7fe40c00b9c0 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7fe420001720 tx=0x7fe40c005830 comp rx=0 tx=0).stop 2024-04-20T21:31:35.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.588+0000 7fe42e000640 1 -- 172.21.15.86:0/785153740 shutdown_connections 2024-04-20T21:31:35.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.588+0000 7fe42e000640 1 --2- 172.21.15.86:0/785153740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe40c0052e0 0x7fe40c00b9c0 unknown :-1 s=CLOSED pgs=2191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.588+0000 7fe42e000640 1 -- 172.21.15.86:0/785153740 >> 172.21.15.86:0/785153740 conn(0x7fe42809d9d0 msgr2=0x7fe42808bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:35.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.588+0000 7fe42e000640 1 -- 172.21.15.86:0/785153740 shutdown_connections 2024-04-20T21:31:35.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.588+0000 7fe42e000640 1 -- 172.21.15.86:0/785153740 wait complete. 2024-04-20T21:31:35.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.590+0000 7fe42e000640 1 Processor -- start 2024-04-20T21:31:35.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.590+0000 7fe42e000640 1 -- start start 2024-04-20T21:31:35.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.591+0000 7fe42e000640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe40c0052e0 0x7fe4281e7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:35.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.591+0000 7fe42e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4281504d0 0x7fe4281e93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:35.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.591+0000 7fe42e000640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4281e98e0 0x7fe42809fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:35.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.591+0000 7fe42e000640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe42806db20 con 0x7fe4281504d0 2024-04-20T21:31:35.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.591+0000 7fe42e000640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7fe42806d9a0 con 0x7fe4281e98e0 2024-04-20T21:31:35.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.591+0000 7fe42e000640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7fe4280d3260 con 0x7fe40c0052e0 2024-04-20T21:31:35.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.591+0000 7fe427e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4281504d0 0x7fe4281e93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:35.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.591+0000 7fe427e00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4281504d0 0x7fe4281e93a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48028/0 (socket says 172.21.15.86:48028) 2024-04-20T21:31:35.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.591+0000 7fe427e00640 1 -- 172.21.15.86:0/4162034010 learned_addr learned my addr 172.21.15.86:0/4162034010 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:31:35.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.592+0000 7fe427e00640 1 -- 172.21.15.86:0/4162034010 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe40c0052e0 msgr2=0x7fe4281e7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:35.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.592+0000 7fe427e00640 1 --2- 172.21.15.86:0/4162034010 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe40c0052e0 0x7fe4281e7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.592+0000 7fe427e00640 1 -- 172.21.15.86:0/4162034010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4281e98e0 msgr2=0x7fe42809fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:35.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.592+0000 7fe427e00640 1 --2- 172.21.15.86:0/4162034010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4281e98e0 0x7fe42809fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.592+0000 7fe427e00640 1 -- 172.21.15.86:0/4162034010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4281e8300 con 0x7fe4281504d0 2024-04-20T21:31:35.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.592+0000 7fe427e00640 1 --2- 172.21.15.86:0/4162034010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4281504d0 0x7fe4281e93a0 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7fe4100046e0 tx=0x7fe410007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:35.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.592+0000 7fe424c00640 1 -- 172.21.15.86:0/4162034010 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe410014610 con 0x7fe4281504d0 2024-04-20T21:31:35.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.592+0000 7fe424c00640 1 -- 172.21.15.86:0/4162034010 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe410019070 con 0x7fe4281504d0 2024-04-20T21:31:35.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.592+0000 7fe424c00640 1 -- 172.21.15.86:0/4162034010 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4100149b0 con 0x7fe4281504d0 2024-04-20T21:31:35.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.592+0000 7fe42e000640 1 -- 172.21.15.86:0/4162034010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4281e85e0 con 0x7fe4281504d0 2024-04-20T21:31:35.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.592+0000 7fe42e000640 1 -- 172.21.15.86:0/4162034010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4281e8ba0 con 0x7fe4281504d0 2024-04-20T21:31:35.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.594+0000 7fe42e000640 1 -- 172.21.15.86:0/4162034010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4280a2520 con 0x7fe4281504d0 2024-04-20T21:31:35.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.595+0000 7fe424c00640 1 -- 172.21.15.86:0/4162034010 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe410014b50 con 0x7fe4281504d0 2024-04-20T21:31:35.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.596+0000 7fe424c00640 1 --2- 172.21.15.86:0/4162034010 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe40406c970 0x7fe40406ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:35.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.596+0000 7fe424c00640 1 -- 172.21.15.86:0/4162034010 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7fe41008c2d0 con 0x7fe4281504d0 2024-04-20T21:31:35.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.596+0000 7fe42cc00640 1 --2- 172.21.15.86:0/4162034010 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe40406c970 0x7fe40406ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:35.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.597+0000 7fe42cc00640 1 --2- 172.21.15.86:0/4162034010 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe40406c970 0x7fe40406ee30 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fe418001940 tx=0x7fe4180018b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:35.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.600+0000 7fe424c00640 1 -- 172.21.15.86:0/4162034010 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe41005a930 con 0x7fe4281504d0 2024-04-20T21:31:35.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.799+0000 7fe42e000640 1 -- 172.21.15.86:0/4162034010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7fe42804fd90 con 0x7fe4281504d0 2024-04-20T21:31:35.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.805+0000 7fe424c00640 1 -- 172.21.15.86:0/4162034010 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v24)=0 v24) ==== 123+0+0 (secure 0 0 0) 0x7fe41005a230 con 0x7fe4281504d0 2024-04-20T21:31:35.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.809+0000 7fe424c00640 1 -- 172.21.15.86:0/4162034010 <== mon.0 v2:172.21.15.86:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fe41005e5e0 con 0x7fe4281504d0 2024-04-20T21:31:35.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.810+0000 7fe42e000640 1 -- 172.21.15.86:0/4162034010 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe40406c970 msgr2=0x7fe40406ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:35.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.810+0000 7fe42e000640 1 --2- 172.21.15.86:0/4162034010 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe40406c970 0x7fe40406ee30 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fe418001940 tx=0x7fe4180018b0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.810+0000 7fe42e000640 1 -- 172.21.15.86:0/4162034010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4281504d0 msgr2=0x7fe4281e93a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:35.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.810+0000 7fe42e000640 1 --2- 172.21.15.86:0/4162034010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4281504d0 0x7fe4281e93a0 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7fe4100046e0 tx=0x7fe410007700 comp rx=0 tx=0).stop 2024-04-20T21:31:35.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.810+0000 7fe42e000640 1 -- 172.21.15.86:0/4162034010 shutdown_connections 2024-04-20T21:31:35.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.810+0000 7fe42e000640 1 --2- 172.21.15.86:0/4162034010 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7fe40406c970 0x7fe40406ee30 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.810+0000 7fe42e000640 1 --2- 172.21.15.86:0/4162034010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4281e98e0 0x7fe42809fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.810+0000 7fe42e000640 1 --2- 172.21.15.86:0/4162034010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4281504d0 0x7fe4281e93a0 unknown :-1 s=CLOSED pgs=2192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.810+0000 7fe42e000640 1 --2- 172.21.15.86:0/4162034010 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7fe40c0052e0 0x7fe4281e7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:35.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.810+0000 7fe42e000640 1 -- 172.21.15.86:0/4162034010 >> 172.21.15.86:0/4162034010 conn(0x7fe42809d9d0 msgr2=0x7fe4280b4150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:35.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.810+0000 7fe42e000640 1 -- 172.21.15.86:0/4162034010 shutdown_connections 2024-04-20T21:31:35.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:35.810+0000 7fe42e000640 1 -- 172.21.15.86:0/4162034010 wait complete. 2024-04-20T21:31:35.828 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-20T21:31:35.829 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T21:31:36.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.014+0000 7f6f1de00640 1 Processor -- start 2024-04-20T21:31:36.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.014+0000 7f6f1de00640 1 -- start start 2024-04-20T21:31:36.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.016+0000 7f6f1de00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6f1815d2c0 0x7f6f1806f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:36.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.016+0000 7f6f1de00640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6f1806f5f0 0x7f6f18071d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:36.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.016+0000 7f6f1de00640 1 -- --> v1:172.21.15.144:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f1806fb70 con 0x7f6f18072240 2024-04-20T21:31:36.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.016+0000 7f6f1de00640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 -- 0x7f6f180726b0 con 0x7f6f1806f5f0 2024-04-20T21:31:36.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.016+0000 7f6f1de00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6f18164210 con 0x7f6f1815d2c0 2024-04-20T21:31:36.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.016+0000 7f6f0e000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6f1806f5f0 0x7f6f18071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:36.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.016+0000 7f6f0e000640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6f1806f5f0 0x7f6f18071d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40212/0 (socket says 172.21.15.86:40212) 2024-04-20T21:31:36.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.016+0000 7f6f0e000640 1 -- 172.21.15.86:0/2331537001 learned_addr learned my addr 172.21.15.86:0/2331537001 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:31:36.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.017+0000 7f6f16a00640 1 --2- 172.21.15.86:0/2331537001 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6f1815d2c0 0x7f6f1806f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:36.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.017+0000 7f6f0e000640 1 -- 172.21.15.86:0/2331537001 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6f1815d2c0 msgr2=0x7f6f1806f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:36.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.017+0000 7f6f0e000640 1 --2- 172.21.15.86:0/2331537001 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6f1815d2c0 0x7f6f1806f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:36.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.017+0000 7f6f0e000640 1 -- 172.21.15.86:0/2331537001 >> v1:172.21.15.144:6789/0 conn(0x7f6f18072240 legacy=0x7f6f1806dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T21:31:36.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.017+0000 7f6f0e000640 1 -- 172.21.15.86:0/2331537001 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f1815d6a0 con 0x7f6f1806f5f0 2024-04-20T21:31:36.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.018+0000 7f6f0e000640 1 --2- 172.21.15.86:0/2331537001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f1806f5f0 0x7f6f18071d00 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7f6f1806dbe0 tx=0x7f6f0002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ac6b408cb91b741 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:36.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.018+0000 7f6f16000640 1 -- 172.21.15.86:0/2331537001 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f00035020 con 0x7f6f1806f5f0 2024-04-20T21:31:36.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.018+0000 7f6f16000640 1 -- 172.21.15.86:0/2331537001 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f6f00031830 con 0x7f6f1806f5f0 2024-04-20T21:31:36.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.018+0000 7f6f16000640 1 -- 172.21.15.86:0/2331537001 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f00031c20 con 0x7f6f1806f5f0 2024-04-20T21:31:36.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.019+0000 7f6f1de00640 1 -- 172.21.15.86:0/2331537001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f1806f5f0 msgr2=0x7f6f18071d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:36.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.019+0000 7f6f1de00640 1 --2- 172.21.15.86:0/2331537001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f1806f5f0 0x7f6f18071d00 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7f6f1806dbe0 tx=0x7f6f0002b8d0 comp rx=0 tx=0).stop 2024-04-20T21:31:36.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.019+0000 7f6f1de00640 1 -- 172.21.15.86:0/2331537001 shutdown_connections 2024-04-20T21:31:36.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.019+0000 7f6f1de00640 1 --2- 172.21.15.86:0/2331537001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f1806f5f0 0x7f6f18071d00 unknown :-1 s=CLOSED pgs=2193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:36.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.019+0000 7f6f1de00640 1 --2- 172.21.15.86:0/2331537001 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6f1815d2c0 0x7f6f1806f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:36.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.019+0000 7f6f1de00640 1 -- 172.21.15.86:0/2331537001 >> 172.21.15.86:0/2331537001 conn(0x7f6f180b2360 msgr2=0x7f6f180b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:36.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.019+0000 7f6f1de00640 1 -- 172.21.15.86:0/2331537001 shutdown_connections 2024-04-20T21:31:36.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.019+0000 7f6f1de00640 1 -- 172.21.15.86:0/2331537001 wait complete. 2024-04-20T21:31:36.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.021+0000 7f6f1de00640 1 Processor -- start 2024-04-20T21:31:36.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.021+0000 7f6f1de00640 1 -- start start 2024-04-20T21:31:36.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.022+0000 7f6f1de00640 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f1806f5f0 0x7f6f1814f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:36.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.022+0000 7f6f1de00640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f18072240 0x7f6f180cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:36.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.022+0000 7f6f1de00640 1 --2- >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6f1815d2c0 0x7f6f18158380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:36.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.022+0000 7f6f1de00640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6f18166ec0 con 0x7f6f18072240 2024-04-20T21:31:36.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.022+0000 7f6f1de00640 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 -- 0x7f6f18166be0 con 0x7f6f1806f5f0 2024-04-20T21:31:36.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.022+0000 7f6f1de00640 1 -- --> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] -- mon_getmap magic: 0 -- 0x7f6f181671a0 con 0x7f6f1815d2c0 2024-04-20T21:31:36.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.022+0000 7f6f0e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f18072240 0x7f6f180cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:36.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.022+0000 7f6f0e000640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f18072240 0x7f6f180cecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40218/0 (socket says 172.21.15.86:40218) 2024-04-20T21:31:36.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.022+0000 7f6f0e000640 1 -- 172.21.15.86:0/1204758209 learned_addr learned my addr 172.21.15.86:0/1204758209 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-20T21:31:36.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.023+0000 7f6f16a00640 1 --2- 172.21.15.86:0/1204758209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f1806f5f0 0x7f6f1814f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:36.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.023+0000 7f6f17400640 1 --2- 172.21.15.86:0/1204758209 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6f1815d2c0 0x7f6f18158380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:36.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.023+0000 7f6f0e000640 1 -- 172.21.15.86:0/1204758209 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6f1815d2c0 msgr2=0x7f6f18158380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:36.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.023+0000 7f6f0e000640 1 --2- 172.21.15.86:0/1204758209 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6f1815d2c0 0x7f6f18158380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:36.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.023+0000 7f6f0e000640 1 -- 172.21.15.86:0/1204758209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f1806f5f0 msgr2=0x7f6f1814f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:36.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.023+0000 7f6f0e000640 1 --2- 172.21.15.86:0/1204758209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f1806f5f0 0x7f6f1814f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:36.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.023+0000 7f6f0e000640 1 -- 172.21.15.86:0/1204758209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f1814fae0 con 0x7f6f18072240 2024-04-20T21:31:36.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.023+0000 7f6f0e000640 1 --2- 172.21.15.86:0/1204758209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f18072240 0x7f6f180cecb0 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7f6f0002b910 tx=0x7f6f000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:36.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.023+0000 7f6f0fe00640 1 -- 172.21.15.86:0/1204758209 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f00040070 con 0x7f6f18072240 2024-04-20T21:31:36.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.024+0000 7f6f0fe00640 1 -- 172.21.15.86:0/1204758209 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f6f00039bf0 con 0x7f6f18072240 2024-04-20T21:31:36.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.024+0000 7f6f1de00640 1 -- 172.21.15.86:0/1204758209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f18154df0 con 0x7f6f18072240 2024-04-20T21:31:36.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.024+0000 7f6f0fe00640 1 -- 172.21.15.86:0/1204758209 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f00035040 con 0x7f6f18072240 2024-04-20T21:31:36.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.024+0000 7f6f1de00640 1 -- 172.21.15.86:0/1204758209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f181553b0 con 0x7f6f18072240 2024-04-20T21:31:36.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.025+0000 7f6f1de00640 1 -- 172.21.15.86:0/1204758209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f180689e0 con 0x7f6f18072240 2024-04-20T21:31:36.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.026+0000 7f6f0fe00640 1 -- 172.21.15.86:0/1204758209 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6f00038020 con 0x7f6f18072240 2024-04-20T21:31:36.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.027+0000 7f6f0fe00640 1 --2- 172.21.15.86:0/1204758209 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6ef406c8f0 0x7f6ef406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T21:31:36.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.027+0000 7f6f0fe00640 1 -- 172.21.15.86:0/1204758209 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5115+0+0 (secure 0 0 0) 0x7f6f000ad570 con 0x7f6f18072240 2024-04-20T21:31:36.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.027+0000 7f6f16a00640 1 --2- 172.21.15.86:0/1204758209 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6ef406c8f0 0x7f6ef406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T21:31:36.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.028+0000 7f6f16a00640 1 --2- 172.21.15.86:0/1204758209 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6ef406c8f0 0x7f6ef406edb0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f6f080070b0 tx=0x7f6f08007040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T21:31:36.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.032+0000 7f6f0fe00640 1 -- 172.21.15.86:0/1204758209 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6f1814fae0 con 0x7f6f18072240 2024-04-20T21:31:36.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.258+0000 7f6f1de00640 1 -- 172.21.15.86:0/1204758209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6f1806ec90 con 0x7f6f18072240 2024-04-20T21:31:36.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.259+0000 7f6f0fe00640 1 -- 172.21.15.86:0/1204758209 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) ==== 95+0+3878 (secure 0 0 0) 0x7f6f0007bb50 con 0x7f6f18072240 2024-04-20T21:31:36.258 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-20T21:31:36.258 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":114,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6253,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6833/3627230007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":3627230007},{"type":"v1","addr":"172.21.15.144:6833","nonce":3627230007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":6256,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6835/2498297133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6834","nonce":2498297133},{"type":"v1","addr":"172.21.15.144:6835","nonce":2498297133}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":9228,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/1120740475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":1120740475},{"type":"v1","addr":"172.21.15.86:6836","nonce":1120740475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":10023,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/4083361845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":4083361845},{"type":"v1","addr":"172.21.15.86:6837","nonce":4083361845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":114}],"filesystems":[]} 2024-04-20T21:31:36.258 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 114 2024-04-20T21:31:36.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.261+0000 7f6f1de00640 1 -- 172.21.15.86:0/1204758209 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6ef406c8f0 msgr2=0x7f6ef406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:36.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.261+0000 7f6f1de00640 1 --2- 172.21.15.86:0/1204758209 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6ef406c8f0 0x7f6ef406edb0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f6f080070b0 tx=0x7f6f08007040 comp rx=0 tx=0).stop 2024-04-20T21:31:36.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.261+0000 7f6f1de00640 1 -- 172.21.15.86:0/1204758209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f18072240 msgr2=0x7f6f180cecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T21:31:36.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.261+0000 7f6f1de00640 1 --2- 172.21.15.86:0/1204758209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f18072240 0x7f6f180cecb0 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7f6f0002b910 tx=0x7f6f000046c0 comp rx=0 tx=0).stop 2024-04-20T21:31:36.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.261+0000 7f6f1de00640 1 -- 172.21.15.86:0/1204758209 shutdown_connections 2024-04-20T21:31:36.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.261+0000 7f6f1de00640 1 --2- 172.21.15.86:0/1204758209 >> [v2:172.21.15.86:6832/302045240,v1:172.21.15.86:6833/302045240] conn(0x7f6ef406c8f0 0x7f6ef406edb0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:36.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.261+0000 7f6f1de00640 1 --2- 172.21.15.86:0/1204758209 >> [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] conn(0x7f6f1815d2c0 0x7f6f18158380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:36.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.261+0000 7f6f1de00640 1 --2- 172.21.15.86:0/1204758209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f18072240 0x7f6f180cecb0 unknown :-1 s=CLOSED pgs=2194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:36.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.261+0000 7f6f1de00640 1 --2- 172.21.15.86:0/1204758209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f1806f5f0 0x7f6f1814f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T21:31:36.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.261+0000 7f6f1de00640 1 -- 172.21.15.86:0/1204758209 >> 172.21.15.86:0/1204758209 conn(0x7f6f180b2360 msgr2=0x7f6f180b3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T21:31:36.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.262+0000 7f6f1de00640 1 -- 172.21.15.86:0/1204758209 shutdown_connections 2024-04-20T21:31:36.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-20T21:31:36.262+0000 7f6f1de00640 1 -- 172.21.15.86:0/1204758209 wait complete. 2024-04-20T21:31:36.278 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-20T21:31:36.278 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-20T21:31:36.279 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-20T21:31:36.279 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:31:36.297 INFO:tasks.ceph.mds.a:Stopped 2024-04-20T21:31:36.297 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-20T21:31:36.297 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:31:36.358 INFO:tasks.ceph.mds.c:Stopped 2024-04-20T21:31:36.358 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-20T21:31:36.358 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:31:36.381 INFO:tasks.ceph.mds.b:Stopped 2024-04-20T21:31:36.381 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-20T21:31:36.381 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:31:36.457 INFO:tasks.ceph.mds.d:Stopped 2024-04-20T21:31:36.457 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-20T21:31:36.457 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-20T21:31:36.457 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:31:36.539 INFO:tasks.ceph.osd.0:Stopped 2024-04-20T21:31:36.539 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-20T21:31:36.539 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:31:36.600 INFO:tasks.ceph.osd.1:Stopped 2024-04-20T21:31:36.600 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-20T21:31:36.600 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:31:36.683 INFO:tasks.ceph.osd.2:Stopped 2024-04-20T21:31:36.683 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-20T21:31:36.683 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:31:36.744 INFO:tasks.ceph.osd.3:Stopped 2024-04-20T21:31:36.744 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-20T21:31:36.745 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:31:36.809 INFO:tasks.ceph.osd.4:Stopped 2024-04-20T21:31:36.809 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-20T21:31:36.809 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:31:36.870 INFO:tasks.ceph.osd.5:Stopped 2024-04-20T21:31:36.870 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-20T21:31:36.870 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:31:36.943 INFO:tasks.ceph.osd.6:Stopped 2024-04-20T21:31:36.944 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-20T21:31:36.944 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:31:37.013 INFO:tasks.ceph.osd.7:Stopped 2024-04-20T21:31:37.013 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-20T21:31:37.013 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-20T21:31:37.013 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:31:37.097 INFO:tasks.ceph.mgr.y:Stopped 2024-04-20T21:31:37.097 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-20T21:31:37.097 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:31:37.176 INFO:tasks.ceph.mgr.x:Stopped 2024-04-20T21:31:37.176 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-20T21:31:37.176 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-20T21:31:37.176 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:31:37.228 INFO:tasks.ceph.mon.a:Stopped 2024-04-20T21:31:37.228 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-20T21:31:37.228 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:31:37.257 INFO:tasks.ceph.mon.b:Stopped 2024-04-20T21:31:37.257 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-20T21:31:37.257 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T21:31:37.314 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-04-20T21:31:37.315 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-04-20T21:31:37.315 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-04-20T21:31:37.315 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-04-20T21:31:37.315 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2024-04-20T21:31:37.315 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-04-20T21:31:37.315 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-04-20T21:31:37.315 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-04-20T21:31:37.328 INFO:tasks.ceph.mon.c:Stopped 2024-04-20T21:31:37.328 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-20T21:31:37.329 DEBUG:teuthology.orchestra.run.smithi086:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-04-20T21:31:37.437 INFO:teuthology.orchestra.run.smithi086.stdout:2024-04-20T21:23:20.238103+0000 mds.a (mds.0) 1 : cluster [WRN] Corrupt dentry 'file_to_be_damaged' in dir frag 0x10000000000: Invalid tag char 'd' pos 0: Malformed input [buffer:3](/subdir) 2024-04-20T21:31:37.439 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2024-04-20T21:31:37.439 DEBUG:teuthology.orchestra.run.smithi086:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-04-20T21:31:37.530 DEBUG:teuthology.orchestra.run.smithi086:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-04-20T21:31:37.614 INFO:teuthology.orchestra.run.smithi086.stdout:2024-04-20T21:25:28.220524+0000 mds.b (mds.0) 1 : cluster [ERR] dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-04-20T21:31:37.615 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi086.front.sepia.ceph.com 2024-04-20T21:31:37.615 DEBUG:teuthology.orchestra.run.smithi086:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-20T21:31:37.906 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi086.front.sepia.ceph.com 2024-04-20T21:31:37.906 DEBUG:teuthology.orchestra.run.smithi086:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-20T21:31:38.065 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi086.front.sepia.ceph.com 2024-04-20T21:31:38.065 DEBUG:teuthology.orchestra.run.smithi086:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-20T21:31:38.202 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi086.front.sepia.ceph.com 2024-04-20T21:31:38.203 DEBUG:teuthology.orchestra.run.smithi086:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-20T21:31:38.352 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi144.front.sepia.ceph.com 2024-04-20T21:31:38.352 DEBUG:teuthology.orchestra.run.smithi144:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-20T21:31:38.656 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi144.front.sepia.ceph.com 2024-04-20T21:31:38.656 DEBUG:teuthology.orchestra.run.smithi144:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-20T21:31:38.788 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi144.front.sepia.ceph.com 2024-04-20T21:31:38.789 DEBUG:teuthology.orchestra.run.smithi144:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-20T21:31:38.929 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi144.front.sepia.ceph.com 2024-04-20T21:31:38.929 DEBUG:teuthology.orchestra.run.smithi144:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-20T21:31:39.095 INFO:tasks.ceph:Archiving mon data... 2024-04-20T21:31:39.096 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665408/data/mon.a.tgz 2024-04-20T21:31:39.096 DEBUG:teuthology.orchestra.run.smithi086:> mktemp 2024-04-20T21:31:39.155 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/tmp.J7an1DXA1D 2024-04-20T21:31:39.156 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.J7an1DXA1D 2024-04-20T21:31:39.345 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 0666 /tmp/tmp.J7an1DXA1D 2024-04-20T21:31:39.482 DEBUG:teuthology.orchestra.remote:smithi086:/tmp/tmp.J7an1DXA1D is 399KB 2024-04-20T21:31:39.589 DEBUG:teuthology.orchestra.run.smithi086:> rm -fr /tmp/tmp.J7an1DXA1D 2024-04-20T21:31:39.645 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665408/data/mon.b.tgz 2024-04-20T21:31:39.645 DEBUG:teuthology.orchestra.run.smithi144:> mktemp 2024-04-20T21:31:39.702 INFO:teuthology.orchestra.run.smithi144.stdout:/tmp/tmp.YT1dBlCHae 2024-04-20T21:31:39.702 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.YT1dBlCHae 2024-04-20T21:31:39.902 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 0666 /tmp/tmp.YT1dBlCHae 2024-04-20T21:31:40.037 DEBUG:teuthology.orchestra.remote:smithi144:/tmp/tmp.YT1dBlCHae is 400KB 2024-04-20T21:31:40.145 DEBUG:teuthology.orchestra.run.smithi144:> rm -fr /tmp/tmp.YT1dBlCHae 2024-04-20T21:31:40.200 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665408/data/mon.c.tgz 2024-04-20T21:31:40.201 DEBUG:teuthology.orchestra.run.smithi144:> mktemp 2024-04-20T21:31:40.253 INFO:teuthology.orchestra.run.smithi144.stdout:/tmp/tmp.LzAIhB9y0G 2024-04-20T21:31:40.254 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.LzAIhB9y0G 2024-04-20T21:31:40.421 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 0666 /tmp/tmp.LzAIhB9y0G 2024-04-20T21:31:40.558 DEBUG:teuthology.orchestra.remote:smithi144:/tmp/tmp.LzAIhB9y0G is 400KB 2024-04-20T21:31:40.667 DEBUG:teuthology.orchestra.run.smithi144:> rm -fr /tmp/tmp.LzAIhB9y0G 2024-04-20T21:31:40.724 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-20T21:31:40.725 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-20T21:31:40.733 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-20T21:31:40.878 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-20T21:31:40.935 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-20T21:31:40.935 INFO:tasks.ceph:Archiving crash dumps... 2024-04-20T21:31:40.937 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665408/remote/smithi086/crash 2024-04-20T21:31:40.938 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-20T21:31:41.025 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665408/remote/smithi144/crash 2024-04-20T21:31:41.026 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-20T21:31:41.117 INFO:tasks.ceph:Shutting down logrotate 2024-04-20T21:31:41.117 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:31:41.126 DEBUG:teuthology.orchestra.run.smithi144:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:31:41.245 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-20T21:31:41.333 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-20T21:31:41.422 INFO:tasks.ceph:Compressing logs... 2024-04-20T21:31:41.422 DEBUG:teuthology.orchestra.run.smithi086:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-20T21:31:41.430 DEBUG:teuthology.orchestra.run.smithi144:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-20T21:31:41.508 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59922.log 2024-04-20T21:31:41.509 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70351.log 2024-04-20T21:31:41.510 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33477.log 2024-04-20T21:31:41.510 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-20T21:31:41.511 INFO:teuthology.orchestra.run.smithi086.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.59922.log.gz 2024-04-20T21:31:41.511 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.70351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42578.log 2024-04-20T21:31:41.511 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.70351.log.gz 2024-04-20T21:31:41.512 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22304.log 2024-04-20T21:31:41.513 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22220.log 2024-04-20T21:31:41.514 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34629.log 2024-04-20T21:31:41.514 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42578.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33477.log.gz 2024-04-20T21:31:41.514 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42578.log.gz 2024-04-20T21:31:41.514 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29553.log 2024-04-20T21:31:41.517 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71505.log 2024-04-20T21:31:41.517 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34629.log.gz 2024-04-20T21:31:41.517 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.29553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58953.log 2024-04-20T21:31:41.517 INFO:teuthology.orchestra.run.smithi086.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.29553.log.gz 2024-04-20T21:31:41.518 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.71505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45468.log 2024-04-20T21:31:41.518 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.71505.log.gz 2024-04-20T21:31:41.519 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.admin.22304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.28934.log 2024-04-20T21:31:41.519 INFO:teuthology.orchestra.run.smithi144.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.22304.log.gz 2024-04-20T21:31:41.519 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.admin.22220.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.22220.log.gz 2024-04-20T21:31:41.519 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-20T21:31:41.519 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.3.28934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22471.log 2024-04-20T21:31:41.519 INFO:teuthology.orchestra.run.smithi144.stderr: 92.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-20T21:31:41.519 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-20T21:31:41.520 INFO:teuthology.orchestra.run.smithi144.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.3.28934.log.gz 2024-04-20T21:31:41.520 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.admin.22471.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.22471.log.gz 2024-04-20T21:31:41.520 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-20T21:31:41.521 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-20T21:31:41.523 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32826.log 2024-04-20T21:31:41.523 INFO:teuthology.orchestra.run.smithi086.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58953.log.gz 2024-04-20T21:31:41.523 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53375.log 2024-04-20T21:31:41.523 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45468.log.gz 2024-04-20T21:31:41.523 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.32826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60977.log 2024-04-20T21:31:41.523 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.32826.log.gz 2024-04-20T21:31:41.523 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57653.log 2024-04-20T21:31:41.523 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53375.log.gz 2024-04-20T21:31:41.523 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35465.log 2024-04-20T21:31:41.523 INFO:teuthology.orchestra.run.smithi086.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.60977.log.gz 2024-04-20T21:31:41.524 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50605.log 2024-04-20T21:31:41.524 INFO:teuthology.orchestra.run.smithi086.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.57653.log.gz 2024-04-20T21:31:41.526 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62697.log 2024-04-20T21:31:41.526 INFO:teuthology.orchestra.run.smithi086.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35465.log.gz 2024-04-20T21:31:41.526 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49199.log 2024-04-20T21:31:41.526 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50605.log.gz 2024-04-20T21:31:41.526 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67675.log 2024-04-20T21:31:41.526 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.62697.log.gz 2024-04-20T21:31:41.527 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-20T21:31:41.527 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-osd.5.log: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-20T21:31:41.528 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22391.log 2024-04-20T21:31:41.528 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-20T21:31:41.528 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.admin.22391.log: gzip -5 --verbose 88.3% -- -- replaced with /var/log/ceph/ceph-client.admin.22391.log.gz /var/log/ceph/ceph-mon.c.log 2024-04-20T21:31:41.528 INFO:teuthology.orchestra.run.smithi144.stderr: 2024-04-20T21:31:41.528 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22431.log 2024-04-20T21:31:41.529 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-20T21:31:41.530 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35757.log 2024-04-20T21:31:41.530 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49199.log.gz 2024-04-20T21:31:41.530 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.67675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57401.log 2024-04-20T21:31:41.530 INFO:teuthology.orchestra.run.smithi086.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.67675.log.gz 2024-04-20T21:31:41.530 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43207.log 2024-04-20T21:31:41.530 INFO:teuthology.orchestra.run.smithi086.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35757.log.gz 2024-04-20T21:31:41.530 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62823.log 2024-04-20T21:31:41.531 INFO:teuthology.orchestra.run.smithi086.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57401.log.gz 2024-04-20T21:31:41.532 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.admin.22431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22351.log 2024-04-20T21:31:41.532 INFO:teuthology.orchestra.run.smithi144.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.22431.log.gz 2024-04-20T21:31:41.532 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57695.log 2024-04-20T21:31:41.533 INFO:teuthology.orchestra.run.smithi086.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43207.log.gz 2024-04-20T21:31:41.533 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62823.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.15815.log 2024-04-20T21:31:41.533 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62823.log.gz 2024-04-20T21:31:41.533 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49947.log 2024-04-20T21:31:41.533 INFO:teuthology.orchestra.run.smithi086.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57695.log.gz 2024-04-20T21:31:41.534 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph.tmp-client.admin.15815.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.15815.log.gz 2024-04-20T21:31:41.534 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65364.log 2024-04-20T21:31:41.534 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-osd.6.log: /var/log/ceph/ceph-client.admin.22351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.28410.log 2024-04-20T21:31:41.535 INFO:teuthology.orchestra.run.smithi144.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.22351.log.gz 2024-04-20T21:31:41.535 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22511.log 2024-04-20T21:31:41.538 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34797.log 2024-04-20T21:31:41.538 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49947.log.gz 2024-04-20T21:31:41.538 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.65364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70634.log 2024-04-20T21:31:41.538 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.65364.log.gz 2024-04-20T21:31:41.539 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42410.log 2024-04-20T21:31:41.539 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34797.log.gz 2024-04-20T21:31:41.539 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.70634.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.70634.log.gz 2024-04-20T21:31:41.539 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53643.log 2024-04-20T21:31:41.551 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.admin.22511.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.22511.log.gz 2024-04-20T21:31:41.552 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.1.28410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22260.log 2024-04-20T21:31:41.552 INFO:teuthology.orchestra.run.smithi144.stderr: 94.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-20T21:31:41.552 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.admin.22260.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.22260.log.gz 2024-04-20T21:31:41.552 INFO:teuthology.orchestra.run.smithi144.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.1.28410.log.gz 2024-04-20T21:31:41.553 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-20T21:31:41.553 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-20T21:31:41.573 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42410.log: gzip -5 --verbose -- 87.9% /var/log/ceph/ceph-client.admin.46475.log 2024-04-20T21:31:41.573 INFO:teuthology.orchestra.run.smithi086.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42410.log.gz 2024-04-20T21:31:41.573 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42248.log 2024-04-20T21:31:41.574 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53643.log.gz 2024-04-20T21:31:41.574 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39050.log 2024-04-20T21:31:41.574 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46475.log.gz 2024-04-20T21:31:41.574 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42248.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42248.log.gz 2024-04-20T21:31:41.574 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46061.log 2024-04-20T21:31:41.574 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48039.log 2024-04-20T21:31:41.575 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39050.log.gz 2024-04-20T21:31:41.575 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46811.log 2024-04-20T21:31:41.575 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46061.log.gz 2024-04-20T21:31:41.575 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49989.log 2024-04-20T21:31:41.575 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48039.log.gz 2024-04-20T21:31:41.575 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55636.log 2024-04-20T21:31:41.576 INFO:teuthology.orchestra.run.smithi086.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46811.log.gz 2024-04-20T21:31:41.576 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50375.log 2024-04-20T21:31:41.576 INFO:teuthology.orchestra.run.smithi086.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.49989.log.gz 2024-04-20T21:31:41.576 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55636.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55636.log.gz 2024-04-20T21:31:41.576 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44155.log 2024-04-20T21:31:41.576 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.50375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31918.log 2024-04-20T21:31:41.577 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52412.log 2024-04-20T21:31:41.577 INFO:teuthology.orchestra.run.smithi086.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.0.50375.log.gz 2024-04-20T21:31:41.577 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.31918.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.44155.log.gz 2024-04-20T21:31:41.577 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.31918.log.gz 2024-04-20T21:31:41.577 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38516.log 2024-04-20T21:31:41.577 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46854.log 2024-04-20T21:31:41.577 INFO:teuthology.orchestra.run.smithi086.stderr: 88.0%/var/log/ceph/ceph-client.admin.38516.log: -- replaced with /var/log/ceph/ceph-client.admin.52412.log.gz 2024-04-20T21:31:41.578 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38516.log.gz 2024-04-20T21:31:41.578 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44708.log 2024-04-20T21:31:41.578 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51521.log 2024-04-20T21:31:41.578 INFO:teuthology.orchestra.run.smithi086.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46854.log.gz 2024-04-20T21:31:41.578 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39903.log 2024-04-20T21:31:41.578 INFO:teuthology.orchestra.run.smithi086.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44708.log.gz 2024-04-20T21:31:41.579 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51052.log 2024-04-20T21:31:41.579 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51521.log.gz 2024-04-20T21:31:41.579 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34257.log 2024-04-20T21:31:41.579 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39903.log.gz 2024-04-20T21:31:41.579 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56935.log 2024-04-20T21:31:41.579 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51052.log.gz 2024-04-20T21:31:41.580 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34671.log 2024-04-20T21:31:41.580 INFO:teuthology.orchestra.run.smithi086.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34257.log.gz 2024-04-20T21:31:41.580 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70080.log 2024-04-20T21:31:41.580 INFO:teuthology.orchestra.run.smithi086.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.56935.log.gz 2024-04-20T21:31:41.580 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48297.log 2024-04-20T21:31:41.580 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.70080.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34671.log.gz 2024-04-20T21:31:41.580 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57863.log 2024-04-20T21:31:41.581 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.70080.log.gz 2024-04-20T21:31:41.581 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32448.log 2024-04-20T21:31:41.581 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48297.log.gz 2024-04-20T21:31:41.581 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33047.log 2024-04-20T21:31:41.581 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57863.log.gz 2024-04-20T21:31:41.582 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.32448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44987.log 2024-04-20T21:31:41.582 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32448.log.gz 2024-04-20T21:31:41.582 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38829.log 2024-04-20T21:31:41.582 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33047.log.gz 2024-04-20T21:31:41.582 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48931.log 2024-04-20T21:31:41.582 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44987.log.gz 2024-04-20T21:31:41.583 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62739.log 2024-04-20T21:31:41.583 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38829.log.gz 2024-04-20T21:31:41.583 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63327.log 2024-04-20T21:31:41.583 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48931.log.gz 2024-04-20T21:31:41.583 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71180.log 2024-04-20T21:31:41.583 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62739.log.gz 2024-04-20T21:31:41.583 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.63327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55136.log 2024-04-20T21:31:41.584 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.63327.log.gz 2024-04-20T21:31:41.584 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.71180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53601.log 2024-04-20T21:31:41.584 INFO:teuthology.orchestra.run.smithi086.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.71180.log.gz 2024-04-20T21:31:41.584 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63880.log 2024-04-20T21:31:41.593 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53601.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.55136.log.gz 2024-04-20T21:31:41.594 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54878.log 2024-04-20T21:31:41.594 INFO:teuthology.orchestra.run.smithi086.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53601.log.gz 2024-04-20T21:31:41.594 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.63880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36236.log 2024-04-20T21:31:41.594 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.63880.log.gz 2024-04-20T21:31:41.594 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56538.log 2024-04-20T21:31:41.594 INFO:teuthology.orchestra.run.smithi086.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.54878.log.gz/var/log/ceph/ceph-client.admin.36236.log: 2024-04-20T21:31:41.594 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69786.log 2024-04-20T21:31:41.594 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36236.log.gz 2024-04-20T21:31:41.594 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.56538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46275.log 2024-04-20T21:31:41.594 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.69786.log: 92.3% -- replaced with /var/log/ceph/ceph-client.0.56538.log.gz 2024-04-20T21:31:41.594 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53333.log 2024-04-20T21:31:41.594 INFO:teuthology.orchestra.run.smithi086.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.69786.log.gz 2024-04-20T21:31:41.594 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46275.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46275.log.gz 2024-04-20T21:31:41.595 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47731.log 2024-04-20T21:31:41.595 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39777.log 2024-04-20T21:31:41.595 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53333.log.gz 2024-04-20T21:31:41.595 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66617.log 2024-04-20T21:31:41.595 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47731.log.gz 2024-04-20T21:31:41.595 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63667.log 2024-04-20T21:31:41.595 INFO:teuthology.orchestra.run.smithi086.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39777.log.gz 2024-04-20T21:31:41.595 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.66617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22480.log 2024-04-20T21:31:41.595 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66617.log.gz 2024-04-20T21:31:41.595 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.63667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34419.log 2024-04-20T21:31:41.595 INFO:teuthology.orchestra.run.smithi086.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.63667.log.gz 2024-04-20T21:31:41.595 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.22480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58540.log 2024-04-20T21:31:41.595 INFO:teuthology.orchestra.run.smithi086.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.22480.log.gz 2024-04-20T21:31:41.596 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71012.log 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34419.log.gz 2024-04-20T21:31:41.596 INFO:teuthology.orchestra.run.smithi086.stderr: 2024-04-20T21:31:41.596 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58540.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58540.log.gz 2024-04-20T21:31:41.596 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51364.log 2024-04-20T21:31:41.596 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.71012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66220.log 2024-04-20T21:31:41.596 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.71012.log.gz 2024-04-20T21:31:41.596 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39693.log 2024-04-20T21:31:41.596 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51364.log.gz 2024-04-20T21:31:41.596 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.66220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46145.log 2024-04-20T21:31:41.596 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.66220.log.gz 2024-04-20T21:31:41.596 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68630.log 2024-04-20T21:31:41.596 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39693.log.gz 2024-04-20T21:31:41.598 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66714.log 2024-04-20T21:31:41.598 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46145.log.gz 2024-04-20T21:31:41.598 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.68630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55452.log 2024-04-20T21:31:41.598 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.68630.log.gz 2024-04-20T21:31:41.598 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.66714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34215.log 2024-04-20T21:31:41.598 INFO:teuthology.orchestra.run.smithi086.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.66714.log.gz 2024-04-20T21:31:41.598 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57569.log 2024-04-20T21:31:41.598 INFO:teuthology.orchestra.run.smithi086.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55452.log.gz 2024-04-20T21:31:41.599 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34215.log: gzip 88.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.34215.log.gz 2024-04-20T21:31:41.599 INFO:teuthology.orchestra.run.smithi086.stderr: --verbose -- /var/log/ceph/ceph.log 2024-04-20T21:31:41.600 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35049.log 2024-04-20T21:31:41.600 INFO:teuthology.orchestra.run.smithi086.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.57569.log.gz 2024-04-20T21:31:41.601 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30171.log 2024-04-20T21:31:41.602 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46327.log 2024-04-20T21:31:41.602 INFO:teuthology.orchestra.run.smithi086.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35049.log.gz 2024-04-20T21:31:41.603 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.30171.log: gzip -5 --verbose 87.4% -- -- replaced with /var/log/ceph/ceph-client.admin.30171.log.gz /var/log/ceph/ceph-client.admin.45113.log 2024-04-20T21:31:41.603 INFO:teuthology.orchestra.run.smithi086.stderr: 2024-04-20T21:31:41.603 INFO:teuthology.orchestra.run.smithi086.stderr: 87.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-20T21:31:41.604 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61800.log 2024-04-20T21:31:41.605 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-mon.b.log: 91.5% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-20T21:31:41.605 INFO:teuthology.orchestra.run.smithi144.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-20T21:31:41.605 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46327.log.gz 2024-04-20T21:31:41.606 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45113.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45113.log.gz 2024-04-20T21:31:41.606 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57080.log 2024-04-20T21:31:41.607 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55552.log 2024-04-20T21:31:41.607 INFO:teuthology.orchestra.run.smithi086.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.61800.log.gz 2024-04-20T21:31:41.608 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65527.log 2024-04-20T21:31:41.608 INFO:teuthology.orchestra.run.smithi086.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57080.log.gz 2024-04-20T21:31:41.609 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30087.log 2024-04-20T21:31:41.609 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55552.log.gz 2024-04-20T21:31:41.610 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.65527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37336.log 2024-04-20T21:31:41.610 INFO:teuthology.orchestra.run.smithi086.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.65527.log.gz 2024-04-20T21:31:41.611 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.30087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48339.log 2024-04-20T21:31:41.611 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.30087.log.gz 2024-04-20T21:31:41.612 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63836.log 2024-04-20T21:31:41.612 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37336.log.gz 2024-04-20T21:31:41.613 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42027.log 2024-04-20T21:31:41.613 INFO:teuthology.orchestra.run.smithi086.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.48339.log.gz 2024-04-20T21:31:41.614 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.63836.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.63836.log.gz 2024-04-20T21:31:41.615 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69828.log 2024-04-20T21:31:41.616 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36148.log 2024-04-20T21:31:41.616 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42027.log.gz 2024-04-20T21:31:41.617 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.69828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59695.log 2024-04-20T21:31:41.617 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69828.log.gz 2024-04-20T21:31:41.618 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58498.log 2024-04-20T21:31:41.618 INFO:teuthology.orchestra.run.smithi086.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36148.log.gz 2024-04-20T21:31:41.619 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33687.log 2024-04-20T21:31:41.619 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59695.log.gz 2024-04-20T21:31:41.620 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22214.log 2024-04-20T21:31:41.620 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58498.log.gz 2024-04-20T21:31:41.621 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55326.log 2024-04-20T21:31:41.621 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33687.log.gz 2024-04-20T21:31:41.622 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.22214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38661.log 2024-04-20T21:31:41.622 INFO:teuthology.orchestra.run.smithi086.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.22214.log.gz 2024-04-20T21:31:41.623 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55326.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-20T21:31:41.623 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55326.log.gz 2024-04-20T21:31:41.624 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70393.log 2024-04-20T21:31:41.624 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38661.log.gz 2024-04-20T21:31:41.625 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64375.log 2024-04-20T21:31:41.626 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.70393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32658.log 2024-04-20T21:31:41.626 INFO:teuthology.orchestra.run.smithi086.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.70393.log.gz 2024-04-20T21:31:41.627 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.64375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37724.log 2024-04-20T21:31:41.627 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.64375.log.gz 2024-04-20T21:31:41.628 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.32658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34461.log 2024-04-20T21:31:41.628 INFO:teuthology.orchestra.run.smithi086.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.32658.log.gz 2024-04-20T21:31:41.629 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46643.log 2024-04-20T21:31:41.629 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37724.log.gz 2024-04-20T21:31:41.630 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63159.log 2024-04-20T21:31:41.630 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34461.log.gz 2024-04-20T21:31:41.631 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46643.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46643.log.gz 2024-04-20T21:31:41.631 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55594.log 2024-04-20T21:31:41.632 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.63159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50097.log 2024-04-20T21:31:41.633 INFO:teuthology.orchestra.run.smithi086.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.63159.log.gz 2024-04-20T21:31:41.633 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70928.log 2024-04-20T21:31:41.633 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55594.log.gz 2024-04-20T21:31:41.634 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30339.log 2024-04-20T21:31:41.634 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50097.log.gz 2024-04-20T21:31:41.635 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.70928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71096.log 2024-04-20T21:31:41.636 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.70928.log.gz 2024-04-20T21:31:41.636 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.30339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54620.log 2024-04-20T21:31:41.637 INFO:teuthology.orchestra.run.smithi086.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.30339.log.gz 2024-04-20T21:31:41.638 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.71096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49621.log 2024-04-20T21:31:41.638 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.71096.log.gz 2024-04-20T21:31:41.638 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61740.log 2024-04-20T21:31:41.644 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49621.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.54620.log.gz 2024-04-20T21:31:41.644 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63075.log 2024-04-20T21:31:41.644 INFO:teuthology.orchestra.run.smithi086.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.49621.log.gz 2024-04-20T21:31:41.644 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58115.log 2024-04-20T21:31:41.644 INFO:teuthology.orchestra.run.smithi086.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.61740.log.gz 2024-04-20T21:31:41.644 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.63075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45720.log 2024-04-20T21:31:41.644 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.63075.log.gz 2024-04-20T21:31:41.645 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49512.log 2024-04-20T21:31:41.645 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58115.log.gz 2024-04-20T21:31:41.645 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37420.log 2024-04-20T21:31:41.645 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45720.log.gz 2024-04-20T21:31:41.645 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56152.log 2024-04-20T21:31:41.645 INFO:teuthology.orchestra.run.smithi086.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.49512.log.gz 2024-04-20T21:31:41.646 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67717.log 2024-04-20T21:31:41.646 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37420.log.gz 2024-04-20T21:31:41.647 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40887.log 2024-04-20T21:31:41.647 INFO:teuthology.orchestra.run.smithi086.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.56152.log.gz 2024-04-20T21:31:41.648 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.67717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61916.log 2024-04-20T21:31:41.648 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.67717.log.gz 2024-04-20T21:31:41.649 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47547.log 2024-04-20T21:31:41.649 INFO:teuthology.orchestra.run.smithi086.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40887.log.gz 2024-04-20T21:31:41.650 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40930.log 2024-04-20T21:31:41.650 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.61916.log.gz 2024-04-20T21:31:41.651 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63285.log 2024-04-20T21:31:41.651 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47547.log.gz 2024-04-20T21:31:41.652 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48405.log 2024-04-20T21:31:41.652 INFO:teuthology.orchestra.run.smithi086.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40930.log.gz 2024-04-20T21:31:41.653 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.63285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29541.log 2024-04-20T21:31:41.653 INFO:teuthology.orchestra.run.smithi086.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.63285.log.gz 2024-04-20T21:31:41.654 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30675.log 2024-04-20T21:31:41.654 INFO:teuthology.orchestra.run.smithi086.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48405.log.gz 2024-04-20T21:31:41.655 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.29541.log: gzip -5 88.2% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.29541.log.gz /var/log/ceph/ceph-client.0.53050.log 2024-04-20T21:31:41.655 INFO:teuthology.orchestra.run.smithi086.stderr: 2024-04-20T21:31:41.656 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.30675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63243.log 2024-04-20T21:31:41.656 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.30675.log.gz 2024-04-20T21:31:41.657 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.53050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.69175.log 2024-04-20T21:31:41.658 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.63243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33929.log 2024-04-20T21:31:41.658 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.63243.log.gz 2024-04-20T21:31:41.659 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.69175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52706.log 2024-04-20T21:31:41.659 INFO:teuthology.orchestra.run.smithi086.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.0.53050.log.gz 2024-04-20T21:31:41.660 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70592.log 2024-04-20T21:31:41.660 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33929.log.gz 2024-04-20T21:31:41.661 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28195.log 2024-04-20T21:31:41.662 INFO:teuthology.orchestra.run.smithi086.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.52706.log.gz 2024-04-20T21:31:41.662 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.70592.log: 93.0% -- replaced with /var/log/ceph/ceph-client.0.69175.log.gz 2024-04-20T21:31:41.663 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.70592.log.gz 2024-04-20T21:31:41.663 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66320.log 2024-04-20T21:31:41.664 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.28195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43057.log 2024-04-20T21:31:41.664 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.28195.log.gz 2024-04-20T21:31:41.665 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.66320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51220.log 2024-04-20T21:31:41.665 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.66320.log.gz 2024-04-20T21:31:41.666 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43057.log: gzip 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43057.log.gz -5 2024-04-20T21:31:41.666 INFO:teuthology.orchestra.run.smithi086.stderr: --verbose -- /var/log/ceph/ceph-client.admin.41588.log 2024-04-20T21:31:41.667 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58073.log 2024-04-20T21:31:41.667 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51220.log.gz 2024-04-20T21:31:41.668 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71390.log 2024-04-20T21:31:41.668 INFO:teuthology.orchestra.run.smithi086.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41588.log.gz 2024-04-20T21:31:41.669 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58073.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-20T21:31:41.669 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58073.log.gz 2024-04-20T21:31:41.670 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.71390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42973.log 2024-04-20T21:31:41.670 INFO:teuthology.orchestra.run.smithi086.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.71390.log.gz 2024-04-20T21:31:41.671 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68683.log 2024-04-20T21:31:41.672 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29555.log 2024-04-20T21:31:41.672 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42973.log.gz 2024-04-20T21:31:41.673 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.68683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51407.log 2024-04-20T21:31:41.673 INFO:teuthology.orchestra.run.smithi086.stderr: 89.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-20T21:31:41.674 INFO:teuthology.orchestra.run.smithi086.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.68683.log.gz 2024-04-20T21:31:41.674 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.29555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36106.log 2024-04-20T21:31:41.675 INFO:teuthology.orchestra.run.smithi086.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.29555.log.gz 2024-04-20T21:31:41.675 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61478.log 2024-04-20T21:31:41.675 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51407.log.gz 2024-04-20T21:31:41.676 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68771.log 2024-04-20T21:31:41.677 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36106.log.gz 2024-04-20T21:31:41.677 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33855.log 2024-04-20T21:31:41.678 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.68771.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.61478.log.gz 2024-04-20T21:31:41.679 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41630.log 2024-04-20T21:31:41.679 INFO:teuthology.orchestra.run.smithi086.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.68771.log.gz 2024-04-20T21:31:41.680 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.36892.log 2024-04-20T21:31:41.680 INFO:teuthology.orchestra.run.smithi086.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33855.log.gz 2024-04-20T21:31:41.681 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54078.log 2024-04-20T21:31:41.681 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41630.log.gz 2024-04-20T21:31:41.681 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.36892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37977.log 2024-04-20T21:31:41.683 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67759.log 2024-04-20T21:31:41.683 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54078.log.gz 2024-04-20T21:31:41.684 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39425.log 2024-04-20T21:31:41.684 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.67759.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.37977.log.gz 2024-04-20T21:31:41.685 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.67759.log.gz 2024-04-20T21:31:41.685 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44750.log 2024-04-20T21:31:41.686 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39425.log: 92.9% -- replaced with /var/log/ceph/ceph-client.0.36892.log.gz 2024-04-20T21:31:41.687 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29203.log 2024-04-20T21:31:41.687 INFO:teuthology.orchestra.run.smithi086.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39425.log.gz 2024-04-20T21:31:41.689 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29533.log 2024-04-20T21:31:41.689 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44750.log.gz 2024-04-20T21:31:41.689 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.29203.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.29203.log.gz 2024-04-20T21:31:41.689 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37646.log 2024-04-20T21:31:41.690 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.29533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32196.log 2024-04-20T21:31:41.690 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.29533.log.gz 2024-04-20T21:31:41.691 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46559.log 2024-04-20T21:31:41.691 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37646.log.gz 2024-04-20T21:31:41.692 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.32196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68470.log 2024-04-20T21:31:41.692 INFO:teuthology.orchestra.run.smithi086.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.32196.log.gz 2024-04-20T21:31:41.693 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45155.log 2024-04-20T21:31:41.693 INFO:teuthology.orchestra.run.smithi086.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46559.log.gz 2024-04-20T21:31:41.694 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.68470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53950.log 2024-04-20T21:31:41.695 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45155.log: 91.9% -- replaced with /var/log/ceph/ceph-client.admin.68470.log.gz 2024-04-20T21:31:41.695 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61019.log 2024-04-20T21:31:41.696 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45155.log.gz 2024-04-20T21:31:41.696 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51689.log 2024-04-20T21:31:41.696 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53950.log.gz 2024-04-20T21:31:41.697 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61019.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.61019.log.gz 2024-04-20T21:31:41.697 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44263.log 2024-04-20T21:31:41.698 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57947.log 2024-04-20T21:31:41.698 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51689.log.gz 2024-04-20T21:31:41.699 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65054.log 2024-04-20T21:31:41.700 INFO:teuthology.orchestra.run.smithi086.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.44263.log.gz 2024-04-20T21:31:41.701 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52622.log 2024-04-20T21:31:41.701 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57947.log.gz 2024-04-20T21:31:41.702 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.65054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56827.log 2024-04-20T21:31:41.702 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65054.log.gz 2024-04-20T21:31:41.702 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47977.log 2024-04-20T21:31:41.703 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52622.log.gz 2024-04-20T21:31:41.704 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57001.log 2024-04-20T21:31:41.704 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47977.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.56827.log.gz 2024-04-20T21:31:41.705 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47977.log.gz 2024-04-20T21:31:41.705 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50968.log 2024-04-20T21:31:41.706 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29037.log 2024-04-20T21:31:41.706 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57001.log.gz 2024-04-20T21:31:41.707 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62949.log 2024-04-20T21:31:41.707 INFO:teuthology.orchestra.run.smithi086.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50968.log.gz 2024-04-20T21:31:41.708 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.29037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52454.log 2024-04-20T21:31:41.708 INFO:teuthology.orchestra.run.smithi086.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.29037.log.gz 2024-04-20T21:31:41.709 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53685.log 2024-04-20T21:31:41.709 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.62949.log.gz 2024-04-20T21:31:41.710 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52454.log: gzip -5 --verbose 88.1% -- -- replaced with /var/log/ceph/ceph-client.admin.52454.log.gz /var/log/ceph/ceph-client.admin.42452.log 2024-04-20T21:31:41.710 INFO:teuthology.orchestra.run.smithi086.stderr: 2024-04-20T21:31:41.711 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61566.log 2024-04-20T21:31:41.711 INFO:teuthology.orchestra.run.smithi086.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53685.log.gz 2024-04-20T21:31:41.712 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64017.log 2024-04-20T21:31:41.712 INFO:teuthology.orchestra.run.smithi086.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42452.log.gz 2024-04-20T21:31:41.713 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40011.log 2024-04-20T21:31:41.714 INFO:teuthology.orchestra.run.smithi086.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.61566.log.gz 2024-04-20T21:31:41.714 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.64017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57164.log 2024-04-20T21:31:41.715 INFO:teuthology.orchestra.run.smithi086.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.64017.log.gz 2024-04-20T21:31:41.716 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34923.log 2024-04-20T21:31:41.716 INFO:teuthology.orchestra.run.smithi086.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40011.log.gz 2024-04-20T21:31:41.717 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57164.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57164.log.gz 2024-04-20T21:31:41.717 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44413.log 2024-04-20T21:31:41.718 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38306.log 2024-04-20T21:31:41.718 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34923.log.gz 2024-04-20T21:31:41.719 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33273.log 2024-04-20T21:31:41.719 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38306.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.44413.log.gz 2024-04-20T21:31:41.720 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53559.log 2024-04-20T21:31:41.720 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38306.log.gz 2024-04-20T21:31:41.721 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33273.log: gzip -5 88.5% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.33273.log.gz /var/log/ceph/ceph-client.admin.55052.log 2024-04-20T21:31:41.721 INFO:teuthology.orchestra.run.smithi086.stderr: 2024-04-20T21:31:41.722 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41225.log 2024-04-20T21:31:41.722 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53559.log.gz 2024-04-20T21:31:41.723 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37294.log 2024-04-20T21:31:41.723 INFO:teuthology.orchestra.run.smithi086.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.55052.log.gz 2024-04-20T21:31:41.724 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67020.log 2024-04-20T21:31:41.724 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41225.log.gz 2024-04-20T21:31:41.725 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44329.log 2024-04-20T21:31:41.725 INFO:teuthology.orchestra.run.smithi086.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37294.log.gz 2024-04-20T21:31:41.726 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.67020.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.67020.log.gz 2024-04-20T21:31:41.727 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41099.log 2024-04-20T21:31:41.728 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29901.log 2024-04-20T21:31:41.729 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41099.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.44329.log.gz 2024-04-20T21:31:41.729 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34299.log 2024-04-20T21:31:41.729 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41099.log.gz 2024-04-20T21:31:41.730 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.29901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42111.log 2024-04-20T21:31:41.730 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.29901.log.gz 2024-04-20T21:31:41.731 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44792.log 2024-04-20T21:31:41.731 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34299.log.gz 2024-04-20T21:31:41.732 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46938.log 2024-04-20T21:31:41.732 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42111.log.gz 2024-04-20T21:31:41.733 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44792.log: gzip 88.7% -5 -- replaced with /var/log/ceph/ceph-client.admin.44792.log.gz 2024-04-20T21:31:41.733 INFO:teuthology.orchestra.run.smithi086.stderr: --verbose -- /var/log/ceph/ceph-client.admin.50689.log 2024-04-20T21:31:41.734 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38474.log 2024-04-20T21:31:41.735 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50689.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.46938.log.gz 2024-04-20T21:31:41.735 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64483.log 2024-04-20T21:31:41.735 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50689.log.gz 2024-04-20T21:31:41.736 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68725.log 2024-04-20T21:31:41.736 INFO:teuthology.orchestra.run.smithi086.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38474.log.gz 2024-04-20T21:31:41.737 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.64483.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-20T21:31:41.737 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.64483.log.gz 2024-04-20T21:31:41.738 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.68725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52370.log 2024-04-20T21:31:41.739 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.68725.log.gz 2024-04-20T21:31:41.739 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30507.log 2024-04-20T21:31:41.740 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47899.log 2024-04-20T21:31:41.741 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52370.log.gz 2024-04-20T21:31:41.742 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.30507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51136.log 2024-04-20T21:31:41.742 INFO:teuthology.orchestra.run.smithi086.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.30507.log.gz 2024-04-20T21:31:41.743 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42536.log 2024-04-20T21:31:41.743 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47899.log.gz 2024-04-20T21:31:41.744 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51136.log: 88.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.51136.log.gz 2024-04-20T21:31:41.744 INFO:teuthology.orchestra.run.smithi086.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.53846.log 2024-04-20T21:31:41.745 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38558.log 2024-04-20T21:31:41.745 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42536.log.gz 2024-04-20T21:31:41.746 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28953.log 2024-04-20T21:31:41.746 INFO:teuthology.orchestra.run.smithi086.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53846.log.gz 2024-04-20T21:31:41.747 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38745.log 2024-04-20T21:31:41.747 INFO:teuthology.orchestra.run.smithi086.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38558.log.gz 2024-04-20T21:31:41.748 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.28953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51563.log 2024-04-20T21:31:41.748 INFO:teuthology.orchestra.run.smithi086.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.28953.log.gz 2024-04-20T21:31:41.749 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41943.log 2024-04-20T21:31:41.749 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38745.log.gz 2024-04-20T21:31:41.750 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54035.log 2024-04-20T21:31:41.750 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51563.log.gz 2024-04-20T21:31:41.751 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68362.log 2024-04-20T21:31:41.752 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41943.log.gz 2024-04-20T21:31:41.753 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30297.log 2024-04-20T21:31:41.753 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54035.log.gz 2024-04-20T21:31:41.753 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.68362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66362.log 2024-04-20T21:31:41.754 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.30297.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.68362.log.gz 2024-04-20T21:31:41.755 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.30297.log.gz 2024-04-20T21:31:41.755 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71222.log 2024-04-20T21:31:41.756 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.66362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56893.log 2024-04-20T21:31:41.756 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.66362.log.gz 2024-04-20T21:31:41.757 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.71222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58252.log 2024-04-20T21:31:41.757 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71222.log.gz 2024-04-20T21:31:41.758 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30633.log 2024-04-20T21:31:41.759 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56893.log.gz 2024-04-20T21:31:41.759 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57248.log 2024-04-20T21:31:41.759 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58252.log.gz 2024-04-20T21:31:41.760 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.30633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34881.log 2024-04-20T21:31:41.760 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.30633.log.gz 2024-04-20T21:31:41.761 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28575.log 2024-04-20T21:31:41.761 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57248.log.gz 2024-04-20T21:31:41.762 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45594.log 2024-04-20T21:31:41.763 INFO:teuthology.orchestra.run.smithi086.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34881.log.gz 2024-04-20T21:31:41.763 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.28575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56008.log 2024-04-20T21:31:41.764 INFO:teuthology.orchestra.run.smithi086.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.28575.log.gz 2024-04-20T21:31:41.765 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38264.log 2024-04-20T21:31:41.765 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45594.log.gz 2024-04-20T21:31:41.766 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68063.log 2024-04-20T21:31:41.766 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56008.log.gz 2024-04-20T21:31:41.766 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52114.log 2024-04-20T21:31:41.767 INFO:teuthology.orchestra.run.smithi086.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38264.log.gz 2024-04-20T21:31:41.768 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.68063.log: 89.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.68063.log.gz 2024-04-20T21:31:41.768 INFO:teuthology.orchestra.run.smithi086.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.69506.log 2024-04-20T21:31:41.769 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51304.log 2024-04-20T21:31:41.769 INFO:teuthology.orchestra.run.smithi086.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52114.log.gz 2024-04-20T21:31:41.770 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.69506.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.69506.log.gz 2024-04-20T21:31:41.770 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68428.log 2024-04-20T21:31:41.771 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32154.log 2024-04-20T21:31:41.771 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51304.log.gz 2024-04-20T21:31:41.772 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.68428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49241.log 2024-04-20T21:31:41.773 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.32154.log: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.68428.log.gz 2024-04-20T21:31:41.774 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50926.log 2024-04-20T21:31:41.774 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32154.log.gz 2024-04-20T21:31:41.774 INFO:teuthology.orchestra.run.smithi086.stderr:gzip/var/log/ceph/ceph-client.admin.49241.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.51094.log 2024-04-20T21:31:41.775 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49241.log.gz 2024-04-20T21:31:41.775 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50926.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50926.log.gz 2024-04-20T21:31:41.776 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44047.log 2024-04-20T21:31:41.777 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45510.log 2024-04-20T21:31:41.777 INFO:teuthology.orchestra.run.smithi086.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51094.log.gz 2024-04-20T21:31:41.778 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38955.log 2024-04-20T21:31:41.779 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45510.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.44047.log.gz 2024-04-20T21:31:41.779 INFO:teuthology.orchestra.run.smithi086.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45510.log.gz 2024-04-20T21:31:41.779 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46727.log 2024-04-20T21:31:41.780 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64189.log 2024-04-20T21:31:41.781 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38955.log.gz 2024-04-20T21:31:41.782 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48973.log 2024-04-20T21:31:41.782 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46727.log.gz 2024-04-20T21:31:41.783 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.64189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36404.log 2024-04-20T21:31:41.783 INFO:teuthology.orchestra.run.smithi086.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.64189.log.gz 2024-04-20T21:31:41.784 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48973.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-20T21:31:41.784 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48973.log.gz 2024-04-20T21:31:41.785 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55840.log 2024-04-20T21:31:41.785 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36404.log.gz 2024-04-20T21:31:41.786 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49729.log 2024-04-20T21:31:41.787 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30465.log 2024-04-20T21:31:41.788 INFO:teuthology.orchestra.run.smithi086.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55840.log.gz 2024-04-20T21:31:41.789 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57359.log 2024-04-20T21:31:41.789 INFO:teuthology.orchestra.run.smithi086.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.49729.log.gz 2024-04-20T21:31:41.790 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.30465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54836.log 2024-04-20T21:31:41.790 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.30465.log.gz 2024-04-20T21:31:41.791 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39819.log 2024-04-20T21:31:41.792 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57359.log.gz 2024-04-20T21:31:41.793 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33131.log 2024-04-20T21:31:41.793 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54836.log.gz 2024-04-20T21:31:41.794 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56110.log 2024-04-20T21:31:41.794 INFO:teuthology.orchestra.run.smithi086.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39819.log.gz 2024-04-20T21:31:41.795 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.67298.log 2024-04-20T21:31:41.795 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33131.log.gz 2024-04-20T21:31:41.796 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47857.log 2024-04-20T21:31:41.797 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56110.log.gz 2024-04-20T21:31:41.797 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.67298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56194.log 2024-04-20T21:31:41.799 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41859.log 2024-04-20T21:31:41.799 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47857.log.gz 2024-04-20T21:31:41.800 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29210.log 2024-04-20T21:31:41.801 INFO:teuthology.orchestra.run.smithi086.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.56194.log.gz 2024-04-20T21:31:41.802 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38390.log 2024-04-20T21:31:41.802 INFO:teuthology.orchestra.run.smithi086.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41859.log.gz 2024-04-20T21:31:41.802 INFO:teuthology.orchestra.run.smithi086.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.0.67298.log.gz 2024-04-20T21:31:41.803 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.29210.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-20T21:31:41.803 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.29210.log.gz 2024-04-20T21:31:41.804 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62991.log 2024-04-20T21:31:41.804 INFO:teuthology.orchestra.run.smithi086.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38390.log.gz 2024-04-20T21:31:41.805 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-20T21:31:41.806 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62991.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62991.log.gz 2024-04-20T21:31:41.806 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31876.log 2024-04-20T21:31:41.807 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51476.log 2024-04-20T21:31:41.808 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.31876.log: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59864.log 2024-04-20T21:31:41.809 INFO:teuthology.orchestra.run.smithi086.stderr: -- replaced with /var/log/ceph/ceph-client.admin.31876.log.gz 2024-04-20T21:31:41.810 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42494.log 2024-04-20T21:31:41.810 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51476.log.gz 2024-04-20T21:31:41.811 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59864.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59864.log.gz 2024-04-20T21:31:41.812 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67985.log 2024-04-20T21:31:41.812 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42494.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42494.log.gz 2024-04-20T21:31:41.813 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55720.log 2024-04-20T21:31:41.814 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38348.log 2024-04-20T21:31:41.814 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.67985.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.67985.log.gz 2024-04-20T21:31:41.815 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55720.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55720.log.gz 2024-04-20T21:31:41.816 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29216.log 2024-04-20T21:31:41.816 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38348.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38348.log.gz 2024-04-20T21:31:41.817 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66262.log 2024-04-20T21:31:41.818 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39651.log 2024-04-20T21:31:41.819 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.29216.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.29216.log.gz 2024-04-20T21:31:41.820 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.66262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43963.log 2024-04-20T21:31:41.820 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.66262.log.gz 2024-04-20T21:31:41.820 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39651.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39651.log.gz 2024-04-20T21:31:41.821 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36362.log 2024-04-20T21:31:41.822 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61061.log 2024-04-20T21:31:41.823 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36362.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.43963.log.gz 2024-04-20T21:31:41.823 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36362.log.gz 2024-04-20T21:31:41.824 INFO:teuthology.orchestra.run.smithi086.stderr:gzip/var/log/ceph/ceph-client.admin.61061.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.49283.log 2024-04-20T21:31:41.824 INFO:teuthology.orchestra.run.smithi086.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.61061.log.gz 2024-04-20T21:31:41.825 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60006.log 2024-04-20T21:31:41.826 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-20T21:31:41.827 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49283.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49283.log.gz 2024-04-20T21:31:41.828 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52244.log 2024-04-20T21:31:41.828 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60006.log.gz 2024-04-20T21:31:41.829 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60048.log 2024-04-20T21:31:41.830 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52244.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.52244.log.gz 2024-04-20T21:31:41.831 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60048.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.60048.log.gz 2024-04-20T21:31:41.832 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37786.log 2024-04-20T21:31:41.834 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53291.log 2024-04-20T21:31:41.835 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37786.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37786.log.gz 2024-04-20T21:31:41.836 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53291.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.53291.log.gz 2024-04-20T21:31:41.837 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22582.log 2024-04-20T21:31:41.839 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63411.log 2024-04-20T21:31:41.840 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.22582.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.22582.log.gz 2024-04-20T21:31:41.841 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.63411.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.63411.log.gz 2024-04-20T21:31:41.841 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70760.log 2024-04-20T21:31:41.843 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42889.log 2024-04-20T21:31:41.844 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.70760.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.70760.log.gz 2024-04-20T21:31:41.846 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42889.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42889.log.gz 2024-04-20T21:31:41.846 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58763.log 2024-04-20T21:31:41.848 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42717.log 2024-04-20T21:31:41.849 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58763.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58763.log.gz 2024-04-20T21:31:41.850 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42717.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42717.log.gz 2024-04-20T21:31:41.851 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55905.log 2024-04-20T21:31:41.853 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39130.log 2024-04-20T21:31:41.856 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55905.log: /var/log/ceph/ceph-client.admin.39130.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39130.log.gz 2024-04-20T21:31:41.856 INFO:teuthology.orchestra.run.smithi086.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.55905.log.gz 2024-04-20T21:31:41.856 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34839.log 2024-04-20T21:31:41.857 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64231.log 2024-04-20T21:31:41.859 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34839.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34839.log.gz 2024-04-20T21:31:41.860 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.64231.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.64231.log.gz 2024-04-20T21:31:41.861 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40522.log 2024-04-20T21:31:41.862 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33315.log 2024-04-20T21:31:41.864 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40522.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.40522.log.gz 2024-04-20T21:31:41.865 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33315.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33315.log.gz 2024-04-20T21:31:41.865 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66824.log 2024-04-20T21:31:41.867 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65222.log 2024-04-20T21:31:41.868 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.66824.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.66824.log.gz 2024-04-20T21:31:41.870 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.65222.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.65222.log.gz 2024-04-20T21:31:41.870 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63975.log 2024-04-20T21:31:41.872 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41141.log 2024-04-20T21:31:41.873 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.63975.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.63975.log.gz 2024-04-20T21:31:41.874 INFO:teuthology.orchestra.run.smithi086.stderr:gzip/var/log/ceph/ceph-client.admin.41141.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.28995.log 2024-04-20T21:31:41.876 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28403.log 2024-04-20T21:31:41.876 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41141.log.gz 2024-04-20T21:31:41.876 INFO:teuthology.orchestra.run.smithi144.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-20T21:31:41.878 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.28995.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.28995.log.gz 2024-04-20T21:31:41.879 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.28403.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.28403.log.gz 2024-04-20T21:31:41.879 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46429.log 2024-04-20T21:31:41.881 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40821.log 2024-04-20T21:31:41.882 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46429.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46429.log.gz 2024-04-20T21:31:41.884 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40821.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.40821.log.gz 2024-04-20T21:31:41.884 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54464.log 2024-04-20T21:31:41.886 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35925.log 2024-04-20T21:31:41.889 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.54464.log: /var/log/ceph/ceph-client.admin.35925.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35925.log.gz 2024-04-20T21:31:41.890 INFO:teuthology.orchestra.run.smithi086.stderr:gzip 93.9% -- replaced with /var/log/ceph/ceph-client.0.54464.log.gz 2024-04-20T21:31:41.890 INFO:teuthology.orchestra.run.smithi086.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.32322.log 2024-04-20T21:31:41.892 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.32322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44479.log 2024-04-20T21:31:41.894 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50731.log 2024-04-20T21:31:41.894 INFO:teuthology.orchestra.run.smithi086.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32322.log.gz 2024-04-20T21:31:41.895 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29213.log 2024-04-20T21:31:41.896 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44479.log: /var/log/ceph/ceph-client.admin.50731.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44479.log.gz 2024-04-20T21:31:41.896 INFO:teuthology.orchestra.run.smithi086.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50731.log.gz 2024-04-20T21:31:41.898 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41714.log 2024-04-20T21:31:41.901 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.29213.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.29213.log.gz 2024-04-20T21:31:41.901 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63752.log 2024-04-20T21:31:41.901 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41714.log.gz 2024-04-20T21:31:41.902 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63201.log 2024-04-20T21:31:41.904 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.63752.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.63752.log.gz 2024-04-20T21:31:41.904 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30129.log/var/log/ceph/ceph-client.admin.63201.log: 2024-04-20T21:31:41.906 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.63201.log.gz 2024-04-20T21:31:41.906 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28099.log 2024-04-20T21:31:41.907 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.30129.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.30129.log.gz 2024-04-20T21:31:41.909 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69590.log/var/log/ceph/ceph-client.admin.28099.log: 2024-04-20T21:31:41.909 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.28099.log.gz 2024-04-20T21:31:41.909 INFO:teuthology.orchestra.run.smithi086.stderr: 91.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28487.log 2024-04-20T21:31:41.910 INFO:teuthology.orchestra.run.smithi086.stderr: -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-20T21:31:41.911 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.69590.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.69590.log.gz 2024-04-20T21:31:41.911 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67591.log 2024-04-20T21:31:41.912 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.28487.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.28487.log.gz 2024-04-20T21:31:41.913 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56677.log 2024-04-20T21:31:41.914 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.67591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29847.log 2024-04-20T21:31:41.914 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.67591.log.gz 2024-04-20T21:31:41.916 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56677.log: 90.7%/var/log/ceph/ceph-client.admin.29847.log: -- replaced with /var/log/ceph/ceph-client.admin.56677.log.gz 2024-04-20T21:31:41.917 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.29847.log.gz 2024-04-20T21:31:41.917 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45678.log 2024-04-20T21:31:41.918 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35385.log 2024-04-20T21:31:41.920 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49115.log 2024-04-20T21:31:41.920 INFO:teuthology.orchestra.run.smithi086.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45678.log.gz 2024-04-20T21:31:41.921 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35385.log: 89.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.35385.log.gz 2024-04-20T21:31:41.921 INFO:teuthology.orchestra.run.smithi086.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.35883.log 2024-04-20T21:31:41.923 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62272.log 2024-04-20T21:31:41.923 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49115.log.gz 2024-04-20T21:31:41.924 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37604.log 2024-04-20T21:31:41.924 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35883.log.gz 2024-04-20T21:31:41.925 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30423.log 2024-04-20T21:31:41.926 INFO:teuthology.orchestra.run.smithi086.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.62272.log.gz 2024-04-20T21:31:41.927 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60230.log 2024-04-20T21:31:41.927 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37604.log.gz 2024-04-20T21:31:41.929 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.30423.log: gzip -5 88.7% -- replaced with /var/log/ceph/ceph-client.admin.30423.log.gz 2024-04-20T21:31:41.929 INFO:teuthology.orchestra.run.smithi086.stderr: --verbose -- /var/log/ceph/ceph-client.admin.71138.log 2024-04-20T21:31:41.930 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60230.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.60230.log.gz 2024-04-20T21:31:41.930 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69632.log 2024-04-20T21:31:41.931 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.71138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45323.log 2024-04-20T21:31:41.932 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.71138.log.gz 2024-04-20T21:31:41.932 INFO:teuthology.orchestra.run.smithi144.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-20T21:31:41.933 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.69632.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.69632.log.gz 2024-04-20T21:31:41.934 INFO:teuthology.orchestra.run.smithi086.stderr:gzip/var/log/ceph/ceph-client.admin.45323.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.32742.log 2024-04-20T21:31:41.934 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45323.log.gz 2024-04-20T21:31:41.935 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35259.log 2024-04-20T21:31:41.936 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.32742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45552.log 2024-04-20T21:31:41.937 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32742.log.gz 2024-04-20T21:31:41.938 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35259.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35259.log.gz 2024-04-20T21:31:41.939 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46685.log 2024-04-20T21:31:41.941 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45552.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45552.log.gz 2024-04-20T21:31:41.941 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64059.log 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46685.log.gz 2024-04-20T21:31:41.941 INFO:teuthology.orchestra.run.smithi086.stderr: 2024-04-20T21:31:41.943 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62357.log 2024-04-20T21:31:41.944 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.64059.log: gzip 88.8% -5 -- replaced with /var/log/ceph/ceph-client.admin.64059.log.gz --verbose 2024-04-20T21:31:41.944 INFO:teuthology.orchestra.run.smithi086.stderr: -- /var/log/ceph/ceph-client.admin.58911.log 2024-04-20T21:31:41.945 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29812.log 2024-04-20T21:31:41.946 INFO:teuthology.orchestra.run.smithi086.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.62357.log.gz 2024-04-20T21:31:41.947 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58911.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58911.log.gz 2024-04-20T21:31:41.947 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34115.log 2024-04-20T21:31:41.950 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.29812.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.29812.log.gz 2024-04-20T21:31:41.951 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34115.log: gzip -5 --verbose 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34115.log.gz 2024-04-20T21:31:41.951 INFO:teuthology.orchestra.run.smithi086.stderr: -- /var/log/ceph/ceph-client.admin.49361.log 2024-04-20T21:31:41.952 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38871.log 2024-04-20T21:31:41.954 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56260.log 2024-04-20T21:31:41.954 INFO:teuthology.orchestra.run.smithi086.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.49361.log.gz 2024-04-20T21:31:41.955 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33089.log 2024-04-20T21:31:41.955 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38871.log.gz 2024-04-20T21:31:41.957 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56260.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56260.log.gz 2024-04-20T21:31:41.957 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54770.log/var/log/ceph/ceph-client.admin.33089.log: 2024-04-20T21:31:41.958 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33089.log.gz 2024-04-20T21:31:41.959 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65406.log 2024-04-20T21:31:41.960 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41672.log 2024-04-20T21:31:41.961 INFO:teuthology.orchestra.run.smithi086.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.54770.log.gz/var/log/ceph/ceph-client.admin.65406.log: 2024-04-20T21:31:41.961 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.65406.log.gz 2024-04-20T21:31:41.962 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59121.log 2024-04-20T21:31:41.963 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41672.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41672.log.gz 2024-04-20T21:31:41.963 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70970.log 2024-04-20T21:31:41.965 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47631.log 2024-04-20T21:31:41.965 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59121.log.gz 2024-04-20T21:31:41.966 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.70970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33771.log 2024-04-20T21:31:41.966 INFO:teuthology.orchestra.run.smithi086.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.70970.log.gz 2024-04-20T21:31:41.968 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41985.log 2024-04-20T21:31:41.968 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47631.log.gz 2024-04-20T21:31:41.969 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58455.log 2024-04-20T21:31:41.969 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33771.log.gz 2024-04-20T21:31:41.970 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41985.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-20T21:31:41.970 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41985.log.gz 2024-04-20T21:31:41.972 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58455.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58455.log.gz 2024-04-20T21:31:41.972 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52664.log 2024-04-20T21:31:41.974 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65096.log 2024-04-20T21:31:41.975 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52664.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52664.log.gz 2024-04-20T21:31:41.976 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.65096.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65096.log.gz 2024-04-20T21:31:41.977 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71054.log 2024-04-20T21:31:41.979 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32070.log 2024-04-20T21:31:41.980 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.71054.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.71054.log.gz 2024-04-20T21:31:41.980 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62907.log 2024-04-20T21:31:41.981 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.32070.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32070.log.gz 2024-04-20T21:31:41.982 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45281.log 2024-04-20T21:31:41.984 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62907.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.62907.log.gz 2024-04-20T21:31:41.984 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68273.log 2024-04-20T21:31:41.985 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45281.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45281.log.gz 2024-04-20T21:31:41.986 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45937.log 2024-04-20T21:31:41.987 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.68273.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.68273.log.gz 2024-04-20T21:31:41.988 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71587.log/var/log/ceph/ceph-client.admin.45937.log: 2024-04-20T21:31:41.989 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45937.log.gz 2024-04-20T21:31:41.990 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29209.log 2024-04-20T21:31:41.991 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.71587.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71587.log.gz 2024-04-20T21:31:41.992 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30549.log/var/log/ceph/ceph-client.admin.29209.log: 2024-04-20T21:31:41.993 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.29209.log.gz 2024-04-20T21:31:41.994 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46601.log 2024-04-20T21:31:41.995 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.30549.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.30549.log.gz 2024-04-20T21:31:41.996 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63033.log 2024-04-20T21:31:41.996 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46601.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46601.log.gz 2024-04-20T21:31:41.997 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49881.log 2024-04-20T21:31:41.999 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.63033.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63033.log.gz 2024-04-20T21:31:42.000 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5/var/log/ceph/ceph-client.admin.49881.log: --verbose -- /var/log/ceph/ceph-client.admin.66136.log 2024-04-20T21:31:42.001 INFO:teuthology.orchestra.run.smithi086.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.49881.log.gz 2024-04-20T21:31:42.002 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51987.log 2024-04-20T21:31:42.002 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.66136.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.66136.log.gz 2024-04-20T21:31:42.004 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51987.log: 91.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-20T21:31:42.004 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51987.log.gz 2024-04-20T21:31:42.005 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36055.log 2024-04-20T21:31:42.006 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.65871.log 2024-04-20T21:31:42.008 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35133.log 2024-04-20T21:31:42.008 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36055.log.gz 2024-04-20T21:31:42.010 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.65871.log: /var/log/ceph/ceph-client.admin.35133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30591.log 2024-04-20T21:31:42.010 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35133.log.gz 2024-04-20T21:31:42.011 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35588.log 2024-04-20T21:31:42.013 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.30591.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.30591.log.gz 2024-04-20T21:31:42.013 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29554.log 2024-04-20T21:31:42.014 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46231.log 2024-04-20T21:31:42.014 INFO:teuthology.orchestra.run.smithi086.stderr: 94.2% -- replaced with /var/log/ceph/ceph-client.0.65871.log.gz 2024-04-20T21:31:42.015 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35588.log.gz 2024-04-20T21:31:42.016 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61161.log 2024-04-20T21:31:42.017 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.29554.log: 88.7%/var/log/ceph/ceph-client.admin.46231.log: -- replaced with /var/log/ceph/ceph-client.admin.29554.log.gz 2024-04-20T21:31:42.017 INFO:teuthology.orchestra.run.smithi086.stderr:gzip 88.1% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.46231.log.gz -- 2024-04-20T21:31:42.017 INFO:teuthology.orchestra.run.smithi086.stderr: /var/log/ceph/ceph-client.admin.55244.log 2024-04-20T21:31:42.019 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38913.log 2024-04-20T21:31:42.019 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.61161.log.gz 2024-04-20T21:31:42.020 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32364.log 2024-04-20T21:31:42.021 INFO:teuthology.orchestra.run.smithi086.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.55244.log.gz 2024-04-20T21:31:42.021 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38913.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38913.log.gz 2024-04-20T21:31:42.022 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64147.log 2024-04-20T21:31:42.023 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.32364.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32364.log.gz 2024-04-20T21:31:42.023 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41057.log 2024-04-20T21:31:42.024 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.64147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29542.log 2024-04-20T21:31:42.024 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.64147.log.gz 2024-04-20T21:31:42.026 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41057.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41057.log.gz 2024-04-20T21:31:42.026 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.29542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52580.log 2024-04-20T21:31:42.027 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.29542.log.gz 2024-04-20T21:31:42.027 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47505.log 2024-04-20T21:31:42.029 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47773.log 2024-04-20T21:31:42.029 INFO:teuthology.orchestra.run.smithi086.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52580.log.gz 2024-04-20T21:31:42.031 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47505.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47505.log.gz 2024-04-20T21:31:42.031 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44834.log 2024-04-20T21:31:42.032 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47773.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47773.log.gz 2024-04-20T21:31:42.033 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44113.log 2024-04-20T21:31:42.034 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59163.log 2024-04-20T21:31:42.034 INFO:teuthology.orchestra.run.smithi086.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44834.log.gz 2024-04-20T21:31:42.036 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44113.log: gzip -5 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44113.log.gz 2024-04-20T21:31:42.036 INFO:teuthology.orchestra.run.smithi086.stderr: --verbose -- /var/log/ceph/ceph-client.admin.30381.log 2024-04-20T21:31:42.037 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42847.log 91.0% -- replaced with /var/log/ceph/ceph-client.admin.59163.log.gz 2024-04-20T21:31:42.037 INFO:teuthology.orchestra.run.smithi086.stderr: 2024-04-20T21:31:42.039 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.30381.log: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33399.log 2024-04-20T21:31:42.039 INFO:teuthology.orchestra.run.smithi086.stderr: -- replaced with /var/log/ceph/ceph-client.admin.30381.log.gz 2024-04-20T21:31:42.040 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42847.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42847.log.gz 2024-04-20T21:31:42.041 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33399.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33399.log.gz 2024-04-20T21:31:42.042 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51605.log 2024-04-20T21:31:42.043 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33729.log 2024-04-20T21:31:42.044 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41183.log 2024-04-20T21:31:42.045 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51605.log.gz 2024-04-20T21:31:42.046 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33729.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33729.log.gz 2024-04-20T21:31:42.046 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.60587.log 2024-04-20T21:31:42.052 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41183.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41183.log.gz 2024-04-20T21:31:42.053 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52156.log 2024-04-20T21:31:42.053 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.60587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31360.log 2024-04-20T21:31:42.053 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52156.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52156.log.gz 2024-04-20T21:31:42.053 INFO:teuthology.orchestra.run.smithi086.stderr:gzip/var/log/ceph/ceph-client.admin.31360.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.44666.log 2024-04-20T21:31:42.053 INFO:teuthology.orchestra.run.smithi086.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.31360.log.gz 2024-04-20T21:31:42.053 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41336.log 2024-04-20T21:31:42.054 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44666.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44666.log.gz 2024-04-20T21:31:42.055 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5/var/log/ceph/ceph-client.admin.41336.log: --verbose -- /var/log/ceph/ceph-client.admin.61479.log 2024-04-20T21:31:42.055 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41336.log.gz 2024-04-20T21:31:42.056 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62612.log 2024-04-20T21:31:42.058 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61479.log: 94.2% -- replaced with /var/log/ceph/ceph-client.0.60587.log.gz 2024-04-20T21:31:42.058 INFO:teuthology.orchestra.run.smithi086.stderr: 87.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52071.log 2024-04-20T21:31:42.058 INFO:teuthology.orchestra.run.smithi086.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61479.log.gz 2024-04-20T21:31:42.060 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62612.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.62612.log.gz 2024-04-20T21:31:42.060 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32238.log 2024-04-20T21:31:42.061 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52071.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52071.log.gz 2024-04-20T21:31:42.061 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51010.log 2024-04-20T21:31:42.062 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.32238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39520.log 2024-04-20T21:31:42.062 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32238.log.gz 2024-04-20T21:31:42.063 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51010.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51010.log.gz 2024-04-20T21:31:42.064 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29219.log 2024-04-20T21:31:42.065 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39520.log: gzip 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39520.log.gz 2024-04-20T21:31:42.065 INFO:teuthology.orchestra.run.smithi086.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.61717.log 2024-04-20T21:31:42.067 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.29219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61245.log 2024-04-20T21:31:42.067 INFO:teuthology.orchestra.run.smithi086.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.29219.log.gz 2024-04-20T21:31:42.068 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61717.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.61717.log.gz 2024-04-20T21:31:42.069 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5/var/log/ceph/ceph-client.admin.61245.log: --verbose -- /var/log/ceph/ceph-client.admin.57821.log 2024-04-20T21:31:42.069 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.61245.log.gz 2024-04-20T21:31:42.070 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55798.log 2024-04-20T21:31:42.072 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29552.log 2024-04-20T21:31:42.072 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57821.log.gz 2024-04-20T21:31:42.073 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55798.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55798.log.gz 2024-04-20T21:31:42.074 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62423.log 2024-04-20T21:31:42.074 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.29552.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.29552.log.gz 2024-04-20T21:31:42.075 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69674.log 2024-04-20T21:31:42.076 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69954.log 2024-04-20T21:31:42.076 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62423.log.gz 2024-04-20T21:31:42.078 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.69674.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.69674.log.gz 2024-04-20T21:31:42.079 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.69954.log: -- /var/log/ceph/ceph-client.admin.33231.log 2024-04-20T21:31:42.079 INFO:teuthology.orchestra.run.smithi086.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.69954.log.gz 2024-04-20T21:31:42.080 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69996.log 2024-04-20T21:31:42.081 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51178.log 2024-04-20T21:31:42.081 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33231.log.gz 2024-04-20T21:31:42.083 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.69996.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69996.log.gz 2024-04-20T21:31:42.083 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68831.log 2024-04-20T21:31:42.084 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51178.log.gz 2024-04-20T21:31:42.084 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34713.log 2024-04-20T21:31:42.086 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.68831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52200.log 2024-04-20T21:31:42.087 INFO:teuthology.orchestra.run.smithi086.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.68831.log.gz 2024-04-20T21:31:42.087 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34713.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34713.log.gz 2024-04-20T21:31:42.088 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34965.log 2024-04-20T21:31:42.089 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52200.log: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36460.log 2024-04-20T21:31:42.089 INFO:teuthology.orchestra.run.smithi086.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52200.log.gz 2024-04-20T21:31:42.090 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69912.log 2024-04-20T21:31:42.090 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34965.log.gz 2024-04-20T21:31:42.092 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36460.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36460.log.gz 2024-04-20T21:31:42.092 INFO:teuthology.orchestra.run.smithi086.stderr:gzip/var/log/ceph/ceph-client.admin.69912.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.39381.log 2024-04-20T21:31:42.093 INFO:teuthology.orchestra.run.smithi086.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.69912.log.gz 2024-04-20T21:31:42.094 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41901.log 2024-04-20T21:31:42.095 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60253.log 2024-04-20T21:31:42.095 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39381.log.gz 2024-04-20T21:31:42.097 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41901.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41901.log.gz 2024-04-20T21:31:42.097 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33813.log 2024-04-20T21:31:42.098 INFO:teuthology.orchestra.run.smithi086.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.60253.log.gz 2024-04-20T21:31:42.098 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65280.log 2024-04-20T21:31:42.100 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37478.log 2024-04-20T21:31:42.100 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33813.log.gz 2024-04-20T21:31:42.101 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.65280.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.65280.log.gz 2024-04-20T21:31:42.101 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51262.log 2024-04-20T21:31:42.102 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37478.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37478.log.gz 2024-04-20T21:31:42.103 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71264.log 2024-04-20T21:31:42.104 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58332.log 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51262.log.gz 2024-04-20T21:31:42.104 INFO:teuthology.orchestra.run.smithi086.stderr: 2024-04-20T21:31:42.106 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.71264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42666.log 2024-04-20T21:31:42.106 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.71264.log.gz 2024-04-20T21:31:42.107 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32616.log 2024-04-20T21:31:42.107 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58332.log.gz 2024-04-20T21:31:42.108 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54686.log 2024-04-20T21:31:42.109 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42666.log.gz 2024-04-20T21:31:42.109 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.32616.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.32616.log.gz 2024-04-20T21:31:42.110 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34503.log 2024-04-20T21:31:42.112 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54686.log: gzip -5 91.2% -- replaced with /var/log/ceph/ceph-client.admin.54686.log.gz 2024-04-20T21:31:42.112 INFO:teuthology.orchestra.run.smithi086.stderr: --verbose -- /var/log/ceph/ceph-client.admin.44521.log 2024-04-20T21:31:42.113 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34503.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34503.log.gz 2024-04-20T21:31:42.113 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38787.log 2024-04-20T21:31:42.115 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44521.log: gzip -5 --verbose/var/log/ceph/ceph-client.admin.38787.log: -- /var/log/ceph/ceph-client.admin.67901.log 2024-04-20T21:31:42.115 INFO:teuthology.orchestra.run.smithi086.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.44521.log.gz 2024-04-20T21:31:42.116 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38787.log.gz 2024-04-20T21:31:42.117 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59964.log 2024-04-20T21:31:42.118 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.67901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48136.log 2024-04-20T21:31:42.118 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.67901.log.gz 2024-04-20T21:31:42.119 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52496.log 2024-04-20T21:31:42.119 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59964.log.gz 2024-04-20T21:31:42.120 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48136.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48136.log.gz 2024-04-20T21:31:42.121 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60893.log 2024-04-20T21:31:42.122 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52496.log: gzip -5 --verbose -- 88.5% /var/log/ceph/ceph-client.admin.32868.log 2024-04-20T21:31:42.122 INFO:teuthology.orchestra.run.smithi086.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52496.log.gz 2024-04-20T21:31:42.123 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66912.log 2024-04-20T21:31:42.124 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.60893.log.gz 2024-04-20T21:31:42.125 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.32868.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.32868.log.gz 2024-04-20T21:31:42.125 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40737.log 2024-04-20T21:31:42.128 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.66912.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.66912.log.gz 2024-04-20T21:31:42.128 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53459.log 2024-04-20T21:31:42.129 INFO:teuthology.orchestra.run.smithi086.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.40737.log.gz 2024-04-20T21:31:42.130 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64105.log 2024-04-20T21:31:42.131 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35217.log 2024-04-20T21:31:42.131 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53459.log.gz 2024-04-20T21:31:42.133 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.64105.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.64105.log.gz 2024-04-20T21:31:42.133 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36320.log 2024-04-20T21:31:42.134 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35217.log.gz 2024-04-20T21:31:42.134 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67943.log 2024-04-20T21:31:42.136 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46517.log 2024-04-20T21:31:42.136 INFO:teuthology.orchestra.run.smithi086.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.36320.log.gz 2024-04-20T21:31:42.137 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.67943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40649.log 2024-04-20T21:31:42.138 INFO:teuthology.orchestra.run.smithi086.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.67943.log.gz 2024-04-20T21:31:42.139 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46517.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46517.log.gz 2024-04-20T21:31:42.139 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49057.log 2024-04-20T21:31:42.141 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48683.log 2024-04-20T21:31:42.141 INFO:teuthology.orchestra.run.smithi086.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.40649.log.gz 2024-04-20T21:31:42.142 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32532.log 2024-04-20T21:31:42.142 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49057.log.gz 2024-04-20T21:31:42.145 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.48683.log: /var/log/ceph/ceph-client.admin.32532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59737.log 2024-04-20T21:31:42.145 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32532.log.gz 2024-04-20T21:31:42.145 INFO:teuthology.orchestra.run.smithi086.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.0.48683.log.gz 2024-04-20T21:31:42.145 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32700.log 2024-04-20T21:31:42.147 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42759.log 2024-04-20T21:31:42.147 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59737.log.gz 2024-04-20T21:31:42.148 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.32700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28445.log 2024-04-20T21:31:42.148 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32700.log.gz 2024-04-20T21:31:42.149 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42759.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42759.log.gz 2024-04-20T21:31:42.150 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49795.log 2024-04-20T21:31:42.151 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.28445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39563.log 2024-04-20T21:31:42.151 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.28445.log.gz 2024-04-20T21:31:42.152 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29836.log 2024-04-20T21:31:42.153 INFO:teuthology.orchestra.run.smithi086.stderr: 91.1%/var/log/ceph/ceph-client.admin.39563.log: -- replaced with /var/log/ceph/ceph-client.admin.49795.log.gz 2024-04-20T21:31:42.154 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39563.log.gz 2024-04-20T21:31:42.154 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62184.log 2024-04-20T21:31:42.155 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.29836.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.29836.log.gz 2024-04-20T21:31:42.156 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67633.log 2024-04-20T21:31:42.157 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43141.log 2024-04-20T21:31:42.158 INFO:teuthology.orchestra.run.smithi086.stderr: 91.2%/var/log/ceph/ceph-client.admin.67633.log: -- replaced with /var/log/ceph/ceph-client.admin.62184.log.gz 2024-04-20T21:31:42.159 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 88.9% -- replaced with /var/log/ceph/ceph-client.admin.67633.log.gz 2024-04-20T21:31:42.159 INFO:teuthology.orchestra.run.smithi086.stderr: --verbose -- /var/log/ceph/ceph-client.admin.36011.log 2024-04-20T21:31:42.160 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51731.log 2024-04-20T21:31:42.160 INFO:teuthology.orchestra.run.smithi086.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.43141.log.gz 2024-04-20T21:31:42.161 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57527.log 2024-04-20T21:31:42.161 INFO:teuthology.orchestra.run.smithi086.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36011.log.gz 2024-04-20T21:31:42.162 INFO:teuthology.orchestra.run.smithi144.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-20T21:31:42.163 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51731.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51731.log.gz 2024-04-20T21:31:42.164 INFO:teuthology.orchestra.run.smithi086.stderr:gzip/var/log/ceph/ceph-client.admin.57527.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.53417.log 2024-04-20T21:31:42.164 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57527.log.gz 2024-04-20T21:31:42.165 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66488.log 2024-04-20T21:31:42.166 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37940.log 2024-04-20T21:31:42.166 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53417.log.gz 2024-04-20T21:31:42.168 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.66488.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.66488.log.gz 2024-04-20T21:31:42.168 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37903.log 2024-04-20T21:31:42.169 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37940.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.37940.log.gz 2024-04-20T21:31:42.169 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27989.log 2024-04-20T21:31:42.171 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53992.log 2024-04-20T21:31:42.171 INFO:teuthology.orchestra.run.smithi086.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.37903.log.gz 2024-04-20T21:31:42.172 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.27989.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.27989.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55863.log 2024-04-20T21:31:42.172 INFO:teuthology.orchestra.run.smithi086.stderr: 2024-04-20T21:31:42.173 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53992.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53992.log.gz 2024-04-20T21:31:42.174 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58413.log 2024-04-20T21:31:42.175 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66094.log 2024-04-20T21:31:42.176 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58413.log: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.55863.log.gz 2024-04-20T21:31:42.177 INFO:teuthology.orchestra.run.smithi086.stderr:gzip 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58413.log.gz 2024-04-20T21:31:42.177 INFO:teuthology.orchestra.run.smithi086.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.60935.log 2024-04-20T21:31:42.178 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.66094.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.66094.log.gz 2024-04-20T21:31:42.179 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64417.log 2024-04-20T21:31:42.180 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64333.log 2024-04-20T21:31:42.180 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60935.log.gz 2024-04-20T21:31:42.182 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.64417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38432.log 2024-04-20T21:31:42.182 INFO:teuthology.orchestra.run.smithi086.stderr: 90.8%/var/log/ceph/ceph-client.admin.64333.log: -- replaced with /var/log/ceph/ceph-client.admin.64417.log.gz 2024-04-20T21:31:42.182 INFO:teuthology.orchestra.run.smithi086.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.64333.log.gz 2024-04-20T21:31:42.183 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33005.log 2024-04-20T21:31:42.184 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28617.log 2024-04-20T21:31:42.184 INFO:teuthology.orchestra.run.smithi086.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.38432.log.gz 2024-04-20T21:31:42.186 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63369.log 2024-04-20T21:31:42.186 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33005.log.gz 2024-04-20T21:31:42.186 INFO:teuthology.orchestra.run.smithi144.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-20T21:31:42.187 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.28617.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.28617.log.gz 2024-04-20T21:31:42.187 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70844.log 2024-04-20T21:31:42.188 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.63369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45762.log 2024-04-20T21:31:42.189 INFO:teuthology.orchestra.run.smithi086.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.63369.log.gz 2024-04-20T21:31:42.190 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.70844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33173.log 2024-04-20T21:31:42.190 INFO:teuthology.orchestra.run.smithi086.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.70844.log.gz 2024-04-20T21:31:42.192 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45762.log: gzip 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45762.log.gz 2024-04-20T21:31:42.192 INFO:teuthology.orchestra.run.smithi086.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.45197.log 2024-04-20T21:31:42.192 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33173.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33173.log.gz 2024-04-20T21:31:42.193 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35301.log 2024-04-20T21:31:42.194 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45071.log 2024-04-20T21:31:42.194 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45197.log.gz 2024-04-20T21:31:42.196 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35301.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35301.log.gz 2024-04-20T21:31:42.197 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45071.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45071.log.gz 2024-04-20T21:31:42.197 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66954.log 2024-04-20T21:31:42.198 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50884.log 2024-04-20T21:31:42.200 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.66954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52328.log 2024-04-20T21:31:42.201 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50884.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.66954.log.gz 2024-04-20T21:31:42.202 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50884.log.gz 2024-04-20T21:31:42.202 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69870.log 2024-04-20T21:31:42.203 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52328.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52328.log.gz 2024-04-20T21:31:42.203 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65180.log 2024-04-20T21:31:42.205 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.69870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42328.log 2024-04-20T21:31:42.205 INFO:teuthology.orchestra.run.smithi086.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.69870.log.gz 2024-04-20T21:31:42.206 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.65180.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.65180.log.gz 2024-04-20T21:31:42.207 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5/var/log/ceph/ceph-client.admin.42328.log: --verbose -- /var/log/ceph/ceph-client.admin.46769.log 2024-04-20T21:31:42.207 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42328.log.gz 2024-04-20T21:31:42.208 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53781.log 2024-04-20T21:31:42.210 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68897.log 2024-04-20T21:31:42.210 INFO:teuthology.orchestra.run.smithi086.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46769.log.gz 2024-04-20T21:31:42.211 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53781.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53781.log.gz 2024-04-20T21:31:42.211 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41420.log 2024-04-20T21:31:42.213 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.68897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68106.log 2024-04-20T21:31:42.213 INFO:teuthology.orchestra.run.smithi086.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.68897.log.gz 2024-04-20T21:31:42.214 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54986.log 2024-04-20T21:31:42.214 INFO:teuthology.orchestra.run.smithi086.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41420.log.gz 2024-04-20T21:31:42.216 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.68106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70266.log 2024-04-20T21:31:42.216 INFO:teuthology.orchestra.run.smithi086.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.68106.log.gz 2024-04-20T21:31:42.217 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54986.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.54986.log.gz 2024-04-20T21:31:42.218 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35715.log/var/log/ceph/ceph-client.admin.70266.log: 2024-04-20T21:31:42.218 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.70266.log.gz 2024-04-20T21:31:42.219 INFO:teuthology.orchestra.run.smithi144.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-20T21:31:42.219 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43015.log 2024-04-20T21:31:42.220 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43494.log 2024-04-20T21:31:42.220 INFO:teuthology.orchestra.run.smithi086.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.35715.log.gz 2024-04-20T21:31:42.223 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43015.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43015.log.gz 2024-04-20T21:31:42.223 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.43494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65138.log 2024-04-20T21:31:42.224 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57989.log 2024-04-20T21:31:42.225 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.65138.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.65138.log.gz 2024-04-20T21:31:42.226 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45407.log 93.7% -- replaced with /var/log/ceph/ceph-client.0.43494.log.gz 2024-04-20T21:31:42.226 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57989.log: 2024-04-20T21:31:42.227 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57989.log.gz 2024-04-20T21:31:42.228 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35673.log 2024-04-20T21:31:42.229 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57122.log 2024-04-20T21:31:42.229 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45407.log.gz 2024-04-20T21:31:42.231 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63924.log 89.6% -- replaced with /var/log/ceph/ceph-client.admin.35673.log.gz 2024-04-20T21:31:42.231 INFO:teuthology.orchestra.run.smithi086.stderr: 2024-04-20T21:31:42.231 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57122.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57122.log.gz 2024-04-20T21:31:42.232 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49015.log 2024-04-20T21:31:42.234 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.63924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28276.log 2024-04-20T21:31:42.234 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.63924.log.gz 2024-04-20T21:31:42.235 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49015.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49015.log.gz 2024-04-20T21:31:42.235 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40650.log 2024-04-20T21:31:42.236 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.28276.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.28276.log.gz 2024-04-20T21:31:42.237 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45636.log 2024-04-20T21:31:42.238 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62185.log 2024-04-20T21:31:42.238 INFO:teuthology.orchestra.run.smithi086.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.40650.log.gz 2024-04-20T21:31:42.240 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53883.log 2024-04-20T21:31:42.240 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45636.log.gz 2024-04-20T21:31:42.241 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70435.log 2024-04-20T21:31:42.241 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.62185.log.gz 2024-04-20T21:31:42.242 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62058.log 2024-04-20T21:31:42.243 INFO:teuthology.orchestra.run.smithi086.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.53883.log.gz/var/log/ceph/ceph-client.admin.70435.log: 2024-04-20T21:31:42.243 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.70435.log.gz 2024-04-20T21:31:42.244 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62058.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.62058.log.gz 2024-04-20T21:31:42.245 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.31772.log 2024-04-20T21:31:42.246 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46019.log 2024-04-20T21:31:42.248 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.2.31772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39339.log 2024-04-20T21:31:42.249 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46019.log: 91.8% -- replaced with /var/log/ceph/ceph-client.2.31772.log.gz 2024-04-20T21:31:42.249 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46019.log.gz 2024-04-20T21:31:42.249 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66446.log 2024-04-20T21:31:42.256 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48232.log 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39339.log.gz 2024-04-20T21:31:42.256 INFO:teuthology.orchestra.run.smithi086.stderr: 2024-04-20T21:31:42.256 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.66446.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.66446.log.gz 2024-04-20T21:31:42.256 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68339.log 2024-04-20T21:31:42.256 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35343.log 2024-04-20T21:31:42.256 INFO:teuthology.orchestra.run.smithi086.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48232.log.gz 2024-04-20T21:31:42.256 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.68339.log: gzip -5 --verbose -- 87.7% -- replaced with /var/log/ceph/ceph-client.admin.68339.log.gz 2024-04-20T21:31:42.256 INFO:teuthology.orchestra.run.smithi086.stderr: /var/log/ceph/ceph-client.admin.57905.log 2024-04-20T21:31:42.257 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35175.log 2024-04-20T21:31:42.257 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35343.log.gz 2024-04-20T21:31:42.258 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57206.log 2024-04-20T21:31:42.258 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57905.log.gz 2024-04-20T21:31:42.259 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35175.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35175.log.gz 2024-04-20T21:31:42.260 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5/var/log/ceph/ceph-client.admin.57206.log: --verbose -- /var/log/ceph/ceph-client.admin.51906.log 2024-04-20T21:31:42.260 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57206.log.gz 2024-04-20T21:31:42.261 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47673.log 2024-04-20T21:31:42.263 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37866.log 2024-04-20T21:31:42.263 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51906.log.gz 2024-04-20T21:31:42.264 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47673.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47673.log.gz 2024-04-20T21:31:42.265 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48190.log 2024-04-20T21:31:42.265 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37866.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.37866.log.gz 2024-04-20T21:31:42.266 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66404.log 2024-04-20T21:31:42.267 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39945.log 2024-04-20T21:31:42.267 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48190.log.gz 2024-04-20T21:31:42.268 INFO:teuthology.orchestra.run.smithi144.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-20T21:31:42.269 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.66404.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.66404.log.gz 2024-04-20T21:31:42.269 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5/var/log/ceph/ceph-client.admin.39945.log: --verbose -- /var/log/ceph/ceph-client.admin.71306.log 2024-04-20T21:31:42.271 INFO:teuthology.orchestra.run.smithi086.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.39945.log.gz 2024-04-20T21:31:42.271 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52772.log 2024-04-20T21:31:42.271 INFO:teuthology.orchestra.run.smithi144.stderr: 2024-04-20T21:31:42.272 INFO:teuthology.orchestra.run.smithi144.stderr:real 0m0.790s 2024-04-20T21:31:42.272 INFO:teuthology.orchestra.run.smithi144.stderr:user 0m3.377s 2024-04-20T21:31:42.272 INFO:teuthology.orchestra.run.smithi144.stderr:sys 0m0.382s 2024-04-20T21:31:42.273 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.71306.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.71306.log.gz 2024-04-20T21:31:42.273 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28531.log 2024-04-20T21:31:42.274 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66866.log 2024-04-20T21:31:42.274 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52772.log.gz 2024-04-20T21:31:42.275 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.28531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28318.log 2024-04-20T21:31:42.275 INFO:teuthology.orchestra.run.smithi086.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.28531.log.gz 2024-04-20T21:31:42.276 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.66866.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66866.log.gz 2024-04-20T21:31:42.277 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.28318.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.28318.log.gz 2024-04-20T21:31:42.278 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57737.log 2024-04-20T21:31:42.279 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42622.log 2024-04-20T21:31:42.281 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36190.log 2024-04-20T21:31:42.281 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57737.log.gz 2024-04-20T21:31:42.282 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42622.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42622.log.gz 2024-04-20T21:31:42.282 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.31256.log 2024-04-20T21:31:42.283 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43745.log 2024-04-20T21:31:42.284 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36190.log.gz 2024-04-20T21:31:42.285 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.31256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61287.log 2024-04-20T21:31:42.286 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43745.log: 91.4% -- replaced with /var/log/ceph/ceph-client.0.31256.log.gz 2024-04-20T21:31:42.286 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61203.log 2024-04-20T21:31:42.287 INFO:teuthology.orchestra.run.smithi086.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43745.log.gz 2024-04-20T21:31:42.288 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34341.log 2024-04-20T21:31:42.288 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.61287.log.gz 2024-04-20T21:31:42.289 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71348.log 2024-04-20T21:31:42.289 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.61203.log.gz 2024-04-20T21:31:42.291 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34341.log: gzip 89.3% -5 --verbose -- /var/log/ceph/ceph-client.admin.34073.log 2024-04-20T21:31:42.291 INFO:teuthology.orchestra.run.smithi086.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34341.log.gz 2024-04-20T21:31:42.291 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.71348.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.71348.log.gz 2024-04-20T21:31:42.292 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43099.log 2024-04-20T21:31:42.293 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34073.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34073.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44945.log 2024-04-20T21:31:42.293 INFO:teuthology.orchestra.run.smithi086.stderr: 2024-04-20T21:31:42.294 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43099.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43099.log.gz 2024-04-20T21:31:42.295 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58805.log 2024-04-20T21:31:42.296 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70038.log 2024-04-20T21:31:42.296 INFO:teuthology.orchestra.run.smithi086.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44945.log.gz 2024-04-20T21:31:42.297 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41816.log 2024-04-20T21:31:42.297 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58805.log.gz 2024-04-20T21:31:42.298 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.70038.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.70038.log.gz 2024-04-20T21:31:42.299 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43897.log 2024-04-20T21:31:42.299 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41816.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41816.log.gz 2024-04-20T21:31:42.300 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59079.log 2024-04-20T21:31:42.301 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61844.log 2024-04-20T21:31:42.302 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59079.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.43897.log.gz 2024-04-20T21:31:42.303 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59079.log.gz 2024-04-20T21:31:42.303 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36506.log 2024-04-20T21:31:42.305 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61844.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.61844.log.gz 2024-04-20T21:31:42.305 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56743.log 2024-04-20T21:31:42.306 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36506.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36506.log.gz 2024-04-20T21:31:42.306 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57779.log 2024-04-20T21:31:42.307 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38618.log 2024-04-20T21:31:42.308 INFO:teuthology.orchestra.run.smithi086.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.56743.log.gz/var/log/ceph/ceph-client.admin.57779.log: 2024-04-20T21:31:42.309 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57779.log.gz 2024-04-20T21:31:42.309 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48889.log 2024-04-20T21:31:42.310 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38618.log: gzip -5 89.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.38618.log.gz -- 2024-04-20T21:31:42.310 INFO:teuthology.orchestra.run.smithi086.stderr: /var/log/ceph/ceph-client.admin.55368.log 2024-04-20T21:31:42.312 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33971.log 2024-04-20T21:31:42.312 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48889.log.gz 2024-04-20T21:31:42.313 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55368.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55368.log.gz 2024-04-20T21:31:42.314 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35799.log 2024-04-20T21:31:42.314 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33971.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33971.log.gz 2024-04-20T21:31:42.315 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41756.log 2024-04-20T21:31:42.315 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-04-20T21:31:42.315 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-04-20T21:31:42.316 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-04-20T21:31:42.316 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-04-20T21:31:42.316 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2024-04-20T21:31:42.316 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-04-20T21:31:42.316 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-04-20T21:31:42.316 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-04-20T21:31:42.316 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34545.log 2024-04-20T21:31:42.316 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35799.log.gz 2024-04-20T21:31:42.317 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47815.log 2024-04-20T21:31:42.317 INFO:teuthology.orchestra.run.smithi086.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41756.log.gz 2024-04-20T21:31:42.319 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34545.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34545.log.gz 2024-04-20T21:31:42.319 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33645.log 2024-04-20T21:31:42.320 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47815.log: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34587.log 2024-04-20T21:31:42.320 INFO:teuthology.orchestra.run.smithi086.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47815.log.gz 2024-04-20T21:31:42.321 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65322.log 2024-04-20T21:31:42.322 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33645.log.gz 2024-04-20T21:31:42.323 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34587.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34587.log.gz 2024-04-20T21:31:42.323 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62781.log 2024-04-20T21:31:42.325 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.65322.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65322.log.gz 2024-04-20T21:31:42.325 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52538.log 2024-04-20T21:31:42.326 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62781.log: gzip -5 --verbose 88.8% -- /var/log/ceph/ceph-client.admin.57443.log -- replaced with /var/log/ceph/ceph-client.admin.62781.log.gz 2024-04-20T21:31:42.326 INFO:teuthology.orchestra.run.smithi086.stderr: 2024-04-20T21:31:42.328 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28827.log 2024-04-20T21:31:42.328 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52538.log.gz 2024-04-20T21:31:42.329 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57485.log 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57443.log.gz 2024-04-20T21:31:42.329 INFO:teuthology.orchestra.run.smithi086.stderr: 2024-04-20T21:31:42.331 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.28827.log: gzip 89.1% -- replaced with /var/log/ceph/ceph-client.admin.28827.log.gz 2024-04-20T21:31:42.331 INFO:teuthology.orchestra.run.smithi086.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.50773.log 2024-04-20T21:31:42.332 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57485.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.57485.log.gz 2024-04-20T21:31:42.332 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44587.log 2024-04-20T21:31:42.333 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29853.log 2024-04-20T21:31:42.334 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50773.log.gz 2024-04-20T21:31:42.334 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44587.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44587.log.gz 2024-04-20T21:31:42.335 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30255.log 2024-04-20T21:31:42.336 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.29853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47004.log 2024-04-20T21:31:42.336 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.29853.log.gz 2024-04-20T21:31:42.337 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.30255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59229.log 2024-04-20T21:31:42.338 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.30255.log.gz 2024-04-20T21:31:42.338 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47004.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47004.log.gz 2024-04-20T21:31:42.339 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32280.log 2024-04-20T21:31:42.340 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70206.log 2024-04-20T21:31:42.340 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59229.log.gz 2024-04-20T21:31:42.342 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.32280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68512.log 2024-04-20T21:31:42.342 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.32280.log.gz 2024-04-20T21:31:42.343 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.70206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28785.log 2024-04-20T21:31:42.343 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.70206.log.gz 2024-04-20T21:31:42.344 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.68512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40289.log 2024-04-20T21:31:42.346 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.28785.log: 92.1% 89.1% -- replaced with /var/log/ceph/ceph-client.admin.28785.log.gz 2024-04-20T21:31:42.346 INFO:teuthology.orchestra.run.smithi086.stderr: -- replaced with /var/log/ceph/ceph-client.admin.68512.log.gz 2024-04-20T21:31:42.348 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.40289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63794.log 2024-04-20T21:31:42.350 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52286.log 2024-04-20T21:31:42.350 INFO:teuthology.orchestra.run.smithi086.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.0.40289.log.gz 2024-04-20T21:31:42.351 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.63794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61651.log 88.8% -- replaced with /var/log/ceph/ceph-client.admin.63794.log.gz 2024-04-20T21:31:42.351 INFO:teuthology.orchestra.run.smithi086.stderr: 2024-04-20T21:31:42.352 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52286.log: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49579.log 2024-04-20T21:31:42.353 INFO:teuthology.orchestra.run.smithi086.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52286.log.gz 2024-04-20T21:31:42.354 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68180.log 2024-04-20T21:31:42.354 INFO:teuthology.orchestra.run.smithi086.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.61651.log.gz 2024-04-20T21:31:42.355 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39469.log 2024-04-20T21:31:42.355 INFO:teuthology.orchestra.run.smithi086.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49579.log.gz 2024-04-20T21:31:42.356 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.68180.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.68180.log.gz 2024-04-20T21:31:42.357 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32112.log 2024-04-20T21:31:42.358 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39469.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39469.log.gz 2024-04-20T21:31:42.358 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36548.log 2024-04-20T21:31:42.358 INFO:teuthology.orchestra.run.smithi086.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-20T21:31:42.359 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.32112.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.32112.log.gz 2024-04-20T21:31:42.359 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61119.log 2024-04-20T21:31:42.360 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29884.log 2024-04-20T21:31:42.361 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61119.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.36548.log.gz 2024-04-20T21:31:42.362 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54944.log 2024-04-20T21:31:42.362 INFO:teuthology.orchestra.run.smithi086.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.61119.log.gz 2024-04-20T21:31:42.363 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.29884.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.29884.log.gz 2024-04-20T21:31:42.363 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58721.log 2024-04-20T21:31:42.364 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58869.log 2024-04-20T21:31:42.364 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54944.log: 87.8%/var/log/ceph/ceph-client.admin.58721.log: -- replaced with /var/log/ceph/ceph-client.admin.54944.log.gz 2024-04-20T21:31:42.365 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58721.log.gz 2024-04-20T21:31:42.365 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35967.log 2024-04-20T21:31:42.366 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51826.log 2024-04-20T21:31:42.366 INFO:teuthology.orchestra.run.smithi086.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58869.log.gz 2024-04-20T21:31:42.367 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35967.log: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46187.log 2024-04-20T21:31:42.367 INFO:teuthology.orchestra.run.smithi086.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35967.log.gz 2024-04-20T21:31:42.368 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52029.log 2024-04-20T21:31:42.368 INFO:teuthology.orchestra.run.smithi086.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51826.log.gz 2024-04-20T21:31:42.369 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46187.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46187.log.gz 2024-04-20T21:31:42.369 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38162.log 2024-04-20T21:31:42.370 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.52029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65448.log 2024-04-20T21:31:42.371 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52029.log.gz 2024-04-20T21:31:42.372 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38162.log: gzip 89.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.38162.log.gz --verbose 2024-04-20T21:31:42.372 INFO:teuthology.orchestra.run.smithi086.stderr: -- /var/log/ceph/ceph-client.admin.28911.log 2024-04-20T21:31:42.372 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.65448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30819.log 2024-04-20T21:31:42.373 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65448.log.gz 2024-04-20T21:31:42.374 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.28911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46369.log 2024-04-20T21:31:42.374 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.28911.log.gz 2024-04-20T21:31:42.375 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.30819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42153.log 2024-04-20T21:31:42.375 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.30819.log.gz 2024-04-20T21:31:42.376 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57611.log 2024-04-20T21:31:42.376 INFO:teuthology.orchestra.run.smithi086.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46369.log.gz 2024-04-20T21:31:42.377 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61970.log 2024-04-20T21:31:42.378 INFO:teuthology.orchestra.run.smithi086.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42153.log.gz 2024-04-20T21:31:42.378 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.57611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43841.log 2024-04-20T21:31:42.379 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57611.log.gz 2024-04-20T21:31:42.379 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55678.log 2024-04-20T21:31:42.379 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.61970.log.gz 2024-04-20T21:31:42.381 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33357.log 2024-04-20T21:31:42.381 INFO:teuthology.orchestra.run.smithi086.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.43841.log.gz 2024-04-20T21:31:42.382 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39212.log 2024-04-20T21:31:42.382 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55678.log.gz 2024-04-20T21:31:42.383 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70122.log 2024-04-20T21:31:42.383 INFO:teuthology.orchestra.run.smithi086.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33357.log.gz 2024-04-20T21:31:42.384 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54120.log 2024-04-20T21:31:42.384 INFO:teuthology.orchestra.run.smithi086.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39212.log.gz 2024-04-20T21:31:42.385 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.70122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62012.log 2024-04-20T21:31:42.385 INFO:teuthology.orchestra.run.smithi086.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.70122.log.gz 2024-04-20T21:31:42.386 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32406.log 2024-04-20T21:31:42.387 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62012.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.54120.log.gz 2024-04-20T21:31:42.387 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58582.log 2024-04-20T21:31:42.388 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.62012.log.gz 2024-04-20T21:31:42.389 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.32406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37562.log 2024-04-20T21:31:42.389 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.32406.log.gz 2024-04-20T21:31:42.390 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58582.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58582.log.gz 2024-04-20T21:31:42.390 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38120.log 2024-04-20T21:31:42.391 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45857.log 2024-04-20T21:31:42.391 INFO:teuthology.orchestra.run.smithi086.stderr: 89.1%/var/log/ceph/ceph-client.admin.38120.log: -- replaced with /var/log/ceph/ceph-client.admin.37562.log.gz 2024-04-20T21:31:42.392 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38120.log.gz 2024-04-20T21:31:42.392 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62446.log 2024-04-20T21:31:42.393 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70164.log 2024-04-20T21:31:42.393 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45857.log.gz 2024-04-20T21:31:42.394 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35007.log 2024-04-20T21:31:42.394 INFO:teuthology.orchestra.run.smithi086.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.62446.log.gz 2024-04-20T21:31:42.395 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.70164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39861.log 2024-04-20T21:31:42.395 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.70164.log.gz 2024-04-20T21:31:42.397 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35007.log: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50647.log 2024-04-20T21:31:42.397 INFO:teuthology.orchestra.run.smithi086.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35007.log.gz 2024-04-20T21:31:42.397 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49157.log 2024-04-20T21:31:42.398 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39861.log.gz 2024-04-20T21:31:42.399 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50647.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50647.log.gz 2024-04-20T21:31:42.399 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68143.log 2024-04-20T21:31:42.402 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5/var/log/ceph/ceph-client.admin.49157.log: --verbose -- /var/log/ceph/ceph-client.admin.55410.log 2024-04-20T21:31:42.402 INFO:teuthology.orchestra.run.smithi086.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49157.log.gz 2024-04-20T21:31:42.402 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.68143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29858.log 2024-04-20T21:31:42.402 INFO:teuthology.orchestra.run.smithi086.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.68143.log.gz 2024-04-20T21:31:42.402 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55410.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-20T21:31:42.403 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55410.log.gz 2024-04-20T21:31:42.403 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.29858.log: gzip 88.5% -- replaced with /var/log/ceph/ceph-client.admin.29858.log.gz -5 2024-04-20T21:31:42.403 INFO:teuthology.orchestra.run.smithi086.stderr: --verbose -- /var/log/ceph/ceph-client.admin.59037.log 2024-04-20T21:31:42.404 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37829.log 2024-04-20T21:31:42.405 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59037.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59037.log.gz 2024-04-20T21:31:42.406 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42801.log 2024-04-20T21:31:42.407 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37829.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.37829.log.gz 2024-04-20T21:31:42.407 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58670.log 2024-04-20T21:31:42.409 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42801.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42801.log.gz 2024-04-20T21:31:42.409 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.59507.log 2024-04-20T21:31:42.410 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58670.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58670.log.gz 2024-04-20T21:31:42.410 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58626.log 2024-04-20T21:31:42.411 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.59507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45365.log 2024-04-20T21:31:42.413 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58626.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.58626.log.gz 2024-04-20T21:31:42.413 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.64761.log 2024-04-20T21:31:42.414 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45365.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45365.log.gz 2024-04-20T21:31:42.415 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55494.log 2024-04-20T21:31:42.416 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.64761.log: 93.9% -- replaced with /var/log/ceph/ceph-client.0.59507.log.gz 2024-04-20T21:31:42.418 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55494.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.55494.log.gz 2024-04-20T21:31:42.418 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39297.log 2024-04-20T21:31:42.420 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35631.log 2024-04-20T21:31:42.421 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39297.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39297.log.gz 2024-04-20T21:31:42.422 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35631.log: 92.7% -- replaced with /var/log/ceph/ceph-client.0.64761.log.gz 2024-04-20T21:31:42.422 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29218.log 2024-04-20T21:31:42.422 INFO:teuthology.orchestra.run.smithi086.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35631.log.gz 2024-04-20T21:31:42.423 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37378.log 2024-04-20T21:31:42.425 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.29218.log: gzip 87.5% -- replaced with /var/log/ceph/ceph-client.admin.29218.log.gz 2024-04-20T21:31:42.425 INFO:teuthology.orchestra.run.smithi086.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.53517.log 2024-04-20T21:31:42.426 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40575.log 2024-04-20T21:31:42.426 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37378.log.gz 2024-04-20T21:31:42.427 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53517.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53517.log.gz 2024-04-20T21:31:42.428 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41546.log 2024-04-20T21:31:42.428 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40575.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40575.log.gz 2024-04-20T21:31:42.429 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41378.log 2024-04-20T21:31:42.430 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69548.log 2024-04-20T21:31:42.430 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41546.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41546.log.gz 2024-04-20T21:31:42.431 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41378.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41378.log.gz 2024-04-20T21:31:42.431 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39605.log 2024-04-20T21:31:42.432 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.69548.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.69548.log.gz 2024-04-20T21:31:42.433 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33561.log 2024-04-20T21:31:42.434 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32784.log 2024-04-20T21:31:42.434 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39605.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39605.log.gz 2024-04-20T21:31:42.435 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33561.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33561.log.gz 2024-04-20T21:31:42.436 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35546.log 2024-04-20T21:31:42.436 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.32784.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32784.log.gz 2024-04-20T21:31:42.437 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63586.log 2024-04-20T21:31:42.438 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35546.log: gzip 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35546.log.gz 2024-04-20T21:31:42.438 INFO:teuthology.orchestra.run.smithi086.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.42931.log 2024-04-20T21:31:42.439 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.63586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62100.log 2024-04-20T21:31:42.439 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.63586.log.gz 2024-04-20T21:31:42.442 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42931.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42931.log.gz 2024-04-20T21:31:42.442 INFO:teuthology.orchestra.run.smithi086.stderr:gzip/var/log/ceph/ceph-client.admin.62100.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.60090.log 2024-04-20T21:31:42.443 INFO:teuthology.orchestra.run.smithi086.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.62100.log.gz 2024-04-20T21:31:42.443 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67817.log 2024-04-20T21:31:42.444 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37252.log 2024-04-20T21:31:42.444 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60090.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60090.log.gz 2024-04-20T21:31:42.445 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.67817.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.67817.log.gz 2024-04-20T21:31:42.446 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58995.log 2024-04-20T21:31:42.446 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37252.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37252.log.gz 2024-04-20T21:31:42.447 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29885.log 2024-04-20T21:31:42.448 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58995.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58995.log.gz 2024-04-20T21:31:42.448 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39254.log 2024-04-20T21:31:42.449 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.29885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35091.log 2024-04-20T21:31:42.450 INFO:teuthology.orchestra.run.smithi086.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.29885.log.gz 2024-04-20T21:31:42.450 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39254.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39254.log.gz 2024-04-20T21:31:42.451 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63506.log 2024-04-20T21:31:42.452 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35091.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35091.log.gz 2024-04-20T21:31:42.452 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41462.log 2024-04-20T21:31:42.453 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58031.log 2024-04-20T21:31:42.454 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.63506.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.63506.log.gz 2024-04-20T21:31:42.454 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68217.log 2024-04-20T21:31:42.455 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41462.log.gz 2024-04-20T21:31:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28743.log 2024-04-20T21:31:42.456 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58031.log.gz 2024-04-20T21:31:42.457 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.68217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32910.log 2024-04-20T21:31:42.457 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.68217.log.gz 2024-04-20T21:31:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.28743.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.28743.log.gz 2024-04-20T21:31:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32028.log 2024-04-20T21:31:42.459 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.32910.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.32910.log.gz 2024-04-20T21:31:42.460 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47282.log 2024-04-20T21:31:42.461 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55947.log 2024-04-20T21:31:42.461 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.32028.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32028.log.gz 2024-04-20T21:31:42.465 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.0.47282.log: /var/log/ceph/ceph-client.admin.55947.log: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.55947.log.gz 2024-04-20T21:31:42.465 INFO:teuthology.orchestra.run.smithi086.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.0.47282.log.gz 2024-04-20T21:31:42.465 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28659.log 2024-04-20T21:31:42.466 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71545.log 2024-04-20T21:31:42.467 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.28659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34173.log 2024-04-20T21:31:42.467 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.28659.log.gz 2024-04-20T21:31:42.469 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.71545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70718.log 2024-04-20T21:31:42.469 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.71545.log.gz 2024-04-20T21:31:42.470 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34173.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34173.log.gz 2024-04-20T21:31:42.470 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43788.log 2024-04-20T21:31:42.471 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.70718.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.70718.log.gz 2024-04-20T21:31:42.471 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64291.log 2024-04-20T21:31:42.472 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29550.log 2024-04-20T21:31:42.473 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.43788.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43788.log.gz 2024-04-20T21:31:42.473 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.64291.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.64291.log.gz 2024-04-20T21:31:42.474 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38204.log 2024-04-20T21:31:42.475 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.29550.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.29550.log.gz 2024-04-20T21:31:42.475 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30213.log 2024-04-20T21:31:42.476 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66178.log 2024-04-20T21:31:42.477 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38204.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38204.log.gz 2024-04-20T21:31:42.477 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.30213.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.30213.log.gz 2024-04-20T21:31:42.478 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33603.log 2024-04-20T21:31:42.479 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.66178.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.66178.log.gz 2024-04-20T21:31:42.479 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59821.log 2024-04-20T21:31:42.480 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32574.log 2024-04-20T21:31:42.481 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33603.log.gz 2024-04-20T21:31:42.482 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59821.log: gzip -5 --verbose -- 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59821.log.gz 2024-04-20T21:31:42.482 INFO:teuthology.orchestra.run.smithi086.stderr: /var/log/ceph/ceph-client.admin.50512.log 2024-04-20T21:31:42.483 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.32574.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.32574.log.gz 2024-04-20T21:31:42.483 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39735.log 2024-04-20T21:31:42.485 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44221.log 2024-04-20T21:31:42.485 INFO:teuthology.orchestra.run.smithi086.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50512.log.gz 2024-04-20T21:31:42.486 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.39735.log: gzip 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39735.log.gz 2024-04-20T21:31:42.486 INFO:teuthology.orchestra.run.smithi086.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.56050.log 2024-04-20T21:31:42.487 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.44221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45239.log 2024-04-20T21:31:42.487 INFO:teuthology.orchestra.run.smithi086.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44221.log.gz 2024-04-20T21:31:42.489 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.56050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35841.log 2024-04-20T21:31:42.489 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56050.log.gz 2024-04-20T21:31:42.489 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45239.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45239.log.gz 2024-04-20T21:31:42.490 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40479.log 2024-04-20T21:31:42.491 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.35841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30717.log 2024-04-20T21:31:42.491 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35841.log.gz 2024-04-20T21:31:42.492 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.40479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49687.log 2024-04-20T21:31:42.492 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40479.log.gz 2024-04-20T21:31:42.494 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.30717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65593.log 2024-04-20T21:31:42.494 INFO:teuthology.orchestra.run.smithi086.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.30717.log.gz 2024-04-20T21:31:42.495 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28701.log 2024-04-20T21:31:42.495 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49687.log.gz 2024-04-20T21:31:42.496 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.65593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34013.log 2024-04-20T21:31:42.496 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.65593.log.gz 2024-04-20T21:31:42.498 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.28701.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.28701.log.gz 2024-04-20T21:31:42.498 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48062.log 2024-04-20T21:31:42.499 INFO:teuthology.orchestra.run.smithi086.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34013.log.gz 2024-04-20T21:31:42.499 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46896.log 2024-04-20T21:31:42.500 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63117.log 2024-04-20T21:31:42.501 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.48062.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.48062.log.gz 2024-04-20T21:31:42.502 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.46896.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46896.log.gz 2024-04-20T21:31:42.502 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32490.log 2024-04-20T21:31:42.503 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.63117.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.63117.log.gz 2024-04-20T21:31:42.503 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63709.log 2024-04-20T21:31:42.504 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53823.log 2024-04-20T21:31:42.505 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.32490.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.32490.log.gz 2024-04-20T21:31:42.505 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.63709.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.63709.log.gz 2024-04-20T21:31:42.506 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54186.log 2024-04-20T21:31:42.507 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.53823.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53823.log.gz 2024-04-20T21:31:42.507 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28361.log 2024-04-20T21:31:42.508 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51647.log 2024-04-20T21:31:42.509 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.54186.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54186.log.gz 2024-04-20T21:31:42.509 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.28361.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.28361.log.gz 2024-04-20T21:31:42.510 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50055.log 2024-04-20T21:31:42.511 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.51647.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51647.log.gz 2024-04-20T21:31:42.511 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70676.log 2024-04-20T21:31:42.512 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.50055.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50055.log.gz 2024-04-20T21:31:42.512 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42069.log 2024-04-20T21:31:42.513 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.70676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62865.log 2024-04-20T21:31:42.514 INFO:teuthology.orchestra.run.smithi086.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.70676.log.gz 2024-04-20T21:31:42.515 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.42069.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42069.log.gz 2024-04-20T21:31:42.515 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55202.log 2024-04-20T21:31:42.516 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62865.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.62865.log.gz 2024-04-20T21:31:42.516 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60168.log 2024-04-20T21:31:42.517 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.55202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67859.log 2024-04-20T21:31:42.518 INFO:teuthology.orchestra.run.smithi086.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55202.log.gz 2024-04-20T21:31:42.519 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60168.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.60168.log.gz 2024-04-20T21:31:42.519 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62655.log 2024-04-20T21:31:42.520 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.67859.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.67859.log.gz 2024-04-20T21:31:42.521 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34755.log 2024-04-20T21:31:42.522 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59779.log 2024-04-20T21:31:42.522 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.62655.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.62655.log.gz 2024-04-20T21:31:42.523 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.34755.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34755.log.gz 2024-04-20T21:31:42.524 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70802.log 2024-04-20T21:31:42.524 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.59779.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.59779.log.gz 2024-04-20T21:31:42.525 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37520.log 2024-04-20T21:31:42.526 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46103.log 2024-04-20T21:31:42.526 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.70802.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.70802.log.gz 2024-04-20T21:31:42.527 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.37520.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37520.log.gz 2024-04-20T21:31:42.528 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5/var/log/ceph/ceph-client.admin.46103.log: --verbose -- /var/log/ceph/ceph-client.admin.70886.log 2024-04-20T21:31:42.528 INFO:teuthology.orchestra.run.smithi086.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46103.log.gz 2024-04-20T21:31:42.529 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61365.log 2024-04-20T21:31:42.530 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.70886.log: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36278.log 2024-04-20T21:31:42.531 INFO:teuthology.orchestra.run.smithi086.stderr: -- replaced with /var/log/ceph/ceph-client.admin.70886.log.gz 2024-04-20T21:31:42.532 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.61365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66770.log 2024-04-20T21:31:42.532 INFO:teuthology.orchestra.run.smithi086.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.61365.log.gz 2024-04-20T21:31:42.533 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36278.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36278.log.gz 2024-04-20T21:31:42.533 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70309.log 2024-04-20T21:31:42.534 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.66770.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.66770.log.gz 2024-04-20T21:31:42.535 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41504.log 2024-04-20T21:31:42.536 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47589.log 2024-04-20T21:31:42.536 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.70309.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.70309.log.gz 2024-04-20T21:31:42.537 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.41504.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41504.log.gz 2024-04-20T21:31:42.538 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49427.log 2024-04-20T21:31:42.538 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.47589.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47589.log.gz 2024-04-20T21:31:42.538 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38703.log 2024-04-20T21:31:42.540 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60309.log 2024-04-20T21:31:42.541 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.49427.log: /var/log/ceph/ceph-client.admin.38703.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.49427.log.gz 2024-04-20T21:31:42.541 INFO:teuthology.orchestra.run.smithi086.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38703.log.gz 2024-04-20T21:31:42.541 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28869.log 2024-04-20T21:31:42.542 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.60309.log: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58157.log 2024-04-20T21:31:42.542 INFO:teuthology.orchestra.run.smithi086.stderr: -- replaced with /var/log/ceph/ceph-client.admin.60309.log.gz 2024-04-20T21:31:42.544 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.28869.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.28869.log.gz 2024-04-20T21:31:42.544 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33519.log 2024-04-20T21:31:42.544 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.58157.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.58157.log.gz 2024-04-20T21:31:42.545 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45029.log 2024-04-20T21:31:42.546 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36614.log 2024-04-20T21:31:42.546 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.33519.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33519.log.gz 2024-04-20T21:31:42.548 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.45029.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45029.log.gz 2024-04-20T21:31:42.548 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29214.log 2024-04-20T21:31:42.549 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.36614.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36614.log.gz 2024-04-20T21:31:42.549 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71432.log 2024-04-20T21:31:42.551 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.29214.log: /var/log/ceph/ceph-client.admin.71432.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.29214.log.gz 2024-04-20T21:31:42.551 INFO:teuthology.orchestra.run.smithi086.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.71432.log.gz 2024-04-20T21:31:42.591 INFO:teuthology.orchestra.run.smithi086.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-20T21:31:42.651 INFO:teuthology.orchestra.run.smithi086.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-20T21:31:42.674 INFO:teuthology.orchestra.run.smithi086.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-20T21:31:42.708 INFO:teuthology.orchestra.run.smithi086.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-20T21:31:42.936 INFO:teuthology.orchestra.run.smithi086.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-20T21:31:42.940 INFO:teuthology.orchestra.run.smithi086.stderr: 2024-04-20T21:31:42.940 INFO:teuthology.orchestra.run.smithi086.stderr:real 0m1.463s 2024-04-20T21:31:42.940 INFO:teuthology.orchestra.run.smithi086.stderr:user 0m4.830s 2024-04-20T21:31:42.940 INFO:teuthology.orchestra.run.smithi086.stderr:sys 0m2.405s 2024-04-20T21:31:42.941 INFO:tasks.ceph:Archiving logs... 2024-04-20T21:31:42.941 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665408/remote/smithi086/log 2024-04-20T21:31:42.942 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-20T21:31:44.487 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665408/remote/smithi144/log 2024-04-20T21:31:44.488 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-20T21:31:45.044 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-20T21:31:45.055 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-20T21:31:45.055 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-20T21:31:45.063 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-20T21:31:45.205 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-20T21:31:45.215 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-20T21:31:45.215 DEBUG:teuthology.orchestra.run.smithi086:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T21:31:45.223 DEBUG:teuthology.orchestra.run.smithi144:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T21:31:45.270 INFO:teuthology.orchestra.run.smithi086.stderr:bash: line 1: ntpq: command not found 2024-04-20T21:31:45.279 INFO:teuthology.orchestra.run.smithi144.stderr:bash: line 1: ntpq: command not found 2024-04-20T21:31:45.279 INFO:teuthology.orchestra.run.smithi086.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T21:31:45.279 INFO:teuthology.orchestra.run.smithi086.stdout:=============================================================================== 2024-04-20T21:31:45.280 INFO:teuthology.orchestra.run.smithi086.stdout:^+ hv01.front.sepia.ceph.com 2 7 377 126 -1024us[-1024us] +/- 93ms 2024-04-20T21:31:45.280 INFO:teuthology.orchestra.run.smithi086.stdout:^* hv02.front.sepia.ceph.com 3 7 377 129 -599us[ -600us] +/- 72ms 2024-04-20T21:31:45.280 INFO:teuthology.orchestra.run.smithi086.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 62 +1462us[+1462us] +/- 71ms 2024-04-20T21:31:45.280 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T21:31:45.289 INFO:teuthology.orchestra.run.smithi144.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T21:31:45.289 INFO:teuthology.orchestra.run.smithi144.stdout:=============================================================================== 2024-04-20T21:31:45.289 INFO:teuthology.orchestra.run.smithi144.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 60 -1051us[-1051us] +/- 94ms 2024-04-20T21:31:45.289 INFO:teuthology.orchestra.run.smithi144.stdout:^* hv02.front.sepia.ceph.com 3 6 377 61 -613us[ -613us] +/- 73ms 2024-04-20T21:31:45.289 INFO:teuthology.orchestra.run.smithi144.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 61 +1444us[+1444us] +/- 71ms 2024-04-20T21:31:45.289 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T21:31:45.291 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-20T21:31:45.301 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-20T21:31:45.301 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-20T21:31:45.310 DEBUG:teuthology.orchestra.run.smithi086:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-20T21:31:45.517 DEBUG:teuthology.orchestra.run.smithi144:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-20T21:31:45.678 DEBUG:teuthology.orchestra.run.smithi086:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-20T21:31:45.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:31:45.773 DEBUG:teuthology.orchestra.run.smithi144:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-20T21:31:45.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:31:45.859 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-20T21:31:45.870 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-20T21:31:45.880 INFO:teuthology.task.internal:Duration was 1246.855105 seconds 2024-04-20T21:31:45.881 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-20T21:31:45.890 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-20T21:31:45.890 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-20T21:31:45.898 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-20T21:31:46.020 INFO:teuthology.orchestra.run.smithi086.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T21:31:46.037 INFO:teuthology.orchestra.run.smithi144.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T21:31:46.255 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-20T21:31:46.255 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi086.front.sepia.ceph.com 2024-04-20T21:31:46.255 DEBUG:teuthology.orchestra.run.smithi086:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-20T21:31:46.328 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi144.front.sepia.ceph.com 2024-04-20T21:31:46.329 DEBUG:teuthology.orchestra.run.smithi144:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-20T21:31:46.408 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-20T21:31:46.409 DEBUG:teuthology.orchestra.run.smithi086:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-20T21:31:46.416 DEBUG:teuthology.orchestra.run.smithi144:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-20T21:31:46.499 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-20T21:31:46.499 DEBUG:teuthology.orchestra.run.smithi086:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-20T21:31:46.506 DEBUG:teuthology.orchestra.run.smithi144:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-20T21:31:46.664 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-20T21:31:46.675 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-20T21:31:46.675 DEBUG:teuthology.orchestra.run.smithi086:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-20T21:31:46.759 DEBUG:teuthology.orchestra.run.smithi144:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-20T21:31:46.846 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-20T21:31:46.857 DEBUG:teuthology.orchestra.run.smithi086:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-20T21:31:46.865 DEBUG:teuthology.orchestra.run.smithi144:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-20T21:31:46.959 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern = core 2024-04-20T21:31:46.961 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern = core 2024-04-20T21:31:47.001 DEBUG:teuthology.orchestra.run.smithi086:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-20T21:31:47.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:31:47.049 DEBUG:teuthology.orchestra.run.smithi144:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-20T21:31:47.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T21:31:47.101 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-20T21:31:47.143 INFO:teuthology.task.internal:Transferring archived files... 2024-04-20T21:31:47.144 DEBUG:teuthology.misc:Transferring archived files from smithi086:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665408/remote/smithi086 2024-04-20T21:31:47.144 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-20T21:31:47.238 DEBUG:teuthology.misc:Transferring archived files from smithi144:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665408/remote/smithi144 2024-04-20T21:31:47.238 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-20T21:31:47.329 INFO:teuthology.task.internal:Removing archive directory... 2024-04-20T21:31:47.329 DEBUG:teuthology.orchestra.run.smithi086:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-20T21:31:47.338 DEBUG:teuthology.orchestra.run.smithi144:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-20T21:31:47.392 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-20T21:31:47.404 INFO:teuthology.task.internal:Not uploading archives. 2024-04-20T21:31:47.404 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-20T21:31:47.414 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-20T21:31:47.414 DEBUG:teuthology.orchestra.run.smithi086:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-20T21:31:47.420 DEBUG:teuthology.orchestra.run.smithi144:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-20T21:31:47.466 INFO:teuthology.orchestra.run.smithi086.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 20 21:31 /home/ubuntu/cephtest 2024-04-20T21:31:47.472 INFO:teuthology.orchestra.run.smithi144.stdout: 393391 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 20 21:31 /home/ubuntu/cephtest 2024-04-20T21:31:47.476 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-20T21:31:47.486 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-20T21:31:47.507 INFO:teuthology.nuke:Checking targets against current locks 2024-04-20T21:31:47.536 DEBUG:teuthology.nuke:shortname: smithi086 2024-04-20T21:31:47.536 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-20T21:31:47.561 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi086.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665408', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-20 20:51:45.034137', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHZnpT/76zTvzEFODAI0hhznpXMK/yubClrbEpPoEgNSA8R/v4zsK8z84AoBNTlVHHCt97Nlq4Z56KKW9fxSmvY='} 2024-04-20T21:31:47.564 DEBUG:teuthology.nuke:shortname: smithi144 2024-04-20T21:31:47.564 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-20T21:31:47.586 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi144.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-20_20:50:37-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7665408', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-20 20:51:45.035608', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNFjKrxoh8SrDM4g/XgOPDuG61tFRgAkBwBCkI70X6ampv0lTPrAbP6bOnQPzoCAqUAMhMswMK2ehmSHQ/ViGVw='} 2024-04-20T21:31:47.615 INFO:teuthology.orchestra.console.smithi086:Power off 2024-04-20T21:31:47.615 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-20T21:31:47.637 DEBUG:teuthology.orchestra.console.smithi086:power off output: Chassis Power Control: Down/Off 2024-04-20T21:31:47.637 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T21:31:47.662 INFO:teuthology.orchestra.console.smithi144:Power off 2024-04-20T21:31:47.662 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-20T21:31:47.672 DEBUG:teuthology.orchestra.console.smithi086:check power output: Chassis Power is on 2024-04-20T21:31:47.684 DEBUG:teuthology.orchestra.console.smithi144:power off output: Chassis Power Control: Down/Off 2024-04-20T21:31:47.684 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T21:31:47.704 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is on 2024-04-20T21:31:51.673 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T21:31:51.705 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T21:31:51.784 DEBUG:teuthology.orchestra.console.smithi086:check power output: Chassis Power is on 2024-04-20T21:31:51.814 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is on 2024-04-20T21:31:55.785 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T21:31:55.814 DEBUG:teuthology.orchestra.console.smithi144:pexpect command: ipmitool -H smithi144.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T21:31:55.896 DEBUG:teuthology.orchestra.console.smithi086:check power output: Chassis Power is on 2024-04-20T21:31:55.924 DEBUG:teuthology.orchestra.console.smithi144:check power output: Chassis Power is off 2024-04-20T21:31:56.024 INFO:teuthology.orchestra.console.smithi144:Power off completed 2024-04-20T21:31:59.899 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-20T21:32:00.015 DEBUG:teuthology.orchestra.console.smithi086:check power output: Chassis Power is off 2024-04-20T21:32:00.116 INFO:teuthology.orchestra.console.smithi086:Power off completed 2024-04-20T21:32:00.221 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/forward-scrub} duration: 1246.8551051616669 failure_reason: '"2024-04-20T21:25:28.220524+0000 mds.b (mds.0) 1 : cluster [ERR] dir 0x10000000000 object missing on disk; some files may be lost (/dir)" in cluster log' flavor: default mon.a-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 mon.b-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 owner: scheduled_rishabh@teuthology success: false 2024-04-20T21:32:00.221 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-20T21:32:00.300 INFO:teuthology.run:FAIL